2023-08-09T13:15:00.349 INFO:root:teuthology version: 0.0.1.dev181+g7fda959 2023-08-09T13:15:00.350 DEBUG:teuthology.run:Teuthology command: teuthology --owner scheduled_vshankar@teuthology --verbose --archive /home/teuthworker/archive/vshankar-2023-08-09_05:46:29-fs-wip-vshankar-testing-20230809.035933-testing-default-smithi/7364139 --description fs/functional/{begin/{0-install 1-ceph 2-logrotate} clusters/1a3s-mds-4c-client conf/{client mds mon osd} distro/{centos_latest} mount/fuse objectstore/bluestore-ec-root overrides/{ignorelist_health ignorelist_wrongly_marked_down no_client_pidfile} tasks/snap-schedule} --name vshankar-2023-08-09_05:46:29-fs-wip-vshankar-testing-20230809.035933-testing-default-smithi -- /home/teuthworker/archive/vshankar-2023-08-09_05:46:29-fs-wip-vshankar-testing-20230809.035933-testing-default-smithi/7364139/orig.config.yaml 2023-08-09T13:15:00.395 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-08-09T13:15:00.473 INFO:teuthology.run:Config: archive_path: /home/teuthworker/archive/vshankar-2023-08-09_05:46:29-fs-wip-vshankar-testing-20230809.035933-testing-default-smithi/7364139 branch: wip-vshankar-testing-20230809.035933 description: fs/functional/{begin/{0-install 1-ceph 2-logrotate} clusters/1a3s-mds-4c-client conf/{client mds mon osd} distro/{centos_latest} mount/fuse objectstore/bluestore-ec-root overrides/{ignorelist_health ignorelist_wrongly_marked_down no_client_pidfile} tasks/snap-schedule} email: vshankar@redhat.com first_in_suite: false job_id: '7364139' kernel: flavor: default kdb: true sha1: a7fb1265323db972dd333f71b9a53e9479f62e37 last_in_suite: false log-rotate: ceph-mds: 10G ceph-osd: 10G machine_type: smithi name: vshankar-2023-08-09_05:46:29-fs-wip-vshankar-testing-20230809.035933-testing-default-smithi no_nested_subset: false nuke-on-error: true openstack: - volumes: count: 4 size: 20 - machine: disk: 200 os_type: centos os_version: 9.stream overrides: admin_socket: branch: wip-vshankar-testing-20230809.035933 ceph: cephfs: ec_profile: - m=2 - k=2 - crush-failure-domain=osd conf: client: client mount timeout: 600 debug client: 20 debug ms: 1 pid file: '' rados mon op timeout: 900 rados osd op timeout: 900 mds: debug mds: 20 debug mds balancer: 20 debug ms: 1 mds debug frag: true mds debug scatterstat: true mds op complaint time: 180 mds verify scatter: true osd op complaint time: 180 rados mon op timeout: 900 rados osd op timeout: 900 mgr: debug client: 20 debug finisher: 20 debug mgr: 20 debug ms: 1 mon: debug mon: 20 debug ms: 1 debug paxos: 20 mon op complaint time: 120 osd: bluestore block size: 96636764160 bluestore fsck on mount: true debug bluefs: 20 debug bluestore: 20 debug ms: 1 debug osd: 20 debug rocksdb: 10 mon osd backfillfull_ratio: 0.85 mon osd full ratio: 0.9 mon osd nearfull ratio: 0.8 osd failsafe full ratio: 0.95 osd objectstore: bluestore osd op complaint time: 180 flavor: default fs: xfs log-ignorelist: - \(MDS_ALL_DOWN\) - \(MDS_UP_LESS_THAN_MAX\) - overall HEALTH_ - \(FS_DEGRADED\) - \(MDS_FAILED\) - \(MDS_DEGRADED\) - \(FS_WITH_FAILED_MDS\) - \(MDS_DAMAGE\) - \(MDS_ALL_DOWN\) - \(MDS_UP_LESS_THAN_MAX\) - \(FS_INLINE_DATA_DEPRECATED\) - overall HEALTH_ - \(OSD_DOWN\) - \(OSD_ - but it is still running - is not responding log-whitelist: - OSD full dropping all updates - OSD near full - pausewr flag - failsafe engaged, dropping updates - failsafe disengaged, no longer dropping - is full \(reached quota - POOL_FULL - POOL_BACKFILLFULL sha1: 668a86f232c9454b2ce5928d1432a29a72964d85 ceph-deploy: bluestore: true conf: client: log file: /var/log/ceph/ceph-$name.$pid.log mon: osd default pool size: 2 osd: bluestore block size: 96636764160 bluestore fsck on mount: true debug bluefs: 20 debug bluestore: 20 debug rocksdb: 10 mon osd backfillfull_ratio: 0.85 mon osd full ratio: 0.9 mon osd nearfull ratio: 0.8 osd failsafe full ratio: 0.95 osd objectstore: bluestore fs: xfs install: ceph: flavor: default sha1: 668a86f232c9454b2ce5928d1432a29a72964d85 thrashosds: bdev_inject_crash: 2 bdev_inject_crash_probability: 0.5 workunit: branch: wip-vshankar-testing-20230809.035933 sha1: 668a86f232c9454b2ce5928d1432a29a72964d85 owner: scheduled_vshankar@teuthology priority: 80 repo: https://git.ceph.com/ceph-ci.git roles: - - mon.a - mgr.y - mds.a - mds.c - osd.0 - osd.1 - osd.2 - osd.3 - client.0 - client.2 - - mon.b - mon.c - mgr.x - mds.b - mds.d - osd.4 - osd.5 - osd.6 - osd.7 - client.1 - client.3 seed: 8976 sha1: 668a86f232c9454b2ce5928d1432a29a72964d85 sleep_before_teardown: 0 subset: 486/1024 suite: fs suite_branch: wip-vshankar-testing-20230809.035933 suite_path: /home/teuthworker/src/git.ceph.com_ceph-c_668a86f232c9454b2ce5928d1432a29a72964d85/qa suite_relpath: qa suite_repo: https://git.ceph.com/ceph-ci.git suite_sha1: 668a86f232c9454b2ce5928d1432a29a72964d85 syslog: ignorelist: - WARNING*.*check_session_state - WARNING*.*__ceph_remove_cap targets: smithi043.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBEKktPs1zHDi/KNgutOyVqOWoAarBLxJwqxEBET4kyZL78XfuBO2Mt+SP5ieccwraLOmO+jlznyREbzKXB1iudI= smithi203.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBD/W9aB9RM/Yz46ZrKxLIqCF48iawl+xUQvYLoz3OmwYWs/kun9L9UcFcFyB+qwqiBQJZebgK5RojyEKSWt07Cw= tasks: - install: extra_packages: deb: - python3-cephfs - cephfs-shell - cephfs-top - cephfs-mirror rpm: - python3-cephfs - cephfs-top - cephfs-mirror extra_system_packages: deb: - bison - flex - libelf-dev - libssl-dev - network-manager - iproute2 - util-linux - dump - indent - libaio-dev - libtool-bin - uuid-dev - xfslibs-dev - postgresql - postgresql-client - postgresql-common - postgresql-contrib rpm: - bison - flex - elfutils-libelf-devel - openssl-devel - NetworkManager - iproute - util-linux - libacl-devel - libaio-devel - libattr-devel - libtool - libuuid-devel - xfsdump - xfsprogs - xfsprogs-devel - libaio-devel - libtool - libuuid-devel - xfsprogs-devel - postgresql - postgresql-server - postgresql-contrib - ceph: null - ceph-fuse: null - cephfs_test_runner: modules: - tasks.cephfs.test_snap_schedules teuthology: fragments_dropped: [] meta: {} postmerge: - local function is_kupstream() return false end - local function is_kdistro() return false end - local function is_fuse() return true end - local function syntax_version() return '' end teuthology_branch: main teuthology_sha1: 7fda95956ac10132c9b74016ba832db907df09fa timestamp: 2023-08-09_05:46:29 tube: smithi user: vshankar verbose: false worker_log: /home/teuthworker/archive/worker_logs/dispatcher.smithi.2224 2023-08-09T13:15:00.473 INFO:teuthology.run:suite_path is set to /home/teuthworker/src/git.ceph.com_ceph-c_668a86f232c9454b2ce5928d1432a29a72964d85/qa; will attempt to use it 2023-08-09T13:15:00.474 INFO:teuthology.run:Found tasks at /home/teuthworker/src/git.ceph.com_ceph-c_668a86f232c9454b2ce5928d1432a29a72964d85/qa/tasks 2023-08-09T13:15:00.474 INFO:teuthology.run_tasks:Running task internal.check_packages... 2023-08-09T13:15:00.476 INFO:teuthology.task.internal:Checking packages... 2023-08-09T13:15:00.504 INFO:teuthology.task.internal:Checking packages for os_type 'centos', flavor 'default' and ceph hash '668a86f232c9454b2ce5928d1432a29a72964d85' 2023-08-09T13:15:00.504 WARNING:teuthology.packaging:More than one of ref, tag, branch, or sha1 supplied; using branch 2023-08-09T13:15:00.505 INFO:teuthology.packaging:ref: None 2023-08-09T13:15:00.505 INFO:teuthology.packaging:tag: None 2023-08-09T13:15:00.505 INFO:teuthology.packaging:branch: wip-vshankar-testing-20230809.035933 2023-08-09T13:15:00.506 INFO:teuthology.packaging:sha1: 668a86f232c9454b2ce5928d1432a29a72964d85 2023-08-09T13:15:00.506 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&ref=wip-vshankar-testing-20230809.035933 2023-08-09T13:15:00.771 INFO:teuthology.task.internal:Found packages for ceph version 18.0.0-5389.g668a86f2 2023-08-09T13:15:00.771 INFO:teuthology.run_tasks:Running task internal.buildpackages_prep... 2023-08-09T13:15:00.778 INFO:teuthology.task.internal:no buildpackages task found 2023-08-09T13:15:00.778 INFO:teuthology.run_tasks:Running task internal.save_config... 2023-08-09T13:15:00.785 INFO:teuthology.task.internal:Saving configuration 2023-08-09T13:15:00.800 INFO:teuthology.run_tasks:Running task internal.check_lock... 2023-08-09T13:15:00.807 INFO:teuthology.task.internal.check_lock:Checking locks... 2023-08-09T13:15:00.832 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi043.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/vshankar-2023-08-09_05:46:29-fs-wip-vshankar-testing-20230809.035933-testing-default-smithi/7364139', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '9', 'arch': 'x86_64', 'locked': True, 'locked_since': '2023-08-09 13:09:34.745232', 'locked_by': 'scheduled_vshankar@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBEKktPs1zHDi/KNgutOyVqOWoAarBLxJwqxEBET4kyZL78XfuBO2Mt+SP5ieccwraLOmO+jlznyREbzKXB1iudI='} 2023-08-09T13:15:00.852 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi203.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/vshankar-2023-08-09_05:46:29-fs-wip-vshankar-testing-20230809.035933-testing-default-smithi/7364139', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '9', 'arch': 'x86_64', 'locked': True, 'locked_since': '2023-08-09 13:09:34.746687', 'locked_by': 'scheduled_vshankar@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBD/W9aB9RM/Yz46ZrKxLIqCF48iawl+xUQvYLoz3OmwYWs/kun9L9UcFcFyB+qwqiBQJZebgK5RojyEKSWt07Cw='} 2023-08-09T13:15:00.853 INFO:teuthology.run_tasks:Running task internal.add_remotes... 2023-08-09T13:15:00.888 INFO:teuthology.task.internal:roles: ubuntu@smithi043.front.sepia.ceph.com - ['mon.a', 'mgr.y', 'mds.a', 'mds.c', 'osd.0', 'osd.1', 'osd.2', 'osd.3', 'client.0', 'client.2'] 2023-08-09T13:15:00.889 INFO:teuthology.task.internal:roles: ubuntu@smithi203.front.sepia.ceph.com - ['mon.b', 'mon.c', 'mgr.x', 'mds.b', 'mds.d', 'osd.4', 'osd.5', 'osd.6', 'osd.7', 'client.1', 'client.3'] 2023-08-09T13:15:00.889 INFO:teuthology.run_tasks:Running task console_log... 2023-08-09T13:15:00.961 DEBUG:teuthology.exit:Installing handler: Handler(exiter=, func=.kill_console_loggers at 0x7f904bc1c040>, signals=[15]) 2023-08-09T13:15:00.963 INFO:teuthology.run_tasks:Running task internal.connect... 2023-08-09T13:15:00.971 INFO:teuthology.task.internal:Opening connections... 2023-08-09T13:15:00.971 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi043.front.sepia.ceph.com 2023-08-09T13:15:00.973 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi043.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-08-09T13:15:01.074 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi203.front.sepia.ceph.com 2023-08-09T13:15:01.075 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi203.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-08-09T13:15:01.157 INFO:teuthology.run_tasks:Running task internal.push_inventory... 2023-08-09T13:15:01.164 DEBUG:teuthology.orchestra.run.smithi043:> uname -m 2023-08-09T13:15:01.181 INFO:teuthology.orchestra.run.smithi043.stdout:x86_64 2023-08-09T13:15:01.182 DEBUG:teuthology.orchestra.run.smithi043:> cat /etc/os-release 2023-08-09T13:15:01.238 INFO:teuthology.orchestra.run.smithi043.stdout:NAME="CentOS Stream" 2023-08-09T13:15:01.238 INFO:teuthology.orchestra.run.smithi043.stdout:VERSION="9" 2023-08-09T13:15:01.239 INFO:teuthology.orchestra.run.smithi043.stdout:ID="centos" 2023-08-09T13:15:01.239 INFO:teuthology.orchestra.run.smithi043.stdout:ID_LIKE="rhel fedora" 2023-08-09T13:15:01.239 INFO:teuthology.orchestra.run.smithi043.stdout:VERSION_ID="9" 2023-08-09T13:15:01.239 INFO:teuthology.orchestra.run.smithi043.stdout:PLATFORM_ID="platform:el9" 2023-08-09T13:15:01.239 INFO:teuthology.orchestra.run.smithi043.stdout:PRETTY_NAME="CentOS Stream 9" 2023-08-09T13:15:01.239 INFO:teuthology.orchestra.run.smithi043.stdout:ANSI_COLOR="0;31" 2023-08-09T13:15:01.240 INFO:teuthology.orchestra.run.smithi043.stdout:LOGO="fedora-logo-icon" 2023-08-09T13:15:01.240 INFO:teuthology.orchestra.run.smithi043.stdout:CPE_NAME="cpe:/o:centos:centos:9" 2023-08-09T13:15:01.240 INFO:teuthology.orchestra.run.smithi043.stdout:HOME_URL="https://centos.org/" 2023-08-09T13:15:01.240 INFO:teuthology.orchestra.run.smithi043.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/" 2023-08-09T13:15:01.240 INFO:teuthology.orchestra.run.smithi043.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 9" 2023-08-09T13:15:01.240 INFO:teuthology.orchestra.run.smithi043.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream" 2023-08-09T13:15:01.241 INFO:teuthology.lock.ops:Updating smithi043.front.sepia.ceph.com on lock server 2023-08-09T13:15:01.265 DEBUG:teuthology.orchestra.run.smithi203:> uname -m 2023-08-09T13:15:01.283 INFO:teuthology.orchestra.run.smithi203.stdout:x86_64 2023-08-09T13:15:01.284 DEBUG:teuthology.orchestra.run.smithi203:> cat /etc/os-release 2023-08-09T13:15:01.341 INFO:teuthology.orchestra.run.smithi203.stdout:NAME="CentOS Stream" 2023-08-09T13:15:01.341 INFO:teuthology.orchestra.run.smithi203.stdout:VERSION="9" 2023-08-09T13:15:01.342 INFO:teuthology.orchestra.run.smithi203.stdout:ID="centos" 2023-08-09T13:15:01.342 INFO:teuthology.orchestra.run.smithi203.stdout:ID_LIKE="rhel fedora" 2023-08-09T13:15:01.342 INFO:teuthology.orchestra.run.smithi203.stdout:VERSION_ID="9" 2023-08-09T13:15:01.342 INFO:teuthology.orchestra.run.smithi203.stdout:PLATFORM_ID="platform:el9" 2023-08-09T13:15:01.342 INFO:teuthology.orchestra.run.smithi203.stdout:PRETTY_NAME="CentOS Stream 9" 2023-08-09T13:15:01.343 INFO:teuthology.orchestra.run.smithi203.stdout:ANSI_COLOR="0;31" 2023-08-09T13:15:01.343 INFO:teuthology.orchestra.run.smithi203.stdout:LOGO="fedora-logo-icon" 2023-08-09T13:15:01.343 INFO:teuthology.orchestra.run.smithi203.stdout:CPE_NAME="cpe:/o:centos:centos:9" 2023-08-09T13:15:01.343 INFO:teuthology.orchestra.run.smithi203.stdout:HOME_URL="https://centos.org/" 2023-08-09T13:15:01.343 INFO:teuthology.orchestra.run.smithi203.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/" 2023-08-09T13:15:01.343 INFO:teuthology.orchestra.run.smithi203.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 9" 2023-08-09T13:15:01.343 INFO:teuthology.orchestra.run.smithi203.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream" 2023-08-09T13:15:01.344 INFO:teuthology.lock.ops:Updating smithi203.front.sepia.ceph.com on lock server 2023-08-09T13:15:01.365 INFO:teuthology.run_tasks:Running task internal.serialize_remote_roles... 2023-08-09T13:15:01.379 INFO:teuthology.run_tasks:Running task internal.check_conflict... 2023-08-09T13:15:01.384 INFO:teuthology.task.internal:Checking for old test directory... 2023-08-09T13:15:01.385 DEBUG:teuthology.orchestra.run.smithi043:> test '!' -e /home/ubuntu/cephtest 2023-08-09T13:15:01.387 DEBUG:teuthology.orchestra.run.smithi203:> test '!' -e /home/ubuntu/cephtest 2023-08-09T13:15:01.404 INFO:teuthology.run_tasks:Running task internal.check_ceph_data... 2023-08-09T13:15:01.410 INFO:teuthology.task.internal:Checking for non-empty /var/lib/ceph... 2023-08-09T13:15:01.411 DEBUG:teuthology.orchestra.run.smithi043:> test -z $(ls -A /var/lib/ceph) 2023-08-09T13:15:01.444 DEBUG:teuthology.orchestra.run.smithi203:> test -z $(ls -A /var/lib/ceph) 2023-08-09T13:15:01.472 INFO:teuthology.run_tasks:Running task internal.vm_setup... 2023-08-09T13:15:01.530 INFO:teuthology.run_tasks:Running task kernel... 2023-08-09T13:15:01.545 INFO:teuthology.task.kernel:normalize config orig: {'flavor': 'default', 'kdb': True, 'sha1': 'a7fb1265323db972dd333f71b9a53e9479f62e37'} 2023-08-09T13:15:01.546 INFO:teuthology.task.kernel:config {'mon.a': {'flavor': 'default', 'kdb': True, 'sha1': 'a7fb1265323db972dd333f71b9a53e9479f62e37'}, 'mon.b': {'flavor': 'default', 'kdb': True, 'sha1': 'a7fb1265323db972dd333f71b9a53e9479f62e37'}}, timeout 300 2023-08-09T13:15:01.546 DEBUG:teuthology.orchestra.run.smithi043:> test -f /run/.containerenv -o -f /.dockerenv 2023-08-09T13:15:01.547 DEBUG:teuthology.orchestra.run.smithi203:> test -f /run/.containerenv -o -f /.dockerenv 2023-08-09T13:15:01.563 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-08-09T13:15:01.564 DEBUG:teuthology.task.kernel:sha1 for mon.b is a7fb1265323db972dd333f71b9a53e9479f62e37 2023-08-09T13:15:01.564 INFO:teuthology.task.kernel:Checking kernel version of mon.b, want "a7fb1265323db972dd333f71b9a53e9479f62e37"... 2023-08-09T13:15:01.564 DEBUG:teuthology.orchestra.run.smithi203:> uname -r 2023-08-09T13:15:01.565 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-08-09T13:15:01.566 DEBUG:teuthology.task.kernel:sha1 for mon.a is a7fb1265323db972dd333f71b9a53e9479f62e37 2023-08-09T13:15:01.566 INFO:teuthology.task.kernel:Checking kernel version of mon.a, want "a7fb1265323db972dd333f71b9a53e9479f62e37"... 2023-08-09T13:15:01.566 DEBUG:teuthology.orchestra.run.smithi043:> uname -r 2023-08-09T13:15:01.620 INFO:teuthology.orchestra.run.smithi203.stdout:5.14.0-239.el9.x86_64 2023-08-09T13:15:01.621 DEBUG:teuthology.task.kernel:current kernel version is 5.14.0-239.el9.x86_64 vs a7fb1265323db972dd333f71b9a53e9479f62e37 2023-08-09T13:15:01.621 DEBUG:teuthology.task.kernel:failed to parse current kernel version 2023-08-09T13:15:01.621 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=kernel&flavor=default&distros=centos%2F9%2Fx86_64&sha1=a7fb1265323db972dd333f71b9a53e9479f62e37 2023-08-09T13:15:01.625 INFO:teuthology.orchestra.run.smithi043.stdout:5.14.0-239.el9.x86_64 2023-08-09T13:15:01.625 DEBUG:teuthology.task.kernel:current kernel version is 5.14.0-239.el9.x86_64 vs a7fb1265323db972dd333f71b9a53e9479f62e37 2023-08-09T13:15:01.626 DEBUG:teuthology.task.kernel:failed to parse current kernel version 2023-08-09T13:15:01.626 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=kernel&flavor=default&distros=centos%2F9%2Fx86_64&sha1=a7fb1265323db972dd333f71b9a53e9479f62e37 2023-08-09T13:15:01.761 DEBUG:teuthology.orchestra.run.smithi203:> sudo yum upgrade -y linux-firmware 2023-08-09T13:15:01.771 DEBUG:teuthology.orchestra.run.smithi043:> sudo yum upgrade -y linux-firmware 2023-08-09T13:15:03.727 INFO:teuthology.orchestra.run.smithi203.stdout:CentOS-9 - AppStream 12 MB/s | 6.3 MB 00:00 2023-08-09T13:15:03.845 INFO:teuthology.orchestra.run.smithi043.stdout:CentOS-9 - AppStream 10 MB/s | 6.3 MB 00:00 2023-08-09T13:15:05.043 INFO:teuthology.orchestra.run.smithi203.stdout:CentOS-9 - BaseOS 6.3 MB/s | 2.0 MB 00:00 2023-08-09T13:15:05.220 INFO:teuthology.orchestra.run.smithi043.stdout:CentOS-9 - BaseOS 6.0 MB/s | 2.0 MB 00:00 2023-08-09T13:15:06.036 INFO:teuthology.orchestra.run.smithi043.stdout:CentOS Stream 9 - BaseOS 16 MB/s | 7.8 MB 00:00 2023-08-09T13:15:09.106 INFO:teuthology.orchestra.run.smithi043.stdout:CentOS Stream 9 - AppStream 8.8 MB/s | 18 MB 00:01 2023-08-09T13:15:10.400 INFO:teuthology.orchestra.run.smithi203.stdout:CentOS Stream 9 - BaseOS 1.5 MB/s | 7.8 MB 00:05 2023-08-09T13:15:13.172 INFO:teuthology.orchestra.run.smithi043.stdout:CentOS Stream 9 - Extras packages 21 kB/s | 14 kB 00:00 2023-08-09T13:15:14.147 INFO:teuthology.orchestra.run.smithi043.stdout:Extra Packages for Enterprise Linux 21 MB/s | 18 MB 00:00 2023-08-09T13:15:17.309 INFO:teuthology.orchestra.run.smithi203.stdout:CentOS Stream 9 - AppStream 3.0 MB/s | 18 MB 00:05 2023-08-09T13:15:20.341 INFO:teuthology.orchestra.run.smithi043.stdout:Dependencies resolved. 2023-08-09T13:15:20.353 INFO:teuthology.orchestra.run.smithi043.stdout:================================================================================ 2023-08-09T13:15:20.353 INFO:teuthology.orchestra.run.smithi043.stdout: Package Arch Version Repository Size 2023-08-09T13:15:20.353 INFO:teuthology.orchestra.run.smithi043.stdout:================================================================================ 2023-08-09T13:15:20.354 INFO:teuthology.orchestra.run.smithi043.stdout:Upgrading: 2023-08-09T13:15:20.354 INFO:teuthology.orchestra.run.smithi043.stdout: linux-firmware noarch 20230726-138.el9 CentOS-BaseOS 294 M 2023-08-09T13:15:20.354 INFO:teuthology.orchestra.run.smithi043.stdout: 2023-08-09T13:15:20.354 INFO:teuthology.orchestra.run.smithi043.stdout:Transaction Summary 2023-08-09T13:15:20.354 INFO:teuthology.orchestra.run.smithi043.stdout:================================================================================ 2023-08-09T13:15:20.354 INFO:teuthology.orchestra.run.smithi043.stdout:Upgrade 1 Package 2023-08-09T13:15:20.354 INFO:teuthology.orchestra.run.smithi043.stdout: 2023-08-09T13:15:20.354 INFO:teuthology.orchestra.run.smithi043.stdout:Total download size: 294 M 2023-08-09T13:15:20.355 INFO:teuthology.orchestra.run.smithi043.stdout:Downloading Packages: 2023-08-09T13:15:21.527 INFO:teuthology.orchestra.run.smithi203.stdout:CentOS Stream 9 - Extras packages 14 kB/s | 14 kB 00:00 2023-08-09T13:15:27.071 INFO:teuthology.orchestra.run.smithi203.stdout:Extra Packages for Enterprise Linux 3.4 MB/s | 18 MB 00:05 2023-08-09T13:15:27.747 INFO:teuthology.orchestra.run.smithi043.stdout:linux-firmware-20230726-138.el9.noarch.rpm 40 MB/s | 294 MB 00:07 2023-08-09T13:15:27.748 INFO:teuthology.orchestra.run.smithi043.stdout:-------------------------------------------------------------------------------- 2023-08-09T13:15:27.749 INFO:teuthology.orchestra.run.smithi043.stdout:Total 40 MB/s | 294 MB 00:07 2023-08-09T13:15:27.749 INFO:teuthology.orchestra.run.smithi043.stdout:Running transaction check 2023-08-09T13:15:27.821 INFO:teuthology.orchestra.run.smithi043.stdout:Transaction check succeeded. 2023-08-09T13:15:27.821 INFO:teuthology.orchestra.run.smithi043.stdout:Running transaction test 2023-08-09T13:15:29.744 INFO:teuthology.orchestra.run.smithi043.stdout:Transaction test succeeded. 2023-08-09T13:15:29.745 INFO:teuthology.orchestra.run.smithi043.stdout:Running transaction 2023-08-09T13:15:33.117 INFO:teuthology.orchestra.run.smithi203.stdout:Dependencies resolved. 2023-08-09T13:15:33.124 INFO:teuthology.orchestra.run.smithi203.stdout:================================================================================ 2023-08-09T13:15:33.124 INFO:teuthology.orchestra.run.smithi203.stdout: Package Arch Version Repository Size 2023-08-09T13:15:33.124 INFO:teuthology.orchestra.run.smithi203.stdout:================================================================================ 2023-08-09T13:15:33.124 INFO:teuthology.orchestra.run.smithi203.stdout:Upgrading: 2023-08-09T13:15:33.124 INFO:teuthology.orchestra.run.smithi203.stdout: linux-firmware noarch 20230726-138.el9 CentOS-BaseOS 294 M 2023-08-09T13:15:33.125 INFO:teuthology.orchestra.run.smithi203.stdout: 2023-08-09T13:15:33.125 INFO:teuthology.orchestra.run.smithi203.stdout:Transaction Summary 2023-08-09T13:15:33.125 INFO:teuthology.orchestra.run.smithi203.stdout:================================================================================ 2023-08-09T13:15:33.125 INFO:teuthology.orchestra.run.smithi203.stdout:Upgrade 1 Package 2023-08-09T13:15:33.125 INFO:teuthology.orchestra.run.smithi203.stdout: 2023-08-09T13:15:33.125 INFO:teuthology.orchestra.run.smithi203.stdout:Total download size: 294 M 2023-08-09T13:15:33.125 INFO:teuthology.orchestra.run.smithi203.stdout:Downloading Packages: 2023-08-09T13:15:35.779 INFO:teuthology.orchestra.run.smithi043.stdout: Preparing : 1/1 2023-08-09T13:15:36.515 INFO:teuthology.orchestra.run.smithi043.stdout: Upgrading : linux-firmware-20230726-138.el9.noarch 1/2 2023-08-09T13:15:36.614 INFO:teuthology.orchestra.run.smithi043.stdout: Cleanup : linux-firmware-20221214-129.el9.noarch 2/2 2023-08-09T13:15:38.115 INFO:teuthology.orchestra.run.smithi043.stdout: Running scriptlet: linux-firmware-20221214-129.el9.noarch 2/2 2023-08-09T13:15:38.116 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : linux-firmware-20230726-138.el9.noarch 1/2 2023-08-09T13:15:38.535 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : linux-firmware-20221214-129.el9.noarch 2/2 2023-08-09T13:15:38.535 INFO:teuthology.orchestra.run.smithi043.stdout: 2023-08-09T13:15:38.535 INFO:teuthology.orchestra.run.smithi043.stdout:Upgraded: 2023-08-09T13:15:38.536 INFO:teuthology.orchestra.run.smithi043.stdout: linux-firmware-20230726-138.el9.noarch 2023-08-09T13:15:38.536 INFO:teuthology.orchestra.run.smithi043.stdout: 2023-08-09T13:15:38.536 INFO:teuthology.orchestra.run.smithi043.stdout:Complete! 2023-08-09T13:15:39.044 INFO:teuthology.task.kernel:Downloading kernel a7fb1265323db972dd333f71b9a53e9479f62e37 on mon.a... 2023-08-09T13:15:39.044 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=kernel&flavor=default&distros=centos%2F9%2Fx86_64&sha1=a7fb1265323db972dd333f71b9a53e9479f62e37 2023-08-09T13:15:39.186 INFO:teuthology.task.kernel:fetching, builder baseurl is https://2.chacra.ceph.com/r/kernel/testing/a7fb1265323db972dd333f71b9a53e9479f62e37/centos/9/flavors/default/x86_64/ 2023-08-09T13:15:39.187 DEBUG:teuthology.orchestra.run.smithi043:> rm -f /tmp/kernel.x86_64.rpm && echo kernel-6.5.0_rc4_ga7fb1265323d-1.x86_64.rpm | wget -nv -O /tmp/kernel.x86_64.rpm --base=https://2.chacra.ceph.com/r/kernel/testing/a7fb1265323db972dd333f71b9a53e9479f62e37/centos/9/flavors/default/x86_64/ --input-file=- 2023-08-09T13:15:39.190 DEBUG:teuthology.task.kernel:Waiting for download/copy to ubuntu@smithi043.front.sepia.ceph.com to complete... 2023-08-09T13:15:40.872 INFO:teuthology.orchestra.run.smithi203.stdout:linux-firmware-20230726-138.el9.noarch.rpm 38 MB/s | 294 MB 00:07 2023-08-09T13:15:40.873 INFO:teuthology.orchestra.run.smithi203.stdout:-------------------------------------------------------------------------------- 2023-08-09T13:15:40.874 INFO:teuthology.orchestra.run.smithi203.stdout:Total 38 MB/s | 294 MB 00:07 2023-08-09T13:15:40.874 INFO:teuthology.orchestra.run.smithi203.stdout:Running transaction check 2023-08-09T13:15:40.957 INFO:teuthology.orchestra.run.smithi203.stdout:Transaction check succeeded. 2023-08-09T13:15:40.957 INFO:teuthology.orchestra.run.smithi203.stdout:Running transaction test 2023-08-09T13:15:42.802 INFO:teuthology.orchestra.run.smithi203.stdout:Transaction test succeeded. 2023-08-09T13:15:42.802 INFO:teuthology.orchestra.run.smithi203.stdout:Running transaction 2023-08-09T13:15:48.716 INFO:teuthology.orchestra.run.smithi203.stdout: Preparing : 1/1 2023-08-09T13:15:49.518 INFO:teuthology.orchestra.run.smithi203.stdout: Upgrading : linux-firmware-20230726-138.el9.noarch 1/2 2023-08-09T13:15:49.605 INFO:teuthology.orchestra.run.smithi203.stdout: Cleanup : linux-firmware-20221214-129.el9.noarch 2/2 2023-08-09T13:15:50.695 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09 13:15:50 URL:https://2.chacra.ceph.com/r/kernel/testing/a7fb1265323db972dd333f71b9a53e9479f62e37/centos/9/flavors/default/x86_64/kernel-6.5.0_rc4_ga7fb1265323d-1.x86_64.rpm [319463472/319463472] -> "/tmp/kernel.x86_64.rpm" [1] 2023-08-09T13:15:50.695 INFO:teuthology.orchestra.run.smithi043.stderr:FINISHED --2023-08-09 13:15:50-- 2023-08-09T13:15:50.695 INFO:teuthology.orchestra.run.smithi043.stderr:Total wall clock time: 11s 2023-08-09T13:15:50.695 INFO:teuthology.orchestra.run.smithi043.stderr:Downloaded: 1 files, 305M in 11s (27.2 MB/s) 2023-08-09T13:15:50.696 INFO:teuthology.task.kernel:Installing kernel a7fb1265323db972dd333f71b9a53e9479f62e37 on mon.a... 2023-08-09T13:15:50.697 DEBUG:teuthology.orchestra.run.smithi043:> sudo rpm -ivh --oldpackage --replacefiles --replacepkgs /tmp/kernel.x86_64.rpm 2023-08-09T13:15:50.915 INFO:teuthology.orchestra.run.smithi203.stdout: Running scriptlet: linux-firmware-20221214-129.el9.noarch 2/2 2023-08-09T13:15:50.916 INFO:teuthology.orchestra.run.smithi203.stdout: Verifying : linux-firmware-20230726-138.el9.noarch 1/2 2023-08-09T13:15:51.280 INFO:teuthology.orchestra.run.smithi203.stdout: Verifying : linux-firmware-20221214-129.el9.noarch 2/2 2023-08-09T13:15:51.280 INFO:teuthology.orchestra.run.smithi203.stdout: 2023-08-09T13:15:51.280 INFO:teuthology.orchestra.run.smithi203.stdout:Upgraded: 2023-08-09T13:15:51.280 INFO:teuthology.orchestra.run.smithi203.stdout: linux-firmware-20230726-138.el9.noarch 2023-08-09T13:15:51.280 INFO:teuthology.orchestra.run.smithi203.stdout: 2023-08-09T13:15:51.281 INFO:teuthology.orchestra.run.smithi203.stdout:Complete! 2023-08-09T13:15:51.610 INFO:teuthology.task.kernel:Downloading kernel a7fb1265323db972dd333f71b9a53e9479f62e37 on mon.b... 2023-08-09T13:15:51.610 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=kernel&flavor=default&distros=centos%2F9%2Fx86_64&sha1=a7fb1265323db972dd333f71b9a53e9479f62e37 2023-08-09T13:15:51.753 INFO:teuthology.task.kernel:fetching, builder baseurl is https://2.chacra.ceph.com/r/kernel/testing/a7fb1265323db972dd333f71b9a53e9479f62e37/centos/9/flavors/default/x86_64/ 2023-08-09T13:15:51.753 DEBUG:teuthology.orchestra.run.smithi203:> rm -f /tmp/kernel.x86_64.rpm && echo kernel-6.5.0_rc4_ga7fb1265323d-1.x86_64.rpm | wget -nv -O /tmp/kernel.x86_64.rpm --base=https://2.chacra.ceph.com/r/kernel/testing/a7fb1265323db972dd333f71b9a53e9479f62e37/centos/9/flavors/default/x86_64/ --input-file=- 2023-08-09T13:15:51.756 DEBUG:teuthology.task.kernel:Waiting for download/copy to ubuntu@smithi203.front.sepia.ceph.com to complete... 2023-08-09T13:15:52.633 INFO:teuthology.orchestra.run.smithi043.stdout:Verifying... ######################################## 2023-08-09T13:15:52.677 INFO:teuthology.orchestra.run.smithi043.stdout:Preparing... ######################################## 2023-08-09T13:15:52.729 INFO:teuthology.orchestra.run.smithi043.stdout:Updating / installing... 2023-08-09T13:16:00.042 INFO:teuthology.orchestra.run.smithi043.stdout:kernel-6.5.0_rc4_ga7fb1265323d-1 ######################################## 2023-08-09T13:16:04.988 INFO:teuthology.orchestra.run.smithi203.stderr:2023-08-09 13:16:04 URL:https://2.chacra.ceph.com/r/kernel/testing/a7fb1265323db972dd333f71b9a53e9479f62e37/centos/9/flavors/default/x86_64/kernel-6.5.0_rc4_ga7fb1265323d-1.x86_64.rpm [319463472/319463472] -> "/tmp/kernel.x86_64.rpm" [1] 2023-08-09T13:16:04.988 INFO:teuthology.orchestra.run.smithi203.stderr:FINISHED --2023-08-09 13:16:04-- 2023-08-09T13:16:04.988 INFO:teuthology.orchestra.run.smithi203.stderr:Total wall clock time: 13s 2023-08-09T13:16:04.988 INFO:teuthology.orchestra.run.smithi203.stderr:Downloaded: 1 files, 305M in 13s (23.6 MB/s) 2023-08-09T13:16:04.989 INFO:teuthology.task.kernel:Installing kernel a7fb1265323db972dd333f71b9a53e9479f62e37 on mon.b... 2023-08-09T13:16:04.990 DEBUG:teuthology.orchestra.run.smithi203:> sudo rpm -ivh --oldpackage --replacefiles --replacepkgs /tmp/kernel.x86_64.rpm 2023-08-09T13:16:06.831 INFO:teuthology.orchestra.run.smithi203.stdout:Verifying... ######################################## 2023-08-09T13:16:06.872 INFO:teuthology.orchestra.run.smithi203.stdout:Preparing... ######################################## 2023-08-09T13:16:06.921 INFO:teuthology.orchestra.run.smithi203.stdout:Updating / installing... 2023-08-09T13:16:13.940 INFO:teuthology.orchestra.run.smithi203.stdout:kernel-6.5.0_rc4_ga7fb1265323d-1 ######################################## 2023-08-09T13:16:40.165 DEBUG:teuthology.task.kernel:install_kernel(remote=ubuntu@smithi043.front.sepia.ceph.com, path=/tmp/kernel.x86_64.rpm, version=None) 2023-08-09T13:16:40.166 DEBUG:teuthology.orchestra.run.smithi043:> rpm -qlp /tmp/kernel.x86_64.rpm 2023-08-09T13:16:40.244 INFO:teuthology.orchestra.run.smithi043.stdout:/boot/System.map-6.5.0-rc4-ga7fb1265323d 2023-08-09T13:16:40.245 INFO:teuthology.orchestra.run.smithi043.stdout:/boot/config-6.5.0-rc4-ga7fb1265323d 2023-08-09T13:16:40.245 INFO:teuthology.orchestra.run.smithi043.stdout:/boot/vmlinuz-6.5.0-rc4-ga7fb1265323d 2023-08-09T13:16:40.245 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d 2023-08-09T13:16:40.245 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel 2023-08-09T13:16:40.245 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/arch 2023-08-09T13:16:40.246 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/arch/x86 2023-08-09T13:16:40.246 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/arch/x86/crypto 2023-08-09T13:16:40.246 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/arch/x86/crypto/blowfish-x86_64.ko 2023-08-09T13:16:40.246 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/arch/x86/crypto/camellia-aesni-avx-x86_64.ko 2023-08-09T13:16:40.246 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/arch/x86/crypto/camellia-aesni-avx2.ko 2023-08-09T13:16:40.246 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/arch/x86/crypto/camellia-x86_64.ko 2023-08-09T13:16:40.246 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/arch/x86/crypto/cast5-avx-x86_64.ko 2023-08-09T13:16:40.246 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/arch/x86/crypto/cast6-avx-x86_64.ko 2023-08-09T13:16:40.247 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/arch/x86/crypto/crc32-pclmul.ko 2023-08-09T13:16:40.247 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/arch/x86/crypto/crc32c-intel.ko 2023-08-09T13:16:40.247 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/arch/x86/crypto/crct10dif-pclmul.ko 2023-08-09T13:16:40.247 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/arch/x86/crypto/des3_ede-x86_64.ko 2023-08-09T13:16:40.247 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/arch/x86/crypto/ghash-clmulni-intel.ko 2023-08-09T13:16:40.247 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/arch/x86/crypto/serpent-avx-x86_64.ko 2023-08-09T13:16:40.247 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/arch/x86/crypto/serpent-avx2.ko 2023-08-09T13:16:40.248 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/arch/x86/crypto/serpent-sse2-x86_64.ko 2023-08-09T13:16:40.248 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/arch/x86/crypto/sha1-ssse3.ko 2023-08-09T13:16:40.248 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/arch/x86/crypto/sha256-ssse3.ko 2023-08-09T13:16:40.248 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/arch/x86/crypto/sha512-ssse3.ko 2023-08-09T13:16:40.248 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/arch/x86/crypto/twofish-avx-x86_64.ko 2023-08-09T13:16:40.248 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/arch/x86/crypto/twofish-x86_64-3way.ko 2023-08-09T13:16:40.248 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/arch/x86/crypto/twofish-x86_64.ko 2023-08-09T13:16:40.249 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/arch/x86/kernel 2023-08-09T13:16:40.249 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/arch/x86/kernel/cpu 2023-08-09T13:16:40.249 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/arch/x86/kernel/cpu/mce 2023-08-09T13:16:40.249 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/arch/x86/kernel/cpu/mce/mce-inject.ko 2023-08-09T13:16:40.249 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/arch/x86/kvm 2023-08-09T13:16:40.249 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/arch/x86/kvm/kvm-amd.ko 2023-08-09T13:16:40.249 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/arch/x86/kvm/kvm-intel.ko 2023-08-09T13:16:40.249 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/arch/x86/kvm/kvm.ko 2023-08-09T13:16:40.250 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/crypto 2023-08-09T13:16:40.250 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/crypto/ansi_cprng.ko 2023-08-09T13:16:40.250 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/crypto/anubis.ko 2023-08-09T13:16:40.250 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/crypto/arc4.ko 2023-08-09T13:16:40.250 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/crypto/async_tx 2023-08-09T13:16:40.250 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/crypto/async_tx/async_memcpy.ko 2023-08-09T13:16:40.250 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/crypto/async_tx/async_pq.ko 2023-08-09T13:16:40.251 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/crypto/async_tx/async_raid6_recov.ko 2023-08-09T13:16:40.251 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/crypto/async_tx/async_tx.ko 2023-08-09T13:16:40.251 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/crypto/async_tx/async_xor.ko 2023-08-09T13:16:40.251 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/crypto/async_tx/raid6test.ko 2023-08-09T13:16:40.251 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/crypto/authenc.ko 2023-08-09T13:16:40.251 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/crypto/authencesn.ko 2023-08-09T13:16:40.251 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/crypto/blake2b_generic.ko 2023-08-09T13:16:40.251 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/crypto/blowfish_common.ko 2023-08-09T13:16:40.252 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/crypto/blowfish_generic.ko 2023-08-09T13:16:40.252 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/crypto/camellia_generic.ko 2023-08-09T13:16:40.252 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/crypto/cast5_generic.ko 2023-08-09T13:16:40.252 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/crypto/cast6_generic.ko 2023-08-09T13:16:40.252 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/crypto/cast_common.ko 2023-08-09T13:16:40.252 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/crypto/ccm.ko 2023-08-09T13:16:40.252 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/crypto/cmac.ko 2023-08-09T13:16:40.253 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/crypto/crc32_generic.ko 2023-08-09T13:16:40.253 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/crypto/crypto_engine.ko 2023-08-09T13:16:40.253 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/crypto/crypto_user.ko 2023-08-09T13:16:40.253 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/crypto/cts.ko 2023-08-09T13:16:40.253 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/crypto/des_generic.ko 2023-08-09T13:16:40.253 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/crypto/dh_generic.ko 2023-08-09T13:16:40.253 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/crypto/echainiv.ko 2023-08-09T13:16:40.253 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/crypto/essiv.ko 2023-08-09T13:16:40.254 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/crypto/fcrypt.ko 2023-08-09T13:16:40.254 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/crypto/khazad.ko 2023-08-09T13:16:40.254 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/crypto/lz4.ko 2023-08-09T13:16:40.254 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/crypto/lz4hc.ko 2023-08-09T13:16:40.254 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/crypto/md4.ko 2023-08-09T13:16:40.254 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/crypto/michael_mic.ko 2023-08-09T13:16:40.254 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/crypto/pcbc.ko 2023-08-09T13:16:40.255 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/crypto/pcrypt.ko 2023-08-09T13:16:40.255 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/crypto/rmd160.ko 2023-08-09T13:16:40.255 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/crypto/seed.ko 2023-08-09T13:16:40.255 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/crypto/serpent_generic.ko 2023-08-09T13:16:40.255 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/crypto/tcrypt.ko 2023-08-09T13:16:40.255 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/crypto/tea.ko 2023-08-09T13:16:40.255 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/crypto/twofish_common.ko 2023-08-09T13:16:40.255 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/crypto/twofish_generic.ko 2023-08-09T13:16:40.256 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/crypto/vmac.ko 2023-08-09T13:16:40.256 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/crypto/wp512.ko 2023-08-09T13:16:40.256 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/crypto/xcbc.ko 2023-08-09T13:16:40.256 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/crypto/xor.ko 2023-08-09T13:16:40.256 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/crypto/xxhash_generic.ko 2023-08-09T13:16:40.256 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers 2023-08-09T13:16:40.256 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/acpi 2023-08-09T13:16:40.257 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/acpi/acpi_ipmi.ko 2023-08-09T13:16:40.257 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/acpi/acpi_pad.ko 2023-08-09T13:16:40.257 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/acpi/custom_method.ko 2023-08-09T13:16:40.257 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/acpi/ec_sys.ko 2023-08-09T13:16:40.257 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/acpi/platform_profile.ko 2023-08-09T13:16:40.257 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/acpi/sbs.ko 2023-08-09T13:16:40.257 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/acpi/sbshc.ko 2023-08-09T13:16:40.257 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/acpi/video.ko 2023-08-09T13:16:40.258 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/ata 2023-08-09T13:16:40.258 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/ata/acard-ahci.ko 2023-08-09T13:16:40.258 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/ata/ahci_platform.ko 2023-08-09T13:16:40.258 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/ata/ata_generic.ko 2023-08-09T13:16:40.258 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/ata/libahci_platform.ko 2023-08-09T13:16:40.259 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/ata/pata_acpi.ko 2023-08-09T13:16:40.259 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/ata/pata_ali.ko 2023-08-09T13:16:40.259 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/ata/pata_amd.ko 2023-08-09T13:16:40.259 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/ata/pata_artop.ko 2023-08-09T13:16:40.259 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/ata/pata_atiixp.ko 2023-08-09T13:16:40.259 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/ata/pata_atp867x.ko 2023-08-09T13:16:40.259 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/ata/pata_cmd640.ko 2023-08-09T13:16:40.259 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/ata/pata_cmd64x.ko 2023-08-09T13:16:40.260 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/ata/pata_cypress.ko 2023-08-09T13:16:40.260 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/ata/pata_efar.ko 2023-08-09T13:16:40.260 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/ata/pata_hpt366.ko 2023-08-09T13:16:40.260 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/ata/pata_hpt37x.ko 2023-08-09T13:16:40.260 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/ata/pata_hpt3x2n.ko 2023-08-09T13:16:40.260 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/ata/pata_hpt3x3.ko 2023-08-09T13:16:40.260 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/ata/pata_it8213.ko 2023-08-09T13:16:40.261 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/ata/pata_it821x.ko 2023-08-09T13:16:40.261 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/ata/pata_jmicron.ko 2023-08-09T13:16:40.261 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/ata/pata_marvell.ko 2023-08-09T13:16:40.261 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/ata/pata_mpiix.ko 2023-08-09T13:16:40.261 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/ata/pata_netcell.ko 2023-08-09T13:16:40.261 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/ata/pata_ninja32.ko 2023-08-09T13:16:40.261 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/ata/pata_ns87410.ko 2023-08-09T13:16:40.261 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/ata/pata_ns87415.ko 2023-08-09T13:16:40.262 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/ata/pata_oldpiix.ko 2023-08-09T13:16:40.262 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/ata/pata_opti.ko 2023-08-09T13:16:40.262 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/ata/pata_optidma.ko 2023-08-09T13:16:40.262 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/ata/pata_pcmcia.ko 2023-08-09T13:16:40.262 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/ata/pata_pdc2027x.ko 2023-08-09T13:16:40.262 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/ata/pata_pdc202xx_old.ko 2023-08-09T13:16:40.263 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/ata/pata_piccolo.ko 2023-08-09T13:16:40.263 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/ata/pata_rdc.ko 2023-08-09T13:16:40.263 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/ata/pata_sch.ko 2023-08-09T13:16:40.263 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/ata/pata_serverworks.ko 2023-08-09T13:16:40.263 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/ata/pata_sil680.ko 2023-08-09T13:16:40.263 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/ata/pata_sis.ko 2023-08-09T13:16:40.263 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/ata/pata_sl82c105.ko 2023-08-09T13:16:40.264 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/ata/pata_triflex.ko 2023-08-09T13:16:40.264 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/ata/pata_via.ko 2023-08-09T13:16:40.264 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/ata/pdc_adma.ko 2023-08-09T13:16:40.264 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/ata/sata_inic162x.ko 2023-08-09T13:16:40.264 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/ata/sata_mv.ko 2023-08-09T13:16:40.264 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/ata/sata_nv.ko 2023-08-09T13:16:40.264 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/ata/sata_promise.ko 2023-08-09T13:16:40.264 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/ata/sata_qstor.ko 2023-08-09T13:16:40.265 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/ata/sata_sil.ko 2023-08-09T13:16:40.265 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/ata/sata_sil24.ko 2023-08-09T13:16:40.265 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/ata/sata_sis.ko 2023-08-09T13:16:40.265 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/ata/sata_svw.ko 2023-08-09T13:16:40.265 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/ata/sata_sx4.ko 2023-08-09T13:16:40.265 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/ata/sata_uli.ko 2023-08-09T13:16:40.265 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/ata/sata_via.ko 2023-08-09T13:16:40.266 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/ata/sata_vsc.ko 2023-08-09T13:16:40.266 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/auxdisplay 2023-08-09T13:16:40.266 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/auxdisplay/ks0108.ko 2023-08-09T13:16:40.266 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/base 2023-08-09T13:16:40.266 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/base/regmap 2023-08-09T13:16:40.266 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/base/regmap/regmap-i2c.ko 2023-08-09T13:16:40.266 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/base/regmap/regmap-mmio.ko 2023-08-09T13:16:40.266 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/bcma 2023-08-09T13:16:40.267 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/bcma/bcma.ko 2023-08-09T13:16:40.267 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/block 2023-08-09T13:16:40.267 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/block/aoe 2023-08-09T13:16:40.267 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/block/aoe/aoe.ko 2023-08-09T13:16:40.267 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/block/brd.ko 2023-08-09T13:16:40.267 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/block/drbd 2023-08-09T13:16:40.267 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/block/drbd/drbd.ko 2023-08-09T13:16:40.267 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/block/floppy.ko 2023-08-09T13:16:40.268 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/block/loop.ko 2023-08-09T13:16:40.268 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/block/mtip32xx 2023-08-09T13:16:40.268 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/block/mtip32xx/mtip32xx.ko 2023-08-09T13:16:40.268 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/block/nbd.ko 2023-08-09T13:16:40.268 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/block/null_blk 2023-08-09T13:16:40.268 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/block/null_blk/null_blk.ko 2023-08-09T13:16:40.268 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/block/pktcdvd.ko 2023-08-09T13:16:40.269 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/block/rbd.ko 2023-08-09T13:16:40.269 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/block/virtio_blk.ko 2023-08-09T13:16:40.269 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/block/xen-blkback 2023-08-09T13:16:40.269 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/block/xen-blkback/xen-blkback.ko 2023-08-09T13:16:40.269 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/block/xen-blkfront.ko 2023-08-09T13:16:40.269 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/block/zram 2023-08-09T13:16:40.269 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/block/zram/zram.ko 2023-08-09T13:16:40.269 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/char 2023-08-09T13:16:40.270 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/char/hangcheck-timer.ko 2023-08-09T13:16:40.270 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/char/hw_random 2023-08-09T13:16:40.270 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/char/hw_random/amd-rng.ko 2023-08-09T13:16:40.270 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/char/hw_random/intel-rng.ko 2023-08-09T13:16:40.270 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/char/hw_random/timeriomem-rng.ko 2023-08-09T13:16:40.270 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/char/hw_random/via-rng.ko 2023-08-09T13:16:40.270 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/char/hw_random/virtio-rng.ko 2023-08-09T13:16:40.270 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/char/ipmi 2023-08-09T13:16:40.271 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/char/ipmi/ipmi_devintf.ko 2023-08-09T13:16:40.271 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/char/ipmi/ipmi_msghandler.ko 2023-08-09T13:16:40.271 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/char/ipmi/ipmi_poweroff.ko 2023-08-09T13:16:40.271 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/char/ipmi/ipmi_si.ko 2023-08-09T13:16:40.271 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/char/ipmi/ipmi_ssif.ko 2023-08-09T13:16:40.271 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/char/ipmi/ipmi_watchdog.ko 2023-08-09T13:16:40.271 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/char/lp.ko 2023-08-09T13:16:40.271 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/char/mwave 2023-08-09T13:16:40.272 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/char/mwave/mwave.ko 2023-08-09T13:16:40.272 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/char/ppdev.ko 2023-08-09T13:16:40.272 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/char/tlclk.ko 2023-08-09T13:16:40.272 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/char/tpm 2023-08-09T13:16:40.272 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/char/tpm/tpm.ko 2023-08-09T13:16:40.272 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/char/tpm/tpm_atmel.ko 2023-08-09T13:16:40.272 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/char/tpm/tpm_infineon.ko 2023-08-09T13:16:40.272 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/char/tpm/tpm_nsc.ko 2023-08-09T13:16:40.273 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/char/tpm/tpm_tis.ko 2023-08-09T13:16:40.273 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/char/tpm/tpm_tis_core.ko 2023-08-09T13:16:40.273 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/char/uv_mmtimer.ko 2023-08-09T13:16:40.273 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/char/virtio_console.ko 2023-08-09T13:16:40.273 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/cpufreq 2023-08-09T13:16:40.273 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/cpufreq/acpi-cpufreq.ko 2023-08-09T13:16:40.273 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/cpufreq/amd_freq_sensitivity.ko 2023-08-09T13:16:40.274 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/cpufreq/p4-clockmod.ko 2023-08-09T13:16:40.274 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/cpufreq/pcc-cpufreq.ko 2023-08-09T13:16:40.274 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/cpufreq/powernow-k8.ko 2023-08-09T13:16:40.274 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/cpufreq/speedstep-lib.ko 2023-08-09T13:16:40.274 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/crypto 2023-08-09T13:16:40.274 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/crypto/ccp 2023-08-09T13:16:40.274 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/crypto/ccp/ccp-crypto.ko 2023-08-09T13:16:40.274 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/crypto/ccp/ccp.ko 2023-08-09T13:16:40.275 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/crypto/intel 2023-08-09T13:16:40.275 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/crypto/intel/qat 2023-08-09T13:16:40.275 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/crypto/intel/qat/qat_common 2023-08-09T13:16:40.275 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/crypto/intel/qat/qat_common/intel_qat.ko 2023-08-09T13:16:40.275 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/crypto/intel/qat/qat_dh895xcc 2023-08-09T13:16:40.275 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/crypto/intel/qat/qat_dh895xcc/qat_dh895xcc.ko 2023-08-09T13:16:40.275 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/crypto/padlock-aes.ko 2023-08-09T13:16:40.275 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/crypto/padlock-sha.ko 2023-08-09T13:16:40.276 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/crypto/virtio 2023-08-09T13:16:40.276 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/crypto/virtio/virtio_crypto.ko 2023-08-09T13:16:40.276 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/dca 2023-08-09T13:16:40.276 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/dca/dca.ko 2023-08-09T13:16:40.276 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/devfreq 2023-08-09T13:16:40.276 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/devfreq/governor_simpleondemand.ko 2023-08-09T13:16:40.276 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/dma 2023-08-09T13:16:40.277 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/dma/dw 2023-08-09T13:16:40.277 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/dma/dw/dw_dmac.ko 2023-08-09T13:16:40.277 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/dma/ioat 2023-08-09T13:16:40.277 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/dma/ioat/ioatdma.ko 2023-08-09T13:16:40.277 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/edac 2023-08-09T13:16:40.277 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/edac/amd64_edac.ko 2023-08-09T13:16:40.278 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/edac/e752x_edac.ko 2023-08-09T13:16:40.278 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/edac/edac_mce_amd.ko 2023-08-09T13:16:40.278 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/edac/i3000_edac.ko 2023-08-09T13:16:40.278 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/edac/i3200_edac.ko 2023-08-09T13:16:40.278 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/edac/i5100_edac.ko 2023-08-09T13:16:40.278 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/edac/i5400_edac.ko 2023-08-09T13:16:40.278 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/edac/i7300_edac.ko 2023-08-09T13:16:40.278 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/edac/i7core_edac.ko 2023-08-09T13:16:40.279 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/edac/i82975x_edac.ko 2023-08-09T13:16:40.279 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/edac/ie31200_edac.ko 2023-08-09T13:16:40.279 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/edac/sb_edac.ko 2023-08-09T13:16:40.279 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/edac/x38_edac.ko 2023-08-09T13:16:40.279 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/firmware 2023-08-09T13:16:40.279 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/firmware/edd.ko 2023-08-09T13:16:40.279 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/firmware/iscsi_ibft.ko 2023-08-09T13:16:40.279 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/gpio 2023-08-09T13:16:40.280 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/gpio/gpio-ich.ko 2023-08-09T13:16:40.280 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/gpio/gpio-viperboard.ko 2023-08-09T13:16:40.280 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/hid 2023-08-09T13:16:40.280 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/hid/hid-appleir.ko 2023-08-09T13:16:40.280 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/hid/hid-aureal.ko 2023-08-09T13:16:40.280 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/hid/hid-axff.ko 2023-08-09T13:16:40.280 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/hid/hid-dr.ko 2023-08-09T13:16:40.281 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/hid/hid-elecom.ko 2023-08-09T13:16:40.281 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/hid/hid-elo.ko 2023-08-09T13:16:40.281 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/hid/hid-emsff.ko 2023-08-09T13:16:40.281 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/hid/hid-gaff.ko 2023-08-09T13:16:40.281 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/hid/hid-gt683r.ko 2023-08-09T13:16:40.281 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/hid/hid-gyration.ko 2023-08-09T13:16:40.281 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/hid/hid-holtek-kbd.ko 2023-08-09T13:16:40.281 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/hid/hid-holtek-mouse.ko 2023-08-09T13:16:40.282 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/hid/hid-holtekff.ko 2023-08-09T13:16:40.282 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/hid/hid-hyperv.ko 2023-08-09T13:16:40.282 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/hid/hid-icade.ko 2023-08-09T13:16:40.282 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/hid/hid-keytouch.ko 2023-08-09T13:16:40.282 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/hid/hid-kye.ko 2023-08-09T13:16:40.282 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/hid/hid-lcpower.ko 2023-08-09T13:16:40.282 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/hid/hid-led.ko 2023-08-09T13:16:40.282 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/hid/hid-lenovo.ko 2023-08-09T13:16:40.283 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/hid/hid-logitech-dj.ko 2023-08-09T13:16:40.283 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/hid/hid-logitech-hidpp.ko 2023-08-09T13:16:40.283 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/hid/hid-multitouch.ko 2023-08-09T13:16:40.283 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/hid/hid-ortek.ko 2023-08-09T13:16:40.283 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/hid/hid-penmount.ko 2023-08-09T13:16:40.283 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/hid/hid-petalynx.ko 2023-08-09T13:16:40.283 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/hid/hid-picolcd.ko 2023-08-09T13:16:40.283 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/hid/hid-pl.ko 2023-08-09T13:16:40.284 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/hid/hid-plantronics.ko 2023-08-09T13:16:40.284 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/hid/hid-primax.ko 2023-08-09T13:16:40.284 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/hid/hid-rmi.ko 2023-08-09T13:16:40.284 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/hid/hid-roccat-arvo.ko 2023-08-09T13:16:40.284 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/hid/hid-roccat-common.ko 2023-08-09T13:16:40.284 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/hid/hid-roccat-isku.ko 2023-08-09T13:16:40.284 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/hid/hid-roccat-kone.ko 2023-08-09T13:16:40.285 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/hid/hid-roccat-koneplus.ko 2023-08-09T13:16:40.285 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/hid/hid-roccat-konepure.ko 2023-08-09T13:16:40.285 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/hid/hid-roccat-kovaplus.ko 2023-08-09T13:16:40.285 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/hid/hid-roccat-lua.ko 2023-08-09T13:16:40.285 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/hid/hid-roccat-pyra.ko 2023-08-09T13:16:40.285 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/hid/hid-roccat-ryos.ko 2023-08-09T13:16:40.285 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/hid/hid-roccat-savu.ko 2023-08-09T13:16:40.285 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/hid/hid-roccat.ko 2023-08-09T13:16:40.286 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/hid/hid-saitek.ko 2023-08-09T13:16:40.286 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/hid/hid-samsung.ko 2023-08-09T13:16:40.286 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/hid/hid-sensor-hub.ko 2023-08-09T13:16:40.286 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/hid/hid-sjoy.ko 2023-08-09T13:16:40.286 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/hid/hid-sony.ko 2023-08-09T13:16:40.286 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/hid/hid-speedlink.ko 2023-08-09T13:16:40.286 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/hid/hid-steelseries.ko 2023-08-09T13:16:40.287 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/hid/hid-sunplus.ko 2023-08-09T13:16:40.287 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/hid/hid-thrustmaster.ko 2023-08-09T13:16:40.287 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/hid/hid-tivo.ko 2023-08-09T13:16:40.287 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/hid/hid-tmff.ko 2023-08-09T13:16:40.287 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/hid/hid-topseed.ko 2023-08-09T13:16:40.287 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/hid/hid-twinhan.ko 2023-08-09T13:16:40.287 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/hid/hid-uclogic.ko 2023-08-09T13:16:40.287 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/hid/hid-waltop.ko 2023-08-09T13:16:40.288 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/hid/hid-wiimote.ko 2023-08-09T13:16:40.288 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/hid/hid-xinmo.ko 2023-08-09T13:16:40.288 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/hid/hid-zpff.ko 2023-08-09T13:16:40.288 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/hid/hid-zydacron.ko 2023-08-09T13:16:40.288 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/hid/uhid.ko 2023-08-09T13:16:40.288 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/hid/wacom.ko 2023-08-09T13:16:40.288 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/hv 2023-08-09T13:16:40.288 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/hv/hv_balloon.ko 2023-08-09T13:16:40.289 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/hv/hv_utils.ko 2023-08-09T13:16:40.289 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/hv/hv_vmbus.ko 2023-08-09T13:16:40.289 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/hwmon 2023-08-09T13:16:40.289 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/hwmon/abituguru.ko 2023-08-09T13:16:40.289 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/hwmon/abituguru3.ko 2023-08-09T13:16:40.289 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/hwmon/acpi_power_meter.ko 2023-08-09T13:16:40.289 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/hwmon/ad7414.ko 2023-08-09T13:16:40.289 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/hwmon/ad7418.ko 2023-08-09T13:16:40.290 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/hwmon/adc128d818.ko 2023-08-09T13:16:40.290 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/hwmon/adm1025.ko 2023-08-09T13:16:40.290 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/hwmon/adm1026.ko 2023-08-09T13:16:40.290 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/hwmon/adm1029.ko 2023-08-09T13:16:40.290 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/hwmon/adm1031.ko 2023-08-09T13:16:40.290 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/hwmon/adm9240.ko 2023-08-09T13:16:40.290 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/hwmon/ads7828.ko 2023-08-09T13:16:40.290 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/hwmon/adt7410.ko 2023-08-09T13:16:40.291 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/hwmon/adt7411.ko 2023-08-09T13:16:40.291 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/hwmon/adt7462.ko 2023-08-09T13:16:40.291 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/hwmon/adt7470.ko 2023-08-09T13:16:40.291 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/hwmon/adt7475.ko 2023-08-09T13:16:40.291 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/hwmon/adt7x10.ko 2023-08-09T13:16:40.291 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/hwmon/amc6821.ko 2023-08-09T13:16:40.291 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/hwmon/applesmc.ko 2023-08-09T13:16:40.291 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/hwmon/asb100.ko 2023-08-09T13:16:40.292 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/hwmon/asc7621.ko 2023-08-09T13:16:40.292 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/hwmon/asus_atk0110.ko 2023-08-09T13:16:40.292 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/hwmon/atxp1.ko 2023-08-09T13:16:40.292 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/hwmon/coretemp.ko 2023-08-09T13:16:40.292 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/hwmon/dell-smm-hwmon.ko 2023-08-09T13:16:40.292 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/hwmon/dme1737.ko 2023-08-09T13:16:40.292 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/hwmon/ds1621.ko 2023-08-09T13:16:40.293 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/hwmon/ds620.ko 2023-08-09T13:16:40.293 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/hwmon/emc1403.ko 2023-08-09T13:16:40.293 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/hwmon/emc6w201.ko 2023-08-09T13:16:40.293 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/hwmon/f71805f.ko 2023-08-09T13:16:40.293 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/hwmon/f71882fg.ko 2023-08-09T13:16:40.293 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/hwmon/f75375s.ko 2023-08-09T13:16:40.293 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/hwmon/fam15h_power.ko 2023-08-09T13:16:40.293 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/hwmon/fschmd.ko 2023-08-09T13:16:40.294 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/hwmon/g760a.ko 2023-08-09T13:16:40.294 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/hwmon/g762.ko 2023-08-09T13:16:40.294 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/hwmon/gl518sm.ko 2023-08-09T13:16:40.294 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/hwmon/gl520sm.ko 2023-08-09T13:16:40.294 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/hwmon/hwmon-vid.ko 2023-08-09T13:16:40.294 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/hwmon/i5500_temp.ko 2023-08-09T13:16:40.294 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/hwmon/i5k_amb.ko 2023-08-09T13:16:40.295 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/hwmon/ibmaem.ko 2023-08-09T13:16:40.295 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/hwmon/ibmpex.ko 2023-08-09T13:16:40.295 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/hwmon/ina209.ko 2023-08-09T13:16:40.295 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/hwmon/ina2xx.ko 2023-08-09T13:16:40.295 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/hwmon/it87.ko 2023-08-09T13:16:40.295 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/hwmon/k10temp.ko 2023-08-09T13:16:40.295 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/hwmon/k8temp.ko 2023-08-09T13:16:40.295 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/hwmon/lineage-pem.ko 2023-08-09T13:16:40.296 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/hwmon/lm63.ko 2023-08-09T13:16:40.296 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/hwmon/lm73.ko 2023-08-09T13:16:40.296 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/hwmon/lm75.ko 2023-08-09T13:16:40.296 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/hwmon/lm77.ko 2023-08-09T13:16:40.296 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/hwmon/lm78.ko 2023-08-09T13:16:40.296 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/hwmon/lm80.ko 2023-08-09T13:16:40.296 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/hwmon/lm83.ko 2023-08-09T13:16:40.296 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/hwmon/lm85.ko 2023-08-09T13:16:40.297 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/hwmon/lm87.ko 2023-08-09T13:16:40.297 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/hwmon/lm90.ko 2023-08-09T13:16:40.297 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/hwmon/lm92.ko 2023-08-09T13:16:40.297 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/hwmon/lm93.ko 2023-08-09T13:16:40.297 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/hwmon/lm95234.ko 2023-08-09T13:16:40.297 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/hwmon/lm95241.ko 2023-08-09T13:16:40.297 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/hwmon/lm95245.ko 2023-08-09T13:16:40.297 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/hwmon/ltc2945.ko 2023-08-09T13:16:40.298 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/hwmon/ltc4151.ko 2023-08-09T13:16:40.298 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/hwmon/ltc4215.ko 2023-08-09T13:16:40.298 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/hwmon/ltc4222.ko 2023-08-09T13:16:40.298 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/hwmon/ltc4245.ko 2023-08-09T13:16:40.298 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/hwmon/ltc4260.ko 2023-08-09T13:16:40.298 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/hwmon/ltc4261.ko 2023-08-09T13:16:40.298 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/hwmon/max16065.ko 2023-08-09T13:16:40.299 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/hwmon/max1619.ko 2023-08-09T13:16:40.299 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/hwmon/max1668.ko 2023-08-09T13:16:40.299 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/hwmon/max197.ko 2023-08-09T13:16:40.299 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/hwmon/max6639.ko 2023-08-09T13:16:40.299 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/hwmon/max6650.ko 2023-08-09T13:16:40.299 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/hwmon/max6697.ko 2023-08-09T13:16:40.299 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/hwmon/mcp3021.ko 2023-08-09T13:16:40.299 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/hwmon/nct6683.ko 2023-08-09T13:16:40.300 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/hwmon/nct6775-core.ko 2023-08-09T13:16:40.300 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/hwmon/nct6775.ko 2023-08-09T13:16:40.300 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/hwmon/nct7802.ko 2023-08-09T13:16:40.300 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/hwmon/pc87360.ko 2023-08-09T13:16:40.300 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/hwmon/pc87427.ko 2023-08-09T13:16:40.300 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/hwmon/pcf8591.ko 2023-08-09T13:16:40.300 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/hwmon/pmbus 2023-08-09T13:16:40.300 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/hwmon/pmbus/adm1275.ko 2023-08-09T13:16:40.301 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/hwmon/pmbus/lm25066.ko 2023-08-09T13:16:40.301 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/hwmon/pmbus/ltc2978.ko 2023-08-09T13:16:40.301 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/hwmon/pmbus/max16064.ko 2023-08-09T13:16:40.301 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/hwmon/pmbus/max34440.ko 2023-08-09T13:16:40.301 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/hwmon/pmbus/max8688.ko 2023-08-09T13:16:40.301 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/hwmon/pmbus/pmbus.ko 2023-08-09T13:16:40.301 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/hwmon/pmbus/pmbus_core.ko 2023-08-09T13:16:40.302 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/hwmon/pmbus/tps40422.ko 2023-08-09T13:16:40.302 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/hwmon/pmbus/ucd9000.ko 2023-08-09T13:16:40.302 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/hwmon/pmbus/ucd9200.ko 2023-08-09T13:16:40.302 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/hwmon/pmbus/zl6100.ko 2023-08-09T13:16:40.302 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/hwmon/powr1220.ko 2023-08-09T13:16:40.302 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/hwmon/sch5627.ko 2023-08-09T13:16:40.302 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/hwmon/sch5636.ko 2023-08-09T13:16:40.302 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/hwmon/sch56xx-common.ko 2023-08-09T13:16:40.303 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/hwmon/sht15.ko 2023-08-09T13:16:40.303 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/hwmon/sht21.ko 2023-08-09T13:16:40.303 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/hwmon/shtc1.ko 2023-08-09T13:16:40.303 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/hwmon/sis5595.ko 2023-08-09T13:16:40.303 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/hwmon/smsc47b397.ko 2023-08-09T13:16:40.303 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/hwmon/smsc47m1.ko 2023-08-09T13:16:40.303 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/hwmon/smsc47m192.ko 2023-08-09T13:16:40.303 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/hwmon/thmc50.ko 2023-08-09T13:16:40.304 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/hwmon/tmp102.ko 2023-08-09T13:16:40.304 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/hwmon/tmp103.ko 2023-08-09T13:16:40.304 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/hwmon/tmp401.ko 2023-08-09T13:16:40.304 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/hwmon/tmp421.ko 2023-08-09T13:16:40.304 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/hwmon/via-cputemp.ko 2023-08-09T13:16:40.304 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/hwmon/via686a.ko 2023-08-09T13:16:40.304 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/hwmon/vt1211.ko 2023-08-09T13:16:40.304 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/hwmon/vt8231.ko 2023-08-09T13:16:40.305 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/hwmon/w83627ehf.ko 2023-08-09T13:16:40.305 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/hwmon/w83627hf.ko 2023-08-09T13:16:40.305 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/hwmon/w83781d.ko 2023-08-09T13:16:40.305 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/hwmon/w83791d.ko 2023-08-09T13:16:40.305 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/hwmon/w83792d.ko 2023-08-09T13:16:40.305 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/hwmon/w83793.ko 2023-08-09T13:16:40.305 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/hwmon/w83795.ko 2023-08-09T13:16:40.305 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/hwmon/w83l785ts.ko 2023-08-09T13:16:40.306 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/hwmon/w83l786ng.ko 2023-08-09T13:16:40.306 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/i2c 2023-08-09T13:16:40.306 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/i2c/algos 2023-08-09T13:16:40.306 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/i2c/algos/i2c-algo-bit.ko 2023-08-09T13:16:40.306 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/i2c/algos/i2c-algo-pca.ko 2023-08-09T13:16:40.306 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/i2c/busses 2023-08-09T13:16:40.306 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/i2c/busses/i2c-amd756-s4882.ko 2023-08-09T13:16:40.307 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/i2c/busses/i2c-amd756.ko 2023-08-09T13:16:40.307 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/i2c/busses/i2c-amd8111.ko 2023-08-09T13:16:40.307 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/i2c/busses/i2c-ccgx-ucsi.ko 2023-08-09T13:16:40.307 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/i2c/busses/i2c-designware-core.ko 2023-08-09T13:16:40.307 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/i2c/busses/i2c-designware-pci.ko 2023-08-09T13:16:40.307 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/i2c/busses/i2c-designware-platform.ko 2023-08-09T13:16:40.307 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/i2c/busses/i2c-diolan-u2c.ko 2023-08-09T13:16:40.307 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/i2c/busses/i2c-i801.ko 2023-08-09T13:16:40.308 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/i2c/busses/i2c-isch.ko 2023-08-09T13:16:40.308 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/i2c/busses/i2c-ismt.ko 2023-08-09T13:16:40.308 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/i2c/busses/i2c-nforce2-s4985.ko 2023-08-09T13:16:40.308 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/i2c/busses/i2c-nforce2.ko 2023-08-09T13:16:40.308 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/i2c/busses/i2c-parport.ko 2023-08-09T13:16:40.308 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/i2c/busses/i2c-pca-platform.ko 2023-08-09T13:16:40.308 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/i2c/busses/i2c-piix4.ko 2023-08-09T13:16:40.309 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/i2c/busses/i2c-scmi.ko 2023-08-09T13:16:40.309 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/i2c/busses/i2c-simtec.ko 2023-08-09T13:16:40.309 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/i2c/busses/i2c-sis96x.ko 2023-08-09T13:16:40.309 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/i2c/busses/i2c-tiny-usb.ko 2023-08-09T13:16:40.309 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/i2c/busses/i2c-via.ko 2023-08-09T13:16:40.309 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/i2c/busses/i2c-viapro.ko 2023-08-09T13:16:40.309 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/i2c/busses/i2c-viperboard.ko 2023-08-09T13:16:40.310 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/i2c/i2c-dev.ko 2023-08-09T13:16:40.310 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/i2c/i2c-mux.ko 2023-08-09T13:16:40.310 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/i2c/i2c-slave-eeprom.ko 2023-08-09T13:16:40.310 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/i2c/i2c-smbus.ko 2023-08-09T13:16:40.310 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/i2c/i2c-stub.ko 2023-08-09T13:16:40.310 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/infiniband 2023-08-09T13:16:40.310 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/infiniband/core 2023-08-09T13:16:40.310 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/infiniband/core/ib_cm.ko 2023-08-09T13:16:40.311 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/infiniband/core/ib_core.ko 2023-08-09T13:16:40.311 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/infiniband/core/ib_umad.ko 2023-08-09T13:16:40.311 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/infiniband/core/ib_uverbs.ko 2023-08-09T13:16:40.311 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/infiniband/core/iw_cm.ko 2023-08-09T13:16:40.311 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/infiniband/core/rdma_cm.ko 2023-08-09T13:16:40.311 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/infiniband/core/rdma_ucm.ko 2023-08-09T13:16:40.311 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/infiniband/hw 2023-08-09T13:16:40.311 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/infiniband/hw/cxgb4 2023-08-09T13:16:40.312 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/infiniband/hw/cxgb4/iw_cxgb4.ko 2023-08-09T13:16:40.312 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/infiniband/hw/mlx4 2023-08-09T13:16:40.312 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/infiniband/hw/mlx4/mlx4_ib.ko 2023-08-09T13:16:40.312 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/infiniband/hw/mlx5 2023-08-09T13:16:40.312 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/infiniband/hw/mlx5/mlx5_ib.ko 2023-08-09T13:16:40.312 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/infiniband/hw/mthca 2023-08-09T13:16:40.312 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/infiniband/hw/mthca/ib_mthca.ko 2023-08-09T13:16:40.312 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/infiniband/ulp 2023-08-09T13:16:40.313 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/infiniband/ulp/ipoib 2023-08-09T13:16:40.313 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/infiniband/ulp/ipoib/ib_ipoib.ko 2023-08-09T13:16:40.313 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/infiniband/ulp/iser 2023-08-09T13:16:40.313 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/infiniband/ulp/iser/ib_iser.ko 2023-08-09T13:16:40.313 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/infiniband/ulp/isert 2023-08-09T13:16:40.313 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/infiniband/ulp/isert/ib_isert.ko 2023-08-09T13:16:40.313 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/infiniband/ulp/srp 2023-08-09T13:16:40.314 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/infiniband/ulp/srp/ib_srp.ko 2023-08-09T13:16:40.314 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/infiniband/ulp/srpt 2023-08-09T13:16:40.314 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/infiniband/ulp/srpt/ib_srpt.ko 2023-08-09T13:16:40.314 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/input 2023-08-09T13:16:40.314 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/input/gameport 2023-08-09T13:16:40.314 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/input/gameport/emu10k1-gp.ko 2023-08-09T13:16:40.314 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/input/gameport/fm801-gp.ko 2023-08-09T13:16:40.315 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/input/gameport/gameport.ko 2023-08-09T13:16:40.315 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/input/gameport/lightning.ko 2023-08-09T13:16:40.315 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/input/gameport/ns558.ko 2023-08-09T13:16:40.315 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/input/joydev.ko 2023-08-09T13:16:40.315 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/input/keyboard 2023-08-09T13:16:40.315 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/input/keyboard/gpio_keys.ko 2023-08-09T13:16:40.315 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/input/mouse 2023-08-09T13:16:40.315 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/input/mouse/appletouch.ko 2023-08-09T13:16:40.316 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/input/mouse/bcm5974.ko 2023-08-09T13:16:40.316 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/input/mouse/cyapatp.ko 2023-08-09T13:16:40.316 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/input/mouse/elan_i2c.ko 2023-08-09T13:16:40.316 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/input/mouse/sermouse.ko 2023-08-09T13:16:40.316 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/input/mouse/synaptics_i2c.ko 2023-08-09T13:16:40.316 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/input/mouse/synaptics_usb.ko 2023-08-09T13:16:40.316 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/input/mouse/vsxxxaa.ko 2023-08-09T13:16:40.316 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/input/rmi4 2023-08-09T13:16:40.317 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/input/rmi4/rmi_core.ko 2023-08-09T13:16:40.317 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/input/serio 2023-08-09T13:16:40.317 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/input/serio/altera_ps2.ko 2023-08-09T13:16:40.317 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/input/serio/arc_ps2.ko 2023-08-09T13:16:40.317 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/input/serio/hyperv-keyboard.ko 2023-08-09T13:16:40.317 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/input/serio/serio_raw.ko 2023-08-09T13:16:40.317 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/input/sparse-keymap.ko 2023-08-09T13:16:40.318 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/iommu 2023-08-09T13:16:40.318 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/iommu/amd 2023-08-09T13:16:40.318 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/iommu/amd/iommu_v2.ko 2023-08-09T13:16:40.318 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/leds 2023-08-09T13:16:40.318 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/leds/leds-blinkm.ko 2023-08-09T13:16:40.318 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/leds/leds-lm3530.ko 2023-08-09T13:16:40.318 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/leds/leds-lp3944.ko 2023-08-09T13:16:40.318 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/leds/leds-ss4200.ko 2023-08-09T13:16:40.319 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/leds/trigger 2023-08-09T13:16:40.319 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/leds/trigger/ledtrig-audio.ko 2023-08-09T13:16:40.319 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/leds/trigger/ledtrig-backlight.ko 2023-08-09T13:16:40.319 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/leds/trigger/ledtrig-camera.ko 2023-08-09T13:16:40.319 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/leds/trigger/ledtrig-default-on.ko 2023-08-09T13:16:40.319 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/leds/trigger/ledtrig-heartbeat.ko 2023-08-09T13:16:40.319 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/leds/trigger/ledtrig-oneshot.ko 2023-08-09T13:16:40.319 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/leds/trigger/ledtrig-timer.ko 2023-08-09T13:16:40.320 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/leds/trigger/ledtrig-transient.ko 2023-08-09T13:16:40.320 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/md 2023-08-09T13:16:40.320 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/md/bcache 2023-08-09T13:16:40.320 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/md/bcache/bcache.ko 2023-08-09T13:16:40.320 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/md/dm-bio-prison.ko 2023-08-09T13:16:40.320 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/md/dm-cache-smq.ko 2023-08-09T13:16:40.320 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/md/dm-cache.ko 2023-08-09T13:16:40.320 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/md/dm-crypt.ko 2023-08-09T13:16:40.321 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/md/dm-delay.ko 2023-08-09T13:16:40.321 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/md/dm-flakey.ko 2023-08-09T13:16:40.321 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/md/dm-log-userspace.ko 2023-08-09T13:16:40.321 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/md/dm-multipath.ko 2023-08-09T13:16:40.321 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/md/dm-queue-length.ko 2023-08-09T13:16:40.321 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/md/dm-raid.ko 2023-08-09T13:16:40.321 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/md/dm-round-robin.ko 2023-08-09T13:16:40.322 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/md/dm-service-time.ko 2023-08-09T13:16:40.322 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/md/dm-switch.ko 2023-08-09T13:16:40.322 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/md/dm-thin-pool.ko 2023-08-09T13:16:40.322 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/md/dm-verity.ko 2023-08-09T13:16:40.322 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/md/faulty.ko 2023-08-09T13:16:40.322 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/md/linear.ko 2023-08-09T13:16:40.322 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/md/multipath.ko 2023-08-09T13:16:40.322 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/md/persistent-data 2023-08-09T13:16:40.323 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/md/persistent-data/dm-persistent-data.ko 2023-08-09T13:16:40.323 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/md/raid0.ko 2023-08-09T13:16:40.323 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/md/raid1.ko 2023-08-09T13:16:40.323 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/md/raid10.ko 2023-08-09T13:16:40.323 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/md/raid456.ko 2023-08-09T13:16:40.323 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/message 2023-08-09T13:16:40.323 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/message/fusion 2023-08-09T13:16:40.324 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/message/fusion/mptbase.ko 2023-08-09T13:16:40.324 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/message/fusion/mptctl.ko 2023-08-09T13:16:40.324 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/message/fusion/mptfc.ko 2023-08-09T13:16:40.324 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/message/fusion/mptlan.ko 2023-08-09T13:16:40.324 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/message/fusion/mptsas.ko 2023-08-09T13:16:40.324 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/message/fusion/mptscsih.ko 2023-08-09T13:16:40.324 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/message/fusion/mptspi.ko 2023-08-09T13:16:40.324 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/mfd 2023-08-09T13:16:40.325 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/mfd/lpc_ich.ko 2023-08-09T13:16:40.325 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/mfd/lpc_sch.ko 2023-08-09T13:16:40.325 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/mfd/mfd-core.ko 2023-08-09T13:16:40.325 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/mfd/sm501.ko 2023-08-09T13:16:40.325 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/mfd/viperboard.ko 2023-08-09T13:16:40.325 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/mfd/vx855.ko 2023-08-09T13:16:40.325 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/mfd/wl1273-core.ko 2023-08-09T13:16:40.325 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/misc 2023-08-09T13:16:40.326 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/misc/altera-stapl 2023-08-09T13:16:40.326 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/misc/altera-stapl/altera-stapl.ko 2023-08-09T13:16:40.326 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/misc/apds9802als.ko 2023-08-09T13:16:40.326 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/misc/apds990x.ko 2023-08-09T13:16:40.326 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/misc/bh1770glc.ko 2023-08-09T13:16:40.326 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/misc/cb710 2023-08-09T13:16:40.326 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/misc/cb710/cb710.ko 2023-08-09T13:16:40.327 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/misc/eeprom 2023-08-09T13:16:40.327 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/misc/eeprom/at24.ko 2023-08-09T13:16:40.327 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/misc/eeprom/eeprom.ko 2023-08-09T13:16:40.327 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/misc/eeprom/eeprom_93cx6.ko 2023-08-09T13:16:40.327 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/misc/eeprom/max6875.ko 2023-08-09T13:16:40.327 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/misc/enclosure.ko 2023-08-09T13:16:40.327 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/misc/hpilo.ko 2023-08-09T13:16:40.327 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/misc/isl29003.ko 2023-08-09T13:16:40.328 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/misc/isl29020.ko 2023-08-09T13:16:40.328 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/misc/lis3lv02d 2023-08-09T13:16:40.328 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/misc/lis3lv02d/lis3lv02d.ko 2023-08-09T13:16:40.328 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/misc/lis3lv02d/lis3lv02d_i2c.ko 2023-08-09T13:16:40.328 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/misc/mei 2023-08-09T13:16:40.328 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/misc/mei/mei-me.ko 2023-08-09T13:16:40.328 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/misc/mei/mei-txe.ko 2023-08-09T13:16:40.328 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/misc/mei/mei.ko 2023-08-09T13:16:40.329 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/misc/sgi-gru 2023-08-09T13:16:40.329 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/misc/sgi-gru/gru.ko 2023-08-09T13:16:40.329 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/misc/sgi-xp 2023-08-09T13:16:40.329 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/misc/sgi-xp/xp.ko 2023-08-09T13:16:40.329 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/misc/sgi-xp/xpc.ko 2023-08-09T13:16:40.329 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/misc/sgi-xp/xpnet.ko 2023-08-09T13:16:40.329 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/misc/tifm_7xx1.ko 2023-08-09T13:16:40.329 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/misc/tifm_core.ko 2023-08-09T13:16:40.330 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/misc/tsl2550.ko 2023-08-09T13:16:40.330 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/misc/vmw_balloon.ko 2023-08-09T13:16:40.330 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/misc/vmw_vmci 2023-08-09T13:16:40.330 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/misc/vmw_vmci/vmw_vmci.ko 2023-08-09T13:16:40.330 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/net 2023-08-09T13:16:40.330 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/net/bonding 2023-08-09T13:16:40.330 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/net/bonding/bonding.ko 2023-08-09T13:16:40.330 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/net/dsa 2023-08-09T13:16:40.331 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/net/dsa/b53 2023-08-09T13:16:40.331 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/net/dsa/b53/b53_common.ko 2023-08-09T13:16:40.331 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/net/dsa/bcm-sf2.ko 2023-08-09T13:16:40.331 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/net/dsa/mv88e6060.ko 2023-08-09T13:16:40.331 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/net/dsa/mv88e6xxx 2023-08-09T13:16:40.331 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/net/dsa/mv88e6xxx/mv88e6xxx.ko 2023-08-09T13:16:40.331 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/net/dummy.ko 2023-08-09T13:16:40.332 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/net/eql.ko 2023-08-09T13:16:40.332 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/net/ethernet 2023-08-09T13:16:40.332 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/net/ethernet/3com 2023-08-09T13:16:40.332 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/net/ethernet/3com/3c574_cs.ko 2023-08-09T13:16:40.332 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/net/ethernet/3com/3c589_cs.ko 2023-08-09T13:16:40.332 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/net/ethernet/3com/3c59x.ko 2023-08-09T13:16:40.332 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/net/ethernet/3com/typhoon.ko 2023-08-09T13:16:40.332 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/net/ethernet/8390 2023-08-09T13:16:40.333 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/net/ethernet/8390/8390.ko 2023-08-09T13:16:40.333 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/net/ethernet/8390/axnet_cs.ko 2023-08-09T13:16:40.333 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/net/ethernet/8390/ne2k-pci.ko 2023-08-09T13:16:40.333 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/net/ethernet/8390/pcnet_cs.ko 2023-08-09T13:16:40.333 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/net/ethernet/adaptec 2023-08-09T13:16:40.333 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/net/ethernet/adaptec/starfire.ko 2023-08-09T13:16:40.333 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/net/ethernet/agere 2023-08-09T13:16:40.333 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/net/ethernet/agere/et131x.ko 2023-08-09T13:16:40.334 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/net/ethernet/alteon 2023-08-09T13:16:40.334 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/net/ethernet/alteon/acenic.ko 2023-08-09T13:16:40.334 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/net/ethernet/altera 2023-08-09T13:16:40.334 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/net/ethernet/altera/altera_tse.ko 2023-08-09T13:16:40.334 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/net/ethernet/amd 2023-08-09T13:16:40.334 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/net/ethernet/amd/amd8111e.ko 2023-08-09T13:16:40.334 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/net/ethernet/amd/nmclan_cs.ko 2023-08-09T13:16:40.334 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/net/ethernet/amd/pcnet32.ko 2023-08-09T13:16:40.335 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/net/ethernet/atheros 2023-08-09T13:16:40.335 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/net/ethernet/atheros/alx 2023-08-09T13:16:40.335 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/net/ethernet/atheros/alx/alx.ko 2023-08-09T13:16:40.335 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/net/ethernet/atheros/atl1c 2023-08-09T13:16:40.335 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/net/ethernet/atheros/atl1c/atl1c.ko 2023-08-09T13:16:40.335 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/net/ethernet/atheros/atl1e 2023-08-09T13:16:40.335 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/net/ethernet/atheros/atl1e/atl1e.ko 2023-08-09T13:16:40.335 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/net/ethernet/atheros/atlx 2023-08-09T13:16:40.336 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/net/ethernet/atheros/atlx/atl1.ko 2023-08-09T13:16:40.336 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/net/ethernet/atheros/atlx/atl2.ko 2023-08-09T13:16:40.336 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/net/ethernet/broadcom 2023-08-09T13:16:40.336 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/net/ethernet/broadcom/b44.ko 2023-08-09T13:16:40.336 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/net/ethernet/broadcom/bnx2.ko 2023-08-09T13:16:40.336 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/net/ethernet/broadcom/bnx2x 2023-08-09T13:16:40.336 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/net/ethernet/broadcom/bnx2x/bnx2x.ko 2023-08-09T13:16:40.337 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/net/ethernet/broadcom/cnic.ko 2023-08-09T13:16:40.337 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/net/ethernet/broadcom/genet 2023-08-09T13:16:40.337 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/net/ethernet/broadcom/genet/genet.ko 2023-08-09T13:16:40.337 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/net/ethernet/broadcom/tg3.ko 2023-08-09T13:16:40.337 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/net/ethernet/brocade 2023-08-09T13:16:40.337 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/net/ethernet/brocade/bna 2023-08-09T13:16:40.337 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/net/ethernet/brocade/bna/bna.ko 2023-08-09T13:16:40.337 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/net/ethernet/chelsio 2023-08-09T13:16:40.338 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/net/ethernet/chelsio/cxgb 2023-08-09T13:16:40.338 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/net/ethernet/chelsio/cxgb/cxgb.ko 2023-08-09T13:16:40.338 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/net/ethernet/chelsio/cxgb3 2023-08-09T13:16:40.338 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/net/ethernet/chelsio/cxgb3/cxgb3.ko 2023-08-09T13:16:40.338 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/net/ethernet/chelsio/cxgb4 2023-08-09T13:16:40.338 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/net/ethernet/chelsio/cxgb4/cxgb4.ko 2023-08-09T13:16:40.338 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/net/ethernet/chelsio/cxgb4vf 2023-08-09T13:16:40.338 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/net/ethernet/chelsio/cxgb4vf/cxgb4vf.ko 2023-08-09T13:16:40.339 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/net/ethernet/chelsio/libcxgb 2023-08-09T13:16:40.339 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/net/ethernet/chelsio/libcxgb/libcxgb.ko 2023-08-09T13:16:40.339 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/net/ethernet/cisco 2023-08-09T13:16:40.339 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/net/ethernet/cisco/enic 2023-08-09T13:16:40.339 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/net/ethernet/cisco/enic/enic.ko 2023-08-09T13:16:40.339 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/net/ethernet/dec 2023-08-09T13:16:40.339 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/net/ethernet/dec/tulip 2023-08-09T13:16:40.340 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/net/ethernet/dec/tulip/de2104x.ko 2023-08-09T13:16:40.340 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/net/ethernet/dec/tulip/dmfe.ko 2023-08-09T13:16:40.340 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/net/ethernet/dec/tulip/tulip.ko 2023-08-09T13:16:40.340 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/net/ethernet/dec/tulip/uli526x.ko 2023-08-09T13:16:40.340 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/net/ethernet/dec/tulip/winbond-840.ko 2023-08-09T13:16:40.340 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/net/ethernet/dec/tulip/xircom_cb.ko 2023-08-09T13:16:40.340 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/net/ethernet/dlink 2023-08-09T13:16:40.340 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/net/ethernet/dlink/dl2k.ko 2023-08-09T13:16:40.341 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/net/ethernet/dlink/sundance.ko 2023-08-09T13:16:40.341 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/net/ethernet/dnet.ko 2023-08-09T13:16:40.341 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/net/ethernet/emulex 2023-08-09T13:16:40.341 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/net/ethernet/emulex/benet 2023-08-09T13:16:40.341 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/net/ethernet/emulex/benet/be2net.ko 2023-08-09T13:16:40.341 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/net/ethernet/ethoc.ko 2023-08-09T13:16:40.341 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/net/ethernet/fealnx.ko 2023-08-09T13:16:40.341 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/net/ethernet/intel 2023-08-09T13:16:40.342 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/net/ethernet/intel/e100.ko 2023-08-09T13:16:40.342 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/net/ethernet/intel/e1000 2023-08-09T13:16:40.342 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/net/ethernet/intel/e1000/e1000.ko 2023-08-09T13:16:40.342 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/net/ethernet/intel/e1000e 2023-08-09T13:16:40.342 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/net/ethernet/intel/e1000e/e1000e.ko 2023-08-09T13:16:40.342 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/net/ethernet/intel/fm10k 2023-08-09T13:16:40.342 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/net/ethernet/intel/fm10k/fm10k.ko 2023-08-09T13:16:40.342 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/net/ethernet/intel/i40e 2023-08-09T13:16:40.343 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/net/ethernet/intel/i40e/i40e.ko 2023-08-09T13:16:40.343 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/net/ethernet/intel/iavf 2023-08-09T13:16:40.343 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/net/ethernet/intel/iavf/iavf.ko 2023-08-09T13:16:40.343 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/net/ethernet/intel/igb 2023-08-09T13:16:40.343 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/net/ethernet/intel/igb/igb.ko 2023-08-09T13:16:40.343 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/net/ethernet/intel/igbvf 2023-08-09T13:16:40.343 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/net/ethernet/intel/igbvf/igbvf.ko 2023-08-09T13:16:40.343 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/net/ethernet/intel/ixgbe 2023-08-09T13:16:40.344 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/net/ethernet/intel/ixgbe/ixgbe.ko 2023-08-09T13:16:40.344 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/net/ethernet/intel/ixgbevf 2023-08-09T13:16:40.344 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/net/ethernet/intel/ixgbevf/ixgbevf.ko 2023-08-09T13:16:40.344 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/net/ethernet/jme.ko 2023-08-09T13:16:40.344 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/net/ethernet/marvell 2023-08-09T13:16:40.344 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/net/ethernet/marvell/mvmdio.ko 2023-08-09T13:16:40.344 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/net/ethernet/marvell/skge.ko 2023-08-09T13:16:40.345 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/net/ethernet/marvell/sky2.ko 2023-08-09T13:16:40.345 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/net/ethernet/mellanox 2023-08-09T13:16:40.345 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/net/ethernet/mellanox/mlx4 2023-08-09T13:16:40.345 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/net/ethernet/mellanox/mlx4/mlx4_core.ko 2023-08-09T13:16:40.345 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/net/ethernet/mellanox/mlx4/mlx4_en.ko 2023-08-09T13:16:40.345 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/net/ethernet/mellanox/mlx5 2023-08-09T13:16:40.345 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/net/ethernet/mellanox/mlx5/core 2023-08-09T13:16:40.345 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/net/ethernet/mellanox/mlx5/core/mlx5_core.ko 2023-08-09T13:16:40.346 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/net/ethernet/micrel 2023-08-09T13:16:40.346 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/net/ethernet/micrel/ksz884x.ko 2023-08-09T13:16:40.346 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/net/ethernet/myricom 2023-08-09T13:16:40.346 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/net/ethernet/myricom/myri10ge 2023-08-09T13:16:40.346 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/net/ethernet/myricom/myri10ge/myri10ge.ko 2023-08-09T13:16:40.346 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/net/ethernet/natsemi 2023-08-09T13:16:40.346 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/net/ethernet/natsemi/natsemi.ko 2023-08-09T13:16:40.346 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/net/ethernet/natsemi/ns83820.ko 2023-08-09T13:16:40.347 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/net/ethernet/neterion 2023-08-09T13:16:40.347 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/net/ethernet/neterion/s2io.ko 2023-08-09T13:16:40.347 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/net/ethernet/nvidia 2023-08-09T13:16:40.347 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/net/ethernet/nvidia/forcedeth.ko 2023-08-09T13:16:40.347 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/net/ethernet/packetengines 2023-08-09T13:16:40.347 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/net/ethernet/packetengines/hamachi.ko 2023-08-09T13:16:40.347 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/net/ethernet/packetengines/yellowfin.ko 2023-08-09T13:16:40.347 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/net/ethernet/qlogic 2023-08-09T13:16:40.348 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/net/ethernet/qlogic/netxen 2023-08-09T13:16:40.348 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/net/ethernet/qlogic/netxen/netxen_nic.ko 2023-08-09T13:16:40.348 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/net/ethernet/qlogic/qla3xxx.ko 2023-08-09T13:16:40.348 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/net/ethernet/qlogic/qlcnic 2023-08-09T13:16:40.348 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/net/ethernet/qlogic/qlcnic/qlcnic.ko 2023-08-09T13:16:40.348 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/net/ethernet/rdc 2023-08-09T13:16:40.348 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/net/ethernet/rdc/r6040.ko 2023-08-09T13:16:40.348 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/net/ethernet/realtek 2023-08-09T13:16:40.349 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/net/ethernet/realtek/8139cp.ko 2023-08-09T13:16:40.349 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/net/ethernet/realtek/8139too.ko 2023-08-09T13:16:40.349 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/net/ethernet/realtek/atp.ko 2023-08-09T13:16:40.349 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/net/ethernet/realtek/r8169.ko 2023-08-09T13:16:40.349 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/net/ethernet/rocker 2023-08-09T13:16:40.349 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/net/ethernet/rocker/rocker.ko 2023-08-09T13:16:40.349 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/net/ethernet/sfc 2023-08-09T13:16:40.350 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/net/ethernet/sfc/sfc.ko 2023-08-09T13:16:40.350 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/net/ethernet/silan 2023-08-09T13:16:40.350 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/net/ethernet/silan/sc92031.ko 2023-08-09T13:16:40.350 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/net/ethernet/sis 2023-08-09T13:16:40.350 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/net/ethernet/sis/sis190.ko 2023-08-09T13:16:40.350 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/net/ethernet/sis/sis900.ko 2023-08-09T13:16:40.350 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/net/ethernet/smsc 2023-08-09T13:16:40.350 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/net/ethernet/smsc/epic100.ko 2023-08-09T13:16:40.351 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/net/ethernet/smsc/smc91c92_cs.ko 2023-08-09T13:16:40.351 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/net/ethernet/smsc/smsc911x.ko 2023-08-09T13:16:40.351 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/net/ethernet/smsc/smsc9420.ko 2023-08-09T13:16:40.351 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/net/ethernet/stmicro 2023-08-09T13:16:40.351 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/net/ethernet/stmicro/stmmac 2023-08-09T13:16:40.351 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/net/ethernet/stmicro/stmmac/dwmac-intel.ko 2023-08-09T13:16:40.351 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/net/ethernet/stmicro/stmmac/stmmac.ko 2023-08-09T13:16:40.351 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/net/ethernet/sun 2023-08-09T13:16:40.352 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/net/ethernet/sun/cassini.ko 2023-08-09T13:16:40.352 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/net/ethernet/sun/niu.ko 2023-08-09T13:16:40.352 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/net/ethernet/sun/sungem.ko 2023-08-09T13:16:40.352 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/net/ethernet/sun/sunhme.ko 2023-08-09T13:16:40.352 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/net/ethernet/tehuti 2023-08-09T13:16:40.352 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/net/ethernet/tehuti/tehuti.ko 2023-08-09T13:16:40.352 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/net/ethernet/ti 2023-08-09T13:16:40.353 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/net/ethernet/ti/tlan.ko 2023-08-09T13:16:40.353 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/net/ethernet/via 2023-08-09T13:16:40.353 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/net/ethernet/via/via-rhine.ko 2023-08-09T13:16:40.353 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/net/ethernet/via/via-velocity.ko 2023-08-09T13:16:40.353 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/net/ethernet/wiznet 2023-08-09T13:16:40.353 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/net/ethernet/wiznet/w5100.ko 2023-08-09T13:16:40.353 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/net/ethernet/wiznet/w5300.ko 2023-08-09T13:16:40.353 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/net/ethernet/xircom 2023-08-09T13:16:40.354 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/net/ethernet/xircom/xirc2ps_cs.ko 2023-08-09T13:16:40.354 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/net/geneve.ko 2023-08-09T13:16:40.354 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/net/hyperv 2023-08-09T13:16:40.354 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/net/hyperv/hv_netvsc.ko 2023-08-09T13:16:40.354 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/net/ieee802154 2023-08-09T13:16:40.354 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/net/ieee802154/fakelb.ko 2023-08-09T13:16:40.354 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/net/ifb.ko 2023-08-09T13:16:40.354 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/net/ipvlan 2023-08-09T13:16:40.355 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/net/ipvlan/ipvlan.ko 2023-08-09T13:16:40.355 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/net/macvlan.ko 2023-08-09T13:16:40.355 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/net/macvtap.ko 2023-08-09T13:16:40.355 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/net/mdio 2023-08-09T13:16:40.355 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/net/mdio.ko 2023-08-09T13:16:40.355 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/net/mdio/mdio-bcm-unimac.ko 2023-08-09T13:16:40.355 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/net/mdio/mdio-bitbang.ko 2023-08-09T13:16:40.355 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/net/mdio/mdio-regmap.ko 2023-08-09T13:16:40.356 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/net/mii.ko 2023-08-09T13:16:40.356 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/net/net_failover.ko 2023-08-09T13:16:40.356 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/net/netconsole.ko 2023-08-09T13:16:40.356 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/net/nlmon.ko 2023-08-09T13:16:40.356 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/net/pcs 2023-08-09T13:16:40.356 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/net/pcs/pcs-lynx.ko 2023-08-09T13:16:40.356 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/net/pcs/pcs_xpcs.ko 2023-08-09T13:16:40.356 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/net/phy 2023-08-09T13:16:40.357 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/net/phy/amd.ko 2023-08-09T13:16:40.357 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/net/phy/bcm-phy-lib.ko 2023-08-09T13:16:40.357 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/net/phy/bcm-phy-ptp.ko 2023-08-09T13:16:40.357 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/net/phy/bcm7xxx.ko 2023-08-09T13:16:40.357 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/net/phy/bcm87xx.ko 2023-08-09T13:16:40.357 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/net/phy/broadcom.ko 2023-08-09T13:16:40.357 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/net/phy/cicada.ko 2023-08-09T13:16:40.357 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/net/phy/davicom.ko 2023-08-09T13:16:40.358 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/net/phy/dp83640.ko 2023-08-09T13:16:40.358 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/net/phy/et1011c.ko 2023-08-09T13:16:40.358 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/net/phy/icplus.ko 2023-08-09T13:16:40.358 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/net/phy/lxt.ko 2023-08-09T13:16:40.358 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/net/phy/marvell.ko 2023-08-09T13:16:40.358 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/net/phy/micrel.ko 2023-08-09T13:16:40.358 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/net/phy/national.ko 2023-08-09T13:16:40.358 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/net/phy/phylink.ko 2023-08-09T13:16:40.359 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/net/phy/qsemi.ko 2023-08-09T13:16:40.359 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/net/phy/realtek.ko 2023-08-09T13:16:40.359 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/net/phy/smsc.ko 2023-08-09T13:16:40.359 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/net/phy/ste10Xp.ko 2023-08-09T13:16:40.359 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/net/phy/vitesse.ko 2023-08-09T13:16:40.359 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/net/ppp 2023-08-09T13:16:40.359 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/net/ppp/bsd_comp.ko 2023-08-09T13:16:40.359 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/net/ppp/ppp_async.ko 2023-08-09T13:16:40.360 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/net/ppp/ppp_deflate.ko 2023-08-09T13:16:40.360 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/net/ppp/ppp_generic.ko 2023-08-09T13:16:40.360 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/net/ppp/ppp_mppe.ko 2023-08-09T13:16:40.360 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/net/ppp/ppp_synctty.ko 2023-08-09T13:16:40.360 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/net/ppp/pppoe.ko 2023-08-09T13:16:40.360 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/net/ppp/pppox.ko 2023-08-09T13:16:40.360 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/net/ppp/pptp.ko 2023-08-09T13:16:40.361 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/net/slip 2023-08-09T13:16:40.361 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/net/slip/slhc.ko 2023-08-09T13:16:40.361 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/net/slip/slip.ko 2023-08-09T13:16:40.361 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/net/sungem_phy.ko 2023-08-09T13:16:40.361 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/net/tap.ko 2023-08-09T13:16:40.361 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/net/team 2023-08-09T13:16:40.361 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/net/team/team.ko 2023-08-09T13:16:40.361 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/net/team/team_mode_activebackup.ko 2023-08-09T13:16:40.362 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/net/team/team_mode_broadcast.ko 2023-08-09T13:16:40.362 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/net/team/team_mode_loadbalance.ko 2023-08-09T13:16:40.362 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/net/team/team_mode_random.ko 2023-08-09T13:16:40.362 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/net/team/team_mode_roundrobin.ko 2023-08-09T13:16:40.362 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/net/tun.ko 2023-08-09T13:16:40.362 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/net/veth.ko 2023-08-09T13:16:40.362 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/net/virtio_net.ko 2023-08-09T13:16:40.362 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/net/vmxnet3 2023-08-09T13:16:40.363 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/net/vmxnet3/vmxnet3.ko 2023-08-09T13:16:40.363 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/net/vxlan 2023-08-09T13:16:40.363 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/net/vxlan/vxlan.ko 2023-08-09T13:16:40.363 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/net/xen-netback 2023-08-09T13:16:40.363 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/net/xen-netback/xen-netback.ko 2023-08-09T13:16:40.363 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/net/xen-netfront.ko 2023-08-09T13:16:40.363 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/ntb 2023-08-09T13:16:40.363 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/ntb/ntb.ko 2023-08-09T13:16:40.364 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/nvme 2023-08-09T13:16:40.364 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/nvme/host 2023-08-09T13:16:40.364 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/nvme/host/nvme-core.ko 2023-08-09T13:16:40.364 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/nvme/host/nvme.ko 2023-08-09T13:16:40.364 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/parport 2023-08-09T13:16:40.364 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/parport/parport.ko 2023-08-09T13:16:40.364 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/parport/parport_cs.ko 2023-08-09T13:16:40.365 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/parport/parport_pc.ko 2023-08-09T13:16:40.365 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/parport/parport_serial.ko 2023-08-09T13:16:40.365 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/pci 2023-08-09T13:16:40.365 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/pci/hotplug 2023-08-09T13:16:40.365 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/pci/hotplug/acpiphp_ibm.ko 2023-08-09T13:16:40.365 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/pci/pcie 2023-08-09T13:16:40.365 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/pci/pcie/aer_inject.ko 2023-08-09T13:16:40.365 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/pci/xen-pcifront.ko 2023-08-09T13:16:40.366 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/pcmcia 2023-08-09T13:16:40.366 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/pcmcia/i82092.ko 2023-08-09T13:16:40.366 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/pcmcia/pd6729.ko 2023-08-09T13:16:40.366 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/pcmcia/yenta_socket.ko 2023-08-09T13:16:40.366 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/pinctrl 2023-08-09T13:16:40.366 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/pinctrl/intel 2023-08-09T13:16:40.366 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/pinctrl/intel/pinctrl-cherryview.ko 2023-08-09T13:16:40.366 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/platform 2023-08-09T13:16:40.367 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/platform/x86 2023-08-09T13:16:40.367 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/platform/x86/acer-wmi.ko 2023-08-09T13:16:40.367 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/platform/x86/acerhdf.ko 2023-08-09T13:16:40.367 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/platform/x86/apple-gmux.ko 2023-08-09T13:16:40.367 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/platform/x86/asus-laptop.ko 2023-08-09T13:16:40.367 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/platform/x86/asus-nb-wmi.ko 2023-08-09T13:16:40.367 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/platform/x86/asus-wmi.ko 2023-08-09T13:16:40.367 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/platform/x86/classmate-laptop.ko 2023-08-09T13:16:40.368 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/platform/x86/eeepc-laptop.ko 2023-08-09T13:16:40.368 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/platform/x86/eeepc-wmi.ko 2023-08-09T13:16:40.368 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/platform/x86/fujitsu-laptop.ko 2023-08-09T13:16:40.368 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/platform/x86/fujitsu-tablet.ko 2023-08-09T13:16:40.368 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/platform/x86/hdaps.ko 2023-08-09T13:16:40.368 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/platform/x86/intel 2023-08-09T13:16:40.368 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/platform/x86/intel/intel-rst.ko 2023-08-09T13:16:40.368 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/platform/x86/intel_ips.ko 2023-08-09T13:16:40.369 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/platform/x86/msi-wmi.ko 2023-08-09T13:16:40.369 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/platform/x86/mxm-wmi.ko 2023-08-09T13:16:40.369 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/platform/x86/panasonic-laptop.ko 2023-08-09T13:16:40.369 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/platform/x86/samsung-laptop.ko 2023-08-09T13:16:40.369 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/platform/x86/samsung-q10.ko 2023-08-09T13:16:40.369 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/platform/x86/topstar-laptop.ko 2023-08-09T13:16:40.369 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/platform/x86/toshiba_bluetooth.ko 2023-08-09T13:16:40.369 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/platform/x86/toshiba_haps.ko 2023-08-09T13:16:40.370 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/platform/x86/wmi-bmof.ko 2023-08-09T13:16:40.370 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/platform/x86/wmi.ko 2023-08-09T13:16:40.370 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/powercap 2023-08-09T13:16:40.370 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/powercap/intel_rapl_common.ko 2023-08-09T13:16:40.370 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/powercap/intel_rapl_msr.ko 2023-08-09T13:16:40.370 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/pps 2023-08-09T13:16:40.370 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/pps/clients 2023-08-09T13:16:40.370 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/pps/clients/pps-gpio.ko 2023-08-09T13:16:40.371 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/pps/clients/pps-ldisc.ko 2023-08-09T13:16:40.371 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/pps/clients/pps_parport.ko 2023-08-09T13:16:40.371 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/pps/pps_core.ko 2023-08-09T13:16:40.371 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/ptp 2023-08-09T13:16:40.371 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/ptp/ptp.ko 2023-08-09T13:16:40.371 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/ptp/ptp_kvm.ko 2023-08-09T13:16:40.371 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/rtc 2023-08-09T13:16:40.372 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/rtc/rtc-bq32k.ko 2023-08-09T13:16:40.372 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/rtc/rtc-bq4802.ko 2023-08-09T13:16:40.372 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/rtc/rtc-ds1286.ko 2023-08-09T13:16:40.372 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/rtc/rtc-ds1307.ko 2023-08-09T13:16:40.372 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/rtc/rtc-ds1374.ko 2023-08-09T13:16:40.372 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/rtc/rtc-ds1511.ko 2023-08-09T13:16:40.372 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/rtc/rtc-ds1553.ko 2023-08-09T13:16:40.372 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/rtc/rtc-ds1672.ko 2023-08-09T13:16:40.373 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/rtc/rtc-ds1742.ko 2023-08-09T13:16:40.373 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/rtc/rtc-ds2404.ko 2023-08-09T13:16:40.373 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/rtc/rtc-ds3232.ko 2023-08-09T13:16:40.373 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/rtc/rtc-em3027.ko 2023-08-09T13:16:40.373 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/rtc/rtc-fm3130.ko 2023-08-09T13:16:40.373 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/rtc/rtc-isl12022.ko 2023-08-09T13:16:40.373 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/rtc/rtc-isl1208.ko 2023-08-09T13:16:40.373 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/rtc/rtc-m41t80.ko 2023-08-09T13:16:40.374 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/rtc/rtc-m48t35.ko 2023-08-09T13:16:40.374 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/rtc/rtc-m48t59.ko 2023-08-09T13:16:40.374 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/rtc/rtc-max6900.ko 2023-08-09T13:16:40.374 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/rtc/rtc-msm6242.ko 2023-08-09T13:16:40.374 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/rtc/rtc-pcf2127.ko 2023-08-09T13:16:40.374 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/rtc/rtc-pcf85063.ko 2023-08-09T13:16:40.374 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/rtc/rtc-pcf8523.ko 2023-08-09T13:16:40.374 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/rtc/rtc-pcf8563.ko 2023-08-09T13:16:40.375 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/rtc/rtc-pcf8583.ko 2023-08-09T13:16:40.375 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/rtc/rtc-rp5c01.ko 2023-08-09T13:16:40.375 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/rtc/rtc-rs5c372.ko 2023-08-09T13:16:40.375 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/rtc/rtc-rv3029c2.ko 2023-08-09T13:16:40.375 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/rtc/rtc-rx8025.ko 2023-08-09T13:16:40.375 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/rtc/rtc-rx8581.ko 2023-08-09T13:16:40.375 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/rtc/rtc-stk17ta8.ko 2023-08-09T13:16:40.375 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/rtc/rtc-x1205.ko 2023-08-09T13:16:40.376 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/scsi 2023-08-09T13:16:40.376 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/scsi/3w-9xxx.ko 2023-08-09T13:16:40.376 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/scsi/3w-sas.ko 2023-08-09T13:16:40.376 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/scsi/3w-xxxx.ko 2023-08-09T13:16:40.376 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/scsi/BusLogic.ko 2023-08-09T13:16:40.376 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/scsi/a100u2w.ko 2023-08-09T13:16:40.376 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/scsi/aacraid 2023-08-09T13:16:40.376 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/scsi/aacraid/aacraid.ko 2023-08-09T13:16:40.377 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/scsi/advansys.ko 2023-08-09T13:16:40.377 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/scsi/aic7xxx 2023-08-09T13:16:40.377 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/scsi/aic7xxx/aic79xx.ko 2023-08-09T13:16:40.377 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/scsi/aic7xxx/aic7xxx.ko 2023-08-09T13:16:40.377 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/scsi/am53c974.ko 2023-08-09T13:16:40.377 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/scsi/arcmsr 2023-08-09T13:16:40.377 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/scsi/arcmsr/arcmsr.ko 2023-08-09T13:16:40.377 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/scsi/atp870u.ko 2023-08-09T13:16:40.378 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/scsi/be2iscsi 2023-08-09T13:16:40.378 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/scsi/be2iscsi/be2iscsi.ko 2023-08-09T13:16:40.378 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/scsi/bfa 2023-08-09T13:16:40.378 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/scsi/bfa/bfa.ko 2023-08-09T13:16:40.378 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/scsi/bnx2fc 2023-08-09T13:16:40.378 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/scsi/bnx2fc/bnx2fc.ko 2023-08-09T13:16:40.378 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/scsi/bnx2i 2023-08-09T13:16:40.378 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/scsi/bnx2i/bnx2i.ko 2023-08-09T13:16:40.379 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/scsi/ch.ko 2023-08-09T13:16:40.379 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/scsi/csiostor 2023-08-09T13:16:40.379 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/scsi/csiostor/csiostor.ko 2023-08-09T13:16:40.379 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/scsi/cxgbi 2023-08-09T13:16:40.379 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/scsi/cxgbi/cxgb3i 2023-08-09T13:16:40.379 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/scsi/cxgbi/cxgb3i/cxgb3i.ko 2023-08-09T13:16:40.379 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/scsi/cxgbi/cxgb4i 2023-08-09T13:16:40.379 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/scsi/cxgbi/cxgb4i/cxgb4i.ko 2023-08-09T13:16:40.380 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/scsi/cxgbi/libcxgbi.ko 2023-08-09T13:16:40.380 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/scsi/dc395x.ko 2023-08-09T13:16:40.380 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/scsi/device_handler 2023-08-09T13:16:40.380 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/scsi/device_handler/scsi_dh_alua.ko 2023-08-09T13:16:40.380 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/scsi/device_handler/scsi_dh_emc.ko 2023-08-09T13:16:40.380 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/scsi/device_handler/scsi_dh_hp_sw.ko 2023-08-09T13:16:40.380 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/scsi/device_handler/scsi_dh_rdac.ko 2023-08-09T13:16:40.381 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/scsi/dmx3191d.ko 2023-08-09T13:16:40.381 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/scsi/esas2r 2023-08-09T13:16:40.381 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/scsi/esas2r/esas2r.ko 2023-08-09T13:16:40.381 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/scsi/esp_scsi.ko 2023-08-09T13:16:40.381 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/scsi/fcoe 2023-08-09T13:16:40.381 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/scsi/fcoe/fcoe.ko 2023-08-09T13:16:40.381 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/scsi/fcoe/libfcoe.ko 2023-08-09T13:16:40.381 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/scsi/fnic 2023-08-09T13:16:40.382 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/scsi/fnic/fnic.ko 2023-08-09T13:16:40.382 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/scsi/hpsa.ko 2023-08-09T13:16:40.382 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/scsi/hptiop.ko 2023-08-09T13:16:40.382 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/scsi/hv_storvsc.ko 2023-08-09T13:16:40.382 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/scsi/initio.ko 2023-08-09T13:16:40.382 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/scsi/ipr.ko 2023-08-09T13:16:40.382 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/scsi/ips.ko 2023-08-09T13:16:40.382 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/scsi/isci 2023-08-09T13:16:40.383 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/scsi/isci/isci.ko 2023-08-09T13:16:40.383 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/scsi/iscsi_boot_sysfs.ko 2023-08-09T13:16:40.383 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/scsi/iscsi_tcp.ko 2023-08-09T13:16:40.383 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/scsi/libfc 2023-08-09T13:16:40.383 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/scsi/libfc/libfc.ko 2023-08-09T13:16:40.383 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/scsi/libiscsi.ko 2023-08-09T13:16:40.383 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/scsi/libiscsi_tcp.ko 2023-08-09T13:16:40.383 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/scsi/libsas 2023-08-09T13:16:40.384 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/scsi/libsas/libsas.ko 2023-08-09T13:16:40.384 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/scsi/lpfc 2023-08-09T13:16:40.384 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/scsi/lpfc/lpfc.ko 2023-08-09T13:16:40.384 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/scsi/megaraid 2023-08-09T13:16:40.384 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/scsi/megaraid.ko 2023-08-09T13:16:40.384 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/scsi/megaraid/megaraid_mbox.ko 2023-08-09T13:16:40.384 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/scsi/megaraid/megaraid_mm.ko 2023-08-09T13:16:40.384 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/scsi/megaraid/megaraid_sas.ko 2023-08-09T13:16:40.385 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/scsi/mpt3sas 2023-08-09T13:16:40.385 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/scsi/mpt3sas/mpt3sas.ko 2023-08-09T13:16:40.385 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/scsi/mvsas 2023-08-09T13:16:40.385 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/scsi/mvsas/mvsas.ko 2023-08-09T13:16:40.385 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/scsi/mvumi.ko 2023-08-09T13:16:40.385 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/scsi/pm8001 2023-08-09T13:16:40.385 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/scsi/pm8001/pm80xx.ko 2023-08-09T13:16:40.385 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/scsi/pmcraid.ko 2023-08-09T13:16:40.386 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/scsi/qla1280.ko 2023-08-09T13:16:40.386 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/scsi/qla2xxx 2023-08-09T13:16:40.386 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/scsi/qla2xxx/qla2xxx.ko 2023-08-09T13:16:40.386 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/scsi/qla2xxx/tcm_qla2xxx.ko 2023-08-09T13:16:40.386 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/scsi/qla4xxx 2023-08-09T13:16:40.386 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/scsi/qla4xxx/qla4xxx.ko 2023-08-09T13:16:40.386 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/scsi/raid_class.ko 2023-08-09T13:16:40.386 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/scsi/scsi_debug.ko 2023-08-09T13:16:40.387 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/scsi/scsi_transport_fc.ko 2023-08-09T13:16:40.387 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/scsi/scsi_transport_iscsi.ko 2023-08-09T13:16:40.387 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/scsi/scsi_transport_sas.ko 2023-08-09T13:16:40.387 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/scsi/scsi_transport_spi.ko 2023-08-09T13:16:40.387 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/scsi/scsi_transport_srp.ko 2023-08-09T13:16:40.387 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/scsi/ses.ko 2023-08-09T13:16:40.387 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/scsi/st.ko 2023-08-09T13:16:40.388 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/scsi/stex.ko 2023-08-09T13:16:40.388 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/scsi/sym53c8xx_2 2023-08-09T13:16:40.388 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/scsi/sym53c8xx_2/sym53c8xx.ko 2023-08-09T13:16:40.388 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/scsi/virtio_scsi.ko 2023-08-09T13:16:40.388 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/scsi/vmw_pvscsi.ko 2023-08-09T13:16:40.388 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/scsi/wd719x.ko 2023-08-09T13:16:40.388 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/ssb 2023-08-09T13:16:40.388 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/ssb/ssb.ko 2023-08-09T13:16:40.389 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/target 2023-08-09T13:16:40.389 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/target/iscsi 2023-08-09T13:16:40.389 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/target/iscsi/iscsi_target_mod.ko 2023-08-09T13:16:40.389 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/target/loopback 2023-08-09T13:16:40.389 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/target/loopback/tcm_loop.ko 2023-08-09T13:16:40.389 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/target/target_core_file.ko 2023-08-09T13:16:40.389 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/target/target_core_iblock.ko 2023-08-09T13:16:40.389 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/target/target_core_mod.ko 2023-08-09T13:16:40.390 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/target/target_core_pscsi.ko 2023-08-09T13:16:40.390 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/target/target_core_user.ko 2023-08-09T13:16:40.390 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/target/tcm_fc 2023-08-09T13:16:40.390 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/target/tcm_fc/tcm_fc.ko 2023-08-09T13:16:40.390 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/thermal 2023-08-09T13:16:40.390 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/thermal/intel 2023-08-09T13:16:40.390 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/thermal/intel/int340x_thermal 2023-08-09T13:16:40.390 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/thermal/intel/int340x_thermal/acpi_thermal_rel.ko 2023-08-09T13:16:40.391 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/thermal/intel/int340x_thermal/int3400_thermal.ko 2023-08-09T13:16:40.391 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/thermal/intel/int340x_thermal/int3401_thermal.ko 2023-08-09T13:16:40.391 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/thermal/intel/int340x_thermal/int3402_thermal.ko 2023-08-09T13:16:40.391 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/thermal/intel/int340x_thermal/int3403_thermal.ko 2023-08-09T13:16:40.391 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/thermal/intel/int340x_thermal/int340x_thermal_zone.ko 2023-08-09T13:16:40.391 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_device.ko 2023-08-09T13:16:40.391 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_device_pci.ko 2023-08-09T13:16:40.391 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_device_pci_legacy.ko 2023-08-09T13:16:40.392 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_mbox.ko 2023-08-09T13:16:40.392 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_rapl.ko 2023-08-09T13:16:40.392 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_rfim.ko 2023-08-09T13:16:40.392 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/thermal/intel/intel_soc_dts_iosf.ko 2023-08-09T13:16:40.392 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/thermal/intel/intel_soc_dts_thermal.ko 2023-08-09T13:16:40.392 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/thermal/intel/x86_pkg_temp_thermal.ko 2023-08-09T13:16:40.392 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/tty 2023-08-09T13:16:40.393 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/tty/ipwireless 2023-08-09T13:16:40.393 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/tty/ipwireless/ipwireless.ko 2023-08-09T13:16:40.393 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/tty/n_gsm.ko 2023-08-09T13:16:40.393 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/tty/n_hdlc.ko 2023-08-09T13:16:40.393 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/tty/nozomi.ko 2023-08-09T13:16:40.393 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/tty/serial 2023-08-09T13:16:40.393 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/tty/serial/8250 2023-08-09T13:16:40.393 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/tty/serial/8250/serial_cs.ko 2023-08-09T13:16:40.394 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/tty/serial/arc_uart.ko 2023-08-09T13:16:40.394 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/tty/serial/jsm 2023-08-09T13:16:40.394 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/tty/serial/jsm/jsm.ko 2023-08-09T13:16:40.394 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/tty/synclink_gt.ko 2023-08-09T13:16:40.394 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/ufs 2023-08-09T13:16:40.394 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/ufs/core 2023-08-09T13:16:40.394 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/ufs/core/ufshcd-core.ko 2023-08-09T13:16:40.394 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/ufs/host 2023-08-09T13:16:40.395 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/ufs/host/ufshcd-pci.ko 2023-08-09T13:16:40.395 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/uio 2023-08-09T13:16:40.395 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/uio/uio.ko 2023-08-09T13:16:40.395 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/uio/uio_aec.ko 2023-08-09T13:16:40.395 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/uio/uio_cif.ko 2023-08-09T13:16:40.395 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/uio/uio_pci_generic.ko 2023-08-09T13:16:40.395 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/uio/uio_sercos3.ko 2023-08-09T13:16:40.395 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/usb 2023-08-09T13:16:40.396 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/usb/class 2023-08-09T13:16:40.396 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/usb/class/cdc-acm.ko 2023-08-09T13:16:40.396 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/usb/class/cdc-wdm.ko 2023-08-09T13:16:40.396 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/usb/class/usblp.ko 2023-08-09T13:16:40.396 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/usb/class/usbtmc.ko 2023-08-09T13:16:40.396 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/usb/host 2023-08-09T13:16:40.396 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/usb/host/sl811-hcd.ko 2023-08-09T13:16:40.396 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/usb/image 2023-08-09T13:16:40.397 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/usb/image/mdc800.ko 2023-08-09T13:16:40.397 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/usb/image/microtek.ko 2023-08-09T13:16:40.397 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/usb/misc 2023-08-09T13:16:40.397 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/usb/misc/adutux.ko 2023-08-09T13:16:40.397 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/usb/misc/appledisplay.ko 2023-08-09T13:16:40.397 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/usb/misc/emi26.ko 2023-08-09T13:16:40.397 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/usb/misc/emi62.ko 2023-08-09T13:16:40.397 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/usb/misc/ezusb.ko 2023-08-09T13:16:40.398 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/usb/misc/idmouse.ko 2023-08-09T13:16:40.398 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/usb/misc/iowarrior.ko 2023-08-09T13:16:40.398 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/usb/misc/isight_firmware.ko 2023-08-09T13:16:40.398 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/usb/misc/ldusb.ko 2023-08-09T13:16:40.398 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/usb/misc/legousbtower.ko 2023-08-09T13:16:40.398 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/usb/misc/sisusbvga 2023-08-09T13:16:40.398 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/usb/misc/sisusbvga/sisusbvga.ko 2023-08-09T13:16:40.399 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/usb/misc/trancevibrator.ko 2023-08-09T13:16:40.399 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/usb/misc/usb3503.ko 2023-08-09T13:16:40.399 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/usb/misc/usblcd.ko 2023-08-09T13:16:40.399 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/usb/misc/usbsevseg.ko 2023-08-09T13:16:40.399 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/usb/misc/uss720.ko 2023-08-09T13:16:40.399 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/usb/misc/yurex.ko 2023-08-09T13:16:40.399 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/usb/phy 2023-08-09T13:16:40.399 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/usb/phy/phy-generic.ko 2023-08-09T13:16:40.400 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/usb/serial 2023-08-09T13:16:40.400 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/usb/serial/aircable.ko 2023-08-09T13:16:40.400 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/usb/serial/ark3116.ko 2023-08-09T13:16:40.400 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/usb/serial/belkin_sa.ko 2023-08-09T13:16:40.400 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/usb/serial/ch341.ko 2023-08-09T13:16:40.400 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/usb/serial/cp210x.ko 2023-08-09T13:16:40.400 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/usb/serial/cyberjack.ko 2023-08-09T13:16:40.400 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/usb/serial/cypress_m8.ko 2023-08-09T13:16:40.401 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/usb/serial/digi_acceleport.ko 2023-08-09T13:16:40.401 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/usb/serial/empeg.ko 2023-08-09T13:16:40.401 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/usb/serial/ftdi_sio.ko 2023-08-09T13:16:40.401 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/usb/serial/garmin_gps.ko 2023-08-09T13:16:40.401 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/usb/serial/io_edgeport.ko 2023-08-09T13:16:40.401 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/usb/serial/io_ti.ko 2023-08-09T13:16:40.401 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/usb/serial/ipaq.ko 2023-08-09T13:16:40.401 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/usb/serial/ipw.ko 2023-08-09T13:16:40.402 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/usb/serial/ir-usb.ko 2023-08-09T13:16:40.402 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/usb/serial/iuu_phoenix.ko 2023-08-09T13:16:40.402 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/usb/serial/keyspan.ko 2023-08-09T13:16:40.402 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/usb/serial/keyspan_pda.ko 2023-08-09T13:16:40.402 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/usb/serial/kl5kusb105.ko 2023-08-09T13:16:40.402 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/usb/serial/kobil_sct.ko 2023-08-09T13:16:40.402 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/usb/serial/mct_u232.ko 2023-08-09T13:16:40.402 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/usb/serial/mos7720.ko 2023-08-09T13:16:40.403 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/usb/serial/mos7840.ko 2023-08-09T13:16:40.403 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/usb/serial/navman.ko 2023-08-09T13:16:40.403 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/usb/serial/omninet.ko 2023-08-09T13:16:40.403 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/usb/serial/opticon.ko 2023-08-09T13:16:40.403 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/usb/serial/option.ko 2023-08-09T13:16:40.403 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/usb/serial/oti6858.ko 2023-08-09T13:16:40.403 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/usb/serial/pl2303.ko 2023-08-09T13:16:40.403 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/usb/serial/qcaux.ko 2023-08-09T13:16:40.404 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/usb/serial/qcserial.ko 2023-08-09T13:16:40.404 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/usb/serial/quatech2.ko 2023-08-09T13:16:40.404 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/usb/serial/safe_serial.ko 2023-08-09T13:16:40.404 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/usb/serial/sierra.ko 2023-08-09T13:16:40.404 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/usb/serial/spcp8x5.ko 2023-08-09T13:16:40.404 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/usb/serial/ssu100.ko 2023-08-09T13:16:40.404 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/usb/serial/symbolserial.ko 2023-08-09T13:16:40.404 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/usb/serial/ti_usb_3410_5052.ko 2023-08-09T13:16:40.405 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/usb/serial/usb-serial-simple.ko 2023-08-09T13:16:40.405 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/usb/serial/usb_debug.ko 2023-08-09T13:16:40.405 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/usb/serial/usb_wwan.ko 2023-08-09T13:16:40.405 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/usb/serial/visor.ko 2023-08-09T13:16:40.405 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/usb/serial/whiteheat.ko 2023-08-09T13:16:40.405 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/usb/serial/xsens_mt.ko 2023-08-09T13:16:40.405 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/usb/storage 2023-08-09T13:16:40.405 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/usb/storage/uas.ko 2023-08-09T13:16:40.406 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/usb/storage/ums-alauda.ko 2023-08-09T13:16:40.406 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/usb/storage/ums-cypress.ko 2023-08-09T13:16:40.406 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/usb/storage/ums-datafab.ko 2023-08-09T13:16:40.406 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/usb/storage/ums-eneub6250.ko 2023-08-09T13:16:40.406 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/usb/storage/ums-freecom.ko 2023-08-09T13:16:40.406 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/usb/storage/ums-isd200.ko 2023-08-09T13:16:40.406 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/usb/storage/ums-jumpshot.ko 2023-08-09T13:16:40.407 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/usb/storage/ums-karma.ko 2023-08-09T13:16:40.407 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/usb/storage/ums-onetouch.ko 2023-08-09T13:16:40.407 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/usb/storage/ums-realtek.ko 2023-08-09T13:16:40.407 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/usb/storage/ums-sddr09.ko 2023-08-09T13:16:40.407 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/usb/storage/ums-sddr55.ko 2023-08-09T13:16:40.407 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/usb/storage/ums-usbat.ko 2023-08-09T13:16:40.407 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/usb/storage/usb-storage.ko 2023-08-09T13:16:40.407 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/usb/usbip 2023-08-09T13:16:40.408 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/usb/usbip/usbip-core.ko 2023-08-09T13:16:40.408 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/usb/usbip/usbip-host.ko 2023-08-09T13:16:40.408 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/usb/usbip/vhci-hcd.ko 2023-08-09T13:16:40.408 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/vfio 2023-08-09T13:16:40.408 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/vfio/pci 2023-08-09T13:16:40.408 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/vfio/pci/vfio-pci-core.ko 2023-08-09T13:16:40.408 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/vfio/pci/vfio-pci.ko 2023-08-09T13:16:40.408 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/vfio/vfio.ko 2023-08-09T13:16:40.409 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/vfio/vfio_iommu_type1.ko 2023-08-09T13:16:40.409 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/vhost 2023-08-09T13:16:40.409 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/vhost/vhost.ko 2023-08-09T13:16:40.409 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/vhost/vhost_iotlb.ko 2023-08-09T13:16:40.409 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/vhost/vhost_net.ko 2023-08-09T13:16:40.409 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/vhost/vhost_scsi.ko 2023-08-09T13:16:40.409 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/video 2023-08-09T13:16:40.409 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/video/backlight 2023-08-09T13:16:40.410 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/video/backlight/apple_bl.ko 2023-08-09T13:16:40.410 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/video/backlight/lcd.ko 2023-08-09T13:16:40.410 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/video/backlight/platform_lcd.ko 2023-08-09T13:16:40.410 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/virtio 2023-08-09T13:16:40.410 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/virtio/virtio.ko 2023-08-09T13:16:40.410 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/virtio/virtio_balloon.ko 2023-08-09T13:16:40.410 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/virtio/virtio_mmio.ko 2023-08-09T13:16:40.410 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/virtio/virtio_pci.ko 2023-08-09T13:16:40.411 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/virtio/virtio_pci_legacy_dev.ko 2023-08-09T13:16:40.411 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/virtio/virtio_pci_modern_dev.ko 2023-08-09T13:16:40.411 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/virtio/virtio_ring.ko 2023-08-09T13:16:40.411 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/w1 2023-08-09T13:16:40.411 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/w1/masters 2023-08-09T13:16:40.411 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/w1/masters/ds2482.ko 2023-08-09T13:16:40.411 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/w1/masters/ds2490.ko 2023-08-09T13:16:40.411 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/w1/slaves 2023-08-09T13:16:40.412 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/w1/slaves/w1_ds2406.ko 2023-08-09T13:16:40.412 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/w1/slaves/w1_ds2408.ko 2023-08-09T13:16:40.412 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/w1/slaves/w1_ds2413.ko 2023-08-09T13:16:40.412 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/w1/slaves/w1_ds2423.ko 2023-08-09T13:16:40.412 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/w1/slaves/w1_ds2431.ko 2023-08-09T13:16:40.412 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/w1/slaves/w1_ds2433.ko 2023-08-09T13:16:40.412 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/w1/slaves/w1_ds2780.ko 2023-08-09T13:16:40.413 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/w1/slaves/w1_ds2781.ko 2023-08-09T13:16:40.413 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/w1/slaves/w1_ds28e04.ko 2023-08-09T13:16:40.413 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/w1/slaves/w1_smem.ko 2023-08-09T13:16:40.413 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/w1/slaves/w1_therm.ko 2023-08-09T13:16:40.413 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/w1/wire.ko 2023-08-09T13:16:40.413 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/watchdog 2023-08-09T13:16:40.413 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/watchdog/alim1535_wdt.ko 2023-08-09T13:16:40.413 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/watchdog/alim7101_wdt.ko 2023-08-09T13:16:40.414 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/watchdog/f71808e_wdt.ko 2023-08-09T13:16:40.414 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/watchdog/hpwdt.ko 2023-08-09T13:16:40.414 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/watchdog/i6300esb.ko 2023-08-09T13:16:40.414 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/watchdog/iTCO_vendor_support.ko 2023-08-09T13:16:40.414 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/watchdog/iTCO_wdt.ko 2023-08-09T13:16:40.414 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/watchdog/ib700wdt.ko 2023-08-09T13:16:40.414 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/watchdog/ibmasr.ko 2023-08-09T13:16:40.414 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/watchdog/ie6xx_wdt.ko 2023-08-09T13:16:40.415 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/watchdog/it8712f_wdt.ko 2023-08-09T13:16:40.415 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/watchdog/it87_wdt.ko 2023-08-09T13:16:40.415 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/watchdog/machzwd.ko 2023-08-09T13:16:40.415 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/watchdog/nv_tco.ko 2023-08-09T13:16:40.415 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/watchdog/pcwd_pci.ko 2023-08-09T13:16:40.415 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/watchdog/pcwd_usb.ko 2023-08-09T13:16:40.415 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/watchdog/sbc_fitpc2_wdt.ko 2023-08-09T13:16:40.415 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/watchdog/sch311x_wdt.ko 2023-08-09T13:16:40.416 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/watchdog/softdog.ko 2023-08-09T13:16:40.416 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/watchdog/sp5100_tco.ko 2023-08-09T13:16:40.416 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/watchdog/via_wdt.ko 2023-08-09T13:16:40.416 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/watchdog/w83627hf_wdt.ko 2023-08-09T13:16:40.416 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/watchdog/w83877f_wdt.ko 2023-08-09T13:16:40.416 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/watchdog/w83977f_wdt.ko 2023-08-09T13:16:40.416 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/watchdog/wdt_pci.ko 2023-08-09T13:16:40.416 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/watchdog/xen_wdt.ko 2023-08-09T13:16:40.417 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/xen 2023-08-09T13:16:40.417 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/xen/xen-acpi-processor.ko 2023-08-09T13:16:40.417 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/xen/xen-evtchn.ko 2023-08-09T13:16:40.417 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/xen/xen-gntalloc.ko 2023-08-09T13:16:40.417 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/xen/xen-gntdev.ko 2023-08-09T13:16:40.417 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/xen/xen-pciback 2023-08-09T13:16:40.417 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/xen/xen-pciback/xen-pciback.ko 2023-08-09T13:16:40.417 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/xen/xen-privcmd.ko 2023-08-09T13:16:40.418 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/xen/xenfs 2023-08-09T13:16:40.418 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/xen/xenfs/xenfs.ko 2023-08-09T13:16:40.418 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/fs 2023-08-09T13:16:40.418 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/fs/9p 2023-08-09T13:16:40.418 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/fs/9p/9p.ko 2023-08-09T13:16:40.418 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/fs/affs 2023-08-09T13:16:40.418 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/fs/affs/affs.ko 2023-08-09T13:16:40.418 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/fs/befs 2023-08-09T13:16:40.419 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/fs/befs/befs.ko 2023-08-09T13:16:40.419 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/fs/binfmt_misc.ko 2023-08-09T13:16:40.419 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/fs/btrfs 2023-08-09T13:16:40.419 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/fs/btrfs/btrfs.ko 2023-08-09T13:16:40.419 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/fs/cachefiles 2023-08-09T13:16:40.419 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/fs/cachefiles/cachefiles.ko 2023-08-09T13:16:40.419 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/fs/ceph 2023-08-09T13:16:40.419 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/fs/ceph/ceph.ko 2023-08-09T13:16:40.420 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/fs/coda 2023-08-09T13:16:40.420 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/fs/coda/coda.ko 2023-08-09T13:16:40.420 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/fs/cramfs 2023-08-09T13:16:40.420 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/fs/cramfs/cramfs.ko 2023-08-09T13:16:40.420 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/fs/dlm 2023-08-09T13:16:40.420 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/fs/dlm/dlm.ko 2023-08-09T13:16:40.420 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/fs/ecryptfs 2023-08-09T13:16:40.420 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/fs/ecryptfs/ecryptfs.ko 2023-08-09T13:16:40.421 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/fs/f2fs 2023-08-09T13:16:40.421 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/fs/f2fs/f2fs.ko 2023-08-09T13:16:40.421 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/fs/fat 2023-08-09T13:16:40.421 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/fs/fat/fat.ko 2023-08-09T13:16:40.421 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/fs/fat/msdos.ko 2023-08-09T13:16:40.421 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/fs/fat/vfat.ko 2023-08-09T13:16:40.421 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/fs/fscache 2023-08-09T13:16:40.421 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/fs/fscache/fscache.ko 2023-08-09T13:16:40.422 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/fs/fuse 2023-08-09T13:16:40.422 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/fs/fuse/cuse.ko 2023-08-09T13:16:40.422 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/fs/fuse/fuse.ko 2023-08-09T13:16:40.422 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/fs/gfs2 2023-08-09T13:16:40.422 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/fs/gfs2/gfs2.ko 2023-08-09T13:16:40.422 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/fs/hfs 2023-08-09T13:16:40.422 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/fs/hfs/hfs.ko 2023-08-09T13:16:40.423 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/fs/hfsplus 2023-08-09T13:16:40.423 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/fs/hfsplus/hfsplus.ko 2023-08-09T13:16:40.423 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/fs/isofs 2023-08-09T13:16:40.423 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/fs/isofs/isofs.ko 2023-08-09T13:16:40.423 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/fs/jfs 2023-08-09T13:16:40.423 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/fs/jfs/jfs.ko 2023-08-09T13:16:40.423 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/fs/lockd 2023-08-09T13:16:40.423 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/fs/lockd/lockd.ko 2023-08-09T13:16:40.424 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/fs/minix 2023-08-09T13:16:40.424 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/fs/minix/minix.ko 2023-08-09T13:16:40.424 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/fs/netfs 2023-08-09T13:16:40.424 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/fs/netfs/netfs.ko 2023-08-09T13:16:40.424 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/fs/nfs 2023-08-09T13:16:40.424 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/fs/nfs/blocklayout 2023-08-09T13:16:40.424 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/fs/nfs/blocklayout/blocklayoutdriver.ko 2023-08-09T13:16:40.424 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/fs/nfs/filelayout 2023-08-09T13:16:40.425 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/fs/nfs/filelayout/nfs_layout_nfsv41_files.ko 2023-08-09T13:16:40.425 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/fs/nfs/flexfilelayout 2023-08-09T13:16:40.425 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/fs/nfs/flexfilelayout/nfs_layout_flexfiles.ko 2023-08-09T13:16:40.425 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/fs/nfs/nfs.ko 2023-08-09T13:16:40.425 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/fs/nfs/nfsv3.ko 2023-08-09T13:16:40.425 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/fs/nfs/nfsv4.ko 2023-08-09T13:16:40.425 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/fs/nfs_common 2023-08-09T13:16:40.425 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/fs/nfs_common/grace.ko 2023-08-09T13:16:40.426 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/fs/nfs_common/nfs_acl.ko 2023-08-09T13:16:40.426 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/fs/nfsd 2023-08-09T13:16:40.426 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/fs/nfsd/nfsd.ko 2023-08-09T13:16:40.426 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/fs/nilfs2 2023-08-09T13:16:40.426 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/fs/nilfs2/nilfs2.ko 2023-08-09T13:16:40.426 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/fs/nls 2023-08-09T13:16:40.426 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/fs/nls/mac-celtic.ko 2023-08-09T13:16:40.426 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/fs/nls/mac-centeuro.ko 2023-08-09T13:16:40.427 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/fs/nls/mac-croatian.ko 2023-08-09T13:16:40.427 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/fs/nls/mac-cyrillic.ko 2023-08-09T13:16:40.427 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/fs/nls/mac-gaelic.ko 2023-08-09T13:16:40.427 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/fs/nls/mac-greek.ko 2023-08-09T13:16:40.428 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/fs/nls/mac-iceland.ko 2023-08-09T13:16:40.428 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/fs/nls/mac-inuit.ko 2023-08-09T13:16:40.428 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/fs/nls/mac-roman.ko 2023-08-09T13:16:40.428 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/fs/nls/mac-romanian.ko 2023-08-09T13:16:40.428 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/fs/nls/mac-turkish.ko 2023-08-09T13:16:40.428 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/fs/nls/nls_cp1250.ko 2023-08-09T13:16:40.428 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/fs/nls/nls_cp1251.ko 2023-08-09T13:16:40.429 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/fs/nls/nls_cp1255.ko 2023-08-09T13:16:40.429 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/fs/nls/nls_cp737.ko 2023-08-09T13:16:40.429 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/fs/nls/nls_cp775.ko 2023-08-09T13:16:40.429 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/fs/nls/nls_cp850.ko 2023-08-09T13:16:40.429 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/fs/nls/nls_cp852.ko 2023-08-09T13:16:40.429 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/fs/nls/nls_cp855.ko 2023-08-09T13:16:40.429 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/fs/nls/nls_cp857.ko 2023-08-09T13:16:40.429 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/fs/nls/nls_cp860.ko 2023-08-09T13:16:40.430 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/fs/nls/nls_cp861.ko 2023-08-09T13:16:40.430 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/fs/nls/nls_cp862.ko 2023-08-09T13:16:40.430 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/fs/nls/nls_cp863.ko 2023-08-09T13:16:40.430 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/fs/nls/nls_cp864.ko 2023-08-09T13:16:40.430 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/fs/nls/nls_cp865.ko 2023-08-09T13:16:40.430 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/fs/nls/nls_cp866.ko 2023-08-09T13:16:40.430 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/fs/nls/nls_cp869.ko 2023-08-09T13:16:40.430 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/fs/nls/nls_cp874.ko 2023-08-09T13:16:40.431 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/fs/nls/nls_cp932.ko 2023-08-09T13:16:40.431 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/fs/nls/nls_cp936.ko 2023-08-09T13:16:40.431 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/fs/nls/nls_cp949.ko 2023-08-09T13:16:40.431 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/fs/nls/nls_cp950.ko 2023-08-09T13:16:40.431 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/fs/nls/nls_euc-jp.ko 2023-08-09T13:16:40.431 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/fs/nls/nls_iso8859-1.ko 2023-08-09T13:16:40.431 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/fs/nls/nls_iso8859-13.ko 2023-08-09T13:16:40.431 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/fs/nls/nls_iso8859-14.ko 2023-08-09T13:16:40.432 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/fs/nls/nls_iso8859-15.ko 2023-08-09T13:16:40.432 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/fs/nls/nls_iso8859-2.ko 2023-08-09T13:16:40.432 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/fs/nls/nls_iso8859-3.ko 2023-08-09T13:16:40.432 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/fs/nls/nls_iso8859-4.ko 2023-08-09T13:16:40.432 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/fs/nls/nls_iso8859-5.ko 2023-08-09T13:16:40.432 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/fs/nls/nls_iso8859-6.ko 2023-08-09T13:16:40.432 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/fs/nls/nls_iso8859-7.ko 2023-08-09T13:16:40.433 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/fs/nls/nls_iso8859-9.ko 2023-08-09T13:16:40.433 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/fs/nls/nls_koi8-r.ko 2023-08-09T13:16:40.433 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/fs/nls/nls_koi8-ru.ko 2023-08-09T13:16:40.433 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/fs/nls/nls_koi8-u.ko 2023-08-09T13:16:40.433 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/fs/nls/nls_utf8.ko 2023-08-09T13:16:40.433 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/fs/ocfs2 2023-08-09T13:16:40.433 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/fs/ocfs2/cluster 2023-08-09T13:16:40.433 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/fs/ocfs2/cluster/ocfs2_nodemanager.ko 2023-08-09T13:16:40.434 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/fs/ocfs2/dlm 2023-08-09T13:16:40.434 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/fs/ocfs2/dlm/ocfs2_dlm.ko 2023-08-09T13:16:40.434 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/fs/ocfs2/dlmfs 2023-08-09T13:16:40.434 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/fs/ocfs2/dlmfs/ocfs2_dlmfs.ko 2023-08-09T13:16:40.434 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/fs/ocfs2/ocfs2.ko 2023-08-09T13:16:40.434 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/fs/ocfs2/ocfs2_stack_o2cb.ko 2023-08-09T13:16:40.434 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/fs/ocfs2/ocfs2_stack_user.ko 2023-08-09T13:16:40.434 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/fs/ocfs2/ocfs2_stackglue.ko 2023-08-09T13:16:40.435 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/fs/overlayfs 2023-08-09T13:16:40.435 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/fs/overlayfs/overlay.ko 2023-08-09T13:16:40.435 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/fs/pstore 2023-08-09T13:16:40.435 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/fs/pstore/ramoops.ko 2023-08-09T13:16:40.435 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/fs/reiserfs 2023-08-09T13:16:40.435 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/fs/reiserfs/reiserfs.ko 2023-08-09T13:16:40.435 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/fs/romfs 2023-08-09T13:16:40.435 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/fs/romfs/romfs.ko 2023-08-09T13:16:40.436 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/fs/smb 2023-08-09T13:16:40.436 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/fs/smb/client 2023-08-09T13:16:40.436 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/fs/smb/client/cifs.ko 2023-08-09T13:16:40.436 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/fs/smb/common 2023-08-09T13:16:40.436 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/fs/smb/common/cifs_arc4.ko 2023-08-09T13:16:40.436 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/fs/smb/common/cifs_md4.ko 2023-08-09T13:16:40.436 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/fs/squashfs 2023-08-09T13:16:40.436 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/fs/squashfs/squashfs.ko 2023-08-09T13:16:40.437 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/fs/sysv 2023-08-09T13:16:40.437 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/fs/sysv/sysv.ko 2023-08-09T13:16:40.437 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/fs/udf 2023-08-09T13:16:40.437 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/fs/udf/udf.ko 2023-08-09T13:16:40.437 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/fs/ufs 2023-08-09T13:16:40.437 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/fs/ufs/ufs.ko 2023-08-09T13:16:40.437 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/fs/xfs 2023-08-09T13:16:40.437 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/fs/xfs/xfs.ko 2023-08-09T13:16:40.438 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/kernel 2023-08-09T13:16:40.438 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/kernel/rcu 2023-08-09T13:16:40.438 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/kernel/rcu/rcutorture.ko 2023-08-09T13:16:40.438 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/kernel/torture.ko 2023-08-09T13:16:40.438 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/kernel/trace 2023-08-09T13:16:40.438 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/kernel/trace/ring_buffer_benchmark.ko 2023-08-09T13:16:40.438 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/lib 2023-08-09T13:16:40.438 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/lib/asn1_encoder.ko 2023-08-09T13:16:40.439 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/lib/crc-itu-t.ko 2023-08-09T13:16:40.439 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/lib/crc8.ko 2023-08-09T13:16:40.439 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/lib/crypto 2023-08-09T13:16:40.439 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/lib/crypto/libarc4.ko 2023-08-09T13:16:40.439 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/lib/crypto/libdes.ko 2023-08-09T13:16:40.439 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/lib/libcrc32c.ko 2023-08-09T13:16:40.439 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/lib/lru_cache.ko 2023-08-09T13:16:40.440 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/lib/lz4 2023-08-09T13:16:40.440 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/lib/lz4/lz4_compress.ko 2023-08-09T13:16:40.440 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/lib/lz4/lz4hc_compress.ko 2023-08-09T13:16:40.440 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/lib/math 2023-08-09T13:16:40.440 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/lib/math/cordic.ko 2023-08-09T13:16:40.440 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/lib/raid6 2023-08-09T13:16:40.440 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/lib/raid6/raid6_pq.ko 2023-08-09T13:16:40.440 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/lib/reed_solomon 2023-08-09T13:16:40.441 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/lib/reed_solomon/reed_solomon.ko 2023-08-09T13:16:40.441 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/lib/ts_bm.ko 2023-08-09T13:16:40.441 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/lib/ts_fsm.ko 2023-08-09T13:16:40.441 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/lib/ts_kmp.ko 2023-08-09T13:16:40.441 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/lib/zstd 2023-08-09T13:16:40.441 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/lib/zstd/zstd_compress.ko 2023-08-09T13:16:40.441 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/mm 2023-08-09T13:16:40.441 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/mm/hwpoison-inject.ko 2023-08-09T13:16:40.442 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/net 2023-08-09T13:16:40.442 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/net/6lowpan 2023-08-09T13:16:40.442 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/net/6lowpan/6lowpan.ko 2023-08-09T13:16:40.442 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/net/6lowpan/nhc_dest.ko 2023-08-09T13:16:40.442 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/net/6lowpan/nhc_fragment.ko 2023-08-09T13:16:40.442 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/net/6lowpan/nhc_hop.ko 2023-08-09T13:16:40.442 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/net/6lowpan/nhc_ipv6.ko 2023-08-09T13:16:40.442 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/net/6lowpan/nhc_mobility.ko 2023-08-09T13:16:40.443 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/net/6lowpan/nhc_routing.ko 2023-08-09T13:16:40.443 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/net/6lowpan/nhc_udp.ko 2023-08-09T13:16:40.443 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/net/802 2023-08-09T13:16:40.443 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/net/802/garp.ko 2023-08-09T13:16:40.443 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/net/802/mrp.ko 2023-08-09T13:16:40.443 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/net/802/p8022.ko 2023-08-09T13:16:40.443 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/net/802/psnap.ko 2023-08-09T13:16:40.443 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/net/802/stp.ko 2023-08-09T13:16:40.444 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/net/8021q 2023-08-09T13:16:40.444 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/net/8021q/8021q.ko 2023-08-09T13:16:40.444 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/net/9p 2023-08-09T13:16:40.444 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/net/9p/9pnet.ko 2023-08-09T13:16:40.444 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/net/9p/9pnet_fd.ko 2023-08-09T13:16:40.444 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/net/9p/9pnet_rdma.ko 2023-08-09T13:16:40.444 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/net/9p/9pnet_virtio.ko 2023-08-09T13:16:40.445 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/net/batman-adv 2023-08-09T13:16:40.445 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/net/batman-adv/batman-adv.ko 2023-08-09T13:16:40.445 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/net/bridge 2023-08-09T13:16:40.445 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/net/bridge/br_netfilter.ko 2023-08-09T13:16:40.445 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/net/bridge/bridge.ko 2023-08-09T13:16:40.445 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/net/bridge/netfilter 2023-08-09T13:16:40.445 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/net/bridge/netfilter/ebt_802_3.ko 2023-08-09T13:16:40.445 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/net/bridge/netfilter/ebt_among.ko 2023-08-09T13:16:40.446 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/net/bridge/netfilter/ebt_arp.ko 2023-08-09T13:16:40.446 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/net/bridge/netfilter/ebt_arpreply.ko 2023-08-09T13:16:40.446 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/net/bridge/netfilter/ebt_dnat.ko 2023-08-09T13:16:40.446 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/net/bridge/netfilter/ebt_ip.ko 2023-08-09T13:16:40.446 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/net/bridge/netfilter/ebt_ip6.ko 2023-08-09T13:16:40.446 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/net/bridge/netfilter/ebt_limit.ko 2023-08-09T13:16:40.446 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/net/bridge/netfilter/ebt_log.ko 2023-08-09T13:16:40.446 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/net/bridge/netfilter/ebt_mark.ko 2023-08-09T13:16:40.447 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/net/bridge/netfilter/ebt_mark_m.ko 2023-08-09T13:16:40.447 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/net/bridge/netfilter/ebt_nflog.ko 2023-08-09T13:16:40.447 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/net/bridge/netfilter/ebt_pkttype.ko 2023-08-09T13:16:40.447 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/net/bridge/netfilter/ebt_redirect.ko 2023-08-09T13:16:40.447 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/net/bridge/netfilter/ebt_snat.ko 2023-08-09T13:16:40.447 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/net/bridge/netfilter/ebt_stp.ko 2023-08-09T13:16:40.447 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/net/bridge/netfilter/ebt_vlan.ko 2023-08-09T13:16:40.447 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/net/bridge/netfilter/ebtable_broute.ko 2023-08-09T13:16:40.448 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/net/bridge/netfilter/ebtable_filter.ko 2023-08-09T13:16:40.448 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/net/bridge/netfilter/ebtable_nat.ko 2023-08-09T13:16:40.448 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/net/bridge/netfilter/ebtables.ko 2023-08-09T13:16:40.448 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/net/bridge/netfilter/nf_conntrack_bridge.ko 2023-08-09T13:16:40.448 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/net/bridge/netfilter/nft_meta_bridge.ko 2023-08-09T13:16:40.448 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/net/bridge/netfilter/nft_reject_bridge.ko 2023-08-09T13:16:40.448 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/net/ceph 2023-08-09T13:16:40.448 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/net/ceph/libceph.ko 2023-08-09T13:16:40.449 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/net/core 2023-08-09T13:16:40.449 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/net/core/failover.ko 2023-08-09T13:16:40.449 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/net/core/pktgen.ko 2023-08-09T13:16:40.449 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/net/dccp 2023-08-09T13:16:40.449 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/net/dccp/dccp.ko 2023-08-09T13:16:40.449 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/net/dccp/dccp_diag.ko 2023-08-09T13:16:40.449 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/net/dccp/dccp_ipv4.ko 2023-08-09T13:16:40.449 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/net/dccp/dccp_ipv6.ko 2023-08-09T13:16:40.450 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/net/dns_resolver 2023-08-09T13:16:40.450 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/net/dns_resolver/dns_resolver.ko 2023-08-09T13:16:40.450 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/net/dsa 2023-08-09T13:16:40.450 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/net/dsa/dsa_core.ko 2023-08-09T13:16:40.450 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/net/dsa/tag_brcm.ko 2023-08-09T13:16:40.450 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/net/dsa/tag_dsa.ko 2023-08-09T13:16:40.450 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/net/dsa/tag_none.ko 2023-08-09T13:16:40.450 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/net/dsa/tag_trailer.ko 2023-08-09T13:16:40.451 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/net/ieee802154 2023-08-09T13:16:40.451 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/net/ieee802154/6lowpan 2023-08-09T13:16:40.451 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/net/ieee802154/6lowpan/ieee802154_6lowpan.ko 2023-08-09T13:16:40.451 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/net/ieee802154/ieee802154.ko 2023-08-09T13:16:40.451 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/net/ieee802154/ieee802154_socket.ko 2023-08-09T13:16:40.451 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/net/ipv4 2023-08-09T13:16:40.451 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/net/ipv4/ah4.ko 2023-08-09T13:16:40.451 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/net/ipv4/esp4.ko 2023-08-09T13:16:40.452 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/net/ipv4/fou.ko 2023-08-09T13:16:40.452 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/net/ipv4/gre.ko 2023-08-09T13:16:40.452 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/net/ipv4/inet_diag.ko 2023-08-09T13:16:40.452 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/net/ipv4/ip_gre.ko 2023-08-09T13:16:40.452 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/net/ipv4/ip_tunnel.ko 2023-08-09T13:16:40.452 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/net/ipv4/ip_vti.ko 2023-08-09T13:16:40.452 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/net/ipv4/ipcomp.ko 2023-08-09T13:16:40.453 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/net/ipv4/ipip.ko 2023-08-09T13:16:40.453 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/net/ipv4/netfilter 2023-08-09T13:16:40.453 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/net/ipv4/netfilter/arp_tables.ko 2023-08-09T13:16:40.453 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/net/ipv4/netfilter/arpt_mangle.ko 2023-08-09T13:16:40.453 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/net/ipv4/netfilter/arptable_filter.ko 2023-08-09T13:16:40.453 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/net/ipv4/netfilter/ipt_ECN.ko 2023-08-09T13:16:40.453 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/net/ipv4/netfilter/ipt_SYNPROXY.ko 2023-08-09T13:16:40.453 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/net/ipv4/netfilter/ipt_ah.ko 2023-08-09T13:16:40.454 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/net/ipv4/netfilter/ipt_rpfilter.ko 2023-08-09T13:16:40.454 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/net/ipv4/netfilter/iptable_mangle.ko 2023-08-09T13:16:40.454 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/net/ipv4/netfilter/iptable_nat.ko 2023-08-09T13:16:40.454 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/net/ipv4/netfilter/iptable_raw.ko 2023-08-09T13:16:40.454 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/net/ipv4/netfilter/iptable_security.ko 2023-08-09T13:16:40.454 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/net/ipv4/netfilter/nf_defrag_ipv4.ko 2023-08-09T13:16:40.454 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/net/ipv4/netfilter/nf_dup_ipv4.ko 2023-08-09T13:16:40.454 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/net/ipv4/netfilter/nf_nat_h323.ko 2023-08-09T13:16:40.455 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/net/ipv4/netfilter/nf_nat_pptp.ko 2023-08-09T13:16:40.455 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/net/ipv4/netfilter/nf_nat_snmp_basic.ko 2023-08-09T13:16:40.455 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/net/ipv4/netfilter/nf_socket_ipv4.ko 2023-08-09T13:16:40.455 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/net/ipv4/netfilter/nf_tproxy_ipv4.ko 2023-08-09T13:16:40.455 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/net/ipv4/netfilter/nft_dup_ipv4.ko 2023-08-09T13:16:40.455 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/net/ipv4/netfilter/nft_fib_ipv4.ko 2023-08-09T13:16:40.455 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/net/ipv4/netfilter/nft_reject_ipv4.ko 2023-08-09T13:16:40.455 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/net/ipv4/tcp_bic.ko 2023-08-09T13:16:40.456 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/net/ipv4/tcp_dctcp.ko 2023-08-09T13:16:40.456 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/net/ipv4/tcp_diag.ko 2023-08-09T13:16:40.456 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/net/ipv4/tcp_highspeed.ko 2023-08-09T13:16:40.456 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/net/ipv4/tcp_htcp.ko 2023-08-09T13:16:40.456 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/net/ipv4/tcp_hybla.ko 2023-08-09T13:16:40.456 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/net/ipv4/tcp_illinois.ko 2023-08-09T13:16:40.456 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/net/ipv4/tcp_lp.ko 2023-08-09T13:16:40.456 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/net/ipv4/tcp_scalable.ko 2023-08-09T13:16:40.457 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/net/ipv4/tcp_vegas.ko 2023-08-09T13:16:40.457 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/net/ipv4/tcp_veno.ko 2023-08-09T13:16:40.457 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/net/ipv4/tcp_westwood.ko 2023-08-09T13:16:40.457 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/net/ipv4/tcp_yeah.ko 2023-08-09T13:16:40.457 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/net/ipv4/tunnel4.ko 2023-08-09T13:16:40.457 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/net/ipv4/udp_diag.ko 2023-08-09T13:16:40.457 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/net/ipv4/udp_tunnel.ko 2023-08-09T13:16:40.458 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/net/ipv4/xfrm4_tunnel.ko 2023-08-09T13:16:40.458 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/net/ipv6 2023-08-09T13:16:40.458 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/net/ipv6/ah6.ko 2023-08-09T13:16:40.458 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/net/ipv6/esp6.ko 2023-08-09T13:16:40.458 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/net/ipv6/fou6.ko 2023-08-09T13:16:40.458 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/net/ipv6/ip6_tunnel.ko 2023-08-09T13:16:40.458 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/net/ipv6/ip6_udp_tunnel.ko 2023-08-09T13:16:40.458 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/net/ipv6/ip6_vti.ko 2023-08-09T13:16:40.459 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/net/ipv6/ipcomp6.ko 2023-08-09T13:16:40.459 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/net/ipv6/netfilter 2023-08-09T13:16:40.459 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/net/ipv6/netfilter/ip6_tables.ko 2023-08-09T13:16:40.459 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/net/ipv6/netfilter/ip6t_REJECT.ko 2023-08-09T13:16:40.459 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/net/ipv6/netfilter/ip6t_SYNPROXY.ko 2023-08-09T13:16:40.459 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/net/ipv6/netfilter/ip6t_ah.ko 2023-08-09T13:16:40.459 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/net/ipv6/netfilter/ip6t_eui64.ko 2023-08-09T13:16:40.459 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/net/ipv6/netfilter/ip6t_frag.ko 2023-08-09T13:16:40.460 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/net/ipv6/netfilter/ip6t_hbh.ko 2023-08-09T13:16:40.460 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/net/ipv6/netfilter/ip6t_ipv6header.ko 2023-08-09T13:16:40.460 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/net/ipv6/netfilter/ip6t_mh.ko 2023-08-09T13:16:40.460 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/net/ipv6/netfilter/ip6t_rpfilter.ko 2023-08-09T13:16:40.460 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/net/ipv6/netfilter/ip6t_rt.ko 2023-08-09T13:16:40.460 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/net/ipv6/netfilter/ip6table_filter.ko 2023-08-09T13:16:40.460 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/net/ipv6/netfilter/ip6table_mangle.ko 2023-08-09T13:16:40.460 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/net/ipv6/netfilter/ip6table_nat.ko 2023-08-09T13:16:40.461 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/net/ipv6/netfilter/ip6table_raw.ko 2023-08-09T13:16:40.461 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/net/ipv6/netfilter/ip6table_security.ko 2023-08-09T13:16:40.461 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/net/ipv6/netfilter/nf_defrag_ipv6.ko 2023-08-09T13:16:40.461 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/net/ipv6/netfilter/nf_dup_ipv6.ko 2023-08-09T13:16:40.461 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/net/ipv6/netfilter/nf_reject_ipv6.ko 2023-08-09T13:16:40.461 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/net/ipv6/netfilter/nf_socket_ipv6.ko 2023-08-09T13:16:40.461 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/net/ipv6/netfilter/nf_tproxy_ipv6.ko 2023-08-09T13:16:40.461 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/net/ipv6/netfilter/nft_dup_ipv6.ko 2023-08-09T13:16:40.462 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/net/ipv6/netfilter/nft_fib_ipv6.ko 2023-08-09T13:16:40.462 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/net/ipv6/netfilter/nft_reject_ipv6.ko 2023-08-09T13:16:40.462 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/net/ipv6/sit.ko 2023-08-09T13:16:40.462 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/net/ipv6/tunnel6.ko 2023-08-09T13:16:40.462 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/net/ipv6/xfrm6_tunnel.ko 2023-08-09T13:16:40.462 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/net/key 2023-08-09T13:16:40.462 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/net/key/af_key.ko 2023-08-09T13:16:40.462 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/net/l2tp 2023-08-09T13:16:40.463 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/net/l2tp/l2tp_core.ko 2023-08-09T13:16:40.463 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/net/l2tp/l2tp_debugfs.ko 2023-08-09T13:16:40.463 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/net/l2tp/l2tp_eth.ko 2023-08-09T13:16:40.463 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/net/l2tp/l2tp_ip.ko 2023-08-09T13:16:40.463 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/net/l2tp/l2tp_ip6.ko 2023-08-09T13:16:40.463 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/net/l2tp/l2tp_netlink.ko 2023-08-09T13:16:40.463 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/net/l2tp/l2tp_ppp.ko 2023-08-09T13:16:40.463 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/net/llc 2023-08-09T13:16:40.464 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/net/llc/llc.ko 2023-08-09T13:16:40.464 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/net/mac802154 2023-08-09T13:16:40.464 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/net/mac802154/mac802154.ko 2023-08-09T13:16:40.464 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/net/mpls 2023-08-09T13:16:40.464 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/net/mpls/mpls_gso.ko 2023-08-09T13:16:40.464 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/net/netfilter 2023-08-09T13:16:40.464 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/net/netfilter/ipset 2023-08-09T13:16:40.465 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/net/netfilter/ipset/ip_set.ko 2023-08-09T13:16:40.465 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/net/netfilter/ipset/ip_set_bitmap_ip.ko 2023-08-09T13:16:40.465 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/net/netfilter/ipset/ip_set_bitmap_ipmac.ko 2023-08-09T13:16:40.465 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/net/netfilter/ipset/ip_set_bitmap_port.ko 2023-08-09T13:16:40.465 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/net/netfilter/ipset/ip_set_hash_ip.ko 2023-08-09T13:16:40.465 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/net/netfilter/ipset/ip_set_hash_ipmark.ko 2023-08-09T13:16:40.465 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/net/netfilter/ipset/ip_set_hash_ipport.ko 2023-08-09T13:16:40.465 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/net/netfilter/ipset/ip_set_hash_ipportip.ko 2023-08-09T13:16:40.466 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/net/netfilter/ipset/ip_set_hash_ipportnet.ko 2023-08-09T13:16:40.466 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/net/netfilter/ipset/ip_set_hash_mac.ko 2023-08-09T13:16:40.466 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/net/netfilter/ipset/ip_set_hash_net.ko 2023-08-09T13:16:40.466 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/net/netfilter/ipset/ip_set_hash_netiface.ko 2023-08-09T13:16:40.466 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/net/netfilter/ipset/ip_set_hash_netnet.ko 2023-08-09T13:16:40.466 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/net/netfilter/ipset/ip_set_hash_netport.ko 2023-08-09T13:16:40.466 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/net/netfilter/ipset/ip_set_hash_netportnet.ko 2023-08-09T13:16:40.466 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/net/netfilter/ipset/ip_set_list_set.ko 2023-08-09T13:16:40.467 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/net/netfilter/ipvs 2023-08-09T13:16:40.467 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/net/netfilter/ipvs/ip_vs.ko 2023-08-09T13:16:40.467 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/net/netfilter/ipvs/ip_vs_dh.ko 2023-08-09T13:16:40.467 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/net/netfilter/ipvs/ip_vs_fo.ko 2023-08-09T13:16:40.467 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/net/netfilter/ipvs/ip_vs_ftp.ko 2023-08-09T13:16:40.467 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/net/netfilter/ipvs/ip_vs_lblc.ko 2023-08-09T13:16:40.467 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/net/netfilter/ipvs/ip_vs_lblcr.ko 2023-08-09T13:16:40.467 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/net/netfilter/ipvs/ip_vs_lc.ko 2023-08-09T13:16:40.468 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/net/netfilter/ipvs/ip_vs_nq.ko 2023-08-09T13:16:40.468 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/net/netfilter/ipvs/ip_vs_pe_sip.ko 2023-08-09T13:16:40.468 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/net/netfilter/ipvs/ip_vs_rr.ko 2023-08-09T13:16:40.468 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/net/netfilter/ipvs/ip_vs_sed.ko 2023-08-09T13:16:40.468 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/net/netfilter/ipvs/ip_vs_sh.ko 2023-08-09T13:16:40.468 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/net/netfilter/ipvs/ip_vs_wlc.ko 2023-08-09T13:16:40.468 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/net/netfilter/ipvs/ip_vs_wrr.ko 2023-08-09T13:16:40.469 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/net/netfilter/nf_conncount.ko 2023-08-09T13:16:40.469 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/net/netfilter/nf_conntrack.ko 2023-08-09T13:16:40.469 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/net/netfilter/nf_conntrack_amanda.ko 2023-08-09T13:16:40.469 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/net/netfilter/nf_conntrack_broadcast.ko 2023-08-09T13:16:40.469 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/net/netfilter/nf_conntrack_ftp.ko 2023-08-09T13:16:40.469 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/net/netfilter/nf_conntrack_h323.ko 2023-08-09T13:16:40.469 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/net/netfilter/nf_conntrack_irc.ko 2023-08-09T13:16:40.469 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/net/netfilter/nf_conntrack_netbios_ns.ko 2023-08-09T13:16:40.470 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/net/netfilter/nf_conntrack_netlink.ko 2023-08-09T13:16:40.470 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/net/netfilter/nf_conntrack_pptp.ko 2023-08-09T13:16:40.470 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/net/netfilter/nf_conntrack_sane.ko 2023-08-09T13:16:40.470 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/net/netfilter/nf_conntrack_sip.ko 2023-08-09T13:16:40.470 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/net/netfilter/nf_conntrack_snmp.ko 2023-08-09T13:16:40.470 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/net/netfilter/nf_conntrack_tftp.ko 2023-08-09T13:16:40.470 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/net/netfilter/nf_dup_netdev.ko 2023-08-09T13:16:40.470 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/net/netfilter/nf_flow_table.ko 2023-08-09T13:16:40.471 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/net/netfilter/nf_flow_table_inet.ko 2023-08-09T13:16:40.471 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/net/netfilter/nf_log_syslog.ko 2023-08-09T13:16:40.471 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/net/netfilter/nf_nat.ko 2023-08-09T13:16:40.471 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/net/netfilter/nf_nat_amanda.ko 2023-08-09T13:16:40.471 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/net/netfilter/nf_nat_ftp.ko 2023-08-09T13:16:40.471 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/net/netfilter/nf_nat_irc.ko 2023-08-09T13:16:40.471 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/net/netfilter/nf_nat_sip.ko 2023-08-09T13:16:40.472 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/net/netfilter/nf_nat_tftp.ko 2023-08-09T13:16:40.472 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/net/netfilter/nf_synproxy_core.ko 2023-08-09T13:16:40.472 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/net/netfilter/nf_tables.ko 2023-08-09T13:16:40.472 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/net/netfilter/nfnetlink.ko 2023-08-09T13:16:40.472 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/net/netfilter/nfnetlink_acct.ko 2023-08-09T13:16:40.472 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/net/netfilter/nfnetlink_log.ko 2023-08-09T13:16:40.472 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/net/netfilter/nfnetlink_osf.ko 2023-08-09T13:16:40.472 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/net/netfilter/nfnetlink_queue.ko 2023-08-09T13:16:40.473 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/net/netfilter/nft_chain_nat.ko 2023-08-09T13:16:40.473 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/net/netfilter/nft_compat.ko 2023-08-09T13:16:40.473 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/net/netfilter/nft_ct.ko 2023-08-09T13:16:40.473 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/net/netfilter/nft_dup_netdev.ko 2023-08-09T13:16:40.473 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/net/netfilter/nft_fib.ko 2023-08-09T13:16:40.473 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/net/netfilter/nft_fwd_netdev.ko 2023-08-09T13:16:40.473 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/net/netfilter/nft_hash.ko 2023-08-09T13:16:40.474 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/net/netfilter/nft_limit.ko 2023-08-09T13:16:40.474 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/net/netfilter/nft_log.ko 2023-08-09T13:16:40.474 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/net/netfilter/nft_masq.ko 2023-08-09T13:16:40.474 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/net/netfilter/nft_nat.ko 2023-08-09T13:16:40.474 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/net/netfilter/nft_numgen.ko 2023-08-09T13:16:40.474 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/net/netfilter/nft_queue.ko 2023-08-09T13:16:40.474 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/net/netfilter/nft_quota.ko 2023-08-09T13:16:40.474 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/net/netfilter/nft_redir.ko 2023-08-09T13:16:40.475 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/net/netfilter/nft_reject.ko 2023-08-09T13:16:40.475 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/net/netfilter/nft_reject_inet.ko 2023-08-09T13:16:40.475 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/net/netfilter/nft_socket.ko 2023-08-09T13:16:40.475 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/net/netfilter/nft_tunnel.ko 2023-08-09T13:16:40.475 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/net/netfilter/xt_AUDIT.ko 2023-08-09T13:16:40.475 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/net/netfilter/xt_CHECKSUM.ko 2023-08-09T13:16:40.475 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/net/netfilter/xt_CLASSIFY.ko 2023-08-09T13:16:40.475 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/net/netfilter/xt_CONNSECMARK.ko 2023-08-09T13:16:40.476 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/net/netfilter/xt_CT.ko 2023-08-09T13:16:40.476 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/net/netfilter/xt_DSCP.ko 2023-08-09T13:16:40.476 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/net/netfilter/xt_HL.ko 2023-08-09T13:16:40.476 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/net/netfilter/xt_HMARK.ko 2023-08-09T13:16:40.476 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/net/netfilter/xt_IDLETIMER.ko 2023-08-09T13:16:40.476 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/net/netfilter/xt_LED.ko 2023-08-09T13:16:40.476 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/net/netfilter/xt_LOG.ko 2023-08-09T13:16:40.476 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/net/netfilter/xt_MASQUERADE.ko 2023-08-09T13:16:40.477 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/net/netfilter/xt_NETMAP.ko 2023-08-09T13:16:40.477 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/net/netfilter/xt_NFLOG.ko 2023-08-09T13:16:40.477 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/net/netfilter/xt_NFQUEUE.ko 2023-08-09T13:16:40.477 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/net/netfilter/xt_RATEEST.ko 2023-08-09T13:16:40.477 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/net/netfilter/xt_REDIRECT.ko 2023-08-09T13:16:40.477 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/net/netfilter/xt_SECMARK.ko 2023-08-09T13:16:40.477 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/net/netfilter/xt_TCPMSS.ko 2023-08-09T13:16:40.477 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/net/netfilter/xt_TCPOPTSTRIP.ko 2023-08-09T13:16:40.478 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/net/netfilter/xt_TEE.ko 2023-08-09T13:16:40.478 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/net/netfilter/xt_TPROXY.ko 2023-08-09T13:16:40.478 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/net/netfilter/xt_TRACE.ko 2023-08-09T13:16:40.478 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/net/netfilter/xt_addrtype.ko 2023-08-09T13:16:40.478 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/net/netfilter/xt_bpf.ko 2023-08-09T13:16:40.478 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/net/netfilter/xt_cgroup.ko 2023-08-09T13:16:40.478 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/net/netfilter/xt_cluster.ko 2023-08-09T13:16:40.478 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/net/netfilter/xt_comment.ko 2023-08-09T13:16:40.479 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/net/netfilter/xt_connbytes.ko 2023-08-09T13:16:40.479 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/net/netfilter/xt_connlabel.ko 2023-08-09T13:16:40.479 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/net/netfilter/xt_connlimit.ko 2023-08-09T13:16:40.479 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/net/netfilter/xt_connmark.ko 2023-08-09T13:16:40.479 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/net/netfilter/xt_conntrack.ko 2023-08-09T13:16:40.479 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/net/netfilter/xt_cpu.ko 2023-08-09T13:16:40.479 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/net/netfilter/xt_dccp.ko 2023-08-09T13:16:40.479 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/net/netfilter/xt_devgroup.ko 2023-08-09T13:16:40.480 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/net/netfilter/xt_dscp.ko 2023-08-09T13:16:40.480 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/net/netfilter/xt_ecn.ko 2023-08-09T13:16:40.480 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/net/netfilter/xt_esp.ko 2023-08-09T13:16:40.480 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/net/netfilter/xt_hashlimit.ko 2023-08-09T13:16:40.480 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/net/netfilter/xt_helper.ko 2023-08-09T13:16:40.480 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/net/netfilter/xt_hl.ko 2023-08-09T13:16:40.480 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/net/netfilter/xt_ipcomp.ko 2023-08-09T13:16:40.481 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/net/netfilter/xt_iprange.ko 2023-08-09T13:16:40.481 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/net/netfilter/xt_ipvs.ko 2023-08-09T13:16:40.481 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/net/netfilter/xt_l2tp.ko 2023-08-09T13:16:40.481 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/net/netfilter/xt_length.ko 2023-08-09T13:16:40.481 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/net/netfilter/xt_limit.ko 2023-08-09T13:16:40.481 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/net/netfilter/xt_mac.ko 2023-08-09T13:16:40.481 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/net/netfilter/xt_mark.ko 2023-08-09T13:16:40.481 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/net/netfilter/xt_multiport.ko 2023-08-09T13:16:40.482 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/net/netfilter/xt_nat.ko 2023-08-09T13:16:40.482 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/net/netfilter/xt_nfacct.ko 2023-08-09T13:16:40.482 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/net/netfilter/xt_osf.ko 2023-08-09T13:16:40.482 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/net/netfilter/xt_owner.ko 2023-08-09T13:16:40.482 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/net/netfilter/xt_physdev.ko 2023-08-09T13:16:40.482 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/net/netfilter/xt_pkttype.ko 2023-08-09T13:16:40.482 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/net/netfilter/xt_policy.ko 2023-08-09T13:16:40.482 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/net/netfilter/xt_quota.ko 2023-08-09T13:16:40.483 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/net/netfilter/xt_rateest.ko 2023-08-09T13:16:40.483 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/net/netfilter/xt_realm.ko 2023-08-09T13:16:40.483 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/net/netfilter/xt_recent.ko 2023-08-09T13:16:40.483 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/net/netfilter/xt_sctp.ko 2023-08-09T13:16:40.483 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/net/netfilter/xt_set.ko 2023-08-09T13:16:40.483 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/net/netfilter/xt_socket.ko 2023-08-09T13:16:40.483 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/net/netfilter/xt_state.ko 2023-08-09T13:16:40.483 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/net/netfilter/xt_statistic.ko 2023-08-09T13:16:40.484 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/net/netfilter/xt_string.ko 2023-08-09T13:16:40.484 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/net/netfilter/xt_tcpmss.ko 2023-08-09T13:16:40.484 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/net/netfilter/xt_time.ko 2023-08-09T13:16:40.484 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/net/netfilter/xt_u32.ko 2023-08-09T13:16:40.484 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/net/netlink 2023-08-09T13:16:40.484 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/net/netlink/netlink_diag.ko 2023-08-09T13:16:40.484 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/net/nsh 2023-08-09T13:16:40.484 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/net/nsh/nsh.ko 2023-08-09T13:16:40.485 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/net/openvswitch 2023-08-09T13:16:40.485 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/net/openvswitch/openvswitch.ko 2023-08-09T13:16:40.485 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/net/openvswitch/vport-geneve.ko 2023-08-09T13:16:40.485 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/net/openvswitch/vport-gre.ko 2023-08-09T13:16:40.485 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/net/openvswitch/vport-vxlan.ko 2023-08-09T13:16:40.485 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/net/packet 2023-08-09T13:16:40.486 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/net/packet/af_packet_diag.ko 2023-08-09T13:16:40.486 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/net/rds 2023-08-09T13:16:40.486 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/net/rds/rds.ko 2023-08-09T13:16:40.486 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/net/rds/rds_rdma.ko 2023-08-09T13:16:40.486 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/net/rds/rds_tcp.ko 2023-08-09T13:16:40.486 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/net/sched 2023-08-09T13:16:40.486 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/net/sched/act_csum.ko 2023-08-09T13:16:40.487 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/net/sched/act_gact.ko 2023-08-09T13:16:40.487 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/net/sched/act_ipt.ko 2023-08-09T13:16:40.487 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/net/sched/act_mirred.ko 2023-08-09T13:16:40.487 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/net/sched/act_nat.ko 2023-08-09T13:16:40.487 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/net/sched/act_pedit.ko 2023-08-09T13:16:40.487 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/net/sched/act_police.ko 2023-08-09T13:16:40.487 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/net/sched/act_simple.ko 2023-08-09T13:16:40.487 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/net/sched/act_skbedit.ko 2023-08-09T13:16:40.488 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/net/sched/act_vlan.ko 2023-08-09T13:16:40.488 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/net/sched/cls_basic.ko 2023-08-09T13:16:40.488 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/net/sched/cls_bpf.ko 2023-08-09T13:16:40.488 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/net/sched/cls_flow.ko 2023-08-09T13:16:40.488 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/net/sched/cls_fw.ko 2023-08-09T13:16:40.488 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/net/sched/cls_route.ko 2023-08-09T13:16:40.488 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/net/sched/cls_u32.ko 2023-08-09T13:16:40.488 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/net/sched/em_cmp.ko 2023-08-09T13:16:40.489 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/net/sched/em_ipset.ko 2023-08-09T13:16:40.489 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/net/sched/em_meta.ko 2023-08-09T13:16:40.489 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/net/sched/em_nbyte.ko 2023-08-09T13:16:40.489 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/net/sched/em_text.ko 2023-08-09T13:16:40.489 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/net/sched/em_u32.ko 2023-08-09T13:16:40.489 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/net/sched/sch_choke.ko 2023-08-09T13:16:40.489 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/net/sched/sch_codel.ko 2023-08-09T13:16:40.489 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/net/sched/sch_drr.ko 2023-08-09T13:16:40.490 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/net/sched/sch_fq.ko 2023-08-09T13:16:40.490 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/net/sched/sch_gred.ko 2023-08-09T13:16:40.490 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/net/sched/sch_hfsc.ko 2023-08-09T13:16:40.490 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/net/sched/sch_hhf.ko 2023-08-09T13:16:40.490 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/net/sched/sch_htb.ko 2023-08-09T13:16:40.490 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/net/sched/sch_ingress.ko 2023-08-09T13:16:40.490 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/net/sched/sch_mqprio.ko 2023-08-09T13:16:40.490 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/net/sched/sch_mqprio_lib.ko 2023-08-09T13:16:40.491 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/net/sched/sch_multiq.ko 2023-08-09T13:16:40.491 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/net/sched/sch_netem.ko 2023-08-09T13:16:40.491 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/net/sched/sch_pie.ko 2023-08-09T13:16:40.491 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/net/sched/sch_plug.ko 2023-08-09T13:16:40.491 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/net/sched/sch_prio.ko 2023-08-09T13:16:40.491 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/net/sched/sch_qfq.ko 2023-08-09T13:16:40.491 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/net/sched/sch_red.ko 2023-08-09T13:16:40.491 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/net/sched/sch_sfb.ko 2023-08-09T13:16:40.492 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/net/sched/sch_sfq.ko 2023-08-09T13:16:40.492 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/net/sched/sch_tbf.ko 2023-08-09T13:16:40.492 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/net/sched/sch_teql.ko 2023-08-09T13:16:40.492 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/net/sctp 2023-08-09T13:16:40.492 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/net/sctp/sctp.ko 2023-08-09T13:16:40.492 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/net/sctp/sctp_diag.ko 2023-08-09T13:16:40.492 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/net/sunrpc 2023-08-09T13:16:40.493 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/net/sunrpc/auth_gss 2023-08-09T13:16:40.493 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/net/sunrpc/auth_gss/auth_rpcgss.ko 2023-08-09T13:16:40.493 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/net/sunrpc/auth_gss/rpcsec_gss_krb5.ko 2023-08-09T13:16:40.493 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/net/sunrpc/sunrpc.ko 2023-08-09T13:16:40.493 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/net/sunrpc/xprtrdma 2023-08-09T13:16:40.493 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/net/sunrpc/xprtrdma/rpcrdma.ko 2023-08-09T13:16:40.493 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/net/tipc 2023-08-09T13:16:40.493 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/net/tipc/diag.ko 2023-08-09T13:16:40.494 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/net/tipc/tipc.ko 2023-08-09T13:16:40.494 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/net/unix 2023-08-09T13:16:40.494 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/net/unix/unix_diag.ko 2023-08-09T13:16:40.494 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/net/vmw_vsock 2023-08-09T13:16:40.494 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/net/vmw_vsock/vmw_vsock_virtio_transport_common.ko 2023-08-09T13:16:40.494 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/net/vmw_vsock/vmw_vsock_vmci_transport.ko 2023-08-09T13:16:40.494 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/net/vmw_vsock/vsock.ko 2023-08-09T13:16:40.495 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/net/vmw_vsock/vsock_diag.ko 2023-08-09T13:16:40.495 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/net/vmw_vsock/vsock_loopback.ko 2023-08-09T13:16:40.495 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/net/xfrm 2023-08-09T13:16:40.495 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/net/xfrm/xfrm_ipcomp.ko 2023-08-09T13:16:40.495 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/security 2023-08-09T13:16:40.495 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/security/keys 2023-08-09T13:16:40.495 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/security/keys/encrypted-keys 2023-08-09T13:16:40.495 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/security/keys/encrypted-keys/encrypted-keys.ko 2023-08-09T13:16:40.496 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/security/keys/trusted-keys 2023-08-09T13:16:40.496 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/security/keys/trusted-keys/trusted.ko 2023-08-09T13:16:40.496 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/virt 2023-08-09T13:16:40.496 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/virt/lib 2023-08-09T13:16:40.496 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/virt/lib/irqbypass.ko 2023-08-09T13:16:40.496 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/modules.alias 2023-08-09T13:16:40.496 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/modules.alias.bin 2023-08-09T13:16:40.496 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/modules.builtin 2023-08-09T13:16:40.497 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/modules.builtin.alias.bin 2023-08-09T13:16:40.497 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/modules.builtin.bin 2023-08-09T13:16:40.497 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/modules.builtin.modinfo 2023-08-09T13:16:40.497 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/modules.dep 2023-08-09T13:16:40.497 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/modules.dep.bin 2023-08-09T13:16:40.497 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/modules.devname 2023-08-09T13:16:40.497 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/modules.order 2023-08-09T13:16:40.497 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/modules.softdep 2023-08-09T13:16:40.498 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/modules.symbols 2023-08-09T13:16:40.498 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/modules.symbols.bin 2023-08-09T13:16:40.499 DEBUG:teuthology.task.kernel:get_image_version: 6.5.0-rc4-ga7fb1265323d 2023-08-09T13:16:40.499 DEBUG:teuthology.orchestra.run.smithi043:> rpm --scripts -qp /tmp/kernel.x86_64.rpm 2023-08-09T13:16:40.523 INFO:teuthology.orchestra.run.smithi043.stdout:postinstall scriptlet (using /bin/sh): 2023-08-09T13:16:40.523 INFO:teuthology.orchestra.run.smithi043.stdout:if [ -x /sbin/installkernel -a -r /boot/vmlinuz-6.5.0-rc4-ga7fb1265323d -a -r /boot/System.map-6.5.0-rc4-ga7fb1265323d ]; then 2023-08-09T13:16:40.523 INFO:teuthology.orchestra.run.smithi043.stdout:cp /boot/vmlinuz-6.5.0-rc4-ga7fb1265323d /boot/.vmlinuz-6.5.0-rc4-ga7fb1265323d-rpm 2023-08-09T13:16:40.523 INFO:teuthology.orchestra.run.smithi043.stdout:cp /boot/System.map-6.5.0-rc4-ga7fb1265323d /boot/.System.map-6.5.0-rc4-ga7fb1265323d-rpm 2023-08-09T13:16:40.523 INFO:teuthology.orchestra.run.smithi043.stdout:rm -f /boot/vmlinuz-6.5.0-rc4-ga7fb1265323d /boot/System.map-6.5.0-rc4-ga7fb1265323d 2023-08-09T13:16:40.524 INFO:teuthology.orchestra.run.smithi043.stdout:/sbin/installkernel 6.5.0-rc4-ga7fb1265323d /boot/.vmlinuz-6.5.0-rc4-ga7fb1265323d-rpm /boot/.System.map-6.5.0-rc4-ga7fb1265323d-rpm 2023-08-09T13:16:40.524 INFO:teuthology.orchestra.run.smithi043.stdout:rm -f /boot/.vmlinuz-6.5.0-rc4-ga7fb1265323d-rpm /boot/.System.map-6.5.0-rc4-ga7fb1265323d-rpm 2023-08-09T13:16:40.524 INFO:teuthology.orchestra.run.smithi043.stdout:fi 2023-08-09T13:16:40.524 INFO:teuthology.orchestra.run.smithi043.stdout:preuninstall scriptlet (using /bin/sh): 2023-08-09T13:16:40.524 INFO:teuthology.orchestra.run.smithi043.stdout:if [ -x /sbin/new-kernel-pkg ]; then 2023-08-09T13:16:40.524 INFO:teuthology.orchestra.run.smithi043.stdout:new-kernel-pkg --remove 6.5.0-rc4-ga7fb1265323d --rminitrd --initrdfile=/boot/initramfs-6.5.0-rc4-ga7fb1265323d.img 2023-08-09T13:16:40.524 INFO:teuthology.orchestra.run.smithi043.stdout:elif [ -x /usr/bin/kernel-install ]; then 2023-08-09T13:16:40.525 INFO:teuthology.orchestra.run.smithi043.stdout:kernel-install remove 6.5.0-rc4-ga7fb1265323d 2023-08-09T13:16:40.525 INFO:teuthology.orchestra.run.smithi043.stdout:fi 2023-08-09T13:16:40.525 INFO:teuthology.orchestra.run.smithi043.stdout:postuninstall scriptlet (using /bin/sh): 2023-08-09T13:16:40.525 INFO:teuthology.orchestra.run.smithi043.stdout:if [ -x /sbin/update-bootloader ]; then 2023-08-09T13:16:40.525 INFO:teuthology.orchestra.run.smithi043.stdout:/sbin/update-bootloader --remove 6.5.0-rc4-ga7fb1265323d 2023-08-09T13:16:40.525 INFO:teuthology.orchestra.run.smithi043.stdout:fi 2023-08-09T13:16:40.526 DEBUG:teuthology.orchestra.run.smithi043:> sudo rpm -qi grub2-tools 2023-08-09T13:16:40.607 INFO:teuthology.orchestra.run.smithi043.stdout:Name : grub2-tools 2023-08-09T13:16:40.607 INFO:teuthology.orchestra.run.smithi043.stdout:Epoch : 1 2023-08-09T13:16:40.608 INFO:teuthology.orchestra.run.smithi043.stdout:Version : 2.06 2023-08-09T13:16:40.608 INFO:teuthology.orchestra.run.smithi043.stdout:Release : 53.el9 2023-08-09T13:16:40.608 INFO:teuthology.orchestra.run.smithi043.stdout:Architecture: x86_64 2023-08-09T13:16:40.608 INFO:teuthology.orchestra.run.smithi043.stdout:Install Date: Mon 30 Jan 2023 10:38:40 AM UTC 2023-08-09T13:16:40.608 INFO:teuthology.orchestra.run.smithi043.stdout:Group : Unspecified 2023-08-09T13:16:40.609 INFO:teuthology.orchestra.run.smithi043.stdout:Size : 8551923 2023-08-09T13:16:40.609 INFO:teuthology.orchestra.run.smithi043.stdout:License : GPLv3+ 2023-08-09T13:16:40.609 INFO:teuthology.orchestra.run.smithi043.stdout:Signature : RSA/SHA256, Wed 18 Jan 2023 04:40:07 PM UTC, Key ID 05b555b38483c65d 2023-08-09T13:16:40.609 INFO:teuthology.orchestra.run.smithi043.stdout:Source RPM : grub2-2.06-53.el9.src.rpm 2023-08-09T13:16:40.609 INFO:teuthology.orchestra.run.smithi043.stdout:Build Date : Fri 13 Jan 2023 09:07:28 PM UTC 2023-08-09T13:16:40.609 INFO:teuthology.orchestra.run.smithi043.stdout:Build Host : x86-05.stream.rdu2.redhat.com 2023-08-09T13:16:40.610 INFO:teuthology.orchestra.run.smithi043.stdout:Packager : builder@centos.org 2023-08-09T13:16:40.610 INFO:teuthology.orchestra.run.smithi043.stdout:Vendor : CentOS 2023-08-09T13:16:40.610 INFO:teuthology.orchestra.run.smithi043.stdout:URL : http://www.gnu.org/software/grub/ 2023-08-09T13:16:40.610 INFO:teuthology.orchestra.run.smithi043.stdout:Summary : Support tools for GRUB. 2023-08-09T13:16:40.610 INFO:teuthology.orchestra.run.smithi043.stdout:Description : 2023-08-09T13:16:40.610 INFO:teuthology.orchestra.run.smithi043.stdout: 2023-08-09T13:16:40.610 INFO:teuthology.orchestra.run.smithi043.stdout:The GRand Unified Bootloader (GRUB) is a highly configurable and 2023-08-09T13:16:40.611 INFO:teuthology.orchestra.run.smithi043.stdout:customizable bootloader with modular architecture. It supports a rich 2023-08-09T13:16:40.611 INFO:teuthology.orchestra.run.smithi043.stdout:variety of kernel formats, file systems, computer architectures and 2023-08-09T13:16:40.611 INFO:teuthology.orchestra.run.smithi043.stdout:hardware devices. 2023-08-09T13:16:40.611 INFO:teuthology.orchestra.run.smithi043.stdout: 2023-08-09T13:16:40.611 INFO:teuthology.orchestra.run.smithi043.stdout:This subpackage provides tools for support of all platforms. 2023-08-09T13:16:40.612 INFO:teuthology.task.kernel:Updating Grub Version: grub2 2023-08-09T13:16:40.612 INFO:teuthology.task.kernel:Updating grub on smithi043 to boot 6.5.0-rc4-ga7fb1265323d 2023-08-09T13:16:40.613 DEBUG:teuthology.orchestra.run.smithi043:> sudo grub2-mkconfig -o /boot/grub2/grub.cfg 2023-08-09T13:16:41.279 INFO:teuthology.orchestra.run.smithi043.stderr:Generating grub configuration file ... 2023-08-09T13:16:42.622 INFO:teuthology.orchestra.run.smithi043.stderr:Adding boot menu entry for UEFI Firmware Settings ... 2023-08-09T13:16:42.658 INFO:teuthology.orchestra.run.smithi043.stderr:done 2023-08-09T13:16:42.667 DEBUG:teuthology.orchestra.run.smithi043:> mktemp 2023-08-09T13:16:42.683 INFO:teuthology.orchestra.run.smithi043.stdout:/tmp/tmp.UX271ZGucH 2023-08-09T13:16:42.683 DEBUG:teuthology.orchestra.run.smithi043:> sudo cp /boot/grub2/grub.cfg /tmp/tmp.UX271ZGucH 2023-08-09T13:16:42.754 DEBUG:teuthology.orchestra.run.smithi043:> sudo chmod 0666 /tmp/tmp.UX271ZGucH 2023-08-09T13:16:42.904 DEBUG:teuthology.orchestra.remote:smithi043:/tmp/tmp.UX271ZGucH is 6KB 2023-08-09T13:16:42.953 DEBUG:teuthology.orchestra.run.smithi043:> rm -fr /tmp/tmp.UX271ZGucH 2023-08-09T13:16:42.968 DEBUG:teuthology.orchestra.run.smithi043:> sudo /bin/ls /boot/loader/entries || true 2023-08-09T13:16:43.036 INFO:teuthology.orchestra.run.smithi043.stdout:e13cba1d25f14c51a1d04da8db823a4e-0-rescue.conf 2023-08-09T13:16:43.036 INFO:teuthology.orchestra.run.smithi043.stdout:e13cba1d25f14c51a1d04da8db823a4e-5.14.0-239.el9.x86_64.conf 2023-08-09T13:16:43.037 INFO:teuthology.orchestra.run.smithi043.stdout:e13cba1d25f14c51a1d04da8db823a4e-6.5.0-rc4-ga7fb1265323d.conf 2023-08-09T13:16:43.038 DEBUG:teuthology.orchestra.run.smithi043:> sudo grub2-set-default e13cba1d25f14c51a1d04da8db823a4e-6.5.0-rc4-ga7fb1265323d 2023-08-09T13:16:43.208 DEBUG:teuthology.orchestra.run.smithi043:> sudo shutdown -r now 2023-08-09T13:16:53.595 DEBUG:teuthology.task.kernel:install_kernel(remote=ubuntu@smithi203.front.sepia.ceph.com, path=/tmp/kernel.x86_64.rpm, version=None) 2023-08-09T13:16:53.596 DEBUG:teuthology.orchestra.run.smithi203:> rpm -qlp /tmp/kernel.x86_64.rpm 2023-08-09T13:16:53.685 INFO:teuthology.orchestra.run.smithi203.stdout:/boot/System.map-6.5.0-rc4-ga7fb1265323d 2023-08-09T13:16:53.686 INFO:teuthology.orchestra.run.smithi203.stdout:/boot/config-6.5.0-rc4-ga7fb1265323d 2023-08-09T13:16:53.686 INFO:teuthology.orchestra.run.smithi203.stdout:/boot/vmlinuz-6.5.0-rc4-ga7fb1265323d 2023-08-09T13:16:53.686 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d 2023-08-09T13:16:53.686 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel 2023-08-09T13:16:53.686 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/arch 2023-08-09T13:16:53.687 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/arch/x86 2023-08-09T13:16:53.687 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/arch/x86/crypto 2023-08-09T13:16:53.687 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/arch/x86/crypto/blowfish-x86_64.ko 2023-08-09T13:16:53.687 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/arch/x86/crypto/camellia-aesni-avx-x86_64.ko 2023-08-09T13:16:53.687 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/arch/x86/crypto/camellia-aesni-avx2.ko 2023-08-09T13:16:53.687 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/arch/x86/crypto/camellia-x86_64.ko 2023-08-09T13:16:53.687 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/arch/x86/crypto/cast5-avx-x86_64.ko 2023-08-09T13:16:53.687 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/arch/x86/crypto/cast6-avx-x86_64.ko 2023-08-09T13:16:53.688 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/arch/x86/crypto/crc32-pclmul.ko 2023-08-09T13:16:53.688 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/arch/x86/crypto/crc32c-intel.ko 2023-08-09T13:16:53.688 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/arch/x86/crypto/crct10dif-pclmul.ko 2023-08-09T13:16:53.688 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/arch/x86/crypto/des3_ede-x86_64.ko 2023-08-09T13:16:53.688 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/arch/x86/crypto/ghash-clmulni-intel.ko 2023-08-09T13:16:53.688 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/arch/x86/crypto/serpent-avx-x86_64.ko 2023-08-09T13:16:53.688 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/arch/x86/crypto/serpent-avx2.ko 2023-08-09T13:16:53.689 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/arch/x86/crypto/serpent-sse2-x86_64.ko 2023-08-09T13:16:53.689 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/arch/x86/crypto/sha1-ssse3.ko 2023-08-09T13:16:53.689 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/arch/x86/crypto/sha256-ssse3.ko 2023-08-09T13:16:53.689 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/arch/x86/crypto/sha512-ssse3.ko 2023-08-09T13:16:53.689 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/arch/x86/crypto/twofish-avx-x86_64.ko 2023-08-09T13:16:53.689 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/arch/x86/crypto/twofish-x86_64-3way.ko 2023-08-09T13:16:53.689 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/arch/x86/crypto/twofish-x86_64.ko 2023-08-09T13:16:53.690 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/arch/x86/kernel 2023-08-09T13:16:53.690 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/arch/x86/kernel/cpu 2023-08-09T13:16:53.690 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/arch/x86/kernel/cpu/mce 2023-08-09T13:16:53.690 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/arch/x86/kernel/cpu/mce/mce-inject.ko 2023-08-09T13:16:53.690 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/arch/x86/kvm 2023-08-09T13:16:53.690 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/arch/x86/kvm/kvm-amd.ko 2023-08-09T13:16:53.690 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/arch/x86/kvm/kvm-intel.ko 2023-08-09T13:16:53.690 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/arch/x86/kvm/kvm.ko 2023-08-09T13:16:53.691 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/crypto 2023-08-09T13:16:53.691 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/crypto/ansi_cprng.ko 2023-08-09T13:16:53.691 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/crypto/anubis.ko 2023-08-09T13:16:53.691 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/crypto/arc4.ko 2023-08-09T13:16:53.691 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/crypto/async_tx 2023-08-09T13:16:53.691 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/crypto/async_tx/async_memcpy.ko 2023-08-09T13:16:53.691 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/crypto/async_tx/async_pq.ko 2023-08-09T13:16:53.691 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/crypto/async_tx/async_raid6_recov.ko 2023-08-09T13:16:53.692 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/crypto/async_tx/async_tx.ko 2023-08-09T13:16:53.692 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/crypto/async_tx/async_xor.ko 2023-08-09T13:16:53.692 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/crypto/async_tx/raid6test.ko 2023-08-09T13:16:53.692 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/crypto/authenc.ko 2023-08-09T13:16:53.692 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/crypto/authencesn.ko 2023-08-09T13:16:53.692 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/crypto/blake2b_generic.ko 2023-08-09T13:16:53.692 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/crypto/blowfish_common.ko 2023-08-09T13:16:53.693 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/crypto/blowfish_generic.ko 2023-08-09T13:16:53.693 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/crypto/camellia_generic.ko 2023-08-09T13:16:53.693 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/crypto/cast5_generic.ko 2023-08-09T13:16:53.693 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/crypto/cast6_generic.ko 2023-08-09T13:16:53.693 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/crypto/cast_common.ko 2023-08-09T13:16:53.693 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/crypto/ccm.ko 2023-08-09T13:16:53.693 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/crypto/cmac.ko 2023-08-09T13:16:53.693 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/crypto/crc32_generic.ko 2023-08-09T13:16:53.694 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/crypto/crypto_engine.ko 2023-08-09T13:16:53.694 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/crypto/crypto_user.ko 2023-08-09T13:16:53.694 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/crypto/cts.ko 2023-08-09T13:16:53.694 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/crypto/des_generic.ko 2023-08-09T13:16:53.694 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/crypto/dh_generic.ko 2023-08-09T13:16:53.694 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/crypto/echainiv.ko 2023-08-09T13:16:53.694 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/crypto/essiv.ko 2023-08-09T13:16:53.694 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/crypto/fcrypt.ko 2023-08-09T13:16:53.695 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/crypto/khazad.ko 2023-08-09T13:16:53.695 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/crypto/lz4.ko 2023-08-09T13:16:53.695 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/crypto/lz4hc.ko 2023-08-09T13:16:53.695 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/crypto/md4.ko 2023-08-09T13:16:53.695 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/crypto/michael_mic.ko 2023-08-09T13:16:53.695 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/crypto/pcbc.ko 2023-08-09T13:16:53.695 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/crypto/pcrypt.ko 2023-08-09T13:16:53.696 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/crypto/rmd160.ko 2023-08-09T13:16:53.696 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/crypto/seed.ko 2023-08-09T13:16:53.696 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/crypto/serpent_generic.ko 2023-08-09T13:16:53.696 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/crypto/tcrypt.ko 2023-08-09T13:16:53.696 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/crypto/tea.ko 2023-08-09T13:16:53.696 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/crypto/twofish_common.ko 2023-08-09T13:16:53.696 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/crypto/twofish_generic.ko 2023-08-09T13:16:53.696 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/crypto/vmac.ko 2023-08-09T13:16:53.697 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/crypto/wp512.ko 2023-08-09T13:16:53.697 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/crypto/xcbc.ko 2023-08-09T13:16:53.697 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/crypto/xor.ko 2023-08-09T13:16:53.697 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/crypto/xxhash_generic.ko 2023-08-09T13:16:53.697 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers 2023-08-09T13:16:53.697 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/acpi 2023-08-09T13:16:53.698 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/acpi/acpi_ipmi.ko 2023-08-09T13:16:53.698 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/acpi/acpi_pad.ko 2023-08-09T13:16:53.698 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/acpi/custom_method.ko 2023-08-09T13:16:53.698 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/acpi/ec_sys.ko 2023-08-09T13:16:53.698 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/acpi/platform_profile.ko 2023-08-09T13:16:53.698 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/acpi/sbs.ko 2023-08-09T13:16:53.698 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/acpi/sbshc.ko 2023-08-09T13:16:53.698 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/acpi/video.ko 2023-08-09T13:16:53.699 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/ata 2023-08-09T13:16:53.699 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/ata/acard-ahci.ko 2023-08-09T13:16:53.699 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/ata/ahci_platform.ko 2023-08-09T13:16:53.699 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/ata/ata_generic.ko 2023-08-09T13:16:53.699 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/ata/libahci_platform.ko 2023-08-09T13:16:53.700 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/ata/pata_acpi.ko 2023-08-09T13:16:53.700 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/ata/pata_ali.ko 2023-08-09T13:16:53.700 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/ata/pata_amd.ko 2023-08-09T13:16:53.700 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/ata/pata_artop.ko 2023-08-09T13:16:53.700 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/ata/pata_atiixp.ko 2023-08-09T13:16:53.700 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/ata/pata_atp867x.ko 2023-08-09T13:16:53.700 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/ata/pata_cmd640.ko 2023-08-09T13:16:53.701 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/ata/pata_cmd64x.ko 2023-08-09T13:16:53.701 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/ata/pata_cypress.ko 2023-08-09T13:16:53.701 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/ata/pata_efar.ko 2023-08-09T13:16:53.701 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/ata/pata_hpt366.ko 2023-08-09T13:16:53.701 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/ata/pata_hpt37x.ko 2023-08-09T13:16:53.701 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/ata/pata_hpt3x2n.ko 2023-08-09T13:16:53.701 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/ata/pata_hpt3x3.ko 2023-08-09T13:16:53.702 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/ata/pata_it8213.ko 2023-08-09T13:16:53.702 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/ata/pata_it821x.ko 2023-08-09T13:16:53.702 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/ata/pata_jmicron.ko 2023-08-09T13:16:53.702 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/ata/pata_marvell.ko 2023-08-09T13:16:53.702 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/ata/pata_mpiix.ko 2023-08-09T13:16:53.702 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/ata/pata_netcell.ko 2023-08-09T13:16:53.702 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/ata/pata_ninja32.ko 2023-08-09T13:16:53.702 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/ata/pata_ns87410.ko 2023-08-09T13:16:53.703 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/ata/pata_ns87415.ko 2023-08-09T13:16:53.703 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/ata/pata_oldpiix.ko 2023-08-09T13:16:53.703 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/ata/pata_opti.ko 2023-08-09T13:16:53.703 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/ata/pata_optidma.ko 2023-08-09T13:16:53.703 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/ata/pata_pcmcia.ko 2023-08-09T13:16:53.703 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/ata/pata_pdc2027x.ko 2023-08-09T13:16:53.703 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/ata/pata_pdc202xx_old.ko 2023-08-09T13:16:53.703 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/ata/pata_piccolo.ko 2023-08-09T13:16:53.704 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/ata/pata_rdc.ko 2023-08-09T13:16:53.704 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/ata/pata_sch.ko 2023-08-09T13:16:53.704 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/ata/pata_serverworks.ko 2023-08-09T13:16:53.704 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/ata/pata_sil680.ko 2023-08-09T13:16:53.704 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/ata/pata_sis.ko 2023-08-09T13:16:53.704 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/ata/pata_sl82c105.ko 2023-08-09T13:16:53.704 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/ata/pata_triflex.ko 2023-08-09T13:16:53.704 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/ata/pata_via.ko 2023-08-09T13:16:53.705 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/ata/pdc_adma.ko 2023-08-09T13:16:53.705 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/ata/sata_inic162x.ko 2023-08-09T13:16:53.705 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/ata/sata_mv.ko 2023-08-09T13:16:53.705 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/ata/sata_nv.ko 2023-08-09T13:16:53.705 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/ata/sata_promise.ko 2023-08-09T13:16:53.705 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/ata/sata_qstor.ko 2023-08-09T13:16:53.705 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/ata/sata_sil.ko 2023-08-09T13:16:53.706 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/ata/sata_sil24.ko 2023-08-09T13:16:53.706 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/ata/sata_sis.ko 2023-08-09T13:16:53.706 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/ata/sata_svw.ko 2023-08-09T13:16:53.706 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/ata/sata_sx4.ko 2023-08-09T13:16:53.706 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/ata/sata_uli.ko 2023-08-09T13:16:53.706 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/ata/sata_via.ko 2023-08-09T13:16:53.706 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/ata/sata_vsc.ko 2023-08-09T13:16:53.706 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/auxdisplay 2023-08-09T13:16:53.707 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/auxdisplay/ks0108.ko 2023-08-09T13:16:53.707 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/base 2023-08-09T13:16:53.707 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/base/regmap 2023-08-09T13:16:53.707 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/base/regmap/regmap-i2c.ko 2023-08-09T13:16:53.707 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/base/regmap/regmap-mmio.ko 2023-08-09T13:16:53.707 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/bcma 2023-08-09T13:16:53.707 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/bcma/bcma.ko 2023-08-09T13:16:53.707 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/block 2023-08-09T13:16:53.708 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/block/aoe 2023-08-09T13:16:53.708 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/block/aoe/aoe.ko 2023-08-09T13:16:53.708 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/block/brd.ko 2023-08-09T13:16:53.708 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/block/drbd 2023-08-09T13:16:53.708 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/block/drbd/drbd.ko 2023-08-09T13:16:53.708 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/block/floppy.ko 2023-08-09T13:16:53.708 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/block/loop.ko 2023-08-09T13:16:53.708 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/block/mtip32xx 2023-08-09T13:16:53.709 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/block/mtip32xx/mtip32xx.ko 2023-08-09T13:16:53.709 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/block/nbd.ko 2023-08-09T13:16:53.709 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/block/null_blk 2023-08-09T13:16:53.709 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/block/null_blk/null_blk.ko 2023-08-09T13:16:53.709 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/block/pktcdvd.ko 2023-08-09T13:16:53.709 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/block/rbd.ko 2023-08-09T13:16:53.709 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/block/virtio_blk.ko 2023-08-09T13:16:53.709 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/block/xen-blkback 2023-08-09T13:16:53.710 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/block/xen-blkback/xen-blkback.ko 2023-08-09T13:16:53.710 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/block/xen-blkfront.ko 2023-08-09T13:16:53.710 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/block/zram 2023-08-09T13:16:53.710 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/block/zram/zram.ko 2023-08-09T13:16:53.710 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/char 2023-08-09T13:16:53.710 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/char/hangcheck-timer.ko 2023-08-09T13:16:53.710 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/char/hw_random 2023-08-09T13:16:53.710 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/char/hw_random/amd-rng.ko 2023-08-09T13:16:53.711 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/char/hw_random/intel-rng.ko 2023-08-09T13:16:53.711 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/char/hw_random/timeriomem-rng.ko 2023-08-09T13:16:53.711 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/char/hw_random/via-rng.ko 2023-08-09T13:16:53.711 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/char/hw_random/virtio-rng.ko 2023-08-09T13:16:53.711 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/char/ipmi 2023-08-09T13:16:53.711 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/char/ipmi/ipmi_devintf.ko 2023-08-09T13:16:53.711 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/char/ipmi/ipmi_msghandler.ko 2023-08-09T13:16:53.711 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/char/ipmi/ipmi_poweroff.ko 2023-08-09T13:16:53.712 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/char/ipmi/ipmi_si.ko 2023-08-09T13:16:53.712 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/char/ipmi/ipmi_ssif.ko 2023-08-09T13:16:53.712 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/char/ipmi/ipmi_watchdog.ko 2023-08-09T13:16:53.712 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/char/lp.ko 2023-08-09T13:16:53.712 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/char/mwave 2023-08-09T13:16:53.712 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/char/mwave/mwave.ko 2023-08-09T13:16:53.712 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/char/ppdev.ko 2023-08-09T13:16:53.713 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/char/tlclk.ko 2023-08-09T13:16:53.713 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/char/tpm 2023-08-09T13:16:53.713 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/char/tpm/tpm.ko 2023-08-09T13:16:53.713 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/char/tpm/tpm_atmel.ko 2023-08-09T13:16:53.713 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/char/tpm/tpm_infineon.ko 2023-08-09T13:16:53.713 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/char/tpm/tpm_nsc.ko 2023-08-09T13:16:53.713 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/char/tpm/tpm_tis.ko 2023-08-09T13:16:53.713 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/char/tpm/tpm_tis_core.ko 2023-08-09T13:16:53.714 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/char/uv_mmtimer.ko 2023-08-09T13:16:53.714 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/char/virtio_console.ko 2023-08-09T13:16:53.714 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/cpufreq 2023-08-09T13:16:53.714 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/cpufreq/acpi-cpufreq.ko 2023-08-09T13:16:53.714 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/cpufreq/amd_freq_sensitivity.ko 2023-08-09T13:16:53.714 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/cpufreq/p4-clockmod.ko 2023-08-09T13:16:53.714 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/cpufreq/pcc-cpufreq.ko 2023-08-09T13:16:53.714 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/cpufreq/powernow-k8.ko 2023-08-09T13:16:53.715 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/cpufreq/speedstep-lib.ko 2023-08-09T13:16:53.715 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/crypto 2023-08-09T13:16:53.715 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/crypto/ccp 2023-08-09T13:16:53.715 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/crypto/ccp/ccp-crypto.ko 2023-08-09T13:16:53.715 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/crypto/ccp/ccp.ko 2023-08-09T13:16:53.715 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/crypto/intel 2023-08-09T13:16:53.715 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/crypto/intel/qat 2023-08-09T13:16:53.715 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/crypto/intel/qat/qat_common 2023-08-09T13:16:53.716 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/crypto/intel/qat/qat_common/intel_qat.ko 2023-08-09T13:16:53.716 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/crypto/intel/qat/qat_dh895xcc 2023-08-09T13:16:53.716 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/crypto/intel/qat/qat_dh895xcc/qat_dh895xcc.ko 2023-08-09T13:16:53.716 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/crypto/padlock-aes.ko 2023-08-09T13:16:53.716 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/crypto/padlock-sha.ko 2023-08-09T13:16:53.716 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/crypto/virtio 2023-08-09T13:16:53.716 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/crypto/virtio/virtio_crypto.ko 2023-08-09T13:16:53.716 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/dca 2023-08-09T13:16:53.717 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/dca/dca.ko 2023-08-09T13:16:53.717 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/devfreq 2023-08-09T13:16:53.717 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/devfreq/governor_simpleondemand.ko 2023-08-09T13:16:53.717 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/dma 2023-08-09T13:16:53.717 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/dma/dw 2023-08-09T13:16:53.717 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/dma/dw/dw_dmac.ko 2023-08-09T13:16:53.717 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/dma/ioat 2023-08-09T13:16:53.718 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/dma/ioat/ioatdma.ko 2023-08-09T13:16:53.718 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/edac 2023-08-09T13:16:53.718 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/edac/amd64_edac.ko 2023-08-09T13:16:53.718 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/edac/e752x_edac.ko 2023-08-09T13:16:53.718 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/edac/edac_mce_amd.ko 2023-08-09T13:16:53.718 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/edac/i3000_edac.ko 2023-08-09T13:16:53.718 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/edac/i3200_edac.ko 2023-08-09T13:16:53.718 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/edac/i5100_edac.ko 2023-08-09T13:16:53.719 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/edac/i5400_edac.ko 2023-08-09T13:16:53.719 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/edac/i7300_edac.ko 2023-08-09T13:16:53.719 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/edac/i7core_edac.ko 2023-08-09T13:16:53.719 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/edac/i82975x_edac.ko 2023-08-09T13:16:53.719 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/edac/ie31200_edac.ko 2023-08-09T13:16:53.719 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/edac/sb_edac.ko 2023-08-09T13:16:53.719 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/edac/x38_edac.ko 2023-08-09T13:16:53.719 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/firmware 2023-08-09T13:16:53.720 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/firmware/edd.ko 2023-08-09T13:16:53.720 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/firmware/iscsi_ibft.ko 2023-08-09T13:16:53.720 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/gpio 2023-08-09T13:16:53.720 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/gpio/gpio-ich.ko 2023-08-09T13:16:53.720 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/gpio/gpio-viperboard.ko 2023-08-09T13:16:53.720 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/hid 2023-08-09T13:16:53.720 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/hid/hid-appleir.ko 2023-08-09T13:16:53.721 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/hid/hid-aureal.ko 2023-08-09T13:16:53.721 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/hid/hid-axff.ko 2023-08-09T13:16:53.721 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/hid/hid-dr.ko 2023-08-09T13:16:53.721 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/hid/hid-elecom.ko 2023-08-09T13:16:53.721 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/hid/hid-elo.ko 2023-08-09T13:16:53.721 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/hid/hid-emsff.ko 2023-08-09T13:16:53.721 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/hid/hid-gaff.ko 2023-08-09T13:16:53.721 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/hid/hid-gt683r.ko 2023-08-09T13:16:53.722 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/hid/hid-gyration.ko 2023-08-09T13:16:53.722 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/hid/hid-holtek-kbd.ko 2023-08-09T13:16:53.722 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/hid/hid-holtek-mouse.ko 2023-08-09T13:16:53.722 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/hid/hid-holtekff.ko 2023-08-09T13:16:53.722 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/hid/hid-hyperv.ko 2023-08-09T13:16:53.722 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/hid/hid-icade.ko 2023-08-09T13:16:53.722 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/hid/hid-keytouch.ko 2023-08-09T13:16:53.722 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/hid/hid-kye.ko 2023-08-09T13:16:53.723 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/hid/hid-lcpower.ko 2023-08-09T13:16:53.723 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/hid/hid-led.ko 2023-08-09T13:16:53.723 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/hid/hid-lenovo.ko 2023-08-09T13:16:53.723 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/hid/hid-logitech-dj.ko 2023-08-09T13:16:53.723 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/hid/hid-logitech-hidpp.ko 2023-08-09T13:16:53.723 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/hid/hid-multitouch.ko 2023-08-09T13:16:53.723 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/hid/hid-ortek.ko 2023-08-09T13:16:53.723 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/hid/hid-penmount.ko 2023-08-09T13:16:53.724 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/hid/hid-petalynx.ko 2023-08-09T13:16:53.724 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/hid/hid-picolcd.ko 2023-08-09T13:16:53.724 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/hid/hid-pl.ko 2023-08-09T13:16:53.724 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/hid/hid-plantronics.ko 2023-08-09T13:16:53.724 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/hid/hid-primax.ko 2023-08-09T13:16:53.724 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/hid/hid-rmi.ko 2023-08-09T13:16:53.724 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/hid/hid-roccat-arvo.ko 2023-08-09T13:16:53.724 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/hid/hid-roccat-common.ko 2023-08-09T13:16:53.725 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/hid/hid-roccat-isku.ko 2023-08-09T13:16:53.725 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/hid/hid-roccat-kone.ko 2023-08-09T13:16:53.725 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/hid/hid-roccat-koneplus.ko 2023-08-09T13:16:53.725 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/hid/hid-roccat-konepure.ko 2023-08-09T13:16:53.725 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/hid/hid-roccat-kovaplus.ko 2023-08-09T13:16:53.725 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/hid/hid-roccat-lua.ko 2023-08-09T13:16:53.725 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/hid/hid-roccat-pyra.ko 2023-08-09T13:16:53.726 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/hid/hid-roccat-ryos.ko 2023-08-09T13:16:53.726 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/hid/hid-roccat-savu.ko 2023-08-09T13:16:53.726 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/hid/hid-roccat.ko 2023-08-09T13:16:53.726 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/hid/hid-saitek.ko 2023-08-09T13:16:53.726 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/hid/hid-samsung.ko 2023-08-09T13:16:53.726 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/hid/hid-sensor-hub.ko 2023-08-09T13:16:53.726 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/hid/hid-sjoy.ko 2023-08-09T13:16:53.726 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/hid/hid-sony.ko 2023-08-09T13:16:53.727 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/hid/hid-speedlink.ko 2023-08-09T13:16:53.727 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/hid/hid-steelseries.ko 2023-08-09T13:16:53.727 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/hid/hid-sunplus.ko 2023-08-09T13:16:53.727 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/hid/hid-thrustmaster.ko 2023-08-09T13:16:53.727 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/hid/hid-tivo.ko 2023-08-09T13:16:53.727 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/hid/hid-tmff.ko 2023-08-09T13:16:53.727 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/hid/hid-topseed.ko 2023-08-09T13:16:53.727 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/hid/hid-twinhan.ko 2023-08-09T13:16:53.728 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/hid/hid-uclogic.ko 2023-08-09T13:16:53.728 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/hid/hid-waltop.ko 2023-08-09T13:16:53.728 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/hid/hid-wiimote.ko 2023-08-09T13:16:53.728 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/hid/hid-xinmo.ko 2023-08-09T13:16:53.728 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/hid/hid-zpff.ko 2023-08-09T13:16:53.728 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/hid/hid-zydacron.ko 2023-08-09T13:16:53.728 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/hid/uhid.ko 2023-08-09T13:16:53.728 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/hid/wacom.ko 2023-08-09T13:16:53.729 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/hv 2023-08-09T13:16:53.729 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/hv/hv_balloon.ko 2023-08-09T13:16:53.729 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/hv/hv_utils.ko 2023-08-09T13:16:53.729 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/hv/hv_vmbus.ko 2023-08-09T13:16:53.729 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/hwmon 2023-08-09T13:16:53.729 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/hwmon/abituguru.ko 2023-08-09T13:16:53.729 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/hwmon/abituguru3.ko 2023-08-09T13:16:53.729 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/hwmon/acpi_power_meter.ko 2023-08-09T13:16:53.730 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/hwmon/ad7414.ko 2023-08-09T13:16:53.730 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/hwmon/ad7418.ko 2023-08-09T13:16:53.730 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/hwmon/adc128d818.ko 2023-08-09T13:16:53.730 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/hwmon/adm1025.ko 2023-08-09T13:16:53.730 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/hwmon/adm1026.ko 2023-08-09T13:16:53.730 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/hwmon/adm1029.ko 2023-08-09T13:16:53.730 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/hwmon/adm1031.ko 2023-08-09T13:16:53.730 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/hwmon/adm9240.ko 2023-08-09T13:16:53.731 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/hwmon/ads7828.ko 2023-08-09T13:16:53.731 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/hwmon/adt7410.ko 2023-08-09T13:16:53.731 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/hwmon/adt7411.ko 2023-08-09T13:16:53.731 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/hwmon/adt7462.ko 2023-08-09T13:16:53.731 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/hwmon/adt7470.ko 2023-08-09T13:16:53.731 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/hwmon/adt7475.ko 2023-08-09T13:16:53.731 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/hwmon/adt7x10.ko 2023-08-09T13:16:53.731 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/hwmon/amc6821.ko 2023-08-09T13:16:53.732 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/hwmon/applesmc.ko 2023-08-09T13:16:53.732 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/hwmon/asb100.ko 2023-08-09T13:16:53.732 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/hwmon/asc7621.ko 2023-08-09T13:16:53.732 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/hwmon/asus_atk0110.ko 2023-08-09T13:16:53.732 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/hwmon/atxp1.ko 2023-08-09T13:16:53.732 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/hwmon/coretemp.ko 2023-08-09T13:16:53.732 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/hwmon/dell-smm-hwmon.ko 2023-08-09T13:16:53.733 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/hwmon/dme1737.ko 2023-08-09T13:16:53.733 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/hwmon/ds1621.ko 2023-08-09T13:16:53.733 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/hwmon/ds620.ko 2023-08-09T13:16:53.733 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/hwmon/emc1403.ko 2023-08-09T13:16:53.733 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/hwmon/emc6w201.ko 2023-08-09T13:16:53.733 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/hwmon/f71805f.ko 2023-08-09T13:16:53.733 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/hwmon/f71882fg.ko 2023-08-09T13:16:53.733 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/hwmon/f75375s.ko 2023-08-09T13:16:53.734 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/hwmon/fam15h_power.ko 2023-08-09T13:16:53.734 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/hwmon/fschmd.ko 2023-08-09T13:16:53.734 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/hwmon/g760a.ko 2023-08-09T13:16:53.734 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/hwmon/g762.ko 2023-08-09T13:16:53.734 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/hwmon/gl518sm.ko 2023-08-09T13:16:53.734 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/hwmon/gl520sm.ko 2023-08-09T13:16:53.734 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/hwmon/hwmon-vid.ko 2023-08-09T13:16:53.734 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/hwmon/i5500_temp.ko 2023-08-09T13:16:53.735 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/hwmon/i5k_amb.ko 2023-08-09T13:16:53.735 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/hwmon/ibmaem.ko 2023-08-09T13:16:53.735 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/hwmon/ibmpex.ko 2023-08-09T13:16:53.735 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/hwmon/ina209.ko 2023-08-09T13:16:53.735 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/hwmon/ina2xx.ko 2023-08-09T13:16:53.735 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/hwmon/it87.ko 2023-08-09T13:16:53.735 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/hwmon/k10temp.ko 2023-08-09T13:16:53.735 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/hwmon/k8temp.ko 2023-08-09T13:16:53.736 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/hwmon/lineage-pem.ko 2023-08-09T13:16:53.736 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/hwmon/lm63.ko 2023-08-09T13:16:53.736 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/hwmon/lm73.ko 2023-08-09T13:16:53.736 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/hwmon/lm75.ko 2023-08-09T13:16:53.736 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/hwmon/lm77.ko 2023-08-09T13:16:53.736 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/hwmon/lm78.ko 2023-08-09T13:16:53.736 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/hwmon/lm80.ko 2023-08-09T13:16:53.736 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/hwmon/lm83.ko 2023-08-09T13:16:53.737 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/hwmon/lm85.ko 2023-08-09T13:16:53.737 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/hwmon/lm87.ko 2023-08-09T13:16:53.737 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/hwmon/lm90.ko 2023-08-09T13:16:53.737 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/hwmon/lm92.ko 2023-08-09T13:16:53.737 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/hwmon/lm93.ko 2023-08-09T13:16:53.737 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/hwmon/lm95234.ko 2023-08-09T13:16:53.737 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/hwmon/lm95241.ko 2023-08-09T13:16:53.737 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/hwmon/lm95245.ko 2023-08-09T13:16:53.738 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/hwmon/ltc2945.ko 2023-08-09T13:16:53.738 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/hwmon/ltc4151.ko 2023-08-09T13:16:53.738 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/hwmon/ltc4215.ko 2023-08-09T13:16:53.738 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/hwmon/ltc4222.ko 2023-08-09T13:16:53.738 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/hwmon/ltc4245.ko 2023-08-09T13:16:53.738 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/hwmon/ltc4260.ko 2023-08-09T13:16:53.738 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/hwmon/ltc4261.ko 2023-08-09T13:16:53.738 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/hwmon/max16065.ko 2023-08-09T13:16:53.739 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/hwmon/max1619.ko 2023-08-09T13:16:53.739 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/hwmon/max1668.ko 2023-08-09T13:16:53.739 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/hwmon/max197.ko 2023-08-09T13:16:53.739 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/hwmon/max6639.ko 2023-08-09T13:16:53.739 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/hwmon/max6650.ko 2023-08-09T13:16:53.739 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/hwmon/max6697.ko 2023-08-09T13:16:53.739 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/hwmon/mcp3021.ko 2023-08-09T13:16:53.739 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/hwmon/nct6683.ko 2023-08-09T13:16:53.740 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/hwmon/nct6775-core.ko 2023-08-09T13:16:53.740 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/hwmon/nct6775.ko 2023-08-09T13:16:53.740 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/hwmon/nct7802.ko 2023-08-09T13:16:53.740 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/hwmon/pc87360.ko 2023-08-09T13:16:53.740 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/hwmon/pc87427.ko 2023-08-09T13:16:53.740 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/hwmon/pcf8591.ko 2023-08-09T13:16:53.740 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/hwmon/pmbus 2023-08-09T13:16:53.741 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/hwmon/pmbus/adm1275.ko 2023-08-09T13:16:53.741 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/hwmon/pmbus/lm25066.ko 2023-08-09T13:16:53.741 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/hwmon/pmbus/ltc2978.ko 2023-08-09T13:16:53.741 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/hwmon/pmbus/max16064.ko 2023-08-09T13:16:53.741 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/hwmon/pmbus/max34440.ko 2023-08-09T13:16:53.741 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/hwmon/pmbus/max8688.ko 2023-08-09T13:16:53.741 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/hwmon/pmbus/pmbus.ko 2023-08-09T13:16:53.741 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/hwmon/pmbus/pmbus_core.ko 2023-08-09T13:16:53.742 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/hwmon/pmbus/tps40422.ko 2023-08-09T13:16:53.742 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/hwmon/pmbus/ucd9000.ko 2023-08-09T13:16:53.742 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/hwmon/pmbus/ucd9200.ko 2023-08-09T13:16:53.742 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/hwmon/pmbus/zl6100.ko 2023-08-09T13:16:53.742 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/hwmon/powr1220.ko 2023-08-09T13:16:53.742 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/hwmon/sch5627.ko 2023-08-09T13:16:53.742 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/hwmon/sch5636.ko 2023-08-09T13:16:53.742 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/hwmon/sch56xx-common.ko 2023-08-09T13:16:53.743 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/hwmon/sht15.ko 2023-08-09T13:16:53.743 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/hwmon/sht21.ko 2023-08-09T13:16:53.743 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/hwmon/shtc1.ko 2023-08-09T13:16:53.743 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/hwmon/sis5595.ko 2023-08-09T13:16:53.743 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/hwmon/smsc47b397.ko 2023-08-09T13:16:53.743 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/hwmon/smsc47m1.ko 2023-08-09T13:16:53.743 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/hwmon/smsc47m192.ko 2023-08-09T13:16:53.743 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/hwmon/thmc50.ko 2023-08-09T13:16:53.744 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/hwmon/tmp102.ko 2023-08-09T13:16:53.744 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/hwmon/tmp103.ko 2023-08-09T13:16:53.744 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/hwmon/tmp401.ko 2023-08-09T13:16:53.744 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/hwmon/tmp421.ko 2023-08-09T13:16:53.744 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/hwmon/via-cputemp.ko 2023-08-09T13:16:53.744 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/hwmon/via686a.ko 2023-08-09T13:16:53.744 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/hwmon/vt1211.ko 2023-08-09T13:16:53.744 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/hwmon/vt8231.ko 2023-08-09T13:16:53.745 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/hwmon/w83627ehf.ko 2023-08-09T13:16:53.745 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/hwmon/w83627hf.ko 2023-08-09T13:16:53.745 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/hwmon/w83781d.ko 2023-08-09T13:16:53.745 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/hwmon/w83791d.ko 2023-08-09T13:16:53.745 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/hwmon/w83792d.ko 2023-08-09T13:16:53.745 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/hwmon/w83793.ko 2023-08-09T13:16:53.745 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/hwmon/w83795.ko 2023-08-09T13:16:53.745 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/hwmon/w83l785ts.ko 2023-08-09T13:16:53.746 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/hwmon/w83l786ng.ko 2023-08-09T13:16:53.746 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/i2c 2023-08-09T13:16:53.746 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/i2c/algos 2023-08-09T13:16:53.746 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/i2c/algos/i2c-algo-bit.ko 2023-08-09T13:16:53.746 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/i2c/algos/i2c-algo-pca.ko 2023-08-09T13:16:53.746 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/i2c/busses 2023-08-09T13:16:53.746 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/i2c/busses/i2c-amd756-s4882.ko 2023-08-09T13:16:53.746 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/i2c/busses/i2c-amd756.ko 2023-08-09T13:16:53.747 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/i2c/busses/i2c-amd8111.ko 2023-08-09T13:16:53.747 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/i2c/busses/i2c-ccgx-ucsi.ko 2023-08-09T13:16:53.747 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/i2c/busses/i2c-designware-core.ko 2023-08-09T13:16:53.747 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/i2c/busses/i2c-designware-pci.ko 2023-08-09T13:16:53.747 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/i2c/busses/i2c-designware-platform.ko 2023-08-09T13:16:53.747 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/i2c/busses/i2c-diolan-u2c.ko 2023-08-09T13:16:53.747 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/i2c/busses/i2c-i801.ko 2023-08-09T13:16:53.748 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/i2c/busses/i2c-isch.ko 2023-08-09T13:16:53.748 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/i2c/busses/i2c-ismt.ko 2023-08-09T13:16:53.748 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/i2c/busses/i2c-nforce2-s4985.ko 2023-08-09T13:16:53.748 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/i2c/busses/i2c-nforce2.ko 2023-08-09T13:16:53.748 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/i2c/busses/i2c-parport.ko 2023-08-09T13:16:53.748 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/i2c/busses/i2c-pca-platform.ko 2023-08-09T13:16:53.748 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/i2c/busses/i2c-piix4.ko 2023-08-09T13:16:53.748 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/i2c/busses/i2c-scmi.ko 2023-08-09T13:16:53.749 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/i2c/busses/i2c-simtec.ko 2023-08-09T13:16:53.749 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/i2c/busses/i2c-sis96x.ko 2023-08-09T13:16:53.749 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/i2c/busses/i2c-tiny-usb.ko 2023-08-09T13:16:53.749 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/i2c/busses/i2c-via.ko 2023-08-09T13:16:53.749 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/i2c/busses/i2c-viapro.ko 2023-08-09T13:16:53.749 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/i2c/busses/i2c-viperboard.ko 2023-08-09T13:16:53.749 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/i2c/i2c-dev.ko 2023-08-09T13:16:53.749 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/i2c/i2c-mux.ko 2023-08-09T13:16:53.750 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/i2c/i2c-slave-eeprom.ko 2023-08-09T13:16:53.750 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/i2c/i2c-smbus.ko 2023-08-09T13:16:53.750 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/i2c/i2c-stub.ko 2023-08-09T13:16:53.750 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/infiniband 2023-08-09T13:16:53.750 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/infiniband/core 2023-08-09T13:16:53.750 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/infiniband/core/ib_cm.ko 2023-08-09T13:16:53.750 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/infiniband/core/ib_core.ko 2023-08-09T13:16:53.750 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/infiniband/core/ib_umad.ko 2023-08-09T13:16:53.751 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/infiniband/core/ib_uverbs.ko 2023-08-09T13:16:53.751 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/infiniband/core/iw_cm.ko 2023-08-09T13:16:53.751 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/infiniband/core/rdma_cm.ko 2023-08-09T13:16:53.751 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/infiniband/core/rdma_ucm.ko 2023-08-09T13:16:53.751 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/infiniband/hw 2023-08-09T13:16:53.751 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/infiniband/hw/cxgb4 2023-08-09T13:16:53.751 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/infiniband/hw/cxgb4/iw_cxgb4.ko 2023-08-09T13:16:53.751 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/infiniband/hw/mlx4 2023-08-09T13:16:53.752 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/infiniband/hw/mlx4/mlx4_ib.ko 2023-08-09T13:16:53.752 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/infiniband/hw/mlx5 2023-08-09T13:16:53.752 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/infiniband/hw/mlx5/mlx5_ib.ko 2023-08-09T13:16:53.752 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/infiniband/hw/mthca 2023-08-09T13:16:53.752 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/infiniband/hw/mthca/ib_mthca.ko 2023-08-09T13:16:53.752 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/infiniband/ulp 2023-08-09T13:16:53.752 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/infiniband/ulp/ipoib 2023-08-09T13:16:53.752 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/infiniband/ulp/ipoib/ib_ipoib.ko 2023-08-09T13:16:53.753 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/infiniband/ulp/iser 2023-08-09T13:16:53.753 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/infiniband/ulp/iser/ib_iser.ko 2023-08-09T13:16:53.753 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/infiniband/ulp/isert 2023-08-09T13:16:53.753 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/infiniband/ulp/isert/ib_isert.ko 2023-08-09T13:16:53.753 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/infiniband/ulp/srp 2023-08-09T13:16:53.753 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/infiniband/ulp/srp/ib_srp.ko 2023-08-09T13:16:53.753 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/infiniband/ulp/srpt 2023-08-09T13:16:53.753 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/infiniband/ulp/srpt/ib_srpt.ko 2023-08-09T13:16:53.754 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/input 2023-08-09T13:16:53.754 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/input/gameport 2023-08-09T13:16:53.754 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/input/gameport/emu10k1-gp.ko 2023-08-09T13:16:53.754 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/input/gameport/fm801-gp.ko 2023-08-09T13:16:53.754 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/input/gameport/gameport.ko 2023-08-09T13:16:53.754 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/input/gameport/lightning.ko 2023-08-09T13:16:53.754 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/input/gameport/ns558.ko 2023-08-09T13:16:53.755 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/input/joydev.ko 2023-08-09T13:16:53.755 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/input/keyboard 2023-08-09T13:16:53.755 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/input/keyboard/gpio_keys.ko 2023-08-09T13:16:53.755 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/input/mouse 2023-08-09T13:16:53.755 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/input/mouse/appletouch.ko 2023-08-09T13:16:53.755 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/input/mouse/bcm5974.ko 2023-08-09T13:16:53.755 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/input/mouse/cyapatp.ko 2023-08-09T13:16:53.755 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/input/mouse/elan_i2c.ko 2023-08-09T13:16:53.756 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/input/mouse/sermouse.ko 2023-08-09T13:16:53.756 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/input/mouse/synaptics_i2c.ko 2023-08-09T13:16:53.756 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/input/mouse/synaptics_usb.ko 2023-08-09T13:16:53.756 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/input/mouse/vsxxxaa.ko 2023-08-09T13:16:53.756 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/input/rmi4 2023-08-09T13:16:53.756 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/input/rmi4/rmi_core.ko 2023-08-09T13:16:53.756 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/input/serio 2023-08-09T13:16:53.756 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/input/serio/altera_ps2.ko 2023-08-09T13:16:53.757 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/input/serio/arc_ps2.ko 2023-08-09T13:16:53.757 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/input/serio/hyperv-keyboard.ko 2023-08-09T13:16:53.757 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/input/serio/serio_raw.ko 2023-08-09T13:16:53.757 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/input/sparse-keymap.ko 2023-08-09T13:16:53.757 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/iommu 2023-08-09T13:16:53.757 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/iommu/amd 2023-08-09T13:16:53.757 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/iommu/amd/iommu_v2.ko 2023-08-09T13:16:53.757 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/leds 2023-08-09T13:16:53.758 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/leds/leds-blinkm.ko 2023-08-09T13:16:53.758 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/leds/leds-lm3530.ko 2023-08-09T13:16:53.758 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/leds/leds-lp3944.ko 2023-08-09T13:16:53.758 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/leds/leds-ss4200.ko 2023-08-09T13:16:53.758 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/leds/trigger 2023-08-09T13:16:53.758 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/leds/trigger/ledtrig-audio.ko 2023-08-09T13:16:53.758 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/leds/trigger/ledtrig-backlight.ko 2023-08-09T13:16:53.758 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/leds/trigger/ledtrig-camera.ko 2023-08-09T13:16:53.759 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/leds/trigger/ledtrig-default-on.ko 2023-08-09T13:16:53.759 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/leds/trigger/ledtrig-heartbeat.ko 2023-08-09T13:16:53.759 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/leds/trigger/ledtrig-oneshot.ko 2023-08-09T13:16:53.759 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/leds/trigger/ledtrig-timer.ko 2023-08-09T13:16:53.759 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/leds/trigger/ledtrig-transient.ko 2023-08-09T13:16:53.759 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/md 2023-08-09T13:16:53.759 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/md/bcache 2023-08-09T13:16:53.759 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/md/bcache/bcache.ko 2023-08-09T13:16:53.760 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/md/dm-bio-prison.ko 2023-08-09T13:16:53.760 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/md/dm-cache-smq.ko 2023-08-09T13:16:53.760 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/md/dm-cache.ko 2023-08-09T13:16:53.760 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/md/dm-crypt.ko 2023-08-09T13:16:53.760 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/md/dm-delay.ko 2023-08-09T13:16:53.760 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/md/dm-flakey.ko 2023-08-09T13:16:53.760 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/md/dm-log-userspace.ko 2023-08-09T13:16:53.760 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/md/dm-multipath.ko 2023-08-09T13:16:53.761 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/md/dm-queue-length.ko 2023-08-09T13:16:53.761 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/md/dm-raid.ko 2023-08-09T13:16:53.761 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/md/dm-round-robin.ko 2023-08-09T13:16:53.761 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/md/dm-service-time.ko 2023-08-09T13:16:53.761 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/md/dm-switch.ko 2023-08-09T13:16:53.761 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/md/dm-thin-pool.ko 2023-08-09T13:16:53.761 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/md/dm-verity.ko 2023-08-09T13:16:53.762 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/md/faulty.ko 2023-08-09T13:16:53.762 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/md/linear.ko 2023-08-09T13:16:53.762 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/md/multipath.ko 2023-08-09T13:16:53.762 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/md/persistent-data 2023-08-09T13:16:53.762 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/md/persistent-data/dm-persistent-data.ko 2023-08-09T13:16:53.762 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/md/raid0.ko 2023-08-09T13:16:53.762 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/md/raid1.ko 2023-08-09T13:16:53.763 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/md/raid10.ko 2023-08-09T13:16:53.763 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/md/raid456.ko 2023-08-09T13:16:53.763 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/message 2023-08-09T13:16:53.763 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/message/fusion 2023-08-09T13:16:53.763 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/message/fusion/mptbase.ko 2023-08-09T13:16:53.763 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/message/fusion/mptctl.ko 2023-08-09T13:16:53.763 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/message/fusion/mptfc.ko 2023-08-09T13:16:53.764 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/message/fusion/mptlan.ko 2023-08-09T13:16:53.764 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/message/fusion/mptsas.ko 2023-08-09T13:16:53.764 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/message/fusion/mptscsih.ko 2023-08-09T13:16:53.764 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/message/fusion/mptspi.ko 2023-08-09T13:16:53.764 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/mfd 2023-08-09T13:16:53.764 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/mfd/lpc_ich.ko 2023-08-09T13:16:53.764 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/mfd/lpc_sch.ko 2023-08-09T13:16:53.764 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/mfd/mfd-core.ko 2023-08-09T13:16:53.765 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/mfd/sm501.ko 2023-08-09T13:16:53.765 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/mfd/viperboard.ko 2023-08-09T13:16:53.765 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/mfd/vx855.ko 2023-08-09T13:16:53.765 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/mfd/wl1273-core.ko 2023-08-09T13:16:53.765 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/misc 2023-08-09T13:16:53.765 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/misc/altera-stapl 2023-08-09T13:16:53.765 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/misc/altera-stapl/altera-stapl.ko 2023-08-09T13:16:53.766 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/misc/apds9802als.ko 2023-08-09T13:16:53.766 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/misc/apds990x.ko 2023-08-09T13:16:53.766 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/misc/bh1770glc.ko 2023-08-09T13:16:53.766 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/misc/cb710 2023-08-09T13:16:53.766 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/misc/cb710/cb710.ko 2023-08-09T13:16:53.766 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/misc/eeprom 2023-08-09T13:16:53.766 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/misc/eeprom/at24.ko 2023-08-09T13:16:53.766 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/misc/eeprom/eeprom.ko 2023-08-09T13:16:53.767 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/misc/eeprom/eeprom_93cx6.ko 2023-08-09T13:16:53.767 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/misc/eeprom/max6875.ko 2023-08-09T13:16:53.767 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/misc/enclosure.ko 2023-08-09T13:16:53.767 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/misc/hpilo.ko 2023-08-09T13:16:53.767 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/misc/isl29003.ko 2023-08-09T13:16:53.767 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/misc/isl29020.ko 2023-08-09T13:16:53.767 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/misc/lis3lv02d 2023-08-09T13:16:53.767 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/misc/lis3lv02d/lis3lv02d.ko 2023-08-09T13:16:53.768 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/misc/lis3lv02d/lis3lv02d_i2c.ko 2023-08-09T13:16:53.768 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/misc/mei 2023-08-09T13:16:53.768 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/misc/mei/mei-me.ko 2023-08-09T13:16:53.768 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/misc/mei/mei-txe.ko 2023-08-09T13:16:53.768 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/misc/mei/mei.ko 2023-08-09T13:16:53.768 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/misc/sgi-gru 2023-08-09T13:16:53.768 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/misc/sgi-gru/gru.ko 2023-08-09T13:16:53.768 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/misc/sgi-xp 2023-08-09T13:16:53.769 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/misc/sgi-xp/xp.ko 2023-08-09T13:16:53.769 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/misc/sgi-xp/xpc.ko 2023-08-09T13:16:53.769 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/misc/sgi-xp/xpnet.ko 2023-08-09T13:16:53.769 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/misc/tifm_7xx1.ko 2023-08-09T13:16:53.769 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/misc/tifm_core.ko 2023-08-09T13:16:53.769 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/misc/tsl2550.ko 2023-08-09T13:16:53.769 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/misc/vmw_balloon.ko 2023-08-09T13:16:53.770 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/misc/vmw_vmci 2023-08-09T13:16:53.770 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/misc/vmw_vmci/vmw_vmci.ko 2023-08-09T13:16:53.770 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/net 2023-08-09T13:16:53.770 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/net/bonding 2023-08-09T13:16:53.770 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/net/bonding/bonding.ko 2023-08-09T13:16:53.770 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/net/dsa 2023-08-09T13:16:53.770 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/net/dsa/b53 2023-08-09T13:16:53.770 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/net/dsa/b53/b53_common.ko 2023-08-09T13:16:53.771 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/net/dsa/bcm-sf2.ko 2023-08-09T13:16:53.771 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/net/dsa/mv88e6060.ko 2023-08-09T13:16:53.771 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/net/dsa/mv88e6xxx 2023-08-09T13:16:53.771 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/net/dsa/mv88e6xxx/mv88e6xxx.ko 2023-08-09T13:16:53.771 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/net/dummy.ko 2023-08-09T13:16:53.771 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/net/eql.ko 2023-08-09T13:16:53.771 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/net/ethernet 2023-08-09T13:16:53.771 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/net/ethernet/3com 2023-08-09T13:16:53.772 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/net/ethernet/3com/3c574_cs.ko 2023-08-09T13:16:53.772 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/net/ethernet/3com/3c589_cs.ko 2023-08-09T13:16:53.772 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/net/ethernet/3com/3c59x.ko 2023-08-09T13:16:53.772 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/net/ethernet/3com/typhoon.ko 2023-08-09T13:16:53.772 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/net/ethernet/8390 2023-08-09T13:16:53.772 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/net/ethernet/8390/8390.ko 2023-08-09T13:16:53.772 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/net/ethernet/8390/axnet_cs.ko 2023-08-09T13:16:53.772 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/net/ethernet/8390/ne2k-pci.ko 2023-08-09T13:16:53.773 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/net/ethernet/8390/pcnet_cs.ko 2023-08-09T13:16:53.773 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/net/ethernet/adaptec 2023-08-09T13:16:53.773 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/net/ethernet/adaptec/starfire.ko 2023-08-09T13:16:53.773 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/net/ethernet/agere 2023-08-09T13:16:53.773 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/net/ethernet/agere/et131x.ko 2023-08-09T13:16:53.773 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/net/ethernet/alteon 2023-08-09T13:16:53.773 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/net/ethernet/alteon/acenic.ko 2023-08-09T13:16:53.773 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/net/ethernet/altera 2023-08-09T13:16:53.774 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/net/ethernet/altera/altera_tse.ko 2023-08-09T13:16:53.774 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/net/ethernet/amd 2023-08-09T13:16:53.774 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/net/ethernet/amd/amd8111e.ko 2023-08-09T13:16:53.774 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/net/ethernet/amd/nmclan_cs.ko 2023-08-09T13:16:53.774 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/net/ethernet/amd/pcnet32.ko 2023-08-09T13:16:53.774 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/net/ethernet/atheros 2023-08-09T13:16:53.774 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/net/ethernet/atheros/alx 2023-08-09T13:16:53.774 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/net/ethernet/atheros/alx/alx.ko 2023-08-09T13:16:53.775 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/net/ethernet/atheros/atl1c 2023-08-09T13:16:53.775 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/net/ethernet/atheros/atl1c/atl1c.ko 2023-08-09T13:16:53.775 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/net/ethernet/atheros/atl1e 2023-08-09T13:16:53.775 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/net/ethernet/atheros/atl1e/atl1e.ko 2023-08-09T13:16:53.775 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/net/ethernet/atheros/atlx 2023-08-09T13:16:53.775 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/net/ethernet/atheros/atlx/atl1.ko 2023-08-09T13:16:53.775 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/net/ethernet/atheros/atlx/atl2.ko 2023-08-09T13:16:53.775 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/net/ethernet/broadcom 2023-08-09T13:16:53.776 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/net/ethernet/broadcom/b44.ko 2023-08-09T13:16:53.776 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/net/ethernet/broadcom/bnx2.ko 2023-08-09T13:16:53.776 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/net/ethernet/broadcom/bnx2x 2023-08-09T13:16:53.776 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/net/ethernet/broadcom/bnx2x/bnx2x.ko 2023-08-09T13:16:53.776 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/net/ethernet/broadcom/cnic.ko 2023-08-09T13:16:53.776 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/net/ethernet/broadcom/genet 2023-08-09T13:16:53.776 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/net/ethernet/broadcom/genet/genet.ko 2023-08-09T13:16:53.777 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/net/ethernet/broadcom/tg3.ko 2023-08-09T13:16:53.777 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/net/ethernet/brocade 2023-08-09T13:16:53.777 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/net/ethernet/brocade/bna 2023-08-09T13:16:53.777 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/net/ethernet/brocade/bna/bna.ko 2023-08-09T13:16:53.777 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/net/ethernet/chelsio 2023-08-09T13:16:53.777 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/net/ethernet/chelsio/cxgb 2023-08-09T13:16:53.777 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/net/ethernet/chelsio/cxgb/cxgb.ko 2023-08-09T13:16:53.777 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/net/ethernet/chelsio/cxgb3 2023-08-09T13:16:53.778 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/net/ethernet/chelsio/cxgb3/cxgb3.ko 2023-08-09T13:16:53.778 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/net/ethernet/chelsio/cxgb4 2023-08-09T13:16:53.778 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/net/ethernet/chelsio/cxgb4/cxgb4.ko 2023-08-09T13:16:53.778 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/net/ethernet/chelsio/cxgb4vf 2023-08-09T13:16:53.778 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/net/ethernet/chelsio/cxgb4vf/cxgb4vf.ko 2023-08-09T13:16:53.778 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/net/ethernet/chelsio/libcxgb 2023-08-09T13:16:53.778 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/net/ethernet/chelsio/libcxgb/libcxgb.ko 2023-08-09T13:16:53.778 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/net/ethernet/cisco 2023-08-09T13:16:53.779 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/net/ethernet/cisco/enic 2023-08-09T13:16:53.779 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/net/ethernet/cisco/enic/enic.ko 2023-08-09T13:16:53.779 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/net/ethernet/dec 2023-08-09T13:16:53.779 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/net/ethernet/dec/tulip 2023-08-09T13:16:53.779 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/net/ethernet/dec/tulip/de2104x.ko 2023-08-09T13:16:53.779 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/net/ethernet/dec/tulip/dmfe.ko 2023-08-09T13:16:53.779 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/net/ethernet/dec/tulip/tulip.ko 2023-08-09T13:16:53.779 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/net/ethernet/dec/tulip/uli526x.ko 2023-08-09T13:16:53.780 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/net/ethernet/dec/tulip/winbond-840.ko 2023-08-09T13:16:53.780 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/net/ethernet/dec/tulip/xircom_cb.ko 2023-08-09T13:16:53.780 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/net/ethernet/dlink 2023-08-09T13:16:53.780 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/net/ethernet/dlink/dl2k.ko 2023-08-09T13:16:53.780 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/net/ethernet/dlink/sundance.ko 2023-08-09T13:16:53.780 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/net/ethernet/dnet.ko 2023-08-09T13:16:53.780 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/net/ethernet/emulex 2023-08-09T13:16:53.780 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/net/ethernet/emulex/benet 2023-08-09T13:16:53.781 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/net/ethernet/emulex/benet/be2net.ko 2023-08-09T13:16:53.781 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/net/ethernet/ethoc.ko 2023-08-09T13:16:53.781 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/net/ethernet/fealnx.ko 2023-08-09T13:16:53.781 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/net/ethernet/intel 2023-08-09T13:16:53.781 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/net/ethernet/intel/e100.ko 2023-08-09T13:16:53.781 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/net/ethernet/intel/e1000 2023-08-09T13:16:53.781 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/net/ethernet/intel/e1000/e1000.ko 2023-08-09T13:16:53.782 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/net/ethernet/intel/e1000e 2023-08-09T13:16:53.782 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/net/ethernet/intel/e1000e/e1000e.ko 2023-08-09T13:16:53.782 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/net/ethernet/intel/fm10k 2023-08-09T13:16:53.782 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/net/ethernet/intel/fm10k/fm10k.ko 2023-08-09T13:16:53.782 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/net/ethernet/intel/i40e 2023-08-09T13:16:53.782 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/net/ethernet/intel/i40e/i40e.ko 2023-08-09T13:16:53.782 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/net/ethernet/intel/iavf 2023-08-09T13:16:53.782 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/net/ethernet/intel/iavf/iavf.ko 2023-08-09T13:16:53.783 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/net/ethernet/intel/igb 2023-08-09T13:16:53.783 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/net/ethernet/intel/igb/igb.ko 2023-08-09T13:16:53.783 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/net/ethernet/intel/igbvf 2023-08-09T13:16:53.783 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/net/ethernet/intel/igbvf/igbvf.ko 2023-08-09T13:16:53.783 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/net/ethernet/intel/ixgbe 2023-08-09T13:16:53.783 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/net/ethernet/intel/ixgbe/ixgbe.ko 2023-08-09T13:16:53.783 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/net/ethernet/intel/ixgbevf 2023-08-09T13:16:53.783 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/net/ethernet/intel/ixgbevf/ixgbevf.ko 2023-08-09T13:16:53.784 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/net/ethernet/jme.ko 2023-08-09T13:16:53.784 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/net/ethernet/marvell 2023-08-09T13:16:53.784 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/net/ethernet/marvell/mvmdio.ko 2023-08-09T13:16:53.784 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/net/ethernet/marvell/skge.ko 2023-08-09T13:16:53.784 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/net/ethernet/marvell/sky2.ko 2023-08-09T13:16:53.784 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/net/ethernet/mellanox 2023-08-09T13:16:53.784 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/net/ethernet/mellanox/mlx4 2023-08-09T13:16:53.784 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/net/ethernet/mellanox/mlx4/mlx4_core.ko 2023-08-09T13:16:53.785 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/net/ethernet/mellanox/mlx4/mlx4_en.ko 2023-08-09T13:16:53.785 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/net/ethernet/mellanox/mlx5 2023-08-09T13:16:53.785 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/net/ethernet/mellanox/mlx5/core 2023-08-09T13:16:53.785 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/net/ethernet/mellanox/mlx5/core/mlx5_core.ko 2023-08-09T13:16:53.785 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/net/ethernet/micrel 2023-08-09T13:16:53.785 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/net/ethernet/micrel/ksz884x.ko 2023-08-09T13:16:53.785 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/net/ethernet/myricom 2023-08-09T13:16:53.785 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/net/ethernet/myricom/myri10ge 2023-08-09T13:16:53.786 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/net/ethernet/myricom/myri10ge/myri10ge.ko 2023-08-09T13:16:53.786 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/net/ethernet/natsemi 2023-08-09T13:16:53.786 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/net/ethernet/natsemi/natsemi.ko 2023-08-09T13:16:53.786 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/net/ethernet/natsemi/ns83820.ko 2023-08-09T13:16:53.786 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/net/ethernet/neterion 2023-08-09T13:16:53.786 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/net/ethernet/neterion/s2io.ko 2023-08-09T13:16:53.786 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/net/ethernet/nvidia 2023-08-09T13:16:53.786 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/net/ethernet/nvidia/forcedeth.ko 2023-08-09T13:16:53.787 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/net/ethernet/packetengines 2023-08-09T13:16:53.787 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/net/ethernet/packetengines/hamachi.ko 2023-08-09T13:16:53.787 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/net/ethernet/packetengines/yellowfin.ko 2023-08-09T13:16:53.787 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/net/ethernet/qlogic 2023-08-09T13:16:53.787 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/net/ethernet/qlogic/netxen 2023-08-09T13:16:53.787 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/net/ethernet/qlogic/netxen/netxen_nic.ko 2023-08-09T13:16:53.787 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/net/ethernet/qlogic/qla3xxx.ko 2023-08-09T13:16:53.787 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/net/ethernet/qlogic/qlcnic 2023-08-09T13:16:53.788 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/net/ethernet/qlogic/qlcnic/qlcnic.ko 2023-08-09T13:16:53.788 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/net/ethernet/rdc 2023-08-09T13:16:53.788 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/net/ethernet/rdc/r6040.ko 2023-08-09T13:16:53.788 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/net/ethernet/realtek 2023-08-09T13:16:53.788 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/net/ethernet/realtek/8139cp.ko 2023-08-09T13:16:53.788 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/net/ethernet/realtek/8139too.ko 2023-08-09T13:16:53.788 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/net/ethernet/realtek/atp.ko 2023-08-09T13:16:53.789 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/net/ethernet/realtek/r8169.ko 2023-08-09T13:16:53.789 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/net/ethernet/rocker 2023-08-09T13:16:53.789 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/net/ethernet/rocker/rocker.ko 2023-08-09T13:16:53.789 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/net/ethernet/sfc 2023-08-09T13:16:53.789 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/net/ethernet/sfc/sfc.ko 2023-08-09T13:16:53.789 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/net/ethernet/silan 2023-08-09T13:16:53.789 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/net/ethernet/silan/sc92031.ko 2023-08-09T13:16:53.789 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/net/ethernet/sis 2023-08-09T13:16:53.790 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/net/ethernet/sis/sis190.ko 2023-08-09T13:16:53.790 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/net/ethernet/sis/sis900.ko 2023-08-09T13:16:53.790 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/net/ethernet/smsc 2023-08-09T13:16:53.790 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/net/ethernet/smsc/epic100.ko 2023-08-09T13:16:53.790 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/net/ethernet/smsc/smc91c92_cs.ko 2023-08-09T13:16:53.790 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/net/ethernet/smsc/smsc911x.ko 2023-08-09T13:16:53.790 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/net/ethernet/smsc/smsc9420.ko 2023-08-09T13:16:53.790 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/net/ethernet/stmicro 2023-08-09T13:16:53.791 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/net/ethernet/stmicro/stmmac 2023-08-09T13:16:53.791 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/net/ethernet/stmicro/stmmac/dwmac-intel.ko 2023-08-09T13:16:53.791 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/net/ethernet/stmicro/stmmac/stmmac.ko 2023-08-09T13:16:53.791 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/net/ethernet/sun 2023-08-09T13:16:53.791 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/net/ethernet/sun/cassini.ko 2023-08-09T13:16:53.791 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/net/ethernet/sun/niu.ko 2023-08-09T13:16:53.791 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/net/ethernet/sun/sungem.ko 2023-08-09T13:16:53.791 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/net/ethernet/sun/sunhme.ko 2023-08-09T13:16:53.792 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/net/ethernet/tehuti 2023-08-09T13:16:53.792 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/net/ethernet/tehuti/tehuti.ko 2023-08-09T13:16:53.792 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/net/ethernet/ti 2023-08-09T13:16:53.792 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/net/ethernet/ti/tlan.ko 2023-08-09T13:16:53.792 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/net/ethernet/via 2023-08-09T13:16:53.792 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/net/ethernet/via/via-rhine.ko 2023-08-09T13:16:53.792 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/net/ethernet/via/via-velocity.ko 2023-08-09T13:16:53.792 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/net/ethernet/wiznet 2023-08-09T13:16:53.793 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/net/ethernet/wiznet/w5100.ko 2023-08-09T13:16:53.793 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/net/ethernet/wiznet/w5300.ko 2023-08-09T13:16:53.793 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/net/ethernet/xircom 2023-08-09T13:16:53.793 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/net/ethernet/xircom/xirc2ps_cs.ko 2023-08-09T13:16:53.793 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/net/geneve.ko 2023-08-09T13:16:53.793 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/net/hyperv 2023-08-09T13:16:53.793 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/net/hyperv/hv_netvsc.ko 2023-08-09T13:16:53.793 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/net/ieee802154 2023-08-09T13:16:53.794 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/net/ieee802154/fakelb.ko 2023-08-09T13:16:53.794 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/net/ifb.ko 2023-08-09T13:16:53.794 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/net/ipvlan 2023-08-09T13:16:53.794 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/net/ipvlan/ipvlan.ko 2023-08-09T13:16:53.794 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/net/macvlan.ko 2023-08-09T13:16:53.794 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/net/macvtap.ko 2023-08-09T13:16:53.794 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/net/mdio 2023-08-09T13:16:53.794 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/net/mdio.ko 2023-08-09T13:16:53.795 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/net/mdio/mdio-bcm-unimac.ko 2023-08-09T13:16:53.795 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/net/mdio/mdio-bitbang.ko 2023-08-09T13:16:53.795 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/net/mdio/mdio-regmap.ko 2023-08-09T13:16:53.795 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/net/mii.ko 2023-08-09T13:16:53.795 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/net/net_failover.ko 2023-08-09T13:16:53.795 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/net/netconsole.ko 2023-08-09T13:16:53.795 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/net/nlmon.ko 2023-08-09T13:16:53.795 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/net/pcs 2023-08-09T13:16:53.796 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/net/pcs/pcs-lynx.ko 2023-08-09T13:16:53.796 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/net/pcs/pcs_xpcs.ko 2023-08-09T13:16:53.796 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/net/phy 2023-08-09T13:16:53.796 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/net/phy/amd.ko 2023-08-09T13:16:53.796 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/net/phy/bcm-phy-lib.ko 2023-08-09T13:16:53.796 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/net/phy/bcm-phy-ptp.ko 2023-08-09T13:16:53.796 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/net/phy/bcm7xxx.ko 2023-08-09T13:16:53.796 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/net/phy/bcm87xx.ko 2023-08-09T13:16:53.797 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/net/phy/broadcom.ko 2023-08-09T13:16:53.797 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/net/phy/cicada.ko 2023-08-09T13:16:53.797 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/net/phy/davicom.ko 2023-08-09T13:16:53.797 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/net/phy/dp83640.ko 2023-08-09T13:16:53.797 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/net/phy/et1011c.ko 2023-08-09T13:16:53.797 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/net/phy/icplus.ko 2023-08-09T13:16:53.797 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/net/phy/lxt.ko 2023-08-09T13:16:53.798 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/net/phy/marvell.ko 2023-08-09T13:16:53.798 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/net/phy/micrel.ko 2023-08-09T13:16:53.798 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/net/phy/national.ko 2023-08-09T13:16:53.798 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/net/phy/phylink.ko 2023-08-09T13:16:53.798 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/net/phy/qsemi.ko 2023-08-09T13:16:53.798 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/net/phy/realtek.ko 2023-08-09T13:16:53.798 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/net/phy/smsc.ko 2023-08-09T13:16:53.798 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/net/phy/ste10Xp.ko 2023-08-09T13:16:53.799 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/net/phy/vitesse.ko 2023-08-09T13:16:53.799 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/net/ppp 2023-08-09T13:16:53.799 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/net/ppp/bsd_comp.ko 2023-08-09T13:16:53.799 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/net/ppp/ppp_async.ko 2023-08-09T13:16:53.799 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/net/ppp/ppp_deflate.ko 2023-08-09T13:16:53.799 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/net/ppp/ppp_generic.ko 2023-08-09T13:16:53.799 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/net/ppp/ppp_mppe.ko 2023-08-09T13:16:53.799 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/net/ppp/ppp_synctty.ko 2023-08-09T13:16:53.800 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/net/ppp/pppoe.ko 2023-08-09T13:16:53.800 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/net/ppp/pppox.ko 2023-08-09T13:16:53.800 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/net/ppp/pptp.ko 2023-08-09T13:16:53.800 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/net/slip 2023-08-09T13:16:53.800 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/net/slip/slhc.ko 2023-08-09T13:16:53.800 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/net/slip/slip.ko 2023-08-09T13:16:53.800 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/net/sungem_phy.ko 2023-08-09T13:16:53.800 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/net/tap.ko 2023-08-09T13:16:53.801 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/net/team 2023-08-09T13:16:53.801 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/net/team/team.ko 2023-08-09T13:16:53.801 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/net/team/team_mode_activebackup.ko 2023-08-09T13:16:53.801 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/net/team/team_mode_broadcast.ko 2023-08-09T13:16:53.801 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/net/team/team_mode_loadbalance.ko 2023-08-09T13:16:53.801 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/net/team/team_mode_random.ko 2023-08-09T13:16:53.801 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/net/team/team_mode_roundrobin.ko 2023-08-09T13:16:53.801 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/net/tun.ko 2023-08-09T13:16:53.802 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/net/veth.ko 2023-08-09T13:16:53.802 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/net/virtio_net.ko 2023-08-09T13:16:53.802 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/net/vmxnet3 2023-08-09T13:16:53.802 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/net/vmxnet3/vmxnet3.ko 2023-08-09T13:16:53.802 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/net/vxlan 2023-08-09T13:16:53.802 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/net/vxlan/vxlan.ko 2023-08-09T13:16:53.802 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/net/xen-netback 2023-08-09T13:16:53.802 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/net/xen-netback/xen-netback.ko 2023-08-09T13:16:53.803 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/net/xen-netfront.ko 2023-08-09T13:16:53.803 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/ntb 2023-08-09T13:16:53.803 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/ntb/ntb.ko 2023-08-09T13:16:53.803 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/nvme 2023-08-09T13:16:53.803 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/nvme/host 2023-08-09T13:16:53.803 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/nvme/host/nvme-core.ko 2023-08-09T13:16:53.803 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/nvme/host/nvme.ko 2023-08-09T13:16:53.803 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/parport 2023-08-09T13:16:53.804 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/parport/parport.ko 2023-08-09T13:16:53.804 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/parport/parport_cs.ko 2023-08-09T13:16:53.804 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/parport/parport_pc.ko 2023-08-09T13:16:53.804 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/parport/parport_serial.ko 2023-08-09T13:16:53.804 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/pci 2023-08-09T13:16:53.804 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/pci/hotplug 2023-08-09T13:16:53.804 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/pci/hotplug/acpiphp_ibm.ko 2023-08-09T13:16:53.804 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/pci/pcie 2023-08-09T13:16:53.805 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/pci/pcie/aer_inject.ko 2023-08-09T13:16:53.805 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/pci/xen-pcifront.ko 2023-08-09T13:16:53.805 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/pcmcia 2023-08-09T13:16:53.805 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/pcmcia/i82092.ko 2023-08-09T13:16:53.805 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/pcmcia/pd6729.ko 2023-08-09T13:16:53.805 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/pcmcia/yenta_socket.ko 2023-08-09T13:16:53.805 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/pinctrl 2023-08-09T13:16:53.805 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/pinctrl/intel 2023-08-09T13:16:53.806 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/pinctrl/intel/pinctrl-cherryview.ko 2023-08-09T13:16:53.806 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/platform 2023-08-09T13:16:53.806 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/platform/x86 2023-08-09T13:16:53.806 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/platform/x86/acer-wmi.ko 2023-08-09T13:16:53.806 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/platform/x86/acerhdf.ko 2023-08-09T13:16:53.806 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/platform/x86/apple-gmux.ko 2023-08-09T13:16:53.806 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/platform/x86/asus-laptop.ko 2023-08-09T13:16:53.807 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/platform/x86/asus-nb-wmi.ko 2023-08-09T13:16:53.807 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/platform/x86/asus-wmi.ko 2023-08-09T13:16:53.807 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/platform/x86/classmate-laptop.ko 2023-08-09T13:16:53.807 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/platform/x86/eeepc-laptop.ko 2023-08-09T13:16:53.807 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/platform/x86/eeepc-wmi.ko 2023-08-09T13:16:53.807 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/platform/x86/fujitsu-laptop.ko 2023-08-09T13:16:53.807 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/platform/x86/fujitsu-tablet.ko 2023-08-09T13:16:53.807 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/platform/x86/hdaps.ko 2023-08-09T13:16:53.808 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/platform/x86/intel 2023-08-09T13:16:53.808 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/platform/x86/intel/intel-rst.ko 2023-08-09T13:16:53.808 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/platform/x86/intel_ips.ko 2023-08-09T13:16:53.808 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/platform/x86/msi-wmi.ko 2023-08-09T13:16:53.808 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/platform/x86/mxm-wmi.ko 2023-08-09T13:16:53.808 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/platform/x86/panasonic-laptop.ko 2023-08-09T13:16:53.808 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/platform/x86/samsung-laptop.ko 2023-08-09T13:16:53.808 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/platform/x86/samsung-q10.ko 2023-08-09T13:16:53.809 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/platform/x86/topstar-laptop.ko 2023-08-09T13:16:53.809 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/platform/x86/toshiba_bluetooth.ko 2023-08-09T13:16:53.809 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/platform/x86/toshiba_haps.ko 2023-08-09T13:16:53.809 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/platform/x86/wmi-bmof.ko 2023-08-09T13:16:53.809 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/platform/x86/wmi.ko 2023-08-09T13:16:53.809 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/powercap 2023-08-09T13:16:53.809 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/powercap/intel_rapl_common.ko 2023-08-09T13:16:53.809 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/powercap/intel_rapl_msr.ko 2023-08-09T13:16:53.810 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/pps 2023-08-09T13:16:53.810 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/pps/clients 2023-08-09T13:16:53.810 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/pps/clients/pps-gpio.ko 2023-08-09T13:16:53.810 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/pps/clients/pps-ldisc.ko 2023-08-09T13:16:53.810 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/pps/clients/pps_parport.ko 2023-08-09T13:16:53.810 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/pps/pps_core.ko 2023-08-09T13:16:53.810 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/ptp 2023-08-09T13:16:53.810 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/ptp/ptp.ko 2023-08-09T13:16:53.811 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/ptp/ptp_kvm.ko 2023-08-09T13:16:53.811 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/rtc 2023-08-09T13:16:53.811 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/rtc/rtc-bq32k.ko 2023-08-09T13:16:53.811 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/rtc/rtc-bq4802.ko 2023-08-09T13:16:53.811 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/rtc/rtc-ds1286.ko 2023-08-09T13:16:53.811 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/rtc/rtc-ds1307.ko 2023-08-09T13:16:53.811 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/rtc/rtc-ds1374.ko 2023-08-09T13:16:53.811 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/rtc/rtc-ds1511.ko 2023-08-09T13:16:53.812 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/rtc/rtc-ds1553.ko 2023-08-09T13:16:53.812 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/rtc/rtc-ds1672.ko 2023-08-09T13:16:53.812 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/rtc/rtc-ds1742.ko 2023-08-09T13:16:53.812 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/rtc/rtc-ds2404.ko 2023-08-09T13:16:53.812 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/rtc/rtc-ds3232.ko 2023-08-09T13:16:53.812 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/rtc/rtc-em3027.ko 2023-08-09T13:16:53.812 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/rtc/rtc-fm3130.ko 2023-08-09T13:16:53.812 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/rtc/rtc-isl12022.ko 2023-08-09T13:16:53.813 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/rtc/rtc-isl1208.ko 2023-08-09T13:16:53.813 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/rtc/rtc-m41t80.ko 2023-08-09T13:16:53.813 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/rtc/rtc-m48t35.ko 2023-08-09T13:16:53.813 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/rtc/rtc-m48t59.ko 2023-08-09T13:16:53.813 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/rtc/rtc-max6900.ko 2023-08-09T13:16:53.813 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/rtc/rtc-msm6242.ko 2023-08-09T13:16:53.813 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/rtc/rtc-pcf2127.ko 2023-08-09T13:16:53.813 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/rtc/rtc-pcf85063.ko 2023-08-09T13:16:53.814 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/rtc/rtc-pcf8523.ko 2023-08-09T13:16:53.814 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/rtc/rtc-pcf8563.ko 2023-08-09T13:16:53.814 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/rtc/rtc-pcf8583.ko 2023-08-09T13:16:53.814 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/rtc/rtc-rp5c01.ko 2023-08-09T13:16:53.814 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/rtc/rtc-rs5c372.ko 2023-08-09T13:16:53.814 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/rtc/rtc-rv3029c2.ko 2023-08-09T13:16:53.814 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/rtc/rtc-rx8025.ko 2023-08-09T13:16:53.815 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/rtc/rtc-rx8581.ko 2023-08-09T13:16:53.815 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/rtc/rtc-stk17ta8.ko 2023-08-09T13:16:53.815 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/rtc/rtc-x1205.ko 2023-08-09T13:16:53.815 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/scsi 2023-08-09T13:16:53.815 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/scsi/3w-9xxx.ko 2023-08-09T13:16:53.815 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/scsi/3w-sas.ko 2023-08-09T13:16:53.815 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/scsi/3w-xxxx.ko 2023-08-09T13:16:53.815 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/scsi/BusLogic.ko 2023-08-09T13:16:53.816 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/scsi/a100u2w.ko 2023-08-09T13:16:53.816 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/scsi/aacraid 2023-08-09T13:16:53.816 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/scsi/aacraid/aacraid.ko 2023-08-09T13:16:53.816 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/scsi/advansys.ko 2023-08-09T13:16:53.816 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/scsi/aic7xxx 2023-08-09T13:16:53.816 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/scsi/aic7xxx/aic79xx.ko 2023-08-09T13:16:53.816 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/scsi/aic7xxx/aic7xxx.ko 2023-08-09T13:16:53.817 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/scsi/am53c974.ko 2023-08-09T13:16:53.817 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/scsi/arcmsr 2023-08-09T13:16:53.817 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/scsi/arcmsr/arcmsr.ko 2023-08-09T13:16:53.817 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/scsi/atp870u.ko 2023-08-09T13:16:53.817 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/scsi/be2iscsi 2023-08-09T13:16:53.817 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/scsi/be2iscsi/be2iscsi.ko 2023-08-09T13:16:53.817 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/scsi/bfa 2023-08-09T13:16:53.817 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/scsi/bfa/bfa.ko 2023-08-09T13:16:53.818 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/scsi/bnx2fc 2023-08-09T13:16:53.818 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/scsi/bnx2fc/bnx2fc.ko 2023-08-09T13:16:53.818 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/scsi/bnx2i 2023-08-09T13:16:53.818 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/scsi/bnx2i/bnx2i.ko 2023-08-09T13:16:53.818 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/scsi/ch.ko 2023-08-09T13:16:53.818 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/scsi/csiostor 2023-08-09T13:16:53.818 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/scsi/csiostor/csiostor.ko 2023-08-09T13:16:53.818 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/scsi/cxgbi 2023-08-09T13:16:53.819 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/scsi/cxgbi/cxgb3i 2023-08-09T13:16:53.819 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/scsi/cxgbi/cxgb3i/cxgb3i.ko 2023-08-09T13:16:53.819 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/scsi/cxgbi/cxgb4i 2023-08-09T13:16:53.819 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/scsi/cxgbi/cxgb4i/cxgb4i.ko 2023-08-09T13:16:53.819 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/scsi/cxgbi/libcxgbi.ko 2023-08-09T13:16:53.819 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/scsi/dc395x.ko 2023-08-09T13:16:53.819 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/scsi/device_handler 2023-08-09T13:16:53.819 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/scsi/device_handler/scsi_dh_alua.ko 2023-08-09T13:16:53.820 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/scsi/device_handler/scsi_dh_emc.ko 2023-08-09T13:16:53.820 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/scsi/device_handler/scsi_dh_hp_sw.ko 2023-08-09T13:16:53.820 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/scsi/device_handler/scsi_dh_rdac.ko 2023-08-09T13:16:53.820 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/scsi/dmx3191d.ko 2023-08-09T13:16:53.820 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/scsi/esas2r 2023-08-09T13:16:53.820 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/scsi/esas2r/esas2r.ko 2023-08-09T13:16:53.820 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/scsi/esp_scsi.ko 2023-08-09T13:16:53.820 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/scsi/fcoe 2023-08-09T13:16:53.821 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/scsi/fcoe/fcoe.ko 2023-08-09T13:16:53.821 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/scsi/fcoe/libfcoe.ko 2023-08-09T13:16:53.821 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/scsi/fnic 2023-08-09T13:16:53.821 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/scsi/fnic/fnic.ko 2023-08-09T13:16:53.821 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/scsi/hpsa.ko 2023-08-09T13:16:53.821 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/scsi/hptiop.ko 2023-08-09T13:16:53.821 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/scsi/hv_storvsc.ko 2023-08-09T13:16:53.822 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/scsi/initio.ko 2023-08-09T13:16:53.822 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/scsi/ipr.ko 2023-08-09T13:16:53.822 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/scsi/ips.ko 2023-08-09T13:16:53.822 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/scsi/isci 2023-08-09T13:16:53.822 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/scsi/isci/isci.ko 2023-08-09T13:16:53.822 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/scsi/iscsi_boot_sysfs.ko 2023-08-09T13:16:53.822 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/scsi/iscsi_tcp.ko 2023-08-09T13:16:53.822 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/scsi/libfc 2023-08-09T13:16:53.823 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/scsi/libfc/libfc.ko 2023-08-09T13:16:53.823 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/scsi/libiscsi.ko 2023-08-09T13:16:53.823 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/scsi/libiscsi_tcp.ko 2023-08-09T13:16:53.823 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/scsi/libsas 2023-08-09T13:16:53.823 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/scsi/libsas/libsas.ko 2023-08-09T13:16:53.823 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/scsi/lpfc 2023-08-09T13:16:53.823 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/scsi/lpfc/lpfc.ko 2023-08-09T13:16:53.823 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/scsi/megaraid 2023-08-09T13:16:53.824 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/scsi/megaraid.ko 2023-08-09T13:16:53.824 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/scsi/megaraid/megaraid_mbox.ko 2023-08-09T13:16:53.824 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/scsi/megaraid/megaraid_mm.ko 2023-08-09T13:16:53.824 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/scsi/megaraid/megaraid_sas.ko 2023-08-09T13:16:53.824 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/scsi/mpt3sas 2023-08-09T13:16:53.824 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/scsi/mpt3sas/mpt3sas.ko 2023-08-09T13:16:53.824 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/scsi/mvsas 2023-08-09T13:16:53.824 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/scsi/mvsas/mvsas.ko 2023-08-09T13:16:53.825 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/scsi/mvumi.ko 2023-08-09T13:16:53.825 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/scsi/pm8001 2023-08-09T13:16:53.825 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/scsi/pm8001/pm80xx.ko 2023-08-09T13:16:53.825 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/scsi/pmcraid.ko 2023-08-09T13:16:53.825 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/scsi/qla1280.ko 2023-08-09T13:16:53.825 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/scsi/qla2xxx 2023-08-09T13:16:53.825 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/scsi/qla2xxx/qla2xxx.ko 2023-08-09T13:16:53.825 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/scsi/qla2xxx/tcm_qla2xxx.ko 2023-08-09T13:16:53.826 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/scsi/qla4xxx 2023-08-09T13:16:53.826 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/scsi/qla4xxx/qla4xxx.ko 2023-08-09T13:16:53.826 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/scsi/raid_class.ko 2023-08-09T13:16:53.826 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/scsi/scsi_debug.ko 2023-08-09T13:16:53.826 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/scsi/scsi_transport_fc.ko 2023-08-09T13:16:53.826 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/scsi/scsi_transport_iscsi.ko 2023-08-09T13:16:53.826 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/scsi/scsi_transport_sas.ko 2023-08-09T13:16:53.826 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/scsi/scsi_transport_spi.ko 2023-08-09T13:16:53.827 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/scsi/scsi_transport_srp.ko 2023-08-09T13:16:53.827 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/scsi/ses.ko 2023-08-09T13:16:53.827 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/scsi/st.ko 2023-08-09T13:16:53.827 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/scsi/stex.ko 2023-08-09T13:16:53.827 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/scsi/sym53c8xx_2 2023-08-09T13:16:53.827 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/scsi/sym53c8xx_2/sym53c8xx.ko 2023-08-09T13:16:53.827 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/scsi/virtio_scsi.ko 2023-08-09T13:16:53.827 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/scsi/vmw_pvscsi.ko 2023-08-09T13:16:53.828 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/scsi/wd719x.ko 2023-08-09T13:16:53.828 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/ssb 2023-08-09T13:16:53.828 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/ssb/ssb.ko 2023-08-09T13:16:53.828 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/target 2023-08-09T13:16:53.828 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/target/iscsi 2023-08-09T13:16:53.828 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/target/iscsi/iscsi_target_mod.ko 2023-08-09T13:16:53.828 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/target/loopback 2023-08-09T13:16:53.828 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/target/loopback/tcm_loop.ko 2023-08-09T13:16:53.829 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/target/target_core_file.ko 2023-08-09T13:16:53.829 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/target/target_core_iblock.ko 2023-08-09T13:16:53.829 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/target/target_core_mod.ko 2023-08-09T13:16:53.829 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/target/target_core_pscsi.ko 2023-08-09T13:16:53.829 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/target/target_core_user.ko 2023-08-09T13:16:53.829 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/target/tcm_fc 2023-08-09T13:16:53.829 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/target/tcm_fc/tcm_fc.ko 2023-08-09T13:16:53.829 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/thermal 2023-08-09T13:16:53.830 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/thermal/intel 2023-08-09T13:16:53.830 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/thermal/intel/int340x_thermal 2023-08-09T13:16:53.830 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/thermal/intel/int340x_thermal/acpi_thermal_rel.ko 2023-08-09T13:16:53.830 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/thermal/intel/int340x_thermal/int3400_thermal.ko 2023-08-09T13:16:53.830 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/thermal/intel/int340x_thermal/int3401_thermal.ko 2023-08-09T13:16:53.830 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/thermal/intel/int340x_thermal/int3402_thermal.ko 2023-08-09T13:16:53.830 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/thermal/intel/int340x_thermal/int3403_thermal.ko 2023-08-09T13:16:53.831 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/thermal/intel/int340x_thermal/int340x_thermal_zone.ko 2023-08-09T13:16:53.831 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_device.ko 2023-08-09T13:16:53.831 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_device_pci.ko 2023-08-09T13:16:53.831 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_device_pci_legacy.ko 2023-08-09T13:16:53.831 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_mbox.ko 2023-08-09T13:16:53.831 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_rapl.ko 2023-08-09T13:16:53.831 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_rfim.ko 2023-08-09T13:16:53.831 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/thermal/intel/intel_soc_dts_iosf.ko 2023-08-09T13:16:53.832 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/thermal/intel/intel_soc_dts_thermal.ko 2023-08-09T13:16:53.832 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/thermal/intel/x86_pkg_temp_thermal.ko 2023-08-09T13:16:53.832 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/tty 2023-08-09T13:16:53.832 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/tty/ipwireless 2023-08-09T13:16:53.832 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/tty/ipwireless/ipwireless.ko 2023-08-09T13:16:53.832 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/tty/n_gsm.ko 2023-08-09T13:16:53.832 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/tty/n_hdlc.ko 2023-08-09T13:16:53.832 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/tty/nozomi.ko 2023-08-09T13:16:53.833 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/tty/serial 2023-08-09T13:16:53.833 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/tty/serial/8250 2023-08-09T13:16:53.833 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/tty/serial/8250/serial_cs.ko 2023-08-09T13:16:53.833 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/tty/serial/arc_uart.ko 2023-08-09T13:16:53.833 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/tty/serial/jsm 2023-08-09T13:16:53.833 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/tty/serial/jsm/jsm.ko 2023-08-09T13:16:53.833 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/tty/synclink_gt.ko 2023-08-09T13:16:53.833 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/ufs 2023-08-09T13:16:53.834 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/ufs/core 2023-08-09T13:16:53.834 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/ufs/core/ufshcd-core.ko 2023-08-09T13:16:53.834 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/ufs/host 2023-08-09T13:16:53.834 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/ufs/host/ufshcd-pci.ko 2023-08-09T13:16:53.834 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/uio 2023-08-09T13:16:53.834 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/uio/uio.ko 2023-08-09T13:16:53.834 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/uio/uio_aec.ko 2023-08-09T13:16:53.834 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/uio/uio_cif.ko 2023-08-09T13:16:53.835 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/uio/uio_pci_generic.ko 2023-08-09T13:16:53.835 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/uio/uio_sercos3.ko 2023-08-09T13:16:53.835 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/usb 2023-08-09T13:16:53.835 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/usb/class 2023-08-09T13:16:53.835 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/usb/class/cdc-acm.ko 2023-08-09T13:16:53.835 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/usb/class/cdc-wdm.ko 2023-08-09T13:16:53.835 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/usb/class/usblp.ko 2023-08-09T13:16:53.835 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/usb/class/usbtmc.ko 2023-08-09T13:16:53.836 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/usb/host 2023-08-09T13:16:53.836 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/usb/host/sl811-hcd.ko 2023-08-09T13:16:53.836 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/usb/image 2023-08-09T13:16:53.836 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/usb/image/mdc800.ko 2023-08-09T13:16:53.836 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/usb/image/microtek.ko 2023-08-09T13:16:53.836 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/usb/misc 2023-08-09T13:16:53.836 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/usb/misc/adutux.ko 2023-08-09T13:16:53.837 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/usb/misc/appledisplay.ko 2023-08-09T13:16:53.837 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/usb/misc/emi26.ko 2023-08-09T13:16:53.837 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/usb/misc/emi62.ko 2023-08-09T13:16:53.837 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/usb/misc/ezusb.ko 2023-08-09T13:16:53.837 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/usb/misc/idmouse.ko 2023-08-09T13:16:53.837 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/usb/misc/iowarrior.ko 2023-08-09T13:16:53.837 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/usb/misc/isight_firmware.ko 2023-08-09T13:16:53.837 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/usb/misc/ldusb.ko 2023-08-09T13:16:53.838 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/usb/misc/legousbtower.ko 2023-08-09T13:16:53.838 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/usb/misc/sisusbvga 2023-08-09T13:16:53.838 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/usb/misc/sisusbvga/sisusbvga.ko 2023-08-09T13:16:53.838 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/usb/misc/trancevibrator.ko 2023-08-09T13:16:53.838 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/usb/misc/usb3503.ko 2023-08-09T13:16:53.838 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/usb/misc/usblcd.ko 2023-08-09T13:16:53.838 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/usb/misc/usbsevseg.ko 2023-08-09T13:16:53.838 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/usb/misc/uss720.ko 2023-08-09T13:16:53.839 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/usb/misc/yurex.ko 2023-08-09T13:16:53.839 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/usb/phy 2023-08-09T13:16:53.839 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/usb/phy/phy-generic.ko 2023-08-09T13:16:53.839 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/usb/serial 2023-08-09T13:16:53.839 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/usb/serial/aircable.ko 2023-08-09T13:16:53.839 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/usb/serial/ark3116.ko 2023-08-09T13:16:53.840 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/usb/serial/belkin_sa.ko 2023-08-09T13:16:53.840 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/usb/serial/ch341.ko 2023-08-09T13:16:53.840 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/usb/serial/cp210x.ko 2023-08-09T13:16:53.840 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/usb/serial/cyberjack.ko 2023-08-09T13:16:53.840 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/usb/serial/cypress_m8.ko 2023-08-09T13:16:53.840 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/usb/serial/digi_acceleport.ko 2023-08-09T13:16:53.840 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/usb/serial/empeg.ko 2023-08-09T13:16:53.841 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/usb/serial/ftdi_sio.ko 2023-08-09T13:16:53.841 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/usb/serial/garmin_gps.ko 2023-08-09T13:16:53.841 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/usb/serial/io_edgeport.ko 2023-08-09T13:16:53.841 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/usb/serial/io_ti.ko 2023-08-09T13:16:53.841 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/usb/serial/ipaq.ko 2023-08-09T13:16:53.841 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/usb/serial/ipw.ko 2023-08-09T13:16:53.841 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/usb/serial/ir-usb.ko 2023-08-09T13:16:53.841 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/usb/serial/iuu_phoenix.ko 2023-08-09T13:16:53.842 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/usb/serial/keyspan.ko 2023-08-09T13:16:53.842 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/usb/serial/keyspan_pda.ko 2023-08-09T13:16:53.842 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/usb/serial/kl5kusb105.ko 2023-08-09T13:16:53.842 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/usb/serial/kobil_sct.ko 2023-08-09T13:16:53.842 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/usb/serial/mct_u232.ko 2023-08-09T13:16:53.842 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/usb/serial/mos7720.ko 2023-08-09T13:16:53.842 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/usb/serial/mos7840.ko 2023-08-09T13:16:53.842 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/usb/serial/navman.ko 2023-08-09T13:16:53.843 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/usb/serial/omninet.ko 2023-08-09T13:16:53.843 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/usb/serial/opticon.ko 2023-08-09T13:16:53.843 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/usb/serial/option.ko 2023-08-09T13:16:53.843 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/usb/serial/oti6858.ko 2023-08-09T13:16:53.843 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/usb/serial/pl2303.ko 2023-08-09T13:16:53.843 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/usb/serial/qcaux.ko 2023-08-09T13:16:53.843 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/usb/serial/qcserial.ko 2023-08-09T13:16:53.843 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/usb/serial/quatech2.ko 2023-08-09T13:16:53.844 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/usb/serial/safe_serial.ko 2023-08-09T13:16:53.844 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/usb/serial/sierra.ko 2023-08-09T13:16:53.844 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/usb/serial/spcp8x5.ko 2023-08-09T13:16:53.844 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/usb/serial/ssu100.ko 2023-08-09T13:16:53.844 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/usb/serial/symbolserial.ko 2023-08-09T13:16:53.844 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/usb/serial/ti_usb_3410_5052.ko 2023-08-09T13:16:53.844 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/usb/serial/usb-serial-simple.ko 2023-08-09T13:16:53.845 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/usb/serial/usb_debug.ko 2023-08-09T13:16:53.845 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/usb/serial/usb_wwan.ko 2023-08-09T13:16:53.845 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/usb/serial/visor.ko 2023-08-09T13:16:53.845 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/usb/serial/whiteheat.ko 2023-08-09T13:16:53.845 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/usb/serial/xsens_mt.ko 2023-08-09T13:16:53.845 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/usb/storage 2023-08-09T13:16:53.845 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/usb/storage/uas.ko 2023-08-09T13:16:53.845 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/usb/storage/ums-alauda.ko 2023-08-09T13:16:53.846 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/usb/storage/ums-cypress.ko 2023-08-09T13:16:53.846 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/usb/storage/ums-datafab.ko 2023-08-09T13:16:53.846 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/usb/storage/ums-eneub6250.ko 2023-08-09T13:16:53.846 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/usb/storage/ums-freecom.ko 2023-08-09T13:16:53.846 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/usb/storage/ums-isd200.ko 2023-08-09T13:16:53.846 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/usb/storage/ums-jumpshot.ko 2023-08-09T13:16:53.846 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/usb/storage/ums-karma.ko 2023-08-09T13:16:53.846 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/usb/storage/ums-onetouch.ko 2023-08-09T13:16:53.847 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/usb/storage/ums-realtek.ko 2023-08-09T13:16:53.847 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/usb/storage/ums-sddr09.ko 2023-08-09T13:16:53.847 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/usb/storage/ums-sddr55.ko 2023-08-09T13:16:53.847 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/usb/storage/ums-usbat.ko 2023-08-09T13:16:53.847 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/usb/storage/usb-storage.ko 2023-08-09T13:16:53.847 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/usb/usbip 2023-08-09T13:16:53.847 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/usb/usbip/usbip-core.ko 2023-08-09T13:16:53.847 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/usb/usbip/usbip-host.ko 2023-08-09T13:16:53.848 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/usb/usbip/vhci-hcd.ko 2023-08-09T13:16:53.848 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/vfio 2023-08-09T13:16:53.848 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/vfio/pci 2023-08-09T13:16:53.848 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/vfio/pci/vfio-pci-core.ko 2023-08-09T13:16:53.848 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/vfio/pci/vfio-pci.ko 2023-08-09T13:16:53.848 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/vfio/vfio.ko 2023-08-09T13:16:53.848 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/vfio/vfio_iommu_type1.ko 2023-08-09T13:16:53.848 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/vhost 2023-08-09T13:16:53.849 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/vhost/vhost.ko 2023-08-09T13:16:53.849 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/vhost/vhost_iotlb.ko 2023-08-09T13:16:53.849 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/vhost/vhost_net.ko 2023-08-09T13:16:53.849 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/vhost/vhost_scsi.ko 2023-08-09T13:16:53.849 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/video 2023-08-09T13:16:53.849 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/video/backlight 2023-08-09T13:16:53.849 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/video/backlight/apple_bl.ko 2023-08-09T13:16:53.849 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/video/backlight/lcd.ko 2023-08-09T13:16:53.850 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/video/backlight/platform_lcd.ko 2023-08-09T13:16:53.850 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/virtio 2023-08-09T13:16:53.850 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/virtio/virtio.ko 2023-08-09T13:16:53.850 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/virtio/virtio_balloon.ko 2023-08-09T13:16:53.850 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/virtio/virtio_mmio.ko 2023-08-09T13:16:53.850 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/virtio/virtio_pci.ko 2023-08-09T13:16:53.850 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/virtio/virtio_pci_legacy_dev.ko 2023-08-09T13:16:53.850 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/virtio/virtio_pci_modern_dev.ko 2023-08-09T13:16:53.851 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/virtio/virtio_ring.ko 2023-08-09T13:16:53.851 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/w1 2023-08-09T13:16:53.851 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/w1/masters 2023-08-09T13:16:53.851 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/w1/masters/ds2482.ko 2023-08-09T13:16:53.851 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/w1/masters/ds2490.ko 2023-08-09T13:16:53.851 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/w1/slaves 2023-08-09T13:16:53.851 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/w1/slaves/w1_ds2406.ko 2023-08-09T13:16:53.852 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/w1/slaves/w1_ds2408.ko 2023-08-09T13:16:53.852 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/w1/slaves/w1_ds2413.ko 2023-08-09T13:16:53.852 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/w1/slaves/w1_ds2423.ko 2023-08-09T13:16:53.852 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/w1/slaves/w1_ds2431.ko 2023-08-09T13:16:53.852 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/w1/slaves/w1_ds2433.ko 2023-08-09T13:16:53.852 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/w1/slaves/w1_ds2780.ko 2023-08-09T13:16:53.852 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/w1/slaves/w1_ds2781.ko 2023-08-09T13:16:53.852 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/w1/slaves/w1_ds28e04.ko 2023-08-09T13:16:53.853 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/w1/slaves/w1_smem.ko 2023-08-09T13:16:53.853 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/w1/slaves/w1_therm.ko 2023-08-09T13:16:53.853 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/w1/wire.ko 2023-08-09T13:16:53.853 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/watchdog 2023-08-09T13:16:53.853 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/watchdog/alim1535_wdt.ko 2023-08-09T13:16:53.853 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/watchdog/alim7101_wdt.ko 2023-08-09T13:16:53.853 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/watchdog/f71808e_wdt.ko 2023-08-09T13:16:53.853 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/watchdog/hpwdt.ko 2023-08-09T13:16:53.854 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/watchdog/i6300esb.ko 2023-08-09T13:16:53.854 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/watchdog/iTCO_vendor_support.ko 2023-08-09T13:16:53.854 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/watchdog/iTCO_wdt.ko 2023-08-09T13:16:53.854 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/watchdog/ib700wdt.ko 2023-08-09T13:16:53.854 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/watchdog/ibmasr.ko 2023-08-09T13:16:53.854 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/watchdog/ie6xx_wdt.ko 2023-08-09T13:16:53.854 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/watchdog/it8712f_wdt.ko 2023-08-09T13:16:53.854 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/watchdog/it87_wdt.ko 2023-08-09T13:16:53.855 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/watchdog/machzwd.ko 2023-08-09T13:16:53.855 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/watchdog/nv_tco.ko 2023-08-09T13:16:53.855 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/watchdog/pcwd_pci.ko 2023-08-09T13:16:53.855 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/watchdog/pcwd_usb.ko 2023-08-09T13:16:53.855 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/watchdog/sbc_fitpc2_wdt.ko 2023-08-09T13:16:53.855 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/watchdog/sch311x_wdt.ko 2023-08-09T13:16:53.855 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/watchdog/softdog.ko 2023-08-09T13:16:53.855 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/watchdog/sp5100_tco.ko 2023-08-09T13:16:53.856 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/watchdog/via_wdt.ko 2023-08-09T13:16:53.856 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/watchdog/w83627hf_wdt.ko 2023-08-09T13:16:53.856 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/watchdog/w83877f_wdt.ko 2023-08-09T13:16:53.856 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/watchdog/w83977f_wdt.ko 2023-08-09T13:16:53.856 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/watchdog/wdt_pci.ko 2023-08-09T13:16:53.856 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/watchdog/xen_wdt.ko 2023-08-09T13:16:53.856 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/xen 2023-08-09T13:16:53.856 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/xen/xen-acpi-processor.ko 2023-08-09T13:16:53.857 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/xen/xen-evtchn.ko 2023-08-09T13:16:53.857 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/xen/xen-gntalloc.ko 2023-08-09T13:16:53.857 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/xen/xen-gntdev.ko 2023-08-09T13:16:53.857 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/xen/xen-pciback 2023-08-09T13:16:53.857 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/xen/xen-pciback/xen-pciback.ko 2023-08-09T13:16:53.857 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/xen/xen-privcmd.ko 2023-08-09T13:16:53.857 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/xen/xenfs 2023-08-09T13:16:53.857 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/xen/xenfs/xenfs.ko 2023-08-09T13:16:53.858 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/fs 2023-08-09T13:16:53.858 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/fs/9p 2023-08-09T13:16:53.858 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/fs/9p/9p.ko 2023-08-09T13:16:53.858 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/fs/affs 2023-08-09T13:16:53.858 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/fs/affs/affs.ko 2023-08-09T13:16:53.858 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/fs/befs 2023-08-09T13:16:53.858 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/fs/befs/befs.ko 2023-08-09T13:16:53.858 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/fs/binfmt_misc.ko 2023-08-09T13:16:53.859 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/fs/btrfs 2023-08-09T13:16:53.859 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/fs/btrfs/btrfs.ko 2023-08-09T13:16:53.859 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/fs/cachefiles 2023-08-09T13:16:53.859 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/fs/cachefiles/cachefiles.ko 2023-08-09T13:16:53.859 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/fs/ceph 2023-08-09T13:16:53.859 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/fs/ceph/ceph.ko 2023-08-09T13:16:53.859 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/fs/coda 2023-08-09T13:16:53.859 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/fs/coda/coda.ko 2023-08-09T13:16:53.860 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/fs/cramfs 2023-08-09T13:16:53.860 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/fs/cramfs/cramfs.ko 2023-08-09T13:16:53.860 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/fs/dlm 2023-08-09T13:16:53.860 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/fs/dlm/dlm.ko 2023-08-09T13:16:53.860 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/fs/ecryptfs 2023-08-09T13:16:53.860 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/fs/ecryptfs/ecryptfs.ko 2023-08-09T13:16:53.860 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/fs/f2fs 2023-08-09T13:16:53.860 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/fs/f2fs/f2fs.ko 2023-08-09T13:16:53.861 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/fs/fat 2023-08-09T13:16:53.861 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/fs/fat/fat.ko 2023-08-09T13:16:53.861 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/fs/fat/msdos.ko 2023-08-09T13:16:53.861 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/fs/fat/vfat.ko 2023-08-09T13:16:53.861 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/fs/fscache 2023-08-09T13:16:53.861 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/fs/fscache/fscache.ko 2023-08-09T13:16:53.861 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/fs/fuse 2023-08-09T13:16:53.862 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/fs/fuse/cuse.ko 2023-08-09T13:16:53.862 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/fs/fuse/fuse.ko 2023-08-09T13:16:53.862 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/fs/gfs2 2023-08-09T13:16:53.862 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/fs/gfs2/gfs2.ko 2023-08-09T13:16:53.862 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/fs/hfs 2023-08-09T13:16:53.862 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/fs/hfs/hfs.ko 2023-08-09T13:16:53.862 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/fs/hfsplus 2023-08-09T13:16:53.862 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/fs/hfsplus/hfsplus.ko 2023-08-09T13:16:53.863 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/fs/isofs 2023-08-09T13:16:53.863 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/fs/isofs/isofs.ko 2023-08-09T13:16:53.863 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/fs/jfs 2023-08-09T13:16:53.863 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/fs/jfs/jfs.ko 2023-08-09T13:16:53.863 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/fs/lockd 2023-08-09T13:16:53.863 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/fs/lockd/lockd.ko 2023-08-09T13:16:53.863 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/fs/minix 2023-08-09T13:16:53.863 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/fs/minix/minix.ko 2023-08-09T13:16:53.864 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/fs/netfs 2023-08-09T13:16:53.864 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/fs/netfs/netfs.ko 2023-08-09T13:16:53.864 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/fs/nfs 2023-08-09T13:16:53.864 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/fs/nfs/blocklayout 2023-08-09T13:16:53.864 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/fs/nfs/blocklayout/blocklayoutdriver.ko 2023-08-09T13:16:53.864 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/fs/nfs/filelayout 2023-08-09T13:16:53.864 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/fs/nfs/filelayout/nfs_layout_nfsv41_files.ko 2023-08-09T13:16:53.864 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/fs/nfs/flexfilelayout 2023-08-09T13:16:53.865 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/fs/nfs/flexfilelayout/nfs_layout_flexfiles.ko 2023-08-09T13:16:53.865 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/fs/nfs/nfs.ko 2023-08-09T13:16:53.865 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/fs/nfs/nfsv3.ko 2023-08-09T13:16:53.865 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/fs/nfs/nfsv4.ko 2023-08-09T13:16:53.865 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/fs/nfs_common 2023-08-09T13:16:53.865 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/fs/nfs_common/grace.ko 2023-08-09T13:16:53.865 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/fs/nfs_common/nfs_acl.ko 2023-08-09T13:16:53.865 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/fs/nfsd 2023-08-09T13:16:53.866 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/fs/nfsd/nfsd.ko 2023-08-09T13:16:53.866 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/fs/nilfs2 2023-08-09T13:16:53.866 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/fs/nilfs2/nilfs2.ko 2023-08-09T13:16:53.866 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/fs/nls 2023-08-09T13:16:53.866 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/fs/nls/mac-celtic.ko 2023-08-09T13:16:53.866 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/fs/nls/mac-centeuro.ko 2023-08-09T13:16:53.866 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/fs/nls/mac-croatian.ko 2023-08-09T13:16:53.866 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/fs/nls/mac-cyrillic.ko 2023-08-09T13:16:53.867 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/fs/nls/mac-gaelic.ko 2023-08-09T13:16:53.867 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/fs/nls/mac-greek.ko 2023-08-09T13:16:53.867 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/fs/nls/mac-iceland.ko 2023-08-09T13:16:53.867 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/fs/nls/mac-inuit.ko 2023-08-09T13:16:53.867 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/fs/nls/mac-roman.ko 2023-08-09T13:16:53.867 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/fs/nls/mac-romanian.ko 2023-08-09T13:16:53.867 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/fs/nls/mac-turkish.ko 2023-08-09T13:16:53.867 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/fs/nls/nls_cp1250.ko 2023-08-09T13:16:53.868 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/fs/nls/nls_cp1251.ko 2023-08-09T13:16:53.868 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/fs/nls/nls_cp1255.ko 2023-08-09T13:16:53.868 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/fs/nls/nls_cp737.ko 2023-08-09T13:16:53.868 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/fs/nls/nls_cp775.ko 2023-08-09T13:16:53.868 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/fs/nls/nls_cp850.ko 2023-08-09T13:16:53.868 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/fs/nls/nls_cp852.ko 2023-08-09T13:16:53.868 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/fs/nls/nls_cp855.ko 2023-08-09T13:16:53.868 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/fs/nls/nls_cp857.ko 2023-08-09T13:16:53.869 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/fs/nls/nls_cp860.ko 2023-08-09T13:16:53.869 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/fs/nls/nls_cp861.ko 2023-08-09T13:16:53.869 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/fs/nls/nls_cp862.ko 2023-08-09T13:16:53.869 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/fs/nls/nls_cp863.ko 2023-08-09T13:16:53.869 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/fs/nls/nls_cp864.ko 2023-08-09T13:16:53.869 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/fs/nls/nls_cp865.ko 2023-08-09T13:16:53.869 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/fs/nls/nls_cp866.ko 2023-08-09T13:16:53.869 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/fs/nls/nls_cp869.ko 2023-08-09T13:16:53.870 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/fs/nls/nls_cp874.ko 2023-08-09T13:16:53.870 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/fs/nls/nls_cp932.ko 2023-08-09T13:16:53.870 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/fs/nls/nls_cp936.ko 2023-08-09T13:16:53.870 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/fs/nls/nls_cp949.ko 2023-08-09T13:16:53.870 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/fs/nls/nls_cp950.ko 2023-08-09T13:16:53.870 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/fs/nls/nls_euc-jp.ko 2023-08-09T13:16:53.870 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/fs/nls/nls_iso8859-1.ko 2023-08-09T13:16:53.870 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/fs/nls/nls_iso8859-13.ko 2023-08-09T13:16:53.871 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/fs/nls/nls_iso8859-14.ko 2023-08-09T13:16:53.871 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/fs/nls/nls_iso8859-15.ko 2023-08-09T13:16:53.871 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/fs/nls/nls_iso8859-2.ko 2023-08-09T13:16:53.871 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/fs/nls/nls_iso8859-3.ko 2023-08-09T13:16:53.871 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/fs/nls/nls_iso8859-4.ko 2023-08-09T13:16:53.871 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/fs/nls/nls_iso8859-5.ko 2023-08-09T13:16:53.871 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/fs/nls/nls_iso8859-6.ko 2023-08-09T13:16:53.872 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/fs/nls/nls_iso8859-7.ko 2023-08-09T13:16:53.872 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/fs/nls/nls_iso8859-9.ko 2023-08-09T13:16:53.872 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/fs/nls/nls_koi8-r.ko 2023-08-09T13:16:53.872 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/fs/nls/nls_koi8-ru.ko 2023-08-09T13:16:53.872 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/fs/nls/nls_koi8-u.ko 2023-08-09T13:16:53.872 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/fs/nls/nls_utf8.ko 2023-08-09T13:16:53.872 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/fs/ocfs2 2023-08-09T13:16:53.872 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/fs/ocfs2/cluster 2023-08-09T13:16:53.873 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/fs/ocfs2/cluster/ocfs2_nodemanager.ko 2023-08-09T13:16:53.873 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/fs/ocfs2/dlm 2023-08-09T13:16:53.873 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/fs/ocfs2/dlm/ocfs2_dlm.ko 2023-08-09T13:16:53.873 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/fs/ocfs2/dlmfs 2023-08-09T13:16:53.873 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/fs/ocfs2/dlmfs/ocfs2_dlmfs.ko 2023-08-09T13:16:53.873 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/fs/ocfs2/ocfs2.ko 2023-08-09T13:16:53.873 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/fs/ocfs2/ocfs2_stack_o2cb.ko 2023-08-09T13:16:53.873 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/fs/ocfs2/ocfs2_stack_user.ko 2023-08-09T13:16:53.874 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/fs/ocfs2/ocfs2_stackglue.ko 2023-08-09T13:16:53.874 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/fs/overlayfs 2023-08-09T13:16:53.874 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/fs/overlayfs/overlay.ko 2023-08-09T13:16:53.874 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/fs/pstore 2023-08-09T13:16:53.874 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/fs/pstore/ramoops.ko 2023-08-09T13:16:53.874 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/fs/reiserfs 2023-08-09T13:16:53.874 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/fs/reiserfs/reiserfs.ko 2023-08-09T13:16:53.874 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/fs/romfs 2023-08-09T13:16:53.875 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/fs/romfs/romfs.ko 2023-08-09T13:16:53.875 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/fs/smb 2023-08-09T13:16:53.875 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/fs/smb/client 2023-08-09T13:16:53.875 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/fs/smb/client/cifs.ko 2023-08-09T13:16:53.875 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/fs/smb/common 2023-08-09T13:16:53.875 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/fs/smb/common/cifs_arc4.ko 2023-08-09T13:16:53.875 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/fs/smb/common/cifs_md4.ko 2023-08-09T13:16:53.875 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/fs/squashfs 2023-08-09T13:16:53.876 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/fs/squashfs/squashfs.ko 2023-08-09T13:16:53.876 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/fs/sysv 2023-08-09T13:16:53.876 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/fs/sysv/sysv.ko 2023-08-09T13:16:53.876 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/fs/udf 2023-08-09T13:16:53.876 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/fs/udf/udf.ko 2023-08-09T13:16:53.876 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/fs/ufs 2023-08-09T13:16:53.876 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/fs/ufs/ufs.ko 2023-08-09T13:16:53.876 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/fs/xfs 2023-08-09T13:16:53.877 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/fs/xfs/xfs.ko 2023-08-09T13:16:53.877 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/kernel 2023-08-09T13:16:53.877 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/kernel/rcu 2023-08-09T13:16:53.877 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/kernel/rcu/rcutorture.ko 2023-08-09T13:16:53.877 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/kernel/torture.ko 2023-08-09T13:16:53.877 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/kernel/trace 2023-08-09T13:16:53.877 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/kernel/trace/ring_buffer_benchmark.ko 2023-08-09T13:16:53.877 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/lib 2023-08-09T13:16:53.878 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/lib/asn1_encoder.ko 2023-08-09T13:16:53.878 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/lib/crc-itu-t.ko 2023-08-09T13:16:53.878 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/lib/crc8.ko 2023-08-09T13:16:53.878 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/lib/crypto 2023-08-09T13:16:53.878 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/lib/crypto/libarc4.ko 2023-08-09T13:16:53.878 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/lib/crypto/libdes.ko 2023-08-09T13:16:53.878 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/lib/libcrc32c.ko 2023-08-09T13:16:53.878 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/lib/lru_cache.ko 2023-08-09T13:16:53.879 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/lib/lz4 2023-08-09T13:16:53.879 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/lib/lz4/lz4_compress.ko 2023-08-09T13:16:53.879 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/lib/lz4/lz4hc_compress.ko 2023-08-09T13:16:53.879 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/lib/math 2023-08-09T13:16:53.879 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/lib/math/cordic.ko 2023-08-09T13:16:53.879 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/lib/raid6 2023-08-09T13:16:53.879 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/lib/raid6/raid6_pq.ko 2023-08-09T13:16:53.880 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/lib/reed_solomon 2023-08-09T13:16:53.880 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/lib/reed_solomon/reed_solomon.ko 2023-08-09T13:16:53.880 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/lib/ts_bm.ko 2023-08-09T13:16:53.880 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/lib/ts_fsm.ko 2023-08-09T13:16:53.880 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/lib/ts_kmp.ko 2023-08-09T13:16:53.880 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/lib/zstd 2023-08-09T13:16:53.880 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/lib/zstd/zstd_compress.ko 2023-08-09T13:16:53.880 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/mm 2023-08-09T13:16:53.881 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/mm/hwpoison-inject.ko 2023-08-09T13:16:53.881 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/net 2023-08-09T13:16:53.881 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/net/6lowpan 2023-08-09T13:16:53.881 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/net/6lowpan/6lowpan.ko 2023-08-09T13:16:53.881 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/net/6lowpan/nhc_dest.ko 2023-08-09T13:16:53.881 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/net/6lowpan/nhc_fragment.ko 2023-08-09T13:16:53.881 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/net/6lowpan/nhc_hop.ko 2023-08-09T13:16:53.881 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/net/6lowpan/nhc_ipv6.ko 2023-08-09T13:16:53.882 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/net/6lowpan/nhc_mobility.ko 2023-08-09T13:16:53.882 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/net/6lowpan/nhc_routing.ko 2023-08-09T13:16:53.882 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/net/6lowpan/nhc_udp.ko 2023-08-09T13:16:53.882 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/net/802 2023-08-09T13:16:53.882 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/net/802/garp.ko 2023-08-09T13:16:53.882 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/net/802/mrp.ko 2023-08-09T13:16:53.882 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/net/802/p8022.ko 2023-08-09T13:16:53.882 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/net/802/psnap.ko 2023-08-09T13:16:53.883 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/net/802/stp.ko 2023-08-09T13:16:53.883 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/net/8021q 2023-08-09T13:16:53.883 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/net/8021q/8021q.ko 2023-08-09T13:16:53.883 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/net/9p 2023-08-09T13:16:53.883 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/net/9p/9pnet.ko 2023-08-09T13:16:53.883 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/net/9p/9pnet_fd.ko 2023-08-09T13:16:53.883 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/net/9p/9pnet_rdma.ko 2023-08-09T13:16:53.883 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/net/9p/9pnet_virtio.ko 2023-08-09T13:16:53.884 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/net/batman-adv 2023-08-09T13:16:53.884 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/net/batman-adv/batman-adv.ko 2023-08-09T13:16:53.884 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/net/bridge 2023-08-09T13:16:53.884 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/net/bridge/br_netfilter.ko 2023-08-09T13:16:53.885 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/net/bridge/bridge.ko 2023-08-09T13:16:53.885 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/net/bridge/netfilter 2023-08-09T13:16:53.885 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/net/bridge/netfilter/ebt_802_3.ko 2023-08-09T13:16:53.885 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/net/bridge/netfilter/ebt_among.ko 2023-08-09T13:16:53.885 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/net/bridge/netfilter/ebt_arp.ko 2023-08-09T13:16:53.885 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/net/bridge/netfilter/ebt_arpreply.ko 2023-08-09T13:16:53.885 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/net/bridge/netfilter/ebt_dnat.ko 2023-08-09T13:16:53.886 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/net/bridge/netfilter/ebt_ip.ko 2023-08-09T13:16:53.886 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/net/bridge/netfilter/ebt_ip6.ko 2023-08-09T13:16:53.886 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/net/bridge/netfilter/ebt_limit.ko 2023-08-09T13:16:53.886 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/net/bridge/netfilter/ebt_log.ko 2023-08-09T13:16:53.886 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/net/bridge/netfilter/ebt_mark.ko 2023-08-09T13:16:53.886 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/net/bridge/netfilter/ebt_mark_m.ko 2023-08-09T13:16:53.886 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/net/bridge/netfilter/ebt_nflog.ko 2023-08-09T13:16:53.887 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/net/bridge/netfilter/ebt_pkttype.ko 2023-08-09T13:16:53.887 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/net/bridge/netfilter/ebt_redirect.ko 2023-08-09T13:16:53.887 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/net/bridge/netfilter/ebt_snat.ko 2023-08-09T13:16:53.887 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/net/bridge/netfilter/ebt_stp.ko 2023-08-09T13:16:53.887 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/net/bridge/netfilter/ebt_vlan.ko 2023-08-09T13:16:53.887 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/net/bridge/netfilter/ebtable_broute.ko 2023-08-09T13:16:53.887 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/net/bridge/netfilter/ebtable_filter.ko 2023-08-09T13:16:53.888 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/net/bridge/netfilter/ebtable_nat.ko 2023-08-09T13:16:53.888 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/net/bridge/netfilter/ebtables.ko 2023-08-09T13:16:53.888 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/net/bridge/netfilter/nf_conntrack_bridge.ko 2023-08-09T13:16:53.888 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/net/bridge/netfilter/nft_meta_bridge.ko 2023-08-09T13:16:53.888 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/net/bridge/netfilter/nft_reject_bridge.ko 2023-08-09T13:16:53.888 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/net/ceph 2023-08-09T13:16:53.889 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/net/ceph/libceph.ko 2023-08-09T13:16:53.889 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/net/core 2023-08-09T13:16:53.889 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/net/core/failover.ko 2023-08-09T13:16:53.889 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/net/core/pktgen.ko 2023-08-09T13:16:53.889 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/net/dccp 2023-08-09T13:16:53.889 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/net/dccp/dccp.ko 2023-08-09T13:16:53.889 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/net/dccp/dccp_diag.ko 2023-08-09T13:16:53.889 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/net/dccp/dccp_ipv4.ko 2023-08-09T13:16:53.890 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/net/dccp/dccp_ipv6.ko 2023-08-09T13:16:53.890 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/net/dns_resolver 2023-08-09T13:16:53.890 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/net/dns_resolver/dns_resolver.ko 2023-08-09T13:16:53.890 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/net/dsa 2023-08-09T13:16:53.890 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/net/dsa/dsa_core.ko 2023-08-09T13:16:53.890 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/net/dsa/tag_brcm.ko 2023-08-09T13:16:53.890 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/net/dsa/tag_dsa.ko 2023-08-09T13:16:53.890 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/net/dsa/tag_none.ko 2023-08-09T13:16:53.891 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/net/dsa/tag_trailer.ko 2023-08-09T13:16:53.891 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/net/ieee802154 2023-08-09T13:16:53.891 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/net/ieee802154/6lowpan 2023-08-09T13:16:53.891 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/net/ieee802154/6lowpan/ieee802154_6lowpan.ko 2023-08-09T13:16:53.891 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/net/ieee802154/ieee802154.ko 2023-08-09T13:16:53.891 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/net/ieee802154/ieee802154_socket.ko 2023-08-09T13:16:53.891 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/net/ipv4 2023-08-09T13:16:53.892 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/net/ipv4/ah4.ko 2023-08-09T13:16:53.892 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/net/ipv4/esp4.ko 2023-08-09T13:16:53.892 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/net/ipv4/fou.ko 2023-08-09T13:16:53.892 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/net/ipv4/gre.ko 2023-08-09T13:16:53.892 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/net/ipv4/inet_diag.ko 2023-08-09T13:16:53.892 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/net/ipv4/ip_gre.ko 2023-08-09T13:16:53.892 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/net/ipv4/ip_tunnel.ko 2023-08-09T13:16:53.893 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/net/ipv4/ip_vti.ko 2023-08-09T13:16:53.893 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/net/ipv4/ipcomp.ko 2023-08-09T13:16:53.893 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/net/ipv4/ipip.ko 2023-08-09T13:16:53.893 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/net/ipv4/netfilter 2023-08-09T13:16:53.893 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/net/ipv4/netfilter/arp_tables.ko 2023-08-09T13:16:53.893 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/net/ipv4/netfilter/arpt_mangle.ko 2023-08-09T13:16:53.893 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/net/ipv4/netfilter/arptable_filter.ko 2023-08-09T13:16:53.893 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/net/ipv4/netfilter/ipt_ECN.ko 2023-08-09T13:16:53.894 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/net/ipv4/netfilter/ipt_SYNPROXY.ko 2023-08-09T13:16:53.894 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/net/ipv4/netfilter/ipt_ah.ko 2023-08-09T13:16:53.894 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/net/ipv4/netfilter/ipt_rpfilter.ko 2023-08-09T13:16:53.894 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/net/ipv4/netfilter/iptable_mangle.ko 2023-08-09T13:16:53.894 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/net/ipv4/netfilter/iptable_nat.ko 2023-08-09T13:16:53.894 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/net/ipv4/netfilter/iptable_raw.ko 2023-08-09T13:16:53.894 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/net/ipv4/netfilter/iptable_security.ko 2023-08-09T13:16:53.895 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/net/ipv4/netfilter/nf_defrag_ipv4.ko 2023-08-09T13:16:53.895 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/net/ipv4/netfilter/nf_dup_ipv4.ko 2023-08-09T13:16:53.895 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/net/ipv4/netfilter/nf_nat_h323.ko 2023-08-09T13:16:53.895 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/net/ipv4/netfilter/nf_nat_pptp.ko 2023-08-09T13:16:53.895 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/net/ipv4/netfilter/nf_nat_snmp_basic.ko 2023-08-09T13:16:53.895 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/net/ipv4/netfilter/nf_socket_ipv4.ko 2023-08-09T13:16:53.895 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/net/ipv4/netfilter/nf_tproxy_ipv4.ko 2023-08-09T13:16:53.896 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/net/ipv4/netfilter/nft_dup_ipv4.ko 2023-08-09T13:16:53.896 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/net/ipv4/netfilter/nft_fib_ipv4.ko 2023-08-09T13:16:53.896 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/net/ipv4/netfilter/nft_reject_ipv4.ko 2023-08-09T13:16:53.896 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/net/ipv4/tcp_bic.ko 2023-08-09T13:16:53.896 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/net/ipv4/tcp_dctcp.ko 2023-08-09T13:16:53.896 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/net/ipv4/tcp_diag.ko 2023-08-09T13:16:53.896 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/net/ipv4/tcp_highspeed.ko 2023-08-09T13:16:53.896 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/net/ipv4/tcp_htcp.ko 2023-08-09T13:16:53.897 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/net/ipv4/tcp_hybla.ko 2023-08-09T13:16:53.897 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/net/ipv4/tcp_illinois.ko 2023-08-09T13:16:53.897 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/net/ipv4/tcp_lp.ko 2023-08-09T13:16:53.897 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/net/ipv4/tcp_scalable.ko 2023-08-09T13:16:53.897 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/net/ipv4/tcp_vegas.ko 2023-08-09T13:16:53.897 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/net/ipv4/tcp_veno.ko 2023-08-09T13:16:53.897 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/net/ipv4/tcp_westwood.ko 2023-08-09T13:16:53.898 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/net/ipv4/tcp_yeah.ko 2023-08-09T13:16:53.898 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/net/ipv4/tunnel4.ko 2023-08-09T13:16:53.898 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/net/ipv4/udp_diag.ko 2023-08-09T13:16:53.898 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/net/ipv4/udp_tunnel.ko 2023-08-09T13:16:53.898 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/net/ipv4/xfrm4_tunnel.ko 2023-08-09T13:16:53.898 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/net/ipv6 2023-08-09T13:16:53.898 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/net/ipv6/ah6.ko 2023-08-09T13:16:53.898 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/net/ipv6/esp6.ko 2023-08-09T13:16:53.899 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/net/ipv6/fou6.ko 2023-08-09T13:16:53.899 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/net/ipv6/ip6_tunnel.ko 2023-08-09T13:16:53.899 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/net/ipv6/ip6_udp_tunnel.ko 2023-08-09T13:16:53.899 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/net/ipv6/ip6_vti.ko 2023-08-09T13:16:53.899 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/net/ipv6/ipcomp6.ko 2023-08-09T13:16:53.899 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/net/ipv6/netfilter 2023-08-09T13:16:53.899 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/net/ipv6/netfilter/ip6_tables.ko 2023-08-09T13:16:53.899 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/net/ipv6/netfilter/ip6t_REJECT.ko 2023-08-09T13:16:53.900 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/net/ipv6/netfilter/ip6t_SYNPROXY.ko 2023-08-09T13:16:53.900 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/net/ipv6/netfilter/ip6t_ah.ko 2023-08-09T13:16:53.900 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/net/ipv6/netfilter/ip6t_eui64.ko 2023-08-09T13:16:53.900 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/net/ipv6/netfilter/ip6t_frag.ko 2023-08-09T13:16:53.900 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/net/ipv6/netfilter/ip6t_hbh.ko 2023-08-09T13:16:53.900 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/net/ipv6/netfilter/ip6t_ipv6header.ko 2023-08-09T13:16:53.900 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/net/ipv6/netfilter/ip6t_mh.ko 2023-08-09T13:16:53.901 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/net/ipv6/netfilter/ip6t_rpfilter.ko 2023-08-09T13:16:53.901 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/net/ipv6/netfilter/ip6t_rt.ko 2023-08-09T13:16:53.901 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/net/ipv6/netfilter/ip6table_filter.ko 2023-08-09T13:16:53.901 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/net/ipv6/netfilter/ip6table_mangle.ko 2023-08-09T13:16:53.901 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/net/ipv6/netfilter/ip6table_nat.ko 2023-08-09T13:16:53.901 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/net/ipv6/netfilter/ip6table_raw.ko 2023-08-09T13:16:53.901 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/net/ipv6/netfilter/ip6table_security.ko 2023-08-09T13:16:53.902 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/net/ipv6/netfilter/nf_defrag_ipv6.ko 2023-08-09T13:16:53.902 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/net/ipv6/netfilter/nf_dup_ipv6.ko 2023-08-09T13:16:53.902 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/net/ipv6/netfilter/nf_reject_ipv6.ko 2023-08-09T13:16:53.902 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/net/ipv6/netfilter/nf_socket_ipv6.ko 2023-08-09T13:16:53.902 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/net/ipv6/netfilter/nf_tproxy_ipv6.ko 2023-08-09T13:16:53.902 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/net/ipv6/netfilter/nft_dup_ipv6.ko 2023-08-09T13:16:53.902 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/net/ipv6/netfilter/nft_fib_ipv6.ko 2023-08-09T13:16:53.903 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/net/ipv6/netfilter/nft_reject_ipv6.ko 2023-08-09T13:16:53.903 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/net/ipv6/sit.ko 2023-08-09T13:16:53.903 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/net/ipv6/tunnel6.ko 2023-08-09T13:16:53.903 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/net/ipv6/xfrm6_tunnel.ko 2023-08-09T13:16:53.903 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/net/key 2023-08-09T13:16:53.903 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/net/key/af_key.ko 2023-08-09T13:16:53.903 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/net/l2tp 2023-08-09T13:16:53.904 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/net/l2tp/l2tp_core.ko 2023-08-09T13:16:53.904 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/net/l2tp/l2tp_debugfs.ko 2023-08-09T13:16:53.904 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/net/l2tp/l2tp_eth.ko 2023-08-09T13:16:53.904 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/net/l2tp/l2tp_ip.ko 2023-08-09T13:16:53.904 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/net/l2tp/l2tp_ip6.ko 2023-08-09T13:16:53.904 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/net/l2tp/l2tp_netlink.ko 2023-08-09T13:16:53.904 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/net/l2tp/l2tp_ppp.ko 2023-08-09T13:16:53.905 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/net/llc 2023-08-09T13:16:53.905 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/net/llc/llc.ko 2023-08-09T13:16:53.905 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/net/mac802154 2023-08-09T13:16:53.905 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/net/mac802154/mac802154.ko 2023-08-09T13:16:53.905 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/net/mpls 2023-08-09T13:16:53.905 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/net/mpls/mpls_gso.ko 2023-08-09T13:16:53.905 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/net/netfilter 2023-08-09T13:16:53.906 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/net/netfilter/ipset 2023-08-09T13:16:53.906 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/net/netfilter/ipset/ip_set.ko 2023-08-09T13:16:53.906 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/net/netfilter/ipset/ip_set_bitmap_ip.ko 2023-08-09T13:16:53.906 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/net/netfilter/ipset/ip_set_bitmap_ipmac.ko 2023-08-09T13:16:53.906 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/net/netfilter/ipset/ip_set_bitmap_port.ko 2023-08-09T13:16:53.906 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/net/netfilter/ipset/ip_set_hash_ip.ko 2023-08-09T13:16:53.906 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/net/netfilter/ipset/ip_set_hash_ipmark.ko 2023-08-09T13:16:53.906 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/net/netfilter/ipset/ip_set_hash_ipport.ko 2023-08-09T13:16:53.907 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/net/netfilter/ipset/ip_set_hash_ipportip.ko 2023-08-09T13:16:53.907 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/net/netfilter/ipset/ip_set_hash_ipportnet.ko 2023-08-09T13:16:53.907 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/net/netfilter/ipset/ip_set_hash_mac.ko 2023-08-09T13:16:53.907 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/net/netfilter/ipset/ip_set_hash_net.ko 2023-08-09T13:16:53.907 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/net/netfilter/ipset/ip_set_hash_netiface.ko 2023-08-09T13:16:53.907 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/net/netfilter/ipset/ip_set_hash_netnet.ko 2023-08-09T13:16:53.907 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/net/netfilter/ipset/ip_set_hash_netport.ko 2023-08-09T13:16:53.908 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/net/netfilter/ipset/ip_set_hash_netportnet.ko 2023-08-09T13:16:53.908 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/net/netfilter/ipset/ip_set_list_set.ko 2023-08-09T13:16:53.908 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/net/netfilter/ipvs 2023-08-09T13:16:53.908 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/net/netfilter/ipvs/ip_vs.ko 2023-08-09T13:16:53.908 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/net/netfilter/ipvs/ip_vs_dh.ko 2023-08-09T13:16:53.908 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/net/netfilter/ipvs/ip_vs_fo.ko 2023-08-09T13:16:53.908 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/net/netfilter/ipvs/ip_vs_ftp.ko 2023-08-09T13:16:53.909 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/net/netfilter/ipvs/ip_vs_lblc.ko 2023-08-09T13:16:53.909 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/net/netfilter/ipvs/ip_vs_lblcr.ko 2023-08-09T13:16:53.909 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/net/netfilter/ipvs/ip_vs_lc.ko 2023-08-09T13:16:53.909 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/net/netfilter/ipvs/ip_vs_nq.ko 2023-08-09T13:16:53.909 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/net/netfilter/ipvs/ip_vs_pe_sip.ko 2023-08-09T13:16:53.909 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/net/netfilter/ipvs/ip_vs_rr.ko 2023-08-09T13:16:53.909 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/net/netfilter/ipvs/ip_vs_sed.ko 2023-08-09T13:16:53.909 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/net/netfilter/ipvs/ip_vs_sh.ko 2023-08-09T13:16:53.910 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/net/netfilter/ipvs/ip_vs_wlc.ko 2023-08-09T13:16:53.910 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/net/netfilter/ipvs/ip_vs_wrr.ko 2023-08-09T13:16:53.910 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/net/netfilter/nf_conncount.ko 2023-08-09T13:16:53.910 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/net/netfilter/nf_conntrack.ko 2023-08-09T13:16:53.910 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/net/netfilter/nf_conntrack_amanda.ko 2023-08-09T13:16:53.910 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/net/netfilter/nf_conntrack_broadcast.ko 2023-08-09T13:16:53.910 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/net/netfilter/nf_conntrack_ftp.ko 2023-08-09T13:16:53.911 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/net/netfilter/nf_conntrack_h323.ko 2023-08-09T13:16:53.911 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/net/netfilter/nf_conntrack_irc.ko 2023-08-09T13:16:53.911 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/net/netfilter/nf_conntrack_netbios_ns.ko 2023-08-09T13:16:53.911 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/net/netfilter/nf_conntrack_netlink.ko 2023-08-09T13:16:53.911 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/net/netfilter/nf_conntrack_pptp.ko 2023-08-09T13:16:53.911 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/net/netfilter/nf_conntrack_sane.ko 2023-08-09T13:16:53.911 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/net/netfilter/nf_conntrack_sip.ko 2023-08-09T13:16:53.911 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/net/netfilter/nf_conntrack_snmp.ko 2023-08-09T13:16:53.912 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/net/netfilter/nf_conntrack_tftp.ko 2023-08-09T13:16:53.912 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/net/netfilter/nf_dup_netdev.ko 2023-08-09T13:16:53.912 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/net/netfilter/nf_flow_table.ko 2023-08-09T13:16:53.912 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/net/netfilter/nf_flow_table_inet.ko 2023-08-09T13:16:53.912 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/net/netfilter/nf_log_syslog.ko 2023-08-09T13:16:53.912 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/net/netfilter/nf_nat.ko 2023-08-09T13:16:53.912 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/net/netfilter/nf_nat_amanda.ko 2023-08-09T13:16:53.913 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/net/netfilter/nf_nat_ftp.ko 2023-08-09T13:16:53.913 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/net/netfilter/nf_nat_irc.ko 2023-08-09T13:16:53.913 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/net/netfilter/nf_nat_sip.ko 2023-08-09T13:16:53.913 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/net/netfilter/nf_nat_tftp.ko 2023-08-09T13:16:53.913 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/net/netfilter/nf_synproxy_core.ko 2023-08-09T13:16:53.913 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/net/netfilter/nf_tables.ko 2023-08-09T13:16:53.913 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/net/netfilter/nfnetlink.ko 2023-08-09T13:16:53.914 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/net/netfilter/nfnetlink_acct.ko 2023-08-09T13:16:53.914 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/net/netfilter/nfnetlink_log.ko 2023-08-09T13:16:53.914 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/net/netfilter/nfnetlink_osf.ko 2023-08-09T13:16:53.914 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/net/netfilter/nfnetlink_queue.ko 2023-08-09T13:16:53.914 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/net/netfilter/nft_chain_nat.ko 2023-08-09T13:16:53.914 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/net/netfilter/nft_compat.ko 2023-08-09T13:16:53.914 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/net/netfilter/nft_ct.ko 2023-08-09T13:16:53.914 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/net/netfilter/nft_dup_netdev.ko 2023-08-09T13:16:53.915 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/net/netfilter/nft_fib.ko 2023-08-09T13:16:53.915 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/net/netfilter/nft_fwd_netdev.ko 2023-08-09T13:16:53.915 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/net/netfilter/nft_hash.ko 2023-08-09T13:16:53.915 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/net/netfilter/nft_limit.ko 2023-08-09T13:16:53.915 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/net/netfilter/nft_log.ko 2023-08-09T13:16:53.915 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/net/netfilter/nft_masq.ko 2023-08-09T13:16:53.915 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/net/netfilter/nft_nat.ko 2023-08-09T13:16:53.916 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/net/netfilter/nft_numgen.ko 2023-08-09T13:16:53.916 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/net/netfilter/nft_queue.ko 2023-08-09T13:16:53.916 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/net/netfilter/nft_quota.ko 2023-08-09T13:16:53.916 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/net/netfilter/nft_redir.ko 2023-08-09T13:16:53.916 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/net/netfilter/nft_reject.ko 2023-08-09T13:16:53.916 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/net/netfilter/nft_reject_inet.ko 2023-08-09T13:16:53.916 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/net/netfilter/nft_socket.ko 2023-08-09T13:16:53.916 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/net/netfilter/nft_tunnel.ko 2023-08-09T13:16:53.917 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/net/netfilter/xt_AUDIT.ko 2023-08-09T13:16:53.917 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/net/netfilter/xt_CHECKSUM.ko 2023-08-09T13:16:53.917 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/net/netfilter/xt_CLASSIFY.ko 2023-08-09T13:16:53.917 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/net/netfilter/xt_CONNSECMARK.ko 2023-08-09T13:16:53.917 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/net/netfilter/xt_CT.ko 2023-08-09T13:16:53.917 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/net/netfilter/xt_DSCP.ko 2023-08-09T13:16:53.917 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/net/netfilter/xt_HL.ko 2023-08-09T13:16:53.918 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/net/netfilter/xt_HMARK.ko 2023-08-09T13:16:53.918 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/net/netfilter/xt_IDLETIMER.ko 2023-08-09T13:16:53.918 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/net/netfilter/xt_LED.ko 2023-08-09T13:16:53.918 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/net/netfilter/xt_LOG.ko 2023-08-09T13:16:53.918 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/net/netfilter/xt_MASQUERADE.ko 2023-08-09T13:16:53.918 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/net/netfilter/xt_NETMAP.ko 2023-08-09T13:16:53.918 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/net/netfilter/xt_NFLOG.ko 2023-08-09T13:16:53.919 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/net/netfilter/xt_NFQUEUE.ko 2023-08-09T13:16:53.919 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/net/netfilter/xt_RATEEST.ko 2023-08-09T13:16:53.919 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/net/netfilter/xt_REDIRECT.ko 2023-08-09T13:16:53.919 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/net/netfilter/xt_SECMARK.ko 2023-08-09T13:16:53.919 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/net/netfilter/xt_TCPMSS.ko 2023-08-09T13:16:53.919 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/net/netfilter/xt_TCPOPTSTRIP.ko 2023-08-09T13:16:53.919 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/net/netfilter/xt_TEE.ko 2023-08-09T13:16:53.919 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/net/netfilter/xt_TPROXY.ko 2023-08-09T13:16:53.920 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/net/netfilter/xt_TRACE.ko 2023-08-09T13:16:53.920 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/net/netfilter/xt_addrtype.ko 2023-08-09T13:16:53.920 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/net/netfilter/xt_bpf.ko 2023-08-09T13:16:53.920 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/net/netfilter/xt_cgroup.ko 2023-08-09T13:16:53.920 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/net/netfilter/xt_cluster.ko 2023-08-09T13:16:53.920 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/net/netfilter/xt_comment.ko 2023-08-09T13:16:53.920 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/net/netfilter/xt_connbytes.ko 2023-08-09T13:16:53.921 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/net/netfilter/xt_connlabel.ko 2023-08-09T13:16:53.921 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/net/netfilter/xt_connlimit.ko 2023-08-09T13:16:53.921 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/net/netfilter/xt_connmark.ko 2023-08-09T13:16:53.921 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/net/netfilter/xt_conntrack.ko 2023-08-09T13:16:53.921 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/net/netfilter/xt_cpu.ko 2023-08-09T13:16:53.921 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/net/netfilter/xt_dccp.ko 2023-08-09T13:16:53.921 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/net/netfilter/xt_devgroup.ko 2023-08-09T13:16:53.921 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/net/netfilter/xt_dscp.ko 2023-08-09T13:16:53.922 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/net/netfilter/xt_ecn.ko 2023-08-09T13:16:53.922 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/net/netfilter/xt_esp.ko 2023-08-09T13:16:53.922 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/net/netfilter/xt_hashlimit.ko 2023-08-09T13:16:53.922 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/net/netfilter/xt_helper.ko 2023-08-09T13:16:53.922 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/net/netfilter/xt_hl.ko 2023-08-09T13:16:53.922 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/net/netfilter/xt_ipcomp.ko 2023-08-09T13:16:53.922 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/net/netfilter/xt_iprange.ko 2023-08-09T13:16:53.923 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/net/netfilter/xt_ipvs.ko 2023-08-09T13:16:53.923 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/net/netfilter/xt_l2tp.ko 2023-08-09T13:16:53.923 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/net/netfilter/xt_length.ko 2023-08-09T13:16:53.923 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/net/netfilter/xt_limit.ko 2023-08-09T13:16:53.923 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/net/netfilter/xt_mac.ko 2023-08-09T13:16:53.923 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/net/netfilter/xt_mark.ko 2023-08-09T13:16:53.923 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/net/netfilter/xt_multiport.ko 2023-08-09T13:16:53.923 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/net/netfilter/xt_nat.ko 2023-08-09T13:16:53.924 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/net/netfilter/xt_nfacct.ko 2023-08-09T13:16:53.924 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/net/netfilter/xt_osf.ko 2023-08-09T13:16:53.924 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/net/netfilter/xt_owner.ko 2023-08-09T13:16:53.924 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/net/netfilter/xt_physdev.ko 2023-08-09T13:16:53.924 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/net/netfilter/xt_pkttype.ko 2023-08-09T13:16:53.924 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/net/netfilter/xt_policy.ko 2023-08-09T13:16:53.924 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/net/netfilter/xt_quota.ko 2023-08-09T13:16:53.924 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/net/netfilter/xt_rateest.ko 2023-08-09T13:16:53.925 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/net/netfilter/xt_realm.ko 2023-08-09T13:16:53.925 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/net/netfilter/xt_recent.ko 2023-08-09T13:16:53.925 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/net/netfilter/xt_sctp.ko 2023-08-09T13:16:53.925 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/net/netfilter/xt_set.ko 2023-08-09T13:16:53.925 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/net/netfilter/xt_socket.ko 2023-08-09T13:16:53.925 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/net/netfilter/xt_state.ko 2023-08-09T13:16:53.925 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/net/netfilter/xt_statistic.ko 2023-08-09T13:16:53.926 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/net/netfilter/xt_string.ko 2023-08-09T13:16:53.926 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/net/netfilter/xt_tcpmss.ko 2023-08-09T13:16:53.926 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/net/netfilter/xt_time.ko 2023-08-09T13:16:53.926 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/net/netfilter/xt_u32.ko 2023-08-09T13:16:53.926 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/net/netlink 2023-08-09T13:16:53.926 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/net/netlink/netlink_diag.ko 2023-08-09T13:16:53.926 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/net/nsh 2023-08-09T13:16:53.927 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/net/nsh/nsh.ko 2023-08-09T13:16:53.927 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/net/openvswitch 2023-08-09T13:16:53.927 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/net/openvswitch/openvswitch.ko 2023-08-09T13:16:53.927 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/net/openvswitch/vport-geneve.ko 2023-08-09T13:16:53.927 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/net/openvswitch/vport-gre.ko 2023-08-09T13:16:53.927 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/net/openvswitch/vport-vxlan.ko 2023-08-09T13:16:53.927 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/net/packet 2023-08-09T13:16:53.927 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/net/packet/af_packet_diag.ko 2023-08-09T13:16:53.928 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/net/rds 2023-08-09T13:16:53.928 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/net/rds/rds.ko 2023-08-09T13:16:53.928 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/net/rds/rds_rdma.ko 2023-08-09T13:16:53.928 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/net/rds/rds_tcp.ko 2023-08-09T13:16:53.928 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/net/sched 2023-08-09T13:16:53.928 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/net/sched/act_csum.ko 2023-08-09T13:16:53.928 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/net/sched/act_gact.ko 2023-08-09T13:16:53.929 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/net/sched/act_ipt.ko 2023-08-09T13:16:53.929 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/net/sched/act_mirred.ko 2023-08-09T13:16:53.929 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/net/sched/act_nat.ko 2023-08-09T13:16:53.929 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/net/sched/act_pedit.ko 2023-08-09T13:16:53.929 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/net/sched/act_police.ko 2023-08-09T13:16:53.929 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/net/sched/act_simple.ko 2023-08-09T13:16:53.929 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/net/sched/act_skbedit.ko 2023-08-09T13:16:53.929 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/net/sched/act_vlan.ko 2023-08-09T13:16:53.930 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/net/sched/cls_basic.ko 2023-08-09T13:16:53.930 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/net/sched/cls_bpf.ko 2023-08-09T13:16:53.930 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/net/sched/cls_flow.ko 2023-08-09T13:16:53.930 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/net/sched/cls_fw.ko 2023-08-09T13:16:53.930 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/net/sched/cls_route.ko 2023-08-09T13:16:53.930 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/net/sched/cls_u32.ko 2023-08-09T13:16:53.930 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/net/sched/em_cmp.ko 2023-08-09T13:16:53.930 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/net/sched/em_ipset.ko 2023-08-09T13:16:53.931 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/net/sched/em_meta.ko 2023-08-09T13:16:53.931 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/net/sched/em_nbyte.ko 2023-08-09T13:16:53.931 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/net/sched/em_text.ko 2023-08-09T13:16:53.931 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/net/sched/em_u32.ko 2023-08-09T13:16:53.931 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/net/sched/sch_choke.ko 2023-08-09T13:16:53.931 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/net/sched/sch_codel.ko 2023-08-09T13:16:53.931 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/net/sched/sch_drr.ko 2023-08-09T13:16:53.931 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/net/sched/sch_fq.ko 2023-08-09T13:16:53.932 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/net/sched/sch_gred.ko 2023-08-09T13:16:53.932 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/net/sched/sch_hfsc.ko 2023-08-09T13:16:53.932 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/net/sched/sch_hhf.ko 2023-08-09T13:16:53.932 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/net/sched/sch_htb.ko 2023-08-09T13:16:53.932 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/net/sched/sch_ingress.ko 2023-08-09T13:16:53.932 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/net/sched/sch_mqprio.ko 2023-08-09T13:16:53.932 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/net/sched/sch_mqprio_lib.ko 2023-08-09T13:16:53.932 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/net/sched/sch_multiq.ko 2023-08-09T13:16:53.933 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/net/sched/sch_netem.ko 2023-08-09T13:16:53.933 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/net/sched/sch_pie.ko 2023-08-09T13:16:53.933 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/net/sched/sch_plug.ko 2023-08-09T13:16:53.933 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/net/sched/sch_prio.ko 2023-08-09T13:16:53.933 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/net/sched/sch_qfq.ko 2023-08-09T13:16:53.933 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/net/sched/sch_red.ko 2023-08-09T13:16:53.933 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/net/sched/sch_sfb.ko 2023-08-09T13:16:53.934 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/net/sched/sch_sfq.ko 2023-08-09T13:16:53.934 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/net/sched/sch_tbf.ko 2023-08-09T13:16:53.934 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/net/sched/sch_teql.ko 2023-08-09T13:16:53.934 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/net/sctp 2023-08-09T13:16:53.934 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/net/sctp/sctp.ko 2023-08-09T13:16:53.934 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/net/sctp/sctp_diag.ko 2023-08-09T13:16:53.934 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/net/sunrpc 2023-08-09T13:16:53.934 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/net/sunrpc/auth_gss 2023-08-09T13:16:53.935 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/net/sunrpc/auth_gss/auth_rpcgss.ko 2023-08-09T13:16:53.935 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/net/sunrpc/auth_gss/rpcsec_gss_krb5.ko 2023-08-09T13:16:53.935 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/net/sunrpc/sunrpc.ko 2023-08-09T13:16:53.935 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/net/sunrpc/xprtrdma 2023-08-09T13:16:53.935 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/net/sunrpc/xprtrdma/rpcrdma.ko 2023-08-09T13:16:53.935 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/net/tipc 2023-08-09T13:16:53.935 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/net/tipc/diag.ko 2023-08-09T13:16:53.936 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/net/tipc/tipc.ko 2023-08-09T13:16:53.936 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/net/unix 2023-08-09T13:16:53.936 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/net/unix/unix_diag.ko 2023-08-09T13:16:53.936 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/net/vmw_vsock 2023-08-09T13:16:53.936 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/net/vmw_vsock/vmw_vsock_virtio_transport_common.ko 2023-08-09T13:16:53.936 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/net/vmw_vsock/vmw_vsock_vmci_transport.ko 2023-08-09T13:16:53.936 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/net/vmw_vsock/vsock.ko 2023-08-09T13:16:53.937 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/net/vmw_vsock/vsock_diag.ko 2023-08-09T13:16:53.937 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/net/vmw_vsock/vsock_loopback.ko 2023-08-09T13:16:53.937 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/net/xfrm 2023-08-09T13:16:53.937 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/net/xfrm/xfrm_ipcomp.ko 2023-08-09T13:16:53.937 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/security 2023-08-09T13:16:53.937 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/security/keys 2023-08-09T13:16:53.937 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/security/keys/encrypted-keys 2023-08-09T13:16:53.937 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/security/keys/encrypted-keys/encrypted-keys.ko 2023-08-09T13:16:53.938 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/security/keys/trusted-keys 2023-08-09T13:16:53.938 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/security/keys/trusted-keys/trusted.ko 2023-08-09T13:16:53.938 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/virt 2023-08-09T13:16:53.938 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/virt/lib 2023-08-09T13:16:53.938 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/virt/lib/irqbypass.ko 2023-08-09T13:16:53.938 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/modules.alias 2023-08-09T13:16:53.938 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/modules.alias.bin 2023-08-09T13:16:53.938 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/modules.builtin 2023-08-09T13:16:53.939 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/modules.builtin.alias.bin 2023-08-09T13:16:53.939 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/modules.builtin.bin 2023-08-09T13:16:53.939 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/modules.builtin.modinfo 2023-08-09T13:16:53.939 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/modules.dep 2023-08-09T13:16:53.939 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/modules.dep.bin 2023-08-09T13:16:53.939 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/modules.devname 2023-08-09T13:16:53.939 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/modules.order 2023-08-09T13:16:53.939 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/modules.softdep 2023-08-09T13:16:53.940 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/modules.symbols 2023-08-09T13:16:53.940 INFO:teuthology.orchestra.run.smithi203.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/modules.symbols.bin 2023-08-09T13:16:53.940 DEBUG:teuthology.task.kernel:get_image_version: 6.5.0-rc4-ga7fb1265323d 2023-08-09T13:16:53.941 DEBUG:teuthology.orchestra.run.smithi203:> rpm --scripts -qp /tmp/kernel.x86_64.rpm 2023-08-09T13:16:53.964 INFO:teuthology.orchestra.run.smithi203.stdout:postinstall scriptlet (using /bin/sh): 2023-08-09T13:16:53.965 INFO:teuthology.orchestra.run.smithi203.stdout:if [ -x /sbin/installkernel -a -r /boot/vmlinuz-6.5.0-rc4-ga7fb1265323d -a -r /boot/System.map-6.5.0-rc4-ga7fb1265323d ]; then 2023-08-09T13:16:53.965 INFO:teuthology.orchestra.run.smithi203.stdout:cp /boot/vmlinuz-6.5.0-rc4-ga7fb1265323d /boot/.vmlinuz-6.5.0-rc4-ga7fb1265323d-rpm 2023-08-09T13:16:53.965 INFO:teuthology.orchestra.run.smithi203.stdout:cp /boot/System.map-6.5.0-rc4-ga7fb1265323d /boot/.System.map-6.5.0-rc4-ga7fb1265323d-rpm 2023-08-09T13:16:53.965 INFO:teuthology.orchestra.run.smithi203.stdout:rm -f /boot/vmlinuz-6.5.0-rc4-ga7fb1265323d /boot/System.map-6.5.0-rc4-ga7fb1265323d 2023-08-09T13:16:53.965 INFO:teuthology.orchestra.run.smithi203.stdout:/sbin/installkernel 6.5.0-rc4-ga7fb1265323d /boot/.vmlinuz-6.5.0-rc4-ga7fb1265323d-rpm /boot/.System.map-6.5.0-rc4-ga7fb1265323d-rpm 2023-08-09T13:16:53.965 INFO:teuthology.orchestra.run.smithi203.stdout:rm -f /boot/.vmlinuz-6.5.0-rc4-ga7fb1265323d-rpm /boot/.System.map-6.5.0-rc4-ga7fb1265323d-rpm 2023-08-09T13:16:53.965 INFO:teuthology.orchestra.run.smithi203.stdout:fi 2023-08-09T13:16:53.966 INFO:teuthology.orchestra.run.smithi203.stdout:preuninstall scriptlet (using /bin/sh): 2023-08-09T13:16:53.966 INFO:teuthology.orchestra.run.smithi203.stdout:if [ -x /sbin/new-kernel-pkg ]; then 2023-08-09T13:16:53.966 INFO:teuthology.orchestra.run.smithi203.stdout:new-kernel-pkg --remove 6.5.0-rc4-ga7fb1265323d --rminitrd --initrdfile=/boot/initramfs-6.5.0-rc4-ga7fb1265323d.img 2023-08-09T13:16:53.966 INFO:teuthology.orchestra.run.smithi203.stdout:elif [ -x /usr/bin/kernel-install ]; then 2023-08-09T13:16:53.966 INFO:teuthology.orchestra.run.smithi203.stdout:kernel-install remove 6.5.0-rc4-ga7fb1265323d 2023-08-09T13:16:53.966 INFO:teuthology.orchestra.run.smithi203.stdout:fi 2023-08-09T13:16:53.966 INFO:teuthology.orchestra.run.smithi203.stdout:postuninstall scriptlet (using /bin/sh): 2023-08-09T13:16:53.967 INFO:teuthology.orchestra.run.smithi203.stdout:if [ -x /sbin/update-bootloader ]; then 2023-08-09T13:16:53.967 INFO:teuthology.orchestra.run.smithi203.stdout:/sbin/update-bootloader --remove 6.5.0-rc4-ga7fb1265323d 2023-08-09T13:16:53.967 INFO:teuthology.orchestra.run.smithi203.stdout:fi 2023-08-09T13:16:53.968 DEBUG:teuthology.orchestra.run.smithi203:> sudo rpm -qi grub2-tools 2023-08-09T13:16:54.044 INFO:teuthology.orchestra.run.smithi203.stdout:Name : grub2-tools 2023-08-09T13:16:54.045 INFO:teuthology.orchestra.run.smithi203.stdout:Epoch : 1 2023-08-09T13:16:54.045 INFO:teuthology.orchestra.run.smithi203.stdout:Version : 2.06 2023-08-09T13:16:54.045 INFO:teuthology.orchestra.run.smithi203.stdout:Release : 53.el9 2023-08-09T13:16:54.045 INFO:teuthology.orchestra.run.smithi203.stdout:Architecture: x86_64 2023-08-09T13:16:54.045 INFO:teuthology.orchestra.run.smithi203.stdout:Install Date: Mon 30 Jan 2023 10:38:40 AM UTC 2023-08-09T13:16:54.045 INFO:teuthology.orchestra.run.smithi203.stdout:Group : Unspecified 2023-08-09T13:16:54.046 INFO:teuthology.orchestra.run.smithi203.stdout:Size : 8551923 2023-08-09T13:16:54.046 INFO:teuthology.orchestra.run.smithi203.stdout:License : GPLv3+ 2023-08-09T13:16:54.046 INFO:teuthology.orchestra.run.smithi203.stdout:Signature : RSA/SHA256, Wed 18 Jan 2023 04:40:07 PM UTC, Key ID 05b555b38483c65d 2023-08-09T13:16:54.046 INFO:teuthology.orchestra.run.smithi203.stdout:Source RPM : grub2-2.06-53.el9.src.rpm 2023-08-09T13:16:54.046 INFO:teuthology.orchestra.run.smithi203.stdout:Build Date : Fri 13 Jan 2023 09:07:28 PM UTC 2023-08-09T13:16:54.046 INFO:teuthology.orchestra.run.smithi203.stdout:Build Host : x86-05.stream.rdu2.redhat.com 2023-08-09T13:16:54.046 INFO:teuthology.orchestra.run.smithi203.stdout:Packager : builder@centos.org 2023-08-09T13:16:54.046 INFO:teuthology.orchestra.run.smithi203.stdout:Vendor : CentOS 2023-08-09T13:16:54.047 INFO:teuthology.orchestra.run.smithi203.stdout:URL : http://www.gnu.org/software/grub/ 2023-08-09T13:16:54.047 INFO:teuthology.orchestra.run.smithi203.stdout:Summary : Support tools for GRUB. 2023-08-09T13:16:54.047 INFO:teuthology.orchestra.run.smithi203.stdout:Description : 2023-08-09T13:16:54.047 INFO:teuthology.orchestra.run.smithi203.stdout: 2023-08-09T13:16:54.047 INFO:teuthology.orchestra.run.smithi203.stdout:The GRand Unified Bootloader (GRUB) is a highly configurable and 2023-08-09T13:16:54.047 INFO:teuthology.orchestra.run.smithi203.stdout:customizable bootloader with modular architecture. It supports a rich 2023-08-09T13:16:54.047 INFO:teuthology.orchestra.run.smithi203.stdout:variety of kernel formats, file systems, computer architectures and 2023-08-09T13:16:54.048 INFO:teuthology.orchestra.run.smithi203.stdout:hardware devices. 2023-08-09T13:16:54.048 INFO:teuthology.orchestra.run.smithi203.stdout: 2023-08-09T13:16:54.048 INFO:teuthology.orchestra.run.smithi203.stdout:This subpackage provides tools for support of all platforms. 2023-08-09T13:16:54.049 INFO:teuthology.task.kernel:Updating Grub Version: grub2 2023-08-09T13:16:54.049 INFO:teuthology.task.kernel:Updating grub on smithi203 to boot 6.5.0-rc4-ga7fb1265323d 2023-08-09T13:16:54.049 DEBUG:teuthology.orchestra.run.smithi203:> sudo grub2-mkconfig -o /boot/grub2/grub.cfg 2023-08-09T13:16:54.628 INFO:teuthology.orchestra.run.smithi203.stderr:Generating grub configuration file ... 2023-08-09T13:16:55.915 INFO:teuthology.orchestra.run.smithi203.stderr:Adding boot menu entry for UEFI Firmware Settings ... 2023-08-09T13:16:55.944 INFO:teuthology.orchestra.run.smithi203.stderr:done 2023-08-09T13:16:55.953 DEBUG:teuthology.orchestra.run.smithi203:> mktemp 2023-08-09T13:16:55.980 INFO:teuthology.orchestra.run.smithi203.stdout:/tmp/tmp.q0O7apqTJW 2023-08-09T13:16:55.981 DEBUG:teuthology.orchestra.run.smithi203:> sudo cp /boot/grub2/grub.cfg /tmp/tmp.q0O7apqTJW 2023-08-09T13:16:56.050 DEBUG:teuthology.orchestra.run.smithi203:> sudo chmod 0666 /tmp/tmp.q0O7apqTJW 2023-08-09T13:16:56.206 DEBUG:teuthology.orchestra.remote:smithi203:/tmp/tmp.q0O7apqTJW is 6KB 2023-08-09T13:16:56.253 DEBUG:teuthology.orchestra.run.smithi203:> rm -fr /tmp/tmp.q0O7apqTJW 2023-08-09T13:16:56.268 DEBUG:teuthology.orchestra.run.smithi203:> sudo /bin/ls /boot/loader/entries || true 2023-08-09T13:16:56.333 INFO:teuthology.orchestra.run.smithi203.stdout:e13cba1d25f14c51a1d04da8db823a4e-0-rescue.conf 2023-08-09T13:16:56.333 INFO:teuthology.orchestra.run.smithi203.stdout:e13cba1d25f14c51a1d04da8db823a4e-5.14.0-239.el9.x86_64.conf 2023-08-09T13:16:56.333 INFO:teuthology.orchestra.run.smithi203.stdout:e13cba1d25f14c51a1d04da8db823a4e-6.5.0-rc4-ga7fb1265323d.conf 2023-08-09T13:16:56.334 DEBUG:teuthology.orchestra.run.smithi203:> sudo grub2-set-default e13cba1d25f14c51a1d04da8db823a4e-6.5.0-rc4-ga7fb1265323d 2023-08-09T13:16:56.487 DEBUG:teuthology.orchestra.run.smithi203:> sudo shutdown -r now 2023-08-09T13:17:13.224 INFO:teuthology.task.kernel:Checking client mon.a for new kernel version... 2023-08-09T13:17:13.225 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi043.front.sepia.ceph.com' 2023-08-09T13:17:13.225 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi043.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-08-09T13:17:26.502 INFO:teuthology.task.kernel:Checking client mon.b for new kernel version... 2023-08-09T13:17:26.502 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi203.front.sepia.ceph.com' 2023-08-09T13:17:26.503 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi203.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-08-09T13:17:31.636 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.43 2023-08-09T13:17:40.644 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi043.front.sepia.ceph.com' 2023-08-09T13:17:40.645 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi043.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-08-09T13:17:44.944 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.203 2023-08-09T13:17:53.952 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi203.front.sepia.ceph.com' 2023-08-09T13:17:53.953 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi203.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-08-09T13:17:57.008 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.203 2023-08-09T13:18:09.020 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi203.front.sepia.ceph.com' 2023-08-09T13:18:09.021 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi203.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-08-09T13:18:40.676 DEBUG:teuthology.orchestra.remote:timed out 2023-08-09T13:18:52.683 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi043.front.sepia.ceph.com' 2023-08-09T13:18:52.684 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi043.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-08-09T13:18:53.004 DEBUG:teuthology.orchestra.run.smithi043:> true 2023-08-09T13:18:53.676 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi043.front.sepia.ceph.com' 2023-08-09T13:18:53.677 INFO:teuthology.task.kernel:Checking kernel version of mon.a, want "6.5.0-rc4-ga7fb1265323d"... 2023-08-09T13:18:53.677 DEBUG:teuthology.orchestra.run.smithi043:> uname -r 2023-08-09T13:18:53.738 INFO:teuthology.orchestra.run.smithi043.stdout:6.5.0-rc4-ga7fb1265323d 2023-08-09T13:18:53.739 DEBUG:teuthology.task.kernel:current kernel version is 6.5.0-rc4-ga7fb1265323d vs 6.5.0-rc4-ga7fb1265323d 2023-08-09T13:18:53.739 DEBUG:teuthology.task.kernel:utsrelease strings match, do not need to install 2023-08-09T13:18:53.739 DEBUG:teuthology.task.kernel:Distro of this test job: centos 2023-08-09T13:18:54.740 INFO:teuthology.task.kernel:Enabling kdb on mon.a... 2023-08-09T13:18:54.740 DEBUG:teuthology.orchestra.run.smithi043:> echo ttyS1 | sudo tee /sys/module/kgdboc/parameters/kgdboc 2023-08-09T13:18:54.901 INFO:teuthology.orchestra.run.smithi043.stdout:ttyS1 2023-08-09T13:18:54.948 DEBUG:teuthology.parallel:result is None 2023-08-09T13:19:09.023 DEBUG:teuthology.orchestra.remote:timed out 2023-08-09T13:19:24.025 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi203.front.sepia.ceph.com' 2023-08-09T13:19:24.025 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi203.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-08-09T13:19:24.432 DEBUG:teuthology.orchestra.run.smithi203:> true 2023-08-09T13:19:25.154 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi203.front.sepia.ceph.com' 2023-08-09T13:19:25.154 INFO:teuthology.task.kernel:Checking kernel version of mon.b, want "6.5.0-rc4-ga7fb1265323d"... 2023-08-09T13:19:25.155 DEBUG:teuthology.orchestra.run.smithi203:> uname -r 2023-08-09T13:19:25.208 INFO:teuthology.orchestra.run.smithi203.stdout:6.5.0-rc4-ga7fb1265323d 2023-08-09T13:19:25.209 DEBUG:teuthology.task.kernel:current kernel version is 6.5.0-rc4-ga7fb1265323d vs 6.5.0-rc4-ga7fb1265323d 2023-08-09T13:19:25.209 DEBUG:teuthology.task.kernel:utsrelease strings match, do not need to install 2023-08-09T13:19:25.209 DEBUG:teuthology.task.kernel:Distro of this test job: centos 2023-08-09T13:19:26.210 INFO:teuthology.task.kernel:Enabling kdb on mon.b... 2023-08-09T13:19:26.211 DEBUG:teuthology.orchestra.run.smithi203:> echo ttyS1 | sudo tee /sys/module/kgdboc/parameters/kgdboc 2023-08-09T13:19:26.361 INFO:teuthology.orchestra.run.smithi203.stdout:ttyS1 2023-08-09T13:19:26.416 DEBUG:teuthology.parallel:result is None 2023-08-09T13:19:26.417 INFO:teuthology.run_tasks:Running task internal.base... 2023-08-09T13:19:26.425 INFO:teuthology.task.internal:Creating test directory... 2023-08-09T13:19:26.426 DEBUG:teuthology.orchestra.run.smithi043:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2023-08-09T13:19:26.435 DEBUG:teuthology.orchestra.run.smithi203:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2023-08-09T13:19:26.494 INFO:teuthology.run_tasks:Running task internal.archive_upload... 2023-08-09T13:19:26.502 INFO:teuthology.run_tasks:Running task internal.archive... 2023-08-09T13:19:26.516 INFO:teuthology.task.internal:Creating archive directory... 2023-08-09T13:19:26.517 DEBUG:teuthology.orchestra.run.smithi043:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2023-08-09T13:19:26.526 DEBUG:teuthology.orchestra.run.smithi203:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2023-08-09T13:19:26.609 INFO:teuthology.run_tasks:Running task internal.coredump... 2023-08-09T13:19:26.623 INFO:teuthology.task.internal:Enabling coredump saving... 2023-08-09T13:19:26.624 DEBUG:teuthology.orchestra.run.smithi043:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2023-08-09T13:19:26.633 DEBUG:teuthology.orchestra.run.smithi203:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2023-08-09T13:19:26.745 INFO:teuthology.orchestra.run.smithi043.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2023-08-09T13:19:26.770 INFO:teuthology.orchestra.run.smithi203.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2023-08-09T13:19:26.791 INFO:teuthology.orchestra.run.smithi043.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2023-08-09T13:19:26.816 INFO:teuthology.orchestra.run.smithi203.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2023-08-09T13:19:26.820 INFO:teuthology.run_tasks:Running task internal.sudo... 2023-08-09T13:19:26.830 INFO:teuthology.task.internal:Configuring sudo... 2023-08-09T13:19:26.830 DEBUG:teuthology.orchestra.run.smithi043:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2023-08-09T13:19:26.840 DEBUG:teuthology.orchestra.run.smithi203:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2023-08-09T13:19:26.942 INFO:teuthology.run_tasks:Running task internal.syslog... 2023-08-09T13:19:26.953 INFO:teuthology.task.internal.syslog:Starting syslog monitoring... 2023-08-09T13:19:26.953 DEBUG:teuthology.orchestra.run.smithi043:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2023-08-09T13:19:26.962 DEBUG:teuthology.orchestra.run.smithi203:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2023-08-09T13:19:27.020 DEBUG:teuthology.orchestra.run.smithi043:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2023-08-09T13:19:27.106 DEBUG:teuthology.orchestra.run.smithi043:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/kern.log 2023-08-09T13:19:27.220 DEBUG:teuthology.orchestra.run.smithi043:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2023-08-09T13:19:27.309 DEBUG:teuthology.orchestra.run.smithi043:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/misc.log 2023-08-09T13:19:27.416 DEBUG:teuthology.orchestra.run.smithi043:> set -ex 2023-08-09T13:19:27.417 DEBUG:teuthology.orchestra.run.smithi043:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2023-08-09T13:19:27.513 DEBUG:teuthology.orchestra.run.smithi203:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2023-08-09T13:19:27.606 DEBUG:teuthology.orchestra.run.smithi203:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/kern.log 2023-08-09T13:19:27.728 DEBUG:teuthology.orchestra.run.smithi203:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2023-08-09T13:19:27.811 DEBUG:teuthology.orchestra.run.smithi203:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/misc.log 2023-08-09T13:19:27.888 DEBUG:teuthology.orchestra.run.smithi203:> set -ex 2023-08-09T13:19:27.888 DEBUG:teuthology.orchestra.run.smithi203:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2023-08-09T13:19:27.962 DEBUG:teuthology.orchestra.run.smithi043:> sudo service rsyslog restart 2023-08-09T13:19:27.971 DEBUG:teuthology.orchestra.run.smithi203:> sudo service rsyslog restart 2023-08-09T13:19:28.097 INFO:teuthology.orchestra.run.smithi203.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2023-08-09T13:19:28.111 INFO:teuthology.orchestra.run.smithi043.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2023-08-09T13:19:28.570 INFO:teuthology.run_tasks:Running task internal.timer... 2023-08-09T13:19:28.579 INFO:teuthology.task.internal:Starting timer... 2023-08-09T13:19:28.579 INFO:teuthology.run_tasks:Running task pcp... 2023-08-09T13:19:28.601 INFO:teuthology.run_tasks:Running task selinux... 2023-08-09T13:19:28.610 DEBUG:teuthology.orchestra.run.smithi043:> sudo service auditd rotate 2023-08-09T13:19:28.738 INFO:teuthology.orchestra.run.smithi043.stdout:Rotating logs: 2023-08-09T13:19:28.745 DEBUG:teuthology.orchestra.run.smithi203:> sudo service auditd rotate 2023-08-09T13:19:28.853 INFO:teuthology.orchestra.run.smithi203.stdout:Rotating logs: 2023-08-09T13:19:28.856 DEBUG:teuthology.task.selinux:Getting current SELinux state 2023-08-09T13:19:28.857 DEBUG:teuthology.orchestra.run.smithi043:> /usr/sbin/getenforce 2023-08-09T13:19:28.929 INFO:teuthology.orchestra.run.smithi043.stdout:Permissive 2023-08-09T13:19:28.930 DEBUG:teuthology.orchestra.run.smithi203:> /usr/sbin/getenforce 2023-08-09T13:19:29.010 INFO:teuthology.orchestra.run.smithi203.stdout:Permissive 2023-08-09T13:19:29.010 DEBUG:teuthology.task.selinux:Existing SELinux modes: {'ubuntu@smithi043.front.sepia.ceph.com': 'permissive', 'ubuntu@smithi203.front.sepia.ceph.com': 'permissive'} 2023-08-09T13:19:29.011 DEBUG:teuthology.orchestra.run.smithi043:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 2023-08-09T13:19:29.105 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-08-09T13:19:29.106 DEBUG:teuthology.orchestra.run.smithi203:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 2023-08-09T13:19:29.208 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-08-09T13:19:29.209 INFO:teuthology.task.selinux:Putting SELinux into permissive mode 2023-08-09T13:19:29.209 DEBUG:teuthology.orchestra.run.smithi043:> sudo /usr/sbin/setenforce permissive 2023-08-09T13:19:29.317 DEBUG:teuthology.orchestra.run.smithi203:> sudo /usr/sbin/setenforce permissive 2023-08-09T13:19:29.422 INFO:teuthology.run_tasks:Running task ansible.cephlab... 2023-08-09T13:19:29.432 DEBUG:teuthology.repo_utils:Setting repo remote to https://git.ceph.com/ceph-cm-ansible.git 2023-08-09T13:19:29.439 INFO:teuthology.repo_utils:Fetching main from origin 2023-08-09T13:19:29.573 INFO:teuthology.repo_utils:Resetting repo at /home/teuthworker/src/git.ceph.com_ceph-cm-ansible_main to origin/main 2023-08-09T13:19:29.597 INFO:teuthology.task.ansible:Playbook: [{'import_playbook': 'ansible_managed.yml'}, {'import_playbook': 'teuthology.yml'}, {'hosts': 'testnodes', 'tasks': [{'set_fact': {'ran_from_cephlab_playbook': True}}]}, {'import_playbook': 'testnodes.yml'}, {'import_playbook': 'container-host.yml'}, {'import_playbook': 'cobbler.yml'}, {'import_playbook': 'paddles.yml'}, {'import_playbook': 'pulpito.yml'}, {'hosts': 'testnodes', 'become': True, 'tasks': [{'name': 'Touch /ceph-qa-ready', 'file': {'path': '/ceph-qa-ready', 'state': 'touch'}, 'when': 'ran_from_cephlab_playbook|bool'}]}] 2023-08-09T13:19:29.599 DEBUG:teuthology.task.ansible:Running ansible-playbook -v --extra-vars '{"ansible_ssh_user": "ubuntu"}' -i /etc/ansible/hosts --limit smithi043.front.sepia.ceph.com,smithi203.front.sepia.ceph.com /home/teuthworker/src/git.ceph.com_ceph-cm-ansible_main/cephlab.yml 2023-08-09T13:30:50.010 DEBUG:teuthology.task.ansible:Reconnecting to [Remote(name='ubuntu@smithi043.front.sepia.ceph.com'), Remote(name='ubuntu@smithi203.front.sepia.ceph.com')] 2023-08-09T13:30:50.011 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi043.front.sepia.ceph.com' 2023-08-09T13:30:50.013 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi043.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-08-09T13:30:50.161 DEBUG:teuthology.orchestra.run.smithi043:> true 2023-08-09T13:30:50.315 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi043.front.sepia.ceph.com' 2023-08-09T13:30:50.316 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi203.front.sepia.ceph.com' 2023-08-09T13:30:50.316 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi203.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-08-09T13:30:50.452 DEBUG:teuthology.orchestra.run.smithi203:> true 2023-08-09T13:30:50.616 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi203.front.sepia.ceph.com' 2023-08-09T13:30:50.617 INFO:teuthology.run_tasks:Running task clock... 2023-08-09T13:30:50.631 INFO:teuthology.task.clock:Syncing clocks and checking initial clock skew... 2023-08-09T13:30:50.631 INFO:teuthology.orchestra.run:Running command with timeout 360 2023-08-09T13:30:50.632 DEBUG:teuthology.orchestra.run.smithi043:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2023-08-09T13:30:50.641 INFO:teuthology.orchestra.run:Running command with timeout 360 2023-08-09T13:30:50.641 DEBUG:teuthology.orchestra.run.smithi203:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2023-08-09T13:30:50.747 INFO:teuthology.orchestra.run.smithi043.stderr:Failed to stop ntp.service: Unit ntp.service not loaded. 2023-08-09T13:30:50.751 INFO:teuthology.orchestra.run.smithi203.stderr:Failed to stop ntp.service: Unit ntp.service not loaded. 2023-08-09T13:30:50.804 INFO:teuthology.orchestra.run.smithi203.stderr:Failed to stop ntpd.service: Unit ntpd.service not loaded. 2023-08-09T13:30:50.805 INFO:teuthology.orchestra.run.smithi043.stderr:Failed to stop ntpd.service: Unit ntpd.service not loaded. 2023-08-09T13:30:50.897 INFO:teuthology.orchestra.run.smithi203.stderr:sudo: ntpd: command not found 2023-08-09T13:30:50.904 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: ntpd: command not found 2023-08-09T13:30:50.938 INFO:teuthology.orchestra.run.smithi203.stdout:506 Cannot talk to daemon 2023-08-09T13:30:50.950 INFO:teuthology.orchestra.run.smithi043.stdout:506 Cannot talk to daemon 2023-08-09T13:30:50.988 INFO:teuthology.orchestra.run.smithi203.stderr:Failed to start ntp.service: Unit ntp.service not found. 2023-08-09T13:30:51.008 INFO:teuthology.orchestra.run.smithi043.stderr:Failed to start ntp.service: Unit ntp.service not found. 2023-08-09T13:30:51.037 INFO:teuthology.orchestra.run.smithi203.stderr:Failed to start ntpd.service: Unit ntpd.service not found. 2023-08-09T13:30:51.070 INFO:teuthology.orchestra.run.smithi043.stderr:Failed to start ntpd.service: Unit ntpd.service not found. 2023-08-09T13:30:51.164 INFO:teuthology.orchestra.run.smithi203.stderr:bash: line 1: ntpq: command not found 2023-08-09T13:30:51.172 INFO:teuthology.orchestra.run.smithi203.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2023-08-09T13:30:51.172 INFO:teuthology.orchestra.run.smithi203.stdout:=============================================================================== 2023-08-09T13:30:51.173 INFO:teuthology.orchestra.run.smithi203.stdout:^? hv01.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2023-08-09T13:30:51.173 INFO:teuthology.orchestra.run.smithi203.stdout:^? hv02.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2023-08-09T13:30:51.173 INFO:teuthology.orchestra.run.smithi203.stdout:^? hv03.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2023-08-09T13:30:51.173 INFO:teuthology.orchestra.run.smithi203.stdout:^? hv04.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2023-08-09T13:30:51.205 INFO:teuthology.orchestra.run.smithi043.stderr:bash: line 1: ntpq: command not found 2023-08-09T13:30:51.215 INFO:teuthology.orchestra.run.smithi043.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2023-08-09T13:30:51.215 INFO:teuthology.orchestra.run.smithi043.stdout:=============================================================================== 2023-08-09T13:30:51.215 INFO:teuthology.orchestra.run.smithi043.stdout:^? hv01.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2023-08-09T13:30:51.215 INFO:teuthology.orchestra.run.smithi043.stdout:^? hv02.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2023-08-09T13:30:51.215 INFO:teuthology.orchestra.run.smithi043.stdout:^? hv03.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2023-08-09T13:30:51.216 INFO:teuthology.orchestra.run.smithi043.stdout:^? hv04.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2023-08-09T13:30:51.217 INFO:teuthology.run_tasks:Running task install... 2023-08-09T13:30:51.224 DEBUG:teuthology.task.install:project ceph 2023-08-09T13:30:51.225 DEBUG:teuthology.task.install:INSTALL overrides: {'ceph': {'flavor': 'default', 'sha1': '668a86f232c9454b2ce5928d1432a29a72964d85'}} 2023-08-09T13:30:51.225 DEBUG:teuthology.task.install:config {'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'flavor': 'default', 'sha1': '668a86f232c9454b2ce5928d1432a29a72964d85'} 2023-08-09T13:30:51.225 INFO:teuthology.task.install:Using flavor: default 2023-08-09T13:30:51.231 DEBUG:teuthology.task.install:Package list is: {'deb': ['ceph', 'cephadm', 'ceph-mds', 'ceph-mgr', 'ceph-common', 'ceph-fuse', 'ceph-test', 'ceph-volume', 'radosgw', 'python3-rados', 'python3-rgw', 'python3-cephfs', 'python3-rbd', 'libcephfs2', 'libcephfs-dev', 'librados2', 'librbd1', 'rbd-fuse'], 'rpm': ['ceph-radosgw', 'ceph-test', 'ceph', 'ceph-base', 'cephadm', 'ceph-immutable-object-cache', 'ceph-mgr', 'ceph-mgr-dashboard', 'ceph-mgr-diskprediction-local', 'ceph-mgr-rook', 'ceph-mgr-cephadm', 'ceph-fuse', 'ceph-volume', 'librados-devel', 'libcephfs2', 'libcephfs-devel', 'librados2', 'librbd1', 'python3-rados', 'python3-rgw', 'python3-cephfs', 'python3-rbd', 'rbd-fuse', 'rbd-mirror', 'rbd-nbd']} 2023-08-09T13:30:51.232 INFO:teuthology.task.install:extra packages: {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-top', 'cephfs-mirror']} 2023-08-09T13:30:51.232 DEBUG:teuthology.task.install.rpm:_update_package_list_and_install: config is {'branch': None, 'cleanup': None, 'debuginfo': None, 'downgrade_packages': [], 'exclude_packages': [], 'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'extras': None, 'enable_coprs': [], 'flavor': 'default', 'install_ceph_packages': True, 'packages': {}, 'project': 'ceph', 'repos_only': False, 'sha1': '668a86f232c9454b2ce5928d1432a29a72964d85', 'tag': None, 'wait_for_package': False} 2023-08-09T13:30:51.233 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=668a86f232c9454b2ce5928d1432a29a72964d85 2023-08-09T13:30:51.235 DEBUG:teuthology.task.install.rpm:_update_package_list_and_install: config is {'branch': None, 'cleanup': None, 'debuginfo': None, 'downgrade_packages': [], 'exclude_packages': [], 'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'extras': None, 'enable_coprs': [], 'flavor': 'default', 'install_ceph_packages': True, 'packages': {}, 'project': 'ceph', 'repos_only': False, 'sha1': '668a86f232c9454b2ce5928d1432a29a72964d85', 'tag': None, 'wait_for_package': False} 2023-08-09T13:30:51.235 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=668a86f232c9454b2ce5928d1432a29a72964d85 2023-08-09T13:30:51.460 INFO:teuthology.task.install.rpm:Pulling from https://2.chacra.ceph.com/r/ceph/wip-vshankar-testing-20230809.035933/668a86f232c9454b2ce5928d1432a29a72964d85/centos/9/flavors/default/ 2023-08-09T13:30:51.461 INFO:teuthology.task.install.rpm:Package version is 18.0.0-5389.g668a86f2 2023-08-09T13:30:51.467 INFO:teuthology.task.install.rpm:Pulling from https://2.chacra.ceph.com/r/ceph/wip-vshankar-testing-20230809.035933/668a86f232c9454b2ce5928d1432a29a72964d85/centos/9/flavors/default/ 2023-08-09T13:30:51.468 INFO:teuthology.task.install.rpm:Package version is 18.0.0-5389.g668a86f2 2023-08-09T13:30:51.580 INFO:teuthology.packaging:Writing yum repo: [ceph] name=ceph packages for $basearch baseurl=https://2.chacra.ceph.com/r/ceph/wip-vshankar-testing-20230809.035933/668a86f232c9454b2ce5928d1432a29a72964d85/centos/9/flavors/default/$basearch enabled=1 gpgcheck=0 type=rpm-md [ceph-noarch] name=ceph noarch packages baseurl=https://2.chacra.ceph.com/r/ceph/wip-vshankar-testing-20230809.035933/668a86f232c9454b2ce5928d1432a29a72964d85/centos/9/flavors/default/noarch enabled=1 gpgcheck=0 type=rpm-md [ceph-source] name=ceph source packages baseurl=https://2.chacra.ceph.com/r/ceph/wip-vshankar-testing-20230809.035933/668a86f232c9454b2ce5928d1432a29a72964d85/centos/9/flavors/default/SRPMS enabled=1 gpgcheck=0 type=rpm-md 2023-08-09T13:30:51.581 DEBUG:teuthology.orchestra.run.smithi043:> set -ex 2023-08-09T13:30:51.581 DEBUG:teuthology.orchestra.run.smithi043:> sudo dd of=/etc/yum.repos.d/ceph.repo 2023-08-09T13:30:51.633 INFO:teuthology.packaging:Writing yum repo: [ceph] name=ceph packages for $basearch baseurl=https://2.chacra.ceph.com/r/ceph/wip-vshankar-testing-20230809.035933/668a86f232c9454b2ce5928d1432a29a72964d85/centos/9/flavors/default/$basearch enabled=1 gpgcheck=0 type=rpm-md [ceph-noarch] name=ceph noarch packages baseurl=https://2.chacra.ceph.com/r/ceph/wip-vshankar-testing-20230809.035933/668a86f232c9454b2ce5928d1432a29a72964d85/centos/9/flavors/default/noarch enabled=1 gpgcheck=0 type=rpm-md [ceph-source] name=ceph source packages baseurl=https://2.chacra.ceph.com/r/ceph/wip-vshankar-testing-20230809.035933/668a86f232c9454b2ce5928d1432a29a72964d85/centos/9/flavors/default/SRPMS enabled=1 gpgcheck=0 type=rpm-md 2023-08-09T13:30:51.634 DEBUG:teuthology.orchestra.run.smithi203:> set -ex 2023-08-09T13:30:51.634 DEBUG:teuthology.orchestra.run.smithi203:> sudo dd of=/etc/yum.repos.d/ceph.repo 2023-08-09T13:30:51.690 INFO:teuthology.task.install.rpm:Installing packages: ceph-radosgw, ceph-test, ceph, ceph-base, cephadm, ceph-immutable-object-cache, ceph-mgr, ceph-mgr-dashboard, ceph-mgr-diskprediction-local, ceph-mgr-rook, ceph-mgr-cephadm, ceph-fuse, ceph-volume, librados-devel, libcephfs2, libcephfs-devel, librados2, librbd1, python3-rados, python3-rgw, python3-cephfs, python3-rbd, rbd-fuse, rbd-mirror, rbd-nbd, python3-cephfs, cephfs-top, cephfs-mirror, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib on remote rpm x86_64 2023-08-09T13:30:51.690 DEBUG:teuthology.orchestra.run.smithi043:> if test -f /etc/yum.repos.d/ceph.repo ; then sudo sed -i -e ':a;N;$!ba;s/enabled=1\ngpg/enabled=1\npriority=1\ngpg/g' -e 's;ref/[a-zA-Z0-9_-]*/;sha1/668a86f232c9454b2ce5928d1432a29a72964d85/;g' /etc/yum.repos.d/ceph.repo ; fi 2023-08-09T13:30:51.738 INFO:teuthology.task.install.rpm:Installing packages: ceph-radosgw, ceph-test, ceph, ceph-base, cephadm, ceph-immutable-object-cache, ceph-mgr, ceph-mgr-dashboard, ceph-mgr-diskprediction-local, ceph-mgr-rook, ceph-mgr-cephadm, ceph-fuse, ceph-volume, librados-devel, libcephfs2, libcephfs-devel, librados2, librbd1, python3-rados, python3-rgw, python3-cephfs, python3-rbd, rbd-fuse, rbd-mirror, rbd-nbd, python3-cephfs, cephfs-top, cephfs-mirror, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib on remote rpm x86_64 2023-08-09T13:30:51.738 DEBUG:teuthology.orchestra.run.smithi203:> if test -f /etc/yum.repos.d/ceph.repo ; then sudo sed -i -e ':a;N;$!ba;s/enabled=1\ngpg/enabled=1\npriority=1\ngpg/g' -e 's;ref/[a-zA-Z0-9_-]*/;sha1/668a86f232c9454b2ce5928d1432a29a72964d85/;g' /etc/yum.repos.d/ceph.repo ; fi 2023-08-09T13:30:51.837 DEBUG:teuthology.orchestra.run.smithi043:> sudo touch -a /etc/yum/pluginconf.d/priorities.conf ; test -e /etc/yum/pluginconf.d/priorities.conf.orig || sudo cp -af /etc/yum/pluginconf.d/priorities.conf /etc/yum/pluginconf.d/priorities.conf.orig 2023-08-09T13:30:51.882 DEBUG:teuthology.orchestra.run.smithi203:> sudo touch -a /etc/yum/pluginconf.d/priorities.conf ; test -e /etc/yum/pluginconf.d/priorities.conf.orig || sudo cp -af /etc/yum/pluginconf.d/priorities.conf /etc/yum/pluginconf.d/priorities.conf.orig 2023-08-09T13:30:51.986 DEBUG:teuthology.orchestra.run.smithi043:> grep check_obsoletes /etc/yum/pluginconf.d/priorities.conf && sudo sed -i 's/check_obsoletes.*0/check_obsoletes = 1/g' /etc/yum/pluginconf.d/priorities.conf || echo 'check_obsoletes = 1' | sudo tee -a /etc/yum/pluginconf.d/priorities.conf 2023-08-09T13:30:52.018 DEBUG:teuthology.orchestra.run.smithi203:> grep check_obsoletes /etc/yum/pluginconf.d/priorities.conf && sudo sed -i 's/check_obsoletes.*0/check_obsoletes = 1/g' /etc/yum/pluginconf.d/priorities.conf || echo 'check_obsoletes = 1' | sudo tee -a /etc/yum/pluginconf.d/priorities.conf 2023-08-09T13:30:52.087 INFO:teuthology.orchestra.run.smithi043.stdout:check_obsoletes = 1 2023-08-09T13:30:52.092 DEBUG:teuthology.orchestra.run.smithi043:> sudo yum clean all 2023-08-09T13:30:52.110 INFO:teuthology.orchestra.run.smithi203.stdout:check_obsoletes = 1 2023-08-09T13:30:52.114 DEBUG:teuthology.orchestra.run.smithi203:> sudo yum clean all 2023-08-09T13:30:52.482 INFO:teuthology.orchestra.run.smithi043.stdout:72 files removed 2023-08-09T13:30:52.488 INFO:teuthology.orchestra.run.smithi203.stdout:72 files removed 2023-08-09T13:30:52.521 DEBUG:teuthology.orchestra.run.smithi043:> sudo yum -y install ceph-radosgw ceph-test ceph ceph-base cephadm ceph-immutable-object-cache ceph-mgr ceph-mgr-dashboard ceph-mgr-diskprediction-local ceph-mgr-rook ceph-mgr-cephadm ceph-fuse ceph-volume librados-devel libcephfs2 libcephfs-devel librados2 librbd1 python3-rados python3-rgw python3-cephfs python3-rbd rbd-fuse rbd-mirror rbd-nbd python3-cephfs cephfs-top cephfs-mirror bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib 2023-08-09T13:30:52.525 DEBUG:teuthology.orchestra.run.smithi203:> sudo yum -y install ceph-radosgw ceph-test ceph ceph-base cephadm ceph-immutable-object-cache ceph-mgr ceph-mgr-dashboard ceph-mgr-diskprediction-local ceph-mgr-rook ceph-mgr-cephadm ceph-fuse ceph-volume librados-devel libcephfs2 libcephfs-devel librados2 librbd1 python3-rados python3-rgw python3-cephfs python3-rbd rbd-fuse rbd-mirror rbd-nbd python3-cephfs cephfs-top cephfs-mirror bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib 2023-08-09T13:30:53.378 INFO:teuthology.orchestra.run.smithi043.stdout:ceph packages for x86_64 194 kB/s | 78 kB 00:00 2023-08-09T13:30:53.515 INFO:teuthology.orchestra.run.smithi203.stdout:ceph packages for x86_64 142 kB/s | 78 kB 00:00 2023-08-09T13:30:53.870 INFO:teuthology.orchestra.run.smithi043.stdout:ceph noarch packages 23 kB/s | 9.5 kB 00:00 2023-08-09T13:30:54.016 INFO:teuthology.orchestra.run.smithi203.stdout:ceph noarch packages 23 kB/s | 9.5 kB 00:00 2023-08-09T13:30:54.454 INFO:teuthology.orchestra.run.smithi043.stdout:ceph source packages 3.6 kB/s | 1.9 kB 00:00 2023-08-09T13:30:54.650 INFO:teuthology.orchestra.run.smithi203.stdout:ceph source packages 3.3 kB/s | 1.9 kB 00:00 2023-08-09T13:30:55.125 INFO:teuthology.orchestra.run.smithi043.stdout:CentOS-9 - AppStream 10 MB/s | 6.3 MB 00:00 2023-08-09T13:30:55.245 INFO:teuthology.orchestra.run.smithi203.stdout:CentOS-9 - AppStream 12 MB/s | 6.3 MB 00:00 2023-08-09T13:30:56.708 INFO:teuthology.orchestra.run.smithi043.stdout:CentOS-9 - BaseOS 5.0 MB/s | 2.0 MB 00:00 2023-08-09T13:30:56.770 INFO:teuthology.orchestra.run.smithi203.stdout:CentOS-9 - BaseOS 5.9 MB/s | 2.0 MB 00:00 2023-08-09T13:30:57.933 INFO:teuthology.orchestra.run.smithi203.stdout:CentOS Stream 9 - BaseOS 9.7 MB/s | 7.8 MB 00:00 2023-08-09T13:30:58.635 INFO:teuthology.orchestra.run.smithi043.stdout:CentOS Stream 9 - BaseOS 5.1 MB/s | 7.8 MB 00:01 2023-08-09T13:31:01.182 INFO:teuthology.orchestra.run.smithi203.stdout:CentOS Stream 9 - AppStream 8.6 MB/s | 18 MB 00:02 2023-08-09T13:31:02.347 INFO:teuthology.orchestra.run.smithi043.stdout:CentOS Stream 9 - AppStream 7.0 MB/s | 18 MB 00:02 2023-08-09T13:31:05.369 INFO:teuthology.orchestra.run.smithi203.stdout:CentOS Stream 9 - CRB 8.3 MB/s | 5.6 MB 00:00 2023-08-09T13:31:07.498 INFO:teuthology.orchestra.run.smithi203.stdout:CentOS Stream 9 - Extras packages 15 kB/s | 14 kB 00:00 2023-08-09T13:31:07.631 INFO:teuthology.orchestra.run.smithi043.stdout:CentOS Stream 9 - CRB 3.4 MB/s | 5.6 MB 00:01 2023-08-09T13:31:09.744 INFO:teuthology.orchestra.run.smithi043.stdout:CentOS Stream 9 - Extras packages 15 kB/s | 14 kB 00:00 2023-08-09T13:31:11.939 INFO:teuthology.orchestra.run.smithi043.stdout:Extra Packages for Enterprise Linux 9.1 MB/s | 18 MB 00:02 2023-08-09T13:31:13.990 INFO:teuthology.orchestra.run.smithi203.stdout:Extra Packages for Enterprise Linux 2.9 MB/s | 18 MB 00:06 2023-08-09T13:31:19.505 INFO:teuthology.orchestra.run.smithi043.stdout:Package librados2-2:16.2.4-5.el9.x86_64 is already installed. 2023-08-09T13:31:19.507 INFO:teuthology.orchestra.run.smithi043.stdout:Package librbd1-2:16.2.4-5.el9.x86_64 is already installed. 2023-08-09T13:31:19.529 INFO:teuthology.orchestra.run.smithi043.stdout:Package NetworkManager-1:1.41.8-1.el9.x86_64 is already installed. 2023-08-09T13:31:19.531 INFO:teuthology.orchestra.run.smithi043.stdout:Package iproute-6.0.0-2.el9.x86_64 is already installed. 2023-08-09T13:31:19.532 INFO:teuthology.orchestra.run.smithi043.stdout:Package util-linux-2.37.4-9.el9.x86_64 is already installed. 2023-08-09T13:31:19.534 INFO:teuthology.orchestra.run.smithi043.stdout:Package libacl-devel-2.3.1-3.el9.x86_64 is already installed. 2023-08-09T13:31:19.536 INFO:teuthology.orchestra.run.smithi043.stdout:Package libattr-devel-2.5.1-3.el9.x86_64 is already installed. 2023-08-09T13:31:19.540 INFO:teuthology.orchestra.run.smithi043.stdout:Package xfsdump-3.1.10-1.el9.x86_64 is already installed. 2023-08-09T13:31:19.542 INFO:teuthology.orchestra.run.smithi043.stdout:Package xfsprogs-5.14.2-1.el9.x86_64 is already installed. 2023-08-09T13:31:19.631 INFO:teuthology.orchestra.run.smithi043.stdout:Dependencies resolved. 2023-08-09T13:31:19.655 INFO:teuthology.orchestra.run.smithi043.stdout:============================================================================================ 2023-08-09T13:31:19.655 INFO:teuthology.orchestra.run.smithi043.stdout: Package Arch Version Repository Size 2023-08-09T13:31:19.655 INFO:teuthology.orchestra.run.smithi043.stdout:============================================================================================ 2023-08-09T13:31:19.656 INFO:teuthology.orchestra.run.smithi043.stdout:Installing: 2023-08-09T13:31:19.656 INFO:teuthology.orchestra.run.smithi043.stdout: bison x86_64 3.7.4-5.el9 CentOS-AppStream 920 k 2023-08-09T13:31:19.656 INFO:teuthology.orchestra.run.smithi043.stdout: ceph x86_64 2:18.0.0-5389.g668a86f2.el9 ceph 6.5 k 2023-08-09T13:31:19.656 INFO:teuthology.orchestra.run.smithi043.stdout: ceph-base x86_64 2:18.0.0-5389.g668a86f2.el9 ceph 5.2 M 2023-08-09T13:31:19.656 INFO:teuthology.orchestra.run.smithi043.stdout: ceph-fuse x86_64 2:18.0.0-5389.g668a86f2.el9 ceph 849 k 2023-08-09T13:31:19.656 INFO:teuthology.orchestra.run.smithi043.stdout: ceph-immutable-object-cache x86_64 2:18.0.0-5389.g668a86f2.el9 ceph 141 k 2023-08-09T13:31:19.656 INFO:teuthology.orchestra.run.smithi043.stdout: ceph-mgr x86_64 2:18.0.0-5389.g668a86f2.el9 ceph 1.5 M 2023-08-09T13:31:19.657 INFO:teuthology.orchestra.run.smithi043.stdout: ceph-mgr-cephadm noarch 2:18.0.0-5389.g668a86f2.el9 ceph-noarch 131 k 2023-08-09T13:31:19.657 INFO:teuthology.orchestra.run.smithi043.stdout: ceph-mgr-dashboard noarch 2:18.0.0-5389.g668a86f2.el9 ceph-noarch 1.7 M 2023-08-09T13:31:19.657 INFO:teuthology.orchestra.run.smithi043.stdout: ceph-mgr-diskprediction-local noarch 2:18.0.0-5389.g668a86f2.el9 ceph-noarch 7.4 M 2023-08-09T13:31:19.657 INFO:teuthology.orchestra.run.smithi043.stdout: ceph-mgr-rook noarch 2:18.0.0-5389.g668a86f2.el9 ceph-noarch 47 k 2023-08-09T13:31:19.657 INFO:teuthology.orchestra.run.smithi043.stdout: ceph-radosgw x86_64 2:18.0.0-5389.g668a86f2.el9 ceph 8.1 M 2023-08-09T13:31:19.657 INFO:teuthology.orchestra.run.smithi043.stdout: ceph-test x86_64 2:18.0.0-5389.g668a86f2.el9 ceph 40 M 2023-08-09T13:31:19.657 INFO:teuthology.orchestra.run.smithi043.stdout: ceph-volume noarch 2:18.0.0-5389.g668a86f2.el9 ceph-noarch 261 k 2023-08-09T13:31:19.657 INFO:teuthology.orchestra.run.smithi043.stdout: cephadm noarch 2:18.0.0-5389.g668a86f2.el9 ceph-noarch 218 k 2023-08-09T13:31:19.658 INFO:teuthology.orchestra.run.smithi043.stdout: cephfs-mirror x86_64 2:18.0.0-5389.g668a86f2.el9 ceph 218 k 2023-08-09T13:31:19.658 INFO:teuthology.orchestra.run.smithi043.stdout: cephfs-top noarch 2:18.0.0-5389.g668a86f2.el9 ceph-noarch 18 k 2023-08-09T13:31:19.658 INFO:teuthology.orchestra.run.smithi043.stdout: elfutils-libelf-devel x86_64 0.189-3.el9 CentOS-AppStream 21 k 2023-08-09T13:31:19.658 INFO:teuthology.orchestra.run.smithi043.stdout: flex x86_64 2.6.4-9.el9 CentOS-AppStream 307 k 2023-08-09T13:31:19.658 INFO:teuthology.orchestra.run.smithi043.stdout: libaio-devel x86_64 0.3.111-13.el9 CentOS-AppStream 10 k 2023-08-09T13:31:19.658 INFO:teuthology.orchestra.run.smithi043.stdout: libcephfs-devel x86_64 2:18.0.0-5389.g668a86f2.el9 ceph 31 k 2023-08-09T13:31:19.658 INFO:teuthology.orchestra.run.smithi043.stdout: libcephfs2 x86_64 2:18.0.0-5389.g668a86f2.el9 ceph 666 k 2023-08-09T13:31:19.658 INFO:teuthology.orchestra.run.smithi043.stdout: librados-devel x86_64 2:18.0.0-5389.g668a86f2.el9 ceph 126 k 2023-08-09T13:31:19.659 INFO:teuthology.orchestra.run.smithi043.stdout: libtool x86_64 2.4.6-45.el9 CentOS-AppStream 578 k 2023-08-09T13:31:19.659 INFO:teuthology.orchestra.run.smithi043.stdout: libuuid-devel x86_64 2.37.4-11.el9 CentOS-AppStream 24 k 2023-08-09T13:31:19.659 INFO:teuthology.orchestra.run.smithi043.stdout: openssl-devel x86_64 1:3.0.7-24.el9 CentOS-AppStream 3.0 M 2023-08-09T13:31:19.659 INFO:teuthology.orchestra.run.smithi043.stdout: postgresql x86_64 13.11-1.el9 CentOS-AppStream 1.5 M 2023-08-09T13:31:19.659 INFO:teuthology.orchestra.run.smithi043.stdout: postgresql-contrib x86_64 13.11-1.el9 CentOS-AppStream 811 k 2023-08-09T13:31:19.659 INFO:teuthology.orchestra.run.smithi043.stdout: postgresql-server x86_64 13.11-1.el9 CentOS-AppStream 5.7 M 2023-08-09T13:31:19.659 INFO:teuthology.orchestra.run.smithi043.stdout: python3-cephfs x86_64 2:18.0.0-5389.g668a86f2.el9 ceph 162 k 2023-08-09T13:31:19.660 INFO:teuthology.orchestra.run.smithi043.stdout: python3-rados x86_64 2:18.0.0-5389.g668a86f2.el9 ceph 320 k 2023-08-09T13:31:19.660 INFO:teuthology.orchestra.run.smithi043.stdout: python3-rbd x86_64 2:18.0.0-5389.g668a86f2.el9 ceph 297 k 2023-08-09T13:31:19.660 INFO:teuthology.orchestra.run.smithi043.stdout: python3-rgw x86_64 2:18.0.0-5389.g668a86f2.el9 ceph 99 k 2023-08-09T13:31:19.660 INFO:teuthology.orchestra.run.smithi043.stdout: rbd-fuse x86_64 2:18.0.0-5389.g668a86f2.el9 ceph 85 k 2023-08-09T13:31:19.660 INFO:teuthology.orchestra.run.smithi043.stdout: rbd-mirror x86_64 2:18.0.0-5389.g668a86f2.el9 ceph 3.0 M 2023-08-09T13:31:19.660 INFO:teuthology.orchestra.run.smithi043.stdout: rbd-nbd x86_64 2:18.0.0-5389.g668a86f2.el9 ceph 169 k 2023-08-09T13:31:19.660 INFO:teuthology.orchestra.run.smithi043.stdout: xfsprogs-devel x86_64 5.19.0-4.el9 appstream 86 k 2023-08-09T13:31:19.660 INFO:teuthology.orchestra.run.smithi043.stdout:Upgrading: 2023-08-09T13:31:19.661 INFO:teuthology.orchestra.run.smithi043.stdout: NetworkManager x86_64 1:1.43.90-1.el9 CentOS-BaseOS 2.2 M 2023-08-09T13:31:19.661 INFO:teuthology.orchestra.run.smithi043.stdout: NetworkManager-libnm x86_64 1:1.43.90-1.el9 CentOS-BaseOS 1.8 M 2023-08-09T13:31:19.661 INFO:teuthology.orchestra.run.smithi043.stdout: NetworkManager-team x86_64 1:1.43.90-1.el9 CentOS-BaseOS 41 k 2023-08-09T13:31:19.661 INFO:teuthology.orchestra.run.smithi043.stdout: NetworkManager-tui x86_64 1:1.43.90-1.el9 CentOS-BaseOS 245 k 2023-08-09T13:31:19.661 INFO:teuthology.orchestra.run.smithi043.stdout: elfutils-debuginfod-client x86_64 0.189-3.el9 CentOS-BaseOS 36 k 2023-08-09T13:31:19.661 INFO:teuthology.orchestra.run.smithi043.stdout: elfutils-libelf x86_64 0.189-3.el9 CentOS-BaseOS 192 k 2023-08-09T13:31:19.661 INFO:teuthology.orchestra.run.smithi043.stdout: elfutils-libs x86_64 0.189-3.el9 CentOS-BaseOS 254 k 2023-08-09T13:31:19.662 INFO:teuthology.orchestra.run.smithi043.stdout: iproute x86_64 6.2.0-5.el9 CentOS-BaseOS 785 k 2023-08-09T13:31:19.662 INFO:teuthology.orchestra.run.smithi043.stdout: iproute-tc x86_64 6.2.0-5.el9 CentOS-BaseOS 448 k 2023-08-09T13:31:19.662 INFO:teuthology.orchestra.run.smithi043.stdout: libblkid x86_64 2.37.4-11.el9 CentOS-BaseOS 107 k 2023-08-09T13:31:19.662 INFO:teuthology.orchestra.run.smithi043.stdout: libfdisk x86_64 2.37.4-11.el9 CentOS-BaseOS 153 k 2023-08-09T13:31:19.662 INFO:teuthology.orchestra.run.smithi043.stdout: libmount x86_64 2.37.4-11.el9 CentOS-BaseOS 133 k 2023-08-09T13:31:19.662 INFO:teuthology.orchestra.run.smithi043.stdout: librados2 x86_64 2:18.0.0-5389.g668a86f2.el9 ceph 3.3 M 2023-08-09T13:31:19.662 INFO:teuthology.orchestra.run.smithi043.stdout: librbd1 x86_64 2:18.0.0-5389.g668a86f2.el9 ceph 3.0 M 2023-08-09T13:31:19.662 INFO:teuthology.orchestra.run.smithi043.stdout: libsmartcols x86_64 2.37.4-11.el9 CentOS-BaseOS 62 k 2023-08-09T13:31:19.663 INFO:teuthology.orchestra.run.smithi043.stdout: libuuid x86_64 2.37.4-11.el9 CentOS-BaseOS 27 k 2023-08-09T13:31:19.663 INFO:teuthology.orchestra.run.smithi043.stdout: openssl x86_64 1:3.0.7-24.el9 CentOS-BaseOS 1.2 M 2023-08-09T13:31:19.663 INFO:teuthology.orchestra.run.smithi043.stdout: openssl-libs x86_64 1:3.0.7-24.el9 CentOS-BaseOS 2.1 M 2023-08-09T13:31:19.663 INFO:teuthology.orchestra.run.smithi043.stdout: util-linux x86_64 2.37.4-11.el9 CentOS-BaseOS 2.2 M 2023-08-09T13:31:19.663 INFO:teuthology.orchestra.run.smithi043.stdout: util-linux-core x86_64 2.37.4-11.el9 CentOS-BaseOS 432 k 2023-08-09T13:31:19.663 INFO:teuthology.orchestra.run.smithi043.stdout: util-linux-user x86_64 2.37.4-11.el9 CentOS-BaseOS 30 k 2023-08-09T13:31:19.663 INFO:teuthology.orchestra.run.smithi043.stdout: xfsdump x86_64 3.1.12-3.el9 CentOS-BaseOS 321 k 2023-08-09T13:31:19.664 INFO:teuthology.orchestra.run.smithi043.stdout: xfsprogs x86_64 5.19.0-4.el9 baseos 1.0 M 2023-08-09T13:31:19.664 INFO:teuthology.orchestra.run.smithi043.stdout:Installing dependencies: 2023-08-09T13:31:19.664 INFO:teuthology.orchestra.run.smithi043.stdout: automake noarch 1.16.2-8.el9 CentOS-AppStream 662 k 2023-08-09T13:31:19.664 INFO:teuthology.orchestra.run.smithi043.stdout: boost-program-options x86_64 1.75.0-8.el9 CentOS-AppStream 106 k 2023-08-09T13:31:19.664 INFO:teuthology.orchestra.run.smithi043.stdout: ceph-common x86_64 2:18.0.0-5389.g668a86f2.el9 ceph 19 M 2023-08-09T13:31:19.664 INFO:teuthology.orchestra.run.smithi043.stdout: ceph-grafana-dashboards noarch 2:18.0.0-5389.g668a86f2.el9 ceph-noarch 23 k 2023-08-09T13:31:19.665 INFO:teuthology.orchestra.run.smithi043.stdout: ceph-mds x86_64 2:18.0.0-5389.g668a86f2.el9 ceph 2.1 M 2023-08-09T13:31:19.665 INFO:teuthology.orchestra.run.smithi043.stdout: ceph-mgr-modules-core noarch 2:18.0.0-5389.g668a86f2.el9 ceph-noarch 242 k 2023-08-09T13:31:19.665 INFO:teuthology.orchestra.run.smithi043.stdout: ceph-mon x86_64 2:18.0.0-5389.g668a86f2.el9 ceph 4.4 M 2023-08-09T13:31:19.665 INFO:teuthology.orchestra.run.smithi043.stdout: ceph-osd x86_64 2:18.0.0-5389.g668a86f2.el9 ceph 18 M 2023-08-09T13:31:19.665 INFO:teuthology.orchestra.run.smithi043.stdout: ceph-prometheus-alerts noarch 2:18.0.0-5389.g668a86f2.el9 ceph-noarch 15 k 2023-08-09T13:31:19.665 INFO:teuthology.orchestra.run.smithi043.stdout: ceph-selinux x86_64 2:18.0.0-5389.g668a86f2.el9 ceph 24 k 2023-08-09T13:31:19.665 INFO:teuthology.orchestra.run.smithi043.stdout: flexiblas x86_64 3.0.4-8.el9 CentOS-AppStream 32 k 2023-08-09T13:31:19.666 INFO:teuthology.orchestra.run.smithi043.stdout: flexiblas-netlib x86_64 3.0.4-8.el9 CentOS-AppStream 3.0 M 2023-08-09T13:31:19.666 INFO:teuthology.orchestra.run.smithi043.stdout: flexiblas-openblas-openmp x86_64 3.0.4-8.el9 CentOS-AppStream 17 k 2023-08-09T13:31:19.666 INFO:teuthology.orchestra.run.smithi043.stdout: fmt x86_64 8.1.1-5.el9 epel 111 k 2023-08-09T13:31:19.666 INFO:teuthology.orchestra.run.smithi043.stdout: fuse x86_64 2.9.9-15.el9 CentOS-BaseOS 79 k 2023-08-09T13:31:19.666 INFO:teuthology.orchestra.run.smithi043.stdout: gperftools-libs x86_64 2.9.1-2.el9 epel 309 k 2023-08-09T13:31:19.666 INFO:teuthology.orchestra.run.smithi043.stdout: jq x86_64 1.6-15.el9 CentOS-AppStream 186 k 2023-08-09T13:31:19.666 INFO:teuthology.orchestra.run.smithi043.stdout: libarrow x86_64 9.0.0-9.el9 epel 4.4 M 2023-08-09T13:31:19.666 INFO:teuthology.orchestra.run.smithi043.stdout: libarrow-doc noarch 9.0.0-9.el9 epel 24 k 2023-08-09T13:31:19.667 INFO:teuthology.orchestra.run.smithi043.stdout: libcephsqlite x86_64 2:18.0.0-5389.g668a86f2.el9 ceph 164 k 2023-08-09T13:31:19.667 INFO:teuthology.orchestra.run.smithi043.stdout: libgfortran x86_64 11.4.1-2.1.el9 CentOS-BaseOS 807 k 2023-08-09T13:31:19.667 INFO:teuthology.orchestra.run.smithi043.stdout: liboath x86_64 2.6.7-2.el9 epel 49 k 2023-08-09T13:31:19.667 INFO:teuthology.orchestra.run.smithi043.stdout: libpmemobj x86_64 1.12.1-1.el9 CentOS-AppStream 159 k 2023-08-09T13:31:19.667 INFO:teuthology.orchestra.run.smithi043.stdout: libquadmath x86_64 11.4.1-2.1.el9 CentOS-BaseOS 197 k 2023-08-09T13:31:19.667 INFO:teuthology.orchestra.run.smithi043.stdout: librabbitmq x86_64 0.11.0-7.el9 CentOS-AppStream 44 k 2023-08-09T13:31:19.667 INFO:teuthology.orchestra.run.smithi043.stdout: libradosstriper1 x86_64 2:18.0.0-5389.g668a86f2.el9 ceph 473 k 2023-08-09T13:31:19.667 INFO:teuthology.orchestra.run.smithi043.stdout: librdkafka x86_64 1.6.1-102.el9 CentOS-AppStream 661 k 2023-08-09T13:31:19.668 INFO:teuthology.orchestra.run.smithi043.stdout: librgw2 x86_64 2:18.0.0-5389.g668a86f2.el9 ceph 4.7 M 2023-08-09T13:31:19.668 INFO:teuthology.orchestra.run.smithi043.stdout: libunwind x86_64 1.6.2-1.el9 epel 67 k 2023-08-09T13:31:19.668 INFO:teuthology.orchestra.run.smithi043.stdout: libzstd-devel x86_64 1.5.1-2.el9 CentOS-AppStream 46 k 2023-08-09T13:31:19.668 INFO:teuthology.orchestra.run.smithi043.stdout: lttng-ust x86_64 2.12.0-6.el9 CentOS-AppStream 282 k 2023-08-09T13:31:19.668 INFO:teuthology.orchestra.run.smithi043.stdout: oniguruma x86_64 6.9.6-1.el9.5 CentOS-AppStream 217 k 2023-08-09T13:31:19.668 INFO:teuthology.orchestra.run.smithi043.stdout: openblas x86_64 0.3.21-2.el9 CentOS-AppStream 34 k 2023-08-09T13:31:19.668 INFO:teuthology.orchestra.run.smithi043.stdout: openblas-openmp x86_64 0.3.21-2.el9 CentOS-AppStream 4.7 M 2023-08-09T13:31:19.669 INFO:teuthology.orchestra.run.smithi043.stdout: parquet-libs x86_64 9.0.0-9.el9 epel 837 k 2023-08-09T13:31:19.669 INFO:teuthology.orchestra.run.smithi043.stdout: postgresql-private-libs x86_64 13.11-1.el9 CentOS-AppStream 134 k 2023-08-09T13:31:19.669 INFO:teuthology.orchestra.run.smithi043.stdout: python3-asyncssh noarch 2.13.2-3.el9 epel 548 k 2023-08-09T13:31:19.669 INFO:teuthology.orchestra.run.smithi043.stdout: python3-babel noarch 2.9.1-2.el9 CentOS-AppStream 5.8 M 2023-08-09T13:31:19.669 INFO:teuthology.orchestra.run.smithi043.stdout: python3-bcrypt x86_64 3.2.2-1.el9 epel 43 k 2023-08-09T13:31:19.669 INFO:teuthology.orchestra.run.smithi043.stdout: python3-cachetools noarch 4.2.4-1.el9 epel 32 k 2023-08-09T13:31:19.669 INFO:teuthology.orchestra.run.smithi043.stdout: python3-ceph-argparse x86_64 2:18.0.0-5389.g668a86f2.el9 ceph 45 k 2023-08-09T13:31:19.669 INFO:teuthology.orchestra.run.smithi043.stdout: python3-ceph-common x86_64 2:18.0.0-5389.g668a86f2.el9 ceph 125 k 2023-08-09T13:31:19.670 INFO:teuthology.orchestra.run.smithi043.stdout: python3-certifi noarch 2021.10.8-2.el9 epel 15 k 2023-08-09T13:31:19.670 INFO:teuthology.orchestra.run.smithi043.stdout: python3-cffi x86_64 1.14.5-5.el9 CentOS-BaseOS 241 k 2023-08-09T13:31:19.670 INFO:teuthology.orchestra.run.smithi043.stdout: python3-cheroot noarch 8.6.0-4.el9 epel 172 k 2023-08-09T13:31:19.670 INFO:teuthology.orchestra.run.smithi043.stdout: python3-cherrypy noarch 18.6.1-2.el9 epel 358 k 2023-08-09T13:31:19.670 INFO:teuthology.orchestra.run.smithi043.stdout: python3-cryptography x86_64 36.0.1-4.el9 CentOS-BaseOS 1.1 M 2023-08-09T13:31:19.670 INFO:teuthology.orchestra.run.smithi043.stdout: python3-devel x86_64 3.9.16-1.el9 appstream 247 k 2023-08-09T13:31:19.670 INFO:teuthology.orchestra.run.smithi043.stdout: python3-google-auth noarch 1:2.22.0-1.el9 epel 197 k 2023-08-09T13:31:19.670 INFO:teuthology.orchestra.run.smithi043.stdout: python3-jaraco noarch 8.2.1-3.el9 epel 11 k 2023-08-09T13:31:19.671 INFO:teuthology.orchestra.run.smithi043.stdout: python3-jaraco-classes noarch 3.2.1-5.el9 epel 18 k 2023-08-09T13:31:19.671 INFO:teuthology.orchestra.run.smithi043.stdout: python3-jaraco-collections noarch 3.0.0-8.el9 epel 23 k 2023-08-09T13:31:19.671 INFO:teuthology.orchestra.run.smithi043.stdout: python3-jaraco-functools noarch 3.5.0-2.el9 epel 19 k 2023-08-09T13:31:19.671 INFO:teuthology.orchestra.run.smithi043.stdout: python3-jaraco-text noarch 3.2.0-6.el9 epel 20 k 2023-08-09T13:31:19.671 INFO:teuthology.orchestra.run.smithi043.stdout: python3-jinja2 noarch 2.11.3-4.el9 CentOS-AppStream 228 k 2023-08-09T13:31:19.671 INFO:teuthology.orchestra.run.smithi043.stdout: python3-jsonpatch noarch 1.21-16.el9 CentOS-AppStream 24 k 2023-08-09T13:31:19.671 INFO:teuthology.orchestra.run.smithi043.stdout: python3-jsonpointer noarch 2.0-4.el9 CentOS-AppStream 18 k 2023-08-09T13:31:19.671 INFO:teuthology.orchestra.run.smithi043.stdout: python3-jwt noarch 2.4.0-1.el9 epel 41 k 2023-08-09T13:31:19.672 INFO:teuthology.orchestra.run.smithi043.stdout: python3-kubernetes noarch 1:26.1.0-2.el9 epel 1.0 M 2023-08-09T13:31:19.672 INFO:teuthology.orchestra.run.smithi043.stdout: python3-logutils noarch 0.3.5-21.el9 epel 46 k 2023-08-09T13:31:19.672 INFO:teuthology.orchestra.run.smithi043.stdout: python3-mako noarch 1.1.4-6.el9 CentOS-AppStream 144 k 2023-08-09T13:31:19.672 INFO:teuthology.orchestra.run.smithi043.stdout: python3-markupsafe x86_64 1.1.1-12.el9 CentOS-AppStream 32 k 2023-08-09T13:31:19.672 INFO:teuthology.orchestra.run.smithi043.stdout: python3-more-itertools noarch 8.12.0-2.el9 epel 79 k 2023-08-09T13:31:19.672 INFO:teuthology.orchestra.run.smithi043.stdout: python3-natsort noarch 7.1.1-5.el9 epel 58 k 2023-08-09T13:31:19.672 INFO:teuthology.orchestra.run.smithi043.stdout: python3-numpy x86_64 1:1.20.1-5.el9 CentOS-AppStream 4.8 M 2023-08-09T13:31:19.672 INFO:teuthology.orchestra.run.smithi043.stdout: python3-numpy-f2py x86_64 1:1.20.1-5.el9 CentOS-AppStream 217 k 2023-08-09T13:31:19.673 INFO:teuthology.orchestra.run.smithi043.stdout: python3-oauthlib noarch 3.1.1-5.el9 CentOS-AppStream 174 k 2023-08-09T13:31:19.673 INFO:teuthology.orchestra.run.smithi043.stdout: python3-pecan noarch 1.4.2-3.el9 epel 272 k 2023-08-09T13:31:19.673 INFO:teuthology.orchestra.run.smithi043.stdout: python3-ply noarch 3.11-14.el9 CentOS-BaseOS 103 k 2023-08-09T13:31:19.673 INFO:teuthology.orchestra.run.smithi043.stdout: python3-portend noarch 3.1.0-2.el9 epel 16 k 2023-08-09T13:31:19.673 INFO:teuthology.orchestra.run.smithi043.stdout: python3-prettytable noarch 0.7.2-27.el9 CentOS-AppStream 41 k 2023-08-09T13:31:19.673 INFO:teuthology.orchestra.run.smithi043.stdout: python3-pyOpenSSL noarch 21.0.0-1.el9 epel 90 k 2023-08-09T13:31:19.673 INFO:teuthology.orchestra.run.smithi043.stdout: python3-pyasn1 noarch 0.4.8-6.el9 CentOS-AppStream 132 k 2023-08-09T13:31:19.673 INFO:teuthology.orchestra.run.smithi043.stdout: python3-pyasn1-modules noarch 0.4.8-6.el9 CentOS-AppStream 211 k 2023-08-09T13:31:19.674 INFO:teuthology.orchestra.run.smithi043.stdout: python3-pycparser noarch 2.20-6.el9 CentOS-BaseOS 124 k 2023-08-09T13:31:19.674 INFO:teuthology.orchestra.run.smithi043.stdout: python3-pytz noarch 2021.1-5.el9 CentOS-AppStream 47 k 2023-08-09T13:31:19.674 INFO:teuthology.orchestra.run.smithi043.stdout: python3-repoze-lru noarch 0.7-16.el9 epel 31 k 2023-08-09T13:31:19.674 INFO:teuthology.orchestra.run.smithi043.stdout: python3-requests-oauthlib noarch 1.3.0-12.el9 CentOS-AppStream 43 k 2023-08-09T13:31:19.674 INFO:teuthology.orchestra.run.smithi043.stdout: python3-routes noarch 2.5.1-5.el9 epel 188 k 2023-08-09T13:31:19.674 INFO:teuthology.orchestra.run.smithi043.stdout: python3-rsa noarch 4.9-2.el9 epel 59 k 2023-08-09T13:31:19.674 INFO:teuthology.orchestra.run.smithi043.stdout: python3-scipy x86_64 1.6.2-8.el9 CentOS-AppStream 16 M 2023-08-09T13:31:19.674 INFO:teuthology.orchestra.run.smithi043.stdout: python3-tempora noarch 5.0.0-2.el9 epel 36 k 2023-08-09T13:31:19.675 INFO:teuthology.orchestra.run.smithi043.stdout: python3-toml noarch 0.10.2-6.el9 CentOS-AppStream 39 k 2023-08-09T13:31:19.675 INFO:teuthology.orchestra.run.smithi043.stdout: python3-typing-extensions noarch 4.4.0-2.el9 epel 51 k 2023-08-09T13:31:19.675 INFO:teuthology.orchestra.run.smithi043.stdout: python3-webob noarch 1.8.7-6.el9 epel 230 k 2023-08-09T13:31:19.675 INFO:teuthology.orchestra.run.smithi043.stdout: python3-websocket-client noarch 1.2.3-2.el9 epel 90 k 2023-08-09T13:31:19.675 INFO:teuthology.orchestra.run.smithi043.stdout: python3-werkzeug noarch 2.0.3-3.el9.1 epel 427 k 2023-08-09T13:31:19.675 INFO:teuthology.orchestra.run.smithi043.stdout: python3-zc-lockfile noarch 2.0-10.el9 epel 20 k 2023-08-09T13:31:19.675 INFO:teuthology.orchestra.run.smithi043.stdout: re2 x86_64 1:20211101-2.el9 epel 192 k 2023-08-09T13:31:19.675 INFO:teuthology.orchestra.run.smithi043.stdout: socat x86_64 1.7.4.1-5.el9 CentOS-AppStream 300 k 2023-08-09T13:31:19.676 INFO:teuthology.orchestra.run.smithi043.stdout: thrift x86_64 0.15.0-2.el9 epel 1.6 M 2023-08-09T13:31:19.676 INFO:teuthology.orchestra.run.smithi043.stdout: uuid x86_64 1.6.2-55.el9 CentOS-AppStream 56 k 2023-08-09T13:31:19.676 INFO:teuthology.orchestra.run.smithi043.stdout: xmlstarlet x86_64 1.6.1-20.el9 CentOS-AppStream 63 k 2023-08-09T13:31:19.676 INFO:teuthology.orchestra.run.smithi043.stdout: zlib-devel x86_64 1.2.11-36.el9 appstream 45 k 2023-08-09T13:31:19.676 INFO:teuthology.orchestra.run.smithi043.stdout:Installing weak dependencies: 2023-08-09T13:31:19.676 INFO:teuthology.orchestra.run.smithi043.stdout: python3-jwt+crypto noarch 2.4.0-1.el9 epel 9.0 k 2023-08-09T13:31:19.676 INFO:teuthology.orchestra.run.smithi043.stdout: 2023-08-09T13:31:19.677 INFO:teuthology.orchestra.run.smithi043.stdout:Transaction Summary 2023-08-09T13:31:19.677 INFO:teuthology.orchestra.run.smithi043.stdout:============================================================================================ 2023-08-09T13:31:19.677 INFO:teuthology.orchestra.run.smithi043.stdout:Install 131 Packages 2023-08-09T13:31:19.677 INFO:teuthology.orchestra.run.smithi043.stdout:Upgrade 23 Packages 2023-08-09T13:31:19.677 INFO:teuthology.orchestra.run.smithi043.stdout: 2023-08-09T13:31:19.678 INFO:teuthology.orchestra.run.smithi043.stdout:Total download size: 205 M 2023-08-09T13:31:19.678 INFO:teuthology.orchestra.run.smithi043.stdout:Downloading Packages: 2023-08-09T13:31:20.859 INFO:teuthology.orchestra.run.smithi043.stdout:(1/154): ceph-18.0.0-5389.g668a86f2.el9.x86_64. 22 kB/s | 6.5 kB 00:00 2023-08-09T13:31:21.094 INFO:teuthology.orchestra.run.smithi203.stdout:Package librados2-2:16.2.4-5.el9.x86_64 is already installed. 2023-08-09T13:31:21.096 INFO:teuthology.orchestra.run.smithi203.stdout:Package librbd1-2:16.2.4-5.el9.x86_64 is already installed. 2023-08-09T13:31:21.115 INFO:teuthology.orchestra.run.smithi203.stdout:Package NetworkManager-1:1.41.8-1.el9.x86_64 is already installed. 2023-08-09T13:31:21.117 INFO:teuthology.orchestra.run.smithi203.stdout:Package iproute-6.0.0-2.el9.x86_64 is already installed. 2023-08-09T13:31:21.118 INFO:teuthology.orchestra.run.smithi203.stdout:Package util-linux-2.37.4-9.el9.x86_64 is already installed. 2023-08-09T13:31:21.119 INFO:teuthology.orchestra.run.smithi203.stdout:Package libacl-devel-2.3.1-3.el9.x86_64 is already installed. 2023-08-09T13:31:21.122 INFO:teuthology.orchestra.run.smithi203.stdout:Package libattr-devel-2.5.1-3.el9.x86_64 is already installed. 2023-08-09T13:31:21.125 INFO:teuthology.orchestra.run.smithi203.stdout:Package xfsdump-3.1.10-1.el9.x86_64 is already installed. 2023-08-09T13:31:21.126 INFO:teuthology.orchestra.run.smithi203.stdout:Package xfsprogs-5.14.2-1.el9.x86_64 is already installed. 2023-08-09T13:31:21.209 INFO:teuthology.orchestra.run.smithi203.stdout:Dependencies resolved. 2023-08-09T13:31:21.246 INFO:teuthology.orchestra.run.smithi203.stdout:============================================================================================ 2023-08-09T13:31:21.247 INFO:teuthology.orchestra.run.smithi203.stdout: Package Arch Version Repository Size 2023-08-09T13:31:21.247 INFO:teuthology.orchestra.run.smithi203.stdout:============================================================================================ 2023-08-09T13:31:21.247 INFO:teuthology.orchestra.run.smithi203.stdout:Installing: 2023-08-09T13:31:21.247 INFO:teuthology.orchestra.run.smithi203.stdout: bison x86_64 3.7.4-5.el9 CentOS-AppStream 920 k 2023-08-09T13:31:21.247 INFO:teuthology.orchestra.run.smithi203.stdout: ceph x86_64 2:18.0.0-5389.g668a86f2.el9 ceph 6.5 k 2023-08-09T13:31:21.247 INFO:teuthology.orchestra.run.smithi203.stdout: ceph-base x86_64 2:18.0.0-5389.g668a86f2.el9 ceph 5.2 M 2023-08-09T13:31:21.247 INFO:teuthology.orchestra.run.smithi203.stdout: ceph-fuse x86_64 2:18.0.0-5389.g668a86f2.el9 ceph 849 k 2023-08-09T13:31:21.248 INFO:teuthology.orchestra.run.smithi203.stdout: ceph-immutable-object-cache x86_64 2:18.0.0-5389.g668a86f2.el9 ceph 141 k 2023-08-09T13:31:21.248 INFO:teuthology.orchestra.run.smithi203.stdout: ceph-mgr x86_64 2:18.0.0-5389.g668a86f2.el9 ceph 1.5 M 2023-08-09T13:31:21.248 INFO:teuthology.orchestra.run.smithi203.stdout: ceph-mgr-cephadm noarch 2:18.0.0-5389.g668a86f2.el9 ceph-noarch 131 k 2023-08-09T13:31:21.248 INFO:teuthology.orchestra.run.smithi203.stdout: ceph-mgr-dashboard noarch 2:18.0.0-5389.g668a86f2.el9 ceph-noarch 1.7 M 2023-08-09T13:31:21.248 INFO:teuthology.orchestra.run.smithi203.stdout: ceph-mgr-diskprediction-local noarch 2:18.0.0-5389.g668a86f2.el9 ceph-noarch 7.4 M 2023-08-09T13:31:21.248 INFO:teuthology.orchestra.run.smithi203.stdout: ceph-mgr-rook noarch 2:18.0.0-5389.g668a86f2.el9 ceph-noarch 47 k 2023-08-09T13:31:21.248 INFO:teuthology.orchestra.run.smithi203.stdout: ceph-radosgw x86_64 2:18.0.0-5389.g668a86f2.el9 ceph 8.1 M 2023-08-09T13:31:21.249 INFO:teuthology.orchestra.run.smithi203.stdout: ceph-test x86_64 2:18.0.0-5389.g668a86f2.el9 ceph 40 M 2023-08-09T13:31:21.249 INFO:teuthology.orchestra.run.smithi203.stdout: ceph-volume noarch 2:18.0.0-5389.g668a86f2.el9 ceph-noarch 261 k 2023-08-09T13:31:21.249 INFO:teuthology.orchestra.run.smithi203.stdout: cephadm noarch 2:18.0.0-5389.g668a86f2.el9 ceph-noarch 218 k 2023-08-09T13:31:21.249 INFO:teuthology.orchestra.run.smithi203.stdout: cephfs-mirror x86_64 2:18.0.0-5389.g668a86f2.el9 ceph 218 k 2023-08-09T13:31:21.249 INFO:teuthology.orchestra.run.smithi203.stdout: cephfs-top noarch 2:18.0.0-5389.g668a86f2.el9 ceph-noarch 18 k 2023-08-09T13:31:21.249 INFO:teuthology.orchestra.run.smithi203.stdout: elfutils-libelf-devel x86_64 0.189-3.el9 CentOS-AppStream 21 k 2023-08-09T13:31:21.249 INFO:teuthology.orchestra.run.smithi203.stdout: flex x86_64 2.6.4-9.el9 CentOS-AppStream 307 k 2023-08-09T13:31:21.249 INFO:teuthology.orchestra.run.smithi203.stdout: libaio-devel x86_64 0.3.111-13.el9 CentOS-AppStream 10 k 2023-08-09T13:31:21.250 INFO:teuthology.orchestra.run.smithi203.stdout: libcephfs-devel x86_64 2:18.0.0-5389.g668a86f2.el9 ceph 31 k 2023-08-09T13:31:21.250 INFO:teuthology.orchestra.run.smithi203.stdout: libcephfs2 x86_64 2:18.0.0-5389.g668a86f2.el9 ceph 666 k 2023-08-09T13:31:21.250 INFO:teuthology.orchestra.run.smithi203.stdout: librados-devel x86_64 2:18.0.0-5389.g668a86f2.el9 ceph 126 k 2023-08-09T13:31:21.250 INFO:teuthology.orchestra.run.smithi203.stdout: libtool x86_64 2.4.6-45.el9 CentOS-AppStream 578 k 2023-08-09T13:31:21.250 INFO:teuthology.orchestra.run.smithi203.stdout: libuuid-devel x86_64 2.37.4-11.el9 CentOS-AppStream 24 k 2023-08-09T13:31:21.250 INFO:teuthology.orchestra.run.smithi203.stdout: openssl-devel x86_64 1:3.0.7-24.el9 CentOS-AppStream 3.0 M 2023-08-09T13:31:21.250 INFO:teuthology.orchestra.run.smithi203.stdout: postgresql x86_64 13.11-1.el9 CentOS-AppStream 1.5 M 2023-08-09T13:31:21.250 INFO:teuthology.orchestra.run.smithi203.stdout: postgresql-contrib x86_64 13.11-1.el9 CentOS-AppStream 811 k 2023-08-09T13:31:21.251 INFO:teuthology.orchestra.run.smithi203.stdout: postgresql-server x86_64 13.11-1.el9 CentOS-AppStream 5.7 M 2023-08-09T13:31:21.251 INFO:teuthology.orchestra.run.smithi203.stdout: python3-cephfs x86_64 2:18.0.0-5389.g668a86f2.el9 ceph 162 k 2023-08-09T13:31:21.251 INFO:teuthology.orchestra.run.smithi203.stdout: python3-rados x86_64 2:18.0.0-5389.g668a86f2.el9 ceph 320 k 2023-08-09T13:31:21.251 INFO:teuthology.orchestra.run.smithi203.stdout: python3-rbd x86_64 2:18.0.0-5389.g668a86f2.el9 ceph 297 k 2023-08-09T13:31:21.251 INFO:teuthology.orchestra.run.smithi203.stdout: python3-rgw x86_64 2:18.0.0-5389.g668a86f2.el9 ceph 99 k 2023-08-09T13:31:21.251 INFO:teuthology.orchestra.run.smithi203.stdout: rbd-fuse x86_64 2:18.0.0-5389.g668a86f2.el9 ceph 85 k 2023-08-09T13:31:21.251 INFO:teuthology.orchestra.run.smithi203.stdout: rbd-mirror x86_64 2:18.0.0-5389.g668a86f2.el9 ceph 3.0 M 2023-08-09T13:31:21.252 INFO:teuthology.orchestra.run.smithi203.stdout: rbd-nbd x86_64 2:18.0.0-5389.g668a86f2.el9 ceph 169 k 2023-08-09T13:31:21.252 INFO:teuthology.orchestra.run.smithi203.stdout: xfsprogs-devel x86_64 5.19.0-4.el9 appstream 86 k 2023-08-09T13:31:21.252 INFO:teuthology.orchestra.run.smithi203.stdout:Upgrading: 2023-08-09T13:31:21.252 INFO:teuthology.orchestra.run.smithi203.stdout: NetworkManager x86_64 1:1.43.90-1.el9 CentOS-BaseOS 2.2 M 2023-08-09T13:31:21.252 INFO:teuthology.orchestra.run.smithi203.stdout: NetworkManager-libnm x86_64 1:1.43.90-1.el9 CentOS-BaseOS 1.8 M 2023-08-09T13:31:21.252 INFO:teuthology.orchestra.run.smithi203.stdout: NetworkManager-team x86_64 1:1.43.90-1.el9 CentOS-BaseOS 41 k 2023-08-09T13:31:21.252 INFO:teuthology.orchestra.run.smithi203.stdout: NetworkManager-tui x86_64 1:1.43.90-1.el9 CentOS-BaseOS 245 k 2023-08-09T13:31:21.252 INFO:teuthology.orchestra.run.smithi203.stdout: elfutils-debuginfod-client x86_64 0.189-3.el9 CentOS-BaseOS 36 k 2023-08-09T13:31:21.253 INFO:teuthology.orchestra.run.smithi203.stdout: elfutils-libelf x86_64 0.189-3.el9 CentOS-BaseOS 192 k 2023-08-09T13:31:21.253 INFO:teuthology.orchestra.run.smithi203.stdout: elfutils-libs x86_64 0.189-3.el9 CentOS-BaseOS 254 k 2023-08-09T13:31:21.253 INFO:teuthology.orchestra.run.smithi203.stdout: iproute x86_64 6.2.0-5.el9 CentOS-BaseOS 785 k 2023-08-09T13:31:21.253 INFO:teuthology.orchestra.run.smithi203.stdout: iproute-tc x86_64 6.2.0-5.el9 CentOS-BaseOS 448 k 2023-08-09T13:31:21.253 INFO:teuthology.orchestra.run.smithi203.stdout: libblkid x86_64 2.37.4-11.el9 CentOS-BaseOS 107 k 2023-08-09T13:31:21.253 INFO:teuthology.orchestra.run.smithi203.stdout: libfdisk x86_64 2.37.4-11.el9 CentOS-BaseOS 153 k 2023-08-09T13:31:21.253 INFO:teuthology.orchestra.run.smithi203.stdout: libmount x86_64 2.37.4-11.el9 CentOS-BaseOS 133 k 2023-08-09T13:31:21.253 INFO:teuthology.orchestra.run.smithi203.stdout: librados2 x86_64 2:18.0.0-5389.g668a86f2.el9 ceph 3.3 M 2023-08-09T13:31:21.254 INFO:teuthology.orchestra.run.smithi203.stdout: librbd1 x86_64 2:18.0.0-5389.g668a86f2.el9 ceph 3.0 M 2023-08-09T13:31:21.254 INFO:teuthology.orchestra.run.smithi203.stdout: libsmartcols x86_64 2.37.4-11.el9 CentOS-BaseOS 62 k 2023-08-09T13:31:21.254 INFO:teuthology.orchestra.run.smithi203.stdout: libuuid x86_64 2.37.4-11.el9 CentOS-BaseOS 27 k 2023-08-09T13:31:21.254 INFO:teuthology.orchestra.run.smithi203.stdout: openssl x86_64 1:3.0.7-24.el9 CentOS-BaseOS 1.2 M 2023-08-09T13:31:21.254 INFO:teuthology.orchestra.run.smithi203.stdout: openssl-libs x86_64 1:3.0.7-24.el9 CentOS-BaseOS 2.1 M 2023-08-09T13:31:21.254 INFO:teuthology.orchestra.run.smithi203.stdout: util-linux x86_64 2.37.4-11.el9 CentOS-BaseOS 2.2 M 2023-08-09T13:31:21.254 INFO:teuthology.orchestra.run.smithi203.stdout: util-linux-core x86_64 2.37.4-11.el9 CentOS-BaseOS 432 k 2023-08-09T13:31:21.255 INFO:teuthology.orchestra.run.smithi203.stdout: util-linux-user x86_64 2.37.4-11.el9 CentOS-BaseOS 30 k 2023-08-09T13:31:21.255 INFO:teuthology.orchestra.run.smithi203.stdout: xfsdump x86_64 3.1.12-3.el9 CentOS-BaseOS 321 k 2023-08-09T13:31:21.255 INFO:teuthology.orchestra.run.smithi203.stdout: xfsprogs x86_64 5.19.0-4.el9 baseos 1.0 M 2023-08-09T13:31:21.255 INFO:teuthology.orchestra.run.smithi203.stdout:Installing dependencies: 2023-08-09T13:31:21.255 INFO:teuthology.orchestra.run.smithi203.stdout: automake noarch 1.16.2-8.el9 CentOS-AppStream 662 k 2023-08-09T13:31:21.255 INFO:teuthology.orchestra.run.smithi203.stdout: boost-program-options x86_64 1.75.0-8.el9 CentOS-AppStream 106 k 2023-08-09T13:31:21.255 INFO:teuthology.orchestra.run.smithi203.stdout: ceph-common x86_64 2:18.0.0-5389.g668a86f2.el9 ceph 19 M 2023-08-09T13:31:21.255 INFO:teuthology.orchestra.run.smithi203.stdout: ceph-grafana-dashboards noarch 2:18.0.0-5389.g668a86f2.el9 ceph-noarch 23 k 2023-08-09T13:31:21.256 INFO:teuthology.orchestra.run.smithi203.stdout: ceph-mds x86_64 2:18.0.0-5389.g668a86f2.el9 ceph 2.1 M 2023-08-09T13:31:21.256 INFO:teuthology.orchestra.run.smithi203.stdout: ceph-mgr-modules-core noarch 2:18.0.0-5389.g668a86f2.el9 ceph-noarch 242 k 2023-08-09T13:31:21.256 INFO:teuthology.orchestra.run.smithi203.stdout: ceph-mon x86_64 2:18.0.0-5389.g668a86f2.el9 ceph 4.4 M 2023-08-09T13:31:21.256 INFO:teuthology.orchestra.run.smithi203.stdout: ceph-osd x86_64 2:18.0.0-5389.g668a86f2.el9 ceph 18 M 2023-08-09T13:31:21.256 INFO:teuthology.orchestra.run.smithi203.stdout: ceph-prometheus-alerts noarch 2:18.0.0-5389.g668a86f2.el9 ceph-noarch 15 k 2023-08-09T13:31:21.256 INFO:teuthology.orchestra.run.smithi203.stdout: ceph-selinux x86_64 2:18.0.0-5389.g668a86f2.el9 ceph 24 k 2023-08-09T13:31:21.256 INFO:teuthology.orchestra.run.smithi203.stdout: flexiblas x86_64 3.0.4-8.el9 CentOS-AppStream 32 k 2023-08-09T13:31:21.256 INFO:teuthology.orchestra.run.smithi203.stdout: flexiblas-netlib x86_64 3.0.4-8.el9 CentOS-AppStream 3.0 M 2023-08-09T13:31:21.257 INFO:teuthology.orchestra.run.smithi203.stdout: flexiblas-openblas-openmp x86_64 3.0.4-8.el9 CentOS-AppStream 17 k 2023-08-09T13:31:21.257 INFO:teuthology.orchestra.run.smithi203.stdout: fmt x86_64 8.1.1-5.el9 epel 111 k 2023-08-09T13:31:21.257 INFO:teuthology.orchestra.run.smithi203.stdout: fuse x86_64 2.9.9-15.el9 CentOS-BaseOS 79 k 2023-08-09T13:31:21.257 INFO:teuthology.orchestra.run.smithi203.stdout: gperftools-libs x86_64 2.9.1-2.el9 epel 309 k 2023-08-09T13:31:21.257 INFO:teuthology.orchestra.run.smithi203.stdout: jq x86_64 1.6-15.el9 CentOS-AppStream 186 k 2023-08-09T13:31:21.257 INFO:teuthology.orchestra.run.smithi203.stdout: libarrow x86_64 9.0.0-9.el9 epel 4.4 M 2023-08-09T13:31:21.257 INFO:teuthology.orchestra.run.smithi203.stdout: libarrow-doc noarch 9.0.0-9.el9 epel 24 k 2023-08-09T13:31:21.257 INFO:teuthology.orchestra.run.smithi203.stdout: libcephsqlite x86_64 2:18.0.0-5389.g668a86f2.el9 ceph 164 k 2023-08-09T13:31:21.258 INFO:teuthology.orchestra.run.smithi203.stdout: libgfortran x86_64 11.4.1-2.1.el9 CentOS-BaseOS 807 k 2023-08-09T13:31:21.258 INFO:teuthology.orchestra.run.smithi203.stdout: liboath x86_64 2.6.7-2.el9 epel 49 k 2023-08-09T13:31:21.258 INFO:teuthology.orchestra.run.smithi203.stdout: libpmemobj x86_64 1.12.1-1.el9 CentOS-AppStream 159 k 2023-08-09T13:31:21.258 INFO:teuthology.orchestra.run.smithi203.stdout: libquadmath x86_64 11.4.1-2.1.el9 CentOS-BaseOS 197 k 2023-08-09T13:31:21.258 INFO:teuthology.orchestra.run.smithi203.stdout: librabbitmq x86_64 0.11.0-7.el9 CentOS-AppStream 44 k 2023-08-09T13:31:21.258 INFO:teuthology.orchestra.run.smithi203.stdout: libradosstriper1 x86_64 2:18.0.0-5389.g668a86f2.el9 ceph 473 k 2023-08-09T13:31:21.258 INFO:teuthology.orchestra.run.smithi203.stdout: librdkafka x86_64 1.6.1-102.el9 CentOS-AppStream 661 k 2023-08-09T13:31:21.259 INFO:teuthology.orchestra.run.smithi203.stdout: librgw2 x86_64 2:18.0.0-5389.g668a86f2.el9 ceph 4.7 M 2023-08-09T13:31:21.259 INFO:teuthology.orchestra.run.smithi203.stdout: libunwind x86_64 1.6.2-1.el9 epel 67 k 2023-08-09T13:31:21.259 INFO:teuthology.orchestra.run.smithi203.stdout: libzstd-devel x86_64 1.5.1-2.el9 CentOS-AppStream 46 k 2023-08-09T13:31:21.259 INFO:teuthology.orchestra.run.smithi203.stdout: lttng-ust x86_64 2.12.0-6.el9 CentOS-AppStream 282 k 2023-08-09T13:31:21.259 INFO:teuthology.orchestra.run.smithi203.stdout: oniguruma x86_64 6.9.6-1.el9.5 CentOS-AppStream 217 k 2023-08-09T13:31:21.259 INFO:teuthology.orchestra.run.smithi203.stdout: openblas x86_64 0.3.21-2.el9 CentOS-AppStream 34 k 2023-08-09T13:31:21.259 INFO:teuthology.orchestra.run.smithi203.stdout: openblas-openmp x86_64 0.3.21-2.el9 CentOS-AppStream 4.7 M 2023-08-09T13:31:21.259 INFO:teuthology.orchestra.run.smithi203.stdout: parquet-libs x86_64 9.0.0-9.el9 epel 837 k 2023-08-09T13:31:21.260 INFO:teuthology.orchestra.run.smithi203.stdout: postgresql-private-libs x86_64 13.11-1.el9 CentOS-AppStream 134 k 2023-08-09T13:31:21.260 INFO:teuthology.orchestra.run.smithi203.stdout: python3-asyncssh noarch 2.13.2-3.el9 epel 548 k 2023-08-09T13:31:21.260 INFO:teuthology.orchestra.run.smithi203.stdout: python3-babel noarch 2.9.1-2.el9 CentOS-AppStream 5.8 M 2023-08-09T13:31:21.260 INFO:teuthology.orchestra.run.smithi203.stdout: python3-bcrypt x86_64 3.2.2-1.el9 epel 43 k 2023-08-09T13:31:21.260 INFO:teuthology.orchestra.run.smithi203.stdout: python3-cachetools noarch 4.2.4-1.el9 epel 32 k 2023-08-09T13:31:21.260 INFO:teuthology.orchestra.run.smithi203.stdout: python3-ceph-argparse x86_64 2:18.0.0-5389.g668a86f2.el9 ceph 45 k 2023-08-09T13:31:21.260 INFO:teuthology.orchestra.run.smithi203.stdout: python3-ceph-common x86_64 2:18.0.0-5389.g668a86f2.el9 ceph 125 k 2023-08-09T13:31:21.260 INFO:teuthology.orchestra.run.smithi203.stdout: python3-certifi noarch 2021.10.8-2.el9 epel 15 k 2023-08-09T13:31:21.261 INFO:teuthology.orchestra.run.smithi203.stdout: python3-cffi x86_64 1.14.5-5.el9 CentOS-BaseOS 241 k 2023-08-09T13:31:21.261 INFO:teuthology.orchestra.run.smithi203.stdout: python3-cheroot noarch 8.6.0-4.el9 epel 172 k 2023-08-09T13:31:21.261 INFO:teuthology.orchestra.run.smithi203.stdout: python3-cherrypy noarch 18.6.1-2.el9 epel 358 k 2023-08-09T13:31:21.261 INFO:teuthology.orchestra.run.smithi203.stdout: python3-cryptography x86_64 36.0.1-4.el9 CentOS-BaseOS 1.1 M 2023-08-09T13:31:21.261 INFO:teuthology.orchestra.run.smithi203.stdout: python3-devel x86_64 3.9.16-1.el9 appstream 247 k 2023-08-09T13:31:21.261 INFO:teuthology.orchestra.run.smithi203.stdout: python3-google-auth noarch 1:2.22.0-1.el9 epel 197 k 2023-08-09T13:31:21.262 INFO:teuthology.orchestra.run.smithi203.stdout: python3-jaraco noarch 8.2.1-3.el9 epel 11 k 2023-08-09T13:31:21.262 INFO:teuthology.orchestra.run.smithi203.stdout: python3-jaraco-classes noarch 3.2.1-5.el9 epel 18 k 2023-08-09T13:31:21.262 INFO:teuthology.orchestra.run.smithi203.stdout: python3-jaraco-collections noarch 3.0.0-8.el9 epel 23 k 2023-08-09T13:31:21.262 INFO:teuthology.orchestra.run.smithi203.stdout: python3-jaraco-functools noarch 3.5.0-2.el9 epel 19 k 2023-08-09T13:31:21.262 INFO:teuthology.orchestra.run.smithi203.stdout: python3-jaraco-text noarch 3.2.0-6.el9 epel 20 k 2023-08-09T13:31:21.262 INFO:teuthology.orchestra.run.smithi203.stdout: python3-jinja2 noarch 2.11.3-4.el9 CentOS-AppStream 228 k 2023-08-09T13:31:21.262 INFO:teuthology.orchestra.run.smithi203.stdout: python3-jsonpatch noarch 1.21-16.el9 CentOS-AppStream 24 k 2023-08-09T13:31:21.262 INFO:teuthology.orchestra.run.smithi203.stdout: python3-jsonpointer noarch 2.0-4.el9 CentOS-AppStream 18 k 2023-08-09T13:31:21.263 INFO:teuthology.orchestra.run.smithi203.stdout: python3-jwt noarch 2.4.0-1.el9 epel 41 k 2023-08-09T13:31:21.263 INFO:teuthology.orchestra.run.smithi203.stdout: python3-kubernetes noarch 1:26.1.0-2.el9 epel 1.0 M 2023-08-09T13:31:21.263 INFO:teuthology.orchestra.run.smithi203.stdout: python3-logutils noarch 0.3.5-21.el9 epel 46 k 2023-08-09T13:31:21.263 INFO:teuthology.orchestra.run.smithi203.stdout: python3-mako noarch 1.1.4-6.el9 CentOS-AppStream 144 k 2023-08-09T13:31:21.263 INFO:teuthology.orchestra.run.smithi203.stdout: python3-markupsafe x86_64 1.1.1-12.el9 CentOS-AppStream 32 k 2023-08-09T13:31:21.263 INFO:teuthology.orchestra.run.smithi203.stdout: python3-more-itertools noarch 8.12.0-2.el9 epel 79 k 2023-08-09T13:31:21.263 INFO:teuthology.orchestra.run.smithi203.stdout: python3-natsort noarch 7.1.1-5.el9 epel 58 k 2023-08-09T13:31:21.263 INFO:teuthology.orchestra.run.smithi203.stdout: python3-numpy x86_64 1:1.20.1-5.el9 CentOS-AppStream 4.8 M 2023-08-09T13:31:21.264 INFO:teuthology.orchestra.run.smithi203.stdout: python3-numpy-f2py x86_64 1:1.20.1-5.el9 CentOS-AppStream 217 k 2023-08-09T13:31:21.264 INFO:teuthology.orchestra.run.smithi203.stdout: python3-oauthlib noarch 3.1.1-5.el9 CentOS-AppStream 174 k 2023-08-09T13:31:21.264 INFO:teuthology.orchestra.run.smithi203.stdout: python3-pecan noarch 1.4.2-3.el9 epel 272 k 2023-08-09T13:31:21.264 INFO:teuthology.orchestra.run.smithi203.stdout: python3-ply noarch 3.11-14.el9 CentOS-BaseOS 103 k 2023-08-09T13:31:21.264 INFO:teuthology.orchestra.run.smithi203.stdout: python3-portend noarch 3.1.0-2.el9 epel 16 k 2023-08-09T13:31:21.264 INFO:teuthology.orchestra.run.smithi203.stdout: python3-prettytable noarch 0.7.2-27.el9 CentOS-AppStream 41 k 2023-08-09T13:31:21.264 INFO:teuthology.orchestra.run.smithi203.stdout: python3-pyOpenSSL noarch 21.0.0-1.el9 epel 90 k 2023-08-09T13:31:21.265 INFO:teuthology.orchestra.run.smithi203.stdout: python3-pyasn1 noarch 0.4.8-6.el9 CentOS-AppStream 132 k 2023-08-09T13:31:21.265 INFO:teuthology.orchestra.run.smithi203.stdout: python3-pyasn1-modules noarch 0.4.8-6.el9 CentOS-AppStream 211 k 2023-08-09T13:31:21.265 INFO:teuthology.orchestra.run.smithi203.stdout: python3-pycparser noarch 2.20-6.el9 CentOS-BaseOS 124 k 2023-08-09T13:31:21.265 INFO:teuthology.orchestra.run.smithi203.stdout: python3-pytz noarch 2021.1-5.el9 CentOS-AppStream 47 k 2023-08-09T13:31:21.265 INFO:teuthology.orchestra.run.smithi203.stdout: python3-repoze-lru noarch 0.7-16.el9 epel 31 k 2023-08-09T13:31:21.265 INFO:teuthology.orchestra.run.smithi203.stdout: python3-requests-oauthlib noarch 1.3.0-12.el9 CentOS-AppStream 43 k 2023-08-09T13:31:21.265 INFO:teuthology.orchestra.run.smithi203.stdout: python3-routes noarch 2.5.1-5.el9 epel 188 k 2023-08-09T13:31:21.265 INFO:teuthology.orchestra.run.smithi203.stdout: python3-rsa noarch 4.9-2.el9 epel 59 k 2023-08-09T13:31:21.266 INFO:teuthology.orchestra.run.smithi203.stdout: python3-scipy x86_64 1.6.2-8.el9 CentOS-AppStream 16 M 2023-08-09T13:31:21.266 INFO:teuthology.orchestra.run.smithi203.stdout: python3-tempora noarch 5.0.0-2.el9 epel 36 k 2023-08-09T13:31:21.266 INFO:teuthology.orchestra.run.smithi203.stdout: python3-toml noarch 0.10.2-6.el9 CentOS-AppStream 39 k 2023-08-09T13:31:21.266 INFO:teuthology.orchestra.run.smithi203.stdout: python3-typing-extensions noarch 4.4.0-2.el9 epel 51 k 2023-08-09T13:31:21.266 INFO:teuthology.orchestra.run.smithi203.stdout: python3-webob noarch 1.8.7-6.el9 epel 230 k 2023-08-09T13:31:21.266 INFO:teuthology.orchestra.run.smithi203.stdout: python3-websocket-client noarch 1.2.3-2.el9 epel 90 k 2023-08-09T13:31:21.266 INFO:teuthology.orchestra.run.smithi203.stdout: python3-werkzeug noarch 2.0.3-3.el9.1 epel 427 k 2023-08-09T13:31:21.267 INFO:teuthology.orchestra.run.smithi203.stdout: python3-zc-lockfile noarch 2.0-10.el9 epel 20 k 2023-08-09T13:31:21.267 INFO:teuthology.orchestra.run.smithi203.stdout: re2 x86_64 1:20211101-2.el9 epel 192 k 2023-08-09T13:31:21.267 INFO:teuthology.orchestra.run.smithi203.stdout: socat x86_64 1.7.4.1-5.el9 CentOS-AppStream 300 k 2023-08-09T13:31:21.267 INFO:teuthology.orchestra.run.smithi203.stdout: thrift x86_64 0.15.0-2.el9 epel 1.6 M 2023-08-09T13:31:21.267 INFO:teuthology.orchestra.run.smithi203.stdout: uuid x86_64 1.6.2-55.el9 CentOS-AppStream 56 k 2023-08-09T13:31:21.267 INFO:teuthology.orchestra.run.smithi203.stdout: xmlstarlet x86_64 1.6.1-20.el9 CentOS-AppStream 63 k 2023-08-09T13:31:21.267 INFO:teuthology.orchestra.run.smithi203.stdout: zlib-devel x86_64 1.2.11-36.el9 appstream 45 k 2023-08-09T13:31:21.268 INFO:teuthology.orchestra.run.smithi203.stdout:Installing weak dependencies: 2023-08-09T13:31:21.268 INFO:teuthology.orchestra.run.smithi203.stdout: python3-jwt+crypto noarch 2.4.0-1.el9 epel 9.0 k 2023-08-09T13:31:21.268 INFO:teuthology.orchestra.run.smithi203.stdout: 2023-08-09T13:31:21.268 INFO:teuthology.orchestra.run.smithi203.stdout:Transaction Summary 2023-08-09T13:31:21.268 INFO:teuthology.orchestra.run.smithi203.stdout:============================================================================================ 2023-08-09T13:31:21.268 INFO:teuthology.orchestra.run.smithi203.stdout:Install 131 Packages 2023-08-09T13:31:21.268 INFO:teuthology.orchestra.run.smithi203.stdout:Upgrade 23 Packages 2023-08-09T13:31:21.268 INFO:teuthology.orchestra.run.smithi203.stdout: 2023-08-09T13:31:21.269 INFO:teuthology.orchestra.run.smithi203.stdout:Total download size: 205 M 2023-08-09T13:31:21.269 INFO:teuthology.orchestra.run.smithi203.stdout:Downloading Packages: 2023-08-09T13:31:21.276 INFO:teuthology.orchestra.run.smithi043.stdout:(2/154): ceph-fuse-18.0.0-5389.g668a86f2.el9.x8 2.0 MB/s | 849 kB 00:00 2023-08-09T13:31:21.376 INFO:teuthology.orchestra.run.smithi043.stdout:(3/154): ceph-immutable-object-cache-18.0.0-538 1.4 MB/s | 141 kB 00:00 2023-08-09T13:31:21.902 INFO:teuthology.orchestra.run.smithi043.stdout:(4/154): ceph-mds-18.0.0-5389.g668a86f2.el9.x86 4.0 MB/s | 2.1 MB 00:00 2023-08-09T13:31:22.127 INFO:teuthology.orchestra.run.smithi043.stdout:(5/154): ceph-base-18.0.0-5389.g668a86f2.el9.x8 3.3 MB/s | 5.2 MB 00:01 2023-08-09T13:31:22.172 INFO:teuthology.orchestra.run.smithi203.stdout:(1/154): ceph-18.0.0-5389.g668a86f2.el9.x86_64. 20 kB/s | 6.5 kB 00:00 2023-08-09T13:31:22.232 INFO:teuthology.orchestra.run.smithi043.stdout:(6/154): ceph-mgr-18.0.0-5389.g668a86f2.el9.x86 4.6 MB/s | 1.5 MB 00:00 2023-08-09T13:31:22.755 INFO:teuthology.orchestra.run.smithi203.stdout:(2/154): ceph-fuse-18.0.0-5389.g668a86f2.el9.x8 1.4 MB/s | 849 kB 00:00 2023-08-09T13:31:22.819 INFO:teuthology.orchestra.run.smithi043.stdout:(7/154): ceph-mon-18.0.0-5389.g668a86f2.el9.x86 6.3 MB/s | 4.4 MB 00:00 2023-08-09T13:31:22.881 INFO:teuthology.orchestra.run.smithi203.stdout:(3/154): ceph-immutable-object-cache-18.0.0-538 1.1 MB/s | 141 kB 00:00 2023-08-09T13:31:23.586 INFO:teuthology.orchestra.run.smithi203.stdout:(4/154): ceph-base-18.0.0-5389.g668a86f2.el9.x8 3.0 MB/s | 5.2 MB 00:01 2023-08-09T13:31:23.940 INFO:teuthology.orchestra.run.smithi203.stdout:(5/154): ceph-mgr-18.0.0-5389.g668a86f2.el9.x86 4.1 MB/s | 1.5 MB 00:00 2023-08-09T13:31:24.023 INFO:teuthology.orchestra.run.smithi203.stdout:(6/154): ceph-mds-18.0.0-5389.g668a86f2.el9.x86 1.8 MB/s | 2.1 MB 00:01 2023-08-09T13:31:24.036 INFO:teuthology.orchestra.run.smithi043.stdout:(8/154): ceph-common-18.0.0-5389.g668a86f2.el9. 5.4 MB/s | 19 MB 00:03 2023-08-09T13:31:24.211 INFO:teuthology.orchestra.run.smithi043.stdout:(9/154): ceph-radosgw-18.0.0-5389.g668a86f2.el9 5.8 MB/s | 8.1 MB 00:01 2023-08-09T13:31:24.236 INFO:teuthology.orchestra.run.smithi043.stdout:(10/154): ceph-selinux-18.0.0-5389.g668a86f2.el 121 kB/s | 24 kB 00:00 2023-08-09T13:31:24.345 INFO:teuthology.orchestra.run.smithi043.stdout:(11/154): cephfs-mirror-18.0.0-5389.g668a86f2.e 2.0 MB/s | 218 kB 00:00 2023-08-09T13:31:24.629 INFO:teuthology.orchestra.run.smithi043.stdout:(12/154): ceph-osd-18.0.0-5389.g668a86f2.el9.x8 7.3 MB/s | 18 MB 00:02 2023-08-09T13:31:24.654 INFO:teuthology.orchestra.run.smithi043.stdout:(13/154): libcephfs-devel-18.0.0-5389.g668a86f2 101 kB/s | 31 kB 00:00 2023-08-09T13:31:24.657 INFO:teuthology.orchestra.run.smithi203.stdout:(7/154): ceph-mon-18.0.0-5389.g668a86f2.el9.x86 6.1 MB/s | 4.4 MB 00:00 2023-08-09T13:31:24.746 INFO:teuthology.orchestra.run.smithi043.stdout:(14/154): libcephsqlite-18.0.0-5389.g668a86f2.e 1.8 MB/s | 164 kB 00:00 2023-08-09T13:31:24.796 INFO:teuthology.orchestra.run.smithi043.stdout:(15/154): libcephfs2-18.0.0-5389.g668a86f2.el9. 3.9 MB/s | 666 kB 00:00 2023-08-09T13:31:24.846 INFO:teuthology.orchestra.run.smithi043.stdout:(16/154): librados-devel-18.0.0-5389.g668a86f2. 1.2 MB/s | 126 kB 00:00 2023-08-09T13:31:24.913 INFO:teuthology.orchestra.run.smithi043.stdout:(17/154): libradosstriper1-18.0.0-5389.g668a86f 4.0 MB/s | 473 kB 00:00 2023-08-09T13:31:25.006 INFO:teuthology.orchestra.run.smithi043.stdout:(18/154): python3-ceph-argparse-18.0.0-5389.g66 497 kB/s | 45 kB 00:00 2023-08-09T13:31:25.122 INFO:teuthology.orchestra.run.smithi043.stdout:(19/154): python3-ceph-common-18.0.0-5389.g668a 1.1 MB/s | 125 kB 00:00 2023-08-09T13:31:25.247 INFO:teuthology.orchestra.run.smithi043.stdout:(20/154): python3-cephfs-18.0.0-5389.g668a86f2. 1.3 MB/s | 162 kB 00:00 2023-08-09T13:31:25.373 INFO:teuthology.orchestra.run.smithi043.stdout:(21/154): python3-rados-18.0.0-5389.g668a86f2.e 2.5 MB/s | 320 kB 00:00 2023-08-09T13:31:25.481 INFO:teuthology.orchestra.run.smithi043.stdout:(22/154): python3-rbd-18.0.0-5389.g668a86f2.el9 2.7 MB/s | 297 kB 00:00 2023-08-09T13:31:25.590 INFO:teuthology.orchestra.run.smithi043.stdout:(23/154): python3-rgw-18.0.0-5389.g668a86f2.el9 928 kB/s | 99 kB 00:00 2023-08-09T13:31:25.690 INFO:teuthology.orchestra.run.smithi043.stdout:(24/154): rbd-fuse-18.0.0-5389.g668a86f2.el9.x8 861 kB/s | 85 kB 00:00 2023-08-09T13:31:25.807 INFO:teuthology.orchestra.run.smithi043.stdout:(25/154): librgw2-18.0.0-5389.g668a86f2.el9.x86 4.9 MB/s | 4.7 MB 00:00 2023-08-09T13:31:25.924 INFO:teuthology.orchestra.run.smithi043.stdout:(26/154): rbd-nbd-18.0.0-5389.g668a86f2.el9.x86 1.4 MB/s | 169 kB 00:00 2023-08-09T13:31:26.041 INFO:teuthology.orchestra.run.smithi043.stdout:(27/154): ceph-grafana-dashboards-18.0.0-5389.g 200 kB/s | 23 kB 00:00 2023-08-09T13:31:26.358 INFO:teuthology.orchestra.run.smithi043.stdout:(28/154): rbd-mirror-18.0.0-5389.g668a86f2.el9. 4.5 MB/s | 3.0 MB 00:00 2023-08-09T13:31:26.466 INFO:teuthology.orchestra.run.smithi043.stdout:(29/154): ceph-mgr-cephadm-18.0.0-5389.g668a86f 308 kB/s | 131 kB 00:00 2023-08-09T13:31:26.515 INFO:teuthology.orchestra.run.smithi203.stdout:(8/154): ceph-common-18.0.0-5389.g668a86f2.el9. 4.0 MB/s | 19 MB 00:04 2023-08-09T13:31:26.607 INFO:teuthology.orchestra.run.smithi203.stdout:(9/154): ceph-selinux-18.0.0-5389.g668a86f2.el9 266 kB/s | 24 kB 00:00 2023-08-09T13:31:26.617 INFO:teuthology.orchestra.run.smithi043.stdout:(30/154): ceph-mgr-dashboard-18.0.0-5389.g668a8 6.8 MB/s | 1.7 MB 00:00 2023-08-09T13:31:26.717 INFO:teuthology.orchestra.run.smithi043.stdout:(31/154): ceph-mgr-modules-core-18.0.0-5389.g66 2.4 MB/s | 242 kB 00:00 2023-08-09T13:31:26.784 INFO:teuthology.orchestra.run.smithi043.stdout:(32/154): ceph-mgr-rook-18.0.0-5389.g668a86f2.e 725 kB/s | 47 kB 00:00 2023-08-09T13:31:26.841 INFO:teuthology.orchestra.run.smithi203.stdout:(10/154): ceph-osd-18.0.0-5389.g668a86f2.el9.x8 6.2 MB/s | 18 MB 00:02 2023-08-09T13:31:26.842 INFO:teuthology.orchestra.run.smithi043.stdout:(33/154): ceph-prometheus-alerts-18.0.0-5389.g6 255 kB/s | 15 kB 00:00 2023-08-09T13:31:26.925 INFO:teuthology.orchestra.run.smithi203.stdout:(11/154): cephfs-mirror-18.0.0-5389.g668a86f2.e 2.6 MB/s | 218 kB 00:00 2023-08-09T13:31:26.943 INFO:teuthology.orchestra.run.smithi043.stdout:(34/154): ceph-volume-18.0.0-5389.g668a86f2.el9 2.6 MB/s | 261 kB 00:00 2023-08-09T13:31:27.016 INFO:teuthology.orchestra.run.smithi203.stdout:(12/154): libcephfs-devel-18.0.0-5389.g668a86f2 344 kB/s | 31 kB 00:00 2023-08-09T13:31:27.035 INFO:teuthology.orchestra.run.smithi043.stdout:(35/154): cephadm-18.0.0-5389.g668a86f2.el9.noa 2.4 MB/s | 218 kB 00:00 2023-08-09T13:31:27.118 INFO:teuthology.orchestra.run.smithi043.stdout:(36/154): cephfs-top-18.0.0-5389.g668a86f2.el9. 220 kB/s | 18 kB 00:00 2023-08-09T13:31:27.183 INFO:teuthology.orchestra.run.smithi203.stdout:(13/154): libcephfs2-18.0.0-5389.g668a86f2.el9. 3.9 MB/s | 666 kB 00:00 2023-08-09T13:31:27.275 INFO:teuthology.orchestra.run.smithi203.stdout:(14/154): libcephsqlite-18.0.0-5389.g668a86f2.e 1.8 MB/s | 164 kB 00:00 2023-08-09T13:31:27.327 INFO:teuthology.orchestra.run.smithi043.stdout:(37/154): automake-1.16.2-8.el9.noarch.rpm 3.1 MB/s | 662 kB 00:00 2023-08-09T13:31:27.359 INFO:teuthology.orchestra.run.smithi203.stdout:(15/154): librados-devel-18.0.0-5389.g668a86f2. 1.5 MB/s | 126 kB 00:00 2023-08-09T13:31:27.394 INFO:teuthology.orchestra.run.smithi043.stdout:(38/154): bison-3.7.4-5.el9.x86_64.rpm 14 MB/s | 920 kB 00:00 2023-08-09T13:31:27.436 INFO:teuthology.orchestra.run.smithi043.stdout:(39/154): boost-program-options-1.75.0-8.el9.x8 2.6 MB/s | 106 kB 00:00 2023-08-09T13:31:27.459 INFO:teuthology.orchestra.run.smithi203.stdout:(16/154): libradosstriper1-18.0.0-5389.g668a86f 4.7 MB/s | 473 kB 00:00 2023-08-09T13:31:27.461 INFO:teuthology.orchestra.run.smithi043.stdout:(40/154): elfutils-libelf-devel-0.189-3.el9.x86 876 kB/s | 21 kB 00:00 2023-08-09T13:31:27.503 INFO:teuthology.orchestra.run.smithi043.stdout:(41/154): flex-2.6.4-9.el9.x86_64.rpm 7.5 MB/s | 307 kB 00:00 2023-08-09T13:31:27.529 INFO:teuthology.orchestra.run.smithi043.stdout:(42/154): flexiblas-3.0.4-8.el9.x86_64.rpm 1.3 MB/s | 32 kB 00:00 2023-08-09T13:31:27.771 INFO:teuthology.orchestra.run.smithi043.stdout:(43/154): flexiblas-netlib-3.0.4-8.el9.x86_64.r 12 MB/s | 3.0 MB 00:00 2023-08-09T13:31:27.805 INFO:teuthology.orchestra.run.smithi043.stdout:(44/154): flexiblas-openblas-openmp-3.0.4-8.el9 529 kB/s | 17 kB 00:00 2023-08-09T13:31:27.847 INFO:teuthology.orchestra.run.smithi043.stdout:(45/154): jq-1.6-15.el9.x86_64.rpm 4.5 MB/s | 186 kB 00:00 2023-08-09T13:31:27.880 INFO:teuthology.orchestra.run.smithi043.stdout:(46/154): libaio-devel-0.3.111-13.el9.x86_64.rp 312 kB/s | 10 kB 00:00 2023-08-09T13:31:27.923 INFO:teuthology.orchestra.run.smithi043.stdout:(47/154): libpmemobj-1.12.1-1.el9.x86_64.rpm 3.8 MB/s | 159 kB 00:00 2023-08-09T13:31:28.056 INFO:teuthology.orchestra.run.smithi043.stdout:(48/154): ceph-mgr-diskprediction-local-18.0.0- 4.7 MB/s | 7.4 MB 00:01 2023-08-09T13:31:28.068 INFO:teuthology.orchestra.run.smithi203.stdout:(17/154): librgw2-18.0.0-5389.g668a86f2.el9.x86 7.7 MB/s | 4.7 MB 00:00 2023-08-09T13:31:28.081 INFO:teuthology.orchestra.run.smithi043.stdout:(49/154): librabbitmq-0.11.0-7.el9.x86_64.rpm 281 kB/s | 44 kB 00:00 2023-08-09T13:31:28.157 INFO:teuthology.orchestra.run.smithi043.stdout:(50/154): libtool-2.4.6-45.el9.x86_64.rpm 7.7 MB/s | 578 kB 00:00 2023-08-09T13:31:28.168 INFO:teuthology.orchestra.run.smithi203.stdout:(18/154): python3-ceph-argparse-18.0.0-5389.g66 456 kB/s | 45 kB 00:00 2023-08-09T13:31:28.198 INFO:teuthology.orchestra.run.smithi043.stdout:(51/154): librdkafka-1.6.1-102.el9.x86_64.rpm 5.6 MB/s | 661 kB 00:00 2023-08-09T13:31:28.224 INFO:teuthology.orchestra.run.smithi043.stdout:(52/154): libuuid-devel-2.37.4-11.el9.x86_64.rp 414 kB/s | 24 kB 00:00 2023-08-09T13:31:28.249 INFO:teuthology.orchestra.run.smithi043.stdout:(53/154): libzstd-devel-1.5.1-2.el9.x86_64.rpm 937 kB/s | 46 kB 00:00 2023-08-09T13:31:28.282 INFO:teuthology.orchestra.run.smithi043.stdout:(54/154): lttng-ust-2.12.0-6.el9.x86_64.rpm 4.8 MB/s | 282 kB 00:00 2023-08-09T13:31:28.316 INFO:teuthology.orchestra.run.smithi043.stdout:(55/154): oniguruma-6.9.6-1.el9.5.x86_64.rpm 3.4 MB/s | 217 kB 00:00 2023-08-09T13:31:28.335 INFO:teuthology.orchestra.run.smithi203.stdout:(19/154): ceph-radosgw-18.0.0-5389.g668a86f2.el 2.2 MB/s | 8.1 MB 00:03 2023-08-09T13:31:28.341 INFO:teuthology.orchestra.run.smithi043.stdout:(56/154): openblas-0.3.21-2.el9.x86_64.rpm 617 kB/s | 34 kB 00:00 2023-08-09T13:31:28.369 INFO:teuthology.orchestra.run.smithi203.stdout:(20/154): python3-ceph-common-18.0.0-5389.g668a 627 kB/s | 125 kB 00:00 2023-08-09T13:31:28.428 INFO:teuthology.orchestra.run.smithi203.stdout:(21/154): python3-cephfs-18.0.0-5389.g668a86f2. 1.7 MB/s | 162 kB 00:00 2023-08-09T13:31:28.484 INFO:teuthology.orchestra.run.smithi043.stdout:(57/154): openblas-openmp-0.3.21-2.el9.x86_64.r 29 MB/s | 4.7 MB 00:00 2023-08-09T13:31:28.511 INFO:teuthology.orchestra.run.smithi203.stdout:(22/154): python3-rados-18.0.0-5389.g668a86f2.e 2.2 MB/s | 320 kB 00:00 2023-08-09T13:31:28.584 INFO:teuthology.orchestra.run.smithi043.stdout:(58/154): openssl-devel-3.0.7-24.el9.x86_64.rpm 13 MB/s | 3.0 MB 00:00 2023-08-09T13:31:28.595 INFO:teuthology.orchestra.run.smithi203.stdout:(23/154): python3-rbd-18.0.0-5389.g668a86f2.el9 1.7 MB/s | 297 kB 00:00 2023-08-09T13:31:28.637 INFO:teuthology.orchestra.run.smithi203.stdout:(24/154): python3-rgw-18.0.0-5389.g668a86f2.el9 800 kB/s | 99 kB 00:00 2023-08-09T13:31:28.651 INFO:teuthology.orchestra.run.smithi043.stdout:(59/154): postgresql-13.11-1.el9.x86_64.rpm 10 MB/s | 1.5 MB 00:00 2023-08-09T13:31:28.671 INFO:teuthology.orchestra.run.smithi203.stdout:(25/154): rbd-fuse-18.0.0-5389.g668a86f2.el9.x8 1.1 MB/s | 85 kB 00:00 2023-08-09T13:31:28.701 INFO:teuthology.orchestra.run.smithi043.stdout:(60/154): postgresql-contrib-13.11-1.el9.x86_64 8.0 MB/s | 811 kB 00:00 2023-08-09T13:31:28.726 INFO:teuthology.orchestra.run.smithi043.stdout:(61/154): postgresql-private-libs-13.11-1.el9.x 2.0 MB/s | 134 kB 00:00 2023-08-09T13:31:28.763 INFO:teuthology.orchestra.run.smithi203.stdout:(26/154): rbd-nbd-18.0.0-5389.g668a86f2.el9.x86 1.8 MB/s | 169 kB 00:00 2023-08-09T13:31:28.821 INFO:teuthology.orchestra.run.smithi203.stdout:(27/154): ceph-grafana-dashboards-18.0.0-5389.g 405 kB/s | 23 kB 00:00 2023-08-09T13:31:28.885 INFO:teuthology.orchestra.run.smithi043.stdout:(62/154): postgresql-server-13.11-1.el9.x86_64. 31 MB/s | 5.7 MB 00:00 2023-08-09T13:31:28.921 INFO:teuthology.orchestra.run.smithi203.stdout:(28/154): ceph-mgr-cephadm-18.0.0-5389.g668a86f 1.3 MB/s | 131 kB 00:00 2023-08-09T13:31:29.030 INFO:teuthology.orchestra.run.smithi203.stdout:(29/154): rbd-mirror-18.0.0-5389.g668a86f2.el9. 7.7 MB/s | 3.0 MB 00:00 2023-08-09T13:31:29.143 INFO:teuthology.orchestra.run.smithi043.stdout:(63/154): python3-babel-2.9.1-2.el9.noarch.rpm 15 MB/s | 5.8 MB 00:00 2023-08-09T13:31:29.168 INFO:teuthology.orchestra.run.smithi043.stdout:(64/154): python3-jinja2-2.11.3-4.el9.noarch.rp 863 kB/s | 228 kB 00:00 2023-08-09T13:31:29.202 INFO:teuthology.orchestra.run.smithi043.stdout:(65/154): python3-jsonpatch-1.21-16.el9.noarch. 763 kB/s | 24 kB 00:00 2023-08-09T13:31:29.219 INFO:teuthology.orchestra.run.smithi043.stdout:(66/154): python3-jsonpointer-2.0-4.el9.noarch. 372 kB/s | 18 kB 00:00 2023-08-09T13:31:29.253 INFO:teuthology.orchestra.run.smithi043.stdout:(67/154): python3-markupsafe-1.1.1-12.el9.x86_6 990 kB/s | 32 kB 00:00 2023-08-09T13:31:29.278 INFO:teuthology.orchestra.run.smithi043.stdout:(68/154): python3-mako-1.1.4-6.el9.noarch.rpm 2.5 MB/s | 144 kB 00:00 2023-08-09T13:31:29.437 INFO:teuthology.orchestra.run.smithi043.stdout:(69/154): python3-numpy-1.20.1-5.el9.x86_64.rpm 26 MB/s | 4.8 MB 00:00 2023-08-09T13:31:29.462 INFO:teuthology.orchestra.run.smithi043.stdout:(70/154): python3-numpy-f2py-1.20.1-5.el9.x86_6 1.3 MB/s | 217 kB 00:00 2023-08-09T13:31:29.504 INFO:teuthology.orchestra.run.smithi043.stdout:(71/154): python3-oauthlib-3.1.1-5.el9.noarch.r 2.7 MB/s | 174 kB 00:00 2023-08-09T13:31:29.529 INFO:teuthology.orchestra.run.smithi043.stdout:(72/154): python3-prettytable-0.7.2-27.el9.noar 623 kB/s | 41 kB 00:00 2023-08-09T13:31:29.564 INFO:teuthology.orchestra.run.smithi203.stdout:(30/154): ceph-mgr-dashboard-18.0.0-5389.g668a8 2.7 MB/s | 1.7 MB 00:00 2023-08-09T13:31:29.571 INFO:teuthology.orchestra.run.smithi043.stdout:(73/154): python3-pyasn1-0.4.8-6.el9.noarch.rpm 2.0 MB/s | 132 kB 00:00 2023-08-09T13:31:29.605 INFO:teuthology.orchestra.run.smithi043.stdout:(74/154): python3-pyasn1-modules-0.4.8-6.el9.no 2.8 MB/s | 211 kB 00:00 2023-08-09T13:31:29.630 INFO:teuthology.orchestra.run.smithi043.stdout:(75/154): python3-pytz-2021.1-5.el9.noarch.rpm 857 kB/s | 47 kB 00:00 2023-08-09T13:31:29.664 INFO:teuthology.orchestra.run.smithi043.stdout:(76/154): python3-requests-oauthlib-1.3.0-12.el 741 kB/s | 43 kB 00:00 2023-08-09T13:31:29.672 INFO:teuthology.orchestra.run.smithi203.stdout:(31/154): ceph-mgr-modules-core-18.0.0-5389.g66 2.2 MB/s | 242 kB 00:00 2023-08-09T13:31:29.739 INFO:teuthology.orchestra.run.smithi203.stdout:(32/154): ceph-mgr-rook-18.0.0-5389.g668a86f2.e 725 kB/s | 47 kB 00:00 2023-08-09T13:31:29.789 INFO:teuthology.orchestra.run.smithi203.stdout:(33/154): ceph-prometheus-alerts-18.0.0-5389.g6 298 kB/s | 15 kB 00:00 2023-08-09T13:31:29.932 INFO:teuthology.orchestra.run.smithi203.stdout:(34/154): ceph-mgr-diskprediction-local-18.0.0- 8.2 MB/s | 7.4 MB 00:00 2023-08-09T13:31:29.939 INFO:teuthology.orchestra.run.smithi043.stdout:(77/154): python3-toml-0.10.2-6.el9.noarch.rpm 141 kB/s | 39 kB 00:00 2023-08-09T13:31:29.999 INFO:teuthology.orchestra.run.smithi203.stdout:(35/154): ceph-volume-18.0.0-5389.g668a86f2.el9 1.2 MB/s | 261 kB 00:00 2023-08-09T13:31:30.032 INFO:teuthology.orchestra.run.smithi203.stdout:(36/154): cephadm-18.0.0-5389.g668a86f2.el9.noa 2.1 MB/s | 218 kB 00:00 2023-08-09T13:31:30.057 INFO:teuthology.orchestra.run.smithi203.stdout:(37/154): cephfs-top-18.0.0-5389.g668a86f2.el9. 315 kB/s | 18 kB 00:00 2023-08-09T13:31:30.281 INFO:teuthology.orchestra.run.smithi043.stdout:(78/154): python3-scipy-1.6.2-8.el9.x86_64.rpm 24 MB/s | 16 MB 00:00 2023-08-09T13:31:30.315 INFO:teuthology.orchestra.run.smithi043.stdout:(79/154): socat-1.7.4.1-5.el9.x86_64.rpm 839 kB/s | 300 kB 00:00 2023-08-09T13:31:30.340 INFO:teuthology.orchestra.run.smithi043.stdout:(80/154): uuid-1.6.2-55.el9.x86_64.rpm 1.0 MB/s | 56 kB 00:00 2023-08-09T13:31:30.382 INFO:teuthology.orchestra.run.smithi043.stdout:(81/154): xmlstarlet-1.6.1-20.el9.x86_64.rpm 947 kB/s | 63 kB 00:00 2023-08-09T13:31:30.409 INFO:teuthology.orchestra.run.smithi203.stdout:(38/154): automake-1.16.2-8.el9.noarch.rpm 1.7 MB/s | 662 kB 00:00 2023-08-09T13:31:30.424 INFO:teuthology.orchestra.run.smithi043.stdout:(82/154): fuse-2.9.9-15.el9.x86_64.rpm 953 kB/s | 79 kB 00:00 2023-08-09T13:31:30.482 INFO:teuthology.orchestra.run.smithi043.stdout:(83/154): libgfortran-11.4.1-2.1.el9.x86_64.rpm 7.9 MB/s | 807 kB 00:00 2023-08-09T13:31:30.500 INFO:teuthology.orchestra.run.smithi203.stdout:(39/154): boost-program-options-1.75.0-8.el9.x8 1.2 MB/s | 106 kB 00:00 2023-08-09T13:31:30.516 INFO:teuthology.orchestra.run.smithi043.stdout:(84/154): libquadmath-11.4.1-2.1.el9.x86_64.rpm 2.4 MB/s | 197 kB 00:00 2023-08-09T13:31:30.558 INFO:teuthology.orchestra.run.smithi043.stdout:(85/154): python3-cffi-1.14.5-5.el9.x86_64.rpm 3.3 MB/s | 241 kB 00:00 2023-08-09T13:31:30.559 INFO:teuthology.orchestra.run.smithi203.stdout:(40/154): bison-3.7.4-5.el9.x86_64.rpm 1.8 MB/s | 920 kB 00:00 2023-08-09T13:31:30.584 INFO:teuthology.orchestra.run.smithi203.stdout:(41/154): elfutils-libelf-devel-0.189-3.el9.x86 259 kB/s | 21 kB 00:00 2023-08-09T13:31:30.643 INFO:teuthology.orchestra.run.smithi203.stdout:(42/154): flex-2.6.4-9.el9.x86_64.rpm 3.6 MB/s | 307 kB 00:00 2023-08-09T13:31:30.649 INFO:teuthology.orchestra.run.smithi043.stdout:(86/154): python3-cryptography-36.0.1-4.el9.x86 8.7 MB/s | 1.1 MB 00:00 2023-08-09T13:31:30.668 INFO:teuthology.orchestra.run.smithi203.stdout:(43/154): flexiblas-3.0.4-8.el9.x86_64.rpm 382 kB/s | 32 kB 00:00 2023-08-09T13:31:30.725 INFO:teuthology.orchestra.run.smithi043.stdout:(87/154): python3-ply-3.11-14.el9.noarch.rpm 685 kB/s | 103 kB 00:00 2023-08-09T13:31:30.750 INFO:teuthology.orchestra.run.smithi043.stdout:(88/154): python3-pycparser-2.20-6.el9.noarch.r 1.2 MB/s | 124 kB 00:00 2023-08-09T13:31:30.835 INFO:teuthology.orchestra.run.smithi203.stdout:(44/154): flexiblas-netlib-3.0.4-8.el9.x86_64.r 16 MB/s | 3.0 MB 00:00 2023-08-09T13:31:30.852 INFO:teuthology.orchestra.run.smithi203.stdout:(45/154): flexiblas-openblas-openmp-3.0.4-8.el9 95 kB/s | 17 kB 00:00 2023-08-09T13:31:30.919 INFO:teuthology.orchestra.run.smithi203.stdout:(46/154): jq-1.6-15.el9.x86_64.rpm 2.8 MB/s | 186 kB 00:00 2023-08-09T13:31:30.935 INFO:teuthology.orchestra.run.smithi203.stdout:(47/154): libaio-devel-0.3.111-13.el9.x86_64.rp 123 kB/s | 10 kB 00:00 2023-08-09T13:31:31.011 INFO:teuthology.orchestra.run.smithi203.stdout:(48/154): libpmemobj-1.12.1-1.el9.x86_64.rpm 2.1 MB/s | 159 kB 00:00 2023-08-09T13:31:31.027 INFO:teuthology.orchestra.run.smithi203.stdout:(49/154): librabbitmq-0.11.0-7.el9.x86_64.rpm 488 kB/s | 44 kB 00:00 2023-08-09T13:31:31.226 INFO:teuthology.orchestra.run.smithi043.stdout:(89/154): ceph-test-18.0.0-5389.g668a86f2.el9.x 5.8 MB/s | 40 MB 00:06 2023-08-09T13:31:31.512 INFO:teuthology.orchestra.run.smithi203.stdout:(50/154): ceph-test-18.0.0-5389.g668a86f2.el9.x 8.2 MB/s | 40 MB 00:04 2023-08-09T13:31:31.551 INFO:teuthology.orchestra.run.smithi043.stdout:(90/154): xfsprogs-devel-5.19.0-4.el9.x86_64.rp 108 kB/s | 86 kB 00:00 2023-08-09T13:31:31.570 INFO:teuthology.orchestra.run.smithi203.stdout:(51/154): librdkafka-1.6.1-102.el9.x86_64.rpm 1.2 MB/s | 661 kB 00:00 2023-08-09T13:31:31.611 INFO:teuthology.orchestra.run.smithi203.stdout:(52/154): libtool-2.4.6-45.el9.x86_64.rpm 991 kB/s | 578 kB 00:00 2023-08-09T13:31:31.629 INFO:teuthology.orchestra.run.smithi203.stdout:(53/154): libuuid-devel-2.37.4-11.el9.x86_64.rp 239 kB/s | 24 kB 00:00 2023-08-09T13:31:31.662 INFO:teuthology.orchestra.run.smithi203.stdout:(54/154): libzstd-devel-1.5.1-2.el9.x86_64.rpm 948 kB/s | 46 kB 00:00 2023-08-09T13:31:31.696 INFO:teuthology.orchestra.run.smithi203.stdout:(55/154): lttng-ust-2.12.0-6.el9.x86_64.rpm 3.4 MB/s | 282 kB 00:00 2023-08-09T13:31:31.701 INFO:teuthology.orchestra.run.smithi043.stdout:(91/154): zlib-devel-1.2.11-36.el9.x86_64.rpm 95 kB/s | 45 kB 00:00 2023-08-09T13:31:31.738 INFO:teuthology.orchestra.run.smithi203.stdout:(56/154): oniguruma-6.9.6-1.el9.5.x86_64.rpm 2.0 MB/s | 217 kB 00:00 2023-08-09T13:31:31.751 INFO:teuthology.orchestra.run.smithi043.stdout:(92/154): python3-devel-3.9.16-1.el9.x86_64.rpm 241 kB/s | 247 kB 00:01 2023-08-09T13:31:31.771 INFO:teuthology.orchestra.run.smithi203.stdout:(57/154): openblas-0.3.21-2.el9.x86_64.rpm 476 kB/s | 34 kB 00:00 2023-08-09T13:31:31.976 INFO:teuthology.orchestra.run.smithi203.stdout:(58/154): openblas-openmp-0.3.21-2.el9.x86_64.r 17 MB/s | 4.7 MB 00:00 2023-08-09T13:31:32.027 INFO:teuthology.orchestra.run.smithi043.stdout:(93/154): fmt-8.1.1-5.el9.x86_64.rpm 233 kB/s | 111 kB 00:00 2023-08-09T13:31:32.097 INFO:teuthology.orchestra.run.smithi203.stdout:(59/154): openssl-devel-3.0.7-24.el9.x86_64.rpm 8.7 MB/s | 3.0 MB 00:00 2023-08-09T13:31:32.175 INFO:teuthology.orchestra.run.smithi203.stdout:(60/154): postgresql-13.11-1.el9.x86_64.rpm 3.9 MB/s | 1.5 MB 00:00 2023-08-09T13:31:32.222 INFO:teuthology.orchestra.run.smithi203.stdout:(61/154): postgresql-contrib-13.11-1.el9.x86_64 3.4 MB/s | 811 kB 00:00 2023-08-09T13:31:32.264 INFO:teuthology.orchestra.run.smithi203.stdout:(62/154): postgresql-private-libs-13.11-1.el9.x 843 kB/s | 134 kB 00:00 2023-08-09T13:31:32.301 INFO:teuthology.orchestra.run.smithi043.stdout:(94/154): gperftools-libs-2.9.1-2.el9.x86_64.rp 516 kB/s | 309 kB 00:00 2023-08-09T13:31:32.352 INFO:teuthology.orchestra.run.smithi043.stdout:(95/154): libarrow-doc-9.0.0-9.el9.noarch.rpm 75 kB/s | 24 kB 00:00 2023-08-09T13:31:32.515 INFO:teuthology.orchestra.run.smithi203.stdout:(63/154): python3-jinja2-2.11.3-4.el9.noarch.rp 951 kB/s | 228 kB 00:00 2023-08-09T13:31:32.707 INFO:teuthology.orchestra.run.smithi203.stdout:(64/154): python3-babel-2.9.1-2.el9.noarch.rpm 12 MB/s | 5.8 MB 00:00 2023-08-09T13:31:32.924 INFO:teuthology.orchestra.run.smithi203.stdout:(65/154): postgresql-server-13.11-1.el9.x86_64. 8.1 MB/s | 5.7 MB 00:00 2023-08-09T13:31:32.969 INFO:teuthology.orchestra.run.smithi043.stdout:(96/154): liboath-2.6.7-2.el9.x86_64.rpm 73 kB/s | 49 kB 00:00 2023-08-09T13:31:32.975 INFO:teuthology.orchestra.run.smithi203.stdout:(66/154): python3-jsonpatch-1.21-16.el9.noarch. 54 kB/s | 24 kB 00:00 2023-08-09T13:31:32.997 INFO:teuthology.orchestra.run.smithi043.stdout:(97/154): libunwind-1.6.2-1.el9.x86_64.rpm 105 kB/s | 67 kB 00:00 2023-08-09T13:31:33.008 INFO:teuthology.orchestra.run.smithi203.stdout:(67/154): python3-jsonpointer-2.0-4.el9.noarch. 62 kB/s | 18 kB 00:00 2023-08-09T13:31:33.050 INFO:teuthology.orchestra.run.smithi203.stdout:(68/154): python3-mako-1.1.4-6.el9.noarch.rpm 1.9 MB/s | 144 kB 00:00 2023-08-09T13:31:33.075 INFO:teuthology.orchestra.run.smithi203.stdout:(69/154): python3-markupsafe-1.1.1-12.el9.x86_6 322 kB/s | 32 kB 00:00 2023-08-09T13:31:33.217 INFO:teuthology.orchestra.run.smithi203.stdout:(70/154): python3-numpy-1.20.1-5.el9.x86_64.rpm 23 MB/s | 4.8 MB 00:00 2023-08-09T13:31:33.250 INFO:teuthology.orchestra.run.smithi203.stdout:(71/154): python3-oauthlib-3.1.1-5.el9.noarch.r 1.1 MB/s | 174 kB 00:00 2023-08-09T13:31:33.284 INFO:teuthology.orchestra.run.smithi203.stdout:(72/154): python3-numpy-f2py-1.20.1-5.el9.x86_6 1.1 MB/s | 217 kB 00:00 2023-08-09T13:31:33.309 INFO:teuthology.orchestra.run.smithi203.stdout:(73/154): python3-prettytable-0.7.2-27.el9.noar 748 kB/s | 41 kB 00:00 2023-08-09T13:31:33.343 INFO:teuthology.orchestra.run.smithi203.stdout:(74/154): python3-pyasn1-0.4.8-6.el9.noarch.rpm 1.5 MB/s | 132 kB 00:00 2023-08-09T13:31:33.376 INFO:teuthology.orchestra.run.smithi203.stdout:(75/154): python3-pyasn1-modules-0.4.8-6.el9.no 2.3 MB/s | 211 kB 00:00 2023-08-09T13:31:33.402 INFO:teuthology.orchestra.run.smithi203.stdout:(76/154): python3-pytz-2021.1-5.el9.noarch.rpm 860 kB/s | 47 kB 00:00 2023-08-09T13:31:33.427 INFO:teuthology.orchestra.run.smithi203.stdout:(77/154): python3-requests-oauthlib-1.3.0-12.el 535 kB/s | 43 kB 00:00 2023-08-09T13:31:33.627 INFO:teuthology.orchestra.run.smithi203.stdout:(78/154): python3-toml-0.10.2-6.el9.noarch.rpm 213 kB/s | 39 kB 00:00 2023-08-09T13:31:33.661 INFO:teuthology.orchestra.run.smithi203.stdout:(79/154): socat-1.7.4.1-5.el9.x86_64.rpm 1.4 MB/s | 300 kB 00:00 2023-08-09T13:31:33.869 INFO:teuthology.orchestra.run.smithi203.stdout:(80/154): python3-scipy-1.6.2-8.el9.x86_64.rpm 32 MB/s | 16 MB 00:00 2023-08-09T13:31:33.886 INFO:teuthology.orchestra.run.smithi203.stdout:(81/154): uuid-1.6.2-55.el9.x86_64.rpm 219 kB/s | 56 kB 00:00 2023-08-09T13:31:33.911 INFO:teuthology.orchestra.run.smithi203.stdout:(82/154): xmlstarlet-1.6.1-20.el9.x86_64.rpm 255 kB/s | 63 kB 00:00 2023-08-09T13:31:33.974 INFO:teuthology.orchestra.run.smithi203.stdout:(83/154): fuse-2.9.9-15.el9.x86_64.rpm 914 kB/s | 79 kB 00:00 2023-08-09T13:31:34.020 INFO:teuthology.orchestra.run.smithi203.stdout:(84/154): libgfortran-11.4.1-2.1.el9.x86_64.rpm 6.0 MB/s | 807 kB 00:00 2023-08-09T13:31:34.062 INFO:teuthology.orchestra.run.smithi203.stdout:(85/154): libquadmath-11.4.1-2.1.el9.x86_64.rpm 1.4 MB/s | 197 kB 00:00 2023-08-09T13:31:34.112 INFO:teuthology.orchestra.run.smithi203.stdout:(86/154): python3-cffi-1.14.5-5.el9.x86_64.rpm 2.6 MB/s | 241 kB 00:00 2023-08-09T13:31:34.163 INFO:teuthology.orchestra.run.smithi203.stdout:(87/154): python3-cryptography-36.0.1-4.el9.x86 8.2 MB/s | 1.1 MB 00:00 2023-08-09T13:31:34.197 INFO:teuthology.orchestra.run.smithi203.stdout:(88/154): python3-ply-3.11-14.el9.noarch.rpm 870 kB/s | 103 kB 00:00 2023-08-09T13:31:34.230 INFO:teuthology.orchestra.run.smithi203.stdout:(89/154): python3-pycparser-2.20-6.el9.noarch.r 1.9 MB/s | 124 kB 00:00 2023-08-09T13:31:34.322 INFO:teuthology.orchestra.run.smithi203.stdout:(90/154): xfsprogs-devel-5.19.0-4.el9.x86_64.rp 704 kB/s | 86 kB 00:00 2023-08-09T13:31:34.339 INFO:teuthology.orchestra.run.smithi203.stdout:(91/154): zlib-devel-1.2.11-36.el9.x86_64.rpm 417 kB/s | 45 kB 00:00 2023-08-09T13:31:34.381 INFO:teuthology.orchestra.run.smithi203.stdout:(92/154): python3-devel-3.9.16-1.el9.x86_64.rpm 1.1 MB/s | 247 kB 00:00 2023-08-09T13:31:34.772 INFO:teuthology.orchestra.run.smithi043.stdout:(98/154): python3-asyncssh-2.13.2-3.el9.noarch. 309 kB/s | 548 kB 00:01 2023-08-09T13:31:34.789 INFO:teuthology.orchestra.run.smithi203.stdout:(93/154): fmt-8.1.1-5.el9.x86_64.rpm 237 kB/s | 111 kB 00:00 2023-08-09T13:31:34.974 INFO:teuthology.orchestra.run.smithi203.stdout:(94/154): libarrow-doc-9.0.0-9.el9.noarch.rpm 133 kB/s | 24 kB 00:00 2023-08-09T13:31:35.015 INFO:teuthology.orchestra.run.smithi203.stdout:(95/154): gperftools-libs-2.9.1-2.el9.x86_64.rp 457 kB/s | 309 kB 00:00 2023-08-09T13:31:35.066 INFO:teuthology.orchestra.run.smithi203.stdout:(96/154): liboath-2.6.7-2.el9.x86_64.rpm 544 kB/s | 49 kB 00:00 2023-08-09T13:31:35.108 INFO:teuthology.orchestra.run.smithi203.stdout:(97/154): libunwind-1.6.2-1.el9.x86_64.rpm 738 kB/s | 67 kB 00:00 2023-08-09T13:31:35.280 INFO:teuthology.orchestra.run.smithi043.stdout:(99/154): parquet-libs-9.0.0-9.el9.x86_64.rpm 362 kB/s | 837 kB 00:02 2023-08-09T13:31:35.647 INFO:teuthology.orchestra.run.smithi043.stdout:(100/154): python3-bcrypt-3.2.2-1.el9.x86_64.rp 50 kB/s | 43 kB 00:00 2023-08-09T13:31:35.816 INFO:teuthology.orchestra.run.smithi203.stdout:(98/154): parquet-libs-9.0.0-9.el9.x86_64.rpm 1.1 MB/s | 837 kB 00:00 2023-08-09T13:31:35.858 INFO:teuthology.orchestra.run.smithi203.stdout:(99/154): python3-asyncssh-2.13.2-3.el9.noarch. 731 kB/s | 548 kB 00:00 2023-08-09T13:31:36.025 INFO:teuthology.orchestra.run.smithi203.stdout:(100/154): python3-bcrypt-3.2.2-1.el9.x86_64.rp 267 kB/s | 43 kB 00:00 2023-08-09T13:31:36.050 INFO:teuthology.orchestra.run.smithi203.stdout:(101/154): python3-cachetools-4.2.4-1.el9.noarc 171 kB/s | 32 kB 00:00 2023-08-09T13:31:36.125 INFO:teuthology.orchestra.run.smithi203.stdout:(102/154): python3-certifi-2021.10.8-2.el9.noar 153 kB/s | 15 kB 00:00 2023-08-09T13:31:36.244 INFO:teuthology.orchestra.run.smithi203.stdout:(103/154): python3-cheroot-8.6.0-4.el9.noarch.r 900 kB/s | 172 kB 00:00 2023-08-09T13:31:36.323 INFO:teuthology.orchestra.run.smithi043.stdout:(101/154): python3-cachetools-4.2.4-1.el9.noarc 31 kB/s | 32 kB 00:01 2023-08-09T13:31:36.459 INFO:teuthology.orchestra.run.smithi203.stdout:(104/154): python3-cherrypy-18.6.1-2.el9.noarch 1.1 MB/s | 358 kB 00:00 2023-08-09T13:31:36.473 INFO:teuthology.orchestra.run.smithi043.stdout:(102/154): python3-certifi-2021.10.8-2.el9.noar 18 kB/s | 15 kB 00:00 2023-08-09T13:31:36.493 INFO:teuthology.orchestra.run.smithi203.stdout:(105/154): python3-google-auth-2.22.0-1.el9.noa 795 kB/s | 197 kB 00:00 2023-08-09T13:31:36.560 INFO:teuthology.orchestra.run.smithi203.stdout:(106/154): python3-jaraco-8.2.1-3.el9.noarch.rp 111 kB/s | 11 kB 00:00 2023-08-09T13:31:36.602 INFO:teuthology.orchestra.run.smithi203.stdout:(107/154): python3-jaraco-classes-3.2.1-5.el9.n 163 kB/s | 18 kB 00:00 2023-08-09T13:31:36.652 INFO:teuthology.orchestra.run.smithi203.stdout:(108/154): python3-jaraco-collections-3.0.0-8.e 258 kB/s | 23 kB 00:00 2023-08-09T13:31:36.686 INFO:teuthology.orchestra.run.smithi203.stdout:(109/154): python3-jaraco-functools-3.5.0-2.el9 234 kB/s | 19 kB 00:00 2023-08-09T13:31:36.744 INFO:teuthology.orchestra.run.smithi203.stdout:(110/154): python3-jaraco-text-3.2.0-6.el9.noar 215 kB/s | 20 kB 00:00 2023-08-09T13:31:36.786 INFO:teuthology.orchestra.run.smithi203.stdout:(111/154): python3-jwt+crypto-2.4.0-1.el9.noarc 91 kB/s | 9.0 kB 00:00 2023-08-09T13:31:36.903 INFO:teuthology.orchestra.run.smithi203.stdout:(112/154): libarrow-9.0.0-9.el9.x86_64.rpm 1.8 MB/s | 4.4 MB 00:02 2023-08-09T13:31:36.928 INFO:teuthology.orchestra.run.smithi203.stdout:(113/154): python3-jwt-2.4.0-1.el9.noarch.rpm 223 kB/s | 41 kB 00:00 2023-08-09T13:31:37.129 INFO:teuthology.orchestra.run.smithi203.stdout:(114/154): python3-logutils-0.3.5-21.el9.noarch 208 kB/s | 46 kB 00:00 2023-08-09T13:31:37.179 INFO:teuthology.orchestra.run.smithi203.stdout:(115/154): python3-more-itertools-8.12.0-2.el9. 316 kB/s | 79 kB 00:00 2023-08-09T13:31:37.181 INFO:teuthology.orchestra.run.smithi043.stdout:(103/154): python3-cheroot-8.6.0-4.el9.noarch.r 200 kB/s | 172 kB 00:00 2023-08-09T13:31:37.313 INFO:teuthology.orchestra.run.smithi203.stdout:(116/154): python3-natsort-7.1.1-5.el9.noarch.r 316 kB/s | 58 kB 00:00 2023-08-09T13:31:37.538 INFO:teuthology.orchestra.run.smithi203.stdout:(117/154): python3-kubernetes-26.1.0-2.el9.noar 1.4 MB/s | 1.0 MB 00:00 2023-08-09T13:31:37.563 INFO:teuthology.orchestra.run.smithi203.stdout:(118/154): python3-portend-3.1.0-2.el9.noarch.r 66 kB/s | 16 kB 00:00 2023-08-09T13:31:37.597 INFO:teuthology.orchestra.run.smithi203.stdout:(119/154): python3-pecan-1.4.2-3.el9.noarch.rpm 654 kB/s | 272 kB 00:00 2023-08-09T13:31:37.630 INFO:teuthology.orchestra.run.smithi203.stdout:(120/154): python3-pyOpenSSL-21.0.0-1.el9.noarc 988 kB/s | 90 kB 00:00 2023-08-09T13:31:37.673 INFO:teuthology.orchestra.run.smithi203.stdout:(121/154): python3-repoze-lru-0.7-16.el9.noarch 284 kB/s | 31 kB 00:00 2023-08-09T13:31:37.756 INFO:teuthology.orchestra.run.smithi203.stdout:(122/154): python3-routes-2.5.1-5.el9.noarch.rp 1.2 MB/s | 188 kB 00:00 2023-08-09T13:31:37.773 INFO:teuthology.orchestra.run.smithi203.stdout:(123/154): python3-rsa-4.9-2.el9.noarch.rpm 417 kB/s | 59 kB 00:00 2023-08-09T13:31:37.790 INFO:teuthology.orchestra.run.smithi203.stdout:(124/154): python3-tempora-5.0.0-2.el9.noarch.r 306 kB/s | 36 kB 00:00 2023-08-09T13:31:37.807 INFO:teuthology.orchestra.run.smithi043.stdout:(104/154): python3-cherrypy-18.6.1-2.el9.noarch 269 kB/s | 358 kB 00:01 2023-08-09T13:31:37.849 INFO:teuthology.orchestra.run.smithi203.stdout:(125/154): python3-typing-extensions-4.4.0-2.el 558 kB/s | 51 kB 00:00 2023-08-09T13:31:38.023 INFO:teuthology.orchestra.run.smithi043.stdout:(105/154): python3-google-auth-2.22.0-1.el9.noa 234 kB/s | 197 kB 00:00 2023-08-09T13:31:38.099 INFO:teuthology.orchestra.run.smithi203.stdout:(126/154): python3-webob-1.8.7-6.el9.noarch.rpm 707 kB/s | 230 kB 00:00 2023-08-09T13:31:38.141 INFO:teuthology.orchestra.run.smithi203.stdout:(127/154): python3-websocket-client-1.2.3-2.el9 256 kB/s | 90 kB 00:00 2023-08-09T13:31:38.233 INFO:teuthology.orchestra.run.smithi203.stdout:(128/154): python3-werkzeug-2.0.3-3.el9.1.noarc 1.1 MB/s | 427 kB 00:00 2023-08-09T13:31:38.258 INFO:teuthology.orchestra.run.smithi203.stdout:(129/154): python3-zc-lockfile-2.0-10.el9.noarc 129 kB/s | 20 kB 00:00 2023-08-09T13:31:38.274 INFO:teuthology.orchestra.run.smithi043.stdout:(106/154): libarrow-9.0.0-9.el9.x86_64.rpm 694 kB/s | 4.4 MB 00:06 2023-08-09T13:31:38.299 INFO:teuthology.orchestra.run.smithi043.stdout:(107/154): python3-jaraco-8.2.1-3.el9.noarch.rp 22 kB/s | 11 kB 00:00 2023-08-09T13:31:38.324 INFO:teuthology.orchestra.run.smithi043.stdout:(108/154): python3-jaraco-classes-3.2.1-5.el9.n 59 kB/s | 18 kB 00:00 2023-08-09T13:31:38.333 INFO:teuthology.orchestra.run.smithi203.stdout:(130/154): re2-20211101-2.el9.x86_64.rpm 1.0 MB/s | 192 kB 00:00 2023-08-09T13:31:38.358 INFO:teuthology.orchestra.run.smithi043.stdout:(109/154): python3-jaraco-collections-3.0.0-8.e 280 kB/s | 23 kB 00:00 2023-08-09T13:31:38.417 INFO:teuthology.orchestra.run.smithi043.stdout:(110/154): python3-jaraco-functools-3.5.0-2.el9 214 kB/s | 19 kB 00:00 2023-08-09T13:31:38.442 INFO:teuthology.orchestra.run.smithi043.stdout:(111/154): python3-jaraco-text-3.2.0-6.el9.noar 170 kB/s | 20 kB 00:00 2023-08-09T13:31:38.467 INFO:teuthology.orchestra.run.smithi043.stdout:(112/154): python3-jwt+crypto-2.4.0-1.el9.noarc 83 kB/s | 9.0 kB 00:00 2023-08-09T13:31:38.534 INFO:teuthology.orchestra.run.smithi043.stdout:(113/154): python3-jwt-2.4.0-1.el9.noarch.rpm 352 kB/s | 41 kB 00:00 2023-08-09T13:31:38.992 INFO:teuthology.orchestra.run.smithi203.stdout:(131/154): thrift-0.15.0-2.el9.x86_64.rpm 2.1 MB/s | 1.6 MB 00:00 2023-08-09T13:31:39.101 INFO:teuthology.orchestra.run.smithi203.stdout:(132/154): NetworkManager-1.43.90-1.el9.x86_64. 21 MB/s | 2.2 MB 00:00 2023-08-09T13:31:39.193 INFO:teuthology.orchestra.run.smithi043.stdout:(114/154): python3-logutils-0.3.5-21.el9.noarch 64 kB/s | 46 kB 00:00 2023-08-09T13:31:39.209 INFO:teuthology.orchestra.run.smithi203.stdout:(133/154): NetworkManager-libnm-1.43.90-1.el9.x 17 MB/s | 1.8 MB 00:00 2023-08-09T13:31:39.226 INFO:teuthology.orchestra.run.smithi043.stdout:(115/154): python3-more-itertools-8.12.0-2.el9. 114 kB/s | 79 kB 00:00 2023-08-09T13:31:39.251 INFO:teuthology.orchestra.run.smithi203.stdout:(134/154): NetworkManager-team-1.43.90-1.el9.x8 1.0 MB/s | 41 kB 00:00 2023-08-09T13:31:39.302 INFO:teuthology.orchestra.run.smithi203.stdout:(135/154): NetworkManager-tui-1.43.90-1.el9.x86 4.8 MB/s | 245 kB 00:00 2023-08-09T13:31:39.343 INFO:teuthology.orchestra.run.smithi203.stdout:(136/154): elfutils-debuginfod-client-0.189-3.e 873 kB/s | 36 kB 00:00 2023-08-09T13:31:39.394 INFO:teuthology.orchestra.run.smithi203.stdout:(137/154): elfutils-libelf-0.189-3.el9.x86_64.r 3.8 MB/s | 192 kB 00:00 2023-08-09T13:31:39.459 INFO:teuthology.orchestra.run.smithi043.stdout:(116/154): python3-kubernetes-26.1.0-2.el9.noar 1.0 MB/s | 1.0 MB 00:01 2023-08-09T13:31:39.473 INFO:teuthology.orchestra.run.smithi203.stdout:(138/154): elfutils-libs-0.189-3.el9.x86_64.rpm 3.2 MB/s | 254 kB 00:00 2023-08-09T13:31:39.485 INFO:teuthology.orchestra.run.smithi043.stdout:(117/154): python3-natsort-7.1.1-5.el9.noarch.r 199 kB/s | 58 kB 00:00 2023-08-09T13:31:39.553 INFO:teuthology.orchestra.run.smithi203.stdout:(139/154): iproute-6.2.0-5.el9.x86_64.rpm 9.7 MB/s | 785 kB 00:00 2023-08-09T13:31:39.603 INFO:teuthology.orchestra.run.smithi203.stdout:(140/154): iproute-tc-6.2.0-5.el9.x86_64.rpm 8.9 MB/s | 448 kB 00:00 2023-08-09T13:31:39.653 INFO:teuthology.orchestra.run.smithi203.stdout:(141/154): libblkid-2.37.4-11.el9.x86_64.rpm 2.1 MB/s | 107 kB 00:00 2023-08-09T13:31:39.660 INFO:teuthology.orchestra.run.smithi043.stdout:(118/154): python3-pecan-1.4.2-3.el9.noarch.rpm 629 kB/s | 272 kB 00:00 2023-08-09T13:31:39.685 INFO:teuthology.orchestra.run.smithi043.stdout:(119/154): python3-portend-3.1.0-2.el9.noarch.r 73 kB/s | 16 kB 00:00 2023-08-09T13:31:39.703 INFO:teuthology.orchestra.run.smithi203.stdout:(142/154): libfdisk-2.37.4-11.el9.x86_64.rpm 3.1 MB/s | 153 kB 00:00 2023-08-09T13:31:39.736 INFO:teuthology.orchestra.run.smithi043.stdout:(120/154): python3-pyOpenSSL-21.0.0-1.el9.noarc 360 kB/s | 90 kB 00:00 2023-08-09T13:31:39.761 INFO:teuthology.orchestra.run.smithi043.stdout:(121/154): python3-repoze-lru-0.7-16.el9.noarch 311 kB/s | 31 kB 00:00 2023-08-09T13:31:39.762 INFO:teuthology.orchestra.run.smithi203.stdout:(143/154): libmount-2.37.4-11.el9.x86_64.rpm 2.3 MB/s | 133 kB 00:00 2023-08-09T13:31:39.854 INFO:teuthology.orchestra.run.smithi203.stdout:(144/154): librados2-18.0.0-5389.g668a86f2.el9. 2.1 MB/s | 3.3 MB 00:01 2023-08-09T13:31:39.879 INFO:teuthology.orchestra.run.smithi203.stdout:(145/154): libsmartcols-2.37.4-11.el9.x86_64.rp 537 kB/s | 62 kB 00:00 2023-08-09T13:31:39.921 INFO:teuthology.orchestra.run.smithi203.stdout:(146/154): libuuid-2.37.4-11.el9.x86_64.rpm 418 kB/s | 27 kB 00:00 2023-08-09T13:31:39.945 INFO:teuthology.orchestra.run.smithi043.stdout:(122/154): python3-routes-2.5.1-5.el9.noarch.rp 731 kB/s | 188 kB 00:00 2023-08-09T13:31:39.986 INFO:teuthology.orchestra.run.smithi043.stdout:(123/154): python3-rsa-4.9-2.el9.noarch.rpm 237 kB/s | 59 kB 00:00 2023-08-09T13:31:40.005 INFO:teuthology.orchestra.run.smithi203.stdout:(147/154): openssl-3.0.7-24.el9.x86_64.rpm 9.3 MB/s | 1.2 MB 00:00 2023-08-09T13:31:40.103 INFO:teuthology.orchestra.run.smithi043.stdout:(124/154): python3-tempora-5.0.0-2.el9.noarch.r 105 kB/s | 36 kB 00:00 2023-08-09T13:31:40.129 INFO:teuthology.orchestra.run.smithi043.stdout:(125/154): python3-typing-extensions-4.4.0-2.el 279 kB/s | 51 kB 00:00 2023-08-09T13:31:40.147 INFO:teuthology.orchestra.run.smithi203.stdout:(148/154): openssl-libs-3.0.7-24.el9.x86_64.rpm 9.6 MB/s | 2.1 MB 00:00 2023-08-09T13:31:40.239 INFO:teuthology.orchestra.run.smithi203.stdout:(149/154): util-linux-2.37.4-11.el9.x86_64.rpm 9.4 MB/s | 2.2 MB 00:00 2023-08-09T13:31:40.281 INFO:teuthology.orchestra.run.smithi203.stdout:(150/154): util-linux-core-2.37.4-11.el9.x86_64 3.6 MB/s | 432 kB 00:00 2023-08-09T13:31:40.298 INFO:teuthology.orchestra.run.smithi203.stdout:(151/154): util-linux-user-2.37.4-11.el9.x86_64 554 kB/s | 30 kB 00:00 2023-08-09T13:31:40.329 INFO:teuthology.orchestra.run.smithi043.stdout:(126/154): python3-webob-1.8.7-6.el9.noarch.rpm 672 kB/s | 230 kB 00:00 2023-08-09T13:31:40.340 INFO:teuthology.orchestra.run.smithi203.stdout:(152/154): xfsdump-3.1.12-3.el9.x86_64.rpm 5.4 MB/s | 321 kB 00:00 2023-08-09T13:31:40.396 INFO:teuthology.orchestra.run.smithi043.stdout:(127/154): python3-websocket-client-1.2.3-2.el9 338 kB/s | 90 kB 00:00 2023-08-09T13:31:40.696 INFO:teuthology.orchestra.run.smithi043.stdout:(128/154): python3-werkzeug-2.0.3-3.el9.1.noarc 755 kB/s | 427 kB 00:00 2023-08-09T13:31:40.722 INFO:teuthology.orchestra.run.smithi043.stdout:(129/154): python3-zc-lockfile-2.0-10.el9.noarc 51 kB/s | 20 kB 00:00 2023-08-09T13:31:40.847 INFO:teuthology.orchestra.run.smithi043.stdout:(130/154): re2-20211101-2.el9.x86_64.rpm 428 kB/s | 192 kB 00:00 2023-08-09T13:31:41.182 INFO:teuthology.orchestra.run.smithi203.stdout:(153/154): xfsprogs-5.19.0-4.el9.x86_64.rpm 1.2 MB/s | 1.0 MB 00:00 2023-08-09T13:31:41.764 INFO:teuthology.orchestra.run.smithi043.stdout:(131/154): thrift-0.15.0-2.el9.x86_64.rpm 1.5 MB/s | 1.6 MB 00:01 2023-08-09T13:31:41.873 INFO:teuthology.orchestra.run.smithi043.stdout:(132/154): NetworkManager-1.43.90-1.el9.x86_64. 21 MB/s | 2.2 MB 00:00 2023-08-09T13:31:42.137 INFO:teuthology.orchestra.run.smithi043.stdout:(133/154): NetworkManager-libnm-1.43.90-1.el9.x 6.9 MB/s | 1.8 MB 00:00 2023-08-09T13:31:42.323 INFO:teuthology.orchestra.run.smithi043.stdout:(134/154): NetworkManager-team-1.43.90-1.el9.x8 223 kB/s | 41 kB 00:00 2023-08-09T13:31:42.365 INFO:teuthology.orchestra.run.smithi043.stdout:(135/154): NetworkManager-tui-1.43.90-1.el9.x86 5.9 MB/s | 245 kB 00:00 2023-08-09T13:31:42.440 INFO:teuthology.orchestra.run.smithi043.stdout:(136/154): librbd1-18.0.0-5389.g668a86f2.el9.x8 1.9 MB/s | 3.0 MB 00:01 2023-08-09T13:31:42.465 INFO:teuthology.orchestra.run.smithi043.stdout:(137/154): elfutils-debuginfod-client-0.189-3.e 361 kB/s | 36 kB 00:00 2023-08-09T13:31:42.491 INFO:teuthology.orchestra.run.smithi043.stdout:(138/154): elfutils-libelf-0.189-3.el9.x86_64.r 3.8 MB/s | 192 kB 00:00 2023-08-09T13:31:42.524 INFO:teuthology.orchestra.run.smithi043.stdout:(139/154): elfutils-libs-0.189-3.el9.x86_64.rpm 4.5 MB/s | 254 kB 00:00 2023-08-09T13:31:42.575 INFO:teuthology.orchestra.run.smithi043.stdout:(140/154): iproute-6.2.0-5.el9.x86_64.rpm 9.6 MB/s | 785 kB 00:00 2023-08-09T13:31:42.608 INFO:teuthology.orchestra.run.smithi043.stdout:(141/154): iproute-tc-6.2.0-5.el9.x86_64.rpm 5.9 MB/s | 448 kB 00:00 2023-08-09T13:31:42.633 INFO:teuthology.orchestra.run.smithi043.stdout:(142/154): libblkid-2.37.4-11.el9.x86_64.rpm 2.0 MB/s | 107 kB 00:00 2023-08-09T13:31:42.659 INFO:teuthology.orchestra.run.smithi043.stdout:(143/154): libfdisk-2.37.4-11.el9.x86_64.rpm 3.1 MB/s | 153 kB 00:00 2023-08-09T13:31:42.684 INFO:teuthology.orchestra.run.smithi043.stdout:(144/154): libmount-2.37.4-11.el9.x86_64.rpm 2.7 MB/s | 133 kB 00:00 2023-08-09T13:31:42.701 INFO:teuthology.orchestra.run.smithi043.stdout:(145/154): libsmartcols-2.37.4-11.el9.x86_64.rp 1.5 MB/s | 62 kB 00:00 2023-08-09T13:31:42.718 INFO:teuthology.orchestra.run.smithi043.stdout:(146/154): libuuid-2.37.4-11.el9.x86_64.rpm 852 kB/s | 27 kB 00:00 2023-08-09T13:31:42.777 INFO:teuthology.orchestra.run.smithi043.stdout:(147/154): openssl-3.0.7-24.el9.x86_64.rpm 16 MB/s | 1.2 MB 00:00 2023-08-09T13:31:42.860 INFO:teuthology.orchestra.run.smithi043.stdout:(148/154): openssl-libs-3.0.7-24.el9.x86_64.rpm 17 MB/s | 2.1 MB 00:00 2023-08-09T13:31:42.944 INFO:teuthology.orchestra.run.smithi043.stdout:(149/154): util-linux-2.37.4-11.el9.x86_64.rpm 15 MB/s | 2.2 MB 00:00 2023-08-09T13:31:43.019 INFO:teuthology.orchestra.run.smithi043.stdout:(150/154): util-linux-core-2.37.4-11.el9.x86_64 3.0 MB/s | 432 kB 00:00 2023-08-09T13:31:43.078 INFO:teuthology.orchestra.run.smithi043.stdout:(151/154): util-linux-user-2.37.4-11.el9.x86_64 231 kB/s | 30 kB 00:00 2023-08-09T13:31:43.111 INFO:teuthology.orchestra.run.smithi043.stdout:(152/154): xfsdump-3.1.12-3.el9.x86_64.rpm 3.5 MB/s | 321 kB 00:00 2023-08-09T13:31:43.195 INFO:teuthology.orchestra.run.smithi043.stdout:(153/154): librados2-18.0.0-5389.g668a86f2.el9. 1.3 MB/s | 3.3 MB 00:02 2023-08-09T13:31:43.954 INFO:teuthology.orchestra.run.smithi043.stdout:(154/154): xfsprogs-5.19.0-4.el9.x86_64.rpm 1.2 MB/s | 1.0 MB 00:00 2023-08-09T13:31:43.966 INFO:teuthology.orchestra.run.smithi043.stdout:-------------------------------------------------------------------------------- 2023-08-09T13:31:43.966 INFO:teuthology.orchestra.run.smithi043.stdout:Total 8.5 MB/s | 205 MB 00:24 2023-08-09T13:31:44.146 INFO:teuthology.orchestra.run.smithi043.stdout:Running transaction check 2023-08-09T13:31:44.478 INFO:teuthology.orchestra.run.smithi043.stdout:Transaction check succeeded. 2023-08-09T13:31:44.478 INFO:teuthology.orchestra.run.smithi043.stdout:Running transaction test 2023-08-09T13:31:44.640 INFO:teuthology.orchestra.run.smithi203.stdout:(154/154): librbd1-18.0.0-5389.g668a86f2.el9.x8 492 kB/s | 3.0 MB 00:06 2023-08-09T13:31:44.654 INFO:teuthology.orchestra.run.smithi203.stdout:-------------------------------------------------------------------------------- 2023-08-09T13:31:44.655 INFO:teuthology.orchestra.run.smithi203.stdout:Total 8.8 MB/s | 205 MB 00:23 2023-08-09T13:31:44.833 INFO:teuthology.orchestra.run.smithi203.stdout:Running transaction check 2023-08-09T13:31:45.153 INFO:teuthology.orchestra.run.smithi203.stdout:Transaction check succeeded. 2023-08-09T13:31:45.154 INFO:teuthology.orchestra.run.smithi203.stdout:Running transaction test 2023-08-09T13:31:46.751 INFO:teuthology.orchestra.run.smithi043.stdout:Transaction test succeeded. 2023-08-09T13:31:46.751 INFO:teuthology.orchestra.run.smithi043.stdout:Running transaction 2023-08-09T13:31:47.262 INFO:teuthology.orchestra.run.smithi203.stdout:Transaction test succeeded. 2023-08-09T13:31:47.263 INFO:teuthology.orchestra.run.smithi203.stdout:Running transaction 2023-08-09T13:31:53.608 INFO:teuthology.orchestra.run.smithi043.stdout: Preparing : 1/1 2023-08-09T13:31:53.698 INFO:teuthology.orchestra.run.smithi043.stdout: Upgrading : openssl-libs-1:3.0.7-24.el9.x86_64 1/177 2023-08-09T13:31:53.733 INFO:teuthology.orchestra.run.smithi043.stdout: Upgrading : libuuid-2.37.4-11.el9.x86_64 2/177 2023-08-09T13:31:53.763 INFO:teuthology.orchestra.run.smithi043.stdout: Upgrading : libblkid-2.37.4-11.el9.x86_64 3/177 2023-08-09T13:31:53.886 INFO:teuthology.orchestra.run.smithi043.stdout: Running scriptlet: libblkid-2.37.4-11.el9.x86_64 3/177 2023-08-09T13:31:53.999 INFO:teuthology.orchestra.run.smithi043.stdout: Installing : thrift-0.15.0-2.el9.x86_64 4/177 2023-08-09T13:31:54.057 INFO:teuthology.orchestra.run.smithi043.stdout: Upgrading : elfutils-libelf-0.189-3.el9.x86_64 5/177 2023-08-09T13:31:54.140 INFO:teuthology.orchestra.run.smithi043.stdout: Installing : librabbitmq-0.11.0-7.el9.x86_64 6/177 2023-08-09T13:31:54.202 INFO:teuthology.orchestra.run.smithi043.stdout: Installing : librdkafka-1.6.1-102.el9.x86_64 7/177 2023-08-09T13:31:54.239 INFO:teuthology.orchestra.run.smithi043.stdout: Installing : python3-more-itertools-8.12.0-2.el9.noarch 8/177 2023-08-09T13:31:54.242 INFO:teuthology.orchestra.run.smithi203.stdout: Preparing : 1/1 2023-08-09T13:31:54.339 INFO:teuthology.orchestra.run.smithi043.stdout: Installing : python3-jaraco-8.2.1-3.el9.noarch 9/177 2023-08-09T13:31:54.371 INFO:teuthology.orchestra.run.smithi203.stdout: Upgrading : openssl-libs-1:3.0.7-24.el9.x86_64 1/177 2023-08-09T13:31:54.420 INFO:teuthology.orchestra.run.smithi203.stdout: Upgrading : libuuid-2.37.4-11.el9.x86_64 2/177 2023-08-09T13:31:54.492 INFO:teuthology.orchestra.run.smithi203.stdout: Upgrading : libblkid-2.37.4-11.el9.x86_64 3/177 2023-08-09T13:31:54.619 INFO:teuthology.orchestra.run.smithi203.stdout: Running scriptlet: libblkid-2.37.4-11.el9.x86_64 3/177 2023-08-09T13:31:54.695 INFO:teuthology.orchestra.run.smithi203.stdout: Installing : thrift-0.15.0-2.el9.x86_64 4/177 2023-08-09T13:31:54.756 INFO:teuthology.orchestra.run.smithi043.stdout: Installing : lttng-ust-2.12.0-6.el9.x86_64 10/177 2023-08-09T13:31:54.767 INFO:teuthology.orchestra.run.smithi203.stdout: Upgrading : elfutils-libelf-0.189-3.el9.x86_64 5/177 2023-08-09T13:31:54.803 INFO:teuthology.orchestra.run.smithi043.stdout: Upgrading : librados2-2:18.0.0-5389.g668a86f2.el9.x86_64 11/177 2023-08-09T13:31:54.859 INFO:teuthology.orchestra.run.smithi203.stdout: Installing : librabbitmq-0.11.0-7.el9.x86_64 6/177 2023-08-09T13:31:54.924 INFO:teuthology.orchestra.run.smithi203.stdout: Installing : librdkafka-1.6.1-102.el9.x86_64 7/177 2023-08-09T13:31:54.957 INFO:teuthology.orchestra.run.smithi043.stdout: Running scriptlet: librados2-2:18.0.0-5389.g668a86f2.el9.x86_64 11/177 2023-08-09T13:31:54.966 INFO:teuthology.orchestra.run.smithi203.stdout: Installing : python3-more-itertools-8.12.0-2.el9.noarch 8/177 2023-08-09T13:31:54.987 INFO:teuthology.orchestra.run.smithi043.stdout: Installing : libcephfs2-2:18.0.0-5389.g668a86f2.el9.x86_64 12/177 2023-08-09T13:31:55.092 INFO:teuthology.orchestra.run.smithi203.stdout: Installing : python3-jaraco-8.2.1-3.el9.noarch 9/177 2023-08-09T13:31:55.106 INFO:teuthology.orchestra.run.smithi043.stdout: Running scriptlet: libcephfs2-2:18.0.0-5389.g668a86f2.el9.x86_64 12/177 2023-08-09T13:31:55.162 INFO:teuthology.orchestra.run.smithi043.stdout: Installing : python3-rados-2:18.0.0-5389.g668a86f2.el9.x86_64 13/177 2023-08-09T13:31:55.428 INFO:teuthology.orchestra.run.smithi043.stdout: Installing : postgresql-private-libs-13.11-1.el9.x86_64 14/177 2023-08-09T13:31:55.484 INFO:teuthology.orchestra.run.smithi043.stdout: Installing : postgresql-13.11-1.el9.x86_64 15/177 2023-08-09T13:31:55.508 INFO:teuthology.orchestra.run.smithi043.stdout: Installing : libcephsqlite-2:18.0.0-5389.g668a86f2.el9.x86_64 16/177 2023-08-09T13:31:55.536 INFO:teuthology.orchestra.run.smithi203.stdout: Installing : lttng-ust-2.12.0-6.el9.x86_64 10/177 2023-08-09T13:31:55.614 INFO:teuthology.orchestra.run.smithi203.stdout: Upgrading : librados2-2:18.0.0-5389.g668a86f2.el9.x86_64 11/177 2023-08-09T13:31:55.681 INFO:teuthology.orchestra.run.smithi043.stdout: Running scriptlet: libcephsqlite-2:18.0.0-5389.g668a86f2.el9.x86_64 16/177 2023-08-09T13:31:55.709 INFO:teuthology.orchestra.run.smithi043.stdout: Installing : libradosstriper1-2:18.0.0-5389.g668a86f2.el9.x86 17/177 2023-08-09T13:31:55.775 INFO:teuthology.orchestra.run.smithi203.stdout: Running scriptlet: librados2-2:18.0.0-5389.g668a86f2.el9.x86_64 11/177 2023-08-09T13:31:55.799 INFO:teuthology.orchestra.run.smithi043.stdout: Running scriptlet: libradosstriper1-2:18.0.0-5389.g668a86f2.el9.x86 17/177 2023-08-09T13:31:55.807 INFO:teuthology.orchestra.run.smithi203.stdout: Installing : libcephfs2-2:18.0.0-5389.g668a86f2.el9.x86_64 12/177 2023-08-09T13:31:55.855 INFO:teuthology.orchestra.run.smithi043.stdout: Upgrading : libmount-2.37.4-11.el9.x86_64 18/177 2023-08-09T13:31:55.933 INFO:teuthology.orchestra.run.smithi203.stdout: Running scriptlet: libcephfs2-2:18.0.0-5389.g668a86f2.el9.x86_64 12/177 2023-08-09T13:31:55.966 INFO:teuthology.orchestra.run.smithi203.stdout: Installing : python3-rados-2:18.0.0-5389.g668a86f2.el9.x86_64 13/177 2023-08-09T13:31:56.216 INFO:teuthology.orchestra.run.smithi043.stdout: Upgrading : libsmartcols-2.37.4-11.el9.x86_64 19/177 2023-08-09T13:31:56.240 INFO:teuthology.orchestra.run.smithi203.stdout: Installing : postgresql-private-libs-13.11-1.el9.x86_64 14/177 2023-08-09T13:31:56.309 INFO:teuthology.orchestra.run.smithi203.stdout: Installing : postgresql-13.11-1.el9.x86_64 15/177 2023-08-09T13:31:56.323 INFO:teuthology.orchestra.run.smithi043.stdout: Upgrading : NetworkManager-libnm-1:1.43.90-1.el9.x86_64 20/177 2023-08-09T13:31:56.345 INFO:teuthology.orchestra.run.smithi203.stdout: Installing : libcephsqlite-2:18.0.0-5389.g668a86f2.el9.x86_64 16/177 2023-08-09T13:31:56.411 INFO:teuthology.orchestra.run.smithi043.stdout: Running scriptlet: NetworkManager-libnm-1:1.43.90-1.el9.x86_64 20/177 2023-08-09T13:31:56.494 INFO:teuthology.orchestra.run.smithi203.stdout: Running scriptlet: libcephsqlite-2:18.0.0-5389.g668a86f2.el9.x86_64 16/177 2023-08-09T13:31:56.528 INFO:teuthology.orchestra.run.smithi203.stdout: Installing : libradosstriper1-2:18.0.0-5389.g668a86f2.el9.x86 17/177 2023-08-09T13:31:56.627 INFO:teuthology.orchestra.run.smithi203.stdout: Running scriptlet: libradosstriper1-2:18.0.0-5389.g668a86f2.el9.x86 17/177 2023-08-09T13:31:56.640 INFO:teuthology.orchestra.run.smithi043.stdout: Running scriptlet: NetworkManager-1:1.43.90-1.el9.x86_64 21/177 2023-08-09T13:31:56.684 INFO:teuthology.orchestra.run.smithi203.stdout: Upgrading : libmount-2.37.4-11.el9.x86_64 18/177 2023-08-09T13:31:56.692 INFO:teuthology.orchestra.run.smithi043.stdout: Upgrading : NetworkManager-1:1.43.90-1.el9.x86_64 21/177 2023-08-09T13:31:57.017 INFO:teuthology.orchestra.run.smithi203.stdout: Upgrading : libsmartcols-2.37.4-11.el9.x86_64 19/177 2023-08-09T13:31:57.162 INFO:teuthology.orchestra.run.smithi043.stdout: Running scriptlet: NetworkManager-1:1.43.90-1.el9.x86_64 21/177 2023-08-09T13:31:57.175 INFO:teuthology.orchestra.run.smithi203.stdout: Upgrading : NetworkManager-libnm-1:1.43.90-1.el9.x86_64 20/177 2023-08-09T13:31:57.208 INFO:teuthology.orchestra.run.smithi043.stdout: Installing : python3-werkzeug-2.0.3-3.el9.1.noarch 22/177 2023-08-09T13:31:57.242 INFO:teuthology.orchestra.run.smithi043.stdout: Installing : liboath-2.6.7-2.el9.x86_64 23/177 2023-08-09T13:31:57.246 INFO:teuthology.orchestra.run.smithi203.stdout: Running scriptlet: NetworkManager-libnm-1:1.43.90-1.el9.x86_64 20/177 2023-08-09T13:31:57.338 INFO:teuthology.orchestra.run.smithi043.stdout: Installing : libquadmath-11.4.1-2.1.el9.x86_64 24/177 2023-08-09T13:31:57.400 INFO:teuthology.orchestra.run.smithi043.stdout: Installing : libgfortran-11.4.1-2.1.el9.x86_64 25/177 2023-08-09T13:31:57.459 INFO:teuthology.orchestra.run.smithi203.stdout: Running scriptlet: NetworkManager-1:1.43.90-1.el9.x86_64 21/177 2023-08-09T13:31:57.517 INFO:teuthology.orchestra.run.smithi043.stdout: Installing : python3-pytz-2021.1-5.el9.noarch 26/177 2023-08-09T13:31:57.537 INFO:teuthology.orchestra.run.smithi203.stdout: Upgrading : NetworkManager-1:1.43.90-1.el9.x86_64 21/177 2023-08-09T13:31:57.561 INFO:teuthology.orchestra.run.smithi043.stdout: Installing : python3-pyasn1-0.4.8-6.el9.noarch 27/177 2023-08-09T13:31:57.594 INFO:teuthology.orchestra.run.smithi043.stdout: Installing : python3-markupsafe-1.1.1-12.el9.x86_64 28/177 2023-08-09T13:31:57.709 INFO:teuthology.orchestra.run.smithi043.stdout: Installing : flexiblas-3.0.4-8.el9.x86_64 29/177 2023-08-09T13:31:57.749 INFO:teuthology.orchestra.run.smithi043.stdout: Installing : python3-ceph-common-2:18.0.0-5389.g668a86f2.el9. 30/177 2023-08-09T13:31:57.800 INFO:teuthology.orchestra.run.smithi043.stdout: Installing : python3-ceph-argparse-2:18.0.0-5389.g668a86f2.el 31/177 2023-08-09T13:31:57.918 INFO:teuthology.orchestra.run.smithi043.stdout: Installing : python3-cephfs-2:18.0.0-5389.g668a86f2.el9.x86_6 32/177 2023-08-09T13:31:57.984 INFO:teuthology.orchestra.run.smithi203.stdout: Running scriptlet: NetworkManager-1:1.43.90-1.el9.x86_64 21/177 2023-08-09T13:31:58.026 INFO:teuthology.orchestra.run.smithi203.stdout: Installing : python3-werkzeug-2.0.3-3.el9.1.noarch 22/177 2023-08-09T13:31:58.061 INFO:teuthology.orchestra.run.smithi203.stdout: Installing : liboath-2.6.7-2.el9.x86_64 23/177 2023-08-09T13:31:58.101 INFO:teuthology.orchestra.run.smithi043.stdout: Installing : python3-mako-1.1.4-6.el9.noarch 33/177 2023-08-09T13:31:58.123 INFO:teuthology.orchestra.run.smithi203.stdout: Installing : libquadmath-11.4.1-2.1.el9.x86_64 24/177 2023-08-09T13:31:58.169 INFO:teuthology.orchestra.run.smithi043.stdout: Installing : python3-pyasn1-modules-0.4.8-6.el9.noarch 34/177 2023-08-09T13:31:58.190 INFO:teuthology.orchestra.run.smithi203.stdout: Installing : libgfortran-11.4.1-2.1.el9.x86_64 25/177 2023-08-09T13:31:58.295 INFO:teuthology.orchestra.run.smithi203.stdout: Installing : python3-pytz-2021.1-5.el9.noarch 26/177 2023-08-09T13:31:58.339 INFO:teuthology.orchestra.run.smithi203.stdout: Installing : python3-pyasn1-0.4.8-6.el9.noarch 27/177 2023-08-09T13:31:58.363 INFO:teuthology.orchestra.run.smithi203.stdout: Installing : python3-markupsafe-1.1.1-12.el9.x86_64 28/177 2023-08-09T13:31:58.459 INFO:teuthology.orchestra.run.smithi203.stdout: Installing : flexiblas-3.0.4-8.el9.x86_64 29/177 2023-08-09T13:31:58.510 INFO:teuthology.orchestra.run.smithi203.stdout: Installing : python3-ceph-common-2:18.0.0-5389.g668a86f2.el9. 30/177 2023-08-09T13:31:58.562 INFO:teuthology.orchestra.run.smithi203.stdout: Installing : python3-ceph-argparse-2:18.0.0-5389.g668a86f2.el 31/177 2023-08-09T13:31:58.675 INFO:teuthology.orchestra.run.smithi203.stdout: Installing : python3-cephfs-2:18.0.0-5389.g668a86f2.el9.x86_6 32/177 2023-08-09T13:31:58.857 INFO:teuthology.orchestra.run.smithi203.stdout: Installing : python3-mako-1.1.4-6.el9.noarch 33/177 2023-08-09T13:31:58.929 INFO:teuthology.orchestra.run.smithi203.stdout: Installing : python3-pyasn1-modules-0.4.8-6.el9.noarch 34/177 2023-08-09T13:31:58.933 INFO:teuthology.orchestra.run.smithi043.stdout: Installing : python3-rsa-4.9-2.el9.noarch 35/177 2023-08-09T13:31:59.045 INFO:teuthology.orchestra.run.smithi043.stdout: Installing : python3-babel-2.9.1-2.el9.noarch 36/177 2023-08-09T13:31:59.193 INFO:teuthology.orchestra.run.smithi043.stdout: Installing : python3-jinja2-2.11.3-4.el9.noarch 37/177 2023-08-09T13:31:59.240 INFO:teuthology.orchestra.run.smithi043.stdout: Upgrading : util-linux-core-2.37.4-11.el9.x86_64 38/177 2023-08-09T13:31:59.287 INFO:teuthology.orchestra.run.smithi043.stdout: Running scriptlet: util-linux-core-2.37.4-11.el9.x86_64 38/177 2023-08-09T13:31:59.333 INFO:teuthology.orchestra.run.smithi043.stdout: Installing : librados-devel-2:18.0.0-5389.g668a86f2.el9.x86_6 39/177 2023-08-09T13:31:59.382 INFO:teuthology.orchestra.run.smithi043.stdout: Installing : python3-jaraco-classes-3.2.1-5.el9.noarch 40/177 2023-08-09T13:31:59.560 INFO:teuthology.orchestra.run.smithi043.stdout: Upgrading : elfutils-libs-0.189-3.el9.x86_64 41/177 2023-08-09T13:31:59.610 INFO:teuthology.orchestra.run.smithi043.stdout: Upgrading : iproute-6.2.0-5.el9.x86_64 42/177 2023-08-09T13:31:59.618 INFO:teuthology.orchestra.run.smithi203.stdout: Installing : python3-rsa-4.9-2.el9.noarch 35/177 2023-08-09T13:31:59.717 INFO:teuthology.orchestra.run.smithi203.stdout: Installing : python3-babel-2.9.1-2.el9.noarch 36/177 2023-08-09T13:31:59.861 INFO:teuthology.orchestra.run.smithi203.stdout: Installing : python3-jinja2-2.11.3-4.el9.noarch 37/177 2023-08-09T13:31:59.910 INFO:teuthology.orchestra.run.smithi203.stdout: Upgrading : util-linux-core-2.37.4-11.el9.x86_64 38/177 2023-08-09T13:31:59.955 INFO:teuthology.orchestra.run.smithi203.stdout: Running scriptlet: util-linux-core-2.37.4-11.el9.x86_64 38/177 2023-08-09T13:31:59.961 INFO:teuthology.orchestra.run.smithi043.stdout: Upgrading : libfdisk-2.37.4-11.el9.x86_64 43/177 2023-08-09T13:32:00.019 INFO:teuthology.orchestra.run.smithi203.stdout: Installing : librados-devel-2:18.0.0-5389.g668a86f2.el9.x86_6 39/177 2023-08-09T13:32:00.060 INFO:teuthology.orchestra.run.smithi203.stdout: Installing : python3-jaraco-classes-3.2.1-5.el9.noarch 40/177 2023-08-09T13:32:00.161 INFO:teuthology.orchestra.run.smithi043.stdout: Upgrading : util-linux-2.37.4-11.el9.x86_64 44/177 2023-08-09T13:32:00.217 INFO:teuthology.orchestra.run.smithi203.stdout: Upgrading : elfutils-libs-0.189-3.el9.x86_64 41/177 2023-08-09T13:32:00.271 INFO:teuthology.orchestra.run.smithi203.stdout: Upgrading : iproute-6.2.0-5.el9.x86_64 42/177 2023-08-09T13:32:00.287 INFO:teuthology.orchestra.run.smithi043.stdout: Upgrading : xfsprogs-5.19.0-4.el9.x86_64 45/177 2023-08-09T13:32:00.356 INFO:teuthology.orchestra.run.smithi043.stdout: Installing : libuuid-devel-2.37.4-11.el9.x86_64 46/177 2023-08-09T13:32:00.569 INFO:teuthology.orchestra.run.smithi043.stdout: Installing : socat-1.7.4.1-5.el9.x86_64 47/177 2023-08-09T13:32:00.604 INFO:teuthology.orchestra.run.smithi203.stdout: Upgrading : libfdisk-2.37.4-11.el9.x86_64 43/177 2023-08-09T13:32:00.646 INFO:teuthology.orchestra.run.smithi043.stdout: Upgrading : openssl-1:3.0.7-24.el9.x86_64 48/177 2023-08-09T13:32:00.808 INFO:teuthology.orchestra.run.smithi043.stdout: Installing : re2-1:20211101-2.el9.x86_64 49/177 2023-08-09T13:32:00.827 INFO:teuthology.orchestra.run.smithi203.stdout: Upgrading : util-linux-2.37.4-11.el9.x86_64 44/177 2023-08-09T13:32:00.857 INFO:teuthology.orchestra.run.smithi043.stdout: Installing : libarrow-9.0.0-9.el9.x86_64 50/177 2023-08-09T13:32:00.936 INFO:teuthology.orchestra.run.smithi043.stdout: Installing : python3-zc-lockfile-2.0-10.el9.noarch 51/177 2023-08-09T13:32:00.998 INFO:teuthology.orchestra.run.smithi203.stdout: Upgrading : xfsprogs-5.19.0-4.el9.x86_64 45/177 2023-08-09T13:32:01.033 INFO:teuthology.orchestra.run.smithi043.stdout: Installing : python3-websocket-client-1.2.3-2.el9.noarch 52/177 2023-08-09T13:32:01.075 INFO:teuthology.orchestra.run.smithi043.stdout: Installing : python3-webob-1.8.7-6.el9.noarch 53/177 2023-08-09T13:32:01.080 INFO:teuthology.orchestra.run.smithi203.stdout: Installing : libuuid-devel-2.37.4-11.el9.x86_64 46/177 2023-08-09T13:32:01.114 INFO:teuthology.orchestra.run.smithi043.stdout: Installing : python3-typing-extensions-4.4.0-2.el9.noarch 54/177 2023-08-09T13:32:01.182 INFO:teuthology.orchestra.run.smithi043.stdout: Installing : python3-repoze-lru-0.7-16.el9.noarch 55/177 2023-08-09T13:32:01.241 INFO:teuthology.orchestra.run.smithi043.stdout: Installing : python3-routes-2.5.1-5.el9.noarch 56/177 2023-08-09T13:32:01.298 INFO:teuthology.orchestra.run.smithi043.stdout: Installing : python3-natsort-7.1.1-5.el9.noarch 57/177 2023-08-09T13:32:01.323 INFO:teuthology.orchestra.run.smithi203.stdout: Installing : socat-1.7.4.1-5.el9.x86_64 47/177 2023-08-09T13:32:01.399 INFO:teuthology.orchestra.run.smithi203.stdout: Upgrading : openssl-1:3.0.7-24.el9.x86_64 48/177 2023-08-09T13:32:01.496 INFO:teuthology.orchestra.run.smithi043.stdout: Installing : python3-logutils-0.3.5-21.el9.noarch 58/177 2023-08-09T13:32:01.548 INFO:teuthology.orchestra.run.smithi043.stdout: Installing : python3-pecan-1.4.2-3.el9.noarch 59/177 2023-08-09T13:32:01.580 INFO:teuthology.orchestra.run.smithi203.stdout: Installing : re2-1:20211101-2.el9.x86_64 49/177 2023-08-09T13:32:01.603 INFO:teuthology.orchestra.run.smithi043.stdout: Installing : python3-certifi-2021.10.8-2.el9.noarch 60/177 2023-08-09T13:32:01.653 INFO:teuthology.orchestra.run.smithi203.stdout: Installing : libarrow-9.0.0-9.el9.x86_64 50/177 2023-08-09T13:32:01.736 INFO:teuthology.orchestra.run.smithi043.stdout: Installing : python3-cachetools-4.2.4-1.el9.noarch 61/177 2023-08-09T13:32:01.756 INFO:teuthology.orchestra.run.smithi203.stdout: Installing : python3-zc-lockfile-2.0-10.el9.noarch 51/177 2023-08-09T13:32:01.780 INFO:teuthology.orchestra.run.smithi043.stdout: Installing : python3-google-auth-1:2.22.0-1.el9.noarch 62/177 2023-08-09T13:32:01.829 INFO:teuthology.orchestra.run.smithi043.stdout: Installing : libunwind-1.6.2-1.el9.x86_64 63/177 2023-08-09T13:32:01.863 INFO:teuthology.orchestra.run.smithi043.stdout: Installing : gperftools-libs-2.9.1-2.el9.x86_64 64/177 2023-08-09T13:32:01.887 INFO:teuthology.orchestra.run.smithi203.stdout: Installing : python3-websocket-client-1.2.3-2.el9.noarch 52/177 2023-08-09T13:32:01.901 INFO:teuthology.orchestra.run.smithi043.stdout: Installing : libarrow-doc-9.0.0-9.el9.noarch 65/177 2023-08-09T13:32:01.945 INFO:teuthology.orchestra.run.smithi203.stdout: Installing : python3-webob-1.8.7-6.el9.noarch 53/177 2023-08-09T13:32:01.946 INFO:teuthology.orchestra.run.smithi043.stdout: Installing : fmt-8.1.1-5.el9.x86_64 66/177 2023-08-09T13:32:02.035 INFO:teuthology.orchestra.run.smithi203.stdout: Installing : python3-typing-extensions-4.4.0-2.el9.noarch 54/177 2023-08-09T13:32:02.121 INFO:teuthology.orchestra.run.smithi043.stdout: Installing : zlib-devel-1.2.11-36.el9.x86_64 67/177 2023-08-09T13:32:02.138 INFO:teuthology.orchestra.run.smithi203.stdout: Installing : python3-repoze-lru-0.7-16.el9.noarch 55/177 2023-08-09T13:32:02.168 INFO:teuthology.orchestra.run.smithi043.stdout: Installing : python3-devel-3.9.16-1.el9.x86_64 68/177 2023-08-09T13:32:02.238 INFO:teuthology.orchestra.run.smithi203.stdout: Installing : python3-routes-2.5.1-5.el9.noarch 56/177 2023-08-09T13:32:02.239 INFO:teuthology.orchestra.run.smithi043.stdout: Installing : python3-ply-3.11-14.el9.noarch 69/177 2023-08-09T13:32:02.324 INFO:teuthology.orchestra.run.smithi203.stdout: Installing : python3-natsort-7.1.1-5.el9.noarch 57/177 2023-08-09T13:32:02.329 INFO:teuthology.orchestra.run.smithi043.stdout: Installing : python3-pycparser-2.20-6.el9.noarch 70/177 2023-08-09T13:32:02.547 INFO:teuthology.orchestra.run.smithi203.stdout: Installing : python3-logutils-0.3.5-21.el9.noarch 58/177 2023-08-09T13:32:02.611 INFO:teuthology.orchestra.run.smithi043.stdout: Installing : python3-cffi-1.14.5-5.el9.x86_64 71/177 2023-08-09T13:32:02.627 INFO:teuthology.orchestra.run.smithi203.stdout: Installing : python3-pecan-1.4.2-3.el9.noarch 59/177 2023-08-09T13:32:02.690 INFO:teuthology.orchestra.run.smithi043.stdout: Installing : python3-cryptography-36.0.1-4.el9.x86_64 72/177 2023-08-09T13:32:02.698 INFO:teuthology.orchestra.run.smithi203.stdout: Installing : python3-certifi-2021.10.8-2.el9.noarch 60/177 2023-08-09T13:32:02.849 INFO:teuthology.orchestra.run.smithi203.stdout: Installing : python3-cachetools-4.2.4-1.el9.noarch 61/177 2023-08-09T13:32:02.866 INFO:teuthology.orchestra.run.smithi043.stdout: Installing : python3-pyOpenSSL-21.0.0-1.el9.noarch 73/177 2023-08-09T13:32:02.917 INFO:teuthology.orchestra.run.smithi203.stdout: Installing : python3-google-auth-1:2.22.0-1.el9.noarch 62/177 2023-08-09T13:32:02.925 INFO:teuthology.orchestra.run.smithi043.stdout: Installing : python3-asyncssh-2.13.2-3.el9.noarch 74/177 2023-08-09T13:32:02.949 INFO:teuthology.orchestra.run.smithi043.stdout: Installing : python3-jwt-2.4.0-1.el9.noarch 75/177 2023-08-09T13:32:02.988 INFO:teuthology.orchestra.run.smithi043.stdout: Installing : python3-jwt+crypto-2.4.0-1.el9.noarch 76/177 2023-08-09T13:32:03.019 INFO:teuthology.orchestra.run.smithi203.stdout: Installing : libunwind-1.6.2-1.el9.x86_64 63/177 2023-08-09T13:32:03.031 INFO:teuthology.orchestra.run.smithi043.stdout: Installing : python3-bcrypt-3.2.2-1.el9.x86_64 77/177 2023-08-09T13:32:03.076 INFO:teuthology.orchestra.run.smithi203.stdout: Installing : gperftools-libs-2.9.1-2.el9.x86_64 64/177 2023-08-09T13:32:03.083 INFO:teuthology.orchestra.run.smithi043.stdout: Installing : fuse-2.9.9-15.el9.x86_64 78/177 2023-08-09T13:32:03.135 INFO:teuthology.orchestra.run.smithi203.stdout: Installing : libarrow-doc-9.0.0-9.el9.noarch 65/177 2023-08-09T13:32:03.136 INFO:teuthology.orchestra.run.smithi043.stdout: Installing : xmlstarlet-1.6.1-20.el9.x86_64 79/177 2023-08-09T13:32:03.193 INFO:teuthology.orchestra.run.smithi203.stdout: Installing : fmt-8.1.1-5.el9.x86_64 66/177 2023-08-09T13:32:03.200 INFO:teuthology.orchestra.run.smithi043.stdout: Installing : uuid-1.6.2-55.el9.x86_64 80/177 2023-08-09T13:32:03.244 INFO:teuthology.orchestra.run.smithi043.stdout: Installing : python3-toml-0.10.2-6.el9.noarch 81/177 2023-08-09T13:32:03.375 INFO:teuthology.orchestra.run.smithi043.stdout: Installing : python3-jaraco-functools-3.5.0-2.el9.noarch 82/177 2023-08-09T13:32:03.397 INFO:teuthology.orchestra.run.smithi203.stdout: Installing : zlib-devel-1.2.11-36.el9.x86_64 67/177 2023-08-09T13:32:03.413 INFO:teuthology.orchestra.run.smithi043.stdout: Installing : python3-cheroot-8.6.0-4.el9.noarch 83/177 2023-08-09T13:32:03.448 INFO:teuthology.orchestra.run.smithi043.stdout: Installing : python3-jaraco-text-3.2.0-6.el9.noarch 84/177 2023-08-09T13:32:03.484 INFO:teuthology.orchestra.run.smithi203.stdout: Installing : python3-devel-3.9.16-1.el9.x86_64 68/177 2023-08-09T13:32:03.509 INFO:teuthology.orchestra.run.smithi043.stdout: Installing : python3-jaraco-collections-3.0.0-8.el9.noarch 85/177 2023-08-09T13:32:03.552 INFO:teuthology.orchestra.run.smithi043.stdout: Installing : python3-tempora-5.0.0-2.el9.noarch 86/177 2023-08-09T13:32:03.585 INFO:teuthology.orchestra.run.smithi203.stdout: Installing : python3-ply-3.11-14.el9.noarch 69/177 2023-08-09T13:32:03.696 INFO:teuthology.orchestra.run.smithi203.stdout: Installing : python3-pycparser-2.20-6.el9.noarch 70/177 2023-08-09T13:32:03.704 INFO:teuthology.orchestra.run.smithi043.stdout: Installing : python3-portend-3.1.0-2.el9.noarch 87/177 2023-08-09T13:32:03.750 INFO:teuthology.orchestra.run.smithi043.stdout: Installing : python3-cherrypy-18.6.1-2.el9.noarch 88/177 2023-08-09T13:32:03.898 INFO:teuthology.orchestra.run.smithi043.stdout: Installing : python3-prettytable-0.7.2-27.el9.noarch 89/177 2023-08-09T13:32:03.972 INFO:teuthology.orchestra.run.smithi203.stdout: Installing : python3-cffi-1.14.5-5.el9.x86_64 71/177 2023-08-09T13:32:03.972 INFO:teuthology.orchestra.run.smithi043.stdout: Installing : python3-oauthlib-3.1.1-5.el9.noarch 90/177 2023-08-09T13:32:04.036 INFO:teuthology.orchestra.run.smithi203.stdout: Installing : python3-cryptography-36.0.1-4.el9.x86_64 72/177 2023-08-09T13:32:04.220 INFO:teuthology.orchestra.run.smithi203.stdout: Installing : python3-pyOpenSSL-21.0.0-1.el9.noarch 73/177 2023-08-09T13:32:04.280 INFO:teuthology.orchestra.run.smithi203.stdout: Installing : python3-asyncssh-2.13.2-3.el9.noarch 74/177 2023-08-09T13:32:04.320 INFO:teuthology.orchestra.run.smithi203.stdout: Installing : python3-jwt-2.4.0-1.el9.noarch 75/177 2023-08-09T13:32:04.369 INFO:teuthology.orchestra.run.smithi203.stdout: Installing : python3-jwt+crypto-2.4.0-1.el9.noarch 76/177 2023-08-09T13:32:04.421 INFO:teuthology.orchestra.run.smithi203.stdout: Installing : python3-bcrypt-3.2.2-1.el9.x86_64 77/177 2023-08-09T13:32:04.483 INFO:teuthology.orchestra.run.smithi203.stdout: Installing : fuse-2.9.9-15.el9.x86_64 78/177 2023-08-09T13:32:04.560 INFO:teuthology.orchestra.run.smithi203.stdout: Installing : xmlstarlet-1.6.1-20.el9.x86_64 79/177 2023-08-09T13:32:04.633 INFO:teuthology.orchestra.run.smithi203.stdout: Installing : uuid-1.6.2-55.el9.x86_64 80/177 2023-08-09T13:32:04.693 INFO:teuthology.orchestra.run.smithi203.stdout: Installing : python3-toml-0.10.2-6.el9.noarch 81/177 2023-08-09T13:32:04.834 INFO:teuthology.orchestra.run.smithi203.stdout: Installing : python3-jaraco-functools-3.5.0-2.el9.noarch 82/177 2023-08-09T13:32:04.878 INFO:teuthology.orchestra.run.smithi203.stdout: Installing : python3-cheroot-8.6.0-4.el9.noarch 83/177 2023-08-09T13:32:04.929 INFO:teuthology.orchestra.run.smithi203.stdout: Installing : python3-jaraco-text-3.2.0-6.el9.noarch 84/177 2023-08-09T13:32:05.001 INFO:teuthology.orchestra.run.smithi203.stdout: Installing : python3-jaraco-collections-3.0.0-8.el9.noarch 85/177 2023-08-09T13:32:05.016 INFO:teuthology.orchestra.run.smithi043.stdout: Installing : python3-requests-oauthlib-1.3.0-12.el9.noarch 91/177 2023-08-09T13:32:05.057 INFO:teuthology.orchestra.run.smithi203.stdout: Installing : python3-tempora-5.0.0-2.el9.noarch 86/177 2023-08-09T13:32:05.080 INFO:teuthology.orchestra.run.smithi043.stdout: Installing : python3-kubernetes-1:26.1.0-2.el9.noarch 92/177 2023-08-09T13:32:05.114 INFO:teuthology.orchestra.run.smithi043.stdout: Installing : python3-jsonpointer-2.0-4.el9.noarch 93/177 2023-08-09T13:32:05.146 INFO:teuthology.orchestra.run.smithi043.stdout: Installing : python3-jsonpatch-1.21-16.el9.noarch 94/177 2023-08-09T13:32:05.218 INFO:teuthology.orchestra.run.smithi203.stdout: Installing : python3-portend-3.1.0-2.el9.noarch 87/177 2023-08-09T13:32:05.270 INFO:teuthology.orchestra.run.smithi203.stdout: Installing : python3-cherrypy-18.6.1-2.el9.noarch 88/177 2023-08-09T13:32:05.421 INFO:teuthology.orchestra.run.smithi203.stdout: Installing : python3-prettytable-0.7.2-27.el9.noarch 89/177 2023-08-09T13:32:05.427 INFO:teuthology.orchestra.run.smithi043.stdout: Installing : openblas-0.3.21-2.el9.x86_64 95/177 2023-08-09T13:32:05.459 INFO:teuthology.orchestra.run.smithi043.stdout: Installing : openblas-openmp-0.3.21-2.el9.x86_64 96/177 2023-08-09T13:32:05.494 INFO:teuthology.orchestra.run.smithi203.stdout: Installing : python3-oauthlib-3.1.1-5.el9.noarch 90/177 2023-08-09T13:32:05.564 INFO:teuthology.orchestra.run.smithi043.stdout: Installing : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 97/177 2023-08-09T13:32:06.435 INFO:teuthology.orchestra.run.smithi043.stdout: Installing : flexiblas-netlib-3.0.4-8.el9.x86_64 98/177 2023-08-09T13:32:06.514 INFO:teuthology.orchestra.run.smithi203.stdout: Installing : python3-requests-oauthlib-1.3.0-12.el9.noarch 91/177 2023-08-09T13:32:06.567 INFO:teuthology.orchestra.run.smithi043.stdout: Installing : python3-numpy-1:1.20.1-5.el9.x86_64 99/177 2023-08-09T13:32:06.585 INFO:teuthology.orchestra.run.smithi203.stdout: Installing : python3-kubernetes-1:26.1.0-2.el9.noarch 92/177 2023-08-09T13:32:06.626 INFO:teuthology.orchestra.run.smithi203.stdout: Installing : python3-jsonpointer-2.0-4.el9.noarch 93/177 2023-08-09T13:32:06.669 INFO:teuthology.orchestra.run.smithi203.stdout: Installing : python3-jsonpatch-1.21-16.el9.noarch 94/177 2023-08-09T13:32:06.960 INFO:teuthology.orchestra.run.smithi203.stdout: Installing : openblas-0.3.21-2.el9.x86_64 95/177 2023-08-09T13:32:07.014 INFO:teuthology.orchestra.run.smithi203.stdout: Installing : openblas-openmp-0.3.21-2.el9.x86_64 96/177 2023-08-09T13:32:07.124 INFO:teuthology.orchestra.run.smithi203.stdout: Installing : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 97/177 2023-08-09T13:32:07.955 INFO:teuthology.orchestra.run.smithi203.stdout: Installing : flexiblas-netlib-3.0.4-8.el9.x86_64 98/177 2023-08-09T13:32:08.105 INFO:teuthology.orchestra.run.smithi203.stdout: Installing : python3-numpy-1:1.20.1-5.el9.x86_64 99/177 2023-08-09T13:32:08.224 INFO:teuthology.orchestra.run.smithi043.stdout: Installing : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 100/177 2023-08-09T13:32:08.322 INFO:teuthology.orchestra.run.smithi043.stdout: Installing : python3-scipy-1.6.2-8.el9.x86_64 101/177 2023-08-09T13:32:08.359 INFO:teuthology.orchestra.run.smithi043.stdout: Installing : oniguruma-6.9.6-1.el9.5.x86_64 102/177 2023-08-09T13:32:08.390 INFO:teuthology.orchestra.run.smithi043.stdout: Installing : jq-1.6-15.el9.x86_64 103/177 2023-08-09T13:32:08.431 INFO:teuthology.orchestra.run.smithi043.stdout: Installing : libzstd-devel-1.5.1-2.el9.x86_64 104/177 2023-08-09T13:32:08.829 INFO:teuthology.orchestra.run.smithi043.stdout: Installing : libpmemobj-1.12.1-1.el9.x86_64 105/177 2023-08-09T13:32:08.862 INFO:teuthology.orchestra.run.smithi043.stdout: Upgrading : librbd1-2:18.0.0-5389.g668a86f2.el9.x86_64 106/177 2023-08-09T13:32:08.983 INFO:teuthology.orchestra.run.smithi043.stdout: Running scriptlet: librbd1-2:18.0.0-5389.g668a86f2.el9.x86_64 106/177 2023-08-09T13:32:09.017 INFO:teuthology.orchestra.run.smithi043.stdout: Installing : python3-rbd-2:18.0.0-5389.g668a86f2.el9.x86_64 107/177 2023-08-09T13:32:09.084 INFO:teuthology.orchestra.run.smithi043.stdout: Installing : boost-program-options-1.75.0-8.el9.x86_64 108/177 2023-08-09T13:32:09.597 INFO:teuthology.orchestra.run.smithi043.stdout: Installing : parquet-libs-9.0.0-9.el9.x86_64 109/177 2023-08-09T13:32:09.631 INFO:teuthology.orchestra.run.smithi043.stdout: Installing : librgw2-2:18.0.0-5389.g668a86f2.el9.x86_64 110/177 2023-08-09T13:32:09.705 INFO:teuthology.orchestra.run.smithi043.stdout: Running scriptlet: librgw2-2:18.0.0-5389.g668a86f2.el9.x86_64 110/177 2023-08-09T13:32:09.733 INFO:teuthology.orchestra.run.smithi043.stdout: Installing : python3-rgw-2:18.0.0-5389.g668a86f2.el9.x86_64 111/177 2023-08-09T13:32:10.341 INFO:teuthology.orchestra.run.smithi203.stdout: Installing : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 100/177 2023-08-09T13:32:10.446 INFO:teuthology.orchestra.run.smithi203.stdout: Installing : python3-scipy-1.6.2-8.el9.x86_64 101/177 2023-08-09T13:32:10.492 INFO:teuthology.orchestra.run.smithi203.stdout: Installing : oniguruma-6.9.6-1.el9.5.x86_64 102/177 2023-08-09T13:32:10.532 INFO:teuthology.orchestra.run.smithi203.stdout: Installing : jq-1.6-15.el9.x86_64 103/177 2023-08-09T13:32:10.572 INFO:teuthology.orchestra.run.smithi203.stdout: Installing : libzstd-devel-1.5.1-2.el9.x86_64 104/177 2023-08-09T13:32:10.945 INFO:teuthology.orchestra.run.smithi203.stdout: Installing : libpmemobj-1.12.1-1.el9.x86_64 105/177 2023-08-09T13:32:10.987 INFO:teuthology.orchestra.run.smithi203.stdout: Upgrading : librbd1-2:18.0.0-5389.g668a86f2.el9.x86_64 106/177 2023-08-09T13:32:11.099 INFO:teuthology.orchestra.run.smithi203.stdout: Running scriptlet: librbd1-2:18.0.0-5389.g668a86f2.el9.x86_64 106/177 2023-08-09T13:32:11.168 INFO:teuthology.orchestra.run.smithi203.stdout: Installing : python3-rbd-2:18.0.0-5389.g668a86f2.el9.x86_64 107/177 2023-08-09T13:32:11.239 INFO:teuthology.orchestra.run.smithi203.stdout: Installing : boost-program-options-1.75.0-8.el9.x86_64 108/177 2023-08-09T13:32:11.744 INFO:teuthology.orchestra.run.smithi203.stdout: Installing : parquet-libs-9.0.0-9.el9.x86_64 109/177 2023-08-09T13:32:11.781 INFO:teuthology.orchestra.run.smithi203.stdout: Installing : librgw2-2:18.0.0-5389.g668a86f2.el9.x86_64 110/177 2023-08-09T13:32:11.872 INFO:teuthology.orchestra.run.smithi203.stdout: Running scriptlet: librgw2-2:18.0.0-5389.g668a86f2.el9.x86_64 110/177 2023-08-09T13:32:11.908 INFO:teuthology.orchestra.run.smithi203.stdout: Installing : python3-rgw-2:18.0.0-5389.g668a86f2.el9.x86_64 111/177 2023-08-09T13:32:12.294 INFO:teuthology.orchestra.run.smithi043.stdout: Running scriptlet: ceph-common-2:18.0.0-5389.g668a86f2.el9.x86_64 112/177 2023-08-09T13:32:12.337 INFO:teuthology.orchestra.run.smithi043.stdout: Installing : ceph-common-2:18.0.0-5389.g668a86f2.el9.x86_64 112/177 2023-08-09T13:32:13.008 INFO:teuthology.orchestra.run.smithi043.stdout: Running scriptlet: ceph-common-2:18.0.0-5389.g668a86f2.el9.x86_64 112/177 2023-08-09T13:32:13.057 INFO:teuthology.orchestra.run.smithi043.stdout: Installing : ceph-base-2:18.0.0-5389.g668a86f2.el9.x86_64 113/177 2023-08-09T13:32:13.240 INFO:teuthology.orchestra.run.smithi043.stdout: Running scriptlet: ceph-base-2:18.0.0-5389.g668a86f2.el9.x86_64 113/177 2023-08-09T13:32:13.240 INFO:teuthology.orchestra.run.smithi043.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target → /usr/lib/systemd/system/ceph.target. 2023-08-09T13:32:13.240 INFO:teuthology.orchestra.run.smithi043.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-crash.service → /usr/lib/systemd/system/ceph-crash.service. 2023-08-09T13:32:13.241 INFO:teuthology.orchestra.run.smithi043.stdout: 2023-08-09T13:32:13.650 INFO:teuthology.orchestra.run.smithi043.stdout: Installing : ceph-selinux-2:18.0.0-5389.g668a86f2.el9.x86_64 114/177 2023-08-09T13:32:14.432 INFO:teuthology.orchestra.run.smithi203.stdout: Running scriptlet: ceph-common-2:18.0.0-5389.g668a86f2.el9.x86_64 112/177 2023-08-09T13:32:14.481 INFO:teuthology.orchestra.run.smithi203.stdout: Installing : ceph-common-2:18.0.0-5389.g668a86f2.el9.x86_64 112/177 2023-08-09T13:32:15.123 INFO:teuthology.orchestra.run.smithi203.stdout: Running scriptlet: ceph-common-2:18.0.0-5389.g668a86f2.el9.x86_64 112/177 2023-08-09T13:32:15.185 INFO:teuthology.orchestra.run.smithi203.stdout: Installing : ceph-base-2:18.0.0-5389.g668a86f2.el9.x86_64 113/177 2023-08-09T13:32:15.349 INFO:teuthology.orchestra.run.smithi203.stdout: Running scriptlet: ceph-base-2:18.0.0-5389.g668a86f2.el9.x86_64 113/177 2023-08-09T13:32:15.349 INFO:teuthology.orchestra.run.smithi203.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target → /usr/lib/systemd/system/ceph.target. 2023-08-09T13:32:15.350 INFO:teuthology.orchestra.run.smithi203.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-crash.service → /usr/lib/systemd/system/ceph-crash.service. 2023-08-09T13:32:15.350 INFO:teuthology.orchestra.run.smithi203.stdout: 2023-08-09T13:32:15.402 INFO:teuthology.orchestra.run.smithi203.stdout: Installing : ceph-selinux-2:18.0.0-5389.g668a86f2.el9.x86_64 114/177 2023-08-09T13:32:27.094 INFO:teuthology.orchestra.run.smithi043.stdout: Running scriptlet: ceph-selinux-2:18.0.0-5389.g668a86f2.el9.x86_64 114/177 2023-08-09T13:32:27.095 INFO:teuthology.orchestra.run.smithi043.stdout:skipping the directory /sys 2023-08-09T13:32:27.095 INFO:teuthology.orchestra.run.smithi043.stdout:skipping the directory /proc 2023-08-09T13:32:27.095 INFO:teuthology.orchestra.run.smithi043.stdout:skipping the directory /mnt 2023-08-09T13:32:27.095 INFO:teuthology.orchestra.run.smithi043.stdout:skipping the directory /var/tmp 2023-08-09T13:32:27.095 INFO:teuthology.orchestra.run.smithi043.stdout:skipping the directory /home 2023-08-09T13:32:27.095 INFO:teuthology.orchestra.run.smithi043.stdout:skipping the directory /root 2023-08-09T13:32:27.096 INFO:teuthology.orchestra.run.smithi043.stdout:skipping the directory /tmp 2023-08-09T13:32:27.096 INFO:teuthology.orchestra.run.smithi043.stdout: 2023-08-09T13:32:27.361 INFO:teuthology.orchestra.run.smithi043.stdout: Installing : ceph-volume-2:18.0.0-5389.g668a86f2.el9.noarch 115/177 2023-08-09T13:32:27.433 INFO:teuthology.orchestra.run.smithi043.stdout: Running scriptlet: ceph-volume-2:18.0.0-5389.g668a86f2.el9.noarch 115/177 2023-08-09T13:32:27.433 INFO:teuthology.orchestra.run.smithi043.stdout:Glob pattern passed to enable, but globs are not supported for this. 2023-08-09T13:32:27.433 INFO:teuthology.orchestra.run.smithi043.stdout:Invalid unit name "ceph-volume@*.service" escaped as "ceph-volume@\x2a.service". 2023-08-09T13:32:27.434 INFO:teuthology.orchestra.run.smithi043.stdout: 2023-08-09T13:32:28.155 INFO:teuthology.orchestra.run.smithi203.stdout: Running scriptlet: ceph-selinux-2:18.0.0-5389.g668a86f2.el9.x86_64 114/177 2023-08-09T13:32:28.156 INFO:teuthology.orchestra.run.smithi203.stdout:skipping the directory /sys 2023-08-09T13:32:28.156 INFO:teuthology.orchestra.run.smithi203.stdout:skipping the directory /proc 2023-08-09T13:32:28.156 INFO:teuthology.orchestra.run.smithi203.stdout:skipping the directory /mnt 2023-08-09T13:32:28.156 INFO:teuthology.orchestra.run.smithi203.stdout:skipping the directory /var/tmp 2023-08-09T13:32:28.156 INFO:teuthology.orchestra.run.smithi203.stdout:skipping the directory /home 2023-08-09T13:32:28.156 INFO:teuthology.orchestra.run.smithi203.stdout:skipping the directory /root 2023-08-09T13:32:28.156 INFO:teuthology.orchestra.run.smithi203.stdout:skipping the directory /tmp 2023-08-09T13:32:28.157 INFO:teuthology.orchestra.run.smithi203.stdout: 2023-08-09T13:32:28.377 INFO:teuthology.orchestra.run.smithi203.stdout: Installing : ceph-volume-2:18.0.0-5389.g668a86f2.el9.noarch 115/177 2023-08-09T13:32:28.445 INFO:teuthology.orchestra.run.smithi203.stdout: Running scriptlet: ceph-volume-2:18.0.0-5389.g668a86f2.el9.noarch 115/177 2023-08-09T13:32:28.445 INFO:teuthology.orchestra.run.smithi203.stdout:Glob pattern passed to enable, but globs are not supported for this. 2023-08-09T13:32:28.446 INFO:teuthology.orchestra.run.smithi203.stdout:Invalid unit name "ceph-volume@*.service" escaped as "ceph-volume@\x2a.service". 2023-08-09T13:32:28.446 INFO:teuthology.orchestra.run.smithi203.stdout: 2023-08-09T13:32:29.236 INFO:teuthology.orchestra.run.smithi043.stdout: Installing : ceph-osd-2:18.0.0-5389.g668a86f2.el9.x86_64 116/177 2023-08-09T13:32:29.343 INFO:teuthology.orchestra.run.smithi043.stdout: Running scriptlet: ceph-osd-2:18.0.0-5389.g668a86f2.el9.x86_64 116/177 2023-08-09T13:32:29.343 INFO:teuthology.orchestra.run.smithi043.stdout:Glob pattern passed to enable, but globs are not supported for this. 2023-08-09T13:32:29.343 INFO:teuthology.orchestra.run.smithi043.stdout:Invalid unit name "ceph-osd@*.service" escaped as "ceph-osd@\x2a.service". 2023-08-09T13:32:29.343 INFO:teuthology.orchestra.run.smithi043.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2023-08-09T13:32:29.344 INFO:teuthology.orchestra.run.smithi043.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2023-08-09T13:32:29.344 INFO:teuthology.orchestra.run.smithi043.stdout: 2023-08-09T13:32:29.586 INFO:teuthology.orchestra.run.smithi043.stdout: Installing : ceph-mds-2:18.0.0-5389.g668a86f2.el9.x86_64 117/177 2023-08-09T13:32:29.672 INFO:teuthology.orchestra.run.smithi043.stdout: Running scriptlet: ceph-mds-2:18.0.0-5389.g668a86f2.el9.x86_64 117/177 2023-08-09T13:32:29.672 INFO:teuthology.orchestra.run.smithi043.stdout:Glob pattern passed to enable, but globs are not supported for this. 2023-08-09T13:32:29.672 INFO:teuthology.orchestra.run.smithi043.stdout:Invalid unit name "ceph-mds@*.service" escaped as "ceph-mds@\x2a.service". 2023-08-09T13:32:29.672 INFO:teuthology.orchestra.run.smithi043.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2023-08-09T13:32:29.672 INFO:teuthology.orchestra.run.smithi043.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2023-08-09T13:32:29.673 INFO:teuthology.orchestra.run.smithi043.stdout: 2023-08-09T13:32:30.160 INFO:teuthology.orchestra.run.smithi203.stdout: Installing : ceph-osd-2:18.0.0-5389.g668a86f2.el9.x86_64 116/177 2023-08-09T13:32:30.229 INFO:teuthology.orchestra.run.smithi043.stdout: Installing : ceph-mon-2:18.0.0-5389.g668a86f2.el9.x86_64 118/177 2023-08-09T13:32:30.279 INFO:teuthology.orchestra.run.smithi203.stdout: Running scriptlet: ceph-osd-2:18.0.0-5389.g668a86f2.el9.x86_64 116/177 2023-08-09T13:32:30.279 INFO:teuthology.orchestra.run.smithi203.stdout:Glob pattern passed to enable, but globs are not supported for this. 2023-08-09T13:32:30.279 INFO:teuthology.orchestra.run.smithi203.stdout:Invalid unit name "ceph-osd@*.service" escaped as "ceph-osd@\x2a.service". 2023-08-09T13:32:30.279 INFO:teuthology.orchestra.run.smithi203.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2023-08-09T13:32:30.279 INFO:teuthology.orchestra.run.smithi203.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2023-08-09T13:32:30.279 INFO:teuthology.orchestra.run.smithi203.stdout: 2023-08-09T13:32:30.341 INFO:teuthology.orchestra.run.smithi043.stdout: Running scriptlet: ceph-mon-2:18.0.0-5389.g668a86f2.el9.x86_64 118/177 2023-08-09T13:32:30.342 INFO:teuthology.orchestra.run.smithi043.stdout:Glob pattern passed to enable, but globs are not supported for this. 2023-08-09T13:32:30.342 INFO:teuthology.orchestra.run.smithi043.stdout:Invalid unit name "ceph-mon@*.service" escaped as "ceph-mon@\x2a.service". 2023-08-09T13:32:30.342 INFO:teuthology.orchestra.run.smithi043.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2023-08-09T13:32:30.342 INFO:teuthology.orchestra.run.smithi043.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2023-08-09T13:32:30.342 INFO:teuthology.orchestra.run.smithi043.stdout: 2023-08-09T13:32:30.481 INFO:teuthology.orchestra.run.smithi043.stdout: Installing : automake-1.16.2-8.el9.noarch 119/177 2023-08-09T13:32:30.527 INFO:teuthology.orchestra.run.smithi203.stdout: Installing : ceph-mds-2:18.0.0-5389.g668a86f2.el9.x86_64 117/177 2023-08-09T13:32:30.635 INFO:teuthology.orchestra.run.smithi203.stdout: Running scriptlet: ceph-mds-2:18.0.0-5389.g668a86f2.el9.x86_64 117/177 2023-08-09T13:32:30.635 INFO:teuthology.orchestra.run.smithi203.stdout:Glob pattern passed to enable, but globs are not supported for this. 2023-08-09T13:32:30.635 INFO:teuthology.orchestra.run.smithi203.stdout:Invalid unit name "ceph-mds@*.service" escaped as "ceph-mds@\x2a.service". 2023-08-09T13:32:30.635 INFO:teuthology.orchestra.run.smithi203.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2023-08-09T13:32:30.636 INFO:teuthology.orchestra.run.smithi203.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2023-08-09T13:32:30.636 INFO:teuthology.orchestra.run.smithi203.stdout: 2023-08-09T13:32:31.116 INFO:teuthology.orchestra.run.smithi043.stdout: Running scriptlet: cephadm-2:18.0.0-5389.g668a86f2.el9.noarch 120/177 2023-08-09T13:32:31.145 INFO:teuthology.orchestra.run.smithi203.stdout: Installing : ceph-mon-2:18.0.0-5389.g668a86f2.el9.x86_64 118/177 2023-08-09T13:32:31.158 INFO:teuthology.orchestra.run.smithi043.stdout: Installing : cephadm-2:18.0.0-5389.g668a86f2.el9.noarch 120/177 2023-08-09T13:32:31.207 INFO:teuthology.orchestra.run.smithi043.stdout: Installing : ceph-prometheus-alerts-2:18.0.0-5389.g668a86f2.e 121/177 2023-08-09T13:32:31.255 INFO:teuthology.orchestra.run.smithi203.stdout: Running scriptlet: ceph-mon-2:18.0.0-5389.g668a86f2.el9.x86_64 118/177 2023-08-09T13:32:31.255 INFO:teuthology.orchestra.run.smithi203.stdout:Glob pattern passed to enable, but globs are not supported for this. 2023-08-09T13:32:31.255 INFO:teuthology.orchestra.run.smithi203.stdout:Invalid unit name "ceph-mon@*.service" escaped as "ceph-mon@\x2a.service". 2023-08-09T13:32:31.255 INFO:teuthology.orchestra.run.smithi203.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2023-08-09T13:32:31.255 INFO:teuthology.orchestra.run.smithi203.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2023-08-09T13:32:31.256 INFO:teuthology.orchestra.run.smithi203.stdout: 2023-08-09T13:32:31.314 INFO:teuthology.orchestra.run.smithi043.stdout: Installing : ceph-grafana-dashboards-2:18.0.0-5389.g668a86f2. 122/177 2023-08-09T13:32:31.343 INFO:teuthology.orchestra.run.smithi043.stdout: Installing : ceph-mgr-cephadm-2:18.0.0-5389.g668a86f2.el9.noa 123/177 2023-08-09T13:32:31.380 INFO:teuthology.orchestra.run.smithi203.stdout: Installing : automake-1.16.2-8.el9.noarch 119/177 2023-08-09T13:32:31.683 INFO:teuthology.orchestra.run.smithi043.stdout: Running scriptlet: ceph-mgr-cephadm-2:18.0.0-5389.g668a86f2.el9.noa 123/177 2023-08-09T13:32:31.719 INFO:teuthology.orchestra.run.smithi043.stdout: Installing : ceph-mgr-dashboard-2:18.0.0-5389.g668a86f2.el9.n 124/177 2023-08-09T13:32:32.052 INFO:teuthology.orchestra.run.smithi203.stdout: Running scriptlet: cephadm-2:18.0.0-5389.g668a86f2.el9.noarch 120/177 2023-08-09T13:32:32.091 INFO:teuthology.orchestra.run.smithi203.stdout: Installing : cephadm-2:18.0.0-5389.g668a86f2.el9.noarch 120/177 2023-08-09T13:32:32.145 INFO:teuthology.orchestra.run.smithi203.stdout: Installing : ceph-prometheus-alerts-2:18.0.0-5389.g668a86f2.e 121/177 2023-08-09T13:32:32.289 INFO:teuthology.orchestra.run.smithi203.stdout: Installing : ceph-grafana-dashboards-2:18.0.0-5389.g668a86f2. 122/177 2023-08-09T13:32:32.344 INFO:teuthology.orchestra.run.smithi203.stdout: Installing : ceph-mgr-cephadm-2:18.0.0-5389.g668a86f2.el9.noa 123/177 2023-08-09T13:32:32.677 INFO:teuthology.orchestra.run.smithi203.stdout: Running scriptlet: ceph-mgr-cephadm-2:18.0.0-5389.g668a86f2.el9.noa 123/177 2023-08-09T13:32:32.720 INFO:teuthology.orchestra.run.smithi203.stdout: Installing : ceph-mgr-dashboard-2:18.0.0-5389.g668a86f2.el9.n 124/177 2023-08-09T13:32:32.943 INFO:teuthology.orchestra.run.smithi043.stdout: Running scriptlet: ceph-mgr-dashboard-2:18.0.0-5389.g668a86f2.el9.n 124/177 2023-08-09T13:32:32.978 INFO:teuthology.orchestra.run.smithi043.stdout: Installing : ceph-mgr-diskprediction-local-2:18.0.0-5389.g668 125/177 2023-08-09T13:32:33.129 INFO:teuthology.orchestra.run.smithi043.stdout: Running scriptlet: ceph-mgr-diskprediction-local-2:18.0.0-5389.g668 125/177 2023-08-09T13:32:33.304 INFO:teuthology.orchestra.run.smithi043.stdout: Installing : ceph-mgr-modules-core-2:18.0.0-5389.g668a86f2.el 126/177 2023-08-09T13:32:33.330 INFO:teuthology.orchestra.run.smithi043.stdout: Installing : ceph-mgr-2:18.0.0-5389.g668a86f2.el9.x86_64 127/177 2023-08-09T13:32:33.414 INFO:teuthology.orchestra.run.smithi043.stdout: Running scriptlet: ceph-mgr-2:18.0.0-5389.g668a86f2.el9.x86_64 127/177 2023-08-09T13:32:33.414 INFO:teuthology.orchestra.run.smithi043.stdout:Glob pattern passed to enable, but globs are not supported for this. 2023-08-09T13:32:33.414 INFO:teuthology.orchestra.run.smithi043.stdout:Invalid unit name "ceph-mgr@*.service" escaped as "ceph-mgr@\x2a.service". 2023-08-09T13:32:33.414 INFO:teuthology.orchestra.run.smithi043.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2023-08-09T13:32:33.414 INFO:teuthology.orchestra.run.smithi043.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2023-08-09T13:32:33.415 INFO:teuthology.orchestra.run.smithi043.stdout: 2023-08-09T13:32:33.472 INFO:teuthology.orchestra.run.smithi043.stdout: Installing : ceph-mgr-rook-2:18.0.0-5389.g668a86f2.el9.noarch 128/177 2023-08-09T13:32:33.513 INFO:teuthology.orchestra.run.smithi043.stdout: Running scriptlet: ceph-mgr-rook-2:18.0.0-5389.g668a86f2.el9.noarch 128/177 2023-08-09T13:32:33.578 INFO:teuthology.orchestra.run.smithi043.stdout: Installing : ceph-2:18.0.0-5389.g668a86f2.el9.x86_64 129/177 2023-08-09T13:32:33.900 INFO:teuthology.orchestra.run.smithi203.stdout: Running scriptlet: ceph-mgr-dashboard-2:18.0.0-5389.g668a86f2.el9.n 124/177 2023-08-09T13:32:33.954 INFO:teuthology.orchestra.run.smithi203.stdout: Installing : ceph-mgr-diskprediction-local-2:18.0.0-5389.g668 125/177 2023-08-09T13:32:34.101 INFO:teuthology.orchestra.run.smithi203.stdout: Running scriptlet: ceph-mgr-diskprediction-local-2:18.0.0-5389.g668 125/177 2023-08-09T13:32:34.281 INFO:teuthology.orchestra.run.smithi203.stdout: Installing : ceph-mgr-modules-core-2:18.0.0-5389.g668a86f2.el 126/177 2023-08-09T13:32:34.322 INFO:teuthology.orchestra.run.smithi203.stdout: Installing : ceph-mgr-2:18.0.0-5389.g668a86f2.el9.x86_64 127/177 2023-08-09T13:32:34.431 INFO:teuthology.orchestra.run.smithi203.stdout: Running scriptlet: ceph-mgr-2:18.0.0-5389.g668a86f2.el9.x86_64 127/177 2023-08-09T13:32:34.432 INFO:teuthology.orchestra.run.smithi203.stdout:Glob pattern passed to enable, but globs are not supported for this. 2023-08-09T13:32:34.432 INFO:teuthology.orchestra.run.smithi203.stdout:Invalid unit name "ceph-mgr@*.service" escaped as "ceph-mgr@\x2a.service". 2023-08-09T13:32:34.432 INFO:teuthology.orchestra.run.smithi203.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2023-08-09T13:32:34.432 INFO:teuthology.orchestra.run.smithi203.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2023-08-09T13:32:34.432 INFO:teuthology.orchestra.run.smithi203.stdout: 2023-08-09T13:32:34.462 INFO:teuthology.orchestra.run.smithi043.stdout: Installing : libtool-2.4.6-45.el9.x86_64 130/177 2023-08-09T13:32:34.498 INFO:teuthology.orchestra.run.smithi203.stdout: Installing : ceph-mgr-rook-2:18.0.0-5389.g668a86f2.el9.noarch 128/177 2023-08-09T13:32:34.499 INFO:teuthology.orchestra.run.smithi043.stdout: Installing : ceph-radosgw-2:18.0.0-5389.g668a86f2.el9.x86_64 131/177 2023-08-09T13:32:34.538 INFO:teuthology.orchestra.run.smithi203.stdout: Running scriptlet: ceph-mgr-rook-2:18.0.0-5389.g668a86f2.el9.noarch 128/177 2023-08-09T13:32:34.590 INFO:teuthology.orchestra.run.smithi043.stdout: Running scriptlet: ceph-radosgw-2:18.0.0-5389.g668a86f2.el9.x86_64 131/177 2023-08-09T13:32:34.590 INFO:teuthology.orchestra.run.smithi043.stdout:Glob pattern passed to enable, but globs are not supported for this. 2023-08-09T13:32:34.590 INFO:teuthology.orchestra.run.smithi043.stdout:Invalid unit name "ceph-radosgw@*.service" escaped as "ceph-radosgw@\x2a.service". 2023-08-09T13:32:34.591 INFO:teuthology.orchestra.run.smithi043.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2023-08-09T13:32:34.591 INFO:teuthology.orchestra.run.smithi043.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2023-08-09T13:32:34.591 INFO:teuthology.orchestra.run.smithi043.stdout: 2023-08-09T13:32:34.641 INFO:teuthology.orchestra.run.smithi043.stdout: Installing : ceph-immutable-object-cache-2:18.0.0-5389.g668a8 132/177 2023-08-09T13:32:34.645 INFO:teuthology.orchestra.run.smithi203.stdout: Installing : ceph-2:18.0.0-5389.g668a86f2.el9.x86_64 129/177 2023-08-09T13:32:34.742 INFO:teuthology.orchestra.run.smithi043.stdout: Running scriptlet: ceph-immutable-object-cache-2:18.0.0-5389.g668a8 132/177 2023-08-09T13:32:34.742 INFO:teuthology.orchestra.run.smithi043.stdout:Glob pattern passed to enable, but globs are not supported for this. 2023-08-09T13:32:34.743 INFO:teuthology.orchestra.run.smithi043.stdout:Invalid unit name "ceph-immutable-object-cache@*.service" escaped as "ceph-immutable-object-cache@\x2a.service". 2023-08-09T13:32:34.743 INFO:teuthology.orchestra.run.smithi043.stdout: 2023-08-09T13:32:34.809 INFO:teuthology.orchestra.run.smithi043.stdout: Installing : cephfs-mirror-2:18.0.0-5389.g668a86f2.el9.x86_64 133/177 2023-08-09T13:32:34.926 INFO:teuthology.orchestra.run.smithi043.stdout: Running scriptlet: cephfs-mirror-2:18.0.0-5389.g668a86f2.el9.x86_64 133/177 2023-08-09T13:32:34.926 INFO:teuthology.orchestra.run.smithi043.stdout:Glob pattern passed to enable, but globs are not supported for this. 2023-08-09T13:32:34.926 INFO:teuthology.orchestra.run.smithi043.stdout:Invalid unit name "cephfs-mirror@*.service" escaped as "cephfs-mirror@\x2a.service". 2023-08-09T13:32:34.926 INFO:teuthology.orchestra.run.smithi043.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2023-08-09T13:32:34.926 INFO:teuthology.orchestra.run.smithi043.stdout:Created symlink /etc/systemd/system/ceph.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2023-08-09T13:32:34.927 INFO:teuthology.orchestra.run.smithi043.stdout: 2023-08-09T13:32:35.318 INFO:teuthology.orchestra.run.smithi043.stdout: Installing : rbd-mirror-2:18.0.0-5389.g668a86f2.el9.x86_64 134/177 2023-08-09T13:32:35.401 INFO:teuthology.orchestra.run.smithi043.stdout: Running scriptlet: rbd-mirror-2:18.0.0-5389.g668a86f2.el9.x86_64 134/177 2023-08-09T13:32:35.402 INFO:teuthology.orchestra.run.smithi043.stdout:Glob pattern passed to enable, but globs are not supported for this. 2023-08-09T13:32:35.402 INFO:teuthology.orchestra.run.smithi043.stdout:Invalid unit name "ceph-rbd-mirror@*.service" escaped as "ceph-rbd-mirror@\x2a.service". 2023-08-09T13:32:35.402 INFO:teuthology.orchestra.run.smithi043.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2023-08-09T13:32:35.402 INFO:teuthology.orchestra.run.smithi043.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2023-08-09T13:32:35.402 INFO:teuthology.orchestra.run.smithi043.stdout: 2023-08-09T13:32:35.516 INFO:teuthology.orchestra.run.smithi203.stdout: Installing : libtool-2.4.6-45.el9.x86_64 130/177 2023-08-09T13:32:35.567 INFO:teuthology.orchestra.run.smithi203.stdout: Installing : ceph-radosgw-2:18.0.0-5389.g668a86f2.el9.x86_64 131/177 2023-08-09T13:32:35.676 INFO:teuthology.orchestra.run.smithi203.stdout: Running scriptlet: ceph-radosgw-2:18.0.0-5389.g668a86f2.el9.x86_64 131/177 2023-08-09T13:32:35.676 INFO:teuthology.orchestra.run.smithi203.stdout:Glob pattern passed to enable, but globs are not supported for this. 2023-08-09T13:32:35.676 INFO:teuthology.orchestra.run.smithi203.stdout:Invalid unit name "ceph-radosgw@*.service" escaped as "ceph-radosgw@\x2a.service". 2023-08-09T13:32:35.676 INFO:teuthology.orchestra.run.smithi203.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2023-08-09T13:32:35.677 INFO:teuthology.orchestra.run.smithi203.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2023-08-09T13:32:35.677 INFO:teuthology.orchestra.run.smithi203.stdout: 2023-08-09T13:32:35.726 INFO:teuthology.orchestra.run.smithi203.stdout: Installing : ceph-immutable-object-cache-2:18.0.0-5389.g668a8 132/177 2023-08-09T13:32:35.838 INFO:teuthology.orchestra.run.smithi203.stdout: Running scriptlet: ceph-immutable-object-cache-2:18.0.0-5389.g668a8 132/177 2023-08-09T13:32:35.839 INFO:teuthology.orchestra.run.smithi203.stdout:Glob pattern passed to enable, but globs are not supported for this. 2023-08-09T13:32:35.839 INFO:teuthology.orchestra.run.smithi203.stdout:Invalid unit name "ceph-immutable-object-cache@*.service" escaped as "ceph-immutable-object-cache@\x2a.service". 2023-08-09T13:32:35.839 INFO:teuthology.orchestra.run.smithi203.stdout: 2023-08-09T13:32:35.936 INFO:teuthology.orchestra.run.smithi203.stdout: Installing : cephfs-mirror-2:18.0.0-5389.g668a86f2.el9.x86_64 133/177 2023-08-09T13:32:36.053 INFO:teuthology.orchestra.run.smithi203.stdout: Running scriptlet: cephfs-mirror-2:18.0.0-5389.g668a86f2.el9.x86_64 133/177 2023-08-09T13:32:36.054 INFO:teuthology.orchestra.run.smithi203.stdout:Glob pattern passed to enable, but globs are not supported for this. 2023-08-09T13:32:36.054 INFO:teuthology.orchestra.run.smithi203.stdout:Invalid unit name "cephfs-mirror@*.service" escaped as "cephfs-mirror@\x2a.service". 2023-08-09T13:32:36.054 INFO:teuthology.orchestra.run.smithi203.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2023-08-09T13:32:36.054 INFO:teuthology.orchestra.run.smithi203.stdout:Created symlink /etc/systemd/system/ceph.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2023-08-09T13:32:36.054 INFO:teuthology.orchestra.run.smithi203.stdout: 2023-08-09T13:32:36.453 INFO:teuthology.orchestra.run.smithi203.stdout: Installing : rbd-mirror-2:18.0.0-5389.g668a86f2.el9.x86_64 134/177 2023-08-09T13:32:36.554 INFO:teuthology.orchestra.run.smithi203.stdout: Running scriptlet: rbd-mirror-2:18.0.0-5389.g668a86f2.el9.x86_64 134/177 2023-08-09T13:32:36.555 INFO:teuthology.orchestra.run.smithi203.stdout:Glob pattern passed to enable, but globs are not supported for this. 2023-08-09T13:32:36.555 INFO:teuthology.orchestra.run.smithi203.stdout:Invalid unit name "ceph-rbd-mirror@*.service" escaped as "ceph-rbd-mirror@\x2a.service". 2023-08-09T13:32:36.555 INFO:teuthology.orchestra.run.smithi203.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2023-08-09T13:32:36.555 INFO:teuthology.orchestra.run.smithi203.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2023-08-09T13:32:36.555 INFO:teuthology.orchestra.run.smithi203.stdout: 2023-08-09T13:32:39.823 INFO:teuthology.orchestra.run.smithi043.stdout: Installing : ceph-test-2:18.0.0-5389.g668a86f2.el9.x86_64 135/177 2023-08-09T13:32:39.876 INFO:teuthology.orchestra.run.smithi043.stdout: Installing : rbd-fuse-2:18.0.0-5389.g668a86f2.el9.x86_64 136/177 2023-08-09T13:32:39.911 INFO:teuthology.orchestra.run.smithi043.stdout: Installing : rbd-nbd-2:18.0.0-5389.g668a86f2.el9.x86_64 137/177 2023-08-09T13:32:40.195 INFO:teuthology.orchestra.run.smithi043.stdout: Installing : elfutils-libelf-devel-0.189-3.el9.x86_64 138/177 2023-08-09T13:32:40.326 INFO:teuthology.orchestra.run.smithi043.stdout: Installing : postgresql-contrib-13.11-1.el9.x86_64 139/177 2023-08-09T13:32:40.372 INFO:teuthology.orchestra.run.smithi043.stdout: Installing : ceph-fuse-2:18.0.0-5389.g668a86f2.el9.x86_64 140/177 2023-08-09T13:32:40.421 INFO:teuthology.orchestra.run.smithi043.stdout: Installing : xfsprogs-devel-5.19.0-4.el9.x86_64 141/177 2023-08-09T13:32:40.512 INFO:teuthology.orchestra.run.smithi043.stdout: Upgrading : xfsdump-3.1.12-3.el9.x86_64 142/177 2023-08-09T13:32:40.636 INFO:teuthology.orchestra.run.smithi043.stdout: Upgrading : util-linux-user-2.37.4-11.el9.x86_64 143/177 2023-08-09T13:32:40.682 INFO:teuthology.orchestra.run.smithi043.stdout: Upgrading : iproute-tc-6.2.0-5.el9.x86_64 144/177 2023-08-09T13:32:40.735 INFO:teuthology.orchestra.run.smithi043.stdout: Upgrading : elfutils-debuginfod-client-0.189-3.el9.x86_64 145/177 2023-08-09T13:32:40.766 INFO:teuthology.orchestra.run.smithi203.stdout: Installing : ceph-test-2:18.0.0-5389.g668a86f2.el9.x86_64 135/177 2023-08-09T13:32:40.770 INFO:teuthology.orchestra.run.smithi043.stdout: Installing : libcephfs-devel-2:18.0.0-5389.g668a86f2.el9.x86_ 146/177 2023-08-09T13:32:40.809 INFO:teuthology.orchestra.run.smithi203.stdout: Installing : rbd-fuse-2:18.0.0-5389.g668a86f2.el9.x86_64 136/177 2023-08-09T13:32:40.825 INFO:teuthology.orchestra.run.smithi043.stdout: Upgrading : NetworkManager-team-1:1.43.90-1.el9.x86_64 147/177 2023-08-09T13:32:40.847 INFO:teuthology.orchestra.run.smithi203.stdout: Installing : rbd-nbd-2:18.0.0-5389.g668a86f2.el9.x86_64 137/177 2023-08-09T13:32:40.887 INFO:teuthology.orchestra.run.smithi043.stdout: Upgrading : NetworkManager-tui-1:1.43.90-1.el9.x86_64 148/177 2023-08-09T13:32:41.134 INFO:teuthology.orchestra.run.smithi203.stdout: Installing : elfutils-libelf-devel-0.189-3.el9.x86_64 138/177 2023-08-09T13:32:41.259 INFO:teuthology.orchestra.run.smithi203.stdout: Installing : postgresql-contrib-13.11-1.el9.x86_64 139/177 2023-08-09T13:32:41.299 INFO:teuthology.orchestra.run.smithi203.stdout: Installing : ceph-fuse-2:18.0.0-5389.g668a86f2.el9.x86_64 140/177 2023-08-09T13:32:41.361 INFO:teuthology.orchestra.run.smithi203.stdout: Installing : xfsprogs-devel-5.19.0-4.el9.x86_64 141/177 2023-08-09T13:32:41.466 INFO:teuthology.orchestra.run.smithi203.stdout: Upgrading : xfsdump-3.1.12-3.el9.x86_64 142/177 2023-08-09T13:32:41.585 INFO:teuthology.orchestra.run.smithi203.stdout: Upgrading : util-linux-user-2.37.4-11.el9.x86_64 143/177 2023-08-09T13:32:41.635 INFO:teuthology.orchestra.run.smithi203.stdout: Upgrading : iproute-tc-6.2.0-5.el9.x86_64 144/177 2023-08-09T13:32:41.687 INFO:teuthology.orchestra.run.smithi203.stdout: Upgrading : elfutils-debuginfod-client-0.189-3.el9.x86_64 145/177 2023-08-09T13:32:41.715 INFO:teuthology.orchestra.run.smithi203.stdout: Installing : libcephfs-devel-2:18.0.0-5389.g668a86f2.el9.x86_ 146/177 2023-08-09T13:32:41.769 INFO:teuthology.orchestra.run.smithi203.stdout: Upgrading : NetworkManager-team-1:1.43.90-1.el9.x86_64 147/177 2023-08-09T13:32:41.784 INFO:teuthology.orchestra.run.smithi043.stdout: Running scriptlet: postgresql-server-13.11-1.el9.x86_64 149/177 2023-08-09T13:32:41.825 INFO:teuthology.orchestra.run.smithi203.stdout: Upgrading : NetworkManager-tui-1:1.43.90-1.el9.x86_64 148/177 2023-08-09T13:32:41.833 INFO:teuthology.orchestra.run.smithi043.stdout: Installing : postgresql-server-13.11-1.el9.x86_64 149/177 2023-08-09T13:32:41.894 INFO:teuthology.orchestra.run.smithi043.stdout: Running scriptlet: postgresql-server-13.11-1.el9.x86_64 149/177 2023-08-09T13:32:42.850 INFO:teuthology.orchestra.run.smithi203.stdout: Running scriptlet: postgresql-server-13.11-1.el9.x86_64 149/177 2023-08-09T13:32:42.898 INFO:teuthology.orchestra.run.smithi203.stdout: Installing : postgresql-server-13.11-1.el9.x86_64 149/177 2023-08-09T13:32:42.962 INFO:teuthology.orchestra.run.smithi203.stdout: Running scriptlet: postgresql-server-13.11-1.el9.x86_64 149/177 2023-08-09T13:32:43.266 INFO:teuthology.orchestra.run.smithi043.stdout: Installing : cephfs-top-2:18.0.0-5389.g668a86f2.el9.noarch 150/177 2023-08-09T13:32:43.346 INFO:teuthology.orchestra.run.smithi043.stdout: Installing : openssl-devel-1:3.0.7-24.el9.x86_64 151/177 2023-08-09T13:32:43.395 INFO:teuthology.orchestra.run.smithi043.stdout: Installing : libaio-devel-0.3.111-13.el9.x86_64 152/177 2023-08-09T13:32:43.525 INFO:teuthology.orchestra.run.smithi043.stdout: Installing : flex-2.6.4-9.el9.x86_64 153/177 2023-08-09T13:32:43.565 INFO:teuthology.orchestra.run.smithi043.stdout: Installing : bison-3.7.4-5.el9.x86_64 154/177 2023-08-09T13:32:43.604 INFO:teuthology.orchestra.run.smithi043.stdout: Cleanup : NetworkManager-tui-1:1.41.8-1.el9.x86_64 155/177 2023-08-09T13:32:43.633 INFO:teuthology.orchestra.run.smithi043.stdout: Cleanup : elfutils-debuginfod-client-0.188-3.el9.x86_64 156/177 2023-08-09T13:32:43.674 INFO:teuthology.orchestra.run.smithi043.stdout: Cleanup : elfutils-libs-0.188-3.el9.x86_64 157/177 2023-08-09T13:32:43.675 INFO:teuthology.orchestra.run.smithi043.stdout: Cleanup : librbd1-2:16.2.4-5.el9.x86_64 158/177 2023-08-09T13:32:43.820 INFO:teuthology.orchestra.run.smithi043.stdout: Running scriptlet: librbd1-2:16.2.4-5.el9.x86_64 158/177 2023-08-09T13:32:43.870 INFO:teuthology.orchestra.run.smithi043.stdout: Cleanup : openssl-1:3.0.7-2.el9.x86_64 159/177 2023-08-09T13:32:43.871 INFO:teuthology.orchestra.run.smithi043.stdout: Cleanup : librados2-2:16.2.4-5.el9.x86_64 160/177 2023-08-09T13:32:44.008 INFO:teuthology.orchestra.run.smithi043.stdout: Running scriptlet: librados2-2:16.2.4-5.el9.x86_64 160/177 2023-08-09T13:32:44.062 INFO:teuthology.orchestra.run.smithi043.stdout: Cleanup : xfsdump-3.1.10-1.el9.x86_64 161/177 2023-08-09T13:32:44.108 INFO:teuthology.orchestra.run.smithi043.stdout: Cleanup : xfsprogs-5.14.2-1.el9.x86_64 162/177 2023-08-09T13:32:44.164 INFO:teuthology.orchestra.run.smithi043.stdout: Cleanup : iproute-tc-6.0.0-2.el9.x86_64 163/177 2023-08-09T13:32:44.206 INFO:teuthology.orchestra.run.smithi043.stdout: Cleanup : iproute-6.0.0-2.el9.x86_64 164/177 2023-08-09T13:32:44.248 INFO:teuthology.orchestra.run.smithi203.stdout: Installing : cephfs-top-2:18.0.0-5389.g668a86f2.el9.noarch 150/177 2023-08-09T13:32:44.303 INFO:teuthology.orchestra.run.smithi043.stdout: Cleanup : util-linux-user-2.37.4-9.el9.x86_64 165/177 2023-08-09T13:32:44.329 INFO:teuthology.orchestra.run.smithi203.stdout: Installing : openssl-devel-1:3.0.7-24.el9.x86_64 151/177 2023-08-09T13:32:44.379 INFO:teuthology.orchestra.run.smithi043.stdout: Cleanup : util-linux-2.37.4-9.el9.x86_64 166/177 2023-08-09T13:32:44.408 INFO:teuthology.orchestra.run.smithi203.stdout: Installing : libaio-devel-0.3.111-13.el9.x86_64 152/177 2023-08-09T13:32:44.436 INFO:teuthology.orchestra.run.smithi043.stdout: Cleanup : util-linux-core-2.37.4-9.el9.x86_64 167/177 2023-08-09T13:32:44.472 INFO:teuthology.orchestra.run.smithi043.stdout: Cleanup : libfdisk-2.37.4-9.el9.x86_64 168/177 2023-08-09T13:32:44.508 INFO:teuthology.orchestra.run.smithi043.stdout: Cleanup : libmount-2.37.4-9.el9.x86_64 169/177 2023-08-09T13:32:44.542 INFO:teuthology.orchestra.run.smithi043.stdout: Cleanup : libblkid-2.37.4-9.el9.x86_64 170/177 2023-08-09T13:32:44.542 INFO:teuthology.orchestra.run.smithi203.stdout: Installing : flex-2.6.4-9.el9.x86_64 153/177 2023-08-09T13:32:44.575 INFO:teuthology.orchestra.run.smithi043.stdout: Cleanup : NetworkManager-team-1:1.41.8-1.el9.x86_64 171/177 2023-08-09T13:32:44.595 INFO:teuthology.orchestra.run.smithi203.stdout: Installing : bison-3.7.4-5.el9.x86_64 154/177 2023-08-09T13:32:44.615 INFO:teuthology.orchestra.run.smithi043.stdout: Running scriptlet: NetworkManager-1:1.41.8-1.el9.x86_64 172/177 2023-08-09T13:32:44.615 INFO:teuthology.orchestra.run.smithi043.stdout: Cleanup : NetworkManager-1:1.41.8-1.el9.x86_64 172/177 2023-08-09T13:32:44.631 INFO:teuthology.orchestra.run.smithi203.stdout: Cleanup : NetworkManager-tui-1:1.41.8-1.el9.x86_64 155/177 2023-08-09T13:32:44.663 INFO:teuthology.orchestra.run.smithi203.stdout: Cleanup : elfutils-debuginfod-client-0.188-3.el9.x86_64 156/177 2023-08-09T13:32:44.705 INFO:teuthology.orchestra.run.smithi203.stdout: Cleanup : elfutils-libs-0.188-3.el9.x86_64 157/177 2023-08-09T13:32:44.706 INFO:teuthology.orchestra.run.smithi203.stdout: Cleanup : librbd1-2:16.2.4-5.el9.x86_64 158/177 2023-08-09T13:32:44.851 INFO:teuthology.orchestra.run.smithi203.stdout: Running scriptlet: librbd1-2:16.2.4-5.el9.x86_64 158/177 2023-08-09T13:32:44.901 INFO:teuthology.orchestra.run.smithi203.stdout: Cleanup : openssl-1:3.0.7-2.el9.x86_64 159/177 2023-08-09T13:32:44.902 INFO:teuthology.orchestra.run.smithi203.stdout: Cleanup : librados2-2:16.2.4-5.el9.x86_64 160/177 2023-08-09T13:32:44.967 INFO:teuthology.orchestra.run.smithi043.stdout: Running scriptlet: NetworkManager-1:1.41.8-1.el9.x86_64 172/177 2023-08-09T13:32:44.967 INFO:teuthology.orchestra.run.smithi043.stdout: Cleanup : NetworkManager-libnm-1:1.41.8-1.el9.x86_64 173/177 2023-08-09T13:32:45.056 INFO:teuthology.orchestra.run.smithi203.stdout: Running scriptlet: librados2-2:16.2.4-5.el9.x86_64 160/177 2023-08-09T13:32:45.101 INFO:teuthology.orchestra.run.smithi203.stdout: Cleanup : xfsdump-3.1.10-1.el9.x86_64 161/177 2023-08-09T13:32:45.105 INFO:teuthology.orchestra.run.smithi043.stdout: Running scriptlet: NetworkManager-libnm-1:1.41.8-1.el9.x86_64 173/177 2023-08-09T13:32:45.136 INFO:teuthology.orchestra.run.smithi043.stdout: Cleanup : libuuid-2.37.4-9.el9.x86_64 174/177 2023-08-09T13:32:45.148 INFO:teuthology.orchestra.run.smithi203.stdout: Cleanup : xfsprogs-5.14.2-1.el9.x86_64 162/177 2023-08-09T13:32:45.170 INFO:teuthology.orchestra.run.smithi043.stdout: Cleanup : libsmartcols-2.37.4-9.el9.x86_64 175/177 2023-08-09T13:32:45.195 INFO:teuthology.orchestra.run.smithi203.stdout: Cleanup : iproute-tc-6.0.0-2.el9.x86_64 163/177 2023-08-09T13:32:45.206 INFO:teuthology.orchestra.run.smithi043.stdout: Cleanup : elfutils-libelf-0.188-3.el9.x86_64 176/177 2023-08-09T13:32:45.250 INFO:teuthology.orchestra.run.smithi203.stdout: Cleanup : iproute-6.0.0-2.el9.x86_64 164/177 2023-08-09T13:32:45.323 INFO:teuthology.orchestra.run.smithi203.stdout: Cleanup : util-linux-user-2.37.4-9.el9.x86_64 165/177 2023-08-09T13:32:45.334 INFO:teuthology.orchestra.run.smithi043.stdout: Cleanup : openssl-libs-1:3.0.7-2.el9.x86_64 177/177 2023-08-09T13:32:45.421 INFO:teuthology.orchestra.run.smithi203.stdout: Cleanup : util-linux-2.37.4-9.el9.x86_64 166/177 2023-08-09T13:32:45.480 INFO:teuthology.orchestra.run.smithi203.stdout: Cleanup : util-linux-core-2.37.4-9.el9.x86_64 167/177 2023-08-09T13:32:45.512 INFO:teuthology.orchestra.run.smithi203.stdout: Cleanup : libfdisk-2.37.4-9.el9.x86_64 168/177 2023-08-09T13:32:45.546 INFO:teuthology.orchestra.run.smithi203.stdout: Cleanup : libmount-2.37.4-9.el9.x86_64 169/177 2023-08-09T13:32:45.580 INFO:teuthology.orchestra.run.smithi203.stdout: Cleanup : libblkid-2.37.4-9.el9.x86_64 170/177 2023-08-09T13:32:45.613 INFO:teuthology.orchestra.run.smithi203.stdout: Cleanup : NetworkManager-team-1:1.41.8-1.el9.x86_64 171/177 2023-08-09T13:32:45.652 INFO:teuthology.orchestra.run.smithi203.stdout: Running scriptlet: NetworkManager-1:1.41.8-1.el9.x86_64 172/177 2023-08-09T13:32:45.652 INFO:teuthology.orchestra.run.smithi203.stdout: Cleanup : NetworkManager-1:1.41.8-1.el9.x86_64 172/177 2023-08-09T13:32:45.800 INFO:teuthology.orchestra.run.smithi043.stdout: Running scriptlet: openssl-libs-1:3.0.7-2.el9.x86_64 177/177 2023-08-09T13:32:45.800 INFO:teuthology.orchestra.run.smithi043.stdout:Couldn't write '0' to 'kernel/yama/ptrace_scope', ignoring: No such file or directory 2023-08-09T13:32:45.801 INFO:teuthology.orchestra.run.smithi043.stdout: 2023-08-09T13:32:45.997 INFO:teuthology.orchestra.run.smithi203.stdout: Running scriptlet: NetworkManager-1:1.41.8-1.el9.x86_64 172/177 2023-08-09T13:32:45.997 INFO:teuthology.orchestra.run.smithi203.stdout: Cleanup : NetworkManager-libnm-1:1.41.8-1.el9.x86_64 173/177 2023-08-09T13:32:46.135 INFO:teuthology.orchestra.run.smithi203.stdout: Running scriptlet: NetworkManager-libnm-1:1.41.8-1.el9.x86_64 173/177 2023-08-09T13:32:46.166 INFO:teuthology.orchestra.run.smithi203.stdout: Cleanup : libuuid-2.37.4-9.el9.x86_64 174/177 2023-08-09T13:32:46.200 INFO:teuthology.orchestra.run.smithi203.stdout: Cleanup : libsmartcols-2.37.4-9.el9.x86_64 175/177 2023-08-09T13:32:46.236 INFO:teuthology.orchestra.run.smithi203.stdout: Cleanup : elfutils-libelf-0.188-3.el9.x86_64 176/177 2023-08-09T13:32:46.360 INFO:teuthology.orchestra.run.smithi203.stdout: Cleanup : openssl-libs-1:3.0.7-2.el9.x86_64 177/177 2023-08-09T13:32:46.825 INFO:teuthology.orchestra.run.smithi203.stdout: Running scriptlet: openssl-libs-1:3.0.7-2.el9.x86_64 177/177 2023-08-09T13:32:46.826 INFO:teuthology.orchestra.run.smithi203.stdout:Couldn't write '0' to 'kernel/yama/ptrace_scope', ignoring: No such file or directory 2023-08-09T13:32:46.826 INFO:teuthology.orchestra.run.smithi203.stdout: 2023-08-09T13:32:50.310 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : ceph-2:18.0.0-5389.g668a86f2.el9.x86_64 1/177 2023-08-09T13:32:50.310 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : ceph-base-2:18.0.0-5389.g668a86f2.el9.x86_64 2/177 2023-08-09T13:32:50.310 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : ceph-common-2:18.0.0-5389.g668a86f2.el9.x86_64 3/177 2023-08-09T13:32:50.310 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : ceph-fuse-2:18.0.0-5389.g668a86f2.el9.x86_64 4/177 2023-08-09T13:32:50.310 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : ceph-immutable-object-cache-2:18.0.0-5389.g668a8 5/177 2023-08-09T13:32:50.311 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : ceph-mds-2:18.0.0-5389.g668a86f2.el9.x86_64 6/177 2023-08-09T13:32:50.311 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : ceph-mgr-2:18.0.0-5389.g668a86f2.el9.x86_64 7/177 2023-08-09T13:32:50.311 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : ceph-mon-2:18.0.0-5389.g668a86f2.el9.x86_64 8/177 2023-08-09T13:32:50.311 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : ceph-osd-2:18.0.0-5389.g668a86f2.el9.x86_64 9/177 2023-08-09T13:32:50.311 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : ceph-radosgw-2:18.0.0-5389.g668a86f2.el9.x86_64 10/177 2023-08-09T13:32:50.311 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : ceph-selinux-2:18.0.0-5389.g668a86f2.el9.x86_64 11/177 2023-08-09T13:32:50.311 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : ceph-test-2:18.0.0-5389.g668a86f2.el9.x86_64 12/177 2023-08-09T13:32:50.311 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : cephfs-mirror-2:18.0.0-5389.g668a86f2.el9.x86_64 13/177 2023-08-09T13:32:50.312 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : libcephfs-devel-2:18.0.0-5389.g668a86f2.el9.x86_ 14/177 2023-08-09T13:32:50.312 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : libcephfs2-2:18.0.0-5389.g668a86f2.el9.x86_64 15/177 2023-08-09T13:32:50.312 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : libcephsqlite-2:18.0.0-5389.g668a86f2.el9.x86_64 16/177 2023-08-09T13:32:50.312 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : librados-devel-2:18.0.0-5389.g668a86f2.el9.x86_6 17/177 2023-08-09T13:32:50.312 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : libradosstriper1-2:18.0.0-5389.g668a86f2.el9.x86 18/177 2023-08-09T13:32:50.312 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : librgw2-2:18.0.0-5389.g668a86f2.el9.x86_64 19/177 2023-08-09T13:32:50.312 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : python3-ceph-argparse-2:18.0.0-5389.g668a86f2.el 20/177 2023-08-09T13:32:50.313 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : python3-ceph-common-2:18.0.0-5389.g668a86f2.el9. 21/177 2023-08-09T13:32:50.313 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : python3-cephfs-2:18.0.0-5389.g668a86f2.el9.x86_6 22/177 2023-08-09T13:32:50.313 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : python3-rados-2:18.0.0-5389.g668a86f2.el9.x86_64 23/177 2023-08-09T13:32:50.313 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : python3-rbd-2:18.0.0-5389.g668a86f2.el9.x86_64 24/177 2023-08-09T13:32:50.313 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : python3-rgw-2:18.0.0-5389.g668a86f2.el9.x86_64 25/177 2023-08-09T13:32:50.313 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : rbd-fuse-2:18.0.0-5389.g668a86f2.el9.x86_64 26/177 2023-08-09T13:32:50.313 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : rbd-mirror-2:18.0.0-5389.g668a86f2.el9.x86_64 27/177 2023-08-09T13:32:50.322 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : rbd-nbd-2:18.0.0-5389.g668a86f2.el9.x86_64 28/177 2023-08-09T13:32:50.322 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : ceph-grafana-dashboards-2:18.0.0-5389.g668a86f2. 29/177 2023-08-09T13:32:50.323 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : ceph-mgr-cephadm-2:18.0.0-5389.g668a86f2.el9.noa 30/177 2023-08-09T13:32:50.323 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : ceph-mgr-dashboard-2:18.0.0-5389.g668a86f2.el9.n 31/177 2023-08-09T13:32:50.323 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : ceph-mgr-diskprediction-local-2:18.0.0-5389.g668 32/177 2023-08-09T13:32:50.323 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : ceph-mgr-modules-core-2:18.0.0-5389.g668a86f2.el 33/177 2023-08-09T13:32:50.323 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : ceph-mgr-rook-2:18.0.0-5389.g668a86f2.el9.noarch 34/177 2023-08-09T13:32:50.323 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : ceph-prometheus-alerts-2:18.0.0-5389.g668a86f2.e 35/177 2023-08-09T13:32:50.323 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : ceph-volume-2:18.0.0-5389.g668a86f2.el9.noarch 36/177 2023-08-09T13:32:50.324 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : cephadm-2:18.0.0-5389.g668a86f2.el9.noarch 37/177 2023-08-09T13:32:50.324 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : cephfs-top-2:18.0.0-5389.g668a86f2.el9.noarch 38/177 2023-08-09T13:32:50.324 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : automake-1.16.2-8.el9.noarch 39/177 2023-08-09T13:32:50.324 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : bison-3.7.4-5.el9.x86_64 40/177 2023-08-09T13:32:50.324 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : boost-program-options-1.75.0-8.el9.x86_64 41/177 2023-08-09T13:32:50.324 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : elfutils-libelf-devel-0.189-3.el9.x86_64 42/177 2023-08-09T13:32:50.324 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : flex-2.6.4-9.el9.x86_64 43/177 2023-08-09T13:32:50.324 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : flexiblas-3.0.4-8.el9.x86_64 44/177 2023-08-09T13:32:50.325 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : flexiblas-netlib-3.0.4-8.el9.x86_64 45/177 2023-08-09T13:32:50.325 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 46/177 2023-08-09T13:32:50.325 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : jq-1.6-15.el9.x86_64 47/177 2023-08-09T13:32:50.325 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : libaio-devel-0.3.111-13.el9.x86_64 48/177 2023-08-09T13:32:50.325 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : libpmemobj-1.12.1-1.el9.x86_64 49/177 2023-08-09T13:32:50.325 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : librabbitmq-0.11.0-7.el9.x86_64 50/177 2023-08-09T13:32:50.325 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : librdkafka-1.6.1-102.el9.x86_64 51/177 2023-08-09T13:32:50.325 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : libtool-2.4.6-45.el9.x86_64 52/177 2023-08-09T13:32:50.326 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : libuuid-devel-2.37.4-11.el9.x86_64 53/177 2023-08-09T13:32:50.326 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : libzstd-devel-1.5.1-2.el9.x86_64 54/177 2023-08-09T13:32:50.326 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : lttng-ust-2.12.0-6.el9.x86_64 55/177 2023-08-09T13:32:50.326 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : oniguruma-6.9.6-1.el9.5.x86_64 56/177 2023-08-09T13:32:50.326 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : openblas-0.3.21-2.el9.x86_64 57/177 2023-08-09T13:32:50.326 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : openblas-openmp-0.3.21-2.el9.x86_64 58/177 2023-08-09T13:32:50.326 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : openssl-devel-1:3.0.7-24.el9.x86_64 59/177 2023-08-09T13:32:50.327 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : postgresql-13.11-1.el9.x86_64 60/177 2023-08-09T13:32:50.327 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : postgresql-contrib-13.11-1.el9.x86_64 61/177 2023-08-09T13:32:50.327 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : postgresql-private-libs-13.11-1.el9.x86_64 62/177 2023-08-09T13:32:50.327 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : postgresql-server-13.11-1.el9.x86_64 63/177 2023-08-09T13:32:50.327 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : python3-babel-2.9.1-2.el9.noarch 64/177 2023-08-09T13:32:50.327 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : python3-jinja2-2.11.3-4.el9.noarch 65/177 2023-08-09T13:32:50.327 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : python3-jsonpatch-1.21-16.el9.noarch 66/177 2023-08-09T13:32:50.328 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : python3-jsonpointer-2.0-4.el9.noarch 67/177 2023-08-09T13:32:50.328 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : python3-mako-1.1.4-6.el9.noarch 68/177 2023-08-09T13:32:50.328 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : python3-markupsafe-1.1.1-12.el9.x86_64 69/177 2023-08-09T13:32:50.328 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : python3-numpy-1:1.20.1-5.el9.x86_64 70/177 2023-08-09T13:32:50.328 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 71/177 2023-08-09T13:32:50.328 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : python3-oauthlib-3.1.1-5.el9.noarch 72/177 2023-08-09T13:32:50.328 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : python3-prettytable-0.7.2-27.el9.noarch 73/177 2023-08-09T13:32:50.328 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : python3-pyasn1-0.4.8-6.el9.noarch 74/177 2023-08-09T13:32:50.329 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : python3-pyasn1-modules-0.4.8-6.el9.noarch 75/177 2023-08-09T13:32:50.329 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : python3-pytz-2021.1-5.el9.noarch 76/177 2023-08-09T13:32:50.329 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : python3-requests-oauthlib-1.3.0-12.el9.noarch 77/177 2023-08-09T13:32:50.329 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : python3-scipy-1.6.2-8.el9.x86_64 78/177 2023-08-09T13:32:50.329 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : python3-toml-0.10.2-6.el9.noarch 79/177 2023-08-09T13:32:50.329 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : socat-1.7.4.1-5.el9.x86_64 80/177 2023-08-09T13:32:50.329 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : uuid-1.6.2-55.el9.x86_64 81/177 2023-08-09T13:32:50.329 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : xmlstarlet-1.6.1-20.el9.x86_64 82/177 2023-08-09T13:32:50.330 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : fuse-2.9.9-15.el9.x86_64 83/177 2023-08-09T13:32:50.330 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : libgfortran-11.4.1-2.1.el9.x86_64 84/177 2023-08-09T13:32:50.330 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : libquadmath-11.4.1-2.1.el9.x86_64 85/177 2023-08-09T13:32:50.330 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : python3-cffi-1.14.5-5.el9.x86_64 86/177 2023-08-09T13:32:50.330 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : python3-cryptography-36.0.1-4.el9.x86_64 87/177 2023-08-09T13:32:50.330 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : python3-ply-3.11-14.el9.noarch 88/177 2023-08-09T13:32:50.330 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : python3-pycparser-2.20-6.el9.noarch 89/177 2023-08-09T13:32:50.330 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : python3-devel-3.9.16-1.el9.x86_64 90/177 2023-08-09T13:32:50.331 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : xfsprogs-devel-5.19.0-4.el9.x86_64 91/177 2023-08-09T13:32:50.331 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : zlib-devel-1.2.11-36.el9.x86_64 92/177 2023-08-09T13:32:50.331 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : fmt-8.1.1-5.el9.x86_64 93/177 2023-08-09T13:32:50.331 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : gperftools-libs-2.9.1-2.el9.x86_64 94/177 2023-08-09T13:32:50.331 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : libarrow-9.0.0-9.el9.x86_64 95/177 2023-08-09T13:32:50.332 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : libarrow-doc-9.0.0-9.el9.noarch 96/177 2023-08-09T13:32:50.332 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : liboath-2.6.7-2.el9.x86_64 97/177 2023-08-09T13:32:50.332 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : libunwind-1.6.2-1.el9.x86_64 98/177 2023-08-09T13:32:50.333 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : parquet-libs-9.0.0-9.el9.x86_64 99/177 2023-08-09T13:32:50.333 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : python3-asyncssh-2.13.2-3.el9.noarch 100/177 2023-08-09T13:32:50.333 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : python3-bcrypt-3.2.2-1.el9.x86_64 101/177 2023-08-09T13:32:50.333 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : python3-cachetools-4.2.4-1.el9.noarch 102/177 2023-08-09T13:32:50.333 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : python3-certifi-2021.10.8-2.el9.noarch 103/177 2023-08-09T13:32:50.334 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : python3-cheroot-8.6.0-4.el9.noarch 104/177 2023-08-09T13:32:50.334 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : python3-cherrypy-18.6.1-2.el9.noarch 105/177 2023-08-09T13:32:50.334 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : python3-google-auth-1:2.22.0-1.el9.noarch 106/177 2023-08-09T13:32:50.334 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : python3-jaraco-8.2.1-3.el9.noarch 107/177 2023-08-09T13:32:50.335 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : python3-jaraco-classes-3.2.1-5.el9.noarch 108/177 2023-08-09T13:32:50.335 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : python3-jaraco-collections-3.0.0-8.el9.noarch 109/177 2023-08-09T13:32:50.335 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : python3-jaraco-functools-3.5.0-2.el9.noarch 110/177 2023-08-09T13:32:50.335 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : python3-jaraco-text-3.2.0-6.el9.noarch 111/177 2023-08-09T13:32:50.335 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : python3-jwt+crypto-2.4.0-1.el9.noarch 112/177 2023-08-09T13:32:50.335 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : python3-jwt-2.4.0-1.el9.noarch 113/177 2023-08-09T13:32:50.335 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : python3-kubernetes-1:26.1.0-2.el9.noarch 114/177 2023-08-09T13:32:50.336 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : python3-logutils-0.3.5-21.el9.noarch 115/177 2023-08-09T13:32:50.336 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : python3-more-itertools-8.12.0-2.el9.noarch 116/177 2023-08-09T13:32:50.336 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : python3-natsort-7.1.1-5.el9.noarch 117/177 2023-08-09T13:32:50.336 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : python3-pecan-1.4.2-3.el9.noarch 118/177 2023-08-09T13:32:50.336 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : python3-portend-3.1.0-2.el9.noarch 119/177 2023-08-09T13:32:50.336 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : python3-pyOpenSSL-21.0.0-1.el9.noarch 120/177 2023-08-09T13:32:50.336 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : python3-repoze-lru-0.7-16.el9.noarch 121/177 2023-08-09T13:32:50.337 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : python3-routes-2.5.1-5.el9.noarch 122/177 2023-08-09T13:32:50.337 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : python3-rsa-4.9-2.el9.noarch 123/177 2023-08-09T13:32:50.337 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : python3-tempora-5.0.0-2.el9.noarch 124/177 2023-08-09T13:32:50.337 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : python3-typing-extensions-4.4.0-2.el9.noarch 125/177 2023-08-09T13:32:50.337 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : python3-webob-1.8.7-6.el9.noarch 126/177 2023-08-09T13:32:50.337 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : python3-websocket-client-1.2.3-2.el9.noarch 127/177 2023-08-09T13:32:50.337 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : python3-werkzeug-2.0.3-3.el9.1.noarch 128/177 2023-08-09T13:32:50.337 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : python3-zc-lockfile-2.0-10.el9.noarch 129/177 2023-08-09T13:32:50.338 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : re2-1:20211101-2.el9.x86_64 130/177 2023-08-09T13:32:50.338 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : thrift-0.15.0-2.el9.x86_64 131/177 2023-08-09T13:32:50.338 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : librados2-2:18.0.0-5389.g668a86f2.el9.x86_64 132/177 2023-08-09T13:32:50.338 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : librados2-2:16.2.4-5.el9.x86_64 133/177 2023-08-09T13:32:50.338 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : librbd1-2:18.0.0-5389.g668a86f2.el9.x86_64 134/177 2023-08-09T13:32:50.338 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : librbd1-2:16.2.4-5.el9.x86_64 135/177 2023-08-09T13:32:50.338 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : NetworkManager-1:1.43.90-1.el9.x86_64 136/177 2023-08-09T13:32:50.339 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : NetworkManager-1:1.41.8-1.el9.x86_64 137/177 2023-08-09T13:32:50.339 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : NetworkManager-libnm-1:1.43.90-1.el9.x86_64 138/177 2023-08-09T13:32:50.339 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : NetworkManager-libnm-1:1.41.8-1.el9.x86_64 139/177 2023-08-09T13:32:50.339 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : NetworkManager-team-1:1.43.90-1.el9.x86_64 140/177 2023-08-09T13:32:50.339 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : NetworkManager-team-1:1.41.8-1.el9.x86_64 141/177 2023-08-09T13:32:50.339 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : NetworkManager-tui-1:1.43.90-1.el9.x86_64 142/177 2023-08-09T13:32:50.339 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : NetworkManager-tui-1:1.41.8-1.el9.x86_64 143/177 2023-08-09T13:32:50.339 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : elfutils-debuginfod-client-0.189-3.el9.x86_64 144/177 2023-08-09T13:32:50.340 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : elfutils-debuginfod-client-0.188-3.el9.x86_64 145/177 2023-08-09T13:32:50.340 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : elfutils-libelf-0.189-3.el9.x86_64 146/177 2023-08-09T13:32:50.340 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : elfutils-libelf-0.188-3.el9.x86_64 147/177 2023-08-09T13:32:50.340 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : elfutils-libs-0.189-3.el9.x86_64 148/177 2023-08-09T13:32:50.340 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : elfutils-libs-0.188-3.el9.x86_64 149/177 2023-08-09T13:32:50.341 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : iproute-6.2.0-5.el9.x86_64 150/177 2023-08-09T13:32:50.341 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : iproute-6.0.0-2.el9.x86_64 151/177 2023-08-09T13:32:50.341 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : iproute-tc-6.2.0-5.el9.x86_64 152/177 2023-08-09T13:32:50.341 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : iproute-tc-6.0.0-2.el9.x86_64 153/177 2023-08-09T13:32:50.341 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : libblkid-2.37.4-11.el9.x86_64 154/177 2023-08-09T13:32:50.341 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : libblkid-2.37.4-9.el9.x86_64 155/177 2023-08-09T13:32:50.342 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : libfdisk-2.37.4-11.el9.x86_64 156/177 2023-08-09T13:32:50.342 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : libfdisk-2.37.4-9.el9.x86_64 157/177 2023-08-09T13:32:50.342 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : libmount-2.37.4-11.el9.x86_64 158/177 2023-08-09T13:32:50.342 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : libmount-2.37.4-9.el9.x86_64 159/177 2023-08-09T13:32:50.342 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : libsmartcols-2.37.4-11.el9.x86_64 160/177 2023-08-09T13:32:50.342 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : libsmartcols-2.37.4-9.el9.x86_64 161/177 2023-08-09T13:32:50.342 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : libuuid-2.37.4-11.el9.x86_64 162/177 2023-08-09T13:32:50.342 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : libuuid-2.37.4-9.el9.x86_64 163/177 2023-08-09T13:32:50.343 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : openssl-1:3.0.7-24.el9.x86_64 164/177 2023-08-09T13:32:50.343 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : openssl-1:3.0.7-2.el9.x86_64 165/177 2023-08-09T13:32:50.343 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : openssl-libs-1:3.0.7-24.el9.x86_64 166/177 2023-08-09T13:32:50.343 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : openssl-libs-1:3.0.7-2.el9.x86_64 167/177 2023-08-09T13:32:50.343 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : util-linux-2.37.4-11.el9.x86_64 168/177 2023-08-09T13:32:50.343 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : util-linux-2.37.4-9.el9.x86_64 169/177 2023-08-09T13:32:50.343 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : util-linux-core-2.37.4-11.el9.x86_64 170/177 2023-08-09T13:32:50.343 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : util-linux-core-2.37.4-9.el9.x86_64 171/177 2023-08-09T13:32:50.344 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : util-linux-user-2.37.4-11.el9.x86_64 172/177 2023-08-09T13:32:50.344 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : util-linux-user-2.37.4-9.el9.x86_64 173/177 2023-08-09T13:32:50.344 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : xfsdump-3.1.12-3.el9.x86_64 174/177 2023-08-09T13:32:50.344 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : xfsdump-3.1.10-1.el9.x86_64 175/177 2023-08-09T13:32:50.344 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : xfsprogs-5.19.0-4.el9.x86_64 176/177 2023-08-09T13:32:51.472 INFO:teuthology.orchestra.run.smithi203.stdout: Verifying : ceph-2:18.0.0-5389.g668a86f2.el9.x86_64 1/177 2023-08-09T13:32:51.473 INFO:teuthology.orchestra.run.smithi203.stdout: Verifying : ceph-base-2:18.0.0-5389.g668a86f2.el9.x86_64 2/177 2023-08-09T13:32:51.473 INFO:teuthology.orchestra.run.smithi203.stdout: Verifying : ceph-common-2:18.0.0-5389.g668a86f2.el9.x86_64 3/177 2023-08-09T13:32:51.473 INFO:teuthology.orchestra.run.smithi203.stdout: Verifying : ceph-fuse-2:18.0.0-5389.g668a86f2.el9.x86_64 4/177 2023-08-09T13:32:51.473 INFO:teuthology.orchestra.run.smithi203.stdout: Verifying : ceph-immutable-object-cache-2:18.0.0-5389.g668a8 5/177 2023-08-09T13:32:51.473 INFO:teuthology.orchestra.run.smithi203.stdout: Verifying : ceph-mds-2:18.0.0-5389.g668a86f2.el9.x86_64 6/177 2023-08-09T13:32:51.474 INFO:teuthology.orchestra.run.smithi203.stdout: Verifying : ceph-mgr-2:18.0.0-5389.g668a86f2.el9.x86_64 7/177 2023-08-09T13:32:51.474 INFO:teuthology.orchestra.run.smithi203.stdout: Verifying : ceph-mon-2:18.0.0-5389.g668a86f2.el9.x86_64 8/177 2023-08-09T13:32:51.474 INFO:teuthology.orchestra.run.smithi203.stdout: Verifying : ceph-osd-2:18.0.0-5389.g668a86f2.el9.x86_64 9/177 2023-08-09T13:32:51.474 INFO:teuthology.orchestra.run.smithi203.stdout: Verifying : ceph-radosgw-2:18.0.0-5389.g668a86f2.el9.x86_64 10/177 2023-08-09T13:32:51.474 INFO:teuthology.orchestra.run.smithi203.stdout: Verifying : ceph-selinux-2:18.0.0-5389.g668a86f2.el9.x86_64 11/177 2023-08-09T13:32:51.474 INFO:teuthology.orchestra.run.smithi203.stdout: Verifying : ceph-test-2:18.0.0-5389.g668a86f2.el9.x86_64 12/177 2023-08-09T13:32:51.474 INFO:teuthology.orchestra.run.smithi203.stdout: Verifying : cephfs-mirror-2:18.0.0-5389.g668a86f2.el9.x86_64 13/177 2023-08-09T13:32:51.475 INFO:teuthology.orchestra.run.smithi203.stdout: Verifying : libcephfs-devel-2:18.0.0-5389.g668a86f2.el9.x86_ 14/177 2023-08-09T13:32:51.475 INFO:teuthology.orchestra.run.smithi203.stdout: Verifying : libcephfs2-2:18.0.0-5389.g668a86f2.el9.x86_64 15/177 2023-08-09T13:32:51.475 INFO:teuthology.orchestra.run.smithi203.stdout: Verifying : libcephsqlite-2:18.0.0-5389.g668a86f2.el9.x86_64 16/177 2023-08-09T13:32:51.475 INFO:teuthology.orchestra.run.smithi203.stdout: Verifying : librados-devel-2:18.0.0-5389.g668a86f2.el9.x86_6 17/177 2023-08-09T13:32:51.475 INFO:teuthology.orchestra.run.smithi203.stdout: Verifying : libradosstriper1-2:18.0.0-5389.g668a86f2.el9.x86 18/177 2023-08-09T13:32:51.475 INFO:teuthology.orchestra.run.smithi203.stdout: Verifying : librgw2-2:18.0.0-5389.g668a86f2.el9.x86_64 19/177 2023-08-09T13:32:51.475 INFO:teuthology.orchestra.run.smithi203.stdout: Verifying : python3-ceph-argparse-2:18.0.0-5389.g668a86f2.el 20/177 2023-08-09T13:32:51.476 INFO:teuthology.orchestra.run.smithi203.stdout: Verifying : python3-ceph-common-2:18.0.0-5389.g668a86f2.el9. 21/177 2023-08-09T13:32:51.476 INFO:teuthology.orchestra.run.smithi203.stdout: Verifying : python3-cephfs-2:18.0.0-5389.g668a86f2.el9.x86_6 22/177 2023-08-09T13:32:51.476 INFO:teuthology.orchestra.run.smithi203.stdout: Verifying : python3-rados-2:18.0.0-5389.g668a86f2.el9.x86_64 23/177 2023-08-09T13:32:51.476 INFO:teuthology.orchestra.run.smithi203.stdout: Verifying : python3-rbd-2:18.0.0-5389.g668a86f2.el9.x86_64 24/177 2023-08-09T13:32:51.476 INFO:teuthology.orchestra.run.smithi203.stdout: Verifying : python3-rgw-2:18.0.0-5389.g668a86f2.el9.x86_64 25/177 2023-08-09T13:32:51.476 INFO:teuthology.orchestra.run.smithi203.stdout: Verifying : rbd-fuse-2:18.0.0-5389.g668a86f2.el9.x86_64 26/177 2023-08-09T13:32:51.476 INFO:teuthology.orchestra.run.smithi203.stdout: Verifying : rbd-mirror-2:18.0.0-5389.g668a86f2.el9.x86_64 27/177 2023-08-09T13:32:51.477 INFO:teuthology.orchestra.run.smithi203.stdout: Verifying : rbd-nbd-2:18.0.0-5389.g668a86f2.el9.x86_64 28/177 2023-08-09T13:32:51.477 INFO:teuthology.orchestra.run.smithi203.stdout: Verifying : ceph-grafana-dashboards-2:18.0.0-5389.g668a86f2. 29/177 2023-08-09T13:32:51.477 INFO:teuthology.orchestra.run.smithi203.stdout: Verifying : ceph-mgr-cephadm-2:18.0.0-5389.g668a86f2.el9.noa 30/177 2023-08-09T13:32:51.477 INFO:teuthology.orchestra.run.smithi203.stdout: Verifying : ceph-mgr-dashboard-2:18.0.0-5389.g668a86f2.el9.n 31/177 2023-08-09T13:32:51.486 INFO:teuthology.orchestra.run.smithi203.stdout: Verifying : ceph-mgr-diskprediction-local-2:18.0.0-5389.g668 32/177 2023-08-09T13:32:51.486 INFO:teuthology.orchestra.run.smithi203.stdout: Verifying : ceph-mgr-modules-core-2:18.0.0-5389.g668a86f2.el 33/177 2023-08-09T13:32:51.487 INFO:teuthology.orchestra.run.smithi203.stdout: Verifying : ceph-mgr-rook-2:18.0.0-5389.g668a86f2.el9.noarch 34/177 2023-08-09T13:32:51.487 INFO:teuthology.orchestra.run.smithi203.stdout: Verifying : ceph-prometheus-alerts-2:18.0.0-5389.g668a86f2.e 35/177 2023-08-09T13:32:51.487 INFO:teuthology.orchestra.run.smithi203.stdout: Verifying : ceph-volume-2:18.0.0-5389.g668a86f2.el9.noarch 36/177 2023-08-09T13:32:51.487 INFO:teuthology.orchestra.run.smithi203.stdout: Verifying : cephadm-2:18.0.0-5389.g668a86f2.el9.noarch 37/177 2023-08-09T13:32:51.487 INFO:teuthology.orchestra.run.smithi203.stdout: Verifying : cephfs-top-2:18.0.0-5389.g668a86f2.el9.noarch 38/177 2023-08-09T13:32:51.487 INFO:teuthology.orchestra.run.smithi203.stdout: Verifying : automake-1.16.2-8.el9.noarch 39/177 2023-08-09T13:32:51.487 INFO:teuthology.orchestra.run.smithi203.stdout: Verifying : bison-3.7.4-5.el9.x86_64 40/177 2023-08-09T13:32:51.487 INFO:teuthology.orchestra.run.smithi203.stdout: Verifying : boost-program-options-1.75.0-8.el9.x86_64 41/177 2023-08-09T13:32:51.488 INFO:teuthology.orchestra.run.smithi203.stdout: Verifying : elfutils-libelf-devel-0.189-3.el9.x86_64 42/177 2023-08-09T13:32:51.488 INFO:teuthology.orchestra.run.smithi203.stdout: Verifying : flex-2.6.4-9.el9.x86_64 43/177 2023-08-09T13:32:51.488 INFO:teuthology.orchestra.run.smithi203.stdout: Verifying : flexiblas-3.0.4-8.el9.x86_64 44/177 2023-08-09T13:32:51.488 INFO:teuthology.orchestra.run.smithi203.stdout: Verifying : flexiblas-netlib-3.0.4-8.el9.x86_64 45/177 2023-08-09T13:32:51.488 INFO:teuthology.orchestra.run.smithi203.stdout: Verifying : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 46/177 2023-08-09T13:32:51.488 INFO:teuthology.orchestra.run.smithi203.stdout: Verifying : jq-1.6-15.el9.x86_64 47/177 2023-08-09T13:32:51.488 INFO:teuthology.orchestra.run.smithi203.stdout: Verifying : libaio-devel-0.3.111-13.el9.x86_64 48/177 2023-08-09T13:32:51.488 INFO:teuthology.orchestra.run.smithi203.stdout: Verifying : libpmemobj-1.12.1-1.el9.x86_64 49/177 2023-08-09T13:32:51.489 INFO:teuthology.orchestra.run.smithi203.stdout: Verifying : librabbitmq-0.11.0-7.el9.x86_64 50/177 2023-08-09T13:32:51.489 INFO:teuthology.orchestra.run.smithi203.stdout: Verifying : librdkafka-1.6.1-102.el9.x86_64 51/177 2023-08-09T13:32:51.489 INFO:teuthology.orchestra.run.smithi203.stdout: Verifying : libtool-2.4.6-45.el9.x86_64 52/177 2023-08-09T13:32:51.489 INFO:teuthology.orchestra.run.smithi203.stdout: Verifying : libuuid-devel-2.37.4-11.el9.x86_64 53/177 2023-08-09T13:32:51.489 INFO:teuthology.orchestra.run.smithi203.stdout: Verifying : libzstd-devel-1.5.1-2.el9.x86_64 54/177 2023-08-09T13:32:51.489 INFO:teuthology.orchestra.run.smithi203.stdout: Verifying : lttng-ust-2.12.0-6.el9.x86_64 55/177 2023-08-09T13:32:51.489 INFO:teuthology.orchestra.run.smithi203.stdout: Verifying : oniguruma-6.9.6-1.el9.5.x86_64 56/177 2023-08-09T13:32:51.490 INFO:teuthology.orchestra.run.smithi203.stdout: Verifying : openblas-0.3.21-2.el9.x86_64 57/177 2023-08-09T13:32:51.490 INFO:teuthology.orchestra.run.smithi203.stdout: Verifying : openblas-openmp-0.3.21-2.el9.x86_64 58/177 2023-08-09T13:32:51.490 INFO:teuthology.orchestra.run.smithi203.stdout: Verifying : openssl-devel-1:3.0.7-24.el9.x86_64 59/177 2023-08-09T13:32:51.490 INFO:teuthology.orchestra.run.smithi203.stdout: Verifying : postgresql-13.11-1.el9.x86_64 60/177 2023-08-09T13:32:51.490 INFO:teuthology.orchestra.run.smithi203.stdout: Verifying : postgresql-contrib-13.11-1.el9.x86_64 61/177 2023-08-09T13:32:51.490 INFO:teuthology.orchestra.run.smithi203.stdout: Verifying : postgresql-private-libs-13.11-1.el9.x86_64 62/177 2023-08-09T13:32:51.490 INFO:teuthology.orchestra.run.smithi203.stdout: Verifying : postgresql-server-13.11-1.el9.x86_64 63/177 2023-08-09T13:32:51.491 INFO:teuthology.orchestra.run.smithi203.stdout: Verifying : python3-babel-2.9.1-2.el9.noarch 64/177 2023-08-09T13:32:51.491 INFO:teuthology.orchestra.run.smithi203.stdout: Verifying : python3-jinja2-2.11.3-4.el9.noarch 65/177 2023-08-09T13:32:51.491 INFO:teuthology.orchestra.run.smithi203.stdout: Verifying : python3-jsonpatch-1.21-16.el9.noarch 66/177 2023-08-09T13:32:51.491 INFO:teuthology.orchestra.run.smithi203.stdout: Verifying : python3-jsonpointer-2.0-4.el9.noarch 67/177 2023-08-09T13:32:51.491 INFO:teuthology.orchestra.run.smithi203.stdout: Verifying : python3-mako-1.1.4-6.el9.noarch 68/177 2023-08-09T13:32:51.491 INFO:teuthology.orchestra.run.smithi203.stdout: Verifying : python3-markupsafe-1.1.1-12.el9.x86_64 69/177 2023-08-09T13:32:51.491 INFO:teuthology.orchestra.run.smithi203.stdout: Verifying : python3-numpy-1:1.20.1-5.el9.x86_64 70/177 2023-08-09T13:32:51.491 INFO:teuthology.orchestra.run.smithi203.stdout: Verifying : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 71/177 2023-08-09T13:32:51.492 INFO:teuthology.orchestra.run.smithi203.stdout: Verifying : python3-oauthlib-3.1.1-5.el9.noarch 72/177 2023-08-09T13:32:51.492 INFO:teuthology.orchestra.run.smithi203.stdout: Verifying : python3-prettytable-0.7.2-27.el9.noarch 73/177 2023-08-09T13:32:51.492 INFO:teuthology.orchestra.run.smithi203.stdout: Verifying : python3-pyasn1-0.4.8-6.el9.noarch 74/177 2023-08-09T13:32:51.492 INFO:teuthology.orchestra.run.smithi203.stdout: Verifying : python3-pyasn1-modules-0.4.8-6.el9.noarch 75/177 2023-08-09T13:32:51.492 INFO:teuthology.orchestra.run.smithi203.stdout: Verifying : python3-pytz-2021.1-5.el9.noarch 76/177 2023-08-09T13:32:51.492 INFO:teuthology.orchestra.run.smithi203.stdout: Verifying : python3-requests-oauthlib-1.3.0-12.el9.noarch 77/177 2023-08-09T13:32:51.492 INFO:teuthology.orchestra.run.smithi203.stdout: Verifying : python3-scipy-1.6.2-8.el9.x86_64 78/177 2023-08-09T13:32:51.492 INFO:teuthology.orchestra.run.smithi203.stdout: Verifying : python3-toml-0.10.2-6.el9.noarch 79/177 2023-08-09T13:32:51.493 INFO:teuthology.orchestra.run.smithi203.stdout: Verifying : socat-1.7.4.1-5.el9.x86_64 80/177 2023-08-09T13:32:51.493 INFO:teuthology.orchestra.run.smithi203.stdout: Verifying : uuid-1.6.2-55.el9.x86_64 81/177 2023-08-09T13:32:51.493 INFO:teuthology.orchestra.run.smithi203.stdout: Verifying : xmlstarlet-1.6.1-20.el9.x86_64 82/177 2023-08-09T13:32:51.493 INFO:teuthology.orchestra.run.smithi203.stdout: Verifying : fuse-2.9.9-15.el9.x86_64 83/177 2023-08-09T13:32:51.493 INFO:teuthology.orchestra.run.smithi203.stdout: Verifying : libgfortran-11.4.1-2.1.el9.x86_64 84/177 2023-08-09T13:32:51.493 INFO:teuthology.orchestra.run.smithi203.stdout: Verifying : libquadmath-11.4.1-2.1.el9.x86_64 85/177 2023-08-09T13:32:51.493 INFO:teuthology.orchestra.run.smithi203.stdout: Verifying : python3-cffi-1.14.5-5.el9.x86_64 86/177 2023-08-09T13:32:51.494 INFO:teuthology.orchestra.run.smithi203.stdout: Verifying : python3-cryptography-36.0.1-4.el9.x86_64 87/177 2023-08-09T13:32:51.494 INFO:teuthology.orchestra.run.smithi203.stdout: Verifying : python3-ply-3.11-14.el9.noarch 88/177 2023-08-09T13:32:51.494 INFO:teuthology.orchestra.run.smithi203.stdout: Verifying : python3-pycparser-2.20-6.el9.noarch 89/177 2023-08-09T13:32:51.494 INFO:teuthology.orchestra.run.smithi203.stdout: Verifying : python3-devel-3.9.16-1.el9.x86_64 90/177 2023-08-09T13:32:51.494 INFO:teuthology.orchestra.run.smithi203.stdout: Verifying : xfsprogs-devel-5.19.0-4.el9.x86_64 91/177 2023-08-09T13:32:51.494 INFO:teuthology.orchestra.run.smithi203.stdout: Verifying : zlib-devel-1.2.11-36.el9.x86_64 92/177 2023-08-09T13:32:51.494 INFO:teuthology.orchestra.run.smithi203.stdout: Verifying : fmt-8.1.1-5.el9.x86_64 93/177 2023-08-09T13:32:51.495 INFO:teuthology.orchestra.run.smithi203.stdout: Verifying : gperftools-libs-2.9.1-2.el9.x86_64 94/177 2023-08-09T13:32:51.495 INFO:teuthology.orchestra.run.smithi203.stdout: Verifying : libarrow-9.0.0-9.el9.x86_64 95/177 2023-08-09T13:32:51.495 INFO:teuthology.orchestra.run.smithi203.stdout: Verifying : libarrow-doc-9.0.0-9.el9.noarch 96/177 2023-08-09T13:32:51.495 INFO:teuthology.orchestra.run.smithi203.stdout: Verifying : liboath-2.6.7-2.el9.x86_64 97/177 2023-08-09T13:32:51.495 INFO:teuthology.orchestra.run.smithi203.stdout: Verifying : libunwind-1.6.2-1.el9.x86_64 98/177 2023-08-09T13:32:51.495 INFO:teuthology.orchestra.run.smithi203.stdout: Verifying : parquet-libs-9.0.0-9.el9.x86_64 99/177 2023-08-09T13:32:51.495 INFO:teuthology.orchestra.run.smithi203.stdout: Verifying : python3-asyncssh-2.13.2-3.el9.noarch 100/177 2023-08-09T13:32:51.496 INFO:teuthology.orchestra.run.smithi203.stdout: Verifying : python3-bcrypt-3.2.2-1.el9.x86_64 101/177 2023-08-09T13:32:51.496 INFO:teuthology.orchestra.run.smithi203.stdout: Verifying : python3-cachetools-4.2.4-1.el9.noarch 102/177 2023-08-09T13:32:51.496 INFO:teuthology.orchestra.run.smithi203.stdout: Verifying : python3-certifi-2021.10.8-2.el9.noarch 103/177 2023-08-09T13:32:51.496 INFO:teuthology.orchestra.run.smithi203.stdout: Verifying : python3-cheroot-8.6.0-4.el9.noarch 104/177 2023-08-09T13:32:51.496 INFO:teuthology.orchestra.run.smithi203.stdout: Verifying : python3-cherrypy-18.6.1-2.el9.noarch 105/177 2023-08-09T13:32:51.496 INFO:teuthology.orchestra.run.smithi203.stdout: Verifying : python3-google-auth-1:2.22.0-1.el9.noarch 106/177 2023-08-09T13:32:51.496 INFO:teuthology.orchestra.run.smithi203.stdout: Verifying : python3-jaraco-8.2.1-3.el9.noarch 107/177 2023-08-09T13:32:51.497 INFO:teuthology.orchestra.run.smithi203.stdout: Verifying : python3-jaraco-classes-3.2.1-5.el9.noarch 108/177 2023-08-09T13:32:51.497 INFO:teuthology.orchestra.run.smithi203.stdout: Verifying : python3-jaraco-collections-3.0.0-8.el9.noarch 109/177 2023-08-09T13:32:51.497 INFO:teuthology.orchestra.run.smithi203.stdout: Verifying : python3-jaraco-functools-3.5.0-2.el9.noarch 110/177 2023-08-09T13:32:51.497 INFO:teuthology.orchestra.run.smithi203.stdout: Verifying : python3-jaraco-text-3.2.0-6.el9.noarch 111/177 2023-08-09T13:32:51.497 INFO:teuthology.orchestra.run.smithi203.stdout: Verifying : python3-jwt+crypto-2.4.0-1.el9.noarch 112/177 2023-08-09T13:32:51.497 INFO:teuthology.orchestra.run.smithi203.stdout: Verifying : python3-jwt-2.4.0-1.el9.noarch 113/177 2023-08-09T13:32:51.497 INFO:teuthology.orchestra.run.smithi203.stdout: Verifying : python3-kubernetes-1:26.1.0-2.el9.noarch 114/177 2023-08-09T13:32:51.498 INFO:teuthology.orchestra.run.smithi203.stdout: Verifying : python3-logutils-0.3.5-21.el9.noarch 115/177 2023-08-09T13:32:51.498 INFO:teuthology.orchestra.run.smithi203.stdout: Verifying : python3-more-itertools-8.12.0-2.el9.noarch 116/177 2023-08-09T13:32:51.498 INFO:teuthology.orchestra.run.smithi203.stdout: Verifying : python3-natsort-7.1.1-5.el9.noarch 117/177 2023-08-09T13:32:51.498 INFO:teuthology.orchestra.run.smithi203.stdout: Verifying : python3-pecan-1.4.2-3.el9.noarch 118/177 2023-08-09T13:32:51.498 INFO:teuthology.orchestra.run.smithi203.stdout: Verifying : python3-portend-3.1.0-2.el9.noarch 119/177 2023-08-09T13:32:51.498 INFO:teuthology.orchestra.run.smithi203.stdout: Verifying : python3-pyOpenSSL-21.0.0-1.el9.noarch 120/177 2023-08-09T13:32:51.498 INFO:teuthology.orchestra.run.smithi203.stdout: Verifying : python3-repoze-lru-0.7-16.el9.noarch 121/177 2023-08-09T13:32:51.498 INFO:teuthology.orchestra.run.smithi203.stdout: Verifying : python3-routes-2.5.1-5.el9.noarch 122/177 2023-08-09T13:32:51.499 INFO:teuthology.orchestra.run.smithi203.stdout: Verifying : python3-rsa-4.9-2.el9.noarch 123/177 2023-08-09T13:32:51.499 INFO:teuthology.orchestra.run.smithi203.stdout: Verifying : python3-tempora-5.0.0-2.el9.noarch 124/177 2023-08-09T13:32:51.499 INFO:teuthology.orchestra.run.smithi203.stdout: Verifying : python3-typing-extensions-4.4.0-2.el9.noarch 125/177 2023-08-09T13:32:51.499 INFO:teuthology.orchestra.run.smithi203.stdout: Verifying : python3-webob-1.8.7-6.el9.noarch 126/177 2023-08-09T13:32:51.499 INFO:teuthology.orchestra.run.smithi203.stdout: Verifying : python3-websocket-client-1.2.3-2.el9.noarch 127/177 2023-08-09T13:32:51.499 INFO:teuthology.orchestra.run.smithi203.stdout: Verifying : python3-werkzeug-2.0.3-3.el9.1.noarch 128/177 2023-08-09T13:32:51.499 INFO:teuthology.orchestra.run.smithi203.stdout: Verifying : python3-zc-lockfile-2.0-10.el9.noarch 129/177 2023-08-09T13:32:51.499 INFO:teuthology.orchestra.run.smithi203.stdout: Verifying : re2-1:20211101-2.el9.x86_64 130/177 2023-08-09T13:32:51.500 INFO:teuthology.orchestra.run.smithi203.stdout: Verifying : thrift-0.15.0-2.el9.x86_64 131/177 2023-08-09T13:32:51.500 INFO:teuthology.orchestra.run.smithi203.stdout: Verifying : librados2-2:18.0.0-5389.g668a86f2.el9.x86_64 132/177 2023-08-09T13:32:51.500 INFO:teuthology.orchestra.run.smithi203.stdout: Verifying : librados2-2:16.2.4-5.el9.x86_64 133/177 2023-08-09T13:32:51.500 INFO:teuthology.orchestra.run.smithi203.stdout: Verifying : librbd1-2:18.0.0-5389.g668a86f2.el9.x86_64 134/177 2023-08-09T13:32:51.500 INFO:teuthology.orchestra.run.smithi203.stdout: Verifying : librbd1-2:16.2.4-5.el9.x86_64 135/177 2023-08-09T13:32:51.500 INFO:teuthology.orchestra.run.smithi203.stdout: Verifying : NetworkManager-1:1.43.90-1.el9.x86_64 136/177 2023-08-09T13:32:51.500 INFO:teuthology.orchestra.run.smithi203.stdout: Verifying : NetworkManager-1:1.41.8-1.el9.x86_64 137/177 2023-08-09T13:32:51.501 INFO:teuthology.orchestra.run.smithi203.stdout: Verifying : NetworkManager-libnm-1:1.43.90-1.el9.x86_64 138/177 2023-08-09T13:32:51.501 INFO:teuthology.orchestra.run.smithi203.stdout: Verifying : NetworkManager-libnm-1:1.41.8-1.el9.x86_64 139/177 2023-08-09T13:32:51.501 INFO:teuthology.orchestra.run.smithi203.stdout: Verifying : NetworkManager-team-1:1.43.90-1.el9.x86_64 140/177 2023-08-09T13:32:51.501 INFO:teuthology.orchestra.run.smithi203.stdout: Verifying : NetworkManager-team-1:1.41.8-1.el9.x86_64 141/177 2023-08-09T13:32:51.501 INFO:teuthology.orchestra.run.smithi203.stdout: Verifying : NetworkManager-tui-1:1.43.90-1.el9.x86_64 142/177 2023-08-09T13:32:51.501 INFO:teuthology.orchestra.run.smithi203.stdout: Verifying : NetworkManager-tui-1:1.41.8-1.el9.x86_64 143/177 2023-08-09T13:32:51.501 INFO:teuthology.orchestra.run.smithi203.stdout: Verifying : elfutils-debuginfod-client-0.189-3.el9.x86_64 144/177 2023-08-09T13:32:51.502 INFO:teuthology.orchestra.run.smithi203.stdout: Verifying : elfutils-debuginfod-client-0.188-3.el9.x86_64 145/177 2023-08-09T13:32:51.502 INFO:teuthology.orchestra.run.smithi203.stdout: Verifying : elfutils-libelf-0.189-3.el9.x86_64 146/177 2023-08-09T13:32:51.502 INFO:teuthology.orchestra.run.smithi203.stdout: Verifying : elfutils-libelf-0.188-3.el9.x86_64 147/177 2023-08-09T13:32:51.502 INFO:teuthology.orchestra.run.smithi203.stdout: Verifying : elfutils-libs-0.189-3.el9.x86_64 148/177 2023-08-09T13:32:51.502 INFO:teuthology.orchestra.run.smithi203.stdout: Verifying : elfutils-libs-0.188-3.el9.x86_64 149/177 2023-08-09T13:32:51.502 INFO:teuthology.orchestra.run.smithi203.stdout: Verifying : iproute-6.2.0-5.el9.x86_64 150/177 2023-08-09T13:32:51.502 INFO:teuthology.orchestra.run.smithi203.stdout: Verifying : iproute-6.0.0-2.el9.x86_64 151/177 2023-08-09T13:32:51.502 INFO:teuthology.orchestra.run.smithi203.stdout: Verifying : iproute-tc-6.2.0-5.el9.x86_64 152/177 2023-08-09T13:32:51.503 INFO:teuthology.orchestra.run.smithi203.stdout: Verifying : iproute-tc-6.0.0-2.el9.x86_64 153/177 2023-08-09T13:32:51.503 INFO:teuthology.orchestra.run.smithi203.stdout: Verifying : libblkid-2.37.4-11.el9.x86_64 154/177 2023-08-09T13:32:51.503 INFO:teuthology.orchestra.run.smithi203.stdout: Verifying : libblkid-2.37.4-9.el9.x86_64 155/177 2023-08-09T13:32:51.503 INFO:teuthology.orchestra.run.smithi203.stdout: Verifying : libfdisk-2.37.4-11.el9.x86_64 156/177 2023-08-09T13:32:51.503 INFO:teuthology.orchestra.run.smithi203.stdout: Verifying : libfdisk-2.37.4-9.el9.x86_64 157/177 2023-08-09T13:32:51.503 INFO:teuthology.orchestra.run.smithi203.stdout: Verifying : libmount-2.37.4-11.el9.x86_64 158/177 2023-08-09T13:32:51.503 INFO:teuthology.orchestra.run.smithi203.stdout: Verifying : libmount-2.37.4-9.el9.x86_64 159/177 2023-08-09T13:32:51.504 INFO:teuthology.orchestra.run.smithi203.stdout: Verifying : libsmartcols-2.37.4-11.el9.x86_64 160/177 2023-08-09T13:32:51.504 INFO:teuthology.orchestra.run.smithi203.stdout: Verifying : libsmartcols-2.37.4-9.el9.x86_64 161/177 2023-08-09T13:32:51.504 INFO:teuthology.orchestra.run.smithi203.stdout: Verifying : libuuid-2.37.4-11.el9.x86_64 162/177 2023-08-09T13:32:51.504 INFO:teuthology.orchestra.run.smithi203.stdout: Verifying : libuuid-2.37.4-9.el9.x86_64 163/177 2023-08-09T13:32:51.504 INFO:teuthology.orchestra.run.smithi203.stdout: Verifying : openssl-1:3.0.7-24.el9.x86_64 164/177 2023-08-09T13:32:51.504 INFO:teuthology.orchestra.run.smithi203.stdout: Verifying : openssl-1:3.0.7-2.el9.x86_64 165/177 2023-08-09T13:32:51.504 INFO:teuthology.orchestra.run.smithi203.stdout: Verifying : openssl-libs-1:3.0.7-24.el9.x86_64 166/177 2023-08-09T13:32:51.505 INFO:teuthology.orchestra.run.smithi203.stdout: Verifying : openssl-libs-1:3.0.7-2.el9.x86_64 167/177 2023-08-09T13:32:51.505 INFO:teuthology.orchestra.run.smithi203.stdout: Verifying : util-linux-2.37.4-11.el9.x86_64 168/177 2023-08-09T13:32:51.505 INFO:teuthology.orchestra.run.smithi203.stdout: Verifying : util-linux-2.37.4-9.el9.x86_64 169/177 2023-08-09T13:32:51.505 INFO:teuthology.orchestra.run.smithi203.stdout: Verifying : util-linux-core-2.37.4-11.el9.x86_64 170/177 2023-08-09T13:32:51.505 INFO:teuthology.orchestra.run.smithi203.stdout: Verifying : util-linux-core-2.37.4-9.el9.x86_64 171/177 2023-08-09T13:32:51.505 INFO:teuthology.orchestra.run.smithi203.stdout: Verifying : util-linux-user-2.37.4-11.el9.x86_64 172/177 2023-08-09T13:32:51.505 INFO:teuthology.orchestra.run.smithi203.stdout: Verifying : util-linux-user-2.37.4-9.el9.x86_64 173/177 2023-08-09T13:32:51.505 INFO:teuthology.orchestra.run.smithi203.stdout: Verifying : xfsdump-3.1.12-3.el9.x86_64 174/177 2023-08-09T13:32:51.506 INFO:teuthology.orchestra.run.smithi203.stdout: Verifying : xfsdump-3.1.10-1.el9.x86_64 175/177 2023-08-09T13:32:51.506 INFO:teuthology.orchestra.run.smithi203.stdout: Verifying : xfsprogs-5.19.0-4.el9.x86_64 176/177 2023-08-09T13:32:52.711 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : xfsprogs-5.14.2-1.el9.x86_64 177/177 2023-08-09T13:32:52.711 INFO:teuthology.orchestra.run.smithi043.stdout: 2023-08-09T13:32:52.711 INFO:teuthology.orchestra.run.smithi043.stdout:Upgraded: 2023-08-09T13:32:52.711 INFO:teuthology.orchestra.run.smithi043.stdout: NetworkManager-1:1.43.90-1.el9.x86_64 2023-08-09T13:32:52.712 INFO:teuthology.orchestra.run.smithi043.stdout: NetworkManager-libnm-1:1.43.90-1.el9.x86_64 2023-08-09T13:32:52.712 INFO:teuthology.orchestra.run.smithi043.stdout: NetworkManager-team-1:1.43.90-1.el9.x86_64 2023-08-09T13:32:52.712 INFO:teuthology.orchestra.run.smithi043.stdout: NetworkManager-tui-1:1.43.90-1.el9.x86_64 2023-08-09T13:32:52.712 INFO:teuthology.orchestra.run.smithi043.stdout: elfutils-debuginfod-client-0.189-3.el9.x86_64 2023-08-09T13:32:52.712 INFO:teuthology.orchestra.run.smithi043.stdout: elfutils-libelf-0.189-3.el9.x86_64 2023-08-09T13:32:52.712 INFO:teuthology.orchestra.run.smithi043.stdout: elfutils-libs-0.189-3.el9.x86_64 2023-08-09T13:32:52.713 INFO:teuthology.orchestra.run.smithi043.stdout: iproute-6.2.0-5.el9.x86_64 2023-08-09T13:32:52.713 INFO:teuthology.orchestra.run.smithi043.stdout: iproute-tc-6.2.0-5.el9.x86_64 2023-08-09T13:32:52.713 INFO:teuthology.orchestra.run.smithi043.stdout: libblkid-2.37.4-11.el9.x86_64 2023-08-09T13:32:52.713 INFO:teuthology.orchestra.run.smithi043.stdout: libfdisk-2.37.4-11.el9.x86_64 2023-08-09T13:32:52.714 INFO:teuthology.orchestra.run.smithi043.stdout: libmount-2.37.4-11.el9.x86_64 2023-08-09T13:32:52.714 INFO:teuthology.orchestra.run.smithi043.stdout: librados2-2:18.0.0-5389.g668a86f2.el9.x86_64 2023-08-09T13:32:52.714 INFO:teuthology.orchestra.run.smithi043.stdout: librbd1-2:18.0.0-5389.g668a86f2.el9.x86_64 2023-08-09T13:32:52.714 INFO:teuthology.orchestra.run.smithi043.stdout: libsmartcols-2.37.4-11.el9.x86_64 2023-08-09T13:32:52.714 INFO:teuthology.orchestra.run.smithi043.stdout: libuuid-2.37.4-11.el9.x86_64 2023-08-09T13:32:52.714 INFO:teuthology.orchestra.run.smithi043.stdout: openssl-1:3.0.7-24.el9.x86_64 2023-08-09T13:32:52.714 INFO:teuthology.orchestra.run.smithi043.stdout: openssl-libs-1:3.0.7-24.el9.x86_64 2023-08-09T13:32:52.714 INFO:teuthology.orchestra.run.smithi043.stdout: util-linux-2.37.4-11.el9.x86_64 2023-08-09T13:32:52.715 INFO:teuthology.orchestra.run.smithi043.stdout: util-linux-core-2.37.4-11.el9.x86_64 2023-08-09T13:32:52.715 INFO:teuthology.orchestra.run.smithi043.stdout: util-linux-user-2.37.4-11.el9.x86_64 2023-08-09T13:32:52.715 INFO:teuthology.orchestra.run.smithi043.stdout: xfsdump-3.1.12-3.el9.x86_64 2023-08-09T13:32:52.715 INFO:teuthology.orchestra.run.smithi043.stdout: xfsprogs-5.19.0-4.el9.x86_64 2023-08-09T13:32:52.715 INFO:teuthology.orchestra.run.smithi043.stdout:Installed: 2023-08-09T13:32:52.715 INFO:teuthology.orchestra.run.smithi043.stdout: automake-1.16.2-8.el9.noarch 2023-08-09T13:32:52.715 INFO:teuthology.orchestra.run.smithi043.stdout: bison-3.7.4-5.el9.x86_64 2023-08-09T13:32:52.716 INFO:teuthology.orchestra.run.smithi043.stdout: boost-program-options-1.75.0-8.el9.x86_64 2023-08-09T13:32:52.716 INFO:teuthology.orchestra.run.smithi043.stdout: ceph-2:18.0.0-5389.g668a86f2.el9.x86_64 2023-08-09T13:32:52.716 INFO:teuthology.orchestra.run.smithi043.stdout: ceph-base-2:18.0.0-5389.g668a86f2.el9.x86_64 2023-08-09T13:32:52.716 INFO:teuthology.orchestra.run.smithi043.stdout: ceph-common-2:18.0.0-5389.g668a86f2.el9.x86_64 2023-08-09T13:32:52.716 INFO:teuthology.orchestra.run.smithi043.stdout: ceph-fuse-2:18.0.0-5389.g668a86f2.el9.x86_64 2023-08-09T13:32:52.716 INFO:teuthology.orchestra.run.smithi043.stdout: ceph-grafana-dashboards-2:18.0.0-5389.g668a86f2.el9.noarch 2023-08-09T13:32:52.716 INFO:teuthology.orchestra.run.smithi043.stdout: ceph-immutable-object-cache-2:18.0.0-5389.g668a86f2.el9.x86_64 2023-08-09T13:32:52.717 INFO:teuthology.orchestra.run.smithi043.stdout: ceph-mds-2:18.0.0-5389.g668a86f2.el9.x86_64 2023-08-09T13:32:52.717 INFO:teuthology.orchestra.run.smithi043.stdout: ceph-mgr-2:18.0.0-5389.g668a86f2.el9.x86_64 2023-08-09T13:32:52.717 INFO:teuthology.orchestra.run.smithi043.stdout: ceph-mgr-cephadm-2:18.0.0-5389.g668a86f2.el9.noarch 2023-08-09T13:32:52.717 INFO:teuthology.orchestra.run.smithi043.stdout: ceph-mgr-dashboard-2:18.0.0-5389.g668a86f2.el9.noarch 2023-08-09T13:32:52.717 INFO:teuthology.orchestra.run.smithi043.stdout: ceph-mgr-diskprediction-local-2:18.0.0-5389.g668a86f2.el9.noarch 2023-08-09T13:32:52.717 INFO:teuthology.orchestra.run.smithi043.stdout: ceph-mgr-modules-core-2:18.0.0-5389.g668a86f2.el9.noarch 2023-08-09T13:32:52.717 INFO:teuthology.orchestra.run.smithi043.stdout: ceph-mgr-rook-2:18.0.0-5389.g668a86f2.el9.noarch 2023-08-09T13:32:52.718 INFO:teuthology.orchestra.run.smithi043.stdout: ceph-mon-2:18.0.0-5389.g668a86f2.el9.x86_64 2023-08-09T13:32:52.718 INFO:teuthology.orchestra.run.smithi043.stdout: ceph-osd-2:18.0.0-5389.g668a86f2.el9.x86_64 2023-08-09T13:32:52.718 INFO:teuthology.orchestra.run.smithi043.stdout: ceph-prometheus-alerts-2:18.0.0-5389.g668a86f2.el9.noarch 2023-08-09T13:32:52.718 INFO:teuthology.orchestra.run.smithi043.stdout: ceph-radosgw-2:18.0.0-5389.g668a86f2.el9.x86_64 2023-08-09T13:32:52.718 INFO:teuthology.orchestra.run.smithi043.stdout: ceph-selinux-2:18.0.0-5389.g668a86f2.el9.x86_64 2023-08-09T13:32:52.718 INFO:teuthology.orchestra.run.smithi043.stdout: ceph-test-2:18.0.0-5389.g668a86f2.el9.x86_64 2023-08-09T13:32:52.718 INFO:teuthology.orchestra.run.smithi043.stdout: ceph-volume-2:18.0.0-5389.g668a86f2.el9.noarch 2023-08-09T13:32:52.718 INFO:teuthology.orchestra.run.smithi043.stdout: cephadm-2:18.0.0-5389.g668a86f2.el9.noarch 2023-08-09T13:32:52.719 INFO:teuthology.orchestra.run.smithi043.stdout: cephfs-mirror-2:18.0.0-5389.g668a86f2.el9.x86_64 2023-08-09T13:32:52.719 INFO:teuthology.orchestra.run.smithi043.stdout: cephfs-top-2:18.0.0-5389.g668a86f2.el9.noarch 2023-08-09T13:32:52.719 INFO:teuthology.orchestra.run.smithi043.stdout: elfutils-libelf-devel-0.189-3.el9.x86_64 2023-08-09T13:32:52.719 INFO:teuthology.orchestra.run.smithi043.stdout: flex-2.6.4-9.el9.x86_64 2023-08-09T13:32:52.719 INFO:teuthology.orchestra.run.smithi043.stdout: flexiblas-3.0.4-8.el9.x86_64 2023-08-09T13:32:52.719 INFO:teuthology.orchestra.run.smithi043.stdout: flexiblas-netlib-3.0.4-8.el9.x86_64 2023-08-09T13:32:52.719 INFO:teuthology.orchestra.run.smithi043.stdout: flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 2023-08-09T13:32:52.719 INFO:teuthology.orchestra.run.smithi043.stdout: fmt-8.1.1-5.el9.x86_64 2023-08-09T13:32:52.720 INFO:teuthology.orchestra.run.smithi043.stdout: fuse-2.9.9-15.el9.x86_64 2023-08-09T13:32:52.720 INFO:teuthology.orchestra.run.smithi043.stdout: gperftools-libs-2.9.1-2.el9.x86_64 2023-08-09T13:32:52.720 INFO:teuthology.orchestra.run.smithi043.stdout: jq-1.6-15.el9.x86_64 2023-08-09T13:32:52.720 INFO:teuthology.orchestra.run.smithi043.stdout: libaio-devel-0.3.111-13.el9.x86_64 2023-08-09T13:32:52.720 INFO:teuthology.orchestra.run.smithi043.stdout: libarrow-9.0.0-9.el9.x86_64 2023-08-09T13:32:52.720 INFO:teuthology.orchestra.run.smithi043.stdout: libarrow-doc-9.0.0-9.el9.noarch 2023-08-09T13:32:52.720 INFO:teuthology.orchestra.run.smithi043.stdout: libcephfs-devel-2:18.0.0-5389.g668a86f2.el9.x86_64 2023-08-09T13:32:52.721 INFO:teuthology.orchestra.run.smithi043.stdout: libcephfs2-2:18.0.0-5389.g668a86f2.el9.x86_64 2023-08-09T13:32:52.721 INFO:teuthology.orchestra.run.smithi043.stdout: libcephsqlite-2:18.0.0-5389.g668a86f2.el9.x86_64 2023-08-09T13:32:52.721 INFO:teuthology.orchestra.run.smithi043.stdout: libgfortran-11.4.1-2.1.el9.x86_64 2023-08-09T13:32:52.721 INFO:teuthology.orchestra.run.smithi043.stdout: liboath-2.6.7-2.el9.x86_64 2023-08-09T13:32:52.721 INFO:teuthology.orchestra.run.smithi043.stdout: libpmemobj-1.12.1-1.el9.x86_64 2023-08-09T13:32:52.721 INFO:teuthology.orchestra.run.smithi043.stdout: libquadmath-11.4.1-2.1.el9.x86_64 2023-08-09T13:32:52.721 INFO:teuthology.orchestra.run.smithi043.stdout: librabbitmq-0.11.0-7.el9.x86_64 2023-08-09T13:32:52.721 INFO:teuthology.orchestra.run.smithi043.stdout: librados-devel-2:18.0.0-5389.g668a86f2.el9.x86_64 2023-08-09T13:32:52.722 INFO:teuthology.orchestra.run.smithi043.stdout: libradosstriper1-2:18.0.0-5389.g668a86f2.el9.x86_64 2023-08-09T13:32:52.722 INFO:teuthology.orchestra.run.smithi043.stdout: librdkafka-1.6.1-102.el9.x86_64 2023-08-09T13:32:52.722 INFO:teuthology.orchestra.run.smithi043.stdout: librgw2-2:18.0.0-5389.g668a86f2.el9.x86_64 2023-08-09T13:32:52.722 INFO:teuthology.orchestra.run.smithi043.stdout: libtool-2.4.6-45.el9.x86_64 2023-08-09T13:32:52.722 INFO:teuthology.orchestra.run.smithi043.stdout: libunwind-1.6.2-1.el9.x86_64 2023-08-09T13:32:52.722 INFO:teuthology.orchestra.run.smithi043.stdout: libuuid-devel-2.37.4-11.el9.x86_64 2023-08-09T13:32:52.722 INFO:teuthology.orchestra.run.smithi043.stdout: libzstd-devel-1.5.1-2.el9.x86_64 2023-08-09T13:32:52.722 INFO:teuthology.orchestra.run.smithi043.stdout: lttng-ust-2.12.0-6.el9.x86_64 2023-08-09T13:32:52.723 INFO:teuthology.orchestra.run.smithi043.stdout: oniguruma-6.9.6-1.el9.5.x86_64 2023-08-09T13:32:52.723 INFO:teuthology.orchestra.run.smithi043.stdout: openblas-0.3.21-2.el9.x86_64 2023-08-09T13:32:52.723 INFO:teuthology.orchestra.run.smithi043.stdout: openblas-openmp-0.3.21-2.el9.x86_64 2023-08-09T13:32:52.723 INFO:teuthology.orchestra.run.smithi043.stdout: openssl-devel-1:3.0.7-24.el9.x86_64 2023-08-09T13:32:52.723 INFO:teuthology.orchestra.run.smithi043.stdout: parquet-libs-9.0.0-9.el9.x86_64 2023-08-09T13:32:52.723 INFO:teuthology.orchestra.run.smithi043.stdout: postgresql-13.11-1.el9.x86_64 2023-08-09T13:32:52.723 INFO:teuthology.orchestra.run.smithi043.stdout: postgresql-contrib-13.11-1.el9.x86_64 2023-08-09T13:32:52.723 INFO:teuthology.orchestra.run.smithi043.stdout: postgresql-private-libs-13.11-1.el9.x86_64 2023-08-09T13:32:52.724 INFO:teuthology.orchestra.run.smithi043.stdout: postgresql-server-13.11-1.el9.x86_64 2023-08-09T13:32:52.724 INFO:teuthology.orchestra.run.smithi043.stdout: python3-asyncssh-2.13.2-3.el9.noarch 2023-08-09T13:32:52.724 INFO:teuthology.orchestra.run.smithi043.stdout: python3-babel-2.9.1-2.el9.noarch 2023-08-09T13:32:52.724 INFO:teuthology.orchestra.run.smithi043.stdout: python3-bcrypt-3.2.2-1.el9.x86_64 2023-08-09T13:32:52.724 INFO:teuthology.orchestra.run.smithi043.stdout: python3-cachetools-4.2.4-1.el9.noarch 2023-08-09T13:32:52.724 INFO:teuthology.orchestra.run.smithi043.stdout: python3-ceph-argparse-2:18.0.0-5389.g668a86f2.el9.x86_64 2023-08-09T13:32:52.725 INFO:teuthology.orchestra.run.smithi043.stdout: python3-ceph-common-2:18.0.0-5389.g668a86f2.el9.x86_64 2023-08-09T13:32:52.725 INFO:teuthology.orchestra.run.smithi043.stdout: python3-cephfs-2:18.0.0-5389.g668a86f2.el9.x86_64 2023-08-09T13:32:52.725 INFO:teuthology.orchestra.run.smithi043.stdout: python3-certifi-2021.10.8-2.el9.noarch 2023-08-09T13:32:52.725 INFO:teuthology.orchestra.run.smithi043.stdout: python3-cffi-1.14.5-5.el9.x86_64 2023-08-09T13:32:52.725 INFO:teuthology.orchestra.run.smithi043.stdout: python3-cheroot-8.6.0-4.el9.noarch 2023-08-09T13:32:52.725 INFO:teuthology.orchestra.run.smithi043.stdout: python3-cherrypy-18.6.1-2.el9.noarch 2023-08-09T13:32:52.725 INFO:teuthology.orchestra.run.smithi043.stdout: python3-cryptography-36.0.1-4.el9.x86_64 2023-08-09T13:32:52.725 INFO:teuthology.orchestra.run.smithi043.stdout: python3-devel-3.9.16-1.el9.x86_64 2023-08-09T13:32:52.726 INFO:teuthology.orchestra.run.smithi043.stdout: python3-google-auth-1:2.22.0-1.el9.noarch 2023-08-09T13:32:52.726 INFO:teuthology.orchestra.run.smithi043.stdout: python3-jaraco-8.2.1-3.el9.noarch 2023-08-09T13:32:52.726 INFO:teuthology.orchestra.run.smithi043.stdout: python3-jaraco-classes-3.2.1-5.el9.noarch 2023-08-09T13:32:52.726 INFO:teuthology.orchestra.run.smithi043.stdout: python3-jaraco-collections-3.0.0-8.el9.noarch 2023-08-09T13:32:52.726 INFO:teuthology.orchestra.run.smithi043.stdout: python3-jaraco-functools-3.5.0-2.el9.noarch 2023-08-09T13:32:52.726 INFO:teuthology.orchestra.run.smithi043.stdout: python3-jaraco-text-3.2.0-6.el9.noarch 2023-08-09T13:32:52.726 INFO:teuthology.orchestra.run.smithi043.stdout: python3-jinja2-2.11.3-4.el9.noarch 2023-08-09T13:32:52.726 INFO:teuthology.orchestra.run.smithi043.stdout: python3-jsonpatch-1.21-16.el9.noarch 2023-08-09T13:32:52.727 INFO:teuthology.orchestra.run.smithi043.stdout: python3-jsonpointer-2.0-4.el9.noarch 2023-08-09T13:32:52.727 INFO:teuthology.orchestra.run.smithi043.stdout: python3-jwt-2.4.0-1.el9.noarch 2023-08-09T13:32:52.727 INFO:teuthology.orchestra.run.smithi043.stdout: python3-jwt+crypto-2.4.0-1.el9.noarch 2023-08-09T13:32:52.727 INFO:teuthology.orchestra.run.smithi043.stdout: python3-kubernetes-1:26.1.0-2.el9.noarch 2023-08-09T13:32:52.727 INFO:teuthology.orchestra.run.smithi043.stdout: python3-logutils-0.3.5-21.el9.noarch 2023-08-09T13:32:52.727 INFO:teuthology.orchestra.run.smithi043.stdout: python3-mako-1.1.4-6.el9.noarch 2023-08-09T13:32:52.727 INFO:teuthology.orchestra.run.smithi043.stdout: python3-markupsafe-1.1.1-12.el9.x86_64 2023-08-09T13:32:52.727 INFO:teuthology.orchestra.run.smithi043.stdout: python3-more-itertools-8.12.0-2.el9.noarch 2023-08-09T13:32:52.728 INFO:teuthology.orchestra.run.smithi043.stdout: python3-natsort-7.1.1-5.el9.noarch 2023-08-09T13:32:52.728 INFO:teuthology.orchestra.run.smithi043.stdout: python3-numpy-1:1.20.1-5.el9.x86_64 2023-08-09T13:32:52.728 INFO:teuthology.orchestra.run.smithi043.stdout: python3-numpy-f2py-1:1.20.1-5.el9.x86_64 2023-08-09T13:32:52.728 INFO:teuthology.orchestra.run.smithi043.stdout: python3-oauthlib-3.1.1-5.el9.noarch 2023-08-09T13:32:52.728 INFO:teuthology.orchestra.run.smithi043.stdout: python3-pecan-1.4.2-3.el9.noarch 2023-08-09T13:32:52.728 INFO:teuthology.orchestra.run.smithi043.stdout: python3-ply-3.11-14.el9.noarch 2023-08-09T13:32:52.728 INFO:teuthology.orchestra.run.smithi043.stdout: python3-portend-3.1.0-2.el9.noarch 2023-08-09T13:32:52.729 INFO:teuthology.orchestra.run.smithi043.stdout: python3-prettytable-0.7.2-27.el9.noarch 2023-08-09T13:32:52.729 INFO:teuthology.orchestra.run.smithi043.stdout: python3-pyOpenSSL-21.0.0-1.el9.noarch 2023-08-09T13:32:52.729 INFO:teuthology.orchestra.run.smithi043.stdout: python3-pyasn1-0.4.8-6.el9.noarch 2023-08-09T13:32:52.729 INFO:teuthology.orchestra.run.smithi043.stdout: python3-pyasn1-modules-0.4.8-6.el9.noarch 2023-08-09T13:32:52.729 INFO:teuthology.orchestra.run.smithi043.stdout: python3-pycparser-2.20-6.el9.noarch 2023-08-09T13:32:52.729 INFO:teuthology.orchestra.run.smithi043.stdout: python3-pytz-2021.1-5.el9.noarch 2023-08-09T13:32:52.729 INFO:teuthology.orchestra.run.smithi043.stdout: python3-rados-2:18.0.0-5389.g668a86f2.el9.x86_64 2023-08-09T13:32:52.729 INFO:teuthology.orchestra.run.smithi043.stdout: python3-rbd-2:18.0.0-5389.g668a86f2.el9.x86_64 2023-08-09T13:32:52.730 INFO:teuthology.orchestra.run.smithi043.stdout: python3-repoze-lru-0.7-16.el9.noarch 2023-08-09T13:32:52.730 INFO:teuthology.orchestra.run.smithi043.stdout: python3-requests-oauthlib-1.3.0-12.el9.noarch 2023-08-09T13:32:52.730 INFO:teuthology.orchestra.run.smithi043.stdout: python3-rgw-2:18.0.0-5389.g668a86f2.el9.x86_64 2023-08-09T13:32:52.730 INFO:teuthology.orchestra.run.smithi043.stdout: python3-routes-2.5.1-5.el9.noarch 2023-08-09T13:32:52.730 INFO:teuthology.orchestra.run.smithi043.stdout: python3-rsa-4.9-2.el9.noarch 2023-08-09T13:32:52.730 INFO:teuthology.orchestra.run.smithi043.stdout: python3-scipy-1.6.2-8.el9.x86_64 2023-08-09T13:32:52.730 INFO:teuthology.orchestra.run.smithi043.stdout: python3-tempora-5.0.0-2.el9.noarch 2023-08-09T13:32:52.731 INFO:teuthology.orchestra.run.smithi043.stdout: python3-toml-0.10.2-6.el9.noarch 2023-08-09T13:32:52.731 INFO:teuthology.orchestra.run.smithi043.stdout: python3-typing-extensions-4.4.0-2.el9.noarch 2023-08-09T13:32:52.731 INFO:teuthology.orchestra.run.smithi043.stdout: python3-webob-1.8.7-6.el9.noarch 2023-08-09T13:32:52.731 INFO:teuthology.orchestra.run.smithi043.stdout: python3-websocket-client-1.2.3-2.el9.noarch 2023-08-09T13:32:52.731 INFO:teuthology.orchestra.run.smithi043.stdout: python3-werkzeug-2.0.3-3.el9.1.noarch 2023-08-09T13:32:52.731 INFO:teuthology.orchestra.run.smithi043.stdout: python3-zc-lockfile-2.0-10.el9.noarch 2023-08-09T13:32:52.731 INFO:teuthology.orchestra.run.smithi043.stdout: rbd-fuse-2:18.0.0-5389.g668a86f2.el9.x86_64 2023-08-09T13:32:52.731 INFO:teuthology.orchestra.run.smithi043.stdout: rbd-mirror-2:18.0.0-5389.g668a86f2.el9.x86_64 2023-08-09T13:32:52.732 INFO:teuthology.orchestra.run.smithi043.stdout: rbd-nbd-2:18.0.0-5389.g668a86f2.el9.x86_64 2023-08-09T13:32:52.732 INFO:teuthology.orchestra.run.smithi043.stdout: re2-1:20211101-2.el9.x86_64 2023-08-09T13:32:52.732 INFO:teuthology.orchestra.run.smithi043.stdout: socat-1.7.4.1-5.el9.x86_64 2023-08-09T13:32:52.732 INFO:teuthology.orchestra.run.smithi043.stdout: thrift-0.15.0-2.el9.x86_64 2023-08-09T13:32:52.732 INFO:teuthology.orchestra.run.smithi043.stdout: uuid-1.6.2-55.el9.x86_64 2023-08-09T13:32:52.732 INFO:teuthology.orchestra.run.smithi043.stdout: xfsprogs-devel-5.19.0-4.el9.x86_64 2023-08-09T13:32:52.732 INFO:teuthology.orchestra.run.smithi043.stdout: xmlstarlet-1.6.1-20.el9.x86_64 2023-08-09T13:32:52.733 INFO:teuthology.orchestra.run.smithi043.stdout: zlib-devel-1.2.11-36.el9.x86_64 2023-08-09T13:32:52.733 INFO:teuthology.orchestra.run.smithi043.stdout: 2023-08-09T13:32:52.733 INFO:teuthology.orchestra.run.smithi043.stdout:Complete! 2023-08-09T13:32:53.290 DEBUG:teuthology.parallel:result is None 2023-08-09T13:32:54.018 INFO:teuthology.orchestra.run.smithi203.stdout: Verifying : xfsprogs-5.14.2-1.el9.x86_64 177/177 2023-08-09T13:32:54.019 INFO:teuthology.orchestra.run.smithi203.stdout: 2023-08-09T13:32:54.019 INFO:teuthology.orchestra.run.smithi203.stdout:Upgraded: 2023-08-09T13:32:54.019 INFO:teuthology.orchestra.run.smithi203.stdout: NetworkManager-1:1.43.90-1.el9.x86_64 2023-08-09T13:32:54.019 INFO:teuthology.orchestra.run.smithi203.stdout: NetworkManager-libnm-1:1.43.90-1.el9.x86_64 2023-08-09T13:32:54.019 INFO:teuthology.orchestra.run.smithi203.stdout: NetworkManager-team-1:1.43.90-1.el9.x86_64 2023-08-09T13:32:54.019 INFO:teuthology.orchestra.run.smithi203.stdout: NetworkManager-tui-1:1.43.90-1.el9.x86_64 2023-08-09T13:32:54.019 INFO:teuthology.orchestra.run.smithi203.stdout: elfutils-debuginfod-client-0.189-3.el9.x86_64 2023-08-09T13:32:54.020 INFO:teuthology.orchestra.run.smithi203.stdout: elfutils-libelf-0.189-3.el9.x86_64 2023-08-09T13:32:54.020 INFO:teuthology.orchestra.run.smithi203.stdout: elfutils-libs-0.189-3.el9.x86_64 2023-08-09T13:32:54.020 INFO:teuthology.orchestra.run.smithi203.stdout: iproute-6.2.0-5.el9.x86_64 2023-08-09T13:32:54.020 INFO:teuthology.orchestra.run.smithi203.stdout: iproute-tc-6.2.0-5.el9.x86_64 2023-08-09T13:32:54.020 INFO:teuthology.orchestra.run.smithi203.stdout: libblkid-2.37.4-11.el9.x86_64 2023-08-09T13:32:54.020 INFO:teuthology.orchestra.run.smithi203.stdout: libfdisk-2.37.4-11.el9.x86_64 2023-08-09T13:32:54.020 INFO:teuthology.orchestra.run.smithi203.stdout: libmount-2.37.4-11.el9.x86_64 2023-08-09T13:32:54.021 INFO:teuthology.orchestra.run.smithi203.stdout: librados2-2:18.0.0-5389.g668a86f2.el9.x86_64 2023-08-09T13:32:54.021 INFO:teuthology.orchestra.run.smithi203.stdout: librbd1-2:18.0.0-5389.g668a86f2.el9.x86_64 2023-08-09T13:32:54.021 INFO:teuthology.orchestra.run.smithi203.stdout: libsmartcols-2.37.4-11.el9.x86_64 2023-08-09T13:32:54.021 INFO:teuthology.orchestra.run.smithi203.stdout: libuuid-2.37.4-11.el9.x86_64 2023-08-09T13:32:54.021 INFO:teuthology.orchestra.run.smithi203.stdout: openssl-1:3.0.7-24.el9.x86_64 2023-08-09T13:32:54.021 INFO:teuthology.orchestra.run.smithi203.stdout: openssl-libs-1:3.0.7-24.el9.x86_64 2023-08-09T13:32:54.021 INFO:teuthology.orchestra.run.smithi203.stdout: util-linux-2.37.4-11.el9.x86_64 2023-08-09T13:32:54.021 INFO:teuthology.orchestra.run.smithi203.stdout: util-linux-core-2.37.4-11.el9.x86_64 2023-08-09T13:32:54.022 INFO:teuthology.orchestra.run.smithi203.stdout: util-linux-user-2.37.4-11.el9.x86_64 2023-08-09T13:32:54.022 INFO:teuthology.orchestra.run.smithi203.stdout: xfsdump-3.1.12-3.el9.x86_64 2023-08-09T13:32:54.022 INFO:teuthology.orchestra.run.smithi203.stdout: xfsprogs-5.19.0-4.el9.x86_64 2023-08-09T13:32:54.022 INFO:teuthology.orchestra.run.smithi203.stdout:Installed: 2023-08-09T13:32:54.022 INFO:teuthology.orchestra.run.smithi203.stdout: automake-1.16.2-8.el9.noarch 2023-08-09T13:32:54.022 INFO:teuthology.orchestra.run.smithi203.stdout: bison-3.7.4-5.el9.x86_64 2023-08-09T13:32:54.022 INFO:teuthology.orchestra.run.smithi203.stdout: boost-program-options-1.75.0-8.el9.x86_64 2023-08-09T13:32:54.023 INFO:teuthology.orchestra.run.smithi203.stdout: ceph-2:18.0.0-5389.g668a86f2.el9.x86_64 2023-08-09T13:32:54.023 INFO:teuthology.orchestra.run.smithi203.stdout: ceph-base-2:18.0.0-5389.g668a86f2.el9.x86_64 2023-08-09T13:32:54.023 INFO:teuthology.orchestra.run.smithi203.stdout: ceph-common-2:18.0.0-5389.g668a86f2.el9.x86_64 2023-08-09T13:32:54.023 INFO:teuthology.orchestra.run.smithi203.stdout: ceph-fuse-2:18.0.0-5389.g668a86f2.el9.x86_64 2023-08-09T13:32:54.023 INFO:teuthology.orchestra.run.smithi203.stdout: ceph-grafana-dashboards-2:18.0.0-5389.g668a86f2.el9.noarch 2023-08-09T13:32:54.023 INFO:teuthology.orchestra.run.smithi203.stdout: ceph-immutable-object-cache-2:18.0.0-5389.g668a86f2.el9.x86_64 2023-08-09T13:32:54.023 INFO:teuthology.orchestra.run.smithi203.stdout: ceph-mds-2:18.0.0-5389.g668a86f2.el9.x86_64 2023-08-09T13:32:54.023 INFO:teuthology.orchestra.run.smithi203.stdout: ceph-mgr-2:18.0.0-5389.g668a86f2.el9.x86_64 2023-08-09T13:32:54.024 INFO:teuthology.orchestra.run.smithi203.stdout: ceph-mgr-cephadm-2:18.0.0-5389.g668a86f2.el9.noarch 2023-08-09T13:32:54.024 INFO:teuthology.orchestra.run.smithi203.stdout: ceph-mgr-dashboard-2:18.0.0-5389.g668a86f2.el9.noarch 2023-08-09T13:32:54.024 INFO:teuthology.orchestra.run.smithi203.stdout: ceph-mgr-diskprediction-local-2:18.0.0-5389.g668a86f2.el9.noarch 2023-08-09T13:32:54.024 INFO:teuthology.orchestra.run.smithi203.stdout: ceph-mgr-modules-core-2:18.0.0-5389.g668a86f2.el9.noarch 2023-08-09T13:32:54.024 INFO:teuthology.orchestra.run.smithi203.stdout: ceph-mgr-rook-2:18.0.0-5389.g668a86f2.el9.noarch 2023-08-09T13:32:54.024 INFO:teuthology.orchestra.run.smithi203.stdout: ceph-mon-2:18.0.0-5389.g668a86f2.el9.x86_64 2023-08-09T13:32:54.024 INFO:teuthology.orchestra.run.smithi203.stdout: ceph-osd-2:18.0.0-5389.g668a86f2.el9.x86_64 2023-08-09T13:32:54.025 INFO:teuthology.orchestra.run.smithi203.stdout: ceph-prometheus-alerts-2:18.0.0-5389.g668a86f2.el9.noarch 2023-08-09T13:32:54.025 INFO:teuthology.orchestra.run.smithi203.stdout: ceph-radosgw-2:18.0.0-5389.g668a86f2.el9.x86_64 2023-08-09T13:32:54.025 INFO:teuthology.orchestra.run.smithi203.stdout: ceph-selinux-2:18.0.0-5389.g668a86f2.el9.x86_64 2023-08-09T13:32:54.025 INFO:teuthology.orchestra.run.smithi203.stdout: ceph-test-2:18.0.0-5389.g668a86f2.el9.x86_64 2023-08-09T13:32:54.025 INFO:teuthology.orchestra.run.smithi203.stdout: ceph-volume-2:18.0.0-5389.g668a86f2.el9.noarch 2023-08-09T13:32:54.025 INFO:teuthology.orchestra.run.smithi203.stdout: cephadm-2:18.0.0-5389.g668a86f2.el9.noarch 2023-08-09T13:32:54.025 INFO:teuthology.orchestra.run.smithi203.stdout: cephfs-mirror-2:18.0.0-5389.g668a86f2.el9.x86_64 2023-08-09T13:32:54.026 INFO:teuthology.orchestra.run.smithi203.stdout: cephfs-top-2:18.0.0-5389.g668a86f2.el9.noarch 2023-08-09T13:32:54.026 INFO:teuthology.orchestra.run.smithi203.stdout: elfutils-libelf-devel-0.189-3.el9.x86_64 2023-08-09T13:32:54.026 INFO:teuthology.orchestra.run.smithi203.stdout: flex-2.6.4-9.el9.x86_64 2023-08-09T13:32:54.026 INFO:teuthology.orchestra.run.smithi203.stdout: flexiblas-3.0.4-8.el9.x86_64 2023-08-09T13:32:54.026 INFO:teuthology.orchestra.run.smithi203.stdout: flexiblas-netlib-3.0.4-8.el9.x86_64 2023-08-09T13:32:54.026 INFO:teuthology.orchestra.run.smithi203.stdout: flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 2023-08-09T13:32:54.026 INFO:teuthology.orchestra.run.smithi203.stdout: fmt-8.1.1-5.el9.x86_64 2023-08-09T13:32:54.026 INFO:teuthology.orchestra.run.smithi203.stdout: fuse-2.9.9-15.el9.x86_64 2023-08-09T13:32:54.027 INFO:teuthology.orchestra.run.smithi203.stdout: gperftools-libs-2.9.1-2.el9.x86_64 2023-08-09T13:32:54.027 INFO:teuthology.orchestra.run.smithi203.stdout: jq-1.6-15.el9.x86_64 2023-08-09T13:32:54.027 INFO:teuthology.orchestra.run.smithi203.stdout: libaio-devel-0.3.111-13.el9.x86_64 2023-08-09T13:32:54.027 INFO:teuthology.orchestra.run.smithi203.stdout: libarrow-9.0.0-9.el9.x86_64 2023-08-09T13:32:54.027 INFO:teuthology.orchestra.run.smithi203.stdout: libarrow-doc-9.0.0-9.el9.noarch 2023-08-09T13:32:54.027 INFO:teuthology.orchestra.run.smithi203.stdout: libcephfs-devel-2:18.0.0-5389.g668a86f2.el9.x86_64 2023-08-09T13:32:54.027 INFO:teuthology.orchestra.run.smithi203.stdout: libcephfs2-2:18.0.0-5389.g668a86f2.el9.x86_64 2023-08-09T13:32:54.027 INFO:teuthology.orchestra.run.smithi203.stdout: libcephsqlite-2:18.0.0-5389.g668a86f2.el9.x86_64 2023-08-09T13:32:54.028 INFO:teuthology.orchestra.run.smithi203.stdout: libgfortran-11.4.1-2.1.el9.x86_64 2023-08-09T13:32:54.028 INFO:teuthology.orchestra.run.smithi203.stdout: liboath-2.6.7-2.el9.x86_64 2023-08-09T13:32:54.028 INFO:teuthology.orchestra.run.smithi203.stdout: libpmemobj-1.12.1-1.el9.x86_64 2023-08-09T13:32:54.028 INFO:teuthology.orchestra.run.smithi203.stdout: libquadmath-11.4.1-2.1.el9.x86_64 2023-08-09T13:32:54.028 INFO:teuthology.orchestra.run.smithi203.stdout: librabbitmq-0.11.0-7.el9.x86_64 2023-08-09T13:32:54.028 INFO:teuthology.orchestra.run.smithi203.stdout: librados-devel-2:18.0.0-5389.g668a86f2.el9.x86_64 2023-08-09T13:32:54.028 INFO:teuthology.orchestra.run.smithi203.stdout: libradosstriper1-2:18.0.0-5389.g668a86f2.el9.x86_64 2023-08-09T13:32:54.029 INFO:teuthology.orchestra.run.smithi203.stdout: librdkafka-1.6.1-102.el9.x86_64 2023-08-09T13:32:54.029 INFO:teuthology.orchestra.run.smithi203.stdout: librgw2-2:18.0.0-5389.g668a86f2.el9.x86_64 2023-08-09T13:32:54.029 INFO:teuthology.orchestra.run.smithi203.stdout: libtool-2.4.6-45.el9.x86_64 2023-08-09T13:32:54.029 INFO:teuthology.orchestra.run.smithi203.stdout: libunwind-1.6.2-1.el9.x86_64 2023-08-09T13:32:54.029 INFO:teuthology.orchestra.run.smithi203.stdout: libuuid-devel-2.37.4-11.el9.x86_64 2023-08-09T13:32:54.029 INFO:teuthology.orchestra.run.smithi203.stdout: libzstd-devel-1.5.1-2.el9.x86_64 2023-08-09T13:32:54.029 INFO:teuthology.orchestra.run.smithi203.stdout: lttng-ust-2.12.0-6.el9.x86_64 2023-08-09T13:32:54.029 INFO:teuthology.orchestra.run.smithi203.stdout: oniguruma-6.9.6-1.el9.5.x86_64 2023-08-09T13:32:54.030 INFO:teuthology.orchestra.run.smithi203.stdout: openblas-0.3.21-2.el9.x86_64 2023-08-09T13:32:54.030 INFO:teuthology.orchestra.run.smithi203.stdout: openblas-openmp-0.3.21-2.el9.x86_64 2023-08-09T13:32:54.030 INFO:teuthology.orchestra.run.smithi203.stdout: openssl-devel-1:3.0.7-24.el9.x86_64 2023-08-09T13:32:54.030 INFO:teuthology.orchestra.run.smithi203.stdout: parquet-libs-9.0.0-9.el9.x86_64 2023-08-09T13:32:54.030 INFO:teuthology.orchestra.run.smithi203.stdout: postgresql-13.11-1.el9.x86_64 2023-08-09T13:32:54.030 INFO:teuthology.orchestra.run.smithi203.stdout: postgresql-contrib-13.11-1.el9.x86_64 2023-08-09T13:32:54.030 INFO:teuthology.orchestra.run.smithi203.stdout: postgresql-private-libs-13.11-1.el9.x86_64 2023-08-09T13:32:54.030 INFO:teuthology.orchestra.run.smithi203.stdout: postgresql-server-13.11-1.el9.x86_64 2023-08-09T13:32:54.031 INFO:teuthology.orchestra.run.smithi203.stdout: python3-asyncssh-2.13.2-3.el9.noarch 2023-08-09T13:32:54.031 INFO:teuthology.orchestra.run.smithi203.stdout: python3-babel-2.9.1-2.el9.noarch 2023-08-09T13:32:54.031 INFO:teuthology.orchestra.run.smithi203.stdout: python3-bcrypt-3.2.2-1.el9.x86_64 2023-08-09T13:32:54.031 INFO:teuthology.orchestra.run.smithi203.stdout: python3-cachetools-4.2.4-1.el9.noarch 2023-08-09T13:32:54.031 INFO:teuthology.orchestra.run.smithi203.stdout: python3-ceph-argparse-2:18.0.0-5389.g668a86f2.el9.x86_64 2023-08-09T13:32:54.031 INFO:teuthology.orchestra.run.smithi203.stdout: python3-ceph-common-2:18.0.0-5389.g668a86f2.el9.x86_64 2023-08-09T13:32:54.031 INFO:teuthology.orchestra.run.smithi203.stdout: python3-cephfs-2:18.0.0-5389.g668a86f2.el9.x86_64 2023-08-09T13:32:54.031 INFO:teuthology.orchestra.run.smithi203.stdout: python3-certifi-2021.10.8-2.el9.noarch 2023-08-09T13:32:54.032 INFO:teuthology.orchestra.run.smithi203.stdout: python3-cffi-1.14.5-5.el9.x86_64 2023-08-09T13:32:54.032 INFO:teuthology.orchestra.run.smithi203.stdout: python3-cheroot-8.6.0-4.el9.noarch 2023-08-09T13:32:54.032 INFO:teuthology.orchestra.run.smithi203.stdout: python3-cherrypy-18.6.1-2.el9.noarch 2023-08-09T13:32:54.032 INFO:teuthology.orchestra.run.smithi203.stdout: python3-cryptography-36.0.1-4.el9.x86_64 2023-08-09T13:32:54.032 INFO:teuthology.orchestra.run.smithi203.stdout: python3-devel-3.9.16-1.el9.x86_64 2023-08-09T13:32:54.032 INFO:teuthology.orchestra.run.smithi203.stdout: python3-google-auth-1:2.22.0-1.el9.noarch 2023-08-09T13:32:54.033 INFO:teuthology.orchestra.run.smithi203.stdout: python3-jaraco-8.2.1-3.el9.noarch 2023-08-09T13:32:54.033 INFO:teuthology.orchestra.run.smithi203.stdout: python3-jaraco-classes-3.2.1-5.el9.noarch 2023-08-09T13:32:54.033 INFO:teuthology.orchestra.run.smithi203.stdout: python3-jaraco-collections-3.0.0-8.el9.noarch 2023-08-09T13:32:54.033 INFO:teuthology.orchestra.run.smithi203.stdout: python3-jaraco-functools-3.5.0-2.el9.noarch 2023-08-09T13:32:54.033 INFO:teuthology.orchestra.run.smithi203.stdout: python3-jaraco-text-3.2.0-6.el9.noarch 2023-08-09T13:32:54.033 INFO:teuthology.orchestra.run.smithi203.stdout: python3-jinja2-2.11.3-4.el9.noarch 2023-08-09T13:32:54.033 INFO:teuthology.orchestra.run.smithi203.stdout: python3-jsonpatch-1.21-16.el9.noarch 2023-08-09T13:32:54.033 INFO:teuthology.orchestra.run.smithi203.stdout: python3-jsonpointer-2.0-4.el9.noarch 2023-08-09T13:32:54.034 INFO:teuthology.orchestra.run.smithi203.stdout: python3-jwt-2.4.0-1.el9.noarch 2023-08-09T13:32:54.034 INFO:teuthology.orchestra.run.smithi203.stdout: python3-jwt+crypto-2.4.0-1.el9.noarch 2023-08-09T13:32:54.034 INFO:teuthology.orchestra.run.smithi203.stdout: python3-kubernetes-1:26.1.0-2.el9.noarch 2023-08-09T13:32:54.034 INFO:teuthology.orchestra.run.smithi203.stdout: python3-logutils-0.3.5-21.el9.noarch 2023-08-09T13:32:54.034 INFO:teuthology.orchestra.run.smithi203.stdout: python3-mako-1.1.4-6.el9.noarch 2023-08-09T13:32:54.034 INFO:teuthology.orchestra.run.smithi203.stdout: python3-markupsafe-1.1.1-12.el9.x86_64 2023-08-09T13:32:54.034 INFO:teuthology.orchestra.run.smithi203.stdout: python3-more-itertools-8.12.0-2.el9.noarch 2023-08-09T13:32:54.034 INFO:teuthology.orchestra.run.smithi203.stdout: python3-natsort-7.1.1-5.el9.noarch 2023-08-09T13:32:54.035 INFO:teuthology.orchestra.run.smithi203.stdout: python3-numpy-1:1.20.1-5.el9.x86_64 2023-08-09T13:32:54.035 INFO:teuthology.orchestra.run.smithi203.stdout: python3-numpy-f2py-1:1.20.1-5.el9.x86_64 2023-08-09T13:32:54.035 INFO:teuthology.orchestra.run.smithi203.stdout: python3-oauthlib-3.1.1-5.el9.noarch 2023-08-09T13:32:54.035 INFO:teuthology.orchestra.run.smithi203.stdout: python3-pecan-1.4.2-3.el9.noarch 2023-08-09T13:32:54.035 INFO:teuthology.orchestra.run.smithi203.stdout: python3-ply-3.11-14.el9.noarch 2023-08-09T13:32:54.035 INFO:teuthology.orchestra.run.smithi203.stdout: python3-portend-3.1.0-2.el9.noarch 2023-08-09T13:32:54.035 INFO:teuthology.orchestra.run.smithi203.stdout: python3-prettytable-0.7.2-27.el9.noarch 2023-08-09T13:32:54.035 INFO:teuthology.orchestra.run.smithi203.stdout: python3-pyOpenSSL-21.0.0-1.el9.noarch 2023-08-09T13:32:54.036 INFO:teuthology.orchestra.run.smithi203.stdout: python3-pyasn1-0.4.8-6.el9.noarch 2023-08-09T13:32:54.036 INFO:teuthology.orchestra.run.smithi203.stdout: python3-pyasn1-modules-0.4.8-6.el9.noarch 2023-08-09T13:32:54.036 INFO:teuthology.orchestra.run.smithi203.stdout: python3-pycparser-2.20-6.el9.noarch 2023-08-09T13:32:54.036 INFO:teuthology.orchestra.run.smithi203.stdout: python3-pytz-2021.1-5.el9.noarch 2023-08-09T13:32:54.036 INFO:teuthology.orchestra.run.smithi203.stdout: python3-rados-2:18.0.0-5389.g668a86f2.el9.x86_64 2023-08-09T13:32:54.036 INFO:teuthology.orchestra.run.smithi203.stdout: python3-rbd-2:18.0.0-5389.g668a86f2.el9.x86_64 2023-08-09T13:32:54.036 INFO:teuthology.orchestra.run.smithi203.stdout: python3-repoze-lru-0.7-16.el9.noarch 2023-08-09T13:32:54.037 INFO:teuthology.orchestra.run.smithi203.stdout: python3-requests-oauthlib-1.3.0-12.el9.noarch 2023-08-09T13:32:54.037 INFO:teuthology.orchestra.run.smithi203.stdout: python3-rgw-2:18.0.0-5389.g668a86f2.el9.x86_64 2023-08-09T13:32:54.037 INFO:teuthology.orchestra.run.smithi203.stdout: python3-routes-2.5.1-5.el9.noarch 2023-08-09T13:32:54.037 INFO:teuthology.orchestra.run.smithi203.stdout: python3-rsa-4.9-2.el9.noarch 2023-08-09T13:32:54.037 INFO:teuthology.orchestra.run.smithi203.stdout: python3-scipy-1.6.2-8.el9.x86_64 2023-08-09T13:32:54.037 INFO:teuthology.orchestra.run.smithi203.stdout: python3-tempora-5.0.0-2.el9.noarch 2023-08-09T13:32:54.037 INFO:teuthology.orchestra.run.smithi203.stdout: python3-toml-0.10.2-6.el9.noarch 2023-08-09T13:32:54.037 INFO:teuthology.orchestra.run.smithi203.stdout: python3-typing-extensions-4.4.0-2.el9.noarch 2023-08-09T13:32:54.038 INFO:teuthology.orchestra.run.smithi203.stdout: python3-webob-1.8.7-6.el9.noarch 2023-08-09T13:32:54.038 INFO:teuthology.orchestra.run.smithi203.stdout: python3-websocket-client-1.2.3-2.el9.noarch 2023-08-09T13:32:54.038 INFO:teuthology.orchestra.run.smithi203.stdout: python3-werkzeug-2.0.3-3.el9.1.noarch 2023-08-09T13:32:54.038 INFO:teuthology.orchestra.run.smithi203.stdout: python3-zc-lockfile-2.0-10.el9.noarch 2023-08-09T13:32:54.038 INFO:teuthology.orchestra.run.smithi203.stdout: rbd-fuse-2:18.0.0-5389.g668a86f2.el9.x86_64 2023-08-09T13:32:54.038 INFO:teuthology.orchestra.run.smithi203.stdout: rbd-mirror-2:18.0.0-5389.g668a86f2.el9.x86_64 2023-08-09T13:32:54.038 INFO:teuthology.orchestra.run.smithi203.stdout: rbd-nbd-2:18.0.0-5389.g668a86f2.el9.x86_64 2023-08-09T13:32:54.038 INFO:teuthology.orchestra.run.smithi203.stdout: re2-1:20211101-2.el9.x86_64 2023-08-09T13:32:54.039 INFO:teuthology.orchestra.run.smithi203.stdout: socat-1.7.4.1-5.el9.x86_64 2023-08-09T13:32:54.039 INFO:teuthology.orchestra.run.smithi203.stdout: thrift-0.15.0-2.el9.x86_64 2023-08-09T13:32:54.039 INFO:teuthology.orchestra.run.smithi203.stdout: uuid-1.6.2-55.el9.x86_64 2023-08-09T13:32:54.039 INFO:teuthology.orchestra.run.smithi203.stdout: xfsprogs-devel-5.19.0-4.el9.x86_64 2023-08-09T13:32:54.039 INFO:teuthology.orchestra.run.smithi203.stdout: xmlstarlet-1.6.1-20.el9.x86_64 2023-08-09T13:32:54.040 INFO:teuthology.orchestra.run.smithi203.stdout: zlib-devel-1.2.11-36.el9.x86_64 2023-08-09T13:32:54.040 INFO:teuthology.orchestra.run.smithi203.stdout: 2023-08-09T13:32:54.040 INFO:teuthology.orchestra.run.smithi203.stdout:Complete! 2023-08-09T13:32:54.691 DEBUG:teuthology.parallel:result is None 2023-08-09T13:32:54.692 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=668a86f232c9454b2ce5928d1432a29a72964d85 2023-08-09T13:32:54.901 DEBUG:teuthology.orchestra.run.smithi043:> rpm -q ceph --qf '%{VERSION}-%{RELEASE}' 2023-08-09T13:32:54.960 INFO:teuthology.orchestra.run.smithi043.stdout:18.0.0-5389.g668a86f2.el9 2023-08-09T13:32:54.961 INFO:teuthology.packaging:The installed version of ceph is 18.0.0-5389.g668a86f2.el9 2023-08-09T13:32:54.961 INFO:teuthology.task.install:The correct ceph version 18.0.0-5389.g668a86f2 is installed. 2023-08-09T13:32:54.962 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=668a86f232c9454b2ce5928d1432a29a72964d85 2023-08-09T13:32:55.170 DEBUG:teuthology.orchestra.run.smithi203:> rpm -q ceph --qf '%{VERSION}-%{RELEASE}' 2023-08-09T13:32:55.234 INFO:teuthology.orchestra.run.smithi203.stdout:18.0.0-5389.g668a86f2.el9 2023-08-09T13:32:55.234 INFO:teuthology.packaging:The installed version of ceph is 18.0.0-5389.g668a86f2.el9 2023-08-09T13:32:55.234 INFO:teuthology.task.install:The correct ceph version 18.0.0-5389.g668a86f2 is installed. 2023-08-09T13:32:55.235 INFO:teuthology.task.install.util:Shipping valgrind.supp... 2023-08-09T13:32:55.236 DEBUG:teuthology.orchestra.run.smithi043:> set -ex 2023-08-09T13:32:55.237 DEBUG:teuthology.orchestra.run.smithi043:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2023-08-09T13:32:55.325 DEBUG:teuthology.orchestra.run.smithi203:> set -ex 2023-08-09T13:32:55.326 DEBUG:teuthology.orchestra.run.smithi203:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2023-08-09T13:32:55.408 INFO:teuthology.task.install.util:Shipping 'daemon-helper'... 2023-08-09T13:32:55.408 DEBUG:teuthology.orchestra.run.smithi043:> set -ex 2023-08-09T13:32:55.409 DEBUG:teuthology.orchestra.run.smithi043:> sudo dd of=/usr/bin/daemon-helper 2023-08-09T13:32:55.518 DEBUG:teuthology.orchestra.run.smithi043:> sudo chmod a=rx -- /usr/bin/daemon-helper 2023-08-09T13:32:55.624 DEBUG:teuthology.orchestra.run.smithi203:> set -ex 2023-08-09T13:32:55.624 DEBUG:teuthology.orchestra.run.smithi203:> sudo dd of=/usr/bin/daemon-helper 2023-08-09T13:32:55.704 DEBUG:teuthology.orchestra.run.smithi203:> sudo chmod a=rx -- /usr/bin/daemon-helper 2023-08-09T13:32:55.818 INFO:teuthology.task.install.util:Shipping 'adjust-ulimits'... 2023-08-09T13:32:55.818 DEBUG:teuthology.orchestra.run.smithi043:> set -ex 2023-08-09T13:32:55.818 DEBUG:teuthology.orchestra.run.smithi043:> sudo dd of=/usr/bin/adjust-ulimits 2023-08-09T13:32:55.905 DEBUG:teuthology.orchestra.run.smithi043:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2023-08-09T13:32:56.023 DEBUG:teuthology.orchestra.run.smithi203:> set -ex 2023-08-09T13:32:56.024 DEBUG:teuthology.orchestra.run.smithi203:> sudo dd of=/usr/bin/adjust-ulimits 2023-08-09T13:32:56.104 DEBUG:teuthology.orchestra.run.smithi203:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2023-08-09T13:32:56.219 INFO:teuthology.task.install.util:Shipping 'stdin-killer'... 2023-08-09T13:32:56.219 DEBUG:teuthology.orchestra.run.smithi043:> set -ex 2023-08-09T13:32:56.220 DEBUG:teuthology.orchestra.run.smithi043:> sudo dd of=/usr/bin/stdin-killer 2023-08-09T13:32:56.313 DEBUG:teuthology.orchestra.run.smithi043:> sudo chmod a=rx -- /usr/bin/stdin-killer 2023-08-09T13:32:56.430 DEBUG:teuthology.orchestra.run.smithi203:> set -ex 2023-08-09T13:32:56.431 DEBUG:teuthology.orchestra.run.smithi203:> sudo dd of=/usr/bin/stdin-killer 2023-08-09T13:32:56.516 DEBUG:teuthology.orchestra.run.smithi203:> sudo chmod a=rx -- /usr/bin/stdin-killer 2023-08-09T13:32:56.635 INFO:teuthology.run_tasks:Running task ceph... 2023-08-09T13:32:56.707 INFO:tasks.ceph:Making ceph log dir writeable by non-root... 2023-08-09T13:32:56.708 DEBUG:teuthology.orchestra.run.smithi043:> sudo chmod 777 /var/log/ceph 2023-08-09T13:32:56.713 DEBUG:teuthology.orchestra.run.smithi203:> sudo chmod 777 /var/log/ceph 2023-08-09T13:32:56.793 INFO:tasks.ceph:Disabling ceph logrotate... 2023-08-09T13:32:56.793 DEBUG:teuthology.orchestra.run.smithi043:> sudo rm -f -- /etc/logrotate.d/ceph 2023-08-09T13:32:56.799 DEBUG:teuthology.orchestra.run.smithi203:> sudo rm -f -- /etc/logrotate.d/ceph 2023-08-09T13:32:56.873 INFO:tasks.ceph:Creating extra log directories... 2023-08-09T13:32:56.874 DEBUG:teuthology.orchestra.run.smithi043:> sudo install -d -m0777 -- /var/log/ceph/valgrind /var/log/ceph/profiling-logger 2023-08-09T13:32:56.879 DEBUG:teuthology.orchestra.run.smithi203:> sudo install -d -m0777 -- /var/log/ceph/valgrind /var/log/ceph/profiling-logger 2023-08-09T13:32:56.963 INFO:tasks.ceph:Setting up log rotation with {'ceph-mds': '10G', 'ceph-osd': '10G'} 2023-08-09T13:32:56.964 INFO:tasks.ceph:writing logrotate stanza for ceph-mds 2023-08-09T13:32:56.964 INFO:tasks.ceph:writing logrotate stanza for ceph-osd 2023-08-09T13:32:56.964 DEBUG:teuthology.orchestra.run.smithi043:> set -ex 2023-08-09T13:32:56.965 DEBUG:teuthology.orchestra.run.smithi043:> dd of=/home/ubuntu/cephtest/logrotate.ceph-test.conf 2023-08-09T13:32:57.006 DEBUG:teuthology.orchestra.run.smithi043:> sudo mv /home/ubuntu/cephtest/logrotate.ceph-test.conf /etc/logrotate.d/ceph-test.conf && sudo chmod 0644 /etc/logrotate.d/ceph-test.conf && sudo chown root.root /etc/logrotate.d/ceph-test.conf 2023-08-09T13:32:57.246 DEBUG:teuthology.orchestra.run.smithi043:> sudo chcon system_u:object_r:etc_t:s0 /etc/logrotate.d/ceph-test.conf 2023-08-09T13:32:57.336 DEBUG:teuthology.orchestra.run.smithi203:> set -ex 2023-08-09T13:32:57.337 DEBUG:teuthology.orchestra.run.smithi203:> dd of=/home/ubuntu/cephtest/logrotate.ceph-test.conf 2023-08-09T13:32:57.380 DEBUG:teuthology.orchestra.run.smithi203:> sudo mv /home/ubuntu/cephtest/logrotate.ceph-test.conf /etc/logrotate.d/ceph-test.conf && sudo chmod 0644 /etc/logrotate.d/ceph-test.conf && sudo chown root.root /etc/logrotate.d/ceph-test.conf 2023-08-09T13:32:57.596 DEBUG:teuthology.orchestra.run.smithi203:> sudo chcon system_u:object_r:etc_t:s0 /etc/logrotate.d/ceph-test.conf 2023-08-09T13:32:57.672 INFO:tasks.ceph:Creating ceph cluster ceph... 2023-08-09T13:32:57.672 INFO:tasks.ceph:config {'conf': {'client': {'client mount timeout': 600, 'debug client': 20, 'debug ms': 1, 'pid file': '', 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'mds': {'debug mds': 20, 'debug mds balancer': 20, 'debug ms': 1, 'mds debug frag': True, 'mds debug scatterstat': True, 'mds op complaint time': 180, 'mds verify scatter': True, 'osd op complaint time': 180, 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'mgr': {'debug client': 20, 'debug finisher': 20, 'debug mgr': 20, 'debug ms': 1}, 'mon': {'debug mon': 20, 'debug ms': 1, 'debug paxos': 20, 'mon op complaint time': 120}, 'osd': {'bluestore block size': 96636764160, 'bluestore fsck on mount': True, 'debug bluefs': 20, 'debug bluestore': 20, 'debug ms': 1, 'debug osd': 20, 'debug rocksdb': 10, 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd failsafe full ratio': 0.95, 'osd objectstore': 'bluestore', 'osd op complaint time': 180}}, 'fs': 'xfs', 'mkfs_options': None, 'mount_options': None, 'skip_mgr_daemons': False, 'log_ignorelist': ['\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', 'overall HEALTH_', '\\(FS_DEGRADED\\)', '\\(MDS_FAILED\\)', '\\(MDS_DEGRADED\\)', '\\(FS_WITH_FAILED_MDS\\)', '\\(MDS_DAMAGE\\)', '\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', '\\(FS_INLINE_DATA_DEPRECATED\\)', 'overall HEALTH_', '\\(OSD_DOWN\\)', '\\(OSD_', 'but it is still running', 'is not responding'], 'cpu_profile': set(), 'cluster': 'ceph', 'mon_bind_msgr2': True, 'mon_bind_addrvec': True} 2023-08-09T13:32:57.672 INFO:tasks.ceph:ctx.config {'archive_path': '/home/teuthworker/archive/vshankar-2023-08-09_05:46:29-fs-wip-vshankar-testing-20230809.035933-testing-default-smithi/7364139', 'branch': 'wip-vshankar-testing-20230809.035933', 'description': 'fs/functional/{begin/{0-install 1-ceph 2-logrotate} clusters/1a3s-mds-4c-client conf/{client mds mon osd} distro/{centos_latest} mount/fuse objectstore/bluestore-ec-root overrides/{ignorelist_health ignorelist_wrongly_marked_down no_client_pidfile} tasks/snap-schedule}', 'email': 'vshankar@redhat.com', 'first_in_suite': False, 'job_id': '7364139', 'kernel': {'flavor': 'default', 'kdb': True, 'sha1': 'a7fb1265323db972dd333f71b9a53e9479f62e37'}, 'last_in_suite': False, 'log-rotate': {'ceph-mds': '10G', 'ceph-osd': '10G'}, 'machine_type': 'smithi', 'name': 'vshankar-2023-08-09_05:46:29-fs-wip-vshankar-testing-20230809.035933-testing-default-smithi', 'no_nested_subset': False, 'nuke-on-error': True, 'openstack': [{'volumes': {'count': 4, 'size': 20}}, {'machine': {'disk': 200}}], 'os_type': 'centos', 'os_version': '9.stream', 'overrides': {'admin_socket': {'branch': 'wip-vshankar-testing-20230809.035933'}, 'ceph': {'cephfs': {'ec_profile': ['m=2', 'k=2', 'crush-failure-domain=osd']}, 'conf': {'client': {'client mount timeout': 600, 'debug client': 20, 'debug ms': 1, 'pid file': '', 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'mds': {'debug mds': 20, 'debug mds balancer': 20, 'debug ms': 1, 'mds debug frag': True, 'mds debug scatterstat': True, 'mds op complaint time': 180, 'mds verify scatter': True, 'osd op complaint time': 180, 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'mgr': {'debug client': 20, 'debug finisher': 20, 'debug mgr': 20, 'debug ms': 1}, 'mon': {'debug mon': 20, 'debug ms': 1, 'debug paxos': 20, 'mon op complaint time': 120}, 'osd': {'bluestore block size': 96636764160, 'bluestore fsck on mount': True, 'debug bluefs': 20, 'debug bluestore': 20, 'debug ms': 1, 'debug osd': 20, 'debug rocksdb': 10, 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd failsafe full ratio': 0.95, 'osd objectstore': 'bluestore', 'osd op complaint time': 180}}, 'flavor': 'default', 'fs': 'xfs', 'log-ignorelist': ['\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', 'overall HEALTH_', '\\(FS_DEGRADED\\)', '\\(MDS_FAILED\\)', '\\(MDS_DEGRADED\\)', '\\(FS_WITH_FAILED_MDS\\)', '\\(MDS_DAMAGE\\)', '\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', '\\(FS_INLINE_DATA_DEPRECATED\\)', 'overall HEALTH_', '\\(OSD_DOWN\\)', '\\(OSD_', 'but it is still running', 'is not responding'], 'log-whitelist': ['OSD full dropping all updates', 'OSD near full', 'pausewr flag', 'failsafe engaged, dropping updates', 'failsafe disengaged, no longer dropping', 'is full \\(reached quota', 'POOL_FULL', 'POOL_BACKFILLFULL'], 'sha1': '668a86f232c9454b2ce5928d1432a29a72964d85'}, 'ceph-deploy': {'bluestore': True, 'conf': {'client': {'log file': '/var/log/ceph/ceph-$name.$pid.log'}, 'mon': {'osd default pool size': 2}, 'osd': {'bluestore block size': 96636764160, 'bluestore fsck on mount': True, 'debug bluefs': 20, 'debug bluestore': 20, 'debug rocksdb': 10, 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd failsafe full ratio': 0.95, 'osd objectstore': 'bluestore'}}, 'fs': 'xfs'}, 'install': {'ceph': {'flavor': 'default', 'sha1': '668a86f232c9454b2ce5928d1432a29a72964d85'}}, 'thrashosds': {'bdev_inject_crash': 2, 'bdev_inject_crash_probability': 0.5}, 'workunit': {'branch': 'wip-vshankar-testing-20230809.035933', 'sha1': '668a86f232c9454b2ce5928d1432a29a72964d85'}}, 'owner': 'scheduled_vshankar@teuthology', 'priority': 80, 'repo': 'https://git.ceph.com/ceph-ci.git', 'roles': [['mon.a', 'mgr.y', 'mds.a', 'mds.c', 'osd.0', 'osd.1', 'osd.2', 'osd.3', 'client.0', 'client.2'], ['mon.b', 'mon.c', 'mgr.x', 'mds.b', 'mds.d', 'osd.4', 'osd.5', 'osd.6', 'osd.7', 'client.1', 'client.3']], 'seed': 8976, 'sha1': '668a86f232c9454b2ce5928d1432a29a72964d85', 'sleep_before_teardown': 0, 'subset': '486/1024', 'suite': 'fs', 'suite_branch': 'wip-vshankar-testing-20230809.035933', 'suite_path': '/home/teuthworker/src/git.ceph.com_ceph-c_668a86f232c9454b2ce5928d1432a29a72964d85/qa', 'suite_relpath': 'qa', 'suite_repo': 'https://git.ceph.com/ceph-ci.git', 'suite_sha1': '668a86f232c9454b2ce5928d1432a29a72964d85', 'syslog': {'ignorelist': ['WARNING*.*check_session_state', 'WARNING*.*__ceph_remove_cap']}, 'targets': {'smithi043.front.sepia.ceph.com': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBEKktPs1zHDi/KNgutOyVqOWoAarBLxJwqxEBET4kyZL78XfuBO2Mt+SP5ieccwraLOmO+jlznyREbzKXB1iudI=', 'smithi203.front.sepia.ceph.com': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBD/W9aB9RM/Yz46ZrKxLIqCF48iawl+xUQvYLoz3OmwYWs/kun9L9UcFcFyB+qwqiBQJZebgK5RojyEKSWt07Cw='}, 'tasks': [{'internal.check_packages': None}, {'internal.buildpackages_prep': None}, {'internal.save_config': None}, {'internal.check_lock': None}, {'internal.add_remotes': None}, {'console_log': None}, {'internal.connect': None}, {'internal.push_inventory': None}, {'internal.serialize_remote_roles': None}, {'internal.check_conflict': None}, {'internal.check_ceph_data': None}, {'internal.vm_setup': None}, {'kernel': {'flavor': 'default', 'kdb': True, 'sha1': 'a7fb1265323db972dd333f71b9a53e9479f62e37'}}, {'internal.base': None}, {'internal.archive_upload': None}, {'internal.archive': None}, {'internal.coredump': None}, {'internal.sudo': None}, {'internal.syslog': None}, {'internal.timer': None}, {'pcp': None}, {'selinux': None}, {'ansible.cephlab': None}, {'clock': None}, {'install': {'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'flavor': 'default', 'sha1': '668a86f232c9454b2ce5928d1432a29a72964d85'}}, {'ceph': None}, {'ceph-fuse': None}, {'cephfs_test_runner': {'modules': ['tasks.cephfs.test_snap_schedules']}}], 'teuthology': {'fragments_dropped': [], 'meta': {}, 'postmerge': ['local function is_kupstream() return false end', 'local function is_kdistro() return false end', 'local function is_fuse() return true end', "local function syntax_version() return '' end"]}, 'teuthology_branch': 'main', 'teuthology_sha1': '7fda95956ac10132c9b74016ba832db907df09fa', 'timestamp': '2023-08-09_05:46:29', 'tube': 'smithi', 'user': 'vshankar', 'verbose': False, 'worker_log': '/home/teuthworker/archive/worker_logs/dispatcher.smithi.2224'} 2023-08-09T13:32:57.673 DEBUG:teuthology.orchestra.run.smithi043:> install -d -m0755 -- /home/ubuntu/cephtest/ceph.data 2023-08-09T13:32:57.679 DEBUG:teuthology.orchestra.run.smithi203:> install -d -m0755 -- /home/ubuntu/cephtest/ceph.data 2023-08-09T13:32:57.720 DEBUG:teuthology.orchestra.run.smithi043:> sudo install -d -m0777 -- /var/run/ceph 2023-08-09T13:32:57.766 DEBUG:teuthology.orchestra.run.smithi203:> sudo install -d -m0777 -- /var/run/ceph 2023-08-09T13:32:57.842 DEBUG:teuthology.orchestra.run.smithi043:> set -ex 2023-08-09T13:32:57.842 DEBUG:teuthology.orchestra.run.smithi043:> dd if=/scratch_devs of=/dev/stdout 2023-08-09T13:32:57.884 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2023-08-09T13:32:57.885 DEBUG:teuthology.orchestra.run.smithi043:> stat /dev/vg_nvme/lv_1 2023-08-09T13:32:57.965 INFO:teuthology.orchestra.run.smithi043.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2023-08-09T13:32:57.965 INFO:teuthology.orchestra.run.smithi043.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2023-08-09T13:32:57.966 INFO:teuthology.orchestra.run.smithi043.stdout:Device: 5h/5d Inode: 622 Links: 1 2023-08-09T13:32:57.966 INFO:teuthology.orchestra.run.smithi043.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2023-08-09T13:32:57.966 INFO:teuthology.orchestra.run.smithi043.stdout:Context: system_u:object_r:device_t:s0 2023-08-09T13:32:57.966 INFO:teuthology.orchestra.run.smithi043.stdout:Access: 2023-08-09 13:32:46.284832995 +0000 2023-08-09T13:32:57.966 INFO:teuthology.orchestra.run.smithi043.stdout:Modify: 2023-08-09 13:29:59.278855934 +0000 2023-08-09T13:32:57.966 INFO:teuthology.orchestra.run.smithi043.stdout:Change: 2023-08-09 13:29:59.278855934 +0000 2023-08-09T13:32:57.966 INFO:teuthology.orchestra.run.smithi043.stdout: Birth: 2023-08-09 13:29:59.278855934 +0000 2023-08-09T13:32:57.967 DEBUG:teuthology.orchestra.run.smithi043:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2023-08-09T13:32:58.078 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2023-08-09T13:32:58.078 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2023-08-09T13:32:58.079 INFO:teuthology.orchestra.run.smithi043.stderr:512 bytes copied, 0.000340896 s, 1.5 MB/s 2023-08-09T13:32:58.081 DEBUG:teuthology.orchestra.run.smithi043:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2023-08-09T13:32:58.124 DEBUG:teuthology.orchestra.run.smithi043:> stat /dev/vg_nvme/lv_2 2023-08-09T13:32:58.207 INFO:teuthology.orchestra.run.smithi043.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2023-08-09T13:32:58.207 INFO:teuthology.orchestra.run.smithi043.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2023-08-09T13:32:58.207 INFO:teuthology.orchestra.run.smithi043.stdout:Device: 5h/5d Inode: 641 Links: 1 2023-08-09T13:32:58.207 INFO:teuthology.orchestra.run.smithi043.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2023-08-09T13:32:58.207 INFO:teuthology.orchestra.run.smithi043.stdout:Context: system_u:object_r:device_t:s0 2023-08-09T13:32:58.207 INFO:teuthology.orchestra.run.smithi043.stdout:Access: 2023-08-09 13:32:46.285832983 +0000 2023-08-09T13:32:58.208 INFO:teuthology.orchestra.run.smithi043.stdout:Modify: 2023-08-09 13:30:00.343843650 +0000 2023-08-09T13:32:58.208 INFO:teuthology.orchestra.run.smithi043.stdout:Change: 2023-08-09 13:30:00.343843650 +0000 2023-08-09T13:32:58.208 INFO:teuthology.orchestra.run.smithi043.stdout: Birth: 2023-08-09 13:30:00.343843650 +0000 2023-08-09T13:32:58.208 DEBUG:teuthology.orchestra.run.smithi043:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2023-08-09T13:32:58.320 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2023-08-09T13:32:58.321 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2023-08-09T13:32:58.321 INFO:teuthology.orchestra.run.smithi043.stderr:512 bytes copied, 0.000273927 s, 1.9 MB/s 2023-08-09T13:32:58.323 DEBUG:teuthology.orchestra.run.smithi043:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2023-08-09T13:32:58.365 DEBUG:teuthology.orchestra.run.smithi043:> stat /dev/vg_nvme/lv_3 2023-08-09T13:32:58.448 INFO:teuthology.orchestra.run.smithi043.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2023-08-09T13:32:58.448 INFO:teuthology.orchestra.run.smithi043.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2023-08-09T13:32:58.448 INFO:teuthology.orchestra.run.smithi043.stdout:Device: 5h/5d Inode: 658 Links: 1 2023-08-09T13:32:58.448 INFO:teuthology.orchestra.run.smithi043.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2023-08-09T13:32:58.449 INFO:teuthology.orchestra.run.smithi043.stdout:Context: system_u:object_r:device_t:s0 2023-08-09T13:32:58.449 INFO:teuthology.orchestra.run.smithi043.stdout:Access: 2023-08-09 13:32:46.287832960 +0000 2023-08-09T13:32:58.449 INFO:teuthology.orchestra.run.smithi043.stdout:Modify: 2023-08-09 13:30:01.429831125 +0000 2023-08-09T13:32:58.449 INFO:teuthology.orchestra.run.smithi043.stdout:Change: 2023-08-09 13:30:01.429831125 +0000 2023-08-09T13:32:58.449 INFO:teuthology.orchestra.run.smithi043.stdout: Birth: 2023-08-09 13:30:01.429831125 +0000 2023-08-09T13:32:58.449 DEBUG:teuthology.orchestra.run.smithi043:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2023-08-09T13:32:58.560 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2023-08-09T13:32:58.561 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2023-08-09T13:32:58.561 INFO:teuthology.orchestra.run.smithi043.stderr:512 bytes copied, 0.000226313 s, 2.3 MB/s 2023-08-09T13:32:58.563 DEBUG:teuthology.orchestra.run.smithi043:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2023-08-09T13:32:58.609 DEBUG:teuthology.orchestra.run.smithi043:> stat /dev/vg_nvme/lv_4 2023-08-09T13:32:58.695 INFO:teuthology.orchestra.run.smithi043.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2023-08-09T13:32:58.695 INFO:teuthology.orchestra.run.smithi043.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2023-08-09T13:32:58.696 INFO:teuthology.orchestra.run.smithi043.stdout:Device: 5h/5d Inode: 676 Links: 1 2023-08-09T13:32:58.696 INFO:teuthology.orchestra.run.smithi043.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2023-08-09T13:32:58.696 INFO:teuthology.orchestra.run.smithi043.stdout:Context: system_u:object_r:device_t:s0 2023-08-09T13:32:58.696 INFO:teuthology.orchestra.run.smithi043.stdout:Access: 2023-08-09 13:32:46.287832960 +0000 2023-08-09T13:32:58.696 INFO:teuthology.orchestra.run.smithi043.stdout:Modify: 2023-08-09 13:30:02.463819199 +0000 2023-08-09T13:32:58.696 INFO:teuthology.orchestra.run.smithi043.stdout:Change: 2023-08-09 13:30:02.463819199 +0000 2023-08-09T13:32:58.696 INFO:teuthology.orchestra.run.smithi043.stdout: Birth: 2023-08-09 13:30:02.463819199 +0000 2023-08-09T13:32:58.697 DEBUG:teuthology.orchestra.run.smithi043:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2023-08-09T13:32:58.809 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2023-08-09T13:32:58.809 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2023-08-09T13:32:58.809 INFO:teuthology.orchestra.run.smithi043.stderr:512 bytes copied, 0.000213182 s, 2.4 MB/s 2023-08-09T13:32:58.812 DEBUG:teuthology.orchestra.run.smithi043:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2023-08-09T13:32:58.854 INFO:tasks.ceph:osd dev map: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'} 2023-08-09T13:32:58.854 DEBUG:teuthology.orchestra.run.smithi203:> set -ex 2023-08-09T13:32:58.854 DEBUG:teuthology.orchestra.run.smithi203:> dd if=/scratch_devs of=/dev/stdout 2023-08-09T13:32:58.903 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2023-08-09T13:32:58.903 DEBUG:teuthology.orchestra.run.smithi203:> stat /dev/vg_nvme/lv_1 2023-08-09T13:32:58.986 INFO:teuthology.orchestra.run.smithi203.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2023-08-09T13:32:58.987 INFO:teuthology.orchestra.run.smithi203.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2023-08-09T13:32:58.987 INFO:teuthology.orchestra.run.smithi203.stdout:Device: 5h/5d Inode: 622 Links: 1 2023-08-09T13:32:58.987 INFO:teuthology.orchestra.run.smithi203.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2023-08-09T13:32:58.987 INFO:teuthology.orchestra.run.smithi203.stdout:Context: system_u:object_r:device_t:s0 2023-08-09T13:32:58.988 INFO:teuthology.orchestra.run.smithi203.stdout:Access: 2023-08-09 13:32:47.286786372 +0000 2023-08-09T13:32:58.988 INFO:teuthology.orchestra.run.smithi203.stdout:Modify: 2023-08-09 13:29:27.195337766 +0000 2023-08-09T13:32:58.988 INFO:teuthology.orchestra.run.smithi203.stdout:Change: 2023-08-09 13:29:27.195337766 +0000 2023-08-09T13:32:58.989 INFO:teuthology.orchestra.run.smithi203.stdout: Birth: 2023-08-09 13:29:27.195337766 +0000 2023-08-09T13:32:58.989 DEBUG:teuthology.orchestra.run.smithi203:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2023-08-09T13:32:59.096 INFO:teuthology.orchestra.run.smithi203.stderr:1+0 records in 2023-08-09T13:32:59.097 INFO:teuthology.orchestra.run.smithi203.stderr:1+0 records out 2023-08-09T13:32:59.097 INFO:teuthology.orchestra.run.smithi203.stderr:512 bytes copied, 0.000345769 s, 1.5 MB/s 2023-08-09T13:32:59.100 DEBUG:teuthology.orchestra.run.smithi203:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2023-08-09T13:32:59.143 DEBUG:teuthology.orchestra.run.smithi203:> stat /dev/vg_nvme/lv_2 2023-08-09T13:32:59.225 INFO:teuthology.orchestra.run.smithi203.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2023-08-09T13:32:59.225 INFO:teuthology.orchestra.run.smithi203.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2023-08-09T13:32:59.225 INFO:teuthology.orchestra.run.smithi203.stdout:Device: 5h/5d Inode: 641 Links: 1 2023-08-09T13:32:59.225 INFO:teuthology.orchestra.run.smithi203.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2023-08-09T13:32:59.225 INFO:teuthology.orchestra.run.smithi203.stdout:Context: system_u:object_r:device_t:s0 2023-08-09T13:32:59.225 INFO:teuthology.orchestra.run.smithi203.stdout:Access: 2023-08-09 13:32:47.288786323 +0000 2023-08-09T13:32:59.226 INFO:teuthology.orchestra.run.smithi203.stdout:Modify: 2023-08-09 13:29:28.235312177 +0000 2023-08-09T13:32:59.226 INFO:teuthology.orchestra.run.smithi203.stdout:Change: 2023-08-09 13:29:28.235312177 +0000 2023-08-09T13:32:59.226 INFO:teuthology.orchestra.run.smithi203.stdout: Birth: 2023-08-09 13:29:28.235312177 +0000 2023-08-09T13:32:59.226 DEBUG:teuthology.orchestra.run.smithi203:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2023-08-09T13:32:59.338 INFO:teuthology.orchestra.run.smithi203.stderr:1+0 records in 2023-08-09T13:32:59.338 INFO:teuthology.orchestra.run.smithi203.stderr:1+0 records out 2023-08-09T13:32:59.338 INFO:teuthology.orchestra.run.smithi203.stderr:512 bytes copied, 0.000285201 s, 1.8 MB/s 2023-08-09T13:32:59.341 DEBUG:teuthology.orchestra.run.smithi203:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2023-08-09T13:32:59.383 DEBUG:teuthology.orchestra.run.smithi203:> stat /dev/vg_nvme/lv_3 2023-08-09T13:32:59.464 INFO:teuthology.orchestra.run.smithi203.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2023-08-09T13:32:59.464 INFO:teuthology.orchestra.run.smithi203.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2023-08-09T13:32:59.465 INFO:teuthology.orchestra.run.smithi203.stdout:Device: 5h/5d Inode: 661 Links: 1 2023-08-09T13:32:59.465 INFO:teuthology.orchestra.run.smithi203.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2023-08-09T13:32:59.465 INFO:teuthology.orchestra.run.smithi203.stdout:Context: system_u:object_r:device_t:s0 2023-08-09T13:32:59.465 INFO:teuthology.orchestra.run.smithi203.stdout:Access: 2023-08-09 13:32:47.289786298 +0000 2023-08-09T13:32:59.465 INFO:teuthology.orchestra.run.smithi203.stdout:Modify: 2023-08-09 13:29:29.271286685 +0000 2023-08-09T13:32:59.465 INFO:teuthology.orchestra.run.smithi203.stdout:Change: 2023-08-09 13:29:29.271286685 +0000 2023-08-09T13:32:59.465 INFO:teuthology.orchestra.run.smithi203.stdout: Birth: 2023-08-09 13:29:29.271286685 +0000 2023-08-09T13:32:59.466 DEBUG:teuthology.orchestra.run.smithi203:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2023-08-09T13:32:59.573 INFO:teuthology.orchestra.run.smithi203.stderr:1+0 records in 2023-08-09T13:32:59.573 INFO:teuthology.orchestra.run.smithi203.stderr:1+0 records out 2023-08-09T13:32:59.574 INFO:teuthology.orchestra.run.smithi203.stderr:512 bytes copied, 0.000283142 s, 1.8 MB/s 2023-08-09T13:32:59.576 DEBUG:teuthology.orchestra.run.smithi203:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2023-08-09T13:32:59.617 DEBUG:teuthology.orchestra.run.smithi203:> stat /dev/vg_nvme/lv_4 2023-08-09T13:32:59.696 INFO:teuthology.orchestra.run.smithi203.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2023-08-09T13:32:59.696 INFO:teuthology.orchestra.run.smithi203.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2023-08-09T13:32:59.696 INFO:teuthology.orchestra.run.smithi203.stdout:Device: 5h/5d Inode: 676 Links: 1 2023-08-09T13:32:59.697 INFO:teuthology.orchestra.run.smithi203.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2023-08-09T13:32:59.697 INFO:teuthology.orchestra.run.smithi203.stdout:Context: system_u:object_r:device_t:s0 2023-08-09T13:32:59.697 INFO:teuthology.orchestra.run.smithi203.stdout:Access: 2023-08-09 13:32:47.289786298 +0000 2023-08-09T13:32:59.697 INFO:teuthology.orchestra.run.smithi203.stdout:Modify: 2023-08-09 13:29:30.285261736 +0000 2023-08-09T13:32:59.697 INFO:teuthology.orchestra.run.smithi203.stdout:Change: 2023-08-09 13:29:30.285261736 +0000 2023-08-09T13:32:59.697 INFO:teuthology.orchestra.run.smithi203.stdout: Birth: 2023-08-09 13:29:30.285261736 +0000 2023-08-09T13:32:59.697 DEBUG:teuthology.orchestra.run.smithi203:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2023-08-09T13:32:59.807 INFO:teuthology.orchestra.run.smithi203.stderr:1+0 records in 2023-08-09T13:32:59.808 INFO:teuthology.orchestra.run.smithi203.stderr:1+0 records out 2023-08-09T13:32:59.808 INFO:teuthology.orchestra.run.smithi203.stderr:512 bytes copied, 0.000354452 s, 1.4 MB/s 2023-08-09T13:32:59.810 DEBUG:teuthology.orchestra.run.smithi203:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2023-08-09T13:32:59.857 INFO:tasks.ceph:osd dev map: {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'} 2023-08-09T13:32:59.858 INFO:tasks.ceph:remote_to_roles_to_devs: {Remote(name='ubuntu@smithi043.front.sepia.ceph.com'): {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'}, Remote(name='ubuntu@smithi203.front.sepia.ceph.com'): {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'}} 2023-08-09T13:32:59.858 INFO:tasks.ceph:Generating config... 2023-08-09T13:32:59.859 INFO:tasks.ceph:[client] client mount timeout = 600 2023-08-09T13:32:59.859 INFO:tasks.ceph:[client] debug client = 20 2023-08-09T13:32:59.860 INFO:tasks.ceph:[client] debug ms = 1 2023-08-09T13:32:59.860 INFO:tasks.ceph:[client] pid file = 2023-08-09T13:32:59.860 INFO:tasks.ceph:[client] rados mon op timeout = 900 2023-08-09T13:32:59.860 INFO:tasks.ceph:[client] rados osd op timeout = 900 2023-08-09T13:32:59.860 INFO:tasks.ceph:[mds] debug mds = 20 2023-08-09T13:32:59.861 INFO:tasks.ceph:[mds] debug mds balancer = 20 2023-08-09T13:32:59.861 INFO:tasks.ceph:[mds] debug ms = 1 2023-08-09T13:32:59.861 INFO:tasks.ceph:[mds] mds debug frag = True 2023-08-09T13:32:59.861 INFO:tasks.ceph:[mds] mds debug scatterstat = True 2023-08-09T13:32:59.861 INFO:tasks.ceph:[mds] mds op complaint time = 180 2023-08-09T13:32:59.862 INFO:tasks.ceph:[mds] mds verify scatter = True 2023-08-09T13:32:59.862 INFO:tasks.ceph:[mds] osd op complaint time = 180 2023-08-09T13:32:59.862 INFO:tasks.ceph:[mds] rados mon op timeout = 900 2023-08-09T13:32:59.862 INFO:tasks.ceph:[mds] rados osd op timeout = 900 2023-08-09T13:32:59.862 INFO:tasks.ceph:[mgr] debug client = 20 2023-08-09T13:32:59.863 INFO:tasks.ceph:[mgr] debug finisher = 20 2023-08-09T13:32:59.863 INFO:tasks.ceph:[mgr] debug mgr = 20 2023-08-09T13:32:59.863 INFO:tasks.ceph:[mgr] debug ms = 1 2023-08-09T13:32:59.863 INFO:tasks.ceph:[mon] debug mon = 20 2023-08-09T13:32:59.863 INFO:tasks.ceph:[mon] debug ms = 1 2023-08-09T13:32:59.863 INFO:tasks.ceph:[mon] debug paxos = 20 2023-08-09T13:32:59.864 INFO:tasks.ceph:[mon] mon op complaint time = 120 2023-08-09T13:32:59.864 INFO:tasks.ceph:[osd] bluestore block size = 96636764160 2023-08-09T13:32:59.864 INFO:tasks.ceph:[osd] bluestore fsck on mount = True 2023-08-09T13:32:59.864 INFO:tasks.ceph:[osd] debug bluefs = 20 2023-08-09T13:32:59.865 INFO:tasks.ceph:[osd] debug bluestore = 20 2023-08-09T13:32:59.865 INFO:tasks.ceph:[osd] debug ms = 1 2023-08-09T13:32:59.865 INFO:tasks.ceph:[osd] debug osd = 20 2023-08-09T13:32:59.865 INFO:tasks.ceph:[osd] debug rocksdb = 10 2023-08-09T13:32:59.865 INFO:tasks.ceph:[osd] mon osd backfillfull_ratio = 0.85 2023-08-09T13:32:59.866 INFO:tasks.ceph:[osd] mon osd full ratio = 0.9 2023-08-09T13:32:59.866 INFO:tasks.ceph:[osd] mon osd nearfull ratio = 0.8 2023-08-09T13:32:59.866 INFO:tasks.ceph:[osd] osd failsafe full ratio = 0.95 2023-08-09T13:32:59.866 INFO:tasks.ceph:[osd] osd objectstore = bluestore 2023-08-09T13:32:59.866 INFO:tasks.ceph:[osd] osd op complaint time = 180 2023-08-09T13:32:59.867 INFO:tasks.ceph:Setting up mon.a... 2023-08-09T13:32:59.867 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring /etc/ceph/ceph.keyring 2023-08-09T13:33:00.028 INFO:teuthology.orchestra.run.smithi043.stdout:creating /etc/ceph/ceph.keyring 2023-08-09T13:33:00.037 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --gen-key --name=mon. /etc/ceph/ceph.keyring 2023-08-09T13:33:00.179 DEBUG:teuthology.orchestra.run.smithi043:> sudo chmod 0644 /etc/ceph/ceph.keyring 2023-08-09T13:33:00.279 DEBUG:tasks.ceph:Ceph mon addresses: [('mon.a', '172.21.15.43'), ('mon.b', '172.21.15.203'), ('mon.c', '[v2:172.21.15.203:3301,v1:172.21.15.203:6790]')] 2023-08-09T13:33:00.280 DEBUG:tasks.ceph:writing out conf {'global': {'chdir': '', 'pid file': '/var/run/ceph/$cluster-$name.pid', 'auth supported': 'cephx', 'filestore xattr use omap': 'true', 'mon clock drift allowed': '1.000', 'osd crush chooseleaf type': '0', 'auth debug': 'true', 'ms die on old message': 'true', 'ms die on bug': 'true', 'mon max pg per osd': '10000', 'mon pg warn max object skew': '0', 'osd_pool_default_pg_autoscale_mode': 'off', 'osd pool default size': '2', 'mon osd allow primary affinity': 'true', 'mon osd allow pg remap': 'true', 'mon warn on legacy crush tunables': 'false', 'mon warn on crush straw calc version zero': 'false', 'mon warn on no sortbitwise': 'false', 'mon warn on osd down out interval zero': 'false', 'mon warn on too few osds': 'false', 'mon_warn_on_pool_pg_num_not_power_of_two': 'false', 'mon_warn_on_pool_no_redundancy': 'false', 'mon_allow_pool_size_one': 'true', 'osd pool default erasure code profile': 'plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd', 'osd default data pool replay window': '5', 'mon allow pool delete': 'true', 'mon cluster log file level': 'debug', 'debug asserts on shutdown': 'true', 'mon health detail to clog': 'false', 'mon host': '172.21.15.43,172.21.15.203,[v2:172.21.15.203:3301,v1:172.21.15.203:6790]'}, 'osd': {'osd journal size': '100', 'osd scrub load threshold': '5.0', 'osd scrub max interval': '600', 'osd mclock profile': 'high_recovery_ops', 'osd recover clone overlap': 'true', 'osd recovery max chunk': '1048576', 'osd debug shutdown': 'true', 'osd debug op order': 'true', 'osd debug verify stray on activate': 'true', 'osd open classes on start': 'true', 'osd debug pg log writeout': 'true', 'osd deep scrub update digest min age': '30', 'osd map max advance': '10', 'journal zero on create': 'true', 'filestore ondisk finisher threads': '3', 'filestore apply finisher threads': '3', 'bdev debug aio': 'true', 'osd debug misdirected ops': 'true', 'bluestore block size': 96636764160, 'bluestore fsck on mount': True, 'debug bluefs': 20, 'debug bluestore': 20, 'debug ms': 1, 'debug osd': 20, 'debug rocksdb': 10, 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd failsafe full ratio': 0.95, 'osd objectstore': 'bluestore', 'osd op complaint time': 180}, 'mgr': {'debug ms': 1, 'debug mgr': 20, 'debug mon': '20', 'debug auth': '20', 'mon reweight min pgs per osd': '4', 'mon reweight min bytes per osd': '10', 'mgr/telemetry/nag': 'false', 'debug client': 20, 'debug finisher': 20}, 'mon': {'debug ms': 1, 'debug mon': 20, 'debug paxos': 20, 'debug auth': '20', 'mon data avail warn': '5', 'mon mgr mkfs grace': '240', 'mon reweight min pgs per osd': '4', 'mon osd reporter subtree level': 'osd', 'mon osd prime pg temp': 'true', 'mon reweight min bytes per osd': '10', 'auth mon ticket ttl': '660', 'auth service ticket ttl': '240', 'mon_warn_on_insecure_global_id_reclaim': 'false', 'mon_warn_on_insecure_global_id_reclaim_allowed': 'false', 'mon_down_mkfs_grace': '2m', 'mon_warn_on_filestore_osds': 'false', 'mon op complaint time': 120}, 'client': {'rgw cache enabled': 'true', 'rgw enable ops log': 'true', 'rgw enable usage log': 'true', 'log file': '/var/log/ceph/$cluster-$name.$pid.log', 'admin socket': '/var/run/ceph/$cluster-$name.$pid.asok', 'client mount timeout': 600, 'debug client': 20, 'debug ms': 1, 'pid file': '', 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'mon.a': {}, 'mon.b': {}, 'mon.c': {}, 'mds.a': {}, 'mds.c': {}, 'mds.b': {}, 'mds.d': {}, 'mds': {'debug mds': 20, 'debug mds balancer': 20, 'debug ms': 1, 'mds debug frag': True, 'mds debug scatterstat': True, 'mds op complaint time': 180, 'mds verify scatter': True, 'osd op complaint time': 180, 'rados mon op timeout': 900, 'rados osd op timeout': 900}} 2023-08-09T13:33:00.281 DEBUG:teuthology.orchestra.run.smithi043:> set -ex 2023-08-09T13:33:00.281 DEBUG:teuthology.orchestra.run.smithi043:> dd of=/home/ubuntu/cephtest/ceph.tmp.conf 2023-08-09T13:33:00.342 DEBUG:teuthology.orchestra.run.smithi043:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage monmaptool -c /home/ubuntu/cephtest/ceph.tmp.conf --create --clobber --enable-all-features --add a 172.21.15.43 --add b 172.21.15.203 --addv c '[v2:172.21.15.203:3301,v1:172.21.15.203:6790]' --print /home/ubuntu/cephtest/ceph.monmap 2023-08-09T13:33:00.460 INFO:teuthology.orchestra.run.smithi043.stdout:monmaptool: monmap file /home/ubuntu/cephtest/ceph.monmap 2023-08-09T13:33:00.461 INFO:teuthology.orchestra.run.smithi043.stdout:monmaptool: generated fsid daac87ee-1337-4250-a955-aea4e408a39b 2023-08-09T13:33:00.461 INFO:teuthology.orchestra.run.smithi043.stdout:setting min_mon_release = pacific 2023-08-09T13:33:00.461 INFO:teuthology.orchestra.run.smithi043.stdout:epoch 0 2023-08-09T13:33:00.461 INFO:teuthology.orchestra.run.smithi043.stdout:fsid daac87ee-1337-4250-a955-aea4e408a39b 2023-08-09T13:33:00.462 INFO:teuthology.orchestra.run.smithi043.stdout:last_changed 2023-08-09T13:33:00.459119+0000 2023-08-09T13:33:00.462 INFO:teuthology.orchestra.run.smithi043.stdout:created 2023-08-09T13:33:00.459119+0000 2023-08-09T13:33:00.462 INFO:teuthology.orchestra.run.smithi043.stdout:min_mon_release 16 (pacific) 2023-08-09T13:33:00.463 INFO:teuthology.orchestra.run.smithi043.stdout:election_strategy: 1 2023-08-09T13:33:00.463 INFO:teuthology.orchestra.run.smithi043.stdout:0: [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] mon.a 2023-08-09T13:33:00.463 INFO:teuthology.orchestra.run.smithi043.stdout:1: [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] mon.b 2023-08-09T13:33:00.463 INFO:teuthology.orchestra.run.smithi043.stdout:2: [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] mon.c 2023-08-09T13:33:00.464 INFO:teuthology.orchestra.run.smithi043.stdout:monmaptool: writing epoch 0 to /home/ubuntu/cephtest/ceph.monmap (3 monitors) 2023-08-09T13:33:00.465 DEBUG:teuthology.orchestra.run.smithi043:> rm -- /home/ubuntu/cephtest/ceph.tmp.conf 2023-08-09T13:33:00.520 INFO:tasks.ceph:Writing /etc/ceph/ceph.conf for FSID daac87ee-1337-4250-a955-aea4e408a39b... 2023-08-09T13:33:00.521 DEBUG:teuthology.orchestra.run.smithi043:> sudo mkdir -p /etc/ceph && sudo chmod 0755 /etc/ceph && sudo tee /etc/ceph/ceph.conf && sudo chmod 0644 /etc/ceph/ceph.conf > /dev/null 2023-08-09T13:33:00.527 DEBUG:teuthology.orchestra.run.smithi203:> sudo mkdir -p /etc/ceph && sudo chmod 0755 /etc/ceph && sudo tee /etc/ceph/ceph.conf && sudo chmod 0644 /etc/ceph/ceph.conf > /dev/null 2023-08-09T13:33:00.682 INFO:teuthology.orchestra.run.smithi203.stdout:[global] 2023-08-09T13:33:00.683 INFO:teuthology.orchestra.run.smithi203.stdout: chdir = "" 2023-08-09T13:33:00.683 INFO:teuthology.orchestra.run.smithi203.stdout: pid file = /var/run/ceph/$cluster-$name.pid 2023-08-09T13:33:00.683 INFO:teuthology.orchestra.run.smithi203.stdout: auth supported = cephx 2023-08-09T13:33:00.683 INFO:teuthology.orchestra.run.smithi203.stdout: 2023-08-09T13:33:00.683 INFO:teuthology.orchestra.run.smithi203.stdout: filestore xattr use omap = true 2023-08-09T13:33:00.683 INFO:teuthology.orchestra.run.smithi203.stdout: 2023-08-09T13:33:00.683 INFO:teuthology.orchestra.run.smithi203.stdout: mon clock drift allowed = 1.000 2023-08-09T13:33:00.684 INFO:teuthology.orchestra.run.smithi203.stdout: 2023-08-09T13:33:00.684 INFO:teuthology.orchestra.run.smithi203.stdout: osd crush chooseleaf type = 0 2023-08-09T13:33:00.684 INFO:teuthology.orchestra.run.smithi203.stdout: auth debug = true 2023-08-09T13:33:00.684 INFO:teuthology.orchestra.run.smithi203.stdout: 2023-08-09T13:33:00.684 INFO:teuthology.orchestra.run.smithi203.stdout: ms die on old message = true 2023-08-09T13:33:00.684 INFO:teuthology.orchestra.run.smithi203.stdout: ms die on bug = true 2023-08-09T13:33:00.684 INFO:teuthology.orchestra.run.smithi203.stdout: 2023-08-09T13:33:00.685 INFO:teuthology.orchestra.run.smithi203.stdout: mon max pg per osd = 10000 # >= luminous 2023-08-09T13:33:00.685 INFO:teuthology.orchestra.run.smithi203.stdout: mon pg warn max object skew = 0 2023-08-09T13:33:00.685 INFO:teuthology.orchestra.run.smithi203.stdout: 2023-08-09T13:33:00.685 INFO:teuthology.orchestra.run.smithi203.stdout: # disable pg_autoscaler by default for new pools 2023-08-09T13:33:00.685 INFO:teuthology.orchestra.run.smithi203.stdout: osd_pool_default_pg_autoscale_mode = off 2023-08-09T13:33:00.685 INFO:teuthology.orchestra.run.smithi203.stdout: 2023-08-09T13:33:00.685 INFO:teuthology.orchestra.run.smithi203.stdout: osd pool default size = 2 2023-08-09T13:33:00.686 INFO:teuthology.orchestra.run.smithi203.stdout: 2023-08-09T13:33:00.686 INFO:teuthology.orchestra.run.smithi203.stdout: mon osd allow primary affinity = true 2023-08-09T13:33:00.686 INFO:teuthology.orchestra.run.smithi203.stdout: mon osd allow pg remap = true 2023-08-09T13:33:00.686 INFO:teuthology.orchestra.run.smithi203.stdout: mon warn on legacy crush tunables = false 2023-08-09T13:33:00.686 INFO:teuthology.orchestra.run.smithi203.stdout: mon warn on crush straw calc version zero = false 2023-08-09T13:33:00.686 INFO:teuthology.orchestra.run.smithi203.stdout: mon warn on no sortbitwise = false 2023-08-09T13:33:00.686 INFO:teuthology.orchestra.run.smithi203.stdout: mon warn on osd down out interval zero = false 2023-08-09T13:33:00.686 INFO:teuthology.orchestra.run.smithi203.stdout: mon warn on too few osds = false 2023-08-09T13:33:00.687 INFO:teuthology.orchestra.run.smithi203.stdout: mon_warn_on_pool_pg_num_not_power_of_two = false 2023-08-09T13:33:00.687 INFO:teuthology.orchestra.run.smithi203.stdout: mon_warn_on_pool_no_redundancy = false 2023-08-09T13:33:00.687 INFO:teuthology.orchestra.run.smithi203.stdout: mon_allow_pool_size_one = true 2023-08-09T13:33:00.687 INFO:teuthology.orchestra.run.smithi203.stdout: 2023-08-09T13:33:00.687 INFO:teuthology.orchestra.run.smithi203.stdout: osd pool default erasure code profile = plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd 2023-08-09T13:33:00.687 INFO:teuthology.orchestra.run.smithi203.stdout: 2023-08-09T13:33:00.687 INFO:teuthology.orchestra.run.smithi203.stdout: osd default data pool replay window = 5 2023-08-09T13:33:00.687 INFO:teuthology.orchestra.run.smithi203.stdout: 2023-08-09T13:33:00.688 INFO:teuthology.orchestra.run.smithi203.stdout: mon allow pool delete = true 2023-08-09T13:33:00.688 INFO:teuthology.orchestra.run.smithi203.stdout: 2023-08-09T13:33:00.688 INFO:teuthology.orchestra.run.smithi203.stdout: mon cluster log file level = debug 2023-08-09T13:33:00.688 INFO:teuthology.orchestra.run.smithi203.stdout: debug asserts on shutdown = true 2023-08-09T13:33:00.688 INFO:teuthology.orchestra.run.smithi203.stdout: mon health detail to clog = false 2023-08-09T13:33:00.688 INFO:teuthology.orchestra.run.smithi203.stdout: mon host = "172.21.15.43,172.21.15.203,[v2:172.21.15.203:3301,v1:172.21.15.203:6790]" 2023-08-09T13:33:00.688 INFO:teuthology.orchestra.run.smithi203.stdout: fsid = daac87ee-1337-4250-a955-aea4e408a39b 2023-08-09T13:33:00.689 INFO:teuthology.orchestra.run.smithi203.stdout: 2023-08-09T13:33:00.689 INFO:teuthology.orchestra.run.smithi203.stdout:[osd] 2023-08-09T13:33:00.689 INFO:teuthology.orchestra.run.smithi203.stdout: osd journal size = 100 2023-08-09T13:33:00.689 INFO:teuthology.orchestra.run.smithi203.stdout: 2023-08-09T13:33:00.689 INFO:teuthology.orchestra.run.smithi203.stdout: osd scrub load threshold = 5.0 2023-08-09T13:33:00.689 INFO:teuthology.orchestra.run.smithi203.stdout: osd scrub max interval = 600 2023-08-09T13:33:00.689 INFO:teuthology.orchestra.run.smithi203.stdout: osd mclock profile = high_recovery_ops 2023-08-09T13:33:00.689 INFO:teuthology.orchestra.run.smithi203.stdout: 2023-08-09T13:33:00.690 INFO:teuthology.orchestra.run.smithi203.stdout: osd recover clone overlap = true 2023-08-09T13:33:00.690 INFO:teuthology.orchestra.run.smithi203.stdout: osd recovery max chunk = 1048576 2023-08-09T13:33:00.690 INFO:teuthology.orchestra.run.smithi203.stdout: 2023-08-09T13:33:00.690 INFO:teuthology.orchestra.run.smithi203.stdout: osd debug shutdown = true 2023-08-09T13:33:00.690 INFO:teuthology.orchestra.run.smithi203.stdout: osd debug op order = true 2023-08-09T13:33:00.690 INFO:teuthology.orchestra.run.smithi203.stdout: osd debug verify stray on activate = true 2023-08-09T13:33:00.690 INFO:teuthology.orchestra.run.smithi203.stdout: 2023-08-09T13:33:00.690 INFO:teuthology.orchestra.run.smithi203.stdout: osd open classes on start = true 2023-08-09T13:33:00.691 INFO:teuthology.orchestra.run.smithi203.stdout: osd debug pg log writeout = true 2023-08-09T13:33:00.691 INFO:teuthology.orchestra.run.smithi203.stdout: 2023-08-09T13:33:00.691 INFO:teuthology.orchestra.run.smithi203.stdout: osd deep scrub update digest min age = 30 2023-08-09T13:33:00.691 INFO:teuthology.orchestra.run.smithi203.stdout: 2023-08-09T13:33:00.691 INFO:teuthology.orchestra.run.smithi203.stdout: osd map max advance = 10 2023-08-09T13:33:00.691 INFO:teuthology.orchestra.run.smithi203.stdout: 2023-08-09T13:33:00.691 INFO:teuthology.orchestra.run.smithi203.stdout: journal zero on create = true 2023-08-09T13:33:00.691 INFO:teuthology.orchestra.run.smithi203.stdout: 2023-08-09T13:33:00.692 INFO:teuthology.orchestra.run.smithi203.stdout: filestore ondisk finisher threads = 3 2023-08-09T13:33:00.692 INFO:teuthology.orchestra.run.smithi203.stdout: filestore apply finisher threads = 3 2023-08-09T13:33:00.692 INFO:teuthology.orchestra.run.smithi203.stdout: 2023-08-09T13:33:00.692 INFO:teuthology.orchestra.run.smithi203.stdout: bdev debug aio = true 2023-08-09T13:33:00.692 INFO:teuthology.orchestra.run.smithi203.stdout: osd debug misdirected ops = true 2023-08-09T13:33:00.692 INFO:teuthology.orchestra.run.smithi203.stdout: bluestore block size = 96636764160 2023-08-09T13:33:00.692 INFO:teuthology.orchestra.run.smithi203.stdout: bluestore fsck on mount = True 2023-08-09T13:33:00.692 INFO:teuthology.orchestra.run.smithi203.stdout: debug bluefs = 20 2023-08-09T13:33:00.693 INFO:teuthology.orchestra.run.smithi203.stdout: debug bluestore = 20 2023-08-09T13:33:00.693 INFO:teuthology.orchestra.run.smithi203.stdout: debug ms = 1 2023-08-09T13:33:00.693 INFO:teuthology.orchestra.run.smithi203.stdout: debug osd = 20 2023-08-09T13:33:00.693 INFO:teuthology.orchestra.run.smithi203.stdout: debug rocksdb = 10 2023-08-09T13:33:00.693 INFO:teuthology.orchestra.run.smithi203.stdout: mon osd backfillfull_ratio = 0.85 2023-08-09T13:33:00.693 INFO:teuthology.orchestra.run.smithi203.stdout: mon osd full ratio = 0.9 2023-08-09T13:33:00.693 INFO:teuthology.orchestra.run.smithi203.stdout: mon osd nearfull ratio = 0.8 2023-08-09T13:33:00.693 INFO:teuthology.orchestra.run.smithi203.stdout: osd failsafe full ratio = 0.95 2023-08-09T13:33:00.694 INFO:teuthology.orchestra.run.smithi203.stdout: osd objectstore = bluestore 2023-08-09T13:33:00.694 INFO:teuthology.orchestra.run.smithi203.stdout: osd op complaint time = 180 2023-08-09T13:33:00.694 INFO:teuthology.orchestra.run.smithi203.stdout: 2023-08-09T13:33:00.694 INFO:teuthology.orchestra.run.smithi203.stdout:[mgr] 2023-08-09T13:33:00.694 INFO:teuthology.orchestra.run.smithi203.stdout: debug ms = 1 2023-08-09T13:33:00.694 INFO:teuthology.orchestra.run.smithi203.stdout: debug mgr = 20 2023-08-09T13:33:00.694 INFO:teuthology.orchestra.run.smithi203.stdout: debug mon = 20 2023-08-09T13:33:00.694 INFO:teuthology.orchestra.run.smithi203.stdout: debug auth = 20 2023-08-09T13:33:00.695 INFO:teuthology.orchestra.run.smithi203.stdout: mon reweight min pgs per osd = 4 2023-08-09T13:33:00.695 INFO:teuthology.orchestra.run.smithi203.stdout: mon reweight min bytes per osd = 10 2023-08-09T13:33:00.695 INFO:teuthology.orchestra.run.smithi203.stdout: mgr/telemetry/nag = false 2023-08-09T13:33:00.695 INFO:teuthology.orchestra.run.smithi203.stdout: debug client = 20 2023-08-09T13:33:00.695 INFO:teuthology.orchestra.run.smithi203.stdout: debug finisher = 20 2023-08-09T13:33:00.695 INFO:teuthology.orchestra.run.smithi203.stdout: 2023-08-09T13:33:00.695 INFO:teuthology.orchestra.run.smithi203.stdout:[mon] 2023-08-09T13:33:00.695 INFO:teuthology.orchestra.run.smithi203.stdout: debug ms = 1 2023-08-09T13:33:00.696 INFO:teuthology.orchestra.run.smithi203.stdout: debug mon = 20 2023-08-09T13:33:00.696 INFO:teuthology.orchestra.run.smithi203.stdout: debug paxos = 20 2023-08-09T13:33:00.696 INFO:teuthology.orchestra.run.smithi203.stdout: debug auth = 20 2023-08-09T13:33:00.696 INFO:teuthology.orchestra.run.smithi203.stdout: mon data avail warn = 5 2023-08-09T13:33:00.696 INFO:teuthology.orchestra.run.smithi203.stdout: mon mgr mkfs grace = 240 2023-08-09T13:33:00.696 INFO:teuthology.orchestra.run.smithi203.stdout: mon reweight min pgs per osd = 4 2023-08-09T13:33:00.696 INFO:teuthology.orchestra.run.smithi203.stdout: mon osd reporter subtree level = osd 2023-08-09T13:33:00.696 INFO:teuthology.orchestra.run.smithi203.stdout: mon osd prime pg temp = true 2023-08-09T13:33:00.697 INFO:teuthology.orchestra.run.smithi203.stdout: mon reweight min bytes per osd = 10 2023-08-09T13:33:00.697 INFO:teuthology.orchestra.run.smithi203.stdout: 2023-08-09T13:33:00.697 INFO:teuthology.orchestra.run.smithi203.stdout: # rotate auth tickets quickly to exercise renewal paths 2023-08-09T13:33:00.697 INFO:teuthology.orchestra.run.smithi203.stdout: auth mon ticket ttl = 660 # 11m 2023-08-09T13:33:00.697 INFO:teuthology.orchestra.run.smithi203.stdout: auth service ticket ttl = 240 # 4m 2023-08-09T13:33:00.697 INFO:teuthology.orchestra.run.smithi203.stdout: 2023-08-09T13:33:00.697 INFO:teuthology.orchestra.run.smithi203.stdout: # don't complain about insecure global_id in the test suite 2023-08-09T13:33:00.697 INFO:teuthology.orchestra.run.smithi203.stdout: mon_warn_on_insecure_global_id_reclaim = false 2023-08-09T13:33:00.698 INFO:teuthology.orchestra.run.smithi203.stdout: mon_warn_on_insecure_global_id_reclaim_allowed = false 2023-08-09T13:33:00.698 INFO:teuthology.orchestra.run.smithi203.stdout: 2023-08-09T13:33:00.698 INFO:teuthology.orchestra.run.smithi203.stdout: # 1m isn't quite enough 2023-08-09T13:33:00.698 INFO:teuthology.orchestra.run.smithi203.stdout: mon_down_mkfs_grace = 2m 2023-08-09T13:33:00.698 INFO:teuthology.orchestra.run.smithi203.stdout: 2023-08-09T13:33:00.698 INFO:teuthology.orchestra.run.smithi203.stdout: mon_warn_on_filestore_osds = false 2023-08-09T13:33:00.698 INFO:teuthology.orchestra.run.smithi203.stdout: mon op complaint time = 120 2023-08-09T13:33:00.698 INFO:teuthology.orchestra.run.smithi203.stdout: 2023-08-09T13:33:00.699 INFO:teuthology.orchestra.run.smithi203.stdout:[client] 2023-08-09T13:33:00.699 INFO:teuthology.orchestra.run.smithi203.stdout: rgw cache enabled = true 2023-08-09T13:33:00.699 INFO:teuthology.orchestra.run.smithi203.stdout: rgw enable ops log = true 2023-08-09T13:33:00.699 INFO:teuthology.orchestra.run.smithi203.stdout: rgw enable usage log = true 2023-08-09T13:33:00.699 INFO:teuthology.orchestra.run.smithi203.stdout: log file = /var/log/ceph/$cluster-$name.$pid.log 2023-08-09T13:33:00.699 INFO:teuthology.orchestra.run.smithi203.stdout: admin socket = /var/run/ceph/$cluster-$name.$pid.asok 2023-08-09T13:33:00.699 INFO:teuthology.orchestra.run.smithi203.stdout: client mount timeout = 600 2023-08-09T13:33:00.699 INFO:teuthology.orchestra.run.smithi203.stdout: debug client = 20 2023-08-09T13:33:00.700 INFO:teuthology.orchestra.run.smithi203.stdout: debug ms = 1 2023-08-09T13:33:00.700 INFO:teuthology.orchestra.run.smithi203.stdout: pid file = "" 2023-08-09T13:33:00.700 INFO:teuthology.orchestra.run.smithi203.stdout: rados mon op timeout = 900 2023-08-09T13:33:00.700 INFO:teuthology.orchestra.run.smithi203.stdout: rados osd op timeout = 900 2023-08-09T13:33:00.700 INFO:teuthology.orchestra.run.smithi203.stdout:[mon.a] 2023-08-09T13:33:00.700 INFO:teuthology.orchestra.run.smithi203.stdout:[mon.b] 2023-08-09T13:33:00.700 INFO:teuthology.orchestra.run.smithi203.stdout:[mon.c] 2023-08-09T13:33:00.700 INFO:teuthology.orchestra.run.smithi203.stdout:[mds.a] 2023-08-09T13:33:00.701 INFO:teuthology.orchestra.run.smithi203.stdout:[mds.c] 2023-08-09T13:33:00.701 INFO:teuthology.orchestra.run.smithi203.stdout:[mds.b] 2023-08-09T13:33:00.701 INFO:teuthology.orchestra.run.smithi203.stdout:[mds.d] 2023-08-09T13:33:00.701 INFO:teuthology.orchestra.run.smithi203.stdout:[mds] 2023-08-09T13:33:00.701 INFO:teuthology.orchestra.run.smithi203.stdout: debug mds = 20 2023-08-09T13:33:00.701 INFO:teuthology.orchestra.run.smithi203.stdout: debug mds balancer = 20 2023-08-09T13:33:00.701 INFO:teuthology.orchestra.run.smithi203.stdout: debug ms = 1 2023-08-09T13:33:00.701 INFO:teuthology.orchestra.run.smithi203.stdout: mds debug frag = True 2023-08-09T13:33:00.702 INFO:teuthology.orchestra.run.smithi203.stdout: mds debug scatterstat = True 2023-08-09T13:33:00.702 INFO:teuthology.orchestra.run.smithi203.stdout: mds op complaint time = 180 2023-08-09T13:33:00.702 INFO:teuthology.orchestra.run.smithi203.stdout: mds verify scatter = True 2023-08-09T13:33:00.702 INFO:teuthology.orchestra.run.smithi203.stdout: osd op complaint time = 180 2023-08-09T13:33:00.702 INFO:teuthology.orchestra.run.smithi203.stdout: rados mon op timeout = 900 2023-08-09T13:33:00.702 INFO:teuthology.orchestra.run.smithi203.stdout: rados osd op timeout = 900 2023-08-09T13:33:00.705 INFO:teuthology.orchestra.run.smithi043.stdout:[global] 2023-08-09T13:33:00.705 INFO:teuthology.orchestra.run.smithi043.stdout: chdir = "" 2023-08-09T13:33:00.705 INFO:teuthology.orchestra.run.smithi043.stdout: pid file = /var/run/ceph/$cluster-$name.pid 2023-08-09T13:33:00.705 INFO:teuthology.orchestra.run.smithi043.stdout: auth supported = cephx 2023-08-09T13:33:00.705 INFO:teuthology.orchestra.run.smithi043.stdout: 2023-08-09T13:33:00.705 INFO:teuthology.orchestra.run.smithi043.stdout: filestore xattr use omap = true 2023-08-09T13:33:00.706 INFO:teuthology.orchestra.run.smithi043.stdout: 2023-08-09T13:33:00.706 INFO:teuthology.orchestra.run.smithi043.stdout: mon clock drift allowed = 1.000 2023-08-09T13:33:00.706 INFO:teuthology.orchestra.run.smithi043.stdout: 2023-08-09T13:33:00.706 INFO:teuthology.orchestra.run.smithi043.stdout: osd crush chooseleaf type = 0 2023-08-09T13:33:00.706 INFO:teuthology.orchestra.run.smithi043.stdout: auth debug = true 2023-08-09T13:33:00.706 INFO:teuthology.orchestra.run.smithi043.stdout: 2023-08-09T13:33:00.706 INFO:teuthology.orchestra.run.smithi043.stdout: ms die on old message = true 2023-08-09T13:33:00.706 INFO:teuthology.orchestra.run.smithi043.stdout: ms die on bug = true 2023-08-09T13:33:00.707 INFO:teuthology.orchestra.run.smithi043.stdout: 2023-08-09T13:33:00.707 INFO:teuthology.orchestra.run.smithi043.stdout: mon max pg per osd = 10000 # >= luminous 2023-08-09T13:33:00.707 INFO:teuthology.orchestra.run.smithi043.stdout: mon pg warn max object skew = 0 2023-08-09T13:33:00.707 INFO:teuthology.orchestra.run.smithi043.stdout: 2023-08-09T13:33:00.707 INFO:teuthology.orchestra.run.smithi043.stdout: # disable pg_autoscaler by default for new pools 2023-08-09T13:33:00.707 INFO:teuthology.orchestra.run.smithi043.stdout: osd_pool_default_pg_autoscale_mode = off 2023-08-09T13:33:00.707 INFO:teuthology.orchestra.run.smithi043.stdout: 2023-08-09T13:33:00.708 INFO:teuthology.orchestra.run.smithi043.stdout: osd pool default size = 2 2023-08-09T13:33:00.708 INFO:teuthology.orchestra.run.smithi043.stdout: 2023-08-09T13:33:00.708 INFO:teuthology.orchestra.run.smithi043.stdout: mon osd allow primary affinity = true 2023-08-09T13:33:00.708 INFO:teuthology.orchestra.run.smithi043.stdout: mon osd allow pg remap = true 2023-08-09T13:33:00.708 INFO:teuthology.orchestra.run.smithi043.stdout: mon warn on legacy crush tunables = false 2023-08-09T13:33:00.708 INFO:teuthology.orchestra.run.smithi043.stdout: mon warn on crush straw calc version zero = false 2023-08-09T13:33:00.708 INFO:teuthology.orchestra.run.smithi043.stdout: mon warn on no sortbitwise = false 2023-08-09T13:33:00.708 INFO:teuthology.orchestra.run.smithi043.stdout: mon warn on osd down out interval zero = false 2023-08-09T13:33:00.709 INFO:teuthology.orchestra.run.smithi043.stdout: mon warn on too few osds = false 2023-08-09T13:33:00.709 INFO:teuthology.orchestra.run.smithi043.stdout: mon_warn_on_pool_pg_num_not_power_of_two = false 2023-08-09T13:33:00.709 INFO:teuthology.orchestra.run.smithi043.stdout: mon_warn_on_pool_no_redundancy = false 2023-08-09T13:33:00.709 INFO:teuthology.orchestra.run.smithi043.stdout: mon_allow_pool_size_one = true 2023-08-09T13:33:00.709 INFO:teuthology.orchestra.run.smithi043.stdout: 2023-08-09T13:33:00.709 INFO:teuthology.orchestra.run.smithi043.stdout: osd pool default erasure code profile = plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd 2023-08-09T13:33:00.709 INFO:teuthology.orchestra.run.smithi043.stdout: 2023-08-09T13:33:00.709 INFO:teuthology.orchestra.run.smithi043.stdout: osd default data pool replay window = 5 2023-08-09T13:33:00.710 INFO:teuthology.orchestra.run.smithi043.stdout: 2023-08-09T13:33:00.710 INFO:teuthology.orchestra.run.smithi043.stdout: mon allow pool delete = true 2023-08-09T13:33:00.710 INFO:teuthology.orchestra.run.smithi043.stdout: 2023-08-09T13:33:00.710 INFO:teuthology.orchestra.run.smithi043.stdout: mon cluster log file level = debug 2023-08-09T13:33:00.710 INFO:teuthology.orchestra.run.smithi043.stdout: debug asserts on shutdown = true 2023-08-09T13:33:00.710 INFO:teuthology.orchestra.run.smithi043.stdout: mon health detail to clog = false 2023-08-09T13:33:00.710 INFO:teuthology.orchestra.run.smithi043.stdout: mon host = "172.21.15.43,172.21.15.203,[v2:172.21.15.203:3301,v1:172.21.15.203:6790]" 2023-08-09T13:33:00.710 INFO:teuthology.orchestra.run.smithi043.stdout: fsid = daac87ee-1337-4250-a955-aea4e408a39b 2023-08-09T13:33:00.711 INFO:teuthology.orchestra.run.smithi043.stdout: 2023-08-09T13:33:00.711 INFO:teuthology.orchestra.run.smithi043.stdout:[osd] 2023-08-09T13:33:00.711 INFO:teuthology.orchestra.run.smithi043.stdout: osd journal size = 100 2023-08-09T13:33:00.711 INFO:teuthology.orchestra.run.smithi043.stdout: 2023-08-09T13:33:00.711 INFO:teuthology.orchestra.run.smithi043.stdout: osd scrub load threshold = 5.0 2023-08-09T13:33:00.711 INFO:teuthology.orchestra.run.smithi043.stdout: osd scrub max interval = 600 2023-08-09T13:33:00.711 INFO:teuthology.orchestra.run.smithi043.stdout: osd mclock profile = high_recovery_ops 2023-08-09T13:33:00.711 INFO:teuthology.orchestra.run.smithi043.stdout: 2023-08-09T13:33:00.712 INFO:teuthology.orchestra.run.smithi043.stdout: osd recover clone overlap = true 2023-08-09T13:33:00.712 INFO:teuthology.orchestra.run.smithi043.stdout: osd recovery max chunk = 1048576 2023-08-09T13:33:00.712 INFO:teuthology.orchestra.run.smithi043.stdout: 2023-08-09T13:33:00.712 INFO:teuthology.orchestra.run.smithi043.stdout: osd debug shutdown = true 2023-08-09T13:33:00.712 INFO:teuthology.orchestra.run.smithi043.stdout: osd debug op order = true 2023-08-09T13:33:00.712 INFO:teuthology.orchestra.run.smithi043.stdout: osd debug verify stray on activate = true 2023-08-09T13:33:00.712 INFO:teuthology.orchestra.run.smithi043.stdout: 2023-08-09T13:33:00.713 INFO:teuthology.orchestra.run.smithi043.stdout: osd open classes on start = true 2023-08-09T13:33:00.713 INFO:teuthology.orchestra.run.smithi043.stdout: osd debug pg log writeout = true 2023-08-09T13:33:00.713 INFO:teuthology.orchestra.run.smithi043.stdout: 2023-08-09T13:33:00.713 INFO:teuthology.orchestra.run.smithi043.stdout: osd deep scrub update digest min age = 30 2023-08-09T13:33:00.713 INFO:teuthology.orchestra.run.smithi043.stdout: 2023-08-09T13:33:00.713 INFO:teuthology.orchestra.run.smithi043.stdout: osd map max advance = 10 2023-08-09T13:33:00.713 INFO:teuthology.orchestra.run.smithi043.stdout: 2023-08-09T13:33:00.713 INFO:teuthology.orchestra.run.smithi043.stdout: journal zero on create = true 2023-08-09T13:33:00.714 INFO:teuthology.orchestra.run.smithi043.stdout: 2023-08-09T13:33:00.714 INFO:teuthology.orchestra.run.smithi043.stdout: filestore ondisk finisher threads = 3 2023-08-09T13:33:00.714 INFO:teuthology.orchestra.run.smithi043.stdout: filestore apply finisher threads = 3 2023-08-09T13:33:00.714 INFO:teuthology.orchestra.run.smithi043.stdout: 2023-08-09T13:33:00.714 INFO:teuthology.orchestra.run.smithi043.stdout: bdev debug aio = true 2023-08-09T13:33:00.714 INFO:teuthology.orchestra.run.smithi043.stdout: osd debug misdirected ops = true 2023-08-09T13:33:00.714 INFO:teuthology.orchestra.run.smithi043.stdout: bluestore block size = 96636764160 2023-08-09T13:33:00.714 INFO:teuthology.orchestra.run.smithi043.stdout: bluestore fsck on mount = True 2023-08-09T13:33:00.715 INFO:teuthology.orchestra.run.smithi043.stdout: debug bluefs = 20 2023-08-09T13:33:00.715 INFO:teuthology.orchestra.run.smithi043.stdout: debug bluestore = 20 2023-08-09T13:33:00.715 INFO:teuthology.orchestra.run.smithi043.stdout: debug ms = 1 2023-08-09T13:33:00.715 INFO:teuthology.orchestra.run.smithi043.stdout: debug osd = 20 2023-08-09T13:33:00.715 INFO:teuthology.orchestra.run.smithi043.stdout: debug rocksdb = 10 2023-08-09T13:33:00.715 INFO:teuthology.orchestra.run.smithi043.stdout: mon osd backfillfull_ratio = 0.85 2023-08-09T13:33:00.715 INFO:teuthology.orchestra.run.smithi043.stdout: mon osd full ratio = 0.9 2023-08-09T13:33:00.715 INFO:teuthology.orchestra.run.smithi043.stdout: mon osd nearfull ratio = 0.8 2023-08-09T13:33:00.716 INFO:teuthology.orchestra.run.smithi043.stdout: osd failsafe full ratio = 0.95 2023-08-09T13:33:00.716 INFO:teuthology.orchestra.run.smithi043.stdout: osd objectstore = bluestore 2023-08-09T13:33:00.716 INFO:teuthology.orchestra.run.smithi043.stdout: osd op complaint time = 180 2023-08-09T13:33:00.716 INFO:teuthology.orchestra.run.smithi043.stdout: 2023-08-09T13:33:00.716 INFO:teuthology.orchestra.run.smithi043.stdout:[mgr] 2023-08-09T13:33:00.716 INFO:teuthology.orchestra.run.smithi043.stdout: debug ms = 1 2023-08-09T13:33:00.716 INFO:teuthology.orchestra.run.smithi043.stdout: debug mgr = 20 2023-08-09T13:33:00.716 INFO:teuthology.orchestra.run.smithi043.stdout: debug mon = 20 2023-08-09T13:33:00.717 INFO:teuthology.orchestra.run.smithi043.stdout: debug auth = 20 2023-08-09T13:33:00.717 INFO:teuthology.orchestra.run.smithi043.stdout: mon reweight min pgs per osd = 4 2023-08-09T13:33:00.717 INFO:teuthology.orchestra.run.smithi043.stdout: mon reweight min bytes per osd = 10 2023-08-09T13:33:00.717 INFO:teuthology.orchestra.run.smithi043.stdout: mgr/telemetry/nag = false 2023-08-09T13:33:00.717 INFO:teuthology.orchestra.run.smithi043.stdout: debug client = 20 2023-08-09T13:33:00.717 INFO:teuthology.orchestra.run.smithi043.stdout: debug finisher = 20 2023-08-09T13:33:00.717 INFO:teuthology.orchestra.run.smithi043.stdout: 2023-08-09T13:33:00.717 INFO:teuthology.orchestra.run.smithi043.stdout:[mon] 2023-08-09T13:33:00.718 INFO:teuthology.orchestra.run.smithi043.stdout: debug ms = 1 2023-08-09T13:33:00.718 INFO:teuthology.orchestra.run.smithi043.stdout: debug mon = 20 2023-08-09T13:33:00.718 INFO:teuthology.orchestra.run.smithi043.stdout: debug paxos = 20 2023-08-09T13:33:00.718 INFO:teuthology.orchestra.run.smithi043.stdout: debug auth = 20 2023-08-09T13:33:00.718 INFO:teuthology.orchestra.run.smithi043.stdout: mon data avail warn = 5 2023-08-09T13:33:00.718 INFO:teuthology.orchestra.run.smithi043.stdout: mon mgr mkfs grace = 240 2023-08-09T13:33:00.718 INFO:teuthology.orchestra.run.smithi043.stdout: mon reweight min pgs per osd = 4 2023-08-09T13:33:00.718 INFO:teuthology.orchestra.run.smithi043.stdout: mon osd reporter subtree level = osd 2023-08-09T13:33:00.719 INFO:teuthology.orchestra.run.smithi043.stdout: mon osd prime pg temp = true 2023-08-09T13:33:00.719 INFO:teuthology.orchestra.run.smithi043.stdout: mon reweight min bytes per osd = 10 2023-08-09T13:33:00.719 INFO:teuthology.orchestra.run.smithi043.stdout: 2023-08-09T13:33:00.719 INFO:teuthology.orchestra.run.smithi043.stdout: # rotate auth tickets quickly to exercise renewal paths 2023-08-09T13:33:00.719 INFO:teuthology.orchestra.run.smithi043.stdout: auth mon ticket ttl = 660 # 11m 2023-08-09T13:33:00.719 INFO:teuthology.orchestra.run.smithi043.stdout: auth service ticket ttl = 240 # 4m 2023-08-09T13:33:00.719 INFO:teuthology.orchestra.run.smithi043.stdout: 2023-08-09T13:33:00.719 INFO:teuthology.orchestra.run.smithi043.stdout: # don't complain about insecure global_id in the test suite 2023-08-09T13:33:00.720 INFO:teuthology.orchestra.run.smithi043.stdout: mon_warn_on_insecure_global_id_reclaim = false 2023-08-09T13:33:00.720 INFO:teuthology.orchestra.run.smithi043.stdout: mon_warn_on_insecure_global_id_reclaim_allowed = false 2023-08-09T13:33:00.720 INFO:teuthology.orchestra.run.smithi043.stdout: 2023-08-09T13:33:00.720 INFO:teuthology.orchestra.run.smithi043.stdout: # 1m isn't quite enough 2023-08-09T13:33:00.720 INFO:teuthology.orchestra.run.smithi043.stdout: mon_down_mkfs_grace = 2m 2023-08-09T13:33:00.720 INFO:teuthology.orchestra.run.smithi043.stdout: 2023-08-09T13:33:00.720 INFO:teuthology.orchestra.run.smithi043.stdout: mon_warn_on_filestore_osds = false 2023-08-09T13:33:00.720 INFO:teuthology.orchestra.run.smithi043.stdout: mon op complaint time = 120 2023-08-09T13:33:00.721 INFO:teuthology.orchestra.run.smithi043.stdout: 2023-08-09T13:33:00.721 INFO:teuthology.orchestra.run.smithi043.stdout:[client] 2023-08-09T13:33:00.721 INFO:teuthology.orchestra.run.smithi043.stdout: rgw cache enabled = true 2023-08-09T13:33:00.721 INFO:teuthology.orchestra.run.smithi043.stdout: rgw enable ops log = true 2023-08-09T13:33:00.721 INFO:teuthology.orchestra.run.smithi043.stdout: rgw enable usage log = true 2023-08-09T13:33:00.721 INFO:teuthology.orchestra.run.smithi043.stdout: log file = /var/log/ceph/$cluster-$name.$pid.log 2023-08-09T13:33:00.721 INFO:teuthology.orchestra.run.smithi043.stdout: admin socket = /var/run/ceph/$cluster-$name.$pid.asok 2023-08-09T13:33:00.721 INFO:teuthology.orchestra.run.smithi043.stdout: client mount timeout = 600 2023-08-09T13:33:00.722 INFO:teuthology.orchestra.run.smithi043.stdout: debug client = 20 2023-08-09T13:33:00.722 INFO:teuthology.orchestra.run.smithi043.stdout: debug ms = 1 2023-08-09T13:33:00.722 INFO:teuthology.orchestra.run.smithi043.stdout: pid file = "" 2023-08-09T13:33:00.722 INFO:teuthology.orchestra.run.smithi043.stdout: rados mon op timeout = 900 2023-08-09T13:33:00.722 INFO:teuthology.orchestra.run.smithi043.stdout: rados osd op timeout = 900 2023-08-09T13:33:00.722 INFO:teuthology.orchestra.run.smithi043.stdout:[mon.a] 2023-08-09T13:33:00.722 INFO:teuthology.orchestra.run.smithi043.stdout:[mon.b] 2023-08-09T13:33:00.722 INFO:teuthology.orchestra.run.smithi043.stdout:[mon.c] 2023-08-09T13:33:00.723 INFO:teuthology.orchestra.run.smithi043.stdout:[mds.a] 2023-08-09T13:33:00.723 INFO:teuthology.orchestra.run.smithi043.stdout:[mds.c] 2023-08-09T13:33:00.723 INFO:teuthology.orchestra.run.smithi043.stdout:[mds.b] 2023-08-09T13:33:00.723 INFO:teuthology.orchestra.run.smithi043.stdout:[mds.d] 2023-08-09T13:33:00.723 INFO:teuthology.orchestra.run.smithi043.stdout:[mds] 2023-08-09T13:33:00.723 INFO:teuthology.orchestra.run.smithi043.stdout: debug mds = 20 2023-08-09T13:33:00.723 INFO:teuthology.orchestra.run.smithi043.stdout: debug mds balancer = 20 2023-08-09T13:33:00.723 INFO:teuthology.orchestra.run.smithi043.stdout: debug ms = 1 2023-08-09T13:33:00.724 INFO:teuthology.orchestra.run.smithi043.stdout: mds debug frag = True 2023-08-09T13:33:00.724 INFO:teuthology.orchestra.run.smithi043.stdout: mds debug scatterstat = True 2023-08-09T13:33:00.724 INFO:teuthology.orchestra.run.smithi043.stdout: mds op complaint time = 180 2023-08-09T13:33:00.724 INFO:teuthology.orchestra.run.smithi043.stdout: mds verify scatter = True 2023-08-09T13:33:00.724 INFO:teuthology.orchestra.run.smithi043.stdout: osd op complaint time = 180 2023-08-09T13:33:00.724 INFO:teuthology.orchestra.run.smithi043.stdout: rados mon op timeout = 900 2023-08-09T13:33:00.724 INFO:teuthology.orchestra.run.smithi043.stdout: rados osd op timeout = 900 2023-08-09T13:33:00.755 INFO:tasks.ceph:Creating admin key on mon.a... 2023-08-09T13:33:00.756 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --gen-key --name=client.admin --cap mon 'allow *' --cap osd 'allow *' --cap mds 'allow *' --cap mgr 'allow *' /etc/ceph/ceph.keyring 2023-08-09T13:33:00.909 INFO:tasks.ceph:Copying monmap to all nodes... 2023-08-09T13:33:00.910 DEBUG:teuthology.orchestra.run.smithi043:> set -ex 2023-08-09T13:33:00.910 DEBUG:teuthology.orchestra.run.smithi043:> dd if=/etc/ceph/ceph.keyring of=/dev/stdout 2023-08-09T13:33:00.961 DEBUG:teuthology.orchestra.run.smithi043:> set -ex 2023-08-09T13:33:00.961 DEBUG:teuthology.orchestra.run.smithi043:> dd if=/home/ubuntu/cephtest/ceph.monmap of=/dev/stdout 2023-08-09T13:33:01.016 INFO:tasks.ceph:Sending monmap to node ubuntu@smithi043.front.sepia.ceph.com 2023-08-09T13:33:01.016 DEBUG:teuthology.orchestra.run.smithi043:> set -ex 2023-08-09T13:33:01.016 DEBUG:teuthology.orchestra.run.smithi043:> sudo dd of=/etc/ceph/ceph.keyring 2023-08-09T13:33:01.017 DEBUG:teuthology.orchestra.run.smithi043:> sudo chmod 0644 /etc/ceph/ceph.keyring 2023-08-09T13:33:01.164 DEBUG:teuthology.orchestra.run.smithi043:> set -ex 2023-08-09T13:33:01.164 DEBUG:teuthology.orchestra.run.smithi043:> dd of=/home/ubuntu/cephtest/ceph.monmap 2023-08-09T13:33:01.265 INFO:tasks.ceph:Sending monmap to node ubuntu@smithi203.front.sepia.ceph.com 2023-08-09T13:33:01.266 DEBUG:teuthology.orchestra.run.smithi203:> set -ex 2023-08-09T13:33:01.266 DEBUG:teuthology.orchestra.run.smithi203:> sudo dd of=/etc/ceph/ceph.keyring 2023-08-09T13:33:01.266 DEBUG:teuthology.orchestra.run.smithi203:> sudo chmod 0644 /etc/ceph/ceph.keyring 2023-08-09T13:33:01.427 DEBUG:teuthology.orchestra.run.smithi203:> set -ex 2023-08-09T13:33:01.427 DEBUG:teuthology.orchestra.run.smithi203:> dd of=/home/ubuntu/cephtest/ceph.monmap 2023-08-09T13:33:01.527 INFO:tasks.ceph:Setting up mon nodes... 2023-08-09T13:33:01.527 INFO:tasks.ceph:Setting up mgr nodes... 2023-08-09T13:33:01.528 DEBUG:teuthology.orchestra.run.smithi043:> sudo mkdir -p /var/lib/ceph/mgr/ceph-y && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mgr.y /var/lib/ceph/mgr/ceph-y/keyring 2023-08-09T13:33:01.742 INFO:teuthology.orchestra.run.smithi043.stdout:creating /var/lib/ceph/mgr/ceph-y/keyring 2023-08-09T13:33:01.752 DEBUG:teuthology.orchestra.run.smithi203:> sudo mkdir -p /var/lib/ceph/mgr/ceph-x && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mgr.x /var/lib/ceph/mgr/ceph-x/keyring 2023-08-09T13:33:01.937 INFO:teuthology.orchestra.run.smithi203.stdout:creating /var/lib/ceph/mgr/ceph-x/keyring 2023-08-09T13:33:01.947 INFO:tasks.ceph:Setting up mds nodes... 2023-08-09T13:33:01.948 DEBUG:teuthology.orchestra.run.smithi043:> sudo mkdir -p /var/lib/ceph/mds/ceph-a && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mds.a /var/lib/ceph/mds/ceph-a/keyring 2023-08-09T13:33:02.160 INFO:teuthology.orchestra.run.smithi043.stdout:creating /var/lib/ceph/mds/ceph-a/keyring 2023-08-09T13:33:02.168 DEBUG:teuthology.orchestra.run.smithi043:> sudo chown -R ceph:ceph /var/lib/ceph/mds/ceph-a 2023-08-09T13:33:02.256 DEBUG:teuthology.orchestra.run.smithi043:> sudo mkdir -p /var/lib/ceph/mds/ceph-c && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mds.c /var/lib/ceph/mds/ceph-c/keyring 2023-08-09T13:33:02.441 INFO:teuthology.orchestra.run.smithi043.stdout:creating /var/lib/ceph/mds/ceph-c/keyring 2023-08-09T13:33:02.450 DEBUG:teuthology.orchestra.run.smithi043:> sudo chown -R ceph:ceph /var/lib/ceph/mds/ceph-c 2023-08-09T13:33:02.544 DEBUG:teuthology.orchestra.run.smithi203:> sudo mkdir -p /var/lib/ceph/mds/ceph-b && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mds.b /var/lib/ceph/mds/ceph-b/keyring 2023-08-09T13:33:02.734 INFO:teuthology.orchestra.run.smithi203.stdout:creating /var/lib/ceph/mds/ceph-b/keyring 2023-08-09T13:33:02.743 DEBUG:teuthology.orchestra.run.smithi203:> sudo chown -R ceph:ceph /var/lib/ceph/mds/ceph-b 2023-08-09T13:33:02.840 DEBUG:teuthology.orchestra.run.smithi203:> sudo mkdir -p /var/lib/ceph/mds/ceph-d && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mds.d /var/lib/ceph/mds/ceph-d/keyring 2023-08-09T13:33:03.019 INFO:teuthology.orchestra.run.smithi203.stdout:creating /var/lib/ceph/mds/ceph-d/keyring 2023-08-09T13:33:03.028 DEBUG:teuthology.orchestra.run.smithi203:> sudo chown -R ceph:ceph /var/lib/ceph/mds/ceph-d 2023-08-09T13:33:03.121 INFO:tasks.ceph_client:Setting up client nodes... 2023-08-09T13:33:03.121 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=client.0 /etc/ceph/ceph.client.0.keyring && sudo chmod 0644 /etc/ceph/ceph.client.0.keyring 2023-08-09T13:33:03.268 INFO:teuthology.orchestra.run.smithi043.stdout:creating /etc/ceph/ceph.client.0.keyring 2023-08-09T13:33:03.323 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=client.2 /etc/ceph/ceph.client.2.keyring && sudo chmod 0644 /etc/ceph/ceph.client.2.keyring 2023-08-09T13:33:03.460 INFO:teuthology.orchestra.run.smithi043.stdout:creating /etc/ceph/ceph.client.2.keyring 2023-08-09T13:33:03.525 DEBUG:teuthology.orchestra.run.smithi203:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=client.1 /etc/ceph/ceph.client.1.keyring && sudo chmod 0644 /etc/ceph/ceph.client.1.keyring 2023-08-09T13:33:03.662 INFO:teuthology.orchestra.run.smithi203.stdout:creating /etc/ceph/ceph.client.1.keyring 2023-08-09T13:33:03.723 DEBUG:teuthology.orchestra.run.smithi203:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=client.3 /etc/ceph/ceph.client.3.keyring && sudo chmod 0644 /etc/ceph/ceph.client.3.keyring 2023-08-09T13:33:03.853 INFO:teuthology.orchestra.run.smithi203.stdout:creating /etc/ceph/ceph.client.3.keyring 2023-08-09T13:33:03.910 INFO:tasks.ceph:Running mkfs on osd nodes... 2023-08-09T13:33:03.911 INFO:tasks.ceph:ctx.disk_config.remote_to_roles_to_dev: {Remote(name='ubuntu@smithi043.front.sepia.ceph.com'): {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'}, Remote(name='ubuntu@smithi203.front.sepia.ceph.com'): {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'}} 2023-08-09T13:33:03.911 DEBUG:teuthology.orchestra.run.smithi043:> sudo mkdir -p /var/lib/ceph/osd/ceph-0 2023-08-09T13:33:04.020 INFO:tasks.ceph:roles_to_devs: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'} 2023-08-09T13:33:04.020 INFO:tasks.ceph:role: osd.0 2023-08-09T13:33:04.021 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_1 on ubuntu@smithi043.front.sepia.ceph.com 2023-08-09T13:33:04.021 DEBUG:teuthology.orchestra.run.smithi043:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_1 2023-08-09T13:33:04.120 INFO:teuthology.orchestra.run.smithi043.stdout:meta-data=/dev/vg_nvme/lv_1 isize=2048 agcount=4, agsize=5859072 blks 2023-08-09T13:33:04.120 INFO:teuthology.orchestra.run.smithi043.stdout: = sectsz=512 attr=2, projid32bit=1 2023-08-09T13:33:04.121 INFO:teuthology.orchestra.run.smithi043.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2023-08-09T13:33:04.121 INFO:teuthology.orchestra.run.smithi043.stdout: = reflink=1 bigtime=1 inobtcount=1 nrext64=0 2023-08-09T13:33:04.121 INFO:teuthology.orchestra.run.smithi043.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2023-08-09T13:33:04.121 INFO:teuthology.orchestra.run.smithi043.stdout: = sunit=0 swidth=0 blks 2023-08-09T13:33:04.121 INFO:teuthology.orchestra.run.smithi043.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2023-08-09T13:33:04.121 INFO:teuthology.orchestra.run.smithi043.stdout:log =internal log bsize=4096 blocks=16384, version=2 2023-08-09T13:33:04.121 INFO:teuthology.orchestra.run.smithi043.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2023-08-09T13:33:04.122 INFO:teuthology.orchestra.run.smithi043.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2023-08-09T13:33:04.154 INFO:teuthology.orchestra.run.smithi043.stdout:Discarding blocks...Done. 2023-08-09T13:33:04.159 INFO:tasks.ceph:mount /dev/vg_nvme/lv_1 on ubuntu@smithi043.front.sepia.ceph.com -o noatime 2023-08-09T13:33:04.160 DEBUG:teuthology.orchestra.run.smithi043:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_1 /var/lib/ceph/osd/ceph-0 2023-08-09T13:33:04.276 DEBUG:teuthology.orchestra.run.smithi043:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-0 2023-08-09T13:33:04.387 DEBUG:teuthology.orchestra.run.smithi043:> sudo mkdir -p /var/lib/ceph/osd/ceph-1 2023-08-09T13:33:04.489 INFO:tasks.ceph:roles_to_devs: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'} 2023-08-09T13:33:04.489 INFO:tasks.ceph:role: osd.1 2023-08-09T13:33:04.489 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_2 on ubuntu@smithi043.front.sepia.ceph.com 2023-08-09T13:33:04.490 DEBUG:teuthology.orchestra.run.smithi043:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_2 2023-08-09T13:33:04.589 INFO:teuthology.orchestra.run.smithi043.stdout:meta-data=/dev/vg_nvme/lv_2 isize=2048 agcount=4, agsize=5859072 blks 2023-08-09T13:33:04.589 INFO:teuthology.orchestra.run.smithi043.stdout: = sectsz=512 attr=2, projid32bit=1 2023-08-09T13:33:04.590 INFO:teuthology.orchestra.run.smithi043.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2023-08-09T13:33:04.590 INFO:teuthology.orchestra.run.smithi043.stdout: = reflink=1 bigtime=1 inobtcount=1 nrext64=0 2023-08-09T13:33:04.590 INFO:teuthology.orchestra.run.smithi043.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2023-08-09T13:33:04.590 INFO:teuthology.orchestra.run.smithi043.stdout: = sunit=0 swidth=0 blks 2023-08-09T13:33:04.590 INFO:teuthology.orchestra.run.smithi043.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2023-08-09T13:33:04.590 INFO:teuthology.orchestra.run.smithi043.stdout:log =internal log bsize=4096 blocks=16384, version=2 2023-08-09T13:33:04.590 INFO:teuthology.orchestra.run.smithi043.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2023-08-09T13:33:04.591 INFO:teuthology.orchestra.run.smithi043.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2023-08-09T13:33:04.624 INFO:teuthology.orchestra.run.smithi043.stdout:Discarding blocks...Done. 2023-08-09T13:33:04.629 INFO:tasks.ceph:mount /dev/vg_nvme/lv_2 on ubuntu@smithi043.front.sepia.ceph.com -o noatime 2023-08-09T13:33:04.630 DEBUG:teuthology.orchestra.run.smithi043:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_2 /var/lib/ceph/osd/ceph-1 2023-08-09T13:33:04.752 DEBUG:teuthology.orchestra.run.smithi043:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-1 2023-08-09T13:33:04.836 DEBUG:teuthology.orchestra.run.smithi043:> sudo mkdir -p /var/lib/ceph/osd/ceph-2 2023-08-09T13:33:04.916 INFO:tasks.ceph:roles_to_devs: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'} 2023-08-09T13:33:04.916 INFO:tasks.ceph:role: osd.2 2023-08-09T13:33:04.916 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_3 on ubuntu@smithi043.front.sepia.ceph.com 2023-08-09T13:33:04.917 DEBUG:teuthology.orchestra.run.smithi043:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_3 2023-08-09T13:33:05.010 INFO:teuthology.orchestra.run.smithi043.stdout:meta-data=/dev/vg_nvme/lv_3 isize=2048 agcount=4, agsize=5859072 blks 2023-08-09T13:33:05.010 INFO:teuthology.orchestra.run.smithi043.stdout: = sectsz=512 attr=2, projid32bit=1 2023-08-09T13:33:05.010 INFO:teuthology.orchestra.run.smithi043.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2023-08-09T13:33:05.010 INFO:teuthology.orchestra.run.smithi043.stdout: = reflink=1 bigtime=1 inobtcount=1 nrext64=0 2023-08-09T13:33:05.011 INFO:teuthology.orchestra.run.smithi043.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2023-08-09T13:33:05.011 INFO:teuthology.orchestra.run.smithi043.stdout: = sunit=0 swidth=0 blks 2023-08-09T13:33:05.011 INFO:teuthology.orchestra.run.smithi043.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2023-08-09T13:33:05.011 INFO:teuthology.orchestra.run.smithi043.stdout:log =internal log bsize=4096 blocks=16384, version=2 2023-08-09T13:33:05.011 INFO:teuthology.orchestra.run.smithi043.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2023-08-09T13:33:05.011 INFO:teuthology.orchestra.run.smithi043.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2023-08-09T13:33:05.044 INFO:teuthology.orchestra.run.smithi043.stdout:Discarding blocks...Done. 2023-08-09T13:33:05.050 INFO:tasks.ceph:mount /dev/vg_nvme/lv_3 on ubuntu@smithi043.front.sepia.ceph.com -o noatime 2023-08-09T13:33:05.050 DEBUG:teuthology.orchestra.run.smithi043:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_3 /var/lib/ceph/osd/ceph-2 2023-08-09T13:33:05.181 DEBUG:teuthology.orchestra.run.smithi043:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-2 2023-08-09T13:33:05.274 DEBUG:teuthology.orchestra.run.smithi043:> sudo mkdir -p /var/lib/ceph/osd/ceph-3 2023-08-09T13:33:05.364 INFO:tasks.ceph:roles_to_devs: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'} 2023-08-09T13:33:05.365 INFO:tasks.ceph:role: osd.3 2023-08-09T13:33:05.365 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_4 on ubuntu@smithi043.front.sepia.ceph.com 2023-08-09T13:33:05.365 DEBUG:teuthology.orchestra.run.smithi043:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_4 2023-08-09T13:33:05.468 INFO:teuthology.orchestra.run.smithi043.stdout:meta-data=/dev/vg_nvme/lv_4 isize=2048 agcount=4, agsize=5859072 blks 2023-08-09T13:33:05.469 INFO:teuthology.orchestra.run.smithi043.stdout: = sectsz=512 attr=2, projid32bit=1 2023-08-09T13:33:05.469 INFO:teuthology.orchestra.run.smithi043.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2023-08-09T13:33:05.469 INFO:teuthology.orchestra.run.smithi043.stdout: = reflink=1 bigtime=1 inobtcount=1 nrext64=0 2023-08-09T13:33:05.469 INFO:teuthology.orchestra.run.smithi043.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2023-08-09T13:33:05.469 INFO:teuthology.orchestra.run.smithi043.stdout: = sunit=0 swidth=0 blks 2023-08-09T13:33:05.469 INFO:teuthology.orchestra.run.smithi043.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2023-08-09T13:33:05.470 INFO:teuthology.orchestra.run.smithi043.stdout:log =internal log bsize=4096 blocks=16384, version=2 2023-08-09T13:33:05.470 INFO:teuthology.orchestra.run.smithi043.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2023-08-09T13:33:05.470 INFO:teuthology.orchestra.run.smithi043.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2023-08-09T13:33:05.502 INFO:teuthology.orchestra.run.smithi043.stdout:Discarding blocks...Done. 2023-08-09T13:33:05.509 INFO:tasks.ceph:mount /dev/vg_nvme/lv_4 on ubuntu@smithi043.front.sepia.ceph.com -o noatime 2023-08-09T13:33:05.510 DEBUG:teuthology.orchestra.run.smithi043:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_4 /var/lib/ceph/osd/ceph-3 2023-08-09T13:33:05.616 DEBUG:teuthology.orchestra.run.smithi043:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-3 2023-08-09T13:33:05.698 DEBUG:teuthology.orchestra.run.smithi043:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 0 --monmap /home/ubuntu/cephtest/ceph.monmap 2023-08-09T13:33:05.854 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:33:05.852+0000 7f03ba3a7500 -1 auth: error reading file: /var/lib/ceph/osd/ceph-0/keyring: can't open /var/lib/ceph/osd/ceph-0/keyring: (2) No such file or directory 2023-08-09T13:33:05.855 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:33:05.853+0000 7f03ba3a7500 -1 created new key in keyring /var/lib/ceph/osd/ceph-0/keyring 2023-08-09T13:33:05.856 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:33:05.853+0000 7f03ba3a7500 -1 bluestore(/var/lib/ceph/osd/ceph-0/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-0/block: (2) No such file or directory 2023-08-09T13:33:05.856 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:33:05.853+0000 7f03ba3a7500 -1 bluestore(/var/lib/ceph/osd/ceph-0/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-0/block: (2) No such file or directory 2023-08-09T13:33:05.856 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:33:05.853+0000 7f03ba3a7500 -1 bluestore(/var/lib/ceph/osd/ceph-0/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-0/block: (2) No such file or directory 2023-08-09T13:33:05.858 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:33:05.856+0000 7f03ba3a7500 -1 bluestore(/var/lib/ceph/osd/ceph-0) _read_fsid unparsable uuid 2023-08-09T13:33:09.047 DEBUG:teuthology.orchestra.run.smithi043:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-0 2023-08-09T13:33:09.160 DEBUG:teuthology.orchestra.run.smithi043:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 1 --monmap /home/ubuntu/cephtest/ceph.monmap 2023-08-09T13:33:09.313 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:33:09.310+0000 7fb10ef89500 -1 auth: error reading file: /var/lib/ceph/osd/ceph-1/keyring: can't open /var/lib/ceph/osd/ceph-1/keyring: (2) No such file or directory 2023-08-09T13:33:09.313 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:33:09.311+0000 7fb10ef89500 -1 created new key in keyring /var/lib/ceph/osd/ceph-1/keyring 2023-08-09T13:33:09.314 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:33:09.311+0000 7fb10ef89500 -1 bluestore(/var/lib/ceph/osd/ceph-1/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-1/block: (2) No such file or directory 2023-08-09T13:33:09.314 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:33:09.311+0000 7fb10ef89500 -1 bluestore(/var/lib/ceph/osd/ceph-1/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-1/block: (2) No such file or directory 2023-08-09T13:33:09.314 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:33:09.311+0000 7fb10ef89500 -1 bluestore(/var/lib/ceph/osd/ceph-1/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-1/block: (2) No such file or directory 2023-08-09T13:33:09.315 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:33:09.313+0000 7fb10ef89500 -1 bluestore(/var/lib/ceph/osd/ceph-1) _read_fsid unparsable uuid 2023-08-09T13:33:12.243 DEBUG:teuthology.orchestra.run.smithi043:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-1 2023-08-09T13:33:12.353 DEBUG:teuthology.orchestra.run.smithi043:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 2 --monmap /home/ubuntu/cephtest/ceph.monmap 2023-08-09T13:33:12.505 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:33:12.502+0000 7f0f99f9f500 -1 auth: error reading file: /var/lib/ceph/osd/ceph-2/keyring: can't open /var/lib/ceph/osd/ceph-2/keyring: (2) No such file or directory 2023-08-09T13:33:12.506 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:33:12.503+0000 7f0f99f9f500 -1 created new key in keyring /var/lib/ceph/osd/ceph-2/keyring 2023-08-09T13:33:12.506 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:33:12.503+0000 7f0f99f9f500 -1 bluestore(/var/lib/ceph/osd/ceph-2/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-2/block: (2) No such file or directory 2023-08-09T13:33:12.506 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:33:12.503+0000 7f0f99f9f500 -1 bluestore(/var/lib/ceph/osd/ceph-2/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-2/block: (2) No such file or directory 2023-08-09T13:33:12.506 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:33:12.503+0000 7f0f99f9f500 -1 bluestore(/var/lib/ceph/osd/ceph-2/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-2/block: (2) No such file or directory 2023-08-09T13:33:12.507 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:33:12.505+0000 7f0f99f9f500 -1 bluestore(/var/lib/ceph/osd/ceph-2) _read_fsid unparsable uuid 2023-08-09T13:33:15.441 DEBUG:teuthology.orchestra.run.smithi043:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-2 2023-08-09T13:33:15.544 DEBUG:teuthology.orchestra.run.smithi043:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 3 --monmap /home/ubuntu/cephtest/ceph.monmap 2023-08-09T13:33:15.701 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:33:15.698+0000 7febe7fbe500 -1 auth: error reading file: /var/lib/ceph/osd/ceph-3/keyring: can't open /var/lib/ceph/osd/ceph-3/keyring: (2) No such file or directory 2023-08-09T13:33:15.701 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:33:15.699+0000 7febe7fbe500 -1 created new key in keyring /var/lib/ceph/osd/ceph-3/keyring 2023-08-09T13:33:15.702 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:33:15.699+0000 7febe7fbe500 -1 bluestore(/var/lib/ceph/osd/ceph-3/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-3/block: (2) No such file or directory 2023-08-09T13:33:15.702 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:33:15.699+0000 7febe7fbe500 -1 bluestore(/var/lib/ceph/osd/ceph-3/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-3/block: (2) No such file or directory 2023-08-09T13:33:15.702 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:33:15.699+0000 7febe7fbe500 -1 bluestore(/var/lib/ceph/osd/ceph-3/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-3/block: (2) No such file or directory 2023-08-09T13:33:15.703 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:33:15.701+0000 7febe7fbe500 -1 bluestore(/var/lib/ceph/osd/ceph-3) _read_fsid unparsable uuid 2023-08-09T13:33:18.879 DEBUG:teuthology.orchestra.run.smithi043:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-3 2023-08-09T13:33:18.989 DEBUG:teuthology.orchestra.run.smithi203:> sudo mkdir -p /var/lib/ceph/osd/ceph-4 2023-08-09T13:33:19.096 INFO:tasks.ceph:roles_to_devs: {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'} 2023-08-09T13:33:19.096 INFO:tasks.ceph:role: osd.4 2023-08-09T13:33:19.096 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_1 on ubuntu@smithi203.front.sepia.ceph.com 2023-08-09T13:33:19.096 DEBUG:teuthology.orchestra.run.smithi203:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_1 2023-08-09T13:33:19.200 INFO:teuthology.orchestra.run.smithi203.stdout:meta-data=/dev/vg_nvme/lv_1 isize=2048 agcount=4, agsize=5859072 blks 2023-08-09T13:33:19.200 INFO:teuthology.orchestra.run.smithi203.stdout: = sectsz=512 attr=2, projid32bit=1 2023-08-09T13:33:19.201 INFO:teuthology.orchestra.run.smithi203.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2023-08-09T13:33:19.201 INFO:teuthology.orchestra.run.smithi203.stdout: = reflink=1 bigtime=1 inobtcount=1 nrext64=0 2023-08-09T13:33:19.201 INFO:teuthology.orchestra.run.smithi203.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2023-08-09T13:33:19.201 INFO:teuthology.orchestra.run.smithi203.stdout: = sunit=0 swidth=0 blks 2023-08-09T13:33:19.201 INFO:teuthology.orchestra.run.smithi203.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2023-08-09T13:33:19.201 INFO:teuthology.orchestra.run.smithi203.stdout:log =internal log bsize=4096 blocks=16384, version=2 2023-08-09T13:33:19.201 INFO:teuthology.orchestra.run.smithi203.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2023-08-09T13:33:19.201 INFO:teuthology.orchestra.run.smithi203.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2023-08-09T13:33:19.696 INFO:teuthology.orchestra.run.smithi203.stdout:Discarding blocks...Done. 2023-08-09T13:33:19.704 INFO:tasks.ceph:mount /dev/vg_nvme/lv_1 on ubuntu@smithi203.front.sepia.ceph.com -o noatime 2023-08-09T13:33:19.705 DEBUG:teuthology.orchestra.run.smithi203:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_1 /var/lib/ceph/osd/ceph-4 2023-08-09T13:33:19.873 DEBUG:teuthology.orchestra.run.smithi203:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-4 2023-08-09T13:33:20.007 DEBUG:teuthology.orchestra.run.smithi203:> sudo mkdir -p /var/lib/ceph/osd/ceph-5 2023-08-09T13:33:20.093 INFO:tasks.ceph:roles_to_devs: {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'} 2023-08-09T13:33:20.093 INFO:tasks.ceph:role: osd.5 2023-08-09T13:33:20.094 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_2 on ubuntu@smithi203.front.sepia.ceph.com 2023-08-09T13:33:20.094 DEBUG:teuthology.orchestra.run.smithi203:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_2 2023-08-09T13:33:20.186 INFO:teuthology.orchestra.run.smithi203.stdout:meta-data=/dev/vg_nvme/lv_2 isize=2048 agcount=4, agsize=5859072 blks 2023-08-09T13:33:20.186 INFO:teuthology.orchestra.run.smithi203.stdout: = sectsz=512 attr=2, projid32bit=1 2023-08-09T13:33:20.186 INFO:teuthology.orchestra.run.smithi203.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2023-08-09T13:33:20.186 INFO:teuthology.orchestra.run.smithi203.stdout: = reflink=1 bigtime=1 inobtcount=1 nrext64=0 2023-08-09T13:33:20.187 INFO:teuthology.orchestra.run.smithi203.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2023-08-09T13:33:20.187 INFO:teuthology.orchestra.run.smithi203.stdout: = sunit=0 swidth=0 blks 2023-08-09T13:33:20.187 INFO:teuthology.orchestra.run.smithi203.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2023-08-09T13:33:20.187 INFO:teuthology.orchestra.run.smithi203.stdout:log =internal log bsize=4096 blocks=16384, version=2 2023-08-09T13:33:20.188 INFO:teuthology.orchestra.run.smithi203.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2023-08-09T13:33:20.188 INFO:teuthology.orchestra.run.smithi203.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2023-08-09T13:33:20.661 INFO:teuthology.orchestra.run.smithi203.stdout:Discarding blocks...Done. 2023-08-09T13:33:20.668 INFO:tasks.ceph:mount /dev/vg_nvme/lv_2 on ubuntu@smithi203.front.sepia.ceph.com -o noatime 2023-08-09T13:33:20.668 DEBUG:teuthology.orchestra.run.smithi203:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_2 /var/lib/ceph/osd/ceph-5 2023-08-09T13:33:20.803 DEBUG:teuthology.orchestra.run.smithi203:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-5 2023-08-09T13:33:20.891 DEBUG:teuthology.orchestra.run.smithi203:> sudo mkdir -p /var/lib/ceph/osd/ceph-6 2023-08-09T13:33:20.977 INFO:tasks.ceph:roles_to_devs: {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'} 2023-08-09T13:33:20.977 INFO:tasks.ceph:role: osd.6 2023-08-09T13:33:20.977 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_3 on ubuntu@smithi203.front.sepia.ceph.com 2023-08-09T13:33:20.977 DEBUG:teuthology.orchestra.run.smithi203:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_3 2023-08-09T13:33:21.069 INFO:teuthology.orchestra.run.smithi203.stdout:meta-data=/dev/vg_nvme/lv_3 isize=2048 agcount=4, agsize=5859072 blks 2023-08-09T13:33:21.069 INFO:teuthology.orchestra.run.smithi203.stdout: = sectsz=512 attr=2, projid32bit=1 2023-08-09T13:33:21.070 INFO:teuthology.orchestra.run.smithi203.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2023-08-09T13:33:21.070 INFO:teuthology.orchestra.run.smithi203.stdout: = reflink=1 bigtime=1 inobtcount=1 nrext64=0 2023-08-09T13:33:21.070 INFO:teuthology.orchestra.run.smithi203.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2023-08-09T13:33:21.070 INFO:teuthology.orchestra.run.smithi203.stdout: = sunit=0 swidth=0 blks 2023-08-09T13:33:21.071 INFO:teuthology.orchestra.run.smithi203.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2023-08-09T13:33:21.071 INFO:teuthology.orchestra.run.smithi203.stdout:log =internal log bsize=4096 blocks=16384, version=2 2023-08-09T13:33:21.071 INFO:teuthology.orchestra.run.smithi203.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2023-08-09T13:33:21.072 INFO:teuthology.orchestra.run.smithi203.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2023-08-09T13:33:21.591 INFO:teuthology.orchestra.run.smithi203.stdout:Discarding blocks...Done. 2023-08-09T13:33:21.597 INFO:tasks.ceph:mount /dev/vg_nvme/lv_3 on ubuntu@smithi203.front.sepia.ceph.com -o noatime 2023-08-09T13:33:21.598 DEBUG:teuthology.orchestra.run.smithi203:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_3 /var/lib/ceph/osd/ceph-6 2023-08-09T13:33:21.786 DEBUG:teuthology.orchestra.run.smithi203:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-6 2023-08-09T13:33:21.892 DEBUG:teuthology.orchestra.run.smithi203:> sudo mkdir -p /var/lib/ceph/osd/ceph-7 2023-08-09T13:33:21.982 INFO:tasks.ceph:roles_to_devs: {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'} 2023-08-09T13:33:21.983 INFO:tasks.ceph:role: osd.7 2023-08-09T13:33:21.983 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_4 on ubuntu@smithi203.front.sepia.ceph.com 2023-08-09T13:33:21.983 DEBUG:teuthology.orchestra.run.smithi203:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_4 2023-08-09T13:33:22.081 INFO:teuthology.orchestra.run.smithi203.stdout:meta-data=/dev/vg_nvme/lv_4 isize=2048 agcount=4, agsize=5859072 blks 2023-08-09T13:33:22.081 INFO:teuthology.orchestra.run.smithi203.stdout: = sectsz=512 attr=2, projid32bit=1 2023-08-09T13:33:22.082 INFO:teuthology.orchestra.run.smithi203.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2023-08-09T13:33:22.082 INFO:teuthology.orchestra.run.smithi203.stdout: = reflink=1 bigtime=1 inobtcount=1 nrext64=0 2023-08-09T13:33:22.082 INFO:teuthology.orchestra.run.smithi203.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2023-08-09T13:33:22.082 INFO:teuthology.orchestra.run.smithi203.stdout: = sunit=0 swidth=0 blks 2023-08-09T13:33:22.082 INFO:teuthology.orchestra.run.smithi203.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2023-08-09T13:33:22.082 INFO:teuthology.orchestra.run.smithi203.stdout:log =internal log bsize=4096 blocks=16384, version=2 2023-08-09T13:33:22.082 INFO:teuthology.orchestra.run.smithi203.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2023-08-09T13:33:22.083 INFO:teuthology.orchestra.run.smithi203.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2023-08-09T13:33:22.543 INFO:teuthology.orchestra.run.smithi203.stdout:Discarding blocks...Done. 2023-08-09T13:33:22.549 INFO:tasks.ceph:mount /dev/vg_nvme/lv_4 on ubuntu@smithi203.front.sepia.ceph.com -o noatime 2023-08-09T13:33:22.550 DEBUG:teuthology.orchestra.run.smithi203:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_4 /var/lib/ceph/osd/ceph-7 2023-08-09T13:33:22.703 DEBUG:teuthology.orchestra.run.smithi203:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-7 2023-08-09T13:33:22.797 DEBUG:teuthology.orchestra.run.smithi203:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 4 --monmap /home/ubuntu/cephtest/ceph.monmap 2023-08-09T13:33:22.942 INFO:teuthology.orchestra.run.smithi203.stderr:2023-08-09T13:33:22.940+0000 7f2081e1f500 -1 auth: error reading file: /var/lib/ceph/osd/ceph-4/keyring: can't open /var/lib/ceph/osd/ceph-4/keyring: (2) No such file or directory 2023-08-09T13:33:22.946 INFO:teuthology.orchestra.run.smithi203.stderr:2023-08-09T13:33:22.944+0000 7f2081e1f500 -1 created new key in keyring /var/lib/ceph/osd/ceph-4/keyring 2023-08-09T13:33:22.947 INFO:teuthology.orchestra.run.smithi203.stderr:2023-08-09T13:33:22.944+0000 7f2081e1f500 -1 bluestore(/var/lib/ceph/osd/ceph-4/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-4/block: (2) No such file or directory 2023-08-09T13:33:22.947 INFO:teuthology.orchestra.run.smithi203.stderr:2023-08-09T13:33:22.944+0000 7f2081e1f500 -1 bluestore(/var/lib/ceph/osd/ceph-4/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-4/block: (2) No such file or directory 2023-08-09T13:33:22.947 INFO:teuthology.orchestra.run.smithi203.stderr:2023-08-09T13:33:22.944+0000 7f2081e1f500 -1 bluestore(/var/lib/ceph/osd/ceph-4/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-4/block: (2) No such file or directory 2023-08-09T13:33:22.950 INFO:teuthology.orchestra.run.smithi203.stderr:2023-08-09T13:33:22.948+0000 7f2081e1f500 -1 bluestore(/var/lib/ceph/osd/ceph-4) _read_fsid unparsable uuid 2023-08-09T13:33:25.878 DEBUG:teuthology.orchestra.run.smithi203:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-4 2023-08-09T13:33:25.981 DEBUG:teuthology.orchestra.run.smithi203:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 5 --monmap /home/ubuntu/cephtest/ceph.monmap 2023-08-09T13:33:26.123 INFO:teuthology.orchestra.run.smithi203.stderr:2023-08-09T13:33:26.121+0000 7fa4dc39f500 -1 auth: error reading file: /var/lib/ceph/osd/ceph-5/keyring: can't open /var/lib/ceph/osd/ceph-5/keyring: (2) No such file or directory 2023-08-09T13:33:26.124 INFO:teuthology.orchestra.run.smithi203.stderr:2023-08-09T13:33:26.122+0000 7fa4dc39f500 -1 created new key in keyring /var/lib/ceph/osd/ceph-5/keyring 2023-08-09T13:33:26.124 INFO:teuthology.orchestra.run.smithi203.stderr:2023-08-09T13:33:26.122+0000 7fa4dc39f500 -1 bluestore(/var/lib/ceph/osd/ceph-5/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-5/block: (2) No such file or directory 2023-08-09T13:33:26.125 INFO:teuthology.orchestra.run.smithi203.stderr:2023-08-09T13:33:26.122+0000 7fa4dc39f500 -1 bluestore(/var/lib/ceph/osd/ceph-5/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-5/block: (2) No such file or directory 2023-08-09T13:33:26.125 INFO:teuthology.orchestra.run.smithi203.stderr:2023-08-09T13:33:26.122+0000 7fa4dc39f500 -1 bluestore(/var/lib/ceph/osd/ceph-5/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-5/block: (2) No such file or directory 2023-08-09T13:33:26.126 INFO:teuthology.orchestra.run.smithi203.stderr:2023-08-09T13:33:26.124+0000 7fa4dc39f500 -1 bluestore(/var/lib/ceph/osd/ceph-5) _read_fsid unparsable uuid 2023-08-09T13:33:27.674 DEBUG:teuthology.orchestra.run.smithi043:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-08-09T13:33:27.684 DEBUG:teuthology.orchestra.run.smithi203:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-08-09T13:33:29.075 DEBUG:teuthology.orchestra.run.smithi203:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-5 2023-08-09T13:33:29.177 DEBUG:teuthology.orchestra.run.smithi203:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 6 --monmap /home/ubuntu/cephtest/ceph.monmap 2023-08-09T13:33:29.319 INFO:teuthology.orchestra.run.smithi203.stderr:2023-08-09T13:33:29.317+0000 7f4daca0c500 -1 auth: error reading file: /var/lib/ceph/osd/ceph-6/keyring: can't open /var/lib/ceph/osd/ceph-6/keyring: (2) No such file or directory 2023-08-09T13:33:29.320 INFO:teuthology.orchestra.run.smithi203.stderr:2023-08-09T13:33:29.318+0000 7f4daca0c500 -1 created new key in keyring /var/lib/ceph/osd/ceph-6/keyring 2023-08-09T13:33:29.320 INFO:teuthology.orchestra.run.smithi203.stderr:2023-08-09T13:33:29.318+0000 7f4daca0c500 -1 bluestore(/var/lib/ceph/osd/ceph-6/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-6/block: (2) No such file or directory 2023-08-09T13:33:29.320 INFO:teuthology.orchestra.run.smithi203.stderr:2023-08-09T13:33:29.318+0000 7f4daca0c500 -1 bluestore(/var/lib/ceph/osd/ceph-6/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-6/block: (2) No such file or directory 2023-08-09T13:33:29.321 INFO:teuthology.orchestra.run.smithi203.stderr:2023-08-09T13:33:29.318+0000 7f4daca0c500 -1 bluestore(/var/lib/ceph/osd/ceph-6/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-6/block: (2) No such file or directory 2023-08-09T13:33:29.322 INFO:teuthology.orchestra.run.smithi203.stderr:2023-08-09T13:33:29.320+0000 7f4daca0c500 -1 bluestore(/var/lib/ceph/osd/ceph-6) _read_fsid unparsable uuid 2023-08-09T13:33:32.505 DEBUG:teuthology.orchestra.run.smithi203:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-6 2023-08-09T13:33:32.607 DEBUG:teuthology.orchestra.run.smithi203:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 7 --monmap /home/ubuntu/cephtest/ceph.monmap 2023-08-09T13:33:32.750 INFO:teuthology.orchestra.run.smithi203.stderr:2023-08-09T13:33:32.748+0000 7f19729be500 -1 auth: error reading file: /var/lib/ceph/osd/ceph-7/keyring: can't open /var/lib/ceph/osd/ceph-7/keyring: (2) No such file or directory 2023-08-09T13:33:32.751 INFO:teuthology.orchestra.run.smithi203.stderr:2023-08-09T13:33:32.749+0000 7f19729be500 -1 created new key in keyring /var/lib/ceph/osd/ceph-7/keyring 2023-08-09T13:33:32.751 INFO:teuthology.orchestra.run.smithi203.stderr:2023-08-09T13:33:32.749+0000 7f19729be500 -1 bluestore(/var/lib/ceph/osd/ceph-7/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-7/block: (2) No such file or directory 2023-08-09T13:33:32.752 INFO:teuthology.orchestra.run.smithi203.stderr:2023-08-09T13:33:32.749+0000 7f19729be500 -1 bluestore(/var/lib/ceph/osd/ceph-7/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-7/block: (2) No such file or directory 2023-08-09T13:33:32.752 INFO:teuthology.orchestra.run.smithi203.stderr:2023-08-09T13:33:32.749+0000 7f19729be500 -1 bluestore(/var/lib/ceph/osd/ceph-7/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-7/block: (2) No such file or directory 2023-08-09T13:33:32.753 INFO:teuthology.orchestra.run.smithi203.stderr:2023-08-09T13:33:32.751+0000 7f19729be500 -1 bluestore(/var/lib/ceph/osd/ceph-7) _read_fsid unparsable uuid 2023-08-09T13:33:35.686 DEBUG:teuthology.orchestra.run.smithi203:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-7 2023-08-09T13:33:35.784 INFO:tasks.ceph:Reading keys from all nodes... 2023-08-09T13:33:35.784 DEBUG:teuthology.orchestra.run.smithi043:> set -ex 2023-08-09T13:33:35.785 DEBUG:teuthology.orchestra.run.smithi043:> sudo dd if=/var/lib/ceph/mgr/ceph-y/keyring of=/dev/stdout 2023-08-09T13:33:35.896 DEBUG:teuthology.orchestra.run.smithi043:> set -ex 2023-08-09T13:33:35.897 DEBUG:teuthology.orchestra.run.smithi043:> sudo dd if=/var/lib/ceph/mds/ceph-a/keyring of=/dev/stdout 2023-08-09T13:33:36.002 DEBUG:teuthology.orchestra.run.smithi043:> set -ex 2023-08-09T13:33:36.002 DEBUG:teuthology.orchestra.run.smithi043:> sudo dd if=/var/lib/ceph/mds/ceph-c/keyring of=/dev/stdout 2023-08-09T13:33:36.104 DEBUG:teuthology.orchestra.run.smithi043:> set -ex 2023-08-09T13:33:36.105 DEBUG:teuthology.orchestra.run.smithi043:> sudo dd if=/var/lib/ceph/osd/ceph-0/keyring of=/dev/stdout 2023-08-09T13:33:36.212 DEBUG:teuthology.orchestra.run.smithi043:> set -ex 2023-08-09T13:33:36.212 DEBUG:teuthology.orchestra.run.smithi043:> sudo dd if=/var/lib/ceph/osd/ceph-1/keyring of=/dev/stdout 2023-08-09T13:33:36.317 DEBUG:teuthology.orchestra.run.smithi043:> set -ex 2023-08-09T13:33:36.317 DEBUG:teuthology.orchestra.run.smithi043:> sudo dd if=/var/lib/ceph/osd/ceph-2/keyring of=/dev/stdout 2023-08-09T13:33:36.425 DEBUG:teuthology.orchestra.run.smithi043:> set -ex 2023-08-09T13:33:36.425 DEBUG:teuthology.orchestra.run.smithi043:> sudo dd if=/var/lib/ceph/osd/ceph-3/keyring of=/dev/stdout 2023-08-09T13:33:36.531 DEBUG:teuthology.orchestra.run.smithi203:> set -ex 2023-08-09T13:33:36.532 DEBUG:teuthology.orchestra.run.smithi203:> sudo dd if=/var/lib/ceph/mgr/ceph-x/keyring of=/dev/stdout 2023-08-09T13:33:36.638 DEBUG:teuthology.orchestra.run.smithi203:> set -ex 2023-08-09T13:33:36.639 DEBUG:teuthology.orchestra.run.smithi203:> sudo dd if=/var/lib/ceph/mds/ceph-b/keyring of=/dev/stdout 2023-08-09T13:33:36.733 DEBUG:teuthology.orchestra.run.smithi203:> set -ex 2023-08-09T13:33:36.734 DEBUG:teuthology.orchestra.run.smithi203:> sudo dd if=/var/lib/ceph/mds/ceph-d/keyring of=/dev/stdout 2023-08-09T13:33:36.829 DEBUG:teuthology.orchestra.run.smithi203:> set -ex 2023-08-09T13:33:36.830 DEBUG:teuthology.orchestra.run.smithi203:> sudo dd if=/var/lib/ceph/osd/ceph-4/keyring of=/dev/stdout 2023-08-09T13:33:36.931 DEBUG:teuthology.orchestra.run.smithi203:> set -ex 2023-08-09T13:33:36.932 DEBUG:teuthology.orchestra.run.smithi203:> sudo dd if=/var/lib/ceph/osd/ceph-5/keyring of=/dev/stdout 2023-08-09T13:33:37.026 DEBUG:teuthology.orchestra.run.smithi203:> set -ex 2023-08-09T13:33:37.027 DEBUG:teuthology.orchestra.run.smithi203:> sudo dd if=/var/lib/ceph/osd/ceph-6/keyring of=/dev/stdout 2023-08-09T13:33:37.122 DEBUG:teuthology.orchestra.run.smithi203:> set -ex 2023-08-09T13:33:37.122 DEBUG:teuthology.orchestra.run.smithi203:> sudo dd if=/var/lib/ceph/osd/ceph-7/keyring of=/dev/stdout 2023-08-09T13:33:37.222 DEBUG:teuthology.orchestra.run.smithi043:> set -ex 2023-08-09T13:33:37.223 DEBUG:teuthology.orchestra.run.smithi043:> dd if=/etc/ceph/ceph.client.0.keyring of=/dev/stdout 2023-08-09T13:33:37.286 DEBUG:teuthology.orchestra.run.smithi043:> set -ex 2023-08-09T13:33:37.287 DEBUG:teuthology.orchestra.run.smithi043:> dd if=/etc/ceph/ceph.client.2.keyring of=/dev/stdout 2023-08-09T13:33:37.344 DEBUG:teuthology.orchestra.run.smithi203:> set -ex 2023-08-09T13:33:37.344 DEBUG:teuthology.orchestra.run.smithi203:> dd if=/etc/ceph/ceph.client.1.keyring of=/dev/stdout 2023-08-09T13:33:37.404 DEBUG:teuthology.orchestra.run.smithi203:> set -ex 2023-08-09T13:33:37.405 DEBUG:teuthology.orchestra.run.smithi203:> dd if=/etc/ceph/ceph.client.3.keyring of=/dev/stdout 2023-08-09T13:33:37.459 INFO:tasks.ceph:Adding keys to all mons... 2023-08-09T13:33:37.459 DEBUG:teuthology.orchestra.run.smithi043:> sudo tee -a /etc/ceph/ceph.keyring 2023-08-09T13:33:37.468 DEBUG:teuthology.orchestra.run.smithi203:> sudo tee -a /etc/ceph/ceph.keyring 2023-08-09T13:33:37.558 INFO:teuthology.orchestra.run.smithi203.stdout:[mgr.y] 2023-08-09T13:33:37.558 INFO:teuthology.orchestra.run.smithi203.stdout: key = AQCNldNk2YgzLBAAA11Q8uAMEgE/bUni+tXIKA== 2023-08-09T13:33:37.558 INFO:teuthology.orchestra.run.smithi203.stdout:[mds.a] 2023-08-09T13:33:37.559 INFO:teuthology.orchestra.run.smithi203.stdout: key = AQCOldNkPu6GCRAAZecwctvMji0Zubb60KMlyQ== 2023-08-09T13:33:37.559 INFO:teuthology.orchestra.run.smithi203.stdout:[mds.c] 2023-08-09T13:33:37.559 INFO:teuthology.orchestra.run.smithi203.stdout: key = AQCOldNk5V46GhAAQBhkvfjVSJCSHLS/YkhgCA== 2023-08-09T13:33:37.559 INFO:teuthology.orchestra.run.smithi203.stdout:[osd.0] 2023-08-09T13:33:37.559 INFO:teuthology.orchestra.run.smithi203.stdout: key = AQCRldNkltPlMhAAAt3rFvraCtI0aSqGvt6UxA== 2023-08-09T13:33:37.559 INFO:teuthology.orchestra.run.smithi203.stdout:[osd.1] 2023-08-09T13:33:37.559 INFO:teuthology.orchestra.run.smithi203.stdout: key = AQCVldNkzpKYEhAA+wen0kV/vUDLQztq3vhfOg== 2023-08-09T13:33:37.560 INFO:teuthology.orchestra.run.smithi203.stdout:[osd.2] 2023-08-09T13:33:37.560 INFO:teuthology.orchestra.run.smithi203.stdout: key = AQCYldNkYbwNHhAAQY8g9dldb1V9M2MM2SHKDg== 2023-08-09T13:33:37.560 INFO:teuthology.orchestra.run.smithi203.stdout:[osd.3] 2023-08-09T13:33:37.560 INFO:teuthology.orchestra.run.smithi203.stdout: key = AQCbldNkeyW6KRAAsd+kgqik2aD4U7PXwccWtA== 2023-08-09T13:33:37.560 INFO:teuthology.orchestra.run.smithi203.stdout:[mgr.x] 2023-08-09T13:33:37.560 INFO:teuthology.orchestra.run.smithi203.stdout: key = AQCNldNk2ufSNxAAU7msRkwLaoLZHTIsk0ck/g== 2023-08-09T13:33:37.560 INFO:teuthology.orchestra.run.smithi203.stdout:[mds.b] 2023-08-09T13:33:37.561 INFO:teuthology.orchestra.run.smithi203.stdout: key = AQCOldNkt2+5KxAAHcckB+S5NZHsF5z22StXmw== 2023-08-09T13:33:37.561 INFO:teuthology.orchestra.run.smithi203.stdout:[mds.d] 2023-08-09T13:33:37.561 INFO:teuthology.orchestra.run.smithi203.stdout: key = AQCPldNkNBoTARAAMVW+rSyXQKaau6I90fUX9Q== 2023-08-09T13:33:37.561 INFO:teuthology.orchestra.run.smithi203.stdout:[osd.4] 2023-08-09T13:33:37.561 INFO:teuthology.orchestra.run.smithi203.stdout: key = AQCildNk80UkOBAAtX+4JGpOVcfZEHetJoeZKw== 2023-08-09T13:33:37.561 INFO:teuthology.orchestra.run.smithi203.stdout:[osd.5] 2023-08-09T13:33:37.561 INFO:teuthology.orchestra.run.smithi203.stdout: key = AQCmldNkCA9SBxAAZj4So6MRj2Eq1I9H9JzLDA== 2023-08-09T13:33:37.561 INFO:teuthology.orchestra.run.smithi203.stdout:[osd.6] 2023-08-09T13:33:37.562 INFO:teuthology.orchestra.run.smithi203.stdout: key = AQCpldNkaOwAExAAraV1tUwr9uU6GheZ+hvdqQ== 2023-08-09T13:33:37.562 INFO:teuthology.orchestra.run.smithi203.stdout:[osd.7] 2023-08-09T13:33:37.562 INFO:teuthology.orchestra.run.smithi203.stdout: key = AQCsldNk2/GwLBAAP42EXPivzLk44X8aMEVVCg== 2023-08-09T13:33:37.562 INFO:teuthology.orchestra.run.smithi203.stdout:[client.0] 2023-08-09T13:33:37.562 INFO:teuthology.orchestra.run.smithi203.stdout: key = AQCPldNkg8zrDxAAM/hnJaE6S9HU/fRwQeBV5w== 2023-08-09T13:33:37.562 INFO:teuthology.orchestra.run.smithi203.stdout:[client.2] 2023-08-09T13:33:37.562 INFO:teuthology.orchestra.run.smithi203.stdout: key = AQCPldNkC2lgGxAAZ8qZboOCUaQlcpkPL1ozHA== 2023-08-09T13:33:37.562 INFO:teuthology.orchestra.run.smithi203.stdout:[client.1] 2023-08-09T13:33:37.563 INFO:teuthology.orchestra.run.smithi203.stdout: key = AQCPldNklfFxJxAA19+L78rW6jU1uOyjey6WNA== 2023-08-09T13:33:37.563 INFO:teuthology.orchestra.run.smithi203.stdout:[client.3] 2023-08-09T13:33:37.563 INFO:teuthology.orchestra.run.smithi203.stdout: key = AQCPldNkl+rXMhAA9rf3KdnSYbvngqmLYnIGGA== 2023-08-09T13:33:37.565 INFO:teuthology.orchestra.run.smithi043.stdout:[mgr.y] 2023-08-09T13:33:37.565 INFO:teuthology.orchestra.run.smithi043.stdout: key = AQCNldNk2YgzLBAAA11Q8uAMEgE/bUni+tXIKA== 2023-08-09T13:33:37.565 INFO:teuthology.orchestra.run.smithi043.stdout:[mds.a] 2023-08-09T13:33:37.565 INFO:teuthology.orchestra.run.smithi043.stdout: key = AQCOldNkPu6GCRAAZecwctvMji0Zubb60KMlyQ== 2023-08-09T13:33:37.566 INFO:teuthology.orchestra.run.smithi043.stdout:[mds.c] 2023-08-09T13:33:37.566 INFO:teuthology.orchestra.run.smithi043.stdout: key = AQCOldNk5V46GhAAQBhkvfjVSJCSHLS/YkhgCA== 2023-08-09T13:33:37.566 INFO:teuthology.orchestra.run.smithi043.stdout:[osd.0] 2023-08-09T13:33:37.566 INFO:teuthology.orchestra.run.smithi043.stdout: key = AQCRldNkltPlMhAAAt3rFvraCtI0aSqGvt6UxA== 2023-08-09T13:33:37.566 INFO:teuthology.orchestra.run.smithi043.stdout:[osd.1] 2023-08-09T13:33:37.566 INFO:teuthology.orchestra.run.smithi043.stdout: key = AQCVldNkzpKYEhAA+wen0kV/vUDLQztq3vhfOg== 2023-08-09T13:33:37.566 INFO:teuthology.orchestra.run.smithi043.stdout:[osd.2] 2023-08-09T13:33:37.567 INFO:teuthology.orchestra.run.smithi043.stdout: key = AQCYldNkYbwNHhAAQY8g9dldb1V9M2MM2SHKDg== 2023-08-09T13:33:37.567 INFO:teuthology.orchestra.run.smithi043.stdout:[osd.3] 2023-08-09T13:33:37.567 INFO:teuthology.orchestra.run.smithi043.stdout: key = AQCbldNkeyW6KRAAsd+kgqik2aD4U7PXwccWtA== 2023-08-09T13:33:37.567 INFO:teuthology.orchestra.run.smithi043.stdout:[mgr.x] 2023-08-09T13:33:37.567 INFO:teuthology.orchestra.run.smithi043.stdout: key = AQCNldNk2ufSNxAAU7msRkwLaoLZHTIsk0ck/g== 2023-08-09T13:33:37.567 INFO:teuthology.orchestra.run.smithi043.stdout:[mds.b] 2023-08-09T13:33:37.567 INFO:teuthology.orchestra.run.smithi043.stdout: key = AQCOldNkt2+5KxAAHcckB+S5NZHsF5z22StXmw== 2023-08-09T13:33:37.567 INFO:teuthology.orchestra.run.smithi043.stdout:[mds.d] 2023-08-09T13:33:37.568 INFO:teuthology.orchestra.run.smithi043.stdout: key = AQCPldNkNBoTARAAMVW+rSyXQKaau6I90fUX9Q== 2023-08-09T13:33:37.568 INFO:teuthology.orchestra.run.smithi043.stdout:[osd.4] 2023-08-09T13:33:37.568 INFO:teuthology.orchestra.run.smithi043.stdout: key = AQCildNk80UkOBAAtX+4JGpOVcfZEHetJoeZKw== 2023-08-09T13:33:37.568 INFO:teuthology.orchestra.run.smithi043.stdout:[osd.5] 2023-08-09T13:33:37.568 INFO:teuthology.orchestra.run.smithi043.stdout: key = AQCmldNkCA9SBxAAZj4So6MRj2Eq1I9H9JzLDA== 2023-08-09T13:33:37.568 INFO:teuthology.orchestra.run.smithi043.stdout:[osd.6] 2023-08-09T13:33:37.568 INFO:teuthology.orchestra.run.smithi043.stdout: key = AQCpldNkaOwAExAAraV1tUwr9uU6GheZ+hvdqQ== 2023-08-09T13:33:37.569 INFO:teuthology.orchestra.run.smithi043.stdout:[osd.7] 2023-08-09T13:33:37.569 INFO:teuthology.orchestra.run.smithi043.stdout: key = AQCsldNk2/GwLBAAP42EXPivzLk44X8aMEVVCg== 2023-08-09T13:33:37.569 INFO:teuthology.orchestra.run.smithi043.stdout:[client.0] 2023-08-09T13:33:37.569 INFO:teuthology.orchestra.run.smithi043.stdout: key = AQCPldNkg8zrDxAAM/hnJaE6S9HU/fRwQeBV5w== 2023-08-09T13:33:37.569 INFO:teuthology.orchestra.run.smithi043.stdout:[client.2] 2023-08-09T13:33:37.569 INFO:teuthology.orchestra.run.smithi043.stdout: key = AQCPldNkC2lgGxAAZ8qZboOCUaQlcpkPL1ozHA== 2023-08-09T13:33:37.569 INFO:teuthology.orchestra.run.smithi043.stdout:[client.1] 2023-08-09T13:33:37.569 INFO:teuthology.orchestra.run.smithi043.stdout: key = AQCPldNklfFxJxAA19+L78rW6jU1uOyjey6WNA== 2023-08-09T13:33:37.570 INFO:teuthology.orchestra.run.smithi043.stdout:[client.3] 2023-08-09T13:33:37.570 INFO:teuthology.orchestra.run.smithi043.stdout: key = AQCPldNkl+rXMhAA9rf3KdnSYbvngqmLYnIGGA== 2023-08-09T13:33:37.570 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mgr.y --cap mon 'allow profile mgr' --cap osd 'allow *' --cap mds 'allow *' 2023-08-09T13:33:37.579 DEBUG:teuthology.orchestra.run.smithi203:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mgr.y --cap mon 'allow profile mgr' --cap osd 'allow *' --cap mds 'allow *' 2023-08-09T13:33:37.726 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.a --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2023-08-09T13:33:37.733 DEBUG:teuthology.orchestra.run.smithi203:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.a --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2023-08-09T13:33:37.884 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.c --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2023-08-09T13:33:37.893 DEBUG:teuthology.orchestra.run.smithi203:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.c --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2023-08-09T13:33:38.042 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.0 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2023-08-09T13:33:38.047 DEBUG:teuthology.orchestra.run.smithi203:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.0 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2023-08-09T13:33:38.198 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.1 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2023-08-09T13:33:38.203 DEBUG:teuthology.orchestra.run.smithi203:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.1 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2023-08-09T13:33:38.351 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.2 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2023-08-09T13:33:38.359 DEBUG:teuthology.orchestra.run.smithi203:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.2 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2023-08-09T13:33:38.507 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.3 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2023-08-09T13:33:38.513 DEBUG:teuthology.orchestra.run.smithi203:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.3 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2023-08-09T13:33:38.659 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mgr.x --cap mon 'allow profile mgr' --cap osd 'allow *' --cap mds 'allow *' 2023-08-09T13:33:38.664 DEBUG:teuthology.orchestra.run.smithi203:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mgr.x --cap mon 'allow profile mgr' --cap osd 'allow *' --cap mds 'allow *' 2023-08-09T13:33:38.810 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.b --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2023-08-09T13:33:38.815 DEBUG:teuthology.orchestra.run.smithi203:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.b --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2023-08-09T13:33:38.968 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.d --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2023-08-09T13:33:38.976 DEBUG:teuthology.orchestra.run.smithi203:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.d --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2023-08-09T13:33:39.125 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.4 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2023-08-09T13:33:39.132 DEBUG:teuthology.orchestra.run.smithi203:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.4 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2023-08-09T13:33:39.274 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.5 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2023-08-09T13:33:39.282 DEBUG:teuthology.orchestra.run.smithi203:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.5 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2023-08-09T13:33:39.426 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.6 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2023-08-09T13:33:39.435 DEBUG:teuthology.orchestra.run.smithi203:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.6 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2023-08-09T13:33:39.584 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.7 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2023-08-09T13:33:39.589 DEBUG:teuthology.orchestra.run.smithi203:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.7 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2023-08-09T13:33:39.736 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.0 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2023-08-09T13:33:39.742 DEBUG:teuthology.orchestra.run.smithi203:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.0 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2023-08-09T13:33:39.892 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.2 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2023-08-09T13:33:39.897 DEBUG:teuthology.orchestra.run.smithi203:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.2 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2023-08-09T13:33:40.047 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.1 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2023-08-09T13:33:40.053 DEBUG:teuthology.orchestra.run.smithi203:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.1 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2023-08-09T13:33:40.203 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.3 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2023-08-09T13:33:40.209 DEBUG:teuthology.orchestra.run.smithi203:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.3 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2023-08-09T13:33:40.357 INFO:tasks.ceph:Running mkfs on mon nodes... 2023-08-09T13:33:40.357 DEBUG:teuthology.orchestra.run.smithi043:> sudo mkdir -p /var/lib/ceph/mon/ceph-a 2023-08-09T13:33:40.457 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-mon --cluster ceph --mkfs -i a --monmap /home/ubuntu/cephtest/ceph.monmap --keyring /etc/ceph/ceph.keyring 2023-08-09T13:33:40.629 DEBUG:teuthology.orchestra.run.smithi043:> sudo chown -R ceph:ceph /var/lib/ceph/mon/ceph-a 2023-08-09T13:33:40.714 DEBUG:teuthology.orchestra.run.smithi203:> sudo mkdir -p /var/lib/ceph/mon/ceph-b 2023-08-09T13:33:40.819 DEBUG:teuthology.orchestra.run.smithi203:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-mon --cluster ceph --mkfs -i b --monmap /home/ubuntu/cephtest/ceph.monmap --keyring /etc/ceph/ceph.keyring 2023-08-09T13:33:40.988 DEBUG:teuthology.orchestra.run.smithi203:> sudo chown -R ceph:ceph /var/lib/ceph/mon/ceph-b 2023-08-09T13:33:41.079 DEBUG:teuthology.orchestra.run.smithi203:> sudo mkdir -p /var/lib/ceph/mon/ceph-c 2023-08-09T13:33:41.179 DEBUG:teuthology.orchestra.run.smithi203:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-mon --cluster ceph --mkfs -i c --monmap /home/ubuntu/cephtest/ceph.monmap --keyring /etc/ceph/ceph.keyring 2023-08-09T13:33:41.338 DEBUG:teuthology.orchestra.run.smithi203:> sudo chown -R ceph:ceph /var/lib/ceph/mon/ceph-c 2023-08-09T13:33:41.421 DEBUG:teuthology.orchestra.run.smithi043:> rm -- /home/ubuntu/cephtest/ceph.monmap 2023-08-09T13:33:41.430 DEBUG:teuthology.orchestra.run.smithi203:> rm -- /home/ubuntu/cephtest/ceph.monmap 2023-08-09T13:33:41.485 INFO:tasks.ceph:Starting mon daemons in cluster ceph... 2023-08-09T13:33:41.485 INFO:tasks.ceph.mon.a:Restarting daemon 2023-08-09T13:33:41.486 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mon -f --cluster ceph -i a 2023-08-09T13:33:41.492 INFO:tasks.ceph.mon.a:Started 2023-08-09T13:33:41.492 INFO:tasks.ceph.mon.b:Restarting daemon 2023-08-09T13:33:41.492 DEBUG:teuthology.orchestra.run.smithi203:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mon -f --cluster ceph -i b 2023-08-09T13:33:41.497 INFO:tasks.ceph.mon.b:Started 2023-08-09T13:33:41.498 INFO:tasks.ceph.mon.c:Restarting daemon 2023-08-09T13:33:41.498 DEBUG:teuthology.orchestra.run.smithi203:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mon -f --cluster ceph -i c 2023-08-09T13:33:41.503 INFO:tasks.ceph.mon.c:Started 2023-08-09T13:33:41.504 INFO:tasks.ceph:Starting mgr daemons in cluster ceph... 2023-08-09T13:33:41.504 INFO:tasks.ceph.mgr.y:Restarting daemon 2023-08-09T13:33:41.504 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mgr -f --cluster ceph -i y 2023-08-09T13:33:41.510 INFO:tasks.ceph.mgr.y:Started 2023-08-09T13:33:41.510 INFO:tasks.ceph.mgr.x:Restarting daemon 2023-08-09T13:33:41.510 DEBUG:teuthology.orchestra.run.smithi203:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mgr -f --cluster ceph -i x 2023-08-09T13:33:41.516 INFO:tasks.ceph.mgr.x:Started 2023-08-09T13:33:41.516 INFO:tasks.ceph:Setting crush tunables to default 2023-08-09T13:33:41.516 DEBUG:teuthology.orchestra.run.smithi043:> sudo ceph --cluster ceph osd crush tunables default 2023-08-09T13:33:41.707 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:33:41.706+0000 7f7145d57640 1 Processor -- start 2023-08-09T13:33:41.707 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:33:41.706+0000 7f7145d57640 1 -- start start 2023-08-09T13:33:41.709 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:33:41.708+0000 7f7145d57640 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7f71400d0a70 0x7f71400cd7f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:33:41.709 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:33:41.708+0000 7f7145d57640 1 -- --> v1:172.21.15.43:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f71400d0f70 con 0x7f714015c680 2023-08-09T13:33:41.710 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:33:41.708+0000 7f7145d57640 1 -- --> v1:172.21.15.203:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f71400ccd10 con 0x7f71400cdd30 2023-08-09T13:33:41.710 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:33:41.708+0000 7f7145d57640 1 -- --> v2:172.21.15.203:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f71400ccff0 con 0x7f71400d0a70 2023-08-09T13:33:41.710 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:33:41.708+0000 7f713effd640 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7f71400d0a70 0x7f71400cd7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:33:41.711 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:33:41.708+0000 7f713effd640 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7f71400d0a70 0x7f71400cd7f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.203:3300/0 says I am v2:172.21.15.43:39496/0 (socket says 172.21.15.43:39496) 2023-08-09T13:33:41.711 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:33:41.708+0000 7f713effd640 1 -- 172.21.15.43:0/61188562 learned_addr learned my addr 172.21.15.43:0/61188562 (peer_addr_for_me v2:172.21.15.43:0/0) 2023-08-09T13:33:41.711 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:33:41.709+0000 7f713effd640 1 -- 172.21.15.43:0/61188562 >> v2:172.21.15.203:3300/0 conn(0x7f71400d0a70 msgr2=0x7f71400cd7f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 14 2023-08-09T13:33:41.711 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:33:41.709+0000 7f713effd640 1 -- 172.21.15.43:0/61188562 >> v2:172.21.15.203:3300/0 conn(0x7f71400d0a70 msgr2=0x7f71400cd7f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2023-08-09T13:33:41.711 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:33:41.709+0000 7f713effd640 1 --2- 172.21.15.43:0/61188562 >> v2:172.21.15.203:3300/0 conn(0x7f71400d0a70 0x7f71400cd7f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2023-08-09T13:33:41.711 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:33:41.709+0000 7f713effd640 1 --2- 172.21.15.43:0/61188562 >> v2:172.21.15.203:3300/0 conn(0x7f71400d0a70 0x7f71400cd7f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-08-09T13:33:41.722 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:33:41.720+0000 7f713e7fc640 1 -- 172.21.15.43:0/61188562 <== mon.0 v1:172.21.15.43:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 287306045 0 0) 0x7f71400d0f70 con 0x7f714015c680 2023-08-09T13:33:41.722 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:33:41.720+0000 7f713e7fc640 1 -- 172.21.15.43:0/61188562 --> v1:172.21.15.43:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f7128003520 con 0x7f714015c680 2023-08-09T13:33:41.724 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:33:41.721+0000 7f713e7fc640 1 -- 172.21.15.43:0/61188562 <== mon.0 v1:172.21.15.43:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 4059189272 0 0) 0x7f7128003520 con 0x7f714015c680 2023-08-09T13:33:41.724 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:33:41.721+0000 7f713e7fc640 1 -- 172.21.15.43:0/61188562 >> v2:172.21.15.203:3300/0 conn(0x7f71400d0a70 msgr2=0x7f71400cd7f0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-08-09T13:33:41.724 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:33:41.721+0000 7f713e7fc640 1 --2- 172.21.15.43:0/61188562 >> v2:172.21.15.203:3300/0 conn(0x7f71400d0a70 0x7f71400cd7f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:33:41.724 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:33:41.721+0000 7f713e7fc640 1 -- 172.21.15.43:0/61188562 >> v1:172.21.15.203:6789/0 conn(0x7f71400cdd30 legacy=0x7f71400b3fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:33:41.724 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:33:41.721+0000 7f713e7fc640 1 -- 172.21.15.43:0/61188562 --> v1:172.21.15.43:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f71400cd2d0 con 0x7f714015c680 2023-08-09T13:33:41.724 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:33:41.721+0000 7f713e7fc640 1 -- 172.21.15.43:0/61188562 <== mon.0 v1:172.21.15.43:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1285150740 0 0) 0x7f7130001670 con 0x7f714015c680 2023-08-09T13:33:41.725 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:33:41.721+0000 7f713e7fc640 1 -- 172.21.15.43:0/61188562 >> v1:172.21.15.43:6789/0 conn(0x7f714015c680 legacy=0x7f71400ce760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:33:41.725 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:33:41.721+0000 7f713e7fc640 1 --2- 172.21.15.43:0/61188562 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f71280039e0 0x7f7128003e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:33:41.725 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:33:41.722+0000 7f713e7fc640 1 --2- 172.21.15.43:0/61188562 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f71280047c0 0x7f7128004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:33:41.725 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:33:41.722+0000 7f713e7fc640 1 --2- 172.21.15.43:0/61188562 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f71280051f0 0x7f712800b980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:33:41.725 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:33:41.722+0000 7f713e7fc640 1 -- 172.21.15.43:0/61188562 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f71400ccff0 con 0x7f71280051f0 2023-08-09T13:33:41.725 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:33:41.722+0000 7f713e7fc640 1 -- 172.21.15.43:0/61188562 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f712800c150 con 0x7f71280047c0 2023-08-09T13:33:41.725 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:33:41.722+0000 7f713e7fc640 1 -- 172.21.15.43:0/61188562 --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f712800c310 con 0x7f71280039e0 2023-08-09T13:33:41.725 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:33:41.722+0000 7f713effd640 1 --2- 172.21.15.43:0/61188562 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f71280047c0 0x7f7128004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:33:41.726 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:33:41.722+0000 7f713ffff640 1 --2- 172.21.15.43:0/61188562 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f71280051f0 0x7f712800b980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:33:41.726 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:33:41.722+0000 7f713f7fe640 1 --2- 172.21.15.43:0/61188562 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f71280039e0 0x7f7128003e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:33:41.726 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:33:41.722+0000 7f713ffff640 1 -- 172.21.15.43:0/61188562 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f71280039e0 msgr2=0x7f7128003e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-09T13:33:41.726 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:33:41.722+0000 7f713ffff640 1 --2- 172.21.15.43:0/61188562 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f71280039e0 0x7f7128003e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:33:41.726 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:33:41.722+0000 7f713ffff640 1 -- 172.21.15.43:0/61188562 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f71280047c0 msgr2=0x7f7128004c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-09T13:33:41.726 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:33:41.722+0000 7f713ffff640 1 --2- 172.21.15.43:0/61188562 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f71280047c0 0x7f7128004c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:33:41.726 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:33:41.722+0000 7f713ffff640 1 -- 172.21.15.43:0/61188562 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7128003520 con 0x7f71280051f0 2023-08-09T13:33:41.727 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:33:41.723+0000 7f713ffff640 1 -- 172.21.15.43:0/61188562 reap_dead start 2023-08-09T13:33:41.727 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:33:41.724+0000 7f713ffff640 1 --2- 172.21.15.43:0/61188562 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f71280051f0 0x7f712800b980 secure :-1 s=READY pgs=4 cs=0 l=1 rev1=1 crypto rx=0x7f7138001e20 tx=0x7f71280057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f0c7c087d147086e server_cookie=0 in_seq=0 out_seq=0 2023-08-09T13:33:41.728 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:33:41.724+0000 7f713e7fc640 1 -- 172.21.15.43:0/61188562 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7138003b70 con 0x7f71280051f0 2023-08-09T13:33:41.728 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:33:41.724+0000 7f713e7fc640 1 -- 172.21.15.43:0/61188562 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7138003cd0 con 0x7f71280051f0 2023-08-09T13:33:41.728 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:33:41.724+0000 7f713e7fc640 1 -- 172.21.15.43:0/61188562 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7138005310 con 0x7f71280051f0 2023-08-09T13:33:41.728 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:33:41.724+0000 7f7145d57640 1 -- 172.21.15.43:0/61188562 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f71280051f0 msgr2=0x7f712800b980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:33:41.728 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:33:41.724+0000 7f7145d57640 1 --2- 172.21.15.43:0/61188562 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f71280051f0 0x7f712800b980 secure :-1 s=READY pgs=4 cs=0 l=1 rev1=1 crypto rx=0x7f7138001e20 tx=0x7f71280057f0 comp rx=0 tx=0).stop 2023-08-09T13:33:41.728 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:33:41.725+0000 7f7145d57640 1 -- 172.21.15.43:0/61188562 shutdown_connections 2023-08-09T13:33:41.728 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:33:41.725+0000 7f7145d57640 1 --2- 172.21.15.43:0/61188562 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f71280051f0 0x7f712800b980 unknown :-1 s=CLOSED pgs=4 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:33:41.729 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:33:41.725+0000 7f7145d57640 1 -- 172.21.15.43:0/61188562 >> 172.21.15.43:0/61188562 conn(0x7f7140071520 msgr2=0x7f71400d2520 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-09T13:33:41.729 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:33:41.725+0000 7f7145d57640 1 -- 172.21.15.43:0/61188562 shutdown_connections 2023-08-09T13:33:41.729 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:33:41.725+0000 7f7145d57640 1 -- 172.21.15.43:0/61188562 wait complete. 2023-08-09T13:33:41.731 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:33:41.727+0000 7f7145d57640 1 Processor -- start 2023-08-09T13:33:41.731 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:33:41.727+0000 7f7145d57640 1 -- start start 2023-08-09T13:33:41.731 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:33:41.728+0000 7f7145d57640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f71280051f0 0x7f7140154180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:33:41.731 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:33:41.728+0000 7f7145d57640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f71401588a0 0x7f71400b6490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:33:41.731 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:33:41.728+0000 7f7145d57640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f7140158d00 0x7f71400b69d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:33:41.731 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:33:41.728+0000 7f7145d57640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f71401f62f0 con 0x7f71401588a0 2023-08-09T13:33:41.732 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:33:41.728+0000 7f7145d57640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f71401f5770 con 0x7f71280051f0 2023-08-09T13:33:41.732 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:33:41.728+0000 7f7145d57640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f71401f5a50 con 0x7f7140158d00 2023-08-09T13:33:41.732 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:33:41.728+0000 7f713effd640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f71401588a0 0x7f71400b6490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:33:41.732 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:33:41.728+0000 7f713effd640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f71401588a0 0x7f71400b6490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:45214/0 (socket says 172.21.15.43:45214) 2023-08-09T13:33:41.732 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:33:41.728+0000 7f713effd640 1 -- 172.21.15.43:0/653292167 learned_addr learned my addr 172.21.15.43:0/653292167 (peer_addr_for_me v2:172.21.15.43:0/0) 2023-08-09T13:33:41.732 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:33:41.728+0000 7f713f7fe640 1 --2- 172.21.15.43:0/653292167 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f71280051f0 0x7f7140154180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:33:41.732 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:33:41.728+0000 7f713ffff640 1 --2- 172.21.15.43:0/653292167 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f7140158d00 0x7f71400b69d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:33:41.733 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:33:41.729+0000 7f713effd640 1 -- 172.21.15.43:0/653292167 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f7140158d00 msgr2=0x7f71400b69d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:33:41.733 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:33:41.729+0000 7f713effd640 1 --2- 172.21.15.43:0/653292167 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f7140158d00 0x7f71400b69d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:33:41.733 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:33:41.729+0000 7f713effd640 1 -- 172.21.15.43:0/653292167 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f71280051f0 msgr2=0x7f7140154180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:33:41.733 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:33:41.729+0000 7f713effd640 1 --2- 172.21.15.43:0/653292167 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f71280051f0 0x7f7140154180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:33:41.733 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:33:41.729+0000 7f713effd640 1 -- 172.21.15.43:0/653292167 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f71401f5d30 con 0x7f71401588a0 2023-08-09T13:33:41.733 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:33:41.729+0000 7f713f7fe640 1 --2- 172.21.15.43:0/653292167 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f71280051f0 0x7f7140154180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-08-09T13:33:41.733 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:33:41.729+0000 7f713effd640 1 --2- 172.21.15.43:0/653292167 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f71401588a0 0x7f71400b6490 secure :-1 s=READY pgs=5 cs=0 l=1 rev1=1 crypto rx=0x7f7134011600 tx=0x7f7134011a00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-09T13:33:41.733 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:33:41.729+0000 7f7127fff640 1 -- 172.21.15.43:0/653292167 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f713400c070 con 0x7f71401588a0 2023-08-09T13:33:41.734 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:33:41.729+0000 7f7127fff640 1 -- 172.21.15.43:0/653292167 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7134015930 con 0x7f71401588a0 2023-08-09T13:33:41.734 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:33:41.729+0000 7f7127fff640 1 -- 172.21.15.43:0/653292167 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7134015ce0 con 0x7f71401588a0 2023-08-09T13:33:41.734 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:33:41.729+0000 7f7145d57640 1 -- 172.21.15.43:0/653292167 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f71401f6010 con 0x7f71401588a0 2023-08-09T13:33:41.734 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:33:41.729+0000 7f7145d57640 1 -- 172.21.15.43:0/653292167 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f71401f5390 con 0x7f71401588a0 2023-08-09T13:33:41.734 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:33:41.730+0000 7f7127fff640 1 -- 172.21.15.43:0/653292167 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 1) v1 ==== 660+0+0 (secure 0 0 0) 0x7f713401e450 con 0x7f71401588a0 2023-08-09T13:33:41.735 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:33:41.730+0000 7f7127fff640 1 -- 172.21.15.43:0/653292167 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(1..1 src has 1..1) v4 ==== 725+0+0 (secure 0 0 0) 0x7f713400f7a0 con 0x7f71401588a0 2023-08-09T13:33:41.735 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:33:41.731+0000 7f7145d57640 1 -- 172.21.15.43:0/653292167 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f71400739f0 con 0x7f71401588a0 2023-08-09T13:33:41.735 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:33:41.734+0000 7f7127fff640 1 -- 172.21.15.43:0/653292167 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+72429 (secure 0 0 0) 0x7f7134023060 con 0x7f71401588a0 2023-08-09T13:33:41.808 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:33:41.806+0000 7f7145d57640 1 -- 172.21.15.43:0/653292167 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "osd crush tunables", "profile": "default"} v 0) v1 -- 0x7f714015dab0 con 0x7f71401588a0 2023-08-09T13:33:41.815 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:33:41.814+0000 7f7127fff640 1 -- 172.21.15.43:0/653292167 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "osd crush tunables", "profile": "default"}]=0 adjusted tunables profile to default v2) v1 ==== 124+0+0 (secure 0 0 0) 0x7f7134019070 con 0x7f71401588a0 2023-08-09T13:33:41.816 INFO:teuthology.orchestra.run.smithi043.stderr:adjusted tunables profile to default 2023-08-09T13:33:41.816 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:33:41.815+0000 7f7145d57640 1 -- 172.21.15.43:0/653292167 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f71401588a0 msgr2=0x7f71400b6490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:33:41.816 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:33:41.815+0000 7f7145d57640 1 --2- 172.21.15.43:0/653292167 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f71401588a0 0x7f71400b6490 secure :-1 s=READY pgs=5 cs=0 l=1 rev1=1 crypto rx=0x7f7134011600 tx=0x7f7134011a00 comp rx=0 tx=0).stop 2023-08-09T13:33:41.817 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:33:41.815+0000 7f7145d57640 1 -- 172.21.15.43:0/653292167 shutdown_connections 2023-08-09T13:33:41.817 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:33:41.815+0000 7f7145d57640 1 --2- 172.21.15.43:0/653292167 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f7140158d00 0x7f71400b69d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:33:41.817 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:33:41.815+0000 7f7145d57640 1 --2- 172.21.15.43:0/653292167 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f71401588a0 0x7f71400b6490 unknown :-1 s=CLOSED pgs=5 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:33:41.818 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:33:41.815+0000 7f7145d57640 1 --2- 172.21.15.43:0/653292167 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f71280051f0 0x7f7140154180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:33:41.818 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:33:41.815+0000 7f7145d57640 1 -- 172.21.15.43:0/653292167 >> 172.21.15.43:0/653292167 conn(0x7f7140071520 msgr2=0x7f71400b75d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-09T13:33:41.818 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:33:41.815+0000 7f7145d57640 1 -- 172.21.15.43:0/653292167 shutdown_connections 2023-08-09T13:33:41.818 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:33:41.815+0000 7f7145d57640 1 -- 172.21.15.43:0/653292167 wait complete. 2023-08-09T13:33:41.834 INFO:tasks.ceph:check_enable_crimson: False 2023-08-09T13:33:41.835 INFO:tasks.ceph:Starting osd daemons in cluster ceph... 2023-08-09T13:33:41.835 DEBUG:teuthology.orchestra.run.smithi043:> set -ex 2023-08-09T13:33:41.835 DEBUG:teuthology.orchestra.run.smithi043:> sudo dd if=/var/lib/ceph/osd/ceph-0/fsid of=/dev/stdout 2023-08-09T13:33:41.923 DEBUG:teuthology.orchestra.run.smithi043:> set -ex 2023-08-09T13:33:41.924 DEBUG:teuthology.orchestra.run.smithi043:> sudo dd if=/var/lib/ceph/osd/ceph-1/fsid of=/dev/stdout 2023-08-09T13:33:42.000 DEBUG:teuthology.orchestra.run.smithi043:> set -ex 2023-08-09T13:33:42.001 DEBUG:teuthology.orchestra.run.smithi043:> sudo dd if=/var/lib/ceph/osd/ceph-2/fsid of=/dev/stdout 2023-08-09T13:33:42.051 INFO:tasks.ceph.mgr.y.smithi043.stderr:2023-08-09T13:33:42.050+0000 7fe23d40f040 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2023-08-09T13:33:42.074 INFO:tasks.ceph.mgr.x.smithi203.stderr:2023-08-09T13:33:42.072+0000 7f943a4b2040 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2023-08-09T13:33:42.135 DEBUG:teuthology.orchestra.run.smithi043:> set -ex 2023-08-09T13:33:42.135 DEBUG:teuthology.orchestra.run.smithi043:> sudo dd if=/var/lib/ceph/osd/ceph-3/fsid of=/dev/stdout 2023-08-09T13:33:42.268 DEBUG:teuthology.orchestra.run.smithi203:> set -ex 2023-08-09T13:33:42.268 DEBUG:teuthology.orchestra.run.smithi203:> sudo dd if=/var/lib/ceph/osd/ceph-4/fsid of=/dev/stdout 2023-08-09T13:33:42.366 DEBUG:teuthology.orchestra.run.smithi203:> set -ex 2023-08-09T13:33:42.367 DEBUG:teuthology.orchestra.run.smithi203:> sudo dd if=/var/lib/ceph/osd/ceph-5/fsid of=/dev/stdout 2023-08-09T13:33:42.452 DEBUG:teuthology.orchestra.run.smithi203:> set -ex 2023-08-09T13:33:42.452 DEBUG:teuthology.orchestra.run.smithi203:> sudo dd if=/var/lib/ceph/osd/ceph-6/fsid of=/dev/stdout 2023-08-09T13:33:42.584 DEBUG:teuthology.orchestra.run.smithi203:> set -ex 2023-08-09T13:33:42.585 DEBUG:teuthology.orchestra.run.smithi203:> sudo dd if=/var/lib/ceph/osd/ceph-7/fsid of=/dev/stdout 2023-08-09T13:33:42.674 DEBUG:teuthology.orchestra.run.smithi203:> sudo ceph --cluster ceph osd new cbd49ecd-e896-4fbd-864e-5c37a829b165 0 2023-08-09T13:33:42.869 INFO:teuthology.orchestra.run.smithi203.stderr:2023-08-09T13:33:42.867+0000 7fc2e4d12640 1 Processor -- start 2023-08-09T13:33:42.870 INFO:teuthology.orchestra.run.smithi203.stderr:2023-08-09T13:33:42.867+0000 7fc2e4d12640 1 -- start start 2023-08-09T13:33:42.871 INFO:teuthology.orchestra.run.smithi203.stderr:2023-08-09T13:33:42.869+0000 7fc2e4d12640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fc2e00b0de0 0x7fc2e006b280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:33:42.871 INFO:teuthology.orchestra.run.smithi203.stderr:2023-08-09T13:33:42.869+0000 7fc2e4d12640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7fc2e00ca810 0x7fc2e00cac70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:33:42.871 INFO:teuthology.orchestra.run.smithi203.stderr:2023-08-09T13:33:42.869+0000 7fc2e4d12640 1 -- --> v1:172.21.15.43:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc2e00cefd0 con 0x7fc2e015c940 2023-08-09T13:33:42.872 INFO:teuthology.orchestra.run.smithi203.stderr:2023-08-09T13:33:42.869+0000 7fc2e4d12640 1 -- --> v2:172.21.15.43:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc2e00b1660 con 0x7fc2e00ca810 2023-08-09T13:33:42.872 INFO:teuthology.orchestra.run.smithi203.stderr:2023-08-09T13:33:42.869+0000 7fc2e4d12640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc2e00cf580 con 0x7fc2e00b0de0 2023-08-09T13:33:42.872 INFO:teuthology.orchestra.run.smithi203.stderr:2023-08-09T13:33:42.869+0000 7fc2ddd9b640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fc2e00b0de0 0x7fc2e006b280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:33:42.872 INFO:teuthology.orchestra.run.smithi203.stderr:2023-08-09T13:33:42.869+0000 7fc2ddd9b640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fc2e00b0de0 0x7fc2e006b280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.203:3301/0 says I am v2:172.21.15.203:59810/0 (socket says 172.21.15.203:59810) 2023-08-09T13:33:42.872 INFO:teuthology.orchestra.run.smithi203.stderr:2023-08-09T13:33:42.869+0000 7fc2ddd9b640 1 -- 172.21.15.203:0/500175880 learned_addr learned my addr 172.21.15.203:0/500175880 (peer_addr_for_me v2:172.21.15.203:0/0) 2023-08-09T13:33:42.873 INFO:teuthology.orchestra.run.smithi203.stderr:2023-08-09T13:33:42.869+0000 7fc2ded9d640 1 --2- 172.21.15.203:0/500175880 >> v2:172.21.15.43:3300/0 conn(0x7fc2e00ca810 0x7fc2e00cac70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:33:42.873 INFO:teuthology.orchestra.run.smithi203.stderr:2023-08-09T13:33:42.869+0000 7fc2ddd9b640 1 -- 172.21.15.203:0/500175880 >> v2:172.21.15.43:3300/0 conn(0x7fc2e00ca810 msgr2=0x7fc2e00cac70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-09T13:33:42.873 INFO:teuthology.orchestra.run.smithi203.stderr:2023-08-09T13:33:42.869+0000 7fc2ddd9b640 1 --2- 172.21.15.203:0/500175880 >> v2:172.21.15.43:3300/0 conn(0x7fc2e00ca810 0x7fc2e00cac70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:33:42.874 INFO:teuthology.orchestra.run.smithi203.stderr:2023-08-09T13:33:42.869+0000 7fc2ddd9b640 1 -- 172.21.15.203:0/500175880 >> v1:172.21.15.43:6789/0 conn(0x7fc2e015c940 legacy=0x7fc2e006ab70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-09T13:33:42.874 INFO:teuthology.orchestra.run.smithi203.stderr:2023-08-09T13:33:42.870+0000 7fc2ddd9b640 1 -- 172.21.15.203:0/500175880 --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc2e015cd20 con 0x7fc2e00b0de0 2023-08-09T13:33:42.874 INFO:teuthology.orchestra.run.smithi203.stderr:2023-08-09T13:33:42.870+0000 7fc2ded9d640 1 --2- 172.21.15.203:0/500175880 >> v2:172.21.15.43:3300/0 conn(0x7fc2e00ca810 0x7fc2e00cac70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-08-09T13:33:42.874 INFO:teuthology.orchestra.run.smithi203.stderr:2023-08-09T13:33:42.871+0000 7fc2ddd9b640 1 --2- 172.21.15.203:0/500175880 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fc2e00b0de0 0x7fc2e006b280 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7fc2d4009870 tx=0x7fc2d402b8f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=9f5646f73546299 server_cookie=0 in_seq=0 out_seq=0 2023-08-09T13:33:42.875 INFO:teuthology.orchestra.run.smithi203.stderr:2023-08-09T13:33:42.872+0000 7fc2dd59a640 1 -- 172.21.15.203:0/500175880 <== mon.2 v2:172.21.15.203:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc2d4031640 con 0x7fc2e00b0de0 2023-08-09T13:33:42.875 INFO:teuthology.orchestra.run.smithi203.stderr:2023-08-09T13:33:42.872+0000 7fc2dd59a640 1 -- 172.21.15.203:0/500175880 <== mon.2 v2:172.21.15.203:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc2d40317a0 con 0x7fc2e00b0de0 2023-08-09T13:33:42.875 INFO:teuthology.orchestra.run.smithi203.stderr:2023-08-09T13:33:42.872+0000 7fc2dd59a640 1 -- 172.21.15.203:0/500175880 <== mon.2 v2:172.21.15.203:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc2d4031b30 con 0x7fc2e00b0de0 2023-08-09T13:33:42.876 INFO:teuthology.orchestra.run.smithi203.stderr:2023-08-09T13:33:42.872+0000 7fc2e4d12640 1 -- 172.21.15.203:0/500175880 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fc2e00b0de0 msgr2=0x7fc2e006b280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:33:42.876 INFO:teuthology.orchestra.run.smithi203.stderr:2023-08-09T13:33:42.872+0000 7fc2e4d12640 1 --2- 172.21.15.203:0/500175880 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fc2e00b0de0 0x7fc2e006b280 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7fc2d4009870 tx=0x7fc2d402b8f0 comp rx=0 tx=0).stop 2023-08-09T13:33:42.876 INFO:teuthology.orchestra.run.smithi203.stderr:2023-08-09T13:33:42.872+0000 7fc2e4d12640 1 -- 172.21.15.203:0/500175880 shutdown_connections 2023-08-09T13:33:42.876 INFO:teuthology.orchestra.run.smithi203.stderr:2023-08-09T13:33:42.872+0000 7fc2e4d12640 1 --2- 172.21.15.203:0/500175880 >> v2:172.21.15.43:3300/0 conn(0x7fc2e00ca810 0x7fc2e00cac70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:33:42.876 INFO:teuthology.orchestra.run.smithi203.stderr:2023-08-09T13:33:42.872+0000 7fc2e4d12640 1 --2- 172.21.15.203:0/500175880 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fc2e00b0de0 0x7fc2e006b280 unknown :-1 s=CLOSED pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:33:42.876 INFO:teuthology.orchestra.run.smithi203.stderr:2023-08-09T13:33:42.872+0000 7fc2e4d12640 1 -- 172.21.15.203:0/500175880 >> 172.21.15.203:0/500175880 conn(0x7fc2e0089fe0 msgr2=0x7fc2e00d01c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-09T13:33:42.877 INFO:teuthology.orchestra.run.smithi203.stderr:2023-08-09T13:33:42.872+0000 7fc2e4d12640 1 -- 172.21.15.203:0/500175880 shutdown_connections 2023-08-09T13:33:42.877 INFO:teuthology.orchestra.run.smithi203.stderr:2023-08-09T13:33:42.873+0000 7fc2e4d12640 1 -- 172.21.15.203:0/500175880 wait complete. 2023-08-09T13:33:42.877 INFO:teuthology.orchestra.run.smithi203.stderr:2023-08-09T13:33:42.874+0000 7fc2e4d12640 1 Processor -- start 2023-08-09T13:33:42.877 INFO:teuthology.orchestra.run.smithi203.stderr:2023-08-09T13:33:42.875+0000 7fc2e4d12640 1 -- start start 2023-08-09T13:33:42.879 INFO:teuthology.orchestra.run.smithi203.stderr:2023-08-09T13:33:42.875+0000 7fc2e4d12640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc2e00b0de0 0x7fc2e01ef830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:33:42.879 INFO:teuthology.orchestra.run.smithi203.stderr:2023-08-09T13:33:42.876+0000 7fc2e4d12640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fc2e00ca810 0x7fc2e00c9690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:33:42.879 INFO:teuthology.orchestra.run.smithi203.stderr:2023-08-09T13:33:42.876+0000 7fc2e4d12640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fc2e015c940 0x7fc2e00c9bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:33:42.879 INFO:teuthology.orchestra.run.smithi203.stderr:2023-08-09T13:33:42.876+0000 7fc2e4d12640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc2e01f31f0 con 0x7fc2e00b0de0 2023-08-09T13:33:42.879 INFO:teuthology.orchestra.run.smithi203.stderr:2023-08-09T13:33:42.876+0000 7fc2e4d12640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc2e01f2370 con 0x7fc2e015c940 2023-08-09T13:33:42.879 INFO:teuthology.orchestra.run.smithi203.stderr:2023-08-09T13:33:42.876+0000 7fc2e4d12640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc2e01f25e0 con 0x7fc2e00ca810 2023-08-09T13:33:42.880 INFO:teuthology.orchestra.run.smithi203.stderr:2023-08-09T13:33:42.876+0000 7fc2ddd9b640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fc2e00ca810 0x7fc2e00c9690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:33:42.880 INFO:teuthology.orchestra.run.smithi203.stderr:2023-08-09T13:33:42.876+0000 7fc2ded9d640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fc2e015c940 0x7fc2e00c9bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:33:42.880 INFO:teuthology.orchestra.run.smithi203.stderr:2023-08-09T13:33:42.876+0000 7fc2ddd9b640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fc2e00ca810 0x7fc2e00c9690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.203:3301/0 says I am v2:172.21.15.203:59816/0 (socket says 172.21.15.203:59816) 2023-08-09T13:33:42.880 INFO:teuthology.orchestra.run.smithi203.stderr:2023-08-09T13:33:42.876+0000 7fc2ddd9b640 1 -- 172.21.15.203:0/442336448 learned_addr learned my addr 172.21.15.203:0/442336448 (peer_addr_for_me v2:172.21.15.203:0/0) 2023-08-09T13:33:42.880 INFO:teuthology.orchestra.run.smithi203.stderr:2023-08-09T13:33:42.876+0000 7fc2de59c640 1 --2- 172.21.15.203:0/442336448 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc2e00b0de0 0x7fc2e01ef830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:33:42.880 INFO:teuthology.orchestra.run.smithi203.stderr:2023-08-09T13:33:42.876+0000 7fc2ddd9b640 1 -- 172.21.15.203:0/442336448 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fc2e015c940 msgr2=0x7fc2e00c9bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:33:42.880 INFO:teuthology.orchestra.run.smithi203.stderr:2023-08-09T13:33:42.876+0000 7fc2ddd9b640 1 --2- 172.21.15.203:0/442336448 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fc2e015c940 0x7fc2e00c9bd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:33:42.881 INFO:teuthology.orchestra.run.smithi203.stderr:2023-08-09T13:33:42.876+0000 7fc2ddd9b640 1 -- 172.21.15.203:0/442336448 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc2e00b0de0 msgr2=0x7fc2e01ef830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:33:42.881 INFO:teuthology.orchestra.run.smithi203.stderr:2023-08-09T13:33:42.876+0000 7fc2ddd9b640 1 --2- 172.21.15.203:0/442336448 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc2e00b0de0 0x7fc2e01ef830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:33:42.881 INFO:teuthology.orchestra.run.smithi203.stderr:2023-08-09T13:33:42.876+0000 7fc2ddd9b640 1 -- 172.21.15.203:0/442336448 --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc2d40094d0 con 0x7fc2e00ca810 2023-08-09T13:33:42.881 INFO:teuthology.orchestra.run.smithi203.stderr:2023-08-09T13:33:42.877+0000 7fc2ddd9b640 1 --2- 172.21.15.203:0/442336448 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fc2e00ca810 0x7fc2e00c9690 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7fc2d402b930 tx=0x7fc2d40046c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-09T13:33:42.882 INFO:teuthology.orchestra.run.smithi203.stderr:2023-08-09T13:33:42.877+0000 7fc2ceffd640 1 -- 172.21.15.203:0/442336448 <== mon.2 v2:172.21.15.203:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc2d403ac60 con 0x7fc2e00ca810 2023-08-09T13:33:42.882 INFO:teuthology.orchestra.run.smithi203.stderr:2023-08-09T13:33:42.877+0000 7fc2ceffd640 1 -- 172.21.15.203:0/442336448 <== mon.2 v2:172.21.15.203:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc2d403ade0 con 0x7fc2e00ca810 2023-08-09T13:33:42.882 INFO:teuthology.orchestra.run.smithi203.stderr:2023-08-09T13:33:42.877+0000 7fc2ceffd640 1 -- 172.21.15.203:0/442336448 <== mon.2 v2:172.21.15.203:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc2d4042630 con 0x7fc2e00ca810 2023-08-09T13:33:42.882 INFO:teuthology.orchestra.run.smithi203.stderr:2023-08-09T13:33:42.877+0000 7fc2e4d12640 1 -- 172.21.15.203:0/442336448 --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc2e01f28c0 con 0x7fc2e00ca810 2023-08-09T13:33:42.882 INFO:teuthology.orchestra.run.smithi203.stderr:2023-08-09T13:33:42.877+0000 7fc2e4d12640 1 -- 172.21.15.203:0/442336448 --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc2e01f2e80 con 0x7fc2e00ca810 2023-08-09T13:33:42.883 INFO:teuthology.orchestra.run.smithi203.stderr:2023-08-09T13:33:42.877+0000 7fc2ceffd640 1 -- 172.21.15.203:0/442336448 <== mon.2 v2:172.21.15.203:3301/0 4 ==== mgrmap(e 1) v1 ==== 660+0+0 (secure 0 0 0) 0x7fc2d402f050 con 0x7fc2e00ca810 2023-08-09T13:33:42.883 INFO:teuthology.orchestra.run.smithi203.stderr:2023-08-09T13:33:42.878+0000 7fc2ceffd640 1 -- 172.21.15.203:0/442336448 <== mon.2 v2:172.21.15.203:3301/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 725+0+0 (secure 0 0 0) 0x7fc2d404b8d0 con 0x7fc2e00ca810 2023-08-09T13:33:42.883 INFO:teuthology.orchestra.run.smithi203.stderr:2023-08-09T13:33:42.879+0000 7fc2e4d12640 1 -- 172.21.15.203:0/442336448 --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc2e01f3360 con 0x7fc2e00ca810 2023-08-09T13:33:42.883 INFO:teuthology.orchestra.run.smithi203.stderr:2023-08-09T13:33:42.881+0000 7fc2ceffd640 1 -- 172.21.15.203:0/442336448 <== mon.2 v2:172.21.15.203:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+72429 (secure 0 0 0) 0x7fc2d4047070 con 0x7fc2e00ca810 2023-08-09T13:33:42.976 INFO:teuthology.orchestra.run.smithi203.stderr:2023-08-09T13:33:42.974+0000 7fc2e4d12640 1 -- 172.21.15.203:0/442336448 --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_command({"prefix": "osd new", "uuid": "cbd49ecd-e896-4fbd-864e-5c37a829b165", "id": 0} v 0) v1 -- 0x7fc2e00739f0 con 0x7fc2e00ca810 2023-08-09T13:33:42.984 INFO:teuthology.orchestra.run.smithi203.stderr:2023-08-09T13:33:42.982+0000 7fc2ceffd640 1 -- 172.21.15.203:0/442336448 <== mon.2 v2:172.21.15.203:3301/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "cbd49ecd-e896-4fbd-864e-5c37a829b165", "id": 0}]=0 v3) v1 ==== 112+0+2 (secure 0 0 0) 0x7fc2d4035020 con 0x7fc2e00ca810 2023-08-09T13:33:42.985 INFO:teuthology.orchestra.run.smithi203.stdout:0 2023-08-09T13:33:42.986 INFO:teuthology.orchestra.run.smithi203.stderr:2023-08-09T13:33:42.984+0000 7fc2e4d12640 1 -- 172.21.15.203:0/442336448 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fc2e00ca810 msgr2=0x7fc2e00c9690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:33:42.986 INFO:teuthology.orchestra.run.smithi203.stderr:2023-08-09T13:33:42.984+0000 7fc2e4d12640 1 --2- 172.21.15.203:0/442336448 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fc2e00ca810 0x7fc2e00c9690 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7fc2d402b930 tx=0x7fc2d40046c0 comp rx=0 tx=0).stop 2023-08-09T13:33:42.986 INFO:teuthology.orchestra.run.smithi203.stderr:2023-08-09T13:33:42.984+0000 7fc2e4d12640 1 -- 172.21.15.203:0/442336448 shutdown_connections 2023-08-09T13:33:42.986 INFO:teuthology.orchestra.run.smithi203.stderr:2023-08-09T13:33:42.984+0000 7fc2e4d12640 1 --2- 172.21.15.203:0/442336448 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fc2e015c940 0x7fc2e00c9bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:33:42.987 INFO:teuthology.orchestra.run.smithi203.stderr:2023-08-09T13:33:42.984+0000 7fc2e4d12640 1 --2- 172.21.15.203:0/442336448 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fc2e00ca810 0x7fc2e00c9690 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:33:42.987 INFO:teuthology.orchestra.run.smithi203.stderr:2023-08-09T13:33:42.984+0000 7fc2e4d12640 1 --2- 172.21.15.203:0/442336448 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc2e00b0de0 0x7fc2e01ef830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:33:42.987 INFO:teuthology.orchestra.run.smithi203.stderr:2023-08-09T13:33:42.984+0000 7fc2e4d12640 1 -- 172.21.15.203:0/442336448 >> 172.21.15.203:0/442336448 conn(0x7fc2e0089fe0 msgr2=0x7fc2e006f0e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-09T13:33:42.987 INFO:teuthology.orchestra.run.smithi203.stderr:2023-08-09T13:33:42.984+0000 7fc2e4d12640 1 -- 172.21.15.203:0/442336448 shutdown_connections 2023-08-09T13:33:42.987 INFO:teuthology.orchestra.run.smithi203.stderr:2023-08-09T13:33:42.984+0000 7fc2e4d12640 1 -- 172.21.15.203:0/442336448 wait complete. 2023-08-09T13:33:43.002 DEBUG:teuthology.orchestra.run.smithi203:> sudo ceph --cluster ceph osd new 75c7e0fe-1ec7-4fe8-a2f2-3661a65b8500 1 2023-08-09T13:33:43.008 INFO:tasks.ceph.mgr.y.smithi043.stderr:2023-08-09T13:33:43.006+0000 7fe23d40f040 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2023-08-09T13:33:43.077 INFO:tasks.ceph.mgr.x.smithi203.stderr:2023-08-09T13:33:43.075+0000 7f943a4b2040 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2023-08-09T13:33:43.189 INFO:tasks.ceph.mgr.y.smithi043.stderr:2023-08-09T13:33:43.187+0000 7fe23d40f040 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2023-08-09T13:33:43.196 INFO:teuthology.orchestra.run.smithi203.stderr:2023-08-09T13:33:43.194+0000 7f5124c26640 1 Processor -- start 2023-08-09T13:33:43.196 INFO:teuthology.orchestra.run.smithi203.stderr:2023-08-09T13:33:43.194+0000 7f5124c26640 1 -- start start 2023-08-09T13:33:43.197 INFO:teuthology.orchestra.run.smithi203.stderr:2023-08-09T13:33:43.195+0000 7f5124c26640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f512015c620 0x7f51200cceb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:33:43.197 INFO:teuthology.orchestra.run.smithi203.stderr:2023-08-09T13:33:43.195+0000 7f5124c26640 1 -- --> v1:172.21.15.43:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f51200cd700 con 0x7f51200cf510 2023-08-09T13:33:43.198 INFO:teuthology.orchestra.run.smithi203.stderr:2023-08-09T13:33:43.195+0000 7f5124c26640 1 -- --> v1:172.21.15.203:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f51200cd9e0 con 0x7f51200b1260 2023-08-09T13:33:43.198 INFO:teuthology.orchestra.run.smithi203.stderr:2023-08-09T13:33:43.195+0000 7f5124c26640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f51200b1b70 con 0x7f512015c620 2023-08-09T13:33:43.198 INFO:teuthology.orchestra.run.smithi203.stderr:2023-08-09T13:33:43.196+0000 7f5117fff640 1 --1- >> v1:172.21.15.203:6789/0 conn(0x7f51200b1260 0x7f51200cee00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.203:6789/0 says I am v1:172.21.15.203:51330/0 (socket says 172.21.15.203:51330) 2023-08-09T13:33:43.199 INFO:teuthology.orchestra.run.smithi203.stderr:2023-08-09T13:33:43.196+0000 7f5117fff640 1 -- 172.21.15.203:0/1944123438 learned_addr learned my addr 172.21.15.203:0/1944123438 (peer_addr_for_me v1:172.21.15.203:0/0) 2023-08-09T13:33:43.199 INFO:teuthology.orchestra.run.smithi203.stderr:2023-08-09T13:33:43.196+0000 7f511e59c640 1 --2- 172.21.15.203:0/1944123438 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f512015c620 0x7f51200cceb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:33:43.199 INFO:teuthology.orchestra.run.smithi203.stderr:2023-08-09T13:33:43.196+0000 7f511dd9b640 1 -- 172.21.15.203:0/1944123438 <== mon.1 v1:172.21.15.203:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 611721274 0 0) 0x7f51200cd9e0 con 0x7f51200b1260 2023-08-09T13:33:43.199 INFO:teuthology.orchestra.run.smithi203.stderr:2023-08-09T13:33:43.196+0000 7f511dd9b640 1 -- 172.21.15.203:0/1944123438 --> v1:172.21.15.203:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f510c003520 con 0x7f51200b1260 2023-08-09T13:33:43.199 INFO:teuthology.orchestra.run.smithi203.stderr:2023-08-09T13:33:43.196+0000 7f511e59c640 1 -- 172.21.15.203:0/1944123438 >> v1:172.21.15.203:6789/0 conn(0x7f51200b1260 legacy=0x7f51200cee00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:33:43.199 INFO:teuthology.orchestra.run.smithi203.stderr:2023-08-09T13:33:43.196+0000 7f511e59c640 1 -- 172.21.15.203:0/1944123438 >> v1:172.21.15.43:6789/0 conn(0x7f51200cf510 legacy=0x7f51200d0350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:33:43.200 INFO:teuthology.orchestra.run.smithi203.stderr:2023-08-09T13:33:43.196+0000 7f511e59c640 1 -- 172.21.15.203:0/1944123438 --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f51200cdcc0 con 0x7f512015c620 2023-08-09T13:33:43.201 INFO:teuthology.orchestra.run.smithi203.stderr:2023-08-09T13:33:43.197+0000 7f511e59c640 1 --2- 172.21.15.203:0/1944123438 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f512015c620 0x7f51200cceb0 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7f51100097c0 tx=0x7f511002b820 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c16ab424830a16f0 server_cookie=0 in_seq=0 out_seq=0 2023-08-09T13:33:43.201 INFO:teuthology.orchestra.run.smithi203.stderr:2023-08-09T13:33:43.198+0000 7f511dd9b640 1 -- 172.21.15.203:0/1944123438 <== mon.2 v2:172.21.15.203:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5110031780 con 0x7f512015c620 2023-08-09T13:33:43.201 INFO:teuthology.orchestra.run.smithi203.stderr:2023-08-09T13:33:43.198+0000 7f511dd9b640 1 -- 172.21.15.203:0/1944123438 <== mon.2 v2:172.21.15.203:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f51100318e0 con 0x7f512015c620 2023-08-09T13:33:43.201 INFO:teuthology.orchestra.run.smithi203.stderr:2023-08-09T13:33:43.198+0000 7f511dd9b640 1 -- 172.21.15.203:0/1944123438 <== mon.2 v2:172.21.15.203:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5110031c70 con 0x7f512015c620 2023-08-09T13:33:43.201 INFO:teuthology.orchestra.run.smithi203.stderr:2023-08-09T13:33:43.198+0000 7f5124c26640 1 -- 172.21.15.203:0/1944123438 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f512015c620 msgr2=0x7f51200cceb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:33:43.201 INFO:teuthology.orchestra.run.smithi203.stderr:2023-08-09T13:33:43.198+0000 7f5124c26640 1 --2- 172.21.15.203:0/1944123438 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f512015c620 0x7f51200cceb0 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7f51100097c0 tx=0x7f511002b820 comp rx=0 tx=0).stop 2023-08-09T13:33:43.201 INFO:teuthology.orchestra.run.smithi203.stderr:2023-08-09T13:33:43.198+0000 7f5124c26640 1 -- 172.21.15.203:0/1944123438 shutdown_connections 2023-08-09T13:33:43.202 INFO:teuthology.orchestra.run.smithi203.stderr:2023-08-09T13:33:43.198+0000 7f5124c26640 1 --2- 172.21.15.203:0/1944123438 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f512015c620 0x7f51200cceb0 unknown :-1 s=CLOSED pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:33:43.202 INFO:teuthology.orchestra.run.smithi203.stderr:2023-08-09T13:33:43.198+0000 7f5124c26640 1 -- 172.21.15.203:0/1944123438 >> 172.21.15.203:0/1944123438 conn(0x7f512009b750 msgr2=0x7f51200d2410 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-09T13:33:43.202 INFO:teuthology.orchestra.run.smithi203.stderr:2023-08-09T13:33:43.198+0000 7f5124c26640 1 -- 172.21.15.203:0/1944123438 shutdown_connections 2023-08-09T13:33:43.202 INFO:teuthology.orchestra.run.smithi203.stderr:2023-08-09T13:33:43.198+0000 7f5124c26640 1 -- 172.21.15.203:0/1944123438 wait complete. 2023-08-09T13:33:43.203 INFO:teuthology.orchestra.run.smithi203.stderr:2023-08-09T13:33:43.200+0000 7f5124c26640 1 Processor -- start 2023-08-09T13:33:43.203 INFO:teuthology.orchestra.run.smithi203.stderr:2023-08-09T13:33:43.200+0000 7f5124c26640 1 -- start start 2023-08-09T13:33:43.204 INFO:teuthology.orchestra.run.smithi203.stderr:2023-08-09T13:33:43.201+0000 7f5124c26640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f51200b1260 0x7f512009fc80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:33:43.204 INFO:teuthology.orchestra.run.smithi203.stderr:2023-08-09T13:33:43.201+0000 7f5124c26640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f51200cf510 0x7f51200b27a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:33:43.204 INFO:teuthology.orchestra.run.smithi203.stderr:2023-08-09T13:33:43.201+0000 7f5124c26640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f512015c620 0x7f51201f1f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:33:43.205 INFO:teuthology.orchestra.run.smithi203.stderr:2023-08-09T13:33:43.201+0000 7f5124c26640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f512009cde0 con 0x7f512015c620 2023-08-09T13:33:43.205 INFO:teuthology.orchestra.run.smithi203.stderr:2023-08-09T13:33:43.201+0000 7f5124c26640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f51200cb280 con 0x7f51200cf510 2023-08-09T13:33:43.205 INFO:teuthology.orchestra.run.smithi203.stderr:2023-08-09T13:33:43.201+0000 7f5124c26640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f51200a01c0 con 0x7f51200b1260 2023-08-09T13:33:43.205 INFO:teuthology.orchestra.run.smithi203.stderr:2023-08-09T13:33:43.201+0000 7f511e59c640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f51200b1260 0x7f512009fc80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:33:43.205 INFO:teuthology.orchestra.run.smithi203.stderr:2023-08-09T13:33:43.201+0000 7f511e59c640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f51200b1260 0x7f512009fc80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.203:3301/0 says I am v2:172.21.15.203:59834/0 (socket says 172.21.15.203:59834) 2023-08-09T13:33:43.205 INFO:teuthology.orchestra.run.smithi203.stderr:2023-08-09T13:33:43.201+0000 7f5117fff640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f51200cf510 0x7f51200b27a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:33:43.205 INFO:teuthology.orchestra.run.smithi203.stderr:2023-08-09T13:33:43.201+0000 7f511e59c640 1 -- 172.21.15.203:0/956161000 learned_addr learned my addr 172.21.15.203:0/956161000 (peer_addr_for_me v2:172.21.15.203:0/0) 2023-08-09T13:33:43.206 INFO:teuthology.orchestra.run.smithi203.stderr:2023-08-09T13:33:43.202+0000 7f511ed9d640 1 --2- 172.21.15.203:0/956161000 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f512015c620 0x7f51201f1f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:33:43.206 INFO:teuthology.orchestra.run.smithi203.stderr:2023-08-09T13:33:43.202+0000 7f511e59c640 1 -- 172.21.15.203:0/956161000 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f51200cf510 msgr2=0x7f51200b27a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:33:43.206 INFO:teuthology.orchestra.run.smithi203.stderr:2023-08-09T13:33:43.202+0000 7f511e59c640 1 --2- 172.21.15.203:0/956161000 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f51200cf510 0x7f51200b27a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:33:43.206 INFO:teuthology.orchestra.run.smithi203.stderr:2023-08-09T13:33:43.202+0000 7f511e59c640 1 -- 172.21.15.203:0/956161000 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f512015c620 msgr2=0x7f51201f1f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:33:43.206 INFO:teuthology.orchestra.run.smithi203.stderr:2023-08-09T13:33:43.202+0000 7f511e59c640 1 --2- 172.21.15.203:0/956161000 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f512015c620 0x7f51201f1f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:33:43.206 INFO:teuthology.orchestra.run.smithi203.stderr:2023-08-09T13:33:43.202+0000 7f511e59c640 1 -- 172.21.15.203:0/956161000 --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f51100093a0 con 0x7f51200b1260 2023-08-09T13:33:43.206 INFO:teuthology.orchestra.run.smithi203.stderr:2023-08-09T13:33:43.202+0000 7f5117fff640 1 --2- 172.21.15.203:0/956161000 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f51200cf510 0x7f51200b27a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-08-09T13:33:43.206 INFO:teuthology.orchestra.run.smithi203.stderr:2023-08-09T13:33:43.202+0000 7f511e59c640 1 --2- 172.21.15.203:0/956161000 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f51200b1260 0x7f512009fc80 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7f51100097c0 tx=0x7f51100046c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-09T13:33:43.207 INFO:teuthology.orchestra.run.smithi203.stderr:2023-08-09T13:33:43.202+0000 7f5116ffd640 1 -- 172.21.15.203:0/956161000 <== mon.2 v2:172.21.15.203:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5110032820 con 0x7f51200b1260 2023-08-09T13:33:43.207 INFO:teuthology.orchestra.run.smithi203.stderr:2023-08-09T13:33:43.202+0000 7f5116ffd640 1 -- 172.21.15.203:0/956161000 <== mon.2 v2:172.21.15.203:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5110032980 con 0x7f51200b1260 2023-08-09T13:33:43.208 INFO:teuthology.orchestra.run.smithi203.stderr:2023-08-09T13:33:43.202+0000 7f5116ffd640 1 -- 172.21.15.203:0/956161000 <== mon.2 v2:172.21.15.203:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5110032d30 con 0x7f51200b1260 2023-08-09T13:33:43.208 INFO:teuthology.orchestra.run.smithi203.stderr:2023-08-09T13:33:43.202+0000 7f5124c26640 1 -- 172.21.15.203:0/956161000 --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f51200cbd40 con 0x7f51200b1260 2023-08-09T13:33:43.208 INFO:teuthology.orchestra.run.smithi203.stderr:2023-08-09T13:33:43.203+0000 7f5124c26640 1 -- 172.21.15.203:0/956161000 --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f51200cc300 con 0x7f51200b1260 2023-08-09T13:33:43.208 INFO:teuthology.orchestra.run.smithi203.stderr:2023-08-09T13:33:43.203+0000 7f5116ffd640 1 -- 172.21.15.203:0/956161000 <== mon.2 v2:172.21.15.203:3301/0 4 ==== mgrmap(e 1) v1 ==== 660+0+0 (secure 0 0 0) 0x7f5110005990 con 0x7f51200b1260 2023-08-09T13:33:43.208 INFO:teuthology.orchestra.run.smithi203.stderr:2023-08-09T13:33:43.203+0000 7f5116ffd640 1 -- 172.21.15.203:0/956161000 <== mon.2 v2:172.21.15.203:3301/0 5 ==== osd_map(3..3 src has 1..3) v4 ==== 840+0+0 (secure 0 0 0) 0x7f51100453f0 con 0x7f51200b1260 2023-08-09T13:33:43.208 INFO:teuthology.orchestra.run.smithi203.stderr:2023-08-09T13:33:43.204+0000 7f5124c26640 1 -- 172.21.15.203:0/956161000 --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f512009cf20 con 0x7f51200b1260 2023-08-09T13:33:43.209 INFO:teuthology.orchestra.run.smithi203.stderr:2023-08-09T13:33:43.206+0000 7f5116ffd640 1 -- 172.21.15.203:0/956161000 <== mon.2 v2:172.21.15.203:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+72429 (secure 0 0 0) 0x7f512009cf20 con 0x7f51200b1260 2023-08-09T13:33:43.248 INFO:tasks.ceph.mgr.y.smithi043.stderr:2023-08-09T13:33:43.246+0000 7fe23d40f040 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2023-08-09T13:33:43.268 INFO:tasks.ceph.mgr.x.smithi203.stderr:2023-08-09T13:33:43.266+0000 7f943a4b2040 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2023-08-09T13:33:43.286 INFO:teuthology.orchestra.run.smithi203.stderr:2023-08-09T13:33:43.284+0000 7f5124c26640 1 -- 172.21.15.203:0/956161000 --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_command({"prefix": "osd new", "uuid": "75c7e0fe-1ec7-4fe8-a2f2-3661a65b8500", "id": 1} v 0) v1 -- 0x7f51200739f0 con 0x7f51200b1260 2023-08-09T13:33:43.293 INFO:teuthology.orchestra.run.smithi203.stderr:2023-08-09T13:33:43.291+0000 7f5116ffd640 1 -- 172.21.15.203:0/956161000 <== mon.2 v2:172.21.15.203:3301/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "75c7e0fe-1ec7-4fe8-a2f2-3661a65b8500", "id": 1}]=0 v4) v1 ==== 112+0+2 (secure 0 0 0) 0x7f5110005dd0 con 0x7f51200b1260 2023-08-09T13:33:43.294 INFO:teuthology.orchestra.run.smithi203.stdout:1 2023-08-09T13:33:43.295 INFO:teuthology.orchestra.run.smithi203.stderr:2023-08-09T13:33:43.292+0000 7f5124c26640 1 -- 172.21.15.203:0/956161000 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f51200b1260 msgr2=0x7f512009fc80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:33:43.295 INFO:teuthology.orchestra.run.smithi203.stderr:2023-08-09T13:33:43.292+0000 7f5124c26640 1 --2- 172.21.15.203:0/956161000 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f51200b1260 0x7f512009fc80 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7f51100097c0 tx=0x7f51100046c0 comp rx=0 tx=0).stop 2023-08-09T13:33:43.296 INFO:teuthology.orchestra.run.smithi203.stderr:2023-08-09T13:33:43.292+0000 7f5124c26640 1 -- 172.21.15.203:0/956161000 shutdown_connections 2023-08-09T13:33:43.296 INFO:teuthology.orchestra.run.smithi203.stderr:2023-08-09T13:33:43.292+0000 7f5124c26640 1 --2- 172.21.15.203:0/956161000 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f512015c620 0x7f51201f1f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:33:43.296 INFO:teuthology.orchestra.run.smithi203.stderr:2023-08-09T13:33:43.292+0000 7f5124c26640 1 --2- 172.21.15.203:0/956161000 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f51200cf510 0x7f51200b27a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:33:43.296 INFO:teuthology.orchestra.run.smithi203.stderr:2023-08-09T13:33:43.292+0000 7f5124c26640 1 --2- 172.21.15.203:0/956161000 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f51200b1260 0x7f512009fc80 unknown :-1 s=CLOSED pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:33:43.297 INFO:teuthology.orchestra.run.smithi203.stderr:2023-08-09T13:33:43.292+0000 7f5124c26640 1 -- 172.21.15.203:0/956161000 >> 172.21.15.203:0/956161000 conn(0x7f512009b750 msgr2=0x7f512006b990 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-09T13:33:43.297 INFO:teuthology.orchestra.run.smithi203.stderr:2023-08-09T13:33:43.292+0000 7f5124c26640 1 -- 172.21.15.203:0/956161000 shutdown_connections 2023-08-09T13:33:43.297 INFO:teuthology.orchestra.run.smithi203.stderr:2023-08-09T13:33:43.292+0000 7f5124c26640 1 -- 172.21.15.203:0/956161000 wait complete. 2023-08-09T13:33:43.303 INFO:tasks.ceph.mgr.y.smithi043.stderr:2023-08-09T13:33:43.301+0000 7fe23d40f040 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2023-08-09T13:33:43.311 DEBUG:teuthology.orchestra.run.smithi203:> sudo ceph --cluster ceph osd new bb5c41e9-7f5f-4125-9b01-3bb8bfc2ea83 2 2023-08-09T13:33:43.331 INFO:tasks.ceph.mgr.x.smithi203.stderr:2023-08-09T13:33:43.329+0000 7f943a4b2040 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2023-08-09T13:33:43.389 INFO:tasks.ceph.mgr.x.smithi203.stderr:2023-08-09T13:33:43.386+0000 7f943a4b2040 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2023-08-09T13:33:43.504 INFO:teuthology.orchestra.run.smithi203.stderr:2023-08-09T13:33:43.502+0000 7f031d5b1640 1 Processor -- start 2023-08-09T13:33:43.505 INFO:teuthology.orchestra.run.smithi203.stderr:2023-08-09T13:33:43.502+0000 7f031d5b1640 1 -- start start 2023-08-09T13:33:43.506 INFO:teuthology.orchestra.run.smithi203.stderr:2023-08-09T13:33:43.504+0000 7f031d5b1640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f031815c620 0x7f03180cceb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:33:43.506 INFO:teuthology.orchestra.run.smithi203.stderr:2023-08-09T13:33:43.504+0000 7f031d5b1640 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7f03180b1260 0x7f03180cee00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:33:43.506 INFO:teuthology.orchestra.run.smithi203.stderr:2023-08-09T13:33:43.504+0000 7f031d5b1640 1 -- --> v1:172.21.15.43:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f03180d1360 con 0x7f03180cf340 2023-08-09T13:33:43.507 INFO:teuthology.orchestra.run.smithi203.stderr:2023-08-09T13:33:43.504+0000 7f031d5b1640 1 -- --> v2:172.21.15.203:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f03180b1b70 con 0x7f03180b1260 2023-08-09T13:33:43.507 INFO:teuthology.orchestra.run.smithi203.stderr:2023-08-09T13:33:43.504+0000 7f031d5b1640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f03180cf7b0 con 0x7f031815c620 2023-08-09T13:33:43.508 INFO:teuthology.orchestra.run.smithi203.stderr:2023-08-09T13:33:43.504+0000 7f030ffff640 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7f03180b1260 0x7f03180cee00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:33:43.508 INFO:teuthology.orchestra.run.smithi203.stderr:2023-08-09T13:33:43.504+0000 7f0316ffd640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f031815c620 0x7f03180cceb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:33:43.508 INFO:teuthology.orchestra.run.smithi203.stderr:2023-08-09T13:33:43.504+0000 7f0316ffd640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f031815c620 0x7f03180cceb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.203:3301/0 says I am v2:172.21.15.203:59850/0 (socket says 172.21.15.203:59850) 2023-08-09T13:33:43.508 INFO:teuthology.orchestra.run.smithi203.stderr:2023-08-09T13:33:43.504+0000 7f030ffff640 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7f03180b1260 0x7f03180cee00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.203:3300/0 says I am v2:172.21.15.203:48876/0 (socket says 172.21.15.203:48876) 2023-08-09T13:33:43.508 INFO:teuthology.orchestra.run.smithi203.stderr:2023-08-09T13:33:43.504+0000 7f0316ffd640 1 -- 172.21.15.203:0/3627178132 learned_addr learned my addr 172.21.15.203:0/3627178132 (peer_addr_for_me v2:172.21.15.203:0/0) 2023-08-09T13:33:43.509 INFO:teuthology.orchestra.run.smithi203.stderr:2023-08-09T13:33:43.505+0000 7f0316ffd640 1 -- 172.21.15.203:0/3627178132 >> v2:172.21.15.203:3300/0 conn(0x7f03180b1260 msgr2=0x7f03180cee00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-09T13:33:43.509 INFO:teuthology.orchestra.run.smithi203.stderr:2023-08-09T13:33:43.505+0000 7f0316ffd640 1 --2- 172.21.15.203:0/3627178132 >> v2:172.21.15.203:3300/0 conn(0x7f03180b1260 0x7f03180cee00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:33:43.509 INFO:teuthology.orchestra.run.smithi203.stderr:2023-08-09T13:33:43.505+0000 7f0316ffd640 1 -- 172.21.15.203:0/3627178132 >> v1:172.21.15.43:6789/0 conn(0x7f03180cf340 legacy=0x7f0318160280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-09T13:33:43.509 INFO:teuthology.orchestra.run.smithi203.stderr:2023-08-09T13:33:43.505+0000 7f0316ffd640 1 -- 172.21.15.203:0/3627178132 --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f03180cd700 con 0x7f031815c620 2023-08-09T13:33:43.510 INFO:teuthology.orchestra.run.smithi203.stderr:2023-08-09T13:33:43.506+0000 7f0316ffd640 1 --2- 172.21.15.203:0/3627178132 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f031815c620 0x7f03180cceb0 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7f0308009850 tx=0x7f030802b7c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=dfe1430ffc7208c2 server_cookie=0 in_seq=0 out_seq=0 2023-08-09T13:33:43.510 INFO:teuthology.orchestra.run.smithi203.stderr:2023-08-09T13:33:43.506+0000 7f03167fc640 1 -- 172.21.15.203:0/3627178132 <== mon.2 v2:172.21.15.203:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0308004030 con 0x7f031815c620 2023-08-09T13:33:43.510 INFO:teuthology.orchestra.run.smithi203.stderr:2023-08-09T13:33:43.506+0000 7f03167fc640 1 -- 172.21.15.203:0/3627178132 <== mon.2 v2:172.21.15.203:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0308004190 con 0x7f031815c620 2023-08-09T13:33:43.510 INFO:teuthology.orchestra.run.smithi203.stderr:2023-08-09T13:33:43.506+0000 7f03167fc640 1 -- 172.21.15.203:0/3627178132 <== mon.2 v2:172.21.15.203:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0308003610 con 0x7f031815c620 2023-08-09T13:33:43.511 INFO:teuthology.orchestra.run.smithi203.stderr:2023-08-09T13:33:43.507+0000 7f031d5b1640 1 -- 172.21.15.203:0/3627178132 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f031815c620 msgr2=0x7f03180cceb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:33:43.511 INFO:teuthology.orchestra.run.smithi203.stderr:2023-08-09T13:33:43.507+0000 7f031d5b1640 1 --2- 172.21.15.203:0/3627178132 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f031815c620 0x7f03180cceb0 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7f0308009850 tx=0x7f030802b7c0 comp rx=0 tx=0).stop 2023-08-09T13:33:43.511 INFO:teuthology.orchestra.run.smithi203.stderr:2023-08-09T13:33:43.507+0000 7f031d5b1640 1 -- 172.21.15.203:0/3627178132 shutdown_connections 2023-08-09T13:33:43.511 INFO:teuthology.orchestra.run.smithi203.stderr:2023-08-09T13:33:43.507+0000 7f031d5b1640 1 --2- 172.21.15.203:0/3627178132 >> v2:172.21.15.203:3300/0 conn(0x7f03180b1260 0x7f03180cee00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:33:43.511 INFO:teuthology.orchestra.run.smithi203.stderr:2023-08-09T13:33:43.507+0000 7f031d5b1640 1 --2- 172.21.15.203:0/3627178132 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f031815c620 0x7f03180cceb0 unknown :-1 s=CLOSED pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:33:43.511 INFO:teuthology.orchestra.run.smithi203.stderr:2023-08-09T13:33:43.507+0000 7f031d5b1640 1 -- 172.21.15.203:0/3627178132 >> 172.21.15.203:0/3627178132 conn(0x7f031809b750 msgr2=0x7f03180d2410 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-09T13:33:43.511 INFO:teuthology.orchestra.run.smithi203.stderr:2023-08-09T13:33:43.507+0000 7f031d5b1640 1 -- 172.21.15.203:0/3627178132 shutdown_connections 2023-08-09T13:33:43.512 INFO:teuthology.orchestra.run.smithi203.stderr:2023-08-09T13:33:43.507+0000 7f031d5b1640 1 -- 172.21.15.203:0/3627178132 wait complete. 2023-08-09T13:33:43.514 INFO:teuthology.orchestra.run.smithi203.stderr:2023-08-09T13:33:43.509+0000 7f031d5b1640 1 Processor -- start 2023-08-09T13:33:43.515 INFO:teuthology.orchestra.run.smithi203.stderr:2023-08-09T13:33:43.509+0000 7f031d5b1640 1 -- start start 2023-08-09T13:33:43.515 INFO:teuthology.orchestra.run.smithi203.stderr:2023-08-09T13:33:43.510+0000 7f031d5b1640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f03180b1260 0x7f031814d7b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:33:43.515 INFO:teuthology.orchestra.run.smithi203.stderr:2023-08-09T13:33:43.510+0000 7f031d5b1640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f03180cf340 0x7f031809fa60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:33:43.516 INFO:teuthology.orchestra.run.smithi203.stderr:2023-08-09T13:33:43.510+0000 7f031d5b1640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f031815c620 0x7f03180b7300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:33:43.516 INFO:teuthology.orchestra.run.smithi203.stderr:2023-08-09T13:33:43.510+0000 7f031d5b1640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f03180cc620 con 0x7f031815c620 2023-08-09T13:33:43.516 INFO:teuthology.orchestra.run.smithi203.stderr:2023-08-09T13:33:43.510+0000 7f031d5b1640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f031814dcf0 con 0x7f03180b1260 2023-08-09T13:33:43.517 INFO:teuthology.orchestra.run.smithi203.stderr:2023-08-09T13:33:43.510+0000 7f031d5b1640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f03180cb6c0 con 0x7f03180cf340 2023-08-09T13:33:43.517 INFO:teuthology.orchestra.run.smithi203.stderr:2023-08-09T13:33:43.510+0000 7f0316ffd640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f03180b1260 0x7f031814d7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:33:43.517 INFO:teuthology.orchestra.run.smithi203.stderr:2023-08-09T13:33:43.510+0000 7f0316ffd640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f03180b1260 0x7f031814d7b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.203:3300/0 says I am v2:172.21.15.203:48892/0 (socket says 172.21.15.203:48892) 2023-08-09T13:33:43.518 INFO:teuthology.orchestra.run.smithi203.stderr:2023-08-09T13:33:43.510+0000 7f0316ffd640 1 -- 172.21.15.203:0/2241535240 learned_addr learned my addr 172.21.15.203:0/2241535240 (peer_addr_for_me v2:172.21.15.203:0/0) 2023-08-09T13:33:43.518 INFO:teuthology.orchestra.run.smithi203.stderr:2023-08-09T13:33:43.510+0000 7f030ffff640 1 --2- 172.21.15.203:0/2241535240 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f03180cf340 0x7f031809fa60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:33:43.518 INFO:teuthology.orchestra.run.smithi203.stderr:2023-08-09T13:33:43.510+0000 7f03177fe640 1 --2- 172.21.15.203:0/2241535240 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f031815c620 0x7f03180b7300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:33:43.518 INFO:teuthology.orchestra.run.smithi203.stderr:2023-08-09T13:33:43.510+0000 7f0316ffd640 1 -- 172.21.15.203:0/2241535240 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f03180cf340 msgr2=0x7f031809fa60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:33:43.519 INFO:teuthology.orchestra.run.smithi203.stderr:2023-08-09T13:33:43.510+0000 7f0316ffd640 1 --2- 172.21.15.203:0/2241535240 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f03180cf340 0x7f031809fa60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:33:43.519 INFO:teuthology.orchestra.run.smithi203.stderr:2023-08-09T13:33:43.511+0000 7f0316ffd640 1 -- 172.21.15.203:0/2241535240 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f031815c620 msgr2=0x7f03180b7300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:33:43.519 INFO:teuthology.orchestra.run.smithi203.stderr:2023-08-09T13:33:43.511+0000 7f0316ffd640 1 --2- 172.21.15.203:0/2241535240 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f031815c620 0x7f03180b7300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:33:43.520 INFO:teuthology.orchestra.run.smithi203.stderr:2023-08-09T13:33:43.511+0000 7f0316ffd640 1 -- 172.21.15.203:0/2241535240 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0308009460 con 0x7f03180b1260 2023-08-09T13:33:43.520 INFO:teuthology.orchestra.run.smithi203.stderr:2023-08-09T13:33:43.511+0000 7f03177fe640 1 --2- 172.21.15.203:0/2241535240 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f031815c620 0x7f03180b7300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-08-09T13:33:43.520 INFO:teuthology.orchestra.run.smithi203.stderr:2023-08-09T13:33:43.511+0000 7f0316ffd640 1 --2- 172.21.15.203:0/2241535240 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f03180b1260 0x7f031814d7b0 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7f0308036040 tx=0x7f03080046c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-09T13:33:43.520 INFO:teuthology.orchestra.run.smithi203.stderr:2023-08-09T13:33:43.511+0000 7f030f7fe640 1 -- 172.21.15.203:0/2241535240 <== mon.1 v2:172.21.15.203:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0308003d50 con 0x7f03180b1260 2023-08-09T13:33:43.520 INFO:teuthology.orchestra.run.smithi203.stderr:2023-08-09T13:33:43.511+0000 7f030f7fe640 1 -- 172.21.15.203:0/2241535240 <== mon.1 v2:172.21.15.203:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0308005e30 con 0x7f03180b1260 2023-08-09T13:33:43.521 INFO:teuthology.orchestra.run.smithi203.stderr:2023-08-09T13:33:43.511+0000 7f030f7fe640 1 -- 172.21.15.203:0/2241535240 <== mon.1 v2:172.21.15.203:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0308006280 con 0x7f03180b1260 2023-08-09T13:33:43.521 INFO:teuthology.orchestra.run.smithi203.stderr:2023-08-09T13:33:43.511+0000 7f031d5b1640 1 -- 172.21.15.203:0/2241535240 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f03180cb9a0 con 0x7f03180b1260 2023-08-09T13:33:43.521 INFO:teuthology.orchestra.run.smithi203.stderr:2023-08-09T13:33:43.511+0000 7f031d5b1640 1 -- 172.21.15.203:0/2241535240 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f03180cbf60 con 0x7f03180b1260 2023-08-09T13:33:43.521 INFO:teuthology.orchestra.run.smithi203.stderr:2023-08-09T13:33:43.512+0000 7f030f7fe640 1 -- 172.21.15.203:0/2241535240 <== mon.1 v2:172.21.15.203:3300/0 4 ==== mgrmap(e 1) v1 ==== 660+0+0 (secure 0 0 0) 0x7f03080066e0 con 0x7f03180b1260 2023-08-09T13:33:43.522 INFO:teuthology.orchestra.run.smithi203.stderr:2023-08-09T13:33:43.512+0000 7f030f7fe640 1 -- 172.21.15.203:0/2241535240 <== mon.1 v2:172.21.15.203:3300/0 5 ==== osd_map(4..4 src has 1..4) v4 ==== 955+0+0 (secure 0 0 0) 0x7f0308031bb0 con 0x7f03180b1260 2023-08-09T13:33:43.522 INFO:teuthology.orchestra.run.smithi203.stderr:2023-08-09T13:33:43.513+0000 7f031d5b1640 1 -- 172.21.15.203:0/2241535240 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f03180739f0 con 0x7f03180b1260 2023-08-09T13:33:43.523 INFO:teuthology.orchestra.run.smithi203.stderr:2023-08-09T13:33:43.515+0000 7f030f7fe640 1 -- 172.21.15.203:0/2241535240 <== mon.1 v2:172.21.15.203:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+72429 (secure 0 0 0) 0x7f0308066b00 con 0x7f03180b1260 2023-08-09T13:33:43.590 INFO:teuthology.orchestra.run.smithi203.stderr:2023-08-09T13:33:43.588+0000 7f031d5b1640 1 -- 172.21.15.203:0/2241535240 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_command({"prefix": "osd new", "uuid": "bb5c41e9-7f5f-4125-9b01-3bb8bfc2ea83", "id": 2} v 0) v1 -- 0x7f03180cfda0 con 0x7f03180b1260 2023-08-09T13:33:43.597 INFO:teuthology.orchestra.run.smithi203.stderr:2023-08-09T13:33:43.595+0000 7f030f7fe640 1 -- 172.21.15.203:0/2241535240 <== mon.1 v2:172.21.15.203:3300/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "bb5c41e9-7f5f-4125-9b01-3bb8bfc2ea83", "id": 2}]=0 v5) v1 ==== 112+0+2 (secure 0 0 0) 0x7f0308055030 con 0x7f03180b1260 2023-08-09T13:33:43.598 INFO:teuthology.orchestra.run.smithi203.stdout:2 2023-08-09T13:33:43.599 INFO:teuthology.orchestra.run.smithi203.stderr:2023-08-09T13:33:43.596+0000 7f031d5b1640 1 -- 172.21.15.203:0/2241535240 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f03180b1260 msgr2=0x7f031814d7b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:33:43.599 INFO:teuthology.orchestra.run.smithi203.stderr:2023-08-09T13:33:43.596+0000 7f031d5b1640 1 --2- 172.21.15.203:0/2241535240 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f03180b1260 0x7f031814d7b0 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7f0308036040 tx=0x7f03080046c0 comp rx=0 tx=0).stop 2023-08-09T13:33:43.599 INFO:teuthology.orchestra.run.smithi203.stderr:2023-08-09T13:33:43.597+0000 7f031d5b1640 1 -- 172.21.15.203:0/2241535240 shutdown_connections 2023-08-09T13:33:43.599 INFO:teuthology.orchestra.run.smithi203.stderr:2023-08-09T13:33:43.597+0000 7f031d5b1640 1 --2- 172.21.15.203:0/2241535240 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f031815c620 0x7f03180b7300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:33:43.599 INFO:teuthology.orchestra.run.smithi203.stderr:2023-08-09T13:33:43.597+0000 7f031d5b1640 1 --2- 172.21.15.203:0/2241535240 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f03180cf340 0x7f031809fa60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:33:43.600 INFO:teuthology.orchestra.run.smithi203.stderr:2023-08-09T13:33:43.597+0000 7f031d5b1640 1 --2- 172.21.15.203:0/2241535240 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f03180b1260 0x7f031814d7b0 unknown :-1 s=CLOSED pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:33:43.600 INFO:teuthology.orchestra.run.smithi203.stderr:2023-08-09T13:33:43.597+0000 7f031d5b1640 1 -- 172.21.15.203:0/2241535240 >> 172.21.15.203:0/2241535240 conn(0x7f031809b750 msgr2=0x7f03180af040 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-09T13:33:43.600 INFO:teuthology.orchestra.run.smithi203.stderr:2023-08-09T13:33:43.597+0000 7f031d5b1640 1 -- 172.21.15.203:0/2241535240 shutdown_connections 2023-08-09T13:33:43.600 INFO:teuthology.orchestra.run.smithi203.stderr:2023-08-09T13:33:43.597+0000 7f031d5b1640 1 -- 172.21.15.203:0/2241535240 wait complete. 2023-08-09T13:33:43.615 DEBUG:teuthology.orchestra.run.smithi203:> sudo ceph --cluster ceph osd new bfe4c8b4-694a-465c-87e6-2b2aab852613 3 2023-08-09T13:33:43.850 INFO:teuthology.orchestra.run.smithi203.stderr:2023-08-09T13:33:43.848+0000 7f7fa7821640 1 Processor -- start 2023-08-09T13:33:43.850 INFO:teuthology.orchestra.run.smithi203.stderr:2023-08-09T13:33:43.848+0000 7f7fa7821640 1 -- start start 2023-08-09T13:33:43.851 INFO:teuthology.orchestra.run.smithi203.stderr:2023-08-09T13:33:43.849+0000 7f7fa7821640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f7fa015c620 0x7f7fa00cceb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:33:43.851 INFO:teuthology.orchestra.run.smithi203.stderr:2023-08-09T13:33:43.849+0000 7f7fa7821640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f7fa00cf510 0x7f7fa00d0350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:33:43.851 INFO:teuthology.orchestra.run.smithi203.stderr:2023-08-09T13:33:43.849+0000 7f7fa7821640 1 -- --> v1:172.21.15.43:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7fa00cd700 con 0x7f7fa00b1260 2023-08-09T13:33:43.852 INFO:teuthology.orchestra.run.smithi203.stderr:2023-08-09T13:33:43.849+0000 7f7fa7821640 1 -- --> v2:172.21.15.43:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7fa00b1b70 con 0x7f7fa00cf510 2023-08-09T13:33:43.852 INFO:teuthology.orchestra.run.smithi203.stderr:2023-08-09T13:33:43.849+0000 7f7fa7821640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7fa00cd9e0 con 0x7f7fa015c620 2023-08-09T13:33:43.852 INFO:teuthology.orchestra.run.smithi203.stderr:2023-08-09T13:33:43.849+0000 7f7fa55bd640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f7fa015c620 0x7f7fa00cceb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:33:43.853 INFO:teuthology.orchestra.run.smithi203.stderr:2023-08-09T13:33:43.849+0000 7f7fa55bd640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f7fa015c620 0x7f7fa00cceb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.203:3301/0 says I am v2:172.21.15.203:59878/0 (socket says 172.21.15.203:59878) 2023-08-09T13:33:43.853 INFO:teuthology.orchestra.run.smithi203.stderr:2023-08-09T13:33:43.849+0000 7f7fa55bd640 1 -- 172.21.15.203:0/2152341613 learned_addr learned my addr 172.21.15.203:0/2152341613 (peer_addr_for_me v2:172.21.15.203:0/0) 2023-08-09T13:33:43.853 INFO:teuthology.orchestra.run.smithi203.stderr:2023-08-09T13:33:43.850+0000 7f7fa5dbe640 1 --2- 172.21.15.203:0/2152341613 >> v2:172.21.15.43:3300/0 conn(0x7f7fa00cf510 0x7f7fa00d0350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:33:43.853 INFO:teuthology.orchestra.run.smithi203.stderr:2023-08-09T13:33:43.850+0000 7f7fa55bd640 1 -- 172.21.15.203:0/2152341613 >> v2:172.21.15.43:3300/0 conn(0x7f7fa00cf510 msgr2=0x7f7fa00d0350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-09T13:33:43.853 INFO:teuthology.orchestra.run.smithi203.stderr:2023-08-09T13:33:43.850+0000 7f7fa55bd640 1 --2- 172.21.15.203:0/2152341613 >> v2:172.21.15.43:3300/0 conn(0x7f7fa00cf510 0x7f7fa00d0350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:33:43.853 INFO:teuthology.orchestra.run.smithi203.stderr:2023-08-09T13:33:43.850+0000 7f7fa55bd640 1 -- 172.21.15.203:0/2152341613 >> v1:172.21.15.43:6789/0 conn(0x7f7fa00b1260 legacy=0x7f7fa00cee00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-09T13:33:43.854 INFO:teuthology.orchestra.run.smithi203.stderr:2023-08-09T13:33:43.850+0000 7f7fa55bd640 1 -- 172.21.15.203:0/2152341613 --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7fa00cdcc0 con 0x7f7fa015c620 2023-08-09T13:33:43.854 INFO:teuthology.orchestra.run.smithi203.stderr:2023-08-09T13:33:43.850+0000 7f7fa5dbe640 1 --2- 172.21.15.203:0/2152341613 >> v2:172.21.15.43:3300/0 conn(0x7f7fa00cf510 0x7f7fa00d0350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-08-09T13:33:43.855 INFO:teuthology.orchestra.run.smithi203.stderr:2023-08-09T13:33:43.851+0000 7f7fa55bd640 1 --2- 172.21.15.203:0/2152341613 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f7fa015c620 0x7f7fa00cceb0 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7f7f9c009860 tx=0x7f7f9c02b7e0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ddcada03ea368526 server_cookie=0 in_seq=0 out_seq=0 2023-08-09T13:33:43.855 INFO:teuthology.orchestra.run.smithi203.stderr:2023-08-09T13:33:43.851+0000 7f7f97fff640 1 -- 172.21.15.203:0/2152341613 <== mon.2 v2:172.21.15.203:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7f9c031740 con 0x7f7fa015c620 2023-08-09T13:33:43.855 INFO:teuthology.orchestra.run.smithi203.stderr:2023-08-09T13:33:43.851+0000 7f7f97fff640 1 -- 172.21.15.203:0/2152341613 <== mon.2 v2:172.21.15.203:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7f9c0318a0 con 0x7f7fa015c620 2023-08-09T13:33:43.855 INFO:teuthology.orchestra.run.smithi203.stderr:2023-08-09T13:33:43.851+0000 7f7f97fff640 1 -- 172.21.15.203:0/2152341613 <== mon.2 v2:172.21.15.203:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7f9c031c30 con 0x7f7fa015c620 2023-08-09T13:33:43.855 INFO:teuthology.orchestra.run.smithi203.stderr:2023-08-09T13:33:43.852+0000 7f7fa7821640 1 -- 172.21.15.203:0/2152341613 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f7fa015c620 msgr2=0x7f7fa00cceb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:33:43.855 INFO:teuthology.orchestra.run.smithi203.stderr:2023-08-09T13:33:43.852+0000 7f7fa7821640 1 --2- 172.21.15.203:0/2152341613 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f7fa015c620 0x7f7fa00cceb0 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7f7f9c009860 tx=0x7f7f9c02b7e0 comp rx=0 tx=0).stop 2023-08-09T13:33:43.856 INFO:teuthology.orchestra.run.smithi203.stderr:2023-08-09T13:33:43.852+0000 7f7fa7821640 1 -- 172.21.15.203:0/2152341613 shutdown_connections 2023-08-09T13:33:43.856 INFO:teuthology.orchestra.run.smithi203.stderr:2023-08-09T13:33:43.852+0000 7f7fa7821640 1 --2- 172.21.15.203:0/2152341613 >> v2:172.21.15.43:3300/0 conn(0x7f7fa00cf510 0x7f7fa00d0350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:33:43.856 INFO:teuthology.orchestra.run.smithi203.stderr:2023-08-09T13:33:43.852+0000 7f7fa7821640 1 --2- 172.21.15.203:0/2152341613 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f7fa015c620 0x7f7fa00cceb0 unknown :-1 s=CLOSED pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:33:43.856 INFO:teuthology.orchestra.run.smithi203.stderr:2023-08-09T13:33:43.852+0000 7f7fa7821640 1 -- 172.21.15.203:0/2152341613 >> 172.21.15.203:0/2152341613 conn(0x7f7fa009b750 msgr2=0x7f7fa00d2410 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-09T13:33:43.856 INFO:teuthology.orchestra.run.smithi203.stderr:2023-08-09T13:33:43.852+0000 7f7fa7821640 1 -- 172.21.15.203:0/2152341613 shutdown_connections 2023-08-09T13:33:43.856 INFO:teuthology.orchestra.run.smithi203.stderr:2023-08-09T13:33:43.852+0000 7f7fa7821640 1 -- 172.21.15.203:0/2152341613 wait complete. 2023-08-09T13:33:43.858 INFO:teuthology.orchestra.run.smithi203.stderr:2023-08-09T13:33:43.853+0000 7f7fa7821640 1 Processor -- start 2023-08-09T13:33:43.858 INFO:teuthology.orchestra.run.smithi203.stderr:2023-08-09T13:33:43.854+0000 7f7fa7821640 1 -- start start 2023-08-09T13:33:43.859 INFO:teuthology.orchestra.run.smithi203.stderr:2023-08-09T13:33:43.854+0000 7f7fa7821640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f7fa00b1260 0x7f7fa014a830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:33:43.859 INFO:teuthology.orchestra.run.smithi203.stderr:2023-08-09T13:33:43.854+0000 7f7fa7821640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f7fa00cf510 0x7f7fa009c0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:33:43.859 INFO:teuthology.orchestra.run.smithi203.stderr:2023-08-09T13:33:43.854+0000 7f7fa7821640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f7fa015c620 0x7f7fa00b7160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:33:43.859 INFO:teuthology.orchestra.run.smithi203.stderr:2023-08-09T13:33:43.854+0000 7f7fa7821640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7fa00b6cd0 con 0x7f7fa00b1260 2023-08-09T13:33:43.859 INFO:teuthology.orchestra.run.smithi203.stderr:2023-08-09T13:33:43.854+0000 7f7fa7821640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7fa014ad70 con 0x7f7fa015c620 2023-08-09T13:33:43.859 INFO:teuthology.orchestra.run.smithi203.stderr:2023-08-09T13:33:43.854+0000 7f7fa7821640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7fa00b60c0 con 0x7f7fa00cf510 2023-08-09T13:33:43.859 INFO:teuthology.orchestra.run.smithi203.stderr:2023-08-09T13:33:43.855+0000 7f7fa4dbc640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f7fa00cf510 0x7f7fa009c0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:33:43.859 INFO:teuthology.orchestra.run.smithi203.stderr:2023-08-09T13:33:43.855+0000 7f7fa5dbe640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f7fa015c620 0x7f7fa00b7160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:33:43.860 INFO:teuthology.orchestra.run.smithi203.stderr:2023-08-09T13:33:43.855+0000 7f7fa4dbc640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f7fa00cf510 0x7f7fa009c0c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.203:3301/0 says I am v2:172.21.15.203:59880/0 (socket says 172.21.15.203:59880) 2023-08-09T13:33:43.860 INFO:teuthology.orchestra.run.smithi203.stderr:2023-08-09T13:33:43.855+0000 7f7fa5dbe640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f7fa015c620 0x7f7fa00b7160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.203:3300/0 says I am v2:172.21.15.203:48900/0 (socket says 172.21.15.203:48900) 2023-08-09T13:33:43.860 INFO:teuthology.orchestra.run.smithi203.stderr:2023-08-09T13:33:43.855+0000 7f7fa4dbc640 1 -- 172.21.15.203:0/3501606709 learned_addr learned my addr 172.21.15.203:0/3501606709 (peer_addr_for_me v2:172.21.15.203:0/0) 2023-08-09T13:33:43.860 INFO:teuthology.orchestra.run.smithi203.stderr:2023-08-09T13:33:43.855+0000 7f7fa55bd640 1 --2- 172.21.15.203:0/3501606709 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f7fa00b1260 0x7f7fa014a830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:33:43.860 INFO:teuthology.orchestra.run.smithi203.stderr:2023-08-09T13:33:43.855+0000 7f7fa4dbc640 1 -- 172.21.15.203:0/3501606709 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f7fa015c620 msgr2=0x7f7fa00b7160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:33:43.860 INFO:teuthology.orchestra.run.smithi203.stderr:2023-08-09T13:33:43.855+0000 7f7fa4dbc640 1 --2- 172.21.15.203:0/3501606709 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f7fa015c620 0x7f7fa00b7160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:33:43.860 INFO:teuthology.orchestra.run.smithi203.stderr:2023-08-09T13:33:43.855+0000 7f7fa4dbc640 1 -- 172.21.15.203:0/3501606709 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f7fa00b1260 msgr2=0x7f7fa014a830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:33:43.861 INFO:teuthology.orchestra.run.smithi203.stderr:2023-08-09T13:33:43.855+0000 7f7fa4dbc640 1 --2- 172.21.15.203:0/3501606709 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f7fa00b1260 0x7f7fa014a830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:33:43.861 INFO:teuthology.orchestra.run.smithi203.stderr:2023-08-09T13:33:43.855+0000 7f7fa4dbc640 1 -- 172.21.15.203:0/3501606709 --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7f9c009470 con 0x7f7fa00cf510 2023-08-09T13:33:43.861 INFO:teuthology.orchestra.run.smithi203.stderr:2023-08-09T13:33:43.855+0000 7f7fa5dbe640 1 --2- 172.21.15.203:0/3501606709 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f7fa015c620 0x7f7fa00b7160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-08-09T13:33:43.861 INFO:teuthology.orchestra.run.smithi203.stderr:2023-08-09T13:33:43.855+0000 7f7fa55bd640 1 --2- 172.21.15.203:0/3501606709 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f7fa00b1260 0x7f7fa014a830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-08-09T13:33:43.861 INFO:teuthology.orchestra.run.smithi203.stderr:2023-08-09T13:33:43.855+0000 7f7fa4dbc640 1 --2- 172.21.15.203:0/3501606709 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f7fa00cf510 0x7f7fa009c0c0 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7f7f9000b410 tx=0x7f7f9000b8d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-09T13:33:43.861 INFO:teuthology.orchestra.run.smithi203.stderr:2023-08-09T13:33:43.856+0000 7f7f95ffb640 1 -- 172.21.15.203:0/3501606709 <== mon.2 v2:172.21.15.203:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7f90011830 con 0x7f7fa00cf510 2023-08-09T13:33:43.861 INFO:teuthology.orchestra.run.smithi203.stderr:2023-08-09T13:33:43.856+0000 7f7f95ffb640 1 -- 172.21.15.203:0/3501606709 <== mon.2 v2:172.21.15.203:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7f90011990 con 0x7f7fa00cf510 2023-08-09T13:33:43.862 INFO:teuthology.orchestra.run.smithi203.stderr:2023-08-09T13:33:43.856+0000 7f7fa7821640 1 -- 172.21.15.203:0/3501606709 --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7fa00b63a0 con 0x7f7fa00cf510 2023-08-09T13:33:43.862 INFO:teuthology.orchestra.run.smithi203.stderr:2023-08-09T13:33:43.856+0000 7f7f95ffb640 1 -- 172.21.15.203:0/3501606709 <== mon.2 v2:172.21.15.203:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7f90011d40 con 0x7f7fa00cf510 2023-08-09T13:33:43.862 INFO:teuthology.orchestra.run.smithi203.stderr:2023-08-09T13:33:43.856+0000 7f7fa7821640 1 -- 172.21.15.203:0/3501606709 --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7fa00b6960 con 0x7f7fa00cf510 2023-08-09T13:33:43.862 INFO:teuthology.orchestra.run.smithi203.stderr:2023-08-09T13:33:43.856+0000 7f7f95ffb640 1 -- 172.21.15.203:0/3501606709 <== mon.2 v2:172.21.15.203:3301/0 4 ==== mgrmap(e 1) v1 ==== 660+0+0 (secure 0 0 0) 0x7f7f90004320 con 0x7f7fa00cf510 2023-08-09T13:33:43.862 INFO:teuthology.orchestra.run.smithi203.stderr:2023-08-09T13:33:43.856+0000 7f7f95ffb640 1 -- 172.21.15.203:0/3501606709 <== mon.2 v2:172.21.15.203:3301/0 5 ==== osd_map(5..5 src has 1..5) v4 ==== 1070+0+0 (secure 0 0 0) 0x7f7f90003a80 con 0x7f7fa00cf510 2023-08-09T13:33:43.863 INFO:teuthology.orchestra.run.smithi203.stderr:2023-08-09T13:33:43.857+0000 7f7fa7821640 1 -- 172.21.15.203:0/3501606709 --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7fa00b6e80 con 0x7f7fa00cf510 2023-08-09T13:33:43.863 INFO:teuthology.orchestra.run.smithi203.stderr:2023-08-09T13:33:43.859+0000 7f7f95ffb640 1 -- 172.21.15.203:0/3501606709 <== mon.2 v2:172.21.15.203:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+72429 (secure 0 0 0) 0x7f7fa00b6e80 con 0x7f7fa00cf510 2023-08-09T13:33:43.939 INFO:teuthology.orchestra.run.smithi203.stderr:2023-08-09T13:33:43.937+0000 7f7fa7821640 1 -- 172.21.15.203:0/3501606709 --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_command({"prefix": "osd new", "uuid": "bfe4c8b4-694a-465c-87e6-2b2aab852613", "id": 3} v 0) v1 -- 0x7f7fa00739f0 con 0x7f7fa00cf510 2023-08-09T13:33:43.946 INFO:teuthology.orchestra.run.smithi203.stderr:2023-08-09T13:33:43.944+0000 7f7f95ffb640 1 -- 172.21.15.203:0/3501606709 <== mon.2 v2:172.21.15.203:3301/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "bfe4c8b4-694a-465c-87e6-2b2aab852613", "id": 3}]=0 v6) v1 ==== 112+0+2 (secure 0 0 0) 0x7f7f90015020 con 0x7f7fa00cf510 2023-08-09T13:33:43.947 INFO:teuthology.orchestra.run.smithi203.stdout:3 2023-08-09T13:33:43.948 INFO:teuthology.orchestra.run.smithi203.stderr:2023-08-09T13:33:43.945+0000 7f7fa7821640 1 -- 172.21.15.203:0/3501606709 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f7fa00cf510 msgr2=0x7f7fa009c0c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:33:43.948 INFO:teuthology.orchestra.run.smithi203.stderr:2023-08-09T13:33:43.945+0000 7f7fa7821640 1 --2- 172.21.15.203:0/3501606709 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f7fa00cf510 0x7f7fa009c0c0 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7f7f9000b410 tx=0x7f7f9000b8d0 comp rx=0 tx=0).stop 2023-08-09T13:33:43.948 INFO:teuthology.orchestra.run.smithi203.stderr:2023-08-09T13:33:43.946+0000 7f7fa7821640 1 -- 172.21.15.203:0/3501606709 shutdown_connections 2023-08-09T13:33:43.949 INFO:teuthology.orchestra.run.smithi203.stderr:2023-08-09T13:33:43.946+0000 7f7fa7821640 1 --2- 172.21.15.203:0/3501606709 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f7fa015c620 0x7f7fa00b7160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:33:43.949 INFO:teuthology.orchestra.run.smithi203.stderr:2023-08-09T13:33:43.946+0000 7f7fa7821640 1 --2- 172.21.15.203:0/3501606709 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f7fa00cf510 0x7f7fa009c0c0 unknown :-1 s=CLOSED pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:33:43.949 INFO:teuthology.orchestra.run.smithi203.stderr:2023-08-09T13:33:43.946+0000 7f7fa7821640 1 --2- 172.21.15.203:0/3501606709 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f7fa00b1260 0x7f7fa014a830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:33:43.949 INFO:teuthology.orchestra.run.smithi203.stderr:2023-08-09T13:33:43.946+0000 7f7fa7821640 1 -- 172.21.15.203:0/3501606709 >> 172.21.15.203:0/3501606709 conn(0x7f7fa009b750 msgr2=0x7f7fa00b2f10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-09T13:33:43.949 INFO:teuthology.orchestra.run.smithi203.stderr:2023-08-09T13:33:43.946+0000 7f7fa7821640 1 -- 172.21.15.203:0/3501606709 shutdown_connections 2023-08-09T13:33:43.949 INFO:teuthology.orchestra.run.smithi203.stderr:2023-08-09T13:33:43.946+0000 7f7fa7821640 1 -- 172.21.15.203:0/3501606709 wait complete. 2023-08-09T13:33:43.964 DEBUG:teuthology.orchestra.run.smithi203:> sudo ceph --cluster ceph osd new fa2e8a77-aba1-4756-853e-0d174a5bdeba 4 2023-08-09T13:33:44.153 INFO:teuthology.orchestra.run.smithi203.stderr:2023-08-09T13:33:44.151+0000 7f7cab59e640 1 Processor -- start 2023-08-09T13:33:44.154 INFO:teuthology.orchestra.run.smithi203.stderr:2023-08-09T13:33:44.151+0000 7f7cab59e640 1 -- start start 2023-08-09T13:33:44.154 INFO:teuthology.orchestra.run.smithi203.stderr:2023-08-09T13:33:44.152+0000 7f7cab59e640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f7cac15a1b0 0x7f7cac0b1ad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:33:44.155 INFO:teuthology.orchestra.run.smithi203.stderr:2023-08-09T13:33:44.152+0000 7f7cab59e640 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7f7cac15e9c0 0x7f7cac0b2010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:33:44.155 INFO:teuthology.orchestra.run.smithi203.stderr:2023-08-09T13:33:44.152+0000 7f7cab59e640 1 -- --> v1:172.21.15.203:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7cac0cc6c0 con 0x7f7cac0cd630 2023-08-09T13:33:44.155 INFO:teuthology.orchestra.run.smithi203.stderr:2023-08-09T13:33:44.152+0000 7f7cab59e640 1 -- --> v2:172.21.15.203:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7cac15dee0 con 0x7f7cac15e9c0 2023-08-09T13:33:44.155 INFO:teuthology.orchestra.run.smithi203.stderr:2023-08-09T13:33:44.152+0000 7f7cab59e640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7cac15e1c0 con 0x7f7cac15a1b0 2023-08-09T13:33:44.156 INFO:teuthology.orchestra.run.smithi203.stderr:2023-08-09T13:33:44.152+0000 7f7caa59c640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f7cac15a1b0 0x7f7cac0b1ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:33:44.156 INFO:teuthology.orchestra.run.smithi203.stderr:2023-08-09T13:33:44.153+0000 7f7caad9d640 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7f7cac15e9c0 0x7f7cac0b2010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:33:44.156 INFO:teuthology.orchestra.run.smithi203.stderr:2023-08-09T13:33:44.153+0000 7f7caa59c640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f7cac15a1b0 0x7f7cac0b1ad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.203:3301/0 says I am v2:172.21.15.203:59892/0 (socket says 172.21.15.203:59892) 2023-08-09T13:33:44.156 INFO:teuthology.orchestra.run.smithi203.stderr:2023-08-09T13:33:44.153+0000 7f7caa59c640 1 -- 172.21.15.203:0/2768051841 learned_addr learned my addr 172.21.15.203:0/2768051841 (peer_addr_for_me v2:172.21.15.203:0/0) 2023-08-09T13:33:44.156 INFO:teuthology.orchestra.run.smithi203.stderr:2023-08-09T13:33:44.153+0000 7f7caa59c640 1 -- 172.21.15.203:0/2768051841 >> v2:172.21.15.203:3300/0 conn(0x7f7cac15e9c0 msgr2=0x7f7cac0b2010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-09T13:33:44.156 INFO:teuthology.orchestra.run.smithi203.stderr:2023-08-09T13:33:44.153+0000 7f7caa59c640 1 --2- 172.21.15.203:0/2768051841 >> v2:172.21.15.203:3300/0 conn(0x7f7cac15e9c0 0x7f7cac0b2010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:33:44.157 INFO:teuthology.orchestra.run.smithi203.stderr:2023-08-09T13:33:44.153+0000 7f7caa59c640 1 -- 172.21.15.203:0/2768051841 >> v1:172.21.15.203:6789/0 conn(0x7f7cac0cd630 legacy=0x7f7cac0cdc20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:33:44.157 INFO:teuthology.orchestra.run.smithi203.stderr:2023-08-09T13:33:44.153+0000 7f7caa59c640 1 -- 172.21.15.203:0/2768051841 --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7cac15e4a0 con 0x7f7cac15a1b0 2023-08-09T13:33:44.157 INFO:teuthology.orchestra.run.smithi203.stderr:2023-08-09T13:33:44.153+0000 7f7caad9d640 1 --2- 172.21.15.203:0/2768051841 >> v2:172.21.15.203:3300/0 conn(0x7f7cac15e9c0 0x7f7cac0b2010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-08-09T13:33:44.158 INFO:teuthology.orchestra.run.smithi203.stderr:2023-08-09T13:33:44.154+0000 7f7caa59c640 1 --2- 172.21.15.203:0/2768051841 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f7cac15a1b0 0x7f7cac0b1ad0 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7f7c9c009870 tx=0x7f7c9c02b980 comp rx=0 tx=0).ready entity=mon.2 client_cookie=2b44769e8a7e14ff server_cookie=0 in_seq=0 out_seq=0 2023-08-09T13:33:44.158 INFO:teuthology.orchestra.run.smithi203.stderr:2023-08-09T13:33:44.154+0000 7f7ca959a640 1 -- 172.21.15.203:0/2768051841 <== mon.2 v2:172.21.15.203:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7c9c030840 con 0x7f7cac15a1b0 2023-08-09T13:33:44.158 INFO:teuthology.orchestra.run.smithi203.stderr:2023-08-09T13:33:44.154+0000 7f7ca959a640 1 -- 172.21.15.203:0/2768051841 <== mon.2 v2:172.21.15.203:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7c9c0309a0 con 0x7f7cac15a1b0 2023-08-09T13:33:44.158 INFO:teuthology.orchestra.run.smithi203.stderr:2023-08-09T13:33:44.154+0000 7f7ca959a640 1 -- 172.21.15.203:0/2768051841 <== mon.2 v2:172.21.15.203:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7c9c030d30 con 0x7f7cac15a1b0 2023-08-09T13:33:44.159 INFO:teuthology.orchestra.run.smithi203.stderr:2023-08-09T13:33:44.155+0000 7f7cab59e640 1 -- 172.21.15.203:0/2768051841 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f7cac15a1b0 msgr2=0x7f7cac0b1ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:33:44.159 INFO:teuthology.orchestra.run.smithi203.stderr:2023-08-09T13:33:44.155+0000 7f7cab59e640 1 --2- 172.21.15.203:0/2768051841 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f7cac15a1b0 0x7f7cac0b1ad0 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7f7c9c009870 tx=0x7f7c9c02b980 comp rx=0 tx=0).stop 2023-08-09T13:33:44.159 INFO:teuthology.orchestra.run.smithi203.stderr:2023-08-09T13:33:44.155+0000 7f7cab59e640 1 -- 172.21.15.203:0/2768051841 shutdown_connections 2023-08-09T13:33:44.159 INFO:teuthology.orchestra.run.smithi203.stderr:2023-08-09T13:33:44.155+0000 7f7cab59e640 1 --2- 172.21.15.203:0/2768051841 >> v2:172.21.15.203:3300/0 conn(0x7f7cac15e9c0 0x7f7cac0b2010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:33:44.159 INFO:teuthology.orchestra.run.smithi203.stderr:2023-08-09T13:33:44.155+0000 7f7cab59e640 1 --2- 172.21.15.203:0/2768051841 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f7cac15a1b0 0x7f7cac0b1ad0 unknown :-1 s=CLOSED pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:33:44.159 INFO:teuthology.orchestra.run.smithi203.stderr:2023-08-09T13:33:44.155+0000 7f7cab59e640 1 -- 172.21.15.203:0/2768051841 >> 172.21.15.203:0/2768051841 conn(0x7f7cac0ccad0 msgr2=0x7f7cac0cceb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-09T13:33:44.159 INFO:teuthology.orchestra.run.smithi203.stderr:2023-08-09T13:33:44.155+0000 7f7cab59e640 1 -- 172.21.15.203:0/2768051841 shutdown_connections 2023-08-09T13:33:44.159 INFO:teuthology.orchestra.run.smithi203.stderr:2023-08-09T13:33:44.155+0000 7f7cab59e640 1 -- 172.21.15.203:0/2768051841 wait complete. 2023-08-09T13:33:44.161 INFO:teuthology.orchestra.run.smithi203.stderr:2023-08-09T13:33:44.157+0000 7f7cab59e640 1 Processor -- start 2023-08-09T13:33:44.161 INFO:teuthology.orchestra.run.smithi203.stderr:2023-08-09T13:33:44.157+0000 7f7cab59e640 1 -- start start 2023-08-09T13:33:44.161 INFO:teuthology.orchestra.run.smithi203.stderr:2023-08-09T13:33:44.158+0000 7f7cab59e640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f7cac0cd630 0x7f7cac14a850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:33:44.161 INFO:teuthology.orchestra.run.smithi203.stderr:2023-08-09T13:33:44.158+0000 7f7cab59e640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f7cac15a1b0 0x7f7cac1f5e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:33:44.161 INFO:teuthology.orchestra.run.smithi203.stderr:2023-08-09T13:33:44.158+0000 7f7cab59e640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f7cac15e9c0 0x7f7cac15c980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:33:44.161 INFO:teuthology.orchestra.run.smithi203.stderr:2023-08-09T13:33:44.158+0000 7f7cab59e640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7cac0c4c90 con 0x7f7cac15a1b0 2023-08-09T13:33:44.162 INFO:teuthology.orchestra.run.smithi203.stderr:2023-08-09T13:33:44.158+0000 7f7cab59e640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7cac06a8d0 con 0x7f7cac0cd630 2023-08-09T13:33:44.162 INFO:teuthology.orchestra.run.smithi203.stderr:2023-08-09T13:33:44.158+0000 7f7cab59e640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7cac1f6380 con 0x7f7cac15e9c0 2023-08-09T13:33:44.162 INFO:teuthology.orchestra.run.smithi203.stderr:2023-08-09T13:33:44.158+0000 7f7caa59c640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f7cac0cd630 0x7f7cac14a850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:33:44.162 INFO:teuthology.orchestra.run.smithi203.stderr:2023-08-09T13:33:44.158+0000 7f7caad9d640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f7cac15e9c0 0x7f7cac15c980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:33:44.162 INFO:teuthology.orchestra.run.smithi203.stderr:2023-08-09T13:33:44.158+0000 7f7caa59c640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f7cac0cd630 0x7f7cac14a850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.203:3300/0 says I am v2:172.21.15.203:48916/0 (socket says 172.21.15.203:48916) 2023-08-09T13:33:44.162 INFO:teuthology.orchestra.run.smithi203.stderr:2023-08-09T13:33:44.158+0000 7f7caad9d640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f7cac15e9c0 0x7f7cac15c980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.203:3301/0 says I am v2:172.21.15.203:59904/0 (socket says 172.21.15.203:59904) 2023-08-09T13:33:44.162 INFO:teuthology.orchestra.run.smithi203.stderr:2023-08-09T13:33:44.158+0000 7f7caa59c640 1 -- 172.21.15.203:0/1369510360 learned_addr learned my addr 172.21.15.203:0/1369510360 (peer_addr_for_me v2:172.21.15.203:0/0) 2023-08-09T13:33:44.164 INFO:teuthology.orchestra.run.smithi203.stderr:2023-08-09T13:33:44.159+0000 7f7ca9d9b640 1 --2- 172.21.15.203:0/1369510360 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f7cac15a1b0 0x7f7cac1f5e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:33:44.164 INFO:teuthology.orchestra.run.smithi203.stderr:2023-08-09T13:33:44.159+0000 7f7caad9d640 1 -- 172.21.15.203:0/1369510360 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f7cac0cd630 msgr2=0x7f7cac14a850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:33:44.164 INFO:teuthology.orchestra.run.smithi203.stderr:2023-08-09T13:33:44.159+0000 7f7caad9d640 1 --2- 172.21.15.203:0/1369510360 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f7cac0cd630 0x7f7cac14a850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:33:44.164 INFO:teuthology.orchestra.run.smithi203.stderr:2023-08-09T13:33:44.159+0000 7f7caad9d640 1 -- 172.21.15.203:0/1369510360 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f7cac15a1b0 msgr2=0x7f7cac1f5e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:33:44.164 INFO:teuthology.orchestra.run.smithi203.stderr:2023-08-09T13:33:44.159+0000 7f7caad9d640 1 --2- 172.21.15.203:0/1369510360 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f7cac15a1b0 0x7f7cac1f5e40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:33:44.164 INFO:teuthology.orchestra.run.smithi203.stderr:2023-08-09T13:33:44.159+0000 7f7caad9d640 1 -- 172.21.15.203:0/1369510360 --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7ca4009a70 con 0x7f7cac15e9c0 2023-08-09T13:33:44.165 INFO:teuthology.orchestra.run.smithi203.stderr:2023-08-09T13:33:44.159+0000 7f7caad9d640 1 --2- 172.21.15.203:0/1369510360 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f7cac15e9c0 0x7f7cac15c980 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7f7ca400ef90 tx=0x7f7ca400c490 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-09T13:33:44.165 INFO:teuthology.orchestra.run.smithi203.stderr:2023-08-09T13:33:44.159+0000 7f7c9affd640 1 -- 172.21.15.203:0/1369510360 <== mon.2 v2:172.21.15.203:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7ca400ccd0 con 0x7f7cac15e9c0 2023-08-09T13:33:44.165 INFO:teuthology.orchestra.run.smithi203.stderr:2023-08-09T13:33:44.159+0000 7f7c9affd640 1 -- 172.21.15.203:0/1369510360 <== mon.2 v2:172.21.15.203:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7ca400f040 con 0x7f7cac15e9c0 2023-08-09T13:33:44.165 INFO:teuthology.orchestra.run.smithi203.stderr:2023-08-09T13:33:44.159+0000 7f7c9affd640 1 -- 172.21.15.203:0/1369510360 <== mon.2 v2:172.21.15.203:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7ca4003bd0 con 0x7f7cac15e9c0 2023-08-09T13:33:44.165 INFO:teuthology.orchestra.run.smithi203.stderr:2023-08-09T13:33:44.159+0000 7f7cab59e640 1 -- 172.21.15.203:0/1369510360 --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7c9c0094d0 con 0x7f7cac15e9c0 2023-08-09T13:33:44.165 INFO:teuthology.orchestra.run.smithi203.stderr:2023-08-09T13:33:44.160+0000 7f7cab59e640 1 -- 172.21.15.203:0/1369510360 --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7cac06b130 con 0x7f7cac15e9c0 2023-08-09T13:33:44.165 INFO:teuthology.orchestra.run.smithi203.stderr:2023-08-09T13:33:44.160+0000 7f7c9affd640 1 -- 172.21.15.203:0/1369510360 <== mon.2 v2:172.21.15.203:3301/0 4 ==== mgrmap(e 1) v1 ==== 660+0+0 (secure 0 0 0) 0x7f7ca4004320 con 0x7f7cac15e9c0 2023-08-09T13:33:44.165 INFO:teuthology.orchestra.run.smithi203.stderr:2023-08-09T13:33:44.160+0000 7f7c9affd640 1 -- 172.21.15.203:0/1369510360 <== mon.2 v2:172.21.15.203:3301/0 5 ==== osd_map(6..6 src has 1..6) v4 ==== 1185+0+0 (secure 0 0 0) 0x7f7ca4014dd0 con 0x7f7cac15e9c0 2023-08-09T13:33:44.166 INFO:teuthology.orchestra.run.smithi203.stderr:2023-08-09T13:33:44.161+0000 7f7cab59e640 1 -- 172.21.15.203:0/1369510360 --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7cac06b2c0 con 0x7f7cac15e9c0 2023-08-09T13:33:44.166 INFO:teuthology.orchestra.run.smithi203.stderr:2023-08-09T13:33:44.163+0000 7f7c9affd640 1 -- 172.21.15.203:0/1369510360 <== mon.2 v2:172.21.15.203:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+72429 (secure 0 0 0) 0x7f7ca4018030 con 0x7f7cac15e9c0 2023-08-09T13:33:44.239 INFO:teuthology.orchestra.run.smithi203.stderr:2023-08-09T13:33:44.236+0000 7f7cab59e640 1 -- 172.21.15.203:0/1369510360 --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_command({"prefix": "osd new", "uuid": "fa2e8a77-aba1-4756-853e-0d174a5bdeba", "id": 4} v 0) v1 -- 0x7f7cac04e390 con 0x7f7cac15e9c0 2023-08-09T13:33:44.245 INFO:teuthology.orchestra.run.smithi203.stderr:2023-08-09T13:33:44.243+0000 7f7c9affd640 1 -- 172.21.15.203:0/1369510360 <== mon.2 v2:172.21.15.203:3301/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "fa2e8a77-aba1-4756-853e-0d174a5bdeba", "id": 4}]=0 v7) v1 ==== 112+0+2 (secure 0 0 0) 0x7f7ca400ce50 con 0x7f7cac15e9c0 2023-08-09T13:33:44.246 INFO:teuthology.orchestra.run.smithi203.stdout:4 2023-08-09T13:33:44.247 INFO:teuthology.orchestra.run.smithi203.stderr:2023-08-09T13:33:44.245+0000 7f7cab59e640 1 -- 172.21.15.203:0/1369510360 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f7cac15e9c0 msgr2=0x7f7cac15c980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:33:44.247 INFO:teuthology.orchestra.run.smithi203.stderr:2023-08-09T13:33:44.245+0000 7f7cab59e640 1 --2- 172.21.15.203:0/1369510360 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f7cac15e9c0 0x7f7cac15c980 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7f7ca400ef90 tx=0x7f7ca400c490 comp rx=0 tx=0).stop 2023-08-09T13:33:44.247 INFO:teuthology.orchestra.run.smithi203.stderr:2023-08-09T13:33:44.245+0000 7f7cab59e640 1 -- 172.21.15.203:0/1369510360 shutdown_connections 2023-08-09T13:33:44.248 INFO:teuthology.orchestra.run.smithi203.stderr:2023-08-09T13:33:44.245+0000 7f7cab59e640 1 --2- 172.21.15.203:0/1369510360 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f7cac15e9c0 0x7f7cac15c980 unknown :-1 s=CLOSED pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:33:44.248 INFO:teuthology.orchestra.run.smithi203.stderr:2023-08-09T13:33:44.245+0000 7f7cab59e640 1 --2- 172.21.15.203:0/1369510360 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f7cac15a1b0 0x7f7cac1f5e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:33:44.248 INFO:teuthology.orchestra.run.smithi203.stderr:2023-08-09T13:33:44.245+0000 7f7cab59e640 1 --2- 172.21.15.203:0/1369510360 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f7cac0cd630 0x7f7cac14a850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:33:44.248 INFO:teuthology.orchestra.run.smithi203.stderr:2023-08-09T13:33:44.245+0000 7f7cab59e640 1 -- 172.21.15.203:0/1369510360 >> 172.21.15.203:0/1369510360 conn(0x7f7cac0ccad0 msgr2=0x7f7cac0d0030 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-09T13:33:44.248 INFO:teuthology.orchestra.run.smithi203.stderr:2023-08-09T13:33:44.245+0000 7f7cab59e640 1 -- 172.21.15.203:0/1369510360 shutdown_connections 2023-08-09T13:33:44.248 INFO:teuthology.orchestra.run.smithi203.stderr:2023-08-09T13:33:44.245+0000 7f7cab59e640 1 -- 172.21.15.203:0/1369510360 wait complete. 2023-08-09T13:33:44.263 DEBUG:teuthology.orchestra.run.smithi203:> sudo ceph --cluster ceph osd new 81811e4b-4d24-4f51-a82c-f1d700a2a00d 5 2023-08-09T13:33:44.332 INFO:tasks.ceph.mgr.y.smithi043.stderr:2023-08-09T13:33:44.331+0000 7fe23d40f040 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2023-08-09T13:33:44.337 INFO:tasks.ceph.mgr.x.smithi203.stderr:2023-08-09T13:33:44.335+0000 7f943a4b2040 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2023-08-09T13:33:44.398 INFO:tasks.ceph.mgr.y.smithi043.stderr:2023-08-09T13:33:44.396+0000 7fe23d40f040 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2023-08-09T13:33:44.402 INFO:tasks.ceph.mgr.x.smithi203.stderr:2023-08-09T13:33:44.400+0000 7f943a4b2040 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2023-08-09T13:33:44.445 INFO:teuthology.orchestra.run.smithi203.stderr:2023-08-09T13:33:44.442+0000 7ff492cec640 1 Processor -- start 2023-08-09T13:33:44.445 INFO:teuthology.orchestra.run.smithi203.stderr:2023-08-09T13:33:44.443+0000 7ff492cec640 1 -- start start 2023-08-09T13:33:44.446 INFO:teuthology.orchestra.run.smithi203.stderr:2023-08-09T13:33:44.444+0000 7ff492cec640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7ff48c15c550 0x7ff48c0cd6d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:33:44.446 INFO:teuthology.orchestra.run.smithi203.stderr:2023-08-09T13:33:44.444+0000 7ff492cec640 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7ff48c0b45f0 0x7ff48c0b59d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:33:44.446 INFO:teuthology.orchestra.run.smithi203.stderr:2023-08-09T13:33:44.444+0000 7ff492cec640 1 -- --> v1:172.21.15.203:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff48c0b7bd0 con 0x7ff48c0d04b0 2023-08-09T13:33:44.446 INFO:teuthology.orchestra.run.smithi203.stderr:2023-08-09T13:33:44.444+0000 7ff492cec640 1 -- --> v2:172.21.15.203:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff48c0d1c00 con 0x7ff48c0b45f0 2023-08-09T13:33:44.447 INFO:teuthology.orchestra.run.smithi203.stderr:2023-08-09T13:33:44.444+0000 7ff492cec640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff48c15c930 con 0x7ff48c15c550 2023-08-09T13:33:44.447 INFO:teuthology.orchestra.run.smithi203.stderr:2023-08-09T13:33:44.444+0000 7ff490a88640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7ff48c15c550 0x7ff48c0cd6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:33:44.448 INFO:teuthology.orchestra.run.smithi203.stderr:2023-08-09T13:33:44.444+0000 7ff490a88640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7ff48c15c550 0x7ff48c0cd6d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.203:3301/0 says I am v2:172.21.15.203:59920/0 (socket says 172.21.15.203:59920) 2023-08-09T13:33:44.448 INFO:teuthology.orchestra.run.smithi203.stderr:2023-08-09T13:33:44.444+0000 7ff490a88640 1 -- 172.21.15.203:0/2725086363 learned_addr learned my addr 172.21.15.203:0/2725086363 (peer_addr_for_me v2:172.21.15.203:0/0) 2023-08-09T13:33:44.448 INFO:teuthology.orchestra.run.smithi203.stderr:2023-08-09T13:33:44.444+0000 7ff491289640 1 --2- 172.21.15.203:0/2725086363 >> v2:172.21.15.203:3300/0 conn(0x7ff48c0b45f0 0x7ff48c0b59d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:33:44.448 INFO:teuthology.orchestra.run.smithi203.stderr:2023-08-09T13:33:44.445+0000 7ff490a88640 1 -- 172.21.15.203:0/2725086363 >> v2:172.21.15.203:3300/0 conn(0x7ff48c0b45f0 msgr2=0x7ff48c0b59d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-09T13:33:44.448 INFO:teuthology.orchestra.run.smithi203.stderr:2023-08-09T13:33:44.445+0000 7ff490a88640 1 --2- 172.21.15.203:0/2725086363 >> v2:172.21.15.203:3300/0 conn(0x7ff48c0b45f0 0x7ff48c0b59d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:33:44.448 INFO:teuthology.orchestra.run.smithi203.stderr:2023-08-09T13:33:44.445+0000 7ff490a88640 1 -- 172.21.15.203:0/2725086363 >> v1:172.21.15.203:6789/0 conn(0x7ff48c0d04b0 legacy=0x7ff48c0b3ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:33:44.448 INFO:teuthology.orchestra.run.smithi203.stderr:2023-08-09T13:33:44.445+0000 7ff490a88640 1 -- 172.21.15.203:0/2725086363 --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff48c0b54f0 con 0x7ff48c15c550 2023-08-09T13:33:44.449 INFO:teuthology.orchestra.run.smithi203.stderr:2023-08-09T13:33:44.446+0000 7ff490a88640 1 --2- 172.21.15.203:0/2725086363 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7ff48c15c550 0x7ff48c0cd6d0 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7ff47c009e50 tx=0x7ff47c02bf60 comp rx=0 tx=0).ready entity=mon.2 client_cookie=37d279a2d7164ead server_cookie=0 in_seq=0 out_seq=0 2023-08-09T13:33:44.449 INFO:teuthology.orchestra.run.smithi203.stderr:2023-08-09T13:33:44.447+0000 7ff48b7fe640 1 -- 172.21.15.203:0/2725086363 <== mon.2 v2:172.21.15.203:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff47c02cb00 con 0x7ff48c15c550 2023-08-09T13:33:44.449 INFO:teuthology.orchestra.run.smithi203.stderr:2023-08-09T13:33:44.447+0000 7ff48b7fe640 1 -- 172.21.15.203:0/2725086363 <== mon.2 v2:172.21.15.203:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff47c02cc60 con 0x7ff48c15c550 2023-08-09T13:33:44.449 INFO:teuthology.orchestra.run.smithi203.stderr:2023-08-09T13:33:44.447+0000 7ff48b7fe640 1 -- 172.21.15.203:0/2725086363 <== mon.2 v2:172.21.15.203:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff47c0343f0 con 0x7ff48c15c550 2023-08-09T13:33:44.450 INFO:teuthology.orchestra.run.smithi203.stderr:2023-08-09T13:33:44.447+0000 7ff492cec640 1 -- 172.21.15.203:0/2725086363 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7ff48c15c550 msgr2=0x7ff48c0cd6d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:33:44.450 INFO:teuthology.orchestra.run.smithi203.stderr:2023-08-09T13:33:44.447+0000 7ff492cec640 1 --2- 172.21.15.203:0/2725086363 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7ff48c15c550 0x7ff48c0cd6d0 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7ff47c009e50 tx=0x7ff47c02bf60 comp rx=0 tx=0).stop 2023-08-09T13:33:44.450 INFO:teuthology.orchestra.run.smithi203.stderr:2023-08-09T13:33:44.447+0000 7ff492cec640 1 -- 172.21.15.203:0/2725086363 shutdown_connections 2023-08-09T13:33:44.451 INFO:teuthology.orchestra.run.smithi203.stderr:2023-08-09T13:33:44.447+0000 7ff492cec640 1 --2- 172.21.15.203:0/2725086363 >> v2:172.21.15.203:3300/0 conn(0x7ff48c0b45f0 0x7ff48c0b59d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:33:44.451 INFO:teuthology.orchestra.run.smithi203.stderr:2023-08-09T13:33:44.447+0000 7ff492cec640 1 --2- 172.21.15.203:0/2725086363 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7ff48c15c550 0x7ff48c0cd6d0 unknown :-1 s=CLOSED pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:33:44.451 INFO:teuthology.orchestra.run.smithi203.stderr:2023-08-09T13:33:44.447+0000 7ff492cec640 1 -- 172.21.15.203:0/2725086363 >> 172.21.15.203:0/2725086363 conn(0x7ff48c071520 msgr2=0x7ff48c0ceee0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-09T13:33:44.451 INFO:teuthology.orchestra.run.smithi203.stderr:2023-08-09T13:33:44.447+0000 7ff492cec640 1 -- 172.21.15.203:0/2725086363 shutdown_connections 2023-08-09T13:33:44.451 INFO:teuthology.orchestra.run.smithi203.stderr:2023-08-09T13:33:44.447+0000 7ff492cec640 1 -- 172.21.15.203:0/2725086363 wait complete. 2023-08-09T13:33:44.452 INFO:teuthology.orchestra.run.smithi203.stderr:2023-08-09T13:33:44.449+0000 7ff492cec640 1 Processor -- start 2023-08-09T13:33:44.452 INFO:teuthology.orchestra.run.smithi203.stderr:2023-08-09T13:33:44.449+0000 7ff492cec640 1 -- start start 2023-08-09T13:33:44.453 INFO:teuthology.orchestra.run.smithi203.stderr:2023-08-09T13:33:44.450+0000 7ff492cec640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7ff48c0b45f0 0x7ff48c14dc40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:33:44.453 INFO:teuthology.orchestra.run.smithi203.stderr:2023-08-09T13:33:44.450+0000 7ff492cec640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff48c0d04b0 0x7ff48c1e7090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:33:44.453 INFO:teuthology.orchestra.run.smithi203.stderr:2023-08-09T13:33:44.450+0000 7ff492cec640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7ff48c15c550 0x7ff48c1e75d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:33:44.453 INFO:teuthology.orchestra.run.smithi203.stderr:2023-08-09T13:33:44.450+0000 7ff492cec640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff48c1e4c40 con 0x7ff48c0d04b0 2023-08-09T13:33:44.453 INFO:teuthology.orchestra.run.smithi203.stderr:2023-08-09T13:33:44.450+0000 7ff492cec640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff48c0c6610 con 0x7ff48c15c550 2023-08-09T13:33:44.453 INFO:teuthology.orchestra.run.smithi203.stderr:2023-08-09T13:33:44.450+0000 7ff492cec640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff48c14b360 con 0x7ff48c0b45f0 2023-08-09T13:33:44.453 INFO:teuthology.orchestra.run.smithi203.stderr:2023-08-09T13:33:44.450+0000 7ff490a88640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7ff48c0b45f0 0x7ff48c14dc40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:33:44.454 INFO:teuthology.orchestra.run.smithi203.stderr:2023-08-09T13:33:44.450+0000 7ff490a88640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7ff48c0b45f0 0x7ff48c14dc40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.203:3301/0 says I am v2:172.21.15.203:59930/0 (socket says 172.21.15.203:59930) 2023-08-09T13:33:44.454 INFO:teuthology.orchestra.run.smithi203.stderr:2023-08-09T13:33:44.450+0000 7ff490a88640 1 -- 172.21.15.203:0/122600510 learned_addr learned my addr 172.21.15.203:0/122600510 (peer_addr_for_me v2:172.21.15.203:0/0) 2023-08-09T13:33:44.454 INFO:teuthology.orchestra.run.smithi203.stderr:2023-08-09T13:33:44.451+0000 7ff491289640 1 --2- 172.21.15.203:0/122600510 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7ff48c15c550 0x7ff48c1e75d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:33:44.454 INFO:tasks.ceph.mgr.y.smithi043.stderr:2023-08-09T13:33:44.452+0000 7fe23d40f040 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2023-08-09T13:33:44.455 INFO:teuthology.orchestra.run.smithi203.stderr:2023-08-09T13:33:44.451+0000 7ff490a88640 1 -- 172.21.15.203:0/122600510 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7ff48c15c550 msgr2=0x7ff48c1e75d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:33:44.456 INFO:teuthology.orchestra.run.smithi203.stderr:2023-08-09T13:33:44.451+0000 7ff490a88640 1 --2- 172.21.15.203:0/122600510 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7ff48c15c550 0x7ff48c1e75d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:33:44.456 INFO:teuthology.orchestra.run.smithi203.stderr:2023-08-09T13:33:44.451+0000 7ff490a88640 1 -- 172.21.15.203:0/122600510 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff48c0d04b0 msgr2=0x7ff48c1e7090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:33:44.456 INFO:teuthology.orchestra.run.smithi203.stderr:2023-08-09T13:33:44.451+0000 7ff48bfff640 1 --2- 172.21.15.203:0/122600510 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff48c0d04b0 0x7ff48c1e7090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:33:44.456 INFO:teuthology.orchestra.run.smithi203.stderr:2023-08-09T13:33:44.451+0000 7ff490a88640 1 --2- 172.21.15.203:0/122600510 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff48c0d04b0 0x7ff48c1e7090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:33:44.456 INFO:teuthology.orchestra.run.smithi203.stderr:2023-08-09T13:33:44.451+0000 7ff490a88640 1 -- 172.21.15.203:0/122600510 --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff47c009b30 con 0x7ff48c0b45f0 2023-08-09T13:33:44.456 INFO:teuthology.orchestra.run.smithi203.stderr:2023-08-09T13:33:44.451+0000 7ff490a88640 1 --2- 172.21.15.203:0/122600510 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7ff48c0b45f0 0x7ff48c14dc40 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7ff47c036040 tx=0x7ff47c004a40 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-09T13:33:44.456 INFO:teuthology.orchestra.run.smithi203.stderr:2023-08-09T13:33:44.451+0000 7ff4897fa640 1 -- 172.21.15.203:0/122600510 <== mon.2 v2:172.21.15.203:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff47c040670 con 0x7ff48c0b45f0 2023-08-09T13:33:44.457 INFO:teuthology.orchestra.run.smithi203.stderr:2023-08-09T13:33:44.452+0000 7ff4897fa640 1 -- 172.21.15.203:0/122600510 <== mon.2 v2:172.21.15.203:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff47c040810 con 0x7ff48c0b45f0 2023-08-09T13:33:44.457 INFO:teuthology.orchestra.run.smithi203.stderr:2023-08-09T13:33:44.452+0000 7ff492cec640 1 -- 172.21.15.203:0/122600510 --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff48c1e4e20 con 0x7ff48c0b45f0 2023-08-09T13:33:44.457 INFO:teuthology.orchestra.run.smithi203.stderr:2023-08-09T13:33:44.452+0000 7ff4897fa640 1 -- 172.21.15.203:0/122600510 <== mon.2 v2:172.21.15.203:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff47c040bc0 con 0x7ff48c0b45f0 2023-08-09T13:33:44.457 INFO:teuthology.orchestra.run.smithi203.stderr:2023-08-09T13:33:44.452+0000 7ff492cec640 1 -- 172.21.15.203:0/122600510 --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff48c1e53e0 con 0x7ff48c0b45f0 2023-08-09T13:33:44.457 INFO:teuthology.orchestra.run.smithi203.stderr:2023-08-09T13:33:44.452+0000 7ff4897fa640 1 -- 172.21.15.203:0/122600510 <== mon.2 v2:172.21.15.203:3301/0 4 ==== mgrmap(e 1) v1 ==== 660+0+0 (secure 0 0 0) 0x7ff47c03e720 con 0x7ff48c0b45f0 2023-08-09T13:33:44.457 INFO:teuthology.orchestra.run.smithi203.stderr:2023-08-09T13:33:44.452+0000 7ff4897fa640 1 -- 172.21.15.203:0/122600510 <== mon.2 v2:172.21.15.203:3301/0 5 ==== osd_map(7..7 src has 1..7) v4 ==== 1300+0+0 (secure 0 0 0) 0x7ff47c040d70 con 0x7ff48c0b45f0 2023-08-09T13:33:44.457 INFO:teuthology.orchestra.run.smithi203.stderr:2023-08-09T13:33:44.453+0000 7ff492cec640 1 -- 172.21.15.203:0/122600510 --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff48c0739f0 con 0x7ff48c0b45f0 2023-08-09T13:33:44.458 INFO:teuthology.orchestra.run.smithi203.stderr:2023-08-09T13:33:44.455+0000 7ff4897fa640 1 -- 172.21.15.203:0/122600510 <== mon.2 v2:172.21.15.203:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+72429 (secure 0 0 0) 0x7ff47c044020 con 0x7ff48c0b45f0 2023-08-09T13:33:44.460 INFO:tasks.ceph.mgr.x.smithi203.stderr:2023-08-09T13:33:44.458+0000 7f943a4b2040 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2023-08-09T13:33:44.529 INFO:teuthology.orchestra.run.smithi203.stderr:2023-08-09T13:33:44.527+0000 7ff492cec640 1 -- 172.21.15.203:0/122600510 --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_command({"prefix": "osd new", "uuid": "81811e4b-4d24-4f51-a82c-f1d700a2a00d", "id": 5} v 0) v1 -- 0x7ff48c0c91d0 con 0x7ff48c0b45f0 2023-08-09T13:33:44.535 INFO:teuthology.orchestra.run.smithi203.stderr:2023-08-09T13:33:44.533+0000 7ff4897fa640 1 -- 172.21.15.203:0/122600510 <== mon.2 v2:172.21.15.203:3301/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "81811e4b-4d24-4f51-a82c-f1d700a2a00d", "id": 5}]=0 v8) v1 ==== 112+0+2 (secure 0 0 0) 0x7ff47c03e9d0 con 0x7ff48c0b45f0 2023-08-09T13:33:44.536 INFO:teuthology.orchestra.run.smithi203.stdout:5 2023-08-09T13:33:44.537 INFO:teuthology.orchestra.run.smithi203.stderr:2023-08-09T13:33:44.534+0000 7ff492cec640 1 -- 172.21.15.203:0/122600510 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7ff48c0b45f0 msgr2=0x7ff48c14dc40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:33:44.537 INFO:teuthology.orchestra.run.smithi203.stderr:2023-08-09T13:33:44.534+0000 7ff492cec640 1 --2- 172.21.15.203:0/122600510 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7ff48c0b45f0 0x7ff48c14dc40 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7ff47c036040 tx=0x7ff47c004a40 comp rx=0 tx=0).stop 2023-08-09T13:33:44.537 INFO:teuthology.orchestra.run.smithi203.stderr:2023-08-09T13:33:44.534+0000 7ff492cec640 1 -- 172.21.15.203:0/122600510 shutdown_connections 2023-08-09T13:33:44.537 INFO:teuthology.orchestra.run.smithi203.stderr:2023-08-09T13:33:44.534+0000 7ff492cec640 1 --2- 172.21.15.203:0/122600510 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7ff48c15c550 0x7ff48c1e75d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:33:44.537 INFO:teuthology.orchestra.run.smithi203.stderr:2023-08-09T13:33:44.534+0000 7ff492cec640 1 --2- 172.21.15.203:0/122600510 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff48c0d04b0 0x7ff48c1e7090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:33:44.537 INFO:teuthology.orchestra.run.smithi203.stderr:2023-08-09T13:33:44.534+0000 7ff492cec640 1 --2- 172.21.15.203:0/122600510 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7ff48c0b45f0 0x7ff48c14dc40 unknown :-1 s=CLOSED pgs=17 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:33:44.538 INFO:teuthology.orchestra.run.smithi203.stderr:2023-08-09T13:33:44.534+0000 7ff492cec640 1 -- 172.21.15.203:0/122600510 >> 172.21.15.203:0/122600510 conn(0x7ff48c071520 msgr2=0x7ff48c15f4a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-09T13:33:44.538 INFO:teuthology.orchestra.run.smithi203.stderr:2023-08-09T13:33:44.535+0000 7ff492cec640 1 -- 172.21.15.203:0/122600510 shutdown_connections 2023-08-09T13:33:44.538 INFO:teuthology.orchestra.run.smithi203.stderr:2023-08-09T13:33:44.535+0000 7ff492cec640 1 -- 172.21.15.203:0/122600510 wait complete. 2023-08-09T13:33:44.553 DEBUG:teuthology.orchestra.run.smithi203:> sudo ceph --cluster ceph osd new 307bac4d-6a5b-443d-af75-56d695d7674f 6 2023-08-09T13:33:44.586 INFO:tasks.ceph.mgr.y.smithi043.stderr:2023-08-09T13:33:44.584+0000 7fe23d40f040 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2023-08-09T13:33:44.592 INFO:tasks.ceph.mgr.x.smithi203.stderr:2023-08-09T13:33:44.590+0000 7f943a4b2040 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2023-08-09T13:33:44.740 INFO:teuthology.orchestra.run.smithi203.stderr:2023-08-09T13:33:44.737+0000 7f77df83f640 1 Processor -- start 2023-08-09T13:33:44.740 INFO:teuthology.orchestra.run.smithi203.stderr:2023-08-09T13:33:44.737+0000 7f77df83f640 1 -- start start 2023-08-09T13:33:44.742 INFO:teuthology.orchestra.run.smithi203.stderr:2023-08-09T13:33:44.739+0000 7f77df83f640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f77d80b1260 0x7f77d80cee00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:33:44.743 INFO:teuthology.orchestra.run.smithi203.stderr:2023-08-09T13:33:44.739+0000 7f77df83f640 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7f77d80cf340 0x7f77d80d0350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:33:44.743 INFO:teuthology.orchestra.run.smithi203.stderr:2023-08-09T13:33:44.739+0000 7f77df83f640 1 -- --> v1:172.21.15.43:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f77d80cd700 con 0x7f77d815c620 2023-08-09T13:33:44.743 INFO:teuthology.orchestra.run.smithi203.stderr:2023-08-09T13:33:44.739+0000 7f77df83f640 1 -- --> v2:172.21.15.203:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f77d80b1b70 con 0x7f77d80cf340 2023-08-09T13:33:44.744 INFO:teuthology.orchestra.run.smithi203.stderr:2023-08-09T13:33:44.739+0000 7f77df83f640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f77d80cf7b0 con 0x7f77d80b1260 2023-08-09T13:33:44.744 INFO:teuthology.orchestra.run.smithi203.stderr:2023-08-09T13:33:44.739+0000 7f77dcdda640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f77d80b1260 0x7f77d80cee00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:33:44.744 INFO:teuthology.orchestra.run.smithi203.stderr:2023-08-09T13:33:44.739+0000 7f77dcdda640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f77d80b1260 0x7f77d80cee00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.203:3301/0 says I am v2:172.21.15.203:59932/0 (socket says 172.21.15.203:59932) 2023-08-09T13:33:44.745 INFO:teuthology.orchestra.run.smithi203.stderr:2023-08-09T13:33:44.739+0000 7f77dcdda640 1 -- 172.21.15.203:0/112982973 learned_addr learned my addr 172.21.15.203:0/112982973 (peer_addr_for_me v2:172.21.15.203:0/0) 2023-08-09T13:33:44.745 INFO:teuthology.orchestra.run.smithi203.stderr:2023-08-09T13:33:44.739+0000 7f77ddddc640 1 --2- 172.21.15.203:0/112982973 >> v2:172.21.15.203:3300/0 conn(0x7f77d80cf340 0x7f77d80d0350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:33:44.745 INFO:teuthology.orchestra.run.smithi203.stderr:2023-08-09T13:33:44.740+0000 7f77dcdda640 1 -- 172.21.15.203:0/112982973 >> v2:172.21.15.203:3300/0 conn(0x7f77d80cf340 msgr2=0x7f77d80d0350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-09T13:33:44.745 INFO:teuthology.orchestra.run.smithi203.stderr:2023-08-09T13:33:44.740+0000 7f77dcdda640 1 --2- 172.21.15.203:0/112982973 >> v2:172.21.15.203:3300/0 conn(0x7f77d80cf340 0x7f77d80d0350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:33:44.746 INFO:teuthology.orchestra.run.smithi203.stderr:2023-08-09T13:33:44.740+0000 7f77dcdda640 1 -- 172.21.15.203:0/112982973 >> v1:172.21.15.43:6789/0 conn(0x7f77d815c620 legacy=0x7f77d80cceb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:33:44.746 INFO:teuthology.orchestra.run.smithi203.stderr:2023-08-09T13:33:44.740+0000 7f77dcdda640 1 -- 172.21.15.203:0/112982973 --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f77d80cd9e0 con 0x7f77d80b1260 2023-08-09T13:33:44.751 INFO:teuthology.orchestra.run.smithi203.stderr:2023-08-09T13:33:44.741+0000 7f77dcdda640 1 --2- 172.21.15.203:0/112982973 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f77d80b1260 0x7f77d80cee00 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7f77c8009870 tx=0x7f77c802b7e0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=7eab4d2546004784 server_cookie=0 in_seq=0 out_seq=0 2023-08-09T13:33:44.751 INFO:teuthology.orchestra.run.smithi203.stderr:2023-08-09T13:33:44.741+0000 7f77cffff640 1 -- 172.21.15.203:0/112982973 <== mon.2 v2:172.21.15.203:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f77c8031740 con 0x7f77d80b1260 2023-08-09T13:33:44.751 INFO:teuthology.orchestra.run.smithi203.stderr:2023-08-09T13:33:44.741+0000 7f77cffff640 1 -- 172.21.15.203:0/112982973 <== mon.2 v2:172.21.15.203:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f77c80318a0 con 0x7f77d80b1260 2023-08-09T13:33:44.752 INFO:teuthology.orchestra.run.smithi203.stderr:2023-08-09T13:33:44.741+0000 7f77cffff640 1 -- 172.21.15.203:0/112982973 <== mon.2 v2:172.21.15.203:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f77c8031c30 con 0x7f77d80b1260 2023-08-09T13:33:44.752 INFO:teuthology.orchestra.run.smithi203.stderr:2023-08-09T13:33:44.741+0000 7f77df83f640 1 -- 172.21.15.203:0/112982973 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f77d80b1260 msgr2=0x7f77d80cee00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:33:44.752 INFO:teuthology.orchestra.run.smithi203.stderr:2023-08-09T13:33:44.741+0000 7f77df83f640 1 --2- 172.21.15.203:0/112982973 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f77d80b1260 0x7f77d80cee00 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7f77c8009870 tx=0x7f77c802b7e0 comp rx=0 tx=0).stop 2023-08-09T13:33:44.753 INFO:teuthology.orchestra.run.smithi203.stderr:2023-08-09T13:33:44.742+0000 7f77df83f640 1 -- 172.21.15.203:0/112982973 shutdown_connections 2023-08-09T13:33:44.753 INFO:teuthology.orchestra.run.smithi203.stderr:2023-08-09T13:33:44.742+0000 7f77df83f640 1 --2- 172.21.15.203:0/112982973 >> v2:172.21.15.203:3300/0 conn(0x7f77d80cf340 0x7f77d80d0350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:33:44.753 INFO:teuthology.orchestra.run.smithi203.stderr:2023-08-09T13:33:44.742+0000 7f77df83f640 1 --2- 172.21.15.203:0/112982973 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f77d80b1260 0x7f77d80cee00 unknown :-1 s=CLOSED pgs=18 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:33:44.753 INFO:teuthology.orchestra.run.smithi203.stderr:2023-08-09T13:33:44.742+0000 7f77df83f640 1 -- 172.21.15.203:0/112982973 >> 172.21.15.203:0/112982973 conn(0x7f77d809b750 msgr2=0x7f77d80d2410 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-09T13:33:44.754 INFO:teuthology.orchestra.run.smithi203.stderr:2023-08-09T13:33:44.742+0000 7f77df83f640 1 -- 172.21.15.203:0/112982973 shutdown_connections 2023-08-09T13:33:44.754 INFO:teuthology.orchestra.run.smithi203.stderr:2023-08-09T13:33:44.742+0000 7f77df83f640 1 -- 172.21.15.203:0/112982973 wait complete. 2023-08-09T13:33:44.754 INFO:teuthology.orchestra.run.smithi203.stderr:2023-08-09T13:33:44.743+0000 7f77df83f640 1 Processor -- start 2023-08-09T13:33:44.755 INFO:teuthology.orchestra.run.smithi203.stderr:2023-08-09T13:33:44.744+0000 7f77df83f640 1 -- start start 2023-08-09T13:33:44.755 INFO:teuthology.orchestra.run.smithi203.stderr:2023-08-09T13:33:44.744+0000 7f77df83f640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f77d80b1260 0x7f77d80b5030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:33:44.755 INFO:teuthology.orchestra.run.smithi203.stderr:2023-08-09T13:33:44.744+0000 7f77df83f640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f77d80cf340 0x7f77d8159c90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:33:44.755 INFO:teuthology.orchestra.run.smithi203.stderr:2023-08-09T13:33:44.744+0000 7f77df83f640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f77d815c620 0x7f77d80b26d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:33:44.756 INFO:teuthology.orchestra.run.smithi203.stderr:2023-08-09T13:33:44.744+0000 7f77df83f640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f77d80b66f0 con 0x7f77d80cf340 2023-08-09T13:33:44.756 INFO:teuthology.orchestra.run.smithi203.stderr:2023-08-09T13:33:44.744+0000 7f77df83f640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f77d80b5610 con 0x7f77d80b1260 2023-08-09T13:33:44.756 INFO:teuthology.orchestra.run.smithi203.stderr:2023-08-09T13:33:44.744+0000 7f77df83f640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f77d80b58f0 con 0x7f77d815c620 2023-08-09T13:33:44.757 INFO:teuthology.orchestra.run.smithi203.stderr:2023-08-09T13:33:44.745+0000 7f77dd5db640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f77d80b1260 0x7f77d80b5030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:33:44.757 INFO:teuthology.orchestra.run.smithi203.stderr:2023-08-09T13:33:44.745+0000 7f77dd5db640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f77d80b1260 0x7f77d80b5030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.203:3300/0 says I am v2:172.21.15.203:48950/0 (socket says 172.21.15.203:48950) 2023-08-09T13:33:44.757 INFO:teuthology.orchestra.run.smithi203.stderr:2023-08-09T13:33:44.745+0000 7f77ddddc640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f77d815c620 0x7f77d80b26d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:33:44.757 INFO:teuthology.orchestra.run.smithi203.stderr:2023-08-09T13:33:44.745+0000 7f77dd5db640 1 -- 172.21.15.203:0/1918451416 learned_addr learned my addr 172.21.15.203:0/1918451416 (peer_addr_for_me v2:172.21.15.203:0/0) 2023-08-09T13:33:44.758 INFO:teuthology.orchestra.run.smithi203.stderr:2023-08-09T13:33:44.745+0000 7f77dcdda640 1 --2- 172.21.15.203:0/1918451416 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f77d80cf340 0x7f77d8159c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:33:44.758 INFO:teuthology.orchestra.run.smithi203.stderr:2023-08-09T13:33:44.745+0000 7f77dd5db640 1 -- 172.21.15.203:0/1918451416 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f77d815c620 msgr2=0x7f77d80b26d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:33:44.758 INFO:teuthology.orchestra.run.smithi203.stderr:2023-08-09T13:33:44.745+0000 7f77dd5db640 1 --2- 172.21.15.203:0/1918451416 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f77d815c620 0x7f77d80b26d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:33:44.759 INFO:teuthology.orchestra.run.smithi203.stderr:2023-08-09T13:33:44.745+0000 7f77dd5db640 1 -- 172.21.15.203:0/1918451416 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f77d80cf340 msgr2=0x7f77d8159c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:33:44.759 INFO:teuthology.orchestra.run.smithi203.stderr:2023-08-09T13:33:44.745+0000 7f77dd5db640 1 --2- 172.21.15.203:0/1918451416 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f77d80cf340 0x7f77d8159c90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:33:44.759 INFO:teuthology.orchestra.run.smithi203.stderr:2023-08-09T13:33:44.745+0000 7f77dd5db640 1 -- 172.21.15.203:0/1918451416 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f77c80094d0 con 0x7f77d80b1260 2023-08-09T13:33:44.759 INFO:teuthology.orchestra.run.smithi203.stderr:2023-08-09T13:33:44.745+0000 7f77dd5db640 1 --2- 172.21.15.203:0/1918451416 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f77d80b1260 0x7f77d80b5030 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7f77d400c3e0 tx=0x7f77d400c8a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-09T13:33:44.760 INFO:teuthology.orchestra.run.smithi203.stderr:2023-08-09T13:33:44.746+0000 7f77cdffb640 1 -- 172.21.15.203:0/1918451416 <== mon.1 v2:172.21.15.203:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f77d4011790 con 0x7f77d80b1260 2023-08-09T13:33:44.760 INFO:teuthology.orchestra.run.smithi203.stderr:2023-08-09T13:33:44.746+0000 7f77cdffb640 1 -- 172.21.15.203:0/1918451416 <== mon.1 v2:172.21.15.203:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f77d40118f0 con 0x7f77d80b1260 2023-08-09T13:33:44.760 INFO:teuthology.orchestra.run.smithi203.stderr:2023-08-09T13:33:44.746+0000 7f77df83f640 1 -- 172.21.15.203:0/1918451416 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f77d80b5bd0 con 0x7f77d80b1260 2023-08-09T13:33:44.761 INFO:teuthology.orchestra.run.smithi203.stderr:2023-08-09T13:33:44.746+0000 7f77cdffb640 1 -- 172.21.15.203:0/1918451416 <== mon.1 v2:172.21.15.203:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f77d4011ce0 con 0x7f77d80b1260 2023-08-09T13:33:44.761 INFO:teuthology.orchestra.run.smithi203.stderr:2023-08-09T13:33:44.746+0000 7f77df83f640 1 -- 172.21.15.203:0/1918451416 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f77d815ac90 con 0x7f77d80b1260 2023-08-09T13:33:44.761 INFO:teuthology.orchestra.run.smithi203.stderr:2023-08-09T13:33:44.746+0000 7f77cdffb640 1 -- 172.21.15.203:0/1918451416 <== mon.1 v2:172.21.15.203:3300/0 4 ==== mgrmap(e 1) v1 ==== 660+0+0 (secure 0 0 0) 0x7f77d401c760 con 0x7f77d80b1260 2023-08-09T13:33:44.761 INFO:teuthology.orchestra.run.smithi203.stderr:2023-08-09T13:33:44.746+0000 7f77cdffb640 1 -- 172.21.15.203:0/1918451416 <== mon.1 v2:172.21.15.203:3300/0 5 ==== osd_map(8..8 src has 1..8) v4 ==== 1415+0+0 (secure 0 0 0) 0x7f77d4011e40 con 0x7f77d80b1260 2023-08-09T13:33:44.762 INFO:teuthology.orchestra.run.smithi203.stderr:2023-08-09T13:33:44.747+0000 7f77df83f640 1 -- 172.21.15.203:0/1918451416 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f77d80cdc70 con 0x7f77d80b1260 2023-08-09T13:33:44.763 INFO:tasks.ceph.mgr.y.smithi043.stderr:2023-08-09T13:33:44.751+0000 7fe23d40f040 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2023-08-09T13:33:44.763 INFO:teuthology.orchestra.run.smithi203.stderr:2023-08-09T13:33:44.750+0000 7f77cdffb640 1 -- 172.21.15.203:0/1918451416 <== mon.1 v2:172.21.15.203:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+72429 (secure 0 0 0) 0x7f77d40128a0 con 0x7f77d80b1260 2023-08-09T13:33:44.764 INFO:tasks.ceph.mgr.x.smithi203.stderr:2023-08-09T13:33:44.758+0000 7f943a4b2040 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2023-08-09T13:33:44.829 INFO:teuthology.orchestra.run.smithi203.stderr:2023-08-09T13:33:44.826+0000 7f77df83f640 1 -- 172.21.15.203:0/1918451416 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_command({"prefix": "osd new", "uuid": "307bac4d-6a5b-443d-af75-56d695d7674f", "id": 6} v 0) v1 -- 0x7f77d80739f0 con 0x7f77d80b1260 2023-08-09T13:33:44.835 INFO:teuthology.orchestra.run.smithi203.stderr:2023-08-09T13:33:44.833+0000 7f77cdffb640 1 -- 172.21.15.203:0/1918451416 <== mon.1 v2:172.21.15.203:3300/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "307bac4d-6a5b-443d-af75-56d695d7674f", "id": 6}]=0 v9) v1 ==== 112+0+2 (secure 0 0 0) 0x7f77d4012a40 con 0x7f77d80b1260 2023-08-09T13:33:44.836 INFO:teuthology.orchestra.run.smithi203.stdout:6 2023-08-09T13:33:44.837 INFO:teuthology.orchestra.run.smithi203.stderr:2023-08-09T13:33:44.834+0000 7f77df83f640 1 -- 172.21.15.203:0/1918451416 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f77d80b1260 msgr2=0x7f77d80b5030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:33:44.837 INFO:teuthology.orchestra.run.smithi203.stderr:2023-08-09T13:33:44.834+0000 7f77df83f640 1 --2- 172.21.15.203:0/1918451416 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f77d80b1260 0x7f77d80b5030 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7f77d400c3e0 tx=0x7f77d400c8a0 comp rx=0 tx=0).stop 2023-08-09T13:33:44.838 INFO:teuthology.orchestra.run.smithi203.stderr:2023-08-09T13:33:44.834+0000 7f77df83f640 1 -- 172.21.15.203:0/1918451416 shutdown_connections 2023-08-09T13:33:44.838 INFO:teuthology.orchestra.run.smithi203.stderr:2023-08-09T13:33:44.834+0000 7f77df83f640 1 --2- 172.21.15.203:0/1918451416 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f77d815c620 0x7f77d80b26d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:33:44.838 INFO:teuthology.orchestra.run.smithi203.stderr:2023-08-09T13:33:44.834+0000 7f77df83f640 1 --2- 172.21.15.203:0/1918451416 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f77d80cf340 0x7f77d8159c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:33:44.838 INFO:teuthology.orchestra.run.smithi203.stderr:2023-08-09T13:33:44.834+0000 7f77df83f640 1 --2- 172.21.15.203:0/1918451416 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f77d80b1260 0x7f77d80b5030 unknown :-1 s=CLOSED pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:33:44.839 INFO:teuthology.orchestra.run.smithi203.stderr:2023-08-09T13:33:44.834+0000 7f77df83f640 1 -- 172.21.15.203:0/1918451416 >> 172.21.15.203:0/1918451416 conn(0x7f77d809b750 msgr2=0x7f77d80ce6a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-09T13:33:44.839 INFO:teuthology.orchestra.run.smithi203.stderr:2023-08-09T13:33:44.835+0000 7f77df83f640 1 -- 172.21.15.203:0/1918451416 shutdown_connections 2023-08-09T13:33:44.839 INFO:teuthology.orchestra.run.smithi203.stderr:2023-08-09T13:33:44.835+0000 7f77df83f640 1 -- 172.21.15.203:0/1918451416 wait complete. 2023-08-09T13:33:44.853 DEBUG:teuthology.orchestra.run.smithi203:> sudo ceph --cluster ceph osd new a95dc7ca-9f1b-4295-af70-68a22a2d7d5e 7 2023-08-09T13:33:44.867 INFO:tasks.ceph.mgr.y.smithi043.stderr:2023-08-09T13:33:44.866+0000 7fe23d40f040 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2023-08-09T13:33:44.874 INFO:tasks.ceph.mgr.x.smithi203.stderr:2023-08-09T13:33:44.871+0000 7f943a4b2040 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2023-08-09T13:33:44.919 INFO:tasks.ceph.mgr.y.smithi043.stderr:2023-08-09T13:33:44.918+0000 7fe23d40f040 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2023-08-09T13:33:44.926 INFO:tasks.ceph.mgr.x.smithi203.stderr:2023-08-09T13:33:44.924+0000 7f943a4b2040 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2023-08-09T13:33:45.058 INFO:teuthology.orchestra.run.smithi203.stderr:2023-08-09T13:33:45.056+0000 7ff7bfd0f640 1 Processor -- start 2023-08-09T13:33:45.058 INFO:teuthology.orchestra.run.smithi203.stderr:2023-08-09T13:33:45.056+0000 7ff7bfd0f640 1 -- start start 2023-08-09T13:33:45.059 INFO:teuthology.orchestra.run.smithi203.stderr:2023-08-09T13:33:45.057+0000 7ff7bfd0f640 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7ff7b815c550 0x7ff7b80cd6d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:33:45.060 INFO:teuthology.orchestra.run.smithi203.stderr:2023-08-09T13:33:45.057+0000 7ff7bfd0f640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7ff7b80d04b0 0x7ff7b80b3ee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:33:45.060 INFO:teuthology.orchestra.run.smithi203.stderr:2023-08-09T13:33:45.057+0000 7ff7bfd0f640 1 -- --> v1:172.21.15.203:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff7b80b4870 con 0x7ff7b80b4420 2023-08-09T13:33:45.060 INFO:teuthology.orchestra.run.smithi203.stderr:2023-08-09T13:33:45.057+0000 7ff7bfd0f640 1 -- --> v2:172.21.15.203:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff7b80b7bd0 con 0x7ff7b815c550 2023-08-09T13:33:45.060 INFO:teuthology.orchestra.run.smithi203.stderr:2023-08-09T13:33:45.057+0000 7ff7bfd0f640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff7b80d1c00 con 0x7ff7b80d04b0 2023-08-09T13:33:45.061 INFO:teuthology.orchestra.run.smithi203.stderr:2023-08-09T13:33:45.058+0000 7ff7bdaab640 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7ff7b815c550 0x7ff7b80cd6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:33:45.061 INFO:teuthology.orchestra.run.smithi203.stderr:2023-08-09T13:33:45.058+0000 7ff7bd2aa640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7ff7b80d04b0 0x7ff7b80b3ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:33:45.061 INFO:teuthology.orchestra.run.smithi203.stderr:2023-08-09T13:33:45.058+0000 7ff7bdaab640 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7ff7b815c550 0x7ff7b80cd6d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.203:3300/0 says I am v2:172.21.15.203:48966/0 (socket says 172.21.15.203:48966) 2023-08-09T13:33:45.061 INFO:teuthology.orchestra.run.smithi203.stderr:2023-08-09T13:33:45.058+0000 7ff7bdaab640 1 -- 172.21.15.203:0/2643290263 learned_addr learned my addr 172.21.15.203:0/2643290263 (peer_addr_for_me v2:172.21.15.203:0/0) 2023-08-09T13:33:45.061 INFO:teuthology.orchestra.run.smithi203.stderr:2023-08-09T13:33:45.058+0000 7ff7bd2aa640 1 -- 172.21.15.203:0/2643290263 >> v2:172.21.15.203:3300/0 conn(0x7ff7b815c550 msgr2=0x7ff7b80cd6d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-09T13:33:45.061 INFO:teuthology.orchestra.run.smithi203.stderr:2023-08-09T13:33:45.058+0000 7ff7bd2aa640 1 --2- 172.21.15.203:0/2643290263 >> v2:172.21.15.203:3300/0 conn(0x7ff7b815c550 0x7ff7b80cd6d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:33:45.062 INFO:teuthology.orchestra.run.smithi203.stderr:2023-08-09T13:33:45.058+0000 7ff7bd2aa640 1 -- 172.21.15.203:0/2643290263 >> v1:172.21.15.203:6789/0 conn(0x7ff7b80b4420 legacy=0x7ff7b806a760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:33:45.062 INFO:teuthology.orchestra.run.smithi203.stderr:2023-08-09T13:33:45.058+0000 7ff7bd2aa640 1 -- 172.21.15.203:0/2643290263 --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff7b815c930 con 0x7ff7b80d04b0 2023-08-09T13:33:45.062 INFO:teuthology.orchestra.run.smithi203.stderr:2023-08-09T13:33:45.058+0000 7ff7bdaab640 1 --2- 172.21.15.203:0/2643290263 >> v2:172.21.15.203:3300/0 conn(0x7ff7b815c550 0x7ff7b80cd6d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-08-09T13:33:45.063 INFO:teuthology.orchestra.run.smithi203.stderr:2023-08-09T13:33:45.059+0000 7ff7bd2aa640 1 --2- 172.21.15.203:0/2643290263 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7ff7b80d04b0 0x7ff7b80b3ee0 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7ff7a80097a0 tx=0x7ff7a802b8b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=eb2a3e1b18018829 server_cookie=0 in_seq=0 out_seq=0 2023-08-09T13:33:45.063 INFO:teuthology.orchestra.run.smithi203.stderr:2023-08-09T13:33:45.060+0000 7ff7bcaa9640 1 -- 172.21.15.203:0/2643290263 <== mon.2 v2:172.21.15.203:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff7a802d040 con 0x7ff7b80d04b0 2023-08-09T13:33:45.063 INFO:teuthology.orchestra.run.smithi203.stderr:2023-08-09T13:33:45.060+0000 7ff7bcaa9640 1 -- 172.21.15.203:0/2643290263 <== mon.2 v2:172.21.15.203:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff7a8031810 con 0x7ff7b80d04b0 2023-08-09T13:33:45.063 INFO:teuthology.orchestra.run.smithi203.stderr:2023-08-09T13:33:45.060+0000 7ff7bcaa9640 1 -- 172.21.15.203:0/2643290263 <== mon.2 v2:172.21.15.203:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff7a8031ba0 con 0x7ff7b80d04b0 2023-08-09T13:33:45.063 INFO:teuthology.orchestra.run.smithi203.stderr:2023-08-09T13:33:45.060+0000 7ff7bfd0f640 1 -- 172.21.15.203:0/2643290263 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7ff7b80d04b0 msgr2=0x7ff7b80b3ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:33:45.064 INFO:teuthology.orchestra.run.smithi203.stderr:2023-08-09T13:33:45.060+0000 7ff7bfd0f640 1 --2- 172.21.15.203:0/2643290263 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7ff7b80d04b0 0x7ff7b80b3ee0 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7ff7a80097a0 tx=0x7ff7a802b8b0 comp rx=0 tx=0).stop 2023-08-09T13:33:45.064 INFO:teuthology.orchestra.run.smithi203.stderr:2023-08-09T13:33:45.060+0000 7ff7bfd0f640 1 -- 172.21.15.203:0/2643290263 shutdown_connections 2023-08-09T13:33:45.064 INFO:teuthology.orchestra.run.smithi203.stderr:2023-08-09T13:33:45.060+0000 7ff7bfd0f640 1 --2- 172.21.15.203:0/2643290263 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7ff7b80d04b0 0x7ff7b80b3ee0 unknown :-1 s=CLOSED pgs=19 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:33:45.064 INFO:teuthology.orchestra.run.smithi203.stderr:2023-08-09T13:33:45.060+0000 7ff7bfd0f640 1 --2- 172.21.15.203:0/2643290263 >> v2:172.21.15.203:3300/0 conn(0x7ff7b815c550 0x7ff7b80cd6d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:33:45.064 INFO:teuthology.orchestra.run.smithi203.stderr:2023-08-09T13:33:45.060+0000 7ff7bfd0f640 1 -- 172.21.15.203:0/2643290263 >> 172.21.15.203:0/2643290263 conn(0x7ff7b8071520 msgr2=0x7ff7b80ceee0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-09T13:33:45.064 INFO:teuthology.orchestra.run.smithi203.stderr:2023-08-09T13:33:45.060+0000 7ff7bfd0f640 1 -- 172.21.15.203:0/2643290263 shutdown_connections 2023-08-09T13:33:45.065 INFO:teuthology.orchestra.run.smithi203.stderr:2023-08-09T13:33:45.060+0000 7ff7bfd0f640 1 -- 172.21.15.203:0/2643290263 wait complete. 2023-08-09T13:33:45.066 INFO:teuthology.orchestra.run.smithi203.stderr:2023-08-09T13:33:45.062+0000 7ff7bfd0f640 1 Processor -- start 2023-08-09T13:33:45.066 INFO:teuthology.orchestra.run.smithi203.stderr:2023-08-09T13:33:45.062+0000 7ff7bfd0f640 1 -- start start 2023-08-09T13:33:45.066 INFO:teuthology.orchestra.run.smithi203.stderr:2023-08-09T13:33:45.063+0000 7ff7bfd0f640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff7b80b4420 0x7ff7b81f1220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:33:45.066 INFO:teuthology.orchestra.run.smithi203.stderr:2023-08-09T13:33:45.063+0000 7ff7bfd0f640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7ff7b80d04b0 0x7ff7b814b1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:33:45.066 INFO:teuthology.orchestra.run.smithi203.stderr:2023-08-09T13:33:45.063+0000 7ff7bfd0f640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7ff7b815c550 0x7ff7b80c6fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:33:45.066 INFO:teuthology.orchestra.run.smithi203.stderr:2023-08-09T13:33:45.063+0000 7ff7bfd0f640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff7b81f1760 con 0x7ff7b80b4420 2023-08-09T13:33:45.066 INFO:teuthology.orchestra.run.smithi203.stderr:2023-08-09T13:33:45.063+0000 7ff7bfd0f640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff7b814c120 con 0x7ff7b815c550 2023-08-09T13:33:45.067 INFO:teuthology.orchestra.run.smithi203.stderr:2023-08-09T13:33:45.063+0000 7ff7bfd0f640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff7b814bb60 con 0x7ff7b80d04b0 2023-08-09T13:33:45.067 INFO:teuthology.orchestra.run.smithi203.stderr:2023-08-09T13:33:45.063+0000 7ff7bd2aa640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7ff7b80d04b0 0x7ff7b814b1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:33:45.067 INFO:teuthology.orchestra.run.smithi203.stderr:2023-08-09T13:33:45.063+0000 7ff7bd2aa640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7ff7b80d04b0 0x7ff7b814b1a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.203:3301/0 says I am v2:172.21.15.203:59950/0 (socket says 172.21.15.203:59950) 2023-08-09T13:33:45.067 INFO:teuthology.orchestra.run.smithi203.stderr:2023-08-09T13:33:45.063+0000 7ff7bd2aa640 1 -- 172.21.15.203:0/586383070 learned_addr learned my addr 172.21.15.203:0/586383070 (peer_addr_for_me v2:172.21.15.203:0/0) 2023-08-09T13:33:45.067 INFO:teuthology.orchestra.run.smithi203.stderr:2023-08-09T13:33:45.063+0000 7ff7be2ac640 1 --2- 172.21.15.203:0/586383070 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7ff7b815c550 0x7ff7b80c6fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:33:45.068 INFO:teuthology.orchestra.run.smithi203.stderr:2023-08-09T13:33:45.064+0000 7ff7bdaab640 1 --2- 172.21.15.203:0/586383070 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff7b80b4420 0x7ff7b81f1220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:33:45.069 INFO:teuthology.orchestra.run.smithi203.stderr:2023-08-09T13:33:45.064+0000 7ff7bd2aa640 1 -- 172.21.15.203:0/586383070 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7ff7b815c550 msgr2=0x7ff7b80c6fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:33:45.069 INFO:teuthology.orchestra.run.smithi203.stderr:2023-08-09T13:33:45.064+0000 7ff7bd2aa640 1 --2- 172.21.15.203:0/586383070 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7ff7b815c550 0x7ff7b80c6fc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:33:45.069 INFO:teuthology.orchestra.run.smithi203.stderr:2023-08-09T13:33:45.064+0000 7ff7bd2aa640 1 -- 172.21.15.203:0/586383070 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff7b80b4420 msgr2=0x7ff7b81f1220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:33:45.069 INFO:teuthology.orchestra.run.smithi203.stderr:2023-08-09T13:33:45.064+0000 7ff7bd2aa640 1 --2- 172.21.15.203:0/586383070 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff7b80b4420 0x7ff7b81f1220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:33:45.069 INFO:teuthology.orchestra.run.smithi203.stderr:2023-08-09T13:33:45.064+0000 7ff7bd2aa640 1 -- 172.21.15.203:0/586383070 --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff7b40094d0 con 0x7ff7b80d04b0 2023-08-09T13:33:45.069 INFO:teuthology.orchestra.run.smithi203.stderr:2023-08-09T13:33:45.064+0000 7ff7bdaab640 1 --2- 172.21.15.203:0/586383070 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff7b80b4420 0x7ff7b81f1220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-08-09T13:33:45.069 INFO:teuthology.orchestra.run.smithi203.stderr:2023-08-09T13:33:45.064+0000 7ff7bd2aa640 1 --2- 172.21.15.203:0/586383070 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7ff7b80d04b0 0x7ff7b814b1a0 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7ff7a80098c0 tx=0x7ff7a80046c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-09T13:33:45.070 INFO:teuthology.orchestra.run.smithi203.stderr:2023-08-09T13:33:45.064+0000 7ff7ae7fc640 1 -- 172.21.15.203:0/586383070 <== mon.2 v2:172.21.15.203:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff7a8006e10 con 0x7ff7b80d04b0 2023-08-09T13:33:45.070 INFO:teuthology.orchestra.run.smithi203.stderr:2023-08-09T13:33:45.065+0000 7ff7ae7fc640 1 -- 172.21.15.203:0/586383070 <== mon.2 v2:172.21.15.203:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff7a803ad60 con 0x7ff7b80d04b0 2023-08-09T13:33:45.070 INFO:teuthology.orchestra.run.smithi203.stderr:2023-08-09T13:33:45.065+0000 7ff7bfd0f640 1 -- 172.21.15.203:0/586383070 --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff7a8009400 con 0x7ff7b80d04b0 2023-08-09T13:33:45.070 INFO:teuthology.orchestra.run.smithi203.stderr:2023-08-09T13:33:45.065+0000 7ff7ae7fc640 1 -- 172.21.15.203:0/586383070 <== mon.2 v2:172.21.15.203:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff7a8044620 con 0x7ff7b80d04b0 2023-08-09T13:33:45.070 INFO:teuthology.orchestra.run.smithi203.stderr:2023-08-09T13:33:45.065+0000 7ff7bfd0f640 1 -- 172.21.15.203:0/586383070 --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff7b8149240 con 0x7ff7b80d04b0 2023-08-09T13:33:45.070 INFO:teuthology.orchestra.run.smithi203.stderr:2023-08-09T13:33:45.065+0000 7ff7ae7fc640 1 -- 172.21.15.203:0/586383070 <== mon.2 v2:172.21.15.203:3301/0 4 ==== mgrmap(e 1) v1 ==== 660+0+0 (secure 0 0 0) 0x7ff7a8044a80 con 0x7ff7b80d04b0 2023-08-09T13:33:45.070 INFO:teuthology.orchestra.run.smithi203.stderr:2023-08-09T13:33:45.065+0000 7ff7ae7fc640 1 -- 172.21.15.203:0/586383070 <== mon.2 v2:172.21.15.203:3301/0 5 ==== osd_map(9..9 src has 1..9) v4 ==== 1530+0+0 (secure 0 0 0) 0x7ff7a8044d30 con 0x7ff7b80d04b0 2023-08-09T13:33:45.070 INFO:teuthology.orchestra.run.smithi203.stderr:2023-08-09T13:33:45.066+0000 7ff7bfd0f640 1 -- 172.21.15.203:0/586383070 --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff7b80739f0 con 0x7ff7b80d04b0 2023-08-09T13:33:45.071 INFO:teuthology.orchestra.run.smithi203.stderr:2023-08-09T13:33:45.068+0000 7ff7ae7fc640 1 -- 172.21.15.203:0/586383070 <== mon.2 v2:172.21.15.203:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+72429 (secure 0 0 0) 0x7ff7a8035020 con 0x7ff7b80d04b0 2023-08-09T13:33:45.092 INFO:tasks.ceph.mgr.y.smithi043.stderr:2023-08-09T13:33:45.091+0000 7fe23d40f040 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2023-08-09T13:33:45.098 INFO:tasks.ceph.mgr.x.smithi203.stderr:2023-08-09T13:33:45.096+0000 7f943a4b2040 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2023-08-09T13:33:45.148 INFO:teuthology.orchestra.run.smithi203.stderr:2023-08-09T13:33:45.146+0000 7ff7bfd0f640 1 -- 172.21.15.203:0/586383070 --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_command({"prefix": "osd new", "uuid": "a95dc7ca-9f1b-4295-af70-68a22a2d7d5e", "id": 7} v 0) v1 -- 0x7ff7b80d1020 con 0x7ff7b80d04b0 2023-08-09T13:33:45.157 INFO:teuthology.orchestra.run.smithi203.stderr:2023-08-09T13:33:45.155+0000 7ff7ae7fc640 1 -- 172.21.15.203:0/586383070 <== mon.2 v2:172.21.15.203:3301/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "a95dc7ca-9f1b-4295-af70-68a22a2d7d5e", "id": 7}]=0 v10) v1 ==== 112+0+2 (secure 0 0 0) 0x7ff7a804d440 con 0x7ff7b80d04b0 2023-08-09T13:33:45.157 INFO:teuthology.orchestra.run.smithi203.stdout:7 2023-08-09T13:33:45.159 INFO:teuthology.orchestra.run.smithi203.stderr:2023-08-09T13:33:45.157+0000 7ff7bfd0f640 1 -- 172.21.15.203:0/586383070 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7ff7b80d04b0 msgr2=0x7ff7b814b1a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:33:45.159 INFO:teuthology.orchestra.run.smithi203.stderr:2023-08-09T13:33:45.157+0000 7ff7bfd0f640 1 --2- 172.21.15.203:0/586383070 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7ff7b80d04b0 0x7ff7b814b1a0 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7ff7a80098c0 tx=0x7ff7a80046c0 comp rx=0 tx=0).stop 2023-08-09T13:33:45.160 INFO:teuthology.orchestra.run.smithi203.stderr:2023-08-09T13:33:45.157+0000 7ff7bfd0f640 1 -- 172.21.15.203:0/586383070 shutdown_connections 2023-08-09T13:33:45.160 INFO:teuthology.orchestra.run.smithi203.stderr:2023-08-09T13:33:45.157+0000 7ff7bfd0f640 1 --2- 172.21.15.203:0/586383070 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7ff7b815c550 0x7ff7b80c6fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:33:45.160 INFO:teuthology.orchestra.run.smithi203.stderr:2023-08-09T13:33:45.157+0000 7ff7bfd0f640 1 --2- 172.21.15.203:0/586383070 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7ff7b80d04b0 0x7ff7b814b1a0 unknown :-1 s=CLOSED pgs=20 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:33:45.160 INFO:teuthology.orchestra.run.smithi203.stderr:2023-08-09T13:33:45.157+0000 7ff7bfd0f640 1 --2- 172.21.15.203:0/586383070 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff7b80b4420 0x7ff7b81f1220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:33:45.160 INFO:teuthology.orchestra.run.smithi203.stderr:2023-08-09T13:33:45.157+0000 7ff7bfd0f640 1 -- 172.21.15.203:0/586383070 >> 172.21.15.203:0/586383070 conn(0x7ff7b8071520 msgr2=0x7ff7b815cac0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-09T13:33:45.160 INFO:teuthology.orchestra.run.smithi203.stderr:2023-08-09T13:33:45.157+0000 7ff7bfd0f640 1 -- 172.21.15.203:0/586383070 shutdown_connections 2023-08-09T13:33:45.160 INFO:teuthology.orchestra.run.smithi203.stderr:2023-08-09T13:33:45.157+0000 7ff7bfd0f640 1 -- 172.21.15.203:0/586383070 wait complete. 2023-08-09T13:33:45.176 INFO:tasks.ceph.osd.0:Restarting daemon 2023-08-09T13:33:45.176 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 0 2023-08-09T13:33:45.185 INFO:tasks.ceph.osd.0:Started 2023-08-09T13:33:45.185 INFO:tasks.ceph.osd.1:Restarting daemon 2023-08-09T13:33:45.186 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 1 2023-08-09T13:33:45.193 INFO:tasks.ceph.osd.1:Started 2023-08-09T13:33:45.194 INFO:tasks.ceph.osd.2:Restarting daemon 2023-08-09T13:33:45.194 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 2 2023-08-09T13:33:45.199 INFO:tasks.ceph.osd.2:Started 2023-08-09T13:33:45.199 INFO:tasks.ceph.osd.3:Restarting daemon 2023-08-09T13:33:45.200 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 3 2023-08-09T13:33:45.207 INFO:tasks.ceph.osd.3:Started 2023-08-09T13:33:45.207 INFO:tasks.ceph.osd.4:Restarting daemon 2023-08-09T13:33:45.208 DEBUG:teuthology.orchestra.run.smithi203:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 4 2023-08-09T13:33:45.216 INFO:tasks.ceph.osd.4:Started 2023-08-09T13:33:45.216 INFO:tasks.ceph.osd.5:Restarting daemon 2023-08-09T13:33:45.216 DEBUG:teuthology.orchestra.run.smithi203:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 5 2023-08-09T13:33:45.224 INFO:tasks.ceph.osd.5:Started 2023-08-09T13:33:45.224 INFO:tasks.ceph.osd.6:Restarting daemon 2023-08-09T13:33:45.224 DEBUG:teuthology.orchestra.run.smithi203:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 6 2023-08-09T13:33:45.229 INFO:tasks.ceph.osd.6:Started 2023-08-09T13:33:45.229 INFO:tasks.ceph.osd.7:Restarting daemon 2023-08-09T13:33:45.229 DEBUG:teuthology.orchestra.run.smithi203:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 7 2023-08-09T13:33:45.235 INFO:tasks.ceph.osd.7:Started 2023-08-09T13:33:45.235 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-08-09T13:33:45.478 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:33:45.476+0000 7f110ae5e640 1 Processor -- start 2023-08-09T13:33:45.478 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:33:45.477+0000 7f110ae5e640 1 -- start start 2023-08-09T13:33:45.480 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:33:45.478+0000 7f110ae5e640 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7f11040c86f0 0x7f11040c8b50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:33:45.480 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:33:45.478+0000 7f110ae5e640 1 -- --> v1:172.21.15.43:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f11040c9090 con 0x7f110406c8d0 2023-08-09T13:33:45.480 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:33:45.478+0000 7f110ae5e640 1 -- --> v1:172.21.15.203:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f110406c440 con 0x7f1104143c20 2023-08-09T13:33:45.480 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:33:45.478+0000 7f110ae5e640 1 -- --> v2:172.21.15.203:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1104144000 con 0x7f11040c86f0 2023-08-09T13:33:45.481 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:33:45.479+0000 7f11093fb640 1 --1- >> v1:172.21.15.43:6789/0 conn(0x7f110406c8d0 0x7f110406cd00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.43:6789/0 says I am v1:172.21.15.43:36018/0 (socket says 172.21.15.43:36018) 2023-08-09T13:33:45.481 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:33:45.479+0000 7f11093fb640 1 -- 172.21.15.43:0/2902460930 learned_addr learned my addr 172.21.15.43:0/2902460930 (peer_addr_for_me v1:172.21.15.43:0/0) 2023-08-09T13:33:45.481 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:33:45.479+0000 7f1103fff640 1 --2- 172.21.15.43:0/2902460930 >> v2:172.21.15.203:3300/0 conn(0x7f11040c86f0 0x7f11040c8b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:33:45.481 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:33:45.479+0000 7f11037fe640 1 -- 172.21.15.43:0/2902460930 <== mon.0 v1:172.21.15.43:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2919031587 0 0) 0x7f11040c9090 con 0x7f110406c8d0 2023-08-09T13:33:45.481 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:33:45.479+0000 7f11037fe640 1 -- 172.21.15.43:0/2902460930 --> v1:172.21.15.43:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f10f4003520 con 0x7f110406c8d0 2023-08-09T13:33:45.482 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:33:45.479+0000 7f11037fe640 1 -- 172.21.15.43:0/2902460930 <== mon.1 v1:172.21.15.203:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2372915555 0 0) 0x7f110406c440 con 0x7f1104143c20 2023-08-09T13:33:45.482 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:33:45.479+0000 7f11037fe640 1 -- 172.21.15.43:0/2902460930 --> v1:172.21.15.203:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f11040c9090 con 0x7f1104143c20 2023-08-09T13:33:45.482 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:33:45.479+0000 7f1103fff640 1 -- 172.21.15.43:0/2902460930 >> v1:172.21.15.203:6789/0 conn(0x7f1104143c20 legacy=0x7f110406f540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:33:45.482 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:33:45.479+0000 7f1103fff640 1 -- 172.21.15.43:0/2902460930 >> v1:172.21.15.43:6789/0 conn(0x7f110406c8d0 legacy=0x7f110406cd00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:33:45.482 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:33:45.479+0000 7f1103fff640 1 -- 172.21.15.43:0/2902460930 --> v2:172.21.15.203:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f110406fc50 con 0x7f11040c86f0 2023-08-09T13:33:45.483 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:33:45.480+0000 7f1103fff640 1 --2- 172.21.15.43:0/2902460930 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f11040c86f0 0x7f11040c8b50 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7f10f00097a0 tx=0x7f10f002b840 comp rx=0 tx=0).ready entity=mon.1 client_cookie=fce4fe874874bf36 server_cookie=0 in_seq=0 out_seq=0 2023-08-09T13:33:45.483 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:33:45.481+0000 7f11037fe640 1 -- 172.21.15.43:0/2902460930 <== mon.1 v2:172.21.15.203:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f10f002e070 con 0x7f11040c86f0 2023-08-09T13:33:45.483 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:33:45.481+0000 7f11037fe640 1 -- 172.21.15.43:0/2902460930 <== mon.1 v2:172.21.15.203:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f10f00315a0 con 0x7f11040c86f0 2023-08-09T13:33:45.483 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:33:45.481+0000 7f11037fe640 1 -- 172.21.15.43:0/2902460930 <== mon.1 v2:172.21.15.203:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f10f0031930 con 0x7f11040c86f0 2023-08-09T13:33:45.484 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:33:45.481+0000 7f110ae5e640 1 -- 172.21.15.43:0/2902460930 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f11040c86f0 msgr2=0x7f11040c8b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:33:45.484 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:33:45.481+0000 7f110ae5e640 1 --2- 172.21.15.43:0/2902460930 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f11040c86f0 0x7f11040c8b50 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7f10f00097a0 tx=0x7f10f002b840 comp rx=0 tx=0).stop 2023-08-09T13:33:45.484 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:33:45.481+0000 7f110ae5e640 1 -- 172.21.15.43:0/2902460930 shutdown_connections 2023-08-09T13:33:45.484 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:33:45.481+0000 7f110ae5e640 1 --2- 172.21.15.43:0/2902460930 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f11040c86f0 0x7f11040c8b50 unknown :-1 s=CLOSED pgs=25 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:33:45.484 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:33:45.481+0000 7f110ae5e640 1 -- 172.21.15.43:0/2902460930 >> 172.21.15.43:0/2902460930 conn(0x7f11040ce0a0 msgr2=0x7f11040ce480 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-09T13:33:45.484 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:33:45.482+0000 7f110ae5e640 1 -- 172.21.15.43:0/2902460930 shutdown_connections 2023-08-09T13:33:45.484 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:33:45.482+0000 7f110ae5e640 1 -- 172.21.15.43:0/2902460930 wait complete. 2023-08-09T13:33:45.485 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:33:45.483+0000 7f110ae5e640 1 Processor -- start 2023-08-09T13:33:45.485 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:33:45.483+0000 7f110ae5e640 1 -- start start 2023-08-09T13:33:45.486 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:33:45.484+0000 7f110ae5e640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f110406c8d0 0x7f11041442b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:33:45.486 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:33:45.484+0000 7f110ae5e640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f11040c86f0 0x7f11040c6b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:33:45.486 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:33:45.484+0000 7f110ae5e640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f1104143c20 0x7f1104138520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:33:45.487 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:33:45.485+0000 7f110ae5e640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f11041447f0 con 0x7f1104143c20 2023-08-09T13:33:45.487 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:33:45.485+0000 7f110ae5e640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f11040ab960 con 0x7f11040c86f0 2023-08-09T13:33:45.487 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:33:45.485+0000 7f110ae5e640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f11040abc00 con 0x7f110406c8d0 2023-08-09T13:33:45.487 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:33:45.485+0000 7f11093fb640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f1104143c20 0x7f1104138520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:33:45.487 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:33:45.485+0000 7f11093fb640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f1104143c20 0x7f1104138520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:52548/0 (socket says 172.21.15.43:52548) 2023-08-09T13:33:45.487 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:33:45.485+0000 7f11093fb640 1 -- 172.21.15.43:0/3993218446 learned_addr learned my addr 172.21.15.43:0/3993218446 (peer_addr_for_me v2:172.21.15.43:0/0) 2023-08-09T13:33:45.487 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:33:45.485+0000 7f1108bfa640 1 --2- 172.21.15.43:0/3993218446 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f110406c8d0 0x7f11041442b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:33:45.487 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:33:45.485+0000 7f1103fff640 1 --2- 172.21.15.43:0/3993218446 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f11040c86f0 0x7f11040c6b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:33:45.489 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:33:45.485+0000 7f11093fb640 1 -- 172.21.15.43:0/3993218446 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f110406c8d0 msgr2=0x7f11041442b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:33:45.489 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:33:45.485+0000 7f11093fb640 1 --2- 172.21.15.43:0/3993218446 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f110406c8d0 0x7f11041442b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:33:45.489 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:33:45.485+0000 7f11093fb640 1 -- 172.21.15.43:0/3993218446 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f11040c86f0 msgr2=0x7f11040c6b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:33:45.489 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:33:45.485+0000 7f11093fb640 1 --2- 172.21.15.43:0/3993218446 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f11040c86f0 0x7f11040c6b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:33:45.489 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:33:45.485+0000 7f11093fb640 1 -- 172.21.15.43:0/3993218446 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f10f0009400 con 0x7f1104143c20 2023-08-09T13:33:45.489 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:33:45.485+0000 7f1108bfa640 1 --2- 172.21.15.43:0/3993218446 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f110406c8d0 0x7f11041442b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-08-09T13:33:45.489 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:33:45.485+0000 7f1103fff640 1 --2- 172.21.15.43:0/3993218446 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f11040c86f0 0x7f11040c6b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-08-09T13:33:45.490 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:33:45.486+0000 7f11093fb640 1 --2- 172.21.15.43:0/3993218446 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f1104143c20 0x7f1104138520 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7f10fc00ca80 tx=0x7f10fc00cf40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-09T13:33:45.490 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:33:45.486+0000 7f11017fa640 1 -- 172.21.15.43:0/3993218446 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f10fc00abd0 con 0x7f1104143c20 2023-08-09T13:33:45.490 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:33:45.486+0000 7f11017fa640 1 -- 172.21.15.43:0/3993218446 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f10fc00ad30 con 0x7f1104143c20 2023-08-09T13:33:45.490 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:33:45.486+0000 7f110ae5e640 1 -- 172.21.15.43:0/3993218446 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f11040abee0 con 0x7f1104143c20 2023-08-09T13:33:45.490 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:33:45.486+0000 7f11017fa640 1 -- 172.21.15.43:0/3993218446 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f10fc00aea0 con 0x7f1104143c20 2023-08-09T13:33:45.490 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:33:45.486+0000 7f110ae5e640 1 -- 172.21.15.43:0/3993218446 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f11040ad870 con 0x7f1104143c20 2023-08-09T13:33:45.490 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:33:45.486+0000 7f11017fa640 1 -- 172.21.15.43:0/3993218446 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 1) v1 ==== 660+0+0 (secure 0 0 0) 0x7f10fc012920 con 0x7f1104143c20 2023-08-09T13:33:45.490 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:33:45.486+0000 7f11017fa640 1 -- 172.21.15.43:0/3993218446 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(10..10 src has 1..10) v4 ==== 1645+0+0 (secure 0 0 0) 0x7f10fc012bd0 con 0x7f1104143c20 2023-08-09T13:33:45.491 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:33:45.487+0000 7f110ae5e640 1 -- 172.21.15.43:0/3993218446 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f11040739f0 con 0x7f1104143c20 2023-08-09T13:33:45.491 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:33:45.490+0000 7f11017fa640 1 -- 172.21.15.43:0/3993218446 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+72429 (secure 0 0 0) 0x7f10fc017070 con 0x7f1104143c20 2023-08-09T13:33:45.556 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:33:45.555+0000 7f110ae5e640 1 -- 172.21.15.43:0/3993218446 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f11040cec90 con 0x7f1104143c20 2023-08-09T13:33:45.557 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:33:45.555+0000 7f11017fa640 1 -- 172.21.15.43:0/3993218446 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v10) v1 ==== 74+0+7057 (secure 0 0 0) 0x7f10fc01c4f0 con 0x7f1104143c20 2023-08-09T13:33:45.557 INFO:teuthology.orchestra.run.smithi043.stdout: 2023-08-09T13:33:45.557 INFO:teuthology.orchestra.run.smithi043.stdout:{"epoch":10,"fsid":"daac87ee-1337-4250-a955-aea4e408a39b","created":"2023-08-09T13:33:41.701198+0000","modified":"2023-08-09T13:33:45.148904+0000","last_up_change":"0.000000","last_in_change":"2023-08-09T13:33:45.148904+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":2,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":0,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[],"osds":[{"osd":0,"uuid":"cbd49ecd-e896-4fbd-864e-5c37a829b165","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":1,"uuid":"75c7e0fe-1ec7-4fe8-a2f2-3661a65b8500","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":2,"uuid":"bb5c41e9-7f5f-4125-9b01-3bb8bfc2ea83","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":3,"uuid":"bfe4c8b4-694a-465c-87e6-2b2aab852613","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":4,"uuid":"fa2e8a77-aba1-4756-853e-0d174a5bdeba","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":5,"uuid":"81811e4b-4d24-4f51-a82c-f1d700a2a00d","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":6,"uuid":"307bac4d-6a5b-443d-af75-56d695d7674f","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":7,"uuid":"a95dc7ca-9f1b-4295-af70-68a22a2d7d5e","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-08-09T13:33:45.558 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:33:45.557+0000 7f110ae5e640 1 -- 172.21.15.43:0/3993218446 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f1104143c20 msgr2=0x7f1104138520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:33:45.558 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:33:45.557+0000 7f110ae5e640 1 --2- 172.21.15.43:0/3993218446 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f1104143c20 0x7f1104138520 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7f10fc00ca80 tx=0x7f10fc00cf40 comp rx=0 tx=0).stop 2023-08-09T13:33:45.559 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:33:45.557+0000 7f110ae5e640 1 -- 172.21.15.43:0/3993218446 shutdown_connections 2023-08-09T13:33:45.559 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:33:45.557+0000 7f110ae5e640 1 --2- 172.21.15.43:0/3993218446 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f1104143c20 0x7f1104138520 unknown :-1 s=CLOSED pgs=19 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:33:45.559 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:33:45.557+0000 7f110ae5e640 1 --2- 172.21.15.43:0/3993218446 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f11040c86f0 0x7f11040c6b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:33:45.559 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:33:45.557+0000 7f110ae5e640 1 --2- 172.21.15.43:0/3993218446 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f110406c8d0 0x7f11041442b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:33:45.559 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:33:45.557+0000 7f110ae5e640 1 -- 172.21.15.43:0/3993218446 >> 172.21.15.43:0/3993218446 conn(0x7f11040ce0a0 msgr2=0x7f1104067560 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-09T13:33:45.559 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:33:45.557+0000 7f110ae5e640 1 -- 172.21.15.43:0/3993218446 shutdown_connections 2023-08-09T13:33:45.560 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:33:45.557+0000 7f110ae5e640 1 -- 172.21.15.43:0/3993218446 wait complete. 2023-08-09T13:33:45.574 INFO:tasks.ceph.ceph_manager.ceph:[] 2023-08-09T13:33:45.574 INFO:tasks.ceph:Waiting for OSDs to come up 2023-08-09T13:33:45.583 INFO:tasks.ceph.mgr.x.smithi203.stderr:2023-08-09T13:33:45.581+0000 7f943a4b2040 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2023-08-09T13:33:45.620 INFO:tasks.ceph.mgr.y.smithi043.stderr:2023-08-09T13:33:45.619+0000 7fe23d40f040 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2023-08-09T13:33:45.803 INFO:tasks.ceph.mgr.x.smithi203.stderr:2023-08-09T13:33:45.801+0000 7f943a4b2040 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2023-08-09T13:33:45.848 INFO:tasks.ceph.mgr.y.smithi043.stderr:2023-08-09T13:33:45.847+0000 7fe23d40f040 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2023-08-09T13:33:45.863 INFO:tasks.ceph.mgr.x.smithi203.stderr:2023-08-09T13:33:45.860+0000 7f943a4b2040 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2023-08-09T13:33:45.915 INFO:tasks.ceph.osd.4.smithi203.stderr:2023-08-09T13:33:45.913+0000 7f21c8411500 -1 Falling back to public interface 2023-08-09T13:33:45.915 INFO:tasks.ceph.mgr.y.smithi043.stderr:2023-08-09T13:33:45.914+0000 7fe23d40f040 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2023-08-09T13:33:45.916 INFO:tasks.ceph.osd.0.smithi043.stderr:2023-08-09T13:33:45.915+0000 7f231c869500 -1 Falling back to public interface 2023-08-09T13:33:45.916 INFO:tasks.ceph.mgr.x.smithi203.stderr:2023-08-09T13:33:45.914+0000 7f943a4b2040 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2023-08-09T13:33:45.926 INFO:tasks.ceph.osd.7.smithi203.stderr:2023-08-09T13:33:45.924+0000 7f07aad74500 -1 Falling back to public interface 2023-08-09T13:33:45.929 INFO:tasks.ceph.osd.1.smithi043.stderr:2023-08-09T13:33:45.927+0000 7f6caaa22500 -1 Falling back to public interface 2023-08-09T13:33:45.932 INFO:tasks.ceph.osd.2.smithi043.stderr:2023-08-09T13:33:45.930+0000 7fa72a80f500 -1 Falling back to public interface 2023-08-09T13:33:45.932 INFO:tasks.ceph.osd.3.smithi043.stderr:2023-08-09T13:33:45.930+0000 7f7c6880c500 -1 Falling back to public interface 2023-08-09T13:33:45.942 INFO:tasks.ceph.osd.5.smithi203.stderr:2023-08-09T13:33:45.940+0000 7ff3c3e1d500 -1 Falling back to public interface 2023-08-09T13:33:45.951 INFO:tasks.ceph.osd.6.smithi203.stderr:2023-08-09T13:33:45.949+0000 7fd591621500 -1 Falling back to public interface 2023-08-09T13:33:45.978 INFO:tasks.ceph.mgr.y.smithi043.stderr:2023-08-09T13:33:45.976+0000 7fe23d40f040 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2023-08-09T13:33:45.983 INFO:tasks.ceph.mgr.x.smithi203.stderr:2023-08-09T13:33:45.981+0000 7f943a4b2040 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2023-08-09T13:33:46.042 INFO:tasks.ceph.mgr.y.smithi043.stderr:2023-08-09T13:33:46.040+0000 7fe23d40f040 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2023-08-09T13:33:46.284 INFO:tasks.ceph.mgr.x.smithi203.stderr:2023-08-09T13:33:46.282+0000 7f943a4b2040 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2023-08-09T13:33:46.344 INFO:tasks.ceph.mgr.x.smithi203.stderr:2023-08-09T13:33:46.341+0000 7f943a4b2040 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2023-08-09T13:33:46.358 INFO:tasks.ceph.mgr.y.smithi043.stderr:2023-08-09T13:33:46.357+0000 7fe23d40f040 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2023-08-09T13:33:46.379 DEBUG:teuthology.orchestra.run.smithi043:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph --cluster ceph osd dump --format=json 2023-08-09T13:33:46.398 INFO:tasks.ceph.mgr.x.smithi203.stderr:2023-08-09T13:33:46.396+0000 7f943a4b2040 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2023-08-09T13:33:46.421 INFO:tasks.ceph.mgr.y.smithi043.stderr:2023-08-09T13:33:46.420+0000 7fe23d40f040 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2023-08-09T13:33:46.479 INFO:tasks.ceph.mgr.y.smithi043.stderr:2023-08-09T13:33:46.478+0000 7fe23d40f040 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2023-08-09T13:33:46.561 INFO:teuthology.misc.health.smithi043.stderr:2023-08-09T13:33:46.559+0000 7f1659acc640 1 Processor -- start 2023-08-09T13:33:46.561 INFO:teuthology.misc.health.smithi043.stderr:2023-08-09T13:33:46.559+0000 7f1659acc640 1 -- start start 2023-08-09T13:33:46.562 INFO:teuthology.misc.health.smithi043.stderr:2023-08-09T13:33:46.560+0000 7f1659acc640 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7f165415c660 0x7f1654160b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:33:46.562 INFO:teuthology.misc.health.smithi043.stderr:2023-08-09T13:33:46.561+0000 7f1659acc640 1 -- --> v1:172.21.15.43:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1654160120 con 0x7f16540d04d0 2023-08-09T13:33:46.562 INFO:teuthology.misc.health.smithi043.stderr:2023-08-09T13:33:46.561+0000 7f1659acc640 1 -- --> v1:172.21.15.203:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f16541606e0 con 0x7f16540d2810 2023-08-09T13:33:46.562 INFO:teuthology.misc.health.smithi043.stderr:2023-08-09T13:33:46.561+0000 7f1659acc640 1 -- --> v2:172.21.15.203:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f165415ca40 con 0x7f165415c660 2023-08-09T13:33:46.563 INFO:tasks.ceph.mgr.x.smithi203.stderr:2023-08-09T13:33:46.560+0000 7f943a4b2040 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2023-08-09T13:33:46.564 INFO:teuthology.misc.health.smithi043.stderr:2023-08-09T13:33:46.561+0000 7f164bfff640 1 --1- >> v1:172.21.15.43:6789/0 conn(0x7f16540d04d0 0x7f16540d2100 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.43:6789/0 says I am v1:172.21.15.43:36028/0 (socket says 172.21.15.43:36028) 2023-08-09T13:33:46.565 INFO:teuthology.misc.health.smithi043.stderr:2023-08-09T13:33:46.561+0000 7f164bfff640 1 -- 172.21.15.43:0/3327515098 learned_addr learned my addr 172.21.15.43:0/3327515098 (peer_addr_for_me v1:172.21.15.43:0/0) 2023-08-09T13:33:46.565 INFO:teuthology.misc.health.smithi043.stderr:2023-08-09T13:33:46.561+0000 7f16537fe640 1 --2- 172.21.15.43:0/3327515098 >> v2:172.21.15.203:3300/0 conn(0x7f165415c660 0x7f1654160b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:33:46.565 INFO:teuthology.misc.health.smithi043.stderr:2023-08-09T13:33:46.561+0000 7f1652ffd640 1 -- 172.21.15.43:0/3327515098 <== mon.0 v1:172.21.15.43:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2441485238 0 0) 0x7f1654160120 con 0x7f16540d04d0 2023-08-09T13:33:46.565 INFO:teuthology.misc.health.smithi043.stderr:2023-08-09T13:33:46.561+0000 7f1652ffd640 1 -- 172.21.15.43:0/3327515098 --> v1:172.21.15.43:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f1640003520 con 0x7f16540d04d0 2023-08-09T13:33:46.565 INFO:teuthology.misc.health.smithi043.stderr:2023-08-09T13:33:46.562+0000 7f1652ffd640 1 -- 172.21.15.43:0/3327515098 <== mon.0 v1:172.21.15.43:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3007425462 0 0) 0x7f1640003520 con 0x7f16540d04d0 2023-08-09T13:33:46.565 INFO:teuthology.misc.health.smithi043.stderr:2023-08-09T13:33:46.562+0000 7f1652ffd640 1 -- 172.21.15.43:0/3327515098 >> v2:172.21.15.203:3300/0 conn(0x7f165415c660 msgr2=0x7f1654160b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-09T13:33:46.565 INFO:teuthology.misc.health.smithi043.stderr:2023-08-09T13:33:46.562+0000 7f1652ffd640 1 --2- 172.21.15.43:0/3327515098 >> v2:172.21.15.203:3300/0 conn(0x7f165415c660 0x7f1654160b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:33:46.566 INFO:teuthology.misc.health.smithi043.stderr:2023-08-09T13:33:46.562+0000 7f1652ffd640 1 -- 172.21.15.43:0/3327515098 >> v1:172.21.15.203:6789/0 conn(0x7f16540d2810 legacy=0x7f16540b12d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:33:46.566 INFO:teuthology.misc.health.smithi043.stderr:2023-08-09T13:33:46.562+0000 7f1652ffd640 1 -- 172.21.15.43:0/3327515098 --> v1:172.21.15.43:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f16540d0ee0 con 0x7f16540d04d0 2023-08-09T13:33:46.566 INFO:teuthology.misc.health.smithi043.stderr:2023-08-09T13:33:46.562+0000 7f1652ffd640 1 -- 172.21.15.43:0/3327515098 <== mon.0 v1:172.21.15.43:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1285150740 0 0) 0x7f163c001670 con 0x7f16540d04d0 2023-08-09T13:33:46.566 INFO:teuthology.misc.health.smithi043.stderr:2023-08-09T13:33:46.562+0000 7f1652ffd640 1 -- 172.21.15.43:0/3327515098 >> v1:172.21.15.43:6789/0 conn(0x7f16540d04d0 legacy=0x7f16540d2100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:33:46.566 INFO:teuthology.misc.health.smithi043.stderr:2023-08-09T13:33:46.562+0000 7f1652ffd640 1 --2- 172.21.15.43:0/3327515098 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f1640003970 0x7f1640003d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:33:46.566 INFO:teuthology.misc.health.smithi043.stderr:2023-08-09T13:33:46.562+0000 7f1652ffd640 1 --2- 172.21.15.43:0/3327515098 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f16400047c0 0x7f1640004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:33:46.566 INFO:teuthology.misc.health.smithi043.stderr:2023-08-09T13:33:46.562+0000 7f1652ffd640 1 --2- 172.21.15.43:0/3327515098 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f1640005180 0x7f164000b980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:33:46.567 INFO:teuthology.misc.health.smithi043.stderr:2023-08-09T13:33:46.562+0000 7f1652ffd640 1 -- 172.21.15.43:0/3327515098 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f165415ca40 con 0x7f1640005180 2023-08-09T13:33:46.567 INFO:teuthology.misc.health.smithi043.stderr:2023-08-09T13:33:46.562+0000 7f1652ffd640 1 -- 172.21.15.43:0/3327515098 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f164000c120 con 0x7f1640003970 2023-08-09T13:33:46.567 INFO:teuthology.misc.health.smithi043.stderr:2023-08-09T13:33:46.562+0000 7f1652ffd640 1 -- 172.21.15.43:0/3327515098 --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f164000c2e0 con 0x7f16400047c0 2023-08-09T13:33:46.567 INFO:teuthology.misc.health.smithi043.stderr:2023-08-09T13:33:46.562+0000 7f1653fff640 1 --2- 172.21.15.43:0/3327515098 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f1640005180 0x7f164000b980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:33:46.567 INFO:teuthology.misc.health.smithi043.stderr:2023-08-09T13:33:46.563+0000 7f16537fe640 1 --2- 172.21.15.43:0/3327515098 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f1640003970 0x7f1640003d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:33:46.567 INFO:teuthology.misc.health.smithi043.stderr:2023-08-09T13:33:46.563+0000 7f164bfff640 1 --2- 172.21.15.43:0/3327515098 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f16400047c0 0x7f1640004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:33:46.567 INFO:teuthology.misc.health.smithi043.stderr:2023-08-09T13:33:46.563+0000 7f1653fff640 1 -- 172.21.15.43:0/3327515098 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f16400047c0 msgr2=0x7f1640004c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-09T13:33:46.567 INFO:teuthology.misc.health.smithi043.stderr:2023-08-09T13:33:46.563+0000 7f1653fff640 1 --2- 172.21.15.43:0/3327515098 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f16400047c0 0x7f1640004c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:33:46.568 INFO:teuthology.misc.health.smithi043.stderr:2023-08-09T13:33:46.563+0000 7f1653fff640 1 -- 172.21.15.43:0/3327515098 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f1640003970 msgr2=0x7f1640003d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-09T13:33:46.568 INFO:teuthology.misc.health.smithi043.stderr:2023-08-09T13:33:46.563+0000 7f1653fff640 1 --2- 172.21.15.43:0/3327515098 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f1640003970 0x7f1640003d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:33:46.568 INFO:teuthology.misc.health.smithi043.stderr:2023-08-09T13:33:46.563+0000 7f1653fff640 1 -- 172.21.15.43:0/3327515098 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1640003520 con 0x7f1640005180 2023-08-09T13:33:46.570 INFO:teuthology.misc.health.smithi043.stderr:2023-08-09T13:33:46.564+0000 7f1653fff640 1 -- 172.21.15.43:0/3327515098 reap_dead start 2023-08-09T13:33:46.570 INFO:teuthology.misc.health.smithi043.stderr:2023-08-09T13:33:46.564+0000 7f1653fff640 1 --2- 172.21.15.43:0/3327515098 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f1640005180 0x7f164000b980 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7f164c001e50 tx=0x7f16400057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3fe7c0bd109f7421 server_cookie=0 in_seq=0 out_seq=0 2023-08-09T13:33:46.570 INFO:teuthology.misc.health.smithi043.stderr:2023-08-09T13:33:46.564+0000 7f1652ffd640 1 -- 172.21.15.43:0/3327515098 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f164c003b70 con 0x7f1640005180 2023-08-09T13:33:46.570 INFO:teuthology.misc.health.smithi043.stderr:2023-08-09T13:33:46.564+0000 7f1652ffd640 1 -- 172.21.15.43:0/3327515098 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f164c003d10 con 0x7f1640005180 2023-08-09T13:33:46.570 INFO:teuthology.misc.health.smithi043.stderr:2023-08-09T13:33:46.564+0000 7f1652ffd640 1 -- 172.21.15.43:0/3327515098 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f164c005310 con 0x7f1640005180 2023-08-09T13:33:46.570 INFO:teuthology.misc.health.smithi043.stderr:2023-08-09T13:33:46.565+0000 7f1659acc640 1 -- 172.21.15.43:0/3327515098 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f1640005180 msgr2=0x7f164000b980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:33:46.571 INFO:teuthology.misc.health.smithi043.stderr:2023-08-09T13:33:46.565+0000 7f1659acc640 1 --2- 172.21.15.43:0/3327515098 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f1640005180 0x7f164000b980 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7f164c001e50 tx=0x7f16400057f0 comp rx=0 tx=0).stop 2023-08-09T13:33:46.571 INFO:teuthology.misc.health.smithi043.stderr:2023-08-09T13:33:46.565+0000 7f1659acc640 1 -- 172.21.15.43:0/3327515098 shutdown_connections 2023-08-09T13:33:46.571 INFO:teuthology.misc.health.smithi043.stderr:2023-08-09T13:33:46.565+0000 7f1659acc640 1 --2- 172.21.15.43:0/3327515098 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f1640005180 0x7f164000b980 unknown :-1 s=CLOSED pgs=21 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:33:46.571 INFO:teuthology.misc.health.smithi043.stderr:2023-08-09T13:33:46.565+0000 7f1659acc640 1 -- 172.21.15.43:0/3327515098 >> 172.21.15.43:0/3327515098 conn(0x7f165408be70 msgr2=0x7f165409b790 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-09T13:33:46.571 INFO:teuthology.misc.health.smithi043.stderr:2023-08-09T13:33:46.565+0000 7f1659acc640 1 -- 172.21.15.43:0/3327515098 shutdown_connections 2023-08-09T13:33:46.571 INFO:teuthology.misc.health.smithi043.stderr:2023-08-09T13:33:46.565+0000 7f1659acc640 1 -- 172.21.15.43:0/3327515098 wait complete. 2023-08-09T13:33:46.571 INFO:teuthology.misc.health.smithi043.stderr:2023-08-09T13:33:46.566+0000 7f1659acc640 1 Processor -- start 2023-08-09T13:33:46.571 INFO:teuthology.misc.health.smithi043.stderr:2023-08-09T13:33:46.567+0000 7f1659acc640 1 -- start start 2023-08-09T13:33:46.572 INFO:teuthology.misc.health.smithi043.stderr:2023-08-09T13:33:46.568+0000 7f1659acc640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f1640005180 0x7f1654159a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:33:46.572 INFO:teuthology.misc.health.smithi043.stderr:2023-08-09T13:33:46.568+0000 7f1659acc640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f16540b1140 0x7f16540b4fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:33:46.572 INFO:teuthology.misc.health.smithi043.stderr:2023-08-09T13:33:46.568+0000 7f1659acc640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f165415edf0 0x7f165415f270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:33:46.572 INFO:teuthology.misc.health.smithi043.stderr:2023-08-09T13:33:46.568+0000 7f1659acc640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f165415f7b0 con 0x7f165415edf0 2023-08-09T13:33:46.572 INFO:teuthology.misc.health.smithi043.stderr:2023-08-09T13:33:46.568+0000 7f1659acc640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f16540c9520 con 0x7f1640005180 2023-08-09T13:33:46.572 INFO:teuthology.misc.health.smithi043.stderr:2023-08-09T13:33:46.568+0000 7f1659acc640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1654159f80 con 0x7f16540b1140 2023-08-09T13:33:46.572 INFO:teuthology.misc.health.smithi043.stderr:2023-08-09T13:33:46.568+0000 7f1653fff640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f165415edf0 0x7f165415f270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:33:46.572 INFO:teuthology.misc.health.smithi043.stderr:2023-08-09T13:33:46.568+0000 7f1653fff640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f165415edf0 0x7f165415f270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:52566/0 (socket says 172.21.15.43:52566) 2023-08-09T13:33:46.573 INFO:teuthology.misc.health.smithi043.stderr:2023-08-09T13:33:46.568+0000 7f1653fff640 1 -- 172.21.15.43:0/3577415060 learned_addr learned my addr 172.21.15.43:0/3577415060 (peer_addr_for_me v2:172.21.15.43:0/0) 2023-08-09T13:33:46.573 INFO:teuthology.misc.health.smithi043.stderr:2023-08-09T13:33:46.568+0000 7f164bfff640 1 --2- 172.21.15.43:0/3577415060 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f16540b1140 0x7f16540b4fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:33:46.573 INFO:teuthology.misc.health.smithi043.stderr:2023-08-09T13:33:46.568+0000 7f16537fe640 1 --2- 172.21.15.43:0/3577415060 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f1640005180 0x7f1654159a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:33:46.573 INFO:teuthology.misc.health.smithi043.stderr:2023-08-09T13:33:46.568+0000 7f1653fff640 1 -- 172.21.15.43:0/3577415060 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f16540b1140 msgr2=0x7f16540b4fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:33:46.573 INFO:teuthology.misc.health.smithi043.stderr:2023-08-09T13:33:46.568+0000 7f1653fff640 1 --2- 172.21.15.43:0/3577415060 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f16540b1140 0x7f16540b4fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:33:46.573 INFO:teuthology.misc.health.smithi043.stderr:2023-08-09T13:33:46.568+0000 7f1653fff640 1 -- 172.21.15.43:0/3577415060 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f1640005180 msgr2=0x7f1654159a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:33:46.573 INFO:teuthology.misc.health.smithi043.stderr:2023-08-09T13:33:46.568+0000 7f1653fff640 1 --2- 172.21.15.43:0/3577415060 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f1640005180 0x7f1654159a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:33:46.574 INFO:teuthology.misc.health.smithi043.stderr:2023-08-09T13:33:46.568+0000 7f1653fff640 1 -- 172.21.15.43:0/3577415060 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f16540b1690 con 0x7f165415edf0 2023-08-09T13:33:46.574 INFO:teuthology.misc.health.smithi043.stderr:2023-08-09T13:33:46.569+0000 7f1653fff640 1 --2- 172.21.15.43:0/3577415060 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f165415edf0 0x7f165415f270 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7f164c02c060 tx=0x7f164c0014a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-09T13:33:46.575 INFO:teuthology.misc.health.smithi043.stderr:2023-08-09T13:33:46.569+0000 7f1650ff9640 1 -- 172.21.15.43:0/3577415060 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f164c006d50 con 0x7f165415edf0 2023-08-09T13:33:46.575 INFO:teuthology.misc.health.smithi043.stderr:2023-08-09T13:33:46.569+0000 7f1650ff9640 1 -- 172.21.15.43:0/3577415060 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f164c007030 con 0x7f165415edf0 2023-08-09T13:33:46.575 INFO:teuthology.misc.health.smithi043.stderr:2023-08-09T13:33:46.569+0000 7f1650ff9640 1 -- 172.21.15.43:0/3577415060 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f164c0073e0 con 0x7f165415edf0 2023-08-09T13:33:46.575 INFO:teuthology.misc.health.smithi043.stderr:2023-08-09T13:33:46.569+0000 7f1659acc640 1 -- 172.21.15.43:0/3577415060 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f16540b7e30 con 0x7f165415edf0 2023-08-09T13:33:46.575 INFO:teuthology.misc.health.smithi043.stderr:2023-08-09T13:33:46.569+0000 7f1659acc640 1 -- 172.21.15.43:0/3577415060 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f16541e6930 con 0x7f165415edf0 2023-08-09T13:33:46.575 INFO:teuthology.misc.health.smithi043.stderr:2023-08-09T13:33:46.569+0000 7f1650ff9640 1 -- 172.21.15.43:0/3577415060 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 1) v1 ==== 660+0+0 (secure 0 0 0) 0x7f164c007540 con 0x7f165415edf0 2023-08-09T13:33:46.576 INFO:teuthology.misc.health.smithi043.stderr:2023-08-09T13:33:46.570+0000 7f1650ff9640 1 -- 172.21.15.43:0/3577415060 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(10..10 src has 1..10) v4 ==== 1645+0+0 (secure 0 0 0) 0x7f164c0077f0 con 0x7f165415edf0 2023-08-09T13:33:46.576 INFO:teuthology.misc.health.smithi043.stderr:2023-08-09T13:33:46.570+0000 7f1659acc640 1 -- 172.21.15.43:0/3577415060 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f16540b8350 con 0x7f165415edf0 2023-08-09T13:33:46.576 INFO:teuthology.misc.health.smithi043.stderr:2023-08-09T13:33:46.573+0000 7f1650ff9640 1 -- 172.21.15.43:0/3577415060 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+72429 (secure 0 0 0) 0x7f164c001a70 con 0x7f165415edf0 2023-08-09T13:33:46.613 INFO:tasks.ceph.mgr.x.smithi203.stderr:2023-08-09T13:33:46.611+0000 7f943a4b2040 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2023-08-09T13:33:46.641 INFO:teuthology.misc.health.smithi043.stderr:2023-08-09T13:33:46.640+0000 7f1659acc640 1 -- 172.21.15.43:0/3577415060 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f16541e6b50 con 0x7f165415edf0 2023-08-09T13:33:46.642 INFO:teuthology.misc.health.smithi043.stderr:2023-08-09T13:33:46.641+0000 7f1650ff9640 1 -- 172.21.15.43:0/3577415060 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v10) v1 ==== 74+0+7057 (secure 0 0 0) 0x7f164c039820 con 0x7f165415edf0 2023-08-09T13:33:46.643 INFO:teuthology.misc.health.smithi043.stdout: 2023-08-09T13:33:46.643 INFO:teuthology.misc.health.smithi043.stdout:{"epoch":10,"fsid":"daac87ee-1337-4250-a955-aea4e408a39b","created":"2023-08-09T13:33:41.701198+0000","modified":"2023-08-09T13:33:45.148904+0000","last_up_change":"0.000000","last_in_change":"2023-08-09T13:33:45.148904+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":2,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":0,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[],"osds":[{"osd":0,"uuid":"cbd49ecd-e896-4fbd-864e-5c37a829b165","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":1,"uuid":"75c7e0fe-1ec7-4fe8-a2f2-3661a65b8500","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":2,"uuid":"bb5c41e9-7f5f-4125-9b01-3bb8bfc2ea83","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":3,"uuid":"bfe4c8b4-694a-465c-87e6-2b2aab852613","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":4,"uuid":"fa2e8a77-aba1-4756-853e-0d174a5bdeba","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":5,"uuid":"81811e4b-4d24-4f51-a82c-f1d700a2a00d","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":6,"uuid":"307bac4d-6a5b-443d-af75-56d695d7674f","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":7,"uuid":"a95dc7ca-9f1b-4295-af70-68a22a2d7d5e","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-08-09T13:33:46.644 INFO:teuthology.misc.health.smithi043.stderr:2023-08-09T13:33:46.642+0000 7f1659acc640 1 -- 172.21.15.43:0/3577415060 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f165415edf0 msgr2=0x7f165415f270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:33:46.644 INFO:teuthology.misc.health.smithi043.stderr:2023-08-09T13:33:46.642+0000 7f1659acc640 1 --2- 172.21.15.43:0/3577415060 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f165415edf0 0x7f165415f270 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7f164c02c060 tx=0x7f164c0014a0 comp rx=0 tx=0).stop 2023-08-09T13:33:46.644 INFO:teuthology.misc.health.smithi043.stderr:2023-08-09T13:33:46.642+0000 7f1659acc640 1 -- 172.21.15.43:0/3577415060 shutdown_connections 2023-08-09T13:33:46.644 INFO:teuthology.misc.health.smithi043.stderr:2023-08-09T13:33:46.642+0000 7f1659acc640 1 --2- 172.21.15.43:0/3577415060 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f165415edf0 0x7f165415f270 unknown :-1 s=CLOSED pgs=22 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:33:46.645 INFO:teuthology.misc.health.smithi043.stderr:2023-08-09T13:33:46.642+0000 7f1659acc640 1 --2- 172.21.15.43:0/3577415060 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f16540b1140 0x7f16540b4fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:33:46.645 INFO:teuthology.misc.health.smithi043.stderr:2023-08-09T13:33:46.642+0000 7f1659acc640 1 --2- 172.21.15.43:0/3577415060 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f1640005180 0x7f1654159a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:33:46.645 INFO:teuthology.misc.health.smithi043.stderr:2023-08-09T13:33:46.642+0000 7f1659acc640 1 -- 172.21.15.43:0/3577415060 >> 172.21.15.43:0/3577415060 conn(0x7f165408be70 msgr2=0x7f165406b490 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-09T13:33:46.645 INFO:teuthology.misc.health.smithi043.stderr:2023-08-09T13:33:46.643+0000 7f1659acc640 1 -- 172.21.15.43:0/3577415060 shutdown_connections 2023-08-09T13:33:46.645 INFO:teuthology.misc.health.smithi043.stderr:2023-08-09T13:33:46.643+0000 7f1659acc640 1 -- 172.21.15.43:0/3577415060 wait complete. 2023-08-09T13:33:46.654 INFO:tasks.ceph.mgr.y.smithi043.stderr:2023-08-09T13:33:46.653+0000 7fe23d40f040 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2023-08-09T13:33:46.656 DEBUG:teuthology.misc:0 of 8 OSDs are up 2023-08-09T13:33:46.672 INFO:tasks.ceph.mgr.x.smithi203.stderr:2023-08-09T13:33:46.670+0000 7f943a4b2040 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2023-08-09T13:33:46.708 INFO:tasks.ceph.mgr.y.smithi043.stderr:2023-08-09T13:33:46.706+0000 7fe23d40f040 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2023-08-09T13:33:46.726 INFO:tasks.ceph.mgr.x.smithi203.stderr:2023-08-09T13:33:46.724+0000 7f943a4b2040 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2023-08-09T13:33:46.775 INFO:tasks.ceph.mgr.y.smithi043.stderr:2023-08-09T13:33:46.773+0000 7fe23d40f040 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2023-08-09T13:33:46.832 INFO:tasks.ceph.mgr.y.smithi043.stderr:2023-08-09T13:33:46.831+0000 7fe23d40f040 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2023-08-09T13:33:47.777 INFO:tasks.ceph.mgr.x.smithi203.stderr:2023-08-09T13:33:47.775+0000 7f93dc03e640 -1 mgr.server handle_report got status from non-daemon mon.b 2023-08-09T13:33:47.784 INFO:tasks.ceph.mgr.x.smithi203.stderr:2023-08-09T13:33:47.781+0000 7f93dc03e640 -1 mgr.server handle_report got status from non-daemon mon.c 2023-08-09T13:33:49.262 INFO:tasks.ceph.osd.5.smithi203.stderr:2023-08-09T13:33:49.260+0000 7ff3c3e1d500 -1 osd.5 0 log_to_monitors true 2023-08-09T13:33:49.447 INFO:tasks.ceph.osd.4.smithi203.stderr:2023-08-09T13:33:49.444+0000 7f21c8411500 -1 osd.4 0 log_to_monitors true 2023-08-09T13:33:49.484 INFO:tasks.ceph.osd.7.smithi203.stderr:2023-08-09T13:33:49.482+0000 7f07aad74500 -1 osd.7 0 log_to_monitors true 2023-08-09T13:33:49.530 INFO:tasks.ceph.osd.6.smithi203.stderr:2023-08-09T13:33:49.528+0000 7fd591621500 -1 osd.6 0 log_to_monitors true 2023-08-09T13:33:49.539 INFO:tasks.ceph.osd.2.smithi043.stderr:2023-08-09T13:33:49.538+0000 7fa72a80f500 -1 osd.2 0 log_to_monitors true 2023-08-09T13:33:49.549 INFO:tasks.ceph.osd.1.smithi043.stderr:2023-08-09T13:33:49.548+0000 7f6caaa22500 -1 osd.1 0 log_to_monitors true 2023-08-09T13:33:49.553 INFO:tasks.ceph.osd.3.smithi043.stderr:2023-08-09T13:33:49.552+0000 7f7c6880c500 -1 osd.3 0 log_to_monitors true 2023-08-09T13:33:49.554 INFO:tasks.ceph.osd.0.smithi043.stderr:2023-08-09T13:33:49.552+0000 7f231c869500 -1 osd.0 0 log_to_monitors true 2023-08-09T13:33:52.779 INFO:tasks.ceph.osd.2.smithi043.stderr:2023-08-09T13:33:52.778+0000 7fa7261a7640 -1 osd.2 0 waiting for initial osdmap 2023-08-09T13:33:52.797 INFO:tasks.ceph.osd.2.smithi043.stderr:2023-08-09T13:33:52.795+0000 7fa71dfc7640 -1 osd.2 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2023-08-09T13:33:52.804 INFO:tasks.ceph.osd.0.smithi043.stderr:2023-08-09T13:33:52.803+0000 7f23187f7640 -1 osd.0 0 waiting for initial osdmap 2023-08-09T13:33:52.805 INFO:tasks.ceph.osd.1.smithi043.stderr:2023-08-09T13:33:52.800+0000 7f6ca62ec640 -1 osd.1 0 waiting for initial osdmap 2023-08-09T13:33:52.817 INFO:tasks.ceph.osd.1.smithi043.stderr:2023-08-09T13:33:52.815+0000 7f6c9e10c640 -1 osd.1 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2023-08-09T13:33:52.817 INFO:tasks.ceph.osd.0.smithi043.stderr:2023-08-09T13:33:52.815+0000 7f2310617640 -1 osd.0 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2023-08-09T13:33:52.835 INFO:tasks.ceph.osd.3.smithi043.stderr:2023-08-09T13:33:52.834+0000 7f7c64095640 -1 osd.3 0 waiting for initial osdmap 2023-08-09T13:33:52.855 INFO:tasks.ceph.osd.3.smithi043.stderr:2023-08-09T13:33:52.854+0000 7f7c5beb5640 -1 osd.3 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2023-08-09T13:33:53.305 INFO:tasks.ceph.osd.4.smithi203.stderr:2023-08-09T13:33:53.303+0000 7f21c3cda640 -1 osd.4 0 waiting for initial osdmap 2023-08-09T13:33:53.320 INFO:tasks.ceph.osd.4.smithi203.stderr:2023-08-09T13:33:53.319+0000 7f21bbafa640 -1 osd.4 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2023-08-09T13:33:53.421 INFO:tasks.ceph.osd.7.smithi203.stderr:2023-08-09T13:33:53.420+0000 7f07a6d02640 -1 osd.7 0 waiting for initial osdmap 2023-08-09T13:33:53.441 INFO:tasks.ceph.osd.7.smithi203.stderr:2023-08-09T13:33:53.439+0000 7f079eb22640 -1 osd.7 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2023-08-09T13:33:53.462 DEBUG:teuthology.orchestra.run.smithi043:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph --cluster ceph osd dump --format=json 2023-08-09T13:33:53.465 INFO:tasks.ceph.osd.6.smithi203.stderr:2023-08-09T13:33:53.464+0000 7fd58d5ad640 -1 osd.6 0 waiting for initial osdmap 2023-08-09T13:33:53.483 INFO:tasks.ceph.osd.6.smithi203.stderr:2023-08-09T13:33:53.481+0000 7fd5853cd640 -1 osd.6 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2023-08-09T13:33:53.495 INFO:tasks.ceph.osd.5.smithi203.stderr:2023-08-09T13:33:53.494+0000 7ff3bf6e8640 -1 osd.5 0 waiting for initial osdmap 2023-08-09T13:33:53.513 INFO:tasks.ceph.osd.5.smithi203.stderr:2023-08-09T13:33:53.511+0000 7ff3b7508640 -1 osd.5 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2023-08-09T13:33:53.657 INFO:teuthology.misc.health.smithi043.stderr:2023-08-09T13:33:53.655+0000 7fc93e687640 1 Processor -- start 2023-08-09T13:33:53.657 INFO:teuthology.misc.health.smithi043.stderr:2023-08-09T13:33:53.656+0000 7fc93e687640 1 -- start start 2023-08-09T13:33:53.659 INFO:teuthology.misc.health.smithi043.stderr:2023-08-09T13:33:53.657+0000 7fc93e687640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fc9380b3f00 0x7fc9380b7e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:33:53.659 INFO:teuthology.misc.health.smithi043.stderr:2023-08-09T13:33:53.657+0000 7fc93e687640 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7fc93815cb30 0x7fc9380b42e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:33:53.660 INFO:teuthology.misc.health.smithi043.stderr:2023-08-09T13:33:53.657+0000 7fc93e687640 1 -- --> v1:172.21.15.43:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc9380d0970 con 0x7fc93806a780 2023-08-09T13:33:53.660 INFO:teuthology.misc.health.smithi043.stderr:2023-08-09T13:33:53.657+0000 7fc93e687640 1 -- --> v2:172.21.15.203:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc9380b8340 con 0x7fc93815cb30 2023-08-09T13:33:53.660 INFO:teuthology.misc.health.smithi043.stderr:2023-08-09T13:33:53.657+0000 7fc93e687640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc9380b4820 con 0x7fc9380b3f00 2023-08-09T13:33:53.660 INFO:teuthology.misc.health.smithi043.stderr:2023-08-09T13:33:53.658+0000 7fc93cc24640 1 --1- >> v1:172.21.15.43:6789/0 conn(0x7fc93806a780 0x7fc93806abf0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.43:6789/0 says I am v1:172.21.15.43:36044/0 (socket says 172.21.15.43:36044) 2023-08-09T13:33:53.660 INFO:teuthology.misc.health.smithi043.stderr:2023-08-09T13:33:53.658+0000 7fc93cc24640 1 -- 172.21.15.43:0/1307414507 learned_addr learned my addr 172.21.15.43:0/1307414507 (peer_addr_for_me v1:172.21.15.43:0/0) 2023-08-09T13:33:53.663 INFO:teuthology.misc.health.smithi043.stderr:2023-08-09T13:33:53.658+0000 7fc92ffff640 1 --2- 172.21.15.43:0/1307414507 >> v2:172.21.15.203:3300/0 conn(0x7fc93815cb30 0x7fc9380b42e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:33:53.663 INFO:teuthology.misc.health.smithi043.stderr:2023-08-09T13:33:53.658+0000 7fc937fff640 1 --2- 172.21.15.43:0/1307414507 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fc9380b3f00 0x7fc9380b7e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:33:53.663 INFO:teuthology.misc.health.smithi043.stderr:2023-08-09T13:33:53.658+0000 7fc9377fe640 1 -- 172.21.15.43:0/1307414507 <== mon.0 v1:172.21.15.43:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1093954849 0 0) 0x7fc9380d0970 con 0x7fc93806a780 2023-08-09T13:33:53.664 INFO:teuthology.misc.health.smithi043.stderr:2023-08-09T13:33:53.658+0000 7fc9377fe640 1 -- 172.21.15.43:0/1307414507 --> v1:172.21.15.43:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fc924003520 con 0x7fc93806a780 2023-08-09T13:33:53.664 INFO:teuthology.misc.health.smithi043.stderr:2023-08-09T13:33:53.659+0000 7fc9377fe640 1 -- 172.21.15.43:0/1307414507 <== mon.0 v1:172.21.15.43:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1502724484 0 0) 0x7fc924003520 con 0x7fc93806a780 2023-08-09T13:33:53.664 INFO:teuthology.misc.health.smithi043.stderr:2023-08-09T13:33:53.659+0000 7fc9377fe640 1 -- 172.21.15.43:0/1307414507 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fc9380b3f00 msgr2=0x7fc9380b7e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-09T13:33:53.664 INFO:teuthology.misc.health.smithi043.stderr:2023-08-09T13:33:53.659+0000 7fc9377fe640 1 --2- 172.21.15.43:0/1307414507 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fc9380b3f00 0x7fc9380b7e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:33:53.664 INFO:teuthology.misc.health.smithi043.stderr:2023-08-09T13:33:53.659+0000 7fc9377fe640 1 -- 172.21.15.43:0/1307414507 >> v2:172.21.15.203:3300/0 conn(0x7fc93815cb30 msgr2=0x7fc9380b42e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-09T13:33:53.664 INFO:teuthology.misc.health.smithi043.stderr:2023-08-09T13:33:53.659+0000 7fc9377fe640 1 --2- 172.21.15.43:0/1307414507 >> v2:172.21.15.203:3300/0 conn(0x7fc93815cb30 0x7fc9380b42e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:33:53.665 INFO:teuthology.misc.health.smithi043.stderr:2023-08-09T13:33:53.659+0000 7fc9377fe640 1 -- 172.21.15.43:0/1307414507 --> v1:172.21.15.43:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc93815d2e0 con 0x7fc93806a780 2023-08-09T13:33:53.665 INFO:teuthology.misc.health.smithi043.stderr:2023-08-09T13:33:53.659+0000 7fc9377fe640 1 -- 172.21.15.43:0/1307414507 <== mon.0 v1:172.21.15.43:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1285150740 0 0) 0x7fc9300019f0 con 0x7fc93806a780 2023-08-09T13:33:53.665 INFO:teuthology.misc.health.smithi043.stderr:2023-08-09T13:33:53.659+0000 7fc9377fe640 1 -- 172.21.15.43:0/1307414507 >> v1:172.21.15.43:6789/0 conn(0x7fc93806a780 legacy=0x7fc93806abf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:33:53.665 INFO:teuthology.misc.health.smithi043.stderr:2023-08-09T13:33:53.659+0000 7fc9377fe640 1 --2- 172.21.15.43:0/1307414507 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fc924003970 0x7fc924003d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:33:53.666 INFO:teuthology.misc.health.smithi043.stderr:2023-08-09T13:33:53.659+0000 7fc9377fe640 1 --2- 172.21.15.43:0/1307414507 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fc9380b3f00 0x7fc9240047c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:33:53.666 INFO:teuthology.misc.health.smithi043.stderr:2023-08-09T13:33:53.659+0000 7fc9377fe640 1 --2- 172.21.15.43:0/1307414507 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc924004d00 0x7fc924005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:33:53.666 INFO:teuthology.misc.health.smithi043.stderr:2023-08-09T13:33:53.659+0000 7fc9377fe640 1 -- 172.21.15.43:0/1307414507 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc9380b8340 con 0x7fc924004d00 2023-08-09T13:33:53.666 INFO:teuthology.misc.health.smithi043.stderr:2023-08-09T13:33:53.659+0000 7fc9377fe640 1 -- 172.21.15.43:0/1307414507 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc9380b4820 con 0x7fc924003970 2023-08-09T13:33:53.666 INFO:teuthology.misc.health.smithi043.stderr:2023-08-09T13:33:53.659+0000 7fc9377fe640 1 -- 172.21.15.43:0/1307414507 --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc92400b980 con 0x7fc9380b3f00 2023-08-09T13:33:53.667 INFO:teuthology.misc.health.smithi043.stderr:2023-08-09T13:33:53.659+0000 7fc93cc24640 1 --2- 172.21.15.43:0/1307414507 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc924004d00 0x7fc924005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:33:53.667 INFO:teuthology.misc.health.smithi043.stderr:2023-08-09T13:33:53.659+0000 7fc937fff640 1 --2- 172.21.15.43:0/1307414507 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fc924003970 0x7fc924003d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:33:53.667 INFO:teuthology.misc.health.smithi043.stderr:2023-08-09T13:33:53.659+0000 7fc92ffff640 1 --2- 172.21.15.43:0/1307414507 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fc9380b3f00 0x7fc9240047c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:33:53.667 INFO:teuthology.misc.health.smithi043.stderr:2023-08-09T13:33:53.659+0000 7fc93cc24640 1 -- 172.21.15.43:0/1307414507 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fc9380b3f00 msgr2=0x7fc9240047c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-09T13:33:53.667 INFO:teuthology.misc.health.smithi043.stderr:2023-08-09T13:33:53.659+0000 7fc93cc24640 1 --2- 172.21.15.43:0/1307414507 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fc9380b3f00 0x7fc9240047c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:33:53.667 INFO:teuthology.misc.health.smithi043.stderr:2023-08-09T13:33:53.660+0000 7fc93cc24640 1 -- 172.21.15.43:0/1307414507 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fc924003970 msgr2=0x7fc924003d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-09T13:33:53.668 INFO:teuthology.misc.health.smithi043.stderr:2023-08-09T13:33:53.660+0000 7fc93cc24640 1 --2- 172.21.15.43:0/1307414507 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fc924003970 0x7fc924003d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:33:53.668 INFO:teuthology.misc.health.smithi043.stderr:2023-08-09T13:33:53.660+0000 7fc93cc24640 1 -- 172.21.15.43:0/1307414507 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc924003520 con 0x7fc924004d00 2023-08-09T13:33:53.668 INFO:teuthology.misc.health.smithi043.stderr:2023-08-09T13:33:53.660+0000 7fc93cc24640 1 --2- 172.21.15.43:0/1307414507 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc924004d00 0x7fc924005180 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7fc93000a8d0 tx=0x7fc9240057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=125cff68e594e3d7 server_cookie=0 in_seq=0 out_seq=0 2023-08-09T13:33:53.668 INFO:teuthology.misc.health.smithi043.stderr:2023-08-09T13:33:53.661+0000 7fc9377fe640 1 -- 172.21.15.43:0/1307414507 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc930034630 con 0x7fc924004d00 2023-08-09T13:33:53.668 INFO:teuthology.misc.health.smithi043.stderr:2023-08-09T13:33:53.661+0000 7fc9377fe640 1 -- 172.21.15.43:0/1307414507 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc93002f070 con 0x7fc924004d00 2023-08-09T13:33:53.668 INFO:teuthology.misc.health.smithi043.stderr:2023-08-09T13:33:53.661+0000 7fc9377fe640 1 -- 172.21.15.43:0/1307414507 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc9300349d0 con 0x7fc924004d00 2023-08-09T13:33:53.668 INFO:teuthology.misc.health.smithi043.stderr:2023-08-09T13:33:53.661+0000 7fc93e687640 1 -- 172.21.15.43:0/1307414507 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc924004d00 msgr2=0x7fc924005180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:33:53.669 INFO:teuthology.misc.health.smithi043.stderr:2023-08-09T13:33:53.661+0000 7fc93e687640 1 --2- 172.21.15.43:0/1307414507 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc924004d00 0x7fc924005180 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7fc93000a8d0 tx=0x7fc9240057f0 comp rx=0 tx=0).stop 2023-08-09T13:33:53.669 INFO:teuthology.misc.health.smithi043.stderr:2023-08-09T13:33:53.661+0000 7fc93cc24640 1 -- 172.21.15.43:0/1307414507 reap_dead start 2023-08-09T13:33:53.669 INFO:teuthology.misc.health.smithi043.stderr:2023-08-09T13:33:53.662+0000 7fc93e687640 1 -- 172.21.15.43:0/1307414507 shutdown_connections 2023-08-09T13:33:53.669 INFO:teuthology.misc.health.smithi043.stderr:2023-08-09T13:33:53.662+0000 7fc93e687640 1 -- 172.21.15.43:0/1307414507 >> 172.21.15.43:0/1307414507 conn(0x7fc938071520 msgr2=0x7fc9380cef00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-09T13:33:53.669 INFO:teuthology.misc.health.smithi043.stderr:2023-08-09T13:33:53.662+0000 7fc93e687640 1 -- 172.21.15.43:0/1307414507 shutdown_connections 2023-08-09T13:33:53.669 INFO:teuthology.misc.health.smithi043.stderr:2023-08-09T13:33:53.662+0000 7fc93e687640 1 -- 172.21.15.43:0/1307414507 wait complete. 2023-08-09T13:33:53.671 INFO:teuthology.misc.health.smithi043.stderr:2023-08-09T13:33:53.663+0000 7fc93e687640 1 Processor -- start 2023-08-09T13:33:53.671 INFO:teuthology.misc.health.smithi043.stderr:2023-08-09T13:33:53.664+0000 7fc93e687640 1 -- start start 2023-08-09T13:33:53.672 INFO:teuthology.misc.health.smithi043.stderr:2023-08-09T13:33:53.664+0000 7fc93e687640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fc924004d00 0x7fc938154c90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:33:53.672 INFO:teuthology.misc.health.smithi043.stderr:2023-08-09T13:33:53.665+0000 7fc93e687640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fc938151290 0x7fc9380ca750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:33:53.672 INFO:teuthology.misc.health.smithi043.stderr:2023-08-09T13:33:53.665+0000 7fc93e687640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc9380cac90 0x7fc93815ac40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:33:53.672 INFO:teuthology.misc.health.smithi043.stderr:2023-08-09T13:33:53.665+0000 7fc93e687640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc938154800 con 0x7fc9380cac90 2023-08-09T13:33:53.672 INFO:teuthology.misc.health.smithi043.stderr:2023-08-09T13:33:53.665+0000 7fc93e687640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc938151710 con 0x7fc924004d00 2023-08-09T13:33:53.672 INFO:teuthology.misc.health.smithi043.stderr:2023-08-09T13:33:53.665+0000 7fc93e687640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc938153910 con 0x7fc938151290 2023-08-09T13:33:53.673 INFO:teuthology.misc.health.smithi043.stderr:2023-08-09T13:33:53.665+0000 7fc93cc24640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc9380cac90 0x7fc93815ac40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:33:53.673 INFO:teuthology.misc.health.smithi043.stderr:2023-08-09T13:33:53.665+0000 7fc93cc24640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc9380cac90 0x7fc93815ac40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:52618/0 (socket says 172.21.15.43:52618) 2023-08-09T13:33:53.673 INFO:teuthology.misc.health.smithi043.stderr:2023-08-09T13:33:53.665+0000 7fc93cc24640 1 -- 172.21.15.43:0/3692191837 learned_addr learned my addr 172.21.15.43:0/3692191837 (peer_addr_for_me v2:172.21.15.43:0/0) 2023-08-09T13:33:53.673 INFO:teuthology.misc.health.smithi043.stderr:2023-08-09T13:33:53.665+0000 7fc92ffff640 1 --2- 172.21.15.43:0/3692191837 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fc938151290 0x7fc9380ca750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:33:53.673 INFO:teuthology.misc.health.smithi043.stderr:2023-08-09T13:33:53.665+0000 7fc937fff640 1 --2- 172.21.15.43:0/3692191837 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fc924004d00 0x7fc938154c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:33:53.673 INFO:teuthology.misc.health.smithi043.stderr:2023-08-09T13:33:53.665+0000 7fc93cc24640 1 -- 172.21.15.43:0/3692191837 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fc938151290 msgr2=0x7fc9380ca750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:33:53.674 INFO:teuthology.misc.health.smithi043.stderr:2023-08-09T13:33:53.665+0000 7fc93cc24640 1 --2- 172.21.15.43:0/3692191837 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fc938151290 0x7fc9380ca750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:33:53.674 INFO:teuthology.misc.health.smithi043.stderr:2023-08-09T13:33:53.665+0000 7fc93cc24640 1 -- 172.21.15.43:0/3692191837 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fc924004d00 msgr2=0x7fc938154c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:33:53.674 INFO:teuthology.misc.health.smithi043.stderr:2023-08-09T13:33:53.665+0000 7fc93cc24640 1 --2- 172.21.15.43:0/3692191837 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fc924004d00 0x7fc938154c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:33:53.674 INFO:teuthology.misc.health.smithi043.stderr:2023-08-09T13:33:53.665+0000 7fc93cc24640 1 -- 172.21.15.43:0/3692191837 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc938153bf0 con 0x7fc9380cac90 2023-08-09T13:33:53.674 INFO:teuthology.misc.health.smithi043.stderr:2023-08-09T13:33:53.666+0000 7fc93cc24640 1 --2- 172.21.15.43:0/3692191837 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc9380cac90 0x7fc93815ac40 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7fc93000a9f0 tx=0x7fc930001360 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-09T13:33:53.674 INFO:teuthology.misc.health.smithi043.stderr:2023-08-09T13:33:53.666+0000 7fc9357fa640 1 -- 172.21.15.43:0/3692191837 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc930034840 con 0x7fc9380cac90 2023-08-09T13:33:53.674 INFO:teuthology.misc.health.smithi043.stderr:2023-08-09T13:33:53.666+0000 7fc9357fa640 1 -- 172.21.15.43:0/3692191837 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc9300349a0 con 0x7fc9380cac90 2023-08-09T13:33:53.674 INFO:teuthology.misc.health.smithi043.stderr:2023-08-09T13:33:53.666+0000 7fc9357fa640 1 -- 172.21.15.43:0/3692191837 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc930034d30 con 0x7fc9380cac90 2023-08-09T13:33:53.675 INFO:teuthology.misc.health.smithi043.stderr:2023-08-09T13:33:53.666+0000 7fc93e687640 1 -- 172.21.15.43:0/3692191837 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc938153ed0 con 0x7fc9380cac90 2023-08-09T13:33:53.675 INFO:teuthology.misc.health.smithi043.stderr:2023-08-09T13:33:53.666+0000 7fc93e687640 1 -- 172.21.15.43:0/3692191837 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc938154490 con 0x7fc9380cac90 2023-08-09T13:33:53.675 INFO:teuthology.misc.health.smithi043.stderr:2023-08-09T13:33:53.667+0000 7fc93e687640 1 -- 172.21.15.43:0/3692191837 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc938154620 con 0x7fc9380cac90 2023-08-09T13:33:53.675 INFO:teuthology.misc.health.smithi043.stderr:2023-08-09T13:33:53.668+0000 7fc9357fa640 1 -- 172.21.15.43:0/3692191837 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 4) v1 ==== 88285+0+0 (secure 0 0 0) 0x7fc930031030 con 0x7fc9380cac90 2023-08-09T13:33:53.675 INFO:teuthology.misc.health.smithi043.stderr:2023-08-09T13:33:53.669+0000 7fc9357fa640 1 --2- 172.21.15.43:0/3692191837 >> [v2:172.21.15.203:6832/830097474,v1:172.21.15.203:6833/830097474] conn(0x7fc918069ff0 0x7fc91806c4b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:33:53.675 INFO:teuthology.misc.health.smithi043.stderr:2023-08-09T13:33:53.669+0000 7fc9357fa640 1 -- 172.21.15.43:0/3692191837 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(12..12 src has 1..12) v4 ==== 2231+0+0 (secure 0 0 0) 0x7fc9300a9990 con 0x7fc9380cac90 2023-08-09T13:33:53.676 INFO:teuthology.misc.health.smithi043.stderr:2023-08-09T13:33:53.670+0000 7fc937fff640 1 --2- 172.21.15.43:0/3692191837 >> [v2:172.21.15.203:6832/830097474,v1:172.21.15.203:6833/830097474] conn(0x7fc918069ff0 0x7fc91806c4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:33:53.676 INFO:teuthology.misc.health.smithi043.stderr:2023-08-09T13:33:53.671+0000 7fc937fff640 1 --2- 172.21.15.43:0/3692191837 >> [v2:172.21.15.203:6832/830097474,v1:172.21.15.203:6833/830097474] conn(0x7fc918069ff0 0x7fc91806c4b0 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0x7fc928009990 tx=0x7fc928008040 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-09T13:33:53.676 INFO:teuthology.misc.health.smithi043.stderr:2023-08-09T13:33:53.674+0000 7fc9357fa640 1 -- 172.21.15.43:0/3692191837 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178066 (secure 0 0 0) 0x7fc930075fb0 con 0x7fc9380cac90 2023-08-09T13:33:53.881 INFO:teuthology.misc.health.smithi043.stderr:2023-08-09T13:33:53.880+0000 7fc93e687640 1 -- 172.21.15.43:0/3692191837 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fc938154620 con 0x7fc9380cac90 2023-08-09T13:33:53.882 INFO:teuthology.misc.health.smithi043.stderr:2023-08-09T13:33:53.881+0000 7fc9357fa640 1 -- 172.21.15.43:0/3692191837 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v13) v1 ==== 74+0+10896 (secure 0 0 0) 0x7fc9300795d0 con 0x7fc9380cac90 2023-08-09T13:33:53.883 INFO:teuthology.misc.health.smithi043.stdout: 2023-08-09T13:33:53.883 INFO:teuthology.misc.health.smithi043.stdout:{"epoch":13,"fsid":"daac87ee-1337-4250-a955-aea4e408a39b","created":"2023-08-09T13:33:41.701198+0000","modified":"2023-08-09T13:33:53.783453+0000","last_up_change":"2023-08-09T13:33:53.783453+0000","last_in_change":"2023-08-09T13:33:45.148904+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":0,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[],"osds":[{"osd":0,"uuid":"cbd49ecd-e896-4fbd-864e-5c37a829b165","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6800","nonce":502667335},{"type":"v1","addr":"172.21.15.43:6801","nonce":502667335}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6802","nonce":502667335},{"type":"v1","addr":"172.21.15.43:6803","nonce":502667335}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6806","nonce":502667335},{"type":"v1","addr":"172.21.15.43:6807","nonce":502667335}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6804","nonce":502667335},{"type":"v1","addr":"172.21.15.43:6805","nonce":502667335}]},"public_addr":"172.21.15.43:6801/502667335","cluster_addr":"172.21.15.43:6803/502667335","heartbeat_back_addr":"172.21.15.43:6807/502667335","heartbeat_front_addr":"172.21.15.43:6805/502667335","state":["exists","up"]},{"osd":1,"uuid":"75c7e0fe-1ec7-4fe8-a2f2-3661a65b8500","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6808","nonce":3289134610},{"type":"v1","addr":"172.21.15.43:6809","nonce":3289134610}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6810","nonce":3289134610},{"type":"v1","addr":"172.21.15.43:6812","nonce":3289134610}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6821","nonce":3289134610},{"type":"v1","addr":"172.21.15.43:6824","nonce":3289134610}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6815","nonce":3289134610},{"type":"v1","addr":"172.21.15.43:6818","nonce":3289134610}]},"public_addr":"172.21.15.43:6809/3289134610","cluster_addr":"172.21.15.43:6812/3289134610","heartbeat_back_addr":"172.21.15.43:6824/3289134610","heartbeat_front_addr":"172.21.15.43:6818/3289134610","state":["exists","up"]},{"osd":2,"uuid":"bb5c41e9-7f5f-4125-9b01-3bb8bfc2ea83","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6811","nonce":4079709736},{"type":"v1","addr":"172.21.15.43:6814","nonce":4079709736}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6817","nonce":4079709736},{"type":"v1","addr":"172.21.15.43:6820","nonce":4079709736}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6829","nonce":4079709736},{"type":"v1","addr":"172.21.15.43:6831","nonce":4079709736}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6823","nonce":4079709736},{"type":"v1","addr":"172.21.15.43:6826","nonce":4079709736}]},"public_addr":"172.21.15.43:6814/4079709736","cluster_addr":"172.21.15.43:6820/4079709736","heartbeat_back_addr":"172.21.15.43:6831/4079709736","heartbeat_front_addr":"172.21.15.43:6826/4079709736","state":["exists","up"]},{"osd":3,"uuid":"bfe4c8b4-694a-465c-87e6-2b2aab852613","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6813","nonce":1709538759},{"type":"v1","addr":"172.21.15.43:6816","nonce":1709538759}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6819","nonce":1709538759},{"type":"v1","addr":"172.21.15.43:6822","nonce":1709538759}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6828","nonce":1709538759},{"type":"v1","addr":"172.21.15.43:6830","nonce":1709538759}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6825","nonce":1709538759},{"type":"v1","addr":"172.21.15.43:6827","nonce":1709538759}]},"public_addr":"172.21.15.43:6816/1709538759","cluster_addr":"172.21.15.43:6822/1709538759","heartbeat_back_addr":"172.21.15.43:6830/1709538759","heartbeat_front_addr":"172.21.15.43:6827/1709538759","state":["exists","up"]},{"osd":4,"uuid":"fa2e8a77-aba1-4756-853e-0d174a5bdeba","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6800","nonce":4196866481},{"type":"v1","addr":"172.21.15.203:6801","nonce":4196866481}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6802","nonce":4196866481},{"type":"v1","addr":"172.21.15.203:6803","nonce":4196866481}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6806","nonce":4196866481},{"type":"v1","addr":"172.21.15.203:6807","nonce":4196866481}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6804","nonce":4196866481},{"type":"v1","addr":"172.21.15.203:6805","nonce":4196866481}]},"public_addr":"172.21.15.203:6801/4196866481","cluster_addr":"172.21.15.203:6803/4196866481","heartbeat_back_addr":"172.21.15.203:6807/4196866481","heartbeat_front_addr":"172.21.15.203:6805/4196866481","state":["exists","up"]},{"osd":5,"uuid":"81811e4b-4d24-4f51-a82c-f1d700a2a00d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6816","nonce":3059290081},{"type":"v1","addr":"172.21.15.203:6817","nonce":3059290081}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6818","nonce":3059290081},{"type":"v1","addr":"172.21.15.203:6819","nonce":3059290081}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6822","nonce":3059290081},{"type":"v1","addr":"172.21.15.203:6823","nonce":3059290081}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6820","nonce":3059290081},{"type":"v1","addr":"172.21.15.203:6821","nonce":3059290081}]},"public_addr":"172.21.15.203:6817/3059290081","cluster_addr":"172.21.15.203:6819/3059290081","heartbeat_back_addr":"172.21.15.203:6823/3059290081","heartbeat_front_addr":"172.21.15.203:6821/3059290081","state":["exists","up"]},{"osd":6,"uuid":"307bac4d-6a5b-443d-af75-56d695d7674f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6824","nonce":512155569},{"type":"v1","addr":"172.21.15.203:6825","nonce":512155569}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6826","nonce":512155569},{"type":"v1","addr":"172.21.15.203:6827","nonce":512155569}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6830","nonce":512155569},{"type":"v1","addr":"172.21.15.203:6831","nonce":512155569}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6828","nonce":512155569},{"type":"v1","addr":"172.21.15.203:6829","nonce":512155569}]},"public_addr":"172.21.15.203:6825/512155569","cluster_addr":"172.21.15.203:6827/512155569","heartbeat_back_addr":"172.21.15.203:6831/512155569","heartbeat_front_addr":"172.21.15.203:6829/512155569","state":["exists","up"]},{"osd":7,"uuid":"a95dc7ca-9f1b-4295-af70-68a22a2d7d5e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6808","nonce":3331450438},{"type":"v1","addr":"172.21.15.203:6809","nonce":3331450438}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6810","nonce":3331450438},{"type":"v1","addr":"172.21.15.203:6811","nonce":3331450438}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6814","nonce":3331450438},{"type":"v1","addr":"172.21.15.203:6815","nonce":3331450438}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6812","nonce":3331450438},{"type":"v1","addr":"172.21.15.203:6813","nonce":3331450438}]},"public_addr":"172.21.15.203:6809/3331450438","cluster_addr":"172.21.15.203:6811/3331450438","heartbeat_back_addr":"172.21.15.203:6815/3331450438","heartbeat_front_addr":"172.21.15.203:6813/3331450438","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-08-09T13:33:53.885 INFO:teuthology.misc.health.smithi043.stderr:2023-08-09T13:33:53.883+0000 7fc93e687640 1 -- 172.21.15.43:0/3692191837 >> [v2:172.21.15.203:6832/830097474,v1:172.21.15.203:6833/830097474] conn(0x7fc918069ff0 msgr2=0x7fc91806c4b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:33:53.886 INFO:teuthology.misc.health.smithi043.stderr:2023-08-09T13:33:53.883+0000 7fc93e687640 1 --2- 172.21.15.43:0/3692191837 >> [v2:172.21.15.203:6832/830097474,v1:172.21.15.203:6833/830097474] conn(0x7fc918069ff0 0x7fc91806c4b0 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0x7fc928009990 tx=0x7fc928008040 comp rx=0 tx=0).stop 2023-08-09T13:33:53.886 INFO:teuthology.misc.health.smithi043.stderr:2023-08-09T13:33:53.884+0000 7fc93e687640 1 -- 172.21.15.43:0/3692191837 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc9380cac90 msgr2=0x7fc93815ac40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:33:53.886 INFO:teuthology.misc.health.smithi043.stderr:2023-08-09T13:33:53.884+0000 7fc93e687640 1 --2- 172.21.15.43:0/3692191837 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc9380cac90 0x7fc93815ac40 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7fc93000a9f0 tx=0x7fc930001360 comp rx=0 tx=0).stop 2023-08-09T13:33:53.886 INFO:teuthology.misc.health.smithi043.stderr:2023-08-09T13:33:53.884+0000 7fc93e687640 1 -- 172.21.15.43:0/3692191837 shutdown_connections 2023-08-09T13:33:53.886 INFO:teuthology.misc.health.smithi043.stderr:2023-08-09T13:33:53.884+0000 7fc93e687640 1 --2- 172.21.15.43:0/3692191837 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc9380cac90 0x7fc93815ac40 unknown :-1 s=CLOSED pgs=29 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:33:53.887 INFO:teuthology.misc.health.smithi043.stderr:2023-08-09T13:33:53.884+0000 7fc93e687640 1 --2- 172.21.15.43:0/3692191837 >> [v2:172.21.15.203:6832/830097474,v1:172.21.15.203:6833/830097474] conn(0x7fc918069ff0 0x7fc91806c4b0 unknown :-1 s=CLOSED pgs=34 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:33:53.887 INFO:teuthology.misc.health.smithi043.stderr:2023-08-09T13:33:53.884+0000 7fc93e687640 1 --2- 172.21.15.43:0/3692191837 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fc938151290 0x7fc9380ca750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:33:53.887 INFO:teuthology.misc.health.smithi043.stderr:2023-08-09T13:33:53.884+0000 7fc93e687640 1 --2- 172.21.15.43:0/3692191837 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fc924004d00 0x7fc938154c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:33:53.887 INFO:teuthology.misc.health.smithi043.stderr:2023-08-09T13:33:53.884+0000 7fc93e687640 1 -- 172.21.15.43:0/3692191837 >> 172.21.15.43:0/3692191837 conn(0x7fc938071520 msgr2=0x7fc93806ab60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-09T13:33:53.887 INFO:teuthology.misc.health.smithi043.stderr:2023-08-09T13:33:53.884+0000 7fc93e687640 1 -- 172.21.15.43:0/3692191837 shutdown_connections 2023-08-09T13:33:53.887 INFO:teuthology.misc.health.smithi043.stderr:2023-08-09T13:33:53.884+0000 7fc93e687640 1 -- 172.21.15.43:0/3692191837 wait complete. 2023-08-09T13:33:53.898 DEBUG:teuthology.misc:8 of 8 OSDs are up 2023-08-09T13:33:53.899 INFO:tasks.ceph:Creating RBD pool 2023-08-09T13:33:53.899 DEBUG:teuthology.orchestra.run.smithi043:> sudo ceph --cluster ceph osd pool create rbd 8 2023-08-09T13:33:54.096 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:33:54.094+0000 7f7f0f071640 1 Processor -- start 2023-08-09T13:33:54.096 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:33:54.094+0000 7f7f0f071640 1 -- start start 2023-08-09T13:33:54.098 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:33:54.096+0000 7f7f0f071640 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7f7f080ce1c0 0x7f7f080ce5a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:33:54.098 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:33:54.096+0000 7f7f0f071640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f7f080c1620 0x7f7f080cbb80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:33:54.098 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:33:54.096+0000 7f7f0f071640 1 -- --> v1:172.21.15.203:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7f080cd0c0 con 0x7f7f08160e20 2023-08-09T13:33:54.098 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:33:54.096+0000 7f7f0f071640 1 -- --> v2:172.21.15.43:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7f080ceae0 con 0x7f7f080c1620 2023-08-09T13:33:54.098 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:33:54.096+0000 7f7f0f071640 1 -- --> v2:172.21.15.203:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7f08161790 con 0x7f7f080ce1c0 2023-08-09T13:33:54.099 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:33:54.097+0000 7f7f07fff640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f7f080c1620 0x7f7f080cbb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:33:54.100 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:33:54.097+0000 7f7f07fff640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f7f080c1620 0x7f7f080cbb80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:52624/0 (socket says 172.21.15.43:52624) 2023-08-09T13:33:54.100 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:33:54.097+0000 7f7f07fff640 1 -- 172.21.15.43:0/1221216922 learned_addr learned my addr 172.21.15.43:0/1221216922 (peer_addr_for_me v2:172.21.15.43:0/0) 2023-08-09T13:33:54.100 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:33:54.097+0000 7f7f0ce0d640 1 --2- 172.21.15.43:0/1221216922 >> v2:172.21.15.203:3300/0 conn(0x7f7f080ce1c0 0x7f7f080ce5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:33:54.100 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:33:54.097+0000 7f7f07fff640 1 -- 172.21.15.43:0/1221216922 >> v2:172.21.15.203:3300/0 conn(0x7f7f080ce1c0 msgr2=0x7f7f080ce5a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-09T13:33:54.100 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:33:54.097+0000 7f7f07fff640 1 --2- 172.21.15.43:0/1221216922 >> v2:172.21.15.203:3300/0 conn(0x7f7f080ce1c0 0x7f7f080ce5a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:33:54.100 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:33:54.097+0000 7f7f07fff640 1 -- 172.21.15.43:0/1221216922 >> v1:172.21.15.203:6789/0 conn(0x7f7f08160e20 legacy=0x7f7f080cadd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:33:54.100 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:33:54.097+0000 7f7f07fff640 1 -- 172.21.15.43:0/1221216922 --> v2:172.21.15.43:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7f08161a70 con 0x7f7f080c1620 2023-08-09T13:33:54.101 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:33:54.099+0000 7f7f07fff640 1 --2- 172.21.15.43:0/1221216922 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f7f080c1620 0x7f7f080cbb80 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0x7f7f080cedb0 tx=0x7f7ef402b910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5fa1b63364fdb66 server_cookie=0 in_seq=0 out_seq=0 2023-08-09T13:33:54.102 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:33:54.099+0000 7f7f077fe640 1 -- 172.21.15.43:0/1221216922 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7ef4031870 con 0x7f7f080c1620 2023-08-09T13:33:54.102 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:33:54.099+0000 7f7f077fe640 1 -- 172.21.15.43:0/1221216922 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7ef40319d0 con 0x7f7f080c1620 2023-08-09T13:33:54.102 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:33:54.099+0000 7f7f077fe640 1 -- 172.21.15.43:0/1221216922 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7ef4031d60 con 0x7f7f080c1620 2023-08-09T13:33:54.102 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:33:54.100+0000 7f7f0f071640 1 -- 172.21.15.43:0/1221216922 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f7f080c1620 msgr2=0x7f7f080cbb80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:33:54.102 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:33:54.100+0000 7f7f0f071640 1 --2- 172.21.15.43:0/1221216922 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f7f080c1620 0x7f7f080cbb80 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0x7f7f080cedb0 tx=0x7f7ef402b910 comp rx=0 tx=0).stop 2023-08-09T13:33:54.102 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:33:54.100+0000 7f7f0f071640 1 -- 172.21.15.43:0/1221216922 shutdown_connections 2023-08-09T13:33:54.103 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:33:54.100+0000 7f7f0f071640 1 --2- 172.21.15.43:0/1221216922 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f7f080c1620 0x7f7f080cbb80 unknown :-1 s=CLOSED pgs=30 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:33:54.103 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:33:54.100+0000 7f7f0f071640 1 --2- 172.21.15.43:0/1221216922 >> v2:172.21.15.203:3300/0 conn(0x7f7f080ce1c0 0x7f7f080ce5a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:33:54.103 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:33:54.100+0000 7f7f0f071640 1 -- 172.21.15.43:0/1221216922 >> 172.21.15.43:0/1221216922 conn(0x7f7f080aff50 msgr2=0x7f7f080b0360 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-09T13:33:54.103 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:33:54.100+0000 7f7f0f071640 1 -- 172.21.15.43:0/1221216922 shutdown_connections 2023-08-09T13:33:54.103 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:33:54.100+0000 7f7f0f071640 1 -- 172.21.15.43:0/1221216922 wait complete. 2023-08-09T13:33:54.104 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:33:54.102+0000 7f7f0f071640 1 Processor -- start 2023-08-09T13:33:54.105 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:33:54.102+0000 7f7f0f071640 1 -- start start 2023-08-09T13:33:54.105 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:33:54.103+0000 7f7f0f071640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f7f080c1620 0x7f7f0814f0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:33:54.105 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:33:54.103+0000 7f7f0f071640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f7f080ce1c0 0x7f7f080c8190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:33:54.105 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:33:54.103+0000 7f7f0f071640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f7f08160e20 0x7f7f080c5080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:33:54.105 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:33:54.103+0000 7f7f0f071640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7f08152910 con 0x7f7f080ce1c0 2023-08-09T13:33:54.105 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:33:54.103+0000 7f7f0f071640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7f080c6040 con 0x7f7f080c1620 2023-08-09T13:33:54.105 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:33:54.103+0000 7f7f0f071640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7f080c7d00 con 0x7f7f08160e20 2023-08-09T13:33:54.106 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:33:54.103+0000 7f7f07fff640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f7f080ce1c0 0x7f7f080c8190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:33:54.106 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:33:54.103+0000 7f7f07fff640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f7f080ce1c0 0x7f7f080c8190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:52628/0 (socket says 172.21.15.43:52628) 2023-08-09T13:33:54.106 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:33:54.103+0000 7f7f07fff640 1 -- 172.21.15.43:0/610325227 learned_addr learned my addr 172.21.15.43:0/610325227 (peer_addr_for_me v2:172.21.15.43:0/0) 2023-08-09T13:33:54.107 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:33:54.103+0000 7f7f07fff640 1 -- 172.21.15.43:0/610325227 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f7f08160e20 msgr2=0x7f7f080c5080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:33:54.107 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:33:54.103+0000 7f7f07fff640 1 --2- 172.21.15.43:0/610325227 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f7f08160e20 0x7f7f080c5080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:33:54.107 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:33:54.103+0000 7f7f07fff640 1 -- 172.21.15.43:0/610325227 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f7f080c1620 msgr2=0x7f7f0814f0a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:33:54.107 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:33:54.103+0000 7f7f07fff640 1 --2- 172.21.15.43:0/610325227 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f7f080c1620 0x7f7f0814f0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:33:54.107 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:33:54.103+0000 7f7f07fff640 1 -- 172.21.15.43:0/610325227 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7ef40094d0 con 0x7f7f080ce1c0 2023-08-09T13:33:54.107 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:33:54.104+0000 7f7f07fff640 1 --2- 172.21.15.43:0/610325227 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f7f080ce1c0 0x7f7f080c8190 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0x7f7ef402b950 tx=0x7f7ef40046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-09T13:33:54.108 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:33:54.104+0000 7f7f057fa640 1 -- 172.21.15.43:0/610325227 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7ef403ad30 con 0x7f7f080ce1c0 2023-08-09T13:33:54.108 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:33:54.104+0000 7f7f057fa640 1 -- 172.21.15.43:0/610325227 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7ef40443f0 con 0x7f7f080ce1c0 2023-08-09T13:33:54.108 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:33:54.104+0000 7f7f057fa640 1 -- 172.21.15.43:0/610325227 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7ef40447a0 con 0x7f7f080ce1c0 2023-08-09T13:33:54.108 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:33:54.104+0000 7f7f0f071640 1 -- 172.21.15.43:0/610325227 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7f080c7430 con 0x7f7f080ce1c0 2023-08-09T13:33:54.108 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:33:54.104+0000 7f7f0f071640 1 -- 172.21.15.43:0/610325227 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7f08152630 con 0x7f7f080ce1c0 2023-08-09T13:33:54.108 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:33:54.105+0000 7f7f0f071640 1 -- 172.21.15.43:0/610325227 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7f080739f0 con 0x7f7f080ce1c0 2023-08-09T13:33:54.109 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:33:54.106+0000 7f7f057fa640 1 -- 172.21.15.43:0/610325227 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 4) v1 ==== 88285+0+0 (secure 0 0 0) 0x7f7ef4044900 con 0x7f7f080ce1c0 2023-08-09T13:33:54.109 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:33:54.107+0000 7f7f057fa640 1 --2- 172.21.15.43:0/610325227 >> [v2:172.21.15.203:6832/830097474,v1:172.21.15.203:6833/830097474] conn(0x7f7eec069ff0 0x7f7eec06c4b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:33:54.109 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:33:54.107+0000 7f7f057fa640 1 -- 172.21.15.43:0/610325227 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(13..13 src has 1..13) v4 ==== 4471+0+0 (secure 0 0 0) 0x7f7ef402d040 con 0x7f7f080ce1c0 2023-08-09T13:33:54.110 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:33:54.108+0000 7f7f0ce0d640 1 --2- 172.21.15.43:0/610325227 >> [v2:172.21.15.203:6832/830097474,v1:172.21.15.203:6833/830097474] conn(0x7f7eec069ff0 0x7f7eec06c4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:33:54.110 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:33:54.109+0000 7f7f0ce0d640 1 --2- 172.21.15.43:0/610325227 >> [v2:172.21.15.203:6832/830097474,v1:172.21.15.203:6833/830097474] conn(0x7f7eec069ff0 0x7f7eec06c4b0 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7f7efc0098f0 tx=0x7f7efc008040 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-09T13:33:54.113 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:33:54.111+0000 7f7f057fa640 1 -- 172.21.15.43:0/610325227 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178066 (secure 0 0 0) 0x7f7ef4076220 con 0x7f7f080ce1c0 2023-08-09T13:33:54.312 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:33:54.310+0000 7f7f0f071640 1 -- 172.21.15.43:0/610325227 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "rbd", "pg_num": 8} v 0) v1 -- 0x7f7f080c7e40 con 0x7f7f080ce1c0 2023-08-09T13:33:54.861 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:33:54.858+0000 7f7f057fa640 1 -- 172.21.15.43:0/610325227 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "rbd", "pg_num": 8}]=0 pool 'rbd' created v14) v1 ==== 109+0+0 (secure 0 0 0) 0x7f7ef4035020 con 0x7f7f080ce1c0 2023-08-09T13:33:54.862 INFO:teuthology.orchestra.run.smithi043.stderr:pool 'rbd' created 2023-08-09T13:33:54.866 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:33:54.864+0000 7f7f0f071640 1 -- 172.21.15.43:0/610325227 >> [v2:172.21.15.203:6832/830097474,v1:172.21.15.203:6833/830097474] conn(0x7f7eec069ff0 msgr2=0x7f7eec06c4b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:33:54.866 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:33:54.864+0000 7f7f0f071640 1 --2- 172.21.15.43:0/610325227 >> [v2:172.21.15.203:6832/830097474,v1:172.21.15.203:6833/830097474] conn(0x7f7eec069ff0 0x7f7eec06c4b0 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7f7efc0098f0 tx=0x7f7efc008040 comp rx=0 tx=0).stop 2023-08-09T13:33:54.866 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:33:54.864+0000 7f7f0f071640 1 -- 172.21.15.43:0/610325227 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f7f080ce1c0 msgr2=0x7f7f080c8190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:33:54.867 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:33:54.864+0000 7f7f0f071640 1 --2- 172.21.15.43:0/610325227 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f7f080ce1c0 0x7f7f080c8190 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0x7f7ef402b950 tx=0x7f7ef40046c0 comp rx=0 tx=0).stop 2023-08-09T13:33:54.867 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:33:54.864+0000 7f7f0f071640 1 -- 172.21.15.43:0/610325227 shutdown_connections 2023-08-09T13:33:54.867 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:33:54.864+0000 7f7f0f071640 1 --2- 172.21.15.43:0/610325227 >> [v2:172.21.15.203:6832/830097474,v1:172.21.15.203:6833/830097474] conn(0x7f7eec069ff0 0x7f7eec06c4b0 unknown :-1 s=CLOSED pgs=43 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:33:54.867 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:33:54.864+0000 7f7f0f071640 1 --2- 172.21.15.43:0/610325227 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f7f08160e20 0x7f7f080c5080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:33:54.867 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:33:54.864+0000 7f7f0f071640 1 --2- 172.21.15.43:0/610325227 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f7f080ce1c0 0x7f7f080c8190 unknown :-1 s=CLOSED pgs=31 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:33:54.867 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:33:54.864+0000 7f7f0f071640 1 --2- 172.21.15.43:0/610325227 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f7f080c1620 0x7f7f0814f0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:33:54.867 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:33:54.864+0000 7f7f0f071640 1 -- 172.21.15.43:0/610325227 >> 172.21.15.43:0/610325227 conn(0x7f7f080aff50 msgr2=0x7f7f080adad0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-09T13:33:54.868 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:33:54.865+0000 7f7f0f071640 1 -- 172.21.15.43:0/610325227 shutdown_connections 2023-08-09T13:33:54.868 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:33:54.865+0000 7f7f0f071640 1 -- 172.21.15.43:0/610325227 wait complete. 2023-08-09T13:33:54.887 DEBUG:teuthology.orchestra.run.smithi043:> sudo ceph --cluster ceph osd pool application enable rbd rbd --yes-i-really-mean-it 2023-08-09T13:33:55.105 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:33:55.104+0000 7fdecd9ea640 1 Processor -- start 2023-08-09T13:33:55.106 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:33:55.104+0000 7fdecd9ea640 1 -- start start 2023-08-09T13:33:55.106 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:33:55.105+0000 7fdecd9ea640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7fdec81592b0 0x7fdec80706e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:33:55.107 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:33:55.105+0000 7fdecd9ea640 1 -- --> v1:172.21.15.43:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fdec80ccd20 con 0x7fdec80caea0 2023-08-09T13:33:55.107 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:33:55.105+0000 7fdecd9ea640 1 -- --> v1:172.21.15.203:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fdec806e9e0 con 0x7fdec8071420 2023-08-09T13:33:55.107 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:33:55.105+0000 7fdecd9ea640 1 -- --> v2:172.21.15.43:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fdec80ce780 con 0x7fdec81592b0 2023-08-09T13:33:55.108 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:33:55.105+0000 7fdec6ffd640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7fdec81592b0 0x7fdec80706e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:33:55.108 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:33:55.106+0000 7fdec6ffd640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7fdec81592b0 0x7fdec80706e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:57562/0 (socket says 172.21.15.43:57562) 2023-08-09T13:33:55.108 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:33:55.106+0000 7fdec6ffd640 1 -- 172.21.15.43:0/3870700081 learned_addr learned my addr 172.21.15.43:0/3870700081 (peer_addr_for_me v2:172.21.15.43:0/0) 2023-08-09T13:33:55.109 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:33:55.106+0000 7fdec6ffd640 1 -- 172.21.15.43:0/3870700081 >> v1:172.21.15.203:6789/0 conn(0x7fdec8071420 legacy=0x7fdec806dbc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-09T13:33:55.109 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:33:55.106+0000 7fdec6ffd640 1 -- 172.21.15.43:0/3870700081 >> v1:172.21.15.43:6789/0 conn(0x7fdec80caea0 legacy=0x7fdec806e2d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:33:55.109 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:33:55.106+0000 7fdec6ffd640 1 -- 172.21.15.43:0/3870700081 --> v2:172.21.15.43:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdec80cb460 con 0x7fdec81592b0 2023-08-09T13:33:55.109 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:33:55.106+0000 7fdec5ffb640 1 -- 172.21.15.43:0/3870700081 <== mon.0 v1:172.21.15.43:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3456869957 0 0) 0x7fdec80ccd20 con 0x7fdec80caea0 2023-08-09T13:33:55.110 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:33:55.107+0000 7fdec6ffd640 1 --2- 172.21.15.43:0/3870700081 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fdec81592b0 0x7fdec80706e0 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0x7fdeb8009e50 tx=0x7fdeb802bf70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1d8833b7bbd67b22 server_cookie=0 in_seq=0 out_seq=0 2023-08-09T13:33:55.111 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:33:55.108+0000 7fdec5ffb640 1 -- 172.21.15.43:0/3870700081 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdeb802cb10 con 0x7fdec81592b0 2023-08-09T13:33:55.111 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:33:55.108+0000 7fdec5ffb640 1 -- 172.21.15.43:0/3870700081 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdeb802cc70 con 0x7fdec81592b0 2023-08-09T13:33:55.111 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:33:55.108+0000 7fdec5ffb640 1 -- 172.21.15.43:0/3870700081 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdeb80343f0 con 0x7fdec81592b0 2023-08-09T13:33:55.111 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:33:55.108+0000 7fdecd9ea640 1 -- 172.21.15.43:0/3870700081 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fdec81592b0 msgr2=0x7fdec80706e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:33:55.111 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:33:55.108+0000 7fdecd9ea640 1 --2- 172.21.15.43:0/3870700081 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fdec81592b0 0x7fdec80706e0 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0x7fdeb8009e50 tx=0x7fdeb802bf70 comp rx=0 tx=0).stop 2023-08-09T13:33:55.111 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:33:55.108+0000 7fdecd9ea640 1 -- 172.21.15.43:0/3870700081 shutdown_connections 2023-08-09T13:33:55.112 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:33:55.108+0000 7fdecd9ea640 1 --2- 172.21.15.43:0/3870700081 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fdec81592b0 0x7fdec80706e0 unknown :-1 s=CLOSED pgs=33 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:33:55.112 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:33:55.108+0000 7fdecd9ea640 1 -- 172.21.15.43:0/3870700081 >> 172.21.15.43:0/3870700081 conn(0x7fdec80cbbd0 msgr2=0x7fdec80cbfe0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-09T13:33:55.112 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:33:55.108+0000 7fdecd9ea640 1 -- 172.21.15.43:0/3870700081 shutdown_connections 2023-08-09T13:33:55.112 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:33:55.109+0000 7fdecd9ea640 1 -- 172.21.15.43:0/3870700081 wait complete. 2023-08-09T13:33:55.113 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:33:55.110+0000 7fdecd9ea640 1 Processor -- start 2023-08-09T13:33:55.113 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:33:55.110+0000 7fdecd9ea640 1 -- start start 2023-08-09T13:33:55.113 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:33:55.111+0000 7fdecd9ea640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fdec8071420 0x7fdec815c0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:33:55.114 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:33:55.111+0000 7fdecd9ea640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fdec80caea0 0x7fdec80afd40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:33:55.114 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:33:55.111+0000 7fdecd9ea640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fdec81592b0 0x7fdec80b0280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:33:55.114 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:33:55.111+0000 7fdecd9ea640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdec81f5e90 con 0x7fdec81592b0 2023-08-09T13:33:55.114 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:33:55.111+0000 7fdecd9ea640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdec81f5560 con 0x7fdec8071420 2023-08-09T13:33:55.114 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:33:55.111+0000 7fdecd9ea640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdec815c5e0 con 0x7fdec80caea0 2023-08-09T13:33:55.114 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:33:55.112+0000 7fdec77fe640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fdec81592b0 0x7fdec80b0280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:33:55.115 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:33:55.112+0000 7fdec77fe640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fdec81592b0 0x7fdec80b0280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:57578/0 (socket says 172.21.15.43:57578) 2023-08-09T13:33:55.115 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:33:55.112+0000 7fdec6ffd640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fdec8071420 0x7fdec815c0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:33:55.115 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:33:55.112+0000 7fdec77fe640 1 -- 172.21.15.43:0/3773305614 learned_addr learned my addr 172.21.15.43:0/3773305614 (peer_addr_for_me v2:172.21.15.43:0/0) 2023-08-09T13:33:55.115 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:33:55.112+0000 7fdec67fc640 1 --2- 172.21.15.43:0/3773305614 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fdec80caea0 0x7fdec80afd40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:33:55.116 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:33:55.112+0000 7fdec77fe640 1 -- 172.21.15.43:0/3773305614 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fdec80caea0 msgr2=0x7fdec80afd40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:33:55.117 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:33:55.112+0000 7fdec77fe640 1 --2- 172.21.15.43:0/3773305614 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fdec80caea0 0x7fdec80afd40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:33:55.117 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:33:55.112+0000 7fdec77fe640 1 -- 172.21.15.43:0/3773305614 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fdec8071420 msgr2=0x7fdec815c0a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:33:55.117 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:33:55.112+0000 7fdec77fe640 1 --2- 172.21.15.43:0/3773305614 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fdec8071420 0x7fdec815c0a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:33:55.117 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:33:55.112+0000 7fdec77fe640 1 -- 172.21.15.43:0/3773305614 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdeb8009b30 con 0x7fdec81592b0 2023-08-09T13:33:55.117 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:33:55.113+0000 7fdec77fe640 1 --2- 172.21.15.43:0/3773305614 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fdec81592b0 0x7fdec80b0280 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0x7fdec000c3c0 tx=0x7fdec000c880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-09T13:33:55.118 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:33:55.113+0000 7fdeaffff640 1 -- 172.21.15.43:0/3773305614 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdec00035f0 con 0x7fdec81592b0 2023-08-09T13:33:55.118 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:33:55.113+0000 7fdeaffff640 1 -- 172.21.15.43:0/3773305614 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdec0003750 con 0x7fdec81592b0 2023-08-09T13:33:55.118 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:33:55.113+0000 7fdeaffff640 1 -- 172.21.15.43:0/3773305614 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdec0003b00 con 0x7fdec81592b0 2023-08-09T13:33:55.118 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:33:55.113+0000 7fdecd9ea640 1 -- 172.21.15.43:0/3773305614 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdec81f57d0 con 0x7fdec81592b0 2023-08-09T13:33:55.118 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:33:55.113+0000 7fdecd9ea640 1 -- 172.21.15.43:0/3773305614 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdec813f0c0 con 0x7fdec81592b0 2023-08-09T13:33:55.118 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:33:55.114+0000 7fdecd9ea640 1 -- 172.21.15.43:0/3773305614 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdec81f5fd0 con 0x7fdec81592b0 2023-08-09T13:33:55.119 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:33:55.116+0000 7fdeaffff640 1 -- 172.21.15.43:0/3773305614 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 4) v1 ==== 88285+0+0 (secure 0 0 0) 0x7fdec0003c60 con 0x7fdec81592b0 2023-08-09T13:33:55.119 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:33:55.117+0000 7fdeaffff640 1 --2- 172.21.15.43:0/3773305614 >> [v2:172.21.15.203:6832/830097474,v1:172.21.15.203:6833/830097474] conn(0x7fdeb4069fc0 0x7fdeb406c480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:33:55.120 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:33:55.117+0000 7fdeaffff640 1 -- 172.21.15.43:0/3773305614 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(14..14 src has 1..14) v4 ==== 5188+0+0 (secure 0 0 0) 0x7fdec0087600 con 0x7fdec81592b0 2023-08-09T13:33:55.120 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:33:55.117+0000 7fdec6ffd640 1 --2- 172.21.15.43:0/3773305614 >> [v2:172.21.15.203:6832/830097474,v1:172.21.15.203:6833/830097474] conn(0x7fdeb4069fc0 0x7fdeb406c480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:33:55.120 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:33:55.118+0000 7fdec6ffd640 1 --2- 172.21.15.43:0/3773305614 >> [v2:172.21.15.203:6832/830097474,v1:172.21.15.203:6833/830097474] conn(0x7fdeb4069fc0 0x7fdeb406c480 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7fdeb8035000 tx=0x7fdeb8030000 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-09T13:33:55.123 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:33:55.121+0000 7fdeaffff640 1 -- 172.21.15.43:0/3773305614 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178066 (secure 0 0 0) 0x7fdec81f5fd0 con 0x7fdec81592b0 2023-08-09T13:33:55.328 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:33:55.327+0000 7fdecd9ea640 1 -- 172.21.15.43:0/3773305614 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "osd pool application enable", "pool": "rbd", "app": "rbd", "yes_i_really_mean_it": true} v 0) v1 -- 0x7fdec81f5fd0 con 0x7fdec81592b0 2023-08-09T13:33:55.876 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:33:55.874+0000 7fdeaffff640 1 -- 172.21.15.43:0/3773305614 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool application enable", "pool": "rbd", "app": "rbd", "yes_i_really_mean_it": true}]=0 enabled application 'rbd' on pool 'rbd' v15) v1 ==== 173+0+0 (secure 0 0 0) 0x7fdec81f5fd0 con 0x7fdec81592b0 2023-08-09T13:33:55.876 INFO:teuthology.orchestra.run.smithi043.stderr:enabled application 'rbd' on pool 'rbd' 2023-08-09T13:33:55.880 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:33:55.879+0000 7fdecd9ea640 1 -- 172.21.15.43:0/3773305614 >> [v2:172.21.15.203:6832/830097474,v1:172.21.15.203:6833/830097474] conn(0x7fdeb4069fc0 msgr2=0x7fdeb406c480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:33:55.881 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:33:55.879+0000 7fdecd9ea640 1 --2- 172.21.15.43:0/3773305614 >> [v2:172.21.15.203:6832/830097474,v1:172.21.15.203:6833/830097474] conn(0x7fdeb4069fc0 0x7fdeb406c480 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7fdeb8035000 tx=0x7fdeb8030000 comp rx=0 tx=0).stop 2023-08-09T13:33:55.881 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:33:55.879+0000 7fdecd9ea640 1 -- 172.21.15.43:0/3773305614 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fdec81592b0 msgr2=0x7fdec80b0280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:33:55.881 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:33:55.879+0000 7fdecd9ea640 1 --2- 172.21.15.43:0/3773305614 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fdec81592b0 0x7fdec80b0280 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0x7fdec000c3c0 tx=0x7fdec000c880 comp rx=0 tx=0).stop 2023-08-09T13:33:55.881 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:33:55.879+0000 7fdecd9ea640 1 -- 172.21.15.43:0/3773305614 shutdown_connections 2023-08-09T13:33:55.882 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:33:55.879+0000 7fdecd9ea640 1 --2- 172.21.15.43:0/3773305614 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fdec81592b0 0x7fdec80b0280 unknown :-1 s=CLOSED pgs=34 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:33:55.882 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:33:55.879+0000 7fdecd9ea640 1 --2- 172.21.15.43:0/3773305614 >> [v2:172.21.15.203:6832/830097474,v1:172.21.15.203:6833/830097474] conn(0x7fdeb4069fc0 0x7fdeb406c480 unknown :-1 s=CLOSED pgs=44 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:33:55.882 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:33:55.879+0000 7fdecd9ea640 1 --2- 172.21.15.43:0/3773305614 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fdec80caea0 0x7fdec80afd40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:33:55.882 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:33:55.879+0000 7fdecd9ea640 1 --2- 172.21.15.43:0/3773305614 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fdec8071420 0x7fdec815c0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:33:55.882 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:33:55.879+0000 7fdecd9ea640 1 -- 172.21.15.43:0/3773305614 >> 172.21.15.43:0/3773305614 conn(0x7fdec80cbbd0 msgr2=0x7fdec80706c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-09T13:33:55.882 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:33:55.880+0000 7fdecd9ea640 1 -- 172.21.15.43:0/3773305614 shutdown_connections 2023-08-09T13:33:55.883 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:33:55.880+0000 7fdecd9ea640 1 -- 172.21.15.43:0/3773305614 wait complete. 2023-08-09T13:33:55.904 INFO:tasks.ceph:Starting mds daemons in cluster ceph... 2023-08-09T13:33:55.905 INFO:tasks.ceph.mds.a:Restarting daemon 2023-08-09T13:33:55.905 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i a 2023-08-09T13:33:55.913 INFO:tasks.ceph.mds.a:Started 2023-08-09T13:33:55.914 INFO:tasks.ceph.mds.c:Restarting daemon 2023-08-09T13:33:55.914 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i c 2023-08-09T13:33:55.922 INFO:tasks.ceph.mds.c:Started 2023-08-09T13:33:55.922 INFO:tasks.ceph.mds.b:Restarting daemon 2023-08-09T13:33:55.923 DEBUG:teuthology.orchestra.run.smithi203:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i b 2023-08-09T13:33:55.931 INFO:tasks.ceph.mds.b:Started 2023-08-09T13:33:55.932 INFO:tasks.ceph.mds.d:Restarting daemon 2023-08-09T13:33:55.932 DEBUG:teuthology.orchestra.run.smithi203:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i d 2023-08-09T13:33:55.942 INFO:tasks.ceph.mds.d:Started 2023-08-09T13:33:55.942 INFO:tasks.ceph:Setting up CephFS filesystem(s)... 2023-08-09T13:33:55.942 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-08-09T13:33:56.091 INFO:tasks.ceph.mds.a.smithi043.stdout:starting mds.a at 2023-08-09T13:33:56.109 INFO:tasks.ceph.mds.c.smithi043.stdout:starting mds.c at 2023-08-09T13:33:56.119 INFO:tasks.ceph.mds.b.smithi203.stdout:starting mds.b at 2023-08-09T13:33:56.119 INFO:tasks.ceph.mds.d.smithi203.stdout:starting mds.d at 2023-08-09T13:33:56.121 INFO:tasks.ceph.mds.b.smithi203.stderr:2023-08-09T13:33:56.120+0000 7f2cd0543640 -1 PosixStack listen unable to listen on v2:0.0.0.0:6834/0: (98) Address already in use 2023-08-09T13:33:56.155 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:33:56.153+0000 7fa7311dd640 1 Processor -- start 2023-08-09T13:33:56.155 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:33:56.153+0000 7fa7311dd640 1 -- start start 2023-08-09T13:33:56.156 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:33:56.155+0000 7fa7311dd640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fa72c15c620 0x7fa72c0cfe70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:33:56.156 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:33:56.155+0000 7fa7311dd640 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7fa72c0d0ef0 0x7fa72c0b27c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:33:56.156 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:33:56.155+0000 7fa7311dd640 1 -- --> v1:172.21.15.203:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa72c0b2330 con 0x7fa72c0cf4f0 2023-08-09T13:33:56.157 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:33:56.155+0000 7fa7311dd640 1 -- --> v2:172.21.15.203:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa72c0d2ec0 con 0x7fa72c0d0ef0 2023-08-09T13:33:56.157 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:33:56.155+0000 7fa7311dd640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa72c0d0500 con 0x7fa72c15c620 2023-08-09T13:33:56.157 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:33:56.155+0000 7fa72bfff640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fa72c15c620 0x7fa72c0cfe70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:33:56.158 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:33:56.155+0000 7fa7309dc640 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7fa72c0d0ef0 0x7fa72c0b27c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:33:56.158 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:33:56.155+0000 7fa723fff640 1 --1- >> v1:172.21.15.203:6789/0 conn(0x7fa72c0cf4f0 0x7fa72c0d07e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.203:6789/0 says I am v1:172.21.15.43:56928/0 (socket says 172.21.15.43:56928) 2023-08-09T13:33:56.158 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:33:56.155+0000 7fa723fff640 1 -- 172.21.15.43:0/3332378098 learned_addr learned my addr 172.21.15.43:0/3332378098 (peer_addr_for_me v1:172.21.15.43:0/0) 2023-08-09T13:33:56.158 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:33:56.156+0000 7fa72bfff640 1 -- 172.21.15.43:0/3332378098 >> v2:172.21.15.203:3300/0 conn(0x7fa72c0d0ef0 msgr2=0x7fa72c0b27c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-09T13:33:56.158 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:33:56.156+0000 7fa72bfff640 1 --2- 172.21.15.43:0/3332378098 >> v2:172.21.15.203:3300/0 conn(0x7fa72c0d0ef0 0x7fa72c0b27c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:33:56.158 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:33:56.156+0000 7fa72bfff640 1 -- 172.21.15.43:0/3332378098 >> v1:172.21.15.203:6789/0 conn(0x7fa72c0cf4f0 legacy=0x7fa72c0d07e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:33:56.158 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:33:56.156+0000 7fa72bfff640 1 -- 172.21.15.43:0/3332378098 --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa72c0b1a00 con 0x7fa72c15c620 2023-08-09T13:33:56.159 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:33:56.157+0000 7fa72bfff640 1 --2- 172.21.15.43:0/3332378098 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fa72c15c620 0x7fa72c0cfe70 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7fa71c009790 tx=0x7fa71c02b810 comp rx=0 tx=0).ready entity=mon.2 client_cookie=83a293b0be945516 server_cookie=0 in_seq=0 out_seq=0 2023-08-09T13:33:56.159 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:33:56.157+0000 7fa72b7fe640 1 -- 172.21.15.43:0/3332378098 <== mon.2 v2:172.21.15.203:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa71c031770 con 0x7fa72c15c620 2023-08-09T13:33:56.160 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:33:56.157+0000 7fa72b7fe640 1 -- 172.21.15.43:0/3332378098 <== mon.2 v2:172.21.15.203:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa71c0318d0 con 0x7fa72c15c620 2023-08-09T13:33:56.160 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:33:56.157+0000 7fa72b7fe640 1 -- 172.21.15.43:0/3332378098 <== mon.2 v2:172.21.15.203:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa71c031c60 con 0x7fa72c15c620 2023-08-09T13:33:56.160 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:33:56.158+0000 7fa7311dd640 1 -- 172.21.15.43:0/3332378098 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fa72c15c620 msgr2=0x7fa72c0cfe70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:33:56.160 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:33:56.158+0000 7fa7311dd640 1 --2- 172.21.15.43:0/3332378098 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fa72c15c620 0x7fa72c0cfe70 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7fa71c009790 tx=0x7fa71c02b810 comp rx=0 tx=0).stop 2023-08-09T13:33:56.160 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:33:56.158+0000 7fa7311dd640 1 -- 172.21.15.43:0/3332378098 shutdown_connections 2023-08-09T13:33:56.160 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:33:56.158+0000 7fa7311dd640 1 --2- 172.21.15.43:0/3332378098 >> v2:172.21.15.203:3300/0 conn(0x7fa72c0d0ef0 0x7fa72c0b27c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:33:56.160 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:33:56.158+0000 7fa7311dd640 1 --2- 172.21.15.43:0/3332378098 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fa72c15c620 0x7fa72c0cfe70 unknown :-1 s=CLOSED pgs=36 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:33:56.161 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:33:56.158+0000 7fa7311dd640 1 -- 172.21.15.43:0/3332378098 >> 172.21.15.43:0/3332378098 conn(0x7fa72c09b650 msgr2=0x7fa72c0d22a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-09T13:33:56.161 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:33:56.158+0000 7fa7311dd640 1 -- 172.21.15.43:0/3332378098 shutdown_connections 2023-08-09T13:33:56.161 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:33:56.158+0000 7fa7311dd640 1 -- 172.21.15.43:0/3332378098 wait complete. 2023-08-09T13:33:56.161 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:33:56.160+0000 7fa7311dd640 1 Processor -- start 2023-08-09T13:33:56.162 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:33:56.160+0000 7fa7311dd640 1 -- start start 2023-08-09T13:33:56.162 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:33:56.161+0000 7fa7311dd640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa72c0cf4f0 0x7fa72c15a590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:33:56.163 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:33:56.161+0000 7fa7311dd640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fa72c0d0ef0 0x7fa72c0c96c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:33:56.163 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:33:56.161+0000 7fa7311dd640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fa72c15c620 0x7fa72c0c67c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:33:56.163 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:33:56.161+0000 7fa7311dd640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa72c0c7380 con 0x7fa72c0cf4f0 2023-08-09T13:33:56.163 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:33:56.161+0000 7fa7311dd640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa72c0cc870 con 0x7fa72c0d0ef0 2023-08-09T13:33:56.163 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:33:56.161+0000 7fa7311dd640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa72c15cba0 con 0x7fa72c15c620 2023-08-09T13:33:56.163 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:33:56.161+0000 7fa72bfff640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa72c0cf4f0 0x7fa72c15a590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:33:56.163 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:33:56.161+0000 7fa72bfff640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa72c0cf4f0 0x7fa72c15a590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:57614/0 (socket says 172.21.15.43:57614) 2023-08-09T13:33:56.164 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:33:56.161+0000 7fa72bfff640 1 -- 172.21.15.43:0/269991865 learned_addr learned my addr 172.21.15.43:0/269991865 (peer_addr_for_me v2:172.21.15.43:0/0) 2023-08-09T13:33:56.164 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:33:56.161+0000 7fa7309dc640 1 --2- 172.21.15.43:0/269991865 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fa72c15c620 0x7fa72c0c67c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:33:56.164 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:33:56.161+0000 7fa723fff640 1 --2- 172.21.15.43:0/269991865 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fa72c0d0ef0 0x7fa72c0c96c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:33:56.165 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:33:56.161+0000 7fa72bfff640 1 -- 172.21.15.43:0/269991865 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fa72c15c620 msgr2=0x7fa72c0c67c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:33:56.165 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:33:56.161+0000 7fa72bfff640 1 --2- 172.21.15.43:0/269991865 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fa72c15c620 0x7fa72c0c67c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:33:56.165 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:33:56.161+0000 7fa72bfff640 1 -- 172.21.15.43:0/269991865 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fa72c0d0ef0 msgr2=0x7fa72c0c96c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:33:56.165 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:33:56.162+0000 7fa72bfff640 1 --2- 172.21.15.43:0/269991865 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fa72c0d0ef0 0x7fa72c0c96c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:33:56.165 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:33:56.162+0000 7fa72bfff640 1 -- 172.21.15.43:0/269991865 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa71c0093a0 con 0x7fa72c0cf4f0 2023-08-09T13:33:56.166 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:33:56.162+0000 7fa72bfff640 1 --2- 172.21.15.43:0/269991865 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa72c0cf4f0 0x7fa72c15a590 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7fa71c000c00 tx=0x7fa71c0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-09T13:33:56.166 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:33:56.162+0000 7fa7297fa640 1 -- 172.21.15.43:0/269991865 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa71c032810 con 0x7fa72c0cf4f0 2023-08-09T13:33:56.166 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:33:56.162+0000 7fa7297fa640 1 -- 172.21.15.43:0/269991865 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa71c032970 con 0x7fa72c0cf4f0 2023-08-09T13:33:56.166 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:33:56.162+0000 7fa7297fa640 1 -- 172.21.15.43:0/269991865 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa71c032d20 con 0x7fa72c0cf4f0 2023-08-09T13:33:56.166 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:33:56.162+0000 7fa7311dd640 1 -- 172.21.15.43:0/269991865 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa72c15ce80 con 0x7fa72c0cf4f0 2023-08-09T13:33:56.166 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:33:56.162+0000 7fa7311dd640 1 -- 172.21.15.43:0/269991865 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa72c15d440 con 0x7fa72c0cf4f0 2023-08-09T13:33:56.166 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:33:56.164+0000 7fa7311dd640 1 -- 172.21.15.43:0/269991865 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa72c0739f0 con 0x7fa72c0cf4f0 2023-08-09T13:33:56.167 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:33:56.165+0000 7fa7297fa640 1 -- 172.21.15.43:0/269991865 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 4) v1 ==== 88285+0+0 (secure 0 0 0) 0x7fa71c04e450 con 0x7fa72c0cf4f0 2023-08-09T13:33:56.167 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:33:56.165+0000 7fa7297fa640 1 --2- 172.21.15.43:0/269991865 >> [v2:172.21.15.203:6832/830097474,v1:172.21.15.203:6833/830097474] conn(0x7fa70c069fc0 0x7fa70c06c480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:33:56.167 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:33:56.166+0000 7fa7297fa640 1 -- 172.21.15.43:0/269991865 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(15..15 src has 1..15) v4 ==== 5210+0+0 (secure 0 0 0) 0x7fa71c0a9ff0 con 0x7fa72c0cf4f0 2023-08-09T13:33:56.168 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:33:56.166+0000 7fa723fff640 1 --2- 172.21.15.43:0/269991865 >> [v2:172.21.15.203:6832/830097474,v1:172.21.15.203:6833/830097474] conn(0x7fa70c069fc0 0x7fa70c06c480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:33:56.168 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:33:56.167+0000 7fa723fff640 1 --2- 172.21.15.43:0/269991865 >> [v2:172.21.15.203:6832/830097474,v1:172.21.15.203:6833/830097474] conn(0x7fa70c069fc0 0x7fa70c06c480 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7fa72c15b470 tx=0x7fa714007e80 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-09T13:33:56.171 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:33:56.170+0000 7fa7297fa640 1 -- 172.21.15.43:0/269991865 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178066 (secure 0 0 0) 0x7fa71c075c60 con 0x7fa72c0cf4f0 2023-08-09T13:33:56.379 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:33:56.377+0000 7fa7311dd640 1 -- 172.21.15.43:0/269991865 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fa72c0b1f90 con 0x7fa72c0cf4f0 2023-08-09T13:33:56.381 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:33:56.380+0000 7fa7297fa640 1 -- 172.21.15.43:0/269991865 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v15) v1 ==== 74+0+14593 (secure 0 0 0) 0x7fa71c079280 con 0x7fa72c0cf4f0 2023-08-09T13:33:56.382 INFO:teuthology.orchestra.run.smithi043.stdout: 2023-08-09T13:33:56.382 INFO:teuthology.orchestra.run.smithi043.stdout:{"epoch":15,"fsid":"daac87ee-1337-4250-a955-aea4e408a39b","created":"2023-08-09T13:33:41.701198+0000","modified":"2023-08-09T13:33:55.857278+0000","last_up_change":"2023-08-09T13:33:53.783453+0000","last_in_change":"2023-08-09T13:33:45.148904+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":2,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":"rbd","create_time":"2023-08-09T13:33:54.365137+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"15","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":".mgr","create_time":"2023-08-09T13:33:54.831803+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"15","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"cbd49ecd-e896-4fbd-864e-5c37a829b165","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6800","nonce":502667335},{"type":"v1","addr":"172.21.15.43:6801","nonce":502667335}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6802","nonce":502667335},{"type":"v1","addr":"172.21.15.43:6803","nonce":502667335}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6806","nonce":502667335},{"type":"v1","addr":"172.21.15.43:6807","nonce":502667335}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6804","nonce":502667335},{"type":"v1","addr":"172.21.15.43:6805","nonce":502667335}]},"public_addr":"172.21.15.43:6801/502667335","cluster_addr":"172.21.15.43:6803/502667335","heartbeat_back_addr":"172.21.15.43:6807/502667335","heartbeat_front_addr":"172.21.15.43:6805/502667335","state":["exists","up"]},{"osd":1,"uuid":"75c7e0fe-1ec7-4fe8-a2f2-3661a65b8500","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":14,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6808","nonce":3289134610},{"type":"v1","addr":"172.21.15.43:6809","nonce":3289134610}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6810","nonce":3289134610},{"type":"v1","addr":"172.21.15.43:6812","nonce":3289134610}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6821","nonce":3289134610},{"type":"v1","addr":"172.21.15.43:6824","nonce":3289134610}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6815","nonce":3289134610},{"type":"v1","addr":"172.21.15.43:6818","nonce":3289134610}]},"public_addr":"172.21.15.43:6809/3289134610","cluster_addr":"172.21.15.43:6812/3289134610","heartbeat_back_addr":"172.21.15.43:6824/3289134610","heartbeat_front_addr":"172.21.15.43:6818/3289134610","state":["exists","up"]},{"osd":2,"uuid":"bb5c41e9-7f5f-4125-9b01-3bb8bfc2ea83","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6811","nonce":4079709736},{"type":"v1","addr":"172.21.15.43:6814","nonce":4079709736}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6817","nonce":4079709736},{"type":"v1","addr":"172.21.15.43:6820","nonce":4079709736}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6829","nonce":4079709736},{"type":"v1","addr":"172.21.15.43:6831","nonce":4079709736}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6823","nonce":4079709736},{"type":"v1","addr":"172.21.15.43:6826","nonce":4079709736}]},"public_addr":"172.21.15.43:6814/4079709736","cluster_addr":"172.21.15.43:6820/4079709736","heartbeat_back_addr":"172.21.15.43:6831/4079709736","heartbeat_front_addr":"172.21.15.43:6826/4079709736","state":["exists","up"]},{"osd":3,"uuid":"bfe4c8b4-694a-465c-87e6-2b2aab852613","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":14,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6813","nonce":1709538759},{"type":"v1","addr":"172.21.15.43:6816","nonce":1709538759}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6819","nonce":1709538759},{"type":"v1","addr":"172.21.15.43:6822","nonce":1709538759}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6828","nonce":1709538759},{"type":"v1","addr":"172.21.15.43:6830","nonce":1709538759}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6825","nonce":1709538759},{"type":"v1","addr":"172.21.15.43:6827","nonce":1709538759}]},"public_addr":"172.21.15.43:6816/1709538759","cluster_addr":"172.21.15.43:6822/1709538759","heartbeat_back_addr":"172.21.15.43:6830/1709538759","heartbeat_front_addr":"172.21.15.43:6827/1709538759","state":["exists","up"]},{"osd":4,"uuid":"fa2e8a77-aba1-4756-853e-0d174a5bdeba","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":14,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6800","nonce":4196866481},{"type":"v1","addr":"172.21.15.203:6801","nonce":4196866481}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6802","nonce":4196866481},{"type":"v1","addr":"172.21.15.203:6803","nonce":4196866481}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6806","nonce":4196866481},{"type":"v1","addr":"172.21.15.203:6807","nonce":4196866481}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6804","nonce":4196866481},{"type":"v1","addr":"172.21.15.203:6805","nonce":4196866481}]},"public_addr":"172.21.15.203:6801/4196866481","cluster_addr":"172.21.15.203:6803/4196866481","heartbeat_back_addr":"172.21.15.203:6807/4196866481","heartbeat_front_addr":"172.21.15.203:6805/4196866481","state":["exists","up"]},{"osd":5,"uuid":"81811e4b-4d24-4f51-a82c-f1d700a2a00d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":14,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6816","nonce":3059290081},{"type":"v1","addr":"172.21.15.203:6817","nonce":3059290081}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6818","nonce":3059290081},{"type":"v1","addr":"172.21.15.203:6819","nonce":3059290081}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6822","nonce":3059290081},{"type":"v1","addr":"172.21.15.203:6823","nonce":3059290081}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6820","nonce":3059290081},{"type":"v1","addr":"172.21.15.203:6821","nonce":3059290081}]},"public_addr":"172.21.15.203:6817/3059290081","cluster_addr":"172.21.15.203:6819/3059290081","heartbeat_back_addr":"172.21.15.203:6823/3059290081","heartbeat_front_addr":"172.21.15.203:6821/3059290081","state":["exists","up"]},{"osd":6,"uuid":"307bac4d-6a5b-443d-af75-56d695d7674f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":14,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6824","nonce":512155569},{"type":"v1","addr":"172.21.15.203:6825","nonce":512155569}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6826","nonce":512155569},{"type":"v1","addr":"172.21.15.203:6827","nonce":512155569}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6830","nonce":512155569},{"type":"v1","addr":"172.21.15.203:6831","nonce":512155569}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6828","nonce":512155569},{"type":"v1","addr":"172.21.15.203:6829","nonce":512155569}]},"public_addr":"172.21.15.203:6825/512155569","cluster_addr":"172.21.15.203:6827/512155569","heartbeat_back_addr":"172.21.15.203:6831/512155569","heartbeat_front_addr":"172.21.15.203:6829/512155569","state":["exists","up"]},{"osd":7,"uuid":"a95dc7ca-9f1b-4295-af70-68a22a2d7d5e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":14,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6808","nonce":3331450438},{"type":"v1","addr":"172.21.15.203:6809","nonce":3331450438}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6810","nonce":3331450438},{"type":"v1","addr":"172.21.15.203:6811","nonce":3331450438}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6814","nonce":3331450438},{"type":"v1","addr":"172.21.15.203:6815","nonce":3331450438}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6812","nonce":3331450438},{"type":"v1","addr":"172.21.15.203:6813","nonce":3331450438}]},"public_addr":"172.21.15.203:6809/3331450438","cluster_addr":"172.21.15.203:6811/3331450438","heartbeat_back_addr":"172.21.15.203:6815/3331450438","heartbeat_front_addr":"172.21.15.203:6813/3331450438","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-09T13:33:50.547652+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-09T13:33:50.589469+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-09T13:33:50.561724+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-09T13:33:50.599650+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-09T13:33:50.411855+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-09T13:33:50.219681+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-09T13:33:50.504664+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-09T13:33:50.441248+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-08-09T13:33:56.384 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:33:56.383+0000 7fa7311dd640 1 -- 172.21.15.43:0/269991865 >> [v2:172.21.15.203:6832/830097474,v1:172.21.15.203:6833/830097474] conn(0x7fa70c069fc0 msgr2=0x7fa70c06c480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:33:56.384 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:33:56.383+0000 7fa7311dd640 1 --2- 172.21.15.43:0/269991865 >> [v2:172.21.15.203:6832/830097474,v1:172.21.15.203:6833/830097474] conn(0x7fa70c069fc0 0x7fa70c06c480 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7fa72c15b470 tx=0x7fa714007e80 comp rx=0 tx=0).stop 2023-08-09T13:33:56.385 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:33:56.383+0000 7fa7311dd640 1 -- 172.21.15.43:0/269991865 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa72c0cf4f0 msgr2=0x7fa72c15a590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:33:56.385 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:33:56.383+0000 7fa7311dd640 1 --2- 172.21.15.43:0/269991865 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa72c0cf4f0 0x7fa72c15a590 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7fa71c000c00 tx=0x7fa71c0046c0 comp rx=0 tx=0).stop 2023-08-09T13:33:56.385 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:33:56.383+0000 7fa7311dd640 1 -- 172.21.15.43:0/269991865 shutdown_connections 2023-08-09T13:33:56.385 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:33:56.383+0000 7fa7311dd640 1 --2- 172.21.15.43:0/269991865 >> [v2:172.21.15.203:6832/830097474,v1:172.21.15.203:6833/830097474] conn(0x7fa70c069fc0 0x7fa70c06c480 unknown :-1 s=CLOSED pgs=47 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:33:56.385 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:33:56.383+0000 7fa7311dd640 1 --2- 172.21.15.43:0/269991865 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fa72c15c620 0x7fa72c0c67c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:33:56.385 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:33:56.383+0000 7fa7311dd640 1 --2- 172.21.15.43:0/269991865 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fa72c0d0ef0 0x7fa72c0c96c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:33:56.386 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:33:56.383+0000 7fa7311dd640 1 --2- 172.21.15.43:0/269991865 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa72c0cf4f0 0x7fa72c15a590 unknown :-1 s=CLOSED pgs=39 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:33:56.386 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:33:56.383+0000 7fa7311dd640 1 -- 172.21.15.43:0/269991865 >> 172.21.15.43:0/269991865 conn(0x7fa72c09b650 msgr2=0x7fa72c0ce8d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-09T13:33:56.386 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:33:56.383+0000 7fa7311dd640 1 -- 172.21.15.43:0/269991865 shutdown_connections 2023-08-09T13:33:56.386 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:33:56.383+0000 7fa7311dd640 1 -- 172.21.15.43:0/269991865 wait complete. 2023-08-09T13:33:56.407 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': 'rbd', 'create_time': '2023-08-09T13:33:54.365137+0000', 'flags': 32769, 'flags_names': 'hashpspool,creating', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '15', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}, 'read_balance': {'score_acting': 2, 'score_stable': 2, 'optimal_score': 1, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': '.mgr', 'create_time': '2023-08-09T13:33:54.831803+0000', 'flags': 32769, 'flags_names': 'hashpspool,creating', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '15', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-08-09T13:33:56.408 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get rbd pg_num 2023-08-09T13:33:56.627 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:33:56.624+0000 7f0b89799640 1 Processor -- start 2023-08-09T13:33:56.627 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:33:56.625+0000 7f0b89799640 1 -- start start 2023-08-09T13:33:56.629 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:33:56.626+0000 7f0b89799640 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7f0b840cf8b0 0x7f0b840ce850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:33:56.629 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:33:56.626+0000 7f0b89799640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f0b840ca890 0x7f0b8406dda0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:33:56.629 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:33:56.626+0000 7f0b89799640 1 -- --> v1:172.21.15.43:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0b840cdbe0 con 0x7f0b8415c730 2023-08-09T13:33:56.629 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:33:56.626+0000 7f0b89799640 1 -- --> v2:172.21.15.43:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0b8406b0c0 con 0x7f0b840ca890 2023-08-09T13:33:56.629 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:33:56.626+0000 7f0b89799640 1 -- --> v2:172.21.15.203:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0b840cd920 con 0x7f0b840cf8b0 2023-08-09T13:33:56.630 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:33:56.627+0000 7f0b82ffd640 1 --1- >> v1:172.21.15.43:6789/0 conn(0x7f0b8415c730 0x7f0b840ce140 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.43:6789/0 says I am v1:172.21.15.43:40896/0 (socket says 172.21.15.43:40896) 2023-08-09T13:33:56.630 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:33:56.627+0000 7f0b82ffd640 1 -- 172.21.15.43:0/999740168 learned_addr learned my addr 172.21.15.43:0/999740168 (peer_addr_for_me v1:172.21.15.43:0/0) 2023-08-09T13:33:56.630 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:33:56.627+0000 7f0b837fe640 1 --2- 172.21.15.43:0/999740168 >> v2:172.21.15.43:3300/0 conn(0x7f0b840ca890 0x7f0b8406dda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:33:56.630 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:33:56.627+0000 7f0b827fc640 1 --2- 172.21.15.43:0/999740168 >> v2:172.21.15.203:3300/0 conn(0x7f0b840cf8b0 0x7f0b840ce850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:33:56.631 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:33:56.627+0000 7f0b81ffb640 1 -- 172.21.15.43:0/999740168 <== mon.0 v1:172.21.15.43:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1194310274 0 0) 0x7f0b840cdbe0 con 0x7f0b8415c730 2023-08-09T13:33:56.631 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:33:56.627+0000 7f0b81ffb640 1 -- 172.21.15.43:0/999740168 --> v1:172.21.15.43:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f0b6c003520 con 0x7f0b8415c730 2023-08-09T13:33:56.631 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:33:56.627+0000 7f0b837fe640 1 -- 172.21.15.43:0/999740168 >> v2:172.21.15.203:3300/0 conn(0x7f0b840cf8b0 msgr2=0x7f0b840ce850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-09T13:33:56.631 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:33:56.627+0000 7f0b837fe640 1 --2- 172.21.15.43:0/999740168 >> v2:172.21.15.203:3300/0 conn(0x7f0b840cf8b0 0x7f0b840ce850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:33:56.631 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:33:56.627+0000 7f0b837fe640 1 -- 172.21.15.43:0/999740168 >> v1:172.21.15.43:6789/0 conn(0x7f0b8415c730 legacy=0x7f0b840ce140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:33:56.631 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:33:56.627+0000 7f0b837fe640 1 -- 172.21.15.43:0/999740168 --> v2:172.21.15.43:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0b8406aa70 con 0x7f0b840ca890 2023-08-09T13:33:56.632 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:33:56.629+0000 7f0b837fe640 1 --2- 172.21.15.43:0/999740168 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f0b840ca890 0x7f0b8406dda0 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7f0b8406cab0 tx=0x7f0b7c02bec0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=275a0bc3f40d5e52 server_cookie=0 in_seq=0 out_seq=0 2023-08-09T13:33:56.633 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:33:56.629+0000 7f0b81ffb640 1 -- 172.21.15.43:0/999740168 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0b7c02cc70 con 0x7f0b840ca890 2023-08-09T13:33:56.633 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:33:56.629+0000 7f0b81ffb640 1 -- 172.21.15.43:0/999740168 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0b7c02cdd0 con 0x7f0b840ca890 2023-08-09T13:33:56.633 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:33:56.629+0000 7f0b81ffb640 1 -- 172.21.15.43:0/999740168 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0b7c034640 con 0x7f0b840ca890 2023-08-09T13:33:56.633 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:33:56.630+0000 7f0b89799640 1 -- 172.21.15.43:0/999740168 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f0b840ca890 msgr2=0x7f0b8406dda0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:33:56.633 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:33:56.630+0000 7f0b89799640 1 --2- 172.21.15.43:0/999740168 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f0b840ca890 0x7f0b8406dda0 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7f0b8406cab0 tx=0x7f0b7c02bec0 comp rx=0 tx=0).stop 2023-08-09T13:33:56.633 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:33:56.630+0000 7f0b89799640 1 -- 172.21.15.43:0/999740168 shutdown_connections 2023-08-09T13:33:56.633 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:33:56.630+0000 7f0b89799640 1 --2- 172.21.15.43:0/999740168 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f0b840ca890 0x7f0b8406dda0 unknown :-1 s=CLOSED pgs=41 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:33:56.634 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:33:56.630+0000 7f0b89799640 1 --2- 172.21.15.43:0/999740168 >> v2:172.21.15.203:3300/0 conn(0x7f0b840cf8b0 0x7f0b840ce850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:33:56.634 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:33:56.630+0000 7f0b89799640 1 -- 172.21.15.43:0/999740168 >> 172.21.15.43:0/999740168 conn(0x7f0b840b1dd0 msgr2=0x7f0b840b21e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-09T13:33:56.634 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:33:56.630+0000 7f0b89799640 1 -- 172.21.15.43:0/999740168 shutdown_connections 2023-08-09T13:33:56.634 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:33:56.630+0000 7f0b89799640 1 -- 172.21.15.43:0/999740168 wait complete. 2023-08-09T13:33:56.634 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:33:56.632+0000 7f0b89799640 1 Processor -- start 2023-08-09T13:33:56.635 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:33:56.632+0000 7f0b89799640 1 -- start start 2023-08-09T13:33:56.636 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:33:56.633+0000 7f0b89799640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f0b840ca890 0x7f0b841573a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:33:56.636 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:33:56.633+0000 7f0b89799640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f0b840cf8b0 0x7f0b841f1c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:33:56.636 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:33:56.633+0000 7f0b89799640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f0b8415c730 0x7f0b841e6040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:33:56.636 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:33:56.633+0000 7f0b89799640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0b84158b30 con 0x7f0b8415c730 2023-08-09T13:33:56.636 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:33:56.633+0000 7f0b89799640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0b841578e0 con 0x7f0b840cf8b0 2023-08-09T13:33:56.636 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:33:56.633+0000 7f0b89799640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0b84159eb0 con 0x7f0b840ca890 2023-08-09T13:33:56.637 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:33:56.633+0000 7f0b837fe640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f0b8415c730 0x7f0b841e6040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:33:56.637 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:33:56.633+0000 7f0b837fe640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f0b8415c730 0x7f0b841e6040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:57624/0 (socket says 172.21.15.43:57624) 2023-08-09T13:33:56.637 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:33:56.633+0000 7f0b837fe640 1 -- 172.21.15.43:0/4144831270 learned_addr learned my addr 172.21.15.43:0/4144831270 (peer_addr_for_me v2:172.21.15.43:0/0) 2023-08-09T13:33:56.638 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:33:56.634+0000 7f0b82ffd640 1 --2- 172.21.15.43:0/4144831270 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f0b840ca890 0x7f0b841573a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:33:56.638 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:33:56.634+0000 7f0b827fc640 1 --2- 172.21.15.43:0/4144831270 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f0b840cf8b0 0x7f0b841f1c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:33:56.638 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:33:56.634+0000 7f0b837fe640 1 -- 172.21.15.43:0/4144831270 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f0b840ca890 msgr2=0x7f0b841573a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:33:56.638 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:33:56.634+0000 7f0b837fe640 1 --2- 172.21.15.43:0/4144831270 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f0b840ca890 0x7f0b841573a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:33:56.638 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:33:56.634+0000 7f0b837fe640 1 -- 172.21.15.43:0/4144831270 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f0b840cf8b0 msgr2=0x7f0b841f1c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:33:56.638 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:33:56.634+0000 7f0b837fe640 1 --2- 172.21.15.43:0/4144831270 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f0b840cf8b0 0x7f0b841f1c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:33:56.639 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:33:56.634+0000 7f0b837fe640 1 -- 172.21.15.43:0/4144831270 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0b7c009a70 con 0x7f0b8415c730 2023-08-09T13:33:56.639 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:33:56.634+0000 7f0b837fe640 1 --2- 172.21.15.43:0/4144831270 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f0b8415c730 0x7f0b841e6040 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7f0b7c02c4d0 tx=0x7f0b7c004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-09T13:33:56.639 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:33:56.635+0000 7f0b6b7fe640 1 -- 172.21.15.43:0/4144831270 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0b7c03e610 con 0x7f0b8415c730 2023-08-09T13:33:56.639 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:33:56.635+0000 7f0b6b7fe640 1 -- 172.21.15.43:0/4144831270 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0b7c03e770 con 0x7f0b8415c730 2023-08-09T13:33:56.639 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:33:56.635+0000 7f0b6b7fe640 1 -- 172.21.15.43:0/4144831270 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0b7c03eb60 con 0x7f0b8415c730 2023-08-09T13:33:56.639 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:33:56.635+0000 7f0b89799640 1 -- 172.21.15.43:0/4144831270 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0b8415a190 con 0x7f0b8415c730 2023-08-09T13:33:56.639 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:33:56.635+0000 7f0b89799640 1 -- 172.21.15.43:0/4144831270 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0b8415a750 con 0x7f0b8415c730 2023-08-09T13:33:56.640 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:33:56.636+0000 7f0b89799640 1 -- 172.21.15.43:0/4144831270 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0b840739f0 con 0x7f0b8415c730 2023-08-09T13:33:56.640 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:33:56.638+0000 7f0b6b7fe640 1 -- 172.21.15.43:0/4144831270 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 4) v1 ==== 88285+0+0 (secure 0 0 0) 0x7f0b7c03b050 con 0x7f0b8415c730 2023-08-09T13:33:56.641 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:33:56.639+0000 7f0b6b7fe640 1 --2- 172.21.15.43:0/4144831270 >> [v2:172.21.15.203:6832/830097474,v1:172.21.15.203:6833/830097474] conn(0x7f0b70069ff0 0x7f0b7006c4b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:33:56.641 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:33:56.639+0000 7f0b6b7fe640 1 -- 172.21.15.43:0/4144831270 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(15..15 src has 1..15) v4 ==== 5210+0+0 (secure 0 0 0) 0x7f0b7c0aa290 con 0x7f0b8415c730 2023-08-09T13:33:56.642 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:33:56.640+0000 7f0b82ffd640 1 --2- 172.21.15.43:0/4144831270 >> [v2:172.21.15.203:6832/830097474,v1:172.21.15.203:6833/830097474] conn(0x7f0b70069ff0 0x7f0b7006c4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:33:56.643 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:33:56.641+0000 7f0b82ffd640 1 --2- 172.21.15.43:0/4144831270 >> [v2:172.21.15.203:6832/830097474,v1:172.21.15.203:6833/830097474] conn(0x7f0b70069ff0 0x7f0b7006c4b0 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7f0b74007fd0 tx=0x7f0b74009040 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-09T13:33:56.646 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:33:56.644+0000 7f0b6b7fe640 1 -- 172.21.15.43:0/4144831270 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178066 (secure 0 0 0) 0x7f0b7c075da0 con 0x7f0b8415c730 2023-08-09T13:33:56.843 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:33:56.841+0000 7f0b89799640 1 -- 172.21.15.43:0/4144831270 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"} v 0) v1 -- 0x7f0b8415a8e0 con 0x7f0b8415c730 2023-08-09T13:33:56.844 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:33:56.842+0000 7f0b6b7fe640 1 -- 172.21.15.43:0/4144831270 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"}]=0 v15) v1 ==== 92+0+10 (secure 0 0 0) 0x7f0b7c009450 con 0x7f0b8415c730 2023-08-09T13:33:56.844 INFO:teuthology.orchestra.run.smithi043.stdout:pg_num: 8 2023-08-09T13:33:56.846 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:33:56.844+0000 7f0b89799640 1 -- 172.21.15.43:0/4144831270 >> [v2:172.21.15.203:6832/830097474,v1:172.21.15.203:6833/830097474] conn(0x7f0b70069ff0 msgr2=0x7f0b7006c4b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:33:56.847 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:33:56.844+0000 7f0b89799640 1 --2- 172.21.15.43:0/4144831270 >> [v2:172.21.15.203:6832/830097474,v1:172.21.15.203:6833/830097474] conn(0x7f0b70069ff0 0x7f0b7006c4b0 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7f0b74007fd0 tx=0x7f0b74009040 comp rx=0 tx=0).stop 2023-08-09T13:33:56.847 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:33:56.844+0000 7f0b89799640 1 -- 172.21.15.43:0/4144831270 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f0b8415c730 msgr2=0x7f0b841e6040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:33:56.847 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:33:56.844+0000 7f0b89799640 1 --2- 172.21.15.43:0/4144831270 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f0b8415c730 0x7f0b841e6040 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7f0b7c02c4d0 tx=0x7f0b7c004a40 comp rx=0 tx=0).stop 2023-08-09T13:33:56.848 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:33:56.844+0000 7f0b89799640 1 -- 172.21.15.43:0/4144831270 shutdown_connections 2023-08-09T13:33:56.848 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:33:56.844+0000 7f0b89799640 1 --2- 172.21.15.43:0/4144831270 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f0b8415c730 0x7f0b841e6040 unknown :-1 s=CLOSED pgs=42 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:33:56.848 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:33:56.844+0000 7f0b89799640 1 --2- 172.21.15.43:0/4144831270 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f0b840cf8b0 0x7f0b841f1c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:33:56.848 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:33:56.844+0000 7f0b89799640 1 --2- 172.21.15.43:0/4144831270 >> [v2:172.21.15.203:6832/830097474,v1:172.21.15.203:6833/830097474] conn(0x7f0b70069ff0 0x7f0b7006c4b0 unknown :-1 s=CLOSED pgs=48 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:33:56.848 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:33:56.844+0000 7f0b89799640 1 --2- 172.21.15.43:0/4144831270 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f0b840ca890 0x7f0b841573a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:33:56.848 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:33:56.844+0000 7f0b89799640 1 -- 172.21.15.43:0/4144831270 >> 172.21.15.43:0/4144831270 conn(0x7f0b840b1dd0 msgr2=0x7f0b840ce820 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-09T13:33:56.848 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:33:56.845+0000 7f0b89799640 1 -- 172.21.15.43:0/4144831270 shutdown_connections 2023-08-09T13:33:56.849 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:33:56.845+0000 7f0b89799640 1 -- 172.21.15.43:0/4144831270 wait complete. 2023-08-09T13:33:56.865 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-08-09T13:33:57.064 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:33:57.061+0000 7f066ca4c640 1 Processor -- start 2023-08-09T13:33:57.064 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:33:57.062+0000 7f066ca4c640 1 -- start start 2023-08-09T13:33:57.066 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:33:57.063+0000 7f066ca4c640 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7f066815a560 0x7f06680c8dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:33:57.066 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:33:57.063+0000 7f066ca4c640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f06680adc00 0x7f066806fbf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:33:57.066 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:33:57.064+0000 7f066ca4c640 1 -- --> v1:172.21.15.203:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0668070eb0 con 0x7f0668070130 2023-08-09T13:33:57.066 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:33:57.064+0000 7f066ca4c640 1 -- --> v2:172.21.15.203:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f06680705c0 con 0x7f066815a560 2023-08-09T13:33:57.067 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:33:57.064+0000 7f066ca4c640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f066815a940 con 0x7f06680adc00 2023-08-09T13:33:57.067 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:33:57.064+0000 7f066659c640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f06680adc00 0x7f066806fbf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:33:57.067 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:33:57.064+0000 7f066659c640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f06680adc00 0x7f066806fbf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.203:3301/0 says I am v2:172.21.15.43:38644/0 (socket says 172.21.15.43:38644) 2023-08-09T13:33:57.068 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:33:57.064+0000 7f066659c640 1 -- 172.21.15.43:0/1213267649 learned_addr learned my addr 172.21.15.43:0/1213267649 (peer_addr_for_me v2:172.21.15.43:0/0) 2023-08-09T13:33:57.068 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:33:57.064+0000 7f066759e640 1 --1- 172.21.15.43:0/1213267649 >> v1:172.21.15.203:6789/0 conn(0x7f0668070130 0x7f066815dfc0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.203:6789/0 says I am v1:172.21.15.43:56940/0 (socket says 172.21.15.43:56940) 2023-08-09T13:33:57.068 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:33:57.064+0000 7f0666d9d640 1 --2- 172.21.15.43:0/1213267649 >> v2:172.21.15.203:3300/0 conn(0x7f066815a560 0x7f06680c8dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:33:57.068 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:33:57.065+0000 7f066659c640 1 -- 172.21.15.43:0/1213267649 >> v2:172.21.15.203:3300/0 conn(0x7f066815a560 msgr2=0x7f06680c8dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-09T13:33:57.069 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:33:57.065+0000 7f066659c640 1 --2- 172.21.15.43:0/1213267649 >> v2:172.21.15.203:3300/0 conn(0x7f066815a560 0x7f06680c8dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:33:57.069 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:33:57.065+0000 7f066659c640 1 -- 172.21.15.43:0/1213267649 >> v1:172.21.15.203:6789/0 conn(0x7f0668070130 legacy=0x7f066815dfc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:33:57.069 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:33:57.065+0000 7f066659c640 1 -- 172.21.15.43:0/1213267649 --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f06680cc7f0 con 0x7f06680adc00 2023-08-09T13:33:57.069 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:33:57.066+0000 7f066659c640 1 --2- 172.21.15.43:0/1213267649 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f06680adc00 0x7f066806fbf0 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7f065c009870 tx=0x7f065c02b8f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c42b6f73ee99ab98 server_cookie=0 in_seq=0 out_seq=0 2023-08-09T13:33:57.070 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:33:57.067+0000 7f0665d9b640 1 -- 172.21.15.43:0/1213267649 <== mon.2 v2:172.21.15.203:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f065c031640 con 0x7f06680adc00 2023-08-09T13:33:57.070 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:33:57.067+0000 7f0665d9b640 1 -- 172.21.15.43:0/1213267649 <== mon.2 v2:172.21.15.203:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f065c0317a0 con 0x7f06680adc00 2023-08-09T13:33:57.070 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:33:57.067+0000 7f0665d9b640 1 -- 172.21.15.43:0/1213267649 <== mon.2 v2:172.21.15.203:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f065c031b30 con 0x7f06680adc00 2023-08-09T13:33:57.070 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:33:57.067+0000 7f066ca4c640 1 -- 172.21.15.43:0/1213267649 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f06680adc00 msgr2=0x7f066806fbf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:33:57.071 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:33:57.067+0000 7f066ca4c640 1 --2- 172.21.15.43:0/1213267649 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f06680adc00 0x7f066806fbf0 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7f065c009870 tx=0x7f065c02b8f0 comp rx=0 tx=0).stop 2023-08-09T13:33:57.071 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:33:57.068+0000 7f066ca4c640 1 -- 172.21.15.43:0/1213267649 shutdown_connections 2023-08-09T13:33:57.071 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:33:57.068+0000 7f066ca4c640 1 --2- 172.21.15.43:0/1213267649 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f06680adc00 0x7f066806fbf0 unknown :-1 s=CLOSED pgs=37 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:33:57.071 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:33:57.068+0000 7f066ca4c640 1 --2- 172.21.15.43:0/1213267649 >> v2:172.21.15.203:3300/0 conn(0x7f066815a560 0x7f06680c8dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:33:57.071 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:33:57.068+0000 7f066ca4c640 1 -- 172.21.15.43:0/1213267649 >> 172.21.15.43:0/1213267649 conn(0x7f06680cb200 msgr2=0x7f06680cb610 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-09T13:33:57.071 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:33:57.068+0000 7f066ca4c640 1 -- 172.21.15.43:0/1213267649 shutdown_connections 2023-08-09T13:33:57.072 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:33:57.068+0000 7f066ca4c640 1 -- 172.21.15.43:0/1213267649 wait complete. 2023-08-09T13:33:57.072 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:33:57.069+0000 7f066ca4c640 1 Processor -- start 2023-08-09T13:33:57.072 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:33:57.070+0000 7f066ca4c640 1 -- start start 2023-08-09T13:33:57.073 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:33:57.070+0000 7f066ca4c640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f0668070130 0x7f06681ef5f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:33:57.073 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:33:57.070+0000 7f066ca4c640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f06680adc00 0x7f066815c610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:33:57.073 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:33:57.070+0000 7f066ca4c640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f066815a560 0x7f0668156260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:33:57.073 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:33:57.070+0000 7f066ca4c640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f066815d620 con 0x7f066815a560 2023-08-09T13:33:57.073 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:33:57.071+0000 7f066ca4c640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f06681451c0 con 0x7f06680adc00 2023-08-09T13:33:57.074 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:33:57.071+0000 7f066ca4c640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0668158780 con 0x7f0668070130 2023-08-09T13:33:57.075 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:33:57.071+0000 7f066759e640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f066815a560 0x7f0668156260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:33:57.075 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:33:57.071+0000 7f066759e640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f066815a560 0x7f0668156260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:57640/0 (socket says 172.21.15.43:57640) 2023-08-09T13:33:57.075 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:33:57.071+0000 7f066759e640 1 -- 172.21.15.43:0/1586663540 learned_addr learned my addr 172.21.15.43:0/1586663540 (peer_addr_for_me v2:172.21.15.43:0/0) 2023-08-09T13:33:57.075 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:33:57.071+0000 7f0666d9d640 1 --2- 172.21.15.43:0/1586663540 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f0668070130 0x7f06681ef5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:33:57.075 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:33:57.071+0000 7f066659c640 1 --2- 172.21.15.43:0/1586663540 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f06680adc00 0x7f066815c610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:33:57.075 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:33:57.071+0000 7f066759e640 1 -- 172.21.15.43:0/1586663540 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f0668070130 msgr2=0x7f06681ef5f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:33:57.076 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:33:57.071+0000 7f066759e640 1 --2- 172.21.15.43:0/1586663540 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f0668070130 0x7f06681ef5f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:33:57.076 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:33:57.071+0000 7f066759e640 1 -- 172.21.15.43:0/1586663540 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f06680adc00 msgr2=0x7f066815c610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:33:57.076 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:33:57.071+0000 7f066759e640 1 --2- 172.21.15.43:0/1586663540 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f06680adc00 0x7f066815c610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:33:57.076 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:33:57.071+0000 7f066759e640 1 -- 172.21.15.43:0/1586663540 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f065c0094d0 con 0x7f066815a560 2023-08-09T13:33:57.076 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:33:57.072+0000 7f066759e640 1 --2- 172.21.15.43:0/1586663540 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f066815a560 0x7f0668156260 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7f066000c730 tx=0x7f066000cbf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-09T13:33:57.076 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:33:57.072+0000 7f064f7fe640 1 -- 172.21.15.43:0/1586663540 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f066000ce40 con 0x7f066815a560 2023-08-09T13:33:57.076 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:33:57.072+0000 7f064f7fe640 1 -- 172.21.15.43:0/1586663540 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f066000a640 con 0x7f066815a560 2023-08-09T13:33:57.077 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:33:57.072+0000 7f064f7fe640 1 -- 172.21.15.43:0/1586663540 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f066000a9f0 con 0x7f066815a560 2023-08-09T13:33:57.077 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:33:57.072+0000 7f066ca4c640 1 -- 172.21.15.43:0/1586663540 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0668144f00 con 0x7f066815a560 2023-08-09T13:33:57.077 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:33:57.072+0000 7f066ca4c640 1 -- 172.21.15.43:0/1586663540 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f066815d060 con 0x7f066815a560 2023-08-09T13:33:57.077 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:33:57.073+0000 7f066ca4c640 1 -- 172.21.15.43:0/1586663540 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f066806d750 con 0x7f066815a560 2023-08-09T13:33:57.078 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:33:57.074+0000 7f064f7fe640 1 -- 172.21.15.43:0/1586663540 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 4) v1 ==== 88285+0+0 (secure 0 0 0) 0x7f066000ab50 con 0x7f066815a560 2023-08-09T13:33:57.078 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:33:57.075+0000 7f064f7fe640 1 --2- 172.21.15.43:0/1586663540 >> [v2:172.21.15.203:6832/830097474,v1:172.21.15.203:6833/830097474] conn(0x7f0654069fc0 0x7f065406c480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:33:57.078 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:33:57.075+0000 7f064f7fe640 1 -- 172.21.15.43:0/1586663540 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(16..16 src has 1..16) v4 ==== 5210+0+0 (secure 0 0 0) 0x7f0660087870 con 0x7f066815a560 2023-08-09T13:33:57.078 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:33:57.076+0000 7f0666d9d640 1 --2- 172.21.15.43:0/1586663540 >> [v2:172.21.15.203:6832/830097474,v1:172.21.15.203:6833/830097474] conn(0x7f0654069fc0 0x7f065406c480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:33:57.079 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:33:57.077+0000 7f0666d9d640 1 --2- 172.21.15.43:0/1586663540 >> [v2:172.21.15.203:6832/830097474,v1:172.21.15.203:6833/830097474] conn(0x7f0654069fc0 0x7f065406c480 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7f06680b2230 tx=0x7f0658008040 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-09T13:33:57.082 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:33:57.080+0000 7f064f7fe640 1 -- 172.21.15.43:0/1586663540 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178066 (secure 0 0 0) 0x7f06600533a0 con 0x7f066815a560 2023-08-09T13:33:57.286 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:33:57.283+0000 7f066ca4c640 1 -- 172.21.15.43:0/1586663540 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f066804e390 con 0x7f066815a560 2023-08-09T13:33:57.287 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:33:57.284+0000 7f064f7fe640 1 -- 172.21.15.43:0/1586663540 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v16) v1 ==== 93+0+10 (secure 0 0 0) 0x7f06600170a0 con 0x7f066815a560 2023-08-09T13:33:57.287 INFO:teuthology.orchestra.run.smithi043.stdout:pg_num: 1 2023-08-09T13:33:57.290 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:33:57.287+0000 7f066ca4c640 1 -- 172.21.15.43:0/1586663540 >> [v2:172.21.15.203:6832/830097474,v1:172.21.15.203:6833/830097474] conn(0x7f0654069fc0 msgr2=0x7f065406c480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:33:57.290 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:33:57.287+0000 7f066ca4c640 1 --2- 172.21.15.43:0/1586663540 >> [v2:172.21.15.203:6832/830097474,v1:172.21.15.203:6833/830097474] conn(0x7f0654069fc0 0x7f065406c480 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7f06680b2230 tx=0x7f0658008040 comp rx=0 tx=0).stop 2023-08-09T13:33:57.290 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:33:57.287+0000 7f066ca4c640 1 -- 172.21.15.43:0/1586663540 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f066815a560 msgr2=0x7f0668156260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:33:57.290 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:33:57.287+0000 7f066ca4c640 1 --2- 172.21.15.43:0/1586663540 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f066815a560 0x7f0668156260 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7f066000c730 tx=0x7f066000cbf0 comp rx=0 tx=0).stop 2023-08-09T13:33:57.291 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:33:57.287+0000 7f066ca4c640 1 -- 172.21.15.43:0/1586663540 shutdown_connections 2023-08-09T13:33:57.291 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:33:57.287+0000 7f066ca4c640 1 --2- 172.21.15.43:0/1586663540 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f066815a560 0x7f0668156260 unknown :-1 s=CLOSED pgs=43 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:33:57.291 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:33:57.287+0000 7f066ca4c640 1 --2- 172.21.15.43:0/1586663540 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f06680adc00 0x7f066815c610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:33:57.291 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:33:57.287+0000 7f066ca4c640 1 --2- 172.21.15.43:0/1586663540 >> [v2:172.21.15.203:6832/830097474,v1:172.21.15.203:6833/830097474] conn(0x7f0654069fc0 0x7f065406c480 unknown :-1 s=CLOSED pgs=49 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:33:57.291 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:33:57.287+0000 7f066ca4c640 1 --2- 172.21.15.43:0/1586663540 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f0668070130 0x7f06681ef5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:33:57.291 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:33:57.287+0000 7f066ca4c640 1 -- 172.21.15.43:0/1586663540 >> 172.21.15.43:0/1586663540 conn(0x7f06680cb200 msgr2=0x7f06680afa60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-09T13:33:57.292 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:33:57.287+0000 7f066ca4c640 1 -- 172.21.15.43:0/1586663540 shutdown_connections 2023-08-09T13:33:57.292 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:33:57.287+0000 7f066ca4c640 1 -- 172.21.15.43:0/1586663540 wait complete. 2023-08-09T13:33:57.308 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-08-09T13:33:57.544 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:33:57.541+0000 7f65fd1b5640 1 Processor -- start 2023-08-09T13:33:57.545 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:33:57.542+0000 7f65fd1b5640 1 -- start start 2023-08-09T13:33:57.552 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:33:57.543+0000 7f65fd1b5640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f65f80b4670 0x7f65f80d2ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:33:57.552 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:33:57.543+0000 7f65fd1b5640 1 -- --> v1:172.21.15.43:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f65f80ce090 con 0x7f65f80b2790 2023-08-09T13:33:57.552 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:33:57.543+0000 7f65fd1b5640 1 -- --> v1:172.21.15.203:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f65f80cd7f0 con 0x7f65f80b2b70 2023-08-09T13:33:57.553 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:33:57.543+0000 7f65fd1b5640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f65f80b3950 con 0x7f65f80b4670 2023-08-09T13:33:57.553 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:33:57.543+0000 7f65f6d9d640 1 --1- >> v1:172.21.15.43:6789/0 conn(0x7f65f80b2790 0x7f65f80b5330 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.43:6789/0 says I am v1:172.21.15.43:40906/0 (socket says 172.21.15.43:40906) 2023-08-09T13:33:57.553 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:33:57.543+0000 7f65f6d9d640 1 -- 172.21.15.43:0/336708461 learned_addr learned my addr 172.21.15.43:0/336708461 (peer_addr_for_me v1:172.21.15.43:0/0) 2023-08-09T13:33:57.554 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:33:57.544+0000 7f65f5d9b640 1 -- 172.21.15.43:0/336708461 <== mon.0 v1:172.21.15.43:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1098053957 0 0) 0x7f65f80ce090 con 0x7f65f80b2790 2023-08-09T13:33:57.554 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:33:57.544+0000 7f65f759e640 1 --2- 172.21.15.43:0/336708461 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f65f80b4670 0x7f65f80d2ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:33:57.554 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:33:57.544+0000 7f65f5d9b640 1 -- 172.21.15.43:0/336708461 --> v1:172.21.15.43:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f65e0003520 con 0x7f65f80b2790 2023-08-09T13:33:57.554 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:33:57.544+0000 7f65f5d9b640 1 -- 172.21.15.43:0/336708461 <== mon.0 v1:172.21.15.43:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 450512708 0 0) 0x7f65e0003520 con 0x7f65f80b2790 2023-08-09T13:33:57.555 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:33:57.544+0000 7f65f5d9b640 1 -- 172.21.15.43:0/336708461 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f65f80b4670 msgr2=0x7f65f80d2ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-09T13:33:57.555 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:33:57.544+0000 7f65f5d9b640 1 --2- 172.21.15.43:0/336708461 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f65f80b4670 0x7f65f80d2ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:33:57.555 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:33:57.544+0000 7f65f5d9b640 1 -- 172.21.15.43:0/336708461 >> v1:172.21.15.203:6789/0 conn(0x7f65f80b2b70 legacy=0x7f65f80b3f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:33:57.555 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:33:57.544+0000 7f65f5d9b640 1 -- 172.21.15.43:0/336708461 --> v1:172.21.15.43:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f65f80cdad0 con 0x7f65f80b2790 2023-08-09T13:33:57.556 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:33:57.544+0000 7f65f5d9b640 1 -- 172.21.15.43:0/336708461 <== mon.0 v1:172.21.15.43:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1285150740 0 0) 0x7f65e8001670 con 0x7f65f80b2790 2023-08-09T13:33:57.556 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:33:57.544+0000 7f65f5d9b640 1 -- 172.21.15.43:0/336708461 >> v1:172.21.15.43:6789/0 conn(0x7f65f80b2790 legacy=0x7f65f80b5330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:33:57.556 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:33:57.544+0000 7f65f5d9b640 1 --2- 172.21.15.43:0/336708461 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f65f80b4670 0x7f65e0003a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:33:57.557 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:33:57.544+0000 7f65f5d9b640 1 --2- 172.21.15.43:0/336708461 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f65e0003f90 0x7f65e00047c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:33:57.557 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:33:57.544+0000 7f65f5d9b640 1 --2- 172.21.15.43:0/336708461 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f65e0004d00 0x7f65e0005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:33:57.557 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:33:57.544+0000 7f65f5d9b640 1 -- 172.21.15.43:0/336708461 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f65f80b3950 con 0x7f65e0003f90 2023-08-09T13:33:57.557 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:33:57.544+0000 7f65f5d9b640 1 -- 172.21.15.43:0/336708461 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f65e000bae0 con 0x7f65e0004d00 2023-08-09T13:33:57.558 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:33:57.544+0000 7f65f5d9b640 1 -- 172.21.15.43:0/336708461 --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f65e000bca0 con 0x7f65f80b4670 2023-08-09T13:33:57.558 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:33:57.545+0000 7f65f659c640 1 --2- 172.21.15.43:0/336708461 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f65e0003f90 0x7f65e00047c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:33:57.558 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:33:57.545+0000 7f65f6d9d640 1 --2- 172.21.15.43:0/336708461 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f65f80b4670 0x7f65e0003a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:33:57.558 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:33:57.545+0000 7f65f659c640 1 -- 172.21.15.43:0/336708461 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f65f80b4670 msgr2=0x7f65e0003a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-09T13:33:57.559 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:33:57.545+0000 7f65f659c640 1 --2- 172.21.15.43:0/336708461 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f65f80b4670 0x7f65e0003a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:33:57.559 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:33:57.545+0000 7f65f759e640 1 --2- 172.21.15.43:0/336708461 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f65e0004d00 0x7f65e0005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:33:57.559 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:33:57.545+0000 7f65f659c640 1 -- 172.21.15.43:0/336708461 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f65e0004d00 msgr2=0x7f65e0005180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-09T13:33:57.560 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:33:57.545+0000 7f65f659c640 1 --2- 172.21.15.43:0/336708461 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f65e0004d00 0x7f65e0005180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:33:57.560 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:33:57.545+0000 7f65f659c640 1 -- 172.21.15.43:0/336708461 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f65e0003520 con 0x7f65e0003f90 2023-08-09T13:33:57.560 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:33:57.546+0000 7f65f659c640 1 --2- 172.21.15.43:0/336708461 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f65e0003f90 0x7f65e00047c0 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7f65ec001750 tx=0x7f65e00057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5a584acfcbcf7365 server_cookie=0 in_seq=0 out_seq=0 2023-08-09T13:33:57.560 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:33:57.547+0000 7f65f5d9b640 1 -- 172.21.15.43:0/336708461 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f65ec0052d0 con 0x7f65e0003f90 2023-08-09T13:33:57.561 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:33:57.547+0000 7f65f5d9b640 1 -- 172.21.15.43:0/336708461 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f65ec02e050 con 0x7f65e0003f90 2023-08-09T13:33:57.561 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:33:57.547+0000 7f65f5d9b640 1 -- 172.21.15.43:0/336708461 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f65ec003830 con 0x7f65e0003f90 2023-08-09T13:33:57.561 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:33:57.547+0000 7f65fd1b5640 1 -- 172.21.15.43:0/336708461 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f65e0003f90 msgr2=0x7f65e00047c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:33:57.561 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:33:57.547+0000 7f65fd1b5640 1 --2- 172.21.15.43:0/336708461 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f65e0003f90 0x7f65e00047c0 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7f65ec001750 tx=0x7f65e00057f0 comp rx=0 tx=0).stop 2023-08-09T13:33:57.562 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:33:57.547+0000 7f65f759e640 1 -- 172.21.15.43:0/336708461 reap_dead start 2023-08-09T13:33:57.562 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:33:57.547+0000 7f65fd1b5640 1 -- 172.21.15.43:0/336708461 shutdown_connections 2023-08-09T13:33:57.562 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:33:57.547+0000 7f65fd1b5640 1 -- 172.21.15.43:0/336708461 >> 172.21.15.43:0/336708461 conn(0x7f65f809b7b0 msgr2=0x7f65f80d24e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-09T13:33:57.563 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:33:57.547+0000 7f65fd1b5640 1 -- 172.21.15.43:0/336708461 shutdown_connections 2023-08-09T13:33:57.563 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:33:57.547+0000 7f65fd1b5640 1 -- 172.21.15.43:0/336708461 wait complete. 2023-08-09T13:33:57.563 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:33:57.549+0000 7f65fd1b5640 1 Processor -- start 2023-08-09T13:33:57.563 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:33:57.549+0000 7f65fd1b5640 1 -- start start 2023-08-09T13:33:57.567 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:33:57.550+0000 7f65fd1b5640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f65e0003f90 0x7f65f815f570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:33:57.567 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:33:57.550+0000 7f65fd1b5640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f65f80c6010 0x7f65f80b10b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:33:57.568 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:33:57.550+0000 7f65fd1b5640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f65f80c6490 0x7f65f80b15f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:33:57.568 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:33:57.550+0000 7f65fd1b5640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f65f80cb380 con 0x7f65f80c6010 2023-08-09T13:33:57.568 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:33:57.550+0000 7f65fd1b5640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f65f80cb660 con 0x7f65f80c6490 2023-08-09T13:33:57.569 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:33:57.550+0000 7f65fd1b5640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f65f80cb940 con 0x7f65e0003f90 2023-08-09T13:33:57.569 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:33:57.551+0000 7f65f659c640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f65f80c6010 0x7f65f80b10b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:33:57.569 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:33:57.551+0000 7f65f659c640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f65f80c6010 0x7f65f80b10b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:57660/0 (socket says 172.21.15.43:57660) 2023-08-09T13:33:57.570 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:33:57.551+0000 7f65f659c640 1 -- 172.21.15.43:0/2408471196 learned_addr learned my addr 172.21.15.43:0/2408471196 (peer_addr_for_me v2:172.21.15.43:0/0) 2023-08-09T13:33:57.570 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:33:57.551+0000 7f65f6d9d640 1 --2- 172.21.15.43:0/2408471196 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f65e0003f90 0x7f65f815f570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:33:57.570 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:33:57.551+0000 7f65f759e640 1 --2- 172.21.15.43:0/2408471196 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f65f80c6490 0x7f65f80b15f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:33:57.571 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:33:57.551+0000 7f65f659c640 1 -- 172.21.15.43:0/2408471196 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f65e0003f90 msgr2=0x7f65f815f570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:33:57.571 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:33:57.551+0000 7f65f659c640 1 --2- 172.21.15.43:0/2408471196 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f65e0003f90 0x7f65f815f570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:33:57.571 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:33:57.551+0000 7f65f659c640 1 -- 172.21.15.43:0/2408471196 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f65f80c6490 msgr2=0x7f65f80b15f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:33:57.572 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:33:57.551+0000 7f65f659c640 1 --2- 172.21.15.43:0/2408471196 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f65f80c6490 0x7f65f80b15f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:33:57.572 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:33:57.551+0000 7f65f659c640 1 -- 172.21.15.43:0/2408471196 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f65f80cbc20 con 0x7f65f80c6010 2023-08-09T13:33:57.572 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:33:57.551+0000 7f65f659c640 1 --2- 172.21.15.43:0/2408471196 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f65f80c6010 0x7f65f80b10b0 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7f65ec008fd0 tx=0x7f65ec001120 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-09T13:33:57.572 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:33:57.552+0000 7f65df7fe640 1 -- 172.21.15.43:0/2408471196 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f65ec007a30 con 0x7f65f80c6010 2023-08-09T13:33:57.573 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:33:57.552+0000 7f65df7fe640 1 -- 172.21.15.43:0/2408471196 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f65ec0069e0 con 0x7f65f80c6010 2023-08-09T13:33:57.573 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:33:57.552+0000 7f65df7fe640 1 -- 172.21.15.43:0/2408471196 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f65ec006d70 con 0x7f65f80c6010 2023-08-09T13:33:57.573 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:33:57.552+0000 7f65fd1b5640 1 -- 172.21.15.43:0/2408471196 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f65f815fab0 con 0x7f65f80c6010 2023-08-09T13:33:57.574 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:33:57.552+0000 7f65fd1b5640 1 -- 172.21.15.43:0/2408471196 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f65f80c7560 con 0x7f65f80c6010 2023-08-09T13:33:57.574 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:33:57.553+0000 7f65fd1b5640 1 -- 172.21.15.43:0/2408471196 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f65f80ce840 con 0x7f65f80c6010 2023-08-09T13:33:57.574 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:33:57.554+0000 7f65df7fe640 1 -- 172.21.15.43:0/2408471196 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 4) v1 ==== 88285+0+0 (secure 0 0 0) 0x7f65ec0035f0 con 0x7f65f80c6010 2023-08-09T13:33:57.574 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:33:57.555+0000 7f65df7fe640 1 --2- 172.21.15.43:0/2408471196 >> [v2:172.21.15.203:6832/830097474,v1:172.21.15.203:6833/830097474] conn(0x7f65e4069fc0 0x7f65e406c480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:33:57.575 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:33:57.555+0000 7f65df7fe640 1 -- 172.21.15.43:0/2408471196 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(16..16 src has 1..16) v4 ==== 5210+0+0 (secure 0 0 0) 0x7f65ec0a9e60 con 0x7f65f80c6010 2023-08-09T13:33:57.575 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:33:57.556+0000 7f65f6d9d640 1 --2- 172.21.15.43:0/2408471196 >> [v2:172.21.15.203:6832/830097474,v1:172.21.15.203:6833/830097474] conn(0x7f65e4069fc0 0x7f65e406c480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:33:57.575 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:33:57.557+0000 7f65f6d9d640 1 --2- 172.21.15.43:0/2408471196 >> [v2:172.21.15.203:6832/830097474,v1:172.21.15.203:6833/830097474] conn(0x7f65e4069fc0 0x7f65e406c480 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f65e80018e0 tx=0x7f65e8007040 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-09T13:33:57.576 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:33:57.560+0000 7f65df7fe640 1 -- 172.21.15.43:0/2408471196 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178066 (secure 0 0 0) 0x7f65ec075a20 con 0x7f65f80c6010 2023-08-09T13:33:57.706 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:33:57.703+0000 7f65df7fe640 1 -- 172.21.15.43:0/2408471196 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mgrmap(e 5) v1 ==== 88341+0+0 (secure 0 0 0) 0x7f65ec0af020 con 0x7f65f80c6010 2023-08-09T13:33:57.802 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:33:57.799+0000 7f65fd1b5640 1 -- 172.21.15.43:0/2408471196 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f65f80cba80 con 0x7f65f80c6010 2023-08-09T13:33:57.803 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:33:57.801+0000 7f65df7fe640 1 -- 172.21.15.43:0/2408471196 <== mon.0 v2:172.21.15.43:3300/0 8 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 3 v3) v1 ==== 93+0+3697 (secure 0 0 0) 0x7f65ec04be90 con 0x7f65f80c6010 2023-08-09T13:33:57.804 INFO:teuthology.orchestra.run.smithi043.stdout: 2023-08-09T13:33:57.804 INFO:teuthology.orchestra.run.smithi043.stdout:{"epoch":3,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4236,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.43:6833/3917539953","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6832","nonce":3917539953},{"type":"v1","addr":"172.21.15.43:6833","nonce":3917539953}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2},{"gid":4247,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.203:6837/2467794159","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6835","nonce":2467794159},{"type":"v1","addr":"172.21.15.203:6837","nonce":2467794159}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":4248,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.43:6835/1818380189","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6834","nonce":1818380189},{"type":"v1","addr":"172.21.15.43:6835","nonce":1818380189}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":4250,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.203:6836/798312463","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6834","nonce":798312463},{"type":"v1","addr":"172.21.15.203:6836","nonce":798312463}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3}],"filesystems":[]} 2023-08-09T13:33:57.804 INFO:teuthology.orchestra.run.smithi043.stderr:dumped fsmap epoch 3 2023-08-09T13:33:57.806 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:33:57.803+0000 7f65fd1b5640 1 -- 172.21.15.43:0/2408471196 >> [v2:172.21.15.203:6832/830097474,v1:172.21.15.203:6833/830097474] conn(0x7f65e4069fc0 msgr2=0x7f65e406c480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:33:57.806 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:33:57.803+0000 7f65fd1b5640 1 --2- 172.21.15.43:0/2408471196 >> [v2:172.21.15.203:6832/830097474,v1:172.21.15.203:6833/830097474] conn(0x7f65e4069fc0 0x7f65e406c480 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f65e80018e0 tx=0x7f65e8007040 comp rx=0 tx=0).stop 2023-08-09T13:33:57.807 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:33:57.803+0000 7f65fd1b5640 1 -- 172.21.15.43:0/2408471196 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f65f80c6010 msgr2=0x7f65f80b10b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:33:57.807 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:33:57.803+0000 7f65fd1b5640 1 --2- 172.21.15.43:0/2408471196 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f65f80c6010 0x7f65f80b10b0 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7f65ec008fd0 tx=0x7f65ec001120 comp rx=0 tx=0).stop 2023-08-09T13:33:57.807 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:33:57.804+0000 7f65fd1b5640 1 -- 172.21.15.43:0/2408471196 shutdown_connections 2023-08-09T13:33:57.807 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:33:57.804+0000 7f65fd1b5640 1 --2- 172.21.15.43:0/2408471196 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f65f80c6490 0x7f65f80b15f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:33:57.807 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:33:57.804+0000 7f65fd1b5640 1 --2- 172.21.15.43:0/2408471196 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f65f80c6010 0x7f65f80b10b0 unknown :-1 s=CLOSED pgs=46 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:33:57.807 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:33:57.804+0000 7f65fd1b5640 1 --2- 172.21.15.43:0/2408471196 >> [v2:172.21.15.203:6832/830097474,v1:172.21.15.203:6833/830097474] conn(0x7f65e4069fc0 0x7f65e406c480 unknown :-1 s=CLOSED pgs=53 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:33:57.807 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:33:57.804+0000 7f65fd1b5640 1 --2- 172.21.15.43:0/2408471196 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f65e0003f90 0x7f65f815f570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:33:57.808 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:33:57.804+0000 7f65fd1b5640 1 -- 172.21.15.43:0/2408471196 >> 172.21.15.43:0/2408471196 conn(0x7f65f809b7b0 msgr2=0x7f65f80b5990 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-09T13:33:57.808 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:33:57.804+0000 7f65fd1b5640 1 -- 172.21.15.43:0/2408471196 shutdown_connections 2023-08-09T13:33:57.808 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:33:57.804+0000 7f65fd1b5640 1 -- 172.21.15.43:0/2408471196 wait complete. 2023-08-09T13:33:57.824 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-08-09T13:33:57.866 DEBUG:teuthology.orchestra.run.smithi043:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-08-09T13:33:57.879 DEBUG:teuthology.orchestra.run.smithi203:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-08-09T13:33:58.042 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:33:58.039+0000 7f2ec6a60640 1 Processor -- start 2023-08-09T13:33:58.042 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:33:58.040+0000 7f2ec6a60640 1 -- start start 2023-08-09T13:33:58.044 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:33:58.041+0000 7f2ec6a60640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f2ec00b2b70 0x7f2ec00d2ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:33:58.044 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:33:58.041+0000 7f2ec6a60640 1 -- --> v1:172.21.15.43:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2ec00ce090 con 0x7f2ec00b3050 2023-08-09T13:33:58.044 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:33:58.041+0000 7f2ec6a60640 1 -- --> v1:172.21.15.203:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2ec00cd7f0 con 0x7f2ec00b2790 2023-08-09T13:33:58.044 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:33:58.042+0000 7f2ec6a60640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2ec00b3950 con 0x7f2ec00b2b70 2023-08-09T13:33:58.045 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:33:58.042+0000 7f2ebffff640 1 --1- >> v1:172.21.15.203:6789/0 conn(0x7f2ec00b2790 0x7f2ec00b5330 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.203:6789/0 says I am v1:172.21.15.43:56954/0 (socket says 172.21.15.43:56954) 2023-08-09T13:33:58.046 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:33:58.042+0000 7f2ebffff640 1 -- 172.21.15.43:0/3696596918 learned_addr learned my addr 172.21.15.43:0/3696596918 (peer_addr_for_me v1:172.21.15.43:0/0) 2023-08-09T13:33:58.046 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:33:58.042+0000 7f2ebf7fe640 1 --2- 172.21.15.43:0/3696596918 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f2ec00b2b70 0x7f2ec00d2ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:33:58.046 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:33:58.042+0000 7f2ebeffd640 1 -- 172.21.15.43:0/3696596918 <== mon.1 v1:172.21.15.203:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2112693820 0 0) 0x7f2ec00cd7f0 con 0x7f2ec00b2790 2023-08-09T13:33:58.046 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:33:58.042+0000 7f2ebeffd640 1 -- 172.21.15.43:0/3696596918 --> v1:172.21.15.203:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f2ea8003520 con 0x7f2ec00b2790 2023-08-09T13:33:58.046 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:33:58.042+0000 7f2ebeffd640 1 -- 172.21.15.43:0/3696596918 <== mon.0 v1:172.21.15.43:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 665254261 0 0) 0x7f2ec00ce090 con 0x7f2ec00b3050 2023-08-09T13:33:58.046 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:33:58.043+0000 7f2ebeffd640 1 -- 172.21.15.43:0/3696596918 --> v1:172.21.15.43:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f2ec00cd7f0 con 0x7f2ec00b3050 2023-08-09T13:33:58.046 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:33:58.043+0000 7f2ebeffd640 1 -- 172.21.15.43:0/3696596918 <== mon.1 v1:172.21.15.203:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3461031229 0 0) 0x7f2ea8003520 con 0x7f2ec00b2790 2023-08-09T13:33:58.047 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:33:58.043+0000 7f2ebeffd640 1 -- 172.21.15.43:0/3696596918 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f2ec00b2b70 msgr2=0x7f2ec00d2ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-09T13:33:58.047 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:33:58.043+0000 7f2ebeffd640 1 --2- 172.21.15.43:0/3696596918 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f2ec00b2b70 0x7f2ec00d2ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:33:58.047 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:33:58.043+0000 7f2ebeffd640 1 -- 172.21.15.43:0/3696596918 >> v1:172.21.15.43:6789/0 conn(0x7f2ec00b3050 legacy=0x7f2ec00b3f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:33:58.047 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:33:58.043+0000 7f2ebeffd640 1 -- 172.21.15.43:0/3696596918 --> v1:172.21.15.203:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2ec00cdad0 con 0x7f2ec00b2790 2023-08-09T13:33:58.047 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:33:58.043+0000 7f2ebeffd640 1 -- 172.21.15.43:0/3696596918 <== mon.1 v1:172.21.15.203:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1285150740 0 0) 0x7f2eb0001670 con 0x7f2ec00b2790 2023-08-09T13:33:58.047 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:33:58.043+0000 7f2ebeffd640 1 -- 172.21.15.43:0/3696596918 >> v1:172.21.15.203:6789/0 conn(0x7f2ec00b2790 legacy=0x7f2ec00b5330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:33:58.047 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:33:58.043+0000 7f2ebeffd640 1 --2- 172.21.15.43:0/3696596918 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f2ea8003b70 0x7f2ea8003f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:33:58.048 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:33:58.043+0000 7f2ebeffd640 1 --2- 172.21.15.43:0/3696596918 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f2ea80047c0 0x7f2ea8004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:33:58.048 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:33:58.043+0000 7f2ebeffd640 1 --2- 172.21.15.43:0/3696596918 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f2ec00b2b70 0x7f2ea8005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:33:58.048 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:33:58.043+0000 7f2ebeffd640 1 -- 172.21.15.43:0/3696596918 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2ec00b3950 con 0x7f2ea8003b70 2023-08-09T13:33:58.048 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:33:58.043+0000 7f2ebeffd640 1 -- 172.21.15.43:0/3696596918 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2ea800ba30 con 0x7f2ea80047c0 2023-08-09T13:33:58.048 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:33:58.043+0000 7f2ebeffd640 1 -- 172.21.15.43:0/3696596918 --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2ea800bbf0 con 0x7f2ec00b2b70 2023-08-09T13:33:58.048 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:33:58.043+0000 7f2ebffff640 1 --2- 172.21.15.43:0/3696596918 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f2ea8003b70 0x7f2ea8003f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:33:58.049 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:33:58.043+0000 7f2ebf7fe640 1 --2- 172.21.15.43:0/3696596918 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f2ea80047c0 0x7f2ea8004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:33:58.050 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:33:58.043+0000 7f2ec4ffd640 1 --2- 172.21.15.43:0/3696596918 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f2ec00b2b70 0x7f2ea8005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:33:58.050 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:33:58.044+0000 7f2ebffff640 1 -- 172.21.15.43:0/3696596918 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f2ec00b2b70 msgr2=0x7f2ea8005180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-09T13:33:58.050 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:33:58.044+0000 7f2ebffff640 1 --2- 172.21.15.43:0/3696596918 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f2ec00b2b70 0x7f2ea8005180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:33:58.050 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:33:58.044+0000 7f2ebffff640 1 -- 172.21.15.43:0/3696596918 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f2ea80047c0 msgr2=0x7f2ea8004c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-09T13:33:58.050 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:33:58.044+0000 7f2ebffff640 1 --2- 172.21.15.43:0/3696596918 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f2ea80047c0 0x7f2ea8004c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:33:58.050 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:33:58.044+0000 7f2ebffff640 1 -- 172.21.15.43:0/3696596918 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2ea8003520 con 0x7f2ea8003b70 2023-08-09T13:33:58.050 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:33:58.045+0000 7f2ebffff640 1 --2- 172.21.15.43:0/3696596918 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f2ea8003b70 0x7f2ea8003f70 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7f2eb0001eb0 tx=0x7f2ea80057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f923ccb7561ee53c server_cookie=0 in_seq=0 out_seq=0 2023-08-09T13:33:58.051 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:33:58.045+0000 7f2ebeffd640 1 -- 172.21.15.43:0/3696596918 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2eb0005ad0 con 0x7f2ea8003b70 2023-08-09T13:33:58.051 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:33:58.045+0000 7f2ebeffd640 1 -- 172.21.15.43:0/3696596918 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2eb0005c50 con 0x7f2ea8003b70 2023-08-09T13:33:58.051 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:33:58.045+0000 7f2ebeffd640 1 -- 172.21.15.43:0/3696596918 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2eb002c7f0 con 0x7f2ea8003b70 2023-08-09T13:33:58.051 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:33:58.046+0000 7f2ec6a60640 1 -- 172.21.15.43:0/3696596918 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f2ea8003b70 msgr2=0x7f2ea8003f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:33:58.051 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:33:58.046+0000 7f2ec6a60640 1 --2- 172.21.15.43:0/3696596918 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f2ea8003b70 0x7f2ea8003f70 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7f2eb0001eb0 tx=0x7f2ea80057f0 comp rx=0 tx=0).stop 2023-08-09T13:33:58.051 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:33:58.046+0000 7f2ec4ffd640 1 -- 172.21.15.43:0/3696596918 reap_dead start 2023-08-09T13:33:58.051 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:33:58.046+0000 7f2ec6a60640 1 -- 172.21.15.43:0/3696596918 shutdown_connections 2023-08-09T13:33:58.051 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:33:58.046+0000 7f2ec6a60640 1 -- 172.21.15.43:0/3696596918 >> 172.21.15.43:0/3696596918 conn(0x7f2ec009b7b0 msgr2=0x7f2ec00d24e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-09T13:33:58.052 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:33:58.046+0000 7f2ec6a60640 1 -- 172.21.15.43:0/3696596918 shutdown_connections 2023-08-09T13:33:58.052 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:33:58.046+0000 7f2ec6a60640 1 -- 172.21.15.43:0/3696596918 wait complete. 2023-08-09T13:33:58.053 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:33:58.048+0000 7f2ec6a60640 1 Processor -- start 2023-08-09T13:33:58.053 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:33:58.048+0000 7f2ec6a60640 1 -- start start 2023-08-09T13:33:58.054 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:33:58.049+0000 7f2ec6a60640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f2ea8003b70 0x7f2ec0154b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:33:58.054 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:33:58.049+0000 7f2ec6a60640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f2ec00af0a0 0x7f2ec00af520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:33:58.054 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:33:58.049+0000 7f2ec6a60640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f2ec00b1560 0x7f2ec00b19e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:33:58.054 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:33:58.049+0000 7f2ec6a60640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2ec00b1f20 con 0x7f2ec00b1560 2023-08-09T13:33:58.054 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:33:58.049+0000 7f2ec6a60640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2ec00b0820 con 0x7f2ec00af0a0 2023-08-09T13:33:58.054 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:33:58.049+0000 7f2ec6a60640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2ec00afc10 con 0x7f2ea8003b70 2023-08-09T13:33:58.054 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:33:58.049+0000 7f2ec4ffd640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f2ec00b1560 0x7f2ec00b19e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:33:58.055 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:33:58.049+0000 7f2ebf7fe640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f2ec00af0a0 0x7f2ec00af520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:33:58.055 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:33:58.049+0000 7f2ec4ffd640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f2ec00b1560 0x7f2ec00b19e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:57678/0 (socket says 172.21.15.43:57678) 2023-08-09T13:33:58.055 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:33:58.049+0000 7f2ec4ffd640 1 -- 172.21.15.43:0/1499222947 learned_addr learned my addr 172.21.15.43:0/1499222947 (peer_addr_for_me v2:172.21.15.43:0/0) 2023-08-09T13:33:58.055 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:33:58.049+0000 7f2ebffff640 1 --2- 172.21.15.43:0/1499222947 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f2ea8003b70 0x7f2ec0154b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:33:58.055 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:33:58.049+0000 7f2ebf7fe640 1 -- 172.21.15.43:0/1499222947 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f2ea8003b70 msgr2=0x7f2ec0154b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:33:58.055 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:33:58.049+0000 7f2ebf7fe640 1 --2- 172.21.15.43:0/1499222947 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f2ea8003b70 0x7f2ec0154b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:33:58.055 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:33:58.049+0000 7f2ebf7fe640 1 -- 172.21.15.43:0/1499222947 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f2ec00b1560 msgr2=0x7f2ec00b19e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:33:58.055 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:33:58.049+0000 7f2ebf7fe640 1 --2- 172.21.15.43:0/1499222947 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f2ec00b1560 0x7f2ec00b19e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:33:58.056 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:33:58.050+0000 7f2ebf7fe640 1 -- 172.21.15.43:0/1499222947 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2ec00afef0 con 0x7f2ec00af0a0 2023-08-09T13:33:58.056 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:33:58.050+0000 7f2ec4ffd640 1 --2- 172.21.15.43:0/1499222947 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f2ec00b1560 0x7f2ec00b19e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-08-09T13:33:58.056 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:33:58.050+0000 7f2ebffff640 1 --2- 172.21.15.43:0/1499222947 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f2ea8003b70 0x7f2ec0154b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-08-09T13:33:58.056 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:33:58.050+0000 7f2ebf7fe640 1 --2- 172.21.15.43:0/1499222947 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f2ec00af0a0 0x7f2ec00af520 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7f2eb4011960 tx=0x7f2eb4011e20 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-09T13:33:58.056 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:33:58.050+0000 7f2ebcff9640 1 -- 172.21.15.43:0/1499222947 <== mon.1 v2:172.21.15.203:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2eb4004300 con 0x7f2ec00af0a0 2023-08-09T13:33:58.056 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:33:58.050+0000 7f2ebcff9640 1 -- 172.21.15.43:0/1499222947 <== mon.1 v2:172.21.15.203:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2eb4004460 con 0x7f2ec00af0a0 2023-08-09T13:33:58.056 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:33:58.050+0000 7f2ebcff9640 1 -- 172.21.15.43:0/1499222947 <== mon.1 v2:172.21.15.203:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2eb400fa40 con 0x7f2ec00af0a0 2023-08-09T13:33:58.057 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:33:58.050+0000 7f2ec6a60640 1 -- 172.21.15.43:0/1499222947 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2ec00b01d0 con 0x7f2ec00af0a0 2023-08-09T13:33:58.057 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:33:58.050+0000 7f2ec6a60640 1 -- 172.21.15.43:0/1499222947 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2ec00c77f0 con 0x7f2ec00af0a0 2023-08-09T13:33:58.057 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:33:58.051+0000 7f2ec6a60640 1 -- 172.21.15.43:0/1499222947 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2ec00739f0 con 0x7f2ec00af0a0 2023-08-09T13:33:58.058 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:33:58.052+0000 7f2ebcff9640 1 -- 172.21.15.43:0/1499222947 <== mon.1 v2:172.21.15.203:3300/0 4 ==== mgrmap(e 5) v1 ==== 88341+0+0 (secure 0 0 0) 0x7f2eb400fba0 con 0x7f2ec00af0a0 2023-08-09T13:33:58.058 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:33:58.053+0000 7f2ebcff9640 1 --2- 172.21.15.43:0/1499222947 >> [v2:172.21.15.203:6832/830097474,v1:172.21.15.203:6833/830097474] conn(0x7f2eac06a080 0x7f2eac06c540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:33:58.058 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:33:58.054+0000 7f2ebcff9640 1 -- 172.21.15.43:0/1499222947 <== mon.1 v2:172.21.15.203:3300/0 5 ==== osd_map(16..16 src has 1..16) v4 ==== 5210+0+0 (secure 0 0 0) 0x7f2eb4088870 con 0x7f2ec00af0a0 2023-08-09T13:33:58.058 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:33:58.054+0000 7f2ebffff640 1 --2- 172.21.15.43:0/1499222947 >> [v2:172.21.15.203:6832/830097474,v1:172.21.15.203:6833/830097474] conn(0x7f2eac06a080 0x7f2eac06c540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:33:58.058 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:33:58.054+0000 7f2ebffff640 1 --2- 172.21.15.43:0/1499222947 >> [v2:172.21.15.203:6832/830097474,v1:172.21.15.203:6833/830097474] conn(0x7f2eac06a080 0x7f2eac06c540 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f2eb0034000 tx=0x7f2eb0007000 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-09T13:33:58.072 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:33:58.070+0000 7f2ebcff9640 1 -- 172.21.15.43:0/1499222947 <== mon.1 v2:172.21.15.203:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178066 (secure 0 0 0) 0x7f2eb40542c0 con 0x7f2ec00af0a0 2023-08-09T13:33:58.270 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:33:58.267+0000 7f2ec6a60640 1 -- 172.21.15.43:0/1499222947 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f2ec00b0960 con 0x7f2ec00af0a0 2023-08-09T13:33:58.272 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:33:58.270+0000 7f2ebcff9640 1 -- 172.21.15.43:0/1499222947 <== mon.1 v2:172.21.15.203:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v16) v1 ==== 74+0+14567 (secure 0 0 0) 0x7f2eb40578e0 con 0x7f2ec00af0a0 2023-08-09T13:33:58.273 INFO:teuthology.orchestra.run.smithi043.stdout: 2023-08-09T13:33:58.273 INFO:teuthology.orchestra.run.smithi043.stdout:{"epoch":16,"fsid":"daac87ee-1337-4250-a955-aea4e408a39b","created":"2023-08-09T13:33:41.701198+0000","modified":"2023-08-09T13:33:56.869470+0000","last_up_change":"2023-08-09T13:33:53.783453+0000","last_in_change":"2023-08-09T13:33:45.148904+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":2,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":"rbd","create_time":"2023-08-09T13:33:54.365137+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":".mgr","create_time":"2023-08-09T13:33:54.831803+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"cbd49ecd-e896-4fbd-864e-5c37a829b165","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6800","nonce":502667335},{"type":"v1","addr":"172.21.15.43:6801","nonce":502667335}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6802","nonce":502667335},{"type":"v1","addr":"172.21.15.43:6803","nonce":502667335}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6806","nonce":502667335},{"type":"v1","addr":"172.21.15.43:6807","nonce":502667335}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6804","nonce":502667335},{"type":"v1","addr":"172.21.15.43:6805","nonce":502667335}]},"public_addr":"172.21.15.43:6801/502667335","cluster_addr":"172.21.15.43:6803/502667335","heartbeat_back_addr":"172.21.15.43:6807/502667335","heartbeat_front_addr":"172.21.15.43:6805/502667335","state":["exists","up"]},{"osd":1,"uuid":"75c7e0fe-1ec7-4fe8-a2f2-3661a65b8500","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":14,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6808","nonce":3289134610},{"type":"v1","addr":"172.21.15.43:6809","nonce":3289134610}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6810","nonce":3289134610},{"type":"v1","addr":"172.21.15.43:6812","nonce":3289134610}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6821","nonce":3289134610},{"type":"v1","addr":"172.21.15.43:6824","nonce":3289134610}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6815","nonce":3289134610},{"type":"v1","addr":"172.21.15.43:6818","nonce":3289134610}]},"public_addr":"172.21.15.43:6809/3289134610","cluster_addr":"172.21.15.43:6812/3289134610","heartbeat_back_addr":"172.21.15.43:6824/3289134610","heartbeat_front_addr":"172.21.15.43:6818/3289134610","state":["exists","up"]},{"osd":2,"uuid":"bb5c41e9-7f5f-4125-9b01-3bb8bfc2ea83","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6811","nonce":4079709736},{"type":"v1","addr":"172.21.15.43:6814","nonce":4079709736}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6817","nonce":4079709736},{"type":"v1","addr":"172.21.15.43:6820","nonce":4079709736}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6829","nonce":4079709736},{"type":"v1","addr":"172.21.15.43:6831","nonce":4079709736}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6823","nonce":4079709736},{"type":"v1","addr":"172.21.15.43:6826","nonce":4079709736}]},"public_addr":"172.21.15.43:6814/4079709736","cluster_addr":"172.21.15.43:6820/4079709736","heartbeat_back_addr":"172.21.15.43:6831/4079709736","heartbeat_front_addr":"172.21.15.43:6826/4079709736","state":["exists","up"]},{"osd":3,"uuid":"bfe4c8b4-694a-465c-87e6-2b2aab852613","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":14,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6813","nonce":1709538759},{"type":"v1","addr":"172.21.15.43:6816","nonce":1709538759}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6819","nonce":1709538759},{"type":"v1","addr":"172.21.15.43:6822","nonce":1709538759}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6828","nonce":1709538759},{"type":"v1","addr":"172.21.15.43:6830","nonce":1709538759}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6825","nonce":1709538759},{"type":"v1","addr":"172.21.15.43:6827","nonce":1709538759}]},"public_addr":"172.21.15.43:6816/1709538759","cluster_addr":"172.21.15.43:6822/1709538759","heartbeat_back_addr":"172.21.15.43:6830/1709538759","heartbeat_front_addr":"172.21.15.43:6827/1709538759","state":["exists","up"]},{"osd":4,"uuid":"fa2e8a77-aba1-4756-853e-0d174a5bdeba","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":14,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6800","nonce":4196866481},{"type":"v1","addr":"172.21.15.203:6801","nonce":4196866481}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6802","nonce":4196866481},{"type":"v1","addr":"172.21.15.203:6803","nonce":4196866481}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6806","nonce":4196866481},{"type":"v1","addr":"172.21.15.203:6807","nonce":4196866481}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6804","nonce":4196866481},{"type":"v1","addr":"172.21.15.203:6805","nonce":4196866481}]},"public_addr":"172.21.15.203:6801/4196866481","cluster_addr":"172.21.15.203:6803/4196866481","heartbeat_back_addr":"172.21.15.203:6807/4196866481","heartbeat_front_addr":"172.21.15.203:6805/4196866481","state":["exists","up"]},{"osd":5,"uuid":"81811e4b-4d24-4f51-a82c-f1d700a2a00d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":14,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6816","nonce":3059290081},{"type":"v1","addr":"172.21.15.203:6817","nonce":3059290081}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6818","nonce":3059290081},{"type":"v1","addr":"172.21.15.203:6819","nonce":3059290081}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6822","nonce":3059290081},{"type":"v1","addr":"172.21.15.203:6823","nonce":3059290081}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6820","nonce":3059290081},{"type":"v1","addr":"172.21.15.203:6821","nonce":3059290081}]},"public_addr":"172.21.15.203:6817/3059290081","cluster_addr":"172.21.15.203:6819/3059290081","heartbeat_back_addr":"172.21.15.203:6823/3059290081","heartbeat_front_addr":"172.21.15.203:6821/3059290081","state":["exists","up"]},{"osd":6,"uuid":"307bac4d-6a5b-443d-af75-56d695d7674f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":14,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6824","nonce":512155569},{"type":"v1","addr":"172.21.15.203:6825","nonce":512155569}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6826","nonce":512155569},{"type":"v1","addr":"172.21.15.203:6827","nonce":512155569}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6830","nonce":512155569},{"type":"v1","addr":"172.21.15.203:6831","nonce":512155569}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6828","nonce":512155569},{"type":"v1","addr":"172.21.15.203:6829","nonce":512155569}]},"public_addr":"172.21.15.203:6825/512155569","cluster_addr":"172.21.15.203:6827/512155569","heartbeat_back_addr":"172.21.15.203:6831/512155569","heartbeat_front_addr":"172.21.15.203:6829/512155569","state":["exists","up"]},{"osd":7,"uuid":"a95dc7ca-9f1b-4295-af70-68a22a2d7d5e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":14,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6808","nonce":3331450438},{"type":"v1","addr":"172.21.15.203:6809","nonce":3331450438}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6810","nonce":3331450438},{"type":"v1","addr":"172.21.15.203:6811","nonce":3331450438}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6814","nonce":3331450438},{"type":"v1","addr":"172.21.15.203:6815","nonce":3331450438}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6812","nonce":3331450438},{"type":"v1","addr":"172.21.15.203:6813","nonce":3331450438}]},"public_addr":"172.21.15.203:6809/3331450438","cluster_addr":"172.21.15.203:6811/3331450438","heartbeat_back_addr":"172.21.15.203:6815/3331450438","heartbeat_front_addr":"172.21.15.203:6813/3331450438","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-09T13:33:50.547652+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-09T13:33:50.589469+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-09T13:33:50.561724+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-09T13:33:50.599650+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-09T13:33:50.411855+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-09T13:33:50.219681+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-09T13:33:50.504664+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-09T13:33:50.441248+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-08-09T13:33:58.275 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:33:58.273+0000 7f2ec6a60640 1 -- 172.21.15.43:0/1499222947 >> [v2:172.21.15.203:6832/830097474,v1:172.21.15.203:6833/830097474] conn(0x7f2eac06a080 msgr2=0x7f2eac06c540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:33:58.276 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:33:58.273+0000 7f2ec6a60640 1 --2- 172.21.15.43:0/1499222947 >> [v2:172.21.15.203:6832/830097474,v1:172.21.15.203:6833/830097474] conn(0x7f2eac06a080 0x7f2eac06c540 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f2eb0034000 tx=0x7f2eb0007000 comp rx=0 tx=0).stop 2023-08-09T13:33:58.276 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:33:58.273+0000 7f2ec6a60640 1 -- 172.21.15.43:0/1499222947 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f2ec00af0a0 msgr2=0x7f2ec00af520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:33:58.276 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:33:58.273+0000 7f2ec6a60640 1 --2- 172.21.15.43:0/1499222947 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f2ec00af0a0 0x7f2ec00af520 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7f2eb4011960 tx=0x7f2eb4011e20 comp rx=0 tx=0).stop 2023-08-09T13:33:58.276 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:33:58.273+0000 7f2ec6a60640 1 -- 172.21.15.43:0/1499222947 shutdown_connections 2023-08-09T13:33:58.276 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:33:58.273+0000 7f2ec6a60640 1 --2- 172.21.15.43:0/1499222947 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f2ec00b1560 0x7f2ec00b19e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:33:58.276 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:33:58.273+0000 7f2ec6a60640 1 --2- 172.21.15.43:0/1499222947 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f2ec00af0a0 0x7f2ec00af520 unknown :-1 s=CLOSED pgs=42 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:33:58.277 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:33:58.273+0000 7f2ec6a60640 1 --2- 172.21.15.43:0/1499222947 >> [v2:172.21.15.203:6832/830097474,v1:172.21.15.203:6833/830097474] conn(0x7f2eac06a080 0x7f2eac06c540 unknown :-1 s=CLOSED pgs=54 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:33:58.277 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:33:58.273+0000 7f2ec6a60640 1 --2- 172.21.15.43:0/1499222947 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f2ea8003b70 0x7f2ec0154b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:33:58.277 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:33:58.273+0000 7f2ec6a60640 1 -- 172.21.15.43:0/1499222947 >> 172.21.15.43:0/1499222947 conn(0x7f2ec009b7b0 msgr2=0x7f2ec0160a30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-09T13:33:58.277 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:33:58.273+0000 7f2ec6a60640 1 -- 172.21.15.43:0/1499222947 shutdown_connections 2023-08-09T13:33:58.277 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:33:58.273+0000 7f2ec6a60640 1 -- 172.21.15.43:0/1499222947 wait complete. 2023-08-09T13:33:58.293 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': 'rbd', 'create_time': '2023-08-09T13:33:54.365137+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}, 'read_balance': {'score_acting': 2, 'score_stable': 2, 'optimal_score': 1, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': '.mgr', 'create_time': '2023-08-09T13:33:54.831803+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-08-09T13:33:58.294 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get rbd pg_num 2023-08-09T13:33:58.547 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:33:58.545+0000 7f9b25cad640 1 Processor -- start 2023-08-09T13:33:58.548 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:33:58.545+0000 7f9b25cad640 1 -- start start 2023-08-09T13:33:58.549 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:33:58.546+0000 7f9b25cad640 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7f9b2015c620 0x7f9b200cceb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:33:58.549 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:33:58.546+0000 7f9b25cad640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f9b200b1260 0x7f9b200cee00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:33:58.549 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:33:58.546+0000 7f9b25cad640 1 -- --> v1:172.21.15.203:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9b200cf750 con 0x7f9b200cf340 2023-08-09T13:33:58.549 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:33:58.547+0000 7f9b25cad640 1 -- --> v2:172.21.15.43:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9b200b1b70 con 0x7f9b200b1260 2023-08-09T13:33:58.549 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:33:58.547+0000 7f9b25cad640 1 -- --> v2:172.21.15.203:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9b200d1360 con 0x7f9b2015c620 2023-08-09T13:33:58.550 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:33:58.547+0000 7f9b1effd640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f9b200b1260 0x7f9b200cee00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:33:58.550 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:33:58.547+0000 7f9b1effd640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f9b200b1260 0x7f9b200cee00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:57680/0 (socket says 172.21.15.43:57680) 2023-08-09T13:33:58.551 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:33:58.547+0000 7f9b1effd640 1 -- 172.21.15.43:0/2703948005 learned_addr learned my addr 172.21.15.43:0/2703948005 (peer_addr_for_me v2:172.21.15.43:0/0) 2023-08-09T13:33:58.551 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:33:58.547+0000 7f9b1f7fe640 1 --2- 172.21.15.43:0/2703948005 >> v2:172.21.15.203:3300/0 conn(0x7f9b2015c620 0x7f9b200cceb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:33:58.552 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:33:58.547+0000 7f9b1effd640 1 -- 172.21.15.43:0/2703948005 >> v2:172.21.15.203:3300/0 conn(0x7f9b2015c620 msgr2=0x7f9b200cceb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-09T13:33:58.553 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:33:58.547+0000 7f9b1effd640 1 --2- 172.21.15.43:0/2703948005 >> v2:172.21.15.203:3300/0 conn(0x7f9b2015c620 0x7f9b200cceb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:33:58.553 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:33:58.547+0000 7f9b1effd640 1 -- 172.21.15.43:0/2703948005 >> v1:172.21.15.203:6789/0 conn(0x7f9b200cf340 legacy=0x7f9b20160280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-09T13:33:58.553 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:33:58.547+0000 7f9b1effd640 1 -- 172.21.15.43:0/2703948005 --> v2:172.21.15.43:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9b200cd700 con 0x7f9b200b1260 2023-08-09T13:33:58.555 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:33:58.549+0000 7f9b1effd640 1 --2- 172.21.15.43:0/2703948005 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f9b200b1260 0x7f9b200cee00 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7f9b2008a6d0 tx=0x7f9b1402b7f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=429794ffd1111b42 server_cookie=0 in_seq=0 out_seq=0 2023-08-09T13:33:58.555 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:33:58.549+0000 7f9b1e7fc640 1 -- 172.21.15.43:0/2703948005 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9b14031750 con 0x7f9b200b1260 2023-08-09T13:33:58.555 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:33:58.549+0000 7f9b1e7fc640 1 -- 172.21.15.43:0/2703948005 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9b140318b0 con 0x7f9b200b1260 2023-08-09T13:33:58.555 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:33:58.549+0000 7f9b1e7fc640 1 -- 172.21.15.43:0/2703948005 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9b14031c40 con 0x7f9b200b1260 2023-08-09T13:33:58.556 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:33:58.550+0000 7f9b25cad640 1 -- 172.21.15.43:0/2703948005 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f9b200b1260 msgr2=0x7f9b200cee00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:33:58.556 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:33:58.550+0000 7f9b25cad640 1 --2- 172.21.15.43:0/2703948005 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f9b200b1260 0x7f9b200cee00 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7f9b2008a6d0 tx=0x7f9b1402b7f0 comp rx=0 tx=0).stop 2023-08-09T13:33:58.556 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:33:58.550+0000 7f9b25cad640 1 -- 172.21.15.43:0/2703948005 shutdown_connections 2023-08-09T13:33:58.556 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:33:58.550+0000 7f9b25cad640 1 --2- 172.21.15.43:0/2703948005 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f9b200b1260 0x7f9b200cee00 unknown :-1 s=CLOSED pgs=50 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:33:58.556 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:33:58.550+0000 7f9b25cad640 1 --2- 172.21.15.43:0/2703948005 >> v2:172.21.15.203:3300/0 conn(0x7f9b2015c620 0x7f9b200cceb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:33:58.557 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:33:58.550+0000 7f9b25cad640 1 -- 172.21.15.43:0/2703948005 >> 172.21.15.43:0/2703948005 conn(0x7f9b2009b750 msgr2=0x7f9b200d2410 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-09T13:33:58.557 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:33:58.550+0000 7f9b25cad640 1 -- 172.21.15.43:0/2703948005 shutdown_connections 2023-08-09T13:33:58.557 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:33:58.550+0000 7f9b25cad640 1 -- 172.21.15.43:0/2703948005 wait complete. 2023-08-09T13:33:58.557 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:33:58.552+0000 7f9b25cad640 1 Processor -- start 2023-08-09T13:33:58.557 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:33:58.552+0000 7f9b25cad640 1 -- start start 2023-08-09T13:33:58.559 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:33:58.553+0000 7f9b25cad640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f9b200b1260 0x7f9b2015f700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:33:58.560 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:33:58.553+0000 7f9b25cad640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f9b200cf340 0x7f9b200c8920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:33:58.560 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:33:58.553+0000 7f9b25cad640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f9b2015c620 0x7f9b2015cd10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:33:58.560 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:33:58.553+0000 7f9b25cad640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9b200cb4e0 con 0x7f9b2015c620 2023-08-09T13:33:58.560 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:33:58.553+0000 7f9b25cad640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9b200b7b90 con 0x7f9b200cf340 2023-08-09T13:33:58.560 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:33:58.553+0000 7f9b25cad640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9b200b7e70 con 0x7f9b200b1260 2023-08-09T13:33:58.561 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:33:58.553+0000 7f9b1ffff640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f9b2015c620 0x7f9b2015cd10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:33:58.561 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:33:58.553+0000 7f9b1ffff640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f9b2015c620 0x7f9b2015cd10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:57682/0 (socket says 172.21.15.43:57682) 2023-08-09T13:33:58.561 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:33:58.553+0000 7f9b1ffff640 1 -- 172.21.15.43:0/536498013 learned_addr learned my addr 172.21.15.43:0/536498013 (peer_addr_for_me v2:172.21.15.43:0/0) 2023-08-09T13:33:58.561 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:33:58.554+0000 7f9b1f7fe640 1 --2- 172.21.15.43:0/536498013 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f9b200b1260 0x7f9b2015f700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:33:58.561 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:33:58.554+0000 7f9b1ffff640 1 -- 172.21.15.43:0/536498013 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f9b200b1260 msgr2=0x7f9b2015f700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:33:58.562 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:33:58.554+0000 7f9b1ffff640 1 --2- 172.21.15.43:0/536498013 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f9b200b1260 0x7f9b2015f700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:33:58.562 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:33:58.554+0000 7f9b1ffff640 1 -- 172.21.15.43:0/536498013 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f9b200cf340 msgr2=0x7f9b200c8920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:33:58.562 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:33:58.554+0000 7f9b1ffff640 1 --2- 172.21.15.43:0/536498013 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f9b200cf340 0x7f9b200c8920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:33:58.562 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:33:58.554+0000 7f9b1ffff640 1 -- 172.21.15.43:0/536498013 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9b140094d0 con 0x7f9b2015c620 2023-08-09T13:33:58.562 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:33:58.554+0000 7f9b1ffff640 1 --2- 172.21.15.43:0/536498013 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f9b2015c620 0x7f9b2015cd10 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7f9b1800c780 tx=0x7f9b1800cc40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-09T13:33:58.562 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:33:58.554+0000 7f9b07fff640 1 -- 172.21.15.43:0/536498013 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9b18004b60 con 0x7f9b2015c620 2023-08-09T13:33:58.563 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:33:58.554+0000 7f9b07fff640 1 -- 172.21.15.43:0/536498013 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9b1800ce90 con 0x7f9b2015c620 2023-08-09T13:33:58.563 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:33:58.554+0000 7f9b07fff640 1 -- 172.21.15.43:0/536498013 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9b1800a850 con 0x7f9b2015c620 2023-08-09T13:33:58.563 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:33:58.555+0000 7f9b25cad640 1 -- 172.21.15.43:0/536498013 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9b200b8150 con 0x7f9b2015c620 2023-08-09T13:33:58.563 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:33:58.555+0000 7f9b25cad640 1 -- 172.21.15.43:0/536498013 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9b2015d250 con 0x7f9b2015c620 2023-08-09T13:33:58.563 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:33:58.556+0000 7f9b25cad640 1 -- 172.21.15.43:0/536498013 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9b200cb650 con 0x7f9b2015c620 2023-08-09T13:33:58.564 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:33:58.557+0000 7f9b07fff640 1 -- 172.21.15.43:0/536498013 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 5) v1 ==== 88341+0+0 (secure 0 0 0) 0x7f9b1800a9b0 con 0x7f9b2015c620 2023-08-09T13:33:58.564 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:33:58.558+0000 7f9b07fff640 1 --2- 172.21.15.43:0/536498013 >> [v2:172.21.15.203:6832/830097474,v1:172.21.15.203:6833/830097474] conn(0x7f9b0c06a080 0x7f9b0c06c540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:33:58.565 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:33:58.558+0000 7f9b07fff640 1 -- 172.21.15.43:0/536498013 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(16..16 src has 1..16) v4 ==== 5210+0+0 (secure 0 0 0) 0x7f9b18088500 con 0x7f9b2015c620 2023-08-09T13:33:58.565 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:33:58.559+0000 7f9b1f7fe640 1 --2- 172.21.15.43:0/536498013 >> [v2:172.21.15.203:6832/830097474,v1:172.21.15.203:6833/830097474] conn(0x7f9b0c06a080 0x7f9b0c06c540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:33:58.565 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:33:58.560+0000 7f9b1f7fe640 1 --2- 172.21.15.43:0/536498013 >> [v2:172.21.15.203:6832/830097474,v1:172.21.15.203:6833/830097474] conn(0x7f9b0c06a080 0x7f9b0c06c540 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f9b1000b9b0 tx=0x7f9b10008040 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-09T13:33:58.567 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:33:58.565+0000 7f9b07fff640 1 -- 172.21.15.43:0/536498013 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178066 (secure 0 0 0) 0x7f9b18053ea0 con 0x7f9b2015c620 2023-08-09T13:33:58.760 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:33:58.757+0000 7f9b25cad640 1 -- 172.21.15.43:0/536498013 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"} v 0) v1 -- 0x7f9b200739f0 con 0x7f9b2015c620 2023-08-09T13:33:58.760 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:33:58.758+0000 7f9b07fff640 1 -- 172.21.15.43:0/536498013 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"}]=0 v16) v1 ==== 92+0+10 (secure 0 0 0) 0x7f9b180574c0 con 0x7f9b2015c620 2023-08-09T13:33:58.761 INFO:teuthology.orchestra.run.smithi043.stdout:pg_num: 8 2023-08-09T13:33:58.763 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:33:58.761+0000 7f9b25cad640 1 -- 172.21.15.43:0/536498013 >> [v2:172.21.15.203:6832/830097474,v1:172.21.15.203:6833/830097474] conn(0x7f9b0c06a080 msgr2=0x7f9b0c06c540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:33:58.764 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:33:58.761+0000 7f9b25cad640 1 --2- 172.21.15.43:0/536498013 >> [v2:172.21.15.203:6832/830097474,v1:172.21.15.203:6833/830097474] conn(0x7f9b0c06a080 0x7f9b0c06c540 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f9b1000b9b0 tx=0x7f9b10008040 comp rx=0 tx=0).stop 2023-08-09T13:33:58.764 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:33:58.761+0000 7f9b25cad640 1 -- 172.21.15.43:0/536498013 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f9b2015c620 msgr2=0x7f9b2015cd10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:33:58.764 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:33:58.761+0000 7f9b25cad640 1 --2- 172.21.15.43:0/536498013 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f9b2015c620 0x7f9b2015cd10 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7f9b1800c780 tx=0x7f9b1800cc40 comp rx=0 tx=0).stop 2023-08-09T13:33:58.764 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:33:58.761+0000 7f9b25cad640 1 -- 172.21.15.43:0/536498013 shutdown_connections 2023-08-09T13:33:58.764 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:33:58.761+0000 7f9b25cad640 1 --2- 172.21.15.43:0/536498013 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f9b2015c620 0x7f9b2015cd10 unknown :-1 s=CLOSED pgs=51 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:33:58.764 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:33:58.761+0000 7f9b25cad640 1 --2- 172.21.15.43:0/536498013 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f9b200cf340 0x7f9b200c8920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:33:58.764 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:33:58.761+0000 7f9b25cad640 1 --2- 172.21.15.43:0/536498013 >> [v2:172.21.15.203:6832/830097474,v1:172.21.15.203:6833/830097474] conn(0x7f9b0c06a080 0x7f9b0c06c540 unknown :-1 s=CLOSED pgs=56 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:33:58.765 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:33:58.761+0000 7f9b25cad640 1 --2- 172.21.15.43:0/536498013 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f9b200b1260 0x7f9b2015f700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:33:58.765 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:33:58.761+0000 7f9b25cad640 1 -- 172.21.15.43:0/536498013 >> 172.21.15.43:0/536498013 conn(0x7f9b2009b750 msgr2=0x7f9b2006b430 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-09T13:33:58.765 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:33:58.761+0000 7f9b25cad640 1 -- 172.21.15.43:0/536498013 shutdown_connections 2023-08-09T13:33:58.765 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:33:58.761+0000 7f9b25cad640 1 -- 172.21.15.43:0/536498013 wait complete. 2023-08-09T13:33:58.783 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-08-09T13:33:59.012 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:33:59.009+0000 7f595b283640 1 Processor -- start 2023-08-09T13:33:59.013 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:33:59.010+0000 7f595b283640 1 -- start start 2023-08-09T13:33:59.013 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:33:59.011+0000 7f595b283640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f595415c550 0x7f59540cd6d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:33:59.013 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:33:59.011+0000 7f595b283640 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7f59540d04b0 0x7f59540b3ee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:33:59.014 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:33:59.011+0000 7f595b283640 1 -- --> v1:172.21.15.203:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f59540b7bd0 con 0x7f59540b4420 2023-08-09T13:33:59.014 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:33:59.011+0000 7f595b283640 1 -- --> v2:172.21.15.43:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f59540b4850 con 0x7f595415c550 2023-08-09T13:33:59.014 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:33:59.011+0000 7f595b283640 1 -- --> v2:172.21.15.203:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f59540d1c00 con 0x7f59540d04b0 2023-08-09T13:33:59.015 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:33:59.011+0000 7f595901f640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f595415c550 0x7f59540cd6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:33:59.015 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:33:59.011+0000 7f595901f640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f595415c550 0x7f59540cd6d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:57686/0 (socket says 172.21.15.43:57686) 2023-08-09T13:33:59.015 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:33:59.011+0000 7f595901f640 1 -- 172.21.15.43:0/19748504 learned_addr learned my addr 172.21.15.43:0/19748504 (peer_addr_for_me v2:172.21.15.43:0/0) 2023-08-09T13:33:59.015 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:33:59.011+0000 7f595881e640 1 --2- 172.21.15.43:0/19748504 >> v2:172.21.15.203:3300/0 conn(0x7f59540d04b0 0x7f59540b3ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:33:59.015 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:33:59.012+0000 7f595901f640 1 -- 172.21.15.43:0/19748504 >> v2:172.21.15.203:3300/0 conn(0x7f59540d04b0 msgr2=0x7f59540b3ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-09T13:33:59.015 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:33:59.012+0000 7f595901f640 1 --2- 172.21.15.43:0/19748504 >> v2:172.21.15.203:3300/0 conn(0x7f59540d04b0 0x7f59540b3ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:33:59.016 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:33:59.012+0000 7f595901f640 1 -- 172.21.15.43:0/19748504 >> v1:172.21.15.203:6789/0 conn(0x7f59540b4420 legacy=0x7f595406a760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-09T13:33:59.016 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:33:59.012+0000 7f595901f640 1 -- 172.21.15.43:0/19748504 --> v2:172.21.15.43:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f595415c930 con 0x7f595415c550 2023-08-09T13:33:59.017 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:33:59.013+0000 7f595901f640 1 --2- 172.21.15.43:0/19748504 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f595415c550 0x7f59540cd6d0 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7f59540b5050 tx=0x7f595002b880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1905b41826cef3e7 server_cookie=0 in_seq=0 out_seq=0 2023-08-09T13:33:59.017 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:33:59.013+0000 7f594bfff640 1 -- 172.21.15.43:0/19748504 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f59500317e0 con 0x7f595415c550 2023-08-09T13:33:59.017 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:33:59.013+0000 7f594bfff640 1 -- 172.21.15.43:0/19748504 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5950031940 con 0x7f595415c550 2023-08-09T13:33:59.017 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:33:59.013+0000 7f594bfff640 1 -- 172.21.15.43:0/19748504 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5950031cd0 con 0x7f595415c550 2023-08-09T13:33:59.017 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:33:59.014+0000 7f595b283640 1 -- 172.21.15.43:0/19748504 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f595415c550 msgr2=0x7f59540cd6d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:33:59.018 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:33:59.014+0000 7f595b283640 1 --2- 172.21.15.43:0/19748504 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f595415c550 0x7f59540cd6d0 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7f59540b5050 tx=0x7f595002b880 comp rx=0 tx=0).stop 2023-08-09T13:33:59.018 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:33:59.014+0000 7f595b283640 1 -- 172.21.15.43:0/19748504 shutdown_connections 2023-08-09T13:33:59.018 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:33:59.014+0000 7f595b283640 1 --2- 172.21.15.43:0/19748504 >> v2:172.21.15.203:3300/0 conn(0x7f59540d04b0 0x7f59540b3ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:33:59.018 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:33:59.014+0000 7f595b283640 1 --2- 172.21.15.43:0/19748504 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f595415c550 0x7f59540cd6d0 unknown :-1 s=CLOSED pgs=52 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:33:59.018 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:33:59.014+0000 7f595b283640 1 -- 172.21.15.43:0/19748504 >> 172.21.15.43:0/19748504 conn(0x7f5954071520 msgr2=0x7f59540ceee0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-09T13:33:59.018 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:33:59.014+0000 7f595b283640 1 -- 172.21.15.43:0/19748504 shutdown_connections 2023-08-09T13:33:59.018 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:33:59.014+0000 7f595b283640 1 -- 172.21.15.43:0/19748504 wait complete. 2023-08-09T13:33:59.019 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:33:59.016+0000 7f595b283640 1 Processor -- start 2023-08-09T13:33:59.019 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:33:59.016+0000 7f595b283640 1 -- start start 2023-08-09T13:33:59.020 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:33:59.017+0000 7f595b283640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f59540b4420 0x7f59540ab8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:33:59.020 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:33:59.017+0000 7f595b283640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f59540d04b0 0x7f59540ca580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:33:59.020 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:33:59.017+0000 7f595b283640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f595415c550 0x7f5954158640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:33:59.020 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:33:59.017+0000 7f595b283640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f59540cbc30 con 0x7f595415c550 2023-08-09T13:33:59.020 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:33:59.017+0000 7f595b283640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f59540cb020 con 0x7f59540d04b0 2023-08-09T13:33:59.021 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:33:59.017+0000 7f595b283640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f59540cb300 con 0x7f59540b4420 2023-08-09T13:33:59.021 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:33:59.017+0000 7f5959820640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f595415c550 0x7f5954158640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:33:59.021 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:33:59.017+0000 7f5959820640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f595415c550 0x7f5954158640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:57702/0 (socket says 172.21.15.43:57702) 2023-08-09T13:33:59.021 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:33:59.017+0000 7f5959820640 1 -- 172.21.15.43:0/465100417 learned_addr learned my addr 172.21.15.43:0/465100417 (peer_addr_for_me v2:172.21.15.43:0/0) 2023-08-09T13:33:59.021 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:33:59.018+0000 7f595881e640 1 --2- 172.21.15.43:0/465100417 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f59540d04b0 0x7f59540ca580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:33:59.022 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:33:59.018+0000 7f5959820640 1 -- 172.21.15.43:0/465100417 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f59540b4420 msgr2=0x7f59540ab8f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:33:59.022 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:33:59.018+0000 7f5959820640 1 --2- 172.21.15.43:0/465100417 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f59540b4420 0x7f59540ab8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:33:59.022 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:33:59.018+0000 7f5959820640 1 -- 172.21.15.43:0/465100417 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f59540d04b0 msgr2=0x7f59540ca580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:33:59.023 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:33:59.018+0000 7f5959820640 1 --2- 172.21.15.43:0/465100417 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f59540d04b0 0x7f59540ca580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:33:59.023 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:33:59.018+0000 7f5959820640 1 -- 172.21.15.43:0/465100417 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f59500094d0 con 0x7f595415c550 2023-08-09T13:33:59.023 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:33:59.018+0000 7f5959820640 1 --2- 172.21.15.43:0/465100417 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f595415c550 0x7f5954158640 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f594c00c780 tx=0x7f594c00cc40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-09T13:33:59.023 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:33:59.018+0000 7f5949ffb640 1 -- 172.21.15.43:0/465100417 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f594c004b60 con 0x7f595415c550 2023-08-09T13:33:59.023 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:33:59.018+0000 7f5949ffb640 1 -- 172.21.15.43:0/465100417 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f594c00ce90 con 0x7f595415c550 2023-08-09T13:33:59.023 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:33:59.018+0000 7f5949ffb640 1 -- 172.21.15.43:0/465100417 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f594c00a850 con 0x7f595415c550 2023-08-09T13:33:59.023 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:33:59.018+0000 7f595b283640 1 -- 172.21.15.43:0/465100417 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f59540cb5e0 con 0x7f595415c550 2023-08-09T13:33:59.023 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:33:59.019+0000 7f595b283640 1 -- 172.21.15.43:0/465100417 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f59540ac3e0 con 0x7f595415c550 2023-08-09T13:33:59.024 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:33:59.020+0000 7f595b283640 1 -- 172.21.15.43:0/465100417 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f59540739f0 con 0x7f595415c550 2023-08-09T13:33:59.024 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:33:59.021+0000 7f5949ffb640 1 -- 172.21.15.43:0/465100417 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 5) v1 ==== 88341+0+0 (secure 0 0 0) 0x7f594c00a9b0 con 0x7f595415c550 2023-08-09T13:33:59.025 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:33:59.023+0000 7f5949ffb640 1 --2- 172.21.15.43:0/465100417 >> [v2:172.21.15.203:6832/830097474,v1:172.21.15.203:6833/830097474] conn(0x7f593806a050 0x7f593806c510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:33:59.025 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:33:59.023+0000 7f5949ffb640 1 -- 172.21.15.43:0/465100417 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(16..16 src has 1..16) v4 ==== 5210+0+0 (secure 0 0 0) 0x7f594c088420 con 0x7f595415c550 2023-08-09T13:33:59.025 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:33:59.023+0000 7f595901f640 1 --2- 172.21.15.43:0/465100417 >> [v2:172.21.15.203:6832/830097474,v1:172.21.15.203:6833/830097474] conn(0x7f593806a050 0x7f593806c510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:33:59.026 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:33:59.024+0000 7f595901f640 1 --2- 172.21.15.43:0/465100417 >> [v2:172.21.15.203:6832/830097474,v1:172.21.15.203:6833/830097474] conn(0x7f593806a050 0x7f593806c510 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f5950036000 tx=0x7f5950030000 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-09T13:33:59.029 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:33:59.027+0000 7f5949ffb640 1 -- 172.21.15.43:0/465100417 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178066 (secure 0 0 0) 0x7f594c053e50 con 0x7f595415c550 2023-08-09T13:33:59.228 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:33:59.225+0000 7f595b283640 1 -- 172.21.15.43:0/465100417 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f59540cbd70 con 0x7f595415c550 2023-08-09T13:33:59.228 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:33:59.225+0000 7f5949ffb640 1 -- 172.21.15.43:0/465100417 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v16) v1 ==== 93+0+10 (secure 0 0 0) 0x7f594c057470 con 0x7f595415c550 2023-08-09T13:33:59.229 INFO:teuthology.orchestra.run.smithi043.stdout:pg_num: 1 2023-08-09T13:33:59.230 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:33:59.228+0000 7f595b283640 1 -- 172.21.15.43:0/465100417 >> [v2:172.21.15.203:6832/830097474,v1:172.21.15.203:6833/830097474] conn(0x7f593806a050 msgr2=0x7f593806c510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:33:59.231 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:33:59.228+0000 7f595b283640 1 --2- 172.21.15.43:0/465100417 >> [v2:172.21.15.203:6832/830097474,v1:172.21.15.203:6833/830097474] conn(0x7f593806a050 0x7f593806c510 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f5950036000 tx=0x7f5950030000 comp rx=0 tx=0).stop 2023-08-09T13:33:59.231 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:33:59.228+0000 7f595b283640 1 -- 172.21.15.43:0/465100417 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f595415c550 msgr2=0x7f5954158640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:33:59.231 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:33:59.228+0000 7f595b283640 1 --2- 172.21.15.43:0/465100417 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f595415c550 0x7f5954158640 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f594c00c780 tx=0x7f594c00cc40 comp rx=0 tx=0).stop 2023-08-09T13:33:59.231 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:33:59.228+0000 7f595b283640 1 -- 172.21.15.43:0/465100417 shutdown_connections 2023-08-09T13:33:59.231 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:33:59.228+0000 7f595b283640 1 --2- 172.21.15.43:0/465100417 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f595415c550 0x7f5954158640 unknown :-1 s=CLOSED pgs=53 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:33:59.231 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:33:59.228+0000 7f595b283640 1 --2- 172.21.15.43:0/465100417 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f59540d04b0 0x7f59540ca580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:33:59.232 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:33:59.228+0000 7f595b283640 1 --2- 172.21.15.43:0/465100417 >> [v2:172.21.15.203:6832/830097474,v1:172.21.15.203:6833/830097474] conn(0x7f593806a050 0x7f593806c510 unknown :-1 s=CLOSED pgs=57 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:33:59.232 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:33:59.228+0000 7f595b283640 1 --2- 172.21.15.43:0/465100417 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f59540b4420 0x7f59540ab8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:33:59.232 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:33:59.228+0000 7f595b283640 1 -- 172.21.15.43:0/465100417 >> 172.21.15.43:0/465100417 conn(0x7f5954071520 msgr2=0x7f59540d2440 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-09T13:33:59.232 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:33:59.228+0000 7f595b283640 1 -- 172.21.15.43:0/465100417 shutdown_connections 2023-08-09T13:33:59.232 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:33:59.228+0000 7f595b283640 1 -- 172.21.15.43:0/465100417 wait complete. 2023-08-09T13:33:59.249 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --format=json-pretty osd lspools 2023-08-09T13:33:59.477 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:33:59.474+0000 7fa734b74640 1 Processor -- start 2023-08-09T13:33:59.477 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:33:59.475+0000 7fa734b74640 1 -- start start 2023-08-09T13:33:59.479 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:33:59.476+0000 7fa734b74640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7fa73015c620 0x7fa7300cfe70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:33:59.479 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:33:59.476+0000 7fa734b74640 1 -- --> v1:172.21.15.43:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa7300b2330 con 0x7fa7300d0ef0 2023-08-09T13:33:59.479 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:33:59.476+0000 7fa734b74640 1 -- --> v1:172.21.15.203:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa7300d2ec0 con 0x7fa7300cf4f0 2023-08-09T13:33:59.480 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:33:59.476+0000 7fa734b74640 1 -- --> v2:172.21.15.43:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa7300d0500 con 0x7fa73015c620 2023-08-09T13:33:59.480 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:33:59.477+0000 7fa72f7fe640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7fa73015c620 0x7fa7300cfe70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:33:59.480 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:33:59.477+0000 7fa72ffff640 1 --1- >> v1:172.21.15.43:6789/0 conn(0x7fa7300d0ef0 0x7fa7300b27c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.43:6789/0 says I am v1:172.21.15.43:40922/0 (socket says 172.21.15.43:40922) 2023-08-09T13:33:59.480 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:33:59.477+0000 7fa72ffff640 1 -- 172.21.15.43:0/2402474922 learned_addr learned my addr 172.21.15.43:0/2402474922 (peer_addr_for_me v1:172.21.15.43:0/0) 2023-08-09T13:33:59.481 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:33:59.477+0000 7fa72f7fe640 1 -- 172.21.15.43:0/2402474922 >> v1:172.21.15.203:6789/0 conn(0x7fa7300cf4f0 legacy=0x7fa7300d07e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-09T13:33:59.481 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:33:59.478+0000 7fa72f7fe640 1 -- 172.21.15.43:0/2402474922 >> v1:172.21.15.43:6789/0 conn(0x7fa7300d0ef0 legacy=0x7fa7300b27c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:33:59.481 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:33:59.478+0000 7fa72f7fe640 1 -- 172.21.15.43:0/2402474922 --> v2:172.21.15.43:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa7300b1a00 con 0x7fa73015c620 2023-08-09T13:33:59.481 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:33:59.479+0000 7fa72f7fe640 1 --2- 172.21.15.43:0/2402474922 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa73015c620 0x7fa7300cfe70 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7fa720009790 tx=0x7fa72002b660 comp rx=0 tx=0).ready entity=mon.0 client_cookie=de4d0953ae0e22cc server_cookie=0 in_seq=0 out_seq=0 2023-08-09T13:33:59.482 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:33:59.480+0000 7fa72e7fc640 1 -- 172.21.15.43:0/2402474922 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa720030710 con 0x7fa73015c620 2023-08-09T13:33:59.482 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:33:59.480+0000 7fa72e7fc640 1 -- 172.21.15.43:0/2402474922 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa72002e050 con 0x7fa73015c620 2023-08-09T13:33:59.482 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:33:59.480+0000 7fa72e7fc640 1 -- 172.21.15.43:0/2402474922 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa720030a90 con 0x7fa73015c620 2023-08-09T13:33:59.483 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:33:59.480+0000 7fa734b74640 1 -- 172.21.15.43:0/2402474922 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa73015c620 msgr2=0x7fa7300cfe70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:33:59.483 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:33:59.480+0000 7fa734b74640 1 --2- 172.21.15.43:0/2402474922 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa73015c620 0x7fa7300cfe70 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7fa720009790 tx=0x7fa72002b660 comp rx=0 tx=0).stop 2023-08-09T13:33:59.483 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:33:59.480+0000 7fa734b74640 1 -- 172.21.15.43:0/2402474922 shutdown_connections 2023-08-09T13:33:59.483 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:33:59.480+0000 7fa734b74640 1 --2- 172.21.15.43:0/2402474922 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa73015c620 0x7fa7300cfe70 unknown :-1 s=CLOSED pgs=55 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:33:59.484 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:33:59.480+0000 7fa734b74640 1 -- 172.21.15.43:0/2402474922 >> 172.21.15.43:0/2402474922 conn(0x7fa73009b650 msgr2=0x7fa7300d22a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-09T13:33:59.484 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:33:59.481+0000 7fa734b74640 1 -- 172.21.15.43:0/2402474922 shutdown_connections 2023-08-09T13:33:59.484 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:33:59.481+0000 7fa734b74640 1 -- 172.21.15.43:0/2402474922 wait complete. 2023-08-09T13:33:59.484 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:33:59.482+0000 7fa734b74640 1 Processor -- start 2023-08-09T13:33:59.485 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:33:59.483+0000 7fa734b74640 1 -- start start 2023-08-09T13:33:59.486 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:33:59.483+0000 7fa734b74640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa7300cf4f0 0x7fa7301f2370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:33:59.486 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:33:59.483+0000 7fa734b74640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fa7300d0ef0 0x7fa7301f5250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:33:59.486 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:33:59.484+0000 7fa734b74640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fa73015c620 0x7fa7300c8700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:33:59.487 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:33:59.484+0000 7fa734b74640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa7300b6040 con 0x7fa7300cf4f0 2023-08-09T13:33:59.487 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:33:59.484+0000 7fa734b74640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa7301f5e60 con 0x7fa73015c620 2023-08-09T13:33:59.487 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:33:59.484+0000 7fa734b74640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa7300b6460 con 0x7fa7300d0ef0 2023-08-09T13:33:59.487 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:33:59.484+0000 7fa72f7fe640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa7300cf4f0 0x7fa7301f2370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:33:59.487 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:33:59.484+0000 7fa72f7fe640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa7300cf4f0 0x7fa7301f2370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:57718/0 (socket says 172.21.15.43:57718) 2023-08-09T13:33:59.487 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:33:59.484+0000 7fa72f7fe640 1 -- 172.21.15.43:0/507839848 learned_addr learned my addr 172.21.15.43:0/507839848 (peer_addr_for_me v2:172.21.15.43:0/0) 2023-08-09T13:33:59.488 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:33:59.484+0000 7fa72ffff640 1 --2- 172.21.15.43:0/507839848 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fa73015c620 0x7fa7300c8700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:33:59.488 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:33:59.484+0000 7fa72effd640 1 --2- 172.21.15.43:0/507839848 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fa7300d0ef0 0x7fa7301f5250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:33:59.489 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:33:59.484+0000 7fa72f7fe640 1 -- 172.21.15.43:0/507839848 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fa7300d0ef0 msgr2=0x7fa7301f5250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:33:59.489 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:33:59.484+0000 7fa72f7fe640 1 --2- 172.21.15.43:0/507839848 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fa7300d0ef0 0x7fa7301f5250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:33:59.489 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:33:59.484+0000 7fa72f7fe640 1 -- 172.21.15.43:0/507839848 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fa73015c620 msgr2=0x7fa7300c8700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:33:59.489 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:33:59.484+0000 7fa72f7fe640 1 --2- 172.21.15.43:0/507839848 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fa73015c620 0x7fa7300c8700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:33:59.489 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:33:59.484+0000 7fa72f7fe640 1 -- 172.21.15.43:0/507839848 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa7200093a0 con 0x7fa7300cf4f0 2023-08-09T13:33:59.489 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:33:59.485+0000 7fa72f7fe640 1 --2- 172.21.15.43:0/507839848 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa7300cf4f0 0x7fa7301f2370 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7fa720035040 tx=0x7fa7200046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-09T13:33:59.489 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:33:59.485+0000 7fa717fff640 1 -- 172.21.15.43:0/507839848 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa720032910 con 0x7fa7300cf4f0 2023-08-09T13:33:59.490 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:33:59.485+0000 7fa717fff640 1 -- 172.21.15.43:0/507839848 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa720032a90 con 0x7fa7300cf4f0 2023-08-09T13:33:59.490 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:33:59.485+0000 7fa717fff640 1 -- 172.21.15.43:0/507839848 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa720032e40 con 0x7fa7300cf4f0 2023-08-09T13:33:59.490 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:33:59.485+0000 7fa734b74640 1 -- 172.21.15.43:0/507839848 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa7300b53c0 con 0x7fa7300cf4f0 2023-08-09T13:33:59.490 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:33:59.485+0000 7fa734b74640 1 -- 172.21.15.43:0/507839848 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa7300b5980 con 0x7fa7300cf4f0 2023-08-09T13:33:59.490 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:33:59.486+0000 7fa734b74640 1 -- 172.21.15.43:0/507839848 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa7300b6180 con 0x7fa7300cf4f0 2023-08-09T13:33:59.491 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:33:59.487+0000 7fa717fff640 1 -- 172.21.15.43:0/507839848 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 5) v1 ==== 88341+0+0 (secure 0 0 0) 0x7fa720039040 con 0x7fa7300cf4f0 2023-08-09T13:33:59.491 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:33:59.488+0000 7fa717fff640 1 --2- 172.21.15.43:0/507839848 >> [v2:172.21.15.203:6832/830097474,v1:172.21.15.203:6833/830097474] conn(0x7fa71c06a080 0x7fa71c06c540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:33:59.491 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:33:59.489+0000 7fa717fff640 1 -- 172.21.15.43:0/507839848 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(16..16 src has 1..16) v4 ==== 5210+0+0 (secure 0 0 0) 0x7fa7200a9a10 con 0x7fa7300cf4f0 2023-08-09T13:33:59.492 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:33:59.489+0000 7fa72effd640 1 --2- 172.21.15.43:0/507839848 >> [v2:172.21.15.203:6832/830097474,v1:172.21.15.203:6833/830097474] conn(0x7fa71c06a080 0x7fa71c06c540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:33:59.492 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:33:59.490+0000 7fa72effd640 1 --2- 172.21.15.43:0/507839848 >> [v2:172.21.15.203:6832/830097474,v1:172.21.15.203:6833/830097474] conn(0x7fa71c06a080 0x7fa71c06c540 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7fa73015e6f0 tx=0x7fa724009040 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-09T13:33:59.495 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:33:59.492+0000 7fa717fff640 1 -- 172.21.15.43:0/507839848 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178066 (secure 0 0 0) 0x7fa720075440 con 0x7fa7300cf4f0 2023-08-09T13:33:59.696 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:33:59.694+0000 7fa734b74640 1 -- 172.21.15.43:0/507839848 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) v1 -- 0x7fa7300739f0 con 0x7fa7300cf4f0 2023-08-09T13:33:59.697 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:33:59.695+0000 7fa717fff640 1 -- 172.21.15.43:0/507839848 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v16) v1 ==== 84+0+126 (secure 0 0 0) 0x7fa720037020 con 0x7fa7300cf4f0 2023-08-09T13:33:59.697 INFO:teuthology.orchestra.run.smithi043.stdout: 2023-08-09T13:33:59.698 INFO:teuthology.orchestra.run.smithi043.stdout:[ 2023-08-09T13:33:59.698 INFO:teuthology.orchestra.run.smithi043.stdout: { 2023-08-09T13:33:59.698 INFO:teuthology.orchestra.run.smithi043.stdout: "poolnum": 1, 2023-08-09T13:33:59.698 INFO:teuthology.orchestra.run.smithi043.stdout: "poolname": "rbd" 2023-08-09T13:33:59.698 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2023-08-09T13:33:59.698 INFO:teuthology.orchestra.run.smithi043.stdout: { 2023-08-09T13:33:59.698 INFO:teuthology.orchestra.run.smithi043.stdout: "poolnum": 2, 2023-08-09T13:33:59.699 INFO:teuthology.orchestra.run.smithi043.stdout: "poolname": ".mgr" 2023-08-09T13:33:59.699 INFO:teuthology.orchestra.run.smithi043.stdout: } 2023-08-09T13:33:59.699 INFO:teuthology.orchestra.run.smithi043.stdout:] 2023-08-09T13:33:59.700 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:33:59.697+0000 7fa734b74640 1 -- 172.21.15.43:0/507839848 >> [v2:172.21.15.203:6832/830097474,v1:172.21.15.203:6833/830097474] conn(0x7fa71c06a080 msgr2=0x7fa71c06c540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:33:59.700 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:33:59.698+0000 7fa734b74640 1 --2- 172.21.15.43:0/507839848 >> [v2:172.21.15.203:6832/830097474,v1:172.21.15.203:6833/830097474] conn(0x7fa71c06a080 0x7fa71c06c540 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7fa73015e6f0 tx=0x7fa724009040 comp rx=0 tx=0).stop 2023-08-09T13:33:59.700 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:33:59.698+0000 7fa734b74640 1 -- 172.21.15.43:0/507839848 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa7300cf4f0 msgr2=0x7fa7301f2370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:33:59.701 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:33:59.698+0000 7fa734b74640 1 --2- 172.21.15.43:0/507839848 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa7300cf4f0 0x7fa7301f2370 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7fa720035040 tx=0x7fa7200046c0 comp rx=0 tx=0).stop 2023-08-09T13:33:59.701 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:33:59.698+0000 7fa734b74640 1 -- 172.21.15.43:0/507839848 shutdown_connections 2023-08-09T13:33:59.701 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:33:59.698+0000 7fa734b74640 1 --2- 172.21.15.43:0/507839848 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fa73015c620 0x7fa7300c8700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:33:59.701 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:33:59.698+0000 7fa734b74640 1 --2- 172.21.15.43:0/507839848 >> [v2:172.21.15.203:6832/830097474,v1:172.21.15.203:6833/830097474] conn(0x7fa71c06a080 0x7fa71c06c540 unknown :-1 s=CLOSED pgs=58 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:33:59.701 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:33:59.698+0000 7fa734b74640 1 --2- 172.21.15.43:0/507839848 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fa7300d0ef0 0x7fa7301f5250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:33:59.701 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:33:59.698+0000 7fa734b74640 1 --2- 172.21.15.43:0/507839848 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa7300cf4f0 0x7fa7301f2370 unknown :-1 s=CLOSED pgs=56 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:33:59.701 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:33:59.698+0000 7fa734b74640 1 -- 172.21.15.43:0/507839848 >> 172.21.15.43:0/507839848 conn(0x7fa73009b650 msgr2=0x7fa7300b0ec0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-09T13:33:59.702 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:33:59.698+0000 7fa734b74640 1 -- 172.21.15.43:0/507839848 shutdown_connections 2023-08-09T13:33:59.702 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:33:59.698+0000 7fa734b74640 1 -- 172.21.15.43:0/507839848 wait complete. 2023-08-09T13:33:59.719 DEBUG:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2023-08-09T13:33:59.720 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_metadata --pg_num_min 64 2023-08-09T13:33:59.944 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:33:59.942+0000 7fa66497f640 1 Processor -- start 2023-08-09T13:33:59.945 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:33:59.942+0000 7fa66497f640 1 -- start start 2023-08-09T13:33:59.946 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:33:59.943+0000 7fa66497f640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7fa66015c400 0x7fa6600d0cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:33:59.946 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:33:59.943+0000 7fa66497f640 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7fa6600d1210 0x7fa6600cec70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:33:59.946 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:33:59.943+0000 7fa66497f640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fa6600cd3c0 0x7fa6600cf1b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:33:59.946 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:33:59.943+0000 7fa66497f640 1 -- --> v2:172.21.15.43:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa66015c7e0 con 0x7fa66015c400 2023-08-09T13:33:59.947 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:33:59.943+0000 7fa66497f640 1 -- --> v2:172.21.15.203:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa6600d1690 con 0x7fa6600d1210 2023-08-09T13:33:59.947 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:33:59.943+0000 7fa66497f640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa6600d1970 con 0x7fa6600cd3c0 2023-08-09T13:33:59.947 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:33:59.944+0000 7fa65f7fe640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7fa66015c400 0x7fa6600d0cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:33:59.947 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:33:59.944+0000 7fa65f7fe640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7fa66015c400 0x7fa6600d0cd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:57726/0 (socket says 172.21.15.43:57726) 2023-08-09T13:33:59.947 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:33:59.944+0000 7fa65f7fe640 1 -- 172.21.15.43:0/2649300923 learned_addr learned my addr 172.21.15.43:0/2649300923 (peer_addr_for_me v2:172.21.15.43:0/0) 2023-08-09T13:33:59.948 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:33:59.944+0000 7fa65effd640 1 --2- 172.21.15.43:0/2649300923 >> v2:172.21.15.203:3300/0 conn(0x7fa6600d1210 0x7fa6600cec70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:33:59.948 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:33:59.944+0000 7fa65ffff640 1 --2- 172.21.15.43:0/2649300923 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fa6600cd3c0 0x7fa6600cf1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:33:59.948 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:33:59.944+0000 7fa65f7fe640 1 -- 172.21.15.43:0/2649300923 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fa6600cd3c0 msgr2=0x7fa6600cf1b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-09T13:33:59.949 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:33:59.944+0000 7fa65f7fe640 1 --2- 172.21.15.43:0/2649300923 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fa6600cd3c0 0x7fa6600cf1b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:33:59.949 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:33:59.944+0000 7fa65f7fe640 1 -- 172.21.15.43:0/2649300923 >> v2:172.21.15.203:3300/0 conn(0x7fa6600d1210 msgr2=0x7fa6600cec70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-09T13:33:59.949 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:33:59.944+0000 7fa65f7fe640 1 --2- 172.21.15.43:0/2649300923 >> v2:172.21.15.203:3300/0 conn(0x7fa6600d1210 0x7fa6600cec70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:33:59.949 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:33:59.944+0000 7fa65f7fe640 1 -- 172.21.15.43:0/2649300923 --> v2:172.21.15.43:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa6600d1c50 con 0x7fa66015c400 2023-08-09T13:33:59.949 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:33:59.944+0000 7fa65ffff640 1 --2- 172.21.15.43:0/2649300923 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fa6600cd3c0 0x7fa6600cf1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-08-09T13:33:59.949 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:33:59.945+0000 7fa65f7fe640 1 --2- 172.21.15.43:0/2649300923 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa66015c400 0x7fa6600d0cd0 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7fa660069dd0 tx=0x7fa65002b9a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=21854808e467272 server_cookie=0 in_seq=0 out_seq=0 2023-08-09T13:33:59.949 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:33:59.946+0000 7fa65e7fc640 1 -- 172.21.15.43:0/2649300923 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa650030860 con 0x7fa66015c400 2023-08-09T13:33:59.949 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:33:59.946+0000 7fa65e7fc640 1 -- 172.21.15.43:0/2649300923 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa6500309c0 con 0x7fa66015c400 2023-08-09T13:33:59.950 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:33:59.946+0000 7fa65e7fc640 1 -- 172.21.15.43:0/2649300923 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa650030d50 con 0x7fa66015c400 2023-08-09T13:33:59.950 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:33:59.946+0000 7fa66497f640 1 -- 172.21.15.43:0/2649300923 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa66015c400 msgr2=0x7fa6600d0cd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:33:59.951 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:33:59.946+0000 7fa66497f640 1 --2- 172.21.15.43:0/2649300923 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa66015c400 0x7fa6600d0cd0 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7fa660069dd0 tx=0x7fa65002b9a0 comp rx=0 tx=0).stop 2023-08-09T13:33:59.951 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:33:59.946+0000 7fa66497f640 1 -- 172.21.15.43:0/2649300923 shutdown_connections 2023-08-09T13:33:59.951 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:33:59.946+0000 7fa66497f640 1 --2- 172.21.15.43:0/2649300923 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fa6600cd3c0 0x7fa6600cf1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:33:59.951 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:33:59.946+0000 7fa66497f640 1 --2- 172.21.15.43:0/2649300923 >> v2:172.21.15.203:3300/0 conn(0x7fa6600d1210 0x7fa6600cec70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:33:59.951 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:33:59.946+0000 7fa66497f640 1 --2- 172.21.15.43:0/2649300923 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa66015c400 0x7fa6600d0cd0 unknown :-1 s=CLOSED pgs=57 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:33:59.951 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:33:59.946+0000 7fa66497f640 1 -- 172.21.15.43:0/2649300923 >> 172.21.15.43:0/2649300923 conn(0x7fa66006a320 msgr2=0x7fa66006a730 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-09T13:33:59.951 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:33:59.947+0000 7fa66497f640 1 -- 172.21.15.43:0/2649300923 shutdown_connections 2023-08-09T13:33:59.952 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:33:59.947+0000 7fa66497f640 1 -- 172.21.15.43:0/2649300923 wait complete. 2023-08-09T13:33:59.953 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:33:59.948+0000 7fa66497f640 1 Processor -- start 2023-08-09T13:33:59.953 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:33:59.949+0000 7fa66497f640 1 -- start start 2023-08-09T13:33:59.953 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:33:59.949+0000 7fa66497f640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fa6600cd3c0 0x7fa6600c9d60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:33:59.953 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:33:59.949+0000 7fa66497f640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fa6600d1210 0x7fa6601583a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:33:59.953 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:33:59.949+0000 7fa66497f640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa66015c400 0x7fa6601592f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:33:59.953 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:33:59.949+0000 7fa66497f640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa66015d740 con 0x7fa66015c400 2023-08-09T13:33:59.954 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:33:59.949+0000 7fa66497f640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa6600ca2a0 con 0x7fa6600d1210 2023-08-09T13:33:59.954 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:33:59.949+0000 7fa66497f640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa66015cac0 con 0x7fa6600cd3c0 2023-08-09T13:33:59.954 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:33:59.950+0000 7fa65ffff640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa66015c400 0x7fa6601592f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:33:59.954 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:33:59.950+0000 7fa65ffff640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa66015c400 0x7fa6601592f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:57740/0 (socket says 172.21.15.43:57740) 2023-08-09T13:33:59.954 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:33:59.950+0000 7fa65ffff640 1 -- 172.21.15.43:0/4206836069 learned_addr learned my addr 172.21.15.43:0/4206836069 (peer_addr_for_me v2:172.21.15.43:0/0) 2023-08-09T13:33:59.954 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:33:59.950+0000 7fa65effd640 1 --2- 172.21.15.43:0/4206836069 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fa6600d1210 0x7fa6601583a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:33:59.954 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:33:59.950+0000 7fa65f7fe640 1 --2- 172.21.15.43:0/4206836069 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fa6600cd3c0 0x7fa6600c9d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:33:59.955 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:33:59.950+0000 7fa65ffff640 1 -- 172.21.15.43:0/4206836069 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fa6600cd3c0 msgr2=0x7fa6600c9d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:33:59.955 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:33:59.950+0000 7fa65ffff640 1 --2- 172.21.15.43:0/4206836069 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fa6600cd3c0 0x7fa6600c9d60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:33:59.955 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:33:59.950+0000 7fa65ffff640 1 -- 172.21.15.43:0/4206836069 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fa6600d1210 msgr2=0x7fa6601583a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:33:59.955 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:33:59.950+0000 7fa65ffff640 1 --2- 172.21.15.43:0/4206836069 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fa6600d1210 0x7fa6601583a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:33:59.955 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:33:59.950+0000 7fa65ffff640 1 -- 172.21.15.43:0/4206836069 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa6500094d0 con 0x7fa66015c400 2023-08-09T13:33:59.956 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:33:59.951+0000 7fa65ffff640 1 --2- 172.21.15.43:0/4206836069 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa66015c400 0x7fa6601592f0 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7fa65800cef0 tx=0x7fa65800a710 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-09T13:33:59.956 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:33:59.951+0000 7fa647fff640 1 -- 172.21.15.43:0/4206836069 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa65800d840 con 0x7fa66015c400 2023-08-09T13:33:59.956 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:33:59.951+0000 7fa647fff640 1 -- 172.21.15.43:0/4206836069 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa65800d9c0 con 0x7fa66015c400 2023-08-09T13:33:59.956 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:33:59.951+0000 7fa647fff640 1 -- 172.21.15.43:0/4206836069 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa65800dd70 con 0x7fa66015c400 2023-08-09T13:33:59.957 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:33:59.951+0000 7fa66497f640 1 -- 172.21.15.43:0/4206836069 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa66015cda0 con 0x7fa66015c400 2023-08-09T13:33:59.957 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:33:59.951+0000 7fa66497f640 1 -- 172.21.15.43:0/4206836069 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa66015d360 con 0x7fa66015c400 2023-08-09T13:33:59.957 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:33:59.952+0000 7fa66497f640 1 -- 172.21.15.43:0/4206836069 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa6600739f0 con 0x7fa66015c400 2023-08-09T13:33:59.957 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:33:59.953+0000 7fa647fff640 1 -- 172.21.15.43:0/4206836069 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 5) v1 ==== 88341+0+0 (secure 0 0 0) 0x7fa65801e3b0 con 0x7fa66015c400 2023-08-09T13:33:59.957 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:33:59.954+0000 7fa647fff640 1 --2- 172.21.15.43:0/4206836069 >> [v2:172.21.15.203:6832/830097474,v1:172.21.15.203:6833/830097474] conn(0x7fa64c06a050 0x7fa64c06c510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:33:59.958 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:33:59.954+0000 7fa647fff640 1 -- 172.21.15.43:0/4206836069 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(16..16 src has 1..16) v4 ==== 5210+0+0 (secure 0 0 0) 0x7fa658088d50 con 0x7fa66015c400 2023-08-09T13:33:59.958 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:33:59.955+0000 7fa65f7fe640 1 --2- 172.21.15.43:0/4206836069 >> [v2:172.21.15.203:6832/830097474,v1:172.21.15.203:6833/830097474] conn(0x7fa64c06a050 0x7fa64c06c510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:33:59.958 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:33:59.956+0000 7fa65f7fe640 1 --2- 172.21.15.43:0/4206836069 >> [v2:172.21.15.203:6832/830097474,v1:172.21.15.203:6833/830097474] conn(0x7fa64c06a050 0x7fa64c06c510 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7fa650036000 tx=0x7fa65003b040 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-09T13:33:59.961 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:33:59.958+0000 7fa647fff640 1 -- 172.21.15.43:0/4206836069 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178066 (secure 0 0 0) 0x7fa658054840 con 0x7fa66015c400 2023-08-09T13:34:00.157 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:00.154+0000 7fa66497f640 1 -- 172.21.15.43:0/4206836069 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64} v 0) v1 -- 0x7fa660066200 con 0x7fa66015c400 2023-08-09T13:34:01.073 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:01.070+0000 7fa647fff640 1 -- 172.21.15.43:0/4206836069 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]=0 pool 'cephfs_metadata' created v17) v1 ==== 138+0+0 (secure 0 0 0) 0x7fa660066200 con 0x7fa66015c400 2023-08-09T13:34:01.073 INFO:teuthology.orchestra.run.smithi043.stderr:pool 'cephfs_metadata' created 2023-08-09T13:34:01.080 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:01.077+0000 7fa66497f640 1 -- 172.21.15.43:0/4206836069 >> [v2:172.21.15.203:6832/830097474,v1:172.21.15.203:6833/830097474] conn(0x7fa64c06a050 msgr2=0x7fa64c06c510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:34:01.080 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:01.078+0000 7fa66497f640 1 --2- 172.21.15.43:0/4206836069 >> [v2:172.21.15.203:6832/830097474,v1:172.21.15.203:6833/830097474] conn(0x7fa64c06a050 0x7fa64c06c510 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7fa650036000 tx=0x7fa65003b040 comp rx=0 tx=0).stop 2023-08-09T13:34:01.080 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:01.078+0000 7fa66497f640 1 -- 172.21.15.43:0/4206836069 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa66015c400 msgr2=0x7fa6601592f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:34:01.081 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:01.078+0000 7fa66497f640 1 --2- 172.21.15.43:0/4206836069 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa66015c400 0x7fa6601592f0 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7fa65800cef0 tx=0x7fa65800a710 comp rx=0 tx=0).stop 2023-08-09T13:34:01.082 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:01.078+0000 7fa66497f640 1 -- 172.21.15.43:0/4206836069 shutdown_connections 2023-08-09T13:34:01.082 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:01.078+0000 7fa66497f640 1 --2- 172.21.15.43:0/4206836069 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa66015c400 0x7fa6601592f0 unknown :-1 s=CLOSED pgs=58 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:34:01.082 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:01.078+0000 7fa66497f640 1 --2- 172.21.15.43:0/4206836069 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fa6600d1210 0x7fa6601583a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:34:01.083 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:01.078+0000 7fa66497f640 1 --2- 172.21.15.43:0/4206836069 >> [v2:172.21.15.203:6832/830097474,v1:172.21.15.203:6833/830097474] conn(0x7fa64c06a050 0x7fa64c06c510 unknown :-1 s=CLOSED pgs=59 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:34:01.083 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:01.078+0000 7fa66497f640 1 --2- 172.21.15.43:0/4206836069 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fa6600cd3c0 0x7fa6600c9d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:34:01.083 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:01.078+0000 7fa66497f640 1 -- 172.21.15.43:0/4206836069 >> 172.21.15.43:0/4206836069 conn(0x7fa66006a320 msgr2=0x7fa6600b3380 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-09T13:34:01.083 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:01.079+0000 7fa66497f640 1 -- 172.21.15.43:0/4206836069 shutdown_connections 2023-08-09T13:34:01.083 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:01.079+0000 7fa66497f640 1 -- 172.21.15.43:0/4206836069 wait complete. 2023-08-09T13:34:01.107 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data 64 --pg_num_min 64 --target_size_ratio 0.05 2023-08-09T13:34:01.336 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:01.333+0000 7f638b1f5640 1 Processor -- start 2023-08-09T13:34:01.336 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:01.333+0000 7f638b1f5640 1 -- start start 2023-08-09T13:34:01.337 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:01.335+0000 7f638b1f5640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f638415c550 0x7f63840cd6d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:34:01.338 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:01.335+0000 7f638b1f5640 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7f63840b45f0 0x7f63840b59d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:34:01.338 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:01.335+0000 7f638b1f5640 1 -- --> v1:172.21.15.43:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f63840b7bd0 con 0x7f63840d04b0 2023-08-09T13:34:01.338 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:01.335+0000 7f638b1f5640 1 -- --> v2:172.21.15.203:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f63840d1c00 con 0x7f63840b45f0 2023-08-09T13:34:01.338 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:01.335+0000 7f638b1f5640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f638415c930 con 0x7f638415c550 2023-08-09T13:34:01.338 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:01.335+0000 7f6383fff640 1 --1- >> v1:172.21.15.43:6789/0 conn(0x7f63840d04b0 0x7f63840b3ee0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.43:6789/0 says I am v1:172.21.15.43:40936/0 (socket says 172.21.15.43:40936) 2023-08-09T13:34:01.338 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:01.335+0000 7f6383fff640 1 -- 172.21.15.43:0/1784755232 learned_addr learned my addr 172.21.15.43:0/1784755232 (peer_addr_for_me v1:172.21.15.43:0/0) 2023-08-09T13:34:01.340 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:01.336+0000 7f6388f91640 1 --2- 172.21.15.43:0/1784755232 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f638415c550 0x7f63840cd6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:34:01.340 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:01.336+0000 7f6389792640 1 --2- 172.21.15.43:0/1784755232 >> v2:172.21.15.203:3300/0 conn(0x7f63840b45f0 0x7f63840b59d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:34:01.341 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:01.336+0000 7f63837fe640 1 -- 172.21.15.43:0/1784755232 <== mon.0 v1:172.21.15.43:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3358017835 0 0) 0x7f63840b7bd0 con 0x7f63840d04b0 2023-08-09T13:34:01.341 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:01.336+0000 7f63837fe640 1 -- 172.21.15.43:0/1784755232 --> v1:172.21.15.43:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f6370003520 con 0x7f63840d04b0 2023-08-09T13:34:01.341 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:01.336+0000 7f63837fe640 1 -- 172.21.15.43:0/1784755232 <== mon.0 v1:172.21.15.43:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3498819856 0 0) 0x7f6370003520 con 0x7f63840d04b0 2023-08-09T13:34:01.341 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:01.336+0000 7f63837fe640 1 -- 172.21.15.43:0/1784755232 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f638415c550 msgr2=0x7f63840cd6d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-09T13:34:01.341 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:01.336+0000 7f63837fe640 1 --2- 172.21.15.43:0/1784755232 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f638415c550 0x7f63840cd6d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:34:01.342 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:01.336+0000 7f63837fe640 1 -- 172.21.15.43:0/1784755232 >> v2:172.21.15.203:3300/0 conn(0x7f63840b45f0 msgr2=0x7f63840b59d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-09T13:34:01.342 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:01.336+0000 7f63837fe640 1 --2- 172.21.15.43:0/1784755232 >> v2:172.21.15.203:3300/0 conn(0x7f63840b45f0 0x7f63840b59d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:34:01.342 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:01.336+0000 7f63837fe640 1 -- 172.21.15.43:0/1784755232 --> v1:172.21.15.43:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f63840b54f0 con 0x7f63840d04b0 2023-08-09T13:34:01.342 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:01.336+0000 7f63837fe640 1 -- 172.21.15.43:0/1784755232 <== mon.0 v1:172.21.15.43:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1285150740 0 0) 0x7f63740019f0 con 0x7f63840d04b0 2023-08-09T13:34:01.342 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:01.336+0000 7f63837fe640 1 -- 172.21.15.43:0/1784755232 >> v1:172.21.15.43:6789/0 conn(0x7f63840d04b0 legacy=0x7f63840b3ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:34:01.342 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:01.336+0000 7f63837fe640 1 --2- 172.21.15.43:0/1784755232 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f6370003970 0x7f6370003d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:34:01.342 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:01.336+0000 7f63837fe640 1 --2- 172.21.15.43:0/1784755232 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f63700047c0 0x7f6370004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:34:01.343 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:01.337+0000 7f63837fe640 1 --2- 172.21.15.43:0/1784755232 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f638415c550 0x7f6370005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:34:01.343 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:01.337+0000 7f63837fe640 1 -- 172.21.15.43:0/1784755232 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f63840d1c00 con 0x7f6370003970 2023-08-09T13:34:01.343 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:01.337+0000 7f63837fe640 1 -- 172.21.15.43:0/1784755232 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f638415c930 con 0x7f63700047c0 2023-08-09T13:34:01.343 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:01.337+0000 7f63837fe640 1 -- 172.21.15.43:0/1784755232 --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f637000b980 con 0x7f638415c550 2023-08-09T13:34:01.343 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:01.337+0000 7f6388f91640 1 --2- 172.21.15.43:0/1784755232 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f6370003970 0x7f6370003d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:34:01.343 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:01.337+0000 7f6389792640 1 --2- 172.21.15.43:0/1784755232 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f638415c550 0x7f6370005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:34:01.343 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:01.337+0000 7f6383fff640 1 --2- 172.21.15.43:0/1784755232 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f63700047c0 0x7f6370004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:34:01.344 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:01.337+0000 7f6388f91640 1 -- 172.21.15.43:0/1784755232 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f638415c550 msgr2=0x7f6370005180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-09T13:34:01.344 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:01.337+0000 7f6388f91640 1 --2- 172.21.15.43:0/1784755232 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f638415c550 0x7f6370005180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:34:01.344 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:01.337+0000 7f6388f91640 1 -- 172.21.15.43:0/1784755232 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f63700047c0 msgr2=0x7f6370004c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-09T13:34:01.344 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:01.337+0000 7f6388f91640 1 --2- 172.21.15.43:0/1784755232 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f63700047c0 0x7f6370004c40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:34:01.344 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:01.337+0000 7f6388f91640 1 -- 172.21.15.43:0/1784755232 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6370003520 con 0x7f6370003970 2023-08-09T13:34:01.344 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:01.338+0000 7f6389792640 1 --2- 172.21.15.43:0/1784755232 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f638415c550 0x7f6370005180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-08-09T13:34:01.347 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:01.339+0000 7f6388f91640 1 --2- 172.21.15.43:0/1784755232 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f6370003970 0x7f6370003d90 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f636c009ac0 tx=0x7f63700057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=77379523e7650ff server_cookie=0 in_seq=0 out_seq=0 2023-08-09T13:34:01.347 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:01.339+0000 7f63837fe640 1 -- 172.21.15.43:0/1784755232 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f636c02c6b0 con 0x7f6370003970 2023-08-09T13:34:01.347 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:01.339+0000 7f63837fe640 1 -- 172.21.15.43:0/1784755232 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f636c02c810 con 0x7f6370003970 2023-08-09T13:34:01.347 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:01.339+0000 7f63837fe640 1 -- 172.21.15.43:0/1784755232 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f636c02cba0 con 0x7f6370003970 2023-08-09T13:34:01.348 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:01.340+0000 7f638b1f5640 1 -- 172.21.15.43:0/1784755232 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f6370003970 msgr2=0x7f6370003d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:34:01.348 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:01.340+0000 7f638b1f5640 1 --2- 172.21.15.43:0/1784755232 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f6370003970 0x7f6370003d90 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f636c009ac0 tx=0x7f63700057f0 comp rx=0 tx=0).stop 2023-08-09T13:34:01.348 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:01.340+0000 7f6389792640 1 -- 172.21.15.43:0/1784755232 reap_dead start 2023-08-09T13:34:01.348 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:01.340+0000 7f638b1f5640 1 -- 172.21.15.43:0/1784755232 shutdown_connections 2023-08-09T13:34:01.348 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:01.340+0000 7f638b1f5640 1 -- 172.21.15.43:0/1784755232 >> 172.21.15.43:0/1784755232 conn(0x7f6384071520 msgr2=0x7f63840ceee0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-09T13:34:01.348 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:01.340+0000 7f638b1f5640 1 -- 172.21.15.43:0/1784755232 shutdown_connections 2023-08-09T13:34:01.348 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:01.340+0000 7f638b1f5640 1 -- 172.21.15.43:0/1784755232 wait complete. 2023-08-09T13:34:01.349 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:01.342+0000 7f638b1f5640 1 Processor -- start 2023-08-09T13:34:01.349 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:01.342+0000 7f638b1f5640 1 -- start start 2023-08-09T13:34:01.349 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:01.343+0000 7f638b1f5640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f6370003970 0x7f6384151480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:34:01.349 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:01.343+0000 7f638b1f5640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f63840d2760 0x7f6384157b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:34:01.349 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:01.343+0000 7f638b1f5640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f63840d2be0 0x7f63841580a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:34:01.349 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:01.343+0000 7f638b1f5640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f63841537e0 con 0x7f6370003970 2023-08-09T13:34:01.350 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:01.343+0000 7f638b1f5640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f63841519c0 con 0x7f63840d2be0 2023-08-09T13:34:01.350 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:01.343+0000 7f638b1f5640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6384152b60 con 0x7f63840d2760 2023-08-09T13:34:01.350 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:01.343+0000 7f6388f91640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f6370003970 0x7f6384151480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:34:01.350 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:01.343+0000 7f6388f91640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f6370003970 0x7f6384151480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:57768/0 (socket says 172.21.15.43:57768) 2023-08-09T13:34:01.350 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:01.343+0000 7f6388f91640 1 -- 172.21.15.43:0/1724370822 learned_addr learned my addr 172.21.15.43:0/1724370822 (peer_addr_for_me v2:172.21.15.43:0/0) 2023-08-09T13:34:01.350 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:01.344+0000 7f6383fff640 1 --2- 172.21.15.43:0/1724370822 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f63840d2760 0x7f6384157b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:34:01.351 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:01.344+0000 7f6389792640 1 --2- 172.21.15.43:0/1724370822 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f63840d2be0 0x7f63841580a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:34:01.351 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:01.344+0000 7f6388f91640 1 -- 172.21.15.43:0/1724370822 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f63840d2760 msgr2=0x7f6384157b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:34:01.351 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:01.344+0000 7f6388f91640 1 --2- 172.21.15.43:0/1724370822 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f63840d2760 0x7f6384157b60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:34:01.351 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:01.344+0000 7f6388f91640 1 -- 172.21.15.43:0/1724370822 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f63840d2be0 msgr2=0x7f63841580a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:34:01.351 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:01.344+0000 7f6388f91640 1 --2- 172.21.15.43:0/1724370822 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f63840d2be0 0x7f63841580a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:34:01.351 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:01.344+0000 7f6388f91640 1 -- 172.21.15.43:0/1724370822 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6384152e40 con 0x7f6370003970 2023-08-09T13:34:01.351 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:01.344+0000 7f6383fff640 1 --2- 172.21.15.43:0/1724370822 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f63840d2760 0x7f6384157b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-08-09T13:34:01.352 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:01.344+0000 7f6388f91640 1 --2- 172.21.15.43:0/1724370822 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f6370003970 0x7f6384151480 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f636c02c250 tx=0x7f636c004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-09T13:34:01.352 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:01.344+0000 7f63817fa640 1 -- 172.21.15.43:0/1724370822 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f636c003810 con 0x7f6370003970 2023-08-09T13:34:01.352 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:01.344+0000 7f63817fa640 1 -- 172.21.15.43:0/1724370822 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f636c003970 con 0x7f6370003970 2023-08-09T13:34:01.352 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:01.344+0000 7f63817fa640 1 -- 172.21.15.43:0/1724370822 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f636c003ce0 con 0x7f6370003970 2023-08-09T13:34:01.352 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:01.345+0000 7f638b1f5640 1 -- 172.21.15.43:0/1724370822 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6384153120 con 0x7f6370003970 2023-08-09T13:34:01.353 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:01.345+0000 7f638b1f5640 1 -- 172.21.15.43:0/1724370822 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f63841526f0 con 0x7f6370003970 2023-08-09T13:34:01.354 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:01.346+0000 7f638b1f5640 1 -- 172.21.15.43:0/1724370822 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f63840739f0 con 0x7f6370003970 2023-08-09T13:34:01.354 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:01.347+0000 7f63817fa640 1 -- 172.21.15.43:0/1724370822 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 5) v1 ==== 88341+0+0 (secure 0 0 0) 0x7f636c034440 con 0x7f6370003970 2023-08-09T13:34:01.354 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:01.348+0000 7f63817fa640 1 --2- 172.21.15.43:0/1724370822 >> [v2:172.21.15.203:6832/830097474,v1:172.21.15.203:6833/830097474] conn(0x7f636406a050 0x7f636406c510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:34:01.354 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:01.348+0000 7f63817fa640 1 -- 172.21.15.43:0/1724370822 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(17..17 src has 1..17) v4 ==== 5580+0+0 (secure 0 0 0) 0x7f636c0aa730 con 0x7f6370003970 2023-08-09T13:34:01.354 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:01.349+0000 7f6383fff640 1 --2- 172.21.15.43:0/1724370822 >> [v2:172.21.15.203:6832/830097474,v1:172.21.15.203:6833/830097474] conn(0x7f636406a050 0x7f636406c510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:34:01.355 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:01.350+0000 7f6383fff640 1 --2- 172.21.15.43:0/1724370822 >> [v2:172.21.15.203:6832/830097474,v1:172.21.15.203:6833/830097474] conn(0x7f636406a050 0x7f636406c510 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f6374002230 tx=0x7f6374009040 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-09T13:34:01.355 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:01.353+0000 7f63817fa640 1 -- 172.21.15.43:0/1724370822 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178066 (secure 0 0 0) 0x7f636c0760a0 con 0x7f6370003970 2023-08-09T13:34:01.563 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:01.561+0000 7f638b1f5640 1 -- 172.21.15.43:0/1724370822 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.05} v 0) v1 -- 0x7f6384152880 con 0x7f6370003970 2023-08-09T13:34:02.080 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:02.077+0000 7f63817fa640 1 -- 172.21.15.43:0/1724370822 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.05}]=0 pool 'cephfs_data' created v18) v1 ==== 171+0+0 (secure 0 0 0) 0x7f636c0044e0 con 0x7f6370003970 2023-08-09T13:34:02.087 INFO:teuthology.orchestra.run.smithi043.stderr:pool 'cephfs_data' created 2023-08-09T13:34:02.106 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:02.104+0000 7f638b1f5640 1 -- 172.21.15.43:0/1724370822 >> [v2:172.21.15.203:6832/830097474,v1:172.21.15.203:6833/830097474] conn(0x7f636406a050 msgr2=0x7f636406c510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:34:02.107 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:02.104+0000 7f638b1f5640 1 --2- 172.21.15.43:0/1724370822 >> [v2:172.21.15.203:6832/830097474,v1:172.21.15.203:6833/830097474] conn(0x7f636406a050 0x7f636406c510 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f6374002230 tx=0x7f6374009040 comp rx=0 tx=0).stop 2023-08-09T13:34:02.107 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:02.104+0000 7f638b1f5640 1 -- 172.21.15.43:0/1724370822 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f6370003970 msgr2=0x7f6384151480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:34:02.107 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:02.104+0000 7f638b1f5640 1 --2- 172.21.15.43:0/1724370822 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f6370003970 0x7f6384151480 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f636c02c250 tx=0x7f636c004730 comp rx=0 tx=0).stop 2023-08-09T13:34:02.108 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:02.106+0000 7f638b1f5640 1 -- 172.21.15.43:0/1724370822 shutdown_connections 2023-08-09T13:34:02.109 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:02.106+0000 7f638b1f5640 1 --2- 172.21.15.43:0/1724370822 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f63840d2be0 0x7f63841580a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:34:02.109 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:02.106+0000 7f638b1f5640 1 --2- 172.21.15.43:0/1724370822 >> [v2:172.21.15.203:6832/830097474,v1:172.21.15.203:6833/830097474] conn(0x7f636406a050 0x7f636406c510 unknown :-1 s=CLOSED pgs=60 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:34:02.109 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:02.106+0000 7f638b1f5640 1 --2- 172.21.15.43:0/1724370822 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f63840d2760 0x7f6384157b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:34:02.109 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:02.106+0000 7f638b1f5640 1 --2- 172.21.15.43:0/1724370822 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f6370003970 0x7f6384151480 unknown :-1 s=CLOSED pgs=61 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:34:02.109 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:02.106+0000 7f638b1f5640 1 -- 172.21.15.43:0/1724370822 >> 172.21.15.43:0/1724370822 conn(0x7f6384071520 msgr2=0x7f63840c6070 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-09T13:34:02.110 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:02.108+0000 7f638b1f5640 1 -- 172.21.15.43:0/1724370822 shutdown_connections 2023-08-09T13:34:02.110 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:02.108+0000 7f638b1f5640 1 -- 172.21.15.43:0/1724370822 wait complete. 2023-08-09T13:34:02.131 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs new cephfs cephfs_metadata cephfs_data 2023-08-09T13:34:02.324 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:02.322+0000 7f3b1b59e640 1 Processor -- start 2023-08-09T13:34:02.324 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:02.322+0000 7f3b1b59e640 1 -- start start 2023-08-09T13:34:02.325 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:02.323+0000 7f3b1b59e640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f3b1c0cd910 0x7f3b1c0ccb20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:34:02.326 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:02.323+0000 7f3b1b59e640 1 -- --> v1:172.21.15.43:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3b1c0b56d0 con 0x7f3b1c15c560 2023-08-09T13:34:02.326 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:02.323+0000 7f3b1b59e640 1 -- --> v1:172.21.15.203:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3b1c1605a0 con 0x7f3b1c0d06d0 2023-08-09T13:34:02.326 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:02.323+0000 7f3b1b59e640 1 -- --> v2:172.21.15.43:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3b1c1611b0 con 0x7f3b1c0cd910 2023-08-09T13:34:02.326 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:02.323+0000 7f3b1a59c640 1 --1- >> v1:172.21.15.43:6789/0 conn(0x7f3b1c15c560 0x7f3b1c0cd200 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.43:6789/0 says I am v1:172.21.15.43:40948/0 (socket says 172.21.15.43:40948) 2023-08-09T13:34:02.326 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:02.323+0000 7f3b1a59c640 1 -- 172.21.15.43:0/2211013017 learned_addr learned my addr 172.21.15.43:0/2211013017 (peer_addr_for_me v1:172.21.15.43:0/0) 2023-08-09T13:34:02.327 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:02.323+0000 7f3b19d9b640 1 --2- 172.21.15.43:0/2211013017 >> v2:172.21.15.43:3300/0 conn(0x7f3b1c0cd910 0x7f3b1c0ccb20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:34:02.327 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:02.324+0000 7f3b1959a640 1 -- 172.21.15.43:0/2211013017 <== mon.0 v1:172.21.15.43:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3779768424 0 0) 0x7f3b1c0b56d0 con 0x7f3b1c15c560 2023-08-09T13:34:02.327 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:02.324+0000 7f3b1959a640 1 -- 172.21.15.43:0/2211013017 --> v1:172.21.15.43:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f3b04003520 con 0x7f3b1c15c560 2023-08-09T13:34:02.327 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:02.324+0000 7f3b19d9b640 1 -- 172.21.15.43:0/2211013017 >> v1:172.21.15.203:6789/0 conn(0x7f3b1c0d06d0 legacy=0x7f3b1c0d0b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:34:02.328 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:02.324+0000 7f3b19d9b640 1 -- 172.21.15.43:0/2211013017 >> v1:172.21.15.43:6789/0 conn(0x7f3b1c15c560 legacy=0x7f3b1c0cd200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:34:02.328 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:02.324+0000 7f3b19d9b640 1 -- 172.21.15.43:0/2211013017 --> v2:172.21.15.43:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3b1c160880 con 0x7f3b1c0cd910 2023-08-09T13:34:02.329 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:02.325+0000 7f3b19d9b640 1 --2- 172.21.15.43:0/2211013017 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f3b1c0cd910 0x7f3b1c0ccb20 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f3b100097a0 tx=0x7f3b1002b840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=85525aa12e30e9d6 server_cookie=0 in_seq=0 out_seq=0 2023-08-09T13:34:02.329 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:02.326+0000 7f3b1959a640 1 -- 172.21.15.43:0/2211013017 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3b1002e070 con 0x7f3b1c0cd910 2023-08-09T13:34:02.329 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:02.326+0000 7f3b1959a640 1 -- 172.21.15.43:0/2211013017 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3b100315a0 con 0x7f3b1c0cd910 2023-08-09T13:34:02.329 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:02.326+0000 7f3b1959a640 1 -- 172.21.15.43:0/2211013017 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3b10031930 con 0x7f3b1c0cd910 2023-08-09T13:34:02.329 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:02.326+0000 7f3b1b59e640 1 -- 172.21.15.43:0/2211013017 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f3b1c0cd910 msgr2=0x7f3b1c0ccb20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:34:02.330 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:02.326+0000 7f3b1b59e640 1 --2- 172.21.15.43:0/2211013017 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f3b1c0cd910 0x7f3b1c0ccb20 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f3b100097a0 tx=0x7f3b1002b840 comp rx=0 tx=0).stop 2023-08-09T13:34:02.330 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:02.326+0000 7f3b1b59e640 1 -- 172.21.15.43:0/2211013017 shutdown_connections 2023-08-09T13:34:02.330 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:02.326+0000 7f3b1b59e640 1 --2- 172.21.15.43:0/2211013017 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f3b1c0cd910 0x7f3b1c0ccb20 unknown :-1 s=CLOSED pgs=63 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:34:02.330 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:02.326+0000 7f3b1b59e640 1 -- 172.21.15.43:0/2211013017 >> 172.21.15.43:0/2211013017 conn(0x7f3b1c06a300 msgr2=0x7f3b1c06a6e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-09T13:34:02.330 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:02.327+0000 7f3b1b59e640 1 -- 172.21.15.43:0/2211013017 shutdown_connections 2023-08-09T13:34:02.330 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:02.327+0000 7f3b1b59e640 1 -- 172.21.15.43:0/2211013017 wait complete. 2023-08-09T13:34:02.331 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:02.328+0000 7f3b1b59e640 1 Processor -- start 2023-08-09T13:34:02.331 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:02.328+0000 7f3b1b59e640 1 -- start start 2023-08-09T13:34:02.332 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:02.329+0000 7f3b1b59e640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f3b1c0cd910 0x7f3b1c157150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:34:02.332 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:02.329+0000 7f3b1b59e640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f3b1c0d06d0 0x7f3b1c15f650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:34:02.332 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:02.329+0000 7f3b1b59e640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f3b1c15c560 0x7f3b1c1f56f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:34:02.332 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:02.329+0000 7f3b1b59e640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3b1c15de20 con 0x7f3b1c15c560 2023-08-09T13:34:02.332 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:02.329+0000 7f3b1b59e640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3b1c15d580 con 0x7f3b1c0cd910 2023-08-09T13:34:02.332 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:02.329+0000 7f3b1b59e640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3b1c15d860 con 0x7f3b1c0d06d0 2023-08-09T13:34:02.334 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:02.330+0000 7f3b1ad9d640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f3b1c15c560 0x7f3b1c1f56f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:34:02.334 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:02.330+0000 7f3b1ad9d640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f3b1c15c560 0x7f3b1c1f56f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:57788/0 (socket says 172.21.15.43:57788) 2023-08-09T13:34:02.334 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:02.330+0000 7f3b1ad9d640 1 -- 172.21.15.43:0/3862163925 learned_addr learned my addr 172.21.15.43:0/3862163925 (peer_addr_for_me v2:172.21.15.43:0/0) 2023-08-09T13:34:02.334 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:02.330+0000 7f3b19d9b640 1 --2- 172.21.15.43:0/3862163925 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f3b1c0d06d0 0x7f3b1c15f650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:34:02.334 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:02.330+0000 7f3b1a59c640 1 --2- 172.21.15.43:0/3862163925 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f3b1c0cd910 0x7f3b1c157150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:34:02.334 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:02.330+0000 7f3b1ad9d640 1 -- 172.21.15.43:0/3862163925 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f3b1c0d06d0 msgr2=0x7f3b1c15f650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:34:02.335 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:02.330+0000 7f3b1ad9d640 1 --2- 172.21.15.43:0/3862163925 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f3b1c0d06d0 0x7f3b1c15f650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:34:02.335 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:02.330+0000 7f3b1ad9d640 1 -- 172.21.15.43:0/3862163925 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f3b1c0cd910 msgr2=0x7f3b1c157150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:34:02.335 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:02.330+0000 7f3b1ad9d640 1 --2- 172.21.15.43:0/3862163925 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f3b1c0cd910 0x7f3b1c157150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:34:02.335 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:02.330+0000 7f3b1ad9d640 1 -- 172.21.15.43:0/3862163925 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3b10009400 con 0x7f3b1c15c560 2023-08-09T13:34:02.335 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:02.331+0000 7f3b1ad9d640 1 --2- 172.21.15.43:0/3862163925 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f3b1c15c560 0x7f3b1c1f56f0 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f3b14001eb0 tx=0x7f3b1400c890 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-09T13:34:02.335 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:02.331+0000 7f3b0affd640 1 -- 172.21.15.43:0/3862163925 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3b140052d0 con 0x7f3b1c15c560 2023-08-09T13:34:02.335 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:02.331+0000 7f3b0affd640 1 -- 172.21.15.43:0/3862163925 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3b14005430 con 0x7f3b1c15c560 2023-08-09T13:34:02.336 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:02.331+0000 7f3b0affd640 1 -- 172.21.15.43:0/3862163925 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3b14003810 con 0x7f3b1c15c560 2023-08-09T13:34:02.336 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:02.331+0000 7f3b1b59e640 1 -- 172.21.15.43:0/3862163925 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3b1c15db40 con 0x7f3b1c15c560 2023-08-09T13:34:02.336 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:02.331+0000 7f3b1b59e640 1 -- 172.21.15.43:0/3862163925 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3b1c157690 con 0x7f3b1c15c560 2023-08-09T13:34:02.336 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:02.332+0000 7f3b1b59e640 1 -- 172.21.15.43:0/3862163925 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3b1c0672d0 con 0x7f3b1c15c560 2023-08-09T13:34:02.337 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:02.333+0000 7f3b0affd640 1 -- 172.21.15.43:0/3862163925 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 5) v1 ==== 88341+0+0 (secure 0 0 0) 0x7f3b14003970 con 0x7f3b1c15c560 2023-08-09T13:34:02.337 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:02.334+0000 7f3b0affd640 1 --2- 172.21.15.43:0/3862163925 >> [v2:172.21.15.203:6832/830097474,v1:172.21.15.203:6833/830097474] conn(0x7f3afc06a050 0x7f3afc06c510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:34:02.337 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:02.334+0000 7f3b0affd640 1 -- 172.21.15.43:0/3862163925 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(18..18 src has 1..18) v4 ==== 5962+0+0 (secure 0 0 0) 0x7f3b14087b90 con 0x7f3b1c15c560 2023-08-09T13:34:02.337 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:02.335+0000 7f3b1a59c640 1 --2- 172.21.15.43:0/3862163925 >> [v2:172.21.15.203:6832/830097474,v1:172.21.15.203:6833/830097474] conn(0x7f3afc06a050 0x7f3afc06c510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:34:02.338 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:02.336+0000 7f3b1a59c640 1 --2- 172.21.15.43:0/3862163925 >> [v2:172.21.15.203:6832/830097474,v1:172.21.15.203:6833/830097474] conn(0x7f3afc06a050 0x7f3afc06c510 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f3b0c001560 tx=0x7f3b0c009040 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-09T13:34:02.341 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:02.339+0000 7f3b0affd640 1 -- 172.21.15.43:0/3862163925 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178066 (secure 0 0 0) 0x7f3b14053240 con 0x7f3b1c15c560 2023-08-09T13:34:02.575 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:02.572+0000 7f3b1b59e640 1 -- 172.21.15.43:0/3862163925 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"} v 0) v1 -- 0x7f3b1c1f5030 con 0x7f3b1c15c560 2023-08-09T13:34:03.154 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:03.151+0000 7f3b0affd640 1 -- 172.21.15.43:0/3862163925 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]=0 new fs with metadata pool 3 and data pool 4 v4) v1 ==== 172+0+0 (secure 0 0 0) 0x7f3b1c1f5030 con 0x7f3b1c15c560 2023-08-09T13:34:03.155 INFO:teuthology.orchestra.run.smithi043.stderr:new fs with metadata pool 3 and data pool 4 2023-08-09T13:34:03.159 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:03.156+0000 7f3b1b59e640 1 -- 172.21.15.43:0/3862163925 >> [v2:172.21.15.203:6832/830097474,v1:172.21.15.203:6833/830097474] conn(0x7f3afc06a050 msgr2=0x7f3afc06c510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:34:03.159 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:03.156+0000 7f3b1b59e640 1 --2- 172.21.15.43:0/3862163925 >> [v2:172.21.15.203:6832/830097474,v1:172.21.15.203:6833/830097474] conn(0x7f3afc06a050 0x7f3afc06c510 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f3b0c001560 tx=0x7f3b0c009040 comp rx=0 tx=0).stop 2023-08-09T13:34:03.160 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:03.157+0000 7f3b1b59e640 1 -- 172.21.15.43:0/3862163925 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f3b1c15c560 msgr2=0x7f3b1c1f56f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:34:03.160 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:03.157+0000 7f3b1b59e640 1 --2- 172.21.15.43:0/3862163925 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f3b1c15c560 0x7f3b1c1f56f0 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f3b14001eb0 tx=0x7f3b1400c890 comp rx=0 tx=0).stop 2023-08-09T13:34:03.162 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:03.159+0000 7f3b1b59e640 1 -- 172.21.15.43:0/3862163925 shutdown_connections 2023-08-09T13:34:03.162 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:03.159+0000 7f3b1b59e640 1 --2- 172.21.15.43:0/3862163925 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f3b1c15c560 0x7f3b1c1f56f0 unknown :-1 s=CLOSED pgs=64 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:34:03.162 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:03.159+0000 7f3b1b59e640 1 --2- 172.21.15.43:0/3862163925 >> [v2:172.21.15.203:6832/830097474,v1:172.21.15.203:6833/830097474] conn(0x7f3afc06a050 0x7f3afc06c510 unknown :-1 s=CLOSED pgs=61 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:34:03.162 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:03.159+0000 7f3b1b59e640 1 --2- 172.21.15.43:0/3862163925 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f3b1c0d06d0 0x7f3b1c15f650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:34:03.162 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:03.159+0000 7f3b1b59e640 1 --2- 172.21.15.43:0/3862163925 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f3b1c0cd910 0x7f3b1c157150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:34:03.163 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:03.159+0000 7f3b1b59e640 1 -- 172.21.15.43:0/3862163925 >> 172.21.15.43:0/3862163925 conn(0x7f3b1c06a300 msgr2=0x7f3b1c0d2460 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-09T13:34:03.165 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:03.163+0000 7f3b1b59e640 1 -- 172.21.15.43:0/3862163925 shutdown_connections 2023-08-09T13:34:03.165 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:03.163+0000 7f3b1b59e640 1 -- 172.21.15.43:0/3862163925 wait complete. 2023-08-09T13:34:03.188 DEBUG:tasks.cephfs.filesystem:EC profile is ['m=2', 'k=2', 'crush-failure-domain=osd'] 2023-08-09T13:34:03.188 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd erasure-code-profile set cephfs_data_ec m=2 k=2 crush-failure-domain=osd 2023-08-09T13:34:03.385 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:03.382+0000 7ffbd6499640 1 Processor -- start 2023-08-09T13:34:03.385 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:03.382+0000 7ffbd6499640 1 -- start start 2023-08-09T13:34:03.386 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:03.384+0000 7ffbd6499640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7ffbd00ceac0 0x7ffbd00712f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:34:03.387 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:03.384+0000 7ffbd6499640 1 -- --> v1:172.21.15.43:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ffbd00cadf0 con 0x7ffbd006f990 2023-08-09T13:34:03.387 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:03.384+0000 7ffbd6499640 1 -- --> v1:172.21.15.203:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ffbd00cb0d0 con 0x7ffbd00ca880 2023-08-09T13:34:03.387 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:03.384+0000 7ffbd6499640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ffbd015a880 con 0x7ffbd00ceac0 2023-08-09T13:34:03.388 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:03.384+0000 7ffbcffff640 1 --1- >> v1:172.21.15.43:6789/0 conn(0x7ffbd006f990 0x7ffbd00ce3b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.43:6789/0 says I am v1:172.21.15.43:40952/0 (socket says 172.21.15.43:40952) 2023-08-09T13:34:03.389 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:03.385+0000 7ffbcffff640 1 -- 172.21.15.43:0/2990495816 learned_addr learned my addr 172.21.15.43:0/2990495816 (peer_addr_for_me v1:172.21.15.43:0/0) 2023-08-09T13:34:03.389 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:03.385+0000 7ffbc7fff640 1 --2- 172.21.15.43:0/2990495816 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7ffbd00ceac0 0x7ffbd00712f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:34:03.389 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:03.385+0000 7ffbcf7fe640 1 -- 172.21.15.43:0/2990495816 <== mon.0 v1:172.21.15.43:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 410187812 0 0) 0x7ffbd00cadf0 con 0x7ffbd006f990 2023-08-09T13:34:03.389 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:03.385+0000 7ffbcf7fe640 1 -- 172.21.15.43:0/2990495816 --> v1:172.21.15.43:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ffbbc003520 con 0x7ffbd006f990 2023-08-09T13:34:03.389 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:03.386+0000 7ffbcf7fe640 1 -- 172.21.15.43:0/2990495816 <== mon.1 v1:172.21.15.203:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4180253087 0 0) 0x7ffbd00cb0d0 con 0x7ffbd00ca880 2023-08-09T13:34:03.389 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:03.386+0000 7ffbcf7fe640 1 -- 172.21.15.43:0/2990495816 --> v1:172.21.15.203:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ffbd00cadf0 con 0x7ffbd00ca880 2023-08-09T13:34:03.389 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:03.386+0000 7ffbcf7fe640 1 -- 172.21.15.43:0/2990495816 <== mon.0 v1:172.21.15.43:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1228506337 0 0) 0x7ffbbc003520 con 0x7ffbd006f990 2023-08-09T13:34:03.389 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:03.386+0000 7ffbcf7fe640 1 -- 172.21.15.43:0/2990495816 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7ffbd00ceac0 msgr2=0x7ffbd00712f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-09T13:34:03.390 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:03.386+0000 7ffbcf7fe640 1 --2- 172.21.15.43:0/2990495816 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7ffbd00ceac0 0x7ffbd00712f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:34:03.390 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:03.386+0000 7ffbcf7fe640 1 -- 172.21.15.43:0/2990495816 >> v1:172.21.15.203:6789/0 conn(0x7ffbd00ca880 legacy=0x7ffbd00c8a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:34:03.390 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:03.386+0000 7ffbcf7fe640 1 -- 172.21.15.43:0/2990495816 --> v1:172.21.15.43:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ffbd00cb3b0 con 0x7ffbd006f990 2023-08-09T13:34:03.390 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:03.386+0000 7ffbcf7fe640 1 -- 172.21.15.43:0/2990495816 <== mon.0 v1:172.21.15.43:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1285150740 0 0) 0x7ffbc0001670 con 0x7ffbd006f990 2023-08-09T13:34:03.390 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:03.386+0000 7ffbcf7fe640 1 -- 172.21.15.43:0/2990495816 >> v1:172.21.15.43:6789/0 conn(0x7ffbd006f990 legacy=0x7ffbd00ce3b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:34:03.390 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:03.386+0000 7ffbcf7fe640 1 --2- 172.21.15.43:0/2990495816 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7ffbd00ceac0 0x7ffbbc003b30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:34:03.390 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:03.386+0000 7ffbcf7fe640 1 --2- 172.21.15.43:0/2990495816 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ffbbc0040e0 0x7ffbbc0047c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:34:03.391 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:03.386+0000 7ffbcf7fe640 1 --2- 172.21.15.43:0/2990495816 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7ffbbc004d00 0x7ffbbc005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:34:03.391 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:03.386+0000 7ffbcf7fe640 1 -- 172.21.15.43:0/2990495816 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffbbc00bb10 con 0x7ffbbc0040e0 2023-08-09T13:34:03.391 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:03.386+0000 7ffbcf7fe640 1 -- 172.21.15.43:0/2990495816 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffbbc00bc80 con 0x7ffbbc004d00 2023-08-09T13:34:03.391 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:03.386+0000 7ffbcf7fe640 1 -- 172.21.15.43:0/2990495816 --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ffbbc00be40 con 0x7ffbd00ceac0 2023-08-09T13:34:03.391 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:03.386+0000 7ffbc7fff640 1 --2- 172.21.15.43:0/2990495816 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ffbbc0040e0 0x7ffbbc0047c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:34:03.392 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:03.386+0000 7ffbcffff640 1 --2- 172.21.15.43:0/2990495816 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7ffbd00ceac0 0x7ffbbc003b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:34:03.393 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:03.387+0000 7ffbc7fff640 1 -- 172.21.15.43:0/2990495816 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7ffbd00ceac0 msgr2=0x7ffbbc003b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-09T13:34:03.393 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:03.387+0000 7ffbc7fff640 1 --2- 172.21.15.43:0/2990495816 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7ffbd00ceac0 0x7ffbbc003b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:34:03.393 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:03.387+0000 7ffbc7fff640 1 -- 172.21.15.43:0/2990495816 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7ffbbc004d00 msgr2=0x7ffbbc005180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-09T13:34:03.393 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:03.387+0000 7ffbd4a36640 1 --2- 172.21.15.43:0/2990495816 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7ffbbc004d00 0x7ffbbc005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:34:03.393 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:03.387+0000 7ffbc7fff640 1 --2- 172.21.15.43:0/2990495816 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7ffbbc004d00 0x7ffbbc005180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:34:03.393 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:03.387+0000 7ffbc7fff640 1 -- 172.21.15.43:0/2990495816 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ffbbc003520 con 0x7ffbbc0040e0 2023-08-09T13:34:03.393 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:03.387+0000 7ffbd4a36640 1 --2- 172.21.15.43:0/2990495816 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7ffbbc004d00 0x7ffbbc005180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-08-09T13:34:03.394 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:03.388+0000 7ffbc7fff640 1 --2- 172.21.15.43:0/2990495816 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ffbbc0040e0 0x7ffbbc0047c0 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7ffbb8009ac0 tx=0x7ffbbc0057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7bd8cf6f3ed3cc03 server_cookie=0 in_seq=0 out_seq=0 2023-08-09T13:34:03.394 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:03.388+0000 7ffbcf7fe640 1 -- 172.21.15.43:0/2990495816 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffbb802c6b0 con 0x7ffbbc0040e0 2023-08-09T13:34:03.394 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:03.388+0000 7ffbcf7fe640 1 -- 172.21.15.43:0/2990495816 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ffbb802c810 con 0x7ffbbc0040e0 2023-08-09T13:34:03.394 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:03.388+0000 7ffbcf7fe640 1 -- 172.21.15.43:0/2990495816 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffbb802cba0 con 0x7ffbbc0040e0 2023-08-09T13:34:03.394 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:03.389+0000 7ffbd6499640 1 -- 172.21.15.43:0/2990495816 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ffbbc0040e0 msgr2=0x7ffbbc0047c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:34:03.394 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:03.389+0000 7ffbd6499640 1 --2- 172.21.15.43:0/2990495816 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ffbbc0040e0 0x7ffbbc0047c0 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7ffbb8009ac0 tx=0x7ffbbc0057f0 comp rx=0 tx=0).stop 2023-08-09T13:34:03.394 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:03.389+0000 7ffbd4a36640 1 -- 172.21.15.43:0/2990495816 reap_dead start 2023-08-09T13:34:03.395 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:03.389+0000 7ffbd6499640 1 -- 172.21.15.43:0/2990495816 shutdown_connections 2023-08-09T13:34:03.395 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:03.389+0000 7ffbd6499640 1 -- 172.21.15.43:0/2990495816 >> 172.21.15.43:0/2990495816 conn(0x7ffbd006b9a0 msgr2=0x7ffbd006bdb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-09T13:34:03.395 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:03.389+0000 7ffbd6499640 1 -- 172.21.15.43:0/2990495816 shutdown_connections 2023-08-09T13:34:03.395 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:03.389+0000 7ffbd6499640 1 -- 172.21.15.43:0/2990495816 wait complete. 2023-08-09T13:34:03.397 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:03.391+0000 7ffbd6499640 1 Processor -- start 2023-08-09T13:34:03.397 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:03.391+0000 7ffbd6499640 1 -- start start 2023-08-09T13:34:03.397 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:03.392+0000 7ffbd6499640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7ffbbc0040e0 0x7ffbd0159860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:34:03.397 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:03.392+0000 7ffbd6499640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ffbd00cc1c0 0x7ffbd00cc640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:34:03.397 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:03.392+0000 7ffbd6499640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7ffbd00aed80 0x7ffbd00af200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:34:03.397 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:03.392+0000 7ffbd6499640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffbd00b05e0 con 0x7ffbd00cc1c0 2023-08-09T13:34:03.397 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:03.392+0000 7ffbd6499640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffbd00ccb80 con 0x7ffbd00aed80 2023-08-09T13:34:03.398 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:03.392+0000 7ffbd6499640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ffbd00af740 con 0x7ffbbc0040e0 2023-08-09T13:34:03.398 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:03.392+0000 7ffbc7fff640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ffbd00cc1c0 0x7ffbd00cc640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:34:03.398 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:03.392+0000 7ffbc7fff640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ffbd00cc1c0 0x7ffbd00cc640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:57802/0 (socket says 172.21.15.43:57802) 2023-08-09T13:34:03.398 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:03.392+0000 7ffbc7fff640 1 -- 172.21.15.43:0/2801956858 learned_addr learned my addr 172.21.15.43:0/2801956858 (peer_addr_for_me v2:172.21.15.43:0/0) 2023-08-09T13:34:03.398 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:03.393+0000 7ffbd4a36640 1 --2- 172.21.15.43:0/2801956858 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7ffbd00aed80 0x7ffbd00af200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:34:03.398 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:03.393+0000 7ffbcffff640 1 --2- 172.21.15.43:0/2801956858 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7ffbbc0040e0 0x7ffbd0159860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:34:03.398 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:03.393+0000 7ffbc7fff640 1 -- 172.21.15.43:0/2801956858 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7ffbbc0040e0 msgr2=0x7ffbd0159860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:34:03.398 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:03.393+0000 7ffbc7fff640 1 --2- 172.21.15.43:0/2801956858 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7ffbbc0040e0 0x7ffbd0159860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:34:03.399 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:03.393+0000 7ffbc7fff640 1 -- 172.21.15.43:0/2801956858 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7ffbd00aed80 msgr2=0x7ffbd00af200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:34:03.399 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:03.393+0000 7ffbc7fff640 1 --2- 172.21.15.43:0/2801956858 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7ffbd00aed80 0x7ffbd00af200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:34:03.399 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:03.393+0000 7ffbc7fff640 1 -- 172.21.15.43:0/2801956858 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ffbd00cd020 con 0x7ffbd00cc1c0 2023-08-09T13:34:03.399 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:03.393+0000 7ffbc7fff640 1 --2- 172.21.15.43:0/2801956858 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ffbd00cc1c0 0x7ffbd00cc640 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7ffbb8009ac0 tx=0x7ffbb8004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-09T13:34:03.399 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:03.394+0000 7ffbcd7fa640 1 -- 172.21.15.43:0/2801956858 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffbb8003810 con 0x7ffbd00cc1c0 2023-08-09T13:34:03.399 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:03.394+0000 7ffbcd7fa640 1 -- 172.21.15.43:0/2801956858 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ffbb8003970 con 0x7ffbd00cc1c0 2023-08-09T13:34:03.399 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:03.394+0000 7ffbcd7fa640 1 -- 172.21.15.43:0/2801956858 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffbb8003d00 con 0x7ffbd00cc1c0 2023-08-09T13:34:03.400 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:03.394+0000 7ffbd6499640 1 -- 172.21.15.43:0/2801956858 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ffbd00af960 con 0x7ffbd00cc1c0 2023-08-09T13:34:03.400 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:03.394+0000 7ffbd6499640 1 -- 172.21.15.43:0/2801956858 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ffbd00aff20 con 0x7ffbd00cc1c0 2023-08-09T13:34:03.400 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:03.395+0000 7ffbd6499640 1 -- 172.21.15.43:0/2801956858 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ffbd00739f0 con 0x7ffbd00cc1c0 2023-08-09T13:34:03.401 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:03.396+0000 7ffbcd7fa640 1 -- 172.21.15.43:0/2801956858 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 5) v1 ==== 88341+0+0 (secure 0 0 0) 0x7ffbb8040450 con 0x7ffbd00cc1c0 2023-08-09T13:34:03.401 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:03.397+0000 7ffbcd7fa640 1 --2- 172.21.15.43:0/2801956858 >> [v2:172.21.15.203:6832/830097474,v1:172.21.15.203:6833/830097474] conn(0x7ffbb006a050 0x7ffbb006c510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:34:03.401 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:03.397+0000 7ffbcd7fa640 1 -- 172.21.15.43:0/2801956858 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(19..19 src has 1..19) v4 ==== 6062+0+0 (secure 0 0 0) 0x7ffbb80aa4c0 con 0x7ffbd00cc1c0 2023-08-09T13:34:03.401 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:03.398+0000 7ffbcffff640 1 --2- 172.21.15.43:0/2801956858 >> [v2:172.21.15.203:6832/830097474,v1:172.21.15.203:6833/830097474] conn(0x7ffbb006a050 0x7ffbb006c510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:34:03.401 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:03.399+0000 7ffbcffff640 1 --2- 172.21.15.43:0/2801956858 >> [v2:172.21.15.203:6832/830097474,v1:172.21.15.203:6833/830097474] conn(0x7ffbb006a050 0x7ffbb006c510 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7ffbc0001eb0 tx=0x7ffbc0009040 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-09T13:34:03.404 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:03.402+0000 7ffbcd7fa640 1 -- 172.21.15.43:0/2801956858 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178066 (secure 0 0 0) 0x7ffbb8075b10 con 0x7ffbd00cc1c0 2023-08-09T13:34:03.606 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:03.604+0000 7ffbd6499640 1 -- 172.21.15.43:0/2801956858 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "osd erasure-code-profile set", "name": "cephfs_data_ec", "profile": ["m=2", "k=2", "crush-failure-domain=osd"]} v 0) v1 -- 0x7ffbd006a260 con 0x7ffbd00cc1c0 2023-08-09T13:34:04.129 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:04.126+0000 7ffbcd7fa640 1 -- 172.21.15.43:0/2801956858 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "osd erasure-code-profile set", "name": "cephfs_data_ec", "profile": ["m=2", "k=2", "crush-failure-domain=osd"]}]=0 v20) v1 ==== 157+0+0 (secure 0 0 0) 0x7ffbb8003e60 con 0x7ffbd00cc1c0 2023-08-09T13:34:04.133 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:04.131+0000 7ffbd6499640 1 -- 172.21.15.43:0/2801956858 >> [v2:172.21.15.203:6832/830097474,v1:172.21.15.203:6833/830097474] conn(0x7ffbb006a050 msgr2=0x7ffbb006c510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:34:04.134 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:04.131+0000 7ffbd6499640 1 --2- 172.21.15.43:0/2801956858 >> [v2:172.21.15.203:6832/830097474,v1:172.21.15.203:6833/830097474] conn(0x7ffbb006a050 0x7ffbb006c510 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7ffbc0001eb0 tx=0x7ffbc0009040 comp rx=0 tx=0).stop 2023-08-09T13:34:04.134 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:04.131+0000 7ffbd6499640 1 -- 172.21.15.43:0/2801956858 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ffbd00cc1c0 msgr2=0x7ffbd00cc640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:34:04.134 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:04.131+0000 7ffbd6499640 1 --2- 172.21.15.43:0/2801956858 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ffbd00cc1c0 0x7ffbd00cc640 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7ffbb8009ac0 tx=0x7ffbb8004730 comp rx=0 tx=0).stop 2023-08-09T13:34:04.134 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:04.131+0000 7ffbd6499640 1 -- 172.21.15.43:0/2801956858 shutdown_connections 2023-08-09T13:34:04.134 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:04.131+0000 7ffbd6499640 1 --2- 172.21.15.43:0/2801956858 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7ffbd00aed80 0x7ffbd00af200 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:34:04.135 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:04.131+0000 7ffbd6499640 1 --2- 172.21.15.43:0/2801956858 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ffbd00cc1c0 0x7ffbd00cc640 unknown :-1 s=CLOSED pgs=67 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:34:04.135 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:04.131+0000 7ffbd6499640 1 --2- 172.21.15.43:0/2801956858 >> [v2:172.21.15.203:6832/830097474,v1:172.21.15.203:6833/830097474] conn(0x7ffbb006a050 0x7ffbb006c510 unknown :-1 s=CLOSED pgs=62 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:34:04.135 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:04.131+0000 7ffbd6499640 1 --2- 172.21.15.43:0/2801956858 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7ffbbc0040e0 0x7ffbd0159860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:34:04.135 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:04.131+0000 7ffbd6499640 1 -- 172.21.15.43:0/2801956858 >> 172.21.15.43:0/2801956858 conn(0x7ffbd006b9a0 msgr2=0x7ffbd015ca70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-09T13:34:04.136 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:04.132+0000 7ffbd6499640 1 -- 172.21.15.43:0/2801956858 shutdown_connections 2023-08-09T13:34:04.136 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:04.132+0000 7ffbd6499640 1 -- 172.21.15.43:0/2801956858 wait complete. 2023-08-09T13:34:04.160 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data_ec erasure cephfs_data_ec --pg_num_min 64 --target_size_ratio 0.9 2023-08-09T13:34:04.397 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:04.395+0000 7fb643fff640 1 Processor -- start 2023-08-09T13:34:04.398 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:04.395+0000 7fb643fff640 1 -- start start 2023-08-09T13:34:04.399 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:04.397+0000 7fb643fff640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7fb6440cf3c0 0x7fb6440d0770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:34:04.399 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:04.397+0000 7fb643fff640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fb6440d0cb0 0x7fb6440b6150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:34:04.399 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:04.397+0000 7fb643fff640 1 -- --> v1:172.21.15.203:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb6440b56b0 con 0x7fb64415c950 2023-08-09T13:34:04.400 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:04.397+0000 7fb643fff640 1 -- --> v2:172.21.15.43:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb6440d1aa0 con 0x7fb6440cf3c0 2023-08-09T13:34:04.400 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:04.397+0000 7fb643fff640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb6440d14e0 con 0x7fb6440d0cb0 2023-08-09T13:34:04.401 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:04.397+0000 7fb6427fc640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7fb6440cf3c0 0x7fb6440d0770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:34:04.401 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:04.397+0000 7fb6427fc640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7fb6440cf3c0 0x7fb6440d0770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:57814/0 (socket says 172.21.15.43:57814) 2023-08-09T13:34:04.401 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:04.397+0000 7fb6427fc640 1 -- 172.21.15.43:0/888432290 learned_addr learned my addr 172.21.15.43:0/888432290 (peer_addr_for_me v2:172.21.15.43:0/0) 2023-08-09T13:34:04.401 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:04.397+0000 7fb6437fe640 1 --2- 172.21.15.43:0/888432290 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fb6440d0cb0 0x7fb6440b6150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:34:04.401 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:04.397+0000 7fb6427fc640 1 -- 172.21.15.43:0/888432290 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fb6440d0cb0 msgr2=0x7fb6440b6150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-09T13:34:04.401 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:04.398+0000 7fb6427fc640 1 --2- 172.21.15.43:0/888432290 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fb6440d0cb0 0x7fb6440b6150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:34:04.401 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:04.398+0000 7fb6427fc640 1 -- 172.21.15.43:0/888432290 >> v1:172.21.15.203:6789/0 conn(0x7fb64415c950 legacy=0x7fb6440cecb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:34:04.402 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:04.398+0000 7fb6427fc640 1 -- 172.21.15.43:0/888432290 --> v2:172.21.15.43:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb6440d17c0 con 0x7fb6440cf3c0 2023-08-09T13:34:04.402 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:04.398+0000 7fb6437fe640 1 --2- 172.21.15.43:0/888432290 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fb6440d0cb0 0x7fb6440b6150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-08-09T13:34:04.403 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:04.399+0000 7fb6427fc640 1 --2- 172.21.15.43:0/888432290 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb6440cf3c0 0x7fb6440d0770 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7fb62c009860 tx=0x7fb62c02b980 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a54d1e6d3edf4941 server_cookie=0 in_seq=0 out_seq=0 2023-08-09T13:34:04.403 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:04.399+0000 7fb641ffb640 1 -- 172.21.15.43:0/888432290 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb62c030840 con 0x7fb6440cf3c0 2023-08-09T13:34:04.403 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:04.400+0000 7fb641ffb640 1 -- 172.21.15.43:0/888432290 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb62c0309a0 con 0x7fb6440cf3c0 2023-08-09T13:34:04.403 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:04.400+0000 7fb641ffb640 1 -- 172.21.15.43:0/888432290 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb62c030d30 con 0x7fb6440cf3c0 2023-08-09T13:34:04.403 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:04.400+0000 7fb643fff640 1 -- 172.21.15.43:0/888432290 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb6440cf3c0 msgr2=0x7fb6440d0770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:34:04.403 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:04.400+0000 7fb643fff640 1 --2- 172.21.15.43:0/888432290 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb6440cf3c0 0x7fb6440d0770 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7fb62c009860 tx=0x7fb62c02b980 comp rx=0 tx=0).stop 2023-08-09T13:34:04.404 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:04.400+0000 7fb643fff640 1 -- 172.21.15.43:0/888432290 shutdown_connections 2023-08-09T13:34:04.404 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:04.400+0000 7fb643fff640 1 --2- 172.21.15.43:0/888432290 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fb6440d0cb0 0x7fb6440b6150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:34:04.404 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:04.400+0000 7fb643fff640 1 --2- 172.21.15.43:0/888432290 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb6440cf3c0 0x7fb6440d0770 unknown :-1 s=CLOSED pgs=68 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:34:04.404 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:04.400+0000 7fb643fff640 1 -- 172.21.15.43:0/888432290 >> 172.21.15.43:0/888432290 conn(0x7fb64406a2e0 msgr2=0x7fb64406a6f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-09T13:34:04.404 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:04.400+0000 7fb643fff640 1 -- 172.21.15.43:0/888432290 shutdown_connections 2023-08-09T13:34:04.404 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:04.400+0000 7fb643fff640 1 -- 172.21.15.43:0/888432290 wait complete. 2023-08-09T13:34:04.405 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:04.402+0000 7fb643fff640 1 Processor -- start 2023-08-09T13:34:04.405 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:04.402+0000 7fb643fff640 1 -- start start 2023-08-09T13:34:04.406 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:04.403+0000 7fb643fff640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fb6440cf3c0 0x7fb6440cb5d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:34:04.406 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:04.403+0000 7fb643fff640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb6440d0cb0 0x7fb6440c82e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:34:04.406 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:04.403+0000 7fb643fff640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fb64415c950 0x7fb6441568f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:34:04.406 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:04.403+0000 7fb643fff640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb6440ca8e0 con 0x7fb6440d0cb0 2023-08-09T13:34:04.406 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:04.403+0000 7fb643fff640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb6440c8820 con 0x7fb64415c950 2023-08-09T13:34:04.407 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:04.403+0000 7fb643fff640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb6441f27e0 con 0x7fb6440cf3c0 2023-08-09T13:34:04.407 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:04.404+0000 7fb6427fc640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb6440d0cb0 0x7fb6440c82e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:34:04.407 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:04.404+0000 7fb6427fc640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb6440d0cb0 0x7fb6440c82e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:57820/0 (socket says 172.21.15.43:57820) 2023-08-09T13:34:04.407 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:04.404+0000 7fb6427fc640 1 -- 172.21.15.43:0/952183098 learned_addr learned my addr 172.21.15.43:0/952183098 (peer_addr_for_me v2:172.21.15.43:0/0) 2023-08-09T13:34:04.408 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:04.404+0000 7fb642ffd640 1 --2- 172.21.15.43:0/952183098 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fb6440cf3c0 0x7fb6440cb5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:34:04.408 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:04.404+0000 7fb6427fc640 1 -- 172.21.15.43:0/952183098 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fb6440cf3c0 msgr2=0x7fb6440cb5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:34:04.408 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:04.404+0000 7fb6427fc640 1 --2- 172.21.15.43:0/952183098 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fb6440cf3c0 0x7fb6440cb5d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:34:04.409 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:04.404+0000 7fb6427fc640 1 -- 172.21.15.43:0/952183098 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fb64415c950 msgr2=0x7fb6441568f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:34:04.409 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:04.404+0000 7fb6427fc640 1 --2- 172.21.15.43:0/952183098 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fb64415c950 0x7fb6441568f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:34:04.409 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:04.404+0000 7fb6427fc640 1 -- 172.21.15.43:0/952183098 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb62c009470 con 0x7fb6440d0cb0 2023-08-09T13:34:04.409 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:04.405+0000 7fb6427fc640 1 --2- 172.21.15.43:0/952183098 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb6440d0cb0 0x7fb6440c82e0 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7fb62c000c00 tx=0x7fb62c004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-09T13:34:04.409 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:04.405+0000 7fb63bfff640 1 -- 172.21.15.43:0/952183098 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb62c030810 con 0x7fb6440d0cb0 2023-08-09T13:34:04.409 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:04.405+0000 7fb63bfff640 1 -- 172.21.15.43:0/952183098 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb62c039d90 con 0x7fb6440d0cb0 2023-08-09T13:34:04.409 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:04.405+0000 7fb643fff640 1 -- 172.21.15.43:0/952183098 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb6440caaf0 con 0x7fb6440d0cb0 2023-08-09T13:34:04.409 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:04.405+0000 7fb63bfff640 1 -- 172.21.15.43:0/952183098 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb62c043670 con 0x7fb6440d0cb0 2023-08-09T13:34:04.410 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:04.405+0000 7fb643fff640 1 -- 172.21.15.43:0/952183098 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb6440cb0b0 con 0x7fb6440d0cb0 2023-08-09T13:34:04.410 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:04.406+0000 7fb643fff640 1 -- 172.21.15.43:0/952183098 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb6440cb2a0 con 0x7fb6440d0cb0 2023-08-09T13:34:04.410 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:04.407+0000 7fb63bfff640 1 -- 172.21.15.43:0/952183098 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 5) v1 ==== 88341+0+0 (secure 0 0 0) 0x7fb62c02f390 con 0x7fb6440d0cb0 2023-08-09T13:34:04.411 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:04.408+0000 7fb63bfff640 1 --2- 172.21.15.43:0/952183098 >> [v2:172.21.15.203:6832/830097474,v1:172.21.15.203:6833/830097474] conn(0x7fb62406a050 0x7fb62406c510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:34:04.411 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:04.408+0000 7fb63bfff640 1 -- 172.21.15.43:0/952183098 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 6288+0+0 (secure 0 0 0) 0x7fb62c0aaac0 con 0x7fb6440d0cb0 2023-08-09T13:34:04.411 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:04.409+0000 7fb642ffd640 1 --2- 172.21.15.43:0/952183098 >> [v2:172.21.15.203:6832/830097474,v1:172.21.15.203:6833/830097474] conn(0x7fb62406a050 0x7fb62406c510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:34:04.412 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:04.410+0000 7fb642ffd640 1 --2- 172.21.15.43:0/952183098 >> [v2:172.21.15.203:6832/830097474,v1:172.21.15.203:6833/830097474] conn(0x7fb62406a050 0x7fb62406c510 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7fb64414eee0 tx=0x7fb634009040 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-09T13:34:04.415 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:04.413+0000 7fb63bfff640 1 -- 172.21.15.43:0/952183098 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178066 (secure 0 0 0) 0x7fb62c076170 con 0x7fb6440d0cb0 2023-08-09T13:34:04.621 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:04.619+0000 7fb643fff640 1 -- 172.21.15.43:0/952183098 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data_ec", "pool_type": "erasure", "erasure_code_profile": "cephfs_data_ec", "pg_num_min": 64, "target_size_ratio": 0.9} v 0) v1 -- 0x7fb6440739f0 con 0x7fb6440d0cb0 2023-08-09T13:34:06.192 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:06.189+0000 7fb63bfff640 1 -- 172.21.15.43:0/952183098 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data_ec", "pool_type": "erasure", "erasure_code_profile": "cephfs_data_ec", "pg_num_min": 64, "target_size_ratio": 0.9}]=0 pool 'cephfs_data_ec' created v22) v1 ==== 228+0+0 (secure 0 0 0) 0x7fb62c079790 con 0x7fb6440d0cb0 2023-08-09T13:34:06.193 INFO:teuthology.orchestra.run.smithi043.stderr:pool 'cephfs_data_ec' created 2023-08-09T13:34:06.200 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:06.195+0000 7fb643fff640 1 -- 172.21.15.43:0/952183098 >> [v2:172.21.15.203:6832/830097474,v1:172.21.15.203:6833/830097474] conn(0x7fb62406a050 msgr2=0x7fb62406c510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:34:06.200 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:06.195+0000 7fb643fff640 1 --2- 172.21.15.43:0/952183098 >> [v2:172.21.15.203:6832/830097474,v1:172.21.15.203:6833/830097474] conn(0x7fb62406a050 0x7fb62406c510 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7fb64414eee0 tx=0x7fb634009040 comp rx=0 tx=0).stop 2023-08-09T13:34:06.200 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:06.195+0000 7fb643fff640 1 -- 172.21.15.43:0/952183098 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb6440d0cb0 msgr2=0x7fb6440c82e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:34:06.201 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:06.195+0000 7fb643fff640 1 --2- 172.21.15.43:0/952183098 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb6440d0cb0 0x7fb6440c82e0 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7fb62c000c00 tx=0x7fb62c004730 comp rx=0 tx=0).stop 2023-08-09T13:34:06.206 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:06.203+0000 7fb643fff640 1 -- 172.21.15.43:0/952183098 shutdown_connections 2023-08-09T13:34:06.206 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:06.204+0000 7fb643fff640 1 --2- 172.21.15.43:0/952183098 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fb64415c950 0x7fb6441568f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:34:06.206 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:06.204+0000 7fb643fff640 1 --2- 172.21.15.43:0/952183098 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb6440d0cb0 0x7fb6440c82e0 unknown :-1 s=CLOSED pgs=69 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:34:06.207 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:06.204+0000 7fb643fff640 1 --2- 172.21.15.43:0/952183098 >> [v2:172.21.15.203:6832/830097474,v1:172.21.15.203:6833/830097474] conn(0x7fb62406a050 0x7fb62406c510 unknown :-1 s=CLOSED pgs=63 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:34:06.207 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:06.204+0000 7fb643fff640 1 --2- 172.21.15.43:0/952183098 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fb6440cf3c0 0x7fb6440cb5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:34:06.207 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:06.204+0000 7fb643fff640 1 -- 172.21.15.43:0/952183098 >> 172.21.15.43:0/952183098 conn(0x7fb64406a2e0 msgr2=0x7fb6440cc840 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-09T13:34:06.207 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:06.205+0000 7fb643fff640 1 -- 172.21.15.43:0/952183098 shutdown_connections 2023-08-09T13:34:06.208 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:06.205+0000 7fb643fff640 1 -- 172.21.15.43:0/952183098 wait complete. 2023-08-09T13:34:06.230 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool set cephfs_data_ec allow_ec_overwrites true 2023-08-09T13:34:06.431 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:06.428+0000 7f9631e4b640 1 Processor -- start 2023-08-09T13:34:06.431 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:06.429+0000 7f9631e4b640 1 -- start start 2023-08-09T13:34:06.433 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:06.430+0000 7f9631e4b640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f962c15c400 0x7f962c0d0cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:34:06.433 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:06.430+0000 7f9631e4b640 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7f962c0d1210 0x7f962c0cec70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:34:06.433 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:06.430+0000 7f9631e4b640 1 -- --> v1:172.21.15.43:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f962c0d1690 con 0x7f962c0cd3c0 2023-08-09T13:34:06.433 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:06.431+0000 7f9631e4b640 1 -- --> v2:172.21.15.203:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f962c15c7e0 con 0x7f962c0d1210 2023-08-09T13:34:06.434 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:06.431+0000 7f9631e4b640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f962c0d1970 con 0x7f962c15c400 2023-08-09T13:34:06.434 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:06.431+0000 7f963164a640 1 --1- >> v1:172.21.15.43:6789/0 conn(0x7f962c0cd3c0 0x7f962c0b3d00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.43:6789/0 says I am v1:172.21.15.43:57480/0 (socket says 172.21.15.43:57480) 2023-08-09T13:34:06.435 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:06.431+0000 7f963164a640 1 -- 172.21.15.43:0/1586878548 learned_addr learned my addr 172.21.15.43:0/1586878548 (peer_addr_for_me v1:172.21.15.43:0/0) 2023-08-09T13:34:06.435 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:06.431+0000 7f9630e49640 1 --2- 172.21.15.43:0/1586878548 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f962c15c400 0x7f962c0d0cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:34:06.435 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:06.431+0000 7f962bfff640 1 --2- 172.21.15.43:0/1586878548 >> v2:172.21.15.203:3300/0 conn(0x7f962c0d1210 0x7f962c0cec70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:34:06.435 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:06.432+0000 7f962b7fe640 1 -- 172.21.15.43:0/1586878548 <== mon.0 v1:172.21.15.43:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4259251798 0 0) 0x7f962c0d1690 con 0x7f962c0cd3c0 2023-08-09T13:34:06.435 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:06.432+0000 7f962b7fe640 1 -- 172.21.15.43:0/1586878548 --> v1:172.21.15.43:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f961c003520 con 0x7f962c0cd3c0 2023-08-09T13:34:06.436 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:06.432+0000 7f962b7fe640 1 -- 172.21.15.43:0/1586878548 <== mon.0 v1:172.21.15.43:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 979161591 0 0) 0x7f961c003520 con 0x7f962c0cd3c0 2023-08-09T13:34:06.436 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:06.432+0000 7f9630e49640 1 -- 172.21.15.43:0/1586878548 >> v2:172.21.15.203:3300/0 conn(0x7f962c0d1210 msgr2=0x7f962c0cec70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-09T13:34:06.436 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:06.432+0000 7f9630e49640 1 --2- 172.21.15.43:0/1586878548 >> v2:172.21.15.203:3300/0 conn(0x7f962c0d1210 0x7f962c0cec70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:34:06.436 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:06.432+0000 7f9630e49640 1 -- 172.21.15.43:0/1586878548 >> v1:172.21.15.43:6789/0 conn(0x7f962c0cd3c0 legacy=0x7f962c0b3d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:34:06.436 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:06.432+0000 7f9630e49640 1 -- 172.21.15.43:0/1586878548 --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f962c0d1c50 con 0x7f962c15c400 2023-08-09T13:34:06.436 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:06.432+0000 7f962bfff640 1 --2- 172.21.15.43:0/1586878548 >> v2:172.21.15.203:3300/0 conn(0x7f962c0d1210 0x7f962c0cec70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-08-09T13:34:06.437 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:06.434+0000 7f9630e49640 1 --2- 172.21.15.43:0/1586878548 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f962c15c400 0x7f962c0d0cd0 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7f96200097a0 tx=0x7f962002b820 comp rx=0 tx=0).ready entity=mon.2 client_cookie=892dbef10ff0dceb server_cookie=0 in_seq=0 out_seq=0 2023-08-09T13:34:06.437 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:06.434+0000 7f962b7fe640 1 -- 172.21.15.43:0/1586878548 <== mon.2 v2:172.21.15.203:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f962002f050 con 0x7f962c15c400 2023-08-09T13:34:06.437 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:06.434+0000 7f962b7fe640 1 -- 172.21.15.43:0/1586878548 <== mon.2 v2:172.21.15.203:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9620031500 con 0x7f962c15c400 2023-08-09T13:34:06.438 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:06.434+0000 7f962b7fe640 1 -- 172.21.15.43:0/1586878548 <== mon.2 v2:172.21.15.203:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f96240019f0 con 0x7f962c15c400 2023-08-09T13:34:06.438 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:06.435+0000 7f9631e4b640 1 -- 172.21.15.43:0/1586878548 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f962c15c400 msgr2=0x7f962c0d0cd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:34:06.438 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:06.435+0000 7f9631e4b640 1 --2- 172.21.15.43:0/1586878548 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f962c15c400 0x7f962c0d0cd0 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7f96200097a0 tx=0x7f962002b820 comp rx=0 tx=0).stop 2023-08-09T13:34:06.438 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:06.435+0000 7f9631e4b640 1 -- 172.21.15.43:0/1586878548 shutdown_connections 2023-08-09T13:34:06.438 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:06.435+0000 7f9631e4b640 1 --2- 172.21.15.43:0/1586878548 >> v2:172.21.15.203:3300/0 conn(0x7f962c0d1210 0x7f962c0cec70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:34:06.438 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:06.435+0000 7f9631e4b640 1 --2- 172.21.15.43:0/1586878548 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f962c15c400 0x7f962c0d0cd0 unknown :-1 s=CLOSED pgs=39 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:34:06.438 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:06.435+0000 7f9631e4b640 1 -- 172.21.15.43:0/1586878548 >> 172.21.15.43:0/1586878548 conn(0x7f962c06a320 msgr2=0x7f962c06a730 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-09T13:34:06.439 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:06.435+0000 7f9631e4b640 1 -- 172.21.15.43:0/1586878548 shutdown_connections 2023-08-09T13:34:06.439 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:06.435+0000 7f9631e4b640 1 -- 172.21.15.43:0/1586878548 wait complete. 2023-08-09T13:34:06.439 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:06.437+0000 7f9631e4b640 1 Processor -- start 2023-08-09T13:34:06.439 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:06.437+0000 7f9631e4b640 1 -- start start 2023-08-09T13:34:06.440 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:06.438+0000 7f9631e4b640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f962c0cd3c0 0x7f962c1f5620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:34:06.441 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:06.438+0000 7f9631e4b640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f962c0d1210 0x7f962c152940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:34:06.441 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:06.438+0000 7f9631e4b640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f962c15c400 0x7f962c0b3c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:34:06.441 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:06.438+0000 7f9631e4b640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f962c0b2990 con 0x7f962c15c400 2023-08-09T13:34:06.441 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:06.438+0000 7f9631e4b640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f962c0b2ba0 con 0x7f962c0cd3c0 2023-08-09T13:34:06.441 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:06.438+0000 7f9631e4b640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f962c0b2e80 con 0x7f962c0d1210 2023-08-09T13:34:06.441 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:06.438+0000 7f963164a640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f962c15c400 0x7f962c0b3c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:34:06.442 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:06.438+0000 7f963164a640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f962c15c400 0x7f962c0b3c40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:42570/0 (socket says 172.21.15.43:42570) 2023-08-09T13:34:06.442 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:06.438+0000 7f963164a640 1 -- 172.21.15.43:0/2719046654 learned_addr learned my addr 172.21.15.43:0/2719046654 (peer_addr_for_me v2:172.21.15.43:0/0) 2023-08-09T13:34:06.443 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:06.439+0000 7f962bfff640 1 --2- 172.21.15.43:0/2719046654 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f962c0d1210 0x7f962c152940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:34:06.443 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:06.439+0000 7f9630e49640 1 --2- 172.21.15.43:0/2719046654 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f962c0cd3c0 0x7f962c1f5620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:34:06.443 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:06.439+0000 7f963164a640 1 -- 172.21.15.43:0/2719046654 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f962c0d1210 msgr2=0x7f962c152940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:34:06.443 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:06.439+0000 7f963164a640 1 --2- 172.21.15.43:0/2719046654 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f962c0d1210 0x7f962c152940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:34:06.443 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:06.439+0000 7f963164a640 1 -- 172.21.15.43:0/2719046654 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f962c0cd3c0 msgr2=0x7f962c1f5620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:34:06.443 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:06.439+0000 7f963164a640 1 --2- 172.21.15.43:0/2719046654 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f962c0cd3c0 0x7f962c1f5620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:34:06.443 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:06.439+0000 7f963164a640 1 -- 172.21.15.43:0/2719046654 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9618009400 con 0x7f962c15c400 2023-08-09T13:34:06.444 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:06.439+0000 7f963164a640 1 --2- 172.21.15.43:0/2719046654 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f962c15c400 0x7f962c0b3c40 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f9624001890 tx=0x7f962400a430 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-09T13:34:06.444 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:06.439+0000 7f96297fa640 1 -- 172.21.15.43:0/2719046654 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9624012400 con 0x7f962c15c400 2023-08-09T13:34:06.444 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:06.439+0000 7f96297fa640 1 -- 172.21.15.43:0/2719046654 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9624012560 con 0x7f962c15c400 2023-08-09T13:34:06.444 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:06.439+0000 7f9631e4b640 1 -- 172.21.15.43:0/2719046654 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9620009400 con 0x7f962c15c400 2023-08-09T13:34:06.444 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:06.439+0000 7f96297fa640 1 -- 172.21.15.43:0/2719046654 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9624012910 con 0x7f962c15c400 2023-08-09T13:34:06.444 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:06.440+0000 7f9631e4b640 1 -- 172.21.15.43:0/2719046654 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f962c0b3440 con 0x7f962c15c400 2023-08-09T13:34:06.445 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:06.441+0000 7f9631e4b640 1 -- 172.21.15.43:0/2719046654 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f962c0739f0 con 0x7f962c15c400 2023-08-09T13:34:06.445 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:06.442+0000 7f96297fa640 1 -- 172.21.15.43:0/2719046654 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 5) v1 ==== 88341+0+0 (secure 0 0 0) 0x7f9624012a70 con 0x7f962c15c400 2023-08-09T13:34:06.445 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:06.443+0000 7f96297fa640 1 --2- 172.21.15.43:0/2719046654 >> [v2:172.21.15.203:6832/830097474,v1:172.21.15.203:6833/830097474] conn(0x7f961006a050 0x7f961006c510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:34:06.446 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:06.443+0000 7f96297fa640 1 -- 172.21.15.43:0/2719046654 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 6781+0+0 (secure 0 0 0) 0x7f9624089b80 con 0x7f962c15c400 2023-08-09T13:34:06.446 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:06.444+0000 7f9630e49640 1 --2- 172.21.15.43:0/2719046654 >> [v2:172.21.15.203:6832/830097474,v1:172.21.15.203:6833/830097474] conn(0x7f961006a050 0x7f961006c510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:34:06.447 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:06.445+0000 7f9630e49640 1 --2- 172.21.15.43:0/2719046654 >> [v2:172.21.15.203:6832/830097474,v1:172.21.15.203:6833/830097474] conn(0x7f961006a050 0x7f961006c510 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f9620035000 tx=0x7f9620008030 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-09T13:34:06.449 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:06.447+0000 7f96297fa640 1 -- 172.21.15.43:0/2719046654 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178066 (secure 0 0 0) 0x7f96240550c0 con 0x7f962c15c400 2023-08-09T13:34:06.643 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:06.641+0000 7f9631e4b640 1 -- 172.21.15.43:0/2719046654 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "osd pool set", "pool": "cephfs_data_ec", "var": "allow_ec_overwrites", "val": "true"} v 0) v1 -- 0x7f962c15ed70 con 0x7f962c15c400 2023-08-09T13:34:07.119 INFO:tasks.ceph.mds.d.smithi203.stderr:2023-08-09T13:34:07.117+0000 7ff9ca2d5640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2023-08-09T13:34:07.216 INFO:teuthology.orchestra.run.smithi043.stderr:set pool 5 allow_ec_overwrites to true 2023-08-09T13:34:07.218 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:07.209+0000 7f96297fa640 1 -- 172.21.15.43:0/2719046654 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool set", "pool": "cephfs_data_ec", "var": "allow_ec_overwrites", "val": "true"}]=0 set pool 5 allow_ec_overwrites to true v23) v1 ==== 169+0+0 (secure 0 0 0) 0x7f96240586e0 con 0x7f962c15c400 2023-08-09T13:34:07.228 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:07.225+0000 7f9631e4b640 1 -- 172.21.15.43:0/2719046654 >> [v2:172.21.15.203:6832/830097474,v1:172.21.15.203:6833/830097474] conn(0x7f961006a050 msgr2=0x7f961006c510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:34:07.228 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:07.225+0000 7f9631e4b640 1 --2- 172.21.15.43:0/2719046654 >> [v2:172.21.15.203:6832/830097474,v1:172.21.15.203:6833/830097474] conn(0x7f961006a050 0x7f961006c510 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f9620035000 tx=0x7f9620008030 comp rx=0 tx=0).stop 2023-08-09T13:34:07.228 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:07.225+0000 7f9631e4b640 1 -- 172.21.15.43:0/2719046654 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f962c15c400 msgr2=0x7f962c0b3c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:34:07.228 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:07.225+0000 7f9631e4b640 1 --2- 172.21.15.43:0/2719046654 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f962c15c400 0x7f962c0b3c40 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f9624001890 tx=0x7f962400a430 comp rx=0 tx=0).stop 2023-08-09T13:34:07.234 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:07.231+0000 7f9631e4b640 1 -- 172.21.15.43:0/2719046654 shutdown_connections 2023-08-09T13:34:07.234 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:07.231+0000 7f9631e4b640 1 --2- 172.21.15.43:0/2719046654 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f962c15c400 0x7f962c0b3c40 unknown :-1 s=CLOSED pgs=71 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:34:07.234 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:07.231+0000 7f9631e4b640 1 --2- 172.21.15.43:0/2719046654 >> [v2:172.21.15.203:6832/830097474,v1:172.21.15.203:6833/830097474] conn(0x7f961006a050 0x7f961006c510 unknown :-1 s=CLOSED pgs=64 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:34:07.234 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:07.231+0000 7f9631e4b640 1 --2- 172.21.15.43:0/2719046654 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f962c0d1210 0x7f962c152940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:34:07.234 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:07.231+0000 7f9631e4b640 1 --2- 172.21.15.43:0/2719046654 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f962c0cd3c0 0x7f962c1f5620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:34:07.234 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:07.231+0000 7f9631e4b640 1 -- 172.21.15.43:0/2719046654 >> 172.21.15.43:0/2719046654 conn(0x7f962c06a320 msgr2=0x7f962c0cec70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-09T13:34:07.236 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:07.234+0000 7f9631e4b640 1 -- 172.21.15.43:0/2719046654 shutdown_connections 2023-08-09T13:34:07.236 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:07.234+0000 7f9631e4b640 1 -- 172.21.15.43:0/2719046654 wait complete. 2023-08-09T13:34:07.264 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs add_data_pool cephfs cephfs_data_ec 2023-08-09T13:34:07.455 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:07.453+0000 7fdf428cd640 1 Processor -- start 2023-08-09T13:34:07.456 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:07.453+0000 7fdf428cd640 1 -- start start 2023-08-09T13:34:07.457 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:07.455+0000 7fdf428cd640 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7fdf3c15c2d0 0x7fdf3c0b7df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:34:07.457 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:07.455+0000 7fdf428cd640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7fdf3c0cebf0 0x7fdf3c0cf040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:34:07.457 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:07.455+0000 7fdf428cd640 1 -- --> v1:172.21.15.203:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fdf3c0cf580 con 0x7fdf3c0d1b30 2023-08-09T13:34:07.457 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:07.455+0000 7fdf428cd640 1 -- --> v2:172.21.15.43:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fdf3c0d16a0 con 0x7fdf3c0cebf0 2023-08-09T13:34:07.458 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:07.455+0000 7fdf428cd640 1 -- --> v2:172.21.15.203:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fdf3c0d1fa0 con 0x7fdf3c15c2d0 2023-08-09T13:34:07.458 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:07.455+0000 7fdf420cc640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7fdf3c0cebf0 0x7fdf3c0cf040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:34:07.459 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:07.455+0000 7fdf420cc640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7fdf3c0cebf0 0x7fdf3c0cf040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:42586/0 (socket says 172.21.15.43:42586) 2023-08-09T13:34:07.459 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:07.455+0000 7fdf420cc640 1 -- 172.21.15.43:0/903421098 learned_addr learned my addr 172.21.15.43:0/903421098 (peer_addr_for_me v2:172.21.15.43:0/0) 2023-08-09T13:34:07.459 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:07.455+0000 7fdf418cb640 1 --2- 172.21.15.43:0/903421098 >> v2:172.21.15.203:3300/0 conn(0x7fdf3c15c2d0 0x7fdf3c0b7df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:34:07.459 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:07.456+0000 7fdf420cc640 1 -- 172.21.15.43:0/903421098 >> v2:172.21.15.203:3300/0 conn(0x7fdf3c15c2d0 msgr2=0x7fdf3c0b7df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-09T13:34:07.459 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:07.456+0000 7fdf420cc640 1 --2- 172.21.15.43:0/903421098 >> v2:172.21.15.203:3300/0 conn(0x7fdf3c15c2d0 0x7fdf3c0b7df0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:34:07.459 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:07.456+0000 7fdf420cc640 1 -- 172.21.15.43:0/903421098 >> v1:172.21.15.203:6789/0 conn(0x7fdf3c0d1b30 legacy=0x7fdf3c0cd020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-09T13:34:07.459 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:07.456+0000 7fdf420cc640 1 -- 172.21.15.43:0/903421098 --> v2:172.21.15.43:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdf3c0d0d70 con 0x7fdf3c0cebf0 2023-08-09T13:34:07.460 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:07.456+0000 7fdf418cb640 1 --2- 172.21.15.43:0/903421098 >> v2:172.21.15.203:3300/0 conn(0x7fdf3c15c2d0 0x7fdf3c0b7df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-08-09T13:34:07.460 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:07.457+0000 7fdf420cc640 1 --2- 172.21.15.43:0/903421098 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fdf3c0cebf0 0x7fdf3c0cf040 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7fdf3c06ae00 tx=0x7fdf3402bee0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d9a1561c77b6e976 server_cookie=0 in_seq=0 out_seq=0 2023-08-09T13:34:07.461 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:07.457+0000 7fdf408c9640 1 -- 172.21.15.43:0/903421098 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdf3402ca80 con 0x7fdf3c0cebf0 2023-08-09T13:34:07.461 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:07.457+0000 7fdf408c9640 1 -- 172.21.15.43:0/903421098 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdf3402cbe0 con 0x7fdf3c0cebf0 2023-08-09T13:34:07.461 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:07.457+0000 7fdf408c9640 1 -- 172.21.15.43:0/903421098 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdf340343f0 con 0x7fdf3c0cebf0 2023-08-09T13:34:07.461 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:07.458+0000 7fdf428cd640 1 -- 172.21.15.43:0/903421098 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fdf3c0cebf0 msgr2=0x7fdf3c0cf040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:34:07.461 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:07.458+0000 7fdf428cd640 1 --2- 172.21.15.43:0/903421098 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fdf3c0cebf0 0x7fdf3c0cf040 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7fdf3c06ae00 tx=0x7fdf3402bee0 comp rx=0 tx=0).stop 2023-08-09T13:34:07.461 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:07.458+0000 7fdf428cd640 1 -- 172.21.15.43:0/903421098 shutdown_connections 2023-08-09T13:34:07.461 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:07.458+0000 7fdf428cd640 1 --2- 172.21.15.43:0/903421098 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fdf3c0cebf0 0x7fdf3c0cf040 unknown :-1 s=CLOSED pgs=72 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:34:07.462 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:07.458+0000 7fdf428cd640 1 --2- 172.21.15.43:0/903421098 >> v2:172.21.15.203:3300/0 conn(0x7fdf3c15c2d0 0x7fdf3c0b7df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:34:07.462 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:07.458+0000 7fdf428cd640 1 -- 172.21.15.43:0/903421098 >> 172.21.15.43:0/903421098 conn(0x7fdf3c06a230 msgr2=0x7fdf3c06a640 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-09T13:34:07.462 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:07.458+0000 7fdf428cd640 1 -- 172.21.15.43:0/903421098 shutdown_connections 2023-08-09T13:34:07.462 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:07.458+0000 7fdf428cd640 1 -- 172.21.15.43:0/903421098 wait complete. 2023-08-09T13:34:07.463 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:07.460+0000 7fdf428cd640 1 Processor -- start 2023-08-09T13:34:07.463 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:07.461+0000 7fdf428cd640 1 -- start start 2023-08-09T13:34:07.464 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:07.462+0000 7fdf428cd640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fdf3c0cebf0 0x7fdf3c0c8650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:34:07.465 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:07.462+0000 7fdf428cd640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fdf3c0d1b30 0x7fdf3c0b2fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:34:07.465 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:07.462+0000 7fdf428cd640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fdf3c15c2d0 0x7fdf3c14c1f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:34:07.465 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:07.462+0000 7fdf428cd640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdf3c0cb790 con 0x7fdf3c0d1b30 2023-08-09T13:34:07.465 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:07.462+0000 7fdf428cd640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdf3c0a9cc0 con 0x7fdf3c0cebf0 2023-08-09T13:34:07.465 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:07.462+0000 7fdf428cd640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdf3c0a9fa0 con 0x7fdf3c15c2d0 2023-08-09T13:34:07.465 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:07.462+0000 7fdf410ca640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fdf3c0d1b30 0x7fdf3c0b2fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:34:07.465 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:07.462+0000 7fdf410ca640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fdf3c0d1b30 0x7fdf3c0b2fc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:42590/0 (socket says 172.21.15.43:42590) 2023-08-09T13:34:07.465 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:07.462+0000 7fdf410ca640 1 -- 172.21.15.43:0/1165747409 learned_addr learned my addr 172.21.15.43:0/1165747409 (peer_addr_for_me v2:172.21.15.43:0/0) 2023-08-09T13:34:07.467 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:07.463+0000 7fdf418cb640 1 --2- 172.21.15.43:0/1165747409 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fdf3c0cebf0 0x7fdf3c0c8650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:34:07.467 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:07.463+0000 7fdf420cc640 1 --2- 172.21.15.43:0/1165747409 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fdf3c15c2d0 0x7fdf3c14c1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:34:07.467 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:07.463+0000 7fdf410ca640 1 -- 172.21.15.43:0/1165747409 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fdf3c15c2d0 msgr2=0x7fdf3c14c1f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:34:07.467 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:07.463+0000 7fdf410ca640 1 --2- 172.21.15.43:0/1165747409 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fdf3c15c2d0 0x7fdf3c14c1f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:34:07.467 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:07.463+0000 7fdf410ca640 1 -- 172.21.15.43:0/1165747409 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fdf3c0cebf0 msgr2=0x7fdf3c0c8650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:34:07.467 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:07.463+0000 7fdf410ca640 1 --2- 172.21.15.43:0/1165747409 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fdf3c0cebf0 0x7fdf3c0c8650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:34:07.467 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:07.463+0000 7fdf410ca640 1 -- 172.21.15.43:0/1165747409 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdf34009b30 con 0x7fdf3c0d1b30 2023-08-09T13:34:07.468 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:07.463+0000 7fdf410ca640 1 --2- 172.21.15.43:0/1165747409 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fdf3c0d1b30 0x7fdf3c0b2fc0 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7fdf2c00c400 tx=0x7fdf2c00c8c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-09T13:34:07.468 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:07.463+0000 7fdf327fc640 1 -- 172.21.15.43:0/1165747409 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdf2c011790 con 0x7fdf3c0d1b30 2023-08-09T13:34:07.468 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:07.464+0000 7fdf327fc640 1 -- 172.21.15.43:0/1165747409 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdf2c011930 con 0x7fdf3c0d1b30 2023-08-09T13:34:07.468 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:07.464+0000 7fdf428cd640 1 -- 172.21.15.43:0/1165747409 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdf3c0cb3b0 con 0x7fdf3c0d1b30 2023-08-09T13:34:07.468 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:07.464+0000 7fdf327fc640 1 -- 172.21.15.43:0/1165747409 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdf2c011ce0 con 0x7fdf3c0d1b30 2023-08-09T13:34:07.468 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:07.464+0000 7fdf428cd640 1 -- 172.21.15.43:0/1165747409 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdf3c0b3500 con 0x7fdf3c0d1b30 2023-08-09T13:34:07.469 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:07.465+0000 7fdf428cd640 1 -- 172.21.15.43:0/1165747409 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdf3c04e390 con 0x7fdf3c0d1b30 2023-08-09T13:34:07.469 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:07.466+0000 7fdf327fc640 1 -- 172.21.15.43:0/1165747409 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 5) v1 ==== 88341+0+0 (secure 0 0 0) 0x7fdf2c01c460 con 0x7fdf3c0d1b30 2023-08-09T13:34:07.469 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:07.467+0000 7fdf327fc640 1 --2- 172.21.15.43:0/1165747409 >> [v2:172.21.15.203:6832/830097474,v1:172.21.15.203:6833/830097474] conn(0x7fdf2006a050 0x7fdf2006c510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:34:07.470 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:07.467+0000 7fdf327fc640 1 -- 172.21.15.43:0/1165747409 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 6781+0+0 (secure 0 0 0) 0x7fdf2c088810 con 0x7fdf3c0d1b30 2023-08-09T13:34:07.470 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:07.468+0000 7fdf418cb640 1 --2- 172.21.15.43:0/1165747409 >> [v2:172.21.15.203:6832/830097474,v1:172.21.15.203:6833/830097474] conn(0x7fdf2006a050 0x7fdf2006c510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:34:07.471 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:07.469+0000 7fdf418cb640 1 --2- 172.21.15.43:0/1165747409 >> [v2:172.21.15.203:6832/830097474,v1:172.21.15.203:6833/830097474] conn(0x7fdf2006a050 0x7fdf2006c510 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7fdf3800a990 tx=0x7fdf38008040 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-09T13:34:07.474 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:07.472+0000 7fdf327fc640 1 -- 172.21.15.43:0/1165747409 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178066 (secure 0 0 0) 0x7fdf2c053c20 con 0x7fdf3c0d1b30 2023-08-09T13:34:07.709 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:07.707+0000 7fdf428cd640 1 -- 172.21.15.43:0/1165747409 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "fs add_data_pool", "fs_name": "cephfs", "pool": "cephfs_data_ec"} v 0) v1 -- 0x7fdf3c072e30 con 0x7fdf3c0d1b30 2023-08-09T13:34:08.700 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:08.697+0000 7fdf327fc640 1 -- 172.21.15.43:0/1165747409 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "fs add_data_pool", "fs_name": "cephfs", "pool": "cephfs_data_ec"}]=0 added data pool 5 to fsmap v7) v1 ==== 137+0+0 (secure 0 0 0) 0x7fdf2c057240 con 0x7fdf3c0d1b30 2023-08-09T13:34:08.700 INFO:teuthology.orchestra.run.smithi043.stderr:added data pool 5 to fsmap 2023-08-09T13:34:08.703 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:08.700+0000 7fdf428cd640 1 -- 172.21.15.43:0/1165747409 >> [v2:172.21.15.203:6832/830097474,v1:172.21.15.203:6833/830097474] conn(0x7fdf2006a050 msgr2=0x7fdf2006c510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:34:08.703 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:08.700+0000 7fdf428cd640 1 --2- 172.21.15.43:0/1165747409 >> [v2:172.21.15.203:6832/830097474,v1:172.21.15.203:6833/830097474] conn(0x7fdf2006a050 0x7fdf2006c510 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7fdf3800a990 tx=0x7fdf38008040 comp rx=0 tx=0).stop 2023-08-09T13:34:08.704 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:08.700+0000 7fdf428cd640 1 -- 172.21.15.43:0/1165747409 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fdf3c0d1b30 msgr2=0x7fdf3c0b2fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:34:08.704 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:08.700+0000 7fdf428cd640 1 --2- 172.21.15.43:0/1165747409 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fdf3c0d1b30 0x7fdf3c0b2fc0 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7fdf2c00c400 tx=0x7fdf2c00c8c0 comp rx=0 tx=0).stop 2023-08-09T13:34:08.704 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:08.700+0000 7fdf428cd640 1 -- 172.21.15.43:0/1165747409 shutdown_connections 2023-08-09T13:34:08.704 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:08.701+0000 7fdf428cd640 1 --2- 172.21.15.43:0/1165747409 >> [v2:172.21.15.203:6832/830097474,v1:172.21.15.203:6833/830097474] conn(0x7fdf2006a050 0x7fdf2006c510 unknown :-1 s=CLOSED pgs=65 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:34:08.704 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:08.701+0000 7fdf428cd640 1 --2- 172.21.15.43:0/1165747409 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fdf3c15c2d0 0x7fdf3c14c1f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:34:08.705 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:08.701+0000 7fdf428cd640 1 --2- 172.21.15.43:0/1165747409 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fdf3c0d1b30 0x7fdf3c0b2fc0 unknown :-1 s=CLOSED pgs=73 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:34:08.705 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:08.701+0000 7fdf428cd640 1 --2- 172.21.15.43:0/1165747409 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fdf3c0cebf0 0x7fdf3c0c8650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:34:08.705 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:08.701+0000 7fdf428cd640 1 -- 172.21.15.43:0/1165747409 >> 172.21.15.43:0/1165747409 conn(0x7fdf3c06a230 msgr2=0x7fdf3c0ccee0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-09T13:34:08.705 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:08.701+0000 7fdf428cd640 1 -- 172.21.15.43:0/1165747409 shutdown_connections 2023-08-09T13:34:08.705 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:08.701+0000 7fdf428cd640 1 -- 172.21.15.43:0/1165747409 wait complete. 2023-08-09T13:34:08.725 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-08-09T13:34:08.948 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:08.945+0000 7f8aaf59e640 1 Processor -- start 2023-08-09T13:34:08.948 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:08.945+0000 7f8aaf59e640 1 -- start start 2023-08-09T13:34:08.949 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:08.947+0000 7f8aaf59e640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f8ab00c8750 0x7f8ab00ad340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:34:08.950 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:08.947+0000 7f8aaf59e640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f8ab00acdf0 0x7f8ab015b2c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:34:08.950 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:08.947+0000 7f8aaf59e640 1 -- --> v1:172.21.15.203:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8ab00cbd90 con 0x7f8ab00c8b30 2023-08-09T13:34:08.950 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:08.947+0000 7f8aaf59e640 1 -- --> v2:172.21.15.43:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8ab015e810 con 0x7f8ab00c8750 2023-08-09T13:34:08.950 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:08.947+0000 7f8aaf59e640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8ab00c9f70 con 0x7f8ab00acdf0 2023-08-09T13:34:08.950 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:08.947+0000 7f8aae59c640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f8ab00c8750 0x7f8ab00ad340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:34:08.950 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:08.947+0000 7f8aae59c640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f8ab00c8750 0x7f8ab00ad340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:42592/0 (socket says 172.21.15.43:42592) 2023-08-09T13:34:08.950 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:08.947+0000 7f8aae59c640 1 -- 172.21.15.43:0/3600298490 learned_addr learned my addr 172.21.15.43:0/3600298490 (peer_addr_for_me v2:172.21.15.43:0/0) 2023-08-09T13:34:08.951 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:08.947+0000 7f8aaed9d640 1 --2- 172.21.15.43:0/3600298490 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f8ab00acdf0 0x7f8ab015b2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:34:08.951 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:08.948+0000 7f8aae59c640 1 -- 172.21.15.43:0/3600298490 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f8ab00acdf0 msgr2=0x7f8ab015b2c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-09T13:34:08.951 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:08.948+0000 7f8aae59c640 1 --2- 172.21.15.43:0/3600298490 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f8ab00acdf0 0x7f8ab015b2c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:34:08.952 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:08.948+0000 7f8aae59c640 1 -- 172.21.15.43:0/3600298490 >> v1:172.21.15.203:6789/0 conn(0x7f8ab00c8b30 legacy=0x7f8ab006f410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-09T13:34:08.952 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:08.948+0000 7f8aae59c640 1 -- 172.21.15.43:0/3600298490 --> v2:172.21.15.43:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8ab00ca2b0 con 0x7f8ab00c8750 2023-08-09T13:34:08.952 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:08.949+0000 7f8aae59c640 1 --2- 172.21.15.43:0/3600298490 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f8ab00c8750 0x7f8ab00ad340 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f8aa0009870 tx=0x7f8aa002b860 comp rx=0 tx=0).ready entity=mon.0 client_cookie=20637da94610725f server_cookie=0 in_seq=0 out_seq=0 2023-08-09T13:34:08.953 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:08.950+0000 7f8aadd9b640 1 -- 172.21.15.43:0/3600298490 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8aa00317c0 con 0x7f8ab00c8750 2023-08-09T13:34:08.953 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:08.950+0000 7f8aadd9b640 1 -- 172.21.15.43:0/3600298490 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8aa0031920 con 0x7f8ab00c8750 2023-08-09T13:34:08.953 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:08.950+0000 7f8aadd9b640 1 -- 172.21.15.43:0/3600298490 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8aa0031cb0 con 0x7f8ab00c8750 2023-08-09T13:34:08.953 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:08.950+0000 7f8aaf59e640 1 -- 172.21.15.43:0/3600298490 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f8ab00c8750 msgr2=0x7f8ab00ad340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:34:08.953 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:08.950+0000 7f8aaf59e640 1 --2- 172.21.15.43:0/3600298490 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f8ab00c8750 0x7f8ab00ad340 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f8aa0009870 tx=0x7f8aa002b860 comp rx=0 tx=0).stop 2023-08-09T13:34:08.954 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:08.950+0000 7f8aaf59e640 1 -- 172.21.15.43:0/3600298490 shutdown_connections 2023-08-09T13:34:08.954 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:08.950+0000 7f8aaf59e640 1 --2- 172.21.15.43:0/3600298490 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f8ab00acdf0 0x7f8ab015b2c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:34:08.954 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:08.950+0000 7f8aaf59e640 1 --2- 172.21.15.43:0/3600298490 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f8ab00c8750 0x7f8ab00ad340 unknown :-1 s=CLOSED pgs=74 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:34:08.954 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:08.950+0000 7f8aaf59e640 1 -- 172.21.15.43:0/3600298490 >> 172.21.15.43:0/3600298490 conn(0x7f8ab00cde90 msgr2=0x7f8ab00ce2a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-09T13:34:08.955 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:08.951+0000 7f8aaf59e640 1 -- 172.21.15.43:0/3600298490 shutdown_connections 2023-08-09T13:34:08.955 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:08.951+0000 7f8aaf59e640 1 -- 172.21.15.43:0/3600298490 wait complete. 2023-08-09T13:34:08.955 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:08.952+0000 7f8aaf59e640 1 Processor -- start 2023-08-09T13:34:08.955 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:08.952+0000 7f8aaf59e640 1 -- start start 2023-08-09T13:34:08.957 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:08.953+0000 7f8aaf59e640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f8ab00acdf0 0x7f8ab0158d60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:34:08.957 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:08.953+0000 7f8aaf59e640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f8ab00c8750 0x7f8ab00b26c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:34:08.957 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:08.953+0000 7f8aaf59e640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f8ab00c8b30 0x7f8ab006c1e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:34:08.957 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:08.953+0000 7f8aaf59e640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8ab00c2640 con 0x7f8ab00c8750 2023-08-09T13:34:08.957 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:08.953+0000 7f8aaf59e640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8ab01f2200 con 0x7f8ab00c8b30 2023-08-09T13:34:08.958 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:08.953+0000 7f8aaf59e640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8ab01592a0 con 0x7f8ab00acdf0 2023-08-09T13:34:08.958 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:08.954+0000 7f8aa7fff640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f8ab00c8750 0x7f8ab00b26c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:34:08.958 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:08.954+0000 7f8aa7fff640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f8ab00c8750 0x7f8ab00b26c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:42606/0 (socket says 172.21.15.43:42606) 2023-08-09T13:34:08.958 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:08.954+0000 7f8aa7fff640 1 -- 172.21.15.43:0/4201051627 learned_addr learned my addr 172.21.15.43:0/4201051627 (peer_addr_for_me v2:172.21.15.43:0/0) 2023-08-09T13:34:08.958 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:08.954+0000 7f8aae59c640 1 --2- 172.21.15.43:0/4201051627 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f8ab00acdf0 0x7f8ab0158d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:34:08.958 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:08.954+0000 7f8aaed9d640 1 --2- 172.21.15.43:0/4201051627 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f8ab00c8b30 0x7f8ab006c1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:34:08.958 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:08.954+0000 7f8aa7fff640 1 -- 172.21.15.43:0/4201051627 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f8ab00acdf0 msgr2=0x7f8ab0158d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:34:08.959 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:08.954+0000 7f8aa7fff640 1 --2- 172.21.15.43:0/4201051627 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f8ab00acdf0 0x7f8ab0158d60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:34:08.959 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:08.954+0000 7f8aa7fff640 1 -- 172.21.15.43:0/4201051627 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f8ab00c8b30 msgr2=0x7f8ab006c1e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:34:08.959 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:08.954+0000 7f8aa7fff640 1 --2- 172.21.15.43:0/4201051627 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f8ab00c8b30 0x7f8ab006c1e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:34:08.959 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:08.954+0000 7f8aa7fff640 1 -- 172.21.15.43:0/4201051627 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8aa00094d0 con 0x7f8ab00c8750 2023-08-09T13:34:08.959 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:08.955+0000 7f8aa7fff640 1 --2- 172.21.15.43:0/4201051627 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f8ab00c8750 0x7f8ab00b26c0 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f8a9800c460 tx=0x7f8a9800c920 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-09T13:34:08.960 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:08.955+0000 7f8aa6ffd640 1 -- 172.21.15.43:0/4201051627 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8a98012830 con 0x7f8ab00c8750 2023-08-09T13:34:08.960 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:08.955+0000 7f8aa6ffd640 1 -- 172.21.15.43:0/4201051627 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8a980129d0 con 0x7f8ab00c8750 2023-08-09T13:34:08.960 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:08.955+0000 7f8aaf59e640 1 -- 172.21.15.43:0/4201051627 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8ab00c19c0 con 0x7f8ab00c8750 2023-08-09T13:34:08.960 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:08.955+0000 7f8aa6ffd640 1 -- 172.21.15.43:0/4201051627 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8a98012d80 con 0x7f8ab00c8750 2023-08-09T13:34:08.960 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:08.955+0000 7f8aaf59e640 1 -- 172.21.15.43:0/4201051627 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8ab00c1f80 con 0x7f8ab00c8750 2023-08-09T13:34:08.961 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:08.956+0000 7f8aaf59e640 1 -- 172.21.15.43:0/4201051627 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8ab00c2780 con 0x7f8ab00c8750 2023-08-09T13:34:08.961 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:08.957+0000 7f8aa6ffd640 1 -- 172.21.15.43:0/4201051627 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 5) v1 ==== 88341+0+0 (secure 0 0 0) 0x7f8a9801a3b0 con 0x7f8ab00c8750 2023-08-09T13:34:08.961 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:08.958+0000 7f8aa6ffd640 1 --2- 172.21.15.43:0/4201051627 >> [v2:172.21.15.203:6832/830097474,v1:172.21.15.203:6833/830097474] conn(0x7f8a9006a050 0x7f8a9006c510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:34:08.961 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:08.958+0000 7f8aa6ffd640 1 -- 172.21.15.43:0/4201051627 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6813+0+0 (secure 0 0 0) 0x7f8a98088880 con 0x7f8ab00c8750 2023-08-09T13:34:08.962 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:08.959+0000 7f8aae59c640 1 --2- 172.21.15.43:0/4201051627 >> [v2:172.21.15.203:6832/830097474,v1:172.21.15.203:6833/830097474] conn(0x7f8a9006a050 0x7f8a9006c510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:34:08.962 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:08.960+0000 7f8aae59c640 1 --2- 172.21.15.43:0/4201051627 >> [v2:172.21.15.203:6832/830097474,v1:172.21.15.203:6833/830097474] conn(0x7f8a9006a050 0x7f8a9006c510 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f8aa0036000 tx=0x7f8aa003d040 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-09T13:34:08.965 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:08.962+0000 7f8aa6ffd640 1 -- 172.21.15.43:0/4201051627 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178066 (secure 0 0 0) 0x7f8ab00c2780 con 0x7f8ab00c8750 2023-08-09T13:34:09.198 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:09.196+0000 7f8aaf59e640 1 -- 172.21.15.43:0/4201051627 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f8ab00c2780 con 0x7f8ab00c8750 2023-08-09T13:34:09.199 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:09.197+0000 7f8aa6ffd640 1 -- 172.21.15.43:0/4201051627 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 7 v7) v1 ==== 93+0+4874 (secure 0 0 0) 0x7f8ab00c2780 con 0x7f8ab00c8750 2023-08-09T13:34:09.200 INFO:teuthology.orchestra.run.smithi043.stdout: 2023-08-09T13:34:09.200 INFO:teuthology.orchestra.run.smithi043.stdout:{"epoch":7,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4236,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.43:6833/3917539953","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6832","nonce":3917539953},{"type":"v1","addr":"172.21.15.43:6833","nonce":3917539953}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2},{"gid":4247,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.203:6837/2467794159","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6835","nonce":2467794159},{"type":"v1","addr":"172.21.15.203:6837","nonce":2467794159}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":4248,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.43:6835/1818380189","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6834","nonce":1818380189},{"type":"v1","addr":"172.21.15.43:6835","nonce":1818380189}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":7,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-08-09T13:34:02.575398+0000","modified":"2023-08-09T13:34:08.690983+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":4250},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4250":{"gid":4250,"name":"d","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.203:6836/798312463","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6834","nonce":798312463},{"type":"v1","addr":"172.21.15.203:6836","nonce":798312463}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[4,5],"metadata_pool":3,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":1},"id":1}]} 2023-08-09T13:34:09.201 INFO:teuthology.orchestra.run.smithi043.stderr:dumped fsmap epoch 7 2023-08-09T13:34:09.202 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:09.200+0000 7f8aaf59e640 1 -- 172.21.15.43:0/4201051627 >> [v2:172.21.15.203:6832/830097474,v1:172.21.15.203:6833/830097474] conn(0x7f8a9006a050 msgr2=0x7f8a9006c510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:34:09.203 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:09.200+0000 7f8aaf59e640 1 --2- 172.21.15.43:0/4201051627 >> [v2:172.21.15.203:6832/830097474,v1:172.21.15.203:6833/830097474] conn(0x7f8a9006a050 0x7f8a9006c510 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f8aa0036000 tx=0x7f8aa003d040 comp rx=0 tx=0).stop 2023-08-09T13:34:09.203 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:09.200+0000 7f8aaf59e640 1 -- 172.21.15.43:0/4201051627 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f8ab00c8750 msgr2=0x7f8ab00b26c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:34:09.203 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:09.200+0000 7f8aaf59e640 1 --2- 172.21.15.43:0/4201051627 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f8ab00c8750 0x7f8ab00b26c0 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f8a9800c460 tx=0x7f8a9800c920 comp rx=0 tx=0).stop 2023-08-09T13:34:09.203 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:09.200+0000 7f8aaf59e640 1 -- 172.21.15.43:0/4201051627 shutdown_connections 2023-08-09T13:34:09.203 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:09.200+0000 7f8aaf59e640 1 --2- 172.21.15.43:0/4201051627 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f8ab00c8b30 0x7f8ab006c1e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:34:09.203 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:09.200+0000 7f8aaf59e640 1 --2- 172.21.15.43:0/4201051627 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f8ab00c8750 0x7f8ab00b26c0 unknown :-1 s=CLOSED pgs=75 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:34:09.204 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:09.200+0000 7f8aaf59e640 1 --2- 172.21.15.43:0/4201051627 >> [v2:172.21.15.203:6832/830097474,v1:172.21.15.203:6833/830097474] conn(0x7f8a9006a050 0x7f8a9006c510 unknown :-1 s=CLOSED pgs=66 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:34:09.204 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:09.200+0000 7f8aaf59e640 1 --2- 172.21.15.43:0/4201051627 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f8ab00acdf0 0x7f8ab0158d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:34:09.204 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:09.200+0000 7f8aaf59e640 1 -- 172.21.15.43:0/4201051627 >> 172.21.15.43:0/4201051627 conn(0x7f8ab00cde90 msgr2=0x7f8ab0159f50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-09T13:34:09.204 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:09.200+0000 7f8aaf59e640 1 -- 172.21.15.43:0/4201051627 shutdown_connections 2023-08-09T13:34:09.205 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:09.200+0000 7f8aaf59e640 1 -- 172.21.15.43:0/4201051627 wait complete. 2023-08-09T13:34:09.223 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-08-09T13:34:09.453 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:09.450+0000 7f28fd2ad640 1 Processor -- start 2023-08-09T13:34:09.453 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:09.450+0000 7f28fd2ad640 1 -- start start 2023-08-09T13:34:09.454 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:09.451+0000 7f28fd2ad640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f28f80c8b30 0x7f28f80ad340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:34:09.454 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:09.452+0000 7f28fd2ad640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f28f80acdf0 0x7f28f815b2c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:34:09.454 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:09.452+0000 7f28fd2ad640 1 -- --> v1:172.21.15.203:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f28f80cbd90 con 0x7f28f80c8750 2023-08-09T13:34:09.454 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:09.452+0000 7f28fd2ad640 1 -- --> v2:172.21.15.43:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f28f815e810 con 0x7f28f80c8b30 2023-08-09T13:34:09.455 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:09.452+0000 7f28fd2ad640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f28f80c9f70 con 0x7f28f80acdf0 2023-08-09T13:34:09.455 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:09.452+0000 7f28f77fe640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f28f80c8b30 0x7f28f80ad340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:34:09.456 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:09.452+0000 7f28f77fe640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f28f80c8b30 0x7f28f80ad340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:42614/0 (socket says 172.21.15.43:42614) 2023-08-09T13:34:09.456 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:09.452+0000 7f28f77fe640 1 -- 172.21.15.43:0/2854216886 learned_addr learned my addr 172.21.15.43:0/2854216886 (peer_addr_for_me v2:172.21.15.43:0/0) 2023-08-09T13:34:09.456 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:09.452+0000 7f28fcaac640 1 --2- 172.21.15.43:0/2854216886 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f28f80acdf0 0x7f28f815b2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:34:09.456 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:09.452+0000 7f28f77fe640 1 -- 172.21.15.43:0/2854216886 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f28f80acdf0 msgr2=0x7f28f815b2c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-09T13:34:09.456 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:09.452+0000 7f28f77fe640 1 --2- 172.21.15.43:0/2854216886 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f28f80acdf0 0x7f28f815b2c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:34:09.456 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:09.452+0000 7f28f77fe640 1 -- 172.21.15.43:0/2854216886 >> v1:172.21.15.203:6789/0 conn(0x7f28f80c8750 legacy=0x7f28f806f410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-09T13:34:09.456 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:09.452+0000 7f28f77fe640 1 -- 172.21.15.43:0/2854216886 --> v2:172.21.15.43:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f28f80ca2b0 con 0x7f28f80c8b30 2023-08-09T13:34:09.457 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:09.452+0000 7f28fcaac640 1 --2- 172.21.15.43:0/2854216886 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f28f80acdf0 0x7f28f815b2c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-08-09T13:34:09.457 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:09.453+0000 7f28f77fe640 1 --2- 172.21.15.43:0/2854216886 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f28f80c8b30 0x7f28f80ad340 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f28ec009870 tx=0x7f28ec02b860 comp rx=0 tx=0).ready entity=mon.0 client_cookie=63055eea5f20db74 server_cookie=0 in_seq=0 out_seq=0 2023-08-09T13:34:09.458 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:09.454+0000 7f28f6ffd640 1 -- 172.21.15.43:0/2854216886 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f28ec0317c0 con 0x7f28f80c8b30 2023-08-09T13:34:09.458 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:09.454+0000 7f28f6ffd640 1 -- 172.21.15.43:0/2854216886 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f28ec031920 con 0x7f28f80c8b30 2023-08-09T13:34:09.460 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:09.454+0000 7f28f6ffd640 1 -- 172.21.15.43:0/2854216886 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f28ec031cb0 con 0x7f28f80c8b30 2023-08-09T13:34:09.460 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:09.454+0000 7f28fd2ad640 1 -- 172.21.15.43:0/2854216886 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f28f80c8b30 msgr2=0x7f28f80ad340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:34:09.460 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:09.454+0000 7f28fd2ad640 1 --2- 172.21.15.43:0/2854216886 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f28f80c8b30 0x7f28f80ad340 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f28ec009870 tx=0x7f28ec02b860 comp rx=0 tx=0).stop 2023-08-09T13:34:09.460 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:09.454+0000 7f28fd2ad640 1 -- 172.21.15.43:0/2854216886 shutdown_connections 2023-08-09T13:34:09.460 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:09.454+0000 7f28fd2ad640 1 --2- 172.21.15.43:0/2854216886 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f28f80acdf0 0x7f28f815b2c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:34:09.460 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:09.454+0000 7f28fd2ad640 1 --2- 172.21.15.43:0/2854216886 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f28f80c8b30 0x7f28f80ad340 unknown :-1 s=CLOSED pgs=76 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:34:09.460 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:09.454+0000 7f28fd2ad640 1 -- 172.21.15.43:0/2854216886 >> 172.21.15.43:0/2854216886 conn(0x7f28f80cde90 msgr2=0x7f28f80ce2a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-09T13:34:09.461 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:09.455+0000 7f28fd2ad640 1 -- 172.21.15.43:0/2854216886 shutdown_connections 2023-08-09T13:34:09.461 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:09.455+0000 7f28fd2ad640 1 -- 172.21.15.43:0/2854216886 wait complete. 2023-08-09T13:34:09.462 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:09.456+0000 7f28fd2ad640 1 Processor -- start 2023-08-09T13:34:09.462 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:09.456+0000 7f28fd2ad640 1 -- start start 2023-08-09T13:34:09.462 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:09.457+0000 7f28fd2ad640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f28f80acdf0 0x7f28f81590d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:34:09.463 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:09.457+0000 7f28fd2ad640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f28f80c8750 0x7f28f80b2310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:34:09.463 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:09.457+0000 7f28fd2ad640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f28f80c8b30 0x7f28f8098a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:34:09.463 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:09.457+0000 7f28fd2ad640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f28f80c6670 con 0x7f28f80c8b30 2023-08-09T13:34:09.463 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:09.457+0000 7f28fd2ad640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f28f806c3d0 con 0x7f28f80acdf0 2023-08-09T13:34:09.463 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:09.457+0000 7f28fd2ad640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f28f80c5dd0 con 0x7f28f80c8750 2023-08-09T13:34:09.463 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:09.458+0000 7f28fcaac640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f28f80c8b30 0x7f28f8098a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:34:09.463 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:09.458+0000 7f28fcaac640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f28f80c8b30 0x7f28f8098a50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:42622/0 (socket says 172.21.15.43:42622) 2023-08-09T13:34:09.463 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:09.458+0000 7f28fcaac640 1 -- 172.21.15.43:0/3063287914 learned_addr learned my addr 172.21.15.43:0/3063287914 (peer_addr_for_me v2:172.21.15.43:0/0) 2023-08-09T13:34:09.464 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:09.458+0000 7f28f77fe640 1 --2- 172.21.15.43:0/3063287914 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f28f80c8750 0x7f28f80b2310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:34:09.464 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:09.458+0000 7f28fcaac640 1 -- 172.21.15.43:0/3063287914 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f28f80c8750 msgr2=0x7f28f80b2310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:34:09.464 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:09.458+0000 7f28fcaac640 1 --2- 172.21.15.43:0/3063287914 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f28f80c8750 0x7f28f80b2310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:34:09.464 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:09.458+0000 7f28fcaac640 1 -- 172.21.15.43:0/3063287914 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f28f80acdf0 msgr2=0x7f28f81590d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:34:09.464 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:09.458+0000 7f28fcaac640 1 --2- 172.21.15.43:0/3063287914 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f28f80acdf0 0x7f28f81590d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:34:09.464 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:09.458+0000 7f28fcaac640 1 -- 172.21.15.43:0/3063287914 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f28ec0094d0 con 0x7f28f80c8b30 2023-08-09T13:34:09.464 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:09.458+0000 7f28fcaac640 1 --2- 172.21.15.43:0/3063287914 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f28f80c8b30 0x7f28f8098a50 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f28f000cef0 tx=0x7f28f000a710 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-09T13:34:09.465 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:09.459+0000 7f28f4ff9640 1 -- 172.21.15.43:0/3063287914 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f28f000d840 con 0x7f28f80c8b30 2023-08-09T13:34:09.465 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:09.459+0000 7f28f4ff9640 1 -- 172.21.15.43:0/3063287914 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f28f000d9c0 con 0x7f28f80c8b30 2023-08-09T13:34:09.465 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:09.459+0000 7f28f4ff9640 1 -- 172.21.15.43:0/3063287914 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f28f000dd70 con 0x7f28f80c8b30 2023-08-09T13:34:09.465 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:09.459+0000 7f28fd2ad640 1 -- 172.21.15.43:0/3063287914 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f28f80c60b0 con 0x7f28f80c8b30 2023-08-09T13:34:09.465 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:09.459+0000 7f28fd2ad640 1 -- 172.21.15.43:0/3063287914 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f28f80c5a10 con 0x7f28f80c8b30 2023-08-09T13:34:09.466 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:09.460+0000 7f28fd2ad640 1 -- 172.21.15.43:0/3063287914 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f28f80c9230 con 0x7f28f80c8b30 2023-08-09T13:34:09.466 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:09.461+0000 7f28f4ff9640 1 -- 172.21.15.43:0/3063287914 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 5) v1 ==== 88341+0+0 (secure 0 0 0) 0x7f28f001e3b0 con 0x7f28f80c8b30 2023-08-09T13:34:09.466 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:09.462+0000 7f28f4ff9640 1 --2- 172.21.15.43:0/3063287914 >> [v2:172.21.15.203:6832/830097474,v1:172.21.15.203:6833/830097474] conn(0x7f28e406a080 0x7f28e406c540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:34:09.466 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:09.462+0000 7f28f4ff9640 1 -- 172.21.15.43:0/3063287914 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6813+0+0 (secure 0 0 0) 0x7f28f00893a0 con 0x7f28f80c8b30 2023-08-09T13:34:09.467 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:09.463+0000 7f28f7fff640 1 --2- 172.21.15.43:0/3063287914 >> [v2:172.21.15.203:6832/830097474,v1:172.21.15.203:6833/830097474] conn(0x7f28e406a080 0x7f28e406c540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:34:09.467 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:09.464+0000 7f28f7fff640 1 --2- 172.21.15.43:0/3063287914 >> [v2:172.21.15.203:6832/830097474,v1:172.21.15.203:6833/830097474] conn(0x7f28e406a080 0x7f28e406c540 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f28e8001560 tx=0x7f28e8007990 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-09T13:34:09.468 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:09.466+0000 7f28f4ff9640 1 -- 172.21.15.43:0/3063287914 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178066 (secure 0 0 0) 0x7f28f00547c0 con 0x7f28f80c8b30 2023-08-09T13:34:09.664 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:09.662+0000 7f28fd2ad640 1 -- 172.21.15.43:0/3063287914 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f28f80739f0 con 0x7f28f80c8b30 2023-08-09T13:34:09.666 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:09.664+0000 7f28f4ff9640 1 -- 172.21.15.43:0/3063287914 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v24) v1 ==== 74+0+20050 (secure 0 0 0) 0x7f28f0057de0 con 0x7f28f80c8b30 2023-08-09T13:34:09.667 INFO:teuthology.orchestra.run.smithi043.stdout: 2023-08-09T13:34:09.667 INFO:teuthology.orchestra.run.smithi043.stdout:{"epoch":24,"fsid":"daac87ee-1337-4250-a955-aea4e408a39b","created":"2023-08-09T13:33:41.701198+0000","modified":"2023-08-09T13:34:07.709429+0000","last_up_change":"2023-08-09T13:33:53.783453+0000","last_in_change":"2023-08-09T13:33:45.148904+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":5,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":"rbd","create_time":"2023-08-09T13:33:54.365137+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":".mgr","create_time":"2023-08-09T13:33:54.831803+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_metadata","create_time":"2023-08-09T13:34:00.207974+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data","create_time":"2023-08-09T13:34:01.614765+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":5,"pool_name":"cephfs_data_ec","create_time":"2023-08-09T13:34:05.239583+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"24","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"cbd49ecd-e896-4fbd-864e-5c37a829b165","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6800","nonce":502667335},{"type":"v1","addr":"172.21.15.43:6801","nonce":502667335}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6802","nonce":502667335},{"type":"v1","addr":"172.21.15.43:6803","nonce":502667335}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6806","nonce":502667335},{"type":"v1","addr":"172.21.15.43:6807","nonce":502667335}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6804","nonce":502667335},{"type":"v1","addr":"172.21.15.43:6805","nonce":502667335}]},"public_addr":"172.21.15.43:6801/502667335","cluster_addr":"172.21.15.43:6803/502667335","heartbeat_back_addr":"172.21.15.43:6807/502667335","heartbeat_front_addr":"172.21.15.43:6805/502667335","state":["exists","up"]},{"osd":1,"uuid":"75c7e0fe-1ec7-4fe8-a2f2-3661a65b8500","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6808","nonce":3289134610},{"type":"v1","addr":"172.21.15.43:6809","nonce":3289134610}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6810","nonce":3289134610},{"type":"v1","addr":"172.21.15.43:6812","nonce":3289134610}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6821","nonce":3289134610},{"type":"v1","addr":"172.21.15.43:6824","nonce":3289134610}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6815","nonce":3289134610},{"type":"v1","addr":"172.21.15.43:6818","nonce":3289134610}]},"public_addr":"172.21.15.43:6809/3289134610","cluster_addr":"172.21.15.43:6812/3289134610","heartbeat_back_addr":"172.21.15.43:6824/3289134610","heartbeat_front_addr":"172.21.15.43:6818/3289134610","state":["exists","up"]},{"osd":2,"uuid":"bb5c41e9-7f5f-4125-9b01-3bb8bfc2ea83","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6811","nonce":4079709736},{"type":"v1","addr":"172.21.15.43:6814","nonce":4079709736}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6817","nonce":4079709736},{"type":"v1","addr":"172.21.15.43:6820","nonce":4079709736}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6829","nonce":4079709736},{"type":"v1","addr":"172.21.15.43:6831","nonce":4079709736}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6823","nonce":4079709736},{"type":"v1","addr":"172.21.15.43:6826","nonce":4079709736}]},"public_addr":"172.21.15.43:6814/4079709736","cluster_addr":"172.21.15.43:6820/4079709736","heartbeat_back_addr":"172.21.15.43:6831/4079709736","heartbeat_front_addr":"172.21.15.43:6826/4079709736","state":["exists","up"]},{"osd":3,"uuid":"bfe4c8b4-694a-465c-87e6-2b2aab852613","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6813","nonce":1709538759},{"type":"v1","addr":"172.21.15.43:6816","nonce":1709538759}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6819","nonce":1709538759},{"type":"v1","addr":"172.21.15.43:6822","nonce":1709538759}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6828","nonce":1709538759},{"type":"v1","addr":"172.21.15.43:6830","nonce":1709538759}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6825","nonce":1709538759},{"type":"v1","addr":"172.21.15.43:6827","nonce":1709538759}]},"public_addr":"172.21.15.43:6816/1709538759","cluster_addr":"172.21.15.43:6822/1709538759","heartbeat_back_addr":"172.21.15.43:6830/1709538759","heartbeat_front_addr":"172.21.15.43:6827/1709538759","state":["exists","up"]},{"osd":4,"uuid":"fa2e8a77-aba1-4756-853e-0d174a5bdeba","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6800","nonce":4196866481},{"type":"v1","addr":"172.21.15.203:6801","nonce":4196866481}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6802","nonce":4196866481},{"type":"v1","addr":"172.21.15.203:6803","nonce":4196866481}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6806","nonce":4196866481},{"type":"v1","addr":"172.21.15.203:6807","nonce":4196866481}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6804","nonce":4196866481},{"type":"v1","addr":"172.21.15.203:6805","nonce":4196866481}]},"public_addr":"172.21.15.203:6801/4196866481","cluster_addr":"172.21.15.203:6803/4196866481","heartbeat_back_addr":"172.21.15.203:6807/4196866481","heartbeat_front_addr":"172.21.15.203:6805/4196866481","state":["exists","up"]},{"osd":5,"uuid":"81811e4b-4d24-4f51-a82c-f1d700a2a00d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6816","nonce":3059290081},{"type":"v1","addr":"172.21.15.203:6817","nonce":3059290081}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6818","nonce":3059290081},{"type":"v1","addr":"172.21.15.203:6819","nonce":3059290081}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6822","nonce":3059290081},{"type":"v1","addr":"172.21.15.203:6823","nonce":3059290081}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6820","nonce":3059290081},{"type":"v1","addr":"172.21.15.203:6821","nonce":3059290081}]},"public_addr":"172.21.15.203:6817/3059290081","cluster_addr":"172.21.15.203:6819/3059290081","heartbeat_back_addr":"172.21.15.203:6823/3059290081","heartbeat_front_addr":"172.21.15.203:6821/3059290081","state":["exists","up"]},{"osd":6,"uuid":"307bac4d-6a5b-443d-af75-56d695d7674f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6824","nonce":512155569},{"type":"v1","addr":"172.21.15.203:6825","nonce":512155569}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6826","nonce":512155569},{"type":"v1","addr":"172.21.15.203:6827","nonce":512155569}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6830","nonce":512155569},{"type":"v1","addr":"172.21.15.203:6831","nonce":512155569}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6828","nonce":512155569},{"type":"v1","addr":"172.21.15.203:6829","nonce":512155569}]},"public_addr":"172.21.15.203:6825/512155569","cluster_addr":"172.21.15.203:6827/512155569","heartbeat_back_addr":"172.21.15.203:6831/512155569","heartbeat_front_addr":"172.21.15.203:6829/512155569","state":["exists","up"]},{"osd":7,"uuid":"a95dc7ca-9f1b-4295-af70-68a22a2d7d5e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6808","nonce":3331450438},{"type":"v1","addr":"172.21.15.203:6809","nonce":3331450438}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6810","nonce":3331450438},{"type":"v1","addr":"172.21.15.203:6811","nonce":3331450438}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6814","nonce":3331450438},{"type":"v1","addr":"172.21.15.203:6815","nonce":3331450438}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6812","nonce":3331450438},{"type":"v1","addr":"172.21.15.203:6813","nonce":3331450438}]},"public_addr":"172.21.15.203:6809/3331450438","cluster_addr":"172.21.15.203:6811/3331450438","heartbeat_back_addr":"172.21.15.203:6815/3331450438","heartbeat_front_addr":"172.21.15.203:6813/3331450438","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-09T13:33:50.547652+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-09T13:33:50.589469+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-09T13:33:50.561724+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-09T13:33:50.599650+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-09T13:33:50.411855+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-09T13:33:50.219681+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-09T13:33:50.504664+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-09T13:33:50.441248+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-08-09T13:34:09.669 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:09.667+0000 7f28fd2ad640 1 -- 172.21.15.43:0/3063287914 >> [v2:172.21.15.203:6832/830097474,v1:172.21.15.203:6833/830097474] conn(0x7f28e406a080 msgr2=0x7f28e406c540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:34:09.670 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:09.667+0000 7f28fd2ad640 1 --2- 172.21.15.43:0/3063287914 >> [v2:172.21.15.203:6832/830097474,v1:172.21.15.203:6833/830097474] conn(0x7f28e406a080 0x7f28e406c540 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f28e8001560 tx=0x7f28e8007990 comp rx=0 tx=0).stop 2023-08-09T13:34:09.670 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:09.667+0000 7f28fd2ad640 1 -- 172.21.15.43:0/3063287914 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f28f80c8b30 msgr2=0x7f28f8098a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:34:09.670 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:09.667+0000 7f28fd2ad640 1 --2- 172.21.15.43:0/3063287914 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f28f80c8b30 0x7f28f8098a50 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f28f000cef0 tx=0x7f28f000a710 comp rx=0 tx=0).stop 2023-08-09T13:34:09.670 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:09.667+0000 7f28fd2ad640 1 -- 172.21.15.43:0/3063287914 shutdown_connections 2023-08-09T13:34:09.670 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:09.667+0000 7f28fd2ad640 1 --2- 172.21.15.43:0/3063287914 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f28f80c8b30 0x7f28f8098a50 unknown :-1 s=CLOSED pgs=77 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:34:09.670 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:09.667+0000 7f28fd2ad640 1 --2- 172.21.15.43:0/3063287914 >> [v2:172.21.15.203:6832/830097474,v1:172.21.15.203:6833/830097474] conn(0x7f28e406a080 0x7f28e406c540 unknown :-1 s=CLOSED pgs=67 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:34:09.671 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:09.667+0000 7f28fd2ad640 1 --2- 172.21.15.43:0/3063287914 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f28f80c8750 0x7f28f80b2310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:34:09.671 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:09.667+0000 7f28fd2ad640 1 --2- 172.21.15.43:0/3063287914 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f28f80acdf0 0x7f28f81590d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:34:09.671 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:09.667+0000 7f28fd2ad640 1 -- 172.21.15.43:0/3063287914 >> 172.21.15.43:0/3063287914 conn(0x7f28f80cde90 msgr2=0x7f28f80ac2f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-09T13:34:09.671 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:09.667+0000 7f28fd2ad640 1 -- 172.21.15.43:0/3063287914 shutdown_connections 2023-08-09T13:34:09.671 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:09.667+0000 7f28fd2ad640 1 -- 172.21.15.43:0/3063287914 wait complete. 2023-08-09T13:34:09.689 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-08-09T13:34:09.923 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:09.921+0000 7f5e598f5640 1 Processor -- start 2023-08-09T13:34:09.924 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:09.921+0000 7f5e598f5640 1 -- start start 2023-08-09T13:34:09.925 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:09.923+0000 7f5e598f5640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f5e5415f7b0 0x7f5e540b4760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:34:09.925 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:09.923+0000 7f5e598f5640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f5e540cd6b0 0x7f5e540d0e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:34:09.925 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:09.923+0000 7f5e598f5640 1 -- --> v1:172.21.15.203:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5e540b7bf0 con 0x7f5e540b1050 2023-08-09T13:34:09.925 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:09.923+0000 7f5e598f5640 1 -- --> v2:172.21.15.43:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5e540b7ed0 con 0x7f5e540cd6b0 2023-08-09T13:34:09.926 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:09.923+0000 7f5e598f5640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5e540d0370 con 0x7f5e5415f7b0 2023-08-09T13:34:09.926 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:09.923+0000 7f5e590f4640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f5e540cd6b0 0x7f5e540d0e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:34:09.927 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:09.923+0000 7f5e590f4640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f5e540cd6b0 0x7f5e540d0e50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:42638/0 (socket says 172.21.15.43:42638) 2023-08-09T13:34:09.927 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:09.923+0000 7f5e590f4640 1 -- 172.21.15.43:0/976571386 learned_addr learned my addr 172.21.15.43:0/976571386 (peer_addr_for_me v2:172.21.15.43:0/0) 2023-08-09T13:34:09.927 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:09.923+0000 7f5e588f3640 1 --2- 172.21.15.43:0/976571386 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f5e5415f7b0 0x7f5e540b4760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:34:09.927 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:09.924+0000 7f5e590f4640 1 -- 172.21.15.43:0/976571386 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f5e5415f7b0 msgr2=0x7f5e540b4760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-09T13:34:09.927 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:09.924+0000 7f5e590f4640 1 --2- 172.21.15.43:0/976571386 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f5e5415f7b0 0x7f5e540b4760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:34:09.927 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:09.924+0000 7f5e590f4640 1 -- 172.21.15.43:0/976571386 >> v1:172.21.15.203:6789/0 conn(0x7f5e540b1050 legacy=0x7f5e540ccfa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-09T13:34:09.927 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:09.924+0000 7f5e590f4640 1 -- 172.21.15.43:0/976571386 --> v2:172.21.15.43:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5e540d0650 con 0x7f5e540cd6b0 2023-08-09T13:34:09.929 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:09.925+0000 7f5e590f4640 1 --2- 172.21.15.43:0/976571386 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f5e540cd6b0 0x7f5e540d0e50 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f5e4c009870 tx=0x7f5e4c02b8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a3c57806e382ba8e server_cookie=0 in_seq=0 out_seq=0 2023-08-09T13:34:09.929 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:09.925+0000 7f5e537fe640 1 -- 172.21.15.43:0/976571386 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5e4c031640 con 0x7f5e540cd6b0 2023-08-09T13:34:09.929 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:09.925+0000 7f5e537fe640 1 -- 172.21.15.43:0/976571386 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5e4c0317a0 con 0x7f5e540cd6b0 2023-08-09T13:34:09.929 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:09.925+0000 7f5e537fe640 1 -- 172.21.15.43:0/976571386 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5e4c031b30 con 0x7f5e540cd6b0 2023-08-09T13:34:09.929 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:09.926+0000 7f5e598f5640 1 -- 172.21.15.43:0/976571386 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f5e540cd6b0 msgr2=0x7f5e540d0e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:34:09.929 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:09.926+0000 7f5e598f5640 1 --2- 172.21.15.43:0/976571386 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f5e540cd6b0 0x7f5e540d0e50 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f5e4c009870 tx=0x7f5e4c02b8f0 comp rx=0 tx=0).stop 2023-08-09T13:34:09.930 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:09.926+0000 7f5e598f5640 1 -- 172.21.15.43:0/976571386 shutdown_connections 2023-08-09T13:34:09.930 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:09.926+0000 7f5e598f5640 1 --2- 172.21.15.43:0/976571386 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f5e540cd6b0 0x7f5e540d0e50 unknown :-1 s=CLOSED pgs=78 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:34:09.930 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:09.926+0000 7f5e598f5640 1 --2- 172.21.15.43:0/976571386 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f5e5415f7b0 0x7f5e540b4760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:34:09.930 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:09.926+0000 7f5e598f5640 1 -- 172.21.15.43:0/976571386 >> 172.21.15.43:0/976571386 conn(0x7f5e5406a2b0 msgr2=0x7f5e5406a690 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-09T13:34:09.930 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:09.926+0000 7f5e598f5640 1 -- 172.21.15.43:0/976571386 shutdown_connections 2023-08-09T13:34:09.930 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:09.926+0000 7f5e598f5640 1 -- 172.21.15.43:0/976571386 wait complete. 2023-08-09T13:34:09.931 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:09.928+0000 7f5e598f5640 1 Processor -- start 2023-08-09T13:34:09.932 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:09.928+0000 7f5e598f5640 1 -- start start 2023-08-09T13:34:09.932 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:09.929+0000 7f5e598f5640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f5e540b1050 0x7f5e5414d830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:34:09.932 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:09.929+0000 7f5e598f5640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f5e540cd6b0 0x7f5e540d2bb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:34:09.932 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:09.929+0000 7f5e598f5640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f5e5415f7b0 0x7f5e540b6230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:34:09.932 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:09.929+0000 7f5e598f5640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5e540d2920 con 0x7f5e540b1050 2023-08-09T13:34:09.932 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:09.929+0000 7f5e598f5640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5e540d11d0 con 0x7f5e540cd6b0 2023-08-09T13:34:09.932 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:09.929+0000 7f5e598f5640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5e540d14b0 con 0x7f5e5415f7b0 2023-08-09T13:34:09.933 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:09.929+0000 7f5e588f3640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f5e540b1050 0x7f5e5414d830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:34:09.933 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:09.929+0000 7f5e588f3640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f5e540b1050 0x7f5e5414d830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:42640/0 (socket says 172.21.15.43:42640) 2023-08-09T13:34:09.933 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:09.929+0000 7f5e588f3640 1 -- 172.21.15.43:0/544965900 learned_addr learned my addr 172.21.15.43:0/544965900 (peer_addr_for_me v2:172.21.15.43:0/0) 2023-08-09T13:34:09.934 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:09.929+0000 7f5e588f3640 1 -- 172.21.15.43:0/544965900 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f5e5415f7b0 msgr2=0x7f5e540b6230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:34:09.934 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:09.929+0000 7f5e588f3640 1 --2- 172.21.15.43:0/544965900 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f5e5415f7b0 0x7f5e540b6230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:34:09.934 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:09.929+0000 7f5e588f3640 1 -- 172.21.15.43:0/544965900 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f5e540cd6b0 msgr2=0x7f5e540d2bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:34:09.934 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:09.929+0000 7f5e588f3640 1 --2- 172.21.15.43:0/544965900 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f5e540cd6b0 0x7f5e540d2bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:34:09.934 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:09.929+0000 7f5e588f3640 1 -- 172.21.15.43:0/544965900 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5e4c0094d0 con 0x7f5e540b1050 2023-08-09T13:34:09.935 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:09.930+0000 7f5e588f3640 1 --2- 172.21.15.43:0/544965900 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f5e540b1050 0x7f5e5414d830 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f5e44007e40 tx=0x7f5e4400e4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-09T13:34:09.935 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:09.930+0000 7f5e517fa640 1 -- 172.21.15.43:0/544965900 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5e44014620 con 0x7f5e540b1050 2023-08-09T13:34:09.935 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:09.930+0000 7f5e517fa640 1 -- 172.21.15.43:0/544965900 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5e4400ee90 con 0x7f5e540b1050 2023-08-09T13:34:09.935 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:09.930+0000 7f5e598f5640 1 -- 172.21.15.43:0/544965900 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5e540d1790 con 0x7f5e540b1050 2023-08-09T13:34:09.935 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:09.930+0000 7f5e517fa640 1 -- 172.21.15.43:0/544965900 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5e440149a0 con 0x7f5e540b1050 2023-08-09T13:34:09.935 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:09.930+0000 7f5e598f5640 1 -- 172.21.15.43:0/544965900 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5e5414dd70 con 0x7f5e540b1050 2023-08-09T13:34:09.935 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:09.931+0000 7f5e598f5640 1 -- 172.21.15.43:0/544965900 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5e541f5430 con 0x7f5e540b1050 2023-08-09T13:34:09.936 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:09.932+0000 7f5e517fa640 1 -- 172.21.15.43:0/544965900 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 5) v1 ==== 88341+0+0 (secure 0 0 0) 0x7f5e44014b00 con 0x7f5e540b1050 2023-08-09T13:34:09.936 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:09.933+0000 7f5e517fa640 1 --2- 172.21.15.43:0/544965900 >> [v2:172.21.15.203:6832/830097474,v1:172.21.15.203:6833/830097474] conn(0x7f5e4006a050 0x7f5e4006c510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:34:09.936 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:09.933+0000 7f5e517fa640 1 -- 172.21.15.43:0/544965900 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6813+0+0 (secure 0 0 0) 0x7f5e440892f0 con 0x7f5e540b1050 2023-08-09T13:34:09.937 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:09.934+0000 7f5e53fff640 1 --2- 172.21.15.43:0/544965900 >> [v2:172.21.15.203:6832/830097474,v1:172.21.15.203:6833/830097474] conn(0x7f5e4006a050 0x7f5e4006c510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:34:09.937 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:09.935+0000 7f5e53fff640 1 --2- 172.21.15.43:0/544965900 >> [v2:172.21.15.203:6832/830097474,v1:172.21.15.203:6833/830097474] conn(0x7f5e4006a050 0x7f5e4006c510 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f5e54151370 tx=0x7f5e480079b0 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-09T13:34:09.940 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:09.938+0000 7f5e517fa640 1 -- 172.21.15.43:0/544965900 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178066 (secure 0 0 0) 0x7f5e44054650 con 0x7f5e540b1050 2023-08-09T13:34:10.137 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:10.134+0000 7f5e598f5640 1 -- 172.21.15.43:0/544965900 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f5e540739f0 con 0x7f5e540b1050 2023-08-09T13:34:10.140 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:10.138+0000 7f5e517fa640 1 -- 172.21.15.43:0/544965900 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v24) v1 ==== 74+0+20050 (secure 0 0 0) 0x7f5e44014db0 con 0x7f5e540b1050 2023-08-09T13:34:10.141 INFO:teuthology.orchestra.run.smithi043.stdout: 2023-08-09T13:34:10.141 INFO:teuthology.orchestra.run.smithi043.stdout:{"epoch":24,"fsid":"daac87ee-1337-4250-a955-aea4e408a39b","created":"2023-08-09T13:33:41.701198+0000","modified":"2023-08-09T13:34:07.709429+0000","last_up_change":"2023-08-09T13:33:53.783453+0000","last_in_change":"2023-08-09T13:33:45.148904+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":5,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":"rbd","create_time":"2023-08-09T13:33:54.365137+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":".mgr","create_time":"2023-08-09T13:33:54.831803+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_metadata","create_time":"2023-08-09T13:34:00.207974+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data","create_time":"2023-08-09T13:34:01.614765+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":5,"pool_name":"cephfs_data_ec","create_time":"2023-08-09T13:34:05.239583+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"24","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"cbd49ecd-e896-4fbd-864e-5c37a829b165","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6800","nonce":502667335},{"type":"v1","addr":"172.21.15.43:6801","nonce":502667335}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6802","nonce":502667335},{"type":"v1","addr":"172.21.15.43:6803","nonce":502667335}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6806","nonce":502667335},{"type":"v1","addr":"172.21.15.43:6807","nonce":502667335}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6804","nonce":502667335},{"type":"v1","addr":"172.21.15.43:6805","nonce":502667335}]},"public_addr":"172.21.15.43:6801/502667335","cluster_addr":"172.21.15.43:6803/502667335","heartbeat_back_addr":"172.21.15.43:6807/502667335","heartbeat_front_addr":"172.21.15.43:6805/502667335","state":["exists","up"]},{"osd":1,"uuid":"75c7e0fe-1ec7-4fe8-a2f2-3661a65b8500","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6808","nonce":3289134610},{"type":"v1","addr":"172.21.15.43:6809","nonce":3289134610}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6810","nonce":3289134610},{"type":"v1","addr":"172.21.15.43:6812","nonce":3289134610}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6821","nonce":3289134610},{"type":"v1","addr":"172.21.15.43:6824","nonce":3289134610}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6815","nonce":3289134610},{"type":"v1","addr":"172.21.15.43:6818","nonce":3289134610}]},"public_addr":"172.21.15.43:6809/3289134610","cluster_addr":"172.21.15.43:6812/3289134610","heartbeat_back_addr":"172.21.15.43:6824/3289134610","heartbeat_front_addr":"172.21.15.43:6818/3289134610","state":["exists","up"]},{"osd":2,"uuid":"bb5c41e9-7f5f-4125-9b01-3bb8bfc2ea83","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6811","nonce":4079709736},{"type":"v1","addr":"172.21.15.43:6814","nonce":4079709736}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6817","nonce":4079709736},{"type":"v1","addr":"172.21.15.43:6820","nonce":4079709736}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6829","nonce":4079709736},{"type":"v1","addr":"172.21.15.43:6831","nonce":4079709736}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6823","nonce":4079709736},{"type":"v1","addr":"172.21.15.43:6826","nonce":4079709736}]},"public_addr":"172.21.15.43:6814/4079709736","cluster_addr":"172.21.15.43:6820/4079709736","heartbeat_back_addr":"172.21.15.43:6831/4079709736","heartbeat_front_addr":"172.21.15.43:6826/4079709736","state":["exists","up"]},{"osd":3,"uuid":"bfe4c8b4-694a-465c-87e6-2b2aab852613","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6813","nonce":1709538759},{"type":"v1","addr":"172.21.15.43:6816","nonce":1709538759}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6819","nonce":1709538759},{"type":"v1","addr":"172.21.15.43:6822","nonce":1709538759}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6828","nonce":1709538759},{"type":"v1","addr":"172.21.15.43:6830","nonce":1709538759}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6825","nonce":1709538759},{"type":"v1","addr":"172.21.15.43:6827","nonce":1709538759}]},"public_addr":"172.21.15.43:6816/1709538759","cluster_addr":"172.21.15.43:6822/1709538759","heartbeat_back_addr":"172.21.15.43:6830/1709538759","heartbeat_front_addr":"172.21.15.43:6827/1709538759","state":["exists","up"]},{"osd":4,"uuid":"fa2e8a77-aba1-4756-853e-0d174a5bdeba","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6800","nonce":4196866481},{"type":"v1","addr":"172.21.15.203:6801","nonce":4196866481}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6802","nonce":4196866481},{"type":"v1","addr":"172.21.15.203:6803","nonce":4196866481}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6806","nonce":4196866481},{"type":"v1","addr":"172.21.15.203:6807","nonce":4196866481}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6804","nonce":4196866481},{"type":"v1","addr":"172.21.15.203:6805","nonce":4196866481}]},"public_addr":"172.21.15.203:6801/4196866481","cluster_addr":"172.21.15.203:6803/4196866481","heartbeat_back_addr":"172.21.15.203:6807/4196866481","heartbeat_front_addr":"172.21.15.203:6805/4196866481","state":["exists","up"]},{"osd":5,"uuid":"81811e4b-4d24-4f51-a82c-f1d700a2a00d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6816","nonce":3059290081},{"type":"v1","addr":"172.21.15.203:6817","nonce":3059290081}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6818","nonce":3059290081},{"type":"v1","addr":"172.21.15.203:6819","nonce":3059290081}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6822","nonce":3059290081},{"type":"v1","addr":"172.21.15.203:6823","nonce":3059290081}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6820","nonce":3059290081},{"type":"v1","addr":"172.21.15.203:6821","nonce":3059290081}]},"public_addr":"172.21.15.203:6817/3059290081","cluster_addr":"172.21.15.203:6819/3059290081","heartbeat_back_addr":"172.21.15.203:6823/3059290081","heartbeat_front_addr":"172.21.15.203:6821/3059290081","state":["exists","up"]},{"osd":6,"uuid":"307bac4d-6a5b-443d-af75-56d695d7674f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6824","nonce":512155569},{"type":"v1","addr":"172.21.15.203:6825","nonce":512155569}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6826","nonce":512155569},{"type":"v1","addr":"172.21.15.203:6827","nonce":512155569}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6830","nonce":512155569},{"type":"v1","addr":"172.21.15.203:6831","nonce":512155569}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6828","nonce":512155569},{"type":"v1","addr":"172.21.15.203:6829","nonce":512155569}]},"public_addr":"172.21.15.203:6825/512155569","cluster_addr":"172.21.15.203:6827/512155569","heartbeat_back_addr":"172.21.15.203:6831/512155569","heartbeat_front_addr":"172.21.15.203:6829/512155569","state":["exists","up"]},{"osd":7,"uuid":"a95dc7ca-9f1b-4295-af70-68a22a2d7d5e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6808","nonce":3331450438},{"type":"v1","addr":"172.21.15.203:6809","nonce":3331450438}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6810","nonce":3331450438},{"type":"v1","addr":"172.21.15.203:6811","nonce":3331450438}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6814","nonce":3331450438},{"type":"v1","addr":"172.21.15.203:6815","nonce":3331450438}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6812","nonce":3331450438},{"type":"v1","addr":"172.21.15.203:6813","nonce":3331450438}]},"public_addr":"172.21.15.203:6809/3331450438","cluster_addr":"172.21.15.203:6811/3331450438","heartbeat_back_addr":"172.21.15.203:6815/3331450438","heartbeat_front_addr":"172.21.15.203:6813/3331450438","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-09T13:33:50.547652+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-09T13:33:50.589469+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-09T13:33:50.561724+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-09T13:33:50.599650+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-09T13:33:50.411855+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-09T13:33:50.219681+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-09T13:33:50.504664+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-09T13:33:50.441248+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-08-09T13:34:10.143 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:10.140+0000 7f5e598f5640 1 -- 172.21.15.43:0/544965900 >> [v2:172.21.15.203:6832/830097474,v1:172.21.15.203:6833/830097474] conn(0x7f5e4006a050 msgr2=0x7f5e4006c510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:34:10.143 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:10.140+0000 7f5e598f5640 1 --2- 172.21.15.43:0/544965900 >> [v2:172.21.15.203:6832/830097474,v1:172.21.15.203:6833/830097474] conn(0x7f5e4006a050 0x7f5e4006c510 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f5e54151370 tx=0x7f5e480079b0 comp rx=0 tx=0).stop 2023-08-09T13:34:10.143 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:10.140+0000 7f5e598f5640 1 -- 172.21.15.43:0/544965900 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f5e540b1050 msgr2=0x7f5e5414d830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:34:10.143 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:10.140+0000 7f5e598f5640 1 --2- 172.21.15.43:0/544965900 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f5e540b1050 0x7f5e5414d830 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f5e44007e40 tx=0x7f5e4400e4b0 comp rx=0 tx=0).stop 2023-08-09T13:34:10.144 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:10.141+0000 7f5e598f5640 1 -- 172.21.15.43:0/544965900 shutdown_connections 2023-08-09T13:34:10.144 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:10.141+0000 7f5e598f5640 1 --2- 172.21.15.43:0/544965900 >> [v2:172.21.15.203:6832/830097474,v1:172.21.15.203:6833/830097474] conn(0x7f5e4006a050 0x7f5e4006c510 unknown :-1 s=CLOSED pgs=68 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:34:10.144 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:10.141+0000 7f5e598f5640 1 --2- 172.21.15.43:0/544965900 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f5e5415f7b0 0x7f5e540b6230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:34:10.144 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:10.141+0000 7f5e598f5640 1 --2- 172.21.15.43:0/544965900 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f5e540cd6b0 0x7f5e540d2bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:34:10.144 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:10.141+0000 7f5e598f5640 1 --2- 172.21.15.43:0/544965900 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f5e540b1050 0x7f5e5414d830 unknown :-1 s=CLOSED pgs=79 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:34:10.145 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:10.141+0000 7f5e598f5640 1 -- 172.21.15.43:0/544965900 >> 172.21.15.43:0/544965900 conn(0x7f5e5406a2b0 msgr2=0x7f5e5415add0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-09T13:34:10.145 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:10.141+0000 7f5e598f5640 1 -- 172.21.15.43:0/544965900 shutdown_connections 2023-08-09T13:34:10.145 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:10.141+0000 7f5e598f5640 1 -- 172.21.15.43:0/544965900 wait complete. 2023-08-09T13:34:10.180 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-08-09T13:34:10.415 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:10.412+0000 7fd4548c5640 1 Processor -- start 2023-08-09T13:34:10.415 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:10.412+0000 7fd4548c5640 1 -- start start 2023-08-09T13:34:10.417 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:10.414+0000 7fd4548c5640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7fd45014a5c0 0x7fd450160f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:34:10.417 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:10.414+0000 7fd4548c5640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fd4500c3130 0x7fd4500cbc70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:34:10.417 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:10.414+0000 7fd4548c5640 1 -- --> v1:172.21.15.203:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd4500c38a0 con 0x7fd4500cc1b0 2023-08-09T13:34:10.417 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:10.414+0000 7fd4548c5640 1 -- --> v2:172.21.15.43:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd4500c4420 con 0x7fd45014a5c0 2023-08-09T13:34:10.417 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:10.414+0000 7fd4548c5640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd4500cc640 con 0x7fd4500c3130 2023-08-09T13:34:10.418 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:10.415+0000 7fd44e59c640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7fd45014a5c0 0x7fd450160f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:34:10.418 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:10.415+0000 7fd44e59c640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7fd45014a5c0 0x7fd450160f20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:42654/0 (socket says 172.21.15.43:42654) 2023-08-09T13:34:10.418 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:10.415+0000 7fd44e59c640 1 -- 172.21.15.43:0/1035359062 learned_addr learned my addr 172.21.15.43:0/1035359062 (peer_addr_for_me v2:172.21.15.43:0/0) 2023-08-09T13:34:10.419 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:10.415+0000 7fd445d9b640 1 --2- 172.21.15.43:0/1035359062 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fd4500c3130 0x7fd4500cbc70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:34:10.419 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:10.415+0000 7fd44e59c640 1 -- 172.21.15.43:0/1035359062 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fd4500c3130 msgr2=0x7fd4500cbc70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-09T13:34:10.419 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:10.415+0000 7fd44e59c640 1 --2- 172.21.15.43:0/1035359062 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fd4500c3130 0x7fd4500cbc70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:34:10.419 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:10.415+0000 7fd44e59c640 1 -- 172.21.15.43:0/1035359062 >> v1:172.21.15.203:6789/0 conn(0x7fd4500cc1b0 legacy=0x7fd450164510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-09T13:34:10.419 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:10.415+0000 7fd44e59c640 1 -- 172.21.15.43:0/1035359062 --> v2:172.21.15.43:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd4500c3b80 con 0x7fd45014a5c0 2023-08-09T13:34:10.419 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:10.415+0000 7fd445d9b640 1 --2- 172.21.15.43:0/1035359062 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fd4500c3130 0x7fd4500cbc70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-08-09T13:34:10.419 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:10.416+0000 7fd44e59c640 1 --2- 172.21.15.43:0/1035359062 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fd45014a5c0 0x7fd450160f20 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7fd440009e50 tx=0x7fd44002be50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6fe4ad5f5cd3b720 server_cookie=0 in_seq=0 out_seq=0 2023-08-09T13:34:10.420 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:10.417+0000 7fd44dd9b640 1 -- 172.21.15.43:0/1035359062 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd44002c9f0 con 0x7fd45014a5c0 2023-08-09T13:34:10.421 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:10.417+0000 7fd44dd9b640 1 -- 172.21.15.43:0/1035359062 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd44002cb50 con 0x7fd45014a5c0 2023-08-09T13:34:10.421 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:10.417+0000 7fd44dd9b640 1 -- 172.21.15.43:0/1035359062 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd4400343f0 con 0x7fd45014a5c0 2023-08-09T13:34:10.421 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:10.417+0000 7fd4548c5640 1 -- 172.21.15.43:0/1035359062 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fd45014a5c0 msgr2=0x7fd450160f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:34:10.421 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:10.417+0000 7fd4548c5640 1 --2- 172.21.15.43:0/1035359062 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fd45014a5c0 0x7fd450160f20 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7fd440009e50 tx=0x7fd44002be50 comp rx=0 tx=0).stop 2023-08-09T13:34:10.421 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:10.417+0000 7fd4548c5640 1 -- 172.21.15.43:0/1035359062 shutdown_connections 2023-08-09T13:34:10.421 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:10.417+0000 7fd4548c5640 1 --2- 172.21.15.43:0/1035359062 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fd4500c3130 0x7fd4500cbc70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:34:10.422 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:10.417+0000 7fd4548c5640 1 --2- 172.21.15.43:0/1035359062 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fd45014a5c0 0x7fd450160f20 unknown :-1 s=CLOSED pgs=80 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:34:10.422 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:10.417+0000 7fd4548c5640 1 -- 172.21.15.43:0/1035359062 >> 172.21.15.43:0/1035359062 conn(0x7fd4500ce0a0 msgr2=0x7fd4500ce480 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-09T13:34:10.422 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:10.418+0000 7fd4548c5640 1 -- 172.21.15.43:0/1035359062 shutdown_connections 2023-08-09T13:34:10.422 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:10.418+0000 7fd4548c5640 1 -- 172.21.15.43:0/1035359062 wait complete. 2023-08-09T13:34:10.422 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:10.419+0000 7fd4548c5640 1 Processor -- start 2023-08-09T13:34:10.423 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:10.420+0000 7fd4548c5640 1 -- start start 2023-08-09T13:34:10.424 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:10.420+0000 7fd4548c5640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fd4500c3130 0x7fd4501f6f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:34:10.424 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:10.421+0000 7fd4548c5640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fd4500cc1b0 0x7fd450160790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:34:10.424 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:10.421+0000 7fd4548c5640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fd45014a5c0 0x7fd4500b2ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:34:10.424 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:10.421+0000 7fd4548c5640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd4501f7460 con 0x7fd4500cc1b0 2023-08-09T13:34:10.424 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:10.421+0000 7fd4548c5640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd4500b1230 con 0x7fd4500c3130 2023-08-09T13:34:10.424 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:10.421+0000 7fd4548c5640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd4500b2090 con 0x7fd45014a5c0 2023-08-09T13:34:10.425 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:10.421+0000 7fd445d9b640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fd4500cc1b0 0x7fd450160790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:34:10.425 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:10.421+0000 7fd445d9b640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fd4500cc1b0 0x7fd450160790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:42666/0 (socket says 172.21.15.43:42666) 2023-08-09T13:34:10.425 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:10.421+0000 7fd445d9b640 1 -- 172.21.15.43:0/255221430 learned_addr learned my addr 172.21.15.43:0/255221430 (peer_addr_for_me v2:172.21.15.43:0/0) 2023-08-09T13:34:10.425 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:10.421+0000 7fd44e59c640 1 --2- 172.21.15.43:0/255221430 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fd4500c3130 0x7fd4501f6f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:34:10.425 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:10.421+0000 7fd44ed9d640 1 --2- 172.21.15.43:0/255221430 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fd45014a5c0 0x7fd4500b2ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:34:10.425 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:10.421+0000 7fd445d9b640 1 -- 172.21.15.43:0/255221430 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fd45014a5c0 msgr2=0x7fd4500b2ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:34:10.425 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:10.421+0000 7fd445d9b640 1 --2- 172.21.15.43:0/255221430 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fd45014a5c0 0x7fd4500b2ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:34:10.426 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:10.421+0000 7fd445d9b640 1 -- 172.21.15.43:0/255221430 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fd4500c3130 msgr2=0x7fd4501f6f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:34:10.426 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:10.421+0000 7fd445d9b640 1 --2- 172.21.15.43:0/255221430 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fd4500c3130 0x7fd4501f6f20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:34:10.426 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:10.421+0000 7fd445d9b640 1 -- 172.21.15.43:0/255221430 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd440009b30 con 0x7fd4500cc1b0 2023-08-09T13:34:10.426 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:10.421+0000 7fd44ed9d640 1 --2- 172.21.15.43:0/255221430 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fd45014a5c0 0x7fd4500b2ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-08-09T13:34:10.426 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:10.422+0000 7fd445d9b640 1 --2- 172.21.15.43:0/255221430 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fd4500cc1b0 0x7fd450160790 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7fd43800c820 tx=0x7fd43800cce0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-09T13:34:10.427 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:10.422+0000 7fd4477fe640 1 -- 172.21.15.43:0/255221430 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd438007a30 con 0x7fd4500cc1b0 2023-08-09T13:34:10.427 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:10.422+0000 7fd4477fe640 1 -- 172.21.15.43:0/255221430 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd438007b90 con 0x7fd4500cc1b0 2023-08-09T13:34:10.427 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:10.422+0000 7fd4477fe640 1 -- 172.21.15.43:0/255221430 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd43800ce10 con 0x7fd4500cc1b0 2023-08-09T13:34:10.427 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:10.422+0000 7fd4548c5640 1 -- 172.21.15.43:0/255221430 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd4500b1480 con 0x7fd4500cc1b0 2023-08-09T13:34:10.428 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:10.422+0000 7fd4548c5640 1 -- 172.21.15.43:0/255221430 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd4500b1a40 con 0x7fd4500cc1b0 2023-08-09T13:34:10.428 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:10.423+0000 7fd4548c5640 1 -- 172.21.15.43:0/255221430 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd4500739f0 con 0x7fd4500cc1b0 2023-08-09T13:34:10.428 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:10.424+0000 7fd4477fe640 1 -- 172.21.15.43:0/255221430 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 5) v1 ==== 88341+0+0 (secure 0 0 0) 0x7fd4380113f0 con 0x7fd4500cc1b0 2023-08-09T13:34:10.428 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:10.425+0000 7fd4477fe640 1 --2- 172.21.15.43:0/255221430 >> [v2:172.21.15.203:6832/830097474,v1:172.21.15.203:6833/830097474] conn(0x7fd43006a050 0x7fd43006c510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:34:10.429 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:10.425+0000 7fd4477fe640 1 -- 172.21.15.43:0/255221430 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6813+0+0 (secure 0 0 0) 0x7fd4380889d0 con 0x7fd4500cc1b0 2023-08-09T13:34:10.429 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:10.426+0000 7fd44e59c640 1 --2- 172.21.15.43:0/255221430 >> [v2:172.21.15.203:6832/830097474,v1:172.21.15.203:6833/830097474] conn(0x7fd43006a050 0x7fd43006c510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:34:10.429 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:10.427+0000 7fd44e59c640 1 --2- 172.21.15.43:0/255221430 >> [v2:172.21.15.203:6832/830097474,v1:172.21.15.203:6833/830097474] conn(0x7fd43006a050 0x7fd43006c510 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7fd440035000 tx=0x7fd440030000 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-09T13:34:10.432 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:10.430+0000 7fd4477fe640 1 -- 172.21.15.43:0/255221430 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178066 (secure 0 0 0) 0x7fd438053d30 con 0x7fd4500cc1b0 2023-08-09T13:34:10.662 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:10.660+0000 7fd4548c5640 1 -- 172.21.15.43:0/255221430 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fd4500cc830 con 0x7fd4500cc1b0 2023-08-09T13:34:10.664 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:10.661+0000 7fd4477fe640 1 -- 172.21.15.43:0/255221430 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 7 v7) v1 ==== 93+0+4874 (secure 0 0 0) 0x7fd438057350 con 0x7fd4500cc1b0 2023-08-09T13:34:10.664 INFO:teuthology.orchestra.run.smithi043.stdout: 2023-08-09T13:34:10.664 INFO:teuthology.orchestra.run.smithi043.stdout:{"epoch":7,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4236,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.43:6833/3917539953","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6832","nonce":3917539953},{"type":"v1","addr":"172.21.15.43:6833","nonce":3917539953}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2},{"gid":4247,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.203:6837/2467794159","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6835","nonce":2467794159},{"type":"v1","addr":"172.21.15.203:6837","nonce":2467794159}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":4248,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.43:6835/1818380189","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6834","nonce":1818380189},{"type":"v1","addr":"172.21.15.43:6835","nonce":1818380189}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":7,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-08-09T13:34:02.575398+0000","modified":"2023-08-09T13:34:08.690983+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":4250},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4250":{"gid":4250,"name":"d","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.203:6836/798312463","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6834","nonce":798312463},{"type":"v1","addr":"172.21.15.203:6836","nonce":798312463}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[4,5],"metadata_pool":3,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":1},"id":1}]} 2023-08-09T13:34:10.665 INFO:teuthology.orchestra.run.smithi043.stderr:dumped fsmap epoch 7 2023-08-09T13:34:10.667 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:10.664+0000 7fd4548c5640 1 -- 172.21.15.43:0/255221430 >> [v2:172.21.15.203:6832/830097474,v1:172.21.15.203:6833/830097474] conn(0x7fd43006a050 msgr2=0x7fd43006c510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:34:10.667 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:10.664+0000 7fd4548c5640 1 --2- 172.21.15.43:0/255221430 >> [v2:172.21.15.203:6832/830097474,v1:172.21.15.203:6833/830097474] conn(0x7fd43006a050 0x7fd43006c510 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7fd440035000 tx=0x7fd440030000 comp rx=0 tx=0).stop 2023-08-09T13:34:10.667 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:10.664+0000 7fd4548c5640 1 -- 172.21.15.43:0/255221430 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fd4500cc1b0 msgr2=0x7fd450160790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:34:10.667 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:10.664+0000 7fd4548c5640 1 --2- 172.21.15.43:0/255221430 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fd4500cc1b0 0x7fd450160790 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7fd43800c820 tx=0x7fd43800cce0 comp rx=0 tx=0).stop 2023-08-09T13:34:10.667 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:10.664+0000 7fd4548c5640 1 -- 172.21.15.43:0/255221430 shutdown_connections 2023-08-09T13:34:10.667 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:10.664+0000 7fd4548c5640 1 --2- 172.21.15.43:0/255221430 >> [v2:172.21.15.203:6832/830097474,v1:172.21.15.203:6833/830097474] conn(0x7fd43006a050 0x7fd43006c510 unknown :-1 s=CLOSED pgs=69 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:34:10.667 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:10.664+0000 7fd4548c5640 1 --2- 172.21.15.43:0/255221430 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fd45014a5c0 0x7fd4500b2ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:34:10.668 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:10.664+0000 7fd4548c5640 1 --2- 172.21.15.43:0/255221430 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fd4500cc1b0 0x7fd450160790 unknown :-1 s=CLOSED pgs=81 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:34:10.668 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:10.664+0000 7fd4548c5640 1 --2- 172.21.15.43:0/255221430 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fd4500c3130 0x7fd4501f6f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:34:10.668 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:10.664+0000 7fd4548c5640 1 -- 172.21.15.43:0/255221430 >> 172.21.15.43:0/255221430 conn(0x7fd4500ce0a0 msgr2=0x7fd4500c98d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-09T13:34:10.668 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:10.664+0000 7fd4548c5640 1 -- 172.21.15.43:0/255221430 shutdown_connections 2023-08-09T13:34:10.668 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:10.664+0000 7fd4548c5640 1 -- 172.21.15.43:0/255221430 wait complete. 2023-08-09T13:34:10.685 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-08-09T13:34:10.914 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:10.911+0000 7f33bbcb8640 1 Processor -- start 2023-08-09T13:34:10.914 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:10.911+0000 7f33bbcb8640 1 -- start start 2023-08-09T13:34:10.915 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:10.913+0000 7f33bbcb8640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f33b4143c20 0x7f33b406f540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:34:10.916 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:10.913+0000 7f33bbcb8640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f33b40c86f0 0x7f33b40c8b50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:34:10.916 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:10.913+0000 7f33bbcb8640 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7f33b406c8d0 0x7f33b406cd60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:34:10.916 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:10.913+0000 7f33bbcb8640 1 -- --> v2:172.21.15.43:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f33b4144000 con 0x7f33b40c86f0 2023-08-09T13:34:10.916 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:10.913+0000 7f33bbcb8640 1 -- --> v2:172.21.15.203:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f33b40c9090 con 0x7f33b406c8d0 2023-08-09T13:34:10.916 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:10.913+0000 7f33bbcb8640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f33b406c440 con 0x7f33b4143c20 2023-08-09T13:34:10.916 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:10.913+0000 7f33b9253640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f33b40c86f0 0x7f33b40c8b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:34:10.917 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:10.913+0000 7f33b9253640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f33b40c86f0 0x7f33b40c8b50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:42668/0 (socket says 172.21.15.43:42668) 2023-08-09T13:34:10.917 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:10.913+0000 7f33b9253640 1 -- 172.21.15.43:0/3995746291 learned_addr learned my addr 172.21.15.43:0/3995746291 (peer_addr_for_me v2:172.21.15.43:0/0) 2023-08-09T13:34:10.917 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:10.914+0000 7f33b9a54640 1 --2- 172.21.15.43:0/3995746291 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f33b4143c20 0x7f33b406f540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:34:10.918 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:10.914+0000 7f33ba255640 1 --2- 172.21.15.43:0/3995746291 >> v2:172.21.15.203:3300/0 conn(0x7f33b406c8d0 0x7f33b406cd60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:34:10.918 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:10.914+0000 7f33b9253640 1 -- 172.21.15.43:0/3995746291 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f33b4143c20 msgr2=0x7f33b406f540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-09T13:34:10.918 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:10.914+0000 7f33b9253640 1 --2- 172.21.15.43:0/3995746291 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f33b4143c20 0x7f33b406f540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:34:10.918 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:10.914+0000 7f33b9253640 1 -- 172.21.15.43:0/3995746291 >> v2:172.21.15.203:3300/0 conn(0x7f33b406c8d0 msgr2=0x7f33b406cd60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-09T13:34:10.918 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:10.914+0000 7f33b9253640 1 --2- 172.21.15.43:0/3995746291 >> v2:172.21.15.203:3300/0 conn(0x7f33b406c8d0 0x7f33b406cd60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:34:10.918 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:10.914+0000 7f33b9253640 1 -- 172.21.15.43:0/3995746291 --> v2:172.21.15.43:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f33b406d2a0 con 0x7f33b40c86f0 2023-08-09T13:34:10.918 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:10.915+0000 7f33b9253640 1 --2- 172.21.15.43:0/3995746291 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f33b40c86f0 0x7f33b40c8b50 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f33b40c8680 tx=0x7f33a402b860 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c6fa7f9933b789d1 server_cookie=0 in_seq=0 out_seq=0 2023-08-09T13:34:10.919 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:10.915+0000 7f33b8a52640 1 -- 172.21.15.43:0/3995746291 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f33a40317c0 con 0x7f33b40c86f0 2023-08-09T13:34:10.919 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:10.915+0000 7f33b8a52640 1 -- 172.21.15.43:0/3995746291 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f33a4031920 con 0x7f33b40c86f0 2023-08-09T13:34:10.920 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:10.915+0000 7f33b8a52640 1 -- 172.21.15.43:0/3995746291 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f33a4031cb0 con 0x7f33b40c86f0 2023-08-09T13:34:10.920 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:10.916+0000 7f33bbcb8640 1 -- 172.21.15.43:0/3995746291 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f33b40c86f0 msgr2=0x7f33b40c8b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:34:10.920 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:10.916+0000 7f33bbcb8640 1 --2- 172.21.15.43:0/3995746291 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f33b40c86f0 0x7f33b40c8b50 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f33b40c8680 tx=0x7f33a402b860 comp rx=0 tx=0).stop 2023-08-09T13:34:10.920 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:10.916+0000 7f33bbcb8640 1 -- 172.21.15.43:0/3995746291 shutdown_connections 2023-08-09T13:34:10.920 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:10.916+0000 7f33bbcb8640 1 --2- 172.21.15.43:0/3995746291 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f33b40c86f0 0x7f33b40c8b50 unknown :-1 s=CLOSED pgs=82 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:34:10.920 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:10.916+0000 7f33bbcb8640 1 --2- 172.21.15.43:0/3995746291 >> v2:172.21.15.203:3300/0 conn(0x7f33b406c8d0 0x7f33b406cd60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:34:10.920 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:10.916+0000 7f33bbcb8640 1 --2- 172.21.15.43:0/3995746291 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f33b4143c20 0x7f33b406f540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:34:10.921 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:10.916+0000 7f33bbcb8640 1 -- 172.21.15.43:0/3995746291 >> 172.21.15.43:0/3995746291 conn(0x7f33b40ce0a0 msgr2=0x7f33b40ce480 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-09T13:34:10.921 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:10.916+0000 7f33bbcb8640 1 -- 172.21.15.43:0/3995746291 shutdown_connections 2023-08-09T13:34:10.921 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:10.916+0000 7f33bbcb8640 1 -- 172.21.15.43:0/3995746291 wait complete. 2023-08-09T13:34:10.922 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:10.918+0000 7f33bbcb8640 1 Processor -- start 2023-08-09T13:34:10.922 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:10.918+0000 7f33bbcb8640 1 -- start start 2023-08-09T13:34:10.922 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:10.919+0000 7f33bbcb8640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f33b406c8d0 0x7f33b413f100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:34:10.922 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:10.919+0000 7f33bbcb8640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f33b40c86f0 0x7f33b40c6f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:34:10.922 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:10.919+0000 7f33bbcb8640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f33b4143c20 0x7f33b40c74a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:34:10.922 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:10.919+0000 7f33bbcb8640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f33b40c6230 con 0x7f33b406c8d0 2023-08-09T13:34:10.923 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:10.919+0000 7f33bbcb8640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f33b415aeb0 con 0x7f33b40c86f0 2023-08-09T13:34:10.923 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:10.919+0000 7f33bbcb8640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f33b40c6480 con 0x7f33b4143c20 2023-08-09T13:34:10.924 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:10.920+0000 7f33b9a54640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f33b406c8d0 0x7f33b413f100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:34:10.924 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:10.920+0000 7f33b9a54640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f33b406c8d0 0x7f33b413f100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:42678/0 (socket says 172.21.15.43:42678) 2023-08-09T13:34:10.924 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:10.920+0000 7f33b9a54640 1 -- 172.21.15.43:0/615434818 learned_addr learned my addr 172.21.15.43:0/615434818 (peer_addr_for_me v2:172.21.15.43:0/0) 2023-08-09T13:34:10.924 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:10.920+0000 7f33ba255640 1 --2- 172.21.15.43:0/615434818 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f33b4143c20 0x7f33b40c74a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:34:10.924 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:10.920+0000 7f33b9253640 1 --2- 172.21.15.43:0/615434818 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f33b40c86f0 0x7f33b40c6f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:34:10.925 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:10.920+0000 7f33b9a54640 1 -- 172.21.15.43:0/615434818 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f33b4143c20 msgr2=0x7f33b40c74a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:34:10.925 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:10.920+0000 7f33b9a54640 1 --2- 172.21.15.43:0/615434818 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f33b4143c20 0x7f33b40c74a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:34:10.925 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:10.920+0000 7f33b9a54640 1 -- 172.21.15.43:0/615434818 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f33b40c86f0 msgr2=0x7f33b40c6f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:34:10.925 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:10.920+0000 7f33b9a54640 1 --2- 172.21.15.43:0/615434818 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f33b40c86f0 0x7f33b40c6f60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:34:10.925 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:10.920+0000 7f33b9a54640 1 -- 172.21.15.43:0/615434818 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f33a40094d0 con 0x7f33b406c8d0 2023-08-09T13:34:10.925 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:10.920+0000 7f33b9a54640 1 --2- 172.21.15.43:0/615434818 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f33b406c8d0 0x7f33b413f100 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f33b000d800 tx=0x7f33b000dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-09T13:34:10.925 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:10.921+0000 7f33aa7fc640 1 -- 172.21.15.43:0/615434818 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f33b000b6c0 con 0x7f33b406c8d0 2023-08-09T13:34:10.926 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:10.921+0000 7f33aa7fc640 1 -- 172.21.15.43:0/615434818 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f33b000b820 con 0x7f33b406c8d0 2023-08-09T13:34:10.926 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:10.921+0000 7f33aa7fc640 1 -- 172.21.15.43:0/615434818 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f33b000bbb0 con 0x7f33b406c8d0 2023-08-09T13:34:10.926 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:10.921+0000 7f33bbcb8640 1 -- 172.21.15.43:0/615434818 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f33b40c6760 con 0x7f33b406c8d0 2023-08-09T13:34:10.926 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:10.921+0000 7f33bbcb8640 1 -- 172.21.15.43:0/615434818 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f33b413f640 con 0x7f33b406c8d0 2023-08-09T13:34:10.927 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:10.922+0000 7f33bbcb8640 1 -- 172.21.15.43:0/615434818 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f33b40739f0 con 0x7f33b406c8d0 2023-08-09T13:34:10.927 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:10.923+0000 7f33aa7fc640 1 -- 172.21.15.43:0/615434818 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 5) v1 ==== 88341+0+0 (secure 0 0 0) 0x7f33b000bd10 con 0x7f33b406c8d0 2023-08-09T13:34:10.927 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:10.924+0000 7f33aa7fc640 1 --2- 172.21.15.43:0/615434818 >> [v2:172.21.15.203:6832/830097474,v1:172.21.15.203:6833/830097474] conn(0x7f339806a050 0x7f339806c510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:34:10.927 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:10.924+0000 7f33aa7fc640 1 -- 172.21.15.43:0/615434818 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6813+0+0 (secure 0 0 0) 0x7f33b0088dc0 con 0x7f33b406c8d0 2023-08-09T13:34:10.927 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:10.925+0000 7f33b9253640 1 --2- 172.21.15.43:0/615434818 >> [v2:172.21.15.203:6832/830097474,v1:172.21.15.203:6833/830097474] conn(0x7f339806a050 0x7f339806c510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:34:10.928 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:10.926+0000 7f33b9253640 1 --2- 172.21.15.43:0/615434818 >> [v2:172.21.15.203:6832/830097474,v1:172.21.15.203:6833/830097474] conn(0x7f339806a050 0x7f339806c510 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f33a4036000 tx=0x7f33a403d040 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-09T13:34:10.931 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:10.928+0000 7f33aa7fc640 1 -- 172.21.15.43:0/615434818 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178066 (secure 0 0 0) 0x7f33b00541b0 con 0x7f33b406c8d0 2023-08-09T13:34:11.127 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:11.124+0000 7f33bbcb8640 1 -- 172.21.15.43:0/615434818 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f33b406c0c0 con 0x7f33b406c8d0 2023-08-09T13:34:11.128 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:11.126+0000 7f33aa7fc640 1 -- 172.21.15.43:0/615434818 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v24) v1 ==== 74+0+20050 (secure 0 0 0) 0x7f33b00577d0 con 0x7f33b406c8d0 2023-08-09T13:34:11.129 INFO:teuthology.orchestra.run.smithi043.stdout: 2023-08-09T13:34:11.129 INFO:teuthology.orchestra.run.smithi043.stdout:{"epoch":24,"fsid":"daac87ee-1337-4250-a955-aea4e408a39b","created":"2023-08-09T13:33:41.701198+0000","modified":"2023-08-09T13:34:07.709429+0000","last_up_change":"2023-08-09T13:33:53.783453+0000","last_in_change":"2023-08-09T13:33:45.148904+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":5,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":"rbd","create_time":"2023-08-09T13:33:54.365137+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":".mgr","create_time":"2023-08-09T13:33:54.831803+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_metadata","create_time":"2023-08-09T13:34:00.207974+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data","create_time":"2023-08-09T13:34:01.614765+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":5,"pool_name":"cephfs_data_ec","create_time":"2023-08-09T13:34:05.239583+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"24","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"cbd49ecd-e896-4fbd-864e-5c37a829b165","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6800","nonce":502667335},{"type":"v1","addr":"172.21.15.43:6801","nonce":502667335}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6802","nonce":502667335},{"type":"v1","addr":"172.21.15.43:6803","nonce":502667335}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6806","nonce":502667335},{"type":"v1","addr":"172.21.15.43:6807","nonce":502667335}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6804","nonce":502667335},{"type":"v1","addr":"172.21.15.43:6805","nonce":502667335}]},"public_addr":"172.21.15.43:6801/502667335","cluster_addr":"172.21.15.43:6803/502667335","heartbeat_back_addr":"172.21.15.43:6807/502667335","heartbeat_front_addr":"172.21.15.43:6805/502667335","state":["exists","up"]},{"osd":1,"uuid":"75c7e0fe-1ec7-4fe8-a2f2-3661a65b8500","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6808","nonce":3289134610},{"type":"v1","addr":"172.21.15.43:6809","nonce":3289134610}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6810","nonce":3289134610},{"type":"v1","addr":"172.21.15.43:6812","nonce":3289134610}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6821","nonce":3289134610},{"type":"v1","addr":"172.21.15.43:6824","nonce":3289134610}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6815","nonce":3289134610},{"type":"v1","addr":"172.21.15.43:6818","nonce":3289134610}]},"public_addr":"172.21.15.43:6809/3289134610","cluster_addr":"172.21.15.43:6812/3289134610","heartbeat_back_addr":"172.21.15.43:6824/3289134610","heartbeat_front_addr":"172.21.15.43:6818/3289134610","state":["exists","up"]},{"osd":2,"uuid":"bb5c41e9-7f5f-4125-9b01-3bb8bfc2ea83","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6811","nonce":4079709736},{"type":"v1","addr":"172.21.15.43:6814","nonce":4079709736}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6817","nonce":4079709736},{"type":"v1","addr":"172.21.15.43:6820","nonce":4079709736}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6829","nonce":4079709736},{"type":"v1","addr":"172.21.15.43:6831","nonce":4079709736}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6823","nonce":4079709736},{"type":"v1","addr":"172.21.15.43:6826","nonce":4079709736}]},"public_addr":"172.21.15.43:6814/4079709736","cluster_addr":"172.21.15.43:6820/4079709736","heartbeat_back_addr":"172.21.15.43:6831/4079709736","heartbeat_front_addr":"172.21.15.43:6826/4079709736","state":["exists","up"]},{"osd":3,"uuid":"bfe4c8b4-694a-465c-87e6-2b2aab852613","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6813","nonce":1709538759},{"type":"v1","addr":"172.21.15.43:6816","nonce":1709538759}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6819","nonce":1709538759},{"type":"v1","addr":"172.21.15.43:6822","nonce":1709538759}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6828","nonce":1709538759},{"type":"v1","addr":"172.21.15.43:6830","nonce":1709538759}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6825","nonce":1709538759},{"type":"v1","addr":"172.21.15.43:6827","nonce":1709538759}]},"public_addr":"172.21.15.43:6816/1709538759","cluster_addr":"172.21.15.43:6822/1709538759","heartbeat_back_addr":"172.21.15.43:6830/1709538759","heartbeat_front_addr":"172.21.15.43:6827/1709538759","state":["exists","up"]},{"osd":4,"uuid":"fa2e8a77-aba1-4756-853e-0d174a5bdeba","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6800","nonce":4196866481},{"type":"v1","addr":"172.21.15.203:6801","nonce":4196866481}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6802","nonce":4196866481},{"type":"v1","addr":"172.21.15.203:6803","nonce":4196866481}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6806","nonce":4196866481},{"type":"v1","addr":"172.21.15.203:6807","nonce":4196866481}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6804","nonce":4196866481},{"type":"v1","addr":"172.21.15.203:6805","nonce":4196866481}]},"public_addr":"172.21.15.203:6801/4196866481","cluster_addr":"172.21.15.203:6803/4196866481","heartbeat_back_addr":"172.21.15.203:6807/4196866481","heartbeat_front_addr":"172.21.15.203:6805/4196866481","state":["exists","up"]},{"osd":5,"uuid":"81811e4b-4d24-4f51-a82c-f1d700a2a00d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6816","nonce":3059290081},{"type":"v1","addr":"172.21.15.203:6817","nonce":3059290081}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6818","nonce":3059290081},{"type":"v1","addr":"172.21.15.203:6819","nonce":3059290081}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6822","nonce":3059290081},{"type":"v1","addr":"172.21.15.203:6823","nonce":3059290081}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6820","nonce":3059290081},{"type":"v1","addr":"172.21.15.203:6821","nonce":3059290081}]},"public_addr":"172.21.15.203:6817/3059290081","cluster_addr":"172.21.15.203:6819/3059290081","heartbeat_back_addr":"172.21.15.203:6823/3059290081","heartbeat_front_addr":"172.21.15.203:6821/3059290081","state":["exists","up"]},{"osd":6,"uuid":"307bac4d-6a5b-443d-af75-56d695d7674f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6824","nonce":512155569},{"type":"v1","addr":"172.21.15.203:6825","nonce":512155569}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6826","nonce":512155569},{"type":"v1","addr":"172.21.15.203:6827","nonce":512155569}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6830","nonce":512155569},{"type":"v1","addr":"172.21.15.203:6831","nonce":512155569}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6828","nonce":512155569},{"type":"v1","addr":"172.21.15.203:6829","nonce":512155569}]},"public_addr":"172.21.15.203:6825/512155569","cluster_addr":"172.21.15.203:6827/512155569","heartbeat_back_addr":"172.21.15.203:6831/512155569","heartbeat_front_addr":"172.21.15.203:6829/512155569","state":["exists","up"]},{"osd":7,"uuid":"a95dc7ca-9f1b-4295-af70-68a22a2d7d5e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6808","nonce":3331450438},{"type":"v1","addr":"172.21.15.203:6809","nonce":3331450438}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6810","nonce":3331450438},{"type":"v1","addr":"172.21.15.203:6811","nonce":3331450438}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6814","nonce":3331450438},{"type":"v1","addr":"172.21.15.203:6815","nonce":3331450438}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6812","nonce":3331450438},{"type":"v1","addr":"172.21.15.203:6813","nonce":3331450438}]},"public_addr":"172.21.15.203:6809/3331450438","cluster_addr":"172.21.15.203:6811/3331450438","heartbeat_back_addr":"172.21.15.203:6815/3331450438","heartbeat_front_addr":"172.21.15.203:6813/3331450438","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-09T13:33:50.547652+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-09T13:33:50.589469+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-09T13:33:50.561724+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-09T13:33:50.599650+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-09T13:33:50.411855+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-09T13:33:50.219681+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-09T13:33:50.504664+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-09T13:33:50.441248+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-08-09T13:34:11.131 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:11.129+0000 7f33bbcb8640 1 -- 172.21.15.43:0/615434818 >> [v2:172.21.15.203:6832/830097474,v1:172.21.15.203:6833/830097474] conn(0x7f339806a050 msgr2=0x7f339806c510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:34:11.132 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:11.129+0000 7f33bbcb8640 1 --2- 172.21.15.43:0/615434818 >> [v2:172.21.15.203:6832/830097474,v1:172.21.15.203:6833/830097474] conn(0x7f339806a050 0x7f339806c510 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f33a4036000 tx=0x7f33a403d040 comp rx=0 tx=0).stop 2023-08-09T13:34:11.132 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:11.129+0000 7f33bbcb8640 1 -- 172.21.15.43:0/615434818 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f33b406c8d0 msgr2=0x7f33b413f100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:34:11.132 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:11.129+0000 7f33bbcb8640 1 --2- 172.21.15.43:0/615434818 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f33b406c8d0 0x7f33b413f100 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f33b000d800 tx=0x7f33b000dcc0 comp rx=0 tx=0).stop 2023-08-09T13:34:11.132 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:11.129+0000 7f33bbcb8640 1 -- 172.21.15.43:0/615434818 shutdown_connections 2023-08-09T13:34:11.132 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:11.129+0000 7f33bbcb8640 1 --2- 172.21.15.43:0/615434818 >> [v2:172.21.15.203:6832/830097474,v1:172.21.15.203:6833/830097474] conn(0x7f339806a050 0x7f339806c510 unknown :-1 s=CLOSED pgs=70 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:34:11.132 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:11.129+0000 7f33bbcb8640 1 --2- 172.21.15.43:0/615434818 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f33b4143c20 0x7f33b40c74a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:34:11.133 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:11.129+0000 7f33bbcb8640 1 --2- 172.21.15.43:0/615434818 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f33b40c86f0 0x7f33b40c6f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:34:11.133 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:11.129+0000 7f33bbcb8640 1 --2- 172.21.15.43:0/615434818 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f33b406c8d0 0x7f33b413f100 unknown :-1 s=CLOSED pgs=83 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:34:11.133 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:11.129+0000 7f33bbcb8640 1 -- 172.21.15.43:0/615434818 >> 172.21.15.43:0/615434818 conn(0x7f33b40ce0a0 msgr2=0x7f33b40c9220 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-09T13:34:11.133 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:11.129+0000 7f33bbcb8640 1 -- 172.21.15.43:0/615434818 shutdown_connections 2023-08-09T13:34:11.133 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:11.129+0000 7f33bbcb8640 1 -- 172.21.15.43:0/615434818 wait complete. 2023-08-09T13:34:11.554 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 7, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2023-08-09T13:34:02.575398+0000', 'modified': '2023-08-09T13:34:08.690983+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 4250}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_4250': {'gid': 4250, 'name': 'd', 'rank': 0, 'incarnation': 4, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.203:6836/798312463', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.203:6834', 'nonce': 798312463}, {'type': 'v1', 'addr': '172.21.15.203:6836', 'nonce': 798312463}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138322906710015, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}}, 'data_pools': [4, 5], 'metadata_pool': 3, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 1} 2023-08-09T13:34:11.555 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2023-08-09T13:34:11.555 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.d status 2023-08-09T13:34:11.796 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:11.793+0000 7f65d8413640 1 Processor -- start 2023-08-09T13:34:11.796 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:11.793+0000 7f65d8413640 1 -- start start 2023-08-09T13:34:11.797 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:11.795+0000 7f65d8413640 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7f65d0143c20 0x7f65d006f540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:34:11.798 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:11.795+0000 7f65d8413640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f65d00c86f0 0x7f65d00c8b50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:34:11.798 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:11.795+0000 7f65d8413640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f65d006c8d0 0x7f65d006cd40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:34:11.798 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:11.795+0000 7f65d8413640 1 -- --> v2:172.21.15.43:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f65d0144000 con 0x7f65d00c86f0 2023-08-09T13:34:11.798 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:11.795+0000 7f65d8413640 1 -- --> v2:172.21.15.203:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f65d00c9090 con 0x7f65d0143c20 2023-08-09T13:34:11.798 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:11.795+0000 7f65d8413640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f65d006c440 con 0x7f65d006c8d0 2023-08-09T13:34:11.799 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:11.795+0000 7f65d59ae640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f65d00c86f0 0x7f65d00c8b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:34:11.799 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:11.796+0000 7f65d59ae640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f65d00c86f0 0x7f65d00c8b50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:42686/0 (socket says 172.21.15.43:42686) 2023-08-09T13:34:11.799 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:11.796+0000 7f65d59ae640 1 -- 172.21.15.43:0/2178197504 learned_addr learned my addr 172.21.15.43:0/2178197504 (peer_addr_for_me v2:172.21.15.43:0/0) 2023-08-09T13:34:11.800 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:11.796+0000 7f65d61af640 1 --2- 172.21.15.43:0/2178197504 >> v2:172.21.15.203:3300/0 conn(0x7f65d0143c20 0x7f65d006f540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:34:11.800 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:11.796+0000 7f65d69b0640 1 --2- 172.21.15.43:0/2178197504 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f65d006c8d0 0x7f65d006cd40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:34:11.800 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:11.796+0000 7f65d59ae640 1 -- 172.21.15.43:0/2178197504 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f65d006c8d0 msgr2=0x7f65d006cd40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-09T13:34:11.800 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:11.796+0000 7f65d59ae640 1 --2- 172.21.15.43:0/2178197504 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f65d006c8d0 0x7f65d006cd40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:34:11.800 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:11.796+0000 7f65d59ae640 1 -- 172.21.15.43:0/2178197504 >> v2:172.21.15.203:3300/0 conn(0x7f65d0143c20 msgr2=0x7f65d006f540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-09T13:34:11.800 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:11.796+0000 7f65d59ae640 1 --2- 172.21.15.43:0/2178197504 >> v2:172.21.15.203:3300/0 conn(0x7f65d0143c20 0x7f65d006f540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:34:11.800 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:11.796+0000 7f65d59ae640 1 -- 172.21.15.43:0/2178197504 --> v2:172.21.15.43:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f65d006d280 con 0x7f65d00c86f0 2023-08-09T13:34:11.801 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:11.796+0000 7f65d69b0640 1 --2- 172.21.15.43:0/2178197504 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f65d006c8d0 0x7f65d006cd40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-08-09T13:34:11.801 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:11.797+0000 7f65d59ae640 1 --2- 172.21.15.43:0/2178197504 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f65d00c86f0 0x7f65d00c8b50 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f65d006e0f0 tx=0x7f65c002b860 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a024ca8b9e4aa048 server_cookie=0 in_seq=0 out_seq=0 2023-08-09T13:34:11.802 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:11.797+0000 7f65d51ad640 1 -- 172.21.15.43:0/2178197504 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f65c00317c0 con 0x7f65d00c86f0 2023-08-09T13:34:11.802 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:11.797+0000 7f65d51ad640 1 -- 172.21.15.43:0/2178197504 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f65c0031920 con 0x7f65d00c86f0 2023-08-09T13:34:11.802 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:11.797+0000 7f65d51ad640 1 -- 172.21.15.43:0/2178197504 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f65c0031cb0 con 0x7f65d00c86f0 2023-08-09T13:34:11.802 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:11.798+0000 7f65d8413640 1 -- 172.21.15.43:0/2178197504 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f65d00c86f0 msgr2=0x7f65d00c8b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:34:11.802 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:11.798+0000 7f65d8413640 1 --2- 172.21.15.43:0/2178197504 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f65d00c86f0 0x7f65d00c8b50 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f65d006e0f0 tx=0x7f65c002b860 comp rx=0 tx=0).stop 2023-08-09T13:34:11.802 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:11.798+0000 7f65d8413640 1 -- 172.21.15.43:0/2178197504 shutdown_connections 2023-08-09T13:34:11.802 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:11.798+0000 7f65d8413640 1 --2- 172.21.15.43:0/2178197504 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f65d00c86f0 0x7f65d00c8b50 unknown :-1 s=CLOSED pgs=84 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:34:11.803 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:11.798+0000 7f65d8413640 1 --2- 172.21.15.43:0/2178197504 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f65d006c8d0 0x7f65d006cd40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:34:11.803 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:11.798+0000 7f65d8413640 1 --2- 172.21.15.43:0/2178197504 >> v2:172.21.15.203:3300/0 conn(0x7f65d0143c20 0x7f65d006f540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:34:11.803 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:11.798+0000 7f65d8413640 1 -- 172.21.15.43:0/2178197504 >> 172.21.15.43:0/2178197504 conn(0x7f65d00ce0a0 msgr2=0x7f65d00ce480 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-09T13:34:11.803 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:11.798+0000 7f65d8413640 1 -- 172.21.15.43:0/2178197504 shutdown_connections 2023-08-09T13:34:11.803 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:11.798+0000 7f65d8413640 1 -- 172.21.15.43:0/2178197504 wait complete. 2023-08-09T13:34:11.804 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:11.800+0000 7f65d8413640 1 Processor -- start 2023-08-09T13:34:11.805 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:11.800+0000 7f65d8413640 1 -- start start 2023-08-09T13:34:11.805 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:11.801+0000 7f65d8413640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f65d006c8d0 0x7f65d00c8060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:34:11.805 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:11.801+0000 7f65d8413640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f65d00c86f0 0x7f65d0141a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:34:11.805 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:11.801+0000 7f65d8413640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f65d0143c20 0x7f65d00c3fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:34:11.805 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:11.801+0000 7f65d8413640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f65d00c59f0 con 0x7f65d0143c20 2023-08-09T13:34:11.805 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:11.801+0000 7f65d8413640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f65d00b3cd0 con 0x7f65d00c86f0 2023-08-09T13:34:11.806 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:11.801+0000 7f65d8413640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f65d0142a80 con 0x7f65d006c8d0 2023-08-09T13:34:11.806 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:11.801+0000 7f65d69b0640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f65d0143c20 0x7f65d00c3fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:34:11.806 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:11.801+0000 7f65d69b0640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f65d0143c20 0x7f65d00c3fb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:42688/0 (socket says 172.21.15.43:42688) 2023-08-09T13:34:11.806 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:11.802+0000 7f65d69b0640 1 -- 172.21.15.43:0/2356950988 learned_addr learned my addr 172.21.15.43:0/2356950988 (peer_addr_for_me v2:172.21.15.43:0/0) 2023-08-09T13:34:11.806 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:11.802+0000 7f65d59ae640 1 --2- 172.21.15.43:0/2356950988 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f65d00c86f0 0x7f65d0141a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:34:11.806 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:11.802+0000 7f65d61af640 1 --2- 172.21.15.43:0/2356950988 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f65d006c8d0 0x7f65d00c8060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:34:11.806 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:11.802+0000 7f65d69b0640 1 -- 172.21.15.43:0/2356950988 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f65d006c8d0 msgr2=0x7f65d00c8060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:34:11.806 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:11.802+0000 7f65d69b0640 1 --2- 172.21.15.43:0/2356950988 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f65d006c8d0 0x7f65d00c8060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:34:11.807 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:11.802+0000 7f65d69b0640 1 -- 172.21.15.43:0/2356950988 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f65d00c86f0 msgr2=0x7f65d0141a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:34:11.807 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:11.802+0000 7f65d69b0640 1 --2- 172.21.15.43:0/2356950988 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f65d00c86f0 0x7f65d0141a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:34:11.807 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:11.802+0000 7f65d69b0640 1 -- 172.21.15.43:0/2356950988 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f65c00094d0 con 0x7f65d0143c20 2023-08-09T13:34:11.807 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:11.802+0000 7f65d61af640 1 --2- 172.21.15.43:0/2356950988 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f65d006c8d0 0x7f65d00c8060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-08-09T13:34:11.807 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:11.802+0000 7f65d69b0640 1 --2- 172.21.15.43:0/2356950988 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f65d0143c20 0x7f65d00c3fb0 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f65c800cd10 tx=0x7f65c800a650 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-09T13:34:11.808 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:11.803+0000 7f65c6ffd640 1 -- 172.21.15.43:0/2356950988 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f65c800f680 con 0x7f65d0143c20 2023-08-09T13:34:11.808 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:11.803+0000 7f65d8413640 1 -- 172.21.15.43:0/2356950988 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f65d00b39f0 con 0x7f65d0143c20 2023-08-09T13:34:11.808 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:11.803+0000 7f65c6ffd640 1 -- 172.21.15.43:0/2356950988 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f65c800f800 con 0x7f65d0143c20 2023-08-09T13:34:11.808 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:11.803+0000 7f65c6ffd640 1 -- 172.21.15.43:0/2356950988 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f65c800fbb0 con 0x7f65d0143c20 2023-08-09T13:34:11.808 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:11.803+0000 7f65d8413640 1 -- 172.21.15.43:0/2356950988 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f65d00c54f0 con 0x7f65d0143c20 2023-08-09T13:34:11.809 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:11.806+0000 7f65c6ffd640 1 -- 172.21.15.43:0/2356950988 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 5) v1 ==== 88341+0+0 (secure 0 0 0) 0x7f65c800b040 con 0x7f65d0143c20 2023-08-09T13:34:11.810 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:11.807+0000 7f65c6ffd640 1 --2- 172.21.15.43:0/2356950988 >> [v2:172.21.15.203:6832/830097474,v1:172.21.15.203:6833/830097474] conn(0x7f65b406a050 0x7f65b406c510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:34:11.810 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:11.808+0000 7f65c6ffd640 1 -- 172.21.15.43:0/2356950988 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6813+0+0 (secure 0 0 0) 0x7f65c8089240 con 0x7f65d0143c20 2023-08-09T13:34:11.811 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:11.808+0000 7f65daf7f740 1 Processor -- start 2023-08-09T13:34:11.811 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:11.808+0000 7f65d61af640 1 --2- 172.21.15.43:0/2356950988 >> [v2:172.21.15.203:6832/830097474,v1:172.21.15.203:6833/830097474] conn(0x7f65b406a050 0x7f65b406c510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:34:11.811 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:11.808+0000 7f65daf7f740 1 -- start start 2023-08-09T13:34:11.812 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:11.809+0000 7f65daf7f740 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x558b7b465c50 0x558b7b466030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:34:11.813 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:11.809+0000 7f65daf7f740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x558b7b435bd0 0x558b7b462ec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:34:11.813 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:11.809+0000 7f65daf7f740 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x558b7b4397d0 0x558b7b463400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:34:11.813 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:11.809+0000 7f65daf7f740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x558b7b3a78f0 con 0x558b7b435bd0 2023-08-09T13:34:11.813 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:11.809+0000 7f65daf7f740 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x558b7b3a4fe0 con 0x558b7b4397d0 2023-08-09T13:34:11.813 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:11.809+0000 7f65daf7f740 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x558b7b3a08d0 con 0x558b7b465c50 2023-08-09T13:34:11.813 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:11.809+0000 7f65d59ae640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x558b7b435bd0 0x558b7b462ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:34:11.814 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:11.809+0000 7f65d59ae640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x558b7b435bd0 0x558b7b462ec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:42692/0 (socket says 172.21.15.43:42692) 2023-08-09T13:34:11.814 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:11.809+0000 7f65d59ae640 1 -- 172.21.15.43:0/375854198 learned_addr learned my addr 172.21.15.43:0/375854198 (peer_addr_for_me v2:172.21.15.43:0/0) 2023-08-09T13:34:11.814 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:11.809+0000 7f65d61af640 1 --2- 172.21.15.43:0/2356950988 >> [v2:172.21.15.203:6832/830097474,v1:172.21.15.203:6833/830097474] conn(0x7f65b406a050 0x7f65b406c510 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f65cc006fd0 tx=0x7f65cc008040 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-09T13:34:11.814 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:11.809+0000 7f65d61af640 1 --2- 172.21.15.43:0/375854198 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x558b7b465c50 0x558b7b466030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:34:11.814 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:11.810+0000 7f65d69b0640 1 --2- 172.21.15.43:0/375854198 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x558b7b4397d0 0x558b7b463400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:34:11.814 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:11.810+0000 7f65d59ae640 1 -- 172.21.15.43:0/375854198 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x558b7b465c50 msgr2=0x558b7b466030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-09T13:34:11.814 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:11.810+0000 7f65d59ae640 1 --2- 172.21.15.43:0/375854198 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x558b7b465c50 0x558b7b466030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:34:11.814 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:11.810+0000 7f65d59ae640 1 -- 172.21.15.43:0/375854198 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x558b7b4397d0 msgr2=0x558b7b463400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-09T13:34:11.815 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:11.810+0000 7f65d59ae640 1 --2- 172.21.15.43:0/375854198 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x558b7b4397d0 0x558b7b463400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:34:11.815 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:11.810+0000 7f65d59ae640 1 -- 172.21.15.43:0/375854198 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x558b7b359ac0 con 0x558b7b435bd0 2023-08-09T13:34:11.815 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:11.810+0000 7f65d59ae640 1 --2- 172.21.15.43:0/375854198 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x558b7b435bd0 0x558b7b462ec0 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f65c002b8a0 tx=0x7f65c0008030 comp rx=0 tx=0).ready entity=mon.0 client_cookie=83b4cc4896ce7919 server_cookie=0 in_seq=0 out_seq=0 2023-08-09T13:34:11.815 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:11.810+0000 7f65b3fff640 1 -- 172.21.15.43:0/375854198 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f65c0031c90 con 0x558b7b435bd0 2023-08-09T13:34:11.815 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:11.810+0000 7f65b3fff640 1 -- 172.21.15.43:0/375854198 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f65c0031df0 con 0x558b7b435bd0 2023-08-09T13:34:11.815 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:11.810+0000 7f65b3fff640 1 -- 172.21.15.43:0/375854198 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f65c0032690 con 0x558b7b435bd0 2023-08-09T13:34:11.817 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:11.811+0000 7f65daf7f740 1 -- 172.21.15.43:0/375854198 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x558b7b435bd0 msgr2=0x558b7b462ec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:34:11.818 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:11.811+0000 7f65daf7f740 1 --2- 172.21.15.43:0/375854198 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x558b7b435bd0 0x558b7b462ec0 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f65c002b8a0 tx=0x7f65c0008030 comp rx=0 tx=0).stop 2023-08-09T13:34:11.818 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:11.811+0000 7f65daf7f740 1 -- 172.21.15.43:0/375854198 shutdown_connections 2023-08-09T13:34:11.818 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:11.811+0000 7f65daf7f740 1 --2- 172.21.15.43:0/375854198 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x558b7b4397d0 0x558b7b463400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:34:11.818 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:11.811+0000 7f65daf7f740 1 --2- 172.21.15.43:0/375854198 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x558b7b435bd0 0x558b7b462ec0 unknown :-1 s=CLOSED pgs=86 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:34:11.818 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:11.811+0000 7f65daf7f740 1 --2- 172.21.15.43:0/375854198 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x558b7b465c50 0x558b7b466030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:34:11.818 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:11.811+0000 7f65daf7f740 1 -- 172.21.15.43:0/375854198 >> 172.21.15.43:0/375854198 conn(0x558b7b437ee0 msgr2=0x558b7b480a60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-09T13:34:11.819 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:11.811+0000 7f65daf7f740 1 -- 172.21.15.43:0/375854198 shutdown_connections 2023-08-09T13:34:11.819 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:11.811+0000 7f65daf7f740 1 -- 172.21.15.43:0/375854198 wait complete. 2023-08-09T13:34:11.819 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:11.812+0000 7f65daf7f740 1 -- start start 2023-08-09T13:34:11.819 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:11.812+0000 7f65daf7f740 1 Processor -- start 2023-08-09T13:34:11.819 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:11.813+0000 7f65daf7f740 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x558b7b4397d0 0x558b7b456f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:34:11.819 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:11.813+0000 7f65daf7f740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x558b7b465c50 0x558b7b468620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:34:11.819 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:11.813+0000 7f65daf7f740 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x558b7b47a540 0x558b7b47a9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:34:11.820 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:11.813+0000 7f65daf7f740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x558b7b340d40 con 0x558b7b465c50 2023-08-09T13:34:11.820 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:11.813+0000 7f65daf7f740 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x558b7b3409e0 con 0x558b7b4397d0 2023-08-09T13:34:11.820 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:11.813+0000 7f65daf7f740 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x558b7b106f20 con 0x558b7b47a540 2023-08-09T13:34:11.820 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:11.813+0000 7f65daf7f740 10 client.0 _ll_register_callbacks cb 0x558b7b359160 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-08-09T13:34:11.820 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:11.814+0000 7f65d59ae640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x558b7b465c50 0x558b7b468620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:34:11.820 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:11.814+0000 7f65d59ae640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x558b7b465c50 0x558b7b468620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:42708/0 (socket says 172.21.15.43:42708) 2023-08-09T13:34:11.820 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:11.814+0000 7f65d59ae640 1 -- 172.21.15.43:0/3013830630 learned_addr learned my addr 172.21.15.43:0/3013830630 (peer_addr_for_me v2:172.21.15.43:0/0) 2023-08-09T13:34:11.820 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:11.814+0000 7f65d61af640 1 --2- 172.21.15.43:0/3013830630 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x558b7b4397d0 0x558b7b456f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:34:11.821 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:11.814+0000 7f65d69b0640 1 --2- 172.21.15.43:0/3013830630 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x558b7b47a540 0x558b7b47a9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:34:11.821 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:11.814+0000 7f65d59ae640 1 -- 172.21.15.43:0/3013830630 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x558b7b47a540 msgr2=0x558b7b47a9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-09T13:34:11.821 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:11.814+0000 7f65d59ae640 1 --2- 172.21.15.43:0/3013830630 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x558b7b47a540 0x558b7b47a9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:34:11.821 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:11.814+0000 7f65d59ae640 1 -- 172.21.15.43:0/3013830630 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x558b7b4397d0 msgr2=0x558b7b456f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-09T13:34:11.821 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:11.814+0000 7f65d59ae640 1 --2- 172.21.15.43:0/3013830630 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x558b7b4397d0 0x558b7b456f90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:34:11.821 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:11.814+0000 7f65d59ae640 1 -- 172.21.15.43:0/3013830630 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f65d006d280 con 0x558b7b465c50 2023-08-09T13:34:11.821 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:11.814+0000 7f65d61af640 1 --2- 172.21.15.43:0/3013830630 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x558b7b4397d0 0x558b7b456f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-08-09T13:34:11.822 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:11.814+0000 7f65d59ae640 1 --2- 172.21.15.43:0/3013830630 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x558b7b465c50 0x558b7b468620 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f65c0047040 tx=0x7f65c0004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d1d6eda1a3bdbcea server_cookie=0 in_seq=0 out_seq=0 2023-08-09T13:34:11.822 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:11.814+0000 7f65b17fa640 10 client.0 ms_handle_connect on v2:172.21.15.43:3300/0 2023-08-09T13:34:11.822 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:11.815+0000 7f65b17fa640 1 -- 172.21.15.43:0/3013830630 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f65c0003810 con 0x558b7b465c50 2023-08-09T13:34:11.822 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:11.815+0000 7f65daf7f740 1 -- 172.21.15.43:0/3013830630 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x558b7b378500 con 0x558b7b465c50 2023-08-09T13:34:11.822 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:11.815+0000 7f65b17fa640 1 -- 172.21.15.43:0/3013830630 <== mon.0 v2:172.21.15.43:3300/0 2 ==== osd_map(24..24 src has 1..24) v4 ==== 6813+0+0 (secure 0 0 0) 0x7f65c0040070 con 0x558b7b465c50 2023-08-09T13:34:11.822 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:11.815+0000 7f65b17fa640 1 -- 172.21.15.43:0/3013830630 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_get_version_reply(handle=1 version=7) v2 ==== 24+0+0 (secure 0 0 0) 0x7f65c0057ea0 con 0x558b7b465c50 2023-08-09T13:34:11.822 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:11.815+0000 7f65daf7f740 10 client.4392 fetch_fsmap learned FSMap version 7 2023-08-09T13:34:11.822 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:11.815+0000 7f65daf7f740 1 -- 172.21.15.43:0/3013830630 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,fsmap=7,monmap=2+}) v3 -- 0x558b7b39ef50 con 0x558b7b465c50 2023-08-09T13:34:11.823 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:11.815+0000 7f65b17fa640 1 -- 172.21.15.43:0/3013830630 <== mon.0 v2:172.21.15.43:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x558b7b378500 con 0x558b7b465c50 2023-08-09T13:34:11.823 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:11.815+0000 7f65b17fa640 1 -- 172.21.15.43:0/3013830630 <== mon.0 v2:172.21.15.43:3300/0 5 ==== fsmap(e 7) v1 ==== 3302+0+0 (secure 0 0 0) 0x7f65c0003970 con 0x558b7b465c50 2023-08-09T13:34:11.823 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:11.815+0000 7f65daf7f740 10 client.4392 fetch_fsmap finished waiting for FSMap version 7 2023-08-09T13:34:11.823 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:11.815+0000 7f65daf7f740 10 client.4392 resolve_mds: resolved name 'd' to daemon mds.d 2023-08-09T13:34:11.823 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:11.815+0000 7f65daf7f740 20 client.4392 populate_metadata read hostname 'smithi043' 2023-08-09T13:34:11.824 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:11.815+0000 7f65daf7f740 1 --2- 172.21.15.43:0/3013830630 >> [v2:172.21.15.203:6834/798312463,v1:172.21.15.203:6836/798312463] conn(0x558b7b4788c0 0x558b7b478ca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:34:11.824 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:11.815+0000 7f65daf7f740 4 client.4392 mds_command: new command op to 4250 tid=0[{"prefix": "get_command_descriptions"}] 2023-08-09T13:34:11.825 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:11.815+0000 7f65daf7f740 1 -- 172.21.15.43:0/3013830630 --> [v2:172.21.15.203:6834/798312463,v1:172.21.15.203:6836/798312463] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x558b7b375bc0 con 0x558b7b4788c0 2023-08-09T13:34:11.825 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:11.816+0000 7f65d61af640 1 --2- 172.21.15.43:0/3013830630 >> [v2:172.21.15.203:6834/798312463,v1:172.21.15.203:6836/798312463] conn(0x558b7b4788c0 0x558b7b478ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:34:11.825 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:11.817+0000 7f65d61af640 1 --2- 172.21.15.43:0/3013830630 >> [v2:172.21.15.203:6834/798312463,v1:172.21.15.203:6836/798312463] conn(0x558b7b4788c0 0x558b7b478ca0 crc :-1 s=READY pgs=11 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=cdb8727ed9eb7938 server_cookie=9ae3764e2b4cc89d in_seq=0 out_seq=0 2023-08-09T13:34:11.825 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:11.817+0000 7f65b17fa640 10 client.4392 ms_handle_connect on v2:172.21.15.203:6834/798312463 2023-08-09T13:34:11.825 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:11.819+0000 7f65b17fa640 1 -- 172.21.15.43:0/3013830630 <== mds.0 v2:172.21.15.203:6834/798312463 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18017 (crc 0 0 0) 0x558b7b375bc0 con 0x558b7b4788c0 2023-08-09T13:34:11.825 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:11.819+0000 7f65b17fa640 10 client.4392 handle_command_reply: tid=0 2023-08-09T13:34:11.825 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:11.819+0000 7f65daf7f740 1 client.4392 shutdown 2023-08-09T13:34:11.826 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:11.820+0000 7f65daf7f740 1 -- 172.21.15.43:0/3013830630 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x558b7b465c50 msgr2=0x558b7b468620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:34:11.826 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:11.820+0000 7f65daf7f740 1 --2- 172.21.15.43:0/3013830630 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x558b7b465c50 0x558b7b468620 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f65c0047040 tx=0x7f65c0004730 comp rx=0 tx=0).stop 2023-08-09T13:34:11.826 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:11.820+0000 7f65daf7f740 1 -- 172.21.15.43:0/3013830630 shutdown_connections 2023-08-09T13:34:11.826 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:11.820+0000 7f65daf7f740 1 --2- 172.21.15.43:0/3013830630 >> [v2:172.21.15.203:6834/798312463,v1:172.21.15.203:6836/798312463] conn(0x558b7b4788c0 0x558b7b478ca0 crc :-1 s=READY pgs=11 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:34:11.826 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:11.820+0000 7f65daf7f740 1 --2- 172.21.15.43:0/3013830630 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x558b7b47a540 0x558b7b47a9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:34:11.826 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:11.820+0000 7f65daf7f740 1 --2- 172.21.15.43:0/3013830630 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x558b7b465c50 0x558b7b468620 unknown :-1 s=CLOSED pgs=87 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:34:11.826 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:11.820+0000 7f65daf7f740 1 --2- 172.21.15.43:0/3013830630 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x558b7b4397d0 0x558b7b456f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:34:11.826 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:11.820+0000 7f65b17fa640 0 client.4392 ms_handle_reset on v2:172.21.15.203:6834/798312463 2023-08-09T13:34:11.827 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:11.820+0000 7f65daf7f740 1 -- 172.21.15.43:0/3013830630 >> 172.21.15.43:0/3013830630 conn(0x558b7b435bd0 msgr2=0x558b7b43a050 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-09T13:34:11.827 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:11.820+0000 7f65daf7f740 1 -- 172.21.15.43:0/3013830630 shutdown_connections 2023-08-09T13:34:11.827 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:11.820+0000 7f65daf7f740 1 -- 172.21.15.43:0/3013830630 wait complete. 2023-08-09T13:34:11.827 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:11.820+0000 7f65daf7f740 20 client.4392 trim_cache size 0 max 16384 2023-08-09T13:34:11.839 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:11.837+0000 7f65daf7f740 1 Processor -- start 2023-08-09T13:34:11.840 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:11.837+0000 7f65daf7f740 1 -- start start 2023-08-09T13:34:11.840 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:11.838+0000 7f65daf7f740 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x558b7b4397d0 0x558b7b462a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:34:11.841 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:11.838+0000 7f65daf7f740 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x558b7b465c50 0x558b7b483f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:34:11.841 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:11.838+0000 7f65daf7f740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x558b7b47a540 0x558b7b4844b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:34:11.841 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:11.838+0000 7f65daf7f740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x558b7b4375a0 con 0x558b7b47a540 2023-08-09T13:34:11.841 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:11.838+0000 7f65daf7f740 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x558b7b358a20 con 0x558b7b4397d0 2023-08-09T13:34:11.841 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:11.838+0000 7f65daf7f740 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x558b7b461eb0 con 0x558b7b465c50 2023-08-09T13:34:11.842 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:11.839+0000 7f65d69b0640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x558b7b47a540 0x558b7b4844b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:34:11.842 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:11.839+0000 7f65d69b0640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x558b7b47a540 0x558b7b4844b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:42710/0 (socket says 172.21.15.43:42710) 2023-08-09T13:34:11.843 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:11.839+0000 7f65d69b0640 1 -- 172.21.15.43:0/4103307457 learned_addr learned my addr 172.21.15.43:0/4103307457 (peer_addr_for_me v2:172.21.15.43:0/0) 2023-08-09T13:34:11.843 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:11.839+0000 7f65d59ae640 1 --2- 172.21.15.43:0/4103307457 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x558b7b465c50 0x558b7b483f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:34:11.843 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:11.839+0000 7f65d61af640 1 --2- 172.21.15.43:0/4103307457 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x558b7b4397d0 0x558b7b462a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:34:11.843 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:11.839+0000 7f65d69b0640 1 -- 172.21.15.43:0/4103307457 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x558b7b465c50 msgr2=0x558b7b483f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-09T13:34:11.843 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:11.839+0000 7f65d69b0640 1 --2- 172.21.15.43:0/4103307457 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x558b7b465c50 0x558b7b483f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:34:11.843 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:11.839+0000 7f65d69b0640 1 -- 172.21.15.43:0/4103307457 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x558b7b4397d0 msgr2=0x558b7b462a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-09T13:34:11.843 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:11.839+0000 7f65d69b0640 1 --2- 172.21.15.43:0/4103307457 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x558b7b4397d0 0x558b7b462a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:34:11.844 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:11.839+0000 7f65d69b0640 1 -- 172.21.15.43:0/4103307457 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x558b7b3d2300 con 0x558b7b47a540 2023-08-09T13:34:11.844 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:11.839+0000 7f65d61af640 1 --2- 172.21.15.43:0/4103307457 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x558b7b4397d0 0x558b7b462a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-08-09T13:34:11.844 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:11.839+0000 7f65d69b0640 1 --2- 172.21.15.43:0/4103307457 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x558b7b47a540 0x558b7b4844b0 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f65c8054770 tx=0x7f65c8057bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f5959a3e0b301df7 server_cookie=0 in_seq=0 out_seq=0 2023-08-09T13:34:11.844 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:11.839+0000 7f65c4e1c640 1 -- 172.21.15.43:0/4103307457 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f65c805ff80 con 0x558b7b47a540 2023-08-09T13:34:11.844 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:11.840+0000 7f65c4e1c640 1 -- 172.21.15.43:0/4103307457 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f65c80563b0 con 0x558b7b47a540 2023-08-09T13:34:11.844 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:11.840+0000 7f65c4e1c640 1 -- 172.21.15.43:0/4103307457 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f65c800fd10 con 0x558b7b47a540 2023-08-09T13:34:11.844 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:11.840+0000 7f65daf7f740 1 -- 172.21.15.43:0/4103307457 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x558b7b47a540 msgr2=0x558b7b4844b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:34:11.845 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:11.840+0000 7f65daf7f740 1 --2- 172.21.15.43:0/4103307457 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x558b7b47a540 0x558b7b4844b0 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f65c8054770 tx=0x7f65c8057bd0 comp rx=0 tx=0).stop 2023-08-09T13:34:11.847 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:11.840+0000 7f65daf7f740 1 -- 172.21.15.43:0/4103307457 shutdown_connections 2023-08-09T13:34:11.847 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:11.840+0000 7f65daf7f740 1 --2- 172.21.15.43:0/4103307457 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x558b7b47a540 0x558b7b4844b0 unknown :-1 s=CLOSED pgs=88 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:34:11.847 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:11.840+0000 7f65daf7f740 1 --2- 172.21.15.43:0/4103307457 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x558b7b465c50 0x558b7b483f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:34:11.847 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:11.840+0000 7f65daf7f740 1 --2- 172.21.15.43:0/4103307457 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x558b7b4397d0 0x558b7b462a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:34:11.847 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:11.840+0000 7f65daf7f740 1 -- 172.21.15.43:0/4103307457 >> 172.21.15.43:0/4103307457 conn(0x558b7b435bd0 msgr2=0x558b7b478ca0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-09T13:34:11.848 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:11.840+0000 7f65daf7f740 1 -- 172.21.15.43:0/4103307457 shutdown_connections 2023-08-09T13:34:11.848 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:11.840+0000 7f65daf7f740 1 -- 172.21.15.43:0/4103307457 wait complete. 2023-08-09T13:34:11.848 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:11.841+0000 7f65daf7f740 1 -- start start 2023-08-09T13:34:11.848 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:11.841+0000 7f65daf7f740 1 Processor -- start 2023-08-09T13:34:11.848 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:11.842+0000 7f65daf7f740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x558b7b4397d0 0x558b7b483f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:34:11.848 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:11.842+0000 7f65daf7f740 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x558b7b465c50 0x558b7b4844b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:34:11.848 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:11.842+0000 7f65daf7f740 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x558b7b47a540 0x558b7b4763f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:34:11.849 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:11.842+0000 7f65daf7f740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x558b7b4f2610 con 0x558b7b4397d0 2023-08-09T13:34:11.849 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:11.842+0000 7f65daf7f740 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x558b7b4901b0 con 0x558b7b465c50 2023-08-09T13:34:11.849 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:11.842+0000 7f65daf7f740 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x558b7b4906e0 con 0x558b7b47a540 2023-08-09T13:34:11.849 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:11.842+0000 7f65daf7f740 10 client.0 _ll_register_callbacks cb 0x7f65c00315c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-08-09T13:34:11.849 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:11.843+0000 7f65d61af640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x558b7b4397d0 0x558b7b483f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:34:11.849 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:11.843+0000 7f65d61af640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x558b7b4397d0 0x558b7b483f70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:42712/0 (socket says 172.21.15.43:42712) 2023-08-09T13:34:11.849 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:11.843+0000 7f65d61af640 1 -- 172.21.15.43:0/2291203725 learned_addr learned my addr 172.21.15.43:0/2291203725 (peer_addr_for_me v2:172.21.15.43:0/0) 2023-08-09T13:34:11.850 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:11.843+0000 7f65d59ae640 1 --2- 172.21.15.43:0/2291203725 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x558b7b465c50 0x558b7b4844b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:34:11.850 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:11.843+0000 7f65d61af640 1 -- 172.21.15.43:0/2291203725 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x558b7b47a540 msgr2=0x558b7b4763f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-09T13:34:11.850 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:11.843+0000 7f65d61af640 1 --2- 172.21.15.43:0/2291203725 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x558b7b47a540 0x558b7b4763f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:34:11.850 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:11.843+0000 7f65d61af640 1 -- 172.21.15.43:0/2291203725 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x558b7b465c50 msgr2=0x558b7b4844b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-09T13:34:11.850 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:11.843+0000 7f65d61af640 1 --2- 172.21.15.43:0/2291203725 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x558b7b465c50 0x558b7b4844b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:34:11.850 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:11.843+0000 7f65d61af640 1 -- 172.21.15.43:0/2291203725 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f65d00c54f0 con 0x558b7b4397d0 2023-08-09T13:34:11.850 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:11.843+0000 7f65d61af640 1 --2- 172.21.15.43:0/2291203725 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x558b7b4397d0 0x558b7b483f70 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f65cc009390 tx=0x7f65cc004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2cc44bf4639cc365 server_cookie=0 in_seq=0 out_seq=0 2023-08-09T13:34:11.850 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:11.843+0000 7f65b17fa640 10 client.0 ms_handle_connect on v2:172.21.15.43:3300/0 2023-08-09T13:34:11.851 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:11.844+0000 7f65b17fa640 1 -- 172.21.15.43:0/2291203725 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f65cc00e7f0 con 0x558b7b4397d0 2023-08-09T13:34:11.851 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:11.844+0000 7f65b17fa640 1 -- 172.21.15.43:0/2291203725 <== mon.0 v2:172.21.15.43:3300/0 2 ==== osd_map(24..24 src has 1..24) v4 ==== 6813+0+0 (secure 0 0 0) 0x7f65cc00e950 con 0x558b7b4397d0 2023-08-09T13:34:11.851 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:11.844+0000 7f65daf7f740 1 -- 172.21.15.43:0/2291203725 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x558b7b3c8800 con 0x558b7b4397d0 2023-08-09T13:34:11.851 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:11.844+0000 7f65b17fa640 1 -- 172.21.15.43:0/2291203725 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_get_version_reply(handle=1 version=7) v2 ==== 24+0+0 (secure 0 0 0) 0x7f65cc00eb10 con 0x558b7b4397d0 2023-08-09T13:34:11.851 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:11.844+0000 7f65daf7f740 10 client.4398 fetch_fsmap learned FSMap version 7 2023-08-09T13:34:11.851 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:11.844+0000 7f65daf7f740 1 -- 172.21.15.43:0/2291203725 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,fsmap=7,monmap=2+}) v3 -- 0x558b7b33fa10 con 0x558b7b4397d0 2023-08-09T13:34:11.851 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:11.844+0000 7f65b17fa640 1 -- 172.21.15.43:0/2291203725 <== mon.0 v2:172.21.15.43:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x558b7b3c8800 con 0x558b7b4397d0 2023-08-09T13:34:11.852 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:11.844+0000 7f65b17fa640 1 -- 172.21.15.43:0/2291203725 <== mon.0 v2:172.21.15.43:3300/0 5 ==== fsmap(e 7) v1 ==== 3302+0+0 (secure 0 0 0) 0x7f65cc021050 con 0x558b7b4397d0 2023-08-09T13:34:11.852 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:11.844+0000 7f65daf7f740 10 client.4398 fetch_fsmap finished waiting for FSMap version 7 2023-08-09T13:34:11.852 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:11.844+0000 7f65daf7f740 10 client.4398 resolve_mds: resolved name 'd' to daemon mds.d 2023-08-09T13:34:11.852 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:11.844+0000 7f65daf7f740 20 client.4398 populate_metadata read hostname 'smithi043' 2023-08-09T13:34:11.852 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:11.844+0000 7f65daf7f740 1 --2- 172.21.15.43:0/2291203725 >> [v2:172.21.15.203:6834/798312463,v1:172.21.15.203:6836/798312463] conn(0x558b7b476ba0 0x558b7b4f1110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:34:11.852 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:11.844+0000 7f65daf7f740 4 client.4398 mds_command: new command op to 4250 tid=0[{"prefix": "status"}] 2023-08-09T13:34:11.852 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:11.844+0000 7f65daf7f740 1 -- 172.21.15.43:0/2291203725 --> [v2:172.21.15.203:6834/798312463,v1:172.21.15.203:6836/798312463] -- command(tid 0: {"prefix": "status"}) v1 -- 0x558b7b3998b0 con 0x558b7b476ba0 2023-08-09T13:34:11.853 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:11.845+0000 7f65d59ae640 1 --2- 172.21.15.43:0/2291203725 >> [v2:172.21.15.203:6834/798312463,v1:172.21.15.203:6836/798312463] conn(0x558b7b476ba0 0x558b7b4f1110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:34:11.854 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:11.846+0000 7f65d59ae640 1 --2- 172.21.15.43:0/2291203725 >> [v2:172.21.15.203:6834/798312463,v1:172.21.15.203:6836/798312463] conn(0x558b7b476ba0 0x558b7b4f1110 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f76f1efd62fe7db8 server_cookie=414db39fa361d430 in_seq=0 out_seq=0 2023-08-09T13:34:11.854 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:11.846+0000 7f65b17fa640 10 client.4398 ms_handle_connect on v2:172.21.15.203:6834/798312463 2023-08-09T13:34:11.854 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:11.846+0000 7f65b17fa640 1 -- 172.21.15.43:0/2291203725 <== mds.0 v2:172.21.15.203:6834/798312463 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+324 (crc 0 0 0) 0x558b7b3998b0 con 0x558b7b476ba0 2023-08-09T13:34:11.854 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:11.846+0000 7f65b17fa640 10 client.4398 handle_command_reply: tid=0 2023-08-09T13:34:11.854 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:11.846+0000 7f65daf7f740 1 client.4398 shutdown 2023-08-09T13:34:11.854 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:11.846+0000 7f65daf7f740 1 -- 172.21.15.43:0/2291203725 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x558b7b4397d0 msgr2=0x558b7b483f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:34:11.854 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:11.846+0000 7f65daf7f740 1 --2- 172.21.15.43:0/2291203725 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x558b7b4397d0 0x558b7b483f70 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f65cc009390 tx=0x7f65cc004730 comp rx=0 tx=0).stop 2023-08-09T13:34:11.855 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:11.847+0000 7f65daf7f740 1 -- 172.21.15.43:0/2291203725 shutdown_connections 2023-08-09T13:34:11.855 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:11.847+0000 7f65daf7f740 1 --2- 172.21.15.43:0/2291203725 >> [v2:172.21.15.203:6834/798312463,v1:172.21.15.203:6836/798312463] conn(0x558b7b476ba0 0x558b7b4f1110 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:34:11.855 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:11.847+0000 7f65daf7f740 1 --2- 172.21.15.43:0/2291203725 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x558b7b47a540 0x558b7b4763f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:34:11.855 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:11.847+0000 7f65daf7f740 1 --2- 172.21.15.43:0/2291203725 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x558b7b465c50 0x558b7b4844b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:34:11.855 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:11.847+0000 7f65b17fa640 0 client.4398 ms_handle_reset on v2:172.21.15.203:6834/798312463 2023-08-09T13:34:11.855 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:11.847+0000 7f65daf7f740 1 --2- 172.21.15.43:0/2291203725 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x558b7b4397d0 0x558b7b483f70 unknown :-1 s=CLOSED pgs=89 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:34:11.855 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:11.847+0000 7f65daf7f740 1 -- 172.21.15.43:0/2291203725 >> 172.21.15.43:0/2291203725 conn(0x558b7b435bd0 msgr2=0x558b7b47b390 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-09T13:34:11.856 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:11.847+0000 7f65daf7f740 1 -- 172.21.15.43:0/2291203725 shutdown_connections 2023-08-09T13:34:11.856 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:11.847+0000 7f65daf7f740 1 -- 172.21.15.43:0/2291203725 wait complete. 2023-08-09T13:34:11.856 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:11.847+0000 7f65daf7f740 20 client.4398 trim_cache size 0 max 16384 2023-08-09T13:34:11.856 INFO:teuthology.orchestra.run.smithi043.stdout:{ 2023-08-09T13:34:11.856 INFO:teuthology.orchestra.run.smithi043.stdout: "cluster_fsid": "daac87ee-1337-4250-a955-aea4e408a39b", 2023-08-09T13:34:11.856 INFO:teuthology.orchestra.run.smithi043.stdout: "whoami": 0, 2023-08-09T13:34:11.857 INFO:teuthology.orchestra.run.smithi043.stdout: "id": 4250, 2023-08-09T13:34:11.857 INFO:teuthology.orchestra.run.smithi043.stdout: "want_state": "up:active", 2023-08-09T13:34:11.857 INFO:teuthology.orchestra.run.smithi043.stdout: "state": "up:active", 2023-08-09T13:34:11.857 INFO:teuthology.orchestra.run.smithi043.stdout: "fs_name": "cephfs", 2023-08-09T13:34:11.857 INFO:teuthology.orchestra.run.smithi043.stdout: "rank_uptime": 8.6932525129999991, 2023-08-09T13:34:11.857 INFO:teuthology.orchestra.run.smithi043.stdout: "mdsmap_epoch": 7, 2023-08-09T13:34:11.857 INFO:teuthology.orchestra.run.smithi043.stdout: "osdmap_epoch": 24, 2023-08-09T13:34:11.857 INFO:teuthology.orchestra.run.smithi043.stdout: "osdmap_epoch_barrier": 24, 2023-08-09T13:34:11.858 INFO:teuthology.orchestra.run.smithi043.stdout: "uptime": 15.725182604 2023-08-09T13:34:11.858 INFO:teuthology.orchestra.run.smithi043.stdout:} 2023-08-09T13:34:11.858 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:11.847+0000 7f65d8413640 1 -- 172.21.15.43:0/2356950988 >> [v2:172.21.15.203:6832/830097474,v1:172.21.15.203:6833/830097474] conn(0x7f65b406a050 msgr2=0x7f65b406c510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:34:11.859 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:11.847+0000 7f65d8413640 1 --2- 172.21.15.43:0/2356950988 >> [v2:172.21.15.203:6832/830097474,v1:172.21.15.203:6833/830097474] conn(0x7f65b406a050 0x7f65b406c510 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f65cc006fd0 tx=0x7f65cc008040 comp rx=0 tx=0).stop 2023-08-09T13:34:11.859 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:11.847+0000 7f65d8413640 1 -- 172.21.15.43:0/2356950988 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f65d0143c20 msgr2=0x7f65d00c3fb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:34:11.859 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:11.847+0000 7f65d8413640 1 --2- 172.21.15.43:0/2356950988 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f65d0143c20 0x7f65d00c3fb0 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f65c800cd10 tx=0x7f65c800a650 comp rx=0 tx=0).stop 2023-08-09T13:34:11.859 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:11.848+0000 7f65d8413640 1 -- 172.21.15.43:0/2356950988 shutdown_connections 2023-08-09T13:34:11.859 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:11.848+0000 7f65d8413640 1 --2- 172.21.15.43:0/2356950988 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f65d0143c20 0x7f65d00c3fb0 unknown :-1 s=CLOSED pgs=85 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:34:11.859 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:11.848+0000 7f65d8413640 1 --2- 172.21.15.43:0/2356950988 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f65d00c86f0 0x7f65d0141a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:34:11.859 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:11.848+0000 7f65d8413640 1 --2- 172.21.15.43:0/2356950988 >> [v2:172.21.15.203:6832/830097474,v1:172.21.15.203:6833/830097474] conn(0x7f65b406a050 0x7f65b406c510 unknown :-1 s=CLOSED pgs=71 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:34:11.860 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:11.848+0000 7f65d8413640 1 --2- 172.21.15.43:0/2356950988 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f65d006c8d0 0x7f65d00c8060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:34:11.860 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:11.848+0000 7f65d8413640 1 -- 172.21.15.43:0/2356950988 >> 172.21.15.43:0/2356950988 conn(0x7f65d00ce0a0 msgr2=0x7f65d00cb3b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-09T13:34:11.860 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:11.848+0000 7f65d8413640 1 -- 172.21.15.43:0/2356950988 shutdown_connections 2023-08-09T13:34:11.860 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:11.848+0000 7f65d8413640 1 -- 172.21.15.43:0/2356950988 wait complete. 2023-08-09T13:34:11.867 INFO:tasks.cephfs.mount:cephfs_mntpt = None 2023-08-09T13:34:11.868 INFO:tasks.cephfs.mount:hostfs_mntpt = /home/ubuntu/cephtest/mnt.admin 2023-08-09T13:34:11.868 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2023-08-09T13:34:12.097 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:12.095+0000 7f0b44757640 1 Processor -- start 2023-08-09T13:34:12.097 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:12.095+0000 7f0b44757640 1 -- start start 2023-08-09T13:34:12.099 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:12.097+0000 7f0b44757640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f0b3c0b4670 0x7f0b3c0d2ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:34:12.099 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:12.097+0000 7f0b44757640 1 -- --> v1:172.21.15.43:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0b3c0ce090 con 0x7f0b3c0b2790 2023-08-09T13:34:12.099 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:12.097+0000 7f0b44757640 1 -- --> v1:172.21.15.203:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0b3c0cd7f0 con 0x7f0b3c0b2b70 2023-08-09T13:34:12.100 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:12.097+0000 7f0b44757640 1 -- --> v2:172.21.15.43:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0b3c0b3950 con 0x7f0b3c0b4670 2023-08-09T13:34:12.100 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:12.097+0000 7f0b424f3640 1 --1- >> v1:172.21.15.43:6789/0 conn(0x7f0b3c0b2790 0x7f0b3c0b5330 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.43:6789/0 says I am v1:172.21.15.43:57486/0 (socket says 172.21.15.43:57486) 2023-08-09T13:34:12.100 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:12.097+0000 7f0b424f3640 1 -- 172.21.15.43:0/4217800680 learned_addr learned my addr 172.21.15.43:0/4217800680 (peer_addr_for_me v1:172.21.15.43:0/0) 2023-08-09T13:34:12.100 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:12.097+0000 7f0b42cf4640 1 --2- 172.21.15.43:0/4217800680 >> v2:172.21.15.43:3300/0 conn(0x7f0b3c0b4670 0x7f0b3c0d2ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:34:12.101 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:12.097+0000 7f0b414f1640 1 -- 172.21.15.43:0/4217800680 <== mon.0 v1:172.21.15.43:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2332683119 0 0) 0x7f0b3c0ce090 con 0x7f0b3c0b2790 2023-08-09T13:34:12.101 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:12.097+0000 7f0b414f1640 1 -- 172.21.15.43:0/4217800680 --> v1:172.21.15.43:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f0b28003520 con 0x7f0b3c0b2790 2023-08-09T13:34:12.101 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:12.098+0000 7f0b42cf4640 1 -- 172.21.15.43:0/4217800680 >> v1:172.21.15.203:6789/0 conn(0x7f0b3c0b2b70 legacy=0x7f0b3c0b3f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:34:12.101 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:12.098+0000 7f0b42cf4640 1 -- 172.21.15.43:0/4217800680 >> v1:172.21.15.43:6789/0 conn(0x7f0b3c0b2790 legacy=0x7f0b3c0b5330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:34:12.101 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:12.098+0000 7f0b42cf4640 1 -- 172.21.15.43:0/4217800680 --> v2:172.21.15.43:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0b3c0cdad0 con 0x7f0b3c0b4670 2023-08-09T13:34:12.102 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:12.099+0000 7f0b42cf4640 1 --2- 172.21.15.43:0/4217800680 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f0b3c0b4670 0x7f0b3c0d2ed0 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f0b34009d90 tx=0x7f0b3402bf50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e202ad36336c6d93 server_cookie=0 in_seq=0 out_seq=0 2023-08-09T13:34:12.102 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:12.099+0000 7f0b414f1640 1 -- 172.21.15.43:0/4217800680 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0b3402cd00 con 0x7f0b3c0b4670 2023-08-09T13:34:12.103 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:12.099+0000 7f0b414f1640 1 -- 172.21.15.43:0/4217800680 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0b3402ce60 con 0x7f0b3c0b4670 2023-08-09T13:34:12.103 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:12.099+0000 7f0b414f1640 1 -- 172.21.15.43:0/4217800680 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0b34034640 con 0x7f0b3c0b4670 2023-08-09T13:34:12.103 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:12.100+0000 7f0b44757640 1 -- 172.21.15.43:0/4217800680 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f0b3c0b4670 msgr2=0x7f0b3c0d2ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:34:12.103 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:12.100+0000 7f0b44757640 1 --2- 172.21.15.43:0/4217800680 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f0b3c0b4670 0x7f0b3c0d2ed0 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f0b34009d90 tx=0x7f0b3402bf50 comp rx=0 tx=0).stop 2023-08-09T13:34:12.103 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:12.100+0000 7f0b44757640 1 -- 172.21.15.43:0/4217800680 shutdown_connections 2023-08-09T13:34:12.103 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:12.100+0000 7f0b44757640 1 --2- 172.21.15.43:0/4217800680 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f0b3c0b4670 0x7f0b3c0d2ed0 unknown :-1 s=CLOSED pgs=91 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:34:12.103 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:12.100+0000 7f0b44757640 1 -- 172.21.15.43:0/4217800680 >> 172.21.15.43:0/4217800680 conn(0x7f0b3c09b7b0 msgr2=0x7f0b3c0d24e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-09T13:34:12.104 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:12.100+0000 7f0b44757640 1 -- 172.21.15.43:0/4217800680 shutdown_connections 2023-08-09T13:34:12.104 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:12.100+0000 7f0b44757640 1 -- 172.21.15.43:0/4217800680 wait complete. 2023-08-09T13:34:12.104 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:12.102+0000 7f0b44757640 1 Processor -- start 2023-08-09T13:34:12.104 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:12.102+0000 7f0b44757640 1 -- start start 2023-08-09T13:34:12.105 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:12.103+0000 7f0b44757640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f0b3c0b2790 0x7f0b3c0c6280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:34:12.106 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:12.103+0000 7f0b44757640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f0b3c0b2b70 0x7f0b3c0cbd80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:34:12.106 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:12.103+0000 7f0b44757640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f0b3c0b4670 0x7f0b3c15a5d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:34:12.106 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:12.103+0000 7f0b44757640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0b3c0c8600 con 0x7f0b3c0b2790 2023-08-09T13:34:12.106 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:12.103+0000 7f0b44757640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0b3c0c79f0 con 0x7f0b3c0b4670 2023-08-09T13:34:12.106 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:12.103+0000 7f0b44757640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0b3c0c7cd0 con 0x7f0b3c0b2b70 2023-08-09T13:34:12.106 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:12.103+0000 7f0b424f3640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f0b3c0b2790 0x7f0b3c0c6280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:34:12.106 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:12.103+0000 7f0b424f3640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f0b3c0b2790 0x7f0b3c0c6280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:42728/0 (socket says 172.21.15.43:42728) 2023-08-09T13:34:12.107 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:12.103+0000 7f0b424f3640 1 -- 172.21.15.43:0/530987284 learned_addr learned my addr 172.21.15.43:0/530987284 (peer_addr_for_me v2:172.21.15.43:0/0) 2023-08-09T13:34:12.108 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:12.104+0000 7f0b41cf2640 1 --2- 172.21.15.43:0/530987284 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f0b3c0b2b70 0x7f0b3c0cbd80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:34:12.108 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:12.104+0000 7f0b42cf4640 1 --2- 172.21.15.43:0/530987284 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f0b3c0b4670 0x7f0b3c15a5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:34:12.108 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:12.104+0000 7f0b424f3640 1 -- 172.21.15.43:0/530987284 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f0b3c0b2b70 msgr2=0x7f0b3c0cbd80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:34:12.108 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:12.104+0000 7f0b424f3640 1 --2- 172.21.15.43:0/530987284 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f0b3c0b2b70 0x7f0b3c0cbd80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:34:12.108 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:12.104+0000 7f0b424f3640 1 -- 172.21.15.43:0/530987284 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f0b3c0b4670 msgr2=0x7f0b3c15a5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:34:12.108 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:12.104+0000 7f0b424f3640 1 --2- 172.21.15.43:0/530987284 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f0b3c0b4670 0x7f0b3c15a5d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:34:12.109 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:12.104+0000 7f0b424f3640 1 -- 172.21.15.43:0/530987284 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0b34009a70 con 0x7f0b3c0b2790 2023-08-09T13:34:12.109 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:12.104+0000 7f0b41cf2640 1 --2- 172.21.15.43:0/530987284 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f0b3c0b2b70 0x7f0b3c0cbd80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-08-09T13:34:12.109 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:12.104+0000 7f0b424f3640 1 --2- 172.21.15.43:0/530987284 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f0b3c0b2790 0x7f0b3c0c6280 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7f0b3800c410 tx=0x7f0b3800c8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-09T13:34:12.109 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:12.104+0000 7f0b32ffd640 1 -- 172.21.15.43:0/530987284 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0b38005ce0 con 0x7f0b3c0b2790 2023-08-09T13:34:12.109 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:12.104+0000 7f0b32ffd640 1 -- 172.21.15.43:0/530987284 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0b38005e40 con 0x7f0b3c0b2790 2023-08-09T13:34:12.109 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:12.104+0000 7f0b32ffd640 1 -- 172.21.15.43:0/530987284 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0b380072f0 con 0x7f0b3c0b2790 2023-08-09T13:34:12.109 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:12.104+0000 7f0b44757640 1 -- 172.21.15.43:0/530987284 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0b3c0c7fb0 con 0x7f0b3c0b2790 2023-08-09T13:34:12.110 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:12.104+0000 7f0b44757640 1 -- 172.21.15.43:0/530987284 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0b3c0c8a90 con 0x7f0b3c0b2790 2023-08-09T13:34:12.110 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:12.106+0000 7f0b44757640 1 -- 172.21.15.43:0/530987284 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0b3c0739f0 con 0x7f0b3c0b2790 2023-08-09T13:34:12.110 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:12.106+0000 7f0b32ffd640 1 -- 172.21.15.43:0/530987284 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 5) v1 ==== 88341+0+0 (secure 0 0 0) 0x7f0b38007510 con 0x7f0b3c0b2790 2023-08-09T13:34:12.111 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:12.107+0000 7f0b32ffd640 1 --2- 172.21.15.43:0/530987284 >> [v2:172.21.15.203:6832/830097474,v1:172.21.15.203:6833/830097474] conn(0x7f0b2006a050 0x7f0b2006c510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:34:12.111 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:12.107+0000 7f0b32ffd640 1 -- 172.21.15.43:0/530987284 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6813+0+0 (secure 0 0 0) 0x7f0b38087fe0 con 0x7f0b3c0b2790 2023-08-09T13:34:12.111 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:12.108+0000 7f0b41cf2640 1 --2- 172.21.15.43:0/530987284 >> [v2:172.21.15.203:6832/830097474,v1:172.21.15.203:6833/830097474] conn(0x7f0b2006a050 0x7f0b2006c510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:34:12.111 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:12.109+0000 7f0b41cf2640 1 --2- 172.21.15.43:0/530987284 >> [v2:172.21.15.203:6832/830097474,v1:172.21.15.203:6833/830097474] conn(0x7f0b2006a050 0x7f0b2006c510 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f0b3c15b9c0 tx=0x7f0b2c009040 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-09T13:34:12.114 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:12.112+0000 7f0b32ffd640 1 -- 172.21.15.43:0/530987284 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178066 (secure 0 0 0) 0x7f0b38053400 con 0x7f0b3c0b2790 2023-08-09T13:34:12.348 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:12.345+0000 7f0b44757640 1 -- 172.21.15.43:0/530987284 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7f0b3c0ce840 con 0x7f0b3c0b2790 2023-08-09T13:34:12.348 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:12.346+0000 7f0b32ffd640 1 -- 172.21.15.43:0/530987284 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v7) v1 ==== 53+0+88 (secure 0 0 0) 0x7f0b38017070 con 0x7f0b3c0b2790 2023-08-09T13:34:12.349 INFO:teuthology.orchestra.run.smithi043.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data cephfs_data_ec ] 2023-08-09T13:34:12.351 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:12.349+0000 7f0b44757640 1 -- 172.21.15.43:0/530987284 >> [v2:172.21.15.203:6832/830097474,v1:172.21.15.203:6833/830097474] conn(0x7f0b2006a050 msgr2=0x7f0b2006c510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:34:12.352 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:12.349+0000 7f0b44757640 1 --2- 172.21.15.43:0/530987284 >> [v2:172.21.15.203:6832/830097474,v1:172.21.15.203:6833/830097474] conn(0x7f0b2006a050 0x7f0b2006c510 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f0b3c15b9c0 tx=0x7f0b2c009040 comp rx=0 tx=0).stop 2023-08-09T13:34:12.352 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:12.349+0000 7f0b44757640 1 -- 172.21.15.43:0/530987284 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f0b3c0b2790 msgr2=0x7f0b3c0c6280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:34:12.352 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:12.349+0000 7f0b44757640 1 --2- 172.21.15.43:0/530987284 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f0b3c0b2790 0x7f0b3c0c6280 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7f0b3800c410 tx=0x7f0b3800c8d0 comp rx=0 tx=0).stop 2023-08-09T13:34:12.352 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:12.349+0000 7f0b44757640 1 -- 172.21.15.43:0/530987284 shutdown_connections 2023-08-09T13:34:12.352 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:12.349+0000 7f0b44757640 1 --2- 172.21.15.43:0/530987284 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f0b3c0b4670 0x7f0b3c15a5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:34:12.352 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:12.349+0000 7f0b44757640 1 --2- 172.21.15.43:0/530987284 >> [v2:172.21.15.203:6832/830097474,v1:172.21.15.203:6833/830097474] conn(0x7f0b2006a050 0x7f0b2006c510 unknown :-1 s=CLOSED pgs=72 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:34:12.353 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:12.349+0000 7f0b44757640 1 --2- 172.21.15.43:0/530987284 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f0b3c0b2b70 0x7f0b3c0cbd80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:34:12.353 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:12.349+0000 7f0b44757640 1 --2- 172.21.15.43:0/530987284 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f0b3c0b2790 0x7f0b3c0c6280 unknown :-1 s=CLOSED pgs=92 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:34:12.353 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:12.349+0000 7f0b44757640 1 -- 172.21.15.43:0/530987284 >> 172.21.15.43:0/530987284 conn(0x7f0b3c09b7b0 msgr2=0x7f0b3c0b3f60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-09T13:34:12.353 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:12.349+0000 7f0b44757640 1 -- 172.21.15.43:0/530987284 shutdown_connections 2023-08-09T13:34:12.353 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:12.349+0000 7f0b44757640 1 -- 172.21.15.43:0/530987284 wait complete. 2023-08-09T13:34:12.371 INFO:tasks.cephfs.mount:Mounting Ceph FS cephfs; just confirmed its presence on cluster 2023-08-09T13:34:12.372 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2023-08-09T13:34:12.372 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi043.front.sepia.ceph.com 2023-08-09T13:34:12.372 INFO:tasks.cephfs.mount:self.client.name = client.admin 2023-08-09T13:34:12.373 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.admin 2023-08-09T13:34:12.373 INFO:tasks.cephfs.mount:self.cephfs_name = cephfs 2023-08-09T13:34:12.373 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2023-08-09T13:34:12.373 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2023-08-09T13:34:12.373 INFO:tasks.cephfs.mount:Setting the 'None' netns for '/home/ubuntu/cephtest/mnt.admin' 2023-08-09T13:34:12.374 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-09T13:34:12.374 DEBUG:teuthology.orchestra.run.smithi043:> ip addr 2023-08-09T13:34:12.430 INFO:teuthology.orchestra.run.smithi043.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2023-08-09T13:34:12.430 INFO:teuthology.orchestra.run.smithi043.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2023-08-09T13:34:12.430 INFO:teuthology.orchestra.run.smithi043.stdout: inet 127.0.0.1/8 scope host lo 2023-08-09T13:34:12.430 INFO:teuthology.orchestra.run.smithi043.stdout: valid_lft forever preferred_lft forever 2023-08-09T13:34:12.431 INFO:teuthology.orchestra.run.smithi043.stdout: inet6 ::1/128 scope host 2023-08-09T13:34:12.431 INFO:teuthology.orchestra.run.smithi043.stdout: valid_lft forever preferred_lft forever 2023-08-09T13:34:12.431 INFO:teuthology.orchestra.run.smithi043.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-08-09T13:34:12.431 INFO:teuthology.orchestra.run.smithi043.stdout: link/ether 0c:c4:7a:6b:fd:9c brd ff:ff:ff:ff:ff:ff 2023-08-09T13:34:12.431 INFO:teuthology.orchestra.run.smithi043.stdout: altname enp7s0f0 2023-08-09T13:34:12.431 INFO:teuthology.orchestra.run.smithi043.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-08-09T13:34:12.431 INFO:teuthology.orchestra.run.smithi043.stdout: link/ether 0c:c4:7a:6b:fd:9d brd ff:ff:ff:ff:ff:ff 2023-08-09T13:34:12.432 INFO:teuthology.orchestra.run.smithi043.stdout: altname enp7s0f1 2023-08-09T13:34:12.432 INFO:teuthology.orchestra.run.smithi043.stdout:4: ens1f0: mtu 1500 qdisc mq state UP group default qlen 1000 2023-08-09T13:34:12.432 INFO:teuthology.orchestra.run.smithi043.stdout: link/ether 0c:c4:7a:bd:13:58 brd ff:ff:ff:ff:ff:ff 2023-08-09T13:34:12.432 INFO:teuthology.orchestra.run.smithi043.stdout: altname enp1s0f0 2023-08-09T13:34:12.432 INFO:teuthology.orchestra.run.smithi043.stdout: inet 172.21.15.43/20 brd 172.21.15.255 scope global dynamic noprefixroute ens1f0 2023-08-09T13:34:12.432 INFO:teuthology.orchestra.run.smithi043.stdout: valid_lft 42251sec preferred_lft 42251sec 2023-08-09T13:34:12.432 INFO:teuthology.orchestra.run.smithi043.stdout: inet6 fe80::ec4:7aff:febd:1358/64 scope link 2023-08-09T13:34:12.433 INFO:teuthology.orchestra.run.smithi043.stdout: valid_lft forever preferred_lft forever 2023-08-09T13:34:12.433 INFO:teuthology.orchestra.run.smithi043.stdout:5: ens1f1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-08-09T13:34:12.433 INFO:teuthology.orchestra.run.smithi043.stdout: link/ether 0c:c4:7a:bd:13:59 brd ff:ff:ff:ff:ff:ff 2023-08-09T13:34:12.433 INFO:teuthology.orchestra.run.smithi043.stdout: altname enp1s0f1 2023-08-09T13:34:12.434 INFO:tasks.cephfs.mount:Setuping the 'ceph-brx' with 192.168.255.254/16 2023-08-09T13:34:12.434 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-09T13:34:12.435 DEBUG:teuthology.orchestra.run.smithi043:> (cd / && exec bash -c ' 2023-08-09T13:34:12.435 DEBUG:teuthology.orchestra.run.smithi043:> set -e 2023-08-09T13:34:12.436 DEBUG:teuthology.orchestra.run.smithi043:> sudo ip link add name ceph-brx type bridge 2023-08-09T13:34:12.436 DEBUG:teuthology.orchestra.run.smithi043:> sudo ip addr flush dev ceph-brx 2023-08-09T13:34:12.436 DEBUG:teuthology.orchestra.run.smithi043:> sudo ip link set ceph-brx up 2023-08-09T13:34:12.436 DEBUG:teuthology.orchestra.run.smithi043:> sudo ip addr add 192.168.255.254/16 brd 192.168.255.255 dev ceph-brx 2023-08-09T13:34:12.437 DEBUG:teuthology.orchestra.run.smithi043:> ') 2023-08-09T13:34:12.945 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-09T13:34:12.945 DEBUG:teuthology.orchestra.run.smithi043:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2023-08-09T13:34:13.030 INFO:teuthology.orchestra.run.smithi043.stdout:1 2023-08-09T13:34:13.033 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-09T13:34:13.034 DEBUG:teuthology.orchestra.run.smithi043:> route 2023-08-09T13:34:13.089 INFO:teuthology.orchestra.run.smithi043.stdout:Kernel IP routing table 2023-08-09T13:34:13.089 INFO:teuthology.orchestra.run.smithi043.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2023-08-09T13:34:13.090 INFO:teuthology.orchestra.run.smithi043.stdout:default _gateway 0.0.0.0 UG 100 0 0 ens1f0 2023-08-09T13:34:13.090 INFO:teuthology.orchestra.run.smithi043.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 ens1f0 2023-08-09T13:34:13.090 INFO:teuthology.orchestra.run.smithi043.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2023-08-09T13:34:13.090 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-09T13:34:13.091 DEBUG:teuthology.orchestra.run.smithi043:> (cd / && exec bash -c ' 2023-08-09T13:34:13.092 DEBUG:teuthology.orchestra.run.smithi043:> set -e 2023-08-09T13:34:13.092 DEBUG:teuthology.orchestra.run.smithi043:> sudo iptables -A FORWARD -o ens1f0 -i ceph-brx -j ACCEPT 2023-08-09T13:34:13.093 DEBUG:teuthology.orchestra.run.smithi043:> sudo iptables -A FORWARD -i ens1f0 -o ceph-brx -j ACCEPT 2023-08-09T13:34:13.093 DEBUG:teuthology.orchestra.run.smithi043:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o ens1f0 -j MASQUERADE 2023-08-09T13:34:13.093 DEBUG:teuthology.orchestra.run.smithi043:> ') 2023-08-09T13:34:13.954 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-09T13:34:13.955 DEBUG:teuthology.orchestra.run.smithi043:> ip netns list 2023-08-09T13:34:14.016 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-09T13:34:14.016 DEBUG:teuthology.orchestra.run.smithi043:> ip netns list-id 2023-08-09T13:34:14.075 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-09T13:34:14.076 DEBUG:teuthology.orchestra.run.smithi043:> (cd / && exec bash -c ' 2023-08-09T13:34:14.076 DEBUG:teuthology.orchestra.run.smithi043:> set -e 2023-08-09T13:34:14.076 DEBUG:teuthology.orchestra.run.smithi043:> sudo ip netns add ceph-ns--home-ubuntu-cephtest-mnt.admin 2023-08-09T13:34:14.077 DEBUG:teuthology.orchestra.run.smithi043:> sudo ip netns set ceph-ns--home-ubuntu-cephtest-mnt.admin 0 2023-08-09T13:34:14.077 DEBUG:teuthology.orchestra.run.smithi043:> ') 2023-08-09T13:34:14.237 INFO:tasks.cephfs.mount:Setuping the netns 'ceph-ns--home-ubuntu-cephtest-mnt.admin' with 192.168.0.1/16 2023-08-09T13:34:14.238 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-09T13:34:14.238 DEBUG:teuthology.orchestra.run.smithi043:> (cd / && exec bash -c ' 2023-08-09T13:34:14.238 DEBUG:teuthology.orchestra.run.smithi043:> set -e 2023-08-09T13:34:14.239 DEBUG:teuthology.orchestra.run.smithi043:> sudo ip link add veth0 netns ceph-ns--home-ubuntu-cephtest-mnt.admin type veth peer name brx.0 2023-08-09T13:34:14.239 DEBUG:teuthology.orchestra.run.smithi043:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.admin ip addr add 192.168.0.1/16 brd 192.168.255.255 dev veth0 2023-08-09T13:34:14.239 DEBUG:teuthology.orchestra.run.smithi043:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.admin ip link set veth0 up 2023-08-09T13:34:14.239 DEBUG:teuthology.orchestra.run.smithi043:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.admin ip link set lo up 2023-08-09T13:34:14.240 DEBUG:teuthology.orchestra.run.smithi043:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.admin ip route add default via 192.168.255.254 2023-08-09T13:34:14.240 DEBUG:teuthology.orchestra.run.smithi043:> ') 2023-08-09T13:34:14.553 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-09T13:34:14.554 DEBUG:teuthology.orchestra.run.smithi043:> (cd / && exec bash -c ' 2023-08-09T13:34:14.554 DEBUG:teuthology.orchestra.run.smithi043:> set -e 2023-08-09T13:34:14.554 DEBUG:teuthology.orchestra.run.smithi043:> sudo ip link set brx.0 up 2023-08-09T13:34:14.554 DEBUG:teuthology.orchestra.run.smithi043:> sudo ip link set dev brx.0 master ceph-brx 2023-08-09T13:34:14.555 DEBUG:teuthology.orchestra.run.smithi043:> ') 2023-08-09T13:34:14.731 INFO:tasks.cephfs.fuse_mount:Client client.admin config is {} 2023-08-09T13:34:14.732 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-08-09T13:34:14.732 DEBUG:teuthology.orchestra.run.smithi043:> mkdir -p -v /home/ubuntu/cephtest/mnt.admin 2023-08-09T13:34:14.782 INFO:teuthology.orchestra.run.smithi043.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.admin' 2023-08-09T13:34:14.782 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-08-09T13:34:14.783 DEBUG:teuthology.orchestra.run.smithi043:> chmod 0000 /home/ubuntu/cephtest/mnt.admin 2023-08-09T13:34:14.828 DEBUG:teuthology.orchestra.run.smithi043:> sudo modprobe fuse 2023-08-09T13:34:14.965 INFO:teuthology.orchestra.run:Running command with timeout 30 2023-08-09T13:34:14.966 DEBUG:teuthology.orchestra.run.smithi043:> sudo mount -t fusectl /sys/fs/fuse/connections /sys/fs/fuse/connections 2023-08-09T13:34:15.061 INFO:teuthology.orchestra.run.smithi043.stderr:mount: /sys/fs/fuse/connections: /sys/fs/fuse/connections already mounted or mount point busy. 2023-08-09T13:34:15.065 DEBUG:teuthology.orchestra.run:got remote process result: 32 2023-08-09T13:34:15.066 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-09T13:34:15.066 DEBUG:teuthology.orchestra.run.smithi043:> ls /sys/fs/fuse/connections 2023-08-09T13:34:15.126 INFO:tasks.cephfs.fuse_mount:Pre-mount connections: [] 2023-08-09T13:34:15.127 DEBUG:teuthology.orchestra.run.smithi043:> (cd /home/ubuntu/cephtest && exec sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.admin sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-fuse -f --admin-socket '/var/run/ceph/$cluster-$name.$pid.asok' /home/ubuntu/cephtest/mnt.admin --id admin --client_fs=cephfs) 2023-08-09T13:34:15.134 DEBUG:teuthology.orchestra.run.smithi043:> sudo modprobe fuse 2023-08-09T13:34:15.235 INFO:teuthology.orchestra.run:Running command with timeout 30 2023-08-09T13:34:15.236 DEBUG:teuthology.orchestra.run.smithi043:> sudo mount -t fusectl /sys/fs/fuse/connections /sys/fs/fuse/connections 2023-08-09T13:34:15.320 INFO:teuthology.orchestra.run.smithi043.stderr:mount: /sys/fs/fuse/connections: /sys/fs/fuse/connections already mounted or mount point busy. 2023-08-09T13:34:15.323 DEBUG:teuthology.orchestra.run:got remote process result: 32 2023-08-09T13:34:15.324 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-09T13:34:15.324 DEBUG:teuthology.orchestra.run.smithi043:> ls /sys/fs/fuse/connections 2023-08-09T13:34:15.349 INFO:tasks.cephfs.fuse_mount.ceph-fuse.admin.smithi043.stderr:2023-08-09T13:34:15.347+0000 7f42a4829480 -1 init, newargv = 0x557609383030 newargc=15 2023-08-09T13:34:15.350 INFO:tasks.cephfs.fuse_mount.ceph-fuse.admin.smithi043.stderr:ceph-fuse[94557]: starting ceph client 2023-08-09T13:34:15.366 INFO:tasks.cephfs.fuse_mount.ceph-fuse.admin.smithi043.stderr:ceph-fuse[94557]: starting fuse 2023-08-09T13:34:15.368 INFO:teuthology.orchestra.run.smithi043.stdout:40 2023-08-09T13:34:15.368 INFO:tasks.cephfs.fuse_mount:Post-mount connections: [40] 2023-08-09T13:34:15.369 DEBUG:teuthology.orchestra.run.smithi043:> sudo stdin-killer -- python3 -c ' 2023-08-09T13:34:15.369 DEBUG:teuthology.orchestra.run.smithi043:> import glob 2023-08-09T13:34:15.369 DEBUG:teuthology.orchestra.run.smithi043:> import re 2023-08-09T13:34:15.370 DEBUG:teuthology.orchestra.run.smithi043:> import os 2023-08-09T13:34:15.370 DEBUG:teuthology.orchestra.run.smithi043:> import subprocess 2023-08-09T13:34:15.370 DEBUG:teuthology.orchestra.run.smithi043:> 2023-08-09T13:34:15.370 DEBUG:teuthology.orchestra.run.smithi043:> def _find_admin_socket(client_name): 2023-08-09T13:34:15.371 DEBUG:teuthology.orchestra.run.smithi043:> asok_path = "/var/run/ceph/ceph-client.admin.*.asok" 2023-08-09T13:34:15.371 DEBUG:teuthology.orchestra.run.smithi043:> files = glob.glob(asok_path) 2023-08-09T13:34:15.371 DEBUG:teuthology.orchestra.run.smithi043:> mountpoint = "/home/ubuntu/cephtest/mnt.admin" 2023-08-09T13:34:15.371 DEBUG:teuthology.orchestra.run.smithi043:> 2023-08-09T13:34:15.372 DEBUG:teuthology.orchestra.run.smithi043:> # Given a non-glob path, it better be there 2023-08-09T13:34:15.372 DEBUG:teuthology.orchestra.run.smithi043:> if "*" not in asok_path: 2023-08-09T13:34:15.372 DEBUG:teuthology.orchestra.run.smithi043:> assert(len(files) == 1) 2023-08-09T13:34:15.373 DEBUG:teuthology.orchestra.run.smithi043:> return files[0] 2023-08-09T13:34:15.373 DEBUG:teuthology.orchestra.run.smithi043:> 2023-08-09T13:34:15.373 DEBUG:teuthology.orchestra.run.smithi043:> for f in files: 2023-08-09T13:34:15.373 DEBUG:teuthology.orchestra.run.smithi043:> pid = re.match(".*\.(\d+)\.asok$", f).group(1) 2023-08-09T13:34:15.374 DEBUG:teuthology.orchestra.run.smithi043:> if os.path.exists("/proc/{0}".format(pid)): 2023-08-09T13:34:15.374 DEBUG:teuthology.orchestra.run.smithi043:> with open("/proc/{0}/cmdline".format(pid), '"'"'r'"'"') as proc_f: 2023-08-09T13:34:15.374 DEBUG:teuthology.orchestra.run.smithi043:> contents = proc_f.read() 2023-08-09T13:34:15.374 DEBUG:teuthology.orchestra.run.smithi043:> if mountpoint in contents: 2023-08-09T13:34:15.375 DEBUG:teuthology.orchestra.run.smithi043:> return f 2023-08-09T13:34:15.375 DEBUG:teuthology.orchestra.run.smithi043:> raise RuntimeError("Client socket {0} not found".format(client_name)) 2023-08-09T13:34:15.375 DEBUG:teuthology.orchestra.run.smithi043:> 2023-08-09T13:34:15.375 DEBUG:teuthology.orchestra.run.smithi043:> print(_find_admin_socket("client.admin")) 2023-08-09T13:34:15.376 DEBUG:teuthology.orchestra.run.smithi043:> ' 2023-08-09T13:34:15.588 INFO:teuthology.orchestra.run.smithi043.stdout:/var/run/ceph/ceph-client.admin.94557.asok 2023-08-09T13:34:15.592 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:15 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2023-08-09T13:34:15.605 INFO:tasks.cephfs.fuse_mount:Found client admin socket at /var/run/ceph/ceph-client.admin.94557.asok 2023-08-09T13:34:15.606 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-09T13:34:15.606 DEBUG:teuthology.orchestra.run.smithi043:> sudo ceph --admin-daemon /var/run/ceph/ceph-client.admin.94557.asok status 2023-08-09T13:34:15.850 INFO:teuthology.orchestra.run.smithi043.stdout:{ 2023-08-09T13:34:15.850 INFO:teuthology.orchestra.run.smithi043.stdout: "metadata": { 2023-08-09T13:34:15.850 INFO:teuthology.orchestra.run.smithi043.stdout: "ceph_sha1": "668a86f232c9454b2ce5928d1432a29a72964d85", 2023-08-09T13:34:15.850 INFO:teuthology.orchestra.run.smithi043.stdout: "ceph_version": "ceph version 18.0.0-5389-g668a86f2 (668a86f232c9454b2ce5928d1432a29a72964d85) reef (dev)", 2023-08-09T13:34:15.850 INFO:teuthology.orchestra.run.smithi043.stdout: "entity_id": "admin", 2023-08-09T13:34:15.851 INFO:teuthology.orchestra.run.smithi043.stdout: "hostname": "smithi043", 2023-08-09T13:34:15.851 INFO:teuthology.orchestra.run.smithi043.stdout: "mount_point": "/home/ubuntu/cephtest/mnt.admin", 2023-08-09T13:34:15.851 INFO:teuthology.orchestra.run.smithi043.stdout: "pid": "94557", 2023-08-09T13:34:15.851 INFO:teuthology.orchestra.run.smithi043.stdout: "root": "/" 2023-08-09T13:34:15.851 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2023-08-09T13:34:15.851 INFO:teuthology.orchestra.run.smithi043.stdout: "dentry_count": 0, 2023-08-09T13:34:15.851 INFO:teuthology.orchestra.run.smithi043.stdout: "dentry_pinned_count": 0, 2023-08-09T13:34:15.852 INFO:teuthology.orchestra.run.smithi043.stdout: "id": 4410, 2023-08-09T13:34:15.852 INFO:teuthology.orchestra.run.smithi043.stdout: "inst": { 2023-08-09T13:34:15.852 INFO:teuthology.orchestra.run.smithi043.stdout: "name": { 2023-08-09T13:34:15.852 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "client", 2023-08-09T13:34:15.852 INFO:teuthology.orchestra.run.smithi043.stdout: "num": 4410 2023-08-09T13:34:15.852 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2023-08-09T13:34:15.852 INFO:teuthology.orchestra.run.smithi043.stdout: "addr": { 2023-08-09T13:34:15.853 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "v1", 2023-08-09T13:34:15.853 INFO:teuthology.orchestra.run.smithi043.stdout: "addr": "192.168.0.1:0", 2023-08-09T13:34:15.853 INFO:teuthology.orchestra.run.smithi043.stdout: "nonce": 1320564111 2023-08-09T13:34:15.853 INFO:teuthology.orchestra.run.smithi043.stdout: } 2023-08-09T13:34:15.853 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2023-08-09T13:34:15.853 INFO:teuthology.orchestra.run.smithi043.stdout: "addr": { 2023-08-09T13:34:15.853 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "v1", 2023-08-09T13:34:15.854 INFO:teuthology.orchestra.run.smithi043.stdout: "addr": "192.168.0.1:0", 2023-08-09T13:34:15.854 INFO:teuthology.orchestra.run.smithi043.stdout: "nonce": 1320564111 2023-08-09T13:34:15.854 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2023-08-09T13:34:15.854 INFO:teuthology.orchestra.run.smithi043.stdout: "inst_str": "client.4410 192.168.0.1:0/1320564111", 2023-08-09T13:34:15.854 INFO:teuthology.orchestra.run.smithi043.stdout: "addr_str": "192.168.0.1:0/1320564111", 2023-08-09T13:34:15.854 INFO:teuthology.orchestra.run.smithi043.stdout: "inode_count": 1, 2023-08-09T13:34:15.854 INFO:teuthology.orchestra.run.smithi043.stdout: "mds_epoch": 7, 2023-08-09T13:34:15.855 INFO:teuthology.orchestra.run.smithi043.stdout: "osd_epoch": 24, 2023-08-09T13:34:15.855 INFO:teuthology.orchestra.run.smithi043.stdout: "osd_epoch_barrier": 0, 2023-08-09T13:34:15.855 INFO:teuthology.orchestra.run.smithi043.stdout: "blocklisted": false, 2023-08-09T13:34:15.855 INFO:teuthology.orchestra.run.smithi043.stdout: "fs_name": "cephfs" 2023-08-09T13:34:15.855 INFO:teuthology.orchestra.run.smithi043.stdout:} 2023-08-09T13:34:15.864 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-09T13:34:15.865 DEBUG:teuthology.orchestra.run.smithi043:> stat --file-system '--printf=%T 2023-08-09T13:34:15.865 DEBUG:teuthology.orchestra.run.smithi043:> ' -- /home/ubuntu/cephtest/mnt.admin 2023-08-09T13:34:15.922 INFO:teuthology.orchestra.run.smithi043.stdout:fuseblk 2023-08-09T13:34:15.922 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.admin 2023-08-09T13:34:15.923 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-09T13:34:15.923 DEBUG:teuthology.orchestra.run.smithi043:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.admin 2023-08-09T13:34:16.028 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-09T13:34:16.028 DEBUG:teuthology.orchestra.run.smithi043:> (cd /home/ubuntu/cephtest/mnt.admin && exec bash -c 'setfattr -n ceph.dir.layout.pool -v cephfs_data_ec . && getfattr -n ceph.dir.layout .') 2023-08-09T13:34:16.136 INFO:teuthology.orchestra.run.smithi043.stdout:# file: . 2023-08-09T13:34:16.137 INFO:teuthology.orchestra.run.smithi043.stdout:ceph.dir.layout="stripe_unit=4194304 stripe_count=1 object_size=4194304 pool=cephfs_data_ec" 2023-08-09T13:34:16.137 INFO:teuthology.orchestra.run.smithi043.stdout: 2023-08-09T13:34:16.138 DEBUG:teuthology.orchestra.run.smithi043:> set -ex 2023-08-09T13:34:16.139 DEBUG:teuthology.orchestra.run.smithi043:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-09T13:34:16.199 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-08-09T13:34:16.431 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:16.428+0000 7fe351d4b640 1 Processor -- start 2023-08-09T13:34:16.431 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:16.428+0000 7fe351d4b640 1 -- start start 2023-08-09T13:34:16.433 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:16.430+0000 7fe351d4b640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7fe34c14a5c0 0x7fe34c160f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:34:16.433 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:16.430+0000 7fe351d4b640 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7fe34c0cc380 0x7fe34c06bb50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:34:16.433 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:16.430+0000 7fe351d4b640 1 -- --> v1:172.21.15.43:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe34c0c35e0 con 0x7fe34c0c3130 2023-08-09T13:34:16.433 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:16.430+0000 7fe351d4b640 1 -- --> v2:172.21.15.43:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe34c0c4420 con 0x7fe34c14a5c0 2023-08-09T13:34:16.433 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:16.430+0000 7fe351d4b640 1 -- --> v2:172.21.15.203:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe34c0c38a0 con 0x7fe34c0cc380 2023-08-09T13:34:16.435 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:16.431+0000 7fe34b7fe640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7fe34c14a5c0 0x7fe34c160f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:34:16.435 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:16.431+0000 7fe34affd640 1 --1- >> v1:172.21.15.43:6789/0 conn(0x7fe34c0c3130 0x7fe34c0cbc70 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.43:6789/0 says I am v1:172.21.15.43:48128/0 (socket says 172.21.15.43:48128) 2023-08-09T13:34:16.435 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:16.431+0000 7fe34affd640 1 -- 172.21.15.43:0/3667138533 learned_addr learned my addr 172.21.15.43:0/3667138533 (peer_addr_for_me v1:172.21.15.43:0/0) 2023-08-09T13:34:16.435 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:16.431+0000 7fe34bfff640 1 --2- 172.21.15.43:0/3667138533 >> v2:172.21.15.203:3300/0 conn(0x7fe34c0cc380 0x7fe34c06bb50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:34:16.435 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:16.431+0000 7fe34a7fc640 1 -- 172.21.15.43:0/3667138533 <== mon.0 v1:172.21.15.43:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 458153918 0 0) 0x7fe34c0c35e0 con 0x7fe34c0c3130 2023-08-09T13:34:16.435 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:16.431+0000 7fe34a7fc640 1 -- 172.21.15.43:0/3667138533 --> v1:172.21.15.43:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fe334003520 con 0x7fe34c0c3130 2023-08-09T13:34:16.436 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:16.431+0000 7fe34b7fe640 1 -- 172.21.15.43:0/3667138533 >> v2:172.21.15.203:3300/0 conn(0x7fe34c0cc380 msgr2=0x7fe34c06bb50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-09T13:34:16.436 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:16.431+0000 7fe34b7fe640 1 --2- 172.21.15.43:0/3667138533 >> v2:172.21.15.203:3300/0 conn(0x7fe34c0cc380 0x7fe34c06bb50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:34:16.436 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:16.431+0000 7fe34b7fe640 1 -- 172.21.15.43:0/3667138533 >> v1:172.21.15.43:6789/0 conn(0x7fe34c0c3130 legacy=0x7fe34c0cbc70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:34:16.436 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:16.431+0000 7fe34b7fe640 1 -- 172.21.15.43:0/3667138533 --> v2:172.21.15.43:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe34c0c3b80 con 0x7fe34c14a5c0 2023-08-09T13:34:16.436 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:16.431+0000 7fe34bfff640 1 --2- 172.21.15.43:0/3667138533 >> v2:172.21.15.203:3300/0 conn(0x7fe34c0cc380 0x7fe34c06bb50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-08-09T13:34:16.437 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:16.433+0000 7fe34b7fe640 1 --2- 172.21.15.43:0/3667138533 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fe34c14a5c0 0x7fe34c160f20 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7fe33c0097a0 tx=0x7fe33c02b840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=33dc1cda20649d82 server_cookie=0 in_seq=0 out_seq=0 2023-08-09T13:34:16.437 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:16.433+0000 7fe34a7fc640 1 -- 172.21.15.43:0/3667138533 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe33c02e070 con 0x7fe34c14a5c0 2023-08-09T13:34:16.437 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:16.433+0000 7fe34a7fc640 1 -- 172.21.15.43:0/3667138533 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe33c0315a0 con 0x7fe34c14a5c0 2023-08-09T13:34:16.437 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:16.433+0000 7fe34a7fc640 1 -- 172.21.15.43:0/3667138533 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe33c031910 con 0x7fe34c14a5c0 2023-08-09T13:34:16.438 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:16.434+0000 7fe351d4b640 1 -- 172.21.15.43:0/3667138533 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fe34c14a5c0 msgr2=0x7fe34c160f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:34:16.438 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:16.434+0000 7fe351d4b640 1 --2- 172.21.15.43:0/3667138533 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fe34c14a5c0 0x7fe34c160f20 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7fe33c0097a0 tx=0x7fe33c02b840 comp rx=0 tx=0).stop 2023-08-09T13:34:16.438 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:16.434+0000 7fe351d4b640 1 -- 172.21.15.43:0/3667138533 shutdown_connections 2023-08-09T13:34:16.438 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:16.434+0000 7fe351d4b640 1 --2- 172.21.15.43:0/3667138533 >> v2:172.21.15.203:3300/0 conn(0x7fe34c0cc380 0x7fe34c06bb50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:34:16.438 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:16.434+0000 7fe351d4b640 1 --2- 172.21.15.43:0/3667138533 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fe34c14a5c0 0x7fe34c160f20 unknown :-1 s=CLOSED pgs=95 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:34:16.438 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:16.434+0000 7fe351d4b640 1 -- 172.21.15.43:0/3667138533 >> 172.21.15.43:0/3667138533 conn(0x7fe34c0ce0a0 msgr2=0x7fe34c0ce480 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-09T13:34:16.438 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:16.434+0000 7fe351d4b640 1 -- 172.21.15.43:0/3667138533 shutdown_connections 2023-08-09T13:34:16.439 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:16.434+0000 7fe351d4b640 1 -- 172.21.15.43:0/3667138533 wait complete. 2023-08-09T13:34:16.439 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:16.436+0000 7fe351d4b640 1 Processor -- start 2023-08-09T13:34:16.440 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:16.436+0000 7fe351d4b640 1 -- start start 2023-08-09T13:34:16.440 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:16.437+0000 7fe351d4b640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fe34c0c3130 0x7fe34c144140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:34:16.440 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:16.437+0000 7fe351d4b640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fe34c0cc380 0x7fe34c145c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:34:16.440 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:16.437+0000 7fe351d4b640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fe34c14a5c0 0x7fe34c164ba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:34:16.441 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:16.437+0000 7fe351d4b640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe34c0c7c50 con 0x7fe34c0cc380 2023-08-09T13:34:16.441 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:16.437+0000 7fe351d4b640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe34c146b40 con 0x7fe34c0c3130 2023-08-09T13:34:16.441 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:16.437+0000 7fe351d4b640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe34c1462a0 con 0x7fe34c14a5c0 2023-08-09T13:34:16.441 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:16.437+0000 7fe34affd640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fe34c0cc380 0x7fe34c145c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:34:16.441 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:16.437+0000 7fe34affd640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fe34c0cc380 0x7fe34c145c80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:44624/0 (socket says 172.21.15.43:44624) 2023-08-09T13:34:16.441 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:16.437+0000 7fe34affd640 1 -- 172.21.15.43:0/3589736539 learned_addr learned my addr 172.21.15.43:0/3589736539 (peer_addr_for_me v2:172.21.15.43:0/0) 2023-08-09T13:34:16.441 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:16.437+0000 7fe34b7fe640 1 --2- 172.21.15.43:0/3589736539 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fe34c0c3130 0x7fe34c144140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:34:16.442 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:16.437+0000 7fe34bfff640 1 --2- 172.21.15.43:0/3589736539 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fe34c14a5c0 0x7fe34c164ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:34:16.443 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:16.437+0000 7fe34affd640 1 -- 172.21.15.43:0/3589736539 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fe34c14a5c0 msgr2=0x7fe34c164ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:34:16.443 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:16.437+0000 7fe34affd640 1 --2- 172.21.15.43:0/3589736539 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fe34c14a5c0 0x7fe34c164ba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:34:16.443 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:16.438+0000 7fe34affd640 1 -- 172.21.15.43:0/3589736539 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fe34c0c3130 msgr2=0x7fe34c144140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:34:16.443 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:16.438+0000 7fe34affd640 1 --2- 172.21.15.43:0/3589736539 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fe34c0c3130 0x7fe34c144140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:34:16.444 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:16.438+0000 7fe34affd640 1 -- 172.21.15.43:0/3589736539 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe33c009400 con 0x7fe34c0cc380 2023-08-09T13:34:16.444 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:16.438+0000 7fe34bfff640 1 --2- 172.21.15.43:0/3589736539 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fe34c14a5c0 0x7fe34c164ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-08-09T13:34:16.444 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:16.438+0000 7fe34b7fe640 1 --2- 172.21.15.43:0/3589736539 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fe34c0c3130 0x7fe34c144140 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-08-09T13:34:16.444 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:16.438+0000 7fe34affd640 1 --2- 172.21.15.43:0/3589736539 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fe34c0cc380 0x7fe34c145c80 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7fe34000c430 tx=0x7fe34000c8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-09T13:34:16.444 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:16.438+0000 7fe333fff640 1 -- 172.21.15.43:0/3589736539 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe340005ce0 con 0x7fe34c0cc380 2023-08-09T13:34:16.444 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:16.438+0000 7fe333fff640 1 -- 172.21.15.43:0/3589736539 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe340005e40 con 0x7fe34c0cc380 2023-08-09T13:34:16.444 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:16.438+0000 7fe333fff640 1 -- 172.21.15.43:0/3589736539 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe3400072f0 con 0x7fe34c0cc380 2023-08-09T13:34:16.445 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:16.438+0000 7fe351d4b640 1 -- 172.21.15.43:0/3589736539 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe34c146580 con 0x7fe34c0cc380 2023-08-09T13:34:16.445 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:16.438+0000 7fe351d4b640 1 -- 172.21.15.43:0/3589736539 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe34c162160 con 0x7fe34c0cc380 2023-08-09T13:34:16.445 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:16.440+0000 7fe351d4b640 1 -- 172.21.15.43:0/3589736539 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe34c162380 con 0x7fe34c0cc380 2023-08-09T13:34:16.445 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:16.441+0000 7fe333fff640 1 -- 172.21.15.43:0/3589736539 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 5) v1 ==== 88341+0+0 (secure 0 0 0) 0x7fe340007450 con 0x7fe34c0cc380 2023-08-09T13:34:16.445 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:16.442+0000 7fe333fff640 1 --2- 172.21.15.43:0/3589736539 >> [v2:172.21.15.203:6832/830097474,v1:172.21.15.203:6833/830097474] conn(0x7fe33806a050 0x7fe33806c510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:34:16.446 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:16.442+0000 7fe333fff640 1 -- 172.21.15.43:0/3589736539 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6813+0+0 (secure 0 0 0) 0x7fe340087f30 con 0x7fe34c0cc380 2023-08-09T13:34:16.446 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:16.442+0000 7fe34b7fe640 1 --2- 172.21.15.43:0/3589736539 >> [v2:172.21.15.203:6832/830097474,v1:172.21.15.203:6833/830097474] conn(0x7fe33806a050 0x7fe33806c510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:34:16.446 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:16.444+0000 7fe34b7fe640 1 --2- 172.21.15.43:0/3589736539 >> [v2:172.21.15.203:6832/830097474,v1:172.21.15.203:6833/830097474] conn(0x7fe33806a050 0x7fe33806c510 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7fe33c036000 tx=0x7fe33c03d040 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-09T13:34:16.450 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:16.448+0000 7fe333fff640 1 -- 172.21.15.43:0/3589736539 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178066 (secure 0 0 0) 0x7fe340056db0 con 0x7fe34c0cc380 2023-08-09T13:34:16.645 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:16.642+0000 7fe351d4b640 1 -- 172.21.15.43:0/3589736539 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fe34c0c4110 con 0x7fe34c0cc380 2023-08-09T13:34:16.647 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:16.645+0000 7fe333fff640 1 -- 172.21.15.43:0/3589736539 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v24) v1 ==== 74+0+20050 (secure 0 0 0) 0x7fe340017090 con 0x7fe34c0cc380 2023-08-09T13:34:16.648 INFO:teuthology.orchestra.run.smithi043.stdout: 2023-08-09T13:34:16.648 INFO:teuthology.orchestra.run.smithi043.stdout:{"epoch":24,"fsid":"daac87ee-1337-4250-a955-aea4e408a39b","created":"2023-08-09T13:33:41.701198+0000","modified":"2023-08-09T13:34:07.709429+0000","last_up_change":"2023-08-09T13:33:53.783453+0000","last_in_change":"2023-08-09T13:33:45.148904+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":5,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":"rbd","create_time":"2023-08-09T13:33:54.365137+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":".mgr","create_time":"2023-08-09T13:33:54.831803+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_metadata","create_time":"2023-08-09T13:34:00.207974+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data","create_time":"2023-08-09T13:34:01.614765+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":5,"pool_name":"cephfs_data_ec","create_time":"2023-08-09T13:34:05.239583+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"24","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"cbd49ecd-e896-4fbd-864e-5c37a829b165","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6800","nonce":502667335},{"type":"v1","addr":"172.21.15.43:6801","nonce":502667335}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6802","nonce":502667335},{"type":"v1","addr":"172.21.15.43:6803","nonce":502667335}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6806","nonce":502667335},{"type":"v1","addr":"172.21.15.43:6807","nonce":502667335}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6804","nonce":502667335},{"type":"v1","addr":"172.21.15.43:6805","nonce":502667335}]},"public_addr":"172.21.15.43:6801/502667335","cluster_addr":"172.21.15.43:6803/502667335","heartbeat_back_addr":"172.21.15.43:6807/502667335","heartbeat_front_addr":"172.21.15.43:6805/502667335","state":["exists","up"]},{"osd":1,"uuid":"75c7e0fe-1ec7-4fe8-a2f2-3661a65b8500","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6808","nonce":3289134610},{"type":"v1","addr":"172.21.15.43:6809","nonce":3289134610}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6810","nonce":3289134610},{"type":"v1","addr":"172.21.15.43:6812","nonce":3289134610}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6821","nonce":3289134610},{"type":"v1","addr":"172.21.15.43:6824","nonce":3289134610}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6815","nonce":3289134610},{"type":"v1","addr":"172.21.15.43:6818","nonce":3289134610}]},"public_addr":"172.21.15.43:6809/3289134610","cluster_addr":"172.21.15.43:6812/3289134610","heartbeat_back_addr":"172.21.15.43:6824/3289134610","heartbeat_front_addr":"172.21.15.43:6818/3289134610","state":["exists","up"]},{"osd":2,"uuid":"bb5c41e9-7f5f-4125-9b01-3bb8bfc2ea83","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6811","nonce":4079709736},{"type":"v1","addr":"172.21.15.43:6814","nonce":4079709736}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6817","nonce":4079709736},{"type":"v1","addr":"172.21.15.43:6820","nonce":4079709736}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6829","nonce":4079709736},{"type":"v1","addr":"172.21.15.43:6831","nonce":4079709736}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6823","nonce":4079709736},{"type":"v1","addr":"172.21.15.43:6826","nonce":4079709736}]},"public_addr":"172.21.15.43:6814/4079709736","cluster_addr":"172.21.15.43:6820/4079709736","heartbeat_back_addr":"172.21.15.43:6831/4079709736","heartbeat_front_addr":"172.21.15.43:6826/4079709736","state":["exists","up"]},{"osd":3,"uuid":"bfe4c8b4-694a-465c-87e6-2b2aab852613","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6813","nonce":1709538759},{"type":"v1","addr":"172.21.15.43:6816","nonce":1709538759}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6819","nonce":1709538759},{"type":"v1","addr":"172.21.15.43:6822","nonce":1709538759}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6828","nonce":1709538759},{"type":"v1","addr":"172.21.15.43:6830","nonce":1709538759}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6825","nonce":1709538759},{"type":"v1","addr":"172.21.15.43:6827","nonce":1709538759}]},"public_addr":"172.21.15.43:6816/1709538759","cluster_addr":"172.21.15.43:6822/1709538759","heartbeat_back_addr":"172.21.15.43:6830/1709538759","heartbeat_front_addr":"172.21.15.43:6827/1709538759","state":["exists","up"]},{"osd":4,"uuid":"fa2e8a77-aba1-4756-853e-0d174a5bdeba","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6800","nonce":4196866481},{"type":"v1","addr":"172.21.15.203:6801","nonce":4196866481}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6802","nonce":4196866481},{"type":"v1","addr":"172.21.15.203:6803","nonce":4196866481}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6806","nonce":4196866481},{"type":"v1","addr":"172.21.15.203:6807","nonce":4196866481}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6804","nonce":4196866481},{"type":"v1","addr":"172.21.15.203:6805","nonce":4196866481}]},"public_addr":"172.21.15.203:6801/4196866481","cluster_addr":"172.21.15.203:6803/4196866481","heartbeat_back_addr":"172.21.15.203:6807/4196866481","heartbeat_front_addr":"172.21.15.203:6805/4196866481","state":["exists","up"]},{"osd":5,"uuid":"81811e4b-4d24-4f51-a82c-f1d700a2a00d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6816","nonce":3059290081},{"type":"v1","addr":"172.21.15.203:6817","nonce":3059290081}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6818","nonce":3059290081},{"type":"v1","addr":"172.21.15.203:6819","nonce":3059290081}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6822","nonce":3059290081},{"type":"v1","addr":"172.21.15.203:6823","nonce":3059290081}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6820","nonce":3059290081},{"type":"v1","addr":"172.21.15.203:6821","nonce":3059290081}]},"public_addr":"172.21.15.203:6817/3059290081","cluster_addr":"172.21.15.203:6819/3059290081","heartbeat_back_addr":"172.21.15.203:6823/3059290081","heartbeat_front_addr":"172.21.15.203:6821/3059290081","state":["exists","up"]},{"osd":6,"uuid":"307bac4d-6a5b-443d-af75-56d695d7674f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6824","nonce":512155569},{"type":"v1","addr":"172.21.15.203:6825","nonce":512155569}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6826","nonce":512155569},{"type":"v1","addr":"172.21.15.203:6827","nonce":512155569}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6830","nonce":512155569},{"type":"v1","addr":"172.21.15.203:6831","nonce":512155569}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6828","nonce":512155569},{"type":"v1","addr":"172.21.15.203:6829","nonce":512155569}]},"public_addr":"172.21.15.203:6825/512155569","cluster_addr":"172.21.15.203:6827/512155569","heartbeat_back_addr":"172.21.15.203:6831/512155569","heartbeat_front_addr":"172.21.15.203:6829/512155569","state":["exists","up"]},{"osd":7,"uuid":"a95dc7ca-9f1b-4295-af70-68a22a2d7d5e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6808","nonce":3331450438},{"type":"v1","addr":"172.21.15.203:6809","nonce":3331450438}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6810","nonce":3331450438},{"type":"v1","addr":"172.21.15.203:6811","nonce":3331450438}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6814","nonce":3331450438},{"type":"v1","addr":"172.21.15.203:6815","nonce":3331450438}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6812","nonce":3331450438},{"type":"v1","addr":"172.21.15.203:6813","nonce":3331450438}]},"public_addr":"172.21.15.203:6809/3331450438","cluster_addr":"172.21.15.203:6811/3331450438","heartbeat_back_addr":"172.21.15.203:6815/3331450438","heartbeat_front_addr":"172.21.15.203:6813/3331450438","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-09T13:33:50.547652+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-09T13:33:50.589469+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-09T13:33:50.561724+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-09T13:33:50.599650+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-09T13:33:50.411855+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-09T13:33:50.219681+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-09T13:33:50.504664+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-09T13:33:50.441248+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-08-09T13:34:16.651 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:16.648+0000 7fe351d4b640 1 -- 172.21.15.43:0/3589736539 >> [v2:172.21.15.203:6832/830097474,v1:172.21.15.203:6833/830097474] conn(0x7fe33806a050 msgr2=0x7fe33806c510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:34:16.651 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:16.648+0000 7fe351d4b640 1 --2- 172.21.15.43:0/3589736539 >> [v2:172.21.15.203:6832/830097474,v1:172.21.15.203:6833/830097474] conn(0x7fe33806a050 0x7fe33806c510 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7fe33c036000 tx=0x7fe33c03d040 comp rx=0 tx=0).stop 2023-08-09T13:34:16.651 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:16.648+0000 7fe351d4b640 1 -- 172.21.15.43:0/3589736539 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fe34c0cc380 msgr2=0x7fe34c145c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:34:16.651 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:16.648+0000 7fe351d4b640 1 --2- 172.21.15.43:0/3589736539 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fe34c0cc380 0x7fe34c145c80 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7fe34000c430 tx=0x7fe34000c8f0 comp rx=0 tx=0).stop 2023-08-09T13:34:16.651 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:16.648+0000 7fe351d4b640 1 -- 172.21.15.43:0/3589736539 shutdown_connections 2023-08-09T13:34:16.651 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:16.648+0000 7fe351d4b640 1 --2- 172.21.15.43:0/3589736539 >> [v2:172.21.15.203:6832/830097474,v1:172.21.15.203:6833/830097474] conn(0x7fe33806a050 0x7fe33806c510 unknown :-1 s=CLOSED pgs=73 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:34:16.652 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:16.648+0000 7fe351d4b640 1 --2- 172.21.15.43:0/3589736539 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fe34c14a5c0 0x7fe34c164ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:34:16.652 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:16.648+0000 7fe351d4b640 1 --2- 172.21.15.43:0/3589736539 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fe34c0cc380 0x7fe34c145c80 unknown :-1 s=CLOSED pgs=96 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:34:16.652 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:16.648+0000 7fe351d4b640 1 --2- 172.21.15.43:0/3589736539 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fe34c0c3130 0x7fe34c144140 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:34:16.652 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:16.648+0000 7fe351d4b640 1 -- 172.21.15.43:0/3589736539 >> 172.21.15.43:0/3589736539 conn(0x7fe34c0ce0a0 msgr2=0x7fe34c06c420 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-09T13:34:16.652 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:16.648+0000 7fe351d4b640 1 -- 172.21.15.43:0/3589736539 shutdown_connections 2023-08-09T13:34:16.652 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:16.648+0000 7fe351d4b640 1 -- 172.21.15.43:0/3589736539 wait complete. 2023-08-09T13:34:16.669 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': 'rbd', 'create_time': '2023-08-09T13:33:54.365137+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}, 'read_balance': {'score_acting': 2, 'score_stable': 2, 'optimal_score': 1, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': '.mgr', 'create_time': '2023-08-09T13:33:54.831803+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_metadata', 'create_time': '2023-08-09T13:34:00.207974+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data', 'create_time': '2023-08-09T13:34:01.614765+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 2, 'score_stable': 2, 'optimal_score': 1, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 5, 'pool_name': 'cephfs_data_ec', 'create_time': '2023-08-09T13:34:05.239583+0000', 'flags': 5, 'flags_names': 'hashpspool,ec_overwrites', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '24', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2023-08-09T13:34:16.670 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get rbd pg_num 2023-08-09T13:34:16.899 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:16.897+0000 7f7da2a8a640 1 Processor -- start 2023-08-09T13:34:16.900 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:16.897+0000 7f7da2a8a640 1 -- start start 2023-08-09T13:34:16.901 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:16.899+0000 7f7da2a8a640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f7d9c0cbde0 0x7f7d9c06fa20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:34:16.901 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:16.899+0000 7f7da2a8a640 1 -- --> v1:172.21.15.43:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7d9c0acfa0 con 0x7f7d9c0cb9a0 2023-08-09T13:34:16.902 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:16.899+0000 7f7da2a8a640 1 -- --> v1:172.21.15.203:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7d9c15c940 con 0x7f7d9c070ef0 2023-08-09T13:34:16.902 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:16.899+0000 7f7da2a8a640 1 -- --> v2:172.21.15.43:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7d9c0ac390 con 0x7f7d9c0cbde0 2023-08-09T13:34:16.902 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:16.899+0000 7f7da2289640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f7d9c0cbde0 0x7f7d9c06fa20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:34:16.902 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:16.899+0000 7f7da2289640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f7d9c0cbde0 0x7f7d9c06fa20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:44632/0 (socket says 172.21.15.43:44632) 2023-08-09T13:34:16.902 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:16.899+0000 7f7da2289640 1 -- 172.21.15.43:0/3575236520 learned_addr learned my addr 172.21.15.43:0/3575236520 (peer_addr_for_me v2:172.21.15.43:0/0) 2023-08-09T13:34:16.903 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:16.899+0000 7f7da2289640 1 -- 172.21.15.43:0/3575236520 >> v1:172.21.15.203:6789/0 conn(0x7f7d9c070ef0 legacy=0x7f7d9c0ab8d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:34:16.903 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:16.899+0000 7f7da2289640 1 -- 172.21.15.43:0/3575236520 >> v1:172.21.15.43:6789/0 conn(0x7f7d9c0cb9a0 legacy=0x7f7d9c06f310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:34:16.903 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:16.899+0000 7f7da2289640 1 -- 172.21.15.43:0/3575236520 --> v2:172.21.15.43:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7d9c0ac670 con 0x7f7d9c0cbde0 2023-08-09T13:34:16.903 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:16.901+0000 7f7da2289640 1 --2- 172.21.15.43:0/3575236520 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f7d9c0cbde0 0x7f7d9c06fa20 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7f7d94009e50 tx=0x7f7d9402c090 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6587b7bacc413bcc server_cookie=0 in_seq=0 out_seq=0 2023-08-09T13:34:16.904 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:16.901+0000 7f7da0a86640 1 -- 172.21.15.43:0/3575236520 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7d9402cc30 con 0x7f7d9c0cbde0 2023-08-09T13:34:16.904 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:16.901+0000 7f7da0a86640 1 -- 172.21.15.43:0/3575236520 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7d9402cd90 con 0x7f7d9c0cbde0 2023-08-09T13:34:16.904 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:16.901+0000 7f7da0a86640 1 -- 172.21.15.43:0/3575236520 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7d94034610 con 0x7f7d9c0cbde0 2023-08-09T13:34:16.904 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:16.902+0000 7f7da2a8a640 1 -- 172.21.15.43:0/3575236520 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f7d9c0cbde0 msgr2=0x7f7d9c06fa20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:34:16.904 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:16.902+0000 7f7da2a8a640 1 --2- 172.21.15.43:0/3575236520 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f7d9c0cbde0 0x7f7d9c06fa20 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7f7d94009e50 tx=0x7f7d9402c090 comp rx=0 tx=0).stop 2023-08-09T13:34:16.905 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:16.902+0000 7f7da2a8a640 1 -- 172.21.15.43:0/3575236520 shutdown_connections 2023-08-09T13:34:16.905 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:16.902+0000 7f7da2a8a640 1 --2- 172.21.15.43:0/3575236520 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f7d9c0cbde0 0x7f7d9c06fa20 unknown :-1 s=CLOSED pgs=98 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:34:16.905 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:16.902+0000 7f7da2a8a640 1 -- 172.21.15.43:0/3575236520 >> 172.21.15.43:0/3575236520 conn(0x7f7d9c0cddd0 msgr2=0x7f7d9c0ce1e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-09T13:34:16.906 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:16.902+0000 7f7da2a8a640 1 -- 172.21.15.43:0/3575236520 shutdown_connections 2023-08-09T13:34:16.906 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:16.902+0000 7f7da2a8a640 1 -- 172.21.15.43:0/3575236520 wait complete. 2023-08-09T13:34:16.906 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:16.904+0000 7f7da2a8a640 1 Processor -- start 2023-08-09T13:34:16.906 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:16.904+0000 7f7da2a8a640 1 -- start start 2023-08-09T13:34:16.908 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:16.905+0000 7f7da2a8a640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f7d9c070ef0 0x7f7d9c157020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:34:16.908 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:16.905+0000 7f7da2a8a640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f7d9c0cb9a0 0x7f7d9c15ee00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:34:16.908 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:16.905+0000 7f7da2a8a640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f7d9c0cbde0 0x7f7d9c15f340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:34:16.908 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:16.905+0000 7f7da2a8a640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7d9c15ce60 con 0x7f7d9c0cb9a0 2023-08-09T13:34:16.908 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:16.905+0000 7f7da2a8a640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7d9c144500 con 0x7f7d9c070ef0 2023-08-09T13:34:16.909 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:16.905+0000 7f7da2a8a640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7d9c0b2290 con 0x7f7d9c0cbde0 2023-08-09T13:34:16.909 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:16.905+0000 7f7da1287640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f7d9c0cb9a0 0x7f7d9c15ee00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:34:16.910 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:16.906+0000 7f7da1287640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f7d9c0cb9a0 0x7f7d9c15ee00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:44642/0 (socket says 172.21.15.43:44642) 2023-08-09T13:34:16.910 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:16.906+0000 7f7da1287640 1 -- 172.21.15.43:0/1279101542 learned_addr learned my addr 172.21.15.43:0/1279101542 (peer_addr_for_me v2:172.21.15.43:0/0) 2023-08-09T13:34:16.910 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:16.906+0000 7f7da2289640 1 --2- 172.21.15.43:0/1279101542 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f7d9c0cbde0 0x7f7d9c15f340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:34:16.910 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:16.906+0000 7f7da1a88640 1 --2- 172.21.15.43:0/1279101542 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f7d9c070ef0 0x7f7d9c157020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:34:16.911 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:16.906+0000 7f7da1287640 1 -- 172.21.15.43:0/1279101542 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f7d9c0cbde0 msgr2=0x7f7d9c15f340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:34:16.911 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:16.906+0000 7f7da1287640 1 --2- 172.21.15.43:0/1279101542 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f7d9c0cbde0 0x7f7d9c15f340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:34:16.911 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:16.906+0000 7f7da1287640 1 -- 172.21.15.43:0/1279101542 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f7d9c070ef0 msgr2=0x7f7d9c157020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:34:16.911 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:16.906+0000 7f7da1287640 1 --2- 172.21.15.43:0/1279101542 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f7d9c070ef0 0x7f7d9c157020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:34:16.911 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:16.906+0000 7f7da1287640 1 -- 172.21.15.43:0/1279101542 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7d94009b30 con 0x7f7d9c0cb9a0 2023-08-09T13:34:16.911 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:16.906+0000 7f7da1a88640 1 --2- 172.21.15.43:0/1279101542 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f7d9c070ef0 0x7f7d9c157020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-08-09T13:34:16.911 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:16.906+0000 7f7da1287640 1 --2- 172.21.15.43:0/1279101542 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f7d9c0cb9a0 0x7f7d9c15ee00 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7f7d8c00c450 tx=0x7f7d8c00c910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-09T13:34:16.911 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:16.907+0000 7f7d927fc640 1 -- 172.21.15.43:0/1279101542 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7d8c0052d0 con 0x7f7d9c0cb9a0 2023-08-09T13:34:16.912 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:16.907+0000 7f7d927fc640 1 -- 172.21.15.43:0/1279101542 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7d8c005430 con 0x7f7d9c0cb9a0 2023-08-09T13:34:16.912 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:16.907+0000 7f7d927fc640 1 -- 172.21.15.43:0/1279101542 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7d8c012870 con 0x7f7d9c0cb9a0 2023-08-09T13:34:16.912 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:16.907+0000 7f7da2a8a640 1 -- 172.21.15.43:0/1279101542 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7d9c0b2550 con 0x7f7d9c0cb9a0 2023-08-09T13:34:16.912 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:16.907+0000 7f7da2a8a640 1 -- 172.21.15.43:0/1279101542 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7d9c0b2b10 con 0x7f7d9c0cb9a0 2023-08-09T13:34:16.913 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:16.908+0000 7f7da2a8a640 1 -- 172.21.15.43:0/1279101542 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7d9c1446c0 con 0x7f7d9c0cb9a0 2023-08-09T13:34:16.913 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:16.909+0000 7f7d927fc640 1 -- 172.21.15.43:0/1279101542 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 5) v1 ==== 88341+0+0 (secure 0 0 0) 0x7f7d8c0129d0 con 0x7f7d9c0cb9a0 2023-08-09T13:34:16.913 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:16.910+0000 7f7d927fc640 1 --2- 172.21.15.43:0/1279101542 >> [v2:172.21.15.203:6832/830097474,v1:172.21.15.203:6833/830097474] conn(0x7f7d8006a050 0x7f7d8006c510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:34:16.913 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:16.910+0000 7f7d927fc640 1 -- 172.21.15.43:0/1279101542 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6813+0+0 (secure 0 0 0) 0x7f7d8c0883e0 con 0x7f7d9c0cb9a0 2023-08-09T13:34:16.914 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:16.911+0000 7f7da1a88640 1 --2- 172.21.15.43:0/1279101542 >> [v2:172.21.15.203:6832/830097474,v1:172.21.15.203:6833/830097474] conn(0x7f7d8006a050 0x7f7d8006c510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:34:16.914 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:16.912+0000 7f7da1a88640 1 --2- 172.21.15.43:0/1279101542 >> [v2:172.21.15.203:6832/830097474,v1:172.21.15.203:6833/830097474] conn(0x7f7d8006a050 0x7f7d8006c510 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f7d98007fd0 tx=0x7f7d98009040 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-09T13:34:16.917 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:16.915+0000 7f7d927fc640 1 -- 172.21.15.43:0/1279101542 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178066 (secure 0 0 0) 0x7f7d8c053740 con 0x7f7d9c0cb9a0 2023-08-09T13:34:17.110 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:17.107+0000 7f7da2a8a640 1 -- 172.21.15.43:0/1279101542 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"} v 0) v1 -- 0x7f7d9c04e390 con 0x7f7d9c0cb9a0 2023-08-09T13:34:17.110 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:17.108+0000 7f7d927fc640 1 -- 172.21.15.43:0/1279101542 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"}]=0 v24) v1 ==== 92+0+10 (secure 0 0 0) 0x7f7d8c056d60 con 0x7f7d9c0cb9a0 2023-08-09T13:34:17.111 INFO:teuthology.orchestra.run.smithi043.stdout:pg_num: 8 2023-08-09T13:34:17.113 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:17.110+0000 7f7da2a8a640 1 -- 172.21.15.43:0/1279101542 >> [v2:172.21.15.203:6832/830097474,v1:172.21.15.203:6833/830097474] conn(0x7f7d8006a050 msgr2=0x7f7d8006c510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:34:17.113 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:17.110+0000 7f7da2a8a640 1 --2- 172.21.15.43:0/1279101542 >> [v2:172.21.15.203:6832/830097474,v1:172.21.15.203:6833/830097474] conn(0x7f7d8006a050 0x7f7d8006c510 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f7d98007fd0 tx=0x7f7d98009040 comp rx=0 tx=0).stop 2023-08-09T13:34:17.113 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:17.110+0000 7f7da2a8a640 1 -- 172.21.15.43:0/1279101542 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f7d9c0cb9a0 msgr2=0x7f7d9c15ee00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:34:17.114 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:17.110+0000 7f7da2a8a640 1 --2- 172.21.15.43:0/1279101542 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f7d9c0cb9a0 0x7f7d9c15ee00 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7f7d8c00c450 tx=0x7f7d8c00c910 comp rx=0 tx=0).stop 2023-08-09T13:34:17.114 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:17.110+0000 7f7da2a8a640 1 -- 172.21.15.43:0/1279101542 shutdown_connections 2023-08-09T13:34:17.114 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:17.110+0000 7f7da2a8a640 1 --2- 172.21.15.43:0/1279101542 >> [v2:172.21.15.203:6832/830097474,v1:172.21.15.203:6833/830097474] conn(0x7f7d8006a050 0x7f7d8006c510 unknown :-1 s=CLOSED pgs=74 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:34:17.114 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:17.110+0000 7f7da2a8a640 1 --2- 172.21.15.43:0/1279101542 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f7d9c0cbde0 0x7f7d9c15f340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:34:17.114 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:17.110+0000 7f7da2a8a640 1 --2- 172.21.15.43:0/1279101542 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f7d9c0cb9a0 0x7f7d9c15ee00 unknown :-1 s=CLOSED pgs=99 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:34:17.115 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:17.110+0000 7f7da2a8a640 1 --2- 172.21.15.43:0/1279101542 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f7d9c070ef0 0x7f7d9c157020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:34:17.115 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:17.110+0000 7f7da2a8a640 1 -- 172.21.15.43:0/1279101542 >> 172.21.15.43:0/1279101542 conn(0x7f7d9c0cddd0 msgr2=0x7f7d9c0ae390 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-09T13:34:17.115 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:17.111+0000 7f7da2a8a640 1 -- 172.21.15.43:0/1279101542 shutdown_connections 2023-08-09T13:34:17.115 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:17.111+0000 7f7da2a8a640 1 -- 172.21.15.43:0/1279101542 wait complete. 2023-08-09T13:34:17.132 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-08-09T13:34:17.364 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:17.361+0000 7f217dc70640 1 Processor -- start 2023-08-09T13:34:17.365 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:17.361+0000 7f217dc70640 1 -- start start 2023-08-09T13:34:17.366 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:17.363+0000 7f217dc70640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f217815c2d0 0x7f21780b7df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:34:17.366 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:17.363+0000 7f217dc70640 1 -- --> v1:172.21.15.43:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f21780d16a0 con 0x7f21780d1b30 2023-08-09T13:34:17.366 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:17.363+0000 7f217dc70640 1 -- --> v1:172.21.15.203:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f21780d1fa0 con 0x7f21780cebf0 2023-08-09T13:34:17.366 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:17.363+0000 7f217dc70640 1 -- --> v2:172.21.15.43:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f21780d0d70 con 0x7f217815c2d0 2023-08-09T13:34:17.366 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:17.363+0000 7f217cc6e640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f217815c2d0 0x7f21780b7df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:34:17.367 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:17.364+0000 7f217cc6e640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f217815c2d0 0x7f21780b7df0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:44658/0 (socket says 172.21.15.43:44658) 2023-08-09T13:34:17.367 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:17.364+0000 7f217cc6e640 1 -- 172.21.15.43:0/632838327 learned_addr learned my addr 172.21.15.43:0/632838327 (peer_addr_for_me v2:172.21.15.43:0/0) 2023-08-09T13:34:17.367 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:17.364+0000 7f217cc6e640 1 -- 172.21.15.43:0/632838327 >> v1:172.21.15.203:6789/0 conn(0x7f21780cebf0 legacy=0x7f21780b5270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-09T13:34:17.368 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:17.364+0000 7f217cc6e640 1 -- 172.21.15.43:0/632838327 >> v1:172.21.15.43:6789/0 conn(0x7f21780d1b30 legacy=0x7f21780cd020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:34:17.368 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:17.364+0000 7f217cc6e640 1 -- 172.21.15.43:0/632838327 --> v2:172.21.15.43:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f21780d1050 con 0x7f217815c2d0 2023-08-09T13:34:17.369 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:17.365+0000 7f217cc6e640 1 --2- 172.21.15.43:0/632838327 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f217815c2d0 0x7f21780b7df0 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7f216c009870 tx=0x7f216c02b9a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=875c2e004c31a263 server_cookie=0 in_seq=0 out_seq=0 2023-08-09T13:34:17.369 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:17.366+0000 7f21777fe640 1 -- 172.21.15.43:0/632838327 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f216c030860 con 0x7f217815c2d0 2023-08-09T13:34:17.369 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:17.366+0000 7f21777fe640 1 -- 172.21.15.43:0/632838327 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f216c0309c0 con 0x7f217815c2d0 2023-08-09T13:34:17.369 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:17.366+0000 7f21777fe640 1 -- 172.21.15.43:0/632838327 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f216c030d50 con 0x7f217815c2d0 2023-08-09T13:34:17.369 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:17.366+0000 7f217dc70640 1 -- 172.21.15.43:0/632838327 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f217815c2d0 msgr2=0x7f21780b7df0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:34:17.370 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:17.366+0000 7f217dc70640 1 --2- 172.21.15.43:0/632838327 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f217815c2d0 0x7f21780b7df0 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7f216c009870 tx=0x7f216c02b9a0 comp rx=0 tx=0).stop 2023-08-09T13:34:17.370 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:17.366+0000 7f217dc70640 1 -- 172.21.15.43:0/632838327 shutdown_connections 2023-08-09T13:34:17.370 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:17.366+0000 7f217dc70640 1 --2- 172.21.15.43:0/632838327 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f217815c2d0 0x7f21780b7df0 unknown :-1 s=CLOSED pgs=101 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:34:17.370 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:17.366+0000 7f217dc70640 1 -- 172.21.15.43:0/632838327 >> 172.21.15.43:0/632838327 conn(0x7f217806a230 msgr2=0x7f217806a640 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-09T13:34:17.370 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:17.366+0000 7f217dc70640 1 -- 172.21.15.43:0/632838327 shutdown_connections 2023-08-09T13:34:17.370 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:17.366+0000 7f217dc70640 1 -- 172.21.15.43:0/632838327 wait complete. 2023-08-09T13:34:17.371 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:17.368+0000 7f217dc70640 1 Processor -- start 2023-08-09T13:34:17.371 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:17.368+0000 7f217dc70640 1 -- start start 2023-08-09T13:34:17.372 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:17.369+0000 7f217dc70640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f21780cebf0 0x7f21780b2710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:34:17.373 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:17.369+0000 7f217dc70640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f21780d1b30 0x7f2178153a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:34:17.373 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:17.369+0000 7f217dc70640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f217815c2d0 0x7f21780b3b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:34:17.373 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:17.369+0000 7f217dc70640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f21780c9ea0 con 0x7f21780cebf0 2023-08-09T13:34:17.373 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:17.369+0000 7f217dc70640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f21780c92b0 con 0x7f217815c2d0 2023-08-09T13:34:17.373 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:17.369+0000 7f217dc70640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f21780c9500 con 0x7f21780d1b30 2023-08-09T13:34:17.373 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:17.370+0000 7f217cc6e640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f21780cebf0 0x7f21780b2710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:34:17.374 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:17.370+0000 7f217cc6e640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f21780cebf0 0x7f21780b2710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:44660/0 (socket says 172.21.15.43:44660) 2023-08-09T13:34:17.374 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:17.370+0000 7f217cc6e640 1 -- 172.21.15.43:0/841225791 learned_addr learned my addr 172.21.15.43:0/841225791 (peer_addr_for_me v2:172.21.15.43:0/0) 2023-08-09T13:34:17.374 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:17.370+0000 7f2177fff640 1 --2- 172.21.15.43:0/841225791 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f21780d1b30 0x7f2178153a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:34:17.374 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:17.370+0000 7f217d46f640 1 --2- 172.21.15.43:0/841225791 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f217815c2d0 0x7f21780b3b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:34:17.374 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:17.370+0000 7f217cc6e640 1 -- 172.21.15.43:0/841225791 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f21780d1b30 msgr2=0x7f2178153a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:34:17.374 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:17.370+0000 7f217cc6e640 1 --2- 172.21.15.43:0/841225791 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f21780d1b30 0x7f2178153a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:34:17.374 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:17.370+0000 7f217cc6e640 1 -- 172.21.15.43:0/841225791 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f217815c2d0 msgr2=0x7f21780b3b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:34:17.375 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:17.370+0000 7f217cc6e640 1 --2- 172.21.15.43:0/841225791 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f217815c2d0 0x7f21780b3b60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:34:17.375 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:17.370+0000 7f217cc6e640 1 -- 172.21.15.43:0/841225791 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f216c0094d0 con 0x7f21780cebf0 2023-08-09T13:34:17.376 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:17.370+0000 7f217cc6e640 1 --2- 172.21.15.43:0/841225791 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f21780cebf0 0x7f21780b2710 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7f216c009870 tx=0x7f216c004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-09T13:34:17.376 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:17.371+0000 7f21757fa640 1 -- 172.21.15.43:0/841225791 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f216c030810 con 0x7f21780cebf0 2023-08-09T13:34:17.376 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:17.371+0000 7f21757fa640 1 -- 172.21.15.43:0/841225791 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f216c039d90 con 0x7f21780cebf0 2023-08-09T13:34:17.376 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:17.371+0000 7f217dc70640 1 -- 172.21.15.43:0/841225791 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f21780c97e0 con 0x7f21780cebf0 2023-08-09T13:34:17.376 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:17.371+0000 7f21757fa640 1 -- 172.21.15.43:0/841225791 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f216c043630 con 0x7f21780cebf0 2023-08-09T13:34:17.376 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:17.371+0000 7f217dc70640 1 -- 172.21.15.43:0/841225791 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f21780b40a0 con 0x7f21780cebf0 2023-08-09T13:34:17.376 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:17.372+0000 7f217dc70640 1 -- 172.21.15.43:0/841225791 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f217804e390 con 0x7f21780cebf0 2023-08-09T13:34:17.377 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:17.373+0000 7f21757fa640 1 -- 172.21.15.43:0/841225791 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 5) v1 ==== 88341+0+0 (secure 0 0 0) 0x7f216c04c3b0 con 0x7f21780cebf0 2023-08-09T13:34:17.377 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:17.374+0000 7f21757fa640 1 --2- 172.21.15.43:0/841225791 >> [v2:172.21.15.203:6832/830097474,v1:172.21.15.203:6833/830097474] conn(0x7f215c06a050 0x7f215c06c510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:34:17.377 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:17.374+0000 7f21757fa640 1 -- 172.21.15.43:0/841225791 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6813+0+0 (secure 0 0 0) 0x7f216c0aafd0 con 0x7f21780cebf0 2023-08-09T13:34:17.378 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:17.375+0000 7f2177fff640 1 --2- 172.21.15.43:0/841225791 >> [v2:172.21.15.203:6832/830097474,v1:172.21.15.203:6833/830097474] conn(0x7f215c06a050 0x7f215c06c510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:34:17.378 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:17.376+0000 7f2177fff640 1 --2- 172.21.15.43:0/841225791 >> [v2:172.21.15.203:6832/830097474,v1:172.21.15.203:6833/830097474] conn(0x7f215c06a050 0x7f215c06c510 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f21780cb660 tx=0x7f2164009040 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-09T13:34:17.381 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:17.379+0000 7f21757fa640 1 -- 172.21.15.43:0/841225791 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178066 (secure 0 0 0) 0x7f216c0763c0 con 0x7f21780cebf0 2023-08-09T13:34:17.578 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:17.575+0000 7f217dc70640 1 -- 172.21.15.43:0/841225791 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f21780b71e0 con 0x7f21780cebf0 2023-08-09T13:34:17.579 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:17.576+0000 7f21757fa640 1 -- 172.21.15.43:0/841225791 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v24) v1 ==== 93+0+10 (secure 0 0 0) 0x7f216c0340a0 con 0x7f21780cebf0 2023-08-09T13:34:17.579 INFO:teuthology.orchestra.run.smithi043.stdout:pg_num: 1 2023-08-09T13:34:17.581 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:17.579+0000 7f217dc70640 1 -- 172.21.15.43:0/841225791 >> [v2:172.21.15.203:6832/830097474,v1:172.21.15.203:6833/830097474] conn(0x7f215c06a050 msgr2=0x7f215c06c510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:34:17.582 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:17.579+0000 7f217dc70640 1 --2- 172.21.15.43:0/841225791 >> [v2:172.21.15.203:6832/830097474,v1:172.21.15.203:6833/830097474] conn(0x7f215c06a050 0x7f215c06c510 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f21780cb660 tx=0x7f2164009040 comp rx=0 tx=0).stop 2023-08-09T13:34:17.582 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:17.579+0000 7f217dc70640 1 -- 172.21.15.43:0/841225791 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f21780cebf0 msgr2=0x7f21780b2710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:34:17.582 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:17.579+0000 7f217dc70640 1 --2- 172.21.15.43:0/841225791 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f21780cebf0 0x7f21780b2710 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7f216c009870 tx=0x7f216c004730 comp rx=0 tx=0).stop 2023-08-09T13:34:17.583 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:17.579+0000 7f217dc70640 1 -- 172.21.15.43:0/841225791 shutdown_connections 2023-08-09T13:34:17.583 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:17.579+0000 7f217dc70640 1 --2- 172.21.15.43:0/841225791 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f217815c2d0 0x7f21780b3b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:34:17.583 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:17.579+0000 7f217dc70640 1 --2- 172.21.15.43:0/841225791 >> [v2:172.21.15.203:6832/830097474,v1:172.21.15.203:6833/830097474] conn(0x7f215c06a050 0x7f215c06c510 unknown :-1 s=CLOSED pgs=75 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:34:17.583 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:17.579+0000 7f217dc70640 1 --2- 172.21.15.43:0/841225791 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f21780d1b30 0x7f2178153a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:34:17.583 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:17.579+0000 7f217dc70640 1 --2- 172.21.15.43:0/841225791 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f21780cebf0 0x7f21780b2710 unknown :-1 s=CLOSED pgs=102 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:34:17.583 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:17.579+0000 7f217dc70640 1 -- 172.21.15.43:0/841225791 >> 172.21.15.43:0/841225791 conn(0x7f217806a230 msgr2=0x7f217815cf10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-09T13:34:17.583 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:17.579+0000 7f217dc70640 1 -- 172.21.15.43:0/841225791 shutdown_connections 2023-08-09T13:34:17.584 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:17.579+0000 7f217dc70640 1 -- 172.21.15.43:0/841225791 wait complete. 2023-08-09T13:34:17.602 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-08-09T13:34:17.824 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:17.822+0000 7fab5cac5640 1 Processor -- start 2023-08-09T13:34:17.825 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:17.822+0000 7fab5cac5640 1 -- start start 2023-08-09T13:34:17.826 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:17.823+0000 7fab5cac5640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7fab5806b140 0x7fab5806b520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:34:17.826 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:17.823+0000 7fab5cac5640 1 -- --> v1:172.21.15.43:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fab580af1a0 con 0x7fab580af6f0 2023-08-09T13:34:17.827 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:17.823+0000 7fab5cac5640 1 -- --> v1:172.21.15.203:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fab580cdc20 con 0x7fab580cf760 2023-08-09T13:34:17.827 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:17.823+0000 7fab5cac5640 1 -- --> v2:172.21.15.43:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fab5806be10 con 0x7fab5806b140 2023-08-09T13:34:17.827 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:17.824+0000 7fab5659c640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7fab5806b140 0x7fab5806b520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:34:17.827 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:17.824+0000 7fab5659c640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7fab5806b140 0x7fab5806b520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:44666/0 (socket says 172.21.15.43:44666) 2023-08-09T13:34:17.827 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:17.824+0000 7fab5659c640 1 -- 172.21.15.43:0/3075568588 learned_addr learned my addr 172.21.15.43:0/3075568588 (peer_addr_for_me v2:172.21.15.43:0/0) 2023-08-09T13:34:17.828 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:17.824+0000 7fab5659c640 1 -- 172.21.15.43:0/3075568588 >> v1:172.21.15.203:6789/0 conn(0x7fab580cf760 legacy=0x7fab5806dcb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-09T13:34:17.828 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:17.824+0000 7fab5659c640 1 -- 172.21.15.43:0/3075568588 >> v1:172.21.15.43:6789/0 conn(0x7fab580af6f0 legacy=0x7fab5806a8d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:34:17.828 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:17.824+0000 7fab5659c640 1 -- 172.21.15.43:0/3075568588 --> v2:172.21.15.43:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fab5806ba60 con 0x7fab5806b140 2023-08-09T13:34:17.828 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:17.826+0000 7fab5659c640 1 --2- 172.21.15.43:0/3075568588 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fab5806b140 0x7fab5806b520 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7fab48009860 tx=0x7fab4802b980 comp rx=0 tx=0).ready entity=mon.0 client_cookie=11d8b7e18ebddfa4 server_cookie=0 in_seq=0 out_seq=0 2023-08-09T13:34:17.830 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:17.827+0000 7fab5559a640 1 -- 172.21.15.43:0/3075568588 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fab48030840 con 0x7fab5806b140 2023-08-09T13:34:17.830 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:17.827+0000 7fab5559a640 1 -- 172.21.15.43:0/3075568588 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fab480309a0 con 0x7fab5806b140 2023-08-09T13:34:17.830 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:17.827+0000 7fab5559a640 1 -- 172.21.15.43:0/3075568588 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fab48030d30 con 0x7fab5806b140 2023-08-09T13:34:17.831 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:17.828+0000 7fab5cac5640 1 -- 172.21.15.43:0/3075568588 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fab5806b140 msgr2=0x7fab5806b520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:34:17.831 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:17.828+0000 7fab5cac5640 1 --2- 172.21.15.43:0/3075568588 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fab5806b140 0x7fab5806b520 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7fab48009860 tx=0x7fab4802b980 comp rx=0 tx=0).stop 2023-08-09T13:34:17.831 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:17.828+0000 7fab5cac5640 1 -- 172.21.15.43:0/3075568588 shutdown_connections 2023-08-09T13:34:17.831 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:17.828+0000 7fab5cac5640 1 --2- 172.21.15.43:0/3075568588 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fab5806b140 0x7fab5806b520 unknown :-1 s=CLOSED pgs=104 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:34:17.831 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:17.828+0000 7fab5cac5640 1 -- 172.21.15.43:0/3075568588 >> 172.21.15.43:0/3075568588 conn(0x7fab580ccd80 msgr2=0x7fab580cd190 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-09T13:34:17.832 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:17.828+0000 7fab5cac5640 1 -- 172.21.15.43:0/3075568588 shutdown_connections 2023-08-09T13:34:17.832 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:17.828+0000 7fab5cac5640 1 -- 172.21.15.43:0/3075568588 wait complete. 2023-08-09T13:34:17.832 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:17.830+0000 7fab5cac5640 1 Processor -- start 2023-08-09T13:34:17.833 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:17.830+0000 7fab5cac5640 1 -- start start 2023-08-09T13:34:17.833 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:17.831+0000 7fab5cac5640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fab5806b140 0x7fab58150810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:34:17.834 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:17.831+0000 7fab5cac5640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fab580af6f0 0x7fab580c8070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:34:17.834 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:17.831+0000 7fab5cac5640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fab580cf760 0x7fab580adf70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:34:17.834 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:17.831+0000 7fab5cac5640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fab5806afe0 con 0x7fab580cf760 2023-08-09T13:34:17.834 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:17.831+0000 7fab5cac5640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fab58150d50 con 0x7fab580af6f0 2023-08-09T13:34:17.834 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:17.831+0000 7fab5cac5640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fab580c85b0 con 0x7fab5806b140 2023-08-09T13:34:17.835 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:17.831+0000 7fab56d9d640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fab580cf760 0x7fab580adf70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:34:17.836 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:17.832+0000 7fab56d9d640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fab580cf760 0x7fab580adf70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:44682/0 (socket says 172.21.15.43:44682) 2023-08-09T13:34:17.836 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:17.832+0000 7fab56d9d640 1 -- 172.21.15.43:0/1148009490 learned_addr learned my addr 172.21.15.43:0/1148009490 (peer_addr_for_me v2:172.21.15.43:0/0) 2023-08-09T13:34:17.836 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:17.832+0000 7fab5659c640 1 --2- 172.21.15.43:0/1148009490 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fab5806b140 0x7fab58150810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:34:17.836 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:17.832+0000 7fab55d9b640 1 --2- 172.21.15.43:0/1148009490 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fab580af6f0 0x7fab580c8070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:34:17.836 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:17.832+0000 7fab56d9d640 1 -- 172.21.15.43:0/1148009490 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fab5806b140 msgr2=0x7fab58150810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:34:17.836 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:17.832+0000 7fab56d9d640 1 --2- 172.21.15.43:0/1148009490 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fab5806b140 0x7fab58150810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:34:17.837 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:17.832+0000 7fab56d9d640 1 -- 172.21.15.43:0/1148009490 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fab580af6f0 msgr2=0x7fab580c8070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:34:17.837 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:17.832+0000 7fab56d9d640 1 --2- 172.21.15.43:0/1148009490 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fab580af6f0 0x7fab580c8070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:34:17.837 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:17.832+0000 7fab56d9d640 1 -- 172.21.15.43:0/1148009490 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fab48009470 con 0x7fab580cf760 2023-08-09T13:34:17.837 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:17.832+0000 7fab56d9d640 1 --2- 172.21.15.43:0/1148009490 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fab580cf760 0x7fab580adf70 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7fab5000c410 tx=0x7fab5000c810 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-09T13:34:17.837 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:17.833+0000 7fab4effd640 1 -- 172.21.15.43:0/1148009490 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fab50011790 con 0x7fab580cf760 2023-08-09T13:34:17.837 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:17.833+0000 7fab4effd640 1 -- 172.21.15.43:0/1148009490 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fab500118f0 con 0x7fab580cf760 2023-08-09T13:34:17.837 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:17.833+0000 7fab5cac5640 1 -- 172.21.15.43:0/1148009490 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fab580c7340 con 0x7fab580cf760 2023-08-09T13:34:17.837 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:17.833+0000 7fab4effd640 1 -- 172.21.15.43:0/1148009490 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fab50011ce0 con 0x7fab580cf760 2023-08-09T13:34:17.838 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:17.833+0000 7fab5cac5640 1 -- 172.21.15.43:0/1148009490 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fab580c7590 con 0x7fab580cf760 2023-08-09T13:34:17.838 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:17.834+0000 7fab5cac5640 1 -- 172.21.15.43:0/1148009490 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fab580739f0 con 0x7fab580cf760 2023-08-09T13:34:17.838 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:17.835+0000 7fab4effd640 1 -- 172.21.15.43:0/1148009490 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 5) v1 ==== 88341+0+0 (secure 0 0 0) 0x7fab5001c460 con 0x7fab580cf760 2023-08-09T13:34:17.839 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:17.836+0000 7fab4effd640 1 --2- 172.21.15.43:0/1148009490 >> [v2:172.21.15.203:6832/830097474,v1:172.21.15.203:6833/830097474] conn(0x7fab3806a080 0x7fab3806c540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:34:17.839 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:17.836+0000 7fab4effd640 1 -- 172.21.15.43:0/1148009490 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6813+0+0 (secure 0 0 0) 0x7fab500884c0 con 0x7fab580cf760 2023-08-09T13:34:17.839 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:17.837+0000 7fab5659c640 1 --2- 172.21.15.43:0/1148009490 >> [v2:172.21.15.203:6832/830097474,v1:172.21.15.203:6833/830097474] conn(0x7fab3806a080 0x7fab3806c540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:34:17.840 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:17.838+0000 7fab5659c640 1 --2- 172.21.15.43:0/1148009490 >> [v2:172.21.15.203:6832/830097474,v1:172.21.15.203:6833/830097474] conn(0x7fab3806a080 0x7fab3806c540 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7fab48036000 tx=0x7fab4803b040 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-09T13:34:17.843 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:17.840+0000 7fab4effd640 1 -- 172.21.15.43:0/1148009490 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178066 (secure 0 0 0) 0x7fab50057370 con 0x7fab580cf760 2023-08-09T13:34:18.035 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:18.032+0000 7fab5cac5640 1 -- 172.21.15.43:0/1148009490 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7fab5815ee80 con 0x7fab580cf760 2023-08-09T13:34:18.035 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:18.033+0000 7fab4effd640 1 -- 172.21.15.43:0/1148009490 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v24) v1 ==== 104+0+11 (secure 0 0 0) 0x7fab50056f00 con 0x7fab580cf760 2023-08-09T13:34:18.036 INFO:teuthology.orchestra.run.smithi043.stdout:pg_num: 32 2023-08-09T13:34:18.038 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:18.036+0000 7fab5cac5640 1 -- 172.21.15.43:0/1148009490 >> [v2:172.21.15.203:6832/830097474,v1:172.21.15.203:6833/830097474] conn(0x7fab3806a080 msgr2=0x7fab3806c540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:34:18.039 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:18.036+0000 7fab5cac5640 1 --2- 172.21.15.43:0/1148009490 >> [v2:172.21.15.203:6832/830097474,v1:172.21.15.203:6833/830097474] conn(0x7fab3806a080 0x7fab3806c540 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7fab48036000 tx=0x7fab4803b040 comp rx=0 tx=0).stop 2023-08-09T13:34:18.039 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:18.036+0000 7fab5cac5640 1 -- 172.21.15.43:0/1148009490 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fab580cf760 msgr2=0x7fab580adf70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:34:18.039 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:18.036+0000 7fab5cac5640 1 --2- 172.21.15.43:0/1148009490 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fab580cf760 0x7fab580adf70 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7fab5000c410 tx=0x7fab5000c810 comp rx=0 tx=0).stop 2023-08-09T13:34:18.039 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:18.036+0000 7fab5cac5640 1 -- 172.21.15.43:0/1148009490 shutdown_connections 2023-08-09T13:34:18.039 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:18.036+0000 7fab5cac5640 1 --2- 172.21.15.43:0/1148009490 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fab580cf760 0x7fab580adf70 unknown :-1 s=CLOSED pgs=105 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:34:18.039 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:18.036+0000 7fab5cac5640 1 --2- 172.21.15.43:0/1148009490 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fab580af6f0 0x7fab580c8070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:34:18.039 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:18.036+0000 7fab5cac5640 1 --2- 172.21.15.43:0/1148009490 >> [v2:172.21.15.203:6832/830097474,v1:172.21.15.203:6833/830097474] conn(0x7fab3806a080 0x7fab3806c540 unknown :-1 s=CLOSED pgs=76 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:34:18.040 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:18.036+0000 7fab5cac5640 1 --2- 172.21.15.43:0/1148009490 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fab5806b140 0x7fab58150810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:34:18.040 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:18.036+0000 7fab5cac5640 1 -- 172.21.15.43:0/1148009490 >> 172.21.15.43:0/1148009490 conn(0x7fab580ccd80 msgr2=0x7fab580cd160 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-09T13:34:18.040 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:18.036+0000 7fab5cac5640 1 -- 172.21.15.43:0/1148009490 shutdown_connections 2023-08-09T13:34:18.041 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:18.036+0000 7fab5cac5640 1 -- 172.21.15.43:0/1148009490 wait complete. 2023-08-09T13:34:18.058 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-08-09T13:34:18.294 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:18.292+0000 7f09a318b640 1 Processor -- start 2023-08-09T13:34:18.294 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:18.292+0000 7f09a318b640 1 -- start start 2023-08-09T13:34:18.295 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:18.293+0000 7f09a318b640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f099c0d04b0 0x7f099c0cd6d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:34:18.296 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:18.293+0000 7f09a318b640 1 -- --> v1:172.21.15.43:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f099c0b7bd0 con 0x7f099c15c550 2023-08-09T13:34:18.296 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:18.293+0000 7f09a318b640 1 -- --> v1:172.21.15.203:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f099c0d1c00 con 0x7f099c0b45f0 2023-08-09T13:34:18.296 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:18.293+0000 7f09a318b640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f099c15c930 con 0x7f099c0d04b0 2023-08-09T13:34:18.296 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:18.293+0000 7f09a0f27640 1 --1- >> v1:172.21.15.43:6789/0 conn(0x7f099c15c550 0x7f099c0b3ee0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.43:6789/0 says I am v1:172.21.15.43:48162/0 (socket says 172.21.15.43:48162) 2023-08-09T13:34:18.297 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:18.294+0000 7f09a0f27640 1 -- 172.21.15.43:0/26165354 learned_addr learned my addr 172.21.15.43:0/26165354 (peer_addr_for_me v1:172.21.15.43:0/0) 2023-08-09T13:34:18.297 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:18.294+0000 7f09a1728640 1 --2- 172.21.15.43:0/26165354 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f099c0d04b0 0x7f099c0cd6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:34:18.297 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:18.294+0000 7f099b7fe640 1 -- 172.21.15.43:0/26165354 <== mon.0 v1:172.21.15.43:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3018046043 0 0) 0x7f099c0b7bd0 con 0x7f099c15c550 2023-08-09T13:34:18.297 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:18.294+0000 7f099b7fe640 1 -- 172.21.15.43:0/26165354 --> v1:172.21.15.43:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f0988003520 con 0x7f099c15c550 2023-08-09T13:34:18.298 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:18.294+0000 7f099b7fe640 1 -- 172.21.15.43:0/26165354 <== mon.0 v1:172.21.15.43:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 86921865 0 0) 0x7f0988003520 con 0x7f099c15c550 2023-08-09T13:34:18.299 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:18.294+0000 7f099b7fe640 1 -- 172.21.15.43:0/26165354 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f099c0d04b0 msgr2=0x7f099c0cd6d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-09T13:34:18.299 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:18.294+0000 7f099b7fe640 1 --2- 172.21.15.43:0/26165354 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f099c0d04b0 0x7f099c0cd6d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:34:18.299 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:18.294+0000 7f099b7fe640 1 -- 172.21.15.43:0/26165354 >> v1:172.21.15.203:6789/0 conn(0x7f099c0b45f0 legacy=0x7f099c0b59d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:34:18.299 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:18.295+0000 7f099b7fe640 1 -- 172.21.15.43:0/26165354 --> v1:172.21.15.43:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f099c0b54f0 con 0x7f099c15c550 2023-08-09T13:34:18.299 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:18.295+0000 7f099b7fe640 1 -- 172.21.15.43:0/26165354 <== mon.0 v1:172.21.15.43:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1285150740 0 0) 0x7f098c0019f0 con 0x7f099c15c550 2023-08-09T13:34:18.299 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:18.295+0000 7f099b7fe640 1 -- 172.21.15.43:0/26165354 >> v1:172.21.15.43:6789/0 conn(0x7f099c15c550 legacy=0x7f099c0b3ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:34:18.299 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:18.295+0000 7f099b7fe640 1 --2- 172.21.15.43:0/26165354 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f09880039a0 0x7f0988003d80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:34:18.300 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:18.295+0000 7f099b7fe640 1 --2- 172.21.15.43:0/26165354 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f099c0d04b0 0x7f09880047c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:34:18.300 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:18.295+0000 7f099b7fe640 1 --2- 172.21.15.43:0/26165354 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f0988004d00 0x7f0988005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:34:18.300 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:18.295+0000 7f099b7fe640 1 -- 172.21.15.43:0/26165354 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f099c15c930 con 0x7f09880039a0 2023-08-09T13:34:18.300 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:18.295+0000 7f099b7fe640 1 -- 172.21.15.43:0/26165354 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f098800b980 con 0x7f0988004d00 2023-08-09T13:34:18.300 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:18.295+0000 7f099b7fe640 1 -- 172.21.15.43:0/26165354 --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f098800bb40 con 0x7f099c0d04b0 2023-08-09T13:34:18.300 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:18.295+0000 7f09a0f27640 1 --2- 172.21.15.43:0/26165354 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f09880039a0 0x7f0988003d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:34:18.300 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:18.295+0000 7f09a1728640 1 --2- 172.21.15.43:0/26165354 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f0988004d00 0x7f0988005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:34:18.301 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:18.295+0000 7f099bfff640 1 --2- 172.21.15.43:0/26165354 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f099c0d04b0 0x7f09880047c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:34:18.301 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:18.296+0000 7f09a0f27640 1 -- 172.21.15.43:0/26165354 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f099c0d04b0 msgr2=0x7f09880047c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-09T13:34:18.301 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:18.296+0000 7f09a0f27640 1 --2- 172.21.15.43:0/26165354 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f099c0d04b0 0x7f09880047c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:34:18.301 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:18.296+0000 7f09a0f27640 1 -- 172.21.15.43:0/26165354 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f0988004d00 msgr2=0x7f0988005180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-09T13:34:18.301 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:18.296+0000 7f09a0f27640 1 --2- 172.21.15.43:0/26165354 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f0988004d00 0x7f0988005180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:34:18.301 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:18.296+0000 7f09a0f27640 1 -- 172.21.15.43:0/26165354 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0988003520 con 0x7f09880039a0 2023-08-09T13:34:18.302 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:18.297+0000 7f09a0f27640 1 --2- 172.21.15.43:0/26165354 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f09880039a0 0x7f0988003d80 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7f098c00a840 tx=0x7f09880057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=885ce9507641fa30 server_cookie=0 in_seq=0 out_seq=0 2023-08-09T13:34:18.302 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:18.297+0000 7f099b7fe640 1 -- 172.21.15.43:0/26165354 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f098c034630 con 0x7f09880039a0 2023-08-09T13:34:18.303 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:18.297+0000 7f099b7fe640 1 -- 172.21.15.43:0/26165354 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f098c02f070 con 0x7f09880039a0 2023-08-09T13:34:18.303 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:18.297+0000 7f099b7fe640 1 -- 172.21.15.43:0/26165354 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f098c0349d0 con 0x7f09880039a0 2023-08-09T13:34:18.303 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:18.298+0000 7f09a318b640 1 -- 172.21.15.43:0/26165354 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f09880039a0 msgr2=0x7f0988003d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:34:18.303 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:18.298+0000 7f09a318b640 1 --2- 172.21.15.43:0/26165354 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f09880039a0 0x7f0988003d80 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7f098c00a840 tx=0x7f09880057f0 comp rx=0 tx=0).stop 2023-08-09T13:34:18.303 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:18.298+0000 7f09a1728640 1 -- 172.21.15.43:0/26165354 reap_dead start 2023-08-09T13:34:18.303 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:18.298+0000 7f09a318b640 1 -- 172.21.15.43:0/26165354 shutdown_connections 2023-08-09T13:34:18.303 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:18.298+0000 7f09a318b640 1 -- 172.21.15.43:0/26165354 >> 172.21.15.43:0/26165354 conn(0x7f099c071520 msgr2=0x7f099c0ceee0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-09T13:34:18.303 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:18.298+0000 7f09a318b640 1 -- 172.21.15.43:0/26165354 shutdown_connections 2023-08-09T13:34:18.304 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:18.298+0000 7f09a318b640 1 -- 172.21.15.43:0/26165354 wait complete. 2023-08-09T13:34:18.304 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:18.300+0000 7f09a318b640 1 Processor -- start 2023-08-09T13:34:18.304 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:18.300+0000 7f09a318b640 1 -- start start 2023-08-09T13:34:18.305 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:18.301+0000 7f09a318b640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f09880039a0 0x7f099c0cd5f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:34:18.306 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:18.301+0000 7f09a318b640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f099c156100 0x7f099c152890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:34:18.306 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:18.301+0000 7f09a318b640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f099c0c77d0 0x7f099c0c7c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:34:18.306 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:18.301+0000 7f09a318b640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f099c0c81a0 con 0x7f099c0c77d0 2023-08-09T13:34:18.306 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:18.301+0000 7f09a318b640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f099c0cdb30 con 0x7f09880039a0 2023-08-09T13:34:18.306 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:18.301+0000 7f09a318b640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f099c155a70 con 0x7f099c156100 2023-08-09T13:34:18.306 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:18.301+0000 7f09a1728640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f099c0c77d0 0x7f099c0c7c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:34:18.306 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:18.301+0000 7f09a1728640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f099c0c77d0 0x7f099c0c7c60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:44690/0 (socket says 172.21.15.43:44690) 2023-08-09T13:34:18.307 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:18.301+0000 7f09a1728640 1 -- 172.21.15.43:0/2112934459 learned_addr learned my addr 172.21.15.43:0/2112934459 (peer_addr_for_me v2:172.21.15.43:0/0) 2023-08-09T13:34:18.307 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:18.301+0000 7f099bfff640 1 --2- 172.21.15.43:0/2112934459 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f099c156100 0x7f099c152890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:34:18.307 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:18.302+0000 7f09a1728640 1 -- 172.21.15.43:0/2112934459 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f099c156100 msgr2=0x7f099c152890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:34:18.307 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:18.302+0000 7f09a1728640 1 --2- 172.21.15.43:0/2112934459 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f099c156100 0x7f099c152890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:34:18.307 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:18.302+0000 7f09a0f27640 1 --2- 172.21.15.43:0/2112934459 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f09880039a0 0x7f099c0cd5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:34:18.307 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:18.302+0000 7f09a1728640 1 -- 172.21.15.43:0/2112934459 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f09880039a0 msgr2=0x7f099c0cd5f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:34:18.307 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:18.302+0000 7f09a1728640 1 --2- 172.21.15.43:0/2112934459 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f09880039a0 0x7f099c0cd5f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:34:18.307 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:18.302+0000 7f09a1728640 1 -- 172.21.15.43:0/2112934459 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f099c1f1dc0 con 0x7f099c0c77d0 2023-08-09T13:34:18.308 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:18.302+0000 7f099bfff640 1 --2- 172.21.15.43:0/2112934459 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f099c156100 0x7f099c152890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-08-09T13:34:18.308 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:18.302+0000 7f09a0f27640 1 --2- 172.21.15.43:0/2112934459 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f09880039a0 0x7f099c0cd5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-08-09T13:34:18.308 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:18.302+0000 7f09a1728640 1 --2- 172.21.15.43:0/2112934459 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f099c0c77d0 0x7f099c0c7c60 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7f099c0d2340 tx=0x7f0994011920 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-09T13:34:18.308 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:18.302+0000 7f09997fa640 1 -- 172.21.15.43:0/2112934459 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0994004440 con 0x7f099c0c77d0 2023-08-09T13:34:18.308 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:18.302+0000 7f09997fa640 1 -- 172.21.15.43:0/2112934459 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0994003610 con 0x7f099c0c77d0 2023-08-09T13:34:18.308 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:18.302+0000 7f09997fa640 1 -- 172.21.15.43:0/2112934459 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f09940039c0 con 0x7f099c0c77d0 2023-08-09T13:34:18.308 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:18.302+0000 7f09a318b640 1 -- 172.21.15.43:0/2112934459 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f099c1595c0 con 0x7f099c0c77d0 2023-08-09T13:34:18.308 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:18.302+0000 7f09a318b640 1 -- 172.21.15.43:0/2112934459 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f099c152dd0 con 0x7f099c0c77d0 2023-08-09T13:34:18.309 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:18.304+0000 7f09a318b640 1 -- 172.21.15.43:0/2112934459 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f099c159420 con 0x7f099c0c77d0 2023-08-09T13:34:18.309 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:18.304+0000 7f09997fa640 1 -- 172.21.15.43:0/2112934459 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 5) v1 ==== 88341+0+0 (secure 0 0 0) 0x7f0994003b20 con 0x7f099c0c77d0 2023-08-09T13:34:18.309 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:18.305+0000 7f09997fa640 1 --2- 172.21.15.43:0/2112934459 >> [v2:172.21.15.203:6832/830097474,v1:172.21.15.203:6833/830097474] conn(0x7f097c06a080 0x7f097c06c540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:34:18.310 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:18.306+0000 7f09997fa640 1 -- 172.21.15.43:0/2112934459 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6813+0+0 (secure 0 0 0) 0x7f0994088ed0 con 0x7f099c0c77d0 2023-08-09T13:34:18.310 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:18.306+0000 7f09a0f27640 1 --2- 172.21.15.43:0/2112934459 >> [v2:172.21.15.203:6832/830097474,v1:172.21.15.203:6833/830097474] conn(0x7f097c06a080 0x7f097c06c540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:34:18.310 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:18.308+0000 7f09a0f27640 1 --2- 172.21.15.43:0/2112934459 >> [v2:172.21.15.203:6832/830097474,v1:172.21.15.203:6833/830097474] conn(0x7f097c06a080 0x7f097c06c540 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f098c034000 tx=0x7f098c02f000 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-09T13:34:18.312 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:18.310+0000 7f09997fa640 1 -- 172.21.15.43:0/2112934459 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178066 (secure 0 0 0) 0x7f099c159420 con 0x7f099c0c77d0 2023-08-09T13:34:18.509 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:18.507+0000 7f09a318b640 1 -- 172.21.15.43:0/2112934459 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f099c159420 con 0x7f099c0c77d0 2023-08-09T13:34:18.510 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:18.508+0000 7f09997fa640 1 -- 172.21.15.43:0/2112934459 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v24) v1 ==== 100+0+11 (secure 0 0 0) 0x7f099c159420 con 0x7f099c0c77d0 2023-08-09T13:34:18.511 INFO:teuthology.orchestra.run.smithi043.stdout:pg_num: 64 2023-08-09T13:34:18.513 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:18.511+0000 7f09a318b640 1 -- 172.21.15.43:0/2112934459 >> [v2:172.21.15.203:6832/830097474,v1:172.21.15.203:6833/830097474] conn(0x7f097c06a080 msgr2=0x7f097c06c540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:34:18.513 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:18.511+0000 7f09a318b640 1 --2- 172.21.15.43:0/2112934459 >> [v2:172.21.15.203:6832/830097474,v1:172.21.15.203:6833/830097474] conn(0x7f097c06a080 0x7f097c06c540 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f098c034000 tx=0x7f098c02f000 comp rx=0 tx=0).stop 2023-08-09T13:34:18.513 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:18.511+0000 7f09a318b640 1 -- 172.21.15.43:0/2112934459 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f099c0c77d0 msgr2=0x7f099c0c7c60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:34:18.514 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:18.511+0000 7f09a318b640 1 --2- 172.21.15.43:0/2112934459 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f099c0c77d0 0x7f099c0c7c60 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7f099c0d2340 tx=0x7f0994011920 comp rx=0 tx=0).stop 2023-08-09T13:34:18.514 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:18.511+0000 7f09a318b640 1 -- 172.21.15.43:0/2112934459 shutdown_connections 2023-08-09T13:34:18.514 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:18.511+0000 7f09a318b640 1 --2- 172.21.15.43:0/2112934459 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f099c0c77d0 0x7f099c0c7c60 unknown :-1 s=CLOSED pgs=108 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:34:18.514 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:18.511+0000 7f09a318b640 1 --2- 172.21.15.43:0/2112934459 >> [v2:172.21.15.203:6832/830097474,v1:172.21.15.203:6833/830097474] conn(0x7f097c06a080 0x7f097c06c540 unknown :-1 s=CLOSED pgs=77 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:34:18.515 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:18.511+0000 7f09a318b640 1 --2- 172.21.15.43:0/2112934459 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f099c156100 0x7f099c152890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:34:18.515 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:18.511+0000 7f09a318b640 1 --2- 172.21.15.43:0/2112934459 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f09880039a0 0x7f099c0cd5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:34:18.515 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:18.511+0000 7f09a318b640 1 -- 172.21.15.43:0/2112934459 >> 172.21.15.43:0/2112934459 conn(0x7f099c071520 msgr2=0x7f099c15ef70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-09T13:34:18.515 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:18.511+0000 7f09a318b640 1 -- 172.21.15.43:0/2112934459 shutdown_connections 2023-08-09T13:34:18.515 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:18.511+0000 7f09a318b640 1 -- 172.21.15.43:0/2112934459 wait complete. 2023-08-09T13:34:18.532 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2023-08-09T13:34:18.765 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:18.762+0000 7f716c3a2640 1 Processor -- start 2023-08-09T13:34:18.765 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:18.762+0000 7f716c3a2640 1 -- start start 2023-08-09T13:34:18.767 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:18.764+0000 7f716c3a2640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f7164143c20 0x7f716406f540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:34:18.767 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:18.764+0000 7f716c3a2640 1 -- --> v1:172.21.15.43:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f716406c440 con 0x7f716406cfe0 2023-08-09T13:34:18.767 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:18.764+0000 7f716c3a2640 1 -- --> v1:172.21.15.203:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f716406b830 con 0x7f71640c86f0 2023-08-09T13:34:18.767 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:18.764+0000 7f716c3a2640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7164144000 con 0x7f7164143c20 2023-08-09T13:34:18.768 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:18.765+0000 7f716a93f640 1 --1- >> v1:172.21.15.43:6789/0 conn(0x7f716406cfe0 0x7f71640ccc00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.43:6789/0 says I am v1:172.21.15.43:48170/0 (socket says 172.21.15.43:48170) 2023-08-09T13:34:18.768 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:18.765+0000 7f716a93f640 1 -- 172.21.15.43:0/1035919520 learned_addr learned my addr 172.21.15.43:0/1035919520 (peer_addr_for_me v1:172.21.15.43:0/0) 2023-08-09T13:34:18.768 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:18.765+0000 7f716a13e640 1 --2- 172.21.15.43:0/1035919520 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f7164143c20 0x7f716406f540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:34:18.769 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:18.765+0000 7f716913c640 1 -- 172.21.15.43:0/1035919520 <== mon.0 v1:172.21.15.43:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 14333295 0 0) 0x7f716406c440 con 0x7f716406cfe0 2023-08-09T13:34:18.769 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:18.765+0000 7f716913c640 1 -- 172.21.15.43:0/1035919520 --> v1:172.21.15.43:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f7150003520 con 0x7f716406cfe0 2023-08-09T13:34:18.769 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:18.766+0000 7f716a13e640 1 -- 172.21.15.43:0/1035919520 >> v1:172.21.15.203:6789/0 conn(0x7f71640c86f0 legacy=0x7f716406c8d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:34:18.769 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:18.766+0000 7f716a13e640 1 -- 172.21.15.43:0/1035919520 >> v1:172.21.15.43:6789/0 conn(0x7f716406cfe0 legacy=0x7f71640ccc00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:34:18.769 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:18.766+0000 7f716a13e640 1 -- 172.21.15.43:0/1035919520 --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f716406bb10 con 0x7f7164143c20 2023-08-09T13:34:18.770 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:18.767+0000 7f716a13e640 1 --2- 172.21.15.43:0/1035919520 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f7164143c20 0x7f716406f540 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7f71600097d0 tx=0x7f716002b8b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=571b55b62ac48a34 server_cookie=0 in_seq=0 out_seq=0 2023-08-09T13:34:18.770 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:18.768+0000 7f716913c640 1 -- 172.21.15.43:0/1035919520 <== mon.2 v2:172.21.15.203:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f716002d040 con 0x7f7164143c20 2023-08-09T13:34:18.770 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:18.768+0000 7f716913c640 1 -- 172.21.15.43:0/1035919520 <== mon.2 v2:172.21.15.203:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7160031810 con 0x7f7164143c20 2023-08-09T13:34:18.770 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:18.768+0000 7f716913c640 1 -- 172.21.15.43:0/1035919520 <== mon.2 v2:172.21.15.203:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7160031ba0 con 0x7f7164143c20 2023-08-09T13:34:18.771 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:18.768+0000 7f716c3a2640 1 -- 172.21.15.43:0/1035919520 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f7164143c20 msgr2=0x7f716406f540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:34:18.771 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:18.768+0000 7f716c3a2640 1 --2- 172.21.15.43:0/1035919520 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f7164143c20 0x7f716406f540 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7f71600097d0 tx=0x7f716002b8b0 comp rx=0 tx=0).stop 2023-08-09T13:34:18.772 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:18.769+0000 7f716c3a2640 1 -- 172.21.15.43:0/1035919520 shutdown_connections 2023-08-09T13:34:18.772 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:18.769+0000 7f716c3a2640 1 --2- 172.21.15.43:0/1035919520 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f7164143c20 0x7f716406f540 unknown :-1 s=CLOSED pgs=40 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:34:18.772 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:18.769+0000 7f716c3a2640 1 -- 172.21.15.43:0/1035919520 >> 172.21.15.43:0/1035919520 conn(0x7f71640ce0a0 msgr2=0x7f71640ce480 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-09T13:34:18.772 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:18.769+0000 7f716c3a2640 1 -- 172.21.15.43:0/1035919520 shutdown_connections 2023-08-09T13:34:18.772 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:18.769+0000 7f716c3a2640 1 -- 172.21.15.43:0/1035919520 wait complete. 2023-08-09T13:34:18.773 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:18.771+0000 7f716c3a2640 1 Processor -- start 2023-08-09T13:34:18.773 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:18.771+0000 7f716c3a2640 1 -- start start 2023-08-09T13:34:18.778 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:18.772+0000 7f716c3a2640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f716406cfe0 0x7f7164141c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:34:18.778 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:18.773+0000 7f716c3a2640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f71640c86f0 0x7f71640b2d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:34:18.778 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:18.773+0000 7f716c3a2640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f7164143c20 0x7f71640b1250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:34:18.779 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:18.773+0000 7f716c3a2640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f71640c6760 con 0x7f7164143c20 2023-08-09T13:34:18.779 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:18.773+0000 7f716c3a2640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f71641421b0 con 0x7f71640c86f0 2023-08-09T13:34:18.779 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:18.773+0000 7f716c3a2640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f71641f29e0 con 0x7f716406cfe0 2023-08-09T13:34:18.779 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:18.774+0000 7f716a13e640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f716406cfe0 0x7f7164141c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:34:18.779 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:18.774+0000 7f716a13e640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f716406cfe0 0x7f7164141c70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.203:3301/0 says I am v2:172.21.15.43:57432/0 (socket says 172.21.15.43:57432) 2023-08-09T13:34:18.780 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:18.774+0000 7f716a13e640 1 -- 172.21.15.43:0/2360620672 learned_addr learned my addr 172.21.15.43:0/2360620672 (peer_addr_for_me v2:172.21.15.43:0/0) 2023-08-09T13:34:18.780 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:18.775+0000 7f716a93f640 1 --2- 172.21.15.43:0/2360620672 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f7164143c20 0x7f71640b1250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:34:18.780 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:18.775+0000 7f716993d640 1 --2- 172.21.15.43:0/2360620672 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f71640c86f0 0x7f71640b2d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:34:18.780 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:18.776+0000 7f716a13e640 1 -- 172.21.15.43:0/2360620672 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f71640c86f0 msgr2=0x7f71640b2d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:34:18.780 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:18.776+0000 7f716a13e640 1 --2- 172.21.15.43:0/2360620672 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f71640c86f0 0x7f71640b2d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:34:18.780 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:18.776+0000 7f716a13e640 1 -- 172.21.15.43:0/2360620672 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f7164143c20 msgr2=0x7f71640b1250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:34:18.780 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:18.776+0000 7f716a13e640 1 --2- 172.21.15.43:0/2360620672 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f7164143c20 0x7f71640b1250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:34:18.781 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:18.776+0000 7f716a13e640 1 -- 172.21.15.43:0/2360620672 --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7160009400 con 0x7f716406cfe0 2023-08-09T13:34:18.781 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:18.776+0000 7f716a93f640 1 --2- 172.21.15.43:0/2360620672 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f7164143c20 0x7f71640b1250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-08-09T13:34:18.781 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:18.776+0000 7f716993d640 1 --2- 172.21.15.43:0/2360620672 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f71640c86f0 0x7f71640b2d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-08-09T13:34:18.782 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:18.776+0000 7f716a13e640 1 --2- 172.21.15.43:0/2360620672 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f716406cfe0 0x7f7164141c70 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7f7160000c00 tx=0x7f71600046c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-09T13:34:18.782 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:18.777+0000 7f715affd640 1 -- 172.21.15.43:0/2360620672 <== mon.2 v2:172.21.15.203:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7160006e10 con 0x7f716406cfe0 2023-08-09T13:34:18.782 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:18.777+0000 7f715affd640 1 -- 172.21.15.43:0/2360620672 <== mon.2 v2:172.21.15.203:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f716003ad60 con 0x7f716406cfe0 2023-08-09T13:34:18.782 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:18.777+0000 7f716c3a2640 1 -- 172.21.15.43:0/2360620672 --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f71640b3620 con 0x7f716406cfe0 2023-08-09T13:34:18.782 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:18.777+0000 7f715affd640 1 -- 172.21.15.43:0/2360620672 <== mon.2 v2:172.21.15.203:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7160044620 con 0x7f716406cfe0 2023-08-09T13:34:18.782 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:18.777+0000 7f716c3a2640 1 -- 172.21.15.43:0/2360620672 --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f71640c63f0 con 0x7f716406cfe0 2023-08-09T13:34:18.782 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:18.778+0000 7f716c3a2640 1 -- 172.21.15.43:0/2360620672 --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f71640739f0 con 0x7f716406cfe0 2023-08-09T13:34:18.783 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:18.780+0000 7f715affd640 1 -- 172.21.15.43:0/2360620672 <== mon.2 v2:172.21.15.203:3301/0 4 ==== mgrmap(e 5) v1 ==== 88341+0+0 (secure 0 0 0) 0x7f716004d440 con 0x7f716406cfe0 2023-08-09T13:34:18.784 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:18.781+0000 7f715affd640 1 --2- 172.21.15.43:0/2360620672 >> [v2:172.21.15.203:6832/830097474,v1:172.21.15.203:6833/830097474] conn(0x7f714806a050 0x7f714806c510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:34:18.784 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:18.781+0000 7f715affd640 1 -- 172.21.15.43:0/2360620672 <== mon.2 v2:172.21.15.203:3301/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6813+0+0 (secure 0 0 0) 0x7f71600ab080 con 0x7f716406cfe0 2023-08-09T13:34:18.784 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:18.782+0000 7f716993d640 1 --2- 172.21.15.43:0/2360620672 >> [v2:172.21.15.203:6832/830097474,v1:172.21.15.203:6833/830097474] conn(0x7f714806a050 0x7f714806c510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:34:18.785 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:18.783+0000 7f716993d640 1 --2- 172.21.15.43:0/2360620672 >> [v2:172.21.15.203:6832/830097474,v1:172.21.15.203:6833/830097474] conn(0x7f714806a050 0x7f714806c510 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f7164142df0 tx=0x7f7154009040 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-09T13:34:18.799 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:18.796+0000 7f715affd640 1 -- 172.21.15.43:0/2360620672 <== mon.2 v2:172.21.15.203:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178066 (secure 0 0 0) 0x7f7160076610 con 0x7f716406cfe0 2023-08-09T13:34:19.002 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:18.999+0000 7f716c3a2640 1 -- 172.21.15.43:0/2360620672 --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) v1 -- 0x7f71640cec90 con 0x7f716406cfe0 2023-08-09T13:34:19.003 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:19.000+0000 7f715affd640 1 -- 172.21.15.43:0/2360620672 <== mon.2 v2:172.21.15.203:3301/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v24) v1 ==== 103+0+11 (secure 0 0 0) 0x7f71600350a0 con 0x7f716406cfe0 2023-08-09T13:34:19.003 INFO:teuthology.orchestra.run.smithi043.stdout:pg_num: 32 2023-08-09T13:34:19.006 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:19.003+0000 7f716c3a2640 1 -- 172.21.15.43:0/2360620672 >> [v2:172.21.15.203:6832/830097474,v1:172.21.15.203:6833/830097474] conn(0x7f714806a050 msgr2=0x7f714806c510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:34:19.006 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:19.003+0000 7f716c3a2640 1 --2- 172.21.15.43:0/2360620672 >> [v2:172.21.15.203:6832/830097474,v1:172.21.15.203:6833/830097474] conn(0x7f714806a050 0x7f714806c510 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f7164142df0 tx=0x7f7154009040 comp rx=0 tx=0).stop 2023-08-09T13:34:19.006 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:19.003+0000 7f716c3a2640 1 -- 172.21.15.43:0/2360620672 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f716406cfe0 msgr2=0x7f7164141c70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:34:19.006 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:19.003+0000 7f716c3a2640 1 --2- 172.21.15.43:0/2360620672 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f716406cfe0 0x7f7164141c70 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7f7160000c00 tx=0x7f71600046c0 comp rx=0 tx=0).stop 2023-08-09T13:34:19.006 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:19.003+0000 7f716c3a2640 1 -- 172.21.15.43:0/2360620672 shutdown_connections 2023-08-09T13:34:19.006 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:19.003+0000 7f716c3a2640 1 --2- 172.21.15.43:0/2360620672 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f7164143c20 0x7f71640b1250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:34:19.007 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:19.003+0000 7f716c3a2640 1 --2- 172.21.15.43:0/2360620672 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f71640c86f0 0x7f71640b2d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:34:19.007 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:19.003+0000 7f716c3a2640 1 --2- 172.21.15.43:0/2360620672 >> [v2:172.21.15.203:6832/830097474,v1:172.21.15.203:6833/830097474] conn(0x7f714806a050 0x7f714806c510 unknown :-1 s=CLOSED pgs=78 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:34:19.007 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:19.003+0000 7f716c3a2640 1 --2- 172.21.15.43:0/2360620672 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f716406cfe0 0x7f7164141c70 unknown :-1 s=CLOSED pgs=41 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:34:19.007 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:19.003+0000 7f716c3a2640 1 -- 172.21.15.43:0/2360620672 >> 172.21.15.43:0/2360620672 conn(0x7f71640ce0a0 msgr2=0x7f716406fdc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-09T13:34:19.007 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:19.003+0000 7f716c3a2640 1 -- 172.21.15.43:0/2360620672 shutdown_connections 2023-08-09T13:34:19.007 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:19.003+0000 7f716c3a2640 1 -- 172.21.15.43:0/2360620672 wait complete. 2023-08-09T13:34:19.027 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2023-08-09T13:34:19.263 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:19.260+0000 7f70a58ec640 1 Processor -- start 2023-08-09T13:34:19.263 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:19.260+0000 7f70a58ec640 1 -- start start 2023-08-09T13:34:19.264 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:19.262+0000 7f70a58ec640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f70a0143c20 0x7f70a006f540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:34:19.265 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:19.262+0000 7f70a58ec640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f70a00c86f0 0x7f70a00c8b50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:34:19.265 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:19.262+0000 7f70a58ec640 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7f70a006c8d0 0x7f70a006cd60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:34:19.265 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:19.262+0000 7f70a58ec640 1 -- --> v2:172.21.15.43:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f70a0144000 con 0x7f70a00c86f0 2023-08-09T13:34:19.265 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:19.262+0000 7f70a58ec640 1 -- --> v2:172.21.15.203:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f70a00c9090 con 0x7f70a006c8d0 2023-08-09T13:34:19.265 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:19.262+0000 7f70a58ec640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f70a006c440 con 0x7f70a0143c20 2023-08-09T13:34:19.266 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:19.263+0000 7f7097fff640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f70a00c86f0 0x7f70a00c8b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:34:19.267 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:19.263+0000 7f7097fff640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f70a00c86f0 0x7f70a00c8b50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:44700/0 (socket says 172.21.15.43:44700) 2023-08-09T13:34:19.267 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:19.263+0000 7f7097fff640 1 -- 172.21.15.43:0/4012240266 learned_addr learned my addr 172.21.15.43:0/4012240266 (peer_addr_for_me v2:172.21.15.43:0/0) 2023-08-09T13:34:19.267 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:19.263+0000 7f709effd640 1 --2- 172.21.15.43:0/4012240266 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f70a0143c20 0x7f70a006f540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:34:19.267 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:19.263+0000 7f709f7fe640 1 --2- 172.21.15.43:0/4012240266 >> v2:172.21.15.203:3300/0 conn(0x7f70a006c8d0 0x7f70a006cd60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:34:19.267 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:19.263+0000 7f7097fff640 1 -- 172.21.15.43:0/4012240266 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f70a0143c20 msgr2=0x7f70a006f540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-09T13:34:19.267 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:19.263+0000 7f7097fff640 1 --2- 172.21.15.43:0/4012240266 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f70a0143c20 0x7f70a006f540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:34:19.267 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:19.263+0000 7f7097fff640 1 -- 172.21.15.43:0/4012240266 >> v2:172.21.15.203:3300/0 conn(0x7f70a006c8d0 msgr2=0x7f70a006cd60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-09T13:34:19.267 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:19.263+0000 7f7097fff640 1 --2- 172.21.15.43:0/4012240266 >> v2:172.21.15.203:3300/0 conn(0x7f70a006c8d0 0x7f70a006cd60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:34:19.268 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:19.263+0000 7f7097fff640 1 -- 172.21.15.43:0/4012240266 --> v2:172.21.15.43:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f70a006d2a0 con 0x7f70a00c86f0 2023-08-09T13:34:19.268 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:19.263+0000 7f709effd640 1 --2- 172.21.15.43:0/4012240266 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f70a0143c20 0x7f70a006f540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-08-09T13:34:19.269 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:19.264+0000 7f7097fff640 1 --2- 172.21.15.43:0/4012240266 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f70a00c86f0 0x7f70a00c8b50 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7f70a00c8680 tx=0x7f708802b860 comp rx=0 tx=0).ready entity=mon.0 client_cookie=56aa28a01ffafdbe server_cookie=0 in_seq=0 out_seq=0 2023-08-09T13:34:19.269 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:19.265+0000 7f709e7fc640 1 -- 172.21.15.43:0/4012240266 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f70880317c0 con 0x7f70a00c86f0 2023-08-09T13:34:19.269 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:19.265+0000 7f709e7fc640 1 -- 172.21.15.43:0/4012240266 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7088031920 con 0x7f70a00c86f0 2023-08-09T13:34:19.269 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:19.265+0000 7f709e7fc640 1 -- 172.21.15.43:0/4012240266 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7088031cb0 con 0x7f70a00c86f0 2023-08-09T13:34:19.269 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:19.265+0000 7f70a58ec640 1 -- 172.21.15.43:0/4012240266 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f70a00c86f0 msgr2=0x7f70a00c8b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:34:19.269 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:19.265+0000 7f70a58ec640 1 --2- 172.21.15.43:0/4012240266 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f70a00c86f0 0x7f70a00c8b50 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7f70a00c8680 tx=0x7f708802b860 comp rx=0 tx=0).stop 2023-08-09T13:34:19.270 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:19.266+0000 7f70a58ec640 1 -- 172.21.15.43:0/4012240266 shutdown_connections 2023-08-09T13:34:19.270 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:19.266+0000 7f70a58ec640 1 --2- 172.21.15.43:0/4012240266 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f70a00c86f0 0x7f70a00c8b50 unknown :-1 s=CLOSED pgs=110 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:34:19.270 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:19.266+0000 7f70a58ec640 1 --2- 172.21.15.43:0/4012240266 >> v2:172.21.15.203:3300/0 conn(0x7f70a006c8d0 0x7f70a006cd60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:34:19.270 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:19.266+0000 7f70a58ec640 1 --2- 172.21.15.43:0/4012240266 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f70a0143c20 0x7f70a006f540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:34:19.270 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:19.266+0000 7f70a58ec640 1 -- 172.21.15.43:0/4012240266 >> 172.21.15.43:0/4012240266 conn(0x7f70a00ce0a0 msgr2=0x7f70a00ce480 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-09T13:34:19.270 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:19.266+0000 7f70a58ec640 1 -- 172.21.15.43:0/4012240266 shutdown_connections 2023-08-09T13:34:19.270 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:19.266+0000 7f70a58ec640 1 -- 172.21.15.43:0/4012240266 wait complete. 2023-08-09T13:34:19.271 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:19.268+0000 7f70a58ec640 1 Processor -- start 2023-08-09T13:34:19.272 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:19.268+0000 7f70a58ec640 1 -- start start 2023-08-09T13:34:19.272 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:19.269+0000 7f70a58ec640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f70a006c8d0 0x7f70a00ac2a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:34:19.272 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:19.269+0000 7f70a58ec640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f70a00c86f0 0x7f70a013d7f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:34:19.272 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:19.269+0000 7f70a58ec640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f70a0143c20 0x7f70a00c7af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:34:19.272 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:19.269+0000 7f70a58ec640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f70a015a5c0 con 0x7f70a006c8d0 2023-08-09T13:34:19.272 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:19.269+0000 7f70a58ec640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f70a00c2fc0 con 0x7f70a0143c20 2023-08-09T13:34:19.272 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:19.269+0000 7f70a58ec640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f70a00ac8b0 con 0x7f70a00c86f0 2023-08-09T13:34:19.273 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:19.269+0000 7f709effd640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f70a006c8d0 0x7f70a00ac2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:34:19.273 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:19.269+0000 7f709effd640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f70a006c8d0 0x7f70a00ac2a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:44708/0 (socket says 172.21.15.43:44708) 2023-08-09T13:34:19.273 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:19.269+0000 7f709effd640 1 -- 172.21.15.43:0/1247121044 learned_addr learned my addr 172.21.15.43:0/1247121044 (peer_addr_for_me v2:172.21.15.43:0/0) 2023-08-09T13:34:19.274 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:19.269+0000 7f709effd640 1 -- 172.21.15.43:0/1247121044 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f70a00c86f0 msgr2=0x7f70a013d7f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:34:19.274 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:19.269+0000 7f709effd640 1 --2- 172.21.15.43:0/1247121044 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f70a00c86f0 0x7f70a013d7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:34:19.274 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:19.269+0000 7f709effd640 1 -- 172.21.15.43:0/1247121044 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f70a0143c20 msgr2=0x7f70a00c7af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:34:19.274 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:19.269+0000 7f709effd640 1 --2- 172.21.15.43:0/1247121044 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f70a0143c20 0x7f70a00c7af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:34:19.274 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:19.269+0000 7f709effd640 1 -- 172.21.15.43:0/1247121044 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f70880094d0 con 0x7f70a006c8d0 2023-08-09T13:34:19.275 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:19.270+0000 7f709effd640 1 --2- 172.21.15.43:0/1247121044 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f70a006c8d0 0x7f70a00ac2a0 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7f709000e810 tx=0x7f709000ecd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-09T13:34:19.275 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:19.270+0000 7f709cff9640 1 -- 172.21.15.43:0/1247121044 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7090004030 con 0x7f70a006c8d0 2023-08-09T13:34:19.275 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:19.270+0000 7f709cff9640 1 -- 172.21.15.43:0/1247121044 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7090004190 con 0x7f70a006c8d0 2023-08-09T13:34:19.275 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:19.270+0000 7f709cff9640 1 -- 172.21.15.43:0/1247121044 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7090004520 con 0x7f70a006c8d0 2023-08-09T13:34:19.275 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:19.270+0000 7f70a58ec640 1 -- 172.21.15.43:0/1247121044 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f70a00acb90 con 0x7f70a006c8d0 2023-08-09T13:34:19.276 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:19.270+0000 7f70a58ec640 1 -- 172.21.15.43:0/1247121044 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f70a00c8030 con 0x7f70a006c8d0 2023-08-09T13:34:19.276 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:19.271+0000 7f70a58ec640 1 -- 172.21.15.43:0/1247121044 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f70a00739f0 con 0x7f70a006c8d0 2023-08-09T13:34:19.276 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:19.273+0000 7f709cff9640 1 -- 172.21.15.43:0/1247121044 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 5) v1 ==== 88341+0+0 (secure 0 0 0) 0x7f709000c6d0 con 0x7f70a006c8d0 2023-08-09T13:34:19.277 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:19.275+0000 7f709cff9640 1 --2- 172.21.15.43:0/1247121044 >> [v2:172.21.15.203:6832/830097474,v1:172.21.15.203:6833/830097474] conn(0x7f708006a050 0x7f708006c510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:34:19.277 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:19.275+0000 7f709cff9640 1 -- 172.21.15.43:0/1247121044 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6813+0+0 (secure 0 0 0) 0x7f7090088550 con 0x7f70a006c8d0 2023-08-09T13:34:19.277 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:19.275+0000 7f7097fff640 1 --2- 172.21.15.43:0/1247121044 >> [v2:172.21.15.203:6832/830097474,v1:172.21.15.203:6833/830097474] conn(0x7f708006a050 0x7f708006c510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:34:19.279 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:19.277+0000 7f7097fff640 1 --2- 172.21.15.43:0/1247121044 >> [v2:172.21.15.203:6832/830097474,v1:172.21.15.203:6833/830097474] conn(0x7f708006a050 0x7f708006c510 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f7088036000 tx=0x7f7088030000 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-09T13:34:19.284 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:19.282+0000 7f709cff9640 1 -- 172.21.15.43:0/1247121044 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178066 (secure 0 0 0) 0x7f70900573c0 con 0x7f70a006c8d0 2023-08-09T13:34:19.492 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:19.489+0000 7f70a58ec640 1 -- 172.21.15.43:0/1247121044 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7f70a00cec90 con 0x7f70a006c8d0 2023-08-09T13:34:19.492 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:19.490+0000 7f709cff9640 1 -- 172.21.15.43:0/1247121044 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v24) v1 ==== 80+0+0 (secure 0 0 0) 0x7f7090056f50 con 0x7f70a006c8d0 2023-08-09T13:34:19.493 INFO:teuthology.orchestra.run.smithi043.stderr:listed 0 entries 2023-08-09T13:34:19.496 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:19.493+0000 7f70a58ec640 1 -- 172.21.15.43:0/1247121044 >> [v2:172.21.15.203:6832/830097474,v1:172.21.15.203:6833/830097474] conn(0x7f708006a050 msgr2=0x7f708006c510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:34:19.496 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:19.493+0000 7f70a58ec640 1 --2- 172.21.15.43:0/1247121044 >> [v2:172.21.15.203:6832/830097474,v1:172.21.15.203:6833/830097474] conn(0x7f708006a050 0x7f708006c510 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f7088036000 tx=0x7f7088030000 comp rx=0 tx=0).stop 2023-08-09T13:34:19.496 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:19.493+0000 7f70a58ec640 1 -- 172.21.15.43:0/1247121044 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f70a006c8d0 msgr2=0x7f70a00ac2a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:34:19.496 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:19.493+0000 7f70a58ec640 1 --2- 172.21.15.43:0/1247121044 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f70a006c8d0 0x7f70a00ac2a0 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7f709000e810 tx=0x7f709000ecd0 comp rx=0 tx=0).stop 2023-08-09T13:34:19.496 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:19.493+0000 7f70a58ec640 1 -- 172.21.15.43:0/1247121044 shutdown_connections 2023-08-09T13:34:19.497 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:19.493+0000 7f70a58ec640 1 --2- 172.21.15.43:0/1247121044 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f70a0143c20 0x7f70a00c7af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:34:19.497 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:19.493+0000 7f70a58ec640 1 --2- 172.21.15.43:0/1247121044 >> [v2:172.21.15.203:6832/830097474,v1:172.21.15.203:6833/830097474] conn(0x7f708006a050 0x7f708006c510 unknown :-1 s=CLOSED pgs=79 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:34:19.497 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:19.493+0000 7f70a58ec640 1 --2- 172.21.15.43:0/1247121044 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f70a00c86f0 0x7f70a013d7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:34:19.497 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:19.493+0000 7f70a58ec640 1 --2- 172.21.15.43:0/1247121044 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f70a006c8d0 0x7f70a00ac2a0 unknown :-1 s=CLOSED pgs=111 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:34:19.497 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:19.493+0000 7f70a58ec640 1 -- 172.21.15.43:0/1247121044 >> 172.21.15.43:0/1247121044 conn(0x7f70a00ce0a0 msgr2=0x7f70a00c9220 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-09T13:34:19.497 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:19.493+0000 7f70a58ec640 1 -- 172.21.15.43:0/1247121044 shutdown_connections 2023-08-09T13:34:19.497 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:19.493+0000 7f70a58ec640 1 -- 172.21.15.43:0/1247121044 wait complete. 2023-08-09T13:34:19.513 DEBUG:teuthology.orchestra.run.smithi043:> set -ex 2023-08-09T13:34:19.514 DEBUG:teuthology.orchestra.run.smithi043:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-09T13:34:19.569 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-08-09T13:34:19.800 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:19.797+0000 7f1dfd48a640 1 Processor -- start 2023-08-09T13:34:19.800 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:19.797+0000 7f1dfd48a640 1 -- start start 2023-08-09T13:34:19.801 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:19.799+0000 7f1dfd48a640 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7f1df80c8750 0x7f1df80ad340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:34:19.802 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:19.799+0000 7f1dfd48a640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f1df80c8b30 0x7f1df806f410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:34:19.802 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:19.799+0000 7f1dfd48a640 1 -- --> v1:172.21.15.203:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1df80cbd90 con 0x7f1df80acdf0 2023-08-09T13:34:19.802 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:19.799+0000 7f1dfd48a640 1 -- --> v2:172.21.15.43:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1df815e810 con 0x7f1df80c8b30 2023-08-09T13:34:19.802 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:19.799+0000 7f1dfd48a640 1 -- --> v2:172.21.15.203:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1df80c9f70 con 0x7f1df80c8750 2023-08-09T13:34:19.803 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:19.799+0000 7f1df77fe640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f1df80c8b30 0x7f1df806f410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:34:19.803 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:19.799+0000 7f1df77fe640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f1df80c8b30 0x7f1df806f410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:44724/0 (socket says 172.21.15.43:44724) 2023-08-09T13:34:19.803 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:19.799+0000 7f1df77fe640 1 -- 172.21.15.43:0/1134815902 learned_addr learned my addr 172.21.15.43:0/1134815902 (peer_addr_for_me v2:172.21.15.43:0/0) 2023-08-09T13:34:19.803 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:19.800+0000 7f1df7fff640 1 --2- 172.21.15.43:0/1134815902 >> v2:172.21.15.203:3300/0 conn(0x7f1df80c8750 0x7f1df80ad340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:34:19.803 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:19.800+0000 7f1df77fe640 1 -- 172.21.15.43:0/1134815902 >> v2:172.21.15.203:3300/0 conn(0x7f1df80c8750 msgr2=0x7f1df80ad340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-09T13:34:19.804 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:19.800+0000 7f1df77fe640 1 --2- 172.21.15.43:0/1134815902 >> v2:172.21.15.203:3300/0 conn(0x7f1df80c8750 0x7f1df80ad340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:34:19.804 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:19.800+0000 7f1df77fe640 1 -- 172.21.15.43:0/1134815902 >> v1:172.21.15.203:6789/0 conn(0x7f1df80acdf0 legacy=0x7f1df8070f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-09T13:34:19.804 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:19.800+0000 7f1df77fe640 1 -- 172.21.15.43:0/1134815902 --> v2:172.21.15.43:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1df80ca2b0 con 0x7f1df80c8b30 2023-08-09T13:34:19.804 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:19.800+0000 7f1df7fff640 1 --2- 172.21.15.43:0/1134815902 >> v2:172.21.15.203:3300/0 conn(0x7f1df80c8750 0x7f1df80ad340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-08-09T13:34:19.805 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:19.801+0000 7f1df77fe640 1 --2- 172.21.15.43:0/1134815902 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f1df80c8b30 0x7f1df806f410 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7f1df80cb9a0 tx=0x7f1dec02b910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6180f3052ba37ab4 server_cookie=0 in_seq=0 out_seq=0 2023-08-09T13:34:19.805 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:19.802+0000 7f1df6ffd640 1 -- 172.21.15.43:0/1134815902 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1dec031870 con 0x7f1df80c8b30 2023-08-09T13:34:19.805 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:19.802+0000 7f1df6ffd640 1 -- 172.21.15.43:0/1134815902 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1dec0319d0 con 0x7f1df80c8b30 2023-08-09T13:34:19.806 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:19.802+0000 7f1df6ffd640 1 -- 172.21.15.43:0/1134815902 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1dec031d60 con 0x7f1df80c8b30 2023-08-09T13:34:19.806 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:19.802+0000 7f1dfd48a640 1 -- 172.21.15.43:0/1134815902 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f1df80c8b30 msgr2=0x7f1df806f410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:34:19.806 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:19.802+0000 7f1dfd48a640 1 --2- 172.21.15.43:0/1134815902 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f1df80c8b30 0x7f1df806f410 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7f1df80cb9a0 tx=0x7f1dec02b910 comp rx=0 tx=0).stop 2023-08-09T13:34:19.806 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:19.802+0000 7f1dfd48a640 1 -- 172.21.15.43:0/1134815902 shutdown_connections 2023-08-09T13:34:19.806 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:19.802+0000 7f1dfd48a640 1 --2- 172.21.15.43:0/1134815902 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f1df80c8b30 0x7f1df806f410 unknown :-1 s=CLOSED pgs=112 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:34:19.806 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:19.802+0000 7f1dfd48a640 1 --2- 172.21.15.43:0/1134815902 >> v2:172.21.15.203:3300/0 conn(0x7f1df80c8750 0x7f1df80ad340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:34:19.806 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:19.802+0000 7f1dfd48a640 1 -- 172.21.15.43:0/1134815902 >> 172.21.15.43:0/1134815902 conn(0x7f1df80cde90 msgr2=0x7f1df80ce2a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-09T13:34:19.806 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:19.802+0000 7f1dfd48a640 1 -- 172.21.15.43:0/1134815902 shutdown_connections 2023-08-09T13:34:19.807 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:19.802+0000 7f1dfd48a640 1 -- 172.21.15.43:0/1134815902 wait complete. 2023-08-09T13:34:19.807 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:19.804+0000 7f1dfd48a640 1 Processor -- start 2023-08-09T13:34:19.807 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:19.804+0000 7f1dfd48a640 1 -- start start 2023-08-09T13:34:19.808 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:19.805+0000 7f1dfd48a640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f1df80acdf0 0x7f1df8159450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:34:19.808 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:19.805+0000 7f1dfd48a640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f1df80c8750 0x7f1df81e4540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:34:19.809 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:19.805+0000 7f1dfd48a640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f1df80c8b30 0x7f1df806ba90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:34:19.809 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:19.805+0000 7f1dfd48a640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1df8159990 con 0x7f1df80c8750 2023-08-09T13:34:19.809 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:19.805+0000 7f1dfd48a640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1df81e4a80 con 0x7f1df80c8b30 2023-08-09T13:34:19.809 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:19.805+0000 7f1dfd48a640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1df80c36a0 con 0x7f1df80acdf0 2023-08-09T13:34:19.809 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:19.806+0000 7f1df77fe640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f1df80c8750 0x7f1df81e4540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:34:19.809 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:19.806+0000 7f1df77fe640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f1df80c8750 0x7f1df81e4540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:44738/0 (socket says 172.21.15.43:44738) 2023-08-09T13:34:19.809 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:19.806+0000 7f1df77fe640 1 -- 172.21.15.43:0/1469486002 learned_addr learned my addr 172.21.15.43:0/1469486002 (peer_addr_for_me v2:172.21.15.43:0/0) 2023-08-09T13:34:19.809 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:19.806+0000 7f1df7fff640 1 --2- 172.21.15.43:0/1469486002 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f1df80acdf0 0x7f1df8159450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:34:19.810 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:19.806+0000 7f1dfcc89640 1 --2- 172.21.15.43:0/1469486002 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f1df80c8b30 0x7f1df806ba90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:34:19.810 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:19.806+0000 7f1df77fe640 1 -- 172.21.15.43:0/1469486002 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f1df80acdf0 msgr2=0x7f1df8159450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:34:19.811 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:19.806+0000 7f1df77fe640 1 --2- 172.21.15.43:0/1469486002 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f1df80acdf0 0x7f1df8159450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:34:19.811 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:19.806+0000 7f1df77fe640 1 -- 172.21.15.43:0/1469486002 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f1df80c8b30 msgr2=0x7f1df806ba90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:34:19.811 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:19.806+0000 7f1df77fe640 1 --2- 172.21.15.43:0/1469486002 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f1df80c8b30 0x7f1df806ba90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:34:19.811 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:19.806+0000 7f1df77fe640 1 -- 172.21.15.43:0/1469486002 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1dec0094d0 con 0x7f1df80c8750 2023-08-09T13:34:19.811 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:19.807+0000 7f1df77fe640 1 --2- 172.21.15.43:0/1469486002 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f1df80c8750 0x7f1df81e4540 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7f1dec000c00 tx=0x7f1dec0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-09T13:34:19.811 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:19.807+0000 7f1df4ff9640 1 -- 172.21.15.43:0/1469486002 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1dec031840 con 0x7f1df80c8750 2023-08-09T13:34:19.812 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:19.807+0000 7f1df4ff9640 1 -- 172.21.15.43:0/1469486002 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1dec0443f0 con 0x7f1df80c8750 2023-08-09T13:34:19.812 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:19.807+0000 7f1df4ff9640 1 -- 172.21.15.43:0/1469486002 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1dec0447a0 con 0x7f1df80c8750 2023-08-09T13:34:19.812 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:19.807+0000 7f1dfd48a640 1 -- 172.21.15.43:0/1469486002 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1df80c3980 con 0x7f1df80c8750 2023-08-09T13:34:19.812 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:19.807+0000 7f1dfd48a640 1 -- 172.21.15.43:0/1469486002 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1df80c3f40 con 0x7f1df80c8750 2023-08-09T13:34:19.813 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:19.808+0000 7f1dfd48a640 1 -- 172.21.15.43:0/1469486002 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1df80c9230 con 0x7f1df80c8750 2023-08-09T13:34:19.813 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:19.809+0000 7f1df4ff9640 1 -- 172.21.15.43:0/1469486002 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 5) v1 ==== 88341+0+0 (secure 0 0 0) 0x7f1dec04d450 con 0x7f1df80c8750 2023-08-09T13:34:19.813 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:19.810+0000 7f1df4ff9640 1 --2- 172.21.15.43:0/1469486002 >> [v2:172.21.15.203:6832/830097474,v1:172.21.15.203:6833/830097474] conn(0x7f1de406a080 0x7f1de406c540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:34:19.813 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:19.810+0000 7f1df4ff9640 1 -- 172.21.15.43:0/1469486002 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6813+0+0 (secure 0 0 0) 0x7f1dec02d040 con 0x7f1df80c8750 2023-08-09T13:34:19.813 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:19.811+0000 7f1df7fff640 1 --2- 172.21.15.43:0/1469486002 >> [v2:172.21.15.203:6832/830097474,v1:172.21.15.203:6833/830097474] conn(0x7f1de406a080 0x7f1de406c540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:34:19.814 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:19.812+0000 7f1df7fff640 1 --2- 172.21.15.43:0/1469486002 >> [v2:172.21.15.203:6832/830097474,v1:172.21.15.203:6833/830097474] conn(0x7f1de406a080 0x7f1de406c540 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f1de800a990 tx=0x7f1de8008040 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-09T13:34:19.817 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:19.815+0000 7f1df4ff9640 1 -- 172.21.15.43:0/1469486002 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178066 (secure 0 0 0) 0x7f1dec076400 con 0x7f1df80c8750 2023-08-09T13:34:20.016 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:20.013+0000 7f1dfd48a640 1 -- 172.21.15.43:0/1469486002 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f1df80739f0 con 0x7f1df80c8750 2023-08-09T13:34:20.018 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:20.016+0000 7f1df4ff9640 1 -- 172.21.15.43:0/1469486002 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v24) v1 ==== 74+0+20050 (secure 0 0 0) 0x7f1dec035020 con 0x7f1df80c8750 2023-08-09T13:34:20.019 INFO:teuthology.orchestra.run.smithi043.stdout: 2023-08-09T13:34:20.019 INFO:teuthology.orchestra.run.smithi043.stdout:{"epoch":24,"fsid":"daac87ee-1337-4250-a955-aea4e408a39b","created":"2023-08-09T13:33:41.701198+0000","modified":"2023-08-09T13:34:07.709429+0000","last_up_change":"2023-08-09T13:33:53.783453+0000","last_in_change":"2023-08-09T13:33:45.148904+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":5,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":"rbd","create_time":"2023-08-09T13:33:54.365137+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":".mgr","create_time":"2023-08-09T13:33:54.831803+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_metadata","create_time":"2023-08-09T13:34:00.207974+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data","create_time":"2023-08-09T13:34:01.614765+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":5,"pool_name":"cephfs_data_ec","create_time":"2023-08-09T13:34:05.239583+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"24","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"cbd49ecd-e896-4fbd-864e-5c37a829b165","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6800","nonce":502667335},{"type":"v1","addr":"172.21.15.43:6801","nonce":502667335}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6802","nonce":502667335},{"type":"v1","addr":"172.21.15.43:6803","nonce":502667335}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6806","nonce":502667335},{"type":"v1","addr":"172.21.15.43:6807","nonce":502667335}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6804","nonce":502667335},{"type":"v1","addr":"172.21.15.43:6805","nonce":502667335}]},"public_addr":"172.21.15.43:6801/502667335","cluster_addr":"172.21.15.43:6803/502667335","heartbeat_back_addr":"172.21.15.43:6807/502667335","heartbeat_front_addr":"172.21.15.43:6805/502667335","state":["exists","up"]},{"osd":1,"uuid":"75c7e0fe-1ec7-4fe8-a2f2-3661a65b8500","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6808","nonce":3289134610},{"type":"v1","addr":"172.21.15.43:6809","nonce":3289134610}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6810","nonce":3289134610},{"type":"v1","addr":"172.21.15.43:6812","nonce":3289134610}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6821","nonce":3289134610},{"type":"v1","addr":"172.21.15.43:6824","nonce":3289134610}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6815","nonce":3289134610},{"type":"v1","addr":"172.21.15.43:6818","nonce":3289134610}]},"public_addr":"172.21.15.43:6809/3289134610","cluster_addr":"172.21.15.43:6812/3289134610","heartbeat_back_addr":"172.21.15.43:6824/3289134610","heartbeat_front_addr":"172.21.15.43:6818/3289134610","state":["exists","up"]},{"osd":2,"uuid":"bb5c41e9-7f5f-4125-9b01-3bb8bfc2ea83","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6811","nonce":4079709736},{"type":"v1","addr":"172.21.15.43:6814","nonce":4079709736}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6817","nonce":4079709736},{"type":"v1","addr":"172.21.15.43:6820","nonce":4079709736}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6829","nonce":4079709736},{"type":"v1","addr":"172.21.15.43:6831","nonce":4079709736}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6823","nonce":4079709736},{"type":"v1","addr":"172.21.15.43:6826","nonce":4079709736}]},"public_addr":"172.21.15.43:6814/4079709736","cluster_addr":"172.21.15.43:6820/4079709736","heartbeat_back_addr":"172.21.15.43:6831/4079709736","heartbeat_front_addr":"172.21.15.43:6826/4079709736","state":["exists","up"]},{"osd":3,"uuid":"bfe4c8b4-694a-465c-87e6-2b2aab852613","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6813","nonce":1709538759},{"type":"v1","addr":"172.21.15.43:6816","nonce":1709538759}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6819","nonce":1709538759},{"type":"v1","addr":"172.21.15.43:6822","nonce":1709538759}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6828","nonce":1709538759},{"type":"v1","addr":"172.21.15.43:6830","nonce":1709538759}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6825","nonce":1709538759},{"type":"v1","addr":"172.21.15.43:6827","nonce":1709538759}]},"public_addr":"172.21.15.43:6816/1709538759","cluster_addr":"172.21.15.43:6822/1709538759","heartbeat_back_addr":"172.21.15.43:6830/1709538759","heartbeat_front_addr":"172.21.15.43:6827/1709538759","state":["exists","up"]},{"osd":4,"uuid":"fa2e8a77-aba1-4756-853e-0d174a5bdeba","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6800","nonce":4196866481},{"type":"v1","addr":"172.21.15.203:6801","nonce":4196866481}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6802","nonce":4196866481},{"type":"v1","addr":"172.21.15.203:6803","nonce":4196866481}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6806","nonce":4196866481},{"type":"v1","addr":"172.21.15.203:6807","nonce":4196866481}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6804","nonce":4196866481},{"type":"v1","addr":"172.21.15.203:6805","nonce":4196866481}]},"public_addr":"172.21.15.203:6801/4196866481","cluster_addr":"172.21.15.203:6803/4196866481","heartbeat_back_addr":"172.21.15.203:6807/4196866481","heartbeat_front_addr":"172.21.15.203:6805/4196866481","state":["exists","up"]},{"osd":5,"uuid":"81811e4b-4d24-4f51-a82c-f1d700a2a00d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6816","nonce":3059290081},{"type":"v1","addr":"172.21.15.203:6817","nonce":3059290081}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6818","nonce":3059290081},{"type":"v1","addr":"172.21.15.203:6819","nonce":3059290081}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6822","nonce":3059290081},{"type":"v1","addr":"172.21.15.203:6823","nonce":3059290081}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6820","nonce":3059290081},{"type":"v1","addr":"172.21.15.203:6821","nonce":3059290081}]},"public_addr":"172.21.15.203:6817/3059290081","cluster_addr":"172.21.15.203:6819/3059290081","heartbeat_back_addr":"172.21.15.203:6823/3059290081","heartbeat_front_addr":"172.21.15.203:6821/3059290081","state":["exists","up"]},{"osd":6,"uuid":"307bac4d-6a5b-443d-af75-56d695d7674f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6824","nonce":512155569},{"type":"v1","addr":"172.21.15.203:6825","nonce":512155569}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6826","nonce":512155569},{"type":"v1","addr":"172.21.15.203:6827","nonce":512155569}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6830","nonce":512155569},{"type":"v1","addr":"172.21.15.203:6831","nonce":512155569}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6828","nonce":512155569},{"type":"v1","addr":"172.21.15.203:6829","nonce":512155569}]},"public_addr":"172.21.15.203:6825/512155569","cluster_addr":"172.21.15.203:6827/512155569","heartbeat_back_addr":"172.21.15.203:6831/512155569","heartbeat_front_addr":"172.21.15.203:6829/512155569","state":["exists","up"]},{"osd":7,"uuid":"a95dc7ca-9f1b-4295-af70-68a22a2d7d5e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6808","nonce":3331450438},{"type":"v1","addr":"172.21.15.203:6809","nonce":3331450438}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6810","nonce":3331450438},{"type":"v1","addr":"172.21.15.203:6811","nonce":3331450438}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6814","nonce":3331450438},{"type":"v1","addr":"172.21.15.203:6815","nonce":3331450438}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6812","nonce":3331450438},{"type":"v1","addr":"172.21.15.203:6813","nonce":3331450438}]},"public_addr":"172.21.15.203:6809/3331450438","cluster_addr":"172.21.15.203:6811/3331450438","heartbeat_back_addr":"172.21.15.203:6815/3331450438","heartbeat_front_addr":"172.21.15.203:6813/3331450438","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-09T13:33:50.547652+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-09T13:33:50.589469+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-09T13:33:50.561724+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-09T13:33:50.599650+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-09T13:33:50.411855+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-09T13:33:50.219681+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-09T13:33:50.504664+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-09T13:33:50.441248+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-08-09T13:34:20.021 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:20.018+0000 7f1dfd48a640 1 -- 172.21.15.43:0/1469486002 >> [v2:172.21.15.203:6832/830097474,v1:172.21.15.203:6833/830097474] conn(0x7f1de406a080 msgr2=0x7f1de406c540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:34:20.021 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:20.018+0000 7f1dfd48a640 1 --2- 172.21.15.43:0/1469486002 >> [v2:172.21.15.203:6832/830097474,v1:172.21.15.203:6833/830097474] conn(0x7f1de406a080 0x7f1de406c540 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f1de800a990 tx=0x7f1de8008040 comp rx=0 tx=0).stop 2023-08-09T13:34:20.021 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:20.018+0000 7f1dfd48a640 1 -- 172.21.15.43:0/1469486002 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f1df80c8750 msgr2=0x7f1df81e4540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:34:20.021 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:20.018+0000 7f1dfd48a640 1 --2- 172.21.15.43:0/1469486002 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f1df80c8750 0x7f1df81e4540 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7f1dec000c00 tx=0x7f1dec0046c0 comp rx=0 tx=0).stop 2023-08-09T13:34:20.022 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:20.018+0000 7f1dfd48a640 1 -- 172.21.15.43:0/1469486002 shutdown_connections 2023-08-09T13:34:20.022 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:20.018+0000 7f1dfd48a640 1 --2- 172.21.15.43:0/1469486002 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f1df80c8b30 0x7f1df806ba90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:34:20.022 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:20.018+0000 7f1dfd48a640 1 --2- 172.21.15.43:0/1469486002 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f1df80c8750 0x7f1df81e4540 unknown :-1 s=CLOSED pgs=113 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:34:20.022 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:20.018+0000 7f1dfd48a640 1 --2- 172.21.15.43:0/1469486002 >> [v2:172.21.15.203:6832/830097474,v1:172.21.15.203:6833/830097474] conn(0x7f1de406a080 0x7f1de406c540 unknown :-1 s=CLOSED pgs=80 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:34:20.022 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:20.018+0000 7f1dfd48a640 1 --2- 172.21.15.43:0/1469486002 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f1df80acdf0 0x7f1df8159450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:34:20.022 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:20.018+0000 7f1dfd48a640 1 -- 172.21.15.43:0/1469486002 >> 172.21.15.43:0/1469486002 conn(0x7f1df80cde90 msgr2=0x7f1df8159b90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-09T13:34:20.022 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:20.019+0000 7f1dfd48a640 1 -- 172.21.15.43:0/1469486002 shutdown_connections 2023-08-09T13:34:20.022 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:20.019+0000 7f1dfd48a640 1 -- 172.21.15.43:0/1469486002 wait complete. 2023-08-09T13:34:20.042 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': 'rbd', 'create_time': '2023-08-09T13:33:54.365137+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}, 'read_balance': {'score_acting': 2, 'score_stable': 2, 'optimal_score': 1, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': '.mgr', 'create_time': '2023-08-09T13:33:54.831803+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_metadata', 'create_time': '2023-08-09T13:34:00.207974+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data', 'create_time': '2023-08-09T13:34:01.614765+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 2, 'score_stable': 2, 'optimal_score': 1, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 5, 'pool_name': 'cephfs_data_ec', 'create_time': '2023-08-09T13:34:05.239583+0000', 'flags': 5, 'flags_names': 'hashpspool,ec_overwrites', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '24', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2023-08-09T13:34:20.043 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get rbd pg_num 2023-08-09T13:34:20.261 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:20.258+0000 7f66a2fa2640 1 Processor -- start 2023-08-09T13:34:20.261 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:20.258+0000 7f66a2fa2640 1 -- start start 2023-08-09T13:34:20.263 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:20.260+0000 7f66a2fa2640 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7f669c15a1b0 0x7f669c0b1ad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:34:20.264 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:20.260+0000 7f66a2fa2640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f669c0cd630 0x7f669c0b2010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:34:20.264 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:20.260+0000 7f66a2fa2640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f669c15e9c0 0x7f669c0cdc20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:34:20.264 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:20.260+0000 7f66a2fa2640 1 -- --> v2:172.21.15.43:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f669c0cc6c0 con 0x7f669c0cd630 2023-08-09T13:34:20.264 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:20.260+0000 7f66a2fa2640 1 -- --> v2:172.21.15.203:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f669c15dee0 con 0x7f669c15a1b0 2023-08-09T13:34:20.265 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:20.260+0000 7f66a2fa2640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f669c15e1c0 con 0x7f669c15e9c0 2023-08-09T13:34:20.265 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:20.260+0000 7f66a179f640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f669c0cd630 0x7f669c0b2010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:34:20.265 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:20.260+0000 7f66a179f640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f669c0cd630 0x7f669c0b2010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:44740/0 (socket says 172.21.15.43:44740) 2023-08-09T13:34:20.265 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:20.260+0000 7f66a179f640 1 -- 172.21.15.43:0/3354313520 learned_addr learned my addr 172.21.15.43:0/3354313520 (peer_addr_for_me v2:172.21.15.43:0/0) 2023-08-09T13:34:20.266 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:20.260+0000 7f66a27a1640 1 --2- 172.21.15.43:0/3354313520 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f669c15e9c0 0x7f669c0cdc20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:34:20.266 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:20.260+0000 7f66a179f640 1 -- 172.21.15.43:0/3354313520 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f669c15e9c0 msgr2=0x7f669c0cdc20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-09T13:34:20.266 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:20.260+0000 7f66a1fa0640 1 --2- 172.21.15.43:0/3354313520 >> v2:172.21.15.203:3300/0 conn(0x7f669c15a1b0 0x7f669c0b1ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:34:20.267 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:20.261+0000 7f66a179f640 1 --2- 172.21.15.43:0/3354313520 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f669c15e9c0 0x7f669c0cdc20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:34:20.267 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:20.261+0000 7f66a179f640 1 -- 172.21.15.43:0/3354313520 >> v2:172.21.15.203:3300/0 conn(0x7f669c15a1b0 msgr2=0x7f669c0b1ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-09T13:34:20.267 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:20.261+0000 7f66a179f640 1 --2- 172.21.15.43:0/3354313520 >> v2:172.21.15.203:3300/0 conn(0x7f669c15a1b0 0x7f669c0b1ad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:34:20.267 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:20.261+0000 7f66a179f640 1 -- 172.21.15.43:0/3354313520 --> v2:172.21.15.43:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f669c15e4a0 con 0x7f669c0cd630 2023-08-09T13:34:20.267 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:20.261+0000 7f66a1fa0640 1 --2- 172.21.15.43:0/3354313520 >> v2:172.21.15.203:3300/0 conn(0x7f669c15a1b0 0x7f669c0b1ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-08-09T13:34:20.269 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:20.262+0000 7f66a179f640 1 --2- 172.21.15.43:0/3354313520 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f669c0cd630 0x7f669c0b2010 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7f669c06a7c0 tx=0x7f668c02b910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=92b3272994d5f084 server_cookie=0 in_seq=0 out_seq=0 2023-08-09T13:34:20.269 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:20.262+0000 7f66a0f9e640 1 -- 172.21.15.43:0/3354313520 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f668c031870 con 0x7f669c0cd630 2023-08-09T13:34:20.269 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:20.262+0000 7f66a0f9e640 1 -- 172.21.15.43:0/3354313520 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f668c0319d0 con 0x7f669c0cd630 2023-08-09T13:34:20.270 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:20.262+0000 7f66a0f9e640 1 -- 172.21.15.43:0/3354313520 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f668c031d60 con 0x7f669c0cd630 2023-08-09T13:34:20.270 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:20.263+0000 7f66a2fa2640 1 -- 172.21.15.43:0/3354313520 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f669c0cd630 msgr2=0x7f669c0b2010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:34:20.270 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:20.263+0000 7f66a2fa2640 1 --2- 172.21.15.43:0/3354313520 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f669c0cd630 0x7f669c0b2010 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7f669c06a7c0 tx=0x7f668c02b910 comp rx=0 tx=0).stop 2023-08-09T13:34:20.270 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:20.263+0000 7f66a2fa2640 1 -- 172.21.15.43:0/3354313520 shutdown_connections 2023-08-09T13:34:20.270 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:20.263+0000 7f66a2fa2640 1 --2- 172.21.15.43:0/3354313520 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f669c0cd630 0x7f669c0b2010 unknown :-1 s=CLOSED pgs=114 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:34:20.270 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:20.263+0000 7f66a2fa2640 1 --2- 172.21.15.43:0/3354313520 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f669c15e9c0 0x7f669c0cdc20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:34:20.271 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:20.263+0000 7f66a2fa2640 1 --2- 172.21.15.43:0/3354313520 >> v2:172.21.15.203:3300/0 conn(0x7f669c15a1b0 0x7f669c0b1ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:34:20.271 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:20.263+0000 7f66a2fa2640 1 -- 172.21.15.43:0/3354313520 >> 172.21.15.43:0/3354313520 conn(0x7f669c0ccad0 msgr2=0x7f669c0cceb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-09T13:34:20.271 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:20.263+0000 7f66a2fa2640 1 -- 172.21.15.43:0/3354313520 shutdown_connections 2023-08-09T13:34:20.271 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:20.263+0000 7f66a2fa2640 1 -- 172.21.15.43:0/3354313520 wait complete. 2023-08-09T13:34:20.271 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:20.265+0000 7f66a2fa2640 1 Processor -- start 2023-08-09T13:34:20.271 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:20.265+0000 7f66a2fa2640 1 -- start start 2023-08-09T13:34:20.271 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:20.266+0000 7f66a2fa2640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f669c0cd630 0x7f669c0c7350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:34:20.271 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:20.266+0000 7f66a2fa2640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f669c15a1b0 0x7f669c1f1570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:34:20.272 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:20.266+0000 7f66a2fa2640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f669c15e9c0 0x7f669c151cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:34:20.272 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:20.266+0000 7f66a2fa2640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f669c1f1cf0 con 0x7f669c15a1b0 2023-08-09T13:34:20.272 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:20.266+0000 7f66a2fa2640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f669c152230 con 0x7f669c0cd630 2023-08-09T13:34:20.272 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:20.266+0000 7f66a2fa2640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f669c1524d0 con 0x7f669c15e9c0 2023-08-09T13:34:20.272 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:20.266+0000 7f66a179f640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f669c15a1b0 0x7f669c1f1570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:34:20.272 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:20.266+0000 7f66a179f640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f669c15a1b0 0x7f669c1f1570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:44750/0 (socket says 172.21.15.43:44750) 2023-08-09T13:34:20.272 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:20.266+0000 7f66a179f640 1 -- 172.21.15.43:0/3197869659 learned_addr learned my addr 172.21.15.43:0/3197869659 (peer_addr_for_me v2:172.21.15.43:0/0) 2023-08-09T13:34:20.273 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:20.267+0000 7f66a1fa0640 1 --2- 172.21.15.43:0/3197869659 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f669c0cd630 0x7f669c0c7350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:34:20.273 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:20.267+0000 7f66a27a1640 1 --2- 172.21.15.43:0/3197869659 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f669c15e9c0 0x7f669c151cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:34:20.273 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:20.267+0000 7f66a179f640 1 -- 172.21.15.43:0/3197869659 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f669c15e9c0 msgr2=0x7f669c151cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:34:20.273 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:20.267+0000 7f66a179f640 1 --2- 172.21.15.43:0/3197869659 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f669c15e9c0 0x7f669c151cf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:34:20.273 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:20.267+0000 7f66a179f640 1 -- 172.21.15.43:0/3197869659 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f669c0cd630 msgr2=0x7f669c0c7350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:34:20.273 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:20.267+0000 7f66a179f640 1 --2- 172.21.15.43:0/3197869659 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f669c0cd630 0x7f669c0c7350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:34:20.273 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:20.267+0000 7f66a179f640 1 -- 172.21.15.43:0/3197869659 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f668c0094d0 con 0x7f669c15a1b0 2023-08-09T13:34:20.274 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:20.267+0000 7f66a27a1640 1 --2- 172.21.15.43:0/3197869659 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f669c15e9c0 0x7f669c151cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-08-09T13:34:20.275 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:20.267+0000 7f66a179f640 1 --2- 172.21.15.43:0/3197869659 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f669c15a1b0 0x7f669c1f1570 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7f668c02b950 tx=0x7f668c0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-09T13:34:20.275 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:20.267+0000 7f66927fc640 1 -- 172.21.15.43:0/3197869659 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f668c03ad30 con 0x7f669c15a1b0 2023-08-09T13:34:20.275 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:20.267+0000 7f66927fc640 1 -- 172.21.15.43:0/3197869659 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f668c0443f0 con 0x7f669c15a1b0 2023-08-09T13:34:20.275 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:20.267+0000 7f66927fc640 1 -- 172.21.15.43:0/3197869659 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f668c0447a0 con 0x7f669c15a1b0 2023-08-09T13:34:20.275 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:20.267+0000 7f66a2fa2640 1 -- 172.21.15.43:0/3197869659 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f669c1527b0 con 0x7f669c15a1b0 2023-08-09T13:34:20.275 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:20.268+0000 7f66a2fa2640 1 -- 172.21.15.43:0/3197869659 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f669c1f2180 con 0x7f669c15a1b0 2023-08-09T13:34:20.275 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:20.269+0000 7f66a2fa2640 1 -- 172.21.15.43:0/3197869659 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f669c04e390 con 0x7f669c15a1b0 2023-08-09T13:34:20.276 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:20.270+0000 7f66927fc640 1 -- 172.21.15.43:0/3197869659 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 5) v1 ==== 88341+0+0 (secure 0 0 0) 0x7f668c04d450 con 0x7f669c15a1b0 2023-08-09T13:34:20.276 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:20.271+0000 7f66927fc640 1 --2- 172.21.15.43:0/3197869659 >> [v2:172.21.15.203:6832/830097474,v1:172.21.15.203:6833/830097474] conn(0x7f668006a050 0x7f668006c510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:34:20.276 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:20.271+0000 7f66927fc640 1 -- 172.21.15.43:0/3197869659 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6813+0+0 (secure 0 0 0) 0x7f668c02d040 con 0x7f669c15a1b0 2023-08-09T13:34:20.276 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:20.272+0000 7f66a1fa0640 1 --2- 172.21.15.43:0/3197869659 >> [v2:172.21.15.203:6832/830097474,v1:172.21.15.203:6833/830097474] conn(0x7f668006a050 0x7f668006c510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:34:20.277 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:20.273+0000 7f66a1fa0640 1 --2- 172.21.15.43:0/3197869659 >> [v2:172.21.15.203:6832/830097474,v1:172.21.15.203:6833/830097474] conn(0x7f668006a050 0x7f668006c510 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f6698008850 tx=0x7f6698008040 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-09T13:34:20.278 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:20.275+0000 7f66927fc640 1 -- 172.21.15.43:0/3197869659 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178066 (secure 0 0 0) 0x7f668c0762e0 con 0x7f669c15a1b0 2023-08-09T13:34:20.480 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:20.477+0000 7f66a2fa2640 1 -- 172.21.15.43:0/3197869659 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"} v 0) v1 -- 0x7f669c1f1e30 con 0x7f669c15a1b0 2023-08-09T13:34:20.481 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:20.478+0000 7f66927fc640 1 -- 172.21.15.43:0/3197869659 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"}]=0 v24) v1 ==== 92+0+10 (secure 0 0 0) 0x7f668c0ad020 con 0x7f669c15a1b0 2023-08-09T13:34:20.481 INFO:teuthology.orchestra.run.smithi043.stdout:pg_num: 8 2023-08-09T13:34:20.483 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:20.481+0000 7f66a2fa2640 1 -- 172.21.15.43:0/3197869659 >> [v2:172.21.15.203:6832/830097474,v1:172.21.15.203:6833/830097474] conn(0x7f668006a050 msgr2=0x7f668006c510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:34:20.484 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:20.481+0000 7f66a2fa2640 1 --2- 172.21.15.43:0/3197869659 >> [v2:172.21.15.203:6832/830097474,v1:172.21.15.203:6833/830097474] conn(0x7f668006a050 0x7f668006c510 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f6698008850 tx=0x7f6698008040 comp rx=0 tx=0).stop 2023-08-09T13:34:20.484 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:20.481+0000 7f66a2fa2640 1 -- 172.21.15.43:0/3197869659 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f669c15a1b0 msgr2=0x7f669c1f1570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:34:20.484 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:20.481+0000 7f66a2fa2640 1 --2- 172.21.15.43:0/3197869659 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f669c15a1b0 0x7f669c1f1570 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7f668c02b950 tx=0x7f668c0046c0 comp rx=0 tx=0).stop 2023-08-09T13:34:20.484 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:20.481+0000 7f66a2fa2640 1 -- 172.21.15.43:0/3197869659 shutdown_connections 2023-08-09T13:34:20.484 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:20.481+0000 7f66a2fa2640 1 --2- 172.21.15.43:0/3197869659 >> [v2:172.21.15.203:6832/830097474,v1:172.21.15.203:6833/830097474] conn(0x7f668006a050 0x7f668006c510 unknown :-1 s=CLOSED pgs=81 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:34:20.484 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:20.481+0000 7f66a2fa2640 1 --2- 172.21.15.43:0/3197869659 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f669c15e9c0 0x7f669c151cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:34:20.485 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:20.481+0000 7f66a2fa2640 1 --2- 172.21.15.43:0/3197869659 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f669c15a1b0 0x7f669c1f1570 unknown :-1 s=CLOSED pgs=115 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:34:20.485 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:20.481+0000 7f66a2fa2640 1 --2- 172.21.15.43:0/3197869659 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f669c0cd630 0x7f669c0c7350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:34:20.485 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:20.481+0000 7f66a2fa2640 1 -- 172.21.15.43:0/3197869659 >> 172.21.15.43:0/3197869659 conn(0x7f669c0ccad0 msgr2=0x7f669c0cdba0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-09T13:34:20.485 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:20.481+0000 7f66a2fa2640 1 -- 172.21.15.43:0/3197869659 shutdown_connections 2023-08-09T13:34:20.486 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:20.481+0000 7f66a2fa2640 1 -- 172.21.15.43:0/3197869659 wait complete. 2023-08-09T13:34:20.502 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-08-09T13:34:20.734 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:20.731+0000 7fc9ff59e640 1 Processor -- start 2023-08-09T13:34:20.734 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:20.731+0000 7fc9ff59e640 1 -- start start 2023-08-09T13:34:20.736 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:20.733+0000 7fc9ff59e640 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7fca000b7df0 0x7fca000d1b30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:34:20.736 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:20.733+0000 7fc9ff59e640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7fca000cebf0 0x7fca000cf040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:34:20.736 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:20.733+0000 7fc9ff59e640 1 -- --> v1:172.21.15.43:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fca000cf580 con 0x7fca0015c2d0 2023-08-09T13:34:20.737 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:20.733+0000 7fc9ff59e640 1 -- --> v2:172.21.15.43:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fca000b8230 con 0x7fca000cebf0 2023-08-09T13:34:20.737 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:20.733+0000 7fc9ff59e640 1 -- --> v2:172.21.15.203:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fca000d16a0 con 0x7fca000b7df0 2023-08-09T13:34:20.737 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:20.733+0000 7fc9fe59c640 1 --1- >> v1:172.21.15.43:6789/0 conn(0x7fca0015c2d0 0x7fca000cd020 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.43:6789/0 says I am v1:172.21.15.43:48174/0 (socket says 172.21.15.43:48174) 2023-08-09T13:34:20.737 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:20.733+0000 7fc9fe59c640 1 -- 172.21.15.43:0/4097602919 learned_addr learned my addr 172.21.15.43:0/4097602919 (peer_addr_for_me v1:172.21.15.43:0/0) 2023-08-09T13:34:20.737 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:20.733+0000 7fc9fed9d640 1 --2- 172.21.15.43:0/4097602919 >> v2:172.21.15.43:3300/0 conn(0x7fca000cebf0 0x7fca000cf040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:34:20.739 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:20.734+0000 7fc9f5d9b640 1 --2- 172.21.15.43:0/4097602919 >> v2:172.21.15.203:3300/0 conn(0x7fca000b7df0 0x7fca000d1b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:34:20.739 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:20.734+0000 7fc9fdd9b640 1 -- 172.21.15.43:0/4097602919 <== mon.0 v1:172.21.15.43:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4034271374 0 0) 0x7fc9f0001110 con 0x7fca0015c2d0 2023-08-09T13:34:20.739 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:20.734+0000 7fc9fdd9b640 1 -- 172.21.15.43:0/4097602919 --> v1:172.21.15.43:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fc9ec003520 con 0x7fca0015c2d0 2023-08-09T13:34:20.739 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:20.734+0000 7fc9fed9d640 1 -- 172.21.15.43:0/4097602919 >> v2:172.21.15.203:3300/0 conn(0x7fca000b7df0 msgr2=0x7fca000d1b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-09T13:34:20.739 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:20.734+0000 7fc9fed9d640 1 --2- 172.21.15.43:0/4097602919 >> v2:172.21.15.203:3300/0 conn(0x7fca000b7df0 0x7fca000d1b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:34:20.739 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:20.734+0000 7fc9fed9d640 1 -- 172.21.15.43:0/4097602919 >> v1:172.21.15.43:6789/0 conn(0x7fca0015c2d0 legacy=0x7fca000cd020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:34:20.739 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:20.734+0000 7fc9fed9d640 1 -- 172.21.15.43:0/4097602919 --> v2:172.21.15.43:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fca000d0d70 con 0x7fca000cebf0 2023-08-09T13:34:20.740 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:20.734+0000 7fc9f5d9b640 1 --2- 172.21.15.43:0/4097602919 >> v2:172.21.15.203:3300/0 conn(0x7fca000b7df0 0x7fca000d1b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-08-09T13:34:20.740 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:20.735+0000 7fc9fed9d640 1 --2- 172.21.15.43:0/4097602919 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fca000cebf0 0x7fca000cf040 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7fca000c5ce0 tx=0x7fc9f802bec0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bcd3ae32e757429f server_cookie=0 in_seq=0 out_seq=0 2023-08-09T13:34:20.740 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:20.735+0000 7fc9fdd9b640 1 -- 172.21.15.43:0/4097602919 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc9f802cc70 con 0x7fca000cebf0 2023-08-09T13:34:20.740 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:20.735+0000 7fc9fdd9b640 1 -- 172.21.15.43:0/4097602919 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc9f802cdd0 con 0x7fca000cebf0 2023-08-09T13:34:20.740 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:20.735+0000 7fc9fdd9b640 1 -- 172.21.15.43:0/4097602919 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc9f8034660 con 0x7fca000cebf0 2023-08-09T13:34:20.740 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:20.736+0000 7fc9ff59e640 1 -- 172.21.15.43:0/4097602919 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fca000cebf0 msgr2=0x7fca000cf040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:34:20.740 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:20.736+0000 7fc9ff59e640 1 --2- 172.21.15.43:0/4097602919 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fca000cebf0 0x7fca000cf040 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7fca000c5ce0 tx=0x7fc9f802bec0 comp rx=0 tx=0).stop 2023-08-09T13:34:20.741 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:20.736+0000 7fc9ff59e640 1 -- 172.21.15.43:0/4097602919 shutdown_connections 2023-08-09T13:34:20.741 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:20.736+0000 7fc9ff59e640 1 --2- 172.21.15.43:0/4097602919 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fca000cebf0 0x7fca000cf040 unknown :-1 s=CLOSED pgs=117 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:34:20.741 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:20.736+0000 7fc9ff59e640 1 --2- 172.21.15.43:0/4097602919 >> v2:172.21.15.203:3300/0 conn(0x7fca000b7df0 0x7fca000d1b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:34:20.741 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:20.736+0000 7fc9ff59e640 1 -- 172.21.15.43:0/4097602919 >> 172.21.15.43:0/4097602919 conn(0x7fca0006a230 msgr2=0x7fca0006a640 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-09T13:34:20.741 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:20.736+0000 7fc9ff59e640 1 -- 172.21.15.43:0/4097602919 shutdown_connections 2023-08-09T13:34:20.741 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:20.736+0000 7fc9ff59e640 1 -- 172.21.15.43:0/4097602919 wait complete. 2023-08-09T13:34:20.743 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:20.738+0000 7fc9ff59e640 1 Processor -- start 2023-08-09T13:34:20.743 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:20.738+0000 7fc9ff59e640 1 -- start start 2023-08-09T13:34:20.743 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:20.739+0000 7fc9ff59e640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fca000b7df0 0x7fca000cbd10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:34:20.743 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:20.739+0000 7fc9ff59e640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fca000cebf0 0x7fca0014dfc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:34:20.743 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:20.739+0000 7fc9ff59e640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fca0015c2d0 0x7fca001f1500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:34:20.743 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:20.739+0000 7fc9ff59e640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fca000cc250 con 0x7fca000cebf0 2023-08-09T13:34:20.744 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:20.739+0000 7fc9ff59e640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fca0014e500 con 0x7fca0015c2d0 2023-08-09T13:34:20.744 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:20.739+0000 7fc9ff59e640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fca000ca150 con 0x7fca000b7df0 2023-08-09T13:34:20.744 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:20.739+0000 7fc9f5d9b640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fca000cebf0 0x7fca0014dfc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:34:20.744 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:20.739+0000 7fc9f5d9b640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fca000cebf0 0x7fca0014dfc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:44764/0 (socket says 172.21.15.43:44764) 2023-08-09T13:34:20.744 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:20.739+0000 7fc9f5d9b640 1 -- 172.21.15.43:0/3899577655 learned_addr learned my addr 172.21.15.43:0/3899577655 (peer_addr_for_me v2:172.21.15.43:0/0) 2023-08-09T13:34:20.744 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:20.740+0000 7fc9fe59c640 1 --2- 172.21.15.43:0/3899577655 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fca000b7df0 0x7fca000cbd10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:34:20.744 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:20.740+0000 7fc9f5d9b640 1 -- 172.21.15.43:0/3899577655 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fca000b7df0 msgr2=0x7fca000cbd10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:34:20.745 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:20.740+0000 7fc9f5d9b640 1 --2- 172.21.15.43:0/3899577655 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fca000b7df0 0x7fca000cbd10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:34:20.745 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:20.740+0000 7fc9f5d9b640 1 -- 172.21.15.43:0/3899577655 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fca0015c2d0 msgr2=0x7fca001f1500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:34:20.745 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:20.740+0000 7fc9f5d9b640 1 --2- 172.21.15.43:0/3899577655 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fca0015c2d0 0x7fca001f1500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:34:20.745 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:20.740+0000 7fc9f5d9b640 1 -- 172.21.15.43:0/3899577655 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc9f8009a70 con 0x7fca000cebf0 2023-08-09T13:34:20.745 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:20.740+0000 7fc9fe59c640 1 --2- 172.21.15.43:0/3899577655 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fca000b7df0 0x7fca000cbd10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-08-09T13:34:20.745 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:20.740+0000 7fc9f5d9b640 1 --2- 172.21.15.43:0/3899577655 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fca000cebf0 0x7fca0014dfc0 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7fc9e800c7f0 tx=0x7fc9e800cbf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-09T13:34:20.745 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:20.740+0000 7fc9f77fe640 1 -- 172.21.15.43:0/3899577655 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc9e8007a30 con 0x7fca000cebf0 2023-08-09T13:34:20.746 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:20.740+0000 7fc9f77fe640 1 -- 172.21.15.43:0/3899577655 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc9e8007b90 con 0x7fca000cebf0 2023-08-09T13:34:20.746 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:20.740+0000 7fc9f77fe640 1 -- 172.21.15.43:0/3899577655 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc9e800ce40 con 0x7fca000cebf0 2023-08-09T13:34:20.746 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:20.740+0000 7fc9ff59e640 1 -- 172.21.15.43:0/3899577655 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fca000ca430 con 0x7fca000cebf0 2023-08-09T13:34:20.746 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:20.740+0000 7fc9ff59e640 1 -- 172.21.15.43:0/3899577655 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fca000ca9f0 con 0x7fca000cebf0 2023-08-09T13:34:20.747 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:20.741+0000 7fc9ff59e640 1 -- 172.21.15.43:0/3899577655 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fca0004e390 con 0x7fca000cebf0 2023-08-09T13:34:20.747 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:20.742+0000 7fc9f77fe640 1 -- 172.21.15.43:0/3899577655 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 5) v1 ==== 88341+0+0 (secure 0 0 0) 0x7fc9e80113f0 con 0x7fca000cebf0 2023-08-09T13:34:20.747 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:20.743+0000 7fc9f77fe640 1 --2- 172.21.15.43:0/3899577655 >> [v2:172.21.15.203:6832/830097474,v1:172.21.15.203:6833/830097474] conn(0x7fc9e006a080 0x7fc9e006c540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:34:20.747 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:20.743+0000 7fc9f77fe640 1 -- 172.21.15.43:0/3899577655 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6813+0+0 (secure 0 0 0) 0x7fc9e8088b00 con 0x7fca000cebf0 2023-08-09T13:34:20.747 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:20.744+0000 7fc9fe59c640 1 --2- 172.21.15.43:0/3899577655 >> [v2:172.21.15.203:6832/830097474,v1:172.21.15.203:6833/830097474] conn(0x7fc9e006a080 0x7fc9e006c540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:34:20.748 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:20.745+0000 7fc9fe59c640 1 --2- 172.21.15.43:0/3899577655 >> [v2:172.21.15.203:6832/830097474,v1:172.21.15.203:6833/830097474] conn(0x7fc9e006a080 0x7fc9e006c540 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7fc9f0001370 tx=0x7fc9f0009040 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-09T13:34:20.750 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:20.748+0000 7fc9f77fe640 1 -- 172.21.15.43:0/3899577655 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178066 (secure 0 0 0) 0x7fc9e808ae60 con 0x7fca000cebf0 2023-08-09T13:34:20.947 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:20.945+0000 7fc9ff59e640 1 -- 172.21.15.43:0/3899577655 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7fca000cce90 con 0x7fca000cebf0 2023-08-09T13:34:20.948 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:20.946+0000 7fc9f77fe640 1 -- 172.21.15.43:0/3899577655 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v24) v1 ==== 93+0+10 (secure 0 0 0) 0x7fc9e801f0a0 con 0x7fca000cebf0 2023-08-09T13:34:20.949 INFO:teuthology.orchestra.run.smithi043.stdout:pg_num: 1 2023-08-09T13:34:20.951 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:20.949+0000 7fc9ff59e640 1 -- 172.21.15.43:0/3899577655 >> [v2:172.21.15.203:6832/830097474,v1:172.21.15.203:6833/830097474] conn(0x7fc9e006a080 msgr2=0x7fc9e006c540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:34:20.951 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:20.949+0000 7fc9ff59e640 1 --2- 172.21.15.43:0/3899577655 >> [v2:172.21.15.203:6832/830097474,v1:172.21.15.203:6833/830097474] conn(0x7fc9e006a080 0x7fc9e006c540 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7fc9f0001370 tx=0x7fc9f0009040 comp rx=0 tx=0).stop 2023-08-09T13:34:20.952 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:20.949+0000 7fc9ff59e640 1 -- 172.21.15.43:0/3899577655 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fca000cebf0 msgr2=0x7fca0014dfc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:34:20.952 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:20.949+0000 7fc9ff59e640 1 --2- 172.21.15.43:0/3899577655 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fca000cebf0 0x7fca0014dfc0 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7fc9e800c7f0 tx=0x7fc9e800cbf0 comp rx=0 tx=0).stop 2023-08-09T13:34:20.952 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:20.949+0000 7fc9ff59e640 1 -- 172.21.15.43:0/3899577655 shutdown_connections 2023-08-09T13:34:20.952 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:20.949+0000 7fc9ff59e640 1 --2- 172.21.15.43:0/3899577655 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fca0015c2d0 0x7fca001f1500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:34:20.952 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:20.949+0000 7fc9ff59e640 1 --2- 172.21.15.43:0/3899577655 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fca000cebf0 0x7fca0014dfc0 unknown :-1 s=CLOSED pgs=118 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:34:20.952 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:20.949+0000 7fc9ff59e640 1 --2- 172.21.15.43:0/3899577655 >> [v2:172.21.15.203:6832/830097474,v1:172.21.15.203:6833/830097474] conn(0x7fc9e006a080 0x7fc9e006c540 unknown :-1 s=CLOSED pgs=82 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:34:20.952 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:20.949+0000 7fc9ff59e640 1 --2- 172.21.15.43:0/3899577655 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fca000b7df0 0x7fca000cbd10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:34:20.953 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:20.949+0000 7fc9ff59e640 1 -- 172.21.15.43:0/3899577655 >> 172.21.15.43:0/3899577655 conn(0x7fca0006a230 msgr2=0x7fca00067560 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-09T13:34:20.953 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:20.949+0000 7fc9ff59e640 1 -- 172.21.15.43:0/3899577655 shutdown_connections 2023-08-09T13:34:20.953 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:20.949+0000 7fc9ff59e640 1 -- 172.21.15.43:0/3899577655 wait complete. 2023-08-09T13:34:20.972 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-08-09T13:34:21.198 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:21.196+0000 7ff1dbfff640 1 Processor -- start 2023-08-09T13:34:21.199 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:21.196+0000 7ff1dbfff640 1 -- start start 2023-08-09T13:34:21.200 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:21.198+0000 7ff1dbfff640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7ff1dc06fbb0 0x7ff1dc0add10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:34:21.201 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:21.198+0000 7ff1dbfff640 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7ff1dc06e0a0 0x7ff1dc0cc210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:34:21.201 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:21.198+0000 7ff1dbfff640 1 -- --> v1:172.21.15.43:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff1dc06cb60 con 0x7ff1dc06ff90 2023-08-09T13:34:21.202 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:21.198+0000 7ff1dbfff640 1 -- --> v2:172.21.15.203:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff1dc0ac8c0 con 0x7ff1dc06e0a0 2023-08-09T13:34:21.202 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:21.198+0000 7ff1dbfff640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff1dc0ac300 con 0x7ff1dc06fbb0 2023-08-09T13:34:21.202 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:21.199+0000 7ff1db7fe640 1 --1- >> v1:172.21.15.43:6789/0 conn(0x7ff1dc06ff90 0x7ff1dc06cff0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.43:6789/0 says I am v1:172.21.15.43:48184/0 (socket says 172.21.15.43:48184) 2023-08-09T13:34:21.202 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:21.199+0000 7ff1db7fe640 1 -- 172.21.15.43:0/3259451846 learned_addr learned my addr 172.21.15.43:0/3259451846 (peer_addr_for_me v1:172.21.15.43:0/0) 2023-08-09T13:34:21.202 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:21.199+0000 7ff1daffd640 1 --2- 172.21.15.43:0/3259451846 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7ff1dc06fbb0 0x7ff1dc0add10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:34:21.202 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:21.199+0000 7ff1da7fc640 1 --2- 172.21.15.43:0/3259451846 >> v2:172.21.15.203:3300/0 conn(0x7ff1dc06e0a0 0x7ff1dc0cc210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:34:21.204 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:21.199+0000 7ff1d9ffb640 1 -- 172.21.15.43:0/3259451846 <== mon.0 v1:172.21.15.43:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 158153371 0 0) 0x7ff1dc06cb60 con 0x7ff1dc06ff90 2023-08-09T13:34:21.204 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:21.199+0000 7ff1d9ffb640 1 -- 172.21.15.43:0/3259451846 --> v1:172.21.15.43:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ff1c4003520 con 0x7ff1dc06ff90 2023-08-09T13:34:21.204 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:21.200+0000 7ff1d9ffb640 1 -- 172.21.15.43:0/3259451846 <== mon.0 v1:172.21.15.43:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2539388347 0 0) 0x7ff1c4003520 con 0x7ff1dc06ff90 2023-08-09T13:34:21.204 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:21.200+0000 7ff1d9ffb640 1 -- 172.21.15.43:0/3259451846 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7ff1dc06fbb0 msgr2=0x7ff1dc0add10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-09T13:34:21.204 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:21.200+0000 7ff1d9ffb640 1 --2- 172.21.15.43:0/3259451846 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7ff1dc06fbb0 0x7ff1dc0add10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:34:21.204 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:21.200+0000 7ff1d9ffb640 1 -- 172.21.15.43:0/3259451846 >> v2:172.21.15.203:3300/0 conn(0x7ff1dc06e0a0 msgr2=0x7ff1dc0cc210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-09T13:34:21.205 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:21.200+0000 7ff1d9ffb640 1 --2- 172.21.15.43:0/3259451846 >> v2:172.21.15.203:3300/0 conn(0x7ff1dc06e0a0 0x7ff1dc0cc210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:34:21.205 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:21.200+0000 7ff1d9ffb640 1 -- 172.21.15.43:0/3259451846 --> v1:172.21.15.43:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff1dc0ac5e0 con 0x7ff1dc06ff90 2023-08-09T13:34:21.205 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:21.200+0000 7ff1d9ffb640 1 -- 172.21.15.43:0/3259451846 <== mon.0 v1:172.21.15.43:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1285150740 0 0) 0x7ff1d40019f0 con 0x7ff1dc06ff90 2023-08-09T13:34:21.205 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:21.200+0000 7ff1d9ffb640 1 -- 172.21.15.43:0/3259451846 >> v1:172.21.15.43:6789/0 conn(0x7ff1dc06ff90 legacy=0x7ff1dc06cff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:34:21.205 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:21.200+0000 7ff1d9ffb640 1 --2- 172.21.15.43:0/3259451846 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff1c4003970 0x7ff1c4003d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:34:21.205 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:21.200+0000 7ff1d9ffb640 1 --2- 172.21.15.43:0/3259451846 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7ff1dc06fbb0 0x7ff1c40047c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:34:21.205 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:21.200+0000 7ff1d9ffb640 1 --2- 172.21.15.43:0/3259451846 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7ff1c4004d00 0x7ff1c4005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:34:21.206 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:21.200+0000 7ff1d9ffb640 1 -- 172.21.15.43:0/3259451846 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff1dc0ac8c0 con 0x7ff1c4003970 2023-08-09T13:34:21.206 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:21.200+0000 7ff1d9ffb640 1 -- 172.21.15.43:0/3259451846 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff1dc0ac300 con 0x7ff1c4004d00 2023-08-09T13:34:21.206 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:21.200+0000 7ff1d9ffb640 1 -- 172.21.15.43:0/3259451846 --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff1c400b980 con 0x7ff1dc06fbb0 2023-08-09T13:34:21.206 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:21.200+0000 7ff1daffd640 1 --2- 172.21.15.43:0/3259451846 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff1c4003970 0x7ff1c4003d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:34:21.206 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:21.201+0000 7ff1db7fe640 1 --2- 172.21.15.43:0/3259451846 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7ff1c4004d00 0x7ff1c4005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:34:21.206 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:21.201+0000 7ff1daffd640 1 -- 172.21.15.43:0/3259451846 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7ff1dc06fbb0 msgr2=0x7ff1c40047c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-09T13:34:21.206 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:21.201+0000 7ff1daffd640 1 --2- 172.21.15.43:0/3259451846 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7ff1dc06fbb0 0x7ff1c40047c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:34:21.206 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:21.201+0000 7ff1daffd640 1 -- 172.21.15.43:0/3259451846 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7ff1c4004d00 msgr2=0x7ff1c4005180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-09T13:34:21.207 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:21.201+0000 7ff1daffd640 1 --2- 172.21.15.43:0/3259451846 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7ff1c4004d00 0x7ff1c4005180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:34:21.207 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:21.201+0000 7ff1daffd640 1 -- 172.21.15.43:0/3259451846 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff1c4003520 con 0x7ff1c4003970 2023-08-09T13:34:21.207 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:21.201+0000 7ff1db7fe640 1 --2- 172.21.15.43:0/3259451846 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7ff1c4004d00 0x7ff1c4005180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-08-09T13:34:21.208 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:21.202+0000 7ff1daffd640 1 --2- 172.21.15.43:0/3259451846 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff1c4003970 0x7ff1c4003d90 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7ff1cc009ac0 tx=0x7ff1c40057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=87a5f140a06e14d4 server_cookie=0 in_seq=0 out_seq=0 2023-08-09T13:34:21.208 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:21.202+0000 7ff1d9ffb640 1 -- 172.21.15.43:0/3259451846 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff1cc02c710 con 0x7ff1c4003970 2023-08-09T13:34:21.208 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:21.202+0000 7ff1d9ffb640 1 -- 172.21.15.43:0/3259451846 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff1cc02c870 con 0x7ff1c4003970 2023-08-09T13:34:21.208 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:21.202+0000 7ff1d9ffb640 1 -- 172.21.15.43:0/3259451846 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff1cc02cc00 con 0x7ff1c4003970 2023-08-09T13:34:21.208 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:21.203+0000 7ff1dbfff640 1 -- 172.21.15.43:0/3259451846 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff1c4003970 msgr2=0x7ff1c4003d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:34:21.209 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:21.203+0000 7ff1dbfff640 1 --2- 172.21.15.43:0/3259451846 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff1c4003970 0x7ff1c4003d90 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7ff1cc009ac0 tx=0x7ff1c40057f0 comp rx=0 tx=0).stop 2023-08-09T13:34:21.209 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:21.203+0000 7ff1db7fe640 1 -- 172.21.15.43:0/3259451846 reap_dead start 2023-08-09T13:34:21.209 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:21.203+0000 7ff1dbfff640 1 -- 172.21.15.43:0/3259451846 shutdown_connections 2023-08-09T13:34:21.209 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:21.203+0000 7ff1dbfff640 1 -- 172.21.15.43:0/3259451846 >> 172.21.15.43:0/3259451846 conn(0x7ff1dc0ce640 msgr2=0x7ff1dc0cea50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-09T13:34:21.209 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:21.203+0000 7ff1dbfff640 1 -- 172.21.15.43:0/3259451846 shutdown_connections 2023-08-09T13:34:21.209 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:21.203+0000 7ff1dbfff640 1 -- 172.21.15.43:0/3259451846 wait complete. 2023-08-09T13:34:21.209 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:21.205+0000 7ff1dbfff640 1 Processor -- start 2023-08-09T13:34:21.210 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:21.205+0000 7ff1dbfff640 1 -- start start 2023-08-09T13:34:21.211 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:21.206+0000 7ff1dbfff640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff1c4003970 0x7ff1dc15eae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:34:21.211 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:21.206+0000 7ff1dbfff640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7ff1dc1592d0 0x7ff1dc14e030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:34:21.211 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:21.206+0000 7ff1dbfff640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7ff1dc069d60 0x7ff1dc06a1e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:34:21.211 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:21.206+0000 7ff1dbfff640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff1dc06a720 con 0x7ff1c4003970 2023-08-09T13:34:21.212 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:21.206+0000 7ff1dbfff640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff1dc15ab60 con 0x7ff1dc069d60 2023-08-09T13:34:21.212 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:21.206+0000 7ff1dbfff640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff1dc159ac0 con 0x7ff1dc1592d0 2023-08-09T13:34:21.212 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:21.206+0000 7ff1daffd640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff1c4003970 0x7ff1dc15eae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:34:21.212 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:21.206+0000 7ff1daffd640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff1c4003970 0x7ff1dc15eae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:44786/0 (socket says 172.21.15.43:44786) 2023-08-09T13:34:21.212 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:21.206+0000 7ff1daffd640 1 -- 172.21.15.43:0/1594278744 learned_addr learned my addr 172.21.15.43:0/1594278744 (peer_addr_for_me v2:172.21.15.43:0/0) 2023-08-09T13:34:21.212 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:21.207+0000 7ff1da7fc640 1 --2- 172.21.15.43:0/1594278744 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7ff1dc1592d0 0x7ff1dc14e030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:34:21.212 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:21.207+0000 7ff1daffd640 1 -- 172.21.15.43:0/1594278744 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7ff1dc1592d0 msgr2=0x7ff1dc14e030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:34:21.213 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:21.207+0000 7ff1daffd640 1 --2- 172.21.15.43:0/1594278744 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7ff1dc1592d0 0x7ff1dc14e030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:34:21.213 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:21.207+0000 7ff1daffd640 1 -- 172.21.15.43:0/1594278744 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7ff1dc069d60 msgr2=0x7ff1dc06a1e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:34:21.213 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:21.207+0000 7ff1daffd640 1 --2- 172.21.15.43:0/1594278744 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7ff1dc069d60 0x7ff1dc06a1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:34:21.213 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:21.207+0000 7ff1daffd640 1 -- 172.21.15.43:0/1594278744 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff1dc159da0 con 0x7ff1c4003970 2023-08-09T13:34:21.213 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:21.207+0000 7ff1daffd640 1 --2- 172.21.15.43:0/1594278744 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff1c4003970 0x7ff1dc15eae0 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7ff1cc000c00 tx=0x7ff1cc004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-09T13:34:21.213 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:21.207+0000 7ff1c37fe640 1 -- 172.21.15.43:0/1594278744 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff1cc003810 con 0x7ff1c4003970 2023-08-09T13:34:21.213 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:21.208+0000 7ff1c37fe640 1 -- 172.21.15.43:0/1594278744 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff1cc003970 con 0x7ff1c4003970 2023-08-09T13:34:21.213 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:21.208+0000 7ff1c37fe640 1 -- 172.21.15.43:0/1594278744 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff1cc003d20 con 0x7ff1c4003970 2023-08-09T13:34:21.214 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:21.208+0000 7ff1dbfff640 1 -- 172.21.15.43:0/1594278744 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff1dc15a080 con 0x7ff1c4003970 2023-08-09T13:34:21.214 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:21.208+0000 7ff1dbfff640 1 -- 172.21.15.43:0/1594278744 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff1dc15a640 con 0x7ff1c4003970 2023-08-09T13:34:21.214 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:21.209+0000 7ff1dbfff640 1 -- 172.21.15.43:0/1594278744 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff1dc15a7d0 con 0x7ff1c4003970 2023-08-09T13:34:21.215 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:21.210+0000 7ff1c37fe640 1 -- 172.21.15.43:0/1594278744 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 5) v1 ==== 88341+0+0 (secure 0 0 0) 0x7ff1cc034440 con 0x7ff1c4003970 2023-08-09T13:34:21.215 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:21.211+0000 7ff1c37fe640 1 --2- 172.21.15.43:0/1594278744 >> [v2:172.21.15.203:6832/830097474,v1:172.21.15.203:6833/830097474] conn(0x7ff1c806a080 0x7ff1c806c540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:34:21.215 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:21.211+0000 7ff1c37fe640 1 -- 172.21.15.43:0/1594278744 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6813+0+0 (secure 0 0 0) 0x7ff1cc0aa8c0 con 0x7ff1c4003970 2023-08-09T13:34:21.215 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:21.211+0000 7ff1da7fc640 1 --2- 172.21.15.43:0/1594278744 >> [v2:172.21.15.203:6832/830097474,v1:172.21.15.203:6833/830097474] conn(0x7ff1c806a080 0x7ff1c806c540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:34:21.215 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:21.213+0000 7ff1da7fc640 1 --2- 172.21.15.43:0/1594278744 >> [v2:172.21.15.203:6832/830097474,v1:172.21.15.203:6833/830097474] conn(0x7ff1c806a080 0x7ff1c806c540 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7ff1dc0cb030 tx=0x7ff1d0008040 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-09T13:34:21.217 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:21.215+0000 7ff1c37fe640 1 -- 172.21.15.43:0/1594278744 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178066 (secure 0 0 0) 0x7ff1cc075df0 con 0x7ff1c4003970 2023-08-09T13:34:21.420 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:21.417+0000 7ff1dbfff640 1 -- 172.21.15.43:0/1594278744 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7ff1dc04e390 con 0x7ff1c4003970 2023-08-09T13:34:21.420 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:21.418+0000 7ff1c37fe640 1 -- 172.21.15.43:0/1594278744 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v24) v1 ==== 104+0+11 (secure 0 0 0) 0x7ff1cc079410 con 0x7ff1c4003970 2023-08-09T13:34:21.421 INFO:teuthology.orchestra.run.smithi043.stdout:pg_num: 32 2023-08-09T13:34:21.424 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:21.421+0000 7ff1dbfff640 1 -- 172.21.15.43:0/1594278744 >> [v2:172.21.15.203:6832/830097474,v1:172.21.15.203:6833/830097474] conn(0x7ff1c806a080 msgr2=0x7ff1c806c540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:34:21.424 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:21.421+0000 7ff1dbfff640 1 --2- 172.21.15.43:0/1594278744 >> [v2:172.21.15.203:6832/830097474,v1:172.21.15.203:6833/830097474] conn(0x7ff1c806a080 0x7ff1c806c540 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7ff1dc0cb030 tx=0x7ff1d0008040 comp rx=0 tx=0).stop 2023-08-09T13:34:21.425 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:21.422+0000 7ff1dbfff640 1 -- 172.21.15.43:0/1594278744 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff1c4003970 msgr2=0x7ff1dc15eae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:34:21.425 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:21.422+0000 7ff1dbfff640 1 --2- 172.21.15.43:0/1594278744 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff1c4003970 0x7ff1dc15eae0 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7ff1cc000c00 tx=0x7ff1cc004730 comp rx=0 tx=0).stop 2023-08-09T13:34:21.425 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:21.422+0000 7ff1dbfff640 1 -- 172.21.15.43:0/1594278744 shutdown_connections 2023-08-09T13:34:21.425 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:21.422+0000 7ff1dbfff640 1 --2- 172.21.15.43:0/1594278744 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7ff1dc069d60 0x7ff1dc06a1e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:34:21.425 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:21.422+0000 7ff1dbfff640 1 --2- 172.21.15.43:0/1594278744 >> [v2:172.21.15.203:6832/830097474,v1:172.21.15.203:6833/830097474] conn(0x7ff1c806a080 0x7ff1c806c540 unknown :-1 s=CLOSED pgs=83 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:34:21.425 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:21.422+0000 7ff1dbfff640 1 --2- 172.21.15.43:0/1594278744 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7ff1dc1592d0 0x7ff1dc14e030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:34:21.426 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:21.422+0000 7ff1dbfff640 1 --2- 172.21.15.43:0/1594278744 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff1c4003970 0x7ff1dc15eae0 unknown :-1 s=CLOSED pgs=121 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:34:21.426 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:21.422+0000 7ff1dbfff640 1 -- 172.21.15.43:0/1594278744 >> 172.21.15.43:0/1594278744 conn(0x7ff1dc0ce640 msgr2=0x7ff1dc0ae630 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-09T13:34:21.426 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:21.422+0000 7ff1dbfff640 1 -- 172.21.15.43:0/1594278744 shutdown_connections 2023-08-09T13:34:21.426 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:21.422+0000 7ff1dbfff640 1 -- 172.21.15.43:0/1594278744 wait complete. 2023-08-09T13:34:21.443 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-08-09T13:34:21.676 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:21.674+0000 7f3cdffff640 1 Processor -- start 2023-08-09T13:34:21.677 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:21.674+0000 7f3cdffff640 1 -- start start 2023-08-09T13:34:21.678 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:21.676+0000 7f3cdffff640 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7f3ce00cf8e0 0x7f3ce00d17b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:34:21.678 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:21.676+0000 7f3cdffff640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f3ce00d1cf0 0x7f3ce00b5670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:34:21.679 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:21.676+0000 7f3cdffff640 1 -- --> v1:172.21.15.203:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3ce00d24a0 con 0x7f3ce015f970 2023-08-09T13:34:21.679 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:21.676+0000 7f3cdffff640 1 -- --> v2:172.21.15.43:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3ce00d1280 con 0x7f3ce00d1cf0 2023-08-09T13:34:21.679 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:21.676+0000 7f3cdffff640 1 -- --> v2:172.21.15.203:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3ce015fd50 con 0x7f3ce00cf8e0 2023-08-09T13:34:21.680 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:21.676+0000 7f3cdf7fe640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f3ce00d1cf0 0x7f3ce00b5670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:34:21.680 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:21.676+0000 7f3cdf7fe640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f3ce00d1cf0 0x7f3ce00b5670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:44798/0 (socket says 172.21.15.43:44798) 2023-08-09T13:34:21.680 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:21.676+0000 7f3cdf7fe640 1 -- 172.21.15.43:0/1961022826 learned_addr learned my addr 172.21.15.43:0/1961022826 (peer_addr_for_me v2:172.21.15.43:0/0) 2023-08-09T13:34:21.680 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:21.676+0000 7f3cde7fc640 1 --2- 172.21.15.43:0/1961022826 >> v2:172.21.15.203:3300/0 conn(0x7f3ce00cf8e0 0x7f3ce00d17b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:34:21.680 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:21.677+0000 7f3cdf7fe640 1 -- 172.21.15.43:0/1961022826 >> v2:172.21.15.203:3300/0 conn(0x7f3ce00cf8e0 msgr2=0x7f3ce00d17b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-09T13:34:21.680 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:21.677+0000 7f3cdf7fe640 1 --2- 172.21.15.43:0/1961022826 >> v2:172.21.15.203:3300/0 conn(0x7f3ce00cf8e0 0x7f3ce00d17b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:34:21.681 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:21.677+0000 7f3cdf7fe640 1 -- 172.21.15.43:0/1961022826 >> v1:172.21.15.203:6789/0 conn(0x7f3ce015f970 legacy=0x7f3ce00cf1d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:34:21.681 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:21.677+0000 7f3cdf7fe640 1 -- 172.21.15.43:0/1961022826 --> v2:172.21.15.43:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3ce00d2780 con 0x7f3ce00d1cf0 2023-08-09T13:34:21.681 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:21.677+0000 7f3cde7fc640 1 --2- 172.21.15.43:0/1961022826 >> v2:172.21.15.203:3300/0 conn(0x7f3ce00cf8e0 0x7f3ce00d17b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-08-09T13:34:21.681 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:21.679+0000 7f3cdf7fe640 1 --2- 172.21.15.43:0/1961022826 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f3ce00d1cf0 0x7f3ce00b5670 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7f3ce00cc850 tx=0x7f3cd802b900 comp rx=0 tx=0).ready entity=mon.0 client_cookie=48ffaeb6705135bb server_cookie=0 in_seq=0 out_seq=0 2023-08-09T13:34:21.682 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:21.679+0000 7f3cddffb640 1 -- 172.21.15.43:0/1961022826 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3cd8031860 con 0x7f3ce00d1cf0 2023-08-09T13:34:21.682 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:21.679+0000 7f3cddffb640 1 -- 172.21.15.43:0/1961022826 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3cd80319c0 con 0x7f3ce00d1cf0 2023-08-09T13:34:21.682 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:21.679+0000 7f3cddffb640 1 -- 172.21.15.43:0/1961022826 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3cd8031d50 con 0x7f3ce00d1cf0 2023-08-09T13:34:21.683 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:21.680+0000 7f3cdffff640 1 -- 172.21.15.43:0/1961022826 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f3ce00d1cf0 msgr2=0x7f3ce00b5670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:34:21.683 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:21.680+0000 7f3cdffff640 1 --2- 172.21.15.43:0/1961022826 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f3ce00d1cf0 0x7f3ce00b5670 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7f3ce00cc850 tx=0x7f3cd802b900 comp rx=0 tx=0).stop 2023-08-09T13:34:21.683 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:21.680+0000 7f3cdffff640 1 -- 172.21.15.43:0/1961022826 shutdown_connections 2023-08-09T13:34:21.683 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:21.680+0000 7f3cdffff640 1 --2- 172.21.15.43:0/1961022826 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f3ce00d1cf0 0x7f3ce00b5670 unknown :-1 s=CLOSED pgs=122 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:34:21.683 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:21.680+0000 7f3cdffff640 1 --2- 172.21.15.43:0/1961022826 >> v2:172.21.15.203:3300/0 conn(0x7f3ce00cf8e0 0x7f3ce00d17b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:34:21.683 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:21.680+0000 7f3cdffff640 1 -- 172.21.15.43:0/1961022826 >> 172.21.15.43:0/1961022826 conn(0x7f3ce00712e0 msgr2=0x7f3ce00716f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-09T13:34:21.684 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:21.680+0000 7f3cdffff640 1 -- 172.21.15.43:0/1961022826 shutdown_connections 2023-08-09T13:34:21.684 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:21.680+0000 7f3cdffff640 1 -- 172.21.15.43:0/1961022826 wait complete. 2023-08-09T13:34:21.684 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:21.682+0000 7f3cdffff640 1 Processor -- start 2023-08-09T13:34:21.685 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:21.682+0000 7f3cdffff640 1 -- start start 2023-08-09T13:34:21.686 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:21.684+0000 7f3cdffff640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f3ce00cf8e0 0x7f3ce014f290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:34:21.686 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:21.684+0000 7f3cdffff640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f3ce00d1cf0 0x7f3ce00c86b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:34:21.686 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:21.684+0000 7f3cdffff640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f3ce015f970 0x7f3ce015e510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:34:21.687 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:21.684+0000 7f3cdffff640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3ce00c5fd0 con 0x7f3ce00d1cf0 2023-08-09T13:34:21.687 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:21.684+0000 7f3cdffff640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3ce00c8bf0 con 0x7f3ce00cf8e0 2023-08-09T13:34:21.687 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:21.684+0000 7f3cdffff640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3ce00cb8e0 con 0x7f3ce015f970 2023-08-09T13:34:21.688 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:21.684+0000 7f3cde7fc640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f3ce00d1cf0 0x7f3ce00c86b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:34:21.688 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:21.684+0000 7f3cde7fc640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f3ce00d1cf0 0x7f3ce00c86b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:44806/0 (socket says 172.21.15.43:44806) 2023-08-09T13:34:21.688 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:21.684+0000 7f3cde7fc640 1 -- 172.21.15.43:0/3635840295 learned_addr learned my addr 172.21.15.43:0/3635840295 (peer_addr_for_me v2:172.21.15.43:0/0) 2023-08-09T13:34:21.689 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:21.684+0000 7f3cdeffd640 1 --2- 172.21.15.43:0/3635840295 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f3ce00cf8e0 0x7f3ce014f290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:34:21.689 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:21.685+0000 7f3cdf7fe640 1 --2- 172.21.15.43:0/3635840295 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f3ce015f970 0x7f3ce015e510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:34:21.689 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:21.685+0000 7f3cde7fc640 1 -- 172.21.15.43:0/3635840295 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f3ce015f970 msgr2=0x7f3ce015e510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:34:21.689 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:21.685+0000 7f3cde7fc640 1 --2- 172.21.15.43:0/3635840295 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f3ce015f970 0x7f3ce015e510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:34:21.689 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:21.685+0000 7f3cde7fc640 1 -- 172.21.15.43:0/3635840295 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f3ce00cf8e0 msgr2=0x7f3ce014f290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:34:21.689 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:21.685+0000 7f3cde7fc640 1 --2- 172.21.15.43:0/3635840295 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f3ce00cf8e0 0x7f3ce014f290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:34:21.689 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:21.685+0000 7f3cde7fc640 1 -- 172.21.15.43:0/3635840295 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3cd80094d0 con 0x7f3ce00d1cf0 2023-08-09T13:34:21.690 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:21.685+0000 7f3cdf7fe640 1 --2- 172.21.15.43:0/3635840295 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f3ce015f970 0x7f3ce015e510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-08-09T13:34:21.690 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:21.685+0000 7f3cde7fc640 1 --2- 172.21.15.43:0/3635840295 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f3ce00d1cf0 0x7f3ce00c86b0 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7f3cd400e7e0 tx=0x7f3cd400eca0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-09T13:34:21.690 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:21.685+0000 7f3cc7fff640 1 -- 172.21.15.43:0/3635840295 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3cd4004230 con 0x7f3ce00d1cf0 2023-08-09T13:34:21.690 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:21.686+0000 7f3cc7fff640 1 -- 172.21.15.43:0/3635840295 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3cd400ee90 con 0x7f3ce00d1cf0 2023-08-09T13:34:21.690 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:21.686+0000 7f3cc7fff640 1 -- 172.21.15.43:0/3635840295 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3cd400c6d0 con 0x7f3ce00d1cf0 2023-08-09T13:34:21.690 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:21.686+0000 7f3cdffff640 1 -- 172.21.15.43:0/3635840295 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3ce00cbbc0 con 0x7f3ce00d1cf0 2023-08-09T13:34:21.691 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:21.686+0000 7f3cdffff640 1 -- 172.21.15.43:0/3635840295 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3ce00cc180 con 0x7f3ce00d1cf0 2023-08-09T13:34:21.691 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:21.687+0000 7f3cdffff640 1 -- 172.21.15.43:0/3635840295 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3ce004e390 con 0x7f3ce00d1cf0 2023-08-09T13:34:21.692 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:21.689+0000 7f3cc7fff640 1 -- 172.21.15.43:0/3635840295 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 5) v1 ==== 88341+0+0 (secure 0 0 0) 0x7f3cd400c830 con 0x7f3ce00d1cf0 2023-08-09T13:34:21.693 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:21.690+0000 7f3cc7fff640 1 --2- 172.21.15.43:0/3635840295 >> [v2:172.21.15.203:6832/830097474,v1:172.21.15.203:6833/830097474] conn(0x7f3ccc06a080 0x7f3ccc06c540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:34:21.693 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:21.690+0000 7f3cc7fff640 1 -- 172.21.15.43:0/3635840295 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6813+0+0 (secure 0 0 0) 0x7f3cd40883d0 con 0x7f3ce00d1cf0 2023-08-09T13:34:21.693 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:21.691+0000 7f3cdeffd640 1 --2- 172.21.15.43:0/3635840295 >> [v2:172.21.15.203:6832/830097474,v1:172.21.15.203:6833/830097474] conn(0x7f3ccc06a080 0x7f3ccc06c540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:34:21.694 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:21.692+0000 7f3cdeffd640 1 --2- 172.21.15.43:0/3635840295 >> [v2:172.21.15.203:6832/830097474,v1:172.21.15.203:6833/830097474] conn(0x7f3ccc06a080 0x7f3ccc06c540 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f3cd00012e0 tx=0x7f3cd0009040 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-09T13:34:21.700 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:21.698+0000 7f3cc7fff640 1 -- 172.21.15.43:0/3635840295 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178066 (secure 0 0 0) 0x7f3cd4053860 con 0x7f3ce00d1cf0 2023-08-09T13:34:21.901 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:21.898+0000 7f3cdffff640 1 -- 172.21.15.43:0/3635840295 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f3ce0066b60 con 0x7f3ce00d1cf0 2023-08-09T13:34:21.901 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:21.899+0000 7f3cc7fff640 1 -- 172.21.15.43:0/3635840295 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v24) v1 ==== 100+0+11 (secure 0 0 0) 0x7f3cd4056e80 con 0x7f3ce00d1cf0 2023-08-09T13:34:21.902 INFO:teuthology.orchestra.run.smithi043.stdout:pg_num: 64 2023-08-09T13:34:21.904 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:21.901+0000 7f3cdffff640 1 -- 172.21.15.43:0/3635840295 >> [v2:172.21.15.203:6832/830097474,v1:172.21.15.203:6833/830097474] conn(0x7f3ccc06a080 msgr2=0x7f3ccc06c540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:34:21.904 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:21.901+0000 7f3cdffff640 1 --2- 172.21.15.43:0/3635840295 >> [v2:172.21.15.203:6832/830097474,v1:172.21.15.203:6833/830097474] conn(0x7f3ccc06a080 0x7f3ccc06c540 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f3cd00012e0 tx=0x7f3cd0009040 comp rx=0 tx=0).stop 2023-08-09T13:34:21.904 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:21.901+0000 7f3cdffff640 1 -- 172.21.15.43:0/3635840295 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f3ce00d1cf0 msgr2=0x7f3ce00c86b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:34:21.905 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:21.901+0000 7f3cdffff640 1 --2- 172.21.15.43:0/3635840295 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f3ce00d1cf0 0x7f3ce00c86b0 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7f3cd400e7e0 tx=0x7f3cd400eca0 comp rx=0 tx=0).stop 2023-08-09T13:34:21.905 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:21.902+0000 7f3cdffff640 1 -- 172.21.15.43:0/3635840295 shutdown_connections 2023-08-09T13:34:21.905 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:21.902+0000 7f3cdffff640 1 --2- 172.21.15.43:0/3635840295 >> [v2:172.21.15.203:6832/830097474,v1:172.21.15.203:6833/830097474] conn(0x7f3ccc06a080 0x7f3ccc06c540 unknown :-1 s=CLOSED pgs=84 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:34:21.905 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:21.902+0000 7f3cdffff640 1 --2- 172.21.15.43:0/3635840295 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f3ce015f970 0x7f3ce015e510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:34:21.905 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:21.902+0000 7f3cdffff640 1 --2- 172.21.15.43:0/3635840295 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f3ce00d1cf0 0x7f3ce00c86b0 unknown :-1 s=CLOSED pgs=123 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:34:21.906 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:21.902+0000 7f3cdffff640 1 --2- 172.21.15.43:0/3635840295 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f3ce00cf8e0 0x7f3ce014f290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:34:21.906 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:21.902+0000 7f3cdffff640 1 -- 172.21.15.43:0/3635840295 >> 172.21.15.43:0/3635840295 conn(0x7f3ce00712e0 msgr2=0x7f3ce006a2e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-09T13:34:21.906 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:21.902+0000 7f3cdffff640 1 -- 172.21.15.43:0/3635840295 shutdown_connections 2023-08-09T13:34:21.906 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:21.902+0000 7f3cdffff640 1 -- 172.21.15.43:0/3635840295 wait complete. 2023-08-09T13:34:21.922 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2023-08-09T13:34:22.150 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:22.147+0000 7fbfe541f640 1 Processor -- start 2023-08-09T13:34:22.150 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:22.147+0000 7fbfe541f640 1 -- start start 2023-08-09T13:34:22.151 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:22.148+0000 7fbfe541f640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fbfe015c680 0x7fbfe00ce760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:34:22.152 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:22.148+0000 7fbfe541f640 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7fbfe00d0a70 0x7fbfe00cd7f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:34:22.152 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:22.148+0000 7fbfe541f640 1 -- --> v1:172.21.15.43:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbfe00d0f70 con 0x7fbfe00cdd30 2023-08-09T13:34:22.152 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:22.148+0000 7fbfe541f640 1 -- --> v2:172.21.15.203:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbfe00ccd10 con 0x7fbfe00d0a70 2023-08-09T13:34:22.152 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:22.148+0000 7fbfe541f640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbfe00ccff0 con 0x7fbfe015c680 2023-08-09T13:34:22.152 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:22.148+0000 7fbfdf7fe640 1 --1- >> v1:172.21.15.43:6789/0 conn(0x7fbfe00cdd30 0x7fbfe00b3fa0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.43:6789/0 says I am v1:172.21.15.43:48186/0 (socket says 172.21.15.43:48186) 2023-08-09T13:34:22.152 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:22.148+0000 7fbfdf7fe640 1 -- 172.21.15.43:0/3038990871 learned_addr learned my addr 172.21.15.43:0/3038990871 (peer_addr_for_me v1:172.21.15.43:0/0) 2023-08-09T13:34:22.152 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:22.149+0000 7fbfde7fc640 1 --2- 172.21.15.43:0/3038990871 >> v2:172.21.15.203:3300/0 conn(0x7fbfe00d0a70 0x7fbfe00cd7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:34:22.153 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:22.149+0000 7fbfdeffd640 1 --2- 172.21.15.43:0/3038990871 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fbfe015c680 0x7fbfe00ce760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:34:22.153 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:22.149+0000 7fbfddffb640 1 -- 172.21.15.43:0/3038990871 <== mon.0 v1:172.21.15.43:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2886420599 0 0) 0x7fbfe00d0f70 con 0x7fbfe00cdd30 2023-08-09T13:34:22.153 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:22.149+0000 7fbfddffb640 1 -- 172.21.15.43:0/3038990871 --> v1:172.21.15.43:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fbfc8003520 con 0x7fbfe00cdd30 2023-08-09T13:34:22.154 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:22.149+0000 7fbfddffb640 1 -- 172.21.15.43:0/3038990871 <== mon.0 v1:172.21.15.43:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 786157398 0 0) 0x7fbfc8003520 con 0x7fbfe00cdd30 2023-08-09T13:34:22.154 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:22.149+0000 7fbfddffb640 1 -- 172.21.15.43:0/3038990871 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fbfe015c680 msgr2=0x7fbfe00ce760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-09T13:34:22.155 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:22.149+0000 7fbfddffb640 1 --2- 172.21.15.43:0/3038990871 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fbfe015c680 0x7fbfe00ce760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:34:22.155 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:22.149+0000 7fbfddffb640 1 -- 172.21.15.43:0/3038990871 >> v2:172.21.15.203:3300/0 conn(0x7fbfe00d0a70 msgr2=0x7fbfe00cd7f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-09T13:34:22.155 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:22.149+0000 7fbfddffb640 1 --2- 172.21.15.43:0/3038990871 >> v2:172.21.15.203:3300/0 conn(0x7fbfe00d0a70 0x7fbfe00cd7f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:34:22.155 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:22.149+0000 7fbfddffb640 1 -- 172.21.15.43:0/3038990871 --> v1:172.21.15.43:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbfe00cd2d0 con 0x7fbfe00cdd30 2023-08-09T13:34:22.155 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:22.149+0000 7fbfddffb640 1 -- 172.21.15.43:0/3038990871 <== mon.0 v1:172.21.15.43:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1285150740 0 0) 0x7fbfd8001670 con 0x7fbfe00cdd30 2023-08-09T13:34:22.155 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:22.149+0000 7fbfddffb640 1 -- 172.21.15.43:0/3038990871 >> v1:172.21.15.43:6789/0 conn(0x7fbfe00cdd30 legacy=0x7fbfe00b3fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:34:22.155 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:22.150+0000 7fbfddffb640 1 --2- 172.21.15.43:0/3038990871 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fbfe015c680 0x7fbfc80039e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:34:22.156 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:22.150+0000 7fbfddffb640 1 --2- 172.21.15.43:0/3038990871 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fbfc8003f60 0x7fbfc80047c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:34:22.156 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:22.150+0000 7fbfddffb640 1 --2- 172.21.15.43:0/3038990871 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fbfc8004d00 0x7fbfc8005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:34:22.156 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:22.150+0000 7fbfddffb640 1 -- 172.21.15.43:0/3038990871 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbfe00ccd10 con 0x7fbfc8004d00 2023-08-09T13:34:22.156 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:22.150+0000 7fbfddffb640 1 -- 172.21.15.43:0/3038990871 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbfe00ccff0 con 0x7fbfc8003f60 2023-08-09T13:34:22.156 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:22.150+0000 7fbfddffb640 1 -- 172.21.15.43:0/3038990871 --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbfc800ba60 con 0x7fbfe015c680 2023-08-09T13:34:22.156 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:22.150+0000 7fbfdf7fe640 1 --2- 172.21.15.43:0/3038990871 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fbfc8004d00 0x7fbfc8005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:34:22.156 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:22.150+0000 7fbfdeffd640 1 --2- 172.21.15.43:0/3038990871 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fbfe015c680 0x7fbfc80039e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:34:22.157 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:22.150+0000 7fbfde7fc640 1 --2- 172.21.15.43:0/3038990871 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fbfc8003f60 0x7fbfc80047c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:34:22.157 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:22.150+0000 7fbfdf7fe640 1 -- 172.21.15.43:0/3038990871 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fbfe015c680 msgr2=0x7fbfc80039e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-09T13:34:22.157 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:22.150+0000 7fbfdf7fe640 1 --2- 172.21.15.43:0/3038990871 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fbfe015c680 0x7fbfc80039e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:34:22.157 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:22.150+0000 7fbfdf7fe640 1 -- 172.21.15.43:0/3038990871 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fbfc8003f60 msgr2=0x7fbfc80047c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-09T13:34:22.157 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:22.150+0000 7fbfdf7fe640 1 --2- 172.21.15.43:0/3038990871 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fbfc8003f60 0x7fbfc80047c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:34:22.157 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:22.151+0000 7fbfdf7fe640 1 -- 172.21.15.43:0/3038990871 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbfc8003520 con 0x7fbfc8004d00 2023-08-09T13:34:22.157 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:22.152+0000 7fbfdf7fe640 1 --2- 172.21.15.43:0/3038990871 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fbfc8004d00 0x7fbfc8005180 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7fbfd800c240 tx=0x7fbfc80057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9cf893fe9398b145 server_cookie=0 in_seq=0 out_seq=0 2023-08-09T13:34:22.158 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:22.152+0000 7fbfddffb640 1 -- 172.21.15.43:0/3038990871 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbfd802cb00 con 0x7fbfc8004d00 2023-08-09T13:34:22.158 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:22.152+0000 7fbfddffb640 1 -- 172.21.15.43:0/3038990871 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbfd802cc60 con 0x7fbfc8004d00 2023-08-09T13:34:22.158 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:22.152+0000 7fbfddffb640 1 -- 172.21.15.43:0/3038990871 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbfd8005af0 con 0x7fbfc8004d00 2023-08-09T13:34:22.159 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:22.153+0000 7fbfe541f640 1 -- 172.21.15.43:0/3038990871 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fbfc8004d00 msgr2=0x7fbfc8005180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:34:22.159 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:22.153+0000 7fbfe541f640 1 --2- 172.21.15.43:0/3038990871 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fbfc8004d00 0x7fbfc8005180 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7fbfd800c240 tx=0x7fbfc80057f0 comp rx=0 tx=0).stop 2023-08-09T13:34:22.159 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:22.153+0000 7fbfdf7fe640 1 -- 172.21.15.43:0/3038990871 reap_dead start 2023-08-09T13:34:22.159 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:22.153+0000 7fbfe541f640 1 -- 172.21.15.43:0/3038990871 shutdown_connections 2023-08-09T13:34:22.159 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:22.153+0000 7fbfe541f640 1 -- 172.21.15.43:0/3038990871 >> 172.21.15.43:0/3038990871 conn(0x7fbfe0071520 msgr2=0x7fbfe00d2520 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-09T13:34:22.160 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:22.153+0000 7fbfe541f640 1 -- 172.21.15.43:0/3038990871 shutdown_connections 2023-08-09T13:34:22.160 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:22.153+0000 7fbfe541f640 1 -- 172.21.15.43:0/3038990871 wait complete. 2023-08-09T13:34:22.160 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:22.155+0000 7fbfe541f640 1 Processor -- start 2023-08-09T13:34:22.160 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:22.156+0000 7fbfe541f640 1 -- start start 2023-08-09T13:34:22.161 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:22.157+0000 7fbfe541f640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fbfc8004d00 0x7fbfe01565c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:34:22.162 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:22.157+0000 7fbfe541f640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fbfe0154180 0x7fbfe0152de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:34:22.162 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:22.157+0000 7fbfe541f640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fbfe015ef70 0x7fbfe015f3e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:34:22.162 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:22.157+0000 7fbfe541f640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbfe015f920 con 0x7fbfe015ef70 2023-08-09T13:34:22.162 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:22.157+0000 7fbfe541f640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbfe0153320 con 0x7fbfe0154180 2023-08-09T13:34:22.162 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:22.157+0000 7fbfe541f640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbfe0155200 con 0x7fbfc8004d00 2023-08-09T13:34:22.162 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:22.157+0000 7fbfdf7fe640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fbfe015ef70 0x7fbfe015f3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:34:22.162 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:22.157+0000 7fbfdf7fe640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fbfe015ef70 0x7fbfe015f3e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:44828/0 (socket says 172.21.15.43:44828) 2023-08-09T13:34:22.163 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:22.157+0000 7fbfdf7fe640 1 -- 172.21.15.43:0/2386783512 learned_addr learned my addr 172.21.15.43:0/2386783512 (peer_addr_for_me v2:172.21.15.43:0/0) 2023-08-09T13:34:22.163 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:22.158+0000 7fbfde7fc640 1 --2- 172.21.15.43:0/2386783512 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fbfe0154180 0x7fbfe0152de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:34:22.163 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:22.158+0000 7fbfdeffd640 1 --2- 172.21.15.43:0/2386783512 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fbfc8004d00 0x7fbfe01565c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:34:22.163 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:22.158+0000 7fbfdf7fe640 1 -- 172.21.15.43:0/2386783512 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fbfc8004d00 msgr2=0x7fbfe01565c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:34:22.163 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:22.158+0000 7fbfdf7fe640 1 --2- 172.21.15.43:0/2386783512 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fbfc8004d00 0x7fbfe01565c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:34:22.163 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:22.158+0000 7fbfdf7fe640 1 -- 172.21.15.43:0/2386783512 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fbfe0154180 msgr2=0x7fbfe0152de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:34:22.163 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:22.158+0000 7fbfdf7fe640 1 --2- 172.21.15.43:0/2386783512 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fbfe0154180 0x7fbfe0152de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:34:22.164 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:22.158+0000 7fbfdf7fe640 1 -- 172.21.15.43:0/2386783512 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbfe0154680 con 0x7fbfe015ef70 2023-08-09T13:34:22.164 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:22.158+0000 7fbfdf7fe640 1 --2- 172.21.15.43:0/2386783512 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fbfe015ef70 0x7fbfe015f3e0 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7fbfd800c240 tx=0x7fbfd8001a70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-09T13:34:22.164 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:22.158+0000 7fbfc77fe640 1 -- 172.21.15.43:0/2386783512 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbfd803c040 con 0x7fbfe015ef70 2023-08-09T13:34:22.164 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:22.158+0000 7fbfc77fe640 1 -- 172.21.15.43:0/2386783512 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbfd8001630 con 0x7fbfe015ef70 2023-08-09T13:34:22.164 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:22.158+0000 7fbfc77fe640 1 -- 172.21.15.43:0/2386783512 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbfd8007270 con 0x7fbfe015ef70 2023-08-09T13:34:22.164 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:22.159+0000 7fbfe541f640 1 -- 172.21.15.43:0/2386783512 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbfe0154960 con 0x7fbfe015ef70 2023-08-09T13:34:22.164 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:22.159+0000 7fbfe541f640 1 -- 172.21.15.43:0/2386783512 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbfe0154f20 con 0x7fbfe015ef70 2023-08-09T13:34:22.165 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:22.160+0000 7fbfe541f640 1 -- 172.21.15.43:0/2386783512 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbfe015ca60 con 0x7fbfe015ef70 2023-08-09T13:34:22.165 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:22.161+0000 7fbfc77fe640 1 -- 172.21.15.43:0/2386783512 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 5) v1 ==== 88341+0+0 (secure 0 0 0) 0x7fbfd8032070 con 0x7fbfe015ef70 2023-08-09T13:34:22.165 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:22.162+0000 7fbfc77fe640 1 --2- 172.21.15.43:0/2386783512 >> [v2:172.21.15.203:6832/830097474,v1:172.21.15.203:6833/830097474] conn(0x7fbfcc06a050 0x7fbfcc06c510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:34:22.165 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:22.163+0000 7fbfc77fe640 1 -- 172.21.15.43:0/2386783512 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6813+0+0 (secure 0 0 0) 0x7fbfd80a9550 con 0x7fbfe015ef70 2023-08-09T13:34:22.166 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:22.163+0000 7fbfdeffd640 1 --2- 172.21.15.43:0/2386783512 >> [v2:172.21.15.203:6832/830097474,v1:172.21.15.203:6833/830097474] conn(0x7fbfcc06a050 0x7fbfcc06c510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:34:22.166 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:22.164+0000 7fbfdeffd640 1 --2- 172.21.15.43:0/2386783512 >> [v2:172.21.15.203:6832/830097474,v1:172.21.15.203:6833/830097474] conn(0x7fbfcc06a050 0x7fbfcc06c510 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7fbfd0004330 tx=0x7fbfd000b040 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-09T13:34:22.169 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:22.167+0000 7fbfc77fe640 1 -- 172.21.15.43:0/2386783512 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178066 (secure 0 0 0) 0x7fbfe015ca60 con 0x7fbfe015ef70 2023-08-09T13:34:22.361 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:22.358+0000 7fbfe541f640 1 -- 172.21.15.43:0/2386783512 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) v1 -- 0x7fbfe015ca60 con 0x7fbfe015ef70 2023-08-09T13:34:22.362 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:22.359+0000 7fbfc77fe640 1 -- 172.21.15.43:0/2386783512 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v24) v1 ==== 103+0+11 (secure 0 0 0) 0x7fbfe015ca60 con 0x7fbfe015ef70 2023-08-09T13:34:22.362 INFO:teuthology.orchestra.run.smithi043.stdout:pg_num: 32 2023-08-09T13:34:22.364 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:22.361+0000 7fbfe541f640 1 -- 172.21.15.43:0/2386783512 >> [v2:172.21.15.203:6832/830097474,v1:172.21.15.203:6833/830097474] conn(0x7fbfcc06a050 msgr2=0x7fbfcc06c510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:34:22.364 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:22.361+0000 7fbfe541f640 1 --2- 172.21.15.43:0/2386783512 >> [v2:172.21.15.203:6832/830097474,v1:172.21.15.203:6833/830097474] conn(0x7fbfcc06a050 0x7fbfcc06c510 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7fbfd0004330 tx=0x7fbfd000b040 comp rx=0 tx=0).stop 2023-08-09T13:34:22.365 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:22.361+0000 7fbfe541f640 1 -- 172.21.15.43:0/2386783512 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fbfe015ef70 msgr2=0x7fbfe015f3e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:34:22.365 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:22.361+0000 7fbfe541f640 1 --2- 172.21.15.43:0/2386783512 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fbfe015ef70 0x7fbfe015f3e0 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7fbfd800c240 tx=0x7fbfd8001a70 comp rx=0 tx=0).stop 2023-08-09T13:34:22.365 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:22.362+0000 7fbfe541f640 1 -- 172.21.15.43:0/2386783512 shutdown_connections 2023-08-09T13:34:22.365 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:22.362+0000 7fbfe541f640 1 --2- 172.21.15.43:0/2386783512 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fbfe015ef70 0x7fbfe015f3e0 unknown :-1 s=CLOSED pgs=126 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:34:22.365 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:22.362+0000 7fbfe541f640 1 --2- 172.21.15.43:0/2386783512 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fbfe0154180 0x7fbfe0152de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:34:22.365 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:22.362+0000 7fbfe541f640 1 --2- 172.21.15.43:0/2386783512 >> [v2:172.21.15.203:6832/830097474,v1:172.21.15.203:6833/830097474] conn(0x7fbfcc06a050 0x7fbfcc06c510 unknown :-1 s=CLOSED pgs=85 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:34:22.365 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:22.362+0000 7fbfe541f640 1 --2- 172.21.15.43:0/2386783512 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fbfc8004d00 0x7fbfe01565c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:34:22.366 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:22.362+0000 7fbfe541f640 1 -- 172.21.15.43:0/2386783512 >> 172.21.15.43:0/2386783512 conn(0x7fbfe0071520 msgr2=0x7fbfe00b6f60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-09T13:34:22.366 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:22.362+0000 7fbfe541f640 1 -- 172.21.15.43:0/2386783512 shutdown_connections 2023-08-09T13:34:22.366 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:22.362+0000 7fbfe541f640 1 -- 172.21.15.43:0/2386783512 wait complete. 2023-08-09T13:34:22.382 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2023-08-09T13:34:22.615 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:22.613+0000 7f680ae34640 1 Processor -- start 2023-08-09T13:34:22.616 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:22.613+0000 7f680ae34640 1 -- start start 2023-08-09T13:34:22.617 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:22.615+0000 7f680ae34640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f680415c680 0x7f68040ce760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:34:22.618 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:22.615+0000 7f680ae34640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f68040d0a70 0x7f68040cd7f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:34:22.618 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:22.615+0000 7f680ae34640 1 -- --> v1:172.21.15.203:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f68040d0f70 con 0x7f68040cdd30 2023-08-09T13:34:22.618 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:22.615+0000 7f680ae34640 1 -- --> v2:172.21.15.43:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f68040ccd10 con 0x7f680415c680 2023-08-09T13:34:22.618 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:22.615+0000 7f680ae34640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f68040ccff0 con 0x7f68040d0a70 2023-08-09T13:34:22.618 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:22.615+0000 7f6808bd0640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f680415c680 0x7f68040ce760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:34:22.618 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:22.615+0000 7f6808bd0640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f680415c680 0x7f68040ce760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:44836/0 (socket says 172.21.15.43:44836) 2023-08-09T13:34:22.618 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:22.615+0000 7f6808bd0640 1 -- 172.21.15.43:0/1278200283 learned_addr learned my addr 172.21.15.43:0/1278200283 (peer_addr_for_me v2:172.21.15.43:0/0) 2023-08-09T13:34:22.619 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:22.615+0000 7f67fbfff640 1 --2- 172.21.15.43:0/1278200283 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f68040d0a70 0x7f68040cd7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:34:22.619 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:22.615+0000 7f6808bd0640 1 -- 172.21.15.43:0/1278200283 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f68040d0a70 msgr2=0x7f68040cd7f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-09T13:34:22.619 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:22.615+0000 7f6808bd0640 1 --2- 172.21.15.43:0/1278200283 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f68040d0a70 0x7f68040cd7f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:34:22.620 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:22.615+0000 7f6808bd0640 1 -- 172.21.15.43:0/1278200283 >> v1:172.21.15.203:6789/0 conn(0x7f68040cdd30 legacy=0x7f68040b3fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-09T13:34:22.620 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:22.615+0000 7f6808bd0640 1 -- 172.21.15.43:0/1278200283 --> v2:172.21.15.43:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f68040cd2d0 con 0x7f680415c680 2023-08-09T13:34:22.620 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:22.617+0000 7f6808bd0640 1 --2- 172.21.15.43:0/1278200283 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f680415c680 0x7f68040ce760 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7f6800009870 tx=0x7f680002b8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=336c38193dfdfbeb server_cookie=0 in_seq=0 out_seq=0 2023-08-09T13:34:22.621 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:22.617+0000 7f67fb7fe640 1 -- 172.21.15.43:0/1278200283 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6800031640 con 0x7f680415c680 2023-08-09T13:34:22.621 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:22.617+0000 7f67fb7fe640 1 -- 172.21.15.43:0/1278200283 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f68000317a0 con 0x7f680415c680 2023-08-09T13:34:22.621 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:22.617+0000 7f67fb7fe640 1 -- 172.21.15.43:0/1278200283 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6800031b30 con 0x7f680415c680 2023-08-09T13:34:22.621 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:22.618+0000 7f680ae34640 1 -- 172.21.15.43:0/1278200283 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f680415c680 msgr2=0x7f68040ce760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:34:22.621 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:22.618+0000 7f680ae34640 1 --2- 172.21.15.43:0/1278200283 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f680415c680 0x7f68040ce760 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7f6800009870 tx=0x7f680002b8f0 comp rx=0 tx=0).stop 2023-08-09T13:34:22.621 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:22.618+0000 7f680ae34640 1 -- 172.21.15.43:0/1278200283 shutdown_connections 2023-08-09T13:34:22.621 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:22.618+0000 7f680ae34640 1 --2- 172.21.15.43:0/1278200283 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f68040d0a70 0x7f68040cd7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:34:22.622 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:22.618+0000 7f680ae34640 1 --2- 172.21.15.43:0/1278200283 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f680415c680 0x7f68040ce760 unknown :-1 s=CLOSED pgs=127 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:34:22.622 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:22.618+0000 7f680ae34640 1 -- 172.21.15.43:0/1278200283 >> 172.21.15.43:0/1278200283 conn(0x7f6804071520 msgr2=0x7f68040d2520 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-09T13:34:22.622 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:22.618+0000 7f680ae34640 1 -- 172.21.15.43:0/1278200283 shutdown_connections 2023-08-09T13:34:22.622 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:22.618+0000 7f680ae34640 1 -- 172.21.15.43:0/1278200283 wait complete. 2023-08-09T13:34:22.622 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:22.620+0000 7f680ae34640 1 Processor -- start 2023-08-09T13:34:22.623 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:22.620+0000 7f680ae34640 1 -- start start 2023-08-09T13:34:22.623 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:22.621+0000 7f680ae34640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f68040d0a70 0x7f680414b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:34:22.624 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:22.621+0000 7f680ae34640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f680415c680 0x7f68040ca7f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:34:22.624 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:22.621+0000 7f680ae34640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f68040cad30 0x7f68040cbb20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:34:22.624 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:22.621+0000 7f680ae34640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f68040c9770 con 0x7f680415c680 2023-08-09T13:34:22.624 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:22.621+0000 7f680ae34640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f68040c9a30 con 0x7f68040cad30 2023-08-09T13:34:22.624 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:22.621+0000 7f680ae34640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f68040c9d10 con 0x7f68040d0a70 2023-08-09T13:34:22.624 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:22.621+0000 7f67fbfff640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f680415c680 0x7f68040ca7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:34:22.624 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:22.621+0000 7f67fbfff640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f680415c680 0x7f68040ca7f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:44844/0 (socket says 172.21.15.43:44844) 2023-08-09T13:34:22.625 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:22.621+0000 7f67fbfff640 1 -- 172.21.15.43:0/760937445 learned_addr learned my addr 172.21.15.43:0/760937445 (peer_addr_for_me v2:172.21.15.43:0/0) 2023-08-09T13:34:22.626 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:22.622+0000 7f6808bd0640 1 --2- 172.21.15.43:0/760937445 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f68040d0a70 0x7f680414b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:34:22.626 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:22.622+0000 7f68093d1640 1 --2- 172.21.15.43:0/760937445 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f68040cad30 0x7f68040cbb20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:34:22.626 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:22.622+0000 7f67fbfff640 1 -- 172.21.15.43:0/760937445 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f68040d0a70 msgr2=0x7f680414b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:34:22.626 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:22.622+0000 7f67fbfff640 1 --2- 172.21.15.43:0/760937445 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f68040d0a70 0x7f680414b9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:34:22.627 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:22.622+0000 7f67fbfff640 1 -- 172.21.15.43:0/760937445 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f68040cad30 msgr2=0x7f68040cbb20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:34:22.627 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:22.622+0000 7f67fbfff640 1 --2- 172.21.15.43:0/760937445 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f68040cad30 0x7f68040cbb20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:34:22.627 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:22.622+0000 7f67fbfff640 1 -- 172.21.15.43:0/760937445 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f68000094d0 con 0x7f680415c680 2023-08-09T13:34:22.627 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:22.622+0000 7f68093d1640 1 --2- 172.21.15.43:0/760937445 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f68040cad30 0x7f68040cbb20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-08-09T13:34:22.627 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:22.622+0000 7f67fbfff640 1 --2- 172.21.15.43:0/760937445 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f680415c680 0x7f68040ca7f0 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7f67f400c7f0 tx=0x7f67f400ccb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-09T13:34:22.627 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:22.622+0000 7f67f97fa640 1 -- 172.21.15.43:0/760937445 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f67f4004220 con 0x7f680415c680 2023-08-09T13:34:22.627 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:22.622+0000 7f67f97fa640 1 -- 172.21.15.43:0/760937445 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f67f40043a0 con 0x7f680415c680 2023-08-09T13:34:22.628 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:22.622+0000 7f67f97fa640 1 -- 172.21.15.43:0/760937445 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f67f4007860 con 0x7f680415c680 2023-08-09T13:34:22.628 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:22.622+0000 7f680ae34640 1 -- 172.21.15.43:0/760937445 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f68040c9ff0 con 0x7f680415c680 2023-08-09T13:34:22.628 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:22.623+0000 7f680ae34640 1 -- 172.21.15.43:0/760937445 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f680414bf00 con 0x7f680415c680 2023-08-09T13:34:22.628 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:22.624+0000 7f680ae34640 1 -- 172.21.15.43:0/760937445 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f68040ca510 con 0x7f680415c680 2023-08-09T13:34:22.629 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:22.625+0000 7f67f97fa640 1 -- 172.21.15.43:0/760937445 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 5) v1 ==== 88341+0+0 (secure 0 0 0) 0x7f67f4007bd0 con 0x7f680415c680 2023-08-09T13:34:22.629 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:22.626+0000 7f67f97fa640 1 --2- 172.21.15.43:0/760937445 >> [v2:172.21.15.203:6832/830097474,v1:172.21.15.203:6833/830097474] conn(0x7f67f006a050 0x7f67f006c510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:34:22.629 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:22.626+0000 7f67f97fa640 1 -- 172.21.15.43:0/760937445 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6813+0+0 (secure 0 0 0) 0x7f67f40888b0 con 0x7f680415c680 2023-08-09T13:34:22.629 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:22.626+0000 7f6808bd0640 1 --2- 172.21.15.43:0/760937445 >> [v2:172.21.15.203:6832/830097474,v1:172.21.15.203:6833/830097474] conn(0x7f67f006a050 0x7f67f006c510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:34:22.630 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:22.628+0000 7f6808bd0640 1 --2- 172.21.15.43:0/760937445 >> [v2:172.21.15.203:6832/830097474,v1:172.21.15.203:6833/830097474] conn(0x7f67f006a050 0x7f67f006c510 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f6800036000 tx=0x7f680003c040 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-09T13:34:22.632 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:22.630+0000 7f67f97fa640 1 -- 172.21.15.43:0/760937445 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178066 (secure 0 0 0) 0x7f68040ca510 con 0x7f680415c680 2023-08-09T13:34:22.846 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:22.843+0000 7f680ae34640 1 -- 172.21.15.43:0/760937445 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7f68040ca510 con 0x7f680415c680 2023-08-09T13:34:22.847 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:22.844+0000 7f67f97fa640 1 -- 172.21.15.43:0/760937445 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v24) v1 ==== 80+0+0 (secure 0 0 0) 0x7f68040ca510 con 0x7f680415c680 2023-08-09T13:34:22.847 INFO:teuthology.orchestra.run.smithi043.stderr:listed 0 entries 2023-08-09T13:34:22.849 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:22.846+0000 7f680ae34640 1 -- 172.21.15.43:0/760937445 >> [v2:172.21.15.203:6832/830097474,v1:172.21.15.203:6833/830097474] conn(0x7f67f006a050 msgr2=0x7f67f006c510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:34:22.849 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:22.846+0000 7f680ae34640 1 --2- 172.21.15.43:0/760937445 >> [v2:172.21.15.203:6832/830097474,v1:172.21.15.203:6833/830097474] conn(0x7f67f006a050 0x7f67f006c510 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f6800036000 tx=0x7f680003c040 comp rx=0 tx=0).stop 2023-08-09T13:34:22.849 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:22.847+0000 7f680ae34640 1 -- 172.21.15.43:0/760937445 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f680415c680 msgr2=0x7f68040ca7f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:34:22.849 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:22.847+0000 7f680ae34640 1 --2- 172.21.15.43:0/760937445 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f680415c680 0x7f68040ca7f0 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7f67f400c7f0 tx=0x7f67f400ccb0 comp rx=0 tx=0).stop 2023-08-09T13:34:22.850 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:22.847+0000 7f680ae34640 1 -- 172.21.15.43:0/760937445 shutdown_connections 2023-08-09T13:34:22.850 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:22.847+0000 7f680ae34640 1 --2- 172.21.15.43:0/760937445 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f68040cad30 0x7f68040cbb20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:34:22.850 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:22.847+0000 7f680ae34640 1 --2- 172.21.15.43:0/760937445 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f680415c680 0x7f68040ca7f0 unknown :-1 s=CLOSED pgs=128 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:34:22.851 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:22.847+0000 7f680ae34640 1 --2- 172.21.15.43:0/760937445 >> [v2:172.21.15.203:6832/830097474,v1:172.21.15.203:6833/830097474] conn(0x7f67f006a050 0x7f67f006c510 unknown :-1 s=CLOSED pgs=86 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:34:22.851 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:22.847+0000 7f680ae34640 1 --2- 172.21.15.43:0/760937445 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f68040d0a70 0x7f680414b9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:34:22.851 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:22.847+0000 7f680ae34640 1 -- 172.21.15.43:0/760937445 >> 172.21.15.43:0/760937445 conn(0x7f6804071520 msgr2=0x7f68040ce120 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-09T13:34:22.851 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:22.847+0000 7f680ae34640 1 -- 172.21.15.43:0/760937445 shutdown_connections 2023-08-09T13:34:22.851 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:22.847+0000 7f680ae34640 1 -- 172.21.15.43:0/760937445 wait complete. 2023-08-09T13:34:22.868 INFO:tasks.cephfs.fuse_mount:Running fusermount -u on ubuntu@smithi043.front.sepia.ceph.com... 2023-08-09T13:34:22.868 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-09T13:34:22.869 DEBUG:teuthology.orchestra.run.smithi043:> sudo fusermount -u /home/ubuntu/cephtest/mnt.admin 2023-08-09T13:34:22.966 INFO:tasks.cephfs.fuse_mount.ceph-fuse.admin.smithi043.stderr:ceph-fuse[94557]: fuse finished with error 0 and tester_r 0 2023-08-09T13:34:22.972 INFO:teuthology.orchestra.run:waiting for 300 2023-08-09T13:34:28.677 DEBUG:teuthology.orchestra.run.smithi043:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-08-09T13:34:28.687 DEBUG:teuthology.orchestra.run.smithi203:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-08-09T13:34:29.074 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi043.front.sepia.ceph.com 2023-08-09T13:34:29.075 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-09T13:34:29.075 DEBUG:teuthology.orchestra.run.smithi043:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.admin) 2023-08-09T13:34:29.150 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-08-09T13:34:29.386 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:29.383+0000 7f7762f67640 1 Processor -- start 2023-08-09T13:34:29.386 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:29.383+0000 7f7762f67640 1 -- start start 2023-08-09T13:34:29.387 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:29.385+0000 7f7762f67640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f775c0c96c0 0x7f775c15ad20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:34:29.388 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:29.385+0000 7f7762f67640 1 -- --> v1:172.21.15.43:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f775c0cdc70 con 0x7f775c15a6a0 2023-08-09T13:34:29.388 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:29.385+0000 7f7762f67640 1 -- --> v1:172.21.15.203:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f775c070910 con 0x7f775c0713b0 2023-08-09T13:34:29.388 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:29.385+0000 7f7762f67640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f775c0c88c0 con 0x7f775c0c96c0 2023-08-09T13:34:29.388 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:29.385+0000 7f7761f65640 1 --1- >> v1:172.21.15.43:6789/0 conn(0x7f775c15a6a0 0x7f775c0cbd90 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.43:6789/0 says I am v1:172.21.15.43:59514/0 (socket says 172.21.15.43:59514) 2023-08-09T13:34:29.388 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:29.385+0000 7f7761f65640 1 -- 172.21.15.43:0/2918675018 learned_addr learned my addr 172.21.15.43:0/2918675018 (peer_addr_for_me v1:172.21.15.43:0/0) 2023-08-09T13:34:29.390 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:29.386+0000 7f7762766640 1 --2- 172.21.15.43:0/2918675018 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f775c0c96c0 0x7f775c15ad20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:34:29.390 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:29.386+0000 7f7760f63640 1 -- 172.21.15.43:0/2918675018 <== mon.0 v1:172.21.15.43:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 726013807 0 0) 0x7f775c0cdc70 con 0x7f775c15a6a0 2023-08-09T13:34:29.390 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:29.386+0000 7f7760f63640 1 -- 172.21.15.43:0/2918675018 --> v1:172.21.15.43:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f7748005730 con 0x7f775c15a6a0 2023-08-09T13:34:29.391 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:29.386+0000 7f7760f63640 1 -- 172.21.15.43:0/2918675018 <== mon.0 v1:172.21.15.43:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3775250559 0 0) 0x7f7748005730 con 0x7f775c15a6a0 2023-08-09T13:34:29.391 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:29.386+0000 7f7760f63640 1 -- 172.21.15.43:0/2918675018 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f775c0c96c0 msgr2=0x7f775c15ad20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-09T13:34:29.391 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:29.386+0000 7f7760f63640 1 --2- 172.21.15.43:0/2918675018 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f775c0c96c0 0x7f775c15ad20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:34:29.391 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:29.386+0000 7f7760f63640 1 -- 172.21.15.43:0/2918675018 >> v1:172.21.15.203:6789/0 conn(0x7f775c0713b0 legacy=0x7f775c0c8fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:34:29.391 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:29.386+0000 7f7760f63640 1 -- 172.21.15.43:0/2918675018 --> v1:172.21.15.43:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f775c070bb0 con 0x7f775c15a6a0 2023-08-09T13:34:29.391 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:29.386+0000 7f7760f63640 1 -- 172.21.15.43:0/2918675018 <== mon.0 v1:172.21.15.43:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1285150740 0 0) 0x7f7758001670 con 0x7f775c15a6a0 2023-08-09T13:34:29.391 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:29.386+0000 7f7760f63640 1 -- 172.21.15.43:0/2918675018 >> v1:172.21.15.43:6789/0 conn(0x7f775c15a6a0 legacy=0x7f775c0cbd90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:34:29.392 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:29.386+0000 7f7760f63640 1 --2- 172.21.15.43:0/2918675018 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f7748005bb0 0x7f7748005fd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:34:29.392 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:29.386+0000 7f7760f63640 1 --2- 172.21.15.43:0/2918675018 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f77480069d0 0x7f7748006e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:34:29.392 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:29.386+0000 7f7760f63640 1 --2- 172.21.15.43:0/2918675018 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f775c0c96c0 0x7f7748007440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:34:29.393 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:29.386+0000 7f7760f63640 1 -- 172.21.15.43:0/2918675018 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f775c0c88c0 con 0x7f77480069d0 2023-08-09T13:34:29.393 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:29.386+0000 7f7760f63640 1 -- 172.21.15.43:0/2918675018 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f774800dd50 con 0x7f7748005bb0 2023-08-09T13:34:29.393 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:29.386+0000 7f7760f63640 1 -- 172.21.15.43:0/2918675018 --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f774800df10 con 0x7f775c0c96c0 2023-08-09T13:34:29.393 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:29.387+0000 7f7761764640 1 --2- 172.21.15.43:0/2918675018 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f77480069d0 0x7f7748006e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:34:29.393 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:29.387+0000 7f7762766640 1 --2- 172.21.15.43:0/2918675018 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f775c0c96c0 0x7f7748007440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:34:29.393 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:29.387+0000 7f7761f65640 1 --2- 172.21.15.43:0/2918675018 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f7748005bb0 0x7f7748005fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:34:29.393 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:29.387+0000 7f7761764640 1 -- 172.21.15.43:0/2918675018 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f775c0c96c0 msgr2=0x7f7748007440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-09T13:34:29.394 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:29.387+0000 7f7761764640 1 --2- 172.21.15.43:0/2918675018 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f775c0c96c0 0x7f7748007440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:34:29.394 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:29.387+0000 7f7761764640 1 -- 172.21.15.43:0/2918675018 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f7748005bb0 msgr2=0x7f7748005fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-09T13:34:29.394 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:29.387+0000 7f7761764640 1 --2- 172.21.15.43:0/2918675018 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f7748005bb0 0x7f7748005fd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:34:29.394 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:29.387+0000 7f7761764640 1 -- 172.21.15.43:0/2918675018 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7748005730 con 0x7f77480069d0 2023-08-09T13:34:29.396 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:29.388+0000 7f7761764640 1 --2- 172.21.15.43:0/2918675018 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f77480069d0 0x7f7748006e50 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7f774c001750 tx=0x7f7748007a00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1bf5632aa6ae09c3 server_cookie=0 in_seq=0 out_seq=0 2023-08-09T13:34:29.396 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:29.389+0000 7f7760f63640 1 -- 172.21.15.43:0/2918675018 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f774c0052d0 con 0x7f77480069d0 2023-08-09T13:34:29.396 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:29.389+0000 7f7760f63640 1 -- 172.21.15.43:0/2918675018 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f774c02e050 con 0x7f77480069d0 2023-08-09T13:34:29.397 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:29.389+0000 7f7760f63640 1 -- 172.21.15.43:0/2918675018 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f774c0035f0 con 0x7f77480069d0 2023-08-09T13:34:29.397 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:29.389+0000 7f7762f67640 1 -- 172.21.15.43:0/2918675018 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f77480069d0 msgr2=0x7f7748006e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:34:29.397 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:29.389+0000 7f7762f67640 1 --2- 172.21.15.43:0/2918675018 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f77480069d0 0x7f7748006e50 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7f774c001750 tx=0x7f7748007a00 comp rx=0 tx=0).stop 2023-08-09T13:34:29.397 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:29.389+0000 7f7762766640 1 -- 172.21.15.43:0/2918675018 reap_dead start 2023-08-09T13:34:29.397 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:29.389+0000 7f7762f67640 1 -- 172.21.15.43:0/2918675018 shutdown_connections 2023-08-09T13:34:29.397 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:29.389+0000 7f7762f67640 1 -- 172.21.15.43:0/2918675018 >> 172.21.15.43:0/2918675018 conn(0x7f775c0cde90 msgr2=0x7f775c0ce2a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-09T13:34:29.397 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:29.389+0000 7f7762f67640 1 -- 172.21.15.43:0/2918675018 shutdown_connections 2023-08-09T13:34:29.398 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:29.389+0000 7f7762f67640 1 -- 172.21.15.43:0/2918675018 wait complete. 2023-08-09T13:34:29.398 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:29.391+0000 7f7762f67640 1 Processor -- start 2023-08-09T13:34:29.398 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:29.392+0000 7f7762f67640 1 -- start start 2023-08-09T13:34:29.398 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:29.393+0000 7f7762f67640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f77480069d0 0x7f775c153410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:34:29.398 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:29.393+0000 7f7762f67640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f775c0aadf0 0x7f775c15ab10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:34:29.398 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:29.393+0000 7f7762f67640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f775c15c400 0x7f775c15cdb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:34:29.398 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:29.393+0000 7f7762f67640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f775c153950 con 0x7f775c15c400 2023-08-09T13:34:29.398 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:29.393+0000 7f7762f67640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f775c15b3a0 con 0x7f775c0aadf0 2023-08-09T13:34:29.399 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:29.393+0000 7f7762f67640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f775c15b640 con 0x7f77480069d0 2023-08-09T13:34:29.399 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:29.393+0000 7f7762766640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f775c15c400 0x7f775c15cdb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:34:29.399 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:29.393+0000 7f7762766640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f775c15c400 0x7f775c15cdb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:37552/0 (socket says 172.21.15.43:37552) 2023-08-09T13:34:29.399 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:29.393+0000 7f7762766640 1 -- 172.21.15.43:0/3252154850 learned_addr learned my addr 172.21.15.43:0/3252154850 (peer_addr_for_me v2:172.21.15.43:0/0) 2023-08-09T13:34:29.399 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:29.393+0000 7f7761f65640 1 --2- 172.21.15.43:0/3252154850 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f77480069d0 0x7f775c153410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:34:29.399 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:29.393+0000 7f7761764640 1 --2- 172.21.15.43:0/3252154850 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f775c0aadf0 0x7f775c15ab10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:34:29.399 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:29.393+0000 7f7762766640 1 -- 172.21.15.43:0/3252154850 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f77480069d0 msgr2=0x7f775c153410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:34:29.399 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:29.393+0000 7f7762766640 1 --2- 172.21.15.43:0/3252154850 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f77480069d0 0x7f775c153410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:34:29.400 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:29.394+0000 7f7762766640 1 -- 172.21.15.43:0/3252154850 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f775c0aadf0 msgr2=0x7f775c15ab10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:34:29.400 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:29.394+0000 7f7762766640 1 --2- 172.21.15.43:0/3252154850 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f775c0aadf0 0x7f775c15ab10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:34:29.400 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:29.394+0000 7f7762766640 1 -- 172.21.15.43:0/3252154850 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f775c15b920 con 0x7f775c15c400 2023-08-09T13:34:29.400 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:29.394+0000 7f7762766640 1 --2- 172.21.15.43:0/3252154850 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f775c15c400 0x7f775c15cdb0 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7f7748006950 tx=0x7f775400f580 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-09T13:34:29.401 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:29.394+0000 7f77527fc640 1 -- 172.21.15.43:0/3252154850 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f775400e040 con 0x7f775c15c400 2023-08-09T13:34:29.401 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:29.394+0000 7f77527fc640 1 -- 172.21.15.43:0/3252154850 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f775400f040 con 0x7f775c15c400 2023-08-09T13:34:29.401 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:29.394+0000 7f77527fc640 1 -- 172.21.15.43:0/3252154850 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7754015670 con 0x7f775c15c400 2023-08-09T13:34:29.401 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:29.394+0000 7f7762f67640 1 -- 172.21.15.43:0/3252154850 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f775c15bc00 con 0x7f775c15c400 2023-08-09T13:34:29.402 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:29.394+0000 7f7762f67640 1 -- 172.21.15.43:0/3252154850 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f775c15b050 con 0x7f775c15c400 2023-08-09T13:34:29.402 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:29.396+0000 7f7762f67640 1 -- 172.21.15.43:0/3252154850 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f775c0739f0 con 0x7f775c15c400 2023-08-09T13:34:29.402 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:29.396+0000 7f77527fc640 1 -- 172.21.15.43:0/3252154850 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 5) v1 ==== 88341+0+0 (secure 0 0 0) 0x7f77540157d0 con 0x7f775c15c400 2023-08-09T13:34:29.402 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:29.397+0000 7f77527fc640 1 --2- 172.21.15.43:0/3252154850 >> [v2:172.21.15.203:6832/830097474,v1:172.21.15.203:6833/830097474] conn(0x7f774006a080 0x7f774006c540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:34:29.402 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:29.398+0000 7f77527fc640 1 -- 172.21.15.43:0/3252154850 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6813+0+0 (secure 0 0 0) 0x7f7754089330 con 0x7f775c15c400 2023-08-09T13:34:29.402 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:29.398+0000 7f7761f65640 1 --2- 172.21.15.43:0/3252154850 >> [v2:172.21.15.203:6832/830097474,v1:172.21.15.203:6833/830097474] conn(0x7f774006a080 0x7f774006c540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:34:29.403 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:29.399+0000 7f7761f65640 1 --2- 172.21.15.43:0/3252154850 >> [v2:172.21.15.203:6832/830097474,v1:172.21.15.203:6833/830097474] conn(0x7f774006a080 0x7f774006c540 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f7758001050 tx=0x7f7758009040 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-09T13:34:29.404 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:29.402+0000 7f77527fc640 1 -- 172.21.15.43:0/3252154850 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178066 (secure 0 0 0) 0x7f7754054720 con 0x7f775c15c400 2023-08-09T13:34:29.602 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:29.600+0000 7f7762f67640 1 -- 172.21.15.43:0/3252154850 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f775c0cec10 con 0x7f775c15c400 2023-08-09T13:34:29.604 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:29.602+0000 7f77527fc640 1 -- 172.21.15.43:0/3252154850 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v24) v1 ==== 74+0+20050 (secure 0 0 0) 0x7f7754057d40 con 0x7f775c15c400 2023-08-09T13:34:29.605 INFO:teuthology.orchestra.run.smithi043.stdout: 2023-08-09T13:34:29.605 INFO:teuthology.orchestra.run.smithi043.stdout:{"epoch":24,"fsid":"daac87ee-1337-4250-a955-aea4e408a39b","created":"2023-08-09T13:33:41.701198+0000","modified":"2023-08-09T13:34:07.709429+0000","last_up_change":"2023-08-09T13:33:53.783453+0000","last_in_change":"2023-08-09T13:33:45.148904+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":5,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":"rbd","create_time":"2023-08-09T13:33:54.365137+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":".mgr","create_time":"2023-08-09T13:33:54.831803+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_metadata","create_time":"2023-08-09T13:34:00.207974+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data","create_time":"2023-08-09T13:34:01.614765+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":5,"pool_name":"cephfs_data_ec","create_time":"2023-08-09T13:34:05.239583+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"24","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"cbd49ecd-e896-4fbd-864e-5c37a829b165","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6800","nonce":502667335},{"type":"v1","addr":"172.21.15.43:6801","nonce":502667335}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6802","nonce":502667335},{"type":"v1","addr":"172.21.15.43:6803","nonce":502667335}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6806","nonce":502667335},{"type":"v1","addr":"172.21.15.43:6807","nonce":502667335}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6804","nonce":502667335},{"type":"v1","addr":"172.21.15.43:6805","nonce":502667335}]},"public_addr":"172.21.15.43:6801/502667335","cluster_addr":"172.21.15.43:6803/502667335","heartbeat_back_addr":"172.21.15.43:6807/502667335","heartbeat_front_addr":"172.21.15.43:6805/502667335","state":["exists","up"]},{"osd":1,"uuid":"75c7e0fe-1ec7-4fe8-a2f2-3661a65b8500","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6808","nonce":3289134610},{"type":"v1","addr":"172.21.15.43:6809","nonce":3289134610}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6810","nonce":3289134610},{"type":"v1","addr":"172.21.15.43:6812","nonce":3289134610}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6821","nonce":3289134610},{"type":"v1","addr":"172.21.15.43:6824","nonce":3289134610}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6815","nonce":3289134610},{"type":"v1","addr":"172.21.15.43:6818","nonce":3289134610}]},"public_addr":"172.21.15.43:6809/3289134610","cluster_addr":"172.21.15.43:6812/3289134610","heartbeat_back_addr":"172.21.15.43:6824/3289134610","heartbeat_front_addr":"172.21.15.43:6818/3289134610","state":["exists","up"]},{"osd":2,"uuid":"bb5c41e9-7f5f-4125-9b01-3bb8bfc2ea83","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6811","nonce":4079709736},{"type":"v1","addr":"172.21.15.43:6814","nonce":4079709736}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6817","nonce":4079709736},{"type":"v1","addr":"172.21.15.43:6820","nonce":4079709736}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6829","nonce":4079709736},{"type":"v1","addr":"172.21.15.43:6831","nonce":4079709736}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6823","nonce":4079709736},{"type":"v1","addr":"172.21.15.43:6826","nonce":4079709736}]},"public_addr":"172.21.15.43:6814/4079709736","cluster_addr":"172.21.15.43:6820/4079709736","heartbeat_back_addr":"172.21.15.43:6831/4079709736","heartbeat_front_addr":"172.21.15.43:6826/4079709736","state":["exists","up"]},{"osd":3,"uuid":"bfe4c8b4-694a-465c-87e6-2b2aab852613","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6813","nonce":1709538759},{"type":"v1","addr":"172.21.15.43:6816","nonce":1709538759}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6819","nonce":1709538759},{"type":"v1","addr":"172.21.15.43:6822","nonce":1709538759}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6828","nonce":1709538759},{"type":"v1","addr":"172.21.15.43:6830","nonce":1709538759}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6825","nonce":1709538759},{"type":"v1","addr":"172.21.15.43:6827","nonce":1709538759}]},"public_addr":"172.21.15.43:6816/1709538759","cluster_addr":"172.21.15.43:6822/1709538759","heartbeat_back_addr":"172.21.15.43:6830/1709538759","heartbeat_front_addr":"172.21.15.43:6827/1709538759","state":["exists","up"]},{"osd":4,"uuid":"fa2e8a77-aba1-4756-853e-0d174a5bdeba","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6800","nonce":4196866481},{"type":"v1","addr":"172.21.15.203:6801","nonce":4196866481}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6802","nonce":4196866481},{"type":"v1","addr":"172.21.15.203:6803","nonce":4196866481}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6806","nonce":4196866481},{"type":"v1","addr":"172.21.15.203:6807","nonce":4196866481}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6804","nonce":4196866481},{"type":"v1","addr":"172.21.15.203:6805","nonce":4196866481}]},"public_addr":"172.21.15.203:6801/4196866481","cluster_addr":"172.21.15.203:6803/4196866481","heartbeat_back_addr":"172.21.15.203:6807/4196866481","heartbeat_front_addr":"172.21.15.203:6805/4196866481","state":["exists","up"]},{"osd":5,"uuid":"81811e4b-4d24-4f51-a82c-f1d700a2a00d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6816","nonce":3059290081},{"type":"v1","addr":"172.21.15.203:6817","nonce":3059290081}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6818","nonce":3059290081},{"type":"v1","addr":"172.21.15.203:6819","nonce":3059290081}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6822","nonce":3059290081},{"type":"v1","addr":"172.21.15.203:6823","nonce":3059290081}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6820","nonce":3059290081},{"type":"v1","addr":"172.21.15.203:6821","nonce":3059290081}]},"public_addr":"172.21.15.203:6817/3059290081","cluster_addr":"172.21.15.203:6819/3059290081","heartbeat_back_addr":"172.21.15.203:6823/3059290081","heartbeat_front_addr":"172.21.15.203:6821/3059290081","state":["exists","up"]},{"osd":6,"uuid":"307bac4d-6a5b-443d-af75-56d695d7674f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6824","nonce":512155569},{"type":"v1","addr":"172.21.15.203:6825","nonce":512155569}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6826","nonce":512155569},{"type":"v1","addr":"172.21.15.203:6827","nonce":512155569}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6830","nonce":512155569},{"type":"v1","addr":"172.21.15.203:6831","nonce":512155569}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6828","nonce":512155569},{"type":"v1","addr":"172.21.15.203:6829","nonce":512155569}]},"public_addr":"172.21.15.203:6825/512155569","cluster_addr":"172.21.15.203:6827/512155569","heartbeat_back_addr":"172.21.15.203:6831/512155569","heartbeat_front_addr":"172.21.15.203:6829/512155569","state":["exists","up"]},{"osd":7,"uuid":"a95dc7ca-9f1b-4295-af70-68a22a2d7d5e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6808","nonce":3331450438},{"type":"v1","addr":"172.21.15.203:6809","nonce":3331450438}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6810","nonce":3331450438},{"type":"v1","addr":"172.21.15.203:6811","nonce":3331450438}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6814","nonce":3331450438},{"type":"v1","addr":"172.21.15.203:6815","nonce":3331450438}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6812","nonce":3331450438},{"type":"v1","addr":"172.21.15.203:6813","nonce":3331450438}]},"public_addr":"172.21.15.203:6809/3331450438","cluster_addr":"172.21.15.203:6811/3331450438","heartbeat_back_addr":"172.21.15.203:6815/3331450438","heartbeat_front_addr":"172.21.15.203:6813/3331450438","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-09T13:33:50.547652+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-09T13:33:50.589469+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-09T13:33:50.561724+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-09T13:33:50.599650+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-09T13:33:50.411855+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-09T13:33:50.219681+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-09T13:33:50.504664+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-09T13:33:50.441248+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-08-09T13:34:29.608 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:29.605+0000 7f7762f67640 1 -- 172.21.15.43:0/3252154850 >> [v2:172.21.15.203:6832/830097474,v1:172.21.15.203:6833/830097474] conn(0x7f774006a080 msgr2=0x7f774006c540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:34:29.608 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:29.605+0000 7f7762f67640 1 --2- 172.21.15.43:0/3252154850 >> [v2:172.21.15.203:6832/830097474,v1:172.21.15.203:6833/830097474] conn(0x7f774006a080 0x7f774006c540 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f7758001050 tx=0x7f7758009040 comp rx=0 tx=0).stop 2023-08-09T13:34:29.608 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:29.605+0000 7f7762f67640 1 -- 172.21.15.43:0/3252154850 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f775c15c400 msgr2=0x7f775c15cdb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:34:29.608 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:29.605+0000 7f7762f67640 1 --2- 172.21.15.43:0/3252154850 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f775c15c400 0x7f775c15cdb0 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7f7748006950 tx=0x7f775400f580 comp rx=0 tx=0).stop 2023-08-09T13:34:29.609 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:29.605+0000 7f7762f67640 1 -- 172.21.15.43:0/3252154850 shutdown_connections 2023-08-09T13:34:29.609 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:29.605+0000 7f7762f67640 1 --2- 172.21.15.43:0/3252154850 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f775c15c400 0x7f775c15cdb0 unknown :-1 s=CLOSED pgs=131 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:34:29.609 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:29.605+0000 7f7762f67640 1 --2- 172.21.15.43:0/3252154850 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f775c0aadf0 0x7f775c15ab10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:34:29.609 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:29.605+0000 7f7762f67640 1 --2- 172.21.15.43:0/3252154850 >> [v2:172.21.15.203:6832/830097474,v1:172.21.15.203:6833/830097474] conn(0x7f774006a080 0x7f774006c540 unknown :-1 s=CLOSED pgs=87 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:34:29.609 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:29.605+0000 7f7762f67640 1 --2- 172.21.15.43:0/3252154850 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f77480069d0 0x7f775c153410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:34:29.609 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:29.605+0000 7f7762f67640 1 -- 172.21.15.43:0/3252154850 >> 172.21.15.43:0/3252154850 conn(0x7f775c0cde90 msgr2=0x7f775c0ae3b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-09T13:34:29.609 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:29.606+0000 7f7762f67640 1 -- 172.21.15.43:0/3252154850 shutdown_connections 2023-08-09T13:34:29.609 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:29.606+0000 7f7762f67640 1 -- 172.21.15.43:0/3252154850 wait complete. 2023-08-09T13:34:29.629 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-08-09T13:34:29.856 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:29.853+0000 7f34ef90c640 1 Processor -- start 2023-08-09T13:34:29.856 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:29.853+0000 7f34ef90c640 1 -- start start 2023-08-09T13:34:29.857 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:29.855+0000 7f34ef90c640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f34e8143c20 0x7f34e806f540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:34:29.858 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:29.855+0000 7f34ef90c640 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7f34e806cfe0 0x7f34e80c8b50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:34:29.858 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:29.855+0000 7f34ef90c640 1 -- --> v1:172.21.15.203:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f34e80c9090 con 0x7f34e80c86f0 2023-08-09T13:34:29.858 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:29.855+0000 7f34ef90c640 1 -- --> v2:172.21.15.203:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f34e8144000 con 0x7f34e806cfe0 2023-08-09T13:34:29.858 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:29.855+0000 7f34ef90c640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f34e806c440 con 0x7f34e8143c20 2023-08-09T13:34:29.859 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:29.856+0000 7f34ed6a8640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f34e8143c20 0x7f34e806f540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:34:29.859 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:29.856+0000 7f34ed6a8640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f34e8143c20 0x7f34e806f540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.203:3301/0 says I am v2:172.21.15.43:36268/0 (socket says 172.21.15.43:36268) 2023-08-09T13:34:29.859 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:29.856+0000 7f34ed6a8640 1 -- 172.21.15.43:0/1369724940 learned_addr learned my addr 172.21.15.43:0/1369724940 (peer_addr_for_me v2:172.21.15.43:0/0) 2023-08-09T13:34:29.859 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:29.856+0000 7f34edea9640 1 --2- 172.21.15.43:0/1369724940 >> v2:172.21.15.203:3300/0 conn(0x7f34e806cfe0 0x7f34e80c8b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:34:29.859 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:29.857+0000 7f34ed6a8640 1 -- 172.21.15.43:0/1369724940 >> v2:172.21.15.203:3300/0 conn(0x7f34e806cfe0 msgr2=0x7f34e80c8b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-09T13:34:29.860 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:29.857+0000 7f34ed6a8640 1 --2- 172.21.15.43:0/1369724940 >> v2:172.21.15.203:3300/0 conn(0x7f34e806cfe0 0x7f34e80c8b50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:34:29.860 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:29.857+0000 7f34ed6a8640 1 -- 172.21.15.43:0/1369724940 >> v1:172.21.15.203:6789/0 conn(0x7f34e80c86f0 legacy=0x7f34e806c8d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:34:29.860 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:29.857+0000 7f34ed6a8640 1 -- 172.21.15.43:0/1369724940 --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f34e806b830 con 0x7f34e8143c20 2023-08-09T13:34:29.861 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:29.858+0000 7f34ed6a8640 1 --2- 172.21.15.43:0/1369724940 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f34e8143c20 0x7f34e806f540 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7f34e4009870 tx=0x7f34e402b8f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c83e5c494b4365fd server_cookie=0 in_seq=0 out_seq=0 2023-08-09T13:34:29.861 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:29.859+0000 7f34dffff640 1 -- 172.21.15.43:0/1369724940 <== mon.2 v2:172.21.15.203:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f34e4031640 con 0x7f34e8143c20 2023-08-09T13:34:29.861 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:29.859+0000 7f34dffff640 1 -- 172.21.15.43:0/1369724940 <== mon.2 v2:172.21.15.203:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f34e40317a0 con 0x7f34e8143c20 2023-08-09T13:34:29.862 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:29.859+0000 7f34dffff640 1 -- 172.21.15.43:0/1369724940 <== mon.2 v2:172.21.15.203:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f34e4031b30 con 0x7f34e8143c20 2023-08-09T13:34:29.862 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:29.860+0000 7f34ef90c640 1 -- 172.21.15.43:0/1369724940 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f34e8143c20 msgr2=0x7f34e806f540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:34:29.862 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:29.860+0000 7f34ef90c640 1 --2- 172.21.15.43:0/1369724940 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f34e8143c20 0x7f34e806f540 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7f34e4009870 tx=0x7f34e402b8f0 comp rx=0 tx=0).stop 2023-08-09T13:34:29.863 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:29.860+0000 7f34ef90c640 1 -- 172.21.15.43:0/1369724940 shutdown_connections 2023-08-09T13:34:29.863 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:29.860+0000 7f34ef90c640 1 --2- 172.21.15.43:0/1369724940 >> v2:172.21.15.203:3300/0 conn(0x7f34e806cfe0 0x7f34e80c8b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:34:29.863 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:29.860+0000 7f34ef90c640 1 --2- 172.21.15.43:0/1369724940 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f34e8143c20 0x7f34e806f540 unknown :-1 s=CLOSED pgs=42 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:34:29.863 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:29.860+0000 7f34ef90c640 1 -- 172.21.15.43:0/1369724940 >> 172.21.15.43:0/1369724940 conn(0x7f34e80ce0a0 msgr2=0x7f34e80ce480 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-09T13:34:29.863 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:29.860+0000 7f34ef90c640 1 -- 172.21.15.43:0/1369724940 shutdown_connections 2023-08-09T13:34:29.863 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:29.860+0000 7f34ef90c640 1 -- 172.21.15.43:0/1369724940 wait complete. 2023-08-09T13:34:29.864 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:29.862+0000 7f34ef90c640 1 Processor -- start 2023-08-09T13:34:29.865 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:29.862+0000 7f34ef90c640 1 -- start start 2023-08-09T13:34:29.867 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:29.864+0000 7f34ef90c640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f34e806cfe0 0x7f34e81598a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:34:29.867 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:29.864+0000 7f34ef90c640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f34e80c86f0 0x7f34e80c5b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:34:29.867 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:29.864+0000 7f34ef90c640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f34e8143c20 0x7f34e80c7630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:34:29.867 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:29.864+0000 7f34ef90c640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f34e815d420 con 0x7f34e80c86f0 2023-08-09T13:34:29.867 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:29.864+0000 7f34ef90c640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f34e815a9d0 con 0x7f34e8143c20 2023-08-09T13:34:29.867 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:29.864+0000 7f34ef90c640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f34e80c6600 con 0x7f34e806cfe0 2023-08-09T13:34:29.868 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:29.864+0000 7f34ecea7640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f34e80c86f0 0x7f34e80c5b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:34:29.868 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:29.864+0000 7f34ecea7640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f34e80c86f0 0x7f34e80c5b80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:37558/0 (socket says 172.21.15.43:37558) 2023-08-09T13:34:29.868 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:29.864+0000 7f34ecea7640 1 -- 172.21.15.43:0/158142038 learned_addr learned my addr 172.21.15.43:0/158142038 (peer_addr_for_me v2:172.21.15.43:0/0) 2023-08-09T13:34:29.868 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:29.864+0000 7f34ed6a8640 1 --2- 172.21.15.43:0/158142038 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f34e806cfe0 0x7f34e81598a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:34:29.869 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:29.864+0000 7f34edea9640 1 --2- 172.21.15.43:0/158142038 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f34e8143c20 0x7f34e80c7630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:34:29.869 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:29.865+0000 7f34ecea7640 1 -- 172.21.15.43:0/158142038 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f34e806cfe0 msgr2=0x7f34e81598a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:34:29.869 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:29.865+0000 7f34ecea7640 1 --2- 172.21.15.43:0/158142038 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f34e806cfe0 0x7f34e81598a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:34:29.870 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:29.865+0000 7f34ecea7640 1 -- 172.21.15.43:0/158142038 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f34e8143c20 msgr2=0x7f34e80c7630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:34:29.870 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:29.865+0000 7f34ecea7640 1 --2- 172.21.15.43:0/158142038 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f34e8143c20 0x7f34e80c7630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:34:29.870 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:29.865+0000 7f34ecea7640 1 -- 172.21.15.43:0/158142038 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f34e40094d0 con 0x7f34e80c86f0 2023-08-09T13:34:29.870 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:29.865+0000 7f34ecea7640 1 --2- 172.21.15.43:0/158142038 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f34e80c86f0 0x7f34e80c5b80 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7f34d800c430 tx=0x7f34d800c8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-09T13:34:29.870 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:29.865+0000 7f34ddffb640 1 -- 172.21.15.43:0/158142038 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f34d80052d0 con 0x7f34e80c86f0 2023-08-09T13:34:29.870 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:29.865+0000 7f34ef90c640 1 -- 172.21.15.43:0/158142038 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f34e80c68e0 con 0x7f34e80c86f0 2023-08-09T13:34:29.870 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:29.866+0000 7f34ef90c640 1 -- 172.21.15.43:0/158142038 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f34e815af00 con 0x7f34e80c86f0 2023-08-09T13:34:29.870 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:29.866+0000 7f34ddffb640 1 -- 172.21.15.43:0/158142038 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f34d8005430 con 0x7f34e80c86f0 2023-08-09T13:34:29.871 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:29.866+0000 7f34ddffb640 1 -- 172.21.15.43:0/158142038 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f34d8003830 con 0x7f34e80c86f0 2023-08-09T13:34:29.871 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:29.867+0000 7f34ef90c640 1 -- 172.21.15.43:0/158142038 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f34e815b0f0 con 0x7f34e80c86f0 2023-08-09T13:34:29.871 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:29.868+0000 7f34ddffb640 1 -- 172.21.15.43:0/158142038 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 5) v1 ==== 88341+0+0 (secure 0 0 0) 0x7f34d8003990 con 0x7f34e80c86f0 2023-08-09T13:34:29.872 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:29.870+0000 7f34ddffb640 1 --2- 172.21.15.43:0/158142038 >> [v2:172.21.15.203:6832/830097474,v1:172.21.15.203:6833/830097474] conn(0x7f34cc06a050 0x7f34cc06c510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:34:29.873 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:29.870+0000 7f34ddffb640 1 -- 172.21.15.43:0/158142038 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6813+0+0 (secure 0 0 0) 0x7f34d8087eb0 con 0x7f34e80c86f0 2023-08-09T13:34:29.873 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:29.871+0000 7f34ed6a8640 1 --2- 172.21.15.43:0/158142038 >> [v2:172.21.15.203:6832/830097474,v1:172.21.15.203:6833/830097474] conn(0x7f34cc06a050 0x7f34cc06c510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:34:29.874 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:29.872+0000 7f34ed6a8640 1 --2- 172.21.15.43:0/158142038 >> [v2:172.21.15.203:6832/830097474,v1:172.21.15.203:6833/830097474] conn(0x7f34cc06a050 0x7f34cc06c510 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f34e4036000 tx=0x7f34e403c040 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-09T13:34:29.876 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:29.874+0000 7f34ddffb640 1 -- 172.21.15.43:0/158142038 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178066 (secure 0 0 0) 0x7f34d8053240 con 0x7f34e80c86f0 2023-08-09T13:34:30.074 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:30.071+0000 7f34ef90c640 1 -- 172.21.15.43:0/158142038 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f34e80739f0 con 0x7f34e80c86f0 2023-08-09T13:34:30.075 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:30.073+0000 7f34ddffb640 1 -- 172.21.15.43:0/158142038 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v24) v1 ==== 74+0+20050 (secure 0 0 0) 0x7f34d8056860 con 0x7f34e80c86f0 2023-08-09T13:34:30.076 INFO:teuthology.orchestra.run.smithi043.stdout: 2023-08-09T13:34:30.076 INFO:teuthology.orchestra.run.smithi043.stdout:{"epoch":24,"fsid":"daac87ee-1337-4250-a955-aea4e408a39b","created":"2023-08-09T13:33:41.701198+0000","modified":"2023-08-09T13:34:07.709429+0000","last_up_change":"2023-08-09T13:33:53.783453+0000","last_in_change":"2023-08-09T13:33:45.148904+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":5,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":"rbd","create_time":"2023-08-09T13:33:54.365137+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":".mgr","create_time":"2023-08-09T13:33:54.831803+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_metadata","create_time":"2023-08-09T13:34:00.207974+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data","create_time":"2023-08-09T13:34:01.614765+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":5,"pool_name":"cephfs_data_ec","create_time":"2023-08-09T13:34:05.239583+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"24","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"cbd49ecd-e896-4fbd-864e-5c37a829b165","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6800","nonce":502667335},{"type":"v1","addr":"172.21.15.43:6801","nonce":502667335}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6802","nonce":502667335},{"type":"v1","addr":"172.21.15.43:6803","nonce":502667335}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6806","nonce":502667335},{"type":"v1","addr":"172.21.15.43:6807","nonce":502667335}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6804","nonce":502667335},{"type":"v1","addr":"172.21.15.43:6805","nonce":502667335}]},"public_addr":"172.21.15.43:6801/502667335","cluster_addr":"172.21.15.43:6803/502667335","heartbeat_back_addr":"172.21.15.43:6807/502667335","heartbeat_front_addr":"172.21.15.43:6805/502667335","state":["exists","up"]},{"osd":1,"uuid":"75c7e0fe-1ec7-4fe8-a2f2-3661a65b8500","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6808","nonce":3289134610},{"type":"v1","addr":"172.21.15.43:6809","nonce":3289134610}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6810","nonce":3289134610},{"type":"v1","addr":"172.21.15.43:6812","nonce":3289134610}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6821","nonce":3289134610},{"type":"v1","addr":"172.21.15.43:6824","nonce":3289134610}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6815","nonce":3289134610},{"type":"v1","addr":"172.21.15.43:6818","nonce":3289134610}]},"public_addr":"172.21.15.43:6809/3289134610","cluster_addr":"172.21.15.43:6812/3289134610","heartbeat_back_addr":"172.21.15.43:6824/3289134610","heartbeat_front_addr":"172.21.15.43:6818/3289134610","state":["exists","up"]},{"osd":2,"uuid":"bb5c41e9-7f5f-4125-9b01-3bb8bfc2ea83","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6811","nonce":4079709736},{"type":"v1","addr":"172.21.15.43:6814","nonce":4079709736}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6817","nonce":4079709736},{"type":"v1","addr":"172.21.15.43:6820","nonce":4079709736}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6829","nonce":4079709736},{"type":"v1","addr":"172.21.15.43:6831","nonce":4079709736}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6823","nonce":4079709736},{"type":"v1","addr":"172.21.15.43:6826","nonce":4079709736}]},"public_addr":"172.21.15.43:6814/4079709736","cluster_addr":"172.21.15.43:6820/4079709736","heartbeat_back_addr":"172.21.15.43:6831/4079709736","heartbeat_front_addr":"172.21.15.43:6826/4079709736","state":["exists","up"]},{"osd":3,"uuid":"bfe4c8b4-694a-465c-87e6-2b2aab852613","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6813","nonce":1709538759},{"type":"v1","addr":"172.21.15.43:6816","nonce":1709538759}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6819","nonce":1709538759},{"type":"v1","addr":"172.21.15.43:6822","nonce":1709538759}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6828","nonce":1709538759},{"type":"v1","addr":"172.21.15.43:6830","nonce":1709538759}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6825","nonce":1709538759},{"type":"v1","addr":"172.21.15.43:6827","nonce":1709538759}]},"public_addr":"172.21.15.43:6816/1709538759","cluster_addr":"172.21.15.43:6822/1709538759","heartbeat_back_addr":"172.21.15.43:6830/1709538759","heartbeat_front_addr":"172.21.15.43:6827/1709538759","state":["exists","up"]},{"osd":4,"uuid":"fa2e8a77-aba1-4756-853e-0d174a5bdeba","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6800","nonce":4196866481},{"type":"v1","addr":"172.21.15.203:6801","nonce":4196866481}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6802","nonce":4196866481},{"type":"v1","addr":"172.21.15.203:6803","nonce":4196866481}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6806","nonce":4196866481},{"type":"v1","addr":"172.21.15.203:6807","nonce":4196866481}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6804","nonce":4196866481},{"type":"v1","addr":"172.21.15.203:6805","nonce":4196866481}]},"public_addr":"172.21.15.203:6801/4196866481","cluster_addr":"172.21.15.203:6803/4196866481","heartbeat_back_addr":"172.21.15.203:6807/4196866481","heartbeat_front_addr":"172.21.15.203:6805/4196866481","state":["exists","up"]},{"osd":5,"uuid":"81811e4b-4d24-4f51-a82c-f1d700a2a00d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6816","nonce":3059290081},{"type":"v1","addr":"172.21.15.203:6817","nonce":3059290081}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6818","nonce":3059290081},{"type":"v1","addr":"172.21.15.203:6819","nonce":3059290081}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6822","nonce":3059290081},{"type":"v1","addr":"172.21.15.203:6823","nonce":3059290081}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6820","nonce":3059290081},{"type":"v1","addr":"172.21.15.203:6821","nonce":3059290081}]},"public_addr":"172.21.15.203:6817/3059290081","cluster_addr":"172.21.15.203:6819/3059290081","heartbeat_back_addr":"172.21.15.203:6823/3059290081","heartbeat_front_addr":"172.21.15.203:6821/3059290081","state":["exists","up"]},{"osd":6,"uuid":"307bac4d-6a5b-443d-af75-56d695d7674f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6824","nonce":512155569},{"type":"v1","addr":"172.21.15.203:6825","nonce":512155569}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6826","nonce":512155569},{"type":"v1","addr":"172.21.15.203:6827","nonce":512155569}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6830","nonce":512155569},{"type":"v1","addr":"172.21.15.203:6831","nonce":512155569}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6828","nonce":512155569},{"type":"v1","addr":"172.21.15.203:6829","nonce":512155569}]},"public_addr":"172.21.15.203:6825/512155569","cluster_addr":"172.21.15.203:6827/512155569","heartbeat_back_addr":"172.21.15.203:6831/512155569","heartbeat_front_addr":"172.21.15.203:6829/512155569","state":["exists","up"]},{"osd":7,"uuid":"a95dc7ca-9f1b-4295-af70-68a22a2d7d5e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6808","nonce":3331450438},{"type":"v1","addr":"172.21.15.203:6809","nonce":3331450438}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6810","nonce":3331450438},{"type":"v1","addr":"172.21.15.203:6811","nonce":3331450438}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6814","nonce":3331450438},{"type":"v1","addr":"172.21.15.203:6815","nonce":3331450438}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6812","nonce":3331450438},{"type":"v1","addr":"172.21.15.203:6813","nonce":3331450438}]},"public_addr":"172.21.15.203:6809/3331450438","cluster_addr":"172.21.15.203:6811/3331450438","heartbeat_back_addr":"172.21.15.203:6815/3331450438","heartbeat_front_addr":"172.21.15.203:6813/3331450438","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-09T13:33:50.547652+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-09T13:33:50.589469+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-09T13:33:50.561724+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-09T13:33:50.599650+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-09T13:33:50.411855+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-09T13:33:50.219681+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-09T13:33:50.504664+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-09T13:33:50.441248+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-08-09T13:34:30.078 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:30.075+0000 7f34ef90c640 1 -- 172.21.15.43:0/158142038 >> [v2:172.21.15.203:6832/830097474,v1:172.21.15.203:6833/830097474] conn(0x7f34cc06a050 msgr2=0x7f34cc06c510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:34:30.078 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:30.075+0000 7f34ef90c640 1 --2- 172.21.15.43:0/158142038 >> [v2:172.21.15.203:6832/830097474,v1:172.21.15.203:6833/830097474] conn(0x7f34cc06a050 0x7f34cc06c510 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f34e4036000 tx=0x7f34e403c040 comp rx=0 tx=0).stop 2023-08-09T13:34:30.078 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:30.075+0000 7f34ef90c640 1 -- 172.21.15.43:0/158142038 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f34e80c86f0 msgr2=0x7f34e80c5b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:34:30.078 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:30.075+0000 7f34ef90c640 1 --2- 172.21.15.43:0/158142038 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f34e80c86f0 0x7f34e80c5b80 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7f34d800c430 tx=0x7f34d800c8f0 comp rx=0 tx=0).stop 2023-08-09T13:34:30.078 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:30.075+0000 7f34ef90c640 1 -- 172.21.15.43:0/158142038 shutdown_connections 2023-08-09T13:34:30.078 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:30.075+0000 7f34ef90c640 1 --2- 172.21.15.43:0/158142038 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f34e8143c20 0x7f34e80c7630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:34:30.079 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:30.075+0000 7f34ef90c640 1 --2- 172.21.15.43:0/158142038 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f34e80c86f0 0x7f34e80c5b80 unknown :-1 s=CLOSED pgs=132 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:34:30.079 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:30.075+0000 7f34ef90c640 1 --2- 172.21.15.43:0/158142038 >> [v2:172.21.15.203:6832/830097474,v1:172.21.15.203:6833/830097474] conn(0x7f34cc06a050 0x7f34cc06c510 unknown :-1 s=CLOSED pgs=88 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:34:30.079 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:30.075+0000 7f34ef90c640 1 --2- 172.21.15.43:0/158142038 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f34e806cfe0 0x7f34e81598a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:34:30.079 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:30.075+0000 7f34ef90c640 1 -- 172.21.15.43:0/158142038 >> 172.21.15.43:0/158142038 conn(0x7f34e80ce0a0 msgr2=0x7f34e80adc20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-09T13:34:30.080 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:30.076+0000 7f34ef90c640 1 -- 172.21.15.43:0/158142038 shutdown_connections 2023-08-09T13:34:30.080 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:30.076+0000 7f34ef90c640 1 -- 172.21.15.43:0/158142038 wait complete. 2023-08-09T13:34:30.100 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs standby_count_wanted 0 2023-08-09T13:34:30.310 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:30.308+0000 7fe372500640 1 Processor -- start 2023-08-09T13:34:30.310 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:30.308+0000 7fe372500640 1 -- start start 2023-08-09T13:34:30.312 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:30.309+0000 7fe372500640 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7fe36c15c680 0x7fe36c0ce760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:34:30.312 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:30.310+0000 7fe372500640 1 -- --> v1:172.21.15.43:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe36c0cdf00 con 0x7fe36c0b3fa0 2023-08-09T13:34:30.313 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:30.310+0000 7fe372500640 1 -- --> v1:172.21.15.203:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe36c0d0f70 con 0x7fe36c0d0a70 2023-08-09T13:34:30.313 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:30.310+0000 7fe372500640 1 -- --> v2:172.21.15.203:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe36c0ccd10 con 0x7fe36c15c680 2023-08-09T13:34:30.313 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:30.310+0000 7fe370a9d640 1 --1- >> v1:172.21.15.43:6789/0 conn(0x7fe36c0b3fa0 0x7fe36c0b43b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.43:6789/0 says I am v1:172.21.15.43:59522/0 (socket says 172.21.15.43:59522) 2023-08-09T13:34:30.313 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:30.310+0000 7fe370a9d640 1 -- 172.21.15.43:0/1596902021 learned_addr learned my addr 172.21.15.43:0/1596902021 (peer_addr_for_me v1:172.21.15.43:0/0) 2023-08-09T13:34:30.314 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:30.311+0000 7fe36bfff640 1 --2- 172.21.15.43:0/1596902021 >> v2:172.21.15.203:3300/0 conn(0x7fe36c15c680 0x7fe36c0ce760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:34:30.315 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:30.311+0000 7fe36affd640 1 -- 172.21.15.43:0/1596902021 <== mon.0 v1:172.21.15.43:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3017351562 0 0) 0x7fe36c0cdf00 con 0x7fe36c0b3fa0 2023-08-09T13:34:30.315 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:30.311+0000 7fe36affd640 1 -- 172.21.15.43:0/1596902021 --> v1:172.21.15.43:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fe358003520 con 0x7fe36c0b3fa0 2023-08-09T13:34:30.315 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:30.311+0000 7fe36affd640 1 -- 172.21.15.43:0/1596902021 <== mon.0 v1:172.21.15.43:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3748452627 0 0) 0x7fe358003520 con 0x7fe36c0b3fa0 2023-08-09T13:34:30.315 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:30.311+0000 7fe36affd640 1 -- 172.21.15.43:0/1596902021 >> v2:172.21.15.203:3300/0 conn(0x7fe36c15c680 msgr2=0x7fe36c0ce760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-09T13:34:30.315 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:30.311+0000 7fe36affd640 1 --2- 172.21.15.43:0/1596902021 >> v2:172.21.15.203:3300/0 conn(0x7fe36c15c680 0x7fe36c0ce760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:34:30.315 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:30.311+0000 7fe36affd640 1 -- 172.21.15.43:0/1596902021 >> v1:172.21.15.203:6789/0 conn(0x7fe36c0d0a70 legacy=0x7fe36c0cd7f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:34:30.315 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:30.311+0000 7fe36affd640 1 -- 172.21.15.43:0/1596902021 --> v1:172.21.15.43:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe36c0ccff0 con 0x7fe36c0b3fa0 2023-08-09T13:34:30.316 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:30.311+0000 7fe36affd640 1 -- 172.21.15.43:0/1596902021 <== mon.0 v1:172.21.15.43:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1285150740 0 0) 0x7fe364001670 con 0x7fe36c0b3fa0 2023-08-09T13:34:30.316 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:30.311+0000 7fe36affd640 1 -- 172.21.15.43:0/1596902021 >> v1:172.21.15.43:6789/0 conn(0x7fe36c0b3fa0 legacy=0x7fe36c0b43b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:34:30.316 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:30.311+0000 7fe36affd640 1 --2- 172.21.15.43:0/1596902021 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fe3580039e0 0x7fe358003e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:34:30.316 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:30.311+0000 7fe36affd640 1 --2- 172.21.15.43:0/1596902021 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fe3580047c0 0x7fe358004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:34:30.316 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:30.311+0000 7fe36affd640 1 --2- 172.21.15.43:0/1596902021 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fe358005180 0x7fe35800b980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:34:30.316 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:30.311+0000 7fe36affd640 1 -- 172.21.15.43:0/1596902021 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe36c0ccd10 con 0x7fe3580047c0 2023-08-09T13:34:30.316 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:30.311+0000 7fe36affd640 1 -- 172.21.15.43:0/1596902021 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe35800c0b0 con 0x7fe358005180 2023-08-09T13:34:30.316 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:30.311+0000 7fe36affd640 1 -- 172.21.15.43:0/1596902021 --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe35800c270 con 0x7fe3580039e0 2023-08-09T13:34:30.317 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:30.312+0000 7fe36b7fe640 1 --2- 172.21.15.43:0/1596902021 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fe3580047c0 0x7fe358004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:34:30.317 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:30.312+0000 7fe36bfff640 1 --2- 172.21.15.43:0/1596902021 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fe3580039e0 0x7fe358003e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:34:30.317 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:30.312+0000 7fe36b7fe640 1 -- 172.21.15.43:0/1596902021 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fe3580039e0 msgr2=0x7fe358003e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-09T13:34:30.317 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:30.312+0000 7fe36b7fe640 1 --2- 172.21.15.43:0/1596902021 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fe3580039e0 0x7fe358003e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:34:30.317 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:30.312+0000 7fe36b7fe640 1 -- 172.21.15.43:0/1596902021 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fe358005180 msgr2=0x7fe35800b980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-09T13:34:30.317 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:30.312+0000 7fe36b7fe640 1 --2- 172.21.15.43:0/1596902021 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fe358005180 0x7fe35800b980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:34:30.317 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:30.312+0000 7fe36b7fe640 1 -- 172.21.15.43:0/1596902021 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe358003520 con 0x7fe3580047c0 2023-08-09T13:34:30.318 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:30.312+0000 7fe370a9d640 1 -- 172.21.15.43:0/1596902021 reap_dead start 2023-08-09T13:34:30.318 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:30.313+0000 7fe36b7fe640 1 --2- 172.21.15.43:0/1596902021 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fe3580047c0 0x7fe358004c40 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7fe354001720 tx=0x7fe3580057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=21c278d7460d9913 server_cookie=0 in_seq=0 out_seq=0 2023-08-09T13:34:30.319 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:30.313+0000 7fe36affd640 1 -- 172.21.15.43:0/1596902021 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe3540052d0 con 0x7fe3580047c0 2023-08-09T13:34:30.319 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:30.313+0000 7fe36affd640 1 -- 172.21.15.43:0/1596902021 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe35402e050 con 0x7fe3580047c0 2023-08-09T13:34:30.319 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:30.313+0000 7fe36affd640 1 -- 172.21.15.43:0/1596902021 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe354003830 con 0x7fe3580047c0 2023-08-09T13:34:30.319 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:30.314+0000 7fe372500640 1 -- 172.21.15.43:0/1596902021 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fe3580047c0 msgr2=0x7fe358004c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:34:30.319 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:30.314+0000 7fe372500640 1 --2- 172.21.15.43:0/1596902021 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fe3580047c0 0x7fe358004c40 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7fe354001720 tx=0x7fe3580057f0 comp rx=0 tx=0).stop 2023-08-09T13:34:30.319 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:30.314+0000 7fe372500640 1 -- 172.21.15.43:0/1596902021 shutdown_connections 2023-08-09T13:34:30.319 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:30.314+0000 7fe372500640 1 --2- 172.21.15.43:0/1596902021 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fe3580047c0 0x7fe358004c40 unknown :-1 s=CLOSED pgs=134 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:34:30.320 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:30.314+0000 7fe372500640 1 -- 172.21.15.43:0/1596902021 >> 172.21.15.43:0/1596902021 conn(0x7fe36c071520 msgr2=0x7fe36c0d2520 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-09T13:34:30.320 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:30.314+0000 7fe372500640 1 -- 172.21.15.43:0/1596902021 shutdown_connections 2023-08-09T13:34:30.320 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:30.314+0000 7fe372500640 1 -- 172.21.15.43:0/1596902021 wait complete. 2023-08-09T13:34:30.320 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:30.316+0000 7fe372500640 1 Processor -- start 2023-08-09T13:34:30.322 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:30.316+0000 7fe372500640 1 -- start start 2023-08-09T13:34:30.322 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:30.317+0000 7fe372500640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fe3580047c0 0x7fe36c0cc0b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:34:30.322 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:30.317+0000 7fe372500640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fe36c14abd0 0x7fe36c0ca000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:34:30.322 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:30.317+0000 7fe372500640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fe36c15e910 0x7fe36c15ed90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:34:30.322 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:30.317+0000 7fe372500640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe36c15f7f0 con 0x7fe36c14abd0 2023-08-09T13:34:30.322 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:30.317+0000 7fe372500640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe36c15f2d0 con 0x7fe36c15e910 2023-08-09T13:34:30.322 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:30.317+0000 7fe372500640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe36c0ca540 con 0x7fe3580047c0 2023-08-09T13:34:30.323 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:30.318+0000 7fe36b7fe640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fe36c14abd0 0x7fe36c0ca000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:34:30.323 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:30.318+0000 7fe36b7fe640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fe36c14abd0 0x7fe36c0ca000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:37574/0 (socket says 172.21.15.43:37574) 2023-08-09T13:34:30.323 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:30.318+0000 7fe36b7fe640 1 -- 172.21.15.43:0/3393870437 learned_addr learned my addr 172.21.15.43:0/3393870437 (peer_addr_for_me v2:172.21.15.43:0/0) 2023-08-09T13:34:30.323 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:30.318+0000 7fe36bfff640 1 --2- 172.21.15.43:0/3393870437 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fe3580047c0 0x7fe36c0cc0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:34:30.323 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:30.318+0000 7fe370a9d640 1 --2- 172.21.15.43:0/3393870437 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fe36c15e910 0x7fe36c15ed90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:34:30.323 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:30.318+0000 7fe36b7fe640 1 -- 172.21.15.43:0/3393870437 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fe3580047c0 msgr2=0x7fe36c0cc0b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:34:30.323 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:30.318+0000 7fe36b7fe640 1 --2- 172.21.15.43:0/3393870437 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fe3580047c0 0x7fe36c0cc0b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:34:30.324 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:30.318+0000 7fe36b7fe640 1 -- 172.21.15.43:0/3393870437 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fe36c15e910 msgr2=0x7fe36c15ed90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:34:30.324 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:30.318+0000 7fe36b7fe640 1 --2- 172.21.15.43:0/3393870437 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fe36c15e910 0x7fe36c15ed90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:34:30.324 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:30.318+0000 7fe36b7fe640 1 -- 172.21.15.43:0/3393870437 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe36c0ca7c0 con 0x7fe36c14abd0 2023-08-09T13:34:30.324 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:30.318+0000 7fe370a9d640 1 --2- 172.21.15.43:0/3393870437 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fe36c15e910 0x7fe36c15ed90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-08-09T13:34:30.324 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:30.319+0000 7fe36b7fe640 1 --2- 172.21.15.43:0/3393870437 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fe36c14abd0 0x7fe36c0ca000 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7fe354005040 tx=0x7fe354001120 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-09T13:34:30.324 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:30.319+0000 7fe368ff9640 1 -- 172.21.15.43:0/3393870437 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe3540076c0 con 0x7fe36c14abd0 2023-08-09T13:34:30.324 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:30.319+0000 7fe368ff9640 1 -- 172.21.15.43:0/3393870437 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe354007a30 con 0x7fe36c14abd0 2023-08-09T13:34:30.324 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:30.319+0000 7fe372500640 1 -- 172.21.15.43:0/3393870437 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe36c0caaa0 con 0x7fe36c14abd0 2023-08-09T13:34:30.325 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:30.319+0000 7fe368ff9640 1 -- 172.21.15.43:0/3393870437 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe354006be0 con 0x7fe36c14abd0 2023-08-09T13:34:30.325 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:30.319+0000 7fe372500640 1 -- 172.21.15.43:0/3393870437 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe36c0cb060 con 0x7fe36c14abd0 2023-08-09T13:34:30.325 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:30.320+0000 7fe372500640 1 -- 172.21.15.43:0/3393870437 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe36c0cb280 con 0x7fe36c14abd0 2023-08-09T13:34:30.326 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:30.321+0000 7fe368ff9640 1 -- 172.21.15.43:0/3393870437 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 5) v1 ==== 88341+0+0 (secure 0 0 0) 0x7fe354006d40 con 0x7fe36c14abd0 2023-08-09T13:34:30.326 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:30.322+0000 7fe368ff9640 1 --2- 172.21.15.43:0/3393870437 >> [v2:172.21.15.203:6832/830097474,v1:172.21.15.203:6833/830097474] conn(0x7fe34c06a080 0x7fe34c06c540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:34:30.326 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:30.322+0000 7fe368ff9640 1 -- 172.21.15.43:0/3393870437 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6813+0+0 (secure 0 0 0) 0x7fe3540aa9c0 con 0x7fe36c14abd0 2023-08-09T13:34:30.326 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:30.323+0000 7fe36bfff640 1 --2- 172.21.15.43:0/3393870437 >> [v2:172.21.15.203:6832/830097474,v1:172.21.15.203:6833/830097474] conn(0x7fe34c06a080 0x7fe34c06c540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:34:30.326 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:30.324+0000 7fe36bfff640 1 --2- 172.21.15.43:0/3393870437 >> [v2:172.21.15.203:6832/830097474,v1:172.21.15.203:6833/830097474] conn(0x7fe34c06a080 0x7fe34c06c540 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7fe36c063390 tx=0x7fe35c00b040 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-09T13:34:30.329 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:30.326+0000 7fe368ff9640 1 -- 172.21.15.43:0/3393870437 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178066 (secure 0 0 0) 0x7fe3540798d0 con 0x7fe36c14abd0 2023-08-09T13:34:30.562 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:30.559+0000 7fe372500640 1 -- 172.21.15.43:0/3393870437 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} v 0) v1 -- 0x7fe36c0739f0 con 0x7fe36c14abd0 2023-08-09T13:34:30.847 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:30.845+0000 7fe368ff9640 1 -- 172.21.15.43:0/3393870437 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]=0 v8) v1 ==== 118+0+0 (secure 0 0 0) 0x7fe354079460 con 0x7fe36c14abd0 2023-08-09T13:34:30.851 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:30.848+0000 7fe372500640 1 -- 172.21.15.43:0/3393870437 >> [v2:172.21.15.203:6832/830097474,v1:172.21.15.203:6833/830097474] conn(0x7fe34c06a080 msgr2=0x7fe34c06c540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:34:30.851 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:30.848+0000 7fe372500640 1 --2- 172.21.15.43:0/3393870437 >> [v2:172.21.15.203:6832/830097474,v1:172.21.15.203:6833/830097474] conn(0x7fe34c06a080 0x7fe34c06c540 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7fe36c063390 tx=0x7fe35c00b040 comp rx=0 tx=0).stop 2023-08-09T13:34:30.851 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:30.848+0000 7fe372500640 1 -- 172.21.15.43:0/3393870437 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fe36c14abd0 msgr2=0x7fe36c0ca000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:34:30.852 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:30.848+0000 7fe372500640 1 --2- 172.21.15.43:0/3393870437 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fe36c14abd0 0x7fe36c0ca000 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7fe354005040 tx=0x7fe354001120 comp rx=0 tx=0).stop 2023-08-09T13:34:30.852 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:30.848+0000 7fe372500640 1 -- 172.21.15.43:0/3393870437 shutdown_connections 2023-08-09T13:34:30.852 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:30.848+0000 7fe372500640 1 --2- 172.21.15.43:0/3393870437 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fe36c15e910 0x7fe36c15ed90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:34:30.852 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:30.848+0000 7fe372500640 1 --2- 172.21.15.43:0/3393870437 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fe36c14abd0 0x7fe36c0ca000 unknown :-1 s=CLOSED pgs=135 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:34:30.852 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:30.848+0000 7fe372500640 1 --2- 172.21.15.43:0/3393870437 >> [v2:172.21.15.203:6832/830097474,v1:172.21.15.203:6833/830097474] conn(0x7fe34c06a080 0x7fe34c06c540 unknown :-1 s=CLOSED pgs=89 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:34:30.852 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:30.848+0000 7fe372500640 1 --2- 172.21.15.43:0/3393870437 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fe3580047c0 0x7fe36c0cc0b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:34:30.852 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:30.848+0000 7fe372500640 1 -- 172.21.15.43:0/3393870437 >> 172.21.15.43:0/3393870437 conn(0x7fe36c071520 msgr2=0x7fe36c0b6b70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-09T13:34:30.852 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:30.849+0000 7fe372500640 1 -- 172.21.15.43:0/3393870437 shutdown_connections 2023-08-09T13:34:30.853 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:30.849+0000 7fe372500640 1 -- 172.21.15.43:0/3393870437 wait complete. 2023-08-09T13:34:30.871 DEBUG:tasks.cephfs.filesystem:fs_config: {'ec_profile': ['m=2', 'k=2', 'crush-failure-domain=osd']} 2023-08-09T13:34:30.871 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs allow_standby_replay false 2023-08-09T13:34:31.111 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:31.108+0000 7f1e95786640 1 Processor -- start 2023-08-09T13:34:31.111 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:31.108+0000 7f1e95786640 1 -- start start 2023-08-09T13:34:31.112 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:31.110+0000 7f1e95786640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f1e90160210 0x7f1e90160670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:34:31.113 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:31.110+0000 7f1e95786640 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7f1e900ceaf0 0x7f1e900ceef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:34:31.113 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:31.110+0000 7f1e95786640 1 -- --> v1:172.21.15.43:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1e9015fff0 con 0x7f1e9015fc10 2023-08-09T13:34:31.113 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:31.110+0000 7f1e95786640 1 -- --> v2:172.21.15.43:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1e900b5c80 con 0x7f1e90160210 2023-08-09T13:34:31.113 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:31.110+0000 7f1e95786640 1 -- --> v2:172.21.15.203:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1e900cce70 con 0x7f1e900ceaf0 2023-08-09T13:34:31.113 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:31.110+0000 7f1e8ffff640 1 --1- >> v1:172.21.15.43:6789/0 conn(0x7f1e9015fc10 0x7f1e900b5570 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.43:6789/0 says I am v1:172.21.15.43:59524/0 (socket says 172.21.15.43:59524) 2023-08-09T13:34:31.113 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:31.110+0000 7f1e8ffff640 1 -- 172.21.15.43:0/777108860 learned_addr learned my addr 172.21.15.43:0/777108860 (peer_addr_for_me v1:172.21.15.43:0/0) 2023-08-09T13:34:31.114 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:31.110+0000 7f1e877fe640 1 --2- 172.21.15.43:0/777108860 >> v2:172.21.15.43:3300/0 conn(0x7f1e90160210 0x7f1e90160670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:34:31.114 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:31.111+0000 7f1e94f85640 1 --2- 172.21.15.43:0/777108860 >> v2:172.21.15.203:3300/0 conn(0x7f1e900ceaf0 0x7f1e900ceef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:34:31.114 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:31.111+0000 7f1e8f7fe640 1 -- 172.21.15.43:0/777108860 <== mon.0 v1:172.21.15.43:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2815872938 0 0) 0x7f1e9015fff0 con 0x7f1e9015fc10 2023-08-09T13:34:31.115 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:31.111+0000 7f1e8f7fe640 1 -- 172.21.15.43:0/777108860 --> v1:172.21.15.43:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f1e7c003520 con 0x7f1e9015fc10 2023-08-09T13:34:31.115 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:31.111+0000 7f1e877fe640 1 -- 172.21.15.43:0/777108860 >> v2:172.21.15.203:3300/0 conn(0x7f1e900ceaf0 msgr2=0x7f1e900ceef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-09T13:34:31.115 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:31.111+0000 7f1e877fe640 1 --2- 172.21.15.43:0/777108860 >> v2:172.21.15.203:3300/0 conn(0x7f1e900ceaf0 0x7f1e900ceef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:34:31.115 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:31.111+0000 7f1e877fe640 1 -- 172.21.15.43:0/777108860 >> v1:172.21.15.43:6789/0 conn(0x7f1e9015fc10 legacy=0x7f1e900b5570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:34:31.115 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:31.111+0000 7f1e877fe640 1 -- 172.21.15.43:0/777108860 --> v2:172.21.15.43:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1e90160d80 con 0x7f1e90160210 2023-08-09T13:34:31.116 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:31.112+0000 7f1e877fe640 1 --2- 172.21.15.43:0/777108860 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f1e90160210 0x7f1e90160670 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7f1e900cff60 tx=0x7f1e7802b840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5cb4c5704d4b09b6 server_cookie=0 in_seq=0 out_seq=0 2023-08-09T13:34:31.116 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:31.112+0000 7f1e8f7fe640 1 -- 172.21.15.43:0/777108860 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1e7802e070 con 0x7f1e90160210 2023-08-09T13:34:31.116 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:31.112+0000 7f1e8f7fe640 1 -- 172.21.15.43:0/777108860 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1e780315a0 con 0x7f1e90160210 2023-08-09T13:34:31.116 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:31.112+0000 7f1e8f7fe640 1 -- 172.21.15.43:0/777108860 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1e78031930 con 0x7f1e90160210 2023-08-09T13:34:31.117 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:31.113+0000 7f1e95786640 1 -- 172.21.15.43:0/777108860 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f1e90160210 msgr2=0x7f1e90160670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:34:31.117 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:31.113+0000 7f1e95786640 1 --2- 172.21.15.43:0/777108860 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f1e90160210 0x7f1e90160670 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7f1e900cff60 tx=0x7f1e7802b840 comp rx=0 tx=0).stop 2023-08-09T13:34:31.117 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:31.113+0000 7f1e95786640 1 -- 172.21.15.43:0/777108860 shutdown_connections 2023-08-09T13:34:31.117 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:31.113+0000 7f1e95786640 1 --2- 172.21.15.43:0/777108860 >> v2:172.21.15.203:3300/0 conn(0x7f1e900ceaf0 0x7f1e900ceef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:34:31.117 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:31.113+0000 7f1e95786640 1 --2- 172.21.15.43:0/777108860 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f1e90160210 0x7f1e90160670 unknown :-1 s=CLOSED pgs=137 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:34:31.117 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:31.113+0000 7f1e95786640 1 -- 172.21.15.43:0/777108860 >> 172.21.15.43:0/777108860 conn(0x7f1e9006a2e0 msgr2=0x7f1e9006a6c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-09T13:34:31.117 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:31.113+0000 7f1e95786640 1 -- 172.21.15.43:0/777108860 shutdown_connections 2023-08-09T13:34:31.118 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:31.113+0000 7f1e95786640 1 -- 172.21.15.43:0/777108860 wait complete. 2023-08-09T13:34:31.118 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:31.115+0000 7f1e95786640 1 Processor -- start 2023-08-09T13:34:31.118 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:31.115+0000 7f1e95786640 1 -- start start 2023-08-09T13:34:31.119 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:31.116+0000 7f1e95786640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f1e900ceaf0 0x7f1e900b6c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:34:31.119 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:31.116+0000 7f1e95786640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f1e9015fc10 0x7f1e9014fb50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:34:31.119 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:31.116+0000 7f1e95786640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f1e90160210 0x7f1e90150090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:34:31.119 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:31.116+0000 7f1e95786640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1e901f1d90 con 0x7f1e9015fc10 2023-08-09T13:34:31.119 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:31.116+0000 7f1e95786640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1e901f2070 con 0x7f1e90160210 2023-08-09T13:34:31.120 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:31.116+0000 7f1e95786640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1e901f2350 con 0x7f1e900ceaf0 2023-08-09T13:34:31.121 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:31.117+0000 7f1e877fe640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f1e9015fc10 0x7f1e9014fb50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:34:31.121 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:31.117+0000 7f1e877fe640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f1e9015fc10 0x7f1e9014fb50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:37596/0 (socket says 172.21.15.43:37596) 2023-08-09T13:34:31.121 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:31.117+0000 7f1e877fe640 1 -- 172.21.15.43:0/2211273213 learned_addr learned my addr 172.21.15.43:0/2211273213 (peer_addr_for_me v2:172.21.15.43:0/0) 2023-08-09T13:34:31.121 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:31.117+0000 7f1e8ffff640 1 --2- 172.21.15.43:0/2211273213 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f1e900ceaf0 0x7f1e900b6c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:34:31.121 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:31.117+0000 7f1e94f85640 1 --2- 172.21.15.43:0/2211273213 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f1e90160210 0x7f1e90150090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:34:31.121 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:31.117+0000 7f1e877fe640 1 -- 172.21.15.43:0/2211273213 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f1e900ceaf0 msgr2=0x7f1e900b6c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:34:31.122 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:31.117+0000 7f1e877fe640 1 --2- 172.21.15.43:0/2211273213 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f1e900ceaf0 0x7f1e900b6c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:34:31.122 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:31.117+0000 7f1e877fe640 1 -- 172.21.15.43:0/2211273213 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f1e90160210 msgr2=0x7f1e90150090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:34:31.122 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:31.117+0000 7f1e877fe640 1 --2- 172.21.15.43:0/2211273213 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f1e90160210 0x7f1e90150090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:34:31.122 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:31.117+0000 7f1e877fe640 1 -- 172.21.15.43:0/2211273213 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1e78009400 con 0x7f1e9015fc10 2023-08-09T13:34:31.122 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:31.118+0000 7f1e877fe640 1 --2- 172.21.15.43:0/2211273213 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f1e9015fc10 0x7f1e9014fb50 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7f1e78009890 tx=0x7f1e780046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-09T13:34:31.122 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:31.118+0000 7f1e8d7fa640 1 -- 172.21.15.43:0/2211273213 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1e78032840 con 0x7f1e9015fc10 2023-08-09T13:34:31.122 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:31.118+0000 7f1e8d7fa640 1 -- 172.21.15.43:0/2211273213 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1e78031f00 con 0x7f1e9015fc10 2023-08-09T13:34:31.122 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:31.118+0000 7f1e8d7fa640 1 -- 172.21.15.43:0/2211273213 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1e78032be0 con 0x7f1e9015fc10 2023-08-09T13:34:31.123 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:31.118+0000 7f1e95786640 1 -- 172.21.15.43:0/2211273213 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1e901f2630 con 0x7f1e9015fc10 2023-08-09T13:34:31.123 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:31.118+0000 7f1e95786640 1 -- 172.21.15.43:0/2211273213 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1e900b7190 con 0x7f1e9015fc10 2023-08-09T13:34:31.123 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:31.120+0000 7f1e95786640 1 -- 172.21.15.43:0/2211273213 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1e900d02d0 con 0x7f1e9015fc10 2023-08-09T13:34:31.123 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:31.120+0000 7f1e8d7fa640 1 -- 172.21.15.43:0/2211273213 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 5) v1 ==== 88341+0+0 (secure 0 0 0) 0x7f1e78032d40 con 0x7f1e9015fc10 2023-08-09T13:34:31.124 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:31.121+0000 7f1e8d7fa640 1 --2- 172.21.15.43:0/2211273213 >> [v2:172.21.15.203:6832/830097474,v1:172.21.15.203:6833/830097474] conn(0x7f1e7006a080 0x7f1e7006c540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:34:31.124 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:31.122+0000 7f1e8d7fa640 1 -- 172.21.15.43:0/2211273213 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6813+0+0 (secure 0 0 0) 0x7f1e780aa730 con 0x7f1e9015fc10 2023-08-09T13:34:31.124 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:31.122+0000 7f1e8ffff640 1 --2- 172.21.15.43:0/2211273213 >> [v2:172.21.15.203:6832/830097474,v1:172.21.15.203:6833/830097474] conn(0x7f1e7006a080 0x7f1e7006c540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:34:31.125 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:31.123+0000 7f1e8ffff640 1 --2- 172.21.15.43:0/2211273213 >> [v2:172.21.15.203:6832/830097474,v1:172.21.15.203:6833/830097474] conn(0x7f1e7006a080 0x7f1e7006c540 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f1e80001560 tx=0x7f1e80009040 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-09T13:34:31.128 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:31.125+0000 7f1e8d7fa640 1 -- 172.21.15.43:0/2211273213 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178066 (secure 0 0 0) 0x7f1e900d02d0 con 0x7f1e9015fc10 2023-08-09T13:34:31.366 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:31.364+0000 7f1e95786640 1 -- 172.21.15.43:0/2211273213 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"} v 0) v1 -- 0x7f1e900d02d0 con 0x7f1e9015fc10 2023-08-09T13:34:31.857 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:31.855+0000 7f1e8d7fa640 1 -- 172.21.15.43:0/2211273213 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"}]=0 v9) v1 ==== 122+0+0 (secure 0 0 0) 0x7f1e900d02d0 con 0x7f1e9015fc10 2023-08-09T13:34:31.861 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:31.858+0000 7f1e95786640 1 -- 172.21.15.43:0/2211273213 >> [v2:172.21.15.203:6832/830097474,v1:172.21.15.203:6833/830097474] conn(0x7f1e7006a080 msgr2=0x7f1e7006c540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:34:31.861 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:31.858+0000 7f1e95786640 1 --2- 172.21.15.43:0/2211273213 >> [v2:172.21.15.203:6832/830097474,v1:172.21.15.203:6833/830097474] conn(0x7f1e7006a080 0x7f1e7006c540 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f1e80001560 tx=0x7f1e80009040 comp rx=0 tx=0).stop 2023-08-09T13:34:31.861 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:31.858+0000 7f1e95786640 1 -- 172.21.15.43:0/2211273213 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f1e9015fc10 msgr2=0x7f1e9014fb50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:34:31.861 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:31.858+0000 7f1e95786640 1 --2- 172.21.15.43:0/2211273213 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f1e9015fc10 0x7f1e9014fb50 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7f1e78009890 tx=0x7f1e780046c0 comp rx=0 tx=0).stop 2023-08-09T13:34:31.861 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:31.858+0000 7f1e95786640 1 -- 172.21.15.43:0/2211273213 shutdown_connections 2023-08-09T13:34:31.861 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:31.858+0000 7f1e95786640 1 --2- 172.21.15.43:0/2211273213 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f1e90160210 0x7f1e90150090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:34:31.862 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:31.858+0000 7f1e95786640 1 --2- 172.21.15.43:0/2211273213 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f1e9015fc10 0x7f1e9014fb50 unknown :-1 s=CLOSED pgs=138 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:34:31.862 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:31.858+0000 7f1e95786640 1 --2- 172.21.15.43:0/2211273213 >> [v2:172.21.15.203:6832/830097474,v1:172.21.15.203:6833/830097474] conn(0x7f1e7006a080 0x7f1e7006c540 unknown :-1 s=CLOSED pgs=90 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:34:31.862 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:31.858+0000 7f1e95786640 1 --2- 172.21.15.43:0/2211273213 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f1e900ceaf0 0x7f1e900b6c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:34:31.862 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:31.858+0000 7f1e95786640 1 -- 172.21.15.43:0/2211273213 >> 172.21.15.43:0/2211273213 conn(0x7f1e9006a2e0 msgr2=0x7f1e90151e50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-09T13:34:31.863 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:31.858+0000 7f1e95786640 1 -- 172.21.15.43:0/2211273213 shutdown_connections 2023-08-09T13:34:31.863 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:31.859+0000 7f1e95786640 1 -- 172.21.15.43:0/2211273213 wait complete. 2023-08-09T13:34:31.883 DEBUG:tasks.cephfs.filesystem:Not Creating any subvols for filesystem 'cephfs' 2023-08-09T13:34:31.883 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-08-09T13:34:32.108 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:32.106+0000 7fa618efb640 1 Processor -- start 2023-08-09T13:34:32.109 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:32.106+0000 7fa618efb640 1 -- start start 2023-08-09T13:34:32.109 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:32.107+0000 7fa618efb640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7fa6140c8750 0x7fa6140ad340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:34:32.110 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:32.107+0000 7fa618efb640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fa6140c8b30 0x7fa61406f410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:34:32.110 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:32.107+0000 7fa618efb640 1 -- --> v1:172.21.15.43:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa6140cbd90 con 0x7fa6140acdf0 2023-08-09T13:34:32.110 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:32.107+0000 7fa618efb640 1 -- --> v2:172.21.15.43:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa61415e810 con 0x7fa6140c8750 2023-08-09T13:34:32.110 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:32.107+0000 7fa618efb640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa6140c9f70 con 0x7fa6140c8b30 2023-08-09T13:34:32.111 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:32.107+0000 7fa6137fe640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7fa6140c8750 0x7fa6140ad340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:34:32.111 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:32.107+0000 7fa613fff640 1 --1- >> v1:172.21.15.43:6789/0 conn(0x7fa6140acdf0 0x7fa614070f00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.43:6789/0 says I am v1:172.21.15.43:59540/0 (socket says 172.21.15.43:59540) 2023-08-09T13:34:32.111 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:32.108+0000 7fa613fff640 1 -- 172.21.15.43:0/2829012873 learned_addr learned my addr 172.21.15.43:0/2829012873 (peer_addr_for_me v1:172.21.15.43:0/0) 2023-08-09T13:34:32.111 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:32.108+0000 7fa60bfff640 1 --2- 172.21.15.43:0/2829012873 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fa6140c8b30 0x7fa61406f410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:34:32.112 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:32.108+0000 7fa6137fe640 1 -- 172.21.15.43:0/2829012873 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fa6140c8b30 msgr2=0x7fa61406f410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-09T13:34:32.112 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:32.108+0000 7fa6137fe640 1 --2- 172.21.15.43:0/2829012873 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fa6140c8b30 0x7fa61406f410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:34:32.112 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:32.108+0000 7fa6137fe640 1 -- 172.21.15.43:0/2829012873 >> v1:172.21.15.43:6789/0 conn(0x7fa6140acdf0 legacy=0x7fa614070f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:34:32.112 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:32.108+0000 7fa6137fe640 1 -- 172.21.15.43:0/2829012873 --> v2:172.21.15.43:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa6140ca2b0 con 0x7fa6140c8750 2023-08-09T13:34:32.112 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:32.108+0000 7fa612ffd640 1 -- 172.21.15.43:0/2829012873 <== mon.0 v1:172.21.15.43:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2230632740 0 0) 0x7fa6140cbd90 con 0x7fa6140acdf0 2023-08-09T13:34:32.113 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:32.109+0000 7fa6137fe640 1 --2- 172.21.15.43:0/2829012873 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa6140c8750 0x7fa6140ad340 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7fa6040097a0 tx=0x7fa60402b8b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1344deb74fe09cf6 server_cookie=0 in_seq=0 out_seq=0 2023-08-09T13:34:32.113 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:32.109+0000 7fa612ffd640 1 -- 172.21.15.43:0/2829012873 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa60402d040 con 0x7fa6140c8750 2023-08-09T13:34:32.114 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:32.110+0000 7fa612ffd640 1 -- 172.21.15.43:0/2829012873 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa604031810 con 0x7fa6140c8750 2023-08-09T13:34:32.114 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:32.110+0000 7fa612ffd640 1 -- 172.21.15.43:0/2829012873 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa604031ba0 con 0x7fa6140c8750 2023-08-09T13:34:32.114 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:32.110+0000 7fa618efb640 1 -- 172.21.15.43:0/2829012873 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa6140c8750 msgr2=0x7fa6140ad340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:34:32.114 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:32.110+0000 7fa618efb640 1 --2- 172.21.15.43:0/2829012873 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa6140c8750 0x7fa6140ad340 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7fa6040097a0 tx=0x7fa60402b8b0 comp rx=0 tx=0).stop 2023-08-09T13:34:32.114 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:32.110+0000 7fa618efb640 1 -- 172.21.15.43:0/2829012873 shutdown_connections 2023-08-09T13:34:32.114 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:32.110+0000 7fa618efb640 1 --2- 172.21.15.43:0/2829012873 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fa6140c8b30 0x7fa61406f410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:34:32.114 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:32.110+0000 7fa618efb640 1 --2- 172.21.15.43:0/2829012873 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa6140c8750 0x7fa6140ad340 unknown :-1 s=CLOSED pgs=140 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:34:32.115 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:32.110+0000 7fa618efb640 1 -- 172.21.15.43:0/2829012873 >> 172.21.15.43:0/2829012873 conn(0x7fa6140cde90 msgr2=0x7fa6140ce2a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-09T13:34:32.115 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:32.110+0000 7fa618efb640 1 -- 172.21.15.43:0/2829012873 shutdown_connections 2023-08-09T13:34:32.115 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:32.110+0000 7fa618efb640 1 -- 172.21.15.43:0/2829012873 wait complete. 2023-08-09T13:34:32.115 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:32.112+0000 7fa618efb640 1 Processor -- start 2023-08-09T13:34:32.116 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:32.112+0000 7fa618efb640 1 -- start start 2023-08-09T13:34:32.116 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:32.113+0000 7fa618efb640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa6140acdf0 0x7fa614098a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:34:32.116 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:32.113+0000 7fa618efb640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fa6140c8750 0x7fa6140b30a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:34:32.116 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:32.113+0000 7fa618efb640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fa6140c8b30 0x7fa614147d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:34:32.116 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:32.113+0000 7fa618efb640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa61406bba0 con 0x7fa6140acdf0 2023-08-09T13:34:32.116 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:32.113+0000 7fa618efb640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa61406bdf0 con 0x7fa6140c8b30 2023-08-09T13:34:32.117 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:32.113+0000 7fa618efb640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa61406c0d0 con 0x7fa6140c8750 2023-08-09T13:34:32.118 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:32.114+0000 7fa6137fe640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa6140acdf0 0x7fa614098a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:34:32.118 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:32.114+0000 7fa6137fe640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa6140acdf0 0x7fa614098a50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:37626/0 (socket says 172.21.15.43:37626) 2023-08-09T13:34:32.118 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:32.114+0000 7fa6137fe640 1 -- 172.21.15.43:0/3953535436 learned_addr learned my addr 172.21.15.43:0/3953535436 (peer_addr_for_me v2:172.21.15.43:0/0) 2023-08-09T13:34:32.118 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:32.114+0000 7fa60bfff640 1 --2- 172.21.15.43:0/3953535436 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fa6140c8750 0x7fa6140b30a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:34:32.118 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:32.114+0000 7fa613fff640 1 --2- 172.21.15.43:0/3953535436 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fa6140c8b30 0x7fa614147d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:34:32.118 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:32.114+0000 7fa6137fe640 1 -- 172.21.15.43:0/3953535436 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fa6140c8750 msgr2=0x7fa6140b30a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:34:32.119 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:32.114+0000 7fa6137fe640 1 --2- 172.21.15.43:0/3953535436 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fa6140c8750 0x7fa6140b30a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:34:32.119 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:32.114+0000 7fa6137fe640 1 -- 172.21.15.43:0/3953535436 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fa6140c8b30 msgr2=0x7fa614147d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:34:32.119 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:32.114+0000 7fa6137fe640 1 --2- 172.21.15.43:0/3953535436 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fa6140c8b30 0x7fa614147d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:34:32.119 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:32.114+0000 7fa6137fe640 1 -- 172.21.15.43:0/3953535436 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa604009400 con 0x7fa6140acdf0 2023-08-09T13:34:32.119 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:32.115+0000 7fa6137fe640 1 --2- 172.21.15.43:0/3953535436 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa6140acdf0 0x7fa614098a50 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7fa604000c00 tx=0x7fa6040046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-09T13:34:32.119 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:32.115+0000 7fa610ff9640 1 -- 172.21.15.43:0/3953535436 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa60403ba60 con 0x7fa6140acdf0 2023-08-09T13:34:32.119 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:32.115+0000 7fa610ff9640 1 -- 172.21.15.43:0/3953535436 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa60403bbc0 con 0x7fa6140acdf0 2023-08-09T13:34:32.120 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:32.115+0000 7fa610ff9640 1 -- 172.21.15.43:0/3953535436 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa604004030 con 0x7fa6140acdf0 2023-08-09T13:34:32.120 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:32.115+0000 7fa618efb640 1 -- 172.21.15.43:0/3953535436 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa61406c3b0 con 0x7fa6140acdf0 2023-08-09T13:34:32.120 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:32.115+0000 7fa618efb640 1 -- 172.21.15.43:0/3953535436 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa614098f90 con 0x7fa6140acdf0 2023-08-09T13:34:32.120 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:32.116+0000 7fa618efb640 1 -- 172.21.15.43:0/3953535436 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa6140ce7a0 con 0x7fa6140acdf0 2023-08-09T13:34:32.121 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:32.117+0000 7fa610ff9640 1 -- 172.21.15.43:0/3953535436 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 5) v1 ==== 88341+0+0 (secure 0 0 0) 0x7fa604043070 con 0x7fa6140acdf0 2023-08-09T13:34:32.121 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:32.118+0000 7fa610ff9640 1 --2- 172.21.15.43:0/3953535436 >> [v2:172.21.15.203:6832/830097474,v1:172.21.15.203:6833/830097474] conn(0x7fa5f406a050 0x7fa5f406c510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:34:32.121 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:32.118+0000 7fa610ff9640 1 -- 172.21.15.43:0/3953535436 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6813+0+0 (secure 0 0 0) 0x7fa6040aa730 con 0x7fa6140acdf0 2023-08-09T13:34:32.121 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:32.119+0000 7fa60bfff640 1 --2- 172.21.15.43:0/3953535436 >> [v2:172.21.15.203:6832/830097474,v1:172.21.15.203:6833/830097474] conn(0x7fa5f406a050 0x7fa5f406c510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:34:32.122 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:32.120+0000 7fa60bfff640 1 --2- 172.21.15.43:0/3953535436 >> [v2:172.21.15.203:6832/830097474,v1:172.21.15.203:6833/830097474] conn(0x7fa5f406a050 0x7fa5f406c510 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7fa5fc006fd0 tx=0x7fa5fc008040 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-09T13:34:32.125 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:32.123+0000 7fa610ff9640 1 -- 172.21.15.43:0/3953535436 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178066 (secure 0 0 0) 0x7fa604075c40 con 0x7fa6140acdf0 2023-08-09T13:34:32.355 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:32.353+0000 7fa618efb640 1 -- 172.21.15.43:0/3953535436 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fa6140c9230 con 0x7fa6140acdf0 2023-08-09T13:34:32.357 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:32.354+0000 7fa610ff9640 1 -- 172.21.15.43:0/3953535436 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 9 v9) v1 ==== 93+0+4874 (secure 0 0 0) 0x7fa604079260 con 0x7fa6140acdf0 2023-08-09T13:34:32.357 INFO:teuthology.orchestra.run.smithi043.stdout: 2023-08-09T13:34:32.357 INFO:teuthology.orchestra.run.smithi043.stdout:{"epoch":9,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4236,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.43:6833/3917539953","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6832","nonce":3917539953},{"type":"v1","addr":"172.21.15.43:6833","nonce":3917539953}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2},{"gid":4247,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.203:6837/2467794159","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6835","nonce":2467794159},{"type":"v1","addr":"172.21.15.203:6837","nonce":2467794159}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":4248,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.43:6835/1818380189","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6834","nonce":1818380189},{"type":"v1","addr":"172.21.15.43:6835","nonce":1818380189}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":9,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-08-09T13:34:02.575398+0000","modified":"2023-08-09T13:34:31.841945+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":4250},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4250":{"gid":4250,"name":"d","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.203:6836/798312463","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6834","nonce":798312463},{"type":"v1","addr":"172.21.15.203:6836","nonce":798312463}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[4,5],"metadata_pool":3,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-08-09T13:34:32.358 INFO:teuthology.orchestra.run.smithi043.stderr:dumped fsmap epoch 9 2023-08-09T13:34:32.359 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:32.357+0000 7fa618efb640 1 -- 172.21.15.43:0/3953535436 >> [v2:172.21.15.203:6832/830097474,v1:172.21.15.203:6833/830097474] conn(0x7fa5f406a050 msgr2=0x7fa5f406c510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:34:32.360 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:32.357+0000 7fa618efb640 1 --2- 172.21.15.43:0/3953535436 >> [v2:172.21.15.203:6832/830097474,v1:172.21.15.203:6833/830097474] conn(0x7fa5f406a050 0x7fa5f406c510 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7fa5fc006fd0 tx=0x7fa5fc008040 comp rx=0 tx=0).stop 2023-08-09T13:34:32.360 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:32.357+0000 7fa618efb640 1 -- 172.21.15.43:0/3953535436 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa6140acdf0 msgr2=0x7fa614098a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:34:32.360 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:32.357+0000 7fa618efb640 1 --2- 172.21.15.43:0/3953535436 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa6140acdf0 0x7fa614098a50 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7fa604000c00 tx=0x7fa6040046c0 comp rx=0 tx=0).stop 2023-08-09T13:34:32.360 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:32.357+0000 7fa618efb640 1 -- 172.21.15.43:0/3953535436 shutdown_connections 2023-08-09T13:34:32.361 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:32.357+0000 7fa618efb640 1 --2- 172.21.15.43:0/3953535436 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fa6140c8b30 0x7fa614147d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:34:32.361 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:32.357+0000 7fa618efb640 1 --2- 172.21.15.43:0/3953535436 >> [v2:172.21.15.203:6832/830097474,v1:172.21.15.203:6833/830097474] conn(0x7fa5f406a050 0x7fa5f406c510 unknown :-1 s=CLOSED pgs=91 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:34:32.361 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:32.357+0000 7fa618efb640 1 --2- 172.21.15.43:0/3953535436 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fa6140c8750 0x7fa6140b30a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:34:32.361 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:32.357+0000 7fa618efb640 1 --2- 172.21.15.43:0/3953535436 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa6140acdf0 0x7fa614098a50 unknown :-1 s=CLOSED pgs=141 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:34:32.361 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:32.357+0000 7fa618efb640 1 -- 172.21.15.43:0/3953535436 >> 172.21.15.43:0/3953535436 conn(0x7fa6140cde90 msgr2=0x7fa614159c90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-09T13:34:32.361 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:32.357+0000 7fa618efb640 1 -- 172.21.15.43:0/3953535436 shutdown_connections 2023-08-09T13:34:32.361 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:32.357+0000 7fa618efb640 1 -- 172.21.15.43:0/3953535436 wait complete. 2023-08-09T13:34:32.381 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-08-09T13:34:32.610 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:32.608+0000 7f463b97f640 1 Processor -- start 2023-08-09T13:34:32.611 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:32.608+0000 7f463b97f640 1 -- start start 2023-08-09T13:34:32.612 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:32.610+0000 7f463b97f640 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7f463415c600 0x7f46340d0fd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:34:32.612 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:32.610+0000 7f463b97f640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f46340cda10 0x7f46340cdea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:34:32.612 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:32.610+0000 7f463b97f640 1 -- --> v1:172.21.15.43:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f46340d0b40 con 0x7f46340cebc0 2023-08-09T13:34:32.613 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:32.610+0000 7f463b97f640 1 -- --> v2:172.21.15.43:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f463415c9e0 con 0x7f46340cda10 2023-08-09T13:34:32.613 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:32.610+0000 7f463b97f640 1 -- --> v2:172.21.15.203:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f46340cff30 con 0x7f463415c600 2023-08-09T13:34:32.613 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:32.610+0000 7f4638f1a640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f46340cda10 0x7f46340cdea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:34:32.614 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:32.610+0000 7f4639f1c640 1 --1- >> v1:172.21.15.43:6789/0 conn(0x7f46340cebc0 0x7f46340cefd0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.43:6789/0 says I am v1:172.21.15.43:59552/0 (socket says 172.21.15.43:59552) 2023-08-09T13:34:32.614 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:32.610+0000 7f4639f1c640 1 -- 172.21.15.43:0/3194040351 learned_addr learned my addr 172.21.15.43:0/3194040351 (peer_addr_for_me v1:172.21.15.43:0/0) 2023-08-09T13:34:32.614 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:32.610+0000 7f463971b640 1 --2- 172.21.15.43:0/3194040351 >> v2:172.21.15.203:3300/0 conn(0x7f463415c600 0x7f46340d0fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:34:32.614 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:32.611+0000 7f4638f1a640 1 -- 172.21.15.43:0/3194040351 >> v2:172.21.15.203:3300/0 conn(0x7f463415c600 msgr2=0x7f46340d0fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-09T13:34:32.614 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:32.611+0000 7f4638f1a640 1 --2- 172.21.15.43:0/3194040351 >> v2:172.21.15.203:3300/0 conn(0x7f463415c600 0x7f46340d0fd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:34:32.614 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:32.611+0000 7f4638f1a640 1 -- 172.21.15.43:0/3194040351 >> v1:172.21.15.43:6789/0 conn(0x7f46340cebc0 legacy=0x7f46340cefd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:34:32.615 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:32.611+0000 7f4638f1a640 1 -- 172.21.15.43:0/3194040351 --> v2:172.21.15.43:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f46340d0210 con 0x7f46340cda10 2023-08-09T13:34:32.615 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:32.611+0000 7f462bfff640 1 -- 172.21.15.43:0/3194040351 <== mon.0 v1:172.21.15.43:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 821410352 0 0) 0x7f46340d0b40 con 0x7f46340cebc0 2023-08-09T13:34:32.615 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:32.612+0000 7f4638f1a640 1 --2- 172.21.15.43:0/3194040351 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f46340cda10 0x7f46340cdea0 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7f463408c710 tx=0x7f462402b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5be3cf17a2d489cb server_cookie=0 in_seq=0 out_seq=0 2023-08-09T13:34:32.616 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:32.613+0000 7f462bfff640 1 -- 172.21.15.43:0/3194040351 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4624031830 con 0x7f46340cda10 2023-08-09T13:34:32.616 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:32.613+0000 7f462bfff640 1 -- 172.21.15.43:0/3194040351 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4624031990 con 0x7f46340cda10 2023-08-09T13:34:32.616 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:32.613+0000 7f462bfff640 1 -- 172.21.15.43:0/3194040351 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4624031d20 con 0x7f46340cda10 2023-08-09T13:34:32.616 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:32.613+0000 7f463b97f640 1 -- 172.21.15.43:0/3194040351 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f46340cda10 msgr2=0x7f46340cdea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:34:32.616 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:32.613+0000 7f463b97f640 1 --2- 172.21.15.43:0/3194040351 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f46340cda10 0x7f46340cdea0 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7f463408c710 tx=0x7f462402b8d0 comp rx=0 tx=0).stop 2023-08-09T13:34:32.617 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:32.613+0000 7f463b97f640 1 -- 172.21.15.43:0/3194040351 shutdown_connections 2023-08-09T13:34:32.617 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:32.613+0000 7f463b97f640 1 --2- 172.21.15.43:0/3194040351 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f46340cda10 0x7f46340cdea0 unknown :-1 s=CLOSED pgs=143 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:34:32.617 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:32.613+0000 7f463b97f640 1 --2- 172.21.15.43:0/3194040351 >> v2:172.21.15.203:3300/0 conn(0x7f463415c600 0x7f46340d0fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:34:32.617 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:32.613+0000 7f463b97f640 1 -- 172.21.15.43:0/3194040351 >> 172.21.15.43:0/3194040351 conn(0x7f463409b6d0 msgr2=0x7f46340d2360 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-09T13:34:32.617 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:32.613+0000 7f463b97f640 1 -- 172.21.15.43:0/3194040351 shutdown_connections 2023-08-09T13:34:32.618 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:32.614+0000 7f463b97f640 1 -- 172.21.15.43:0/3194040351 wait complete. 2023-08-09T13:34:32.618 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:32.615+0000 7f463b97f640 1 Processor -- start 2023-08-09T13:34:32.618 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:32.616+0000 7f463b97f640 1 -- start start 2023-08-09T13:34:32.619 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:32.616+0000 7f463b97f640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f46340cda10 0x7f4634159f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:34:32.619 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:32.616+0000 7f463b97f640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f46340cebc0 0x7f46340c8070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:34:32.619 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:32.616+0000 7f463b97f640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f463415c600 0x7f46340b74b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:34:32.619 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:32.616+0000 7f463b97f640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f46341f4c60 con 0x7f46340cda10 2023-08-09T13:34:32.620 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:32.616+0000 7f463b97f640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f46341f18a0 con 0x7f463415c600 2023-08-09T13:34:32.620 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:32.616+0000 7f463b97f640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f46340b5ee0 con 0x7f46340cebc0 2023-08-09T13:34:32.620 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:32.617+0000 7f463971b640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f46340cda10 0x7f4634159f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:34:32.620 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:32.617+0000 7f463971b640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f46340cda10 0x7f4634159f90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:37638/0 (socket says 172.21.15.43:37638) 2023-08-09T13:34:32.620 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:32.617+0000 7f463971b640 1 -- 172.21.15.43:0/498333824 learned_addr learned my addr 172.21.15.43:0/498333824 (peer_addr_for_me v2:172.21.15.43:0/0) 2023-08-09T13:34:32.621 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:32.617+0000 7f4638f1a640 1 --2- 172.21.15.43:0/498333824 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f46340cebc0 0x7f46340c8070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:34:32.621 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:32.617+0000 7f463971b640 1 -- 172.21.15.43:0/498333824 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f46340cebc0 msgr2=0x7f46340c8070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:34:32.622 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:32.617+0000 7f4639f1c640 1 --2- 172.21.15.43:0/498333824 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f463415c600 0x7f46340b74b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:34:32.622 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:32.617+0000 7f463971b640 1 --2- 172.21.15.43:0/498333824 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f46340cebc0 0x7f46340c8070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:34:32.622 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:32.617+0000 7f463971b640 1 -- 172.21.15.43:0/498333824 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f463415c600 msgr2=0x7f46340b74b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:34:32.622 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:32.617+0000 7f463971b640 1 --2- 172.21.15.43:0/498333824 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f463415c600 0x7f46340b74b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:34:32.622 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:32.617+0000 7f463971b640 1 -- 172.21.15.43:0/498333824 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4624009400 con 0x7f46340cda10 2023-08-09T13:34:32.622 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:32.618+0000 7f463971b640 1 --2- 172.21.15.43:0/498333824 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f46340cda10 0x7f4634159f90 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7f463000e870 tx=0x7f463000ed30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-09T13:34:32.623 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:32.618+0000 7f4629ffb640 1 -- 172.21.15.43:0/498333824 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f46300094f0 con 0x7f46340cda10 2023-08-09T13:34:32.623 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:32.618+0000 7f4629ffb640 1 -- 172.21.15.43:0/498333824 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f463000c940 con 0x7f46340cda10 2023-08-09T13:34:32.623 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:32.618+0000 7f4629ffb640 1 -- 172.21.15.43:0/498333824 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f463000ccf0 con 0x7f46340cda10 2023-08-09T13:34:32.623 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:32.618+0000 7f463b97f640 1 -- 172.21.15.43:0/498333824 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f46340b61c0 con 0x7f46340cda10 2023-08-09T13:34:32.623 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:32.618+0000 7f463b97f640 1 -- 172.21.15.43:0/498333824 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f46340b6780 con 0x7f46340cda10 2023-08-09T13:34:32.624 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:32.619+0000 7f463b97f640 1 -- 172.21.15.43:0/498333824 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f46340739f0 con 0x7f46340cda10 2023-08-09T13:34:32.624 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:32.620+0000 7f4629ffb640 1 -- 172.21.15.43:0/498333824 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 5) v1 ==== 88341+0+0 (secure 0 0 0) 0x7f4630013470 con 0x7f46340cda10 2023-08-09T13:34:32.624 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:32.621+0000 7f4629ffb640 1 --2- 172.21.15.43:0/498333824 >> [v2:172.21.15.203:6832/830097474,v1:172.21.15.203:6833/830097474] conn(0x7f461806a080 0x7f461806c540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:34:32.624 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:32.621+0000 7f4629ffb640 1 -- 172.21.15.43:0/498333824 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6813+0+0 (secure 0 0 0) 0x7f4630088940 con 0x7f46340cda10 2023-08-09T13:34:32.625 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:32.622+0000 7f4638f1a640 1 --2- 172.21.15.43:0/498333824 >> [v2:172.21.15.203:6832/830097474,v1:172.21.15.203:6833/830097474] conn(0x7f461806a080 0x7f461806c540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:34:32.625 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:32.623+0000 7f4638f1a640 1 --2- 172.21.15.43:0/498333824 >> [v2:172.21.15.203:6832/830097474,v1:172.21.15.203:6833/830097474] conn(0x7f461806a080 0x7f461806c540 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7f4624036000 tx=0x7f4624008030 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-09T13:34:32.628 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:32.626+0000 7f4629ffb640 1 -- 172.21.15.43:0/498333824 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178066 (secure 0 0 0) 0x7f4630053d30 con 0x7f46340cda10 2023-08-09T13:34:32.824 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:32.822+0000 7f463b97f640 1 -- 172.21.15.43:0/498333824 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f46340cf910 con 0x7f46340cda10 2023-08-09T13:34:32.826 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:32.824+0000 7f4629ffb640 1 -- 172.21.15.43:0/498333824 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v24) v1 ==== 74+0+20050 (secure 0 0 0) 0x7f4630057350 con 0x7f46340cda10 2023-08-09T13:34:32.827 INFO:teuthology.orchestra.run.smithi043.stdout: 2023-08-09T13:34:32.827 INFO:teuthology.orchestra.run.smithi043.stdout:{"epoch":24,"fsid":"daac87ee-1337-4250-a955-aea4e408a39b","created":"2023-08-09T13:33:41.701198+0000","modified":"2023-08-09T13:34:07.709429+0000","last_up_change":"2023-08-09T13:33:53.783453+0000","last_in_change":"2023-08-09T13:33:45.148904+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":5,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":"rbd","create_time":"2023-08-09T13:33:54.365137+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":".mgr","create_time":"2023-08-09T13:33:54.831803+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_metadata","create_time":"2023-08-09T13:34:00.207974+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data","create_time":"2023-08-09T13:34:01.614765+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":5,"pool_name":"cephfs_data_ec","create_time":"2023-08-09T13:34:05.239583+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"24","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"cbd49ecd-e896-4fbd-864e-5c37a829b165","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6800","nonce":502667335},{"type":"v1","addr":"172.21.15.43:6801","nonce":502667335}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6802","nonce":502667335},{"type":"v1","addr":"172.21.15.43:6803","nonce":502667335}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6806","nonce":502667335},{"type":"v1","addr":"172.21.15.43:6807","nonce":502667335}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6804","nonce":502667335},{"type":"v1","addr":"172.21.15.43:6805","nonce":502667335}]},"public_addr":"172.21.15.43:6801/502667335","cluster_addr":"172.21.15.43:6803/502667335","heartbeat_back_addr":"172.21.15.43:6807/502667335","heartbeat_front_addr":"172.21.15.43:6805/502667335","state":["exists","up"]},{"osd":1,"uuid":"75c7e0fe-1ec7-4fe8-a2f2-3661a65b8500","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6808","nonce":3289134610},{"type":"v1","addr":"172.21.15.43:6809","nonce":3289134610}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6810","nonce":3289134610},{"type":"v1","addr":"172.21.15.43:6812","nonce":3289134610}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6821","nonce":3289134610},{"type":"v1","addr":"172.21.15.43:6824","nonce":3289134610}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6815","nonce":3289134610},{"type":"v1","addr":"172.21.15.43:6818","nonce":3289134610}]},"public_addr":"172.21.15.43:6809/3289134610","cluster_addr":"172.21.15.43:6812/3289134610","heartbeat_back_addr":"172.21.15.43:6824/3289134610","heartbeat_front_addr":"172.21.15.43:6818/3289134610","state":["exists","up"]},{"osd":2,"uuid":"bb5c41e9-7f5f-4125-9b01-3bb8bfc2ea83","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6811","nonce":4079709736},{"type":"v1","addr":"172.21.15.43:6814","nonce":4079709736}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6817","nonce":4079709736},{"type":"v1","addr":"172.21.15.43:6820","nonce":4079709736}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6829","nonce":4079709736},{"type":"v1","addr":"172.21.15.43:6831","nonce":4079709736}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6823","nonce":4079709736},{"type":"v1","addr":"172.21.15.43:6826","nonce":4079709736}]},"public_addr":"172.21.15.43:6814/4079709736","cluster_addr":"172.21.15.43:6820/4079709736","heartbeat_back_addr":"172.21.15.43:6831/4079709736","heartbeat_front_addr":"172.21.15.43:6826/4079709736","state":["exists","up"]},{"osd":3,"uuid":"bfe4c8b4-694a-465c-87e6-2b2aab852613","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6813","nonce":1709538759},{"type":"v1","addr":"172.21.15.43:6816","nonce":1709538759}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6819","nonce":1709538759},{"type":"v1","addr":"172.21.15.43:6822","nonce":1709538759}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6828","nonce":1709538759},{"type":"v1","addr":"172.21.15.43:6830","nonce":1709538759}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6825","nonce":1709538759},{"type":"v1","addr":"172.21.15.43:6827","nonce":1709538759}]},"public_addr":"172.21.15.43:6816/1709538759","cluster_addr":"172.21.15.43:6822/1709538759","heartbeat_back_addr":"172.21.15.43:6830/1709538759","heartbeat_front_addr":"172.21.15.43:6827/1709538759","state":["exists","up"]},{"osd":4,"uuid":"fa2e8a77-aba1-4756-853e-0d174a5bdeba","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6800","nonce":4196866481},{"type":"v1","addr":"172.21.15.203:6801","nonce":4196866481}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6802","nonce":4196866481},{"type":"v1","addr":"172.21.15.203:6803","nonce":4196866481}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6806","nonce":4196866481},{"type":"v1","addr":"172.21.15.203:6807","nonce":4196866481}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6804","nonce":4196866481},{"type":"v1","addr":"172.21.15.203:6805","nonce":4196866481}]},"public_addr":"172.21.15.203:6801/4196866481","cluster_addr":"172.21.15.203:6803/4196866481","heartbeat_back_addr":"172.21.15.203:6807/4196866481","heartbeat_front_addr":"172.21.15.203:6805/4196866481","state":["exists","up"]},{"osd":5,"uuid":"81811e4b-4d24-4f51-a82c-f1d700a2a00d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6816","nonce":3059290081},{"type":"v1","addr":"172.21.15.203:6817","nonce":3059290081}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6818","nonce":3059290081},{"type":"v1","addr":"172.21.15.203:6819","nonce":3059290081}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6822","nonce":3059290081},{"type":"v1","addr":"172.21.15.203:6823","nonce":3059290081}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6820","nonce":3059290081},{"type":"v1","addr":"172.21.15.203:6821","nonce":3059290081}]},"public_addr":"172.21.15.203:6817/3059290081","cluster_addr":"172.21.15.203:6819/3059290081","heartbeat_back_addr":"172.21.15.203:6823/3059290081","heartbeat_front_addr":"172.21.15.203:6821/3059290081","state":["exists","up"]},{"osd":6,"uuid":"307bac4d-6a5b-443d-af75-56d695d7674f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6824","nonce":512155569},{"type":"v1","addr":"172.21.15.203:6825","nonce":512155569}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6826","nonce":512155569},{"type":"v1","addr":"172.21.15.203:6827","nonce":512155569}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6830","nonce":512155569},{"type":"v1","addr":"172.21.15.203:6831","nonce":512155569}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6828","nonce":512155569},{"type":"v1","addr":"172.21.15.203:6829","nonce":512155569}]},"public_addr":"172.21.15.203:6825/512155569","cluster_addr":"172.21.15.203:6827/512155569","heartbeat_back_addr":"172.21.15.203:6831/512155569","heartbeat_front_addr":"172.21.15.203:6829/512155569","state":["exists","up"]},{"osd":7,"uuid":"a95dc7ca-9f1b-4295-af70-68a22a2d7d5e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6808","nonce":3331450438},{"type":"v1","addr":"172.21.15.203:6809","nonce":3331450438}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6810","nonce":3331450438},{"type":"v1","addr":"172.21.15.203:6811","nonce":3331450438}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6814","nonce":3331450438},{"type":"v1","addr":"172.21.15.203:6815","nonce":3331450438}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6812","nonce":3331450438},{"type":"v1","addr":"172.21.15.203:6813","nonce":3331450438}]},"public_addr":"172.21.15.203:6809/3331450438","cluster_addr":"172.21.15.203:6811/3331450438","heartbeat_back_addr":"172.21.15.203:6815/3331450438","heartbeat_front_addr":"172.21.15.203:6813/3331450438","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-09T13:33:50.547652+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-09T13:33:50.589469+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-09T13:33:50.561724+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-09T13:33:50.599650+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-09T13:33:50.411855+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-09T13:33:50.219681+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-09T13:33:50.504664+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-09T13:33:50.441248+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-08-09T13:34:32.829 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:32.826+0000 7f463b97f640 1 -- 172.21.15.43:0/498333824 >> [v2:172.21.15.203:6832/830097474,v1:172.21.15.203:6833/830097474] conn(0x7f461806a080 msgr2=0x7f461806c540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:34:32.829 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:32.826+0000 7f463b97f640 1 --2- 172.21.15.43:0/498333824 >> [v2:172.21.15.203:6832/830097474,v1:172.21.15.203:6833/830097474] conn(0x7f461806a080 0x7f461806c540 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7f4624036000 tx=0x7f4624008030 comp rx=0 tx=0).stop 2023-08-09T13:34:32.829 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:32.826+0000 7f463b97f640 1 -- 172.21.15.43:0/498333824 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f46340cda10 msgr2=0x7f4634159f90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:34:32.830 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:32.826+0000 7f463b97f640 1 --2- 172.21.15.43:0/498333824 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f46340cda10 0x7f4634159f90 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7f463000e870 tx=0x7f463000ed30 comp rx=0 tx=0).stop 2023-08-09T13:34:32.830 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:32.827+0000 7f463b97f640 1 -- 172.21.15.43:0/498333824 shutdown_connections 2023-08-09T13:34:32.830 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:32.827+0000 7f463b97f640 1 --2- 172.21.15.43:0/498333824 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f463415c600 0x7f46340b74b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:34:32.830 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:32.827+0000 7f463b97f640 1 --2- 172.21.15.43:0/498333824 >> [v2:172.21.15.203:6832/830097474,v1:172.21.15.203:6833/830097474] conn(0x7f461806a080 0x7f461806c540 unknown :-1 s=CLOSED pgs=92 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:34:32.830 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:32.827+0000 7f463b97f640 1 --2- 172.21.15.43:0/498333824 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f46340cebc0 0x7f46340c8070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:34:32.830 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:32.827+0000 7f463b97f640 1 --2- 172.21.15.43:0/498333824 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f46340cda10 0x7f4634159f90 unknown :-1 s=CLOSED pgs=144 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:34:32.830 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:32.827+0000 7f463b97f640 1 -- 172.21.15.43:0/498333824 >> 172.21.15.43:0/498333824 conn(0x7f463409b6d0 msgr2=0x7f463415dc30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-09T13:34:32.831 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:32.827+0000 7f463b97f640 1 -- 172.21.15.43:0/498333824 shutdown_connections 2023-08-09T13:34:32.831 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:32.827+0000 7f463b97f640 1 -- 172.21.15.43:0/498333824 wait complete. 2023-08-09T13:34:32.848 INFO:tasks.cephfs.filesystem.ceph_manager:waiting for clean 2023-08-09T13:34:32.849 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-08-09T13:34:33.079 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:33.075+0000 7fbde325e640 1 Processor -- start 2023-08-09T13:34:33.079 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:33.076+0000 7fbde325e640 1 -- start start 2023-08-09T13:34:33.080 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:33.077+0000 7fbde325e640 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7fbddc0b2790 0x7fbddc0d2ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:34:33.080 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:33.077+0000 7fbde325e640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7fbddc0b3050 0x7fbddc0b3f60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:34:33.080 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:33.077+0000 7fbde325e640 1 -- --> v1:172.21.15.43:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbddc0ce090 con 0x7fbddc0b2b70 2023-08-09T13:34:33.080 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:33.077+0000 7fbde325e640 1 -- --> v2:172.21.15.43:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbddc0b3950 con 0x7fbddc0b3050 2023-08-09T13:34:33.080 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:33.077+0000 7fbde325e640 1 -- --> v2:172.21.15.203:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbddc0cd7f0 con 0x7fbddc0b2790 2023-08-09T13:34:33.081 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:33.078+0000 7fbdd3fff640 1 --1- >> v1:172.21.15.43:6789/0 conn(0x7fbddc0b2b70 0x7fbddc0b5330 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.43:6789/0 says I am v1:172.21.15.43:59562/0 (socket says 172.21.15.43:59562) 2023-08-09T13:34:33.081 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:33.078+0000 7fbdd3fff640 1 -- 172.21.15.43:0/1292921256 learned_addr learned my addr 172.21.15.43:0/1292921256 (peer_addr_for_me v1:172.21.15.43:0/0) 2023-08-09T13:34:33.081 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:33.078+0000 7fbde17fb640 1 --2- 172.21.15.43:0/1292921256 >> v2:172.21.15.43:3300/0 conn(0x7fbddc0b3050 0x7fbddc0b3f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:34:33.082 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:33.078+0000 7fbde0ffa640 1 --2- 172.21.15.43:0/1292921256 >> v2:172.21.15.203:3300/0 conn(0x7fbddc0b2790 0x7fbddc0d2ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:34:33.082 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:33.078+0000 7fbdd37fe640 1 -- 172.21.15.43:0/1292921256 <== mon.0 v1:172.21.15.43:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4069219595 0 0) 0x7fbddc0ce090 con 0x7fbddc0b2b70 2023-08-09T13:34:33.082 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:33.078+0000 7fbde17fb640 1 -- 172.21.15.43:0/1292921256 >> v2:172.21.15.203:3300/0 conn(0x7fbddc0b2790 msgr2=0x7fbddc0d2ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-09T13:34:33.082 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:33.079+0000 7fbde17fb640 1 --2- 172.21.15.43:0/1292921256 >> v2:172.21.15.203:3300/0 conn(0x7fbddc0b2790 0x7fbddc0d2ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:34:33.082 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:33.079+0000 7fbde17fb640 1 -- 172.21.15.43:0/1292921256 >> v1:172.21.15.43:6789/0 conn(0x7fbddc0b2b70 legacy=0x7fbddc0b5330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:34:33.082 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:33.079+0000 7fbde17fb640 1 -- 172.21.15.43:0/1292921256 --> v2:172.21.15.43:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbddc0cdad0 con 0x7fbddc0b3050 2023-08-09T13:34:33.083 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:33.080+0000 7fbde17fb640 1 --2- 172.21.15.43:0/1292921256 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fbddc0b3050 0x7fbddc0b3f60 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7fbddc08a4a0 tx=0x7fbdd402bed0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ac8660caa2625831 server_cookie=0 in_seq=0 out_seq=0 2023-08-09T13:34:33.084 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:33.080+0000 7fbdd37fe640 1 -- 172.21.15.43:0/1292921256 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbdd402cc80 con 0x7fbddc0b3050 2023-08-09T13:34:33.084 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:33.080+0000 7fbdd37fe640 1 -- 172.21.15.43:0/1292921256 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbdd402cde0 con 0x7fbddc0b3050 2023-08-09T13:34:33.084 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:33.080+0000 7fbdd37fe640 1 -- 172.21.15.43:0/1292921256 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbdd4034660 con 0x7fbddc0b3050 2023-08-09T13:34:33.084 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:33.081+0000 7fbde325e640 1 -- 172.21.15.43:0/1292921256 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fbddc0b3050 msgr2=0x7fbddc0b3f60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:34:33.084 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:33.081+0000 7fbde325e640 1 --2- 172.21.15.43:0/1292921256 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fbddc0b3050 0x7fbddc0b3f60 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7fbddc08a4a0 tx=0x7fbdd402bed0 comp rx=0 tx=0).stop 2023-08-09T13:34:33.084 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:33.081+0000 7fbde325e640 1 -- 172.21.15.43:0/1292921256 shutdown_connections 2023-08-09T13:34:33.084 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:33.081+0000 7fbde325e640 1 --2- 172.21.15.43:0/1292921256 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fbddc0b3050 0x7fbddc0b3f60 unknown :-1 s=CLOSED pgs=146 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:34:33.085 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:33.081+0000 7fbde325e640 1 --2- 172.21.15.43:0/1292921256 >> v2:172.21.15.203:3300/0 conn(0x7fbddc0b2790 0x7fbddc0d2ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:34:33.085 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:33.081+0000 7fbde325e640 1 -- 172.21.15.43:0/1292921256 >> 172.21.15.43:0/1292921256 conn(0x7fbddc09b7b0 msgr2=0x7fbddc0d24e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-09T13:34:33.085 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:33.081+0000 7fbde325e640 1 -- 172.21.15.43:0/1292921256 shutdown_connections 2023-08-09T13:34:33.085 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:33.081+0000 7fbde325e640 1 -- 172.21.15.43:0/1292921256 wait complete. 2023-08-09T13:34:33.085 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:33.083+0000 7fbde325e640 1 Processor -- start 2023-08-09T13:34:33.086 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:33.083+0000 7fbde325e640 1 -- start start 2023-08-09T13:34:33.087 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:33.084+0000 7fbde325e640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fbddc0b2790 0x7fbddc0c8b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:34:33.087 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:33.084+0000 7fbde325e640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fbddc0b2b70 0x7fbddc156670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:34:33.087 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:33.084+0000 7fbde325e640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fbddc0b3050 0x7fbddc1f9d70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:34:33.088 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:33.084+0000 7fbde325e640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbddc156bb0 con 0x7fbddc0b2790 2023-08-09T13:34:33.088 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:33.084+0000 7fbde325e640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbddc0cb6f0 con 0x7fbddc0b3050 2023-08-09T13:34:33.088 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:33.084+0000 7fbde325e640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbddc0c96e0 con 0x7fbddc0b2b70 2023-08-09T13:34:33.088 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:33.085+0000 7fbde0ffa640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fbddc0b2790 0x7fbddc0c8b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:34:33.088 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:33.085+0000 7fbde0ffa640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fbddc0b2790 0x7fbddc0c8b00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:37666/0 (socket says 172.21.15.43:37666) 2023-08-09T13:34:33.088 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:33.085+0000 7fbde0ffa640 1 -- 172.21.15.43:0/1934943957 learned_addr learned my addr 172.21.15.43:0/1934943957 (peer_addr_for_me v2:172.21.15.43:0/0) 2023-08-09T13:34:33.089 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:33.085+0000 7fbde17fb640 1 --2- 172.21.15.43:0/1934943957 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fbddc0b3050 0x7fbddc1f9d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:34:33.089 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:33.085+0000 7fbdd3fff640 1 --2- 172.21.15.43:0/1934943957 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fbddc0b2b70 0x7fbddc156670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:34:33.090 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:33.085+0000 7fbde0ffa640 1 -- 172.21.15.43:0/1934943957 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fbddc0b2b70 msgr2=0x7fbddc156670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:34:33.090 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:33.085+0000 7fbde0ffa640 1 --2- 172.21.15.43:0/1934943957 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fbddc0b2b70 0x7fbddc156670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:34:33.090 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:33.085+0000 7fbde0ffa640 1 -- 172.21.15.43:0/1934943957 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fbddc0b3050 msgr2=0x7fbddc1f9d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:34:33.090 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:33.085+0000 7fbde0ffa640 1 --2- 172.21.15.43:0/1934943957 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fbddc0b3050 0x7fbddc1f9d70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:34:33.090 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:33.085+0000 7fbde0ffa640 1 -- 172.21.15.43:0/1934943957 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbdd4009a70 con 0x7fbddc0b2790 2023-08-09T13:34:33.090 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:33.085+0000 7fbde17fb640 1 --2- 172.21.15.43:0/1934943957 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fbddc0b3050 0x7fbddc1f9d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-08-09T13:34:33.091 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:33.086+0000 7fbde0ffa640 1 --2- 172.21.15.43:0/1934943957 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fbddc0b2790 0x7fbddc0c8b00 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7fbdd800e870 tx=0x7fbdd800ed30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-09T13:34:33.091 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:33.086+0000 7fbdd17fa640 1 -- 172.21.15.43:0/1934943957 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbdd80094f0 con 0x7fbddc0b2790 2023-08-09T13:34:33.091 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:33.086+0000 7fbdd17fa640 1 -- 172.21.15.43:0/1934943957 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbdd800c940 con 0x7fbddc0b2790 2023-08-09T13:34:33.091 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:33.086+0000 7fbdd17fa640 1 -- 172.21.15.43:0/1934943957 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbdd800ccf0 con 0x7fbddc0b2790 2023-08-09T13:34:33.091 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:33.086+0000 7fbde325e640 1 -- 172.21.15.43:0/1934943957 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbddc0c99c0 con 0x7fbddc0b2790 2023-08-09T13:34:33.091 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:33.086+0000 7fbde325e640 1 -- 172.21.15.43:0/1934943957 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbddc0cb030 con 0x7fbddc0b2790 2023-08-09T13:34:33.091 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:33.087+0000 7fbde325e640 1 -- 172.21.15.43:0/1934943957 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbddc0c9be0 con 0x7fbddc0b2790 2023-08-09T13:34:33.092 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:33.088+0000 7fbdd17fa640 1 -- 172.21.15.43:0/1934943957 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 5) v1 ==== 88341+0+0 (secure 0 0 0) 0x7fbdd8013470 con 0x7fbddc0b2790 2023-08-09T13:34:33.092 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:33.089+0000 7fbdd17fa640 1 --2- 172.21.15.43:0/1934943957 >> [v2:172.21.15.203:6832/830097474,v1:172.21.15.203:6833/830097474] conn(0x7fbdc806a050 0x7fbdc806c510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:34:33.092 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:33.090+0000 7fbdd17fa640 1 -- 172.21.15.43:0/1934943957 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6813+0+0 (secure 0 0 0) 0x7fbdd8088970 con 0x7fbddc0b2790 2023-08-09T13:34:33.093 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:33.090+0000 7fbdd3fff640 1 --2- 172.21.15.43:0/1934943957 >> [v2:172.21.15.203:6832/830097474,v1:172.21.15.203:6833/830097474] conn(0x7fbdc806a050 0x7fbdc806c510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:34:33.093 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:33.091+0000 7fbdd3fff640 1 --2- 172.21.15.43:0/1934943957 >> [v2:172.21.15.203:6832/830097474,v1:172.21.15.203:6833/830097474] conn(0x7fbdc806a050 0x7fbdc806c510 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7fbddc1573d0 tx=0x7fbdcc007c00 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-09T13:34:33.096 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:33.094+0000 7fbdd17fa640 1 -- 172.21.15.43:0/1934943957 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178066 (secure 0 0 0) 0x7fbdd8053d60 con 0x7fbddc0b2790 2023-08-09T13:34:33.283 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:33.281+0000 7fbde325e640 1 -- 172.21.15.43:0/1934943957 --> [v2:172.21.15.203:6832/830097474,v1:172.21.15.203:6833/830097474] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7fbddc0d0930 con 0x7fbdc806a050 2023-08-09T13:34:33.318 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:33.316+0000 7fbdd17fa640 1 -- 172.21.15.43:0/1934943957 <== mgr.4114 v2:172.21.15.203:6832/830097474 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+367303 (secure 0 0 0) 0x7fbdcc007270 con 0x7fbdc806a050 2023-08-09T13:34:33.320 INFO:teuthology.orchestra.run.smithi043.stdout: 2023-08-09T13:34:33.325 INFO:teuthology.orchestra.run.smithi043.stderr:dumped all 2023-08-09T13:34:33.326 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:33.323+0000 7fbde325e640 1 -- 172.21.15.43:0/1934943957 >> [v2:172.21.15.203:6832/830097474,v1:172.21.15.203:6833/830097474] conn(0x7fbdc806a050 msgr2=0x7fbdc806c510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:34:33.327 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:33.323+0000 7fbde325e640 1 --2- 172.21.15.43:0/1934943957 >> [v2:172.21.15.203:6832/830097474,v1:172.21.15.203:6833/830097474] conn(0x7fbdc806a050 0x7fbdc806c510 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7fbddc1573d0 tx=0x7fbdcc007c00 comp rx=0 tx=0).stop 2023-08-09T13:34:33.327 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:33.323+0000 7fbde325e640 1 -- 172.21.15.43:0/1934943957 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fbddc0b2790 msgr2=0x7fbddc0c8b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:34:33.327 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:33.323+0000 7fbde325e640 1 --2- 172.21.15.43:0/1934943957 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fbddc0b2790 0x7fbddc0c8b00 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7fbdd800e870 tx=0x7fbdd800ed30 comp rx=0 tx=0).stop 2023-08-09T13:34:33.327 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:33.323+0000 7fbde325e640 1 -- 172.21.15.43:0/1934943957 shutdown_connections 2023-08-09T13:34:33.327 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:33.324+0000 7fbde325e640 1 --2- 172.21.15.43:0/1934943957 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fbddc0b3050 0x7fbddc1f9d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:34:33.328 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:33.324+0000 7fbde325e640 1 --2- 172.21.15.43:0/1934943957 >> [v2:172.21.15.203:6832/830097474,v1:172.21.15.203:6833/830097474] conn(0x7fbdc806a050 0x7fbdc806c510 unknown :-1 s=CLOSED pgs=93 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:34:33.328 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:33.324+0000 7fbde325e640 1 --2- 172.21.15.43:0/1934943957 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fbddc0b2b70 0x7fbddc156670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:34:33.328 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:33.324+0000 7fbde325e640 1 --2- 172.21.15.43:0/1934943957 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fbddc0b2790 0x7fbddc0c8b00 unknown :-1 s=CLOSED pgs=147 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:34:33.328 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:33.324+0000 7fbde325e640 1 -- 172.21.15.43:0/1934943957 >> 172.21.15.43:0/1934943957 conn(0x7fbddc09b7b0 msgr2=0x7fbddc0b10b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-09T13:34:33.328 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:33.324+0000 7fbde325e640 1 -- 172.21.15.43:0/1934943957 shutdown_connections 2023-08-09T13:34:33.328 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:33.324+0000 7fbde325e640 1 -- 172.21.15.43:0/1934943957 wait complete. 2023-08-09T13:34:33.349 INFO:teuthology.orchestra.run.smithi043.stdout:{"pg_ready":true,"pg_map":{"version":37,"stamp":"2023-08-09T13:34:32.771588+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":595007,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":208,"num_write_kb":2435,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160,"ondisk_log_size":160,"up":338,"acting":338,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":338,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":235764,"kb_used_data":3200,"kb_used_omap":13,"kb_used_meta":232434,"kb_avail":754738956,"statfs":{"total":773094113280,"available":772852690944,"internally_reserved":0,"allocated":3276800,"data_stored":2025132,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":14248,"internal_metadata":238012504},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":141,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.005783"},"pg_stats":[{"pgid":"5.19","version":"0'0","reported_seq":13,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-08-09T13:34:08.367616+0000","last_change":"2023-08-09T13:34:07.232493+0000","last_active":"2023-08-09T13:34:08.367616+0000","last_peered":"2023-08-09T13:34:08.367616+0000","last_clean":"2023-08-09T13:34:08.367616+0000","last_became_active":"2023-08-09T13:34:07.231969+0000","last_became_peered":"2023-08-09T13:34:07.231969+0000","last_unstale":"2023-08-09T13:34:08.367616+0000","last_undegraded":"2023-08-09T13:34:08.367616+0000","last_fullsized":"2023-08-09T13:34:08.367616+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:34:06.173547+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:34:06.173547+0000","last_clean_scrub_stamp":"2023-08-09T13:34:06.173547+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T19:42:37.842719+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,7,2],"acting":[1,5,7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.1f","version":"19'4","reported_seq":23,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-08-09T13:34:08.209913+0000","last_change":"2023-08-09T13:34:02.108915+0000","last_active":"2023-08-09T13:34:08.209913+0000","last_peered":"2023-08-09T13:34:08.209913+0000","last_clean":"2023-08-09T13:34:08.209913+0000","last_became_active":"2023-08-09T13:34:02.108573+0000","last_became_peered":"2023-08-09T13:34:02.108573+0000","last_unstale":"2023-08-09T13:34:08.209913+0000","last_undegraded":"2023-08-09T13:34:08.209913+0000","last_fullsized":"2023-08-09T13:34:08.209913+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:34:01.052721+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:34:01.052721+0000","last_clean_scrub_stamp":"2023-08-09T13:34:01.052721+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T23:52:26.726953+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-08-09T13:34:08.370536+0000","last_change":"2023-08-09T13:34:03.156516+0000","last_active":"2023-08-09T13:34:08.370536+0000","last_peered":"2023-08-09T13:34:08.370536+0000","last_clean":"2023-08-09T13:34:08.370536+0000","last_became_active":"2023-08-09T13:34:03.156109+0000","last_became_peered":"2023-08-09T13:34:03.156109+0000","last_unstale":"2023-08-09T13:34:08.370536+0000","last_undegraded":"2023-08-09T13:34:08.370536+0000","last_fullsized":"2023-08-09T13:34:08.370536+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:34:02.064154+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:34:02.064154+0000","last_clean_scrub_stamp":"2023-08-09T13:34:02.064154+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T13:40:56.937864+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.18","version":"0'0","reported_seq":13,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-08-09T13:34:08.369809+0000","last_change":"2023-08-09T13:34:07.234694+0000","last_active":"2023-08-09T13:34:08.369809+0000","last_peered":"2023-08-09T13:34:08.369809+0000","last_clean":"2023-08-09T13:34:08.369809+0000","last_became_active":"2023-08-09T13:34:07.234241+0000","last_became_peered":"2023-08-09T13:34:07.234241+0000","last_unstale":"2023-08-09T13:34:08.369809+0000","last_undegraded":"2023-08-09T13:34:08.369809+0000","last_fullsized":"2023-08-09T13:34:08.369809+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:34:06.173547+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:34:06.173547+0000","last_clean_scrub_stamp":"2023-08-09T13:34:06.173547+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T15:06:44.662571+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,1,2],"acting":[4,6,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.1e","version":"19'1","reported_seq":22,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-08-09T13:34:08.370464+0000","last_change":"2023-08-09T13:34:02.105244+0000","last_active":"2023-08-09T13:34:08.370464+0000","last_peered":"2023-08-09T13:34:08.370464+0000","last_clean":"2023-08-09T13:34:08.370464+0000","last_became_active":"2023-08-09T13:34:02.104920+0000","last_became_peered":"2023-08-09T13:34:02.104920+0000","last_unstale":"2023-08-09T13:34:08.370464+0000","last_undegraded":"2023-08-09T13:34:08.370464+0000","last_fullsized":"2023-08-09T13:34:08.370464+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:34:01.052721+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:34:01.052721+0000","last_clean_scrub_stamp":"2023-08-09T13:34:01.052721+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T21:57:53.171937+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-08-09T13:34:08.368933+0000","last_change":"2023-08-09T13:34:03.156960+0000","last_active":"2023-08-09T13:34:08.368933+0000","last_peered":"2023-08-09T13:34:08.368933+0000","last_clean":"2023-08-09T13:34:08.368933+0000","last_became_active":"2023-08-09T13:34:03.156668+0000","last_became_peered":"2023-08-09T13:34:03.156668+0000","last_unstale":"2023-08-09T13:34:08.368933+0000","last_undegraded":"2023-08-09T13:34:08.368933+0000","last_fullsized":"2023-08-09T13:34:08.368933+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:34:02.064154+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:34:02.064154+0000","last_clean_scrub_stamp":"2023-08-09T13:34:02.064154+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T14:55:32.816063+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.1b","version":"0'0","reported_seq":13,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-08-09T13:34:08.370229+0000","last_change":"2023-08-09T13:34:07.232500+0000","last_active":"2023-08-09T13:34:08.370229+0000","last_peered":"2023-08-09T13:34:08.370229+0000","last_clean":"2023-08-09T13:34:08.370229+0000","last_became_active":"2023-08-09T13:34:07.232045+0000","last_became_peered":"2023-08-09T13:34:07.232045+0000","last_unstale":"2023-08-09T13:34:08.370229+0000","last_undegraded":"2023-08-09T13:34:08.370229+0000","last_fullsized":"2023-08-09T13:34:08.370229+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:34:06.173547+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:34:06.173547+0000","last_clean_scrub_stamp":"2023-08-09T13:34:06.173547+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T18:36:44.021636+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,7,4],"acting":[5,0,7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1d","version":"19'2","reported_seq":21,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-08-09T13:34:08.370340+0000","last_change":"2023-08-09T13:34:02.109750+0000","last_active":"2023-08-09T13:34:08.370340+0000","last_peered":"2023-08-09T13:34:08.370340+0000","last_clean":"2023-08-09T13:34:08.370340+0000","last_became_active":"2023-08-09T13:34:02.109445+0000","last_became_peered":"2023-08-09T13:34:02.109445+0000","last_unstale":"2023-08-09T13:34:08.370340+0000","last_undegraded":"2023-08-09T13:34:08.370340+0000","last_fullsized":"2023-08-09T13:34:08.370340+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:34:01.052721+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:34:01.052721+0000","last_clean_scrub_stamp":"2023-08-09T13:34:01.052721+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T22:30:05.448240+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1a","version":"0'0","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-08-09T13:34:08.371675+0000","last_change":"2023-08-09T13:34:03.145198+0000","last_active":"2023-08-09T13:34:08.371675+0000","last_peered":"2023-08-09T13:34:08.371675+0000","last_clean":"2023-08-09T13:34:08.371675+0000","last_became_active":"2023-08-09T13:34:03.144756+0000","last_became_peered":"2023-08-09T13:34:03.144756+0000","last_unstale":"2023-08-09T13:34:08.371675+0000","last_undegraded":"2023-08-09T13:34:08.371675+0000","last_fullsized":"2023-08-09T13:34:08.371675+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:34:02.064154+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:34:02.064154+0000","last_clean_scrub_stamp":"2023-08-09T13:34:02.064154+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T16:10:58.188522+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1a","version":"0'0","reported_seq":13,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-08-09T13:34:08.369069+0000","last_change":"2023-08-09T13:34:07.238058+0000","last_active":"2023-08-09T13:34:08.369069+0000","last_peered":"2023-08-09T13:34:08.369069+0000","last_clean":"2023-08-09T13:34:08.369069+0000","last_became_active":"2023-08-09T13:34:07.237575+0000","last_became_peered":"2023-08-09T13:34:07.237575+0000","last_unstale":"2023-08-09T13:34:08.369069+0000","last_undegraded":"2023-08-09T13:34:08.369069+0000","last_fullsized":"2023-08-09T13:34:08.369069+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:34:06.173547+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:34:06.173547+0000","last_clean_scrub_stamp":"2023-08-09T13:34:06.173547+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T21:18:23.660943+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,1,6],"acting":[7,4,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1c","version":"19'2","reported_seq":21,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-08-09T13:34:08.369633+0000","last_change":"2023-08-09T13:34:02.107853+0000","last_active":"2023-08-09T13:34:08.369633+0000","last_peered":"2023-08-09T13:34:08.369633+0000","last_clean":"2023-08-09T13:34:08.369633+0000","last_became_active":"2023-08-09T13:34:02.107571+0000","last_became_peered":"2023-08-09T13:34:02.107571+0000","last_unstale":"2023-08-09T13:34:08.369633+0000","last_undegraded":"2023-08-09T13:34:08.369633+0000","last_fullsized":"2023-08-09T13:34:08.369633+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:34:01.052721+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:34:01.052721+0000","last_clean_scrub_stamp":"2023-08-09T13:34:01.052721+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T19:42:32.721893+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1b","version":"0'0","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-08-09T13:34:08.371599+0000","last_change":"2023-08-09T13:34:03.148927+0000","last_active":"2023-08-09T13:34:08.371599+0000","last_peered":"2023-08-09T13:34:08.371599+0000","last_clean":"2023-08-09T13:34:08.371599+0000","last_became_active":"2023-08-09T13:34:03.148582+0000","last_became_peered":"2023-08-09T13:34:03.148582+0000","last_unstale":"2023-08-09T13:34:08.371599+0000","last_undegraded":"2023-08-09T13:34:08.371599+0000","last_fullsized":"2023-08-09T13:34:08.371599+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:34:02.064154+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:34:02.064154+0000","last_clean_scrub_stamp":"2023-08-09T13:34:02.064154+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T17:58:47.470353+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1d","version":"0'0","reported_seq":13,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-08-09T13:34:08.372087+0000","last_change":"2023-08-09T13:34:07.246175+0000","last_active":"2023-08-09T13:34:08.372087+0000","last_peered":"2023-08-09T13:34:08.372087+0000","last_clean":"2023-08-09T13:34:08.372087+0000","last_became_active":"2023-08-09T13:34:07.245757+0000","last_became_peered":"2023-08-09T13:34:07.245757+0000","last_unstale":"2023-08-09T13:34:08.372087+0000","last_undegraded":"2023-08-09T13:34:08.372087+0000","last_fullsized":"2023-08-09T13:34:08.372087+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:34:06.173547+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:34:06.173547+0000","last_clean_scrub_stamp":"2023-08-09T13:34:06.173547+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T18:07:35.814562+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,4,0],"acting":[1,3,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.1b","version":"19'1","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-08-09T13:34:08.210113+0000","last_change":"2023-08-09T13:34:02.109573+0000","last_active":"2023-08-09T13:34:08.210113+0000","last_peered":"2023-08-09T13:34:08.210113+0000","last_clean":"2023-08-09T13:34:08.210113+0000","last_became_active":"2023-08-09T13:34:02.109015+0000","last_became_peered":"2023-08-09T13:34:02.109015+0000","last_unstale":"2023-08-09T13:34:08.210113+0000","last_undegraded":"2023-08-09T13:34:08.210113+0000","last_fullsized":"2023-08-09T13:34:08.210113+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:34:01.052721+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:34:01.052721+0000","last_clean_scrub_stamp":"2023-08-09T13:34:01.052721+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T13:44:49.753930+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1c","version":"0'0","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-08-09T13:34:08.212417+0000","last_change":"2023-08-09T13:34:03.162651+0000","last_active":"2023-08-09T13:34:08.212417+0000","last_peered":"2023-08-09T13:34:08.212417+0000","last_clean":"2023-08-09T13:34:08.212417+0000","last_became_active":"2023-08-09T13:34:03.161575+0000","last_became_peered":"2023-08-09T13:34:03.161575+0000","last_unstale":"2023-08-09T13:34:08.212417+0000","last_undegraded":"2023-08-09T13:34:08.212417+0000","last_fullsized":"2023-08-09T13:34:08.212417+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:34:02.064154+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:34:02.064154+0000","last_clean_scrub_stamp":"2023-08-09T13:34:02.064154+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-11T00:30:20.150060+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.1c","version":"0'0","reported_seq":13,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-08-09T13:34:08.372392+0000","last_change":"2023-08-09T13:34:07.242334+0000","last_active":"2023-08-09T13:34:08.372392+0000","last_peered":"2023-08-09T13:34:08.372392+0000","last_clean":"2023-08-09T13:34:08.372392+0000","last_became_active":"2023-08-09T13:34:07.241841+0000","last_became_peered":"2023-08-09T13:34:07.241841+0000","last_unstale":"2023-08-09T13:34:08.372392+0000","last_undegraded":"2023-08-09T13:34:08.372392+0000","last_fullsized":"2023-08-09T13:34:08.372392+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:34:06.173547+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:34:06.173547+0000","last_clean_scrub_stamp":"2023-08-09T13:34:06.173547+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T14:35:09.213789+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,2,0],"acting":[4,3,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.1a","version":"19'1","reported_seq":22,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-08-09T13:34:08.372077+0000","last_change":"2023-08-09T13:34:02.104004+0000","last_active":"2023-08-09T13:34:08.372077+0000","last_peered":"2023-08-09T13:34:08.372077+0000","last_clean":"2023-08-09T13:34:08.372077+0000","last_became_active":"2023-08-09T13:34:02.103701+0000","last_became_peered":"2023-08-09T13:34:02.103701+0000","last_unstale":"2023-08-09T13:34:08.372077+0000","last_undegraded":"2023-08-09T13:34:08.372077+0000","last_fullsized":"2023-08-09T13:34:08.372077+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:34:01.052721+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:34:01.052721+0000","last_clean_scrub_stamp":"2023-08-09T13:34:01.052721+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T18:58:30.540971+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.1d","version":"0'0","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-08-09T13:34:08.371028+0000","last_change":"2023-08-09T13:34:03.156422+0000","last_active":"2023-08-09T13:34:08.371028+0000","last_peered":"2023-08-09T13:34:08.371028+0000","last_clean":"2023-08-09T13:34:08.371028+0000","last_became_active":"2023-08-09T13:34:03.155091+0000","last_became_peered":"2023-08-09T13:34:03.155091+0000","last_unstale":"2023-08-09T13:34:08.371028+0000","last_undegraded":"2023-08-09T13:34:08.371028+0000","last_fullsized":"2023-08-09T13:34:08.371028+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:34:02.064154+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:34:02.064154+0000","last_clean_scrub_stamp":"2023-08-09T13:34:02.064154+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T15:06:40.553178+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.1f","version":"0'0","reported_seq":13,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-08-09T13:34:08.213034+0000","last_change":"2023-08-09T13:34:07.241585+0000","last_active":"2023-08-09T13:34:08.213034+0000","last_peered":"2023-08-09T13:34:08.213034+0000","last_clean":"2023-08-09T13:34:08.213034+0000","last_became_active":"2023-08-09T13:34:07.241206+0000","last_became_peered":"2023-08-09T13:34:07.241206+0000","last_unstale":"2023-08-09T13:34:08.213034+0000","last_undegraded":"2023-08-09T13:34:08.213034+0000","last_fullsized":"2023-08-09T13:34:08.213034+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:34:06.173547+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:34:06.173547+0000","last_clean_scrub_stamp":"2023-08-09T13:34:06.173547+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-11T00:43:38.951241+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,3,7],"acting":[6,4,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":21,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-08-09T13:34:08.368085+0000","last_change":"2023-08-09T13:34:02.104251+0000","last_active":"2023-08-09T13:34:08.368085+0000","last_peered":"2023-08-09T13:34:08.368085+0000","last_clean":"2023-08-09T13:34:08.368085+0000","last_became_active":"2023-08-09T13:34:02.103920+0000","last_became_peered":"2023-08-09T13:34:02.103920+0000","last_unstale":"2023-08-09T13:34:08.368085+0000","last_undegraded":"2023-08-09T13:34:08.368085+0000","last_fullsized":"2023-08-09T13:34:08.368085+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:34:01.052721+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:34:01.052721+0000","last_clean_scrub_stamp":"2023-08-09T13:34:01.052721+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T18:36:38.900810+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1e","version":"0'0","reported_seq":17,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-08-09T13:34:08.208574+0000","last_change":"2023-08-09T13:34:03.172919+0000","last_active":"2023-08-09T13:34:08.208574+0000","last_peered":"2023-08-09T13:34:08.208574+0000","last_clean":"2023-08-09T13:34:08.208574+0000","last_became_active":"2023-08-09T13:34:03.172587+0000","last_became_peered":"2023-08-09T13:34:03.172587+0000","last_unstale":"2023-08-09T13:34:08.208574+0000","last_undegraded":"2023-08-09T13:34:08.208574+0000","last_fullsized":"2023-08-09T13:34:08.208574+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:34:02.064154+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:34:02.064154+0000","last_clean_scrub_stamp":"2023-08-09T13:34:02.064154+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T15:53:25.307248+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.1e","version":"0'0","reported_seq":13,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-08-09T13:34:08.209416+0000","last_change":"2023-08-09T13:34:07.236079+0000","last_active":"2023-08-09T13:34:08.209416+0000","last_peered":"2023-08-09T13:34:08.209416+0000","last_clean":"2023-08-09T13:34:08.209416+0000","last_became_active":"2023-08-09T13:34:07.234955+0000","last_became_peered":"2023-08-09T13:34:07.234955+0000","last_unstale":"2023-08-09T13:34:08.209416+0000","last_undegraded":"2023-08-09T13:34:08.209416+0000","last_fullsized":"2023-08-09T13:34:08.209416+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:34:06.173547+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:34:06.173547+0000","last_clean_scrub_stamp":"2023-08-09T13:34:06.173547+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T22:30:10.569066+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,1,2],"acting":[0,7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":21,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-08-09T13:34:08.368549+0000","last_change":"2023-08-09T13:34:02.103043+0000","last_active":"2023-08-09T13:34:08.368549+0000","last_peered":"2023-08-09T13:34:08.368549+0000","last_clean":"2023-08-09T13:34:08.368549+0000","last_became_active":"2023-08-09T13:34:02.102724+0000","last_became_peered":"2023-08-09T13:34:02.102724+0000","last_unstale":"2023-08-09T13:34:08.368549+0000","last_undegraded":"2023-08-09T13:34:08.368549+0000","last_fullsized":"2023-08-09T13:34:08.368549+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:34:01.052721+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:34:01.052721+0000","last_clean_scrub_stamp":"2023-08-09T13:34:01.052721+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T17:59:27.409131+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1f","version":"0'0","reported_seq":17,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-08-09T13:34:08.213083+0000","last_change":"2023-08-09T13:34:03.150663+0000","last_active":"2023-08-09T13:34:08.213083+0000","last_peered":"2023-08-09T13:34:08.213083+0000","last_clean":"2023-08-09T13:34:08.213083+0000","last_became_active":"2023-08-09T13:34:03.150322+0000","last_became_peered":"2023-08-09T13:34:03.150322+0000","last_unstale":"2023-08-09T13:34:08.213083+0000","last_undegraded":"2023-08-09T13:34:08.213083+0000","last_fullsized":"2023-08-09T13:34:08.213083+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:34:02.064154+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:34:02.064154+0000","last_clean_scrub_stamp":"2023-08-09T13:34:02.064154+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T22:09:11.225000+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.10","version":"0'0","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-08-09T13:34:08.368129+0000","last_change":"2023-08-09T13:34:03.154639+0000","last_active":"2023-08-09T13:34:08.368129+0000","last_peered":"2023-08-09T13:34:08.368129+0000","last_clean":"2023-08-09T13:34:08.368129+0000","last_became_active":"2023-08-09T13:34:03.151967+0000","last_became_peered":"2023-08-09T13:34:03.151967+0000","last_unstale":"2023-08-09T13:34:08.368129+0000","last_undegraded":"2023-08-09T13:34:08.368129+0000","last_fullsized":"2023-08-09T13:34:08.368129+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:34:02.064154+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:34:02.064154+0000","last_clean_scrub_stamp":"2023-08-09T13:34:02.064154+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-11T00:11:21.887434+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-08-09T13:34:08.210224+0000","last_change":"2023-08-09T13:34:02.108763+0000","last_active":"2023-08-09T13:34:08.210224+0000","last_peered":"2023-08-09T13:34:08.210224+0000","last_clean":"2023-08-09T13:34:08.210224+0000","last_became_active":"2023-08-09T13:34:02.108439+0000","last_became_peered":"2023-08-09T13:34:02.108439+0000","last_unstale":"2023-08-09T13:34:08.210224+0000","last_undegraded":"2023-08-09T13:34:08.210224+0000","last_fullsized":"2023-08-09T13:34:08.210224+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:34:01.052721+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:34:01.052721+0000","last_clean_scrub_stamp":"2023-08-09T13:34:01.052721+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T17:08:06.492548+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.11","version":"0'0","reported_seq":13,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-08-09T13:34:08.211048+0000","last_change":"2023-08-09T13:34:07.241202+0000","last_active":"2023-08-09T13:34:08.211048+0000","last_peered":"2023-08-09T13:34:08.211048+0000","last_clean":"2023-08-09T13:34:08.211048+0000","last_became_active":"2023-08-09T13:34:07.240678+0000","last_became_peered":"2023-08-09T13:34:07.240678+0000","last_unstale":"2023-08-09T13:34:08.211048+0000","last_undegraded":"2023-08-09T13:34:08.211048+0000","last_fullsized":"2023-08-09T13:34:08.211048+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:34:06.173547+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:34:06.173547+0000","last_clean_scrub_stamp":"2023-08-09T13:34:06.173547+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T22:29:57.384386+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,7,1],"acting":[6,4,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-08-09T13:34:08.369040+0000","last_change":"2023-08-09T13:34:03.163640+0000","last_active":"2023-08-09T13:34:08.369040+0000","last_peered":"2023-08-09T13:34:08.369040+0000","last_clean":"2023-08-09T13:34:08.369040+0000","last_became_active":"2023-08-09T13:34:03.163333+0000","last_became_peered":"2023-08-09T13:34:03.163333+0000","last_unstale":"2023-08-09T13:34:08.369040+0000","last_undegraded":"2023-08-09T13:34:08.369040+0000","last_fullsized":"2023-08-09T13:34:08.369040+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:34:02.064154+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:34:02.064154+0000","last_clean_scrub_stamp":"2023-08-09T13:34:02.064154+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T17:55:41.057906+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-08-09T13:34:08.371698+0000","last_change":"2023-08-09T13:34:02.112947+0000","last_active":"2023-08-09T13:34:08.371698+0000","last_peered":"2023-08-09T13:34:08.371698+0000","last_clean":"2023-08-09T13:34:08.371698+0000","last_became_active":"2023-08-09T13:34:02.112545+0000","last_became_peered":"2023-08-09T13:34:02.112545+0000","last_unstale":"2023-08-09T13:34:08.371698+0000","last_undegraded":"2023-08-09T13:34:08.371698+0000","last_fullsized":"2023-08-09T13:34:08.371698+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:34:01.052721+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:34:01.052721+0000","last_clean_scrub_stamp":"2023-08-09T13:34:01.052721+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T18:07:30.693736+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.10","version":"0'0","reported_seq":13,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-08-09T13:34:08.367894+0000","last_change":"2023-08-09T13:34:07.231662+0000","last_active":"2023-08-09T13:34:08.367894+0000","last_peered":"2023-08-09T13:34:08.367894+0000","last_clean":"2023-08-09T13:34:08.367894+0000","last_became_active":"2023-08-09T13:34:07.231214+0000","last_became_peered":"2023-08-09T13:34:07.231214+0000","last_unstale":"2023-08-09T13:34:08.367894+0000","last_undegraded":"2023-08-09T13:34:08.367894+0000","last_fullsized":"2023-08-09T13:34:08.367894+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:34:06.173547+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:34:06.173547+0000","last_clean_scrub_stamp":"2023-08-09T13:34:06.173547+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T21:35:05.575947+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6,0],"acting":[7,4,6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.15","version":"19'3","reported_seq":22,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-08-09T13:34:08.370972+0000","last_change":"2023-08-09T13:34:02.112344+0000","last_active":"2023-08-09T13:34:08.370972+0000","last_peered":"2023-08-09T13:34:08.370972+0000","last_clean":"2023-08-09T13:34:08.370972+0000","last_became_active":"2023-08-09T13:34:02.111749+0000","last_became_peered":"2023-08-09T13:34:02.111749+0000","last_unstale":"2023-08-09T13:34:08.370972+0000","last_undegraded":"2023-08-09T13:34:08.370972+0000","last_fullsized":"2023-08-09T13:34:08.370972+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:34:01.052721+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:34:01.052721+0000","last_clean_scrub_stamp":"2023-08-09T13:34:01.052721+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T15:33:08.276561+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-08-09T13:34:08.369248+0000","last_change":"2023-08-09T13:34:03.154995+0000","last_active":"2023-08-09T13:34:08.369248+0000","last_peered":"2023-08-09T13:34:08.369248+0000","last_clean":"2023-08-09T13:34:08.369248+0000","last_became_active":"2023-08-09T13:34:03.154715+0000","last_became_peered":"2023-08-09T13:34:03.154715+0000","last_unstale":"2023-08-09T13:34:08.369248+0000","last_undegraded":"2023-08-09T13:34:08.369248+0000","last_fullsized":"2023-08-09T13:34:08.369248+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:34:02.064154+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:34:02.064154+0000","last_clean_scrub_stamp":"2023-08-09T13:34:02.064154+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T22:30:06.459673+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.13","version":"0'0","reported_seq":13,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-08-09T13:34:08.368692+0000","last_change":"2023-08-09T13:34:07.241979+0000","last_active":"2023-08-09T13:34:08.368692+0000","last_peered":"2023-08-09T13:34:08.368692+0000","last_clean":"2023-08-09T13:34:08.368692+0000","last_became_active":"2023-08-09T13:34:07.241025+0000","last_became_peered":"2023-08-09T13:34:07.241025+0000","last_unstale":"2023-08-09T13:34:08.368692+0000","last_undegraded":"2023-08-09T13:34:08.368692+0000","last_fullsized":"2023-08-09T13:34:08.368692+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:34:06.173547+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:34:06.173547+0000","last_clean_scrub_stamp":"2023-08-09T13:34:06.173547+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T17:42:24.789866+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,1,5],"acting":[3,0,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.14","version":"24'10","reported_seq":31,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-08-09T13:34:22.970697+0000","last_change":"2023-08-09T13:34:02.113338+0000","last_active":"2023-08-09T13:34:22.970697+0000","last_peered":"2023-08-09T13:34:22.970697+0000","last_clean":"2023-08-09T13:34:22.970697+0000","last_became_active":"2023-08-09T13:34:02.113042+0000","last_became_peered":"2023-08-09T13:34:02.113042+0000","last_unstale":"2023-08-09T13:34:22.970697+0000","last_undegraded":"2023-08-09T13:34:22.970697+0000","last_fullsized":"2023-08-09T13:34:22.970697+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:34:01.052721+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:34:01.052721+0000","last_clean_scrub_stamp":"2023-08-09T13:34:01.052721+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T21:15:54.123137+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":3841,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.13","version":"0'0","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-08-09T13:34:08.371930+0000","last_change":"2023-08-09T13:34:03.148233+0000","last_active":"2023-08-09T13:34:08.371930+0000","last_peered":"2023-08-09T13:34:08.371930+0000","last_clean":"2023-08-09T13:34:08.371930+0000","last_became_active":"2023-08-09T13:34:03.147860+0000","last_became_peered":"2023-08-09T13:34:03.147860+0000","last_unstale":"2023-08-09T13:34:08.371930+0000","last_undegraded":"2023-08-09T13:34:08.371930+0000","last_fullsized":"2023-08-09T13:34:08.371930+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:34:02.064154+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:34:02.064154+0000","last_clean_scrub_stamp":"2023-08-09T13:34:02.064154+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T18:17:08.524229+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.12","version":"0'0","reported_seq":13,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-08-09T13:34:08.368474+0000","last_change":"2023-08-09T13:34:07.236661+0000","last_active":"2023-08-09T13:34:08.368474+0000","last_peered":"2023-08-09T13:34:08.368474+0000","last_clean":"2023-08-09T13:34:08.368474+0000","last_became_active":"2023-08-09T13:34:07.236264+0000","last_became_peered":"2023-08-09T13:34:07.236264+0000","last_unstale":"2023-08-09T13:34:08.368474+0000","last_undegraded":"2023-08-09T13:34:08.368474+0000","last_fullsized":"2023-08-09T13:34:08.368474+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:34:06.173547+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:34:06.173547+0000","last_clean_scrub_stamp":"2023-08-09T13:34:06.173547+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T21:35:05.575947+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,3,7],"acting":[1,5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-08-09T13:34:08.369602+0000","last_change":"2023-08-09T13:34:03.149954+0000","last_active":"2023-08-09T13:34:08.369602+0000","last_peered":"2023-08-09T13:34:08.369602+0000","last_clean":"2023-08-09T13:34:08.369602+0000","last_became_active":"2023-08-09T13:34:03.149648+0000","last_became_peered":"2023-08-09T13:34:03.149648+0000","last_unstale":"2023-08-09T13:34:08.369602+0000","last_undegraded":"2023-08-09T13:34:08.369602+0000","last_fullsized":"2023-08-09T13:34:08.369602+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:34:02.064154+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:34:02.064154+0000","last_clean_scrub_stamp":"2023-08-09T13:34:02.064154+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T18:50:05.285807+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":25,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-08-09T13:34:08.213159+0000","last_change":"2023-08-09T13:33:55.886389+0000","last_active":"2023-08-09T13:34:08.213159+0000","last_peered":"2023-08-09T13:34:08.213159+0000","last_clean":"2023-08-09T13:34:08.213159+0000","last_became_active":"2023-08-09T13:33:55.885638+0000","last_became_peered":"2023-08-09T13:33:55.885638+0000","last_unstale":"2023-08-09T13:34:08.213159+0000","last_undegraded":"2023-08-09T13:34:08.213159+0000","last_fullsized":"2023-08-09T13:34:08.213159+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:33:54.845869+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:33:54.845869+0000","last_clean_scrub_stamp":"2023-08-09T13:33:54.845869+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T23:11:03.836310+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-08-09T13:34:08.370437+0000","last_change":"2023-08-09T13:34:02.114759+0000","last_active":"2023-08-09T13:34:08.370437+0000","last_peered":"2023-08-09T13:34:08.370437+0000","last_clean":"2023-08-09T13:34:08.370437+0000","last_became_active":"2023-08-09T13:34:02.114423+0000","last_became_peered":"2023-08-09T13:34:02.114423+0000","last_unstale":"2023-08-09T13:34:08.370437+0000","last_undegraded":"2023-08-09T13:34:08.370437+0000","last_fullsized":"2023-08-09T13:34:08.370437+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:34:01.052721+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:34:01.052721+0000","last_clean_scrub_stamp":"2023-08-09T13:34:01.052721+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T18:50:04.274374+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.3","version":"0'0","reported_seq":13,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-08-09T13:34:08.209248+0000","last_change":"2023-08-09T13:34:07.244416+0000","last_active":"2023-08-09T13:34:08.209248+0000","last_peered":"2023-08-09T13:34:08.209248+0000","last_clean":"2023-08-09T13:34:08.209248+0000","last_became_active":"2023-08-09T13:34:07.243294+0000","last_became_peered":"2023-08-09T13:34:07.243294+0000","last_unstale":"2023-08-09T13:34:08.209248+0000","last_undegraded":"2023-08-09T13:34:08.209248+0000","last_fullsized":"2023-08-09T13:34:08.209248+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:34:06.173547+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:34:06.173547+0000","last_clean_scrub_stamp":"2023-08-09T13:34:06.173547+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T21:35:05.575947+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,6,5],"acting":[0,1,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":17,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-08-09T13:34:08.209747+0000","last_change":"2023-08-09T13:34:03.151674+0000","last_active":"2023-08-09T13:34:08.209747+0000","last_peered":"2023-08-09T13:34:08.209747+0000","last_clean":"2023-08-09T13:34:08.209747+0000","last_became_active":"2023-08-09T13:34:03.151137+0000","last_became_peered":"2023-08-09T13:34:03.151137+0000","last_unstale":"2023-08-09T13:34:08.209747+0000","last_undegraded":"2023-08-09T13:34:08.209747+0000","last_fullsized":"2023-08-09T13:34:08.209747+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:34:02.064154+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:34:02.064154+0000","last_clean_scrub_stamp":"2023-08-09T13:34:02.064154+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T21:11:40.494329+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":25,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-08-09T13:34:08.212939+0000","last_change":"2023-08-09T13:33:56.401781+0000","last_active":"2023-08-09T13:34:08.212939+0000","last_peered":"2023-08-09T13:34:08.212939+0000","last_clean":"2023-08-09T13:34:08.212939+0000","last_became_active":"2023-08-09T13:33:56.400336+0000","last_became_peered":"2023-08-09T13:33:56.400336+0000","last_unstale":"2023-08-09T13:34:08.212939+0000","last_undegraded":"2023-08-09T13:34:08.212939+0000","last_fullsized":"2023-08-09T13:34:08.212939+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:33:54.845869+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:33:54.845869+0000","last_clean_scrub_stamp":"2023-08-09T13:33:54.845869+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T19:43:07.034435+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":21,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-08-09T13:34:08.372061+0000","last_change":"2023-08-09T13:34:02.106587+0000","last_active":"2023-08-09T13:34:08.372061+0000","last_peered":"2023-08-09T13:34:08.372061+0000","last_clean":"2023-08-09T13:34:08.372061+0000","last_became_active":"2023-08-09T13:34:02.106236+0000","last_became_peered":"2023-08-09T13:34:02.106236+0000","last_unstale":"2023-08-09T13:34:08.372061+0000","last_undegraded":"2023-08-09T13:34:08.372061+0000","last_fullsized":"2023-08-09T13:34:08.372061+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:34:01.052721+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:34:01.052721+0000","last_clean_scrub_stamp":"2023-08-09T13:34:01.052721+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T13:44:49.753930+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.2","version":"0'0","reported_seq":13,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-08-09T13:34:08.211286+0000","last_change":"2023-08-09T13:34:07.242738+0000","last_active":"2023-08-09T13:34:08.211286+0000","last_peered":"2023-08-09T13:34:08.211286+0000","last_clean":"2023-08-09T13:34:08.211286+0000","last_became_active":"2023-08-09T13:34:07.242314+0000","last_became_peered":"2023-08-09T13:34:07.242314+0000","last_unstale":"2023-08-09T13:34:08.211286+0000","last_undegraded":"2023-08-09T13:34:08.211286+0000","last_fullsized":"2023-08-09T13:34:08.211286+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:34:06.173547+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:34:06.173547+0000","last_clean_scrub_stamp":"2023-08-09T13:34:06.173547+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T18:00:37.387381+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,5,2],"acting":[6,0,5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-08-09T13:34:08.368126+0000","last_change":"2023-08-09T13:34:03.163805+0000","last_active":"2023-08-09T13:34:08.368126+0000","last_peered":"2023-08-09T13:34:08.368126+0000","last_clean":"2023-08-09T13:34:08.368126+0000","last_became_active":"2023-08-09T13:34:03.163407+0000","last_became_peered":"2023-08-09T13:34:03.163407+0000","last_unstale":"2023-08-09T13:34:08.368126+0000","last_undegraded":"2023-08-09T13:34:08.368126+0000","last_fullsized":"2023-08-09T13:34:08.368126+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:34:02.064154+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:34:02.064154+0000","last_clean_scrub_stamp":"2023-08-09T13:34:02.064154+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T23:33:49.011321+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":27,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-08-09T13:34:08.373438+0000","last_change":"2023-08-09T13:33:56.404236+0000","last_active":"2023-08-09T13:34:08.373438+0000","last_peered":"2023-08-09T13:34:08.373438+0000","last_clean":"2023-08-09T13:34:08.373438+0000","last_became_active":"2023-08-09T13:33:56.402485+0000","last_became_peered":"2023-08-09T13:33:56.402485+0000","last_unstale":"2023-08-09T13:34:08.373438+0000","last_undegraded":"2023-08-09T13:34:08.373438+0000","last_fullsized":"2023-08-09T13:34:08.373438+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:33:54.845869+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:33:54.845869+0000","last_clean_scrub_stamp":"2023-08-09T13:33:54.845869+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T18:59:14.817691+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.7","version":"19'2","reported_seq":23,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-08-09T13:34:08.369861+0000","last_change":"2023-08-09T13:34:02.108061+0000","last_active":"2023-08-09T13:34:08.369861+0000","last_peered":"2023-08-09T13:34:08.369861+0000","last_clean":"2023-08-09T13:34:08.369861+0000","last_became_active":"2023-08-09T13:34:02.107766+0000","last_became_peered":"2023-08-09T13:34:02.107766+0000","last_unstale":"2023-08-09T13:34:08.369861+0000","last_undegraded":"2023-08-09T13:34:08.369861+0000","last_fullsized":"2023-08-09T13:34:08.369861+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:34:01.052721+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:34:01.052721+0000","last_clean_scrub_stamp":"2023-08-09T13:34:01.052721+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T16:14:05.720686+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.1","version":"0'0","reported_seq":13,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-08-09T13:34:08.368770+0000","last_change":"2023-08-09T13:34:07.236332+0000","last_active":"2023-08-09T13:34:08.368770+0000","last_peered":"2023-08-09T13:34:08.368770+0000","last_clean":"2023-08-09T13:34:08.368770+0000","last_became_active":"2023-08-09T13:34:07.235923+0000","last_became_peered":"2023-08-09T13:34:07.235923+0000","last_unstale":"2023-08-09T13:34:08.368770+0000","last_undegraded":"2023-08-09T13:34:08.368770+0000","last_fullsized":"2023-08-09T13:34:08.368770+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:34:06.173547+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:34:06.173547+0000","last_clean_scrub_stamp":"2023-08-09T13:34:06.173547+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-11T01:07:37.438752+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,7,0],"acting":[4,3,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-08-09T13:34:08.369699+0000","last_change":"2023-08-09T13:34:03.149492+0000","last_active":"2023-08-09T13:34:08.369699+0000","last_peered":"2023-08-09T13:34:08.369699+0000","last_clean":"2023-08-09T13:34:08.369699+0000","last_became_active":"2023-08-09T13:34:03.148649+0000","last_became_peered":"2023-08-09T13:34:03.148649+0000","last_unstale":"2023-08-09T13:34:08.369699+0000","last_undegraded":"2023-08-09T13:34:08.369699+0000","last_fullsized":"2023-08-09T13:34:08.369699+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:34:02.064154+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:34:02.064154+0000","last_clean_scrub_stamp":"2023-08-09T13:34:02.064154+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T21:57:54.183369+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":27,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-08-09T13:34:08.370409+0000","last_change":"2023-08-09T13:33:55.889565+0000","last_active":"2023-08-09T13:34:08.370409+0000","last_peered":"2023-08-09T13:34:08.370409+0000","last_clean":"2023-08-09T13:34:08.370409+0000","last_became_active":"2023-08-09T13:33:55.889164+0000","last_became_peered":"2023-08-09T13:33:55.889164+0000","last_unstale":"2023-08-09T13:34:08.370409+0000","last_undegraded":"2023-08-09T13:34:08.370409+0000","last_fullsized":"2023-08-09T13:34:08.370409+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:33:54.845869+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:33:54.845869+0000","last_clean_scrub_stamp":"2023-08-09T13:33:54.845869+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-11T01:07:26.111074+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.6","version":"21'1","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-08-09T13:34:08.209027+0000","last_change":"2023-08-09T13:34:02.096482+0000","last_active":"2023-08-09T13:34:08.209027+0000","last_peered":"2023-08-09T13:34:08.209027+0000","last_clean":"2023-08-09T13:34:08.209027+0000","last_became_active":"2023-08-09T13:34:02.095287+0000","last_became_peered":"2023-08-09T13:34:02.095287+0000","last_unstale":"2023-08-09T13:34:08.209027+0000","last_undegraded":"2023-08-09T13:34:08.209027+0000","last_fullsized":"2023-08-09T13:34:08.209027+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:34:01.052721+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:34:01.052721+0000","last_clean_scrub_stamp":"2023-08-09T13:34:01.052721+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T18:36:38.900810+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.0","version":"0'0","reported_seq":13,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-08-09T13:34:08.367857+0000","last_change":"2023-08-09T13:34:07.244568+0000","last_active":"2023-08-09T13:34:08.367857+0000","last_peered":"2023-08-09T13:34:08.367857+0000","last_clean":"2023-08-09T13:34:08.367857+0000","last_became_active":"2023-08-09T13:34:07.243669+0000","last_became_peered":"2023-08-09T13:34:07.243669+0000","last_unstale":"2023-08-09T13:34:08.367857+0000","last_undegraded":"2023-08-09T13:34:08.367857+0000","last_fullsized":"2023-08-09T13:34:08.367857+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:34:06.173547+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:34:06.173547+0000","last_clean_scrub_stamp":"2023-08-09T13:34:06.173547+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T20:31:33.361620+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,4,0],"acting":[3,1,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.6","version":"0'0","reported_seq":17,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-08-09T13:34:08.208668+0000","last_change":"2023-08-09T13:34:03.139485+0000","last_active":"2023-08-09T13:34:08.208668+0000","last_peered":"2023-08-09T13:34:08.208668+0000","last_clean":"2023-08-09T13:34:08.208668+0000","last_became_active":"2023-08-09T13:34:03.139178+0000","last_became_peered":"2023-08-09T13:34:03.139178+0000","last_unstale":"2023-08-09T13:34:08.208668+0000","last_undegraded":"2023-08-09T13:34:08.208668+0000","last_fullsized":"2023-08-09T13:34:08.208668+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:34:02.064154+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:34:02.064154+0000","last_clean_scrub_stamp":"2023-08-09T13:34:02.064154+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T19:42:33.733326+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-08-09T13:34:08.207236+0000","last_change":"2023-08-09T13:34:02.110719+0000","last_active":"2023-08-09T13:34:08.207236+0000","last_peered":"2023-08-09T13:34:08.207236+0000","last_clean":"2023-08-09T13:34:08.207236+0000","last_became_active":"2023-08-09T13:34:02.110431+0000","last_became_peered":"2023-08-09T13:34:02.110431+0000","last_unstale":"2023-08-09T13:34:08.207236+0000","last_undegraded":"2023-08-09T13:34:08.207236+0000","last_fullsized":"2023-08-09T13:34:08.207236+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:34:01.052721+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:34:01.052721+0000","last_clean_scrub_stamp":"2023-08-09T13:34:01.052721+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T23:24:07.266385+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.0","version":"16'117","reported_seq":175,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-08-09T13:34:08.368906+0000","last_change":"2023-08-09T13:33:55.886345+0000","last_active":"2023-08-09T13:34:08.368906+0000","last_peered":"2023-08-09T13:34:08.368906+0000","last_clean":"2023-08-09T13:34:08.368906+0000","last_became_active":"2023-08-09T13:33:55.885781+0000","last_became_peered":"2023-08-09T13:33:55.885781+0000","last_unstale":"2023-08-09T13:34:08.368906+0000","last_undegraded":"2023-08-09T13:34:08.368906+0000","last_fullsized":"2023-08-09T13:34:08.368906+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:33:54.845869+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:33:54.845869+0000","last_clean_scrub_stamp":"2023-08-09T13:33:54.845869+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T18:49:58.067523+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.3","version":"0'0","reported_seq":27,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-08-09T13:34:08.370928+0000","last_change":"2023-08-09T13:33:55.886749+0000","last_active":"2023-08-09T13:34:08.370928+0000","last_peered":"2023-08-09T13:34:08.370928+0000","last_clean":"2023-08-09T13:34:08.370928+0000","last_became_active":"2023-08-09T13:33:55.886116+0000","last_became_peered":"2023-08-09T13:33:55.886116+0000","last_unstale":"2023-08-09T13:34:08.370928+0000","last_undegraded":"2023-08-09T13:34:08.370928+0000","last_fullsized":"2023-08-09T13:34:08.370928+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:33:54.845869+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:33:54.845869+0000","last_clean_scrub_stamp":"2023-08-09T13:33:54.845869+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T17:46:15.820922+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.7","version":"0'0","reported_seq":13,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-08-09T13:34:08.371420+0000","last_change":"2023-08-09T13:34:07.235784+0000","last_active":"2023-08-09T13:34:08.371420+0000","last_peered":"2023-08-09T13:34:08.371420+0000","last_clean":"2023-08-09T13:34:08.371420+0000","last_became_active":"2023-08-09T13:34:07.235282+0000","last_became_peered":"2023-08-09T13:34:07.235282+0000","last_unstale":"2023-08-09T13:34:08.371420+0000","last_undegraded":"2023-08-09T13:34:08.371420+0000","last_fullsized":"2023-08-09T13:34:08.371420+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:34:06.173547+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:34:06.173547+0000","last_clean_scrub_stamp":"2023-08-09T13:34:06.173547+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T23:52:31.847779+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,0,4],"acting":[5,1,0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.7","version":"0'0","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-08-09T13:34:08.372407+0000","last_change":"2023-08-09T13:34:03.149568+0000","last_active":"2023-08-09T13:34:08.372407+0000","last_peered":"2023-08-09T13:34:08.372407+0000","last_clean":"2023-08-09T13:34:08.372407+0000","last_became_active":"2023-08-09T13:34:03.148937+0000","last_became_peered":"2023-08-09T13:34:03.148937+0000","last_unstale":"2023-08-09T13:34:08.372407+0000","last_undegraded":"2023-08-09T13:34:08.372407+0000","last_fullsized":"2023-08-09T13:34:08.372407+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:34:02.064154+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:34:02.064154+0000","last_clean_scrub_stamp":"2023-08-09T13:34:02.064154+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T23:34:54.506258+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":21,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-08-09T13:34:08.367679+0000","last_change":"2023-08-09T13:34:02.102795+0000","last_active":"2023-08-09T13:34:08.367679+0000","last_peered":"2023-08-09T13:34:08.367679+0000","last_clean":"2023-08-09T13:34:08.367679+0000","last_became_active":"2023-08-09T13:34:02.101883+0000","last_became_peered":"2023-08-09T13:34:02.101883+0000","last_unstale":"2023-08-09T13:34:08.367679+0000","last_undegraded":"2023-08-09T13:34:08.367679+0000","last_fullsized":"2023-08-09T13:34:08.367679+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:34:01.052721+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:34:01.052721+0000","last_clean_scrub_stamp":"2023-08-09T13:34:01.052721+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T19:22:07.585850+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.2","version":"0'0","reported_seq":25,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-08-09T13:34:08.369310+0000","last_change":"2023-08-09T13:33:56.402794+0000","last_active":"2023-08-09T13:34:08.369310+0000","last_peered":"2023-08-09T13:34:08.369310+0000","last_clean":"2023-08-09T13:34:08.369310+0000","last_became_active":"2023-08-09T13:33:56.402038+0000","last_became_peered":"2023-08-09T13:33:56.402038+0000","last_unstale":"2023-08-09T13:34:08.369310+0000","last_undegraded":"2023-08-09T13:34:08.369310+0000","last_fullsized":"2023-08-09T13:34:08.369310+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:33:54.845869+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:33:54.845869+0000","last_clean_scrub_stamp":"2023-08-09T13:33:54.845869+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T18:07:24.486884+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.6","version":"0'0","reported_seq":13,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-08-09T13:34:08.212013+0000","last_change":"2023-08-09T13:34:07.244168+0000","last_active":"2023-08-09T13:34:08.212013+0000","last_peered":"2023-08-09T13:34:08.212013+0000","last_clean":"2023-08-09T13:34:08.212013+0000","last_became_active":"2023-08-09T13:34:07.241903+0000","last_became_peered":"2023-08-09T13:34:07.241903+0000","last_unstale":"2023-08-09T13:34:08.212013+0000","last_undegraded":"2023-08-09T13:34:08.212013+0000","last_fullsized":"2023-08-09T13:34:08.212013+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:34:06.173547+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:34:06.173547+0000","last_clean_scrub_stamp":"2023-08-09T13:34:06.173547+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-11T00:23:22.192340+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,1,7],"acting":[2,5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-08-09T13:34:08.371697+0000","last_change":"2023-08-09T13:34:03.153039+0000","last_active":"2023-08-09T13:34:08.371697+0000","last_peered":"2023-08-09T13:34:08.371697+0000","last_clean":"2023-08-09T13:34:08.371697+0000","last_became_active":"2023-08-09T13:34:03.152401+0000","last_became_peered":"2023-08-09T13:34:03.152401+0000","last_unstale":"2023-08-09T13:34:08.371697+0000","last_undegraded":"2023-08-09T13:34:08.371697+0000","last_fullsized":"2023-08-09T13:34:08.371697+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:34:02.064154+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:34:02.064154+0000","last_clean_scrub_stamp":"2023-08-09T13:34:02.064154+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T21:18:19.551549+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":21,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-08-09T13:34:08.372303+0000","last_change":"2023-08-09T13:34:02.108657+0000","last_active":"2023-08-09T13:34:08.372303+0000","last_peered":"2023-08-09T13:34:08.372303+0000","last_clean":"2023-08-09T13:34:08.372303+0000","last_became_active":"2023-08-09T13:34:02.105481+0000","last_became_peered":"2023-08-09T13:34:02.105481+0000","last_unstale":"2023-08-09T13:34:08.372303+0000","last_undegraded":"2023-08-09T13:34:08.372303+0000","last_fullsized":"2023-08-09T13:34:08.372303+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:34:01.052721+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:34:01.052721+0000","last_clean_scrub_stamp":"2023-08-09T13:34:01.052721+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T22:51:18.775151+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":25,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-08-09T13:34:08.368026+0000","last_change":"2023-08-09T13:33:56.401123+0000","last_active":"2023-08-09T13:34:08.368026+0000","last_peered":"2023-08-09T13:34:08.368026+0000","last_clean":"2023-08-09T13:34:08.368026+0000","last_became_active":"2023-08-09T13:33:56.399933+0000","last_became_peered":"2023-08-09T13:33:56.399933+0000","last_unstale":"2023-08-09T13:34:08.368026+0000","last_undegraded":"2023-08-09T13:34:08.368026+0000","last_fullsized":"2023-08-09T13:34:08.368026+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:33:54.845869+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:33:54.845869+0000","last_clean_scrub_stamp":"2023-08-09T13:33:54.845869+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T23:24:01.059533+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.5","version":"0'0","reported_seq":13,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-08-09T13:34:08.208281+0000","last_change":"2023-08-09T13:34:07.235418+0000","last_active":"2023-08-09T13:34:08.208281+0000","last_peered":"2023-08-09T13:34:08.208281+0000","last_clean":"2023-08-09T13:34:08.208281+0000","last_became_active":"2023-08-09T13:34:07.234889+0000","last_became_peered":"2023-08-09T13:34:07.234889+0000","last_unstale":"2023-08-09T13:34:08.208281+0000","last_undegraded":"2023-08-09T13:34:08.208281+0000","last_fullsized":"2023-08-09T13:34:08.208281+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:34:06.173547+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:34:06.173547+0000","last_clean_scrub_stamp":"2023-08-09T13:34:06.173547+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T20:46:19.756813+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,4,6],"acting":[0,1,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.5","version":"0'0","reported_seq":17,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-08-09T13:34:08.212552+0000","last_change":"2023-08-09T13:34:03.159726+0000","last_active":"2023-08-09T13:34:08.212552+0000","last_peered":"2023-08-09T13:34:08.212552+0000","last_clean":"2023-08-09T13:34:08.212552+0000","last_became_active":"2023-08-09T13:34:03.159238+0000","last_became_peered":"2023-08-09T13:34:03.159238+0000","last_unstale":"2023-08-09T13:34:08.212552+0000","last_undegraded":"2023-08-09T13:34:08.212552+0000","last_fullsized":"2023-08-09T13:34:08.212552+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:34:02.064154+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:34:02.064154+0000","last_clean_scrub_stamp":"2023-08-09T13:34:02.064154+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T17:32:16.845908+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":21,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-08-09T13:34:08.368904+0000","last_change":"2023-08-09T13:34:02.114840+0000","last_active":"2023-08-09T13:34:08.368904+0000","last_peered":"2023-08-09T13:34:08.368904+0000","last_clean":"2023-08-09T13:34:08.368904+0000","last_became_active":"2023-08-09T13:34:02.114547+0000","last_became_peered":"2023-08-09T13:34:02.114547+0000","last_unstale":"2023-08-09T13:34:08.368904+0000","last_undegraded":"2023-08-09T13:34:08.368904+0000","last_fullsized":"2023-08-09T13:34:08.368904+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:34:01.052721+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:34:01.052721+0000","last_clean_scrub_stamp":"2023-08-09T13:34:01.052721+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T13:43:19.071421+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":25,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-08-09T13:34:08.369269+0000","last_change":"2023-08-09T13:33:56.401406+0000","last_active":"2023-08-09T13:34:08.369269+0000","last_peered":"2023-08-09T13:34:08.369269+0000","last_clean":"2023-08-09T13:34:08.369269+0000","last_became_active":"2023-08-09T13:33:56.400487+0000","last_became_peered":"2023-08-09T13:33:56.400487+0000","last_unstale":"2023-08-09T13:34:08.369269+0000","last_undegraded":"2023-08-09T13:34:08.369269+0000","last_fullsized":"2023-08-09T13:34:08.369269+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:33:54.845869+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:33:54.845869+0000","last_clean_scrub_stamp":"2023-08-09T13:33:54.845869+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T23:52:20.520101+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.4","version":"0'0","reported_seq":13,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-08-09T13:34:08.370501+0000","last_change":"2023-08-09T13:34:07.249799+0000","last_active":"2023-08-09T13:34:08.370501+0000","last_peered":"2023-08-09T13:34:08.370501+0000","last_clean":"2023-08-09T13:34:08.370501+0000","last_became_active":"2023-08-09T13:34:07.249286+0000","last_became_peered":"2023-08-09T13:34:07.249286+0000","last_unstale":"2023-08-09T13:34:08.370501+0000","last_undegraded":"2023-08-09T13:34:08.370501+0000","last_fullsized":"2023-08-09T13:34:08.370501+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:34:06.173547+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:34:06.173547+0000","last_clean_scrub_stamp":"2023-08-09T13:34:06.173547+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T15:43:48.184972+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2,5,3],"acting":[7,2,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.f","version":"0'0","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-08-09T13:34:08.372421+0000","last_change":"2023-08-09T13:34:03.151508+0000","last_active":"2023-08-09T13:34:08.372421+0000","last_peered":"2023-08-09T13:34:08.372421+0000","last_clean":"2023-08-09T13:34:08.372421+0000","last_became_active":"2023-08-09T13:34:03.149550+0000","last_became_peered":"2023-08-09T13:34:03.149550+0000","last_unstale":"2023-08-09T13:34:08.372421+0000","last_undegraded":"2023-08-09T13:34:08.372421+0000","last_fullsized":"2023-08-09T13:34:08.372421+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:34:02.064154+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:34:02.064154+0000","last_clean_scrub_stamp":"2023-08-09T13:34:02.064154+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T22:51:11.201456+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":21,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-08-09T13:34:08.368991+0000","last_change":"2023-08-09T13:34:02.095946+0000","last_active":"2023-08-09T13:34:08.368991+0000","last_peered":"2023-08-09T13:34:08.368991+0000","last_clean":"2023-08-09T13:34:08.368991+0000","last_became_active":"2023-08-09T13:34:02.095325+0000","last_became_peered":"2023-08-09T13:34:02.095325+0000","last_unstale":"2023-08-09T13:34:08.368991+0000","last_undegraded":"2023-08-09T13:34:08.368991+0000","last_fullsized":"2023-08-09T13:34:08.368991+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:34:01.052721+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:34:01.052721+0000","last_clean_scrub_stamp":"2023-08-09T13:34:01.052721+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-11T00:06:29.000035+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.e","version":"0'0","reported_seq":13,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-08-09T13:34:08.373803+0000","last_change":"2023-08-09T13:34:07.232690+0000","last_active":"2023-08-09T13:34:08.373803+0000","last_peered":"2023-08-09T13:34:08.373803+0000","last_clean":"2023-08-09T13:34:08.373803+0000","last_became_active":"2023-08-09T13:34:07.232326+0000","last_became_peered":"2023-08-09T13:34:07.232326+0000","last_unstale":"2023-08-09T13:34:08.373803+0000","last_undegraded":"2023-08-09T13:34:08.373803+0000","last_fullsized":"2023-08-09T13:34:08.373803+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:34:06.173547+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:34:06.173547+0000","last_clean_scrub_stamp":"2023-08-09T13:34:06.173547+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T13:37:38.237696+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,0,2],"acting":[4,5,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.e","version":"0'0","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-08-09T13:34:08.373562+0000","last_change":"2023-08-09T13:34:03.149791+0000","last_active":"2023-08-09T13:34:08.373562+0000","last_peered":"2023-08-09T13:34:08.373562+0000","last_clean":"2023-08-09T13:34:08.373562+0000","last_became_active":"2023-08-09T13:34:03.149525+0000","last_became_peered":"2023-08-09T13:34:03.149525+0000","last_unstale":"2023-08-09T13:34:08.373562+0000","last_undegraded":"2023-08-09T13:34:08.373562+0000","last_fullsized":"2023-08-09T13:34:08.373562+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:34:02.064154+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:34:02.064154+0000","last_clean_scrub_stamp":"2023-08-09T13:34:02.064154+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T14:53:59.337109+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":21,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-08-09T13:34:08.369963+0000","last_change":"2023-08-09T13:34:02.105359+0000","last_active":"2023-08-09T13:34:08.369963+0000","last_peered":"2023-08-09T13:34:08.369963+0000","last_clean":"2023-08-09T13:34:08.369963+0000","last_became_active":"2023-08-09T13:34:02.105027+0000","last_became_peered":"2023-08-09T13:34:02.105027+0000","last_unstale":"2023-08-09T13:34:08.369963+0000","last_undegraded":"2023-08-09T13:34:08.369963+0000","last_fullsized":"2023-08-09T13:34:08.369963+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:34:01.052721+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:34:01.052721+0000","last_clean_scrub_stamp":"2023-08-09T13:34:01.052721+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T13:45:26.240878+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.f","version":"0'0","reported_seq":13,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-08-09T13:34:08.371141+0000","last_change":"2023-08-09T13:34:07.239625+0000","last_active":"2023-08-09T13:34:08.371141+0000","last_peered":"2023-08-09T13:34:08.371141+0000","last_clean":"2023-08-09T13:34:08.371141+0000","last_became_active":"2023-08-09T13:34:07.239002+0000","last_became_peered":"2023-08-09T13:34:07.239002+0000","last_unstale":"2023-08-09T13:34:08.371141+0000","last_undegraded":"2023-08-09T13:34:08.371141+0000","last_fullsized":"2023-08-09T13:34:08.371141+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:34:06.173547+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:34:06.173547+0000","last_clean_scrub_stamp":"2023-08-09T13:34:06.173547+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T21:11:44.603723+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,6,2],"acting":[5,4,6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.d","version":"0'0","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-08-09T13:34:08.373090+0000","last_change":"2023-08-09T13:34:03.152579+0000","last_active":"2023-08-09T13:34:08.373090+0000","last_peered":"2023-08-09T13:34:08.373090+0000","last_clean":"2023-08-09T13:34:08.373090+0000","last_became_active":"2023-08-09T13:34:03.152287+0000","last_became_peered":"2023-08-09T13:34:03.152287+0000","last_unstale":"2023-08-09T13:34:08.373090+0000","last_undegraded":"2023-08-09T13:34:08.373090+0000","last_fullsized":"2023-08-09T13:34:08.373090+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:34:02.064154+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:34:02.064154+0000","last_clean_scrub_stamp":"2023-08-09T13:34:02.064154+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T17:37:51.284066+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-08-09T13:34:08.211685+0000","last_change":"2023-08-09T13:34:02.109847+0000","last_active":"2023-08-09T13:34:08.211685+0000","last_peered":"2023-08-09T13:34:08.211685+0000","last_clean":"2023-08-09T13:34:08.211685+0000","last_became_active":"2023-08-09T13:34:02.109516+0000","last_became_peered":"2023-08-09T13:34:02.109516+0000","last_unstale":"2023-08-09T13:34:08.211685+0000","last_undegraded":"2023-08-09T13:34:08.211685+0000","last_fullsized":"2023-08-09T13:34:08.211685+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:34:01.052721+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:34:01.052721+0000","last_clean_scrub_stamp":"2023-08-09T13:34:01.052721+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T22:40:21.488794+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.c","version":"0'0","reported_seq":13,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-08-09T13:34:08.371235+0000","last_change":"2023-08-09T13:34:07.231138+0000","last_active":"2023-08-09T13:34:08.371235+0000","last_peered":"2023-08-09T13:34:08.371235+0000","last_clean":"2023-08-09T13:34:08.371235+0000","last_became_active":"2023-08-09T13:34:07.230741+0000","last_became_peered":"2023-08-09T13:34:07.230741+0000","last_unstale":"2023-08-09T13:34:08.371235+0000","last_undegraded":"2023-08-09T13:34:08.371235+0000","last_fullsized":"2023-08-09T13:34:08.371235+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:34:06.173547+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:34:06.173547+0000","last_clean_scrub_stamp":"2023-08-09T13:34:06.173547+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T21:11:44.603723+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4,0,6],"acting":[1,4,0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.c","version":"0'0","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-08-09T13:34:08.372932+0000","last_change":"2023-08-09T13:34:03.159095+0000","last_active":"2023-08-09T13:34:08.372932+0000","last_peered":"2023-08-09T13:34:08.372932+0000","last_clean":"2023-08-09T13:34:08.372932+0000","last_became_active":"2023-08-09T13:34:03.158807+0000","last_became_peered":"2023-08-09T13:34:03.158807+0000","last_unstale":"2023-08-09T13:34:08.372932+0000","last_undegraded":"2023-08-09T13:34:08.372932+0000","last_fullsized":"2023-08-09T13:34:08.372932+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:34:02.064154+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:34:02.064154+0000","last_clean_scrub_stamp":"2023-08-09T13:34:02.064154+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T21:06:37.115870+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.b","version":"19'2","reported_seq":23,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-08-09T13:34:08.369938+0000","last_change":"2023-08-09T13:34:02.103259+0000","last_active":"2023-08-09T13:34:08.369938+0000","last_peered":"2023-08-09T13:34:08.369938+0000","last_clean":"2023-08-09T13:34:08.369938+0000","last_became_active":"2023-08-09T13:34:02.102644+0000","last_became_peered":"2023-08-09T13:34:02.102644+0000","last_unstale":"2023-08-09T13:34:08.369938+0000","last_undegraded":"2023-08-09T13:34:08.369938+0000","last_fullsized":"2023-08-09T13:34:08.369938+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:34:01.052721+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:34:01.052721+0000","last_clean_scrub_stamp":"2023-08-09T13:34:01.052721+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T19:48:11.160512+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.d","version":"0'0","reported_seq":13,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-08-09T13:34:08.211821+0000","last_change":"2023-08-09T13:34:07.242458+0000","last_active":"2023-08-09T13:34:08.211821+0000","last_peered":"2023-08-09T13:34:08.211821+0000","last_clean":"2023-08-09T13:34:08.211821+0000","last_became_active":"2023-08-09T13:34:07.241393+0000","last_became_peered":"2023-08-09T13:34:07.241393+0000","last_unstale":"2023-08-09T13:34:08.211821+0000","last_undegraded":"2023-08-09T13:34:08.211821+0000","last_fullsized":"2023-08-09T13:34:08.211821+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:34:06.173547+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:34:06.173547+0000","last_clean_scrub_stamp":"2023-08-09T13:34:06.173547+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T19:00:12.926085+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7,5,6],"acting":[2,7,5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.b","version":"0'0","reported_seq":17,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-08-09T13:34:08.209500+0000","last_change":"2023-08-09T13:34:03.155061+0000","last_active":"2023-08-09T13:34:08.209500+0000","last_peered":"2023-08-09T13:34:08.209500+0000","last_clean":"2023-08-09T13:34:08.209500+0000","last_became_active":"2023-08-09T13:34:03.154792+0000","last_became_peered":"2023-08-09T13:34:03.154792+0000","last_unstale":"2023-08-09T13:34:08.209500+0000","last_undegraded":"2023-08-09T13:34:08.209500+0000","last_fullsized":"2023-08-09T13:34:08.209500+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:34:02.064154+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:34:02.064154+0000","last_clean_scrub_stamp":"2023-08-09T13:34:02.064154+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T18:50:05.285807+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-08-09T13:34:08.369085+0000","last_change":"2023-08-09T13:34:02.111888+0000","last_active":"2023-08-09T13:34:08.369085+0000","last_peered":"2023-08-09T13:34:08.369085+0000","last_clean":"2023-08-09T13:34:08.369085+0000","last_became_active":"2023-08-09T13:34:02.111588+0000","last_became_peered":"2023-08-09T13:34:02.111588+0000","last_unstale":"2023-08-09T13:34:08.369085+0000","last_undegraded":"2023-08-09T13:34:08.369085+0000","last_fullsized":"2023-08-09T13:34:08.369085+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:34:01.052721+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:34:01.052721+0000","last_clean_scrub_stamp":"2023-08-09T13:34:01.052721+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T15:53:24.295815+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.a","version":"0'0","reported_seq":13,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-08-09T13:34:08.211349+0000","last_change":"2023-08-09T13:34:07.241400+0000","last_active":"2023-08-09T13:34:08.211349+0000","last_peered":"2023-08-09T13:34:08.211349+0000","last_clean":"2023-08-09T13:34:08.211349+0000","last_became_active":"2023-08-09T13:34:07.240945+0000","last_became_peered":"2023-08-09T13:34:07.240945+0000","last_unstale":"2023-08-09T13:34:08.211349+0000","last_undegraded":"2023-08-09T13:34:08.211349+0000","last_fullsized":"2023-08-09T13:34:08.211349+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:34:06.173547+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:34:06.173547+0000","last_clean_scrub_stamp":"2023-08-09T13:34:06.173547+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T20:24:07.327648+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4,3,6],"acting":[2,4,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":17,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-08-09T13:34:08.211229+0000","last_change":"2023-08-09T13:34:03.159173+0000","last_active":"2023-08-09T13:34:08.211229+0000","last_peered":"2023-08-09T13:34:08.211229+0000","last_clean":"2023-08-09T13:34:08.211229+0000","last_became_active":"2023-08-09T13:34:03.157726+0000","last_became_peered":"2023-08-09T13:34:03.157726+0000","last_unstale":"2023-08-09T13:34:08.211229+0000","last_undegraded":"2023-08-09T13:34:08.211229+0000","last_fullsized":"2023-08-09T13:34:08.211229+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:34:02.064154+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:34:02.064154+0000","last_clean_scrub_stamp":"2023-08-09T13:34:02.064154+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T18:49:41.829768+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.d","version":"19'1","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-08-09T13:34:08.371386+0000","last_change":"2023-08-09T13:34:02.115845+0000","last_active":"2023-08-09T13:34:08.371386+0000","last_peered":"2023-08-09T13:34:08.371386+0000","last_clean":"2023-08-09T13:34:08.371386+0000","last_became_active":"2023-08-09T13:34:02.115519+0000","last_became_peered":"2023-08-09T13:34:02.115519+0000","last_unstale":"2023-08-09T13:34:08.371386+0000","last_undegraded":"2023-08-09T13:34:08.371386+0000","last_fullsized":"2023-08-09T13:34:08.371386+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:34:01.052721+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:34:01.052721+0000","last_clean_scrub_stamp":"2023-08-09T13:34:01.052721+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T22:00:42.967697+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.b","version":"0'0","reported_seq":13,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-08-09T13:34:08.211258+0000","last_change":"2023-08-09T13:34:07.245045+0000","last_active":"2023-08-09T13:34:08.211258+0000","last_peered":"2023-08-09T13:34:08.211258+0000","last_clean":"2023-08-09T13:34:08.211258+0000","last_became_active":"2023-08-09T13:34:07.242923+0000","last_became_peered":"2023-08-09T13:34:07.242923+0000","last_unstale":"2023-08-09T13:34:08.211258+0000","last_undegraded":"2023-08-09T13:34:08.211258+0000","last_fullsized":"2023-08-09T13:34:08.211258+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:34:06.173547+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:34:06.173547+0000","last_clean_scrub_stamp":"2023-08-09T13:34:06.173547+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T20:20:56.644233+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,5,6],"acting":[2,0,5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-08-09T13:34:08.369085+0000","last_change":"2023-08-09T13:34:03.157124+0000","last_active":"2023-08-09T13:34:08.369085+0000","last_peered":"2023-08-09T13:34:08.369085+0000","last_clean":"2023-08-09T13:34:08.369085+0000","last_became_active":"2023-08-09T13:34:03.156039+0000","last_became_peered":"2023-08-09T13:34:03.156039+0000","last_unstale":"2023-08-09T13:34:08.369085+0000","last_undegraded":"2023-08-09T13:34:08.369085+0000","last_fullsized":"2023-08-09T13:34:08.369085+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:34:02.064154+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:34:02.064154+0000","last_clean_scrub_stamp":"2023-08-09T13:34:02.064154+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T17:18:45.337355+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.e","version":"19'2","reported_seq":21,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-08-09T13:34:08.371732+0000","last_change":"2023-08-09T13:34:02.116789+0000","last_active":"2023-08-09T13:34:08.371732+0000","last_peered":"2023-08-09T13:34:08.371732+0000","last_clean":"2023-08-09T13:34:08.371732+0000","last_became_active":"2023-08-09T13:34:02.116543+0000","last_became_peered":"2023-08-09T13:34:02.116543+0000","last_unstale":"2023-08-09T13:34:08.371732+0000","last_undegraded":"2023-08-09T13:34:08.371732+0000","last_fullsized":"2023-08-09T13:34:08.371732+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:34:01.052721+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:34:01.052721+0000","last_clean_scrub_stamp":"2023-08-09T13:34:01.052721+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T21:52:55.252712+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.8","version":"0'0","reported_seq":13,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-08-09T13:34:08.211204+0000","last_change":"2023-08-09T13:34:07.241851+0000","last_active":"2023-08-09T13:34:08.211204+0000","last_peered":"2023-08-09T13:34:08.211204+0000","last_clean":"2023-08-09T13:34:08.211204+0000","last_became_active":"2023-08-09T13:34:07.241329+0000","last_became_peered":"2023-08-09T13:34:07.241329+0000","last_unstale":"2023-08-09T13:34:08.211204+0000","last_undegraded":"2023-08-09T13:34:08.211204+0000","last_fullsized":"2023-08-09T13:34:08.211204+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:34:06.173547+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:34:06.173547+0000","last_clean_scrub_stamp":"2023-08-09T13:34:06.173547+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T20:54:41.878672+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,1,5],"acting":[2,0,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":17,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-08-09T13:34:08.368062+0000","last_change":"2023-08-09T13:34:03.158365+0000","last_active":"2023-08-09T13:34:08.368062+0000","last_peered":"2023-08-09T13:34:08.368062+0000","last_clean":"2023-08-09T13:34:08.368062+0000","last_became_active":"2023-08-09T13:34:03.158042+0000","last_became_peered":"2023-08-09T13:34:03.158042+0000","last_unstale":"2023-08-09T13:34:08.368062+0000","last_undegraded":"2023-08-09T13:34:08.368062+0000","last_fullsized":"2023-08-09T13:34:08.368062+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:34:02.064154+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:34:02.064154+0000","last_clean_scrub_stamp":"2023-08-09T13:34:02.064154+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T22:36:24.124090+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.f","version":"19'1","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-08-09T13:34:08.373105+0000","last_change":"2023-08-09T13:34:02.116130+0000","last_active":"2023-08-09T13:34:08.373105+0000","last_peered":"2023-08-09T13:34:08.373105+0000","last_clean":"2023-08-09T13:34:08.373105+0000","last_became_active":"2023-08-09T13:34:02.115758+0000","last_became_peered":"2023-08-09T13:34:02.115758+0000","last_unstale":"2023-08-09T13:34:08.373105+0000","last_undegraded":"2023-08-09T13:34:08.373105+0000","last_fullsized":"2023-08-09T13:34:08.373105+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:34:01.052721+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:34:01.052721+0000","last_clean_scrub_stamp":"2023-08-09T13:34:01.052721+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-11T00:08:17.437427+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.9","version":"0'0","reported_seq":13,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-08-09T13:34:08.369072+0000","last_change":"2023-08-09T13:34:07.241507+0000","last_active":"2023-08-09T13:34:08.369072+0000","last_peered":"2023-08-09T13:34:08.369072+0000","last_clean":"2023-08-09T13:34:08.369072+0000","last_became_active":"2023-08-09T13:34:07.240810+0000","last_became_peered":"2023-08-09T13:34:07.240810+0000","last_unstale":"2023-08-09T13:34:08.369072+0000","last_undegraded":"2023-08-09T13:34:08.369072+0000","last_fullsized":"2023-08-09T13:34:08.369072+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:34:06.173547+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:34:06.173547+0000","last_clean_scrub_stamp":"2023-08-09T13:34:06.173547+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T17:46:27.148600+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,3,4],"acting":[7,6,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.16","version":"0'0","reported_seq":13,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-08-09T13:34:08.370805+0000","last_change":"2023-08-09T13:34:07.238069+0000","last_active":"2023-08-09T13:34:08.370805+0000","last_peered":"2023-08-09T13:34:08.370805+0000","last_clean":"2023-08-09T13:34:08.370805+0000","last_became_active":"2023-08-09T13:34:07.237560+0000","last_became_peered":"2023-08-09T13:34:07.237560+0000","last_unstale":"2023-08-09T13:34:08.370805+0000","last_undegraded":"2023-08-09T13:34:08.370805+0000","last_fullsized":"2023-08-09T13:34:08.370805+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:34:06.173547+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:34:06.173547+0000","last_clean_scrub_stamp":"2023-08-09T13:34:06.173547+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T17:46:27.148600+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,1,6],"acting":[5,3,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.10","version":"19'8","reported_seq":27,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-08-09T13:34:08.211427+0000","last_change":"2023-08-09T13:34:02.103042+0000","last_active":"2023-08-09T13:34:08.211427+0000","last_peered":"2023-08-09T13:34:08.211427+0000","last_clean":"2023-08-09T13:34:08.211427+0000","last_became_active":"2023-08-09T13:34:02.102730+0000","last_became_peered":"2023-08-09T13:34:02.102730+0000","last_unstale":"2023-08-09T13:34:08.211427+0000","last_undegraded":"2023-08-09T13:34:08.211427+0000","last_fullsized":"2023-08-09T13:34:08.211427+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:34:01.052721+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:34:01.052721+0000","last_clean_scrub_stamp":"2023-08-09T13:34:01.052721+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T19:01:23.853811+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-08-09T13:34:08.369924+0000","last_change":"2023-08-09T13:34:03.163917+0000","last_active":"2023-08-09T13:34:08.369924+0000","last_peered":"2023-08-09T13:34:08.369924+0000","last_clean":"2023-08-09T13:34:08.369924+0000","last_became_active":"2023-08-09T13:34:03.163598+0000","last_became_peered":"2023-08-09T13:34:03.163598+0000","last_unstale":"2023-08-09T13:34:08.369924+0000","last_undegraded":"2023-08-09T13:34:08.369924+0000","last_fullsized":"2023-08-09T13:34:08.369924+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:34:02.064154+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:34:02.064154+0000","last_clean_scrub_stamp":"2023-08-09T13:34:02.064154+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T20:01:36.200073+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.17","version":"0'0","reported_seq":13,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-08-09T13:34:08.369281+0000","last_change":"2023-08-09T13:34:07.249196+0000","last_active":"2023-08-09T13:34:08.369281+0000","last_peered":"2023-08-09T13:34:08.369281+0000","last_clean":"2023-08-09T13:34:08.369281+0000","last_became_active":"2023-08-09T13:34:07.248758+0000","last_became_peered":"2023-08-09T13:34:07.248758+0000","last_unstale":"2023-08-09T13:34:08.369281+0000","last_undegraded":"2023-08-09T13:34:08.369281+0000","last_fullsized":"2023-08-09T13:34:08.369281+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:34:06.173547+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:34:06.173547+0000","last_clean_scrub_stamp":"2023-08-09T13:34:06.173547+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T19:50:51.608237+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,7,0],"acting":[3,1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-08-09T13:34:08.369394+0000","last_change":"2023-08-09T13:34:02.112054+0000","last_active":"2023-08-09T13:34:08.369394+0000","last_peered":"2023-08-09T13:34:08.369394+0000","last_clean":"2023-08-09T13:34:08.369394+0000","last_became_active":"2023-08-09T13:34:02.111753+0000","last_became_peered":"2023-08-09T13:34:02.111753+0000","last_unstale":"2023-08-09T13:34:08.369394+0000","last_undegraded":"2023-08-09T13:34:08.369394+0000","last_fullsized":"2023-08-09T13:34:08.369394+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:34:01.052721+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:34:01.052721+0000","last_clean_scrub_stamp":"2023-08-09T13:34:01.052721+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T17:08:06.492548+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":17,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-08-09T13:34:08.208919+0000","last_change":"2023-08-09T13:34:03.151791+0000","last_active":"2023-08-09T13:34:08.208919+0000","last_peered":"2023-08-09T13:34:08.208919+0000","last_clean":"2023-08-09T13:34:08.208919+0000","last_became_active":"2023-08-09T13:34:03.151265+0000","last_became_peered":"2023-08-09T13:34:03.151265+0000","last_unstale":"2023-08-09T13:34:08.208919+0000","last_undegraded":"2023-08-09T13:34:08.208919+0000","last_fullsized":"2023-08-09T13:34:08.208919+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:34:02.064154+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:34:02.064154+0000","last_clean_scrub_stamp":"2023-08-09T13:34:02.064154+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T19:22:08.597283+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-08-09T13:34:08.207862+0000","last_change":"2023-08-09T13:34:02.109334+0000","last_active":"2023-08-09T13:34:08.207862+0000","last_peered":"2023-08-09T13:34:08.207862+0000","last_clean":"2023-08-09T13:34:08.207862+0000","last_became_active":"2023-08-09T13:34:02.108390+0000","last_became_peered":"2023-08-09T13:34:02.108390+0000","last_unstale":"2023-08-09T13:34:08.207862+0000","last_undegraded":"2023-08-09T13:34:08.207862+0000","last_fullsized":"2023-08-09T13:34:08.207862+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:34:01.052721+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:34:01.052721+0000","last_clean_scrub_stamp":"2023-08-09T13:34:01.052721+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T22:36:23.112658+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.15","version":"0'0","reported_seq":17,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-08-09T13:34:08.370140+0000","last_change":"2023-08-09T13:34:03.150611+0000","last_active":"2023-08-09T13:34:08.370140+0000","last_peered":"2023-08-09T13:34:08.370140+0000","last_clean":"2023-08-09T13:34:08.370140+0000","last_became_active":"2023-08-09T13:34:03.150016+0000","last_became_peered":"2023-08-09T13:34:03.150016+0000","last_unstale":"2023-08-09T13:34:08.370140+0000","last_undegraded":"2023-08-09T13:34:08.370140+0000","last_fullsized":"2023-08-09T13:34:08.370140+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:34:02.064154+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:34:02.064154+0000","last_clean_scrub_stamp":"2023-08-09T13:34:02.064154+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T21:35:01.466554+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.14","version":"0'0","reported_seq":13,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-08-09T13:34:08.369980+0000","last_change":"2023-08-09T13:34:07.251464+0000","last_active":"2023-08-09T13:34:08.369980+0000","last_peered":"2023-08-09T13:34:08.369980+0000","last_clean":"2023-08-09T13:34:08.369980+0000","last_became_active":"2023-08-09T13:34:07.251035+0000","last_became_peered":"2023-08-09T13:34:07.251035+0000","last_unstale":"2023-08-09T13:34:08.369980+0000","last_undegraded":"2023-08-09T13:34:08.369980+0000","last_fullsized":"2023-08-09T13:34:08.369980+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:34:06.173547+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:34:06.173547+0000","last_clean_scrub_stamp":"2023-08-09T13:34:06.173547+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T15:06:44.996627+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,2,1],"acting":[3,7,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.13","version":"19'2","reported_seq":21,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-08-09T13:34:08.370349+0000","last_change":"2023-08-09T13:34:02.115081+0000","last_active":"2023-08-09T13:34:08.370349+0000","last_peered":"2023-08-09T13:34:08.370349+0000","last_clean":"2023-08-09T13:34:08.370349+0000","last_became_active":"2023-08-09T13:34:02.114825+0000","last_became_peered":"2023-08-09T13:34:02.114825+0000","last_unstale":"2023-08-09T13:34:08.370349+0000","last_undegraded":"2023-08-09T13:34:08.370349+0000","last_fullsized":"2023-08-09T13:34:08.370349+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:34:01.052721+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:34:01.052721+0000","last_clean_scrub_stamp":"2023-08-09T13:34:01.052721+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T22:51:10.190023+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.14","version":"0'0","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-08-09T13:34:08.370421+0000","last_change":"2023-08-09T13:34:03.154171+0000","last_active":"2023-08-09T13:34:08.370421+0000","last_peered":"2023-08-09T13:34:08.370421+0000","last_clean":"2023-08-09T13:34:08.370421+0000","last_became_active":"2023-08-09T13:34:03.150789+0000","last_became_peered":"2023-08-09T13:34:03.150789+0000","last_unstale":"2023-08-09T13:34:08.370421+0000","last_undegraded":"2023-08-09T13:34:08.370421+0000","last_fullsized":"2023-08-09T13:34:08.370421+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:34:02.064154+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:34:02.064154+0000","last_clean_scrub_stamp":"2023-08-09T13:34:02.064154+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T17:18:45.337355+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.15","version":"0'0","reported_seq":13,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-08-09T13:34:08.369475+0000","last_change":"2023-08-09T13:34:07.241401+0000","last_active":"2023-08-09T13:34:08.369475+0000","last_peered":"2023-08-09T13:34:08.369475+0000","last_clean":"2023-08-09T13:34:08.369475+0000","last_became_active":"2023-08-09T13:34:07.240937+0000","last_became_peered":"2023-08-09T13:34:07.240937+0000","last_unstale":"2023-08-09T13:34:08.369475+0000","last_undegraded":"2023-08-09T13:34:08.369475+0000","last_fullsized":"2023-08-09T13:34:08.369475+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:34:06.173547+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:34:06.173547+0000","last_clean_scrub_stamp":"2023-08-09T13:34:06.173547+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T18:50:25.863045+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,3,0],"acting":[5,1,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.20","version":"0'0","reported_seq":17,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-08-09T13:34:08.210886+0000","last_change":"2023-08-09T13:34:03.157698+0000","last_active":"2023-08-09T13:34:08.210886+0000","last_peered":"2023-08-09T13:34:08.210886+0000","last_clean":"2023-08-09T13:34:08.210886+0000","last_became_active":"2023-08-09T13:34:03.153351+0000","last_became_peered":"2023-08-09T13:34:03.153351+0000","last_unstale":"2023-08-09T13:34:08.210886+0000","last_undegraded":"2023-08-09T13:34:08.210886+0000","last_fullsized":"2023-08-09T13:34:08.210886+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:34:02.064154+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:34:02.064154+0000","last_clean_scrub_stamp":"2023-08-09T13:34:02.064154+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T19:40:06.763332+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.21","version":"0'0","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-08-09T13:34:08.368201+0000","last_change":"2023-08-09T13:34:03.150065+0000","last_active":"2023-08-09T13:34:08.368201+0000","last_peered":"2023-08-09T13:34:08.368201+0000","last_clean":"2023-08-09T13:34:08.368201+0000","last_became_active":"2023-08-09T13:34:03.149650+0000","last_became_peered":"2023-08-09T13:34:03.149650+0000","last_unstale":"2023-08-09T13:34:08.368201+0000","last_undegraded":"2023-08-09T13:34:08.368201+0000","last_fullsized":"2023-08-09T13:34:08.368201+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:34:02.064154+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:34:02.064154+0000","last_clean_scrub_stamp":"2023-08-09T13:34:02.064154+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T13:47:48.941725+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.22","version":"0'0","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-08-09T13:34:08.372000+0000","last_change":"2023-08-09T13:34:03.152515+0000","last_active":"2023-08-09T13:34:08.372000+0000","last_peered":"2023-08-09T13:34:08.372000+0000","last_clean":"2023-08-09T13:34:08.372000+0000","last_became_active":"2023-08-09T13:34:03.152123+0000","last_became_peered":"2023-08-09T13:34:03.152123+0000","last_unstale":"2023-08-09T13:34:08.372000+0000","last_undegraded":"2023-08-09T13:34:08.372000+0000","last_fullsized":"2023-08-09T13:34:08.372000+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:34:02.064154+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:34:02.064154+0000","last_clean_scrub_stamp":"2023-08-09T13:34:02.064154+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T15:53:52.598510+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.23","version":"0'0","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-08-09T13:34:08.369171+0000","last_change":"2023-08-09T13:34:03.159919+0000","last_active":"2023-08-09T13:34:08.369171+0000","last_peered":"2023-08-09T13:34:08.369171+0000","last_clean":"2023-08-09T13:34:08.369171+0000","last_became_active":"2023-08-09T13:34:03.154344+0000","last_became_peered":"2023-08-09T13:34:03.154344+0000","last_unstale":"2023-08-09T13:34:08.369171+0000","last_undegraded":"2023-08-09T13:34:08.369171+0000","last_fullsized":"2023-08-09T13:34:08.369171+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:34:02.064154+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:34:02.064154+0000","last_clean_scrub_stamp":"2023-08-09T13:34:02.064154+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T22:33:11.246239+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.24","version":"0'0","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-08-09T13:34:08.371698+0000","last_change":"2023-08-09T13:34:03.164662+0000","last_active":"2023-08-09T13:34:08.371698+0000","last_peered":"2023-08-09T13:34:08.371698+0000","last_clean":"2023-08-09T13:34:08.371698+0000","last_became_active":"2023-08-09T13:34:03.163711+0000","last_became_peered":"2023-08-09T13:34:03.163711+0000","last_unstale":"2023-08-09T13:34:08.371698+0000","last_undegraded":"2023-08-09T13:34:08.371698+0000","last_fullsized":"2023-08-09T13:34:08.371698+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:34:02.064154+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:34:02.064154+0000","last_clean_scrub_stamp":"2023-08-09T13:34:02.064154+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T23:52:27.738385+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.25","version":"0'0","reported_seq":17,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-08-09T13:34:08.369863+0000","last_change":"2023-08-09T13:34:03.175502+0000","last_active":"2023-08-09T13:34:08.369863+0000","last_peered":"2023-08-09T13:34:08.369863+0000","last_clean":"2023-08-09T13:34:08.369863+0000","last_became_active":"2023-08-09T13:34:03.175192+0000","last_became_peered":"2023-08-09T13:34:03.175192+0000","last_unstale":"2023-08-09T13:34:08.369863+0000","last_undegraded":"2023-08-09T13:34:08.369863+0000","last_fullsized":"2023-08-09T13:34:08.369863+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:34:02.064154+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:34:02.064154+0000","last_clean_scrub_stamp":"2023-08-09T13:34:02.064154+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T19:22:08.597283+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.26","version":"0'0","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-08-09T13:34:08.372808+0000","last_change":"2023-08-09T13:34:03.155984+0000","last_active":"2023-08-09T13:34:08.372808+0000","last_peered":"2023-08-09T13:34:08.372808+0000","last_clean":"2023-08-09T13:34:08.372808+0000","last_became_active":"2023-08-09T13:34:03.155686+0000","last_became_peered":"2023-08-09T13:34:03.155686+0000","last_unstale":"2023-08-09T13:34:08.372808+0000","last_undegraded":"2023-08-09T13:34:08.372808+0000","last_fullsized":"2023-08-09T13:34:08.372808+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:34:02.064154+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:34:02.064154+0000","last_clean_scrub_stamp":"2023-08-09T13:34:02.064154+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T15:43:44.075578+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.27","version":"0'0","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-08-09T13:34:08.372180+0000","last_change":"2023-08-09T13:34:03.154003+0000","last_active":"2023-08-09T13:34:08.372180+0000","last_peered":"2023-08-09T13:34:08.372180+0000","last_clean":"2023-08-09T13:34:08.372180+0000","last_became_active":"2023-08-09T13:34:03.153707+0000","last_became_peered":"2023-08-09T13:34:03.153707+0000","last_unstale":"2023-08-09T13:34:08.372180+0000","last_undegraded":"2023-08-09T13:34:08.372180+0000","last_fullsized":"2023-08-09T13:34:08.372180+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:34:02.064154+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:34:02.064154+0000","last_clean_scrub_stamp":"2023-08-09T13:34:02.064154+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T17:08:07.503980+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.28","version":"0'0","reported_seq":17,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-08-09T13:34:08.368537+0000","last_change":"2023-08-09T13:34:03.170907+0000","last_active":"2023-08-09T13:34:08.368537+0000","last_peered":"2023-08-09T13:34:08.368537+0000","last_clean":"2023-08-09T13:34:08.368537+0000","last_became_active":"2023-08-09T13:34:03.162204+0000","last_became_peered":"2023-08-09T13:34:03.162204+0000","last_unstale":"2023-08-09T13:34:08.368537+0000","last_undegraded":"2023-08-09T13:34:08.368537+0000","last_fullsized":"2023-08-09T13:34:08.368537+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:34:02.064154+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:34:02.064154+0000","last_clean_scrub_stamp":"2023-08-09T13:34:02.064154+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T22:30:06.459673+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.29","version":"0'0","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-08-09T13:34:08.368673+0000","last_change":"2023-08-09T13:34:03.144064+0000","last_active":"2023-08-09T13:34:08.368673+0000","last_peered":"2023-08-09T13:34:08.368673+0000","last_clean":"2023-08-09T13:34:08.368673+0000","last_became_active":"2023-08-09T13:34:03.143612+0000","last_became_peered":"2023-08-09T13:34:03.143612+0000","last_unstale":"2023-08-09T13:34:08.368673+0000","last_undegraded":"2023-08-09T13:34:08.368673+0000","last_fullsized":"2023-08-09T13:34:08.368673+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:34:02.064154+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:34:02.064154+0000","last_clean_scrub_stamp":"2023-08-09T13:34:02.064154+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T18:50:26.789814+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.2a","version":"0'0","reported_seq":17,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-08-09T13:34:08.208605+0000","last_change":"2023-08-09T13:34:03.140127+0000","last_active":"2023-08-09T13:34:08.208605+0000","last_peered":"2023-08-09T13:34:08.208605+0000","last_clean":"2023-08-09T13:34:08.208605+0000","last_became_active":"2023-08-09T13:34:03.139616+0000","last_became_peered":"2023-08-09T13:34:03.139616+0000","last_unstale":"2023-08-09T13:34:08.208605+0000","last_undegraded":"2023-08-09T13:34:08.208605+0000","last_fullsized":"2023-08-09T13:34:08.208605+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:34:02.064154+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:34:02.064154+0000","last_clean_scrub_stamp":"2023-08-09T13:34:02.064154+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T18:50:21.753652+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.2b","version":"0'0","reported_seq":17,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-08-09T13:34:08.369974+0000","last_change":"2023-08-09T13:34:03.170969+0000","last_active":"2023-08-09T13:34:08.369974+0000","last_peered":"2023-08-09T13:34:08.369974+0000","last_clean":"2023-08-09T13:34:08.369974+0000","last_became_active":"2023-08-09T13:34:03.162334+0000","last_became_peered":"2023-08-09T13:34:03.162334+0000","last_unstale":"2023-08-09T13:34:08.369974+0000","last_undegraded":"2023-08-09T13:34:08.369974+0000","last_fullsized":"2023-08-09T13:34:08.369974+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:34:02.064154+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:34:02.064154+0000","last_clean_scrub_stamp":"2023-08-09T13:34:02.064154+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T23:34:54.506258+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.2c","version":"0'0","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-08-09T13:34:08.372790+0000","last_change":"2023-08-09T13:34:03.149273+0000","last_active":"2023-08-09T13:34:08.372790+0000","last_peered":"2023-08-09T13:34:08.372790+0000","last_clean":"2023-08-09T13:34:08.372790+0000","last_became_active":"2023-08-09T13:34:03.148460+0000","last_became_peered":"2023-08-09T13:34:03.148460+0000","last_unstale":"2023-08-09T13:34:08.372790+0000","last_undegraded":"2023-08-09T13:34:08.372790+0000","last_fullsized":"2023-08-09T13:34:08.372790+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:34:02.064154+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:34:02.064154+0000","last_clean_scrub_stamp":"2023-08-09T13:34:02.064154+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-11T01:12:24.927676+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.2d","version":"0'0","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-08-09T13:34:08.211794+0000","last_change":"2023-08-09T13:34:03.160727+0000","last_active":"2023-08-09T13:34:08.211794+0000","last_peered":"2023-08-09T13:34:08.211794+0000","last_clean":"2023-08-09T13:34:08.211794+0000","last_became_active":"2023-08-09T13:34:03.160406+0000","last_became_peered":"2023-08-09T13:34:03.160406+0000","last_unstale":"2023-08-09T13:34:08.211794+0000","last_undegraded":"2023-08-09T13:34:08.211794+0000","last_fullsized":"2023-08-09T13:34:08.211794+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:34:02.064154+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:34:02.064154+0000","last_clean_scrub_stamp":"2023-08-09T13:34:02.064154+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T20:28:43.534176+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.2e","version":"0'0","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-08-09T13:34:08.373221+0000","last_change":"2023-08-09T13:34:03.154714+0000","last_active":"2023-08-09T13:34:08.373221+0000","last_peered":"2023-08-09T13:34:08.373221+0000","last_clean":"2023-08-09T13:34:08.373221+0000","last_became_active":"2023-08-09T13:34:03.154384+0000","last_became_peered":"2023-08-09T13:34:03.154384+0000","last_unstale":"2023-08-09T13:34:08.373221+0000","last_undegraded":"2023-08-09T13:34:08.373221+0000","last_fullsized":"2023-08-09T13:34:08.373221+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:34:02.064154+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:34:02.064154+0000","last_clean_scrub_stamp":"2023-08-09T13:34:02.064154+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T16:21:31.389815+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.2f","version":"0'0","reported_seq":17,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-08-09T13:34:08.209885+0000","last_change":"2023-08-09T13:34:03.144822+0000","last_active":"2023-08-09T13:34:08.209885+0000","last_peered":"2023-08-09T13:34:08.209885+0000","last_clean":"2023-08-09T13:34:08.209885+0000","last_became_active":"2023-08-09T13:34:03.144470+0000","last_became_peered":"2023-08-09T13:34:03.144470+0000","last_unstale":"2023-08-09T13:34:08.209885+0000","last_undegraded":"2023-08-09T13:34:08.209885+0000","last_fullsized":"2023-08-09T13:34:08.209885+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:34:02.064154+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:34:02.064154+0000","last_clean_scrub_stamp":"2023-08-09T13:34:02.064154+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T21:18:19.551549+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.30","version":"0'0","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-08-09T13:34:08.370238+0000","last_change":"2023-08-09T13:34:03.140680+0000","last_active":"2023-08-09T13:34:08.370238+0000","last_peered":"2023-08-09T13:34:08.370238+0000","last_clean":"2023-08-09T13:34:08.370238+0000","last_became_active":"2023-08-09T13:34:03.139105+0000","last_became_peered":"2023-08-09T13:34:03.139105+0000","last_unstale":"2023-08-09T13:34:08.370238+0000","last_undegraded":"2023-08-09T13:34:08.370238+0000","last_fullsized":"2023-08-09T13:34:08.370238+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:34:02.064154+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:34:02.064154+0000","last_clean_scrub_stamp":"2023-08-09T13:34:02.064154+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T18:16:48.383340+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.31","version":"0'0","reported_seq":17,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-08-09T13:34:08.369342+0000","last_change":"2023-08-09T13:34:03.174569+0000","last_active":"2023-08-09T13:34:08.369342+0000","last_peered":"2023-08-09T13:34:08.369342+0000","last_clean":"2023-08-09T13:34:08.369342+0000","last_became_active":"2023-08-09T13:34:03.173450+0000","last_became_peered":"2023-08-09T13:34:03.173450+0000","last_unstale":"2023-08-09T13:34:08.369342+0000","last_undegraded":"2023-08-09T13:34:08.369342+0000","last_fullsized":"2023-08-09T13:34:08.369342+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:34:02.064154+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:34:02.064154+0000","last_clean_scrub_stamp":"2023-08-09T13:34:02.064154+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T13:44:50.765363+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.32","version":"0'0","reported_seq":17,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-08-09T13:34:08.368679+0000","last_change":"2023-08-09T13:34:03.156031+0000","last_active":"2023-08-09T13:34:08.368679+0000","last_peered":"2023-08-09T13:34:08.368679+0000","last_clean":"2023-08-09T13:34:08.368679+0000","last_became_active":"2023-08-09T13:34:03.155720+0000","last_became_peered":"2023-08-09T13:34:03.155720+0000","last_unstale":"2023-08-09T13:34:08.368679+0000","last_undegraded":"2023-08-09T13:34:08.368679+0000","last_fullsized":"2023-08-09T13:34:08.368679+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:34:02.064154+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:34:02.064154+0000","last_clean_scrub_stamp":"2023-08-09T13:34:02.064154+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T20:46:15.647419+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.33","version":"0'0","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-08-09T13:34:08.211563+0000","last_change":"2023-08-09T13:34:03.159410+0000","last_active":"2023-08-09T13:34:08.211563+0000","last_peered":"2023-08-09T13:34:08.211563+0000","last_clean":"2023-08-09T13:34:08.211563+0000","last_became_active":"2023-08-09T13:34:03.159006+0000","last_became_peered":"2023-08-09T13:34:03.159006+0000","last_unstale":"2023-08-09T13:34:08.211563+0000","last_undegraded":"2023-08-09T13:34:08.211563+0000","last_fullsized":"2023-08-09T13:34:08.211563+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:34:02.064154+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:34:02.064154+0000","last_clean_scrub_stamp":"2023-08-09T13:34:02.064154+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T21:55:49.311446+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.34","version":"0'0","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-08-09T13:34:08.212297+0000","last_change":"2023-08-09T13:34:03.165655+0000","last_active":"2023-08-09T13:34:08.212297+0000","last_peered":"2023-08-09T13:34:08.212297+0000","last_clean":"2023-08-09T13:34:08.212297+0000","last_became_active":"2023-08-09T13:34:03.165308+0000","last_became_peered":"2023-08-09T13:34:03.165308+0000","last_unstale":"2023-08-09T13:34:08.212297+0000","last_undegraded":"2023-08-09T13:34:08.212297+0000","last_fullsized":"2023-08-09T13:34:08.212297+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:34:02.064154+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:34:02.064154+0000","last_clean_scrub_stamp":"2023-08-09T13:34:02.064154+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T21:47:59.693735+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.35","version":"0'0","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-08-09T13:34:08.372839+0000","last_change":"2023-08-09T13:34:03.138842+0000","last_active":"2023-08-09T13:34:08.372839+0000","last_peered":"2023-08-09T13:34:08.372839+0000","last_clean":"2023-08-09T13:34:08.372839+0000","last_became_active":"2023-08-09T13:34:03.138554+0000","last_became_peered":"2023-08-09T13:34:03.138554+0000","last_unstale":"2023-08-09T13:34:08.372839+0000","last_undegraded":"2023-08-09T13:34:08.372839+0000","last_fullsized":"2023-08-09T13:34:08.372839+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:34:02.064154+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:34:02.064154+0000","last_clean_scrub_stamp":"2023-08-09T13:34:02.064154+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T23:05:06.664154+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.36","version":"0'0","reported_seq":17,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-08-09T13:34:08.371821+0000","last_change":"2023-08-09T13:34:03.148574+0000","last_active":"2023-08-09T13:34:08.371821+0000","last_peered":"2023-08-09T13:34:08.371821+0000","last_clean":"2023-08-09T13:34:08.371821+0000","last_became_active":"2023-08-09T13:34:03.148292+0000","last_became_peered":"2023-08-09T13:34:03.148292+0000","last_unstale":"2023-08-09T13:34:08.371821+0000","last_undegraded":"2023-08-09T13:34:08.371821+0000","last_fullsized":"2023-08-09T13:34:08.371821+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:34:02.064154+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:34:02.064154+0000","last_clean_scrub_stamp":"2023-08-09T13:34:02.064154+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T13:44:50.765363+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.37","version":"0'0","reported_seq":17,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-08-09T13:34:08.209550+0000","last_change":"2023-08-09T13:34:03.152770+0000","last_active":"2023-08-09T13:34:08.209550+0000","last_peered":"2023-08-09T13:34:08.209550+0000","last_clean":"2023-08-09T13:34:08.209550+0000","last_became_active":"2023-08-09T13:34:03.151703+0000","last_became_peered":"2023-08-09T13:34:03.151703+0000","last_unstale":"2023-08-09T13:34:08.209550+0000","last_undegraded":"2023-08-09T13:34:08.209550+0000","last_fullsized":"2023-08-09T13:34:08.209550+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:34:02.064154+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:34:02.064154+0000","last_clean_scrub_stamp":"2023-08-09T13:34:02.064154+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T17:46:23.039207+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.38","version":"0'0","reported_seq":17,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-08-09T13:34:08.367726+0000","last_change":"2023-08-09T13:34:03.146016+0000","last_active":"2023-08-09T13:34:08.367726+0000","last_peered":"2023-08-09T13:34:08.367726+0000","last_clean":"2023-08-09T13:34:08.367726+0000","last_became_active":"2023-08-09T13:34:03.145766+0000","last_became_peered":"2023-08-09T13:34:03.145766+0000","last_unstale":"2023-08-09T13:34:08.367726+0000","last_undegraded":"2023-08-09T13:34:08.367726+0000","last_fullsized":"2023-08-09T13:34:08.367726+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:34:02.064154+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:34:02.064154+0000","last_clean_scrub_stamp":"2023-08-09T13:34:02.064154+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T13:51:34.177397+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.39","version":"0'0","reported_seq":17,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-08-09T13:34:08.368967+0000","last_change":"2023-08-09T13:34:03.173368+0000","last_active":"2023-08-09T13:34:08.368967+0000","last_peered":"2023-08-09T13:34:08.368967+0000","last_clean":"2023-08-09T13:34:08.368967+0000","last_became_active":"2023-08-09T13:34:03.173037+0000","last_became_peered":"2023-08-09T13:34:03.173037+0000","last_unstale":"2023-08-09T13:34:08.368967+0000","last_undegraded":"2023-08-09T13:34:08.368967+0000","last_fullsized":"2023-08-09T13:34:08.368967+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:34:02.064154+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:34:02.064154+0000","last_clean_scrub_stamp":"2023-08-09T13:34:02.064154+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T21:11:40.494329+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.3a","version":"0'0","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-08-09T13:34:08.371086+0000","last_change":"2023-08-09T13:34:03.144394+0000","last_active":"2023-08-09T13:34:08.371086+0000","last_peered":"2023-08-09T13:34:08.371086+0000","last_clean":"2023-08-09T13:34:08.371086+0000","last_became_active":"2023-08-09T13:34:03.143958+0000","last_became_peered":"2023-08-09T13:34:03.143958+0000","last_unstale":"2023-08-09T13:34:08.371086+0000","last_undegraded":"2023-08-09T13:34:08.371086+0000","last_fullsized":"2023-08-09T13:34:08.371086+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:34:02.064154+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:34:02.064154+0000","last_clean_scrub_stamp":"2023-08-09T13:34:02.064154+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T19:49:18.157706+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.3b","version":"0'0","reported_seq":17,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-08-09T13:34:08.210114+0000","last_change":"2023-08-09T13:34:03.144695+0000","last_active":"2023-08-09T13:34:08.210114+0000","last_peered":"2023-08-09T13:34:08.210114+0000","last_clean":"2023-08-09T13:34:08.210114+0000","last_became_active":"2023-08-09T13:34:03.144303+0000","last_became_peered":"2023-08-09T13:34:03.144303+0000","last_unstale":"2023-08-09T13:34:08.210114+0000","last_undegraded":"2023-08-09T13:34:08.210114+0000","last_fullsized":"2023-08-09T13:34:08.210114+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:34:02.064154+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:34:02.064154+0000","last_clean_scrub_stamp":"2023-08-09T13:34:02.064154+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T18:07:31.705168+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.3c","version":"0'0","reported_seq":17,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-08-09T13:34:08.212037+0000","last_change":"2023-08-09T13:34:03.151741+0000","last_active":"2023-08-09T13:34:08.212037+0000","last_peered":"2023-08-09T13:34:08.212037+0000","last_clean":"2023-08-09T13:34:08.212037+0000","last_became_active":"2023-08-09T13:34:03.151331+0000","last_became_peered":"2023-08-09T13:34:03.151331+0000","last_unstale":"2023-08-09T13:34:08.212037+0000","last_undegraded":"2023-08-09T13:34:08.212037+0000","last_fullsized":"2023-08-09T13:34:08.212037+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:34:02.064154+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:34:02.064154+0000","last_clean_scrub_stamp":"2023-08-09T13:34:02.064154+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-11T00:09:38.635751+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.3d","version":"0'0","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-08-09T13:34:08.372764+0000","last_change":"2023-08-09T13:34:03.144010+0000","last_active":"2023-08-09T13:34:08.372764+0000","last_peered":"2023-08-09T13:34:08.372764+0000","last_clean":"2023-08-09T13:34:08.372764+0000","last_became_active":"2023-08-09T13:34:03.143673+0000","last_became_peered":"2023-08-09T13:34:03.143673+0000","last_unstale":"2023-08-09T13:34:08.372764+0000","last_undegraded":"2023-08-09T13:34:08.372764+0000","last_fullsized":"2023-08-09T13:34:08.372764+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:34:02.064154+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:34:02.064154+0000","last_clean_scrub_stamp":"2023-08-09T13:34:02.064154+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T16:58:18.499551+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.3e","version":"0'0","reported_seq":17,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-08-09T13:34:08.371258+0000","last_change":"2023-08-09T13:34:03.150894+0000","last_active":"2023-08-09T13:34:08.371258+0000","last_peered":"2023-08-09T13:34:08.371258+0000","last_clean":"2023-08-09T13:34:08.371258+0000","last_became_active":"2023-08-09T13:34:03.149237+0000","last_became_peered":"2023-08-09T13:34:03.149237+0000","last_unstale":"2023-08-09T13:34:08.371258+0000","last_undegraded":"2023-08-09T13:34:08.371258+0000","last_fullsized":"2023-08-09T13:34:08.371258+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:34:02.064154+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:34:02.064154+0000","last_clean_scrub_stamp":"2023-08-09T13:34:02.064154+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T21:18:19.551549+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.3f","version":"0'0","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-08-09T13:34:08.212027+0000","last_change":"2023-08-09T13:34:03.160785+0000","last_active":"2023-08-09T13:34:08.212027+0000","last_peered":"2023-08-09T13:34:08.212027+0000","last_clean":"2023-08-09T13:34:08.212027+0000","last_became_active":"2023-08-09T13:34:03.160452+0000","last_became_peered":"2023-08-09T13:34:03.160452+0000","last_unstale":"2023-08-09T13:34:08.212027+0000","last_undegraded":"2023-08-09T13:34:08.212027+0000","last_fullsized":"2023-08-09T13:34:08.212027+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:34:02.064154+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:34:02.064154+0000","last_clean_scrub_stamp":"2023-08-09T13:34:02.064154+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T18:03:19.093604+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]}],"pool_stats":[{"poolid":5,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":4,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":3,"num_pg":32,"stat_sum":{"num_bytes":4639,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":49,"num_write_kb":19,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":9278,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":43,"ondisk_log_size":43,"up":64,"acting":64,"num_store_stats":8},{"poolid":2,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":8}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574856,"num_pgs":41,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27872,"kb_used_data":856,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343968,"statfs":{"total":96636764160,"available":96608223232,"internally_reserved":0,"allocated":876544,"data_stored":699155,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1781,"internal_metadata":27654411},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.806}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.7110000000000001}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.0269999999999999}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.1479999999999999}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.6379999999999999}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.9079999999999999}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":2.367}]}]},{"osd":6,"up_from":13,"seq":55834574856,"num_pgs":34,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27268,"kb_used_data":244,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344572,"statfs":{"total":96636764160,"available":96608841728,"internally_reserved":0,"allocated":249856,"data_stored":103369,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1781,"internal_metadata":27654411},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.0920000000000001}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.3180000000000001}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.55}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.141}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.228}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.4319999999999999}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.359}]}]},{"osd":5,"up_from":13,"seq":55834574856,"num_pgs":39,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27280,"kb_used_data":256,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344560,"statfs":{"total":96636764160,"available":96608829440,"internally_reserved":0,"allocated":262144,"data_stored":105382,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1781,"internal_metadata":27654411},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.1040000000000001}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.599}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.167}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.2130000000000001}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.4279999999999999}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.3720000000000001}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.3959999999999999}]}]},{"osd":4,"up_from":13,"seq":55834574856,"num_pgs":50,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27284,"kb_used_data":268,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344556,"statfs":{"total":96636764160,"available":96608825344,"internally_reserved":0,"allocated":274432,"data_stored":108627,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1781,"internal_metadata":27654411},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.996}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.429}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.2909999999999999}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.357}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.772}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.5760000000000001}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.649}]}]},{"osd":3,"up_from":13,"seq":55834574857,"num_pgs":48,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31372,"kb_used_data":252,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":94340468,"statfs":{"total":96636764160,"available":96604639232,"internally_reserved":0,"allocated":258048,"data_stored":104820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1781,"internal_metadata":31848715},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.3560000000000001}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.181}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.98099999999999998}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.034}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.476}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.4159999999999999}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.804}]}]},{"osd":2,"up_from":13,"seq":55834574856,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31368,"kb_used_data":248,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":94340472,"statfs":{"total":96636764160,"available":96604643328,"internally_reserved":0,"allocated":253952,"data_stored":104730,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1781,"internal_metadata":31848715},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.204}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.462}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.0940000000000001}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.73}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.411}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.385}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.5269999999999999}]}]},{"osd":1,"up_from":13,"seq":55834574856,"num_pgs":49,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31952,"kb_used_data":828,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":94339888,"statfs":{"total":96636764160,"available":96604045312,"internally_reserved":0,"allocated":847872,"data_stored":695098,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1781,"internal_metadata":31848715},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.98699999999999999}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.746}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.2689999999999999}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.216}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.1699999999999999}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.3009999999999999}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":2.0699999999999998}]}]},{"osd":0,"up_from":13,"seq":55834574856,"num_pgs":47,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31368,"kb_used_data":248,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":94340472,"statfs":{"total":96636764160,"available":96604643328,"internally_reserved":0,"allocated":253952,"data_stored":103951,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1781,"internal_metadata":31848715},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.95899999999999996}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.2270000000000001}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.1240000000000001}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.5329999999999999}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.3089999999999999}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.7210000000000001}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.623}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":3897,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":652,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":4057,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-08-09T13:34:33.360 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-08-09T13:34:33.623 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:33.621+0000 7f34eae57640 1 Processor -- start 2023-08-09T13:34:33.624 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:33.621+0000 7f34eae57640 1 -- start start 2023-08-09T13:34:33.625 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:33.623+0000 7f34eae57640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f34e40cd510 0x7f34e40c96e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:34:33.625 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:33.623+0000 7f34eae57640 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7f34e415ce40 0x7f34e40c3290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:34:33.626 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:33.623+0000 7f34eae57640 1 -- --> v1:172.21.15.203:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f34e4143080 con 0x7f34e4142ca0 2023-08-09T13:34:33.626 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:33.623+0000 7f34eae57640 1 -- --> v2:172.21.15.43:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f34e406d400 con 0x7f34e40cd510 2023-08-09T13:34:33.626 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:33.623+0000 7f34eae57640 1 -- --> v2:172.21.15.203:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f34e40c9d90 con 0x7f34e415ce40 2023-08-09T13:34:33.627 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:33.623+0000 7f34e3fff640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f34e40cd510 0x7f34e40c96e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:34:33.627 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:33.623+0000 7f34e3fff640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f34e40cd510 0x7f34e40c96e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:37668/0 (socket says 172.21.15.43:37668) 2023-08-09T13:34:33.627 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:33.623+0000 7f34e3fff640 1 -- 172.21.15.43:0/2066727145 learned_addr learned my addr 172.21.15.43:0/2066727145 (peer_addr_for_me v2:172.21.15.43:0/0) 2023-08-09T13:34:33.627 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:33.624+0000 7f34e93f4640 1 --2- 172.21.15.43:0/2066727145 >> v2:172.21.15.203:3300/0 conn(0x7f34e415ce40 0x7f34e40c3290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:34:33.627 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:33.624+0000 7f34e3fff640 1 -- 172.21.15.43:0/2066727145 >> v2:172.21.15.203:3300/0 conn(0x7f34e415ce40 msgr2=0x7f34e40c3290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-09T13:34:33.627 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:33.624+0000 7f34e3fff640 1 --2- 172.21.15.43:0/2066727145 >> v2:172.21.15.203:3300/0 conn(0x7f34e415ce40 0x7f34e40c3290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:34:33.628 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:33.624+0000 7f34e3fff640 1 -- 172.21.15.43:0/2066727145 >> v1:172.21.15.203:6789/0 conn(0x7f34e4142ca0 legacy=0x7f34e40cce00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:34:33.628 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:33.624+0000 7f34e3fff640 1 -- 172.21.15.43:0/2066727145 --> v2:172.21.15.43:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f34e40ca070 con 0x7f34e40cd510 2023-08-09T13:34:33.629 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:33.625+0000 7f34e3fff640 1 --2- 172.21.15.43:0/2066727145 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f34e40cd510 0x7f34e40c96e0 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7f34e40cf100 tx=0x7f34d002e170 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2f1dbbad55a2bc48 server_cookie=0 in_seq=0 out_seq=0 2023-08-09T13:34:33.629 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:33.626+0000 7f34e37fe640 1 -- 172.21.15.43:0/2066727145 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f34d0036440 con 0x7f34e40cd510 2023-08-09T13:34:33.629 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:33.626+0000 7f34e37fe640 1 -- 172.21.15.43:0/2066727145 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f34d00365a0 con 0x7f34e40cd510 2023-08-09T13:34:33.629 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:33.626+0000 7f34e37fe640 1 -- 172.21.15.43:0/2066727145 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f34d0036930 con 0x7f34e40cd510 2023-08-09T13:34:33.629 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:33.626+0000 7f34eae57640 1 -- 172.21.15.43:0/2066727145 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f34e40cd510 msgr2=0x7f34e40c96e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:34:33.630 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:33.626+0000 7f34eae57640 1 --2- 172.21.15.43:0/2066727145 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f34e40cd510 0x7f34e40c96e0 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7f34e40cf100 tx=0x7f34d002e170 comp rx=0 tx=0).stop 2023-08-09T13:34:33.630 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:33.626+0000 7f34eae57640 1 -- 172.21.15.43:0/2066727145 shutdown_connections 2023-08-09T13:34:33.630 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:33.626+0000 7f34eae57640 1 --2- 172.21.15.43:0/2066727145 >> v2:172.21.15.203:3300/0 conn(0x7f34e415ce40 0x7f34e40c3290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:34:33.630 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:33.626+0000 7f34eae57640 1 --2- 172.21.15.43:0/2066727145 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f34e40cd510 0x7f34e40c96e0 unknown :-1 s=CLOSED pgs=148 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:34:33.630 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:33.626+0000 7f34eae57640 1 -- 172.21.15.43:0/2066727145 >> 172.21.15.43:0/2066727145 conn(0x7f34e40ce0a0 msgr2=0x7f34e40ce480 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-09T13:34:33.630 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:33.627+0000 7f34eae57640 1 -- 172.21.15.43:0/2066727145 shutdown_connections 2023-08-09T13:34:33.630 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:33.627+0000 7f34eae57640 1 -- 172.21.15.43:0/2066727145 wait complete. 2023-08-09T13:34:33.631 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:33.628+0000 7f34eae57640 1 Processor -- start 2023-08-09T13:34:33.631 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:33.629+0000 7f34eae57640 1 -- start start 2023-08-09T13:34:33.633 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:33.630+0000 7f34eae57640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f34e40cd510 0x7f34e4137e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:34:33.633 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:33.630+0000 7f34eae57640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f34e4142ca0 0x7f34e406bcb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:34:33.633 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:33.630+0000 7f34eae57640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f34e415ce40 0x7f34e40b0d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:34:33.633 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:33.630+0000 7f34eae57640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f34e40ab8c0 con 0x7f34e415ce40 2023-08-09T13:34:33.633 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:33.630+0000 7f34eae57640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f34e406c670 con 0x7f34e4142ca0 2023-08-09T13:34:33.634 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:33.630+0000 7f34eae57640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f34e406c950 con 0x7f34e40cd510 2023-08-09T13:34:33.634 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:33.630+0000 7f34e93f4640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f34e415ce40 0x7f34e40b0d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:34:33.634 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:33.630+0000 7f34e93f4640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f34e415ce40 0x7f34e40b0d80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:37670/0 (socket says 172.21.15.43:37670) 2023-08-09T13:34:33.634 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:33.630+0000 7f34e93f4640 1 -- 172.21.15.43:0/2142563548 learned_addr learned my addr 172.21.15.43:0/2142563548 (peer_addr_for_me v2:172.21.15.43:0/0) 2023-08-09T13:34:33.634 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:33.630+0000 7f34e3fff640 1 --2- 172.21.15.43:0/2142563548 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f34e4142ca0 0x7f34e406bcb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:34:33.634 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:33.630+0000 7f34e8bf3640 1 --2- 172.21.15.43:0/2142563548 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f34e40cd510 0x7f34e4137e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:34:33.635 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:33.630+0000 7f34e93f4640 1 -- 172.21.15.43:0/2142563548 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f34e40cd510 msgr2=0x7f34e4137e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:34:33.635 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:33.630+0000 7f34e93f4640 1 --2- 172.21.15.43:0/2142563548 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f34e40cd510 0x7f34e4137e70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:34:33.635 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:33.630+0000 7f34e93f4640 1 -- 172.21.15.43:0/2142563548 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f34e4142ca0 msgr2=0x7f34e406bcb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:34:33.635 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:33.630+0000 7f34e93f4640 1 --2- 172.21.15.43:0/2142563548 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f34e4142ca0 0x7f34e406bcb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:34:33.635 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:33.631+0000 7f34e93f4640 1 -- 172.21.15.43:0/2142563548 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f34d000b770 con 0x7f34e415ce40 2023-08-09T13:34:33.636 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:33.631+0000 7f34e93f4640 1 --2- 172.21.15.43:0/2142563548 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f34e415ce40 0x7f34e40b0d80 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7f34dc00ec10 tx=0x7f34dc00c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-09T13:34:33.636 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:33.631+0000 7f34e17fa640 1 -- 172.21.15.43:0/2142563548 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f34dc00cd20 con 0x7f34e415ce40 2023-08-09T13:34:33.636 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:33.631+0000 7f34e17fa640 1 -- 172.21.15.43:0/2142563548 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f34dc00f040 con 0x7f34e415ce40 2023-08-09T13:34:33.636 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:33.631+0000 7f34e17fa640 1 -- 172.21.15.43:0/2142563548 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f34dc0146c0 con 0x7f34e415ce40 2023-08-09T13:34:33.637 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:33.631+0000 7f34eae57640 1 -- 172.21.15.43:0/2142563548 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f34e406cc30 con 0x7f34e415ce40 2023-08-09T13:34:33.637 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:33.631+0000 7f34eae57640 1 -- 172.21.15.43:0/2142563548 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f34e406c1f0 con 0x7f34e415ce40 2023-08-09T13:34:33.637 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:33.633+0000 7f34eae57640 1 -- 172.21.15.43:0/2142563548 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f34e40aba70 con 0x7f34e415ce40 2023-08-09T13:34:33.637 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:33.633+0000 7f34e17fa640 1 -- 172.21.15.43:0/2142563548 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 5) v1 ==== 88341+0+0 (secure 0 0 0) 0x7f34dc014a30 con 0x7f34e415ce40 2023-08-09T13:34:33.638 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:33.634+0000 7f34e17fa640 1 --2- 172.21.15.43:0/2142563548 >> [v2:172.21.15.203:6832/830097474,v1:172.21.15.203:6833/830097474] conn(0x7f34c806a050 0x7f34c806c510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:34:33.638 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:33.634+0000 7f34e17fa640 1 -- 172.21.15.43:0/2142563548 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6813+0+0 (secure 0 0 0) 0x7f34dc089600 con 0x7f34e415ce40 2023-08-09T13:34:33.638 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:33.635+0000 7f34e8bf3640 1 --2- 172.21.15.43:0/2142563548 >> [v2:172.21.15.203:6832/830097474,v1:172.21.15.203:6833/830097474] conn(0x7f34c806a050 0x7f34c806c510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:34:33.638 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:33.636+0000 7f34e8bf3640 1 --2- 172.21.15.43:0/2142563548 >> [v2:172.21.15.203:6832/830097474,v1:172.21.15.203:6833/830097474] conn(0x7f34c806a050 0x7f34c806c510 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7f34d8000f60 tx=0x7f34d8009040 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-09T13:34:33.641 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:33.639+0000 7f34e17fa640 1 -- 172.21.15.43:0/2142563548 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178066 (secure 0 0 0) 0x7f34e40aba70 con 0x7f34e415ce40 2023-08-09T13:34:33.830 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:33.828+0000 7f34eae57640 1 -- 172.21.15.43:0/2142563548 --> [v2:172.21.15.203:6832/830097474,v1:172.21.15.203:6833/830097474] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f34e40ca630 con 0x7f34c806a050 2023-08-09T13:34:33.859 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:33.857+0000 7f34e17fa640 1 -- 172.21.15.43:0/2142563548 <== mgr.4114 v2:172.21.15.203:6832/830097474 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+367303 (secure 0 0 0) 0x7f34d8001390 con 0x7f34c806a050 2023-08-09T13:34:33.863 INFO:teuthology.orchestra.run.smithi043.stdout: 2023-08-09T13:34:33.864 INFO:teuthology.orchestra.run.smithi043.stderr:dumped all 2023-08-09T13:34:33.866 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:33.862+0000 7f34eae57640 1 -- 172.21.15.43:0/2142563548 >> [v2:172.21.15.203:6832/830097474,v1:172.21.15.203:6833/830097474] conn(0x7f34c806a050 msgr2=0x7f34c806c510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:34:33.866 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:33.862+0000 7f34eae57640 1 --2- 172.21.15.43:0/2142563548 >> [v2:172.21.15.203:6832/830097474,v1:172.21.15.203:6833/830097474] conn(0x7f34c806a050 0x7f34c806c510 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7f34d8000f60 tx=0x7f34d8009040 comp rx=0 tx=0).stop 2023-08-09T13:34:33.866 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:33.862+0000 7f34eae57640 1 -- 172.21.15.43:0/2142563548 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f34e415ce40 msgr2=0x7f34e40b0d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:34:33.866 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:33.862+0000 7f34eae57640 1 --2- 172.21.15.43:0/2142563548 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f34e415ce40 0x7f34e40b0d80 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7f34dc00ec10 tx=0x7f34dc00c490 comp rx=0 tx=0).stop 2023-08-09T13:34:33.867 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:33.862+0000 7f34eae57640 1 -- 172.21.15.43:0/2142563548 shutdown_connections 2023-08-09T13:34:33.867 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:33.862+0000 7f34eae57640 1 --2- 172.21.15.43:0/2142563548 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f34e415ce40 0x7f34e40b0d80 unknown :-1 s=CLOSED pgs=149 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:34:33.867 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:33.862+0000 7f34eae57640 1 --2- 172.21.15.43:0/2142563548 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f34e4142ca0 0x7f34e406bcb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:34:33.867 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:33.862+0000 7f34eae57640 1 --2- 172.21.15.43:0/2142563548 >> [v2:172.21.15.203:6832/830097474,v1:172.21.15.203:6833/830097474] conn(0x7f34c806a050 0x7f34c806c510 unknown :-1 s=CLOSED pgs=94 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:34:33.867 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:33.862+0000 7f34eae57640 1 --2- 172.21.15.43:0/2142563548 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f34e40cd510 0x7f34e4137e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:34:33.867 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:33.862+0000 7f34eae57640 1 -- 172.21.15.43:0/2142563548 >> 172.21.15.43:0/2142563548 conn(0x7f34e40ce0a0 msgr2=0x7f34e415d300 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-09T13:34:33.867 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:33.862+0000 7f34eae57640 1 -- 172.21.15.43:0/2142563548 shutdown_connections 2023-08-09T13:34:33.867 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:33.863+0000 7f34eae57640 1 -- 172.21.15.43:0/2142563548 wait complete. 2023-08-09T13:34:33.886 INFO:teuthology.orchestra.run.smithi043.stdout:{"pg_ready":true,"pg_map":{"version":37,"stamp":"2023-08-09T13:34:32.771588+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":595007,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":208,"num_write_kb":2435,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160,"ondisk_log_size":160,"up":338,"acting":338,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":338,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":235764,"kb_used_data":3200,"kb_used_omap":13,"kb_used_meta":232434,"kb_avail":754738956,"statfs":{"total":773094113280,"available":772852690944,"internally_reserved":0,"allocated":3276800,"data_stored":2025132,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":14248,"internal_metadata":238012504},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":141,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.005783"},"pg_stats":[{"pgid":"5.19","version":"0'0","reported_seq":13,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-08-09T13:34:08.367616+0000","last_change":"2023-08-09T13:34:07.232493+0000","last_active":"2023-08-09T13:34:08.367616+0000","last_peered":"2023-08-09T13:34:08.367616+0000","last_clean":"2023-08-09T13:34:08.367616+0000","last_became_active":"2023-08-09T13:34:07.231969+0000","last_became_peered":"2023-08-09T13:34:07.231969+0000","last_unstale":"2023-08-09T13:34:08.367616+0000","last_undegraded":"2023-08-09T13:34:08.367616+0000","last_fullsized":"2023-08-09T13:34:08.367616+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:34:06.173547+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:34:06.173547+0000","last_clean_scrub_stamp":"2023-08-09T13:34:06.173547+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T19:42:37.842719+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,7,2],"acting":[1,5,7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.1f","version":"19'4","reported_seq":23,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-08-09T13:34:08.209913+0000","last_change":"2023-08-09T13:34:02.108915+0000","last_active":"2023-08-09T13:34:08.209913+0000","last_peered":"2023-08-09T13:34:08.209913+0000","last_clean":"2023-08-09T13:34:08.209913+0000","last_became_active":"2023-08-09T13:34:02.108573+0000","last_became_peered":"2023-08-09T13:34:02.108573+0000","last_unstale":"2023-08-09T13:34:08.209913+0000","last_undegraded":"2023-08-09T13:34:08.209913+0000","last_fullsized":"2023-08-09T13:34:08.209913+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:34:01.052721+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:34:01.052721+0000","last_clean_scrub_stamp":"2023-08-09T13:34:01.052721+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T23:52:26.726953+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-08-09T13:34:08.370536+0000","last_change":"2023-08-09T13:34:03.156516+0000","last_active":"2023-08-09T13:34:08.370536+0000","last_peered":"2023-08-09T13:34:08.370536+0000","last_clean":"2023-08-09T13:34:08.370536+0000","last_became_active":"2023-08-09T13:34:03.156109+0000","last_became_peered":"2023-08-09T13:34:03.156109+0000","last_unstale":"2023-08-09T13:34:08.370536+0000","last_undegraded":"2023-08-09T13:34:08.370536+0000","last_fullsized":"2023-08-09T13:34:08.370536+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:34:02.064154+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:34:02.064154+0000","last_clean_scrub_stamp":"2023-08-09T13:34:02.064154+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T13:40:56.937864+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.18","version":"0'0","reported_seq":13,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-08-09T13:34:08.369809+0000","last_change":"2023-08-09T13:34:07.234694+0000","last_active":"2023-08-09T13:34:08.369809+0000","last_peered":"2023-08-09T13:34:08.369809+0000","last_clean":"2023-08-09T13:34:08.369809+0000","last_became_active":"2023-08-09T13:34:07.234241+0000","last_became_peered":"2023-08-09T13:34:07.234241+0000","last_unstale":"2023-08-09T13:34:08.369809+0000","last_undegraded":"2023-08-09T13:34:08.369809+0000","last_fullsized":"2023-08-09T13:34:08.369809+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:34:06.173547+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:34:06.173547+0000","last_clean_scrub_stamp":"2023-08-09T13:34:06.173547+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T15:06:44.662571+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,1,2],"acting":[4,6,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.1e","version":"19'1","reported_seq":22,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-08-09T13:34:08.370464+0000","last_change":"2023-08-09T13:34:02.105244+0000","last_active":"2023-08-09T13:34:08.370464+0000","last_peered":"2023-08-09T13:34:08.370464+0000","last_clean":"2023-08-09T13:34:08.370464+0000","last_became_active":"2023-08-09T13:34:02.104920+0000","last_became_peered":"2023-08-09T13:34:02.104920+0000","last_unstale":"2023-08-09T13:34:08.370464+0000","last_undegraded":"2023-08-09T13:34:08.370464+0000","last_fullsized":"2023-08-09T13:34:08.370464+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:34:01.052721+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:34:01.052721+0000","last_clean_scrub_stamp":"2023-08-09T13:34:01.052721+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T21:57:53.171937+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-08-09T13:34:08.368933+0000","last_change":"2023-08-09T13:34:03.156960+0000","last_active":"2023-08-09T13:34:08.368933+0000","last_peered":"2023-08-09T13:34:08.368933+0000","last_clean":"2023-08-09T13:34:08.368933+0000","last_became_active":"2023-08-09T13:34:03.156668+0000","last_became_peered":"2023-08-09T13:34:03.156668+0000","last_unstale":"2023-08-09T13:34:08.368933+0000","last_undegraded":"2023-08-09T13:34:08.368933+0000","last_fullsized":"2023-08-09T13:34:08.368933+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:34:02.064154+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:34:02.064154+0000","last_clean_scrub_stamp":"2023-08-09T13:34:02.064154+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T14:55:32.816063+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.1b","version":"0'0","reported_seq":13,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-08-09T13:34:08.370229+0000","last_change":"2023-08-09T13:34:07.232500+0000","last_active":"2023-08-09T13:34:08.370229+0000","last_peered":"2023-08-09T13:34:08.370229+0000","last_clean":"2023-08-09T13:34:08.370229+0000","last_became_active":"2023-08-09T13:34:07.232045+0000","last_became_peered":"2023-08-09T13:34:07.232045+0000","last_unstale":"2023-08-09T13:34:08.370229+0000","last_undegraded":"2023-08-09T13:34:08.370229+0000","last_fullsized":"2023-08-09T13:34:08.370229+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:34:06.173547+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:34:06.173547+0000","last_clean_scrub_stamp":"2023-08-09T13:34:06.173547+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T18:36:44.021636+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,7,4],"acting":[5,0,7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1d","version":"19'2","reported_seq":21,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-08-09T13:34:08.370340+0000","last_change":"2023-08-09T13:34:02.109750+0000","last_active":"2023-08-09T13:34:08.370340+0000","last_peered":"2023-08-09T13:34:08.370340+0000","last_clean":"2023-08-09T13:34:08.370340+0000","last_became_active":"2023-08-09T13:34:02.109445+0000","last_became_peered":"2023-08-09T13:34:02.109445+0000","last_unstale":"2023-08-09T13:34:08.370340+0000","last_undegraded":"2023-08-09T13:34:08.370340+0000","last_fullsized":"2023-08-09T13:34:08.370340+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:34:01.052721+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:34:01.052721+0000","last_clean_scrub_stamp":"2023-08-09T13:34:01.052721+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T22:30:05.448240+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1a","version":"0'0","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-08-09T13:34:08.371675+0000","last_change":"2023-08-09T13:34:03.145198+0000","last_active":"2023-08-09T13:34:08.371675+0000","last_peered":"2023-08-09T13:34:08.371675+0000","last_clean":"2023-08-09T13:34:08.371675+0000","last_became_active":"2023-08-09T13:34:03.144756+0000","last_became_peered":"2023-08-09T13:34:03.144756+0000","last_unstale":"2023-08-09T13:34:08.371675+0000","last_undegraded":"2023-08-09T13:34:08.371675+0000","last_fullsized":"2023-08-09T13:34:08.371675+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:34:02.064154+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:34:02.064154+0000","last_clean_scrub_stamp":"2023-08-09T13:34:02.064154+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T16:10:58.188522+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1a","version":"0'0","reported_seq":13,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-08-09T13:34:08.369069+0000","last_change":"2023-08-09T13:34:07.238058+0000","last_active":"2023-08-09T13:34:08.369069+0000","last_peered":"2023-08-09T13:34:08.369069+0000","last_clean":"2023-08-09T13:34:08.369069+0000","last_became_active":"2023-08-09T13:34:07.237575+0000","last_became_peered":"2023-08-09T13:34:07.237575+0000","last_unstale":"2023-08-09T13:34:08.369069+0000","last_undegraded":"2023-08-09T13:34:08.369069+0000","last_fullsized":"2023-08-09T13:34:08.369069+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:34:06.173547+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:34:06.173547+0000","last_clean_scrub_stamp":"2023-08-09T13:34:06.173547+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T21:18:23.660943+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,1,6],"acting":[7,4,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1c","version":"19'2","reported_seq":21,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-08-09T13:34:08.369633+0000","last_change":"2023-08-09T13:34:02.107853+0000","last_active":"2023-08-09T13:34:08.369633+0000","last_peered":"2023-08-09T13:34:08.369633+0000","last_clean":"2023-08-09T13:34:08.369633+0000","last_became_active":"2023-08-09T13:34:02.107571+0000","last_became_peered":"2023-08-09T13:34:02.107571+0000","last_unstale":"2023-08-09T13:34:08.369633+0000","last_undegraded":"2023-08-09T13:34:08.369633+0000","last_fullsized":"2023-08-09T13:34:08.369633+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:34:01.052721+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:34:01.052721+0000","last_clean_scrub_stamp":"2023-08-09T13:34:01.052721+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T19:42:32.721893+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1b","version":"0'0","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-08-09T13:34:08.371599+0000","last_change":"2023-08-09T13:34:03.148927+0000","last_active":"2023-08-09T13:34:08.371599+0000","last_peered":"2023-08-09T13:34:08.371599+0000","last_clean":"2023-08-09T13:34:08.371599+0000","last_became_active":"2023-08-09T13:34:03.148582+0000","last_became_peered":"2023-08-09T13:34:03.148582+0000","last_unstale":"2023-08-09T13:34:08.371599+0000","last_undegraded":"2023-08-09T13:34:08.371599+0000","last_fullsized":"2023-08-09T13:34:08.371599+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:34:02.064154+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:34:02.064154+0000","last_clean_scrub_stamp":"2023-08-09T13:34:02.064154+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T17:58:47.470353+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1d","version":"0'0","reported_seq":13,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-08-09T13:34:08.372087+0000","last_change":"2023-08-09T13:34:07.246175+0000","last_active":"2023-08-09T13:34:08.372087+0000","last_peered":"2023-08-09T13:34:08.372087+0000","last_clean":"2023-08-09T13:34:08.372087+0000","last_became_active":"2023-08-09T13:34:07.245757+0000","last_became_peered":"2023-08-09T13:34:07.245757+0000","last_unstale":"2023-08-09T13:34:08.372087+0000","last_undegraded":"2023-08-09T13:34:08.372087+0000","last_fullsized":"2023-08-09T13:34:08.372087+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:34:06.173547+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:34:06.173547+0000","last_clean_scrub_stamp":"2023-08-09T13:34:06.173547+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T18:07:35.814562+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,4,0],"acting":[1,3,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.1b","version":"19'1","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-08-09T13:34:08.210113+0000","last_change":"2023-08-09T13:34:02.109573+0000","last_active":"2023-08-09T13:34:08.210113+0000","last_peered":"2023-08-09T13:34:08.210113+0000","last_clean":"2023-08-09T13:34:08.210113+0000","last_became_active":"2023-08-09T13:34:02.109015+0000","last_became_peered":"2023-08-09T13:34:02.109015+0000","last_unstale":"2023-08-09T13:34:08.210113+0000","last_undegraded":"2023-08-09T13:34:08.210113+0000","last_fullsized":"2023-08-09T13:34:08.210113+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:34:01.052721+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:34:01.052721+0000","last_clean_scrub_stamp":"2023-08-09T13:34:01.052721+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T13:44:49.753930+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1c","version":"0'0","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-08-09T13:34:08.212417+0000","last_change":"2023-08-09T13:34:03.162651+0000","last_active":"2023-08-09T13:34:08.212417+0000","last_peered":"2023-08-09T13:34:08.212417+0000","last_clean":"2023-08-09T13:34:08.212417+0000","last_became_active":"2023-08-09T13:34:03.161575+0000","last_became_peered":"2023-08-09T13:34:03.161575+0000","last_unstale":"2023-08-09T13:34:08.212417+0000","last_undegraded":"2023-08-09T13:34:08.212417+0000","last_fullsized":"2023-08-09T13:34:08.212417+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:34:02.064154+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:34:02.064154+0000","last_clean_scrub_stamp":"2023-08-09T13:34:02.064154+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-11T00:30:20.150060+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.1c","version":"0'0","reported_seq":13,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-08-09T13:34:08.372392+0000","last_change":"2023-08-09T13:34:07.242334+0000","last_active":"2023-08-09T13:34:08.372392+0000","last_peered":"2023-08-09T13:34:08.372392+0000","last_clean":"2023-08-09T13:34:08.372392+0000","last_became_active":"2023-08-09T13:34:07.241841+0000","last_became_peered":"2023-08-09T13:34:07.241841+0000","last_unstale":"2023-08-09T13:34:08.372392+0000","last_undegraded":"2023-08-09T13:34:08.372392+0000","last_fullsized":"2023-08-09T13:34:08.372392+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:34:06.173547+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:34:06.173547+0000","last_clean_scrub_stamp":"2023-08-09T13:34:06.173547+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T14:35:09.213789+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,2,0],"acting":[4,3,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.1a","version":"19'1","reported_seq":22,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-08-09T13:34:08.372077+0000","last_change":"2023-08-09T13:34:02.104004+0000","last_active":"2023-08-09T13:34:08.372077+0000","last_peered":"2023-08-09T13:34:08.372077+0000","last_clean":"2023-08-09T13:34:08.372077+0000","last_became_active":"2023-08-09T13:34:02.103701+0000","last_became_peered":"2023-08-09T13:34:02.103701+0000","last_unstale":"2023-08-09T13:34:08.372077+0000","last_undegraded":"2023-08-09T13:34:08.372077+0000","last_fullsized":"2023-08-09T13:34:08.372077+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:34:01.052721+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:34:01.052721+0000","last_clean_scrub_stamp":"2023-08-09T13:34:01.052721+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T18:58:30.540971+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.1d","version":"0'0","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-08-09T13:34:08.371028+0000","last_change":"2023-08-09T13:34:03.156422+0000","last_active":"2023-08-09T13:34:08.371028+0000","last_peered":"2023-08-09T13:34:08.371028+0000","last_clean":"2023-08-09T13:34:08.371028+0000","last_became_active":"2023-08-09T13:34:03.155091+0000","last_became_peered":"2023-08-09T13:34:03.155091+0000","last_unstale":"2023-08-09T13:34:08.371028+0000","last_undegraded":"2023-08-09T13:34:08.371028+0000","last_fullsized":"2023-08-09T13:34:08.371028+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:34:02.064154+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:34:02.064154+0000","last_clean_scrub_stamp":"2023-08-09T13:34:02.064154+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T15:06:40.553178+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.1f","version":"0'0","reported_seq":13,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-08-09T13:34:08.213034+0000","last_change":"2023-08-09T13:34:07.241585+0000","last_active":"2023-08-09T13:34:08.213034+0000","last_peered":"2023-08-09T13:34:08.213034+0000","last_clean":"2023-08-09T13:34:08.213034+0000","last_became_active":"2023-08-09T13:34:07.241206+0000","last_became_peered":"2023-08-09T13:34:07.241206+0000","last_unstale":"2023-08-09T13:34:08.213034+0000","last_undegraded":"2023-08-09T13:34:08.213034+0000","last_fullsized":"2023-08-09T13:34:08.213034+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:34:06.173547+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:34:06.173547+0000","last_clean_scrub_stamp":"2023-08-09T13:34:06.173547+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-11T00:43:38.951241+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,3,7],"acting":[6,4,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":21,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-08-09T13:34:08.368085+0000","last_change":"2023-08-09T13:34:02.104251+0000","last_active":"2023-08-09T13:34:08.368085+0000","last_peered":"2023-08-09T13:34:08.368085+0000","last_clean":"2023-08-09T13:34:08.368085+0000","last_became_active":"2023-08-09T13:34:02.103920+0000","last_became_peered":"2023-08-09T13:34:02.103920+0000","last_unstale":"2023-08-09T13:34:08.368085+0000","last_undegraded":"2023-08-09T13:34:08.368085+0000","last_fullsized":"2023-08-09T13:34:08.368085+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:34:01.052721+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:34:01.052721+0000","last_clean_scrub_stamp":"2023-08-09T13:34:01.052721+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T18:36:38.900810+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1e","version":"0'0","reported_seq":17,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-08-09T13:34:08.208574+0000","last_change":"2023-08-09T13:34:03.172919+0000","last_active":"2023-08-09T13:34:08.208574+0000","last_peered":"2023-08-09T13:34:08.208574+0000","last_clean":"2023-08-09T13:34:08.208574+0000","last_became_active":"2023-08-09T13:34:03.172587+0000","last_became_peered":"2023-08-09T13:34:03.172587+0000","last_unstale":"2023-08-09T13:34:08.208574+0000","last_undegraded":"2023-08-09T13:34:08.208574+0000","last_fullsized":"2023-08-09T13:34:08.208574+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:34:02.064154+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:34:02.064154+0000","last_clean_scrub_stamp":"2023-08-09T13:34:02.064154+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T15:53:25.307248+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.1e","version":"0'0","reported_seq":13,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-08-09T13:34:08.209416+0000","last_change":"2023-08-09T13:34:07.236079+0000","last_active":"2023-08-09T13:34:08.209416+0000","last_peered":"2023-08-09T13:34:08.209416+0000","last_clean":"2023-08-09T13:34:08.209416+0000","last_became_active":"2023-08-09T13:34:07.234955+0000","last_became_peered":"2023-08-09T13:34:07.234955+0000","last_unstale":"2023-08-09T13:34:08.209416+0000","last_undegraded":"2023-08-09T13:34:08.209416+0000","last_fullsized":"2023-08-09T13:34:08.209416+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:34:06.173547+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:34:06.173547+0000","last_clean_scrub_stamp":"2023-08-09T13:34:06.173547+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T22:30:10.569066+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,1,2],"acting":[0,7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":21,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-08-09T13:34:08.368549+0000","last_change":"2023-08-09T13:34:02.103043+0000","last_active":"2023-08-09T13:34:08.368549+0000","last_peered":"2023-08-09T13:34:08.368549+0000","last_clean":"2023-08-09T13:34:08.368549+0000","last_became_active":"2023-08-09T13:34:02.102724+0000","last_became_peered":"2023-08-09T13:34:02.102724+0000","last_unstale":"2023-08-09T13:34:08.368549+0000","last_undegraded":"2023-08-09T13:34:08.368549+0000","last_fullsized":"2023-08-09T13:34:08.368549+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:34:01.052721+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:34:01.052721+0000","last_clean_scrub_stamp":"2023-08-09T13:34:01.052721+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T17:59:27.409131+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1f","version":"0'0","reported_seq":17,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-08-09T13:34:08.213083+0000","last_change":"2023-08-09T13:34:03.150663+0000","last_active":"2023-08-09T13:34:08.213083+0000","last_peered":"2023-08-09T13:34:08.213083+0000","last_clean":"2023-08-09T13:34:08.213083+0000","last_became_active":"2023-08-09T13:34:03.150322+0000","last_became_peered":"2023-08-09T13:34:03.150322+0000","last_unstale":"2023-08-09T13:34:08.213083+0000","last_undegraded":"2023-08-09T13:34:08.213083+0000","last_fullsized":"2023-08-09T13:34:08.213083+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:34:02.064154+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:34:02.064154+0000","last_clean_scrub_stamp":"2023-08-09T13:34:02.064154+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T22:09:11.225000+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.10","version":"0'0","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-08-09T13:34:08.368129+0000","last_change":"2023-08-09T13:34:03.154639+0000","last_active":"2023-08-09T13:34:08.368129+0000","last_peered":"2023-08-09T13:34:08.368129+0000","last_clean":"2023-08-09T13:34:08.368129+0000","last_became_active":"2023-08-09T13:34:03.151967+0000","last_became_peered":"2023-08-09T13:34:03.151967+0000","last_unstale":"2023-08-09T13:34:08.368129+0000","last_undegraded":"2023-08-09T13:34:08.368129+0000","last_fullsized":"2023-08-09T13:34:08.368129+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:34:02.064154+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:34:02.064154+0000","last_clean_scrub_stamp":"2023-08-09T13:34:02.064154+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-11T00:11:21.887434+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-08-09T13:34:08.210224+0000","last_change":"2023-08-09T13:34:02.108763+0000","last_active":"2023-08-09T13:34:08.210224+0000","last_peered":"2023-08-09T13:34:08.210224+0000","last_clean":"2023-08-09T13:34:08.210224+0000","last_became_active":"2023-08-09T13:34:02.108439+0000","last_became_peered":"2023-08-09T13:34:02.108439+0000","last_unstale":"2023-08-09T13:34:08.210224+0000","last_undegraded":"2023-08-09T13:34:08.210224+0000","last_fullsized":"2023-08-09T13:34:08.210224+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:34:01.052721+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:34:01.052721+0000","last_clean_scrub_stamp":"2023-08-09T13:34:01.052721+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T17:08:06.492548+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.11","version":"0'0","reported_seq":13,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-08-09T13:34:08.211048+0000","last_change":"2023-08-09T13:34:07.241202+0000","last_active":"2023-08-09T13:34:08.211048+0000","last_peered":"2023-08-09T13:34:08.211048+0000","last_clean":"2023-08-09T13:34:08.211048+0000","last_became_active":"2023-08-09T13:34:07.240678+0000","last_became_peered":"2023-08-09T13:34:07.240678+0000","last_unstale":"2023-08-09T13:34:08.211048+0000","last_undegraded":"2023-08-09T13:34:08.211048+0000","last_fullsized":"2023-08-09T13:34:08.211048+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:34:06.173547+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:34:06.173547+0000","last_clean_scrub_stamp":"2023-08-09T13:34:06.173547+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T22:29:57.384386+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,7,1],"acting":[6,4,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-08-09T13:34:08.369040+0000","last_change":"2023-08-09T13:34:03.163640+0000","last_active":"2023-08-09T13:34:08.369040+0000","last_peered":"2023-08-09T13:34:08.369040+0000","last_clean":"2023-08-09T13:34:08.369040+0000","last_became_active":"2023-08-09T13:34:03.163333+0000","last_became_peered":"2023-08-09T13:34:03.163333+0000","last_unstale":"2023-08-09T13:34:08.369040+0000","last_undegraded":"2023-08-09T13:34:08.369040+0000","last_fullsized":"2023-08-09T13:34:08.369040+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:34:02.064154+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:34:02.064154+0000","last_clean_scrub_stamp":"2023-08-09T13:34:02.064154+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T17:55:41.057906+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-08-09T13:34:08.371698+0000","last_change":"2023-08-09T13:34:02.112947+0000","last_active":"2023-08-09T13:34:08.371698+0000","last_peered":"2023-08-09T13:34:08.371698+0000","last_clean":"2023-08-09T13:34:08.371698+0000","last_became_active":"2023-08-09T13:34:02.112545+0000","last_became_peered":"2023-08-09T13:34:02.112545+0000","last_unstale":"2023-08-09T13:34:08.371698+0000","last_undegraded":"2023-08-09T13:34:08.371698+0000","last_fullsized":"2023-08-09T13:34:08.371698+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:34:01.052721+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:34:01.052721+0000","last_clean_scrub_stamp":"2023-08-09T13:34:01.052721+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T18:07:30.693736+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.10","version":"0'0","reported_seq":13,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-08-09T13:34:08.367894+0000","last_change":"2023-08-09T13:34:07.231662+0000","last_active":"2023-08-09T13:34:08.367894+0000","last_peered":"2023-08-09T13:34:08.367894+0000","last_clean":"2023-08-09T13:34:08.367894+0000","last_became_active":"2023-08-09T13:34:07.231214+0000","last_became_peered":"2023-08-09T13:34:07.231214+0000","last_unstale":"2023-08-09T13:34:08.367894+0000","last_undegraded":"2023-08-09T13:34:08.367894+0000","last_fullsized":"2023-08-09T13:34:08.367894+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:34:06.173547+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:34:06.173547+0000","last_clean_scrub_stamp":"2023-08-09T13:34:06.173547+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T21:35:05.575947+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6,0],"acting":[7,4,6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.15","version":"19'3","reported_seq":22,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-08-09T13:34:08.370972+0000","last_change":"2023-08-09T13:34:02.112344+0000","last_active":"2023-08-09T13:34:08.370972+0000","last_peered":"2023-08-09T13:34:08.370972+0000","last_clean":"2023-08-09T13:34:08.370972+0000","last_became_active":"2023-08-09T13:34:02.111749+0000","last_became_peered":"2023-08-09T13:34:02.111749+0000","last_unstale":"2023-08-09T13:34:08.370972+0000","last_undegraded":"2023-08-09T13:34:08.370972+0000","last_fullsized":"2023-08-09T13:34:08.370972+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:34:01.052721+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:34:01.052721+0000","last_clean_scrub_stamp":"2023-08-09T13:34:01.052721+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T15:33:08.276561+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-08-09T13:34:08.369248+0000","last_change":"2023-08-09T13:34:03.154995+0000","last_active":"2023-08-09T13:34:08.369248+0000","last_peered":"2023-08-09T13:34:08.369248+0000","last_clean":"2023-08-09T13:34:08.369248+0000","last_became_active":"2023-08-09T13:34:03.154715+0000","last_became_peered":"2023-08-09T13:34:03.154715+0000","last_unstale":"2023-08-09T13:34:08.369248+0000","last_undegraded":"2023-08-09T13:34:08.369248+0000","last_fullsized":"2023-08-09T13:34:08.369248+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:34:02.064154+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:34:02.064154+0000","last_clean_scrub_stamp":"2023-08-09T13:34:02.064154+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T22:30:06.459673+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.13","version":"0'0","reported_seq":13,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-08-09T13:34:08.368692+0000","last_change":"2023-08-09T13:34:07.241979+0000","last_active":"2023-08-09T13:34:08.368692+0000","last_peered":"2023-08-09T13:34:08.368692+0000","last_clean":"2023-08-09T13:34:08.368692+0000","last_became_active":"2023-08-09T13:34:07.241025+0000","last_became_peered":"2023-08-09T13:34:07.241025+0000","last_unstale":"2023-08-09T13:34:08.368692+0000","last_undegraded":"2023-08-09T13:34:08.368692+0000","last_fullsized":"2023-08-09T13:34:08.368692+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:34:06.173547+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:34:06.173547+0000","last_clean_scrub_stamp":"2023-08-09T13:34:06.173547+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T17:42:24.789866+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,1,5],"acting":[3,0,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.14","version":"24'10","reported_seq":31,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-08-09T13:34:22.970697+0000","last_change":"2023-08-09T13:34:02.113338+0000","last_active":"2023-08-09T13:34:22.970697+0000","last_peered":"2023-08-09T13:34:22.970697+0000","last_clean":"2023-08-09T13:34:22.970697+0000","last_became_active":"2023-08-09T13:34:02.113042+0000","last_became_peered":"2023-08-09T13:34:02.113042+0000","last_unstale":"2023-08-09T13:34:22.970697+0000","last_undegraded":"2023-08-09T13:34:22.970697+0000","last_fullsized":"2023-08-09T13:34:22.970697+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:34:01.052721+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:34:01.052721+0000","last_clean_scrub_stamp":"2023-08-09T13:34:01.052721+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T21:15:54.123137+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":3841,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.13","version":"0'0","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-08-09T13:34:08.371930+0000","last_change":"2023-08-09T13:34:03.148233+0000","last_active":"2023-08-09T13:34:08.371930+0000","last_peered":"2023-08-09T13:34:08.371930+0000","last_clean":"2023-08-09T13:34:08.371930+0000","last_became_active":"2023-08-09T13:34:03.147860+0000","last_became_peered":"2023-08-09T13:34:03.147860+0000","last_unstale":"2023-08-09T13:34:08.371930+0000","last_undegraded":"2023-08-09T13:34:08.371930+0000","last_fullsized":"2023-08-09T13:34:08.371930+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:34:02.064154+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:34:02.064154+0000","last_clean_scrub_stamp":"2023-08-09T13:34:02.064154+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T18:17:08.524229+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.12","version":"0'0","reported_seq":13,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-08-09T13:34:08.368474+0000","last_change":"2023-08-09T13:34:07.236661+0000","last_active":"2023-08-09T13:34:08.368474+0000","last_peered":"2023-08-09T13:34:08.368474+0000","last_clean":"2023-08-09T13:34:08.368474+0000","last_became_active":"2023-08-09T13:34:07.236264+0000","last_became_peered":"2023-08-09T13:34:07.236264+0000","last_unstale":"2023-08-09T13:34:08.368474+0000","last_undegraded":"2023-08-09T13:34:08.368474+0000","last_fullsized":"2023-08-09T13:34:08.368474+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:34:06.173547+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:34:06.173547+0000","last_clean_scrub_stamp":"2023-08-09T13:34:06.173547+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T21:35:05.575947+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,3,7],"acting":[1,5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-08-09T13:34:08.369602+0000","last_change":"2023-08-09T13:34:03.149954+0000","last_active":"2023-08-09T13:34:08.369602+0000","last_peered":"2023-08-09T13:34:08.369602+0000","last_clean":"2023-08-09T13:34:08.369602+0000","last_became_active":"2023-08-09T13:34:03.149648+0000","last_became_peered":"2023-08-09T13:34:03.149648+0000","last_unstale":"2023-08-09T13:34:08.369602+0000","last_undegraded":"2023-08-09T13:34:08.369602+0000","last_fullsized":"2023-08-09T13:34:08.369602+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:34:02.064154+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:34:02.064154+0000","last_clean_scrub_stamp":"2023-08-09T13:34:02.064154+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T18:50:05.285807+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":25,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-08-09T13:34:08.213159+0000","last_change":"2023-08-09T13:33:55.886389+0000","last_active":"2023-08-09T13:34:08.213159+0000","last_peered":"2023-08-09T13:34:08.213159+0000","last_clean":"2023-08-09T13:34:08.213159+0000","last_became_active":"2023-08-09T13:33:55.885638+0000","last_became_peered":"2023-08-09T13:33:55.885638+0000","last_unstale":"2023-08-09T13:34:08.213159+0000","last_undegraded":"2023-08-09T13:34:08.213159+0000","last_fullsized":"2023-08-09T13:34:08.213159+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:33:54.845869+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:33:54.845869+0000","last_clean_scrub_stamp":"2023-08-09T13:33:54.845869+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T23:11:03.836310+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-08-09T13:34:08.370437+0000","last_change":"2023-08-09T13:34:02.114759+0000","last_active":"2023-08-09T13:34:08.370437+0000","last_peered":"2023-08-09T13:34:08.370437+0000","last_clean":"2023-08-09T13:34:08.370437+0000","last_became_active":"2023-08-09T13:34:02.114423+0000","last_became_peered":"2023-08-09T13:34:02.114423+0000","last_unstale":"2023-08-09T13:34:08.370437+0000","last_undegraded":"2023-08-09T13:34:08.370437+0000","last_fullsized":"2023-08-09T13:34:08.370437+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:34:01.052721+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:34:01.052721+0000","last_clean_scrub_stamp":"2023-08-09T13:34:01.052721+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T18:50:04.274374+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.3","version":"0'0","reported_seq":13,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-08-09T13:34:08.209248+0000","last_change":"2023-08-09T13:34:07.244416+0000","last_active":"2023-08-09T13:34:08.209248+0000","last_peered":"2023-08-09T13:34:08.209248+0000","last_clean":"2023-08-09T13:34:08.209248+0000","last_became_active":"2023-08-09T13:34:07.243294+0000","last_became_peered":"2023-08-09T13:34:07.243294+0000","last_unstale":"2023-08-09T13:34:08.209248+0000","last_undegraded":"2023-08-09T13:34:08.209248+0000","last_fullsized":"2023-08-09T13:34:08.209248+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:34:06.173547+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:34:06.173547+0000","last_clean_scrub_stamp":"2023-08-09T13:34:06.173547+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T21:35:05.575947+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,6,5],"acting":[0,1,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":17,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-08-09T13:34:08.209747+0000","last_change":"2023-08-09T13:34:03.151674+0000","last_active":"2023-08-09T13:34:08.209747+0000","last_peered":"2023-08-09T13:34:08.209747+0000","last_clean":"2023-08-09T13:34:08.209747+0000","last_became_active":"2023-08-09T13:34:03.151137+0000","last_became_peered":"2023-08-09T13:34:03.151137+0000","last_unstale":"2023-08-09T13:34:08.209747+0000","last_undegraded":"2023-08-09T13:34:08.209747+0000","last_fullsized":"2023-08-09T13:34:08.209747+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:34:02.064154+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:34:02.064154+0000","last_clean_scrub_stamp":"2023-08-09T13:34:02.064154+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T21:11:40.494329+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":25,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-08-09T13:34:08.212939+0000","last_change":"2023-08-09T13:33:56.401781+0000","last_active":"2023-08-09T13:34:08.212939+0000","last_peered":"2023-08-09T13:34:08.212939+0000","last_clean":"2023-08-09T13:34:08.212939+0000","last_became_active":"2023-08-09T13:33:56.400336+0000","last_became_peered":"2023-08-09T13:33:56.400336+0000","last_unstale":"2023-08-09T13:34:08.212939+0000","last_undegraded":"2023-08-09T13:34:08.212939+0000","last_fullsized":"2023-08-09T13:34:08.212939+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:33:54.845869+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:33:54.845869+0000","last_clean_scrub_stamp":"2023-08-09T13:33:54.845869+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T19:43:07.034435+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":21,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-08-09T13:34:08.372061+0000","last_change":"2023-08-09T13:34:02.106587+0000","last_active":"2023-08-09T13:34:08.372061+0000","last_peered":"2023-08-09T13:34:08.372061+0000","last_clean":"2023-08-09T13:34:08.372061+0000","last_became_active":"2023-08-09T13:34:02.106236+0000","last_became_peered":"2023-08-09T13:34:02.106236+0000","last_unstale":"2023-08-09T13:34:08.372061+0000","last_undegraded":"2023-08-09T13:34:08.372061+0000","last_fullsized":"2023-08-09T13:34:08.372061+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:34:01.052721+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:34:01.052721+0000","last_clean_scrub_stamp":"2023-08-09T13:34:01.052721+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T13:44:49.753930+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.2","version":"0'0","reported_seq":13,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-08-09T13:34:08.211286+0000","last_change":"2023-08-09T13:34:07.242738+0000","last_active":"2023-08-09T13:34:08.211286+0000","last_peered":"2023-08-09T13:34:08.211286+0000","last_clean":"2023-08-09T13:34:08.211286+0000","last_became_active":"2023-08-09T13:34:07.242314+0000","last_became_peered":"2023-08-09T13:34:07.242314+0000","last_unstale":"2023-08-09T13:34:08.211286+0000","last_undegraded":"2023-08-09T13:34:08.211286+0000","last_fullsized":"2023-08-09T13:34:08.211286+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:34:06.173547+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:34:06.173547+0000","last_clean_scrub_stamp":"2023-08-09T13:34:06.173547+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T18:00:37.387381+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,5,2],"acting":[6,0,5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-08-09T13:34:08.368126+0000","last_change":"2023-08-09T13:34:03.163805+0000","last_active":"2023-08-09T13:34:08.368126+0000","last_peered":"2023-08-09T13:34:08.368126+0000","last_clean":"2023-08-09T13:34:08.368126+0000","last_became_active":"2023-08-09T13:34:03.163407+0000","last_became_peered":"2023-08-09T13:34:03.163407+0000","last_unstale":"2023-08-09T13:34:08.368126+0000","last_undegraded":"2023-08-09T13:34:08.368126+0000","last_fullsized":"2023-08-09T13:34:08.368126+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:34:02.064154+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:34:02.064154+0000","last_clean_scrub_stamp":"2023-08-09T13:34:02.064154+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T23:33:49.011321+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":27,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-08-09T13:34:08.373438+0000","last_change":"2023-08-09T13:33:56.404236+0000","last_active":"2023-08-09T13:34:08.373438+0000","last_peered":"2023-08-09T13:34:08.373438+0000","last_clean":"2023-08-09T13:34:08.373438+0000","last_became_active":"2023-08-09T13:33:56.402485+0000","last_became_peered":"2023-08-09T13:33:56.402485+0000","last_unstale":"2023-08-09T13:34:08.373438+0000","last_undegraded":"2023-08-09T13:34:08.373438+0000","last_fullsized":"2023-08-09T13:34:08.373438+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:33:54.845869+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:33:54.845869+0000","last_clean_scrub_stamp":"2023-08-09T13:33:54.845869+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T18:59:14.817691+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.7","version":"19'2","reported_seq":23,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-08-09T13:34:08.369861+0000","last_change":"2023-08-09T13:34:02.108061+0000","last_active":"2023-08-09T13:34:08.369861+0000","last_peered":"2023-08-09T13:34:08.369861+0000","last_clean":"2023-08-09T13:34:08.369861+0000","last_became_active":"2023-08-09T13:34:02.107766+0000","last_became_peered":"2023-08-09T13:34:02.107766+0000","last_unstale":"2023-08-09T13:34:08.369861+0000","last_undegraded":"2023-08-09T13:34:08.369861+0000","last_fullsized":"2023-08-09T13:34:08.369861+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:34:01.052721+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:34:01.052721+0000","last_clean_scrub_stamp":"2023-08-09T13:34:01.052721+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T16:14:05.720686+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.1","version":"0'0","reported_seq":13,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-08-09T13:34:08.368770+0000","last_change":"2023-08-09T13:34:07.236332+0000","last_active":"2023-08-09T13:34:08.368770+0000","last_peered":"2023-08-09T13:34:08.368770+0000","last_clean":"2023-08-09T13:34:08.368770+0000","last_became_active":"2023-08-09T13:34:07.235923+0000","last_became_peered":"2023-08-09T13:34:07.235923+0000","last_unstale":"2023-08-09T13:34:08.368770+0000","last_undegraded":"2023-08-09T13:34:08.368770+0000","last_fullsized":"2023-08-09T13:34:08.368770+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:34:06.173547+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:34:06.173547+0000","last_clean_scrub_stamp":"2023-08-09T13:34:06.173547+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-11T01:07:37.438752+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,7,0],"acting":[4,3,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-08-09T13:34:08.369699+0000","last_change":"2023-08-09T13:34:03.149492+0000","last_active":"2023-08-09T13:34:08.369699+0000","last_peered":"2023-08-09T13:34:08.369699+0000","last_clean":"2023-08-09T13:34:08.369699+0000","last_became_active":"2023-08-09T13:34:03.148649+0000","last_became_peered":"2023-08-09T13:34:03.148649+0000","last_unstale":"2023-08-09T13:34:08.369699+0000","last_undegraded":"2023-08-09T13:34:08.369699+0000","last_fullsized":"2023-08-09T13:34:08.369699+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:34:02.064154+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:34:02.064154+0000","last_clean_scrub_stamp":"2023-08-09T13:34:02.064154+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T21:57:54.183369+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":27,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-08-09T13:34:08.370409+0000","last_change":"2023-08-09T13:33:55.889565+0000","last_active":"2023-08-09T13:34:08.370409+0000","last_peered":"2023-08-09T13:34:08.370409+0000","last_clean":"2023-08-09T13:34:08.370409+0000","last_became_active":"2023-08-09T13:33:55.889164+0000","last_became_peered":"2023-08-09T13:33:55.889164+0000","last_unstale":"2023-08-09T13:34:08.370409+0000","last_undegraded":"2023-08-09T13:34:08.370409+0000","last_fullsized":"2023-08-09T13:34:08.370409+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:33:54.845869+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:33:54.845869+0000","last_clean_scrub_stamp":"2023-08-09T13:33:54.845869+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-11T01:07:26.111074+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.6","version":"21'1","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-08-09T13:34:08.209027+0000","last_change":"2023-08-09T13:34:02.096482+0000","last_active":"2023-08-09T13:34:08.209027+0000","last_peered":"2023-08-09T13:34:08.209027+0000","last_clean":"2023-08-09T13:34:08.209027+0000","last_became_active":"2023-08-09T13:34:02.095287+0000","last_became_peered":"2023-08-09T13:34:02.095287+0000","last_unstale":"2023-08-09T13:34:08.209027+0000","last_undegraded":"2023-08-09T13:34:08.209027+0000","last_fullsized":"2023-08-09T13:34:08.209027+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:34:01.052721+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:34:01.052721+0000","last_clean_scrub_stamp":"2023-08-09T13:34:01.052721+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T18:36:38.900810+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.0","version":"0'0","reported_seq":13,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-08-09T13:34:08.367857+0000","last_change":"2023-08-09T13:34:07.244568+0000","last_active":"2023-08-09T13:34:08.367857+0000","last_peered":"2023-08-09T13:34:08.367857+0000","last_clean":"2023-08-09T13:34:08.367857+0000","last_became_active":"2023-08-09T13:34:07.243669+0000","last_became_peered":"2023-08-09T13:34:07.243669+0000","last_unstale":"2023-08-09T13:34:08.367857+0000","last_undegraded":"2023-08-09T13:34:08.367857+0000","last_fullsized":"2023-08-09T13:34:08.367857+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:34:06.173547+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:34:06.173547+0000","last_clean_scrub_stamp":"2023-08-09T13:34:06.173547+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T20:31:33.361620+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,4,0],"acting":[3,1,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.6","version":"0'0","reported_seq":17,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-08-09T13:34:08.208668+0000","last_change":"2023-08-09T13:34:03.139485+0000","last_active":"2023-08-09T13:34:08.208668+0000","last_peered":"2023-08-09T13:34:08.208668+0000","last_clean":"2023-08-09T13:34:08.208668+0000","last_became_active":"2023-08-09T13:34:03.139178+0000","last_became_peered":"2023-08-09T13:34:03.139178+0000","last_unstale":"2023-08-09T13:34:08.208668+0000","last_undegraded":"2023-08-09T13:34:08.208668+0000","last_fullsized":"2023-08-09T13:34:08.208668+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:34:02.064154+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:34:02.064154+0000","last_clean_scrub_stamp":"2023-08-09T13:34:02.064154+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T19:42:33.733326+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-08-09T13:34:08.207236+0000","last_change":"2023-08-09T13:34:02.110719+0000","last_active":"2023-08-09T13:34:08.207236+0000","last_peered":"2023-08-09T13:34:08.207236+0000","last_clean":"2023-08-09T13:34:08.207236+0000","last_became_active":"2023-08-09T13:34:02.110431+0000","last_became_peered":"2023-08-09T13:34:02.110431+0000","last_unstale":"2023-08-09T13:34:08.207236+0000","last_undegraded":"2023-08-09T13:34:08.207236+0000","last_fullsized":"2023-08-09T13:34:08.207236+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:34:01.052721+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:34:01.052721+0000","last_clean_scrub_stamp":"2023-08-09T13:34:01.052721+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T23:24:07.266385+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.0","version":"16'117","reported_seq":175,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-08-09T13:34:08.368906+0000","last_change":"2023-08-09T13:33:55.886345+0000","last_active":"2023-08-09T13:34:08.368906+0000","last_peered":"2023-08-09T13:34:08.368906+0000","last_clean":"2023-08-09T13:34:08.368906+0000","last_became_active":"2023-08-09T13:33:55.885781+0000","last_became_peered":"2023-08-09T13:33:55.885781+0000","last_unstale":"2023-08-09T13:34:08.368906+0000","last_undegraded":"2023-08-09T13:34:08.368906+0000","last_fullsized":"2023-08-09T13:34:08.368906+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:33:54.845869+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:33:54.845869+0000","last_clean_scrub_stamp":"2023-08-09T13:33:54.845869+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T18:49:58.067523+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.3","version":"0'0","reported_seq":27,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-08-09T13:34:08.370928+0000","last_change":"2023-08-09T13:33:55.886749+0000","last_active":"2023-08-09T13:34:08.370928+0000","last_peered":"2023-08-09T13:34:08.370928+0000","last_clean":"2023-08-09T13:34:08.370928+0000","last_became_active":"2023-08-09T13:33:55.886116+0000","last_became_peered":"2023-08-09T13:33:55.886116+0000","last_unstale":"2023-08-09T13:34:08.370928+0000","last_undegraded":"2023-08-09T13:34:08.370928+0000","last_fullsized":"2023-08-09T13:34:08.370928+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:33:54.845869+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:33:54.845869+0000","last_clean_scrub_stamp":"2023-08-09T13:33:54.845869+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T17:46:15.820922+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.7","version":"0'0","reported_seq":13,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-08-09T13:34:08.371420+0000","last_change":"2023-08-09T13:34:07.235784+0000","last_active":"2023-08-09T13:34:08.371420+0000","last_peered":"2023-08-09T13:34:08.371420+0000","last_clean":"2023-08-09T13:34:08.371420+0000","last_became_active":"2023-08-09T13:34:07.235282+0000","last_became_peered":"2023-08-09T13:34:07.235282+0000","last_unstale":"2023-08-09T13:34:08.371420+0000","last_undegraded":"2023-08-09T13:34:08.371420+0000","last_fullsized":"2023-08-09T13:34:08.371420+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:34:06.173547+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:34:06.173547+0000","last_clean_scrub_stamp":"2023-08-09T13:34:06.173547+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T23:52:31.847779+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,0,4],"acting":[5,1,0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.7","version":"0'0","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-08-09T13:34:08.372407+0000","last_change":"2023-08-09T13:34:03.149568+0000","last_active":"2023-08-09T13:34:08.372407+0000","last_peered":"2023-08-09T13:34:08.372407+0000","last_clean":"2023-08-09T13:34:08.372407+0000","last_became_active":"2023-08-09T13:34:03.148937+0000","last_became_peered":"2023-08-09T13:34:03.148937+0000","last_unstale":"2023-08-09T13:34:08.372407+0000","last_undegraded":"2023-08-09T13:34:08.372407+0000","last_fullsized":"2023-08-09T13:34:08.372407+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:34:02.064154+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:34:02.064154+0000","last_clean_scrub_stamp":"2023-08-09T13:34:02.064154+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T23:34:54.506258+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":21,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-08-09T13:34:08.367679+0000","last_change":"2023-08-09T13:34:02.102795+0000","last_active":"2023-08-09T13:34:08.367679+0000","last_peered":"2023-08-09T13:34:08.367679+0000","last_clean":"2023-08-09T13:34:08.367679+0000","last_became_active":"2023-08-09T13:34:02.101883+0000","last_became_peered":"2023-08-09T13:34:02.101883+0000","last_unstale":"2023-08-09T13:34:08.367679+0000","last_undegraded":"2023-08-09T13:34:08.367679+0000","last_fullsized":"2023-08-09T13:34:08.367679+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:34:01.052721+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:34:01.052721+0000","last_clean_scrub_stamp":"2023-08-09T13:34:01.052721+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T19:22:07.585850+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.2","version":"0'0","reported_seq":25,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-08-09T13:34:08.369310+0000","last_change":"2023-08-09T13:33:56.402794+0000","last_active":"2023-08-09T13:34:08.369310+0000","last_peered":"2023-08-09T13:34:08.369310+0000","last_clean":"2023-08-09T13:34:08.369310+0000","last_became_active":"2023-08-09T13:33:56.402038+0000","last_became_peered":"2023-08-09T13:33:56.402038+0000","last_unstale":"2023-08-09T13:34:08.369310+0000","last_undegraded":"2023-08-09T13:34:08.369310+0000","last_fullsized":"2023-08-09T13:34:08.369310+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:33:54.845869+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:33:54.845869+0000","last_clean_scrub_stamp":"2023-08-09T13:33:54.845869+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T18:07:24.486884+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.6","version":"0'0","reported_seq":13,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-08-09T13:34:08.212013+0000","last_change":"2023-08-09T13:34:07.244168+0000","last_active":"2023-08-09T13:34:08.212013+0000","last_peered":"2023-08-09T13:34:08.212013+0000","last_clean":"2023-08-09T13:34:08.212013+0000","last_became_active":"2023-08-09T13:34:07.241903+0000","last_became_peered":"2023-08-09T13:34:07.241903+0000","last_unstale":"2023-08-09T13:34:08.212013+0000","last_undegraded":"2023-08-09T13:34:08.212013+0000","last_fullsized":"2023-08-09T13:34:08.212013+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:34:06.173547+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:34:06.173547+0000","last_clean_scrub_stamp":"2023-08-09T13:34:06.173547+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-11T00:23:22.192340+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,1,7],"acting":[2,5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-08-09T13:34:08.371697+0000","last_change":"2023-08-09T13:34:03.153039+0000","last_active":"2023-08-09T13:34:08.371697+0000","last_peered":"2023-08-09T13:34:08.371697+0000","last_clean":"2023-08-09T13:34:08.371697+0000","last_became_active":"2023-08-09T13:34:03.152401+0000","last_became_peered":"2023-08-09T13:34:03.152401+0000","last_unstale":"2023-08-09T13:34:08.371697+0000","last_undegraded":"2023-08-09T13:34:08.371697+0000","last_fullsized":"2023-08-09T13:34:08.371697+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:34:02.064154+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:34:02.064154+0000","last_clean_scrub_stamp":"2023-08-09T13:34:02.064154+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T21:18:19.551549+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":21,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-08-09T13:34:08.372303+0000","last_change":"2023-08-09T13:34:02.108657+0000","last_active":"2023-08-09T13:34:08.372303+0000","last_peered":"2023-08-09T13:34:08.372303+0000","last_clean":"2023-08-09T13:34:08.372303+0000","last_became_active":"2023-08-09T13:34:02.105481+0000","last_became_peered":"2023-08-09T13:34:02.105481+0000","last_unstale":"2023-08-09T13:34:08.372303+0000","last_undegraded":"2023-08-09T13:34:08.372303+0000","last_fullsized":"2023-08-09T13:34:08.372303+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:34:01.052721+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:34:01.052721+0000","last_clean_scrub_stamp":"2023-08-09T13:34:01.052721+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T22:51:18.775151+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":25,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-08-09T13:34:08.368026+0000","last_change":"2023-08-09T13:33:56.401123+0000","last_active":"2023-08-09T13:34:08.368026+0000","last_peered":"2023-08-09T13:34:08.368026+0000","last_clean":"2023-08-09T13:34:08.368026+0000","last_became_active":"2023-08-09T13:33:56.399933+0000","last_became_peered":"2023-08-09T13:33:56.399933+0000","last_unstale":"2023-08-09T13:34:08.368026+0000","last_undegraded":"2023-08-09T13:34:08.368026+0000","last_fullsized":"2023-08-09T13:34:08.368026+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:33:54.845869+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:33:54.845869+0000","last_clean_scrub_stamp":"2023-08-09T13:33:54.845869+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T23:24:01.059533+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.5","version":"0'0","reported_seq":13,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-08-09T13:34:08.208281+0000","last_change":"2023-08-09T13:34:07.235418+0000","last_active":"2023-08-09T13:34:08.208281+0000","last_peered":"2023-08-09T13:34:08.208281+0000","last_clean":"2023-08-09T13:34:08.208281+0000","last_became_active":"2023-08-09T13:34:07.234889+0000","last_became_peered":"2023-08-09T13:34:07.234889+0000","last_unstale":"2023-08-09T13:34:08.208281+0000","last_undegraded":"2023-08-09T13:34:08.208281+0000","last_fullsized":"2023-08-09T13:34:08.208281+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:34:06.173547+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:34:06.173547+0000","last_clean_scrub_stamp":"2023-08-09T13:34:06.173547+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T20:46:19.756813+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,4,6],"acting":[0,1,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.5","version":"0'0","reported_seq":17,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-08-09T13:34:08.212552+0000","last_change":"2023-08-09T13:34:03.159726+0000","last_active":"2023-08-09T13:34:08.212552+0000","last_peered":"2023-08-09T13:34:08.212552+0000","last_clean":"2023-08-09T13:34:08.212552+0000","last_became_active":"2023-08-09T13:34:03.159238+0000","last_became_peered":"2023-08-09T13:34:03.159238+0000","last_unstale":"2023-08-09T13:34:08.212552+0000","last_undegraded":"2023-08-09T13:34:08.212552+0000","last_fullsized":"2023-08-09T13:34:08.212552+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:34:02.064154+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:34:02.064154+0000","last_clean_scrub_stamp":"2023-08-09T13:34:02.064154+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T17:32:16.845908+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":21,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-08-09T13:34:08.368904+0000","last_change":"2023-08-09T13:34:02.114840+0000","last_active":"2023-08-09T13:34:08.368904+0000","last_peered":"2023-08-09T13:34:08.368904+0000","last_clean":"2023-08-09T13:34:08.368904+0000","last_became_active":"2023-08-09T13:34:02.114547+0000","last_became_peered":"2023-08-09T13:34:02.114547+0000","last_unstale":"2023-08-09T13:34:08.368904+0000","last_undegraded":"2023-08-09T13:34:08.368904+0000","last_fullsized":"2023-08-09T13:34:08.368904+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:34:01.052721+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:34:01.052721+0000","last_clean_scrub_stamp":"2023-08-09T13:34:01.052721+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T13:43:19.071421+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":25,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-08-09T13:34:08.369269+0000","last_change":"2023-08-09T13:33:56.401406+0000","last_active":"2023-08-09T13:34:08.369269+0000","last_peered":"2023-08-09T13:34:08.369269+0000","last_clean":"2023-08-09T13:34:08.369269+0000","last_became_active":"2023-08-09T13:33:56.400487+0000","last_became_peered":"2023-08-09T13:33:56.400487+0000","last_unstale":"2023-08-09T13:34:08.369269+0000","last_undegraded":"2023-08-09T13:34:08.369269+0000","last_fullsized":"2023-08-09T13:34:08.369269+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:33:54.845869+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:33:54.845869+0000","last_clean_scrub_stamp":"2023-08-09T13:33:54.845869+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T23:52:20.520101+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.4","version":"0'0","reported_seq":13,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-08-09T13:34:08.370501+0000","last_change":"2023-08-09T13:34:07.249799+0000","last_active":"2023-08-09T13:34:08.370501+0000","last_peered":"2023-08-09T13:34:08.370501+0000","last_clean":"2023-08-09T13:34:08.370501+0000","last_became_active":"2023-08-09T13:34:07.249286+0000","last_became_peered":"2023-08-09T13:34:07.249286+0000","last_unstale":"2023-08-09T13:34:08.370501+0000","last_undegraded":"2023-08-09T13:34:08.370501+0000","last_fullsized":"2023-08-09T13:34:08.370501+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:34:06.173547+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:34:06.173547+0000","last_clean_scrub_stamp":"2023-08-09T13:34:06.173547+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T15:43:48.184972+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2,5,3],"acting":[7,2,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.f","version":"0'0","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-08-09T13:34:08.372421+0000","last_change":"2023-08-09T13:34:03.151508+0000","last_active":"2023-08-09T13:34:08.372421+0000","last_peered":"2023-08-09T13:34:08.372421+0000","last_clean":"2023-08-09T13:34:08.372421+0000","last_became_active":"2023-08-09T13:34:03.149550+0000","last_became_peered":"2023-08-09T13:34:03.149550+0000","last_unstale":"2023-08-09T13:34:08.372421+0000","last_undegraded":"2023-08-09T13:34:08.372421+0000","last_fullsized":"2023-08-09T13:34:08.372421+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:34:02.064154+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:34:02.064154+0000","last_clean_scrub_stamp":"2023-08-09T13:34:02.064154+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T22:51:11.201456+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":21,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-08-09T13:34:08.368991+0000","last_change":"2023-08-09T13:34:02.095946+0000","last_active":"2023-08-09T13:34:08.368991+0000","last_peered":"2023-08-09T13:34:08.368991+0000","last_clean":"2023-08-09T13:34:08.368991+0000","last_became_active":"2023-08-09T13:34:02.095325+0000","last_became_peered":"2023-08-09T13:34:02.095325+0000","last_unstale":"2023-08-09T13:34:08.368991+0000","last_undegraded":"2023-08-09T13:34:08.368991+0000","last_fullsized":"2023-08-09T13:34:08.368991+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:34:01.052721+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:34:01.052721+0000","last_clean_scrub_stamp":"2023-08-09T13:34:01.052721+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-11T00:06:29.000035+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.e","version":"0'0","reported_seq":13,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-08-09T13:34:08.373803+0000","last_change":"2023-08-09T13:34:07.232690+0000","last_active":"2023-08-09T13:34:08.373803+0000","last_peered":"2023-08-09T13:34:08.373803+0000","last_clean":"2023-08-09T13:34:08.373803+0000","last_became_active":"2023-08-09T13:34:07.232326+0000","last_became_peered":"2023-08-09T13:34:07.232326+0000","last_unstale":"2023-08-09T13:34:08.373803+0000","last_undegraded":"2023-08-09T13:34:08.373803+0000","last_fullsized":"2023-08-09T13:34:08.373803+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:34:06.173547+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:34:06.173547+0000","last_clean_scrub_stamp":"2023-08-09T13:34:06.173547+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T13:37:38.237696+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,0,2],"acting":[4,5,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.e","version":"0'0","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-08-09T13:34:08.373562+0000","last_change":"2023-08-09T13:34:03.149791+0000","last_active":"2023-08-09T13:34:08.373562+0000","last_peered":"2023-08-09T13:34:08.373562+0000","last_clean":"2023-08-09T13:34:08.373562+0000","last_became_active":"2023-08-09T13:34:03.149525+0000","last_became_peered":"2023-08-09T13:34:03.149525+0000","last_unstale":"2023-08-09T13:34:08.373562+0000","last_undegraded":"2023-08-09T13:34:08.373562+0000","last_fullsized":"2023-08-09T13:34:08.373562+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:34:02.064154+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:34:02.064154+0000","last_clean_scrub_stamp":"2023-08-09T13:34:02.064154+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T14:53:59.337109+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":21,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-08-09T13:34:08.369963+0000","last_change":"2023-08-09T13:34:02.105359+0000","last_active":"2023-08-09T13:34:08.369963+0000","last_peered":"2023-08-09T13:34:08.369963+0000","last_clean":"2023-08-09T13:34:08.369963+0000","last_became_active":"2023-08-09T13:34:02.105027+0000","last_became_peered":"2023-08-09T13:34:02.105027+0000","last_unstale":"2023-08-09T13:34:08.369963+0000","last_undegraded":"2023-08-09T13:34:08.369963+0000","last_fullsized":"2023-08-09T13:34:08.369963+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:34:01.052721+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:34:01.052721+0000","last_clean_scrub_stamp":"2023-08-09T13:34:01.052721+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T13:45:26.240878+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.f","version":"0'0","reported_seq":13,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-08-09T13:34:08.371141+0000","last_change":"2023-08-09T13:34:07.239625+0000","last_active":"2023-08-09T13:34:08.371141+0000","last_peered":"2023-08-09T13:34:08.371141+0000","last_clean":"2023-08-09T13:34:08.371141+0000","last_became_active":"2023-08-09T13:34:07.239002+0000","last_became_peered":"2023-08-09T13:34:07.239002+0000","last_unstale":"2023-08-09T13:34:08.371141+0000","last_undegraded":"2023-08-09T13:34:08.371141+0000","last_fullsized":"2023-08-09T13:34:08.371141+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:34:06.173547+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:34:06.173547+0000","last_clean_scrub_stamp":"2023-08-09T13:34:06.173547+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T21:11:44.603723+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,6,2],"acting":[5,4,6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.d","version":"0'0","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-08-09T13:34:08.373090+0000","last_change":"2023-08-09T13:34:03.152579+0000","last_active":"2023-08-09T13:34:08.373090+0000","last_peered":"2023-08-09T13:34:08.373090+0000","last_clean":"2023-08-09T13:34:08.373090+0000","last_became_active":"2023-08-09T13:34:03.152287+0000","last_became_peered":"2023-08-09T13:34:03.152287+0000","last_unstale":"2023-08-09T13:34:08.373090+0000","last_undegraded":"2023-08-09T13:34:08.373090+0000","last_fullsized":"2023-08-09T13:34:08.373090+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:34:02.064154+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:34:02.064154+0000","last_clean_scrub_stamp":"2023-08-09T13:34:02.064154+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T17:37:51.284066+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-08-09T13:34:08.211685+0000","last_change":"2023-08-09T13:34:02.109847+0000","last_active":"2023-08-09T13:34:08.211685+0000","last_peered":"2023-08-09T13:34:08.211685+0000","last_clean":"2023-08-09T13:34:08.211685+0000","last_became_active":"2023-08-09T13:34:02.109516+0000","last_became_peered":"2023-08-09T13:34:02.109516+0000","last_unstale":"2023-08-09T13:34:08.211685+0000","last_undegraded":"2023-08-09T13:34:08.211685+0000","last_fullsized":"2023-08-09T13:34:08.211685+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:34:01.052721+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:34:01.052721+0000","last_clean_scrub_stamp":"2023-08-09T13:34:01.052721+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T22:40:21.488794+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.c","version":"0'0","reported_seq":13,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-08-09T13:34:08.371235+0000","last_change":"2023-08-09T13:34:07.231138+0000","last_active":"2023-08-09T13:34:08.371235+0000","last_peered":"2023-08-09T13:34:08.371235+0000","last_clean":"2023-08-09T13:34:08.371235+0000","last_became_active":"2023-08-09T13:34:07.230741+0000","last_became_peered":"2023-08-09T13:34:07.230741+0000","last_unstale":"2023-08-09T13:34:08.371235+0000","last_undegraded":"2023-08-09T13:34:08.371235+0000","last_fullsized":"2023-08-09T13:34:08.371235+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:34:06.173547+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:34:06.173547+0000","last_clean_scrub_stamp":"2023-08-09T13:34:06.173547+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T21:11:44.603723+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4,0,6],"acting":[1,4,0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.c","version":"0'0","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-08-09T13:34:08.372932+0000","last_change":"2023-08-09T13:34:03.159095+0000","last_active":"2023-08-09T13:34:08.372932+0000","last_peered":"2023-08-09T13:34:08.372932+0000","last_clean":"2023-08-09T13:34:08.372932+0000","last_became_active":"2023-08-09T13:34:03.158807+0000","last_became_peered":"2023-08-09T13:34:03.158807+0000","last_unstale":"2023-08-09T13:34:08.372932+0000","last_undegraded":"2023-08-09T13:34:08.372932+0000","last_fullsized":"2023-08-09T13:34:08.372932+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:34:02.064154+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:34:02.064154+0000","last_clean_scrub_stamp":"2023-08-09T13:34:02.064154+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T21:06:37.115870+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.b","version":"19'2","reported_seq":23,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-08-09T13:34:08.369938+0000","last_change":"2023-08-09T13:34:02.103259+0000","last_active":"2023-08-09T13:34:08.369938+0000","last_peered":"2023-08-09T13:34:08.369938+0000","last_clean":"2023-08-09T13:34:08.369938+0000","last_became_active":"2023-08-09T13:34:02.102644+0000","last_became_peered":"2023-08-09T13:34:02.102644+0000","last_unstale":"2023-08-09T13:34:08.369938+0000","last_undegraded":"2023-08-09T13:34:08.369938+0000","last_fullsized":"2023-08-09T13:34:08.369938+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:34:01.052721+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:34:01.052721+0000","last_clean_scrub_stamp":"2023-08-09T13:34:01.052721+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T19:48:11.160512+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.d","version":"0'0","reported_seq":13,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-08-09T13:34:08.211821+0000","last_change":"2023-08-09T13:34:07.242458+0000","last_active":"2023-08-09T13:34:08.211821+0000","last_peered":"2023-08-09T13:34:08.211821+0000","last_clean":"2023-08-09T13:34:08.211821+0000","last_became_active":"2023-08-09T13:34:07.241393+0000","last_became_peered":"2023-08-09T13:34:07.241393+0000","last_unstale":"2023-08-09T13:34:08.211821+0000","last_undegraded":"2023-08-09T13:34:08.211821+0000","last_fullsized":"2023-08-09T13:34:08.211821+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:34:06.173547+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:34:06.173547+0000","last_clean_scrub_stamp":"2023-08-09T13:34:06.173547+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T19:00:12.926085+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7,5,6],"acting":[2,7,5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.b","version":"0'0","reported_seq":17,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-08-09T13:34:08.209500+0000","last_change":"2023-08-09T13:34:03.155061+0000","last_active":"2023-08-09T13:34:08.209500+0000","last_peered":"2023-08-09T13:34:08.209500+0000","last_clean":"2023-08-09T13:34:08.209500+0000","last_became_active":"2023-08-09T13:34:03.154792+0000","last_became_peered":"2023-08-09T13:34:03.154792+0000","last_unstale":"2023-08-09T13:34:08.209500+0000","last_undegraded":"2023-08-09T13:34:08.209500+0000","last_fullsized":"2023-08-09T13:34:08.209500+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:34:02.064154+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:34:02.064154+0000","last_clean_scrub_stamp":"2023-08-09T13:34:02.064154+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T18:50:05.285807+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-08-09T13:34:08.369085+0000","last_change":"2023-08-09T13:34:02.111888+0000","last_active":"2023-08-09T13:34:08.369085+0000","last_peered":"2023-08-09T13:34:08.369085+0000","last_clean":"2023-08-09T13:34:08.369085+0000","last_became_active":"2023-08-09T13:34:02.111588+0000","last_became_peered":"2023-08-09T13:34:02.111588+0000","last_unstale":"2023-08-09T13:34:08.369085+0000","last_undegraded":"2023-08-09T13:34:08.369085+0000","last_fullsized":"2023-08-09T13:34:08.369085+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:34:01.052721+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:34:01.052721+0000","last_clean_scrub_stamp":"2023-08-09T13:34:01.052721+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T15:53:24.295815+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.a","version":"0'0","reported_seq":13,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-08-09T13:34:08.211349+0000","last_change":"2023-08-09T13:34:07.241400+0000","last_active":"2023-08-09T13:34:08.211349+0000","last_peered":"2023-08-09T13:34:08.211349+0000","last_clean":"2023-08-09T13:34:08.211349+0000","last_became_active":"2023-08-09T13:34:07.240945+0000","last_became_peered":"2023-08-09T13:34:07.240945+0000","last_unstale":"2023-08-09T13:34:08.211349+0000","last_undegraded":"2023-08-09T13:34:08.211349+0000","last_fullsized":"2023-08-09T13:34:08.211349+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:34:06.173547+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:34:06.173547+0000","last_clean_scrub_stamp":"2023-08-09T13:34:06.173547+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T20:24:07.327648+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4,3,6],"acting":[2,4,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":17,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-08-09T13:34:08.211229+0000","last_change":"2023-08-09T13:34:03.159173+0000","last_active":"2023-08-09T13:34:08.211229+0000","last_peered":"2023-08-09T13:34:08.211229+0000","last_clean":"2023-08-09T13:34:08.211229+0000","last_became_active":"2023-08-09T13:34:03.157726+0000","last_became_peered":"2023-08-09T13:34:03.157726+0000","last_unstale":"2023-08-09T13:34:08.211229+0000","last_undegraded":"2023-08-09T13:34:08.211229+0000","last_fullsized":"2023-08-09T13:34:08.211229+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:34:02.064154+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:34:02.064154+0000","last_clean_scrub_stamp":"2023-08-09T13:34:02.064154+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T18:49:41.829768+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.d","version":"19'1","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-08-09T13:34:08.371386+0000","last_change":"2023-08-09T13:34:02.115845+0000","last_active":"2023-08-09T13:34:08.371386+0000","last_peered":"2023-08-09T13:34:08.371386+0000","last_clean":"2023-08-09T13:34:08.371386+0000","last_became_active":"2023-08-09T13:34:02.115519+0000","last_became_peered":"2023-08-09T13:34:02.115519+0000","last_unstale":"2023-08-09T13:34:08.371386+0000","last_undegraded":"2023-08-09T13:34:08.371386+0000","last_fullsized":"2023-08-09T13:34:08.371386+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:34:01.052721+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:34:01.052721+0000","last_clean_scrub_stamp":"2023-08-09T13:34:01.052721+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T22:00:42.967697+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.b","version":"0'0","reported_seq":13,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-08-09T13:34:08.211258+0000","last_change":"2023-08-09T13:34:07.245045+0000","last_active":"2023-08-09T13:34:08.211258+0000","last_peered":"2023-08-09T13:34:08.211258+0000","last_clean":"2023-08-09T13:34:08.211258+0000","last_became_active":"2023-08-09T13:34:07.242923+0000","last_became_peered":"2023-08-09T13:34:07.242923+0000","last_unstale":"2023-08-09T13:34:08.211258+0000","last_undegraded":"2023-08-09T13:34:08.211258+0000","last_fullsized":"2023-08-09T13:34:08.211258+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:34:06.173547+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:34:06.173547+0000","last_clean_scrub_stamp":"2023-08-09T13:34:06.173547+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T20:20:56.644233+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,5,6],"acting":[2,0,5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-08-09T13:34:08.369085+0000","last_change":"2023-08-09T13:34:03.157124+0000","last_active":"2023-08-09T13:34:08.369085+0000","last_peered":"2023-08-09T13:34:08.369085+0000","last_clean":"2023-08-09T13:34:08.369085+0000","last_became_active":"2023-08-09T13:34:03.156039+0000","last_became_peered":"2023-08-09T13:34:03.156039+0000","last_unstale":"2023-08-09T13:34:08.369085+0000","last_undegraded":"2023-08-09T13:34:08.369085+0000","last_fullsized":"2023-08-09T13:34:08.369085+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:34:02.064154+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:34:02.064154+0000","last_clean_scrub_stamp":"2023-08-09T13:34:02.064154+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T17:18:45.337355+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.e","version":"19'2","reported_seq":21,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-08-09T13:34:08.371732+0000","last_change":"2023-08-09T13:34:02.116789+0000","last_active":"2023-08-09T13:34:08.371732+0000","last_peered":"2023-08-09T13:34:08.371732+0000","last_clean":"2023-08-09T13:34:08.371732+0000","last_became_active":"2023-08-09T13:34:02.116543+0000","last_became_peered":"2023-08-09T13:34:02.116543+0000","last_unstale":"2023-08-09T13:34:08.371732+0000","last_undegraded":"2023-08-09T13:34:08.371732+0000","last_fullsized":"2023-08-09T13:34:08.371732+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:34:01.052721+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:34:01.052721+0000","last_clean_scrub_stamp":"2023-08-09T13:34:01.052721+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T21:52:55.252712+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.8","version":"0'0","reported_seq":13,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-08-09T13:34:08.211204+0000","last_change":"2023-08-09T13:34:07.241851+0000","last_active":"2023-08-09T13:34:08.211204+0000","last_peered":"2023-08-09T13:34:08.211204+0000","last_clean":"2023-08-09T13:34:08.211204+0000","last_became_active":"2023-08-09T13:34:07.241329+0000","last_became_peered":"2023-08-09T13:34:07.241329+0000","last_unstale":"2023-08-09T13:34:08.211204+0000","last_undegraded":"2023-08-09T13:34:08.211204+0000","last_fullsized":"2023-08-09T13:34:08.211204+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:34:06.173547+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:34:06.173547+0000","last_clean_scrub_stamp":"2023-08-09T13:34:06.173547+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T20:54:41.878672+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,1,5],"acting":[2,0,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":17,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-08-09T13:34:08.368062+0000","last_change":"2023-08-09T13:34:03.158365+0000","last_active":"2023-08-09T13:34:08.368062+0000","last_peered":"2023-08-09T13:34:08.368062+0000","last_clean":"2023-08-09T13:34:08.368062+0000","last_became_active":"2023-08-09T13:34:03.158042+0000","last_became_peered":"2023-08-09T13:34:03.158042+0000","last_unstale":"2023-08-09T13:34:08.368062+0000","last_undegraded":"2023-08-09T13:34:08.368062+0000","last_fullsized":"2023-08-09T13:34:08.368062+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:34:02.064154+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:34:02.064154+0000","last_clean_scrub_stamp":"2023-08-09T13:34:02.064154+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T22:36:24.124090+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.f","version":"19'1","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-08-09T13:34:08.373105+0000","last_change":"2023-08-09T13:34:02.116130+0000","last_active":"2023-08-09T13:34:08.373105+0000","last_peered":"2023-08-09T13:34:08.373105+0000","last_clean":"2023-08-09T13:34:08.373105+0000","last_became_active":"2023-08-09T13:34:02.115758+0000","last_became_peered":"2023-08-09T13:34:02.115758+0000","last_unstale":"2023-08-09T13:34:08.373105+0000","last_undegraded":"2023-08-09T13:34:08.373105+0000","last_fullsized":"2023-08-09T13:34:08.373105+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:34:01.052721+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:34:01.052721+0000","last_clean_scrub_stamp":"2023-08-09T13:34:01.052721+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-11T00:08:17.437427+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.9","version":"0'0","reported_seq":13,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-08-09T13:34:08.369072+0000","last_change":"2023-08-09T13:34:07.241507+0000","last_active":"2023-08-09T13:34:08.369072+0000","last_peered":"2023-08-09T13:34:08.369072+0000","last_clean":"2023-08-09T13:34:08.369072+0000","last_became_active":"2023-08-09T13:34:07.240810+0000","last_became_peered":"2023-08-09T13:34:07.240810+0000","last_unstale":"2023-08-09T13:34:08.369072+0000","last_undegraded":"2023-08-09T13:34:08.369072+0000","last_fullsized":"2023-08-09T13:34:08.369072+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:34:06.173547+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:34:06.173547+0000","last_clean_scrub_stamp":"2023-08-09T13:34:06.173547+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T17:46:27.148600+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,3,4],"acting":[7,6,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.16","version":"0'0","reported_seq":13,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-08-09T13:34:08.370805+0000","last_change":"2023-08-09T13:34:07.238069+0000","last_active":"2023-08-09T13:34:08.370805+0000","last_peered":"2023-08-09T13:34:08.370805+0000","last_clean":"2023-08-09T13:34:08.370805+0000","last_became_active":"2023-08-09T13:34:07.237560+0000","last_became_peered":"2023-08-09T13:34:07.237560+0000","last_unstale":"2023-08-09T13:34:08.370805+0000","last_undegraded":"2023-08-09T13:34:08.370805+0000","last_fullsized":"2023-08-09T13:34:08.370805+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:34:06.173547+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:34:06.173547+0000","last_clean_scrub_stamp":"2023-08-09T13:34:06.173547+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T17:46:27.148600+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,1,6],"acting":[5,3,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.10","version":"19'8","reported_seq":27,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-08-09T13:34:08.211427+0000","last_change":"2023-08-09T13:34:02.103042+0000","last_active":"2023-08-09T13:34:08.211427+0000","last_peered":"2023-08-09T13:34:08.211427+0000","last_clean":"2023-08-09T13:34:08.211427+0000","last_became_active":"2023-08-09T13:34:02.102730+0000","last_became_peered":"2023-08-09T13:34:02.102730+0000","last_unstale":"2023-08-09T13:34:08.211427+0000","last_undegraded":"2023-08-09T13:34:08.211427+0000","last_fullsized":"2023-08-09T13:34:08.211427+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:34:01.052721+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:34:01.052721+0000","last_clean_scrub_stamp":"2023-08-09T13:34:01.052721+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T19:01:23.853811+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-08-09T13:34:08.369924+0000","last_change":"2023-08-09T13:34:03.163917+0000","last_active":"2023-08-09T13:34:08.369924+0000","last_peered":"2023-08-09T13:34:08.369924+0000","last_clean":"2023-08-09T13:34:08.369924+0000","last_became_active":"2023-08-09T13:34:03.163598+0000","last_became_peered":"2023-08-09T13:34:03.163598+0000","last_unstale":"2023-08-09T13:34:08.369924+0000","last_undegraded":"2023-08-09T13:34:08.369924+0000","last_fullsized":"2023-08-09T13:34:08.369924+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:34:02.064154+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:34:02.064154+0000","last_clean_scrub_stamp":"2023-08-09T13:34:02.064154+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T20:01:36.200073+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.17","version":"0'0","reported_seq":13,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-08-09T13:34:08.369281+0000","last_change":"2023-08-09T13:34:07.249196+0000","last_active":"2023-08-09T13:34:08.369281+0000","last_peered":"2023-08-09T13:34:08.369281+0000","last_clean":"2023-08-09T13:34:08.369281+0000","last_became_active":"2023-08-09T13:34:07.248758+0000","last_became_peered":"2023-08-09T13:34:07.248758+0000","last_unstale":"2023-08-09T13:34:08.369281+0000","last_undegraded":"2023-08-09T13:34:08.369281+0000","last_fullsized":"2023-08-09T13:34:08.369281+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:34:06.173547+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:34:06.173547+0000","last_clean_scrub_stamp":"2023-08-09T13:34:06.173547+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T19:50:51.608237+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,7,0],"acting":[3,1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-08-09T13:34:08.369394+0000","last_change":"2023-08-09T13:34:02.112054+0000","last_active":"2023-08-09T13:34:08.369394+0000","last_peered":"2023-08-09T13:34:08.369394+0000","last_clean":"2023-08-09T13:34:08.369394+0000","last_became_active":"2023-08-09T13:34:02.111753+0000","last_became_peered":"2023-08-09T13:34:02.111753+0000","last_unstale":"2023-08-09T13:34:08.369394+0000","last_undegraded":"2023-08-09T13:34:08.369394+0000","last_fullsized":"2023-08-09T13:34:08.369394+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:34:01.052721+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:34:01.052721+0000","last_clean_scrub_stamp":"2023-08-09T13:34:01.052721+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T17:08:06.492548+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":17,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-08-09T13:34:08.208919+0000","last_change":"2023-08-09T13:34:03.151791+0000","last_active":"2023-08-09T13:34:08.208919+0000","last_peered":"2023-08-09T13:34:08.208919+0000","last_clean":"2023-08-09T13:34:08.208919+0000","last_became_active":"2023-08-09T13:34:03.151265+0000","last_became_peered":"2023-08-09T13:34:03.151265+0000","last_unstale":"2023-08-09T13:34:08.208919+0000","last_undegraded":"2023-08-09T13:34:08.208919+0000","last_fullsized":"2023-08-09T13:34:08.208919+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:34:02.064154+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:34:02.064154+0000","last_clean_scrub_stamp":"2023-08-09T13:34:02.064154+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T19:22:08.597283+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-08-09T13:34:08.207862+0000","last_change":"2023-08-09T13:34:02.109334+0000","last_active":"2023-08-09T13:34:08.207862+0000","last_peered":"2023-08-09T13:34:08.207862+0000","last_clean":"2023-08-09T13:34:08.207862+0000","last_became_active":"2023-08-09T13:34:02.108390+0000","last_became_peered":"2023-08-09T13:34:02.108390+0000","last_unstale":"2023-08-09T13:34:08.207862+0000","last_undegraded":"2023-08-09T13:34:08.207862+0000","last_fullsized":"2023-08-09T13:34:08.207862+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:34:01.052721+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:34:01.052721+0000","last_clean_scrub_stamp":"2023-08-09T13:34:01.052721+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T22:36:23.112658+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.15","version":"0'0","reported_seq":17,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-08-09T13:34:08.370140+0000","last_change":"2023-08-09T13:34:03.150611+0000","last_active":"2023-08-09T13:34:08.370140+0000","last_peered":"2023-08-09T13:34:08.370140+0000","last_clean":"2023-08-09T13:34:08.370140+0000","last_became_active":"2023-08-09T13:34:03.150016+0000","last_became_peered":"2023-08-09T13:34:03.150016+0000","last_unstale":"2023-08-09T13:34:08.370140+0000","last_undegraded":"2023-08-09T13:34:08.370140+0000","last_fullsized":"2023-08-09T13:34:08.370140+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:34:02.064154+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:34:02.064154+0000","last_clean_scrub_stamp":"2023-08-09T13:34:02.064154+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T21:35:01.466554+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.14","version":"0'0","reported_seq":13,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-08-09T13:34:08.369980+0000","last_change":"2023-08-09T13:34:07.251464+0000","last_active":"2023-08-09T13:34:08.369980+0000","last_peered":"2023-08-09T13:34:08.369980+0000","last_clean":"2023-08-09T13:34:08.369980+0000","last_became_active":"2023-08-09T13:34:07.251035+0000","last_became_peered":"2023-08-09T13:34:07.251035+0000","last_unstale":"2023-08-09T13:34:08.369980+0000","last_undegraded":"2023-08-09T13:34:08.369980+0000","last_fullsized":"2023-08-09T13:34:08.369980+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:34:06.173547+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:34:06.173547+0000","last_clean_scrub_stamp":"2023-08-09T13:34:06.173547+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T15:06:44.996627+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,2,1],"acting":[3,7,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.13","version":"19'2","reported_seq":21,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-08-09T13:34:08.370349+0000","last_change":"2023-08-09T13:34:02.115081+0000","last_active":"2023-08-09T13:34:08.370349+0000","last_peered":"2023-08-09T13:34:08.370349+0000","last_clean":"2023-08-09T13:34:08.370349+0000","last_became_active":"2023-08-09T13:34:02.114825+0000","last_became_peered":"2023-08-09T13:34:02.114825+0000","last_unstale":"2023-08-09T13:34:08.370349+0000","last_undegraded":"2023-08-09T13:34:08.370349+0000","last_fullsized":"2023-08-09T13:34:08.370349+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:34:01.052721+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:34:01.052721+0000","last_clean_scrub_stamp":"2023-08-09T13:34:01.052721+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T22:51:10.190023+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.14","version":"0'0","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-08-09T13:34:08.370421+0000","last_change":"2023-08-09T13:34:03.154171+0000","last_active":"2023-08-09T13:34:08.370421+0000","last_peered":"2023-08-09T13:34:08.370421+0000","last_clean":"2023-08-09T13:34:08.370421+0000","last_became_active":"2023-08-09T13:34:03.150789+0000","last_became_peered":"2023-08-09T13:34:03.150789+0000","last_unstale":"2023-08-09T13:34:08.370421+0000","last_undegraded":"2023-08-09T13:34:08.370421+0000","last_fullsized":"2023-08-09T13:34:08.370421+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:34:02.064154+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:34:02.064154+0000","last_clean_scrub_stamp":"2023-08-09T13:34:02.064154+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T17:18:45.337355+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.15","version":"0'0","reported_seq":13,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-08-09T13:34:08.369475+0000","last_change":"2023-08-09T13:34:07.241401+0000","last_active":"2023-08-09T13:34:08.369475+0000","last_peered":"2023-08-09T13:34:08.369475+0000","last_clean":"2023-08-09T13:34:08.369475+0000","last_became_active":"2023-08-09T13:34:07.240937+0000","last_became_peered":"2023-08-09T13:34:07.240937+0000","last_unstale":"2023-08-09T13:34:08.369475+0000","last_undegraded":"2023-08-09T13:34:08.369475+0000","last_fullsized":"2023-08-09T13:34:08.369475+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:34:06.173547+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:34:06.173547+0000","last_clean_scrub_stamp":"2023-08-09T13:34:06.173547+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T18:50:25.863045+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,3,0],"acting":[5,1,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.20","version":"0'0","reported_seq":17,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-08-09T13:34:08.210886+0000","last_change":"2023-08-09T13:34:03.157698+0000","last_active":"2023-08-09T13:34:08.210886+0000","last_peered":"2023-08-09T13:34:08.210886+0000","last_clean":"2023-08-09T13:34:08.210886+0000","last_became_active":"2023-08-09T13:34:03.153351+0000","last_became_peered":"2023-08-09T13:34:03.153351+0000","last_unstale":"2023-08-09T13:34:08.210886+0000","last_undegraded":"2023-08-09T13:34:08.210886+0000","last_fullsized":"2023-08-09T13:34:08.210886+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:34:02.064154+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:34:02.064154+0000","last_clean_scrub_stamp":"2023-08-09T13:34:02.064154+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T19:40:06.763332+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.21","version":"0'0","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-08-09T13:34:08.368201+0000","last_change":"2023-08-09T13:34:03.150065+0000","last_active":"2023-08-09T13:34:08.368201+0000","last_peered":"2023-08-09T13:34:08.368201+0000","last_clean":"2023-08-09T13:34:08.368201+0000","last_became_active":"2023-08-09T13:34:03.149650+0000","last_became_peered":"2023-08-09T13:34:03.149650+0000","last_unstale":"2023-08-09T13:34:08.368201+0000","last_undegraded":"2023-08-09T13:34:08.368201+0000","last_fullsized":"2023-08-09T13:34:08.368201+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:34:02.064154+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:34:02.064154+0000","last_clean_scrub_stamp":"2023-08-09T13:34:02.064154+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T13:47:48.941725+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.22","version":"0'0","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-08-09T13:34:08.372000+0000","last_change":"2023-08-09T13:34:03.152515+0000","last_active":"2023-08-09T13:34:08.372000+0000","last_peered":"2023-08-09T13:34:08.372000+0000","last_clean":"2023-08-09T13:34:08.372000+0000","last_became_active":"2023-08-09T13:34:03.152123+0000","last_became_peered":"2023-08-09T13:34:03.152123+0000","last_unstale":"2023-08-09T13:34:08.372000+0000","last_undegraded":"2023-08-09T13:34:08.372000+0000","last_fullsized":"2023-08-09T13:34:08.372000+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:34:02.064154+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:34:02.064154+0000","last_clean_scrub_stamp":"2023-08-09T13:34:02.064154+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T15:53:52.598510+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.23","version":"0'0","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-08-09T13:34:08.369171+0000","last_change":"2023-08-09T13:34:03.159919+0000","last_active":"2023-08-09T13:34:08.369171+0000","last_peered":"2023-08-09T13:34:08.369171+0000","last_clean":"2023-08-09T13:34:08.369171+0000","last_became_active":"2023-08-09T13:34:03.154344+0000","last_became_peered":"2023-08-09T13:34:03.154344+0000","last_unstale":"2023-08-09T13:34:08.369171+0000","last_undegraded":"2023-08-09T13:34:08.369171+0000","last_fullsized":"2023-08-09T13:34:08.369171+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:34:02.064154+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:34:02.064154+0000","last_clean_scrub_stamp":"2023-08-09T13:34:02.064154+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T22:33:11.246239+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.24","version":"0'0","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-08-09T13:34:08.371698+0000","last_change":"2023-08-09T13:34:03.164662+0000","last_active":"2023-08-09T13:34:08.371698+0000","last_peered":"2023-08-09T13:34:08.371698+0000","last_clean":"2023-08-09T13:34:08.371698+0000","last_became_active":"2023-08-09T13:34:03.163711+0000","last_became_peered":"2023-08-09T13:34:03.163711+0000","last_unstale":"2023-08-09T13:34:08.371698+0000","last_undegraded":"2023-08-09T13:34:08.371698+0000","last_fullsized":"2023-08-09T13:34:08.371698+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:34:02.064154+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:34:02.064154+0000","last_clean_scrub_stamp":"2023-08-09T13:34:02.064154+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T23:52:27.738385+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.25","version":"0'0","reported_seq":17,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-08-09T13:34:08.369863+0000","last_change":"2023-08-09T13:34:03.175502+0000","last_active":"2023-08-09T13:34:08.369863+0000","last_peered":"2023-08-09T13:34:08.369863+0000","last_clean":"2023-08-09T13:34:08.369863+0000","last_became_active":"2023-08-09T13:34:03.175192+0000","last_became_peered":"2023-08-09T13:34:03.175192+0000","last_unstale":"2023-08-09T13:34:08.369863+0000","last_undegraded":"2023-08-09T13:34:08.369863+0000","last_fullsized":"2023-08-09T13:34:08.369863+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:34:02.064154+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:34:02.064154+0000","last_clean_scrub_stamp":"2023-08-09T13:34:02.064154+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T19:22:08.597283+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.26","version":"0'0","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-08-09T13:34:08.372808+0000","last_change":"2023-08-09T13:34:03.155984+0000","last_active":"2023-08-09T13:34:08.372808+0000","last_peered":"2023-08-09T13:34:08.372808+0000","last_clean":"2023-08-09T13:34:08.372808+0000","last_became_active":"2023-08-09T13:34:03.155686+0000","last_became_peered":"2023-08-09T13:34:03.155686+0000","last_unstale":"2023-08-09T13:34:08.372808+0000","last_undegraded":"2023-08-09T13:34:08.372808+0000","last_fullsized":"2023-08-09T13:34:08.372808+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:34:02.064154+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:34:02.064154+0000","last_clean_scrub_stamp":"2023-08-09T13:34:02.064154+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T15:43:44.075578+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.27","version":"0'0","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-08-09T13:34:08.372180+0000","last_change":"2023-08-09T13:34:03.154003+0000","last_active":"2023-08-09T13:34:08.372180+0000","last_peered":"2023-08-09T13:34:08.372180+0000","last_clean":"2023-08-09T13:34:08.372180+0000","last_became_active":"2023-08-09T13:34:03.153707+0000","last_became_peered":"2023-08-09T13:34:03.153707+0000","last_unstale":"2023-08-09T13:34:08.372180+0000","last_undegraded":"2023-08-09T13:34:08.372180+0000","last_fullsized":"2023-08-09T13:34:08.372180+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:34:02.064154+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:34:02.064154+0000","last_clean_scrub_stamp":"2023-08-09T13:34:02.064154+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T17:08:07.503980+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.28","version":"0'0","reported_seq":17,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-08-09T13:34:08.368537+0000","last_change":"2023-08-09T13:34:03.170907+0000","last_active":"2023-08-09T13:34:08.368537+0000","last_peered":"2023-08-09T13:34:08.368537+0000","last_clean":"2023-08-09T13:34:08.368537+0000","last_became_active":"2023-08-09T13:34:03.162204+0000","last_became_peered":"2023-08-09T13:34:03.162204+0000","last_unstale":"2023-08-09T13:34:08.368537+0000","last_undegraded":"2023-08-09T13:34:08.368537+0000","last_fullsized":"2023-08-09T13:34:08.368537+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:34:02.064154+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:34:02.064154+0000","last_clean_scrub_stamp":"2023-08-09T13:34:02.064154+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T22:30:06.459673+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.29","version":"0'0","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-08-09T13:34:08.368673+0000","last_change":"2023-08-09T13:34:03.144064+0000","last_active":"2023-08-09T13:34:08.368673+0000","last_peered":"2023-08-09T13:34:08.368673+0000","last_clean":"2023-08-09T13:34:08.368673+0000","last_became_active":"2023-08-09T13:34:03.143612+0000","last_became_peered":"2023-08-09T13:34:03.143612+0000","last_unstale":"2023-08-09T13:34:08.368673+0000","last_undegraded":"2023-08-09T13:34:08.368673+0000","last_fullsized":"2023-08-09T13:34:08.368673+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:34:02.064154+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:34:02.064154+0000","last_clean_scrub_stamp":"2023-08-09T13:34:02.064154+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T18:50:26.789814+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.2a","version":"0'0","reported_seq":17,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-08-09T13:34:08.208605+0000","last_change":"2023-08-09T13:34:03.140127+0000","last_active":"2023-08-09T13:34:08.208605+0000","last_peered":"2023-08-09T13:34:08.208605+0000","last_clean":"2023-08-09T13:34:08.208605+0000","last_became_active":"2023-08-09T13:34:03.139616+0000","last_became_peered":"2023-08-09T13:34:03.139616+0000","last_unstale":"2023-08-09T13:34:08.208605+0000","last_undegraded":"2023-08-09T13:34:08.208605+0000","last_fullsized":"2023-08-09T13:34:08.208605+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:34:02.064154+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:34:02.064154+0000","last_clean_scrub_stamp":"2023-08-09T13:34:02.064154+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T18:50:21.753652+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.2b","version":"0'0","reported_seq":17,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-08-09T13:34:08.369974+0000","last_change":"2023-08-09T13:34:03.170969+0000","last_active":"2023-08-09T13:34:08.369974+0000","last_peered":"2023-08-09T13:34:08.369974+0000","last_clean":"2023-08-09T13:34:08.369974+0000","last_became_active":"2023-08-09T13:34:03.162334+0000","last_became_peered":"2023-08-09T13:34:03.162334+0000","last_unstale":"2023-08-09T13:34:08.369974+0000","last_undegraded":"2023-08-09T13:34:08.369974+0000","last_fullsized":"2023-08-09T13:34:08.369974+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:34:02.064154+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:34:02.064154+0000","last_clean_scrub_stamp":"2023-08-09T13:34:02.064154+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T23:34:54.506258+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.2c","version":"0'0","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-08-09T13:34:08.372790+0000","last_change":"2023-08-09T13:34:03.149273+0000","last_active":"2023-08-09T13:34:08.372790+0000","last_peered":"2023-08-09T13:34:08.372790+0000","last_clean":"2023-08-09T13:34:08.372790+0000","last_became_active":"2023-08-09T13:34:03.148460+0000","last_became_peered":"2023-08-09T13:34:03.148460+0000","last_unstale":"2023-08-09T13:34:08.372790+0000","last_undegraded":"2023-08-09T13:34:08.372790+0000","last_fullsized":"2023-08-09T13:34:08.372790+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:34:02.064154+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:34:02.064154+0000","last_clean_scrub_stamp":"2023-08-09T13:34:02.064154+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-11T01:12:24.927676+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.2d","version":"0'0","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-08-09T13:34:08.211794+0000","last_change":"2023-08-09T13:34:03.160727+0000","last_active":"2023-08-09T13:34:08.211794+0000","last_peered":"2023-08-09T13:34:08.211794+0000","last_clean":"2023-08-09T13:34:08.211794+0000","last_became_active":"2023-08-09T13:34:03.160406+0000","last_became_peered":"2023-08-09T13:34:03.160406+0000","last_unstale":"2023-08-09T13:34:08.211794+0000","last_undegraded":"2023-08-09T13:34:08.211794+0000","last_fullsized":"2023-08-09T13:34:08.211794+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:34:02.064154+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:34:02.064154+0000","last_clean_scrub_stamp":"2023-08-09T13:34:02.064154+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T20:28:43.534176+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.2e","version":"0'0","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-08-09T13:34:08.373221+0000","last_change":"2023-08-09T13:34:03.154714+0000","last_active":"2023-08-09T13:34:08.373221+0000","last_peered":"2023-08-09T13:34:08.373221+0000","last_clean":"2023-08-09T13:34:08.373221+0000","last_became_active":"2023-08-09T13:34:03.154384+0000","last_became_peered":"2023-08-09T13:34:03.154384+0000","last_unstale":"2023-08-09T13:34:08.373221+0000","last_undegraded":"2023-08-09T13:34:08.373221+0000","last_fullsized":"2023-08-09T13:34:08.373221+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:34:02.064154+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:34:02.064154+0000","last_clean_scrub_stamp":"2023-08-09T13:34:02.064154+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T16:21:31.389815+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.2f","version":"0'0","reported_seq":17,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-08-09T13:34:08.209885+0000","last_change":"2023-08-09T13:34:03.144822+0000","last_active":"2023-08-09T13:34:08.209885+0000","last_peered":"2023-08-09T13:34:08.209885+0000","last_clean":"2023-08-09T13:34:08.209885+0000","last_became_active":"2023-08-09T13:34:03.144470+0000","last_became_peered":"2023-08-09T13:34:03.144470+0000","last_unstale":"2023-08-09T13:34:08.209885+0000","last_undegraded":"2023-08-09T13:34:08.209885+0000","last_fullsized":"2023-08-09T13:34:08.209885+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:34:02.064154+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:34:02.064154+0000","last_clean_scrub_stamp":"2023-08-09T13:34:02.064154+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T21:18:19.551549+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.30","version":"0'0","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-08-09T13:34:08.370238+0000","last_change":"2023-08-09T13:34:03.140680+0000","last_active":"2023-08-09T13:34:08.370238+0000","last_peered":"2023-08-09T13:34:08.370238+0000","last_clean":"2023-08-09T13:34:08.370238+0000","last_became_active":"2023-08-09T13:34:03.139105+0000","last_became_peered":"2023-08-09T13:34:03.139105+0000","last_unstale":"2023-08-09T13:34:08.370238+0000","last_undegraded":"2023-08-09T13:34:08.370238+0000","last_fullsized":"2023-08-09T13:34:08.370238+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:34:02.064154+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:34:02.064154+0000","last_clean_scrub_stamp":"2023-08-09T13:34:02.064154+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T18:16:48.383340+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.31","version":"0'0","reported_seq":17,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-08-09T13:34:08.369342+0000","last_change":"2023-08-09T13:34:03.174569+0000","last_active":"2023-08-09T13:34:08.369342+0000","last_peered":"2023-08-09T13:34:08.369342+0000","last_clean":"2023-08-09T13:34:08.369342+0000","last_became_active":"2023-08-09T13:34:03.173450+0000","last_became_peered":"2023-08-09T13:34:03.173450+0000","last_unstale":"2023-08-09T13:34:08.369342+0000","last_undegraded":"2023-08-09T13:34:08.369342+0000","last_fullsized":"2023-08-09T13:34:08.369342+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:34:02.064154+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:34:02.064154+0000","last_clean_scrub_stamp":"2023-08-09T13:34:02.064154+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T13:44:50.765363+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.32","version":"0'0","reported_seq":17,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-08-09T13:34:08.368679+0000","last_change":"2023-08-09T13:34:03.156031+0000","last_active":"2023-08-09T13:34:08.368679+0000","last_peered":"2023-08-09T13:34:08.368679+0000","last_clean":"2023-08-09T13:34:08.368679+0000","last_became_active":"2023-08-09T13:34:03.155720+0000","last_became_peered":"2023-08-09T13:34:03.155720+0000","last_unstale":"2023-08-09T13:34:08.368679+0000","last_undegraded":"2023-08-09T13:34:08.368679+0000","last_fullsized":"2023-08-09T13:34:08.368679+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:34:02.064154+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:34:02.064154+0000","last_clean_scrub_stamp":"2023-08-09T13:34:02.064154+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T20:46:15.647419+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.33","version":"0'0","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-08-09T13:34:08.211563+0000","last_change":"2023-08-09T13:34:03.159410+0000","last_active":"2023-08-09T13:34:08.211563+0000","last_peered":"2023-08-09T13:34:08.211563+0000","last_clean":"2023-08-09T13:34:08.211563+0000","last_became_active":"2023-08-09T13:34:03.159006+0000","last_became_peered":"2023-08-09T13:34:03.159006+0000","last_unstale":"2023-08-09T13:34:08.211563+0000","last_undegraded":"2023-08-09T13:34:08.211563+0000","last_fullsized":"2023-08-09T13:34:08.211563+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:34:02.064154+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:34:02.064154+0000","last_clean_scrub_stamp":"2023-08-09T13:34:02.064154+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T21:55:49.311446+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.34","version":"0'0","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-08-09T13:34:08.212297+0000","last_change":"2023-08-09T13:34:03.165655+0000","last_active":"2023-08-09T13:34:08.212297+0000","last_peered":"2023-08-09T13:34:08.212297+0000","last_clean":"2023-08-09T13:34:08.212297+0000","last_became_active":"2023-08-09T13:34:03.165308+0000","last_became_peered":"2023-08-09T13:34:03.165308+0000","last_unstale":"2023-08-09T13:34:08.212297+0000","last_undegraded":"2023-08-09T13:34:08.212297+0000","last_fullsized":"2023-08-09T13:34:08.212297+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:34:02.064154+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:34:02.064154+0000","last_clean_scrub_stamp":"2023-08-09T13:34:02.064154+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T21:47:59.693735+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.35","version":"0'0","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-08-09T13:34:08.372839+0000","last_change":"2023-08-09T13:34:03.138842+0000","last_active":"2023-08-09T13:34:08.372839+0000","last_peered":"2023-08-09T13:34:08.372839+0000","last_clean":"2023-08-09T13:34:08.372839+0000","last_became_active":"2023-08-09T13:34:03.138554+0000","last_became_peered":"2023-08-09T13:34:03.138554+0000","last_unstale":"2023-08-09T13:34:08.372839+0000","last_undegraded":"2023-08-09T13:34:08.372839+0000","last_fullsized":"2023-08-09T13:34:08.372839+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:34:02.064154+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:34:02.064154+0000","last_clean_scrub_stamp":"2023-08-09T13:34:02.064154+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T23:05:06.664154+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.36","version":"0'0","reported_seq":17,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-08-09T13:34:08.371821+0000","last_change":"2023-08-09T13:34:03.148574+0000","last_active":"2023-08-09T13:34:08.371821+0000","last_peered":"2023-08-09T13:34:08.371821+0000","last_clean":"2023-08-09T13:34:08.371821+0000","last_became_active":"2023-08-09T13:34:03.148292+0000","last_became_peered":"2023-08-09T13:34:03.148292+0000","last_unstale":"2023-08-09T13:34:08.371821+0000","last_undegraded":"2023-08-09T13:34:08.371821+0000","last_fullsized":"2023-08-09T13:34:08.371821+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:34:02.064154+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:34:02.064154+0000","last_clean_scrub_stamp":"2023-08-09T13:34:02.064154+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T13:44:50.765363+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.37","version":"0'0","reported_seq":17,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-08-09T13:34:08.209550+0000","last_change":"2023-08-09T13:34:03.152770+0000","last_active":"2023-08-09T13:34:08.209550+0000","last_peered":"2023-08-09T13:34:08.209550+0000","last_clean":"2023-08-09T13:34:08.209550+0000","last_became_active":"2023-08-09T13:34:03.151703+0000","last_became_peered":"2023-08-09T13:34:03.151703+0000","last_unstale":"2023-08-09T13:34:08.209550+0000","last_undegraded":"2023-08-09T13:34:08.209550+0000","last_fullsized":"2023-08-09T13:34:08.209550+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:34:02.064154+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:34:02.064154+0000","last_clean_scrub_stamp":"2023-08-09T13:34:02.064154+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T17:46:23.039207+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.38","version":"0'0","reported_seq":17,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-08-09T13:34:08.367726+0000","last_change":"2023-08-09T13:34:03.146016+0000","last_active":"2023-08-09T13:34:08.367726+0000","last_peered":"2023-08-09T13:34:08.367726+0000","last_clean":"2023-08-09T13:34:08.367726+0000","last_became_active":"2023-08-09T13:34:03.145766+0000","last_became_peered":"2023-08-09T13:34:03.145766+0000","last_unstale":"2023-08-09T13:34:08.367726+0000","last_undegraded":"2023-08-09T13:34:08.367726+0000","last_fullsized":"2023-08-09T13:34:08.367726+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:34:02.064154+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:34:02.064154+0000","last_clean_scrub_stamp":"2023-08-09T13:34:02.064154+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T13:51:34.177397+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.39","version":"0'0","reported_seq":17,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-08-09T13:34:08.368967+0000","last_change":"2023-08-09T13:34:03.173368+0000","last_active":"2023-08-09T13:34:08.368967+0000","last_peered":"2023-08-09T13:34:08.368967+0000","last_clean":"2023-08-09T13:34:08.368967+0000","last_became_active":"2023-08-09T13:34:03.173037+0000","last_became_peered":"2023-08-09T13:34:03.173037+0000","last_unstale":"2023-08-09T13:34:08.368967+0000","last_undegraded":"2023-08-09T13:34:08.368967+0000","last_fullsized":"2023-08-09T13:34:08.368967+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:34:02.064154+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:34:02.064154+0000","last_clean_scrub_stamp":"2023-08-09T13:34:02.064154+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T21:11:40.494329+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.3a","version":"0'0","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-08-09T13:34:08.371086+0000","last_change":"2023-08-09T13:34:03.144394+0000","last_active":"2023-08-09T13:34:08.371086+0000","last_peered":"2023-08-09T13:34:08.371086+0000","last_clean":"2023-08-09T13:34:08.371086+0000","last_became_active":"2023-08-09T13:34:03.143958+0000","last_became_peered":"2023-08-09T13:34:03.143958+0000","last_unstale":"2023-08-09T13:34:08.371086+0000","last_undegraded":"2023-08-09T13:34:08.371086+0000","last_fullsized":"2023-08-09T13:34:08.371086+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:34:02.064154+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:34:02.064154+0000","last_clean_scrub_stamp":"2023-08-09T13:34:02.064154+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T19:49:18.157706+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.3b","version":"0'0","reported_seq":17,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-08-09T13:34:08.210114+0000","last_change":"2023-08-09T13:34:03.144695+0000","last_active":"2023-08-09T13:34:08.210114+0000","last_peered":"2023-08-09T13:34:08.210114+0000","last_clean":"2023-08-09T13:34:08.210114+0000","last_became_active":"2023-08-09T13:34:03.144303+0000","last_became_peered":"2023-08-09T13:34:03.144303+0000","last_unstale":"2023-08-09T13:34:08.210114+0000","last_undegraded":"2023-08-09T13:34:08.210114+0000","last_fullsized":"2023-08-09T13:34:08.210114+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:34:02.064154+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:34:02.064154+0000","last_clean_scrub_stamp":"2023-08-09T13:34:02.064154+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T18:07:31.705168+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.3c","version":"0'0","reported_seq":17,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-08-09T13:34:08.212037+0000","last_change":"2023-08-09T13:34:03.151741+0000","last_active":"2023-08-09T13:34:08.212037+0000","last_peered":"2023-08-09T13:34:08.212037+0000","last_clean":"2023-08-09T13:34:08.212037+0000","last_became_active":"2023-08-09T13:34:03.151331+0000","last_became_peered":"2023-08-09T13:34:03.151331+0000","last_unstale":"2023-08-09T13:34:08.212037+0000","last_undegraded":"2023-08-09T13:34:08.212037+0000","last_fullsized":"2023-08-09T13:34:08.212037+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:34:02.064154+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:34:02.064154+0000","last_clean_scrub_stamp":"2023-08-09T13:34:02.064154+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-11T00:09:38.635751+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.3d","version":"0'0","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-08-09T13:34:08.372764+0000","last_change":"2023-08-09T13:34:03.144010+0000","last_active":"2023-08-09T13:34:08.372764+0000","last_peered":"2023-08-09T13:34:08.372764+0000","last_clean":"2023-08-09T13:34:08.372764+0000","last_became_active":"2023-08-09T13:34:03.143673+0000","last_became_peered":"2023-08-09T13:34:03.143673+0000","last_unstale":"2023-08-09T13:34:08.372764+0000","last_undegraded":"2023-08-09T13:34:08.372764+0000","last_fullsized":"2023-08-09T13:34:08.372764+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:34:02.064154+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:34:02.064154+0000","last_clean_scrub_stamp":"2023-08-09T13:34:02.064154+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T16:58:18.499551+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.3e","version":"0'0","reported_seq":17,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-08-09T13:34:08.371258+0000","last_change":"2023-08-09T13:34:03.150894+0000","last_active":"2023-08-09T13:34:08.371258+0000","last_peered":"2023-08-09T13:34:08.371258+0000","last_clean":"2023-08-09T13:34:08.371258+0000","last_became_active":"2023-08-09T13:34:03.149237+0000","last_became_peered":"2023-08-09T13:34:03.149237+0000","last_unstale":"2023-08-09T13:34:08.371258+0000","last_undegraded":"2023-08-09T13:34:08.371258+0000","last_fullsized":"2023-08-09T13:34:08.371258+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:34:02.064154+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:34:02.064154+0000","last_clean_scrub_stamp":"2023-08-09T13:34:02.064154+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T21:18:19.551549+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.3f","version":"0'0","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-08-09T13:34:08.212027+0000","last_change":"2023-08-09T13:34:03.160785+0000","last_active":"2023-08-09T13:34:08.212027+0000","last_peered":"2023-08-09T13:34:08.212027+0000","last_clean":"2023-08-09T13:34:08.212027+0000","last_became_active":"2023-08-09T13:34:03.160452+0000","last_became_peered":"2023-08-09T13:34:03.160452+0000","last_unstale":"2023-08-09T13:34:08.212027+0000","last_undegraded":"2023-08-09T13:34:08.212027+0000","last_fullsized":"2023-08-09T13:34:08.212027+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:34:02.064154+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:34:02.064154+0000","last_clean_scrub_stamp":"2023-08-09T13:34:02.064154+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T18:03:19.093604+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]}],"pool_stats":[{"poolid":5,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":4,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":3,"num_pg":32,"stat_sum":{"num_bytes":4639,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":49,"num_write_kb":19,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":9278,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":43,"ondisk_log_size":43,"up":64,"acting":64,"num_store_stats":8},{"poolid":2,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":8}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574856,"num_pgs":41,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27872,"kb_used_data":856,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343968,"statfs":{"total":96636764160,"available":96608223232,"internally_reserved":0,"allocated":876544,"data_stored":699155,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1781,"internal_metadata":27654411},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.806}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.7110000000000001}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.0269999999999999}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.1479999999999999}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.6379999999999999}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.9079999999999999}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":2.367}]}]},{"osd":6,"up_from":13,"seq":55834574856,"num_pgs":34,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27268,"kb_used_data":244,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344572,"statfs":{"total":96636764160,"available":96608841728,"internally_reserved":0,"allocated":249856,"data_stored":103369,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1781,"internal_metadata":27654411},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.0920000000000001}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.3180000000000001}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.55}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.141}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.228}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.4319999999999999}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.359}]}]},{"osd":5,"up_from":13,"seq":55834574856,"num_pgs":39,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27280,"kb_used_data":256,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344560,"statfs":{"total":96636764160,"available":96608829440,"internally_reserved":0,"allocated":262144,"data_stored":105382,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1781,"internal_metadata":27654411},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.1040000000000001}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.599}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.167}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.2130000000000001}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.4279999999999999}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.3720000000000001}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.3959999999999999}]}]},{"osd":4,"up_from":13,"seq":55834574856,"num_pgs":50,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27284,"kb_used_data":268,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344556,"statfs":{"total":96636764160,"available":96608825344,"internally_reserved":0,"allocated":274432,"data_stored":108627,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1781,"internal_metadata":27654411},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.996}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.429}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.2909999999999999}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.357}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.772}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.5760000000000001}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.649}]}]},{"osd":3,"up_from":13,"seq":55834574857,"num_pgs":48,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31372,"kb_used_data":252,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":94340468,"statfs":{"total":96636764160,"available":96604639232,"internally_reserved":0,"allocated":258048,"data_stored":104820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1781,"internal_metadata":31848715},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.3560000000000001}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.181}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.98099999999999998}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.034}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.476}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.4159999999999999}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.804}]}]},{"osd":2,"up_from":13,"seq":55834574856,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31368,"kb_used_data":248,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":94340472,"statfs":{"total":96636764160,"available":96604643328,"internally_reserved":0,"allocated":253952,"data_stored":104730,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1781,"internal_metadata":31848715},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.204}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.462}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.0940000000000001}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.73}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.411}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.385}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.5269999999999999}]}]},{"osd":1,"up_from":13,"seq":55834574856,"num_pgs":49,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31952,"kb_used_data":828,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":94339888,"statfs":{"total":96636764160,"available":96604045312,"internally_reserved":0,"allocated":847872,"data_stored":695098,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1781,"internal_metadata":31848715},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.98699999999999999}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.746}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.2689999999999999}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.216}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.1699999999999999}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.3009999999999999}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":2.0699999999999998}]}]},{"osd":0,"up_from":13,"seq":55834574856,"num_pgs":47,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31368,"kb_used_data":248,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":94340472,"statfs":{"total":96636764160,"available":96604643328,"internally_reserved":0,"allocated":253952,"data_stored":103951,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1781,"internal_metadata":31848715},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.95899999999999996}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.2270000000000001}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.1240000000000001}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.5329999999999999}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.3089999999999999}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.7210000000000001}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.623}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":3897,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":652,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":4057,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-08-09T13:34:33.894 INFO:tasks.cephfs.filesystem.ceph_manager:clean! 2023-08-09T13:34:33.894 INFO:tasks.ceph:Waiting until ceph daemons up and pgs clean... 2023-08-09T13:34:33.894 INFO:tasks.ceph.ceph_manager.ceph:waiting for mgr available 2023-08-09T13:34:33.895 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph mgr dump --format=json 2023-08-09T13:34:33.895 INFO:tasks.daemonwatchdog.daemon_watchdog:watchdog starting 2023-08-09T13:34:34.159 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:34.157+0000 7fa8b90fb640 1 Processor -- start 2023-08-09T13:34:34.160 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:34.157+0000 7fa8b90fb640 1 -- start start 2023-08-09T13:34:34.161 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:34.159+0000 7fa8b90fb640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7fa8b415a6a0 0x7fa8b40713b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:34:34.161 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:34.159+0000 7fa8b90fb640 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7fa8b40c96c0 0x7fa8b415ad20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:34:34.161 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:34.159+0000 7fa8b90fb640 1 -- --> v1:172.21.15.203:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa8b40cdc70 con 0x7fa8b40cbd90 2023-08-09T13:34:34.162 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:34.159+0000 7fa8b90fb640 1 -- --> v2:172.21.15.43:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa8b4070910 con 0x7fa8b415a6a0 2023-08-09T13:34:34.162 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:34.159+0000 7fa8b90fb640 1 -- --> v2:172.21.15.203:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa8b40c88c0 con 0x7fa8b40c96c0 2023-08-09T13:34:34.163 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:34.159+0000 7fa8b3fff640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7fa8b415a6a0 0x7fa8b40713b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:34:34.163 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:34.159+0000 7fa8b3fff640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7fa8b415a6a0 0x7fa8b40713b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:37684/0 (socket says 172.21.15.43:37684) 2023-08-09T13:34:34.163 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:34.159+0000 7fa8b3fff640 1 -- 172.21.15.43:0/4286463846 learned_addr learned my addr 172.21.15.43:0/4286463846 (peer_addr_for_me v2:172.21.15.43:0/0) 2023-08-09T13:34:34.163 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:34.159+0000 7fa8b88fa640 1 --2- 172.21.15.43:0/4286463846 >> v2:172.21.15.203:3300/0 conn(0x7fa8b40c96c0 0x7fa8b415ad20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:34:34.163 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:34.160+0000 7fa8b3fff640 1 -- 172.21.15.43:0/4286463846 >> v2:172.21.15.203:3300/0 conn(0x7fa8b40c96c0 msgr2=0x7fa8b415ad20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-09T13:34:34.163 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:34.160+0000 7fa8b3fff640 1 --2- 172.21.15.43:0/4286463846 >> v2:172.21.15.203:3300/0 conn(0x7fa8b40c96c0 0x7fa8b415ad20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:34:34.164 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:34.160+0000 7fa8b3fff640 1 -- 172.21.15.43:0/4286463846 >> v1:172.21.15.203:6789/0 conn(0x7fa8b40cbd90 legacy=0x7fa8b40c8fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:34:34.164 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:34.160+0000 7fa8b3fff640 1 -- 172.21.15.43:0/4286463846 --> v2:172.21.15.43:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa8b4070bb0 con 0x7fa8b415a6a0 2023-08-09T13:34:34.165 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:34.161+0000 7fa8b3fff640 1 --2- 172.21.15.43:0/4286463846 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa8b415a6a0 0x7fa8b40713b0 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7fa8b40b3dc0 tx=0x7fa8a402db20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4300fc4b5f649a01 server_cookie=0 in_seq=0 out_seq=0 2023-08-09T13:34:34.165 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:34.161+0000 7fa8b37fe640 1 -- 172.21.15.43:0/4286463846 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa8a402f8e0 con 0x7fa8b415a6a0 2023-08-09T13:34:34.165 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:34.162+0000 7fa8b37fe640 1 -- 172.21.15.43:0/4286463846 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa8a402fa40 con 0x7fa8b415a6a0 2023-08-09T13:34:34.165 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:34.162+0000 7fa8b37fe640 1 -- 172.21.15.43:0/4286463846 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa8a402fdd0 con 0x7fa8b415a6a0 2023-08-09T13:34:34.165 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:34.162+0000 7fa8b90fb640 1 -- 172.21.15.43:0/4286463846 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa8b415a6a0 msgr2=0x7fa8b40713b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:34:34.166 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:34.162+0000 7fa8b90fb640 1 --2- 172.21.15.43:0/4286463846 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa8b415a6a0 0x7fa8b40713b0 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7fa8b40b3dc0 tx=0x7fa8a402db20 comp rx=0 tx=0).stop 2023-08-09T13:34:34.166 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:34.162+0000 7fa8b90fb640 1 -- 172.21.15.43:0/4286463846 shutdown_connections 2023-08-09T13:34:34.166 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:34.162+0000 7fa8b90fb640 1 --2- 172.21.15.43:0/4286463846 >> v2:172.21.15.203:3300/0 conn(0x7fa8b40c96c0 0x7fa8b415ad20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:34:34.166 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:34.162+0000 7fa8b90fb640 1 --2- 172.21.15.43:0/4286463846 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa8b415a6a0 0x7fa8b40713b0 unknown :-1 s=CLOSED pgs=150 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:34:34.166 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:34.162+0000 7fa8b90fb640 1 -- 172.21.15.43:0/4286463846 >> 172.21.15.43:0/4286463846 conn(0x7fa8b40cde90 msgr2=0x7fa8b40ce2a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-09T13:34:34.166 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:34.162+0000 7fa8b90fb640 1 -- 172.21.15.43:0/4286463846 shutdown_connections 2023-08-09T13:34:34.166 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:34.162+0000 7fa8b90fb640 1 -- 172.21.15.43:0/4286463846 wait complete. 2023-08-09T13:34:34.167 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:34.164+0000 7fa8b90fb640 1 Processor -- start 2023-08-09T13:34:34.167 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:34.164+0000 7fa8b90fb640 1 -- start start 2023-08-09T13:34:34.168 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:34.165+0000 7fa8b90fb640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fa8b40c96c0 0x7fa8b4158990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:34:34.168 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:34.165+0000 7fa8b90fb640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa8b40cbd90 0x7fa8b40b22b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:34:34.168 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:34.165+0000 7fa8b90fb640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fa8b415a6a0 0x7fa8b40b27f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:34:34.169 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:34.165+0000 7fa8b90fb640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa8b406cc00 con 0x7fa8b40cbd90 2023-08-09T13:34:34.169 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:34.165+0000 7fa8b90fb640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa8b409a550 con 0x7fa8b40c96c0 2023-08-09T13:34:34.169 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:34.165+0000 7fa8b90fb640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa8b409a830 con 0x7fa8b415a6a0 2023-08-09T13:34:34.169 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:34.166+0000 7fa8abfff640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa8b40cbd90 0x7fa8b40b22b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:34:34.169 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:34.166+0000 7fa8abfff640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa8b40cbd90 0x7fa8b40b22b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:37688/0 (socket says 172.21.15.43:37688) 2023-08-09T13:34:34.169 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:34.166+0000 7fa8abfff640 1 -- 172.21.15.43:0/2362471364 learned_addr learned my addr 172.21.15.43:0/2362471364 (peer_addr_for_me v2:172.21.15.43:0/0) 2023-08-09T13:34:34.170 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:34.166+0000 7fa8b3fff640 1 --2- 172.21.15.43:0/2362471364 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fa8b40c96c0 0x7fa8b4158990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:34:34.171 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:34.166+0000 7fa8b88fa640 1 --2- 172.21.15.43:0/2362471364 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fa8b415a6a0 0x7fa8b40b27f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:34:34.171 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:34.166+0000 7fa8abfff640 1 -- 172.21.15.43:0/2362471364 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fa8b415a6a0 msgr2=0x7fa8b40b27f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:34:34.171 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:34.166+0000 7fa8abfff640 1 --2- 172.21.15.43:0/2362471364 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fa8b415a6a0 0x7fa8b40b27f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:34:34.171 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:34.166+0000 7fa8abfff640 1 -- 172.21.15.43:0/2362471364 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fa8b40c96c0 msgr2=0x7fa8b4158990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:34:34.171 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:34.166+0000 7fa8abfff640 1 --2- 172.21.15.43:0/2362471364 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fa8b40c96c0 0x7fa8b4158990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:34:34.171 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:34.166+0000 7fa8abfff640 1 -- 172.21.15.43:0/2362471364 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa8a40094d0 con 0x7fa8b40cbd90 2023-08-09T13:34:34.172 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:34.166+0000 7fa8b88fa640 1 --2- 172.21.15.43:0/2362471364 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fa8b415a6a0 0x7fa8b40b27f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-08-09T13:34:34.172 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:34.167+0000 7fa8abfff640 1 --2- 172.21.15.43:0/2362471364 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa8b40cbd90 0x7fa8b40b22b0 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7fa89c00c400 tx=0x7fa89c00c8c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-09T13:34:34.172 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:34.167+0000 7fa8b17fa640 1 -- 172.21.15.43:0/2362471364 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa89c0052d0 con 0x7fa8b40cbd90 2023-08-09T13:34:34.172 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:34.167+0000 7fa8b17fa640 1 -- 172.21.15.43:0/2362471364 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa89c005450 con 0x7fa8b40cbd90 2023-08-09T13:34:34.172 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:34.167+0000 7fa8b17fa640 1 -- 172.21.15.43:0/2362471364 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa89c007260 con 0x7fa8b40cbd90 2023-08-09T13:34:34.172 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:34.167+0000 7fa8b90fb640 1 -- 172.21.15.43:0/2362471364 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa8b409ab10 con 0x7fa8b40cbd90 2023-08-09T13:34:34.172 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:34.167+0000 7fa8b90fb640 1 -- 172.21.15.43:0/2362471364 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa8b409b310 con 0x7fa8b40cbd90 2023-08-09T13:34:34.173 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:34.168+0000 7fa8b90fb640 1 -- 172.21.15.43:0/2362471364 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa8b409b030 con 0x7fa8b40cbd90 2023-08-09T13:34:34.173 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:34.169+0000 7fa8b17fa640 1 -- 172.21.15.43:0/2362471364 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 5) v1 ==== 88341+0+0 (secure 0 0 0) 0x7fa89c0073c0 con 0x7fa8b40cbd90 2023-08-09T13:34:34.173 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:34.170+0000 7fa8b17fa640 1 --2- 172.21.15.43:0/2362471364 >> [v2:172.21.15.203:6832/830097474,v1:172.21.15.203:6833/830097474] conn(0x7fa89406a050 0x7fa89406c510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:34:34.174 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:34.170+0000 7fa8b17fa640 1 -- 172.21.15.43:0/2362471364 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6813+0+0 (secure 0 0 0) 0x7fa89c087d50 con 0x7fa8b40cbd90 2023-08-09T13:34:34.174 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:34.171+0000 7fa8b3fff640 1 --2- 172.21.15.43:0/2362471364 >> [v2:172.21.15.203:6832/830097474,v1:172.21.15.203:6833/830097474] conn(0x7fa89406a050 0x7fa89406c510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:34:34.174 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:34.172+0000 7fa8b3fff640 1 --2- 172.21.15.43:0/2362471364 >> [v2:172.21.15.203:6832/830097474,v1:172.21.15.203:6833/830097474] conn(0x7fa89406a050 0x7fa89406c510 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7fa8a4038000 tx=0x7fa8a4008030 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-09T13:34:34.177 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:34.174+0000 7fa8b17fa640 1 -- 172.21.15.43:0/2362471364 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178066 (secure 0 0 0) 0x7fa8b409b030 con 0x7fa8b40cbd90 2023-08-09T13:34:34.421 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:34.418+0000 7fa8b90fb640 1 -- 172.21.15.43:0/2362471364 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "mgr dump", "format": "json"} v 0) v1 -- 0x7fa8b409b030 con 0x7fa8b40cbd90 2023-08-09T13:34:34.430 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:34.427+0000 7fa8b17fa640 1 -- 172.21.15.43:0/2362471364 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "mgr dump", "format": "json"}]=0 v5) v1 ==== 74+0+169495 (secure 0 0 0) 0x7fa8b409b030 con 0x7fa8b40cbd90 2023-08-09T13:34:34.432 INFO:teuthology.orchestra.run.smithi043.stdout: 2023-08-09T13:34:34.434 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:34.432+0000 7fa8b90fb640 1 -- 172.21.15.43:0/2362471364 >> [v2:172.21.15.203:6832/830097474,v1:172.21.15.203:6833/830097474] conn(0x7fa89406a050 msgr2=0x7fa89406c510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:34:34.435 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:34.432+0000 7fa8b90fb640 1 --2- 172.21.15.43:0/2362471364 >> [v2:172.21.15.203:6832/830097474,v1:172.21.15.203:6833/830097474] conn(0x7fa89406a050 0x7fa89406c510 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7fa8a4038000 tx=0x7fa8a4008030 comp rx=0 tx=0).stop 2023-08-09T13:34:34.435 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:34.432+0000 7fa8b90fb640 1 -- 172.21.15.43:0/2362471364 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa8b40cbd90 msgr2=0x7fa8b40b22b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:34:34.435 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:34.432+0000 7fa8b90fb640 1 --2- 172.21.15.43:0/2362471364 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa8b40cbd90 0x7fa8b40b22b0 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7fa89c00c400 tx=0x7fa89c00c8c0 comp rx=0 tx=0).stop 2023-08-09T13:34:34.435 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:34.432+0000 7fa8b90fb640 1 -- 172.21.15.43:0/2362471364 shutdown_connections 2023-08-09T13:34:34.435 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:34.432+0000 7fa8b90fb640 1 --2- 172.21.15.43:0/2362471364 >> [v2:172.21.15.203:6832/830097474,v1:172.21.15.203:6833/830097474] conn(0x7fa89406a050 0x7fa89406c510 unknown :-1 s=CLOSED pgs=95 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:34:34.435 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:34.432+0000 7fa8b90fb640 1 --2- 172.21.15.43:0/2362471364 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fa8b415a6a0 0x7fa8b40b27f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:34:34.436 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:34.432+0000 7fa8b90fb640 1 --2- 172.21.15.43:0/2362471364 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa8b40cbd90 0x7fa8b40b22b0 unknown :-1 s=CLOSED pgs=151 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:34:34.436 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:34.432+0000 7fa8b90fb640 1 --2- 172.21.15.43:0/2362471364 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fa8b40c96c0 0x7fa8b4158990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:34:34.436 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:34.432+0000 7fa8b90fb640 1 -- 172.21.15.43:0/2362471364 >> 172.21.15.43:0/2362471364 conn(0x7fa8b40cde90 msgr2=0x7fa8b415c310 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-09T13:34:34.436 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:34.432+0000 7fa8b90fb640 1 -- 172.21.15.43:0/2362471364 shutdown_connections 2023-08-09T13:34:34.436 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:34.432+0000 7fa8b90fb640 1 -- 172.21.15.43:0/2362471364 wait complete. 2023-08-09T13:34:34.455 INFO:teuthology.orchestra.run.smithi043.stdout:{"epoch":5,"active_gid":4114,"active_name":"x","active_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6832","nonce":830097474},{"type":"v1","addr":"172.21.15.203:6833","nonce":830097474}]},"active_addr":"172.21.15.203:6833/830097474","active_change":"2023-08-09T13:33:46.735660+0000","active_mgr_features":4540138322906710015,"available":true,"standbys":[{"gid":4107,"name":"y","mgr_features":4540138322906710015,"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2359","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"upmap","min":"","max":"","enum_allowed":["crush-compat","none","upmap"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"cephadm","can_run":true,"error_string":"","module_options":{"agent_down_multiplier":{"name":"agent_down_multiplier","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"","max":"","enum_allowed":[],"desc":"Multiplied by agent refresh rate to calculate how long agent must not report before being marked down","long_desc":"","tags":[],"see_also":[]},"agent_refresh_rate":{"name":"agent_refresh_rate","type":"secs","level":"advanced","flags":0,"default_value":"20","min":"","max":"","enum_allowed":[],"desc":"How often agent on each host will try to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"agent_starting_port":{"name":"agent_starting_port","type":"int","level":"advanced","flags":0,"default_value":"4721","min":"","max":"","enum_allowed":[],"desc":"First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)","long_desc":"","tags":[],"see_also":[]},"allow_ptrace":{"name":"allow_ptrace","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow SYS_PTRACE capability on ceph containers","long_desc":"The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.","tags":[],"see_also":[]},"autotune_interval":{"name":"autotune_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to autotune daemon memory","long_desc":"","tags":[],"see_also":[]},"autotune_memory_target_ratio":{"name":"autotune_memory_target_ratio","type":"float","level":"advanced","flags":0,"default_value":"0.7","min":"","max":"","enum_allowed":[],"desc":"ratio of total system memory to divide amongst autotuned daemons","long_desc":"","tags":[],"see_also":[]},"cgroups_split":{"name":"cgroups_split","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Pass --cgroups=split when cephadm creates containers (currently podman only)","long_desc":"","tags":[],"see_also":[]},"config_checks_enabled":{"name":"config_checks_enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable or disable the cephadm configuration analysis","long_desc":"","tags":[],"see_also":[]},"config_dashboard":{"name":"config_dashboard","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"manage configs like API endpoints in Dashboard.","long_desc":"","tags":[],"see_also":[]},"container_image_alertmanager":{"name":"container_image_alertmanager","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/alertmanager:v0.25.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_base":{"name":"container_image_base","type":"str","level":"advanced","flags":1,"default_value":"quay.io/ceph/ceph","min":"","max":"","enum_allowed":[],"desc":"Container image name, without the tag","long_desc":"","tags":[],"see_also":[]},"container_image_elasticsearch":{"name":"container_image_elasticsearch","type":"str","level":"advanced","flags":0,"default_value":"quay.io/omrizeneva/elasticsearch:6.8.23","min":"","max":"","enum_allowed":[],"desc":"elasticsearch container image","long_desc":"","tags":[],"see_also":[]},"container_image_grafana":{"name":"container_image_grafana","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/ceph-grafana:9.4.7","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_haproxy":{"name":"container_image_haproxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/haproxy:2.3","min":"","max":"","enum_allowed":[],"desc":"HAproxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_agent":{"name":"container_image_jaeger_agent","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-agent:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger agent container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_collector":{"name":"container_image_jaeger_collector","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-collector:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger collector container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_query":{"name":"container_image_jaeger_query","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-query:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger query container image","long_desc":"","tags":[],"see_also":[]},"container_image_keepalived":{"name":"container_image_keepalived","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/keepalived:2.2.4","min":"","max":"","enum_allowed":[],"desc":"Keepalived container image","long_desc":"","tags":[],"see_also":[]},"container_image_loki":{"name":"container_image_loki","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/loki:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Loki container image","long_desc":"","tags":[],"see_also":[]},"container_image_node_exporter":{"name":"container_image_node_exporter","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/node-exporter:v1.5.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_nvmeof":{"name":"container_image_nvmeof","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/nvmeof:0.0.1","min":"","max":"","enum_allowed":[],"desc":"Nvme-of container image","long_desc":"","tags":[],"see_also":[]},"container_image_prometheus":{"name":"container_image_prometheus","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/prometheus:v2.43.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_promtail":{"name":"container_image_promtail","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/promtail:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Promtail container image","long_desc":"","tags":[],"see_also":[]},"container_image_snmp_gateway":{"name":"container_image_snmp_gateway","type":"str","level":"advanced","flags":0,"default_value":"docker.io/maxwo/snmp-notifier:v1.2.1","min":"","max":"","enum_allowed":[],"desc":"SNMP Gateway container image","long_desc":"","tags":[],"see_also":[]},"container_init":{"name":"container_init","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Run podman/docker with `--init`","long_desc":"","tags":[],"see_also":[]},"daemon_cache_timeout":{"name":"daemon_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"seconds to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"default_cephadm_command_timeout":{"name":"default_cephadm_command_timeout","type":"secs","level":"advanced","flags":0,"default_value":"900","min":"","max":"","enum_allowed":[],"desc":"Default timeout applied to cephadm commands run directly on the host (in seconds)","long_desc":"","tags":[],"see_also":[]},"default_registry":{"name":"default_registry","type":"str","level":"advanced","flags":0,"default_value":"docker.io","min":"","max":"","enum_allowed":[],"desc":"Search-registry to which we should normalize unqualified image names. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"device_cache_timeout":{"name":"device_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"seconds to cache device inventory","long_desc":"","tags":[],"see_also":[]},"device_enhanced_scan":{"name":"device_enhanced_scan","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use libstoragemgmt during device scans","long_desc":"","tags":[],"see_also":[]},"facts_cache_timeout":{"name":"facts_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"seconds to cache host facts data","long_desc":"","tags":[],"see_also":[]},"host_check_interval":{"name":"host_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to perform a host check","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_refresh_metadata":{"name":"log_refresh_metadata","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"log to the \"cephadm\" cluster log channel\"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf":{"name":"manage_etc_ceph_ceph_conf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Manage and own /etc/ceph/ceph.conf on the hosts.","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf_hosts":{"name":"manage_etc_ceph_ceph_conf_hosts","type":"str","level":"advanced","flags":0,"default_value":"*","min":"","max":"","enum_allowed":[],"desc":"PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf","long_desc":"","tags":[],"see_also":[]},"max_count_per_host":{"name":"max_count_per_host","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of daemons per service per host","long_desc":"","tags":[],"see_also":[]},"max_osd_draining_count":{"name":"max_osd_draining_count","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of osds that will be drained simultaneously when osds are removed","long_desc":"","tags":[],"see_also":[]},"migration_current":{"name":"migration_current","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"internal - do not modify","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["cephadm-package","root"],"desc":"mode for remote execution of cephadm","long_desc":"","tags":[],"see_also":[]},"prometheus_alerts_path":{"name":"prometheus_alerts_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/prometheus/ceph/ceph_default_alerts.yml","min":"","max":"","enum_allowed":[],"desc":"location of alerts to include in prometheus deployments","long_desc":"","tags":[],"see_also":[]},"registry_insecure":{"name":"registry_insecure","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Registry is to be considered insecure (no TLS available). Only for development purposes.","long_desc":"","tags":[],"see_also":[]},"registry_password":{"name":"registry_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository password. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"registry_url":{"name":"registry_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Registry url for login purposes. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"registry_username":{"name":"registry_username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository username. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"secure_monitoring_stack":{"name":"secure_monitoring_stack","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable TLS security for all the monitoring stack daemons","long_desc":"","tags":[],"see_also":[]},"service_discovery_port":{"name":"service_discovery_port","type":"int","level":"advanced","flags":0,"default_value":"8765","min":"","max":"","enum_allowed":[],"desc":"cephadm service discovery port","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"customized SSH config file to connect to managed hosts","long_desc":"","tags":[],"see_also":[]},"use_agent":{"name":"use_agent","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use cephadm agent on each host to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"use_repo_digest":{"name":"use_repo_digest","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Automatically convert image tags to image digest. Make sure all daemons use the same image","long_desc":"","tags":[],"see_also":[]},"warn_on_failed_host_check":{"name":"warn_on_failed_host_check","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if the host check fails","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_daemons":{"name":"warn_on_stray_daemons","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected that are not managed by cephadm","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_hosts":{"name":"warn_on_stray_hosts","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected on a host that is not managed by cephadm","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ACCOUNT_LOCKOUT_ATTEMPTS":{"name":"ACCOUNT_LOCKOUT_ATTEMPTS","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_SSL_VERIFY":{"name":"ALERTMANAGER_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_CEPHFS":{"name":"FEATURE_TOGGLE_CEPHFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_DASHBOARD":{"name":"FEATURE_TOGGLE_DASHBOARD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_ISCSI":{"name":"FEATURE_TOGGLE_ISCSI","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_MIRRORING":{"name":"FEATURE_TOGGLE_MIRRORING","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_NFS":{"name":"FEATURE_TOGGLE_NFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RBD":{"name":"FEATURE_TOGGLE_RBD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RGW":{"name":"FEATURE_TOGGLE_RGW","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_FRONTEND_API_URL":{"name":"GRAFANA_FRONTEND_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISSUE_TRACKER_API_KEY":{"name":"ISSUE_TRACKER_API_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_SSL_VERIFY":{"name":"PROMETHEUS_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"cross_origin_url":{"name":"cross_origin_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"motd":{"name":"motd","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"The message of the day","long_desc":"","tags":[],"see_also":[]},"redirect_resolve_ip_addr":{"name":"redirect_resolve_ip_addr","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"diskprediction_local","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predict_interval":{"name":"predict_interval","type":"str","level":"advanced","flags":0,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predictor_model":{"name":"predictor_model","type":"str","level":"advanced","flags":0,"default_value":"prophetstor","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"int","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"How big batches of data points should be when sending to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server hostname","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"30","min":"5","max":"","enum_allowed":[],"desc":"Time between reports to InfluxDB. Default 30 seconds.","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"password of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"int","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server port","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"Use https connection for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"int","level":"advanced","flags":0,"default_value":"5","min":"1","max":"32","enum_allowed":[],"desc":"How many worker threads should be spawned for sending data to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"username of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"Verify https cert for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"what failure domain we should separate data replicas across.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"value to set min_size to (unchanged from Ceph's default if this option is not set)","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"which CRUSH subtree type the module should create a pool for.","tags":[],"see_also":[]}}},{"name":"mds_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"mirroring","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"nfs","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"fail_fs":{"name":"fail_fs","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Fail filesystem for rapid multi-rank mds upgrade","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_perf_query","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"noautoscale":{"name":"noautoscale","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"global autoscale flag","long_desc":"Option to turn on/off the autoscaler for all pools","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threshold":{"name":"threshold","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"1.0","max":"","enum_allowed":[],"desc":"scaling threshold","long_desc":"The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"allow_pg_recovery_event":{"name":"allow_pg_recovery_event","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow the module to show pg recovery progress","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how long the module is going to sleep","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"cache":{"name":"cache","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"exclude_perf_counters":{"name":"exclude_perf_counters","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Do not include perf-counters in the metrics output","long_desc":"Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"float","level":"advanced","flags":0,"default_value":"15.0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"the IPv4 or IPv6 address on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":1,"default_value":"9283","min":"","max":"","enum_allowed":[],"desc":"the port on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"log","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":1,"default_value":"default","min":"","max":"","enum_allowed":["default","error"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":1,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"restful","can_run":true,"error_string":"","module_options":{"enable_auth":{"name":"enable_auth","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rgw","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rook","can_run":true,"error_string":"","module_options":{"drive_group_interval":{"name":"drive_group_interval","type":"float","level":"advanced","flags":0,"default_value":"300.0","min":"","max":"","enum_allowed":[],"desc":"interval in seconds between re-application of applied drive_groups","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"storage_class":{"name":"storage_class","type":"str","level":"advanced","flags":0,"default_value":"local","min":"","max":"","enum_allowed":[],"desc":"storage class name for LSO-discovered PVs","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption7":{"name":"rwoption7","type":"int","level":"advanced","flags":0,"default_value":"","min":"1","max":"42","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"snap_schedule","can_run":true,"error_string":"","module_options":{"allow_m_granularity":{"name":"allow_m_granularity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow minute scheduled snapshots","long_desc":"","tags":[],"see_also":[]},"dump_on_update":{"name":"dump_on_update","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"dump database to debug log on update","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"stats","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share metadata about Ceph daemon crashes (version, stack straces, etc)","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share a user-provided description and/or contact email for the cluster","long_desc":"","tags":[],"see_also":[]},"channel_perf":{"name":"channel_perf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share various performance metrics of a cluster","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard_description":{"name":"leaderboard_description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_delay":{"name":"snapshot_clone_delay","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"Delay clone begin operation by snapshot_clone_delay seconds","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"discovery_interval":{"name":"discovery_interval","type":"uint","level":"advanced","flags":0,"default_value":"100","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}]}],"modules":["iostat","nfs","restful"],"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2359","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"upmap","min":"","max":"","enum_allowed":["crush-compat","none","upmap"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"cephadm","can_run":true,"error_string":"","module_options":{"agent_down_multiplier":{"name":"agent_down_multiplier","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"","max":"","enum_allowed":[],"desc":"Multiplied by agent refresh rate to calculate how long agent must not report before being marked down","long_desc":"","tags":[],"see_also":[]},"agent_refresh_rate":{"name":"agent_refresh_rate","type":"secs","level":"advanced","flags":0,"default_value":"20","min":"","max":"","enum_allowed":[],"desc":"How often agent on each host will try to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"agent_starting_port":{"name":"agent_starting_port","type":"int","level":"advanced","flags":0,"default_value":"4721","min":"","max":"","enum_allowed":[],"desc":"First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)","long_desc":"","tags":[],"see_also":[]},"allow_ptrace":{"name":"allow_ptrace","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow SYS_PTRACE capability on ceph containers","long_desc":"The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.","tags":[],"see_also":[]},"autotune_interval":{"name":"autotune_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to autotune daemon memory","long_desc":"","tags":[],"see_also":[]},"autotune_memory_target_ratio":{"name":"autotune_memory_target_ratio","type":"float","level":"advanced","flags":0,"default_value":"0.7","min":"","max":"","enum_allowed":[],"desc":"ratio of total system memory to divide amongst autotuned daemons","long_desc":"","tags":[],"see_also":[]},"cgroups_split":{"name":"cgroups_split","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Pass --cgroups=split when cephadm creates containers (currently podman only)","long_desc":"","tags":[],"see_also":[]},"config_checks_enabled":{"name":"config_checks_enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable or disable the cephadm configuration analysis","long_desc":"","tags":[],"see_also":[]},"config_dashboard":{"name":"config_dashboard","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"manage configs like API endpoints in Dashboard.","long_desc":"","tags":[],"see_also":[]},"container_image_alertmanager":{"name":"container_image_alertmanager","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/alertmanager:v0.25.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_base":{"name":"container_image_base","type":"str","level":"advanced","flags":1,"default_value":"quay.io/ceph/ceph","min":"","max":"","enum_allowed":[],"desc":"Container image name, without the tag","long_desc":"","tags":[],"see_also":[]},"container_image_elasticsearch":{"name":"container_image_elasticsearch","type":"str","level":"advanced","flags":0,"default_value":"quay.io/omrizeneva/elasticsearch:6.8.23","min":"","max":"","enum_allowed":[],"desc":"elasticsearch container image","long_desc":"","tags":[],"see_also":[]},"container_image_grafana":{"name":"container_image_grafana","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/ceph-grafana:9.4.7","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_haproxy":{"name":"container_image_haproxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/haproxy:2.3","min":"","max":"","enum_allowed":[],"desc":"HAproxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_agent":{"name":"container_image_jaeger_agent","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-agent:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger agent container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_collector":{"name":"container_image_jaeger_collector","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-collector:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger collector container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_query":{"name":"container_image_jaeger_query","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-query:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger query container image","long_desc":"","tags":[],"see_also":[]},"container_image_keepalived":{"name":"container_image_keepalived","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/keepalived:2.2.4","min":"","max":"","enum_allowed":[],"desc":"Keepalived container image","long_desc":"","tags":[],"see_also":[]},"container_image_loki":{"name":"container_image_loki","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/loki:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Loki container image","long_desc":"","tags":[],"see_also":[]},"container_image_node_exporter":{"name":"container_image_node_exporter","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/node-exporter:v1.5.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_nvmeof":{"name":"container_image_nvmeof","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/nvmeof:0.0.1","min":"","max":"","enum_allowed":[],"desc":"Nvme-of container image","long_desc":"","tags":[],"see_also":[]},"container_image_prometheus":{"name":"container_image_prometheus","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/prometheus:v2.43.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_promtail":{"name":"container_image_promtail","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/promtail:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Promtail container image","long_desc":"","tags":[],"see_also":[]},"container_image_snmp_gateway":{"name":"container_image_snmp_gateway","type":"str","level":"advanced","flags":0,"default_value":"docker.io/maxwo/snmp-notifier:v1.2.1","min":"","max":"","enum_allowed":[],"desc":"SNMP Gateway container image","long_desc":"","tags":[],"see_also":[]},"container_init":{"name":"container_init","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Run podman/docker with `--init`","long_desc":"","tags":[],"see_also":[]},"daemon_cache_timeout":{"name":"daemon_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"seconds to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"default_cephadm_command_timeout":{"name":"default_cephadm_command_timeout","type":"secs","level":"advanced","flags":0,"default_value":"900","min":"","max":"","enum_allowed":[],"desc":"Default timeout applied to cephadm commands run directly on the host (in seconds)","long_desc":"","tags":[],"see_also":[]},"default_registry":{"name":"default_registry","type":"str","level":"advanced","flags":0,"default_value":"docker.io","min":"","max":"","enum_allowed":[],"desc":"Search-registry to which we should normalize unqualified image names. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"device_cache_timeout":{"name":"device_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"seconds to cache device inventory","long_desc":"","tags":[],"see_also":[]},"device_enhanced_scan":{"name":"device_enhanced_scan","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use libstoragemgmt during device scans","long_desc":"","tags":[],"see_also":[]},"facts_cache_timeout":{"name":"facts_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"seconds to cache host facts data","long_desc":"","tags":[],"see_also":[]},"host_check_interval":{"name":"host_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to perform a host check","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_refresh_metadata":{"name":"log_refresh_metadata","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"log to the \"cephadm\" cluster log channel\"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf":{"name":"manage_etc_ceph_ceph_conf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Manage and own /etc/ceph/ceph.conf on the hosts.","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf_hosts":{"name":"manage_etc_ceph_ceph_conf_hosts","type":"str","level":"advanced","flags":0,"default_value":"*","min":"","max":"","enum_allowed":[],"desc":"PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf","long_desc":"","tags":[],"see_also":[]},"max_count_per_host":{"name":"max_count_per_host","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of daemons per service per host","long_desc":"","tags":[],"see_also":[]},"max_osd_draining_count":{"name":"max_osd_draining_count","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of osds that will be drained simultaneously when osds are removed","long_desc":"","tags":[],"see_also":[]},"migration_current":{"name":"migration_current","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"internal - do not modify","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["cephadm-package","root"],"desc":"mode for remote execution of cephadm","long_desc":"","tags":[],"see_also":[]},"prometheus_alerts_path":{"name":"prometheus_alerts_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/prometheus/ceph/ceph_default_alerts.yml","min":"","max":"","enum_allowed":[],"desc":"location of alerts to include in prometheus deployments","long_desc":"","tags":[],"see_also":[]},"registry_insecure":{"name":"registry_insecure","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Registry is to be considered insecure (no TLS available). Only for development purposes.","long_desc":"","tags":[],"see_also":[]},"registry_password":{"name":"registry_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository password. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"registry_url":{"name":"registry_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Registry url for login purposes. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"registry_username":{"name":"registry_username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository username. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"secure_monitoring_stack":{"name":"secure_monitoring_stack","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable TLS security for all the monitoring stack daemons","long_desc":"","tags":[],"see_also":[]},"service_discovery_port":{"name":"service_discovery_port","type":"int","level":"advanced","flags":0,"default_value":"8765","min":"","max":"","enum_allowed":[],"desc":"cephadm service discovery port","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"customized SSH config file to connect to managed hosts","long_desc":"","tags":[],"see_also":[]},"use_agent":{"name":"use_agent","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use cephadm agent on each host to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"use_repo_digest":{"name":"use_repo_digest","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Automatically convert image tags to image digest. Make sure all daemons use the same image","long_desc":"","tags":[],"see_also":[]},"warn_on_failed_host_check":{"name":"warn_on_failed_host_check","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if the host check fails","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_daemons":{"name":"warn_on_stray_daemons","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected that are not managed by cephadm","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_hosts":{"name":"warn_on_stray_hosts","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected on a host that is not managed by cephadm","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ACCOUNT_LOCKOUT_ATTEMPTS":{"name":"ACCOUNT_LOCKOUT_ATTEMPTS","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_SSL_VERIFY":{"name":"ALERTMANAGER_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_CEPHFS":{"name":"FEATURE_TOGGLE_CEPHFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_DASHBOARD":{"name":"FEATURE_TOGGLE_DASHBOARD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_ISCSI":{"name":"FEATURE_TOGGLE_ISCSI","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_MIRRORING":{"name":"FEATURE_TOGGLE_MIRRORING","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_NFS":{"name":"FEATURE_TOGGLE_NFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RBD":{"name":"FEATURE_TOGGLE_RBD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RGW":{"name":"FEATURE_TOGGLE_RGW","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_FRONTEND_API_URL":{"name":"GRAFANA_FRONTEND_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISSUE_TRACKER_API_KEY":{"name":"ISSUE_TRACKER_API_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_SSL_VERIFY":{"name":"PROMETHEUS_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"cross_origin_url":{"name":"cross_origin_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"motd":{"name":"motd","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"The message of the day","long_desc":"","tags":[],"see_also":[]},"redirect_resolve_ip_addr":{"name":"redirect_resolve_ip_addr","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"diskprediction_local","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predict_interval":{"name":"predict_interval","type":"str","level":"advanced","flags":0,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predictor_model":{"name":"predictor_model","type":"str","level":"advanced","flags":0,"default_value":"prophetstor","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"int","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"How big batches of data points should be when sending to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server hostname","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"30","min":"5","max":"","enum_allowed":[],"desc":"Time between reports to InfluxDB. Default 30 seconds.","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"password of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"int","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server port","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"Use https connection for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"int","level":"advanced","flags":0,"default_value":"5","min":"1","max":"32","enum_allowed":[],"desc":"How many worker threads should be spawned for sending data to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"username of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"Verify https cert for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"what failure domain we should separate data replicas across.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"value to set min_size to (unchanged from Ceph's default if this option is not set)","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"which CRUSH subtree type the module should create a pool for.","tags":[],"see_also":[]}}},{"name":"mds_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"mirroring","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"nfs","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"fail_fs":{"name":"fail_fs","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Fail filesystem for rapid multi-rank mds upgrade","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_perf_query","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"noautoscale":{"name":"noautoscale","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"global autoscale flag","long_desc":"Option to turn on/off the autoscaler for all pools","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threshold":{"name":"threshold","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"1.0","max":"","enum_allowed":[],"desc":"scaling threshold","long_desc":"The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"allow_pg_recovery_event":{"name":"allow_pg_recovery_event","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow the module to show pg recovery progress","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how long the module is going to sleep","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"cache":{"name":"cache","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"exclude_perf_counters":{"name":"exclude_perf_counters","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Do not include perf-counters in the metrics output","long_desc":"Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"float","level":"advanced","flags":0,"default_value":"15.0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"the IPv4 or IPv6 address on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":1,"default_value":"9283","min":"","max":"","enum_allowed":[],"desc":"the port on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"log","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":1,"default_value":"default","min":"","max":"","enum_allowed":["default","error"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":1,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"restful","can_run":true,"error_string":"","module_options":{"enable_auth":{"name":"enable_auth","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rgw","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rook","can_run":true,"error_string":"","module_options":{"drive_group_interval":{"name":"drive_group_interval","type":"float","level":"advanced","flags":0,"default_value":"300.0","min":"","max":"","enum_allowed":[],"desc":"interval in seconds between re-application of applied drive_groups","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"storage_class":{"name":"storage_class","type":"str","level":"advanced","flags":0,"default_value":"local","min":"","max":"","enum_allowed":[],"desc":"storage class name for LSO-discovered PVs","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption7":{"name":"rwoption7","type":"int","level":"advanced","flags":0,"default_value":"","min":"1","max":"42","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"snap_schedule","can_run":true,"error_string":"","module_options":{"allow_m_granularity":{"name":"allow_m_granularity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow minute scheduled snapshots","long_desc":"","tags":[],"see_also":[]},"dump_on_update":{"name":"dump_on_update","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"dump database to debug log on update","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"stats","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share metadata about Ceph daemon crashes (version, stack straces, etc)","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share a user-provided description and/or contact email for the cluster","long_desc":"","tags":[],"see_also":[]},"channel_perf":{"name":"channel_perf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share various performance metrics of a cluster","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard_description":{"name":"leaderboard_description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_delay":{"name":"snapshot_clone_delay","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"Delay clone begin operation by snapshot_clone_delay seconds","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"discovery_interval":{"name":"discovery_interval","type":"uint","level":"advanced","flags":0,"default_value":"100","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}],"services":{},"always_on_modules":{"octopus":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"pacific":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"quincy":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"reef":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"]},"last_failure_osd_epoch":0,"active_clients":[{"name":"devicehealth","addrvec":[{"type":"v2","addr":"172.21.15.203:0","nonce":3309257000}]},{"name":"libcephsqlite","addrvec":[{"type":"v2","addr":"172.21.15.203:0","nonce":890377927}]},{"name":"rbd_support","addrvec":[{"type":"v2","addr":"172.21.15.203:0","nonce":2439593041}]},{"name":"volumes","addrvec":[{"type":"v2","addr":"172.21.15.203:0","nonce":1761968483}]}]} 2023-08-09T13:34:34.459 INFO:tasks.ceph.ceph_manager.ceph:mgr available! 2023-08-09T13:34:34.459 INFO:tasks.ceph.ceph_manager.ceph:waiting for all up 2023-08-09T13:34:34.460 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-08-09T13:34:34.691 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:34.689+0000 7f1b136ec640 1 Processor -- start 2023-08-09T13:34:34.692 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:34.689+0000 7f1b136ec640 1 -- start start 2023-08-09T13:34:34.693 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:34.691+0000 7f1b136ec640 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7f1b0c06cfe0 0x7f1b0c0c8b50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:34:34.693 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:34.691+0000 7f1b136ec640 1 -- --> v1:172.21.15.43:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1b0c0c9090 con 0x7f1b0c143c20 2023-08-09T13:34:34.694 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:34.691+0000 7f1b136ec640 1 -- --> v1:172.21.15.203:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1b0c06c440 con 0x7f1b0c0c86f0 2023-08-09T13:34:34.694 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:34.691+0000 7f1b136ec640 1 -- --> v2:172.21.15.203:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1b0c144000 con 0x7f1b0c06cfe0 2023-08-09T13:34:34.694 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:34.691+0000 7f1b11488640 1 --1- >> v1:172.21.15.43:6789/0 conn(0x7f1b0c143c20 0x7f1b0c06f540 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.43:6789/0 says I am v1:172.21.15.43:59568/0 (socket says 172.21.15.43:59568) 2023-08-09T13:34:34.694 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:34.691+0000 7f1b11488640 1 -- 172.21.15.43:0/1200410542 learned_addr learned my addr 172.21.15.43:0/1200410542 (peer_addr_for_me v1:172.21.15.43:0/0) 2023-08-09T13:34:34.695 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:34.691+0000 7f1b03fff640 1 -- 172.21.15.43:0/1200410542 <== mon.0 v1:172.21.15.43:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 383618394 0 0) 0x7f1b0c0c9090 con 0x7f1b0c143c20 2023-08-09T13:34:34.696 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:34.692+0000 7f1b03fff640 1 -- 172.21.15.43:0/1200410542 --> v1:172.21.15.43:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f1af8003520 con 0x7f1b0c143c20 2023-08-09T13:34:34.696 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:34.692+0000 7f1b11c89640 1 --2- 172.21.15.43:0/1200410542 >> v2:172.21.15.203:3300/0 conn(0x7f1b0c06cfe0 0x7f1b0c0c8b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:34:34.696 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:34.692+0000 7f1b03fff640 1 -- 172.21.15.43:0/1200410542 <== mon.0 v1:172.21.15.43:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 763407104 0 0) 0x7f1af8003520 con 0x7f1b0c143c20 2023-08-09T13:34:34.696 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:34.692+0000 7f1b03fff640 1 -- 172.21.15.43:0/1200410542 >> v2:172.21.15.203:3300/0 conn(0x7f1b0c06cfe0 msgr2=0x7f1b0c0c8b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-09T13:34:34.696 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:34.692+0000 7f1b03fff640 1 --2- 172.21.15.43:0/1200410542 >> v2:172.21.15.203:3300/0 conn(0x7f1b0c06cfe0 0x7f1b0c0c8b50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:34:34.696 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:34.692+0000 7f1b03fff640 1 -- 172.21.15.43:0/1200410542 >> v1:172.21.15.203:6789/0 conn(0x7f1b0c0c86f0 legacy=0x7f1b0c06c8d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:34:34.697 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:34.692+0000 7f1b03fff640 1 -- 172.21.15.43:0/1200410542 --> v1:172.21.15.43:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1b0c06fc50 con 0x7f1b0c143c20 2023-08-09T13:34:34.697 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:34.692+0000 7f1b03fff640 1 -- 172.21.15.43:0/1200410542 <== mon.0 v1:172.21.15.43:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1285150740 0 0) 0x7f1b08001670 con 0x7f1b0c143c20 2023-08-09T13:34:34.697 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:34.692+0000 7f1b03fff640 1 -- 172.21.15.43:0/1200410542 >> v1:172.21.15.43:6789/0 conn(0x7f1b0c143c20 legacy=0x7f1b0c06f540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:34:34.697 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:34.692+0000 7f1b03fff640 1 --2- 172.21.15.43:0/1200410542 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f1af80039e0 0x7f1af8003e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:34:34.697 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:34.692+0000 7f1b03fff640 1 --2- 172.21.15.43:0/1200410542 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f1af80047c0 0x7f1af8004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:34:34.697 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:34.692+0000 7f1b03fff640 1 --2- 172.21.15.43:0/1200410542 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f1af8005330 0x7f1af800fdb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:34:34.697 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:34.692+0000 7f1b03fff640 1 -- 172.21.15.43:0/1200410542 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1b0c144000 con 0x7f1af80039e0 2023-08-09T13:34:34.697 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:34.692+0000 7f1b03fff640 1 -- 172.21.15.43:0/1200410542 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1af8010690 con 0x7f1af8005330 2023-08-09T13:34:34.698 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:34.692+0000 7f1b03fff640 1 -- 172.21.15.43:0/1200410542 --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1af8010850 con 0x7f1af80047c0 2023-08-09T13:34:34.698 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:34.692+0000 7f1b11c89640 1 --2- 172.21.15.43:0/1200410542 >> v2:172.21.15.203:3300/0 conn(0x7f1b0c06cfe0 0x7f1b0c0c8b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-08-09T13:34:34.698 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:34.692+0000 7f1b11488640 1 --2- 172.21.15.43:0/1200410542 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f1af80039e0 0x7f1af8003e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:34:34.698 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:34.693+0000 7f1b11c89640 1 --2- 172.21.15.43:0/1200410542 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f1af8005330 0x7f1af800fdb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:34:34.698 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:34.693+0000 7f1b10c87640 1 --2- 172.21.15.43:0/1200410542 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f1af80047c0 0x7f1af8004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:34:34.698 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:34.693+0000 7f1b11488640 1 -- 172.21.15.43:0/1200410542 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f1af80047c0 msgr2=0x7f1af8004c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-09T13:34:34.698 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:34.693+0000 7f1b11488640 1 --2- 172.21.15.43:0/1200410542 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f1af80047c0 0x7f1af8004c40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:34:34.698 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:34.693+0000 7f1b11488640 1 -- 172.21.15.43:0/1200410542 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f1af8005330 msgr2=0x7f1af800fdb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-09T13:34:34.699 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:34.693+0000 7f1b11488640 1 --2- 172.21.15.43:0/1200410542 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f1af8005330 0x7f1af800fdb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:34:34.699 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:34.693+0000 7f1b11488640 1 -- 172.21.15.43:0/1200410542 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1af8003520 con 0x7f1af80039e0 2023-08-09T13:34:34.699 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:34.693+0000 7f1b11c89640 1 --2- 172.21.15.43:0/1200410542 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f1af8005330 0x7f1af800fdb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-08-09T13:34:34.699 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:34.693+0000 7f1b11c89640 1 -- 172.21.15.43:0/1200410542 reap_dead start 2023-08-09T13:34:34.700 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:34.694+0000 7f1b11488640 1 --2- 172.21.15.43:0/1200410542 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f1af80039e0 0x7f1af8003e00 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7f1b0800c240 tx=0x7f1af80057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9201914701cdb4d5 server_cookie=0 in_seq=0 out_seq=0 2023-08-09T13:34:34.700 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:34.694+0000 7f1b03fff640 1 -- 172.21.15.43:0/1200410542 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1b08005ad0 con 0x7f1af80039e0 2023-08-09T13:34:34.700 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:34.694+0000 7f1b03fff640 1 -- 172.21.15.43:0/1200410542 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1b08005c30 con 0x7f1af80039e0 2023-08-09T13:34:34.701 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:34.694+0000 7f1b03fff640 1 -- 172.21.15.43:0/1200410542 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1b0802caf0 con 0x7f1af80039e0 2023-08-09T13:34:34.701 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:34.695+0000 7f1b136ec640 1 -- 172.21.15.43:0/1200410542 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f1af80039e0 msgr2=0x7f1af8003e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:34:34.701 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:34.695+0000 7f1b136ec640 1 --2- 172.21.15.43:0/1200410542 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f1af80039e0 0x7f1af8003e00 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7f1b0800c240 tx=0x7f1af80057f0 comp rx=0 tx=0).stop 2023-08-09T13:34:34.701 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:34.695+0000 7f1b136ec640 1 -- 172.21.15.43:0/1200410542 shutdown_connections 2023-08-09T13:34:34.701 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:34.695+0000 7f1b136ec640 1 --2- 172.21.15.43:0/1200410542 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f1af80039e0 0x7f1af8003e00 unknown :-1 s=CLOSED pgs=153 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:34:34.701 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:34.695+0000 7f1b136ec640 1 -- 172.21.15.43:0/1200410542 >> 172.21.15.43:0/1200410542 conn(0x7f1b0c0ce0a0 msgr2=0x7f1b0c0ce480 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-09T13:34:34.701 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:34.695+0000 7f1b136ec640 1 -- 172.21.15.43:0/1200410542 shutdown_connections 2023-08-09T13:34:34.701 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:34.695+0000 7f1b136ec640 1 -- 172.21.15.43:0/1200410542 wait complete. 2023-08-09T13:34:34.702 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:34.697+0000 7f1b136ec640 1 Processor -- start 2023-08-09T13:34:34.702 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:34.697+0000 7f1b136ec640 1 -- start start 2023-08-09T13:34:34.703 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:34.698+0000 7f1b136ec640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f1af80039e0 0x7f1b0c13d870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:34:34.703 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:34.698+0000 7f1b136ec640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f1b0c15c040 0x7f1b0c0b0d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:34:34.704 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:34.698+0000 7f1b136ec640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f1b0c1408c0 0x7f1b0c140d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:34:34.704 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:34.698+0000 7f1b136ec640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1b0c141280 con 0x7f1af80039e0 2023-08-09T13:34:34.704 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:34.698+0000 7f1b136ec640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1b0c13e120 con 0x7f1b0c15c040 2023-08-09T13:34:34.704 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:34.698+0000 7f1b136ec640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1b0c15c600 con 0x7f1b0c1408c0 2023-08-09T13:34:34.704 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:34.698+0000 7f1b11488640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f1af80039e0 0x7f1b0c13d870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:34:34.704 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:34.698+0000 7f1b11488640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f1af80039e0 0x7f1b0c13d870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:37696/0 (socket says 172.21.15.43:37696) 2023-08-09T13:34:34.704 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:34.698+0000 7f1b11488640 1 -- 172.21.15.43:0/1237375473 learned_addr learned my addr 172.21.15.43:0/1237375473 (peer_addr_for_me v2:172.21.15.43:0/0) 2023-08-09T13:34:34.705 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:34.699+0000 7f1b10c87640 1 --2- 172.21.15.43:0/1237375473 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f1b0c15c040 0x7f1b0c0b0d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:34:34.705 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:34.699+0000 7f1b11488640 1 -- 172.21.15.43:0/1237375473 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f1b0c1408c0 msgr2=0x7f1b0c140d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:34:34.705 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:34.699+0000 7f1b11c89640 1 --2- 172.21.15.43:0/1237375473 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f1b0c1408c0 0x7f1b0c140d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:34:34.705 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:34.699+0000 7f1b11488640 1 --2- 172.21.15.43:0/1237375473 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f1b0c1408c0 0x7f1b0c140d40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:34:34.705 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:34.699+0000 7f1b11488640 1 -- 172.21.15.43:0/1237375473 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f1b0c15c040 msgr2=0x7f1b0c0b0d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:34:34.705 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:34.699+0000 7f1b11488640 1 --2- 172.21.15.43:0/1237375473 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f1b0c15c040 0x7f1b0c0b0d80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:34:34.705 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:34.699+0000 7f1b11488640 1 -- 172.21.15.43:0/1237375473 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1b0c15c8e0 con 0x7f1af80039e0 2023-08-09T13:34:34.705 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:34.699+0000 7f1b10c87640 1 --2- 172.21.15.43:0/1237375473 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f1b0c15c040 0x7f1b0c0b0d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-08-09T13:34:34.706 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:34.699+0000 7f1b11488640 1 --2- 172.21.15.43:0/1237375473 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f1af80039e0 0x7f1b0c13d870 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7f1b0800c240 tx=0x7f1b0802e070 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-09T13:34:34.706 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:34.700+0000 7f1b01ffb640 1 -- 172.21.15.43:0/1237375473 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1b0803c040 con 0x7f1af80039e0 2023-08-09T13:34:34.706 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:34.700+0000 7f1b01ffb640 1 -- 172.21.15.43:0/1237375473 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1b0802ccc0 con 0x7f1af80039e0 2023-08-09T13:34:34.706 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:34.700+0000 7f1b01ffb640 1 -- 172.21.15.43:0/1237375473 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1b080072b0 con 0x7f1af80039e0 2023-08-09T13:34:34.706 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:34.700+0000 7f1b136ec640 1 -- 172.21.15.43:0/1237375473 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1b0c15cbc0 con 0x7f1af80039e0 2023-08-09T13:34:34.706 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:34.700+0000 7f1b136ec640 1 -- 172.21.15.43:0/1237375473 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1b0c15d180 con 0x7f1af80039e0 2023-08-09T13:34:34.706 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:34.701+0000 7f1b136ec640 1 -- 172.21.15.43:0/1237375473 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1b0c15d310 con 0x7f1af80039e0 2023-08-09T13:34:34.707 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:34.702+0000 7f1b01ffb640 1 -- 172.21.15.43:0/1237375473 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 5) v1 ==== 88341+0+0 (secure 0 0 0) 0x7f1b08032070 con 0x7f1af80039e0 2023-08-09T13:34:34.707 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:34.703+0000 7f1b01ffb640 1 --2- 172.21.15.43:0/1237375473 >> [v2:172.21.15.203:6832/830097474,v1:172.21.15.203:6833/830097474] conn(0x7f1af006a050 0x7f1af006c510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:34:34.707 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:34.703+0000 7f1b01ffb640 1 -- 172.21.15.43:0/1237375473 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6813+0+0 (secure 0 0 0) 0x7f1b080a9ea0 con 0x7f1af80039e0 2023-08-09T13:34:34.707 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:34.704+0000 7f1b10c87640 1 --2- 172.21.15.43:0/1237375473 >> [v2:172.21.15.203:6832/830097474,v1:172.21.15.203:6833/830097474] conn(0x7f1af006a050 0x7f1af006c510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:34:34.708 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:34.705+0000 7f1b10c87640 1 --2- 172.21.15.43:0/1237375473 >> [v2:172.21.15.203:6832/830097474,v1:172.21.15.203:6833/830097474] conn(0x7f1af006a050 0x7f1af006c510 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7f1b0c0af3a0 tx=0x7f1afc009040 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-09T13:34:34.710 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:34.708+0000 7f1b01ffb640 1 -- 172.21.15.43:0/1237375473 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178066 (secure 0 0 0) 0x7f1b08075320 con 0x7f1af80039e0 2023-08-09T13:34:34.908 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:34.905+0000 7f1b136ec640 1 -- 172.21.15.43:0/1237375473 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f1b0c143480 con 0x7f1af80039e0 2023-08-09T13:34:34.911 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:34.909+0000 7f1b01ffb640 1 -- 172.21.15.43:0/1237375473 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v24) v1 ==== 74+0+20050 (secure 0 0 0) 0x7f1b08078940 con 0x7f1af80039e0 2023-08-09T13:34:34.912 INFO:teuthology.orchestra.run.smithi043.stdout: 2023-08-09T13:34:34.912 INFO:teuthology.orchestra.run.smithi043.stdout:{"epoch":24,"fsid":"daac87ee-1337-4250-a955-aea4e408a39b","created":"2023-08-09T13:33:41.701198+0000","modified":"2023-08-09T13:34:07.709429+0000","last_up_change":"2023-08-09T13:33:53.783453+0000","last_in_change":"2023-08-09T13:33:45.148904+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":5,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":"rbd","create_time":"2023-08-09T13:33:54.365137+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":".mgr","create_time":"2023-08-09T13:33:54.831803+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_metadata","create_time":"2023-08-09T13:34:00.207974+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data","create_time":"2023-08-09T13:34:01.614765+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":5,"pool_name":"cephfs_data_ec","create_time":"2023-08-09T13:34:05.239583+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"24","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"cbd49ecd-e896-4fbd-864e-5c37a829b165","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6800","nonce":502667335},{"type":"v1","addr":"172.21.15.43:6801","nonce":502667335}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6802","nonce":502667335},{"type":"v1","addr":"172.21.15.43:6803","nonce":502667335}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6806","nonce":502667335},{"type":"v1","addr":"172.21.15.43:6807","nonce":502667335}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6804","nonce":502667335},{"type":"v1","addr":"172.21.15.43:6805","nonce":502667335}]},"public_addr":"172.21.15.43:6801/502667335","cluster_addr":"172.21.15.43:6803/502667335","heartbeat_back_addr":"172.21.15.43:6807/502667335","heartbeat_front_addr":"172.21.15.43:6805/502667335","state":["exists","up"]},{"osd":1,"uuid":"75c7e0fe-1ec7-4fe8-a2f2-3661a65b8500","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6808","nonce":3289134610},{"type":"v1","addr":"172.21.15.43:6809","nonce":3289134610}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6810","nonce":3289134610},{"type":"v1","addr":"172.21.15.43:6812","nonce":3289134610}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6821","nonce":3289134610},{"type":"v1","addr":"172.21.15.43:6824","nonce":3289134610}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6815","nonce":3289134610},{"type":"v1","addr":"172.21.15.43:6818","nonce":3289134610}]},"public_addr":"172.21.15.43:6809/3289134610","cluster_addr":"172.21.15.43:6812/3289134610","heartbeat_back_addr":"172.21.15.43:6824/3289134610","heartbeat_front_addr":"172.21.15.43:6818/3289134610","state":["exists","up"]},{"osd":2,"uuid":"bb5c41e9-7f5f-4125-9b01-3bb8bfc2ea83","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6811","nonce":4079709736},{"type":"v1","addr":"172.21.15.43:6814","nonce":4079709736}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6817","nonce":4079709736},{"type":"v1","addr":"172.21.15.43:6820","nonce":4079709736}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6829","nonce":4079709736},{"type":"v1","addr":"172.21.15.43:6831","nonce":4079709736}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6823","nonce":4079709736},{"type":"v1","addr":"172.21.15.43:6826","nonce":4079709736}]},"public_addr":"172.21.15.43:6814/4079709736","cluster_addr":"172.21.15.43:6820/4079709736","heartbeat_back_addr":"172.21.15.43:6831/4079709736","heartbeat_front_addr":"172.21.15.43:6826/4079709736","state":["exists","up"]},{"osd":3,"uuid":"bfe4c8b4-694a-465c-87e6-2b2aab852613","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6813","nonce":1709538759},{"type":"v1","addr":"172.21.15.43:6816","nonce":1709538759}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6819","nonce":1709538759},{"type":"v1","addr":"172.21.15.43:6822","nonce":1709538759}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6828","nonce":1709538759},{"type":"v1","addr":"172.21.15.43:6830","nonce":1709538759}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6825","nonce":1709538759},{"type":"v1","addr":"172.21.15.43:6827","nonce":1709538759}]},"public_addr":"172.21.15.43:6816/1709538759","cluster_addr":"172.21.15.43:6822/1709538759","heartbeat_back_addr":"172.21.15.43:6830/1709538759","heartbeat_front_addr":"172.21.15.43:6827/1709538759","state":["exists","up"]},{"osd":4,"uuid":"fa2e8a77-aba1-4756-853e-0d174a5bdeba","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6800","nonce":4196866481},{"type":"v1","addr":"172.21.15.203:6801","nonce":4196866481}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6802","nonce":4196866481},{"type":"v1","addr":"172.21.15.203:6803","nonce":4196866481}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6806","nonce":4196866481},{"type":"v1","addr":"172.21.15.203:6807","nonce":4196866481}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6804","nonce":4196866481},{"type":"v1","addr":"172.21.15.203:6805","nonce":4196866481}]},"public_addr":"172.21.15.203:6801/4196866481","cluster_addr":"172.21.15.203:6803/4196866481","heartbeat_back_addr":"172.21.15.203:6807/4196866481","heartbeat_front_addr":"172.21.15.203:6805/4196866481","state":["exists","up"]},{"osd":5,"uuid":"81811e4b-4d24-4f51-a82c-f1d700a2a00d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6816","nonce":3059290081},{"type":"v1","addr":"172.21.15.203:6817","nonce":3059290081}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6818","nonce":3059290081},{"type":"v1","addr":"172.21.15.203:6819","nonce":3059290081}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6822","nonce":3059290081},{"type":"v1","addr":"172.21.15.203:6823","nonce":3059290081}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6820","nonce":3059290081},{"type":"v1","addr":"172.21.15.203:6821","nonce":3059290081}]},"public_addr":"172.21.15.203:6817/3059290081","cluster_addr":"172.21.15.203:6819/3059290081","heartbeat_back_addr":"172.21.15.203:6823/3059290081","heartbeat_front_addr":"172.21.15.203:6821/3059290081","state":["exists","up"]},{"osd":6,"uuid":"307bac4d-6a5b-443d-af75-56d695d7674f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6824","nonce":512155569},{"type":"v1","addr":"172.21.15.203:6825","nonce":512155569}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6826","nonce":512155569},{"type":"v1","addr":"172.21.15.203:6827","nonce":512155569}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6830","nonce":512155569},{"type":"v1","addr":"172.21.15.203:6831","nonce":512155569}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6828","nonce":512155569},{"type":"v1","addr":"172.21.15.203:6829","nonce":512155569}]},"public_addr":"172.21.15.203:6825/512155569","cluster_addr":"172.21.15.203:6827/512155569","heartbeat_back_addr":"172.21.15.203:6831/512155569","heartbeat_front_addr":"172.21.15.203:6829/512155569","state":["exists","up"]},{"osd":7,"uuid":"a95dc7ca-9f1b-4295-af70-68a22a2d7d5e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6808","nonce":3331450438},{"type":"v1","addr":"172.21.15.203:6809","nonce":3331450438}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6810","nonce":3331450438},{"type":"v1","addr":"172.21.15.203:6811","nonce":3331450438}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6814","nonce":3331450438},{"type":"v1","addr":"172.21.15.203:6815","nonce":3331450438}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6812","nonce":3331450438},{"type":"v1","addr":"172.21.15.203:6813","nonce":3331450438}]},"public_addr":"172.21.15.203:6809/3331450438","cluster_addr":"172.21.15.203:6811/3331450438","heartbeat_back_addr":"172.21.15.203:6815/3331450438","heartbeat_front_addr":"172.21.15.203:6813/3331450438","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-09T13:33:50.547652+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-09T13:33:50.589469+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-09T13:33:50.561724+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-09T13:33:50.599650+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-09T13:33:50.411855+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-09T13:33:50.219681+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-09T13:33:50.504664+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-09T13:33:50.441248+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-08-09T13:34:34.914 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:34.912+0000 7f1b136ec640 1 -- 172.21.15.43:0/1237375473 >> [v2:172.21.15.203:6832/830097474,v1:172.21.15.203:6833/830097474] conn(0x7f1af006a050 msgr2=0x7f1af006c510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:34:34.914 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:34.912+0000 7f1b136ec640 1 --2- 172.21.15.43:0/1237375473 >> [v2:172.21.15.203:6832/830097474,v1:172.21.15.203:6833/830097474] conn(0x7f1af006a050 0x7f1af006c510 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7f1b0c0af3a0 tx=0x7f1afc009040 comp rx=0 tx=0).stop 2023-08-09T13:34:34.915 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:34.912+0000 7f1b136ec640 1 -- 172.21.15.43:0/1237375473 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f1af80039e0 msgr2=0x7f1b0c13d870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:34:34.915 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:34.912+0000 7f1b136ec640 1 --2- 172.21.15.43:0/1237375473 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f1af80039e0 0x7f1b0c13d870 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7f1b0800c240 tx=0x7f1b0802e070 comp rx=0 tx=0).stop 2023-08-09T13:34:34.915 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:34.912+0000 7f1b136ec640 1 -- 172.21.15.43:0/1237375473 shutdown_connections 2023-08-09T13:34:34.915 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:34.912+0000 7f1b136ec640 1 --2- 172.21.15.43:0/1237375473 >> [v2:172.21.15.203:6832/830097474,v1:172.21.15.203:6833/830097474] conn(0x7f1af006a050 0x7f1af006c510 unknown :-1 s=CLOSED pgs=96 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:34:34.915 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:34.912+0000 7f1b136ec640 1 --2- 172.21.15.43:0/1237375473 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f1b0c1408c0 0x7f1b0c140d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:34:34.915 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:34.912+0000 7f1b136ec640 1 --2- 172.21.15.43:0/1237375473 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f1b0c15c040 0x7f1b0c0b0d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:34:34.915 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:34.912+0000 7f1b136ec640 1 --2- 172.21.15.43:0/1237375473 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f1af80039e0 0x7f1b0c13d870 unknown :-1 s=CLOSED pgs=154 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:34:34.916 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:34.912+0000 7f1b136ec640 1 -- 172.21.15.43:0/1237375473 >> 172.21.15.43:0/1237375473 conn(0x7f1b0c0ce0a0 msgr2=0x7f1b0c0c4050 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-09T13:34:34.916 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:34.912+0000 7f1b136ec640 1 -- 172.21.15.43:0/1237375473 shutdown_connections 2023-08-09T13:34:34.916 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:34.912+0000 7f1b136ec640 1 -- 172.21.15.43:0/1237375473 wait complete. 2023-08-09T13:34:34.937 INFO:tasks.ceph.ceph_manager.ceph:all up! 2023-08-09T13:34:34.938 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-08-09T13:34:35.169 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:35.166+0000 7f77cb59e640 1 Processor -- start 2023-08-09T13:34:35.169 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:35.166+0000 7f77cb59e640 1 -- start start 2023-08-09T13:34:35.170 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:35.168+0000 7f77cb59e640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f77cc0acdf0 0x7f77cc0ad340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:34:35.171 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:35.168+0000 7f77cb59e640 1 -- --> v1:172.21.15.43:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f77cc0cbd90 con 0x7f77cc0c8b30 2023-08-09T13:34:35.171 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:35.168+0000 7f77cb59e640 1 -- --> v1:172.21.15.203:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f77cc0ca2b0 con 0x7f77cc0c8750 2023-08-09T13:34:35.171 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:35.168+0000 7f77cb59e640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f77cc15e810 con 0x7f77cc0acdf0 2023-08-09T13:34:35.172 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:35.169+0000 7f77c9d9b640 1 --1- >> v1:172.21.15.43:6789/0 conn(0x7f77cc0c8b30 0x7f77cc070f00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.43:6789/0 says I am v1:172.21.15.43:35580/0 (socket says 172.21.15.43:35580) 2023-08-09T13:34:35.172 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:35.169+0000 7f77ca59c640 1 --1- >> v1:172.21.15.203:6789/0 conn(0x7f77cc0c8750 0x7f77cc06f410 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.203:6789/0 says I am v1:172.21.15.43:33708/0 (socket says 172.21.15.43:33708) 2023-08-09T13:34:35.172 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:35.169+0000 7f77c9d9b640 1 -- 172.21.15.43:0/1569842410 learned_addr learned my addr 172.21.15.43:0/1569842410 (peer_addr_for_me v1:172.21.15.43:0/0) 2023-08-09T13:34:35.172 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:35.169+0000 7f77cad9d640 1 --2- 172.21.15.43:0/1569842410 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f77cc0acdf0 0x7f77cc0ad340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:34:35.172 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:35.169+0000 7f77c959a640 1 -- 172.21.15.43:0/1569842410 <== mon.0 v1:172.21.15.43:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 951389571 0 0) 0x7f77cc0cbd90 con 0x7f77cc0c8b30 2023-08-09T13:34:35.172 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:35.169+0000 7f77c959a640 1 -- 172.21.15.43:0/1569842410 --> v1:172.21.15.43:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f77b4003520 con 0x7f77cc0c8b30 2023-08-09T13:34:35.173 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:35.169+0000 7f77c959a640 1 -- 172.21.15.43:0/1569842410 <== mon.1 v1:172.21.15.203:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 500587110 0 0) 0x7f77cc0ca2b0 con 0x7f77cc0c8750 2023-08-09T13:34:35.173 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:35.169+0000 7f77c959a640 1 -- 172.21.15.43:0/1569842410 --> v1:172.21.15.203:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f77cc0cbd90 con 0x7f77cc0c8750 2023-08-09T13:34:35.173 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:35.170+0000 7f77cad9d640 1 -- 172.21.15.43:0/1569842410 >> v1:172.21.15.203:6789/0 conn(0x7f77cc0c8750 legacy=0x7f77cc06f410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:34:35.173 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:35.170+0000 7f77cad9d640 1 -- 172.21.15.43:0/1569842410 >> v1:172.21.15.43:6789/0 conn(0x7f77cc0c8b30 legacy=0x7f77cc070f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:34:35.174 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:35.170+0000 7f77c959a640 1 -- 172.21.15.43:0/1569842410 <== mon.0 v1:172.21.15.43:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3271409200 0 0) 0x7f77b4003520 con 0x7f77cc0c8b30 2023-08-09T13:34:35.174 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:35.170+0000 7f77cad9d640 1 -- 172.21.15.43:0/1569842410 --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f77cc0c9a20 con 0x7f77cc0acdf0 2023-08-09T13:34:35.174 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:35.171+0000 7f77cad9d640 1 --2- 172.21.15.43:0/1569842410 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f77cc0acdf0 0x7f77cc0ad340 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7f77c4009dc0 tx=0x7f77c402bd50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=2467a55c8f94cae1 server_cookie=0 in_seq=0 out_seq=0 2023-08-09T13:34:35.174 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:35.171+0000 7f77c959a640 1 -- 172.21.15.43:0/1569842410 <== mon.2 v2:172.21.15.203:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f77c0001670 con 0x7f77cc0acdf0 2023-08-09T13:34:35.175 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:35.172+0000 7f77c959a640 1 -- 172.21.15.43:0/1569842410 <== mon.2 v2:172.21.15.203:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f77c402c880 con 0x7f77cc0acdf0 2023-08-09T13:34:35.175 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:35.172+0000 7f77c959a640 1 -- 172.21.15.43:0/1569842410 <== mon.2 v2:172.21.15.203:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f77c402cc10 con 0x7f77cc0acdf0 2023-08-09T13:34:35.175 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:35.172+0000 7f77cb59e640 1 -- 172.21.15.43:0/1569842410 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f77cc0acdf0 msgr2=0x7f77cc0ad340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:34:35.175 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:35.172+0000 7f77cb59e640 1 --2- 172.21.15.43:0/1569842410 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f77cc0acdf0 0x7f77cc0ad340 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7f77c4009dc0 tx=0x7f77c402bd50 comp rx=0 tx=0).stop 2023-08-09T13:34:35.176 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:35.172+0000 7f77cb59e640 1 -- 172.21.15.43:0/1569842410 shutdown_connections 2023-08-09T13:34:35.176 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:35.172+0000 7f77cb59e640 1 --2- 172.21.15.43:0/1569842410 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f77cc0acdf0 0x7f77cc0ad340 unknown :-1 s=CLOSED pgs=43 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:34:35.176 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:35.172+0000 7f77cb59e640 1 -- 172.21.15.43:0/1569842410 >> 172.21.15.43:0/1569842410 conn(0x7f77cc0cde90 msgr2=0x7f77cc0ce2a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-09T13:34:35.176 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:35.172+0000 7f77cb59e640 1 -- 172.21.15.43:0/1569842410 shutdown_connections 2023-08-09T13:34:35.176 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:35.173+0000 7f77cb59e640 1 -- 172.21.15.43:0/1569842410 wait complete. 2023-08-09T13:34:35.177 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:35.174+0000 7f77cb59e640 1 Processor -- start 2023-08-09T13:34:35.177 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:35.175+0000 7f77cb59e640 1 -- start start 2023-08-09T13:34:35.178 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:35.175+0000 7f77cb59e640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f77cc0acdf0 0x7f77cc09b280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:34:35.178 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:35.176+0000 7f77cb59e640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f77cc0c8750 0x7f77cc155f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:34:35.178 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:35.176+0000 7f77cb59e640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f77cc0c8b30 0x7f77cc1564c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:34:35.178 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:35.176+0000 7f77cb59e640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f77cc0c9f70 con 0x7f77cc0c8750 2023-08-09T13:34:35.179 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:35.176+0000 7f77cb59e640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f77cc0c5a10 con 0x7f77cc0c8b30 2023-08-09T13:34:35.179 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:35.176+0000 7f77cb59e640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f77cc0c5cf0 con 0x7f77cc0acdf0 2023-08-09T13:34:35.180 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:35.176+0000 7f77c9d9b640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f77cc0c8750 0x7f77cc155f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:34:35.180 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:35.176+0000 7f77c9d9b640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f77cc0c8750 0x7f77cc155f80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:51564/0 (socket says 172.21.15.43:51564) 2023-08-09T13:34:35.180 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:35.176+0000 7f77c9d9b640 1 -- 172.21.15.43:0/1485669796 learned_addr learned my addr 172.21.15.43:0/1485669796 (peer_addr_for_me v2:172.21.15.43:0/0) 2023-08-09T13:34:35.180 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:35.176+0000 7f77ca59c640 1 --2- 172.21.15.43:0/1485669796 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f77cc0acdf0 0x7f77cc09b280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:34:35.181 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:35.176+0000 7f77cad9d640 1 --2- 172.21.15.43:0/1485669796 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f77cc0c8b30 0x7f77cc1564c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:34:35.181 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:35.176+0000 7f77c9d9b640 1 -- 172.21.15.43:0/1485669796 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f77cc0acdf0 msgr2=0x7f77cc09b280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:34:35.181 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:35.176+0000 7f77c9d9b640 1 --2- 172.21.15.43:0/1485669796 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f77cc0acdf0 0x7f77cc09b280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:34:35.181 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:35.176+0000 7f77c9d9b640 1 -- 172.21.15.43:0/1485669796 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f77cc0c8b30 msgr2=0x7f77cc1564c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:34:35.181 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:35.176+0000 7f77c9d9b640 1 --2- 172.21.15.43:0/1485669796 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f77cc0c8b30 0x7f77cc1564c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:34:35.181 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:35.177+0000 7f77c9d9b640 1 -- 172.21.15.43:0/1485669796 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f77c4009a70 con 0x7f77cc0c8750 2023-08-09T13:34:35.181 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:35.177+0000 7f77c9d9b640 1 --2- 172.21.15.43:0/1485669796 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f77cc0c8750 0x7f77cc155f80 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7f77c000c750 tx=0x7f77c000cc10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-09T13:34:35.181 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:35.177+0000 7f77baffd640 1 -- 172.21.15.43:0/1485669796 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f77c0005ad0 con 0x7f77cc0c8750 2023-08-09T13:34:35.182 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:35.177+0000 7f77baffd640 1 -- 172.21.15.43:0/1485669796 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f77c0005c30 con 0x7f77cc0c8750 2023-08-09T13:34:35.182 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:35.177+0000 7f77baffd640 1 -- 172.21.15.43:0/1485669796 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f77c0007070 con 0x7f77cc0c8750 2023-08-09T13:34:35.182 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:35.177+0000 7f77cb59e640 1 -- 172.21.15.43:0/1485669796 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f77cc0c5fd0 con 0x7f77cc0c8750 2023-08-09T13:34:35.182 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:35.177+0000 7f77cb59e640 1 -- 172.21.15.43:0/1485669796 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f77cc0c6590 con 0x7f77cc0c8750 2023-08-09T13:34:35.182 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:35.178+0000 7f77cb59e640 1 -- 172.21.15.43:0/1485669796 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f77cc0c9230 con 0x7f77cc0c8750 2023-08-09T13:34:35.183 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:35.180+0000 7f77baffd640 1 -- 172.21.15.43:0/1485669796 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 5) v1 ==== 88341+0+0 (secure 0 0 0) 0x7f77c00071d0 con 0x7f77cc0c8750 2023-08-09T13:34:35.183 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:35.180+0000 7f77baffd640 1 --2- 172.21.15.43:0/1485669796 >> [v2:172.21.15.203:6832/830097474,v1:172.21.15.203:6833/830097474] conn(0x7f77ac06a050 0x7f77ac06c510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:34:35.183 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:35.181+0000 7f77baffd640 1 -- 172.21.15.43:0/1485669796 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6813+0+0 (secure 0 0 0) 0x7f77c0088730 con 0x7f77cc0c8750 2023-08-09T13:34:35.183 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:35.181+0000 7f77ca59c640 1 --2- 172.21.15.43:0/1485669796 >> [v2:172.21.15.203:6832/830097474,v1:172.21.15.203:6833/830097474] conn(0x7f77ac06a050 0x7f77ac06c510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:34:35.184 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:35.182+0000 7f77ca59c640 1 --2- 172.21.15.43:0/1485669796 >> [v2:172.21.15.203:6832/830097474,v1:172.21.15.203:6833/830097474] conn(0x7f77ac06a050 0x7f77ac06c510 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7f77bc0011e0 tx=0x7f77bc007b50 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-09T13:34:35.187 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:35.185+0000 7f77baffd640 1 -- 172.21.15.43:0/1485669796 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178066 (secure 0 0 0) 0x7f77c0053a00 con 0x7f77cc0c8750 2023-08-09T13:34:35.383 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:35.380+0000 7f77cb59e640 1 -- 172.21.15.43:0/1485669796 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f77cc0739f0 con 0x7f77cc0c8750 2023-08-09T13:34:35.386 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:35.383+0000 7f77baffd640 1 -- 172.21.15.43:0/1485669796 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v24) v1 ==== 74+0+20050 (secure 0 0 0) 0x7f77c0057020 con 0x7f77cc0c8750 2023-08-09T13:34:35.386 INFO:teuthology.orchestra.run.smithi043.stdout: 2023-08-09T13:34:35.386 INFO:teuthology.orchestra.run.smithi043.stdout:{"epoch":24,"fsid":"daac87ee-1337-4250-a955-aea4e408a39b","created":"2023-08-09T13:33:41.701198+0000","modified":"2023-08-09T13:34:07.709429+0000","last_up_change":"2023-08-09T13:33:53.783453+0000","last_in_change":"2023-08-09T13:33:45.148904+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":5,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":"rbd","create_time":"2023-08-09T13:33:54.365137+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":".mgr","create_time":"2023-08-09T13:33:54.831803+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_metadata","create_time":"2023-08-09T13:34:00.207974+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data","create_time":"2023-08-09T13:34:01.614765+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":5,"pool_name":"cephfs_data_ec","create_time":"2023-08-09T13:34:05.239583+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"24","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"cbd49ecd-e896-4fbd-864e-5c37a829b165","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6800","nonce":502667335},{"type":"v1","addr":"172.21.15.43:6801","nonce":502667335}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6802","nonce":502667335},{"type":"v1","addr":"172.21.15.43:6803","nonce":502667335}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6806","nonce":502667335},{"type":"v1","addr":"172.21.15.43:6807","nonce":502667335}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6804","nonce":502667335},{"type":"v1","addr":"172.21.15.43:6805","nonce":502667335}]},"public_addr":"172.21.15.43:6801/502667335","cluster_addr":"172.21.15.43:6803/502667335","heartbeat_back_addr":"172.21.15.43:6807/502667335","heartbeat_front_addr":"172.21.15.43:6805/502667335","state":["exists","up"]},{"osd":1,"uuid":"75c7e0fe-1ec7-4fe8-a2f2-3661a65b8500","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6808","nonce":3289134610},{"type":"v1","addr":"172.21.15.43:6809","nonce":3289134610}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6810","nonce":3289134610},{"type":"v1","addr":"172.21.15.43:6812","nonce":3289134610}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6821","nonce":3289134610},{"type":"v1","addr":"172.21.15.43:6824","nonce":3289134610}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6815","nonce":3289134610},{"type":"v1","addr":"172.21.15.43:6818","nonce":3289134610}]},"public_addr":"172.21.15.43:6809/3289134610","cluster_addr":"172.21.15.43:6812/3289134610","heartbeat_back_addr":"172.21.15.43:6824/3289134610","heartbeat_front_addr":"172.21.15.43:6818/3289134610","state":["exists","up"]},{"osd":2,"uuid":"bb5c41e9-7f5f-4125-9b01-3bb8bfc2ea83","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6811","nonce":4079709736},{"type":"v1","addr":"172.21.15.43:6814","nonce":4079709736}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6817","nonce":4079709736},{"type":"v1","addr":"172.21.15.43:6820","nonce":4079709736}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6829","nonce":4079709736},{"type":"v1","addr":"172.21.15.43:6831","nonce":4079709736}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6823","nonce":4079709736},{"type":"v1","addr":"172.21.15.43:6826","nonce":4079709736}]},"public_addr":"172.21.15.43:6814/4079709736","cluster_addr":"172.21.15.43:6820/4079709736","heartbeat_back_addr":"172.21.15.43:6831/4079709736","heartbeat_front_addr":"172.21.15.43:6826/4079709736","state":["exists","up"]},{"osd":3,"uuid":"bfe4c8b4-694a-465c-87e6-2b2aab852613","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6813","nonce":1709538759},{"type":"v1","addr":"172.21.15.43:6816","nonce":1709538759}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6819","nonce":1709538759},{"type":"v1","addr":"172.21.15.43:6822","nonce":1709538759}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6828","nonce":1709538759},{"type":"v1","addr":"172.21.15.43:6830","nonce":1709538759}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6825","nonce":1709538759},{"type":"v1","addr":"172.21.15.43:6827","nonce":1709538759}]},"public_addr":"172.21.15.43:6816/1709538759","cluster_addr":"172.21.15.43:6822/1709538759","heartbeat_back_addr":"172.21.15.43:6830/1709538759","heartbeat_front_addr":"172.21.15.43:6827/1709538759","state":["exists","up"]},{"osd":4,"uuid":"fa2e8a77-aba1-4756-853e-0d174a5bdeba","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6800","nonce":4196866481},{"type":"v1","addr":"172.21.15.203:6801","nonce":4196866481}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6802","nonce":4196866481},{"type":"v1","addr":"172.21.15.203:6803","nonce":4196866481}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6806","nonce":4196866481},{"type":"v1","addr":"172.21.15.203:6807","nonce":4196866481}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6804","nonce":4196866481},{"type":"v1","addr":"172.21.15.203:6805","nonce":4196866481}]},"public_addr":"172.21.15.203:6801/4196866481","cluster_addr":"172.21.15.203:6803/4196866481","heartbeat_back_addr":"172.21.15.203:6807/4196866481","heartbeat_front_addr":"172.21.15.203:6805/4196866481","state":["exists","up"]},{"osd":5,"uuid":"81811e4b-4d24-4f51-a82c-f1d700a2a00d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6816","nonce":3059290081},{"type":"v1","addr":"172.21.15.203:6817","nonce":3059290081}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6818","nonce":3059290081},{"type":"v1","addr":"172.21.15.203:6819","nonce":3059290081}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6822","nonce":3059290081},{"type":"v1","addr":"172.21.15.203:6823","nonce":3059290081}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6820","nonce":3059290081},{"type":"v1","addr":"172.21.15.203:6821","nonce":3059290081}]},"public_addr":"172.21.15.203:6817/3059290081","cluster_addr":"172.21.15.203:6819/3059290081","heartbeat_back_addr":"172.21.15.203:6823/3059290081","heartbeat_front_addr":"172.21.15.203:6821/3059290081","state":["exists","up"]},{"osd":6,"uuid":"307bac4d-6a5b-443d-af75-56d695d7674f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6824","nonce":512155569},{"type":"v1","addr":"172.21.15.203:6825","nonce":512155569}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6826","nonce":512155569},{"type":"v1","addr":"172.21.15.203:6827","nonce":512155569}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6830","nonce":512155569},{"type":"v1","addr":"172.21.15.203:6831","nonce":512155569}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6828","nonce":512155569},{"type":"v1","addr":"172.21.15.203:6829","nonce":512155569}]},"public_addr":"172.21.15.203:6825/512155569","cluster_addr":"172.21.15.203:6827/512155569","heartbeat_back_addr":"172.21.15.203:6831/512155569","heartbeat_front_addr":"172.21.15.203:6829/512155569","state":["exists","up"]},{"osd":7,"uuid":"a95dc7ca-9f1b-4295-af70-68a22a2d7d5e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6808","nonce":3331450438},{"type":"v1","addr":"172.21.15.203:6809","nonce":3331450438}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6810","nonce":3331450438},{"type":"v1","addr":"172.21.15.203:6811","nonce":3331450438}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6814","nonce":3331450438},{"type":"v1","addr":"172.21.15.203:6815","nonce":3331450438}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6812","nonce":3331450438},{"type":"v1","addr":"172.21.15.203:6813","nonce":3331450438}]},"public_addr":"172.21.15.203:6809/3331450438","cluster_addr":"172.21.15.203:6811/3331450438","heartbeat_back_addr":"172.21.15.203:6815/3331450438","heartbeat_front_addr":"172.21.15.203:6813/3331450438","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-09T13:33:50.547652+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-09T13:33:50.589469+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-09T13:33:50.561724+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-09T13:33:50.599650+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-09T13:33:50.411855+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-09T13:33:50.219681+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-09T13:33:50.504664+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-09T13:33:50.441248+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-08-09T13:34:35.388 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:35.386+0000 7f77cb59e640 1 -- 172.21.15.43:0/1485669796 >> [v2:172.21.15.203:6832/830097474,v1:172.21.15.203:6833/830097474] conn(0x7f77ac06a050 msgr2=0x7f77ac06c510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:34:35.388 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:35.386+0000 7f77cb59e640 1 --2- 172.21.15.43:0/1485669796 >> [v2:172.21.15.203:6832/830097474,v1:172.21.15.203:6833/830097474] conn(0x7f77ac06a050 0x7f77ac06c510 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7f77bc0011e0 tx=0x7f77bc007b50 comp rx=0 tx=0).stop 2023-08-09T13:34:35.389 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:35.386+0000 7f77cb59e640 1 -- 172.21.15.43:0/1485669796 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f77cc0c8750 msgr2=0x7f77cc155f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:34:35.389 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:35.386+0000 7f77cb59e640 1 --2- 172.21.15.43:0/1485669796 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f77cc0c8750 0x7f77cc155f80 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7f77c000c750 tx=0x7f77c000cc10 comp rx=0 tx=0).stop 2023-08-09T13:34:35.389 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:35.386+0000 7f77cb59e640 1 -- 172.21.15.43:0/1485669796 shutdown_connections 2023-08-09T13:34:35.389 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:35.386+0000 7f77cb59e640 1 --2- 172.21.15.43:0/1485669796 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f77cc0c8b30 0x7f77cc1564c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:34:35.390 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:35.386+0000 7f77cb59e640 1 --2- 172.21.15.43:0/1485669796 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f77cc0c8750 0x7f77cc155f80 unknown :-1 s=CLOSED pgs=156 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:34:35.390 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:35.386+0000 7f77cb59e640 1 --2- 172.21.15.43:0/1485669796 >> [v2:172.21.15.203:6832/830097474,v1:172.21.15.203:6833/830097474] conn(0x7f77ac06a050 0x7f77ac06c510 unknown :-1 s=CLOSED pgs=97 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:34:35.390 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:35.386+0000 7f77cb59e640 1 --2- 172.21.15.43:0/1485669796 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f77cc0acdf0 0x7f77cc09b280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:34:35.390 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:35.386+0000 7f77cb59e640 1 -- 172.21.15.43:0/1485669796 >> 172.21.15.43:0/1485669796 conn(0x7f77cc0cde90 msgr2=0x7f77cc0cbf20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-09T13:34:35.390 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:35.386+0000 7f77cb59e640 1 -- 172.21.15.43:0/1485669796 shutdown_connections 2023-08-09T13:34:35.390 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:35.386+0000 7f77cb59e640 1 -- 172.21.15.43:0/1485669796 wait complete. 2023-08-09T13:34:35.408 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.0 flush_pg_stats 2023-08-09T13:34:35.409 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.1 flush_pg_stats 2023-08-09T13:34:35.410 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.2 flush_pg_stats 2023-08-09T13:34:35.410 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.3 flush_pg_stats 2023-08-09T13:34:35.410 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.4 flush_pg_stats 2023-08-09T13:34:35.411 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.5 flush_pg_stats 2023-08-09T13:34:35.412 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.6 flush_pg_stats 2023-08-09T13:34:35.412 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.7 flush_pg_stats 2023-08-09T13:34:35.734 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:35.732+0000 7fc17a1ab640 1 Processor -- start 2023-08-09T13:34:35.735 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:35.732+0000 7fc17a1ab640 1 -- start start 2023-08-09T13:34:35.735 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:35.733+0000 7f276e015640 1 Processor -- start 2023-08-09T13:34:35.736 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:35.733+0000 7f276e015640 1 -- start start 2023-08-09T13:34:35.737 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:35.734+0000 7fc17a1ab640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fc1740c84d0 0x7fc1740c8930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:34:35.737 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:35.734+0000 7fc17a1ab640 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7fc1740cba50 0x7fc1740cbea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:34:35.737 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:35.734+0000 7fc17a1ab640 1 -- --> v1:172.21.15.43:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc17415cbd0 con 0x7fc1740cd3e0 2023-08-09T13:34:35.738 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:35.734+0000 7fc17a1ab640 1 -- --> v2:172.21.15.203:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc1740ac440 con 0x7fc1740cba50 2023-08-09T13:34:35.738 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:35.734+0000 7fc17a1ab640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc1740ac720 con 0x7fc1740c84d0 2023-08-09T13:34:35.738 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:35.734+0000 7fc1737fe640 1 --1- >> v1:172.21.15.43:6789/0 conn(0x7fc1740cd3e0 0x7fc1740ab8d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.43:6789/0 says I am v1:172.21.15.43:35588/0 (socket says 172.21.15.43:35588) 2023-08-09T13:34:35.738 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:35.735+0000 7fc1737fe640 1 -- 172.21.15.43:0/1451189023 learned_addr learned my addr 172.21.15.43:0/1451189023 (peer_addr_for_me v1:172.21.15.43:0/0) 2023-08-09T13:34:35.738 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:35.735+0000 7fc173fff640 1 --2- 172.21.15.43:0/1451189023 >> v2:172.21.15.203:3300/0 conn(0x7fc1740cba50 0x7fc1740cbea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:34:35.739 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:35.735+0000 7fc172ffd640 1 --2- 172.21.15.43:0/1451189023 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fc1740c84d0 0x7fc1740c8930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:34:35.739 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:35.735+0000 7fc1727fc640 1 -- 172.21.15.43:0/1451189023 <== mon.0 v1:172.21.15.43:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3216312825 0 0) 0x7fc17415cbd0 con 0x7fc1740cd3e0 2023-08-09T13:34:35.739 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:35.735+0000 7fc1727fc640 1 -- 172.21.15.43:0/1451189023 --> v1:172.21.15.43:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fc160003520 con 0x7fc1740cd3e0 2023-08-09T13:34:35.739 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:35.736+0000 7fc1727fc640 1 -- 172.21.15.43:0/1451189023 <== mon.0 v1:172.21.15.43:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1338649693 0 0) 0x7fc160003520 con 0x7fc1740cd3e0 2023-08-09T13:34:35.739 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:35.736+0000 7fc1727fc640 1 -- 172.21.15.43:0/1451189023 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fc1740c84d0 msgr2=0x7fc1740c8930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-09T13:34:35.739 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:35.736+0000 7fc1727fc640 1 --2- 172.21.15.43:0/1451189023 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fc1740c84d0 0x7fc1740c8930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:34:35.739 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:35.736+0000 7fc1727fc640 1 -- 172.21.15.43:0/1451189023 >> v2:172.21.15.203:3300/0 conn(0x7fc1740cba50 msgr2=0x7fc1740cbea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-09T13:34:35.740 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:35.736+0000 7fc1727fc640 1 --2- 172.21.15.43:0/1451189023 >> v2:172.21.15.203:3300/0 conn(0x7fc1740cba50 0x7fc1740cbea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:34:35.740 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:35.736+0000 7fc1727fc640 1 -- 172.21.15.43:0/1451189023 --> v1:172.21.15.43:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc1740c8e70 con 0x7fc1740cd3e0 2023-08-09T13:34:35.740 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:35.736+0000 7fc1727fc640 1 -- 172.21.15.43:0/1451189023 <== mon.0 v1:172.21.15.43:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1285150740 0 0) 0x7fc164001670 con 0x7fc1740cd3e0 2023-08-09T13:34:35.740 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:35.736+0000 7fc1727fc640 1 -- 172.21.15.43:0/1451189023 >> v1:172.21.15.43:6789/0 conn(0x7fc1740cd3e0 legacy=0x7fc1740ab8d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:34:35.740 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:35.736+0000 7fc1727fc640 1 --2- 172.21.15.43:0/1451189023 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fc1740c84d0 0x7fc160003970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:34:35.740 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:35.736+0000 7fc1727fc640 1 --2- 172.21.15.43:0/1451189023 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fc160003ef0 0x7fc1600047c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:34:35.740 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:35.736+0000 7fc1727fc640 1 --2- 172.21.15.43:0/1451189023 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc160004d00 0x7fc160005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:34:35.741 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:35.736+0000 7fc1727fc640 1 -- 172.21.15.43:0/1451189023 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc1740ac440 con 0x7fc160004d00 2023-08-09T13:34:35.741 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:35.736+0000 7fc1727fc640 1 -- 172.21.15.43:0/1451189023 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc1740ac720 con 0x7fc160003ef0 2023-08-09T13:34:35.741 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:35.736+0000 7fc1727fc640 1 -- 172.21.15.43:0/1451189023 --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc16000b980 con 0x7fc1740c84d0 2023-08-09T13:34:35.741 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:35.736+0000 7f276e015640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f27681546f0 0x7f276815e640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:34:35.741 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:35.736+0000 7f276e015640 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7f276815b460 0x7f276806b3d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:34:35.741 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:35.736+0000 7f276e015640 1 -- --> v1:172.21.15.203:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f276806d770 con 0x7f2768070ab0 2023-08-09T13:34:35.742 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:35.736+0000 7f276e015640 1 -- --> v2:172.21.15.203:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f276806cf30 con 0x7f276815b460 2023-08-09T13:34:35.742 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:35.736+0000 7f276e015640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f276806d1b0 con 0x7f27681546f0 2023-08-09T13:34:35.742 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:35.736+0000 7fc173fff640 1 --2- 172.21.15.43:0/1451189023 >> v2:172.21.15.203:3300/0 conn(0x7fc1740cba50 0x7fc1740cbea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-08-09T13:34:35.742 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:35.737+0000 7fc172ffd640 1 --2- 172.21.15.43:0/1451189023 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fc160003ef0 0x7fc1600047c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:34:35.743 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:35.737+0000 7f276c812640 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7f276815b460 0x7f276806b3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:34:35.743 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:35.737+0000 7fc1737fe640 1 --2- 172.21.15.43:0/1451189023 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fc1740c84d0 0x7fc160003970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:34:35.743 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:35.737+0000 7f276c812640 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7f276815b460 0x7f276806b3d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.203:3300/0 says I am v2:172.21.15.43:34230/0 (socket says 172.21.15.43:34230) 2023-08-09T13:34:35.743 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:35.737+0000 7f276c812640 1 -- 172.21.15.43:0/620296085 learned_addr learned my addr 172.21.15.43:0/620296085 (peer_addr_for_me v2:172.21.15.43:0/0) 2023-08-09T13:34:35.744 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:35.737+0000 7fc173fff640 1 --2- 172.21.15.43:0/1451189023 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc160004d00 0x7fc160005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:34:35.744 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:35.737+0000 7f276d013640 1 --2- 172.21.15.43:0/620296085 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f27681546f0 0x7f276815e640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:34:35.745 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:35.738+0000 7fc172ffd640 1 -- 172.21.15.43:0/1451189023 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fc1740c84d0 msgr2=0x7fc160003970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-09T13:34:35.745 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:35.738+0000 7fc172ffd640 1 --2- 172.21.15.43:0/1451189023 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fc1740c84d0 0x7fc160003970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:34:35.745 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:35.738+0000 7fc172ffd640 1 -- 172.21.15.43:0/1451189023 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc160004d00 msgr2=0x7fc160005180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-09T13:34:35.745 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:35.738+0000 7fc172ffd640 1 --2- 172.21.15.43:0/1451189023 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc160004d00 0x7fc160005180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:34:35.745 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:35.738+0000 7fc172ffd640 1 -- 172.21.15.43:0/1451189023 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc1740c8e70 con 0x7fc160003ef0 2023-08-09T13:34:35.746 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:35.738+0000 7f276c812640 1 -- 172.21.15.43:0/620296085 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f27681546f0 msgr2=0x7f276815e640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-09T13:34:35.746 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:35.738+0000 7f276c812640 1 --2- 172.21.15.43:0/620296085 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f27681546f0 0x7f276815e640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:34:35.746 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:35.738+0000 7f276c812640 1 -- 172.21.15.43:0/620296085 >> v1:172.21.15.203:6789/0 conn(0x7f2768070ab0 legacy=0x7f27680c84b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:34:35.746 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:35.738+0000 7f276c812640 1 -- 172.21.15.43:0/620296085 --> v2:172.21.15.203:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f276806d490 con 0x7f276815b460 2023-08-09T13:34:35.746 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:35.739+0000 7f276c812640 1 --2- 172.21.15.43:0/620296085 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f276815b460 0x7f276806b3d0 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f2758009870 tx=0x7f275802b8f0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=f4ee17ef23160575 server_cookie=0 in_seq=0 out_seq=0 2023-08-09T13:34:35.746 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:35.740+0000 7fc172ffd640 1 --2- 172.21.15.43:0/1451189023 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fc160003ef0 0x7fc1600047c0 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7fc15c009ac0 tx=0x7fc1600057f0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=3b6efb58f8b3f2fd server_cookie=0 in_seq=0 out_seq=0 2023-08-09T13:34:35.747 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:35.740+0000 7f275ffff640 1 -- 172.21.15.43:0/620296085 <== mon.1 v2:172.21.15.203:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2758031640 con 0x7f276815b460 2023-08-09T13:34:35.747 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:35.740+0000 7f275ffff640 1 -- 172.21.15.43:0/620296085 <== mon.1 v2:172.21.15.203:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f27580317a0 con 0x7f276815b460 2023-08-09T13:34:35.747 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:35.740+0000 7f275ffff640 1 -- 172.21.15.43:0/620296085 <== mon.1 v2:172.21.15.203:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2758031b30 con 0x7f276815b460 2023-08-09T13:34:35.747 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:35.740+0000 7fc1727fc640 1 -- 172.21.15.43:0/1451189023 <== mon.1 v2:172.21.15.203:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc15c02c6b0 con 0x7fc160003ef0 2023-08-09T13:34:35.748 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:35.740+0000 7fc1727fc640 1 -- 172.21.15.43:0/1451189023 <== mon.1 v2:172.21.15.203:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc15c02c810 con 0x7fc160003ef0 2023-08-09T13:34:35.748 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:35.740+0000 7fc1727fc640 1 -- 172.21.15.43:0/1451189023 <== mon.1 v2:172.21.15.203:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc15c02cba0 con 0x7fc160003ef0 2023-08-09T13:34:35.748 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:35.741+0000 7f276e015640 1 -- 172.21.15.43:0/620296085 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f276815b460 msgr2=0x7f276806b3d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:34:35.748 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:35.741+0000 7f276e015640 1 --2- 172.21.15.43:0/620296085 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f276815b460 0x7f276806b3d0 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f2758009870 tx=0x7f275802b8f0 comp rx=0 tx=0).stop 2023-08-09T13:34:35.749 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:35.741+0000 7f276e015640 1 -- 172.21.15.43:0/620296085 shutdown_connections 2023-08-09T13:34:35.749 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:35.741+0000 7f276e015640 1 --2- 172.21.15.43:0/620296085 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f276815b460 0x7f276806b3d0 unknown :-1 s=CLOSED pgs=73 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:34:35.749 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:35.741+0000 7f276e015640 1 --2- 172.21.15.43:0/620296085 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f27681546f0 0x7f276815e640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:34:35.749 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:35.741+0000 7f276e015640 1 -- 172.21.15.43:0/620296085 >> 172.21.15.43:0/620296085 conn(0x7f27680af950 msgr2=0x7f27680afd60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-09T13:34:35.749 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:35.741+0000 7fc17a1ab640 1 -- 172.21.15.43:0/1451189023 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fc160003ef0 msgr2=0x7fc1600047c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:34:35.749 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:35.741+0000 7fc17a1ab640 1 --2- 172.21.15.43:0/1451189023 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fc160003ef0 0x7fc1600047c0 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7fc15c009ac0 tx=0x7fc1600057f0 comp rx=0 tx=0).stop 2023-08-09T13:34:35.750 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:35.741+0000 7fc173fff640 1 -- 172.21.15.43:0/1451189023 reap_dead start 2023-08-09T13:34:35.750 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:35.742+0000 7f276e015640 1 -- 172.21.15.43:0/620296085 shutdown_connections 2023-08-09T13:34:35.750 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:35.742+0000 7f276e015640 1 -- 172.21.15.43:0/620296085 wait complete. 2023-08-09T13:34:35.750 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:35.742+0000 7fc17a1ab640 1 -- 172.21.15.43:0/1451189023 shutdown_connections 2023-08-09T13:34:35.751 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:35.742+0000 7fc17a1ab640 1 -- 172.21.15.43:0/1451189023 >> 172.21.15.43:0/1451189023 conn(0x7fc1740cde80 msgr2=0x7fc1740ce290 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-09T13:34:35.751 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:35.742+0000 7fc17a1ab640 1 -- 172.21.15.43:0/1451189023 shutdown_connections 2023-08-09T13:34:35.751 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:35.742+0000 7fc17a1ab640 1 -- 172.21.15.43:0/1451189023 wait complete. 2023-08-09T13:34:35.751 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:35.744+0000 7f276e015640 1 Processor -- start 2023-08-09T13:34:35.752 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:35.745+0000 7fc17a1ab640 1 Processor -- start 2023-08-09T13:34:35.752 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:35.745+0000 7f276e015640 1 -- start start 2023-08-09T13:34:35.752 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:35.746+0000 7f276e015640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f2768070ab0 0x7f2768148f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:34:35.752 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:35.746+0000 7f276e015640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f27681546f0 0x7f27680c2850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:34:35.753 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:35.745+0000 7fc17a1ab640 1 -- start start 2023-08-09T13:34:35.753 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:35.747+0000 7fc17a1ab640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fc160003ef0 0x7fc174070b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:34:35.753 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:35.747+0000 7f276d013640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f2768070ab0 0x7f2768148f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:34:35.754 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:35.747+0000 7f276d013640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f2768070ab0 0x7f2768148f60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:51582/0 (socket says 172.21.15.43:51582) 2023-08-09T13:34:35.754 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:35.747+0000 7f276d013640 1 -- 172.21.15.43:0/904747059 learned_addr learned my addr 172.21.15.43:0/904747059 (peer_addr_for_me v2:172.21.15.43:0/0) 2023-08-09T13:34:35.754 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:35.747+0000 7fc17a1ab640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fc1740ad860 0x7fc1740b3020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:34:35.754 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:35.747+0000 7fc17a1ab640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc1740b3560 0x7fc1740b0aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:34:35.754 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:35.747+0000 7f276e015640 1 --2- 172.21.15.43:0/904747059 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f2768149a30 0x7f27680c4260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:34:35.755 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:35.747+0000 7fc17a1ab640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc1740c4f50 con 0x7fc1740b3560 2023-08-09T13:34:35.755 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:35.747+0000 7fc17a1ab640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc1740c5160 con 0x7fc1740ad860 2023-08-09T13:34:35.755 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:35.747+0000 7fc17a1ab640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc1740c5440 con 0x7fc160003ef0 2023-08-09T13:34:35.755 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:35.747+0000 7fc173fff640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc1740b3560 0x7fc1740b0aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:34:35.756 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:35.747+0000 7fc173fff640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc1740b3560 0x7fc1740b0aa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:51588/0 (socket says 172.21.15.43:51588) 2023-08-09T13:34:35.756 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:35.747+0000 7fc173fff640 1 -- 172.21.15.43:0/49380364 learned_addr learned my addr 172.21.15.43:0/49380364 (peer_addr_for_me v2:172.21.15.43:0/0) 2023-08-09T13:34:35.756 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:35.748+0000 7f276c812640 1 --2- 172.21.15.43:0/904747059 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f27681546f0 0x7f27680c2850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:34:35.756 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:35.748+0000 7f7d190ad640 1 Processor -- start 2023-08-09T13:34:35.757 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:35.748+0000 7fc172ffd640 1 --2- 172.21.15.43:0/49380364 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fc1740ad860 0x7fc1740b3020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:34:35.757 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:35.748+0000 7fc1737fe640 1 --2- 172.21.15.43:0/49380364 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fc160003ef0 0x7fc174070b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:34:35.757 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:35.748+0000 7fc173fff640 1 -- 172.21.15.43:0/49380364 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fc160003ef0 msgr2=0x7fc174070b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:34:35.758 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:35.748+0000 7fc173fff640 1 --2- 172.21.15.43:0/49380364 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fc160003ef0 0x7fc174070b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:34:35.758 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:35.748+0000 7fc173fff640 1 -- 172.21.15.43:0/49380364 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fc1740ad860 msgr2=0x7fc1740b3020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:34:35.758 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:35.748+0000 7fc173fff640 1 --2- 172.21.15.43:0/49380364 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fc1740ad860 0x7fc1740b3020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:34:35.758 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:35.748+0000 7fc173fff640 1 -- 172.21.15.43:0/49380364 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc1740c5720 con 0x7fc1740b3560 2023-08-09T13:34:35.758 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:35.748+0000 7f7d190ad640 1 -- start start 2023-08-09T13:34:35.759 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:35.749+0000 7fc173fff640 1 --2- 172.21.15.43:0/49380364 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc1740b3560 0x7fc1740b0aa0 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7fc17406ec60 tx=0x7fc16c00f580 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-09T13:34:35.759 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:35.749+0000 7f276e015640 1 -- 172.21.15.43:0/904747059 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f27680c3e40 con 0x7f2768070ab0 2023-08-09T13:34:35.759 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:35.749+0000 7f276e015640 1 -- 172.21.15.43:0/904747059 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f27681494a0 con 0x7f2768149a30 2023-08-09T13:34:35.759 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:35.750+0000 7f276e015640 1 -- 172.21.15.43:0/904747059 --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f27680c31c0 con 0x7f27681546f0 2023-08-09T13:34:35.760 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:35.750+0000 7f7d190ad640 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7f7d140c9810 0x7f7d140cca60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:34:35.760 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:35.750+0000 7f7d190ad640 1 -- --> v1:172.21.15.43:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7d140cdaa0 con 0x7f7d140b0490 2023-08-09T13:34:35.760 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:35.750+0000 7f7d190ad640 1 -- --> v1:172.21.15.203:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7d1406fec0 con 0x7f7d1406f690 2023-08-09T13:34:35.760 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:35.750+0000 7f7d190ad640 1 -- --> v2:172.21.15.203:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7d140701a0 con 0x7f7d140c9810 2023-08-09T13:34:35.760 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:35.751+0000 7f7d12d9d640 1 --1- >> v1:172.21.15.43:6789/0 conn(0x7f7d140b0490 0x7f7d14159260 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.43:6789/0 says I am v1:172.21.15.43:35592/0 (socket says 172.21.15.43:35592) 2023-08-09T13:34:35.761 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:35.751+0000 7f7d12d9d640 1 -- 172.21.15.43:0/1189678328 learned_addr learned my addr 172.21.15.43:0/1189678328 (peer_addr_for_me v1:172.21.15.43:0/0) 2023-08-09T13:34:35.761 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:35.751+0000 7f7d1359e640 1 --2- 172.21.15.43:0/1189678328 >> v2:172.21.15.203:3300/0 conn(0x7f7d140c9810 0x7f7d140cca60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:34:35.761 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:35.750+0000 7fc16bfff640 1 -- 172.21.15.43:0/49380364 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc16c00e040 con 0x7fc1740b3560 2023-08-09T13:34:35.761 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:35.750+0000 7fc16bfff640 1 -- 172.21.15.43:0/49380364 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc16c00f040 con 0x7fc1740b3560 2023-08-09T13:34:35.762 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:35.750+0000 7fc16bfff640 1 -- 172.21.15.43:0/49380364 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc16c015670 con 0x7fc1740b3560 2023-08-09T13:34:35.762 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:35.750+0000 7fc17a1ab640 1 -- 172.21.15.43:0/49380364 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc1740c5a00 con 0x7fc1740b3560 2023-08-09T13:34:35.762 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:35.750+0000 7fc17a1ab640 1 -- 172.21.15.43:0/49380364 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc174071040 con 0x7fc1740b3560 2023-08-09T13:34:35.762 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:35.751+0000 7f276d814640 1 --2- 172.21.15.43:0/904747059 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f2768149a30 0x7f27680c4260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:34:35.762 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:35.751+0000 7f7d11d9b640 1 -- 172.21.15.43:0/1189678328 <== mon.0 v1:172.21.15.43:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2536736361 0 0) 0x7f7d140cdaa0 con 0x7f7d140b0490 2023-08-09T13:34:35.763 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:35.751+0000 7f7d11d9b640 1 -- 172.21.15.43:0/1189678328 --> v1:172.21.15.43:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f7cfc003520 con 0x7f7d140b0490 2023-08-09T13:34:35.763 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:35.752+0000 7f7b54999640 1 Processor -- start 2023-08-09T13:34:35.763 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:35.752+0000 7fc17a1ab640 1 -- 172.21.15.43:0/49380364 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7fc158000fc0 con 0x7fc1740b3560 2023-08-09T13:34:35.764 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:35.752+0000 7f276d814640 1 -- 172.21.15.43:0/904747059 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f27681546f0 msgr2=0x7f27680c2850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:34:35.764 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:35.752+0000 7f276d814640 1 --2- 172.21.15.43:0/904747059 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f27681546f0 0x7f27680c2850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:34:35.764 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:35.752+0000 7f276d814640 1 -- 172.21.15.43:0/904747059 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f2768070ab0 msgr2=0x7f2768148f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:34:35.764 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:35.752+0000 7f276d814640 1 --2- 172.21.15.43:0/904747059 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f2768070ab0 0x7f2768148f60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:34:35.764 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:35.752+0000 7f276d814640 1 -- 172.21.15.43:0/904747059 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f27580094d0 con 0x7f2768149a30 2023-08-09T13:34:35.765 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:35.752+0000 7f7b54999640 1 -- start start 2023-08-09T13:34:35.765 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:35.753+0000 7fb8e15fe640 1 Processor -- start 2023-08-09T13:34:35.765 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:35.753+0000 7f276d814640 1 --2- 172.21.15.43:0/904747059 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f2768149a30 0x7f27680c4260 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f276400c390 tx=0x7f276400c850 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-09T13:34:35.766 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:35.753+0000 7f275dffb640 1 -- 172.21.15.43:0/904747059 <== mon.1 v2:172.21.15.203:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f27640052d0 con 0x7f2768149a30 2023-08-09T13:34:35.766 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:35.754+0000 7f276e015640 1 -- 172.21.15.43:0/904747059 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f27680c34a0 con 0x7f2768149a30 2023-08-09T13:34:35.766 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:35.754+0000 7f276e015640 1 -- 172.21.15.43:0/904747059 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f27680c3a60 con 0x7f2768149a30 2023-08-09T13:34:35.766 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:35.754+0000 7f275dffb640 1 -- 172.21.15.43:0/904747059 <== mon.1 v2:172.21.15.203:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2764005430 con 0x7f2768149a30 2023-08-09T13:34:35.766 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:35.754+0000 7f275dffb640 1 -- 172.21.15.43:0/904747059 <== mon.1 v2:172.21.15.203:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2764003810 con 0x7f2768149a30 2023-08-09T13:34:35.766 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:35.753+0000 7fb8e15fe640 1 -- start start 2023-08-09T13:34:35.767 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:35.752+0000 7f7d1359e640 1 -- 172.21.15.43:0/1189678328 >> v1:172.21.15.203:6789/0 conn(0x7f7d1406f690 legacy=0x7f7d140c9f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:34:35.767 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:35.752+0000 7f7d1359e640 1 -- 172.21.15.43:0/1189678328 >> v1:172.21.15.43:6789/0 conn(0x7f7d140b0490 legacy=0x7f7d14159260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:34:35.767 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:35.752+0000 7f7d1359e640 1 -- 172.21.15.43:0/1189678328 --> v2:172.21.15.203:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7d14070480 con 0x7f7d140c9810 2023-08-09T13:34:35.767 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:35.754+0000 7f7b54999640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f7b480eac80 0x7f7b48047bb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:34:35.768 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:35.754+0000 7f7b54999640 1 -- --> v1:172.21.15.43:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7b48037b50 con 0x7f7b48006a50 2023-08-09T13:34:35.768 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:35.754+0000 7f7b54999640 1 -- --> v1:172.21.15.203:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7b4804f710 con 0x7f7b48005ec0 2023-08-09T13:34:35.768 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:35.754+0000 7f7b54999640 1 -- --> v2:172.21.15.43:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7b4804f150 con 0x7f7b480eac80 2023-08-09T13:34:35.768 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:35.755+0000 7f7b4ed9d640 1 --1- >> v1:172.21.15.43:6789/0 conn(0x7f7b48006a50 0x7f7b48052f40 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.43:6789/0 says I am v1:172.21.15.43:35600/0 (socket says 172.21.15.43:35600) 2023-08-09T13:34:35.768 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:35.755+0000 7f7b4ed9d640 1 -- 172.21.15.43:0/206689942 learned_addr learned my addr 172.21.15.43:0/206689942 (peer_addr_for_me v1:172.21.15.43:0/0) 2023-08-09T13:34:35.769 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:35.755+0000 7f276e015640 1 -- 172.21.15.43:0/904747059 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f2748000fc0 con 0x7f2768149a30 2023-08-09T13:34:35.769 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:35.755+0000 7f7b4dd9b640 1 --1- >> v1:172.21.15.203:6789/0 conn(0x7f7b48005ec0 0x7f7b48006340 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.203:6789/0 says I am v1:172.21.15.43:33736/0 (socket says 172.21.15.43:33736) 2023-08-09T13:34:35.770 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:35.756+0000 7f7b4e59c640 1 --2- 172.21.15.43:0/206689942 >> v2:172.21.15.43:3300/0 conn(0x7f7b480eac80 0x7f7b48047bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:34:35.770 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:35.756+0000 7f7b4d59a640 1 -- 172.21.15.43:0/206689942 <== mon.1 v1:172.21.15.203:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 188833156 0 0) 0x7f7b4804f710 con 0x7f7b48005ec0 2023-08-09T13:34:35.770 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:35.756+0000 7f7b4d59a640 1 -- 172.21.15.43:0/206689942 --> v1:172.21.15.203:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f7b38003520 con 0x7f7b48005ec0 2023-08-09T13:34:35.770 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:35.757+0000 7f7b4d59a640 1 -- 172.21.15.43:0/206689942 <== mon.0 v1:172.21.15.43:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3072158192 0 0) 0x7f7b48037b50 con 0x7f7b48006a50 2023-08-09T13:34:35.770 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:35.757+0000 7f7b4d59a640 1 -- 172.21.15.43:0/206689942 --> v1:172.21.15.43:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f7b4804f710 con 0x7f7b48006a50 2023-08-09T13:34:35.770 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:35.757+0000 7f7b4d59a640 1 -- 172.21.15.43:0/206689942 <== mon.1 v1:172.21.15.203:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2774761850 0 0) 0x7f7b38003520 con 0x7f7b48005ec0 2023-08-09T13:34:35.770 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:35.757+0000 7f7b4d59a640 1 -- 172.21.15.43:0/206689942 >> v2:172.21.15.43:3300/0 conn(0x7f7b480eac80 msgr2=0x7f7b48047bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-09T13:34:35.771 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:35.757+0000 7f7b4d59a640 1 --2- 172.21.15.43:0/206689942 >> v2:172.21.15.43:3300/0 conn(0x7f7b480eac80 0x7f7b48047bb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:34:35.771 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:35.757+0000 7f7b4d59a640 1 -- 172.21.15.43:0/206689942 >> v1:172.21.15.43:6789/0 conn(0x7f7b48006a50 legacy=0x7f7b48052f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:34:35.771 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:35.757+0000 7f7b4d59a640 1 -- 172.21.15.43:0/206689942 --> v1:172.21.15.203:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7b4804f430 con 0x7f7b48005ec0 2023-08-09T13:34:35.771 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:35.757+0000 7f7b4d59a640 1 -- 172.21.15.43:0/206689942 <== mon.1 v1:172.21.15.203:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1285150740 0 0) 0x7f7b44001670 con 0x7f7b48005ec0 2023-08-09T13:34:35.771 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:35.757+0000 7f7b4d59a640 1 -- 172.21.15.43:0/206689942 >> v1:172.21.15.203:6789/0 conn(0x7f7b48005ec0 legacy=0x7f7b48006340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:34:35.771 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:35.757+0000 7f7b4d59a640 1 --2- 172.21.15.43:0/206689942 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f7b38003b00 0x7f7b38003ee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:34:35.771 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:35.757+0000 7f7b4d59a640 1 --2- 172.21.15.43:0/206689942 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f7b380047c0 0x7f7b38004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:34:35.771 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:35.757+0000 7f7b4d59a640 1 --2- 172.21.15.43:0/206689942 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f7b38005180 0x7f7b3800b980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:34:35.772 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:35.757+0000 7f7b4d59a640 1 -- 172.21.15.43:0/206689942 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7b500613d0 con 0x7f7b380047c0 2023-08-09T13:34:35.772 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:35.757+0000 7f7b4d59a640 1 -- 172.21.15.43:0/206689942 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7b4804f150 con 0x7f7b38005180 2023-08-09T13:34:35.772 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:35.758+0000 7f7b4d59a640 1 -- 172.21.15.43:0/206689942 --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7b3800bec0 con 0x7f7b38003b00 2023-08-09T13:34:35.772 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:35.758+0000 7f7b4e59c640 1 --2- 172.21.15.43:0/206689942 >> v2:172.21.15.43:3300/0 conn(0x7f7b480eac80 0x7f7b48047bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-08-09T13:34:35.772 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:35.758+0000 7f7b4ed9d640 1 --2- 172.21.15.43:0/206689942 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f7b38005180 0x7f7b3800b980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:34:35.772 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:35.759+0000 7f7b4dd9b640 1 --2- 172.21.15.43:0/206689942 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f7b380047c0 0x7f7b38004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:34:35.773 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:35.757+0000 7fc15e5c8640 1 Processor -- start 2023-08-09T13:34:35.773 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:35.757+0000 7fc15e5c8640 1 -- start start 2023-08-09T13:34:35.773 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:35.759+0000 7f275dffb640 1 -- 172.21.15.43:0/904747059 <== mon.1 v2:172.21.15.203:3300/0 4 ==== mgrmap(e 5) v1 ==== 88341+0+0 (secure 0 0 0) 0x7f2764003970 con 0x7f2768149a30 2023-08-09T13:34:35.774 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:35.759+0000 7f7b4e59c640 1 --2- 172.21.15.43:0/206689942 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f7b38003b00 0x7f7b38003ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:34:35.774 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:35.759+0000 7f7b4ed9d640 1 -- 172.21.15.43:0/206689942 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f7b38003b00 msgr2=0x7f7b38003ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-09T13:34:35.774 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:35.759+0000 7f7b4ed9d640 1 --2- 172.21.15.43:0/206689942 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f7b38003b00 0x7f7b38003ee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:34:35.774 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:35.759+0000 7f7b4ed9d640 1 -- 172.21.15.43:0/206689942 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f7b380047c0 msgr2=0x7f7b38004c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-09T13:34:35.774 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:35.757+0000 7fc16bfff640 1 -- 172.21.15.43:0/49380364 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 5) v1 ==== 88341+0+0 (secure 0 0 0) 0x7fc16c0157d0 con 0x7fc1740b3560 2023-08-09T13:34:35.775 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:35.759+0000 7fc16bfff640 1 --2- 172.21.15.43:0/49380364 >> [v2:172.21.15.203:6832/830097474,v1:172.21.15.203:6833/830097474] conn(0x7fc15406a120 0x7fc15406c5e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:34:35.775 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:35.759+0000 7fc16bfff640 1 -- 172.21.15.43:0/49380364 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6813+0+0 (secure 0 0 0) 0x7fc16c089ef0 con 0x7fc1740b3560 2023-08-09T13:34:35.775 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:35.759+0000 7fc16bfff640 1 --2- 172.21.15.43:0/49380364 >> [v2:172.21.15.43:6800/502667335,v1:172.21.15.43:6801/502667335] conn(0x7fc154073e00 0x7fc154076240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:34:35.775 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:35.759+0000 7fc16bfff640 1 -- 172.21.15.43:0/49380364 --> [v2:172.21.15.43:6800/502667335,v1:172.21.15.43:6801/502667335] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7fc154076910 con 0x7fc154073e00 2023-08-09T13:34:35.775 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:35.759+0000 7fc16bfff640 1 -- 172.21.15.43:0/49380364 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_get_version_reply(handle=1 version=24) v2 ==== 24+0+0 (secure 0 0 0) 0x7fc16c08a2a0 con 0x7fc1740b3560 2023-08-09T13:34:35.775 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:35.759+0000 7f7b4ed9d640 1 --2- 172.21.15.43:0/206689942 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f7b380047c0 0x7f7b38004c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:34:35.776 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:35.759+0000 7f7b4ed9d640 1 -- 172.21.15.43:0/206689942 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7b4804f430 con 0x7f7b38005180 2023-08-09T13:34:35.776 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:35.760+0000 7f7b4dd9b640 1 --2- 172.21.15.43:0/206689942 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f7b380047c0 0x7f7b38004c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-08-09T13:34:35.776 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:35.760+0000 7fc15e5c8640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7fc1580cc0c0 0x7fc1580cc550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:34:35.777 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:35.755+0000 7fb8e15fe640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7fb8d40eac80 0x7fb8d4047bb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:34:35.777 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:35.755+0000 7fb8e15fe640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fb8d4005ec0 0x7fb8d4006340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:34:35.777 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:35.755+0000 7fb8e15fe640 1 -- --> v1:172.21.15.43:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb8d4037b50 con 0x7fb8d4006880 2023-08-09T13:34:35.777 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:35.755+0000 7fb8e15fe640 1 -- --> v2:172.21.15.43:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb8d4006d10 con 0x7fb8d40eac80 2023-08-09T13:34:35.777 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:35.755+0000 7fb8e15fe640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb8d404f710 con 0x7fb8d4005ec0 2023-08-09T13:34:35.777 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:35.756+0000 7fb8dbfff640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7fb8d40eac80 0x7fb8d4047bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:34:35.778 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:35.756+0000 7fb8dbfff640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7fb8d40eac80 0x7fb8d4047bb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:51600/0 (socket says 172.21.15.43:51600) 2023-08-09T13:34:35.778 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:35.756+0000 7fb8dbfff640 1 -- 172.21.15.43:0/4044232821 learned_addr learned my addr 172.21.15.43:0/4044232821 (peer_addr_for_me v2:172.21.15.43:0/0) 2023-08-09T13:34:35.778 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:35.756+0000 7fb8dbfff640 1 -- 172.21.15.43:0/4044232821 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fb8d4005ec0 msgr2=0x7fb8d4006340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-09T13:34:35.778 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:35.756+0000 7fb8dbfff640 1 --2- 172.21.15.43:0/4044232821 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fb8d4005ec0 0x7fb8d4006340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:34:35.778 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:35.756+0000 7fb8dbfff640 1 -- 172.21.15.43:0/4044232821 >> v1:172.21.15.43:6789/0 conn(0x7fb8d4006880 legacy=0x7fb8d40eb130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-09T13:34:35.778 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:35.756+0000 7fb8dbfff640 1 -- 172.21.15.43:0/4044232821 --> v2:172.21.15.43:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb8d404f150 con 0x7fb8d40eac80 2023-08-09T13:34:35.778 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:35.759+0000 7fb8dbfff640 1 --2- 172.21.15.43:0/4044232821 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb8d40eac80 0x7fb8d4047bb0 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7fb8d0009870 tx=0x7fb8d002b9a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c8948e6c7fd60f19 server_cookie=0 in_seq=0 out_seq=0 2023-08-09T13:34:35.778 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:35.759+0000 7fb8daffd640 1 -- 172.21.15.43:0/4044232821 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb8d0030860 con 0x7fb8d40eac80 2023-08-09T13:34:35.779 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:35.759+0000 7fb8daffd640 1 -- 172.21.15.43:0/4044232821 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb8d00309c0 con 0x7fb8d40eac80 2023-08-09T13:34:35.779 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:35.759+0000 7fb8daffd640 1 -- 172.21.15.43:0/4044232821 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb8d0030d90 con 0x7fb8d40eac80 2023-08-09T13:34:35.779 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:35.760+0000 7fc1737fe640 1 --2- 172.21.15.43:0/49380364 >> [v2:172.21.15.203:6832/830097474,v1:172.21.15.203:6833/830097474] conn(0x7fc15406a120 0x7fc15406c5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:34:35.779 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:35.760+0000 7fc172ffd640 1 --2- 172.21.15.43:0/49380364 >> [v2:172.21.15.43:6800/502667335,v1:172.21.15.43:6801/502667335] conn(0x7fc154073e00 0x7fc154076240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:34:35.780 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:35.760+0000 7fc15e5c8640 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7fc1580ca710 0x7fc1580cabc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:34:35.780 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:35.760+0000 7fc15e5c8640 1 -- --> v1:172.21.15.203:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc15806f870 con 0x7fc15806b790 2023-08-09T13:34:35.780 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:35.760+0000 7f7b4ed9d640 1 -- 172.21.15.43:0/206689942 reap_dead start 2023-08-09T13:34:35.780 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:35.760+0000 7fc15e5c8640 1 -- --> v2:172.21.15.43:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc15806c5d0 con 0x7fc1580cc0c0 2023-08-09T13:34:35.781 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:35.760+0000 7fc15e5c8640 1 -- --> v2:172.21.15.203:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc15806f540 con 0x7fc1580ca710 2023-08-09T13:34:35.781 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:35.760+0000 7fb8e15fe640 1 -- 172.21.15.43:0/4044232821 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb8d40eac80 msgr2=0x7fb8d4047bb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:34:35.781 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:35.760+0000 7fb8e15fe640 1 --2- 172.21.15.43:0/4044232821 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb8d40eac80 0x7fb8d4047bb0 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7fb8d0009870 tx=0x7fb8d002b9a0 comp rx=0 tx=0).stop 2023-08-09T13:34:35.781 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:35.761+0000 7fb8e15fe640 1 -- 172.21.15.43:0/4044232821 shutdown_connections 2023-08-09T13:34:35.782 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:35.761+0000 7fb8e15fe640 1 --2- 172.21.15.43:0/4044232821 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fb8d4005ec0 0x7fb8d4006340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:34:35.782 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:35.761+0000 7fb8e15fe640 1 --2- 172.21.15.43:0/4044232821 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb8d40eac80 0x7fb8d4047bb0 unknown :-1 s=CLOSED pgs=161 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:34:35.782 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:35.761+0000 7fb8e15fe640 1 -- 172.21.15.43:0/4044232821 >> 172.21.15.43:0/4044232821 conn(0x7fb8d4009fe0 msgr2=0x7fb8d400a3c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-09T13:34:35.782 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:35.761+0000 7f7d1359e640 1 --2- 172.21.15.43:0/1189678328 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f7d140c9810 0x7f7d140cca60 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f7d0c009d90 tx=0x7f7d0c02bf60 comp rx=0 tx=0).ready entity=mon.1 client_cookie=2c1f2199af31f36d server_cookie=0 in_seq=0 out_seq=0 2023-08-09T13:34:35.782 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:35.761+0000 7f275dffb640 1 --2- 172.21.15.43:0/904747059 >> [v2:172.21.15.203:6832/830097474,v1:172.21.15.203:6833/830097474] conn(0x7f274c06a050 0x7f274c06c510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:34:35.783 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:35.761+0000 7f7b4ed9d640 1 --2- 172.21.15.43:0/206689942 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f7b38005180 0x7f7b3800b980 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f7b5002c700 tx=0x7f7b380057f0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=3a10f42b75d995ef server_cookie=0 in_seq=0 out_seq=0 2023-08-09T13:34:35.783 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:35.761+0000 7fc1577fe640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7fc1580cc0c0 0x7fc1580cc550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:34:35.783 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:35.761+0000 7fc15cb65640 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7fc1580ca710 0x7fc1580cabc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:34:35.784 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:35.761+0000 7fc15cb65640 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7fc1580ca710 0x7fc1580cabc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.203:3300/0 says I am v2:172.21.15.43:34270/0 (socket says 172.21.15.43:34270) 2023-08-09T13:34:35.784 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:35.761+0000 7fc15cb65640 1 -- 172.21.15.43:0/675462142 learned_addr learned my addr 172.21.15.43:0/675462142 (peer_addr_for_me v2:172.21.15.43:0/0) 2023-08-09T13:34:35.784 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:35.761+0000 7fb8e15fe640 1 -- 172.21.15.43:0/4044232821 shutdown_connections 2023-08-09T13:34:35.784 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:35.761+0000 7fb8e15fe640 1 -- 172.21.15.43:0/4044232821 wait complete. 2023-08-09T13:34:35.784 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:35.761+0000 7f275dffb640 1 -- 172.21.15.43:0/904747059 <== mon.1 v2:172.21.15.203:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6813+0+0 (secure 0 0 0) 0x7f2764088170 con 0x7f2768149a30 2023-08-09T13:34:35.785 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:35.762+0000 7fc172ffd640 1 --2- 172.21.15.43:0/49380364 >> [v2:172.21.15.43:6800/502667335,v1:172.21.15.43:6801/502667335] conn(0x7fc154073e00 0x7fc154076240 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-09T13:34:35.785 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:35.762+0000 7f7d11d9b640 1 -- 172.21.15.43:0/1189678328 <== mon.1 v2:172.21.15.203:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7d0c02cd10 con 0x7f7d140c9810 2023-08-09T13:34:35.785 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:35.762+0000 7f7d11d9b640 1 -- 172.21.15.43:0/1189678328 <== mon.1 v2:172.21.15.203:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7d0c02ce70 con 0x7f7d140c9810 2023-08-09T13:34:35.785 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:35.762+0000 7f7d11d9b640 1 -- 172.21.15.43:0/1189678328 <== mon.1 v2:172.21.15.203:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7d0c034640 con 0x7f7d140c9810 2023-08-09T13:34:35.786 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:35.762+0000 7f7b4d59a640 1 -- 172.21.15.43:0/206689942 <== mon.1 v2:172.21.15.203:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7b5006a940 con 0x7f7b38005180 2023-08-09T13:34:35.786 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:35.762+0000 7f7b4d59a640 1 -- 172.21.15.43:0/206689942 <== mon.1 v2:172.21.15.203:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7b5006aae0 con 0x7f7b38005180 2023-08-09T13:34:35.786 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:35.762+0000 7f7b4d59a640 1 -- 172.21.15.43:0/206689942 <== mon.1 v2:172.21.15.203:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7b5006ae90 con 0x7f7b38005180 2023-08-09T13:34:35.786 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:35.762+0000 7f275dffb640 1 --2- 172.21.15.43:0/904747059 >> [v2:172.21.15.43:6808/3289134610,v1:172.21.15.43:6809/3289134610] conn(0x7f274c073d30 0x7f274c076170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:34:35.787 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:35.762+0000 7f275dffb640 1 -- 172.21.15.43:0/904747059 --> [v2:172.21.15.43:6808/3289134610,v1:172.21.15.43:6809/3289134610] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f274c076840 con 0x7f274c073d30 2023-08-09T13:34:35.787 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:35.762+0000 7f275dffb640 1 -- 172.21.15.43:0/904747059 <== mon.1 v2:172.21.15.203:3300/0 6 ==== mon_get_version_reply(handle=1 version=24) v2 ==== 24+0+0 (secure 0 0 0) 0x7f2764088520 con 0x7f2768149a30 2023-08-09T13:34:35.787 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:35.762+0000 7fc1737fe640 1 --2- 172.21.15.43:0/49380364 >> [v2:172.21.15.203:6832/830097474,v1:172.21.15.203:6833/830097474] conn(0x7fc15406a120 0x7fc15406c5e0 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7fc164001b80 tx=0x7fc164001af0 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-09T13:34:35.787 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:35.762+0000 7f276c812640 1 --2- 172.21.15.43:0/904747059 >> [v2:172.21.15.43:6808/3289134610,v1:172.21.15.43:6809/3289134610] conn(0x7f274c073d30 0x7f274c076170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:34:35.788 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:35.763+0000 7f7d190ad640 1 -- 172.21.15.43:0/1189678328 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f7d140c9810 msgr2=0x7f7d140cca60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:34:35.788 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:35.763+0000 7f7d190ad640 1 --2- 172.21.15.43:0/1189678328 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f7d140c9810 0x7f7d140cca60 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f7d0c009d90 tx=0x7f7d0c02bf60 comp rx=0 tx=0).stop 2023-08-09T13:34:35.788 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:35.763+0000 7f40fc477640 1 Processor -- start 2023-08-09T13:34:35.789 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:35.763+0000 7f7b54999640 1 -- 172.21.15.43:0/206689942 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f7b38005180 msgr2=0x7f7b3800b980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:34:35.789 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:35.763+0000 7f7b54999640 1 --2- 172.21.15.43:0/206689942 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f7b38005180 0x7f7b3800b980 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f7b5002c700 tx=0x7f7b380057f0 comp rx=0 tx=0).stop 2023-08-09T13:34:35.789 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:35.763+0000 7f7b54999640 1 -- 172.21.15.43:0/206689942 shutdown_connections 2023-08-09T13:34:35.789 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:35.763+0000 7f7b54999640 1 --2- 172.21.15.43:0/206689942 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f7b38005180 0x7f7b3800b980 unknown :-1 s=CLOSED pgs=79 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:34:35.789 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:35.763+0000 7f7b54999640 1 -- 172.21.15.43:0/206689942 >> 172.21.15.43:0/206689942 conn(0x7f7b48009fe0 msgr2=0x7f7b4800a3c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-09T13:34:35.790 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:35.763+0000 7f7d190ad640 1 -- 172.21.15.43:0/1189678328 shutdown_connections 2023-08-09T13:34:35.790 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:35.763+0000 7f7d190ad640 1 --2- 172.21.15.43:0/1189678328 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f7d140c9810 0x7f7d140cca60 unknown :-1 s=CLOSED pgs=78 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:34:35.790 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:35.763+0000 7f7d190ad640 1 -- 172.21.15.43:0/1189678328 >> 172.21.15.43:0/1189678328 conn(0x7f7d140af970 msgr2=0x7f7d140afd80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-09T13:34:35.790 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:35.763+0000 7fc156ffd640 1 -- 172.21.15.43:0/675462142 <== mon.1 v1:172.21.15.203:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 131142737 0 0) 0x7fc15806f870 con 0x7fc15806b790 2023-08-09T13:34:35.790 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:35.763+0000 7f40fc477640 1 -- start start 2023-08-09T13:34:35.791 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:35.763+0000 7f276c812640 1 --2- 172.21.15.43:0/904747059 >> [v2:172.21.15.43:6808/3289134610,v1:172.21.15.43:6809/3289134610] conn(0x7f274c073d30 0x7f274c076170 crc :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-09T13:34:35.791 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:35.763+0000 7f276d013640 1 --2- 172.21.15.43:0/904747059 >> [v2:172.21.15.203:6832/830097474,v1:172.21.15.203:6833/830097474] conn(0x7f274c06a050 0x7f274c06c510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:34:35.791 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:35.763+0000 7f7d190ad640 1 -- 172.21.15.43:0/1189678328 shutdown_connections 2023-08-09T13:34:35.792 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:35.763+0000 7fc156ffd640 1 -- 172.21.15.43:0/675462142 --> v1:172.21.15.203:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fc140003520 con 0x7fc15806b790 2023-08-09T13:34:35.792 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:35.764+0000 7fc15cb65640 1 -- 172.21.15.43:0/675462142 >> v2:172.21.15.43:3300/0 conn(0x7fc1580cc0c0 msgr2=0x7fc1580cc550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-09T13:34:35.792 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:35.764+0000 7f7d190ad640 1 -- 172.21.15.43:0/1189678328 wait complete. 2023-08-09T13:34:35.792 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:35.764+0000 7f7b54999640 1 -- 172.21.15.43:0/206689942 shutdown_connections 2023-08-09T13:34:35.793 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:35.764+0000 7fc16bfff640 1 -- 172.21.15.43:0/49380364 <== osd.0 v2:172.21.15.43:6800/502667335 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+25607 (crc 0 0 0) 0x7fc154076910 con 0x7fc154073e00 2023-08-09T13:34:35.793 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:35.764+0000 7fb8e15fe640 1 Processor -- start 2023-08-09T13:34:35.793 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:35.765+0000 7fb8e15fe640 1 -- start start 2023-08-09T13:34:35.794 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:35.764+0000 7fc15cb65640 1 --2- 172.21.15.43:0/675462142 >> v2:172.21.15.43:3300/0 conn(0x7fc1580cc0c0 0x7fc1580cc550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:34:35.794 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:35.764+0000 7fc15cb65640 1 -- 172.21.15.43:0/675462142 >> v1:172.21.15.203:6789/0 conn(0x7fc15806b790 legacy=0x7fc15806bb70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:34:35.794 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:35.764+0000 7fc15cb65640 1 -- 172.21.15.43:0/675462142 --> v2:172.21.15.203:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc1580c3b40 con 0x7fc1580ca710 2023-08-09T13:34:35.794 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:35.764+0000 7fc1577fe640 1 --2- 172.21.15.43:0/675462142 >> v2:172.21.15.43:3300/0 conn(0x7fc1580cc0c0 0x7fc1580cc550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-08-09T13:34:35.795 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:35.765+0000 7f40fc477640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f40ec0dffd0 0x7f40ec0399a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:34:35.795 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:35.765+0000 7f40fc477640 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7f40ec050f30 0x7f40ec052810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:34:35.795 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:35.765+0000 7f40fc477640 1 -- --> v1:172.21.15.203:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f40ec04fea0 con 0x7f40ec050330 2023-08-09T13:34:35.795 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:35.765+0000 7f40fc477640 1 -- --> v2:172.21.15.203:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f40ec0e03b0 con 0x7f40ec050f30 2023-08-09T13:34:35.795 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:35.765+0000 7f40fc477640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f40ec04f290 con 0x7f40ec0dffd0 2023-08-09T13:34:35.795 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:35.765+0000 7f275dffb640 1 -- 172.21.15.43:0/904747059 <== osd.1 v2:172.21.15.43:6808/3289134610 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+25607 (crc 0 0 0) 0x7f274c076840 con 0x7f274c073d30 2023-08-09T13:34:35.796 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:35.765+0000 7f276d013640 1 --2- 172.21.15.43:0/904747059 >> [v2:172.21.15.203:6832/830097474,v1:172.21.15.203:6833/830097474] conn(0x7f274c06a050 0x7f274c06c510 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7f2760009a40 tx=0x7f276000b040 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-09T13:34:35.796 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:35.765+0000 7f7b54999640 1 -- 172.21.15.43:0/206689942 wait complete. 2023-08-09T13:34:35.796 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:35.766+0000 7f40f9a12640 1 --1- >> v1:172.21.15.203:6789/0 conn(0x7f40ec050330 0x7f40ec050820 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.203:6789/0 says I am v1:172.21.15.43:33752/0 (socket says 172.21.15.43:33752) 2023-08-09T13:34:35.797 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:35.766+0000 7f40f9a12640 1 -- 172.21.15.43:0/4201619068 learned_addr learned my addr 172.21.15.43:0/4201619068 (peer_addr_for_me v1:172.21.15.43:0/0) 2023-08-09T13:34:35.797 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:35.766+0000 7f40faa14640 1 --2- 172.21.15.43:0/4201619068 >> v2:172.21.15.203:3300/0 conn(0x7f40ec050f30 0x7f40ec052810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:34:35.797 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:35.766+0000 7fc15cb65640 1 --2- 172.21.15.43:0/675462142 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fc1580ca710 0x7fc1580cabc0 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7fc1580b0b40 tx=0x7fc14c02de90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=9c1a64c9a136ec0f server_cookie=0 in_seq=0 out_seq=0 2023-08-09T13:34:35.797 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:35.767+0000 7f7d190ad640 1 Processor -- start 2023-08-09T13:34:35.798 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:35.767+0000 7fb8e15fe640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fb8d4005ec0 0x7fb8d40dd190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:34:35.798 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:35.767+0000 7f40fa213640 1 --2- 172.21.15.43:0/4201619068 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f40ec0dffd0 0x7f40ec0399a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:34:35.798 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:35.767+0000 7fb8e15fe640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb8d4006880 0x7fb8d404a320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:34:35.799 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:35.767+0000 7f40faa14640 1 -- 172.21.15.43:0/4201619068 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f40ec0dffd0 msgr2=0x7f40ec0399a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-09T13:34:35.799 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:35.768+0000 7f40faa14640 1 --2- 172.21.15.43:0/4201619068 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f40ec0dffd0 0x7f40ec0399a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:34:35.799 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:35.768+0000 7f40faa14640 1 -- 172.21.15.43:0/4201619068 >> v1:172.21.15.203:6789/0 conn(0x7f40ec050330 legacy=0x7f40ec050820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:34:35.799 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:35.767+0000 7f40f9211640 1 -- 172.21.15.43:0/4201619068 <== mon.1 v1:172.21.15.203:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1303701575 0 0) 0x7f40ec04fea0 con 0x7f40ec050330 2023-08-09T13:34:35.799 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:35.768+0000 7f40faa14640 1 -- 172.21.15.43:0/4201619068 --> v2:172.21.15.203:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f40ec04f570 con 0x7f40ec050f30 2023-08-09T13:34:35.800 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:35.768+0000 7fb8e15fe640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fb8d40eac80 0x7fb8d4048940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:34:35.800 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:35.768+0000 7fb8e15fe640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb8d4048e80 con 0x7fb8d4006880 2023-08-09T13:34:35.800 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:35.768+0000 7fb8e15fe640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb8d404a970 con 0x7fb8d4005ec0 2023-08-09T13:34:35.800 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:35.768+0000 7fb8e15fe640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb8d404ac50 con 0x7fb8d40eac80 2023-08-09T13:34:35.800 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:35.768+0000 7fb8db7fe640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb8d4006880 0x7fb8d404a320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:34:35.801 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:35.768+0000 7fb8dbfff640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fb8d4005ec0 0x7fb8d40dd190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:34:35.801 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:35.768+0000 7fb8db7fe640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb8d4006880 0x7fb8d404a320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:51628/0 (socket says 172.21.15.43:51628) 2023-08-09T13:34:35.801 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:35.768+0000 7fb8dbfff640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fb8d4005ec0 0x7fb8d40dd190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.203:3300/0 says I am v2:172.21.15.43:34288/0 (socket says 172.21.15.43:34288) 2023-08-09T13:34:35.801 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:35.768+0000 7fb8db7fe640 1 -- 172.21.15.43:0/3122861890 learned_addr learned my addr 172.21.15.43:0/3122861890 (peer_addr_for_me v2:172.21.15.43:0/0) 2023-08-09T13:34:35.801 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:35.768+0000 7fb8e0dfd640 1 --2- 172.21.15.43:0/3122861890 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fb8d40eac80 0x7fb8d4048940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:34:35.801 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:35.768+0000 7f7b54999640 1 Processor -- start 2023-08-09T13:34:35.802 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:35.768+0000 7f7b54999640 1 -- start start 2023-08-09T13:34:35.802 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:35.769+0000 7fb8db7fe640 1 -- 172.21.15.43:0/3122861890 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fb8d40eac80 msgr2=0x7fb8d4048940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:34:35.802 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:35.769+0000 7fb8db7fe640 1 --2- 172.21.15.43:0/3122861890 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fb8d40eac80 0x7fb8d4048940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:34:35.802 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:35.769+0000 7fb8db7fe640 1 -- 172.21.15.43:0/3122861890 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fb8d4005ec0 msgr2=0x7fb8d40dd190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:34:35.802 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:35.769+0000 7fb8db7fe640 1 --2- 172.21.15.43:0/3122861890 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fb8d4005ec0 0x7fb8d40dd190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:34:35.802 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:35.769+0000 7fb8db7fe640 1 -- 172.21.15.43:0/3122861890 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb8d00094d0 con 0x7fb8d4006880 2023-08-09T13:34:35.803 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:35.769+0000 7f40faa14640 1 --2- 172.21.15.43:0/4201619068 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f40ec050f30 0x7f40ec052810 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f40f4067f10 tx=0x7f40f40a50e0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=1d5868feb9493677 server_cookie=0 in_seq=0 out_seq=0 2023-08-09T13:34:35.803 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:35.769+0000 7f7b54999640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f7b38005180 0x7f7b48049030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:34:35.803 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:35.769+0000 7fc156ffd640 1 -- 172.21.15.43:0/675462142 <== mon.1 v2:172.21.15.203:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc14c02ec40 con 0x7fc1580ca710 2023-08-09T13:34:35.804 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:35.769+0000 7f7b54999640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f7b48181120 0x7f7b481807f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:34:35.804 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:35.769+0000 7fb8db7fe640 1 --2- 172.21.15.43:0/3122861890 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb8d4006880 0x7fb8d404a320 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7fb8c800b410 tx=0x7fb8c800b810 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-09T13:34:35.804 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:35.770+0000 7f7b54999640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f7b480e94b0 0x7f7b4804e0f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:34:35.805 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:35.769+0000 7fc156ffd640 1 -- 172.21.15.43:0/675462142 <== mon.1 v2:172.21.15.203:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc14c032070 con 0x7fc1580ca710 2023-08-09T13:34:35.805 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:35.769+0000 7fc156ffd640 1 -- 172.21.15.43:0/675462142 <== mon.1 v2:172.21.15.203:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc14c038400 con 0x7fc1580ca710 2023-08-09T13:34:35.805 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:35.770+0000 7f40f9211640 1 -- 172.21.15.43:0/4201619068 <== mon.1 v2:172.21.15.203:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f40f40a5ce0 con 0x7f40ec050f30 2023-08-09T13:34:35.805 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:35.770+0000 7f40f9211640 1 -- 172.21.15.43:0/4201619068 <== mon.1 v2:172.21.15.203:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f40f40a5e80 con 0x7f40ec050f30 2023-08-09T13:34:35.806 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:35.770+0000 7f40f9211640 1 -- 172.21.15.43:0/4201619068 <== mon.1 v2:172.21.15.203:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f40f40ad640 con 0x7f40ec050f30 2023-08-09T13:34:35.806 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:35.770+0000 7fb8d8ff9640 1 -- 172.21.15.43:0/3122861890 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb8c8011830 con 0x7fb8d4006880 2023-08-09T13:34:35.806 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:35.770+0000 7f7b54999640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7b480e7030 con 0x7f7b480e94b0 2023-08-09T13:34:35.806 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:35.770+0000 7fb8d8ff9640 1 -- 172.21.15.43:0/3122861890 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb8c8011990 con 0x7fb8d4006880 2023-08-09T13:34:35.807 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:35.770+0000 7f7b4e59c640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f7b38005180 0x7f7b48049030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:34:35.807 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:35.770+0000 7f7b4e59c640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f7b38005180 0x7f7b48049030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.203:3301/0 says I am v2:172.21.15.43:44882/0 (socket says 172.21.15.43:44882) 2023-08-09T13:34:35.807 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:35.770+0000 7f7b4e59c640 1 -- 172.21.15.43:0/4169296907 learned_addr learned my addr 172.21.15.43:0/4169296907 (peer_addr_for_me v2:172.21.15.43:0/0) 2023-08-09T13:34:35.807 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:35.770+0000 7f7b54999640 1 -- 172.21.15.43:0/4169296907 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7b48049930 con 0x7f7b48181120 2023-08-09T13:34:35.808 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:35.770+0000 7f7b54999640 1 -- 172.21.15.43:0/4169296907 --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7b48049c10 con 0x7f7b38005180 2023-08-09T13:34:35.808 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:35.770+0000 7f7b4dd9b640 1 --2- 172.21.15.43:0/4169296907 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f7b48181120 0x7f7b481807f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:34:35.808 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:35.771+0000 7f7b4ed9d640 1 --2- 172.21.15.43:0/4169296907 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f7b480e94b0 0x7f7b4804e0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:34:35.808 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:35.771+0000 7f40fc477640 1 -- 172.21.15.43:0/4201619068 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f40ec050f30 msgr2=0x7f40ec052810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:34:35.809 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:35.771+0000 7f40fc477640 1 --2- 172.21.15.43:0/4201619068 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f40ec050f30 0x7f40ec052810 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f40f4067f10 tx=0x7f40f40a50e0 comp rx=0 tx=0).stop 2023-08-09T13:34:35.809 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:35.771+0000 7f40fc477640 1 -- 172.21.15.43:0/4201619068 shutdown_connections 2023-08-09T13:34:35.809 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:35.771+0000 7f40fc477640 1 --2- 172.21.15.43:0/4201619068 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f40ec050f30 0x7f40ec052810 unknown :-1 s=CLOSED pgs=83 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:34:35.809 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:35.771+0000 7f40fc477640 1 --2- 172.21.15.43:0/4201619068 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f40ec0dffd0 0x7f40ec0399a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:34:35.809 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:35.771+0000 7f40fc477640 1 -- 172.21.15.43:0/4201619068 >> 172.21.15.43:0/4201619068 conn(0x7f40ec054c40 msgr2=0x7f40ec055020 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-09T13:34:35.809 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:35.771+0000 7f7b4dd9b640 1 -- 172.21.15.43:0/4169296907 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f7b38005180 msgr2=0x7f7b48049030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:34:35.810 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:35.771+0000 7f7b4dd9b640 1 --2- 172.21.15.43:0/4169296907 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f7b38005180 0x7f7b48049030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:34:35.810 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:35.771+0000 7f7b4dd9b640 1 -- 172.21.15.43:0/4169296907 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f7b480e94b0 msgr2=0x7f7b4804e0f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:34:35.810 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:35.771+0000 7f7b4dd9b640 1 --2- 172.21.15.43:0/4169296907 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f7b480e94b0 0x7f7b4804e0f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:34:35.810 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:35.771+0000 7f7b4dd9b640 1 -- 172.21.15.43:0/4169296907 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7b48049ef0 con 0x7f7b48181120 2023-08-09T13:34:35.810 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:35.771+0000 7f7b4e59c640 1 --2- 172.21.15.43:0/4169296907 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f7b38005180 0x7f7b48049030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-08-09T13:34:35.810 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:35.771+0000 7fb8d8ff9640 1 -- 172.21.15.43:0/3122861890 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb8c8011d40 con 0x7fb8d4006880 2023-08-09T13:34:35.811 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:35.770+0000 7f7d190ad640 1 -- start start 2023-08-09T13:34:35.811 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:35.771+0000 7f7d190ad640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f7d1406f690 0x7f7d141f3520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:34:35.811 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:35.771+0000 7f7d190ad640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f7d140b0490 0x7f7d1415d050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:34:35.811 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:35.772+0000 7fb8e15fe640 1 -- 172.21.15.43:0/3122861890 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb8d404af30 con 0x7fb8d4006880 2023-08-09T13:34:35.812 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:35.771+0000 7f7d190ad640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f7d140c9810 0x7f7d140b0aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:34:35.812 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:35.771+0000 7f7d190ad640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7d1414b870 con 0x7f7d140b0490 2023-08-09T13:34:35.812 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:35.771+0000 7f7d190ad640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7d140b1bb0 con 0x7f7d140c9810 2023-08-09T13:34:35.812 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:35.771+0000 7f7d190ad640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7d14158780 con 0x7f7d1406f690 2023-08-09T13:34:35.813 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:35.772+0000 7f7b4dd9b640 1 --2- 172.21.15.43:0/4169296907 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f7b48181120 0x7f7b481807f0 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f7b44004a60 tx=0x7f7b44004f20 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-09T13:34:35.813 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:35.772+0000 7f7b3effd640 1 -- 172.21.15.43:0/4169296907 <== mon.1 v2:172.21.15.203:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7b4400fa20 con 0x7f7b48181120 2023-08-09T13:34:35.813 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:35.772+0000 7f7b3effd640 1 -- 172.21.15.43:0/4169296907 <== mon.1 v2:172.21.15.203:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7b4400fb80 con 0x7f7b48181120 2023-08-09T13:34:35.813 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:35.772+0000 7f7b3effd640 1 -- 172.21.15.43:0/4169296907 <== mon.1 v2:172.21.15.203:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7b440133f0 con 0x7f7b48181120 2023-08-09T13:34:35.813 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:35.772+0000 7f7b54999640 1 -- 172.21.15.43:0/4169296907 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7b4804a1d0 con 0x7f7b48181120 2023-08-09T13:34:35.813 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:35.773+0000 7f7b54999640 1 -- 172.21.15.43:0/4169296907 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7b48180d30 con 0x7f7b48181120 2023-08-09T13:34:35.814 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:35.773+0000 7fc15e5c8640 1 -- 172.21.15.43:0/675462142 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fc1580ca710 msgr2=0x7fc1580cabc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:34:35.814 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:35.773+0000 7fc15e5c8640 1 --2- 172.21.15.43:0/675462142 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fc1580ca710 0x7fc1580cabc0 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7fc1580b0b40 tx=0x7fc14c02de90 comp rx=0 tx=0).stop 2023-08-09T13:34:35.814 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:35.773+0000 7fb8e15fe640 1 -- 172.21.15.43:0/3122861890 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb8d404b4f0 con 0x7fb8d4006880 2023-08-09T13:34:35.815 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:35.773+0000 7f7d12d9d640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f7d1406f690 0x7f7d141f3520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:34:35.815 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:35.773+0000 7f7d12d9d640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f7d1406f690 0x7f7d141f3520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.203:3301/0 says I am v2:172.21.15.43:44890/0 (socket says 172.21.15.43:44890) 2023-08-09T13:34:35.815 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:35.773+0000 7f7d12d9d640 1 -- 172.21.15.43:0/6031396 learned_addr learned my addr 172.21.15.43:0/6031396 (peer_addr_for_me v2:172.21.15.43:0/0) 2023-08-09T13:34:35.815 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:35.774+0000 7fc15e5c8640 1 -- 172.21.15.43:0/675462142 shutdown_connections 2023-08-09T13:34:35.815 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:35.774+0000 7fc15e5c8640 1 --2- 172.21.15.43:0/675462142 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fc1580ca710 0x7fc1580cabc0 unknown :-1 s=CLOSED pgs=81 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:34:35.816 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:35.774+0000 7fc15e5c8640 1 --2- 172.21.15.43:0/675462142 >> v2:172.21.15.43:3300/0 conn(0x7fc1580cc0c0 0x7fc1580cc550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:34:35.816 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:35.774+0000 7fc15e5c8640 1 -- 172.21.15.43:0/675462142 >> 172.21.15.43:0/675462142 conn(0x7fc1580afcb0 msgr2=0x7fc1580b0090 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-09T13:34:35.816 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:35.774+0000 7f7d1359e640 1 --2- 172.21.15.43:0/6031396 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f7d140c9810 0x7f7d140b0aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:34:35.816 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:35.774+0000 7fdaabfff640 1 Processor -- start 2023-08-09T13:34:35.817 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:35.774+0000 7fb8e15fe640 1 -- 172.21.15.43:0/3122861890 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7fb8d404fb60 con 0x7fb8d4006880 2023-08-09T13:34:35.817 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:35.774+0000 7fdaabfff640 1 -- start start 2023-08-09T13:34:35.817 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:35.774+0000 7f7b54999640 1 -- 172.21.15.43:0/4169296907 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f7b4804fe40 con 0x7f7b48181120 2023-08-09T13:34:35.818 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:35.775+0000 7f7d12d9d640 1 -- 172.21.15.43:0/6031396 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f7d140c9810 msgr2=0x7f7d140b0aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:34:35.818 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:35.775+0000 7f7d12d9d640 1 --2- 172.21.15.43:0/6031396 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f7d140c9810 0x7f7d140b0aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:34:35.818 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:35.775+0000 7f7d12d9d640 1 -- 172.21.15.43:0/6031396 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f7d140b0490 msgr2=0x7f7d1415d050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:34:35.818 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:35.775+0000 7f7d12d9d640 1 --2- 172.21.15.43:0/6031396 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f7d140b0490 0x7f7d1415d050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:34:35.818 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:35.775+0000 7f7d12d9d640 1 -- 172.21.15.43:0/6031396 --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7d0c009a70 con 0x7f7d1406f690 2023-08-09T13:34:35.819 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:35.775+0000 7fb8d8ff9640 1 -- 172.21.15.43:0/3122861890 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 5) v1 ==== 88341+0+0 (secure 0 0 0) 0x7fb8c801a400 con 0x7fb8d4006880 2023-08-09T13:34:35.819 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:35.775+0000 7fc15e5c8640 1 -- 172.21.15.43:0/675462142 shutdown_connections 2023-08-09T13:34:35.819 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:35.775+0000 7f40fc477640 1 -- 172.21.15.43:0/4201619068 shutdown_connections 2023-08-09T13:34:35.819 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:35.776+0000 7f40fc477640 1 -- 172.21.15.43:0/4201619068 wait complete. 2023-08-09T13:34:35.820 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:35.776+0000 7fdaabfff640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fdaa4053b30 0x7fdaa403c640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:34:35.820 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:35.777+0000 7fdaaaffd640 1 --1- >> v1:172.21.15.43:6789/0 conn(0x7fdaa40e33f0 0x7fdaa4055a70 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.43:6789/0 says I am v1:172.21.15.43:35612/0 (socket says 172.21.15.43:35612) 2023-08-09T13:34:35.820 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:35.777+0000 7fdaaa7fc640 1 --1- >> v1:172.21.15.203:6789/0 conn(0x7fdaa4053680 0x7fdaa403bf30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.203:6789/0 says I am v1:172.21.15.43:33758/0 (socket says 172.21.15.43:33758) 2023-08-09T13:34:35.820 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:35.777+0000 7fdaaaffd640 1 -- 172.21.15.43:0/4234937518 learned_addr learned my addr 172.21.15.43:0/4234937518 (peer_addr_for_me v1:172.21.15.43:0/0) 2023-08-09T13:34:35.821 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:35.777+0000 7fdaab7fe640 1 --2- 172.21.15.43:0/4234937518 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fdaa4053b30 0x7fdaa403c640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:34:35.821 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:35.777+0000 7f7b3effd640 1 -- 172.21.15.43:0/4169296907 <== mon.1 v2:172.21.15.203:3300/0 4 ==== mgrmap(e 5) v1 ==== 88341+0+0 (secure 0 0 0) 0x7f7b44013550 con 0x7f7b48181120 2023-08-09T13:34:35.821 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:35.778+0000 7fdaabfff640 1 -- 172.21.15.43:0/4234937518 --> v1:172.21.15.43:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fdaa4053220 con 0x7fdaa40e33f0 2023-08-09T13:34:35.822 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:35.776+0000 7f7d12d9d640 1 --2- 172.21.15.43:0/6031396 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f7d1406f690 0x7f7d141f3520 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7f7d0400c530 tx=0x7f7d0400c9f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-09T13:34:35.822 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:35.776+0000 7f7cfb7fe640 1 -- 172.21.15.43:0/6031396 <== mon.2 v2:172.21.15.203:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7d04005ce0 con 0x7f7d1406f690 2023-08-09T13:34:35.822 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:35.776+0000 7f7cfb7fe640 1 -- 172.21.15.43:0/6031396 <== mon.2 v2:172.21.15.203:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7d04005e40 con 0x7f7d1406f690 2023-08-09T13:34:35.822 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:35.776+0000 7f7cfb7fe640 1 -- 172.21.15.43:0/6031396 <== mon.2 v2:172.21.15.203:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7d04007270 con 0x7f7d1406f690 2023-08-09T13:34:35.822 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:35.778+0000 7fb8d8ff9640 1 --2- 172.21.15.43:0/3122861890 >> [v2:172.21.15.203:6832/830097474,v1:172.21.15.203:6833/830097474] conn(0x7fb8c006a0d0 0x7fb8c006c590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:34:35.823 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:35.775+0000 7fc15e5c8640 1 -- 172.21.15.43:0/675462142 wait complete. 2023-08-09T13:34:35.823 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:35.778+0000 7f40fc477640 1 Processor -- start 2023-08-09T13:34:35.823 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:35.778+0000 7fc15e5c8640 1 Processor -- start 2023-08-09T13:34:35.824 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:35.778+0000 7f7d190ad640 1 -- 172.21.15.43:0/6031396 --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7d140b15f0 con 0x7f7d1406f690 2023-08-09T13:34:35.824 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:35.778+0000 7f7d190ad640 1 -- 172.21.15.43:0/6031396 --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7d140b2870 con 0x7f7d1406f690 2023-08-09T13:34:35.824 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:35.778+0000 7fdaabfff640 1 -- 172.21.15.43:0/4234937518 --> v1:172.21.15.203:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fdaa4053f80 con 0x7fdaa4053680 2023-08-09T13:34:35.824 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:35.779+0000 7fc15e5c8640 1 -- start start 2023-08-09T13:34:35.825 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:35.779+0000 7fb8dbfff640 1 --2- 172.21.15.43:0/3122861890 >> [v2:172.21.15.203:6832/830097474,v1:172.21.15.203:6833/830097474] conn(0x7fb8c006a0d0 0x7fb8c006c590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:34:35.825 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:35.778+0000 7fdaa9ffb640 1 -- 172.21.15.43:0/4234937518 <== mon.0 v1:172.21.15.43:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2543960147 0 0) 0x7fdaa4053220 con 0x7fdaa40e33f0 2023-08-09T13:34:35.825 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:35.779+0000 7fdaabfff640 1 -- 172.21.15.43:0/4234937518 --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdaa40e37d0 con 0x7fdaa4053b30 2023-08-09T13:34:35.825 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:35.779+0000 7fdaa9ffb640 1 -- 172.21.15.43:0/4234937518 --> v1:172.21.15.43:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fda94003520 con 0x7fdaa40e33f0 2023-08-09T13:34:35.826 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:35.779+0000 7fb8d8ff9640 1 -- 172.21.15.43:0/3122861890 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6813+0+0 (secure 0 0 0) 0x7fb8c8087a20 con 0x7fb8d4006880 2023-08-09T13:34:35.826 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:35.779+0000 7fdaa9ffb640 1 -- 172.21.15.43:0/4234937518 <== mon.1 v1:172.21.15.203:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2665352316 0 0) 0x7fdaa4053f80 con 0x7fdaa4053680 2023-08-09T13:34:35.826 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:35.779+0000 7fb8d8ff9640 1 --2- 172.21.15.43:0/3122861890 >> [v2:172.21.15.43:6813/1709538759,v1:172.21.15.43:6816/1709538759] conn(0x7fb8c0073c40 0x7fb8c0076080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:34:35.827 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:35.779+0000 7fb8d8ff9640 1 -- 172.21.15.43:0/3122861890 --> [v2:172.21.15.43:6813/1709538759,v1:172.21.15.43:6816/1709538759] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7fb8c0076750 con 0x7fb8c0073c40 2023-08-09T13:34:35.827 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:35.779+0000 7fb8d8ff9640 1 -- 172.21.15.43:0/3122861890 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_get_version_reply(handle=1 version=24) v2 ==== 24+0+0 (secure 0 0 0) 0x7fb8c8087da0 con 0x7fb8d4006880 2023-08-09T13:34:35.827 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:35.779+0000 7fdaa9ffb640 1 -- 172.21.15.43:0/4234937518 --> v1:172.21.15.203:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fdaa4053220 con 0x7fdaa4053680 2023-08-09T13:34:35.827 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:35.779+0000 7f7b3effd640 1 --2- 172.21.15.43:0/4169296907 >> [v2:172.21.15.203:6832/830097474,v1:172.21.15.203:6833/830097474] conn(0x7f7b3006a080 0x7f7b3006c540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:34:35.828 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:35.779+0000 7f7b3effd640 1 -- 172.21.15.43:0/4169296907 <== mon.1 v2:172.21.15.203:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6813+0+0 (secure 0 0 0) 0x7f7b44089510 con 0x7f7b48181120 2023-08-09T13:34:35.828 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:35.779+0000 7f7b3effd640 1 --2- 172.21.15.43:0/4169296907 >> [v2:172.21.15.43:6811/4079709736,v1:172.21.15.43:6814/4079709736] conn(0x7f7b30073ce0 0x7f7b30076120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:34:35.828 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:35.779+0000 7f7b3effd640 1 -- 172.21.15.43:0/4169296907 --> [v2:172.21.15.43:6811/4079709736,v1:172.21.15.43:6814/4079709736] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f7b300767f0 con 0x7f7b30073ce0 2023-08-09T13:34:35.828 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:35.779+0000 7f7b3effd640 1 -- 172.21.15.43:0/4169296907 <== mon.1 v2:172.21.15.203:3300/0 6 ==== mon_get_version_reply(handle=1 version=24) v2 ==== 24+0+0 (secure 0 0 0) 0x7f7b440898c0 con 0x7f7b48181120 2023-08-09T13:34:35.828 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:35.780+0000 7f7b4ed9d640 1 --2- 172.21.15.43:0/4169296907 >> [v2:172.21.15.43:6811/4079709736,v1:172.21.15.43:6814/4079709736] conn(0x7f7b30073ce0 0x7f7b30076120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:34:35.829 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:35.778+0000 7f40fc477640 1 -- start start 2023-08-09T13:34:35.829 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:35.780+0000 7f40fc477640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f40ec050330 0x7f40ec04b4c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:34:35.829 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:35.780+0000 7f7b4e59c640 1 --2- 172.21.15.43:0/4169296907 >> [v2:172.21.15.203:6832/830097474,v1:172.21.15.203:6833/830097474] conn(0x7f7b3006a080 0x7f7b3006c540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:34:35.829 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:35.780+0000 7fb8e0dfd640 1 --2- 172.21.15.43:0/3122861890 >> [v2:172.21.15.43:6813/1709538759,v1:172.21.15.43:6816/1709538759] conn(0x7fb8c0073c40 0x7fb8c0076080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:34:35.830 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:35.781+0000 7f40fc477640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f40ec050f30 0x7f40ec0db660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:34:35.830 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:35.781+0000 7f40fa213640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f40ec050330 0x7f40ec04b4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:34:35.830 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:35.781+0000 7f40fc477640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f40ec0dffd0 0x7f40ec0d5b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:34:35.830 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:35.781+0000 7f40fa213640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f40ec050330 0x7f40ec04b4c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:51662/0 (socket says 172.21.15.43:51662) 2023-08-09T13:34:35.830 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:35.780+0000 7fdaab7fe640 1 -- 172.21.15.43:0/4234937518 >> v1:172.21.15.203:6789/0 conn(0x7fdaa4053680 legacy=0x7fdaa403bf30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:34:35.831 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:35.780+0000 7fdaab7fe640 1 -- 172.21.15.43:0/4234937518 >> v1:172.21.15.43:6789/0 conn(0x7fdaa40e33f0 legacy=0x7fdaa4055a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:34:35.831 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:35.780+0000 7fdaab7fe640 1 -- 172.21.15.43:0/4234937518 --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdaa4056540 con 0x7fdaa4053b30 2023-08-09T13:34:35.831 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:35.781+0000 7fc15e5c8640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fc15806b790 0x7fc158088b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:34:35.831 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:35.781+0000 7f40fa213640 1 -- 172.21.15.43:0/2754910972 learned_addr learned my addr 172.21.15.43:0/2754910972 (peer_addr_for_me v2:172.21.15.43:0/0) 2023-08-09T13:34:35.832 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:35.781+0000 7f7b4ed9d640 1 --2- 172.21.15.43:0/4169296907 >> [v2:172.21.15.43:6811/4079709736,v1:172.21.15.43:6814/4079709736] conn(0x7f7b30073ce0 0x7f7b30076120 crc :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-09T13:34:35.832 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:35.781+0000 7fc15e5c8640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fc1580ca710 0x7fc15816b6f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:34:35.832 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:35.781+0000 7f40fc477640 1 -- 172.21.15.43:0/2754910972 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f40ec0d5760 con 0x7f40ec050330 2023-08-09T13:34:35.833 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:35.781+0000 7fb8dbfff640 1 --2- 172.21.15.43:0/3122861890 >> [v2:172.21.15.203:6832/830097474,v1:172.21.15.203:6833/830097474] conn(0x7fb8c006a0d0 0x7fb8c006c590 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7fb8d0036000 tx=0x7fb8d0008030 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-09T13:34:35.833 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:35.781+0000 7fb8e0dfd640 1 --2- 172.21.15.43:0/3122861890 >> [v2:172.21.15.43:6813/1709538759,v1:172.21.15.43:6816/1709538759] conn(0x7fb8c0073c40 0x7fb8c0076080 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-09T13:34:35.833 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:35.781+0000 7f40fc477640 1 -- 172.21.15.43:0/2754910972 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f40ec0de540 con 0x7f40ec0dffd0 2023-08-09T13:34:35.833 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:35.781+0000 7f40fc477640 1 -- 172.21.15.43:0/2754910972 --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f40ec0de820 con 0x7f40ec050f30 2023-08-09T13:34:35.834 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:35.781+0000 7fdaab7fe640 1 --2- 172.21.15.43:0/4234937518 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fdaa4053b30 0x7fdaa403c640 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7fdaac02c700 tx=0x7fdaac0a4f00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=28931a6d010c176d server_cookie=0 in_seq=0 out_seq=0 2023-08-09T13:34:35.834 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:35.781+0000 7fc15e5c8640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc1580cc0c0 0x7fc15816bc30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:34:35.834 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:35.782+0000 7f40f9a12640 1 --2- 172.21.15.43:0/2754910972 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f40ec050f30 0x7f40ec0db660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:34:35.835 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:35.782+0000 7fc157fff640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fc15806b790 0x7fc158088b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:34:35.835 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:35.782+0000 7fc157fff640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fc15806b790 0x7fc158088b40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.203:3300/0 says I am v2:172.21.15.43:34334/0 (socket says 172.21.15.43:34334) 2023-08-09T13:34:35.835 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:35.782+0000 7fc1577fe640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fc1580ca710 0x7fc15816b6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:34:35.835 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:35.782+0000 7fc1577fe640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fc1580ca710 0x7fc15816b6f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.203:3301/0 says I am v2:172.21.15.43:44926/0 (socket says 172.21.15.43:44926) 2023-08-09T13:34:35.836 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:35.782+0000 7fdaa9ffb640 1 -- 172.21.15.43:0/4234937518 <== mon.2 v2:172.21.15.203:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdaac0a5d10 con 0x7fdaa4053b30 2023-08-09T13:34:35.836 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:35.782+0000 7fc157fff640 1 -- 172.21.15.43:0/1890653340 learned_addr learned my addr 172.21.15.43:0/1890653340 (peer_addr_for_me v2:172.21.15.43:0/0) 2023-08-09T13:34:35.836 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:35.782+0000 7fc15e5c8640 1 -- 172.21.15.43:0/1890653340 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc1580c82a0 con 0x7fc1580cc0c0 2023-08-09T13:34:35.836 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:35.782+0000 7fb8d8ff9640 1 -- 172.21.15.43:0/3122861890 <== osd.3 v2:172.21.15.43:6813/1709538759 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+25607 (crc 0 0 0) 0x7fb8c0076750 con 0x7fb8c0073c40 2023-08-09T13:34:35.837 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:35.782+0000 7fc15e5c8640 1 -- 172.21.15.43:0/1890653340 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc1580c61f0 con 0x7fc15806b790 2023-08-09T13:34:35.837 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:35.782+0000 7fdaa9ffb640 1 -- 172.21.15.43:0/4234937518 <== mon.2 v2:172.21.15.203:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdaac0ad440 con 0x7fdaa4053b30 2023-08-09T13:34:35.837 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:35.782+0000 7fc15e5c8640 1 -- 172.21.15.43:0/1890653340 --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc1580c64d0 con 0x7fc1580ca710 2023-08-09T13:34:35.838 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:35.782+0000 7f40faa14640 1 --2- 172.21.15.43:0/2754910972 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f40ec0dffd0 0x7f40ec0d5b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:34:35.838 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:35.783+0000 7f7b4e59c640 1 --2- 172.21.15.43:0/4169296907 >> [v2:172.21.15.203:6832/830097474,v1:172.21.15.203:6833/830097474] conn(0x7f7b3006a080 0x7f7b3006c540 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7f7b400134d0 tx=0x7f7b40008040 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-09T13:34:35.838 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:35.783+0000 7fc15cb65640 1 --2- 172.21.15.43:0/1890653340 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc1580cc0c0 0x7fc15816bc30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:34:35.839 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:35.783+0000 7f7b3effd640 1 -- 172.21.15.43:0/4169296907 <== osd.2 v2:172.21.15.43:6811/4079709736 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+25607 (crc 0 0 0) 0x7f7b300767f0 con 0x7f7b30073ce0 2023-08-09T13:34:35.839 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:35.784+0000 7f7d190ad640 1 -- 172.21.15.43:0/6031396 --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f7d140ca610 con 0x7f7d1406f690 2023-08-09T13:34:35.839 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:35.783+0000 7fc15cb65640 1 -- 172.21.15.43:0/1890653340 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fc1580ca710 msgr2=0x7fc15816b6f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:34:35.840 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:35.783+0000 7fc15cb65640 1 --2- 172.21.15.43:0/1890653340 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fc1580ca710 0x7fc15816b6f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:34:35.840 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:35.783+0000 7fc15cb65640 1 -- 172.21.15.43:0/1890653340 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fc15806b790 msgr2=0x7fc158088b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:34:35.840 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:35.783+0000 7fc15cb65640 1 --2- 172.21.15.43:0/1890653340 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fc15806b790 0x7fc158088b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:34:35.840 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:35.783+0000 7fc15cb65640 1 -- 172.21.15.43:0/1890653340 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc148009400 con 0x7fc1580cc0c0 2023-08-09T13:34:35.840 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:35.784+0000 7fc15cb65640 1 --2- 172.21.15.43:0/1890653340 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc1580cc0c0 0x7fc15816bc30 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7fc14c036000 tx=0x7fc14c004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-09T13:34:35.840 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:35.783+0000 7f40faa14640 1 -- 172.21.15.43:0/2754910972 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f40ec050f30 msgr2=0x7f40ec0db660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:34:35.841 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:35.783+0000 7f40faa14640 1 --2- 172.21.15.43:0/2754910972 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f40ec050f30 0x7f40ec0db660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:34:35.841 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:35.783+0000 7f40faa14640 1 -- 172.21.15.43:0/2754910972 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f40ec050330 msgr2=0x7f40ec04b4c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:34:35.841 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:35.783+0000 7f40faa14640 1 --2- 172.21.15.43:0/2754910972 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f40ec050330 0x7f40ec04b4c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:34:35.841 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:35.783+0000 7f40faa14640 1 -- 172.21.15.43:0/2754910972 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f40ec04ba00 con 0x7f40ec0dffd0 2023-08-09T13:34:35.841 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:35.783+0000 7fdaabfff640 1 -- 172.21.15.43:0/4234937518 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fdaa4053b30 msgr2=0x7fdaa403c640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:34:35.842 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:35.783+0000 7fdaabfff640 1 --2- 172.21.15.43:0/4234937518 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fdaa4053b30 0x7fdaa403c640 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7fdaac02c700 tx=0x7fdaac0a4f00 comp rx=0 tx=0).stop 2023-08-09T13:34:35.842 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:35.784+0000 7fdaabfff640 1 -- 172.21.15.43:0/4234937518 shutdown_connections 2023-08-09T13:34:35.842 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:35.784+0000 7fdaabfff640 1 --2- 172.21.15.43:0/4234937518 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fdaa4053b30 0x7fdaa403c640 unknown :-1 s=CLOSED pgs=45 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:34:35.842 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:35.784+0000 7fdaabfff640 1 -- 172.21.15.43:0/4234937518 >> 172.21.15.43:0/4234937518 conn(0x7fdaa40200d0 msgr2=0x7fdaa4056e00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-09T13:34:35.842 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:35.784+0000 7f7cfb7fe640 1 -- 172.21.15.43:0/6031396 <== mon.2 v2:172.21.15.203:3301/0 4 ==== mgrmap(e 5) v1 ==== 88341+0+0 (secure 0 0 0) 0x7f7d040073d0 con 0x7f7d1406f690 2023-08-09T13:34:35.843 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:35.785+0000 7fc154ff9640 1 -- 172.21.15.43:0/1890653340 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc14c046070 con 0x7fc1580cc0c0 2023-08-09T13:34:35.843 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:35.785+0000 7fc154ff9640 1 -- 172.21.15.43:0/1890653340 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc14c00c330 con 0x7fc1580cc0c0 2023-08-09T13:34:35.843 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:35.785+0000 7fc154ff9640 1 -- 172.21.15.43:0/1890653340 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc14c009c20 con 0x7fc1580cc0c0 2023-08-09T13:34:35.843 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:35.785+0000 7fc15e5c8640 1 -- 172.21.15.43:0/1890653340 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc14c00c650 con 0x7fc1580cc0c0 2023-08-09T13:34:35.843 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:35.785+0000 7fdaabfff640 1 -- 172.21.15.43:0/4234937518 shutdown_connections 2023-08-09T13:34:35.844 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:35.785+0000 7f7cfb7fe640 1 --2- 172.21.15.43:0/6031396 >> [v2:172.21.15.203:6832/830097474,v1:172.21.15.203:6833/830097474] conn(0x7f7d0006a120 0x7f7d0006c5e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:34:35.844 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:35.785+0000 7f7cfb7fe640 1 -- 172.21.15.43:0/6031396 <== mon.2 v2:172.21.15.203:3301/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6813+0+0 (secure 0 0 0) 0x7f7d04087cd0 con 0x7f7d1406f690 2023-08-09T13:34:35.844 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:35.786+0000 7f7cfb7fe640 1 --2- 172.21.15.43:0/6031396 >> [v2:172.21.15.203:6800/4196866481,v1:172.21.15.203:6801/4196866481] conn(0x7f7d00073e00 0x7f7d00076240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:34:35.844 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:35.786+0000 7f7cfb7fe640 1 -- 172.21.15.43:0/6031396 --> [v2:172.21.15.203:6800/4196866481,v1:172.21.15.203:6801/4196866481] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f7d00076910 con 0x7f7d00073e00 2023-08-09T13:34:35.844 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:35.786+0000 7f7cfb7fe640 1 -- 172.21.15.43:0/6031396 <== mon.2 v2:172.21.15.203:3301/0 6 ==== mon_get_version_reply(handle=1 version=24) v2 ==== 24+0+0 (secure 0 0 0) 0x7f7d0408a2f0 con 0x7f7d1406f690 2023-08-09T13:34:35.844 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:35.787+0000 7f7d1359e640 1 --2- 172.21.15.43:0/6031396 >> [v2:172.21.15.203:6800/4196866481,v1:172.21.15.203:6801/4196866481] conn(0x7f7d00073e00 0x7f7d00076240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:34:35.845 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:35.787+0000 7f7d1259c640 1 --2- 172.21.15.43:0/6031396 >> [v2:172.21.15.203:6832/830097474,v1:172.21.15.203:6833/830097474] conn(0x7f7d0006a120 0x7f7d0006c5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:34:35.845 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:35.786+0000 7fc15e5c8640 1 -- 172.21.15.43:0/1890653340 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc1580c6a90 con 0x7fc1580cc0c0 2023-08-09T13:34:35.845 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:35.786+0000 7f40faa14640 1 --2- 172.21.15.43:0/2754910972 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f40ec0dffd0 0x7f40ec0d5b50 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f40f4063820 tx=0x7f40f40721b0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-09T13:34:35.846 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:35.787+0000 7f40eaffd640 1 -- 172.21.15.43:0/2754910972 <== mon.1 v2:172.21.15.203:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f40f40b9650 con 0x7f40ec0dffd0 2023-08-09T13:34:35.846 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:35.787+0000 7f40eaffd640 1 -- 172.21.15.43:0/2754910972 <== mon.1 v2:172.21.15.203:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f40f40b97b0 con 0x7f40ec0dffd0 2023-08-09T13:34:35.846 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:35.787+0000 7f40eaffd640 1 -- 172.21.15.43:0/2754910972 <== mon.1 v2:172.21.15.203:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f40f40b9ba0 con 0x7f40ec0dffd0 2023-08-09T13:34:35.846 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:35.787+0000 7f40fc477640 1 -- 172.21.15.43:0/2754910972 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f40ec0dbba0 con 0x7f40ec0dffd0 2023-08-09T13:34:35.846 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:35.787+0000 7f40fc477640 1 -- 172.21.15.43:0/2754910972 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f40ec0d53a0 con 0x7f40ec0dffd0 2023-08-09T13:34:35.846 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:35.787+0000 7fc15e5c8640 1 -- 172.21.15.43:0/1890653340 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7fc1580c4100 con 0x7fc1580cc0c0 2023-08-09T13:34:35.847 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:35.787+0000 7fdaabfff640 1 -- 172.21.15.43:0/4234937518 wait complete. 2023-08-09T13:34:35.847 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:35.788+0000 7f7d1359e640 1 --2- 172.21.15.43:0/6031396 >> [v2:172.21.15.203:6800/4196866481,v1:172.21.15.203:6801/4196866481] conn(0x7f7d00073e00 0x7f7d00076240 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-09T13:34:35.847 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:35.788+0000 7f7d1259c640 1 --2- 172.21.15.43:0/6031396 >> [v2:172.21.15.203:6832/830097474,v1:172.21.15.203:6833/830097474] conn(0x7f7d0006a120 0x7f7d0006c5e0 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7f7d08000f60 tx=0x7f7d080079b0 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-09T13:34:35.848 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:35.789+0000 7f40fc477640 1 -- 172.21.15.43:0/2754910972 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f40dc000fc0 con 0x7f40ec0dffd0 2023-08-09T13:34:35.848 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:35.790+0000 7fdaabfff640 1 Processor -- start 2023-08-09T13:34:35.848 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:35.790+0000 7fdaabfff640 1 -- start start 2023-08-09T13:34:35.848 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:35.790+0000 7f7cfb7fe640 1 -- 172.21.15.43:0/6031396 <== osd.4 v2:172.21.15.203:6800/4196866481 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+25607 (crc 0 0 0) 0x7f7d00076910 con 0x7f7d00073e00 2023-08-09T13:34:35.849 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:35.791+0000 7fc154ff9640 1 -- 172.21.15.43:0/1890653340 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 5) v1 ==== 88341+0+0 (secure 0 0 0) 0x7fc14c042430 con 0x7fc1580cc0c0 2023-08-09T13:34:35.849 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:35.791+0000 7f40eaffd640 1 -- 172.21.15.43:0/2754910972 <== mon.1 v2:172.21.15.203:3300/0 4 ==== mgrmap(e 5) v1 ==== 88341+0+0 (secure 0 0 0) 0x7f40f40b9d00 con 0x7f40ec0dffd0 2023-08-09T13:34:35.849 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:35.792+0000 7fdaabfff640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fdaa4053680 0x7fdaa404d0f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:34:35.850 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:35.792+0000 7fdaaaffd640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fdaa4053680 0x7fdaa404d0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:34:35.850 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:35.792+0000 7fc154ff9640 1 --2- 172.21.15.43:0/1890653340 >> [v2:172.21.15.203:6832/830097474,v1:172.21.15.203:6833/830097474] conn(0x7fc14406a120 0x7fc14406c5e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:34:35.850 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:35.792+0000 7fdaaaffd640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fdaa4053680 0x7fdaa404d0f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:51682/0 (socket says 172.21.15.43:51682) 2023-08-09T13:34:35.850 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:35.793+0000 7fdaaaffd640 1 -- 172.21.15.43:0/1068176335 learned_addr learned my addr 172.21.15.43:0/1068176335 (peer_addr_for_me v2:172.21.15.43:0/0) 2023-08-09T13:34:35.851 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:35.793+0000 7fdaabfff640 1 --2- 172.21.15.43:0/1068176335 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fdaa4053b30 0x7fdaa403bda0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:34:35.851 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:35.793+0000 7fc154ff9640 1 -- 172.21.15.43:0/1890653340 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6813+0+0 (secure 0 0 0) 0x7fc14c0ab140 con 0x7fc1580cc0c0 2023-08-09T13:34:35.851 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:35.793+0000 7fdaabfff640 1 --2- 172.21.15.43:0/1068176335 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fdaa40e33f0 0x7fdaa403ab60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:34:35.851 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:35.793+0000 7fdaabfff640 1 -- 172.21.15.43:0/1068176335 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdaa403a6d0 con 0x7fdaa4053680 2023-08-09T13:34:35.852 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:35.793+0000 7fdaabfff640 1 -- 172.21.15.43:0/1068176335 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdaa4039ac0 con 0x7fdaa4053b30 2023-08-09T13:34:35.852 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:35.793+0000 7fdaabfff640 1 -- 172.21.15.43:0/1068176335 --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdaa4039da0 con 0x7fdaa40e33f0 2023-08-09T13:34:35.852 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:35.792+0000 7f40eaffd640 1 --2- 172.21.15.43:0/2754910972 >> [v2:172.21.15.203:6832/830097474,v1:172.21.15.203:6833/830097474] conn(0x7f40d806a120 0x7f40d806c5e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:34:35.852 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:35.793+0000 7f40eaffd640 1 -- 172.21.15.43:0/2754910972 <== mon.1 v2:172.21.15.203:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6813+0+0 (secure 0 0 0) 0x7f40f4123c30 con 0x7f40ec0dffd0 2023-08-09T13:34:35.852 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:35.793+0000 7f40eaffd640 1 --2- 172.21.15.43:0/2754910972 >> [v2:172.21.15.203:6824/512155569,v1:172.21.15.203:6825/512155569] conn(0x7f40d8073e00 0x7f40d8076240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:34:35.853 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:35.793+0000 7f40eaffd640 1 -- 172.21.15.43:0/2754910972 --> [v2:172.21.15.203:6824/512155569,v1:172.21.15.203:6825/512155569] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f40d8076910 con 0x7f40d8073e00 2023-08-09T13:34:35.853 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:35.793+0000 7f40eaffd640 1 -- 172.21.15.43:0/2754910972 <== mon.1 v2:172.21.15.203:3300/0 6 ==== mon_get_version_reply(handle=1 version=24) v2 ==== 24+0+0 (secure 0 0 0) 0x7f40f4128240 con 0x7f40ec0dffd0 2023-08-09T13:34:35.853 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:35.793+0000 7fc157fff640 1 --2- 172.21.15.43:0/1890653340 >> [v2:172.21.15.203:6832/830097474,v1:172.21.15.203:6833/830097474] conn(0x7fc14406a120 0x7fc14406c5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:34:35.853 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:35.793+0000 7fc154ff9640 1 --2- 172.21.15.43:0/1890653340 >> [v2:172.21.15.203:6816/3059290081,v1:172.21.15.203:6817/3059290081] conn(0x7fc144073e00 0x7fc144076240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:34:35.854 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:35.793+0000 7f40fa213640 1 --2- 172.21.15.43:0/2754910972 >> [v2:172.21.15.203:6832/830097474,v1:172.21.15.203:6833/830097474] conn(0x7f40d806a120 0x7f40d806c5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:34:35.854 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:35.793+0000 7fdaaa7fc640 1 --2- 172.21.15.43:0/1068176335 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fdaa4053b30 0x7fdaa403bda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:34:35.854 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:35.793+0000 7fdaaaffd640 1 -- 172.21.15.43:0/1068176335 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fdaa40e33f0 msgr2=0x7fdaa403ab60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:34:35.854 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:35.793+0000 7fdaaaffd640 1 --2- 172.21.15.43:0/1068176335 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fdaa40e33f0 0x7fdaa403ab60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:34:35.854 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:35.793+0000 7fdaaaffd640 1 -- 172.21.15.43:0/1068176335 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fdaa4053b30 msgr2=0x7fdaa403bda0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:34:35.854 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:35.793+0000 7fdaaaffd640 1 --2- 172.21.15.43:0/1068176335 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fdaa4053b30 0x7fdaa403bda0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:34:35.855 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:35.793+0000 7fdaaaffd640 1 -- 172.21.15.43:0/1068176335 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdaac072400 con 0x7fdaa4053680 2023-08-09T13:34:35.855 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:35.794+0000 7fdaaa7fc640 1 --2- 172.21.15.43:0/1068176335 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fdaa4053b30 0x7fdaa403bda0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-08-09T13:34:35.855 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:35.794+0000 7fc154ff9640 1 -- 172.21.15.43:0/1890653340 --> [v2:172.21.15.203:6816/3059290081,v1:172.21.15.203:6817/3059290081] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7fc144076910 con 0x7fc144073e00 2023-08-09T13:34:35.855 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:35.794+0000 7fc154ff9640 1 -- 172.21.15.43:0/1890653340 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_get_version_reply(handle=1 version=24) v2 ==== 24+0+0 (secure 0 0 0) 0x7fc14c0ab4f0 con 0x7fc1580cc0c0 2023-08-09T13:34:35.856 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:35.794+0000 7fdaaaffd640 1 --2- 172.21.15.43:0/1068176335 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fdaa4053680 0x7fdaa404d0f0 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7fdaa0001eb0 tx=0x7fdaa000c960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-09T13:34:35.856 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:35.794+0000 7fc1577fe640 1 --2- 172.21.15.43:0/1890653340 >> [v2:172.21.15.203:6816/3059290081,v1:172.21.15.203:6817/3059290081] conn(0x7fc144073e00 0x7fc144076240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:34:35.856 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:35.794+0000 7f40f9a12640 1 --2- 172.21.15.43:0/2754910972 >> [v2:172.21.15.203:6824/512155569,v1:172.21.15.203:6825/512155569] conn(0x7f40d8073e00 0x7f40d8076240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:34:35.857 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:35.795+0000 7fc157fff640 1 --2- 172.21.15.43:0/1890653340 >> [v2:172.21.15.203:6832/830097474,v1:172.21.15.203:6833/830097474] conn(0x7fc14406a120 0x7fc14406c5e0 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7fc150001560 tx=0x7fc150009040 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-09T13:34:35.857 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:35.795+0000 7fc1577fe640 1 --2- 172.21.15.43:0/1890653340 >> [v2:172.21.15.203:6816/3059290081,v1:172.21.15.203:6817/3059290081] conn(0x7fc144073e00 0x7fc144076240 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-09T13:34:35.857 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:35.795+0000 7f40f9a12640 1 --2- 172.21.15.43:0/2754910972 >> [v2:172.21.15.203:6824/512155569,v1:172.21.15.203:6825/512155569] conn(0x7f40d8073e00 0x7f40d8076240 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-09T13:34:35.857 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:35.794+0000 7fda93fff640 1 -- 172.21.15.43:0/1068176335 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdaa0005ce0 con 0x7fdaa4053680 2023-08-09T13:34:35.858 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:35.794+0000 7fda93fff640 1 -- 172.21.15.43:0/1068176335 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdaa0005e40 con 0x7fdaa4053680 2023-08-09T13:34:35.858 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:35.794+0000 7fda93fff640 1 -- 172.21.15.43:0/1068176335 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdaa0007270 con 0x7fdaa4053680 2023-08-09T13:34:35.858 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:35.794+0000 7fdaabfff640 1 -- 172.21.15.43:0/1068176335 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdaa403a080 con 0x7fdaa4053680 2023-08-09T13:34:35.858 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:35.796+0000 7f40fa213640 1 --2- 172.21.15.43:0/2754910972 >> [v2:172.21.15.203:6832/830097474,v1:172.21.15.203:6833/830097474] conn(0x7f40d806a120 0x7f40d806c5e0 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7f40f0007a60 tx=0x7f40f0008040 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-09T13:34:35.859 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:35.796+0000 7fdaabfff640 1 -- 172.21.15.43:0/1068176335 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdaa404d630 con 0x7fdaa4053680 2023-08-09T13:34:35.859 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:35.798+0000 7fda93fff640 1 -- 172.21.15.43:0/1068176335 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 5) v1 ==== 88341+0+0 (secure 0 0 0) 0x7fdaa0007490 con 0x7fdaa4053680 2023-08-09T13:34:35.859 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:35.798+0000 7fc17a1ab640 1 -- 172.21.15.43:0/49380364 --> [v2:172.21.15.43:6800/502667335,v1:172.21.15.43:6801/502667335] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7fc158002cb0 con 0x7fc154073e00 2023-08-09T13:34:35.859 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:35.798+0000 7fdaabfff640 1 -- 172.21.15.43:0/1068176335 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7fdaa4051730 con 0x7fdaa4053680 2023-08-09T13:34:35.860 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:35.799+0000 7fc16bfff640 1 -- 172.21.15.43:0/49380364 <== osd.0 v2:172.21.15.43:6800/502667335 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+11 (crc 0 0 0) 0x7fc158002cb0 con 0x7fc154073e00 2023-08-09T13:34:35.860 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:35.799+0000 7fc154ff9640 1 -- 172.21.15.43:0/1890653340 <== osd.5 v2:172.21.15.203:6816/3059290081 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+25607 (crc 0 0 0) 0x7fc144076910 con 0x7fc144073e00 2023-08-09T13:34:35.860 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:35.799+0000 7fda93fff640 1 --2- 172.21.15.43:0/1068176335 >> [v2:172.21.15.203:6832/830097474,v1:172.21.15.203:6833/830097474] conn(0x7fda9806a0a0 0x7fda9806c560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:34:35.861 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:35.799+0000 7fc17a1ab640 1 -- 172.21.15.43:0/49380364 >> [v2:172.21.15.43:6800/502667335,v1:172.21.15.43:6801/502667335] conn(0x7fc154073e00 msgr2=0x7fc154076240 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:34:35.861 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:35.799+0000 7fc17a1ab640 1 --2- 172.21.15.43:0/49380364 >> [v2:172.21.15.43:6800/502667335,v1:172.21.15.43:6801/502667335] conn(0x7fc154073e00 0x7fc154076240 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:34:35.861 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:35.799+0000 7f40eaffd640 1 -- 172.21.15.43:0/2754910972 <== osd.6 v2:172.21.15.203:6824/512155569 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+25607 (crc 0 0 0) 0x7f40d8076910 con 0x7f40d8073e00 2023-08-09T13:34:35.862 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:35.799+0000 7fda93fff640 1 -- 172.21.15.43:0/1068176335 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6813+0+0 (secure 0 0 0) 0x7fdaa0087be0 con 0x7fdaa4053680 2023-08-09T13:34:35.862 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:35.800+0000 7fda93fff640 1 --2- 172.21.15.43:0/1068176335 >> [v2:172.21.15.203:6808/3331450438,v1:172.21.15.203:6809/3331450438] conn(0x7fda98073d80 0x7fda980761c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:34:35.862 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:35.800+0000 7fda93fff640 1 -- 172.21.15.43:0/1068176335 --> [v2:172.21.15.203:6808/3331450438,v1:172.21.15.203:6809/3331450438] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7fda98076890 con 0x7fda98073d80 2023-08-09T13:34:35.862 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:35.800+0000 7fda93fff640 1 -- 172.21.15.43:0/1068176335 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_get_version_reply(handle=1 version=24) v2 ==== 24+0+0 (secure 0 0 0) 0x7fdaa008a270 con 0x7fdaa4053680 2023-08-09T13:34:35.862 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:35.800+0000 7fdaaa7fc640 1 --2- 172.21.15.43:0/1068176335 >> [v2:172.21.15.203:6832/830097474,v1:172.21.15.203:6833/830097474] conn(0x7fda9806a0a0 0x7fda9806c560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:34:35.863 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:35.800+0000 7fc17a1ab640 1 -- 172.21.15.43:0/49380364 >> [v2:172.21.15.203:6832/830097474,v1:172.21.15.203:6833/830097474] conn(0x7fc15406a120 msgr2=0x7fc15406c5e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:34:35.863 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:35.800+0000 7fc17a1ab640 1 --2- 172.21.15.43:0/49380364 >> [v2:172.21.15.203:6832/830097474,v1:172.21.15.203:6833/830097474] conn(0x7fc15406a120 0x7fc15406c5e0 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7fc164001b80 tx=0x7fc164001af0 comp rx=0 tx=0).stop 2023-08-09T13:34:35.863 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:35.800+0000 7fc17a1ab640 1 -- 172.21.15.43:0/49380364 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc1740b3560 msgr2=0x7fc1740b0aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:34:35.863 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:35.800+0000 7fc17a1ab640 1 --2- 172.21.15.43:0/49380364 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc1740b3560 0x7fc1740b0aa0 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7fc17406ec60 tx=0x7fc16c00f580 comp rx=0 tx=0).stop 2023-08-09T13:34:35.863 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:35.800+0000 7fc173fff640 1 -- 172.21.15.43:0/49380364 reap_dead start 2023-08-09T13:34:35.863 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:35.800+0000 7fc17a1ab640 1 -- 172.21.15.43:0/49380364 shutdown_connections 2023-08-09T13:34:35.864 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:35.800+0000 7fdaab7fe640 1 --2- 172.21.15.43:0/1068176335 >> [v2:172.21.15.203:6808/3331450438,v1:172.21.15.203:6809/3331450438] conn(0x7fda98073d80 0x7fda980761c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:34:35.864 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:35.801+0000 7fc17a1ab640 1 -- 172.21.15.43:0/49380364 >> 172.21.15.43:0/49380364 conn(0x7fc1740cde80 msgr2=0x7fc1740cbe60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-09T13:34:35.864 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:35.801+0000 7fc17a1ab640 1 -- 172.21.15.43:0/49380364 shutdown_connections 2023-08-09T13:34:35.864 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:35.801+0000 7fdaaa7fc640 1 --2- 172.21.15.43:0/1068176335 >> [v2:172.21.15.203:6832/830097474,v1:172.21.15.203:6833/830097474] conn(0x7fda9806a0a0 0x7fda9806c560 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7fdaa40dd1c0 tx=0x7fda9c009040 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-09T13:34:35.865 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:35.801+0000 7fc17a1ab640 1 -- 172.21.15.43:0/49380364 wait complete. 2023-08-09T13:34:35.865 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:35.802+0000 7fdaab7fe640 1 --2- 172.21.15.43:0/1068176335 >> [v2:172.21.15.203:6808/3331450438,v1:172.21.15.203:6809/3331450438] conn(0x7fda98073d80 0x7fda980761c0 crc :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-09T13:34:35.865 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:35.804+0000 7fda93fff640 1 -- 172.21.15.43:0/1068176335 <== osd.7 v2:172.21.15.203:6808/3331450438 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+25607 (crc 0 0 0) 0x7fda98076890 con 0x7fda98073d80 2023-08-09T13:34:35.866 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:35.806+0000 7f276e015640 1 -- 172.21.15.43:0/904747059 --> [v2:172.21.15.43:6808/3289134610,v1:172.21.15.43:6809/3289134610] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7f2748002cb0 con 0x7f274c073d30 2023-08-09T13:34:35.866 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:35.806+0000 7f275dffb640 1 -- 172.21.15.43:0/904747059 <== osd.1 v2:172.21.15.43:6808/3289134610 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+11 (crc 0 0 0) 0x7f2748002cb0 con 0x7f274c073d30 2023-08-09T13:34:35.867 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:35.808+0000 7f27537fe640 1 -- 172.21.15.43:0/904747059 >> [v2:172.21.15.43:6808/3289134610,v1:172.21.15.43:6809/3289134610] conn(0x7f274c073d30 msgr2=0x7f274c076170 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:34:35.867 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:35.808+0000 7f27537fe640 1 --2- 172.21.15.43:0/904747059 >> [v2:172.21.15.43:6808/3289134610,v1:172.21.15.43:6809/3289134610] conn(0x7f274c073d30 0x7f274c076170 crc :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:34:35.867 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:35.809+0000 7f27537fe640 1 -- 172.21.15.43:0/904747059 >> [v2:172.21.15.203:6832/830097474,v1:172.21.15.203:6833/830097474] conn(0x7f274c06a050 msgr2=0x7f274c06c510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:34:35.868 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:35.809+0000 7f27537fe640 1 --2- 172.21.15.43:0/904747059 >> [v2:172.21.15.203:6832/830097474,v1:172.21.15.203:6833/830097474] conn(0x7f274c06a050 0x7f274c06c510 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7f2760009a40 tx=0x7f276000b040 comp rx=0 tx=0).stop 2023-08-09T13:34:35.868 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:35.809+0000 7f27537fe640 1 -- 172.21.15.43:0/904747059 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f2768149a30 msgr2=0x7f27680c4260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:34:35.868 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:35.809+0000 7f27537fe640 1 --2- 172.21.15.43:0/904747059 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f2768149a30 0x7f27680c4260 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f276400c390 tx=0x7f276400c850 comp rx=0 tx=0).stop 2023-08-09T13:34:35.868 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:35.809+0000 7f276d814640 1 -- 172.21.15.43:0/904747059 reap_dead start 2023-08-09T13:34:35.868 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:35.810+0000 7f27537fe640 1 -- 172.21.15.43:0/904747059 shutdown_connections 2023-08-09T13:34:35.868 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:35.810+0000 7f27537fe640 1 -- 172.21.15.43:0/904747059 >> 172.21.15.43:0/904747059 conn(0x7f27680af950 msgr2=0x7f2768152a60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-09T13:34:35.868 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:35.810+0000 7f27537fe640 1 -- 172.21.15.43:0/904747059 shutdown_connections 2023-08-09T13:34:35.868 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:35.810+0000 7f27537fe640 1 -- 172.21.15.43:0/904747059 wait complete. 2023-08-09T13:34:35.869 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:35.819+0000 7f7b54999640 1 -- 172.21.15.43:0/4169296907 --> [v2:172.21.15.43:6811/4079709736,v1:172.21.15.43:6814/4079709736] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7f7b4804fb60 con 0x7f7b30073ce0 2023-08-09T13:34:35.869 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:35.820+0000 7f7b3effd640 1 -- 172.21.15.43:0/4169296907 <== osd.2 v2:172.21.15.43:6811/4079709736 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+11 (crc 0 0 0) 0x7f7b5004ea30 con 0x7f7b30073ce0 2023-08-09T13:34:35.869 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:35.821+0000 7f7b54999640 1 -- 172.21.15.43:0/4169296907 >> [v2:172.21.15.43:6811/4079709736,v1:172.21.15.43:6814/4079709736] conn(0x7f7b30073ce0 msgr2=0x7f7b30076120 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:34:35.870 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:35.821+0000 7f7b54999640 1 --2- 172.21.15.43:0/4169296907 >> [v2:172.21.15.43:6811/4079709736,v1:172.21.15.43:6814/4079709736] conn(0x7f7b30073ce0 0x7f7b30076120 crc :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:34:35.870 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:35.822+0000 7fb8e15fe640 1 -- 172.21.15.43:0/3122861890 --> [v2:172.21.15.43:6813/1709538759,v1:172.21.15.43:6816/1709538759] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7fb8d404f430 con 0x7fb8c0073c40 2023-08-09T13:34:35.870 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:35.823+0000 7f7b54999640 1 -- 172.21.15.43:0/4169296907 >> [v2:172.21.15.203:6832/830097474,v1:172.21.15.203:6833/830097474] conn(0x7f7b3006a080 msgr2=0x7f7b3006c540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:34:35.871 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:35.823+0000 7f7b54999640 1 --2- 172.21.15.43:0/4169296907 >> [v2:172.21.15.203:6832/830097474,v1:172.21.15.203:6833/830097474] conn(0x7f7b3006a080 0x7f7b3006c540 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7f7b400134d0 tx=0x7f7b40008040 comp rx=0 tx=0).stop 2023-08-09T13:34:35.871 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:35.823+0000 7f7b54999640 1 -- 172.21.15.43:0/4169296907 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f7b48181120 msgr2=0x7f7b481807f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:34:35.871 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:35.823+0000 7f7b54999640 1 --2- 172.21.15.43:0/4169296907 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f7b48181120 0x7f7b481807f0 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f7b44004a60 tx=0x7f7b44004f20 comp rx=0 tx=0).stop 2023-08-09T13:34:35.871 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:35.823+0000 7f7b4ed9d640 1 -- 172.21.15.43:0/4169296907 reap_dead start 2023-08-09T13:34:35.871 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:35.823+0000 7f7b54999640 1 -- 172.21.15.43:0/4169296907 shutdown_connections 2023-08-09T13:34:35.871 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:35.823+0000 7f7b54999640 1 -- 172.21.15.43:0/4169296907 >> 172.21.15.43:0/4169296907 conn(0x7f7b48009fe0 msgr2=0x7f7b48039d10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-09T13:34:35.872 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:35.824+0000 7fb8d8ff9640 1 -- 172.21.15.43:0/3122861890 <== osd.3 v2:172.21.15.43:6813/1709538759 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+11 (crc 0 0 0) 0x7fb8dc04ea30 con 0x7fb8c0073c40 2023-08-09T13:34:35.872 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:35.825+0000 7f7b54999640 1 -- 172.21.15.43:0/4169296907 shutdown_connections 2023-08-09T13:34:35.872 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:35.825+0000 7f7b54999640 1 -- 172.21.15.43:0/4169296907 wait complete. 2023-08-09T13:34:35.873 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:35.825+0000 7f7d190ad640 1 -- 172.21.15.43:0/6031396 --> [v2:172.21.15.203:6800/4196866481,v1:172.21.15.203:6801/4196866481] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7f7d14070760 con 0x7f7d00073e00 2023-08-09T13:34:35.873 INFO:teuthology.orchestra.run.smithi043.stdout:55834574858 2023-08-09T13:34:35.873 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:35.826+0000 7fb8e15fe640 1 -- 172.21.15.43:0/3122861890 >> [v2:172.21.15.43:6813/1709538759,v1:172.21.15.43:6816/1709538759] conn(0x7fb8c0073c40 msgr2=0x7fb8c0076080 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:34:35.874 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:35.826+0000 7fb8e15fe640 1 --2- 172.21.15.43:0/3122861890 >> [v2:172.21.15.43:6813/1709538759,v1:172.21.15.43:6816/1709538759] conn(0x7fb8c0073c40 0x7fb8c0076080 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:34:35.874 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2023-08-09T13:34:35.875 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:35.826+0000 7fb8e15fe640 1 -- 172.21.15.43:0/3122861890 >> [v2:172.21.15.203:6832/830097474,v1:172.21.15.203:6833/830097474] conn(0x7fb8c006a0d0 msgr2=0x7fb8c006c590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:34:35.875 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:35.826+0000 7fb8e15fe640 1 --2- 172.21.15.43:0/3122861890 >> [v2:172.21.15.203:6832/830097474,v1:172.21.15.203:6833/830097474] conn(0x7fb8c006a0d0 0x7fb8c006c590 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7fb8d0036000 tx=0x7fb8d0008030 comp rx=0 tx=0).stop 2023-08-09T13:34:35.875 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:35.826+0000 7fb8e15fe640 1 -- 172.21.15.43:0/3122861890 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb8d4006880 msgr2=0x7fb8d404a320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:34:35.875 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:35.826+0000 7fb8e15fe640 1 --2- 172.21.15.43:0/3122861890 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb8d4006880 0x7fb8d404a320 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7fb8c800b410 tx=0x7fb8c800b810 comp rx=0 tx=0).stop 2023-08-09T13:34:35.875 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:35.826+0000 7fb8e0dfd640 1 -- 172.21.15.43:0/3122861890 reap_dead start 2023-08-09T13:34:35.876 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:35.826+0000 7f7cfb7fe640 1 -- 172.21.15.43:0/6031396 <== osd.4 v2:172.21.15.203:6800/4196866481 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+11 (crc 0 0 0) 0x7f7d0c034440 con 0x7f7d00073e00 2023-08-09T13:34:35.876 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:35.827+0000 7fb8e15fe640 1 -- 172.21.15.43:0/3122861890 shutdown_connections 2023-08-09T13:34:35.876 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:35.827+0000 7fb8e15fe640 1 -- 172.21.15.43:0/3122861890 >> 172.21.15.43:0/3122861890 conn(0x7fb8d4009fe0 msgr2=0x7fb8d4038fc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-09T13:34:35.877 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:35.827+0000 7fb8e15fe640 1 -- 172.21.15.43:0/3122861890 shutdown_connections 2023-08-09T13:34:35.877 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:35.827+0000 7fb8e15fe640 1 -- 172.21.15.43:0/3122861890 wait complete. 2023-08-09T13:34:35.877 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:35.827+0000 7f7d190ad640 1 -- 172.21.15.43:0/6031396 >> [v2:172.21.15.203:6800/4196866481,v1:172.21.15.203:6801/4196866481] conn(0x7f7d00073e00 msgr2=0x7f7d00076240 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:34:35.877 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:35.827+0000 7f7d190ad640 1 --2- 172.21.15.43:0/6031396 >> [v2:172.21.15.203:6800/4196866481,v1:172.21.15.203:6801/4196866481] conn(0x7f7d00073e00 0x7f7d00076240 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:34:35.877 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:35.828+0000 7f7d190ad640 1 -- 172.21.15.43:0/6031396 >> [v2:172.21.15.203:6832/830097474,v1:172.21.15.203:6833/830097474] conn(0x7f7d0006a120 msgr2=0x7f7d0006c5e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:34:35.877 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:35.828+0000 7f7d190ad640 1 --2- 172.21.15.43:0/6031396 >> [v2:172.21.15.203:6832/830097474,v1:172.21.15.203:6833/830097474] conn(0x7f7d0006a120 0x7f7d0006c5e0 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7f7d08000f60 tx=0x7f7d080079b0 comp rx=0 tx=0).stop 2023-08-09T13:34:35.878 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:35.828+0000 7f7d190ad640 1 -- 172.21.15.43:0/6031396 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f7d1406f690 msgr2=0x7f7d141f3520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:34:35.878 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:35.828+0000 7f7d190ad640 1 --2- 172.21.15.43:0/6031396 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f7d1406f690 0x7f7d141f3520 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7f7d0400c530 tx=0x7f7d0400c9f0 comp rx=0 tx=0).stop 2023-08-09T13:34:35.878 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:35.828+0000 7f7d1359e640 1 -- 172.21.15.43:0/6031396 reap_dead start 2023-08-09T13:34:35.878 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:35.828+0000 7f7d190ad640 1 -- 172.21.15.43:0/6031396 shutdown_connections 2023-08-09T13:34:35.878 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:35.828+0000 7f7d190ad640 1 -- 172.21.15.43:0/6031396 >> 172.21.15.43:0/6031396 conn(0x7f7d140af970 msgr2=0x7f7d1415b420 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-09T13:34:35.878 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:35.828+0000 7f7d190ad640 1 -- 172.21.15.43:0/6031396 shutdown_connections 2023-08-09T13:34:35.878 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:35.829+0000 7f40fc477640 1 -- 172.21.15.43:0/2754910972 --> [v2:172.21.15.203:6824/512155569,v1:172.21.15.203:6825/512155569] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7f40dc002cb0 con 0x7f40d8073e00 2023-08-09T13:34:35.879 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:35.830+0000 7f7d190ad640 1 -- 172.21.15.43:0/6031396 wait complete. 2023-08-09T13:34:35.879 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:35.831+0000 7f40eaffd640 1 -- 172.21.15.43:0/2754910972 <== osd.6 v2:172.21.15.203:6824/512155569 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+11 (crc 0 0 0) 0x7f40dc002cb0 con 0x7f40d8073e00 2023-08-09T13:34:35.880 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:35.831+0000 7fc15e5c8640 1 -- 172.21.15.43:0/1890653340 --> [v2:172.21.15.203:6816/3059290081,v1:172.21.15.203:6817/3059290081] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7fc1580c3e20 con 0x7fc144073e00 2023-08-09T13:34:35.880 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:35.831+0000 7f40fc477640 1 -- 172.21.15.43:0/2754910972 >> [v2:172.21.15.203:6824/512155569,v1:172.21.15.203:6825/512155569] conn(0x7f40d8073e00 msgr2=0x7f40d8076240 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:34:35.880 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:35.831+0000 7f40fc477640 1 --2- 172.21.15.43:0/2754910972 >> [v2:172.21.15.203:6824/512155569,v1:172.21.15.203:6825/512155569] conn(0x7f40d8073e00 0x7f40d8076240 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:34:35.881 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:35.831+0000 7f40fc477640 1 -- 172.21.15.43:0/2754910972 >> [v2:172.21.15.203:6832/830097474,v1:172.21.15.203:6833/830097474] conn(0x7f40d806a120 msgr2=0x7f40d806c5e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:34:35.881 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:35.832+0000 7f40fc477640 1 --2- 172.21.15.43:0/2754910972 >> [v2:172.21.15.203:6832/830097474,v1:172.21.15.203:6833/830097474] conn(0x7f40d806a120 0x7f40d806c5e0 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7f40f0007a60 tx=0x7f40f0008040 comp rx=0 tx=0).stop 2023-08-09T13:34:35.881 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:35.832+0000 7f40fc477640 1 -- 172.21.15.43:0/2754910972 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f40ec0dffd0 msgr2=0x7f40ec0d5b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:34:35.881 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:35.832+0000 7f40fc477640 1 --2- 172.21.15.43:0/2754910972 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f40ec0dffd0 0x7f40ec0d5b50 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f40f4063820 tx=0x7f40f40721b0 comp rx=0 tx=0).stop 2023-08-09T13:34:35.881 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:35.832+0000 7f40faa14640 1 -- 172.21.15.43:0/2754910972 reap_dead start 2023-08-09T13:34:35.881 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:35.832+0000 7f40fc477640 1 -- 172.21.15.43:0/2754910972 shutdown_connections 2023-08-09T13:34:35.881 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:35.832+0000 7f40fc477640 1 -- 172.21.15.43:0/2754910972 >> 172.21.15.43:0/2754910972 conn(0x7f40ec054c40 msgr2=0x7f40ec006e20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-09T13:34:35.882 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:35.832+0000 7f40fc477640 1 -- 172.21.15.43:0/2754910972 shutdown_connections 2023-08-09T13:34:35.882 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:35.832+0000 7f40fc477640 1 -- 172.21.15.43:0/2754910972 wait complete. 2023-08-09T13:34:35.882 INFO:teuthology.orchestra.run.smithi043.stdout:55834574858 2023-08-09T13:34:35.882 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:35.833+0000 7fc154ff9640 1 -- 172.21.15.43:0/1890653340 <== osd.5 v2:172.21.15.203:6816/3059290081 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+11 (crc 0 0 0) 0x7fc148009280 con 0x7fc144073e00 2023-08-09T13:34:35.883 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2023-08-09T13:34:35.884 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:35.833+0000 7fc15e5c8640 1 -- 172.21.15.43:0/1890653340 >> [v2:172.21.15.203:6816/3059290081,v1:172.21.15.203:6817/3059290081] conn(0x7fc144073e00 msgr2=0x7fc144076240 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:34:35.884 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:35.833+0000 7fc15e5c8640 1 --2- 172.21.15.43:0/1890653340 >> [v2:172.21.15.203:6816/3059290081,v1:172.21.15.203:6817/3059290081] conn(0x7fc144073e00 0x7fc144076240 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:34:35.884 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:35.833+0000 7fc15e5c8640 1 -- 172.21.15.43:0/1890653340 >> [v2:172.21.15.203:6832/830097474,v1:172.21.15.203:6833/830097474] conn(0x7fc14406a120 msgr2=0x7fc14406c5e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:34:35.885 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:35.833+0000 7fc15e5c8640 1 --2- 172.21.15.43:0/1890653340 >> [v2:172.21.15.203:6832/830097474,v1:172.21.15.203:6833/830097474] conn(0x7fc14406a120 0x7fc14406c5e0 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7fc150001560 tx=0x7fc150009040 comp rx=0 tx=0).stop 2023-08-09T13:34:35.885 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:35.834+0000 7fc15e5c8640 1 -- 172.21.15.43:0/1890653340 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc1580cc0c0 msgr2=0x7fc15816bc30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:34:35.885 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:35.834+0000 7fc15e5c8640 1 --2- 172.21.15.43:0/1890653340 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc1580cc0c0 0x7fc15816bc30 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7fc14c036000 tx=0x7fc14c004730 comp rx=0 tx=0).stop 2023-08-09T13:34:35.885 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:35.834+0000 7fc15cb65640 1 -- 172.21.15.43:0/1890653340 reap_dead start 2023-08-09T13:34:35.885 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:35.834+0000 7fc15e5c8640 1 -- 172.21.15.43:0/1890653340 shutdown_connections 2023-08-09T13:34:35.885 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:35.834+0000 7fc15e5c8640 1 -- 172.21.15.43:0/1890653340 >> 172.21.15.43:0/1890653340 conn(0x7fc1580afcb0 msgr2=0x7fc158168c10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-09T13:34:35.885 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:35.834+0000 7fc15e5c8640 1 -- 172.21.15.43:0/1890653340 shutdown_connections 2023-08-09T13:34:35.885 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:35.834+0000 7fc15e5c8640 1 -- 172.21.15.43:0/1890653340 wait complete. 2023-08-09T13:34:35.886 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:35.835+0000 7fdaabfff640 1 -- 172.21.15.43:0/1068176335 --> [v2:172.21.15.203:6808/3331450438,v1:172.21.15.203:6809/3331450438] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7fdaa4051450 con 0x7fda98073d80 2023-08-09T13:34:35.886 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:35.836+0000 7fda93fff640 1 -- 172.21.15.43:0/1068176335 <== osd.7 v2:172.21.15.203:6808/3331450438 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+11 (crc 0 0 0) 0x7fdaac0ad6d0 con 0x7fda98073d80 2023-08-09T13:34:35.887 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:35.837+0000 7fdaabfff640 1 -- 172.21.15.43:0/1068176335 >> [v2:172.21.15.203:6808/3331450438,v1:172.21.15.203:6809/3331450438] conn(0x7fda98073d80 msgr2=0x7fda980761c0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:34:35.887 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:35.837+0000 7fdaabfff640 1 --2- 172.21.15.43:0/1068176335 >> [v2:172.21.15.203:6808/3331450438,v1:172.21.15.203:6809/3331450438] conn(0x7fda98073d80 0x7fda980761c0 crc :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:34:35.887 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:35.837+0000 7fdaabfff640 1 -- 172.21.15.43:0/1068176335 >> [v2:172.21.15.203:6832/830097474,v1:172.21.15.203:6833/830097474] conn(0x7fda9806a0a0 msgr2=0x7fda9806c560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:34:35.887 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:35.837+0000 7fdaabfff640 1 --2- 172.21.15.43:0/1068176335 >> [v2:172.21.15.203:6832/830097474,v1:172.21.15.203:6833/830097474] conn(0x7fda9806a0a0 0x7fda9806c560 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7fdaa40dd1c0 tx=0x7fda9c009040 comp rx=0 tx=0).stop 2023-08-09T13:34:35.887 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:35.837+0000 7fdaabfff640 1 -- 172.21.15.43:0/1068176335 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fdaa4053680 msgr2=0x7fdaa404d0f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:34:35.887 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:35.837+0000 7fdaabfff640 1 --2- 172.21.15.43:0/1068176335 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fdaa4053680 0x7fdaa404d0f0 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7fdaa0001eb0 tx=0x7fdaa000c960 comp rx=0 tx=0).stop 2023-08-09T13:34:35.888 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:35.837+0000 7fdaab7fe640 1 -- 172.21.15.43:0/1068176335 reap_dead start 2023-08-09T13:34:35.888 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:35.837+0000 7fdaabfff640 1 -- 172.21.15.43:0/1068176335 shutdown_connections 2023-08-09T13:34:35.888 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:35.837+0000 7fdaabfff640 1 -- 172.21.15.43:0/1068176335 >> 172.21.15.43:0/1068176335 conn(0x7fdaa40200d0 msgr2=0x7fdaa400c310 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-09T13:34:35.888 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:35.838+0000 7fdaabfff640 1 -- 172.21.15.43:0/1068176335 shutdown_connections 2023-08-09T13:34:35.888 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:35.838+0000 7fdaabfff640 1 -- 172.21.15.43:0/1068176335 wait complete. 2023-08-09T13:34:35.889 INFO:teuthology.orchestra.run.smithi043.stdout:55834574858 2023-08-09T13:34:35.889 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2023-08-09T13:34:35.890 INFO:teuthology.orchestra.run.smithi043.stdout:55834574859 2023-08-09T13:34:35.890 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.3 2023-08-09T13:34:35.891 INFO:teuthology.orchestra.run.smithi043.stdout:55834574858 2023-08-09T13:34:35.891 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.4 2023-08-09T13:34:35.892 INFO:teuthology.orchestra.run.smithi043.stdout:55834574858 2023-08-09T13:34:35.892 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.6 2023-08-09T13:34:35.893 INFO:teuthology.orchestra.run.smithi043.stdout:55834574858 2023-08-09T13:34:35.893 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.5 2023-08-09T13:34:35.894 INFO:teuthology.orchestra.run.smithi043.stdout:55834574858 2023-08-09T13:34:35.895 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.7 2023-08-09T13:34:36.227 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:36.225+0000 7f50db59e640 1 Processor -- start 2023-08-09T13:34:36.230 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:36.225+0000 7f50db59e640 1 -- start start 2023-08-09T13:34:36.230 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:36.227+0000 7f50db59e640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f50dc15be00 0x7f50dc15c260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:34:36.230 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:36.227+0000 7f50db59e640 1 -- --> v1:172.21.15.43:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f50dc15c7a0 con 0x7f50dc0b0470 2023-08-09T13:34:36.230 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:36.227+0000 7f50db59e640 1 -- --> v1:172.21.15.203:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f50dc0c98e0 con 0x7f50dc152760 2023-08-09T13:34:36.231 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:36.228+0000 7f50db59e640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f50dc0cbc60 con 0x7f50dc15be00 2023-08-09T13:34:36.231 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:36.228+0000 7f50dad9d640 1 --1- >> v1:172.21.15.43:6789/0 conn(0x7f50dc0b0470 0x7f50dc0cc940 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.43:6789/0 says I am v1:172.21.15.43:35616/0 (socket says 172.21.15.43:35616) 2023-08-09T13:34:36.231 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:36.228+0000 7f50dad9d640 1 -- 172.21.15.43:0/99145978 learned_addr learned my addr 172.21.15.43:0/99145978 (peer_addr_for_me v1:172.21.15.43:0/0) 2023-08-09T13:34:36.232 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:36.229+0000 7f50d9d9b640 1 --2- 172.21.15.43:0/99145978 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f50dc15be00 0x7f50dc15c260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:34:36.232 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:36.229+0000 7f50d959a640 1 -- 172.21.15.43:0/99145978 <== mon.1 v1:172.21.15.203:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 703173978 0 0) 0x7f50dc0c98e0 con 0x7f50dc152760 2023-08-09T13:34:36.232 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:36.229+0000 7f50d959a640 1 -- 172.21.15.43:0/99145978 --> v1:172.21.15.203:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f50c4005730 con 0x7f50dc152760 2023-08-09T13:34:36.232 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:36.229+0000 7f50d959a640 1 -- 172.21.15.43:0/99145978 <== mon.0 v1:172.21.15.43:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3357105001 0 0) 0x7f50dc15c7a0 con 0x7f50dc0b0470 2023-08-09T13:34:36.232 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:36.229+0000 7f50d959a640 1 -- 172.21.15.43:0/99145978 --> v1:172.21.15.43:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f50dc0c98e0 con 0x7f50dc0b0470 2023-08-09T13:34:36.233 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:36.230+0000 7f50d959a640 1 -- 172.21.15.43:0/99145978 <== mon.0 v1:172.21.15.43:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 553236190 0 0) 0x7f50dc0c98e0 con 0x7f50dc0b0470 2023-08-09T13:34:36.233 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:36.230+0000 7f50d959a640 1 -- 172.21.15.43:0/99145978 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f50dc15be00 msgr2=0x7f50dc15c260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-09T13:34:36.233 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:36.230+0000 7f50d959a640 1 --2- 172.21.15.43:0/99145978 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f50dc15be00 0x7f50dc15c260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:34:36.233 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:36.230+0000 7f50d959a640 1 -- 172.21.15.43:0/99145978 >> v1:172.21.15.203:6789/0 conn(0x7f50dc152760 legacy=0x7f50dc0c8e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:34:36.233 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:36.230+0000 7f50d959a640 1 -- 172.21.15.43:0/99145978 --> v1:172.21.15.43:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f50dc0c9bc0 con 0x7f50dc0b0470 2023-08-09T13:34:36.234 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:36.230+0000 7f50d959a640 1 -- 172.21.15.43:0/99145978 <== mon.0 v1:172.21.15.43:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1285150740 0 0) 0x7f50cc001670 con 0x7f50dc0b0470 2023-08-09T13:34:36.234 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:36.230+0000 7f50d959a640 1 -- 172.21.15.43:0/99145978 >> v1:172.21.15.43:6789/0 conn(0x7f50dc0b0470 legacy=0x7f50dc0cc940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:34:36.234 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:36.230+0000 7f50d959a640 1 --2- 172.21.15.43:0/99145978 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f50c4006da0 0x7f50c40071a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:34:36.234 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:36.230+0000 7f50d959a640 1 --2- 172.21.15.43:0/99145978 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f50c4005970 0x7f50c4005df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:34:36.234 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:36.230+0000 7f50d959a640 1 --2- 172.21.15.43:0/99145978 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f50c4006330 0x7f50c400ff90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:34:36.234 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:36.230+0000 7f50d959a640 1 -- 172.21.15.43:0/99145978 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f50dc0cbc60 con 0x7f50c4006330 2023-08-09T13:34:36.234 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:36.230+0000 7f50d959a640 1 -- 172.21.15.43:0/99145978 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f50c40104d0 con 0x7f50c4005970 2023-08-09T13:34:36.234 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:36.230+0000 7f50d959a640 1 -- 172.21.15.43:0/99145978 --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f50c4010640 con 0x7f50c4006da0 2023-08-09T13:34:36.235 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:36.231+0000 7f50da59c640 1 --2- 172.21.15.43:0/99145978 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f50c4006da0 0x7f50c40071a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:34:36.235 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:36.231+0000 7f50dad9d640 1 --2- 172.21.15.43:0/99145978 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f50c4006330 0x7f50c400ff90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:34:36.235 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:36.231+0000 7f8533fff640 1 Processor -- start 2023-08-09T13:34:36.236 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:36.231+0000 7f8533fff640 1 -- start start 2023-08-09T13:34:36.236 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:36.231+0000 7f50d9d9b640 1 --2- 172.21.15.43:0/99145978 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f50c4005970 0x7f50c4005df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:34:36.236 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:36.232+0000 7f50da59c640 1 -- 172.21.15.43:0/99145978 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f50c4005970 msgr2=0x7f50c4005df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-09T13:34:36.236 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:36.232+0000 7f50da59c640 1 --2- 172.21.15.43:0/99145978 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f50c4005970 0x7f50c4005df0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:34:36.237 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:36.232+0000 7f50da59c640 1 -- 172.21.15.43:0/99145978 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f50c4006330 msgr2=0x7f50c400ff90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-09T13:34:36.237 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:36.232+0000 7f50da59c640 1 --2- 172.21.15.43:0/99145978 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f50c4006330 0x7f50c400ff90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:34:36.237 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:36.232+0000 7f50da59c640 1 -- 172.21.15.43:0/99145978 --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f50dc0c9bc0 con 0x7f50c4006da0 2023-08-09T13:34:36.237 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:36.234+0000 7f8533fff640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f852c055730 0x7f852c007580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:34:36.237 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:36.234+0000 7f8533fff640 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7f852c005fe0 0x7f852c054260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:34:36.238 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:36.234+0000 7f8533fff640 1 -- --> v1:172.21.15.203:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f852c0067c0 con 0x7f852c0df290 2023-08-09T13:34:36.238 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:36.234+0000 7f8533fff640 1 -- --> v2:172.21.15.203:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f852c006aa0 con 0x7f852c005fe0 2023-08-09T13:34:36.238 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:36.234+0000 7f8533fff640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f852c006d80 con 0x7f852c055730 2023-08-09T13:34:36.238 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:36.234+0000 7f50da59c640 1 --2- 172.21.15.43:0/99145978 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f50c4006da0 0x7f50c40071a0 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7f50d4001f60 tx=0x7f50c4007a00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=4b368c7327a4c5fd server_cookie=0 in_seq=0 out_seq=0 2023-08-09T13:34:36.239 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:36.234+0000 7f85327fc640 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7f852c005fe0 0x7f852c054260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:34:36.239 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:36.235+0000 7f85327fc640 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7f852c005fe0 0x7f852c054260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.203:3300/0 says I am v2:172.21.15.43:34362/0 (socket says 172.21.15.43:34362) 2023-08-09T13:34:36.239 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:36.235+0000 7f85327fc640 1 -- 172.21.15.43:0/4118271887 learned_addr learned my addr 172.21.15.43:0/4118271887 (peer_addr_for_me v2:172.21.15.43:0/0) 2023-08-09T13:34:36.239 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:36.235+0000 7f8532ffd640 1 --2- 172.21.15.43:0/4118271887 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f852c055730 0x7f852c007580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:34:36.240 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:36.235+0000 7f50d959a640 1 -- 172.21.15.43:0/99145978 <== mon.2 v2:172.21.15.203:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f50d4003b70 con 0x7f50c4006da0 2023-08-09T13:34:36.240 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:36.235+0000 7f50d959a640 1 -- 172.21.15.43:0/99145978 <== mon.2 v2:172.21.15.203:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f50d4003cf0 con 0x7f50c4006da0 2023-08-09T13:34:36.240 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:36.235+0000 7f50d959a640 1 -- 172.21.15.43:0/99145978 <== mon.2 v2:172.21.15.203:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f50d4003ea0 con 0x7f50c4006da0 2023-08-09T13:34:36.240 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:36.235+0000 7f8531ffb640 1 -- 172.21.15.43:0/4118271887 <== mon.1 v1:172.21.15.203:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 803302499 0 0) 0x7f852c0067c0 con 0x7f852c0df290 2023-08-09T13:34:36.241 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:36.236+0000 7f8531ffb640 1 -- 172.21.15.43:0/4118271887 --> v1:172.21.15.203:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f851c003520 con 0x7f852c0df290 2023-08-09T13:34:36.241 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:36.236+0000 7f85327fc640 1 -- 172.21.15.43:0/4118271887 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f852c055730 msgr2=0x7f852c007580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-09T13:34:36.241 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:36.236+0000 7f85327fc640 1 --2- 172.21.15.43:0/4118271887 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f852c055730 0x7f852c007580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:34:36.241 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:36.236+0000 7f85327fc640 1 -- 172.21.15.43:0/4118271887 >> v1:172.21.15.203:6789/0 conn(0x7f852c0df290 legacy=0x7f852c04fc70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:34:36.241 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:36.236+0000 7f85327fc640 1 -- 172.21.15.43:0/4118271887 --> v2:172.21.15.203:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f852c007060 con 0x7f852c005fe0 2023-08-09T13:34:36.242 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:36.236+0000 7f50db59e640 1 -- 172.21.15.43:0/99145978 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f50c4006da0 msgr2=0x7f50c40071a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:34:36.242 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:36.236+0000 7f50db59e640 1 --2- 172.21.15.43:0/99145978 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f50c4006da0 0x7f50c40071a0 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7f50d4001f60 tx=0x7f50c4007a00 comp rx=0 tx=0).stop 2023-08-09T13:34:36.242 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:36.236+0000 7f50dad9d640 1 -- 172.21.15.43:0/99145978 reap_dead start 2023-08-09T13:34:36.242 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:36.236+0000 7f50db59e640 1 -- 172.21.15.43:0/99145978 shutdown_connections 2023-08-09T13:34:36.242 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:36.236+0000 7f50db59e640 1 -- 172.21.15.43:0/99145978 >> 172.21.15.43:0/99145978 conn(0x7f50dc0af950 msgr2=0x7f50dc0afd60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-09T13:34:36.242 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:36.237+0000 7f50db59e640 1 -- 172.21.15.43:0/99145978 shutdown_connections 2023-08-09T13:34:36.243 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:36.237+0000 7f85327fc640 1 --2- 172.21.15.43:0/4118271887 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f852c005fe0 0x7f852c054260 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f8534065230 tx=0x7f85340a5240 comp rx=0 tx=0).ready entity=mon.1 client_cookie=b7833b8ab36a209c server_cookie=0 in_seq=0 out_seq=0 2023-08-09T13:34:36.243 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:36.238+0000 7f8531ffb640 1 -- 172.21.15.43:0/4118271887 <== mon.1 v2:172.21.15.203:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f85340a5e20 con 0x7f852c005fe0 2023-08-09T13:34:36.243 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:36.238+0000 7f8531ffb640 1 -- 172.21.15.43:0/4118271887 <== mon.1 v2:172.21.15.203:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f85340ad440 con 0x7f852c005fe0 2023-08-09T13:34:36.243 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:36.238+0000 7f8531ffb640 1 -- 172.21.15.43:0/4118271887 <== mon.1 v2:172.21.15.203:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f85340ad830 con 0x7f852c005fe0 2023-08-09T13:34:36.243 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:36.239+0000 7f8533fff640 1 -- 172.21.15.43:0/4118271887 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f852c005fe0 msgr2=0x7f852c054260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:34:36.244 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:36.239+0000 7f8533fff640 1 --2- 172.21.15.43:0/4118271887 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f852c005fe0 0x7f852c054260 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f8534065230 tx=0x7f85340a5240 comp rx=0 tx=0).stop 2023-08-09T13:34:36.244 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:36.237+0000 7f50db59e640 1 -- 172.21.15.43:0/99145978 wait complete. 2023-08-09T13:34:36.244 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:36.240+0000 7f50db59e640 1 Processor -- start 2023-08-09T13:34:36.245 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:36.240+0000 7f8533fff640 1 -- 172.21.15.43:0/4118271887 shutdown_connections 2023-08-09T13:34:36.245 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:36.240+0000 7f8533fff640 1 --2- 172.21.15.43:0/4118271887 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f852c005fe0 0x7f852c054260 unknown :-1 s=CLOSED pgs=89 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:34:36.245 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:36.240+0000 7f8533fff640 1 --2- 172.21.15.43:0/4118271887 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f852c055730 0x7f852c007580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:34:36.245 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:36.240+0000 7f8533fff640 1 -- 172.21.15.43:0/4118271887 >> 172.21.15.43:0/4118271887 conn(0x7f852c054c10 msgr2=0x7f852c055020 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-09T13:34:36.245 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:36.240+0000 7f50db59e640 1 -- start start 2023-08-09T13:34:36.246 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:36.240+0000 7f8533fff640 1 -- 172.21.15.43:0/4118271887 shutdown_connections 2023-08-09T13:34:36.246 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:36.240+0000 7f8533fff640 1 -- 172.21.15.43:0/4118271887 wait complete. 2023-08-09T13:34:36.246 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:36.242+0000 7f50db59e640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f50c4006da0 0x7f50dc14b410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:34:36.246 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:36.242+0000 7f50db59e640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f50dc1eda40 0x7f50dc1edec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:34:36.247 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:36.242+0000 7f50db59e640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f50dc06f2d0 0x7f50dc06f780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:34:36.247 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:36.243+0000 7f8533fff640 1 Processor -- start 2023-08-09T13:34:36.247 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:36.243+0000 7f50d9d9b640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f50dc1eda40 0x7f50dc1edec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:34:36.248 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:36.243+0000 7f50d9d9b640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f50dc1eda40 0x7f50dc1edec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.203:3301/0 says I am v2:172.21.15.43:44966/0 (socket says 172.21.15.43:44966) 2023-08-09T13:34:36.248 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:36.243+0000 7f50dad9d640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f50dc06f2d0 0x7f50dc06f780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:34:36.248 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:36.243+0000 7f50dad9d640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f50dc06f2d0 0x7f50dc06f780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:51712/0 (socket says 172.21.15.43:51712) 2023-08-09T13:34:36.248 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:36.243+0000 7f8533fff640 1 -- start start 2023-08-09T13:34:36.249 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:36.243+0000 7f50d9d9b640 1 -- 172.21.15.43:0/203964691 learned_addr learned my addr 172.21.15.43:0/203964691 (peer_addr_for_me v2:172.21.15.43:0/0) 2023-08-09T13:34:36.249 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:36.244+0000 7f50db59e640 1 -- 172.21.15.43:0/203964691 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f50dc06fcc0 con 0x7f50dc06f2d0 2023-08-09T13:34:36.249 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:36.244+0000 7f50db59e640 1 -- 172.21.15.43:0/203964691 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f50dc0b0a10 con 0x7f50c4006da0 2023-08-09T13:34:36.250 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:36.244+0000 7f50db59e640 1 -- 172.21.15.43:0/203964691 --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f50dc14b950 con 0x7f50dc1eda40 2023-08-09T13:34:36.250 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:36.245+0000 7f50da59c640 1 --2- 172.21.15.43:0/203964691 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f50c4006da0 0x7f50dc14b410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:34:36.250 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:36.245+0000 7f50d9d9b640 1 -- 172.21.15.43:0/203964691 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f50c4006da0 msgr2=0x7f50dc14b410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:34:36.250 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:36.245+0000 7f50d9d9b640 1 --2- 172.21.15.43:0/203964691 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f50c4006da0 0x7f50dc14b410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:34:36.250 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:36.245+0000 7f50d9d9b640 1 -- 172.21.15.43:0/203964691 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f50dc06f2d0 msgr2=0x7f50dc06f780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:34:36.250 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:36.245+0000 7f50d9d9b640 1 --2- 172.21.15.43:0/203964691 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f50dc06f2d0 0x7f50dc06f780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:34:36.250 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:36.245+0000 7f50d9d9b640 1 -- 172.21.15.43:0/203964691 --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f50dc1ee4f0 con 0x7f50dc1eda40 2023-08-09T13:34:36.251 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:36.245+0000 7f8533fff640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f852c005fe0 0x7f852c04e970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:34:36.251 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:36.245+0000 7f8533fff640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f852c055730 0x7f852c0e1ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:34:36.251 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:36.245+0000 7f8533fff640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f852c0df290 0x7f852c0e2410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:34:36.251 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:36.245+0000 7f50dad9d640 1 --2- 172.21.15.43:0/203964691 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f50dc06f2d0 0x7f50dc06f780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-08-09T13:34:36.252 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:36.245+0000 7f50da59c640 1 --2- 172.21.15.43:0/203964691 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f50c4006da0 0x7f50dc14b410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-08-09T13:34:36.252 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:36.245+0000 7f50d9d9b640 1 --2- 172.21.15.43:0/203964691 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f50dc1eda40 0x7f50dc1edec0 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7f50d000f530 tx=0x7f50d000f9f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-09T13:34:36.252 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:36.246+0000 7f8532ffd640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f852c005fe0 0x7f852c04e970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:34:36.252 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:36.246+0000 7f8532ffd640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f852c005fe0 0x7f852c04e970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.203:3301/0 says I am v2:172.21.15.43:44974/0 (socket says 172.21.15.43:44974) 2023-08-09T13:34:36.253 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:36.246+0000 7f85327fc640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f852c055730 0x7f852c0e1ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:34:36.253 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:36.246+0000 7f50caffd640 1 -- 172.21.15.43:0/203964691 <== mon.2 v2:172.21.15.203:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f50d0004020 con 0x7f50dc1eda40 2023-08-09T13:34:36.253 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:36.246+0000 7f50caffd640 1 -- 172.21.15.43:0/203964691 <== mon.2 v2:172.21.15.203:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f50d0004180 con 0x7f50dc1eda40 2023-08-09T13:34:36.253 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:36.246+0000 7f50caffd640 1 -- 172.21.15.43:0/203964691 <== mon.2 v2:172.21.15.203:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f50d0014670 con 0x7f50dc1eda40 2023-08-09T13:34:36.254 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:36.246+0000 7f85337fe640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f852c0df290 0x7f852c0e2410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:34:36.254 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:36.246+0000 7f85337fe640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f852c0df290 0x7f852c0e2410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:51728/0 (socket says 172.21.15.43:51728) 2023-08-09T13:34:36.254 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:36.246+0000 7f85327fc640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f852c055730 0x7f852c0e1ed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.203:3300/0 says I am v2:172.21.15.43:34382/0 (socket says 172.21.15.43:34382) 2023-08-09T13:34:36.254 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:36.246+0000 7f8532ffd640 1 -- 172.21.15.43:0/338620758 learned_addr learned my addr 172.21.15.43:0/338620758 (peer_addr_for_me v2:172.21.15.43:0/0) 2023-08-09T13:34:36.255 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:36.246+0000 7f8533fff640 1 -- 172.21.15.43:0/338620758 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f852c049a60 con 0x7f852c0df290 2023-08-09T13:34:36.255 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:36.246+0000 7f8533fff640 1 -- 172.21.15.43:0/338620758 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f852c04eeb0 con 0x7f852c055730 2023-08-09T13:34:36.255 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:36.246+0000 7f8533fff640 1 -- 172.21.15.43:0/338620758 --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f852c0e0630 con 0x7f852c005fe0 2023-08-09T13:34:36.255 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:36.247+0000 7f8532ffd640 1 -- 172.21.15.43:0/338620758 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f852c055730 msgr2=0x7f852c0e1ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:34:36.255 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:36.247+0000 7f8532ffd640 1 --2- 172.21.15.43:0/338620758 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f852c055730 0x7f852c0e1ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:34:36.255 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:36.247+0000 7f8532ffd640 1 -- 172.21.15.43:0/338620758 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f852c0df290 msgr2=0x7f852c0e2410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:34:36.255 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:36.247+0000 7f8532ffd640 1 --2- 172.21.15.43:0/338620758 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f852c0df290 0x7f852c0e2410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:34:36.255 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:36.247+0000 7f8532ffd640 1 -- 172.21.15.43:0/338620758 --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f852c0e0910 con 0x7f852c005fe0 2023-08-09T13:34:36.256 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:36.248+0000 7f8532ffd640 1 --2- 172.21.15.43:0/338620758 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f852c005fe0 0x7f852c04e970 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7f852800e870 tx=0x7f852800ed30 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-09T13:34:36.256 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:36.246+0000 7f50db59e640 1 -- 172.21.15.43:0/203964691 --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f50dc1ee790 con 0x7f50dc1eda40 2023-08-09T13:34:36.256 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:36.246+0000 7f50db59e640 1 -- 172.21.15.43:0/203964691 --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f50dc1eed50 con 0x7f50dc1eda40 2023-08-09T13:34:36.257 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:36.248+0000 7f851b7fe640 1 -- 172.21.15.43:0/338620758 <== mon.2 v2:172.21.15.203:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f85280094f0 con 0x7f852c005fe0 2023-08-09T13:34:36.257 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:36.248+0000 7f851b7fe640 1 -- 172.21.15.43:0/338620758 <== mon.2 v2:172.21.15.203:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f852800c940 con 0x7f852c005fe0 2023-08-09T13:34:36.257 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:36.248+0000 7f851b7fe640 1 -- 172.21.15.43:0/338620758 <== mon.2 v2:172.21.15.203:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f852800ccf0 con 0x7f852c005fe0 2023-08-09T13:34:36.257 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:36.248+0000 7f8533fff640 1 -- 172.21.15.43:0/338620758 --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f852c0e0bf0 con 0x7f852c005fe0 2023-08-09T13:34:36.257 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:36.248+0000 7f8533fff640 1 -- 172.21.15.43:0/338620758 --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f852c0e11b0 con 0x7f852c005fe0 2023-08-09T13:34:36.258 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:36.248+0000 7f50db59e640 1 -- 172.21.15.43:0/203964691 --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f50dc0739f0 con 0x7f50dc1eda40 2023-08-09T13:34:36.258 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:36.250+0000 7f8533fff640 1 -- 172.21.15.43:0/338620758 --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f852c050d70 con 0x7f852c005fe0 2023-08-09T13:34:36.258 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:36.252+0000 7f50caffd640 1 -- 172.21.15.43:0/203964691 <== mon.2 v2:172.21.15.203:3301/0 4 ==== mgrmap(e 5) v1 ==== 88341+0+0 (secure 0 0 0) 0x7f50d00147d0 con 0x7f50dc1eda40 2023-08-09T13:34:36.259 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:36.253+0000 7f50caffd640 1 --2- 172.21.15.43:0/203964691 >> [v2:172.21.15.203:6832/830097474,v1:172.21.15.203:6833/830097474] conn(0x7f50bc06a150 0x7f50bc06c610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:34:36.259 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:36.254+0000 7f50caffd640 1 -- 172.21.15.43:0/203964691 <== mon.2 v2:172.21.15.203:3301/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6813+0+0 (secure 0 0 0) 0x7f50d0089600 con 0x7f50dc1eda40 2023-08-09T13:34:36.259 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:36.254+0000 7f851b7fe640 1 -- 172.21.15.43:0/338620758 <== mon.2 v2:172.21.15.203:3301/0 4 ==== mgrmap(e 5) v1 ==== 88341+0+0 (secure 0 0 0) 0x7f8528013470 con 0x7f852c005fe0 2023-08-09T13:34:36.259 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:36.254+0000 7fe5e9e26640 1 Processor -- start 2023-08-09T13:34:36.260 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:36.254+0000 7fe5e9e26640 1 -- start start 2023-08-09T13:34:36.260 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:36.255+0000 7f50da59c640 1 --2- 172.21.15.43:0/203964691 >> [v2:172.21.15.203:6832/830097474,v1:172.21.15.203:6833/830097474] conn(0x7f50bc06a150 0x7f50bc06c610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:34:36.260 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:36.255+0000 7f851b7fe640 1 --2- 172.21.15.43:0/338620758 >> [v2:172.21.15.203:6832/830097474,v1:172.21.15.203:6833/830097474] conn(0x7f852006a120 0x7f852006c5e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:34:36.261 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:36.255+0000 7f851b7fe640 1 -- 172.21.15.43:0/338620758 <== mon.2 v2:172.21.15.203:3301/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6813+0+0 (secure 0 0 0) 0x7f85280891d0 con 0x7f852c005fe0 2023-08-09T13:34:36.261 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:36.256+0000 7fe5e9e26640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fe5dc054350 0x7fe5dc0eddb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:34:36.261 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:36.256+0000 7fe5e9e26640 1 -- --> v1:172.21.15.43:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe5dc03c060 con 0x7fe5dc0e3060 2023-08-09T13:34:36.261 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:36.256+0000 7fe5e9e26640 1 -- --> v1:172.21.15.203:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe5dc0070e0 con 0x7fe5dc006b60 2023-08-09T13:34:36.261 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:36.256+0000 7fe5e9e26640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe5dc0073a0 con 0x7fe5dc054350 2023-08-09T13:34:36.262 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:36.256+0000 7f85327fc640 1 --2- 172.21.15.43:0/338620758 >> [v2:172.21.15.203:6832/830097474,v1:172.21.15.203:6833/830097474] conn(0x7f852006a120 0x7f852006c5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:34:36.262 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:36.256+0000 7fe5e37fe640 1 --1- >> v1:172.21.15.43:6789/0 conn(0x7fe5dc0e3060 0x7fe5dc03ea30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.43:6789/0 says I am v1:172.21.15.43:35626/0 (socket says 172.21.15.43:35626) 2023-08-09T13:34:36.262 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:36.256+0000 7fe5e37fe640 1 -- 172.21.15.43:0/490346383 learned_addr learned my addr 172.21.15.43:0/490346383 (peer_addr_for_me v1:172.21.15.43:0/0) 2023-08-09T13:34:36.263 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:36.256+0000 7fe5e3fff640 1 --2- 172.21.15.43:0/490346383 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fe5dc054350 0x7fe5dc0eddb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:34:36.263 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:36.257+0000 7f50da59c640 1 --2- 172.21.15.43:0/203964691 >> [v2:172.21.15.203:6832/830097474,v1:172.21.15.203:6833/830097474] conn(0x7f50bc06a150 0x7f50bc06c610 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7f50d4034000 tx=0x7f50d4009030 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-09T13:34:36.264 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:36.257+0000 7fe5e27fc640 1 -- 172.21.15.43:0/490346383 <== mon.0 v1:172.21.15.43:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 833113774 0 0) 0x7fe5dc03c060 con 0x7fe5dc0e3060 2023-08-09T13:34:36.264 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:36.257+0000 7fe5e27fc640 1 -- 172.21.15.43:0/490346383 --> v1:172.21.15.43:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fe5cc003520 con 0x7fe5dc0e3060 2023-08-09T13:34:36.264 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:36.257+0000 7fe5e27fc640 1 -- 172.21.15.43:0/490346383 <== mon.1 v1:172.21.15.203:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 768267939 0 0) 0x7fe5dc0070e0 con 0x7fe5dc006b60 2023-08-09T13:34:36.264 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:36.257+0000 7fe5e27fc640 1 -- 172.21.15.43:0/490346383 --> v1:172.21.15.203:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fe5dc03c060 con 0x7fe5dc006b60 2023-08-09T13:34:36.264 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:36.257+0000 7fe5e27fc640 1 -- 172.21.15.43:0/490346383 <== mon.0 v1:172.21.15.43:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 809781795 0 0) 0x7fe5cc003520 con 0x7fe5dc0e3060 2023-08-09T13:34:36.264 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:36.257+0000 7fe5e27fc640 1 -- 172.21.15.43:0/490346383 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fe5dc054350 msgr2=0x7fe5dc0eddb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-09T13:34:36.265 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:36.257+0000 7fe5e27fc640 1 --2- 172.21.15.43:0/490346383 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fe5dc054350 0x7fe5dc0eddb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:34:36.265 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:36.257+0000 7fe5e27fc640 1 -- 172.21.15.43:0/490346383 >> v1:172.21.15.203:6789/0 conn(0x7fe5dc006b60 legacy=0x7fe5dc053c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:34:36.265 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:36.257+0000 7fe5e27fc640 1 -- 172.21.15.43:0/490346383 --> v1:172.21.15.43:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe5dc007680 con 0x7fe5dc0e3060 2023-08-09T13:34:36.265 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:36.257+0000 7fe5e27fc640 1 -- 172.21.15.43:0/490346383 <== mon.0 v1:172.21.15.43:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1285150740 0 0) 0x7fe5d8001670 con 0x7fe5dc0e3060 2023-08-09T13:34:36.265 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:36.257+0000 7fe5e27fc640 1 -- 172.21.15.43:0/490346383 >> v1:172.21.15.43:6789/0 conn(0x7fe5dc0e3060 legacy=0x7fe5dc03ea30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:34:36.265 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:36.258+0000 7f85327fc640 1 --2- 172.21.15.43:0/338620758 >> [v2:172.21.15.203:6832/830097474,v1:172.21.15.203:6833/830097474] conn(0x7f852006a120 0x7f852006c5e0 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7f852c055d10 tx=0x7f85340a9000 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-09T13:34:36.266 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:36.258+0000 7fe5e27fc640 1 --2- 172.21.15.43:0/490346383 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fe5cc004b90 0x7fe5cc004fd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:34:36.266 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:36.258+0000 7fe5e27fc640 1 --2- 172.21.15.43:0/490346383 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fe5cc003760 0x7fe5cc003be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:34:36.266 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:36.258+0000 7fe5e27fc640 1 --2- 172.21.15.43:0/490346383 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fe5dc054350 0x7fe5cc004140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:34:36.267 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:36.258+0000 7fe5e27fc640 1 -- 172.21.15.43:0/490346383 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe5dc0073a0 con 0x7fe5cc003760 2023-08-09T13:34:36.267 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:36.258+0000 7fe5e27fc640 1 -- 172.21.15.43:0/490346383 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe5cc00bac0 con 0x7fe5cc004b90 2023-08-09T13:34:36.267 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:36.258+0000 7fe5e27fc640 1 -- 172.21.15.43:0/490346383 --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe5cc00bc80 con 0x7fe5dc054350 2023-08-09T13:34:36.267 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:36.258+0000 7fe5e2ffd640 1 --2- 172.21.15.43:0/490346383 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fe5cc003760 0x7fe5cc003be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:34:36.267 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:36.258+0000 7fe5e2ffd640 1 -- 172.21.15.43:0/490346383 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fe5dc054350 msgr2=0x7fe5cc004140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-09T13:34:36.267 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:36.258+0000 7fe5e2ffd640 1 --2- 172.21.15.43:0/490346383 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fe5dc054350 0x7fe5cc004140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:34:36.267 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:36.258+0000 7fe5e2ffd640 1 -- 172.21.15.43:0/490346383 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fe5cc004b90 msgr2=0x7fe5cc004fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-09T13:34:36.268 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:36.258+0000 7fe5e2ffd640 1 --2- 172.21.15.43:0/490346383 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fe5cc004b90 0x7fe5cc004fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:34:36.268 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:36.259+0000 7fe5e2ffd640 1 -- 172.21.15.43:0/490346383 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe5dc007680 con 0x7fe5cc003760 2023-08-09T13:34:36.268 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:36.260+0000 7fe5e2ffd640 1 --2- 172.21.15.43:0/490346383 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fe5cc003760 0x7fe5cc003be0 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7fe5d40018d0 tx=0x7fe5cc0057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=28f3312448724db5 server_cookie=0 in_seq=0 out_seq=0 2023-08-09T13:34:36.268 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:36.261+0000 7fe5e27fc640 1 -- 172.21.15.43:0/490346383 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe5d40052d0 con 0x7fe5cc003760 2023-08-09T13:34:36.268 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:36.261+0000 7fe5e27fc640 1 -- 172.21.15.43:0/490346383 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe5d402e070 con 0x7fe5cc003760 2023-08-09T13:34:36.268 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:36.261+0000 7fe5e27fc640 1 -- 172.21.15.43:0/490346383 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe5d4003810 con 0x7fe5cc003760 2023-08-09T13:34:36.269 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:36.261+0000 7f33948ad640 1 Processor -- start 2023-08-09T13:34:36.269 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:36.261+0000 7fe5e9e26640 1 -- 172.21.15.43:0/490346383 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fe5cc003760 msgr2=0x7fe5cc003be0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:34:36.269 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:36.261+0000 7fe5e9e26640 1 --2- 172.21.15.43:0/490346383 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fe5cc003760 0x7fe5cc003be0 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7fe5d40018d0 tx=0x7fe5cc0057f0 comp rx=0 tx=0).stop 2023-08-09T13:34:36.269 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:36.261+0000 7f33948ad640 1 -- start start 2023-08-09T13:34:36.270 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:36.262+0000 7fe5e3fff640 1 -- 172.21.15.43:0/490346383 reap_dead start 2023-08-09T13:34:36.270 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:36.262+0000 7fe5e9e26640 1 -- 172.21.15.43:0/490346383 shutdown_connections 2023-08-09T13:34:36.270 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:36.262+0000 7fe5e9e26640 1 -- 172.21.15.43:0/490346383 >> 172.21.15.43:0/490346383 conn(0x7fe5dc055a80 msgr2=0x7fe5dc055e60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-09T13:34:36.270 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:36.262+0000 7fe5e9e26640 1 -- 172.21.15.43:0/490346383 shutdown_connections 2023-08-09T13:34:36.271 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:36.263+0000 7f6905871640 1 Processor -- start 2023-08-09T13:34:36.271 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:36.263+0000 7f6905871640 1 -- start start 2023-08-09T13:34:36.271 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:36.263+0000 7f33948ad640 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7f339015c430 0x7f33900cd0a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:34:36.272 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:36.262+0000 7fe5e9e26640 1 -- 172.21.15.43:0/490346383 wait complete. 2023-08-09T13:34:36.272 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:36.264+0000 7fe5e9e26640 1 Processor -- start 2023-08-09T13:34:36.273 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:36.264+0000 7f338ed9d640 1 --1- >> v1:172.21.15.43:6789/0 conn(0x7f33900cf2b0 0x7f33900b51b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.43:6789/0 says I am v1:172.21.15.43:35628/0 (socket says 172.21.15.43:35628) 2023-08-09T13:34:36.273 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:36.264+0000 7f338ed9d640 1 -- 172.21.15.43:0/3326254086 learned_addr learned my addr 172.21.15.43:0/3326254086 (peer_addr_for_me v1:172.21.15.43:0/0) 2023-08-09T13:34:36.273 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:36.264+0000 7f338e59c640 1 --2- 172.21.15.43:0/3326254086 >> v2:172.21.15.203:3300/0 conn(0x7f339015c430 0x7f33900cd0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:34:36.273 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:36.265+0000 7f33948ad640 1 -- 172.21.15.43:0/3326254086 --> v1:172.21.15.43:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f33900b7e80 con 0x7f33900cf2b0 2023-08-09T13:34:36.273 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:36.265+0000 7f1e79fbd640 1 Processor -- start 2023-08-09T13:34:36.274 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:36.265+0000 7f338d59a640 1 -- 172.21.15.43:0/3326254086 <== mon.0 v1:172.21.15.43:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4074680311 0 0) 0x7f33900b7e80 con 0x7f33900cf2b0 2023-08-09T13:34:36.274 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:36.265+0000 7f33948ad640 1 -- 172.21.15.43:0/3326254086 --> v1:172.21.15.203:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f33900d1070 con 0x7f33900cd5e0 2023-08-09T13:34:36.274 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:36.264+0000 7fe5e9e26640 1 -- start start 2023-08-09T13:34:36.275 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:36.265+0000 7fe5e9e26640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fe5cc003760 0x7fe5dc1853d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:34:36.275 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:36.265+0000 7f33948ad640 1 -- 172.21.15.43:0/3326254086 --> v2:172.21.15.203:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f33900d1350 con 0x7f339015c430 2023-08-09T13:34:36.275 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:36.265+0000 7f338d59a640 1 -- 172.21.15.43:0/3326254086 --> v1:172.21.15.43:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f3378003520 con 0x7f33900cf2b0 2023-08-09T13:34:36.275 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:36.266+0000 7f6905871640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f690015c500 0x7f69000d0240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:34:36.276 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:36.266+0000 7fe5e9e26640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fe5dc051090 0x7fe5dc0514f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:34:36.276 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:36.266+0000 7f6905871640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f69000b7f90 0x7f69000b5310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:34:36.277 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:36.266+0000 7fe5e9e26640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fe5dc0cbca0 0x7fe5dc0cc120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:34:36.277 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:36.266+0000 7fe5e9e26640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe5dc051a30 con 0x7fe5cc003760 2023-08-09T13:34:36.277 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:36.266+0000 7fe5e9e26640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe5dc00d8d0 con 0x7fe5dc051090 2023-08-09T13:34:36.277 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:36.266+0000 7f338d59a640 1 -- 172.21.15.43:0/3326254086 <== mon.0 v1:172.21.15.43:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2404410215 0 0) 0x7f3378003520 con 0x7f33900cf2b0 2023-08-09T13:34:36.278 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:36.266+0000 7fe5e37fe640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fe5cc003760 0x7fe5dc1853d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:34:36.278 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:36.266+0000 7f6905871640 1 -- --> v1:172.21.15.203:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f690015c8e0 con 0x7f69000b5850 2023-08-09T13:34:36.278 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:36.266+0000 7fe5e37fe640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fe5cc003760 0x7fe5dc1853d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:51750/0 (socket says 172.21.15.43:51750) 2023-08-09T13:34:36.279 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:36.266+0000 7fe5e37fe640 1 -- 172.21.15.43:0/2009738792 learned_addr learned my addr 172.21.15.43:0/2009738792 (peer_addr_for_me v2:172.21.15.43:0/0) 2023-08-09T13:34:36.279 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:36.266+0000 7fe5e9e26640 1 -- 172.21.15.43:0/2009738792 --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe5dc0cb1c0 con 0x7fe5dc0cbca0 2023-08-09T13:34:36.279 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:36.266+0000 7fe5e2ffd640 1 --2- 172.21.15.43:0/2009738792 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fe5dc051090 0x7fe5dc0514f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:34:36.279 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:36.266+0000 7f68feffd640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f690015c500 0x7f69000d0240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:34:36.279 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:36.266+0000 7f6905871640 1 -- --> v2:172.21.15.43:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f69000d2bd0 con 0x7f69000b7f90 2023-08-09T13:34:36.280 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:36.266+0000 7f68feffd640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f690015c500 0x7f69000d0240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.203:3301/0 says I am v2:172.21.15.43:44998/0 (socket says 172.21.15.43:44998) 2023-08-09T13:34:36.280 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:36.266+0000 7f68feffd640 1 -- 172.21.15.43:0/2751113570 learned_addr learned my addr 172.21.15.43:0/2751113570 (peer_addr_for_me v2:172.21.15.43:0/0) 2023-08-09T13:34:36.280 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:36.266+0000 7f338d59a640 1 -- 172.21.15.43:0/3326254086 >> v2:172.21.15.203:3300/0 conn(0x7f339015c430 msgr2=0x7f33900cd0a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-09T13:34:36.280 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:36.267+0000 7f6905871640 1 -- 172.21.15.43:0/2751113570 --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f69000d0b20 con 0x7f690015c500 2023-08-09T13:34:36.281 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:36.267+0000 7fe5e3fff640 1 --2- 172.21.15.43:0/2009738792 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fe5dc0cbca0 0x7fe5dc0cc120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:34:36.281 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:36.265+0000 7f1e79fbd640 1 -- start start 2023-08-09T13:34:36.281 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:36.266+0000 7f338d59a640 1 --2- 172.21.15.43:0/3326254086 >> v2:172.21.15.203:3300/0 conn(0x7f339015c430 0x7f33900cd0a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:34:36.282 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:36.266+0000 7f338d59a640 1 -- 172.21.15.43:0/3326254086 >> v1:172.21.15.203:6789/0 conn(0x7f33900cd5e0 legacy=0x7f33900ceba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:34:36.282 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:36.267+0000 7f1e79fbd640 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7f1e6c0e3310 0x7f1e6c057180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:34:36.283 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:36.266+0000 7f338d59a640 1 -- 172.21.15.43:0/3326254086 --> v1:172.21.15.43:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f33900d1630 con 0x7f33900cf2b0 2023-08-09T13:34:36.283 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:36.266+0000 7f338d59a640 1 -- 172.21.15.43:0/3326254086 <== mon.0 v1:172.21.15.43:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1285150740 0 0) 0x7f3388001890 con 0x7f33900cf2b0 2023-08-09T13:34:36.283 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:36.266+0000 7f338d59a640 1 -- 172.21.15.43:0/3326254086 >> v1:172.21.15.43:6789/0 conn(0x7f33900cf2b0 legacy=0x7f33900b51b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:34:36.283 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:36.266+0000 7f338d59a640 1 --2- 172.21.15.43:0/3326254086 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f3378003970 0x7f3378003d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:34:36.283 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:36.266+0000 7f338d59a640 1 --2- 172.21.15.43:0/3326254086 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f33780047c0 0x7f3378004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:34:36.283 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:36.266+0000 7f338d59a640 1 --2- 172.21.15.43:0/3326254086 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f3378005180 0x7f337800b980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:34:36.283 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:36.266+0000 7f338d59a640 1 -- 172.21.15.43:0/3326254086 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f33900d1350 con 0x7f3378003970 2023-08-09T13:34:36.284 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:36.266+0000 7f338d59a640 1 -- 172.21.15.43:0/3326254086 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f337800c030 con 0x7f3378005180 2023-08-09T13:34:36.284 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:36.266+0000 7f338d59a640 1 -- 172.21.15.43:0/3326254086 --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f337800c1f0 con 0x7f33780047c0 2023-08-09T13:34:36.284 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:36.267+0000 7f338ed9d640 1 --2- 172.21.15.43:0/3326254086 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f3378005180 0x7f337800b980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:34:36.284 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:36.267+0000 7fe5e2ffd640 1 -- 172.21.15.43:0/2009738792 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fe5dc0cbca0 msgr2=0x7fe5dc0cc120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:34:36.284 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:36.267+0000 7fe5e2ffd640 1 --2- 172.21.15.43:0/2009738792 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fe5dc0cbca0 0x7fe5dc0cc120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:34:36.285 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:36.267+0000 7fe5e2ffd640 1 -- 172.21.15.43:0/2009738792 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fe5cc003760 msgr2=0x7fe5dc1853d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:34:36.285 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:36.267+0000 7fe5e2ffd640 1 --2- 172.21.15.43:0/2009738792 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fe5cc003760 0x7fe5dc1853d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:34:36.285 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:36.267+0000 7fe5e2ffd640 1 -- 172.21.15.43:0/2009738792 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe5dc0cb4a0 con 0x7fe5dc051090 2023-08-09T13:34:36.285 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:36.267+0000 7f68fe7fc640 1 --2- 172.21.15.43:0/2751113570 >> v2:172.21.15.43:3300/0 conn(0x7f69000b7f90 0x7f69000b5310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:34:36.286 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:36.267+0000 7fe5e3fff640 1 --2- 172.21.15.43:0/2009738792 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fe5dc0cbca0 0x7fe5dc0cc120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-08-09T13:34:36.286 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:36.268+0000 7fe5e37fe640 1 --2- 172.21.15.43:0/2009738792 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fe5cc003760 0x7fe5dc1853d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-08-09T13:34:36.286 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:36.267+0000 7f68feffd640 1 -- 172.21.15.43:0/2751113570 >> v2:172.21.15.43:3300/0 conn(0x7f69000b7f90 msgr2=0x7f69000b5310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-09T13:34:36.286 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:36.267+0000 7f68feffd640 1 --2- 172.21.15.43:0/2751113570 >> v2:172.21.15.43:3300/0 conn(0x7f69000b7f90 0x7f69000b5310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:34:36.286 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:36.267+0000 7f68feffd640 1 -- 172.21.15.43:0/2751113570 >> v1:172.21.15.203:6789/0 conn(0x7f69000b5850 legacy=0x7f69000b3d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:34:36.287 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:36.267+0000 7f68feffd640 1 -- 172.21.15.43:0/2751113570 --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f69000d0e00 con 0x7f690015c500 2023-08-09T13:34:36.287 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:36.267+0000 7f1e79fbd640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f1e6c054570 0x7f1e6c03c1c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:34:36.287 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:36.267+0000 7f1e79fbd640 1 -- --> v1:172.21.15.203:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1e6c0e36f0 con 0x7f1e6c0576c0 2023-08-09T13:34:36.287 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:36.267+0000 7f1e79fbd640 1 -- --> v2:172.21.15.43:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1e6c057c40 con 0x7f1e6c054570 2023-08-09T13:34:36.287 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:36.267+0000 7f1e79fbd640 1 -- --> v2:172.21.15.203:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1e6c057f20 con 0x7f1e6c0e3310 2023-08-09T13:34:36.288 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:36.268+0000 7f1e797bc640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f1e6c054570 0x7f1e6c03c1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:34:36.288 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:36.268+0000 7f1e78fbb640 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7f1e6c0e3310 0x7f1e6c057180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:34:36.288 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:36.268+0000 7f1e797bc640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f1e6c054570 0x7f1e6c03c1c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:51772/0 (socket says 172.21.15.43:51772) 2023-08-09T13:34:36.288 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:36.268+0000 7f1e797bc640 1 -- 172.21.15.43:0/2623443260 learned_addr learned my addr 172.21.15.43:0/2623443260 (peer_addr_for_me v2:172.21.15.43:0/0) 2023-08-09T13:34:36.288 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:36.268+0000 7f1e78fbb640 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7f1e6c0e3310 0x7f1e6c057180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.203:3300/0 says I am v2:172.21.15.43:34416/0 (socket says 172.21.15.43:34416) 2023-08-09T13:34:36.288 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:36.268+0000 7f338e59c640 1 --2- 172.21.15.43:0/3326254086 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f3378003970 0x7f3378003d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:34:36.289 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:36.268+0000 7fe5e2ffd640 1 --2- 172.21.15.43:0/2009738792 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fe5dc051090 0x7fe5dc0514f0 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7fe5d4004000 tx=0x7fe5d40012d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-09T13:34:36.289 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:36.268+0000 7f338ed9d640 1 -- 172.21.15.43:0/3326254086 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f33780047c0 msgr2=0x7f3378004c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-09T13:34:36.290 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:36.268+0000 7f338ed9d640 1 --2- 172.21.15.43:0/3326254086 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f33780047c0 0x7f3378004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:34:36.290 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:36.268+0000 7f338ed9d640 1 -- 172.21.15.43:0/3326254086 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f3378003970 msgr2=0x7f3378003d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-09T13:34:36.290 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:36.268+0000 7f338ed9d640 1 --2- 172.21.15.43:0/3326254086 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f3378003970 0x7f3378003d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:34:36.290 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:36.268+0000 7f338ed9d640 1 -- 172.21.15.43:0/3326254086 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f33900d1630 con 0x7f3378005180 2023-08-09T13:34:36.290 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:36.269+0000 7f1e797bc640 1 -- 172.21.15.43:0/2623443260 >> v2:172.21.15.203:3300/0 conn(0x7f1e6c0e3310 msgr2=0x7f1e6c057180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-09T13:34:36.290 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:36.269+0000 7f1e797bc640 1 --2- 172.21.15.43:0/2623443260 >> v2:172.21.15.203:3300/0 conn(0x7f1e6c0e3310 0x7f1e6c057180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:34:36.291 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:36.269+0000 7f1e797bc640 1 -- 172.21.15.43:0/2623443260 >> v1:172.21.15.203:6789/0 conn(0x7f1e6c0576c0 legacy=0x7f1e6c053e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-09T13:34:36.291 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:36.269+0000 7f1e797bc640 1 -- 172.21.15.43:0/2623443260 --> v2:172.21.15.43:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1e6c058200 con 0x7f1e6c054570 2023-08-09T13:34:36.291 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:36.269+0000 7f50caffd640 1 -- 172.21.15.43:0/203964691 <== mon.2 v2:172.21.15.203:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178066 (secure 0 0 0) 0x7f50d00549e0 con 0x7f50dc1eda40 2023-08-09T13:34:36.292 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:36.269+0000 7fe5cbfff640 1 -- 172.21.15.43:0/2009738792 <== mon.1 v2:172.21.15.203:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe5d4007040 con 0x7fe5dc051090 2023-08-09T13:34:36.292 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:36.269+0000 7fe5cbfff640 1 -- 172.21.15.43:0/2009738792 <== mon.1 v2:172.21.15.203:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe5d40071a0 con 0x7fe5dc051090 2023-08-09T13:34:36.292 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:36.269+0000 7fe5cbfff640 1 -- 172.21.15.43:0/2009738792 <== mon.1 v2:172.21.15.203:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe5d4007550 con 0x7fe5dc051090 2023-08-09T13:34:36.292 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:36.269+0000 7fe5e9e26640 1 -- 172.21.15.43:0/2009738792 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe5dc0cb780 con 0x7fe5dc051090 2023-08-09T13:34:36.292 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:36.269+0000 7fe5e9e26640 1 -- 172.21.15.43:0/2009738792 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe5dc0cdd20 con 0x7fe5dc051090 2023-08-09T13:34:36.293 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:36.270+0000 7f338ed9d640 1 -- 172.21.15.43:0/3326254086 reap_dead start 2023-08-09T13:34:36.293 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:36.270+0000 7f1e78fbb640 1 --2- 172.21.15.43:0/2623443260 >> v2:172.21.15.203:3300/0 conn(0x7f1e6c0e3310 0x7f1e6c057180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-08-09T13:34:36.293 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:36.270+0000 7f68feffd640 1 --2- 172.21.15.43:0/2751113570 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f690015c500 0x7f69000d0240 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7f68f0009870 tx=0x7f68f002ba10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=20e9d13da9a99956 server_cookie=0 in_seq=0 out_seq=0 2023-08-09T13:34:36.294 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:36.270+0000 7f1e797bc640 1 --2- 172.21.15.43:0/2623443260 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f1e6c054570 0x7f1e6c03c1c0 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7f1e6c011ef0 tx=0x7f1e6802b9c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=59752258cc70db16 server_cookie=0 in_seq=0 out_seq=0 2023-08-09T13:34:36.294 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:36.271+0000 7f338ed9d640 1 --2- 172.21.15.43:0/3326254086 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f3378005180 0x7f337800b980 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7f33900cf6c0 tx=0x7f33780057f0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=ebbe36997af018d2 server_cookie=0 in_seq=0 out_seq=0 2023-08-09T13:34:36.294 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:36.271+0000 7f1e737fe640 1 -- 172.21.15.43:0/2623443260 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1e68030880 con 0x7f1e6c054570 2023-08-09T13:34:36.295 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:36.271+0000 7f1e737fe640 1 -- 172.21.15.43:0/2623443260 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1e680309e0 con 0x7f1e6c054570 2023-08-09T13:34:36.295 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:36.271+0000 7f1e737fe640 1 -- 172.21.15.43:0/2623443260 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1e68030db0 con 0x7f1e6c054570 2023-08-09T13:34:36.295 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:36.271+0000 7fe5e9e26640 1 -- 172.21.15.43:0/2009738792 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe5c4005250 con 0x7fe5dc051090 2023-08-09T13:34:36.295 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:36.271+0000 7f338d59a640 1 -- 172.21.15.43:0/3326254086 <== mon.1 v2:172.21.15.203:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3388034630 con 0x7f3378005180 2023-08-09T13:34:36.296 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:36.271+0000 7f338d59a640 1 -- 172.21.15.43:0/3326254086 <== mon.1 v2:172.21.15.203:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f338802f070 con 0x7f3378005180 2023-08-09T13:34:36.296 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:36.271+0000 7f338d59a640 1 -- 172.21.15.43:0/3326254086 <== mon.1 v2:172.21.15.203:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f33880349d0 con 0x7f3378005180 2023-08-09T13:34:36.296 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:36.272+0000 7f1e79fbd640 1 -- 172.21.15.43:0/2623443260 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f1e6c054570 msgr2=0x7f1e6c03c1c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:34:36.296 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:36.272+0000 7f1e79fbd640 1 --2- 172.21.15.43:0/2623443260 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f1e6c054570 0x7f1e6c03c1c0 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7f1e6c011ef0 tx=0x7f1e6802b9c0 comp rx=0 tx=0).stop 2023-08-09T13:34:36.297 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:36.272+0000 7f33948ad640 1 -- 172.21.15.43:0/3326254086 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f3378005180 msgr2=0x7f337800b980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:34:36.297 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:36.272+0000 7f33948ad640 1 --2- 172.21.15.43:0/3326254086 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f3378005180 0x7f337800b980 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7f33900cf6c0 tx=0x7f33780057f0 comp rx=0 tx=0).stop 2023-08-09T13:34:36.297 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:36.272+0000 7f1e79fbd640 1 -- 172.21.15.43:0/2623443260 shutdown_connections 2023-08-09T13:34:36.297 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:36.272+0000 7f1e79fbd640 1 --2- 172.21.15.43:0/2623443260 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f1e6c054570 0x7f1e6c03c1c0 unknown :-1 s=CLOSED pgs=170 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:34:36.298 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:36.272+0000 7f1e79fbd640 1 --2- 172.21.15.43:0/2623443260 >> v2:172.21.15.203:3300/0 conn(0x7f1e6c0e3310 0x7f1e6c057180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:34:36.298 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:36.272+0000 7f1e79fbd640 1 -- 172.21.15.43:0/2623443260 >> 172.21.15.43:0/2623443260 conn(0x7f1e6c03ab80 msgr2=0x7f1e6c03af90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-09T13:34:36.298 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:36.272+0000 7f1e79fbd640 1 -- 172.21.15.43:0/2623443260 shutdown_connections 2023-08-09T13:34:36.298 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:36.272+0000 7f1e79fbd640 1 -- 172.21.15.43:0/2623443260 wait complete. 2023-08-09T13:34:36.298 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:36.272+0000 7f33948ad640 1 -- 172.21.15.43:0/3326254086 shutdown_connections 2023-08-09T13:34:36.299 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:36.272+0000 7f33948ad640 1 --2- 172.21.15.43:0/3326254086 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f3378005180 0x7f337800b980 unknown :-1 s=CLOSED pgs=94 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:34:36.299 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:36.272+0000 7f33948ad640 1 -- 172.21.15.43:0/3326254086 >> 172.21.15.43:0/3326254086 conn(0x7f339006a2b0 msgr2=0x7f339006a6c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-09T13:34:36.299 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:36.272+0000 7f33948ad640 1 -- 172.21.15.43:0/3326254086 shutdown_connections 2023-08-09T13:34:36.299 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:36.273+0000 7f33948ad640 1 -- 172.21.15.43:0/3326254086 wait complete. 2023-08-09T13:34:36.299 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:36.274+0000 7f1e79fbd640 1 Processor -- start 2023-08-09T13:34:36.300 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:36.274+0000 7f1e79fbd640 1 -- start start 2023-08-09T13:34:36.300 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:36.274+0000 7fe5cbfff640 1 -- 172.21.15.43:0/2009738792 <== mon.1 v2:172.21.15.203:3300/0 4 ==== mgrmap(e 5) v1 ==== 88341+0+0 (secure 0 0 0) 0x7fe5d40035f0 con 0x7fe5dc051090 2023-08-09T13:34:36.300 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:36.275+0000 7f33948ad640 1 Processor -- start 2023-08-09T13:34:36.300 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:36.275+0000 7f33948ad640 1 -- start start 2023-08-09T13:34:36.301 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:36.275+0000 7fb0e89fc640 1 Processor -- start 2023-08-09T13:34:36.301 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:36.276+0000 7fb0e89fc640 1 -- start start 2023-08-09T13:34:36.301 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:36.276+0000 7f1e79fbd640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f1e6c054570 0x7f1e6c0e39c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:34:36.302 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:36.276+0000 7f1e79fbd640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f1e6c0576c0 0x7f1e6c03d480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:34:36.302 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:36.276+0000 7fe5cbfff640 1 --2- 172.21.15.43:0/2009738792 >> [v2:172.21.15.203:6832/830097474,v1:172.21.15.203:6833/830097474] conn(0x7fe5d006a0a0 0x7fe5d006c560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:34:36.302 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:36.276+0000 7fe5cbfff640 1 -- 172.21.15.43:0/2009738792 <== mon.1 v2:172.21.15.203:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6813+0+0 (secure 0 0 0) 0x7fe5d40aabf0 con 0x7fe5dc051090 2023-08-09T13:34:36.302 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:36.276+0000 7f1e79fbd640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f1e6c0e3310 0x7f1e6c0d30f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:34:36.303 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:36.276+0000 7f1e79fbd640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1e6c051220 con 0x7f1e6c0e3310 2023-08-09T13:34:36.303 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:36.277+0000 7fe5e37fe640 1 --2- 172.21.15.43:0/2009738792 >> [v2:172.21.15.203:6832/830097474,v1:172.21.15.203:6833/830097474] conn(0x7fe5d006a0a0 0x7fe5d006c560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:34:36.303 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:36.277+0000 7f1e79fbd640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1e6c17c720 con 0x7f1e6c054570 2023-08-09T13:34:36.304 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:36.277+0000 7f1e79fbd640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1e6c17ca00 con 0x7f1e6c0576c0 2023-08-09T13:34:36.304 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:36.277+0000 7f1e797bc640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f1e6c0e3310 0x7f1e6c0d30f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:34:36.304 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:36.277+0000 7f1e797bc640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f1e6c0e3310 0x7f1e6c0d30f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:51776/0 (socket says 172.21.15.43:51776) 2023-08-09T13:34:36.304 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:36.277+0000 7f1e797bc640 1 -- 172.21.15.43:0/3024525694 learned_addr learned my addr 172.21.15.43:0/3024525694 (peer_addr_for_me v2:172.21.15.43:0/0) 2023-08-09T13:34:36.304 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:36.277+0000 7fb0e89fc640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7fb0e406a970 0x7fb0e40ce860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:34:36.305 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:36.277+0000 7fb0e89fc640 1 -- --> v1:172.21.15.43:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb0e40caba0 con 0x7fb0e406ad50 2023-08-09T13:34:36.305 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:36.277+0000 7fb0e89fc640 1 -- --> v1:172.21.15.203:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb0e41606b0 con 0x7fb0e40cb030 2023-08-09T13:34:36.305 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:36.277+0000 7f1e73fff640 1 --2- 172.21.15.43:0/3024525694 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f1e6c0576c0 0x7f1e6c03d480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:34:36.305 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:36.277+0000 7fb0e89fc640 1 -- --> v2:172.21.15.43:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb0e4160990 con 0x7fb0e406a970 2023-08-09T13:34:36.306 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:36.277+0000 7f33948ad640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f3378005180 0x7f339014e4e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:34:36.306 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:36.277+0000 7f1e797bc640 1 -- 172.21.15.43:0/3024525694 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f1e6c0576c0 msgr2=0x7f1e6c03d480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:34:36.306 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:36.277+0000 7f1e797bc640 1 --2- 172.21.15.43:0/3024525694 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f1e6c0576c0 0x7f1e6c03d480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:34:36.306 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:36.277+0000 7f1e797bc640 1 -- 172.21.15.43:0/3024525694 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f1e6c054570 msgr2=0x7f1e6c0e39c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:34:36.307 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:36.277+0000 7f1e78fbb640 1 --2- 172.21.15.43:0/3024525694 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f1e6c054570 0x7f1e6c0e39c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:34:36.307 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:36.277+0000 7f1e797bc640 1 --2- 172.21.15.43:0/3024525694 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f1e6c054570 0x7f1e6c0e39c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:34:36.307 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:36.277+0000 7f1e797bc640 1 -- 172.21.15.43:0/3024525694 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1e60009400 con 0x7f1e6c0e3310 2023-08-09T13:34:36.307 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:36.277+0000 7fb0e37fe640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7fb0e406a970 0x7fb0e40ce860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:34:36.307 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:36.278+0000 7f1e78fbb640 1 --2- 172.21.15.43:0/3024525694 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f1e6c054570 0x7f1e6c0e39c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-08-09T13:34:36.308 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:36.278+0000 7fb0e37fe640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7fb0e406a970 0x7fb0e40ce860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:51778/0 (socket says 172.21.15.43:51778) 2023-08-09T13:34:36.308 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:36.278+0000 7fb0e37fe640 1 -- 172.21.15.43:0/1412669098 learned_addr learned my addr 172.21.15.43:0/1412669098 (peer_addr_for_me v2:172.21.15.43:0/0) 2023-08-09T13:34:36.308 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:36.278+0000 7f1e797bc640 1 --2- 172.21.15.43:0/3024525694 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f1e6c0e3310 0x7f1e6c0d30f0 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7f1e680090b0 tx=0x7f1e68004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-09T13:34:36.309 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:36.279+0000 7fe5e37fe640 1 --2- 172.21.15.43:0/2009738792 >> [v2:172.21.15.203:6832/830097474,v1:172.21.15.203:6833/830097474] conn(0x7fe5d006a0a0 0x7fe5d006c560 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7fe5d80011e0 tx=0x7fe5d8009040 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-09T13:34:36.309 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:36.279+0000 7f33948ad640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f339014cf80 0x7f33900cb480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:34:36.309 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:36.279+0000 7f33948ad640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f33900b25b0 0x7f33900b60a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:34:36.310 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:36.279+0000 7f33948ad640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f33900cb9c0 con 0x7f33900b25b0 2023-08-09T13:34:36.310 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:36.279+0000 7f33948ad640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f339014d400 con 0x7f339014cf80 2023-08-09T13:34:36.310 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:36.279+0000 7f338e59c640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f3378005180 0x7f339014e4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:34:36.310 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:36.279+0000 7f33948ad640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f339014e050 con 0x7f3378005180 2023-08-09T13:34:36.310 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:36.279+0000 7f338e59c640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f3378005180 0x7f339014e4e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.203:3301/0 says I am v2:172.21.15.43:45038/0 (socket says 172.21.15.43:45038) 2023-08-09T13:34:36.310 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:36.279+0000 7f338e59c640 1 -- 172.21.15.43:0/713296298 learned_addr learned my addr 172.21.15.43:0/713296298 (peer_addr_for_me v2:172.21.15.43:0/0) 2023-08-09T13:34:36.311 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:36.279+0000 7fb0e37fe640 1 -- 172.21.15.43:0/1412669098 >> v1:172.21.15.203:6789/0 conn(0x7fb0e40cb030 legacy=0x7fb0e40703e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:34:36.311 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:36.279+0000 7fb0e37fe640 1 -- 172.21.15.43:0/1412669098 >> v1:172.21.15.43:6789/0 conn(0x7fb0e406ad50 legacy=0x7fb0e415fe80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:34:36.311 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:36.279+0000 7f338ed9d640 1 --2- 172.21.15.43:0/713296298 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f33900b25b0 0x7f33900b60a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:34:36.311 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:36.280+0000 7f338e59c640 1 -- 172.21.15.43:0/713296298 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f339014cf80 msgr2=0x7f33900cb480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:34:36.312 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:36.280+0000 7f338e59c640 1 --2- 172.21.15.43:0/713296298 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f339014cf80 0x7f33900cb480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:34:36.312 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:36.280+0000 7f338e59c640 1 -- 172.21.15.43:0/713296298 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f33900b25b0 msgr2=0x7f33900b60a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:34:36.312 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:36.280+0000 7f338e59c640 1 --2- 172.21.15.43:0/713296298 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f33900b25b0 0x7f33900b60a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:34:36.312 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:36.280+0000 7f338e59c640 1 -- 172.21.15.43:0/713296298 --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f339014d720 con 0x7f3378005180 2023-08-09T13:34:36.312 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:36.279+0000 7f1e717fa640 1 -- 172.21.15.43:0/3024525694 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1e68039c10 con 0x7f1e6c0e3310 2023-08-09T13:34:36.312 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:36.279+0000 7f1e717fa640 1 -- 172.21.15.43:0/3024525694 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1e68039d70 con 0x7f1e6c0e3310 2023-08-09T13:34:36.313 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:36.279+0000 7f1e717fa640 1 -- 172.21.15.43:0/3024525694 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1e68043600 con 0x7f1e6c0e3310 2023-08-09T13:34:36.313 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:36.279+0000 7f1e79fbd640 1 -- 172.21.15.43:0/3024525694 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1e680094d0 con 0x7f1e6c0e3310 2023-08-09T13:34:36.313 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:36.279+0000 7f1e79fbd640 1 -- 172.21.15.43:0/3024525694 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1e6c17cfc0 con 0x7f1e6c0e3310 2023-08-09T13:34:36.313 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:36.280+0000 7fb0e37fe640 1 -- 172.21.15.43:0/1412669098 --> v2:172.21.15.43:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb0e4160c70 con 0x7fb0e406a970 2023-08-09T13:34:36.313 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:36.280+0000 7f338e59c640 1 --2- 172.21.15.43:0/713296298 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f3378005180 0x7f339014e4e0 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7f338000e260 tx=0x7f338000e720 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-09T13:34:36.314 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:36.281+0000 7fb0e37fe640 1 --2- 172.21.15.43:0/1412669098 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb0e406a970 0x7fb0e40ce860 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7fb0d4009870 tx=0x7fb0d402ba20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=97dab33e4ef332e0 server_cookie=0 in_seq=0 out_seq=0 2023-08-09T13:34:36.314 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:36.281+0000 7fb67ff64640 1 Processor -- start 2023-08-09T13:34:36.315 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:36.281+0000 7fb0e27fc640 1 -- 172.21.15.43:0/1412669098 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb0d402f840 con 0x7fb0e406a970 2023-08-09T13:34:36.315 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:36.281+0000 7fb0e27fc640 1 -- 172.21.15.43:0/1412669098 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb0d402f9a0 con 0x7fb0e406a970 2023-08-09T13:34:36.315 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:36.281+0000 7fb0e27fc640 1 -- 172.21.15.43:0/1412669098 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb0d402fd30 con 0x7fb0e406a970 2023-08-09T13:34:36.315 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:36.281+0000 7f1e79fbd640 1 -- 172.21.15.43:0/3024525694 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1e6c17d1b0 con 0x7f1e6c0e3310 2023-08-09T13:34:36.316 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:36.281+0000 7fb67ff64640 1 -- start start 2023-08-09T13:34:36.316 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:36.282+0000 7fb0e89fc640 1 -- 172.21.15.43:0/1412669098 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb0e406a970 msgr2=0x7fb0e40ce860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:34:36.316 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:36.282+0000 7fb0e89fc640 1 --2- 172.21.15.43:0/1412669098 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb0e406a970 0x7fb0e40ce860 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7fb0d4009870 tx=0x7fb0d402ba20 comp rx=0 tx=0).stop 2023-08-09T13:34:36.316 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:36.282+0000 7fb0e89fc640 1 -- 172.21.15.43:0/1412669098 shutdown_connections 2023-08-09T13:34:36.317 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:36.282+0000 7fb0e89fc640 1 --2- 172.21.15.43:0/1412669098 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb0e406a970 0x7fb0e40ce860 unknown :-1 s=CLOSED pgs=173 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:34:36.317 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:36.282+0000 7fb0e89fc640 1 -- 172.21.15.43:0/1412669098 >> 172.21.15.43:0/1412669098 conn(0x7fb0e40cb9e0 msgr2=0x7fb0e40cbdc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-09T13:34:36.317 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:36.283+0000 7f1e717fa640 1 -- 172.21.15.43:0/3024525694 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 5) v1 ==== 88341+0+0 (secure 0 0 0) 0x7f1e68043760 con 0x7f1e6c0e3310 2023-08-09T13:34:36.317 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:36.283+0000 7f851b7fe640 1 -- 172.21.15.43:0/338620758 <== mon.2 v2:172.21.15.203:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178066 (secure 0 0 0) 0x7f85280545d0 con 0x7f852c005fe0 2023-08-09T13:34:36.318 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:36.283+0000 7f1e717fa640 1 --2- 172.21.15.43:0/3024525694 >> [v2:172.21.15.203:6832/830097474,v1:172.21.15.203:6833/830097474] conn(0x7f1e5806a0a0 0x7f1e5806c560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:34:36.318 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:36.284+0000 7f1e717fa640 1 -- 172.21.15.43:0/3024525694 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6813+0+0 (secure 0 0 0) 0x7f1e6803e070 con 0x7f1e6c0e3310 2023-08-09T13:34:36.318 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:36.283+0000 7f68fdffb640 1 -- 172.21.15.43:0/2751113570 <== mon.2 v2:172.21.15.203:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f68f002f830 con 0x7f690015c500 2023-08-09T13:34:36.319 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:36.283+0000 7f68fdffb640 1 -- 172.21.15.43:0/2751113570 <== mon.2 v2:172.21.15.203:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f68f002f990 con 0x7f690015c500 2023-08-09T13:34:36.319 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:36.283+0000 7f68fdffb640 1 -- 172.21.15.43:0/2751113570 <== mon.2 v2:172.21.15.203:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f68f002fd20 con 0x7f690015c500 2023-08-09T13:34:36.319 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:36.284+0000 7f6905871640 1 -- 172.21.15.43:0/2751113570 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f690015c500 msgr2=0x7f69000d0240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:34:36.319 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:36.284+0000 7f6905871640 1 --2- 172.21.15.43:0/2751113570 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f690015c500 0x7f69000d0240 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7f68f0009870 tx=0x7f68f002ba10 comp rx=0 tx=0).stop 2023-08-09T13:34:36.319 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:36.284+0000 7f6905871640 1 -- 172.21.15.43:0/2751113570 shutdown_connections 2023-08-09T13:34:36.319 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:36.284+0000 7f6905871640 1 --2- 172.21.15.43:0/2751113570 >> v2:172.21.15.43:3300/0 conn(0x7f69000b7f90 0x7f69000b5310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:34:36.319 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:36.284+0000 7f6905871640 1 --2- 172.21.15.43:0/2751113570 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f690015c500 0x7f69000d0240 unknown :-1 s=CLOSED pgs=49 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:34:36.319 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:36.284+0000 7f6905871640 1 -- 172.21.15.43:0/2751113570 >> 172.21.15.43:0/2751113570 conn(0x7f69000712e0 msgr2=0x7f69000716f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-09T13:34:36.320 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:36.284+0000 7fb0e89fc640 1 -- 172.21.15.43:0/1412669098 shutdown_connections 2023-08-09T13:34:36.320 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:36.284+0000 7fb0e89fc640 1 -- 172.21.15.43:0/1412669098 wait complete. 2023-08-09T13:34:36.320 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:36.285+0000 7f1e78fbb640 1 --2- 172.21.15.43:0/3024525694 >> [v2:172.21.15.203:6832/830097474,v1:172.21.15.203:6833/830097474] conn(0x7f1e5806a0a0 0x7f1e5806c560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:34:36.321 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:36.285+0000 7fb67ff64640 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7fb67815a510 0x7fb6780cfd40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:34:36.321 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:36.285+0000 7fb67ff64640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fb6780d0280 0x7fb6780caba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:34:36.321 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:36.285+0000 7fb67ff64640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7fb6780cb0e0 0x7fb6780cca60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:34:36.321 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:36.286+0000 7fb67dd00640 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7fb67815a510 0x7fb6780cfd40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:34:36.322 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:36.286+0000 7fb67dd00640 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7fb67815a510 0x7fb6780cfd40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.203:3300/0 says I am v2:172.21.15.43:34446/0 (socket says 172.21.15.43:34446) 2023-08-09T13:34:36.322 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:36.286+0000 7f6905871640 1 -- 172.21.15.43:0/2751113570 shutdown_connections 2023-08-09T13:34:36.322 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:36.286+0000 7f6905871640 1 -- 172.21.15.43:0/2751113570 wait complete. 2023-08-09T13:34:36.322 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:36.287+0000 7fb67dd00640 1 -- 172.21.15.43:0/1302058555 learned_addr learned my addr 172.21.15.43:0/1302058555 (peer_addr_for_me v2:172.21.15.43:0/0) 2023-08-09T13:34:36.323 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:36.287+0000 7fb0e89fc640 1 Processor -- start 2023-08-09T13:34:36.323 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:36.287+0000 7fb67e501640 1 --2- 172.21.15.43:0/1302058555 >> v2:172.21.15.43:3300/0 conn(0x7fb6780cb0e0 0x7fb6780cca60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:34:36.323 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:36.284+0000 7f337effd640 1 -- 172.21.15.43:0/713296298 <== mon.2 v2:172.21.15.203:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f338001c070 con 0x7f3378005180 2023-08-09T13:34:36.324 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:36.284+0000 7f337effd640 1 -- 172.21.15.43:0/713296298 <== mon.2 v2:172.21.15.203:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f338001d480 con 0x7f3378005180 2023-08-09T13:34:36.324 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:36.284+0000 7f337effd640 1 -- 172.21.15.43:0/713296298 <== mon.2 v2:172.21.15.203:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f338001d5f0 con 0x7f3378005180 2023-08-09T13:34:36.324 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:36.284+0000 7f33948ad640 1 -- 172.21.15.43:0/713296298 --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f339014da00 con 0x7f3378005180 2023-08-09T13:34:36.324 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:36.284+0000 7f33948ad640 1 -- 172.21.15.43:0/713296298 --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f33900b2a30 con 0x7f3378005180 2023-08-09T13:34:36.324 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:36.286+0000 7f33948ad640 1 -- 172.21.15.43:0/713296298 --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f339004e390 con 0x7f3378005180 2023-08-09T13:34:36.325 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:36.287+0000 7fb0e89fc640 1 -- start start 2023-08-09T13:34:36.325 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:36.289+0000 7fb0e89fc640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fb0e406a970 0x7fb0e4156cd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:34:36.325 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:36.289+0000 7f1e78fbb640 1 --2- 172.21.15.43:0/3024525694 >> [v2:172.21.15.203:6832/830097474,v1:172.21.15.203:6833/830097474] conn(0x7f1e5806a0a0 0x7f1e5806c560 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7f1e60009bb0 tx=0x7f1e60008040 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-09T13:34:36.325 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:36.289+0000 7fb0e89fc640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fb0e40cb030 0x7fb0e40b1090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:34:36.326 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:36.289+0000 7f337effd640 1 -- 172.21.15.43:0/713296298 <== mon.2 v2:172.21.15.203:3301/0 4 ==== mgrmap(e 5) v1 ==== 88341+0+0 (secure 0 0 0) 0x7f338001d750 con 0x7f3378005180 2023-08-09T13:34:36.326 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:36.289+0000 7fb0e89fc640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb0e4159520 0x7fb0e40c4c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:34:36.326 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:36.289+0000 7fb0e37fe640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fb0e406a970 0x7fb0e4156cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:34:36.327 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:36.290+0000 7fb0e37fe640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fb0e406a970 0x7fb0e4156cd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.203:3301/0 says I am v2:172.21.15.43:45056/0 (socket says 172.21.15.43:45056) 2023-08-09T13:34:36.327 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:36.290+0000 7fb0e2ffd640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fb0e40cb030 0x7fb0e40b1090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:34:36.327 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:36.290+0000 7fb0e2ffd640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fb0e40cb030 0x7fb0e40b1090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.203:3300/0 says I am v2:172.21.15.43:34462/0 (socket says 172.21.15.43:34462) 2023-08-09T13:34:36.327 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:36.288+0000 7f6905871640 1 Processor -- start 2023-08-09T13:34:36.327 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:36.289+0000 7f6905871640 1 -- start start 2023-08-09T13:34:36.328 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:36.290+0000 7fb0e2ffd640 1 -- 172.21.15.43:0/1321082850 learned_addr learned my addr 172.21.15.43:0/1321082850 (peer_addr_for_me v2:172.21.15.43:0/0) 2023-08-09T13:34:36.328 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:36.290+0000 7fb0e89fc640 1 -- 172.21.15.43:0/1321082850 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb0e4157210 con 0x7fb0e4159520 2023-08-09T13:34:36.328 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:36.290+0000 7fb0e89fc640 1 -- 172.21.15.43:0/1321082850 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb0e4159090 con 0x7fb0e40cb030 2023-08-09T13:34:36.328 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:36.290+0000 7fb0e89fc640 1 -- 172.21.15.43:0/1321082850 --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb0e4158480 con 0x7fb0e406a970 2023-08-09T13:34:36.329 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:36.290+0000 7f6905871640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f69000b5850 0x7f690015a6a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:34:36.329 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:36.290+0000 7f6905871640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f69000b7f90 0x7f690006d7d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:34:36.329 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:36.289+0000 7fb67ff64640 1 -- 172.21.15.43:0/1302058555 --> v2:172.21.15.43:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb6780cc120 con 0x7fb6780cb0e0 2023-08-09T13:34:36.329 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:36.289+0000 7fb67ff64640 1 -- 172.21.15.43:0/1302058555 --> v2:172.21.15.203:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb67815a8f0 con 0x7fb67815a510 2023-08-09T13:34:36.329 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:36.289+0000 7fb67ff64640 1 -- 172.21.15.43:0/1302058555 --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb6780c3e80 con 0x7fb6780d0280 2023-08-09T13:34:36.329 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:36.290+0000 7fb67d4ff640 1 --2- 172.21.15.43:0/1302058555 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fb6780d0280 0x7fb6780caba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:34:36.330 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:36.291+0000 7f6905871640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f690015c500 0x7f69000b1790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:34:36.330 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:36.291+0000 7f6905871640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f690006cae0 con 0x7f69000b7f90 2023-08-09T13:34:36.330 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:36.291+0000 7f6905871640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f690006d340 con 0x7f69000b5850 2023-08-09T13:34:36.330 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:36.291+0000 7f6905871640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f690015abe0 con 0x7f690015c500 2023-08-09T13:34:36.331 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:36.291+0000 7fb67d4ff640 1 -- 172.21.15.43:0/1302058555 >> v2:172.21.15.203:3300/0 conn(0x7fb67815a510 msgr2=0x7fb6780cfd40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-09T13:34:36.331 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:36.291+0000 7fb67d4ff640 1 --2- 172.21.15.43:0/1302058555 >> v2:172.21.15.203:3300/0 conn(0x7fb67815a510 0x7fb6780cfd40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:34:36.331 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:36.291+0000 7fb67d4ff640 1 -- 172.21.15.43:0/1302058555 >> v2:172.21.15.43:3300/0 conn(0x7fb6780cb0e0 msgr2=0x7fb6780cca60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-09T13:34:36.331 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:36.291+0000 7fb67d4ff640 1 --2- 172.21.15.43:0/1302058555 >> v2:172.21.15.43:3300/0 conn(0x7fb6780cb0e0 0x7fb6780cca60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:34:36.332 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:36.291+0000 7f68fe7fc640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f69000b7f90 0x7f690006d7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:34:36.332 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:36.291+0000 7f68fe7fc640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f69000b7f90 0x7f690006d7d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:51824/0 (socket says 172.21.15.43:51824) 2023-08-09T13:34:36.332 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:36.291+0000 7f68fe7fc640 1 -- 172.21.15.43:0/404075115 learned_addr learned my addr 172.21.15.43:0/404075115 (peer_addr_for_me v2:172.21.15.43:0/0) 2023-08-09T13:34:36.332 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:36.291+0000 7fb0e2ffd640 1 -- 172.21.15.43:0/1321082850 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fb0e406a970 msgr2=0x7fb0e4156cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:34:36.333 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:36.291+0000 7fb0e2ffd640 1 --2- 172.21.15.43:0/1321082850 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fb0e406a970 0x7fb0e4156cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:34:36.333 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:36.291+0000 7fb0e2ffd640 1 -- 172.21.15.43:0/1321082850 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb0e4159520 msgr2=0x7fb0e40c4c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:34:36.333 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:36.291+0000 7fb0e2ffd640 1 --2- 172.21.15.43:0/1321082850 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb0e4159520 0x7fb0e40c4c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:34:36.333 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:36.291+0000 7fb0e2ffd640 1 -- 172.21.15.43:0/1321082850 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb0d40094d0 con 0x7fb0e40cb030 2023-08-09T13:34:36.333 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:36.292+0000 7fb67d4ff640 1 -- 172.21.15.43:0/1302058555 --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb6780ce000 con 0x7fb6780d0280 2023-08-09T13:34:36.334 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:36.292+0000 7f68feffd640 1 --2- 172.21.15.43:0/404075115 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f69000b5850 0x7f690015a6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:34:36.334 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:36.292+0000 7fe5cbfff640 1 -- 172.21.15.43:0/2009738792 <== mon.1 v2:172.21.15.203:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178066 (secure 0 0 0) 0x7fe5d40760f0 con 0x7fe5dc051090 2023-08-09T13:34:36.334 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:36.292+0000 7f1e717fa640 1 -- 172.21.15.43:0/3024525694 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178066 (secure 0 0 0) 0x7f1e68077970 con 0x7f1e6c0e3310 2023-08-09T13:34:36.334 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:36.292+0000 7fb67e501640 1 --2- 172.21.15.43:0/1302058555 >> v2:172.21.15.43:3300/0 conn(0x7fb6780cb0e0 0x7fb6780cca60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-08-09T13:34:36.335 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:36.292+0000 7f337effd640 1 --2- 172.21.15.43:0/713296298 >> [v2:172.21.15.203:6832/830097474,v1:172.21.15.203:6833/830097474] conn(0x7f337006a150 0x7f337006c610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:34:36.335 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:36.292+0000 7f68ff7fe640 1 --2- 172.21.15.43:0/404075115 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f690015c500 0x7f69000b1790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:34:36.335 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:36.292+0000 7f337effd640 1 -- 172.21.15.43:0/713296298 <== mon.2 v2:172.21.15.203:3301/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6813+0+0 (secure 0 0 0) 0x7f3380090dd0 con 0x7f3378005180 2023-08-09T13:34:36.336 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:36.292+0000 7f68fe7fc640 1 -- 172.21.15.43:0/404075115 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f690015c500 msgr2=0x7f69000b1790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:34:36.336 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:36.292+0000 7f68fe7fc640 1 --2- 172.21.15.43:0/404075115 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f690015c500 0x7f69000b1790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:34:36.336 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:36.292+0000 7f68fe7fc640 1 -- 172.21.15.43:0/404075115 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f69000b5850 msgr2=0x7f690015a6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:34:36.336 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:36.292+0000 7f68fe7fc640 1 --2- 172.21.15.43:0/404075115 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f69000b5850 0x7f690015a6a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:34:36.336 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:36.292+0000 7f68fe7fc640 1 -- 172.21.15.43:0/404075115 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f68f00094d0 con 0x7f69000b7f90 2023-08-09T13:34:36.337 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:36.293+0000 7f338dd9b640 1 --2- 172.21.15.43:0/713296298 >> [v2:172.21.15.203:6832/830097474,v1:172.21.15.203:6833/830097474] conn(0x7f337006a150 0x7f337006c610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:34:36.337 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:36.292+0000 7fb0e2ffd640 1 --2- 172.21.15.43:0/1321082850 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fb0e40cb030 0x7fb0e40b1090 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7fb0d800c450 tx=0x7fb0d800c910 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-09T13:34:36.337 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:36.292+0000 7fb0cbfff640 1 -- 172.21.15.43:0/1321082850 <== mon.1 v2:172.21.15.203:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb0d80052d0 con 0x7fb0e40cb030 2023-08-09T13:34:36.337 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:36.293+0000 7fb0cbfff640 1 -- 172.21.15.43:0/1321082850 <== mon.1 v2:172.21.15.203:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb0d8005430 con 0x7fb0e40cb030 2023-08-09T13:34:36.337 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:36.293+0000 7fb0cbfff640 1 -- 172.21.15.43:0/1321082850 <== mon.1 v2:172.21.15.203:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb0d8012870 con 0x7fb0e40cb030 2023-08-09T13:34:36.338 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:36.293+0000 7fb0e89fc640 1 -- 172.21.15.43:0/1321082850 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb0e4158760 con 0x7fb0e40cb030 2023-08-09T13:34:36.338 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:36.293+0000 7fb0e89fc640 1 -- 172.21.15.43:0/1321082850 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb0e4158d20 con 0x7fb0e40cb030 2023-08-09T13:34:36.338 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:36.293+0000 7f68fe7fc640 1 --2- 172.21.15.43:0/404075115 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f69000b7f90 0x7f690006d7d0 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7f68f400d830 tx=0x7f68f400dcf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-09T13:34:36.338 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:36.293+0000 7fb67d4ff640 1 --2- 172.21.15.43:0/1302058555 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fb6780d0280 0x7fb6780caba0 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7fb67806d4f0 tx=0x7fb66802b840 comp rx=0 tx=0).ready entity=mon.2 client_cookie=73b1a95ddb07ce17 server_cookie=0 in_seq=0 out_seq=0 2023-08-09T13:34:36.339 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:36.294+0000 7f338dd9b640 1 --2- 172.21.15.43:0/713296298 >> [v2:172.21.15.203:6832/830097474,v1:172.21.15.203:6833/830097474] conn(0x7f337006a150 0x7f337006c610 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7f3390157fa0 tx=0x7f3384007990 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-09T13:34:36.339 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:36.294+0000 7f68e7fff640 1 -- 172.21.15.43:0/404075115 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f68f400b6c0 con 0x7f69000b7f90 2023-08-09T13:34:36.339 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:36.294+0000 7f68e7fff640 1 -- 172.21.15.43:0/404075115 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f68f400b820 con 0x7f69000b7f90 2023-08-09T13:34:36.339 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:36.294+0000 7f68e7fff640 1 -- 172.21.15.43:0/404075115 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f68f400bbb0 con 0x7f69000b7f90 2023-08-09T13:34:36.339 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:36.294+0000 7f6905871640 1 -- 172.21.15.43:0/404075115 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f69001ff4f0 con 0x7f69000b7f90 2023-08-09T13:34:36.340 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:36.294+0000 7f6905871640 1 -- 172.21.15.43:0/404075115 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f690006cfd0 con 0x7f69000b7f90 2023-08-09T13:34:36.340 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:36.295+0000 7fb0e89fc640 1 -- 172.21.15.43:0/1321082850 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb0e4158ee0 con 0x7fb0e40cb030 2023-08-09T13:34:36.340 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:36.297+0000 7f68e7fff640 1 -- 172.21.15.43:0/404075115 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 5) v1 ==== 88341+0+0 (secure 0 0 0) 0x7f68f400bd10 con 0x7f69000b7f90 2023-08-09T13:34:36.340 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:36.299+0000 7f68e7fff640 1 --2- 172.21.15.43:0/404075115 >> [v2:172.21.15.203:6832/830097474,v1:172.21.15.203:6833/830097474] conn(0x7f68ec06a0d0 0x7f68ec06c590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:34:36.341 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:36.299+0000 7f68e7fff640 1 -- 172.21.15.43:0/404075115 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6813+0+0 (secure 0 0 0) 0x7f68f4088c80 con 0x7f69000b7f90 2023-08-09T13:34:36.341 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:36.299+0000 7fb0cbfff640 1 -- 172.21.15.43:0/1321082850 <== mon.1 v2:172.21.15.203:3300/0 4 ==== mgrmap(e 5) v1 ==== 88341+0+0 (secure 0 0 0) 0x7fb0d80129d0 con 0x7fb0e40cb030 2023-08-09T13:34:36.341 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:36.300+0000 7f68feffd640 1 --2- 172.21.15.43:0/404075115 >> [v2:172.21.15.203:6832/830097474,v1:172.21.15.203:6833/830097474] conn(0x7f68ec06a0d0 0x7f68ec06c590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:34:36.341 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:36.300+0000 7f6905871640 1 -- 172.21.15.43:0/404075115 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f68e0005250 con 0x7f69000b7f90 2023-08-09T13:34:36.342 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:36.301+0000 7fb0cbfff640 1 --2- 172.21.15.43:0/1321082850 >> [v2:172.21.15.203:6832/830097474,v1:172.21.15.203:6833/830097474] conn(0x7fb0d006a120 0x7fb0d006c5e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:34:36.342 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:36.301+0000 7fb0cbfff640 1 -- 172.21.15.43:0/1321082850 <== mon.1 v2:172.21.15.203:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6813+0+0 (secure 0 0 0) 0x7fb0d8088330 con 0x7fb0e40cb030 2023-08-09T13:34:36.342 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:36.301+0000 7f68feffd640 1 --2- 172.21.15.43:0/404075115 >> [v2:172.21.15.203:6832/830097474,v1:172.21.15.203:6833/830097474] conn(0x7f68ec06a0d0 0x7f68ec06c590 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7f68f0036000 tx=0x7f68f0008030 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-09T13:34:36.343 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:36.301+0000 7fb0e37fe640 1 --2- 172.21.15.43:0/1321082850 >> [v2:172.21.15.203:6832/830097474,v1:172.21.15.203:6833/830097474] conn(0x7fb0d006a120 0x7fb0d006c5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:34:36.343 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:36.303+0000 7fb0e37fe640 1 --2- 172.21.15.43:0/1321082850 >> [v2:172.21.15.203:6832/830097474,v1:172.21.15.203:6833/830097474] conn(0x7fb0d006a120 0x7fb0d006c5e0 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7fb0d4036000 tx=0x7fb0d4008030 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-09T13:34:36.343 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:36.304+0000 7f337effd640 1 -- 172.21.15.43:0/713296298 <== mon.2 v2:172.21.15.203:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178066 (secure 0 0 0) 0x7f338005c260 con 0x7f3378005180 2023-08-09T13:34:36.344 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:36.303+0000 7fb67ccfe640 1 -- 172.21.15.43:0/1302058555 <== mon.2 v2:172.21.15.203:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb66802e070 con 0x7fb6780d0280 2023-08-09T13:34:36.344 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:36.304+0000 7fb67ccfe640 1 -- 172.21.15.43:0/1302058555 <== mon.2 v2:172.21.15.203:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb6680315a0 con 0x7fb6780d0280 2023-08-09T13:34:36.344 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:36.304+0000 7fb67ccfe640 1 -- 172.21.15.43:0/1302058555 <== mon.2 v2:172.21.15.203:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb668031930 con 0x7fb6780d0280 2023-08-09T13:34:36.344 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:36.305+0000 7fb67ff64640 1 -- 172.21.15.43:0/1302058555 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fb6780d0280 msgr2=0x7fb6780caba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:34:36.344 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:36.305+0000 7fb67ff64640 1 --2- 172.21.15.43:0/1302058555 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fb6780d0280 0x7fb6780caba0 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7fb67806d4f0 tx=0x7fb66802b840 comp rx=0 tx=0).stop 2023-08-09T13:34:36.345 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:36.305+0000 7fb67ff64640 1 -- 172.21.15.43:0/1302058555 shutdown_connections 2023-08-09T13:34:36.345 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:36.305+0000 7fb67ff64640 1 --2- 172.21.15.43:0/1302058555 >> v2:172.21.15.43:3300/0 conn(0x7fb6780cb0e0 0x7fb6780cca60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:34:36.345 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:36.305+0000 7fb67ff64640 1 --2- 172.21.15.43:0/1302058555 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fb6780d0280 0x7fb6780caba0 unknown :-1 s=CLOSED pgs=51 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:34:36.345 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:36.305+0000 7fb67ff64640 1 --2- 172.21.15.43:0/1302058555 >> v2:172.21.15.203:3300/0 conn(0x7fb67815a510 0x7fb6780cfd40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:34:36.345 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:36.305+0000 7fb67ff64640 1 -- 172.21.15.43:0/1302058555 >> 172.21.15.43:0/1302058555 conn(0x7fb67806a1f0 msgr2=0x7fb6780994b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-09T13:34:36.345 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:36.305+0000 7fb67ff64640 1 -- 172.21.15.43:0/1302058555 shutdown_connections 2023-08-09T13:34:36.345 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:36.305+0000 7fb67ff64640 1 -- 172.21.15.43:0/1302058555 wait complete. 2023-08-09T13:34:36.346 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:36.307+0000 7f68e7fff640 1 -- 172.21.15.43:0/404075115 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178066 (secure 0 0 0) 0x7f68f4054160 con 0x7f69000b7f90 2023-08-09T13:34:36.347 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:36.308+0000 7fb67ff64640 1 Processor -- start 2023-08-09T13:34:36.347 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:36.308+0000 7fb67ff64640 1 -- start start 2023-08-09T13:34:36.347 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:36.310+0000 7fb67ff64640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fb6780cb0e0 0x7fb67815d520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:34:36.347 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:36.310+0000 7fb67ff64640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb6780d0280 0x7fb67815bce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:34:36.347 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:36.311+0000 7fb67ff64640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fb67815a510 0x7fb6781504f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:34:36.347 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:36.311+0000 7fb67d4ff640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb6780d0280 0x7fb67815bce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:34:36.347 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:36.311+0000 7fb67d4ff640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb6780d0280 0x7fb67815bce0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:51828/0 (socket says 172.21.15.43:51828) 2023-08-09T13:34:36.348 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:36.311+0000 7fb67ff64640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb67815d090 con 0x7fb6780d0280 2023-08-09T13:34:36.348 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:36.311+0000 7fb67ff64640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb67815c370 con 0x7fb6780cb0e0 2023-08-09T13:34:36.348 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:36.311+0000 7fb67ff64640 1 -- 172.21.15.43:0/3604201571 --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb67815c7a0 con 0x7fb67815a510 2023-08-09T13:34:36.348 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:36.311+0000 7fb67d4ff640 1 -- 172.21.15.43:0/3604201571 learned_addr learned my addr 172.21.15.43:0/3604201571 (peer_addr_for_me v2:172.21.15.43:0/0) 2023-08-09T13:34:36.348 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:36.311+0000 7fb67dd00640 1 --2- 172.21.15.43:0/3604201571 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fb6780cb0e0 0x7fb67815d520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:34:36.348 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:36.311+0000 7fb67e501640 1 --2- 172.21.15.43:0/3604201571 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fb67815a510 0x7fb6781504f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:34:36.348 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:36.312+0000 7fb67d4ff640 1 -- 172.21.15.43:0/3604201571 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fb67815a510 msgr2=0x7fb6781504f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:34:36.348 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:36.312+0000 7fb67d4ff640 1 --2- 172.21.15.43:0/3604201571 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fb67815a510 0x7fb6781504f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:34:36.349 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:36.312+0000 7fb67d4ff640 1 -- 172.21.15.43:0/3604201571 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fb6780cb0e0 msgr2=0x7fb67815d520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:34:36.349 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:36.312+0000 7fb67d4ff640 1 --2- 172.21.15.43:0/3604201571 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fb6780cb0e0 0x7fb67815d520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:34:36.349 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:36.312+0000 7fb67d4ff640 1 -- 172.21.15.43:0/3604201571 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb668009400 con 0x7fb6780d0280 2023-08-09T13:34:36.349 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:36.312+0000 7fb0cbfff640 1 -- 172.21.15.43:0/1321082850 <== mon.1 v2:172.21.15.203:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178066 (secure 0 0 0) 0x7fb0d8053710 con 0x7fb0e40cb030 2023-08-09T13:34:36.350 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:36.313+0000 7fb67d4ff640 1 --2- 172.21.15.43:0/3604201571 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb6780d0280 0x7fb67815bce0 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7fb668009890 tx=0x7fb6680046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-09T13:34:36.350 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:36.313+0000 7fb66e7fc640 1 -- 172.21.15.43:0/3604201571 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb668032900 con 0x7fb6780d0280 2023-08-09T13:34:36.350 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:36.313+0000 7fb66e7fc640 1 -- 172.21.15.43:0/3604201571 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb668032a80 con 0x7fb6780d0280 2023-08-09T13:34:36.350 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:36.313+0000 7fb66e7fc640 1 -- 172.21.15.43:0/3604201571 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb668032e30 con 0x7fb6780d0280 2023-08-09T13:34:36.350 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:36.314+0000 7fb67ff64640 1 -- 172.21.15.43:0/3604201571 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb67815ca40 con 0x7fb6780d0280 2023-08-09T13:34:36.350 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:36.317+0000 7fb66e7fc640 1 -- 172.21.15.43:0/3604201571 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 5) v1 ==== 88341+0+0 (secure 0 0 0) 0x7fb66803b3f0 con 0x7fb6780d0280 2023-08-09T13:34:36.351 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:36.319+0000 7fb66e7fc640 1 --2- 172.21.15.43:0/3604201571 >> [v2:172.21.15.203:6832/830097474,v1:172.21.15.203:6833/830097474] conn(0x7fb65c06a0a0 0x7fb65c06c560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:34:36.351 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:36.319+0000 7fb67ff64640 1 -- 172.21.15.43:0/3604201571 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb6780c88b0 con 0x7fb6780d0280 2023-08-09T13:34:36.351 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:36.320+0000 7fb67dd00640 1 --2- 172.21.15.43:0/3604201571 >> [v2:172.21.15.203:6832/830097474,v1:172.21.15.203:6833/830097474] conn(0x7fb65c06a0a0 0x7fb65c06c560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:34:36.351 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:36.320+0000 7fb66e7fc640 1 -- 172.21.15.43:0/3604201571 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6813+0+0 (secure 0 0 0) 0x7fb668079510 con 0x7fb6780d0280 2023-08-09T13:34:36.351 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:36.321+0000 7fb67dd00640 1 --2- 172.21.15.43:0/3604201571 >> [v2:172.21.15.203:6832/830097474,v1:172.21.15.203:6833/830097474] conn(0x7fb65c06a0a0 0x7fb65c06c560 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7fb674005f80 tx=0x7fb674008040 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-09T13:34:36.351 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:36.327+0000 7fb67ff64640 1 -- 172.21.15.43:0/3604201571 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb6780739f0 con 0x7fb6780d0280 2023-08-09T13:34:36.351 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:36.337+0000 7fb66e7fc640 1 -- 172.21.15.43:0/3604201571 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178066 (secure 0 0 0) 0x7fb6680751c0 con 0x7fb6780d0280 2023-08-09T13:34:36.685 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:36.682+0000 7f50db59e640 1 -- 172.21.15.43:0/203964691 --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 0} v 0) v1 -- 0x7f50dc14bb80 con 0x7f50dc1eda40 2023-08-09T13:34:36.686 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:36.683+0000 7f50caffd640 1 -- 172.21.15.43:0/203964691 <== mon.2 v2:172.21.15.203:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 0}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f50d0058000 con 0x7f50dc1eda40 2023-08-09T13:34:36.689 INFO:teuthology.orchestra.run.smithi043.stdout:55834574857 2023-08-09T13:34:36.690 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:36.687+0000 7f50c8ff9640 1 -- 172.21.15.43:0/203964691 >> [v2:172.21.15.203:6832/830097474,v1:172.21.15.203:6833/830097474] conn(0x7f50bc06a150 msgr2=0x7f50bc06c610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:34:36.690 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:36.687+0000 7f50c8ff9640 1 --2- 172.21.15.43:0/203964691 >> [v2:172.21.15.203:6832/830097474,v1:172.21.15.203:6833/830097474] conn(0x7f50bc06a150 0x7f50bc06c610 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7f50d4034000 tx=0x7f50d4009030 comp rx=0 tx=0).stop 2023-08-09T13:34:36.690 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:36.687+0000 7f50c8ff9640 1 -- 172.21.15.43:0/203964691 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f50dc1eda40 msgr2=0x7f50dc1edec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:34:36.690 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:36.687+0000 7f50c8ff9640 1 --2- 172.21.15.43:0/203964691 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f50dc1eda40 0x7f50dc1edec0 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7f50d000f530 tx=0x7f50d000f9f0 comp rx=0 tx=0).stop 2023-08-09T13:34:36.691 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:36.687+0000 7f50c8ff9640 1 -- 172.21.15.43:0/203964691 shutdown_connections 2023-08-09T13:34:36.691 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:36.687+0000 7f50c8ff9640 1 --2- 172.21.15.43:0/203964691 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f50dc06f2d0 0x7f50dc06f780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:34:36.691 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:36.687+0000 7f50c8ff9640 1 --2- 172.21.15.43:0/203964691 >> [v2:172.21.15.203:6832/830097474,v1:172.21.15.203:6833/830097474] conn(0x7f50bc06a150 0x7f50bc06c610 unknown :-1 s=CLOSED pgs=106 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:34:36.691 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:36.687+0000 7f50c8ff9640 1 --2- 172.21.15.43:0/203964691 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f50dc1eda40 0x7f50dc1edec0 unknown :-1 s=CLOSED pgs=47 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:34:36.691 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:36.687+0000 7f50c8ff9640 1 --2- 172.21.15.43:0/203964691 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f50c4006da0 0x7f50dc14b410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:34:36.691 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:36.688+0000 7f50c8ff9640 1 -- 172.21.15.43:0/203964691 >> 172.21.15.43:0/203964691 conn(0x7f50dc0af950 msgr2=0x7f50dc14c620 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-09T13:34:36.692 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:36.688+0000 7f50c8ff9640 1 -- 172.21.15.43:0/203964691 shutdown_connections 2023-08-09T13:34:36.692 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:36.688+0000 7f50c8ff9640 1 -- 172.21.15.43:0/203964691 wait complete. 2023-08-09T13:34:36.714 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:36.711+0000 7f8533fff640 1 -- 172.21.15.43:0/338620758 --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 1} v 0) v1 -- 0x7f852c008b20 con 0x7f852c005fe0 2023-08-09T13:34:36.715 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:36.712+0000 7f851b7fe640 1 -- 172.21.15.43:0/338620758 <== mon.2 v2:172.21.15.203:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 1}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f8528021090 con 0x7f852c005fe0 2023-08-09T13:34:36.715 INFO:teuthology.orchestra.run.smithi043.stdout:55834574857 2023-08-09T13:34:36.719 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574858 got 55834574857 for osd.0 2023-08-09T13:34:36.720 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:36.716+0000 7f8533fff640 1 -- 172.21.15.43:0/338620758 >> [v2:172.21.15.203:6832/830097474,v1:172.21.15.203:6833/830097474] conn(0x7f852006a120 msgr2=0x7f852006c5e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:34:36.720 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:36.716+0000 7f8533fff640 1 --2- 172.21.15.43:0/338620758 >> [v2:172.21.15.203:6832/830097474,v1:172.21.15.203:6833/830097474] conn(0x7f852006a120 0x7f852006c5e0 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7f852c055d10 tx=0x7f85340a9000 comp rx=0 tx=0).stop 2023-08-09T13:34:36.720 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:36.717+0000 7f8533fff640 1 -- 172.21.15.43:0/338620758 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f852c005fe0 msgr2=0x7f852c04e970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:34:36.721 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:36.717+0000 7f8533fff640 1 --2- 172.21.15.43:0/338620758 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f852c005fe0 0x7f852c04e970 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7f852800e870 tx=0x7f852800ed30 comp rx=0 tx=0).stop 2023-08-09T13:34:36.721 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:36.717+0000 7f8533fff640 1 -- 172.21.15.43:0/338620758 shutdown_connections 2023-08-09T13:34:36.721 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:36.717+0000 7f8533fff640 1 --2- 172.21.15.43:0/338620758 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f852c0df290 0x7f852c0e2410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:34:36.721 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:36.717+0000 7f8533fff640 1 --2- 172.21.15.43:0/338620758 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f852c055730 0x7f852c0e1ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:34:36.721 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:36.717+0000 7f8533fff640 1 --2- 172.21.15.43:0/338620758 >> [v2:172.21.15.203:6832/830097474,v1:172.21.15.203:6833/830097474] conn(0x7f852006a120 0x7f852006c5e0 unknown :-1 s=CLOSED pgs=107 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:34:36.721 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:36.717+0000 7f8533fff640 1 --2- 172.21.15.43:0/338620758 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f852c005fe0 0x7f852c04e970 unknown :-1 s=CLOSED pgs=48 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:34:36.721 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:36.717+0000 7f8533fff640 1 -- 172.21.15.43:0/338620758 >> 172.21.15.43:0/338620758 conn(0x7f852c054c10 msgr2=0x7f852c0381e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-09T13:34:36.722 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:36.717+0000 7f8533fff640 1 -- 172.21.15.43:0/338620758 shutdown_connections 2023-08-09T13:34:36.722 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:36.717+0000 7f8533fff640 1 -- 172.21.15.43:0/338620758 wait complete. 2023-08-09T13:34:36.722 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:36.718+0000 7fe5e9e26640 1 -- 172.21.15.43:0/2009738792 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 2} v 0) v1 -- 0x7fe5c40057d0 con 0x7fe5dc051090 2023-08-09T13:34:36.722 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:36.719+0000 7f1e79fbd640 1 -- 172.21.15.43:0/3024525694 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 4} v 0) v1 -- 0x7f1e6c051360 con 0x7f1e6c0e3310 2023-08-09T13:34:36.723 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:36.719+0000 7fe5cbfff640 1 -- 172.21.15.43:0/2009738792 <== mon.1 v2:172.21.15.203:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 2}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7fe5d4037020 con 0x7fe5dc051090 2023-08-09T13:34:36.723 INFO:teuthology.orchestra.run.smithi043.stdout:55834574857 2023-08-09T13:34:36.724 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:36.720+0000 7f1e717fa640 1 -- 172.21.15.43:0/3024525694 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 4}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f1e68034020 con 0x7f1e6c0e3310 2023-08-09T13:34:36.724 INFO:teuthology.orchestra.run.smithi043.stdout:55834574857 2023-08-09T13:34:36.725 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:36.723+0000 7f1e79fbd640 1 -- 172.21.15.43:0/3024525694 >> [v2:172.21.15.203:6832/830097474,v1:172.21.15.203:6833/830097474] conn(0x7f1e5806a0a0 msgr2=0x7f1e5806c560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:34:36.725 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:36.723+0000 7f1e79fbd640 1 --2- 172.21.15.43:0/3024525694 >> [v2:172.21.15.203:6832/830097474,v1:172.21.15.203:6833/830097474] conn(0x7f1e5806a0a0 0x7f1e5806c560 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7f1e60009bb0 tx=0x7f1e60008040 comp rx=0 tx=0).stop 2023-08-09T13:34:36.725 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:36.723+0000 7f1e79fbd640 1 -- 172.21.15.43:0/3024525694 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f1e6c0e3310 msgr2=0x7f1e6c0d30f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:34:36.726 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:36.723+0000 7f1e79fbd640 1 --2- 172.21.15.43:0/3024525694 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f1e6c0e3310 0x7f1e6c0d30f0 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7f1e680090b0 tx=0x7f1e68004730 comp rx=0 tx=0).stop 2023-08-09T13:34:36.726 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:36.723+0000 7f1e79fbd640 1 -- 172.21.15.43:0/3024525694 shutdown_connections 2023-08-09T13:34:36.726 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:36.723+0000 7f1e79fbd640 1 --2- 172.21.15.43:0/3024525694 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f1e6c0e3310 0x7f1e6c0d30f0 unknown :-1 s=CLOSED pgs=171 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:34:36.727 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:36.723+0000 7f1e79fbd640 1 --2- 172.21.15.43:0/3024525694 >> [v2:172.21.15.203:6832/830097474,v1:172.21.15.203:6833/830097474] conn(0x7f1e5806a0a0 0x7f1e5806c560 unknown :-1 s=CLOSED pgs=109 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:34:36.727 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:36.723+0000 7f1e79fbd640 1 --2- 172.21.15.43:0/3024525694 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f1e6c0576c0 0x7f1e6c03d480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:34:36.727 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:36.723+0000 7f1e79fbd640 1 --2- 172.21.15.43:0/3024525694 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f1e6c054570 0x7f1e6c0e39c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:34:36.727 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:36.723+0000 7f1e79fbd640 1 -- 172.21.15.43:0/3024525694 >> 172.21.15.43:0/3024525694 conn(0x7f1e6c03ab80 msgr2=0x7f1e6c0ddbe0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-09T13:34:36.727 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:36.723+0000 7f1e79fbd640 1 -- 172.21.15.43:0/3024525694 shutdown_connections 2023-08-09T13:34:36.727 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:36.723+0000 7fe5e9e26640 1 -- 172.21.15.43:0/2009738792 >> [v2:172.21.15.203:6832/830097474,v1:172.21.15.203:6833/830097474] conn(0x7fe5d006a0a0 msgr2=0x7fe5d006c560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:34:36.728 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:36.723+0000 7f1e79fbd640 1 -- 172.21.15.43:0/3024525694 wait complete. 2023-08-09T13:34:36.729 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:36.723+0000 7fe5e9e26640 1 --2- 172.21.15.43:0/2009738792 >> [v2:172.21.15.203:6832/830097474,v1:172.21.15.203:6833/830097474] conn(0x7fe5d006a0a0 0x7fe5d006c560 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7fe5d80011e0 tx=0x7fe5d8009040 comp rx=0 tx=0).stop 2023-08-09T13:34:36.729 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:36.724+0000 7fe5e9e26640 1 -- 172.21.15.43:0/2009738792 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fe5dc051090 msgr2=0x7fe5dc0514f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:34:36.729 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:36.724+0000 7fe5e9e26640 1 --2- 172.21.15.43:0/2009738792 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fe5dc051090 0x7fe5dc0514f0 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7fe5d4004000 tx=0x7fe5d40012d0 comp rx=0 tx=0).stop 2023-08-09T13:34:36.729 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:36.724+0000 7fe5e9e26640 1 -- 172.21.15.43:0/2009738792 shutdown_connections 2023-08-09T13:34:36.729 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:36.724+0000 7fe5e9e26640 1 --2- 172.21.15.43:0/2009738792 >> [v2:172.21.15.203:6832/830097474,v1:172.21.15.203:6833/830097474] conn(0x7fe5d006a0a0 0x7fe5d006c560 unknown :-1 s=CLOSED pgs=108 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:34:36.729 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:36.724+0000 7fe5e9e26640 1 --2- 172.21.15.43:0/2009738792 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fe5dc0cbca0 0x7fe5dc0cc120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:34:36.730 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:36.724+0000 7fe5e9e26640 1 --2- 172.21.15.43:0/2009738792 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fe5dc051090 0x7fe5dc0514f0 unknown :-1 s=CLOSED pgs=93 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:34:36.730 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:36.724+0000 7fe5e9e26640 1 --2- 172.21.15.43:0/2009738792 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fe5cc003760 0x7fe5dc1853d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:34:36.730 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:36.724+0000 7fe5e9e26640 1 -- 172.21.15.43:0/2009738792 >> 172.21.15.43:0/2009738792 conn(0x7fe5dc055a80 msgr2=0x7fe5dc054950 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-09T13:34:36.730 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:36.724+0000 7fe5e9e26640 1 -- 172.21.15.43:0/2009738792 shutdown_connections 2023-08-09T13:34:36.730 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:36.724+0000 7fe5e9e26640 1 -- 172.21.15.43:0/2009738792 wait complete. 2023-08-09T13:34:36.736 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:36.733+0000 7f33948ad640 1 -- 172.21.15.43:0/713296298 --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 3} v 0) v1 -- 0x7f33901e5e80 con 0x7f3378005180 2023-08-09T13:34:36.738 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:36.734+0000 7f337effd640 1 -- 172.21.15.43:0/713296298 <== mon.2 v2:172.21.15.203:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 3}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f338002f020 con 0x7f3378005180 2023-08-09T13:34:36.738 INFO:teuthology.orchestra.run.smithi043.stdout:55834574858 2023-08-09T13:34:36.740 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:36.737+0000 7f33948ad640 1 -- 172.21.15.43:0/713296298 >> [v2:172.21.15.203:6832/830097474,v1:172.21.15.203:6833/830097474] conn(0x7f337006a150 msgr2=0x7f337006c610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:34:36.740 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:36.737+0000 7f33948ad640 1 --2- 172.21.15.43:0/713296298 >> [v2:172.21.15.203:6832/830097474,v1:172.21.15.203:6833/830097474] conn(0x7f337006a150 0x7f337006c610 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7f3390157fa0 tx=0x7f3384007990 comp rx=0 tx=0).stop 2023-08-09T13:34:36.740 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:36.737+0000 7f33948ad640 1 -- 172.21.15.43:0/713296298 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f3378005180 msgr2=0x7f339014e4e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:34:36.740 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:36.737+0000 7f33948ad640 1 --2- 172.21.15.43:0/713296298 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f3378005180 0x7f339014e4e0 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7f338000e260 tx=0x7f338000e720 comp rx=0 tx=0).stop 2023-08-09T13:34:36.741 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:36.737+0000 7f33948ad640 1 -- 172.21.15.43:0/713296298 shutdown_connections 2023-08-09T13:34:36.741 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:36.737+0000 7f33948ad640 1 --2- 172.21.15.43:0/713296298 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f33900b25b0 0x7f33900b60a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:34:36.741 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:36.737+0000 7f33948ad640 1 --2- 172.21.15.43:0/713296298 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f339014cf80 0x7f33900cb480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:34:36.741 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:36.737+0000 7f33948ad640 1 --2- 172.21.15.43:0/713296298 >> [v2:172.21.15.203:6832/830097474,v1:172.21.15.203:6833/830097474] conn(0x7f337006a150 0x7f337006c610 unknown :-1 s=CLOSED pgs=110 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:34:36.741 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:36.737+0000 7f33948ad640 1 --2- 172.21.15.43:0/713296298 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f3378005180 0x7f339014e4e0 unknown :-1 s=CLOSED pgs=50 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:34:36.741 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:36.737+0000 7f33948ad640 1 -- 172.21.15.43:0/713296298 >> 172.21.15.43:0/713296298 conn(0x7f339006a2b0 msgr2=0x7f33901e6680 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-09T13:34:36.741 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:36.738+0000 7f33948ad640 1 -- 172.21.15.43:0/713296298 shutdown_connections 2023-08-09T13:34:36.742 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:36.738+0000 7f33948ad640 1 -- 172.21.15.43:0/713296298 wait complete. 2023-08-09T13:34:36.745 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:36.743+0000 7f6905871640 1 -- 172.21.15.43:0/404075115 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 6} v 0) v1 -- 0x7f68e00057d0 con 0x7f69000b7f90 2023-08-09T13:34:36.746 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:36.744+0000 7f68e7fff640 1 -- 172.21.15.43:0/404075115 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 6}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f68f4057780 con 0x7f69000b7f90 2023-08-09T13:34:36.746 INFO:teuthology.orchestra.run.smithi043.stdout:55834574857 2023-08-09T13:34:36.751 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574858 got 55834574857 for osd.1 2023-08-09T13:34:36.752 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:36.748+0000 7f6905871640 1 -- 172.21.15.43:0/404075115 >> [v2:172.21.15.203:6832/830097474,v1:172.21.15.203:6833/830097474] conn(0x7f68ec06a0d0 msgr2=0x7f68ec06c590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:34:36.752 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:36.748+0000 7f6905871640 1 --2- 172.21.15.43:0/404075115 >> [v2:172.21.15.203:6832/830097474,v1:172.21.15.203:6833/830097474] conn(0x7f68ec06a0d0 0x7f68ec06c590 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7f68f0036000 tx=0x7f68f0008030 comp rx=0 tx=0).stop 2023-08-09T13:34:36.752 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:36.748+0000 7f6905871640 1 -- 172.21.15.43:0/404075115 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f69000b7f90 msgr2=0x7f690006d7d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:34:36.752 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:36.748+0000 7f6905871640 1 --2- 172.21.15.43:0/404075115 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f69000b7f90 0x7f690006d7d0 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7f68f400d830 tx=0x7f68f400dcf0 comp rx=0 tx=0).stop 2023-08-09T13:34:36.752 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:36.748+0000 7f6905871640 1 -- 172.21.15.43:0/404075115 shutdown_connections 2023-08-09T13:34:36.752 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:36.748+0000 7f6905871640 1 --2- 172.21.15.43:0/404075115 >> [v2:172.21.15.203:6832/830097474,v1:172.21.15.203:6833/830097474] conn(0x7f68ec06a0d0 0x7f68ec06c590 unknown :-1 s=CLOSED pgs=111 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:34:36.752 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:36.748+0000 7f6905871640 1 --2- 172.21.15.43:0/404075115 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f690015c500 0x7f69000b1790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:34:36.753 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:36.748+0000 7f6905871640 1 --2- 172.21.15.43:0/404075115 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f69000b7f90 0x7f690006d7d0 unknown :-1 s=CLOSED pgs=174 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:34:36.753 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:36.748+0000 7f6905871640 1 --2- 172.21.15.43:0/404075115 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f69000b5850 0x7f690015a6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:34:36.753 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:36.748+0000 7f6905871640 1 -- 172.21.15.43:0/404075115 >> 172.21.15.43:0/404075115 conn(0x7f69000712e0 msgr2=0x7f69000ccce0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-09T13:34:36.753 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:36.749+0000 7f6905871640 1 -- 172.21.15.43:0/404075115 shutdown_connections 2023-08-09T13:34:36.753 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:36.749+0000 7f6905871640 1 -- 172.21.15.43:0/404075115 wait complete. 2023-08-09T13:34:36.754 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574858 got 55834574857 for osd.4 2023-08-09T13:34:36.755 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574858 got 55834574857 for osd.2 2023-08-09T13:34:36.755 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:36.753+0000 7fb0e89fc640 1 -- 172.21.15.43:0/1321082850 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 5} v 0) v1 -- 0x7fb0e4159240 con 0x7fb0e40cb030 2023-08-09T13:34:36.756 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:36.753+0000 7fb67ff64640 1 -- 172.21.15.43:0/3604201571 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 7} v 0) v1 -- 0x7fb6780b1760 con 0x7fb6780d0280 2023-08-09T13:34:36.756 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:36.754+0000 7fb66e7fc640 1 -- 172.21.15.43:0/3604201571 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 7}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7fb6680787e0 con 0x7fb6780d0280 2023-08-09T13:34:36.756 INFO:teuthology.orchestra.run.smithi043.stdout:55834574857 2023-08-09T13:34:36.757 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:36.754+0000 7fb0cbfff640 1 -- 172.21.15.43:0/1321082850 <== mon.1 v2:172.21.15.203:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 5}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7fb0d8056d30 con 0x7fb0e40cb030 2023-08-09T13:34:36.757 INFO:teuthology.orchestra.run.smithi043.stdout:55834574857 2023-08-09T13:34:36.759 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:36.756+0000 7fb0e89fc640 1 -- 172.21.15.43:0/1321082850 >> [v2:172.21.15.203:6832/830097474,v1:172.21.15.203:6833/830097474] conn(0x7fb0d006a120 msgr2=0x7fb0d006c5e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:34:36.759 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:36.756+0000 7fb0e89fc640 1 --2- 172.21.15.43:0/1321082850 >> [v2:172.21.15.203:6832/830097474,v1:172.21.15.203:6833/830097474] conn(0x7fb0d006a120 0x7fb0d006c5e0 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7fb0d4036000 tx=0x7fb0d4008030 comp rx=0 tx=0).stop 2023-08-09T13:34:36.759 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:36.756+0000 7fb0e89fc640 1 -- 172.21.15.43:0/1321082850 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fb0e40cb030 msgr2=0x7fb0e40b1090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:34:36.759 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:36.756+0000 7fb0e89fc640 1 --2- 172.21.15.43:0/1321082850 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fb0e40cb030 0x7fb0e40b1090 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7fb0d800c450 tx=0x7fb0d800c910 comp rx=0 tx=0).stop 2023-08-09T13:34:36.760 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:36.756+0000 7fb0e89fc640 1 -- 172.21.15.43:0/1321082850 shutdown_connections 2023-08-09T13:34:36.760 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:36.756+0000 7fb0e89fc640 1 --2- 172.21.15.43:0/1321082850 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb0e4159520 0x7fb0e40c4c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:34:36.760 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:36.756+0000 7fb0e89fc640 1 --2- 172.21.15.43:0/1321082850 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fb0e40cb030 0x7fb0e40b1090 unknown :-1 s=CLOSED pgs=96 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:34:36.760 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:36.756+0000 7fb0e89fc640 1 --2- 172.21.15.43:0/1321082850 >> [v2:172.21.15.203:6832/830097474,v1:172.21.15.203:6833/830097474] conn(0x7fb0d006a120 0x7fb0d006c5e0 unknown :-1 s=CLOSED pgs=112 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:34:36.760 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:36.756+0000 7fb0e89fc640 1 --2- 172.21.15.43:0/1321082850 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fb0e406a970 0x7fb0e4156cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:34:36.760 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:36.756+0000 7fb0e89fc640 1 -- 172.21.15.43:0/1321082850 >> 172.21.15.43:0/1321082850 conn(0x7fb0e40cb9e0 msgr2=0x7fb0e415f230 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-09T13:34:36.760 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:36.757+0000 7fb0e89fc640 1 -- 172.21.15.43:0/1321082850 shutdown_connections 2023-08-09T13:34:36.761 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:36.757+0000 7fb0e89fc640 1 -- 172.21.15.43:0/1321082850 wait complete. 2023-08-09T13:34:36.761 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:36.757+0000 7fb67ff64640 1 -- 172.21.15.43:0/3604201571 >> [v2:172.21.15.203:6832/830097474,v1:172.21.15.203:6833/830097474] conn(0x7fb65c06a0a0 msgr2=0x7fb65c06c560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:34:36.761 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:36.757+0000 7fb67ff64640 1 --2- 172.21.15.43:0/3604201571 >> [v2:172.21.15.203:6832/830097474,v1:172.21.15.203:6833/830097474] conn(0x7fb65c06a0a0 0x7fb65c06c560 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7fb674005f80 tx=0x7fb674008040 comp rx=0 tx=0).stop 2023-08-09T13:34:36.762 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:36.757+0000 7fb67ff64640 1 -- 172.21.15.43:0/3604201571 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb6780d0280 msgr2=0x7fb67815bce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:34:36.762 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:36.757+0000 7fb67ff64640 1 --2- 172.21.15.43:0/3604201571 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb6780d0280 0x7fb67815bce0 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7fb668009890 tx=0x7fb6680046c0 comp rx=0 tx=0).stop 2023-08-09T13:34:36.762 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:36.758+0000 7fb67ff64640 1 -- 172.21.15.43:0/3604201571 shutdown_connections 2023-08-09T13:34:36.762 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:36.758+0000 7fb67ff64640 1 --2- 172.21.15.43:0/3604201571 >> [v2:172.21.15.203:6832/830097474,v1:172.21.15.203:6833/830097474] conn(0x7fb65c06a0a0 0x7fb65c06c560 unknown :-1 s=CLOSED pgs=113 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:34:36.762 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:36.758+0000 7fb67ff64640 1 --2- 172.21.15.43:0/3604201571 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fb67815a510 0x7fb6781504f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:34:36.762 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:36.758+0000 7fb67ff64640 1 --2- 172.21.15.43:0/3604201571 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb6780d0280 0x7fb67815bce0 unknown :-1 s=CLOSED pgs=175 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:34:36.762 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:36.758+0000 7fb67ff64640 1 --2- 172.21.15.43:0/3604201571 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fb6780cb0e0 0x7fb67815d520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:34:36.763 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:36.758+0000 7fb67ff64640 1 -- 172.21.15.43:0/3604201571 >> 172.21.15.43:0/3604201571 conn(0x7fb67806a1f0 msgr2=0x7fb678153bb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-09T13:34:36.763 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:36.758+0000 7fb67ff64640 1 -- 172.21.15.43:0/3604201571 shutdown_connections 2023-08-09T13:34:36.763 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:36.758+0000 7fb67ff64640 1 -- 172.21.15.43:0/3604201571 wait complete. 2023-08-09T13:34:36.763 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574859 got 55834574858 for osd.3 2023-08-09T13:34:36.778 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574858 got 55834574857 for osd.6 2023-08-09T13:34:36.778 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574858 got 55834574857 for osd.5 2023-08-09T13:34:36.780 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574858 got 55834574857 for osd.7 2023-08-09T13:34:37.720 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2023-08-09T13:34:37.751 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2023-08-09T13:34:37.755 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.4 2023-08-09T13:34:37.756 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2023-08-09T13:34:37.764 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.3 2023-08-09T13:34:37.778 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.6 2023-08-09T13:34:37.779 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.5 2023-08-09T13:34:37.781 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.7 2023-08-09T13:34:38.011 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:38.009+0000 7efd8afb1640 1 Processor -- start 2023-08-09T13:34:38.014 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:38.009+0000 7efd8afb1640 1 -- start start 2023-08-09T13:34:38.014 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:38.011+0000 7efd8afb1640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7efd840ca170 0x7efd84159260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:34:38.014 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:38.011+0000 7efd8afb1640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7efd8406f690 0x7efd840cca60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:34:38.014 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:38.011+0000 7efd8afb1640 1 -- --> v1:172.21.15.43:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7efd840cdaa0 con 0x7efd840ccfa0 2023-08-09T13:34:38.015 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:38.011+0000 7efd8afb1640 1 -- --> v2:172.21.15.43:3300/0 -- mon_getmap magic: 0 v1 -- 0x7efd8406fbe0 con 0x7efd840ca170 2023-08-09T13:34:38.015 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:38.011+0000 7efd8afb1640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7efd8406fec0 con 0x7efd8406f690 2023-08-09T13:34:38.015 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:38.012+0000 7efd83fff640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7efd8406f690 0x7efd840cca60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:34:38.016 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:38.012+0000 7efd83fff640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7efd8406f690 0x7efd840cca60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.203:3301/0 says I am v2:172.21.15.43:45074/0 (socket says 172.21.15.43:45074) 2023-08-09T13:34:38.016 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:38.012+0000 7efd83fff640 1 -- 172.21.15.43:0/2930201526 learned_addr learned my addr 172.21.15.43:0/2930201526 (peer_addr_for_me v2:172.21.15.43:0/0) 2023-08-09T13:34:38.016 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:38.012+0000 7efd88d4d640 1 --2- 172.21.15.43:0/2930201526 >> v2:172.21.15.43:3300/0 conn(0x7efd840ca170 0x7efd84159260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:34:38.016 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:38.013+0000 7efd83fff640 1 -- 172.21.15.43:0/2930201526 >> v2:172.21.15.43:3300/0 conn(0x7efd840ca170 msgr2=0x7efd84159260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-09T13:34:38.016 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:38.013+0000 7efd83fff640 1 --2- 172.21.15.43:0/2930201526 >> v2:172.21.15.43:3300/0 conn(0x7efd840ca170 0x7efd84159260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:34:38.016 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:38.013+0000 7efd83fff640 1 -- 172.21.15.43:0/2930201526 >> v1:172.21.15.43:6789/0 conn(0x7efd840ccfa0 legacy=0x7efd8414ec50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-09T13:34:38.017 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:38.013+0000 7efd83fff640 1 -- 172.21.15.43:0/2930201526 --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efd840cd400 con 0x7efd8406f690 2023-08-09T13:34:38.018 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:38.015+0000 7efd83fff640 1 --2- 172.21.15.43:0/2930201526 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7efd8406f690 0x7efd840cca60 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7efd7c00d370 tx=0x7efd7c02e1f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=29de9d0d04b7bb01 server_cookie=0 in_seq=0 out_seq=0 2023-08-09T13:34:38.018 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:38.016+0000 7efd837fe640 1 -- 172.21.15.43:0/2930201526 <== mon.2 v2:172.21.15.203:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efd7c038400 con 0x7efd8406f690 2023-08-09T13:34:38.018 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:38.016+0000 7efd837fe640 1 -- 172.21.15.43:0/2930201526 <== mon.2 v2:172.21.15.203:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efd7c032070 con 0x7efd8406f690 2023-08-09T13:34:38.020 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:38.017+0000 7efd8afb1640 1 -- 172.21.15.43:0/2930201526 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7efd8406f690 msgr2=0x7efd840cca60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:34:38.020 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:38.017+0000 7efd8afb1640 1 --2- 172.21.15.43:0/2930201526 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7efd8406f690 0x7efd840cca60 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7efd7c00d370 tx=0x7efd7c02e1f0 comp rx=0 tx=0).stop 2023-08-09T13:34:38.020 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:38.018+0000 7efd8afb1640 1 -- 172.21.15.43:0/2930201526 shutdown_connections 2023-08-09T13:34:38.020 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:38.018+0000 7efd8afb1640 1 --2- 172.21.15.43:0/2930201526 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7efd8406f690 0x7efd840cca60 unknown :-1 s=CLOSED pgs=52 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:34:38.021 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:38.018+0000 7efd8afb1640 1 --2- 172.21.15.43:0/2930201526 >> v2:172.21.15.43:3300/0 conn(0x7efd840ca170 0x7efd84159260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:34:38.021 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:38.018+0000 7efd8afb1640 1 -- 172.21.15.43:0/2930201526 >> 172.21.15.43:0/2930201526 conn(0x7efd840af970 msgr2=0x7efd840afd80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-09T13:34:38.021 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:38.018+0000 7efd8afb1640 1 -- 172.21.15.43:0/2930201526 shutdown_connections 2023-08-09T13:34:38.023 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:38.018+0000 7efd8afb1640 1 -- 172.21.15.43:0/2930201526 wait complete. 2023-08-09T13:34:38.024 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:38.021+0000 7efd8afb1640 1 Processor -- start 2023-08-09T13:34:38.026 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:38.022+0000 7efd8afb1640 1 -- start start 2023-08-09T13:34:38.026 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:38.023+0000 7efd8afb1640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7efd840ca170 0x7efd840b2a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:34:38.026 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:38.024+0000 7efd8afb1640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7efd840ccfa0 0x7efd840c91b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:34:38.027 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:38.024+0000 7efd8afb1640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7efd8414a810 0x7efd840c96f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:34:38.027 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:38.024+0000 7efd8afb1640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efd841f7290 con 0x7efd840ccfa0 2023-08-09T13:34:38.027 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:38.024+0000 7efd88d4d640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7efd840ca170 0x7efd840b2a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:34:38.027 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:38.025+0000 7efd88d4d640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7efd840ca170 0x7efd840b2a00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.203:3300/0 says I am v2:172.21.15.43:34484/0 (socket says 172.21.15.43:34484) 2023-08-09T13:34:38.027 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:38.025+0000 7efd88d4d640 1 -- 172.21.15.43:0/1847632179 learned_addr learned my addr 172.21.15.43:0/1847632179 (peer_addr_for_me v2:172.21.15.43:0/0) 2023-08-09T13:34:38.028 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:38.025+0000 7efd83fff640 1 --2- 172.21.15.43:0/1847632179 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7efd840ccfa0 0x7efd840c91b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:34:38.028 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:38.025+0000 7efd8afb1640 1 -- 172.21.15.43:0/1847632179 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efd840c6e00 con 0x7efd840ca170 2023-08-09T13:34:38.029 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:38.025+0000 7efd8afb1640 1 -- 172.21.15.43:0/1847632179 --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7efd840c6040 con 0x7efd8414a810 2023-08-09T13:34:38.029 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:38.026+0000 7efd8954e640 1 --2- 172.21.15.43:0/1847632179 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7efd8414a810 0x7efd840c96f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:34:38.029 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:38.026+0000 7efd83fff640 1 -- 172.21.15.43:0/1847632179 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7efd8414a810 msgr2=0x7efd840c96f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:34:38.030 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:38.026+0000 7efd83fff640 1 --2- 172.21.15.43:0/1847632179 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7efd8414a810 0x7efd840c96f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:34:38.030 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:38.026+0000 7efd83fff640 1 -- 172.21.15.43:0/1847632179 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7efd840ca170 msgr2=0x7efd840b2a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:34:38.030 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:38.026+0000 7efd83fff640 1 --2- 172.21.15.43:0/1847632179 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7efd840ca170 0x7efd840b2a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:34:38.031 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:38.026+0000 7efd83fff640 1 -- 172.21.15.43:0/1847632179 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efd7c00d050 con 0x7efd840ccfa0 2023-08-09T13:34:38.031 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:38.027+0000 7efd83fff640 1 --2- 172.21.15.43:0/1847632179 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7efd840ccfa0 0x7efd840c91b0 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7efd7c036000 tx=0x7efd7c004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-09T13:34:38.032 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:38.028+0000 7efd817fa640 1 -- 172.21.15.43:0/1847632179 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efd7c046070 con 0x7efd840ccfa0 2023-08-09T13:34:38.032 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:38.028+0000 7efd8afb1640 1 -- 172.21.15.43:0/1847632179 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7efd840c6320 con 0x7efd840ccfa0 2023-08-09T13:34:38.032 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:38.029+0000 7efd8afb1640 1 -- 172.21.15.43:0/1847632179 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7efd840c68e0 con 0x7efd840ccfa0 2023-08-09T13:34:38.035 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:38.032+0000 7efd817fa640 1 -- 172.21.15.43:0/1847632179 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efd7c038cb0 con 0x7efd840ccfa0 2023-08-09T13:34:38.035 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:38.032+0000 7efd817fa640 1 -- 172.21.15.43:0/1847632179 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efd7c008280 con 0x7efd840ccfa0 2023-08-09T13:34:38.036 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:38.033+0000 7efd817fa640 1 -- 172.21.15.43:0/1847632179 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 5) v1 ==== 88341+0+0 (secure 0 0 0) 0x7efd7c0084a0 con 0x7efd840ccfa0 2023-08-09T13:34:38.037 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:38.035+0000 7efd817fa640 1 --2- 172.21.15.43:0/1847632179 >> [v2:172.21.15.203:6832/830097474,v1:172.21.15.203:6833/830097474] conn(0x7efd7006a1a0 0x7efd7006c660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:34:38.038 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:38.035+0000 7efd817fa640 1 -- 172.21.15.43:0/1847632179 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6813+0+0 (secure 0 0 0) 0x7efd7c0aa9f0 con 0x7efd840ccfa0 2023-08-09T13:34:38.038 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:38.036+0000 7efd88d4d640 1 --2- 172.21.15.43:0/1847632179 >> [v2:172.21.15.203:6832/830097474,v1:172.21.15.203:6833/830097474] conn(0x7efd7006a1a0 0x7efd7006c660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:34:38.039 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:38.036+0000 7efd8afb1640 1 -- 172.21.15.43:0/1847632179 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7efd64005250 con 0x7efd840ccfa0 2023-08-09T13:34:38.049 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:38.047+0000 7efd817fa640 1 -- 172.21.15.43:0/1847632179 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178066 (secure 0 0 0) 0x7efd7c075f10 con 0x7efd840ccfa0 2023-08-09T13:34:38.050 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:38.047+0000 7efd88d4d640 1 --2- 172.21.15.43:0/1847632179 >> [v2:172.21.15.203:6832/830097474,v1:172.21.15.203:6833/830097474] conn(0x7efd7006a1a0 0x7efd7006c660 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7efd840b1c50 tx=0x7efd74009190 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-09T13:34:38.071 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:38.067+0000 7f02dd2e5640 1 Processor -- start 2023-08-09T13:34:38.071 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:38.067+0000 7f02dd2e5640 1 -- start start 2023-08-09T13:34:38.072 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:38.069+0000 7f02dd2e5640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f02d003a860 0x7f02d0038170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:34:38.072 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:38.069+0000 7f02dd2e5640 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7f02d0005fe0 0x7f02d00386b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:34:38.072 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:38.069+0000 7f02dd2e5640 1 -- --> v1:172.21.15.43:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f02d0037d40 con 0x7f02d00545a0 2023-08-09T13:34:38.073 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:38.069+0000 7f02dd2e5640 1 -- --> v2:172.21.15.43:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f02d00079c0 con 0x7f02d003a860 2023-08-09T13:34:38.073 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:38.069+0000 7f02dd2e5640 1 -- --> v2:172.21.15.203:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f02d0054340 con 0x7f02d0005fe0 2023-08-09T13:34:38.074 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:38.070+0000 7f02d7fff640 1 --1- >> v1:172.21.15.43:6789/0 conn(0x7f02d00545a0 0x7f02d0006870 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.43:6789/0 says I am v1:172.21.15.43:35660/0 (socket says 172.21.15.43:35660) 2023-08-09T13:34:38.074 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:38.070+0000 7f02d7fff640 1 -- 172.21.15.43:0/2840426600 learned_addr learned my addr 172.21.15.43:0/2840426600 (peer_addr_for_me v1:172.21.15.43:0/0) 2023-08-09T13:34:38.074 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:38.070+0000 7f02d77fe640 1 --2- 172.21.15.43:0/2840426600 >> v2:172.21.15.43:3300/0 conn(0x7f02d003a860 0x7f02d0038170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:34:38.074 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:38.070+0000 7f02dcae4640 1 --2- 172.21.15.43:0/2840426600 >> v2:172.21.15.203:3300/0 conn(0x7f02d0005fe0 0x7f02d00386b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:34:38.075 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:38.071+0000 7f63fb5e8640 1 Processor -- start 2023-08-09T13:34:38.076 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:38.071+0000 7f63fb5e8640 1 -- start start 2023-08-09T13:34:38.076 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:38.071+0000 7f02dcae4640 1 -- 172.21.15.43:0/2840426600 >> v2:172.21.15.43:3300/0 conn(0x7f02d003a860 msgr2=0x7f02d0038170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-09T13:34:38.077 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:38.071+0000 7f02dcae4640 1 --2- 172.21.15.43:0/2840426600 >> v2:172.21.15.43:3300/0 conn(0x7f02d003a860 0x7f02d0038170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:34:38.077 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:38.071+0000 7f02dcae4640 1 -- 172.21.15.43:0/2840426600 >> v1:172.21.15.43:6789/0 conn(0x7f02d00545a0 legacy=0x7f02d0006870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:34:38.077 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:38.071+0000 7f02dcae4640 1 -- 172.21.15.43:0/2840426600 --> v2:172.21.15.203:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f02d0007090 con 0x7f02d0005fe0 2023-08-09T13:34:38.077 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:38.073+0000 7f02dcae4640 1 --2- 172.21.15.43:0/2840426600 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f02d0005fe0 0x7f02d00386b0 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7f02d8067f10 tx=0x7f02d8094200 comp rx=0 tx=0).ready entity=mon.1 client_cookie=e18a17c7dc19afa6 server_cookie=0 in_seq=0 out_seq=0 2023-08-09T13:34:38.078 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:38.073+0000 7f63fb5e8640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f63ec0dffd0 0x7f63ec0399a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:34:38.079 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:38.073+0000 7f63fb5e8640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f63ec050330 0x7f63ec050820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:34:38.079 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:38.073+0000 7f63fb5e8640 1 -- --> v1:172.21.15.203:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f63ec04fea0 con 0x7f63ec039ee0 2023-08-09T13:34:38.079 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:38.073+0000 7f63fb5e8640 1 -- --> v2:172.21.15.43:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f63ec0e03b0 con 0x7f63ec050330 2023-08-09T13:34:38.079 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:38.073+0000 7f63fb5e8640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f63ec04f290 con 0x7f63ec0dffd0 2023-08-09T13:34:38.079 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:38.074+0000 7f63f8b83640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f63ec050330 0x7f63ec050820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:34:38.079 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:38.074+0000 7f63f8b83640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f63ec050330 0x7f63ec050820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:51864/0 (socket says 172.21.15.43:51864) 2023-08-09T13:34:38.079 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:38.074+0000 7f63f8b83640 1 -- 172.21.15.43:0/1927044715 learned_addr learned my addr 172.21.15.43:0/1927044715 (peer_addr_for_me v2:172.21.15.43:0/0) 2023-08-09T13:34:38.080 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:38.073+0000 7f02d6ffd640 1 -- 172.21.15.43:0/2840426600 <== mon.1 v2:172.21.15.203:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f02d8094e00 con 0x7f02d0005fe0 2023-08-09T13:34:38.080 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:38.073+0000 7f02d6ffd640 1 -- 172.21.15.43:0/2840426600 <== mon.1 v2:172.21.15.203:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f02d809c440 con 0x7f02d0005fe0 2023-08-09T13:34:38.080 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:38.073+0000 7f02d6ffd640 1 -- 172.21.15.43:0/2840426600 <== mon.1 v2:172.21.15.203:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f02d809c870 con 0x7f02d0005fe0 2023-08-09T13:34:38.080 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:38.074+0000 7f02dd2e5640 1 -- 172.21.15.43:0/2840426600 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f02d0005fe0 msgr2=0x7f02d00386b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:34:38.081 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:38.074+0000 7f02dd2e5640 1 --2- 172.21.15.43:0/2840426600 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f02d0005fe0 0x7f02d00386b0 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7f02d8067f10 tx=0x7f02d8094200 comp rx=0 tx=0).stop 2023-08-09T13:34:38.081 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:38.075+0000 7f63f8b83640 1 -- 172.21.15.43:0/1927044715 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f63ec0dffd0 msgr2=0x7f63ec0399a0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-08-09T13:34:38.081 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:38.075+0000 7f63f8b83640 1 --2- 172.21.15.43:0/1927044715 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f63ec0dffd0 0x7f63ec0399a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:34:38.081 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:38.075+0000 7f63f8b83640 1 -- 172.21.15.43:0/1927044715 >> v1:172.21.15.203:6789/0 conn(0x7f63ec039ee0 legacy=0x7f63ec0e3ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-09T13:34:38.081 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:38.075+0000 7f63f8b83640 1 -- 172.21.15.43:0/1927044715 --> v2:172.21.15.43:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f63ec04f570 con 0x7f63ec050330 2023-08-09T13:34:38.082 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:38.075+0000 7f02dd2e5640 1 -- 172.21.15.43:0/2840426600 shutdown_connections 2023-08-09T13:34:38.082 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:38.075+0000 7f02dd2e5640 1 --2- 172.21.15.43:0/2840426600 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f02d0005fe0 0x7f02d00386b0 unknown :-1 s=CLOSED pgs=97 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:34:38.082 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:38.075+0000 7f02dd2e5640 1 --2- 172.21.15.43:0/2840426600 >> v2:172.21.15.43:3300/0 conn(0x7f02d003a860 0x7f02d0038170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:34:38.082 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:38.075+0000 7f02dd2e5640 1 -- 172.21.15.43:0/2840426600 >> 172.21.15.43:0/2840426600 conn(0x7f02d0054c10 msgr2=0x7f02d0055020 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-09T13:34:38.082 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:38.075+0000 7f02dd2e5640 1 -- 172.21.15.43:0/2840426600 shutdown_connections 2023-08-09T13:34:38.083 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:38.075+0000 7f02dd2e5640 1 -- 172.21.15.43:0/2840426600 wait complete. 2023-08-09T13:34:38.083 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:38.077+0000 7f63f8b83640 1 --2- 172.21.15.43:0/1927044715 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f63ec050330 0x7f63ec050820 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7f63f000bb10 tx=0x7f63f002e0e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=feb1dc6ea33a5c7a server_cookie=0 in_seq=0 out_seq=0 2023-08-09T13:34:38.083 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:38.077+0000 7f63ebfff640 1 -- 172.21.15.43:0/1927044715 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f63f002ee90 con 0x7f63ec050330 2023-08-09T13:34:38.083 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:38.077+0000 7f63ebfff640 1 -- 172.21.15.43:0/1927044715 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f63f0036440 con 0x7f63ec050330 2023-08-09T13:34:38.083 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:38.077+0000 7f63ebfff640 1 -- 172.21.15.43:0/1927044715 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f63f00367d0 con 0x7f63ec050330 2023-08-09T13:34:38.084 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:38.078+0000 7f63fb5e8640 1 -- 172.21.15.43:0/1927044715 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f63ec050330 msgr2=0x7f63ec050820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:34:38.084 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:38.078+0000 7f63fb5e8640 1 --2- 172.21.15.43:0/1927044715 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f63ec050330 0x7f63ec050820 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7f63f000bb10 tx=0x7f63f002e0e0 comp rx=0 tx=0).stop 2023-08-09T13:34:38.084 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:38.078+0000 7f02dd2e5640 1 Processor -- start 2023-08-09T13:34:38.084 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:38.079+0000 7f63fb5e8640 1 -- 172.21.15.43:0/1927044715 shutdown_connections 2023-08-09T13:34:38.085 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:38.079+0000 7f63fb5e8640 1 --2- 172.21.15.43:0/1927044715 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f63ec050330 0x7f63ec050820 unknown :-1 s=CLOSED pgs=179 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:34:38.085 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:38.079+0000 7f63fb5e8640 1 --2- 172.21.15.43:0/1927044715 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f63ec0dffd0 0x7f63ec0399a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:34:38.085 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:38.079+0000 7f63fb5e8640 1 -- 172.21.15.43:0/1927044715 >> 172.21.15.43:0/1927044715 conn(0x7f63ec054c40 msgr2=0x7f63ec055020 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-09T13:34:38.085 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:38.079+0000 7f02dd2e5640 1 -- start start 2023-08-09T13:34:38.086 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:38.081+0000 7f02dd2e5640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f02d0005fe0 0x7f02d00475a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:34:38.086 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:38.081+0000 7f02dd2e5640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f02d003a860 0x7f02d004ac30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:34:38.086 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:38.081+0000 7f02dd2e5640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f02d00545a0 0x7f02d004b170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:34:38.086 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:38.081+0000 7f63fb5e8640 1 -- 172.21.15.43:0/1927044715 shutdown_connections 2023-08-09T13:34:38.086 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:38.081+0000 7f63fb5e8640 1 -- 172.21.15.43:0/1927044715 wait complete. 2023-08-09T13:34:38.087 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:38.082+0000 7f02d7fff640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f02d0005fe0 0x7f02d00475a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:34:38.087 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:38.082+0000 7f02d7fff640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f02d0005fe0 0x7f02d00475a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.203:3301/0 says I am v2:172.21.15.43:45092/0 (socket says 172.21.15.43:45092) 2023-08-09T13:34:38.087 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:38.082+0000 7f02d77fe640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f02d003a860 0x7f02d004ac30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:34:38.087 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:38.082+0000 7f02d77fe640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f02d003a860 0x7f02d004ac30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.203:3300/0 says I am v2:172.21.15.43:34500/0 (socket says 172.21.15.43:34500) 2023-08-09T13:34:38.088 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:38.083+0000 7f63fb5e8640 1 Processor -- start 2023-08-09T13:34:38.088 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:38.083+0000 7f02d7fff640 1 -- 172.21.15.43:0/375074373 learned_addr learned my addr 172.21.15.43:0/375074373 (peer_addr_for_me v2:172.21.15.43:0/0) 2023-08-09T13:34:38.088 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:38.083+0000 7f02dd2e5640 1 -- 172.21.15.43:0/375074373 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f02d010d1a0 con 0x7f02d00545a0 2023-08-09T13:34:38.088 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:38.083+0000 7f02dd2e5640 1 -- 172.21.15.43:0/375074373 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f02d010d460 con 0x7f02d003a860 2023-08-09T13:34:38.088 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:38.083+0000 7f02dd2e5640 1 -- 172.21.15.43:0/375074373 --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f02d010d740 con 0x7f02d0005fe0 2023-08-09T13:34:38.089 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:38.083+0000 7f02dcae4640 1 --2- 172.21.15.43:0/375074373 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f02d00545a0 0x7f02d004b170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:34:38.089 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:38.084+0000 7f63fb5e8640 1 -- start start 2023-08-09T13:34:38.089 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:38.084+0000 7f02d7fff640 1 -- 172.21.15.43:0/375074373 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f02d003a860 msgr2=0x7f02d004ac30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:34:38.090 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:38.084+0000 7f02d7fff640 1 --2- 172.21.15.43:0/375074373 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f02d003a860 0x7f02d004ac30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:34:38.090 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:38.084+0000 7f02d7fff640 1 -- 172.21.15.43:0/375074373 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f02d00545a0 msgr2=0x7f02d004b170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:34:38.090 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:38.084+0000 7f02d7fff640 1 --2- 172.21.15.43:0/375074373 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f02d00545a0 0x7f02d004b170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:34:38.090 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:38.084+0000 7f02d7fff640 1 -- 172.21.15.43:0/375074373 --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f02d010da20 con 0x7f02d0005fe0 2023-08-09T13:34:38.090 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:38.084+0000 7f02d7fff640 1 --2- 172.21.15.43:0/375074373 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f02d0005fe0 0x7f02d00475a0 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f02c400c3c0 tx=0x7f02c400c880 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-09T13:34:38.091 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:38.080+0000 7fdf9cd86640 1 Processor -- start 2023-08-09T13:34:38.091 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:38.080+0000 7fdf9cd86640 1 -- start start 2023-08-09T13:34:38.091 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:38.082+0000 7fdf9cd86640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7fdf9806fe10 0x7fdf980c8cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:34:38.091 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:38.082+0000 7fdf9cd86640 1 -- --> v1:172.21.15.43:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fdf980c8820 con 0x7fdf980b0490 2023-08-09T13:34:38.091 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:38.082+0000 7fdf9cd86640 1 -- --> v1:172.21.15.203:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fdf980703f0 con 0x7fdf9816c570 2023-08-09T13:34:38.092 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:38.082+0000 7fdf9cd86640 1 -- --> v2:172.21.15.43:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fdf980cd510 con 0x7fdf9806fe10 2023-08-09T13:34:38.092 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:38.083+0000 7fdf9659c640 1 --1- >> v1:172.21.15.203:6789/0 conn(0x7fdf9816c570 0x7fdf9806f700 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.203:6789/0 says I am v1:172.21.15.43:33844/0 (socket says 172.21.15.43:33844) 2023-08-09T13:34:38.092 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:38.083+0000 7fdf9659c640 1 -- 172.21.15.43:0/2621260210 learned_addr learned my addr 172.21.15.43:0/2621260210 (peer_addr_for_me v1:172.21.15.43:0/0) 2023-08-09T13:34:38.092 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:38.083+0000 7fdf9559a640 1 -- 172.21.15.43:0/2621260210 <== mon.1 v1:172.21.15.203:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3570086656 0 0) 0x7fdf980703f0 con 0x7fdf9816c570 2023-08-09T13:34:38.092 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:38.084+0000 7fdf95d9b640 1 --2- 172.21.15.43:0/2621260210 >> v2:172.21.15.43:3300/0 conn(0x7fdf9806fe10 0x7fdf980c8cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:34:38.092 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:38.084+0000 7fdf9559a640 1 -- 172.21.15.43:0/2621260210 --> v1:172.21.15.203:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fdf80003db0 con 0x7fdf9816c570 2023-08-09T13:34:38.092 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:38.084+0000 7fdf9559a640 1 -- 172.21.15.43:0/2621260210 <== mon.0 v1:172.21.15.43:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2077301956 0 0) 0x7fdf980c8820 con 0x7fdf980b0490 2023-08-09T13:34:38.092 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:38.084+0000 7fdf9559a640 1 -- 172.21.15.43:0/2621260210 --> v1:172.21.15.43:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fdf980703f0 con 0x7fdf980b0490 2023-08-09T13:34:38.093 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:38.085+0000 7f63fb5e8640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f63ec039ee0 0x7f63ec04a1d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:34:38.093 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:38.085+0000 7fdf9559a640 1 -- 172.21.15.43:0/2621260210 <== mon.1 v1:172.21.15.203:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2965020526 0 0) 0x7fdf80003db0 con 0x7fdf9816c570 2023-08-09T13:34:38.093 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:38.085+0000 7fdf9559a640 1 -- 172.21.15.43:0/2621260210 >> v2:172.21.15.43:3300/0 conn(0x7fdf9806fe10 msgr2=0x7fdf980c8cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-09T13:34:38.093 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:38.085+0000 7fdf9559a640 1 --2- 172.21.15.43:0/2621260210 >> v2:172.21.15.43:3300/0 conn(0x7fdf9806fe10 0x7fdf980c8cb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:34:38.094 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:38.085+0000 7fdf9559a640 1 -- 172.21.15.43:0/2621260210 >> v1:172.21.15.43:6789/0 conn(0x7fdf980b0490 legacy=0x7fdf980c91f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:34:38.094 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:38.085+0000 7fdf9559a640 1 -- 172.21.15.43:0/2621260210 --> v1:172.21.15.203:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdf980c84d0 con 0x7fdf9816c570 2023-08-09T13:34:38.094 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:38.085+0000 7fdf9559a640 1 -- 172.21.15.43:0/2621260210 <== mon.1 v1:172.21.15.203:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1285150740 0 0) 0x7fdf88001670 con 0x7fdf9816c570 2023-08-09T13:34:38.094 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:38.085+0000 7fdf9559a640 1 -- 172.21.15.43:0/2621260210 >> v1:172.21.15.203:6789/0 conn(0x7fdf9816c570 legacy=0x7fdf9806f700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:34:38.094 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:38.085+0000 7f58686b8640 1 Processor -- start 2023-08-09T13:34:38.095 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:38.085+0000 7fdf9559a640 1 --2- 172.21.15.43:0/2621260210 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fdf800038c0 0x7fdf80005050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:34:38.095 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:38.085+0000 7fdf9559a640 1 --2- 172.21.15.43:0/2621260210 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fdf800055e0 0x7fdf80005a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:34:38.095 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:38.085+0000 7fdf9559a640 1 --2- 172.21.15.43:0/2621260210 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fdf80004100 0x7fdf800045b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:34:38.095 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:38.085+0000 7fdf9559a640 1 -- 172.21.15.43:0/2621260210 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdf980cd510 con 0x7fdf800038c0 2023-08-09T13:34:38.095 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:38.085+0000 7fdf9559a640 1 -- 172.21.15.43:0/2621260210 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdf80004e90 con 0x7fdf800055e0 2023-08-09T13:34:38.096 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:38.085+0000 7fdf9559a640 1 -- 172.21.15.43:0/2621260210 --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdf80008240 con 0x7fdf80004100 2023-08-09T13:34:38.096 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:38.086+0000 7f63fb5e8640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f63ec050330 0x7f63ec006940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:34:38.096 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:38.086+0000 7f02d4ff9640 1 -- 172.21.15.43:0/375074373 <== mon.2 v2:172.21.15.203:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f02c4005ce0 con 0x7f02d0005fe0 2023-08-09T13:34:38.097 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:38.086+0000 7f63fb5e8640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f63ec0dffd0 0x7f63ec0dad10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:34:38.097 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:38.086+0000 7f63fb5e8640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f63ec04db10 con 0x7f63ec039ee0 2023-08-09T13:34:38.098 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:38.086+0000 7f63fb5e8640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f63ec04ddf0 con 0x7f63ec050330 2023-08-09T13:34:38.098 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:38.086+0000 7f63fb5e8640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f63ec04e0d0 con 0x7f63ec0dffd0 2023-08-09T13:34:38.098 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:38.086+0000 7fdf96d9d640 1 --2- 172.21.15.43:0/2621260210 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fdf80004100 0x7fdf800045b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:34:38.099 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:38.087+0000 7f63f8b83640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f63ec050330 0x7f63ec006940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:34:38.099 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:38.087+0000 7f63f9384640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f63ec039ee0 0x7f63ec04a1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:34:38.099 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:38.087+0000 7f63f9384640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f63ec039ee0 0x7f63ec04a1d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:51888/0 (socket says 172.21.15.43:51888) 2023-08-09T13:34:38.099 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:38.087+0000 7f63f9384640 1 -- 172.21.15.43:0/246394760 learned_addr learned my addr 172.21.15.43:0/246394760 (peer_addr_for_me v2:172.21.15.43:0/0) 2023-08-09T13:34:38.100 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:38.087+0000 7f63f9b85640 1 --2- 172.21.15.43:0/246394760 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f63ec0dffd0 0x7f63ec0dad10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:34:38.100 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:38.087+0000 7f63f8b83640 1 -- 172.21.15.43:0/246394760 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f63ec0dffd0 msgr2=0x7f63ec0dad10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:34:38.100 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:38.087+0000 7f63f8b83640 1 --2- 172.21.15.43:0/246394760 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f63ec0dffd0 0x7f63ec0dad10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:34:38.100 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:38.087+0000 7f63f8b83640 1 -- 172.21.15.43:0/246394760 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f63ec039ee0 msgr2=0x7f63ec04a1d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:34:38.100 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:38.087+0000 7f63f8b83640 1 --2- 172.21.15.43:0/246394760 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f63ec039ee0 0x7f63ec04a1d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:34:38.100 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:38.087+0000 7f63f8b83640 1 -- 172.21.15.43:0/246394760 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f63f000b770 con 0x7f63ec050330 2023-08-09T13:34:38.101 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:38.087+0000 7f02d4ff9640 1 -- 172.21.15.43:0/375074373 <== mon.2 v2:172.21.15.203:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f02c4005e40 con 0x7f02d0005fe0 2023-08-09T13:34:38.101 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:38.087+0000 7f02dd2e5640 1 -- 172.21.15.43:0/375074373 --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f02d010dd00 con 0x7f02d0005fe0 2023-08-09T13:34:38.101 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:38.087+0000 7f02dd2e5640 1 -- 172.21.15.43:0/375074373 --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f02d004d170 con 0x7f02d0005fe0 2023-08-09T13:34:38.101 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:38.087+0000 7fdf9659c640 1 --2- 172.21.15.43:0/2621260210 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fdf800038c0 0x7fdf80005050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:34:38.102 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:38.089+0000 7f02d4ff9640 1 -- 172.21.15.43:0/375074373 <== mon.2 v2:172.21.15.203:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f02c40072d0 con 0x7f02d0005fe0 2023-08-09T13:34:38.102 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:38.088+0000 7fdf9659c640 1 -- 172.21.15.43:0/2621260210 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fdf80004100 msgr2=0x7fdf800045b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-09T13:34:38.102 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:38.088+0000 7fdf9659c640 1 --2- 172.21.15.43:0/2621260210 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fdf80004100 0x7fdf800045b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:34:38.102 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:38.088+0000 7fdf9659c640 1 -- 172.21.15.43:0/2621260210 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fdf800055e0 msgr2=0x7fdf80005a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-09T13:34:38.103 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:38.088+0000 7fdf9659c640 1 --2- 172.21.15.43:0/2621260210 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fdf800055e0 0x7fdf80005a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:34:38.103 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:38.088+0000 7fdf9659c640 1 -- 172.21.15.43:0/2621260210 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdf980c84d0 con 0x7fdf800038c0 2023-08-09T13:34:38.103 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:38.089+0000 7fdf96d9d640 1 -- 172.21.15.43:0/2621260210 reap_dead start 2023-08-09T13:34:38.103 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:38.089+0000 7f63f8b83640 1 --2- 172.21.15.43:0/246394760 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f63ec050330 0x7f63ec006940 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7f63f0035000 tx=0x7f63f0004730 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-09T13:34:38.103 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:38.090+0000 7f63e9ffb640 1 -- 172.21.15.43:0/246394760 <== mon.1 v2:172.21.15.203:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f63f0045070 con 0x7f63ec050330 2023-08-09T13:34:38.104 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:38.090+0000 7f63e9ffb640 1 -- 172.21.15.43:0/246394760 <== mon.1 v2:172.21.15.203:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f63f0036e30 con 0x7f63ec050330 2023-08-09T13:34:38.104 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:38.090+0000 7f63e9ffb640 1 -- 172.21.15.43:0/246394760 <== mon.1 v2:172.21.15.203:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f63f0034ba0 con 0x7f63ec050330 2023-08-09T13:34:38.104 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:38.090+0000 7f63fb5e8640 1 -- 172.21.15.43:0/246394760 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f63ec04e3b0 con 0x7f63ec050330 2023-08-09T13:34:38.104 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:38.090+0000 7f63fb5e8640 1 -- 172.21.15.43:0/246394760 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f63ec04a710 con 0x7f63ec050330 2023-08-09T13:34:38.105 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:38.085+0000 7f58686b8640 1 -- start start 2023-08-09T13:34:38.105 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:38.091+0000 7fdf9659c640 1 --2- 172.21.15.43:0/2621260210 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fdf800038c0 0x7fdf80005050 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7fdf8800c1e0 tx=0x7fdf80006080 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b8033c73d8e19f79 server_cookie=0 in_seq=0 out_seq=0 2023-08-09T13:34:38.105 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:38.091+0000 7fdf9559a640 1 -- 172.21.15.43:0/2621260210 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdf88005ad0 con 0x7fdf800038c0 2023-08-09T13:34:38.106 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:38.091+0000 7fdf9559a640 1 -- 172.21.15.43:0/2621260210 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdf88005c30 con 0x7fdf800038c0 2023-08-09T13:34:38.106 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:38.091+0000 7fdf9559a640 1 -- 172.21.15.43:0/2621260210 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdf8802ca70 con 0x7fdf800038c0 2023-08-09T13:34:38.106 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:38.088+0000 7f58686b8640 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7f58600ca3d0 0x7f58600ccb40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:34:38.106 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:38.088+0000 7f58686b8640 1 -- --> v1:172.21.15.43:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f586006fbf0 con 0x7f58600c84d0 2023-08-09T13:34:38.107 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:38.092+0000 7f02dd2e5640 1 -- 172.21.15.43:0/375074373 --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f02b8005250 con 0x7f02d0005fe0 2023-08-09T13:34:38.107 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:38.092+0000 7fdf9cd86640 1 -- 172.21.15.43:0/2621260210 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fdf800038c0 msgr2=0x7fdf80005050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:34:38.107 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:38.092+0000 7fdf9cd86640 1 --2- 172.21.15.43:0/2621260210 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fdf800038c0 0x7fdf80005050 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7fdf8800c1e0 tx=0x7fdf80006080 comp rx=0 tx=0).stop 2023-08-09T13:34:38.107 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:38.088+0000 7f58686b8640 1 -- --> v1:172.21.15.203:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5860071060 con 0x7f58600c9070 2023-08-09T13:34:38.108 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:38.088+0000 7f58686b8640 1 -- --> v2:172.21.15.203:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f58600ca220 con 0x7f58600ca3d0 2023-08-09T13:34:38.108 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:38.089+0000 7f5865c53640 1 --1- >> v1:172.21.15.203:6789/0 conn(0x7f58600c9070 0x7f58600cc430 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.203:6789/0 says I am v1:172.21.15.43:33854/0 (socket says 172.21.15.43:33854) 2023-08-09T13:34:38.108 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:38.090+0000 7f5865c53640 1 -- 172.21.15.43:0/2014758962 learned_addr learned my addr 172.21.15.43:0/2014758962 (peer_addr_for_me v1:172.21.15.43:0/0) 2023-08-09T13:34:38.108 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:38.090+0000 7f5866c55640 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7f58600ca3d0 0x7f58600ccb40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:34:38.108 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:38.091+0000 7f5865452640 1 -- 172.21.15.43:0/2014758962 <== mon.1 v1:172.21.15.203:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3526366133 0 0) 0x7f5860071060 con 0x7f58600c9070 2023-08-09T13:34:38.108 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:38.091+0000 7f5865452640 1 -- 172.21.15.43:0/2014758962 --> v1:172.21.15.203:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f584c003520 con 0x7f58600c9070 2023-08-09T13:34:38.109 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:38.091+0000 7f5866c55640 1 -- 172.21.15.43:0/2014758962 >> v1:172.21.15.203:6789/0 conn(0x7f58600c9070 legacy=0x7f58600cc430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:34:38.109 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:38.091+0000 7f5866c55640 1 -- 172.21.15.43:0/2014758962 >> v1:172.21.15.43:6789/0 conn(0x7f58600c84d0 legacy=0x7f58600c88b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:34:38.109 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:38.091+0000 7f5866c55640 1 -- 172.21.15.43:0/2014758962 --> v2:172.21.15.203:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5860071300 con 0x7f58600ca3d0 2023-08-09T13:34:38.109 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:38.095+0000 7f63e9ffb640 1 -- 172.21.15.43:0/246394760 <== mon.1 v2:172.21.15.203:3300/0 4 ==== mgrmap(e 5) v1 ==== 88341+0+0 (secure 0 0 0) 0x7f63f0034d00 con 0x7f63ec050330 2023-08-09T13:34:38.109 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:38.095+0000 7fdf9cd86640 1 -- 172.21.15.43:0/2621260210 shutdown_connections 2023-08-09T13:34:38.110 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:38.095+0000 7fdf9cd86640 1 --2- 172.21.15.43:0/2621260210 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fdf800038c0 0x7fdf80005050 unknown :-1 s=CLOSED pgs=182 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:34:38.110 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:38.095+0000 7fdf9cd86640 1 -- 172.21.15.43:0/2621260210 >> 172.21.15.43:0/2621260210 conn(0x7fdf980af970 msgr2=0x7fdf980afd80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-09T13:34:38.110 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:38.095+0000 7f5866c55640 1 --2- 172.21.15.43:0/2014758962 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f58600ca3d0 0x7f58600ccb40 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7f5858009d90 tx=0x7f585802be40 comp rx=0 tx=0).ready entity=mon.1 client_cookie=25c55cebe99a4a43 server_cookie=0 in_seq=0 out_seq=0 2023-08-09T13:34:38.110 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:38.096+0000 7fdf9cd86640 1 -- 172.21.15.43:0/2621260210 shutdown_connections 2023-08-09T13:34:38.111 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:38.095+0000 7f63df7fe640 1 -- 172.21.15.43:0/246394760 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f63ec17cf10 con 0x7f63ec050330 2023-08-09T13:34:38.111 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:38.095+0000 7f02d4ff9640 1 -- 172.21.15.43:0/375074373 <== mon.2 v2:172.21.15.203:3301/0 4 ==== mgrmap(e 5) v1 ==== 88341+0+0 (secure 0 0 0) 0x7f02c4007430 con 0x7f02d0005fe0 2023-08-09T13:34:38.112 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:38.097+0000 7f63e9ffb640 1 --2- 172.21.15.43:0/246394760 >> [v2:172.21.15.203:6832/830097474,v1:172.21.15.203:6833/830097474] conn(0x7f63d806a0a0 0x7f63d806c560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:34:38.112 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:38.097+0000 7f63e9ffb640 1 -- 172.21.15.43:0/246394760 <== mon.1 v2:172.21.15.203:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6813+0+0 (secure 0 0 0) 0x7f63f00ab080 con 0x7f63ec050330 2023-08-09T13:34:38.112 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:38.097+0000 7f63f9384640 1 --2- 172.21.15.43:0/246394760 >> [v2:172.21.15.203:6832/830097474,v1:172.21.15.203:6833/830097474] conn(0x7f63d806a0a0 0x7f63d806c560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:34:38.112 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:38.098+0000 7f5865452640 1 -- 172.21.15.43:0/2014758962 <== mon.1 v2:172.21.15.203:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f585802cbf0 con 0x7f58600ca3d0 2023-08-09T13:34:38.113 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:38.098+0000 7f5865452640 1 -- 172.21.15.43:0/2014758962 <== mon.1 v2:172.21.15.203:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f585802cd50 con 0x7f58600ca3d0 2023-08-09T13:34:38.113 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:38.098+0000 7f5865452640 1 -- 172.21.15.43:0/2014758962 <== mon.1 v2:172.21.15.203:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5858034660 con 0x7f58600ca3d0 2023-08-09T13:34:38.113 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:38.098+0000 7fdf9cd86640 1 -- 172.21.15.43:0/2621260210 wait complete. 2023-08-09T13:34:38.113 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:38.099+0000 7f63f9384640 1 --2- 172.21.15.43:0/246394760 >> [v2:172.21.15.203:6832/830097474,v1:172.21.15.203:6833/830097474] conn(0x7f63d806a0a0 0x7f63d806c560 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7f63e4009a30 tx=0x7f63e4009190 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-09T13:34:38.114 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:38.099+0000 7f58686b8640 1 -- 172.21.15.43:0/2014758962 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f58600ca3d0 msgr2=0x7f58600ccb40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:34:38.114 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:38.099+0000 7f58686b8640 1 --2- 172.21.15.43:0/2014758962 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f58600ca3d0 0x7f58600ccb40 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7f5858009d90 tx=0x7f585802be40 comp rx=0 tx=0).stop 2023-08-09T13:34:38.114 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:38.099+0000 7f02d4ff9640 1 --2- 172.21.15.43:0/375074373 >> [v2:172.21.15.203:6832/830097474,v1:172.21.15.203:6833/830097474] conn(0x7f02bc06a150 0x7f02bc06c610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:34:38.114 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:38.099+0000 7f02d4ff9640 1 -- 172.21.15.43:0/375074373 <== mon.2 v2:172.21.15.203:3301/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6813+0+0 (secure 0 0 0) 0x7f02c4087cc0 con 0x7f02d0005fe0 2023-08-09T13:34:38.115 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:38.099+0000 7f58686b8640 1 -- 172.21.15.43:0/2014758962 shutdown_connections 2023-08-09T13:34:38.115 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:38.099+0000 7f58686b8640 1 --2- 172.21.15.43:0/2014758962 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f58600ca3d0 0x7f58600ccb40 unknown :-1 s=CLOSED pgs=102 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:34:38.115 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:38.099+0000 7f58686b8640 1 -- 172.21.15.43:0/2014758962 >> 172.21.15.43:0/2014758962 conn(0x7f58600cde80 msgr2=0x7f58600ce290 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-09T13:34:38.115 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:38.099+0000 7f58686b8640 1 -- 172.21.15.43:0/2014758962 shutdown_connections 2023-08-09T13:34:38.116 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:38.100+0000 7f02d77fe640 1 --2- 172.21.15.43:0/375074373 >> [v2:172.21.15.203:6832/830097474,v1:172.21.15.203:6833/830097474] conn(0x7f02bc06a150 0x7f02bc06c610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:34:38.116 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:38.100+0000 7f58686b8640 1 -- 172.21.15.43:0/2014758962 wait complete. 2023-08-09T13:34:38.116 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:38.101+0000 7fdf9cd86640 1 Processor -- start 2023-08-09T13:34:38.117 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:38.101+0000 7f02d77fe640 1 --2- 172.21.15.43:0/375074373 >> [v2:172.21.15.203:6832/830097474,v1:172.21.15.203:6833/830097474] conn(0x7f02bc06a150 0x7f02bc06c610 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7f02d0103d60 tx=0x7f02cc009190 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-09T13:34:38.117 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:38.101+0000 7fdf9cd86640 1 -- start start 2023-08-09T13:34:38.117 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:38.103+0000 7fdf9cd86640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fdf800038c0 0x7fdf98207b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:34:38.118 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:38.103+0000 7f58686b8640 1 Processor -- start 2023-08-09T13:34:38.118 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:38.103+0000 7fdf9cd86640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fdf98168860 0x7fdf98169cb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:34:38.118 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:38.103+0000 7fdf9cd86640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fdf98208280 0x7fdf9820acc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:34:38.119 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:38.103+0000 7fdf9cd86640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdf982080a0 con 0x7fdf98168860 2023-08-09T13:34:38.119 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:38.103+0000 7fdf9cd86640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdf980b3c10 con 0x7fdf98208280 2023-08-09T13:34:38.119 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:38.104+0000 7fdf9cd86640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdf9820a7e0 con 0x7fdf800038c0 2023-08-09T13:34:38.119 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:38.104+0000 7fdf9659c640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fdf800038c0 0x7fdf98207b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:34:38.119 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:38.104+0000 7fdf9659c640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fdf800038c0 0x7fdf98207b60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.203:3301/0 says I am v2:172.21.15.43:45104/0 (socket says 172.21.15.43:45104) 2023-08-09T13:34:38.119 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:38.104+0000 7fdf95d9b640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fdf98168860 0x7fdf98169cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:34:38.119 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:38.104+0000 7fdf9659c640 1 -- 172.21.15.43:0/4060401720 learned_addr learned my addr 172.21.15.43:0/4060401720 (peer_addr_for_me v2:172.21.15.43:0/0) 2023-08-09T13:34:38.120 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:38.104+0000 7f58686b8640 1 -- start start 2023-08-09T13:34:38.120 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:38.104+0000 7fdf96d9d640 1 --2- 172.21.15.43:0/4060401720 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fdf98208280 0x7fdf9820acc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:34:38.120 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:38.105+0000 7f58686b8640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f58600c84d0 0x7f5860151c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:34:38.121 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:38.105+0000 7f58686b8640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f58600c9070 0x7f58600c6150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:34:38.121 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:38.105+0000 7f58686b8640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f58600ca3d0 0x7f5860152f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:34:38.121 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:38.105+0000 7f58686b8640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f58600c5cc0 con 0x7f58600c9070 2023-08-09T13:34:38.121 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:38.105+0000 7f58686b8640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f58600c50d0 con 0x7f58600ca3d0 2023-08-09T13:34:38.121 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:38.105+0000 7f58686b8640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f58600c5390 con 0x7f58600c84d0 2023-08-09T13:34:38.122 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:38.105+0000 7fdf9659c640 1 -- 172.21.15.43:0/4060401720 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fdf98208280 msgr2=0x7fdf9820acc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:34:38.122 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:38.105+0000 7fdf9659c640 1 --2- 172.21.15.43:0/4060401720 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fdf98208280 0x7fdf9820acc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:34:38.122 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:38.105+0000 7fdf9659c640 1 -- 172.21.15.43:0/4060401720 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fdf98168860 msgr2=0x7fdf98169cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:34:38.122 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:38.105+0000 7fdf9659c640 1 --2- 172.21.15.43:0/4060401720 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fdf98168860 0x7fdf98169cb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:34:38.122 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:38.105+0000 7fdf9659c640 1 -- 172.21.15.43:0/4060401720 --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdf980b3370 con 0x7fdf800038c0 2023-08-09T13:34:38.122 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:38.105+0000 7fdf95d9b640 1 --2- 172.21.15.43:0/4060401720 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fdf98168860 0x7fdf98169cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-08-09T13:34:38.123 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:38.105+0000 7f5865c53640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f58600c9070 0x7f58600c6150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:34:38.123 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:38.106+0000 7f5865c53640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f58600c9070 0x7f58600c6150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:51902/0 (socket says 172.21.15.43:51902) 2023-08-09T13:34:38.123 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:38.106+0000 7f5865c53640 1 -- 172.21.15.43:0/2825405935 learned_addr learned my addr 172.21.15.43:0/2825405935 (peer_addr_for_me v2:172.21.15.43:0/0) 2023-08-09T13:34:38.123 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:38.106+0000 7f5866454640 1 --2- 172.21.15.43:0/2825405935 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f58600c84d0 0x7f5860151c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:34:38.124 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:38.106+0000 7f5866c55640 1 --2- 172.21.15.43:0/2825405935 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f58600ca3d0 0x7f5860152f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:34:38.124 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:38.106+0000 7f5865c53640 1 -- 172.21.15.43:0/2825405935 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f58600c84d0 msgr2=0x7f5860151c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:34:38.124 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:38.106+0000 7f5865c53640 1 --2- 172.21.15.43:0/2825405935 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f58600c84d0 0x7f5860151c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:34:38.124 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:38.106+0000 7f5865c53640 1 -- 172.21.15.43:0/2825405935 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f58600ca3d0 msgr2=0x7f5860152f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:34:38.124 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:38.106+0000 7f5865c53640 1 --2- 172.21.15.43:0/2825405935 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f58600ca3d0 0x7f5860152f00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:34:38.124 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:38.106+0000 7f5865c53640 1 -- 172.21.15.43:0/2825405935 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5858009a70 con 0x7f58600c9070 2023-08-09T13:34:38.124 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:38.106+0000 7f5866c55640 1 --2- 172.21.15.43:0/2825405935 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f58600ca3d0 0x7f5860152f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-08-09T13:34:38.125 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:38.106+0000 7fdf9659c640 1 --2- 172.21.15.43:0/4060401720 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fdf800038c0 0x7fdf98207b60 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7fdf8800c1e0 tx=0x7fdf88001930 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-09T13:34:38.125 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:38.107+0000 7f5865c53640 1 --2- 172.21.15.43:0/2825405935 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f58600c9070 0x7f58600c6150 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7f585c01d520 tx=0x7f585c01d9e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-09T13:34:38.126 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:38.107+0000 7f5852ffd640 1 -- 172.21.15.43:0/2825405935 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f585c0184e0 con 0x7f58600c9070 2023-08-09T13:34:38.126 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:38.107+0000 7f5852ffd640 1 -- 172.21.15.43:0/2825405935 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f585c018640 con 0x7f58600c9070 2023-08-09T13:34:38.126 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:38.107+0000 7f5852ffd640 1 -- 172.21.15.43:0/2825405935 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f585c0159f0 con 0x7f58600c9070 2023-08-09T13:34:38.126 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:38.108+0000 7f58686b8640 1 -- 172.21.15.43:0/2825405935 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f58600c5670 con 0x7f58600c9070 2023-08-09T13:34:38.126 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:38.109+0000 7f58686b8640 1 -- 172.21.15.43:0/2825405935 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5860152180 con 0x7f58600c9070 2023-08-09T13:34:38.126 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:38.111+0000 7f58686b8640 1 -- 172.21.15.43:0/2825405935 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f58600c5e70 con 0x7f58600c9070 2023-08-09T13:34:38.127 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:38.111+0000 7fdf86ffd640 1 -- 172.21.15.43:0/4060401720 <== mon.2 v2:172.21.15.203:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdf8802cc60 con 0x7fdf800038c0 2023-08-09T13:34:38.127 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:38.112+0000 7fdf9cd86640 1 -- 172.21.15.43:0/4060401720 --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdf980b3650 con 0x7fdf800038c0 2023-08-09T13:34:38.127 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:38.112+0000 7fdf86ffd640 1 -- 172.21.15.43:0/4060401720 <== mon.2 v2:172.21.15.203:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdf8803a070 con 0x7fdf800038c0 2023-08-09T13:34:38.127 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:38.112+0000 7fdf86ffd640 1 -- 172.21.15.43:0/4060401720 <== mon.2 v2:172.21.15.203:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdf88007070 con 0x7fdf800038c0 2023-08-09T13:34:38.127 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:38.112+0000 7f5852ffd640 1 -- 172.21.15.43:0/2825405935 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 5) v1 ==== 88341+0+0 (secure 0 0 0) 0x7f585c015b50 con 0x7f58600c9070 2023-08-09T13:34:38.128 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:38.113+0000 7f63e9ffb640 1 -- 172.21.15.43:0/246394760 <== mon.1 v2:172.21.15.203:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178066 (secure 0 0 0) 0x7f63f00dd7b0 con 0x7f63ec050330 2023-08-09T13:34:38.128 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:38.112+0000 7f02d4ff9640 1 -- 172.21.15.43:0/375074373 <== mon.2 v2:172.21.15.203:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178066 (secure 0 0 0) 0x7f02c40531e0 con 0x7f02d0005fe0 2023-08-09T13:34:38.128 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:38.113+0000 7fdf9cd86640 1 -- 172.21.15.43:0/4060401720 --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdf9816a1f0 con 0x7fdf800038c0 2023-08-09T13:34:38.129 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:38.113+0000 7f5852ffd640 1 --2- 172.21.15.43:0/2825405935 >> [v2:172.21.15.203:6832/830097474,v1:172.21.15.203:6833/830097474] conn(0x7f584406a0d0 0x7f584406c590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:34:38.129 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:38.114+0000 7f5852ffd640 1 -- 172.21.15.43:0/2825405935 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6813+0+0 (secure 0 0 0) 0x7f585c098d10 con 0x7f58600c9070 2023-08-09T13:34:38.129 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:38.117+0000 7fdf86ffd640 1 -- 172.21.15.43:0/4060401720 <== mon.2 v2:172.21.15.203:3301/0 4 ==== mgrmap(e 5) v1 ==== 88341+0+0 (secure 0 0 0) 0x7fdf88032070 con 0x7fdf800038c0 2023-08-09T13:34:38.129 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:38.119+0000 7fdf86ffd640 1 --2- 172.21.15.43:0/4060401720 >> [v2:172.21.15.203:6832/830097474,v1:172.21.15.203:6833/830097474] conn(0x7fdf7806a050 0x7fdf7806c510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:34:38.130 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:38.119+0000 7fdf86ffd640 1 -- 172.21.15.43:0/4060401720 <== mon.2 v2:172.21.15.203:3301/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6813+0+0 (secure 0 0 0) 0x7fdf880a9d60 con 0x7fdf800038c0 2023-08-09T13:34:38.130 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:38.120+0000 7fdf95d9b640 1 --2- 172.21.15.43:0/4060401720 >> [v2:172.21.15.203:6832/830097474,v1:172.21.15.203:6833/830097474] conn(0x7fdf7806a050 0x7fdf7806c510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:34:38.130 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:38.120+0000 7fdf84ff9640 1 -- 172.21.15.43:0/4060401720 --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdf9820a9a0 con 0x7fdf800038c0 2023-08-09T13:34:38.130 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:38.115+0000 7f5866454640 1 --2- 172.21.15.43:0/2825405935 >> [v2:172.21.15.203:6832/830097474,v1:172.21.15.203:6833/830097474] conn(0x7f584406a0d0 0x7f584406c590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:34:38.130 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:38.117+0000 7f5866454640 1 --2- 172.21.15.43:0/2825405935 >> [v2:172.21.15.203:6832/830097474,v1:172.21.15.203:6833/830097474] conn(0x7f584406a0d0 0x7f584406c590 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7f5860163ec0 tx=0x7f5854009040 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-09T13:34:38.131 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:38.121+0000 7fdf95d9b640 1 --2- 172.21.15.43:0/4060401720 >> [v2:172.21.15.203:6832/830097474,v1:172.21.15.203:6833/830097474] conn(0x7fdf7806a050 0x7fdf7806c510 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7fdf980c7af0 tx=0x7fdf8c0097c0 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-09T13:34:38.131 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:38.124+0000 7f5852ffd640 1 -- 172.21.15.43:0/2825405935 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178066 (secure 0 0 0) 0x7f58600c5e70 con 0x7f58600c9070 2023-08-09T13:34:38.136 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:38.134+0000 7fdf86ffd640 1 -- 172.21.15.43:0/4060401720 <== mon.2 v2:172.21.15.203:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178066 (secure 0 0 0) 0x7fdf88075200 con 0x7fdf800038c0 2023-08-09T13:34:38.143 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:38.138+0000 7ff7a18c2640 1 Processor -- start 2023-08-09T13:34:38.143 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:38.139+0000 7ff7a18c2640 1 -- start start 2023-08-09T13:34:38.144 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:38.141+0000 7ff7a18c2640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7ff79403a970 0x7ff794100da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:34:38.144 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:38.141+0000 7ff7a18c2640 1 -- --> v1:172.21.15.43:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff7941012e0 con 0x7ff794053400 2023-08-09T13:34:38.144 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:38.141+0000 7ff7a18c2640 1 -- --> v1:172.21.15.203:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff794053c80 con 0x7ff794052fc0 2023-08-09T13:34:38.145 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:38.141+0000 7ff7a18c2640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff794054a40 con 0x7ff79403a970 2023-08-09T13:34:38.145 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:38.142+0000 7ff7a10c1640 1 --1- >> v1:172.21.15.43:6789/0 conn(0x7ff794053400 0x7ff794006ff0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.43:6789/0 says I am v1:172.21.15.43:35692/0 (socket says 172.21.15.43:35692) 2023-08-09T13:34:38.145 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:38.142+0000 7ff7a10c1640 1 -- 172.21.15.43:0/2855052223 learned_addr learned my addr 172.21.15.43:0/2855052223 (peer_addr_for_me v1:172.21.15.43:0/0) 2023-08-09T13:34:38.145 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:38.143+0000 7ff79b7fe640 1 -- 172.21.15.43:0/2855052223 <== mon.0 v1:172.21.15.43:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1103160099 0 0) 0x7ff7941012e0 con 0x7ff794053400 2023-08-09T13:34:38.145 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:38.143+0000 7ff79b7fe640 1 -- 172.21.15.43:0/2855052223 --> v1:172.21.15.43:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ff78c003520 con 0x7ff794053400 2023-08-09T13:34:38.146 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:38.143+0000 7ff7a08c0640 1 --2- 172.21.15.43:0/2855052223 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7ff79403a970 0x7ff794100da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:34:38.146 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:38.143+0000 7ff79b7fe640 1 -- 172.21.15.43:0/2855052223 <== mon.1 v1:172.21.15.203:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3362513432 0 0) 0x7ff794053c80 con 0x7ff794052fc0 2023-08-09T13:34:38.146 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:38.143+0000 7ff79b7fe640 1 -- 172.21.15.43:0/2855052223 --> v1:172.21.15.203:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ff7941012e0 con 0x7ff794052fc0 2023-08-09T13:34:38.146 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:38.143+0000 7ff79b7fe640 1 -- 172.21.15.43:0/2855052223 <== mon.0 v1:172.21.15.43:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3538531883 0 0) 0x7ff78c003520 con 0x7ff794053400 2023-08-09T13:34:38.147 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:38.143+0000 7ff79b7fe640 1 -- 172.21.15.43:0/2855052223 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7ff79403a970 msgr2=0x7ff794100da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-09T13:34:38.147 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:38.143+0000 7ff79b7fe640 1 --2- 172.21.15.43:0/2855052223 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7ff79403a970 0x7ff794100da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:34:38.147 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:38.143+0000 7ff79b7fe640 1 -- 172.21.15.43:0/2855052223 >> v1:172.21.15.203:6789/0 conn(0x7ff794052fc0 legacy=0x7ff794037d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:34:38.147 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:38.143+0000 7ff79b7fe640 1 -- 172.21.15.43:0/2855052223 --> v1:172.21.15.43:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff794053f60 con 0x7ff794053400 2023-08-09T13:34:38.148 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:38.144+0000 7ff79b7fe640 1 -- 172.21.15.43:0/2855052223 <== mon.0 v1:172.21.15.43:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1285150740 0 0) 0x7ff79c04e390 con 0x7ff794053400 2023-08-09T13:34:38.148 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:38.144+0000 7ff79b7fe640 1 -- 172.21.15.43:0/2855052223 >> v1:172.21.15.43:6789/0 conn(0x7ff794053400 legacy=0x7ff794006ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:34:38.148 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:38.144+0000 7ff79b7fe640 1 --2- 172.21.15.43:0/2855052223 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff78c003bb0 0x7ff78c003f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:34:38.148 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:38.144+0000 7ff79b7fe640 1 --2- 172.21.15.43:0/2855052223 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7ff78c0047c0 0x7ff78c004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:34:38.148 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:38.144+0000 7ff79b7fe640 1 --2- 172.21.15.43:0/2855052223 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7ff79403a970 0x7ff78c005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:34:38.149 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:38.144+0000 7ff79b7fe640 1 -- 172.21.15.43:0/2855052223 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff794054a40 con 0x7ff78c003bb0 2023-08-09T13:34:38.149 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:38.144+0000 7ff79b7fe640 1 -- 172.21.15.43:0/2855052223 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff78c00ba30 con 0x7ff78c0047c0 2023-08-09T13:34:38.149 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:38.144+0000 7ff79b7fe640 1 -- 172.21.15.43:0/2855052223 --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff78c00bbc0 con 0x7ff79403a970 2023-08-09T13:34:38.149 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:38.145+0000 7ff7a08c0640 1 --2- 172.21.15.43:0/2855052223 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff78c003bb0 0x7ff78c003f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:34:38.149 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:38.145+0000 7ff79bfff640 1 --2- 172.21.15.43:0/2855052223 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7ff78c0047c0 0x7ff78c004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:34:38.149 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:38.145+0000 7ff7a10c1640 1 --2- 172.21.15.43:0/2855052223 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7ff79403a970 0x7ff78c005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:34:38.150 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:38.146+0000 7ff79bfff640 1 -- 172.21.15.43:0/2855052223 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7ff79403a970 msgr2=0x7ff78c005180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-09T13:34:38.150 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:38.146+0000 7ff79bfff640 1 --2- 172.21.15.43:0/2855052223 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7ff79403a970 0x7ff78c005180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:34:38.150 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:38.146+0000 7ff79bfff640 1 -- 172.21.15.43:0/2855052223 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff78c003bb0 msgr2=0x7ff78c003f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-09T13:34:38.150 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:38.146+0000 7ff79bfff640 1 --2- 172.21.15.43:0/2855052223 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff78c003bb0 0x7ff78c003f90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:34:38.150 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:38.146+0000 7ff79bfff640 1 -- 172.21.15.43:0/2855052223 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff78c003520 con 0x7ff78c0047c0 2023-08-09T13:34:38.151 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:38.146+0000 7ff7a10c1640 1 --2- 172.21.15.43:0/2855052223 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7ff79403a970 0x7ff78c005180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-08-09T13:34:38.151 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:38.146+0000 7ff7a08c0640 1 --2- 172.21.15.43:0/2855052223 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff78c003bb0 0x7ff78c003f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-08-09T13:34:38.151 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:38.147+0000 7f8333256640 1 Processor -- start 2023-08-09T13:34:38.151 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:38.148+0000 7ff79bfff640 1 --2- 172.21.15.43:0/2855052223 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7ff78c0047c0 0x7ff78c004c40 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7ff7900017d0 tx=0x7ff78c0057f0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=896405a2e5ce6297 server_cookie=0 in_seq=0 out_seq=0 2023-08-09T13:34:38.152 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:38.148+0000 7ff79b7fe640 1 -- 172.21.15.43:0/2855052223 <== mon.1 v2:172.21.15.203:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff7900052d0 con 0x7ff78c0047c0 2023-08-09T13:34:38.152 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:38.148+0000 7ff79b7fe640 1 -- 172.21.15.43:0/2855052223 <== mon.1 v2:172.21.15.203:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff79002e050 con 0x7ff78c0047c0 2023-08-09T13:34:38.152 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:38.148+0000 7ff79b7fe640 1 -- 172.21.15.43:0/2855052223 <== mon.1 v2:172.21.15.203:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff7900035f0 con 0x7ff78c0047c0 2023-08-09T13:34:38.153 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:38.151+0000 7ff7a18c2640 1 -- 172.21.15.43:0/2855052223 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7ff78c0047c0 msgr2=0x7ff78c004c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:34:38.154 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:38.151+0000 7ff7a18c2640 1 --2- 172.21.15.43:0/2855052223 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7ff78c0047c0 0x7ff78c004c40 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7ff7900017d0 tx=0x7ff78c0057f0 comp rx=0 tx=0).stop 2023-08-09T13:34:38.154 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:38.148+0000 7f8333256640 1 -- start start 2023-08-09T13:34:38.154 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:38.150+0000 7f8333256640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f832c0c0e60 0x7f832c160e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:34:38.154 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:38.150+0000 7f8333256640 1 -- --> v1:172.21.15.43:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f832c1614b0 con 0x7f832c0c9620 2023-08-09T13:34:38.154 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:38.150+0000 7f8333256640 1 -- --> v1:172.21.15.203:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f832c161790 con 0x7f832c149e10 2023-08-09T13:34:38.155 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:38.150+0000 7f8333256640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f832c164540 con 0x7f832c0c0e60 2023-08-09T13:34:38.155 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:38.151+0000 7f8332a55640 1 --1- >> v1:172.21.15.43:6789/0 conn(0x7f832c0c9620 0x7f832c164960 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.43:6789/0 says I am v1:172.21.15.43:35700/0 (socket says 172.21.15.43:35700) 2023-08-09T13:34:38.155 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:38.151+0000 7f8332a55640 1 -- 172.21.15.43:0/2179001740 learned_addr learned my addr 172.21.15.43:0/2179001740 (peer_addr_for_me v1:172.21.15.43:0/0) 2023-08-09T13:34:38.155 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:38.151+0000 7ff7a10c1640 1 -- 172.21.15.43:0/2855052223 reap_dead start 2023-08-09T13:34:38.156 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:38.152+0000 7ff7a18c2640 1 -- 172.21.15.43:0/2855052223 shutdown_connections 2023-08-09T13:34:38.156 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:38.152+0000 7ff7a18c2640 1 -- 172.21.15.43:0/2855052223 >> 172.21.15.43:0/2855052223 conn(0x7ff794054c10 msgr2=0x7ff794055020 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-09T13:34:38.156 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:38.152+0000 7ff7a18c2640 1 -- 172.21.15.43:0/2855052223 shutdown_connections 2023-08-09T13:34:38.156 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:38.152+0000 7ff7a18c2640 1 -- 172.21.15.43:0/2855052223 wait complete. 2023-08-09T13:34:38.157 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:38.152+0000 7f8331252640 1 -- 172.21.15.43:0/2179001740 <== mon.0 v1:172.21.15.43:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 131273000 0 0) 0x7f832c1614b0 con 0x7f832c0c9620 2023-08-09T13:34:38.157 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:38.152+0000 7f8331252640 1 -- 172.21.15.43:0/2179001740 --> v1:172.21.15.43:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f8318003520 con 0x7f832c0c9620 2023-08-09T13:34:38.157 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:38.153+0000 7f8331252640 1 -- 172.21.15.43:0/2179001740 <== mon.0 v1:172.21.15.43:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1254637357 0 0) 0x7f8318003520 con 0x7f832c0c9620 2023-08-09T13:34:38.157 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:38.153+0000 7f8331252640 1 -- 172.21.15.43:0/2179001740 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f832c0c0e60 msgr2=0x7f832c160e70 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2023-08-09T13:34:38.157 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:38.153+0000 7f8331252640 1 --2- 172.21.15.43:0/2179001740 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f832c0c0e60 0x7f832c160e70 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:34:38.157 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:38.153+0000 7f8331252640 1 -- 172.21.15.43:0/2179001740 >> v1:172.21.15.203:6789/0 conn(0x7f832c149e10 legacy=0x7f832c0c1240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-09T13:34:38.158 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:38.153+0000 7f8331252640 1 -- 172.21.15.43:0/2179001740 --> v1:172.21.15.43:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f832c161a70 con 0x7f832c0c9620 2023-08-09T13:34:38.158 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:38.153+0000 7f8331252640 1 -- 172.21.15.43:0/2179001740 <== mon.0 v1:172.21.15.43:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1285150740 0 0) 0x7f83240019f0 con 0x7f832c0c9620 2023-08-09T13:34:38.158 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:38.153+0000 7f8331252640 1 -- 172.21.15.43:0/2179001740 >> v1:172.21.15.43:6789/0 conn(0x7f832c0c9620 legacy=0x7f832c164960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:34:38.158 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:38.153+0000 7f8331252640 1 --2- 172.21.15.43:0/2179001740 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f83180039a0 0x7f8318003da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:34:38.158 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:38.153+0000 7f8331252640 1 --2- 172.21.15.43:0/2179001740 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f83180047c0 0x7f8318004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:34:38.158 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:38.153+0000 7f8331252640 1 --2- 172.21.15.43:0/2179001740 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f83180051d0 0x7f831800b980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:34:38.158 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:38.153+0000 7f8331252640 1 -- 172.21.15.43:0/2179001740 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f832c164540 con 0x7f83180051d0 2023-08-09T13:34:38.158 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:38.153+0000 7f8331252640 1 -- 172.21.15.43:0/2179001740 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f831800c100 con 0x7f83180047c0 2023-08-09T13:34:38.159 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:38.153+0000 7f8331252640 1 -- 172.21.15.43:0/2179001740 --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f831800c2c0 con 0x7f83180039a0 2023-08-09T13:34:38.159 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:38.154+0000 7f8332a55640 1 --2- 172.21.15.43:0/2179001740 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f83180051d0 0x7f831800b980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:34:38.159 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:38.155+0000 7ff7a18c2640 1 Processor -- start 2023-08-09T13:34:38.159 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:38.155+0000 7ff7a18c2640 1 -- start start 2023-08-09T13:34:38.160 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:38.154+0000 7f8332a55640 1 -- 172.21.15.43:0/2179001740 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f83180039a0 msgr2=0x7f8318003da0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-08-09T13:34:38.160 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:38.154+0000 7f8332a55640 1 --2- 172.21.15.43:0/2179001740 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f83180039a0 0x7f8318003da0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:34:38.160 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:38.154+0000 7f8332a55640 1 -- 172.21.15.43:0/2179001740 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f83180047c0 msgr2=0x7f8318004c40 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-08-09T13:34:38.160 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:38.154+0000 7f8332a55640 1 --2- 172.21.15.43:0/2179001740 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f83180047c0 0x7f8318004c40 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:34:38.161 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:38.154+0000 7f8332a55640 1 -- 172.21.15.43:0/2179001740 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f832c161a70 con 0x7f83180051d0 2023-08-09T13:34:38.161 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:38.156+0000 7f8332a55640 1 --2- 172.21.15.43:0/2179001740 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f83180051d0 0x7f831800b980 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7f832401b8d0 tx=0x7f83180057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d1c21b9e76417ef7 server_cookie=0 in_seq=0 out_seq=0 2023-08-09T13:34:38.161 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:38.157+0000 7ff7a18c2640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff78c0047c0 0x7ff7940f5030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:34:38.161 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:38.157+0000 7f8331252640 1 -- 172.21.15.43:0/2179001740 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8324045630 con 0x7f83180051d0 2023-08-09T13:34:38.162 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:38.157+0000 7f8331252640 1 -- 172.21.15.43:0/2179001740 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8324040070 con 0x7f83180051d0 2023-08-09T13:34:38.162 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:38.157+0000 7f8331252640 1 -- 172.21.15.43:0/2179001740 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8324045a30 con 0x7f83180051d0 2023-08-09T13:34:38.162 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:38.158+0000 7f8333256640 1 -- 172.21.15.43:0/2179001740 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f83180051d0 msgr2=0x7f831800b980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:34:38.162 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:38.158+0000 7f8333256640 1 --2- 172.21.15.43:0/2179001740 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f83180051d0 0x7f831800b980 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7f832401b8d0 tx=0x7f83180057f0 comp rx=0 tx=0).stop 2023-08-09T13:34:38.163 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:38.158+0000 7ff7a08c0640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff78c0047c0 0x7ff7940f5030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:34:38.163 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:38.158+0000 7ff7a08c0640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff78c0047c0 0x7ff7940f5030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:51924/0 (socket says 172.21.15.43:51924) 2023-08-09T13:34:38.163 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:38.158+0000 7ff7a08c0640 1 -- 172.21.15.43:0/617417946 learned_addr learned my addr 172.21.15.43:0/617417946 (peer_addr_for_me v2:172.21.15.43:0/0) 2023-08-09T13:34:38.163 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:38.158+0000 7ff7a18c2640 1 --2- 172.21.15.43:0/617417946 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7ff79404c420 0x7ff79404def0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:34:38.163 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:38.158+0000 7f8332a55640 1 -- 172.21.15.43:0/2179001740 reap_dead start 2023-08-09T13:34:38.164 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:38.158+0000 7f8333256640 1 -- 172.21.15.43:0/2179001740 shutdown_connections 2023-08-09T13:34:38.164 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:38.158+0000 7f8333256640 1 -- 172.21.15.43:0/2179001740 >> 172.21.15.43:0/2179001740 conn(0x7f832c0cde90 msgr2=0x7f832c0ce2a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-09T13:34:38.164 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:38.159+0000 7ff7a18c2640 1 --2- 172.21.15.43:0/617417946 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7ff79404e430 0x7ff794049880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:34:38.165 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:38.159+0000 7f8333256640 1 -- 172.21.15.43:0/2179001740 shutdown_connections 2023-08-09T13:34:38.165 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:38.159+0000 7f8333256640 1 -- 172.21.15.43:0/2179001740 wait complete. 2023-08-09T13:34:38.165 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:38.159+0000 7ff79bfff640 1 --2- 172.21.15.43:0/617417946 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7ff79404c420 0x7ff79404def0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:34:38.166 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:38.159+0000 7ff7a18c2640 1 -- 172.21.15.43:0/617417946 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff7940f28f0 con 0x7ff78c0047c0 2023-08-09T13:34:38.166 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:38.160+0000 7ff7a18c2640 1 -- 172.21.15.43:0/617417946 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff7940f20e0 con 0x7ff79404c420 2023-08-09T13:34:38.166 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:38.160+0000 7ff7a10c1640 1 --2- 172.21.15.43:0/617417946 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7ff79404e430 0x7ff794049880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:34:38.166 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:38.160+0000 7ff7a18c2640 1 -- 172.21.15.43:0/617417946 --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff79404d770 con 0x7ff79404e430 2023-08-09T13:34:38.166 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:38.160+0000 7f8333256640 1 Processor -- start 2023-08-09T13:34:38.167 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:38.161+0000 7f8333256640 1 -- start start 2023-08-09T13:34:38.167 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:38.154+0000 7f1356de8640 1 Processor -- start 2023-08-09T13:34:38.167 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:38.161+0000 7ff7a08c0640 1 -- 172.21.15.43:0/617417946 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7ff79404e430 msgr2=0x7ff794049880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:34:38.168 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:38.161+0000 7ff7a08c0640 1 --2- 172.21.15.43:0/617417946 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7ff79404e430 0x7ff794049880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:34:38.168 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:38.161+0000 7ff7a08c0640 1 -- 172.21.15.43:0/617417946 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7ff79404c420 msgr2=0x7ff79404def0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:34:38.168 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:38.161+0000 7ff7a08c0640 1 --2- 172.21.15.43:0/617417946 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7ff79404c420 0x7ff79404def0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:34:38.168 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:38.161+0000 7ff7a08c0640 1 -- 172.21.15.43:0/617417946 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff7940ebdb0 con 0x7ff78c0047c0 2023-08-09T13:34:38.168 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:38.161+0000 7ff79bfff640 1 --2- 172.21.15.43:0/617417946 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7ff79404c420 0x7ff79404def0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-08-09T13:34:38.169 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:38.154+0000 7f1356de8640 1 -- start start 2023-08-09T13:34:38.169 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:38.156+0000 7f1356de8640 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7f135016d620 0x7f13500c3130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:34:38.169 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:38.156+0000 7f1356de8640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f135016cd10 0x7f13500cbc70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:34:38.169 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:38.156+0000 7f1356de8640 1 -- --> v1:172.21.15.203:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f13500c39f0 con 0x7f13500cc1b0 2023-08-09T13:34:38.169 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:38.156+0000 7f1356de8640 1 -- --> v2:172.21.15.43:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f13500c4570 con 0x7f135016cd10 2023-08-09T13:34:38.170 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:38.156+0000 7f1356de8640 1 -- --> v2:172.21.15.203:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f13500cc640 con 0x7f135016d620 2023-08-09T13:34:38.170 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:38.157+0000 7f134ffff640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f135016cd10 0x7f13500cbc70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:34:38.170 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:38.157+0000 7f134ffff640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f135016cd10 0x7f13500cbc70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:51920/0 (socket says 172.21.15.43:51920) 2023-08-09T13:34:38.170 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:38.157+0000 7f134ffff640 1 -- 172.21.15.43:0/874155189 learned_addr learned my addr 172.21.15.43:0/874155189 (peer_addr_for_me v2:172.21.15.43:0/0) 2023-08-09T13:34:38.170 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:38.162+0000 7f8333256640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f832c0ac170 0x7f832c148db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:34:38.171 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:38.159+0000 7f134f7fe640 1 -- 172.21.15.43:0/874155189 <== mon.1 v1:172.21.15.203:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3262897982 0 0) 0x7f13500c39f0 con 0x7f13500cc1b0 2023-08-09T13:34:38.171 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:38.158+0000 7f1354b84640 1 --2- 172.21.15.43:0/874155189 >> v2:172.21.15.203:3300/0 conn(0x7f135016d620 0x7f13500c3130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:34:38.171 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:38.161+0000 7f134f7fe640 1 -- 172.21.15.43:0/874155189 --> v1:172.21.15.203:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f1338005730 con 0x7f13500cc1b0 2023-08-09T13:34:38.171 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:38.162+0000 7f134ffff640 1 -- 172.21.15.43:0/874155189 >> v2:172.21.15.203:3300/0 conn(0x7f135016d620 msgr2=0x7f13500c3130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-09T13:34:38.171 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:38.162+0000 7f134ffff640 1 --2- 172.21.15.43:0/874155189 >> v2:172.21.15.203:3300/0 conn(0x7f135016d620 0x7f13500c3130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:34:38.171 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:38.162+0000 7f134ffff640 1 -- 172.21.15.43:0/874155189 >> v1:172.21.15.203:6789/0 conn(0x7f13500cc1b0 legacy=0x7f13500ca710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:34:38.172 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:38.162+0000 7f134ffff640 1 -- 172.21.15.43:0/874155189 --> v2:172.21.15.43:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f13500c3cd0 con 0x7f135016cd10 2023-08-09T13:34:38.172 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:38.162+0000 7ff7a08c0640 1 --2- 172.21.15.43:0/617417946 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff78c0047c0 0x7ff7940f5030 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7ff788004650 tx=0x7ff788003ce0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-09T13:34:38.172 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:38.162+0000 7f8333256640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f832c0ac550 0x7f832c0b1630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:34:38.172 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:38.162+0000 7f8333256640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f832c071100 0x7f832c06f740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:34:38.173 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:38.163+0000 7f8333256640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f832c099d00 con 0x7f832c0ac170 2023-08-09T13:34:38.173 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:38.163+0000 7f8333256640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f832c09a230 con 0x7f832c0ac550 2023-08-09T13:34:38.173 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:38.163+0000 7f8333256640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f832c070ee0 con 0x7f832c071100 2023-08-09T13:34:38.173 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:38.163+0000 7ff7997fa640 1 -- 172.21.15.43:0/617417946 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff788010a40 con 0x7ff78c0047c0 2023-08-09T13:34:38.173 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:38.163+0000 7ff7997fa640 1 -- 172.21.15.43:0/617417946 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff788010ba0 con 0x7ff78c0047c0 2023-08-09T13:34:38.174 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:38.163+0000 7f8332254640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f832c0ac170 0x7f832c148db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:34:38.174 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:38.163+0000 7f8332254640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f832c0ac170 0x7f832c148db0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:51932/0 (socket says 172.21.15.43:51932) 2023-08-09T13:34:38.174 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:38.163+0000 7f8332254640 1 -- 172.21.15.43:0/3804042763 learned_addr learned my addr 172.21.15.43:0/3804042763 (peer_addr_for_me v2:172.21.15.43:0/0) 2023-08-09T13:34:38.174 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:38.164+0000 7f8331a53640 1 --2- 172.21.15.43:0/3804042763 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f832c0ac550 0x7f832c0b1630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:34:38.174 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:38.164+0000 7f8332254640 1 -- 172.21.15.43:0/3804042763 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f832c071100 msgr2=0x7f832c06f740 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2023-08-09T13:34:38.174 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:38.164+0000 7f8332254640 1 --2- 172.21.15.43:0/3804042763 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f832c071100 0x7f832c06f740 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:34:38.175 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:38.164+0000 7f8332254640 1 -- 172.21.15.43:0/3804042763 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f832c0ac550 msgr2=0x7f832c0b1630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:34:38.175 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:38.164+0000 7ff7997fa640 1 -- 172.21.15.43:0/617417946 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff788010a40 con 0x7ff78c0047c0 2023-08-09T13:34:38.175 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:38.164+0000 7f8332254640 1 --2- 172.21.15.43:0/3804042763 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f832c0ac550 0x7f832c0b1630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:34:38.175 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:38.164+0000 7f8332254640 1 -- 172.21.15.43:0/3804042763 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f832c0b1b70 con 0x7f832c0ac170 2023-08-09T13:34:38.176 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:38.164+0000 7ff7a18c2640 1 -- 172.21.15.43:0/617417946 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff7940f2330 con 0x7ff78c0047c0 2023-08-09T13:34:38.176 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:38.164+0000 7f8331a53640 1 --2- 172.21.15.43:0/3804042763 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f832c0ac550 0x7f832c0b1630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-08-09T13:34:38.176 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:38.165+0000 7f134ffff640 1 --2- 172.21.15.43:0/874155189 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f135016cd10 0x7f13500cbc70 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7f13500b4270 tx=0x7f1348034900 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2010d2a0e0a5a815 server_cookie=0 in_seq=0 out_seq=0 2023-08-09T13:34:38.177 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:38.165+0000 7ff7a18c2640 1 -- 172.21.15.43:0/617417946 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff79404d9d0 con 0x7ff78c0047c0 2023-08-09T13:34:38.177 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:38.166+0000 7f8332254640 1 --2- 172.21.15.43:0/3804042763 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f832c0ac170 0x7f832c148db0 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7f832800b340 tx=0x7f832800b800 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-09T13:34:38.177 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:38.166+0000 7f134f7fe640 1 -- 172.21.15.43:0/874155189 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f134803a860 con 0x7f135016cd10 2023-08-09T13:34:38.178 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:38.166+0000 7f134f7fe640 1 -- 172.21.15.43:0/874155189 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f134803a9c0 con 0x7f135016cd10 2023-08-09T13:34:38.178 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:38.166+0000 7f134f7fe640 1 -- 172.21.15.43:0/874155189 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f134803ad90 con 0x7f135016cd10 2023-08-09T13:34:38.178 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:38.167+0000 7f1356de8640 1 -- 172.21.15.43:0/874155189 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f135016cd10 msgr2=0x7f13500cbc70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:34:38.178 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:38.167+0000 7f1356de8640 1 --2- 172.21.15.43:0/874155189 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f135016cd10 0x7f13500cbc70 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7f13500b4270 tx=0x7f1348034900 comp rx=0 tx=0).stop 2023-08-09T13:34:38.178 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:38.167+0000 7f8322ffd640 1 -- 172.21.15.43:0/3804042763 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8328011830 con 0x7f832c0ac170 2023-08-09T13:34:38.179 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:38.167+0000 7f8322ffd640 1 -- 172.21.15.43:0/3804042763 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8328011990 con 0x7f832c0ac170 2023-08-09T13:34:38.179 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:38.167+0000 7f8322ffd640 1 -- 172.21.15.43:0/3804042763 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8328011d40 con 0x7f832c0ac170 2023-08-09T13:34:38.179 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:38.168+0000 7f8333256640 1 -- 172.21.15.43:0/3804042763 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f832c099f50 con 0x7f832c0ac170 2023-08-09T13:34:38.179 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:38.168+0000 7f8333256640 1 -- 172.21.15.43:0/3804042763 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f832c070c00 con 0x7f832c0ac170 2023-08-09T13:34:38.180 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:38.170+0000 7f1356de8640 1 -- 172.21.15.43:0/874155189 shutdown_connections 2023-08-09T13:34:38.180 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:38.170+0000 7f1356de8640 1 --2- 172.21.15.43:0/874155189 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f135016cd10 0x7f13500cbc70 unknown :-1 s=CLOSED pgs=188 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:34:38.180 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:38.170+0000 7f1356de8640 1 --2- 172.21.15.43:0/874155189 >> v2:172.21.15.203:3300/0 conn(0x7f135016d620 0x7f13500c3130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:34:38.180 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:38.170+0000 7f1356de8640 1 -- 172.21.15.43:0/874155189 >> 172.21.15.43:0/874155189 conn(0x7f13500afcb0 msgr2=0x7f13500b0090 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-09T13:34:38.180 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:38.170+0000 7f1356de8640 1 -- 172.21.15.43:0/874155189 shutdown_connections 2023-08-09T13:34:38.181 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:38.169+0000 7ff7997fa640 1 -- 172.21.15.43:0/617417946 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 5) v1 ==== 88341+0+0 (secure 0 0 0) 0x7ff788010d10 con 0x7ff78c0047c0 2023-08-09T13:34:38.181 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:38.171+0000 7ff7997fa640 1 --2- 172.21.15.43:0/617417946 >> [v2:172.21.15.203:6832/830097474,v1:172.21.15.203:6833/830097474] conn(0x7ff78006a110 0x7ff78006c5d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:34:38.181 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:38.171+0000 7ff7997fa640 1 -- 172.21.15.43:0/617417946 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6813+0+0 (secure 0 0 0) 0x7ff788088a10 con 0x7ff78c0047c0 2023-08-09T13:34:38.181 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:38.172+0000 7f8322ffd640 1 -- 172.21.15.43:0/3804042763 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 5) v1 ==== 88341+0+0 (secure 0 0 0) 0x7f832801a400 con 0x7f832c0ac170 2023-08-09T13:34:38.182 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:38.167+0000 7ff7a18c2640 1 -- 172.21.15.43:0/617417946 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff794008b20 con 0x7ff78c0047c0 2023-08-09T13:34:38.182 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:38.172+0000 7ff79bfff640 1 --2- 172.21.15.43:0/617417946 >> [v2:172.21.15.203:6832/830097474,v1:172.21.15.203:6833/830097474] conn(0x7ff78006a110 0x7ff78006c5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:34:38.182 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:38.170+0000 7f1356de8640 1 -- 172.21.15.43:0/874155189 wait complete. 2023-08-09T13:34:38.182 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:38.173+0000 7f1356de8640 1 Processor -- start 2023-08-09T13:34:38.183 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:38.172+0000 7f8333256640 1 -- 172.21.15.43:0/3804042763 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f832c0739f0 con 0x7f832c0ac170 2023-08-09T13:34:38.183 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:38.174+0000 7f8322ffd640 1 --2- 172.21.15.43:0/3804042763 >> [v2:172.21.15.203:6832/830097474,v1:172.21.15.203:6833/830097474] conn(0x7f831006a0a0 0x7f831006c560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:34:38.183 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:38.174+0000 7f8322ffd640 1 -- 172.21.15.43:0/3804042763 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6813+0+0 (secure 0 0 0) 0x7f83280887a0 con 0x7f832c0ac170 2023-08-09T13:34:38.183 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:38.173+0000 7f1356de8640 1 -- start start 2023-08-09T13:34:38.184 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:38.175+0000 7f1356de8640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f13500cc1b0 0x7f13500ca180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:34:38.184 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:38.175+0000 7f1356de8640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f135016cd10 0x7f13501f2920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:34:38.184 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:38.175+0000 7f1356de8640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f135016d620 0x7f13501f2e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:34:38.184 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:38.175+0000 7f8331a53640 1 --2- 172.21.15.43:0/3804042763 >> [v2:172.21.15.203:6832/830097474,v1:172.21.15.203:6833/830097474] conn(0x7f831006a0a0 0x7f831006c560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:34:38.185 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:38.175+0000 7f1354b84640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f13500cc1b0 0x7f13500ca180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:34:38.186 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:38.175+0000 7f1354b84640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f13500cc1b0 0x7f13500ca180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:51946/0 (socket says 172.21.15.43:51946) 2023-08-09T13:34:38.186 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:38.176+0000 7f134ffff640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f135016cd10 0x7f13501f2920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:34:38.186 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:38.176+0000 7f134ffff640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f135016cd10 0x7f13501f2920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.203:3300/0 says I am v2:172.21.15.43:34584/0 (socket says 172.21.15.43:34584) 2023-08-09T13:34:38.186 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:38.176+0000 7f1354b84640 1 -- 172.21.15.43:0/3552648038 learned_addr learned my addr 172.21.15.43:0/3552648038 (peer_addr_for_me v2:172.21.15.43:0/0) 2023-08-09T13:34:38.187 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:38.176+0000 7ff79bfff640 1 --2- 172.21.15.43:0/617417946 >> [v2:172.21.15.203:6832/830097474,v1:172.21.15.203:6833/830097474] conn(0x7ff78006a110 0x7ff78006c5d0 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7ff7900017d0 tx=0x7ff790030000 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-09T13:34:38.187 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:38.176+0000 7f1356de8640 1 -- 172.21.15.43:0/3552648038 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f135016b570 con 0x7f13500cc1b0 2023-08-09T13:34:38.188 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:38.177+0000 7f8331a53640 1 --2- 172.21.15.43:0/3804042763 >> [v2:172.21.15.203:6832/830097474,v1:172.21.15.203:6833/830097474] conn(0x7f831006a0a0 0x7f831006c560 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7f831c003e30 tx=0x7f831c009040 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-09T13:34:38.190 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:38.177+0000 7f1355385640 1 --2- 172.21.15.43:0/3552648038 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f135016d620 0x7f13501f2e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:34:38.190 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:38.177+0000 7f1356de8640 1 -- 172.21.15.43:0/3552648038 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f135016a8f0 con 0x7f135016cd10 2023-08-09T13:34:38.190 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:38.177+0000 7f1356de8640 1 -- 172.21.15.43:0/3552648038 --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f135016abd0 con 0x7f135016d620 2023-08-09T13:34:38.191 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:38.178+0000 7f1354b84640 1 -- 172.21.15.43:0/3552648038 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f135016d620 msgr2=0x7f13501f2e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:34:38.191 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:38.178+0000 7f1354b84640 1 --2- 172.21.15.43:0/3552648038 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f135016d620 0x7f13501f2e60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:34:38.191 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:38.178+0000 7f1354b84640 1 -- 172.21.15.43:0/3552648038 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f135016cd10 msgr2=0x7f13501f2920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:34:38.192 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:38.178+0000 7f1354b84640 1 --2- 172.21.15.43:0/3552648038 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f135016cd10 0x7f13501f2920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:34:38.192 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:38.178+0000 7f1354b84640 1 -- 172.21.15.43:0/3552648038 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1348014ac0 con 0x7f13500cc1b0 2023-08-09T13:34:38.192 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:38.178+0000 7f134ffff640 1 --2- 172.21.15.43:0/3552648038 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f135016cd10 0x7f13501f2920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-08-09T13:34:38.192 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:38.178+0000 7f1355385640 1 --2- 172.21.15.43:0/3552648038 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f135016d620 0x7f13501f2e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-08-09T13:34:38.193 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:38.178+0000 7f1354b84640 1 --2- 172.21.15.43:0/3552648038 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f13500cc1b0 0x7f13500ca180 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7f134400b920 tx=0x7f134400bde0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-09T13:34:38.193 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:38.181+0000 7f134d7fa640 1 -- 172.21.15.43:0/3552648038 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1344006c50 con 0x7f13500cc1b0 2023-08-09T13:34:38.193 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:38.182+0000 7f1356de8640 1 -- 172.21.15.43:0/3552648038 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f135016aeb0 con 0x7f13500cc1b0 2023-08-09T13:34:38.194 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:38.182+0000 7f1356de8640 1 -- 172.21.15.43:0/3552648038 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f13500ca6c0 con 0x7f13500cc1b0 2023-08-09T13:34:38.194 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:38.182+0000 7ff7997fa640 1 -- 172.21.15.43:0/617417946 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178066 (secure 0 0 0) 0x7ff788053da0 con 0x7ff78c0047c0 2023-08-09T13:34:38.195 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:38.185+0000 7f1336ffd640 1 -- 172.21.15.43:0/3552648038 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f13500ca850 con 0x7f13500cc1b0 2023-08-09T13:34:38.195 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:38.190+0000 7f8322ffd640 1 -- 172.21.15.43:0/3804042763 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178066 (secure 0 0 0) 0x7f8328053d10 con 0x7f832c0ac170 2023-08-09T13:34:38.197 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:38.193+0000 7f134d7fa640 1 -- 172.21.15.43:0/3552648038 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f134400a8c0 con 0x7f13500cc1b0 2023-08-09T13:34:38.198 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:38.193+0000 7f134d7fa640 1 -- 172.21.15.43:0/3552648038 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f134400ac50 con 0x7f13500cc1b0 2023-08-09T13:34:38.198 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:38.193+0000 7f134d7fa640 1 -- 172.21.15.43:0/3552648038 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 5) v1 ==== 88341+0+0 (secure 0 0 0) 0x7f13440203b0 con 0x7f13500cc1b0 2023-08-09T13:34:38.198 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:38.195+0000 7f134d7fa640 1 --2- 172.21.15.43:0/3552648038 >> [v2:172.21.15.203:6832/830097474,v1:172.21.15.203:6833/830097474] conn(0x7f133c06a0d0 0x7f133c06c590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:34:38.198 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:38.195+0000 7f134d7fa640 1 -- 172.21.15.43:0/3552648038 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6813+0+0 (secure 0 0 0) 0x7f1344088b70 con 0x7f13500cc1b0 2023-08-09T13:34:38.201 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:38.199+0000 7f134ffff640 1 --2- 172.21.15.43:0/3552648038 >> [v2:172.21.15.203:6832/830097474,v1:172.21.15.203:6833/830097474] conn(0x7f133c06a0d0 0x7f133c06c590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:34:38.202 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:38.200+0000 7f134d7fa640 1 -- 172.21.15.43:0/3552648038 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178066 (secure 0 0 0) 0x7f1344054020 con 0x7f13500cc1b0 2023-08-09T13:34:38.203 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:38.201+0000 7f134ffff640 1 --2- 172.21.15.43:0/3552648038 >> [v2:172.21.15.203:6832/830097474,v1:172.21.15.203:6833/830097474] conn(0x7f133c06a0d0 0x7f133c06c590 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7f134803f000 tx=0x7f1348045040 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-09T13:34:38.460 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:38.458+0000 7efd8afb1640 1 -- 172.21.15.43:0/1847632179 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 0} v 0) v1 -- 0x7efd64005500 con 0x7efd840ccfa0 2023-08-09T13:34:38.461 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:38.458+0000 7efd817fa640 1 -- 172.21.15.43:0/1847632179 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 0}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7efd7c079530 con 0x7efd840ccfa0 2023-08-09T13:34:38.463 INFO:teuthology.orchestra.run.smithi043.stdout:55834574858 2023-08-09T13:34:38.464 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:38.462+0000 7efd6affd640 1 -- 172.21.15.43:0/1847632179 >> [v2:172.21.15.203:6832/830097474,v1:172.21.15.203:6833/830097474] conn(0x7efd7006a1a0 msgr2=0x7efd7006c660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:34:38.464 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:38.462+0000 7efd6affd640 1 --2- 172.21.15.43:0/1847632179 >> [v2:172.21.15.203:6832/830097474,v1:172.21.15.203:6833/830097474] conn(0x7efd7006a1a0 0x7efd7006c660 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7efd840b1c50 tx=0x7efd74009190 comp rx=0 tx=0).stop 2023-08-09T13:34:38.465 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:38.462+0000 7efd6affd640 1 -- 172.21.15.43:0/1847632179 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7efd840ccfa0 msgr2=0x7efd840c91b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:34:38.465 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:38.462+0000 7efd6affd640 1 --2- 172.21.15.43:0/1847632179 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7efd840ccfa0 0x7efd840c91b0 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7efd7c036000 tx=0x7efd7c004a40 comp rx=0 tx=0).stop 2023-08-09T13:34:38.465 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:38.462+0000 7efd6affd640 1 -- 172.21.15.43:0/1847632179 shutdown_connections 2023-08-09T13:34:38.465 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:38.462+0000 7efd6affd640 1 --2- 172.21.15.43:0/1847632179 >> [v2:172.21.15.203:6832/830097474,v1:172.21.15.203:6833/830097474] conn(0x7efd7006a1a0 0x7efd7006c660 unknown :-1 s=CLOSED pgs=114 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:34:38.465 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:38.462+0000 7efd6affd640 1 --2- 172.21.15.43:0/1847632179 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7efd8414a810 0x7efd840c96f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:34:38.465 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:38.462+0000 7efd6affd640 1 --2- 172.21.15.43:0/1847632179 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7efd840ccfa0 0x7efd840c91b0 unknown :-1 s=CLOSED pgs=177 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:34:38.465 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:38.462+0000 7efd6affd640 1 --2- 172.21.15.43:0/1847632179 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7efd840ca170 0x7efd840b2a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:34:38.466 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:38.462+0000 7efd6affd640 1 -- 172.21.15.43:0/1847632179 >> 172.21.15.43:0/1847632179 conn(0x7efd840af970 msgr2=0x7efd840c7650 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-09T13:34:38.466 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:38.462+0000 7efd6affd640 1 -- 172.21.15.43:0/1847632179 shutdown_connections 2023-08-09T13:34:38.466 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:38.462+0000 7efd6affd640 1 -- 172.21.15.43:0/1847632179 wait complete. 2023-08-09T13:34:38.492 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574858 got 55834574858 for osd.0 2023-08-09T13:34:38.492 DEBUG:teuthology.parallel:result is None 2023-08-09T13:34:38.513 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:38.511+0000 7f02dd2e5640 1 -- 172.21.15.43:0/375074373 --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 4} v 0) v1 -- 0x7f02b8005500 con 0x7f02d0005fe0 2023-08-09T13:34:38.514 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:38.512+0000 7f02d4ff9640 1 -- 172.21.15.43:0/375074373 <== mon.2 v2:172.21.15.203:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 4}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f02c4056800 con 0x7f02d0005fe0 2023-08-09T13:34:38.515 INFO:teuthology.orchestra.run.smithi043.stdout:55834574858 2023-08-09T13:34:38.517 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:38.514+0000 7f02dd2e5640 1 -- 172.21.15.43:0/375074373 >> [v2:172.21.15.203:6832/830097474,v1:172.21.15.203:6833/830097474] conn(0x7f02bc06a150 msgr2=0x7f02bc06c610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:34:38.517 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:38.514+0000 7f02dd2e5640 1 --2- 172.21.15.43:0/375074373 >> [v2:172.21.15.203:6832/830097474,v1:172.21.15.203:6833/830097474] conn(0x7f02bc06a150 0x7f02bc06c610 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7f02d0103d60 tx=0x7f02cc009190 comp rx=0 tx=0).stop 2023-08-09T13:34:38.517 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:38.514+0000 7f02dd2e5640 1 -- 172.21.15.43:0/375074373 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f02d0005fe0 msgr2=0x7f02d00475a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:34:38.518 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:38.514+0000 7f02dd2e5640 1 --2- 172.21.15.43:0/375074373 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f02d0005fe0 0x7f02d00475a0 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f02c400c3c0 tx=0x7f02c400c880 comp rx=0 tx=0).stop 2023-08-09T13:34:38.518 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:38.514+0000 7f02dd2e5640 1 -- 172.21.15.43:0/375074373 shutdown_connections 2023-08-09T13:34:38.518 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:38.514+0000 7f02dd2e5640 1 --2- 172.21.15.43:0/375074373 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f02d00545a0 0x7f02d004b170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:34:38.518 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:38.514+0000 7f02dd2e5640 1 --2- 172.21.15.43:0/375074373 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f02d003a860 0x7f02d004ac30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:34:38.519 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:38.514+0000 7f02dd2e5640 1 --2- 172.21.15.43:0/375074373 >> [v2:172.21.15.203:6832/830097474,v1:172.21.15.203:6833/830097474] conn(0x7f02bc06a150 0x7f02bc06c610 unknown :-1 s=CLOSED pgs=116 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:34:38.519 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:38.514+0000 7f02dd2e5640 1 --2- 172.21.15.43:0/375074373 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f02d0005fe0 0x7f02d00475a0 unknown :-1 s=CLOSED pgs=53 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:34:38.519 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:38.514+0000 7f02dd2e5640 1 -- 172.21.15.43:0/375074373 >> 172.21.15.43:0/375074373 conn(0x7f02d0054c10 msgr2=0x7f02d00f6b00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-09T13:34:38.519 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:38.515+0000 7f02dd2e5640 1 -- 172.21.15.43:0/375074373 shutdown_connections 2023-08-09T13:34:38.520 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:38.515+0000 7f02dd2e5640 1 -- 172.21.15.43:0/375074373 wait complete. 2023-08-09T13:34:38.534 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574858 got 55834574858 for osd.4 2023-08-09T13:34:38.534 DEBUG:teuthology.parallel:result is None 2023-08-09T13:34:38.541 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:38.538+0000 7f63df7fe640 1 -- 172.21.15.43:0/246394760 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 1} v 0) v1 -- 0x7f63ec04df30 con 0x7f63ec050330 2023-08-09T13:34:38.542 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:38.539+0000 7f63e9ffb640 1 -- 172.21.15.43:0/246394760 <== mon.1 v2:172.21.15.203:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 1}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f63ec04df30 con 0x7f63ec050330 2023-08-09T13:34:38.542 INFO:teuthology.orchestra.run.smithi043.stdout:55834574858 2023-08-09T13:34:38.545 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:38.543+0000 7f63df7fe640 1 -- 172.21.15.43:0/246394760 >> [v2:172.21.15.203:6832/830097474,v1:172.21.15.203:6833/830097474] conn(0x7f63d806a0a0 msgr2=0x7f63d806c560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:34:38.545 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:38.543+0000 7f63df7fe640 1 --2- 172.21.15.43:0/246394760 >> [v2:172.21.15.203:6832/830097474,v1:172.21.15.203:6833/830097474] conn(0x7f63d806a0a0 0x7f63d806c560 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7f63e4009a30 tx=0x7f63e4009190 comp rx=0 tx=0).stop 2023-08-09T13:34:38.546 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:38.543+0000 7f63df7fe640 1 -- 172.21.15.43:0/246394760 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f63ec050330 msgr2=0x7f63ec006940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:34:38.546 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:38.543+0000 7f63df7fe640 1 --2- 172.21.15.43:0/246394760 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f63ec050330 0x7f63ec006940 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7f63f0035000 tx=0x7f63f0004730 comp rx=0 tx=0).stop 2023-08-09T13:34:38.546 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:38.543+0000 7f63df7fe640 1 -- 172.21.15.43:0/246394760 shutdown_connections 2023-08-09T13:34:38.547 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:38.543+0000 7f63df7fe640 1 --2- 172.21.15.43:0/246394760 >> [v2:172.21.15.203:6832/830097474,v1:172.21.15.203:6833/830097474] conn(0x7f63d806a0a0 0x7f63d806c560 unknown :-1 s=CLOSED pgs=115 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:34:38.547 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:38.543+0000 7f63df7fe640 1 --2- 172.21.15.43:0/246394760 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f63ec0dffd0 0x7f63ec0dad10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:34:38.547 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:38.543+0000 7f63df7fe640 1 --2- 172.21.15.43:0/246394760 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f63ec050330 0x7f63ec006940 unknown :-1 s=CLOSED pgs=100 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:34:38.547 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:38.543+0000 7f63df7fe640 1 --2- 172.21.15.43:0/246394760 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f63ec039ee0 0x7f63ec04a1d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:34:38.547 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:38.543+0000 7f63df7fe640 1 -- 172.21.15.43:0/246394760 >> 172.21.15.43:0/246394760 conn(0x7f63ec054c40 msgr2=0x7f63ec038e50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-09T13:34:38.547 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:38.543+0000 7f63df7fe640 1 -- 172.21.15.43:0/246394760 shutdown_connections 2023-08-09T13:34:38.547 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:38.544+0000 7f63df7fe640 1 -- 172.21.15.43:0/246394760 wait complete. 2023-08-09T13:34:38.560 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:38.557+0000 7fdf84ff9640 1 -- 172.21.15.43:0/4060401720 --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 2} v 0) v1 -- 0x7fdf9816a380 con 0x7fdf800038c0 2023-08-09T13:34:38.561 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:38.558+0000 7fdf86ffd640 1 -- 172.21.15.43:0/4060401720 <== mon.2 v2:172.21.15.203:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 2}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7fdf88045020 con 0x7fdf800038c0 2023-08-09T13:34:38.561 INFO:teuthology.orchestra.run.smithi043.stdout:55834574858 2023-08-09T13:34:38.563 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:38.561+0000 7f58686b8640 1 -- 172.21.15.43:0/2825405935 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 3} v 0) v1 -- 0x7f586006fe70 con 0x7f58600c9070 2023-08-09T13:34:38.563 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:38.561+0000 7f5852ffd640 1 -- 172.21.15.43:0/2825405935 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 3}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f585c029090 con 0x7f58600c9070 2023-08-09T13:34:38.564 INFO:teuthology.orchestra.run.smithi043.stdout:55834574859 2023-08-09T13:34:38.565 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574858 got 55834574858 for osd.1 2023-08-09T13:34:38.565 DEBUG:teuthology.parallel:result is None 2023-08-09T13:34:38.566 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:38.562+0000 7fdf84ff9640 1 -- 172.21.15.43:0/4060401720 >> [v2:172.21.15.203:6832/830097474,v1:172.21.15.203:6833/830097474] conn(0x7fdf7806a050 msgr2=0x7fdf7806c510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:34:38.566 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:38.562+0000 7fdf84ff9640 1 --2- 172.21.15.43:0/4060401720 >> [v2:172.21.15.203:6832/830097474,v1:172.21.15.203:6833/830097474] conn(0x7fdf7806a050 0x7fdf7806c510 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7fdf980c7af0 tx=0x7fdf8c0097c0 comp rx=0 tx=0).stop 2023-08-09T13:34:38.566 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:38.562+0000 7fdf84ff9640 1 -- 172.21.15.43:0/4060401720 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fdf800038c0 msgr2=0x7fdf98207b60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:34:38.566 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:38.562+0000 7fdf84ff9640 1 --2- 172.21.15.43:0/4060401720 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fdf800038c0 0x7fdf98207b60 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7fdf8800c1e0 tx=0x7fdf88001930 comp rx=0 tx=0).stop 2023-08-09T13:34:38.566 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:38.562+0000 7fdf84ff9640 1 -- 172.21.15.43:0/4060401720 shutdown_connections 2023-08-09T13:34:38.566 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:38.562+0000 7fdf84ff9640 1 --2- 172.21.15.43:0/4060401720 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fdf98208280 0x7fdf9820acc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:34:38.567 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:38.562+0000 7fdf84ff9640 1 --2- 172.21.15.43:0/4060401720 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fdf98168860 0x7fdf98169cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:34:38.567 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:38.562+0000 7fdf84ff9640 1 --2- 172.21.15.43:0/4060401720 >> [v2:172.21.15.203:6832/830097474,v1:172.21.15.203:6833/830097474] conn(0x7fdf7806a050 0x7fdf7806c510 unknown :-1 s=CLOSED pgs=118 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:34:38.567 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:38.562+0000 7fdf84ff9640 1 --2- 172.21.15.43:0/4060401720 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fdf800038c0 0x7fdf98207b60 unknown :-1 s=CLOSED pgs=54 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:34:38.567 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:38.562+0000 7fdf84ff9640 1 -- 172.21.15.43:0/4060401720 >> 172.21.15.43:0/4060401720 conn(0x7fdf980af970 msgr2=0x7fdf9806d490 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-09T13:34:38.567 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:38.562+0000 7fdf84ff9640 1 -- 172.21.15.43:0/4060401720 shutdown_connections 2023-08-09T13:34:38.567 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:38.562+0000 7fdf84ff9640 1 -- 172.21.15.43:0/4060401720 wait complete. 2023-08-09T13:34:38.568 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:38.564+0000 7f58686b8640 1 -- 172.21.15.43:0/2825405935 >> [v2:172.21.15.203:6832/830097474,v1:172.21.15.203:6833/830097474] conn(0x7f584406a0d0 msgr2=0x7f584406c590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:34:38.568 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:38.564+0000 7f58686b8640 1 --2- 172.21.15.43:0/2825405935 >> [v2:172.21.15.203:6832/830097474,v1:172.21.15.203:6833/830097474] conn(0x7f584406a0d0 0x7f584406c590 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7f5860163ec0 tx=0x7f5854009040 comp rx=0 tx=0).stop 2023-08-09T13:34:38.568 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:38.564+0000 7f58686b8640 1 -- 172.21.15.43:0/2825405935 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f58600c9070 msgr2=0x7f58600c6150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:34:38.569 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:38.564+0000 7f58686b8640 1 --2- 172.21.15.43:0/2825405935 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f58600c9070 0x7f58600c6150 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7f585c01d520 tx=0x7f585c01d9e0 comp rx=0 tx=0).stop 2023-08-09T13:34:38.569 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:38.564+0000 7f58686b8640 1 -- 172.21.15.43:0/2825405935 shutdown_connections 2023-08-09T13:34:38.569 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:38.564+0000 7f58686b8640 1 --2- 172.21.15.43:0/2825405935 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f58600ca3d0 0x7f5860152f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:34:38.569 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:38.565+0000 7f58686b8640 1 --2- 172.21.15.43:0/2825405935 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f58600c9070 0x7f58600c6150 unknown :-1 s=CLOSED pgs=183 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:34:38.569 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:38.565+0000 7f58686b8640 1 --2- 172.21.15.43:0/2825405935 >> [v2:172.21.15.203:6832/830097474,v1:172.21.15.203:6833/830097474] conn(0x7f584406a0d0 0x7f584406c590 unknown :-1 s=CLOSED pgs=117 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:34:38.569 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:38.565+0000 7f58686b8640 1 --2- 172.21.15.43:0/2825405935 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f58600c84d0 0x7f5860151c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:34:38.569 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:38.565+0000 7f58686b8640 1 -- 172.21.15.43:0/2825405935 >> 172.21.15.43:0/2825405935 conn(0x7f58600cde80 msgr2=0x7f58601641b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-09T13:34:38.569 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:38.565+0000 7f58686b8640 1 -- 172.21.15.43:0/2825405935 shutdown_connections 2023-08-09T13:34:38.570 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:38.565+0000 7f58686b8640 1 -- 172.21.15.43:0/2825405935 wait complete. 2023-08-09T13:34:38.572 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:38.570+0000 7ff7a18c2640 1 -- 172.21.15.43:0/617417946 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 6} v 0) v1 -- 0x7ff794006b20 con 0x7ff78c0047c0 2023-08-09T13:34:38.572 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:38.570+0000 7ff7997fa640 1 -- 172.21.15.43:0/617417946 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 6}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7ff7880573c0 con 0x7ff78c0047c0 2023-08-09T13:34:38.573 INFO:teuthology.orchestra.run.smithi043.stdout:55834574858 2023-08-09T13:34:38.575 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:38.573+0000 7ff7a18c2640 1 -- 172.21.15.43:0/617417946 >> [v2:172.21.15.203:6832/830097474,v1:172.21.15.203:6833/830097474] conn(0x7ff78006a110 msgr2=0x7ff78006c5d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:34:38.575 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:38.573+0000 7ff7a18c2640 1 --2- 172.21.15.43:0/617417946 >> [v2:172.21.15.203:6832/830097474,v1:172.21.15.203:6833/830097474] conn(0x7ff78006a110 0x7ff78006c5d0 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7ff7900017d0 tx=0x7ff790030000 comp rx=0 tx=0).stop 2023-08-09T13:34:38.576 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:38.573+0000 7ff7a18c2640 1 -- 172.21.15.43:0/617417946 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff78c0047c0 msgr2=0x7ff7940f5030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:34:38.576 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:38.573+0000 7ff7a18c2640 1 --2- 172.21.15.43:0/617417946 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff78c0047c0 0x7ff7940f5030 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7ff788004650 tx=0x7ff788003ce0 comp rx=0 tx=0).stop 2023-08-09T13:34:38.576 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:38.573+0000 7ff7a18c2640 1 -- 172.21.15.43:0/617417946 shutdown_connections 2023-08-09T13:34:38.576 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:38.573+0000 7ff7a18c2640 1 --2- 172.21.15.43:0/617417946 >> [v2:172.21.15.203:6832/830097474,v1:172.21.15.203:6833/830097474] conn(0x7ff78006a110 0x7ff78006c5d0 unknown :-1 s=CLOSED pgs=119 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:34:38.576 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:38.573+0000 7ff7a18c2640 1 --2- 172.21.15.43:0/617417946 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7ff79404e430 0x7ff794049880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:34:38.576 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:38.573+0000 7ff7a18c2640 1 --2- 172.21.15.43:0/617417946 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7ff79404c420 0x7ff79404def0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:34:38.577 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:38.573+0000 7ff7a18c2640 1 --2- 172.21.15.43:0/617417946 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff78c0047c0 0x7ff7940f5030 unknown :-1 s=CLOSED pgs=187 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:34:38.577 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:38.573+0000 7ff7a18c2640 1 -- 172.21.15.43:0/617417946 >> 172.21.15.43:0/617417946 conn(0x7ff794054c10 msgr2=0x7ff794100340 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-09T13:34:38.577 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:38.573+0000 7ff7a18c2640 1 -- 172.21.15.43:0/617417946 shutdown_connections 2023-08-09T13:34:38.577 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:38.573+0000 7ff7a18c2640 1 -- 172.21.15.43:0/617417946 wait complete. 2023-08-09T13:34:38.587 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574859 got 55834574859 for osd.3 2023-08-09T13:34:38.588 DEBUG:teuthology.parallel:result is None 2023-08-09T13:34:38.592 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574858 got 55834574858 for osd.2 2023-08-09T13:34:38.592 DEBUG:teuthology.parallel:result is None 2023-08-09T13:34:38.594 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574858 got 55834574858 for osd.6 2023-08-09T13:34:38.594 DEBUG:teuthology.parallel:result is None 2023-08-09T13:34:38.598 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:38.596+0000 7f1336ffd640 1 -- 172.21.15.43:0/3552648038 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 5} v 0) v1 -- 0x7f13500739f0 con 0x7f13500cc1b0 2023-08-09T13:34:38.599 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:38.597+0000 7f134d7fa640 1 -- 172.21.15.43:0/3552648038 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 5}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f1344057640 con 0x7f13500cc1b0 2023-08-09T13:34:38.599 INFO:teuthology.orchestra.run.smithi043.stdout:55834574858 2023-08-09T13:34:38.600 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:38.598+0000 7f8333256640 1 -- 172.21.15.43:0/3804042763 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 7} v 0) v1 -- 0x7f832c0cebd0 con 0x7f832c0ac170 2023-08-09T13:34:38.601 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:38.598+0000 7f8322ffd640 1 -- 172.21.15.43:0/3804042763 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 7}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f8328015070 con 0x7f832c0ac170 2023-08-09T13:34:38.601 INFO:teuthology.orchestra.run.smithi043.stdout:55834574858 2023-08-09T13:34:38.602 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:38.599+0000 7f1336ffd640 1 -- 172.21.15.43:0/3552648038 >> [v2:172.21.15.203:6832/830097474,v1:172.21.15.203:6833/830097474] conn(0x7f133c06a0d0 msgr2=0x7f133c06c590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:34:38.602 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:38.599+0000 7f1336ffd640 1 --2- 172.21.15.43:0/3552648038 >> [v2:172.21.15.203:6832/830097474,v1:172.21.15.203:6833/830097474] conn(0x7f133c06a0d0 0x7f133c06c590 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7f134803f000 tx=0x7f1348045040 comp rx=0 tx=0).stop 2023-08-09T13:34:38.602 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:38.599+0000 7f1336ffd640 1 -- 172.21.15.43:0/3552648038 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f13500cc1b0 msgr2=0x7f13500ca180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:34:38.602 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:38.599+0000 7f1336ffd640 1 --2- 172.21.15.43:0/3552648038 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f13500cc1b0 0x7f13500ca180 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7f134400b920 tx=0x7f134400bde0 comp rx=0 tx=0).stop 2023-08-09T13:34:38.602 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:38.599+0000 7f1336ffd640 1 -- 172.21.15.43:0/3552648038 shutdown_connections 2023-08-09T13:34:38.602 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:38.599+0000 7f1336ffd640 1 --2- 172.21.15.43:0/3552648038 >> [v2:172.21.15.203:6832/830097474,v1:172.21.15.203:6833/830097474] conn(0x7f133c06a0d0 0x7f133c06c590 unknown :-1 s=CLOSED pgs=121 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:34:38.603 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:38.599+0000 7f1336ffd640 1 --2- 172.21.15.43:0/3552648038 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f135016d620 0x7f13501f2e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:34:38.603 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:38.599+0000 7f1336ffd640 1 --2- 172.21.15.43:0/3552648038 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f135016cd10 0x7f13501f2920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:34:38.603 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:38.599+0000 7f1336ffd640 1 --2- 172.21.15.43:0/3552648038 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f13500cc1b0 0x7f13500ca180 unknown :-1 s=CLOSED pgs=190 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:34:38.603 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:38.599+0000 7f1336ffd640 1 -- 172.21.15.43:0/3552648038 >> 172.21.15.43:0/3552648038 conn(0x7f13500afcb0 msgr2=0x7f1350067560 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-09T13:34:38.603 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:38.599+0000 7f1336ffd640 1 -- 172.21.15.43:0/3552648038 shutdown_connections 2023-08-09T13:34:38.603 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:38.599+0000 7f1336ffd640 1 -- 172.21.15.43:0/3552648038 wait complete. 2023-08-09T13:34:38.604 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:38.601+0000 7f8333256640 1 -- 172.21.15.43:0/3804042763 >> [v2:172.21.15.203:6832/830097474,v1:172.21.15.203:6833/830097474] conn(0x7f831006a0a0 msgr2=0x7f831006c560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:34:38.604 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:38.601+0000 7f8333256640 1 --2- 172.21.15.43:0/3804042763 >> [v2:172.21.15.203:6832/830097474,v1:172.21.15.203:6833/830097474] conn(0x7f831006a0a0 0x7f831006c560 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7f831c003e30 tx=0x7f831c009040 comp rx=0 tx=0).stop 2023-08-09T13:34:38.604 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:38.601+0000 7f8333256640 1 -- 172.21.15.43:0/3804042763 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f832c0ac170 msgr2=0x7f832c148db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:34:38.605 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:38.601+0000 7f8333256640 1 --2- 172.21.15.43:0/3804042763 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f832c0ac170 0x7f832c148db0 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7f832800b340 tx=0x7f832800b800 comp rx=0 tx=0).stop 2023-08-09T13:34:38.605 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:38.601+0000 7f8333256640 1 -- 172.21.15.43:0/3804042763 shutdown_connections 2023-08-09T13:34:38.605 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:38.601+0000 7f8333256640 1 --2- 172.21.15.43:0/3804042763 >> [v2:172.21.15.203:6832/830097474,v1:172.21.15.203:6833/830097474] conn(0x7f831006a0a0 0x7f831006c560 unknown :-1 s=CLOSED pgs=120 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:34:38.605 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:38.601+0000 7f8333256640 1 --2- 172.21.15.43:0/3804042763 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f832c071100 0x7f832c06f740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:34:38.605 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:38.601+0000 7f8333256640 1 --2- 172.21.15.43:0/3804042763 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f832c0ac550 0x7f832c0b1630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:34:38.605 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:38.601+0000 7f8333256640 1 --2- 172.21.15.43:0/3804042763 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f832c0ac170 0x7f832c148db0 unknown :-1 s=CLOSED pgs=189 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:34:38.605 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:38.601+0000 7f8333256640 1 -- 172.21.15.43:0/3804042763 >> 172.21.15.43:0/3804042763 conn(0x7f832c0cde90 msgr2=0x7f832c0ce2a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-09T13:34:38.606 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:38.601+0000 7f8333256640 1 -- 172.21.15.43:0/3804042763 shutdown_connections 2023-08-09T13:34:38.606 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:38.601+0000 7f8333256640 1 -- 172.21.15.43:0/3804042763 wait complete. 2023-08-09T13:34:38.618 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574858 got 55834574858 for osd.5 2023-08-09T13:34:38.618 DEBUG:teuthology.parallel:result is None 2023-08-09T13:34:38.619 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574858 got 55834574858 for osd.7 2023-08-09T13:34:38.620 DEBUG:teuthology.parallel:result is None 2023-08-09T13:34:38.620 INFO:tasks.ceph.ceph_manager.ceph:waiting for clean 2023-08-09T13:34:38.620 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-08-09T13:34:38.889 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:38.886+0000 7f6cf245d640 1 Processor -- start 2023-08-09T13:34:38.889 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:38.886+0000 7f6cf245d640 1 -- start start 2023-08-09T13:34:38.891 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:38.888+0000 7f6cf245d640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f6cec15f7b0 0x7f6cec0b4760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:34:38.891 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:38.888+0000 7f6cf245d640 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7f6cec0b1050 0x7f6cec0ccfa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:34:38.891 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:38.888+0000 7f6cf245d640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f6cec0cd4e0 0x7f6cec0ceae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:34:38.891 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:38.888+0000 7f6cf245d640 1 -- --> v2:172.21.15.43:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6cec0cd930 con 0x7f6cec0cd4e0 2023-08-09T13:34:38.891 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:38.888+0000 7f6cf245d640 1 -- --> v2:172.21.15.203:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6cec0b7bf0 con 0x7f6cec0b1050 2023-08-09T13:34:38.891 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:38.888+0000 7f6cf245d640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6cec0b7ed0 con 0x7f6cec15f7b0 2023-08-09T13:34:38.892 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:38.889+0000 7f6cf1c5c640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f6cec0cd4e0 0x7f6cec0ceae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:34:38.892 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:38.889+0000 7f6cf1c5c640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f6cec0cd4e0 0x7f6cec0ceae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:51962/0 (socket says 172.21.15.43:51962) 2023-08-09T13:34:38.892 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:38.889+0000 7f6cf1c5c640 1 -- 172.21.15.43:0/654645673 learned_addr learned my addr 172.21.15.43:0/654645673 (peer_addr_for_me v2:172.21.15.43:0/0) 2023-08-09T13:34:38.893 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:38.889+0000 7f6cf145b640 1 --2- 172.21.15.43:0/654645673 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f6cec15f7b0 0x7f6cec0b4760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:34:38.893 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:38.889+0000 7f6cf0c5a640 1 --2- 172.21.15.43:0/654645673 >> v2:172.21.15.203:3300/0 conn(0x7f6cec0b1050 0x7f6cec0ccfa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:34:38.893 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:38.889+0000 7f6cf1c5c640 1 -- 172.21.15.43:0/654645673 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f6cec15f7b0 msgr2=0x7f6cec0b4760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-09T13:34:38.893 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:38.889+0000 7f6cf1c5c640 1 --2- 172.21.15.43:0/654645673 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f6cec15f7b0 0x7f6cec0b4760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:34:38.893 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:38.889+0000 7f6cf1c5c640 1 -- 172.21.15.43:0/654645673 >> v2:172.21.15.203:3300/0 conn(0x7f6cec0b1050 msgr2=0x7f6cec0ccfa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-09T13:34:38.893 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:38.889+0000 7f6cf1c5c640 1 --2- 172.21.15.43:0/654645673 >> v2:172.21.15.203:3300/0 conn(0x7f6cec0b1050 0x7f6cec0ccfa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:34:38.893 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:38.889+0000 7f6cf1c5c640 1 -- 172.21.15.43:0/654645673 --> v2:172.21.15.43:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6cec0d0370 con 0x7f6cec0cd4e0 2023-08-09T13:34:38.894 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:38.890+0000 7f6cf1c5c640 1 --2- 172.21.15.43:0/654645673 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f6cec0cd4e0 0x7f6cec0ceae0 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7f6cec0b4e00 tx=0x7f6ce402bee0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=866c5b7bce24f9a0 server_cookie=0 in_seq=0 out_seq=0 2023-08-09T13:34:38.894 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:38.891+0000 7f6ce3fff640 1 -- 172.21.15.43:0/654645673 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6ce402ca80 con 0x7f6cec0cd4e0 2023-08-09T13:34:38.895 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:38.891+0000 7f6ce3fff640 1 -- 172.21.15.43:0/654645673 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6ce402cbe0 con 0x7f6cec0cd4e0 2023-08-09T13:34:38.895 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:38.891+0000 7f6ce3fff640 1 -- 172.21.15.43:0/654645673 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6ce40343f0 con 0x7f6cec0cd4e0 2023-08-09T13:34:38.895 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:38.891+0000 7f6cf245d640 1 -- 172.21.15.43:0/654645673 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f6cec0cd4e0 msgr2=0x7f6cec0ceae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:34:38.895 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:38.891+0000 7f6cf245d640 1 --2- 172.21.15.43:0/654645673 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f6cec0cd4e0 0x7f6cec0ceae0 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7f6cec0b4e00 tx=0x7f6ce402bee0 comp rx=0 tx=0).stop 2023-08-09T13:34:38.895 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:38.891+0000 7f6cf245d640 1 -- 172.21.15.43:0/654645673 shutdown_connections 2023-08-09T13:34:38.895 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:38.891+0000 7f6cf245d640 1 --2- 172.21.15.43:0/654645673 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f6cec0cd4e0 0x7f6cec0ceae0 unknown :-1 s=CLOSED pgs=191 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:34:38.895 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:38.891+0000 7f6cf245d640 1 --2- 172.21.15.43:0/654645673 >> v2:172.21.15.203:3300/0 conn(0x7f6cec0b1050 0x7f6cec0ccfa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:34:38.896 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:38.891+0000 7f6cf245d640 1 --2- 172.21.15.43:0/654645673 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f6cec15f7b0 0x7f6cec0b4760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:34:38.896 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:38.891+0000 7f6cf245d640 1 -- 172.21.15.43:0/654645673 >> 172.21.15.43:0/654645673 conn(0x7f6cec06a2b0 msgr2=0x7f6cec06a690 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-09T13:34:38.896 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:38.891+0000 7f6cf245d640 1 -- 172.21.15.43:0/654645673 shutdown_connections 2023-08-09T13:34:38.896 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:38.892+0000 7f6cf245d640 1 -- 172.21.15.43:0/654645673 wait complete. 2023-08-09T13:34:38.897 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:38.893+0000 7f6cf245d640 1 Processor -- start 2023-08-09T13:34:38.897 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:38.893+0000 7f6cf245d640 1 -- start start 2023-08-09T13:34:38.897 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:38.894+0000 7f6cf245d640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f6cec0b1050 0x7f6cec152b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:34:38.897 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:38.894+0000 7f6cf245d640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f6cec0cd4e0 0x7f6cec0b1b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:34:38.897 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:38.894+0000 7f6cf245d640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f6cec15f7b0 0x7f6cec1550a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:34:38.897 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:38.894+0000 7f6cf245d640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6cec0b2ae0 con 0x7f6cec0cd4e0 2023-08-09T13:34:38.897 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:38.894+0000 7f6cf245d640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6cec0b31a0 con 0x7f6cec15f7b0 2023-08-09T13:34:38.898 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:38.894+0000 7f6cf245d640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6cec1530d0 con 0x7f6cec0b1050 2023-08-09T13:34:38.899 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:38.895+0000 7f6cf0c5a640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f6cec0cd4e0 0x7f6cec0b1b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:34:38.899 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:38.895+0000 7f6cf0c5a640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f6cec0cd4e0 0x7f6cec0b1b60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:51974/0 (socket says 172.21.15.43:51974) 2023-08-09T13:34:38.899 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:38.895+0000 7f6cf0c5a640 1 -- 172.21.15.43:0/492415803 learned_addr learned my addr 172.21.15.43:0/492415803 (peer_addr_for_me v2:172.21.15.43:0/0) 2023-08-09T13:34:38.899 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:38.895+0000 7f6cf145b640 1 --2- 172.21.15.43:0/492415803 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f6cec0b1050 0x7f6cec152b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:34:38.899 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:38.895+0000 7f6cf1c5c640 1 --2- 172.21.15.43:0/492415803 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f6cec15f7b0 0x7f6cec1550a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:34:38.899 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:38.895+0000 7f6cf0c5a640 1 -- 172.21.15.43:0/492415803 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f6cec0b1050 msgr2=0x7f6cec152b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:34:38.899 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:38.895+0000 7f6cf0c5a640 1 --2- 172.21.15.43:0/492415803 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f6cec0b1050 0x7f6cec152b90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:34:38.900 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:38.895+0000 7f6cf0c5a640 1 -- 172.21.15.43:0/492415803 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f6cec15f7b0 msgr2=0x7f6cec1550a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:34:38.900 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:38.895+0000 7f6cf0c5a640 1 --2- 172.21.15.43:0/492415803 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f6cec15f7b0 0x7f6cec1550a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:34:38.900 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:38.895+0000 7f6cf0c5a640 1 -- 172.21.15.43:0/492415803 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6ce4009b30 con 0x7f6cec0cd4e0 2023-08-09T13:34:38.900 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:38.895+0000 7f6cf145b640 1 --2- 172.21.15.43:0/492415803 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f6cec0b1050 0x7f6cec152b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-08-09T13:34:38.900 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:38.895+0000 7f6cf1c5c640 1 --2- 172.21.15.43:0/492415803 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f6cec15f7b0 0x7f6cec1550a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-08-09T13:34:38.900 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:38.895+0000 7f6cf0c5a640 1 --2- 172.21.15.43:0/492415803 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f6cec0cd4e0 0x7f6cec0b1b60 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7f6cdc00d860 tx=0x7f6cdc00dd20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-09T13:34:38.901 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:38.897+0000 7f6ce1ffb640 1 -- 172.21.15.43:0/492415803 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6cdc0096a0 con 0x7f6cec0cd4e0 2023-08-09T13:34:38.901 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:38.897+0000 7f6ce1ffb640 1 -- 172.21.15.43:0/492415803 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6cdc009800 con 0x7f6cec0cd4e0 2023-08-09T13:34:38.901 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:38.897+0000 7f6ce1ffb640 1 -- 172.21.15.43:0/492415803 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6cdc0099b0 con 0x7f6cec0cd4e0 2023-08-09T13:34:38.901 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:38.897+0000 7f6cf245d640 1 -- 172.21.15.43:0/492415803 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6cec0b2520 con 0x7f6cec0cd4e0 2023-08-09T13:34:38.902 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:38.897+0000 7f6cf245d640 1 -- 172.21.15.43:0/492415803 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6cec0b2e30 con 0x7f6cec0cd4e0 2023-08-09T13:34:38.902 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:38.898+0000 7f6cf245d640 1 -- 172.21.15.43:0/492415803 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6cec0739f0 con 0x7f6cec0cd4e0 2023-08-09T13:34:38.902 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:38.899+0000 7f6ce1ffb640 1 -- 172.21.15.43:0/492415803 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 5) v1 ==== 88341+0+0 (secure 0 0 0) 0x7f6cdc009b10 con 0x7f6cec0cd4e0 2023-08-09T13:34:38.902 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:38.900+0000 7f6ce1ffb640 1 --2- 172.21.15.43:0/492415803 >> [v2:172.21.15.203:6832/830097474,v1:172.21.15.203:6833/830097474] conn(0x7f6cd006a050 0x7f6cd006c510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:34:38.903 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:38.900+0000 7f6ce1ffb640 1 -- 172.21.15.43:0/492415803 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6813+0+0 (secure 0 0 0) 0x7f6cdc088de0 con 0x7f6cec0cd4e0 2023-08-09T13:34:38.903 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:38.901+0000 7f6cf145b640 1 --2- 172.21.15.43:0/492415803 >> [v2:172.21.15.203:6832/830097474,v1:172.21.15.203:6833/830097474] conn(0x7f6cd006a050 0x7f6cd006c510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:34:38.904 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:38.902+0000 7f6cf145b640 1 --2- 172.21.15.43:0/492415803 >> [v2:172.21.15.203:6832/830097474,v1:172.21.15.203:6833/830097474] conn(0x7f6cd006a050 0x7f6cd006c510 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7f6ce8006fd0 tx=0x7f6ce8008040 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-09T13:34:38.906 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:38.904+0000 7f6ce1ffb640 1 -- 172.21.15.43:0/492415803 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178066 (secure 0 0 0) 0x7f6cdc054140 con 0x7f6cec0cd4e0 2023-08-09T13:34:39.087 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:39.085+0000 7f6cf245d640 1 -- 172.21.15.43:0/492415803 --> [v2:172.21.15.203:6832/830097474,v1:172.21.15.203:6833/830097474] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f6cec0d0930 con 0x7f6cd006a050 2023-08-09T13:34:39.117 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:39.114+0000 7f6ce1ffb640 1 -- 172.21.15.43:0/492415803 <== mgr.4114 v2:172.21.15.203:6832/830097474 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+367422 (secure 0 0 0) 0x7f6ce8009e10 con 0x7f6cd006a050 2023-08-09T13:34:39.121 INFO:teuthology.orchestra.run.smithi043.stdout: 2023-08-09T13:34:39.122 INFO:teuthology.orchestra.run.smithi043.stderr:dumped all 2023-08-09T13:34:39.123 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:39.120+0000 7f6cf245d640 1 -- 172.21.15.43:0/492415803 >> [v2:172.21.15.203:6832/830097474,v1:172.21.15.203:6833/830097474] conn(0x7f6cd006a050 msgr2=0x7f6cd006c510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:34:39.124 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:39.120+0000 7f6cf245d640 1 --2- 172.21.15.43:0/492415803 >> [v2:172.21.15.203:6832/830097474,v1:172.21.15.203:6833/830097474] conn(0x7f6cd006a050 0x7f6cd006c510 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7f6ce8006fd0 tx=0x7f6ce8008040 comp rx=0 tx=0).stop 2023-08-09T13:34:39.124 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:39.120+0000 7f6cf245d640 1 -- 172.21.15.43:0/492415803 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f6cec0cd4e0 msgr2=0x7f6cec0b1b60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:34:39.124 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:39.120+0000 7f6cf245d640 1 --2- 172.21.15.43:0/492415803 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f6cec0cd4e0 0x7f6cec0b1b60 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7f6cdc00d860 tx=0x7f6cdc00dd20 comp rx=0 tx=0).stop 2023-08-09T13:34:39.124 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:39.120+0000 7f6cf245d640 1 -- 172.21.15.43:0/492415803 shutdown_connections 2023-08-09T13:34:39.124 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:39.120+0000 7f6cf245d640 1 --2- 172.21.15.43:0/492415803 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f6cec15f7b0 0x7f6cec1550a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:34:39.124 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:39.120+0000 7f6cf245d640 1 --2- 172.21.15.43:0/492415803 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f6cec0cd4e0 0x7f6cec0b1b60 unknown :-1 s=CLOSED pgs=192 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:34:39.125 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:39.120+0000 7f6cf245d640 1 --2- 172.21.15.43:0/492415803 >> [v2:172.21.15.203:6832/830097474,v1:172.21.15.203:6833/830097474] conn(0x7f6cd006a050 0x7f6cd006c510 unknown :-1 s=CLOSED pgs=122 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:34:39.125 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:39.120+0000 7f6cf245d640 1 --2- 172.21.15.43:0/492415803 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f6cec0b1050 0x7f6cec152b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:34:39.125 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:39.120+0000 7f6cf245d640 1 -- 172.21.15.43:0/492415803 >> 172.21.15.43:0/492415803 conn(0x7f6cec06a2b0 msgr2=0x7f6cec15bf10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-09T13:34:39.125 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:39.120+0000 7f6cf245d640 1 -- 172.21.15.43:0/492415803 shutdown_connections 2023-08-09T13:34:39.125 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:39.120+0000 7f6cf245d640 1 -- 172.21.15.43:0/492415803 wait complete. 2023-08-09T13:34:39.140 INFO:teuthology.orchestra.run.smithi043.stdout:{"pg_ready":true,"pg_map":{"version":40,"stamp":"2023-08-09T13:34:38.775015+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":595007,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":208,"num_write_kb":2435,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160,"ondisk_log_size":160,"up":338,"acting":338,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":338,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":235764,"kb_used_data":3200,"kb_used_omap":13,"kb_used_meta":232434,"kb_avail":754738956,"statfs":{"total":773094113280,"available":772852690944,"internally_reserved":0,"allocated":3276800,"data_stored":2025132,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":14248,"internal_metadata":238012504},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.005938"},"pg_stats":[{"pgid":"5.19","version":"0'0","reported_seq":13,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-08-09T13:34:08.367616+0000","last_change":"2023-08-09T13:34:07.232493+0000","last_active":"2023-08-09T13:34:08.367616+0000","last_peered":"2023-08-09T13:34:08.367616+0000","last_clean":"2023-08-09T13:34:08.367616+0000","last_became_active":"2023-08-09T13:34:07.231969+0000","last_became_peered":"2023-08-09T13:34:07.231969+0000","last_unstale":"2023-08-09T13:34:08.367616+0000","last_undegraded":"2023-08-09T13:34:08.367616+0000","last_fullsized":"2023-08-09T13:34:08.367616+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:34:06.173547+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:34:06.173547+0000","last_clean_scrub_stamp":"2023-08-09T13:34:06.173547+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T19:42:37.842719+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,7,2],"acting":[1,5,7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.1f","version":"19'4","reported_seq":23,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-08-09T13:34:08.209913+0000","last_change":"2023-08-09T13:34:02.108915+0000","last_active":"2023-08-09T13:34:08.209913+0000","last_peered":"2023-08-09T13:34:08.209913+0000","last_clean":"2023-08-09T13:34:08.209913+0000","last_became_active":"2023-08-09T13:34:02.108573+0000","last_became_peered":"2023-08-09T13:34:02.108573+0000","last_unstale":"2023-08-09T13:34:08.209913+0000","last_undegraded":"2023-08-09T13:34:08.209913+0000","last_fullsized":"2023-08-09T13:34:08.209913+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:34:01.052721+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:34:01.052721+0000","last_clean_scrub_stamp":"2023-08-09T13:34:01.052721+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T23:52:26.726953+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-08-09T13:34:08.370536+0000","last_change":"2023-08-09T13:34:03.156516+0000","last_active":"2023-08-09T13:34:08.370536+0000","last_peered":"2023-08-09T13:34:08.370536+0000","last_clean":"2023-08-09T13:34:08.370536+0000","last_became_active":"2023-08-09T13:34:03.156109+0000","last_became_peered":"2023-08-09T13:34:03.156109+0000","last_unstale":"2023-08-09T13:34:08.370536+0000","last_undegraded":"2023-08-09T13:34:08.370536+0000","last_fullsized":"2023-08-09T13:34:08.370536+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:34:02.064154+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:34:02.064154+0000","last_clean_scrub_stamp":"2023-08-09T13:34:02.064154+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T13:40:56.937864+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.18","version":"0'0","reported_seq":13,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-08-09T13:34:08.369809+0000","last_change":"2023-08-09T13:34:07.234694+0000","last_active":"2023-08-09T13:34:08.369809+0000","last_peered":"2023-08-09T13:34:08.369809+0000","last_clean":"2023-08-09T13:34:08.369809+0000","last_became_active":"2023-08-09T13:34:07.234241+0000","last_became_peered":"2023-08-09T13:34:07.234241+0000","last_unstale":"2023-08-09T13:34:08.369809+0000","last_undegraded":"2023-08-09T13:34:08.369809+0000","last_fullsized":"2023-08-09T13:34:08.369809+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:34:06.173547+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:34:06.173547+0000","last_clean_scrub_stamp":"2023-08-09T13:34:06.173547+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T15:06:44.662571+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,1,2],"acting":[4,6,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.1e","version":"19'1","reported_seq":22,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-08-09T13:34:08.370464+0000","last_change":"2023-08-09T13:34:02.105244+0000","last_active":"2023-08-09T13:34:08.370464+0000","last_peered":"2023-08-09T13:34:08.370464+0000","last_clean":"2023-08-09T13:34:08.370464+0000","last_became_active":"2023-08-09T13:34:02.104920+0000","last_became_peered":"2023-08-09T13:34:02.104920+0000","last_unstale":"2023-08-09T13:34:08.370464+0000","last_undegraded":"2023-08-09T13:34:08.370464+0000","last_fullsized":"2023-08-09T13:34:08.370464+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:34:01.052721+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:34:01.052721+0000","last_clean_scrub_stamp":"2023-08-09T13:34:01.052721+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T21:57:53.171937+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-08-09T13:34:08.368933+0000","last_change":"2023-08-09T13:34:03.156960+0000","last_active":"2023-08-09T13:34:08.368933+0000","last_peered":"2023-08-09T13:34:08.368933+0000","last_clean":"2023-08-09T13:34:08.368933+0000","last_became_active":"2023-08-09T13:34:03.156668+0000","last_became_peered":"2023-08-09T13:34:03.156668+0000","last_unstale":"2023-08-09T13:34:08.368933+0000","last_undegraded":"2023-08-09T13:34:08.368933+0000","last_fullsized":"2023-08-09T13:34:08.368933+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:34:02.064154+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:34:02.064154+0000","last_clean_scrub_stamp":"2023-08-09T13:34:02.064154+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T14:55:32.816063+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.1b","version":"0'0","reported_seq":13,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-08-09T13:34:08.370229+0000","last_change":"2023-08-09T13:34:07.232500+0000","last_active":"2023-08-09T13:34:08.370229+0000","last_peered":"2023-08-09T13:34:08.370229+0000","last_clean":"2023-08-09T13:34:08.370229+0000","last_became_active":"2023-08-09T13:34:07.232045+0000","last_became_peered":"2023-08-09T13:34:07.232045+0000","last_unstale":"2023-08-09T13:34:08.370229+0000","last_undegraded":"2023-08-09T13:34:08.370229+0000","last_fullsized":"2023-08-09T13:34:08.370229+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:34:06.173547+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:34:06.173547+0000","last_clean_scrub_stamp":"2023-08-09T13:34:06.173547+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T18:36:44.021636+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,7,4],"acting":[5,0,7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1d","version":"19'2","reported_seq":21,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-08-09T13:34:08.370340+0000","last_change":"2023-08-09T13:34:02.109750+0000","last_active":"2023-08-09T13:34:08.370340+0000","last_peered":"2023-08-09T13:34:08.370340+0000","last_clean":"2023-08-09T13:34:08.370340+0000","last_became_active":"2023-08-09T13:34:02.109445+0000","last_became_peered":"2023-08-09T13:34:02.109445+0000","last_unstale":"2023-08-09T13:34:08.370340+0000","last_undegraded":"2023-08-09T13:34:08.370340+0000","last_fullsized":"2023-08-09T13:34:08.370340+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:34:01.052721+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:34:01.052721+0000","last_clean_scrub_stamp":"2023-08-09T13:34:01.052721+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T22:30:05.448240+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1a","version":"0'0","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-08-09T13:34:08.371675+0000","last_change":"2023-08-09T13:34:03.145198+0000","last_active":"2023-08-09T13:34:08.371675+0000","last_peered":"2023-08-09T13:34:08.371675+0000","last_clean":"2023-08-09T13:34:08.371675+0000","last_became_active":"2023-08-09T13:34:03.144756+0000","last_became_peered":"2023-08-09T13:34:03.144756+0000","last_unstale":"2023-08-09T13:34:08.371675+0000","last_undegraded":"2023-08-09T13:34:08.371675+0000","last_fullsized":"2023-08-09T13:34:08.371675+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:34:02.064154+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:34:02.064154+0000","last_clean_scrub_stamp":"2023-08-09T13:34:02.064154+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T16:10:58.188522+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1a","version":"0'0","reported_seq":13,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-08-09T13:34:08.369069+0000","last_change":"2023-08-09T13:34:07.238058+0000","last_active":"2023-08-09T13:34:08.369069+0000","last_peered":"2023-08-09T13:34:08.369069+0000","last_clean":"2023-08-09T13:34:08.369069+0000","last_became_active":"2023-08-09T13:34:07.237575+0000","last_became_peered":"2023-08-09T13:34:07.237575+0000","last_unstale":"2023-08-09T13:34:08.369069+0000","last_undegraded":"2023-08-09T13:34:08.369069+0000","last_fullsized":"2023-08-09T13:34:08.369069+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:34:06.173547+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:34:06.173547+0000","last_clean_scrub_stamp":"2023-08-09T13:34:06.173547+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T21:18:23.660943+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,1,6],"acting":[7,4,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1c","version":"19'2","reported_seq":21,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-08-09T13:34:08.369633+0000","last_change":"2023-08-09T13:34:02.107853+0000","last_active":"2023-08-09T13:34:08.369633+0000","last_peered":"2023-08-09T13:34:08.369633+0000","last_clean":"2023-08-09T13:34:08.369633+0000","last_became_active":"2023-08-09T13:34:02.107571+0000","last_became_peered":"2023-08-09T13:34:02.107571+0000","last_unstale":"2023-08-09T13:34:08.369633+0000","last_undegraded":"2023-08-09T13:34:08.369633+0000","last_fullsized":"2023-08-09T13:34:08.369633+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:34:01.052721+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:34:01.052721+0000","last_clean_scrub_stamp":"2023-08-09T13:34:01.052721+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T19:42:32.721893+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1b","version":"0'0","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-08-09T13:34:08.371599+0000","last_change":"2023-08-09T13:34:03.148927+0000","last_active":"2023-08-09T13:34:08.371599+0000","last_peered":"2023-08-09T13:34:08.371599+0000","last_clean":"2023-08-09T13:34:08.371599+0000","last_became_active":"2023-08-09T13:34:03.148582+0000","last_became_peered":"2023-08-09T13:34:03.148582+0000","last_unstale":"2023-08-09T13:34:08.371599+0000","last_undegraded":"2023-08-09T13:34:08.371599+0000","last_fullsized":"2023-08-09T13:34:08.371599+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:34:02.064154+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:34:02.064154+0000","last_clean_scrub_stamp":"2023-08-09T13:34:02.064154+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T17:58:47.470353+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1d","version":"0'0","reported_seq":13,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-08-09T13:34:08.372087+0000","last_change":"2023-08-09T13:34:07.246175+0000","last_active":"2023-08-09T13:34:08.372087+0000","last_peered":"2023-08-09T13:34:08.372087+0000","last_clean":"2023-08-09T13:34:08.372087+0000","last_became_active":"2023-08-09T13:34:07.245757+0000","last_became_peered":"2023-08-09T13:34:07.245757+0000","last_unstale":"2023-08-09T13:34:08.372087+0000","last_undegraded":"2023-08-09T13:34:08.372087+0000","last_fullsized":"2023-08-09T13:34:08.372087+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:34:06.173547+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:34:06.173547+0000","last_clean_scrub_stamp":"2023-08-09T13:34:06.173547+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T18:07:35.814562+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,4,0],"acting":[1,3,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.1b","version":"19'1","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-08-09T13:34:08.210113+0000","last_change":"2023-08-09T13:34:02.109573+0000","last_active":"2023-08-09T13:34:08.210113+0000","last_peered":"2023-08-09T13:34:08.210113+0000","last_clean":"2023-08-09T13:34:08.210113+0000","last_became_active":"2023-08-09T13:34:02.109015+0000","last_became_peered":"2023-08-09T13:34:02.109015+0000","last_unstale":"2023-08-09T13:34:08.210113+0000","last_undegraded":"2023-08-09T13:34:08.210113+0000","last_fullsized":"2023-08-09T13:34:08.210113+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:34:01.052721+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:34:01.052721+0000","last_clean_scrub_stamp":"2023-08-09T13:34:01.052721+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T13:44:49.753930+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1c","version":"0'0","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-08-09T13:34:08.212417+0000","last_change":"2023-08-09T13:34:03.162651+0000","last_active":"2023-08-09T13:34:08.212417+0000","last_peered":"2023-08-09T13:34:08.212417+0000","last_clean":"2023-08-09T13:34:08.212417+0000","last_became_active":"2023-08-09T13:34:03.161575+0000","last_became_peered":"2023-08-09T13:34:03.161575+0000","last_unstale":"2023-08-09T13:34:08.212417+0000","last_undegraded":"2023-08-09T13:34:08.212417+0000","last_fullsized":"2023-08-09T13:34:08.212417+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:34:02.064154+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:34:02.064154+0000","last_clean_scrub_stamp":"2023-08-09T13:34:02.064154+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-11T00:30:20.150060+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.1c","version":"0'0","reported_seq":13,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-08-09T13:34:08.372392+0000","last_change":"2023-08-09T13:34:07.242334+0000","last_active":"2023-08-09T13:34:08.372392+0000","last_peered":"2023-08-09T13:34:08.372392+0000","last_clean":"2023-08-09T13:34:08.372392+0000","last_became_active":"2023-08-09T13:34:07.241841+0000","last_became_peered":"2023-08-09T13:34:07.241841+0000","last_unstale":"2023-08-09T13:34:08.372392+0000","last_undegraded":"2023-08-09T13:34:08.372392+0000","last_fullsized":"2023-08-09T13:34:08.372392+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:34:06.173547+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:34:06.173547+0000","last_clean_scrub_stamp":"2023-08-09T13:34:06.173547+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T14:35:09.213789+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,2,0],"acting":[4,3,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.1a","version":"19'1","reported_seq":22,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-08-09T13:34:08.372077+0000","last_change":"2023-08-09T13:34:02.104004+0000","last_active":"2023-08-09T13:34:08.372077+0000","last_peered":"2023-08-09T13:34:08.372077+0000","last_clean":"2023-08-09T13:34:08.372077+0000","last_became_active":"2023-08-09T13:34:02.103701+0000","last_became_peered":"2023-08-09T13:34:02.103701+0000","last_unstale":"2023-08-09T13:34:08.372077+0000","last_undegraded":"2023-08-09T13:34:08.372077+0000","last_fullsized":"2023-08-09T13:34:08.372077+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:34:01.052721+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:34:01.052721+0000","last_clean_scrub_stamp":"2023-08-09T13:34:01.052721+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T18:58:30.540971+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.1d","version":"0'0","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-08-09T13:34:08.371028+0000","last_change":"2023-08-09T13:34:03.156422+0000","last_active":"2023-08-09T13:34:08.371028+0000","last_peered":"2023-08-09T13:34:08.371028+0000","last_clean":"2023-08-09T13:34:08.371028+0000","last_became_active":"2023-08-09T13:34:03.155091+0000","last_became_peered":"2023-08-09T13:34:03.155091+0000","last_unstale":"2023-08-09T13:34:08.371028+0000","last_undegraded":"2023-08-09T13:34:08.371028+0000","last_fullsized":"2023-08-09T13:34:08.371028+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:34:02.064154+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:34:02.064154+0000","last_clean_scrub_stamp":"2023-08-09T13:34:02.064154+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T15:06:40.553178+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.1f","version":"0'0","reported_seq":13,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-08-09T13:34:08.213034+0000","last_change":"2023-08-09T13:34:07.241585+0000","last_active":"2023-08-09T13:34:08.213034+0000","last_peered":"2023-08-09T13:34:08.213034+0000","last_clean":"2023-08-09T13:34:08.213034+0000","last_became_active":"2023-08-09T13:34:07.241206+0000","last_became_peered":"2023-08-09T13:34:07.241206+0000","last_unstale":"2023-08-09T13:34:08.213034+0000","last_undegraded":"2023-08-09T13:34:08.213034+0000","last_fullsized":"2023-08-09T13:34:08.213034+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:34:06.173547+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:34:06.173547+0000","last_clean_scrub_stamp":"2023-08-09T13:34:06.173547+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-11T00:43:38.951241+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,3,7],"acting":[6,4,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":21,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-08-09T13:34:08.368085+0000","last_change":"2023-08-09T13:34:02.104251+0000","last_active":"2023-08-09T13:34:08.368085+0000","last_peered":"2023-08-09T13:34:08.368085+0000","last_clean":"2023-08-09T13:34:08.368085+0000","last_became_active":"2023-08-09T13:34:02.103920+0000","last_became_peered":"2023-08-09T13:34:02.103920+0000","last_unstale":"2023-08-09T13:34:08.368085+0000","last_undegraded":"2023-08-09T13:34:08.368085+0000","last_fullsized":"2023-08-09T13:34:08.368085+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:34:01.052721+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:34:01.052721+0000","last_clean_scrub_stamp":"2023-08-09T13:34:01.052721+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T18:36:38.900810+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1e","version":"0'0","reported_seq":17,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-08-09T13:34:08.208574+0000","last_change":"2023-08-09T13:34:03.172919+0000","last_active":"2023-08-09T13:34:08.208574+0000","last_peered":"2023-08-09T13:34:08.208574+0000","last_clean":"2023-08-09T13:34:08.208574+0000","last_became_active":"2023-08-09T13:34:03.172587+0000","last_became_peered":"2023-08-09T13:34:03.172587+0000","last_unstale":"2023-08-09T13:34:08.208574+0000","last_undegraded":"2023-08-09T13:34:08.208574+0000","last_fullsized":"2023-08-09T13:34:08.208574+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:34:02.064154+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:34:02.064154+0000","last_clean_scrub_stamp":"2023-08-09T13:34:02.064154+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T15:53:25.307248+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.1e","version":"0'0","reported_seq":13,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-08-09T13:34:08.209416+0000","last_change":"2023-08-09T13:34:07.236079+0000","last_active":"2023-08-09T13:34:08.209416+0000","last_peered":"2023-08-09T13:34:08.209416+0000","last_clean":"2023-08-09T13:34:08.209416+0000","last_became_active":"2023-08-09T13:34:07.234955+0000","last_became_peered":"2023-08-09T13:34:07.234955+0000","last_unstale":"2023-08-09T13:34:08.209416+0000","last_undegraded":"2023-08-09T13:34:08.209416+0000","last_fullsized":"2023-08-09T13:34:08.209416+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:34:06.173547+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:34:06.173547+0000","last_clean_scrub_stamp":"2023-08-09T13:34:06.173547+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T22:30:10.569066+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,1,2],"acting":[0,7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":21,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-08-09T13:34:08.368549+0000","last_change":"2023-08-09T13:34:02.103043+0000","last_active":"2023-08-09T13:34:08.368549+0000","last_peered":"2023-08-09T13:34:08.368549+0000","last_clean":"2023-08-09T13:34:08.368549+0000","last_became_active":"2023-08-09T13:34:02.102724+0000","last_became_peered":"2023-08-09T13:34:02.102724+0000","last_unstale":"2023-08-09T13:34:08.368549+0000","last_undegraded":"2023-08-09T13:34:08.368549+0000","last_fullsized":"2023-08-09T13:34:08.368549+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:34:01.052721+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:34:01.052721+0000","last_clean_scrub_stamp":"2023-08-09T13:34:01.052721+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T17:59:27.409131+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1f","version":"0'0","reported_seq":17,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-08-09T13:34:08.213083+0000","last_change":"2023-08-09T13:34:03.150663+0000","last_active":"2023-08-09T13:34:08.213083+0000","last_peered":"2023-08-09T13:34:08.213083+0000","last_clean":"2023-08-09T13:34:08.213083+0000","last_became_active":"2023-08-09T13:34:03.150322+0000","last_became_peered":"2023-08-09T13:34:03.150322+0000","last_unstale":"2023-08-09T13:34:08.213083+0000","last_undegraded":"2023-08-09T13:34:08.213083+0000","last_fullsized":"2023-08-09T13:34:08.213083+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:34:02.064154+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:34:02.064154+0000","last_clean_scrub_stamp":"2023-08-09T13:34:02.064154+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T22:09:11.225000+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.10","version":"0'0","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-08-09T13:34:08.368129+0000","last_change":"2023-08-09T13:34:03.154639+0000","last_active":"2023-08-09T13:34:08.368129+0000","last_peered":"2023-08-09T13:34:08.368129+0000","last_clean":"2023-08-09T13:34:08.368129+0000","last_became_active":"2023-08-09T13:34:03.151967+0000","last_became_peered":"2023-08-09T13:34:03.151967+0000","last_unstale":"2023-08-09T13:34:08.368129+0000","last_undegraded":"2023-08-09T13:34:08.368129+0000","last_fullsized":"2023-08-09T13:34:08.368129+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:34:02.064154+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:34:02.064154+0000","last_clean_scrub_stamp":"2023-08-09T13:34:02.064154+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-11T00:11:21.887434+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-08-09T13:34:08.210224+0000","last_change":"2023-08-09T13:34:02.108763+0000","last_active":"2023-08-09T13:34:08.210224+0000","last_peered":"2023-08-09T13:34:08.210224+0000","last_clean":"2023-08-09T13:34:08.210224+0000","last_became_active":"2023-08-09T13:34:02.108439+0000","last_became_peered":"2023-08-09T13:34:02.108439+0000","last_unstale":"2023-08-09T13:34:08.210224+0000","last_undegraded":"2023-08-09T13:34:08.210224+0000","last_fullsized":"2023-08-09T13:34:08.210224+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:34:01.052721+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:34:01.052721+0000","last_clean_scrub_stamp":"2023-08-09T13:34:01.052721+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T17:08:06.492548+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.11","version":"0'0","reported_seq":13,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-08-09T13:34:08.211048+0000","last_change":"2023-08-09T13:34:07.241202+0000","last_active":"2023-08-09T13:34:08.211048+0000","last_peered":"2023-08-09T13:34:08.211048+0000","last_clean":"2023-08-09T13:34:08.211048+0000","last_became_active":"2023-08-09T13:34:07.240678+0000","last_became_peered":"2023-08-09T13:34:07.240678+0000","last_unstale":"2023-08-09T13:34:08.211048+0000","last_undegraded":"2023-08-09T13:34:08.211048+0000","last_fullsized":"2023-08-09T13:34:08.211048+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:34:06.173547+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:34:06.173547+0000","last_clean_scrub_stamp":"2023-08-09T13:34:06.173547+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T22:29:57.384386+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,7,1],"acting":[6,4,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-08-09T13:34:08.369040+0000","last_change":"2023-08-09T13:34:03.163640+0000","last_active":"2023-08-09T13:34:08.369040+0000","last_peered":"2023-08-09T13:34:08.369040+0000","last_clean":"2023-08-09T13:34:08.369040+0000","last_became_active":"2023-08-09T13:34:03.163333+0000","last_became_peered":"2023-08-09T13:34:03.163333+0000","last_unstale":"2023-08-09T13:34:08.369040+0000","last_undegraded":"2023-08-09T13:34:08.369040+0000","last_fullsized":"2023-08-09T13:34:08.369040+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:34:02.064154+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:34:02.064154+0000","last_clean_scrub_stamp":"2023-08-09T13:34:02.064154+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T17:55:41.057906+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-08-09T13:34:08.371698+0000","last_change":"2023-08-09T13:34:02.112947+0000","last_active":"2023-08-09T13:34:08.371698+0000","last_peered":"2023-08-09T13:34:08.371698+0000","last_clean":"2023-08-09T13:34:08.371698+0000","last_became_active":"2023-08-09T13:34:02.112545+0000","last_became_peered":"2023-08-09T13:34:02.112545+0000","last_unstale":"2023-08-09T13:34:08.371698+0000","last_undegraded":"2023-08-09T13:34:08.371698+0000","last_fullsized":"2023-08-09T13:34:08.371698+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:34:01.052721+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:34:01.052721+0000","last_clean_scrub_stamp":"2023-08-09T13:34:01.052721+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T18:07:30.693736+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.10","version":"0'0","reported_seq":13,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-08-09T13:34:08.367894+0000","last_change":"2023-08-09T13:34:07.231662+0000","last_active":"2023-08-09T13:34:08.367894+0000","last_peered":"2023-08-09T13:34:08.367894+0000","last_clean":"2023-08-09T13:34:08.367894+0000","last_became_active":"2023-08-09T13:34:07.231214+0000","last_became_peered":"2023-08-09T13:34:07.231214+0000","last_unstale":"2023-08-09T13:34:08.367894+0000","last_undegraded":"2023-08-09T13:34:08.367894+0000","last_fullsized":"2023-08-09T13:34:08.367894+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:34:06.173547+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:34:06.173547+0000","last_clean_scrub_stamp":"2023-08-09T13:34:06.173547+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T21:35:05.575947+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6,0],"acting":[7,4,6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.15","version":"19'3","reported_seq":22,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-08-09T13:34:08.370972+0000","last_change":"2023-08-09T13:34:02.112344+0000","last_active":"2023-08-09T13:34:08.370972+0000","last_peered":"2023-08-09T13:34:08.370972+0000","last_clean":"2023-08-09T13:34:08.370972+0000","last_became_active":"2023-08-09T13:34:02.111749+0000","last_became_peered":"2023-08-09T13:34:02.111749+0000","last_unstale":"2023-08-09T13:34:08.370972+0000","last_undegraded":"2023-08-09T13:34:08.370972+0000","last_fullsized":"2023-08-09T13:34:08.370972+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:34:01.052721+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:34:01.052721+0000","last_clean_scrub_stamp":"2023-08-09T13:34:01.052721+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T15:33:08.276561+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-08-09T13:34:08.369248+0000","last_change":"2023-08-09T13:34:03.154995+0000","last_active":"2023-08-09T13:34:08.369248+0000","last_peered":"2023-08-09T13:34:08.369248+0000","last_clean":"2023-08-09T13:34:08.369248+0000","last_became_active":"2023-08-09T13:34:03.154715+0000","last_became_peered":"2023-08-09T13:34:03.154715+0000","last_unstale":"2023-08-09T13:34:08.369248+0000","last_undegraded":"2023-08-09T13:34:08.369248+0000","last_fullsized":"2023-08-09T13:34:08.369248+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:34:02.064154+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:34:02.064154+0000","last_clean_scrub_stamp":"2023-08-09T13:34:02.064154+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T22:30:06.459673+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.13","version":"0'0","reported_seq":13,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-08-09T13:34:08.368692+0000","last_change":"2023-08-09T13:34:07.241979+0000","last_active":"2023-08-09T13:34:08.368692+0000","last_peered":"2023-08-09T13:34:08.368692+0000","last_clean":"2023-08-09T13:34:08.368692+0000","last_became_active":"2023-08-09T13:34:07.241025+0000","last_became_peered":"2023-08-09T13:34:07.241025+0000","last_unstale":"2023-08-09T13:34:08.368692+0000","last_undegraded":"2023-08-09T13:34:08.368692+0000","last_fullsized":"2023-08-09T13:34:08.368692+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:34:06.173547+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:34:06.173547+0000","last_clean_scrub_stamp":"2023-08-09T13:34:06.173547+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T17:42:24.789866+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,1,5],"acting":[3,0,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.14","version":"24'10","reported_seq":31,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-08-09T13:34:22.970697+0000","last_change":"2023-08-09T13:34:02.113338+0000","last_active":"2023-08-09T13:34:22.970697+0000","last_peered":"2023-08-09T13:34:22.970697+0000","last_clean":"2023-08-09T13:34:22.970697+0000","last_became_active":"2023-08-09T13:34:02.113042+0000","last_became_peered":"2023-08-09T13:34:02.113042+0000","last_unstale":"2023-08-09T13:34:22.970697+0000","last_undegraded":"2023-08-09T13:34:22.970697+0000","last_fullsized":"2023-08-09T13:34:22.970697+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:34:01.052721+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:34:01.052721+0000","last_clean_scrub_stamp":"2023-08-09T13:34:01.052721+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T21:15:54.123137+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":3841,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.13","version":"0'0","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-08-09T13:34:08.371930+0000","last_change":"2023-08-09T13:34:03.148233+0000","last_active":"2023-08-09T13:34:08.371930+0000","last_peered":"2023-08-09T13:34:08.371930+0000","last_clean":"2023-08-09T13:34:08.371930+0000","last_became_active":"2023-08-09T13:34:03.147860+0000","last_became_peered":"2023-08-09T13:34:03.147860+0000","last_unstale":"2023-08-09T13:34:08.371930+0000","last_undegraded":"2023-08-09T13:34:08.371930+0000","last_fullsized":"2023-08-09T13:34:08.371930+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:34:02.064154+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:34:02.064154+0000","last_clean_scrub_stamp":"2023-08-09T13:34:02.064154+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T18:17:08.524229+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.12","version":"0'0","reported_seq":13,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-08-09T13:34:08.368474+0000","last_change":"2023-08-09T13:34:07.236661+0000","last_active":"2023-08-09T13:34:08.368474+0000","last_peered":"2023-08-09T13:34:08.368474+0000","last_clean":"2023-08-09T13:34:08.368474+0000","last_became_active":"2023-08-09T13:34:07.236264+0000","last_became_peered":"2023-08-09T13:34:07.236264+0000","last_unstale":"2023-08-09T13:34:08.368474+0000","last_undegraded":"2023-08-09T13:34:08.368474+0000","last_fullsized":"2023-08-09T13:34:08.368474+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:34:06.173547+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:34:06.173547+0000","last_clean_scrub_stamp":"2023-08-09T13:34:06.173547+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T21:35:05.575947+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,3,7],"acting":[1,5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-08-09T13:34:08.369602+0000","last_change":"2023-08-09T13:34:03.149954+0000","last_active":"2023-08-09T13:34:08.369602+0000","last_peered":"2023-08-09T13:34:08.369602+0000","last_clean":"2023-08-09T13:34:08.369602+0000","last_became_active":"2023-08-09T13:34:03.149648+0000","last_became_peered":"2023-08-09T13:34:03.149648+0000","last_unstale":"2023-08-09T13:34:08.369602+0000","last_undegraded":"2023-08-09T13:34:08.369602+0000","last_fullsized":"2023-08-09T13:34:08.369602+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:34:02.064154+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:34:02.064154+0000","last_clean_scrub_stamp":"2023-08-09T13:34:02.064154+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T18:50:05.285807+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":25,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-08-09T13:34:08.213159+0000","last_change":"2023-08-09T13:33:55.886389+0000","last_active":"2023-08-09T13:34:08.213159+0000","last_peered":"2023-08-09T13:34:08.213159+0000","last_clean":"2023-08-09T13:34:08.213159+0000","last_became_active":"2023-08-09T13:33:55.885638+0000","last_became_peered":"2023-08-09T13:33:55.885638+0000","last_unstale":"2023-08-09T13:34:08.213159+0000","last_undegraded":"2023-08-09T13:34:08.213159+0000","last_fullsized":"2023-08-09T13:34:08.213159+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:33:54.845869+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:33:54.845869+0000","last_clean_scrub_stamp":"2023-08-09T13:33:54.845869+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T23:11:03.836310+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-08-09T13:34:08.370437+0000","last_change":"2023-08-09T13:34:02.114759+0000","last_active":"2023-08-09T13:34:08.370437+0000","last_peered":"2023-08-09T13:34:08.370437+0000","last_clean":"2023-08-09T13:34:08.370437+0000","last_became_active":"2023-08-09T13:34:02.114423+0000","last_became_peered":"2023-08-09T13:34:02.114423+0000","last_unstale":"2023-08-09T13:34:08.370437+0000","last_undegraded":"2023-08-09T13:34:08.370437+0000","last_fullsized":"2023-08-09T13:34:08.370437+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:34:01.052721+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:34:01.052721+0000","last_clean_scrub_stamp":"2023-08-09T13:34:01.052721+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T18:50:04.274374+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.3","version":"0'0","reported_seq":13,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-08-09T13:34:08.209248+0000","last_change":"2023-08-09T13:34:07.244416+0000","last_active":"2023-08-09T13:34:08.209248+0000","last_peered":"2023-08-09T13:34:08.209248+0000","last_clean":"2023-08-09T13:34:08.209248+0000","last_became_active":"2023-08-09T13:34:07.243294+0000","last_became_peered":"2023-08-09T13:34:07.243294+0000","last_unstale":"2023-08-09T13:34:08.209248+0000","last_undegraded":"2023-08-09T13:34:08.209248+0000","last_fullsized":"2023-08-09T13:34:08.209248+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:34:06.173547+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:34:06.173547+0000","last_clean_scrub_stamp":"2023-08-09T13:34:06.173547+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T21:35:05.575947+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,6,5],"acting":[0,1,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":17,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-08-09T13:34:08.209747+0000","last_change":"2023-08-09T13:34:03.151674+0000","last_active":"2023-08-09T13:34:08.209747+0000","last_peered":"2023-08-09T13:34:08.209747+0000","last_clean":"2023-08-09T13:34:08.209747+0000","last_became_active":"2023-08-09T13:34:03.151137+0000","last_became_peered":"2023-08-09T13:34:03.151137+0000","last_unstale":"2023-08-09T13:34:08.209747+0000","last_undegraded":"2023-08-09T13:34:08.209747+0000","last_fullsized":"2023-08-09T13:34:08.209747+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:34:02.064154+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:34:02.064154+0000","last_clean_scrub_stamp":"2023-08-09T13:34:02.064154+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T21:11:40.494329+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":25,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-08-09T13:34:08.212939+0000","last_change":"2023-08-09T13:33:56.401781+0000","last_active":"2023-08-09T13:34:08.212939+0000","last_peered":"2023-08-09T13:34:08.212939+0000","last_clean":"2023-08-09T13:34:08.212939+0000","last_became_active":"2023-08-09T13:33:56.400336+0000","last_became_peered":"2023-08-09T13:33:56.400336+0000","last_unstale":"2023-08-09T13:34:08.212939+0000","last_undegraded":"2023-08-09T13:34:08.212939+0000","last_fullsized":"2023-08-09T13:34:08.212939+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:33:54.845869+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:33:54.845869+0000","last_clean_scrub_stamp":"2023-08-09T13:33:54.845869+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T19:43:07.034435+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":21,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-08-09T13:34:08.372061+0000","last_change":"2023-08-09T13:34:02.106587+0000","last_active":"2023-08-09T13:34:08.372061+0000","last_peered":"2023-08-09T13:34:08.372061+0000","last_clean":"2023-08-09T13:34:08.372061+0000","last_became_active":"2023-08-09T13:34:02.106236+0000","last_became_peered":"2023-08-09T13:34:02.106236+0000","last_unstale":"2023-08-09T13:34:08.372061+0000","last_undegraded":"2023-08-09T13:34:08.372061+0000","last_fullsized":"2023-08-09T13:34:08.372061+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:34:01.052721+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:34:01.052721+0000","last_clean_scrub_stamp":"2023-08-09T13:34:01.052721+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T13:44:49.753930+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.2","version":"0'0","reported_seq":13,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-08-09T13:34:08.211286+0000","last_change":"2023-08-09T13:34:07.242738+0000","last_active":"2023-08-09T13:34:08.211286+0000","last_peered":"2023-08-09T13:34:08.211286+0000","last_clean":"2023-08-09T13:34:08.211286+0000","last_became_active":"2023-08-09T13:34:07.242314+0000","last_became_peered":"2023-08-09T13:34:07.242314+0000","last_unstale":"2023-08-09T13:34:08.211286+0000","last_undegraded":"2023-08-09T13:34:08.211286+0000","last_fullsized":"2023-08-09T13:34:08.211286+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:34:06.173547+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:34:06.173547+0000","last_clean_scrub_stamp":"2023-08-09T13:34:06.173547+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T18:00:37.387381+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,5,2],"acting":[6,0,5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-08-09T13:34:08.368126+0000","last_change":"2023-08-09T13:34:03.163805+0000","last_active":"2023-08-09T13:34:08.368126+0000","last_peered":"2023-08-09T13:34:08.368126+0000","last_clean":"2023-08-09T13:34:08.368126+0000","last_became_active":"2023-08-09T13:34:03.163407+0000","last_became_peered":"2023-08-09T13:34:03.163407+0000","last_unstale":"2023-08-09T13:34:08.368126+0000","last_undegraded":"2023-08-09T13:34:08.368126+0000","last_fullsized":"2023-08-09T13:34:08.368126+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:34:02.064154+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:34:02.064154+0000","last_clean_scrub_stamp":"2023-08-09T13:34:02.064154+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T23:33:49.011321+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":27,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-08-09T13:34:08.373438+0000","last_change":"2023-08-09T13:33:56.404236+0000","last_active":"2023-08-09T13:34:08.373438+0000","last_peered":"2023-08-09T13:34:08.373438+0000","last_clean":"2023-08-09T13:34:08.373438+0000","last_became_active":"2023-08-09T13:33:56.402485+0000","last_became_peered":"2023-08-09T13:33:56.402485+0000","last_unstale":"2023-08-09T13:34:08.373438+0000","last_undegraded":"2023-08-09T13:34:08.373438+0000","last_fullsized":"2023-08-09T13:34:08.373438+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:33:54.845869+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:33:54.845869+0000","last_clean_scrub_stamp":"2023-08-09T13:33:54.845869+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T18:59:14.817691+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.7","version":"19'2","reported_seq":23,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-08-09T13:34:08.369861+0000","last_change":"2023-08-09T13:34:02.108061+0000","last_active":"2023-08-09T13:34:08.369861+0000","last_peered":"2023-08-09T13:34:08.369861+0000","last_clean":"2023-08-09T13:34:08.369861+0000","last_became_active":"2023-08-09T13:34:02.107766+0000","last_became_peered":"2023-08-09T13:34:02.107766+0000","last_unstale":"2023-08-09T13:34:08.369861+0000","last_undegraded":"2023-08-09T13:34:08.369861+0000","last_fullsized":"2023-08-09T13:34:08.369861+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:34:01.052721+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:34:01.052721+0000","last_clean_scrub_stamp":"2023-08-09T13:34:01.052721+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T16:14:05.720686+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.1","version":"0'0","reported_seq":13,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-08-09T13:34:08.368770+0000","last_change":"2023-08-09T13:34:07.236332+0000","last_active":"2023-08-09T13:34:08.368770+0000","last_peered":"2023-08-09T13:34:08.368770+0000","last_clean":"2023-08-09T13:34:08.368770+0000","last_became_active":"2023-08-09T13:34:07.235923+0000","last_became_peered":"2023-08-09T13:34:07.235923+0000","last_unstale":"2023-08-09T13:34:08.368770+0000","last_undegraded":"2023-08-09T13:34:08.368770+0000","last_fullsized":"2023-08-09T13:34:08.368770+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:34:06.173547+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:34:06.173547+0000","last_clean_scrub_stamp":"2023-08-09T13:34:06.173547+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-11T01:07:37.438752+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,7,0],"acting":[4,3,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-08-09T13:34:08.369699+0000","last_change":"2023-08-09T13:34:03.149492+0000","last_active":"2023-08-09T13:34:08.369699+0000","last_peered":"2023-08-09T13:34:08.369699+0000","last_clean":"2023-08-09T13:34:08.369699+0000","last_became_active":"2023-08-09T13:34:03.148649+0000","last_became_peered":"2023-08-09T13:34:03.148649+0000","last_unstale":"2023-08-09T13:34:08.369699+0000","last_undegraded":"2023-08-09T13:34:08.369699+0000","last_fullsized":"2023-08-09T13:34:08.369699+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:34:02.064154+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:34:02.064154+0000","last_clean_scrub_stamp":"2023-08-09T13:34:02.064154+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T21:57:54.183369+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":27,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-08-09T13:34:08.370409+0000","last_change":"2023-08-09T13:33:55.889565+0000","last_active":"2023-08-09T13:34:08.370409+0000","last_peered":"2023-08-09T13:34:08.370409+0000","last_clean":"2023-08-09T13:34:08.370409+0000","last_became_active":"2023-08-09T13:33:55.889164+0000","last_became_peered":"2023-08-09T13:33:55.889164+0000","last_unstale":"2023-08-09T13:34:08.370409+0000","last_undegraded":"2023-08-09T13:34:08.370409+0000","last_fullsized":"2023-08-09T13:34:08.370409+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:33:54.845869+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:33:54.845869+0000","last_clean_scrub_stamp":"2023-08-09T13:33:54.845869+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-11T01:07:26.111074+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.6","version":"21'1","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-08-09T13:34:08.209027+0000","last_change":"2023-08-09T13:34:02.096482+0000","last_active":"2023-08-09T13:34:08.209027+0000","last_peered":"2023-08-09T13:34:08.209027+0000","last_clean":"2023-08-09T13:34:08.209027+0000","last_became_active":"2023-08-09T13:34:02.095287+0000","last_became_peered":"2023-08-09T13:34:02.095287+0000","last_unstale":"2023-08-09T13:34:08.209027+0000","last_undegraded":"2023-08-09T13:34:08.209027+0000","last_fullsized":"2023-08-09T13:34:08.209027+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:34:01.052721+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:34:01.052721+0000","last_clean_scrub_stamp":"2023-08-09T13:34:01.052721+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T18:36:38.900810+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.0","version":"0'0","reported_seq":13,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-08-09T13:34:08.367857+0000","last_change":"2023-08-09T13:34:07.244568+0000","last_active":"2023-08-09T13:34:08.367857+0000","last_peered":"2023-08-09T13:34:08.367857+0000","last_clean":"2023-08-09T13:34:08.367857+0000","last_became_active":"2023-08-09T13:34:07.243669+0000","last_became_peered":"2023-08-09T13:34:07.243669+0000","last_unstale":"2023-08-09T13:34:08.367857+0000","last_undegraded":"2023-08-09T13:34:08.367857+0000","last_fullsized":"2023-08-09T13:34:08.367857+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:34:06.173547+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:34:06.173547+0000","last_clean_scrub_stamp":"2023-08-09T13:34:06.173547+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T20:31:33.361620+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,4,0],"acting":[3,1,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.6","version":"0'0","reported_seq":17,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-08-09T13:34:08.208668+0000","last_change":"2023-08-09T13:34:03.139485+0000","last_active":"2023-08-09T13:34:08.208668+0000","last_peered":"2023-08-09T13:34:08.208668+0000","last_clean":"2023-08-09T13:34:08.208668+0000","last_became_active":"2023-08-09T13:34:03.139178+0000","last_became_peered":"2023-08-09T13:34:03.139178+0000","last_unstale":"2023-08-09T13:34:08.208668+0000","last_undegraded":"2023-08-09T13:34:08.208668+0000","last_fullsized":"2023-08-09T13:34:08.208668+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:34:02.064154+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:34:02.064154+0000","last_clean_scrub_stamp":"2023-08-09T13:34:02.064154+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T19:42:33.733326+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-08-09T13:34:08.207236+0000","last_change":"2023-08-09T13:34:02.110719+0000","last_active":"2023-08-09T13:34:08.207236+0000","last_peered":"2023-08-09T13:34:08.207236+0000","last_clean":"2023-08-09T13:34:08.207236+0000","last_became_active":"2023-08-09T13:34:02.110431+0000","last_became_peered":"2023-08-09T13:34:02.110431+0000","last_unstale":"2023-08-09T13:34:08.207236+0000","last_undegraded":"2023-08-09T13:34:08.207236+0000","last_fullsized":"2023-08-09T13:34:08.207236+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:34:01.052721+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:34:01.052721+0000","last_clean_scrub_stamp":"2023-08-09T13:34:01.052721+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T23:24:07.266385+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.0","version":"16'117","reported_seq":175,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-08-09T13:34:08.368906+0000","last_change":"2023-08-09T13:33:55.886345+0000","last_active":"2023-08-09T13:34:08.368906+0000","last_peered":"2023-08-09T13:34:08.368906+0000","last_clean":"2023-08-09T13:34:08.368906+0000","last_became_active":"2023-08-09T13:33:55.885781+0000","last_became_peered":"2023-08-09T13:33:55.885781+0000","last_unstale":"2023-08-09T13:34:08.368906+0000","last_undegraded":"2023-08-09T13:34:08.368906+0000","last_fullsized":"2023-08-09T13:34:08.368906+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:33:54.845869+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:33:54.845869+0000","last_clean_scrub_stamp":"2023-08-09T13:33:54.845869+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T18:49:58.067523+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.3","version":"0'0","reported_seq":27,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-08-09T13:34:08.370928+0000","last_change":"2023-08-09T13:33:55.886749+0000","last_active":"2023-08-09T13:34:08.370928+0000","last_peered":"2023-08-09T13:34:08.370928+0000","last_clean":"2023-08-09T13:34:08.370928+0000","last_became_active":"2023-08-09T13:33:55.886116+0000","last_became_peered":"2023-08-09T13:33:55.886116+0000","last_unstale":"2023-08-09T13:34:08.370928+0000","last_undegraded":"2023-08-09T13:34:08.370928+0000","last_fullsized":"2023-08-09T13:34:08.370928+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:33:54.845869+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:33:54.845869+0000","last_clean_scrub_stamp":"2023-08-09T13:33:54.845869+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T17:46:15.820922+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.7","version":"0'0","reported_seq":13,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-08-09T13:34:08.371420+0000","last_change":"2023-08-09T13:34:07.235784+0000","last_active":"2023-08-09T13:34:08.371420+0000","last_peered":"2023-08-09T13:34:08.371420+0000","last_clean":"2023-08-09T13:34:08.371420+0000","last_became_active":"2023-08-09T13:34:07.235282+0000","last_became_peered":"2023-08-09T13:34:07.235282+0000","last_unstale":"2023-08-09T13:34:08.371420+0000","last_undegraded":"2023-08-09T13:34:08.371420+0000","last_fullsized":"2023-08-09T13:34:08.371420+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:34:06.173547+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:34:06.173547+0000","last_clean_scrub_stamp":"2023-08-09T13:34:06.173547+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T23:52:31.847779+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,0,4],"acting":[5,1,0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.7","version":"0'0","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-08-09T13:34:08.372407+0000","last_change":"2023-08-09T13:34:03.149568+0000","last_active":"2023-08-09T13:34:08.372407+0000","last_peered":"2023-08-09T13:34:08.372407+0000","last_clean":"2023-08-09T13:34:08.372407+0000","last_became_active":"2023-08-09T13:34:03.148937+0000","last_became_peered":"2023-08-09T13:34:03.148937+0000","last_unstale":"2023-08-09T13:34:08.372407+0000","last_undegraded":"2023-08-09T13:34:08.372407+0000","last_fullsized":"2023-08-09T13:34:08.372407+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:34:02.064154+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:34:02.064154+0000","last_clean_scrub_stamp":"2023-08-09T13:34:02.064154+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T23:34:54.506258+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":21,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-08-09T13:34:08.367679+0000","last_change":"2023-08-09T13:34:02.102795+0000","last_active":"2023-08-09T13:34:08.367679+0000","last_peered":"2023-08-09T13:34:08.367679+0000","last_clean":"2023-08-09T13:34:08.367679+0000","last_became_active":"2023-08-09T13:34:02.101883+0000","last_became_peered":"2023-08-09T13:34:02.101883+0000","last_unstale":"2023-08-09T13:34:08.367679+0000","last_undegraded":"2023-08-09T13:34:08.367679+0000","last_fullsized":"2023-08-09T13:34:08.367679+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:34:01.052721+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:34:01.052721+0000","last_clean_scrub_stamp":"2023-08-09T13:34:01.052721+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T19:22:07.585850+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.2","version":"0'0","reported_seq":25,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-08-09T13:34:08.369310+0000","last_change":"2023-08-09T13:33:56.402794+0000","last_active":"2023-08-09T13:34:08.369310+0000","last_peered":"2023-08-09T13:34:08.369310+0000","last_clean":"2023-08-09T13:34:08.369310+0000","last_became_active":"2023-08-09T13:33:56.402038+0000","last_became_peered":"2023-08-09T13:33:56.402038+0000","last_unstale":"2023-08-09T13:34:08.369310+0000","last_undegraded":"2023-08-09T13:34:08.369310+0000","last_fullsized":"2023-08-09T13:34:08.369310+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:33:54.845869+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:33:54.845869+0000","last_clean_scrub_stamp":"2023-08-09T13:33:54.845869+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T18:07:24.486884+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.6","version":"0'0","reported_seq":13,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-08-09T13:34:08.212013+0000","last_change":"2023-08-09T13:34:07.244168+0000","last_active":"2023-08-09T13:34:08.212013+0000","last_peered":"2023-08-09T13:34:08.212013+0000","last_clean":"2023-08-09T13:34:08.212013+0000","last_became_active":"2023-08-09T13:34:07.241903+0000","last_became_peered":"2023-08-09T13:34:07.241903+0000","last_unstale":"2023-08-09T13:34:08.212013+0000","last_undegraded":"2023-08-09T13:34:08.212013+0000","last_fullsized":"2023-08-09T13:34:08.212013+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:34:06.173547+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:34:06.173547+0000","last_clean_scrub_stamp":"2023-08-09T13:34:06.173547+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-11T00:23:22.192340+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,1,7],"acting":[2,5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-08-09T13:34:08.371697+0000","last_change":"2023-08-09T13:34:03.153039+0000","last_active":"2023-08-09T13:34:08.371697+0000","last_peered":"2023-08-09T13:34:08.371697+0000","last_clean":"2023-08-09T13:34:08.371697+0000","last_became_active":"2023-08-09T13:34:03.152401+0000","last_became_peered":"2023-08-09T13:34:03.152401+0000","last_unstale":"2023-08-09T13:34:08.371697+0000","last_undegraded":"2023-08-09T13:34:08.371697+0000","last_fullsized":"2023-08-09T13:34:08.371697+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:34:02.064154+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:34:02.064154+0000","last_clean_scrub_stamp":"2023-08-09T13:34:02.064154+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T21:18:19.551549+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":21,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-08-09T13:34:08.372303+0000","last_change":"2023-08-09T13:34:02.108657+0000","last_active":"2023-08-09T13:34:08.372303+0000","last_peered":"2023-08-09T13:34:08.372303+0000","last_clean":"2023-08-09T13:34:08.372303+0000","last_became_active":"2023-08-09T13:34:02.105481+0000","last_became_peered":"2023-08-09T13:34:02.105481+0000","last_unstale":"2023-08-09T13:34:08.372303+0000","last_undegraded":"2023-08-09T13:34:08.372303+0000","last_fullsized":"2023-08-09T13:34:08.372303+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:34:01.052721+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:34:01.052721+0000","last_clean_scrub_stamp":"2023-08-09T13:34:01.052721+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T22:51:18.775151+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":25,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-08-09T13:34:08.368026+0000","last_change":"2023-08-09T13:33:56.401123+0000","last_active":"2023-08-09T13:34:08.368026+0000","last_peered":"2023-08-09T13:34:08.368026+0000","last_clean":"2023-08-09T13:34:08.368026+0000","last_became_active":"2023-08-09T13:33:56.399933+0000","last_became_peered":"2023-08-09T13:33:56.399933+0000","last_unstale":"2023-08-09T13:34:08.368026+0000","last_undegraded":"2023-08-09T13:34:08.368026+0000","last_fullsized":"2023-08-09T13:34:08.368026+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:33:54.845869+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:33:54.845869+0000","last_clean_scrub_stamp":"2023-08-09T13:33:54.845869+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T23:24:01.059533+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.5","version":"0'0","reported_seq":13,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-08-09T13:34:08.208281+0000","last_change":"2023-08-09T13:34:07.235418+0000","last_active":"2023-08-09T13:34:08.208281+0000","last_peered":"2023-08-09T13:34:08.208281+0000","last_clean":"2023-08-09T13:34:08.208281+0000","last_became_active":"2023-08-09T13:34:07.234889+0000","last_became_peered":"2023-08-09T13:34:07.234889+0000","last_unstale":"2023-08-09T13:34:08.208281+0000","last_undegraded":"2023-08-09T13:34:08.208281+0000","last_fullsized":"2023-08-09T13:34:08.208281+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:34:06.173547+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:34:06.173547+0000","last_clean_scrub_stamp":"2023-08-09T13:34:06.173547+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T20:46:19.756813+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,4,6],"acting":[0,1,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.5","version":"0'0","reported_seq":17,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-08-09T13:34:08.212552+0000","last_change":"2023-08-09T13:34:03.159726+0000","last_active":"2023-08-09T13:34:08.212552+0000","last_peered":"2023-08-09T13:34:08.212552+0000","last_clean":"2023-08-09T13:34:08.212552+0000","last_became_active":"2023-08-09T13:34:03.159238+0000","last_became_peered":"2023-08-09T13:34:03.159238+0000","last_unstale":"2023-08-09T13:34:08.212552+0000","last_undegraded":"2023-08-09T13:34:08.212552+0000","last_fullsized":"2023-08-09T13:34:08.212552+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:34:02.064154+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:34:02.064154+0000","last_clean_scrub_stamp":"2023-08-09T13:34:02.064154+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T17:32:16.845908+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":21,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-08-09T13:34:08.368904+0000","last_change":"2023-08-09T13:34:02.114840+0000","last_active":"2023-08-09T13:34:08.368904+0000","last_peered":"2023-08-09T13:34:08.368904+0000","last_clean":"2023-08-09T13:34:08.368904+0000","last_became_active":"2023-08-09T13:34:02.114547+0000","last_became_peered":"2023-08-09T13:34:02.114547+0000","last_unstale":"2023-08-09T13:34:08.368904+0000","last_undegraded":"2023-08-09T13:34:08.368904+0000","last_fullsized":"2023-08-09T13:34:08.368904+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:34:01.052721+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:34:01.052721+0000","last_clean_scrub_stamp":"2023-08-09T13:34:01.052721+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T13:43:19.071421+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":25,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-08-09T13:34:08.369269+0000","last_change":"2023-08-09T13:33:56.401406+0000","last_active":"2023-08-09T13:34:08.369269+0000","last_peered":"2023-08-09T13:34:08.369269+0000","last_clean":"2023-08-09T13:34:08.369269+0000","last_became_active":"2023-08-09T13:33:56.400487+0000","last_became_peered":"2023-08-09T13:33:56.400487+0000","last_unstale":"2023-08-09T13:34:08.369269+0000","last_undegraded":"2023-08-09T13:34:08.369269+0000","last_fullsized":"2023-08-09T13:34:08.369269+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:33:54.845869+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:33:54.845869+0000","last_clean_scrub_stamp":"2023-08-09T13:33:54.845869+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T23:52:20.520101+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.4","version":"0'0","reported_seq":13,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-08-09T13:34:08.370501+0000","last_change":"2023-08-09T13:34:07.249799+0000","last_active":"2023-08-09T13:34:08.370501+0000","last_peered":"2023-08-09T13:34:08.370501+0000","last_clean":"2023-08-09T13:34:08.370501+0000","last_became_active":"2023-08-09T13:34:07.249286+0000","last_became_peered":"2023-08-09T13:34:07.249286+0000","last_unstale":"2023-08-09T13:34:08.370501+0000","last_undegraded":"2023-08-09T13:34:08.370501+0000","last_fullsized":"2023-08-09T13:34:08.370501+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:34:06.173547+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:34:06.173547+0000","last_clean_scrub_stamp":"2023-08-09T13:34:06.173547+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T15:43:48.184972+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2,5,3],"acting":[7,2,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.f","version":"0'0","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-08-09T13:34:08.372421+0000","last_change":"2023-08-09T13:34:03.151508+0000","last_active":"2023-08-09T13:34:08.372421+0000","last_peered":"2023-08-09T13:34:08.372421+0000","last_clean":"2023-08-09T13:34:08.372421+0000","last_became_active":"2023-08-09T13:34:03.149550+0000","last_became_peered":"2023-08-09T13:34:03.149550+0000","last_unstale":"2023-08-09T13:34:08.372421+0000","last_undegraded":"2023-08-09T13:34:08.372421+0000","last_fullsized":"2023-08-09T13:34:08.372421+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:34:02.064154+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:34:02.064154+0000","last_clean_scrub_stamp":"2023-08-09T13:34:02.064154+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T22:51:11.201456+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":21,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-08-09T13:34:08.368991+0000","last_change":"2023-08-09T13:34:02.095946+0000","last_active":"2023-08-09T13:34:08.368991+0000","last_peered":"2023-08-09T13:34:08.368991+0000","last_clean":"2023-08-09T13:34:08.368991+0000","last_became_active":"2023-08-09T13:34:02.095325+0000","last_became_peered":"2023-08-09T13:34:02.095325+0000","last_unstale":"2023-08-09T13:34:08.368991+0000","last_undegraded":"2023-08-09T13:34:08.368991+0000","last_fullsized":"2023-08-09T13:34:08.368991+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:34:01.052721+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:34:01.052721+0000","last_clean_scrub_stamp":"2023-08-09T13:34:01.052721+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-11T00:06:29.000035+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.e","version":"0'0","reported_seq":13,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-08-09T13:34:08.373803+0000","last_change":"2023-08-09T13:34:07.232690+0000","last_active":"2023-08-09T13:34:08.373803+0000","last_peered":"2023-08-09T13:34:08.373803+0000","last_clean":"2023-08-09T13:34:08.373803+0000","last_became_active":"2023-08-09T13:34:07.232326+0000","last_became_peered":"2023-08-09T13:34:07.232326+0000","last_unstale":"2023-08-09T13:34:08.373803+0000","last_undegraded":"2023-08-09T13:34:08.373803+0000","last_fullsized":"2023-08-09T13:34:08.373803+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:34:06.173547+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:34:06.173547+0000","last_clean_scrub_stamp":"2023-08-09T13:34:06.173547+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T13:37:38.237696+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,0,2],"acting":[4,5,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.e","version":"0'0","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-08-09T13:34:08.373562+0000","last_change":"2023-08-09T13:34:03.149791+0000","last_active":"2023-08-09T13:34:08.373562+0000","last_peered":"2023-08-09T13:34:08.373562+0000","last_clean":"2023-08-09T13:34:08.373562+0000","last_became_active":"2023-08-09T13:34:03.149525+0000","last_became_peered":"2023-08-09T13:34:03.149525+0000","last_unstale":"2023-08-09T13:34:08.373562+0000","last_undegraded":"2023-08-09T13:34:08.373562+0000","last_fullsized":"2023-08-09T13:34:08.373562+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:34:02.064154+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:34:02.064154+0000","last_clean_scrub_stamp":"2023-08-09T13:34:02.064154+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T14:53:59.337109+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":21,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-08-09T13:34:08.369963+0000","last_change":"2023-08-09T13:34:02.105359+0000","last_active":"2023-08-09T13:34:08.369963+0000","last_peered":"2023-08-09T13:34:08.369963+0000","last_clean":"2023-08-09T13:34:08.369963+0000","last_became_active":"2023-08-09T13:34:02.105027+0000","last_became_peered":"2023-08-09T13:34:02.105027+0000","last_unstale":"2023-08-09T13:34:08.369963+0000","last_undegraded":"2023-08-09T13:34:08.369963+0000","last_fullsized":"2023-08-09T13:34:08.369963+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:34:01.052721+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:34:01.052721+0000","last_clean_scrub_stamp":"2023-08-09T13:34:01.052721+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T13:45:26.240878+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.f","version":"0'0","reported_seq":13,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-08-09T13:34:08.371141+0000","last_change":"2023-08-09T13:34:07.239625+0000","last_active":"2023-08-09T13:34:08.371141+0000","last_peered":"2023-08-09T13:34:08.371141+0000","last_clean":"2023-08-09T13:34:08.371141+0000","last_became_active":"2023-08-09T13:34:07.239002+0000","last_became_peered":"2023-08-09T13:34:07.239002+0000","last_unstale":"2023-08-09T13:34:08.371141+0000","last_undegraded":"2023-08-09T13:34:08.371141+0000","last_fullsized":"2023-08-09T13:34:08.371141+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:34:06.173547+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:34:06.173547+0000","last_clean_scrub_stamp":"2023-08-09T13:34:06.173547+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T21:11:44.603723+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,6,2],"acting":[5,4,6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.d","version":"0'0","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-08-09T13:34:08.373090+0000","last_change":"2023-08-09T13:34:03.152579+0000","last_active":"2023-08-09T13:34:08.373090+0000","last_peered":"2023-08-09T13:34:08.373090+0000","last_clean":"2023-08-09T13:34:08.373090+0000","last_became_active":"2023-08-09T13:34:03.152287+0000","last_became_peered":"2023-08-09T13:34:03.152287+0000","last_unstale":"2023-08-09T13:34:08.373090+0000","last_undegraded":"2023-08-09T13:34:08.373090+0000","last_fullsized":"2023-08-09T13:34:08.373090+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:34:02.064154+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:34:02.064154+0000","last_clean_scrub_stamp":"2023-08-09T13:34:02.064154+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T17:37:51.284066+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-08-09T13:34:08.211685+0000","last_change":"2023-08-09T13:34:02.109847+0000","last_active":"2023-08-09T13:34:08.211685+0000","last_peered":"2023-08-09T13:34:08.211685+0000","last_clean":"2023-08-09T13:34:08.211685+0000","last_became_active":"2023-08-09T13:34:02.109516+0000","last_became_peered":"2023-08-09T13:34:02.109516+0000","last_unstale":"2023-08-09T13:34:08.211685+0000","last_undegraded":"2023-08-09T13:34:08.211685+0000","last_fullsized":"2023-08-09T13:34:08.211685+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:34:01.052721+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:34:01.052721+0000","last_clean_scrub_stamp":"2023-08-09T13:34:01.052721+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T22:40:21.488794+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.c","version":"0'0","reported_seq":13,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-08-09T13:34:08.371235+0000","last_change":"2023-08-09T13:34:07.231138+0000","last_active":"2023-08-09T13:34:08.371235+0000","last_peered":"2023-08-09T13:34:08.371235+0000","last_clean":"2023-08-09T13:34:08.371235+0000","last_became_active":"2023-08-09T13:34:07.230741+0000","last_became_peered":"2023-08-09T13:34:07.230741+0000","last_unstale":"2023-08-09T13:34:08.371235+0000","last_undegraded":"2023-08-09T13:34:08.371235+0000","last_fullsized":"2023-08-09T13:34:08.371235+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:34:06.173547+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:34:06.173547+0000","last_clean_scrub_stamp":"2023-08-09T13:34:06.173547+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T21:11:44.603723+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4,0,6],"acting":[1,4,0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.c","version":"0'0","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-08-09T13:34:08.372932+0000","last_change":"2023-08-09T13:34:03.159095+0000","last_active":"2023-08-09T13:34:08.372932+0000","last_peered":"2023-08-09T13:34:08.372932+0000","last_clean":"2023-08-09T13:34:08.372932+0000","last_became_active":"2023-08-09T13:34:03.158807+0000","last_became_peered":"2023-08-09T13:34:03.158807+0000","last_unstale":"2023-08-09T13:34:08.372932+0000","last_undegraded":"2023-08-09T13:34:08.372932+0000","last_fullsized":"2023-08-09T13:34:08.372932+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:34:02.064154+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:34:02.064154+0000","last_clean_scrub_stamp":"2023-08-09T13:34:02.064154+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T21:06:37.115870+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.b","version":"19'2","reported_seq":23,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-08-09T13:34:08.369938+0000","last_change":"2023-08-09T13:34:02.103259+0000","last_active":"2023-08-09T13:34:08.369938+0000","last_peered":"2023-08-09T13:34:08.369938+0000","last_clean":"2023-08-09T13:34:08.369938+0000","last_became_active":"2023-08-09T13:34:02.102644+0000","last_became_peered":"2023-08-09T13:34:02.102644+0000","last_unstale":"2023-08-09T13:34:08.369938+0000","last_undegraded":"2023-08-09T13:34:08.369938+0000","last_fullsized":"2023-08-09T13:34:08.369938+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:34:01.052721+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:34:01.052721+0000","last_clean_scrub_stamp":"2023-08-09T13:34:01.052721+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T19:48:11.160512+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.d","version":"0'0","reported_seq":13,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-08-09T13:34:08.211821+0000","last_change":"2023-08-09T13:34:07.242458+0000","last_active":"2023-08-09T13:34:08.211821+0000","last_peered":"2023-08-09T13:34:08.211821+0000","last_clean":"2023-08-09T13:34:08.211821+0000","last_became_active":"2023-08-09T13:34:07.241393+0000","last_became_peered":"2023-08-09T13:34:07.241393+0000","last_unstale":"2023-08-09T13:34:08.211821+0000","last_undegraded":"2023-08-09T13:34:08.211821+0000","last_fullsized":"2023-08-09T13:34:08.211821+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:34:06.173547+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:34:06.173547+0000","last_clean_scrub_stamp":"2023-08-09T13:34:06.173547+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T19:00:12.926085+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7,5,6],"acting":[2,7,5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.b","version":"0'0","reported_seq":17,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-08-09T13:34:08.209500+0000","last_change":"2023-08-09T13:34:03.155061+0000","last_active":"2023-08-09T13:34:08.209500+0000","last_peered":"2023-08-09T13:34:08.209500+0000","last_clean":"2023-08-09T13:34:08.209500+0000","last_became_active":"2023-08-09T13:34:03.154792+0000","last_became_peered":"2023-08-09T13:34:03.154792+0000","last_unstale":"2023-08-09T13:34:08.209500+0000","last_undegraded":"2023-08-09T13:34:08.209500+0000","last_fullsized":"2023-08-09T13:34:08.209500+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:34:02.064154+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:34:02.064154+0000","last_clean_scrub_stamp":"2023-08-09T13:34:02.064154+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T18:50:05.285807+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-08-09T13:34:08.369085+0000","last_change":"2023-08-09T13:34:02.111888+0000","last_active":"2023-08-09T13:34:08.369085+0000","last_peered":"2023-08-09T13:34:08.369085+0000","last_clean":"2023-08-09T13:34:08.369085+0000","last_became_active":"2023-08-09T13:34:02.111588+0000","last_became_peered":"2023-08-09T13:34:02.111588+0000","last_unstale":"2023-08-09T13:34:08.369085+0000","last_undegraded":"2023-08-09T13:34:08.369085+0000","last_fullsized":"2023-08-09T13:34:08.369085+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:34:01.052721+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:34:01.052721+0000","last_clean_scrub_stamp":"2023-08-09T13:34:01.052721+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T15:53:24.295815+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.a","version":"0'0","reported_seq":13,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-08-09T13:34:08.211349+0000","last_change":"2023-08-09T13:34:07.241400+0000","last_active":"2023-08-09T13:34:08.211349+0000","last_peered":"2023-08-09T13:34:08.211349+0000","last_clean":"2023-08-09T13:34:08.211349+0000","last_became_active":"2023-08-09T13:34:07.240945+0000","last_became_peered":"2023-08-09T13:34:07.240945+0000","last_unstale":"2023-08-09T13:34:08.211349+0000","last_undegraded":"2023-08-09T13:34:08.211349+0000","last_fullsized":"2023-08-09T13:34:08.211349+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:34:06.173547+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:34:06.173547+0000","last_clean_scrub_stamp":"2023-08-09T13:34:06.173547+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T20:24:07.327648+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4,3,6],"acting":[2,4,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":17,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-08-09T13:34:08.211229+0000","last_change":"2023-08-09T13:34:03.159173+0000","last_active":"2023-08-09T13:34:08.211229+0000","last_peered":"2023-08-09T13:34:08.211229+0000","last_clean":"2023-08-09T13:34:08.211229+0000","last_became_active":"2023-08-09T13:34:03.157726+0000","last_became_peered":"2023-08-09T13:34:03.157726+0000","last_unstale":"2023-08-09T13:34:08.211229+0000","last_undegraded":"2023-08-09T13:34:08.211229+0000","last_fullsized":"2023-08-09T13:34:08.211229+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:34:02.064154+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:34:02.064154+0000","last_clean_scrub_stamp":"2023-08-09T13:34:02.064154+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T18:49:41.829768+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.d","version":"19'1","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-08-09T13:34:08.371386+0000","last_change":"2023-08-09T13:34:02.115845+0000","last_active":"2023-08-09T13:34:08.371386+0000","last_peered":"2023-08-09T13:34:08.371386+0000","last_clean":"2023-08-09T13:34:08.371386+0000","last_became_active":"2023-08-09T13:34:02.115519+0000","last_became_peered":"2023-08-09T13:34:02.115519+0000","last_unstale":"2023-08-09T13:34:08.371386+0000","last_undegraded":"2023-08-09T13:34:08.371386+0000","last_fullsized":"2023-08-09T13:34:08.371386+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:34:01.052721+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:34:01.052721+0000","last_clean_scrub_stamp":"2023-08-09T13:34:01.052721+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T22:00:42.967697+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.b","version":"0'0","reported_seq":13,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-08-09T13:34:08.211258+0000","last_change":"2023-08-09T13:34:07.245045+0000","last_active":"2023-08-09T13:34:08.211258+0000","last_peered":"2023-08-09T13:34:08.211258+0000","last_clean":"2023-08-09T13:34:08.211258+0000","last_became_active":"2023-08-09T13:34:07.242923+0000","last_became_peered":"2023-08-09T13:34:07.242923+0000","last_unstale":"2023-08-09T13:34:08.211258+0000","last_undegraded":"2023-08-09T13:34:08.211258+0000","last_fullsized":"2023-08-09T13:34:08.211258+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:34:06.173547+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:34:06.173547+0000","last_clean_scrub_stamp":"2023-08-09T13:34:06.173547+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T20:20:56.644233+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,5,6],"acting":[2,0,5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-08-09T13:34:08.369085+0000","last_change":"2023-08-09T13:34:03.157124+0000","last_active":"2023-08-09T13:34:08.369085+0000","last_peered":"2023-08-09T13:34:08.369085+0000","last_clean":"2023-08-09T13:34:08.369085+0000","last_became_active":"2023-08-09T13:34:03.156039+0000","last_became_peered":"2023-08-09T13:34:03.156039+0000","last_unstale":"2023-08-09T13:34:08.369085+0000","last_undegraded":"2023-08-09T13:34:08.369085+0000","last_fullsized":"2023-08-09T13:34:08.369085+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:34:02.064154+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:34:02.064154+0000","last_clean_scrub_stamp":"2023-08-09T13:34:02.064154+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T17:18:45.337355+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.e","version":"19'2","reported_seq":21,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-08-09T13:34:08.371732+0000","last_change":"2023-08-09T13:34:02.116789+0000","last_active":"2023-08-09T13:34:08.371732+0000","last_peered":"2023-08-09T13:34:08.371732+0000","last_clean":"2023-08-09T13:34:08.371732+0000","last_became_active":"2023-08-09T13:34:02.116543+0000","last_became_peered":"2023-08-09T13:34:02.116543+0000","last_unstale":"2023-08-09T13:34:08.371732+0000","last_undegraded":"2023-08-09T13:34:08.371732+0000","last_fullsized":"2023-08-09T13:34:08.371732+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:34:01.052721+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:34:01.052721+0000","last_clean_scrub_stamp":"2023-08-09T13:34:01.052721+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T21:52:55.252712+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.8","version":"0'0","reported_seq":13,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-08-09T13:34:08.211204+0000","last_change":"2023-08-09T13:34:07.241851+0000","last_active":"2023-08-09T13:34:08.211204+0000","last_peered":"2023-08-09T13:34:08.211204+0000","last_clean":"2023-08-09T13:34:08.211204+0000","last_became_active":"2023-08-09T13:34:07.241329+0000","last_became_peered":"2023-08-09T13:34:07.241329+0000","last_unstale":"2023-08-09T13:34:08.211204+0000","last_undegraded":"2023-08-09T13:34:08.211204+0000","last_fullsized":"2023-08-09T13:34:08.211204+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:34:06.173547+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:34:06.173547+0000","last_clean_scrub_stamp":"2023-08-09T13:34:06.173547+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T20:54:41.878672+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,1,5],"acting":[2,0,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":17,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-08-09T13:34:08.368062+0000","last_change":"2023-08-09T13:34:03.158365+0000","last_active":"2023-08-09T13:34:08.368062+0000","last_peered":"2023-08-09T13:34:08.368062+0000","last_clean":"2023-08-09T13:34:08.368062+0000","last_became_active":"2023-08-09T13:34:03.158042+0000","last_became_peered":"2023-08-09T13:34:03.158042+0000","last_unstale":"2023-08-09T13:34:08.368062+0000","last_undegraded":"2023-08-09T13:34:08.368062+0000","last_fullsized":"2023-08-09T13:34:08.368062+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:34:02.064154+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:34:02.064154+0000","last_clean_scrub_stamp":"2023-08-09T13:34:02.064154+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T22:36:24.124090+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.f","version":"19'1","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-08-09T13:34:08.373105+0000","last_change":"2023-08-09T13:34:02.116130+0000","last_active":"2023-08-09T13:34:08.373105+0000","last_peered":"2023-08-09T13:34:08.373105+0000","last_clean":"2023-08-09T13:34:08.373105+0000","last_became_active":"2023-08-09T13:34:02.115758+0000","last_became_peered":"2023-08-09T13:34:02.115758+0000","last_unstale":"2023-08-09T13:34:08.373105+0000","last_undegraded":"2023-08-09T13:34:08.373105+0000","last_fullsized":"2023-08-09T13:34:08.373105+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:34:01.052721+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:34:01.052721+0000","last_clean_scrub_stamp":"2023-08-09T13:34:01.052721+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-11T00:08:17.437427+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.9","version":"0'0","reported_seq":13,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-08-09T13:34:08.369072+0000","last_change":"2023-08-09T13:34:07.241507+0000","last_active":"2023-08-09T13:34:08.369072+0000","last_peered":"2023-08-09T13:34:08.369072+0000","last_clean":"2023-08-09T13:34:08.369072+0000","last_became_active":"2023-08-09T13:34:07.240810+0000","last_became_peered":"2023-08-09T13:34:07.240810+0000","last_unstale":"2023-08-09T13:34:08.369072+0000","last_undegraded":"2023-08-09T13:34:08.369072+0000","last_fullsized":"2023-08-09T13:34:08.369072+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:34:06.173547+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:34:06.173547+0000","last_clean_scrub_stamp":"2023-08-09T13:34:06.173547+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T17:46:27.148600+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,3,4],"acting":[7,6,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.16","version":"0'0","reported_seq":13,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-08-09T13:34:08.370805+0000","last_change":"2023-08-09T13:34:07.238069+0000","last_active":"2023-08-09T13:34:08.370805+0000","last_peered":"2023-08-09T13:34:08.370805+0000","last_clean":"2023-08-09T13:34:08.370805+0000","last_became_active":"2023-08-09T13:34:07.237560+0000","last_became_peered":"2023-08-09T13:34:07.237560+0000","last_unstale":"2023-08-09T13:34:08.370805+0000","last_undegraded":"2023-08-09T13:34:08.370805+0000","last_fullsized":"2023-08-09T13:34:08.370805+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:34:06.173547+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:34:06.173547+0000","last_clean_scrub_stamp":"2023-08-09T13:34:06.173547+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T17:46:27.148600+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,1,6],"acting":[5,3,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.10","version":"19'8","reported_seq":27,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-08-09T13:34:08.211427+0000","last_change":"2023-08-09T13:34:02.103042+0000","last_active":"2023-08-09T13:34:08.211427+0000","last_peered":"2023-08-09T13:34:08.211427+0000","last_clean":"2023-08-09T13:34:08.211427+0000","last_became_active":"2023-08-09T13:34:02.102730+0000","last_became_peered":"2023-08-09T13:34:02.102730+0000","last_unstale":"2023-08-09T13:34:08.211427+0000","last_undegraded":"2023-08-09T13:34:08.211427+0000","last_fullsized":"2023-08-09T13:34:08.211427+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:34:01.052721+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:34:01.052721+0000","last_clean_scrub_stamp":"2023-08-09T13:34:01.052721+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T19:01:23.853811+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-08-09T13:34:08.369924+0000","last_change":"2023-08-09T13:34:03.163917+0000","last_active":"2023-08-09T13:34:08.369924+0000","last_peered":"2023-08-09T13:34:08.369924+0000","last_clean":"2023-08-09T13:34:08.369924+0000","last_became_active":"2023-08-09T13:34:03.163598+0000","last_became_peered":"2023-08-09T13:34:03.163598+0000","last_unstale":"2023-08-09T13:34:08.369924+0000","last_undegraded":"2023-08-09T13:34:08.369924+0000","last_fullsized":"2023-08-09T13:34:08.369924+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:34:02.064154+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:34:02.064154+0000","last_clean_scrub_stamp":"2023-08-09T13:34:02.064154+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T20:01:36.200073+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.17","version":"0'0","reported_seq":13,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-08-09T13:34:08.369281+0000","last_change":"2023-08-09T13:34:07.249196+0000","last_active":"2023-08-09T13:34:08.369281+0000","last_peered":"2023-08-09T13:34:08.369281+0000","last_clean":"2023-08-09T13:34:08.369281+0000","last_became_active":"2023-08-09T13:34:07.248758+0000","last_became_peered":"2023-08-09T13:34:07.248758+0000","last_unstale":"2023-08-09T13:34:08.369281+0000","last_undegraded":"2023-08-09T13:34:08.369281+0000","last_fullsized":"2023-08-09T13:34:08.369281+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:34:06.173547+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:34:06.173547+0000","last_clean_scrub_stamp":"2023-08-09T13:34:06.173547+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T19:50:51.608237+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,7,0],"acting":[3,1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-08-09T13:34:08.369394+0000","last_change":"2023-08-09T13:34:02.112054+0000","last_active":"2023-08-09T13:34:08.369394+0000","last_peered":"2023-08-09T13:34:08.369394+0000","last_clean":"2023-08-09T13:34:08.369394+0000","last_became_active":"2023-08-09T13:34:02.111753+0000","last_became_peered":"2023-08-09T13:34:02.111753+0000","last_unstale":"2023-08-09T13:34:08.369394+0000","last_undegraded":"2023-08-09T13:34:08.369394+0000","last_fullsized":"2023-08-09T13:34:08.369394+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:34:01.052721+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:34:01.052721+0000","last_clean_scrub_stamp":"2023-08-09T13:34:01.052721+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T17:08:06.492548+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":17,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-08-09T13:34:08.208919+0000","last_change":"2023-08-09T13:34:03.151791+0000","last_active":"2023-08-09T13:34:08.208919+0000","last_peered":"2023-08-09T13:34:08.208919+0000","last_clean":"2023-08-09T13:34:08.208919+0000","last_became_active":"2023-08-09T13:34:03.151265+0000","last_became_peered":"2023-08-09T13:34:03.151265+0000","last_unstale":"2023-08-09T13:34:08.208919+0000","last_undegraded":"2023-08-09T13:34:08.208919+0000","last_fullsized":"2023-08-09T13:34:08.208919+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:34:02.064154+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:34:02.064154+0000","last_clean_scrub_stamp":"2023-08-09T13:34:02.064154+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T19:22:08.597283+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-08-09T13:34:08.207862+0000","last_change":"2023-08-09T13:34:02.109334+0000","last_active":"2023-08-09T13:34:08.207862+0000","last_peered":"2023-08-09T13:34:08.207862+0000","last_clean":"2023-08-09T13:34:08.207862+0000","last_became_active":"2023-08-09T13:34:02.108390+0000","last_became_peered":"2023-08-09T13:34:02.108390+0000","last_unstale":"2023-08-09T13:34:08.207862+0000","last_undegraded":"2023-08-09T13:34:08.207862+0000","last_fullsized":"2023-08-09T13:34:08.207862+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:34:01.052721+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:34:01.052721+0000","last_clean_scrub_stamp":"2023-08-09T13:34:01.052721+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T22:36:23.112658+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.15","version":"0'0","reported_seq":17,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-08-09T13:34:08.370140+0000","last_change":"2023-08-09T13:34:03.150611+0000","last_active":"2023-08-09T13:34:08.370140+0000","last_peered":"2023-08-09T13:34:08.370140+0000","last_clean":"2023-08-09T13:34:08.370140+0000","last_became_active":"2023-08-09T13:34:03.150016+0000","last_became_peered":"2023-08-09T13:34:03.150016+0000","last_unstale":"2023-08-09T13:34:08.370140+0000","last_undegraded":"2023-08-09T13:34:08.370140+0000","last_fullsized":"2023-08-09T13:34:08.370140+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:34:02.064154+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:34:02.064154+0000","last_clean_scrub_stamp":"2023-08-09T13:34:02.064154+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T21:35:01.466554+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.14","version":"0'0","reported_seq":13,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-08-09T13:34:08.369980+0000","last_change":"2023-08-09T13:34:07.251464+0000","last_active":"2023-08-09T13:34:08.369980+0000","last_peered":"2023-08-09T13:34:08.369980+0000","last_clean":"2023-08-09T13:34:08.369980+0000","last_became_active":"2023-08-09T13:34:07.251035+0000","last_became_peered":"2023-08-09T13:34:07.251035+0000","last_unstale":"2023-08-09T13:34:08.369980+0000","last_undegraded":"2023-08-09T13:34:08.369980+0000","last_fullsized":"2023-08-09T13:34:08.369980+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:34:06.173547+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:34:06.173547+0000","last_clean_scrub_stamp":"2023-08-09T13:34:06.173547+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T15:06:44.996627+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,2,1],"acting":[3,7,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.13","version":"19'2","reported_seq":21,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-08-09T13:34:08.370349+0000","last_change":"2023-08-09T13:34:02.115081+0000","last_active":"2023-08-09T13:34:08.370349+0000","last_peered":"2023-08-09T13:34:08.370349+0000","last_clean":"2023-08-09T13:34:08.370349+0000","last_became_active":"2023-08-09T13:34:02.114825+0000","last_became_peered":"2023-08-09T13:34:02.114825+0000","last_unstale":"2023-08-09T13:34:08.370349+0000","last_undegraded":"2023-08-09T13:34:08.370349+0000","last_fullsized":"2023-08-09T13:34:08.370349+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:34:01.052721+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:34:01.052721+0000","last_clean_scrub_stamp":"2023-08-09T13:34:01.052721+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T22:51:10.190023+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.14","version":"0'0","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-08-09T13:34:08.370421+0000","last_change":"2023-08-09T13:34:03.154171+0000","last_active":"2023-08-09T13:34:08.370421+0000","last_peered":"2023-08-09T13:34:08.370421+0000","last_clean":"2023-08-09T13:34:08.370421+0000","last_became_active":"2023-08-09T13:34:03.150789+0000","last_became_peered":"2023-08-09T13:34:03.150789+0000","last_unstale":"2023-08-09T13:34:08.370421+0000","last_undegraded":"2023-08-09T13:34:08.370421+0000","last_fullsized":"2023-08-09T13:34:08.370421+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:34:02.064154+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:34:02.064154+0000","last_clean_scrub_stamp":"2023-08-09T13:34:02.064154+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T17:18:45.337355+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.15","version":"0'0","reported_seq":13,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-08-09T13:34:08.369475+0000","last_change":"2023-08-09T13:34:07.241401+0000","last_active":"2023-08-09T13:34:08.369475+0000","last_peered":"2023-08-09T13:34:08.369475+0000","last_clean":"2023-08-09T13:34:08.369475+0000","last_became_active":"2023-08-09T13:34:07.240937+0000","last_became_peered":"2023-08-09T13:34:07.240937+0000","last_unstale":"2023-08-09T13:34:08.369475+0000","last_undegraded":"2023-08-09T13:34:08.369475+0000","last_fullsized":"2023-08-09T13:34:08.369475+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:34:06.173547+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:34:06.173547+0000","last_clean_scrub_stamp":"2023-08-09T13:34:06.173547+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T18:50:25.863045+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,3,0],"acting":[5,1,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.20","version":"0'0","reported_seq":17,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-08-09T13:34:08.210886+0000","last_change":"2023-08-09T13:34:03.157698+0000","last_active":"2023-08-09T13:34:08.210886+0000","last_peered":"2023-08-09T13:34:08.210886+0000","last_clean":"2023-08-09T13:34:08.210886+0000","last_became_active":"2023-08-09T13:34:03.153351+0000","last_became_peered":"2023-08-09T13:34:03.153351+0000","last_unstale":"2023-08-09T13:34:08.210886+0000","last_undegraded":"2023-08-09T13:34:08.210886+0000","last_fullsized":"2023-08-09T13:34:08.210886+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:34:02.064154+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:34:02.064154+0000","last_clean_scrub_stamp":"2023-08-09T13:34:02.064154+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T19:40:06.763332+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.21","version":"0'0","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-08-09T13:34:08.368201+0000","last_change":"2023-08-09T13:34:03.150065+0000","last_active":"2023-08-09T13:34:08.368201+0000","last_peered":"2023-08-09T13:34:08.368201+0000","last_clean":"2023-08-09T13:34:08.368201+0000","last_became_active":"2023-08-09T13:34:03.149650+0000","last_became_peered":"2023-08-09T13:34:03.149650+0000","last_unstale":"2023-08-09T13:34:08.368201+0000","last_undegraded":"2023-08-09T13:34:08.368201+0000","last_fullsized":"2023-08-09T13:34:08.368201+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:34:02.064154+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:34:02.064154+0000","last_clean_scrub_stamp":"2023-08-09T13:34:02.064154+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T13:47:48.941725+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.22","version":"0'0","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-08-09T13:34:08.372000+0000","last_change":"2023-08-09T13:34:03.152515+0000","last_active":"2023-08-09T13:34:08.372000+0000","last_peered":"2023-08-09T13:34:08.372000+0000","last_clean":"2023-08-09T13:34:08.372000+0000","last_became_active":"2023-08-09T13:34:03.152123+0000","last_became_peered":"2023-08-09T13:34:03.152123+0000","last_unstale":"2023-08-09T13:34:08.372000+0000","last_undegraded":"2023-08-09T13:34:08.372000+0000","last_fullsized":"2023-08-09T13:34:08.372000+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:34:02.064154+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:34:02.064154+0000","last_clean_scrub_stamp":"2023-08-09T13:34:02.064154+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T15:53:52.598510+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.23","version":"0'0","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-08-09T13:34:08.369171+0000","last_change":"2023-08-09T13:34:03.159919+0000","last_active":"2023-08-09T13:34:08.369171+0000","last_peered":"2023-08-09T13:34:08.369171+0000","last_clean":"2023-08-09T13:34:08.369171+0000","last_became_active":"2023-08-09T13:34:03.154344+0000","last_became_peered":"2023-08-09T13:34:03.154344+0000","last_unstale":"2023-08-09T13:34:08.369171+0000","last_undegraded":"2023-08-09T13:34:08.369171+0000","last_fullsized":"2023-08-09T13:34:08.369171+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:34:02.064154+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:34:02.064154+0000","last_clean_scrub_stamp":"2023-08-09T13:34:02.064154+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T22:33:11.246239+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.24","version":"0'0","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-08-09T13:34:08.371698+0000","last_change":"2023-08-09T13:34:03.164662+0000","last_active":"2023-08-09T13:34:08.371698+0000","last_peered":"2023-08-09T13:34:08.371698+0000","last_clean":"2023-08-09T13:34:08.371698+0000","last_became_active":"2023-08-09T13:34:03.163711+0000","last_became_peered":"2023-08-09T13:34:03.163711+0000","last_unstale":"2023-08-09T13:34:08.371698+0000","last_undegraded":"2023-08-09T13:34:08.371698+0000","last_fullsized":"2023-08-09T13:34:08.371698+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:34:02.064154+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:34:02.064154+0000","last_clean_scrub_stamp":"2023-08-09T13:34:02.064154+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T23:52:27.738385+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.25","version":"0'0","reported_seq":17,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-08-09T13:34:08.369863+0000","last_change":"2023-08-09T13:34:03.175502+0000","last_active":"2023-08-09T13:34:08.369863+0000","last_peered":"2023-08-09T13:34:08.369863+0000","last_clean":"2023-08-09T13:34:08.369863+0000","last_became_active":"2023-08-09T13:34:03.175192+0000","last_became_peered":"2023-08-09T13:34:03.175192+0000","last_unstale":"2023-08-09T13:34:08.369863+0000","last_undegraded":"2023-08-09T13:34:08.369863+0000","last_fullsized":"2023-08-09T13:34:08.369863+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:34:02.064154+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:34:02.064154+0000","last_clean_scrub_stamp":"2023-08-09T13:34:02.064154+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T19:22:08.597283+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.26","version":"0'0","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-08-09T13:34:08.372808+0000","last_change":"2023-08-09T13:34:03.155984+0000","last_active":"2023-08-09T13:34:08.372808+0000","last_peered":"2023-08-09T13:34:08.372808+0000","last_clean":"2023-08-09T13:34:08.372808+0000","last_became_active":"2023-08-09T13:34:03.155686+0000","last_became_peered":"2023-08-09T13:34:03.155686+0000","last_unstale":"2023-08-09T13:34:08.372808+0000","last_undegraded":"2023-08-09T13:34:08.372808+0000","last_fullsized":"2023-08-09T13:34:08.372808+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:34:02.064154+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:34:02.064154+0000","last_clean_scrub_stamp":"2023-08-09T13:34:02.064154+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T15:43:44.075578+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.27","version":"0'0","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-08-09T13:34:08.372180+0000","last_change":"2023-08-09T13:34:03.154003+0000","last_active":"2023-08-09T13:34:08.372180+0000","last_peered":"2023-08-09T13:34:08.372180+0000","last_clean":"2023-08-09T13:34:08.372180+0000","last_became_active":"2023-08-09T13:34:03.153707+0000","last_became_peered":"2023-08-09T13:34:03.153707+0000","last_unstale":"2023-08-09T13:34:08.372180+0000","last_undegraded":"2023-08-09T13:34:08.372180+0000","last_fullsized":"2023-08-09T13:34:08.372180+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:34:02.064154+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:34:02.064154+0000","last_clean_scrub_stamp":"2023-08-09T13:34:02.064154+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T17:08:07.503980+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.28","version":"0'0","reported_seq":17,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-08-09T13:34:08.368537+0000","last_change":"2023-08-09T13:34:03.170907+0000","last_active":"2023-08-09T13:34:08.368537+0000","last_peered":"2023-08-09T13:34:08.368537+0000","last_clean":"2023-08-09T13:34:08.368537+0000","last_became_active":"2023-08-09T13:34:03.162204+0000","last_became_peered":"2023-08-09T13:34:03.162204+0000","last_unstale":"2023-08-09T13:34:08.368537+0000","last_undegraded":"2023-08-09T13:34:08.368537+0000","last_fullsized":"2023-08-09T13:34:08.368537+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:34:02.064154+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:34:02.064154+0000","last_clean_scrub_stamp":"2023-08-09T13:34:02.064154+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T22:30:06.459673+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.29","version":"0'0","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-08-09T13:34:08.368673+0000","last_change":"2023-08-09T13:34:03.144064+0000","last_active":"2023-08-09T13:34:08.368673+0000","last_peered":"2023-08-09T13:34:08.368673+0000","last_clean":"2023-08-09T13:34:08.368673+0000","last_became_active":"2023-08-09T13:34:03.143612+0000","last_became_peered":"2023-08-09T13:34:03.143612+0000","last_unstale":"2023-08-09T13:34:08.368673+0000","last_undegraded":"2023-08-09T13:34:08.368673+0000","last_fullsized":"2023-08-09T13:34:08.368673+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:34:02.064154+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:34:02.064154+0000","last_clean_scrub_stamp":"2023-08-09T13:34:02.064154+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T18:50:26.789814+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.2a","version":"0'0","reported_seq":17,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-08-09T13:34:08.208605+0000","last_change":"2023-08-09T13:34:03.140127+0000","last_active":"2023-08-09T13:34:08.208605+0000","last_peered":"2023-08-09T13:34:08.208605+0000","last_clean":"2023-08-09T13:34:08.208605+0000","last_became_active":"2023-08-09T13:34:03.139616+0000","last_became_peered":"2023-08-09T13:34:03.139616+0000","last_unstale":"2023-08-09T13:34:08.208605+0000","last_undegraded":"2023-08-09T13:34:08.208605+0000","last_fullsized":"2023-08-09T13:34:08.208605+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:34:02.064154+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:34:02.064154+0000","last_clean_scrub_stamp":"2023-08-09T13:34:02.064154+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T18:50:21.753652+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.2b","version":"0'0","reported_seq":17,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-08-09T13:34:08.369974+0000","last_change":"2023-08-09T13:34:03.170969+0000","last_active":"2023-08-09T13:34:08.369974+0000","last_peered":"2023-08-09T13:34:08.369974+0000","last_clean":"2023-08-09T13:34:08.369974+0000","last_became_active":"2023-08-09T13:34:03.162334+0000","last_became_peered":"2023-08-09T13:34:03.162334+0000","last_unstale":"2023-08-09T13:34:08.369974+0000","last_undegraded":"2023-08-09T13:34:08.369974+0000","last_fullsized":"2023-08-09T13:34:08.369974+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:34:02.064154+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:34:02.064154+0000","last_clean_scrub_stamp":"2023-08-09T13:34:02.064154+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T23:34:54.506258+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.2c","version":"0'0","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-08-09T13:34:08.372790+0000","last_change":"2023-08-09T13:34:03.149273+0000","last_active":"2023-08-09T13:34:08.372790+0000","last_peered":"2023-08-09T13:34:08.372790+0000","last_clean":"2023-08-09T13:34:08.372790+0000","last_became_active":"2023-08-09T13:34:03.148460+0000","last_became_peered":"2023-08-09T13:34:03.148460+0000","last_unstale":"2023-08-09T13:34:08.372790+0000","last_undegraded":"2023-08-09T13:34:08.372790+0000","last_fullsized":"2023-08-09T13:34:08.372790+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:34:02.064154+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:34:02.064154+0000","last_clean_scrub_stamp":"2023-08-09T13:34:02.064154+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-11T01:12:24.927676+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.2d","version":"0'0","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-08-09T13:34:08.211794+0000","last_change":"2023-08-09T13:34:03.160727+0000","last_active":"2023-08-09T13:34:08.211794+0000","last_peered":"2023-08-09T13:34:08.211794+0000","last_clean":"2023-08-09T13:34:08.211794+0000","last_became_active":"2023-08-09T13:34:03.160406+0000","last_became_peered":"2023-08-09T13:34:03.160406+0000","last_unstale":"2023-08-09T13:34:08.211794+0000","last_undegraded":"2023-08-09T13:34:08.211794+0000","last_fullsized":"2023-08-09T13:34:08.211794+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:34:02.064154+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:34:02.064154+0000","last_clean_scrub_stamp":"2023-08-09T13:34:02.064154+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T20:28:43.534176+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.2e","version":"0'0","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-08-09T13:34:08.373221+0000","last_change":"2023-08-09T13:34:03.154714+0000","last_active":"2023-08-09T13:34:08.373221+0000","last_peered":"2023-08-09T13:34:08.373221+0000","last_clean":"2023-08-09T13:34:08.373221+0000","last_became_active":"2023-08-09T13:34:03.154384+0000","last_became_peered":"2023-08-09T13:34:03.154384+0000","last_unstale":"2023-08-09T13:34:08.373221+0000","last_undegraded":"2023-08-09T13:34:08.373221+0000","last_fullsized":"2023-08-09T13:34:08.373221+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:34:02.064154+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:34:02.064154+0000","last_clean_scrub_stamp":"2023-08-09T13:34:02.064154+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T16:21:31.389815+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.2f","version":"0'0","reported_seq":17,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-08-09T13:34:08.209885+0000","last_change":"2023-08-09T13:34:03.144822+0000","last_active":"2023-08-09T13:34:08.209885+0000","last_peered":"2023-08-09T13:34:08.209885+0000","last_clean":"2023-08-09T13:34:08.209885+0000","last_became_active":"2023-08-09T13:34:03.144470+0000","last_became_peered":"2023-08-09T13:34:03.144470+0000","last_unstale":"2023-08-09T13:34:08.209885+0000","last_undegraded":"2023-08-09T13:34:08.209885+0000","last_fullsized":"2023-08-09T13:34:08.209885+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:34:02.064154+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:34:02.064154+0000","last_clean_scrub_stamp":"2023-08-09T13:34:02.064154+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T21:18:19.551549+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.30","version":"0'0","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-08-09T13:34:08.370238+0000","last_change":"2023-08-09T13:34:03.140680+0000","last_active":"2023-08-09T13:34:08.370238+0000","last_peered":"2023-08-09T13:34:08.370238+0000","last_clean":"2023-08-09T13:34:08.370238+0000","last_became_active":"2023-08-09T13:34:03.139105+0000","last_became_peered":"2023-08-09T13:34:03.139105+0000","last_unstale":"2023-08-09T13:34:08.370238+0000","last_undegraded":"2023-08-09T13:34:08.370238+0000","last_fullsized":"2023-08-09T13:34:08.370238+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:34:02.064154+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:34:02.064154+0000","last_clean_scrub_stamp":"2023-08-09T13:34:02.064154+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T18:16:48.383340+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.31","version":"0'0","reported_seq":17,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-08-09T13:34:08.369342+0000","last_change":"2023-08-09T13:34:03.174569+0000","last_active":"2023-08-09T13:34:08.369342+0000","last_peered":"2023-08-09T13:34:08.369342+0000","last_clean":"2023-08-09T13:34:08.369342+0000","last_became_active":"2023-08-09T13:34:03.173450+0000","last_became_peered":"2023-08-09T13:34:03.173450+0000","last_unstale":"2023-08-09T13:34:08.369342+0000","last_undegraded":"2023-08-09T13:34:08.369342+0000","last_fullsized":"2023-08-09T13:34:08.369342+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:34:02.064154+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:34:02.064154+0000","last_clean_scrub_stamp":"2023-08-09T13:34:02.064154+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T13:44:50.765363+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.32","version":"0'0","reported_seq":17,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-08-09T13:34:08.368679+0000","last_change":"2023-08-09T13:34:03.156031+0000","last_active":"2023-08-09T13:34:08.368679+0000","last_peered":"2023-08-09T13:34:08.368679+0000","last_clean":"2023-08-09T13:34:08.368679+0000","last_became_active":"2023-08-09T13:34:03.155720+0000","last_became_peered":"2023-08-09T13:34:03.155720+0000","last_unstale":"2023-08-09T13:34:08.368679+0000","last_undegraded":"2023-08-09T13:34:08.368679+0000","last_fullsized":"2023-08-09T13:34:08.368679+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:34:02.064154+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:34:02.064154+0000","last_clean_scrub_stamp":"2023-08-09T13:34:02.064154+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T20:46:15.647419+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.33","version":"0'0","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-08-09T13:34:08.211563+0000","last_change":"2023-08-09T13:34:03.159410+0000","last_active":"2023-08-09T13:34:08.211563+0000","last_peered":"2023-08-09T13:34:08.211563+0000","last_clean":"2023-08-09T13:34:08.211563+0000","last_became_active":"2023-08-09T13:34:03.159006+0000","last_became_peered":"2023-08-09T13:34:03.159006+0000","last_unstale":"2023-08-09T13:34:08.211563+0000","last_undegraded":"2023-08-09T13:34:08.211563+0000","last_fullsized":"2023-08-09T13:34:08.211563+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:34:02.064154+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:34:02.064154+0000","last_clean_scrub_stamp":"2023-08-09T13:34:02.064154+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T21:55:49.311446+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.34","version":"0'0","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-08-09T13:34:08.212297+0000","last_change":"2023-08-09T13:34:03.165655+0000","last_active":"2023-08-09T13:34:08.212297+0000","last_peered":"2023-08-09T13:34:08.212297+0000","last_clean":"2023-08-09T13:34:08.212297+0000","last_became_active":"2023-08-09T13:34:03.165308+0000","last_became_peered":"2023-08-09T13:34:03.165308+0000","last_unstale":"2023-08-09T13:34:08.212297+0000","last_undegraded":"2023-08-09T13:34:08.212297+0000","last_fullsized":"2023-08-09T13:34:08.212297+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:34:02.064154+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:34:02.064154+0000","last_clean_scrub_stamp":"2023-08-09T13:34:02.064154+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T21:47:59.693735+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.35","version":"0'0","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-08-09T13:34:08.372839+0000","last_change":"2023-08-09T13:34:03.138842+0000","last_active":"2023-08-09T13:34:08.372839+0000","last_peered":"2023-08-09T13:34:08.372839+0000","last_clean":"2023-08-09T13:34:08.372839+0000","last_became_active":"2023-08-09T13:34:03.138554+0000","last_became_peered":"2023-08-09T13:34:03.138554+0000","last_unstale":"2023-08-09T13:34:08.372839+0000","last_undegraded":"2023-08-09T13:34:08.372839+0000","last_fullsized":"2023-08-09T13:34:08.372839+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:34:02.064154+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:34:02.064154+0000","last_clean_scrub_stamp":"2023-08-09T13:34:02.064154+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T23:05:06.664154+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.36","version":"0'0","reported_seq":17,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-08-09T13:34:08.371821+0000","last_change":"2023-08-09T13:34:03.148574+0000","last_active":"2023-08-09T13:34:08.371821+0000","last_peered":"2023-08-09T13:34:08.371821+0000","last_clean":"2023-08-09T13:34:08.371821+0000","last_became_active":"2023-08-09T13:34:03.148292+0000","last_became_peered":"2023-08-09T13:34:03.148292+0000","last_unstale":"2023-08-09T13:34:08.371821+0000","last_undegraded":"2023-08-09T13:34:08.371821+0000","last_fullsized":"2023-08-09T13:34:08.371821+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:34:02.064154+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:34:02.064154+0000","last_clean_scrub_stamp":"2023-08-09T13:34:02.064154+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T13:44:50.765363+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.37","version":"0'0","reported_seq":17,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-08-09T13:34:08.209550+0000","last_change":"2023-08-09T13:34:03.152770+0000","last_active":"2023-08-09T13:34:08.209550+0000","last_peered":"2023-08-09T13:34:08.209550+0000","last_clean":"2023-08-09T13:34:08.209550+0000","last_became_active":"2023-08-09T13:34:03.151703+0000","last_became_peered":"2023-08-09T13:34:03.151703+0000","last_unstale":"2023-08-09T13:34:08.209550+0000","last_undegraded":"2023-08-09T13:34:08.209550+0000","last_fullsized":"2023-08-09T13:34:08.209550+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:34:02.064154+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:34:02.064154+0000","last_clean_scrub_stamp":"2023-08-09T13:34:02.064154+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T17:46:23.039207+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.38","version":"0'0","reported_seq":17,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-08-09T13:34:08.367726+0000","last_change":"2023-08-09T13:34:03.146016+0000","last_active":"2023-08-09T13:34:08.367726+0000","last_peered":"2023-08-09T13:34:08.367726+0000","last_clean":"2023-08-09T13:34:08.367726+0000","last_became_active":"2023-08-09T13:34:03.145766+0000","last_became_peered":"2023-08-09T13:34:03.145766+0000","last_unstale":"2023-08-09T13:34:08.367726+0000","last_undegraded":"2023-08-09T13:34:08.367726+0000","last_fullsized":"2023-08-09T13:34:08.367726+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:34:02.064154+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:34:02.064154+0000","last_clean_scrub_stamp":"2023-08-09T13:34:02.064154+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T13:51:34.177397+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.39","version":"0'0","reported_seq":17,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-08-09T13:34:08.368967+0000","last_change":"2023-08-09T13:34:03.173368+0000","last_active":"2023-08-09T13:34:08.368967+0000","last_peered":"2023-08-09T13:34:08.368967+0000","last_clean":"2023-08-09T13:34:08.368967+0000","last_became_active":"2023-08-09T13:34:03.173037+0000","last_became_peered":"2023-08-09T13:34:03.173037+0000","last_unstale":"2023-08-09T13:34:08.368967+0000","last_undegraded":"2023-08-09T13:34:08.368967+0000","last_fullsized":"2023-08-09T13:34:08.368967+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:34:02.064154+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:34:02.064154+0000","last_clean_scrub_stamp":"2023-08-09T13:34:02.064154+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T21:11:40.494329+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.3a","version":"0'0","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-08-09T13:34:08.371086+0000","last_change":"2023-08-09T13:34:03.144394+0000","last_active":"2023-08-09T13:34:08.371086+0000","last_peered":"2023-08-09T13:34:08.371086+0000","last_clean":"2023-08-09T13:34:08.371086+0000","last_became_active":"2023-08-09T13:34:03.143958+0000","last_became_peered":"2023-08-09T13:34:03.143958+0000","last_unstale":"2023-08-09T13:34:08.371086+0000","last_undegraded":"2023-08-09T13:34:08.371086+0000","last_fullsized":"2023-08-09T13:34:08.371086+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:34:02.064154+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:34:02.064154+0000","last_clean_scrub_stamp":"2023-08-09T13:34:02.064154+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T19:49:18.157706+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.3b","version":"0'0","reported_seq":17,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-08-09T13:34:08.210114+0000","last_change":"2023-08-09T13:34:03.144695+0000","last_active":"2023-08-09T13:34:08.210114+0000","last_peered":"2023-08-09T13:34:08.210114+0000","last_clean":"2023-08-09T13:34:08.210114+0000","last_became_active":"2023-08-09T13:34:03.144303+0000","last_became_peered":"2023-08-09T13:34:03.144303+0000","last_unstale":"2023-08-09T13:34:08.210114+0000","last_undegraded":"2023-08-09T13:34:08.210114+0000","last_fullsized":"2023-08-09T13:34:08.210114+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:34:02.064154+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:34:02.064154+0000","last_clean_scrub_stamp":"2023-08-09T13:34:02.064154+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T18:07:31.705168+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.3c","version":"0'0","reported_seq":17,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-08-09T13:34:08.212037+0000","last_change":"2023-08-09T13:34:03.151741+0000","last_active":"2023-08-09T13:34:08.212037+0000","last_peered":"2023-08-09T13:34:08.212037+0000","last_clean":"2023-08-09T13:34:08.212037+0000","last_became_active":"2023-08-09T13:34:03.151331+0000","last_became_peered":"2023-08-09T13:34:03.151331+0000","last_unstale":"2023-08-09T13:34:08.212037+0000","last_undegraded":"2023-08-09T13:34:08.212037+0000","last_fullsized":"2023-08-09T13:34:08.212037+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:34:02.064154+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:34:02.064154+0000","last_clean_scrub_stamp":"2023-08-09T13:34:02.064154+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-11T00:09:38.635751+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.3d","version":"0'0","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-08-09T13:34:08.372764+0000","last_change":"2023-08-09T13:34:03.144010+0000","last_active":"2023-08-09T13:34:08.372764+0000","last_peered":"2023-08-09T13:34:08.372764+0000","last_clean":"2023-08-09T13:34:08.372764+0000","last_became_active":"2023-08-09T13:34:03.143673+0000","last_became_peered":"2023-08-09T13:34:03.143673+0000","last_unstale":"2023-08-09T13:34:08.372764+0000","last_undegraded":"2023-08-09T13:34:08.372764+0000","last_fullsized":"2023-08-09T13:34:08.372764+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:34:02.064154+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:34:02.064154+0000","last_clean_scrub_stamp":"2023-08-09T13:34:02.064154+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T16:58:18.499551+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.3e","version":"0'0","reported_seq":17,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-08-09T13:34:08.371258+0000","last_change":"2023-08-09T13:34:03.150894+0000","last_active":"2023-08-09T13:34:08.371258+0000","last_peered":"2023-08-09T13:34:08.371258+0000","last_clean":"2023-08-09T13:34:08.371258+0000","last_became_active":"2023-08-09T13:34:03.149237+0000","last_became_peered":"2023-08-09T13:34:03.149237+0000","last_unstale":"2023-08-09T13:34:08.371258+0000","last_undegraded":"2023-08-09T13:34:08.371258+0000","last_fullsized":"2023-08-09T13:34:08.371258+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:34:02.064154+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:34:02.064154+0000","last_clean_scrub_stamp":"2023-08-09T13:34:02.064154+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T21:18:19.551549+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.3f","version":"0'0","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-08-09T13:34:08.212027+0000","last_change":"2023-08-09T13:34:03.160785+0000","last_active":"2023-08-09T13:34:08.212027+0000","last_peered":"2023-08-09T13:34:08.212027+0000","last_clean":"2023-08-09T13:34:08.212027+0000","last_became_active":"2023-08-09T13:34:03.160452+0000","last_became_peered":"2023-08-09T13:34:03.160452+0000","last_unstale":"2023-08-09T13:34:08.212027+0000","last_undegraded":"2023-08-09T13:34:08.212027+0000","last_fullsized":"2023-08-09T13:34:08.212027+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:34:02.064154+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:34:02.064154+0000","last_clean_scrub_stamp":"2023-08-09T13:34:02.064154+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T18:03:19.093604+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]}],"pool_stats":[{"poolid":5,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":4,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":3,"num_pg":32,"stat_sum":{"num_bytes":4639,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":49,"num_write_kb":19,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":9278,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":43,"ondisk_log_size":43,"up":64,"acting":64,"num_store_stats":8},{"poolid":2,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":8}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574858,"num_pgs":41,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27872,"kb_used_data":856,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343968,"statfs":{"total":96636764160,"available":96608223232,"internally_reserved":0,"allocated":876544,"data_stored":699155,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1781,"internal_metadata":27654411},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.98299999999999998}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.71999999999999997}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.92900000000000005}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.327}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.391}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.9750000000000001}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.859}]}]},{"osd":6,"up_from":13,"seq":55834574858,"num_pgs":34,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27268,"kb_used_data":244,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344572,"statfs":{"total":96636764160,"available":96608841728,"internally_reserved":0,"allocated":249856,"data_stored":103369,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1781,"internal_metadata":27654411},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.3320000000000001}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.101}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.99299999999999999}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.2210000000000001}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.5900000000000001}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.2809999999999999}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.6379999999999999}]}]},{"osd":5,"up_from":13,"seq":55834574858,"num_pgs":39,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27280,"kb_used_data":256,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344560,"statfs":{"total":96636764160,"available":96608829440,"internally_reserved":0,"allocated":262144,"data_stored":105382,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1781,"internal_metadata":27654411},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.97599999999999998}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.323}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.90200000000000002}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.82799999999999996}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.488}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.014}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.369}]}]},{"osd":4,"up_from":13,"seq":55834574858,"num_pgs":50,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27284,"kb_used_data":268,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344556,"statfs":{"total":96636764160,"available":96608825344,"internally_reserved":0,"allocated":274432,"data_stored":108627,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1781,"internal_metadata":27654411},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.92400000000000004}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.3720000000000001}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.3149999999999999}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.284}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.5980000000000001}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.1479999999999999}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.514}]}]},{"osd":3,"up_from":13,"seq":55834574859,"num_pgs":48,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31372,"kb_used_data":252,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":94340468,"statfs":{"total":96636764160,"available":96604639232,"internally_reserved":0,"allocated":258048,"data_stored":104820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1781,"internal_metadata":31848715},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.753}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.80000000000000004}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.90900000000000003}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.361}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.3460000000000001}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.4490000000000001}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.669}]}]},{"osd":2,"up_from":13,"seq":55834574858,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31368,"kb_used_data":248,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":94340472,"statfs":{"total":96636764160,"available":96604643328,"internally_reserved":0,"allocated":253952,"data_stored":104730,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1781,"internal_metadata":31848715},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.871}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.2250000000000001}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.61599999999999999}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.4299999999999999}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.4079999999999999}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.2130000000000001}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.5489999999999999}]}]},{"osd":1,"up_from":13,"seq":55834574858,"num_pgs":49,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31952,"kb_used_data":828,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":94339888,"statfs":{"total":96636764160,"available":96604045312,"internally_reserved":0,"allocated":847872,"data_stored":695098,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1781,"internal_metadata":31848715},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.51000000000000001}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.006}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.82199999999999995}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.073}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.151}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.1799999999999999}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.3500000000000001}]}]},{"osd":0,"up_from":13,"seq":55834574858,"num_pgs":47,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31368,"kb_used_data":248,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":94340472,"statfs":{"total":96636764160,"available":96604643328,"internally_reserved":0,"allocated":253952,"data_stored":103951,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1781,"internal_metadata":31848715},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.69399999999999995}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.59099999999999997}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.55200000000000005}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.93999999999999995}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.89200000000000002}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.2190000000000001}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.1990000000000001}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":3897,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":652,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":4057,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-08-09T13:34:39.147 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-08-09T13:34:39.410 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:39.407+0000 7f997d57f640 1 Processor -- start 2023-08-09T13:34:39.410 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:39.407+0000 7f997d57f640 1 -- start start 2023-08-09T13:34:39.411 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:39.409+0000 7f997d57f640 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7f997815a6a0 0x7f99780713b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:34:39.411 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:39.409+0000 7f997d57f640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f99780cbd90 0x7f99780c8fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:34:39.412 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:39.409+0000 7f997d57f640 1 -- --> v1:172.21.15.43:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f99780cdc70 con 0x7f99780c94f0 2023-08-09T13:34:39.412 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:39.409+0000 7f997d57f640 1 -- --> v2:172.21.15.203:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f99780c9960 con 0x7f997815a6a0 2023-08-09T13:34:39.412 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:39.409+0000 7f997d57f640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9978070910 con 0x7f99780cbd90 2023-08-09T13:34:39.413 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:39.409+0000 7f997cd7e640 1 --1- >> v1:172.21.15.43:6789/0 conn(0x7f99780c94f0 0x7f997815ad20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.43:6789/0 says I am v1:172.21.15.43:35710/0 (socket says 172.21.15.43:35710) 2023-08-09T13:34:39.414 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:39.409+0000 7f997cd7e640 1 -- 172.21.15.43:0/2516331991 learned_addr learned my addr 172.21.15.43:0/2516331991 (peer_addr_for_me v1:172.21.15.43:0/0) 2023-08-09T13:34:39.414 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:39.410+0000 7f99777fe640 1 --2- 172.21.15.43:0/2516331991 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f99780cbd90 0x7f99780c8fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:34:39.414 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:39.410+0000 7f9977fff640 1 --2- 172.21.15.43:0/2516331991 >> v2:172.21.15.203:3300/0 conn(0x7f997815a6a0 0x7f99780713b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:34:39.414 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:39.410+0000 7f9976ffd640 1 -- 172.21.15.43:0/2516331991 <== mon.0 v1:172.21.15.43:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2441290693 0 0) 0x7f99780cdc70 con 0x7f99780c94f0 2023-08-09T13:34:39.414 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:39.410+0000 7f9976ffd640 1 -- 172.21.15.43:0/2516331991 --> v1:172.21.15.43:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f9960003520 con 0x7f99780c94f0 2023-08-09T13:34:39.414 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:39.410+0000 7f9976ffd640 1 -- 172.21.15.43:0/2516331991 <== mon.0 v1:172.21.15.43:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2464285868 0 0) 0x7f9960003520 con 0x7f99780c94f0 2023-08-09T13:34:39.415 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:39.410+0000 7f9976ffd640 1 -- 172.21.15.43:0/2516331991 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f99780cbd90 msgr2=0x7f99780c8fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-09T13:34:39.415 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:39.410+0000 7f9976ffd640 1 --2- 172.21.15.43:0/2516331991 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f99780cbd90 0x7f99780c8fb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:34:39.415 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:39.410+0000 7f9976ffd640 1 -- 172.21.15.43:0/2516331991 >> v2:172.21.15.203:3300/0 conn(0x7f997815a6a0 msgr2=0x7f99780713b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-09T13:34:39.415 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:39.410+0000 7f9976ffd640 1 --2- 172.21.15.43:0/2516331991 >> v2:172.21.15.203:3300/0 conn(0x7f997815a6a0 0x7f99780713b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:34:39.415 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:39.410+0000 7f9976ffd640 1 -- 172.21.15.43:0/2516331991 --> v1:172.21.15.43:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f99780c88c0 con 0x7f99780c94f0 2023-08-09T13:34:39.415 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:39.410+0000 7f9976ffd640 1 -- 172.21.15.43:0/2516331991 <== mon.0 v1:172.21.15.43:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1285150740 0 0) 0x7f99700019f0 con 0x7f99780c94f0 2023-08-09T13:34:39.415 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:39.410+0000 7f9976ffd640 1 -- 172.21.15.43:0/2516331991 >> v1:172.21.15.43:6789/0 conn(0x7f99780c94f0 legacy=0x7f997815ad20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:34:39.415 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:39.410+0000 7f9976ffd640 1 --2- 172.21.15.43:0/2516331991 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f9960003970 0x7f9960003d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:34:39.416 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:39.410+0000 7f9976ffd640 1 --2- 172.21.15.43:0/2516331991 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f99780cbd90 0x7f99600047c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:34:39.416 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:39.410+0000 7f9976ffd640 1 --2- 172.21.15.43:0/2516331991 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f9960004d00 0x7f9960005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:34:39.416 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:39.410+0000 7f9976ffd640 1 -- 172.21.15.43:0/2516331991 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f99780c9960 con 0x7f9960004d00 2023-08-09T13:34:39.416 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:39.410+0000 7f9976ffd640 1 -- 172.21.15.43:0/2516331991 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9978070910 con 0x7f9960003970 2023-08-09T13:34:39.416 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:39.410+0000 7f9976ffd640 1 -- 172.21.15.43:0/2516331991 --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f996000dbc0 con 0x7f99780cbd90 2023-08-09T13:34:39.416 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:39.411+0000 7f997cd7e640 1 --2- 172.21.15.43:0/2516331991 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f9960004d00 0x7f9960005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:34:39.416 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:39.411+0000 7f9977fff640 1 --2- 172.21.15.43:0/2516331991 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f9960003970 0x7f9960003d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:34:39.417 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:39.411+0000 7f99777fe640 1 --2- 172.21.15.43:0/2516331991 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f99780cbd90 0x7f99600047c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:34:39.417 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:39.411+0000 7f997cd7e640 1 -- 172.21.15.43:0/2516331991 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f99780cbd90 msgr2=0x7f99600047c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-09T13:34:39.417 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:39.411+0000 7f997cd7e640 1 --2- 172.21.15.43:0/2516331991 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f99780cbd90 0x7f99600047c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:34:39.417 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:39.411+0000 7f997cd7e640 1 -- 172.21.15.43:0/2516331991 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f9960003970 msgr2=0x7f9960003d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-09T13:34:39.417 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:39.411+0000 7f997cd7e640 1 --2- 172.21.15.43:0/2516331991 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f9960003970 0x7f9960003d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:34:39.417 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:39.411+0000 7f997cd7e640 1 -- 172.21.15.43:0/2516331991 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9960003520 con 0x7f9960004d00 2023-08-09T13:34:39.418 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:39.412+0000 7f997cd7e640 1 --2- 172.21.15.43:0/2516331991 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f9960004d00 0x7f9960005180 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7f997000a880 tx=0x7f99600057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d1ab18b1a91b5f5d server_cookie=0 in_seq=0 out_seq=0 2023-08-09T13:34:39.419 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:39.412+0000 7f9976ffd640 1 -- 172.21.15.43:0/2516331991 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9970034630 con 0x7f9960004d00 2023-08-09T13:34:39.419 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:39.412+0000 7f9976ffd640 1 -- 172.21.15.43:0/2516331991 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f997002f070 con 0x7f9960004d00 2023-08-09T13:34:39.419 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:39.412+0000 7f9976ffd640 1 -- 172.21.15.43:0/2516331991 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f99700349f0 con 0x7f9960004d00 2023-08-09T13:34:39.419 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:39.413+0000 7f997d57f640 1 -- 172.21.15.43:0/2516331991 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f9960004d00 msgr2=0x7f9960005180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:34:39.419 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:39.413+0000 7f997d57f640 1 --2- 172.21.15.43:0/2516331991 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f9960004d00 0x7f9960005180 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7f997000a880 tx=0x7f99600057f0 comp rx=0 tx=0).stop 2023-08-09T13:34:39.419 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:39.413+0000 7f997cd7e640 1 -- 172.21.15.43:0/2516331991 reap_dead start 2023-08-09T13:34:39.420 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:39.413+0000 7f997d57f640 1 -- 172.21.15.43:0/2516331991 shutdown_connections 2023-08-09T13:34:39.420 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:39.413+0000 7f997d57f640 1 -- 172.21.15.43:0/2516331991 >> 172.21.15.43:0/2516331991 conn(0x7f99780cde90 msgr2=0x7f99780ce2a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-09T13:34:39.420 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:39.413+0000 7f997d57f640 1 -- 172.21.15.43:0/2516331991 shutdown_connections 2023-08-09T13:34:39.420 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:39.413+0000 7f997d57f640 1 -- 172.21.15.43:0/2516331991 wait complete. 2023-08-09T13:34:39.420 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:39.415+0000 7f997d57f640 1 Processor -- start 2023-08-09T13:34:39.420 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:39.415+0000 7f997d57f640 1 -- start start 2023-08-09T13:34:39.420 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:39.416+0000 7f997d57f640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f9960004d00 0x7f997809a2e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:34:39.421 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:39.416+0000 7f997d57f640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f997809b390 0x7f997806b190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:34:39.421 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:39.416+0000 7f997d57f640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f997806b6d0 0x7f99780b2410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:34:39.421 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:39.416+0000 7f997d57f640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f997814d8e0 con 0x7f997809b390 2023-08-09T13:34:39.421 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:39.416+0000 7f997d57f640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9978099e50 con 0x7f997806b6d0 2023-08-09T13:34:39.421 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:39.416+0000 7f997d57f640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f997809b820 con 0x7f9960004d00 2023-08-09T13:34:39.422 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:39.416+0000 7f99777fe640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f997809b390 0x7f997806b190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:34:39.423 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:39.417+0000 7f99777fe640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f997809b390 0x7f997806b190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:51998/0 (socket says 172.21.15.43:51998) 2023-08-09T13:34:39.423 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:39.417+0000 7f99777fe640 1 -- 172.21.15.43:0/3215998306 learned_addr learned my addr 172.21.15.43:0/3215998306 (peer_addr_for_me v2:172.21.15.43:0/0) 2023-08-09T13:34:39.423 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:39.417+0000 7f9977fff640 1 --2- 172.21.15.43:0/3215998306 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f9960004d00 0x7f997809a2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:34:39.423 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:39.417+0000 7f997cd7e640 1 --2- 172.21.15.43:0/3215998306 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f997806b6d0 0x7f99780b2410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:34:39.423 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:39.417+0000 7f99777fe640 1 -- 172.21.15.43:0/3215998306 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f9960004d00 msgr2=0x7f997809a2e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:34:39.423 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:39.417+0000 7f99777fe640 1 --2- 172.21.15.43:0/3215998306 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f9960004d00 0x7f997809a2e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:34:39.423 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:39.417+0000 7f99777fe640 1 -- 172.21.15.43:0/3215998306 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f997806b6d0 msgr2=0x7f99780b2410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:34:39.423 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:39.417+0000 7f99777fe640 1 --2- 172.21.15.43:0/3215998306 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f997806b6d0 0x7f99780b2410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:34:39.424 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:39.417+0000 7f99777fe640 1 -- 172.21.15.43:0/3215998306 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9978098af0 con 0x7f997809b390 2023-08-09T13:34:39.424 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:39.417+0000 7f997cd7e640 1 --2- 172.21.15.43:0/3215998306 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f997806b6d0 0x7f99780b2410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-08-09T13:34:39.424 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:39.417+0000 7f99777fe640 1 --2- 172.21.15.43:0/3215998306 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f997809b390 0x7f997806b190 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7f996c011600 tx=0x7f996c011ac0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-09T13:34:39.424 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:39.417+0000 7f9974ff9640 1 -- 172.21.15.43:0/3215998306 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f996c00f920 con 0x7f997809b390 2023-08-09T13:34:39.424 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:39.418+0000 7f9974ff9640 1 -- 172.21.15.43:0/3215998306 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f996c00fa80 con 0x7f997809b390 2023-08-09T13:34:39.424 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:39.418+0000 7f9974ff9640 1 -- 172.21.15.43:0/3215998306 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f996c00fdf0 con 0x7f997809b390 2023-08-09T13:34:39.424 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:39.418+0000 7f997d57f640 1 -- 172.21.15.43:0/3215998306 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9978098dd0 con 0x7f997809b390 2023-08-09T13:34:39.425 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:39.418+0000 7f997d57f640 1 -- 172.21.15.43:0/3215998306 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f99780990b0 con 0x7f997809b390 2023-08-09T13:34:39.425 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:39.419+0000 7f997d57f640 1 -- 172.21.15.43:0/3215998306 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f997809a000 con 0x7f997809b390 2023-08-09T13:34:39.425 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:39.420+0000 7f9974ff9640 1 -- 172.21.15.43:0/3215998306 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 5) v1 ==== 88341+0+0 (secure 0 0 0) 0x7f996c015460 con 0x7f997809b390 2023-08-09T13:34:39.426 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:39.421+0000 7f9974ff9640 1 --2- 172.21.15.43:0/3215998306 >> [v2:172.21.15.203:6832/830097474,v1:172.21.15.203:6833/830097474] conn(0x7f996406a080 0x7f996406c540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:34:39.426 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:39.421+0000 7f9974ff9640 1 -- 172.21.15.43:0/3215998306 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6813+0+0 (secure 0 0 0) 0x7f996c088d60 con 0x7f997809b390 2023-08-09T13:34:39.426 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:39.421+0000 7f9977fff640 1 --2- 172.21.15.43:0/3215998306 >> [v2:172.21.15.203:6832/830097474,v1:172.21.15.203:6833/830097474] conn(0x7f996406a080 0x7f996406c540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:34:39.426 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:39.422+0000 7f9977fff640 1 --2- 172.21.15.43:0/3215998306 >> [v2:172.21.15.203:6832/830097474,v1:172.21.15.203:6833/830097474] conn(0x7f996406a080 0x7f996406c540 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7f9968009a90 tx=0x7f9968008040 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-09T13:34:39.427 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:39.425+0000 7f9974ff9640 1 -- 172.21.15.43:0/3215998306 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178066 (secure 0 0 0) 0x7f997809a000 con 0x7f997809b390 2023-08-09T13:34:39.611 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:39.608+0000 7f997d57f640 1 -- 172.21.15.43:0/3215998306 --> [v2:172.21.15.203:6832/830097474,v1:172.21.15.203:6833/830097474] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f9978070e90 con 0x7f996406a080 2023-08-09T13:34:39.639 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:39.636+0000 7f9974ff9640 1 -- 172.21.15.43:0/3215998306 <== mgr.4114 v2:172.21.15.203:6832/830097474 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+367422 (secure 0 0 0) 0x7f9968013a20 con 0x7f996406a080 2023-08-09T13:34:39.643 INFO:teuthology.orchestra.run.smithi043.stdout: 2023-08-09T13:34:39.644 INFO:teuthology.orchestra.run.smithi043.stderr:dumped all 2023-08-09T13:34:39.646 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:39.642+0000 7f997d57f640 1 -- 172.21.15.43:0/3215998306 >> [v2:172.21.15.203:6832/830097474,v1:172.21.15.203:6833/830097474] conn(0x7f996406a080 msgr2=0x7f996406c540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:34:39.646 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:39.642+0000 7f997d57f640 1 --2- 172.21.15.43:0/3215998306 >> [v2:172.21.15.203:6832/830097474,v1:172.21.15.203:6833/830097474] conn(0x7f996406a080 0x7f996406c540 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7f9968009a90 tx=0x7f9968008040 comp rx=0 tx=0).stop 2023-08-09T13:34:39.646 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:39.642+0000 7f997d57f640 1 -- 172.21.15.43:0/3215998306 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f997809b390 msgr2=0x7f997806b190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:34:39.646 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:39.642+0000 7f997d57f640 1 --2- 172.21.15.43:0/3215998306 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f997809b390 0x7f997806b190 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7f996c011600 tx=0x7f996c011ac0 comp rx=0 tx=0).stop 2023-08-09T13:34:39.647 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:39.642+0000 7f997d57f640 1 -- 172.21.15.43:0/3215998306 shutdown_connections 2023-08-09T13:34:39.647 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:39.642+0000 7f997d57f640 1 --2- 172.21.15.43:0/3215998306 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f997806b6d0 0x7f99780b2410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:34:39.647 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:39.642+0000 7f997d57f640 1 --2- 172.21.15.43:0/3215998306 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f997809b390 0x7f997806b190 unknown :-1 s=CLOSED pgs=195 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:34:39.647 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:39.642+0000 7f997d57f640 1 --2- 172.21.15.43:0/3215998306 >> [v2:172.21.15.203:6832/830097474,v1:172.21.15.203:6833/830097474] conn(0x7f996406a080 0x7f996406c540 unknown :-1 s=CLOSED pgs=123 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:34:39.647 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:39.642+0000 7f997d57f640 1 --2- 172.21.15.43:0/3215998306 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f9960004d00 0x7f997809a2e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:34:39.647 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:39.642+0000 7f997d57f640 1 -- 172.21.15.43:0/3215998306 >> 172.21.15.43:0/3215998306 conn(0x7f99780cde90 msgr2=0x7f99780af130 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-09T13:34:39.648 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:39.643+0000 7f997d57f640 1 -- 172.21.15.43:0/3215998306 shutdown_connections 2023-08-09T13:34:39.648 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:39.643+0000 7f997d57f640 1 -- 172.21.15.43:0/3215998306 wait complete. 2023-08-09T13:34:39.665 INFO:teuthology.orchestra.run.smithi043.stdout:{"pg_ready":true,"pg_map":{"version":40,"stamp":"2023-08-09T13:34:38.775015+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":595007,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":208,"num_write_kb":2435,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160,"ondisk_log_size":160,"up":338,"acting":338,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":338,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":235764,"kb_used_data":3200,"kb_used_omap":13,"kb_used_meta":232434,"kb_avail":754738956,"statfs":{"total":773094113280,"available":772852690944,"internally_reserved":0,"allocated":3276800,"data_stored":2025132,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":14248,"internal_metadata":238012504},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.005938"},"pg_stats":[{"pgid":"5.19","version":"0'0","reported_seq":13,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-08-09T13:34:08.367616+0000","last_change":"2023-08-09T13:34:07.232493+0000","last_active":"2023-08-09T13:34:08.367616+0000","last_peered":"2023-08-09T13:34:08.367616+0000","last_clean":"2023-08-09T13:34:08.367616+0000","last_became_active":"2023-08-09T13:34:07.231969+0000","last_became_peered":"2023-08-09T13:34:07.231969+0000","last_unstale":"2023-08-09T13:34:08.367616+0000","last_undegraded":"2023-08-09T13:34:08.367616+0000","last_fullsized":"2023-08-09T13:34:08.367616+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:34:06.173547+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:34:06.173547+0000","last_clean_scrub_stamp":"2023-08-09T13:34:06.173547+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T19:42:37.842719+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,7,2],"acting":[1,5,7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.1f","version":"19'4","reported_seq":23,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-08-09T13:34:08.209913+0000","last_change":"2023-08-09T13:34:02.108915+0000","last_active":"2023-08-09T13:34:08.209913+0000","last_peered":"2023-08-09T13:34:08.209913+0000","last_clean":"2023-08-09T13:34:08.209913+0000","last_became_active":"2023-08-09T13:34:02.108573+0000","last_became_peered":"2023-08-09T13:34:02.108573+0000","last_unstale":"2023-08-09T13:34:08.209913+0000","last_undegraded":"2023-08-09T13:34:08.209913+0000","last_fullsized":"2023-08-09T13:34:08.209913+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:34:01.052721+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:34:01.052721+0000","last_clean_scrub_stamp":"2023-08-09T13:34:01.052721+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T23:52:26.726953+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-08-09T13:34:08.370536+0000","last_change":"2023-08-09T13:34:03.156516+0000","last_active":"2023-08-09T13:34:08.370536+0000","last_peered":"2023-08-09T13:34:08.370536+0000","last_clean":"2023-08-09T13:34:08.370536+0000","last_became_active":"2023-08-09T13:34:03.156109+0000","last_became_peered":"2023-08-09T13:34:03.156109+0000","last_unstale":"2023-08-09T13:34:08.370536+0000","last_undegraded":"2023-08-09T13:34:08.370536+0000","last_fullsized":"2023-08-09T13:34:08.370536+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:34:02.064154+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:34:02.064154+0000","last_clean_scrub_stamp":"2023-08-09T13:34:02.064154+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T13:40:56.937864+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.18","version":"0'0","reported_seq":13,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-08-09T13:34:08.369809+0000","last_change":"2023-08-09T13:34:07.234694+0000","last_active":"2023-08-09T13:34:08.369809+0000","last_peered":"2023-08-09T13:34:08.369809+0000","last_clean":"2023-08-09T13:34:08.369809+0000","last_became_active":"2023-08-09T13:34:07.234241+0000","last_became_peered":"2023-08-09T13:34:07.234241+0000","last_unstale":"2023-08-09T13:34:08.369809+0000","last_undegraded":"2023-08-09T13:34:08.369809+0000","last_fullsized":"2023-08-09T13:34:08.369809+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:34:06.173547+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:34:06.173547+0000","last_clean_scrub_stamp":"2023-08-09T13:34:06.173547+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T15:06:44.662571+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,1,2],"acting":[4,6,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.1e","version":"19'1","reported_seq":22,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-08-09T13:34:08.370464+0000","last_change":"2023-08-09T13:34:02.105244+0000","last_active":"2023-08-09T13:34:08.370464+0000","last_peered":"2023-08-09T13:34:08.370464+0000","last_clean":"2023-08-09T13:34:08.370464+0000","last_became_active":"2023-08-09T13:34:02.104920+0000","last_became_peered":"2023-08-09T13:34:02.104920+0000","last_unstale":"2023-08-09T13:34:08.370464+0000","last_undegraded":"2023-08-09T13:34:08.370464+0000","last_fullsized":"2023-08-09T13:34:08.370464+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:34:01.052721+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:34:01.052721+0000","last_clean_scrub_stamp":"2023-08-09T13:34:01.052721+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T21:57:53.171937+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-08-09T13:34:08.368933+0000","last_change":"2023-08-09T13:34:03.156960+0000","last_active":"2023-08-09T13:34:08.368933+0000","last_peered":"2023-08-09T13:34:08.368933+0000","last_clean":"2023-08-09T13:34:08.368933+0000","last_became_active":"2023-08-09T13:34:03.156668+0000","last_became_peered":"2023-08-09T13:34:03.156668+0000","last_unstale":"2023-08-09T13:34:08.368933+0000","last_undegraded":"2023-08-09T13:34:08.368933+0000","last_fullsized":"2023-08-09T13:34:08.368933+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:34:02.064154+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:34:02.064154+0000","last_clean_scrub_stamp":"2023-08-09T13:34:02.064154+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T14:55:32.816063+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.1b","version":"0'0","reported_seq":13,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-08-09T13:34:08.370229+0000","last_change":"2023-08-09T13:34:07.232500+0000","last_active":"2023-08-09T13:34:08.370229+0000","last_peered":"2023-08-09T13:34:08.370229+0000","last_clean":"2023-08-09T13:34:08.370229+0000","last_became_active":"2023-08-09T13:34:07.232045+0000","last_became_peered":"2023-08-09T13:34:07.232045+0000","last_unstale":"2023-08-09T13:34:08.370229+0000","last_undegraded":"2023-08-09T13:34:08.370229+0000","last_fullsized":"2023-08-09T13:34:08.370229+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:34:06.173547+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:34:06.173547+0000","last_clean_scrub_stamp":"2023-08-09T13:34:06.173547+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T18:36:44.021636+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,7,4],"acting":[5,0,7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1d","version":"19'2","reported_seq":21,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-08-09T13:34:08.370340+0000","last_change":"2023-08-09T13:34:02.109750+0000","last_active":"2023-08-09T13:34:08.370340+0000","last_peered":"2023-08-09T13:34:08.370340+0000","last_clean":"2023-08-09T13:34:08.370340+0000","last_became_active":"2023-08-09T13:34:02.109445+0000","last_became_peered":"2023-08-09T13:34:02.109445+0000","last_unstale":"2023-08-09T13:34:08.370340+0000","last_undegraded":"2023-08-09T13:34:08.370340+0000","last_fullsized":"2023-08-09T13:34:08.370340+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:34:01.052721+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:34:01.052721+0000","last_clean_scrub_stamp":"2023-08-09T13:34:01.052721+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T22:30:05.448240+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1a","version":"0'0","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-08-09T13:34:08.371675+0000","last_change":"2023-08-09T13:34:03.145198+0000","last_active":"2023-08-09T13:34:08.371675+0000","last_peered":"2023-08-09T13:34:08.371675+0000","last_clean":"2023-08-09T13:34:08.371675+0000","last_became_active":"2023-08-09T13:34:03.144756+0000","last_became_peered":"2023-08-09T13:34:03.144756+0000","last_unstale":"2023-08-09T13:34:08.371675+0000","last_undegraded":"2023-08-09T13:34:08.371675+0000","last_fullsized":"2023-08-09T13:34:08.371675+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:34:02.064154+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:34:02.064154+0000","last_clean_scrub_stamp":"2023-08-09T13:34:02.064154+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T16:10:58.188522+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1a","version":"0'0","reported_seq":13,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-08-09T13:34:08.369069+0000","last_change":"2023-08-09T13:34:07.238058+0000","last_active":"2023-08-09T13:34:08.369069+0000","last_peered":"2023-08-09T13:34:08.369069+0000","last_clean":"2023-08-09T13:34:08.369069+0000","last_became_active":"2023-08-09T13:34:07.237575+0000","last_became_peered":"2023-08-09T13:34:07.237575+0000","last_unstale":"2023-08-09T13:34:08.369069+0000","last_undegraded":"2023-08-09T13:34:08.369069+0000","last_fullsized":"2023-08-09T13:34:08.369069+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:34:06.173547+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:34:06.173547+0000","last_clean_scrub_stamp":"2023-08-09T13:34:06.173547+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T21:18:23.660943+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,1,6],"acting":[7,4,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1c","version":"19'2","reported_seq":21,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-08-09T13:34:08.369633+0000","last_change":"2023-08-09T13:34:02.107853+0000","last_active":"2023-08-09T13:34:08.369633+0000","last_peered":"2023-08-09T13:34:08.369633+0000","last_clean":"2023-08-09T13:34:08.369633+0000","last_became_active":"2023-08-09T13:34:02.107571+0000","last_became_peered":"2023-08-09T13:34:02.107571+0000","last_unstale":"2023-08-09T13:34:08.369633+0000","last_undegraded":"2023-08-09T13:34:08.369633+0000","last_fullsized":"2023-08-09T13:34:08.369633+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:34:01.052721+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:34:01.052721+0000","last_clean_scrub_stamp":"2023-08-09T13:34:01.052721+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T19:42:32.721893+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1b","version":"0'0","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-08-09T13:34:08.371599+0000","last_change":"2023-08-09T13:34:03.148927+0000","last_active":"2023-08-09T13:34:08.371599+0000","last_peered":"2023-08-09T13:34:08.371599+0000","last_clean":"2023-08-09T13:34:08.371599+0000","last_became_active":"2023-08-09T13:34:03.148582+0000","last_became_peered":"2023-08-09T13:34:03.148582+0000","last_unstale":"2023-08-09T13:34:08.371599+0000","last_undegraded":"2023-08-09T13:34:08.371599+0000","last_fullsized":"2023-08-09T13:34:08.371599+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:34:02.064154+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:34:02.064154+0000","last_clean_scrub_stamp":"2023-08-09T13:34:02.064154+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T17:58:47.470353+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1d","version":"0'0","reported_seq":13,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-08-09T13:34:08.372087+0000","last_change":"2023-08-09T13:34:07.246175+0000","last_active":"2023-08-09T13:34:08.372087+0000","last_peered":"2023-08-09T13:34:08.372087+0000","last_clean":"2023-08-09T13:34:08.372087+0000","last_became_active":"2023-08-09T13:34:07.245757+0000","last_became_peered":"2023-08-09T13:34:07.245757+0000","last_unstale":"2023-08-09T13:34:08.372087+0000","last_undegraded":"2023-08-09T13:34:08.372087+0000","last_fullsized":"2023-08-09T13:34:08.372087+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:34:06.173547+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:34:06.173547+0000","last_clean_scrub_stamp":"2023-08-09T13:34:06.173547+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T18:07:35.814562+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,4,0],"acting":[1,3,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.1b","version":"19'1","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-08-09T13:34:08.210113+0000","last_change":"2023-08-09T13:34:02.109573+0000","last_active":"2023-08-09T13:34:08.210113+0000","last_peered":"2023-08-09T13:34:08.210113+0000","last_clean":"2023-08-09T13:34:08.210113+0000","last_became_active":"2023-08-09T13:34:02.109015+0000","last_became_peered":"2023-08-09T13:34:02.109015+0000","last_unstale":"2023-08-09T13:34:08.210113+0000","last_undegraded":"2023-08-09T13:34:08.210113+0000","last_fullsized":"2023-08-09T13:34:08.210113+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:34:01.052721+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:34:01.052721+0000","last_clean_scrub_stamp":"2023-08-09T13:34:01.052721+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T13:44:49.753930+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1c","version":"0'0","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-08-09T13:34:08.212417+0000","last_change":"2023-08-09T13:34:03.162651+0000","last_active":"2023-08-09T13:34:08.212417+0000","last_peered":"2023-08-09T13:34:08.212417+0000","last_clean":"2023-08-09T13:34:08.212417+0000","last_became_active":"2023-08-09T13:34:03.161575+0000","last_became_peered":"2023-08-09T13:34:03.161575+0000","last_unstale":"2023-08-09T13:34:08.212417+0000","last_undegraded":"2023-08-09T13:34:08.212417+0000","last_fullsized":"2023-08-09T13:34:08.212417+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:34:02.064154+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:34:02.064154+0000","last_clean_scrub_stamp":"2023-08-09T13:34:02.064154+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-11T00:30:20.150060+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.1c","version":"0'0","reported_seq":13,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-08-09T13:34:08.372392+0000","last_change":"2023-08-09T13:34:07.242334+0000","last_active":"2023-08-09T13:34:08.372392+0000","last_peered":"2023-08-09T13:34:08.372392+0000","last_clean":"2023-08-09T13:34:08.372392+0000","last_became_active":"2023-08-09T13:34:07.241841+0000","last_became_peered":"2023-08-09T13:34:07.241841+0000","last_unstale":"2023-08-09T13:34:08.372392+0000","last_undegraded":"2023-08-09T13:34:08.372392+0000","last_fullsized":"2023-08-09T13:34:08.372392+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:34:06.173547+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:34:06.173547+0000","last_clean_scrub_stamp":"2023-08-09T13:34:06.173547+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T14:35:09.213789+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,2,0],"acting":[4,3,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.1a","version":"19'1","reported_seq":22,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-08-09T13:34:08.372077+0000","last_change":"2023-08-09T13:34:02.104004+0000","last_active":"2023-08-09T13:34:08.372077+0000","last_peered":"2023-08-09T13:34:08.372077+0000","last_clean":"2023-08-09T13:34:08.372077+0000","last_became_active":"2023-08-09T13:34:02.103701+0000","last_became_peered":"2023-08-09T13:34:02.103701+0000","last_unstale":"2023-08-09T13:34:08.372077+0000","last_undegraded":"2023-08-09T13:34:08.372077+0000","last_fullsized":"2023-08-09T13:34:08.372077+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:34:01.052721+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:34:01.052721+0000","last_clean_scrub_stamp":"2023-08-09T13:34:01.052721+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T18:58:30.540971+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.1d","version":"0'0","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-08-09T13:34:08.371028+0000","last_change":"2023-08-09T13:34:03.156422+0000","last_active":"2023-08-09T13:34:08.371028+0000","last_peered":"2023-08-09T13:34:08.371028+0000","last_clean":"2023-08-09T13:34:08.371028+0000","last_became_active":"2023-08-09T13:34:03.155091+0000","last_became_peered":"2023-08-09T13:34:03.155091+0000","last_unstale":"2023-08-09T13:34:08.371028+0000","last_undegraded":"2023-08-09T13:34:08.371028+0000","last_fullsized":"2023-08-09T13:34:08.371028+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:34:02.064154+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:34:02.064154+0000","last_clean_scrub_stamp":"2023-08-09T13:34:02.064154+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T15:06:40.553178+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.1f","version":"0'0","reported_seq":13,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-08-09T13:34:08.213034+0000","last_change":"2023-08-09T13:34:07.241585+0000","last_active":"2023-08-09T13:34:08.213034+0000","last_peered":"2023-08-09T13:34:08.213034+0000","last_clean":"2023-08-09T13:34:08.213034+0000","last_became_active":"2023-08-09T13:34:07.241206+0000","last_became_peered":"2023-08-09T13:34:07.241206+0000","last_unstale":"2023-08-09T13:34:08.213034+0000","last_undegraded":"2023-08-09T13:34:08.213034+0000","last_fullsized":"2023-08-09T13:34:08.213034+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:34:06.173547+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:34:06.173547+0000","last_clean_scrub_stamp":"2023-08-09T13:34:06.173547+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-11T00:43:38.951241+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,3,7],"acting":[6,4,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":21,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-08-09T13:34:08.368085+0000","last_change":"2023-08-09T13:34:02.104251+0000","last_active":"2023-08-09T13:34:08.368085+0000","last_peered":"2023-08-09T13:34:08.368085+0000","last_clean":"2023-08-09T13:34:08.368085+0000","last_became_active":"2023-08-09T13:34:02.103920+0000","last_became_peered":"2023-08-09T13:34:02.103920+0000","last_unstale":"2023-08-09T13:34:08.368085+0000","last_undegraded":"2023-08-09T13:34:08.368085+0000","last_fullsized":"2023-08-09T13:34:08.368085+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:34:01.052721+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:34:01.052721+0000","last_clean_scrub_stamp":"2023-08-09T13:34:01.052721+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T18:36:38.900810+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1e","version":"0'0","reported_seq":17,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-08-09T13:34:08.208574+0000","last_change":"2023-08-09T13:34:03.172919+0000","last_active":"2023-08-09T13:34:08.208574+0000","last_peered":"2023-08-09T13:34:08.208574+0000","last_clean":"2023-08-09T13:34:08.208574+0000","last_became_active":"2023-08-09T13:34:03.172587+0000","last_became_peered":"2023-08-09T13:34:03.172587+0000","last_unstale":"2023-08-09T13:34:08.208574+0000","last_undegraded":"2023-08-09T13:34:08.208574+0000","last_fullsized":"2023-08-09T13:34:08.208574+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:34:02.064154+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:34:02.064154+0000","last_clean_scrub_stamp":"2023-08-09T13:34:02.064154+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T15:53:25.307248+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.1e","version":"0'0","reported_seq":13,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-08-09T13:34:08.209416+0000","last_change":"2023-08-09T13:34:07.236079+0000","last_active":"2023-08-09T13:34:08.209416+0000","last_peered":"2023-08-09T13:34:08.209416+0000","last_clean":"2023-08-09T13:34:08.209416+0000","last_became_active":"2023-08-09T13:34:07.234955+0000","last_became_peered":"2023-08-09T13:34:07.234955+0000","last_unstale":"2023-08-09T13:34:08.209416+0000","last_undegraded":"2023-08-09T13:34:08.209416+0000","last_fullsized":"2023-08-09T13:34:08.209416+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:34:06.173547+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:34:06.173547+0000","last_clean_scrub_stamp":"2023-08-09T13:34:06.173547+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T22:30:10.569066+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,1,2],"acting":[0,7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":21,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-08-09T13:34:08.368549+0000","last_change":"2023-08-09T13:34:02.103043+0000","last_active":"2023-08-09T13:34:08.368549+0000","last_peered":"2023-08-09T13:34:08.368549+0000","last_clean":"2023-08-09T13:34:08.368549+0000","last_became_active":"2023-08-09T13:34:02.102724+0000","last_became_peered":"2023-08-09T13:34:02.102724+0000","last_unstale":"2023-08-09T13:34:08.368549+0000","last_undegraded":"2023-08-09T13:34:08.368549+0000","last_fullsized":"2023-08-09T13:34:08.368549+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:34:01.052721+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:34:01.052721+0000","last_clean_scrub_stamp":"2023-08-09T13:34:01.052721+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T17:59:27.409131+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1f","version":"0'0","reported_seq":17,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-08-09T13:34:08.213083+0000","last_change":"2023-08-09T13:34:03.150663+0000","last_active":"2023-08-09T13:34:08.213083+0000","last_peered":"2023-08-09T13:34:08.213083+0000","last_clean":"2023-08-09T13:34:08.213083+0000","last_became_active":"2023-08-09T13:34:03.150322+0000","last_became_peered":"2023-08-09T13:34:03.150322+0000","last_unstale":"2023-08-09T13:34:08.213083+0000","last_undegraded":"2023-08-09T13:34:08.213083+0000","last_fullsized":"2023-08-09T13:34:08.213083+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:34:02.064154+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:34:02.064154+0000","last_clean_scrub_stamp":"2023-08-09T13:34:02.064154+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T22:09:11.225000+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.10","version":"0'0","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-08-09T13:34:08.368129+0000","last_change":"2023-08-09T13:34:03.154639+0000","last_active":"2023-08-09T13:34:08.368129+0000","last_peered":"2023-08-09T13:34:08.368129+0000","last_clean":"2023-08-09T13:34:08.368129+0000","last_became_active":"2023-08-09T13:34:03.151967+0000","last_became_peered":"2023-08-09T13:34:03.151967+0000","last_unstale":"2023-08-09T13:34:08.368129+0000","last_undegraded":"2023-08-09T13:34:08.368129+0000","last_fullsized":"2023-08-09T13:34:08.368129+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:34:02.064154+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:34:02.064154+0000","last_clean_scrub_stamp":"2023-08-09T13:34:02.064154+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-11T00:11:21.887434+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-08-09T13:34:08.210224+0000","last_change":"2023-08-09T13:34:02.108763+0000","last_active":"2023-08-09T13:34:08.210224+0000","last_peered":"2023-08-09T13:34:08.210224+0000","last_clean":"2023-08-09T13:34:08.210224+0000","last_became_active":"2023-08-09T13:34:02.108439+0000","last_became_peered":"2023-08-09T13:34:02.108439+0000","last_unstale":"2023-08-09T13:34:08.210224+0000","last_undegraded":"2023-08-09T13:34:08.210224+0000","last_fullsized":"2023-08-09T13:34:08.210224+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:34:01.052721+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:34:01.052721+0000","last_clean_scrub_stamp":"2023-08-09T13:34:01.052721+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T17:08:06.492548+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.11","version":"0'0","reported_seq":13,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-08-09T13:34:08.211048+0000","last_change":"2023-08-09T13:34:07.241202+0000","last_active":"2023-08-09T13:34:08.211048+0000","last_peered":"2023-08-09T13:34:08.211048+0000","last_clean":"2023-08-09T13:34:08.211048+0000","last_became_active":"2023-08-09T13:34:07.240678+0000","last_became_peered":"2023-08-09T13:34:07.240678+0000","last_unstale":"2023-08-09T13:34:08.211048+0000","last_undegraded":"2023-08-09T13:34:08.211048+0000","last_fullsized":"2023-08-09T13:34:08.211048+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:34:06.173547+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:34:06.173547+0000","last_clean_scrub_stamp":"2023-08-09T13:34:06.173547+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T22:29:57.384386+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,7,1],"acting":[6,4,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-08-09T13:34:08.369040+0000","last_change":"2023-08-09T13:34:03.163640+0000","last_active":"2023-08-09T13:34:08.369040+0000","last_peered":"2023-08-09T13:34:08.369040+0000","last_clean":"2023-08-09T13:34:08.369040+0000","last_became_active":"2023-08-09T13:34:03.163333+0000","last_became_peered":"2023-08-09T13:34:03.163333+0000","last_unstale":"2023-08-09T13:34:08.369040+0000","last_undegraded":"2023-08-09T13:34:08.369040+0000","last_fullsized":"2023-08-09T13:34:08.369040+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:34:02.064154+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:34:02.064154+0000","last_clean_scrub_stamp":"2023-08-09T13:34:02.064154+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T17:55:41.057906+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-08-09T13:34:08.371698+0000","last_change":"2023-08-09T13:34:02.112947+0000","last_active":"2023-08-09T13:34:08.371698+0000","last_peered":"2023-08-09T13:34:08.371698+0000","last_clean":"2023-08-09T13:34:08.371698+0000","last_became_active":"2023-08-09T13:34:02.112545+0000","last_became_peered":"2023-08-09T13:34:02.112545+0000","last_unstale":"2023-08-09T13:34:08.371698+0000","last_undegraded":"2023-08-09T13:34:08.371698+0000","last_fullsized":"2023-08-09T13:34:08.371698+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:34:01.052721+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:34:01.052721+0000","last_clean_scrub_stamp":"2023-08-09T13:34:01.052721+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T18:07:30.693736+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.10","version":"0'0","reported_seq":13,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-08-09T13:34:08.367894+0000","last_change":"2023-08-09T13:34:07.231662+0000","last_active":"2023-08-09T13:34:08.367894+0000","last_peered":"2023-08-09T13:34:08.367894+0000","last_clean":"2023-08-09T13:34:08.367894+0000","last_became_active":"2023-08-09T13:34:07.231214+0000","last_became_peered":"2023-08-09T13:34:07.231214+0000","last_unstale":"2023-08-09T13:34:08.367894+0000","last_undegraded":"2023-08-09T13:34:08.367894+0000","last_fullsized":"2023-08-09T13:34:08.367894+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:34:06.173547+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:34:06.173547+0000","last_clean_scrub_stamp":"2023-08-09T13:34:06.173547+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T21:35:05.575947+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6,0],"acting":[7,4,6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.15","version":"19'3","reported_seq":22,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-08-09T13:34:08.370972+0000","last_change":"2023-08-09T13:34:02.112344+0000","last_active":"2023-08-09T13:34:08.370972+0000","last_peered":"2023-08-09T13:34:08.370972+0000","last_clean":"2023-08-09T13:34:08.370972+0000","last_became_active":"2023-08-09T13:34:02.111749+0000","last_became_peered":"2023-08-09T13:34:02.111749+0000","last_unstale":"2023-08-09T13:34:08.370972+0000","last_undegraded":"2023-08-09T13:34:08.370972+0000","last_fullsized":"2023-08-09T13:34:08.370972+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:34:01.052721+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:34:01.052721+0000","last_clean_scrub_stamp":"2023-08-09T13:34:01.052721+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T15:33:08.276561+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-08-09T13:34:08.369248+0000","last_change":"2023-08-09T13:34:03.154995+0000","last_active":"2023-08-09T13:34:08.369248+0000","last_peered":"2023-08-09T13:34:08.369248+0000","last_clean":"2023-08-09T13:34:08.369248+0000","last_became_active":"2023-08-09T13:34:03.154715+0000","last_became_peered":"2023-08-09T13:34:03.154715+0000","last_unstale":"2023-08-09T13:34:08.369248+0000","last_undegraded":"2023-08-09T13:34:08.369248+0000","last_fullsized":"2023-08-09T13:34:08.369248+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:34:02.064154+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:34:02.064154+0000","last_clean_scrub_stamp":"2023-08-09T13:34:02.064154+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T22:30:06.459673+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.13","version":"0'0","reported_seq":13,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-08-09T13:34:08.368692+0000","last_change":"2023-08-09T13:34:07.241979+0000","last_active":"2023-08-09T13:34:08.368692+0000","last_peered":"2023-08-09T13:34:08.368692+0000","last_clean":"2023-08-09T13:34:08.368692+0000","last_became_active":"2023-08-09T13:34:07.241025+0000","last_became_peered":"2023-08-09T13:34:07.241025+0000","last_unstale":"2023-08-09T13:34:08.368692+0000","last_undegraded":"2023-08-09T13:34:08.368692+0000","last_fullsized":"2023-08-09T13:34:08.368692+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:34:06.173547+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:34:06.173547+0000","last_clean_scrub_stamp":"2023-08-09T13:34:06.173547+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T17:42:24.789866+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,1,5],"acting":[3,0,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.14","version":"24'10","reported_seq":31,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-08-09T13:34:22.970697+0000","last_change":"2023-08-09T13:34:02.113338+0000","last_active":"2023-08-09T13:34:22.970697+0000","last_peered":"2023-08-09T13:34:22.970697+0000","last_clean":"2023-08-09T13:34:22.970697+0000","last_became_active":"2023-08-09T13:34:02.113042+0000","last_became_peered":"2023-08-09T13:34:02.113042+0000","last_unstale":"2023-08-09T13:34:22.970697+0000","last_undegraded":"2023-08-09T13:34:22.970697+0000","last_fullsized":"2023-08-09T13:34:22.970697+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:34:01.052721+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:34:01.052721+0000","last_clean_scrub_stamp":"2023-08-09T13:34:01.052721+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T21:15:54.123137+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":3841,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.13","version":"0'0","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-08-09T13:34:08.371930+0000","last_change":"2023-08-09T13:34:03.148233+0000","last_active":"2023-08-09T13:34:08.371930+0000","last_peered":"2023-08-09T13:34:08.371930+0000","last_clean":"2023-08-09T13:34:08.371930+0000","last_became_active":"2023-08-09T13:34:03.147860+0000","last_became_peered":"2023-08-09T13:34:03.147860+0000","last_unstale":"2023-08-09T13:34:08.371930+0000","last_undegraded":"2023-08-09T13:34:08.371930+0000","last_fullsized":"2023-08-09T13:34:08.371930+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:34:02.064154+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:34:02.064154+0000","last_clean_scrub_stamp":"2023-08-09T13:34:02.064154+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T18:17:08.524229+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.12","version":"0'0","reported_seq":13,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-08-09T13:34:08.368474+0000","last_change":"2023-08-09T13:34:07.236661+0000","last_active":"2023-08-09T13:34:08.368474+0000","last_peered":"2023-08-09T13:34:08.368474+0000","last_clean":"2023-08-09T13:34:08.368474+0000","last_became_active":"2023-08-09T13:34:07.236264+0000","last_became_peered":"2023-08-09T13:34:07.236264+0000","last_unstale":"2023-08-09T13:34:08.368474+0000","last_undegraded":"2023-08-09T13:34:08.368474+0000","last_fullsized":"2023-08-09T13:34:08.368474+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:34:06.173547+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:34:06.173547+0000","last_clean_scrub_stamp":"2023-08-09T13:34:06.173547+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T21:35:05.575947+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,3,7],"acting":[1,5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-08-09T13:34:08.369602+0000","last_change":"2023-08-09T13:34:03.149954+0000","last_active":"2023-08-09T13:34:08.369602+0000","last_peered":"2023-08-09T13:34:08.369602+0000","last_clean":"2023-08-09T13:34:08.369602+0000","last_became_active":"2023-08-09T13:34:03.149648+0000","last_became_peered":"2023-08-09T13:34:03.149648+0000","last_unstale":"2023-08-09T13:34:08.369602+0000","last_undegraded":"2023-08-09T13:34:08.369602+0000","last_fullsized":"2023-08-09T13:34:08.369602+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:34:02.064154+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:34:02.064154+0000","last_clean_scrub_stamp":"2023-08-09T13:34:02.064154+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T18:50:05.285807+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":25,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-08-09T13:34:08.213159+0000","last_change":"2023-08-09T13:33:55.886389+0000","last_active":"2023-08-09T13:34:08.213159+0000","last_peered":"2023-08-09T13:34:08.213159+0000","last_clean":"2023-08-09T13:34:08.213159+0000","last_became_active":"2023-08-09T13:33:55.885638+0000","last_became_peered":"2023-08-09T13:33:55.885638+0000","last_unstale":"2023-08-09T13:34:08.213159+0000","last_undegraded":"2023-08-09T13:34:08.213159+0000","last_fullsized":"2023-08-09T13:34:08.213159+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:33:54.845869+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:33:54.845869+0000","last_clean_scrub_stamp":"2023-08-09T13:33:54.845869+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T23:11:03.836310+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-08-09T13:34:08.370437+0000","last_change":"2023-08-09T13:34:02.114759+0000","last_active":"2023-08-09T13:34:08.370437+0000","last_peered":"2023-08-09T13:34:08.370437+0000","last_clean":"2023-08-09T13:34:08.370437+0000","last_became_active":"2023-08-09T13:34:02.114423+0000","last_became_peered":"2023-08-09T13:34:02.114423+0000","last_unstale":"2023-08-09T13:34:08.370437+0000","last_undegraded":"2023-08-09T13:34:08.370437+0000","last_fullsized":"2023-08-09T13:34:08.370437+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:34:01.052721+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:34:01.052721+0000","last_clean_scrub_stamp":"2023-08-09T13:34:01.052721+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T18:50:04.274374+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.3","version":"0'0","reported_seq":13,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-08-09T13:34:08.209248+0000","last_change":"2023-08-09T13:34:07.244416+0000","last_active":"2023-08-09T13:34:08.209248+0000","last_peered":"2023-08-09T13:34:08.209248+0000","last_clean":"2023-08-09T13:34:08.209248+0000","last_became_active":"2023-08-09T13:34:07.243294+0000","last_became_peered":"2023-08-09T13:34:07.243294+0000","last_unstale":"2023-08-09T13:34:08.209248+0000","last_undegraded":"2023-08-09T13:34:08.209248+0000","last_fullsized":"2023-08-09T13:34:08.209248+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:34:06.173547+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:34:06.173547+0000","last_clean_scrub_stamp":"2023-08-09T13:34:06.173547+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T21:35:05.575947+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,6,5],"acting":[0,1,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":17,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-08-09T13:34:08.209747+0000","last_change":"2023-08-09T13:34:03.151674+0000","last_active":"2023-08-09T13:34:08.209747+0000","last_peered":"2023-08-09T13:34:08.209747+0000","last_clean":"2023-08-09T13:34:08.209747+0000","last_became_active":"2023-08-09T13:34:03.151137+0000","last_became_peered":"2023-08-09T13:34:03.151137+0000","last_unstale":"2023-08-09T13:34:08.209747+0000","last_undegraded":"2023-08-09T13:34:08.209747+0000","last_fullsized":"2023-08-09T13:34:08.209747+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:34:02.064154+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:34:02.064154+0000","last_clean_scrub_stamp":"2023-08-09T13:34:02.064154+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T21:11:40.494329+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":25,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-08-09T13:34:08.212939+0000","last_change":"2023-08-09T13:33:56.401781+0000","last_active":"2023-08-09T13:34:08.212939+0000","last_peered":"2023-08-09T13:34:08.212939+0000","last_clean":"2023-08-09T13:34:08.212939+0000","last_became_active":"2023-08-09T13:33:56.400336+0000","last_became_peered":"2023-08-09T13:33:56.400336+0000","last_unstale":"2023-08-09T13:34:08.212939+0000","last_undegraded":"2023-08-09T13:34:08.212939+0000","last_fullsized":"2023-08-09T13:34:08.212939+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:33:54.845869+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:33:54.845869+0000","last_clean_scrub_stamp":"2023-08-09T13:33:54.845869+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T19:43:07.034435+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":21,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-08-09T13:34:08.372061+0000","last_change":"2023-08-09T13:34:02.106587+0000","last_active":"2023-08-09T13:34:08.372061+0000","last_peered":"2023-08-09T13:34:08.372061+0000","last_clean":"2023-08-09T13:34:08.372061+0000","last_became_active":"2023-08-09T13:34:02.106236+0000","last_became_peered":"2023-08-09T13:34:02.106236+0000","last_unstale":"2023-08-09T13:34:08.372061+0000","last_undegraded":"2023-08-09T13:34:08.372061+0000","last_fullsized":"2023-08-09T13:34:08.372061+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:34:01.052721+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:34:01.052721+0000","last_clean_scrub_stamp":"2023-08-09T13:34:01.052721+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T13:44:49.753930+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.2","version":"0'0","reported_seq":13,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-08-09T13:34:08.211286+0000","last_change":"2023-08-09T13:34:07.242738+0000","last_active":"2023-08-09T13:34:08.211286+0000","last_peered":"2023-08-09T13:34:08.211286+0000","last_clean":"2023-08-09T13:34:08.211286+0000","last_became_active":"2023-08-09T13:34:07.242314+0000","last_became_peered":"2023-08-09T13:34:07.242314+0000","last_unstale":"2023-08-09T13:34:08.211286+0000","last_undegraded":"2023-08-09T13:34:08.211286+0000","last_fullsized":"2023-08-09T13:34:08.211286+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:34:06.173547+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:34:06.173547+0000","last_clean_scrub_stamp":"2023-08-09T13:34:06.173547+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T18:00:37.387381+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,5,2],"acting":[6,0,5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-08-09T13:34:08.368126+0000","last_change":"2023-08-09T13:34:03.163805+0000","last_active":"2023-08-09T13:34:08.368126+0000","last_peered":"2023-08-09T13:34:08.368126+0000","last_clean":"2023-08-09T13:34:08.368126+0000","last_became_active":"2023-08-09T13:34:03.163407+0000","last_became_peered":"2023-08-09T13:34:03.163407+0000","last_unstale":"2023-08-09T13:34:08.368126+0000","last_undegraded":"2023-08-09T13:34:08.368126+0000","last_fullsized":"2023-08-09T13:34:08.368126+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:34:02.064154+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:34:02.064154+0000","last_clean_scrub_stamp":"2023-08-09T13:34:02.064154+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T23:33:49.011321+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":27,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-08-09T13:34:08.373438+0000","last_change":"2023-08-09T13:33:56.404236+0000","last_active":"2023-08-09T13:34:08.373438+0000","last_peered":"2023-08-09T13:34:08.373438+0000","last_clean":"2023-08-09T13:34:08.373438+0000","last_became_active":"2023-08-09T13:33:56.402485+0000","last_became_peered":"2023-08-09T13:33:56.402485+0000","last_unstale":"2023-08-09T13:34:08.373438+0000","last_undegraded":"2023-08-09T13:34:08.373438+0000","last_fullsized":"2023-08-09T13:34:08.373438+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:33:54.845869+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:33:54.845869+0000","last_clean_scrub_stamp":"2023-08-09T13:33:54.845869+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T18:59:14.817691+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.7","version":"19'2","reported_seq":23,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-08-09T13:34:08.369861+0000","last_change":"2023-08-09T13:34:02.108061+0000","last_active":"2023-08-09T13:34:08.369861+0000","last_peered":"2023-08-09T13:34:08.369861+0000","last_clean":"2023-08-09T13:34:08.369861+0000","last_became_active":"2023-08-09T13:34:02.107766+0000","last_became_peered":"2023-08-09T13:34:02.107766+0000","last_unstale":"2023-08-09T13:34:08.369861+0000","last_undegraded":"2023-08-09T13:34:08.369861+0000","last_fullsized":"2023-08-09T13:34:08.369861+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:34:01.052721+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:34:01.052721+0000","last_clean_scrub_stamp":"2023-08-09T13:34:01.052721+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T16:14:05.720686+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.1","version":"0'0","reported_seq":13,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-08-09T13:34:08.368770+0000","last_change":"2023-08-09T13:34:07.236332+0000","last_active":"2023-08-09T13:34:08.368770+0000","last_peered":"2023-08-09T13:34:08.368770+0000","last_clean":"2023-08-09T13:34:08.368770+0000","last_became_active":"2023-08-09T13:34:07.235923+0000","last_became_peered":"2023-08-09T13:34:07.235923+0000","last_unstale":"2023-08-09T13:34:08.368770+0000","last_undegraded":"2023-08-09T13:34:08.368770+0000","last_fullsized":"2023-08-09T13:34:08.368770+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:34:06.173547+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:34:06.173547+0000","last_clean_scrub_stamp":"2023-08-09T13:34:06.173547+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-11T01:07:37.438752+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,7,0],"acting":[4,3,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-08-09T13:34:08.369699+0000","last_change":"2023-08-09T13:34:03.149492+0000","last_active":"2023-08-09T13:34:08.369699+0000","last_peered":"2023-08-09T13:34:08.369699+0000","last_clean":"2023-08-09T13:34:08.369699+0000","last_became_active":"2023-08-09T13:34:03.148649+0000","last_became_peered":"2023-08-09T13:34:03.148649+0000","last_unstale":"2023-08-09T13:34:08.369699+0000","last_undegraded":"2023-08-09T13:34:08.369699+0000","last_fullsized":"2023-08-09T13:34:08.369699+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:34:02.064154+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:34:02.064154+0000","last_clean_scrub_stamp":"2023-08-09T13:34:02.064154+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T21:57:54.183369+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":27,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-08-09T13:34:08.370409+0000","last_change":"2023-08-09T13:33:55.889565+0000","last_active":"2023-08-09T13:34:08.370409+0000","last_peered":"2023-08-09T13:34:08.370409+0000","last_clean":"2023-08-09T13:34:08.370409+0000","last_became_active":"2023-08-09T13:33:55.889164+0000","last_became_peered":"2023-08-09T13:33:55.889164+0000","last_unstale":"2023-08-09T13:34:08.370409+0000","last_undegraded":"2023-08-09T13:34:08.370409+0000","last_fullsized":"2023-08-09T13:34:08.370409+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:33:54.845869+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:33:54.845869+0000","last_clean_scrub_stamp":"2023-08-09T13:33:54.845869+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-11T01:07:26.111074+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.6","version":"21'1","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-08-09T13:34:08.209027+0000","last_change":"2023-08-09T13:34:02.096482+0000","last_active":"2023-08-09T13:34:08.209027+0000","last_peered":"2023-08-09T13:34:08.209027+0000","last_clean":"2023-08-09T13:34:08.209027+0000","last_became_active":"2023-08-09T13:34:02.095287+0000","last_became_peered":"2023-08-09T13:34:02.095287+0000","last_unstale":"2023-08-09T13:34:08.209027+0000","last_undegraded":"2023-08-09T13:34:08.209027+0000","last_fullsized":"2023-08-09T13:34:08.209027+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:34:01.052721+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:34:01.052721+0000","last_clean_scrub_stamp":"2023-08-09T13:34:01.052721+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T18:36:38.900810+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.0","version":"0'0","reported_seq":13,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-08-09T13:34:08.367857+0000","last_change":"2023-08-09T13:34:07.244568+0000","last_active":"2023-08-09T13:34:08.367857+0000","last_peered":"2023-08-09T13:34:08.367857+0000","last_clean":"2023-08-09T13:34:08.367857+0000","last_became_active":"2023-08-09T13:34:07.243669+0000","last_became_peered":"2023-08-09T13:34:07.243669+0000","last_unstale":"2023-08-09T13:34:08.367857+0000","last_undegraded":"2023-08-09T13:34:08.367857+0000","last_fullsized":"2023-08-09T13:34:08.367857+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:34:06.173547+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:34:06.173547+0000","last_clean_scrub_stamp":"2023-08-09T13:34:06.173547+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T20:31:33.361620+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,4,0],"acting":[3,1,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.6","version":"0'0","reported_seq":17,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-08-09T13:34:08.208668+0000","last_change":"2023-08-09T13:34:03.139485+0000","last_active":"2023-08-09T13:34:08.208668+0000","last_peered":"2023-08-09T13:34:08.208668+0000","last_clean":"2023-08-09T13:34:08.208668+0000","last_became_active":"2023-08-09T13:34:03.139178+0000","last_became_peered":"2023-08-09T13:34:03.139178+0000","last_unstale":"2023-08-09T13:34:08.208668+0000","last_undegraded":"2023-08-09T13:34:08.208668+0000","last_fullsized":"2023-08-09T13:34:08.208668+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:34:02.064154+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:34:02.064154+0000","last_clean_scrub_stamp":"2023-08-09T13:34:02.064154+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T19:42:33.733326+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-08-09T13:34:08.207236+0000","last_change":"2023-08-09T13:34:02.110719+0000","last_active":"2023-08-09T13:34:08.207236+0000","last_peered":"2023-08-09T13:34:08.207236+0000","last_clean":"2023-08-09T13:34:08.207236+0000","last_became_active":"2023-08-09T13:34:02.110431+0000","last_became_peered":"2023-08-09T13:34:02.110431+0000","last_unstale":"2023-08-09T13:34:08.207236+0000","last_undegraded":"2023-08-09T13:34:08.207236+0000","last_fullsized":"2023-08-09T13:34:08.207236+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:34:01.052721+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:34:01.052721+0000","last_clean_scrub_stamp":"2023-08-09T13:34:01.052721+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T23:24:07.266385+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.0","version":"16'117","reported_seq":175,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-08-09T13:34:08.368906+0000","last_change":"2023-08-09T13:33:55.886345+0000","last_active":"2023-08-09T13:34:08.368906+0000","last_peered":"2023-08-09T13:34:08.368906+0000","last_clean":"2023-08-09T13:34:08.368906+0000","last_became_active":"2023-08-09T13:33:55.885781+0000","last_became_peered":"2023-08-09T13:33:55.885781+0000","last_unstale":"2023-08-09T13:34:08.368906+0000","last_undegraded":"2023-08-09T13:34:08.368906+0000","last_fullsized":"2023-08-09T13:34:08.368906+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:33:54.845869+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:33:54.845869+0000","last_clean_scrub_stamp":"2023-08-09T13:33:54.845869+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T18:49:58.067523+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.3","version":"0'0","reported_seq":27,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-08-09T13:34:08.370928+0000","last_change":"2023-08-09T13:33:55.886749+0000","last_active":"2023-08-09T13:34:08.370928+0000","last_peered":"2023-08-09T13:34:08.370928+0000","last_clean":"2023-08-09T13:34:08.370928+0000","last_became_active":"2023-08-09T13:33:55.886116+0000","last_became_peered":"2023-08-09T13:33:55.886116+0000","last_unstale":"2023-08-09T13:34:08.370928+0000","last_undegraded":"2023-08-09T13:34:08.370928+0000","last_fullsized":"2023-08-09T13:34:08.370928+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:33:54.845869+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:33:54.845869+0000","last_clean_scrub_stamp":"2023-08-09T13:33:54.845869+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T17:46:15.820922+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.7","version":"0'0","reported_seq":13,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-08-09T13:34:08.371420+0000","last_change":"2023-08-09T13:34:07.235784+0000","last_active":"2023-08-09T13:34:08.371420+0000","last_peered":"2023-08-09T13:34:08.371420+0000","last_clean":"2023-08-09T13:34:08.371420+0000","last_became_active":"2023-08-09T13:34:07.235282+0000","last_became_peered":"2023-08-09T13:34:07.235282+0000","last_unstale":"2023-08-09T13:34:08.371420+0000","last_undegraded":"2023-08-09T13:34:08.371420+0000","last_fullsized":"2023-08-09T13:34:08.371420+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:34:06.173547+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:34:06.173547+0000","last_clean_scrub_stamp":"2023-08-09T13:34:06.173547+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T23:52:31.847779+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,0,4],"acting":[5,1,0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.7","version":"0'0","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-08-09T13:34:08.372407+0000","last_change":"2023-08-09T13:34:03.149568+0000","last_active":"2023-08-09T13:34:08.372407+0000","last_peered":"2023-08-09T13:34:08.372407+0000","last_clean":"2023-08-09T13:34:08.372407+0000","last_became_active":"2023-08-09T13:34:03.148937+0000","last_became_peered":"2023-08-09T13:34:03.148937+0000","last_unstale":"2023-08-09T13:34:08.372407+0000","last_undegraded":"2023-08-09T13:34:08.372407+0000","last_fullsized":"2023-08-09T13:34:08.372407+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:34:02.064154+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:34:02.064154+0000","last_clean_scrub_stamp":"2023-08-09T13:34:02.064154+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T23:34:54.506258+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":21,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-08-09T13:34:08.367679+0000","last_change":"2023-08-09T13:34:02.102795+0000","last_active":"2023-08-09T13:34:08.367679+0000","last_peered":"2023-08-09T13:34:08.367679+0000","last_clean":"2023-08-09T13:34:08.367679+0000","last_became_active":"2023-08-09T13:34:02.101883+0000","last_became_peered":"2023-08-09T13:34:02.101883+0000","last_unstale":"2023-08-09T13:34:08.367679+0000","last_undegraded":"2023-08-09T13:34:08.367679+0000","last_fullsized":"2023-08-09T13:34:08.367679+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:34:01.052721+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:34:01.052721+0000","last_clean_scrub_stamp":"2023-08-09T13:34:01.052721+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T19:22:07.585850+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.2","version":"0'0","reported_seq":25,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-08-09T13:34:08.369310+0000","last_change":"2023-08-09T13:33:56.402794+0000","last_active":"2023-08-09T13:34:08.369310+0000","last_peered":"2023-08-09T13:34:08.369310+0000","last_clean":"2023-08-09T13:34:08.369310+0000","last_became_active":"2023-08-09T13:33:56.402038+0000","last_became_peered":"2023-08-09T13:33:56.402038+0000","last_unstale":"2023-08-09T13:34:08.369310+0000","last_undegraded":"2023-08-09T13:34:08.369310+0000","last_fullsized":"2023-08-09T13:34:08.369310+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:33:54.845869+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:33:54.845869+0000","last_clean_scrub_stamp":"2023-08-09T13:33:54.845869+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T18:07:24.486884+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.6","version":"0'0","reported_seq":13,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-08-09T13:34:08.212013+0000","last_change":"2023-08-09T13:34:07.244168+0000","last_active":"2023-08-09T13:34:08.212013+0000","last_peered":"2023-08-09T13:34:08.212013+0000","last_clean":"2023-08-09T13:34:08.212013+0000","last_became_active":"2023-08-09T13:34:07.241903+0000","last_became_peered":"2023-08-09T13:34:07.241903+0000","last_unstale":"2023-08-09T13:34:08.212013+0000","last_undegraded":"2023-08-09T13:34:08.212013+0000","last_fullsized":"2023-08-09T13:34:08.212013+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:34:06.173547+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:34:06.173547+0000","last_clean_scrub_stamp":"2023-08-09T13:34:06.173547+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-11T00:23:22.192340+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,1,7],"acting":[2,5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-08-09T13:34:08.371697+0000","last_change":"2023-08-09T13:34:03.153039+0000","last_active":"2023-08-09T13:34:08.371697+0000","last_peered":"2023-08-09T13:34:08.371697+0000","last_clean":"2023-08-09T13:34:08.371697+0000","last_became_active":"2023-08-09T13:34:03.152401+0000","last_became_peered":"2023-08-09T13:34:03.152401+0000","last_unstale":"2023-08-09T13:34:08.371697+0000","last_undegraded":"2023-08-09T13:34:08.371697+0000","last_fullsized":"2023-08-09T13:34:08.371697+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:34:02.064154+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:34:02.064154+0000","last_clean_scrub_stamp":"2023-08-09T13:34:02.064154+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T21:18:19.551549+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":21,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-08-09T13:34:08.372303+0000","last_change":"2023-08-09T13:34:02.108657+0000","last_active":"2023-08-09T13:34:08.372303+0000","last_peered":"2023-08-09T13:34:08.372303+0000","last_clean":"2023-08-09T13:34:08.372303+0000","last_became_active":"2023-08-09T13:34:02.105481+0000","last_became_peered":"2023-08-09T13:34:02.105481+0000","last_unstale":"2023-08-09T13:34:08.372303+0000","last_undegraded":"2023-08-09T13:34:08.372303+0000","last_fullsized":"2023-08-09T13:34:08.372303+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:34:01.052721+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:34:01.052721+0000","last_clean_scrub_stamp":"2023-08-09T13:34:01.052721+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T22:51:18.775151+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":25,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-08-09T13:34:08.368026+0000","last_change":"2023-08-09T13:33:56.401123+0000","last_active":"2023-08-09T13:34:08.368026+0000","last_peered":"2023-08-09T13:34:08.368026+0000","last_clean":"2023-08-09T13:34:08.368026+0000","last_became_active":"2023-08-09T13:33:56.399933+0000","last_became_peered":"2023-08-09T13:33:56.399933+0000","last_unstale":"2023-08-09T13:34:08.368026+0000","last_undegraded":"2023-08-09T13:34:08.368026+0000","last_fullsized":"2023-08-09T13:34:08.368026+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:33:54.845869+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:33:54.845869+0000","last_clean_scrub_stamp":"2023-08-09T13:33:54.845869+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T23:24:01.059533+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.5","version":"0'0","reported_seq":13,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-08-09T13:34:08.208281+0000","last_change":"2023-08-09T13:34:07.235418+0000","last_active":"2023-08-09T13:34:08.208281+0000","last_peered":"2023-08-09T13:34:08.208281+0000","last_clean":"2023-08-09T13:34:08.208281+0000","last_became_active":"2023-08-09T13:34:07.234889+0000","last_became_peered":"2023-08-09T13:34:07.234889+0000","last_unstale":"2023-08-09T13:34:08.208281+0000","last_undegraded":"2023-08-09T13:34:08.208281+0000","last_fullsized":"2023-08-09T13:34:08.208281+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:34:06.173547+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:34:06.173547+0000","last_clean_scrub_stamp":"2023-08-09T13:34:06.173547+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T20:46:19.756813+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,4,6],"acting":[0,1,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.5","version":"0'0","reported_seq":17,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-08-09T13:34:08.212552+0000","last_change":"2023-08-09T13:34:03.159726+0000","last_active":"2023-08-09T13:34:08.212552+0000","last_peered":"2023-08-09T13:34:08.212552+0000","last_clean":"2023-08-09T13:34:08.212552+0000","last_became_active":"2023-08-09T13:34:03.159238+0000","last_became_peered":"2023-08-09T13:34:03.159238+0000","last_unstale":"2023-08-09T13:34:08.212552+0000","last_undegraded":"2023-08-09T13:34:08.212552+0000","last_fullsized":"2023-08-09T13:34:08.212552+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:34:02.064154+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:34:02.064154+0000","last_clean_scrub_stamp":"2023-08-09T13:34:02.064154+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T17:32:16.845908+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":21,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-08-09T13:34:08.368904+0000","last_change":"2023-08-09T13:34:02.114840+0000","last_active":"2023-08-09T13:34:08.368904+0000","last_peered":"2023-08-09T13:34:08.368904+0000","last_clean":"2023-08-09T13:34:08.368904+0000","last_became_active":"2023-08-09T13:34:02.114547+0000","last_became_peered":"2023-08-09T13:34:02.114547+0000","last_unstale":"2023-08-09T13:34:08.368904+0000","last_undegraded":"2023-08-09T13:34:08.368904+0000","last_fullsized":"2023-08-09T13:34:08.368904+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:34:01.052721+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:34:01.052721+0000","last_clean_scrub_stamp":"2023-08-09T13:34:01.052721+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T13:43:19.071421+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":25,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-08-09T13:34:08.369269+0000","last_change":"2023-08-09T13:33:56.401406+0000","last_active":"2023-08-09T13:34:08.369269+0000","last_peered":"2023-08-09T13:34:08.369269+0000","last_clean":"2023-08-09T13:34:08.369269+0000","last_became_active":"2023-08-09T13:33:56.400487+0000","last_became_peered":"2023-08-09T13:33:56.400487+0000","last_unstale":"2023-08-09T13:34:08.369269+0000","last_undegraded":"2023-08-09T13:34:08.369269+0000","last_fullsized":"2023-08-09T13:34:08.369269+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:33:54.845869+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:33:54.845869+0000","last_clean_scrub_stamp":"2023-08-09T13:33:54.845869+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T23:52:20.520101+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.4","version":"0'0","reported_seq":13,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-08-09T13:34:08.370501+0000","last_change":"2023-08-09T13:34:07.249799+0000","last_active":"2023-08-09T13:34:08.370501+0000","last_peered":"2023-08-09T13:34:08.370501+0000","last_clean":"2023-08-09T13:34:08.370501+0000","last_became_active":"2023-08-09T13:34:07.249286+0000","last_became_peered":"2023-08-09T13:34:07.249286+0000","last_unstale":"2023-08-09T13:34:08.370501+0000","last_undegraded":"2023-08-09T13:34:08.370501+0000","last_fullsized":"2023-08-09T13:34:08.370501+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:34:06.173547+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:34:06.173547+0000","last_clean_scrub_stamp":"2023-08-09T13:34:06.173547+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T15:43:48.184972+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2,5,3],"acting":[7,2,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.f","version":"0'0","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-08-09T13:34:08.372421+0000","last_change":"2023-08-09T13:34:03.151508+0000","last_active":"2023-08-09T13:34:08.372421+0000","last_peered":"2023-08-09T13:34:08.372421+0000","last_clean":"2023-08-09T13:34:08.372421+0000","last_became_active":"2023-08-09T13:34:03.149550+0000","last_became_peered":"2023-08-09T13:34:03.149550+0000","last_unstale":"2023-08-09T13:34:08.372421+0000","last_undegraded":"2023-08-09T13:34:08.372421+0000","last_fullsized":"2023-08-09T13:34:08.372421+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:34:02.064154+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:34:02.064154+0000","last_clean_scrub_stamp":"2023-08-09T13:34:02.064154+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T22:51:11.201456+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":21,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-08-09T13:34:08.368991+0000","last_change":"2023-08-09T13:34:02.095946+0000","last_active":"2023-08-09T13:34:08.368991+0000","last_peered":"2023-08-09T13:34:08.368991+0000","last_clean":"2023-08-09T13:34:08.368991+0000","last_became_active":"2023-08-09T13:34:02.095325+0000","last_became_peered":"2023-08-09T13:34:02.095325+0000","last_unstale":"2023-08-09T13:34:08.368991+0000","last_undegraded":"2023-08-09T13:34:08.368991+0000","last_fullsized":"2023-08-09T13:34:08.368991+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:34:01.052721+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:34:01.052721+0000","last_clean_scrub_stamp":"2023-08-09T13:34:01.052721+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-11T00:06:29.000035+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.e","version":"0'0","reported_seq":13,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-08-09T13:34:08.373803+0000","last_change":"2023-08-09T13:34:07.232690+0000","last_active":"2023-08-09T13:34:08.373803+0000","last_peered":"2023-08-09T13:34:08.373803+0000","last_clean":"2023-08-09T13:34:08.373803+0000","last_became_active":"2023-08-09T13:34:07.232326+0000","last_became_peered":"2023-08-09T13:34:07.232326+0000","last_unstale":"2023-08-09T13:34:08.373803+0000","last_undegraded":"2023-08-09T13:34:08.373803+0000","last_fullsized":"2023-08-09T13:34:08.373803+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:34:06.173547+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:34:06.173547+0000","last_clean_scrub_stamp":"2023-08-09T13:34:06.173547+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T13:37:38.237696+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,0,2],"acting":[4,5,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.e","version":"0'0","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-08-09T13:34:08.373562+0000","last_change":"2023-08-09T13:34:03.149791+0000","last_active":"2023-08-09T13:34:08.373562+0000","last_peered":"2023-08-09T13:34:08.373562+0000","last_clean":"2023-08-09T13:34:08.373562+0000","last_became_active":"2023-08-09T13:34:03.149525+0000","last_became_peered":"2023-08-09T13:34:03.149525+0000","last_unstale":"2023-08-09T13:34:08.373562+0000","last_undegraded":"2023-08-09T13:34:08.373562+0000","last_fullsized":"2023-08-09T13:34:08.373562+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:34:02.064154+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:34:02.064154+0000","last_clean_scrub_stamp":"2023-08-09T13:34:02.064154+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T14:53:59.337109+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":21,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-08-09T13:34:08.369963+0000","last_change":"2023-08-09T13:34:02.105359+0000","last_active":"2023-08-09T13:34:08.369963+0000","last_peered":"2023-08-09T13:34:08.369963+0000","last_clean":"2023-08-09T13:34:08.369963+0000","last_became_active":"2023-08-09T13:34:02.105027+0000","last_became_peered":"2023-08-09T13:34:02.105027+0000","last_unstale":"2023-08-09T13:34:08.369963+0000","last_undegraded":"2023-08-09T13:34:08.369963+0000","last_fullsized":"2023-08-09T13:34:08.369963+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:34:01.052721+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:34:01.052721+0000","last_clean_scrub_stamp":"2023-08-09T13:34:01.052721+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T13:45:26.240878+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.f","version":"0'0","reported_seq":13,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-08-09T13:34:08.371141+0000","last_change":"2023-08-09T13:34:07.239625+0000","last_active":"2023-08-09T13:34:08.371141+0000","last_peered":"2023-08-09T13:34:08.371141+0000","last_clean":"2023-08-09T13:34:08.371141+0000","last_became_active":"2023-08-09T13:34:07.239002+0000","last_became_peered":"2023-08-09T13:34:07.239002+0000","last_unstale":"2023-08-09T13:34:08.371141+0000","last_undegraded":"2023-08-09T13:34:08.371141+0000","last_fullsized":"2023-08-09T13:34:08.371141+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:34:06.173547+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:34:06.173547+0000","last_clean_scrub_stamp":"2023-08-09T13:34:06.173547+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T21:11:44.603723+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,6,2],"acting":[5,4,6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.d","version":"0'0","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-08-09T13:34:08.373090+0000","last_change":"2023-08-09T13:34:03.152579+0000","last_active":"2023-08-09T13:34:08.373090+0000","last_peered":"2023-08-09T13:34:08.373090+0000","last_clean":"2023-08-09T13:34:08.373090+0000","last_became_active":"2023-08-09T13:34:03.152287+0000","last_became_peered":"2023-08-09T13:34:03.152287+0000","last_unstale":"2023-08-09T13:34:08.373090+0000","last_undegraded":"2023-08-09T13:34:08.373090+0000","last_fullsized":"2023-08-09T13:34:08.373090+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:34:02.064154+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:34:02.064154+0000","last_clean_scrub_stamp":"2023-08-09T13:34:02.064154+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T17:37:51.284066+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-08-09T13:34:08.211685+0000","last_change":"2023-08-09T13:34:02.109847+0000","last_active":"2023-08-09T13:34:08.211685+0000","last_peered":"2023-08-09T13:34:08.211685+0000","last_clean":"2023-08-09T13:34:08.211685+0000","last_became_active":"2023-08-09T13:34:02.109516+0000","last_became_peered":"2023-08-09T13:34:02.109516+0000","last_unstale":"2023-08-09T13:34:08.211685+0000","last_undegraded":"2023-08-09T13:34:08.211685+0000","last_fullsized":"2023-08-09T13:34:08.211685+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:34:01.052721+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:34:01.052721+0000","last_clean_scrub_stamp":"2023-08-09T13:34:01.052721+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T22:40:21.488794+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.c","version":"0'0","reported_seq":13,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-08-09T13:34:08.371235+0000","last_change":"2023-08-09T13:34:07.231138+0000","last_active":"2023-08-09T13:34:08.371235+0000","last_peered":"2023-08-09T13:34:08.371235+0000","last_clean":"2023-08-09T13:34:08.371235+0000","last_became_active":"2023-08-09T13:34:07.230741+0000","last_became_peered":"2023-08-09T13:34:07.230741+0000","last_unstale":"2023-08-09T13:34:08.371235+0000","last_undegraded":"2023-08-09T13:34:08.371235+0000","last_fullsized":"2023-08-09T13:34:08.371235+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:34:06.173547+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:34:06.173547+0000","last_clean_scrub_stamp":"2023-08-09T13:34:06.173547+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T21:11:44.603723+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4,0,6],"acting":[1,4,0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.c","version":"0'0","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-08-09T13:34:08.372932+0000","last_change":"2023-08-09T13:34:03.159095+0000","last_active":"2023-08-09T13:34:08.372932+0000","last_peered":"2023-08-09T13:34:08.372932+0000","last_clean":"2023-08-09T13:34:08.372932+0000","last_became_active":"2023-08-09T13:34:03.158807+0000","last_became_peered":"2023-08-09T13:34:03.158807+0000","last_unstale":"2023-08-09T13:34:08.372932+0000","last_undegraded":"2023-08-09T13:34:08.372932+0000","last_fullsized":"2023-08-09T13:34:08.372932+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:34:02.064154+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:34:02.064154+0000","last_clean_scrub_stamp":"2023-08-09T13:34:02.064154+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T21:06:37.115870+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.b","version":"19'2","reported_seq":23,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-08-09T13:34:08.369938+0000","last_change":"2023-08-09T13:34:02.103259+0000","last_active":"2023-08-09T13:34:08.369938+0000","last_peered":"2023-08-09T13:34:08.369938+0000","last_clean":"2023-08-09T13:34:08.369938+0000","last_became_active":"2023-08-09T13:34:02.102644+0000","last_became_peered":"2023-08-09T13:34:02.102644+0000","last_unstale":"2023-08-09T13:34:08.369938+0000","last_undegraded":"2023-08-09T13:34:08.369938+0000","last_fullsized":"2023-08-09T13:34:08.369938+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:34:01.052721+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:34:01.052721+0000","last_clean_scrub_stamp":"2023-08-09T13:34:01.052721+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T19:48:11.160512+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.d","version":"0'0","reported_seq":13,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-08-09T13:34:08.211821+0000","last_change":"2023-08-09T13:34:07.242458+0000","last_active":"2023-08-09T13:34:08.211821+0000","last_peered":"2023-08-09T13:34:08.211821+0000","last_clean":"2023-08-09T13:34:08.211821+0000","last_became_active":"2023-08-09T13:34:07.241393+0000","last_became_peered":"2023-08-09T13:34:07.241393+0000","last_unstale":"2023-08-09T13:34:08.211821+0000","last_undegraded":"2023-08-09T13:34:08.211821+0000","last_fullsized":"2023-08-09T13:34:08.211821+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:34:06.173547+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:34:06.173547+0000","last_clean_scrub_stamp":"2023-08-09T13:34:06.173547+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T19:00:12.926085+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7,5,6],"acting":[2,7,5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.b","version":"0'0","reported_seq":17,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-08-09T13:34:08.209500+0000","last_change":"2023-08-09T13:34:03.155061+0000","last_active":"2023-08-09T13:34:08.209500+0000","last_peered":"2023-08-09T13:34:08.209500+0000","last_clean":"2023-08-09T13:34:08.209500+0000","last_became_active":"2023-08-09T13:34:03.154792+0000","last_became_peered":"2023-08-09T13:34:03.154792+0000","last_unstale":"2023-08-09T13:34:08.209500+0000","last_undegraded":"2023-08-09T13:34:08.209500+0000","last_fullsized":"2023-08-09T13:34:08.209500+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:34:02.064154+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:34:02.064154+0000","last_clean_scrub_stamp":"2023-08-09T13:34:02.064154+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T18:50:05.285807+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-08-09T13:34:08.369085+0000","last_change":"2023-08-09T13:34:02.111888+0000","last_active":"2023-08-09T13:34:08.369085+0000","last_peered":"2023-08-09T13:34:08.369085+0000","last_clean":"2023-08-09T13:34:08.369085+0000","last_became_active":"2023-08-09T13:34:02.111588+0000","last_became_peered":"2023-08-09T13:34:02.111588+0000","last_unstale":"2023-08-09T13:34:08.369085+0000","last_undegraded":"2023-08-09T13:34:08.369085+0000","last_fullsized":"2023-08-09T13:34:08.369085+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:34:01.052721+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:34:01.052721+0000","last_clean_scrub_stamp":"2023-08-09T13:34:01.052721+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T15:53:24.295815+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.a","version":"0'0","reported_seq":13,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-08-09T13:34:08.211349+0000","last_change":"2023-08-09T13:34:07.241400+0000","last_active":"2023-08-09T13:34:08.211349+0000","last_peered":"2023-08-09T13:34:08.211349+0000","last_clean":"2023-08-09T13:34:08.211349+0000","last_became_active":"2023-08-09T13:34:07.240945+0000","last_became_peered":"2023-08-09T13:34:07.240945+0000","last_unstale":"2023-08-09T13:34:08.211349+0000","last_undegraded":"2023-08-09T13:34:08.211349+0000","last_fullsized":"2023-08-09T13:34:08.211349+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:34:06.173547+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:34:06.173547+0000","last_clean_scrub_stamp":"2023-08-09T13:34:06.173547+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T20:24:07.327648+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4,3,6],"acting":[2,4,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":17,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-08-09T13:34:08.211229+0000","last_change":"2023-08-09T13:34:03.159173+0000","last_active":"2023-08-09T13:34:08.211229+0000","last_peered":"2023-08-09T13:34:08.211229+0000","last_clean":"2023-08-09T13:34:08.211229+0000","last_became_active":"2023-08-09T13:34:03.157726+0000","last_became_peered":"2023-08-09T13:34:03.157726+0000","last_unstale":"2023-08-09T13:34:08.211229+0000","last_undegraded":"2023-08-09T13:34:08.211229+0000","last_fullsized":"2023-08-09T13:34:08.211229+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:34:02.064154+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:34:02.064154+0000","last_clean_scrub_stamp":"2023-08-09T13:34:02.064154+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T18:49:41.829768+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.d","version":"19'1","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-08-09T13:34:08.371386+0000","last_change":"2023-08-09T13:34:02.115845+0000","last_active":"2023-08-09T13:34:08.371386+0000","last_peered":"2023-08-09T13:34:08.371386+0000","last_clean":"2023-08-09T13:34:08.371386+0000","last_became_active":"2023-08-09T13:34:02.115519+0000","last_became_peered":"2023-08-09T13:34:02.115519+0000","last_unstale":"2023-08-09T13:34:08.371386+0000","last_undegraded":"2023-08-09T13:34:08.371386+0000","last_fullsized":"2023-08-09T13:34:08.371386+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:34:01.052721+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:34:01.052721+0000","last_clean_scrub_stamp":"2023-08-09T13:34:01.052721+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T22:00:42.967697+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.b","version":"0'0","reported_seq":13,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-08-09T13:34:08.211258+0000","last_change":"2023-08-09T13:34:07.245045+0000","last_active":"2023-08-09T13:34:08.211258+0000","last_peered":"2023-08-09T13:34:08.211258+0000","last_clean":"2023-08-09T13:34:08.211258+0000","last_became_active":"2023-08-09T13:34:07.242923+0000","last_became_peered":"2023-08-09T13:34:07.242923+0000","last_unstale":"2023-08-09T13:34:08.211258+0000","last_undegraded":"2023-08-09T13:34:08.211258+0000","last_fullsized":"2023-08-09T13:34:08.211258+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:34:06.173547+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:34:06.173547+0000","last_clean_scrub_stamp":"2023-08-09T13:34:06.173547+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T20:20:56.644233+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,5,6],"acting":[2,0,5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-08-09T13:34:08.369085+0000","last_change":"2023-08-09T13:34:03.157124+0000","last_active":"2023-08-09T13:34:08.369085+0000","last_peered":"2023-08-09T13:34:08.369085+0000","last_clean":"2023-08-09T13:34:08.369085+0000","last_became_active":"2023-08-09T13:34:03.156039+0000","last_became_peered":"2023-08-09T13:34:03.156039+0000","last_unstale":"2023-08-09T13:34:08.369085+0000","last_undegraded":"2023-08-09T13:34:08.369085+0000","last_fullsized":"2023-08-09T13:34:08.369085+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:34:02.064154+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:34:02.064154+0000","last_clean_scrub_stamp":"2023-08-09T13:34:02.064154+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T17:18:45.337355+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.e","version":"19'2","reported_seq":21,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-08-09T13:34:08.371732+0000","last_change":"2023-08-09T13:34:02.116789+0000","last_active":"2023-08-09T13:34:08.371732+0000","last_peered":"2023-08-09T13:34:08.371732+0000","last_clean":"2023-08-09T13:34:08.371732+0000","last_became_active":"2023-08-09T13:34:02.116543+0000","last_became_peered":"2023-08-09T13:34:02.116543+0000","last_unstale":"2023-08-09T13:34:08.371732+0000","last_undegraded":"2023-08-09T13:34:08.371732+0000","last_fullsized":"2023-08-09T13:34:08.371732+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:34:01.052721+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:34:01.052721+0000","last_clean_scrub_stamp":"2023-08-09T13:34:01.052721+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T21:52:55.252712+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.8","version":"0'0","reported_seq":13,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-08-09T13:34:08.211204+0000","last_change":"2023-08-09T13:34:07.241851+0000","last_active":"2023-08-09T13:34:08.211204+0000","last_peered":"2023-08-09T13:34:08.211204+0000","last_clean":"2023-08-09T13:34:08.211204+0000","last_became_active":"2023-08-09T13:34:07.241329+0000","last_became_peered":"2023-08-09T13:34:07.241329+0000","last_unstale":"2023-08-09T13:34:08.211204+0000","last_undegraded":"2023-08-09T13:34:08.211204+0000","last_fullsized":"2023-08-09T13:34:08.211204+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:34:06.173547+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:34:06.173547+0000","last_clean_scrub_stamp":"2023-08-09T13:34:06.173547+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T20:54:41.878672+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,1,5],"acting":[2,0,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":17,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-08-09T13:34:08.368062+0000","last_change":"2023-08-09T13:34:03.158365+0000","last_active":"2023-08-09T13:34:08.368062+0000","last_peered":"2023-08-09T13:34:08.368062+0000","last_clean":"2023-08-09T13:34:08.368062+0000","last_became_active":"2023-08-09T13:34:03.158042+0000","last_became_peered":"2023-08-09T13:34:03.158042+0000","last_unstale":"2023-08-09T13:34:08.368062+0000","last_undegraded":"2023-08-09T13:34:08.368062+0000","last_fullsized":"2023-08-09T13:34:08.368062+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:34:02.064154+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:34:02.064154+0000","last_clean_scrub_stamp":"2023-08-09T13:34:02.064154+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T22:36:24.124090+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.f","version":"19'1","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-08-09T13:34:08.373105+0000","last_change":"2023-08-09T13:34:02.116130+0000","last_active":"2023-08-09T13:34:08.373105+0000","last_peered":"2023-08-09T13:34:08.373105+0000","last_clean":"2023-08-09T13:34:08.373105+0000","last_became_active":"2023-08-09T13:34:02.115758+0000","last_became_peered":"2023-08-09T13:34:02.115758+0000","last_unstale":"2023-08-09T13:34:08.373105+0000","last_undegraded":"2023-08-09T13:34:08.373105+0000","last_fullsized":"2023-08-09T13:34:08.373105+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:34:01.052721+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:34:01.052721+0000","last_clean_scrub_stamp":"2023-08-09T13:34:01.052721+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-11T00:08:17.437427+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.9","version":"0'0","reported_seq":13,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-08-09T13:34:08.369072+0000","last_change":"2023-08-09T13:34:07.241507+0000","last_active":"2023-08-09T13:34:08.369072+0000","last_peered":"2023-08-09T13:34:08.369072+0000","last_clean":"2023-08-09T13:34:08.369072+0000","last_became_active":"2023-08-09T13:34:07.240810+0000","last_became_peered":"2023-08-09T13:34:07.240810+0000","last_unstale":"2023-08-09T13:34:08.369072+0000","last_undegraded":"2023-08-09T13:34:08.369072+0000","last_fullsized":"2023-08-09T13:34:08.369072+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:34:06.173547+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:34:06.173547+0000","last_clean_scrub_stamp":"2023-08-09T13:34:06.173547+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T17:46:27.148600+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,3,4],"acting":[7,6,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.16","version":"0'0","reported_seq":13,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-08-09T13:34:08.370805+0000","last_change":"2023-08-09T13:34:07.238069+0000","last_active":"2023-08-09T13:34:08.370805+0000","last_peered":"2023-08-09T13:34:08.370805+0000","last_clean":"2023-08-09T13:34:08.370805+0000","last_became_active":"2023-08-09T13:34:07.237560+0000","last_became_peered":"2023-08-09T13:34:07.237560+0000","last_unstale":"2023-08-09T13:34:08.370805+0000","last_undegraded":"2023-08-09T13:34:08.370805+0000","last_fullsized":"2023-08-09T13:34:08.370805+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:34:06.173547+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:34:06.173547+0000","last_clean_scrub_stamp":"2023-08-09T13:34:06.173547+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T17:46:27.148600+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,1,6],"acting":[5,3,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.10","version":"19'8","reported_seq":27,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-08-09T13:34:08.211427+0000","last_change":"2023-08-09T13:34:02.103042+0000","last_active":"2023-08-09T13:34:08.211427+0000","last_peered":"2023-08-09T13:34:08.211427+0000","last_clean":"2023-08-09T13:34:08.211427+0000","last_became_active":"2023-08-09T13:34:02.102730+0000","last_became_peered":"2023-08-09T13:34:02.102730+0000","last_unstale":"2023-08-09T13:34:08.211427+0000","last_undegraded":"2023-08-09T13:34:08.211427+0000","last_fullsized":"2023-08-09T13:34:08.211427+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:34:01.052721+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:34:01.052721+0000","last_clean_scrub_stamp":"2023-08-09T13:34:01.052721+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T19:01:23.853811+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-08-09T13:34:08.369924+0000","last_change":"2023-08-09T13:34:03.163917+0000","last_active":"2023-08-09T13:34:08.369924+0000","last_peered":"2023-08-09T13:34:08.369924+0000","last_clean":"2023-08-09T13:34:08.369924+0000","last_became_active":"2023-08-09T13:34:03.163598+0000","last_became_peered":"2023-08-09T13:34:03.163598+0000","last_unstale":"2023-08-09T13:34:08.369924+0000","last_undegraded":"2023-08-09T13:34:08.369924+0000","last_fullsized":"2023-08-09T13:34:08.369924+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:34:02.064154+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:34:02.064154+0000","last_clean_scrub_stamp":"2023-08-09T13:34:02.064154+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T20:01:36.200073+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.17","version":"0'0","reported_seq":13,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-08-09T13:34:08.369281+0000","last_change":"2023-08-09T13:34:07.249196+0000","last_active":"2023-08-09T13:34:08.369281+0000","last_peered":"2023-08-09T13:34:08.369281+0000","last_clean":"2023-08-09T13:34:08.369281+0000","last_became_active":"2023-08-09T13:34:07.248758+0000","last_became_peered":"2023-08-09T13:34:07.248758+0000","last_unstale":"2023-08-09T13:34:08.369281+0000","last_undegraded":"2023-08-09T13:34:08.369281+0000","last_fullsized":"2023-08-09T13:34:08.369281+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:34:06.173547+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:34:06.173547+0000","last_clean_scrub_stamp":"2023-08-09T13:34:06.173547+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T19:50:51.608237+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,7,0],"acting":[3,1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-08-09T13:34:08.369394+0000","last_change":"2023-08-09T13:34:02.112054+0000","last_active":"2023-08-09T13:34:08.369394+0000","last_peered":"2023-08-09T13:34:08.369394+0000","last_clean":"2023-08-09T13:34:08.369394+0000","last_became_active":"2023-08-09T13:34:02.111753+0000","last_became_peered":"2023-08-09T13:34:02.111753+0000","last_unstale":"2023-08-09T13:34:08.369394+0000","last_undegraded":"2023-08-09T13:34:08.369394+0000","last_fullsized":"2023-08-09T13:34:08.369394+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:34:01.052721+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:34:01.052721+0000","last_clean_scrub_stamp":"2023-08-09T13:34:01.052721+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T17:08:06.492548+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":17,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-08-09T13:34:08.208919+0000","last_change":"2023-08-09T13:34:03.151791+0000","last_active":"2023-08-09T13:34:08.208919+0000","last_peered":"2023-08-09T13:34:08.208919+0000","last_clean":"2023-08-09T13:34:08.208919+0000","last_became_active":"2023-08-09T13:34:03.151265+0000","last_became_peered":"2023-08-09T13:34:03.151265+0000","last_unstale":"2023-08-09T13:34:08.208919+0000","last_undegraded":"2023-08-09T13:34:08.208919+0000","last_fullsized":"2023-08-09T13:34:08.208919+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:34:02.064154+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:34:02.064154+0000","last_clean_scrub_stamp":"2023-08-09T13:34:02.064154+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T19:22:08.597283+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-08-09T13:34:08.207862+0000","last_change":"2023-08-09T13:34:02.109334+0000","last_active":"2023-08-09T13:34:08.207862+0000","last_peered":"2023-08-09T13:34:08.207862+0000","last_clean":"2023-08-09T13:34:08.207862+0000","last_became_active":"2023-08-09T13:34:02.108390+0000","last_became_peered":"2023-08-09T13:34:02.108390+0000","last_unstale":"2023-08-09T13:34:08.207862+0000","last_undegraded":"2023-08-09T13:34:08.207862+0000","last_fullsized":"2023-08-09T13:34:08.207862+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:34:01.052721+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:34:01.052721+0000","last_clean_scrub_stamp":"2023-08-09T13:34:01.052721+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T22:36:23.112658+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.15","version":"0'0","reported_seq":17,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-08-09T13:34:08.370140+0000","last_change":"2023-08-09T13:34:03.150611+0000","last_active":"2023-08-09T13:34:08.370140+0000","last_peered":"2023-08-09T13:34:08.370140+0000","last_clean":"2023-08-09T13:34:08.370140+0000","last_became_active":"2023-08-09T13:34:03.150016+0000","last_became_peered":"2023-08-09T13:34:03.150016+0000","last_unstale":"2023-08-09T13:34:08.370140+0000","last_undegraded":"2023-08-09T13:34:08.370140+0000","last_fullsized":"2023-08-09T13:34:08.370140+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:34:02.064154+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:34:02.064154+0000","last_clean_scrub_stamp":"2023-08-09T13:34:02.064154+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T21:35:01.466554+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.14","version":"0'0","reported_seq":13,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-08-09T13:34:08.369980+0000","last_change":"2023-08-09T13:34:07.251464+0000","last_active":"2023-08-09T13:34:08.369980+0000","last_peered":"2023-08-09T13:34:08.369980+0000","last_clean":"2023-08-09T13:34:08.369980+0000","last_became_active":"2023-08-09T13:34:07.251035+0000","last_became_peered":"2023-08-09T13:34:07.251035+0000","last_unstale":"2023-08-09T13:34:08.369980+0000","last_undegraded":"2023-08-09T13:34:08.369980+0000","last_fullsized":"2023-08-09T13:34:08.369980+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:34:06.173547+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:34:06.173547+0000","last_clean_scrub_stamp":"2023-08-09T13:34:06.173547+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T15:06:44.996627+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,2,1],"acting":[3,7,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.13","version":"19'2","reported_seq":21,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-08-09T13:34:08.370349+0000","last_change":"2023-08-09T13:34:02.115081+0000","last_active":"2023-08-09T13:34:08.370349+0000","last_peered":"2023-08-09T13:34:08.370349+0000","last_clean":"2023-08-09T13:34:08.370349+0000","last_became_active":"2023-08-09T13:34:02.114825+0000","last_became_peered":"2023-08-09T13:34:02.114825+0000","last_unstale":"2023-08-09T13:34:08.370349+0000","last_undegraded":"2023-08-09T13:34:08.370349+0000","last_fullsized":"2023-08-09T13:34:08.370349+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:34:01.052721+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:34:01.052721+0000","last_clean_scrub_stamp":"2023-08-09T13:34:01.052721+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T22:51:10.190023+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.14","version":"0'0","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-08-09T13:34:08.370421+0000","last_change":"2023-08-09T13:34:03.154171+0000","last_active":"2023-08-09T13:34:08.370421+0000","last_peered":"2023-08-09T13:34:08.370421+0000","last_clean":"2023-08-09T13:34:08.370421+0000","last_became_active":"2023-08-09T13:34:03.150789+0000","last_became_peered":"2023-08-09T13:34:03.150789+0000","last_unstale":"2023-08-09T13:34:08.370421+0000","last_undegraded":"2023-08-09T13:34:08.370421+0000","last_fullsized":"2023-08-09T13:34:08.370421+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:34:02.064154+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:34:02.064154+0000","last_clean_scrub_stamp":"2023-08-09T13:34:02.064154+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T17:18:45.337355+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.15","version":"0'0","reported_seq":13,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-08-09T13:34:08.369475+0000","last_change":"2023-08-09T13:34:07.241401+0000","last_active":"2023-08-09T13:34:08.369475+0000","last_peered":"2023-08-09T13:34:08.369475+0000","last_clean":"2023-08-09T13:34:08.369475+0000","last_became_active":"2023-08-09T13:34:07.240937+0000","last_became_peered":"2023-08-09T13:34:07.240937+0000","last_unstale":"2023-08-09T13:34:08.369475+0000","last_undegraded":"2023-08-09T13:34:08.369475+0000","last_fullsized":"2023-08-09T13:34:08.369475+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:34:06.173547+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:34:06.173547+0000","last_clean_scrub_stamp":"2023-08-09T13:34:06.173547+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T18:50:25.863045+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,3,0],"acting":[5,1,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.20","version":"0'0","reported_seq":17,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-08-09T13:34:08.210886+0000","last_change":"2023-08-09T13:34:03.157698+0000","last_active":"2023-08-09T13:34:08.210886+0000","last_peered":"2023-08-09T13:34:08.210886+0000","last_clean":"2023-08-09T13:34:08.210886+0000","last_became_active":"2023-08-09T13:34:03.153351+0000","last_became_peered":"2023-08-09T13:34:03.153351+0000","last_unstale":"2023-08-09T13:34:08.210886+0000","last_undegraded":"2023-08-09T13:34:08.210886+0000","last_fullsized":"2023-08-09T13:34:08.210886+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:34:02.064154+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:34:02.064154+0000","last_clean_scrub_stamp":"2023-08-09T13:34:02.064154+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T19:40:06.763332+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.21","version":"0'0","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-08-09T13:34:08.368201+0000","last_change":"2023-08-09T13:34:03.150065+0000","last_active":"2023-08-09T13:34:08.368201+0000","last_peered":"2023-08-09T13:34:08.368201+0000","last_clean":"2023-08-09T13:34:08.368201+0000","last_became_active":"2023-08-09T13:34:03.149650+0000","last_became_peered":"2023-08-09T13:34:03.149650+0000","last_unstale":"2023-08-09T13:34:08.368201+0000","last_undegraded":"2023-08-09T13:34:08.368201+0000","last_fullsized":"2023-08-09T13:34:08.368201+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:34:02.064154+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:34:02.064154+0000","last_clean_scrub_stamp":"2023-08-09T13:34:02.064154+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T13:47:48.941725+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.22","version":"0'0","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-08-09T13:34:08.372000+0000","last_change":"2023-08-09T13:34:03.152515+0000","last_active":"2023-08-09T13:34:08.372000+0000","last_peered":"2023-08-09T13:34:08.372000+0000","last_clean":"2023-08-09T13:34:08.372000+0000","last_became_active":"2023-08-09T13:34:03.152123+0000","last_became_peered":"2023-08-09T13:34:03.152123+0000","last_unstale":"2023-08-09T13:34:08.372000+0000","last_undegraded":"2023-08-09T13:34:08.372000+0000","last_fullsized":"2023-08-09T13:34:08.372000+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:34:02.064154+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:34:02.064154+0000","last_clean_scrub_stamp":"2023-08-09T13:34:02.064154+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T15:53:52.598510+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.23","version":"0'0","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-08-09T13:34:08.369171+0000","last_change":"2023-08-09T13:34:03.159919+0000","last_active":"2023-08-09T13:34:08.369171+0000","last_peered":"2023-08-09T13:34:08.369171+0000","last_clean":"2023-08-09T13:34:08.369171+0000","last_became_active":"2023-08-09T13:34:03.154344+0000","last_became_peered":"2023-08-09T13:34:03.154344+0000","last_unstale":"2023-08-09T13:34:08.369171+0000","last_undegraded":"2023-08-09T13:34:08.369171+0000","last_fullsized":"2023-08-09T13:34:08.369171+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:34:02.064154+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:34:02.064154+0000","last_clean_scrub_stamp":"2023-08-09T13:34:02.064154+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T22:33:11.246239+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.24","version":"0'0","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-08-09T13:34:08.371698+0000","last_change":"2023-08-09T13:34:03.164662+0000","last_active":"2023-08-09T13:34:08.371698+0000","last_peered":"2023-08-09T13:34:08.371698+0000","last_clean":"2023-08-09T13:34:08.371698+0000","last_became_active":"2023-08-09T13:34:03.163711+0000","last_became_peered":"2023-08-09T13:34:03.163711+0000","last_unstale":"2023-08-09T13:34:08.371698+0000","last_undegraded":"2023-08-09T13:34:08.371698+0000","last_fullsized":"2023-08-09T13:34:08.371698+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:34:02.064154+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:34:02.064154+0000","last_clean_scrub_stamp":"2023-08-09T13:34:02.064154+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T23:52:27.738385+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.25","version":"0'0","reported_seq":17,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-08-09T13:34:08.369863+0000","last_change":"2023-08-09T13:34:03.175502+0000","last_active":"2023-08-09T13:34:08.369863+0000","last_peered":"2023-08-09T13:34:08.369863+0000","last_clean":"2023-08-09T13:34:08.369863+0000","last_became_active":"2023-08-09T13:34:03.175192+0000","last_became_peered":"2023-08-09T13:34:03.175192+0000","last_unstale":"2023-08-09T13:34:08.369863+0000","last_undegraded":"2023-08-09T13:34:08.369863+0000","last_fullsized":"2023-08-09T13:34:08.369863+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:34:02.064154+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:34:02.064154+0000","last_clean_scrub_stamp":"2023-08-09T13:34:02.064154+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T19:22:08.597283+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.26","version":"0'0","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-08-09T13:34:08.372808+0000","last_change":"2023-08-09T13:34:03.155984+0000","last_active":"2023-08-09T13:34:08.372808+0000","last_peered":"2023-08-09T13:34:08.372808+0000","last_clean":"2023-08-09T13:34:08.372808+0000","last_became_active":"2023-08-09T13:34:03.155686+0000","last_became_peered":"2023-08-09T13:34:03.155686+0000","last_unstale":"2023-08-09T13:34:08.372808+0000","last_undegraded":"2023-08-09T13:34:08.372808+0000","last_fullsized":"2023-08-09T13:34:08.372808+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:34:02.064154+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:34:02.064154+0000","last_clean_scrub_stamp":"2023-08-09T13:34:02.064154+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T15:43:44.075578+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.27","version":"0'0","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-08-09T13:34:08.372180+0000","last_change":"2023-08-09T13:34:03.154003+0000","last_active":"2023-08-09T13:34:08.372180+0000","last_peered":"2023-08-09T13:34:08.372180+0000","last_clean":"2023-08-09T13:34:08.372180+0000","last_became_active":"2023-08-09T13:34:03.153707+0000","last_became_peered":"2023-08-09T13:34:03.153707+0000","last_unstale":"2023-08-09T13:34:08.372180+0000","last_undegraded":"2023-08-09T13:34:08.372180+0000","last_fullsized":"2023-08-09T13:34:08.372180+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:34:02.064154+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:34:02.064154+0000","last_clean_scrub_stamp":"2023-08-09T13:34:02.064154+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T17:08:07.503980+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.28","version":"0'0","reported_seq":17,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-08-09T13:34:08.368537+0000","last_change":"2023-08-09T13:34:03.170907+0000","last_active":"2023-08-09T13:34:08.368537+0000","last_peered":"2023-08-09T13:34:08.368537+0000","last_clean":"2023-08-09T13:34:08.368537+0000","last_became_active":"2023-08-09T13:34:03.162204+0000","last_became_peered":"2023-08-09T13:34:03.162204+0000","last_unstale":"2023-08-09T13:34:08.368537+0000","last_undegraded":"2023-08-09T13:34:08.368537+0000","last_fullsized":"2023-08-09T13:34:08.368537+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:34:02.064154+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:34:02.064154+0000","last_clean_scrub_stamp":"2023-08-09T13:34:02.064154+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T22:30:06.459673+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.29","version":"0'0","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-08-09T13:34:08.368673+0000","last_change":"2023-08-09T13:34:03.144064+0000","last_active":"2023-08-09T13:34:08.368673+0000","last_peered":"2023-08-09T13:34:08.368673+0000","last_clean":"2023-08-09T13:34:08.368673+0000","last_became_active":"2023-08-09T13:34:03.143612+0000","last_became_peered":"2023-08-09T13:34:03.143612+0000","last_unstale":"2023-08-09T13:34:08.368673+0000","last_undegraded":"2023-08-09T13:34:08.368673+0000","last_fullsized":"2023-08-09T13:34:08.368673+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:34:02.064154+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:34:02.064154+0000","last_clean_scrub_stamp":"2023-08-09T13:34:02.064154+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T18:50:26.789814+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.2a","version":"0'0","reported_seq":17,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-08-09T13:34:08.208605+0000","last_change":"2023-08-09T13:34:03.140127+0000","last_active":"2023-08-09T13:34:08.208605+0000","last_peered":"2023-08-09T13:34:08.208605+0000","last_clean":"2023-08-09T13:34:08.208605+0000","last_became_active":"2023-08-09T13:34:03.139616+0000","last_became_peered":"2023-08-09T13:34:03.139616+0000","last_unstale":"2023-08-09T13:34:08.208605+0000","last_undegraded":"2023-08-09T13:34:08.208605+0000","last_fullsized":"2023-08-09T13:34:08.208605+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:34:02.064154+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:34:02.064154+0000","last_clean_scrub_stamp":"2023-08-09T13:34:02.064154+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T18:50:21.753652+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.2b","version":"0'0","reported_seq":17,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-08-09T13:34:08.369974+0000","last_change":"2023-08-09T13:34:03.170969+0000","last_active":"2023-08-09T13:34:08.369974+0000","last_peered":"2023-08-09T13:34:08.369974+0000","last_clean":"2023-08-09T13:34:08.369974+0000","last_became_active":"2023-08-09T13:34:03.162334+0000","last_became_peered":"2023-08-09T13:34:03.162334+0000","last_unstale":"2023-08-09T13:34:08.369974+0000","last_undegraded":"2023-08-09T13:34:08.369974+0000","last_fullsized":"2023-08-09T13:34:08.369974+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:34:02.064154+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:34:02.064154+0000","last_clean_scrub_stamp":"2023-08-09T13:34:02.064154+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T23:34:54.506258+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.2c","version":"0'0","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-08-09T13:34:08.372790+0000","last_change":"2023-08-09T13:34:03.149273+0000","last_active":"2023-08-09T13:34:08.372790+0000","last_peered":"2023-08-09T13:34:08.372790+0000","last_clean":"2023-08-09T13:34:08.372790+0000","last_became_active":"2023-08-09T13:34:03.148460+0000","last_became_peered":"2023-08-09T13:34:03.148460+0000","last_unstale":"2023-08-09T13:34:08.372790+0000","last_undegraded":"2023-08-09T13:34:08.372790+0000","last_fullsized":"2023-08-09T13:34:08.372790+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:34:02.064154+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:34:02.064154+0000","last_clean_scrub_stamp":"2023-08-09T13:34:02.064154+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-11T01:12:24.927676+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.2d","version":"0'0","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-08-09T13:34:08.211794+0000","last_change":"2023-08-09T13:34:03.160727+0000","last_active":"2023-08-09T13:34:08.211794+0000","last_peered":"2023-08-09T13:34:08.211794+0000","last_clean":"2023-08-09T13:34:08.211794+0000","last_became_active":"2023-08-09T13:34:03.160406+0000","last_became_peered":"2023-08-09T13:34:03.160406+0000","last_unstale":"2023-08-09T13:34:08.211794+0000","last_undegraded":"2023-08-09T13:34:08.211794+0000","last_fullsized":"2023-08-09T13:34:08.211794+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:34:02.064154+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:34:02.064154+0000","last_clean_scrub_stamp":"2023-08-09T13:34:02.064154+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T20:28:43.534176+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.2e","version":"0'0","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-08-09T13:34:08.373221+0000","last_change":"2023-08-09T13:34:03.154714+0000","last_active":"2023-08-09T13:34:08.373221+0000","last_peered":"2023-08-09T13:34:08.373221+0000","last_clean":"2023-08-09T13:34:08.373221+0000","last_became_active":"2023-08-09T13:34:03.154384+0000","last_became_peered":"2023-08-09T13:34:03.154384+0000","last_unstale":"2023-08-09T13:34:08.373221+0000","last_undegraded":"2023-08-09T13:34:08.373221+0000","last_fullsized":"2023-08-09T13:34:08.373221+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:34:02.064154+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:34:02.064154+0000","last_clean_scrub_stamp":"2023-08-09T13:34:02.064154+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T16:21:31.389815+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.2f","version":"0'0","reported_seq":17,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-08-09T13:34:08.209885+0000","last_change":"2023-08-09T13:34:03.144822+0000","last_active":"2023-08-09T13:34:08.209885+0000","last_peered":"2023-08-09T13:34:08.209885+0000","last_clean":"2023-08-09T13:34:08.209885+0000","last_became_active":"2023-08-09T13:34:03.144470+0000","last_became_peered":"2023-08-09T13:34:03.144470+0000","last_unstale":"2023-08-09T13:34:08.209885+0000","last_undegraded":"2023-08-09T13:34:08.209885+0000","last_fullsized":"2023-08-09T13:34:08.209885+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:34:02.064154+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:34:02.064154+0000","last_clean_scrub_stamp":"2023-08-09T13:34:02.064154+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T21:18:19.551549+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.30","version":"0'0","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-08-09T13:34:08.370238+0000","last_change":"2023-08-09T13:34:03.140680+0000","last_active":"2023-08-09T13:34:08.370238+0000","last_peered":"2023-08-09T13:34:08.370238+0000","last_clean":"2023-08-09T13:34:08.370238+0000","last_became_active":"2023-08-09T13:34:03.139105+0000","last_became_peered":"2023-08-09T13:34:03.139105+0000","last_unstale":"2023-08-09T13:34:08.370238+0000","last_undegraded":"2023-08-09T13:34:08.370238+0000","last_fullsized":"2023-08-09T13:34:08.370238+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:34:02.064154+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:34:02.064154+0000","last_clean_scrub_stamp":"2023-08-09T13:34:02.064154+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T18:16:48.383340+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.31","version":"0'0","reported_seq":17,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-08-09T13:34:08.369342+0000","last_change":"2023-08-09T13:34:03.174569+0000","last_active":"2023-08-09T13:34:08.369342+0000","last_peered":"2023-08-09T13:34:08.369342+0000","last_clean":"2023-08-09T13:34:08.369342+0000","last_became_active":"2023-08-09T13:34:03.173450+0000","last_became_peered":"2023-08-09T13:34:03.173450+0000","last_unstale":"2023-08-09T13:34:08.369342+0000","last_undegraded":"2023-08-09T13:34:08.369342+0000","last_fullsized":"2023-08-09T13:34:08.369342+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:34:02.064154+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:34:02.064154+0000","last_clean_scrub_stamp":"2023-08-09T13:34:02.064154+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T13:44:50.765363+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.32","version":"0'0","reported_seq":17,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-08-09T13:34:08.368679+0000","last_change":"2023-08-09T13:34:03.156031+0000","last_active":"2023-08-09T13:34:08.368679+0000","last_peered":"2023-08-09T13:34:08.368679+0000","last_clean":"2023-08-09T13:34:08.368679+0000","last_became_active":"2023-08-09T13:34:03.155720+0000","last_became_peered":"2023-08-09T13:34:03.155720+0000","last_unstale":"2023-08-09T13:34:08.368679+0000","last_undegraded":"2023-08-09T13:34:08.368679+0000","last_fullsized":"2023-08-09T13:34:08.368679+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:34:02.064154+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:34:02.064154+0000","last_clean_scrub_stamp":"2023-08-09T13:34:02.064154+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T20:46:15.647419+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.33","version":"0'0","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-08-09T13:34:08.211563+0000","last_change":"2023-08-09T13:34:03.159410+0000","last_active":"2023-08-09T13:34:08.211563+0000","last_peered":"2023-08-09T13:34:08.211563+0000","last_clean":"2023-08-09T13:34:08.211563+0000","last_became_active":"2023-08-09T13:34:03.159006+0000","last_became_peered":"2023-08-09T13:34:03.159006+0000","last_unstale":"2023-08-09T13:34:08.211563+0000","last_undegraded":"2023-08-09T13:34:08.211563+0000","last_fullsized":"2023-08-09T13:34:08.211563+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:34:02.064154+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:34:02.064154+0000","last_clean_scrub_stamp":"2023-08-09T13:34:02.064154+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T21:55:49.311446+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.34","version":"0'0","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-08-09T13:34:08.212297+0000","last_change":"2023-08-09T13:34:03.165655+0000","last_active":"2023-08-09T13:34:08.212297+0000","last_peered":"2023-08-09T13:34:08.212297+0000","last_clean":"2023-08-09T13:34:08.212297+0000","last_became_active":"2023-08-09T13:34:03.165308+0000","last_became_peered":"2023-08-09T13:34:03.165308+0000","last_unstale":"2023-08-09T13:34:08.212297+0000","last_undegraded":"2023-08-09T13:34:08.212297+0000","last_fullsized":"2023-08-09T13:34:08.212297+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:34:02.064154+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:34:02.064154+0000","last_clean_scrub_stamp":"2023-08-09T13:34:02.064154+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T21:47:59.693735+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.35","version":"0'0","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-08-09T13:34:08.372839+0000","last_change":"2023-08-09T13:34:03.138842+0000","last_active":"2023-08-09T13:34:08.372839+0000","last_peered":"2023-08-09T13:34:08.372839+0000","last_clean":"2023-08-09T13:34:08.372839+0000","last_became_active":"2023-08-09T13:34:03.138554+0000","last_became_peered":"2023-08-09T13:34:03.138554+0000","last_unstale":"2023-08-09T13:34:08.372839+0000","last_undegraded":"2023-08-09T13:34:08.372839+0000","last_fullsized":"2023-08-09T13:34:08.372839+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:34:02.064154+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:34:02.064154+0000","last_clean_scrub_stamp":"2023-08-09T13:34:02.064154+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T23:05:06.664154+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.36","version":"0'0","reported_seq":17,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-08-09T13:34:08.371821+0000","last_change":"2023-08-09T13:34:03.148574+0000","last_active":"2023-08-09T13:34:08.371821+0000","last_peered":"2023-08-09T13:34:08.371821+0000","last_clean":"2023-08-09T13:34:08.371821+0000","last_became_active":"2023-08-09T13:34:03.148292+0000","last_became_peered":"2023-08-09T13:34:03.148292+0000","last_unstale":"2023-08-09T13:34:08.371821+0000","last_undegraded":"2023-08-09T13:34:08.371821+0000","last_fullsized":"2023-08-09T13:34:08.371821+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:34:02.064154+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:34:02.064154+0000","last_clean_scrub_stamp":"2023-08-09T13:34:02.064154+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T13:44:50.765363+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.37","version":"0'0","reported_seq":17,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-08-09T13:34:08.209550+0000","last_change":"2023-08-09T13:34:03.152770+0000","last_active":"2023-08-09T13:34:08.209550+0000","last_peered":"2023-08-09T13:34:08.209550+0000","last_clean":"2023-08-09T13:34:08.209550+0000","last_became_active":"2023-08-09T13:34:03.151703+0000","last_became_peered":"2023-08-09T13:34:03.151703+0000","last_unstale":"2023-08-09T13:34:08.209550+0000","last_undegraded":"2023-08-09T13:34:08.209550+0000","last_fullsized":"2023-08-09T13:34:08.209550+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:34:02.064154+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:34:02.064154+0000","last_clean_scrub_stamp":"2023-08-09T13:34:02.064154+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T17:46:23.039207+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.38","version":"0'0","reported_seq":17,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-08-09T13:34:08.367726+0000","last_change":"2023-08-09T13:34:03.146016+0000","last_active":"2023-08-09T13:34:08.367726+0000","last_peered":"2023-08-09T13:34:08.367726+0000","last_clean":"2023-08-09T13:34:08.367726+0000","last_became_active":"2023-08-09T13:34:03.145766+0000","last_became_peered":"2023-08-09T13:34:03.145766+0000","last_unstale":"2023-08-09T13:34:08.367726+0000","last_undegraded":"2023-08-09T13:34:08.367726+0000","last_fullsized":"2023-08-09T13:34:08.367726+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:34:02.064154+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:34:02.064154+0000","last_clean_scrub_stamp":"2023-08-09T13:34:02.064154+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T13:51:34.177397+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.39","version":"0'0","reported_seq":17,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-08-09T13:34:08.368967+0000","last_change":"2023-08-09T13:34:03.173368+0000","last_active":"2023-08-09T13:34:08.368967+0000","last_peered":"2023-08-09T13:34:08.368967+0000","last_clean":"2023-08-09T13:34:08.368967+0000","last_became_active":"2023-08-09T13:34:03.173037+0000","last_became_peered":"2023-08-09T13:34:03.173037+0000","last_unstale":"2023-08-09T13:34:08.368967+0000","last_undegraded":"2023-08-09T13:34:08.368967+0000","last_fullsized":"2023-08-09T13:34:08.368967+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:34:02.064154+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:34:02.064154+0000","last_clean_scrub_stamp":"2023-08-09T13:34:02.064154+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T21:11:40.494329+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.3a","version":"0'0","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-08-09T13:34:08.371086+0000","last_change":"2023-08-09T13:34:03.144394+0000","last_active":"2023-08-09T13:34:08.371086+0000","last_peered":"2023-08-09T13:34:08.371086+0000","last_clean":"2023-08-09T13:34:08.371086+0000","last_became_active":"2023-08-09T13:34:03.143958+0000","last_became_peered":"2023-08-09T13:34:03.143958+0000","last_unstale":"2023-08-09T13:34:08.371086+0000","last_undegraded":"2023-08-09T13:34:08.371086+0000","last_fullsized":"2023-08-09T13:34:08.371086+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:34:02.064154+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:34:02.064154+0000","last_clean_scrub_stamp":"2023-08-09T13:34:02.064154+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T19:49:18.157706+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.3b","version":"0'0","reported_seq":17,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-08-09T13:34:08.210114+0000","last_change":"2023-08-09T13:34:03.144695+0000","last_active":"2023-08-09T13:34:08.210114+0000","last_peered":"2023-08-09T13:34:08.210114+0000","last_clean":"2023-08-09T13:34:08.210114+0000","last_became_active":"2023-08-09T13:34:03.144303+0000","last_became_peered":"2023-08-09T13:34:03.144303+0000","last_unstale":"2023-08-09T13:34:08.210114+0000","last_undegraded":"2023-08-09T13:34:08.210114+0000","last_fullsized":"2023-08-09T13:34:08.210114+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:34:02.064154+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:34:02.064154+0000","last_clean_scrub_stamp":"2023-08-09T13:34:02.064154+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T18:07:31.705168+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.3c","version":"0'0","reported_seq":17,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-08-09T13:34:08.212037+0000","last_change":"2023-08-09T13:34:03.151741+0000","last_active":"2023-08-09T13:34:08.212037+0000","last_peered":"2023-08-09T13:34:08.212037+0000","last_clean":"2023-08-09T13:34:08.212037+0000","last_became_active":"2023-08-09T13:34:03.151331+0000","last_became_peered":"2023-08-09T13:34:03.151331+0000","last_unstale":"2023-08-09T13:34:08.212037+0000","last_undegraded":"2023-08-09T13:34:08.212037+0000","last_fullsized":"2023-08-09T13:34:08.212037+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:34:02.064154+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:34:02.064154+0000","last_clean_scrub_stamp":"2023-08-09T13:34:02.064154+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-11T00:09:38.635751+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.3d","version":"0'0","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-08-09T13:34:08.372764+0000","last_change":"2023-08-09T13:34:03.144010+0000","last_active":"2023-08-09T13:34:08.372764+0000","last_peered":"2023-08-09T13:34:08.372764+0000","last_clean":"2023-08-09T13:34:08.372764+0000","last_became_active":"2023-08-09T13:34:03.143673+0000","last_became_peered":"2023-08-09T13:34:03.143673+0000","last_unstale":"2023-08-09T13:34:08.372764+0000","last_undegraded":"2023-08-09T13:34:08.372764+0000","last_fullsized":"2023-08-09T13:34:08.372764+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:34:02.064154+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:34:02.064154+0000","last_clean_scrub_stamp":"2023-08-09T13:34:02.064154+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T16:58:18.499551+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.3e","version":"0'0","reported_seq":17,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-08-09T13:34:08.371258+0000","last_change":"2023-08-09T13:34:03.150894+0000","last_active":"2023-08-09T13:34:08.371258+0000","last_peered":"2023-08-09T13:34:08.371258+0000","last_clean":"2023-08-09T13:34:08.371258+0000","last_became_active":"2023-08-09T13:34:03.149237+0000","last_became_peered":"2023-08-09T13:34:03.149237+0000","last_unstale":"2023-08-09T13:34:08.371258+0000","last_undegraded":"2023-08-09T13:34:08.371258+0000","last_fullsized":"2023-08-09T13:34:08.371258+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:34:02.064154+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:34:02.064154+0000","last_clean_scrub_stamp":"2023-08-09T13:34:02.064154+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T21:18:19.551549+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.3f","version":"0'0","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-08-09T13:34:08.212027+0000","last_change":"2023-08-09T13:34:03.160785+0000","last_active":"2023-08-09T13:34:08.212027+0000","last_peered":"2023-08-09T13:34:08.212027+0000","last_clean":"2023-08-09T13:34:08.212027+0000","last_became_active":"2023-08-09T13:34:03.160452+0000","last_became_peered":"2023-08-09T13:34:03.160452+0000","last_unstale":"2023-08-09T13:34:08.212027+0000","last_undegraded":"2023-08-09T13:34:08.212027+0000","last_fullsized":"2023-08-09T13:34:08.212027+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:34:02.064154+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:34:02.064154+0000","last_clean_scrub_stamp":"2023-08-09T13:34:02.064154+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T18:03:19.093604+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]}],"pool_stats":[{"poolid":5,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":4,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":3,"num_pg":32,"stat_sum":{"num_bytes":4639,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":49,"num_write_kb":19,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":9278,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":43,"ondisk_log_size":43,"up":64,"acting":64,"num_store_stats":8},{"poolid":2,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":8}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574858,"num_pgs":41,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27872,"kb_used_data":856,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343968,"statfs":{"total":96636764160,"available":96608223232,"internally_reserved":0,"allocated":876544,"data_stored":699155,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1781,"internal_metadata":27654411},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.98299999999999998}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.71999999999999997}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.92900000000000005}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.327}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.391}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.9750000000000001}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.859}]}]},{"osd":6,"up_from":13,"seq":55834574858,"num_pgs":34,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27268,"kb_used_data":244,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344572,"statfs":{"total":96636764160,"available":96608841728,"internally_reserved":0,"allocated":249856,"data_stored":103369,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1781,"internal_metadata":27654411},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.3320000000000001}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.101}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.99299999999999999}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.2210000000000001}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.5900000000000001}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.2809999999999999}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.6379999999999999}]}]},{"osd":5,"up_from":13,"seq":55834574858,"num_pgs":39,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27280,"kb_used_data":256,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344560,"statfs":{"total":96636764160,"available":96608829440,"internally_reserved":0,"allocated":262144,"data_stored":105382,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1781,"internal_metadata":27654411},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.97599999999999998}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.323}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.90200000000000002}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.82799999999999996}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.488}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.014}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.369}]}]},{"osd":4,"up_from":13,"seq":55834574858,"num_pgs":50,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27284,"kb_used_data":268,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344556,"statfs":{"total":96636764160,"available":96608825344,"internally_reserved":0,"allocated":274432,"data_stored":108627,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1781,"internal_metadata":27654411},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.92400000000000004}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.3720000000000001}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.3149999999999999}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.284}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.5980000000000001}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.1479999999999999}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.514}]}]},{"osd":3,"up_from":13,"seq":55834574859,"num_pgs":48,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31372,"kb_used_data":252,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":94340468,"statfs":{"total":96636764160,"available":96604639232,"internally_reserved":0,"allocated":258048,"data_stored":104820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1781,"internal_metadata":31848715},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.753}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.80000000000000004}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.90900000000000003}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.361}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.3460000000000001}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.4490000000000001}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.669}]}]},{"osd":2,"up_from":13,"seq":55834574858,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31368,"kb_used_data":248,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":94340472,"statfs":{"total":96636764160,"available":96604643328,"internally_reserved":0,"allocated":253952,"data_stored":104730,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1781,"internal_metadata":31848715},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.871}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.2250000000000001}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.61599999999999999}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.4299999999999999}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.4079999999999999}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.2130000000000001}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.5489999999999999}]}]},{"osd":1,"up_from":13,"seq":55834574858,"num_pgs":49,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31952,"kb_used_data":828,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":94339888,"statfs":{"total":96636764160,"available":96604045312,"internally_reserved":0,"allocated":847872,"data_stored":695098,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1781,"internal_metadata":31848715},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.51000000000000001}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.006}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.82199999999999995}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.073}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.151}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.1799999999999999}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.3500000000000001}]}]},{"osd":0,"up_from":13,"seq":55834574858,"num_pgs":47,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31368,"kb_used_data":248,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":94340472,"statfs":{"total":96636764160,"available":96604643328,"internally_reserved":0,"allocated":253952,"data_stored":103951,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1781,"internal_metadata":31848715},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.69399999999999995}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.59099999999999997}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.55200000000000005}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.93999999999999995}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.89200000000000002}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.2190000000000001}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.1990000000000001}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":3897,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":652,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":4057,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-08-09T13:34:39.673 INFO:tasks.ceph.ceph_manager.ceph:clean! 2023-08-09T13:34:39.673 INFO:tasks.ceph:Waiting until ceph cluster ceph is healthy... 2023-08-09T13:34:39.674 INFO:tasks.ceph.ceph_manager.ceph:wait_until_healthy 2023-08-09T13:34:39.674 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph health --format=json 2023-08-09T13:34:39.929 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:39.927+0000 7efcfec1a640 1 Processor -- start 2023-08-09T13:34:39.930 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:39.927+0000 7efcfec1a640 1 -- start start 2023-08-09T13:34:39.931 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:39.929+0000 7efcfec1a640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7efcf80c8b30 0x7efcf80ad340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:34:39.931 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:39.929+0000 7efcfec1a640 1 -- --> v1:172.21.15.43:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7efcf80cbd90 con 0x7efcf80acdf0 2023-08-09T13:34:39.931 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:39.929+0000 7efcfec1a640 1 -- --> v1:172.21.15.203:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7efcf80ca2b0 con 0x7efcf80c8750 2023-08-09T13:34:39.932 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:39.929+0000 7efcfec1a640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7efcf815e810 con 0x7efcf80c8b30 2023-08-09T13:34:39.933 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:39.929+0000 7efcfe419640 1 --1- >> v1:172.21.15.43:6789/0 conn(0x7efcf80acdf0 0x7efcf8070f00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.43:6789/0 says I am v1:172.21.15.43:35716/0 (socket says 172.21.15.43:35716) 2023-08-09T13:34:39.933 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:39.929+0000 7efcfe419640 1 -- 172.21.15.43:0/1990714114 learned_addr learned my addr 172.21.15.43:0/1990714114 (peer_addr_for_me v1:172.21.15.43:0/0) 2023-08-09T13:34:39.933 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:39.929+0000 7efcfd417640 1 --2- 172.21.15.43:0/1990714114 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7efcf80c8b30 0x7efcf80ad340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:34:39.934 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:39.930+0000 7efcfcc16640 1 -- 172.21.15.43:0/1990714114 <== mon.0 v1:172.21.15.43:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3112457144 0 0) 0x7efcf80cbd90 con 0x7efcf80acdf0 2023-08-09T13:34:39.934 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:39.930+0000 7efcfcc16640 1 -- 172.21.15.43:0/1990714114 --> v1:172.21.15.43:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7efce4003520 con 0x7efcf80acdf0 2023-08-09T13:34:39.934 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:39.930+0000 7efcfcc16640 1 -- 172.21.15.43:0/1990714114 <== mon.1 v1:172.21.15.203:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3970654843 0 0) 0x7efcf80ca2b0 con 0x7efcf80c8750 2023-08-09T13:34:39.934 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:39.930+0000 7efcfcc16640 1 -- 172.21.15.43:0/1990714114 --> v1:172.21.15.203:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7efcf80cbd90 con 0x7efcf80c8750 2023-08-09T13:34:39.934 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:39.930+0000 7efcfcc16640 1 -- 172.21.15.43:0/1990714114 <== mon.0 v1:172.21.15.43:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 731804576 0 0) 0x7efce4003520 con 0x7efcf80acdf0 2023-08-09T13:34:39.934 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:39.930+0000 7efcfcc16640 1 -- 172.21.15.43:0/1990714114 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7efcf80c8b30 msgr2=0x7efcf80ad340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-09T13:34:39.935 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:39.930+0000 7efcfcc16640 1 --2- 172.21.15.43:0/1990714114 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7efcf80c8b30 0x7efcf80ad340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:34:39.935 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:39.930+0000 7efcfcc16640 1 -- 172.21.15.43:0/1990714114 >> v1:172.21.15.203:6789/0 conn(0x7efcf80c8750 legacy=0x7efcf806f410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:34:39.935 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:39.930+0000 7efcfcc16640 1 -- 172.21.15.43:0/1990714114 --> v1:172.21.15.43:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efcf80c9a20 con 0x7efcf80acdf0 2023-08-09T13:34:39.935 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:39.930+0000 7efcfcc16640 1 -- 172.21.15.43:0/1990714114 <== mon.0 v1:172.21.15.43:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1285150740 0 0) 0x7efcf0001970 con 0x7efcf80acdf0 2023-08-09T13:34:39.935 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:39.930+0000 7efcfcc16640 1 -- 172.21.15.43:0/1990714114 >> v1:172.21.15.43:6789/0 conn(0x7efcf80acdf0 legacy=0x7efcf8070f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:34:39.935 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:39.930+0000 7efcfcc16640 1 --2- 172.21.15.43:0/1990714114 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7efce4003b30 0x7efce4003f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:34:39.935 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:39.930+0000 7efcfcc16640 1 --2- 172.21.15.43:0/1990714114 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7efcf80c8b30 0x7efce40047c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:34:39.936 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:39.930+0000 7efcfcc16640 1 --2- 172.21.15.43:0/1990714114 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7efce4004d00 0x7efce4005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:34:39.936 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:39.930+0000 7efcfcc16640 1 -- 172.21.15.43:0/1990714114 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efcf815e810 con 0x7efce4003b30 2023-08-09T13:34:39.936 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:39.930+0000 7efcfcc16640 1 -- 172.21.15.43:0/1990714114 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efce400bb10 con 0x7efce4004d00 2023-08-09T13:34:39.936 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:39.930+0000 7efcfcc16640 1 -- 172.21.15.43:0/1990714114 --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7efce400bca0 con 0x7efcf80c8b30 2023-08-09T13:34:39.936 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:39.931+0000 7efcfdc18640 1 --2- 172.21.15.43:0/1990714114 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7efce4003b30 0x7efce4003f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:34:39.936 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:39.931+0000 7efcfe419640 1 --2- 172.21.15.43:0/1990714114 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7efce4004d00 0x7efce4005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:34:39.937 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:39.931+0000 7efcfd417640 1 --2- 172.21.15.43:0/1990714114 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7efcf80c8b30 0x7efce40047c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:34:39.938 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:39.931+0000 7efcfdc18640 1 -- 172.21.15.43:0/1990714114 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7efcf80c8b30 msgr2=0x7efce40047c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-09T13:34:39.938 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:39.931+0000 7efcfdc18640 1 --2- 172.21.15.43:0/1990714114 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7efcf80c8b30 0x7efce40047c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:34:39.938 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:39.931+0000 7efcfdc18640 1 -- 172.21.15.43:0/1990714114 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7efce4004d00 msgr2=0x7efce4005180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-09T13:34:39.938 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:39.931+0000 7efcfdc18640 1 --2- 172.21.15.43:0/1990714114 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7efce4004d00 0x7efce4005180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:34:39.938 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:39.931+0000 7efcfdc18640 1 -- 172.21.15.43:0/1990714114 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efce4003520 con 0x7efce4003b30 2023-08-09T13:34:39.938 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:39.931+0000 7efcfd417640 1 --2- 172.21.15.43:0/1990714114 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7efcf80c8b30 0x7efce40047c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-08-09T13:34:39.939 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:39.932+0000 7efcfdc18640 1 --2- 172.21.15.43:0/1990714114 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7efce4003b30 0x7efce4003f30 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7efcf4001900 tx=0x7efce40057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=564b0f4fda89bc3d server_cookie=0 in_seq=0 out_seq=0 2023-08-09T13:34:39.939 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:39.932+0000 7efcfcc16640 1 -- 172.21.15.43:0/1990714114 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efcf40052d0 con 0x7efce4003b30 2023-08-09T13:34:39.939 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:39.933+0000 7efcfcc16640 1 -- 172.21.15.43:0/1990714114 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efcf4005430 con 0x7efce4003b30 2023-08-09T13:34:39.939 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:39.933+0000 7efcfcc16640 1 -- 172.21.15.43:0/1990714114 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efcf4003810 con 0x7efce4003b30 2023-08-09T13:34:39.939 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:39.933+0000 7efcfec1a640 1 -- 172.21.15.43:0/1990714114 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7efce4003b30 msgr2=0x7efce4003f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:34:39.939 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:39.933+0000 7efcfec1a640 1 --2- 172.21.15.43:0/1990714114 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7efce4003b30 0x7efce4003f30 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7efcf4001900 tx=0x7efce40057f0 comp rx=0 tx=0).stop 2023-08-09T13:34:39.940 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:39.933+0000 7efcfe419640 1 -- 172.21.15.43:0/1990714114 reap_dead start 2023-08-09T13:34:39.940 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:39.933+0000 7efcfec1a640 1 -- 172.21.15.43:0/1990714114 shutdown_connections 2023-08-09T13:34:39.940 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:39.933+0000 7efcfec1a640 1 -- 172.21.15.43:0/1990714114 >> 172.21.15.43:0/1990714114 conn(0x7efcf80cde90 msgr2=0x7efcf80ce2a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-09T13:34:39.940 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:39.933+0000 7efcfec1a640 1 -- 172.21.15.43:0/1990714114 shutdown_connections 2023-08-09T13:34:39.940 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:39.933+0000 7efcfec1a640 1 -- 172.21.15.43:0/1990714114 wait complete. 2023-08-09T13:34:39.940 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:39.936+0000 7efcfec1a640 1 Processor -- start 2023-08-09T13:34:39.942 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:39.936+0000 7efcfec1a640 1 -- start start 2023-08-09T13:34:39.942 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:39.937+0000 7efcfec1a640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7efce4003b30 0x7efcf814c1e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:34:39.942 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:39.937+0000 7efcfec1a640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7efcf81591c0 0x7efcf80b2e00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:34:39.942 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:39.937+0000 7efcfec1a640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7efcf80c5190 0x7efcf80c5610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:34:39.942 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:39.937+0000 7efcfec1a640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efcf80c9f70 con 0x7efcf81591c0 2023-08-09T13:34:39.943 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:39.937+0000 7efcfec1a640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efcf80b2970 con 0x7efcf80c5190 2023-08-09T13:34:39.943 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:39.937+0000 7efcfec1a640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7efcf80c7d90 con 0x7efce4003b30 2023-08-09T13:34:39.943 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:39.937+0000 7efcfd417640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7efcf81591c0 0x7efcf80b2e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:34:39.943 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:39.937+0000 7efcfd417640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7efcf81591c0 0x7efcf80b2e00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:52014/0 (socket says 172.21.15.43:52014) 2023-08-09T13:34:39.943 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:39.937+0000 7efcfd417640 1 -- 172.21.15.43:0/2615490896 learned_addr learned my addr 172.21.15.43:0/2615490896 (peer_addr_for_me v2:172.21.15.43:0/0) 2023-08-09T13:34:39.943 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:39.938+0000 7efcfe419640 1 --2- 172.21.15.43:0/2615490896 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7efcf80c5190 0x7efcf80c5610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:34:39.943 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:39.938+0000 7efcfdc18640 1 --2- 172.21.15.43:0/2615490896 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7efce4003b30 0x7efcf814c1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:34:39.943 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:39.938+0000 7efcfd417640 1 -- 172.21.15.43:0/2615490896 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7efce4003b30 msgr2=0x7efcf814c1e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:34:39.944 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:39.938+0000 7efcfd417640 1 --2- 172.21.15.43:0/2615490896 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7efce4003b30 0x7efcf814c1e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:34:39.944 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:39.938+0000 7efcfd417640 1 -- 172.21.15.43:0/2615490896 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7efcf80c5190 msgr2=0x7efcf80c5610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:34:39.944 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:39.938+0000 7efcfd417640 1 --2- 172.21.15.43:0/2615490896 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7efcf80c5190 0x7efcf80c5610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:34:39.944 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:39.938+0000 7efcfd417640 1 -- 172.21.15.43:0/2615490896 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efcf80b2640 con 0x7efcf81591c0 2023-08-09T13:34:39.944 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:39.938+0000 7efcfe419640 1 --2- 172.21.15.43:0/2615490896 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7efcf80c5190 0x7efcf80c5610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-08-09T13:34:39.944 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:39.938+0000 7efcfd417640 1 --2- 172.21.15.43:0/2615490896 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7efcf81591c0 0x7efcf80b2e00 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7efce8011630 tx=0x7efce8011af0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-09T13:34:39.944 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:39.938+0000 7efcee7fc640 1 -- 172.21.15.43:0/2615490896 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efce800f920 con 0x7efcf81591c0 2023-08-09T13:34:39.945 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:39.939+0000 7efcee7fc640 1 -- 172.21.15.43:0/2615490896 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efce800fa80 con 0x7efcf81591c0 2023-08-09T13:34:39.945 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:39.939+0000 7efcee7fc640 1 -- 172.21.15.43:0/2615490896 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efce800fe10 con 0x7efcf81591c0 2023-08-09T13:34:39.945 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:39.939+0000 7efcfec1a640 1 -- 172.21.15.43:0/2615490896 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7efcf814c720 con 0x7efcf81591c0 2023-08-09T13:34:39.945 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:39.939+0000 7efcfec1a640 1 -- 172.21.15.43:0/2615490896 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7efcf80c1dd0 con 0x7efcf81591c0 2023-08-09T13:34:39.946 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:39.940+0000 7efcfec1a640 1 -- 172.21.15.43:0/2615490896 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7efcf80702a0 con 0x7efcf81591c0 2023-08-09T13:34:39.946 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:39.941+0000 7efcee7fc640 1 -- 172.21.15.43:0/2615490896 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 5) v1 ==== 88341+0+0 (secure 0 0 0) 0x7efce8015460 con 0x7efcf81591c0 2023-08-09T13:34:39.946 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:39.941+0000 7efcee7fc640 1 --2- 172.21.15.43:0/2615490896 >> [v2:172.21.15.203:6832/830097474,v1:172.21.15.203:6833/830097474] conn(0x7efcdc06a050 0x7efcdc06c510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:34:39.946 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:39.942+0000 7efcee7fc640 1 -- 172.21.15.43:0/2615490896 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6813+0+0 (secure 0 0 0) 0x7efce8089120 con 0x7efcf81591c0 2023-08-09T13:34:39.947 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:39.942+0000 7efcfdc18640 1 --2- 172.21.15.43:0/2615490896 >> [v2:172.21.15.203:6832/830097474,v1:172.21.15.203:6833/830097474] conn(0x7efcdc06a050 0x7efcdc06c510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:34:39.947 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:39.943+0000 7efcfdc18640 1 --2- 172.21.15.43:0/2615490896 >> [v2:172.21.15.203:6832/830097474,v1:172.21.15.203:6833/830097474] conn(0x7efcdc06a050 0x7efcdc06c510 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7efcf4034000 tx=0x7efcf4031000 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-09T13:34:39.948 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:39.946+0000 7efcee7fc640 1 -- 172.21.15.43:0/2615490896 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178066 (secure 0 0 0) 0x7efce8054510 con 0x7efcf81591c0 2023-08-09T13:34:40.172 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:40.169+0000 7efcfec1a640 1 -- 172.21.15.43:0/2615490896 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "health", "format": "json"} v 0) v1 -- 0x7efcf80c9230 con 0x7efcf81591c0 2023-08-09T13:34:40.172 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:40.170+0000 7efcee7fc640 1 -- 172.21.15.43:0/2615490896 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "health", "format": "json"}]=0 v0) v1 ==== 72+0+46 (secure 0 0 0) 0x7efce8057b30 con 0x7efcf81591c0 2023-08-09T13:34:40.173 INFO:teuthology.orchestra.run.smithi043.stdout: 2023-08-09T13:34:40.173 INFO:teuthology.orchestra.run.smithi043.stdout:{"status":"HEALTH_OK","checks":{},"mutes":[]} 2023-08-09T13:34:40.176 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:40.173+0000 7efcfec1a640 1 -- 172.21.15.43:0/2615490896 >> [v2:172.21.15.203:6832/830097474,v1:172.21.15.203:6833/830097474] conn(0x7efcdc06a050 msgr2=0x7efcdc06c510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:34:40.176 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:40.173+0000 7efcfec1a640 1 --2- 172.21.15.43:0/2615490896 >> [v2:172.21.15.203:6832/830097474,v1:172.21.15.203:6833/830097474] conn(0x7efcdc06a050 0x7efcdc06c510 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7efcf4034000 tx=0x7efcf4031000 comp rx=0 tx=0).stop 2023-08-09T13:34:40.176 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:40.173+0000 7efcfec1a640 1 -- 172.21.15.43:0/2615490896 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7efcf81591c0 msgr2=0x7efcf80b2e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:34:40.176 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:40.173+0000 7efcfec1a640 1 --2- 172.21.15.43:0/2615490896 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7efcf81591c0 0x7efcf80b2e00 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7efce8011630 tx=0x7efce8011af0 comp rx=0 tx=0).stop 2023-08-09T13:34:40.177 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:40.173+0000 7efcfec1a640 1 -- 172.21.15.43:0/2615490896 shutdown_connections 2023-08-09T13:34:40.177 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:40.173+0000 7efcfec1a640 1 --2- 172.21.15.43:0/2615490896 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7efcf80c5190 0x7efcf80c5610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:34:40.177 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:40.173+0000 7efcfec1a640 1 --2- 172.21.15.43:0/2615490896 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7efcf81591c0 0x7efcf80b2e00 unknown :-1 s=CLOSED pgs=198 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:34:40.177 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:40.173+0000 7efcfec1a640 1 --2- 172.21.15.43:0/2615490896 >> [v2:172.21.15.203:6832/830097474,v1:172.21.15.203:6833/830097474] conn(0x7efcdc06a050 0x7efcdc06c510 unknown :-1 s=CLOSED pgs=124 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:34:40.177 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:40.173+0000 7efcfec1a640 1 --2- 172.21.15.43:0/2615490896 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7efce4003b30 0x7efcf814c1e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:34:40.177 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:40.173+0000 7efcfec1a640 1 -- 172.21.15.43:0/2615490896 >> 172.21.15.43:0/2615490896 conn(0x7efcf80cde90 msgr2=0x7efcf80aeb10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-09T13:34:40.177 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:40.174+0000 7efcfec1a640 1 -- 172.21.15.43:0/2615490896 shutdown_connections 2023-08-09T13:34:40.178 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:40.174+0000 7efcfec1a640 1 -- 172.21.15.43:0/2615490896 wait complete. 2023-08-09T13:34:40.195 INFO:tasks.ceph.ceph_manager.ceph:wait_until_healthy done 2023-08-09T13:34:40.195 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-08-09T13:34:40.410 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:40.408+0000 7fb77c3a6640 1 Processor -- start 2023-08-09T13:34:40.410 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:40.408+0000 7fb77c3a6640 1 -- start start 2023-08-09T13:34:40.411 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:40.409+0000 7fb77c3a6640 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7fb7740b2790 0x7fb7740d2ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:34:40.411 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:40.409+0000 7fb77c3a6640 1 -- --> v1:172.21.15.43:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb7740ce090 con 0x7fb7740b3050 2023-08-09T13:34:40.412 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:40.409+0000 7fb77c3a6640 1 -- --> v1:172.21.15.203:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb7740cd7f0 con 0x7fb7740b2b70 2023-08-09T13:34:40.412 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:40.409+0000 7fb77c3a6640 1 -- --> v2:172.21.15.203:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb7740b3950 con 0x7fb7740b2790 2023-08-09T13:34:40.414 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:40.409+0000 7fb77a943640 1 --1- >> v1:172.21.15.43:6789/0 conn(0x7fb7740b3050 0x7fb7740b3f60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.43:6789/0 says I am v1:172.21.15.43:35720/0 (socket says 172.21.15.43:35720) 2023-08-09T13:34:40.414 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:40.409+0000 7fb77a943640 1 -- 172.21.15.43:0/2537961183 learned_addr learned my addr 172.21.15.43:0/2537961183 (peer_addr_for_me v1:172.21.15.43:0/0) 2023-08-09T13:34:40.414 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:40.410+0000 7fb77a142640 1 --2- 172.21.15.43:0/2537961183 >> v2:172.21.15.203:3300/0 conn(0x7fb7740b2790 0x7fb7740d2ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:34:40.414 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:40.410+0000 7fb779140640 1 -- 172.21.15.43:0/2537961183 <== mon.0 v1:172.21.15.43:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 71463846 0 0) 0x7fb7740ce090 con 0x7fb7740b3050 2023-08-09T13:34:40.414 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:40.410+0000 7fb779140640 1 -- 172.21.15.43:0/2537961183 --> v1:172.21.15.43:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fb764003520 con 0x7fb7740b3050 2023-08-09T13:34:40.414 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:40.410+0000 7fb779140640 1 -- 172.21.15.43:0/2537961183 <== mon.0 v1:172.21.15.43:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 657220314 0 0) 0x7fb764003520 con 0x7fb7740b3050 2023-08-09T13:34:40.414 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:40.410+0000 7fb779140640 1 -- 172.21.15.43:0/2537961183 >> v2:172.21.15.203:3300/0 conn(0x7fb7740b2790 msgr2=0x7fb7740d2ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-09T13:34:40.415 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:40.410+0000 7fb779140640 1 --2- 172.21.15.43:0/2537961183 >> v2:172.21.15.203:3300/0 conn(0x7fb7740b2790 0x7fb7740d2ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:34:40.415 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:40.410+0000 7fb779140640 1 -- 172.21.15.43:0/2537961183 >> v1:172.21.15.203:6789/0 conn(0x7fb7740b2b70 legacy=0x7fb7740b5330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:34:40.415 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:40.410+0000 7fb779140640 1 -- 172.21.15.43:0/2537961183 --> v1:172.21.15.43:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb7740cdad0 con 0x7fb7740b3050 2023-08-09T13:34:40.415 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:40.410+0000 7fb779140640 1 -- 172.21.15.43:0/2537961183 <== mon.0 v1:172.21.15.43:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1285150740 0 0) 0x7fb76c001670 con 0x7fb7740b3050 2023-08-09T13:34:40.415 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:40.410+0000 7fb779140640 1 -- 172.21.15.43:0/2537961183 >> v1:172.21.15.43:6789/0 conn(0x7fb7740b3050 legacy=0x7fb7740b3f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:34:40.415 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:40.410+0000 7fb779140640 1 --2- 172.21.15.43:0/2537961183 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fb7640039e0 0x7fb764003e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:34:40.415 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:40.410+0000 7fb779140640 1 --2- 172.21.15.43:0/2537961183 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb7640047c0 0x7fb764004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:34:40.416 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:40.410+0000 7fb779140640 1 --2- 172.21.15.43:0/2537961183 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fb764005180 0x7fb76400b980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:34:40.416 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:40.410+0000 7fb779140640 1 -- 172.21.15.43:0/2537961183 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb7740b3950 con 0x7fb7640047c0 2023-08-09T13:34:40.416 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:40.410+0000 7fb779140640 1 -- 172.21.15.43:0/2537961183 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb76400c0b0 con 0x7fb7640039e0 2023-08-09T13:34:40.416 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:40.410+0000 7fb779140640 1 -- 172.21.15.43:0/2537961183 --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb76400c270 con 0x7fb764005180 2023-08-09T13:34:40.416 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:40.411+0000 7fb779941640 1 --2- 172.21.15.43:0/2537961183 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb7640047c0 0x7fb764004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:34:40.416 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:40.411+0000 7fb77a142640 1 --2- 172.21.15.43:0/2537961183 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fb7640039e0 0x7fb764003e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:34:40.416 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:40.411+0000 7fb779941640 1 -- 172.21.15.43:0/2537961183 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fb764005180 msgr2=0x7fb76400b980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-09T13:34:40.416 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:40.411+0000 7fb77a943640 1 --2- 172.21.15.43:0/2537961183 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fb764005180 0x7fb76400b980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:34:40.417 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:40.411+0000 7fb779941640 1 --2- 172.21.15.43:0/2537961183 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fb764005180 0x7fb76400b980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:34:40.417 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:40.411+0000 7fb779941640 1 -- 172.21.15.43:0/2537961183 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fb7640039e0 msgr2=0x7fb764003e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-09T13:34:40.417 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:40.411+0000 7fb779941640 1 --2- 172.21.15.43:0/2537961183 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fb7640039e0 0x7fb764003e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:34:40.417 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:40.411+0000 7fb779941640 1 -- 172.21.15.43:0/2537961183 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb764003520 con 0x7fb7640047c0 2023-08-09T13:34:40.417 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:40.411+0000 7fb77a142640 1 --2- 172.21.15.43:0/2537961183 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fb7640039e0 0x7fb764003e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-08-09T13:34:40.417 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:40.411+0000 7fb77a943640 1 -- 172.21.15.43:0/2537961183 reap_dead start 2023-08-09T13:34:40.419 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:40.412+0000 7fb779941640 1 --2- 172.21.15.43:0/2537961183 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb7640047c0 0x7fb764004c40 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7fb760001750 tx=0x7fb7640057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6bdd7ef686008da4 server_cookie=0 in_seq=0 out_seq=0 2023-08-09T13:34:40.419 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:40.413+0000 7fb779140640 1 -- 172.21.15.43:0/2537961183 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb7600052d0 con 0x7fb7640047c0 2023-08-09T13:34:40.419 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:40.413+0000 7fb779140640 1 -- 172.21.15.43:0/2537961183 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb76002e050 con 0x7fb7640047c0 2023-08-09T13:34:40.419 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:40.413+0000 7fb779140640 1 -- 172.21.15.43:0/2537961183 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb760003830 con 0x7fb7640047c0 2023-08-09T13:34:40.419 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:40.413+0000 7fb77c3a6640 1 -- 172.21.15.43:0/2537961183 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb7640047c0 msgr2=0x7fb764004c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:34:40.419 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:40.413+0000 7fb77c3a6640 1 --2- 172.21.15.43:0/2537961183 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb7640047c0 0x7fb764004c40 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7fb760001750 tx=0x7fb7640057f0 comp rx=0 tx=0).stop 2023-08-09T13:34:40.420 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:40.413+0000 7fb77c3a6640 1 -- 172.21.15.43:0/2537961183 shutdown_connections 2023-08-09T13:34:40.420 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:40.413+0000 7fb77c3a6640 1 --2- 172.21.15.43:0/2537961183 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb7640047c0 0x7fb764004c40 unknown :-1 s=CLOSED pgs=200 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:34:40.420 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:40.413+0000 7fb77c3a6640 1 -- 172.21.15.43:0/2537961183 >> 172.21.15.43:0/2537961183 conn(0x7fb77409b7b0 msgr2=0x7fb7740d24e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-09T13:34:40.420 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:40.413+0000 7fb77c3a6640 1 -- 172.21.15.43:0/2537961183 shutdown_connections 2023-08-09T13:34:40.420 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:40.413+0000 7fb77c3a6640 1 -- 172.21.15.43:0/2537961183 wait complete. 2023-08-09T13:34:40.421 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:40.415+0000 7fb77c3a6640 1 Processor -- start 2023-08-09T13:34:40.421 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:40.415+0000 7fb77c3a6640 1 -- start start 2023-08-09T13:34:40.421 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:40.416+0000 7fb77c3a6640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb7640047c0 0x7fb774146fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:34:40.422 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:40.416+0000 7fb77c3a6640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fb7741478f0 0x7fb7740b0650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:34:40.422 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:40.416+0000 7fb77c3a6640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fb774148ee0 0x7fb774149360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:34:40.422 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:40.416+0000 7fb77c3a6640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb774147e40 con 0x7fb7640047c0 2023-08-09T13:34:40.422 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:40.416+0000 7fb77c3a6640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb774148120 con 0x7fb774148ee0 2023-08-09T13:34:40.423 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:40.416+0000 7fb77c3a6640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb774148400 con 0x7fb7741478f0 2023-08-09T13:34:40.426 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:40.417+0000 7fb77a142640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb7640047c0 0x7fb774146fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:34:40.426 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:40.417+0000 7fb77a142640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb7640047c0 0x7fb774146fe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:52034/0 (socket says 172.21.15.43:52034) 2023-08-09T13:34:40.427 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:40.417+0000 7fb77a142640 1 -- 172.21.15.43:0/2436797578 learned_addr learned my addr 172.21.15.43:0/2436797578 (peer_addr_for_me v2:172.21.15.43:0/0) 2023-08-09T13:34:40.427 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:40.417+0000 7fb77a943640 1 --2- 172.21.15.43:0/2436797578 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fb774148ee0 0x7fb774149360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:34:40.427 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:40.417+0000 7fb779941640 1 --2- 172.21.15.43:0/2436797578 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fb7741478f0 0x7fb7740b0650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:34:40.427 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:40.417+0000 7fb77a142640 1 -- 172.21.15.43:0/2436797578 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fb7741478f0 msgr2=0x7fb7740b0650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:34:40.427 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:40.417+0000 7fb77a142640 1 --2- 172.21.15.43:0/2436797578 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fb7741478f0 0x7fb7740b0650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:34:40.427 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:40.417+0000 7fb77a142640 1 -- 172.21.15.43:0/2436797578 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fb774148ee0 msgr2=0x7fb774149360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:34:40.428 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:40.417+0000 7fb77a142640 1 --2- 172.21.15.43:0/2436797578 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fb774148ee0 0x7fb774149360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:34:40.428 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:40.417+0000 7fb77a142640 1 -- 172.21.15.43:0/2436797578 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb7741486e0 con 0x7fb7640047c0 2023-08-09T13:34:40.428 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:40.417+0000 7fb77a943640 1 --2- 172.21.15.43:0/2436797578 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fb774148ee0 0x7fb774149360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-08-09T13:34:40.428 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:40.417+0000 7fb77a142640 1 --2- 172.21.15.43:0/2436797578 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb7640047c0 0x7fb774146fe0 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7fb768009f90 tx=0x7fb768010590 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-09T13:34:40.428 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:40.418+0000 7fb772ffd640 1 -- 172.21.15.43:0/2436797578 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb76800e050 con 0x7fb7640047c0 2023-08-09T13:34:40.428 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:40.418+0000 7fb772ffd640 1 -- 172.21.15.43:0/2436797578 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb76800f040 con 0x7fb7640047c0 2023-08-09T13:34:40.429 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:40.418+0000 7fb772ffd640 1 -- 172.21.15.43:0/2436797578 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb768003bb0 con 0x7fb7640047c0 2023-08-09T13:34:40.429 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:40.418+0000 7fb77c3a6640 1 -- 172.21.15.43:0/2436797578 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb7741489c0 con 0x7fb7640047c0 2023-08-09T13:34:40.429 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:40.418+0000 7fb77c3a6640 1 -- 172.21.15.43:0/2436797578 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb7740b0b90 con 0x7fb7640047c0 2023-08-09T13:34:40.429 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:40.419+0000 7fb77c3a6640 1 -- 172.21.15.43:0/2436797578 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb774148b50 con 0x7fb7640047c0 2023-08-09T13:34:40.429 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:40.420+0000 7fb772ffd640 1 -- 172.21.15.43:0/2436797578 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 5) v1 ==== 88341+0+0 (secure 0 0 0) 0x7fb768003d10 con 0x7fb7640047c0 2023-08-09T13:34:40.429 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:40.421+0000 7fb772ffd640 1 --2- 172.21.15.43:0/2436797578 >> [v2:172.21.15.203:6832/830097474,v1:172.21.15.203:6833/830097474] conn(0x7fb75806a050 0x7fb75806c510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:34:40.429 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:40.421+0000 7fb772ffd640 1 -- 172.21.15.43:0/2436797578 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6813+0+0 (secure 0 0 0) 0x7fb768089680 con 0x7fb7640047c0 2023-08-09T13:34:40.429 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:40.422+0000 7fb779941640 1 --2- 172.21.15.43:0/2436797578 >> [v2:172.21.15.203:6832/830097474,v1:172.21.15.203:6833/830097474] conn(0x7fb75806a050 0x7fb75806c510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:34:40.430 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:40.423+0000 7fb779941640 1 --2- 172.21.15.43:0/2436797578 >> [v2:172.21.15.203:6832/830097474,v1:172.21.15.203:6833/830097474] conn(0x7fb75806a050 0x7fb75806c510 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7fb760033000 tx=0x7fb760009030 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-09T13:34:40.430 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:40.426+0000 7fb772ffd640 1 -- 172.21.15.43:0/2436797578 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178066 (secure 0 0 0) 0x7fb7680549f0 con 0x7fb7640047c0 2023-08-09T13:34:40.627 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:40.624+0000 7fb77c3a6640 1 -- 172.21.15.43:0/2436797578 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fb7740739f0 con 0x7fb7640047c0 2023-08-09T13:34:40.630 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:40.627+0000 7fb772ffd640 1 -- 172.21.15.43:0/2436797578 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v24) v1 ==== 74+0+20050 (secure 0 0 0) 0x7fb768058010 con 0x7fb7640047c0 2023-08-09T13:34:40.631 INFO:teuthology.orchestra.run.smithi043.stdout: 2023-08-09T13:34:40.631 INFO:teuthology.orchestra.run.smithi043.stdout:{"epoch":24,"fsid":"daac87ee-1337-4250-a955-aea4e408a39b","created":"2023-08-09T13:33:41.701198+0000","modified":"2023-08-09T13:34:07.709429+0000","last_up_change":"2023-08-09T13:33:53.783453+0000","last_in_change":"2023-08-09T13:33:45.148904+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":5,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":"rbd","create_time":"2023-08-09T13:33:54.365137+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":".mgr","create_time":"2023-08-09T13:33:54.831803+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_metadata","create_time":"2023-08-09T13:34:00.207974+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data","create_time":"2023-08-09T13:34:01.614765+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":5,"pool_name":"cephfs_data_ec","create_time":"2023-08-09T13:34:05.239583+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"24","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"cbd49ecd-e896-4fbd-864e-5c37a829b165","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6800","nonce":502667335},{"type":"v1","addr":"172.21.15.43:6801","nonce":502667335}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6802","nonce":502667335},{"type":"v1","addr":"172.21.15.43:6803","nonce":502667335}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6806","nonce":502667335},{"type":"v1","addr":"172.21.15.43:6807","nonce":502667335}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6804","nonce":502667335},{"type":"v1","addr":"172.21.15.43:6805","nonce":502667335}]},"public_addr":"172.21.15.43:6801/502667335","cluster_addr":"172.21.15.43:6803/502667335","heartbeat_back_addr":"172.21.15.43:6807/502667335","heartbeat_front_addr":"172.21.15.43:6805/502667335","state":["exists","up"]},{"osd":1,"uuid":"75c7e0fe-1ec7-4fe8-a2f2-3661a65b8500","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6808","nonce":3289134610},{"type":"v1","addr":"172.21.15.43:6809","nonce":3289134610}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6810","nonce":3289134610},{"type":"v1","addr":"172.21.15.43:6812","nonce":3289134610}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6821","nonce":3289134610},{"type":"v1","addr":"172.21.15.43:6824","nonce":3289134610}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6815","nonce":3289134610},{"type":"v1","addr":"172.21.15.43:6818","nonce":3289134610}]},"public_addr":"172.21.15.43:6809/3289134610","cluster_addr":"172.21.15.43:6812/3289134610","heartbeat_back_addr":"172.21.15.43:6824/3289134610","heartbeat_front_addr":"172.21.15.43:6818/3289134610","state":["exists","up"]},{"osd":2,"uuid":"bb5c41e9-7f5f-4125-9b01-3bb8bfc2ea83","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6811","nonce":4079709736},{"type":"v1","addr":"172.21.15.43:6814","nonce":4079709736}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6817","nonce":4079709736},{"type":"v1","addr":"172.21.15.43:6820","nonce":4079709736}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6829","nonce":4079709736},{"type":"v1","addr":"172.21.15.43:6831","nonce":4079709736}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6823","nonce":4079709736},{"type":"v1","addr":"172.21.15.43:6826","nonce":4079709736}]},"public_addr":"172.21.15.43:6814/4079709736","cluster_addr":"172.21.15.43:6820/4079709736","heartbeat_back_addr":"172.21.15.43:6831/4079709736","heartbeat_front_addr":"172.21.15.43:6826/4079709736","state":["exists","up"]},{"osd":3,"uuid":"bfe4c8b4-694a-465c-87e6-2b2aab852613","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6813","nonce":1709538759},{"type":"v1","addr":"172.21.15.43:6816","nonce":1709538759}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6819","nonce":1709538759},{"type":"v1","addr":"172.21.15.43:6822","nonce":1709538759}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6828","nonce":1709538759},{"type":"v1","addr":"172.21.15.43:6830","nonce":1709538759}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6825","nonce":1709538759},{"type":"v1","addr":"172.21.15.43:6827","nonce":1709538759}]},"public_addr":"172.21.15.43:6816/1709538759","cluster_addr":"172.21.15.43:6822/1709538759","heartbeat_back_addr":"172.21.15.43:6830/1709538759","heartbeat_front_addr":"172.21.15.43:6827/1709538759","state":["exists","up"]},{"osd":4,"uuid":"fa2e8a77-aba1-4756-853e-0d174a5bdeba","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6800","nonce":4196866481},{"type":"v1","addr":"172.21.15.203:6801","nonce":4196866481}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6802","nonce":4196866481},{"type":"v1","addr":"172.21.15.203:6803","nonce":4196866481}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6806","nonce":4196866481},{"type":"v1","addr":"172.21.15.203:6807","nonce":4196866481}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6804","nonce":4196866481},{"type":"v1","addr":"172.21.15.203:6805","nonce":4196866481}]},"public_addr":"172.21.15.203:6801/4196866481","cluster_addr":"172.21.15.203:6803/4196866481","heartbeat_back_addr":"172.21.15.203:6807/4196866481","heartbeat_front_addr":"172.21.15.203:6805/4196866481","state":["exists","up"]},{"osd":5,"uuid":"81811e4b-4d24-4f51-a82c-f1d700a2a00d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6816","nonce":3059290081},{"type":"v1","addr":"172.21.15.203:6817","nonce":3059290081}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6818","nonce":3059290081},{"type":"v1","addr":"172.21.15.203:6819","nonce":3059290081}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6822","nonce":3059290081},{"type":"v1","addr":"172.21.15.203:6823","nonce":3059290081}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6820","nonce":3059290081},{"type":"v1","addr":"172.21.15.203:6821","nonce":3059290081}]},"public_addr":"172.21.15.203:6817/3059290081","cluster_addr":"172.21.15.203:6819/3059290081","heartbeat_back_addr":"172.21.15.203:6823/3059290081","heartbeat_front_addr":"172.21.15.203:6821/3059290081","state":["exists","up"]},{"osd":6,"uuid":"307bac4d-6a5b-443d-af75-56d695d7674f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6824","nonce":512155569},{"type":"v1","addr":"172.21.15.203:6825","nonce":512155569}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6826","nonce":512155569},{"type":"v1","addr":"172.21.15.203:6827","nonce":512155569}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6830","nonce":512155569},{"type":"v1","addr":"172.21.15.203:6831","nonce":512155569}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6828","nonce":512155569},{"type":"v1","addr":"172.21.15.203:6829","nonce":512155569}]},"public_addr":"172.21.15.203:6825/512155569","cluster_addr":"172.21.15.203:6827/512155569","heartbeat_back_addr":"172.21.15.203:6831/512155569","heartbeat_front_addr":"172.21.15.203:6829/512155569","state":["exists","up"]},{"osd":7,"uuid":"a95dc7ca-9f1b-4295-af70-68a22a2d7d5e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6808","nonce":3331450438},{"type":"v1","addr":"172.21.15.203:6809","nonce":3331450438}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6810","nonce":3331450438},{"type":"v1","addr":"172.21.15.203:6811","nonce":3331450438}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6814","nonce":3331450438},{"type":"v1","addr":"172.21.15.203:6815","nonce":3331450438}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6812","nonce":3331450438},{"type":"v1","addr":"172.21.15.203:6813","nonce":3331450438}]},"public_addr":"172.21.15.203:6809/3331450438","cluster_addr":"172.21.15.203:6811/3331450438","heartbeat_back_addr":"172.21.15.203:6815/3331450438","heartbeat_front_addr":"172.21.15.203:6813/3331450438","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-09T13:33:50.547652+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-09T13:33:50.589469+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-09T13:33:50.561724+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-09T13:33:50.599650+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-09T13:33:50.411855+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-09T13:33:50.219681+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-09T13:33:50.504664+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-09T13:33:50.441248+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-08-09T13:34:40.633 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:40.630+0000 7fb77c3a6640 1 -- 172.21.15.43:0/2436797578 >> [v2:172.21.15.203:6832/830097474,v1:172.21.15.203:6833/830097474] conn(0x7fb75806a050 msgr2=0x7fb75806c510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:34:40.633 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:40.630+0000 7fb77c3a6640 1 --2- 172.21.15.43:0/2436797578 >> [v2:172.21.15.203:6832/830097474,v1:172.21.15.203:6833/830097474] conn(0x7fb75806a050 0x7fb75806c510 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7fb760033000 tx=0x7fb760009030 comp rx=0 tx=0).stop 2023-08-09T13:34:40.633 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:40.630+0000 7fb77c3a6640 1 -- 172.21.15.43:0/2436797578 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb7640047c0 msgr2=0x7fb774146fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:34:40.633 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:40.630+0000 7fb77c3a6640 1 --2- 172.21.15.43:0/2436797578 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb7640047c0 0x7fb774146fe0 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7fb768009f90 tx=0x7fb768010590 comp rx=0 tx=0).stop 2023-08-09T13:34:40.634 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:40.630+0000 7fb77c3a6640 1 -- 172.21.15.43:0/2436797578 shutdown_connections 2023-08-09T13:34:40.634 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:40.630+0000 7fb77c3a6640 1 --2- 172.21.15.43:0/2436797578 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fb774148ee0 0x7fb774149360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:34:40.634 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:40.630+0000 7fb77c3a6640 1 --2- 172.21.15.43:0/2436797578 >> [v2:172.21.15.203:6832/830097474,v1:172.21.15.203:6833/830097474] conn(0x7fb75806a050 0x7fb75806c510 unknown :-1 s=CLOSED pgs=125 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:34:40.634 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:40.630+0000 7fb77c3a6640 1 --2- 172.21.15.43:0/2436797578 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fb7741478f0 0x7fb7740b0650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:34:40.634 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:40.630+0000 7fb77c3a6640 1 --2- 172.21.15.43:0/2436797578 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb7640047c0 0x7fb774146fe0 unknown :-1 s=CLOSED pgs=201 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:34:40.634 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:40.630+0000 7fb77c3a6640 1 -- 172.21.15.43:0/2436797578 >> 172.21.15.43:0/2436797578 conn(0x7fb77409b7b0 msgr2=0x7fb77406a5f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-09T13:34:40.634 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:40.631+0000 7fb77c3a6640 1 -- 172.21.15.43:0/2436797578 shutdown_connections 2023-08-09T13:34:40.635 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:40.631+0000 7fb77c3a6640 1 -- 172.21.15.43:0/2436797578 wait complete. 2023-08-09T13:34:40.655 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': 'rbd', 'create_time': '2023-08-09T13:33:54.365137+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}, 'read_balance': {'score_acting': 2, 'score_stable': 2, 'optimal_score': 1, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': '.mgr', 'create_time': '2023-08-09T13:33:54.831803+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_metadata', 'create_time': '2023-08-09T13:34:00.207974+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data', 'create_time': '2023-08-09T13:34:01.614765+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 2, 'score_stable': 2, 'optimal_score': 1, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 5, 'pool_name': 'cephfs_data_ec', 'create_time': '2023-08-09T13:34:05.239583+0000', 'flags': 5, 'flags_names': 'hashpspool,ec_overwrites', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '24', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2023-08-09T13:34:40.656 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get rbd pg_num 2023-08-09T13:34:40.870 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:40.868+0000 7fa86a7a6640 1 Processor -- start 2023-08-09T13:34:40.871 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:40.868+0000 7fa86a7a6640 1 -- start start 2023-08-09T13:34:40.872 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:40.869+0000 7fa86a7a6640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7fa864160e20 0x7fa8640cadd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:34:40.872 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:40.869+0000 7fa86a7a6640 1 -- --> v1:172.21.15.43:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa8640cd0c0 con 0x7fa8640c1620 2023-08-09T13:34:40.872 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:40.869+0000 7fa86a7a6640 1 -- --> v1:172.21.15.203:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa864161790 con 0x7fa8640ce1c0 2023-08-09T13:34:40.872 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:40.869+0000 7fa86a7a6640 1 -- --> v2:172.21.15.43:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa8640cc290 con 0x7fa864160e20 2023-08-09T13:34:40.873 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:40.870+0000 7fa868d43640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7fa864160e20 0x7fa8640cadd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:34:40.873 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:40.870+0000 7fa8637fe640 1 --1- >> v1:172.21.15.43:6789/0 conn(0x7fa8640c1620 0x7fa8640cbb80 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.43:6789/0 says I am v1:172.21.15.43:35730/0 (socket says 172.21.15.43:35730) 2023-08-09T13:34:40.873 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:40.870+0000 7fa8637fe640 1 -- 172.21.15.43:0/1251688143 learned_addr learned my addr 172.21.15.43:0/1251688143 (peer_addr_for_me v1:172.21.15.43:0/0) 2023-08-09T13:34:40.873 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:40.870+0000 7fa868d43640 1 -- 172.21.15.43:0/1251688143 >> v1:172.21.15.203:6789/0 conn(0x7fa8640ce1c0 legacy=0x7fa8640ce5a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:34:40.874 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:40.870+0000 7fa868d43640 1 -- 172.21.15.43:0/1251688143 >> v1:172.21.15.43:6789/0 conn(0x7fa8640c1620 legacy=0x7fa8640cbb80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:34:40.874 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:40.871+0000 7fa868d43640 1 -- 172.21.15.43:0/1251688143 --> v2:172.21.15.43:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa864161a70 con 0x7fa864160e20 2023-08-09T13:34:40.874 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:40.872+0000 7fa868d43640 1 --2- 172.21.15.43:0/1251688143 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa864160e20 0x7fa8640cadd0 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7fa85c0097d0 tx=0x7fa85c02b8b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b74ed57b540ab16b server_cookie=0 in_seq=0 out_seq=0 2023-08-09T13:34:40.875 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:40.872+0000 7fa862ffd640 1 -- 172.21.15.43:0/1251688143 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa85c02d040 con 0x7fa864160e20 2023-08-09T13:34:40.875 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:40.872+0000 7fa862ffd640 1 -- 172.21.15.43:0/1251688143 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa85c031810 con 0x7fa864160e20 2023-08-09T13:34:40.875 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:40.872+0000 7fa862ffd640 1 -- 172.21.15.43:0/1251688143 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa85c031ba0 con 0x7fa864160e20 2023-08-09T13:34:40.876 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:40.873+0000 7fa86a7a6640 1 -- 172.21.15.43:0/1251688143 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa864160e20 msgr2=0x7fa8640cadd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:34:40.876 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:40.873+0000 7fa86a7a6640 1 --2- 172.21.15.43:0/1251688143 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa864160e20 0x7fa8640cadd0 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7fa85c0097d0 tx=0x7fa85c02b8b0 comp rx=0 tx=0).stop 2023-08-09T13:34:40.876 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:40.873+0000 7fa86a7a6640 1 -- 172.21.15.43:0/1251688143 shutdown_connections 2023-08-09T13:34:40.876 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:40.873+0000 7fa86a7a6640 1 --2- 172.21.15.43:0/1251688143 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa864160e20 0x7fa8640cadd0 unknown :-1 s=CLOSED pgs=203 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:34:40.876 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:40.873+0000 7fa86a7a6640 1 -- 172.21.15.43:0/1251688143 >> 172.21.15.43:0/1251688143 conn(0x7fa8640aff50 msgr2=0x7fa8640b0360 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-09T13:34:40.876 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:40.873+0000 7fa86a7a6640 1 -- 172.21.15.43:0/1251688143 shutdown_connections 2023-08-09T13:34:40.877 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:40.873+0000 7fa86a7a6640 1 -- 172.21.15.43:0/1251688143 wait complete. 2023-08-09T13:34:40.877 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:40.875+0000 7fa86a7a6640 1 Processor -- start 2023-08-09T13:34:40.877 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:40.875+0000 7fa86a7a6640 1 -- start start 2023-08-09T13:34:40.878 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:40.876+0000 7fa86a7a6640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fa8640c1620 0x7fa8640c1ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:34:40.878 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:40.876+0000 7fa86a7a6640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fa8640ce1c0 0x7fa86415d320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:34:40.879 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:40.876+0000 7fa86a7a6640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa864160e20 0x7fa8640c3160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:34:40.879 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:40.876+0000 7fa86a7a6640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa8640c2cd0 con 0x7fa864160e20 2023-08-09T13:34:40.879 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:40.876+0000 7fa86a7a6640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa8640c20c0 con 0x7fa8640c1620 2023-08-09T13:34:40.879 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:40.876+0000 7fa86a7a6640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa8640c23a0 con 0x7fa8640ce1c0 2023-08-09T13:34:40.880 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:40.876+0000 7fa868d43640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa864160e20 0x7fa8640c3160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:34:40.880 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:40.876+0000 7fa868d43640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa864160e20 0x7fa8640c3160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:52048/0 (socket says 172.21.15.43:52048) 2023-08-09T13:34:40.880 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:40.876+0000 7fa868d43640 1 -- 172.21.15.43:0/574579187 learned_addr learned my addr 172.21.15.43:0/574579187 (peer_addr_for_me v2:172.21.15.43:0/0) 2023-08-09T13:34:40.881 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:40.876+0000 7fa863fff640 1 --2- 172.21.15.43:0/574579187 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fa8640c1620 0x7fa8640c1ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:34:40.881 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:40.877+0000 7fa8637fe640 1 --2- 172.21.15.43:0/574579187 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fa8640ce1c0 0x7fa86415d320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:34:40.881 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:40.877+0000 7fa868d43640 1 -- 172.21.15.43:0/574579187 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fa8640ce1c0 msgr2=0x7fa86415d320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:34:40.881 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:40.877+0000 7fa868d43640 1 --2- 172.21.15.43:0/574579187 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fa8640ce1c0 0x7fa86415d320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:34:40.881 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:40.877+0000 7fa868d43640 1 -- 172.21.15.43:0/574579187 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fa8640c1620 msgr2=0x7fa8640c1ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:34:40.881 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:40.877+0000 7fa868d43640 1 --2- 172.21.15.43:0/574579187 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fa8640c1620 0x7fa8640c1ae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:34:40.882 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:40.877+0000 7fa868d43640 1 -- 172.21.15.43:0/574579187 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa85c009400 con 0x7fa864160e20 2023-08-09T13:34:40.882 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:40.877+0000 7fa868d43640 1 --2- 172.21.15.43:0/574579187 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa864160e20 0x7fa8640c3160 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7fa85c02b8f0 tx=0x7fa85c0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-09T13:34:40.882 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:40.877+0000 7fa860ff9640 1 -- 172.21.15.43:0/574579187 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa85c03ace0 con 0x7fa864160e20 2023-08-09T13:34:40.882 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:40.878+0000 7fa860ff9640 1 -- 172.21.15.43:0/574579187 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa85c03ae60 con 0x7fa864160e20 2023-08-09T13:34:40.882 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:40.878+0000 7fa860ff9640 1 -- 172.21.15.43:0/574579187 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa85c044630 con 0x7fa864160e20 2023-08-09T13:34:40.882 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:40.878+0000 7fa86a7a6640 1 -- 172.21.15.43:0/574579187 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa8640c2680 con 0x7fa864160e20 2023-08-09T13:34:40.882 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:40.878+0000 7fa86a7a6640 1 -- 172.21.15.43:0/574579187 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa86415db00 con 0x7fa864160e20 2023-08-09T13:34:40.883 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:40.879+0000 7fa86a7a6640 1 -- 172.21.15.43:0/574579187 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa8640c2e80 con 0x7fa864160e20 2023-08-09T13:34:40.883 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:40.880+0000 7fa860ff9640 1 -- 172.21.15.43:0/574579187 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 5) v1 ==== 88341+0+0 (secure 0 0 0) 0x7fa85c044790 con 0x7fa864160e20 2023-08-09T13:34:40.883 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:40.881+0000 7fa860ff9640 1 --2- 172.21.15.43:0/574579187 >> [v2:172.21.15.203:6832/830097474,v1:172.21.15.203:6833/830097474] conn(0x7fa85006a050 0x7fa85006c510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:34:40.883 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:40.881+0000 7fa860ff9640 1 -- 172.21.15.43:0/574579187 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6813+0+0 (secure 0 0 0) 0x7fa85c0aad70 con 0x7fa864160e20 2023-08-09T13:34:40.884 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:40.881+0000 7fa863fff640 1 --2- 172.21.15.43:0/574579187 >> [v2:172.21.15.203:6832/830097474,v1:172.21.15.203:6833/830097474] conn(0x7fa85006a050 0x7fa85006c510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:34:40.884 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:40.882+0000 7fa863fff640 1 --2- 172.21.15.43:0/574579187 >> [v2:172.21.15.203:6832/830097474,v1:172.21.15.203:6833/830097474] conn(0x7fa85006a050 0x7fa85006c510 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7fa8640b3e60 tx=0x7fa854009040 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-09T13:34:40.888 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:40.885+0000 7fa860ff9640 1 -- 172.21.15.43:0/574579187 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178066 (secure 0 0 0) 0x7fa8640c2e80 con 0x7fa864160e20 2023-08-09T13:34:41.080 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:41.077+0000 7fa86a7a6640 1 -- 172.21.15.43:0/574579187 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"} v 0) v1 -- 0x7fa8640c2e80 con 0x7fa864160e20 2023-08-09T13:34:41.081 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:41.078+0000 7fa860ff9640 1 -- 172.21.15.43:0/574579187 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"}]=0 v24) v1 ==== 92+0+10 (secure 0 0 0) 0x7fa8640c2e80 con 0x7fa864160e20 2023-08-09T13:34:41.081 INFO:teuthology.orchestra.run.smithi043.stdout:pg_num: 8 2023-08-09T13:34:41.084 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:41.081+0000 7fa86a7a6640 1 -- 172.21.15.43:0/574579187 >> [v2:172.21.15.203:6832/830097474,v1:172.21.15.203:6833/830097474] conn(0x7fa85006a050 msgr2=0x7fa85006c510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:34:41.084 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:41.081+0000 7fa86a7a6640 1 --2- 172.21.15.43:0/574579187 >> [v2:172.21.15.203:6832/830097474,v1:172.21.15.203:6833/830097474] conn(0x7fa85006a050 0x7fa85006c510 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7fa8640b3e60 tx=0x7fa854009040 comp rx=0 tx=0).stop 2023-08-09T13:34:41.084 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:41.081+0000 7fa86a7a6640 1 -- 172.21.15.43:0/574579187 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa864160e20 msgr2=0x7fa8640c3160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:34:41.084 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:41.081+0000 7fa86a7a6640 1 --2- 172.21.15.43:0/574579187 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa864160e20 0x7fa8640c3160 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7fa85c02b8f0 tx=0x7fa85c0046c0 comp rx=0 tx=0).stop 2023-08-09T13:34:41.084 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:41.081+0000 7fa86a7a6640 1 -- 172.21.15.43:0/574579187 shutdown_connections 2023-08-09T13:34:41.084 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:41.081+0000 7fa86a7a6640 1 --2- 172.21.15.43:0/574579187 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa864160e20 0x7fa8640c3160 unknown :-1 s=CLOSED pgs=204 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:34:41.085 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:41.081+0000 7fa86a7a6640 1 --2- 172.21.15.43:0/574579187 >> [v2:172.21.15.203:6832/830097474,v1:172.21.15.203:6833/830097474] conn(0x7fa85006a050 0x7fa85006c510 unknown :-1 s=CLOSED pgs=126 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:34:41.085 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:41.081+0000 7fa86a7a6640 1 --2- 172.21.15.43:0/574579187 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fa8640ce1c0 0x7fa86415d320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:34:41.085 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:41.081+0000 7fa86a7a6640 1 --2- 172.21.15.43:0/574579187 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fa8640c1620 0x7fa8640c1ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:34:41.085 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:41.081+0000 7fa86a7a6640 1 -- 172.21.15.43:0/574579187 >> 172.21.15.43:0/574579187 conn(0x7fa8640aff50 msgr2=0x7fa864162270 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-09T13:34:41.085 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:41.081+0000 7fa86a7a6640 1 -- 172.21.15.43:0/574579187 shutdown_connections 2023-08-09T13:34:41.085 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:41.081+0000 7fa86a7a6640 1 -- 172.21.15.43:0/574579187 wait complete. 2023-08-09T13:34:41.103 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-08-09T13:34:41.330 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:41.327+0000 7f923fcda640 1 Processor -- start 2023-08-09T13:34:41.330 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:41.327+0000 7f923fcda640 1 -- start start 2023-08-09T13:34:41.331 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:41.329+0000 7f923fcda640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f92380cbb80 0x7f92380cb2b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:34:41.331 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:41.329+0000 7f923fcda640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f92380ce1c0 0x7f92380ce5f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:34:41.332 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:41.329+0000 7f923fcda640 1 -- --> v1:172.21.15.43:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f92380ceb30 con 0x7f923815a0c0 2023-08-09T13:34:41.332 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:41.329+0000 7f923fcda640 1 -- --> v2:172.21.15.43:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f923815a4a0 con 0x7f92380ce1c0 2023-08-09T13:34:41.332 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:41.329+0000 7f923fcda640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f92380cc000 con 0x7f92380cbb80 2023-08-09T13:34:41.333 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:41.329+0000 7f923da76640 1 --1- >> v1:172.21.15.43:6789/0 conn(0x7f923815a0c0 0x7f923815d510 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.43:6789/0 says I am v1:172.21.15.43:35740/0 (socket says 172.21.15.43:35740) 2023-08-09T13:34:41.333 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:41.329+0000 7f923e277640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f92380ce1c0 0x7f92380ce5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:34:41.333 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:41.329+0000 7f923da76640 1 -- 172.21.15.43:0/1291681501 learned_addr learned my addr 172.21.15.43:0/1291681501 (peer_addr_for_me v1:172.21.15.43:0/0) 2023-08-09T13:34:41.333 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:41.330+0000 7f923d275640 1 --2- 172.21.15.43:0/1291681501 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f92380cbb80 0x7f92380cb2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:34:41.333 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:41.330+0000 7f923e277640 1 -- 172.21.15.43:0/1291681501 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f92380cbb80 msgr2=0x7f92380cb2b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-09T13:34:41.333 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:41.330+0000 7f923e277640 1 --2- 172.21.15.43:0/1291681501 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f92380cbb80 0x7f92380cb2b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:34:41.334 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:41.330+0000 7f923e277640 1 -- 172.21.15.43:0/1291681501 >> v1:172.21.15.43:6789/0 conn(0x7f923815a0c0 legacy=0x7f923815d510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:34:41.334 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:41.330+0000 7f923e277640 1 -- 172.21.15.43:0/1291681501 --> v2:172.21.15.43:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f92380cc2d0 con 0x7f92380ce1c0 2023-08-09T13:34:41.334 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:41.330+0000 7f923d275640 1 --2- 172.21.15.43:0/1291681501 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f92380cbb80 0x7f92380cb2b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-08-09T13:34:41.335 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:41.331+0000 7f923e277640 1 --2- 172.21.15.43:0/1291681501 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f92380ce1c0 0x7f92380ce5f0 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7f9230009d90 tx=0x7f923002bea0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2656cca67fd10641 server_cookie=0 in_seq=0 out_seq=0 2023-08-09T13:34:41.335 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:41.331+0000 7f923ca74640 1 -- 172.21.15.43:0/1291681501 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f923002cc50 con 0x7f92380ce1c0 2023-08-09T13:34:41.335 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:41.331+0000 7f923ca74640 1 -- 172.21.15.43:0/1291681501 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f923002cdb0 con 0x7f92380ce1c0 2023-08-09T13:34:41.335 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:41.331+0000 7f923ca74640 1 -- 172.21.15.43:0/1291681501 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9230034660 con 0x7f92380ce1c0 2023-08-09T13:34:41.335 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:41.332+0000 7f923fcda640 1 -- 172.21.15.43:0/1291681501 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f92380ce1c0 msgr2=0x7f92380ce5f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:34:41.336 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:41.332+0000 7f923fcda640 1 --2- 172.21.15.43:0/1291681501 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f92380ce1c0 0x7f92380ce5f0 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7f9230009d90 tx=0x7f923002bea0 comp rx=0 tx=0).stop 2023-08-09T13:34:41.336 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:41.332+0000 7f923fcda640 1 -- 172.21.15.43:0/1291681501 shutdown_connections 2023-08-09T13:34:41.336 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:41.332+0000 7f923fcda640 1 --2- 172.21.15.43:0/1291681501 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f92380ce1c0 0x7f92380ce5f0 unknown :-1 s=CLOSED pgs=206 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:34:41.336 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:41.332+0000 7f923fcda640 1 --2- 172.21.15.43:0/1291681501 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f92380cbb80 0x7f92380cb2b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:34:41.336 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:41.332+0000 7f923fcda640 1 -- 172.21.15.43:0/1291681501 >> 172.21.15.43:0/1291681501 conn(0x7f92380aff50 msgr2=0x7f92380b0360 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-09T13:34:41.336 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:41.332+0000 7f923fcda640 1 -- 172.21.15.43:0/1291681501 shutdown_connections 2023-08-09T13:34:41.336 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:41.332+0000 7f923fcda640 1 -- 172.21.15.43:0/1291681501 wait complete. 2023-08-09T13:34:41.337 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:41.334+0000 7f923fcda640 1 Processor -- start 2023-08-09T13:34:41.338 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:41.334+0000 7f923fcda640 1 -- start start 2023-08-09T13:34:41.338 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:41.335+0000 7f923fcda640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f92380cbb80 0x7f92380ae5e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:34:41.338 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:41.335+0000 7f923fcda640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f92380ce1c0 0x7f92380abb40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:34:41.338 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:41.335+0000 7f923fcda640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f923815a0c0 0x7f92380706d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:34:41.338 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:41.335+0000 7f923fcda640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f92380af7d0 con 0x7f92380ce1c0 2023-08-09T13:34:41.338 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:41.335+0000 7f923fcda640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f92380aebc0 con 0x7f92380cbb80 2023-08-09T13:34:41.338 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:41.335+0000 7f923fcda640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f92380aeea0 con 0x7f923815a0c0 2023-08-09T13:34:41.339 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:41.335+0000 7f923d275640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f92380ce1c0 0x7f92380abb40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:34:41.339 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:41.335+0000 7f923d275640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f92380ce1c0 0x7f92380abb40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:52076/0 (socket says 172.21.15.43:52076) 2023-08-09T13:34:41.339 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:41.335+0000 7f923d275640 1 -- 172.21.15.43:0/4205723294 learned_addr learned my addr 172.21.15.43:0/4205723294 (peer_addr_for_me v2:172.21.15.43:0/0) 2023-08-09T13:34:41.340 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:41.336+0000 7f923d275640 1 -- 172.21.15.43:0/4205723294 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f923815a0c0 msgr2=0x7f92380706d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:34:41.340 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:41.336+0000 7f923d275640 1 --2- 172.21.15.43:0/4205723294 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f923815a0c0 0x7f92380706d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:34:41.340 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:41.336+0000 7f923d275640 1 -- 172.21.15.43:0/4205723294 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f92380cbb80 msgr2=0x7f92380ae5e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:34:41.340 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:41.336+0000 7f923d275640 1 --2- 172.21.15.43:0/4205723294 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f92380cbb80 0x7f92380ae5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:34:41.341 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:41.336+0000 7f923d275640 1 -- 172.21.15.43:0/4205723294 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9230009a70 con 0x7f92380ce1c0 2023-08-09T13:34:41.341 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:41.336+0000 7f923d275640 1 --2- 172.21.15.43:0/4205723294 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f92380ce1c0 0x7f92380abb40 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7f922800c7f0 tx=0x7f922800cbf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-09T13:34:41.341 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:41.336+0000 7f922e7fc640 1 -- 172.21.15.43:0/4205723294 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9228007a30 con 0x7f92380ce1c0 2023-08-09T13:34:41.341 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:41.336+0000 7f922e7fc640 1 -- 172.21.15.43:0/4205723294 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9228007b90 con 0x7f92380ce1c0 2023-08-09T13:34:41.341 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:41.336+0000 7f923fcda640 1 -- 172.21.15.43:0/4205723294 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f92380af180 con 0x7f92380ce1c0 2023-08-09T13:34:41.341 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:41.336+0000 7f922e7fc640 1 -- 172.21.15.43:0/4205723294 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f922800ce40 con 0x7f92380ce1c0 2023-08-09T13:34:41.341 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:41.336+0000 7f923fcda640 1 -- 172.21.15.43:0/4205723294 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f92380ac080 con 0x7f92380ce1c0 2023-08-09T13:34:41.342 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:41.338+0000 7f923fcda640 1 -- 172.21.15.43:0/4205723294 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f92380739f0 con 0x7f92380ce1c0 2023-08-09T13:34:41.342 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:41.338+0000 7f922e7fc640 1 -- 172.21.15.43:0/4205723294 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 5) v1 ==== 88341+0+0 (secure 0 0 0) 0x7f92280113f0 con 0x7f92380ce1c0 2023-08-09T13:34:41.342 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:41.339+0000 7f922e7fc640 1 --2- 172.21.15.43:0/4205723294 >> [v2:172.21.15.203:6832/830097474,v1:172.21.15.203:6833/830097474] conn(0x7f921c06a050 0x7f921c06c510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:34:41.343 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:41.339+0000 7f922e7fc640 1 -- 172.21.15.43:0/4205723294 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6813+0+0 (secure 0 0 0) 0x7f9228088a30 con 0x7f92380ce1c0 2023-08-09T13:34:41.343 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:41.340+0000 7f923da76640 1 --2- 172.21.15.43:0/4205723294 >> [v2:172.21.15.203:6832/830097474,v1:172.21.15.203:6833/830097474] conn(0x7f921c06a050 0x7f921c06c510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:34:41.343 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:41.341+0000 7f923da76640 1 --2- 172.21.15.43:0/4205723294 >> [v2:172.21.15.203:6832/830097474,v1:172.21.15.203:6833/830097474] conn(0x7f921c06a050 0x7f921c06c510 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7f9234005880 tx=0x7f9234007a00 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-09T13:34:41.346 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:41.344+0000 7f922e7fc640 1 -- 172.21.15.43:0/4205723294 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178066 (secure 0 0 0) 0x7f9228053e30 con 0x7f92380ce1c0 2023-08-09T13:34:41.547 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:41.545+0000 7f923fcda640 1 -- 172.21.15.43:0/4205723294 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f923806a3d0 con 0x7f92380ce1c0 2023-08-09T13:34:41.548 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:41.545+0000 7f922e7fc640 1 -- 172.21.15.43:0/4205723294 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v24) v1 ==== 93+0+10 (secure 0 0 0) 0x7f9228057450 con 0x7f92380ce1c0 2023-08-09T13:34:41.548 INFO:teuthology.orchestra.run.smithi043.stdout:pg_num: 1 2023-08-09T13:34:41.551 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:41.548+0000 7f923fcda640 1 -- 172.21.15.43:0/4205723294 >> [v2:172.21.15.203:6832/830097474,v1:172.21.15.203:6833/830097474] conn(0x7f921c06a050 msgr2=0x7f921c06c510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:34:41.551 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:41.548+0000 7f923fcda640 1 --2- 172.21.15.43:0/4205723294 >> [v2:172.21.15.203:6832/830097474,v1:172.21.15.203:6833/830097474] conn(0x7f921c06a050 0x7f921c06c510 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7f9234005880 tx=0x7f9234007a00 comp rx=0 tx=0).stop 2023-08-09T13:34:41.552 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:41.548+0000 7f923fcda640 1 -- 172.21.15.43:0/4205723294 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f92380ce1c0 msgr2=0x7f92380abb40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:34:41.552 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:41.548+0000 7f923fcda640 1 --2- 172.21.15.43:0/4205723294 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f92380ce1c0 0x7f92380abb40 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7f922800c7f0 tx=0x7f922800cbf0 comp rx=0 tx=0).stop 2023-08-09T13:34:41.552 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:41.549+0000 7f923fcda640 1 -- 172.21.15.43:0/4205723294 shutdown_connections 2023-08-09T13:34:41.552 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:41.549+0000 7f923fcda640 1 --2- 172.21.15.43:0/4205723294 >> [v2:172.21.15.203:6832/830097474,v1:172.21.15.203:6833/830097474] conn(0x7f921c06a050 0x7f921c06c510 unknown :-1 s=CLOSED pgs=127 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:34:41.552 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:41.549+0000 7f923fcda640 1 --2- 172.21.15.43:0/4205723294 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f923815a0c0 0x7f92380706d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:34:41.552 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:41.549+0000 7f923fcda640 1 --2- 172.21.15.43:0/4205723294 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f92380ce1c0 0x7f92380abb40 unknown :-1 s=CLOSED pgs=207 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:34:41.553 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:41.549+0000 7f923fcda640 1 --2- 172.21.15.43:0/4205723294 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f92380cbb80 0x7f92380ae5e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:34:41.553 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:41.549+0000 7f923fcda640 1 -- 172.21.15.43:0/4205723294 >> 172.21.15.43:0/4205723294 conn(0x7f92380aff50 msgr2=0x7f92380c3af0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-09T13:34:41.553 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:41.549+0000 7f923fcda640 1 -- 172.21.15.43:0/4205723294 shutdown_connections 2023-08-09T13:34:41.553 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:41.549+0000 7f923fcda640 1 -- 172.21.15.43:0/4205723294 wait complete. 2023-08-09T13:34:41.569 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-08-09T13:34:41.787 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:41.784+0000 7fda73d8b640 1 Processor -- start 2023-08-09T13:34:41.788 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:41.785+0000 7fda73d8b640 1 -- start start 2023-08-09T13:34:41.789 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:41.786+0000 7fda73d8b640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7fda6c0b3fa0 0x7fda6c0b43b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:34:41.789 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:41.787+0000 7fda73d8b640 1 -- --> v1:172.21.15.43:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fda6c0b48f0 con 0x7fda6c15c680 2023-08-09T13:34:41.789 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:41.787+0000 7fda73d8b640 1 -- --> v1:172.21.15.203:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fda6c0cdf00 con 0x7fda6c0d0a70 2023-08-09T13:34:41.789 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:41.787+0000 7fda73d8b640 1 -- --> v2:172.21.15.43:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fda6c0d0f70 con 0x7fda6c0b3fa0 2023-08-09T13:34:41.790 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:41.787+0000 7fda72328640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7fda6c0b3fa0 0x7fda6c0b43b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:34:41.790 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:41.787+0000 7fda71b27640 1 --1- >> v1:172.21.15.43:6789/0 conn(0x7fda6c15c680 0x7fda6c0ce760 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.43:6789/0 says I am v1:172.21.15.43:35744/0 (socket says 172.21.15.43:35744) 2023-08-09T13:34:41.791 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:41.787+0000 7fda71b27640 1 -- 172.21.15.43:0/3154341098 learned_addr learned my addr 172.21.15.43:0/3154341098 (peer_addr_for_me v1:172.21.15.43:0/0) 2023-08-09T13:34:41.791 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:41.787+0000 7fda72328640 1 -- 172.21.15.43:0/3154341098 >> v1:172.21.15.203:6789/0 conn(0x7fda6c0d0a70 legacy=0x7fda6c0cd7f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-09T13:34:41.791 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:41.787+0000 7fda72328640 1 -- 172.21.15.43:0/3154341098 >> v1:172.21.15.43:6789/0 conn(0x7fda6c15c680 legacy=0x7fda6c0ce760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:34:41.791 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:41.787+0000 7fda72328640 1 -- 172.21.15.43:0/3154341098 --> v2:172.21.15.43:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fda6c0ccd10 con 0x7fda6c0b3fa0 2023-08-09T13:34:41.792 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:41.789+0000 7fda72328640 1 --2- 172.21.15.43:0/3154341098 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fda6c0b3fa0 0x7fda6c0b43b0 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7fda64009d90 tx=0x7fda6402bf40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=92eef62ec10f98d9 server_cookie=0 in_seq=0 out_seq=0 2023-08-09T13:34:41.792 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:41.789+0000 7fda70b25640 1 -- 172.21.15.43:0/3154341098 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fda6402ccf0 con 0x7fda6c0b3fa0 2023-08-09T13:34:41.792 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:41.789+0000 7fda70b25640 1 -- 172.21.15.43:0/3154341098 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fda6402ce50 con 0x7fda6c0b3fa0 2023-08-09T13:34:41.793 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:41.789+0000 7fda70b25640 1 -- 172.21.15.43:0/3154341098 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fda64034660 con 0x7fda6c0b3fa0 2023-08-09T13:34:41.793 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:41.789+0000 7fda73d8b640 1 -- 172.21.15.43:0/3154341098 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fda6c0b3fa0 msgr2=0x7fda6c0b43b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:34:41.793 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:41.789+0000 7fda73d8b640 1 --2- 172.21.15.43:0/3154341098 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fda6c0b3fa0 0x7fda6c0b43b0 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7fda64009d90 tx=0x7fda6402bf40 comp rx=0 tx=0).stop 2023-08-09T13:34:41.793 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:41.789+0000 7fda73d8b640 1 -- 172.21.15.43:0/3154341098 shutdown_connections 2023-08-09T13:34:41.793 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:41.789+0000 7fda73d8b640 1 --2- 172.21.15.43:0/3154341098 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fda6c0b3fa0 0x7fda6c0b43b0 unknown :-1 s=CLOSED pgs=209 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:34:41.793 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:41.789+0000 7fda73d8b640 1 -- 172.21.15.43:0/3154341098 >> 172.21.15.43:0/3154341098 conn(0x7fda6c071520 msgr2=0x7fda6c0d2520 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-09T13:34:41.793 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:41.790+0000 7fda73d8b640 1 -- 172.21.15.43:0/3154341098 shutdown_connections 2023-08-09T13:34:41.794 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:41.790+0000 7fda73d8b640 1 -- 172.21.15.43:0/3154341098 wait complete. 2023-08-09T13:34:41.794 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:41.791+0000 7fda73d8b640 1 Processor -- start 2023-08-09T13:34:41.794 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:41.791+0000 7fda73d8b640 1 -- start start 2023-08-09T13:34:41.795 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:41.792+0000 7fda73d8b640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fda6c0b3fa0 0x7fda6c1556e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:34:41.795 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:41.792+0000 7fda73d8b640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fda6c0d0a70 0x7fda6c15d470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:34:41.796 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:41.792+0000 7fda73d8b640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fda6c15c680 0x7fda6c0c81a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:34:41.796 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:41.793+0000 7fda73d8b640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fda6c15d9b0 con 0x7fda6c15c680 2023-08-09T13:34:41.796 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:41.793+0000 7fda73d8b640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fda6c15ec80 con 0x7fda6c0d0a70 2023-08-09T13:34:41.796 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:41.793+0000 7fda73d8b640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fda6c15dbe0 con 0x7fda6c0b3fa0 2023-08-09T13:34:41.796 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:41.793+0000 7fda72328640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fda6c15c680 0x7fda6c0c81a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:34:41.796 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:41.793+0000 7fda72328640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fda6c15c680 0x7fda6c0c81a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:52096/0 (socket says 172.21.15.43:52096) 2023-08-09T13:34:41.796 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:41.793+0000 7fda72328640 1 -- 172.21.15.43:0/2863121938 learned_addr learned my addr 172.21.15.43:0/2863121938 (peer_addr_for_me v2:172.21.15.43:0/0) 2023-08-09T13:34:41.797 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:41.793+0000 7fda71b27640 1 --2- 172.21.15.43:0/2863121938 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fda6c0b3fa0 0x7fda6c1556e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:34:41.798 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:41.793+0000 7fda72328640 1 -- 172.21.15.43:0/2863121938 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fda6c0b3fa0 msgr2=0x7fda6c1556e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:34:41.798 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:41.793+0000 7fda71326640 1 --2- 172.21.15.43:0/2863121938 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fda6c0d0a70 0x7fda6c15d470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:34:41.798 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:41.793+0000 7fda72328640 1 --2- 172.21.15.43:0/2863121938 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fda6c0b3fa0 0x7fda6c1556e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:34:41.798 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:41.793+0000 7fda72328640 1 -- 172.21.15.43:0/2863121938 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fda6c0d0a70 msgr2=0x7fda6c15d470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:34:41.798 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:41.793+0000 7fda72328640 1 --2- 172.21.15.43:0/2863121938 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fda6c0d0a70 0x7fda6c15d470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:34:41.798 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:41.793+0000 7fda72328640 1 -- 172.21.15.43:0/2863121938 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fda64009a70 con 0x7fda6c15c680 2023-08-09T13:34:41.798 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:41.794+0000 7fda72328640 1 --2- 172.21.15.43:0/2863121938 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fda6c15c680 0x7fda6c0c81a0 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7fda64037040 tx=0x7fda64004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-09T13:34:41.799 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:41.794+0000 7fda627fc640 1 -- 172.21.15.43:0/2863121938 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fda6403d600 con 0x7fda6c15c680 2023-08-09T13:34:41.799 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:41.794+0000 7fda627fc640 1 -- 172.21.15.43:0/2863121938 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fda6403d760 con 0x7fda6c15c680 2023-08-09T13:34:41.799 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:41.794+0000 7fda627fc640 1 -- 172.21.15.43:0/2863121938 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fda6403daf0 con 0x7fda6c15c680 2023-08-09T13:34:41.799 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:41.794+0000 7fda73d8b640 1 -- 172.21.15.43:0/2863121938 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fda6c15dec0 con 0x7fda6c15c680 2023-08-09T13:34:41.799 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:41.794+0000 7fda73d8b640 1 -- 172.21.15.43:0/2863121938 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fda6c15e480 con 0x7fda6c15c680 2023-08-09T13:34:41.800 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:41.795+0000 7fda73d8b640 1 -- 172.21.15.43:0/2863121938 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fda6c0739f0 con 0x7fda6c15c680 2023-08-09T13:34:41.800 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:41.796+0000 7fda627fc640 1 -- 172.21.15.43:0/2863121938 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 5) v1 ==== 88341+0+0 (secure 0 0 0) 0x7fda6403b040 con 0x7fda6c15c680 2023-08-09T13:34:41.800 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:41.797+0000 7fda627fc640 1 --2- 172.21.15.43:0/2863121938 >> [v2:172.21.15.203:6832/830097474,v1:172.21.15.203:6833/830097474] conn(0x7fda5006a080 0x7fda5006c540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:34:41.801 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:41.798+0000 7fda627fc640 1 -- 172.21.15.43:0/2863121938 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6813+0+0 (secure 0 0 0) 0x7fda640aaff0 con 0x7fda6c15c680 2023-08-09T13:34:41.801 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:41.798+0000 7fda71b27640 1 --2- 172.21.15.43:0/2863121938 >> [v2:172.21.15.203:6832/830097474,v1:172.21.15.203:6833/830097474] conn(0x7fda5006a080 0x7fda5006c540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:34:41.801 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:41.799+0000 7fda71b27640 1 --2- 172.21.15.43:0/2863121938 >> [v2:172.21.15.203:6832/830097474,v1:172.21.15.203:6833/830097474] conn(0x7fda5006a080 0x7fda5006c540 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7fda68005880 tx=0x7fda68009040 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-09T13:34:41.805 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:41.802+0000 7fda627fc640 1 -- 172.21.15.43:0/2863121938 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178066 (secure 0 0 0) 0x7fda64076360 con 0x7fda6c15c680 2023-08-09T13:34:41.998 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:41.996+0000 7fda73d8b640 1 -- 172.21.15.43:0/2863121938 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7fda6c15fe60 con 0x7fda6c15c680 2023-08-09T13:34:41.999 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:41.997+0000 7fda627fc640 1 -- 172.21.15.43:0/2863121938 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v24) v1 ==== 104+0+11 (secure 0 0 0) 0x7fda6404e020 con 0x7fda6c15c680 2023-08-09T13:34:42.000 INFO:teuthology.orchestra.run.smithi043.stdout:pg_num: 32 2023-08-09T13:34:42.002 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:41.999+0000 7fda73d8b640 1 -- 172.21.15.43:0/2863121938 >> [v2:172.21.15.203:6832/830097474,v1:172.21.15.203:6833/830097474] conn(0x7fda5006a080 msgr2=0x7fda5006c540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:34:42.002 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:41.999+0000 7fda73d8b640 1 --2- 172.21.15.43:0/2863121938 >> [v2:172.21.15.203:6832/830097474,v1:172.21.15.203:6833/830097474] conn(0x7fda5006a080 0x7fda5006c540 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7fda68005880 tx=0x7fda68009040 comp rx=0 tx=0).stop 2023-08-09T13:34:42.002 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:41.999+0000 7fda73d8b640 1 -- 172.21.15.43:0/2863121938 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fda6c15c680 msgr2=0x7fda6c0c81a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:34:42.003 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:41.999+0000 7fda73d8b640 1 --2- 172.21.15.43:0/2863121938 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fda6c15c680 0x7fda6c0c81a0 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7fda64037040 tx=0x7fda64004a40 comp rx=0 tx=0).stop 2023-08-09T13:34:42.003 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:42.000+0000 7fda73d8b640 1 -- 172.21.15.43:0/2863121938 shutdown_connections 2023-08-09T13:34:42.003 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:42.000+0000 7fda73d8b640 1 --2- 172.21.15.43:0/2863121938 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fda6c15c680 0x7fda6c0c81a0 unknown :-1 s=CLOSED pgs=210 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:34:42.003 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:42.000+0000 7fda73d8b640 1 --2- 172.21.15.43:0/2863121938 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fda6c0d0a70 0x7fda6c15d470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:34:42.003 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:42.000+0000 7fda73d8b640 1 --2- 172.21.15.43:0/2863121938 >> [v2:172.21.15.203:6832/830097474,v1:172.21.15.203:6833/830097474] conn(0x7fda5006a080 0x7fda5006c540 unknown :-1 s=CLOSED pgs=128 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:34:42.003 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:42.000+0000 7fda73d8b640 1 --2- 172.21.15.43:0/2863121938 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fda6c0b3fa0 0x7fda6c1556e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:34:42.003 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:42.000+0000 7fda73d8b640 1 -- 172.21.15.43:0/2863121938 >> 172.21.15.43:0/2863121938 conn(0x7fda6c071520 msgr2=0x7fda6c15ac80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-09T13:34:42.003 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:42.000+0000 7fda73d8b640 1 -- 172.21.15.43:0/2863121938 shutdown_connections 2023-08-09T13:34:42.004 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:42.000+0000 7fda73d8b640 1 -- 172.21.15.43:0/2863121938 wait complete. 2023-08-09T13:34:42.020 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-08-09T13:34:42.245 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:42.242+0000 7f104a885640 1 Processor -- start 2023-08-09T13:34:42.245 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:42.242+0000 7f104a885640 1 -- start start 2023-08-09T13:34:42.246 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:42.244+0000 7f104a885640 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7f10440ce1c0 0x7f10440ce5a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:34:42.247 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:42.244+0000 7f104a885640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f10440c1620 0x7f10440cbb80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:34:42.247 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:42.244+0000 7f104a885640 1 -- --> v1:172.21.15.203:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f10440cd0c0 con 0x7f1044160e20 2023-08-09T13:34:42.247 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:42.244+0000 7f104a885640 1 -- --> v2:172.21.15.203:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f10440ceae0 con 0x7f10440ce1c0 2023-08-09T13:34:42.247 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:42.244+0000 7f104a885640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1044161790 con 0x7f10440c1620 2023-08-09T13:34:42.248 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:42.245+0000 7f10437fe640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f10440c1620 0x7f10440cbb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:34:42.248 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:42.245+0000 7f10437fe640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f10440c1620 0x7f10440cbb80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.203:3301/0 says I am v2:172.21.15.43:45306/0 (socket says 172.21.15.43:45306) 2023-08-09T13:34:42.248 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:42.245+0000 7f10437fe640 1 -- 172.21.15.43:0/2305065710 learned_addr learned my addr 172.21.15.43:0/2305065710 (peer_addr_for_me v2:172.21.15.43:0/0) 2023-08-09T13:34:42.248 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:42.245+0000 7f1043fff640 1 --2- 172.21.15.43:0/2305065710 >> v2:172.21.15.203:3300/0 conn(0x7f10440ce1c0 0x7f10440ce5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:34:42.249 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:42.246+0000 7f1042ffd640 1 -- 172.21.15.43:0/2305065710 <== mon.1 v1:172.21.15.203:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1196901646 0 0) 0x7f10440cd0c0 con 0x7f1044160e20 2023-08-09T13:34:42.249 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:42.246+0000 7f1042ffd640 1 -- 172.21.15.43:0/2305065710 --> v1:172.21.15.203:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f102c003520 con 0x7f1044160e20 2023-08-09T13:34:42.249 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:42.246+0000 7f1043fff640 1 -- 172.21.15.43:0/2305065710 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f10440c1620 msgr2=0x7f10440cbb80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-09T13:34:42.249 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:42.246+0000 7f1043fff640 1 --2- 172.21.15.43:0/2305065710 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f10440c1620 0x7f10440cbb80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:34:42.249 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:42.246+0000 7f1043fff640 1 -- 172.21.15.43:0/2305065710 >> v1:172.21.15.203:6789/0 conn(0x7f1044160e20 legacy=0x7f10440cadd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:34:42.249 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:42.246+0000 7f1043fff640 1 -- 172.21.15.43:0/2305065710 --> v2:172.21.15.203:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1044161a70 con 0x7f10440ce1c0 2023-08-09T13:34:42.250 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:42.246+0000 7f10437fe640 1 --2- 172.21.15.43:0/2305065710 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f10440c1620 0x7f10440cbb80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-08-09T13:34:42.250 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:42.248+0000 7f1043fff640 1 --2- 172.21.15.43:0/2305065710 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f10440ce1c0 0x7f10440ce5a0 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7f10340097a0 tx=0x7f103402b820 comp rx=0 tx=0).ready entity=mon.1 client_cookie=2077b3fea3b5e7ab server_cookie=0 in_seq=0 out_seq=0 2023-08-09T13:34:42.251 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:42.248+0000 7f1042ffd640 1 -- 172.21.15.43:0/2305065710 <== mon.1 v2:172.21.15.203:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1034031780 con 0x7f10440ce1c0 2023-08-09T13:34:42.251 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:42.248+0000 7f1042ffd640 1 -- 172.21.15.43:0/2305065710 <== mon.1 v2:172.21.15.203:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f10340318e0 con 0x7f10440ce1c0 2023-08-09T13:34:42.251 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:42.248+0000 7f1042ffd640 1 -- 172.21.15.43:0/2305065710 <== mon.1 v2:172.21.15.203:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1034031c70 con 0x7f10440ce1c0 2023-08-09T13:34:42.252 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:42.249+0000 7f104a885640 1 -- 172.21.15.43:0/2305065710 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f10440ce1c0 msgr2=0x7f10440ce5a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:34:42.252 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:42.249+0000 7f104a885640 1 --2- 172.21.15.43:0/2305065710 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f10440ce1c0 0x7f10440ce5a0 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7f10340097a0 tx=0x7f103402b820 comp rx=0 tx=0).stop 2023-08-09T13:34:42.252 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:42.249+0000 7f104a885640 1 -- 172.21.15.43:0/2305065710 shutdown_connections 2023-08-09T13:34:42.252 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:42.249+0000 7f104a885640 1 --2- 172.21.15.43:0/2305065710 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f10440c1620 0x7f10440cbb80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:34:42.252 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:42.249+0000 7f104a885640 1 --2- 172.21.15.43:0/2305065710 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f10440ce1c0 0x7f10440ce5a0 unknown :-1 s=CLOSED pgs=112 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:34:42.253 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:42.249+0000 7f104a885640 1 -- 172.21.15.43:0/2305065710 >> 172.21.15.43:0/2305065710 conn(0x7f10440aff50 msgr2=0x7f10440b0360 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-09T13:34:42.253 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:42.249+0000 7f104a885640 1 -- 172.21.15.43:0/2305065710 shutdown_connections 2023-08-09T13:34:42.253 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:42.249+0000 7f104a885640 1 -- 172.21.15.43:0/2305065710 wait complete. 2023-08-09T13:34:42.253 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:42.251+0000 7f104a885640 1 Processor -- start 2023-08-09T13:34:42.254 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:42.251+0000 7f104a885640 1 -- start start 2023-08-09T13:34:42.255 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:42.252+0000 7f104a885640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f10440c1620 0x7f104414f010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:34:42.255 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:42.252+0000 7f104a885640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f10440ce1c0 0x7f10440c80f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:34:42.255 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:42.252+0000 7f104a885640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f1044160e20 0x7f10440c5080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:34:42.255 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:42.252+0000 7f104a885640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f10441f3ea0 con 0x7f10440c1620 2023-08-09T13:34:42.255 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:42.252+0000 7f104a885640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f10440c7330 con 0x7f1044160e20 2023-08-09T13:34:42.255 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:42.252+0000 7f104a885640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f10440c7610 con 0x7f10440ce1c0 2023-08-09T13:34:42.257 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:42.253+0000 7f1043fff640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f10440c1620 0x7f104414f010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:34:42.257 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:42.253+0000 7f1043fff640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f10440c1620 0x7f104414f010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:52100/0 (socket says 172.21.15.43:52100) 2023-08-09T13:34:42.257 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:42.253+0000 7f1043fff640 1 -- 172.21.15.43:0/324684915 learned_addr learned my addr 172.21.15.43:0/324684915 (peer_addr_for_me v2:172.21.15.43:0/0) 2023-08-09T13:34:42.257 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:42.253+0000 7f10437fe640 1 --2- 172.21.15.43:0/324684915 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f10440ce1c0 0x7f10440c80f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:34:42.257 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:42.253+0000 7f1048e22640 1 --2- 172.21.15.43:0/324684915 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f1044160e20 0x7f10440c5080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:34:42.257 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:42.253+0000 7f1043fff640 1 -- 172.21.15.43:0/324684915 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f10440ce1c0 msgr2=0x7f10440c80f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:34:42.258 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:42.253+0000 7f1043fff640 1 --2- 172.21.15.43:0/324684915 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f10440ce1c0 0x7f10440c80f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:34:42.258 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:42.253+0000 7f1043fff640 1 -- 172.21.15.43:0/324684915 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f1044160e20 msgr2=0x7f10440c5080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:34:42.258 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:42.253+0000 7f1043fff640 1 --2- 172.21.15.43:0/324684915 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f1044160e20 0x7f10440c5080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:34:42.258 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:42.253+0000 7f1043fff640 1 -- 172.21.15.43:0/324684915 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f10380094d0 con 0x7f10440c1620 2023-08-09T13:34:42.258 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:42.253+0000 7f1048e22640 1 --2- 172.21.15.43:0/324684915 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f1044160e20 0x7f10440c5080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-08-09T13:34:42.258 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:42.254+0000 7f1043fff640 1 --2- 172.21.15.43:0/324684915 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f10440c1620 0x7f104414f010 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7f1034034040 tx=0x7f10340046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-09T13:34:42.258 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:42.254+0000 7f1040ff9640 1 -- 172.21.15.43:0/324684915 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1034032820 con 0x7f10440c1620 2023-08-09T13:34:42.258 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:42.254+0000 7f1040ff9640 1 -- 172.21.15.43:0/324684915 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f10340329a0 con 0x7f10440c1620 2023-08-09T13:34:42.259 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:42.254+0000 7f1040ff9640 1 -- 172.21.15.43:0/324684915 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1034032d50 con 0x7f10440c1620 2023-08-09T13:34:42.259 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:42.254+0000 7f104a885640 1 -- 172.21.15.43:0/324684915 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1034009400 con 0x7f10440c1620 2023-08-09T13:34:42.259 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:42.254+0000 7f104a885640 1 -- 172.21.15.43:0/324684915 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f10440c7bd0 con 0x7f10440c1620 2023-08-09T13:34:42.259 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:42.256+0000 7f104a885640 1 -- 172.21.15.43:0/324684915 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f10440739f0 con 0x7f10440c1620 2023-08-09T13:34:42.260 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:42.256+0000 7f1040ff9640 1 -- 172.21.15.43:0/324684915 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 5) v1 ==== 88341+0+0 (secure 0 0 0) 0x7f103404d3f0 con 0x7f10440c1620 2023-08-09T13:34:42.260 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:42.257+0000 7f1040ff9640 1 --2- 172.21.15.43:0/324684915 >> [v2:172.21.15.203:6832/830097474,v1:172.21.15.203:6833/830097474] conn(0x7f103006a020 0x7f103006c4e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:34:42.260 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:42.257+0000 7f1040ff9640 1 -- 172.21.15.43:0/324684915 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6813+0+0 (secure 0 0 0) 0x7f10340aa4c0 con 0x7f10440c1620 2023-08-09T13:34:42.260 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:42.258+0000 7f10437fe640 1 --2- 172.21.15.43:0/324684915 >> [v2:172.21.15.203:6832/830097474,v1:172.21.15.203:6833/830097474] conn(0x7f103006a020 0x7f103006c4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:34:42.261 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:42.259+0000 7f10437fe640 1 --2- 172.21.15.43:0/324684915 >> [v2:172.21.15.203:6832/830097474,v1:172.21.15.203:6833/830097474] conn(0x7f103006a020 0x7f103006c4e0 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7f10441609b0 tx=0x7f1038008040 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-09T13:34:42.264 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:42.262+0000 7f1040ff9640 1 -- 172.21.15.43:0/324684915 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178066 (secure 0 0 0) 0x7f1034075820 con 0x7f10440c1620 2023-08-09T13:34:42.463 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:42.460+0000 7f104a885640 1 -- 172.21.15.43:0/324684915 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f10440c0bb0 con 0x7f10440c1620 2023-08-09T13:34:42.464 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:42.461+0000 7f1040ff9640 1 -- 172.21.15.43:0/324684915 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v24) v1 ==== 100+0+11 (secure 0 0 0) 0x7f1034078e40 con 0x7f10440c1620 2023-08-09T13:34:42.464 INFO:teuthology.orchestra.run.smithi043.stdout:pg_num: 64 2023-08-09T13:34:42.467 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:42.464+0000 7f104a885640 1 -- 172.21.15.43:0/324684915 >> [v2:172.21.15.203:6832/830097474,v1:172.21.15.203:6833/830097474] conn(0x7f103006a020 msgr2=0x7f103006c4e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:34:42.467 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:42.464+0000 7f104a885640 1 --2- 172.21.15.43:0/324684915 >> [v2:172.21.15.203:6832/830097474,v1:172.21.15.203:6833/830097474] conn(0x7f103006a020 0x7f103006c4e0 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7f10441609b0 tx=0x7f1038008040 comp rx=0 tx=0).stop 2023-08-09T13:34:42.467 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:42.464+0000 7f104a885640 1 -- 172.21.15.43:0/324684915 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f10440c1620 msgr2=0x7f104414f010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:34:42.467 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:42.464+0000 7f104a885640 1 --2- 172.21.15.43:0/324684915 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f10440c1620 0x7f104414f010 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7f1034034040 tx=0x7f10340046c0 comp rx=0 tx=0).stop 2023-08-09T13:34:42.467 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:42.464+0000 7f104a885640 1 -- 172.21.15.43:0/324684915 shutdown_connections 2023-08-09T13:34:42.468 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:42.464+0000 7f104a885640 1 --2- 172.21.15.43:0/324684915 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f1044160e20 0x7f10440c5080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:34:42.468 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:42.464+0000 7f104a885640 1 --2- 172.21.15.43:0/324684915 >> [v2:172.21.15.203:6832/830097474,v1:172.21.15.203:6833/830097474] conn(0x7f103006a020 0x7f103006c4e0 unknown :-1 s=CLOSED pgs=129 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:34:42.468 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:42.464+0000 7f104a885640 1 --2- 172.21.15.43:0/324684915 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f10440ce1c0 0x7f10440c80f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:34:42.468 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:42.464+0000 7f104a885640 1 --2- 172.21.15.43:0/324684915 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f10440c1620 0x7f104414f010 unknown :-1 s=CLOSED pgs=211 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:34:42.468 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:42.464+0000 7f104a885640 1 -- 172.21.15.43:0/324684915 >> 172.21.15.43:0/324684915 conn(0x7f10440aff50 msgr2=0x7f10440ae7b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-09T13:34:42.468 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:42.464+0000 7f104a885640 1 -- 172.21.15.43:0/324684915 shutdown_connections 2023-08-09T13:34:42.468 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:42.465+0000 7f104a885640 1 -- 172.21.15.43:0/324684915 wait complete. 2023-08-09T13:34:42.486 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2023-08-09T13:34:42.719 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:42.717+0000 7f8c7359e640 1 Processor -- start 2023-08-09T13:34:42.720 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:42.717+0000 7f8c7359e640 1 -- start start 2023-08-09T13:34:42.721 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:42.718+0000 7f8c7359e640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f8c740cc9e0 0x7f8c740ce390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:34:42.721 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:42.719+0000 7f8c7359e640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f8c7406b2e0 0x7f8c7406b710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:34:42.721 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:42.719+0000 7f8c7359e640 1 -- --> v1:172.21.15.43:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8c7406bc50 con 0x7f8c7415c630 2023-08-09T13:34:42.722 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:42.719+0000 7f8c7359e640 1 -- --> v2:172.21.15.43:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8c740af310 con 0x7f8c740cc9e0 2023-08-09T13:34:42.722 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:42.719+0000 7f8c7359e640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8c740cd9f0 con 0x7f8c7406b2e0 2023-08-09T13:34:42.722 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:42.719+0000 7f8c71d9b640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f8c740cc9e0 0x7f8c740ce390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:34:42.722 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:42.719+0000 7f8c7259c640 1 --1- >> v1:172.21.15.43:6789/0 conn(0x7f8c7415c630 0x7f8c7406aae0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.43:6789/0 says I am v1:172.21.15.43:35752/0 (socket says 172.21.15.43:35752) 2023-08-09T13:34:42.723 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:42.719+0000 7f8c7259c640 1 -- 172.21.15.43:0/3006476913 learned_addr learned my addr 172.21.15.43:0/3006476913 (peer_addr_for_me v1:172.21.15.43:0/0) 2023-08-09T13:34:42.723 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:42.719+0000 7f8c72d9d640 1 --2- 172.21.15.43:0/3006476913 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f8c7406b2e0 0x7f8c7406b710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:34:42.724 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:42.719+0000 7f8c71d9b640 1 -- 172.21.15.43:0/3006476913 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f8c7406b2e0 msgr2=0x7f8c7406b710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-09T13:34:42.724 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:42.719+0000 7f8c71d9b640 1 --2- 172.21.15.43:0/3006476913 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f8c7406b2e0 0x7f8c7406b710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:34:42.724 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:42.719+0000 7f8c71d9b640 1 -- 172.21.15.43:0/3006476913 >> v1:172.21.15.43:6789/0 conn(0x7f8c7415c630 legacy=0x7f8c7406aae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:34:42.724 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:42.720+0000 7f8c71d9b640 1 -- 172.21.15.43:0/3006476913 --> v2:172.21.15.43:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8c740cce70 con 0x7f8c740cc9e0 2023-08-09T13:34:42.724 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:42.720+0000 7f8c7159a640 1 -- 172.21.15.43:0/3006476913 <== mon.0 v1:172.21.15.43:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2093043348 0 0) 0x7f8c7406bc50 con 0x7f8c7415c630 2023-08-09T13:34:42.724 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:42.721+0000 7f8c71d9b640 1 --2- 172.21.15.43:0/3006476913 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f8c740cc9e0 0x7f8c740ce390 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7f8c680097a0 tx=0x7f8c6802b820 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e8dbb0623ac6e351 server_cookie=0 in_seq=0 out_seq=0 2023-08-09T13:34:42.725 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:42.721+0000 7f8c7159a640 1 -- 172.21.15.43:0/3006476913 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8c68031780 con 0x7f8c740cc9e0 2023-08-09T13:34:42.725 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:42.721+0000 7f8c7159a640 1 -- 172.21.15.43:0/3006476913 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8c680318e0 con 0x7f8c740cc9e0 2023-08-09T13:34:42.725 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:42.721+0000 7f8c7159a640 1 -- 172.21.15.43:0/3006476913 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8c68031c70 con 0x7f8c740cc9e0 2023-08-09T13:34:42.726 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:42.722+0000 7f8c7359e640 1 -- 172.21.15.43:0/3006476913 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f8c740cc9e0 msgr2=0x7f8c740ce390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:34:42.726 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:42.722+0000 7f8c7359e640 1 --2- 172.21.15.43:0/3006476913 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f8c740cc9e0 0x7f8c740ce390 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7f8c680097a0 tx=0x7f8c6802b820 comp rx=0 tx=0).stop 2023-08-09T13:34:42.726 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:42.722+0000 7f8c7359e640 1 -- 172.21.15.43:0/3006476913 shutdown_connections 2023-08-09T13:34:42.726 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:42.722+0000 7f8c7359e640 1 --2- 172.21.15.43:0/3006476913 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f8c7406b2e0 0x7f8c7406b710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:34:42.726 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:42.722+0000 7f8c7359e640 1 --2- 172.21.15.43:0/3006476913 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f8c740cc9e0 0x7f8c740ce390 unknown :-1 s=CLOSED pgs=213 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:34:42.727 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:42.722+0000 7f8c7359e640 1 -- 172.21.15.43:0/3006476913 >> 172.21.15.43:0/3006476913 conn(0x7f8c74099370 msgr2=0x7f8c74089fe0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-09T13:34:42.727 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:42.722+0000 7f8c7359e640 1 -- 172.21.15.43:0/3006476913 shutdown_connections 2023-08-09T13:34:42.727 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:42.722+0000 7f8c7359e640 1 -- 172.21.15.43:0/3006476913 wait complete. 2023-08-09T13:34:42.727 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:42.724+0000 7f8c7359e640 1 Processor -- start 2023-08-09T13:34:42.728 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:42.724+0000 7f8c7359e640 1 -- start start 2023-08-09T13:34:42.728 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:42.725+0000 7f8c7359e640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f8c7406b2e0 0x7f8c7409a9b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:34:42.729 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:42.725+0000 7f8c7359e640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f8c7415c630 0x7f8c7409ce70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:34:42.729 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:42.725+0000 7f8c7359e640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f8c741534a0 0x7f8c740acbf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:34:42.729 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:42.725+0000 7f8c7359e640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8c7409d3b0 con 0x7f8c7415c630 2023-08-09T13:34:42.729 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:42.725+0000 7f8c7359e640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8c7409d5e0 con 0x7f8c741534a0 2023-08-09T13:34:42.729 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:42.725+0000 7f8c7359e640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8c7409d8c0 con 0x7f8c7406b2e0 2023-08-09T13:34:42.729 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:42.725+0000 7f8c71d9b640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f8c7415c630 0x7f8c7409ce70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:34:42.729 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:42.725+0000 7f8c71d9b640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f8c7415c630 0x7f8c7409ce70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:52128/0 (socket says 172.21.15.43:52128) 2023-08-09T13:34:42.729 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:42.725+0000 7f8c71d9b640 1 -- 172.21.15.43:0/2641526503 learned_addr learned my addr 172.21.15.43:0/2641526503 (peer_addr_for_me v2:172.21.15.43:0/0) 2023-08-09T13:34:42.730 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:42.725+0000 7f8c7259c640 1 --2- 172.21.15.43:0/2641526503 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f8c7406b2e0 0x7f8c7409a9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:34:42.730 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:42.726+0000 7f8c71d9b640 1 -- 172.21.15.43:0/2641526503 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f8c7406b2e0 msgr2=0x7f8c7409a9b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:34:42.730 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:42.726+0000 7f8c71d9b640 1 --2- 172.21.15.43:0/2641526503 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f8c7406b2e0 0x7f8c7409a9b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:34:42.730 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:42.726+0000 7f8c71d9b640 1 -- 172.21.15.43:0/2641526503 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f8c741534a0 msgr2=0x7f8c740acbf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:34:42.730 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:42.726+0000 7f8c71d9b640 1 --2- 172.21.15.43:0/2641526503 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f8c741534a0 0x7f8c740acbf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:34:42.731 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:42.726+0000 7f8c71d9b640 1 -- 172.21.15.43:0/2641526503 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8c68009400 con 0x7f8c7415c630 2023-08-09T13:34:42.731 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:42.726+0000 7f8c71d9b640 1 --2- 172.21.15.43:0/2641526503 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f8c7415c630 0x7f8c7409ce70 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7f8c68034040 tx=0x7f8c680046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-09T13:34:42.732 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:42.726+0000 7f8c62ffd640 1 -- 172.21.15.43:0/2641526503 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8c68032960 con 0x7f8c7415c630 2023-08-09T13:34:42.732 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:42.726+0000 7f8c62ffd640 1 -- 172.21.15.43:0/2641526503 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8c68032ac0 con 0x7f8c7415c630 2023-08-09T13:34:42.733 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:42.726+0000 7f8c62ffd640 1 -- 172.21.15.43:0/2641526503 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8c68032e70 con 0x7f8c7415c630 2023-08-09T13:34:42.733 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:42.726+0000 7f8c7359e640 1 -- 172.21.15.43:0/2641526503 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8c7409dba0 con 0x7f8c7415c630 2023-08-09T13:34:42.733 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:42.726+0000 7f8c7359e640 1 -- 172.21.15.43:0/2641526503 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8c740c4fe0 con 0x7f8c7415c630 2023-08-09T13:34:42.733 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:42.728+0000 7f8c7359e640 1 -- 172.21.15.43:0/2641526503 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8c740660b0 con 0x7f8c7415c630 2023-08-09T13:34:42.733 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:42.729+0000 7f8c62ffd640 1 -- 172.21.15.43:0/2641526503 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 5) v1 ==== 88341+0+0 (secure 0 0 0) 0x7f8c68045400 con 0x7f8c7415c630 2023-08-09T13:34:42.733 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:42.729+0000 7f8c62ffd640 1 --2- 172.21.15.43:0/2641526503 >> [v2:172.21.15.203:6832/830097474,v1:172.21.15.203:6833/830097474] conn(0x7f8c5406a050 0x7f8c5406c510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:34:42.733 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:42.730+0000 7f8c62ffd640 1 -- 172.21.15.43:0/2641526503 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6813+0+0 (secure 0 0 0) 0x7f8c680aad20 con 0x7f8c7415c630 2023-08-09T13:34:42.734 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:42.730+0000 7f8c7259c640 1 --2- 172.21.15.43:0/2641526503 >> [v2:172.21.15.203:6832/830097474,v1:172.21.15.203:6833/830097474] conn(0x7f8c5406a050 0x7f8c5406c510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:34:42.734 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:42.731+0000 7f8c7259c640 1 --2- 172.21.15.43:0/2641526503 >> [v2:172.21.15.203:6832/830097474,v1:172.21.15.203:6833/830097474] conn(0x7f8c5406a050 0x7f8c5406c510 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7f8c64000f60 tx=0x7f8c64009040 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-09T13:34:42.736 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:42.734+0000 7f8c62ffd640 1 -- 172.21.15.43:0/2641526503 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178066 (secure 0 0 0) 0x7f8c68076230 con 0x7f8c7415c630 2023-08-09T13:34:42.930 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:42.927+0000 7f8c7359e640 1 -- 172.21.15.43:0/2641526503 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) v1 -- 0x7f8c740c51d0 con 0x7f8c7415c630 2023-08-09T13:34:42.930 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:42.928+0000 7f8c62ffd640 1 -- 172.21.15.43:0/2641526503 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v24) v1 ==== 103+0+11 (secure 0 0 0) 0x7f8c68079850 con 0x7f8c7415c630 2023-08-09T13:34:42.931 INFO:teuthology.orchestra.run.smithi043.stdout:pg_num: 32 2023-08-09T13:34:42.933 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:42.930+0000 7f8c7359e640 1 -- 172.21.15.43:0/2641526503 >> [v2:172.21.15.203:6832/830097474,v1:172.21.15.203:6833/830097474] conn(0x7f8c5406a050 msgr2=0x7f8c5406c510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:34:42.933 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:42.930+0000 7f8c7359e640 1 --2- 172.21.15.43:0/2641526503 >> [v2:172.21.15.203:6832/830097474,v1:172.21.15.203:6833/830097474] conn(0x7f8c5406a050 0x7f8c5406c510 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7f8c64000f60 tx=0x7f8c64009040 comp rx=0 tx=0).stop 2023-08-09T13:34:42.933 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:42.931+0000 7f8c7359e640 1 -- 172.21.15.43:0/2641526503 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f8c7415c630 msgr2=0x7f8c7409ce70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:34:42.933 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:42.931+0000 7f8c7359e640 1 --2- 172.21.15.43:0/2641526503 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f8c7415c630 0x7f8c7409ce70 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7f8c68034040 tx=0x7f8c680046c0 comp rx=0 tx=0).stop 2023-08-09T13:34:42.934 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:42.931+0000 7f8c7359e640 1 -- 172.21.15.43:0/2641526503 shutdown_connections 2023-08-09T13:34:42.934 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:42.931+0000 7f8c7359e640 1 --2- 172.21.15.43:0/2641526503 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f8c741534a0 0x7f8c740acbf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:34:42.934 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:42.931+0000 7f8c7359e640 1 --2- 172.21.15.43:0/2641526503 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f8c7415c630 0x7f8c7409ce70 unknown :-1 s=CLOSED pgs=214 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:34:42.934 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:42.931+0000 7f8c7359e640 1 --2- 172.21.15.43:0/2641526503 >> [v2:172.21.15.203:6832/830097474,v1:172.21.15.203:6833/830097474] conn(0x7f8c5406a050 0x7f8c5406c510 unknown :-1 s=CLOSED pgs=130 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:34:42.935 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:42.931+0000 7f8c7359e640 1 --2- 172.21.15.43:0/2641526503 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f8c7406b2e0 0x7f8c7409a9b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:34:42.935 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:42.931+0000 7f8c7359e640 1 -- 172.21.15.43:0/2641526503 >> 172.21.15.43:0/2641526503 conn(0x7f8c74099370 msgr2=0x7f8c7406ea40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-09T13:34:42.935 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:42.931+0000 7f8c7359e640 1 -- 172.21.15.43:0/2641526503 shutdown_connections 2023-08-09T13:34:42.935 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:42.931+0000 7f8c7359e640 1 -- 172.21.15.43:0/2641526503 wait complete. 2023-08-09T13:34:42.951 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-08-09T13:34:43.183 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:43.181+0000 7f81de871640 1 Processor -- start 2023-08-09T13:34:43.184 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:43.181+0000 7f81de871640 1 -- start start 2023-08-09T13:34:43.185 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:43.183+0000 7f81de871640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f81d8143c20 0x7f81d806f540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:34:43.186 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:43.183+0000 7f81de871640 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7f81d80c86f0 0x7f81d80c8b50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:34:43.186 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:43.183+0000 7f81de871640 1 -- --> v1:172.21.15.43:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f81d80c9090 con 0x7f81d806c8d0 2023-08-09T13:34:43.186 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:43.183+0000 7f81de871640 1 -- --> v2:172.21.15.43:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f81d8144000 con 0x7f81d8143c20 2023-08-09T13:34:43.186 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:43.183+0000 7f81de871640 1 -- --> v2:172.21.15.203:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f81d806c440 con 0x7f81d80c86f0 2023-08-09T13:34:43.187 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:43.183+0000 7f81dce0e640 1 --1- >> v1:172.21.15.43:6789/0 conn(0x7f81d806c8d0 0x7f81d806cd00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.43:6789/0 says I am v1:172.21.15.43:35758/0 (socket says 172.21.15.43:35758) 2023-08-09T13:34:43.187 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:43.183+0000 7f81d7fff640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f81d8143c20 0x7f81d806f540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:34:43.187 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:43.183+0000 7f81dce0e640 1 -- 172.21.15.43:0/1894976212 learned_addr learned my addr 172.21.15.43:0/1894976212 (peer_addr_for_me v1:172.21.15.43:0/0) 2023-08-09T13:34:43.187 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:43.184+0000 7f81d77fe640 1 --2- 172.21.15.43:0/1894976212 >> v2:172.21.15.203:3300/0 conn(0x7f81d80c86f0 0x7f81d80c8b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:34:43.187 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:43.184+0000 7f81d6ffd640 1 -- 172.21.15.43:0/1894976212 <== mon.0 v1:172.21.15.43:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1853928537 0 0) 0x7f81d80c9090 con 0x7f81d806c8d0 2023-08-09T13:34:43.187 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:43.184+0000 7f81d6ffd640 1 -- 172.21.15.43:0/1894976212 --> v1:172.21.15.43:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f81c0003520 con 0x7f81d806c8d0 2023-08-09T13:34:43.188 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:43.184+0000 7f81d7fff640 1 -- 172.21.15.43:0/1894976212 >> v2:172.21.15.203:3300/0 conn(0x7f81d80c86f0 msgr2=0x7f81d80c8b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-09T13:34:43.188 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:43.184+0000 7f81d7fff640 1 --2- 172.21.15.43:0/1894976212 >> v2:172.21.15.203:3300/0 conn(0x7f81d80c86f0 0x7f81d80c8b50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:34:43.188 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:43.184+0000 7f81d7fff640 1 -- 172.21.15.43:0/1894976212 >> v1:172.21.15.43:6789/0 conn(0x7f81d806c8d0 legacy=0x7f81d806cd00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:34:43.188 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:43.184+0000 7f81d7fff640 1 -- 172.21.15.43:0/1894976212 --> v2:172.21.15.43:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f81d806b830 con 0x7f81d8143c20 2023-08-09T13:34:43.189 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:43.185+0000 7f81d7fff640 1 --2- 172.21.15.43:0/1894976212 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f81d8143c20 0x7f81d806f540 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7f81d80c9b70 tx=0x7f81c802b7b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e99811c2d70f6b77 server_cookie=0 in_seq=0 out_seq=0 2023-08-09T13:34:43.189 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:43.186+0000 7f81d6ffd640 1 -- 172.21.15.43:0/1894976212 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f81c8031710 con 0x7f81d8143c20 2023-08-09T13:34:43.189 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:43.186+0000 7f81d6ffd640 1 -- 172.21.15.43:0/1894976212 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f81c802f050 con 0x7f81d8143c20 2023-08-09T13:34:43.190 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:43.186+0000 7f81d6ffd640 1 -- 172.21.15.43:0/1894976212 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f81c8031a90 con 0x7f81d8143c20 2023-08-09T13:34:43.190 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:43.186+0000 7f81de871640 1 -- 172.21.15.43:0/1894976212 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f81d8143c20 msgr2=0x7f81d806f540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:34:43.190 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:43.186+0000 7f81de871640 1 --2- 172.21.15.43:0/1894976212 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f81d8143c20 0x7f81d806f540 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7f81d80c9b70 tx=0x7f81c802b7b0 comp rx=0 tx=0).stop 2023-08-09T13:34:43.190 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:43.186+0000 7f81de871640 1 -- 172.21.15.43:0/1894976212 shutdown_connections 2023-08-09T13:34:43.190 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:43.186+0000 7f81de871640 1 --2- 172.21.15.43:0/1894976212 >> v2:172.21.15.203:3300/0 conn(0x7f81d80c86f0 0x7f81d80c8b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:34:43.190 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:43.186+0000 7f81de871640 1 --2- 172.21.15.43:0/1894976212 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f81d8143c20 0x7f81d806f540 unknown :-1 s=CLOSED pgs=216 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:34:43.190 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:43.186+0000 7f81de871640 1 -- 172.21.15.43:0/1894976212 >> 172.21.15.43:0/1894976212 conn(0x7f81d80ce0a0 msgr2=0x7f81d80ce480 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-09T13:34:43.191 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:43.186+0000 7f81de871640 1 -- 172.21.15.43:0/1894976212 shutdown_connections 2023-08-09T13:34:43.191 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:43.186+0000 7f81de871640 1 -- 172.21.15.43:0/1894976212 wait complete. 2023-08-09T13:34:43.191 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:43.188+0000 7f81de871640 1 Processor -- start 2023-08-09T13:34:43.191 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:43.188+0000 7f81de871640 1 -- start start 2023-08-09T13:34:43.192 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:43.189+0000 7f81de871640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f81d806c8d0 0x7f81d813cea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:34:43.193 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:43.189+0000 7f81de871640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f81d80c86f0 0x7f81d80c6c90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:34:43.193 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:43.189+0000 7f81de871640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f81d8143c20 0x7f81d80c71d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:34:43.193 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:43.189+0000 7f81de871640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f81d815ad20 con 0x7f81d80c86f0 2023-08-09T13:34:43.193 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:43.189+0000 7f81de871640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f81d813d740 con 0x7f81d8143c20 2023-08-09T13:34:43.193 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:43.189+0000 7f81de871640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f81d813da20 con 0x7f81d806c8d0 2023-08-09T13:34:43.193 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:43.190+0000 7f81d77fe640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f81d80c86f0 0x7f81d80c6c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:34:43.193 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:43.190+0000 7f81d77fe640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f81d80c86f0 0x7f81d80c6c90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:52154/0 (socket says 172.21.15.43:52154) 2023-08-09T13:34:43.194 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:43.190+0000 7f81d77fe640 1 -- 172.21.15.43:0/1502479313 learned_addr learned my addr 172.21.15.43:0/1502479313 (peer_addr_for_me v2:172.21.15.43:0/0) 2023-08-09T13:34:43.194 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:43.190+0000 7f81d7fff640 1 --2- 172.21.15.43:0/1502479313 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f81d806c8d0 0x7f81d813cea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:34:43.194 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:43.190+0000 7f81dce0e640 1 --2- 172.21.15.43:0/1502479313 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f81d8143c20 0x7f81d80c71d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:34:43.194 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:43.190+0000 7f81d77fe640 1 -- 172.21.15.43:0/1502479313 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f81d806c8d0 msgr2=0x7f81d813cea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:34:43.194 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:43.190+0000 7f81d77fe640 1 --2- 172.21.15.43:0/1502479313 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f81d806c8d0 0x7f81d813cea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:34:43.194 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:43.190+0000 7f81d77fe640 1 -- 172.21.15.43:0/1502479313 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f81d8143c20 msgr2=0x7f81d80c71d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:34:43.194 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:43.190+0000 7f81d77fe640 1 --2- 172.21.15.43:0/1502479313 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f81d8143c20 0x7f81d80c71d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:34:43.194 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:43.190+0000 7f81d77fe640 1 -- 172.21.15.43:0/1502479313 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f81c8009400 con 0x7f81d80c86f0 2023-08-09T13:34:43.195 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:43.191+0000 7f81d77fe640 1 --2- 172.21.15.43:0/1502479313 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f81d80c86f0 0x7f81d80c6c90 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7f81cc00d5f0 tx=0x7f81cc00dab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-09T13:34:43.196 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:43.191+0000 7f81d4ff9640 1 -- 172.21.15.43:0/1502479313 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f81cc0125f0 con 0x7f81d80c86f0 2023-08-09T13:34:43.196 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:43.191+0000 7f81d4ff9640 1 -- 172.21.15.43:0/1502479313 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f81cc012750 con 0x7f81d80c86f0 2023-08-09T13:34:43.196 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:43.191+0000 7f81de871640 1 -- 172.21.15.43:0/1502479313 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f81d813dd00 con 0x7f81d80c86f0 2023-08-09T13:34:43.196 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:43.191+0000 7f81d4ff9640 1 -- 172.21.15.43:0/1502479313 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f81cc012b00 con 0x7f81d80c86f0 2023-08-09T13:34:43.196 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:43.191+0000 7f81de871640 1 -- 172.21.15.43:0/1502479313 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f81d813d3e0 con 0x7f81d80c86f0 2023-08-09T13:34:43.196 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:43.192+0000 7f81de871640 1 -- 172.21.15.43:0/1502479313 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f81d815af90 con 0x7f81d80c86f0 2023-08-09T13:34:43.196 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:43.193+0000 7f81d4ff9640 1 -- 172.21.15.43:0/1502479313 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 5) v1 ==== 88341+0+0 (secure 0 0 0) 0x7f81cc012c60 con 0x7f81d80c86f0 2023-08-09T13:34:43.197 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:43.194+0000 7f81d4ff9640 1 --2- 172.21.15.43:0/1502479313 >> [v2:172.21.15.203:6832/830097474,v1:172.21.15.203:6833/830097474] conn(0x7f81c406a050 0x7f81c406c510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:34:43.197 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:43.194+0000 7f81d4ff9640 1 -- 172.21.15.43:0/1502479313 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6813+0+0 (secure 0 0 0) 0x7f81cc088d00 con 0x7f81d80c86f0 2023-08-09T13:34:43.197 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:43.195+0000 7f81d7fff640 1 --2- 172.21.15.43:0/1502479313 >> [v2:172.21.15.203:6832/830097474,v1:172.21.15.203:6833/830097474] conn(0x7f81c406a050 0x7f81c406c510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:34:43.198 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:43.196+0000 7f81d7fff640 1 --2- 172.21.15.43:0/1502479313 >> [v2:172.21.15.203:6832/830097474,v1:172.21.15.203:6833/830097474] conn(0x7f81c406a050 0x7f81c406c510 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7f81c8035000 tx=0x7f81c8008030 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-09T13:34:43.200 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:43.198+0000 7f81d4ff9640 1 -- 172.21.15.43:0/1502479313 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178066 (secure 0 0 0) 0x7f81cc0540f0 con 0x7f81d80c86f0 2023-08-09T13:34:43.437 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:43.434+0000 7f81de871640 1 -- 172.21.15.43:0/1502479313 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f81d80739f0 con 0x7f81d80c86f0 2023-08-09T13:34:43.438 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:43.435+0000 7f81d4ff9640 1 -- 172.21.15.43:0/1502479313 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 9 v9) v1 ==== 93+0+4874 (secure 0 0 0) 0x7f81cc057710 con 0x7f81d80c86f0 2023-08-09T13:34:43.438 INFO:teuthology.orchestra.run.smithi043.stdout: 2023-08-09T13:34:43.438 INFO:teuthology.orchestra.run.smithi043.stdout:{"epoch":9,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4236,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.43:6833/3917539953","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6832","nonce":3917539953},{"type":"v1","addr":"172.21.15.43:6833","nonce":3917539953}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2},{"gid":4247,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.203:6837/2467794159","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6835","nonce":2467794159},{"type":"v1","addr":"172.21.15.203:6837","nonce":2467794159}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":4248,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.43:6835/1818380189","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6834","nonce":1818380189},{"type":"v1","addr":"172.21.15.43:6835","nonce":1818380189}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":9,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-08-09T13:34:02.575398+0000","modified":"2023-08-09T13:34:31.841945+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":4250},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4250":{"gid":4250,"name":"d","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.203:6836/798312463","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6834","nonce":798312463},{"type":"v1","addr":"172.21.15.203:6836","nonce":798312463}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[4,5],"metadata_pool":3,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-08-09T13:34:43.439 INFO:teuthology.orchestra.run.smithi043.stderr:dumped fsmap epoch 9 2023-08-09T13:34:43.441 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:43.438+0000 7f81de871640 1 -- 172.21.15.43:0/1502479313 >> [v2:172.21.15.203:6832/830097474,v1:172.21.15.203:6833/830097474] conn(0x7f81c406a050 msgr2=0x7f81c406c510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:34:43.441 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:43.438+0000 7f81de871640 1 --2- 172.21.15.43:0/1502479313 >> [v2:172.21.15.203:6832/830097474,v1:172.21.15.203:6833/830097474] conn(0x7f81c406a050 0x7f81c406c510 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7f81c8035000 tx=0x7f81c8008030 comp rx=0 tx=0).stop 2023-08-09T13:34:43.441 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:43.438+0000 7f81de871640 1 -- 172.21.15.43:0/1502479313 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f81d80c86f0 msgr2=0x7f81d80c6c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:34:43.442 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:43.438+0000 7f81de871640 1 --2- 172.21.15.43:0/1502479313 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f81d80c86f0 0x7f81d80c6c90 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7f81cc00d5f0 tx=0x7f81cc00dab0 comp rx=0 tx=0).stop 2023-08-09T13:34:43.442 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:43.439+0000 7f81de871640 1 -- 172.21.15.43:0/1502479313 shutdown_connections 2023-08-09T13:34:43.442 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:43.439+0000 7f81de871640 1 --2- 172.21.15.43:0/1502479313 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f81d8143c20 0x7f81d80c71d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:34:43.442 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:43.439+0000 7f81de871640 1 --2- 172.21.15.43:0/1502479313 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f81d80c86f0 0x7f81d80c6c90 unknown :-1 s=CLOSED pgs=217 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:34:43.442 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:43.439+0000 7f81de871640 1 --2- 172.21.15.43:0/1502479313 >> [v2:172.21.15.203:6832/830097474,v1:172.21.15.203:6833/830097474] conn(0x7f81c406a050 0x7f81c406c510 unknown :-1 s=CLOSED pgs=131 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:34:43.442 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:43.439+0000 7f81de871640 1 --2- 172.21.15.43:0/1502479313 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f81d806c8d0 0x7f81d813cea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:34:43.443 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:43.439+0000 7f81de871640 1 -- 172.21.15.43:0/1502479313 >> 172.21.15.43:0/1502479313 conn(0x7f81d80ce0a0 msgr2=0x7f81d806f520 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-09T13:34:43.443 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:43.439+0000 7f81de871640 1 -- 172.21.15.43:0/1502479313 shutdown_connections 2023-08-09T13:34:43.443 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:43.439+0000 7f81de871640 1 -- 172.21.15.43:0/1502479313 wait complete. 2023-08-09T13:34:43.461 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-08-09T13:34:43.684 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:43.681+0000 7f670ec4b640 1 Processor -- start 2023-08-09T13:34:43.684 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:43.682+0000 7f670ec4b640 1 -- start start 2023-08-09T13:34:43.686 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:43.683+0000 7f670ec4b640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f670815f7b0 0x7f67080b4760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:34:43.686 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:43.683+0000 7f670ec4b640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f67080b1050 0x7f67080ccfa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:34:43.686 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:43.683+0000 7f670ec4b640 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7f67080cd4e0 0x7f67080ceae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:34:43.687 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:43.683+0000 7f670ec4b640 1 -- --> v2:172.21.15.43:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f67080cd930 con 0x7f67080b1050 2023-08-09T13:34:43.687 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:43.683+0000 7f670ec4b640 1 -- --> v2:172.21.15.203:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f67080b7bf0 con 0x7f67080cd4e0 2023-08-09T13:34:43.687 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:43.683+0000 7f670ec4b640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f67080b7ed0 con 0x7f670815f7b0 2023-08-09T13:34:43.687 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:43.684+0000 7f670d448640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f67080b1050 0x7f67080ccfa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:34:43.687 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:43.684+0000 7f670d448640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f67080b1050 0x7f67080ccfa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:52162/0 (socket says 172.21.15.43:52162) 2023-08-09T13:34:43.687 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:43.684+0000 7f670d448640 1 -- 172.21.15.43:0/1098220862 learned_addr learned my addr 172.21.15.43:0/1098220862 (peer_addr_for_me v2:172.21.15.43:0/0) 2023-08-09T13:34:43.688 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:43.684+0000 7f670e44a640 1 --2- 172.21.15.43:0/1098220862 >> v2:172.21.15.203:3300/0 conn(0x7f67080cd4e0 0x7f67080ceae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:34:43.688 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:43.684+0000 7f670dc49640 1 --2- 172.21.15.43:0/1098220862 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f670815f7b0 0x7f67080b4760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:34:43.688 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:43.684+0000 7f670d448640 1 -- 172.21.15.43:0/1098220862 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f670815f7b0 msgr2=0x7f67080b4760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-09T13:34:43.689 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:43.684+0000 7f670d448640 1 --2- 172.21.15.43:0/1098220862 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f670815f7b0 0x7f67080b4760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:34:43.689 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:43.684+0000 7f670d448640 1 -- 172.21.15.43:0/1098220862 >> v2:172.21.15.203:3300/0 conn(0x7f67080cd4e0 msgr2=0x7f67080ceae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-09T13:34:43.689 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:43.684+0000 7f670d448640 1 --2- 172.21.15.43:0/1098220862 >> v2:172.21.15.203:3300/0 conn(0x7f67080cd4e0 0x7f67080ceae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:34:43.689 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:43.685+0000 7f670d448640 1 -- 172.21.15.43:0/1098220862 --> v2:172.21.15.43:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f67080d0370 con 0x7f67080b1050 2023-08-09T13:34:43.689 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:43.685+0000 7f670dc49640 1 --2- 172.21.15.43:0/1098220862 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f670815f7b0 0x7f67080b4760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-08-09T13:34:43.689 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:43.686+0000 7f670d448640 1 --2- 172.21.15.43:0/1098220862 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f67080b1050 0x7f67080ccfa0 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7f67080c92d0 tx=0x7f66f802b910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dd7c131b83a80c20 server_cookie=0 in_seq=0 out_seq=0 2023-08-09T13:34:43.690 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:43.686+0000 7f670cc47640 1 -- 172.21.15.43:0/1098220862 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f66f8031870 con 0x7f67080b1050 2023-08-09T13:34:43.690 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:43.686+0000 7f670cc47640 1 -- 172.21.15.43:0/1098220862 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f66f80319d0 con 0x7f67080b1050 2023-08-09T13:34:43.690 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:43.686+0000 7f670cc47640 1 -- 172.21.15.43:0/1098220862 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f66f8031d60 con 0x7f67080b1050 2023-08-09T13:34:43.691 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:43.687+0000 7f670ec4b640 1 -- 172.21.15.43:0/1098220862 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f67080b1050 msgr2=0x7f67080ccfa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:34:43.691 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:43.687+0000 7f670ec4b640 1 --2- 172.21.15.43:0/1098220862 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f67080b1050 0x7f67080ccfa0 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7f67080c92d0 tx=0x7f66f802b910 comp rx=0 tx=0).stop 2023-08-09T13:34:43.691 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:43.687+0000 7f670ec4b640 1 -- 172.21.15.43:0/1098220862 shutdown_connections 2023-08-09T13:34:43.692 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:43.687+0000 7f670ec4b640 1 --2- 172.21.15.43:0/1098220862 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f67080b1050 0x7f67080ccfa0 unknown :-1 s=CLOSED pgs=218 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:34:43.692 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:43.687+0000 7f670ec4b640 1 --2- 172.21.15.43:0/1098220862 >> v2:172.21.15.203:3300/0 conn(0x7f67080cd4e0 0x7f67080ceae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:34:43.692 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:43.687+0000 7f670ec4b640 1 --2- 172.21.15.43:0/1098220862 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f670815f7b0 0x7f67080b4760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:34:43.692 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:43.687+0000 7f670ec4b640 1 -- 172.21.15.43:0/1098220862 >> 172.21.15.43:0/1098220862 conn(0x7f670806a2b0 msgr2=0x7f670806a690 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-09T13:34:43.692 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:43.687+0000 7f670ec4b640 1 -- 172.21.15.43:0/1098220862 shutdown_connections 2023-08-09T13:34:43.692 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:43.687+0000 7f670ec4b640 1 -- 172.21.15.43:0/1098220862 wait complete. 2023-08-09T13:34:43.692 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:43.689+0000 7f670ec4b640 1 Processor -- start 2023-08-09T13:34:43.693 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:43.689+0000 7f670ec4b640 1 -- start start 2023-08-09T13:34:43.694 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:43.690+0000 7f670ec4b640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f67080b1050 0x7f6708150620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:34:43.694 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:43.690+0000 7f670ec4b640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f67080cd4e0 0x7f67081573e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:34:43.694 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:43.690+0000 7f670ec4b640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f670815f7b0 0x7f6708157920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:34:43.694 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:43.690+0000 7f670ec4b640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6708156470 con 0x7f670815f7b0 2023-08-09T13:34:43.695 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:43.690+0000 7f670ec4b640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f67080d0df0 con 0x7f67080b1050 2023-08-09T13:34:43.695 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:43.690+0000 7f670ec4b640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f67080d10d0 con 0x7f67080cd4e0 2023-08-09T13:34:43.695 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:43.690+0000 7f670e44a640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f670815f7b0 0x7f6708157920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:34:43.695 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:43.690+0000 7f670e44a640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f670815f7b0 0x7f6708157920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:52170/0 (socket says 172.21.15.43:52170) 2023-08-09T13:34:43.695 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:43.690+0000 7f670e44a640 1 -- 172.21.15.43:0/2015922994 learned_addr learned my addr 172.21.15.43:0/2015922994 (peer_addr_for_me v2:172.21.15.43:0/0) 2023-08-09T13:34:43.695 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:43.690+0000 7f670dc49640 1 --2- 172.21.15.43:0/2015922994 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f67080b1050 0x7f6708150620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:34:43.695 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:43.690+0000 7f670d448640 1 --2- 172.21.15.43:0/2015922994 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f67080cd4e0 0x7f67081573e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:34:43.696 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:43.691+0000 7f670e44a640 1 -- 172.21.15.43:0/2015922994 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f67080cd4e0 msgr2=0x7f67081573e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:34:43.696 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:43.691+0000 7f670e44a640 1 --2- 172.21.15.43:0/2015922994 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f67080cd4e0 0x7f67081573e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:34:43.696 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:43.691+0000 7f670e44a640 1 -- 172.21.15.43:0/2015922994 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f67080b1050 msgr2=0x7f6708150620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:34:43.696 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:43.691+0000 7f670e44a640 1 --2- 172.21.15.43:0/2015922994 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f67080b1050 0x7f6708150620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:34:43.696 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:43.691+0000 7f670e44a640 1 -- 172.21.15.43:0/2015922994 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f66f80094d0 con 0x7f670815f7b0 2023-08-09T13:34:43.696 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:43.691+0000 7f670dc49640 1 --2- 172.21.15.43:0/2015922994 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f67080b1050 0x7f6708150620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-08-09T13:34:43.696 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:43.691+0000 7f670e44a640 1 --2- 172.21.15.43:0/2015922994 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f670815f7b0 0x7f6708157920 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7f6700007e60 tx=0x7f670000e4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-09T13:34:43.697 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:43.691+0000 7f66fe7fc640 1 -- 172.21.15.43:0/2015922994 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6700014620 con 0x7f670815f7b0 2023-08-09T13:34:43.697 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:43.691+0000 7f66fe7fc640 1 -- 172.21.15.43:0/2015922994 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f670000ee90 con 0x7f670815f7b0 2023-08-09T13:34:43.697 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:43.691+0000 7f66fe7fc640 1 -- 172.21.15.43:0/2015922994 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f67000149e0 con 0x7f670815f7b0 2023-08-09T13:34:43.697 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:43.691+0000 7f670ec4b640 1 -- 172.21.15.43:0/2015922994 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f67080d13b0 con 0x7f670815f7b0 2023-08-09T13:34:43.697 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:43.692+0000 7f670ec4b640 1 -- 172.21.15.43:0/2015922994 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6708150b60 con 0x7f670815f7b0 2023-08-09T13:34:43.698 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:43.693+0000 7f670ec4b640 1 -- 172.21.15.43:0/2015922994 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f67080d15a0 con 0x7f670815f7b0 2023-08-09T13:34:43.698 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:43.694+0000 7f66fe7fc640 1 -- 172.21.15.43:0/2015922994 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 5) v1 ==== 88341+0+0 (secure 0 0 0) 0x7f6700014b40 con 0x7f670815f7b0 2023-08-09T13:34:43.698 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:43.695+0000 7f66fe7fc640 1 --2- 172.21.15.43:0/2015922994 >> [v2:172.21.15.203:6832/830097474,v1:172.21.15.203:6833/830097474] conn(0x7f66ec06a050 0x7f66ec06c510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:34:43.698 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:43.695+0000 7f66fe7fc640 1 -- 172.21.15.43:0/2015922994 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6813+0+0 (secure 0 0 0) 0x7f67000892a0 con 0x7f670815f7b0 2023-08-09T13:34:43.699 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:43.695+0000 7f670dc49640 1 --2- 172.21.15.43:0/2015922994 >> [v2:172.21.15.203:6832/830097474,v1:172.21.15.203:6833/830097474] conn(0x7f66ec06a050 0x7f66ec06c510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:34:43.699 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:43.696+0000 7f670dc49640 1 --2- 172.21.15.43:0/2015922994 >> [v2:172.21.15.203:6832/830097474,v1:172.21.15.203:6833/830097474] conn(0x7f66ec06a050 0x7f66ec06c510 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7f6704006fd0 tx=0x7f6704008040 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-09T13:34:43.701 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:43.699+0000 7f66fe7fc640 1 -- 172.21.15.43:0/2015922994 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178066 (secure 0 0 0) 0x7f6700054600 con 0x7f670815f7b0 2023-08-09T13:34:43.900 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:43.897+0000 7f670ec4b640 1 -- 172.21.15.43:0/2015922994 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f67080739f0 con 0x7f670815f7b0 2023-08-09T13:34:43.903 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:43.901+0000 7f66fe7fc640 1 -- 172.21.15.43:0/2015922994 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v24) v1 ==== 74+0+20050 (secure 0 0 0) 0x7f6700057c20 con 0x7f670815f7b0 2023-08-09T13:34:43.904 INFO:teuthology.orchestra.run.smithi043.stdout: 2023-08-09T13:34:43.904 INFO:teuthology.orchestra.run.smithi043.stdout:{"epoch":24,"fsid":"daac87ee-1337-4250-a955-aea4e408a39b","created":"2023-08-09T13:33:41.701198+0000","modified":"2023-08-09T13:34:07.709429+0000","last_up_change":"2023-08-09T13:33:53.783453+0000","last_in_change":"2023-08-09T13:33:45.148904+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":5,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":"rbd","create_time":"2023-08-09T13:33:54.365137+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":".mgr","create_time":"2023-08-09T13:33:54.831803+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_metadata","create_time":"2023-08-09T13:34:00.207974+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data","create_time":"2023-08-09T13:34:01.614765+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":5,"pool_name":"cephfs_data_ec","create_time":"2023-08-09T13:34:05.239583+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"24","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"cbd49ecd-e896-4fbd-864e-5c37a829b165","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6800","nonce":502667335},{"type":"v1","addr":"172.21.15.43:6801","nonce":502667335}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6802","nonce":502667335},{"type":"v1","addr":"172.21.15.43:6803","nonce":502667335}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6806","nonce":502667335},{"type":"v1","addr":"172.21.15.43:6807","nonce":502667335}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6804","nonce":502667335},{"type":"v1","addr":"172.21.15.43:6805","nonce":502667335}]},"public_addr":"172.21.15.43:6801/502667335","cluster_addr":"172.21.15.43:6803/502667335","heartbeat_back_addr":"172.21.15.43:6807/502667335","heartbeat_front_addr":"172.21.15.43:6805/502667335","state":["exists","up"]},{"osd":1,"uuid":"75c7e0fe-1ec7-4fe8-a2f2-3661a65b8500","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6808","nonce":3289134610},{"type":"v1","addr":"172.21.15.43:6809","nonce":3289134610}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6810","nonce":3289134610},{"type":"v1","addr":"172.21.15.43:6812","nonce":3289134610}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6821","nonce":3289134610},{"type":"v1","addr":"172.21.15.43:6824","nonce":3289134610}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6815","nonce":3289134610},{"type":"v1","addr":"172.21.15.43:6818","nonce":3289134610}]},"public_addr":"172.21.15.43:6809/3289134610","cluster_addr":"172.21.15.43:6812/3289134610","heartbeat_back_addr":"172.21.15.43:6824/3289134610","heartbeat_front_addr":"172.21.15.43:6818/3289134610","state":["exists","up"]},{"osd":2,"uuid":"bb5c41e9-7f5f-4125-9b01-3bb8bfc2ea83","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6811","nonce":4079709736},{"type":"v1","addr":"172.21.15.43:6814","nonce":4079709736}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6817","nonce":4079709736},{"type":"v1","addr":"172.21.15.43:6820","nonce":4079709736}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6829","nonce":4079709736},{"type":"v1","addr":"172.21.15.43:6831","nonce":4079709736}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6823","nonce":4079709736},{"type":"v1","addr":"172.21.15.43:6826","nonce":4079709736}]},"public_addr":"172.21.15.43:6814/4079709736","cluster_addr":"172.21.15.43:6820/4079709736","heartbeat_back_addr":"172.21.15.43:6831/4079709736","heartbeat_front_addr":"172.21.15.43:6826/4079709736","state":["exists","up"]},{"osd":3,"uuid":"bfe4c8b4-694a-465c-87e6-2b2aab852613","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6813","nonce":1709538759},{"type":"v1","addr":"172.21.15.43:6816","nonce":1709538759}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6819","nonce":1709538759},{"type":"v1","addr":"172.21.15.43:6822","nonce":1709538759}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6828","nonce":1709538759},{"type":"v1","addr":"172.21.15.43:6830","nonce":1709538759}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6825","nonce":1709538759},{"type":"v1","addr":"172.21.15.43:6827","nonce":1709538759}]},"public_addr":"172.21.15.43:6816/1709538759","cluster_addr":"172.21.15.43:6822/1709538759","heartbeat_back_addr":"172.21.15.43:6830/1709538759","heartbeat_front_addr":"172.21.15.43:6827/1709538759","state":["exists","up"]},{"osd":4,"uuid":"fa2e8a77-aba1-4756-853e-0d174a5bdeba","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6800","nonce":4196866481},{"type":"v1","addr":"172.21.15.203:6801","nonce":4196866481}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6802","nonce":4196866481},{"type":"v1","addr":"172.21.15.203:6803","nonce":4196866481}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6806","nonce":4196866481},{"type":"v1","addr":"172.21.15.203:6807","nonce":4196866481}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6804","nonce":4196866481},{"type":"v1","addr":"172.21.15.203:6805","nonce":4196866481}]},"public_addr":"172.21.15.203:6801/4196866481","cluster_addr":"172.21.15.203:6803/4196866481","heartbeat_back_addr":"172.21.15.203:6807/4196866481","heartbeat_front_addr":"172.21.15.203:6805/4196866481","state":["exists","up"]},{"osd":5,"uuid":"81811e4b-4d24-4f51-a82c-f1d700a2a00d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6816","nonce":3059290081},{"type":"v1","addr":"172.21.15.203:6817","nonce":3059290081}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6818","nonce":3059290081},{"type":"v1","addr":"172.21.15.203:6819","nonce":3059290081}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6822","nonce":3059290081},{"type":"v1","addr":"172.21.15.203:6823","nonce":3059290081}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6820","nonce":3059290081},{"type":"v1","addr":"172.21.15.203:6821","nonce":3059290081}]},"public_addr":"172.21.15.203:6817/3059290081","cluster_addr":"172.21.15.203:6819/3059290081","heartbeat_back_addr":"172.21.15.203:6823/3059290081","heartbeat_front_addr":"172.21.15.203:6821/3059290081","state":["exists","up"]},{"osd":6,"uuid":"307bac4d-6a5b-443d-af75-56d695d7674f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6824","nonce":512155569},{"type":"v1","addr":"172.21.15.203:6825","nonce":512155569}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6826","nonce":512155569},{"type":"v1","addr":"172.21.15.203:6827","nonce":512155569}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6830","nonce":512155569},{"type":"v1","addr":"172.21.15.203:6831","nonce":512155569}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6828","nonce":512155569},{"type":"v1","addr":"172.21.15.203:6829","nonce":512155569}]},"public_addr":"172.21.15.203:6825/512155569","cluster_addr":"172.21.15.203:6827/512155569","heartbeat_back_addr":"172.21.15.203:6831/512155569","heartbeat_front_addr":"172.21.15.203:6829/512155569","state":["exists","up"]},{"osd":7,"uuid":"a95dc7ca-9f1b-4295-af70-68a22a2d7d5e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6808","nonce":3331450438},{"type":"v1","addr":"172.21.15.203:6809","nonce":3331450438}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6810","nonce":3331450438},{"type":"v1","addr":"172.21.15.203:6811","nonce":3331450438}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6814","nonce":3331450438},{"type":"v1","addr":"172.21.15.203:6815","nonce":3331450438}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6812","nonce":3331450438},{"type":"v1","addr":"172.21.15.203:6813","nonce":3331450438}]},"public_addr":"172.21.15.203:6809/3331450438","cluster_addr":"172.21.15.203:6811/3331450438","heartbeat_back_addr":"172.21.15.203:6815/3331450438","heartbeat_front_addr":"172.21.15.203:6813/3331450438","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-09T13:33:50.547652+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-09T13:33:50.589469+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-09T13:33:50.561724+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-09T13:33:50.599650+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-09T13:33:50.411855+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-09T13:33:50.219681+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-09T13:33:50.504664+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-09T13:33:50.441248+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-08-09T13:34:43.906 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:43.903+0000 7f670ec4b640 1 -- 172.21.15.43:0/2015922994 >> [v2:172.21.15.203:6832/830097474,v1:172.21.15.203:6833/830097474] conn(0x7f66ec06a050 msgr2=0x7f66ec06c510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:34:43.907 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:43.903+0000 7f670ec4b640 1 --2- 172.21.15.43:0/2015922994 >> [v2:172.21.15.203:6832/830097474,v1:172.21.15.203:6833/830097474] conn(0x7f66ec06a050 0x7f66ec06c510 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7f6704006fd0 tx=0x7f6704008040 comp rx=0 tx=0).stop 2023-08-09T13:34:43.907 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:43.903+0000 7f670ec4b640 1 -- 172.21.15.43:0/2015922994 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f670815f7b0 msgr2=0x7f6708157920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:34:43.907 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:43.903+0000 7f670ec4b640 1 --2- 172.21.15.43:0/2015922994 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f670815f7b0 0x7f6708157920 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7f6700007e60 tx=0x7f670000e4b0 comp rx=0 tx=0).stop 2023-08-09T13:34:43.907 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:43.904+0000 7f670ec4b640 1 -- 172.21.15.43:0/2015922994 shutdown_connections 2023-08-09T13:34:43.907 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:43.904+0000 7f670ec4b640 1 --2- 172.21.15.43:0/2015922994 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f670815f7b0 0x7f6708157920 unknown :-1 s=CLOSED pgs=219 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:34:43.908 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:43.904+0000 7f670ec4b640 1 --2- 172.21.15.43:0/2015922994 >> [v2:172.21.15.203:6832/830097474,v1:172.21.15.203:6833/830097474] conn(0x7f66ec06a050 0x7f66ec06c510 unknown :-1 s=CLOSED pgs=132 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:34:43.908 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:43.904+0000 7f670ec4b640 1 --2- 172.21.15.43:0/2015922994 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f67080cd4e0 0x7f67081573e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:34:43.908 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:43.904+0000 7f670ec4b640 1 --2- 172.21.15.43:0/2015922994 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f67080b1050 0x7f6708150620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:34:43.908 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:43.904+0000 7f670ec4b640 1 -- 172.21.15.43:0/2015922994 >> 172.21.15.43:0/2015922994 conn(0x7f670806a2b0 msgr2=0x7f670815a000 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-09T13:34:43.908 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:43.904+0000 7f670ec4b640 1 -- 172.21.15.43:0/2015922994 shutdown_connections 2023-08-09T13:34:43.908 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:43.904+0000 7f670ec4b640 1 -- 172.21.15.43:0/2015922994 wait complete. 2023-08-09T13:34:43.927 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': 'rbd', 'create_time': '2023-08-09T13:33:54.365137+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}, 'read_balance': {'score_acting': 2, 'score_stable': 2, 'optimal_score': 1, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': '.mgr', 'create_time': '2023-08-09T13:33:54.831803+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_metadata', 'create_time': '2023-08-09T13:34:00.207974+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data', 'create_time': '2023-08-09T13:34:01.614765+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 2, 'score_stable': 2, 'optimal_score': 1, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 5, 'pool_name': 'cephfs_data_ec', 'create_time': '2023-08-09T13:34:05.239583+0000', 'flags': 5, 'flags_names': 'hashpspool,ec_overwrites', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '24', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2023-08-09T13:34:43.928 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get rbd pg_num 2023-08-09T13:34:44.161 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:44.159+0000 7f0adf122640 1 Processor -- start 2023-08-09T13:34:44.162 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:44.159+0000 7f0adf122640 1 -- start start 2023-08-09T13:34:44.163 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:44.161+0000 7f0adf122640 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7f0ad80c9560 0x7f0ad80cb9a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:34:44.163 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:44.161+0000 7f0adf122640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f0ad80c8900 0x7f0ad815a270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:34:44.163 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:44.161+0000 7f0adf122640 1 -- --> v1:172.21.15.203:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0ad815a7b0 con 0x7f0ad8159400 2023-08-09T13:34:44.164 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:44.161+0000 7f0adf122640 1 -- --> v2:172.21.15.203:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0ad8070250 con 0x7f0ad80c9560 2023-08-09T13:34:44.164 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:44.161+0000 7f0adf122640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0ad80c86c0 con 0x7f0ad80c8900 2023-08-09T13:34:44.165 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:44.161+0000 7f0ade921640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f0ad80c8900 0x7f0ad815a270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:34:44.165 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:44.161+0000 7f0add91f640 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7f0ad80c9560 0x7f0ad80cb9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:34:44.165 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:44.162+0000 7f0add91f640 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7f0ad80c9560 0x7f0ad80cb9a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.203:3300/0 says I am v2:172.21.15.43:34788/0 (socket says 172.21.15.43:34788) 2023-08-09T13:34:44.165 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:44.162+0000 7f0ade921640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f0ad80c8900 0x7f0ad815a270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.203:3301/0 says I am v2:172.21.15.43:45364/0 (socket says 172.21.15.43:45364) 2023-08-09T13:34:44.165 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:44.161+0000 7f0ade120640 1 --1- >> v1:172.21.15.203:6789/0 conn(0x7f0ad8159400 0x7f0ad80c8e50 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.203:6789/0 says I am v1:172.21.15.43:33948/0 (socket says 172.21.15.43:33948) 2023-08-09T13:34:44.165 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:44.162+0000 7f0add91f640 1 -- 172.21.15.43:0/1468452920 learned_addr learned my addr 172.21.15.43:0/1468452920 (peer_addr_for_me v2:172.21.15.43:0/0) 2023-08-09T13:34:44.165 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:44.162+0000 7f0add91f640 1 -- 172.21.15.43:0/1468452920 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f0ad80c8900 msgr2=0x7f0ad815a270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-09T13:34:44.166 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:44.162+0000 7f0add91f640 1 --2- 172.21.15.43:0/1468452920 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f0ad80c8900 0x7f0ad815a270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:34:44.166 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:44.162+0000 7f0add91f640 1 -- 172.21.15.43:0/1468452920 >> v1:172.21.15.203:6789/0 conn(0x7f0ad8159400 legacy=0x7f0ad80c8e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:34:44.166 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:44.162+0000 7f0add91f640 1 -- 172.21.15.43:0/1468452920 --> v2:172.21.15.203:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0ad806fc60 con 0x7f0ad80c9560 2023-08-09T13:34:44.167 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:44.164+0000 7f0add91f640 1 --2- 172.21.15.43:0/1468452920 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f0ad80c9560 0x7f0ad80cb9a0 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7f0acc00bb10 tx=0x7f0acc02e0d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=3da598a78efcdb19 server_cookie=0 in_seq=0 out_seq=0 2023-08-09T13:34:44.167 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:44.164+0000 7f0add11e640 1 -- 172.21.15.43:0/1468452920 <== mon.1 v2:172.21.15.203:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0acc02ee80 con 0x7f0ad80c9560 2023-08-09T13:34:44.167 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:44.164+0000 7f0add11e640 1 -- 172.21.15.43:0/1468452920 <== mon.1 v2:172.21.15.203:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0acc036440 con 0x7f0ad80c9560 2023-08-09T13:34:44.167 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:44.164+0000 7f0add11e640 1 -- 172.21.15.43:0/1468452920 <== mon.1 v2:172.21.15.203:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0acc0367b0 con 0x7f0ad80c9560 2023-08-09T13:34:44.168 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:44.165+0000 7f0adf122640 1 -- 172.21.15.43:0/1468452920 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f0ad80c9560 msgr2=0x7f0ad80cb9a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:34:44.168 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:44.165+0000 7f0adf122640 1 --2- 172.21.15.43:0/1468452920 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f0ad80c9560 0x7f0ad80cb9a0 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7f0acc00bb10 tx=0x7f0acc02e0d0 comp rx=0 tx=0).stop 2023-08-09T13:34:44.168 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:44.165+0000 7f0adf122640 1 -- 172.21.15.43:0/1468452920 shutdown_connections 2023-08-09T13:34:44.168 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:44.165+0000 7f0adf122640 1 --2- 172.21.15.43:0/1468452920 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f0ad80c8900 0x7f0ad815a270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:34:44.168 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:44.165+0000 7f0adf122640 1 --2- 172.21.15.43:0/1468452920 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f0ad80c9560 0x7f0ad80cb9a0 unknown :-1 s=CLOSED pgs=114 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:34:44.169 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:44.165+0000 7f0adf122640 1 -- 172.21.15.43:0/1468452920 >> 172.21.15.43:0/1468452920 conn(0x7f0ad80cddd0 msgr2=0x7f0ad80ce1e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-09T13:34:44.169 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:44.165+0000 7f0adf122640 1 -- 172.21.15.43:0/1468452920 shutdown_connections 2023-08-09T13:34:44.169 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:44.165+0000 7f0adf122640 1 -- 172.21.15.43:0/1468452920 wait complete. 2023-08-09T13:34:44.170 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:44.167+0000 7f0adf122640 1 Processor -- start 2023-08-09T13:34:44.170 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:44.168+0000 7f0adf122640 1 -- start start 2023-08-09T13:34:44.172 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:44.169+0000 7f0adf122640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f0ad80c8900 0x7f0ad81564f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:34:44.172 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:44.169+0000 7f0adf122640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f0ad80c9560 0x7f0ad80ab8d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:34:44.172 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:44.169+0000 7f0adf122640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f0ad80abe10 0x7f0ad8155050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:34:44.172 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:44.169+0000 7f0adf122640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0ad80c59b0 con 0x7f0ad80c8900 2023-08-09T13:34:44.172 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:44.169+0000 7f0adf122640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0ad80ac390 con 0x7f0ad80c9560 2023-08-09T13:34:44.172 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:44.169+0000 7f0adf122640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0ad80ac670 con 0x7f0ad80abe10 2023-08-09T13:34:44.173 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:44.169+0000 7f0ade120640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f0ad80c8900 0x7f0ad81564f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:34:44.173 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:44.169+0000 7f0ade120640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f0ad80c8900 0x7f0ad81564f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:52182/0 (socket says 172.21.15.43:52182) 2023-08-09T13:34:44.173 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:44.170+0000 7f0ade120640 1 -- 172.21.15.43:0/4175544555 learned_addr learned my addr 172.21.15.43:0/4175544555 (peer_addr_for_me v2:172.21.15.43:0/0) 2023-08-09T13:34:44.174 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:44.170+0000 7f0add91f640 1 --2- 172.21.15.43:0/4175544555 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f0ad80c9560 0x7f0ad80ab8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:34:44.174 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:44.170+0000 7f0ade921640 1 --2- 172.21.15.43:0/4175544555 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f0ad80abe10 0x7f0ad8155050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:34:44.174 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:44.170+0000 7f0ade120640 1 -- 172.21.15.43:0/4175544555 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f0ad80abe10 msgr2=0x7f0ad8155050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:34:44.174 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:44.170+0000 7f0ade120640 1 --2- 172.21.15.43:0/4175544555 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f0ad80abe10 0x7f0ad8155050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:34:44.175 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:44.170+0000 7f0ade120640 1 -- 172.21.15.43:0/4175544555 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f0ad80c9560 msgr2=0x7f0ad80ab8d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:34:44.175 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:44.170+0000 7f0ade120640 1 --2- 172.21.15.43:0/4175544555 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f0ad80c9560 0x7f0ad80ab8d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:34:44.175 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:44.170+0000 7f0ade120640 1 -- 172.21.15.43:0/4175544555 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0acc00b770 con 0x7f0ad80c8900 2023-08-09T13:34:44.175 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:44.170+0000 7f0ade921640 1 --2- 172.21.15.43:0/4175544555 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f0ad80abe10 0x7f0ad8155050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-08-09T13:34:44.175 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:44.170+0000 7f0ade120640 1 --2- 172.21.15.43:0/4175544555 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f0ad80c8900 0x7f0ad81564f0 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7f0ad400c3e0 tx=0x7f0ad400c8a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-09T13:34:44.175 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:44.171+0000 7f0acaffd640 1 -- 172.21.15.43:0/4175544555 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0ad40074e0 con 0x7f0ad80c8900 2023-08-09T13:34:44.175 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:44.171+0000 7f0acaffd640 1 -- 172.21.15.43:0/4175544555 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0ad4007640 con 0x7f0ad80c8900 2023-08-09T13:34:44.176 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:44.171+0000 7f0acaffd640 1 -- 172.21.15.43:0/4175544555 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0ad40049e0 con 0x7f0ad80c8900 2023-08-09T13:34:44.176 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:44.171+0000 7f0adf122640 1 -- 172.21.15.43:0/4175544555 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0ad80ac950 con 0x7f0ad80c8900 2023-08-09T13:34:44.176 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:44.171+0000 7f0adf122640 1 -- 172.21.15.43:0/4175544555 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0ad815c450 con 0x7f0ad80c8900 2023-08-09T13:34:44.176 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:44.172+0000 7f0adf122640 1 -- 172.21.15.43:0/4175544555 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0ad80ace70 con 0x7f0ad80c8900 2023-08-09T13:34:44.177 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:44.173+0000 7f0acaffd640 1 -- 172.21.15.43:0/4175544555 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 5) v1 ==== 88341+0+0 (secure 0 0 0) 0x7f0ad4004b40 con 0x7f0ad80c8900 2023-08-09T13:34:44.177 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:44.174+0000 7f0acaffd640 1 --2- 172.21.15.43:0/4175544555 >> [v2:172.21.15.203:6832/830097474,v1:172.21.15.203:6833/830097474] conn(0x7f0abc06a050 0x7f0abc06c510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:34:44.177 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:44.175+0000 7f0acaffd640 1 -- 172.21.15.43:0/4175544555 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6813+0+0 (secure 0 0 0) 0x7f0ad40881b0 con 0x7f0ad80c8900 2023-08-09T13:34:44.178 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:44.175+0000 7f0add91f640 1 --2- 172.21.15.43:0/4175544555 >> [v2:172.21.15.203:6832/830097474,v1:172.21.15.203:6833/830097474] conn(0x7f0abc06a050 0x7f0abc06c510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:34:44.178 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:44.176+0000 7f0add91f640 1 --2- 172.21.15.43:0/4175544555 >> [v2:172.21.15.203:6832/830097474,v1:172.21.15.203:6833/830097474] conn(0x7f0abc06a050 0x7f0abc06c510 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7f0acc038000 tx=0x7f0acc008040 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-09T13:34:44.181 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:44.179+0000 7f0acaffd640 1 -- 172.21.15.43:0/4175544555 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178066 (secure 0 0 0) 0x7f0ad80ace70 con 0x7f0ad80c8900 2023-08-09T13:34:44.381 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:44.379+0000 7f0adf122640 1 -- 172.21.15.43:0/4175544555 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"} v 0) v1 -- 0x7f0ad80ace70 con 0x7f0ad80c8900 2023-08-09T13:34:44.382 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:44.379+0000 7f0acaffd640 1 -- 172.21.15.43:0/4175544555 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"}]=0 v24) v1 ==== 92+0+10 (secure 0 0 0) 0x7f0ad80ace70 con 0x7f0ad80c8900 2023-08-09T13:34:44.383 INFO:teuthology.orchestra.run.smithi043.stdout:pg_num: 8 2023-08-09T13:34:44.385 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:44.382+0000 7f0adf122640 1 -- 172.21.15.43:0/4175544555 >> [v2:172.21.15.203:6832/830097474,v1:172.21.15.203:6833/830097474] conn(0x7f0abc06a050 msgr2=0x7f0abc06c510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:34:44.386 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:44.382+0000 7f0adf122640 1 --2- 172.21.15.43:0/4175544555 >> [v2:172.21.15.203:6832/830097474,v1:172.21.15.203:6833/830097474] conn(0x7f0abc06a050 0x7f0abc06c510 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7f0acc038000 tx=0x7f0acc008040 comp rx=0 tx=0).stop 2023-08-09T13:34:44.386 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:44.383+0000 7f0adf122640 1 -- 172.21.15.43:0/4175544555 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f0ad80c8900 msgr2=0x7f0ad81564f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:34:44.386 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:44.383+0000 7f0adf122640 1 --2- 172.21.15.43:0/4175544555 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f0ad80c8900 0x7f0ad81564f0 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7f0ad400c3e0 tx=0x7f0ad400c8a0 comp rx=0 tx=0).stop 2023-08-09T13:34:44.386 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:44.383+0000 7f0adf122640 1 -- 172.21.15.43:0/4175544555 shutdown_connections 2023-08-09T13:34:44.386 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:44.383+0000 7f0adf122640 1 --2- 172.21.15.43:0/4175544555 >> [v2:172.21.15.203:6832/830097474,v1:172.21.15.203:6833/830097474] conn(0x7f0abc06a050 0x7f0abc06c510 unknown :-1 s=CLOSED pgs=133 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:34:44.386 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:44.383+0000 7f0adf122640 1 --2- 172.21.15.43:0/4175544555 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f0ad80abe10 0x7f0ad8155050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:34:44.386 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:44.383+0000 7f0adf122640 1 --2- 172.21.15.43:0/4175544555 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f0ad80c9560 0x7f0ad80ab8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:34:44.386 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:44.383+0000 7f0adf122640 1 --2- 172.21.15.43:0/4175544555 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f0ad80c8900 0x7f0ad81564f0 unknown :-1 s=CLOSED pgs=220 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:34:44.387 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:44.383+0000 7f0adf122640 1 -- 172.21.15.43:0/4175544555 >> 172.21.15.43:0/4175544555 conn(0x7f0ad80cddd0 msgr2=0x7f0ad80b2a00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-09T13:34:44.387 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:44.383+0000 7f0adf122640 1 -- 172.21.15.43:0/4175544555 shutdown_connections 2023-08-09T13:34:44.387 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:44.383+0000 7f0adf122640 1 -- 172.21.15.43:0/4175544555 wait complete. 2023-08-09T13:34:44.405 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-08-09T13:34:44.635 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:44.633+0000 7fbcf1949640 1 Processor -- start 2023-08-09T13:34:44.635 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:44.633+0000 7fbcf1949640 1 -- start start 2023-08-09T13:34:44.636 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:44.634+0000 7fbcf1949640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7fbcec15f970 0x7fbcec0cf1d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:34:44.637 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:44.634+0000 7fbcf1949640 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7fbcec0cf710 0x7fbcec0d17b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:34:44.637 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:44.634+0000 7fbcf1949640 1 -- --> v1:172.21.15.43:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbcec0d24a0 con 0x7fbcec0d1cf0 2023-08-09T13:34:44.637 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:44.634+0000 7fbcf1949640 1 -- --> v2:172.21.15.43:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbcec0d1280 con 0x7fbcec15f970 2023-08-09T13:34:44.637 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:44.634+0000 7fbcf1949640 1 -- --> v2:172.21.15.203:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbcec0cfba0 con 0x7fbcec0cf710 2023-08-09T13:34:44.638 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:44.635+0000 7fbcf0947640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7fbcec15f970 0x7fbcec0cf1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:34:44.638 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:44.635+0000 7fbcf1148640 1 --1- >> v1:172.21.15.43:6789/0 conn(0x7fbcec0d1cf0 0x7fbcec0cfe40 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.43:6789/0 says I am v1:172.21.15.43:35772/0 (socket says 172.21.15.43:35772) 2023-08-09T13:34:44.638 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:44.635+0000 7fbcf1148640 1 -- 172.21.15.43:0/3518226332 learned_addr learned my addr 172.21.15.43:0/3518226332 (peer_addr_for_me v1:172.21.15.43:0/0) 2023-08-09T13:34:44.638 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:44.635+0000 7fbcebfff640 1 --2- 172.21.15.43:0/3518226332 >> v2:172.21.15.203:3300/0 conn(0x7fbcec0cf710 0x7fbcec0d17b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:34:44.639 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:44.635+0000 7fbcf0947640 1 -- 172.21.15.43:0/3518226332 >> v2:172.21.15.203:3300/0 conn(0x7fbcec0cf710 msgr2=0x7fbcec0d17b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-09T13:34:44.639 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:44.635+0000 7fbcf0947640 1 --2- 172.21.15.43:0/3518226332 >> v2:172.21.15.203:3300/0 conn(0x7fbcec0cf710 0x7fbcec0d17b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:34:44.639 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:44.635+0000 7fbcf0947640 1 -- 172.21.15.43:0/3518226332 >> v1:172.21.15.43:6789/0 conn(0x7fbcec0d1cf0 legacy=0x7fbcec0cfe40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:34:44.639 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:44.635+0000 7fbcf0947640 1 -- 172.21.15.43:0/3518226332 --> v2:172.21.15.43:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbcec0d2780 con 0x7fbcec15f970 2023-08-09T13:34:44.639 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:44.635+0000 7fbceb7fe640 1 -- 172.21.15.43:0/3518226332 <== mon.0 v1:172.21.15.43:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 841530089 0 0) 0x7fbcec0d24a0 con 0x7fbcec0d1cf0 2023-08-09T13:34:44.640 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:44.636+0000 7fbcf0947640 1 --2- 172.21.15.43:0/3518226332 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fbcec15f970 0x7fbcec0cf1d0 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7fbcec0b4480 tx=0x7fbcdc02be30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9efaafc7bc6c966e server_cookie=0 in_seq=0 out_seq=0 2023-08-09T13:34:44.640 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:44.637+0000 7fbceb7fe640 1 -- 172.21.15.43:0/3518226332 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbcdc02cbe0 con 0x7fbcec15f970 2023-08-09T13:34:44.640 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:44.637+0000 7fbceb7fe640 1 -- 172.21.15.43:0/3518226332 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbcdc02cd40 con 0x7fbcec15f970 2023-08-09T13:34:44.641 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:44.637+0000 7fbceb7fe640 1 -- 172.21.15.43:0/3518226332 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbcdc034660 con 0x7fbcec15f970 2023-08-09T13:34:44.641 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:44.637+0000 7fbcf1949640 1 -- 172.21.15.43:0/3518226332 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fbcec15f970 msgr2=0x7fbcec0cf1d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:34:44.641 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:44.637+0000 7fbcf1949640 1 --2- 172.21.15.43:0/3518226332 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fbcec15f970 0x7fbcec0cf1d0 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7fbcec0b4480 tx=0x7fbcdc02be30 comp rx=0 tx=0).stop 2023-08-09T13:34:44.641 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:44.637+0000 7fbcf1949640 1 -- 172.21.15.43:0/3518226332 shutdown_connections 2023-08-09T13:34:44.641 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:44.637+0000 7fbcf1949640 1 --2- 172.21.15.43:0/3518226332 >> v2:172.21.15.203:3300/0 conn(0x7fbcec0cf710 0x7fbcec0d17b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:34:44.641 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:44.637+0000 7fbcf1949640 1 --2- 172.21.15.43:0/3518226332 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fbcec15f970 0x7fbcec0cf1d0 unknown :-1 s=CLOSED pgs=222 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:34:44.641 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:44.637+0000 7fbcf1949640 1 -- 172.21.15.43:0/3518226332 >> 172.21.15.43:0/3518226332 conn(0x7fbcec0712e0 msgr2=0x7fbcec0716f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-09T13:34:44.642 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:44.638+0000 7fbcf1949640 1 -- 172.21.15.43:0/3518226332 shutdown_connections 2023-08-09T13:34:44.642 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:44.638+0000 7fbcf1949640 1 -- 172.21.15.43:0/3518226332 wait complete. 2023-08-09T13:34:44.642 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:44.639+0000 7fbcf1949640 1 Processor -- start 2023-08-09T13:34:44.642 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:44.640+0000 7fbcf1949640 1 -- start start 2023-08-09T13:34:44.643 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:44.641+0000 7fbcf1949640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fbcec0cf710 0x7fbcec15ab20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:34:44.643 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:44.641+0000 7fbcf1949640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fbcec0d1cf0 0x7fbcec0cb510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:34:44.644 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:44.641+0000 7fbcf1949640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fbcec15f970 0x7fbcec0cba50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:34:44.644 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:44.641+0000 7fbcf1949640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbcec0a8d00 con 0x7fbcec0d1cf0 2023-08-09T13:34:44.644 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:44.641+0000 7fbcf1949640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbcec0a8fa0 con 0x7fbcec15f970 2023-08-09T13:34:44.644 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:44.641+0000 7fbcf1949640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbcec0a9280 con 0x7fbcec0cf710 2023-08-09T13:34:44.644 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:44.641+0000 7fbcebfff640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fbcec0d1cf0 0x7fbcec0cb510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:34:44.644 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:44.641+0000 7fbcebfff640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fbcec0d1cf0 0x7fbcec0cb510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:52194/0 (socket says 172.21.15.43:52194) 2023-08-09T13:34:44.644 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:44.641+0000 7fbcebfff640 1 -- 172.21.15.43:0/3890875532 learned_addr learned my addr 172.21.15.43:0/3890875532 (peer_addr_for_me v2:172.21.15.43:0/0) 2023-08-09T13:34:44.645 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:44.641+0000 7fbcf0947640 1 --2- 172.21.15.43:0/3890875532 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fbcec0cf710 0x7fbcec15ab20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:34:44.645 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:44.641+0000 7fbcf1148640 1 --2- 172.21.15.43:0/3890875532 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fbcec15f970 0x7fbcec0cba50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:34:44.646 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:44.641+0000 7fbcebfff640 1 -- 172.21.15.43:0/3890875532 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fbcec0cf710 msgr2=0x7fbcec15ab20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:34:44.646 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:44.641+0000 7fbcebfff640 1 --2- 172.21.15.43:0/3890875532 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fbcec0cf710 0x7fbcec15ab20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:34:44.646 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:44.641+0000 7fbcebfff640 1 -- 172.21.15.43:0/3890875532 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fbcec15f970 msgr2=0x7fbcec0cba50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:34:44.646 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:44.641+0000 7fbcebfff640 1 --2- 172.21.15.43:0/3890875532 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fbcec15f970 0x7fbcec0cba50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:34:44.646 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:44.642+0000 7fbcebfff640 1 -- 172.21.15.43:0/3890875532 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbcdc009a70 con 0x7fbcec0d1cf0 2023-08-09T13:34:44.646 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:44.642+0000 7fbcf0947640 1 --2- 172.21.15.43:0/3890875532 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fbcec0cf710 0x7fbcec15ab20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-08-09T13:34:44.646 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:44.642+0000 7fbcebfff640 1 --2- 172.21.15.43:0/3890875532 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fbcec0d1cf0 0x7fbcec0cb510 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7fbce000d800 tx=0x7fbce000dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-09T13:34:44.647 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:44.642+0000 7fbce97fa640 1 -- 172.21.15.43:0/3890875532 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbce000b6c0 con 0x7fbcec0d1cf0 2023-08-09T13:34:44.647 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:44.642+0000 7fbce97fa640 1 -- 172.21.15.43:0/3890875532 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbce000b820 con 0x7fbcec0d1cf0 2023-08-09T13:34:44.647 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:44.642+0000 7fbcf1949640 1 -- 172.21.15.43:0/3890875532 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbcec0a9560 con 0x7fbcec0d1cf0 2023-08-09T13:34:44.647 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:44.642+0000 7fbce97fa640 1 -- 172.21.15.43:0/3890875532 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbce000bbb0 con 0x7fbcec0d1cf0 2023-08-09T13:34:44.647 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:44.642+0000 7fbcf1949640 1 -- 172.21.15.43:0/3890875532 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbcec15b060 con 0x7fbcec0d1cf0 2023-08-09T13:34:44.648 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:44.643+0000 7fbcf1949640 1 -- 172.21.15.43:0/3890875532 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbcec0d2a40 con 0x7fbcec0d1cf0 2023-08-09T13:34:44.648 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:44.644+0000 7fbce97fa640 1 -- 172.21.15.43:0/3890875532 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 5) v1 ==== 88341+0+0 (secure 0 0 0) 0x7fbce000bd10 con 0x7fbcec0d1cf0 2023-08-09T13:34:44.648 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:44.645+0000 7fbce97fa640 1 --2- 172.21.15.43:0/3890875532 >> [v2:172.21.15.203:6832/830097474,v1:172.21.15.203:6833/830097474] conn(0x7fbcd806a050 0x7fbcd806c510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:34:44.648 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:44.645+0000 7fbce97fa640 1 -- 172.21.15.43:0/3890875532 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6813+0+0 (secure 0 0 0) 0x7fbce0088d80 con 0x7fbcec0d1cf0 2023-08-09T13:34:44.649 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:44.646+0000 7fbcf0947640 1 --2- 172.21.15.43:0/3890875532 >> [v2:172.21.15.203:6832/830097474,v1:172.21.15.203:6833/830097474] conn(0x7fbcd806a050 0x7fbcd806c510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:34:44.649 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:44.647+0000 7fbcf0947640 1 --2- 172.21.15.43:0/3890875532 >> [v2:172.21.15.203:6832/830097474,v1:172.21.15.203:6833/830097474] conn(0x7fbcd806a050 0x7fbcd806c510 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7fbcdc036000 tx=0x7fbcdc030000 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-09T13:34:44.652 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:44.650+0000 7fbce97fa640 1 -- 172.21.15.43:0/3890875532 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178066 (secure 0 0 0) 0x7fbcec0d2a40 con 0x7fbcec0d1cf0 2023-08-09T13:34:44.846 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:44.843+0000 7fbcf1949640 1 -- 172.21.15.43:0/3890875532 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7fbcec0d2a10 con 0x7fbcec0d1cf0 2023-08-09T13:34:44.847 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:44.844+0000 7fbce97fa640 1 -- 172.21.15.43:0/3890875532 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v24) v1 ==== 93+0+10 (secure 0 0 0) 0x7fbce0057ca0 con 0x7fbcec0d1cf0 2023-08-09T13:34:44.847 INFO:teuthology.orchestra.run.smithi043.stdout:pg_num: 1 2023-08-09T13:34:44.850 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:44.847+0000 7fbcf1949640 1 -- 172.21.15.43:0/3890875532 >> [v2:172.21.15.203:6832/830097474,v1:172.21.15.203:6833/830097474] conn(0x7fbcd806a050 msgr2=0x7fbcd806c510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:34:44.850 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:44.847+0000 7fbcf1949640 1 --2- 172.21.15.43:0/3890875532 >> [v2:172.21.15.203:6832/830097474,v1:172.21.15.203:6833/830097474] conn(0x7fbcd806a050 0x7fbcd806c510 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7fbcdc036000 tx=0x7fbcdc030000 comp rx=0 tx=0).stop 2023-08-09T13:34:44.850 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:44.847+0000 7fbcf1949640 1 -- 172.21.15.43:0/3890875532 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fbcec0d1cf0 msgr2=0x7fbcec0cb510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:34:44.850 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:44.847+0000 7fbcf1949640 1 --2- 172.21.15.43:0/3890875532 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fbcec0d1cf0 0x7fbcec0cb510 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7fbce000d800 tx=0x7fbce000dcc0 comp rx=0 tx=0).stop 2023-08-09T13:34:44.850 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:44.847+0000 7fbcf1949640 1 -- 172.21.15.43:0/3890875532 shutdown_connections 2023-08-09T13:34:44.850 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:44.847+0000 7fbcf1949640 1 --2- 172.21.15.43:0/3890875532 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fbcec15f970 0x7fbcec0cba50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:34:44.850 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:44.847+0000 7fbcf1949640 1 --2- 172.21.15.43:0/3890875532 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fbcec0d1cf0 0x7fbcec0cb510 unknown :-1 s=CLOSED pgs=223 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:34:44.851 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:44.847+0000 7fbcf1949640 1 --2- 172.21.15.43:0/3890875532 >> [v2:172.21.15.203:6832/830097474,v1:172.21.15.203:6833/830097474] conn(0x7fbcd806a050 0x7fbcd806c510 unknown :-1 s=CLOSED pgs=134 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:34:44.851 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:44.847+0000 7fbcf1949640 1 --2- 172.21.15.43:0/3890875532 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fbcec0cf710 0x7fbcec15ab20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:34:44.851 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:44.847+0000 7fbcf1949640 1 -- 172.21.15.43:0/3890875532 >> 172.21.15.43:0/3890875532 conn(0x7fbcec0712e0 msgr2=0x7fbcec0b5570 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-09T13:34:44.851 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:44.847+0000 7fbcf1949640 1 -- 172.21.15.43:0/3890875532 shutdown_connections 2023-08-09T13:34:44.851 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:44.847+0000 7fbcf1949640 1 -- 172.21.15.43:0/3890875532 wait complete. 2023-08-09T13:34:44.869 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-08-09T13:34:45.101 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:45.098+0000 7f7cbb59e640 1 Processor -- start 2023-08-09T13:34:45.101 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:45.098+0000 7f7cbb59e640 1 -- start start 2023-08-09T13:34:45.102 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:45.100+0000 7f7cbb59e640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f7cbc1590b0 0x7f7cbc0cafa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:34:45.103 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:45.100+0000 7f7cbb59e640 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7f7cbc0c96a0 0x7f7cbc0cb9f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:34:45.103 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:45.100+0000 7f7cbb59e640 1 -- --> v1:172.21.15.203:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7cbc0b07e0 con 0x7f7cbc06fd40 2023-08-09T13:34:45.103 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:45.100+0000 7f7cbb59e640 1 -- --> v2:172.21.15.43:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7cbc0701d0 con 0x7f7cbc1590b0 2023-08-09T13:34:45.103 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:45.100+0000 7f7cbb59e640 1 -- --> v2:172.21.15.203:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7cbc070420 con 0x7f7cbc0c96a0 2023-08-09T13:34:45.104 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:45.100+0000 7f7cba59c640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f7cbc1590b0 0x7f7cbc0cafa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:34:45.104 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:45.101+0000 7f7cba59c640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f7cbc1590b0 0x7f7cbc0cafa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:44100/0 (socket says 172.21.15.43:44100) 2023-08-09T13:34:45.104 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:45.101+0000 7f7cba59c640 1 -- 172.21.15.43:0/208994064 learned_addr learned my addr 172.21.15.43:0/208994064 (peer_addr_for_me v2:172.21.15.43:0/0) 2023-08-09T13:34:45.104 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:45.101+0000 7f7cba59c640 1 -- 172.21.15.43:0/208994064 >> v2:172.21.15.203:3300/0 conn(0x7f7cbc0c96a0 msgr2=0x7f7cbc0cb9f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-09T13:34:45.104 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:45.101+0000 7f7cba59c640 1 --2- 172.21.15.43:0/208994064 >> v2:172.21.15.203:3300/0 conn(0x7f7cbc0c96a0 0x7f7cbc0cb9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:34:45.105 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:45.101+0000 7f7cba59c640 1 -- 172.21.15.43:0/208994064 >> v1:172.21.15.203:6789/0 conn(0x7f7cbc06fd40 legacy=0x7f7cbc0c8f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-09T13:34:45.105 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:45.101+0000 7f7cba59c640 1 -- 172.21.15.43:0/208994064 --> v2:172.21.15.43:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7cbc070700 con 0x7f7cbc1590b0 2023-08-09T13:34:45.106 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:45.102+0000 7f7cba59c640 1 --2- 172.21.15.43:0/208994064 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f7cbc1590b0 0x7f7cbc0cafa0 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7f7cbc06f250 tx=0x7f7cac02b910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f29293bf1ffd3d2a server_cookie=0 in_seq=0 out_seq=0 2023-08-09T13:34:45.106 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:45.103+0000 7f7cb9d9b640 1 -- 172.21.15.43:0/208994064 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7cac031870 con 0x7f7cbc1590b0 2023-08-09T13:34:45.106 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:45.103+0000 7f7cb9d9b640 1 -- 172.21.15.43:0/208994064 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7cac0319d0 con 0x7f7cbc1590b0 2023-08-09T13:34:45.106 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:45.103+0000 7f7cb9d9b640 1 -- 172.21.15.43:0/208994064 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7cac031d60 con 0x7f7cbc1590b0 2023-08-09T13:34:45.106 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:45.103+0000 7f7cbb59e640 1 -- 172.21.15.43:0/208994064 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f7cbc1590b0 msgr2=0x7f7cbc0cafa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:34:45.106 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:45.103+0000 7f7cbb59e640 1 --2- 172.21.15.43:0/208994064 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f7cbc1590b0 0x7f7cbc0cafa0 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7f7cbc06f250 tx=0x7f7cac02b910 comp rx=0 tx=0).stop 2023-08-09T13:34:45.107 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:45.103+0000 7f7cbb59e640 1 -- 172.21.15.43:0/208994064 shutdown_connections 2023-08-09T13:34:45.107 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:45.103+0000 7f7cbb59e640 1 --2- 172.21.15.43:0/208994064 >> v2:172.21.15.203:3300/0 conn(0x7f7cbc0c96a0 0x7f7cbc0cb9f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:34:45.107 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:45.103+0000 7f7cbb59e640 1 --2- 172.21.15.43:0/208994064 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f7cbc1590b0 0x7f7cbc0cafa0 unknown :-1 s=CLOSED pgs=224 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:34:45.107 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:45.103+0000 7f7cbb59e640 1 -- 172.21.15.43:0/208994064 >> 172.21.15.43:0/208994064 conn(0x7f7cbc0af9f0 msgr2=0x7f7cbc0afdd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-09T13:34:45.107 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:45.103+0000 7f7cbb59e640 1 -- 172.21.15.43:0/208994064 shutdown_connections 2023-08-09T13:34:45.107 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:45.103+0000 7f7cbb59e640 1 -- 172.21.15.43:0/208994064 wait complete. 2023-08-09T13:34:45.108 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:45.105+0000 7f7cbb59e640 1 Processor -- start 2023-08-09T13:34:45.108 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:45.105+0000 7f7cbb59e640 1 -- start start 2023-08-09T13:34:45.109 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:45.106+0000 7f7cbb59e640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f7cbc06fd40 0x7f7cbc0b2d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:34:45.109 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:45.106+0000 7f7cbb59e640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f7cbc0c96a0 0x7f7cbc0aeb10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:34:45.109 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:45.106+0000 7f7cbb59e640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f7cbc1590b0 0x7f7cbc148bb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:34:45.109 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:45.106+0000 7f7cbb59e640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7cbc1f5600 con 0x7f7cbc1590b0 2023-08-09T13:34:45.109 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:45.106+0000 7f7cbb59e640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7cbc1f49f0 con 0x7f7cbc06fd40 2023-08-09T13:34:45.110 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:45.106+0000 7f7cbb59e640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7cbc1f4cd0 con 0x7f7cbc0c96a0 2023-08-09T13:34:45.111 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:45.107+0000 7f7cbad9d640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f7cbc1590b0 0x7f7cbc148bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:34:45.111 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:45.107+0000 7f7cbad9d640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f7cbc1590b0 0x7f7cbc148bb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:44104/0 (socket says 172.21.15.43:44104) 2023-08-09T13:34:45.111 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:45.107+0000 7f7cbad9d640 1 -- 172.21.15.43:0/101549383 learned_addr learned my addr 172.21.15.43:0/101549383 (peer_addr_for_me v2:172.21.15.43:0/0) 2023-08-09T13:34:45.111 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:45.107+0000 7f7cb3fff640 1 --2- 172.21.15.43:0/101549383 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f7cbc0c96a0 0x7f7cbc0aeb10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:34:45.111 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:45.107+0000 7f7cba59c640 1 --2- 172.21.15.43:0/101549383 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f7cbc06fd40 0x7f7cbc0b2d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:34:45.111 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:45.107+0000 7f7cbad9d640 1 -- 172.21.15.43:0/101549383 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f7cbc0c96a0 msgr2=0x7f7cbc0aeb10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:34:45.112 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:45.107+0000 7f7cbad9d640 1 --2- 172.21.15.43:0/101549383 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f7cbc0c96a0 0x7f7cbc0aeb10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:34:45.112 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:45.107+0000 7f7cbad9d640 1 -- 172.21.15.43:0/101549383 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f7cbc06fd40 msgr2=0x7f7cbc0b2d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:34:45.112 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:45.107+0000 7f7cbad9d640 1 --2- 172.21.15.43:0/101549383 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f7cbc06fd40 0x7f7cbc0b2d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:34:45.112 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:45.107+0000 7f7cbad9d640 1 -- 172.21.15.43:0/101549383 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7cac0094d0 con 0x7f7cbc1590b0 2023-08-09T13:34:45.112 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:45.107+0000 7f7cb3fff640 1 --2- 172.21.15.43:0/101549383 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f7cbc0c96a0 0x7f7cbc0aeb10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-08-09T13:34:45.112 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:45.108+0000 7f7cbad9d640 1 --2- 172.21.15.43:0/101549383 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f7cbc1590b0 0x7f7cbc148bb0 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7f7cb400b9e0 tx=0x7f7cb400bea0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-09T13:34:45.112 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:45.108+0000 7f7cb2ffd640 1 -- 172.21.15.43:0/101549383 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7cb400cb20 con 0x7f7cbc1590b0 2023-08-09T13:34:45.113 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:45.108+0000 7f7cb2ffd640 1 -- 172.21.15.43:0/101549383 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7cb400cc80 con 0x7f7cbc1590b0 2023-08-09T13:34:45.113 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:45.108+0000 7f7cbb59e640 1 -- 172.21.15.43:0/101549383 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7cbc1f4fb0 con 0x7f7cbc1590b0 2023-08-09T13:34:45.113 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:45.108+0000 7f7cb2ffd640 1 -- 172.21.15.43:0/101549383 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7cb4014680 con 0x7f7cbc1590b0 2023-08-09T13:34:45.113 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:45.108+0000 7f7cbb59e640 1 -- 172.21.15.43:0/101549383 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7cbc0af050 con 0x7f7cbc1590b0 2023-08-09T13:34:45.113 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:45.109+0000 7f7cbb59e640 1 -- 172.21.15.43:0/101549383 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7cbc04e390 con 0x7f7cbc1590b0 2023-08-09T13:34:45.114 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:45.110+0000 7f7cb2ffd640 1 -- 172.21.15.43:0/101549383 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 5) v1 ==== 88341+0+0 (secure 0 0 0) 0x7f7cb401d3b0 con 0x7f7cbc1590b0 2023-08-09T13:34:45.114 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:45.111+0000 7f7cb2ffd640 1 --2- 172.21.15.43:0/101549383 >> [v2:172.21.15.203:6832/830097474,v1:172.21.15.203:6833/830097474] conn(0x7f7c9c06a0a0 0x7f7c9c06c560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:34:45.114 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:45.112+0000 7f7cb2ffd640 1 -- 172.21.15.43:0/101549383 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6813+0+0 (secure 0 0 0) 0x7f7cb4088840 con 0x7f7cbc1590b0 2023-08-09T13:34:45.114 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:45.112+0000 7f7cba59c640 1 --2- 172.21.15.43:0/101549383 >> [v2:172.21.15.203:6832/830097474,v1:172.21.15.203:6833/830097474] conn(0x7f7c9c06a0a0 0x7f7c9c06c560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:34:45.115 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:45.113+0000 7f7cba59c640 1 --2- 172.21.15.43:0/101549383 >> [v2:172.21.15.203:6832/830097474,v1:172.21.15.203:6833/830097474] conn(0x7f7c9c06a0a0 0x7f7c9c06c560 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7f7cac036000 tx=0x7f7cac03c040 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-09T13:34:45.118 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:45.115+0000 7f7cb2ffd640 1 -- 172.21.15.43:0/101549383 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178066 (secure 0 0 0) 0x7f7cb4053c20 con 0x7f7cbc1590b0 2023-08-09T13:34:45.314 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:45.311+0000 7f7cbb59e640 1 -- 172.21.15.43:0/101549383 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f7cbc073110 con 0x7f7cbc1590b0 2023-08-09T13:34:45.315 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:45.312+0000 7f7cb2ffd640 1 -- 172.21.15.43:0/101549383 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v24) v1 ==== 104+0+11 (secure 0 0 0) 0x7f7cb4057240 con 0x7f7cbc1590b0 2023-08-09T13:34:45.315 INFO:teuthology.orchestra.run.smithi043.stdout:pg_num: 32 2023-08-09T13:34:45.318 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:45.315+0000 7f7cbb59e640 1 -- 172.21.15.43:0/101549383 >> [v2:172.21.15.203:6832/830097474,v1:172.21.15.203:6833/830097474] conn(0x7f7c9c06a0a0 msgr2=0x7f7c9c06c560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:34:45.318 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:45.315+0000 7f7cbb59e640 1 --2- 172.21.15.43:0/101549383 >> [v2:172.21.15.203:6832/830097474,v1:172.21.15.203:6833/830097474] conn(0x7f7c9c06a0a0 0x7f7c9c06c560 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7f7cac036000 tx=0x7f7cac03c040 comp rx=0 tx=0).stop 2023-08-09T13:34:45.318 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:45.315+0000 7f7cbb59e640 1 -- 172.21.15.43:0/101549383 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f7cbc1590b0 msgr2=0x7f7cbc148bb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:34:45.318 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:45.315+0000 7f7cbb59e640 1 --2- 172.21.15.43:0/101549383 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f7cbc1590b0 0x7f7cbc148bb0 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7f7cb400b9e0 tx=0x7f7cb400bea0 comp rx=0 tx=0).stop 2023-08-09T13:34:45.318 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:45.315+0000 7f7cbb59e640 1 -- 172.21.15.43:0/101549383 shutdown_connections 2023-08-09T13:34:45.318 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:45.315+0000 7f7cbb59e640 1 --2- 172.21.15.43:0/101549383 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f7cbc1590b0 0x7f7cbc148bb0 unknown :-1 s=CLOSED pgs=225 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:34:45.319 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:45.315+0000 7f7cbb59e640 1 --2- 172.21.15.43:0/101549383 >> [v2:172.21.15.203:6832/830097474,v1:172.21.15.203:6833/830097474] conn(0x7f7c9c06a0a0 0x7f7c9c06c560 unknown :-1 s=CLOSED pgs=135 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:34:45.319 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:45.315+0000 7f7cbb59e640 1 --2- 172.21.15.43:0/101549383 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f7cbc0c96a0 0x7f7cbc0aeb10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:34:45.319 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:45.315+0000 7f7cbb59e640 1 --2- 172.21.15.43:0/101549383 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f7cbc06fd40 0x7f7cbc0b2d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:34:45.319 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:45.315+0000 7f7cbb59e640 1 -- 172.21.15.43:0/101549383 >> 172.21.15.43:0/101549383 conn(0x7f7cbc0af9f0 msgr2=0x7f7cbc0abb70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-09T13:34:45.319 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:45.315+0000 7f7cbb59e640 1 -- 172.21.15.43:0/101549383 shutdown_connections 2023-08-09T13:34:45.319 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:45.315+0000 7f7cbb59e640 1 -- 172.21.15.43:0/101549383 wait complete. 2023-08-09T13:34:45.337 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-08-09T13:34:45.569 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:45.566+0000 7f013a463640 1 Processor -- start 2023-08-09T13:34:45.569 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:45.567+0000 7f013a463640 1 -- start start 2023-08-09T13:34:45.571 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:45.568+0000 7f013a463640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f01340cb9a0 0x7f01340cbde0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:34:45.571 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:45.568+0000 7f013a463640 1 -- --> v1:172.21.15.43:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f01340acfa0 con 0x7f0134070ef0 2023-08-09T13:34:45.571 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:45.568+0000 7f013a463640 1 -- --> v1:172.21.15.203:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f01340cc320 con 0x7f013406f310 2023-08-09T13:34:45.571 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:45.568+0000 7f013a463640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f013415c940 con 0x7f01340cb9a0 2023-08-09T13:34:45.571 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:45.569+0000 7f0139461640 1 --1- >> v1:172.21.15.43:6789/0 conn(0x7f0134070ef0 0x7f01340ab8d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.43:6789/0 says I am v1:172.21.15.43:42388/0 (socket says 172.21.15.43:42388) 2023-08-09T13:34:45.572 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:45.569+0000 7f0139461640 1 -- 172.21.15.43:0/316348914 learned_addr learned my addr 172.21.15.43:0/316348914 (peer_addr_for_me v1:172.21.15.43:0/0) 2023-08-09T13:34:45.574 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:45.569+0000 7f0138c60640 1 --2- 172.21.15.43:0/316348914 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f01340cb9a0 0x7f01340cbde0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:34:45.574 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:45.569+0000 7f012bfff640 1 -- 172.21.15.43:0/316348914 <== mon.0 v1:172.21.15.43:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 730517020 0 0) 0x7f01340acfa0 con 0x7f0134070ef0 2023-08-09T13:34:45.575 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:45.569+0000 7f012bfff640 1 -- 172.21.15.43:0/316348914 --> v1:172.21.15.43:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f0120003520 con 0x7f0134070ef0 2023-08-09T13:34:45.575 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:45.569+0000 7f012bfff640 1 -- 172.21.15.43:0/316348914 <== mon.0 v1:172.21.15.43:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 4242113235 0 0) 0x7f0120003520 con 0x7f0134070ef0 2023-08-09T13:34:45.575 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:45.569+0000 7f012bfff640 1 -- 172.21.15.43:0/316348914 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f01340cb9a0 msgr2=0x7f01340cbde0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-09T13:34:45.575 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:45.569+0000 7f012bfff640 1 --2- 172.21.15.43:0/316348914 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f01340cb9a0 0x7f01340cbde0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:34:45.575 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:45.569+0000 7f012bfff640 1 -- 172.21.15.43:0/316348914 >> v1:172.21.15.203:6789/0 conn(0x7f013406f310 legacy=0x7f013406f7a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:34:45.575 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:45.569+0000 7f012bfff640 1 -- 172.21.15.43:0/316348914 --> v1:172.21.15.43:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f01340ac390 con 0x7f0134070ef0 2023-08-09T13:34:45.575 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:45.569+0000 7f012bfff640 1 -- 172.21.15.43:0/316348914 <== mon.0 v1:172.21.15.43:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1285150740 0 0) 0x7f01300019f0 con 0x7f0134070ef0 2023-08-09T13:34:45.575 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:45.569+0000 7f012bfff640 1 -- 172.21.15.43:0/316348914 >> v1:172.21.15.43:6789/0 conn(0x7f0134070ef0 legacy=0x7f01340ab8d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:34:45.576 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:45.570+0000 7f012bfff640 1 --2- 172.21.15.43:0/316348914 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f01200039a0 0x7f0120003dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:34:45.576 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:45.570+0000 7f012bfff640 1 --2- 172.21.15.43:0/316348914 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f01340cb9a0 0x7f01200047c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:34:45.576 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:45.570+0000 7f012bfff640 1 --2- 172.21.15.43:0/316348914 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f0120004d00 0x7f0120005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:34:45.576 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:45.570+0000 7f012bfff640 1 -- 172.21.15.43:0/316348914 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f013415c940 con 0x7f01200039a0 2023-08-09T13:34:45.576 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:45.570+0000 7f012bfff640 1 -- 172.21.15.43:0/316348914 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f012000bba0 con 0x7f0120004d00 2023-08-09T13:34:45.576 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:45.570+0000 7f012bfff640 1 -- 172.21.15.43:0/316348914 --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f012000bd30 con 0x7f01340cb9a0 2023-08-09T13:34:45.576 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:45.570+0000 7f0139461640 1 --2- 172.21.15.43:0/316348914 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f01200039a0 0x7f0120003dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:34:45.577 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:45.570+0000 7f0139c62640 1 --2- 172.21.15.43:0/316348914 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f0120004d00 0x7f0120005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:34:45.577 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:45.570+0000 7f0139461640 1 -- 172.21.15.43:0/316348914 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f01340cb9a0 msgr2=0x7f01200047c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-09T13:34:45.577 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:45.570+0000 7f0138c60640 1 --2- 172.21.15.43:0/316348914 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f01340cb9a0 0x7f01200047c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:34:45.577 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:45.570+0000 7f0139461640 1 --2- 172.21.15.43:0/316348914 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f01340cb9a0 0x7f01200047c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:34:45.577 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:45.570+0000 7f0139461640 1 -- 172.21.15.43:0/316348914 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f0120004d00 msgr2=0x7f0120005180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-09T13:34:45.577 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:45.570+0000 7f0139461640 1 --2- 172.21.15.43:0/316348914 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f0120004d00 0x7f0120005180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:34:45.577 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:45.570+0000 7f0139461640 1 -- 172.21.15.43:0/316348914 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0120003520 con 0x7f01200039a0 2023-08-09T13:34:45.577 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:45.570+0000 7f0138c60640 1 --2- 172.21.15.43:0/316348914 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f01340cb9a0 0x7f01200047c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-08-09T13:34:45.578 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:45.572+0000 7f0139461640 1 --2- 172.21.15.43:0/316348914 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f01200039a0 0x7f0120003dc0 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7f013000a860 tx=0x7f01200057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f9a92809ba66c4d server_cookie=0 in_seq=0 out_seq=0 2023-08-09T13:34:45.578 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:45.572+0000 7f012bfff640 1 -- 172.21.15.43:0/316348914 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f013002cea0 con 0x7f01200039a0 2023-08-09T13:34:45.578 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:45.572+0000 7f012bfff640 1 -- 172.21.15.43:0/316348914 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f013002f070 con 0x7f01200039a0 2023-08-09T13:34:45.578 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:45.572+0000 7f012bfff640 1 -- 172.21.15.43:0/316348914 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0130007550 con 0x7f01200039a0 2023-08-09T13:34:45.580 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:45.572+0000 7f013a463640 1 -- 172.21.15.43:0/316348914 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f01200039a0 msgr2=0x7f0120003dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:34:45.580 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:45.572+0000 7f013a463640 1 --2- 172.21.15.43:0/316348914 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f01200039a0 0x7f0120003dc0 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7f013000a860 tx=0x7f01200057f0 comp rx=0 tx=0).stop 2023-08-09T13:34:45.580 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:45.572+0000 7f0139c62640 1 -- 172.21.15.43:0/316348914 reap_dead start 2023-08-09T13:34:45.580 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:45.572+0000 7f013a463640 1 -- 172.21.15.43:0/316348914 shutdown_connections 2023-08-09T13:34:45.581 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:45.572+0000 7f013a463640 1 -- 172.21.15.43:0/316348914 >> 172.21.15.43:0/316348914 conn(0x7f01340cddd0 msgr2=0x7f01340ce1e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-09T13:34:45.581 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:45.573+0000 7f013a463640 1 -- 172.21.15.43:0/316348914 shutdown_connections 2023-08-09T13:34:45.581 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:45.573+0000 7f013a463640 1 -- 172.21.15.43:0/316348914 wait complete. 2023-08-09T13:34:45.581 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:45.574+0000 7f013a463640 1 Processor -- start 2023-08-09T13:34:45.581 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:45.575+0000 7f013a463640 1 -- start start 2023-08-09T13:34:45.581 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:45.575+0000 7f013a463640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f01200039a0 0x7f01340c38e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:34:45.581 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:45.575+0000 7f013a463640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f013415d100 0x7f013415d580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:34:45.582 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:45.575+0000 7f013a463640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f01341f1a60 0x7f01341f1ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:34:45.582 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:45.575+0000 7f013a463640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f013415dac0 con 0x7f013415d100 2023-08-09T13:34:45.582 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:45.575+0000 7f013a463640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f01340b2650 con 0x7f01341f1a60 2023-08-09T13:34:45.582 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:45.575+0000 7f013a463640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f01340b2930 con 0x7f01200039a0 2023-08-09T13:34:45.582 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:45.576+0000 7f0138c60640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f013415d100 0x7f013415d580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:34:45.582 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:45.576+0000 7f0138c60640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f013415d100 0x7f013415d580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:44118/0 (socket says 172.21.15.43:44118) 2023-08-09T13:34:45.583 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:45.576+0000 7f0138c60640 1 -- 172.21.15.43:0/667850769 learned_addr learned my addr 172.21.15.43:0/667850769 (peer_addr_for_me v2:172.21.15.43:0/0) 2023-08-09T13:34:45.583 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:45.576+0000 7f0139461640 1 --2- 172.21.15.43:0/667850769 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f01200039a0 0x7f01340c38e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:34:45.583 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:45.576+0000 7f0139c62640 1 --2- 172.21.15.43:0/667850769 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f01341f1a60 0x7f01341f1ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:34:45.583 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:45.576+0000 7f0138c60640 1 -- 172.21.15.43:0/667850769 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f01200039a0 msgr2=0x7f01340c38e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:34:45.583 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:45.576+0000 7f0138c60640 1 --2- 172.21.15.43:0/667850769 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f01200039a0 0x7f01340c38e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:34:45.583 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:45.576+0000 7f0138c60640 1 -- 172.21.15.43:0/667850769 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f01341f1a60 msgr2=0x7f01341f1ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:34:45.583 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:45.576+0000 7f0138c60640 1 --2- 172.21.15.43:0/667850769 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f01341f1a60 0x7f01341f1ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:34:45.584 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:45.576+0000 7f0138c60640 1 -- 172.21.15.43:0/667850769 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f01340b2c10 con 0x7f013415d100 2023-08-09T13:34:45.584 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:45.577+0000 7f0138c60640 1 --2- 172.21.15.43:0/667850769 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f013415d100 0x7f013415d580 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7f012400e7e0 tx=0x7f012400eca0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-09T13:34:45.584 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:45.577+0000 7f0129ffb640 1 -- 172.21.15.43:0/667850769 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f012400c6d0 con 0x7f013415d100 2023-08-09T13:34:45.584 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:45.577+0000 7f0129ffb640 1 -- 172.21.15.43:0/667850769 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f012400c830 con 0x7f013415d100 2023-08-09T13:34:45.584 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:45.577+0000 7f0129ffb640 1 -- 172.21.15.43:0/667850769 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f012400cba0 con 0x7f013415d100 2023-08-09T13:34:45.584 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:45.577+0000 7f013a463640 1 -- 172.21.15.43:0/667850769 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f01340b2ef0 con 0x7f013415d100 2023-08-09T13:34:45.584 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:45.577+0000 7f013a463640 1 -- 172.21.15.43:0/667850769 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f01340b36f0 con 0x7f013415d100 2023-08-09T13:34:45.585 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:45.579+0000 7f013a463640 1 -- 172.21.15.43:0/667850769 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f013404e390 con 0x7f013415d100 2023-08-09T13:34:45.585 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:45.579+0000 7f0129ffb640 1 -- 172.21.15.43:0/667850769 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 5) v1 ==== 88341+0+0 (secure 0 0 0) 0x7f012400cd00 con 0x7f013415d100 2023-08-09T13:34:45.586 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:45.580+0000 7f0129ffb640 1 --2- 172.21.15.43:0/667850769 >> [v2:172.21.15.203:6832/830097474,v1:172.21.15.203:6833/830097474] conn(0x7f011806a080 0x7f011806c540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:34:45.586 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:45.581+0000 7f0129ffb640 1 -- 172.21.15.43:0/667850769 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6813+0+0 (secure 0 0 0) 0x7f01240885e0 con 0x7f013415d100 2023-08-09T13:34:45.586 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:45.581+0000 7f0139461640 1 --2- 172.21.15.43:0/667850769 >> [v2:172.21.15.203:6832/830097474,v1:172.21.15.203:6833/830097474] conn(0x7f011806a080 0x7f011806c540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:34:45.586 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:45.582+0000 7f0139461640 1 --2- 172.21.15.43:0/667850769 >> [v2:172.21.15.203:6832/830097474,v1:172.21.15.203:6833/830097474] conn(0x7f011806a080 0x7f011806c540 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7f0130034000 tx=0x7f0130007000 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-09T13:34:45.587 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:45.584+0000 7f0129ffb640 1 -- 172.21.15.43:0/667850769 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178066 (secure 0 0 0) 0x7f0124053940 con 0x7f013415d100 2023-08-09T13:34:45.784 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:45.782+0000 7f013a463640 1 -- 172.21.15.43:0/667850769 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f01340b3d50 con 0x7f013415d100 2023-08-09T13:34:45.785 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:45.783+0000 7f0129ffb640 1 -- 172.21.15.43:0/667850769 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v24) v1 ==== 100+0+11 (secure 0 0 0) 0x7f0124056f60 con 0x7f013415d100 2023-08-09T13:34:45.785 INFO:teuthology.orchestra.run.smithi043.stdout:pg_num: 64 2023-08-09T13:34:45.788 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:45.785+0000 7f013a463640 1 -- 172.21.15.43:0/667850769 >> [v2:172.21.15.203:6832/830097474,v1:172.21.15.203:6833/830097474] conn(0x7f011806a080 msgr2=0x7f011806c540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:34:45.788 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:45.785+0000 7f013a463640 1 --2- 172.21.15.43:0/667850769 >> [v2:172.21.15.203:6832/830097474,v1:172.21.15.203:6833/830097474] conn(0x7f011806a080 0x7f011806c540 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7f0130034000 tx=0x7f0130007000 comp rx=0 tx=0).stop 2023-08-09T13:34:45.789 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:45.785+0000 7f013a463640 1 -- 172.21.15.43:0/667850769 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f013415d100 msgr2=0x7f013415d580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:34:45.789 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:45.785+0000 7f013a463640 1 --2- 172.21.15.43:0/667850769 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f013415d100 0x7f013415d580 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7f012400e7e0 tx=0x7f012400eca0 comp rx=0 tx=0).stop 2023-08-09T13:34:45.789 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:45.786+0000 7f013a463640 1 -- 172.21.15.43:0/667850769 shutdown_connections 2023-08-09T13:34:45.789 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:45.786+0000 7f013a463640 1 --2- 172.21.15.43:0/667850769 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f01341f1a60 0x7f01341f1ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:34:45.789 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:45.786+0000 7f013a463640 1 --2- 172.21.15.43:0/667850769 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f013415d100 0x7f013415d580 unknown :-1 s=CLOSED pgs=228 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:34:45.790 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:45.786+0000 7f013a463640 1 --2- 172.21.15.43:0/667850769 >> [v2:172.21.15.203:6832/830097474,v1:172.21.15.203:6833/830097474] conn(0x7f011806a080 0x7f011806c540 unknown :-1 s=CLOSED pgs=136 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:34:45.790 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:45.786+0000 7f013a463640 1 --2- 172.21.15.43:0/667850769 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f01200039a0 0x7f01340c38e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:34:45.790 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:45.786+0000 7f013a463640 1 -- 172.21.15.43:0/667850769 >> 172.21.15.43:0/667850769 conn(0x7f01340cddd0 msgr2=0x7f01340ab8d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-09T13:34:45.790 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:45.786+0000 7f013a463640 1 -- 172.21.15.43:0/667850769 shutdown_connections 2023-08-09T13:34:45.790 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:45.786+0000 7f013a463640 1 -- 172.21.15.43:0/667850769 wait complete. 2023-08-09T13:34:45.807 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2023-08-09T13:34:46.040 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:46.037+0000 7fa5affff640 1 Processor -- start 2023-08-09T13:34:46.040 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:46.037+0000 7fa5affff640 1 -- start start 2023-08-09T13:34:46.042 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:46.039+0000 7fa5affff640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fa5b0157ce0 0x7fa5b015b0f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:34:46.042 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:46.039+0000 7fa5affff640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7fa5b00c8470 0x7fa5b00c88c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:34:46.042 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:46.039+0000 7fa5affff640 1 -- --> v1:172.21.15.203:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa5b00c8e00 con 0x7fa5b006f2c0 2023-08-09T13:34:46.042 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:46.039+0000 7fa5affff640 1 -- --> v2:172.21.15.43:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa5b01580c0 con 0x7fa5b00c8470 2023-08-09T13:34:46.042 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:46.039+0000 7fa5affff640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa5b00c92c0 con 0x7fa5b0157ce0 2023-08-09T13:34:46.043 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:46.040+0000 7fa5af7fe640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7fa5b00c8470 0x7fa5b00c88c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:34:46.043 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:46.040+0000 7fa5af7fe640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7fa5b00c8470 0x7fa5b00c88c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:44132/0 (socket says 172.21.15.43:44132) 2023-08-09T13:34:46.043 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:46.040+0000 7fa5af7fe640 1 -- 172.21.15.43:0/2274943099 learned_addr learned my addr 172.21.15.43:0/2274943099 (peer_addr_for_me v2:172.21.15.43:0/0) 2023-08-09T13:34:46.043 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:46.040+0000 7fa5aeffd640 1 --2- 172.21.15.43:0/2274943099 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fa5b0157ce0 0x7fa5b015b0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:34:46.044 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:46.040+0000 7fa5af7fe640 1 -- 172.21.15.43:0/2274943099 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fa5b0157ce0 msgr2=0x7fa5b015b0f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-09T13:34:46.044 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:46.040+0000 7fa5af7fe640 1 --2- 172.21.15.43:0/2274943099 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fa5b0157ce0 0x7fa5b015b0f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:34:46.044 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:46.040+0000 7fa5af7fe640 1 -- 172.21.15.43:0/2274943099 >> v1:172.21.15.203:6789/0 conn(0x7fa5b006f2c0 legacy=0x7fa5b006bc40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:34:46.044 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:46.040+0000 7fa5af7fe640 1 -- 172.21.15.43:0/2274943099 --> v2:172.21.15.43:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa5b006b480 con 0x7fa5b00c8470 2023-08-09T13:34:46.045 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:46.042+0000 7fa5af7fe640 1 --2- 172.21.15.43:0/2274943099 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa5b00c8470 0x7fa5b00c88c0 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7fa5a8009e50 tx=0x7fa5a802bed0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d96f87536c804829 server_cookie=0 in_seq=0 out_seq=0 2023-08-09T13:34:46.045 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:46.042+0000 7fa5ae7fc640 1 -- 172.21.15.43:0/2274943099 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa5a802ca70 con 0x7fa5b00c8470 2023-08-09T13:34:46.046 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:46.042+0000 7fa5ae7fc640 1 -- 172.21.15.43:0/2274943099 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa5a802cbd0 con 0x7fa5b00c8470 2023-08-09T13:34:46.046 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:46.042+0000 7fa5ae7fc640 1 -- 172.21.15.43:0/2274943099 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa5a80343f0 con 0x7fa5b00c8470 2023-08-09T13:34:46.046 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:46.043+0000 7fa5affff640 1 -- 172.21.15.43:0/2274943099 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa5b00c8470 msgr2=0x7fa5b00c88c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:34:46.046 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:46.043+0000 7fa5affff640 1 --2- 172.21.15.43:0/2274943099 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa5b00c8470 0x7fa5b00c88c0 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7fa5a8009e50 tx=0x7fa5a802bed0 comp rx=0 tx=0).stop 2023-08-09T13:34:46.047 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:46.043+0000 7fa5affff640 1 -- 172.21.15.43:0/2274943099 shutdown_connections 2023-08-09T13:34:46.047 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:46.043+0000 7fa5affff640 1 --2- 172.21.15.43:0/2274943099 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa5b00c8470 0x7fa5b00c88c0 unknown :-1 s=CLOSED pgs=229 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:34:46.047 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:46.043+0000 7fa5affff640 1 --2- 172.21.15.43:0/2274943099 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fa5b0157ce0 0x7fa5b015b0f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:34:46.047 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:46.043+0000 7fa5affff640 1 -- 172.21.15.43:0/2274943099 >> 172.21.15.43:0/2274943099 conn(0x7fa5b00af9f0 msgr2=0x7fa5b00afdd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-09T13:34:46.047 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:46.043+0000 7fa5affff640 1 -- 172.21.15.43:0/2274943099 shutdown_connections 2023-08-09T13:34:46.047 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:46.043+0000 7fa5affff640 1 -- 172.21.15.43:0/2274943099 wait complete. 2023-08-09T13:34:46.048 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:46.045+0000 7fa5affff640 1 Processor -- start 2023-08-09T13:34:46.048 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:46.045+0000 7fa5affff640 1 -- start start 2023-08-09T13:34:46.049 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:46.046+0000 7fa5affff640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fa5b006f2c0 0x7fa5b015d560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:34:46.049 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:46.046+0000 7fa5affff640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fa5b00c8470 0x7fa5b00c29a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:34:46.049 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:46.046+0000 7fa5affff640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa5b0157ce0 0x7fa5b015deb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:34:46.050 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:46.046+0000 7fa5affff640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa5b015daa0 con 0x7fa5b0157ce0 2023-08-09T13:34:46.050 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:46.046+0000 7fa5affff640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa5b0156170 con 0x7fa5b006f2c0 2023-08-09T13:34:46.050 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:46.046+0000 7fa5affff640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa5b0155bb0 con 0x7fa5b00c8470 2023-08-09T13:34:46.050 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:46.046+0000 7fa5af7fe640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa5b0157ce0 0x7fa5b015deb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:34:46.050 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:46.046+0000 7fa5af7fe640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa5b0157ce0 0x7fa5b015deb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:44146/0 (socket says 172.21.15.43:44146) 2023-08-09T13:34:46.050 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:46.046+0000 7fa5af7fe640 1 -- 172.21.15.43:0/1587439576 learned_addr learned my addr 172.21.15.43:0/1587439576 (peer_addr_for_me v2:172.21.15.43:0/0) 2023-08-09T13:34:46.050 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:46.047+0000 7fa5aeffd640 1 --2- 172.21.15.43:0/1587439576 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fa5b006f2c0 0x7fa5b015d560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:34:46.050 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:46.047+0000 7fa5a7fff640 1 --2- 172.21.15.43:0/1587439576 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fa5b00c8470 0x7fa5b00c29a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:34:46.051 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:46.047+0000 7fa5af7fe640 1 -- 172.21.15.43:0/1587439576 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fa5b00c8470 msgr2=0x7fa5b00c29a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:34:46.051 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:46.047+0000 7fa5af7fe640 1 --2- 172.21.15.43:0/1587439576 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fa5b00c8470 0x7fa5b00c29a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:34:46.051 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:46.047+0000 7fa5af7fe640 1 -- 172.21.15.43:0/1587439576 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fa5b006f2c0 msgr2=0x7fa5b015d560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:34:46.051 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:46.047+0000 7fa5af7fe640 1 --2- 172.21.15.43:0/1587439576 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fa5b006f2c0 0x7fa5b015d560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:34:46.051 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:46.047+0000 7fa5af7fe640 1 -- 172.21.15.43:0/1587439576 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa5a8009b30 con 0x7fa5b0157ce0 2023-08-09T13:34:46.052 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:46.047+0000 7fa5af7fe640 1 --2- 172.21.15.43:0/1587439576 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa5b0157ce0 0x7fa5b015deb0 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7fa5a8009e50 tx=0x7fa5a8004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-09T13:34:46.052 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:46.047+0000 7fa5a77fe640 1 -- 172.21.15.43:0/1587439576 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa5a8009510 con 0x7fa5b0157ce0 2023-08-09T13:34:46.052 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:46.047+0000 7fa5a77fe640 1 -- 172.21.15.43:0/1587439576 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa5a803e450 con 0x7fa5b0157ce0 2023-08-09T13:34:46.052 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:46.047+0000 7fa5affff640 1 -- 172.21.15.43:0/1587439576 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa5b0155e90 con 0x7fa5b0157ce0 2023-08-09T13:34:46.053 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:46.047+0000 7fa5a77fe640 1 -- 172.21.15.43:0/1587439576 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa5a803e800 con 0x7fa5b0157ce0 2023-08-09T13:34:46.053 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:46.048+0000 7fa5affff640 1 -- 172.21.15.43:0/1587439576 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa5b015bb90 con 0x7fa5b0157ce0 2023-08-09T13:34:46.053 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:46.049+0000 7fa5affff640 1 -- 172.21.15.43:0/1587439576 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa5b015bd80 con 0x7fa5b0157ce0 2023-08-09T13:34:46.053 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:46.050+0000 7fa5a77fe640 1 -- 172.21.15.43:0/1587439576 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 5) v1 ==== 88341+0+0 (secure 0 0 0) 0x7fa5a803e960 con 0x7fa5b0157ce0 2023-08-09T13:34:46.054 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:46.051+0000 7fa5a77fe640 1 --2- 172.21.15.43:0/1587439576 >> [v2:172.21.15.203:6832/830097474,v1:172.21.15.203:6833/830097474] conn(0x7fa59006a050 0x7fa59006c510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:34:46.054 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:46.051+0000 7fa5a77fe640 1 -- 172.21.15.43:0/1587439576 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6813+0+0 (secure 0 0 0) 0x7fa5a80ab5d0 con 0x7fa5b0157ce0 2023-08-09T13:34:46.054 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:46.051+0000 7fa5aeffd640 1 --2- 172.21.15.43:0/1587439576 >> [v2:172.21.15.203:6832/830097474,v1:172.21.15.203:6833/830097474] conn(0x7fa59006a050 0x7fa59006c510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:34:46.055 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:46.052+0000 7fa5aeffd640 1 --2- 172.21.15.43:0/1587439576 >> [v2:172.21.15.203:6832/830097474,v1:172.21.15.203:6833/830097474] conn(0x7fa59006a050 0x7fa59006c510 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7fa5a0009ad0 tx=0x7fa5a0008040 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-09T13:34:46.057 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:46.055+0000 7fa5a77fe640 1 -- 172.21.15.43:0/1587439576 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178066 (secure 0 0 0) 0x7fa5a80769c0 con 0x7fa5b0157ce0 2023-08-09T13:34:46.257 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:46.255+0000 7fa5affff640 1 -- 172.21.15.43:0/1587439576 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) v1 -- 0x7fa5b004e390 con 0x7fa5b0157ce0 2023-08-09T13:34:46.258 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:46.256+0000 7fa5a77fe640 1 -- 172.21.15.43:0/1587439576 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v24) v1 ==== 103+0+11 (secure 0 0 0) 0x7fa5a8079fe0 con 0x7fa5b0157ce0 2023-08-09T13:34:46.258 INFO:teuthology.orchestra.run.smithi043.stdout:pg_num: 32 2023-08-09T13:34:46.260 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:46.258+0000 7fa5affff640 1 -- 172.21.15.43:0/1587439576 >> [v2:172.21.15.203:6832/830097474,v1:172.21.15.203:6833/830097474] conn(0x7fa59006a050 msgr2=0x7fa59006c510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:34:46.261 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:46.258+0000 7fa5affff640 1 --2- 172.21.15.43:0/1587439576 >> [v2:172.21.15.203:6832/830097474,v1:172.21.15.203:6833/830097474] conn(0x7fa59006a050 0x7fa59006c510 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7fa5a0009ad0 tx=0x7fa5a0008040 comp rx=0 tx=0).stop 2023-08-09T13:34:46.261 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:46.258+0000 7fa5affff640 1 -- 172.21.15.43:0/1587439576 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa5b0157ce0 msgr2=0x7fa5b015deb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:34:46.261 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:46.258+0000 7fa5affff640 1 --2- 172.21.15.43:0/1587439576 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa5b0157ce0 0x7fa5b015deb0 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7fa5a8009e50 tx=0x7fa5a8004a40 comp rx=0 tx=0).stop 2023-08-09T13:34:46.262 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:46.258+0000 7fa5affff640 1 -- 172.21.15.43:0/1587439576 shutdown_connections 2023-08-09T13:34:46.262 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:46.258+0000 7fa5affff640 1 --2- 172.21.15.43:0/1587439576 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa5b0157ce0 0x7fa5b015deb0 unknown :-1 s=CLOSED pgs=230 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:34:46.262 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:46.258+0000 7fa5affff640 1 --2- 172.21.15.43:0/1587439576 >> [v2:172.21.15.203:6832/830097474,v1:172.21.15.203:6833/830097474] conn(0x7fa59006a050 0x7fa59006c510 unknown :-1 s=CLOSED pgs=137 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:34:46.262 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:46.258+0000 7fa5affff640 1 --2- 172.21.15.43:0/1587439576 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fa5b00c8470 0x7fa5b00c29a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:34:46.262 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:46.258+0000 7fa5affff640 1 --2- 172.21.15.43:0/1587439576 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fa5b006f2c0 0x7fa5b015d560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:34:46.262 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:46.258+0000 7fa5affff640 1 -- 172.21.15.43:0/1587439576 >> 172.21.15.43:0/1587439576 conn(0x7fa5b00af9f0 msgr2=0x7fa5b00cba50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-09T13:34:46.263 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:46.259+0000 7fa5affff640 1 -- 172.21.15.43:0/1587439576 shutdown_connections 2023-08-09T13:34:46.263 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:46.259+0000 7fa5affff640 1 -- 172.21.15.43:0/1587439576 wait complete. 2023-08-09T13:34:46.279 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-08-09T13:34:46.504 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:46.502+0000 7f741940a640 1 Processor -- start 2023-08-09T13:34:46.505 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:46.502+0000 7f741940a640 1 -- start start 2023-08-09T13:34:46.506 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:46.503+0000 7f741940a640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f74140d0ef0 0x7f74140cfe70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:34:46.506 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:46.503+0000 7f741940a640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f74140cf4f0 0x7f74140b27c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:34:46.506 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:46.504+0000 7f741940a640 1 -- --> v1:172.21.15.203:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f74140b2330 con 0x7f741415c620 2023-08-09T13:34:46.507 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:46.504+0000 7f741940a640 1 -- --> v2:172.21.15.43:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f74140d2ec0 con 0x7f74140cf4f0 2023-08-09T13:34:46.507 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:46.504+0000 7f741940a640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f74140d0500 con 0x7f74140d0ef0 2023-08-09T13:34:46.508 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:46.504+0000 7f7418c09640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f74140cf4f0 0x7f74140b27c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:34:46.508 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:46.504+0000 7f7418c09640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f74140cf4f0 0x7f74140b27c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:44160/0 (socket says 172.21.15.43:44160) 2023-08-09T13:34:46.508 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:46.504+0000 7f7418c09640 1 -- 172.21.15.43:0/4089843181 learned_addr learned my addr 172.21.15.43:0/4089843181 (peer_addr_for_me v2:172.21.15.43:0/0) 2023-08-09T13:34:46.508 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:46.504+0000 7f7413fff640 1 --1- 172.21.15.43:0/4089843181 >> v1:172.21.15.203:6789/0 conn(0x7f741415c620 0x7f74140d07e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.203:6789/0 says I am v1:172.21.15.43:59100/0 (socket says 172.21.15.43:59100) 2023-08-09T13:34:46.508 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:46.504+0000 7f74137fe640 1 --2- 172.21.15.43:0/4089843181 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f74140d0ef0 0x7f74140cfe70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:34:46.508 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:46.505+0000 7f7418c09640 1 -- 172.21.15.43:0/4089843181 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f74140d0ef0 msgr2=0x7f74140cfe70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-09T13:34:46.508 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:46.505+0000 7f7418c09640 1 --2- 172.21.15.43:0/4089843181 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f74140d0ef0 0x7f74140cfe70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:34:46.509 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:46.505+0000 7f7418c09640 1 -- 172.21.15.43:0/4089843181 >> v1:172.21.15.203:6789/0 conn(0x7f741415c620 legacy=0x7f74140d07e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:34:46.509 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:46.505+0000 7f7418c09640 1 -- 172.21.15.43:0/4089843181 --> v2:172.21.15.43:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f74140b1a00 con 0x7f74140cf4f0 2023-08-09T13:34:46.510 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:46.506+0000 7f7418c09640 1 --2- 172.21.15.43:0/4089843181 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f74140cf4f0 0x7f74140b27c0 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7f740c009e50 tx=0x7f740c02bed0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e2a50abc78374a9b server_cookie=0 in_seq=0 out_seq=0 2023-08-09T13:34:46.510 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:46.506+0000 7f7412ffd640 1 -- 172.21.15.43:0/4089843181 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f740c02ca70 con 0x7f74140cf4f0 2023-08-09T13:34:46.510 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:46.506+0000 7f7412ffd640 1 -- 172.21.15.43:0/4089843181 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f740c02cbd0 con 0x7f74140cf4f0 2023-08-09T13:34:46.510 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:46.506+0000 7f7412ffd640 1 -- 172.21.15.43:0/4089843181 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f740c0343f0 con 0x7f74140cf4f0 2023-08-09T13:34:46.510 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:46.507+0000 7f741940a640 1 -- 172.21.15.43:0/4089843181 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f74140cf4f0 msgr2=0x7f74140b27c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:34:46.510 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:46.507+0000 7f741940a640 1 --2- 172.21.15.43:0/4089843181 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f74140cf4f0 0x7f74140b27c0 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7f740c009e50 tx=0x7f740c02bed0 comp rx=0 tx=0).stop 2023-08-09T13:34:46.511 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:46.507+0000 7f741940a640 1 -- 172.21.15.43:0/4089843181 shutdown_connections 2023-08-09T13:34:46.511 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:46.507+0000 7f741940a640 1 --2- 172.21.15.43:0/4089843181 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f74140cf4f0 0x7f74140b27c0 unknown :-1 s=CLOSED pgs=231 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:34:46.511 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:46.507+0000 7f741940a640 1 --2- 172.21.15.43:0/4089843181 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f74140d0ef0 0x7f74140cfe70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:34:46.511 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:46.507+0000 7f741940a640 1 -- 172.21.15.43:0/4089843181 >> 172.21.15.43:0/4089843181 conn(0x7f741409b650 msgr2=0x7f74140d22a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-09T13:34:46.511 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:46.507+0000 7f741940a640 1 -- 172.21.15.43:0/4089843181 shutdown_connections 2023-08-09T13:34:46.511 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:46.507+0000 7f741940a640 1 -- 172.21.15.43:0/4089843181 wait complete. 2023-08-09T13:34:46.512 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:46.509+0000 7f741940a640 1 Processor -- start 2023-08-09T13:34:46.512 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:46.509+0000 7f741940a640 1 -- start start 2023-08-09T13:34:46.512 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:46.510+0000 7f741940a640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f74140cf4f0 0x7f741414c670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:34:46.513 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:46.510+0000 7f741940a640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f74140d0ef0 0x7f74140c7800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:34:46.513 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:46.510+0000 7f741940a640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f741415c620 0x7f74140b4ec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:34:46.513 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:46.510+0000 7f741940a640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f74140c8280 con 0x7f741415c620 2023-08-09T13:34:46.513 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:46.510+0000 7f741940a640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f74140c8560 con 0x7f74140cf4f0 2023-08-09T13:34:46.513 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:46.510+0000 7f741940a640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f74140c9c10 con 0x7f74140d0ef0 2023-08-09T13:34:46.514 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:46.510+0000 7f7418c09640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f741415c620 0x7f74140b4ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:34:46.515 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:46.511+0000 7f7418c09640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f741415c620 0x7f74140b4ec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:44168/0 (socket says 172.21.15.43:44168) 2023-08-09T13:34:46.515 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:46.511+0000 7f7418c09640 1 -- 172.21.15.43:0/3139238895 learned_addr learned my addr 172.21.15.43:0/3139238895 (peer_addr_for_me v2:172.21.15.43:0/0) 2023-08-09T13:34:46.515 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:46.511+0000 7f74137fe640 1 --2- 172.21.15.43:0/3139238895 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f74140d0ef0 0x7f74140c7800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:34:46.515 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:46.511+0000 7f7413fff640 1 --2- 172.21.15.43:0/3139238895 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f74140cf4f0 0x7f741414c670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:34:46.515 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:46.511+0000 7f7418c09640 1 -- 172.21.15.43:0/3139238895 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f74140d0ef0 msgr2=0x7f74140c7800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:34:46.515 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:46.511+0000 7f7418c09640 1 --2- 172.21.15.43:0/3139238895 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f74140d0ef0 0x7f74140c7800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:34:46.516 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:46.511+0000 7f7418c09640 1 -- 172.21.15.43:0/3139238895 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f74140cf4f0 msgr2=0x7f741414c670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:34:46.516 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:46.511+0000 7f7418c09640 1 --2- 172.21.15.43:0/3139238895 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f74140cf4f0 0x7f741414c670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:34:46.516 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:46.511+0000 7f7418c09640 1 -- 172.21.15.43:0/3139238895 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f740c009b30 con 0x7f741415c620 2023-08-09T13:34:46.516 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:46.511+0000 7f7418c09640 1 --2- 172.21.15.43:0/3139238895 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f741415c620 0x7f74140b4ec0 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7f740c036040 tx=0x7f740c004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-09T13:34:46.516 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:46.512+0000 7f7410ff9640 1 -- 172.21.15.43:0/3139238895 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f740c009510 con 0x7f741415c620 2023-08-09T13:34:46.516 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:46.512+0000 7f7410ff9640 1 -- 172.21.15.43:0/3139238895 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f740c03e450 con 0x7f741415c620 2023-08-09T13:34:46.516 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:46.512+0000 7f741940a640 1 -- 172.21.15.43:0/3139238895 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f74140c9ef0 con 0x7f741415c620 2023-08-09T13:34:46.517 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:46.512+0000 7f7410ff9640 1 -- 172.21.15.43:0/3139238895 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f740c03e800 con 0x7f741415c620 2023-08-09T13:34:46.517 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:46.512+0000 7f741940a640 1 -- 172.21.15.43:0/3139238895 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7414158bf0 con 0x7f741415c620 2023-08-09T13:34:46.517 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:46.513+0000 7f741940a640 1 -- 172.21.15.43:0/3139238895 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f74140ca110 con 0x7f741415c620 2023-08-09T13:34:46.517 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:46.514+0000 7f7410ff9640 1 -- 172.21.15.43:0/3139238895 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 5) v1 ==== 88341+0+0 (secure 0 0 0) 0x7f740c03e960 con 0x7f741415c620 2023-08-09T13:34:46.518 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:46.515+0000 7f7410ff9640 1 --2- 172.21.15.43:0/3139238895 >> [v2:172.21.15.203:6832/830097474,v1:172.21.15.203:6833/830097474] conn(0x7f73f406a0a0 0x7f73f406c560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:34:46.518 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:46.515+0000 7f7410ff9640 1 -- 172.21.15.43:0/3139238895 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6813+0+0 (secure 0 0 0) 0x7f740c0ab610 con 0x7f741415c620 2023-08-09T13:34:46.518 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:46.516+0000 7f7413fff640 1 --2- 172.21.15.43:0/3139238895 >> [v2:172.21.15.203:6832/830097474,v1:172.21.15.203:6833/830097474] conn(0x7f73f406a0a0 0x7f73f406c560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:34:46.519 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:46.517+0000 7f7413fff640 1 --2- 172.21.15.43:0/3139238895 >> [v2:172.21.15.203:6832/830097474,v1:172.21.15.203:6833/830097474] conn(0x7f73f406a0a0 0x7f73f406c560 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7f7404000fb0 tx=0x7f7404007c50 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-09T13:34:46.522 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:46.520+0000 7f7410ff9640 1 -- 172.21.15.43:0/3139238895 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178066 (secure 0 0 0) 0x7f740c0769f0 con 0x7f741415c620 2023-08-09T13:34:46.756 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:46.753+0000 7f741940a640 1 -- 172.21.15.43:0/3139238895 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f74140739f0 con 0x7f741415c620 2023-08-09T13:34:46.757 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:46.754+0000 7f7410ff9640 1 -- 172.21.15.43:0/3139238895 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 9 v9) v1 ==== 93+0+4874 (secure 0 0 0) 0x7f740c043090 con 0x7f741415c620 2023-08-09T13:34:46.757 INFO:teuthology.orchestra.run.smithi043.stdout: 2023-08-09T13:34:46.757 INFO:teuthology.orchestra.run.smithi043.stdout:{"epoch":9,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4236,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.43:6833/3917539953","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6832","nonce":3917539953},{"type":"v1","addr":"172.21.15.43:6833","nonce":3917539953}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2},{"gid":4247,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.203:6837/2467794159","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6835","nonce":2467794159},{"type":"v1","addr":"172.21.15.203:6837","nonce":2467794159}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":4248,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.43:6835/1818380189","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6834","nonce":1818380189},{"type":"v1","addr":"172.21.15.43:6835","nonce":1818380189}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":9,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-08-09T13:34:02.575398+0000","modified":"2023-08-09T13:34:31.841945+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":4250},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4250":{"gid":4250,"name":"d","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.203:6836/798312463","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6834","nonce":798312463},{"type":"v1","addr":"172.21.15.203:6836","nonce":798312463}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[4,5],"metadata_pool":3,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-08-09T13:34:46.758 INFO:teuthology.orchestra.run.smithi043.stderr:dumped fsmap epoch 9 2023-08-09T13:34:46.759 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:46.757+0000 7f741940a640 1 -- 172.21.15.43:0/3139238895 >> [v2:172.21.15.203:6832/830097474,v1:172.21.15.203:6833/830097474] conn(0x7f73f406a0a0 msgr2=0x7f73f406c560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:34:46.759 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:46.757+0000 7f741940a640 1 --2- 172.21.15.43:0/3139238895 >> [v2:172.21.15.203:6832/830097474,v1:172.21.15.203:6833/830097474] conn(0x7f73f406a0a0 0x7f73f406c560 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7f7404000fb0 tx=0x7f7404007c50 comp rx=0 tx=0).stop 2023-08-09T13:34:46.760 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:46.757+0000 7f741940a640 1 -- 172.21.15.43:0/3139238895 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f741415c620 msgr2=0x7f74140b4ec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:34:46.760 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:46.757+0000 7f741940a640 1 --2- 172.21.15.43:0/3139238895 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f741415c620 0x7f74140b4ec0 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7f740c036040 tx=0x7f740c004a40 comp rx=0 tx=0).stop 2023-08-09T13:34:46.760 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:46.757+0000 7f741940a640 1 -- 172.21.15.43:0/3139238895 shutdown_connections 2023-08-09T13:34:46.760 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:46.757+0000 7f741940a640 1 --2- 172.21.15.43:0/3139238895 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f741415c620 0x7f74140b4ec0 unknown :-1 s=CLOSED pgs=232 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:34:46.761 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:46.757+0000 7f741940a640 1 --2- 172.21.15.43:0/3139238895 >> [v2:172.21.15.203:6832/830097474,v1:172.21.15.203:6833/830097474] conn(0x7f73f406a0a0 0x7f73f406c560 unknown :-1 s=CLOSED pgs=138 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:34:46.761 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:46.757+0000 7f741940a640 1 --2- 172.21.15.43:0/3139238895 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f74140d0ef0 0x7f74140c7800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:34:46.761 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:46.757+0000 7f741940a640 1 --2- 172.21.15.43:0/3139238895 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f74140cf4f0 0x7f741414c670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:34:46.761 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:46.757+0000 7f741940a640 1 -- 172.21.15.43:0/3139238895 >> 172.21.15.43:0/3139238895 conn(0x7f741409b650 msgr2=0x7f741406b450 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-09T13:34:46.761 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:46.757+0000 7f741940a640 1 -- 172.21.15.43:0/3139238895 shutdown_connections 2023-08-09T13:34:46.761 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:46.757+0000 7f741940a640 1 -- 172.21.15.43:0/3139238895 wait complete. 2023-08-09T13:34:46.779 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-08-09T13:34:47.004 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:47.002+0000 7ffb3789b640 1 Processor -- start 2023-08-09T13:34:47.004 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:47.002+0000 7ffb3789b640 1 -- start start 2023-08-09T13:34:47.006 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:47.003+0000 7ffb3789b640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7ffb300b2b70 0x7ffb300d2ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:34:47.006 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:47.003+0000 7ffb3789b640 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7ffb300b3050 0x7ffb300b3f60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:34:47.006 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:47.003+0000 7ffb3789b640 1 -- --> v1:172.21.15.43:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ffb300ce090 con 0x7ffb300b2790 2023-08-09T13:34:47.006 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:47.003+0000 7ffb3789b640 1 -- --> v2:172.21.15.203:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ffb300b3950 con 0x7ffb300b3050 2023-08-09T13:34:47.006 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:47.003+0000 7ffb3789b640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ffb300cd7f0 con 0x7ffb300b2b70 2023-08-09T13:34:47.006 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:47.004+0000 7ffb35637640 1 --1- >> v1:172.21.15.43:6789/0 conn(0x7ffb300b2790 0x7ffb300b5330 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.43:6789/0 says I am v1:172.21.15.43:42390/0 (socket says 172.21.15.43:42390) 2023-08-09T13:34:47.007 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:47.004+0000 7ffb35637640 1 -- 172.21.15.43:0/2818803992 learned_addr learned my addr 172.21.15.43:0/2818803992 (peer_addr_for_me v1:172.21.15.43:0/0) 2023-08-09T13:34:47.008 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:47.004+0000 7ffb35e38640 1 --2- 172.21.15.43:0/2818803992 >> v2:172.21.15.203:3300/0 conn(0x7ffb300b3050 0x7ffb300b3f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:34:47.009 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:47.004+0000 7ffb34e36640 1 --2- 172.21.15.43:0/2818803992 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7ffb300b2b70 0x7ffb300d2ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:34:47.009 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:47.004+0000 7ffb2ffff640 1 -- 172.21.15.43:0/2818803992 <== mon.0 v1:172.21.15.43:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3645245574 0 0) 0x7ffb300ce090 con 0x7ffb300b2790 2023-08-09T13:34:47.009 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:47.004+0000 7ffb2ffff640 1 -- 172.21.15.43:0/2818803992 --> v1:172.21.15.43:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ffb18003520 con 0x7ffb300b2790 2023-08-09T13:34:47.009 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:47.005+0000 7ffb2ffff640 1 -- 172.21.15.43:0/2818803992 <== mon.0 v1:172.21.15.43:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3763601119 0 0) 0x7ffb18003520 con 0x7ffb300b2790 2023-08-09T13:34:47.009 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:47.005+0000 7ffb2ffff640 1 -- 172.21.15.43:0/2818803992 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7ffb300b2b70 msgr2=0x7ffb300d2ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-09T13:34:47.009 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:47.005+0000 7ffb2ffff640 1 --2- 172.21.15.43:0/2818803992 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7ffb300b2b70 0x7ffb300d2ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:34:47.010 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:47.005+0000 7ffb2ffff640 1 -- 172.21.15.43:0/2818803992 >> v2:172.21.15.203:3300/0 conn(0x7ffb300b3050 msgr2=0x7ffb300b3f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-09T13:34:47.010 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:47.005+0000 7ffb2ffff640 1 --2- 172.21.15.43:0/2818803992 >> v2:172.21.15.203:3300/0 conn(0x7ffb300b3050 0x7ffb300b3f60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:34:47.010 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:47.005+0000 7ffb2ffff640 1 -- 172.21.15.43:0/2818803992 --> v1:172.21.15.43:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ffb300cdad0 con 0x7ffb300b2790 2023-08-09T13:34:47.010 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:47.005+0000 7ffb2ffff640 1 -- 172.21.15.43:0/2818803992 <== mon.0 v1:172.21.15.43:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1285150740 0 0) 0x7ffb20001670 con 0x7ffb300b2790 2023-08-09T13:34:47.010 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:47.005+0000 7ffb2ffff640 1 -- 172.21.15.43:0/2818803992 >> v1:172.21.15.43:6789/0 conn(0x7ffb300b2790 legacy=0x7ffb300b5330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:34:47.010 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:47.005+0000 7ffb2ffff640 1 --2- 172.21.15.43:0/2818803992 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7ffb300b2b70 0x7ffb180039e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:34:47.010 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:47.005+0000 7ffb2ffff640 1 --2- 172.21.15.43:0/2818803992 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7ffb18003f60 0x7ffb180047c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:34:47.010 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:47.005+0000 7ffb2ffff640 1 --2- 172.21.15.43:0/2818803992 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ffb18004d00 0x7ffb18005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:34:47.011 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:47.005+0000 7ffb2ffff640 1 -- 172.21.15.43:0/2818803992 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffb300b3950 con 0x7ffb18004d00 2023-08-09T13:34:47.011 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:47.005+0000 7ffb2ffff640 1 -- 172.21.15.43:0/2818803992 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffb300cd7f0 con 0x7ffb18003f60 2023-08-09T13:34:47.011 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:47.005+0000 7ffb2ffff640 1 -- 172.21.15.43:0/2818803992 --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ffb1800b9b0 con 0x7ffb300b2b70 2023-08-09T13:34:47.011 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:47.005+0000 7ffb35e38640 1 --2- 172.21.15.43:0/2818803992 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ffb18004d00 0x7ffb18005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:34:47.011 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:47.005+0000 7ffb35637640 1 --2- 172.21.15.43:0/2818803992 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7ffb300b2b70 0x7ffb180039e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:34:47.011 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:47.005+0000 7ffb34e36640 1 --2- 172.21.15.43:0/2818803992 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7ffb18003f60 0x7ffb180047c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:34:47.011 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:47.005+0000 7ffb35e38640 1 -- 172.21.15.43:0/2818803992 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7ffb300b2b70 msgr2=0x7ffb180039e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-09T13:34:47.012 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:47.006+0000 7ffb35e38640 1 --2- 172.21.15.43:0/2818803992 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7ffb300b2b70 0x7ffb180039e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:34:47.012 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:47.006+0000 7ffb35e38640 1 -- 172.21.15.43:0/2818803992 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7ffb18003f60 msgr2=0x7ffb180047c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-09T13:34:47.012 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:47.006+0000 7ffb35e38640 1 --2- 172.21.15.43:0/2818803992 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7ffb18003f60 0x7ffb180047c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:34:47.012 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:47.006+0000 7ffb35e38640 1 -- 172.21.15.43:0/2818803992 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ffb18003520 con 0x7ffb18004d00 2023-08-09T13:34:47.012 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:47.006+0000 7ffb35637640 1 --2- 172.21.15.43:0/2818803992 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7ffb300b2b70 0x7ffb180039e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-08-09T13:34:47.012 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:47.006+0000 7ffb34e36640 1 --2- 172.21.15.43:0/2818803992 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7ffb18003f60 0x7ffb180047c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-08-09T13:34:47.014 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:47.007+0000 7ffb35e38640 1 --2- 172.21.15.43:0/2818803992 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ffb18004d00 0x7ffb18005180 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7ffb28009fc0 tx=0x7ffb180057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=babf827cfb247be1 server_cookie=0 in_seq=0 out_seq=0 2023-08-09T13:34:47.015 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:47.007+0000 7ffb2ffff640 1 -- 172.21.15.43:0/2818803992 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffb2802cc30 con 0x7ffb18004d00 2023-08-09T13:34:47.015 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:47.007+0000 7ffb2ffff640 1 -- 172.21.15.43:0/2818803992 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ffb2800f040 con 0x7ffb18004d00 2023-08-09T13:34:47.015 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:47.007+0000 7ffb2ffff640 1 -- 172.21.15.43:0/2818803992 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffb28034440 con 0x7ffb18004d00 2023-08-09T13:34:47.015 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:47.007+0000 7ffb3789b640 1 -- 172.21.15.43:0/2818803992 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ffb18004d00 msgr2=0x7ffb18005180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:34:47.015 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:47.007+0000 7ffb3789b640 1 --2- 172.21.15.43:0/2818803992 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ffb18004d00 0x7ffb18005180 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7ffb28009fc0 tx=0x7ffb180057f0 comp rx=0 tx=0).stop 2023-08-09T13:34:47.015 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:47.007+0000 7ffb35e38640 1 -- 172.21.15.43:0/2818803992 reap_dead start 2023-08-09T13:34:47.015 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:47.008+0000 7ffb3789b640 1 -- 172.21.15.43:0/2818803992 shutdown_connections 2023-08-09T13:34:47.015 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:47.008+0000 7ffb3789b640 1 -- 172.21.15.43:0/2818803992 >> 172.21.15.43:0/2818803992 conn(0x7ffb3009b7b0 msgr2=0x7ffb300d24e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-09T13:34:47.016 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:47.008+0000 7ffb3789b640 1 -- 172.21.15.43:0/2818803992 shutdown_connections 2023-08-09T13:34:47.016 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:47.008+0000 7ffb3789b640 1 -- 172.21.15.43:0/2818803992 wait complete. 2023-08-09T13:34:47.016 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:47.009+0000 7ffb3789b640 1 Processor -- start 2023-08-09T13:34:47.016 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:47.010+0000 7ffb3789b640 1 -- start start 2023-08-09T13:34:47.016 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:47.011+0000 7ffb3789b640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ffb18004d00 0x7ffb30158480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:34:47.016 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:47.011+0000 7ffb3789b640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7ffb300c7840 0x7ffb300b5170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:34:47.016 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:47.011+0000 7ffb3789b640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7ffb300b56b0 0x7ffb301f5520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:34:47.017 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:47.011+0000 7ffb3789b640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffb300b5b30 con 0x7ffb18004d00 2023-08-09T13:34:47.017 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:47.011+0000 7ffb3789b640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffb30158ad0 con 0x7ffb300c7840 2023-08-09T13:34:47.017 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:47.011+0000 7ffb3789b640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ffb30158db0 con 0x7ffb300b56b0 2023-08-09T13:34:47.017 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:47.011+0000 7ffb35637640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ffb18004d00 0x7ffb30158480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:34:47.017 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:47.011+0000 7ffb35637640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ffb18004d00 0x7ffb30158480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:44180/0 (socket says 172.21.15.43:44180) 2023-08-09T13:34:47.017 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:47.011+0000 7ffb35637640 1 -- 172.21.15.43:0/3344457918 learned_addr learned my addr 172.21.15.43:0/3344457918 (peer_addr_for_me v2:172.21.15.43:0/0) 2023-08-09T13:34:47.017 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:47.011+0000 7ffb34e36640 1 --2- 172.21.15.43:0/3344457918 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7ffb300c7840 0x7ffb300b5170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:34:47.017 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:47.011+0000 7ffb35e38640 1 --2- 172.21.15.43:0/3344457918 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7ffb300b56b0 0x7ffb301f5520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:34:47.018 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:47.011+0000 7ffb35637640 1 -- 172.21.15.43:0/3344457918 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7ffb300b56b0 msgr2=0x7ffb301f5520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:34:47.018 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:47.011+0000 7ffb35637640 1 --2- 172.21.15.43:0/3344457918 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7ffb300b56b0 0x7ffb301f5520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:34:47.018 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:47.011+0000 7ffb35637640 1 -- 172.21.15.43:0/3344457918 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7ffb300c7840 msgr2=0x7ffb300b5170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:34:47.018 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:47.011+0000 7ffb35637640 1 --2- 172.21.15.43:0/3344457918 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7ffb300c7840 0x7ffb300b5170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:34:47.018 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:47.011+0000 7ffb35637640 1 -- 172.21.15.43:0/3344457918 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ffb30159090 con 0x7ffb18004d00 2023-08-09T13:34:47.018 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:47.011+0000 7ffb35e38640 1 --2- 172.21.15.43:0/3344457918 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7ffb300b56b0 0x7ffb301f5520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-08-09T13:34:47.018 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:47.012+0000 7ffb35637640 1 --2- 172.21.15.43:0/3344457918 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ffb18004d00 0x7ffb30158480 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7ffb200018a0 tx=0x7ffb2000dd20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-09T13:34:47.019 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:47.012+0000 7ffb2dffb640 1 -- 172.21.15.43:0/3344457918 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffb200074e0 con 0x7ffb18004d00 2023-08-09T13:34:47.019 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:47.012+0000 7ffb2dffb640 1 -- 172.21.15.43:0/3344457918 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ffb20007640 con 0x7ffb18004d00 2023-08-09T13:34:47.019 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:47.012+0000 7ffb3789b640 1 -- 172.21.15.43:0/3344457918 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ffb30159370 con 0x7ffb18004d00 2023-08-09T13:34:47.019 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:47.012+0000 7ffb2dffb640 1 -- 172.21.15.43:0/3344457918 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffb200049e0 con 0x7ffb18004d00 2023-08-09T13:34:47.019 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:47.012+0000 7ffb3789b640 1 -- 172.21.15.43:0/3344457918 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ffb301f5c40 con 0x7ffb18004d00 2023-08-09T13:34:47.020 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:47.013+0000 7ffb3789b640 1 -- 172.21.15.43:0/3344457918 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ffb301595b0 con 0x7ffb18004d00 2023-08-09T13:34:47.020 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:47.014+0000 7ffb2dffb640 1 -- 172.21.15.43:0/3344457918 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 5) v1 ==== 88341+0+0 (secure 0 0 0) 0x7ffb20004b40 con 0x7ffb18004d00 2023-08-09T13:34:47.020 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:47.015+0000 7ffb2dffb640 1 --2- 172.21.15.43:0/3344457918 >> [v2:172.21.15.203:6832/830097474,v1:172.21.15.203:6833/830097474] conn(0x7ffb1c06a080 0x7ffb1c06c540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:34:47.020 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:47.016+0000 7ffb2dffb640 1 -- 172.21.15.43:0/3344457918 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6813+0+0 (secure 0 0 0) 0x7ffb200886e0 con 0x7ffb18004d00 2023-08-09T13:34:47.020 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:47.016+0000 7ffb34e36640 1 --2- 172.21.15.43:0/3344457918 >> [v2:172.21.15.203:6832/830097474,v1:172.21.15.203:6833/830097474] conn(0x7ffb1c06a080 0x7ffb1c06c540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:34:47.021 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:47.017+0000 7ffb34e36640 1 --2- 172.21.15.43:0/3344457918 >> [v2:172.21.15.203:6832/830097474,v1:172.21.15.203:6833/830097474] conn(0x7ffb1c06a080 0x7ffb1c06c540 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7ffb3015d080 tx=0x7ffb24008040 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-09T13:34:47.022 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:47.020+0000 7ffb2dffb640 1 -- 172.21.15.43:0/3344457918 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178066 (secure 0 0 0) 0x7ffb301595b0 con 0x7ffb18004d00 2023-08-09T13:34:47.216 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:47.214+0000 7ffb3789b640 1 -- 172.21.15.43:0/3344457918 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7ffb301595b0 con 0x7ffb18004d00 2023-08-09T13:34:47.219 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:47.217+0000 7ffb2dffb640 1 -- 172.21.15.43:0/3344457918 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v24) v1 ==== 74+0+20050 (secure 0 0 0) 0x7ffb301595b0 con 0x7ffb18004d00 2023-08-09T13:34:47.220 INFO:teuthology.orchestra.run.smithi043.stdout: 2023-08-09T13:34:47.220 INFO:teuthology.orchestra.run.smithi043.stdout:{"epoch":24,"fsid":"daac87ee-1337-4250-a955-aea4e408a39b","created":"2023-08-09T13:33:41.701198+0000","modified":"2023-08-09T13:34:07.709429+0000","last_up_change":"2023-08-09T13:33:53.783453+0000","last_in_change":"2023-08-09T13:33:45.148904+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":5,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":"rbd","create_time":"2023-08-09T13:33:54.365137+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":".mgr","create_time":"2023-08-09T13:33:54.831803+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_metadata","create_time":"2023-08-09T13:34:00.207974+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data","create_time":"2023-08-09T13:34:01.614765+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":5,"pool_name":"cephfs_data_ec","create_time":"2023-08-09T13:34:05.239583+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"24","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"cbd49ecd-e896-4fbd-864e-5c37a829b165","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6800","nonce":502667335},{"type":"v1","addr":"172.21.15.43:6801","nonce":502667335}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6802","nonce":502667335},{"type":"v1","addr":"172.21.15.43:6803","nonce":502667335}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6806","nonce":502667335},{"type":"v1","addr":"172.21.15.43:6807","nonce":502667335}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6804","nonce":502667335},{"type":"v1","addr":"172.21.15.43:6805","nonce":502667335}]},"public_addr":"172.21.15.43:6801/502667335","cluster_addr":"172.21.15.43:6803/502667335","heartbeat_back_addr":"172.21.15.43:6807/502667335","heartbeat_front_addr":"172.21.15.43:6805/502667335","state":["exists","up"]},{"osd":1,"uuid":"75c7e0fe-1ec7-4fe8-a2f2-3661a65b8500","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6808","nonce":3289134610},{"type":"v1","addr":"172.21.15.43:6809","nonce":3289134610}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6810","nonce":3289134610},{"type":"v1","addr":"172.21.15.43:6812","nonce":3289134610}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6821","nonce":3289134610},{"type":"v1","addr":"172.21.15.43:6824","nonce":3289134610}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6815","nonce":3289134610},{"type":"v1","addr":"172.21.15.43:6818","nonce":3289134610}]},"public_addr":"172.21.15.43:6809/3289134610","cluster_addr":"172.21.15.43:6812/3289134610","heartbeat_back_addr":"172.21.15.43:6824/3289134610","heartbeat_front_addr":"172.21.15.43:6818/3289134610","state":["exists","up"]},{"osd":2,"uuid":"bb5c41e9-7f5f-4125-9b01-3bb8bfc2ea83","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6811","nonce":4079709736},{"type":"v1","addr":"172.21.15.43:6814","nonce":4079709736}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6817","nonce":4079709736},{"type":"v1","addr":"172.21.15.43:6820","nonce":4079709736}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6829","nonce":4079709736},{"type":"v1","addr":"172.21.15.43:6831","nonce":4079709736}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6823","nonce":4079709736},{"type":"v1","addr":"172.21.15.43:6826","nonce":4079709736}]},"public_addr":"172.21.15.43:6814/4079709736","cluster_addr":"172.21.15.43:6820/4079709736","heartbeat_back_addr":"172.21.15.43:6831/4079709736","heartbeat_front_addr":"172.21.15.43:6826/4079709736","state":["exists","up"]},{"osd":3,"uuid":"bfe4c8b4-694a-465c-87e6-2b2aab852613","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6813","nonce":1709538759},{"type":"v1","addr":"172.21.15.43:6816","nonce":1709538759}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6819","nonce":1709538759},{"type":"v1","addr":"172.21.15.43:6822","nonce":1709538759}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6828","nonce":1709538759},{"type":"v1","addr":"172.21.15.43:6830","nonce":1709538759}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6825","nonce":1709538759},{"type":"v1","addr":"172.21.15.43:6827","nonce":1709538759}]},"public_addr":"172.21.15.43:6816/1709538759","cluster_addr":"172.21.15.43:6822/1709538759","heartbeat_back_addr":"172.21.15.43:6830/1709538759","heartbeat_front_addr":"172.21.15.43:6827/1709538759","state":["exists","up"]},{"osd":4,"uuid":"fa2e8a77-aba1-4756-853e-0d174a5bdeba","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6800","nonce":4196866481},{"type":"v1","addr":"172.21.15.203:6801","nonce":4196866481}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6802","nonce":4196866481},{"type":"v1","addr":"172.21.15.203:6803","nonce":4196866481}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6806","nonce":4196866481},{"type":"v1","addr":"172.21.15.203:6807","nonce":4196866481}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6804","nonce":4196866481},{"type":"v1","addr":"172.21.15.203:6805","nonce":4196866481}]},"public_addr":"172.21.15.203:6801/4196866481","cluster_addr":"172.21.15.203:6803/4196866481","heartbeat_back_addr":"172.21.15.203:6807/4196866481","heartbeat_front_addr":"172.21.15.203:6805/4196866481","state":["exists","up"]},{"osd":5,"uuid":"81811e4b-4d24-4f51-a82c-f1d700a2a00d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6816","nonce":3059290081},{"type":"v1","addr":"172.21.15.203:6817","nonce":3059290081}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6818","nonce":3059290081},{"type":"v1","addr":"172.21.15.203:6819","nonce":3059290081}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6822","nonce":3059290081},{"type":"v1","addr":"172.21.15.203:6823","nonce":3059290081}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6820","nonce":3059290081},{"type":"v1","addr":"172.21.15.203:6821","nonce":3059290081}]},"public_addr":"172.21.15.203:6817/3059290081","cluster_addr":"172.21.15.203:6819/3059290081","heartbeat_back_addr":"172.21.15.203:6823/3059290081","heartbeat_front_addr":"172.21.15.203:6821/3059290081","state":["exists","up"]},{"osd":6,"uuid":"307bac4d-6a5b-443d-af75-56d695d7674f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6824","nonce":512155569},{"type":"v1","addr":"172.21.15.203:6825","nonce":512155569}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6826","nonce":512155569},{"type":"v1","addr":"172.21.15.203:6827","nonce":512155569}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6830","nonce":512155569},{"type":"v1","addr":"172.21.15.203:6831","nonce":512155569}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6828","nonce":512155569},{"type":"v1","addr":"172.21.15.203:6829","nonce":512155569}]},"public_addr":"172.21.15.203:6825/512155569","cluster_addr":"172.21.15.203:6827/512155569","heartbeat_back_addr":"172.21.15.203:6831/512155569","heartbeat_front_addr":"172.21.15.203:6829/512155569","state":["exists","up"]},{"osd":7,"uuid":"a95dc7ca-9f1b-4295-af70-68a22a2d7d5e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6808","nonce":3331450438},{"type":"v1","addr":"172.21.15.203:6809","nonce":3331450438}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6810","nonce":3331450438},{"type":"v1","addr":"172.21.15.203:6811","nonce":3331450438}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6814","nonce":3331450438},{"type":"v1","addr":"172.21.15.203:6815","nonce":3331450438}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6812","nonce":3331450438},{"type":"v1","addr":"172.21.15.203:6813","nonce":3331450438}]},"public_addr":"172.21.15.203:6809/3331450438","cluster_addr":"172.21.15.203:6811/3331450438","heartbeat_back_addr":"172.21.15.203:6815/3331450438","heartbeat_front_addr":"172.21.15.203:6813/3331450438","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-09T13:33:50.547652+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-09T13:33:50.589469+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-09T13:33:50.561724+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-09T13:33:50.599650+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-09T13:33:50.411855+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-09T13:33:50.219681+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-09T13:33:50.504664+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-09T13:33:50.441248+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-08-09T13:34:47.222 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:47.219+0000 7ffb3789b640 1 -- 172.21.15.43:0/3344457918 >> [v2:172.21.15.203:6832/830097474,v1:172.21.15.203:6833/830097474] conn(0x7ffb1c06a080 msgr2=0x7ffb1c06c540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:34:47.222 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:47.220+0000 7ffb3789b640 1 --2- 172.21.15.43:0/3344457918 >> [v2:172.21.15.203:6832/830097474,v1:172.21.15.203:6833/830097474] conn(0x7ffb1c06a080 0x7ffb1c06c540 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7ffb3015d080 tx=0x7ffb24008040 comp rx=0 tx=0).stop 2023-08-09T13:34:47.222 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:47.220+0000 7ffb3789b640 1 -- 172.21.15.43:0/3344457918 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ffb18004d00 msgr2=0x7ffb30158480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:34:47.223 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:47.220+0000 7ffb3789b640 1 --2- 172.21.15.43:0/3344457918 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ffb18004d00 0x7ffb30158480 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7ffb200018a0 tx=0x7ffb2000dd20 comp rx=0 tx=0).stop 2023-08-09T13:34:47.223 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:47.220+0000 7ffb3789b640 1 -- 172.21.15.43:0/3344457918 shutdown_connections 2023-08-09T13:34:47.223 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:47.220+0000 7ffb3789b640 1 --2- 172.21.15.43:0/3344457918 >> [v2:172.21.15.203:6832/830097474,v1:172.21.15.203:6833/830097474] conn(0x7ffb1c06a080 0x7ffb1c06c540 unknown :-1 s=CLOSED pgs=139 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:34:47.223 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:47.220+0000 7ffb3789b640 1 --2- 172.21.15.43:0/3344457918 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7ffb300b56b0 0x7ffb301f5520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:34:47.223 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:47.220+0000 7ffb3789b640 1 --2- 172.21.15.43:0/3344457918 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7ffb300c7840 0x7ffb300b5170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:34:47.223 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:47.220+0000 7ffb3789b640 1 --2- 172.21.15.43:0/3344457918 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ffb18004d00 0x7ffb30158480 unknown :-1 s=CLOSED pgs=235 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:34:47.223 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:47.220+0000 7ffb3789b640 1 -- 172.21.15.43:0/3344457918 >> 172.21.15.43:0/3344457918 conn(0x7ffb3009b7b0 msgr2=0x7ffb300b10b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-09T13:34:47.224 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:47.220+0000 7ffb3789b640 1 -- 172.21.15.43:0/3344457918 shutdown_connections 2023-08-09T13:34:47.224 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:47.220+0000 7ffb3789b640 1 -- 172.21.15.43:0/3344457918 wait complete. 2023-08-09T13:34:47.245 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-08-09T13:34:47.463 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:47.461+0000 7fa780d8b640 1 Processor -- start 2023-08-09T13:34:47.464 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:47.461+0000 7fa780d8b640 1 -- start start 2023-08-09T13:34:47.465 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:47.462+0000 7fa780d8b640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fa77c0b2790 0x7fa77c0d2ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:34:47.466 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:47.462+0000 7fa780d8b640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7fa77c0b3050 0x7fa77c0b3f60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:34:47.466 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:47.462+0000 7fa780d8b640 1 -- --> v1:172.21.15.43:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa77c0ce090 con 0x7fa77c0b2b70 2023-08-09T13:34:47.466 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:47.462+0000 7fa780d8b640 1 -- --> v2:172.21.15.43:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa77c0b3950 con 0x7fa77c0b3050 2023-08-09T13:34:47.467 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:47.462+0000 7fa780d8b640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa77c0cd7f0 con 0x7fa77c0b2790 2023-08-09T13:34:47.467 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:47.462+0000 7fa77ad9d640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7fa77c0b3050 0x7fa77c0b3f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:34:47.467 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:47.462+0000 7fa773fff640 1 --1- >> v1:172.21.15.43:6789/0 conn(0x7fa77c0b2b70 0x7fa77c0b5330 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.43:6789/0 says I am v1:172.21.15.43:42402/0 (socket says 172.21.15.43:42402) 2023-08-09T13:34:47.467 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:47.462+0000 7fa773fff640 1 -- 172.21.15.43:0/3884531200 learned_addr learned my addr 172.21.15.43:0/3884531200 (peer_addr_for_me v1:172.21.15.43:0/0) 2023-08-09T13:34:47.467 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:47.463+0000 7fa77a59c640 1 --2- 172.21.15.43:0/3884531200 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fa77c0b2790 0x7fa77c0d2ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:34:47.467 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:47.463+0000 7fa77ad9d640 1 -- 172.21.15.43:0/3884531200 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fa77c0b2790 msgr2=0x7fa77c0d2ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-09T13:34:47.468 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:47.463+0000 7fa77ad9d640 1 --2- 172.21.15.43:0/3884531200 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fa77c0b2790 0x7fa77c0d2ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:34:47.468 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:47.463+0000 7fa77ad9d640 1 -- 172.21.15.43:0/3884531200 >> v1:172.21.15.43:6789/0 conn(0x7fa77c0b2b70 legacy=0x7fa77c0b5330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:34:47.468 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:47.463+0000 7fa779d9b640 1 -- 172.21.15.43:0/3884531200 <== mon.0 v1:172.21.15.43:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3053172629 0 0) 0x7fa77c0ce090 con 0x7fa77c0b2b70 2023-08-09T13:34:47.468 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:47.463+0000 7fa77ad9d640 1 -- 172.21.15.43:0/3884531200 --> v2:172.21.15.43:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa77c0cdad0 con 0x7fa77c0b3050 2023-08-09T13:34:47.469 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:47.464+0000 7fa77ad9d640 1 --2- 172.21.15.43:0/3884531200 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa77c0b3050 0x7fa77c0b3f60 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7fa774009d90 tx=0x7fa77402bea0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1697c2b63082887 server_cookie=0 in_seq=0 out_seq=0 2023-08-09T13:34:47.470 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:47.465+0000 7fa779d9b640 1 -- 172.21.15.43:0/3884531200 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa77402cc50 con 0x7fa77c0b3050 2023-08-09T13:34:47.470 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:47.465+0000 7fa779d9b640 1 -- 172.21.15.43:0/3884531200 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa77402cdb0 con 0x7fa77c0b3050 2023-08-09T13:34:47.470 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:47.465+0000 7fa779d9b640 1 -- 172.21.15.43:0/3884531200 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa774034660 con 0x7fa77c0b3050 2023-08-09T13:34:47.470 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:47.465+0000 7fa780d8b640 1 -- 172.21.15.43:0/3884531200 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa77c0b3050 msgr2=0x7fa77c0b3f60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:34:47.470 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:47.465+0000 7fa780d8b640 1 --2- 172.21.15.43:0/3884531200 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa77c0b3050 0x7fa77c0b3f60 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7fa774009d90 tx=0x7fa77402bea0 comp rx=0 tx=0).stop 2023-08-09T13:34:47.471 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:47.465+0000 7fa780d8b640 1 -- 172.21.15.43:0/3884531200 shutdown_connections 2023-08-09T13:34:47.471 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:47.465+0000 7fa780d8b640 1 --2- 172.21.15.43:0/3884531200 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa77c0b3050 0x7fa77c0b3f60 unknown :-1 s=CLOSED pgs=237 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:34:47.471 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:47.465+0000 7fa780d8b640 1 --2- 172.21.15.43:0/3884531200 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fa77c0b2790 0x7fa77c0d2ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:34:47.471 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:47.465+0000 7fa780d8b640 1 -- 172.21.15.43:0/3884531200 >> 172.21.15.43:0/3884531200 conn(0x7fa77c09b7b0 msgr2=0x7fa77c0d24e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-09T13:34:47.471 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:47.465+0000 7fa780d8b640 1 -- 172.21.15.43:0/3884531200 shutdown_connections 2023-08-09T13:34:47.471 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:47.465+0000 7fa780d8b640 1 -- 172.21.15.43:0/3884531200 wait complete. 2023-08-09T13:34:47.474 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:47.467+0000 7fa780d8b640 1 Processor -- start 2023-08-09T13:34:47.474 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:47.467+0000 7fa780d8b640 1 -- start start 2023-08-09T13:34:47.474 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:47.468+0000 7fa780d8b640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fa77c0b2790 0x7fa77c0cabc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:34:47.475 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:47.468+0000 7fa780d8b640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa77c0b2b70 0x7fa77c1549a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:34:47.475 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:47.468+0000 7fa780d8b640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fa77c0b3050 0x7fa77c0af0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:34:47.475 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:47.468+0000 7fa780d8b640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa77c0ca730 con 0x7fa77c0b2b70 2023-08-09T13:34:47.475 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:47.468+0000 7fa780d8b640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa77c0c9b20 con 0x7fa77c0b2790 2023-08-09T13:34:47.475 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:47.468+0000 7fa780d8b640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa77c0c9e00 con 0x7fa77c0b3050 2023-08-09T13:34:47.475 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:47.469+0000 7fa773fff640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa77c0b2b70 0x7fa77c1549a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:34:47.475 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:47.469+0000 7fa773fff640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa77c0b2b70 0x7fa77c1549a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:44196/0 (socket says 172.21.15.43:44196) 2023-08-09T13:34:47.476 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:47.469+0000 7fa773fff640 1 -- 172.21.15.43:0/899853908 learned_addr learned my addr 172.21.15.43:0/899853908 (peer_addr_for_me v2:172.21.15.43:0/0) 2023-08-09T13:34:47.476 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:47.469+0000 7fa77ad9d640 1 --2- 172.21.15.43:0/899853908 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fa77c0b3050 0x7fa77c0af0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:34:47.476 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:47.469+0000 7fa77a59c640 1 --2- 172.21.15.43:0/899853908 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fa77c0b2790 0x7fa77c0cabc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:34:47.476 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:47.469+0000 7fa773fff640 1 -- 172.21.15.43:0/899853908 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fa77c0b3050 msgr2=0x7fa77c0af0a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:34:47.476 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:47.469+0000 7fa773fff640 1 --2- 172.21.15.43:0/899853908 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fa77c0b3050 0x7fa77c0af0a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:34:47.476 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:47.469+0000 7fa773fff640 1 -- 172.21.15.43:0/899853908 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fa77c0b2790 msgr2=0x7fa77c0cabc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:34:47.476 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:47.469+0000 7fa773fff640 1 --2- 172.21.15.43:0/899853908 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fa77c0b2790 0x7fa77c0cabc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:34:47.477 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:47.469+0000 7fa773fff640 1 -- 172.21.15.43:0/899853908 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa774009a70 con 0x7fa77c0b2b70 2023-08-09T13:34:47.477 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:47.470+0000 7fa773fff640 1 --2- 172.21.15.43:0/899853908 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa77c0b2b70 0x7fa77c1549a0 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7fa76400c430 tx=0x7fa76400c8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-09T13:34:47.477 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:47.470+0000 7fa772ffd640 1 -- 172.21.15.43:0/899853908 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa7640074e0 con 0x7fa77c0b2b70 2023-08-09T13:34:47.477 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:47.470+0000 7fa772ffd640 1 -- 172.21.15.43:0/899853908 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa764007660 con 0x7fa77c0b2b70 2023-08-09T13:34:47.477 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:47.470+0000 7fa772ffd640 1 -- 172.21.15.43:0/899853908 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa764004a20 con 0x7fa77c0b2b70 2023-08-09T13:34:47.477 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:47.470+0000 7fa780d8b640 1 -- 172.21.15.43:0/899853908 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa77c0ca0e0 con 0x7fa77c0b2b70 2023-08-09T13:34:47.477 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:47.470+0000 7fa780d8b640 1 -- 172.21.15.43:0/899853908 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa77c0afd90 con 0x7fa77c0b2b70 2023-08-09T13:34:47.477 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:47.471+0000 7fa780d8b640 1 -- 172.21.15.43:0/899853908 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa77c0ca8a0 con 0x7fa77c0b2b70 2023-08-09T13:34:47.478 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:47.472+0000 7fa772ffd640 1 -- 172.21.15.43:0/899853908 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 5) v1 ==== 88341+0+0 (secure 0 0 0) 0x7fa764004b80 con 0x7fa77c0b2b70 2023-08-09T13:34:47.478 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:47.473+0000 7fa772ffd640 1 --2- 172.21.15.43:0/899853908 >> [v2:172.21.15.203:6832/830097474,v1:172.21.15.203:6833/830097474] conn(0x7fa75c06a050 0x7fa75c06c510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:34:47.479 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:47.473+0000 7fa772ffd640 1 -- 172.21.15.43:0/899853908 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6813+0+0 (secure 0 0 0) 0x7fa764088690 con 0x7fa77c0b2b70 2023-08-09T13:34:47.479 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:47.474+0000 7fa77a59c640 1 --2- 172.21.15.43:0/899853908 >> [v2:172.21.15.203:6832/830097474,v1:172.21.15.203:6833/830097474] conn(0x7fa75c06a050 0x7fa75c06c510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:34:47.479 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:47.475+0000 7fa77a59c640 1 --2- 172.21.15.43:0/899853908 >> [v2:172.21.15.203:6832/830097474,v1:172.21.15.203:6833/830097474] conn(0x7fa75c06a050 0x7fa75c06c510 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7fa76c009a30 tx=0x7fa76c008040 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-09T13:34:47.480 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:47.477+0000 7fa772ffd640 1 -- 172.21.15.43:0/899853908 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178066 (secure 0 0 0) 0x7fa7640539f0 con 0x7fa77c0b2b70 2023-08-09T13:34:47.722 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:47.719+0000 7fa780d8b640 1 -- 172.21.15.43:0/899853908 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fa77c0739f0 con 0x7fa77c0b2b70 2023-08-09T13:34:47.727 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:47.724+0000 7fa772ffd640 1 -- 172.21.15.43:0/899853908 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 9 v9) v1 ==== 93+0+4874 (secure 0 0 0) 0x7fa764057010 con 0x7fa77c0b2b70 2023-08-09T13:34:47.727 INFO:teuthology.orchestra.run.smithi043.stdout: 2023-08-09T13:34:47.728 INFO:teuthology.orchestra.run.smithi043.stdout:{"epoch":9,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4236,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.43:6833/3917539953","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6832","nonce":3917539953},{"type":"v1","addr":"172.21.15.43:6833","nonce":3917539953}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2},{"gid":4247,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.203:6837/2467794159","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6835","nonce":2467794159},{"type":"v1","addr":"172.21.15.203:6837","nonce":2467794159}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":4248,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.43:6835/1818380189","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6834","nonce":1818380189},{"type":"v1","addr":"172.21.15.43:6835","nonce":1818380189}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":9,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-08-09T13:34:02.575398+0000","modified":"2023-08-09T13:34:31.841945+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":4250},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4250":{"gid":4250,"name":"d","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.203:6836/798312463","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6834","nonce":798312463},{"type":"v1","addr":"172.21.15.203:6836","nonce":798312463}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[4,5],"metadata_pool":3,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-08-09T13:34:47.728 INFO:teuthology.orchestra.run.smithi043.stderr:dumped fsmap epoch 9 2023-08-09T13:34:47.731 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:47.728+0000 7fa780d8b640 1 -- 172.21.15.43:0/899853908 >> [v2:172.21.15.203:6832/830097474,v1:172.21.15.203:6833/830097474] conn(0x7fa75c06a050 msgr2=0x7fa75c06c510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:34:47.731 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:47.728+0000 7fa780d8b640 1 --2- 172.21.15.43:0/899853908 >> [v2:172.21.15.203:6832/830097474,v1:172.21.15.203:6833/830097474] conn(0x7fa75c06a050 0x7fa75c06c510 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7fa76c009a30 tx=0x7fa76c008040 comp rx=0 tx=0).stop 2023-08-09T13:34:47.731 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:47.728+0000 7fa780d8b640 1 -- 172.21.15.43:0/899853908 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa77c0b2b70 msgr2=0x7fa77c1549a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:34:47.731 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:47.728+0000 7fa780d8b640 1 --2- 172.21.15.43:0/899853908 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa77c0b2b70 0x7fa77c1549a0 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7fa76400c430 tx=0x7fa76400c8f0 comp rx=0 tx=0).stop 2023-08-09T13:34:47.731 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:47.728+0000 7fa780d8b640 1 -- 172.21.15.43:0/899853908 shutdown_connections 2023-08-09T13:34:47.731 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:47.728+0000 7fa780d8b640 1 --2- 172.21.15.43:0/899853908 >> [v2:172.21.15.203:6832/830097474,v1:172.21.15.203:6833/830097474] conn(0x7fa75c06a050 0x7fa75c06c510 unknown :-1 s=CLOSED pgs=140 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:34:47.731 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:47.728+0000 7fa780d8b640 1 --2- 172.21.15.43:0/899853908 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fa77c0b3050 0x7fa77c0af0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:34:47.732 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:47.728+0000 7fa780d8b640 1 --2- 172.21.15.43:0/899853908 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa77c0b2b70 0x7fa77c1549a0 unknown :-1 s=CLOSED pgs=238 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:34:47.732 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:47.728+0000 7fa780d8b640 1 --2- 172.21.15.43:0/899853908 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fa77c0b2790 0x7fa77c0cabc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:34:47.732 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:47.728+0000 7fa780d8b640 1 -- 172.21.15.43:0/899853908 >> 172.21.15.43:0/899853908 conn(0x7fa77c09b7b0 msgr2=0x7fa77c15acb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-09T13:34:47.732 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:47.729+0000 7fa780d8b640 1 -- 172.21.15.43:0/899853908 shutdown_connections 2023-08-09T13:34:47.733 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:47.729+0000 7fa780d8b640 1 -- 172.21.15.43:0/899853908 wait complete. 2023-08-09T13:34:48.152 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 9, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2023-08-09T13:34:02.575398+0000', 'modified': '2023-08-09T13:34:31.841945+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 4250}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_4250': {'gid': 4250, 'name': 'd', 'rank': 0, 'incarnation': 4, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.203:6836/798312463', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.203:6834', 'nonce': 798312463}, {'type': 'v1', 'addr': '172.21.15.203:6836', 'nonce': 798312463}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138322906710015, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}}, 'data_pools': [4, 5], 'metadata_pool': 3, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0} 2023-08-09T13:34:48.153 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2023-08-09T13:34:48.153 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.d status 2023-08-09T13:34:48.396 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:48.394+0000 7f04caa63640 1 Processor -- start 2023-08-09T13:34:48.396 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:48.394+0000 7f04caa63640 1 -- start start 2023-08-09T13:34:48.397 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:48.396+0000 7f04caa63640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f04c415b460 0x7f04c406b3d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:34:48.398 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:48.396+0000 7f04caa63640 1 -- --> v1:172.21.15.43:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f04c406d770 con 0x7f04c4070ab0 2023-08-09T13:34:48.398 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:48.396+0000 7f04caa63640 1 -- --> v1:172.21.15.203:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f04c406cf30 con 0x7f04c41546f0 2023-08-09T13:34:48.398 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:48.396+0000 7f04caa63640 1 -- --> v2:172.21.15.43:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f04c406d1b0 con 0x7f04c415b460 2023-08-09T13:34:48.399 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:48.396+0000 7f04c9260640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f04c415b460 0x7f04c406b3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:34:48.399 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:48.396+0000 7f04ca262640 1 --1- >> v1:172.21.15.43:6789/0 conn(0x7f04c4070ab0 0x7f04c40c84b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.43:6789/0 says I am v1:172.21.15.43:42408/0 (socket says 172.21.15.43:42408) 2023-08-09T13:34:48.399 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:48.396+0000 7f04ca262640 1 -- 172.21.15.43:0/1361667201 learned_addr learned my addr 172.21.15.43:0/1361667201 (peer_addr_for_me v1:172.21.15.43:0/0) 2023-08-09T13:34:48.399 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:48.396+0000 7f04c9260640 1 -- 172.21.15.43:0/1361667201 >> v1:172.21.15.203:6789/0 conn(0x7f04c41546f0 legacy=0x7f04c415e640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:34:48.399 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:48.396+0000 7f04c9260640 1 -- 172.21.15.43:0/1361667201 >> v1:172.21.15.43:6789/0 conn(0x7f04c4070ab0 legacy=0x7f04c40c84b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:34:48.399 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:48.396+0000 7f04c9260640 1 -- 172.21.15.43:0/1361667201 --> v2:172.21.15.43:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f04c406d490 con 0x7f04c415b460 2023-08-09T13:34:48.400 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:48.397+0000 7f04c9260640 1 --2- 172.21.15.43:0/1361667201 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f04c415b460 0x7f04c406b3d0 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7f04b40097d0 tx=0x7f04b402b8b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2494b035a926f8c9 server_cookie=0 in_seq=0 out_seq=0 2023-08-09T13:34:48.400 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:48.397+0000 7f04c8a5f640 1 -- 172.21.15.43:0/1361667201 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f04b402d040 con 0x7f04c415b460 2023-08-09T13:34:48.400 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:48.398+0000 7f04c8a5f640 1 -- 172.21.15.43:0/1361667201 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f04b4031810 con 0x7f04c415b460 2023-08-09T13:34:48.401 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:48.398+0000 7f04c8a5f640 1 -- 172.21.15.43:0/1361667201 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f04b4031ba0 con 0x7f04c415b460 2023-08-09T13:34:48.401 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:48.398+0000 7f04caa63640 1 -- 172.21.15.43:0/1361667201 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f04c415b460 msgr2=0x7f04c406b3d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:34:48.401 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:48.398+0000 7f04caa63640 1 --2- 172.21.15.43:0/1361667201 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f04c415b460 0x7f04c406b3d0 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7f04b40097d0 tx=0x7f04b402b8b0 comp rx=0 tx=0).stop 2023-08-09T13:34:48.402 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:48.398+0000 7f04caa63640 1 -- 172.21.15.43:0/1361667201 shutdown_connections 2023-08-09T13:34:48.402 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:48.398+0000 7f04caa63640 1 --2- 172.21.15.43:0/1361667201 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f04c415b460 0x7f04c406b3d0 unknown :-1 s=CLOSED pgs=240 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:34:48.402 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:48.398+0000 7f04caa63640 1 -- 172.21.15.43:0/1361667201 >> 172.21.15.43:0/1361667201 conn(0x7f04c40af950 msgr2=0x7f04c40afd60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-09T13:34:48.402 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:48.399+0000 7f04caa63640 1 -- 172.21.15.43:0/1361667201 shutdown_connections 2023-08-09T13:34:48.402 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:48.399+0000 7f04caa63640 1 -- 172.21.15.43:0/1361667201 wait complete. 2023-08-09T13:34:48.403 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:48.400+0000 7f04caa63640 1 Processor -- start 2023-08-09T13:34:48.403 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:48.400+0000 7f04caa63640 1 -- start start 2023-08-09T13:34:48.404 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:48.401+0000 7f04caa63640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f04c4070ab0 0x7f04c40b2660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:34:48.404 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:48.401+0000 7f04caa63640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f04c41546f0 0x7f04c40b2ba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:34:48.404 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:48.401+0000 7f04caa63640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f04c40c4310 0x7f04c414f0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:34:48.404 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:48.401+0000 7f04caa63640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f04c40b1730 con 0x7f04c40c4310 2023-08-09T13:34:48.404 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:48.401+0000 7f04caa63640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f04c40b3490 con 0x7f04c41546f0 2023-08-09T13:34:48.405 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:48.401+0000 7f04caa63640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f04c40b0b20 con 0x7f04c4070ab0 2023-08-09T13:34:48.406 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:48.402+0000 7f04ca262640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f04c40c4310 0x7f04c414f0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:34:48.406 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:48.402+0000 7f04ca262640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f04c40c4310 0x7f04c414f0a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:44214/0 (socket says 172.21.15.43:44214) 2023-08-09T13:34:48.406 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:48.402+0000 7f04ca262640 1 -- 172.21.15.43:0/1717845697 learned_addr learned my addr 172.21.15.43:0/1717845697 (peer_addr_for_me v2:172.21.15.43:0/0) 2023-08-09T13:34:48.406 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:48.402+0000 7f04c9a61640 1 --2- 172.21.15.43:0/1717845697 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f04c4070ab0 0x7f04c40b2660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:34:48.406 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:48.402+0000 7f04c9260640 1 --2- 172.21.15.43:0/1717845697 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f04c41546f0 0x7f04c40b2ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:34:48.407 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:48.402+0000 7f04ca262640 1 -- 172.21.15.43:0/1717845697 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f04c4070ab0 msgr2=0x7f04c40b2660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:34:48.407 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:48.402+0000 7f04ca262640 1 --2- 172.21.15.43:0/1717845697 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f04c4070ab0 0x7f04c40b2660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:34:48.407 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:48.402+0000 7f04ca262640 1 -- 172.21.15.43:0/1717845697 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f04c41546f0 msgr2=0x7f04c40b2ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:34:48.407 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:48.402+0000 7f04ca262640 1 --2- 172.21.15.43:0/1717845697 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f04c41546f0 0x7f04c40b2ba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:34:48.407 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:48.402+0000 7f04ca262640 1 -- 172.21.15.43:0/1717845697 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f04b4009400 con 0x7f04c40c4310 2023-08-09T13:34:48.407 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:48.403+0000 7f04ca262640 1 --2- 172.21.15.43:0/1717845697 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f04c40c4310 0x7f04c414f0a0 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7f04bc00cbc0 tx=0x7f04bc00a430 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-09T13:34:48.407 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:48.403+0000 7f04ba7fc640 1 -- 172.21.15.43:0/1717845697 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f04bc00ad40 con 0x7f04c40c4310 2023-08-09T13:34:48.408 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:48.403+0000 7f04ba7fc640 1 -- 172.21.15.43:0/1717845697 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f04bc012400 con 0x7f04c40c4310 2023-08-09T13:34:48.408 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:48.403+0000 7f04ba7fc640 1 -- 172.21.15.43:0/1717845697 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f04bc00aea0 con 0x7f04c40c4310 2023-08-09T13:34:48.408 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:48.403+0000 7f04caa63640 1 -- 172.21.15.43:0/1717845697 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f04c40b0e00 con 0x7f04c40c4310 2023-08-09T13:34:48.408 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:48.403+0000 7f04caa63640 1 -- 172.21.15.43:0/1717845697 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f04c40b13c0 con 0x7f04c40c4310 2023-08-09T13:34:48.408 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:48.406+0000 7f04ba7fc640 1 -- 172.21.15.43:0/1717845697 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 5) v1 ==== 88341+0+0 (secure 0 0 0) 0x7f04bc007070 con 0x7f04c40c4310 2023-08-09T13:34:48.409 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:48.407+0000 7f04ba7fc640 1 --2- 172.21.15.43:0/1717845697 >> [v2:172.21.15.203:6832/830097474,v1:172.21.15.203:6833/830097474] conn(0x7f04a806a0a0 0x7f04a806c560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:34:48.409 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:48.407+0000 7f04ba7fc640 1 -- 172.21.15.43:0/1717845697 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6871+0+0 (secure 0 0 0) 0x7f04bc088530 con 0x7f04c40c4310 2023-08-09T13:34:48.410 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:48.407+0000 7f04c9a61640 1 --2- 172.21.15.43:0/1717845697 >> [v2:172.21.15.203:6832/830097474,v1:172.21.15.203:6833/830097474] conn(0x7f04a806a0a0 0x7f04a806c560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:34:48.410 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:48.407+0000 7f04ce839740 1 Processor -- start 2023-08-09T13:34:48.410 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:48.408+0000 7f04ce839740 1 -- start start 2023-08-09T13:34:48.411 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:48.408+0000 7f04c9a61640 1 --2- 172.21.15.43:0/1717845697 >> [v2:172.21.15.203:6832/830097474,v1:172.21.15.203:6833/830097474] conn(0x7f04a806a0a0 0x7f04a806c560 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7f04c0000f60 tx=0x7f04c0009040 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-09T13:34:48.411 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:48.408+0000 7f04ce839740 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x559cfd7bbe30 0x559cfd7bc210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:34:48.411 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:48.408+0000 7f04ce839740 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x559cfd78bdb0 0x559cfd7b90a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:34:48.412 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:48.408+0000 7f04ce839740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x559cfd78f9b0 0x559cfd7b95e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:34:48.412 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:48.408+0000 7f04ce839740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x559cfd6fd8f0 con 0x559cfd78f9b0 2023-08-09T13:34:48.412 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:48.408+0000 7f04ce839740 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x559cfd6fafe0 con 0x559cfd78bdb0 2023-08-09T13:34:48.412 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:48.408+0000 7f04ce839740 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x559cfd6f68d0 con 0x559cfd7bbe30 2023-08-09T13:34:48.412 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:48.409+0000 7f04ca262640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x559cfd78f9b0 0x559cfd7b95e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:34:48.412 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:48.409+0000 7f04ca262640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x559cfd78f9b0 0x559cfd7b95e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:44228/0 (socket says 172.21.15.43:44228) 2023-08-09T13:34:48.413 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:48.409+0000 7f04ca262640 1 -- 172.21.15.43:0/2200865888 learned_addr learned my addr 172.21.15.43:0/2200865888 (peer_addr_for_me v2:172.21.15.43:0/0) 2023-08-09T13:34:48.413 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:48.409+0000 7f04c9a61640 1 --2- 172.21.15.43:0/2200865888 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x559cfd7bbe30 0x559cfd7bc210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:34:48.414 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:48.409+0000 7f04c9260640 1 --2- 172.21.15.43:0/2200865888 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x559cfd78bdb0 0x559cfd7b90a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:34:48.414 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:48.409+0000 7f04ca262640 1 -- 172.21.15.43:0/2200865888 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x559cfd7bbe30 msgr2=0x559cfd7bc210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-09T13:34:48.415 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:48.409+0000 7f04ca262640 1 --2- 172.21.15.43:0/2200865888 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x559cfd7bbe30 0x559cfd7bc210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:34:48.415 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:48.409+0000 7f04ca262640 1 -- 172.21.15.43:0/2200865888 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x559cfd78bdb0 msgr2=0x559cfd7b90a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-09T13:34:48.415 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:48.409+0000 7f04ca262640 1 --2- 172.21.15.43:0/2200865888 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x559cfd78bdb0 0x559cfd7b90a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:34:48.415 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:48.409+0000 7f04ca262640 1 -- 172.21.15.43:0/2200865888 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x559cfd6afac0 con 0x559cfd78f9b0 2023-08-09T13:34:48.415 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:48.409+0000 7f04c9260640 1 --2- 172.21.15.43:0/2200865888 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x559cfd78bdb0 0x559cfd7b90a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-08-09T13:34:48.415 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:48.409+0000 7f04ca262640 1 --2- 172.21.15.43:0/2200865888 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x559cfd78f9b0 0x559cfd7b95e0 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7f04bc05ee70 tx=0x7f04bc05f3a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ae225149e5871d33 server_cookie=0 in_seq=0 out_seq=0 2023-08-09T13:34:48.415 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:48.410+0000 7f04a77fe640 1 -- 172.21.15.43:0/2200865888 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f04bc053e50 con 0x559cfd78f9b0 2023-08-09T13:34:48.416 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:48.410+0000 7f04a77fe640 1 -- 172.21.15.43:0/2200865888 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f04bc053fb0 con 0x559cfd78f9b0 2023-08-09T13:34:48.416 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:48.410+0000 7f04a77fe640 1 -- 172.21.15.43:0/2200865888 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f04bc05d300 con 0x559cfd78f9b0 2023-08-09T13:34:48.416 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:48.410+0000 7f04ce839740 1 -- 172.21.15.43:0/2200865888 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x559cfd78f9b0 msgr2=0x559cfd7b95e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:34:48.416 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:48.410+0000 7f04ce839740 1 --2- 172.21.15.43:0/2200865888 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x559cfd78f9b0 0x559cfd7b95e0 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7f04bc05ee70 tx=0x7f04bc05f3a0 comp rx=0 tx=0).stop 2023-08-09T13:34:48.416 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:48.410+0000 7f04ce839740 1 -- 172.21.15.43:0/2200865888 shutdown_connections 2023-08-09T13:34:48.416 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:48.410+0000 7f04ce839740 1 --2- 172.21.15.43:0/2200865888 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x559cfd78f9b0 0x559cfd7b95e0 unknown :-1 s=CLOSED pgs=242 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:34:48.416 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:48.410+0000 7f04ce839740 1 --2- 172.21.15.43:0/2200865888 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x559cfd78bdb0 0x559cfd7b90a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:34:48.417 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:48.410+0000 7f04ce839740 1 --2- 172.21.15.43:0/2200865888 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x559cfd7bbe30 0x559cfd7bc210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:34:48.417 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:48.410+0000 7f04ce839740 1 -- 172.21.15.43:0/2200865888 >> 172.21.15.43:0/2200865888 conn(0x559cfd78e0c0 msgr2=0x559cfd7d6c40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-09T13:34:48.417 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:48.411+0000 7f04ce839740 1 -- 172.21.15.43:0/2200865888 shutdown_connections 2023-08-09T13:34:48.417 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:48.411+0000 7f04ce839740 1 -- 172.21.15.43:0/2200865888 wait complete. 2023-08-09T13:34:48.417 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:48.411+0000 7f04ce839740 1 -- start start 2023-08-09T13:34:48.417 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:48.411+0000 7f04ce839740 1 Processor -- start 2023-08-09T13:34:48.420 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:48.412+0000 7f04ce839740 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x559cfd78f9b0 0x559cfd7b48b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:34:48.420 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:48.412+0000 7f04ce839740 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x559cfd7bbe30 0x559cfd79fe10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:34:48.420 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:48.413+0000 7f04ce839740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x559cfd7e7680 0x559cfd7b4e80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:34:48.420 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:48.413+0000 7f04ce839740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x559cfd696d40 con 0x559cfd7e7680 2023-08-09T13:34:48.420 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:48.413+0000 7f04ce839740 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x559cfd6969e0 con 0x559cfd78f9b0 2023-08-09T13:34:48.421 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:48.413+0000 7f04ce839740 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x559cfd45cf20 con 0x559cfd7bbe30 2023-08-09T13:34:48.421 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:48.413+0000 7f04ce839740 10 client.0 _ll_register_callbacks cb 0x559cfd6af160 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-08-09T13:34:48.421 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:48.413+0000 7f04ca262640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x559cfd7e7680 0x559cfd7b4e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:34:48.421 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:48.413+0000 7f04ca262640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x559cfd7e7680 0x559cfd7b4e80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:44232/0 (socket says 172.21.15.43:44232) 2023-08-09T13:34:48.421 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:48.413+0000 7f04ca262640 1 -- 172.21.15.43:0/1863666229 learned_addr learned my addr 172.21.15.43:0/1863666229 (peer_addr_for_me v2:172.21.15.43:0/0) 2023-08-09T13:34:48.421 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:48.413+0000 7f04c9a61640 1 --2- 172.21.15.43:0/1863666229 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x559cfd78f9b0 0x559cfd7b48b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:34:48.421 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:48.413+0000 7f04c9260640 1 --2- 172.21.15.43:0/1863666229 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x559cfd7bbe30 0x559cfd79fe10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:34:48.422 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:48.413+0000 7f04ca262640 1 -- 172.21.15.43:0/1863666229 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x559cfd7bbe30 msgr2=0x559cfd79fe10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-09T13:34:48.422 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:48.413+0000 7f04ca262640 1 --2- 172.21.15.43:0/1863666229 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x559cfd7bbe30 0x559cfd79fe10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:34:48.422 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:48.413+0000 7f04ca262640 1 -- 172.21.15.43:0/1863666229 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x559cfd78f9b0 msgr2=0x559cfd7b48b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-09T13:34:48.422 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:48.413+0000 7f04ca262640 1 --2- 172.21.15.43:0/1863666229 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x559cfd78f9b0 0x559cfd7b48b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:34:48.422 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:48.413+0000 7f04ca262640 1 -- 172.21.15.43:0/1863666229 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f04c40b13c0 con 0x559cfd7e7680 2023-08-09T13:34:48.422 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:48.413+0000 7f04c9a61640 1 --2- 172.21.15.43:0/1863666229 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x559cfd78f9b0 0x559cfd7b48b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-08-09T13:34:48.422 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:48.414+0000 7f04ca262640 1 --2- 172.21.15.43:0/1863666229 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x559cfd7e7680 0x559cfd7b4e80 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7f04bc053c40 tx=0x7f04bc014070 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9a00c5f2e986fac1 server_cookie=0 in_seq=0 out_seq=0 2023-08-09T13:34:48.423 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:48.414+0000 7f04a4ff9640 10 client.0 ms_handle_connect on v2:172.21.15.43:3300/0 2023-08-09T13:34:48.423 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:48.414+0000 7f04a4ff9640 1 -- 172.21.15.43:0/1863666229 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f04bc05f120 con 0x559cfd7e7680 2023-08-09T13:34:48.423 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:48.414+0000 7f04a4ff9640 1 -- 172.21.15.43:0/1863666229 <== mon.0 v2:172.21.15.43:3300/0 2 ==== osd_map(25..25 src has 1..25) v4 ==== 6871+0+0 (secure 0 0 0) 0x7f04bc086220 con 0x559cfd7e7680 2023-08-09T13:34:48.423 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:48.414+0000 7f04ce839740 1 -- 172.21.15.43:0/1863666229 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x559cfd6ce500 con 0x559cfd7e7680 2023-08-09T13:34:48.423 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:48.414+0000 7f04a4ff9640 1 -- 172.21.15.43:0/1863666229 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_get_version_reply(handle=1 version=9) v2 ==== 24+0+0 (secure 0 0 0) 0x7f04bc086400 con 0x559cfd7e7680 2023-08-09T13:34:48.423 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:48.414+0000 7f04ce839740 10 client.4863 fetch_fsmap learned FSMap version 9 2023-08-09T13:34:48.423 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:48.414+0000 7f04ce839740 1 -- 172.21.15.43:0/1863666229 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,fsmap=9,monmap=2+}) v3 -- 0x559cfd6f4f50 con 0x559cfd7e7680 2023-08-09T13:34:48.424 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:48.414+0000 7f04a4ff9640 1 -- 172.21.15.43:0/1863666229 <== mon.0 v2:172.21.15.43:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f04bc079080 con 0x559cfd7e7680 2023-08-09T13:34:48.424 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:48.414+0000 7f04a4ff9640 1 -- 172.21.15.43:0/1863666229 <== mon.0 v2:172.21.15.43:3300/0 5 ==== fsmap(e 9) v1 ==== 3302+0+0 (secure 0 0 0) 0x7f04bc004b60 con 0x559cfd7e7680 2023-08-09T13:34:48.424 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:48.414+0000 7f04ce839740 10 client.4863 fetch_fsmap finished waiting for FSMap version 9 2023-08-09T13:34:48.424 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:48.414+0000 7f04ce839740 10 client.4863 resolve_mds: resolved name 'd' to daemon mds.d 2023-08-09T13:34:48.424 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:48.415+0000 7f04ce839740 20 client.4863 populate_metadata read hostname 'smithi043' 2023-08-09T13:34:48.424 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:48.415+0000 7f04ce839740 1 --2- 172.21.15.43:0/1863666229 >> [v2:172.21.15.203:6834/798312463,v1:172.21.15.203:6836/798312463] conn(0x559cfd7b1fc0 0x559cfd7ea4d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:34:48.424 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:48.415+0000 7f04ce839740 4 client.4863 mds_command: new command op to 4250 tid=0[{"prefix": "get_command_descriptions"}] 2023-08-09T13:34:48.425 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:48.415+0000 7f04ce839740 1 -- 172.21.15.43:0/1863666229 --> [v2:172.21.15.203:6834/798312463,v1:172.21.15.203:6836/798312463] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x559cfd6cbbc0 con 0x559cfd7b1fc0 2023-08-09T13:34:48.425 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:48.415+0000 7f04c9a61640 1 --2- 172.21.15.43:0/1863666229 >> [v2:172.21.15.203:6834/798312463,v1:172.21.15.203:6836/798312463] conn(0x559cfd7b1fc0 0x559cfd7ea4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:34:48.425 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:48.416+0000 7f04c9a61640 1 --2- 172.21.15.43:0/1863666229 >> [v2:172.21.15.203:6834/798312463,v1:172.21.15.203:6836/798312463] conn(0x559cfd7b1fc0 0x559cfd7ea4d0 crc :-1 s=READY pgs=14 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=5e1c92a853500557 server_cookie=9dc15bb50bd4d054 in_seq=0 out_seq=0 2023-08-09T13:34:48.425 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:48.416+0000 7f04a4ff9640 10 client.4863 ms_handle_connect on v2:172.21.15.203:6834/798312463 2023-08-09T13:34:48.426 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:48.419+0000 7f04a4ff9640 1 -- 172.21.15.43:0/1863666229 <== mds.0 v2:172.21.15.203:6834/798312463 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18017 (crc 0 0 0) 0x559cfd6cbbc0 con 0x559cfd7b1fc0 2023-08-09T13:34:48.426 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:48.419+0000 7f04a4ff9640 10 client.4863 handle_command_reply: tid=0 2023-08-09T13:34:48.426 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:48.419+0000 7f04ce839740 1 client.4863 shutdown 2023-08-09T13:34:48.427 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:48.419+0000 7f04ce839740 1 -- 172.21.15.43:0/1863666229 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x559cfd7e7680 msgr2=0x559cfd7b4e80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:34:48.427 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:48.419+0000 7f04ce839740 1 --2- 172.21.15.43:0/1863666229 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x559cfd7e7680 0x559cfd7b4e80 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7f04bc053c40 tx=0x7f04bc014070 comp rx=0 tx=0).stop 2023-08-09T13:34:48.427 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:48.419+0000 7f04ce839740 1 -- 172.21.15.43:0/1863666229 shutdown_connections 2023-08-09T13:34:48.427 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:48.419+0000 7f04ce839740 1 --2- 172.21.15.43:0/1863666229 >> [v2:172.21.15.203:6834/798312463,v1:172.21.15.203:6836/798312463] conn(0x559cfd7b1fc0 0x559cfd7ea4d0 crc :-1 s=READY pgs=14 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:34:48.427 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:48.419+0000 7f04ce839740 1 --2- 172.21.15.43:0/1863666229 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x559cfd7e7680 0x559cfd7b4e80 unknown :-1 s=CLOSED pgs=243 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:34:48.427 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:48.419+0000 7f04ce839740 1 --2- 172.21.15.43:0/1863666229 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x559cfd7bbe30 0x559cfd79fe10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:34:48.427 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:48.419+0000 7f04ce839740 1 --2- 172.21.15.43:0/1863666229 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x559cfd78f9b0 0x559cfd7b48b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:34:48.428 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:48.419+0000 7f04a4ff9640 0 client.4863 ms_handle_reset on v2:172.21.15.203:6834/798312463 2023-08-09T13:34:48.428 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:48.419+0000 7f04ce839740 1 -- 172.21.15.43:0/1863666229 >> 172.21.15.43:0/1863666229 conn(0x559cfd78bdb0 msgr2=0x559cfd790230 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-09T13:34:48.428 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:48.419+0000 7f04ce839740 1 -- 172.21.15.43:0/1863666229 shutdown_connections 2023-08-09T13:34:48.428 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:48.419+0000 7f04ce839740 1 -- 172.21.15.43:0/1863666229 wait complete. 2023-08-09T13:34:48.428 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:48.419+0000 7f04ce839740 20 client.4863 trim_cache size 0 max 16384 2023-08-09T13:34:48.438 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:48.436+0000 7f04ce839740 1 Processor -- start 2023-08-09T13:34:48.438 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:48.436+0000 7f04ce839740 1 -- start start 2023-08-09T13:34:48.439 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:48.436+0000 7f04ce839740 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x559cfd78f9b0 0x559cfd79fe90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:34:48.439 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:48.437+0000 7f04ce839740 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x559cfd7bbe30 0x559cfd7da820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:34:48.439 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:48.437+0000 7f04ce839740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x559cfd7e7680 0x559cfd78ffe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:34:48.440 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:48.437+0000 7f04ce839740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x559cfd7bf460 con 0x559cfd7e7680 2023-08-09T13:34:48.440 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:48.437+0000 7f04ce839740 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x559cfd7b40b0 con 0x559cfd7bbe30 2023-08-09T13:34:48.440 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:48.437+0000 7f04ce839740 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x559cfd79fc20 con 0x559cfd78f9b0 2023-08-09T13:34:48.440 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:48.437+0000 7f04ca262640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x559cfd7e7680 0x559cfd78ffe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:34:48.440 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:48.437+0000 7f04ca262640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x559cfd7e7680 0x559cfd78ffe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:44242/0 (socket says 172.21.15.43:44242) 2023-08-09T13:34:48.440 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:48.437+0000 7f04ca262640 1 -- 172.21.15.43:0/1823370526 learned_addr learned my addr 172.21.15.43:0/1823370526 (peer_addr_for_me v2:172.21.15.43:0/0) 2023-08-09T13:34:48.442 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:48.437+0000 7f04c9260640 1 --2- 172.21.15.43:0/1823370526 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x559cfd7bbe30 0x559cfd7da820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:34:48.442 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:48.437+0000 7f04c9a61640 1 --2- 172.21.15.43:0/1823370526 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x559cfd78f9b0 0x559cfd79fe90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:34:48.442 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:48.437+0000 7f04ca262640 1 -- 172.21.15.43:0/1823370526 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x559cfd78f9b0 msgr2=0x559cfd79fe90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-09T13:34:48.442 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:48.437+0000 7f04ca262640 1 --2- 172.21.15.43:0/1823370526 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x559cfd78f9b0 0x559cfd79fe90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:34:48.442 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:48.437+0000 7f04ca262640 1 -- 172.21.15.43:0/1823370526 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x559cfd7bbe30 msgr2=0x559cfd7da820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-09T13:34:48.442 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:48.437+0000 7f04ca262640 1 --2- 172.21.15.43:0/1823370526 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x559cfd7bbe30 0x559cfd7da820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:34:48.443 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:48.437+0000 7f04ca262640 1 -- 172.21.15.43:0/1823370526 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x559cfd728300 con 0x559cfd7e7680 2023-08-09T13:34:48.443 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:48.437+0000 7f04c9a61640 1 --2- 172.21.15.43:0/1823370526 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x559cfd78f9b0 0x559cfd79fe90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-08-09T13:34:48.443 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:48.437+0000 7f04c9260640 1 --2- 172.21.15.43:0/1823370526 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x559cfd7bbe30 0x559cfd7da820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-08-09T13:34:48.443 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:48.438+0000 7f04ca262640 1 --2- 172.21.15.43:0/1823370526 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x559cfd7e7680 0x559cfd78ffe0 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7f04bc0576d0 tx=0x7f04bc013000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f4adfc1dc25207ca server_cookie=0 in_seq=0 out_seq=0 2023-08-09T13:34:48.443 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:48.438+0000 7f04a7fff640 1 -- 172.21.15.43:0/1823370526 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f04bc05f120 con 0x559cfd7e7680 2023-08-09T13:34:48.443 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:48.438+0000 7f04a7fff640 1 -- 172.21.15.43:0/1823370526 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x559cfd6ce500 con 0x559cfd7e7680 2023-08-09T13:34:48.443 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:48.438+0000 7f04a7fff640 1 -- 172.21.15.43:0/1823370526 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f04bc086220 con 0x559cfd7e7680 2023-08-09T13:34:48.444 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:48.438+0000 7f04ce839740 1 -- 172.21.15.43:0/1823370526 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x559cfd7e7680 msgr2=0x559cfd78ffe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:34:48.444 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:48.438+0000 7f04ce839740 1 --2- 172.21.15.43:0/1823370526 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x559cfd7e7680 0x559cfd78ffe0 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7f04bc0576d0 tx=0x7f04bc013000 comp rx=0 tx=0).stop 2023-08-09T13:34:48.444 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:48.438+0000 7f04ce839740 1 -- 172.21.15.43:0/1823370526 shutdown_connections 2023-08-09T13:34:48.444 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:48.438+0000 7f04ce839740 1 --2- 172.21.15.43:0/1823370526 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x559cfd7e7680 0x559cfd78ffe0 unknown :-1 s=CLOSED pgs=244 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:34:48.444 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:48.438+0000 7f04ce839740 1 --2- 172.21.15.43:0/1823370526 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x559cfd7bbe30 0x559cfd7da820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:34:48.444 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:48.438+0000 7f04ce839740 1 --2- 172.21.15.43:0/1823370526 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x559cfd78f9b0 0x559cfd79fe90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:34:48.444 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:48.438+0000 7f04ce839740 1 -- 172.21.15.43:0/1823370526 >> 172.21.15.43:0/1823370526 conn(0x559cfd78bdb0 msgr2=0x559cfd7bc5f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-09T13:34:48.445 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:48.439+0000 7f04ce839740 1 -- 172.21.15.43:0/1823370526 shutdown_connections 2023-08-09T13:34:48.445 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:48.439+0000 7f04ce839740 1 -- 172.21.15.43:0/1823370526 wait complete. 2023-08-09T13:34:48.445 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:48.439+0000 7f04ce839740 1 -- start start 2023-08-09T13:34:48.445 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:48.439+0000 7f04ce839740 1 Processor -- start 2023-08-09T13:34:48.447 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:48.440+0000 7f04ce839740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x559cfd78f9b0 0x559cfd7b8cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:34:48.447 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:48.440+0000 7f04ce839740 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x559cfd7bbe30 0x559cfd8251b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:34:48.447 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:48.441+0000 7f04ce839740 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x559cfd7e7680 0x559cfd78ffe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:34:48.448 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:48.441+0000 7f04ce839740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x559cfd7b12e0 con 0x559cfd78f9b0 2023-08-09T13:34:48.448 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:48.441+0000 7f04ce839740 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x559cfd790bf0 con 0x559cfd7bbe30 2023-08-09T13:34:48.448 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:48.441+0000 7f04ce839740 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x559cfd791ee0 con 0x559cfd7e7680 2023-08-09T13:34:48.448 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:48.441+0000 7f04ce839740 10 client.0 _ll_register_callbacks cb 0x7f04bc057170 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-08-09T13:34:48.448 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:48.441+0000 7f04c9a61640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x559cfd78f9b0 0x559cfd7b8cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:34:48.448 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:48.441+0000 7f04c9a61640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x559cfd78f9b0 0x559cfd7b8cc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:44252/0 (socket says 172.21.15.43:44252) 2023-08-09T13:34:48.448 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:48.441+0000 7f04c9a61640 1 -- 172.21.15.43:0/50577295 learned_addr learned my addr 172.21.15.43:0/50577295 (peer_addr_for_me v2:172.21.15.43:0/0) 2023-08-09T13:34:48.449 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:48.441+0000 7f04c9260640 1 --2- 172.21.15.43:0/50577295 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x559cfd7bbe30 0x559cfd8251b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:34:48.449 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:48.441+0000 7f04c9a61640 1 -- 172.21.15.43:0/50577295 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x559cfd7e7680 msgr2=0x559cfd78ffe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-09T13:34:48.449 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:48.441+0000 7f04c9a61640 1 --2- 172.21.15.43:0/50577295 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x559cfd7e7680 0x559cfd78ffe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:34:48.449 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:48.441+0000 7f04c9a61640 1 -- 172.21.15.43:0/50577295 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x559cfd7bbe30 msgr2=0x559cfd8251b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-09T13:34:48.449 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:48.441+0000 7f04c9a61640 1 --2- 172.21.15.43:0/50577295 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x559cfd7bbe30 0x559cfd8251b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:34:48.449 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:48.441+0000 7f04c9a61640 1 -- 172.21.15.43:0/50577295 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x559cfd6f4f50 con 0x559cfd78f9b0 2023-08-09T13:34:48.449 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:48.442+0000 7f04c9a61640 1 --2- 172.21.15.43:0/50577295 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x559cfd78f9b0 0x559cfd7b8cc0 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x559cfd7b2450 tx=0x7f04c0000ed0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5d2107bad283dd2 server_cookie=0 in_seq=0 out_seq=0 2023-08-09T13:34:48.450 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:48.442+0000 7f04a4ff9640 10 client.0 ms_handle_connect on v2:172.21.15.43:3300/0 2023-08-09T13:34:48.450 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:48.442+0000 7f04a4ff9640 1 -- 172.21.15.43:0/50577295 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f04c0005480 con 0x559cfd78f9b0 2023-08-09T13:34:48.450 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:48.442+0000 7f04a4ff9640 1 -- 172.21.15.43:0/50577295 <== mon.0 v2:172.21.15.43:3300/0 2 ==== osd_map(25..25 src has 1..25) v4 ==== 6871+0+0 (secure 0 0 0) 0x7f04c00248f0 con 0x559cfd78f9b0 2023-08-09T13:34:48.450 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:48.442+0000 7f04ce839740 1 -- 172.21.15.43:0/50577295 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x559cfd71e800 con 0x559cfd78f9b0 2023-08-09T13:34:48.450 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:48.442+0000 7f04a4ff9640 1 -- 172.21.15.43:0/50577295 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_get_version_reply(handle=1 version=9) v2 ==== 24+0+0 (secure 0 0 0) 0x7f04c0024ab0 con 0x559cfd78f9b0 2023-08-09T13:34:48.450 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:48.442+0000 7f04ce839740 10 client.4869 fetch_fsmap learned FSMap version 9 2023-08-09T13:34:48.450 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:48.442+0000 7f04ce839740 1 -- 172.21.15.43:0/50577295 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,fsmap=9,monmap=2+}) v3 -- 0x559cfd695a10 con 0x559cfd78f9b0 2023-08-09T13:34:48.451 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:48.442+0000 7f04a4ff9640 1 -- 172.21.15.43:0/50577295 <== mon.0 v2:172.21.15.43:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x559cfd71e800 con 0x559cfd78f9b0 2023-08-09T13:34:48.451 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:48.443+0000 7f04a4ff9640 1 -- 172.21.15.43:0/50577295 <== mon.0 v2:172.21.15.43:3300/0 5 ==== fsmap(e 9) v1 ==== 3302+0+0 (secure 0 0 0) 0x7f04c0024c10 con 0x559cfd78f9b0 2023-08-09T13:34:48.451 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:48.443+0000 7f04ce839740 10 client.4869 fetch_fsmap finished waiting for FSMap version 9 2023-08-09T13:34:48.451 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:48.443+0000 7f04ce839740 10 client.4869 resolve_mds: resolved name 'd' to daemon mds.d 2023-08-09T13:34:48.451 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:48.443+0000 7f04ce839740 20 client.4869 populate_metadata read hostname 'smithi043' 2023-08-09T13:34:48.451 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:48.443+0000 7f04ce839740 1 --2- 172.21.15.43:0/50577295 >> [v2:172.21.15.203:6834/798312463,v1:172.21.15.203:6836/798312463] conn(0x559cfd7b03c0 0x559cfd7b07a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:34:48.451 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:48.443+0000 7f04ce839740 4 client.4869 mds_command: new command op to 4250 tid=0[{"prefix": "status"}] 2023-08-09T13:34:48.452 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:48.443+0000 7f04ce839740 1 -- 172.21.15.43:0/50577295 --> [v2:172.21.15.203:6834/798312463,v1:172.21.15.203:6836/798312463] -- command(tid 0: {"prefix": "status"}) v1 -- 0x559cfd6ef8b0 con 0x559cfd7b03c0 2023-08-09T13:34:48.452 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:48.443+0000 7f04c9260640 1 --2- 172.21.15.43:0/50577295 >> [v2:172.21.15.203:6834/798312463,v1:172.21.15.203:6836/798312463] conn(0x559cfd7b03c0 0x559cfd7b07a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:34:48.452 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:48.444+0000 7f04c9260640 1 --2- 172.21.15.43:0/50577295 >> [v2:172.21.15.203:6834/798312463,v1:172.21.15.203:6836/798312463] conn(0x559cfd7b03c0 0x559cfd7b07a0 crc :-1 s=READY pgs=15 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ab077e5045d02fbf server_cookie=8af97010d075305d in_seq=0 out_seq=0 2023-08-09T13:34:48.452 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:48.444+0000 7f04a4ff9640 10 client.4869 ms_handle_connect on v2:172.21.15.203:6834/798312463 2023-08-09T13:34:48.452 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:48.445+0000 7f04a4ff9640 1 -- 172.21.15.43:0/50577295 <== mds.0 v2:172.21.15.203:6834/798312463 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+330 (crc 0 0 0) 0x559cfd6ef8b0 con 0x559cfd7b03c0 2023-08-09T13:34:48.452 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:48.445+0000 7f04a4ff9640 10 client.4869 handle_command_reply: tid=0 2023-08-09T13:34:48.452 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:48.445+0000 7f04ce839740 1 client.4869 shutdown 2023-08-09T13:34:48.453 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:48.445+0000 7f04ce839740 1 -- 172.21.15.43:0/50577295 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x559cfd78f9b0 msgr2=0x559cfd7b8cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:34:48.454 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:48.445+0000 7f04ce839740 1 --2- 172.21.15.43:0/50577295 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x559cfd78f9b0 0x559cfd7b8cc0 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x559cfd7b2450 tx=0x7f04c0000ed0 comp rx=0 tx=0).stop 2023-08-09T13:34:48.454 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:48.445+0000 7f04ce839740 1 -- 172.21.15.43:0/50577295 shutdown_connections 2023-08-09T13:34:48.454 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:48.445+0000 7f04ce839740 1 --2- 172.21.15.43:0/50577295 >> [v2:172.21.15.203:6834/798312463,v1:172.21.15.203:6836/798312463] conn(0x559cfd7b03c0 0x559cfd7b07a0 crc :-1 s=READY pgs=15 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:34:48.454 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:48.445+0000 7f04a4ff9640 0 client.4869 ms_handle_reset on v2:172.21.15.203:6834/798312463 2023-08-09T13:34:48.454 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:48.445+0000 7f04ce839740 1 --2- 172.21.15.43:0/50577295 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x559cfd7e7680 0x559cfd78ffe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:34:48.454 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:48.445+0000 7f04ce839740 1 --2- 172.21.15.43:0/50577295 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x559cfd7bbe30 0x559cfd8251b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:34:48.454 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:48.445+0000 7f04ce839740 1 --2- 172.21.15.43:0/50577295 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x559cfd78f9b0 0x559cfd7b8cc0 unknown :-1 s=CLOSED pgs=245 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:34:48.455 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:48.445+0000 7f04ce839740 1 -- 172.21.15.43:0/50577295 >> 172.21.15.43:0/50577295 conn(0x559cfd78bdb0 msgr2=0x559cfd79fc20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-09T13:34:48.455 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:48.445+0000 7f04ce839740 1 -- 172.21.15.43:0/50577295 shutdown_connections 2023-08-09T13:34:48.455 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:48.445+0000 7f04ce839740 1 -- 172.21.15.43:0/50577295 wait complete. 2023-08-09T13:34:48.455 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:48.446+0000 7f04ce839740 20 client.4869 trim_cache size 0 max 16384 2023-08-09T13:34:48.455 INFO:teuthology.orchestra.run.smithi043.stdout:{ 2023-08-09T13:34:48.455 INFO:teuthology.orchestra.run.smithi043.stdout: "cluster_fsid": "daac87ee-1337-4250-a955-aea4e408a39b", 2023-08-09T13:34:48.456 INFO:teuthology.orchestra.run.smithi043.stdout: "whoami": 0, 2023-08-09T13:34:48.456 INFO:teuthology.orchestra.run.smithi043.stdout: "id": 4250, 2023-08-09T13:34:48.456 INFO:teuthology.orchestra.run.smithi043.stdout: "want_state": "up:active", 2023-08-09T13:34:48.456 INFO:teuthology.orchestra.run.smithi043.stdout: "state": "up:active", 2023-08-09T13:34:48.456 INFO:teuthology.orchestra.run.smithi043.stdout: "fs_name": "cephfs", 2023-08-09T13:34:48.456 INFO:teuthology.orchestra.run.smithi043.stdout: "rank_uptime": 45.291578530000002, 2023-08-09T13:34:48.457 INFO:teuthology.orchestra.run.smithi043.stdout: "mdsmap_epoch": 9, 2023-08-09T13:34:48.457 INFO:teuthology.orchestra.run.smithi043.stdout: "osdmap_epoch": 25, 2023-08-09T13:34:48.457 INFO:teuthology.orchestra.run.smithi043.stdout: "osdmap_epoch_barrier": 24, 2023-08-09T13:34:48.457 INFO:teuthology.orchestra.run.smithi043.stdout: "uptime": 52.323509713999997 2023-08-09T13:34:48.457 INFO:teuthology.orchestra.run.smithi043.stdout:} 2023-08-09T13:34:48.458 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:48.446+0000 7f04caa63640 1 -- 172.21.15.43:0/1717845697 >> [v2:172.21.15.203:6832/830097474,v1:172.21.15.203:6833/830097474] conn(0x7f04a806a0a0 msgr2=0x7f04a806c560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:34:48.458 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:48.446+0000 7f04caa63640 1 --2- 172.21.15.43:0/1717845697 >> [v2:172.21.15.203:6832/830097474,v1:172.21.15.203:6833/830097474] conn(0x7f04a806a0a0 0x7f04a806c560 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7f04c0000f60 tx=0x7f04c0009040 comp rx=0 tx=0).stop 2023-08-09T13:34:48.458 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:48.446+0000 7f04caa63640 1 -- 172.21.15.43:0/1717845697 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f04c40c4310 msgr2=0x7f04c414f0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:34:48.458 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:48.446+0000 7f04caa63640 1 --2- 172.21.15.43:0/1717845697 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f04c40c4310 0x7f04c414f0a0 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7f04bc00cbc0 tx=0x7f04bc00a430 comp rx=0 tx=0).stop 2023-08-09T13:34:48.458 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:48.446+0000 7f04caa63640 1 -- 172.21.15.43:0/1717845697 shutdown_connections 2023-08-09T13:34:48.459 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:48.446+0000 7f04caa63640 1 --2- 172.21.15.43:0/1717845697 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f04c40c4310 0x7f04c414f0a0 unknown :-1 s=CLOSED pgs=241 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:34:48.459 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:48.446+0000 7f04caa63640 1 --2- 172.21.15.43:0/1717845697 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f04c41546f0 0x7f04c40b2ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:34:48.459 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:48.446+0000 7f04caa63640 1 --2- 172.21.15.43:0/1717845697 >> [v2:172.21.15.203:6832/830097474,v1:172.21.15.203:6833/830097474] conn(0x7f04a806a0a0 0x7f04a806c560 unknown :-1 s=CLOSED pgs=141 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:34:48.459 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:48.446+0000 7f04caa63640 1 --2- 172.21.15.43:0/1717845697 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f04c4070ab0 0x7f04c40b2660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:34:48.459 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:48.446+0000 7f04caa63640 1 -- 172.21.15.43:0/1717845697 >> 172.21.15.43:0/1717845697 conn(0x7f04c40af950 msgr2=0x7f04c406fc70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-09T13:34:48.459 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:48.446+0000 7f04caa63640 1 -- 172.21.15.43:0/1717845697 shutdown_connections 2023-08-09T13:34:48.459 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:48.446+0000 7f04caa63640 1 -- 172.21.15.43:0/1717845697 wait complete. 2023-08-09T13:34:48.466 INFO:teuthology.run_tasks:Running task ceph-fuse... 2023-08-09T13:34:48.475 INFO:tasks.ceph_fuse:Running ceph_fuse task... 2023-08-09T13:34:48.476 INFO:tasks.ceph_fuse:config is {'client.0': {}, 'client.2': {}, 'client.1': {}, 'client.3': {}} 2023-08-09T13:34:48.476 INFO:tasks.ceph_fuse:client.0 config is {} 2023-08-09T13:34:48.476 INFO:tasks.cephfs.mount:cephfs_mntpt = None 2023-08-09T13:34:48.476 INFO:tasks.cephfs.mount:hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2023-08-09T13:34:48.477 INFO:tasks.ceph_fuse:client.2 config is {} 2023-08-09T13:34:48.477 INFO:tasks.cephfs.mount:cephfs_mntpt = None 2023-08-09T13:34:48.477 INFO:tasks.cephfs.mount:hostfs_mntpt = /home/ubuntu/cephtest/mnt.2 2023-08-09T13:34:48.477 INFO:tasks.ceph_fuse:client.1 config is {} 2023-08-09T13:34:48.478 INFO:tasks.cephfs.mount:cephfs_mntpt = None 2023-08-09T13:34:48.478 INFO:tasks.cephfs.mount:hostfs_mntpt = /home/ubuntu/cephtest/mnt.1 2023-08-09T13:34:48.478 INFO:tasks.ceph_fuse:client.3 config is {} 2023-08-09T13:34:48.478 INFO:tasks.cephfs.mount:cephfs_mntpt = None 2023-08-09T13:34:48.478 INFO:tasks.cephfs.mount:hostfs_mntpt = /home/ubuntu/cephtest/mnt.3 2023-08-09T13:34:48.479 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-09T13:34:48.479 DEBUG:teuthology.orchestra.run.smithi043:> ip netns list 2023-08-09T13:34:48.536 INFO:teuthology.orchestra.run.smithi043.stdout:ceph-ns--home-ubuntu-cephtest-mnt.admin (id: 0) 2023-08-09T13:34:48.537 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-09T13:34:48.537 DEBUG:teuthology.orchestra.run.smithi043:> sudo ip netns delete ceph-ns--home-ubuntu-cephtest-mnt.admin 2023-08-09T13:34:48.646 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-09T13:34:48.646 DEBUG:teuthology.orchestra.run.smithi043:> sudo ip link delete ceph-brx 2023-08-09T13:34:48.792 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-09T13:34:48.793 DEBUG:teuthology.orchestra.run.smithi203:> ip netns list 2023-08-09T13:34:48.851 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-09T13:34:48.852 DEBUG:teuthology.orchestra.run.smithi203:> sudo ip link delete ceph-brx 2023-08-09T13:34:48.947 INFO:teuthology.orchestra.run.smithi203.stderr:Cannot find device "ceph-brx" 2023-08-09T13:34:48.950 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-08-09T13:34:48.951 INFO:tasks.ceph_fuse:Mounting ceph-fuse clients... 2023-08-09T13:34:48.951 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2023-08-09T13:34:49.181 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:49.179+0000 7f6bbb57f640 1 Processor -- start 2023-08-09T13:34:49.182 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:49.179+0000 7f6bbb57f640 1 -- start start 2023-08-09T13:34:49.183 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:49.181+0000 7f6bbb57f640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f6bb40c41c0 0x7f6bb4142290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:34:49.183 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:49.182+0000 7f6bbb57f640 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7f6bb40697c0 0x7f6bb4069c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:34:49.183 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:49.182+0000 7f6bbb57f640 1 -- --> v1:172.21.15.203:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6bb4141b00 con 0x7f6bb40c3de0 2023-08-09T13:34:49.183 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:49.182+0000 7f6bbb57f640 1 -- --> v2:172.21.15.203:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6bb415d040 con 0x7f6bb40697c0 2023-08-09T13:34:49.184 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:49.182+0000 7f6bbb57f640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6bb406a170 con 0x7f6bb40c41c0 2023-08-09T13:34:49.184 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:49.182+0000 7f6bb9d7c640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f6bb40c41c0 0x7f6bb4142290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:34:49.184 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:49.182+0000 7f6bb9d7c640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f6bb40c41c0 0x7f6bb4142290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.203:3301/0 says I am v2:172.21.15.43:57954/0 (socket says 172.21.15.43:57954) 2023-08-09T13:34:49.184 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:49.182+0000 7f6bb9d7c640 1 -- 172.21.15.43:0/2307810806 learned_addr learned my addr 172.21.15.43:0/2307810806 (peer_addr_for_me v2:172.21.15.43:0/0) 2023-08-09T13:34:49.185 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:49.182+0000 7f6bbad7e640 1 --2- 172.21.15.43:0/2307810806 >> v2:172.21.15.203:3300/0 conn(0x7f6bb40697c0 0x7f6bb4069c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:34:49.185 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:49.182+0000 7f6bb9d7c640 1 -- 172.21.15.43:0/2307810806 >> v2:172.21.15.203:3300/0 conn(0x7f6bb40697c0 msgr2=0x7f6bb4069c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-09T13:34:49.185 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:49.182+0000 7f6bb9d7c640 1 --2- 172.21.15.43:0/2307810806 >> v2:172.21.15.203:3300/0 conn(0x7f6bb40697c0 0x7f6bb4069c30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:34:49.185 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:49.182+0000 7f6bb9d7c640 1 -- 172.21.15.43:0/2307810806 >> v1:172.21.15.203:6789/0 conn(0x7f6bb40c3de0 legacy=0x7f6bb415c930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:34:49.186 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:49.182+0000 7f6bb9d7c640 1 -- 172.21.15.43:0/2307810806 --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6bb40c4640 con 0x7f6bb40c41c0 2023-08-09T13:34:49.186 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:49.184+0000 7f6bb9d7c640 1 --2- 172.21.15.43:0/2307810806 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f6bb40c41c0 0x7f6bb4142290 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f6ba4009870 tx=0x7f6ba402b980 comp rx=0 tx=0).ready entity=mon.2 client_cookie=9ccffaebc187fa5 server_cookie=0 in_seq=0 out_seq=0 2023-08-09T13:34:49.187 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:49.184+0000 7f6bb957b640 1 -- 172.21.15.43:0/2307810806 <== mon.2 v2:172.21.15.203:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6ba4030840 con 0x7f6bb40c41c0 2023-08-09T13:34:49.187 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:49.184+0000 7f6bb957b640 1 -- 172.21.15.43:0/2307810806 <== mon.2 v2:172.21.15.203:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6ba40309a0 con 0x7f6bb40c41c0 2023-08-09T13:34:49.187 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:49.184+0000 7f6bb957b640 1 -- 172.21.15.43:0/2307810806 <== mon.2 v2:172.21.15.203:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6ba4030d30 con 0x7f6bb40c41c0 2023-08-09T13:34:49.188 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:49.185+0000 7f6bbb57f640 1 -- 172.21.15.43:0/2307810806 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f6bb40c41c0 msgr2=0x7f6bb4142290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:34:49.188 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:49.185+0000 7f6bbb57f640 1 --2- 172.21.15.43:0/2307810806 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f6bb40c41c0 0x7f6bb4142290 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f6ba4009870 tx=0x7f6ba402b980 comp rx=0 tx=0).stop 2023-08-09T13:34:49.188 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:49.185+0000 7f6bbb57f640 1 -- 172.21.15.43:0/2307810806 shutdown_connections 2023-08-09T13:34:49.188 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:49.185+0000 7f6bbb57f640 1 --2- 172.21.15.43:0/2307810806 >> v2:172.21.15.203:3300/0 conn(0x7f6bb40697c0 0x7f6bb4069c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:34:49.188 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:49.185+0000 7f6bbb57f640 1 --2- 172.21.15.43:0/2307810806 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f6bb40c41c0 0x7f6bb4142290 unknown :-1 s=CLOSED pgs=55 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:34:49.189 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:49.185+0000 7f6bbb57f640 1 -- 172.21.15.43:0/2307810806 >> 172.21.15.43:0/2307810806 conn(0x7f6bb40b0b70 msgr2=0x7f6bb40b0f50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-09T13:34:49.189 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:49.185+0000 7f6bbb57f640 1 -- 172.21.15.43:0/2307810806 shutdown_connections 2023-08-09T13:34:49.189 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:49.185+0000 7f6bbb57f640 1 -- 172.21.15.43:0/2307810806 wait complete. 2023-08-09T13:34:49.190 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:49.189+0000 7f6bbb57f640 1 Processor -- start 2023-08-09T13:34:49.190 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:49.189+0000 7f6bbb57f640 1 -- start start 2023-08-09T13:34:49.191 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:49.190+0000 7f6bbb57f640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f6bb40697c0 0x7f6bb40b2870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:34:49.192 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:49.190+0000 7f6bbb57f640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f6bb40c3de0 0x7f6bb40b4c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:34:49.192 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:49.190+0000 7f6bbb57f640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f6bb40c41c0 0x7f6bb415e8c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:34:49.192 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:49.190+0000 7f6bbb57f640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6bb40ac250 con 0x7f6bb40c41c0 2023-08-09T13:34:49.192 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:49.190+0000 7f6bbb57f640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6bb41f29e0 con 0x7f6bb40697c0 2023-08-09T13:34:49.192 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:49.190+0000 7f6bbb57f640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6bb40b23e0 con 0x7f6bb40c3de0 2023-08-09T13:34:49.193 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:49.190+0000 7f6bbad7e640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f6bb40c41c0 0x7f6bb415e8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:34:49.194 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:49.190+0000 7f6bbad7e640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f6bb40c41c0 0x7f6bb415e8c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:44254/0 (socket says 172.21.15.43:44254) 2023-08-09T13:34:49.194 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:49.190+0000 7f6bbad7e640 1 -- 172.21.15.43:0/2789000340 learned_addr learned my addr 172.21.15.43:0/2789000340 (peer_addr_for_me v2:172.21.15.43:0/0) 2023-08-09T13:34:49.194 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:49.190+0000 7f6bb9d7c640 1 --2- 172.21.15.43:0/2789000340 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f6bb40c3de0 0x7f6bb40b4c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:34:49.194 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:49.190+0000 7f6bba57d640 1 --2- 172.21.15.43:0/2789000340 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f6bb40697c0 0x7f6bb40b2870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:34:49.194 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:49.190+0000 7f6bbad7e640 1 -- 172.21.15.43:0/2789000340 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f6bb40c3de0 msgr2=0x7f6bb40b4c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:34:49.194 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:49.190+0000 7f6bbad7e640 1 --2- 172.21.15.43:0/2789000340 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f6bb40c3de0 0x7f6bb40b4c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:34:49.194 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:49.190+0000 7f6bbad7e640 1 -- 172.21.15.43:0/2789000340 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f6bb40697c0 msgr2=0x7f6bb40b2870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:34:49.195 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:49.190+0000 7f6bbad7e640 1 --2- 172.21.15.43:0/2789000340 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f6bb40697c0 0x7f6bb40b2870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:34:49.195 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:49.190+0000 7f6bbad7e640 1 -- 172.21.15.43:0/2789000340 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6ba40094d0 con 0x7f6bb40c41c0 2023-08-09T13:34:49.195 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:49.191+0000 7f6bbad7e640 1 --2- 172.21.15.43:0/2789000340 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f6bb40c41c0 0x7f6bb415e8c0 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7f6bac00e930 tx=0x7f6bac00edf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-09T13:34:49.195 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:49.191+0000 7f6baaffd640 1 -- 172.21.15.43:0/2789000340 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6bac009690 con 0x7f6bb40c41c0 2023-08-09T13:34:49.195 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:49.191+0000 7f6baaffd640 1 -- 172.21.15.43:0/2789000340 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6bac00c980 con 0x7f6bb40c41c0 2023-08-09T13:34:49.195 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:49.191+0000 7f6bbb57f640 1 -- 172.21.15.43:0/2789000340 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6bb40b51c0 con 0x7f6bb40c41c0 2023-08-09T13:34:49.195 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:49.191+0000 7f6baaffd640 1 -- 172.21.15.43:0/2789000340 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6bac00cd30 con 0x7f6bb40c41c0 2023-08-09T13:34:49.196 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:49.191+0000 7f6bbb57f640 1 -- 172.21.15.43:0/2789000340 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6bb40b1d90 con 0x7f6bb40c41c0 2023-08-09T13:34:49.196 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:49.192+0000 7f6bbb57f640 1 -- 172.21.15.43:0/2789000340 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6bb40b2550 con 0x7f6bb40c41c0 2023-08-09T13:34:49.196 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:49.193+0000 7f6baaffd640 1 -- 172.21.15.43:0/2789000340 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 5) v1 ==== 88341+0+0 (secure 0 0 0) 0x7f6bac013470 con 0x7f6bb40c41c0 2023-08-09T13:34:49.197 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:49.195+0000 7f6baaffd640 1 --2- 172.21.15.43:0/2789000340 >> [v2:172.21.15.203:6832/830097474,v1:172.21.15.203:6833/830097474] conn(0x7f6b9806a050 0x7f6b9806c510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:34:49.198 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:49.195+0000 7f6baaffd640 1 -- 172.21.15.43:0/2789000340 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 6871+0+0 (secure 0 0 0) 0x7f6bac088a70 con 0x7f6bb40c41c0 2023-08-09T13:34:49.198 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:49.196+0000 7f6bba57d640 1 --2- 172.21.15.43:0/2789000340 >> [v2:172.21.15.203:6832/830097474,v1:172.21.15.203:6833/830097474] conn(0x7f6b9806a050 0x7f6b9806c510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:34:49.199 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:49.197+0000 7f6bba57d640 1 --2- 172.21.15.43:0/2789000340 >> [v2:172.21.15.203:6832/830097474,v1:172.21.15.203:6833/830097474] conn(0x7f6b9806a050 0x7f6b9806c510 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7f6bb0001f10 tx=0x7f6bb0009040 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-09T13:34:49.202 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:49.199+0000 7f6baaffd640 1 -- 172.21.15.43:0/2789000340 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178066 (secure 0 0 0) 0x7f6bac053f10 con 0x7f6bb40c41c0 2023-08-09T13:34:49.439 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:49.438+0000 7f6bbb57f640 1 -- 172.21.15.43:0/2789000340 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7f6bb40739f0 con 0x7f6bb40c41c0 2023-08-09T13:34:49.440 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:49.439+0000 7f6baaffd640 1 -- 172.21.15.43:0/2789000340 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v9) v1 ==== 53+0+88 (secure 0 0 0) 0x7f6bac057530 con 0x7f6bb40c41c0 2023-08-09T13:34:49.440 INFO:teuthology.orchestra.run.smithi043.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data cephfs_data_ec ] 2023-08-09T13:34:49.443 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:49.441+0000 7f6bbb57f640 1 -- 172.21.15.43:0/2789000340 >> [v2:172.21.15.203:6832/830097474,v1:172.21.15.203:6833/830097474] conn(0x7f6b9806a050 msgr2=0x7f6b9806c510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:34:49.443 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:49.441+0000 7f6bbb57f640 1 --2- 172.21.15.43:0/2789000340 >> [v2:172.21.15.203:6832/830097474,v1:172.21.15.203:6833/830097474] conn(0x7f6b9806a050 0x7f6b9806c510 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7f6bb0001f10 tx=0x7f6bb0009040 comp rx=0 tx=0).stop 2023-08-09T13:34:49.443 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:49.441+0000 7f6bbb57f640 1 -- 172.21.15.43:0/2789000340 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f6bb40c41c0 msgr2=0x7f6bb415e8c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:34:49.443 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:49.441+0000 7f6bbb57f640 1 --2- 172.21.15.43:0/2789000340 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f6bb40c41c0 0x7f6bb415e8c0 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7f6bac00e930 tx=0x7f6bac00edf0 comp rx=0 tx=0).stop 2023-08-09T13:34:49.444 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:49.441+0000 7f6bbb57f640 1 -- 172.21.15.43:0/2789000340 shutdown_connections 2023-08-09T13:34:49.444 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:49.441+0000 7f6bbb57f640 1 --2- 172.21.15.43:0/2789000340 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f6bb40c41c0 0x7f6bb415e8c0 unknown :-1 s=CLOSED pgs=246 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:34:49.444 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:49.441+0000 7f6bbb57f640 1 --2- 172.21.15.43:0/2789000340 >> [v2:172.21.15.203:6832/830097474,v1:172.21.15.203:6833/830097474] conn(0x7f6b9806a050 0x7f6b9806c510 unknown :-1 s=CLOSED pgs=142 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:34:49.444 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:49.441+0000 7f6bbb57f640 1 --2- 172.21.15.43:0/2789000340 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f6bb40c3de0 0x7f6bb40b4c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:34:49.444 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:49.441+0000 7f6bbb57f640 1 --2- 172.21.15.43:0/2789000340 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f6bb40697c0 0x7f6bb40b2870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:34:49.444 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:49.441+0000 7f6bbb57f640 1 -- 172.21.15.43:0/2789000340 >> 172.21.15.43:0/2789000340 conn(0x7f6bb40b0b70 msgr2=0x7f6bb40ce100 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-09T13:34:49.444 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:49.442+0000 7f6bbb57f640 1 -- 172.21.15.43:0/2789000340 shutdown_connections 2023-08-09T13:34:49.445 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:49.442+0000 7f6bbb57f640 1 -- 172.21.15.43:0/2789000340 wait complete. 2023-08-09T13:34:49.462 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2023-08-09T13:34:49.463 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2023-08-09T13:34:49.463 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi043.front.sepia.ceph.com 2023-08-09T13:34:49.463 INFO:tasks.cephfs.mount:self.client.name = client.0 2023-08-09T13:34:49.463 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2023-08-09T13:34:49.464 INFO:tasks.cephfs.mount:self.cephfs_name = None 2023-08-09T13:34:49.464 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2023-08-09T13:34:49.464 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2023-08-09T13:34:49.464 INFO:tasks.cephfs.mount:Setting the 'None' netns for '/home/ubuntu/cephtest/mnt.0' 2023-08-09T13:34:49.465 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-09T13:34:49.465 DEBUG:teuthology.orchestra.run.smithi043:> ip addr 2023-08-09T13:34:49.520 INFO:teuthology.orchestra.run.smithi043.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2023-08-09T13:34:49.520 INFO:teuthology.orchestra.run.smithi043.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2023-08-09T13:34:49.520 INFO:teuthology.orchestra.run.smithi043.stdout: inet 127.0.0.1/8 scope host lo 2023-08-09T13:34:49.520 INFO:teuthology.orchestra.run.smithi043.stdout: valid_lft forever preferred_lft forever 2023-08-09T13:34:49.521 INFO:teuthology.orchestra.run.smithi043.stdout: inet6 ::1/128 scope host 2023-08-09T13:34:49.521 INFO:teuthology.orchestra.run.smithi043.stdout: valid_lft forever preferred_lft forever 2023-08-09T13:34:49.521 INFO:teuthology.orchestra.run.smithi043.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-08-09T13:34:49.521 INFO:teuthology.orchestra.run.smithi043.stdout: link/ether 0c:c4:7a:6b:fd:9c brd ff:ff:ff:ff:ff:ff 2023-08-09T13:34:49.521 INFO:teuthology.orchestra.run.smithi043.stdout: altname enp7s0f0 2023-08-09T13:34:49.521 INFO:teuthology.orchestra.run.smithi043.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-08-09T13:34:49.521 INFO:teuthology.orchestra.run.smithi043.stdout: link/ether 0c:c4:7a:6b:fd:9d brd ff:ff:ff:ff:ff:ff 2023-08-09T13:34:49.522 INFO:teuthology.orchestra.run.smithi043.stdout: altname enp7s0f1 2023-08-09T13:34:49.522 INFO:teuthology.orchestra.run.smithi043.stdout:4: ens1f0: mtu 1500 qdisc mq state UP group default qlen 1000 2023-08-09T13:34:49.522 INFO:teuthology.orchestra.run.smithi043.stdout: link/ether 0c:c4:7a:bd:13:58 brd ff:ff:ff:ff:ff:ff 2023-08-09T13:34:49.522 INFO:teuthology.orchestra.run.smithi043.stdout: altname enp1s0f0 2023-08-09T13:34:49.522 INFO:teuthology.orchestra.run.smithi043.stdout: inet 172.21.15.43/20 brd 172.21.15.255 scope global dynamic noprefixroute ens1f0 2023-08-09T13:34:49.522 INFO:teuthology.orchestra.run.smithi043.stdout: valid_lft 42214sec preferred_lft 42214sec 2023-08-09T13:34:49.522 INFO:teuthology.orchestra.run.smithi043.stdout: inet6 fe80::ec4:7aff:febd:1358/64 scope link 2023-08-09T13:34:49.522 INFO:teuthology.orchestra.run.smithi043.stdout: valid_lft forever preferred_lft forever 2023-08-09T13:34:49.523 INFO:teuthology.orchestra.run.smithi043.stdout:5: ens1f1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-08-09T13:34:49.523 INFO:teuthology.orchestra.run.smithi043.stdout: link/ether 0c:c4:7a:bd:13:59 brd ff:ff:ff:ff:ff:ff 2023-08-09T13:34:49.523 INFO:teuthology.orchestra.run.smithi043.stdout: altname enp1s0f1 2023-08-09T13:34:49.524 INFO:tasks.cephfs.mount:Setuping the 'ceph-brx' with 192.168.255.254/16 2023-08-09T13:34:49.524 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-09T13:34:49.524 DEBUG:teuthology.orchestra.run.smithi043:> (cd / && exec bash -c ' 2023-08-09T13:34:49.525 DEBUG:teuthology.orchestra.run.smithi043:> set -e 2023-08-09T13:34:49.525 DEBUG:teuthology.orchestra.run.smithi043:> sudo ip link add name ceph-brx type bridge 2023-08-09T13:34:49.525 DEBUG:teuthology.orchestra.run.smithi043:> sudo ip addr flush dev ceph-brx 2023-08-09T13:34:49.525 DEBUG:teuthology.orchestra.run.smithi043:> sudo ip link set ceph-brx up 2023-08-09T13:34:49.526 DEBUG:teuthology.orchestra.run.smithi043:> sudo ip addr add 192.168.255.254/16 brd 192.168.255.255 dev ceph-brx 2023-08-09T13:34:49.526 DEBUG:teuthology.orchestra.run.smithi043:> ') 2023-08-09T13:34:49.777 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-09T13:34:49.778 DEBUG:teuthology.orchestra.run.smithi043:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2023-08-09T13:34:49.865 INFO:teuthology.orchestra.run.smithi043.stdout:1 2023-08-09T13:34:49.869 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-09T13:34:49.869 DEBUG:teuthology.orchestra.run.smithi043:> route 2023-08-09T13:34:49.919 INFO:teuthology.orchestra.run.smithi043.stdout:Kernel IP routing table 2023-08-09T13:34:49.919 INFO:teuthology.orchestra.run.smithi043.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2023-08-09T13:34:49.919 INFO:teuthology.orchestra.run.smithi043.stdout:default _gateway 0.0.0.0 UG 100 0 0 ens1f0 2023-08-09T13:34:49.920 INFO:teuthology.orchestra.run.smithi043.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 ens1f0 2023-08-09T13:34:49.920 INFO:teuthology.orchestra.run.smithi043.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2023-08-09T13:34:49.921 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-09T13:34:49.921 DEBUG:teuthology.orchestra.run.smithi043:> (cd / && exec bash -c ' 2023-08-09T13:34:49.921 DEBUG:teuthology.orchestra.run.smithi043:> set -e 2023-08-09T13:34:49.921 DEBUG:teuthology.orchestra.run.smithi043:> sudo iptables -A FORWARD -o ens1f0 -i ceph-brx -j ACCEPT 2023-08-09T13:34:49.922 DEBUG:teuthology.orchestra.run.smithi043:> sudo iptables -A FORWARD -i ens1f0 -o ceph-brx -j ACCEPT 2023-08-09T13:34:49.922 DEBUG:teuthology.orchestra.run.smithi043:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o ens1f0 -j MASQUERADE 2023-08-09T13:34:49.922 DEBUG:teuthology.orchestra.run.smithi043:> ') 2023-08-09T13:34:50.111 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-09T13:34:50.112 DEBUG:teuthology.orchestra.run.smithi043:> ip netns list 2023-08-09T13:34:50.170 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-09T13:34:50.171 DEBUG:teuthology.orchestra.run.smithi043:> ip netns list-id 2023-08-09T13:34:50.228 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-09T13:34:50.228 DEBUG:teuthology.orchestra.run.smithi043:> (cd / && exec bash -c ' 2023-08-09T13:34:50.229 DEBUG:teuthology.orchestra.run.smithi043:> set -e 2023-08-09T13:34:50.229 DEBUG:teuthology.orchestra.run.smithi043:> sudo ip netns add ceph-ns--home-ubuntu-cephtest-mnt.0 2023-08-09T13:34:50.230 DEBUG:teuthology.orchestra.run.smithi043:> sudo ip netns set ceph-ns--home-ubuntu-cephtest-mnt.0 0 2023-08-09T13:34:50.230 DEBUG:teuthology.orchestra.run.smithi043:> ') 2023-08-09T13:34:50.386 INFO:tasks.cephfs.mount:Setuping the netns 'ceph-ns--home-ubuntu-cephtest-mnt.0' with 192.168.0.1/16 2023-08-09T13:34:50.386 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-09T13:34:50.387 DEBUG:teuthology.orchestra.run.smithi043:> (cd / && exec bash -c ' 2023-08-09T13:34:50.387 DEBUG:teuthology.orchestra.run.smithi043:> set -e 2023-08-09T13:34:50.387 DEBUG:teuthology.orchestra.run.smithi043:> sudo ip link add veth0 netns ceph-ns--home-ubuntu-cephtest-mnt.0 type veth peer name brx.0 2023-08-09T13:34:50.387 DEBUG:teuthology.orchestra.run.smithi043:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip addr add 192.168.0.1/16 brd 192.168.255.255 dev veth0 2023-08-09T13:34:50.388 DEBUG:teuthology.orchestra.run.smithi043:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip link set veth0 up 2023-08-09T13:34:50.388 DEBUG:teuthology.orchestra.run.smithi043:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip link set lo up 2023-08-09T13:34:50.388 DEBUG:teuthology.orchestra.run.smithi043:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip route add default via 192.168.255.254 2023-08-09T13:34:50.388 DEBUG:teuthology.orchestra.run.smithi043:> ') 2023-08-09T13:34:50.657 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-09T13:34:50.658 DEBUG:teuthology.orchestra.run.smithi043:> (cd / && exec bash -c ' 2023-08-09T13:34:50.658 DEBUG:teuthology.orchestra.run.smithi043:> set -e 2023-08-09T13:34:50.658 DEBUG:teuthology.orchestra.run.smithi043:> sudo ip link set brx.0 up 2023-08-09T13:34:50.658 DEBUG:teuthology.orchestra.run.smithi043:> sudo ip link set dev brx.0 master ceph-brx 2023-08-09T13:34:50.659 DEBUG:teuthology.orchestra.run.smithi043:> ') 2023-08-09T13:34:50.833 INFO:tasks.cephfs.fuse_mount:Client client.0 config is {} 2023-08-09T13:34:50.834 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-08-09T13:34:50.834 DEBUG:teuthology.orchestra.run.smithi043:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2023-08-09T13:34:50.888 INFO:teuthology.orchestra.run.smithi043.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2023-08-09T13:34:50.889 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-08-09T13:34:50.889 DEBUG:teuthology.orchestra.run.smithi043:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2023-08-09T13:34:50.944 DEBUG:teuthology.orchestra.run.smithi043:> sudo modprobe fuse 2023-08-09T13:34:51.045 INFO:teuthology.orchestra.run:Running command with timeout 30 2023-08-09T13:34:51.045 DEBUG:teuthology.orchestra.run.smithi043:> sudo mount -t fusectl /sys/fs/fuse/connections /sys/fs/fuse/connections 2023-08-09T13:34:51.151 INFO:teuthology.orchestra.run.smithi043.stderr:mount: /sys/fs/fuse/connections: /sys/fs/fuse/connections already mounted or mount point busy. 2023-08-09T13:34:51.156 DEBUG:teuthology.orchestra.run:got remote process result: 32 2023-08-09T13:34:51.157 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-09T13:34:51.158 DEBUG:teuthology.orchestra.run.smithi043:> ls /sys/fs/fuse/connections 2023-08-09T13:34:51.217 INFO:tasks.cephfs.fuse_mount:Pre-mount connections: [] 2023-08-09T13:34:51.217 DEBUG:teuthology.orchestra.run.smithi043:> (cd /home/ubuntu/cephtest && exec sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-fuse -f --admin-socket '/var/run/ceph/$cluster-$name.$pid.asok' /home/ubuntu/cephtest/mnt.0 --id 0) 2023-08-09T13:34:51.224 DEBUG:teuthology.orchestra.run.smithi043:> sudo modprobe fuse 2023-08-09T13:34:51.304 INFO:teuthology.orchestra.run:Running command with timeout 30 2023-08-09T13:34:51.305 DEBUG:teuthology.orchestra.run.smithi043:> sudo mount -t fusectl /sys/fs/fuse/connections /sys/fs/fuse/connections 2023-08-09T13:34:51.384 INFO:teuthology.orchestra.run.smithi043.stderr:mount: /sys/fs/fuse/connections: /sys/fs/fuse/connections already mounted or mount point busy. 2023-08-09T13:34:51.387 DEBUG:teuthology.orchestra.run:got remote process result: 32 2023-08-09T13:34:51.389 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-09T13:34:51.390 DEBUG:teuthology.orchestra.run.smithi043:> ls /sys/fs/fuse/connections 2023-08-09T13:34:51.428 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi043.stderr:2023-08-09T13:34:51.427+0000 7fc95f133480 -1 init, newargv = 0x5574dbaa94b0 newargc=15 2023-08-09T13:34:51.429 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi043.stderr:ceph-fuse[98188]: starting ceph client 2023-08-09T13:34:51.442 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi043.stderr:ceph-fuse[98188]: starting fuse 2023-08-09T13:34:52.541 DEBUG:teuthology.orchestra.run.smithi043:> sudo modprobe fuse 2023-08-09T13:34:52.656 INFO:teuthology.orchestra.run:Running command with timeout 30 2023-08-09T13:34:52.656 DEBUG:teuthology.orchestra.run.smithi043:> sudo mount -t fusectl /sys/fs/fuse/connections /sys/fs/fuse/connections 2023-08-09T13:34:52.759 INFO:teuthology.orchestra.run.smithi043.stderr:mount: /sys/fs/fuse/connections: /sys/fs/fuse/connections already mounted or mount point busy. 2023-08-09T13:34:52.762 DEBUG:teuthology.orchestra.run:got remote process result: 32 2023-08-09T13:34:52.763 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-09T13:34:52.764 DEBUG:teuthology.orchestra.run.smithi043:> ls /sys/fs/fuse/connections 2023-08-09T13:34:52.825 INFO:teuthology.orchestra.run.smithi043.stdout:40 2023-08-09T13:34:52.826 INFO:tasks.cephfs.fuse_mount:Post-mount connections: [40] 2023-08-09T13:34:52.826 DEBUG:teuthology.orchestra.run.smithi043:> sudo stdin-killer -- python3 -c ' 2023-08-09T13:34:52.827 DEBUG:teuthology.orchestra.run.smithi043:> import glob 2023-08-09T13:34:52.827 DEBUG:teuthology.orchestra.run.smithi043:> import re 2023-08-09T13:34:52.827 DEBUG:teuthology.orchestra.run.smithi043:> import os 2023-08-09T13:34:52.827 DEBUG:teuthology.orchestra.run.smithi043:> import subprocess 2023-08-09T13:34:52.828 DEBUG:teuthology.orchestra.run.smithi043:> 2023-08-09T13:34:52.828 DEBUG:teuthology.orchestra.run.smithi043:> def _find_admin_socket(client_name): 2023-08-09T13:34:52.828 DEBUG:teuthology.orchestra.run.smithi043:> asok_path = "/var/run/ceph/ceph-client.0.*.asok" 2023-08-09T13:34:52.828 DEBUG:teuthology.orchestra.run.smithi043:> files = glob.glob(asok_path) 2023-08-09T13:34:52.829 DEBUG:teuthology.orchestra.run.smithi043:> mountpoint = "/home/ubuntu/cephtest/mnt.0" 2023-08-09T13:34:52.829 DEBUG:teuthology.orchestra.run.smithi043:> 2023-08-09T13:34:52.829 DEBUG:teuthology.orchestra.run.smithi043:> # Given a non-glob path, it better be there 2023-08-09T13:34:52.829 DEBUG:teuthology.orchestra.run.smithi043:> if "*" not in asok_path: 2023-08-09T13:34:52.830 DEBUG:teuthology.orchestra.run.smithi043:> assert(len(files) == 1) 2023-08-09T13:34:52.830 DEBUG:teuthology.orchestra.run.smithi043:> return files[0] 2023-08-09T13:34:52.830 DEBUG:teuthology.orchestra.run.smithi043:> 2023-08-09T13:34:52.830 DEBUG:teuthology.orchestra.run.smithi043:> for f in files: 2023-08-09T13:34:52.831 DEBUG:teuthology.orchestra.run.smithi043:> pid = re.match(".*\.(\d+)\.asok$", f).group(1) 2023-08-09T13:34:52.831 DEBUG:teuthology.orchestra.run.smithi043:> if os.path.exists("/proc/{0}".format(pid)): 2023-08-09T13:34:52.831 DEBUG:teuthology.orchestra.run.smithi043:> with open("/proc/{0}/cmdline".format(pid), '"'"'r'"'"') as proc_f: 2023-08-09T13:34:52.831 DEBUG:teuthology.orchestra.run.smithi043:> contents = proc_f.read() 2023-08-09T13:34:52.831 DEBUG:teuthology.orchestra.run.smithi043:> if mountpoint in contents: 2023-08-09T13:34:52.832 DEBUG:teuthology.orchestra.run.smithi043:> return f 2023-08-09T13:34:52.832 DEBUG:teuthology.orchestra.run.smithi043:> raise RuntimeError("Client socket {0} not found".format(client_name)) 2023-08-09T13:34:52.832 DEBUG:teuthology.orchestra.run.smithi043:> 2023-08-09T13:34:52.833 DEBUG:teuthology.orchestra.run.smithi043:> print(_find_admin_socket("client.0")) 2023-08-09T13:34:52.833 DEBUG:teuthology.orchestra.run.smithi043:> ' 2023-08-09T13:34:53.028 INFO:teuthology.orchestra.run.smithi043.stdout:/var/run/ceph/ceph-client.0.98188.asok 2023-08-09T13:34:53.033 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:53 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2023-08-09T13:34:53.046 INFO:tasks.cephfs.fuse_mount:Found client admin socket at /var/run/ceph/ceph-client.0.98188.asok 2023-08-09T13:34:53.046 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-09T13:34:53.047 DEBUG:teuthology.orchestra.run.smithi043:> sudo ceph --admin-daemon /var/run/ceph/ceph-client.0.98188.asok status 2023-08-09T13:34:53.299 INFO:teuthology.orchestra.run.smithi043.stdout:{ 2023-08-09T13:34:53.299 INFO:teuthology.orchestra.run.smithi043.stdout: "metadata": { 2023-08-09T13:34:53.299 INFO:teuthology.orchestra.run.smithi043.stdout: "ceph_sha1": "668a86f232c9454b2ce5928d1432a29a72964d85", 2023-08-09T13:34:53.299 INFO:teuthology.orchestra.run.smithi043.stdout: "ceph_version": "ceph version 18.0.0-5389-g668a86f2 (668a86f232c9454b2ce5928d1432a29a72964d85) reef (dev)", 2023-08-09T13:34:53.300 INFO:teuthology.orchestra.run.smithi043.stdout: "entity_id": "0", 2023-08-09T13:34:53.300 INFO:teuthology.orchestra.run.smithi043.stdout: "hostname": "smithi043", 2023-08-09T13:34:53.300 INFO:teuthology.orchestra.run.smithi043.stdout: "mount_point": "/home/ubuntu/cephtest/mnt.0", 2023-08-09T13:34:53.300 INFO:teuthology.orchestra.run.smithi043.stdout: "pid": "98188", 2023-08-09T13:34:53.300 INFO:teuthology.orchestra.run.smithi043.stdout: "root": "/" 2023-08-09T13:34:53.300 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2023-08-09T13:34:53.300 INFO:teuthology.orchestra.run.smithi043.stdout: "dentry_count": 0, 2023-08-09T13:34:53.301 INFO:teuthology.orchestra.run.smithi043.stdout: "dentry_pinned_count": 0, 2023-08-09T13:34:53.301 INFO:teuthology.orchestra.run.smithi043.stdout: "id": 4875, 2023-08-09T13:34:53.301 INFO:teuthology.orchestra.run.smithi043.stdout: "inst": { 2023-08-09T13:34:53.301 INFO:teuthology.orchestra.run.smithi043.stdout: "name": { 2023-08-09T13:34:53.301 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "client", 2023-08-09T13:34:53.301 INFO:teuthology.orchestra.run.smithi043.stdout: "num": 4875 2023-08-09T13:34:53.301 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2023-08-09T13:34:53.301 INFO:teuthology.orchestra.run.smithi043.stdout: "addr": { 2023-08-09T13:34:53.302 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "v1", 2023-08-09T13:34:53.302 INFO:teuthology.orchestra.run.smithi043.stdout: "addr": "172.21.15.43:0", 2023-08-09T13:34:53.302 INFO:teuthology.orchestra.run.smithi043.stdout: "nonce": 2316698253 2023-08-09T13:34:53.302 INFO:teuthology.orchestra.run.smithi043.stdout: } 2023-08-09T13:34:53.302 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2023-08-09T13:34:53.302 INFO:teuthology.orchestra.run.smithi043.stdout: "addr": { 2023-08-09T13:34:53.302 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "v1", 2023-08-09T13:34:53.302 INFO:teuthology.orchestra.run.smithi043.stdout: "addr": "172.21.15.43:0", 2023-08-09T13:34:53.303 INFO:teuthology.orchestra.run.smithi043.stdout: "nonce": 2316698253 2023-08-09T13:34:53.303 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2023-08-09T13:34:53.303 INFO:teuthology.orchestra.run.smithi043.stdout: "inst_str": "client.4875 172.21.15.43:0/2316698253", 2023-08-09T13:34:53.303 INFO:teuthology.orchestra.run.smithi043.stdout: "addr_str": "172.21.15.43:0/2316698253", 2023-08-09T13:34:53.303 INFO:teuthology.orchestra.run.smithi043.stdout: "inode_count": 1, 2023-08-09T13:34:53.303 INFO:teuthology.orchestra.run.smithi043.stdout: "mds_epoch": 9, 2023-08-09T13:34:53.303 INFO:teuthology.orchestra.run.smithi043.stdout: "osd_epoch": 26, 2023-08-09T13:34:53.303 INFO:teuthology.orchestra.run.smithi043.stdout: "osd_epoch_barrier": 0, 2023-08-09T13:34:53.304 INFO:teuthology.orchestra.run.smithi043.stdout: "blocklisted": false, 2023-08-09T13:34:53.304 INFO:teuthology.orchestra.run.smithi043.stdout: "fs_name": "cephfs" 2023-08-09T13:34:53.304 INFO:teuthology.orchestra.run.smithi043.stdout:} 2023-08-09T13:34:53.314 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2023-08-09T13:34:53.538 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:53.536+0000 7f7797fff640 1 Processor -- start 2023-08-09T13:34:53.538 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:53.536+0000 7f7797fff640 1 -- start start 2023-08-09T13:34:53.539 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:53.538+0000 7f7797fff640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f77980c8750 0x7f77980ad340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:34:53.539 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:53.538+0000 7f7797fff640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f77980c8b30 0x7f779806f410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:34:53.539 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:53.538+0000 7f7797fff640 1 -- --> v1:172.21.15.203:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f77980cbd90 con 0x7f77980acdf0 2023-08-09T13:34:53.540 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:53.538+0000 7f7797fff640 1 -- --> v2:172.21.15.43:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f779815e810 con 0x7f77980c8750 2023-08-09T13:34:53.540 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:53.538+0000 7f7797fff640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f77980c9f70 con 0x7f77980c8b30 2023-08-09T13:34:53.540 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:53.538+0000 7f7796ffd640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f77980c8750 0x7f77980ad340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:34:53.541 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:53.538+0000 7f7796ffd640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f77980c8750 0x7f77980ad340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:44256/0 (socket says 172.21.15.43:44256) 2023-08-09T13:34:53.541 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:53.538+0000 7f7796ffd640 1 -- 172.21.15.43:0/743928322 learned_addr learned my addr 172.21.15.43:0/743928322 (peer_addr_for_me v2:172.21.15.43:0/0) 2023-08-09T13:34:53.541 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:53.538+0000 7f778ffff640 1 --2- 172.21.15.43:0/743928322 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f77980c8b30 0x7f779806f410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:34:53.541 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:53.539+0000 7f7796ffd640 1 -- 172.21.15.43:0/743928322 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f77980c8b30 msgr2=0x7f779806f410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-09T13:34:53.541 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:53.539+0000 7f7796ffd640 1 --2- 172.21.15.43:0/743928322 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f77980c8b30 0x7f779806f410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:34:53.542 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:53.539+0000 7f7796ffd640 1 -- 172.21.15.43:0/743928322 >> v1:172.21.15.203:6789/0 conn(0x7f77980acdf0 legacy=0x7f7798070f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:34:53.542 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:53.539+0000 7f7796ffd640 1 -- 172.21.15.43:0/743928322 --> v2:172.21.15.43:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f77980ca2b0 con 0x7f77980c8750 2023-08-09T13:34:53.542 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:53.540+0000 7f7796ffd640 1 --2- 172.21.15.43:0/743928322 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f77980c8750 0x7f77980ad340 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7f7788009870 tx=0x7f778802b980 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a23224148af79efd server_cookie=0 in_seq=0 out_seq=0 2023-08-09T13:34:53.543 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:53.540+0000 7f77967fc640 1 -- 172.21.15.43:0/743928322 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7788030840 con 0x7f77980c8750 2023-08-09T13:34:53.543 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:53.540+0000 7f77967fc640 1 -- 172.21.15.43:0/743928322 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f77880309a0 con 0x7f77980c8750 2023-08-09T13:34:53.543 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:53.540+0000 7f77967fc640 1 -- 172.21.15.43:0/743928322 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7788030d30 con 0x7f77980c8750 2023-08-09T13:34:53.543 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:53.541+0000 7f7797fff640 1 -- 172.21.15.43:0/743928322 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f77980c8750 msgr2=0x7f77980ad340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:34:53.543 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:53.541+0000 7f7797fff640 1 --2- 172.21.15.43:0/743928322 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f77980c8750 0x7f77980ad340 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7f7788009870 tx=0x7f778802b980 comp rx=0 tx=0).stop 2023-08-09T13:34:53.543 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:53.541+0000 7f7797fff640 1 -- 172.21.15.43:0/743928322 shutdown_connections 2023-08-09T13:34:53.544 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:53.541+0000 7f7797fff640 1 --2- 172.21.15.43:0/743928322 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f77980c8b30 0x7f779806f410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:34:53.544 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:53.541+0000 7f7797fff640 1 --2- 172.21.15.43:0/743928322 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f77980c8750 0x7f77980ad340 unknown :-1 s=CLOSED pgs=248 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:34:53.544 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:53.541+0000 7f7797fff640 1 -- 172.21.15.43:0/743928322 >> 172.21.15.43:0/743928322 conn(0x7f77980cde90 msgr2=0x7f77980ce2a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-09T13:34:53.544 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:53.541+0000 7f7797fff640 1 -- 172.21.15.43:0/743928322 shutdown_connections 2023-08-09T13:34:53.544 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:53.541+0000 7f7797fff640 1 -- 172.21.15.43:0/743928322 wait complete. 2023-08-09T13:34:53.545 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:53.543+0000 7f7797fff640 1 Processor -- start 2023-08-09T13:34:53.545 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:53.543+0000 7f7797fff640 1 -- start start 2023-08-09T13:34:53.545 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:53.544+0000 7f7797fff640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f77980acdf0 0x7f779814d290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:34:53.546 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:53.544+0000 7f7797fff640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f77980c8750 0x7f77980c3c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:34:53.546 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:53.544+0000 7f7797fff640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f77980c8b30 0x7f779809a110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:34:53.546 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:53.544+0000 7f7797fff640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f779814d7d0 con 0x7f77980c8750 2023-08-09T13:34:53.546 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:53.544+0000 7f7797fff640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f779809b830 con 0x7f77980c8b30 2023-08-09T13:34:53.546 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:53.544+0000 7f7797fff640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f779809af90 con 0x7f77980acdf0 2023-08-09T13:34:53.547 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:53.545+0000 7f778ffff640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f77980c8750 0x7f77980c3c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:34:53.548 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:53.545+0000 7f778ffff640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f77980c8750 0x7f77980c3c70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:44272/0 (socket says 172.21.15.43:44272) 2023-08-09T13:34:53.548 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:53.545+0000 7f778ffff640 1 -- 172.21.15.43:0/3461638924 learned_addr learned my addr 172.21.15.43:0/3461638924 (peer_addr_for_me v2:172.21.15.43:0/0) 2023-08-09T13:34:53.548 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:53.545+0000 7f7796ffd640 1 --2- 172.21.15.43:0/3461638924 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f77980acdf0 0x7f779814d290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:34:53.548 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:53.545+0000 7f77977fe640 1 --2- 172.21.15.43:0/3461638924 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f77980c8b30 0x7f779809a110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:34:53.548 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:53.545+0000 7f778ffff640 1 -- 172.21.15.43:0/3461638924 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f77980acdf0 msgr2=0x7f779814d290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:34:53.548 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:53.545+0000 7f778ffff640 1 --2- 172.21.15.43:0/3461638924 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f77980acdf0 0x7f779814d290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:34:53.548 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:53.545+0000 7f778ffff640 1 -- 172.21.15.43:0/3461638924 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f77980c8b30 msgr2=0x7f779809a110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:34:53.549 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:53.545+0000 7f778ffff640 1 --2- 172.21.15.43:0/3461638924 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f77980c8b30 0x7f779809a110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:34:53.549 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:53.545+0000 7f778ffff640 1 -- 172.21.15.43:0/3461638924 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f77880094d0 con 0x7f77980c8750 2023-08-09T13:34:53.549 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:53.545+0000 7f77977fe640 1 --2- 172.21.15.43:0/3461638924 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f77980c8b30 0x7f779809a110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-08-09T13:34:53.549 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:53.546+0000 7f778ffff640 1 --2- 172.21.15.43:0/3461638924 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f77980c8750 0x7f77980c3c70 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7f778000d5d0 tx=0x7f778000da90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-09T13:34:53.549 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:53.546+0000 7f778f7fe640 1 -- 172.21.15.43:0/3461638924 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f77800125f0 con 0x7f77980c8750 2023-08-09T13:34:53.549 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:53.546+0000 7f778f7fe640 1 -- 172.21.15.43:0/3461638924 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7780012770 con 0x7f77980c8750 2023-08-09T13:34:53.549 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:53.546+0000 7f778f7fe640 1 -- 172.21.15.43:0/3461638924 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7780012b20 con 0x7f77980c8750 2023-08-09T13:34:53.549 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:53.546+0000 7f7797fff640 1 -- 172.21.15.43:0/3461638924 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f779809b270 con 0x7f77980c8750 2023-08-09T13:34:53.550 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:53.546+0000 7f7797fff640 1 -- 172.21.15.43:0/3461638924 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f779815ab90 con 0x7f77980c8750 2023-08-09T13:34:53.550 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:53.547+0000 7f7797fff640 1 -- 172.21.15.43:0/3461638924 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f77980c1920 con 0x7f77980c8750 2023-08-09T13:34:53.550 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:53.548+0000 7f778f7fe640 1 -- 172.21.15.43:0/3461638924 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 5) v1 ==== 88341+0+0 (secure 0 0 0) 0x7f7780012c80 con 0x7f77980c8750 2023-08-09T13:34:53.551 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:53.549+0000 7f778f7fe640 1 --2- 172.21.15.43:0/3461638924 >> [v2:172.21.15.203:6832/830097474,v1:172.21.15.203:6833/830097474] conn(0x7f777806a050 0x7f777806c510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:34:53.551 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:53.549+0000 7f778f7fe640 1 -- 172.21.15.43:0/3461638924 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 6871+0+0 (secure 0 0 0) 0x7f7780088d90 con 0x7f77980c8750 2023-08-09T13:34:53.551 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:53.550+0000 7f7796ffd640 1 --2- 172.21.15.43:0/3461638924 >> [v2:172.21.15.203:6832/830097474,v1:172.21.15.203:6833/830097474] conn(0x7f777806a050 0x7f777806c510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:34:53.552 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:53.551+0000 7f7796ffd640 1 --2- 172.21.15.43:0/3461638924 >> [v2:172.21.15.203:6832/830097474,v1:172.21.15.203:6833/830097474] conn(0x7f777806a050 0x7f777806c510 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7f7788036000 tx=0x7f7788008030 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-09T13:34:53.555 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:53.554+0000 7f778f7fe640 1 -- 172.21.15.43:0/3461638924 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178066 (secure 0 0 0) 0x7f77980c1920 con 0x7f77980c8750 2023-08-09T13:34:53.788 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:53.786+0000 7f7797fff640 1 -- 172.21.15.43:0/3461638924 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7f77980c1920 con 0x7f77980c8750 2023-08-09T13:34:53.788 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:53.787+0000 7f778f7fe640 1 -- 172.21.15.43:0/3461638924 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v9) v1 ==== 53+0+88 (secure 0 0 0) 0x7f77980c1920 con 0x7f77980c8750 2023-08-09T13:34:53.788 INFO:teuthology.orchestra.run.smithi043.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data cephfs_data_ec ] 2023-08-09T13:34:53.791 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:53.789+0000 7f7797fff640 1 -- 172.21.15.43:0/3461638924 >> [v2:172.21.15.203:6832/830097474,v1:172.21.15.203:6833/830097474] conn(0x7f777806a050 msgr2=0x7f777806c510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:34:53.791 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:53.789+0000 7f7797fff640 1 --2- 172.21.15.43:0/3461638924 >> [v2:172.21.15.203:6832/830097474,v1:172.21.15.203:6833/830097474] conn(0x7f777806a050 0x7f777806c510 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7f7788036000 tx=0x7f7788008030 comp rx=0 tx=0).stop 2023-08-09T13:34:53.791 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:53.789+0000 7f7797fff640 1 -- 172.21.15.43:0/3461638924 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f77980c8750 msgr2=0x7f77980c3c70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:34:53.791 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:53.789+0000 7f7797fff640 1 --2- 172.21.15.43:0/3461638924 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f77980c8750 0x7f77980c3c70 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7f778000d5d0 tx=0x7f778000da90 comp rx=0 tx=0).stop 2023-08-09T13:34:53.791 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:53.789+0000 7f7797fff640 1 -- 172.21.15.43:0/3461638924 shutdown_connections 2023-08-09T13:34:53.791 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:53.789+0000 7f7797fff640 1 --2- 172.21.15.43:0/3461638924 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f77980c8b30 0x7f779809a110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:34:53.792 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:53.789+0000 7f7797fff640 1 --2- 172.21.15.43:0/3461638924 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f77980c8750 0x7f77980c3c70 unknown :-1 s=CLOSED pgs=249 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:34:53.792 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:53.789+0000 7f7797fff640 1 --2- 172.21.15.43:0/3461638924 >> [v2:172.21.15.203:6832/830097474,v1:172.21.15.203:6833/830097474] conn(0x7f777806a050 0x7f777806c510 unknown :-1 s=CLOSED pgs=143 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:34:53.792 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:53.789+0000 7f7797fff640 1 --2- 172.21.15.43:0/3461638924 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f77980acdf0 0x7f779814d290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:34:53.792 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:53.789+0000 7f7797fff640 1 -- 172.21.15.43:0/3461638924 >> 172.21.15.43:0/3461638924 conn(0x7f77980cde90 msgr2=0x7f77980abde0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-09T13:34:53.792 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:53.789+0000 7f7797fff640 1 -- 172.21.15.43:0/3461638924 shutdown_connections 2023-08-09T13:34:53.793 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:53.789+0000 7f7797fff640 1 -- 172.21.15.43:0/3461638924 wait complete. 2023-08-09T13:34:53.812 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2023-08-09T13:34:53.812 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2023-08-09T13:34:53.813 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi043.front.sepia.ceph.com 2023-08-09T13:34:53.813 INFO:tasks.cephfs.mount:self.client.name = client.2 2023-08-09T13:34:53.813 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.2 2023-08-09T13:34:53.813 INFO:tasks.cephfs.mount:self.cephfs_name = None 2023-08-09T13:34:53.813 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2023-08-09T13:34:53.814 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2023-08-09T13:34:53.814 INFO:tasks.cephfs.mount:Setting the 'None' netns for '/home/ubuntu/cephtest/mnt.2' 2023-08-09T13:34:53.814 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-09T13:34:53.814 DEBUG:teuthology.orchestra.run.smithi043:> ip addr 2023-08-09T13:34:53.873 INFO:teuthology.orchestra.run.smithi043.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2023-08-09T13:34:53.874 INFO:teuthology.orchestra.run.smithi043.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2023-08-09T13:34:53.874 INFO:teuthology.orchestra.run.smithi043.stdout: inet 127.0.0.1/8 scope host lo 2023-08-09T13:34:53.874 INFO:teuthology.orchestra.run.smithi043.stdout: valid_lft forever preferred_lft forever 2023-08-09T13:34:53.874 INFO:teuthology.orchestra.run.smithi043.stdout: inet6 ::1/128 scope host 2023-08-09T13:34:53.874 INFO:teuthology.orchestra.run.smithi043.stdout: valid_lft forever preferred_lft forever 2023-08-09T13:34:53.874 INFO:teuthology.orchestra.run.smithi043.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-08-09T13:34:53.875 INFO:teuthology.orchestra.run.smithi043.stdout: link/ether 0c:c4:7a:6b:fd:9c brd ff:ff:ff:ff:ff:ff 2023-08-09T13:34:53.875 INFO:teuthology.orchestra.run.smithi043.stdout: altname enp7s0f0 2023-08-09T13:34:53.875 INFO:teuthology.orchestra.run.smithi043.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-08-09T13:34:53.875 INFO:teuthology.orchestra.run.smithi043.stdout: link/ether 0c:c4:7a:6b:fd:9d brd ff:ff:ff:ff:ff:ff 2023-08-09T13:34:53.875 INFO:teuthology.orchestra.run.smithi043.stdout: altname enp7s0f1 2023-08-09T13:34:53.875 INFO:teuthology.orchestra.run.smithi043.stdout:4: ens1f0: mtu 1500 qdisc mq state UP group default qlen 1000 2023-08-09T13:34:53.875 INFO:teuthology.orchestra.run.smithi043.stdout: link/ether 0c:c4:7a:bd:13:58 brd ff:ff:ff:ff:ff:ff 2023-08-09T13:34:53.876 INFO:teuthology.orchestra.run.smithi043.stdout: altname enp1s0f0 2023-08-09T13:34:53.876 INFO:teuthology.orchestra.run.smithi043.stdout: inet 172.21.15.43/20 brd 172.21.15.255 scope global dynamic noprefixroute ens1f0 2023-08-09T13:34:53.876 INFO:teuthology.orchestra.run.smithi043.stdout: valid_lft 42210sec preferred_lft 42210sec 2023-08-09T13:34:53.876 INFO:teuthology.orchestra.run.smithi043.stdout: inet6 fe80::ec4:7aff:febd:1358/64 scope link 2023-08-09T13:34:53.876 INFO:teuthology.orchestra.run.smithi043.stdout: valid_lft forever preferred_lft forever 2023-08-09T13:34:53.876 INFO:teuthology.orchestra.run.smithi043.stdout:5: ens1f1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-08-09T13:34:53.876 INFO:teuthology.orchestra.run.smithi043.stdout: link/ether 0c:c4:7a:bd:13:59 brd ff:ff:ff:ff:ff:ff 2023-08-09T13:34:53.876 INFO:teuthology.orchestra.run.smithi043.stdout: altname enp1s0f1 2023-08-09T13:34:53.877 INFO:teuthology.orchestra.run.smithi043.stdout:8: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2023-08-09T13:34:53.877 INFO:teuthology.orchestra.run.smithi043.stdout: link/ether 7e:93:26:fa:07:8f brd ff:ff:ff:ff:ff:ff 2023-08-09T13:34:53.877 INFO:teuthology.orchestra.run.smithi043.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2023-08-09T13:34:53.877 INFO:teuthology.orchestra.run.smithi043.stdout: valid_lft forever preferred_lft forever 2023-08-09T13:34:53.877 INFO:teuthology.orchestra.run.smithi043.stdout: inet6 fe80::e494:65ff:fe92:a182/64 scope link 2023-08-09T13:34:53.877 INFO:teuthology.orchestra.run.smithi043.stdout: valid_lft forever preferred_lft forever 2023-08-09T13:34:53.877 INFO:teuthology.orchestra.run.smithi043.stdout:9: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2023-08-09T13:34:53.878 INFO:teuthology.orchestra.run.smithi043.stdout: link/ether 7e:93:26:fa:07:8f brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.0 2023-08-09T13:34:53.878 INFO:teuthology.orchestra.run.smithi043.stdout: inet6 fe80::7c93:26ff:fefa:78f/64 scope link 2023-08-09T13:34:53.878 INFO:teuthology.orchestra.run.smithi043.stdout: valid_lft forever preferred_lft forever 2023-08-09T13:34:53.879 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-09T13:34:53.879 DEBUG:teuthology.orchestra.run.smithi043:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2023-08-09T13:34:53.978 INFO:teuthology.orchestra.run.smithi043.stdout:1 2023-08-09T13:34:53.982 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-09T13:34:53.983 DEBUG:teuthology.orchestra.run.smithi043:> route 2023-08-09T13:34:54.042 INFO:teuthology.orchestra.run.smithi043.stdout:Kernel IP routing table 2023-08-09T13:34:54.043 INFO:teuthology.orchestra.run.smithi043.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2023-08-09T13:34:54.043 INFO:teuthology.orchestra.run.smithi043.stdout:default _gateway 0.0.0.0 UG 100 0 0 ens1f0 2023-08-09T13:34:54.043 INFO:teuthology.orchestra.run.smithi043.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 ens1f0 2023-08-09T13:34:54.043 INFO:teuthology.orchestra.run.smithi043.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2023-08-09T13:34:54.044 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-09T13:34:54.044 DEBUG:teuthology.orchestra.run.smithi043:> (cd / && exec bash -c ' 2023-08-09T13:34:54.044 DEBUG:teuthology.orchestra.run.smithi043:> set -e 2023-08-09T13:34:54.044 DEBUG:teuthology.orchestra.run.smithi043:> sudo iptables -A FORWARD -o ens1f0 -i ceph-brx -j ACCEPT 2023-08-09T13:34:54.045 DEBUG:teuthology.orchestra.run.smithi043:> sudo iptables -A FORWARD -i ens1f0 -o ceph-brx -j ACCEPT 2023-08-09T13:34:54.045 DEBUG:teuthology.orchestra.run.smithi043:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o ens1f0 -j MASQUERADE 2023-08-09T13:34:54.045 DEBUG:teuthology.orchestra.run.smithi043:> ') 2023-08-09T13:34:54.255 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-09T13:34:54.256 DEBUG:teuthology.orchestra.run.smithi043:> ip netns list 2023-08-09T13:34:54.312 INFO:teuthology.orchestra.run.smithi043.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2023-08-09T13:34:54.313 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-09T13:34:54.314 DEBUG:teuthology.orchestra.run.smithi043:> ip netns list-id 2023-08-09T13:34:54.368 INFO:teuthology.orchestra.run.smithi043.stdout:nsid 0 (iproute2 netns name: ceph-ns--home-ubuntu-cephtest-mnt.0) 2023-08-09T13:34:54.370 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-09T13:34:54.370 DEBUG:teuthology.orchestra.run.smithi043:> (cd / && exec bash -c ' 2023-08-09T13:34:54.370 DEBUG:teuthology.orchestra.run.smithi043:> set -e 2023-08-09T13:34:54.370 DEBUG:teuthology.orchestra.run.smithi043:> sudo ip netns add ceph-ns--home-ubuntu-cephtest-mnt.2 2023-08-09T13:34:54.371 DEBUG:teuthology.orchestra.run.smithi043:> sudo ip netns set ceph-ns--home-ubuntu-cephtest-mnt.2 1 2023-08-09T13:34:54.371 DEBUG:teuthology.orchestra.run.smithi043:> ') 2023-08-09T13:34:54.528 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-09T13:34:54.528 DEBUG:teuthology.orchestra.run.smithi043:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip addr 2023-08-09T13:34:54.623 INFO:teuthology.orchestra.run.smithi043.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2023-08-09T13:34:54.623 INFO:teuthology.orchestra.run.smithi043.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2023-08-09T13:34:54.623 INFO:teuthology.orchestra.run.smithi043.stdout: inet 127.0.0.1/8 scope host lo 2023-08-09T13:34:54.624 INFO:teuthology.orchestra.run.smithi043.stdout: valid_lft forever preferred_lft forever 2023-08-09T13:34:54.624 INFO:teuthology.orchestra.run.smithi043.stdout: inet6 ::1/128 scope host 2023-08-09T13:34:54.624 INFO:teuthology.orchestra.run.smithi043.stdout: valid_lft forever preferred_lft forever 2023-08-09T13:34:54.624 INFO:teuthology.orchestra.run.smithi043.stdout:2: veth0@if9: mtu 1500 qdisc noqueue state UP group default qlen 1000 2023-08-09T13:34:54.624 INFO:teuthology.orchestra.run.smithi043.stdout: link/ether ae:dc:1f:6e:fb:f4 brd ff:ff:ff:ff:ff:ff link-netnsid 0 2023-08-09T13:34:54.624 INFO:teuthology.orchestra.run.smithi043.stdout: inet 192.168.0.1/16 brd 192.168.255.255 scope global veth0 2023-08-09T13:34:54.624 INFO:teuthology.orchestra.run.smithi043.stdout: valid_lft forever preferred_lft forever 2023-08-09T13:34:54.625 INFO:teuthology.orchestra.run.smithi043.stdout: inet6 fe80::acdc:1fff:fe6e:fbf4/64 scope link 2023-08-09T13:34:54.625 INFO:teuthology.orchestra.run.smithi043.stdout: valid_lft forever preferred_lft forever 2023-08-09T13:34:54.628 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-09T13:34:54.628 DEBUG:teuthology.orchestra.run.smithi043:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip addr 2023-08-09T13:34:54.731 INFO:teuthology.orchestra.run.smithi043.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2023-08-09T13:34:54.731 INFO:teuthology.orchestra.run.smithi043.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2023-08-09T13:34:54.732 INFO:teuthology.orchestra.run.smithi043.stdout: inet 127.0.0.1/8 scope host lo 2023-08-09T13:34:54.732 INFO:teuthology.orchestra.run.smithi043.stdout: valid_lft forever preferred_lft forever 2023-08-09T13:34:54.732 INFO:teuthology.orchestra.run.smithi043.stdout: inet6 ::1/128 scope host 2023-08-09T13:34:54.732 INFO:teuthology.orchestra.run.smithi043.stdout: valid_lft forever preferred_lft forever 2023-08-09T13:34:54.733 INFO:teuthology.orchestra.run.smithi043.stdout:2: veth0@if9: mtu 1500 qdisc noqueue state UP group default qlen 1000 2023-08-09T13:34:54.733 INFO:teuthology.orchestra.run.smithi043.stdout: link/ether ae:dc:1f:6e:fb:f4 brd ff:ff:ff:ff:ff:ff link-netnsid 0 2023-08-09T13:34:54.733 INFO:teuthology.orchestra.run.smithi043.stdout: inet 192.168.0.1/16 brd 192.168.255.255 scope global veth0 2023-08-09T13:34:54.733 INFO:teuthology.orchestra.run.smithi043.stdout: valid_lft forever preferred_lft forever 2023-08-09T13:34:54.734 INFO:teuthology.orchestra.run.smithi043.stdout: inet6 fe80::acdc:1fff:fe6e:fbf4/64 scope link 2023-08-09T13:34:54.734 INFO:teuthology.orchestra.run.smithi043.stdout: valid_lft forever preferred_lft forever 2023-08-09T13:34:54.736 INFO:tasks.cephfs.mount:Setuping the netns 'ceph-ns--home-ubuntu-cephtest-mnt.2' with 192.168.0.2/16 2023-08-09T13:34:54.736 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-09T13:34:54.737 DEBUG:teuthology.orchestra.run.smithi043:> (cd / && exec bash -c ' 2023-08-09T13:34:54.737 DEBUG:teuthology.orchestra.run.smithi043:> set -e 2023-08-09T13:34:54.738 DEBUG:teuthology.orchestra.run.smithi043:> sudo ip link add veth0 netns ceph-ns--home-ubuntu-cephtest-mnt.2 type veth peer name brx.1 2023-08-09T13:34:54.738 DEBUG:teuthology.orchestra.run.smithi043:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.2 ip addr add 192.168.0.2/16 brd 192.168.255.255 dev veth0 2023-08-09T13:34:54.739 DEBUG:teuthology.orchestra.run.smithi043:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.2 ip link set veth0 up 2023-08-09T13:34:54.739 DEBUG:teuthology.orchestra.run.smithi043:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.2 ip link set lo up 2023-08-09T13:34:54.739 DEBUG:teuthology.orchestra.run.smithi043:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.2 ip route add default via 192.168.255.254 2023-08-09T13:34:54.739 DEBUG:teuthology.orchestra.run.smithi043:> ') 2023-08-09T13:34:55.003 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-09T13:34:55.004 DEBUG:teuthology.orchestra.run.smithi043:> (cd / && exec bash -c ' 2023-08-09T13:34:55.004 DEBUG:teuthology.orchestra.run.smithi043:> set -e 2023-08-09T13:34:55.004 DEBUG:teuthology.orchestra.run.smithi043:> sudo ip link set brx.1 up 2023-08-09T13:34:55.005 DEBUG:teuthology.orchestra.run.smithi043:> sudo ip link set dev brx.1 master ceph-brx 2023-08-09T13:34:55.005 DEBUG:teuthology.orchestra.run.smithi043:> ') 2023-08-09T13:34:55.168 INFO:tasks.cephfs.fuse_mount:Client client.2 config is {} 2023-08-09T13:34:55.169 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-08-09T13:34:55.169 DEBUG:teuthology.orchestra.run.smithi043:> mkdir -p -v /home/ubuntu/cephtest/mnt.2 2023-08-09T13:34:55.222 INFO:teuthology.orchestra.run.smithi043.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.2' 2023-08-09T13:34:55.222 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-08-09T13:34:55.223 DEBUG:teuthology.orchestra.run.smithi043:> chmod 0000 /home/ubuntu/cephtest/mnt.2 2023-08-09T13:34:55.279 DEBUG:teuthology.orchestra.run.smithi043:> sudo modprobe fuse 2023-08-09T13:34:55.379 INFO:teuthology.orchestra.run:Running command with timeout 30 2023-08-09T13:34:55.380 DEBUG:teuthology.orchestra.run.smithi043:> sudo mount -t fusectl /sys/fs/fuse/connections /sys/fs/fuse/connections 2023-08-09T13:34:55.485 INFO:teuthology.orchestra.run.smithi043.stderr:mount: /sys/fs/fuse/connections: /sys/fs/fuse/connections already mounted or mount point busy. 2023-08-09T13:34:55.488 DEBUG:teuthology.orchestra.run:got remote process result: 32 2023-08-09T13:34:55.489 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-09T13:34:55.489 DEBUG:teuthology.orchestra.run.smithi043:> ls /sys/fs/fuse/connections 2023-08-09T13:34:55.545 INFO:teuthology.orchestra.run.smithi043.stdout:40 2023-08-09T13:34:55.546 INFO:tasks.cephfs.fuse_mount:Pre-mount connections: [40] 2023-08-09T13:34:55.546 DEBUG:teuthology.orchestra.run.smithi043:> (cd /home/ubuntu/cephtest && exec sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-fuse -f --admin-socket '/var/run/ceph/$cluster-$name.$pid.asok' /home/ubuntu/cephtest/mnt.2 --id 2) 2023-08-09T13:34:55.552 DEBUG:teuthology.orchestra.run.smithi043:> sudo modprobe fuse 2023-08-09T13:34:55.637 INFO:teuthology.orchestra.run:Running command with timeout 30 2023-08-09T13:34:55.637 DEBUG:teuthology.orchestra.run.smithi043:> sudo mount -t fusectl /sys/fs/fuse/connections /sys/fs/fuse/connections 2023-08-09T13:34:55.714 INFO:teuthology.orchestra.run.smithi043.stderr:mount: /sys/fs/fuse/connections: /sys/fs/fuse/connections already mounted or mount point busy. 2023-08-09T13:34:55.717 DEBUG:teuthology.orchestra.run:got remote process result: 32 2023-08-09T13:34:55.718 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-09T13:34:55.718 DEBUG:teuthology.orchestra.run.smithi043:> ls /sys/fs/fuse/connections 2023-08-09T13:34:55.760 INFO:tasks.cephfs.fuse_mount.ceph-fuse.2.smithi043.stderr:2023-08-09T13:34:55.759+0000 7f31f6a4f480 -1 init, newargv = 0x55fea2ead4b0 newargc=15 2023-08-09T13:34:55.761 INFO:tasks.cephfs.fuse_mount.ceph-fuse.2.smithi043.stderr:ceph-fuse[98750]: starting ceph client 2023-08-09T13:34:55.764 INFO:teuthology.orchestra.run.smithi043.stdout:40 2023-08-09T13:34:55.772 INFO:tasks.cephfs.fuse_mount.ceph-fuse.2.smithi043.stderr:ceph-fuse[98750]: starting fuse 2023-08-09T13:34:56.866 DEBUG:teuthology.orchestra.run.smithi043:> sudo modprobe fuse 2023-08-09T13:34:56.983 INFO:teuthology.orchestra.run:Running command with timeout 30 2023-08-09T13:34:56.983 DEBUG:teuthology.orchestra.run.smithi043:> sudo mount -t fusectl /sys/fs/fuse/connections /sys/fs/fuse/connections 2023-08-09T13:34:57.085 INFO:teuthology.orchestra.run.smithi043.stderr:mount: /sys/fs/fuse/connections: /sys/fs/fuse/connections already mounted or mount point busy. 2023-08-09T13:34:57.088 DEBUG:teuthology.orchestra.run:got remote process result: 32 2023-08-09T13:34:57.089 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-09T13:34:57.089 DEBUG:teuthology.orchestra.run.smithi043:> ls /sys/fs/fuse/connections 2023-08-09T13:34:57.146 INFO:teuthology.orchestra.run.smithi043.stdout:40 2023-08-09T13:34:57.146 INFO:teuthology.orchestra.run.smithi043.stdout:41 2023-08-09T13:34:57.147 INFO:tasks.cephfs.fuse_mount:Post-mount connections: [40, 41] 2023-08-09T13:34:57.147 DEBUG:teuthology.orchestra.run.smithi043:> sudo stdin-killer -- python3 -c ' 2023-08-09T13:34:57.147 DEBUG:teuthology.orchestra.run.smithi043:> import glob 2023-08-09T13:34:57.147 DEBUG:teuthology.orchestra.run.smithi043:> import re 2023-08-09T13:34:57.148 DEBUG:teuthology.orchestra.run.smithi043:> import os 2023-08-09T13:34:57.148 DEBUG:teuthology.orchestra.run.smithi043:> import subprocess 2023-08-09T13:34:57.148 DEBUG:teuthology.orchestra.run.smithi043:> 2023-08-09T13:34:57.148 DEBUG:teuthology.orchestra.run.smithi043:> def _find_admin_socket(client_name): 2023-08-09T13:34:57.149 DEBUG:teuthology.orchestra.run.smithi043:> asok_path = "/var/run/ceph/ceph-client.2.*.asok" 2023-08-09T13:34:57.149 DEBUG:teuthology.orchestra.run.smithi043:> files = glob.glob(asok_path) 2023-08-09T13:34:57.149 DEBUG:teuthology.orchestra.run.smithi043:> mountpoint = "/home/ubuntu/cephtest/mnt.2" 2023-08-09T13:34:57.149 DEBUG:teuthology.orchestra.run.smithi043:> 2023-08-09T13:34:57.150 DEBUG:teuthology.orchestra.run.smithi043:> # Given a non-glob path, it better be there 2023-08-09T13:34:57.150 DEBUG:teuthology.orchestra.run.smithi043:> if "*" not in asok_path: 2023-08-09T13:34:57.150 DEBUG:teuthology.orchestra.run.smithi043:> assert(len(files) == 1) 2023-08-09T13:34:57.150 DEBUG:teuthology.orchestra.run.smithi043:> return files[0] 2023-08-09T13:34:57.150 DEBUG:teuthology.orchestra.run.smithi043:> 2023-08-09T13:34:57.151 DEBUG:teuthology.orchestra.run.smithi043:> for f in files: 2023-08-09T13:34:57.151 DEBUG:teuthology.orchestra.run.smithi043:> pid = re.match(".*\.(\d+)\.asok$", f).group(1) 2023-08-09T13:34:57.151 DEBUG:teuthology.orchestra.run.smithi043:> if os.path.exists("/proc/{0}".format(pid)): 2023-08-09T13:34:57.151 DEBUG:teuthology.orchestra.run.smithi043:> with open("/proc/{0}/cmdline".format(pid), '"'"'r'"'"') as proc_f: 2023-08-09T13:34:57.152 DEBUG:teuthology.orchestra.run.smithi043:> contents = proc_f.read() 2023-08-09T13:34:57.152 DEBUG:teuthology.orchestra.run.smithi043:> if mountpoint in contents: 2023-08-09T13:34:57.152 DEBUG:teuthology.orchestra.run.smithi043:> return f 2023-08-09T13:34:57.152 DEBUG:teuthology.orchestra.run.smithi043:> raise RuntimeError("Client socket {0} not found".format(client_name)) 2023-08-09T13:34:57.152 DEBUG:teuthology.orchestra.run.smithi043:> 2023-08-09T13:34:57.153 DEBUG:teuthology.orchestra.run.smithi043:> print(_find_admin_socket("client.2")) 2023-08-09T13:34:57.153 DEBUG:teuthology.orchestra.run.smithi043:> ' 2023-08-09T13:34:57.346 INFO:teuthology.orchestra.run.smithi043.stdout:/var/run/ceph/ceph-client.2.98750.asok 2023-08-09T13:34:57.350 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:57 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2023-08-09T13:34:57.364 INFO:tasks.cephfs.fuse_mount:Found client admin socket at /var/run/ceph/ceph-client.2.98750.asok 2023-08-09T13:34:57.364 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-09T13:34:57.364 DEBUG:teuthology.orchestra.run.smithi043:> sudo ceph --admin-daemon /var/run/ceph/ceph-client.2.98750.asok status 2023-08-09T13:34:57.622 INFO:teuthology.orchestra.run.smithi043.stdout:{ 2023-08-09T13:34:57.622 INFO:teuthology.orchestra.run.smithi043.stdout: "metadata": { 2023-08-09T13:34:57.622 INFO:teuthology.orchestra.run.smithi043.stdout: "ceph_sha1": "668a86f232c9454b2ce5928d1432a29a72964d85", 2023-08-09T13:34:57.622 INFO:teuthology.orchestra.run.smithi043.stdout: "ceph_version": "ceph version 18.0.0-5389-g668a86f2 (668a86f232c9454b2ce5928d1432a29a72964d85) reef (dev)", 2023-08-09T13:34:57.623 INFO:teuthology.orchestra.run.smithi043.stdout: "entity_id": "2", 2023-08-09T13:34:57.623 INFO:teuthology.orchestra.run.smithi043.stdout: "hostname": "smithi043", 2023-08-09T13:34:57.623 INFO:teuthology.orchestra.run.smithi043.stdout: "mount_point": "/home/ubuntu/cephtest/mnt.2", 2023-08-09T13:34:57.623 INFO:teuthology.orchestra.run.smithi043.stdout: "pid": "98750", 2023-08-09T13:34:57.623 INFO:teuthology.orchestra.run.smithi043.stdout: "root": "/" 2023-08-09T13:34:57.623 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2023-08-09T13:34:57.623 INFO:teuthology.orchestra.run.smithi043.stdout: "dentry_count": 0, 2023-08-09T13:34:57.623 INFO:teuthology.orchestra.run.smithi043.stdout: "dentry_pinned_count": 0, 2023-08-09T13:34:57.624 INFO:teuthology.orchestra.run.smithi043.stdout: "id": 4887, 2023-08-09T13:34:57.624 INFO:teuthology.orchestra.run.smithi043.stdout: "inst": { 2023-08-09T13:34:57.624 INFO:teuthology.orchestra.run.smithi043.stdout: "name": { 2023-08-09T13:34:57.624 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "client", 2023-08-09T13:34:57.624 INFO:teuthology.orchestra.run.smithi043.stdout: "num": 4887 2023-08-09T13:34:57.624 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2023-08-09T13:34:57.624 INFO:teuthology.orchestra.run.smithi043.stdout: "addr": { 2023-08-09T13:34:57.625 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "v1", 2023-08-09T13:34:57.625 INFO:teuthology.orchestra.run.smithi043.stdout: "addr": "192.168.0.2:0", 2023-08-09T13:34:57.625 INFO:teuthology.orchestra.run.smithi043.stdout: "nonce": 4025077506 2023-08-09T13:34:57.625 INFO:teuthology.orchestra.run.smithi043.stdout: } 2023-08-09T13:34:57.625 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2023-08-09T13:34:57.625 INFO:teuthology.orchestra.run.smithi043.stdout: "addr": { 2023-08-09T13:34:57.625 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "v1", 2023-08-09T13:34:57.625 INFO:teuthology.orchestra.run.smithi043.stdout: "addr": "192.168.0.2:0", 2023-08-09T13:34:57.626 INFO:teuthology.orchestra.run.smithi043.stdout: "nonce": 4025077506 2023-08-09T13:34:57.626 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2023-08-09T13:34:57.626 INFO:teuthology.orchestra.run.smithi043.stdout: "inst_str": "client.4887 192.168.0.2:0/4025077506", 2023-08-09T13:34:57.626 INFO:teuthology.orchestra.run.smithi043.stdout: "addr_str": "192.168.0.2:0/4025077506", 2023-08-09T13:34:57.626 INFO:teuthology.orchestra.run.smithi043.stdout: "inode_count": 1, 2023-08-09T13:34:57.626 INFO:teuthology.orchestra.run.smithi043.stdout: "mds_epoch": 9, 2023-08-09T13:34:57.626 INFO:teuthology.orchestra.run.smithi043.stdout: "osd_epoch": 26, 2023-08-09T13:34:57.626 INFO:teuthology.orchestra.run.smithi043.stdout: "osd_epoch_barrier": 0, 2023-08-09T13:34:57.627 INFO:teuthology.orchestra.run.smithi043.stdout: "blocklisted": false, 2023-08-09T13:34:57.627 INFO:teuthology.orchestra.run.smithi043.stdout: "fs_name": "cephfs" 2023-08-09T13:34:57.627 INFO:teuthology.orchestra.run.smithi043.stdout:} 2023-08-09T13:34:57.637 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2023-08-09T13:34:57.861 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:57.859+0000 7f34cfe71640 1 Processor -- start 2023-08-09T13:34:57.861 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:57.859+0000 7f34cfe71640 1 -- start start 2023-08-09T13:34:57.863 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:57.861+0000 7f34cfe71640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f34c814a5c0 0x7f34c8160f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:34:57.863 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:57.861+0000 7f34cfe71640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f34c80c3130 0x7f34c80cbc70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:34:57.863 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:57.861+0000 7f34cfe71640 1 -- --> v1:172.21.15.203:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f34c80c38a0 con 0x7f34c80cc1b0 2023-08-09T13:34:57.863 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:57.861+0000 7f34cfe71640 1 -- --> v2:172.21.15.43:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f34c80c4420 con 0x7f34c814a5c0 2023-08-09T13:34:57.863 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:57.861+0000 7f34cfe71640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f34c80cc640 con 0x7f34c80c3130 2023-08-09T13:34:57.864 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:57.861+0000 7f34cdc0d640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f34c814a5c0 0x7f34c8160f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:34:57.864 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:57.862+0000 7f34cdc0d640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f34c814a5c0 0x7f34c8160f20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:58668/0 (socket says 172.21.15.43:58668) 2023-08-09T13:34:57.864 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:57.862+0000 7f34cdc0d640 1 -- 172.21.15.43:0/4061773739 learned_addr learned my addr 172.21.15.43:0/4061773739 (peer_addr_for_me v2:172.21.15.43:0/0) 2023-08-09T13:34:57.865 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:57.862+0000 7f34cd40c640 1 --2- 172.21.15.43:0/4061773739 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f34c80c3130 0x7f34c80cbc70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:34:57.865 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:57.862+0000 7f34cdc0d640 1 -- 172.21.15.43:0/4061773739 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f34c80c3130 msgr2=0x7f34c80cbc70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-09T13:34:57.865 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:57.862+0000 7f34cdc0d640 1 --2- 172.21.15.43:0/4061773739 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f34c80c3130 0x7f34c80cbc70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:34:57.865 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:57.862+0000 7f34cdc0d640 1 -- 172.21.15.43:0/4061773739 >> v1:172.21.15.203:6789/0 conn(0x7f34c80cc1b0 legacy=0x7f34c8164510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:34:57.865 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:57.862+0000 7f34cdc0d640 1 -- 172.21.15.43:0/4061773739 --> v2:172.21.15.43:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f34c80c3b80 con 0x7f34c814a5c0 2023-08-09T13:34:57.865 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:57.863+0000 7f34cdc0d640 1 --2- 172.21.15.43:0/4061773739 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f34c814a5c0 0x7f34c8160f20 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7f34c4009870 tx=0x7f34c402b8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=147bba41bc13f3be server_cookie=0 in_seq=0 out_seq=0 2023-08-09T13:34:57.866 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:57.864+0000 7f34ccc0b640 1 -- 172.21.15.43:0/4061773739 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f34c4031640 con 0x7f34c814a5c0 2023-08-09T13:34:57.866 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:57.864+0000 7f34ccc0b640 1 -- 172.21.15.43:0/4061773739 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f34c40317a0 con 0x7f34c814a5c0 2023-08-09T13:34:57.867 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:57.864+0000 7f34ccc0b640 1 -- 172.21.15.43:0/4061773739 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f34c4031b30 con 0x7f34c814a5c0 2023-08-09T13:34:57.867 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:57.864+0000 7f34cfe71640 1 -- 172.21.15.43:0/4061773739 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f34c814a5c0 msgr2=0x7f34c8160f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:34:57.867 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:57.864+0000 7f34cfe71640 1 --2- 172.21.15.43:0/4061773739 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f34c814a5c0 0x7f34c8160f20 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7f34c4009870 tx=0x7f34c402b8f0 comp rx=0 tx=0).stop 2023-08-09T13:34:57.867 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:57.865+0000 7f34cfe71640 1 -- 172.21.15.43:0/4061773739 shutdown_connections 2023-08-09T13:34:57.867 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:57.865+0000 7f34cfe71640 1 --2- 172.21.15.43:0/4061773739 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f34c80c3130 0x7f34c80cbc70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:34:57.867 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:57.865+0000 7f34cfe71640 1 --2- 172.21.15.43:0/4061773739 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f34c814a5c0 0x7f34c8160f20 unknown :-1 s=CLOSED pgs=252 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:34:57.867 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:57.865+0000 7f34cfe71640 1 -- 172.21.15.43:0/4061773739 >> 172.21.15.43:0/4061773739 conn(0x7f34c80ce0a0 msgr2=0x7f34c80ce480 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-09T13:34:57.868 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:57.865+0000 7f34cfe71640 1 -- 172.21.15.43:0/4061773739 shutdown_connections 2023-08-09T13:34:57.868 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:57.865+0000 7f34cfe71640 1 -- 172.21.15.43:0/4061773739 wait complete. 2023-08-09T13:34:57.868 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:57.866+0000 7f34cfe71640 1 Processor -- start 2023-08-09T13:34:57.868 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:57.867+0000 7f34cfe71640 1 -- start start 2023-08-09T13:34:57.869 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:57.868+0000 7f34cfe71640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f34c80c3130 0x7f34c81f5bc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:34:57.869 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:57.868+0000 7f34cfe71640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f34c80cc1b0 0x7f34c80b1a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:34:57.869 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:57.868+0000 7f34cfe71640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f34c814a5c0 0x7f34c806f9a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:34:57.869 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:57.868+0000 7f34cfe71640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f34c8162840 con 0x7f34c80c3130 2023-08-09T13:34:57.869 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:57.868+0000 7f34cfe71640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f34c8161cc0 con 0x7f34c80cc1b0 2023-08-09T13:34:57.870 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:57.868+0000 7f34cfe71640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f34c8161fa0 con 0x7f34c814a5c0 2023-08-09T13:34:57.871 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:57.868+0000 7f34cdc0d640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f34c80c3130 0x7f34c81f5bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:34:57.871 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:57.868+0000 7f34cdc0d640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f34c80c3130 0x7f34c81f5bc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:58672/0 (socket says 172.21.15.43:58672) 2023-08-09T13:34:57.871 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:57.868+0000 7f34cdc0d640 1 -- 172.21.15.43:0/3689373713 learned_addr learned my addr 172.21.15.43:0/3689373713 (peer_addr_for_me v2:172.21.15.43:0/0) 2023-08-09T13:34:57.871 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:57.868+0000 7f34ce40e640 1 --2- 172.21.15.43:0/3689373713 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f34c814a5c0 0x7f34c806f9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:34:57.871 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:57.868+0000 7f34cd40c640 1 --2- 172.21.15.43:0/3689373713 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f34c80cc1b0 0x7f34c80b1a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:34:57.871 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:57.869+0000 7f34cdc0d640 1 -- 172.21.15.43:0/3689373713 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f34c814a5c0 msgr2=0x7f34c806f9a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:34:57.872 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:57.869+0000 7f34cdc0d640 1 --2- 172.21.15.43:0/3689373713 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f34c814a5c0 0x7f34c806f9a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:34:57.872 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:57.869+0000 7f34cdc0d640 1 -- 172.21.15.43:0/3689373713 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f34c80cc1b0 msgr2=0x7f34c80b1a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:34:57.872 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:57.869+0000 7f34cdc0d640 1 --2- 172.21.15.43:0/3689373713 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f34c80cc1b0 0x7f34c80b1a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:34:57.872 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:57.869+0000 7f34cdc0d640 1 -- 172.21.15.43:0/3689373713 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f34c40094d0 con 0x7f34c80c3130 2023-08-09T13:34:57.872 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:57.869+0000 7f34cdc0d640 1 --2- 172.21.15.43:0/3689373713 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f34c80c3130 0x7f34c81f5bc0 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7f34c4009870 tx=0x7f34c40046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-09T13:34:57.872 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:57.869+0000 7f34be7fc640 1 -- 172.21.15.43:0/3689373713 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f34c403ac60 con 0x7f34c80c3130 2023-08-09T13:34:57.872 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:57.869+0000 7f34be7fc640 1 -- 172.21.15.43:0/3689373713 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f34c403ade0 con 0x7f34c80c3130 2023-08-09T13:34:57.873 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:57.869+0000 7f34be7fc640 1 -- 172.21.15.43:0/3689373713 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f34c4044630 con 0x7f34c80c3130 2023-08-09T13:34:57.873 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:57.869+0000 7f34cfe71640 1 -- 172.21.15.43:0/3689373713 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f34c8162280 con 0x7f34c80c3130 2023-08-09T13:34:57.873 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:57.870+0000 7f34cfe71640 1 -- 172.21.15.43:0/3689373713 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f34c81f6100 con 0x7f34c80c3130 2023-08-09T13:34:57.873 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:57.871+0000 7f34cfe71640 1 -- 172.21.15.43:0/3689373713 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f34c8162ce0 con 0x7f34c80c3130 2023-08-09T13:34:57.873 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:57.872+0000 7f34be7fc640 1 -- 172.21.15.43:0/3689373713 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 5) v1 ==== 88341+0+0 (secure 0 0 0) 0x7f34c402f050 con 0x7f34c80c3130 2023-08-09T13:34:57.874 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:57.873+0000 7f34be7fc640 1 --2- 172.21.15.43:0/3689373713 >> [v2:172.21.15.203:6832/830097474,v1:172.21.15.203:6833/830097474] conn(0x7f34ac06a080 0x7f34ac06c540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:34:57.875 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:57.873+0000 7f34be7fc640 1 -- 172.21.15.43:0/3689373713 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 6871+0+0 (secure 0 0 0) 0x7f34c40aa790 con 0x7f34c80c3130 2023-08-09T13:34:57.875 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:57.874+0000 7f34cd40c640 1 --2- 172.21.15.43:0/3689373713 >> [v2:172.21.15.203:6832/830097474,v1:172.21.15.203:6833/830097474] conn(0x7f34ac06a080 0x7f34ac06c540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:34:57.876 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:57.875+0000 7f34cd40c640 1 --2- 172.21.15.43:0/3689373713 >> [v2:172.21.15.203:6832/830097474,v1:172.21.15.203:6833/830097474] conn(0x7f34ac06a080 0x7f34ac06c540 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7f34b8006fd0 tx=0x7f34b8008040 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-09T13:34:57.879 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:57.878+0000 7f34be7fc640 1 -- 172.21.15.43:0/3689373713 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178066 (secure 0 0 0) 0x7f34c8162ce0 con 0x7f34c80c3130 2023-08-09T13:34:58.117 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:58.115+0000 7f34cfe71640 1 -- 172.21.15.43:0/3689373713 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7f34c8162ce0 con 0x7f34c80c3130 2023-08-09T13:34:58.117 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:58.116+0000 7f34be7fc640 1 -- 172.21.15.43:0/3689373713 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v9) v1 ==== 53+0+88 (secure 0 0 0) 0x7f34c8162ce0 con 0x7f34c80c3130 2023-08-09T13:34:58.118 INFO:teuthology.orchestra.run.smithi043.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data cephfs_data_ec ] 2023-08-09T13:34:58.120 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:58.118+0000 7f34cfe71640 1 -- 172.21.15.43:0/3689373713 >> [v2:172.21.15.203:6832/830097474,v1:172.21.15.203:6833/830097474] conn(0x7f34ac06a080 msgr2=0x7f34ac06c540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:34:58.120 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:58.118+0000 7f34cfe71640 1 --2- 172.21.15.43:0/3689373713 >> [v2:172.21.15.203:6832/830097474,v1:172.21.15.203:6833/830097474] conn(0x7f34ac06a080 0x7f34ac06c540 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7f34b8006fd0 tx=0x7f34b8008040 comp rx=0 tx=0).stop 2023-08-09T13:34:58.121 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:58.118+0000 7f34cfe71640 1 -- 172.21.15.43:0/3689373713 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f34c80c3130 msgr2=0x7f34c81f5bc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:34:58.121 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:58.118+0000 7f34cfe71640 1 --2- 172.21.15.43:0/3689373713 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f34c80c3130 0x7f34c81f5bc0 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7f34c4009870 tx=0x7f34c40046c0 comp rx=0 tx=0).stop 2023-08-09T13:34:58.121 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:58.118+0000 7f34cfe71640 1 -- 172.21.15.43:0/3689373713 shutdown_connections 2023-08-09T13:34:58.121 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:58.118+0000 7f34cfe71640 1 --2- 172.21.15.43:0/3689373713 >> [v2:172.21.15.203:6832/830097474,v1:172.21.15.203:6833/830097474] conn(0x7f34ac06a080 0x7f34ac06c540 unknown :-1 s=CLOSED pgs=144 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:34:58.121 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:58.118+0000 7f34cfe71640 1 --2- 172.21.15.43:0/3689373713 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f34c814a5c0 0x7f34c806f9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:34:58.121 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:58.118+0000 7f34cfe71640 1 --2- 172.21.15.43:0/3689373713 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f34c80cc1b0 0x7f34c80b1a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:34:58.121 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:58.118+0000 7f34cfe71640 1 --2- 172.21.15.43:0/3689373713 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f34c80c3130 0x7f34c81f5bc0 unknown :-1 s=CLOSED pgs=253 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:34:58.121 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:58.119+0000 7f34cfe71640 1 -- 172.21.15.43:0/3689373713 >> 172.21.15.43:0/3689373713 conn(0x7f34c80ce0a0 msgr2=0x7f34c80c86b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-09T13:34:58.122 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:58.119+0000 7f34cfe71640 1 -- 172.21.15.43:0/3689373713 shutdown_connections 2023-08-09T13:34:58.122 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:34:58.119+0000 7f34cfe71640 1 -- 172.21.15.43:0/3689373713 wait complete. 2023-08-09T13:34:58.140 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2023-08-09T13:34:58.140 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2023-08-09T13:34:58.140 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi203.front.sepia.ceph.com 2023-08-09T13:34:58.140 INFO:tasks.cephfs.mount:self.client.name = client.1 2023-08-09T13:34:58.141 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.1 2023-08-09T13:34:58.141 INFO:tasks.cephfs.mount:self.cephfs_name = None 2023-08-09T13:34:58.141 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2023-08-09T13:34:58.141 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2023-08-09T13:34:58.141 INFO:tasks.cephfs.mount:Setting the 'None' netns for '/home/ubuntu/cephtest/mnt.1' 2023-08-09T13:34:58.142 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-09T13:34:58.142 DEBUG:teuthology.orchestra.run.smithi203:> ip addr 2023-08-09T13:34:58.205 INFO:teuthology.orchestra.run.smithi203.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2023-08-09T13:34:58.206 INFO:teuthology.orchestra.run.smithi203.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2023-08-09T13:34:58.206 INFO:teuthology.orchestra.run.smithi203.stdout: inet 127.0.0.1/8 scope host lo 2023-08-09T13:34:58.206 INFO:teuthology.orchestra.run.smithi203.stdout: valid_lft forever preferred_lft forever 2023-08-09T13:34:58.206 INFO:teuthology.orchestra.run.smithi203.stdout: inet6 ::1/128 scope host 2023-08-09T13:34:58.206 INFO:teuthology.orchestra.run.smithi203.stdout: valid_lft forever preferred_lft forever 2023-08-09T13:34:58.207 INFO:teuthology.orchestra.run.smithi203.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-08-09T13:34:58.207 INFO:teuthology.orchestra.run.smithi203.stdout: link/ether 0c:c4:7a:d9:19:44 brd ff:ff:ff:ff:ff:ff 2023-08-09T13:34:58.207 INFO:teuthology.orchestra.run.smithi203.stdout: altname enp7s0f0 2023-08-09T13:34:58.207 INFO:teuthology.orchestra.run.smithi203.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-08-09T13:34:58.207 INFO:teuthology.orchestra.run.smithi203.stdout: link/ether 0c:c4:7a:d9:19:45 brd ff:ff:ff:ff:ff:ff 2023-08-09T13:34:58.207 INFO:teuthology.orchestra.run.smithi203.stdout: altname enp7s0f1 2023-08-09T13:34:58.208 INFO:teuthology.orchestra.run.smithi203.stdout:4: enp3s0f0: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-08-09T13:34:58.208 INFO:teuthology.orchestra.run.smithi203.stdout: link/ether 0c:c4:7a:8f:4a:22 brd ff:ff:ff:ff:ff:ff 2023-08-09T13:34:58.208 INFO:teuthology.orchestra.run.smithi203.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2023-08-09T13:34:58.208 INFO:teuthology.orchestra.run.smithi203.stdout: link/ether 0c:c4:7a:8f:4a:23 brd ff:ff:ff:ff:ff:ff 2023-08-09T13:34:58.208 INFO:teuthology.orchestra.run.smithi203.stdout: inet 172.21.15.203/20 brd 172.21.15.255 scope global dynamic noprefixroute enp3s0f1 2023-08-09T13:34:58.208 INFO:teuthology.orchestra.run.smithi203.stdout: valid_lft 42233sec preferred_lft 42233sec 2023-08-09T13:34:58.208 INFO:teuthology.orchestra.run.smithi203.stdout: inet6 fe80::ec4:7aff:fe8f:4a23/64 scope link noprefixroute 2023-08-09T13:34:58.208 INFO:teuthology.orchestra.run.smithi203.stdout: valid_lft forever preferred_lft forever 2023-08-09T13:34:58.209 INFO:tasks.cephfs.mount:Setuping the 'ceph-brx' with 192.168.255.254/16 2023-08-09T13:34:58.210 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-09T13:34:58.210 DEBUG:teuthology.orchestra.run.smithi203:> (cd / && exec bash -c ' 2023-08-09T13:34:58.210 DEBUG:teuthology.orchestra.run.smithi203:> set -e 2023-08-09T13:34:58.210 DEBUG:teuthology.orchestra.run.smithi203:> sudo ip link add name ceph-brx type bridge 2023-08-09T13:34:58.211 DEBUG:teuthology.orchestra.run.smithi203:> sudo ip addr flush dev ceph-brx 2023-08-09T13:34:58.211 DEBUG:teuthology.orchestra.run.smithi203:> sudo ip link set ceph-brx up 2023-08-09T13:34:58.211 DEBUG:teuthology.orchestra.run.smithi203:> sudo ip addr add 192.168.255.254/16 brd 192.168.255.255 dev ceph-brx 2023-08-09T13:34:58.211 DEBUG:teuthology.orchestra.run.smithi203:> ') 2023-08-09T13:34:58.714 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-09T13:34:58.714 DEBUG:teuthology.orchestra.run.smithi203:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2023-08-09T13:34:58.796 INFO:teuthology.orchestra.run.smithi203.stdout:1 2023-08-09T13:34:58.799 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-09T13:34:58.800 DEBUG:teuthology.orchestra.run.smithi203:> route 2023-08-09T13:34:58.826 DEBUG:teuthology.orchestra.run.smithi043:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-08-09T13:34:58.836 DEBUG:teuthology.orchestra.run.smithi203:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-08-09T13:34:58.850 INFO:teuthology.orchestra.run.smithi203.stdout:Kernel IP routing table 2023-08-09T13:34:58.850 INFO:teuthology.orchestra.run.smithi203.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2023-08-09T13:34:58.851 INFO:teuthology.orchestra.run.smithi203.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2023-08-09T13:34:58.851 INFO:teuthology.orchestra.run.smithi203.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2023-08-09T13:34:58.851 INFO:teuthology.orchestra.run.smithi203.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2023-08-09T13:34:58.851 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-09T13:34:58.852 DEBUG:teuthology.orchestra.run.smithi203:> (cd / && exec bash -c ' 2023-08-09T13:34:58.852 DEBUG:teuthology.orchestra.run.smithi203:> set -e 2023-08-09T13:34:58.852 DEBUG:teuthology.orchestra.run.smithi203:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2023-08-09T13:34:58.852 DEBUG:teuthology.orchestra.run.smithi203:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2023-08-09T13:34:58.853 DEBUG:teuthology.orchestra.run.smithi203:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2023-08-09T13:34:58.853 DEBUG:teuthology.orchestra.run.smithi203:> ') 2023-08-09T13:34:59.751 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-09T13:34:59.752 DEBUG:teuthology.orchestra.run.smithi203:> ip netns list 2023-08-09T13:34:59.805 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-09T13:34:59.806 DEBUG:teuthology.orchestra.run.smithi203:> ip netns list-id 2023-08-09T13:34:59.861 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-09T13:34:59.861 DEBUG:teuthology.orchestra.run.smithi203:> (cd / && exec bash -c ' 2023-08-09T13:34:59.861 DEBUG:teuthology.orchestra.run.smithi203:> set -e 2023-08-09T13:34:59.862 DEBUG:teuthology.orchestra.run.smithi203:> sudo ip netns add ceph-ns--home-ubuntu-cephtest-mnt.1 2023-08-09T13:34:59.862 DEBUG:teuthology.orchestra.run.smithi203:> sudo ip netns set ceph-ns--home-ubuntu-cephtest-mnt.1 0 2023-08-09T13:34:59.862 DEBUG:teuthology.orchestra.run.smithi203:> ') 2023-08-09T13:35:00.013 INFO:tasks.cephfs.mount:Setuping the netns 'ceph-ns--home-ubuntu-cephtest-mnt.1' with 192.168.0.1/16 2023-08-09T13:35:00.013 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-09T13:35:00.014 DEBUG:teuthology.orchestra.run.smithi203:> (cd / && exec bash -c ' 2023-08-09T13:35:00.014 DEBUG:teuthology.orchestra.run.smithi203:> set -e 2023-08-09T13:35:00.014 DEBUG:teuthology.orchestra.run.smithi203:> sudo ip link add veth0 netns ceph-ns--home-ubuntu-cephtest-mnt.1 type veth peer name brx.0 2023-08-09T13:35:00.015 DEBUG:teuthology.orchestra.run.smithi203:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.1 ip addr add 192.168.0.1/16 brd 192.168.255.255 dev veth0 2023-08-09T13:35:00.015 DEBUG:teuthology.orchestra.run.smithi203:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.1 ip link set veth0 up 2023-08-09T13:35:00.015 DEBUG:teuthology.orchestra.run.smithi203:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.1 ip link set lo up 2023-08-09T13:35:00.015 DEBUG:teuthology.orchestra.run.smithi203:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.1 ip route add default via 192.168.255.254 2023-08-09T13:35:00.016 DEBUG:teuthology.orchestra.run.smithi203:> ') 2023-08-09T13:35:00.329 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-09T13:35:00.329 DEBUG:teuthology.orchestra.run.smithi203:> (cd / && exec bash -c ' 2023-08-09T13:35:00.329 DEBUG:teuthology.orchestra.run.smithi203:> set -e 2023-08-09T13:35:00.330 DEBUG:teuthology.orchestra.run.smithi203:> sudo ip link set brx.0 up 2023-08-09T13:35:00.330 DEBUG:teuthology.orchestra.run.smithi203:> sudo ip link set dev brx.0 master ceph-brx 2023-08-09T13:35:00.330 DEBUG:teuthology.orchestra.run.smithi203:> ') 2023-08-09T13:35:00.498 INFO:tasks.cephfs.fuse_mount:Client client.1 config is {} 2023-08-09T13:35:00.499 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-08-09T13:35:00.499 DEBUG:teuthology.orchestra.run.smithi203:> mkdir -p -v /home/ubuntu/cephtest/mnt.1 2023-08-09T13:35:00.546 INFO:teuthology.orchestra.run.smithi203.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.1' 2023-08-09T13:35:00.547 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-08-09T13:35:00.547 DEBUG:teuthology.orchestra.run.smithi203:> chmod 0000 /home/ubuntu/cephtest/mnt.1 2023-08-09T13:35:00.645 DEBUG:teuthology.orchestra.run.smithi203:> sudo modprobe fuse 2023-08-09T13:35:00.732 INFO:teuthology.orchestra.run:Running command with timeout 30 2023-08-09T13:35:00.733 DEBUG:teuthology.orchestra.run.smithi203:> sudo mount -t fusectl /sys/fs/fuse/connections /sys/fs/fuse/connections 2023-08-09T13:35:00.818 INFO:teuthology.orchestra.run.smithi203.stderr:mount: /sys/fs/fuse/connections: /sys/fs/fuse/connections already mounted or mount point busy. 2023-08-09T13:35:00.821 DEBUG:teuthology.orchestra.run:got remote process result: 32 2023-08-09T13:35:00.822 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-09T13:35:00.822 DEBUG:teuthology.orchestra.run.smithi203:> ls /sys/fs/fuse/connections 2023-08-09T13:35:00.876 INFO:tasks.cephfs.fuse_mount:Pre-mount connections: [] 2023-08-09T13:35:00.876 DEBUG:teuthology.orchestra.run.smithi203:> (cd /home/ubuntu/cephtest && exec sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-fuse -f --admin-socket '/var/run/ceph/$cluster-$name.$pid.asok' /home/ubuntu/cephtest/mnt.1 --id 1) 2023-08-09T13:35:00.881 DEBUG:teuthology.orchestra.run.smithi203:> sudo modprobe fuse 2023-08-09T13:35:00.957 INFO:teuthology.orchestra.run:Running command with timeout 30 2023-08-09T13:35:00.958 DEBUG:teuthology.orchestra.run.smithi203:> sudo mount -t fusectl /sys/fs/fuse/connections /sys/fs/fuse/connections 2023-08-09T13:35:01.035 INFO:teuthology.orchestra.run.smithi203.stderr:mount: /sys/fs/fuse/connections: /sys/fs/fuse/connections already mounted or mount point busy. 2023-08-09T13:35:01.037 DEBUG:teuthology.orchestra.run:got remote process result: 32 2023-08-09T13:35:01.038 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-09T13:35:01.039 DEBUG:teuthology.orchestra.run.smithi203:> ls /sys/fs/fuse/connections 2023-08-09T13:35:01.072 INFO:tasks.cephfs.fuse_mount.ceph-fuse.1.smithi203.stderr:2023-08-09T13:35:01.069+0000 7f86fb645480 -1 init, newargv = 0x555b326c9640 newargc=15 2023-08-09T13:35:01.073 INFO:tasks.cephfs.fuse_mount.ceph-fuse.1.smithi203.stderr:ceph-fuse[94798]: starting ceph client 2023-08-09T13:35:01.081 INFO:tasks.cephfs.fuse_mount.ceph-fuse.1.smithi203.stderr:ceph-fuse[94798]: starting fuse 2023-08-09T13:35:02.181 DEBUG:teuthology.orchestra.run.smithi203:> sudo modprobe fuse 2023-08-09T13:35:02.286 INFO:teuthology.orchestra.run:Running command with timeout 30 2023-08-09T13:35:02.287 DEBUG:teuthology.orchestra.run.smithi203:> sudo mount -t fusectl /sys/fs/fuse/connections /sys/fs/fuse/connections 2023-08-09T13:35:02.385 INFO:teuthology.orchestra.run.smithi203.stderr:mount: /sys/fs/fuse/connections: /sys/fs/fuse/connections already mounted or mount point busy. 2023-08-09T13:35:02.390 DEBUG:teuthology.orchestra.run:got remote process result: 32 2023-08-09T13:35:02.391 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-09T13:35:02.392 DEBUG:teuthology.orchestra.run.smithi203:> ls /sys/fs/fuse/connections 2023-08-09T13:35:02.451 INFO:teuthology.orchestra.run.smithi203.stdout:40 2023-08-09T13:35:02.452 INFO:tasks.cephfs.fuse_mount:Post-mount connections: [40] 2023-08-09T13:35:02.452 DEBUG:teuthology.orchestra.run.smithi203:> sudo stdin-killer -- python3 -c ' 2023-08-09T13:35:02.453 DEBUG:teuthology.orchestra.run.smithi203:> import glob 2023-08-09T13:35:02.453 DEBUG:teuthology.orchestra.run.smithi203:> import re 2023-08-09T13:35:02.453 DEBUG:teuthology.orchestra.run.smithi203:> import os 2023-08-09T13:35:02.453 DEBUG:teuthology.orchestra.run.smithi203:> import subprocess 2023-08-09T13:35:02.454 DEBUG:teuthology.orchestra.run.smithi203:> 2023-08-09T13:35:02.454 DEBUG:teuthology.orchestra.run.smithi203:> def _find_admin_socket(client_name): 2023-08-09T13:35:02.454 DEBUG:teuthology.orchestra.run.smithi203:> asok_path = "/var/run/ceph/ceph-client.1.*.asok" 2023-08-09T13:35:02.454 DEBUG:teuthology.orchestra.run.smithi203:> files = glob.glob(asok_path) 2023-08-09T13:35:02.455 DEBUG:teuthology.orchestra.run.smithi203:> mountpoint = "/home/ubuntu/cephtest/mnt.1" 2023-08-09T13:35:02.455 DEBUG:teuthology.orchestra.run.smithi203:> 2023-08-09T13:35:02.455 DEBUG:teuthology.orchestra.run.smithi203:> # Given a non-glob path, it better be there 2023-08-09T13:35:02.456 DEBUG:teuthology.orchestra.run.smithi203:> if "*" not in asok_path: 2023-08-09T13:35:02.456 DEBUG:teuthology.orchestra.run.smithi203:> assert(len(files) == 1) 2023-08-09T13:35:02.456 DEBUG:teuthology.orchestra.run.smithi203:> return files[0] 2023-08-09T13:35:02.456 DEBUG:teuthology.orchestra.run.smithi203:> 2023-08-09T13:35:02.457 DEBUG:teuthology.orchestra.run.smithi203:> for f in files: 2023-08-09T13:35:02.457 DEBUG:teuthology.orchestra.run.smithi203:> pid = re.match(".*\.(\d+)\.asok$", f).group(1) 2023-08-09T13:35:02.457 DEBUG:teuthology.orchestra.run.smithi203:> if os.path.exists("/proc/{0}".format(pid)): 2023-08-09T13:35:02.457 DEBUG:teuthology.orchestra.run.smithi203:> with open("/proc/{0}/cmdline".format(pid), '"'"'r'"'"') as proc_f: 2023-08-09T13:35:02.457 DEBUG:teuthology.orchestra.run.smithi203:> contents = proc_f.read() 2023-08-09T13:35:02.458 DEBUG:teuthology.orchestra.run.smithi203:> if mountpoint in contents: 2023-08-09T13:35:02.458 DEBUG:teuthology.orchestra.run.smithi203:> return f 2023-08-09T13:35:02.458 DEBUG:teuthology.orchestra.run.smithi203:> raise RuntimeError("Client socket {0} not found".format(client_name)) 2023-08-09T13:35:02.458 DEBUG:teuthology.orchestra.run.smithi203:> 2023-08-09T13:35:02.459 DEBUG:teuthology.orchestra.run.smithi203:> print(_find_admin_socket("client.1")) 2023-08-09T13:35:02.459 DEBUG:teuthology.orchestra.run.smithi203:> ' 2023-08-09T13:35:02.633 INFO:teuthology.orchestra.run.smithi203.stdout:/var/run/ceph/ceph-client.1.94798.asok 2023-08-09T13:35:02.638 INFO:teuthology.orchestra.run.smithi203.stderr:2023-08-09T13:35:02 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2023-08-09T13:35:02.650 INFO:tasks.cephfs.fuse_mount:Found client admin socket at /var/run/ceph/ceph-client.1.94798.asok 2023-08-09T13:35:02.651 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-09T13:35:02.651 DEBUG:teuthology.orchestra.run.smithi203:> sudo ceph --admin-daemon /var/run/ceph/ceph-client.1.94798.asok status 2023-08-09T13:35:02.893 INFO:teuthology.orchestra.run.smithi203.stdout:{ 2023-08-09T13:35:02.894 INFO:teuthology.orchestra.run.smithi203.stdout: "metadata": { 2023-08-09T13:35:02.894 INFO:teuthology.orchestra.run.smithi203.stdout: "ceph_sha1": "668a86f232c9454b2ce5928d1432a29a72964d85", 2023-08-09T13:35:02.894 INFO:teuthology.orchestra.run.smithi203.stdout: "ceph_version": "ceph version 18.0.0-5389-g668a86f2 (668a86f232c9454b2ce5928d1432a29a72964d85) reef (dev)", 2023-08-09T13:35:02.894 INFO:teuthology.orchestra.run.smithi203.stdout: "entity_id": "1", 2023-08-09T13:35:02.894 INFO:teuthology.orchestra.run.smithi203.stdout: "hostname": "smithi203", 2023-08-09T13:35:02.894 INFO:teuthology.orchestra.run.smithi203.stdout: "mount_point": "/home/ubuntu/cephtest/mnt.1", 2023-08-09T13:35:02.894 INFO:teuthology.orchestra.run.smithi203.stdout: "pid": "94798", 2023-08-09T13:35:02.895 INFO:teuthology.orchestra.run.smithi203.stdout: "root": "/" 2023-08-09T13:35:02.895 INFO:teuthology.orchestra.run.smithi203.stdout: }, 2023-08-09T13:35:02.895 INFO:teuthology.orchestra.run.smithi203.stdout: "dentry_count": 0, 2023-08-09T13:35:02.895 INFO:teuthology.orchestra.run.smithi203.stdout: "dentry_pinned_count": 0, 2023-08-09T13:35:02.895 INFO:teuthology.orchestra.run.smithi203.stdout: "id": 4765, 2023-08-09T13:35:02.895 INFO:teuthology.orchestra.run.smithi203.stdout: "inst": { 2023-08-09T13:35:02.895 INFO:teuthology.orchestra.run.smithi203.stdout: "name": { 2023-08-09T13:35:02.896 INFO:teuthology.orchestra.run.smithi203.stdout: "type": "client", 2023-08-09T13:35:02.896 INFO:teuthology.orchestra.run.smithi203.stdout: "num": 4765 2023-08-09T13:35:02.896 INFO:teuthology.orchestra.run.smithi203.stdout: }, 2023-08-09T13:35:02.896 INFO:teuthology.orchestra.run.smithi203.stdout: "addr": { 2023-08-09T13:35:02.896 INFO:teuthology.orchestra.run.smithi203.stdout: "type": "v1", 2023-08-09T13:35:02.896 INFO:teuthology.orchestra.run.smithi203.stdout: "addr": "192.168.0.1:0", 2023-08-09T13:35:02.896 INFO:teuthology.orchestra.run.smithi203.stdout: "nonce": 642099177 2023-08-09T13:35:02.897 INFO:teuthology.orchestra.run.smithi203.stdout: } 2023-08-09T13:35:02.897 INFO:teuthology.orchestra.run.smithi203.stdout: }, 2023-08-09T13:35:02.897 INFO:teuthology.orchestra.run.smithi203.stdout: "addr": { 2023-08-09T13:35:02.897 INFO:teuthology.orchestra.run.smithi203.stdout: "type": "v1", 2023-08-09T13:35:02.897 INFO:teuthology.orchestra.run.smithi203.stdout: "addr": "192.168.0.1:0", 2023-08-09T13:35:02.897 INFO:teuthology.orchestra.run.smithi203.stdout: "nonce": 642099177 2023-08-09T13:35:02.897 INFO:teuthology.orchestra.run.smithi203.stdout: }, 2023-08-09T13:35:02.897 INFO:teuthology.orchestra.run.smithi203.stdout: "inst_str": "client.4765 192.168.0.1:0/642099177", 2023-08-09T13:35:02.898 INFO:teuthology.orchestra.run.smithi203.stdout: "addr_str": "192.168.0.1:0/642099177", 2023-08-09T13:35:02.898 INFO:teuthology.orchestra.run.smithi203.stdout: "inode_count": 1, 2023-08-09T13:35:02.898 INFO:teuthology.orchestra.run.smithi203.stdout: "mds_epoch": 9, 2023-08-09T13:35:02.898 INFO:teuthology.orchestra.run.smithi203.stdout: "osd_epoch": 26, 2023-08-09T13:35:02.898 INFO:teuthology.orchestra.run.smithi203.stdout: "osd_epoch_barrier": 0, 2023-08-09T13:35:02.898 INFO:teuthology.orchestra.run.smithi203.stdout: "blocklisted": false, 2023-08-09T13:35:02.898 INFO:teuthology.orchestra.run.smithi203.stdout: "fs_name": "cephfs" 2023-08-09T13:35:02.898 INFO:teuthology.orchestra.run.smithi203.stdout:} 2023-08-09T13:35:02.908 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2023-08-09T13:35:03.146 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:03.144+0000 7f077c283640 1 Processor -- start 2023-08-09T13:35:03.146 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:03.144+0000 7f077c283640 1 -- start start 2023-08-09T13:35:03.148 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:03.146+0000 7f077c283640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f077406cfe0 0x7f07740c8b50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:35:03.148 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:03.146+0000 7f077c283640 1 -- --> v1:172.21.15.43:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f07740c9090 con 0x7f07740c86f0 2023-08-09T13:35:03.148 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:03.146+0000 7f077c283640 1 -- --> v1:172.21.15.203:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f077406c440 con 0x7f0774143c20 2023-08-09T13:35:03.148 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:03.146+0000 7f077c283640 1 -- --> v2:172.21.15.43:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0774144000 con 0x7f077406cfe0 2023-08-09T13:35:03.149 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:03.146+0000 7f077981e640 1 --1- >> v1:172.21.15.43:6789/0 conn(0x7f07740c86f0 0x7f077406c8d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.43:6789/0 says I am v1:172.21.15.43:47678/0 (socket says 172.21.15.43:47678) 2023-08-09T13:35:03.149 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:03.147+0000 7f077981e640 1 -- 172.21.15.43:0/3059713705 learned_addr learned my addr 172.21.15.43:0/3059713705 (peer_addr_for_me v1:172.21.15.43:0/0) 2023-08-09T13:35:03.149 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:03.147+0000 7f077a820640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f077406cfe0 0x7f07740c8b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:35:03.149 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:03.147+0000 7f077a820640 1 -- 172.21.15.43:0/3059713705 >> v1:172.21.15.203:6789/0 conn(0x7f0774143c20 legacy=0x7f077406f540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-09T13:35:03.149 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:03.147+0000 7f077a820640 1 -- 172.21.15.43:0/3059713705 >> v1:172.21.15.43:6789/0 conn(0x7f07740c86f0 legacy=0x7f077406c8d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:35:03.149 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:03.147+0000 7f077a820640 1 -- 172.21.15.43:0/3059713705 --> v2:172.21.15.43:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f077406fc50 con 0x7f077406cfe0 2023-08-09T13:35:03.150 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:03.148+0000 7f077a820640 1 --2- 172.21.15.43:0/3059713705 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f077406cfe0 0x7f07740c8b50 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7f076c009d90 tx=0x7f076c02bd90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7ba14f33c82a4817 server_cookie=0 in_seq=0 out_seq=0 2023-08-09T13:35:03.151 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:03.148+0000 7f077901d640 1 -- 172.21.15.43:0/3059713705 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f076c02cb40 con 0x7f077406cfe0 2023-08-09T13:35:03.151 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:03.148+0000 7f077901d640 1 -- 172.21.15.43:0/3059713705 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f076c02cca0 con 0x7f077406cfe0 2023-08-09T13:35:03.151 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:03.149+0000 7f077901d640 1 -- 172.21.15.43:0/3059713705 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f076c034660 con 0x7f077406cfe0 2023-08-09T13:35:03.151 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:03.149+0000 7f077c283640 1 -- 172.21.15.43:0/3059713705 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f077406cfe0 msgr2=0x7f07740c8b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:35:03.151 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:03.149+0000 7f077c283640 1 --2- 172.21.15.43:0/3059713705 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f077406cfe0 0x7f07740c8b50 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7f076c009d90 tx=0x7f076c02bd90 comp rx=0 tx=0).stop 2023-08-09T13:35:03.151 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:03.149+0000 7f077c283640 1 -- 172.21.15.43:0/3059713705 shutdown_connections 2023-08-09T13:35:03.152 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:03.149+0000 7f077c283640 1 --2- 172.21.15.43:0/3059713705 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f077406cfe0 0x7f07740c8b50 unknown :-1 s=CLOSED pgs=257 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:35:03.152 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:03.149+0000 7f077c283640 1 -- 172.21.15.43:0/3059713705 >> 172.21.15.43:0/3059713705 conn(0x7f07740ce0a0 msgr2=0x7f07740ce480 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-09T13:35:03.152 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:03.149+0000 7f077c283640 1 -- 172.21.15.43:0/3059713705 shutdown_connections 2023-08-09T13:35:03.152 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:03.149+0000 7f077c283640 1 -- 172.21.15.43:0/3059713705 wait complete. 2023-08-09T13:35:03.153 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:03.151+0000 7f077c283640 1 Processor -- start 2023-08-09T13:35:03.153 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:03.151+0000 7f077c283640 1 -- start start 2023-08-09T13:35:03.154 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:03.152+0000 7f077c283640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f077406cfe0 0x7f077415c9d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:35:03.154 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:03.152+0000 7f077c283640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f07740c86f0 0x7f077413d4f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:35:03.154 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:03.152+0000 7f077c283640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f0774143c20 0x7f07740c7960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:35:03.154 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:03.152+0000 7f077c283640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f07740b2b60 con 0x7f07740c86f0 2023-08-09T13:35:03.154 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:03.152+0000 7f077c283640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f07740b1ee0 con 0x7f0774143c20 2023-08-09T13:35:03.155 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:03.152+0000 7f077c283640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f07740b21c0 con 0x7f077406cfe0 2023-08-09T13:35:03.155 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:03.152+0000 7f077981e640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f07740c86f0 0x7f077413d4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:35:03.155 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:03.153+0000 7f077981e640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f07740c86f0 0x7f077413d4f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:58694/0 (socket says 172.21.15.43:58694) 2023-08-09T13:35:03.155 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:03.153+0000 7f077981e640 1 -- 172.21.15.43:0/2963255152 learned_addr learned my addr 172.21.15.43:0/2963255152 (peer_addr_for_me v2:172.21.15.43:0/0) 2023-08-09T13:35:03.156 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:03.153+0000 7f077a01f640 1 --2- 172.21.15.43:0/2963255152 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f077406cfe0 0x7f077415c9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:35:03.156 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:03.153+0000 7f077a820640 1 --2- 172.21.15.43:0/2963255152 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f0774143c20 0x7f07740c7960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:35:03.156 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:03.153+0000 7f077981e640 1 -- 172.21.15.43:0/2963255152 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f077406cfe0 msgr2=0x7f077415c9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:35:03.157 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:03.153+0000 7f077981e640 1 --2- 172.21.15.43:0/2963255152 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f077406cfe0 0x7f077415c9d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:35:03.157 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:03.153+0000 7f077981e640 1 -- 172.21.15.43:0/2963255152 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f0774143c20 msgr2=0x7f07740c7960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:35:03.157 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:03.153+0000 7f077981e640 1 --2- 172.21.15.43:0/2963255152 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f0774143c20 0x7f07740c7960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:35:03.157 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:03.153+0000 7f077981e640 1 -- 172.21.15.43:0/2963255152 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f076c009a70 con 0x7f07740c86f0 2023-08-09T13:35:03.157 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:03.153+0000 7f077a820640 1 --2- 172.21.15.43:0/2963255152 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f0774143c20 0x7f07740c7960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-08-09T13:35:03.157 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:03.153+0000 7f077981e640 1 --2- 172.21.15.43:0/2963255152 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f07740c86f0 0x7f077413d4f0 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7f076400c4a0 tx=0x7f076400c960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-09T13:35:03.158 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:03.154+0000 7f076affd640 1 -- 172.21.15.43:0/2963255152 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0764005ce0 con 0x7f07740c86f0 2023-08-09T13:35:03.158 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:03.154+0000 7f076affd640 1 -- 172.21.15.43:0/2963255152 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0764005e40 con 0x7f07740c86f0 2023-08-09T13:35:03.158 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:03.154+0000 7f076affd640 1 -- 172.21.15.43:0/2963255152 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0764007270 con 0x7f07740c86f0 2023-08-09T13:35:03.158 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:03.154+0000 7f077c283640 1 -- 172.21.15.43:0/2963255152 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f07740b24a0 con 0x7f07740c86f0 2023-08-09T13:35:03.158 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:03.154+0000 7f077c283640 1 -- 172.21.15.43:0/2963255152 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f07740c7ea0 con 0x7f07740c86f0 2023-08-09T13:35:03.158 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:03.155+0000 7f077c283640 1 -- 172.21.15.43:0/2963255152 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f07740b2ca0 con 0x7f07740c86f0 2023-08-09T13:35:03.159 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:03.156+0000 7f076affd640 1 -- 172.21.15.43:0/2963255152 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 5) v1 ==== 88341+0+0 (secure 0 0 0) 0x7f0764007490 con 0x7f07740c86f0 2023-08-09T13:35:03.159 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:03.157+0000 7f076affd640 1 --2- 172.21.15.43:0/2963255152 >> [v2:172.21.15.203:6832/830097474,v1:172.21.15.203:6833/830097474] conn(0x7f075806a050 0x7f075806c510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:35:03.160 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:03.158+0000 7f076affd640 1 -- 172.21.15.43:0/2963255152 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 6871+0+0 (secure 0 0 0) 0x7f0764087e30 con 0x7f07740c86f0 2023-08-09T13:35:03.160 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:03.158+0000 7f077a01f640 1 --2- 172.21.15.43:0/2963255152 >> [v2:172.21.15.203:6832/830097474,v1:172.21.15.203:6833/830097474] conn(0x7f075806a050 0x7f075806c510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:35:03.161 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:03.159+0000 7f077a01f640 1 --2- 172.21.15.43:0/2963255152 >> [v2:172.21.15.203:6832/830097474,v1:172.21.15.203:6833/830097474] conn(0x7f075806a050 0x7f075806c510 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7f07741e8bd0 tx=0x7f0770009040 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-09T13:35:03.163 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:03.162+0000 7f076affd640 1 -- 172.21.15.43:0/2963255152 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178066 (secure 0 0 0) 0x7f07740b2ca0 con 0x7f07740c86f0 2023-08-09T13:35:03.401 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:03.399+0000 7f077c283640 1 -- 172.21.15.43:0/2963255152 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7f07740b2ca0 con 0x7f07740c86f0 2023-08-09T13:35:03.402 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:03.400+0000 7f076affd640 1 -- 172.21.15.43:0/2963255152 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v9) v1 ==== 53+0+88 (secure 0 0 0) 0x7f07740b2ca0 con 0x7f07740c86f0 2023-08-09T13:35:03.402 INFO:teuthology.orchestra.run.smithi043.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data cephfs_data_ec ] 2023-08-09T13:35:03.404 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:03.403+0000 7f077c283640 1 -- 172.21.15.43:0/2963255152 >> [v2:172.21.15.203:6832/830097474,v1:172.21.15.203:6833/830097474] conn(0x7f075806a050 msgr2=0x7f075806c510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:35:03.405 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:03.403+0000 7f077c283640 1 --2- 172.21.15.43:0/2963255152 >> [v2:172.21.15.203:6832/830097474,v1:172.21.15.203:6833/830097474] conn(0x7f075806a050 0x7f075806c510 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7f07741e8bd0 tx=0x7f0770009040 comp rx=0 tx=0).stop 2023-08-09T13:35:03.405 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:03.403+0000 7f077c283640 1 -- 172.21.15.43:0/2963255152 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f07740c86f0 msgr2=0x7f077413d4f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:35:03.405 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:03.403+0000 7f077c283640 1 --2- 172.21.15.43:0/2963255152 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f07740c86f0 0x7f077413d4f0 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7f076400c4a0 tx=0x7f076400c960 comp rx=0 tx=0).stop 2023-08-09T13:35:03.405 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:03.403+0000 7f077c283640 1 -- 172.21.15.43:0/2963255152 shutdown_connections 2023-08-09T13:35:03.405 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:03.403+0000 7f077c283640 1 --2- 172.21.15.43:0/2963255152 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f0774143c20 0x7f07740c7960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:35:03.405 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:03.403+0000 7f077c283640 1 --2- 172.21.15.43:0/2963255152 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f07740c86f0 0x7f077413d4f0 unknown :-1 s=CLOSED pgs=258 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:35:03.406 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:03.403+0000 7f077c283640 1 --2- 172.21.15.43:0/2963255152 >> [v2:172.21.15.203:6832/830097474,v1:172.21.15.203:6833/830097474] conn(0x7f075806a050 0x7f075806c510 unknown :-1 s=CLOSED pgs=145 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:35:03.406 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:03.403+0000 7f077c283640 1 --2- 172.21.15.43:0/2963255152 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f077406cfe0 0x7f077415c9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:35:03.406 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:03.403+0000 7f077c283640 1 -- 172.21.15.43:0/2963255152 >> 172.21.15.43:0/2963255152 conn(0x7f07740ce0a0 msgr2=0x7f077406c8d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-09T13:35:03.406 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:03.403+0000 7f077c283640 1 -- 172.21.15.43:0/2963255152 shutdown_connections 2023-08-09T13:35:03.407 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:03.403+0000 7f077c283640 1 -- 172.21.15.43:0/2963255152 wait complete. 2023-08-09T13:35:03.423 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2023-08-09T13:35:03.423 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2023-08-09T13:35:03.423 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi203.front.sepia.ceph.com 2023-08-09T13:35:03.423 INFO:tasks.cephfs.mount:self.client.name = client.3 2023-08-09T13:35:03.424 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.3 2023-08-09T13:35:03.424 INFO:tasks.cephfs.mount:self.cephfs_name = None 2023-08-09T13:35:03.424 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2023-08-09T13:35:03.424 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2023-08-09T13:35:03.424 INFO:tasks.cephfs.mount:Setting the 'None' netns for '/home/ubuntu/cephtest/mnt.3' 2023-08-09T13:35:03.425 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-09T13:35:03.425 DEBUG:teuthology.orchestra.run.smithi203:> ip addr 2023-08-09T13:35:03.485 INFO:teuthology.orchestra.run.smithi203.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2023-08-09T13:35:03.485 INFO:teuthology.orchestra.run.smithi203.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2023-08-09T13:35:03.485 INFO:teuthology.orchestra.run.smithi203.stdout: inet 127.0.0.1/8 scope host lo 2023-08-09T13:35:03.485 INFO:teuthology.orchestra.run.smithi203.stdout: valid_lft forever preferred_lft forever 2023-08-09T13:35:03.486 INFO:teuthology.orchestra.run.smithi203.stdout: inet6 ::1/128 scope host 2023-08-09T13:35:03.486 INFO:teuthology.orchestra.run.smithi203.stdout: valid_lft forever preferred_lft forever 2023-08-09T13:35:03.486 INFO:teuthology.orchestra.run.smithi203.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-08-09T13:35:03.486 INFO:teuthology.orchestra.run.smithi203.stdout: link/ether 0c:c4:7a:d9:19:44 brd ff:ff:ff:ff:ff:ff 2023-08-09T13:35:03.486 INFO:teuthology.orchestra.run.smithi203.stdout: altname enp7s0f0 2023-08-09T13:35:03.486 INFO:teuthology.orchestra.run.smithi203.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-08-09T13:35:03.486 INFO:teuthology.orchestra.run.smithi203.stdout: link/ether 0c:c4:7a:d9:19:45 brd ff:ff:ff:ff:ff:ff 2023-08-09T13:35:03.487 INFO:teuthology.orchestra.run.smithi203.stdout: altname enp7s0f1 2023-08-09T13:35:03.487 INFO:teuthology.orchestra.run.smithi203.stdout:4: enp3s0f0: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-08-09T13:35:03.487 INFO:teuthology.orchestra.run.smithi203.stdout: link/ether 0c:c4:7a:8f:4a:22 brd ff:ff:ff:ff:ff:ff 2023-08-09T13:35:03.487 INFO:teuthology.orchestra.run.smithi203.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2023-08-09T13:35:03.487 INFO:teuthology.orchestra.run.smithi203.stdout: link/ether 0c:c4:7a:8f:4a:23 brd ff:ff:ff:ff:ff:ff 2023-08-09T13:35:03.487 INFO:teuthology.orchestra.run.smithi203.stdout: inet 172.21.15.203/20 brd 172.21.15.255 scope global dynamic noprefixroute enp3s0f1 2023-08-09T13:35:03.487 INFO:teuthology.orchestra.run.smithi203.stdout: valid_lft 42227sec preferred_lft 42227sec 2023-08-09T13:35:03.487 INFO:teuthology.orchestra.run.smithi203.stdout: inet6 fe80::ec4:7aff:fe8f:4a23/64 scope link noprefixroute 2023-08-09T13:35:03.488 INFO:teuthology.orchestra.run.smithi203.stdout: valid_lft forever preferred_lft forever 2023-08-09T13:35:03.488 INFO:teuthology.orchestra.run.smithi203.stdout:6: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2023-08-09T13:35:03.488 INFO:teuthology.orchestra.run.smithi203.stdout: link/ether 3a:3f:1d:76:ff:3b brd ff:ff:ff:ff:ff:ff 2023-08-09T13:35:03.488 INFO:teuthology.orchestra.run.smithi203.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2023-08-09T13:35:03.488 INFO:teuthology.orchestra.run.smithi203.stdout: valid_lft forever preferred_lft forever 2023-08-09T13:35:03.488 INFO:teuthology.orchestra.run.smithi203.stdout: inet6 fe80::e01b:a0ff:fe4c:e2ba/64 scope link 2023-08-09T13:35:03.488 INFO:teuthology.orchestra.run.smithi203.stdout: valid_lft forever preferred_lft forever 2023-08-09T13:35:03.489 INFO:teuthology.orchestra.run.smithi203.stdout:7: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2023-08-09T13:35:03.489 INFO:teuthology.orchestra.run.smithi203.stdout: link/ether 3a:3f:1d:76:ff:3b brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.1 2023-08-09T13:35:03.489 INFO:teuthology.orchestra.run.smithi203.stdout: inet6 fe80::383f:1dff:fe76:ff3b/64 scope link 2023-08-09T13:35:03.489 INFO:teuthology.orchestra.run.smithi203.stdout: valid_lft forever preferred_lft forever 2023-08-09T13:35:03.489 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-09T13:35:03.490 DEBUG:teuthology.orchestra.run.smithi203:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2023-08-09T13:35:03.580 INFO:teuthology.orchestra.run.smithi203.stdout:1 2023-08-09T13:35:03.584 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-09T13:35:03.584 DEBUG:teuthology.orchestra.run.smithi203:> route 2023-08-09T13:35:03.640 INFO:teuthology.orchestra.run.smithi203.stdout:Kernel IP routing table 2023-08-09T13:35:03.640 INFO:teuthology.orchestra.run.smithi203.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2023-08-09T13:35:03.641 INFO:teuthology.orchestra.run.smithi203.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2023-08-09T13:35:03.641 INFO:teuthology.orchestra.run.smithi203.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2023-08-09T13:35:03.641 INFO:teuthology.orchestra.run.smithi203.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2023-08-09T13:35:03.642 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-09T13:35:03.642 DEBUG:teuthology.orchestra.run.smithi203:> (cd / && exec bash -c ' 2023-08-09T13:35:03.642 DEBUG:teuthology.orchestra.run.smithi203:> set -e 2023-08-09T13:35:03.643 DEBUG:teuthology.orchestra.run.smithi203:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2023-08-09T13:35:03.643 DEBUG:teuthology.orchestra.run.smithi203:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2023-08-09T13:35:03.643 DEBUG:teuthology.orchestra.run.smithi203:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2023-08-09T13:35:03.643 DEBUG:teuthology.orchestra.run.smithi203:> ') 2023-08-09T13:35:03.835 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-09T13:35:03.836 DEBUG:teuthology.orchestra.run.smithi203:> ip netns list 2023-08-09T13:35:03.889 INFO:teuthology.orchestra.run.smithi203.stdout:ceph-ns--home-ubuntu-cephtest-mnt.1 (id: 0) 2023-08-09T13:35:03.893 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-09T13:35:03.894 DEBUG:teuthology.orchestra.run.smithi203:> ip netns list-id 2023-08-09T13:35:03.946 INFO:teuthology.orchestra.run.smithi203.stdout:nsid 0 (iproute2 netns name: ceph-ns--home-ubuntu-cephtest-mnt.1) 2023-08-09T13:35:03.947 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-09T13:35:03.948 DEBUG:teuthology.orchestra.run.smithi203:> (cd / && exec bash -c ' 2023-08-09T13:35:03.948 DEBUG:teuthology.orchestra.run.smithi203:> set -e 2023-08-09T13:35:03.948 DEBUG:teuthology.orchestra.run.smithi203:> sudo ip netns add ceph-ns--home-ubuntu-cephtest-mnt.3 2023-08-09T13:35:03.948 DEBUG:teuthology.orchestra.run.smithi203:> sudo ip netns set ceph-ns--home-ubuntu-cephtest-mnt.3 1 2023-08-09T13:35:03.949 DEBUG:teuthology.orchestra.run.smithi203:> ') 2023-08-09T13:35:04.094 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-09T13:35:04.094 DEBUG:teuthology.orchestra.run.smithi203:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.1 ip addr 2023-08-09T13:35:04.188 INFO:teuthology.orchestra.run.smithi203.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2023-08-09T13:35:04.189 INFO:teuthology.orchestra.run.smithi203.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2023-08-09T13:35:04.189 INFO:teuthology.orchestra.run.smithi203.stdout: inet 127.0.0.1/8 scope host lo 2023-08-09T13:35:04.189 INFO:teuthology.orchestra.run.smithi203.stdout: valid_lft forever preferred_lft forever 2023-08-09T13:35:04.189 INFO:teuthology.orchestra.run.smithi203.stdout: inet6 ::1/128 scope host 2023-08-09T13:35:04.189 INFO:teuthology.orchestra.run.smithi203.stdout: valid_lft forever preferred_lft forever 2023-08-09T13:35:04.189 INFO:teuthology.orchestra.run.smithi203.stdout:2: veth0@if7: mtu 1500 qdisc noqueue state UP group default qlen 1000 2023-08-09T13:35:04.189 INFO:teuthology.orchestra.run.smithi203.stdout: link/ether 42:b3:07:af:83:da brd ff:ff:ff:ff:ff:ff link-netnsid 0 2023-08-09T13:35:04.189 INFO:teuthology.orchestra.run.smithi203.stdout: inet 192.168.0.1/16 brd 192.168.255.255 scope global veth0 2023-08-09T13:35:04.190 INFO:teuthology.orchestra.run.smithi203.stdout: valid_lft forever preferred_lft forever 2023-08-09T13:35:04.190 INFO:teuthology.orchestra.run.smithi203.stdout: inet6 fe80::40b3:7ff:feaf:83da/64 scope link 2023-08-09T13:35:04.190 INFO:teuthology.orchestra.run.smithi203.stdout: valid_lft forever preferred_lft forever 2023-08-09T13:35:04.193 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-09T13:35:04.193 DEBUG:teuthology.orchestra.run.smithi203:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.1 ip addr 2023-08-09T13:35:04.293 INFO:teuthology.orchestra.run.smithi203.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2023-08-09T13:35:04.293 INFO:teuthology.orchestra.run.smithi203.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2023-08-09T13:35:04.293 INFO:teuthology.orchestra.run.smithi203.stdout: inet 127.0.0.1/8 scope host lo 2023-08-09T13:35:04.294 INFO:teuthology.orchestra.run.smithi203.stdout: valid_lft forever preferred_lft forever 2023-08-09T13:35:04.294 INFO:teuthology.orchestra.run.smithi203.stdout: inet6 ::1/128 scope host 2023-08-09T13:35:04.294 INFO:teuthology.orchestra.run.smithi203.stdout: valid_lft forever preferred_lft forever 2023-08-09T13:35:04.294 INFO:teuthology.orchestra.run.smithi203.stdout:2: veth0@if7: mtu 1500 qdisc noqueue state UP group default qlen 1000 2023-08-09T13:35:04.294 INFO:teuthology.orchestra.run.smithi203.stdout: link/ether 42:b3:07:af:83:da brd ff:ff:ff:ff:ff:ff link-netnsid 0 2023-08-09T13:35:04.294 INFO:teuthology.orchestra.run.smithi203.stdout: inet 192.168.0.1/16 brd 192.168.255.255 scope global veth0 2023-08-09T13:35:04.294 INFO:teuthology.orchestra.run.smithi203.stdout: valid_lft forever preferred_lft forever 2023-08-09T13:35:04.294 INFO:teuthology.orchestra.run.smithi203.stdout: inet6 fe80::40b3:7ff:feaf:83da/64 scope link 2023-08-09T13:35:04.295 INFO:teuthology.orchestra.run.smithi203.stdout: valid_lft forever preferred_lft forever 2023-08-09T13:35:04.297 INFO:tasks.cephfs.mount:Setuping the netns 'ceph-ns--home-ubuntu-cephtest-mnt.3' with 192.168.0.2/16 2023-08-09T13:35:04.298 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-09T13:35:04.298 DEBUG:teuthology.orchestra.run.smithi203:> (cd / && exec bash -c ' 2023-08-09T13:35:04.298 DEBUG:teuthology.orchestra.run.smithi203:> set -e 2023-08-09T13:35:04.299 DEBUG:teuthology.orchestra.run.smithi203:> sudo ip link add veth0 netns ceph-ns--home-ubuntu-cephtest-mnt.3 type veth peer name brx.1 2023-08-09T13:35:04.299 DEBUG:teuthology.orchestra.run.smithi203:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.3 ip addr add 192.168.0.2/16 brd 192.168.255.255 dev veth0 2023-08-09T13:35:04.299 DEBUG:teuthology.orchestra.run.smithi203:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.3 ip link set veth0 up 2023-08-09T13:35:04.299 DEBUG:teuthology.orchestra.run.smithi203:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.3 ip link set lo up 2023-08-09T13:35:04.299 DEBUG:teuthology.orchestra.run.smithi203:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.3 ip route add default via 192.168.255.254 2023-08-09T13:35:04.300 DEBUG:teuthology.orchestra.run.smithi203:> ') 2023-08-09T13:35:04.570 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-09T13:35:04.570 DEBUG:teuthology.orchestra.run.smithi203:> (cd / && exec bash -c ' 2023-08-09T13:35:04.570 DEBUG:teuthology.orchestra.run.smithi203:> set -e 2023-08-09T13:35:04.570 DEBUG:teuthology.orchestra.run.smithi203:> sudo ip link set brx.1 up 2023-08-09T13:35:04.571 DEBUG:teuthology.orchestra.run.smithi203:> sudo ip link set dev brx.1 master ceph-brx 2023-08-09T13:35:04.571 DEBUG:teuthology.orchestra.run.smithi203:> ') 2023-08-09T13:35:04.743 INFO:tasks.cephfs.fuse_mount:Client client.3 config is {} 2023-08-09T13:35:04.743 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-08-09T13:35:04.744 DEBUG:teuthology.orchestra.run.smithi203:> mkdir -p -v /home/ubuntu/cephtest/mnt.3 2023-08-09T13:35:04.795 INFO:teuthology.orchestra.run.smithi203.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.3' 2023-08-09T13:35:04.795 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-08-09T13:35:04.795 DEBUG:teuthology.orchestra.run.smithi203:> chmod 0000 /home/ubuntu/cephtest/mnt.3 2023-08-09T13:35:04.845 DEBUG:teuthology.orchestra.run.smithi203:> sudo modprobe fuse 2023-08-09T13:35:04.944 INFO:teuthology.orchestra.run:Running command with timeout 30 2023-08-09T13:35:04.944 DEBUG:teuthology.orchestra.run.smithi203:> sudo mount -t fusectl /sys/fs/fuse/connections /sys/fs/fuse/connections 2023-08-09T13:35:05.036 INFO:teuthology.orchestra.run.smithi203.stderr:mount: /sys/fs/fuse/connections: /sys/fs/fuse/connections already mounted or mount point busy. 2023-08-09T13:35:05.039 DEBUG:teuthology.orchestra.run:got remote process result: 32 2023-08-09T13:35:05.040 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-09T13:35:05.041 DEBUG:teuthology.orchestra.run.smithi203:> ls /sys/fs/fuse/connections 2023-08-09T13:35:05.089 INFO:teuthology.orchestra.run.smithi203.stdout:40 2023-08-09T13:35:05.090 INFO:tasks.cephfs.fuse_mount:Pre-mount connections: [40] 2023-08-09T13:35:05.090 DEBUG:teuthology.orchestra.run.smithi203:> (cd /home/ubuntu/cephtest && exec sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-fuse -f --admin-socket '/var/run/ceph/$cluster-$name.$pid.asok' /home/ubuntu/cephtest/mnt.3 --id 3) 2023-08-09T13:35:05.098 DEBUG:teuthology.orchestra.run.smithi203:> sudo modprobe fuse 2023-08-09T13:35:05.178 INFO:teuthology.orchestra.run:Running command with timeout 30 2023-08-09T13:35:05.179 DEBUG:teuthology.orchestra.run.smithi203:> sudo mount -t fusectl /sys/fs/fuse/connections /sys/fs/fuse/connections 2023-08-09T13:35:05.248 INFO:teuthology.orchestra.run.smithi203.stderr:mount: /sys/fs/fuse/connections: /sys/fs/fuse/connections already mounted or mount point busy. 2023-08-09T13:35:05.251 DEBUG:teuthology.orchestra.run:got remote process result: 32 2023-08-09T13:35:05.252 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-09T13:35:05.252 DEBUG:teuthology.orchestra.run.smithi203:> ls /sys/fs/fuse/connections 2023-08-09T13:35:05.286 INFO:tasks.cephfs.fuse_mount.ceph-fuse.3.smithi203.stderr:2023-08-09T13:35:05.284+0000 7ffabd9d6480 -1 init, newargv = 0x563fac1c3250 newargc=15 2023-08-09T13:35:05.287 INFO:tasks.cephfs.fuse_mount.ceph-fuse.3.smithi203.stderr:ceph-fuse[95318]: starting ceph client 2023-08-09T13:35:05.294 INFO:tasks.cephfs.fuse_mount.ceph-fuse.3.smithi203.stderr:ceph-fuse[95318]: starting fuse 2023-08-09T13:35:05.295 INFO:teuthology.orchestra.run.smithi203.stdout:40 2023-08-09T13:35:05.296 INFO:teuthology.orchestra.run.smithi203.stdout:41 2023-08-09T13:35:05.296 INFO:tasks.cephfs.fuse_mount:Post-mount connections: [40, 41] 2023-08-09T13:35:05.297 DEBUG:teuthology.orchestra.run.smithi203:> sudo stdin-killer -- python3 -c ' 2023-08-09T13:35:05.297 DEBUG:teuthology.orchestra.run.smithi203:> import glob 2023-08-09T13:35:05.298 DEBUG:teuthology.orchestra.run.smithi203:> import re 2023-08-09T13:35:05.298 DEBUG:teuthology.orchestra.run.smithi203:> import os 2023-08-09T13:35:05.299 DEBUG:teuthology.orchestra.run.smithi203:> import subprocess 2023-08-09T13:35:05.299 DEBUG:teuthology.orchestra.run.smithi203:> 2023-08-09T13:35:05.299 DEBUG:teuthology.orchestra.run.smithi203:> def _find_admin_socket(client_name): 2023-08-09T13:35:05.299 DEBUG:teuthology.orchestra.run.smithi203:> asok_path = "/var/run/ceph/ceph-client.3.*.asok" 2023-08-09T13:35:05.300 DEBUG:teuthology.orchestra.run.smithi203:> files = glob.glob(asok_path) 2023-08-09T13:35:05.300 DEBUG:teuthology.orchestra.run.smithi203:> mountpoint = "/home/ubuntu/cephtest/mnt.3" 2023-08-09T13:35:05.300 DEBUG:teuthology.orchestra.run.smithi203:> 2023-08-09T13:35:05.300 DEBUG:teuthology.orchestra.run.smithi203:> # Given a non-glob path, it better be there 2023-08-09T13:35:05.301 DEBUG:teuthology.orchestra.run.smithi203:> if "*" not in asok_path: 2023-08-09T13:35:05.301 DEBUG:teuthology.orchestra.run.smithi203:> assert(len(files) == 1) 2023-08-09T13:35:05.301 DEBUG:teuthology.orchestra.run.smithi203:> return files[0] 2023-08-09T13:35:05.301 DEBUG:teuthology.orchestra.run.smithi203:> 2023-08-09T13:35:05.302 DEBUG:teuthology.orchestra.run.smithi203:> for f in files: 2023-08-09T13:35:05.302 DEBUG:teuthology.orchestra.run.smithi203:> pid = re.match(".*\.(\d+)\.asok$", f).group(1) 2023-08-09T13:35:05.302 DEBUG:teuthology.orchestra.run.smithi203:> if os.path.exists("/proc/{0}".format(pid)): 2023-08-09T13:35:05.302 DEBUG:teuthology.orchestra.run.smithi203:> with open("/proc/{0}/cmdline".format(pid), '"'"'r'"'"') as proc_f: 2023-08-09T13:35:05.302 DEBUG:teuthology.orchestra.run.smithi203:> contents = proc_f.read() 2023-08-09T13:35:05.303 DEBUG:teuthology.orchestra.run.smithi203:> if mountpoint in contents: 2023-08-09T13:35:05.303 DEBUG:teuthology.orchestra.run.smithi203:> return f 2023-08-09T13:35:05.303 DEBUG:teuthology.orchestra.run.smithi203:> raise RuntimeError("Client socket {0} not found".format(client_name)) 2023-08-09T13:35:05.303 DEBUG:teuthology.orchestra.run.smithi203:> 2023-08-09T13:35:05.304 DEBUG:teuthology.orchestra.run.smithi203:> print(_find_admin_socket("client.3")) 2023-08-09T13:35:05.304 DEBUG:teuthology.orchestra.run.smithi203:> ' 2023-08-09T13:35:05.502 INFO:teuthology.orchestra.run.smithi203.stdout:/var/run/ceph/ceph-client.3.95318.asok 2023-08-09T13:35:05.506 INFO:teuthology.orchestra.run.smithi203.stderr:2023-08-09T13:35:05 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2023-08-09T13:35:05.520 INFO:tasks.cephfs.fuse_mount:Found client admin socket at /var/run/ceph/ceph-client.3.95318.asok 2023-08-09T13:35:05.520 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-09T13:35:05.520 DEBUG:teuthology.orchestra.run.smithi203:> sudo ceph --admin-daemon /var/run/ceph/ceph-client.3.95318.asok status 2023-08-09T13:35:05.761 INFO:teuthology.orchestra.run.smithi203.stdout:{ 2023-08-09T13:35:05.761 INFO:teuthology.orchestra.run.smithi203.stdout: "metadata": { 2023-08-09T13:35:05.761 INFO:teuthology.orchestra.run.smithi203.stdout: "ceph_sha1": "668a86f232c9454b2ce5928d1432a29a72964d85", 2023-08-09T13:35:05.761 INFO:teuthology.orchestra.run.smithi203.stdout: "ceph_version": "ceph version 18.0.0-5389-g668a86f2 (668a86f232c9454b2ce5928d1432a29a72964d85) reef (dev)", 2023-08-09T13:35:05.761 INFO:teuthology.orchestra.run.smithi203.stdout: "entity_id": "3", 2023-08-09T13:35:05.762 INFO:teuthology.orchestra.run.smithi203.stdout: "hostname": "smithi203", 2023-08-09T13:35:05.762 INFO:teuthology.orchestra.run.smithi203.stdout: "mount_point": "/home/ubuntu/cephtest/mnt.3", 2023-08-09T13:35:05.762 INFO:teuthology.orchestra.run.smithi203.stdout: "pid": "95318", 2023-08-09T13:35:05.762 INFO:teuthology.orchestra.run.smithi203.stdout: "root": "/" 2023-08-09T13:35:05.762 INFO:teuthology.orchestra.run.smithi203.stdout: }, 2023-08-09T13:35:05.762 INFO:teuthology.orchestra.run.smithi203.stdout: "dentry_count": 0, 2023-08-09T13:35:05.762 INFO:teuthology.orchestra.run.smithi203.stdout: "dentry_pinned_count": 0, 2023-08-09T13:35:05.762 INFO:teuthology.orchestra.run.smithi203.stdout: "id": 4625, 2023-08-09T13:35:05.763 INFO:teuthology.orchestra.run.smithi203.stdout: "inst": { 2023-08-09T13:35:05.763 INFO:teuthology.orchestra.run.smithi203.stdout: "name": { 2023-08-09T13:35:05.763 INFO:teuthology.orchestra.run.smithi203.stdout: "type": "client", 2023-08-09T13:35:05.763 INFO:teuthology.orchestra.run.smithi203.stdout: "num": 4625 2023-08-09T13:35:05.763 INFO:teuthology.orchestra.run.smithi203.stdout: }, 2023-08-09T13:35:05.763 INFO:teuthology.orchestra.run.smithi203.stdout: "addr": { 2023-08-09T13:35:05.763 INFO:teuthology.orchestra.run.smithi203.stdout: "type": "v1", 2023-08-09T13:35:05.763 INFO:teuthology.orchestra.run.smithi203.stdout: "addr": "192.168.0.2:0", 2023-08-09T13:35:05.764 INFO:teuthology.orchestra.run.smithi203.stdout: "nonce": 1510570118 2023-08-09T13:35:05.764 INFO:teuthology.orchestra.run.smithi203.stdout: } 2023-08-09T13:35:05.764 INFO:teuthology.orchestra.run.smithi203.stdout: }, 2023-08-09T13:35:05.764 INFO:teuthology.orchestra.run.smithi203.stdout: "addr": { 2023-08-09T13:35:05.764 INFO:teuthology.orchestra.run.smithi203.stdout: "type": "v1", 2023-08-09T13:35:05.764 INFO:teuthology.orchestra.run.smithi203.stdout: "addr": "192.168.0.2:0", 2023-08-09T13:35:05.764 INFO:teuthology.orchestra.run.smithi203.stdout: "nonce": 1510570118 2023-08-09T13:35:05.765 INFO:teuthology.orchestra.run.smithi203.stdout: }, 2023-08-09T13:35:05.765 INFO:teuthology.orchestra.run.smithi203.stdout: "inst_str": "client.4625 192.168.0.2:0/1510570118", 2023-08-09T13:35:05.765 INFO:teuthology.orchestra.run.smithi203.stdout: "addr_str": "192.168.0.2:0/1510570118", 2023-08-09T13:35:05.765 INFO:teuthology.orchestra.run.smithi203.stdout: "inode_count": 1, 2023-08-09T13:35:05.765 INFO:teuthology.orchestra.run.smithi203.stdout: "mds_epoch": 9, 2023-08-09T13:35:05.765 INFO:teuthology.orchestra.run.smithi203.stdout: "osd_epoch": 26, 2023-08-09T13:35:05.766 INFO:teuthology.orchestra.run.smithi203.stdout: "osd_epoch_barrier": 0, 2023-08-09T13:35:05.766 INFO:teuthology.orchestra.run.smithi203.stdout: "blocklisted": false, 2023-08-09T13:35:05.766 INFO:teuthology.orchestra.run.smithi203.stdout: "fs_name": "cephfs" 2023-08-09T13:35:05.766 INFO:teuthology.orchestra.run.smithi203.stdout:} 2023-08-09T13:35:05.775 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-09T13:35:05.776 DEBUG:teuthology.orchestra.run.smithi043:> stat --file-system '--printf=%T 2023-08-09T13:35:05.776 DEBUG:teuthology.orchestra.run.smithi043:> ' -- /home/ubuntu/cephtest/mnt.0 2023-08-09T13:35:05.841 INFO:teuthology.orchestra.run.smithi043.stdout:fuseblk 2023-08-09T13:35:05.841 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.0 2023-08-09T13:35:05.842 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-09T13:35:05.842 DEBUG:teuthology.orchestra.run.smithi043:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2023-08-09T13:35:05.948 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-09T13:35:05.949 DEBUG:teuthology.orchestra.run.smithi043:> stat --file-system '--printf=%T 2023-08-09T13:35:05.950 DEBUG:teuthology.orchestra.run.smithi043:> ' -- /home/ubuntu/cephtest/mnt.2 2023-08-09T13:35:06.007 INFO:teuthology.orchestra.run.smithi043.stdout:fuseblk 2023-08-09T13:35:06.008 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.2 2023-08-09T13:35:06.008 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-09T13:35:06.009 DEBUG:teuthology.orchestra.run.smithi043:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.2 2023-08-09T13:35:06.115 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-09T13:35:06.115 DEBUG:teuthology.orchestra.run.smithi203:> stat --file-system '--printf=%T 2023-08-09T13:35:06.115 DEBUG:teuthology.orchestra.run.smithi203:> ' -- /home/ubuntu/cephtest/mnt.1 2023-08-09T13:35:06.175 INFO:teuthology.orchestra.run.smithi203.stdout:fuseblk 2023-08-09T13:35:06.176 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.1 2023-08-09T13:35:06.176 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-09T13:35:06.176 DEBUG:teuthology.orchestra.run.smithi203:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.1 2023-08-09T13:35:06.274 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-09T13:35:06.274 DEBUG:teuthology.orchestra.run.smithi203:> stat --file-system '--printf=%T 2023-08-09T13:35:06.275 DEBUG:teuthology.orchestra.run.smithi203:> ' -- /home/ubuntu/cephtest/mnt.3 2023-08-09T13:35:06.325 INFO:teuthology.orchestra.run.smithi203.stdout:fuseblk 2023-08-09T13:35:06.326 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.3 2023-08-09T13:35:06.326 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-09T13:35:06.326 DEBUG:teuthology.orchestra.run.smithi203:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.3 2023-08-09T13:35:06.422 INFO:teuthology.run_tasks:Running task cephfs_test_runner... 2023-08-09T13:35:06.438 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-08-09T13:35:06.678 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:06.677+0000 7fa81ca13640 1 Processor -- start 2023-08-09T13:35:06.679 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:06.677+0000 7fa81ca13640 1 -- start start 2023-08-09T13:35:06.679 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:06.678+0000 7fa81ca13640 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7fa8180c8750 0x7fa8180ad340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:35:06.680 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:06.678+0000 7fa81ca13640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fa8180c8b30 0x7fa81806f410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:35:06.680 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:06.678+0000 7fa81ca13640 1 -- --> v1:172.21.15.203:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa8180cbd90 con 0x7fa8180acdf0 2023-08-09T13:35:06.680 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:06.678+0000 7fa81ca13640 1 -- --> v2:172.21.15.203:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa81815e810 con 0x7fa8180c8750 2023-08-09T13:35:06.680 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:06.678+0000 7fa81ca13640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa8180c9f70 con 0x7fa8180c8b30 2023-08-09T13:35:06.681 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:06.679+0000 7fa816d9d640 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7fa8180c8750 0x7fa8180ad340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:35:06.681 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:06.679+0000 7fa80e59c640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fa8180c8b30 0x7fa81806f410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:35:06.681 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:06.679+0000 7fa816d9d640 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7fa8180c8750 0x7fa8180ad340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.203:3300/0 says I am v2:172.21.15.43:43794/0 (socket says 172.21.15.43:43794) 2023-08-09T13:35:06.681 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:06.679+0000 7fa816d9d640 1 -- 172.21.15.43:0/4058318741 learned_addr learned my addr 172.21.15.43:0/4058318741 (peer_addr_for_me v2:172.21.15.43:0/0) 2023-08-09T13:35:06.682 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:06.680+0000 7fa816d9d640 1 -- 172.21.15.43:0/4058318741 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fa8180c8b30 msgr2=0x7fa81806f410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-09T13:35:06.682 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:06.680+0000 7fa816d9d640 1 --2- 172.21.15.43:0/4058318741 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fa8180c8b30 0x7fa81806f410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:35:06.682 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:06.680+0000 7fa816d9d640 1 -- 172.21.15.43:0/4058318741 >> v1:172.21.15.203:6789/0 conn(0x7fa8180acdf0 legacy=0x7fa818070f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:35:06.682 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:06.680+0000 7fa816d9d640 1 -- 172.21.15.43:0/4058318741 --> v2:172.21.15.203:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa8180ca2b0 con 0x7fa8180c8750 2023-08-09T13:35:06.682 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:06.680+0000 7fa81659c640 1 -- 172.21.15.43:0/4058318741 <== mon.1 v1:172.21.15.203:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3097900098 0 0) 0x7fa8180cbd90 con 0x7fa8180acdf0 2023-08-09T13:35:06.683 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:06.680+0000 7fa80e59c640 1 --2- 172.21.15.43:0/4058318741 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fa8180c8b30 0x7fa81806f410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-08-09T13:35:06.683 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:06.682+0000 7fa816d9d640 1 --2- 172.21.15.43:0/4058318741 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fa8180c8750 0x7fa8180ad340 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7fa808009870 tx=0x7fa80802b980 comp rx=0 tx=0).ready entity=mon.1 client_cookie=f99f4662a912d3ce server_cookie=0 in_seq=0 out_seq=0 2023-08-09T13:35:06.684 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:06.683+0000 7fa81659c640 1 -- 172.21.15.43:0/4058318741 <== mon.1 v2:172.21.15.203:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa808030840 con 0x7fa8180c8750 2023-08-09T13:35:06.684 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:06.683+0000 7fa81659c640 1 -- 172.21.15.43:0/4058318741 <== mon.1 v2:172.21.15.203:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa8080309a0 con 0x7fa8180c8750 2023-08-09T13:35:06.684 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:06.683+0000 7fa81659c640 1 -- 172.21.15.43:0/4058318741 <== mon.1 v2:172.21.15.203:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa808030d30 con 0x7fa8180c8750 2023-08-09T13:35:06.685 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:06.683+0000 7fa81ca13640 1 -- 172.21.15.43:0/4058318741 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fa8180c8750 msgr2=0x7fa8180ad340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:35:06.685 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:06.683+0000 7fa81ca13640 1 --2- 172.21.15.43:0/4058318741 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fa8180c8750 0x7fa8180ad340 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7fa808009870 tx=0x7fa80802b980 comp rx=0 tx=0).stop 2023-08-09T13:35:06.685 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:06.683+0000 7fa81ca13640 1 -- 172.21.15.43:0/4058318741 shutdown_connections 2023-08-09T13:35:06.685 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:06.683+0000 7fa81ca13640 1 --2- 172.21.15.43:0/4058318741 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fa8180c8b30 0x7fa81806f410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:35:06.686 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:06.683+0000 7fa81ca13640 1 --2- 172.21.15.43:0/4058318741 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fa8180c8750 0x7fa8180ad340 unknown :-1 s=CLOSED pgs=129 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:35:06.686 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:06.683+0000 7fa81ca13640 1 -- 172.21.15.43:0/4058318741 >> 172.21.15.43:0/4058318741 conn(0x7fa8180cde90 msgr2=0x7fa8180ce2a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-09T13:35:06.686 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:06.683+0000 7fa81ca13640 1 -- 172.21.15.43:0/4058318741 shutdown_connections 2023-08-09T13:35:06.686 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:06.684+0000 7fa81ca13640 1 -- 172.21.15.43:0/4058318741 wait complete. 2023-08-09T13:35:06.687 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:06.685+0000 7fa81ca13640 1 Processor -- start 2023-08-09T13:35:06.687 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:06.685+0000 7fa81ca13640 1 -- start start 2023-08-09T13:35:06.688 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:06.686+0000 7fa81ca13640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fa8180acdf0 0x7fa8181591d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:35:06.688 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:06.686+0000 7fa81ca13640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fa8180c8750 0x7fa8180c74e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:35:06.688 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:06.686+0000 7fa81ca13640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa8180c8b30 0x7fa8180b0ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:35:06.688 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:06.686+0000 7fa81ca13640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa8180c7a20 con 0x7fa8180c8b30 2023-08-09T13:35:06.688 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:06.686+0000 7fa81ca13640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa8180b3800 con 0x7fa8180c8750 2023-08-09T13:35:06.688 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:06.686+0000 7fa81ca13640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa8180b2ed0 con 0x7fa8180acdf0 2023-08-09T13:35:06.689 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:06.687+0000 7fa81759e640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa8180c8b30 0x7fa8180b0ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:35:06.689 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:06.687+0000 7fa81759e640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa8180c8b30 0x7fa8180b0ab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:36930/0 (socket says 172.21.15.43:36930) 2023-08-09T13:35:06.690 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:06.687+0000 7fa81759e640 1 -- 172.21.15.43:0/3905422412 learned_addr learned my addr 172.21.15.43:0/3905422412 (peer_addr_for_me v2:172.21.15.43:0/0) 2023-08-09T13:35:06.690 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:06.687+0000 7fa80e59c640 1 --2- 172.21.15.43:0/3905422412 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fa8180c8750 0x7fa8180c74e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:35:06.690 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:06.687+0000 7fa816d9d640 1 --2- 172.21.15.43:0/3905422412 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fa8180acdf0 0x7fa8181591d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:35:06.690 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:06.687+0000 7fa81759e640 1 -- 172.21.15.43:0/3905422412 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fa8180acdf0 msgr2=0x7fa8181591d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:35:06.690 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:06.687+0000 7fa81759e640 1 --2- 172.21.15.43:0/3905422412 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fa8180acdf0 0x7fa8181591d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:35:06.690 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:06.688+0000 7fa81759e640 1 -- 172.21.15.43:0/3905422412 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fa8180c8750 msgr2=0x7fa8180c74e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:35:06.690 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:06.688+0000 7fa81759e640 1 --2- 172.21.15.43:0/3905422412 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fa8180c8750 0x7fa8180c74e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:35:06.691 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:06.688+0000 7fa81759e640 1 -- 172.21.15.43:0/3905422412 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa800009400 con 0x7fa8180c8b30 2023-08-09T13:35:06.691 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:06.688+0000 7fa80e59c640 1 --2- 172.21.15.43:0/3905422412 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fa8180c8750 0x7fa8180c74e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-08-09T13:35:06.691 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:06.688+0000 7fa81759e640 1 --2- 172.21.15.43:0/3905422412 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa8180c8b30 0x7fa8180b0ab0 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7fa810001560 tx=0x7fa81000cc70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-09T13:35:06.691 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:06.688+0000 7fa80ffff640 1 -- 172.21.15.43:0/3905422412 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa810003da0 con 0x7fa8180c8b30 2023-08-09T13:35:06.692 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:06.688+0000 7fa80ffff640 1 -- 172.21.15.43:0/3905422412 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa810005070 con 0x7fa8180c8b30 2023-08-09T13:35:06.692 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:06.688+0000 7fa80ffff640 1 -- 172.21.15.43:0/3905422412 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa810005420 con 0x7fa8180c8b30 2023-08-09T13:35:06.692 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:06.688+0000 7fa81ca13640 1 -- 172.21.15.43:0/3905422412 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa8080094d0 con 0x7fa8180c8b30 2023-08-09T13:35:06.692 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:06.689+0000 7fa81ca13640 1 -- 172.21.15.43:0/3905422412 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa8180b3490 con 0x7fa8180c8b30 2023-08-09T13:35:06.692 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:06.690+0000 7fa81ca13640 1 -- 172.21.15.43:0/3905422412 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa8180c9230 con 0x7fa8180c8b30 2023-08-09T13:35:06.693 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:06.691+0000 7fa80ffff640 1 -- 172.21.15.43:0/3905422412 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 5) v1 ==== 88341+0+0 (secure 0 0 0) 0x7fa810005580 con 0x7fa8180c8b30 2023-08-09T13:35:06.693 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:06.692+0000 7fa80ffff640 1 --2- 172.21.15.43:0/3905422412 >> [v2:172.21.15.203:6832/830097474,v1:172.21.15.203:6833/830097474] conn(0x7fa7f806a050 0x7fa7f806c510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:35:06.694 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:06.692+0000 7fa80ffff640 1 -- 172.21.15.43:0/3905422412 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 6871+0+0 (secure 0 0 0) 0x7fa8100881a0 con 0x7fa8180c8b30 2023-08-09T13:35:06.694 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:06.693+0000 7fa816d9d640 1 --2- 172.21.15.43:0/3905422412 >> [v2:172.21.15.203:6832/830097474,v1:172.21.15.203:6833/830097474] conn(0x7fa7f806a050 0x7fa7f806c510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:35:06.696 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:06.694+0000 7fa816d9d640 1 --2- 172.21.15.43:0/3905422412 >> [v2:172.21.15.203:6832/830097474,v1:172.21.15.203:6833/830097474] conn(0x7fa7f806a050 0x7fa7f806c510 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7fa808036000 tx=0x7fa808008030 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-09T13:35:06.698 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:06.697+0000 7fa80ffff640 1 -- 172.21.15.43:0/3905422412 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178066 (secure 0 0 0) 0x7fa810053560 con 0x7fa8180c8b30 2023-08-09T13:35:06.895 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:06.893+0000 7fa81ca13640 1 -- 172.21.15.43:0/3905422412 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fa8180739f0 con 0x7fa8180c8b30 2023-08-09T13:35:06.897 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:06.896+0000 7fa80ffff640 1 -- 172.21.15.43:0/3905422412 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v26) v1 ==== 74+0+20146 (secure 0 0 0) 0x7fa810056b80 con 0x7fa8180c8b30 2023-08-09T13:35:06.898 INFO:teuthology.orchestra.run.smithi043.stdout: 2023-08-09T13:35:06.898 INFO:teuthology.orchestra.run.smithi043.stdout:{"epoch":26,"fsid":"daac87ee-1337-4250-a955-aea4e408a39b","created":"2023-08-09T13:33:41.701198+0000","modified":"2023-08-09T13:34:48.711911+0000","last_up_change":"2023-08-09T13:33:53.783453+0000","last_in_change":"2023-08-09T13:33:45.148904+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":5,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":"rbd","create_time":"2023-08-09T13:33:54.365137+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":".mgr","create_time":"2023-08-09T13:33:54.831803+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_metadata","create_time":"2023-08-09T13:34:00.207974+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data","create_time":"2023-08-09T13:34:01.614765+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":5,"pool_name":"cephfs_data_ec","create_time":"2023-08-09T13:34:05.239583+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"24","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"cbd49ecd-e896-4fbd-864e-5c37a829b165","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6800","nonce":502667335},{"type":"v1","addr":"172.21.15.43:6801","nonce":502667335}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6802","nonce":502667335},{"type":"v1","addr":"172.21.15.43:6803","nonce":502667335}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6806","nonce":502667335},{"type":"v1","addr":"172.21.15.43:6807","nonce":502667335}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6804","nonce":502667335},{"type":"v1","addr":"172.21.15.43:6805","nonce":502667335}]},"public_addr":"172.21.15.43:6801/502667335","cluster_addr":"172.21.15.43:6803/502667335","heartbeat_back_addr":"172.21.15.43:6807/502667335","heartbeat_front_addr":"172.21.15.43:6805/502667335","state":["exists","up"]},{"osd":1,"uuid":"75c7e0fe-1ec7-4fe8-a2f2-3661a65b8500","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6808","nonce":3289134610},{"type":"v1","addr":"172.21.15.43:6809","nonce":3289134610}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6810","nonce":3289134610},{"type":"v1","addr":"172.21.15.43:6812","nonce":3289134610}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6821","nonce":3289134610},{"type":"v1","addr":"172.21.15.43:6824","nonce":3289134610}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6815","nonce":3289134610},{"type":"v1","addr":"172.21.15.43:6818","nonce":3289134610}]},"public_addr":"172.21.15.43:6809/3289134610","cluster_addr":"172.21.15.43:6812/3289134610","heartbeat_back_addr":"172.21.15.43:6824/3289134610","heartbeat_front_addr":"172.21.15.43:6818/3289134610","state":["exists","up"]},{"osd":2,"uuid":"bb5c41e9-7f5f-4125-9b01-3bb8bfc2ea83","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6811","nonce":4079709736},{"type":"v1","addr":"172.21.15.43:6814","nonce":4079709736}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6817","nonce":4079709736},{"type":"v1","addr":"172.21.15.43:6820","nonce":4079709736}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6829","nonce":4079709736},{"type":"v1","addr":"172.21.15.43:6831","nonce":4079709736}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6823","nonce":4079709736},{"type":"v1","addr":"172.21.15.43:6826","nonce":4079709736}]},"public_addr":"172.21.15.43:6814/4079709736","cluster_addr":"172.21.15.43:6820/4079709736","heartbeat_back_addr":"172.21.15.43:6831/4079709736","heartbeat_front_addr":"172.21.15.43:6826/4079709736","state":["exists","up"]},{"osd":3,"uuid":"bfe4c8b4-694a-465c-87e6-2b2aab852613","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6813","nonce":1709538759},{"type":"v1","addr":"172.21.15.43:6816","nonce":1709538759}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6819","nonce":1709538759},{"type":"v1","addr":"172.21.15.43:6822","nonce":1709538759}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6828","nonce":1709538759},{"type":"v1","addr":"172.21.15.43:6830","nonce":1709538759}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6825","nonce":1709538759},{"type":"v1","addr":"172.21.15.43:6827","nonce":1709538759}]},"public_addr":"172.21.15.43:6816/1709538759","cluster_addr":"172.21.15.43:6822/1709538759","heartbeat_back_addr":"172.21.15.43:6830/1709538759","heartbeat_front_addr":"172.21.15.43:6827/1709538759","state":["exists","up"]},{"osd":4,"uuid":"fa2e8a77-aba1-4756-853e-0d174a5bdeba","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6800","nonce":4196866481},{"type":"v1","addr":"172.21.15.203:6801","nonce":4196866481}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6802","nonce":4196866481},{"type":"v1","addr":"172.21.15.203:6803","nonce":4196866481}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6806","nonce":4196866481},{"type":"v1","addr":"172.21.15.203:6807","nonce":4196866481}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6804","nonce":4196866481},{"type":"v1","addr":"172.21.15.203:6805","nonce":4196866481}]},"public_addr":"172.21.15.203:6801/4196866481","cluster_addr":"172.21.15.203:6803/4196866481","heartbeat_back_addr":"172.21.15.203:6807/4196866481","heartbeat_front_addr":"172.21.15.203:6805/4196866481","state":["exists","up"]},{"osd":5,"uuid":"81811e4b-4d24-4f51-a82c-f1d700a2a00d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6816","nonce":3059290081},{"type":"v1","addr":"172.21.15.203:6817","nonce":3059290081}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6818","nonce":3059290081},{"type":"v1","addr":"172.21.15.203:6819","nonce":3059290081}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6822","nonce":3059290081},{"type":"v1","addr":"172.21.15.203:6823","nonce":3059290081}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6820","nonce":3059290081},{"type":"v1","addr":"172.21.15.203:6821","nonce":3059290081}]},"public_addr":"172.21.15.203:6817/3059290081","cluster_addr":"172.21.15.203:6819/3059290081","heartbeat_back_addr":"172.21.15.203:6823/3059290081","heartbeat_front_addr":"172.21.15.203:6821/3059290081","state":["exists","up"]},{"osd":6,"uuid":"307bac4d-6a5b-443d-af75-56d695d7674f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6824","nonce":512155569},{"type":"v1","addr":"172.21.15.203:6825","nonce":512155569}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6826","nonce":512155569},{"type":"v1","addr":"172.21.15.203:6827","nonce":512155569}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6830","nonce":512155569},{"type":"v1","addr":"172.21.15.203:6831","nonce":512155569}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6828","nonce":512155569},{"type":"v1","addr":"172.21.15.203:6829","nonce":512155569}]},"public_addr":"172.21.15.203:6825/512155569","cluster_addr":"172.21.15.203:6827/512155569","heartbeat_back_addr":"172.21.15.203:6831/512155569","heartbeat_front_addr":"172.21.15.203:6829/512155569","state":["exists","up"]},{"osd":7,"uuid":"a95dc7ca-9f1b-4295-af70-68a22a2d7d5e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6808","nonce":3331450438},{"type":"v1","addr":"172.21.15.203:6809","nonce":3331450438}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6810","nonce":3331450438},{"type":"v1","addr":"172.21.15.203:6811","nonce":3331450438}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6814","nonce":3331450438},{"type":"v1","addr":"172.21.15.203:6815","nonce":3331450438}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6812","nonce":3331450438},{"type":"v1","addr":"172.21.15.203:6813","nonce":3331450438}]},"public_addr":"172.21.15.203:6809/3331450438","cluster_addr":"172.21.15.203:6811/3331450438","heartbeat_back_addr":"172.21.15.203:6815/3331450438","heartbeat_front_addr":"172.21.15.203:6813/3331450438","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-09T13:33:50.547652+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-09T13:33:50.589469+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-09T13:33:50.561724+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-09T13:33:50.599650+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-09T13:33:50.411855+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-09T13:33:50.219681+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-09T13:33:50.504664+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-09T13:33:50.441248+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"4.14","mappings":[{"from":1,"to":5}]},{"pgid":"4.29","mappings":[{"from":3,"to":2}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-08-09T13:35:06.900 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:06.898+0000 7fa81ca13640 1 -- 172.21.15.43:0/3905422412 >> [v2:172.21.15.203:6832/830097474,v1:172.21.15.203:6833/830097474] conn(0x7fa7f806a050 msgr2=0x7fa7f806c510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:35:06.900 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:06.898+0000 7fa81ca13640 1 --2- 172.21.15.43:0/3905422412 >> [v2:172.21.15.203:6832/830097474,v1:172.21.15.203:6833/830097474] conn(0x7fa7f806a050 0x7fa7f806c510 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7fa808036000 tx=0x7fa808008030 comp rx=0 tx=0).stop 2023-08-09T13:35:06.900 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:06.898+0000 7fa81ca13640 1 -- 172.21.15.43:0/3905422412 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa8180c8b30 msgr2=0x7fa8180b0ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:35:06.900 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:06.898+0000 7fa81ca13640 1 --2- 172.21.15.43:0/3905422412 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa8180c8b30 0x7fa8180b0ab0 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7fa810001560 tx=0x7fa81000cc70 comp rx=0 tx=0).stop 2023-08-09T13:35:06.900 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:06.898+0000 7fa81ca13640 1 -- 172.21.15.43:0/3905422412 shutdown_connections 2023-08-09T13:35:06.901 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:06.898+0000 7fa81ca13640 1 --2- 172.21.15.43:0/3905422412 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa8180c8b30 0x7fa8180b0ab0 unknown :-1 s=CLOSED pgs=259 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:35:06.901 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:06.898+0000 7fa81ca13640 1 --2- 172.21.15.43:0/3905422412 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fa8180c8750 0x7fa8180c74e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:35:06.901 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:06.898+0000 7fa81ca13640 1 --2- 172.21.15.43:0/3905422412 >> [v2:172.21.15.203:6832/830097474,v1:172.21.15.203:6833/830097474] conn(0x7fa7f806a050 0x7fa7f806c510 unknown :-1 s=CLOSED pgs=146 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:35:06.901 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:06.898+0000 7fa81ca13640 1 --2- 172.21.15.43:0/3905422412 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fa8180acdf0 0x7fa8181591d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:35:06.901 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:06.898+0000 7fa81ca13640 1 -- 172.21.15.43:0/3905422412 >> 172.21.15.43:0/3905422412 conn(0x7fa8180cde90 msgr2=0x7fa8180abb90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-09T13:35:06.901 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:06.899+0000 7fa81ca13640 1 -- 172.21.15.43:0/3905422412 shutdown_connections 2023-08-09T13:35:06.901 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:06.899+0000 7fa81ca13640 1 -- 172.21.15.43:0/3905422412 wait complete. 2023-08-09T13:35:06.921 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': 'rbd', 'create_time': '2023-08-09T13:33:54.365137+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}, 'read_balance': {'score_acting': 2, 'score_stable': 2, 'optimal_score': 1, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': '.mgr', 'create_time': '2023-08-09T13:33:54.831803+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_metadata', 'create_time': '2023-08-09T13:34:00.207974+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data', 'create_time': '2023-08-09T13:34:01.614765+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 2, 'score_stable': 2, 'optimal_score': 1, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 5, 'pool_name': 'cephfs_data_ec', 'create_time': '2023-08-09T13:34:05.239583+0000', 'flags': 5, 'flags_names': 'hashpspool,ec_overwrites', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '24', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2023-08-09T13:35:06.921 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get rbd pg_num 2023-08-09T13:35:07.160 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:07.158+0000 7fcaf84cc640 1 Processor -- start 2023-08-09T13:35:07.160 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:07.158+0000 7fcaf84cc640 1 -- start start 2023-08-09T13:35:07.162 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:07.160+0000 7fcaf84cc640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fcaf00b1260 0x7fcaf00cee00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:35:07.163 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:07.160+0000 7fcaf84cc640 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7fcaf00cf340 0x7fcaf00d0350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:35:07.163 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:07.160+0000 7fcaf84cc640 1 -- --> v1:172.21.15.43:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fcaf00cd700 con 0x7fcaf015c620 2023-08-09T13:35:07.163 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:07.160+0000 7fcaf84cc640 1 -- --> v2:172.21.15.203:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fcaf00b1b70 con 0x7fcaf00cf340 2023-08-09T13:35:07.164 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:07.160+0000 7fcaf84cc640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcaf00cf7b0 con 0x7fcaf00b1260 2023-08-09T13:35:07.164 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:07.161+0000 7fcaf6268640 1 --1- >> v1:172.21.15.43:6789/0 conn(0x7fcaf015c620 0x7fcaf00cceb0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.43:6789/0 says I am v1:172.21.15.43:57508/0 (socket says 172.21.15.43:57508) 2023-08-09T13:35:07.164 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:07.161+0000 7fcaf6268640 1 -- 172.21.15.43:0/3915705318 learned_addr learned my addr 172.21.15.43:0/3915705318 (peer_addr_for_me v1:172.21.15.43:0/0) 2023-08-09T13:35:07.165 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:07.161+0000 7fcaf5a67640 1 --2- 172.21.15.43:0/3915705318 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fcaf00b1260 0x7fcaf00cee00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:35:07.165 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:07.161+0000 7fcaf6a69640 1 --2- 172.21.15.43:0/3915705318 >> v2:172.21.15.203:3300/0 conn(0x7fcaf00cf340 0x7fcaf00d0350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:35:07.167 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:07.161+0000 7fcaf5266640 1 -- 172.21.15.43:0/3915705318 <== mon.0 v1:172.21.15.43:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1787188500 0 0) 0x7fcaf00cd700 con 0x7fcaf015c620 2023-08-09T13:35:07.167 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:07.161+0000 7fcaf5266640 1 -- 172.21.15.43:0/3915705318 --> v1:172.21.15.43:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fcadc003520 con 0x7fcaf015c620 2023-08-09T13:35:07.168 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:07.162+0000 7fcaf5a67640 1 -- 172.21.15.43:0/3915705318 >> v2:172.21.15.203:3300/0 conn(0x7fcaf00cf340 msgr2=0x7fcaf00d0350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-09T13:35:07.168 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:07.162+0000 7fcaf5a67640 1 --2- 172.21.15.43:0/3915705318 >> v2:172.21.15.203:3300/0 conn(0x7fcaf00cf340 0x7fcaf00d0350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:35:07.168 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:07.162+0000 7fcaf5a67640 1 -- 172.21.15.43:0/3915705318 >> v1:172.21.15.43:6789/0 conn(0x7fcaf015c620 legacy=0x7fcaf00cceb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:35:07.169 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:07.162+0000 7fcaf5266640 1 -- 172.21.15.43:0/3915705318 <== mon.0 v1:172.21.15.43:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3700571585 0 0) 0x7fcadc003520 con 0x7fcaf015c620 2023-08-09T13:35:07.169 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:07.162+0000 7fcaf5a67640 1 -- 172.21.15.43:0/3915705318 --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcaf00cd9e0 con 0x7fcaf00b1260 2023-08-09T13:35:07.169 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:07.163+0000 7fcaf5a67640 1 --2- 172.21.15.43:0/3915705318 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fcaf00b1260 0x7fcaf00cee00 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7fcae00097a0 tx=0x7fcae002b660 comp rx=0 tx=0).ready entity=mon.2 client_cookie=97b23b3b0dd6b733 server_cookie=0 in_seq=0 out_seq=0 2023-08-09T13:35:07.169 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:07.164+0000 7fcaf5266640 1 -- 172.21.15.43:0/3915705318 <== mon.2 v2:172.21.15.203:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcaec005090 con 0x7fcaf00b1260 2023-08-09T13:35:07.169 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:07.164+0000 7fcaf5266640 1 -- 172.21.15.43:0/3915705318 <== mon.2 v2:172.21.15.203:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcae002e050 con 0x7fcaf00b1260 2023-08-09T13:35:07.170 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:07.164+0000 7fcaf5266640 1 -- 172.21.15.43:0/3915705318 <== mon.2 v2:172.21.15.203:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcae0030710 con 0x7fcaf00b1260 2023-08-09T13:35:07.170 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:07.164+0000 7fcaf84cc640 1 -- 172.21.15.43:0/3915705318 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fcaf00b1260 msgr2=0x7fcaf00cee00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:35:07.170 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:07.164+0000 7fcaf84cc640 1 --2- 172.21.15.43:0/3915705318 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fcaf00b1260 0x7fcaf00cee00 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7fcae00097a0 tx=0x7fcae002b660 comp rx=0 tx=0).stop 2023-08-09T13:35:07.170 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:07.165+0000 7fcaf84cc640 1 -- 172.21.15.43:0/3915705318 shutdown_connections 2023-08-09T13:35:07.170 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:07.165+0000 7fcaf84cc640 1 --2- 172.21.15.43:0/3915705318 >> v2:172.21.15.203:3300/0 conn(0x7fcaf00cf340 0x7fcaf00d0350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:35:07.170 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:07.165+0000 7fcaf84cc640 1 --2- 172.21.15.43:0/3915705318 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fcaf00b1260 0x7fcaf00cee00 unknown :-1 s=CLOSED pgs=58 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:35:07.170 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:07.165+0000 7fcaf84cc640 1 -- 172.21.15.43:0/3915705318 >> 172.21.15.43:0/3915705318 conn(0x7fcaf009b750 msgr2=0x7fcaf00d2410 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-09T13:35:07.171 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:07.165+0000 7fcaf84cc640 1 -- 172.21.15.43:0/3915705318 shutdown_connections 2023-08-09T13:35:07.171 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:07.165+0000 7fcaf84cc640 1 -- 172.21.15.43:0/3915705318 wait complete. 2023-08-09T13:35:07.173 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:07.167+0000 7fcaf84cc640 1 Processor -- start 2023-08-09T13:35:07.173 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:07.167+0000 7fcaf84cc640 1 -- start start 2023-08-09T13:35:07.173 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:07.168+0000 7fcaf84cc640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fcaf00b1260 0x7fcaf0157f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:35:07.173 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:07.168+0000 7fcaf84cc640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fcaf00cf340 0x7fcaf00b6d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:35:07.173 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:07.168+0000 7fcaf84cc640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fcaf015c620 0x7fcaf00af040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:35:07.173 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:07.168+0000 7fcaf84cc640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcaf00b0760 con 0x7fcaf00cf340 2023-08-09T13:35:07.174 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:07.168+0000 7fcaf84cc640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcaf009c510 con 0x7fcaf015c620 2023-08-09T13:35:07.174 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:07.168+0000 7fcaf84cc640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcaf00b01a0 con 0x7fcaf00b1260 2023-08-09T13:35:07.174 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:07.168+0000 7fcaf5a67640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fcaf00cf340 0x7fcaf00b6d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:35:07.174 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:07.168+0000 7fcaf5a67640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fcaf00cf340 0x7fcaf00b6d30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:36942/0 (socket says 172.21.15.43:36942) 2023-08-09T13:35:07.174 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:07.168+0000 7fcaf5a67640 1 -- 172.21.15.43:0/298409574 learned_addr learned my addr 172.21.15.43:0/298409574 (peer_addr_for_me v2:172.21.15.43:0/0) 2023-08-09T13:35:07.174 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:07.169+0000 7fcaf6268640 1 --2- 172.21.15.43:0/298409574 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fcaf00b1260 0x7fcaf0157f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:35:07.174 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:07.169+0000 7fcaf6a69640 1 --2- 172.21.15.43:0/298409574 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fcaf015c620 0x7fcaf00af040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:35:07.175 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:07.169+0000 7fcaf5a67640 1 -- 172.21.15.43:0/298409574 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fcaf00b1260 msgr2=0x7fcaf0157f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:35:07.175 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:07.169+0000 7fcaf5a67640 1 --2- 172.21.15.43:0/298409574 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fcaf00b1260 0x7fcaf0157f50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:35:07.175 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:07.169+0000 7fcaf5a67640 1 -- 172.21.15.43:0/298409574 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fcaf015c620 msgr2=0x7fcaf00af040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:35:07.175 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:07.169+0000 7fcaf5a67640 1 --2- 172.21.15.43:0/298409574 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fcaf015c620 0x7fcaf00af040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:35:07.175 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:07.169+0000 7fcaf5a67640 1 -- 172.21.15.43:0/298409574 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcae0009400 con 0x7fcaf00cf340 2023-08-09T13:35:07.175 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:07.169+0000 7fcaf5a67640 1 --2- 172.21.15.43:0/298409574 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fcaf00cf340 0x7fcaf00b6d30 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7fcae0000c00 tx=0x7fcae00046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-09T13:35:07.176 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:07.169+0000 7fcae6ffd640 1 -- 172.21.15.43:0/298409574 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcae003c620 con 0x7fcaf00cf340 2023-08-09T13:35:07.176 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:07.170+0000 7fcae6ffd640 1 -- 172.21.15.43:0/298409574 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcae003c7a0 con 0x7fcaf00cf340 2023-08-09T13:35:07.176 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:07.170+0000 7fcae6ffd640 1 -- 172.21.15.43:0/298409574 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcae003cb50 con 0x7fcaf00cf340 2023-08-09T13:35:07.176 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:07.170+0000 7fcaf84cc640 1 -- 172.21.15.43:0/298409574 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcaf00b0480 con 0x7fcaf00cf340 2023-08-09T13:35:07.176 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:07.170+0000 7fcaf84cc640 1 -- 172.21.15.43:0/298409574 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcaf009c7f0 con 0x7fcaf00cf340 2023-08-09T13:35:07.176 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:07.171+0000 7fcaf84cc640 1 -- 172.21.15.43:0/298409574 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcaf00739f0 con 0x7fcaf00cf340 2023-08-09T13:35:07.177 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:07.172+0000 7fcae6ffd640 1 -- 172.21.15.43:0/298409574 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 5) v1 ==== 88341+0+0 (secure 0 0 0) 0x7fcae0039070 con 0x7fcaf00cf340 2023-08-09T13:35:07.177 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:07.173+0000 7fcae6ffd640 1 --2- 172.21.15.43:0/298409574 >> [v2:172.21.15.203:6832/830097474,v1:172.21.15.203:6833/830097474] conn(0x7fcad406a050 0x7fcad406c510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:35:07.177 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:07.173+0000 7fcae6ffd640 1 -- 172.21.15.43:0/298409574 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 6871+0+0 (secure 0 0 0) 0x7fcae00a9ee0 con 0x7fcaf00cf340 2023-08-09T13:35:07.177 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:07.173+0000 7fcaf6268640 1 --2- 172.21.15.43:0/298409574 >> [v2:172.21.15.203:6832/830097474,v1:172.21.15.203:6833/830097474] conn(0x7fcad406a050 0x7fcad406c510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:35:07.178 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:07.175+0000 7fcaf6268640 1 --2- 172.21.15.43:0/298409574 >> [v2:172.21.15.203:6832/830097474,v1:172.21.15.203:6833/830097474] conn(0x7fcad406a050 0x7fcad406c510 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7fcaec004c20 tx=0x7fcaec009040 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-09T13:35:07.179 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:07.177+0000 7fcae6ffd640 1 -- 172.21.15.43:0/298409574 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178066 (secure 0 0 0) 0x7fcae0075290 con 0x7fcaf00cf340 2023-08-09T13:35:07.374 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:07.372+0000 7fcaf84cc640 1 -- 172.21.15.43:0/298409574 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"} v 0) v1 -- 0x7fcaf015b030 con 0x7fcaf00cf340 2023-08-09T13:35:07.375 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:07.373+0000 7fcae6ffd640 1 -- 172.21.15.43:0/298409574 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"}]=0 v26) v1 ==== 92+0+10 (secure 0 0 0) 0x7fcae0037020 con 0x7fcaf00cf340 2023-08-09T13:35:07.375 INFO:teuthology.orchestra.run.smithi043.stdout:pg_num: 8 2023-08-09T13:35:07.377 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:07.376+0000 7fcaf84cc640 1 -- 172.21.15.43:0/298409574 >> [v2:172.21.15.203:6832/830097474,v1:172.21.15.203:6833/830097474] conn(0x7fcad406a050 msgr2=0x7fcad406c510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:35:07.378 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:07.376+0000 7fcaf84cc640 1 --2- 172.21.15.43:0/298409574 >> [v2:172.21.15.203:6832/830097474,v1:172.21.15.203:6833/830097474] conn(0x7fcad406a050 0x7fcad406c510 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7fcaec004c20 tx=0x7fcaec009040 comp rx=0 tx=0).stop 2023-08-09T13:35:07.378 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:07.376+0000 7fcaf84cc640 1 -- 172.21.15.43:0/298409574 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fcaf00cf340 msgr2=0x7fcaf00b6d30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:35:07.378 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:07.376+0000 7fcaf84cc640 1 --2- 172.21.15.43:0/298409574 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fcaf00cf340 0x7fcaf00b6d30 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7fcae0000c00 tx=0x7fcae00046c0 comp rx=0 tx=0).stop 2023-08-09T13:35:07.378 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:07.376+0000 7fcaf84cc640 1 -- 172.21.15.43:0/298409574 shutdown_connections 2023-08-09T13:35:07.378 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:07.376+0000 7fcaf84cc640 1 --2- 172.21.15.43:0/298409574 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fcaf015c620 0x7fcaf00af040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:35:07.378 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:07.376+0000 7fcaf84cc640 1 --2- 172.21.15.43:0/298409574 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fcaf00cf340 0x7fcaf00b6d30 unknown :-1 s=CLOSED pgs=261 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:35:07.379 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:07.376+0000 7fcaf84cc640 1 --2- 172.21.15.43:0/298409574 >> [v2:172.21.15.203:6832/830097474,v1:172.21.15.203:6833/830097474] conn(0x7fcad406a050 0x7fcad406c510 unknown :-1 s=CLOSED pgs=147 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:35:07.379 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:07.376+0000 7fcaf84cc640 1 --2- 172.21.15.43:0/298409574 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fcaf00b1260 0x7fcaf0157f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:35:07.379 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:07.376+0000 7fcaf84cc640 1 -- 172.21.15.43:0/298409574 >> 172.21.15.43:0/298409574 conn(0x7fcaf009b750 msgr2=0x7fcaf00d0350 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-09T13:35:07.379 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:07.376+0000 7fcaf84cc640 1 -- 172.21.15.43:0/298409574 shutdown_connections 2023-08-09T13:35:07.379 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:07.376+0000 7fcaf84cc640 1 -- 172.21.15.43:0/298409574 wait complete. 2023-08-09T13:35:07.395 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-08-09T13:35:07.628 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:07.627+0000 7f253f139640 1 Processor -- start 2023-08-09T13:35:07.629 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:07.627+0000 7f253f139640 1 -- start start 2023-08-09T13:35:07.630 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:07.629+0000 7f253f139640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f253815c940 0x7f253806ab70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:35:07.630 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:07.629+0000 7f253f139640 1 -- --> v1:172.21.15.43:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f25380cefd0 con 0x7f25380ca810 2023-08-09T13:35:07.630 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:07.629+0000 7f253f139640 1 -- --> v1:172.21.15.203:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f25380cf580 con 0x7f25380b0de0 2023-08-09T13:35:07.631 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:07.629+0000 7f253f139640 1 -- --> v2:172.21.15.43:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f25380b1660 con 0x7f253815c940 2023-08-09T13:35:07.631 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:07.629+0000 7f253d6d6640 1 --1- >> v1:172.21.15.43:6789/0 conn(0x7f25380ca810 0x7f25380caca0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.43:6789/0 says I am v1:172.21.15.43:57524/0 (socket says 172.21.15.43:57524) 2023-08-09T13:35:07.631 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:07.629+0000 7f253ced5640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f253815c940 0x7f253806ab70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:35:07.632 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:07.629+0000 7f253d6d6640 1 -- 172.21.15.43:0/4022759104 learned_addr learned my addr 172.21.15.43:0/4022759104 (peer_addr_for_me v1:172.21.15.43:0/0) 2023-08-09T13:35:07.632 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:07.630+0000 7f253ced5640 1 -- 172.21.15.43:0/4022759104 >> v1:172.21.15.203:6789/0 conn(0x7f25380b0de0 legacy=0x7f253806b0b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-09T13:35:07.632 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:07.630+0000 7f253ced5640 1 -- 172.21.15.43:0/4022759104 >> v1:172.21.15.43:6789/0 conn(0x7f25380ca810 legacy=0x7f25380caca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:35:07.632 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:07.630+0000 7f253ced5640 1 -- 172.21.15.43:0/4022759104 --> v2:172.21.15.43:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f253815cd20 con 0x7f253815c940 2023-08-09T13:35:07.633 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:07.630+0000 7f25377fe640 1 -- 172.21.15.43:0/4022759104 <== mon.0 v1:172.21.15.43:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3270159687 0 0) 0x7f2530001110 con 0x7f25380ca810 2023-08-09T13:35:07.633 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:07.631+0000 7f253ced5640 1 --2- 172.21.15.43:0/4022759104 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f253815c940 0x7f253806ab70 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7f2528009790 tx=0x7f252802b8b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=75d6fd8413e19546 server_cookie=0 in_seq=0 out_seq=0 2023-08-09T13:35:07.634 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:07.631+0000 7f25377fe640 1 -- 172.21.15.43:0/4022759104 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f252802d040 con 0x7f253815c940 2023-08-09T13:35:07.634 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:07.631+0000 7f25377fe640 1 -- 172.21.15.43:0/4022759104 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2528031810 con 0x7f253815c940 2023-08-09T13:35:07.634 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:07.631+0000 7f25377fe640 1 -- 172.21.15.43:0/4022759104 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2528031ba0 con 0x7f253815c940 2023-08-09T13:35:07.634 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:07.632+0000 7f253f139640 1 -- 172.21.15.43:0/4022759104 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f253815c940 msgr2=0x7f253806ab70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:35:07.634 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:07.632+0000 7f253f139640 1 --2- 172.21.15.43:0/4022759104 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f253815c940 0x7f253806ab70 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7f2528009790 tx=0x7f252802b8b0 comp rx=0 tx=0).stop 2023-08-09T13:35:07.634 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:07.632+0000 7f253f139640 1 -- 172.21.15.43:0/4022759104 shutdown_connections 2023-08-09T13:35:07.634 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:07.632+0000 7f253f139640 1 --2- 172.21.15.43:0/4022759104 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f253815c940 0x7f253806ab70 unknown :-1 s=CLOSED pgs=263 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:35:07.635 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:07.632+0000 7f253f139640 1 -- 172.21.15.43:0/4022759104 >> 172.21.15.43:0/4022759104 conn(0x7f2538089fe0 msgr2=0x7f25380d01c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-09T13:35:07.635 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:07.632+0000 7f253f139640 1 -- 172.21.15.43:0/4022759104 shutdown_connections 2023-08-09T13:35:07.635 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:07.632+0000 7f253f139640 1 -- 172.21.15.43:0/4022759104 wait complete. 2023-08-09T13:35:07.635 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:07.634+0000 7f253f139640 1 Processor -- start 2023-08-09T13:35:07.636 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:07.634+0000 7f253f139640 1 -- start start 2023-08-09T13:35:07.636 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:07.635+0000 7f253f139640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f25380b0de0 0x7f25380c9510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:35:07.636 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:07.635+0000 7f253f139640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f25380ca810 0x7f253814c060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:35:07.636 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:07.635+0000 7f253f139640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f253815c940 0x7f253814b1e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:35:07.637 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:07.635+0000 7f253f139640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f25380c3d50 con 0x7f253815c940 2023-08-09T13:35:07.637 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:07.635+0000 7f253f139640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f25380c4030 con 0x7f25380b0de0 2023-08-09T13:35:07.637 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:07.635+0000 7f253f139640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f25380c4310 con 0x7f25380ca810 2023-08-09T13:35:07.638 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:07.635+0000 7f253d6d6640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f253815c940 0x7f253814b1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:35:07.638 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:07.635+0000 7f253d6d6640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f253815c940 0x7f253814b1e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:36962/0 (socket says 172.21.15.43:36962) 2023-08-09T13:35:07.638 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:07.635+0000 7f253d6d6640 1 -- 172.21.15.43:0/1340260041 learned_addr learned my addr 172.21.15.43:0/1340260041 (peer_addr_for_me v2:172.21.15.43:0/0) 2023-08-09T13:35:07.638 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:07.636+0000 7f253d6d6640 1 -- 172.21.15.43:0/1340260041 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f25380ca810 msgr2=0x7f253814c060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:35:07.639 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:07.636+0000 7f253d6d6640 1 --2- 172.21.15.43:0/1340260041 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f25380ca810 0x7f253814c060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:35:07.639 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:07.636+0000 7f253ced5640 1 --2- 172.21.15.43:0/1340260041 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f25380b0de0 0x7f25380c9510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:35:07.639 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:07.636+0000 7f253d6d6640 1 -- 172.21.15.43:0/1340260041 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f25380b0de0 msgr2=0x7f25380c9510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:35:07.639 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:07.636+0000 7f253d6d6640 1 --2- 172.21.15.43:0/1340260041 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f25380b0de0 0x7f25380c9510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:35:07.639 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:07.636+0000 7f253d6d6640 1 -- 172.21.15.43:0/1340260041 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f25280093a0 con 0x7f253815c940 2023-08-09T13:35:07.639 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:07.636+0000 7f253d6d6640 1 --2- 172.21.15.43:0/1340260041 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f253815c940 0x7f253814b1e0 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7f253000c780 tx=0x7f253000cc40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-09T13:35:07.639 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:07.636+0000 7f25357fa640 1 -- 172.21.15.43:0/1340260041 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f25300072d0 con 0x7f253815c940 2023-08-09T13:35:07.640 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:07.636+0000 7f25357fa640 1 -- 172.21.15.43:0/1340260041 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2530007450 con 0x7f253815c940 2023-08-09T13:35:07.640 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:07.636+0000 7f25357fa640 1 -- 172.21.15.43:0/1340260041 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2530001d10 con 0x7f253815c940 2023-08-09T13:35:07.640 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:07.636+0000 7f253f139640 1 -- 172.21.15.43:0/1340260041 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f25380c45f0 con 0x7f253815c940 2023-08-09T13:35:07.640 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:07.636+0000 7f253f139640 1 -- 172.21.15.43:0/1340260041 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f25380c9a50 con 0x7f253815c940 2023-08-09T13:35:07.640 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:07.638+0000 7f253f139640 1 -- 172.21.15.43:0/1340260041 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f25380b1ab0 con 0x7f253815c940 2023-08-09T13:35:07.641 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:07.639+0000 7f25357fa640 1 -- 172.21.15.43:0/1340260041 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 5) v1 ==== 88341+0+0 (secure 0 0 0) 0x7f25300047e0 con 0x7f253815c940 2023-08-09T13:35:07.641 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:07.639+0000 7f25357fa640 1 --2- 172.21.15.43:0/1340260041 >> [v2:172.21.15.203:6832/830097474,v1:172.21.15.203:6833/830097474] conn(0x7f252406a050 0x7f252406c510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:35:07.641 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:07.640+0000 7f25357fa640 1 -- 172.21.15.43:0/1340260041 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 6871+0+0 (secure 0 0 0) 0x7f25300879e0 con 0x7f253815c940 2023-08-09T13:35:07.642 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:07.640+0000 7f253ced5640 1 --2- 172.21.15.43:0/1340260041 >> [v2:172.21.15.203:6832/830097474,v1:172.21.15.203:6833/830097474] conn(0x7f252406a050 0x7f252406c510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:35:07.643 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:07.642+0000 7f253ced5640 1 --2- 172.21.15.43:0/1340260041 >> [v2:172.21.15.203:6832/830097474,v1:172.21.15.203:6833/830097474] conn(0x7f252406a050 0x7f252406c510 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7f2528036000 tx=0x7f252803c040 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-09T13:35:07.646 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:07.644+0000 7f25357fa640 1 -- 172.21.15.43:0/1340260041 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178066 (secure 0 0 0) 0x7f2530052d90 con 0x7f253815c940 2023-08-09T13:35:07.841 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:07.839+0000 7f253f139640 1 -- 172.21.15.43:0/1340260041 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f25380c4b10 con 0x7f253815c940 2023-08-09T13:35:07.841 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:07.840+0000 7f25357fa640 1 -- 172.21.15.43:0/1340260041 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v26) v1 ==== 93+0+10 (secure 0 0 0) 0x7f25380c4b10 con 0x7f253815c940 2023-08-09T13:35:07.842 INFO:teuthology.orchestra.run.smithi043.stdout:pg_num: 1 2023-08-09T13:35:07.844 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:07.842+0000 7f253f139640 1 -- 172.21.15.43:0/1340260041 >> [v2:172.21.15.203:6832/830097474,v1:172.21.15.203:6833/830097474] conn(0x7f252406a050 msgr2=0x7f252406c510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:35:07.844 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:07.842+0000 7f253f139640 1 --2- 172.21.15.43:0/1340260041 >> [v2:172.21.15.203:6832/830097474,v1:172.21.15.203:6833/830097474] conn(0x7f252406a050 0x7f252406c510 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7f2528036000 tx=0x7f252803c040 comp rx=0 tx=0).stop 2023-08-09T13:35:07.844 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:07.842+0000 7f253f139640 1 -- 172.21.15.43:0/1340260041 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f253815c940 msgr2=0x7f253814b1e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:35:07.845 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:07.842+0000 7f253f139640 1 --2- 172.21.15.43:0/1340260041 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f253815c940 0x7f253814b1e0 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7f253000c780 tx=0x7f253000cc40 comp rx=0 tx=0).stop 2023-08-09T13:35:07.845 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:07.842+0000 7f253f139640 1 -- 172.21.15.43:0/1340260041 shutdown_connections 2023-08-09T13:35:07.845 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:07.842+0000 7f253f139640 1 --2- 172.21.15.43:0/1340260041 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f253815c940 0x7f253814b1e0 unknown :-1 s=CLOSED pgs=264 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:35:07.845 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:07.842+0000 7f253f139640 1 --2- 172.21.15.43:0/1340260041 >> [v2:172.21.15.203:6832/830097474,v1:172.21.15.203:6833/830097474] conn(0x7f252406a050 0x7f252406c510 unknown :-1 s=CLOSED pgs=148 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:35:07.845 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:07.842+0000 7f253f139640 1 --2- 172.21.15.43:0/1340260041 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f25380ca810 0x7f253814c060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:35:07.845 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:07.842+0000 7f253f139640 1 --2- 172.21.15.43:0/1340260041 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f25380b0de0 0x7f25380c9510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:35:07.845 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:07.842+0000 7f253f139640 1 -- 172.21.15.43:0/1340260041 >> 172.21.15.43:0/1340260041 conn(0x7f2538089fe0 msgr2=0x7f253806d9f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-09T13:35:07.846 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:07.843+0000 7f253f139640 1 -- 172.21.15.43:0/1340260041 shutdown_connections 2023-08-09T13:35:07.846 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:07.843+0000 7f253f139640 1 -- 172.21.15.43:0/1340260041 wait complete. 2023-08-09T13:35:07.865 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-08-09T13:35:08.100 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:08.098+0000 7f839faa9640 1 Processor -- start 2023-08-09T13:35:08.100 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:08.098+0000 7f839faa9640 1 -- start start 2023-08-09T13:35:08.102 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:08.100+0000 7f839faa9640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f83980ccf20 0x7f83980cd550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:35:08.102 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:08.100+0000 7f839faa9640 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7f83980cda90 0x7f839815ca10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:35:08.102 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:08.100+0000 7f839faa9640 1 -- --> v1:172.21.15.203:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f83980cca90 con 0x7f839815c630 2023-08-09T13:35:08.102 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:08.100+0000 7f839faa9640 1 -- --> v2:172.21.15.43:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f83980ac600 con 0x7f83980ccf20 2023-08-09T13:35:08.102 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:08.100+0000 7f839faa9640 1 -- --> v2:172.21.15.203:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f83980cbc70 con 0x7f83980cda90 2023-08-09T13:35:08.103 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:08.100+0000 7f839d044640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f83980ccf20 0x7f83980cd550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:35:08.103 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:08.100+0000 7f839d044640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f83980ccf20 0x7f83980cd550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:36976/0 (socket says 172.21.15.43:36976) 2023-08-09T13:35:08.103 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:08.101+0000 7f839d044640 1 -- 172.21.15.43:0/2286221427 learned_addr learned my addr 172.21.15.43:0/2286221427 (peer_addr_for_me v2:172.21.15.43:0/0) 2023-08-09T13:35:08.103 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:08.101+0000 7f839e046640 1 --2- 172.21.15.43:0/2286221427 >> v2:172.21.15.203:3300/0 conn(0x7f83980cda90 0x7f839815ca10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:35:08.104 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:08.101+0000 7f839d044640 1 -- 172.21.15.43:0/2286221427 >> v2:172.21.15.203:3300/0 conn(0x7f83980cda90 msgr2=0x7f839815ca10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-09T13:35:08.104 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:08.101+0000 7f839d044640 1 --2- 172.21.15.43:0/2286221427 >> v2:172.21.15.203:3300/0 conn(0x7f83980cda90 0x7f839815ca10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:35:08.104 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:08.101+0000 7f839d044640 1 -- 172.21.15.43:0/2286221427 >> v1:172.21.15.203:6789/0 conn(0x7f839815c630 legacy=0x7f83980abef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-09T13:35:08.104 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:08.101+0000 7f839d044640 1 -- 172.21.15.43:0/2286221427 --> v2:172.21.15.43:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f83980cdea0 con 0x7f83980ccf20 2023-08-09T13:35:08.105 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:08.101+0000 7f839e046640 1 --2- 172.21.15.43:0/2286221427 >> v2:172.21.15.203:3300/0 conn(0x7f83980cda90 0x7f839815ca10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-08-09T13:35:08.105 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:08.103+0000 7f839d044640 1 --2- 172.21.15.43:0/2286221427 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f83980ccf20 0x7f83980cd550 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7f83980ca9b0 tx=0x7f838802b9a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=76215caa3811ca34 server_cookie=0 in_seq=0 out_seq=0 2023-08-09T13:35:08.106 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:08.103+0000 7f839c843640 1 -- 172.21.15.43:0/2286221427 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8388030860 con 0x7f83980ccf20 2023-08-09T13:35:08.106 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:08.103+0000 7f839c843640 1 -- 172.21.15.43:0/2286221427 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f83880309c0 con 0x7f83980ccf20 2023-08-09T13:35:08.106 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:08.103+0000 7f839c843640 1 -- 172.21.15.43:0/2286221427 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8388030d50 con 0x7f83980ccf20 2023-08-09T13:35:08.106 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:08.104+0000 7f839faa9640 1 -- 172.21.15.43:0/2286221427 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f83980ccf20 msgr2=0x7f83980cd550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:35:08.106 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:08.104+0000 7f839faa9640 1 --2- 172.21.15.43:0/2286221427 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f83980ccf20 0x7f83980cd550 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7f83980ca9b0 tx=0x7f838802b9a0 comp rx=0 tx=0).stop 2023-08-09T13:35:08.107 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:08.104+0000 7f839faa9640 1 -- 172.21.15.43:0/2286221427 shutdown_connections 2023-08-09T13:35:08.107 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:08.104+0000 7f839faa9640 1 --2- 172.21.15.43:0/2286221427 >> v2:172.21.15.203:3300/0 conn(0x7f83980cda90 0x7f839815ca10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:35:08.107 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:08.104+0000 7f839faa9640 1 --2- 172.21.15.43:0/2286221427 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f83980ccf20 0x7f83980cd550 unknown :-1 s=CLOSED pgs=265 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:35:08.107 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:08.104+0000 7f839faa9640 1 -- 172.21.15.43:0/2286221427 >> 172.21.15.43:0/2286221427 conn(0x7f83980ce0a0 msgr2=0x7f83980ce480 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-09T13:35:08.107 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:08.104+0000 7f839faa9640 1 -- 172.21.15.43:0/2286221427 shutdown_connections 2023-08-09T13:35:08.108 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:08.104+0000 7f839faa9640 1 -- 172.21.15.43:0/2286221427 wait complete. 2023-08-09T13:35:08.108 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:08.106+0000 7f839faa9640 1 Processor -- start 2023-08-09T13:35:08.108 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:08.106+0000 7f839faa9640 1 -- start start 2023-08-09T13:35:08.109 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:08.108+0000 7f839faa9640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f83980ccf20 0x7f83980c6650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:35:08.110 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:08.108+0000 7f839faa9640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f83980cda90 0x7f839814cef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:35:08.110 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:08.108+0000 7f839faa9640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f839815c630 0x7f839815ef60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:35:08.110 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:08.108+0000 7f839faa9640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f83980c7840 con 0x7f839815c630 2023-08-09T13:35:08.110 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:08.108+0000 7f839faa9640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f839814d430 con 0x7f83980cda90 2023-08-09T13:35:08.110 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:08.108+0000 7f839faa9640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f83981f9b10 con 0x7f83980ccf20 2023-08-09T13:35:08.110 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:08.108+0000 7f839e046640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f839815c630 0x7f839815ef60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:35:08.112 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:08.108+0000 7f839e046640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f839815c630 0x7f839815ef60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:36986/0 (socket says 172.21.15.43:36986) 2023-08-09T13:35:08.112 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:08.108+0000 7f839e046640 1 -- 172.21.15.43:0/309460226 learned_addr learned my addr 172.21.15.43:0/309460226 (peer_addr_for_me v2:172.21.15.43:0/0) 2023-08-09T13:35:08.112 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:08.109+0000 7f839d044640 1 --2- 172.21.15.43:0/309460226 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f83980cda90 0x7f839814cef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:35:08.112 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:08.109+0000 7f839d845640 1 --2- 172.21.15.43:0/309460226 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f83980ccf20 0x7f83980c6650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:35:08.112 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:08.109+0000 7f839e046640 1 -- 172.21.15.43:0/309460226 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f83980ccf20 msgr2=0x7f83980c6650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:35:08.112 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:08.109+0000 7f839e046640 1 --2- 172.21.15.43:0/309460226 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f83980ccf20 0x7f83980c6650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:35:08.113 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:08.109+0000 7f839e046640 1 -- 172.21.15.43:0/309460226 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f83980cda90 msgr2=0x7f839814cef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:35:08.113 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:08.109+0000 7f839e046640 1 --2- 172.21.15.43:0/309460226 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f83980cda90 0x7f839814cef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:35:08.113 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:08.109+0000 7f839e046640 1 -- 172.21.15.43:0/309460226 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f83880094d0 con 0x7f839815c630 2023-08-09T13:35:08.113 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:08.109+0000 7f839e046640 1 --2- 172.21.15.43:0/309460226 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f839815c630 0x7f839815ef60 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7f8390007e10 tx=0x7f839000d4a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-09T13:35:08.113 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:08.109+0000 7f838e7fc640 1 -- 172.21.15.43:0/309460226 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8390003990 con 0x7f839815c630 2023-08-09T13:35:08.113 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:08.110+0000 7f838e7fc640 1 -- 172.21.15.43:0/309460226 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f839000f040 con 0x7f839815c630 2023-08-09T13:35:08.113 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:08.110+0000 7f838e7fc640 1 -- 172.21.15.43:0/309460226 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8390003d10 con 0x7f839815c630 2023-08-09T13:35:08.114 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:08.110+0000 7f839faa9640 1 -- 172.21.15.43:0/309460226 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f83980c6c30 con 0x7f839815c630 2023-08-09T13:35:08.114 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:08.110+0000 7f839faa9640 1 -- 172.21.15.43:0/309460226 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f83980c71f0 con 0x7f839815c630 2023-08-09T13:35:08.114 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:08.111+0000 7f839faa9640 1 -- 172.21.15.43:0/309460226 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f83980739f0 con 0x7f839815c630 2023-08-09T13:35:08.115 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:08.112+0000 7f838e7fc640 1 -- 172.21.15.43:0/309460226 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 5) v1 ==== 88341+0+0 (secure 0 0 0) 0x7f8390004030 con 0x7f839815c630 2023-08-09T13:35:08.115 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:08.113+0000 7f838e7fc640 1 --2- 172.21.15.43:0/309460226 >> [v2:172.21.15.203:6832/830097474,v1:172.21.15.203:6833/830097474] conn(0x7f837c06a050 0x7f837c06c510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:35:08.115 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:08.113+0000 7f838e7fc640 1 -- 172.21.15.43:0/309460226 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 6871+0+0 (secure 0 0 0) 0x7f8390088b70 con 0x7f839815c630 2023-08-09T13:35:08.115 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:08.114+0000 7f839d845640 1 --2- 172.21.15.43:0/309460226 >> [v2:172.21.15.203:6832/830097474,v1:172.21.15.203:6833/830097474] conn(0x7f837c06a050 0x7f837c06c510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:35:08.116 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:08.115+0000 7f839d845640 1 --2- 172.21.15.43:0/309460226 >> [v2:172.21.15.203:6832/830097474,v1:172.21.15.203:6833/830097474] conn(0x7f837c06a050 0x7f837c06c510 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7f8394000f60 tx=0x7f8394009040 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-09T13:35:08.118 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:08.117+0000 7f838e7fc640 1 -- 172.21.15.43:0/309460226 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178066 (secure 0 0 0) 0x7f8390053f20 con 0x7f839815c630 2023-08-09T13:35:08.312 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:08.311+0000 7f839faa9640 1 -- 172.21.15.43:0/309460226 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f8398067be0 con 0x7f839815c630 2023-08-09T13:35:08.313 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:08.311+0000 7f838e7fc640 1 -- 172.21.15.43:0/309460226 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v26) v1 ==== 104+0+11 (secure 0 0 0) 0x7f8390057540 con 0x7f839815c630 2023-08-09T13:35:08.313 INFO:teuthology.orchestra.run.smithi043.stdout:pg_num: 32 2023-08-09T13:35:08.316 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:08.314+0000 7f839faa9640 1 -- 172.21.15.43:0/309460226 >> [v2:172.21.15.203:6832/830097474,v1:172.21.15.203:6833/830097474] conn(0x7f837c06a050 msgr2=0x7f837c06c510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:35:08.316 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:08.314+0000 7f839faa9640 1 --2- 172.21.15.43:0/309460226 >> [v2:172.21.15.203:6832/830097474,v1:172.21.15.203:6833/830097474] conn(0x7f837c06a050 0x7f837c06c510 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7f8394000f60 tx=0x7f8394009040 comp rx=0 tx=0).stop 2023-08-09T13:35:08.316 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:08.314+0000 7f839faa9640 1 -- 172.21.15.43:0/309460226 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f839815c630 msgr2=0x7f839815ef60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:35:08.316 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:08.314+0000 7f839faa9640 1 --2- 172.21.15.43:0/309460226 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f839815c630 0x7f839815ef60 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7f8390007e10 tx=0x7f839000d4a0 comp rx=0 tx=0).stop 2023-08-09T13:35:08.316 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:08.314+0000 7f839faa9640 1 -- 172.21.15.43:0/309460226 shutdown_connections 2023-08-09T13:35:08.317 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:08.314+0000 7f839faa9640 1 --2- 172.21.15.43:0/309460226 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f839815c630 0x7f839815ef60 unknown :-1 s=CLOSED pgs=266 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:35:08.317 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:08.314+0000 7f839faa9640 1 --2- 172.21.15.43:0/309460226 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f83980cda90 0x7f839814cef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:35:08.317 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:08.314+0000 7f839faa9640 1 --2- 172.21.15.43:0/309460226 >> [v2:172.21.15.203:6832/830097474,v1:172.21.15.203:6833/830097474] conn(0x7f837c06a050 0x7f837c06c510 unknown :-1 s=CLOSED pgs=149 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:35:08.317 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:08.314+0000 7f839faa9640 1 --2- 172.21.15.43:0/309460226 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f83980ccf20 0x7f83980c6650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:35:08.317 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:08.314+0000 7f839faa9640 1 -- 172.21.15.43:0/309460226 >> 172.21.15.43:0/309460226 conn(0x7f83980ce0a0 msgr2=0x7f839815ca10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-09T13:35:08.317 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:08.314+0000 7f839faa9640 1 -- 172.21.15.43:0/309460226 shutdown_connections 2023-08-09T13:35:08.317 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:08.314+0000 7f839faa9640 1 -- 172.21.15.43:0/309460226 wait complete. 2023-08-09T13:35:08.333 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-08-09T13:35:08.568 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:08.567+0000 7f5c5822d640 1 Processor -- start 2023-08-09T13:35:08.569 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:08.567+0000 7f5c5822d640 1 -- start start 2023-08-09T13:35:08.570 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:08.568+0000 7f5c5822d640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f5c5015dae0 0x7f5c500cabb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:35:08.570 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:08.569+0000 7f5c5822d640 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7f5c5015e940 0x7f5c50070600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:35:08.570 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:08.569+0000 7f5c5822d640 1 -- --> v1:172.21.15.43:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5c5015edb0 con 0x7f5c500cb220 2023-08-09T13:35:08.570 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:08.569+0000 7f5c5822d640 1 -- --> v2:172.21.15.43:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5c500c8f90 con 0x7f5c5015dae0 2023-08-09T13:35:08.571 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:08.569+0000 7f5c5822d640 1 -- --> v2:172.21.15.203:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5c500c9270 con 0x7f5c5015e940 2023-08-09T13:35:08.571 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:08.569+0000 7f5c567ca640 1 --1- >> v1:172.21.15.43:6789/0 conn(0x7f5c500cb220 0x7f5c500b1660 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.43:6789/0 says I am v1:172.21.15.43:57526/0 (socket says 172.21.15.43:57526) 2023-08-09T13:35:08.572 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:08.569+0000 7f5c567ca640 1 -- 172.21.15.43:0/479150966 learned_addr learned my addr 172.21.15.43:0/479150966 (peer_addr_for_me v1:172.21.15.43:0/0) 2023-08-09T13:35:08.572 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:08.569+0000 7f5c55fc9640 1 --2- 172.21.15.43:0/479150966 >> v2:172.21.15.43:3300/0 conn(0x7f5c5015dae0 0x7f5c500cabb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:35:08.572 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:08.569+0000 7f5c557c8640 1 --2- 172.21.15.43:0/479150966 >> v2:172.21.15.203:3300/0 conn(0x7f5c5015e940 0x7f5c50070600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:35:08.572 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:08.569+0000 7f5c54fc7640 1 -- 172.21.15.43:0/479150966 <== mon.0 v1:172.21.15.43:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3664208981 0 0) 0x7f5c5015edb0 con 0x7f5c500cb220 2023-08-09T13:35:08.572 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:08.569+0000 7f5c54fc7640 1 -- 172.21.15.43:0/479150966 --> v1:172.21.15.43:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f5c3c003520 con 0x7f5c500cb220 2023-08-09T13:35:08.573 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:08.570+0000 7f5c55fc9640 1 -- 172.21.15.43:0/479150966 >> v2:172.21.15.203:3300/0 conn(0x7f5c5015e940 msgr2=0x7f5c50070600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-09T13:35:08.573 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:08.570+0000 7f5c55fc9640 1 --2- 172.21.15.43:0/479150966 >> v2:172.21.15.203:3300/0 conn(0x7f5c5015e940 0x7f5c50070600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:35:08.573 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:08.570+0000 7f5c55fc9640 1 -- 172.21.15.43:0/479150966 >> v1:172.21.15.43:6789/0 conn(0x7f5c500cb220 legacy=0x7f5c500b1660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:35:08.573 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:08.570+0000 7f5c55fc9640 1 -- 172.21.15.43:0/479150966 --> v2:172.21.15.43:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5c5006fe00 con 0x7f5c5015dae0 2023-08-09T13:35:08.574 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:08.571+0000 7f5c55fc9640 1 --2- 172.21.15.43:0/479150966 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f5c5015dae0 0x7f5c500cabb0 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7f5c5006fcf0 tx=0x7f5c4c02b840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ce7d61f1d9608a40 server_cookie=0 in_seq=0 out_seq=0 2023-08-09T13:35:08.575 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:08.571+0000 7f5c54fc7640 1 -- 172.21.15.43:0/479150966 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5c4c02e070 con 0x7f5c5015dae0 2023-08-09T13:35:08.575 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:08.571+0000 7f5c54fc7640 1 -- 172.21.15.43:0/479150966 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5c4c0315a0 con 0x7f5c5015dae0 2023-08-09T13:35:08.575 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:08.571+0000 7f5c54fc7640 1 -- 172.21.15.43:0/479150966 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5c4c031910 con 0x7f5c5015dae0 2023-08-09T13:35:08.576 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:08.572+0000 7f5c5822d640 1 -- 172.21.15.43:0/479150966 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f5c5015dae0 msgr2=0x7f5c500cabb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:35:08.576 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:08.572+0000 7f5c5822d640 1 --2- 172.21.15.43:0/479150966 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f5c5015dae0 0x7f5c500cabb0 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7f5c5006fcf0 tx=0x7f5c4c02b840 comp rx=0 tx=0).stop 2023-08-09T13:35:08.576 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:08.572+0000 7f5c5822d640 1 -- 172.21.15.43:0/479150966 shutdown_connections 2023-08-09T13:35:08.576 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:08.572+0000 7f5c5822d640 1 --2- 172.21.15.43:0/479150966 >> v2:172.21.15.203:3300/0 conn(0x7f5c5015e940 0x7f5c50070600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:35:08.576 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:08.572+0000 7f5c5822d640 1 --2- 172.21.15.43:0/479150966 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f5c5015dae0 0x7f5c500cabb0 unknown :-1 s=CLOSED pgs=268 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:35:08.577 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:08.572+0000 7f5c5822d640 1 -- 172.21.15.43:0/479150966 >> 172.21.15.43:0/479150966 conn(0x7f5c500b0360 msgr2=0x7f5c500b0740 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-09T13:35:08.577 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:08.572+0000 7f5c5822d640 1 -- 172.21.15.43:0/479150966 shutdown_connections 2023-08-09T13:35:08.577 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:08.572+0000 7f5c5822d640 1 -- 172.21.15.43:0/479150966 wait complete. 2023-08-09T13:35:08.580 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:08.574+0000 7f5c5822d640 1 Processor -- start 2023-08-09T13:35:08.580 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:08.574+0000 7f5c5822d640 1 -- start start 2023-08-09T13:35:08.580 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:08.575+0000 7f5c5822d640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f5c500cb220 0x7f5c500c68f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:35:08.580 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:08.575+0000 7f5c5822d640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f5c5015dae0 0x7f5c501596d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:35:08.580 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:08.575+0000 7f5c5822d640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f5c5015e940 0x7f5c500c7210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:35:08.581 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:08.575+0000 7f5c5822d640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5c500c64d0 con 0x7f5c5015e940 2023-08-09T13:35:08.581 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:08.575+0000 7f5c5822d640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5c500c6e30 con 0x7f5c500cb220 2023-08-09T13:35:08.581 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:08.575+0000 7f5c5822d640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5c500c58e0 con 0x7f5c5015dae0 2023-08-09T13:35:08.581 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:08.575+0000 7f5c567ca640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f5c5015e940 0x7f5c500c7210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:35:08.581 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:08.575+0000 7f5c567ca640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f5c5015e940 0x7f5c500c7210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:37002/0 (socket says 172.21.15.43:37002) 2023-08-09T13:35:08.582 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:08.575+0000 7f5c567ca640 1 -- 172.21.15.43:0/3769379607 learned_addr learned my addr 172.21.15.43:0/3769379607 (peer_addr_for_me v2:172.21.15.43:0/0) 2023-08-09T13:35:08.582 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:08.576+0000 7f5c55fc9640 1 --2- 172.21.15.43:0/3769379607 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f5c500cb220 0x7f5c500c68f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:35:08.582 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:08.576+0000 7f5c557c8640 1 --2- 172.21.15.43:0/3769379607 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f5c5015dae0 0x7f5c501596d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:35:08.582 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:08.576+0000 7f5c567ca640 1 -- 172.21.15.43:0/3769379607 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f5c5015dae0 msgr2=0x7f5c501596d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:35:08.582 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:08.576+0000 7f5c567ca640 1 --2- 172.21.15.43:0/3769379607 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f5c5015dae0 0x7f5c501596d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:35:08.582 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:08.576+0000 7f5c567ca640 1 -- 172.21.15.43:0/3769379607 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f5c500cb220 msgr2=0x7f5c500c68f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:35:08.582 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:08.576+0000 7f5c567ca640 1 --2- 172.21.15.43:0/3769379607 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f5c500cb220 0x7f5c500c68f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:35:08.583 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:08.576+0000 7f5c567ca640 1 -- 172.21.15.43:0/3769379607 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5c4c009400 con 0x7f5c5015e940 2023-08-09T13:35:08.583 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:08.576+0000 7f5c55fc9640 1 --2- 172.21.15.43:0/3769379607 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f5c500cb220 0x7f5c500c68f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-08-09T13:35:08.583 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:08.576+0000 7f5c567ca640 1 --2- 172.21.15.43:0/3769379607 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f5c5015e940 0x7f5c500c7210 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7f5c4800cb40 tx=0x7f5c4800a060 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-09T13:35:08.583 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:08.577+0000 7f5c467fc640 1 -- 172.21.15.43:0/3769379607 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5c4800aca0 con 0x7f5c5015e940 2023-08-09T13:35:08.584 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:08.577+0000 7f5c467fc640 1 -- 172.21.15.43:0/3769379607 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5c4800ae00 con 0x7f5c5015e940 2023-08-09T13:35:08.584 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:08.577+0000 7f5c5822d640 1 -- 172.21.15.43:0/3769379607 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5c500c5b30 con 0x7f5c5015e940 2023-08-09T13:35:08.584 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:08.577+0000 7f5c467fc640 1 -- 172.21.15.43:0/3769379607 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5c48012620 con 0x7f5c5015e940 2023-08-09T13:35:08.584 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:08.577+0000 7f5c5822d640 1 -- 172.21.15.43:0/3769379607 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5c500c60f0 con 0x7f5c5015e940 2023-08-09T13:35:08.584 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:08.578+0000 7f5c5822d640 1 -- 172.21.15.43:0/3769379607 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5c500c6610 con 0x7f5c5015e940 2023-08-09T13:35:08.586 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:08.579+0000 7f5c467fc640 1 -- 172.21.15.43:0/3769379607 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 5) v1 ==== 88341+0+0 (secure 0 0 0) 0x7f5c48007070 con 0x7f5c5015e940 2023-08-09T13:35:08.586 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:08.580+0000 7f5c467fc640 1 --2- 172.21.15.43:0/3769379607 >> [v2:172.21.15.203:6832/830097474,v1:172.21.15.203:6833/830097474] conn(0x7f5c3406a050 0x7f5c3406c510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:35:08.586 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:08.580+0000 7f5c467fc640 1 -- 172.21.15.43:0/3769379607 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 6871+0+0 (secure 0 0 0) 0x7f5c48088970 con 0x7f5c5015e940 2023-08-09T13:35:08.586 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:08.581+0000 7f5c55fc9640 1 --2- 172.21.15.43:0/3769379607 >> [v2:172.21.15.203:6832/830097474,v1:172.21.15.203:6833/830097474] conn(0x7f5c3406a050 0x7f5c3406c510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:35:08.586 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:08.582+0000 7f5c55fc9640 1 --2- 172.21.15.43:0/3769379607 >> [v2:172.21.15.203:6832/830097474,v1:172.21.15.203:6833/830097474] conn(0x7f5c3406a050 0x7f5c3406c510 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7f5c4c036000 tx=0x7f5c4c03d040 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-09T13:35:08.587 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:08.584+0000 7f5c467fc640 1 -- 172.21.15.43:0/3769379607 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178066 (secure 0 0 0) 0x7f5c500c6610 con 0x7f5c5015e940 2023-08-09T13:35:08.782 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:08.780+0000 7f5c5822d640 1 -- 172.21.15.43:0/3769379607 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f5c500c6610 con 0x7f5c5015e940 2023-08-09T13:35:08.783 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:08.781+0000 7f5c467fc640 1 -- 172.21.15.43:0/3769379607 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v26) v1 ==== 100+0+11 (secure 0 0 0) 0x7f5c500c6610 con 0x7f5c5015e940 2023-08-09T13:35:08.783 INFO:teuthology.orchestra.run.smithi043.stdout:pg_num: 64 2023-08-09T13:35:08.785 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:08.784+0000 7f5c5822d640 1 -- 172.21.15.43:0/3769379607 >> [v2:172.21.15.203:6832/830097474,v1:172.21.15.203:6833/830097474] conn(0x7f5c3406a050 msgr2=0x7f5c3406c510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:35:08.785 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:08.784+0000 7f5c5822d640 1 --2- 172.21.15.43:0/3769379607 >> [v2:172.21.15.203:6832/830097474,v1:172.21.15.203:6833/830097474] conn(0x7f5c3406a050 0x7f5c3406c510 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7f5c4c036000 tx=0x7f5c4c03d040 comp rx=0 tx=0).stop 2023-08-09T13:35:08.785 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:08.784+0000 7f5c5822d640 1 -- 172.21.15.43:0/3769379607 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f5c5015e940 msgr2=0x7f5c500c7210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:35:08.786 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:08.784+0000 7f5c5822d640 1 --2- 172.21.15.43:0/3769379607 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f5c5015e940 0x7f5c500c7210 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7f5c4800cb40 tx=0x7f5c4800a060 comp rx=0 tx=0).stop 2023-08-09T13:35:08.786 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:08.784+0000 7f5c5822d640 1 -- 172.21.15.43:0/3769379607 shutdown_connections 2023-08-09T13:35:08.787 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:08.784+0000 7f5c5822d640 1 --2- 172.21.15.43:0/3769379607 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f5c5015e940 0x7f5c500c7210 unknown :-1 s=CLOSED pgs=269 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:35:08.787 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:08.784+0000 7f5c5822d640 1 --2- 172.21.15.43:0/3769379607 >> [v2:172.21.15.203:6832/830097474,v1:172.21.15.203:6833/830097474] conn(0x7f5c3406a050 0x7f5c3406c510 unknown :-1 s=CLOSED pgs=150 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:35:08.787 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:08.784+0000 7f5c5822d640 1 --2- 172.21.15.43:0/3769379607 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f5c5015dae0 0x7f5c501596d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:35:08.787 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:08.784+0000 7f5c5822d640 1 --2- 172.21.15.43:0/3769379607 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f5c500cb220 0x7f5c500c68f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:35:08.787 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:08.784+0000 7f5c5822d640 1 -- 172.21.15.43:0/3769379607 >> 172.21.15.43:0/3769379607 conn(0x7f5c500b0360 msgr2=0x7f5c5015cb10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-09T13:35:08.787 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:08.784+0000 7f5c5822d640 1 -- 172.21.15.43:0/3769379607 shutdown_connections 2023-08-09T13:35:08.787 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:08.784+0000 7f5c5822d640 1 -- 172.21.15.43:0/3769379607 wait complete. 2023-08-09T13:35:08.805 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2023-08-09T13:35:09.035 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:09.034+0000 7fa96ffff640 1 Processor -- start 2023-08-09T13:35:09.036 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:09.034+0000 7fa96ffff640 1 -- start start 2023-08-09T13:35:09.037 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:09.036+0000 7fa96ffff640 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7fa970158f10 0x7fa97006a8d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:35:09.037 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:09.036+0000 7fa96ffff640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fa97006ae10 0x7fa97015d4a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:35:09.038 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:09.036+0000 7fa96ffff640 1 -- --> v1:172.21.15.43:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa97015a150 con 0x7fa9701592f0 2023-08-09T13:35:09.038 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:09.036+0000 7fa96ffff640 1 -- --> v2:172.21.15.203:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa97006b2c0 con 0x7fa970158f10 2023-08-09T13:35:09.038 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:09.036+0000 7fa96ffff640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa970159820 con 0x7fa97006ae10 2023-08-09T13:35:09.039 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:09.036+0000 7fa96e7fc640 1 --1- >> v1:172.21.15.43:6789/0 conn(0x7fa9701592f0 0x7fa97015cd90 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.43:6789/0 says I am v1:172.21.15.43:57536/0 (socket says 172.21.15.43:57536) 2023-08-09T13:35:09.039 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:09.036+0000 7fa96e7fc640 1 -- 172.21.15.43:0/3184977288 learned_addr learned my addr 172.21.15.43:0/3184977288 (peer_addr_for_me v1:172.21.15.43:0/0) 2023-08-09T13:35:09.039 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:09.037+0000 7fa96f7fe640 1 --2- 172.21.15.43:0/3184977288 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fa97006ae10 0x7fa97015d4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:35:09.039 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:09.037+0000 7fa96effd640 1 --2- 172.21.15.43:0/3184977288 >> v2:172.21.15.203:3300/0 conn(0x7fa970158f10 0x7fa97006a8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:35:09.040 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:09.037+0000 7fa96dffb640 1 -- 172.21.15.43:0/3184977288 <== mon.0 v1:172.21.15.43:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2199780036 0 0) 0x7fa97015a150 con 0x7fa9701592f0 2023-08-09T13:35:09.040 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:09.037+0000 7fa96dffb640 1 -- 172.21.15.43:0/3184977288 --> v1:172.21.15.43:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fa958003520 con 0x7fa9701592f0 2023-08-09T13:35:09.040 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:09.037+0000 7fa96dffb640 1 -- 172.21.15.43:0/3184977288 <== mon.0 v1:172.21.15.43:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 189826471 0 0) 0x7fa958003520 con 0x7fa9701592f0 2023-08-09T13:35:09.040 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:09.037+0000 7fa96dffb640 1 -- 172.21.15.43:0/3184977288 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fa97006ae10 msgr2=0x7fa97015d4a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-09T13:35:09.040 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:09.037+0000 7fa96dffb640 1 --2- 172.21.15.43:0/3184977288 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fa97006ae10 0x7fa97015d4a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:35:09.040 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:09.037+0000 7fa96dffb640 1 -- 172.21.15.43:0/3184977288 >> v2:172.21.15.203:3300/0 conn(0x7fa970158f10 msgr2=0x7fa97006a8d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-09T13:35:09.040 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:09.037+0000 7fa96dffb640 1 --2- 172.21.15.43:0/3184977288 >> v2:172.21.15.203:3300/0 conn(0x7fa970158f10 0x7fa97006a8d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:35:09.041 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:09.037+0000 7fa96dffb640 1 -- 172.21.15.43:0/3184977288 --> v1:172.21.15.43:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa970159b00 con 0x7fa9701592f0 2023-08-09T13:35:09.041 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:09.037+0000 7fa96dffb640 1 -- 172.21.15.43:0/3184977288 <== mon.0 v1:172.21.15.43:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1285150740 0 0) 0x7fa9640019f0 con 0x7fa9701592f0 2023-08-09T13:35:09.041 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:09.037+0000 7fa96dffb640 1 -- 172.21.15.43:0/3184977288 >> v1:172.21.15.43:6789/0 conn(0x7fa9701592f0 legacy=0x7fa97015cd90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:35:09.041 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:09.037+0000 7fa96dffb640 1 --2- 172.21.15.43:0/3184977288 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fa97006ae10 0x7fa958003970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:35:09.041 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:09.037+0000 7fa96dffb640 1 --2- 172.21.15.43:0/3184977288 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa958003ef0 0x7fa9580047c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:35:09.041 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:09.037+0000 7fa96dffb640 1 --2- 172.21.15.43:0/3184977288 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fa958004d00 0x7fa958005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:35:09.041 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:09.037+0000 7fa96dffb640 1 -- 172.21.15.43:0/3184977288 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa97006b2c0 con 0x7fa958003ef0 2023-08-09T13:35:09.041 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:09.037+0000 7fa96dffb640 1 -- 172.21.15.43:0/3184977288 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa970159820 con 0x7fa958004d00 2023-08-09T13:35:09.042 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:09.037+0000 7fa96dffb640 1 -- 172.21.15.43:0/3184977288 --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa95800db90 con 0x7fa97006ae10 2023-08-09T13:35:09.042 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:09.038+0000 7fa96e7fc640 1 --2- 172.21.15.43:0/3184977288 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa958003ef0 0x7fa9580047c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:35:09.043 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:09.038+0000 7fa96effd640 1 --2- 172.21.15.43:0/3184977288 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fa97006ae10 0x7fa958003970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:35:09.043 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:09.038+0000 7fa96f7fe640 1 --2- 172.21.15.43:0/3184977288 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fa958004d00 0x7fa958005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:35:09.043 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:09.038+0000 7fa96e7fc640 1 -- 172.21.15.43:0/3184977288 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fa97006ae10 msgr2=0x7fa958003970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-09T13:35:09.043 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:09.038+0000 7fa96e7fc640 1 --2- 172.21.15.43:0/3184977288 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fa97006ae10 0x7fa958003970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:35:09.044 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:09.038+0000 7fa96e7fc640 1 -- 172.21.15.43:0/3184977288 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fa958004d00 msgr2=0x7fa958005180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-09T13:35:09.044 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:09.038+0000 7fa96e7fc640 1 --2- 172.21.15.43:0/3184977288 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fa958004d00 0x7fa958005180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:35:09.044 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:09.038+0000 7fa96e7fc640 1 -- 172.21.15.43:0/3184977288 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa958003520 con 0x7fa958003ef0 2023-08-09T13:35:09.044 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:09.040+0000 7fa96e7fc640 1 --2- 172.21.15.43:0/3184977288 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa958003ef0 0x7fa9580047c0 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7fa96400a830 tx=0x7fa9580057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=15cf9aeb0a4960f4 server_cookie=0 in_seq=0 out_seq=0 2023-08-09T13:35:09.044 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:09.040+0000 7fa96dffb640 1 -- 172.21.15.43:0/3184977288 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa96402cea0 con 0x7fa958003ef0 2023-08-09T13:35:09.044 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:09.040+0000 7fa96dffb640 1 -- 172.21.15.43:0/3184977288 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa96402f070 con 0x7fa958003ef0 2023-08-09T13:35:09.044 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:09.040+0000 7fa96dffb640 1 -- 172.21.15.43:0/3184977288 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa9640348d0 con 0x7fa958003ef0 2023-08-09T13:35:09.045 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:09.041+0000 7fa96ffff640 1 -- 172.21.15.43:0/3184977288 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa958003ef0 msgr2=0x7fa9580047c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:35:09.045 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:09.041+0000 7fa96ffff640 1 --2- 172.21.15.43:0/3184977288 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa958003ef0 0x7fa9580047c0 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7fa96400a830 tx=0x7fa9580057f0 comp rx=0 tx=0).stop 2023-08-09T13:35:09.045 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:09.041+0000 7fa96f7fe640 1 -- 172.21.15.43:0/3184977288 reap_dead start 2023-08-09T13:35:09.045 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:09.041+0000 7fa96ffff640 1 -- 172.21.15.43:0/3184977288 shutdown_connections 2023-08-09T13:35:09.045 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:09.041+0000 7fa96ffff640 1 -- 172.21.15.43:0/3184977288 >> 172.21.15.43:0/3184977288 conn(0x7fa97006ce10 msgr2=0x7fa9700cef70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-09T13:35:09.045 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:09.041+0000 7fa96ffff640 1 -- 172.21.15.43:0/3184977288 shutdown_connections 2023-08-09T13:35:09.045 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:09.041+0000 7fa96ffff640 1 -- 172.21.15.43:0/3184977288 wait complete. 2023-08-09T13:35:09.047 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:09.043+0000 7fa96ffff640 1 Processor -- start 2023-08-09T13:35:09.047 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:09.043+0000 7fa96ffff640 1 -- start start 2023-08-09T13:35:09.047 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:09.044+0000 7fa96ffff640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa958003ef0 0x7fa9700b27e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:35:09.047 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:09.044+0000 7fa96ffff640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fa9700b2d20 0x7fa9700b0bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:35:09.047 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:09.044+0000 7fa96ffff640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fa9700b1130 0x7fa97015b830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:35:09.047 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:09.044+0000 7fa96ffff640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa9700b16d0 con 0x7fa958003ef0 2023-08-09T13:35:09.047 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:09.044+0000 7fa96ffff640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa9700b19b0 con 0x7fa9700b2d20 2023-08-09T13:35:09.048 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:09.044+0000 7fa96ffff640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa9700b1d00 con 0x7fa9700b1130 2023-08-09T13:35:09.048 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:09.044+0000 7fa96effd640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa958003ef0 0x7fa9700b27e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:35:09.048 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:09.044+0000 7fa96effd640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa958003ef0 0x7fa9700b27e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:37012/0 (socket says 172.21.15.43:37012) 2023-08-09T13:35:09.048 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:09.044+0000 7fa96effd640 1 -- 172.21.15.43:0/1227167924 learned_addr learned my addr 172.21.15.43:0/1227167924 (peer_addr_for_me v2:172.21.15.43:0/0) 2023-08-09T13:35:09.048 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:09.044+0000 7fa96effd640 1 -- 172.21.15.43:0/1227167924 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fa9700b1130 msgr2=0x7fa97015b830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:35:09.048 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:09.044+0000 7fa96effd640 1 --2- 172.21.15.43:0/1227167924 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fa9700b1130 0x7fa97015b830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:35:09.048 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:09.044+0000 7fa96effd640 1 -- 172.21.15.43:0/1227167924 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fa9700b2d20 msgr2=0x7fa9700b0bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:35:09.049 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:09.044+0000 7fa96effd640 1 --2- 172.21.15.43:0/1227167924 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fa9700b2d20 0x7fa9700b0bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:35:09.049 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:09.044+0000 7fa96effd640 1 -- 172.21.15.43:0/1227167924 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa9700b1fe0 con 0x7fa958003ef0 2023-08-09T13:35:09.049 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:09.045+0000 7fa96effd640 1 --2- 172.21.15.43:0/1227167924 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa958003ef0 0x7fa9700b27e0 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7fa9600116d0 tx=0x7fa960011b90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-09T13:35:09.049 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:09.045+0000 7fa9577fe640 1 -- 172.21.15.43:0/1227167924 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa96000f940 con 0x7fa958003ef0 2023-08-09T13:35:09.049 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:09.045+0000 7fa9577fe640 1 -- 172.21.15.43:0/1227167924 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa96000faa0 con 0x7fa958003ef0 2023-08-09T13:35:09.049 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:09.045+0000 7fa9577fe640 1 -- 172.21.15.43:0/1227167924 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa96000fe10 con 0x7fa958003ef0 2023-08-09T13:35:09.049 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:09.045+0000 7fa96ffff640 1 -- 172.21.15.43:0/1227167924 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa9700b22c0 con 0x7fa958003ef0 2023-08-09T13:35:09.050 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:09.045+0000 7fa96ffff640 1 -- 172.21.15.43:0/1227167924 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa970152210 con 0x7fa958003ef0 2023-08-09T13:35:09.051 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:09.046+0000 7fa96ffff640 1 -- 172.21.15.43:0/1227167924 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa97004e390 con 0x7fa958003ef0 2023-08-09T13:35:09.051 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:09.047+0000 7fa9577fe640 1 -- 172.21.15.43:0/1227167924 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 5) v1 ==== 88341+0+0 (secure 0 0 0) 0x7fa960015460 con 0x7fa958003ef0 2023-08-09T13:35:09.051 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:09.048+0000 7fa9577fe640 1 --2- 172.21.15.43:0/1227167924 >> [v2:172.21.15.203:6832/830097474,v1:172.21.15.203:6833/830097474] conn(0x7fa95c06a080 0x7fa95c06c540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:35:09.051 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:09.048+0000 7fa9577fe640 1 -- 172.21.15.43:0/1227167924 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 6871+0+0 (secure 0 0 0) 0x7fa960088db0 con 0x7fa958003ef0 2023-08-09T13:35:09.052 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:09.049+0000 7fa96e7fc640 1 --2- 172.21.15.43:0/1227167924 >> [v2:172.21.15.203:6832/830097474,v1:172.21.15.203:6833/830097474] conn(0x7fa95c06a080 0x7fa95c06c540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:35:09.052 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:09.050+0000 7fa96e7fc640 1 --2- 172.21.15.43:0/1227167924 >> [v2:172.21.15.203:6832/830097474,v1:172.21.15.203:6833/830097474] conn(0x7fa95c06a080 0x7fa95c06c540 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7fa964034000 tx=0x7fa964007000 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-09T13:35:09.054 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:09.052+0000 7fa9577fe640 1 -- 172.21.15.43:0/1227167924 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178066 (secure 0 0 0) 0x7fa96008d070 con 0x7fa958003ef0 2023-08-09T13:35:09.247 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:09.245+0000 7fa96ffff640 1 -- 172.21.15.43:0/1227167924 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) v1 -- 0x7fa97006ecd0 con 0x7fa958003ef0 2023-08-09T13:35:09.247 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:09.246+0000 7fa9577fe640 1 -- 172.21.15.43:0/1227167924 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v26) v1 ==== 103+0+11 (secure 0 0 0) 0x7fa960057ca0 con 0x7fa958003ef0 2023-08-09T13:35:09.248 INFO:teuthology.orchestra.run.smithi043.stdout:pg_num: 32 2023-08-09T13:35:09.250 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:09.248+0000 7fa96ffff640 1 -- 172.21.15.43:0/1227167924 >> [v2:172.21.15.203:6832/830097474,v1:172.21.15.203:6833/830097474] conn(0x7fa95c06a080 msgr2=0x7fa95c06c540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:35:09.250 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:09.249+0000 7fa96ffff640 1 --2- 172.21.15.43:0/1227167924 >> [v2:172.21.15.203:6832/830097474,v1:172.21.15.203:6833/830097474] conn(0x7fa95c06a080 0x7fa95c06c540 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7fa964034000 tx=0x7fa964007000 comp rx=0 tx=0).stop 2023-08-09T13:35:09.251 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:09.249+0000 7fa96ffff640 1 -- 172.21.15.43:0/1227167924 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa958003ef0 msgr2=0x7fa9700b27e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:35:09.251 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:09.249+0000 7fa96ffff640 1 --2- 172.21.15.43:0/1227167924 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa958003ef0 0x7fa9700b27e0 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7fa9600116d0 tx=0x7fa960011b90 comp rx=0 tx=0).stop 2023-08-09T13:35:09.251 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:09.249+0000 7fa96ffff640 1 -- 172.21.15.43:0/1227167924 shutdown_connections 2023-08-09T13:35:09.251 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:09.249+0000 7fa96ffff640 1 --2- 172.21.15.43:0/1227167924 >> [v2:172.21.15.203:6832/830097474,v1:172.21.15.203:6833/830097474] conn(0x7fa95c06a080 0x7fa95c06c540 unknown :-1 s=CLOSED pgs=151 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:35:09.251 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:09.249+0000 7fa96ffff640 1 --2- 172.21.15.43:0/1227167924 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fa9700b1130 0x7fa97015b830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:35:09.251 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:09.249+0000 7fa96ffff640 1 --2- 172.21.15.43:0/1227167924 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fa9700b2d20 0x7fa9700b0bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:35:09.251 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:09.249+0000 7fa96ffff640 1 --2- 172.21.15.43:0/1227167924 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa958003ef0 0x7fa9700b27e0 unknown :-1 s=CLOSED pgs=272 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:35:09.252 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:09.249+0000 7fa96ffff640 1 -- 172.21.15.43:0/1227167924 >> 172.21.15.43:0/1227167924 conn(0x7fa97006ce10 msgr2=0x7fa97006a160 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-09T13:35:09.252 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:09.249+0000 7fa96ffff640 1 -- 172.21.15.43:0/1227167924 shutdown_connections 2023-08-09T13:35:09.252 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:09.249+0000 7fa96ffff640 1 -- 172.21.15.43:0/1227167924 wait complete. 2023-08-09T13:35:09.269 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-08-09T13:35:09.489 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:09.486+0000 7fe133c38640 1 Processor -- start 2023-08-09T13:35:09.489 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:09.486+0000 7fe133c38640 1 -- start start 2023-08-09T13:35:09.490 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:09.488+0000 7fe133c38640 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7fe12c143c20 0x7fe12c06f540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:35:09.490 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:09.488+0000 7fe133c38640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fe12c0c86f0 0x7fe12c0c8b50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:35:09.490 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:09.488+0000 7fe133c38640 1 -- --> v1:172.21.15.43:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe12c0c9090 con 0x7fe12c06c8d0 2023-08-09T13:35:09.491 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:09.488+0000 7fe133c38640 1 -- --> v2:172.21.15.203:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe12c144000 con 0x7fe12c143c20 2023-08-09T13:35:09.491 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:09.488+0000 7fe133c38640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe12c06c440 con 0x7fe12c0c86f0 2023-08-09T13:35:09.491 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:09.488+0000 7fe1321d5640 1 --1- >> v1:172.21.15.43:6789/0 conn(0x7fe12c06c8d0 0x7fe12c06cd60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.43:6789/0 says I am v1:172.21.15.43:57552/0 (socket says 172.21.15.43:57552) 2023-08-09T13:35:09.491 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:09.488+0000 7fe1321d5640 1 -- 172.21.15.43:0/142806719 learned_addr learned my addr 172.21.15.43:0/142806719 (peer_addr_for_me v1:172.21.15.43:0/0) 2023-08-09T13:35:09.491 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:09.489+0000 7fe1319d4640 1 --2- 172.21.15.43:0/142806719 >> v2:172.21.15.203:3300/0 conn(0x7fe12c143c20 0x7fe12c06f540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:35:09.491 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:09.489+0000 7fe1311d3640 1 --2- 172.21.15.43:0/142806719 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fe12c0c86f0 0x7fe12c0c8b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:35:09.491 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:09.489+0000 7fe1309d2640 1 -- 172.21.15.43:0/142806719 <== mon.0 v1:172.21.15.43:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1599535281 0 0) 0x7fe12c0c9090 con 0x7fe12c06c8d0 2023-08-09T13:35:09.492 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:09.489+0000 7fe1309d2640 1 -- 172.21.15.43:0/142806719 --> v1:172.21.15.43:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fe118003520 con 0x7fe12c06c8d0 2023-08-09T13:35:09.493 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:09.489+0000 7fe1309d2640 1 -- 172.21.15.43:0/142806719 <== mon.0 v1:172.21.15.43:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 595759730 0 0) 0x7fe118003520 con 0x7fe12c06c8d0 2023-08-09T13:35:09.493 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:09.489+0000 7fe1309d2640 1 -- 172.21.15.43:0/142806719 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fe12c0c86f0 msgr2=0x7fe12c0c8b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-09T13:35:09.494 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:09.489+0000 7fe1309d2640 1 --2- 172.21.15.43:0/142806719 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fe12c0c86f0 0x7fe12c0c8b50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:35:09.494 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:09.489+0000 7fe1309d2640 1 -- 172.21.15.43:0/142806719 >> v2:172.21.15.203:3300/0 conn(0x7fe12c143c20 msgr2=0x7fe12c06f540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-09T13:35:09.494 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:09.489+0000 7fe1309d2640 1 --2- 172.21.15.43:0/142806719 >> v2:172.21.15.203:3300/0 conn(0x7fe12c143c20 0x7fe12c06f540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:35:09.494 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:09.489+0000 7fe1309d2640 1 -- 172.21.15.43:0/142806719 --> v1:172.21.15.43:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe12c06b830 con 0x7fe12c06c8d0 2023-08-09T13:35:09.494 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:09.489+0000 7fe1309d2640 1 -- 172.21.15.43:0/142806719 <== mon.0 v1:172.21.15.43:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1285150740 0 0) 0x7fe1240019f0 con 0x7fe12c06c8d0 2023-08-09T13:35:09.495 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:09.489+0000 7fe1309d2640 1 -- 172.21.15.43:0/142806719 >> v1:172.21.15.43:6789/0 conn(0x7fe12c06c8d0 legacy=0x7fe12c06cd60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:35:09.495 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:09.489+0000 7fe1309d2640 1 --2- 172.21.15.43:0/142806719 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fe1180039e0 0x7fe118003e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:35:09.495 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:09.489+0000 7fe1309d2640 1 --2- 172.21.15.43:0/142806719 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fe12c0c86f0 0x7fe1180047c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:35:09.495 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:09.490+0000 7fe1309d2640 1 --2- 172.21.15.43:0/142806719 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fe118004d00 0x7fe118005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:35:09.495 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:09.490+0000 7fe1309d2640 1 -- 172.21.15.43:0/142806719 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe12c144000 con 0x7fe118004d00 2023-08-09T13:35:09.495 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:09.490+0000 7fe1309d2640 1 -- 172.21.15.43:0/142806719 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe12c06c440 con 0x7fe1180039e0 2023-08-09T13:35:09.495 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:09.490+0000 7fe1309d2640 1 -- 172.21.15.43:0/142806719 --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe11800ba30 con 0x7fe12c0c86f0 2023-08-09T13:35:09.495 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:09.490+0000 7fe1321d5640 1 --2- 172.21.15.43:0/142806719 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fe118004d00 0x7fe118005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:35:09.496 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:09.490+0000 7fe1319d4640 1 --2- 172.21.15.43:0/142806719 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fe1180039e0 0x7fe118003e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:35:09.496 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:09.490+0000 7fe1311d3640 1 --2- 172.21.15.43:0/142806719 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fe12c0c86f0 0x7fe1180047c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:35:09.496 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:09.490+0000 7fe1321d5640 1 -- 172.21.15.43:0/142806719 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fe12c0c86f0 msgr2=0x7fe1180047c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-09T13:35:09.496 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:09.490+0000 7fe1321d5640 1 --2- 172.21.15.43:0/142806719 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fe12c0c86f0 0x7fe1180047c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:35:09.496 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:09.490+0000 7fe1321d5640 1 -- 172.21.15.43:0/142806719 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fe1180039e0 msgr2=0x7fe118003e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-09T13:35:09.496 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:09.490+0000 7fe1321d5640 1 --2- 172.21.15.43:0/142806719 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fe1180039e0 0x7fe118003e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:35:09.496 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:09.490+0000 7fe1321d5640 1 -- 172.21.15.43:0/142806719 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe118003520 con 0x7fe118004d00 2023-08-09T13:35:09.497 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:09.491+0000 7fe1321d5640 1 --2- 172.21.15.43:0/142806719 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fe118004d00 0x7fe118005180 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7fe12c06d470 tx=0x7fe1180057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=17269b7572b669dd server_cookie=0 in_seq=0 out_seq=0 2023-08-09T13:35:09.497 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:09.492+0000 7fe1309d2640 1 -- 172.21.15.43:0/142806719 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe124034630 con 0x7fe118004d00 2023-08-09T13:35:09.497 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:09.492+0000 7fe1309d2640 1 -- 172.21.15.43:0/142806719 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe12402f070 con 0x7fe118004d00 2023-08-09T13:35:09.497 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:09.492+0000 7fe1309d2640 1 -- 172.21.15.43:0/142806719 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe1240349f0 con 0x7fe118004d00 2023-08-09T13:35:09.499 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:09.492+0000 7fe133c38640 1 -- 172.21.15.43:0/142806719 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fe118004d00 msgr2=0x7fe118005180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:35:09.499 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:09.492+0000 7fe133c38640 1 --2- 172.21.15.43:0/142806719 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fe118004d00 0x7fe118005180 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7fe12c06d470 tx=0x7fe1180057f0 comp rx=0 tx=0).stop 2023-08-09T13:35:09.499 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:09.492+0000 7fe1321d5640 1 -- 172.21.15.43:0/142806719 reap_dead start 2023-08-09T13:35:09.499 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:09.492+0000 7fe133c38640 1 -- 172.21.15.43:0/142806719 shutdown_connections 2023-08-09T13:35:09.499 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:09.492+0000 7fe133c38640 1 -- 172.21.15.43:0/142806719 >> 172.21.15.43:0/142806719 conn(0x7fe12c0ce0a0 msgr2=0x7fe12c0ce480 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-09T13:35:09.499 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:09.492+0000 7fe133c38640 1 -- 172.21.15.43:0/142806719 shutdown_connections 2023-08-09T13:35:09.499 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:09.493+0000 7fe133c38640 1 -- 172.21.15.43:0/142806719 wait complete. 2023-08-09T13:35:09.500 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:09.494+0000 7fe133c38640 1 Processor -- start 2023-08-09T13:35:09.500 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:09.494+0000 7fe133c38640 1 -- start start 2023-08-09T13:35:09.500 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:09.495+0000 7fe133c38640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fe118004d00 0x7fe12c0b28d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:35:09.500 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:09.495+0000 7fe133c38640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fe12c138870 0x7fe12c138cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:35:09.500 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:09.495+0000 7fe133c38640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fe12c0b3220 0x7fe12c0b36a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:35:09.500 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:09.495+0000 7fe133c38640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe12c1e2960 con 0x7fe12c138870 2023-08-09T13:35:09.500 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:09.495+0000 7fe133c38640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe12c139230 con 0x7fe118004d00 2023-08-09T13:35:09.501 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:09.495+0000 7fe133c38640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe12c1e3780 con 0x7fe12c0b3220 2023-08-09T13:35:09.501 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:09.496+0000 7fe1311d3640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fe12c138870 0x7fe12c138cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:35:09.501 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:09.496+0000 7fe1311d3640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fe12c138870 0x7fe12c138cf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:37026/0 (socket says 172.21.15.43:37026) 2023-08-09T13:35:09.501 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:09.496+0000 7fe1311d3640 1 -- 172.21.15.43:0/2080745257 learned_addr learned my addr 172.21.15.43:0/2080745257 (peer_addr_for_me v2:172.21.15.43:0/0) 2023-08-09T13:35:09.501 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:09.496+0000 7fe1321d5640 1 --2- 172.21.15.43:0/2080745257 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fe12c0b3220 0x7fe12c0b36a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:35:09.501 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:09.496+0000 7fe1319d4640 1 --2- 172.21.15.43:0/2080745257 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fe118004d00 0x7fe12c0b28d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:35:09.501 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:09.496+0000 7fe1311d3640 1 -- 172.21.15.43:0/2080745257 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fe12c0b3220 msgr2=0x7fe12c0b36a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:35:09.501 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:09.496+0000 7fe1311d3640 1 --2- 172.21.15.43:0/2080745257 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fe12c0b3220 0x7fe12c0b36a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:35:09.502 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:09.496+0000 7fe1311d3640 1 -- 172.21.15.43:0/2080745257 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fe118004d00 msgr2=0x7fe12c0b28d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:35:09.502 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:09.496+0000 7fe1311d3640 1 --2- 172.21.15.43:0/2080745257 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fe118004d00 0x7fe12c0b28d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:35:09.502 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:09.496+0000 7fe1311d3640 1 -- 172.21.15.43:0/2080745257 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe12c0b3be0 con 0x7fe12c138870 2023-08-09T13:35:09.502 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:09.497+0000 7fe1311d3640 1 --2- 172.21.15.43:0/2080745257 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fe12c138870 0x7fe12c138cf0 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7fe11c011600 tx=0x7fe11c011ac0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-09T13:35:09.502 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:09.497+0000 7fe1227fc640 1 -- 172.21.15.43:0/2080745257 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe11c00f920 con 0x7fe12c138870 2023-08-09T13:35:09.502 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:09.497+0000 7fe1227fc640 1 -- 172.21.15.43:0/2080745257 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe11c00fa80 con 0x7fe12c138870 2023-08-09T13:35:09.502 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:09.497+0000 7fe1227fc640 1 -- 172.21.15.43:0/2080745257 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe11c00fdf0 con 0x7fe12c138870 2023-08-09T13:35:09.503 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:09.497+0000 7fe133c38640 1 -- 172.21.15.43:0/2080745257 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe12c1e2b70 con 0x7fe12c138870 2023-08-09T13:35:09.503 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:09.497+0000 7fe133c38640 1 -- 172.21.15.43:0/2080745257 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe12c1e3130 con 0x7fe12c138870 2023-08-09T13:35:09.503 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:09.498+0000 7fe133c38640 1 -- 172.21.15.43:0/2080745257 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe12c0739f0 con 0x7fe12c138870 2023-08-09T13:35:09.503 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:09.499+0000 7fe1227fc640 1 -- 172.21.15.43:0/2080745257 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 5) v1 ==== 88341+0+0 (secure 0 0 0) 0x7fe11c015460 con 0x7fe12c138870 2023-08-09T13:35:09.504 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:09.501+0000 7fe1227fc640 1 --2- 172.21.15.43:0/2080745257 >> [v2:172.21.15.203:6832/830097474,v1:172.21.15.203:6833/830097474] conn(0x7fe11006a050 0x7fe11006c510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:35:09.504 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:09.501+0000 7fe1227fc640 1 -- 172.21.15.43:0/2080745257 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 6871+0+0 (secure 0 0 0) 0x7fe11c088e90 con 0x7fe12c138870 2023-08-09T13:35:09.504 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:09.502+0000 7fe1319d4640 1 --2- 172.21.15.43:0/2080745257 >> [v2:172.21.15.203:6832/830097474,v1:172.21.15.203:6833/830097474] conn(0x7fe11006a050 0x7fe11006c510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:35:09.504 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:09.503+0000 7fe1319d4640 1 --2- 172.21.15.43:0/2080745257 >> [v2:172.21.15.203:6832/830097474,v1:172.21.15.203:6833/830097474] conn(0x7fe11006a050 0x7fe11006c510 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7fe128009a90 tx=0x7fe128008040 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-09T13:35:09.506 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:09.505+0000 7fe1227fc640 1 -- 172.21.15.43:0/2080745257 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178066 (secure 0 0 0) 0x7fe11c0541b0 con 0x7fe12c138870 2023-08-09T13:35:09.700 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:09.699+0000 7fe133c38640 1 -- 172.21.15.43:0/2080745257 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fe12c071160 con 0x7fe12c138870 2023-08-09T13:35:09.702 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:09.700+0000 7fe1227fc640 1 -- 172.21.15.43:0/2080745257 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v26) v1 ==== 74+0+20146 (secure 0 0 0) 0x7fe11c0577d0 con 0x7fe12c138870 2023-08-09T13:35:09.702 INFO:teuthology.orchestra.run.smithi043.stdout: 2023-08-09T13:35:09.703 INFO:teuthology.orchestra.run.smithi043.stdout:{"epoch":26,"fsid":"daac87ee-1337-4250-a955-aea4e408a39b","created":"2023-08-09T13:33:41.701198+0000","modified":"2023-08-09T13:34:48.711911+0000","last_up_change":"2023-08-09T13:33:53.783453+0000","last_in_change":"2023-08-09T13:33:45.148904+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":5,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":"rbd","create_time":"2023-08-09T13:33:54.365137+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":".mgr","create_time":"2023-08-09T13:33:54.831803+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_metadata","create_time":"2023-08-09T13:34:00.207974+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data","create_time":"2023-08-09T13:34:01.614765+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":5,"pool_name":"cephfs_data_ec","create_time":"2023-08-09T13:34:05.239583+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"24","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"cbd49ecd-e896-4fbd-864e-5c37a829b165","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6800","nonce":502667335},{"type":"v1","addr":"172.21.15.43:6801","nonce":502667335}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6802","nonce":502667335},{"type":"v1","addr":"172.21.15.43:6803","nonce":502667335}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6806","nonce":502667335},{"type":"v1","addr":"172.21.15.43:6807","nonce":502667335}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6804","nonce":502667335},{"type":"v1","addr":"172.21.15.43:6805","nonce":502667335}]},"public_addr":"172.21.15.43:6801/502667335","cluster_addr":"172.21.15.43:6803/502667335","heartbeat_back_addr":"172.21.15.43:6807/502667335","heartbeat_front_addr":"172.21.15.43:6805/502667335","state":["exists","up"]},{"osd":1,"uuid":"75c7e0fe-1ec7-4fe8-a2f2-3661a65b8500","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6808","nonce":3289134610},{"type":"v1","addr":"172.21.15.43:6809","nonce":3289134610}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6810","nonce":3289134610},{"type":"v1","addr":"172.21.15.43:6812","nonce":3289134610}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6821","nonce":3289134610},{"type":"v1","addr":"172.21.15.43:6824","nonce":3289134610}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6815","nonce":3289134610},{"type":"v1","addr":"172.21.15.43:6818","nonce":3289134610}]},"public_addr":"172.21.15.43:6809/3289134610","cluster_addr":"172.21.15.43:6812/3289134610","heartbeat_back_addr":"172.21.15.43:6824/3289134610","heartbeat_front_addr":"172.21.15.43:6818/3289134610","state":["exists","up"]},{"osd":2,"uuid":"bb5c41e9-7f5f-4125-9b01-3bb8bfc2ea83","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6811","nonce":4079709736},{"type":"v1","addr":"172.21.15.43:6814","nonce":4079709736}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6817","nonce":4079709736},{"type":"v1","addr":"172.21.15.43:6820","nonce":4079709736}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6829","nonce":4079709736},{"type":"v1","addr":"172.21.15.43:6831","nonce":4079709736}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6823","nonce":4079709736},{"type":"v1","addr":"172.21.15.43:6826","nonce":4079709736}]},"public_addr":"172.21.15.43:6814/4079709736","cluster_addr":"172.21.15.43:6820/4079709736","heartbeat_back_addr":"172.21.15.43:6831/4079709736","heartbeat_front_addr":"172.21.15.43:6826/4079709736","state":["exists","up"]},{"osd":3,"uuid":"bfe4c8b4-694a-465c-87e6-2b2aab852613","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6813","nonce":1709538759},{"type":"v1","addr":"172.21.15.43:6816","nonce":1709538759}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6819","nonce":1709538759},{"type":"v1","addr":"172.21.15.43:6822","nonce":1709538759}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6828","nonce":1709538759},{"type":"v1","addr":"172.21.15.43:6830","nonce":1709538759}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6825","nonce":1709538759},{"type":"v1","addr":"172.21.15.43:6827","nonce":1709538759}]},"public_addr":"172.21.15.43:6816/1709538759","cluster_addr":"172.21.15.43:6822/1709538759","heartbeat_back_addr":"172.21.15.43:6830/1709538759","heartbeat_front_addr":"172.21.15.43:6827/1709538759","state":["exists","up"]},{"osd":4,"uuid":"fa2e8a77-aba1-4756-853e-0d174a5bdeba","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6800","nonce":4196866481},{"type":"v1","addr":"172.21.15.203:6801","nonce":4196866481}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6802","nonce":4196866481},{"type":"v1","addr":"172.21.15.203:6803","nonce":4196866481}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6806","nonce":4196866481},{"type":"v1","addr":"172.21.15.203:6807","nonce":4196866481}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6804","nonce":4196866481},{"type":"v1","addr":"172.21.15.203:6805","nonce":4196866481}]},"public_addr":"172.21.15.203:6801/4196866481","cluster_addr":"172.21.15.203:6803/4196866481","heartbeat_back_addr":"172.21.15.203:6807/4196866481","heartbeat_front_addr":"172.21.15.203:6805/4196866481","state":["exists","up"]},{"osd":5,"uuid":"81811e4b-4d24-4f51-a82c-f1d700a2a00d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6816","nonce":3059290081},{"type":"v1","addr":"172.21.15.203:6817","nonce":3059290081}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6818","nonce":3059290081},{"type":"v1","addr":"172.21.15.203:6819","nonce":3059290081}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6822","nonce":3059290081},{"type":"v1","addr":"172.21.15.203:6823","nonce":3059290081}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6820","nonce":3059290081},{"type":"v1","addr":"172.21.15.203:6821","nonce":3059290081}]},"public_addr":"172.21.15.203:6817/3059290081","cluster_addr":"172.21.15.203:6819/3059290081","heartbeat_back_addr":"172.21.15.203:6823/3059290081","heartbeat_front_addr":"172.21.15.203:6821/3059290081","state":["exists","up"]},{"osd":6,"uuid":"307bac4d-6a5b-443d-af75-56d695d7674f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6824","nonce":512155569},{"type":"v1","addr":"172.21.15.203:6825","nonce":512155569}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6826","nonce":512155569},{"type":"v1","addr":"172.21.15.203:6827","nonce":512155569}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6830","nonce":512155569},{"type":"v1","addr":"172.21.15.203:6831","nonce":512155569}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6828","nonce":512155569},{"type":"v1","addr":"172.21.15.203:6829","nonce":512155569}]},"public_addr":"172.21.15.203:6825/512155569","cluster_addr":"172.21.15.203:6827/512155569","heartbeat_back_addr":"172.21.15.203:6831/512155569","heartbeat_front_addr":"172.21.15.203:6829/512155569","state":["exists","up"]},{"osd":7,"uuid":"a95dc7ca-9f1b-4295-af70-68a22a2d7d5e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6808","nonce":3331450438},{"type":"v1","addr":"172.21.15.203:6809","nonce":3331450438}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6810","nonce":3331450438},{"type":"v1","addr":"172.21.15.203:6811","nonce":3331450438}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6814","nonce":3331450438},{"type":"v1","addr":"172.21.15.203:6815","nonce":3331450438}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6812","nonce":3331450438},{"type":"v1","addr":"172.21.15.203:6813","nonce":3331450438}]},"public_addr":"172.21.15.203:6809/3331450438","cluster_addr":"172.21.15.203:6811/3331450438","heartbeat_back_addr":"172.21.15.203:6815/3331450438","heartbeat_front_addr":"172.21.15.203:6813/3331450438","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-09T13:33:50.547652+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-09T13:33:50.589469+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-09T13:33:50.561724+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-09T13:33:50.599650+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-09T13:33:50.411855+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-09T13:33:50.219681+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-09T13:33:50.504664+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-09T13:33:50.441248+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"4.14","mappings":[{"from":1,"to":5}]},{"pgid":"4.29","mappings":[{"from":3,"to":2}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-08-09T13:35:09.705 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:09.703+0000 7fe133c38640 1 -- 172.21.15.43:0/2080745257 >> [v2:172.21.15.203:6832/830097474,v1:172.21.15.203:6833/830097474] conn(0x7fe11006a050 msgr2=0x7fe11006c510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:35:09.705 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:09.703+0000 7fe133c38640 1 --2- 172.21.15.43:0/2080745257 >> [v2:172.21.15.203:6832/830097474,v1:172.21.15.203:6833/830097474] conn(0x7fe11006a050 0x7fe11006c510 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7fe128009a90 tx=0x7fe128008040 comp rx=0 tx=0).stop 2023-08-09T13:35:09.705 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:09.703+0000 7fe133c38640 1 -- 172.21.15.43:0/2080745257 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fe12c138870 msgr2=0x7fe12c138cf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:35:09.706 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:09.703+0000 7fe133c38640 1 --2- 172.21.15.43:0/2080745257 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fe12c138870 0x7fe12c138cf0 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7fe11c011600 tx=0x7fe11c011ac0 comp rx=0 tx=0).stop 2023-08-09T13:35:09.706 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:09.704+0000 7fe133c38640 1 -- 172.21.15.43:0/2080745257 shutdown_connections 2023-08-09T13:35:09.706 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:09.704+0000 7fe133c38640 1 --2- 172.21.15.43:0/2080745257 >> [v2:172.21.15.203:6832/830097474,v1:172.21.15.203:6833/830097474] conn(0x7fe11006a050 0x7fe11006c510 unknown :-1 s=CLOSED pgs=152 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:35:09.706 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:09.704+0000 7fe133c38640 1 --2- 172.21.15.43:0/2080745257 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fe12c0b3220 0x7fe12c0b36a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:35:09.706 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:09.704+0000 7fe133c38640 1 --2- 172.21.15.43:0/2080745257 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fe12c138870 0x7fe12c138cf0 unknown :-1 s=CLOSED pgs=275 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:35:09.706 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:09.704+0000 7fe133c38640 1 --2- 172.21.15.43:0/2080745257 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fe118004d00 0x7fe12c0b28d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:35:09.706 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:09.704+0000 7fe133c38640 1 -- 172.21.15.43:0/2080745257 >> 172.21.15.43:0/2080745257 conn(0x7fe12c0ce0a0 msgr2=0x7fe12c0af3a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-09T13:35:09.707 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:09.704+0000 7fe133c38640 1 -- 172.21.15.43:0/2080745257 shutdown_connections 2023-08-09T13:35:09.707 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:09.704+0000 7fe133c38640 1 -- 172.21.15.43:0/2080745257 wait complete. 2023-08-09T13:35:09.724 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': 'rbd', 'create_time': '2023-08-09T13:33:54.365137+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}, 'read_balance': {'score_acting': 2, 'score_stable': 2, 'optimal_score': 1, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': '.mgr', 'create_time': '2023-08-09T13:33:54.831803+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_metadata', 'create_time': '2023-08-09T13:34:00.207974+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data', 'create_time': '2023-08-09T13:34:01.614765+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 2, 'score_stable': 2, 'optimal_score': 1, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 5, 'pool_name': 'cephfs_data_ec', 'create_time': '2023-08-09T13:34:05.239583+0000', 'flags': 5, 'flags_names': 'hashpspool,ec_overwrites', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '24', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2023-08-09T13:35:09.725 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get rbd pg_num 2023-08-09T13:35:09.957 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:09.955+0000 7f4388e37640 1 Processor -- start 2023-08-09T13:35:09.957 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:09.955+0000 7f4388e37640 1 -- start start 2023-08-09T13:35:09.958 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:09.956+0000 7f4388e37640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f4384157ea0 0x7f43840c97d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:35:09.958 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:09.956+0000 7f4388e37640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f438415af10 0x7f43840c9d10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:35:09.958 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:09.957+0000 7f4388e37640 1 -- --> v1:172.21.15.203:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4384158280 con 0x7f43840c90a0 2023-08-09T13:35:09.958 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:09.957+0000 7f4388e37640 1 -- --> v2:172.21.15.43:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f438415b840 con 0x7f4384157ea0 2023-08-09T13:35:09.959 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:09.957+0000 7f4388e37640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f43840b3810 con 0x7f438415af10 2023-08-09T13:35:09.959 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:09.957+0000 7f43837fe640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f4384157ea0 0x7f43840c97d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:35:09.960 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:09.957+0000 7f43837fe640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f4384157ea0 0x7f43840c97d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:37030/0 (socket says 172.21.15.43:37030) 2023-08-09T13:35:09.960 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:09.957+0000 7f43837fe640 1 -- 172.21.15.43:0/4235196341 learned_addr learned my addr 172.21.15.43:0/4235196341 (peer_addr_for_me v2:172.21.15.43:0/0) 2023-08-09T13:35:09.960 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:09.957+0000 7f4383fff640 1 --2- 172.21.15.43:0/4235196341 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f438415af10 0x7f43840c9d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:35:09.960 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:09.958+0000 7f43837fe640 1 -- 172.21.15.43:0/4235196341 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f438415af10 msgr2=0x7f43840c9d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-09T13:35:09.961 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:09.958+0000 7f43837fe640 1 --2- 172.21.15.43:0/4235196341 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f438415af10 0x7f43840c9d10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:35:09.961 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:09.958+0000 7f43837fe640 1 -- 172.21.15.43:0/4235196341 >> v1:172.21.15.203:6789/0 conn(0x7f43840c90a0 legacy=0x7f438415a800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-09T13:35:09.961 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:09.958+0000 7f43837fe640 1 -- 172.21.15.43:0/4235196341 --> v2:172.21.15.43:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f438415b560 con 0x7f4384157ea0 2023-08-09T13:35:09.966 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:09.959+0000 7f43837fe640 1 --2- 172.21.15.43:0/4235196341 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f4384157ea0 0x7f43840c97d0 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7f43740098a0 tx=0x7f437402b860 comp rx=0 tx=0).ready entity=mon.0 client_cookie=747a3efd3993ad80 server_cookie=0 in_seq=0 out_seq=0 2023-08-09T13:35:09.966 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:09.960+0000 7f43827fc640 1 -- 172.21.15.43:0/4235196341 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f43740317c0 con 0x7f4384157ea0 2023-08-09T13:35:09.967 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:09.960+0000 7f43827fc640 1 -- 172.21.15.43:0/4235196341 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4374031920 con 0x7f4384157ea0 2023-08-09T13:35:09.967 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:09.960+0000 7f43827fc640 1 -- 172.21.15.43:0/4235196341 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4374031cb0 con 0x7f4384157ea0 2023-08-09T13:35:09.967 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:09.960+0000 7f4388e37640 1 -- 172.21.15.43:0/4235196341 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f4384157ea0 msgr2=0x7f43840c97d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:35:09.967 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:09.960+0000 7f4388e37640 1 --2- 172.21.15.43:0/4235196341 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f4384157ea0 0x7f43840c97d0 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7f43740098a0 tx=0x7f437402b860 comp rx=0 tx=0).stop 2023-08-09T13:35:09.967 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:09.960+0000 7f4388e37640 1 -- 172.21.15.43:0/4235196341 shutdown_connections 2023-08-09T13:35:09.967 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:09.960+0000 7f4388e37640 1 --2- 172.21.15.43:0/4235196341 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f438415af10 0x7f43840c9d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:35:09.967 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:09.960+0000 7f4388e37640 1 --2- 172.21.15.43:0/4235196341 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f4384157ea0 0x7f43840c97d0 unknown :-1 s=CLOSED pgs=276 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:35:09.968 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:09.960+0000 7f4388e37640 1 -- 172.21.15.43:0/4235196341 >> 172.21.15.43:0/4235196341 conn(0x7f43840cddd0 msgr2=0x7f43840ce1e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-09T13:35:09.968 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:09.960+0000 7f4388e37640 1 -- 172.21.15.43:0/4235196341 shutdown_connections 2023-08-09T13:35:09.968 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:09.960+0000 7f4388e37640 1 -- 172.21.15.43:0/4235196341 wait complete. 2023-08-09T13:35:09.968 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:09.962+0000 7f4388e37640 1 Processor -- start 2023-08-09T13:35:09.968 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:09.962+0000 7f4388e37640 1 -- start start 2023-08-09T13:35:09.968 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:09.963+0000 7f4388e37640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f43840c90a0 0x7f43840ccd30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:35:09.968 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:09.963+0000 7f4388e37640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f4384157ea0 0x7f43840cd270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:35:09.968 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:09.963+0000 7f4388e37640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f438415af10 0x7f43841f5030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:35:09.969 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:09.963+0000 7f4388e37640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f43840cbf30 con 0x7f43840c90a0 2023-08-09T13:35:09.969 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:09.963+0000 7f4388e37640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f43840cb3b0 con 0x7f4384157ea0 2023-08-09T13:35:09.969 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:09.963+0000 7f4388e37640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f43840cb600 con 0x7f438415af10 2023-08-09T13:35:09.969 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:09.963+0000 7f43837fe640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f43840c90a0 0x7f43840ccd30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:35:09.969 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:09.964+0000 7f43837fe640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f43840c90a0 0x7f43840ccd30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:37044/0 (socket says 172.21.15.43:37044) 2023-08-09T13:35:09.969 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:09.964+0000 7f43837fe640 1 -- 172.21.15.43:0/1146143904 learned_addr learned my addr 172.21.15.43:0/1146143904 (peer_addr_for_me v2:172.21.15.43:0/0) 2023-08-09T13:35:09.969 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:09.964+0000 7f43837fe640 1 -- 172.21.15.43:0/1146143904 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f438415af10 msgr2=0x7f43841f5030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:35:09.970 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:09.964+0000 7f4383fff640 1 --2- 172.21.15.43:0/1146143904 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f438415af10 0x7f43841f5030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:35:09.970 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:09.964+0000 7f43837fe640 1 --2- 172.21.15.43:0/1146143904 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f438415af10 0x7f43841f5030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:35:09.970 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:09.964+0000 7f4382ffd640 1 --2- 172.21.15.43:0/1146143904 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f4384157ea0 0x7f43840cd270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:35:09.970 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:09.964+0000 7f43837fe640 1 -- 172.21.15.43:0/1146143904 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f4384157ea0 msgr2=0x7f43840cd270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:35:09.970 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:09.964+0000 7f43837fe640 1 --2- 172.21.15.43:0/1146143904 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f4384157ea0 0x7f43840cd270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:35:09.970 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:09.964+0000 7f43837fe640 1 -- 172.21.15.43:0/1146143904 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f43740094d0 con 0x7f43840c90a0 2023-08-09T13:35:09.970 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:09.964+0000 7f4383fff640 1 --2- 172.21.15.43:0/1146143904 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f438415af10 0x7f43841f5030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-08-09T13:35:09.970 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:09.964+0000 7f43837fe640 1 --2- 172.21.15.43:0/1146143904 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f43840c90a0 0x7f43840ccd30 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7f43740098a0 tx=0x7f43740046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-09T13:35:09.971 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:09.965+0000 7f436bfff640 1 -- 172.21.15.43:0/1146143904 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4374032860 con 0x7f43840c90a0 2023-08-09T13:35:09.971 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:09.965+0000 7f436bfff640 1 -- 172.21.15.43:0/1146143904 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f43740329e0 con 0x7f43840c90a0 2023-08-09T13:35:09.971 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:09.965+0000 7f436bfff640 1 -- 172.21.15.43:0/1146143904 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4374032d90 con 0x7f43840c90a0 2023-08-09T13:35:09.971 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:09.965+0000 7f4388e37640 1 -- 172.21.15.43:0/1146143904 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f43840cb8e0 con 0x7f43840c90a0 2023-08-09T13:35:09.971 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:09.965+0000 7f4388e37640 1 -- 172.21.15.43:0/1146143904 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f43840cd7b0 con 0x7f43840c90a0 2023-08-09T13:35:09.972 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:09.966+0000 7f4388e37640 1 -- 172.21.15.43:0/1146143904 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f438404e390 con 0x7f43840c90a0 2023-08-09T13:35:09.972 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:09.967+0000 7f436bfff640 1 -- 172.21.15.43:0/1146143904 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 5) v1 ==== 88341+0+0 (secure 0 0 0) 0x7f437404d3f0 con 0x7f43840c90a0 2023-08-09T13:35:09.972 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:09.968+0000 7f436bfff640 1 --2- 172.21.15.43:0/1146143904 >> [v2:172.21.15.203:6832/830097474,v1:172.21.15.203:6833/830097474] conn(0x7f437006a050 0x7f437006c510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:35:09.972 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:09.968+0000 7f436bfff640 1 -- 172.21.15.43:0/1146143904 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 6871+0+0 (secure 0 0 0) 0x7f43740aa520 con 0x7f43840c90a0 2023-08-09T13:35:09.973 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:09.969+0000 7f4382ffd640 1 --2- 172.21.15.43:0/1146143904 >> [v2:172.21.15.203:6832/830097474,v1:172.21.15.203:6833/830097474] conn(0x7f437006a050 0x7f437006c510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:35:09.973 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:09.970+0000 7f4382ffd640 1 --2- 172.21.15.43:0/1146143904 >> [v2:172.21.15.203:6832/830097474,v1:172.21.15.203:6833/830097474] conn(0x7f437006a050 0x7f437006c510 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7f438406f780 tx=0x7f4378009040 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-09T13:35:09.974 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:09.972+0000 7f436bfff640 1 -- 172.21.15.43:0/1146143904 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178066 (secure 0 0 0) 0x7f4374075980 con 0x7f43840c90a0 2023-08-09T13:35:10.167 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:10.166+0000 7f4388e37640 1 -- 172.21.15.43:0/1146143904 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"} v 0) v1 -- 0x7f4384158280 con 0x7f43840c90a0 2023-08-09T13:35:10.168 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:10.166+0000 7f436bfff640 1 -- 172.21.15.43:0/1146143904 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"}]=0 v26) v1 ==== 92+0+10 (secure 0 0 0) 0x7f4384158280 con 0x7f43840c90a0 2023-08-09T13:35:10.168 INFO:teuthology.orchestra.run.smithi043.stdout:pg_num: 8 2023-08-09T13:35:10.171 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:10.169+0000 7f4388e37640 1 -- 172.21.15.43:0/1146143904 >> [v2:172.21.15.203:6832/830097474,v1:172.21.15.203:6833/830097474] conn(0x7f437006a050 msgr2=0x7f437006c510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:35:10.171 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:10.169+0000 7f4388e37640 1 --2- 172.21.15.43:0/1146143904 >> [v2:172.21.15.203:6832/830097474,v1:172.21.15.203:6833/830097474] conn(0x7f437006a050 0x7f437006c510 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7f438406f780 tx=0x7f4378009040 comp rx=0 tx=0).stop 2023-08-09T13:35:10.171 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:10.169+0000 7f4388e37640 1 -- 172.21.15.43:0/1146143904 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f43840c90a0 msgr2=0x7f43840ccd30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:35:10.171 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:10.169+0000 7f4388e37640 1 --2- 172.21.15.43:0/1146143904 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f43840c90a0 0x7f43840ccd30 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7f43740098a0 tx=0x7f43740046c0 comp rx=0 tx=0).stop 2023-08-09T13:35:10.171 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:10.169+0000 7f4388e37640 1 -- 172.21.15.43:0/1146143904 shutdown_connections 2023-08-09T13:35:10.171 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:10.169+0000 7f4388e37640 1 --2- 172.21.15.43:0/1146143904 >> [v2:172.21.15.203:6832/830097474,v1:172.21.15.203:6833/830097474] conn(0x7f437006a050 0x7f437006c510 unknown :-1 s=CLOSED pgs=153 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:35:10.172 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:10.169+0000 7f4388e37640 1 --2- 172.21.15.43:0/1146143904 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f438415af10 0x7f43841f5030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:35:10.172 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:10.169+0000 7f4388e37640 1 --2- 172.21.15.43:0/1146143904 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f4384157ea0 0x7f43840cd270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:35:10.172 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:10.169+0000 7f4388e37640 1 --2- 172.21.15.43:0/1146143904 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f43840c90a0 0x7f43840ccd30 unknown :-1 s=CLOSED pgs=277 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:35:10.172 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:10.169+0000 7f4388e37640 1 -- 172.21.15.43:0/1146143904 >> 172.21.15.43:0/1146143904 conn(0x7f43840cddd0 msgr2=0x7f438406b3e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-09T13:35:10.172 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:10.169+0000 7f4388e37640 1 -- 172.21.15.43:0/1146143904 shutdown_connections 2023-08-09T13:35:10.172 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:10.169+0000 7f4388e37640 1 -- 172.21.15.43:0/1146143904 wait complete. 2023-08-09T13:35:10.191 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-08-09T13:35:10.410 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:10.409+0000 7f831dd0f640 1 Processor -- start 2023-08-09T13:35:10.411 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:10.409+0000 7f831dd0f640 1 -- start start 2023-08-09T13:35:10.412 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:10.411+0000 7f831dd0f640 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7f831815df60 0x7f83180ce1c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:35:10.413 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:10.411+0000 7f831dd0f640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f831815e3d0 0x7f83180ce700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:35:10.413 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:10.411+0000 7f831dd0f640 1 -- --> v1:172.21.15.203:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f831815aef0 con 0x7f83180cb2d0 2023-08-09T13:35:10.413 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:10.411+0000 7f831dd0f640 1 -- --> v2:172.21.15.43:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f831815a970 con 0x7f831815e3d0 2023-08-09T13:35:10.413 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:10.411+0000 7f831dd0f640 1 -- --> v2:172.21.15.203:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f831815e940 con 0x7f831815df60 2023-08-09T13:35:10.414 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:10.411+0000 7f8317fff640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f831815e3d0 0x7f83180ce700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:35:10.414 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:10.411+0000 7f8317fff640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f831815e3d0 0x7f83180ce700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:37046/0 (socket says 172.21.15.43:37046) 2023-08-09T13:35:10.414 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:10.411+0000 7f8317fff640 1 -- 172.21.15.43:0/4047382457 learned_addr learned my addr 172.21.15.43:0/4047382457 (peer_addr_for_me v2:172.21.15.43:0/0) 2023-08-09T13:35:10.414 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:10.412+0000 7f8316ffd640 1 --2- 172.21.15.43:0/4047382457 >> v2:172.21.15.203:3300/0 conn(0x7f831815df60 0x7f83180ce1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:35:10.414 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:10.412+0000 7f8317fff640 1 -- 172.21.15.43:0/4047382457 >> v2:172.21.15.203:3300/0 conn(0x7f831815df60 msgr2=0x7f83180ce1c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-09T13:35:10.415 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:10.412+0000 7f8317fff640 1 --2- 172.21.15.43:0/4047382457 >> v2:172.21.15.203:3300/0 conn(0x7f831815df60 0x7f83180ce1c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:35:10.415 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:10.412+0000 7f8317fff640 1 -- 172.21.15.43:0/4047382457 >> v1:172.21.15.203:6789/0 conn(0x7f83180cb2d0 legacy=0x7f831815b380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-09T13:35:10.415 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:10.412+0000 7f8317fff640 1 -- 172.21.15.43:0/4047382457 --> v2:172.21.15.43:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f831815ec20 con 0x7f831815e3d0 2023-08-09T13:35:10.416 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:10.413+0000 7f8317fff640 1 --2- 172.21.15.43:0/4047382457 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f831815e3d0 0x7f83180ce700 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7f83180cf0b0 tx=0x7f831002be50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f14a4479c08602e8 server_cookie=0 in_seq=0 out_seq=0 2023-08-09T13:35:10.416 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:10.413+0000 7f83167fc640 1 -- 172.21.15.43:0/4047382457 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f831002c9f0 con 0x7f831815e3d0 2023-08-09T13:35:10.416 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:10.414+0000 7f83167fc640 1 -- 172.21.15.43:0/4047382457 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f831002cb50 con 0x7f831815e3d0 2023-08-09T13:35:10.416 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:10.414+0000 7f83167fc640 1 -- 172.21.15.43:0/4047382457 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f83100343f0 con 0x7f831815e3d0 2023-08-09T13:35:10.416 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:10.414+0000 7f831dd0f640 1 -- 172.21.15.43:0/4047382457 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f831815e3d0 msgr2=0x7f83180ce700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:35:10.417 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:10.414+0000 7f831dd0f640 1 --2- 172.21.15.43:0/4047382457 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f831815e3d0 0x7f83180ce700 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7f83180cf0b0 tx=0x7f831002be50 comp rx=0 tx=0).stop 2023-08-09T13:35:10.417 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:10.414+0000 7f831dd0f640 1 -- 172.21.15.43:0/4047382457 shutdown_connections 2023-08-09T13:35:10.417 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:10.414+0000 7f831dd0f640 1 --2- 172.21.15.43:0/4047382457 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f831815e3d0 0x7f83180ce700 unknown :-1 s=CLOSED pgs=278 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:35:10.417 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:10.414+0000 7f831dd0f640 1 --2- 172.21.15.43:0/4047382457 >> v2:172.21.15.203:3300/0 conn(0x7f831815df60 0x7f83180ce1c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:35:10.417 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:10.414+0000 7f831dd0f640 1 -- 172.21.15.43:0/4047382457 >> 172.21.15.43:0/4047382457 conn(0x7f83180aff50 msgr2=0x7f83180b0360 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-09T13:35:10.417 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:10.415+0000 7f831dd0f640 1 -- 172.21.15.43:0/4047382457 shutdown_connections 2023-08-09T13:35:10.417 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:10.415+0000 7f831dd0f640 1 -- 172.21.15.43:0/4047382457 wait complete. 2023-08-09T13:35:10.418 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:10.417+0000 7f831dd0f640 1 Processor -- start 2023-08-09T13:35:10.419 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:10.417+0000 7f831dd0f640 1 -- start start 2023-08-09T13:35:10.420 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:10.419+0000 7f831dd0f640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f83180cb2d0 0x7f83180b3710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:35:10.420 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:10.419+0000 7f831dd0f640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f831815df60 0x7f83180c5a90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:35:10.421 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:10.419+0000 7f831dd0f640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f831815e3d0 0x7f83181f46c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:35:10.421 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:10.419+0000 7f831dd0f640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f83180b0cf0 con 0x7f83180cb2d0 2023-08-09T13:35:10.421 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:10.419+0000 7f831dd0f640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f83181f4c00 con 0x7f831815df60 2023-08-09T13:35:10.421 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:10.419+0000 7f831dd0f640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f83180c6150 con 0x7f831815e3d0 2023-08-09T13:35:10.421 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:10.419+0000 7f83177fe640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f83180cb2d0 0x7f83180b3710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:35:10.421 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:10.419+0000 7f83177fe640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f83180cb2d0 0x7f83180b3710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:37060/0 (socket says 172.21.15.43:37060) 2023-08-09T13:35:10.421 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:10.419+0000 7f83177fe640 1 -- 172.21.15.43:0/287613317 learned_addr learned my addr 172.21.15.43:0/287613317 (peer_addr_for_me v2:172.21.15.43:0/0) 2023-08-09T13:35:10.422 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:10.419+0000 7f8316ffd640 1 --2- 172.21.15.43:0/287613317 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f831815df60 0x7f83180c5a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:35:10.423 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:10.419+0000 7f8317fff640 1 --2- 172.21.15.43:0/287613317 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f831815e3d0 0x7f83181f46c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:35:10.423 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:10.419+0000 7f83177fe640 1 -- 172.21.15.43:0/287613317 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f831815e3d0 msgr2=0x7f83181f46c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:35:10.423 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:10.419+0000 7f83177fe640 1 --2- 172.21.15.43:0/287613317 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f831815e3d0 0x7f83181f46c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:35:10.423 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:10.419+0000 7f83177fe640 1 -- 172.21.15.43:0/287613317 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f831815df60 msgr2=0x7f83180c5a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:35:10.423 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:10.419+0000 7f83177fe640 1 --2- 172.21.15.43:0/287613317 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f831815df60 0x7f83180c5a90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:35:10.423 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:10.420+0000 7f83177fe640 1 -- 172.21.15.43:0/287613317 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8310009b30 con 0x7f83180cb2d0 2023-08-09T13:35:10.423 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:10.420+0000 7f8316ffd640 1 --2- 172.21.15.43:0/287613317 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f831815df60 0x7f83180c5a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-08-09T13:35:10.424 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:10.420+0000 7f83177fe640 1 --2- 172.21.15.43:0/287613317 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f83180cb2d0 0x7f83180b3710 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7f830800c430 tx=0x7f830800c8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-09T13:35:10.424 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:10.420+0000 7f82fffff640 1 -- 172.21.15.43:0/287613317 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8308011790 con 0x7f83180cb2d0 2023-08-09T13:35:10.424 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:10.420+0000 7f82fffff640 1 -- 172.21.15.43:0/287613317 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f83080118f0 con 0x7f83180cb2d0 2023-08-09T13:35:10.424 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:10.420+0000 7f82fffff640 1 -- 172.21.15.43:0/287613317 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8308011ce0 con 0x7f83180cb2d0 2023-08-09T13:35:10.424 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:10.420+0000 7f831dd0f640 1 -- 172.21.15.43:0/287613317 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f83180b0f90 con 0x7f83180cb2d0 2023-08-09T13:35:10.424 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:10.420+0000 7f831dd0f640 1 -- 172.21.15.43:0/287613317 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f83181f2a70 con 0x7f83180cb2d0 2023-08-09T13:35:10.425 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:10.422+0000 7f831dd0f640 1 -- 172.21.15.43:0/287613317 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f83180b14b0 con 0x7f83180cb2d0 2023-08-09T13:35:10.425 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:10.423+0000 7f82fffff640 1 -- 172.21.15.43:0/287613317 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 5) v1 ==== 88341+0+0 (secure 0 0 0) 0x7f830801c460 con 0x7f83180cb2d0 2023-08-09T13:35:10.425 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:10.424+0000 7f82fffff640 1 --2- 172.21.15.43:0/287613317 >> [v2:172.21.15.203:6832/830097474,v1:172.21.15.203:6833/830097474] conn(0x7f830406a050 0x7f830406c510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:35:10.426 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:10.424+0000 7f82fffff640 1 -- 172.21.15.43:0/287613317 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 6871+0+0 (secure 0 0 0) 0x7f8308088860 con 0x7f83180cb2d0 2023-08-09T13:35:10.426 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:10.424+0000 7f8316ffd640 1 --2- 172.21.15.43:0/287613317 >> [v2:172.21.15.203:6832/830097474,v1:172.21.15.203:6833/830097474] conn(0x7f830406a050 0x7f830406c510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:35:10.427 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:10.426+0000 7f8316ffd640 1 --2- 172.21.15.43:0/287613317 >> [v2:172.21.15.203:6832/830097474,v1:172.21.15.203:6833/830097474] conn(0x7f830406a050 0x7f830406c510 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7f83180715a0 tx=0x7f830c008040 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-09T13:35:10.429 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:10.428+0000 7f82fffff640 1 -- 172.21.15.43:0/287613317 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178066 (secure 0 0 0) 0x7f83180b14b0 con 0x7f83180cb2d0 2023-08-09T13:35:10.625 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:10.623+0000 7f831dd0f640 1 -- 172.21.15.43:0/287613317 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f83180b14b0 con 0x7f83180cb2d0 2023-08-09T13:35:10.625 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:10.624+0000 7f82fffff640 1 -- 172.21.15.43:0/287613317 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v26) v1 ==== 93+0+10 (secure 0 0 0) 0x7f83080576b0 con 0x7f83180cb2d0 2023-08-09T13:35:10.626 INFO:teuthology.orchestra.run.smithi043.stdout:pg_num: 1 2023-08-09T13:35:10.628 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:10.626+0000 7f831dd0f640 1 -- 172.21.15.43:0/287613317 >> [v2:172.21.15.203:6832/830097474,v1:172.21.15.203:6833/830097474] conn(0x7f830406a050 msgr2=0x7f830406c510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:35:10.628 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:10.626+0000 7f831dd0f640 1 --2- 172.21.15.43:0/287613317 >> [v2:172.21.15.203:6832/830097474,v1:172.21.15.203:6833/830097474] conn(0x7f830406a050 0x7f830406c510 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7f83180715a0 tx=0x7f830c008040 comp rx=0 tx=0).stop 2023-08-09T13:35:10.628 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:10.626+0000 7f831dd0f640 1 -- 172.21.15.43:0/287613317 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f83180cb2d0 msgr2=0x7f83180b3710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:35:10.628 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:10.626+0000 7f831dd0f640 1 --2- 172.21.15.43:0/287613317 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f83180cb2d0 0x7f83180b3710 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7f830800c430 tx=0x7f830800c8f0 comp rx=0 tx=0).stop 2023-08-09T13:35:10.628 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:10.626+0000 7f831dd0f640 1 -- 172.21.15.43:0/287613317 shutdown_connections 2023-08-09T13:35:10.628 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:10.626+0000 7f831dd0f640 1 --2- 172.21.15.43:0/287613317 >> [v2:172.21.15.203:6832/830097474,v1:172.21.15.203:6833/830097474] conn(0x7f830406a050 0x7f830406c510 unknown :-1 s=CLOSED pgs=154 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:35:10.629 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:10.626+0000 7f831dd0f640 1 --2- 172.21.15.43:0/287613317 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f831815e3d0 0x7f83181f46c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:35:10.629 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:10.626+0000 7f831dd0f640 1 --2- 172.21.15.43:0/287613317 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f831815df60 0x7f83180c5a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:35:10.629 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:10.626+0000 7f831dd0f640 1 --2- 172.21.15.43:0/287613317 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f83180cb2d0 0x7f83180b3710 unknown :-1 s=CLOSED pgs=279 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:35:10.629 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:10.626+0000 7f831dd0f640 1 -- 172.21.15.43:0/287613317 >> 172.21.15.43:0/287613317 conn(0x7f83180aff50 msgr2=0x7f8318159440 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-09T13:35:10.629 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:10.626+0000 7f831dd0f640 1 -- 172.21.15.43:0/287613317 shutdown_connections 2023-08-09T13:35:10.629 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:10.626+0000 7f831dd0f640 1 -- 172.21.15.43:0/287613317 wait complete. 2023-08-09T13:35:10.648 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-08-09T13:35:10.878 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:10.876+0000 7f3657c3f640 1 Processor -- start 2023-08-09T13:35:10.878 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:10.877+0000 7f3657c3f640 1 -- start start 2023-08-09T13:35:10.880 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:10.878+0000 7f3657c3f640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f365015a250 0x7f36500697c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:35:10.880 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:10.878+0000 7f3657c3f640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f365015a630 0x7f3650069d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:35:10.880 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:10.878+0000 7f3657c3f640 1 -- --> v1:172.21.15.203:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f36500716b0 con 0x7f36500cb750 2023-08-09T13:35:10.880 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:10.878+0000 7f3657c3f640 1 -- --> v2:172.21.15.43:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3650070aa0 con 0x7f365015a250 2023-08-09T13:35:10.881 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:10.878+0000 7f3657c3f640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3650070d80 con 0x7f365015a630 2023-08-09T13:35:10.882 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:10.879+0000 7f36559db640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f365015a250 0x7f36500697c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:35:10.882 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:10.879+0000 7f36559db640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f365015a250 0x7f36500697c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:37068/0 (socket says 172.21.15.43:37068) 2023-08-09T13:35:10.882 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:10.879+0000 7f36559db640 1 -- 172.21.15.43:0/1572337864 learned_addr learned my addr 172.21.15.43:0/1572337864 (peer_addr_for_me v2:172.21.15.43:0/0) 2023-08-09T13:35:10.882 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:10.879+0000 7f36551da640 1 --2- 172.21.15.43:0/1572337864 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f365015a630 0x7f3650069d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:35:10.882 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:10.879+0000 7f36559db640 1 -- 172.21.15.43:0/1572337864 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f365015a630 msgr2=0x7f3650069d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-09T13:35:10.882 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:10.879+0000 7f36559db640 1 --2- 172.21.15.43:0/1572337864 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f365015a630 0x7f3650069d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:35:10.883 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:10.879+0000 7f36559db640 1 -- 172.21.15.43:0/1572337864 >> v1:172.21.15.203:6789/0 conn(0x7f36500cb750 legacy=0x7f365015e030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-09T13:35:10.883 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:10.879+0000 7f36559db640 1 -- 172.21.15.43:0/1572337864 --> v2:172.21.15.43:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3650071060 con 0x7f365015a250 2023-08-09T13:35:10.883 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:10.879+0000 7f36551da640 1 --2- 172.21.15.43:0/1572337864 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f365015a630 0x7f3650069d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-08-09T13:35:10.883 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:10.880+0000 7f36559db640 1 --2- 172.21.15.43:0/1572337864 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f365015a250 0x7f36500697c0 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7f364c009870 tx=0x7f364c02b980 comp rx=0 tx=0).ready entity=mon.0 client_cookie=130bdb0712823a36 server_cookie=0 in_seq=0 out_seq=0 2023-08-09T13:35:10.884 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:10.881+0000 7f36549d9640 1 -- 172.21.15.43:0/1572337864 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f364c030840 con 0x7f365015a250 2023-08-09T13:35:10.884 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:10.881+0000 7f36549d9640 1 -- 172.21.15.43:0/1572337864 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f364c0309a0 con 0x7f365015a250 2023-08-09T13:35:10.884 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:10.881+0000 7f36549d9640 1 -- 172.21.15.43:0/1572337864 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f364c030d30 con 0x7f365015a250 2023-08-09T13:35:10.884 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:10.881+0000 7f3657c3f640 1 -- 172.21.15.43:0/1572337864 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f365015a250 msgr2=0x7f36500697c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:35:10.884 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:10.881+0000 7f3657c3f640 1 --2- 172.21.15.43:0/1572337864 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f365015a250 0x7f36500697c0 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7f364c009870 tx=0x7f364c02b980 comp rx=0 tx=0).stop 2023-08-09T13:35:10.885 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:10.882+0000 7f3657c3f640 1 -- 172.21.15.43:0/1572337864 shutdown_connections 2023-08-09T13:35:10.885 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:10.882+0000 7f3657c3f640 1 --2- 172.21.15.43:0/1572337864 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f365015a630 0x7f3650069d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:35:10.885 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:10.882+0000 7f3657c3f640 1 --2- 172.21.15.43:0/1572337864 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f365015a250 0x7f36500697c0 unknown :-1 s=CLOSED pgs=280 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:35:10.885 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:10.882+0000 7f3657c3f640 1 -- 172.21.15.43:0/1572337864 >> 172.21.15.43:0/1572337864 conn(0x7f36500cbc80 msgr2=0x7f36500cc090 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-09T13:35:10.885 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:10.882+0000 7f3657c3f640 1 -- 172.21.15.43:0/1572337864 shutdown_connections 2023-08-09T13:35:10.885 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:10.882+0000 7f3657c3f640 1 -- 172.21.15.43:0/1572337864 wait complete. 2023-08-09T13:35:10.886 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:10.883+0000 7f3657c3f640 1 Processor -- start 2023-08-09T13:35:10.886 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:10.884+0000 7f3657c3f640 1 -- start start 2023-08-09T13:35:10.887 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:10.885+0000 7f3657c3f640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f36500cb750 0x7f36501f1b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:35:10.887 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:10.885+0000 7f3657c3f640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f365015a250 0x7f36500a9110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:35:10.887 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:10.885+0000 7f3657c3f640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f365015a630 0x7f36500a9650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:35:10.887 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:10.885+0000 7f3657c3f640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f36500a7760 con 0x7f36500cb750 2023-08-09T13:35:10.887 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:10.885+0000 7f3657c3f640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f36501f2b40 con 0x7f365015a630 2023-08-09T13:35:10.887 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:10.885+0000 7f3657c3f640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f36500c8480 con 0x7f365015a250 2023-08-09T13:35:10.887 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:10.885+0000 7f36559db640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f36500cb750 0x7f36501f1b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:35:10.887 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:10.885+0000 7f36559db640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f36500cb750 0x7f36501f1b00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:37076/0 (socket says 172.21.15.43:37076) 2023-08-09T13:35:10.888 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:10.885+0000 7f36559db640 1 -- 172.21.15.43:0/3374910179 learned_addr learned my addr 172.21.15.43:0/3374910179 (peer_addr_for_me v2:172.21.15.43:0/0) 2023-08-09T13:35:10.888 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:10.885+0000 7f36559db640 1 -- 172.21.15.43:0/3374910179 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f365015a250 msgr2=0x7f36500a9110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:35:10.889 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:10.885+0000 7f36559db640 1 --2- 172.21.15.43:0/3374910179 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f365015a250 0x7f36500a9110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:35:10.889 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:10.885+0000 7f36559db640 1 -- 172.21.15.43:0/3374910179 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f365015a630 msgr2=0x7f36500a9650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:35:10.889 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:10.885+0000 7f36559db640 1 --2- 172.21.15.43:0/3374910179 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f365015a630 0x7f36500a9650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:35:10.889 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:10.885+0000 7f36559db640 1 -- 172.21.15.43:0/3374910179 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f364c0094d0 con 0x7f36500cb750 2023-08-09T13:35:10.889 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:10.886+0000 7f36559db640 1 --2- 172.21.15.43:0/3374910179 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f36500cb750 0x7f36501f1b00 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7f364c009870 tx=0x7f364c004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-09T13:35:10.889 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:10.886+0000 7f36467fc640 1 -- 172.21.15.43:0/3374910179 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f364c0307f0 con 0x7f36500cb750 2023-08-09T13:35:10.889 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:10.886+0000 7f36467fc640 1 -- 172.21.15.43:0/3374910179 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f364c043400 con 0x7f36500cb750 2023-08-09T13:35:10.890 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:10.886+0000 7f36467fc640 1 -- 172.21.15.43:0/3374910179 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f364c0437b0 con 0x7f36500cb750 2023-08-09T13:35:10.890 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:10.886+0000 7f3657c3f640 1 -- 172.21.15.43:0/3374910179 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f36501f2040 con 0x7f36500cb750 2023-08-09T13:35:10.890 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:10.886+0000 7f3657c3f640 1 -- 172.21.15.43:0/3374910179 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f36501f2580 con 0x7f36500cb750 2023-08-09T13:35:10.890 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:10.887+0000 7f3657c3f640 1 -- 172.21.15.43:0/3374910179 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f36500a8be0 con 0x7f36500cb750 2023-08-09T13:35:10.891 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:10.888+0000 7f36467fc640 1 -- 172.21.15.43:0/3374910179 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 5) v1 ==== 88341+0+0 (secure 0 0 0) 0x7f364c043910 con 0x7f36500cb750 2023-08-09T13:35:10.891 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:10.889+0000 7f36467fc640 1 --2- 172.21.15.43:0/3374910179 >> [v2:172.21.15.203:6832/830097474,v1:172.21.15.203:6833/830097474] conn(0x7f363406a080 0x7f363406c540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:35:10.891 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:10.889+0000 7f36467fc640 1 -- 172.21.15.43:0/3374910179 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 6871+0+0 (secure 0 0 0) 0x7f364c03e070 con 0x7f36500cb750 2023-08-09T13:35:10.891 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:10.890+0000 7f36551da640 1 --2- 172.21.15.43:0/3374910179 >> [v2:172.21.15.203:6832/830097474,v1:172.21.15.203:6833/830097474] conn(0x7f363406a080 0x7f363406c540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:35:10.892 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:10.891+0000 7f36551da640 1 --2- 172.21.15.43:0/3374910179 >> [v2:172.21.15.203:6832/830097474,v1:172.21.15.203:6833/830097474] conn(0x7f363406a080 0x7f363406c540 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7f3640006fd0 tx=0x7f3640008040 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-09T13:35:10.895 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:10.893+0000 7f36467fc640 1 -- 172.21.15.43:0/3374910179 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178066 (secure 0 0 0) 0x7f364c076f10 con 0x7f36500cb750 2023-08-09T13:35:11.088 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:11.086+0000 7f3657c3f640 1 -- 172.21.15.43:0/3374910179 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f36500a74c0 con 0x7f36500cb750 2023-08-09T13:35:11.089 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:11.087+0000 7f36467fc640 1 -- 172.21.15.43:0/3374910179 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v26) v1 ==== 104+0+11 (secure 0 0 0) 0x7f36500a74c0 con 0x7f36500cb750 2023-08-09T13:35:11.089 INFO:teuthology.orchestra.run.smithi043.stdout:pg_num: 32 2023-08-09T13:35:11.092 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:11.090+0000 7f3657c3f640 1 -- 172.21.15.43:0/3374910179 >> [v2:172.21.15.203:6832/830097474,v1:172.21.15.203:6833/830097474] conn(0x7f363406a080 msgr2=0x7f363406c540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:35:11.092 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:11.090+0000 7f3657c3f640 1 --2- 172.21.15.43:0/3374910179 >> [v2:172.21.15.203:6832/830097474,v1:172.21.15.203:6833/830097474] conn(0x7f363406a080 0x7f363406c540 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7f3640006fd0 tx=0x7f3640008040 comp rx=0 tx=0).stop 2023-08-09T13:35:11.092 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:11.090+0000 7f3657c3f640 1 -- 172.21.15.43:0/3374910179 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f36500cb750 msgr2=0x7f36501f1b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:35:11.092 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:11.090+0000 7f3657c3f640 1 --2- 172.21.15.43:0/3374910179 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f36500cb750 0x7f36501f1b00 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7f364c009870 tx=0x7f364c004730 comp rx=0 tx=0).stop 2023-08-09T13:35:11.092 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:11.090+0000 7f3657c3f640 1 -- 172.21.15.43:0/3374910179 shutdown_connections 2023-08-09T13:35:11.092 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:11.090+0000 7f3657c3f640 1 --2- 172.21.15.43:0/3374910179 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f365015a630 0x7f36500a9650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:35:11.093 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:11.090+0000 7f3657c3f640 1 --2- 172.21.15.43:0/3374910179 >> [v2:172.21.15.203:6832/830097474,v1:172.21.15.203:6833/830097474] conn(0x7f363406a080 0x7f363406c540 unknown :-1 s=CLOSED pgs=155 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:35:11.093 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:11.090+0000 7f3657c3f640 1 --2- 172.21.15.43:0/3374910179 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f365015a250 0x7f36500a9110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:35:11.093 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:11.090+0000 7f3657c3f640 1 --2- 172.21.15.43:0/3374910179 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f36500cb750 0x7f36501f1b00 unknown :-1 s=CLOSED pgs=281 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:35:11.093 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:11.090+0000 7f3657c3f640 1 -- 172.21.15.43:0/3374910179 >> 172.21.15.43:0/3374910179 conn(0x7f36500cbc80 msgr2=0x7f36500cc380 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-09T13:35:11.093 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:11.090+0000 7f3657c3f640 1 -- 172.21.15.43:0/3374910179 shutdown_connections 2023-08-09T13:35:11.094 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:11.090+0000 7f3657c3f640 1 -- 172.21.15.43:0/3374910179 wait complete. 2023-08-09T13:35:11.110 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-08-09T13:35:11.358 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:11.356+0000 7f038df6a640 1 Processor -- start 2023-08-09T13:35:11.358 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:11.356+0000 7f038df6a640 1 -- start start 2023-08-09T13:35:11.360 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:11.358+0000 7f038df6a640 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7f03880ce850 0x7f03880c3d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:35:11.360 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:11.358+0000 7f038df6a640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f03880cf790 0x7f03880cd630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:35:11.360 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:11.358+0000 7f038df6a640 1 -- --> v1:172.21.15.43:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f038815e470 con 0x7f038815a550 2023-08-09T13:35:11.360 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:11.358+0000 7f038df6a640 1 -- --> v2:172.21.15.203:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f038815e750 con 0x7f03880ce850 2023-08-09T13:35:11.360 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:11.358+0000 7f038df6a640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f038815ea30 con 0x7f03880cf790 2023-08-09T13:35:11.360 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:11.358+0000 7f03877fe640 1 --1- >> v1:172.21.15.43:6789/0 conn(0x7f038815a550 0x7f03880ce140 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.43:6789/0 says I am v1:172.21.15.43:57566/0 (socket says 172.21.15.43:57566) 2023-08-09T13:35:11.361 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:11.358+0000 7f03877fe640 1 -- 172.21.15.43:0/2073340407 learned_addr learned my addr 172.21.15.43:0/2073340407 (peer_addr_for_me v1:172.21.15.43:0/0) 2023-08-09T13:35:11.362 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:11.359+0000 7f0386ffd640 1 --2- 172.21.15.43:0/2073340407 >> v2:172.21.15.203:3300/0 conn(0x7f03880ce850 0x7f03880c3d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:35:11.362 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:11.359+0000 7f0387fff640 1 --2- 172.21.15.43:0/2073340407 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f03880cf790 0x7f03880cd630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:35:11.362 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:11.359+0000 7f03867fc640 1 -- 172.21.15.43:0/2073340407 <== mon.0 v1:172.21.15.43:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 823069607 0 0) 0x7f038815e470 con 0x7f038815a550 2023-08-09T13:35:11.362 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:11.359+0000 7f03867fc640 1 -- 172.21.15.43:0/2073340407 --> v1:172.21.15.43:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f0370003520 con 0x7f038815a550 2023-08-09T13:35:11.363 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:11.359+0000 7f03867fc640 1 -- 172.21.15.43:0/2073340407 <== mon.0 v1:172.21.15.43:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2347758717 0 0) 0x7f0370003520 con 0x7f038815a550 2023-08-09T13:35:11.363 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:11.359+0000 7f03867fc640 1 -- 172.21.15.43:0/2073340407 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f03880cf790 msgr2=0x7f03880cd630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-09T13:35:11.363 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:11.359+0000 7f03867fc640 1 --2- 172.21.15.43:0/2073340407 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f03880cf790 0x7f03880cd630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:35:11.363 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:11.359+0000 7f03867fc640 1 -- 172.21.15.43:0/2073340407 >> v2:172.21.15.203:3300/0 conn(0x7f03880ce850 msgr2=0x7f03880c3d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-09T13:35:11.363 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:11.359+0000 7f03867fc640 1 --2- 172.21.15.43:0/2073340407 >> v2:172.21.15.203:3300/0 conn(0x7f03880ce850 0x7f03880c3d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:35:11.363 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:11.359+0000 7f03867fc640 1 -- 172.21.15.43:0/2073340407 --> v1:172.21.15.43:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f038815ed10 con 0x7f038815a550 2023-08-09T13:35:11.363 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:11.359+0000 7f03867fc640 1 -- 172.21.15.43:0/2073340407 <== mon.0 v1:172.21.15.43:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1285150740 0 0) 0x7f0378001670 con 0x7f038815a550 2023-08-09T13:35:11.363 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:11.359+0000 7f03867fc640 1 -- 172.21.15.43:0/2073340407 >> v1:172.21.15.43:6789/0 conn(0x7f038815a550 legacy=0x7f03880ce140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:35:11.364 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:11.359+0000 7f03867fc640 1 --2- 172.21.15.43:0/2073340407 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f0370003970 0x7f0370003d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:35:11.364 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:11.359+0000 7f03867fc640 1 --2- 172.21.15.43:0/2073340407 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f03880cf790 0x7f03700047c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:35:11.364 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:11.360+0000 7f03867fc640 1 --2- 172.21.15.43:0/2073340407 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f0370004d00 0x7f0370005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:35:11.364 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:11.360+0000 7f03867fc640 1 -- 172.21.15.43:0/2073340407 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f038815e750 con 0x7f0370004d00 2023-08-09T13:35:11.364 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:11.360+0000 7f03867fc640 1 -- 172.21.15.43:0/2073340407 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f038815ea30 con 0x7f0370003970 2023-08-09T13:35:11.364 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:11.360+0000 7f03867fc640 1 -- 172.21.15.43:0/2073340407 --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f037000ba00 con 0x7f03880cf790 2023-08-09T13:35:11.364 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:11.360+0000 7f0387fff640 1 --2- 172.21.15.43:0/2073340407 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f0370004d00 0x7f0370005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:35:11.365 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:11.360+0000 7f03877fe640 1 --2- 172.21.15.43:0/2073340407 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f0370003970 0x7f0370003d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:35:11.365 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:11.360+0000 7f0386ffd640 1 --2- 172.21.15.43:0/2073340407 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f03880cf790 0x7f03700047c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:35:11.365 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:11.360+0000 7f0387fff640 1 -- 172.21.15.43:0/2073340407 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f03880cf790 msgr2=0x7f03700047c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-09T13:35:11.365 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:11.360+0000 7f0387fff640 1 --2- 172.21.15.43:0/2073340407 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f03880cf790 0x7f03700047c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:35:11.365 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:11.360+0000 7f0387fff640 1 -- 172.21.15.43:0/2073340407 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f0370003970 msgr2=0x7f0370003d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-09T13:35:11.365 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:11.360+0000 7f0387fff640 1 --2- 172.21.15.43:0/2073340407 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f0370003970 0x7f0370003d90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:35:11.366 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:11.360+0000 7f0387fff640 1 -- 172.21.15.43:0/2073340407 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0370003520 con 0x7f0370004d00 2023-08-09T13:35:11.366 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:11.361+0000 7f0387fff640 1 --2- 172.21.15.43:0/2073340407 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f0370004d00 0x7f0370005180 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7f0380009fc0 tx=0x7f03700057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=70c1f953dabf14 server_cookie=0 in_seq=0 out_seq=0 2023-08-09T13:35:11.367 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:11.362+0000 7f03867fc640 1 -- 172.21.15.43:0/2073340407 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f038002cc20 con 0x7f0370004d00 2023-08-09T13:35:11.367 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:11.362+0000 7f03867fc640 1 -- 172.21.15.43:0/2073340407 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f038000f040 con 0x7f0370004d00 2023-08-09T13:35:11.367 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:11.362+0000 7f03867fc640 1 -- 172.21.15.43:0/2073340407 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0380034440 con 0x7f0370004d00 2023-08-09T13:35:11.367 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:11.362+0000 7f038df6a640 1 -- 172.21.15.43:0/2073340407 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f0370004d00 msgr2=0x7f0370005180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:35:11.367 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:11.362+0000 7f038df6a640 1 --2- 172.21.15.43:0/2073340407 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f0370004d00 0x7f0370005180 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7f0380009fc0 tx=0x7f03700057f0 comp rx=0 tx=0).stop 2023-08-09T13:35:11.367 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:11.362+0000 7f0387fff640 1 -- 172.21.15.43:0/2073340407 reap_dead start 2023-08-09T13:35:11.367 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:11.362+0000 7f038df6a640 1 -- 172.21.15.43:0/2073340407 shutdown_connections 2023-08-09T13:35:11.368 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:11.362+0000 7f038df6a640 1 -- 172.21.15.43:0/2073340407 >> 172.21.15.43:0/2073340407 conn(0x7f03880b1dd0 msgr2=0x7f03880b21e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-09T13:35:11.368 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:11.363+0000 7f038df6a640 1 -- 172.21.15.43:0/2073340407 shutdown_connections 2023-08-09T13:35:11.368 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:11.363+0000 7f038df6a640 1 -- 172.21.15.43:0/2073340407 wait complete. 2023-08-09T13:35:11.368 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:11.364+0000 7f038df6a640 1 Processor -- start 2023-08-09T13:35:11.368 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:11.365+0000 7f038df6a640 1 -- start start 2023-08-09T13:35:11.370 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:11.365+0000 7f038df6a640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f0370004d00 0x7f0388158890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:35:11.370 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:11.366+0000 7f038df6a640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f038815b4c0 0x7f03880b0850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:35:11.370 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:11.366+0000 7f038df6a640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f03880ae1e0 0x7f03880b0d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:35:11.370 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:11.366+0000 7f038df6a640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0388158dd0 con 0x7f0370004d00 2023-08-09T13:35:11.370 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:11.366+0000 7f038df6a640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f038815b950 con 0x7f03880ae1e0 2023-08-09T13:35:11.370 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:11.366+0000 7f038df6a640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f038815bc30 con 0x7f038815b4c0 2023-08-09T13:35:11.371 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:11.366+0000 7f03877fe640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f0370004d00 0x7f0388158890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:35:11.371 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:11.366+0000 7f03877fe640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f0370004d00 0x7f0388158890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:37098/0 (socket says 172.21.15.43:37098) 2023-08-09T13:35:11.371 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:11.366+0000 7f03877fe640 1 -- 172.21.15.43:0/1589150106 learned_addr learned my addr 172.21.15.43:0/1589150106 (peer_addr_for_me v2:172.21.15.43:0/0) 2023-08-09T13:35:11.371 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:11.366+0000 7f0387fff640 1 --2- 172.21.15.43:0/1589150106 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f03880ae1e0 0x7f03880b0d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:35:11.371 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:11.366+0000 7f03877fe640 1 -- 172.21.15.43:0/1589150106 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f038815b4c0 msgr2=0x7f03880b0850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:35:11.371 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:11.366+0000 7f03877fe640 1 --2- 172.21.15.43:0/1589150106 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f038815b4c0 0x7f03880b0850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:35:11.371 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:11.366+0000 7f03877fe640 1 -- 172.21.15.43:0/1589150106 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f03880ae1e0 msgr2=0x7f03880b0d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:35:11.372 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:11.366+0000 7f03877fe640 1 --2- 172.21.15.43:0/1589150106 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f03880ae1e0 0x7f03880b0d90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:35:11.372 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:11.366+0000 7f03877fe640 1 -- 172.21.15.43:0/1589150106 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f038815bf10 con 0x7f0370004d00 2023-08-09T13:35:11.372 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:11.367+0000 7f03877fe640 1 --2- 172.21.15.43:0/1589150106 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f0370004d00 0x7f0388158890 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7f03780018a0 tx=0x7f037800dc70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-09T13:35:11.372 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:11.367+0000 7f036ffff640 1 -- 172.21.15.43:0/1589150106 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f03780074e0 con 0x7f0370004d00 2023-08-09T13:35:11.372 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:11.367+0000 7f036ffff640 1 -- 172.21.15.43:0/1589150106 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0378007640 con 0x7f0370004d00 2023-08-09T13:35:11.372 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:11.367+0000 7f036ffff640 1 -- 172.21.15.43:0/1589150106 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0378004a00 con 0x7f0370004d00 2023-08-09T13:35:11.372 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:11.367+0000 7f038df6a640 1 -- 172.21.15.43:0/1589150106 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f038815c1f0 con 0x7f0370004d00 2023-08-09T13:35:11.372 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:11.367+0000 7f038df6a640 1 -- 172.21.15.43:0/1589150106 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f03880ada80 con 0x7f0370004d00 2023-08-09T13:35:11.373 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:11.368+0000 7f038df6a640 1 -- 172.21.15.43:0/1589150106 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f03880adca0 con 0x7f0370004d00 2023-08-09T13:35:11.373 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:11.369+0000 7f036ffff640 1 -- 172.21.15.43:0/1589150106 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 5) v1 ==== 88341+0+0 (secure 0 0 0) 0x7f0378004b60 con 0x7f0370004d00 2023-08-09T13:35:11.373 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:11.370+0000 7f036ffff640 1 --2- 172.21.15.43:0/1589150106 >> [v2:172.21.15.203:6832/830097474,v1:172.21.15.203:6833/830097474] conn(0x7f037406a050 0x7f037406c510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:35:11.374 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:11.370+0000 7f036ffff640 1 -- 172.21.15.43:0/1589150106 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 6871+0+0 (secure 0 0 0) 0x7f0378088350 con 0x7f0370004d00 2023-08-09T13:35:11.374 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:11.371+0000 7f0386ffd640 1 --2- 172.21.15.43:0/1589150106 >> [v2:172.21.15.203:6832/830097474,v1:172.21.15.203:6833/830097474] conn(0x7f037406a050 0x7f037406c510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:35:11.374 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:11.372+0000 7f0386ffd640 1 --2- 172.21.15.43:0/1589150106 >> [v2:172.21.15.203:6832/830097474,v1:172.21.15.203:6833/830097474] conn(0x7f037406a050 0x7f037406c510 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7f03880cb350 tx=0x7f037c008040 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-09T13:35:11.376 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:11.375+0000 7f036ffff640 1 -- 172.21.15.43:0/1589150106 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178066 (secure 0 0 0) 0x7f03880adca0 con 0x7f0370004d00 2023-08-09T13:35:11.570 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:11.569+0000 7f038df6a640 1 -- 172.21.15.43:0/1589150106 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f03880adca0 con 0x7f0370004d00 2023-08-09T13:35:11.571 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:11.570+0000 7f036ffff640 1 -- 172.21.15.43:0/1589150106 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v26) v1 ==== 100+0+11 (secure 0 0 0) 0x7f03880adca0 con 0x7f0370004d00 2023-08-09T13:35:11.571 INFO:teuthology.orchestra.run.smithi043.stdout:pg_num: 64 2023-08-09T13:35:11.574 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:11.572+0000 7f038df6a640 1 -- 172.21.15.43:0/1589150106 >> [v2:172.21.15.203:6832/830097474,v1:172.21.15.203:6833/830097474] conn(0x7f037406a050 msgr2=0x7f037406c510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:35:11.574 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:11.572+0000 7f038df6a640 1 --2- 172.21.15.43:0/1589150106 >> [v2:172.21.15.203:6832/830097474,v1:172.21.15.203:6833/830097474] conn(0x7f037406a050 0x7f037406c510 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7f03880cb350 tx=0x7f037c008040 comp rx=0 tx=0).stop 2023-08-09T13:35:11.574 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:11.572+0000 7f038df6a640 1 -- 172.21.15.43:0/1589150106 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f0370004d00 msgr2=0x7f0388158890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:35:11.574 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:11.572+0000 7f038df6a640 1 --2- 172.21.15.43:0/1589150106 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f0370004d00 0x7f0388158890 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7f03780018a0 tx=0x7f037800dc70 comp rx=0 tx=0).stop 2023-08-09T13:35:11.574 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:11.572+0000 7f038df6a640 1 -- 172.21.15.43:0/1589150106 shutdown_connections 2023-08-09T13:35:11.575 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:11.572+0000 7f038df6a640 1 --2- 172.21.15.43:0/1589150106 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f03880ae1e0 0x7f03880b0d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:35:11.575 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:11.572+0000 7f038df6a640 1 --2- 172.21.15.43:0/1589150106 >> [v2:172.21.15.203:6832/830097474,v1:172.21.15.203:6833/830097474] conn(0x7f037406a050 0x7f037406c510 unknown :-1 s=CLOSED pgs=156 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:35:11.575 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:11.572+0000 7f038df6a640 1 --2- 172.21.15.43:0/1589150106 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f038815b4c0 0x7f03880b0850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:35:11.575 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:11.572+0000 7f038df6a640 1 --2- 172.21.15.43:0/1589150106 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f0370004d00 0x7f0388158890 unknown :-1 s=CLOSED pgs=284 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:35:11.575 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:11.572+0000 7f038df6a640 1 -- 172.21.15.43:0/1589150106 >> 172.21.15.43:0/1589150106 conn(0x7f03880b1dd0 msgr2=0x7f03880c38e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-09T13:35:11.575 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:11.573+0000 7f038df6a640 1 -- 172.21.15.43:0/1589150106 shutdown_connections 2023-08-09T13:35:11.576 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:11.573+0000 7f038df6a640 1 -- 172.21.15.43:0/1589150106 wait complete. 2023-08-09T13:35:11.594 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2023-08-09T13:35:11.831 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:11.830+0000 7f41f0c3f640 1 Processor -- start 2023-08-09T13:35:11.831 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:11.830+0000 7f41f0c3f640 1 -- start start 2023-08-09T13:35:11.833 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:11.831+0000 7f41f0c3f640 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7f41ec15c680 0x7f41ec0ce760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:35:11.833 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:11.831+0000 7f41f0c3f640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f41ec0d0a70 0x7f41ec0cd7f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:35:11.833 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:11.831+0000 7f41f0c3f640 1 -- --> v1:172.21.15.43:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f41ec0d0f70 con 0x7f41ec0cdd30 2023-08-09T13:35:11.833 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:11.831+0000 7f41f0c3f640 1 -- --> v2:172.21.15.43:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f41ec0ccd10 con 0x7f41ec0d0a70 2023-08-09T13:35:11.833 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:11.831+0000 7f41f0c3f640 1 -- --> v2:172.21.15.203:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f41ec0ccff0 con 0x7f41ec15c680 2023-08-09T13:35:11.834 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:11.832+0000 7f41ead9d640 1 --1- >> v1:172.21.15.43:6789/0 conn(0x7f41ec0cdd30 0x7f41ec0b3fa0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.43:6789/0 says I am v1:172.21.15.43:57568/0 (socket says 172.21.15.43:57568) 2023-08-09T13:35:11.834 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:11.832+0000 7f41ead9d640 1 -- 172.21.15.43:0/714533006 learned_addr learned my addr 172.21.15.43:0/714533006 (peer_addr_for_me v1:172.21.15.43:0/0) 2023-08-09T13:35:11.834 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:11.832+0000 7f41e9d9b640 1 --2- 172.21.15.43:0/714533006 >> v2:172.21.15.43:3300/0 conn(0x7f41ec0d0a70 0x7f41ec0cd7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:35:11.835 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:11.832+0000 7f41ea59c640 1 --2- 172.21.15.43:0/714533006 >> v2:172.21.15.203:3300/0 conn(0x7f41ec15c680 0x7f41ec0ce760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:35:11.835 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:11.832+0000 7f41e959a640 1 -- 172.21.15.43:0/714533006 <== mon.0 v1:172.21.15.43:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2271570506 0 0) 0x7f41ec0d0f70 con 0x7f41ec0cdd30 2023-08-09T13:35:11.835 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:11.832+0000 7f41e959a640 1 -- 172.21.15.43:0/714533006 --> v1:172.21.15.43:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f41d8003520 con 0x7f41ec0cdd30 2023-08-09T13:35:11.835 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:11.832+0000 7f41e9d9b640 1 -- 172.21.15.43:0/714533006 >> v2:172.21.15.203:3300/0 conn(0x7f41ec15c680 msgr2=0x7f41ec0ce760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-09T13:35:11.835 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:11.832+0000 7f41e9d9b640 1 --2- 172.21.15.43:0/714533006 >> v2:172.21.15.203:3300/0 conn(0x7f41ec15c680 0x7f41ec0ce760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:35:11.835 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:11.832+0000 7f41e9d9b640 1 -- 172.21.15.43:0/714533006 >> v1:172.21.15.43:6789/0 conn(0x7f41ec0cdd30 legacy=0x7f41ec0b3fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:35:11.835 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:11.833+0000 7f41e9d9b640 1 -- 172.21.15.43:0/714533006 --> v2:172.21.15.43:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f41ec0cd2d0 con 0x7f41ec0d0a70 2023-08-09T13:35:11.836 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:11.834+0000 7f41e9d9b640 1 --2- 172.21.15.43:0/714533006 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f41ec0d0a70 0x7f41ec0cd7f0 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7f41ec06b7d0 tx=0x7f41d402b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6143f689e4d1598 server_cookie=0 in_seq=0 out_seq=0 2023-08-09T13:35:11.836 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:11.834+0000 7f41e959a640 1 -- 172.21.15.43:0/714533006 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f41d4031830 con 0x7f41ec0d0a70 2023-08-09T13:35:11.837 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:11.834+0000 7f41e959a640 1 -- 172.21.15.43:0/714533006 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f41d4031990 con 0x7f41ec0d0a70 2023-08-09T13:35:11.837 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:11.834+0000 7f41e959a640 1 -- 172.21.15.43:0/714533006 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f41d4031d20 con 0x7f41ec0d0a70 2023-08-09T13:35:11.837 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:11.835+0000 7f41f0c3f640 1 -- 172.21.15.43:0/714533006 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f41ec0d0a70 msgr2=0x7f41ec0cd7f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:35:11.837 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:11.835+0000 7f41f0c3f640 1 --2- 172.21.15.43:0/714533006 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f41ec0d0a70 0x7f41ec0cd7f0 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7f41ec06b7d0 tx=0x7f41d402b8d0 comp rx=0 tx=0).stop 2023-08-09T13:35:11.837 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:11.835+0000 7f41f0c3f640 1 -- 172.21.15.43:0/714533006 shutdown_connections 2023-08-09T13:35:11.837 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:11.835+0000 7f41f0c3f640 1 --2- 172.21.15.43:0/714533006 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f41ec0d0a70 0x7f41ec0cd7f0 unknown :-1 s=CLOSED pgs=286 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:35:11.837 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:11.835+0000 7f41f0c3f640 1 --2- 172.21.15.43:0/714533006 >> v2:172.21.15.203:3300/0 conn(0x7f41ec15c680 0x7f41ec0ce760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:35:11.838 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:11.835+0000 7f41f0c3f640 1 -- 172.21.15.43:0/714533006 >> 172.21.15.43:0/714533006 conn(0x7f41ec071520 msgr2=0x7f41ec0d2520 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-09T13:35:11.838 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:11.835+0000 7f41f0c3f640 1 -- 172.21.15.43:0/714533006 shutdown_connections 2023-08-09T13:35:11.838 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:11.835+0000 7f41f0c3f640 1 -- 172.21.15.43:0/714533006 wait complete. 2023-08-09T13:35:11.838 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:11.837+0000 7f41f0c3f640 1 Processor -- start 2023-08-09T13:35:11.838 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:11.837+0000 7f41f0c3f640 1 -- start start 2023-08-09T13:35:11.839 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:11.838+0000 7f41f0c3f640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f41ec0d0a70 0x7f41ec153a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:35:11.840 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:11.838+0000 7f41f0c3f640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f41ec15c680 0x7f41ec1f1c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:35:11.840 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:11.838+0000 7f41f0c3f640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f41ec1f2150 0x7f41ec0c95f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:35:11.840 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:11.838+0000 7f41f0c3f640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f41ec0ca600 con 0x7f41ec15c680 2023-08-09T13:35:11.840 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:11.838+0000 7f41f0c3f640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f41ec153fa0 con 0x7f41ec1f2150 2023-08-09T13:35:11.840 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:11.838+0000 7f41f0c3f640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f41ec0ca040 con 0x7f41ec0d0a70 2023-08-09T13:35:11.841 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:11.838+0000 7f41e9d9b640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f41ec15c680 0x7f41ec1f1c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:35:11.841 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:11.838+0000 7f41e9d9b640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f41ec15c680 0x7f41ec1f1c10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:37104/0 (socket says 172.21.15.43:37104) 2023-08-09T13:35:11.842 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:11.838+0000 7f41e9d9b640 1 -- 172.21.15.43:0/3520028722 learned_addr learned my addr 172.21.15.43:0/3520028722 (peer_addr_for_me v2:172.21.15.43:0/0) 2023-08-09T13:35:11.842 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:11.839+0000 7f41ea59c640 1 --2- 172.21.15.43:0/3520028722 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f41ec0d0a70 0x7f41ec153a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:35:11.842 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:11.839+0000 7f41e9d9b640 1 -- 172.21.15.43:0/3520028722 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f41ec0d0a70 msgr2=0x7f41ec153a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:35:11.842 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:11.839+0000 7f41e9d9b640 1 --2- 172.21.15.43:0/3520028722 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f41ec0d0a70 0x7f41ec153a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:35:11.842 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:11.839+0000 7f41e9d9b640 1 -- 172.21.15.43:0/3520028722 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f41ec1f2150 msgr2=0x7f41ec0c95f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:35:11.842 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:11.839+0000 7f41e9d9b640 1 --2- 172.21.15.43:0/3520028722 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f41ec1f2150 0x7f41ec0c95f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:35:11.842 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:11.839+0000 7f41e9d9b640 1 -- 172.21.15.43:0/3520028722 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f41d4009400 con 0x7f41ec15c680 2023-08-09T13:35:11.842 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:11.839+0000 7f41e9d9b640 1 --2- 172.21.15.43:0/3520028722 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f41ec15c680 0x7f41ec1f1c10 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7f41d4009890 tx=0x7f41d40046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-09T13:35:11.843 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:11.839+0000 7f41e2ffd640 1 -- 172.21.15.43:0/3520028722 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f41d403a6f0 con 0x7f41ec15c680 2023-08-09T13:35:11.843 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:11.840+0000 7f41e2ffd640 1 -- 172.21.15.43:0/3520028722 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f41d403a870 con 0x7f41ec15c680 2023-08-09T13:35:11.843 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:11.840+0000 7f41e2ffd640 1 -- 172.21.15.43:0/3520028722 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f41d403ac20 con 0x7f41ec15c680 2023-08-09T13:35:11.843 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:11.840+0000 7f41f0c3f640 1 -- 172.21.15.43:0/3520028722 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f41ec0ca320 con 0x7f41ec15c680 2023-08-09T13:35:11.843 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:11.840+0000 7f41f0c3f640 1 -- 172.21.15.43:0/3520028722 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f41ec0cb4b0 con 0x7f41ec15c680 2023-08-09T13:35:11.844 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:11.841+0000 7f41f0c3f640 1 -- 172.21.15.43:0/3520028722 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f41ec0cb030 con 0x7f41ec15c680 2023-08-09T13:35:11.844 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:11.842+0000 7f41e2ffd640 1 -- 172.21.15.43:0/3520028722 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 5) v1 ==== 88341+0+0 (secure 0 0 0) 0x7f41d4043410 con 0x7f41ec15c680 2023-08-09T13:35:11.844 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:11.843+0000 7f41e2ffd640 1 --2- 172.21.15.43:0/3520028722 >> [v2:172.21.15.203:6832/830097474,v1:172.21.15.203:6833/830097474] conn(0x7f41cc06a050 0x7f41cc06c510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:35:11.844 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:11.843+0000 7f41e2ffd640 1 -- 172.21.15.43:0/3520028722 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 6871+0+0 (secure 0 0 0) 0x7f41d40aa490 con 0x7f41ec15c680 2023-08-09T13:35:11.845 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:11.843+0000 7f41ea59c640 1 --2- 172.21.15.43:0/3520028722 >> [v2:172.21.15.203:6832/830097474,v1:172.21.15.203:6833/830097474] conn(0x7f41cc06a050 0x7f41cc06c510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:35:11.846 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:11.845+0000 7f41ea59c640 1 --2- 172.21.15.43:0/3520028722 >> [v2:172.21.15.203:6832/830097474,v1:172.21.15.203:6833/830097474] conn(0x7f41cc06a050 0x7f41cc06c510 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7f41dc0099c0 tx=0x7f41dc003990 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-09T13:35:11.849 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:11.848+0000 7f41e2ffd640 1 -- 172.21.15.43:0/3520028722 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178066 (secure 0 0 0) 0x7f41ec0cb030 con 0x7f41ec15c680 2023-08-09T13:35:12.042 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:12.040+0000 7f41f0c3f640 1 -- 172.21.15.43:0/3520028722 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) v1 -- 0x7f41ec0cb030 con 0x7f41ec15c680 2023-08-09T13:35:12.042 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:12.041+0000 7f41e2ffd640 1 -- 172.21.15.43:0/3520028722 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v26) v1 ==== 103+0+11 (secure 0 0 0) 0x7f41ec0cb030 con 0x7f41ec15c680 2023-08-09T13:35:12.043 INFO:teuthology.orchestra.run.smithi043.stdout:pg_num: 32 2023-08-09T13:35:12.045 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:12.043+0000 7f41f0c3f640 1 -- 172.21.15.43:0/3520028722 >> [v2:172.21.15.203:6832/830097474,v1:172.21.15.203:6833/830097474] conn(0x7f41cc06a050 msgr2=0x7f41cc06c510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:35:12.045 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:12.043+0000 7f41f0c3f640 1 --2- 172.21.15.43:0/3520028722 >> [v2:172.21.15.203:6832/830097474,v1:172.21.15.203:6833/830097474] conn(0x7f41cc06a050 0x7f41cc06c510 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7f41dc0099c0 tx=0x7f41dc003990 comp rx=0 tx=0).stop 2023-08-09T13:35:12.045 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:12.043+0000 7f41f0c3f640 1 -- 172.21.15.43:0/3520028722 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f41ec15c680 msgr2=0x7f41ec1f1c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:35:12.046 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:12.043+0000 7f41f0c3f640 1 --2- 172.21.15.43:0/3520028722 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f41ec15c680 0x7f41ec1f1c10 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7f41d4009890 tx=0x7f41d40046c0 comp rx=0 tx=0).stop 2023-08-09T13:35:12.046 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:12.044+0000 7f41f0c3f640 1 -- 172.21.15.43:0/3520028722 shutdown_connections 2023-08-09T13:35:12.046 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:12.044+0000 7f41f0c3f640 1 --2- 172.21.15.43:0/3520028722 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f41ec1f2150 0x7f41ec0c95f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:35:12.046 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:12.044+0000 7f41f0c3f640 1 --2- 172.21.15.43:0/3520028722 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f41ec15c680 0x7f41ec1f1c10 unknown :-1 s=CLOSED pgs=287 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:35:12.046 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:12.044+0000 7f41f0c3f640 1 --2- 172.21.15.43:0/3520028722 >> [v2:172.21.15.203:6832/830097474,v1:172.21.15.203:6833/830097474] conn(0x7f41cc06a050 0x7f41cc06c510 unknown :-1 s=CLOSED pgs=157 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:35:12.046 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:12.044+0000 7f41f0c3f640 1 --2- 172.21.15.43:0/3520028722 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f41ec0d0a70 0x7f41ec153a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:35:12.046 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:12.044+0000 7f41f0c3f640 1 -- 172.21.15.43:0/3520028722 >> 172.21.15.43:0/3520028722 conn(0x7f41ec071520 msgr2=0x7f41ec158fa0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-09T13:35:12.047 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:12.044+0000 7f41f0c3f640 1 -- 172.21.15.43:0/3520028722 shutdown_connections 2023-08-09T13:35:12.047 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:12.044+0000 7f41f0c3f640 1 -- 172.21.15.43:0/3520028722 wait complete. 2023-08-09T13:35:12.065 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-08-09T13:35:12.296 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:12.295+0000 7f9bf4c63640 1 Processor -- start 2023-08-09T13:35:12.297 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:12.295+0000 7f9bf4c63640 1 -- start start 2023-08-09T13:35:12.298 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:12.296+0000 7f9bf4c63640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f9bf00d0ef0 0x7f9bf00cfe70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:35:12.298 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:12.296+0000 7f9bf4c63640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f9bf00cf4f0 0x7f9bf00b27c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:35:12.298 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:12.296+0000 7f9bf4c63640 1 -- --> v1:172.21.15.203:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9bf00b2330 con 0x7f9bf015c620 2023-08-09T13:35:12.298 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:12.296+0000 7f9bf4c63640 1 -- --> v2:172.21.15.43:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9bf00d2ec0 con 0x7f9bf00cf4f0 2023-08-09T13:35:12.298 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:12.296+0000 7f9bf4c63640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9bf00d0500 con 0x7f9bf00d0ef0 2023-08-09T13:35:12.299 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:12.297+0000 7f9beffff640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f9bf00cf4f0 0x7f9bf00b27c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:35:12.299 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:12.297+0000 7f9beffff640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f9bf00cf4f0 0x7f9bf00b27c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:37118/0 (socket says 172.21.15.43:37118) 2023-08-09T13:35:12.299 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:12.297+0000 7f9beffff640 1 -- 172.21.15.43:0/669103998 learned_addr learned my addr 172.21.15.43:0/669103998 (peer_addr_for_me v2:172.21.15.43:0/0) 2023-08-09T13:35:12.300 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:12.297+0000 7f9be6ffd640 1 --2- 172.21.15.43:0/669103998 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f9bf00d0ef0 0x7f9bf00cfe70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:35:12.300 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:12.298+0000 7f9beffff640 1 -- 172.21.15.43:0/669103998 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f9bf00d0ef0 msgr2=0x7f9bf00cfe70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-09T13:35:12.300 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:12.298+0000 7f9beffff640 1 --2- 172.21.15.43:0/669103998 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f9bf00d0ef0 0x7f9bf00cfe70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:35:12.300 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:12.298+0000 7f9beffff640 1 -- 172.21.15.43:0/669103998 >> v1:172.21.15.203:6789/0 conn(0x7f9bf015c620 legacy=0x7f9bf00d07e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:35:12.300 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:12.298+0000 7f9beffff640 1 -- 172.21.15.43:0/669103998 --> v2:172.21.15.43:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9bf00b1a00 con 0x7f9bf00cf4f0 2023-08-09T13:35:12.301 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:12.299+0000 7f9beffff640 1 --2- 172.21.15.43:0/669103998 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f9bf00cf4f0 0x7f9bf00b27c0 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7f9be8009e50 tx=0x7f9be802bed0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5ca62f4152399311 server_cookie=0 in_seq=0 out_seq=0 2023-08-09T13:35:12.301 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:12.300+0000 7f9beeffd640 1 -- 172.21.15.43:0/669103998 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9be802ca70 con 0x7f9bf00cf4f0 2023-08-09T13:35:12.301 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:12.300+0000 7f9beeffd640 1 -- 172.21.15.43:0/669103998 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9be802cbd0 con 0x7f9bf00cf4f0 2023-08-09T13:35:12.301 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:12.300+0000 7f9beeffd640 1 -- 172.21.15.43:0/669103998 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9be80343f0 con 0x7f9bf00cf4f0 2023-08-09T13:35:12.302 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:12.300+0000 7f9bf4c63640 1 -- 172.21.15.43:0/669103998 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f9bf00cf4f0 msgr2=0x7f9bf00b27c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:35:12.302 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:12.300+0000 7f9bf4c63640 1 --2- 172.21.15.43:0/669103998 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f9bf00cf4f0 0x7f9bf00b27c0 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7f9be8009e50 tx=0x7f9be802bed0 comp rx=0 tx=0).stop 2023-08-09T13:35:12.302 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:12.300+0000 7f9bf4c63640 1 -- 172.21.15.43:0/669103998 shutdown_connections 2023-08-09T13:35:12.302 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:12.300+0000 7f9bf4c63640 1 --2- 172.21.15.43:0/669103998 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f9bf00cf4f0 0x7f9bf00b27c0 unknown :-1 s=CLOSED pgs=288 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:35:12.303 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:12.300+0000 7f9bf4c63640 1 --2- 172.21.15.43:0/669103998 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f9bf00d0ef0 0x7f9bf00cfe70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:35:12.303 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:12.300+0000 7f9bf4c63640 1 -- 172.21.15.43:0/669103998 >> 172.21.15.43:0/669103998 conn(0x7f9bf009b650 msgr2=0x7f9bf00d22a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-09T13:35:12.303 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:12.301+0000 7f9bf4c63640 1 -- 172.21.15.43:0/669103998 shutdown_connections 2023-08-09T13:35:12.303 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:12.301+0000 7f9bf4c63640 1 -- 172.21.15.43:0/669103998 wait complete. 2023-08-09T13:35:12.303 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:12.302+0000 7f9bf4c63640 1 Processor -- start 2023-08-09T13:35:12.304 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:12.303+0000 7f9bf4c63640 1 -- start start 2023-08-09T13:35:12.305 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:12.303+0000 7f9bf4c63640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f9bf00cf4f0 0x7f9bf01f12a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:35:12.305 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:12.303+0000 7f9bf4c63640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f9bf00d0ef0 0x7f9bf00c6f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:35:12.305 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:12.304+0000 7f9bf4c63640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f9bf015c620 0x7f9bf00c7440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:35:12.305 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:12.304+0000 7f9bf4c63640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9bf00b7c30 con 0x7f9bf00cf4f0 2023-08-09T13:35:12.306 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:12.304+0000 7f9bf4c63640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9bf00c86d0 con 0x7f9bf015c620 2023-08-09T13:35:12.306 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:12.304+0000 7f9bf4c63640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9bf00c7ac0 con 0x7f9bf00d0ef0 2023-08-09T13:35:12.306 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:12.304+0000 7f9bef7fe640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f9bf00cf4f0 0x7f9bf01f12a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:35:12.306 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:12.304+0000 7f9bef7fe640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f9bf00cf4f0 0x7f9bf01f12a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:37134/0 (socket says 172.21.15.43:37134) 2023-08-09T13:35:12.306 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:12.304+0000 7f9bef7fe640 1 -- 172.21.15.43:0/1212016239 learned_addr learned my addr 172.21.15.43:0/1212016239 (peer_addr_for_me v2:172.21.15.43:0/0) 2023-08-09T13:35:12.307 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:12.304+0000 7f9be6ffd640 1 --2- 172.21.15.43:0/1212016239 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f9bf00d0ef0 0x7f9bf00c6f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:35:12.307 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:12.304+0000 7f9beffff640 1 --2- 172.21.15.43:0/1212016239 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f9bf015c620 0x7f9bf00c7440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:35:12.307 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:12.304+0000 7f9bef7fe640 1 -- 172.21.15.43:0/1212016239 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f9bf00d0ef0 msgr2=0x7f9bf00c6f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:35:12.308 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:12.304+0000 7f9bef7fe640 1 --2- 172.21.15.43:0/1212016239 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f9bf00d0ef0 0x7f9bf00c6f00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:35:12.308 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:12.304+0000 7f9bef7fe640 1 -- 172.21.15.43:0/1212016239 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f9bf015c620 msgr2=0x7f9bf00c7440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:35:12.308 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:12.304+0000 7f9bef7fe640 1 --2- 172.21.15.43:0/1212016239 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f9bf015c620 0x7f9bf00c7440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:35:12.308 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:12.304+0000 7f9bef7fe640 1 -- 172.21.15.43:0/1212016239 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9be8009b30 con 0x7f9bf00cf4f0 2023-08-09T13:35:12.308 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:12.305+0000 7f9bef7fe640 1 --2- 172.21.15.43:0/1212016239 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f9bf00cf4f0 0x7f9bf01f12a0 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7f9be000c3a0 tx=0x7f9be000c860 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-09T13:35:12.308 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:12.305+0000 7f9becff9640 1 -- 172.21.15.43:0/1212016239 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9be00052d0 con 0x7f9bf00cf4f0 2023-08-09T13:35:12.308 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:12.305+0000 7f9becff9640 1 -- 172.21.15.43:0/1212016239 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9be0005450 con 0x7f9bf00cf4f0 2023-08-09T13:35:12.308 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:12.305+0000 7f9becff9640 1 -- 172.21.15.43:0/1212016239 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9be0003880 con 0x7f9bf00cf4f0 2023-08-09T13:35:12.309 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:12.305+0000 7f9bf4c63640 1 -- 172.21.15.43:0/1212016239 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9bf00c7da0 con 0x7f9bf00cf4f0 2023-08-09T13:35:12.309 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:12.305+0000 7f9bf4c63640 1 -- 172.21.15.43:0/1212016239 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9bf00c8360 con 0x7f9bf00cf4f0 2023-08-09T13:35:12.309 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:12.306+0000 7f9bf4c63640 1 -- 172.21.15.43:0/1212016239 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9bf00c8520 con 0x7f9bf00cf4f0 2023-08-09T13:35:12.310 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:12.307+0000 7f9becff9640 1 -- 172.21.15.43:0/1212016239 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 5) v1 ==== 88341+0+0 (secure 0 0 0) 0x7f9be00039e0 con 0x7f9bf00cf4f0 2023-08-09T13:35:12.310 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:12.308+0000 7f9becff9640 1 --2- 172.21.15.43:0/1212016239 >> [v2:172.21.15.203:6832/830097474,v1:172.21.15.203:6833/830097474] conn(0x7f9bd006a050 0x7f9bd006c510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:35:12.310 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:12.308+0000 7f9becff9640 1 -- 172.21.15.43:0/1212016239 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 6871+0+0 (secure 0 0 0) 0x7f9be0087ed0 con 0x7f9bf00cf4f0 2023-08-09T13:35:12.310 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:12.309+0000 7f9be6ffd640 1 --2- 172.21.15.43:0/1212016239 >> [v2:172.21.15.203:6832/830097474,v1:172.21.15.203:6833/830097474] conn(0x7f9bd006a050 0x7f9bd006c510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:35:12.311 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:12.310+0000 7f9be6ffd640 1 --2- 172.21.15.43:0/1212016239 >> [v2:172.21.15.203:6832/830097474,v1:172.21.15.203:6833/830097474] conn(0x7f9bd006a050 0x7f9bd006c510 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7f9bf00b2900 tx=0x7f9bd8008040 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-09T13:35:12.314 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:12.312+0000 7f9becff9640 1 -- 172.21.15.43:0/1212016239 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178066 (secure 0 0 0) 0x7f9be00531f0 con 0x7f9bf00cf4f0 2023-08-09T13:35:12.509 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:12.508+0000 7f9bf4c63640 1 -- 172.21.15.43:0/1212016239 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f9bf00c89b0 con 0x7f9bf00cf4f0 2023-08-09T13:35:12.513 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:12.511+0000 7f9becff9640 1 -- 172.21.15.43:0/1212016239 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v26) v1 ==== 74+0+20146 (secure 0 0 0) 0x7f9be0056810 con 0x7f9bf00cf4f0 2023-08-09T13:35:12.514 INFO:teuthology.orchestra.run.smithi043.stdout: 2023-08-09T13:35:12.514 INFO:teuthology.orchestra.run.smithi043.stdout:{"epoch":26,"fsid":"daac87ee-1337-4250-a955-aea4e408a39b","created":"2023-08-09T13:33:41.701198+0000","modified":"2023-08-09T13:34:48.711911+0000","last_up_change":"2023-08-09T13:33:53.783453+0000","last_in_change":"2023-08-09T13:33:45.148904+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":5,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":"rbd","create_time":"2023-08-09T13:33:54.365137+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":".mgr","create_time":"2023-08-09T13:33:54.831803+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_metadata","create_time":"2023-08-09T13:34:00.207974+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data","create_time":"2023-08-09T13:34:01.614765+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":5,"pool_name":"cephfs_data_ec","create_time":"2023-08-09T13:34:05.239583+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"24","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"cbd49ecd-e896-4fbd-864e-5c37a829b165","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6800","nonce":502667335},{"type":"v1","addr":"172.21.15.43:6801","nonce":502667335}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6802","nonce":502667335},{"type":"v1","addr":"172.21.15.43:6803","nonce":502667335}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6806","nonce":502667335},{"type":"v1","addr":"172.21.15.43:6807","nonce":502667335}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6804","nonce":502667335},{"type":"v1","addr":"172.21.15.43:6805","nonce":502667335}]},"public_addr":"172.21.15.43:6801/502667335","cluster_addr":"172.21.15.43:6803/502667335","heartbeat_back_addr":"172.21.15.43:6807/502667335","heartbeat_front_addr":"172.21.15.43:6805/502667335","state":["exists","up"]},{"osd":1,"uuid":"75c7e0fe-1ec7-4fe8-a2f2-3661a65b8500","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6808","nonce":3289134610},{"type":"v1","addr":"172.21.15.43:6809","nonce":3289134610}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6810","nonce":3289134610},{"type":"v1","addr":"172.21.15.43:6812","nonce":3289134610}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6821","nonce":3289134610},{"type":"v1","addr":"172.21.15.43:6824","nonce":3289134610}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6815","nonce":3289134610},{"type":"v1","addr":"172.21.15.43:6818","nonce":3289134610}]},"public_addr":"172.21.15.43:6809/3289134610","cluster_addr":"172.21.15.43:6812/3289134610","heartbeat_back_addr":"172.21.15.43:6824/3289134610","heartbeat_front_addr":"172.21.15.43:6818/3289134610","state":["exists","up"]},{"osd":2,"uuid":"bb5c41e9-7f5f-4125-9b01-3bb8bfc2ea83","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6811","nonce":4079709736},{"type":"v1","addr":"172.21.15.43:6814","nonce":4079709736}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6817","nonce":4079709736},{"type":"v1","addr":"172.21.15.43:6820","nonce":4079709736}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6829","nonce":4079709736},{"type":"v1","addr":"172.21.15.43:6831","nonce":4079709736}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6823","nonce":4079709736},{"type":"v1","addr":"172.21.15.43:6826","nonce":4079709736}]},"public_addr":"172.21.15.43:6814/4079709736","cluster_addr":"172.21.15.43:6820/4079709736","heartbeat_back_addr":"172.21.15.43:6831/4079709736","heartbeat_front_addr":"172.21.15.43:6826/4079709736","state":["exists","up"]},{"osd":3,"uuid":"bfe4c8b4-694a-465c-87e6-2b2aab852613","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6813","nonce":1709538759},{"type":"v1","addr":"172.21.15.43:6816","nonce":1709538759}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6819","nonce":1709538759},{"type":"v1","addr":"172.21.15.43:6822","nonce":1709538759}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6828","nonce":1709538759},{"type":"v1","addr":"172.21.15.43:6830","nonce":1709538759}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6825","nonce":1709538759},{"type":"v1","addr":"172.21.15.43:6827","nonce":1709538759}]},"public_addr":"172.21.15.43:6816/1709538759","cluster_addr":"172.21.15.43:6822/1709538759","heartbeat_back_addr":"172.21.15.43:6830/1709538759","heartbeat_front_addr":"172.21.15.43:6827/1709538759","state":["exists","up"]},{"osd":4,"uuid":"fa2e8a77-aba1-4756-853e-0d174a5bdeba","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6800","nonce":4196866481},{"type":"v1","addr":"172.21.15.203:6801","nonce":4196866481}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6802","nonce":4196866481},{"type":"v1","addr":"172.21.15.203:6803","nonce":4196866481}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6806","nonce":4196866481},{"type":"v1","addr":"172.21.15.203:6807","nonce":4196866481}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6804","nonce":4196866481},{"type":"v1","addr":"172.21.15.203:6805","nonce":4196866481}]},"public_addr":"172.21.15.203:6801/4196866481","cluster_addr":"172.21.15.203:6803/4196866481","heartbeat_back_addr":"172.21.15.203:6807/4196866481","heartbeat_front_addr":"172.21.15.203:6805/4196866481","state":["exists","up"]},{"osd":5,"uuid":"81811e4b-4d24-4f51-a82c-f1d700a2a00d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6816","nonce":3059290081},{"type":"v1","addr":"172.21.15.203:6817","nonce":3059290081}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6818","nonce":3059290081},{"type":"v1","addr":"172.21.15.203:6819","nonce":3059290081}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6822","nonce":3059290081},{"type":"v1","addr":"172.21.15.203:6823","nonce":3059290081}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6820","nonce":3059290081},{"type":"v1","addr":"172.21.15.203:6821","nonce":3059290081}]},"public_addr":"172.21.15.203:6817/3059290081","cluster_addr":"172.21.15.203:6819/3059290081","heartbeat_back_addr":"172.21.15.203:6823/3059290081","heartbeat_front_addr":"172.21.15.203:6821/3059290081","state":["exists","up"]},{"osd":6,"uuid":"307bac4d-6a5b-443d-af75-56d695d7674f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6824","nonce":512155569},{"type":"v1","addr":"172.21.15.203:6825","nonce":512155569}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6826","nonce":512155569},{"type":"v1","addr":"172.21.15.203:6827","nonce":512155569}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6830","nonce":512155569},{"type":"v1","addr":"172.21.15.203:6831","nonce":512155569}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6828","nonce":512155569},{"type":"v1","addr":"172.21.15.203:6829","nonce":512155569}]},"public_addr":"172.21.15.203:6825/512155569","cluster_addr":"172.21.15.203:6827/512155569","heartbeat_back_addr":"172.21.15.203:6831/512155569","heartbeat_front_addr":"172.21.15.203:6829/512155569","state":["exists","up"]},{"osd":7,"uuid":"a95dc7ca-9f1b-4295-af70-68a22a2d7d5e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6808","nonce":3331450438},{"type":"v1","addr":"172.21.15.203:6809","nonce":3331450438}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6810","nonce":3331450438},{"type":"v1","addr":"172.21.15.203:6811","nonce":3331450438}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6814","nonce":3331450438},{"type":"v1","addr":"172.21.15.203:6815","nonce":3331450438}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6812","nonce":3331450438},{"type":"v1","addr":"172.21.15.203:6813","nonce":3331450438}]},"public_addr":"172.21.15.203:6809/3331450438","cluster_addr":"172.21.15.203:6811/3331450438","heartbeat_back_addr":"172.21.15.203:6815/3331450438","heartbeat_front_addr":"172.21.15.203:6813/3331450438","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-09T13:33:50.547652+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-09T13:33:50.589469+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-09T13:33:50.561724+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-09T13:33:50.599650+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-09T13:33:50.411855+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-09T13:33:50.219681+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-09T13:33:50.504664+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-09T13:33:50.441248+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"4.14","mappings":[{"from":1,"to":5}]},{"pgid":"4.29","mappings":[{"from":3,"to":2}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-08-09T13:35:12.515 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:12.514+0000 7f9bf4c63640 1 -- 172.21.15.43:0/1212016239 >> [v2:172.21.15.203:6832/830097474,v1:172.21.15.203:6833/830097474] conn(0x7f9bd006a050 msgr2=0x7f9bd006c510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:35:12.515 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:12.514+0000 7f9bf4c63640 1 --2- 172.21.15.43:0/1212016239 >> [v2:172.21.15.203:6832/830097474,v1:172.21.15.203:6833/830097474] conn(0x7f9bd006a050 0x7f9bd006c510 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7f9bf00b2900 tx=0x7f9bd8008040 comp rx=0 tx=0).stop 2023-08-09T13:35:12.515 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:12.514+0000 7f9bf4c63640 1 -- 172.21.15.43:0/1212016239 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f9bf00cf4f0 msgr2=0x7f9bf01f12a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:35:12.516 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:12.514+0000 7f9bf4c63640 1 --2- 172.21.15.43:0/1212016239 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f9bf00cf4f0 0x7f9bf01f12a0 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7f9be000c3a0 tx=0x7f9be000c860 comp rx=0 tx=0).stop 2023-08-09T13:35:12.516 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:12.514+0000 7f9bf4c63640 1 -- 172.21.15.43:0/1212016239 shutdown_connections 2023-08-09T13:35:12.516 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:12.514+0000 7f9bf4c63640 1 --2- 172.21.15.43:0/1212016239 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f9bf015c620 0x7f9bf00c7440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:35:12.516 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:12.514+0000 7f9bf4c63640 1 --2- 172.21.15.43:0/1212016239 >> [v2:172.21.15.203:6832/830097474,v1:172.21.15.203:6833/830097474] conn(0x7f9bd006a050 0x7f9bd006c510 unknown :-1 s=CLOSED pgs=158 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:35:12.517 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:12.514+0000 7f9bf4c63640 1 --2- 172.21.15.43:0/1212016239 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f9bf00d0ef0 0x7f9bf00c6f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:35:12.517 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:12.514+0000 7f9bf4c63640 1 --2- 172.21.15.43:0/1212016239 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f9bf00cf4f0 0x7f9bf01f12a0 unknown :-1 s=CLOSED pgs=289 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:35:12.517 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:12.514+0000 7f9bf4c63640 1 -- 172.21.15.43:0/1212016239 >> 172.21.15.43:0/1212016239 conn(0x7f9bf009b650 msgr2=0x7f9bf00ce8d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-09T13:35:12.517 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:12.514+0000 7f9bf4c63640 1 -- 172.21.15.43:0/1212016239 shutdown_connections 2023-08-09T13:35:12.517 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:12.514+0000 7f9bf4c63640 1 -- 172.21.15.43:0/1212016239 wait complete. 2023-08-09T13:35:12.533 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': 'rbd', 'create_time': '2023-08-09T13:33:54.365137+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}, 'read_balance': {'score_acting': 2, 'score_stable': 2, 'optimal_score': 1, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': '.mgr', 'create_time': '2023-08-09T13:33:54.831803+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_metadata', 'create_time': '2023-08-09T13:34:00.207974+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data', 'create_time': '2023-08-09T13:34:01.614765+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 2, 'score_stable': 2, 'optimal_score': 1, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 5, 'pool_name': 'cephfs_data_ec', 'create_time': '2023-08-09T13:34:05.239583+0000', 'flags': 5, 'flags_names': 'hashpspool,ec_overwrites', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '24', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2023-08-09T13:35:12.534 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get rbd pg_num 2023-08-09T13:35:12.760 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:12.758+0000 7fa40be04640 1 Processor -- start 2023-08-09T13:35:12.760 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:12.758+0000 7fa40be04640 1 -- start start 2023-08-09T13:35:12.761 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:12.760+0000 7fa40be04640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7fa4040cb2d0 0x7fa40415b380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:35:12.762 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:12.760+0000 7fa40be04640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fa4040ce8d0 0x7fa4040c9f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:35:12.762 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:12.760+0000 7fa40be04640 1 -- --> v1:172.21.15.43:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa40415aef0 con 0x7fa40415df60 2023-08-09T13:35:12.762 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:12.760+0000 7fa40be04640 1 -- --> v2:172.21.15.43:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa40415a970 con 0x7fa4040cb2d0 2023-08-09T13:35:12.762 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:12.760+0000 7fa40be04640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa40415b8c0 con 0x7fa4040ce8d0 2023-08-09T13:35:12.763 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:12.760+0000 7fa409ba0640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7fa4040cb2d0 0x7fa40415b380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:35:12.763 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:12.760+0000 7fa40939f640 1 --1- >> v1:172.21.15.43:6789/0 conn(0x7fa40415df60 0x7fa4040ce1c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.43:6789/0 says I am v1:172.21.15.43:57580/0 (socket says 172.21.15.43:57580) 2023-08-09T13:35:12.763 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:12.760+0000 7fa40939f640 1 -- 172.21.15.43:0/357246584 learned_addr learned my addr 172.21.15.43:0/357246584 (peer_addr_for_me v1:172.21.15.43:0/0) 2023-08-09T13:35:12.763 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:12.761+0000 7fa40a3a1640 1 --2- 172.21.15.43:0/357246584 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fa4040ce8d0 0x7fa4040c9f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:35:12.763 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:12.761+0000 7fa409ba0640 1 -- 172.21.15.43:0/357246584 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fa4040ce8d0 msgr2=0x7fa4040c9f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-09T13:35:12.764 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:12.761+0000 7fa409ba0640 1 --2- 172.21.15.43:0/357246584 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fa4040ce8d0 0x7fa4040c9f90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:35:12.764 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:12.761+0000 7fa409ba0640 1 -- 172.21.15.43:0/357246584 >> v1:172.21.15.43:6789/0 conn(0x7fa40415df60 legacy=0x7fa4040ce1c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:35:12.764 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:12.761+0000 7fa409ba0640 1 -- 172.21.15.43:0/357246584 --> v2:172.21.15.43:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa40415e940 con 0x7fa4040cb2d0 2023-08-09T13:35:12.764 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:12.761+0000 7fa408b9e640 1 -- 172.21.15.43:0/357246584 <== mon.0 v1:172.21.15.43:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1367710471 0 0) 0x7fa40415aef0 con 0x7fa40415df60 2023-08-09T13:35:12.765 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:12.762+0000 7fa409ba0640 1 --2- 172.21.15.43:0/357246584 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa4040cb2d0 0x7fa40415b380 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x7fa4000097a0 tx=0x7fa40002b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=89ef137a3615684b server_cookie=0 in_seq=0 out_seq=0 2023-08-09T13:35:12.765 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:12.762+0000 7fa408b9e640 1 -- 172.21.15.43:0/357246584 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa400031830 con 0x7fa4040cb2d0 2023-08-09T13:35:12.765 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:12.762+0000 7fa408b9e640 1 -- 172.21.15.43:0/357246584 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa400031990 con 0x7fa4040cb2d0 2023-08-09T13:35:12.766 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:12.762+0000 7fa408b9e640 1 -- 172.21.15.43:0/357246584 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa400031d20 con 0x7fa4040cb2d0 2023-08-09T13:35:12.766 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:12.763+0000 7fa40be04640 1 -- 172.21.15.43:0/357246584 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa4040cb2d0 msgr2=0x7fa40415b380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:35:12.766 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:12.763+0000 7fa40be04640 1 --2- 172.21.15.43:0/357246584 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa4040cb2d0 0x7fa40415b380 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x7fa4000097a0 tx=0x7fa40002b8d0 comp rx=0 tx=0).stop 2023-08-09T13:35:12.766 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:12.763+0000 7fa40be04640 1 -- 172.21.15.43:0/357246584 shutdown_connections 2023-08-09T13:35:12.766 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:12.763+0000 7fa40be04640 1 --2- 172.21.15.43:0/357246584 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fa4040ce8d0 0x7fa4040c9f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:35:12.766 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:12.763+0000 7fa40be04640 1 --2- 172.21.15.43:0/357246584 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa4040cb2d0 0x7fa40415b380 unknown :-1 s=CLOSED pgs=291 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:35:12.766 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:12.763+0000 7fa40be04640 1 -- 172.21.15.43:0/357246584 >> 172.21.15.43:0/357246584 conn(0x7fa4040aff50 msgr2=0x7fa4040b0360 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-09T13:35:12.766 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:12.763+0000 7fa40be04640 1 -- 172.21.15.43:0/357246584 shutdown_connections 2023-08-09T13:35:12.767 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:12.763+0000 7fa40be04640 1 -- 172.21.15.43:0/357246584 wait complete. 2023-08-09T13:35:12.767 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:12.765+0000 7fa40be04640 1 Processor -- start 2023-08-09T13:35:12.767 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:12.765+0000 7fa40be04640 1 -- start start 2023-08-09T13:35:12.769 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:12.766+0000 7fa40be04640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fa4040cb2d0 0x7fa4041ef5f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:35:12.769 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:12.766+0000 7fa40be04640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fa4040ce8d0 0x7fa4040c9ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:35:12.769 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:12.766+0000 7fa40be04640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa40415df60 0x7fa4040b1ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:35:12.769 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:12.766+0000 7fa40be04640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa404155000 con 0x7fa40415df60 2023-08-09T13:35:12.769 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:12.766+0000 7fa40be04640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa4041552e0 con 0x7fa4040ce8d0 2023-08-09T13:35:12.769 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:12.766+0000 7fa40be04640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa4041555c0 con 0x7fa4040cb2d0 2023-08-09T13:35:12.770 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:12.766+0000 7fa40a3a1640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa40415df60 0x7fa4040b1ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:35:12.770 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:12.766+0000 7fa40a3a1640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa40415df60 0x7fa4040b1ef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:37152/0 (socket says 172.21.15.43:37152) 2023-08-09T13:35:12.770 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:12.766+0000 7fa40a3a1640 1 -- 172.21.15.43:0/1348448990 learned_addr learned my addr 172.21.15.43:0/1348448990 (peer_addr_for_me v2:172.21.15.43:0/0) 2023-08-09T13:35:12.770 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:12.767+0000 7fa40a3a1640 1 -- 172.21.15.43:0/1348448990 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fa4040cb2d0 msgr2=0x7fa4041ef5f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:35:12.770 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:12.767+0000 7fa40a3a1640 1 --2- 172.21.15.43:0/1348448990 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fa4040cb2d0 0x7fa4041ef5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:35:12.770 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:12.767+0000 7fa40a3a1640 1 -- 172.21.15.43:0/1348448990 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fa4040ce8d0 msgr2=0x7fa4040c9ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:35:12.770 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:12.767+0000 7fa40a3a1640 1 --2- 172.21.15.43:0/1348448990 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fa4040ce8d0 0x7fa4040c9ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:35:12.770 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:12.767+0000 7fa40a3a1640 1 -- 172.21.15.43:0/1348448990 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa400009400 con 0x7fa40415df60 2023-08-09T13:35:12.771 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:12.767+0000 7fa40a3a1640 1 --2- 172.21.15.43:0/1348448990 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa40415df60 0x7fa4040b1ef0 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7fa3fc007e40 tx=0x7fa3fc00e4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-09T13:35:12.771 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:12.767+0000 7fa3fa7fc640 1 -- 172.21.15.43:0/1348448990 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa3fc014620 con 0x7fa40415df60 2023-08-09T13:35:12.771 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:12.767+0000 7fa3fa7fc640 1 -- 172.21.15.43:0/1348448990 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa3fc00ee90 con 0x7fa40415df60 2023-08-09T13:35:12.771 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:12.767+0000 7fa3fa7fc640 1 -- 172.21.15.43:0/1348448990 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa3fc0149e0 con 0x7fa40415df60 2023-08-09T13:35:12.771 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:12.767+0000 7fa40be04640 1 -- 172.21.15.43:0/1348448990 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa4041558a0 con 0x7fa40415df60 2023-08-09T13:35:12.772 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:12.768+0000 7fa40be04640 1 -- 172.21.15.43:0/1348448990 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa4041efb30 con 0x7fa40415df60 2023-08-09T13:35:12.772 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:12.769+0000 7fa40be04640 1 -- 172.21.15.43:0/1348448990 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa4040739f0 con 0x7fa40415df60 2023-08-09T13:35:12.772 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:12.770+0000 7fa3fa7fc640 1 -- 172.21.15.43:0/1348448990 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 5) v1 ==== 88341+0+0 (secure 0 0 0) 0x7fa3fc014b40 con 0x7fa40415df60 2023-08-09T13:35:12.772 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:12.770+0000 7fa3fa7fc640 1 --2- 172.21.15.43:0/1348448990 >> [v2:172.21.15.203:6832/830097474,v1:172.21.15.203:6833/830097474] conn(0x7fa3e806a050 0x7fa3e806c510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:35:12.772 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:12.771+0000 7fa3fa7fc640 1 -- 172.21.15.43:0/1348448990 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 6871+0+0 (secure 0 0 0) 0x7fa3fc089360 con 0x7fa40415df60 2023-08-09T13:35:12.773 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:12.771+0000 7fa409ba0640 1 --2- 172.21.15.43:0/1348448990 >> [v2:172.21.15.203:6832/830097474,v1:172.21.15.203:6833/830097474] conn(0x7fa3e806a050 0x7fa3e806c510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:35:12.774 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:12.773+0000 7fa409ba0640 1 --2- 172.21.15.43:0/1348448990 >> [v2:172.21.15.203:6832/830097474,v1:172.21.15.203:6833/830097474] conn(0x7fa3e806a050 0x7fa3e806c510 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7fa400036000 tx=0x7fa400030000 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-09T13:35:12.776 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:12.775+0000 7fa3fa7fc640 1 -- 172.21.15.43:0/1348448990 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178066 (secure 0 0 0) 0x7fa3fc054680 con 0x7fa40415df60 2023-08-09T13:35:12.975 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:12.973+0000 7fa40be04640 1 -- 172.21.15.43:0/1348448990 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"} v 0) v1 -- 0x7fa40415a3b0 con 0x7fa40415df60 2023-08-09T13:35:12.976 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:12.974+0000 7fa3fa7fc640 1 -- 172.21.15.43:0/1348448990 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"}]=0 v26) v1 ==== 92+0+10 (secure 0 0 0) 0x7fa3fc057ca0 con 0x7fa40415df60 2023-08-09T13:35:12.976 INFO:teuthology.orchestra.run.smithi043.stdout:pg_num: 8 2023-08-09T13:35:12.979 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:12.977+0000 7fa40be04640 1 -- 172.21.15.43:0/1348448990 >> [v2:172.21.15.203:6832/830097474,v1:172.21.15.203:6833/830097474] conn(0x7fa3e806a050 msgr2=0x7fa3e806c510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:35:12.979 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:12.977+0000 7fa40be04640 1 --2- 172.21.15.43:0/1348448990 >> [v2:172.21.15.203:6832/830097474,v1:172.21.15.203:6833/830097474] conn(0x7fa3e806a050 0x7fa3e806c510 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7fa400036000 tx=0x7fa400030000 comp rx=0 tx=0).stop 2023-08-09T13:35:12.980 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:12.977+0000 7fa40be04640 1 -- 172.21.15.43:0/1348448990 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa40415df60 msgr2=0x7fa4040b1ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:35:12.980 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:12.977+0000 7fa40be04640 1 --2- 172.21.15.43:0/1348448990 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa40415df60 0x7fa4040b1ef0 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7fa3fc007e40 tx=0x7fa3fc00e4b0 comp rx=0 tx=0).stop 2023-08-09T13:35:12.980 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:12.978+0000 7fa40be04640 1 -- 172.21.15.43:0/1348448990 shutdown_connections 2023-08-09T13:35:12.980 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:12.978+0000 7fa40be04640 1 --2- 172.21.15.43:0/1348448990 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa40415df60 0x7fa4040b1ef0 unknown :-1 s=CLOSED pgs=292 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:35:12.980 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:12.978+0000 7fa40be04640 1 --2- 172.21.15.43:0/1348448990 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fa4040ce8d0 0x7fa4040c9ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:35:12.980 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:12.978+0000 7fa40be04640 1 --2- 172.21.15.43:0/1348448990 >> [v2:172.21.15.203:6832/830097474,v1:172.21.15.203:6833/830097474] conn(0x7fa3e806a050 0x7fa3e806c510 unknown :-1 s=CLOSED pgs=159 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:35:12.981 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:12.978+0000 7fa40be04640 1 --2- 172.21.15.43:0/1348448990 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fa4040cb2d0 0x7fa4041ef5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:35:12.981 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:12.978+0000 7fa40be04640 1 -- 172.21.15.43:0/1348448990 >> 172.21.15.43:0/1348448990 conn(0x7fa4040aff50 msgr2=0x7fa40406f410 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-09T13:35:12.981 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:12.978+0000 7fa40be04640 1 -- 172.21.15.43:0/1348448990 shutdown_connections 2023-08-09T13:35:12.981 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:12.978+0000 7fa40be04640 1 -- 172.21.15.43:0/1348448990 wait complete. 2023-08-09T13:35:13.000 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-08-09T13:35:13.233 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:13.231+0000 7efd8d4ec640 1 Processor -- start 2023-08-09T13:35:13.233 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:13.231+0000 7efd8d4ec640 1 -- start start 2023-08-09T13:35:13.234 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:13.233+0000 7efd8d4ec640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7efd880ce8d0 0x7efd880c9f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:35:13.234 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:13.233+0000 7efd8d4ec640 1 -- --> v1:172.21.15.43:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7efd8815aef0 con 0x7efd8815df60 2023-08-09T13:35:13.235 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:13.233+0000 7efd8d4ec640 1 -- --> v1:172.21.15.203:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7efd8815e940 con 0x7efd880cb2d0 2023-08-09T13:35:13.235 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:13.233+0000 7efd8d4ec640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7efd8815a970 con 0x7efd880ce8d0 2023-08-09T13:35:13.236 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:13.233+0000 7efd867fc640 1 --1- >> v1:172.21.15.43:6789/0 conn(0x7efd8815df60 0x7efd880ce1c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.43:6789/0 says I am v1:172.21.15.43:57582/0 (socket says 172.21.15.43:57582) 2023-08-09T13:35:13.236 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:13.233+0000 7efd867fc640 1 -- 172.21.15.43:0/1588411264 learned_addr learned my addr 172.21.15.43:0/1588411264 (peer_addr_for_me v1:172.21.15.43:0/0) 2023-08-09T13:35:13.236 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:13.234+0000 7efd877fe640 1 --2- 172.21.15.43:0/1588411264 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7efd880ce8d0 0x7efd880c9f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:35:13.237 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:13.234+0000 7efd85ffb640 1 -- 172.21.15.43:0/1588411264 <== mon.0 v1:172.21.15.43:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 568124451 0 0) 0x7efd8815aef0 con 0x7efd8815df60 2023-08-09T13:35:13.237 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:13.234+0000 7efd85ffb640 1 -- 172.21.15.43:0/1588411264 --> v1:172.21.15.43:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7efd70003520 con 0x7efd8815df60 2023-08-09T13:35:13.237 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:13.234+0000 7efd85ffb640 1 -- 172.21.15.43:0/1588411264 <== mon.0 v1:172.21.15.43:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3974973623 0 0) 0x7efd70003520 con 0x7efd8815df60 2023-08-09T13:35:13.237 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:13.234+0000 7efd85ffb640 1 -- 172.21.15.43:0/1588411264 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7efd880ce8d0 msgr2=0x7efd880c9f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-09T13:35:13.237 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:13.234+0000 7efd85ffb640 1 --2- 172.21.15.43:0/1588411264 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7efd880ce8d0 0x7efd880c9f90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:35:13.237 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:13.234+0000 7efd85ffb640 1 -- 172.21.15.43:0/1588411264 >> v1:172.21.15.203:6789/0 conn(0x7efd880cb2d0 legacy=0x7efd8815b380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:35:13.237 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:13.234+0000 7efd85ffb640 1 -- 172.21.15.43:0/1588411264 --> v1:172.21.15.43:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efd8815ec20 con 0x7efd8815df60 2023-08-09T13:35:13.238 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:13.234+0000 7efd85ffb640 1 -- 172.21.15.43:0/1588411264 <== mon.0 v1:172.21.15.43:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1285150740 0 0) 0x7efd7c001670 con 0x7efd8815df60 2023-08-09T13:35:13.238 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:13.234+0000 7efd85ffb640 1 -- 172.21.15.43:0/1588411264 >> v1:172.21.15.43:6789/0 conn(0x7efd8815df60 legacy=0x7efd880ce1c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:35:13.238 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:13.234+0000 7efd85ffb640 1 --2- 172.21.15.43:0/1588411264 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7efd700039a0 0x7efd70003dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:35:13.238 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:13.234+0000 7efd85ffb640 1 --2- 172.21.15.43:0/1588411264 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7efd700047c0 0x7efd70004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:35:13.238 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:13.235+0000 7efd85ffb640 1 --2- 172.21.15.43:0/1588411264 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7efd880ce8d0 0x7efd70005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:35:13.238 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:13.235+0000 7efd85ffb640 1 -- 172.21.15.43:0/1588411264 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efd8815a970 con 0x7efd700039a0 2023-08-09T13:35:13.238 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:13.235+0000 7efd85ffb640 1 -- 172.21.15.43:0/1588411264 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efd7000ba30 con 0x7efd700047c0 2023-08-09T13:35:13.238 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:13.235+0000 7efd85ffb640 1 -- 172.21.15.43:0/1588411264 --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7efd7000bbf0 con 0x7efd880ce8d0 2023-08-09T13:35:13.239 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:13.235+0000 7efd86ffd640 1 --2- 172.21.15.43:0/1588411264 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7efd700039a0 0x7efd70003dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:35:13.240 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:13.235+0000 7efd877fe640 1 --2- 172.21.15.43:0/1588411264 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7efd880ce8d0 0x7efd70005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:35:13.240 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:13.235+0000 7efd867fc640 1 --2- 172.21.15.43:0/1588411264 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7efd700047c0 0x7efd70004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:35:13.240 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:13.235+0000 7efd86ffd640 1 -- 172.21.15.43:0/1588411264 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7efd880ce8d0 msgr2=0x7efd70005180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-09T13:35:13.240 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:13.235+0000 7efd86ffd640 1 --2- 172.21.15.43:0/1588411264 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7efd880ce8d0 0x7efd70005180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:35:13.240 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:13.235+0000 7efd86ffd640 1 -- 172.21.15.43:0/1588411264 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7efd700047c0 msgr2=0x7efd70004c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-09T13:35:13.241 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:13.235+0000 7efd86ffd640 1 --2- 172.21.15.43:0/1588411264 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7efd700047c0 0x7efd70004c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:35:13.241 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:13.235+0000 7efd86ffd640 1 -- 172.21.15.43:0/1588411264 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efd70003520 con 0x7efd700039a0 2023-08-09T13:35:13.241 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:13.235+0000 7efd867fc640 1 --2- 172.21.15.43:0/1588411264 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7efd700047c0 0x7efd70004c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-08-09T13:35:13.241 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:13.237+0000 7efd86ffd640 1 --2- 172.21.15.43:0/1588411264 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7efd700039a0 0x7efd70003dc0 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x7efd78001720 tx=0x7efd700057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2fc23cc4333e90f1 server_cookie=0 in_seq=0 out_seq=0 2023-08-09T13:35:13.241 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:13.237+0000 7efd85ffb640 1 -- 172.21.15.43:0/1588411264 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efd780052d0 con 0x7efd700039a0 2023-08-09T13:35:13.241 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:13.237+0000 7efd85ffb640 1 -- 172.21.15.43:0/1588411264 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efd7802e050 con 0x7efd700039a0 2023-08-09T13:35:13.241 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:13.237+0000 7efd85ffb640 1 -- 172.21.15.43:0/1588411264 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efd780035f0 con 0x7efd700039a0 2023-08-09T13:35:13.241 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:13.238+0000 7efd8d4ec640 1 -- 172.21.15.43:0/1588411264 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7efd700039a0 msgr2=0x7efd70003dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:35:13.242 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:13.238+0000 7efd8d4ec640 1 --2- 172.21.15.43:0/1588411264 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7efd700039a0 0x7efd70003dc0 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x7efd78001720 tx=0x7efd700057f0 comp rx=0 tx=0).stop 2023-08-09T13:35:13.242 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:13.238+0000 7efd877fe640 1 -- 172.21.15.43:0/1588411264 reap_dead start 2023-08-09T13:35:13.242 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:13.238+0000 7efd8d4ec640 1 -- 172.21.15.43:0/1588411264 shutdown_connections 2023-08-09T13:35:13.242 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:13.238+0000 7efd8d4ec640 1 -- 172.21.15.43:0/1588411264 >> 172.21.15.43:0/1588411264 conn(0x7efd880aff50 msgr2=0x7efd880b0360 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-09T13:35:13.242 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:13.238+0000 7efd8d4ec640 1 -- 172.21.15.43:0/1588411264 shutdown_connections 2023-08-09T13:35:13.242 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:13.238+0000 7efd8d4ec640 1 -- 172.21.15.43:0/1588411264 wait complete. 2023-08-09T13:35:13.244 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:13.240+0000 7efd8d4ec640 1 Processor -- start 2023-08-09T13:35:13.244 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:13.240+0000 7efd8d4ec640 1 -- start start 2023-08-09T13:35:13.244 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:13.241+0000 7efd8d4ec640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7efd700039a0 0x7efd8814f130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:35:13.244 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:13.241+0000 7efd8d4ec640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7efd8814f9c0 0x7efd8815be10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:35:13.244 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:13.241+0000 7efd8d4ec640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7efd8815c350 0x7efd880c50f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:35:13.244 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:13.241+0000 7efd8d4ec640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efd8815c7d0 con 0x7efd8814f9c0 2023-08-09T13:35:13.244 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:13.241+0000 7efd8d4ec640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efd880c6410 con 0x7efd8815c350 2023-08-09T13:35:13.245 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:13.241+0000 7efd8d4ec640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7efd880c5e20 con 0x7efd700039a0 2023-08-09T13:35:13.245 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:13.242+0000 7efd867fc640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7efd8814f9c0 0x7efd8815be10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:35:13.245 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:13.242+0000 7efd867fc640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7efd8814f9c0 0x7efd8815be10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:37174/0 (socket says 172.21.15.43:37174) 2023-08-09T13:35:13.245 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:13.242+0000 7efd867fc640 1 -- 172.21.15.43:0/8897540 learned_addr learned my addr 172.21.15.43:0/8897540 (peer_addr_for_me v2:172.21.15.43:0/0) 2023-08-09T13:35:13.245 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:13.242+0000 7efd86ffd640 1 --2- 172.21.15.43:0/8897540 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7efd700039a0 0x7efd8814f130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:35:13.245 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:13.242+0000 7efd867fc640 1 -- 172.21.15.43:0/8897540 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7efd700039a0 msgr2=0x7efd8814f130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:35:13.246 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:13.242+0000 7efd867fc640 1 --2- 172.21.15.43:0/8897540 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7efd700039a0 0x7efd8814f130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:35:13.246 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:13.242+0000 7efd867fc640 1 -- 172.21.15.43:0/8897540 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7efd8815c350 msgr2=0x7efd880c50f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:35:13.246 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:13.242+0000 7efd877fe640 1 --2- 172.21.15.43:0/8897540 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7efd8815c350 0x7efd880c50f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:35:13.246 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:13.242+0000 7efd867fc640 1 --2- 172.21.15.43:0/8897540 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7efd8815c350 0x7efd880c50f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:35:13.246 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:13.242+0000 7efd867fc640 1 -- 172.21.15.43:0/8897540 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efd880c6660 con 0x7efd8814f9c0 2023-08-09T13:35:13.246 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:13.242+0000 7efd867fc640 1 --2- 172.21.15.43:0/8897540 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7efd8814f9c0 0x7efd8815be10 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x7efd7c00d7d0 tx=0x7efd7c00dc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-09T13:35:13.247 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:13.243+0000 7efd6f7fe640 1 -- 172.21.15.43:0/8897540 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efd7c0074e0 con 0x7efd8814f9c0 2023-08-09T13:35:13.247 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:13.243+0000 7efd6f7fe640 1 -- 172.21.15.43:0/8897540 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efd7c007640 con 0x7efd8814f9c0 2023-08-09T13:35:13.247 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:13.243+0000 7efd8d4ec640 1 -- 172.21.15.43:0/8897540 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7efd880c6940 con 0x7efd8814f9c0 2023-08-09T13:35:13.248 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:13.243+0000 7efd6f7fe640 1 -- 172.21.15.43:0/8897540 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efd7c0049e0 con 0x7efd8814f9c0 2023-08-09T13:35:13.248 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:13.243+0000 7efd8d4ec640 1 -- 172.21.15.43:0/8897540 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7efd880c5630 con 0x7efd8814f9c0 2023-08-09T13:35:13.248 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:13.244+0000 7efd8d4ec640 1 -- 172.21.15.43:0/8897540 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7efd880c6e60 con 0x7efd8814f9c0 2023-08-09T13:35:13.248 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:13.245+0000 7efd6f7fe640 1 -- 172.21.15.43:0/8897540 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 5) v1 ==== 88341+0+0 (secure 0 0 0) 0x7efd7c004b40 con 0x7efd8814f9c0 2023-08-09T13:35:13.248 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:13.246+0000 7efd6f7fe640 1 --2- 172.21.15.43:0/8897540 >> [v2:172.21.15.203:6832/830097474,v1:172.21.15.203:6833/830097474] conn(0x7efd7406a080 0x7efd7406c540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:35:13.248 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:13.246+0000 7efd6f7fe640 1 -- 172.21.15.43:0/8897540 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 6871+0+0 (secure 0 0 0) 0x7efd7c088540 con 0x7efd8814f9c0 2023-08-09T13:35:13.249 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:13.247+0000 7efd86ffd640 1 --2- 172.21.15.43:0/8897540 >> [v2:172.21.15.203:6832/830097474,v1:172.21.15.203:6833/830097474] conn(0x7efd7406a080 0x7efd7406c540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:35:13.249 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:13.248+0000 7efd86ffd640 1 --2- 172.21.15.43:0/8897540 >> [v2:172.21.15.203:6832/830097474,v1:172.21.15.203:6833/830097474] conn(0x7efd7406a080 0x7efd7406c540 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7efd78033000 tx=0x7efd78009030 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-09T13:35:13.251 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:13.250+0000 7efd6f7fe640 1 -- 172.21.15.43:0/8897540 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178066 (secure 0 0 0) 0x7efd880c6e60 con 0x7efd8814f9c0 2023-08-09T13:35:13.447 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:13.446+0000 7efd8d4ec640 1 -- 172.21.15.43:0/8897540 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7efd880c6e60 con 0x7efd8814f9c0 2023-08-09T13:35:13.448 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:13.447+0000 7efd6f7fe640 1 -- 172.21.15.43:0/8897540 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v26) v1 ==== 93+0+10 (secure 0 0 0) 0x7efd880c6e60 con 0x7efd8814f9c0 2023-08-09T13:35:13.448 INFO:teuthology.orchestra.run.smithi043.stdout:pg_num: 1 2023-08-09T13:35:13.450 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:13.449+0000 7efd8d4ec640 1 -- 172.21.15.43:0/8897540 >> [v2:172.21.15.203:6832/830097474,v1:172.21.15.203:6833/830097474] conn(0x7efd7406a080 msgr2=0x7efd7406c540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:35:13.451 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:13.449+0000 7efd8d4ec640 1 --2- 172.21.15.43:0/8897540 >> [v2:172.21.15.203:6832/830097474,v1:172.21.15.203:6833/830097474] conn(0x7efd7406a080 0x7efd7406c540 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7efd78033000 tx=0x7efd78009030 comp rx=0 tx=0).stop 2023-08-09T13:35:13.451 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:13.449+0000 7efd8d4ec640 1 -- 172.21.15.43:0/8897540 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7efd8814f9c0 msgr2=0x7efd8815be10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:35:13.451 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:13.449+0000 7efd8d4ec640 1 --2- 172.21.15.43:0/8897540 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7efd8814f9c0 0x7efd8815be10 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x7efd7c00d7d0 tx=0x7efd7c00dc90 comp rx=0 tx=0).stop 2023-08-09T13:35:13.451 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:13.449+0000 7efd8d4ec640 1 -- 172.21.15.43:0/8897540 shutdown_connections 2023-08-09T13:35:13.451 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:13.449+0000 7efd8d4ec640 1 --2- 172.21.15.43:0/8897540 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7efd8815c350 0x7efd880c50f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:35:13.452 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:13.449+0000 7efd8d4ec640 1 --2- 172.21.15.43:0/8897540 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7efd8814f9c0 0x7efd8815be10 unknown :-1 s=CLOSED pgs=295 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:35:13.452 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:13.449+0000 7efd8d4ec640 1 --2- 172.21.15.43:0/8897540 >> [v2:172.21.15.203:6832/830097474,v1:172.21.15.203:6833/830097474] conn(0x7efd7406a080 0x7efd7406c540 unknown :-1 s=CLOSED pgs=160 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:35:13.452 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:13.449+0000 7efd8d4ec640 1 --2- 172.21.15.43:0/8897540 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7efd700039a0 0x7efd8814f130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:35:13.452 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:13.449+0000 7efd8d4ec640 1 -- 172.21.15.43:0/8897540 >> 172.21.15.43:0/8897540 conn(0x7efd880aff50 msgr2=0x7efd880ca5f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-09T13:35:13.452 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:13.449+0000 7efd8d4ec640 1 -- 172.21.15.43:0/8897540 shutdown_connections 2023-08-09T13:35:13.452 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:13.449+0000 7efd8d4ec640 1 -- 172.21.15.43:0/8897540 wait complete. 2023-08-09T13:35:13.471 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-08-09T13:35:13.698 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:13.697+0000 7f3cfba0d640 1 Processor -- start 2023-08-09T13:35:13.699 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:13.697+0000 7f3cfba0d640 1 -- start start 2023-08-09T13:35:13.700 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:13.698+0000 7f3cfba0d640 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7f3cf40b3fa0 0x7f3cf40b43b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:35:13.700 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:13.698+0000 7f3cfba0d640 1 -- --> v1:172.21.15.43:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3cf40b48f0 con 0x7f3cf40d0a70 2023-08-09T13:35:13.700 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:13.698+0000 7f3cfba0d640 1 -- --> v1:172.21.15.203:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3cf40cdf00 con 0x7f3cf415c680 2023-08-09T13:35:13.701 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:13.698+0000 7f3cfba0d640 1 -- --> v2:172.21.15.203:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3cf40d0f70 con 0x7f3cf40b3fa0 2023-08-09T13:35:13.701 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:13.699+0000 7f3cf8fa8640 1 --1- >> v1:172.21.15.43:6789/0 conn(0x7f3cf40d0a70 0x7f3cf40cd7f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.43:6789/0 says I am v1:172.21.15.43:57594/0 (socket says 172.21.15.43:57594) 2023-08-09T13:35:13.701 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:13.699+0000 7f3cf8fa8640 1 -- 172.21.15.43:0/346564508 learned_addr learned my addr 172.21.15.43:0/346564508 (peer_addr_for_me v1:172.21.15.43:0/0) 2023-08-09T13:35:13.703 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:13.699+0000 7f3cf9faa640 1 --2- 172.21.15.43:0/346564508 >> v2:172.21.15.203:3300/0 conn(0x7f3cf40b3fa0 0x7f3cf40b43b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:35:13.703 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:13.699+0000 7f3cebfff640 1 -- 172.21.15.43:0/346564508 <== mon.0 v1:172.21.15.43:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1895576586 0 0) 0x7f3cf40b48f0 con 0x7f3cf40d0a70 2023-08-09T13:35:13.703 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:13.699+0000 7f3cebfff640 1 -- 172.21.15.43:0/346564508 --> v1:172.21.15.43:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f3ce0003520 con 0x7f3cf40d0a70 2023-08-09T13:35:13.703 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:13.700+0000 7f3cebfff640 1 -- 172.21.15.43:0/346564508 <== mon.0 v1:172.21.15.43:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3052059444 0 0) 0x7f3ce0003520 con 0x7f3cf40d0a70 2023-08-09T13:35:13.704 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:13.700+0000 7f3cebfff640 1 -- 172.21.15.43:0/346564508 >> v2:172.21.15.203:3300/0 conn(0x7f3cf40b3fa0 msgr2=0x7f3cf40b43b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-09T13:35:13.704 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:13.700+0000 7f3cebfff640 1 --2- 172.21.15.43:0/346564508 >> v2:172.21.15.203:3300/0 conn(0x7f3cf40b3fa0 0x7f3cf40b43b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:35:13.704 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:13.700+0000 7f3cebfff640 1 -- 172.21.15.43:0/346564508 >> v1:172.21.15.203:6789/0 conn(0x7f3cf415c680 legacy=0x7f3cf40ce760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:35:13.704 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:13.700+0000 7f3cebfff640 1 -- 172.21.15.43:0/346564508 --> v1:172.21.15.43:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3cf40ccd10 con 0x7f3cf40d0a70 2023-08-09T13:35:13.705 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:13.700+0000 7f3cebfff640 1 -- 172.21.15.43:0/346564508 <== mon.0 v1:172.21.15.43:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1285150740 0 0) 0x7f3ce4001670 con 0x7f3cf40d0a70 2023-08-09T13:35:13.705 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:13.700+0000 7f3cebfff640 1 -- 172.21.15.43:0/346564508 >> v1:172.21.15.43:6789/0 conn(0x7f3cf40d0a70 legacy=0x7f3cf40cd7f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:35:13.705 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:13.700+0000 7f3cebfff640 1 --2- 172.21.15.43:0/346564508 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f3ce00039e0 0x7f3ce0003e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:35:13.705 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:13.700+0000 7f3cebfff640 1 --2- 172.21.15.43:0/346564508 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f3ce00047c0 0x7f3ce0004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:35:13.705 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:13.700+0000 7f3cebfff640 1 --2- 172.21.15.43:0/346564508 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f3ce0005180 0x7f3ce000b980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:35:13.706 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:13.700+0000 7f3cebfff640 1 -- 172.21.15.43:0/346564508 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3cf40d0f70 con 0x7f3ce00047c0 2023-08-09T13:35:13.706 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:13.700+0000 7f3cebfff640 1 -- 172.21.15.43:0/346564508 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3ce000c0b0 con 0x7f3ce0005180 2023-08-09T13:35:13.706 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:13.700+0000 7f3cebfff640 1 -- 172.21.15.43:0/346564508 --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3ce000c270 con 0x7f3ce00039e0 2023-08-09T13:35:13.706 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:13.700+0000 7f3cf8fa8640 1 --2- 172.21.15.43:0/346564508 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f3ce00047c0 0x7f3ce0004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:35:13.706 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:13.700+0000 7f3cf97a9640 1 --2- 172.21.15.43:0/346564508 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f3ce00039e0 0x7f3ce0003e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:35:13.706 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:13.701+0000 7f3cf9faa640 1 --2- 172.21.15.43:0/346564508 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f3ce0005180 0x7f3ce000b980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:35:13.706 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:13.701+0000 7f3cf8fa8640 1 -- 172.21.15.43:0/346564508 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f3ce00039e0 msgr2=0x7f3ce0003e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-09T13:35:13.707 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:13.701+0000 7f3cf8fa8640 1 --2- 172.21.15.43:0/346564508 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f3ce00039e0 0x7f3ce0003e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:35:13.707 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:13.701+0000 7f3cf8fa8640 1 -- 172.21.15.43:0/346564508 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f3ce0005180 msgr2=0x7f3ce000b980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-09T13:35:13.707 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:13.701+0000 7f3cf8fa8640 1 --2- 172.21.15.43:0/346564508 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f3ce0005180 0x7f3ce000b980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:35:13.707 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:13.701+0000 7f3cf8fa8640 1 -- 172.21.15.43:0/346564508 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3ce0003520 con 0x7f3ce00047c0 2023-08-09T13:35:13.707 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:13.701+0000 7f3cf9faa640 1 --2- 172.21.15.43:0/346564508 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f3ce0005180 0x7f3ce000b980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-08-09T13:35:13.707 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:13.701+0000 7f3cf9faa640 1 -- 172.21.15.43:0/346564508 reap_dead start 2023-08-09T13:35:13.710 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:13.702+0000 7f3cf8fa8640 1 --2- 172.21.15.43:0/346564508 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f3ce00047c0 0x7f3ce0004c40 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x7f3ce400c210 tx=0x7f3ce00057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7c9d1f476c6e693c server_cookie=0 in_seq=0 out_seq=0 2023-08-09T13:35:13.710 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:13.702+0000 7f3cebfff640 1 -- 172.21.15.43:0/346564508 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3ce4005ad0 con 0x7f3ce00047c0 2023-08-09T13:35:13.710 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:13.702+0000 7f3cebfff640 1 -- 172.21.15.43:0/346564508 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3ce4005c30 con 0x7f3ce00047c0 2023-08-09T13:35:13.710 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:13.702+0000 7f3cebfff640 1 -- 172.21.15.43:0/346564508 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3ce402ca50 con 0x7f3ce00047c0 2023-08-09T13:35:13.711 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:13.703+0000 7f3cfba0d640 1 -- 172.21.15.43:0/346564508 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f3ce00047c0 msgr2=0x7f3ce0004c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:35:13.711 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:13.703+0000 7f3cfba0d640 1 --2- 172.21.15.43:0/346564508 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f3ce00047c0 0x7f3ce0004c40 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x7f3ce400c210 tx=0x7f3ce00057f0 comp rx=0 tx=0).stop 2023-08-09T13:35:13.711 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:13.703+0000 7f3cfba0d640 1 -- 172.21.15.43:0/346564508 shutdown_connections 2023-08-09T13:35:13.711 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:13.703+0000 7f3cfba0d640 1 --2- 172.21.15.43:0/346564508 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f3ce00047c0 0x7f3ce0004c40 unknown :-1 s=CLOSED pgs=297 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:35:13.711 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:13.703+0000 7f3cfba0d640 1 -- 172.21.15.43:0/346564508 >> 172.21.15.43:0/346564508 conn(0x7f3cf4071520 msgr2=0x7f3cf40d2520 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-09T13:35:13.711 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:13.703+0000 7f3cfba0d640 1 -- 172.21.15.43:0/346564508 shutdown_connections 2023-08-09T13:35:13.711 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:13.703+0000 7f3cfba0d640 1 -- 172.21.15.43:0/346564508 wait complete. 2023-08-09T13:35:13.712 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:13.705+0000 7f3cfba0d640 1 Processor -- start 2023-08-09T13:35:13.712 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:13.705+0000 7f3cfba0d640 1 -- start start 2023-08-09T13:35:13.712 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:13.706+0000 7f3cfba0d640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f3ce00047c0 0x7f3cf41408a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:35:13.712 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:13.706+0000 7f3cfba0d640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f3cf40ca000 0x7f3cf415efd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:35:13.712 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:13.706+0000 7f3cfba0d640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f3cf415f510 0x7f3cf415d100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:35:13.712 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:13.706+0000 7f3cfba0d640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3cf40cc460 con 0x7f3cf415f510 2023-08-09T13:35:13.713 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:13.706+0000 7f3cfba0d640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3cf415f990 con 0x7f3ce00047c0 2023-08-09T13:35:13.713 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:13.706+0000 7f3cfba0d640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3cf40ca770 con 0x7f3cf40ca000 2023-08-09T13:35:13.713 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:13.706+0000 7f3cf9faa640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f3cf415f510 0x7f3cf415d100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:35:13.713 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:13.706+0000 7f3cf9faa640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f3cf415f510 0x7f3cf415d100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:37194/0 (socket says 172.21.15.43:37194) 2023-08-09T13:35:13.713 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:13.706+0000 7f3cf9faa640 1 -- 172.21.15.43:0/370762948 learned_addr learned my addr 172.21.15.43:0/370762948 (peer_addr_for_me v2:172.21.15.43:0/0) 2023-08-09T13:35:13.713 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:13.706+0000 7f3cf97a9640 1 --2- 172.21.15.43:0/370762948 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f3ce00047c0 0x7f3cf41408a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:35:13.713 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:13.706+0000 7f3cf8fa8640 1 --2- 172.21.15.43:0/370762948 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f3cf40ca000 0x7f3cf415efd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:35:13.714 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:13.707+0000 7f3cf9faa640 1 -- 172.21.15.43:0/370762948 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f3cf40ca000 msgr2=0x7f3cf415efd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:35:13.714 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:13.707+0000 7f3cf9faa640 1 --2- 172.21.15.43:0/370762948 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f3cf40ca000 0x7f3cf415efd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:35:13.714 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:13.707+0000 7f3cf9faa640 1 -- 172.21.15.43:0/370762948 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f3ce00047c0 msgr2=0x7f3cf41408a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:35:13.714 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:13.707+0000 7f3cf9faa640 1 --2- 172.21.15.43:0/370762948 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f3ce00047c0 0x7f3cf41408a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:35:13.714 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:13.707+0000 7f3cf9faa640 1 -- 172.21.15.43:0/370762948 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3cf40caa50 con 0x7f3cf415f510 2023-08-09T13:35:13.714 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:13.707+0000 7f3cf8fa8640 1 --2- 172.21.15.43:0/370762948 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f3cf40ca000 0x7f3cf415efd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-08-09T13:35:13.714 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:13.707+0000 7f3cf9faa640 1 --2- 172.21.15.43:0/370762948 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f3cf415f510 0x7f3cf415d100 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x7f3cf40d23c0 tx=0x7f3cec010c20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-09T13:35:13.715 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:13.707+0000 7f3ce9ffb640 1 -- 172.21.15.43:0/370762948 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3cec004230 con 0x7f3cf415f510 2023-08-09T13:35:13.715 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:13.707+0000 7f3ce9ffb640 1 -- 172.21.15.43:0/370762948 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3cec004390 con 0x7f3cf415f510 2023-08-09T13:35:13.715 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:13.707+0000 7f3ce9ffb640 1 -- 172.21.15.43:0/370762948 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3cec003810 con 0x7f3cf415f510 2023-08-09T13:35:13.715 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:13.707+0000 7f3cfba0d640 1 -- 172.21.15.43:0/370762948 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3cf40cad30 con 0x7f3cf415f510 2023-08-09T13:35:13.715 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:13.708+0000 7f3cfba0d640 1 -- 172.21.15.43:0/370762948 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3cf40ca480 con 0x7f3cf415f510 2023-08-09T13:35:13.715 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:13.709+0000 7f3cfba0d640 1 -- 172.21.15.43:0/370762948 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3cf415ce20 con 0x7f3cf415f510 2023-08-09T13:35:13.716 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:13.710+0000 7f3ce9ffb640 1 -- 172.21.15.43:0/370762948 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 5) v1 ==== 88341+0+0 (secure 0 0 0) 0x7f3cec003970 con 0x7f3cf415f510 2023-08-09T13:35:13.716 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:13.710+0000 7f3ce9ffb640 1 --2- 172.21.15.43:0/370762948 >> [v2:172.21.15.203:6832/830097474,v1:172.21.15.203:6833/830097474] conn(0x7f3cd806a080 0x7f3cd806c540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:35:13.716 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:13.711+0000 7f3ce9ffb640 1 -- 172.21.15.43:0/370762948 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 6871+0+0 (secure 0 0 0) 0x7f3cec0888f0 con 0x7f3cf415f510 2023-08-09T13:35:13.716 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:13.711+0000 7f3cf97a9640 1 --2- 172.21.15.43:0/370762948 >> [v2:172.21.15.203:6832/830097474,v1:172.21.15.203:6833/830097474] conn(0x7f3cd806a080 0x7f3cd806c540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:35:13.717 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:13.712+0000 7f3cf97a9640 1 --2- 172.21.15.43:0/370762948 >> [v2:172.21.15.203:6832/830097474,v1:172.21.15.203:6833/830097474] conn(0x7f3cd806a080 0x7f3cd806c540 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7f3cf0007fd0 tx=0x7f3cf0009040 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-09T13:35:13.717 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:13.715+0000 7f3ce9ffb640 1 -- 172.21.15.43:0/370762948 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178066 (secure 0 0 0) 0x7f3cf415ce20 con 0x7f3cf415f510 2023-08-09T13:35:13.911 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:13.909+0000 7f3cfba0d640 1 -- 172.21.15.43:0/370762948 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f3cf415ce20 con 0x7f3cf415f510 2023-08-09T13:35:13.911 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:13.910+0000 7f3ce9ffb640 1 -- 172.21.15.43:0/370762948 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v26) v1 ==== 104+0+11 (secure 0 0 0) 0x7f3cf415ce20 con 0x7f3cf415f510 2023-08-09T13:35:13.912 INFO:teuthology.orchestra.run.smithi043.stdout:pg_num: 32 2023-08-09T13:35:13.914 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:13.912+0000 7f3cfba0d640 1 -- 172.21.15.43:0/370762948 >> [v2:172.21.15.203:6832/830097474,v1:172.21.15.203:6833/830097474] conn(0x7f3cd806a080 msgr2=0x7f3cd806c540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:35:13.914 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:13.912+0000 7f3cfba0d640 1 --2- 172.21.15.43:0/370762948 >> [v2:172.21.15.203:6832/830097474,v1:172.21.15.203:6833/830097474] conn(0x7f3cd806a080 0x7f3cd806c540 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7f3cf0007fd0 tx=0x7f3cf0009040 comp rx=0 tx=0).stop 2023-08-09T13:35:13.914 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:13.912+0000 7f3cfba0d640 1 -- 172.21.15.43:0/370762948 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f3cf415f510 msgr2=0x7f3cf415d100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:35:13.915 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:13.912+0000 7f3cfba0d640 1 --2- 172.21.15.43:0/370762948 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f3cf415f510 0x7f3cf415d100 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x7f3cf40d23c0 tx=0x7f3cec010c20 comp rx=0 tx=0).stop 2023-08-09T13:35:13.915 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:13.913+0000 7f3cfba0d640 1 -- 172.21.15.43:0/370762948 shutdown_connections 2023-08-09T13:35:13.915 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:13.913+0000 7f3cfba0d640 1 --2- 172.21.15.43:0/370762948 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f3cf415f510 0x7f3cf415d100 unknown :-1 s=CLOSED pgs=298 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:35:13.915 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:13.913+0000 7f3cfba0d640 1 --2- 172.21.15.43:0/370762948 >> [v2:172.21.15.203:6832/830097474,v1:172.21.15.203:6833/830097474] conn(0x7f3cd806a080 0x7f3cd806c540 unknown :-1 s=CLOSED pgs=161 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:35:13.915 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:13.913+0000 7f3cfba0d640 1 --2- 172.21.15.43:0/370762948 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f3cf40ca000 0x7f3cf415efd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:35:13.915 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:13.913+0000 7f3cfba0d640 1 --2- 172.21.15.43:0/370762948 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f3ce00047c0 0x7f3cf41408a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:35:13.915 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:13.913+0000 7f3cfba0d640 1 -- 172.21.15.43:0/370762948 >> 172.21.15.43:0/370762948 conn(0x7f3cf4071520 msgr2=0x7f3cf40b71d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-09T13:35:13.916 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:13.913+0000 7f3cfba0d640 1 -- 172.21.15.43:0/370762948 shutdown_connections 2023-08-09T13:35:13.916 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:13.913+0000 7f3cfba0d640 1 -- 172.21.15.43:0/370762948 wait complete. 2023-08-09T13:35:13.932 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-08-09T13:35:14.169 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:14.168+0000 7f5c9bb4b640 1 Processor -- start 2023-08-09T13:35:14.169 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:14.168+0000 7f5c9bb4b640 1 -- start start 2023-08-09T13:35:14.171 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:14.169+0000 7f5c9bb4b640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f5c9415c550 0x7f5c940cd6d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:35:14.171 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:14.170+0000 7f5c9bb4b640 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7f5c940d04b0 0x7f5c940b3ee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:35:14.171 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:14.170+0000 7f5c9bb4b640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f5c940b4420 0x7f5c9406a760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:35:14.172 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:14.170+0000 7f5c9bb4b640 1 -- --> v2:172.21.15.43:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5c940b4870 con 0x7f5c9415c550 2023-08-09T13:35:14.172 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:14.170+0000 7f5c9bb4b640 1 -- --> v2:172.21.15.203:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5c940b7bd0 con 0x7f5c940d04b0 2023-08-09T13:35:14.172 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:14.170+0000 7f5c9bb4b640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5c940d1c00 con 0x7f5c940b4420 2023-08-09T13:35:14.173 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:14.170+0000 7f5c998e7640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f5c9415c550 0x7f5c940cd6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:35:14.173 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:14.170+0000 7f5c998e7640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f5c9415c550 0x7f5c940cd6d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:37200/0 (socket says 172.21.15.43:37200) 2023-08-09T13:35:14.173 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:14.170+0000 7f5c998e7640 1 -- 172.21.15.43:0/3412945574 learned_addr learned my addr 172.21.15.43:0/3412945574 (peer_addr_for_me v2:172.21.15.43:0/0) 2023-08-09T13:35:14.173 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:14.170+0000 7f5c990e6640 1 --2- 172.21.15.43:0/3412945574 >> v2:172.21.15.203:3300/0 conn(0x7f5c940d04b0 0x7f5c940b3ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:35:14.173 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:14.170+0000 7f5c9a0e8640 1 --2- 172.21.15.43:0/3412945574 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f5c940b4420 0x7f5c9406a760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:35:14.173 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:14.171+0000 7f5c998e7640 1 -- 172.21.15.43:0/3412945574 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f5c940b4420 msgr2=0x7f5c9406a760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-09T13:35:14.174 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:14.171+0000 7f5c998e7640 1 --2- 172.21.15.43:0/3412945574 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f5c940b4420 0x7f5c9406a760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:35:14.174 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:14.171+0000 7f5c998e7640 1 -- 172.21.15.43:0/3412945574 >> v2:172.21.15.203:3300/0 conn(0x7f5c940d04b0 msgr2=0x7f5c940b3ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-09T13:35:14.174 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:14.171+0000 7f5c998e7640 1 --2- 172.21.15.43:0/3412945574 >> v2:172.21.15.203:3300/0 conn(0x7f5c940d04b0 0x7f5c940b3ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:35:14.174 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:14.171+0000 7f5c998e7640 1 -- 172.21.15.43:0/3412945574 --> v2:172.21.15.43:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5c9415c930 con 0x7f5c9415c550 2023-08-09T13:35:14.175 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:14.172+0000 7f5c998e7640 1 --2- 172.21.15.43:0/3412945574 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f5c9415c550 0x7f5c940cd6d0 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 crypto rx=0x7f5c940b5050 tx=0x7f5c9002b880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a1988028bfac7827 server_cookie=0 in_seq=0 out_seq=0 2023-08-09T13:35:14.175 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:14.172+0000 7f5c988e5640 1 -- 172.21.15.43:0/3412945574 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5c900317e0 con 0x7f5c9415c550 2023-08-09T13:35:14.175 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:14.172+0000 7f5c988e5640 1 -- 172.21.15.43:0/3412945574 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5c90031940 con 0x7f5c9415c550 2023-08-09T13:35:14.175 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:14.172+0000 7f5c988e5640 1 -- 172.21.15.43:0/3412945574 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5c90031cd0 con 0x7f5c9415c550 2023-08-09T13:35:14.176 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:14.173+0000 7f5c9bb4b640 1 -- 172.21.15.43:0/3412945574 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f5c9415c550 msgr2=0x7f5c940cd6d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:35:14.176 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:14.173+0000 7f5c9bb4b640 1 --2- 172.21.15.43:0/3412945574 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f5c9415c550 0x7f5c940cd6d0 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 crypto rx=0x7f5c940b5050 tx=0x7f5c9002b880 comp rx=0 tx=0).stop 2023-08-09T13:35:14.176 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:14.173+0000 7f5c9bb4b640 1 -- 172.21.15.43:0/3412945574 shutdown_connections 2023-08-09T13:35:14.176 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:14.173+0000 7f5c9bb4b640 1 --2- 172.21.15.43:0/3412945574 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f5c940b4420 0x7f5c9406a760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:35:14.176 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:14.173+0000 7f5c9bb4b640 1 --2- 172.21.15.43:0/3412945574 >> v2:172.21.15.203:3300/0 conn(0x7f5c940d04b0 0x7f5c940b3ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:35:14.176 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:14.173+0000 7f5c9bb4b640 1 --2- 172.21.15.43:0/3412945574 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f5c9415c550 0x7f5c940cd6d0 unknown :-1 s=CLOSED pgs=299 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:35:14.176 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:14.173+0000 7f5c9bb4b640 1 -- 172.21.15.43:0/3412945574 >> 172.21.15.43:0/3412945574 conn(0x7f5c94071520 msgr2=0x7f5c940ceee0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-09T13:35:14.176 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:14.173+0000 7f5c9bb4b640 1 -- 172.21.15.43:0/3412945574 shutdown_connections 2023-08-09T13:35:14.177 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:14.174+0000 7f5c9bb4b640 1 -- 172.21.15.43:0/3412945574 wait complete. 2023-08-09T13:35:14.177 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:14.175+0000 7f5c9bb4b640 1 Processor -- start 2023-08-09T13:35:14.177 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:14.175+0000 7f5c9bb4b640 1 -- start start 2023-08-09T13:35:14.178 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:14.176+0000 7f5c9bb4b640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f5c940b4420 0x7f5c941f12b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:35:14.178 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:14.176+0000 7f5c9bb4b640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f5c940d04b0 0x7f5c940caf60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:35:14.178 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:14.176+0000 7f5c9bb4b640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f5c9415c550 0x7f5c940ab280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:35:14.179 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:14.176+0000 7f5c9bb4b640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5c941f17f0 con 0x7f5c940d04b0 2023-08-09T13:35:14.179 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:14.176+0000 7f5c9bb4b640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5c940cb970 con 0x7f5c9415c550 2023-08-09T13:35:14.179 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:14.176+0000 7f5c9bb4b640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5c940cbc50 con 0x7f5c940b4420 2023-08-09T13:35:14.179 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:14.177+0000 7f5c990e6640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f5c940d04b0 0x7f5c940caf60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:35:14.179 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:14.177+0000 7f5c990e6640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f5c940d04b0 0x7f5c940caf60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:37212/0 (socket says 172.21.15.43:37212) 2023-08-09T13:35:14.180 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:14.177+0000 7f5c990e6640 1 -- 172.21.15.43:0/1738531146 learned_addr learned my addr 172.21.15.43:0/1738531146 (peer_addr_for_me v2:172.21.15.43:0/0) 2023-08-09T13:35:14.180 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:14.177+0000 7f5c998e7640 1 --2- 172.21.15.43:0/1738531146 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f5c940b4420 0x7f5c941f12b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:35:14.181 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:14.177+0000 7f5c9a0e8640 1 --2- 172.21.15.43:0/1738531146 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f5c9415c550 0x7f5c940ab280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:35:14.181 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:14.177+0000 7f5c990e6640 1 -- 172.21.15.43:0/1738531146 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f5c940b4420 msgr2=0x7f5c941f12b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:35:14.182 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:14.177+0000 7f5c990e6640 1 --2- 172.21.15.43:0/1738531146 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f5c940b4420 0x7f5c941f12b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:35:14.182 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:14.177+0000 7f5c990e6640 1 -- 172.21.15.43:0/1738531146 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f5c9415c550 msgr2=0x7f5c940ab280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:35:14.182 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:14.177+0000 7f5c990e6640 1 --2- 172.21.15.43:0/1738531146 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f5c9415c550 0x7f5c940ab280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:35:14.182 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:14.177+0000 7f5c990e6640 1 -- 172.21.15.43:0/1738531146 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5c900094d0 con 0x7f5c940d04b0 2023-08-09T13:35:14.182 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:14.177+0000 7f5c998e7640 1 --2- 172.21.15.43:0/1738531146 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f5c940b4420 0x7f5c941f12b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-08-09T13:35:14.182 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:14.178+0000 7f5c990e6640 1 --2- 172.21.15.43:0/1738531146 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f5c940d04b0 0x7f5c940caf60 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x7f5c8400d800 tx=0x7f5c8400dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-09T13:35:14.182 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:14.178+0000 7f5c8a7fc640 1 -- 172.21.15.43:0/1738531146 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5c8400b6c0 con 0x7f5c940d04b0 2023-08-09T13:35:14.183 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:14.178+0000 7f5c8a7fc640 1 -- 172.21.15.43:0/1738531146 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5c8400b820 con 0x7f5c940d04b0 2023-08-09T13:35:14.183 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:14.178+0000 7f5c8a7fc640 1 -- 172.21.15.43:0/1738531146 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5c8400bbb0 con 0x7f5c940d04b0 2023-08-09T13:35:14.183 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:14.178+0000 7f5c9bb4b640 1 -- 172.21.15.43:0/1738531146 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5c940cbf30 con 0x7f5c940d04b0 2023-08-09T13:35:14.183 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:14.178+0000 7f5c9bb4b640 1 -- 172.21.15.43:0/1738531146 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5c940a9140 con 0x7f5c940d04b0 2023-08-09T13:35:14.183 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:14.179+0000 7f5c9bb4b640 1 -- 172.21.15.43:0/1738531146 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5c940739f0 con 0x7f5c940d04b0 2023-08-09T13:35:14.184 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:14.180+0000 7f5c8a7fc640 1 -- 172.21.15.43:0/1738531146 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 5) v1 ==== 88341+0+0 (secure 0 0 0) 0x7f5c8400bd10 con 0x7f5c940d04b0 2023-08-09T13:35:14.184 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:14.181+0000 7f5c8a7fc640 1 --2- 172.21.15.43:0/1738531146 >> [v2:172.21.15.203:6832/830097474,v1:172.21.15.203:6833/830097474] conn(0x7f5c7806a050 0x7f5c7806c510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:35:14.184 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:14.181+0000 7f5c8a7fc640 1 -- 172.21.15.43:0/1738531146 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 6871+0+0 (secure 0 0 0) 0x7f5c84088e20 con 0x7f5c940d04b0 2023-08-09T13:35:14.184 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:14.182+0000 7f5c998e7640 1 --2- 172.21.15.43:0/1738531146 >> [v2:172.21.15.203:6832/830097474,v1:172.21.15.203:6833/830097474] conn(0x7f5c7806a050 0x7f5c7806c510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:35:14.185 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:14.183+0000 7f5c998e7640 1 --2- 172.21.15.43:0/1738531146 >> [v2:172.21.15.203:6832/830097474,v1:172.21.15.203:6833/830097474] conn(0x7f5c7806a050 0x7f5c7806c510 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7f5c90036000 tx=0x7f5c9003d040 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-09T13:35:14.187 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:14.186+0000 7f5c8a7fc640 1 -- 172.21.15.43:0/1738531146 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178066 (secure 0 0 0) 0x7f5c84054140 con 0x7f5c940d04b0 2023-08-09T13:35:14.380 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:14.379+0000 7f5c9bb4b640 1 -- 172.21.15.43:0/1738531146 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f5c940b76d0 con 0x7f5c940d04b0 2023-08-09T13:35:14.381 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:14.380+0000 7f5c8a7fc640 1 -- 172.21.15.43:0/1738531146 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v26) v1 ==== 100+0+11 (secure 0 0 0) 0x7f5c84057760 con 0x7f5c940d04b0 2023-08-09T13:35:14.382 INFO:teuthology.orchestra.run.smithi043.stdout:pg_num: 64 2023-08-09T13:35:14.384 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:14.382+0000 7f5c9bb4b640 1 -- 172.21.15.43:0/1738531146 >> [v2:172.21.15.203:6832/830097474,v1:172.21.15.203:6833/830097474] conn(0x7f5c7806a050 msgr2=0x7f5c7806c510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:35:14.384 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:14.382+0000 7f5c9bb4b640 1 --2- 172.21.15.43:0/1738531146 >> [v2:172.21.15.203:6832/830097474,v1:172.21.15.203:6833/830097474] conn(0x7f5c7806a050 0x7f5c7806c510 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7f5c90036000 tx=0x7f5c9003d040 comp rx=0 tx=0).stop 2023-08-09T13:35:14.384 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:14.382+0000 7f5c9bb4b640 1 -- 172.21.15.43:0/1738531146 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f5c940d04b0 msgr2=0x7f5c940caf60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:35:14.385 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:14.382+0000 7f5c9bb4b640 1 --2- 172.21.15.43:0/1738531146 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f5c940d04b0 0x7f5c940caf60 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x7f5c8400d800 tx=0x7f5c8400dcc0 comp rx=0 tx=0).stop 2023-08-09T13:35:14.385 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:14.382+0000 7f5c9bb4b640 1 -- 172.21.15.43:0/1738531146 shutdown_connections 2023-08-09T13:35:14.385 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:14.382+0000 7f5c9bb4b640 1 --2- 172.21.15.43:0/1738531146 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f5c9415c550 0x7f5c940ab280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:35:14.385 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:14.382+0000 7f5c9bb4b640 1 --2- 172.21.15.43:0/1738531146 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f5c940d04b0 0x7f5c940caf60 unknown :-1 s=CLOSED pgs=300 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:35:14.386 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:14.382+0000 7f5c9bb4b640 1 --2- 172.21.15.43:0/1738531146 >> [v2:172.21.15.203:6832/830097474,v1:172.21.15.203:6833/830097474] conn(0x7f5c7806a050 0x7f5c7806c510 unknown :-1 s=CLOSED pgs=162 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:35:14.386 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:14.382+0000 7f5c9bb4b640 1 --2- 172.21.15.43:0/1738531146 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f5c940b4420 0x7f5c941f12b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:35:14.386 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:14.382+0000 7f5c9bb4b640 1 -- 172.21.15.43:0/1738531146 >> 172.21.15.43:0/1738531146 conn(0x7f5c94071520 msgr2=0x7f5c9415cac0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-09T13:35:14.386 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:14.382+0000 7f5c9bb4b640 1 -- 172.21.15.43:0/1738531146 shutdown_connections 2023-08-09T13:35:14.386 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:14.382+0000 7f5c9bb4b640 1 -- 172.21.15.43:0/1738531146 wait complete. 2023-08-09T13:35:14.404 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2023-08-09T13:35:14.637 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:14.635+0000 7f027ac78640 1 Processor -- start 2023-08-09T13:35:14.637 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:14.635+0000 7f027ac78640 1 -- start start 2023-08-09T13:35:14.638 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:14.637+0000 7f027ac78640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f027415c200 0x7f027406bed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:35:14.639 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:14.637+0000 7f027ac78640 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7f0274160b30 0x7f02740cb9f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:35:14.639 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:14.637+0000 7f027ac78640 1 -- --> v1:172.21.15.43:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f02740cc320 con 0x7f02740ca800 2023-08-09T13:35:14.639 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:14.637+0000 7f027ac78640 1 -- --> v2:172.21.15.203:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f027406b3f0 con 0x7f0274160b30 2023-08-09T13:35:14.639 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:14.637+0000 7f027ac78640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f027406b6d0 con 0x7f027415c200 2023-08-09T13:35:14.640 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:14.637+0000 7f027a477640 1 --1- >> v1:172.21.15.43:6789/0 conn(0x7f02740ca800 0x7f02740c8470 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.43:6789/0 says I am v1:172.21.15.43:57596/0 (socket says 172.21.15.43:57596) 2023-08-09T13:35:14.640 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:14.637+0000 7f027a477640 1 -- 172.21.15.43:0/1218246643 learned_addr learned my addr 172.21.15.43:0/1218246643 (peer_addr_for_me v1:172.21.15.43:0/0) 2023-08-09T13:35:14.640 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:14.638+0000 7f0279475640 1 --2- 172.21.15.43:0/1218246643 >> v2:172.21.15.203:3300/0 conn(0x7f0274160b30 0x7f02740cb9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:35:14.640 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:14.638+0000 7f0279c76640 1 --2- 172.21.15.43:0/1218246643 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f027415c200 0x7f027406bed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:35:14.640 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:14.638+0000 7f0278c74640 1 -- 172.21.15.43:0/1218246643 <== mon.0 v1:172.21.15.43:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4093354146 0 0) 0x7f02740cc320 con 0x7f02740ca800 2023-08-09T13:35:14.640 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:14.638+0000 7f0278c74640 1 -- 172.21.15.43:0/1218246643 --> v1:172.21.15.43:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f0260003520 con 0x7f02740ca800 2023-08-09T13:35:14.641 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:14.639+0000 7f0279c76640 1 -- 172.21.15.43:0/1218246643 >> v2:172.21.15.203:3300/0 conn(0x7f0274160b30 msgr2=0x7f02740cb9f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-09T13:35:14.641 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:14.639+0000 7f0279c76640 1 --2- 172.21.15.43:0/1218246643 >> v2:172.21.15.203:3300/0 conn(0x7f0274160b30 0x7f02740cb9f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:35:14.641 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:14.639+0000 7f0279c76640 1 -- 172.21.15.43:0/1218246643 >> v1:172.21.15.43:6789/0 conn(0x7f02740ca800 legacy=0x7f02740c8470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:35:14.641 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:14.639+0000 7f0279c76640 1 -- 172.21.15.43:0/1218246643 --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f027406b9b0 con 0x7f027415c200 2023-08-09T13:35:14.641 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:14.639+0000 7f0279475640 1 --2- 172.21.15.43:0/1218246643 >> v2:172.21.15.203:3300/0 conn(0x7f0274160b30 0x7f02740cb9f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-08-09T13:35:14.642 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:14.640+0000 7f0279c76640 1 --2- 172.21.15.43:0/1218246643 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f027415c200 0x7f027406bed0 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f02700097a0 tx=0x7f027002b8b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=1bbd8a6c829fe6f8 server_cookie=0 in_seq=0 out_seq=0 2023-08-09T13:35:14.642 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:14.641+0000 7f0278c74640 1 -- 172.21.15.43:0/1218246643 <== mon.2 v2:172.21.15.203:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f027002d040 con 0x7f027415c200 2023-08-09T13:35:14.643 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:14.641+0000 7f0278c74640 1 -- 172.21.15.43:0/1218246643 <== mon.2 v2:172.21.15.203:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0270031810 con 0x7f027415c200 2023-08-09T13:35:14.643 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:14.641+0000 7f0278c74640 1 -- 172.21.15.43:0/1218246643 <== mon.2 v2:172.21.15.203:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0270031ba0 con 0x7f027415c200 2023-08-09T13:35:14.643 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:14.642+0000 7f027ac78640 1 -- 172.21.15.43:0/1218246643 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f027415c200 msgr2=0x7f027406bed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:35:14.644 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:14.642+0000 7f027ac78640 1 --2- 172.21.15.43:0/1218246643 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f027415c200 0x7f027406bed0 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f02700097a0 tx=0x7f027002b8b0 comp rx=0 tx=0).stop 2023-08-09T13:35:14.644 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:14.642+0000 7f027ac78640 1 -- 172.21.15.43:0/1218246643 shutdown_connections 2023-08-09T13:35:14.644 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:14.642+0000 7f027ac78640 1 --2- 172.21.15.43:0/1218246643 >> v2:172.21.15.203:3300/0 conn(0x7f0274160b30 0x7f02740cb9f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:35:14.644 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:14.642+0000 7f027ac78640 1 --2- 172.21.15.43:0/1218246643 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f027415c200 0x7f027406bed0 unknown :-1 s=CLOSED pgs=59 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:35:14.644 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:14.642+0000 7f027ac78640 1 -- 172.21.15.43:0/1218246643 >> 172.21.15.43:0/1218246643 conn(0x7f02740af9f0 msgr2=0x7f02740afdd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-09T13:35:14.644 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:14.642+0000 7f027ac78640 1 -- 172.21.15.43:0/1218246643 shutdown_connections 2023-08-09T13:35:14.645 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:14.642+0000 7f027ac78640 1 -- 172.21.15.43:0/1218246643 wait complete. 2023-08-09T13:35:14.645 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:14.644+0000 7f027ac78640 1 Processor -- start 2023-08-09T13:35:14.645 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:14.644+0000 7f027ac78640 1 -- start start 2023-08-09T13:35:14.646 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:14.645+0000 7f027ac78640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f02740ca800 0x7f0274156810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:35:14.647 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:14.645+0000 7f027ac78640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f027415c200 0x7f027414c3a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:35:14.647 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:14.645+0000 7f027ac78640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f0274160b30 0x7f0274070210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:35:14.647 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:14.645+0000 7f027ac78640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f02740c7690 con 0x7f0274160b30 2023-08-09T13:35:14.647 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:14.645+0000 7f027ac78640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f02740c1a20 con 0x7f02740ca800 2023-08-09T13:35:14.647 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:14.645+0000 7f027ac78640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f02740c4420 con 0x7f027415c200 2023-08-09T13:35:14.648 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:14.645+0000 7f027a477640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f0274160b30 0x7f0274070210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:35:14.649 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:14.646+0000 7f027a477640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f0274160b30 0x7f0274070210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:37226/0 (socket says 172.21.15.43:37226) 2023-08-09T13:35:14.649 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:14.646+0000 7f027a477640 1 -- 172.21.15.43:0/1184949433 learned_addr learned my addr 172.21.15.43:0/1184949433 (peer_addr_for_me v2:172.21.15.43:0/0) 2023-08-09T13:35:14.649 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:14.646+0000 7f0279c76640 1 --2- 172.21.15.43:0/1184949433 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f02740ca800 0x7f0274156810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:35:14.649 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:14.646+0000 7f0279475640 1 --2- 172.21.15.43:0/1184949433 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f027415c200 0x7f027414c3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:35:14.649 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:14.646+0000 7f027a477640 1 -- 172.21.15.43:0/1184949433 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f027415c200 msgr2=0x7f027414c3a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:35:14.649 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:14.646+0000 7f027a477640 1 --2- 172.21.15.43:0/1184949433 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f027415c200 0x7f027414c3a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:35:14.649 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:14.646+0000 7f027a477640 1 -- 172.21.15.43:0/1184949433 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f02740ca800 msgr2=0x7f0274156810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:35:14.650 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:14.646+0000 7f027a477640 1 --2- 172.21.15.43:0/1184949433 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f02740ca800 0x7f0274156810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:35:14.650 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:14.646+0000 7f027a477640 1 -- 172.21.15.43:0/1184949433 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0264009400 con 0x7f0274160b30 2023-08-09T13:35:14.650 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:14.646+0000 7f027a477640 1 --2- 172.21.15.43:0/1184949433 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f0274160b30 0x7f0274070210 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 crypto rx=0x7f026c00cb30 tx=0x7f026c00a060 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-09T13:35:14.650 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:14.647+0000 7f026a7fc640 1 -- 172.21.15.43:0/1184949433 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f026c00aca0 con 0x7f0274160b30 2023-08-09T13:35:14.650 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:14.647+0000 7f026a7fc640 1 -- 172.21.15.43:0/1184949433 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f026c00ae00 con 0x7f0274160b30 2023-08-09T13:35:14.650 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:14.647+0000 7f026a7fc640 1 -- 172.21.15.43:0/1184949433 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f026c012600 con 0x7f0274160b30 2023-08-09T13:35:14.650 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:14.647+0000 7f027ac78640 1 -- 172.21.15.43:0/1184949433 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0270009400 con 0x7f0274160b30 2023-08-09T13:35:14.651 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:14.647+0000 7f027ac78640 1 -- 172.21.15.43:0/1184949433 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f02740c1d00 con 0x7f0274160b30 2023-08-09T13:35:14.651 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:14.648+0000 7f027ac78640 1 -- 172.21.15.43:0/1184949433 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f02740c7800 con 0x7f0274160b30 2023-08-09T13:35:14.651 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:14.649+0000 7f026a7fc640 1 -- 172.21.15.43:0/1184949433 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 5) v1 ==== 88341+0+0 (secure 0 0 0) 0x7f026c012820 con 0x7f0274160b30 2023-08-09T13:35:14.652 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:14.650+0000 7f026a7fc640 1 --2- 172.21.15.43:0/1184949433 >> [v2:172.21.15.203:6832/830097474,v1:172.21.15.203:6833/830097474] conn(0x7f025806a050 0x7f025806c510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:35:14.652 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:14.650+0000 7f026a7fc640 1 -- 172.21.15.43:0/1184949433 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 6871+0+0 (secure 0 0 0) 0x7f026c088830 con 0x7f0274160b30 2023-08-09T13:35:14.652 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:14.651+0000 7f0279c76640 1 --2- 172.21.15.43:0/1184949433 >> [v2:172.21.15.203:6832/830097474,v1:172.21.15.203:6833/830097474] conn(0x7f025806a050 0x7f025806c510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:35:14.653 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:14.652+0000 7f0279c76640 1 --2- 172.21.15.43:0/1184949433 >> [v2:172.21.15.203:6832/830097474,v1:172.21.15.203:6833/830097474] conn(0x7f025806a050 0x7f025806c510 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7f0270036000 tx=0x7f027003c040 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-09T13:35:14.657 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:14.656+0000 7f026a7fc640 1 -- 172.21.15.43:0/1184949433 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178066 (secure 0 0 0) 0x7f02740c7800 con 0x7f0274160b30 2023-08-09T13:35:14.852 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:14.850+0000 7f027ac78640 1 -- 172.21.15.43:0/1184949433 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) v1 -- 0x7f02740c7800 con 0x7f0274160b30 2023-08-09T13:35:14.852 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:14.851+0000 7f026a7fc640 1 -- 172.21.15.43:0/1184949433 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v26) v1 ==== 103+0+11 (secure 0 0 0) 0x7f02740c7800 con 0x7f0274160b30 2023-08-09T13:35:14.853 INFO:teuthology.orchestra.run.smithi043.stdout:pg_num: 32 2023-08-09T13:35:14.856 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:14.854+0000 7f027ac78640 1 -- 172.21.15.43:0/1184949433 >> [v2:172.21.15.203:6832/830097474,v1:172.21.15.203:6833/830097474] conn(0x7f025806a050 msgr2=0x7f025806c510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:35:14.856 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:14.854+0000 7f027ac78640 1 --2- 172.21.15.43:0/1184949433 >> [v2:172.21.15.203:6832/830097474,v1:172.21.15.203:6833/830097474] conn(0x7f025806a050 0x7f025806c510 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7f0270036000 tx=0x7f027003c040 comp rx=0 tx=0).stop 2023-08-09T13:35:14.857 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:14.854+0000 7f027ac78640 1 -- 172.21.15.43:0/1184949433 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f0274160b30 msgr2=0x7f0274070210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:35:14.857 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:14.854+0000 7f027ac78640 1 --2- 172.21.15.43:0/1184949433 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f0274160b30 0x7f0274070210 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 crypto rx=0x7f026c00cb30 tx=0x7f026c00a060 comp rx=0 tx=0).stop 2023-08-09T13:35:14.857 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:14.854+0000 7f027ac78640 1 -- 172.21.15.43:0/1184949433 shutdown_connections 2023-08-09T13:35:14.857 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:14.854+0000 7f027ac78640 1 --2- 172.21.15.43:0/1184949433 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f0274160b30 0x7f0274070210 unknown :-1 s=CLOSED pgs=302 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:35:14.858 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:14.854+0000 7f027ac78640 1 --2- 172.21.15.43:0/1184949433 >> [v2:172.21.15.203:6832/830097474,v1:172.21.15.203:6833/830097474] conn(0x7f025806a050 0x7f025806c510 unknown :-1 s=CLOSED pgs=163 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:35:14.858 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:14.854+0000 7f027ac78640 1 --2- 172.21.15.43:0/1184949433 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f027415c200 0x7f027414c3a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:35:14.858 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:14.854+0000 7f027ac78640 1 --2- 172.21.15.43:0/1184949433 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f02740ca800 0x7f0274156810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:35:14.858 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:14.854+0000 7f027ac78640 1 -- 172.21.15.43:0/1184949433 >> 172.21.15.43:0/1184949433 conn(0x7f02740af9f0 msgr2=0x7f02740adb90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-09T13:35:14.859 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:14.854+0000 7f027ac78640 1 -- 172.21.15.43:0/1184949433 shutdown_connections 2023-08-09T13:35:14.859 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:14.854+0000 7f027ac78640 1 -- 172.21.15.43:0/1184949433 wait complete. 2023-08-09T13:35:14.876 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-08-09T13:35:15.096 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:15.094+0000 7f9d12c06640 1 Processor -- start 2023-08-09T13:35:15.096 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:15.094+0000 7f9d12c06640 1 -- start start 2023-08-09T13:35:15.100 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:15.095+0000 7f9d12c06640 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7f9d0c14a5c0 0x7f9d0c160f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:35:15.101 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:15.096+0000 7f9d12c06640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f9d0c0c3130 0x7f9d0c0cbc70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:35:15.101 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:15.096+0000 7f9d12c06640 1 -- --> v1:172.21.15.43:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9d0c0c38a0 con 0x7f9d0c0cc1b0 2023-08-09T13:35:15.102 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:15.096+0000 7f9d12c06640 1 -- --> v2:172.21.15.203:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9d0c0c4420 con 0x7f9d0c14a5c0 2023-08-09T13:35:15.102 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:15.096+0000 7f9d12c06640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9d0c0cc640 con 0x7f9d0c0c3130 2023-08-09T13:35:15.102 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:15.096+0000 7f9d111a3640 1 --1- >> v1:172.21.15.43:6789/0 conn(0x7f9d0c0cc1b0 0x7f9d0c164510 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.43:6789/0 says I am v1:172.21.15.43:49108/0 (socket says 172.21.15.43:49108) 2023-08-09T13:35:15.102 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:15.096+0000 7f9d111a3640 1 -- 172.21.15.43:0/1589879310 learned_addr learned my addr 172.21.15.43:0/1589879310 (peer_addr_for_me v1:172.21.15.43:0/0) 2023-08-09T13:35:15.103 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:15.096+0000 7f9d0bfff640 1 --2- 172.21.15.43:0/1589879310 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f9d0c0c3130 0x7f9d0c0cbc70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:35:15.103 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:15.096+0000 7f9d109a2640 1 --2- 172.21.15.43:0/1589879310 >> v2:172.21.15.203:3300/0 conn(0x7f9d0c14a5c0 0x7f9d0c160f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:35:15.103 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:15.097+0000 7f9d0b7fe640 1 -- 172.21.15.43:0/1589879310 <== mon.0 v1:172.21.15.43:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 509552273 0 0) 0x7f9d0c0c38a0 con 0x7f9d0c0cc1b0 2023-08-09T13:35:15.103 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:15.097+0000 7f9d0b7fe640 1 -- 172.21.15.43:0/1589879310 --> v1:172.21.15.43:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f9cfc003520 con 0x7f9d0c0cc1b0 2023-08-09T13:35:15.103 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:15.097+0000 7f9d0b7fe640 1 -- 172.21.15.43:0/1589879310 <== mon.0 v1:172.21.15.43:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 892083386 0 0) 0x7f9cfc003520 con 0x7f9d0c0cc1b0 2023-08-09T13:35:15.104 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:15.097+0000 7f9d0b7fe640 1 -- 172.21.15.43:0/1589879310 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f9d0c0c3130 msgr2=0x7f9d0c0cbc70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-09T13:35:15.104 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:15.097+0000 7f9d0b7fe640 1 --2- 172.21.15.43:0/1589879310 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f9d0c0c3130 0x7f9d0c0cbc70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:35:15.105 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:15.097+0000 7f9d0b7fe640 1 -- 172.21.15.43:0/1589879310 >> v2:172.21.15.203:3300/0 conn(0x7f9d0c14a5c0 msgr2=0x7f9d0c160f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-09T13:35:15.105 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:15.097+0000 7f9d0b7fe640 1 --2- 172.21.15.43:0/1589879310 >> v2:172.21.15.203:3300/0 conn(0x7f9d0c14a5c0 0x7f9d0c160f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:35:15.105 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:15.097+0000 7f9d0b7fe640 1 -- 172.21.15.43:0/1589879310 --> v1:172.21.15.43:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9d0c0c3b80 con 0x7f9d0c0cc1b0 2023-08-09T13:35:15.105 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:15.097+0000 7f9d0b7fe640 1 -- 172.21.15.43:0/1589879310 <== mon.0 v1:172.21.15.43:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1285150740 0 0) 0x7f9d040019f0 con 0x7f9d0c0cc1b0 2023-08-09T13:35:15.105 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:15.097+0000 7f9d0b7fe640 1 -- 172.21.15.43:0/1589879310 >> v1:172.21.15.43:6789/0 conn(0x7f9d0c0cc1b0 legacy=0x7f9d0c164510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:35:15.105 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:15.097+0000 7f9d0b7fe640 1 --2- 172.21.15.43:0/1589879310 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f9d0c0c3130 0x7f9cfc0039e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:35:15.106 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:15.098+0000 7f9d0b7fe640 1 --2- 172.21.15.43:0/1589879310 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f9cfc003f60 0x7f9cfc0047c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:35:15.106 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:15.098+0000 7f9d0b7fe640 1 --2- 172.21.15.43:0/1589879310 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f9cfc004d00 0x7f9cfc005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:35:15.106 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:15.098+0000 7f9d0b7fe640 1 -- 172.21.15.43:0/1589879310 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9d0c0c4420 con 0x7f9cfc003f60 2023-08-09T13:35:15.106 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:15.098+0000 7f9d0b7fe640 1 -- 172.21.15.43:0/1589879310 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9d0c0cc640 con 0x7f9cfc004d00 2023-08-09T13:35:15.106 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:15.098+0000 7f9d0b7fe640 1 -- 172.21.15.43:0/1589879310 --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9cfc00ba30 con 0x7f9d0c0c3130 2023-08-09T13:35:15.106 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:15.098+0000 7f9d0bfff640 1 --2- 172.21.15.43:0/1589879310 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f9cfc003f60 0x7f9cfc0047c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:35:15.106 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:15.098+0000 7f9d109a2640 1 --2- 172.21.15.43:0/1589879310 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f9d0c0c3130 0x7f9cfc0039e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:35:15.107 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:15.098+0000 7f9d111a3640 1 --2- 172.21.15.43:0/1589879310 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f9cfc004d00 0x7f9cfc005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:35:15.107 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:15.098+0000 7f9d0bfff640 1 -- 172.21.15.43:0/1589879310 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f9d0c0c3130 msgr2=0x7f9cfc0039e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-09T13:35:15.107 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:15.098+0000 7f9d0bfff640 1 --2- 172.21.15.43:0/1589879310 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f9d0c0c3130 0x7f9cfc0039e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:35:15.107 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:15.098+0000 7f9d0bfff640 1 -- 172.21.15.43:0/1589879310 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f9cfc004d00 msgr2=0x7f9cfc005180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-09T13:35:15.107 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:15.098+0000 7f9d0bfff640 1 --2- 172.21.15.43:0/1589879310 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f9cfc004d00 0x7f9cfc005180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:35:15.107 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:15.098+0000 7f9d0bfff640 1 -- 172.21.15.43:0/1589879310 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9cfc003520 con 0x7f9cfc003f60 2023-08-09T13:35:15.110 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:15.099+0000 7f9d0bfff640 1 --2- 172.21.15.43:0/1589879310 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f9cfc003f60 0x7f9cfc0047c0 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 crypto rx=0x7f9cf8009ac0 tx=0x7f9cfc0057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6e563925042c2d1e server_cookie=0 in_seq=0 out_seq=0 2023-08-09T13:35:15.110 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:15.100+0000 7f9d0b7fe640 1 -- 172.21.15.43:0/1589879310 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9cf802c6b0 con 0x7f9cfc003f60 2023-08-09T13:35:15.110 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:15.100+0000 7f9d0b7fe640 1 -- 172.21.15.43:0/1589879310 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9cf802c810 con 0x7f9cfc003f60 2023-08-09T13:35:15.110 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:15.100+0000 7f9d0b7fe640 1 -- 172.21.15.43:0/1589879310 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9cf802cba0 con 0x7f9cfc003f60 2023-08-09T13:35:15.111 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:15.100+0000 7f9d12c06640 1 -- 172.21.15.43:0/1589879310 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f9cfc003f60 msgr2=0x7f9cfc0047c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:35:15.111 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:15.100+0000 7f9d12c06640 1 --2- 172.21.15.43:0/1589879310 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f9cfc003f60 0x7f9cfc0047c0 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 crypto rx=0x7f9cf8009ac0 tx=0x7f9cfc0057f0 comp rx=0 tx=0).stop 2023-08-09T13:35:15.111 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:15.100+0000 7f9d111a3640 1 -- 172.21.15.43:0/1589879310 reap_dead start 2023-08-09T13:35:15.111 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:15.100+0000 7f9d12c06640 1 -- 172.21.15.43:0/1589879310 shutdown_connections 2023-08-09T13:35:15.111 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:15.100+0000 7f9d12c06640 1 -- 172.21.15.43:0/1589879310 >> 172.21.15.43:0/1589879310 conn(0x7f9d0c0ce0a0 msgr2=0x7f9d0c0ce480 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-09T13:35:15.111 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:15.100+0000 7f9d12c06640 1 -- 172.21.15.43:0/1589879310 shutdown_connections 2023-08-09T13:35:15.111 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:15.100+0000 7f9d12c06640 1 -- 172.21.15.43:0/1589879310 wait complete. 2023-08-09T13:35:15.111 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:15.102+0000 7f9d12c06640 1 Processor -- start 2023-08-09T13:35:15.112 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:15.102+0000 7f9d12c06640 1 -- start start 2023-08-09T13:35:15.112 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:15.103+0000 7f9d12c06640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f9cfc003f60 0x7f9d0c1f7250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:35:15.112 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:15.103+0000 7f9d12c06640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f9d0c1f7790 0x7f9d0c0b3460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:35:15.112 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:15.103+0000 7f9d12c06640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f9d0c0b39a0 0x7f9d0c0c6090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:35:15.112 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:15.103+0000 7f9d12c06640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9d0c0b3e50 con 0x7f9d0c1f7790 2023-08-09T13:35:15.112 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:15.103+0000 7f9d12c06640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9d0c0c5470 con 0x7f9d0c0b39a0 2023-08-09T13:35:15.113 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:15.103+0000 7f9d12c06640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9d0c0b0d80 con 0x7f9cfc003f60 2023-08-09T13:35:15.113 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:15.104+0000 7f9d0bfff640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f9d0c1f7790 0x7f9d0c0b3460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:35:15.113 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:15.104+0000 7f9d0bfff640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f9d0c1f7790 0x7f9d0c0b3460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:42342/0 (socket says 172.21.15.43:42342) 2023-08-09T13:35:15.113 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:15.104+0000 7f9d0bfff640 1 -- 172.21.15.43:0/36402311 learned_addr learned my addr 172.21.15.43:0/36402311 (peer_addr_for_me v2:172.21.15.43:0/0) 2023-08-09T13:35:15.113 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:15.104+0000 7f9d109a2640 1 --2- 172.21.15.43:0/36402311 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f9cfc003f60 0x7f9d0c1f7250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:35:15.113 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:15.104+0000 7f9d111a3640 1 --2- 172.21.15.43:0/36402311 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f9d0c0b39a0 0x7f9d0c0c6090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:35:15.113 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:15.104+0000 7f9d0bfff640 1 -- 172.21.15.43:0/36402311 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f9cfc003f60 msgr2=0x7f9d0c1f7250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:35:15.114 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:15.104+0000 7f9d0bfff640 1 --2- 172.21.15.43:0/36402311 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f9cfc003f60 0x7f9d0c1f7250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:35:15.114 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:15.104+0000 7f9d0bfff640 1 -- 172.21.15.43:0/36402311 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f9d0c0b39a0 msgr2=0x7f9d0c0c6090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:35:15.114 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:15.104+0000 7f9d0bfff640 1 --2- 172.21.15.43:0/36402311 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f9d0c0b39a0 0x7f9d0c0c6090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:35:15.114 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:15.104+0000 7f9d0bfff640 1 -- 172.21.15.43:0/36402311 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9d0c0b1020 con 0x7f9d0c1f7790 2023-08-09T13:35:15.114 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:15.104+0000 7f9d111a3640 1 --2- 172.21.15.43:0/36402311 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f9d0c0b39a0 0x7f9d0c0c6090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-08-09T13:35:15.114 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:15.105+0000 7f9d0bfff640 1 --2- 172.21.15.43:0/36402311 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f9d0c1f7790 0x7f9d0c0b3460 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 crypto rx=0x7f9cf8037040 tx=0x7f9cf8004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-09T13:35:15.114 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:15.105+0000 7f9d097fa640 1 -- 172.21.15.43:0/36402311 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9cf8003810 con 0x7f9d0c1f7790 2023-08-09T13:35:15.115 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:15.105+0000 7f9d097fa640 1 -- 172.21.15.43:0/36402311 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9cf8003970 con 0x7f9d0c1f7790 2023-08-09T13:35:15.115 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:15.105+0000 7f9d097fa640 1 -- 172.21.15.43:0/36402311 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9cf8003d20 con 0x7f9d0c1f7790 2023-08-09T13:35:15.115 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:15.105+0000 7f9d12c06640 1 -- 172.21.15.43:0/36402311 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9d0c0b1300 con 0x7f9d0c1f7790 2023-08-09T13:35:15.115 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:15.105+0000 7f9d12c06640 1 -- 172.21.15.43:0/36402311 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9d0c0b18c0 con 0x7f9d0c1f7790 2023-08-09T13:35:15.115 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:15.106+0000 7f9d12c06640 1 -- 172.21.15.43:0/36402311 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9d0c0b1a50 con 0x7f9d0c1f7790 2023-08-09T13:35:15.115 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:15.107+0000 7f9d097fa640 1 -- 172.21.15.43:0/36402311 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 5) v1 ==== 88341+0+0 (secure 0 0 0) 0x7f9cf8040450 con 0x7f9d0c1f7790 2023-08-09T13:35:15.115 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:15.108+0000 7f9d097fa640 1 --2- 172.21.15.43:0/36402311 >> [v2:172.21.15.203:6832/830097474,v1:172.21.15.203:6833/830097474] conn(0x7f9cf006a050 0x7f9cf006c510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:35:15.116 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:15.108+0000 7f9d097fa640 1 -- 172.21.15.43:0/36402311 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 6871+0+0 (secure 0 0 0) 0x7f9cf80aa800 con 0x7f9d0c1f7790 2023-08-09T13:35:15.116 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:15.109+0000 7f9d109a2640 1 --2- 172.21.15.43:0/36402311 >> [v2:172.21.15.203:6832/830097474,v1:172.21.15.203:6833/830097474] conn(0x7f9cf006a050 0x7f9cf006c510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:35:15.116 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:15.110+0000 7f9d109a2640 1 --2- 172.21.15.43:0/36402311 >> [v2:172.21.15.203:6832/830097474,v1:172.21.15.203:6833/830097474] conn(0x7f9cf006a050 0x7f9cf006c510 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7f9d00009a90 tx=0x7f9d00008040 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-09T13:35:15.117 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:15.113+0000 7f9d097fa640 1 -- 172.21.15.43:0/36402311 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178066 (secure 0 0 0) 0x7f9cf8075bb0 con 0x7f9d0c1f7790 2023-08-09T13:35:15.317 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:15.315+0000 7f9d12c06640 1 -- 172.21.15.43:0/36402311 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f9d0c0739f0 con 0x7f9d0c1f7790 2023-08-09T13:35:15.320 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:15.318+0000 7f9d097fa640 1 -- 172.21.15.43:0/36402311 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v26) v1 ==== 74+0+20146 (secure 0 0 0) 0x7f9cf80044e0 con 0x7f9d0c1f7790 2023-08-09T13:35:15.320 INFO:teuthology.orchestra.run.smithi043.stdout: 2023-08-09T13:35:15.321 INFO:teuthology.orchestra.run.smithi043.stdout:{"epoch":26,"fsid":"daac87ee-1337-4250-a955-aea4e408a39b","created":"2023-08-09T13:33:41.701198+0000","modified":"2023-08-09T13:34:48.711911+0000","last_up_change":"2023-08-09T13:33:53.783453+0000","last_in_change":"2023-08-09T13:33:45.148904+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":5,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":"rbd","create_time":"2023-08-09T13:33:54.365137+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":".mgr","create_time":"2023-08-09T13:33:54.831803+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_metadata","create_time":"2023-08-09T13:34:00.207974+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data","create_time":"2023-08-09T13:34:01.614765+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":5,"pool_name":"cephfs_data_ec","create_time":"2023-08-09T13:34:05.239583+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"24","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"cbd49ecd-e896-4fbd-864e-5c37a829b165","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6800","nonce":502667335},{"type":"v1","addr":"172.21.15.43:6801","nonce":502667335}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6802","nonce":502667335},{"type":"v1","addr":"172.21.15.43:6803","nonce":502667335}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6806","nonce":502667335},{"type":"v1","addr":"172.21.15.43:6807","nonce":502667335}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6804","nonce":502667335},{"type":"v1","addr":"172.21.15.43:6805","nonce":502667335}]},"public_addr":"172.21.15.43:6801/502667335","cluster_addr":"172.21.15.43:6803/502667335","heartbeat_back_addr":"172.21.15.43:6807/502667335","heartbeat_front_addr":"172.21.15.43:6805/502667335","state":["exists","up"]},{"osd":1,"uuid":"75c7e0fe-1ec7-4fe8-a2f2-3661a65b8500","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6808","nonce":3289134610},{"type":"v1","addr":"172.21.15.43:6809","nonce":3289134610}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6810","nonce":3289134610},{"type":"v1","addr":"172.21.15.43:6812","nonce":3289134610}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6821","nonce":3289134610},{"type":"v1","addr":"172.21.15.43:6824","nonce":3289134610}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6815","nonce":3289134610},{"type":"v1","addr":"172.21.15.43:6818","nonce":3289134610}]},"public_addr":"172.21.15.43:6809/3289134610","cluster_addr":"172.21.15.43:6812/3289134610","heartbeat_back_addr":"172.21.15.43:6824/3289134610","heartbeat_front_addr":"172.21.15.43:6818/3289134610","state":["exists","up"]},{"osd":2,"uuid":"bb5c41e9-7f5f-4125-9b01-3bb8bfc2ea83","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6811","nonce":4079709736},{"type":"v1","addr":"172.21.15.43:6814","nonce":4079709736}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6817","nonce":4079709736},{"type":"v1","addr":"172.21.15.43:6820","nonce":4079709736}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6829","nonce":4079709736},{"type":"v1","addr":"172.21.15.43:6831","nonce":4079709736}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6823","nonce":4079709736},{"type":"v1","addr":"172.21.15.43:6826","nonce":4079709736}]},"public_addr":"172.21.15.43:6814/4079709736","cluster_addr":"172.21.15.43:6820/4079709736","heartbeat_back_addr":"172.21.15.43:6831/4079709736","heartbeat_front_addr":"172.21.15.43:6826/4079709736","state":["exists","up"]},{"osd":3,"uuid":"bfe4c8b4-694a-465c-87e6-2b2aab852613","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6813","nonce":1709538759},{"type":"v1","addr":"172.21.15.43:6816","nonce":1709538759}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6819","nonce":1709538759},{"type":"v1","addr":"172.21.15.43:6822","nonce":1709538759}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6828","nonce":1709538759},{"type":"v1","addr":"172.21.15.43:6830","nonce":1709538759}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6825","nonce":1709538759},{"type":"v1","addr":"172.21.15.43:6827","nonce":1709538759}]},"public_addr":"172.21.15.43:6816/1709538759","cluster_addr":"172.21.15.43:6822/1709538759","heartbeat_back_addr":"172.21.15.43:6830/1709538759","heartbeat_front_addr":"172.21.15.43:6827/1709538759","state":["exists","up"]},{"osd":4,"uuid":"fa2e8a77-aba1-4756-853e-0d174a5bdeba","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6800","nonce":4196866481},{"type":"v1","addr":"172.21.15.203:6801","nonce":4196866481}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6802","nonce":4196866481},{"type":"v1","addr":"172.21.15.203:6803","nonce":4196866481}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6806","nonce":4196866481},{"type":"v1","addr":"172.21.15.203:6807","nonce":4196866481}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6804","nonce":4196866481},{"type":"v1","addr":"172.21.15.203:6805","nonce":4196866481}]},"public_addr":"172.21.15.203:6801/4196866481","cluster_addr":"172.21.15.203:6803/4196866481","heartbeat_back_addr":"172.21.15.203:6807/4196866481","heartbeat_front_addr":"172.21.15.203:6805/4196866481","state":["exists","up"]},{"osd":5,"uuid":"81811e4b-4d24-4f51-a82c-f1d700a2a00d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6816","nonce":3059290081},{"type":"v1","addr":"172.21.15.203:6817","nonce":3059290081}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6818","nonce":3059290081},{"type":"v1","addr":"172.21.15.203:6819","nonce":3059290081}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6822","nonce":3059290081},{"type":"v1","addr":"172.21.15.203:6823","nonce":3059290081}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6820","nonce":3059290081},{"type":"v1","addr":"172.21.15.203:6821","nonce":3059290081}]},"public_addr":"172.21.15.203:6817/3059290081","cluster_addr":"172.21.15.203:6819/3059290081","heartbeat_back_addr":"172.21.15.203:6823/3059290081","heartbeat_front_addr":"172.21.15.203:6821/3059290081","state":["exists","up"]},{"osd":6,"uuid":"307bac4d-6a5b-443d-af75-56d695d7674f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6824","nonce":512155569},{"type":"v1","addr":"172.21.15.203:6825","nonce":512155569}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6826","nonce":512155569},{"type":"v1","addr":"172.21.15.203:6827","nonce":512155569}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6830","nonce":512155569},{"type":"v1","addr":"172.21.15.203:6831","nonce":512155569}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6828","nonce":512155569},{"type":"v1","addr":"172.21.15.203:6829","nonce":512155569}]},"public_addr":"172.21.15.203:6825/512155569","cluster_addr":"172.21.15.203:6827/512155569","heartbeat_back_addr":"172.21.15.203:6831/512155569","heartbeat_front_addr":"172.21.15.203:6829/512155569","state":["exists","up"]},{"osd":7,"uuid":"a95dc7ca-9f1b-4295-af70-68a22a2d7d5e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6808","nonce":3331450438},{"type":"v1","addr":"172.21.15.203:6809","nonce":3331450438}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6810","nonce":3331450438},{"type":"v1","addr":"172.21.15.203:6811","nonce":3331450438}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6814","nonce":3331450438},{"type":"v1","addr":"172.21.15.203:6815","nonce":3331450438}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6812","nonce":3331450438},{"type":"v1","addr":"172.21.15.203:6813","nonce":3331450438}]},"public_addr":"172.21.15.203:6809/3331450438","cluster_addr":"172.21.15.203:6811/3331450438","heartbeat_back_addr":"172.21.15.203:6815/3331450438","heartbeat_front_addr":"172.21.15.203:6813/3331450438","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-09T13:33:50.547652+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-09T13:33:50.589469+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-09T13:33:50.561724+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-09T13:33:50.599650+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-09T13:33:50.411855+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-09T13:33:50.219681+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-09T13:33:50.504664+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-09T13:33:50.441248+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"4.14","mappings":[{"from":1,"to":5}]},{"pgid":"4.29","mappings":[{"from":3,"to":2}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-08-09T13:35:15.323 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:15.321+0000 7f9d12c06640 1 -- 172.21.15.43:0/36402311 >> [v2:172.21.15.203:6832/830097474,v1:172.21.15.203:6833/830097474] conn(0x7f9cf006a050 msgr2=0x7f9cf006c510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:35:15.323 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:15.321+0000 7f9d12c06640 1 --2- 172.21.15.43:0/36402311 >> [v2:172.21.15.203:6832/830097474,v1:172.21.15.203:6833/830097474] conn(0x7f9cf006a050 0x7f9cf006c510 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7f9d00009a90 tx=0x7f9d00008040 comp rx=0 tx=0).stop 2023-08-09T13:35:15.323 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:15.321+0000 7f9d12c06640 1 -- 172.21.15.43:0/36402311 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f9d0c1f7790 msgr2=0x7f9d0c0b3460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:35:15.324 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:15.321+0000 7f9d12c06640 1 --2- 172.21.15.43:0/36402311 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f9d0c1f7790 0x7f9d0c0b3460 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 crypto rx=0x7f9cf8037040 tx=0x7f9cf8004730 comp rx=0 tx=0).stop 2023-08-09T13:35:15.324 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:15.321+0000 7f9d12c06640 1 -- 172.21.15.43:0/36402311 shutdown_connections 2023-08-09T13:35:15.324 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:15.321+0000 7f9d12c06640 1 --2- 172.21.15.43:0/36402311 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f9d0c0b39a0 0x7f9d0c0c6090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:35:15.324 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:15.321+0000 7f9d12c06640 1 --2- 172.21.15.43:0/36402311 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f9d0c1f7790 0x7f9d0c0b3460 unknown :-1 s=CLOSED pgs=305 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:35:15.324 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:15.321+0000 7f9d12c06640 1 --2- 172.21.15.43:0/36402311 >> [v2:172.21.15.203:6832/830097474,v1:172.21.15.203:6833/830097474] conn(0x7f9cf006a050 0x7f9cf006c510 unknown :-1 s=CLOSED pgs=164 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:35:15.324 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:15.321+0000 7f9d12c06640 1 --2- 172.21.15.43:0/36402311 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f9cfc003f60 0x7f9d0c1f7250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:35:15.325 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:15.321+0000 7f9d12c06640 1 -- 172.21.15.43:0/36402311 >> 172.21.15.43:0/36402311 conn(0x7f9d0c0ce0a0 msgr2=0x7f9d0c070550 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-09T13:35:15.325 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:15.322+0000 7f9d12c06640 1 -- 172.21.15.43:0/36402311 shutdown_connections 2023-08-09T13:35:15.325 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:15.322+0000 7f9d12c06640 1 -- 172.21.15.43:0/36402311 wait complete. 2023-08-09T13:35:15.342 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': 'rbd', 'create_time': '2023-08-09T13:33:54.365137+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}, 'read_balance': {'score_acting': 2, 'score_stable': 2, 'optimal_score': 1, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': '.mgr', 'create_time': '2023-08-09T13:33:54.831803+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_metadata', 'create_time': '2023-08-09T13:34:00.207974+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data', 'create_time': '2023-08-09T13:34:01.614765+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 2, 'score_stable': 2, 'optimal_score': 1, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 5, 'pool_name': 'cephfs_data_ec', 'create_time': '2023-08-09T13:34:05.239583+0000', 'flags': 5, 'flags_names': 'hashpspool,ec_overwrites', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '24', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2023-08-09T13:35:15.343 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get rbd pg_num 2023-08-09T13:35:15.572 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:15.570+0000 7f7a55b12640 1 Processor -- start 2023-08-09T13:35:15.572 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:15.571+0000 7f7a55b12640 1 -- start start 2023-08-09T13:35:15.574 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:15.572+0000 7f7a55b12640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f7a500cb2d0 0x7f7a5015b380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:35:15.574 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:15.572+0000 7f7a55b12640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f7a5015df60 0x7f7a500ce1c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:35:15.574 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:15.572+0000 7f7a55b12640 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7f7a5015e3c0 0x7f7a500ce700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:35:15.574 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:15.572+0000 7f7a55b12640 1 -- --> v2:172.21.15.43:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7a5015a970 con 0x7f7a500cb2d0 2023-08-09T13:35:15.574 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:15.572+0000 7f7a55b12640 1 -- --> v2:172.21.15.203:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7a5015aef0 con 0x7f7a5015e3c0 2023-08-09T13:35:15.574 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:15.572+0000 7f7a55b12640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7a5015b8c0 con 0x7f7a5015df60 2023-08-09T13:35:15.575 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:15.573+0000 7f7a4f7fe640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f7a500cb2d0 0x7f7a5015b380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:35:15.575 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:15.573+0000 7f7a4f7fe640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f7a500cb2d0 0x7f7a5015b380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:42358/0 (socket says 172.21.15.43:42358) 2023-08-09T13:35:15.576 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:15.573+0000 7f7a4f7fe640 1 -- 172.21.15.43:0/1486509594 learned_addr learned my addr 172.21.15.43:0/1486509594 (peer_addr_for_me v2:172.21.15.43:0/0) 2023-08-09T13:35:15.576 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:15.573+0000 7f7a4effd640 1 --2- 172.21.15.43:0/1486509594 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f7a5015df60 0x7f7a500ce1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:35:15.576 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:15.573+0000 7f7a4ffff640 1 --2- 172.21.15.43:0/1486509594 >> v2:172.21.15.203:3300/0 conn(0x7f7a5015e3c0 0x7f7a500ce700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:35:15.576 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:15.573+0000 7f7a4f7fe640 1 -- 172.21.15.43:0/1486509594 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f7a5015df60 msgr2=0x7f7a500ce1c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-09T13:35:15.576 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:15.573+0000 7f7a4f7fe640 1 --2- 172.21.15.43:0/1486509594 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f7a5015df60 0x7f7a500ce1c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:35:15.576 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:15.573+0000 7f7a4f7fe640 1 -- 172.21.15.43:0/1486509594 >> v2:172.21.15.203:3300/0 conn(0x7f7a5015e3c0 msgr2=0x7f7a500ce700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-09T13:35:15.576 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:15.573+0000 7f7a4f7fe640 1 --2- 172.21.15.43:0/1486509594 >> v2:172.21.15.203:3300/0 conn(0x7f7a5015e3c0 0x7f7a500ce700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:35:15.577 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:15.573+0000 7f7a4f7fe640 1 -- 172.21.15.43:0/1486509594 --> v2:172.21.15.43:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7a5015e940 con 0x7f7a500cb2d0 2023-08-09T13:35:15.577 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:15.574+0000 7f7a4f7fe640 1 --2- 172.21.15.43:0/1486509594 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f7a500cb2d0 0x7f7a5015b380 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 crypto rx=0x7f7a50069aa0 tx=0x7f7a4002b900 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b9759af4ca4c720 server_cookie=0 in_seq=0 out_seq=0 2023-08-09T13:35:15.578 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:15.575+0000 7f7a4e7fc640 1 -- 172.21.15.43:0/1486509594 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7a40031860 con 0x7f7a500cb2d0 2023-08-09T13:35:15.578 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:15.575+0000 7f7a4e7fc640 1 -- 172.21.15.43:0/1486509594 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7a400319c0 con 0x7f7a500cb2d0 2023-08-09T13:35:15.578 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:15.575+0000 7f7a4e7fc640 1 -- 172.21.15.43:0/1486509594 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7a40031d50 con 0x7f7a500cb2d0 2023-08-09T13:35:15.578 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:15.575+0000 7f7a55b12640 1 -- 172.21.15.43:0/1486509594 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f7a500cb2d0 msgr2=0x7f7a5015b380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:35:15.578 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:15.575+0000 7f7a55b12640 1 --2- 172.21.15.43:0/1486509594 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f7a500cb2d0 0x7f7a5015b380 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 crypto rx=0x7f7a50069aa0 tx=0x7f7a4002b900 comp rx=0 tx=0).stop 2023-08-09T13:35:15.578 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:15.576+0000 7f7a55b12640 1 -- 172.21.15.43:0/1486509594 shutdown_connections 2023-08-09T13:35:15.578 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:15.576+0000 7f7a55b12640 1 --2- 172.21.15.43:0/1486509594 >> v2:172.21.15.203:3300/0 conn(0x7f7a5015e3c0 0x7f7a500ce700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:35:15.579 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:15.576+0000 7f7a55b12640 1 --2- 172.21.15.43:0/1486509594 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f7a5015df60 0x7f7a500ce1c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:35:15.579 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:15.576+0000 7f7a55b12640 1 --2- 172.21.15.43:0/1486509594 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f7a500cb2d0 0x7f7a5015b380 unknown :-1 s=CLOSED pgs=306 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:35:15.579 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:15.576+0000 7f7a55b12640 1 -- 172.21.15.43:0/1486509594 >> 172.21.15.43:0/1486509594 conn(0x7f7a500aff50 msgr2=0x7f7a500b0360 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-09T13:35:15.579 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:15.576+0000 7f7a55b12640 1 -- 172.21.15.43:0/1486509594 shutdown_connections 2023-08-09T13:35:15.579 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:15.576+0000 7f7a55b12640 1 -- 172.21.15.43:0/1486509594 wait complete. 2023-08-09T13:35:15.580 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:15.577+0000 7f7a55b12640 1 Processor -- start 2023-08-09T13:35:15.580 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:15.578+0000 7f7a55b12640 1 -- start start 2023-08-09T13:35:15.580 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:15.578+0000 7f7a55b12640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f7a500cb2d0 0x7f7a50141ba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:35:15.581 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:15.579+0000 7f7a55b12640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f7a5015df60 0x7f7a500b3850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:35:15.581 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:15.579+0000 7f7a55b12640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f7a5015e3c0 0x7f7a500c7290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:35:15.581 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:15.579+0000 7f7a55b12640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7a500c4470 con 0x7f7a5015df60 2023-08-09T13:35:15.581 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:15.579+0000 7f7a55b12640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7a500c6bb0 con 0x7f7a500cb2d0 2023-08-09T13:35:15.581 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:15.579+0000 7f7a55b12640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7a50151ab0 con 0x7f7a5015e3c0 2023-08-09T13:35:15.582 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:15.579+0000 7f7a4effd640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f7a5015df60 0x7f7a500b3850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:35:15.583 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:15.579+0000 7f7a4effd640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f7a5015df60 0x7f7a500b3850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:42374/0 (socket says 172.21.15.43:42374) 2023-08-09T13:35:15.583 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:15.579+0000 7f7a4effd640 1 -- 172.21.15.43:0/2977052714 learned_addr learned my addr 172.21.15.43:0/2977052714 (peer_addr_for_me v2:172.21.15.43:0/0) 2023-08-09T13:35:15.584 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:15.579+0000 7f7a4f7fe640 1 --2- 172.21.15.43:0/2977052714 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f7a500cb2d0 0x7f7a50141ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:35:15.585 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:15.579+0000 7f7a4ffff640 1 --2- 172.21.15.43:0/2977052714 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f7a5015e3c0 0x7f7a500c7290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:35:15.585 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:15.579+0000 7f7a4effd640 1 -- 172.21.15.43:0/2977052714 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f7a5015e3c0 msgr2=0x7f7a500c7290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:35:15.585 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:15.579+0000 7f7a4effd640 1 --2- 172.21.15.43:0/2977052714 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f7a5015e3c0 0x7f7a500c7290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:35:15.585 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:15.579+0000 7f7a4effd640 1 -- 172.21.15.43:0/2977052714 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f7a500cb2d0 msgr2=0x7f7a50141ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:35:15.585 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:15.579+0000 7f7a4effd640 1 --2- 172.21.15.43:0/2977052714 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f7a500cb2d0 0x7f7a50141ba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:35:15.585 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:15.579+0000 7f7a4effd640 1 -- 172.21.15.43:0/2977052714 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7a400094d0 con 0x7f7a5015df60 2023-08-09T13:35:15.586 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:15.580+0000 7f7a4effd640 1 --2- 172.21.15.43:0/2977052714 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f7a5015df60 0x7f7a500b3850 secure :-1 s=READY pgs=307 cs=0 l=1 rev1=1 crypto rx=0x7f7a4400d800 tx=0x7f7a4400dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-09T13:35:15.586 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:15.580+0000 7f7a37fff640 1 -- 172.21.15.43:0/2977052714 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7a440096a0 con 0x7f7a5015df60 2023-08-09T13:35:15.586 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:15.580+0000 7f7a37fff640 1 -- 172.21.15.43:0/2977052714 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7a44009800 con 0x7f7a5015df60 2023-08-09T13:35:15.586 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:15.580+0000 7f7a37fff640 1 -- 172.21.15.43:0/2977052714 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7a44009b90 con 0x7f7a5015df60 2023-08-09T13:35:15.586 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:15.580+0000 7f7a55b12640 1 -- 172.21.15.43:0/2977052714 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7a500c68d0 con 0x7f7a5015df60 2023-08-09T13:35:15.586 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:15.580+0000 7f7a55b12640 1 -- 172.21.15.43:0/2977052714 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7a500b1060 con 0x7f7a5015df60 2023-08-09T13:35:15.586 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:15.581+0000 7f7a55b12640 1 -- 172.21.15.43:0/2977052714 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7a500739f0 con 0x7f7a5015df60 2023-08-09T13:35:15.586 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:15.582+0000 7f7a37fff640 1 -- 172.21.15.43:0/2977052714 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 5) v1 ==== 88341+0+0 (secure 0 0 0) 0x7f7a44009cf0 con 0x7f7a5015df60 2023-08-09T13:35:15.587 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:15.584+0000 7f7a37fff640 1 --2- 172.21.15.43:0/2977052714 >> [v2:172.21.15.203:6832/830097474,v1:172.21.15.203:6833/830097474] conn(0x7f7a3c06a080 0x7f7a3c06c540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:35:15.587 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:15.584+0000 7f7a37fff640 1 -- 172.21.15.43:0/2977052714 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 6871+0+0 (secure 0 0 0) 0x7f7a44088fc0 con 0x7f7a5015df60 2023-08-09T13:35:15.587 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:15.585+0000 7f7a4f7fe640 1 --2- 172.21.15.43:0/2977052714 >> [v2:172.21.15.203:6832/830097474,v1:172.21.15.203:6833/830097474] conn(0x7f7a3c06a080 0x7f7a3c06c540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:35:15.588 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:15.586+0000 7f7a4f7fe640 1 --2- 172.21.15.43:0/2977052714 >> [v2:172.21.15.203:6832/830097474,v1:172.21.15.203:6833/830097474] conn(0x7f7a3c06a080 0x7f7a3c06c540 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7f7a40036000 tx=0x7f7a4003c040 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-09T13:35:15.590 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:15.589+0000 7f7a37fff640 1 -- 172.21.15.43:0/2977052714 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178066 (secure 0 0 0) 0x7f7a44054370 con 0x7f7a5015df60 2023-08-09T13:35:15.787 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:15.786+0000 7f7a55b12640 1 -- 172.21.15.43:0/2977052714 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"} v 0) v1 -- 0x7f7a500cd4a0 con 0x7f7a5015df60 2023-08-09T13:35:15.788 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:15.787+0000 7f7a37fff640 1 -- 172.21.15.43:0/2977052714 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"}]=0 v26) v1 ==== 92+0+10 (secure 0 0 0) 0x7f7a4400b440 con 0x7f7a5015df60 2023-08-09T13:35:15.788 INFO:teuthology.orchestra.run.smithi043.stdout:pg_num: 8 2023-08-09T13:35:15.791 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:15.789+0000 7f7a55b12640 1 -- 172.21.15.43:0/2977052714 >> [v2:172.21.15.203:6832/830097474,v1:172.21.15.203:6833/830097474] conn(0x7f7a3c06a080 msgr2=0x7f7a3c06c540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:35:15.791 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:15.789+0000 7f7a55b12640 1 --2- 172.21.15.43:0/2977052714 >> [v2:172.21.15.203:6832/830097474,v1:172.21.15.203:6833/830097474] conn(0x7f7a3c06a080 0x7f7a3c06c540 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7f7a40036000 tx=0x7f7a4003c040 comp rx=0 tx=0).stop 2023-08-09T13:35:15.791 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:15.789+0000 7f7a55b12640 1 -- 172.21.15.43:0/2977052714 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f7a5015df60 msgr2=0x7f7a500b3850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:35:15.791 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:15.789+0000 7f7a55b12640 1 --2- 172.21.15.43:0/2977052714 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f7a5015df60 0x7f7a500b3850 secure :-1 s=READY pgs=307 cs=0 l=1 rev1=1 crypto rx=0x7f7a4400d800 tx=0x7f7a4400dcc0 comp rx=0 tx=0).stop 2023-08-09T13:35:15.791 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:15.789+0000 7f7a55b12640 1 -- 172.21.15.43:0/2977052714 shutdown_connections 2023-08-09T13:35:15.791 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:15.789+0000 7f7a55b12640 1 --2- 172.21.15.43:0/2977052714 >> [v2:172.21.15.203:6832/830097474,v1:172.21.15.203:6833/830097474] conn(0x7f7a3c06a080 0x7f7a3c06c540 unknown :-1 s=CLOSED pgs=165 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:35:15.792 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:15.789+0000 7f7a55b12640 1 --2- 172.21.15.43:0/2977052714 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f7a5015e3c0 0x7f7a500c7290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:35:15.792 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:15.789+0000 7f7a55b12640 1 --2- 172.21.15.43:0/2977052714 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f7a5015df60 0x7f7a500b3850 unknown :-1 s=CLOSED pgs=307 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:35:15.792 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:15.789+0000 7f7a55b12640 1 --2- 172.21.15.43:0/2977052714 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f7a500cb2d0 0x7f7a50141ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:35:15.792 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:15.789+0000 7f7a55b12640 1 -- 172.21.15.43:0/2977052714 >> 172.21.15.43:0/2977052714 conn(0x7f7a500aff50 msgr2=0x7f7a500af220 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-09T13:35:15.793 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:15.789+0000 7f7a55b12640 1 -- 172.21.15.43:0/2977052714 shutdown_connections 2023-08-09T13:35:15.793 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:15.789+0000 7f7a55b12640 1 -- 172.21.15.43:0/2977052714 wait complete. 2023-08-09T13:35:15.809 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-08-09T13:35:16.038 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:16.036+0000 7febde434640 1 Processor -- start 2023-08-09T13:35:16.038 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:16.036+0000 7febde434640 1 -- start start 2023-08-09T13:35:16.039 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:16.038+0000 7febde434640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7febd815df60 0x7febd80ce1c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:35:16.039 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:16.038+0000 7febde434640 1 -- --> v1:172.21.15.43:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7febd815aef0 con 0x7febd80cb2d0 2023-08-09T13:35:16.039 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:16.038+0000 7febde434640 1 -- --> v1:172.21.15.203:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7febd815e940 con 0x7febd815e3d0 2023-08-09T13:35:16.040 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:16.038+0000 7febde434640 1 -- --> v2:172.21.15.43:3300/0 -- mon_getmap magic: 0 v1 -- 0x7febd815a970 con 0x7febd815df60 2023-08-09T13:35:16.040 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:16.038+0000 7febd7fff640 1 --1- >> v1:172.21.15.43:6789/0 conn(0x7febd80cb2d0 0x7febd815b380 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.43:6789/0 says I am v1:172.21.15.43:49124/0 (socket says 172.21.15.43:49124) 2023-08-09T13:35:16.040 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:16.038+0000 7febd7fff640 1 -- 172.21.15.43:0/2486115958 learned_addr learned my addr 172.21.15.43:0/2486115958 (peer_addr_for_me v1:172.21.15.43:0/0) 2023-08-09T13:35:16.041 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:16.038+0000 7febd77fe640 1 --2- 172.21.15.43:0/2486115958 >> v2:172.21.15.43:3300/0 conn(0x7febd815df60 0x7febd80ce1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:35:16.041 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:16.039+0000 7febd6ffd640 1 -- 172.21.15.43:0/2486115958 <== mon.0 v1:172.21.15.43:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1239424013 0 0) 0x7febd815aef0 con 0x7febd80cb2d0 2023-08-09T13:35:16.041 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:16.039+0000 7febd6ffd640 1 -- 172.21.15.43:0/2486115958 --> v1:172.21.15.43:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7febc0003520 con 0x7febd80cb2d0 2023-08-09T13:35:16.041 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:16.039+0000 7febd77fe640 1 -- 172.21.15.43:0/2486115958 >> v1:172.21.15.203:6789/0 conn(0x7febd815e3d0 legacy=0x7febd80c9f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-09T13:35:16.041 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:16.039+0000 7febd77fe640 1 -- 172.21.15.43:0/2486115958 >> v1:172.21.15.43:6789/0 conn(0x7febd80cb2d0 legacy=0x7febd815b380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:35:16.041 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:16.039+0000 7febd77fe640 1 -- 172.21.15.43:0/2486115958 --> v2:172.21.15.43:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7febd815ec20 con 0x7febd815df60 2023-08-09T13:35:16.042 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:16.040+0000 7febd77fe640 1 --2- 172.21.15.43:0/2486115958 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7febd815df60 0x7febd80ce1c0 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 crypto rx=0x7febcc0097a0 tx=0x7febcc02b840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9c56d67320d983cc server_cookie=0 in_seq=0 out_seq=0 2023-08-09T13:35:16.042 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:16.040+0000 7febd6ffd640 1 -- 172.21.15.43:0/2486115958 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7febcc02e070 con 0x7febd815df60 2023-08-09T13:35:16.042 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:16.040+0000 7febd6ffd640 1 -- 172.21.15.43:0/2486115958 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7febcc0315a0 con 0x7febd815df60 2023-08-09T13:35:16.042 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:16.040+0000 7febd6ffd640 1 -- 172.21.15.43:0/2486115958 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7febcc031910 con 0x7febd815df60 2023-08-09T13:35:16.043 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:16.041+0000 7febde434640 1 -- 172.21.15.43:0/2486115958 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7febd815df60 msgr2=0x7febd80ce1c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:35:16.043 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:16.041+0000 7febde434640 1 --2- 172.21.15.43:0/2486115958 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7febd815df60 0x7febd80ce1c0 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 crypto rx=0x7febcc0097a0 tx=0x7febcc02b840 comp rx=0 tx=0).stop 2023-08-09T13:35:16.043 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:16.041+0000 7febde434640 1 -- 172.21.15.43:0/2486115958 shutdown_connections 2023-08-09T13:35:16.043 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:16.041+0000 7febde434640 1 --2- 172.21.15.43:0/2486115958 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7febd815df60 0x7febd80ce1c0 unknown :-1 s=CLOSED pgs=309 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:35:16.044 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:16.041+0000 7febde434640 1 -- 172.21.15.43:0/2486115958 >> 172.21.15.43:0/2486115958 conn(0x7febd80aff50 msgr2=0x7febd80b0360 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-09T13:35:16.044 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:16.041+0000 7febde434640 1 -- 172.21.15.43:0/2486115958 shutdown_connections 2023-08-09T13:35:16.044 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:16.041+0000 7febde434640 1 -- 172.21.15.43:0/2486115958 wait complete. 2023-08-09T13:35:16.044 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:16.043+0000 7febde434640 1 Processor -- start 2023-08-09T13:35:16.044 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:16.043+0000 7febde434640 1 -- start start 2023-08-09T13:35:16.046 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:16.044+0000 7febde434640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7febd80cb2d0 0x7febd80c5ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:35:16.046 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:16.044+0000 7febde434640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7febd815df60 0x7febd80c98a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:35:16.046 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:16.044+0000 7febde434640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7febd815e3d0 0x7febd80b1be0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:35:16.046 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:16.044+0000 7febde434640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7febd80b3c10 con 0x7febd815e3d0 2023-08-09T13:35:16.046 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:16.044+0000 7febde434640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7febd80b26a0 con 0x7febd80cb2d0 2023-08-09T13:35:16.046 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:16.044+0000 7febde434640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7febd80c5120 con 0x7febd815df60 2023-08-09T13:35:16.046 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:16.044+0000 7febdc9d1640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7febd815e3d0 0x7febd80b1be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:35:16.047 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:16.044+0000 7febdc9d1640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7febd815e3d0 0x7febd80b1be0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:42394/0 (socket says 172.21.15.43:42394) 2023-08-09T13:35:16.047 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:16.044+0000 7febdc9d1640 1 -- 172.21.15.43:0/2559774157 learned_addr learned my addr 172.21.15.43:0/2559774157 (peer_addr_for_me v2:172.21.15.43:0/0) 2023-08-09T13:35:16.048 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:16.045+0000 7febd7fff640 1 --2- 172.21.15.43:0/2559774157 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7febd80cb2d0 0x7febd80c5ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:35:16.048 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:16.045+0000 7febdc9d1640 1 -- 172.21.15.43:0/2559774157 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7febd815df60 msgr2=0x7febd80c98a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:35:16.049 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:16.045+0000 7febd77fe640 1 --2- 172.21.15.43:0/2559774157 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7febd815df60 0x7febd80c98a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:35:16.049 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:16.045+0000 7febdc9d1640 1 --2- 172.21.15.43:0/2559774157 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7febd815df60 0x7febd80c98a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:35:16.049 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:16.045+0000 7febdc9d1640 1 -- 172.21.15.43:0/2559774157 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7febd80cb2d0 msgr2=0x7febd80c5ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:35:16.049 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:16.045+0000 7febdc9d1640 1 --2- 172.21.15.43:0/2559774157 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7febd80cb2d0 0x7febd80c5ee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:35:16.049 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:16.045+0000 7febdc9d1640 1 -- 172.21.15.43:0/2559774157 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7febcc009400 con 0x7febd815e3d0 2023-08-09T13:35:16.049 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:16.045+0000 7febdc9d1640 1 --2- 172.21.15.43:0/2559774157 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7febd815e3d0 0x7febd80b1be0 secure :-1 s=READY pgs=310 cs=0 l=1 rev1=1 crypto rx=0x7febd000b970 tx=0x7febd000be30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-09T13:35:16.049 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:16.045+0000 7febd4ff9640 1 -- 172.21.15.43:0/2559774157 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7febd000cb40 con 0x7febd815e3d0 2023-08-09T13:35:16.050 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:16.045+0000 7febd4ff9640 1 -- 172.21.15.43:0/2559774157 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7febd000cca0 con 0x7febd815e3d0 2023-08-09T13:35:16.050 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:16.045+0000 7febd4ff9640 1 -- 172.21.15.43:0/2559774157 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7febd0014680 con 0x7febd815e3d0 2023-08-09T13:35:16.050 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:16.045+0000 7febde434640 1 -- 172.21.15.43:0/2559774157 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7febd80c5400 con 0x7febd815e3d0 2023-08-09T13:35:16.050 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:16.046+0000 7febde434640 1 -- 172.21.15.43:0/2559774157 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7febd80c59c0 con 0x7febd815e3d0 2023-08-09T13:35:16.050 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:16.047+0000 7febde434640 1 -- 172.21.15.43:0/2559774157 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7febd80739f0 con 0x7febd815e3d0 2023-08-09T13:35:16.050 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:16.048+0000 7febd4ff9640 1 -- 172.21.15.43:0/2559774157 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 5) v1 ==== 88341+0+0 (secure 0 0 0) 0x7febd001d3b0 con 0x7febd815e3d0 2023-08-09T13:35:16.051 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:16.049+0000 7febd4ff9640 1 --2- 172.21.15.43:0/2559774157 >> [v2:172.21.15.203:6832/830097474,v1:172.21.15.203:6833/830097474] conn(0x7febc406a050 0x7febc406c510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:35:16.051 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:16.050+0000 7febd4ff9640 1 -- 172.21.15.43:0/2559774157 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 6871+0+0 (secure 0 0 0) 0x7febd00886d0 con 0x7febd815e3d0 2023-08-09T13:35:16.051 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:16.050+0000 7febd7fff640 1 --2- 172.21.15.43:0/2559774157 >> [v2:172.21.15.203:6832/830097474,v1:172.21.15.203:6833/830097474] conn(0x7febc406a050 0x7febc406c510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:35:16.053 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:16.051+0000 7febd7fff640 1 --2- 172.21.15.43:0/2559774157 >> [v2:172.21.15.203:6832/830097474,v1:172.21.15.203:6833/830097474] conn(0x7febc406a050 0x7febc406c510 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7febc80011e0 tx=0x7febc8009040 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-09T13:35:16.055 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:16.054+0000 7febd4ff9640 1 -- 172.21.15.43:0/2559774157 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178066 (secure 0 0 0) 0x7febd0053a80 con 0x7febd815e3d0 2023-08-09T13:35:16.251 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:16.249+0000 7febde434640 1 -- 172.21.15.43:0/2559774157 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7febd806fce0 con 0x7febd815e3d0 2023-08-09T13:35:16.252 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:16.250+0000 7febd4ff9640 1 -- 172.21.15.43:0/2559774157 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v26) v1 ==== 93+0+10 (secure 0 0 0) 0x7febd00570a0 con 0x7febd815e3d0 2023-08-09T13:35:16.252 INFO:teuthology.orchestra.run.smithi043.stdout:pg_num: 1 2023-08-09T13:35:16.255 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:16.253+0000 7febde434640 1 -- 172.21.15.43:0/2559774157 >> [v2:172.21.15.203:6832/830097474,v1:172.21.15.203:6833/830097474] conn(0x7febc406a050 msgr2=0x7febc406c510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:35:16.255 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:16.253+0000 7febde434640 1 --2- 172.21.15.43:0/2559774157 >> [v2:172.21.15.203:6832/830097474,v1:172.21.15.203:6833/830097474] conn(0x7febc406a050 0x7febc406c510 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7febc80011e0 tx=0x7febc8009040 comp rx=0 tx=0).stop 2023-08-09T13:35:16.255 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:16.253+0000 7febde434640 1 -- 172.21.15.43:0/2559774157 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7febd815e3d0 msgr2=0x7febd80b1be0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:35:16.255 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:16.253+0000 7febde434640 1 --2- 172.21.15.43:0/2559774157 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7febd815e3d0 0x7febd80b1be0 secure :-1 s=READY pgs=310 cs=0 l=1 rev1=1 crypto rx=0x7febd000b970 tx=0x7febd000be30 comp rx=0 tx=0).stop 2023-08-09T13:35:16.256 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:16.253+0000 7febde434640 1 -- 172.21.15.43:0/2559774157 shutdown_connections 2023-08-09T13:35:16.256 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:16.253+0000 7febde434640 1 --2- 172.21.15.43:0/2559774157 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7febd815e3d0 0x7febd80b1be0 unknown :-1 s=CLOSED pgs=310 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:35:16.256 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:16.253+0000 7febde434640 1 --2- 172.21.15.43:0/2559774157 >> [v2:172.21.15.203:6832/830097474,v1:172.21.15.203:6833/830097474] conn(0x7febc406a050 0x7febc406c510 unknown :-1 s=CLOSED pgs=166 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:35:16.256 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:16.253+0000 7febde434640 1 --2- 172.21.15.43:0/2559774157 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7febd815df60 0x7febd80c98a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:35:16.256 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:16.253+0000 7febde434640 1 --2- 172.21.15.43:0/2559774157 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7febd80cb2d0 0x7febd80c5ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:35:16.256 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:16.253+0000 7febde434640 1 -- 172.21.15.43:0/2559774157 >> 172.21.15.43:0/2559774157 conn(0x7febd80aff50 msgr2=0x7febd80ce190 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-09T13:35:16.257 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:16.253+0000 7febde434640 1 -- 172.21.15.43:0/2559774157 shutdown_connections 2023-08-09T13:35:16.257 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:16.253+0000 7febde434640 1 -- 172.21.15.43:0/2559774157 wait complete. 2023-08-09T13:35:16.275 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-08-09T13:35:16.509 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:16.507+0000 7f85d6a3f640 1 Processor -- start 2023-08-09T13:35:16.509 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:16.507+0000 7f85d6a3f640 1 -- start start 2023-08-09T13:35:16.510 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:16.509+0000 7f85d6a3f640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f85d00b3fa0 0x7f85d00b43b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:35:16.511 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:16.509+0000 7f85d6a3f640 1 -- --> v1:172.21.15.43:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f85d00b48f0 con 0x7f85d015c680 2023-08-09T13:35:16.511 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:16.509+0000 7f85d6a3f640 1 -- --> v1:172.21.15.203:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f85d00cdf00 con 0x7f85d00d0a70 2023-08-09T13:35:16.511 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:16.509+0000 7f85d6a3f640 1 -- --> v2:172.21.15.43:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f85d00d0f70 con 0x7f85d00b3fa0 2023-08-09T13:35:16.511 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:16.509+0000 7f85cffff640 1 --1- >> v1:172.21.15.43:6789/0 conn(0x7f85d015c680 0x7f85d00ce760 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.43:6789/0 says I am v1:172.21.15.43:49128/0 (socket says 172.21.15.43:49128) 2023-08-09T13:35:16.512 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:16.509+0000 7f85cffff640 1 -- 172.21.15.43:0/1586898962 learned_addr learned my addr 172.21.15.43:0/1586898962 (peer_addr_for_me v1:172.21.15.43:0/0) 2023-08-09T13:35:16.512 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:16.509+0000 7f85d4fdc640 1 --2- 172.21.15.43:0/1586898962 >> v2:172.21.15.43:3300/0 conn(0x7f85d00b3fa0 0x7f85d00b43b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:35:16.512 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:16.510+0000 7f85cf7fe640 1 -- 172.21.15.43:0/1586898962 <== mon.0 v1:172.21.15.43:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1203993087 0 0) 0x7f85d00b48f0 con 0x7f85d015c680 2023-08-09T13:35:16.512 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:16.510+0000 7f85d4fdc640 1 -- 172.21.15.43:0/1586898962 >> v1:172.21.15.203:6789/0 conn(0x7f85d00d0a70 legacy=0x7f85d00cd7f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:35:16.512 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:16.510+0000 7f85d4fdc640 1 -- 172.21.15.43:0/1586898962 >> v1:172.21.15.43:6789/0 conn(0x7f85d015c680 legacy=0x7f85d00ce760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:35:16.512 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:16.510+0000 7f85d4fdc640 1 -- 172.21.15.43:0/1586898962 --> v2:172.21.15.43:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f85d00ccd10 con 0x7f85d00b3fa0 2023-08-09T13:35:16.513 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:16.511+0000 7f85d4fdc640 1 --2- 172.21.15.43:0/1586898962 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f85d00b3fa0 0x7f85d00b43b0 secure :-1 s=READY pgs=312 cs=0 l=1 rev1=1 crypto rx=0x7f85c8009d90 tx=0x7f85c802bf40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=60e104732653cddc server_cookie=0 in_seq=0 out_seq=0 2023-08-09T13:35:16.513 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:16.512+0000 7f85cf7fe640 1 -- 172.21.15.43:0/1586898962 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f85c802ccf0 con 0x7f85d00b3fa0 2023-08-09T13:35:16.513 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:16.512+0000 7f85cf7fe640 1 -- 172.21.15.43:0/1586898962 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f85c802ce50 con 0x7f85d00b3fa0 2023-08-09T13:35:16.513 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:16.512+0000 7f85cf7fe640 1 -- 172.21.15.43:0/1586898962 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f85c8034660 con 0x7f85d00b3fa0 2023-08-09T13:35:16.514 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:16.512+0000 7f85d6a3f640 1 -- 172.21.15.43:0/1586898962 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f85d00b3fa0 msgr2=0x7f85d00b43b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:35:16.514 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:16.512+0000 7f85d6a3f640 1 --2- 172.21.15.43:0/1586898962 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f85d00b3fa0 0x7f85d00b43b0 secure :-1 s=READY pgs=312 cs=0 l=1 rev1=1 crypto rx=0x7f85c8009d90 tx=0x7f85c802bf40 comp rx=0 tx=0).stop 2023-08-09T13:35:16.514 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:16.513+0000 7f85d6a3f640 1 -- 172.21.15.43:0/1586898962 shutdown_connections 2023-08-09T13:35:16.514 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:16.513+0000 7f85d6a3f640 1 --2- 172.21.15.43:0/1586898962 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f85d00b3fa0 0x7f85d00b43b0 unknown :-1 s=CLOSED pgs=312 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:35:16.515 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:16.513+0000 7f85d6a3f640 1 -- 172.21.15.43:0/1586898962 >> 172.21.15.43:0/1586898962 conn(0x7f85d0071520 msgr2=0x7f85d00d2520 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-09T13:35:16.515 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:16.513+0000 7f85d6a3f640 1 -- 172.21.15.43:0/1586898962 shutdown_connections 2023-08-09T13:35:16.515 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:16.513+0000 7f85d6a3f640 1 -- 172.21.15.43:0/1586898962 wait complete. 2023-08-09T13:35:16.516 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:16.514+0000 7f85d6a3f640 1 Processor -- start 2023-08-09T13:35:16.516 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:16.515+0000 7f85d6a3f640 1 -- start start 2023-08-09T13:35:16.517 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:16.516+0000 7f85d6a3f640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f85d00b3fa0 0x7f85d00ca000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:35:16.517 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:16.516+0000 7f85d6a3f640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f85d00d0a70 0x7f85d0156fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:35:16.517 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:16.516+0000 7f85d6a3f640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f85d015c680 0x7f85d00cbe30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:35:16.518 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:16.516+0000 7f85d6a3f640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f85d0154db0 con 0x7f85d00b3fa0 2023-08-09T13:35:16.518 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:16.516+0000 7f85d6a3f640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f85d00cb160 con 0x7f85d00d0a70 2023-08-09T13:35:16.518 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:16.516+0000 7f85d6a3f640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f85d0157520 con 0x7f85d015c680 2023-08-09T13:35:16.518 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:16.516+0000 7f85cffff640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f85d00b3fa0 0x7f85d00ca000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:35:16.518 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:16.516+0000 7f85cffff640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f85d00b3fa0 0x7f85d00ca000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:42424/0 (socket says 172.21.15.43:42424) 2023-08-09T13:35:16.518 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:16.516+0000 7f85cffff640 1 -- 172.21.15.43:0/2590204203 learned_addr learned my addr 172.21.15.43:0/2590204203 (peer_addr_for_me v2:172.21.15.43:0/0) 2023-08-09T13:35:16.519 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:16.516+0000 7f85c77fe640 1 --2- 172.21.15.43:0/2590204203 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f85d00d0a70 0x7f85d0156fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:35:16.519 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:16.516+0000 7f85cffff640 1 -- 172.21.15.43:0/2590204203 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f85d015c680 msgr2=0x7f85d00cbe30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:35:16.519 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:16.516+0000 7f85cffff640 1 --2- 172.21.15.43:0/2590204203 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f85d015c680 0x7f85d00cbe30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:35:16.520 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:16.516+0000 7f85cffff640 1 -- 172.21.15.43:0/2590204203 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f85d00d0a70 msgr2=0x7f85d0156fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:35:16.520 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:16.516+0000 7f85cffff640 1 --2- 172.21.15.43:0/2590204203 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f85d00d0a70 0x7f85d0156fe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:35:16.520 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:16.516+0000 7f85cffff640 1 -- 172.21.15.43:0/2590204203 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f85c8009a70 con 0x7f85d00b3fa0 2023-08-09T13:35:16.520 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:16.517+0000 7f85cffff640 1 --2- 172.21.15.43:0/2590204203 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f85d00b3fa0 0x7f85d00ca000 secure :-1 s=READY pgs=313 cs=0 l=1 rev1=1 crypto rx=0x7f85c000c3f0 tx=0x7f85c000c8b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-09T13:35:16.520 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:16.517+0000 7f85cd7fa640 1 -- 172.21.15.43:0/2590204203 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f85c00074e0 con 0x7f85d00b3fa0 2023-08-09T13:35:16.520 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:16.517+0000 7f85cd7fa640 1 -- 172.21.15.43:0/2590204203 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f85c0007640 con 0x7f85d00b3fa0 2023-08-09T13:35:16.521 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:16.517+0000 7f85cd7fa640 1 -- 172.21.15.43:0/2590204203 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f85c0004a00 con 0x7f85d00b3fa0 2023-08-09T13:35:16.521 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:16.517+0000 7f85d6a3f640 1 -- 172.21.15.43:0/2590204203 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f85d00ca5e0 con 0x7f85d00b3fa0 2023-08-09T13:35:16.521 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:16.517+0000 7f85d6a3f640 1 -- 172.21.15.43:0/2590204203 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f85d00caba0 con 0x7f85d00b3fa0 2023-08-09T13:35:16.521 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:16.519+0000 7f85d6a3f640 1 -- 172.21.15.43:0/2590204203 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f85d0154f20 con 0x7f85d00b3fa0 2023-08-09T13:35:16.521 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:16.520+0000 7f85cd7fa640 1 -- 172.21.15.43:0/2590204203 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 5) v1 ==== 88341+0+0 (secure 0 0 0) 0x7f85c0004b60 con 0x7f85d00b3fa0 2023-08-09T13:35:16.522 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:16.521+0000 7f85cd7fa640 1 --2- 172.21.15.43:0/2590204203 >> [v2:172.21.15.203:6832/830097474,v1:172.21.15.203:6833/830097474] conn(0x7f85b006a080 0x7f85b006c540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:35:16.522 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:16.521+0000 7f85cd7fa640 1 -- 172.21.15.43:0/2590204203 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 6871+0+0 (secure 0 0 0) 0x7f85c0088630 con 0x7f85d00b3fa0 2023-08-09T13:35:16.523 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:16.522+0000 7f85c77fe640 1 --2- 172.21.15.43:0/2590204203 >> [v2:172.21.15.203:6832/830097474,v1:172.21.15.203:6833/830097474] conn(0x7f85b006a080 0x7f85b006c540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:35:16.524 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:16.523+0000 7f85c77fe640 1 --2- 172.21.15.43:0/2590204203 >> [v2:172.21.15.203:6832/830097474,v1:172.21.15.203:6833/830097474] conn(0x7f85b006a080 0x7f85b006c540 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7f85d0159f10 tx=0x7f85b8009040 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-09T13:35:16.526 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:16.525+0000 7f85cd7fa640 1 -- 172.21.15.43:0/2590204203 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178066 (secure 0 0 0) 0x7f85c00539f0 con 0x7f85d00b3fa0 2023-08-09T13:35:16.720 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:16.718+0000 7f85d6a3f640 1 -- 172.21.15.43:0/2590204203 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f85d00739f0 con 0x7f85d00b3fa0 2023-08-09T13:35:16.721 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:16.719+0000 7f85cd7fa640 1 -- 172.21.15.43:0/2590204203 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v26) v1 ==== 104+0+11 (secure 0 0 0) 0x7f85c0057010 con 0x7f85d00b3fa0 2023-08-09T13:35:16.721 INFO:teuthology.orchestra.run.smithi043.stdout:pg_num: 32 2023-08-09T13:35:16.724 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:16.722+0000 7f85d6a3f640 1 -- 172.21.15.43:0/2590204203 >> [v2:172.21.15.203:6832/830097474,v1:172.21.15.203:6833/830097474] conn(0x7f85b006a080 msgr2=0x7f85b006c540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:35:16.724 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:16.722+0000 7f85d6a3f640 1 --2- 172.21.15.43:0/2590204203 >> [v2:172.21.15.203:6832/830097474,v1:172.21.15.203:6833/830097474] conn(0x7f85b006a080 0x7f85b006c540 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7f85d0159f10 tx=0x7f85b8009040 comp rx=0 tx=0).stop 2023-08-09T13:35:16.724 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:16.722+0000 7f85d6a3f640 1 -- 172.21.15.43:0/2590204203 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f85d00b3fa0 msgr2=0x7f85d00ca000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:35:16.724 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:16.722+0000 7f85d6a3f640 1 --2- 172.21.15.43:0/2590204203 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f85d00b3fa0 0x7f85d00ca000 secure :-1 s=READY pgs=313 cs=0 l=1 rev1=1 crypto rx=0x7f85c000c3f0 tx=0x7f85c000c8b0 comp rx=0 tx=0).stop 2023-08-09T13:35:16.725 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:16.722+0000 7f85d6a3f640 1 -- 172.21.15.43:0/2590204203 shutdown_connections 2023-08-09T13:35:16.725 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:16.722+0000 7f85d6a3f640 1 --2- 172.21.15.43:0/2590204203 >> [v2:172.21.15.203:6832/830097474,v1:172.21.15.203:6833/830097474] conn(0x7f85b006a080 0x7f85b006c540 unknown :-1 s=CLOSED pgs=167 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:35:16.725 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:16.722+0000 7f85d6a3f640 1 --2- 172.21.15.43:0/2590204203 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f85d015c680 0x7f85d00cbe30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:35:16.725 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:16.722+0000 7f85d6a3f640 1 --2- 172.21.15.43:0/2590204203 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f85d00d0a70 0x7f85d0156fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:35:16.725 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:16.722+0000 7f85d6a3f640 1 --2- 172.21.15.43:0/2590204203 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f85d00b3fa0 0x7f85d00ca000 unknown :-1 s=CLOSED pgs=313 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:35:16.726 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:16.722+0000 7f85d6a3f640 1 -- 172.21.15.43:0/2590204203 >> 172.21.15.43:0/2590204203 conn(0x7f85d0071520 msgr2=0x7f85d00cf3d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-09T13:35:16.726 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:16.722+0000 7f85d6a3f640 1 -- 172.21.15.43:0/2590204203 shutdown_connections 2023-08-09T13:35:16.726 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:16.722+0000 7f85d6a3f640 1 -- 172.21.15.43:0/2590204203 wait complete. 2023-08-09T13:35:16.742 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-08-09T13:35:16.959 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:16.958+0000 7fc2d4d37640 1 Processor -- start 2023-08-09T13:35:16.960 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:16.958+0000 7fc2d4d37640 1 -- start start 2023-08-09T13:35:16.961 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:16.959+0000 7fc2d4d37640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fc2d00b7df0 0x7fc2d00d1b30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:35:16.961 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:16.959+0000 7fc2d4d37640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7fc2d00cebf0 0x7fc2d00cf060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:35:16.961 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:16.959+0000 7fc2d4d37640 1 -- --> v1:172.21.15.203:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc2d00d16a0 con 0x7fc2d015c2d0 2023-08-09T13:35:16.961 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:16.959+0000 7fc2d4d37640 1 -- --> v2:172.21.15.43:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc2d00b8230 con 0x7fc2d00cebf0 2023-08-09T13:35:16.961 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:16.959+0000 7fc2d4d37640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc2d00cf5a0 con 0x7fc2d00b7df0 2023-08-09T13:35:16.961 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:16.960+0000 7fc2cffff640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7fc2d00cebf0 0x7fc2d00cf060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:35:16.962 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:16.960+0000 7fc2cffff640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7fc2d00cebf0 0x7fc2d00cf060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:42428/0 (socket says 172.21.15.43:42428) 2023-08-09T13:35:16.962 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:16.960+0000 7fc2cffff640 1 -- 172.21.15.43:0/1820859060 learned_addr learned my addr 172.21.15.43:0/1820859060 (peer_addr_for_me v2:172.21.15.43:0/0) 2023-08-09T13:35:16.963 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:16.960+0000 7fc2c6ffd640 1 --2- 172.21.15.43:0/1820859060 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fc2d00b7df0 0x7fc2d00d1b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:35:16.963 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:16.960+0000 7fc2cffff640 1 -- 172.21.15.43:0/1820859060 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fc2d00b7df0 msgr2=0x7fc2d00d1b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-09T13:35:16.963 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:16.960+0000 7fc2cffff640 1 --2- 172.21.15.43:0/1820859060 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fc2d00b7df0 0x7fc2d00d1b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:35:16.963 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:16.960+0000 7fc2cffff640 1 -- 172.21.15.43:0/1820859060 >> v1:172.21.15.203:6789/0 conn(0x7fc2d015c2d0 legacy=0x7fc2d00cd020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-09T13:35:16.963 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:16.960+0000 7fc2cffff640 1 -- 172.21.15.43:0/1820859060 --> v2:172.21.15.43:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc2d00d0d70 con 0x7fc2d00cebf0 2023-08-09T13:35:16.964 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:16.962+0000 7fc2cffff640 1 --2- 172.21.15.43:0/1820859060 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc2d00cebf0 0x7fc2d00cf060 secure :-1 s=READY pgs=314 cs=0 l=1 rev1=1 crypto rx=0x7fc2c8009e50 tx=0x7fc2c802bed0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=91b642ca62e055b server_cookie=0 in_seq=0 out_seq=0 2023-08-09T13:35:16.964 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:16.962+0000 7fc2ceffd640 1 -- 172.21.15.43:0/1820859060 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc2c802ca70 con 0x7fc2d00cebf0 2023-08-09T13:35:16.964 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:16.962+0000 7fc2ceffd640 1 -- 172.21.15.43:0/1820859060 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc2c802cbd0 con 0x7fc2d00cebf0 2023-08-09T13:35:16.964 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:16.962+0000 7fc2ceffd640 1 -- 172.21.15.43:0/1820859060 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc2c80343f0 con 0x7fc2d00cebf0 2023-08-09T13:35:16.965 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:16.963+0000 7fc2d4d37640 1 -- 172.21.15.43:0/1820859060 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc2d00cebf0 msgr2=0x7fc2d00cf060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:35:16.965 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:16.963+0000 7fc2d4d37640 1 --2- 172.21.15.43:0/1820859060 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc2d00cebf0 0x7fc2d00cf060 secure :-1 s=READY pgs=314 cs=0 l=1 rev1=1 crypto rx=0x7fc2c8009e50 tx=0x7fc2c802bed0 comp rx=0 tx=0).stop 2023-08-09T13:35:16.965 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:16.963+0000 7fc2d4d37640 1 -- 172.21.15.43:0/1820859060 shutdown_connections 2023-08-09T13:35:16.965 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:16.963+0000 7fc2d4d37640 1 --2- 172.21.15.43:0/1820859060 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc2d00cebf0 0x7fc2d00cf060 unknown :-1 s=CLOSED pgs=314 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:35:16.966 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:16.963+0000 7fc2d4d37640 1 --2- 172.21.15.43:0/1820859060 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fc2d00b7df0 0x7fc2d00d1b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:35:16.966 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:16.963+0000 7fc2d4d37640 1 -- 172.21.15.43:0/1820859060 >> 172.21.15.43:0/1820859060 conn(0x7fc2d006a230 msgr2=0x7fc2d006a640 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-09T13:35:16.966 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:16.963+0000 7fc2d4d37640 1 -- 172.21.15.43:0/1820859060 shutdown_connections 2023-08-09T13:35:16.966 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:16.963+0000 7fc2d4d37640 1 -- 172.21.15.43:0/1820859060 wait complete. 2023-08-09T13:35:16.966 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:16.965+0000 7fc2d4d37640 1 Processor -- start 2023-08-09T13:35:16.967 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:16.965+0000 7fc2d4d37640 1 -- start start 2023-08-09T13:35:16.968 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:16.966+0000 7fc2d4d37640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc2d00b7df0 0x7fc2d00b62c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:35:16.968 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:16.966+0000 7fc2d4d37640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fc2d00cebf0 0x7fc2d00c92b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:35:16.968 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:16.966+0000 7fc2d4d37640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fc2d015c2d0 0x7fc2d00c97f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:35:16.968 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:16.966+0000 7fc2d4d37640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc2d00cb2e0 con 0x7fc2d00b7df0 2023-08-09T13:35:16.968 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:16.966+0000 7fc2d4d37640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc2d00cb540 con 0x7fc2d00cebf0 2023-08-09T13:35:16.968 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:16.966+0000 7fc2d4d37640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc2d00cb820 con 0x7fc2d015c2d0 2023-08-09T13:35:16.969 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:16.966+0000 7fc2cf7fe640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc2d00b7df0 0x7fc2d00b62c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:35:16.969 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:16.966+0000 7fc2cf7fe640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc2d00b7df0 0x7fc2d00b62c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:42440/0 (socket says 172.21.15.43:42440) 2023-08-09T13:35:16.969 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:16.966+0000 7fc2cf7fe640 1 -- 172.21.15.43:0/2971577532 learned_addr learned my addr 172.21.15.43:0/2971577532 (peer_addr_for_me v2:172.21.15.43:0/0) 2023-08-09T13:35:16.970 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:16.967+0000 7fc2c6ffd640 1 --2- 172.21.15.43:0/2971577532 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fc2d00cebf0 0x7fc2d00c92b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:35:16.970 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:16.967+0000 7fc2cf7fe640 1 -- 172.21.15.43:0/2971577532 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fc2d015c2d0 msgr2=0x7fc2d00c97f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:35:16.970 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:16.967+0000 7fc2cf7fe640 1 --2- 172.21.15.43:0/2971577532 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fc2d015c2d0 0x7fc2d00c97f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:35:16.971 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:16.967+0000 7fc2cf7fe640 1 -- 172.21.15.43:0/2971577532 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fc2d00cebf0 msgr2=0x7fc2d00c92b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:35:16.971 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:16.967+0000 7fc2cf7fe640 1 --2- 172.21.15.43:0/2971577532 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fc2d00cebf0 0x7fc2d00c92b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:35:16.971 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:16.967+0000 7fc2cf7fe640 1 -- 172.21.15.43:0/2971577532 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc2c8009b30 con 0x7fc2d00b7df0 2023-08-09T13:35:16.971 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:16.967+0000 7fc2cf7fe640 1 --2- 172.21.15.43:0/2971577532 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc2d00b7df0 0x7fc2d00b62c0 secure :-1 s=READY pgs=315 cs=0 l=1 rev1=1 crypto rx=0x7fc2c000c430 tx=0x7fc2c000c8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-09T13:35:16.971 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:16.968+0000 7fc2ccff9640 1 -- 172.21.15.43:0/2971577532 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc2c0011790 con 0x7fc2d00b7df0 2023-08-09T13:35:16.971 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:16.968+0000 7fc2ccff9640 1 -- 172.21.15.43:0/2971577532 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc2c00118f0 con 0x7fc2d00b7df0 2023-08-09T13:35:16.971 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:16.968+0000 7fc2d4d37640 1 -- 172.21.15.43:0/2971577532 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc2d00cbb00 con 0x7fc2d00b7df0 2023-08-09T13:35:16.972 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:16.968+0000 7fc2ccff9640 1 -- 172.21.15.43:0/2971577532 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc2c0011ce0 con 0x7fc2d00b7df0 2023-08-09T13:35:16.972 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:16.968+0000 7fc2d4d37640 1 -- 172.21.15.43:0/2971577532 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc2d015da80 con 0x7fc2d00b7df0 2023-08-09T13:35:16.972 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:16.969+0000 7fc2d4d37640 1 -- 172.21.15.43:0/2971577532 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc2d004e390 con 0x7fc2d00b7df0 2023-08-09T13:35:16.973 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:16.970+0000 7fc2ccff9640 1 -- 172.21.15.43:0/2971577532 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 5) v1 ==== 88341+0+0 (secure 0 0 0) 0x7fc2c001c460 con 0x7fc2d00b7df0 2023-08-09T13:35:16.973 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:16.971+0000 7fc2ccff9640 1 --2- 172.21.15.43:0/2971577532 >> [v2:172.21.15.203:6832/830097474,v1:172.21.15.203:6833/830097474] conn(0x7fc2b006a050 0x7fc2b006c510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:35:16.973 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:16.971+0000 7fc2ccff9640 1 -- 172.21.15.43:0/2971577532 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 6871+0+0 (secure 0 0 0) 0x7fc2c00888e0 con 0x7fc2d00b7df0 2023-08-09T13:35:16.973 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:16.972+0000 7fc2c6ffd640 1 --2- 172.21.15.43:0/2971577532 >> [v2:172.21.15.203:6832/830097474,v1:172.21.15.203:6833/830097474] conn(0x7fc2b006a050 0x7fc2b006c510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:35:16.974 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:16.973+0000 7fc2c6ffd640 1 --2- 172.21.15.43:0/2971577532 >> [v2:172.21.15.203:6832/830097474,v1:172.21.15.203:6833/830097474] conn(0x7fc2b006a050 0x7fc2b006c510 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7fc2b8006fd0 tx=0x7fc2b8008040 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-09T13:35:16.977 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:16.976+0000 7fc2ccff9640 1 -- 172.21.15.43:0/2971577532 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178066 (secure 0 0 0) 0x7fc2c0053c00 con 0x7fc2d00b7df0 2023-08-09T13:35:17.173 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:17.171+0000 7fc2d4d37640 1 -- 172.21.15.43:0/2971577532 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7fc2d0153d00 con 0x7fc2d00b7df0 2023-08-09T13:35:17.173 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:17.172+0000 7fc2ccff9640 1 -- 172.21.15.43:0/2971577532 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v26) v1 ==== 100+0+11 (secure 0 0 0) 0x7fc2c0057220 con 0x7fc2d00b7df0 2023-08-09T13:35:17.174 INFO:teuthology.orchestra.run.smithi043.stdout:pg_num: 64 2023-08-09T13:35:17.176 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:17.174+0000 7fc2d4d37640 1 -- 172.21.15.43:0/2971577532 >> [v2:172.21.15.203:6832/830097474,v1:172.21.15.203:6833/830097474] conn(0x7fc2b006a050 msgr2=0x7fc2b006c510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:35:17.177 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:17.174+0000 7fc2d4d37640 1 --2- 172.21.15.43:0/2971577532 >> [v2:172.21.15.203:6832/830097474,v1:172.21.15.203:6833/830097474] conn(0x7fc2b006a050 0x7fc2b006c510 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7fc2b8006fd0 tx=0x7fc2b8008040 comp rx=0 tx=0).stop 2023-08-09T13:35:17.177 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:17.174+0000 7fc2d4d37640 1 -- 172.21.15.43:0/2971577532 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc2d00b7df0 msgr2=0x7fc2d00b62c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:35:17.177 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:17.174+0000 7fc2d4d37640 1 --2- 172.21.15.43:0/2971577532 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc2d00b7df0 0x7fc2d00b62c0 secure :-1 s=READY pgs=315 cs=0 l=1 rev1=1 crypto rx=0x7fc2c000c430 tx=0x7fc2c000c8f0 comp rx=0 tx=0).stop 2023-08-09T13:35:17.177 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:17.175+0000 7fc2d4d37640 1 -- 172.21.15.43:0/2971577532 shutdown_connections 2023-08-09T13:35:17.177 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:17.175+0000 7fc2d4d37640 1 --2- 172.21.15.43:0/2971577532 >> [v2:172.21.15.203:6832/830097474,v1:172.21.15.203:6833/830097474] conn(0x7fc2b006a050 0x7fc2b006c510 unknown :-1 s=CLOSED pgs=168 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:35:17.178 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:17.175+0000 7fc2d4d37640 1 --2- 172.21.15.43:0/2971577532 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fc2d015c2d0 0x7fc2d00c97f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:35:17.178 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:17.175+0000 7fc2d4d37640 1 --2- 172.21.15.43:0/2971577532 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fc2d00cebf0 0x7fc2d00c92b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:35:17.178 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:17.175+0000 7fc2d4d37640 1 --2- 172.21.15.43:0/2971577532 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc2d00b7df0 0x7fc2d00b62c0 unknown :-1 s=CLOSED pgs=315 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:35:17.178 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:17.175+0000 7fc2d4d37640 1 -- 172.21.15.43:0/2971577532 >> 172.21.15.43:0/2971577532 conn(0x7fc2d006a230 msgr2=0x7fc2d00cfd90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-09T13:35:17.178 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:17.175+0000 7fc2d4d37640 1 -- 172.21.15.43:0/2971577532 shutdown_connections 2023-08-09T13:35:17.178 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:17.175+0000 7fc2d4d37640 1 -- 172.21.15.43:0/2971577532 wait complete. 2023-08-09T13:35:17.196 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2023-08-09T13:35:17.426 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:17.424+0000 7f0a0e856640 1 Processor -- start 2023-08-09T13:35:17.426 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:17.425+0000 7f0a0e856640 1 -- start start 2023-08-09T13:35:17.427 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:17.426+0000 7f0a0e856640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f0a08157ce0 0x7f0a0815b0f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:35:17.428 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:17.426+0000 7f0a0e856640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f0a080c8470 0x7f0a080c88c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:35:17.428 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:17.426+0000 7f0a0e856640 1 -- --> v1:172.21.15.43:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0a080c8e00 con 0x7f0a0806f2c0 2023-08-09T13:35:17.428 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:17.426+0000 7f0a0e856640 1 -- --> v2:172.21.15.43:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0a081580c0 con 0x7f0a080c8470 2023-08-09T13:35:17.428 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:17.426+0000 7f0a0e856640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0a080c92c0 con 0x7f0a08157ce0 2023-08-09T13:35:17.429 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:17.426+0000 7f0a0d053640 1 --1- >> v1:172.21.15.43:6789/0 conn(0x7f0a0806f2c0 0x7f0a0806bc40 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.43:6789/0 says I am v1:172.21.15.43:49142/0 (socket says 172.21.15.43:49142) 2023-08-09T13:35:17.429 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:17.426+0000 7f0a0d053640 1 -- 172.21.15.43:0/921229421 learned_addr learned my addr 172.21.15.43:0/921229421 (peer_addr_for_me v1:172.21.15.43:0/0) 2023-08-09T13:35:17.429 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:17.427+0000 7f0a0e055640 1 --2- 172.21.15.43:0/921229421 >> v2:172.21.15.43:3300/0 conn(0x7f0a080c8470 0x7f0a080c88c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:35:17.430 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:17.427+0000 7f0a0d854640 1 --2- 172.21.15.43:0/921229421 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f0a08157ce0 0x7f0a0815b0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:35:17.430 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:17.427+0000 7f0a0c852640 1 -- 172.21.15.43:0/921229421 <== mon.0 v1:172.21.15.43:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 377852141 0 0) 0x7f09f8001110 con 0x7f0a0806f2c0 2023-08-09T13:35:17.430 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:17.427+0000 7f0a0c852640 1 -- 172.21.15.43:0/921229421 --> v1:172.21.15.43:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f09f4003520 con 0x7f0a0806f2c0 2023-08-09T13:35:17.430 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:17.427+0000 7f0a0e055640 1 -- 172.21.15.43:0/921229421 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f0a08157ce0 msgr2=0x7f0a0815b0f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-09T13:35:17.430 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:17.427+0000 7f0a0e055640 1 --2- 172.21.15.43:0/921229421 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f0a08157ce0 0x7f0a0815b0f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:35:17.431 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:17.427+0000 7f0a0e055640 1 -- 172.21.15.43:0/921229421 >> v1:172.21.15.43:6789/0 conn(0x7f0a0806f2c0 legacy=0x7f0a0806bc40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:35:17.431 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:17.427+0000 7f0a0e055640 1 -- 172.21.15.43:0/921229421 --> v2:172.21.15.43:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0a0806b480 con 0x7f0a080c8470 2023-08-09T13:35:17.432 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:17.429+0000 7f0a0e055640 1 --2- 172.21.15.43:0/921229421 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f0a080c8470 0x7f0a080c88c0 secure :-1 s=READY pgs=317 cs=0 l=1 rev1=1 crypto rx=0x7f0a00009d90 tx=0x7f0a0002be20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f280cf6f03658df8 server_cookie=0 in_seq=0 out_seq=0 2023-08-09T13:35:17.432 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:17.429+0000 7f0a0c852640 1 -- 172.21.15.43:0/921229421 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0a0002cbd0 con 0x7f0a080c8470 2023-08-09T13:35:17.432 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:17.429+0000 7f0a0c852640 1 -- 172.21.15.43:0/921229421 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0a0002cd30 con 0x7f0a080c8470 2023-08-09T13:35:17.432 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:17.429+0000 7f0a0c852640 1 -- 172.21.15.43:0/921229421 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0a0002cea0 con 0x7f0a080c8470 2023-08-09T13:35:17.432 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:17.430+0000 7f0a0e856640 1 -- 172.21.15.43:0/921229421 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f0a080c8470 msgr2=0x7f0a080c88c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:35:17.433 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:17.430+0000 7f0a0e856640 1 --2- 172.21.15.43:0/921229421 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f0a080c8470 0x7f0a080c88c0 secure :-1 s=READY pgs=317 cs=0 l=1 rev1=1 crypto rx=0x7f0a00009d90 tx=0x7f0a0002be20 comp rx=0 tx=0).stop 2023-08-09T13:35:17.433 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:17.430+0000 7f0a0e856640 1 -- 172.21.15.43:0/921229421 shutdown_connections 2023-08-09T13:35:17.433 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:17.430+0000 7f0a0e856640 1 --2- 172.21.15.43:0/921229421 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f0a080c8470 0x7f0a080c88c0 unknown :-1 s=CLOSED pgs=317 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:35:17.433 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:17.430+0000 7f0a0e856640 1 --2- 172.21.15.43:0/921229421 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f0a08157ce0 0x7f0a0815b0f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:35:17.433 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:17.430+0000 7f0a0e856640 1 -- 172.21.15.43:0/921229421 >> 172.21.15.43:0/921229421 conn(0x7f0a080af9f0 msgr2=0x7f0a080afdd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-09T13:35:17.433 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:17.430+0000 7f0a0e856640 1 -- 172.21.15.43:0/921229421 shutdown_connections 2023-08-09T13:35:17.433 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:17.430+0000 7f0a0e856640 1 -- 172.21.15.43:0/921229421 wait complete. 2023-08-09T13:35:17.435 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:17.431+0000 7f0a0e856640 1 Processor -- start 2023-08-09T13:35:17.436 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:17.432+0000 7f0a0e856640 1 -- start start 2023-08-09T13:35:17.436 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:17.433+0000 7f0a0e856640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f0a0806f2c0 0x7f0a081f1700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:35:17.436 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:17.433+0000 7f0a0e856640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f0a080c8470 0x7f0a080c08a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:35:17.436 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:17.433+0000 7f0a0e856640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f0a08157ce0 0x7f0a080c0de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:35:17.437 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:17.433+0000 7f0a0e856640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0a081f2d20 con 0x7f0a0806f2c0 2023-08-09T13:35:17.437 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:17.433+0000 7f0a0e856640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0a080c7b90 con 0x7f0a08157ce0 2023-08-09T13:35:17.437 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:17.433+0000 7f0a0e856640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0a080c78b0 con 0x7f0a080c8470 2023-08-09T13:35:17.437 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:17.433+0000 7f0a0d854640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f0a0806f2c0 0x7f0a081f1700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:35:17.437 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:17.433+0000 7f0a0d854640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f0a0806f2c0 0x7f0a081f1700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:42468/0 (socket says 172.21.15.43:42468) 2023-08-09T13:35:17.437 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:17.433+0000 7f0a0d854640 1 -- 172.21.15.43:0/2536203219 learned_addr learned my addr 172.21.15.43:0/2536203219 (peer_addr_for_me v2:172.21.15.43:0/0) 2023-08-09T13:35:17.438 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:17.433+0000 7f0a0d053640 1 --2- 172.21.15.43:0/2536203219 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f0a080c8470 0x7f0a080c08a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:35:17.438 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:17.433+0000 7f0a0d854640 1 -- 172.21.15.43:0/2536203219 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f0a080c8470 msgr2=0x7f0a080c08a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:35:17.438 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:17.433+0000 7f0a0d854640 1 --2- 172.21.15.43:0/2536203219 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f0a080c8470 0x7f0a080c08a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:35:17.438 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:17.433+0000 7f0a0d854640 1 -- 172.21.15.43:0/2536203219 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f0a08157ce0 msgr2=0x7f0a080c0de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:35:17.438 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:17.433+0000 7f0a0d854640 1 --2- 172.21.15.43:0/2536203219 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f0a08157ce0 0x7f0a080c0de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:35:17.439 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:17.433+0000 7f0a0d854640 1 -- 172.21.15.43:0/2536203219 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0a00009a70 con 0x7f0a0806f2c0 2023-08-09T13:35:17.439 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:17.434+0000 7f0a0d854640 1 --2- 172.21.15.43:0/2536203219 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f0a0806f2c0 0x7f0a081f1700 secure :-1 s=READY pgs=318 cs=0 l=1 rev1=1 crypto rx=0x7f0a0400d800 tx=0x7f0a0400dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-09T13:35:17.439 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:17.434+0000 7f09fe7fc640 1 -- 172.21.15.43:0/2536203219 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0a0400b6c0 con 0x7f0a0806f2c0 2023-08-09T13:35:17.439 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:17.434+0000 7f09fe7fc640 1 -- 172.21.15.43:0/2536203219 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0a0400b820 con 0x7f0a0806f2c0 2023-08-09T13:35:17.439 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:17.434+0000 7f0a0e856640 1 -- 172.21.15.43:0/2536203219 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0a081f26d0 con 0x7f0a0806f2c0 2023-08-09T13:35:17.439 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:17.434+0000 7f09fe7fc640 1 -- 172.21.15.43:0/2536203219 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0a0400bbb0 con 0x7f0a0806f2c0 2023-08-09T13:35:17.439 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:17.434+0000 7f0a0e856640 1 -- 172.21.15.43:0/2536203219 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0a081f1c40 con 0x7f0a0806f2c0 2023-08-09T13:35:17.440 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:17.435+0000 7f0a0e856640 1 -- 172.21.15.43:0/2536203219 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0a081f1e30 con 0x7f0a0806f2c0 2023-08-09T13:35:17.440 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:17.436+0000 7f09fe7fc640 1 -- 172.21.15.43:0/2536203219 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 5) v1 ==== 88341+0+0 (secure 0 0 0) 0x7f0a0400bd10 con 0x7f0a0806f2c0 2023-08-09T13:35:17.440 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:17.437+0000 7f09fe7fc640 1 --2- 172.21.15.43:0/2536203219 >> [v2:172.21.15.203:6832/830097474,v1:172.21.15.203:6833/830097474] conn(0x7f09ec06a050 0x7f09ec06c510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:35:17.441 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:17.437+0000 7f09fe7fc640 1 -- 172.21.15.43:0/2536203219 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 6871+0+0 (secure 0 0 0) 0x7f0a04088e60 con 0x7f0a0806f2c0 2023-08-09T13:35:17.441 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:17.438+0000 7f0a0d053640 1 --2- 172.21.15.43:0/2536203219 >> [v2:172.21.15.203:6832/830097474,v1:172.21.15.203:6833/830097474] conn(0x7f09ec06a050 0x7f09ec06c510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:35:17.441 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:17.439+0000 7f0a0d053640 1 --2- 172.21.15.43:0/2536203219 >> [v2:172.21.15.203:6832/830097474,v1:172.21.15.203:6833/830097474] conn(0x7f09ec06a050 0x7f09ec06c510 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7f09f8001270 tx=0x7f09f8009040 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-09T13:35:17.442 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:17.441+0000 7f09fe7fc640 1 -- 172.21.15.43:0/2536203219 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178066 (secure 0 0 0) 0x7f0a04054210 con 0x7f0a0806f2c0 2023-08-09T13:35:17.627 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:17.626+0000 7f0a0e856640 1 -- 172.21.15.43:0/2536203219 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) v1 -- 0x7f0a0804e390 con 0x7f0a0806f2c0 2023-08-09T13:35:17.628 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:17.626+0000 7f09fe7fc640 1 -- 172.21.15.43:0/2536203219 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v26) v1 ==== 103+0+11 (secure 0 0 0) 0x7f0a04057830 con 0x7f0a0806f2c0 2023-08-09T13:35:17.628 INFO:teuthology.orchestra.run.smithi043.stdout:pg_num: 32 2023-08-09T13:35:17.630 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:17.629+0000 7f0a0e856640 1 -- 172.21.15.43:0/2536203219 >> [v2:172.21.15.203:6832/830097474,v1:172.21.15.203:6833/830097474] conn(0x7f09ec06a050 msgr2=0x7f09ec06c510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:35:17.631 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:17.629+0000 7f0a0e856640 1 --2- 172.21.15.43:0/2536203219 >> [v2:172.21.15.203:6832/830097474,v1:172.21.15.203:6833/830097474] conn(0x7f09ec06a050 0x7f09ec06c510 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7f09f8001270 tx=0x7f09f8009040 comp rx=0 tx=0).stop 2023-08-09T13:35:17.631 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:17.629+0000 7f0a0e856640 1 -- 172.21.15.43:0/2536203219 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f0a0806f2c0 msgr2=0x7f0a081f1700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:35:17.631 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:17.629+0000 7f0a0e856640 1 --2- 172.21.15.43:0/2536203219 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f0a0806f2c0 0x7f0a081f1700 secure :-1 s=READY pgs=318 cs=0 l=1 rev1=1 crypto rx=0x7f0a0400d800 tx=0x7f0a0400dcc0 comp rx=0 tx=0).stop 2023-08-09T13:35:17.631 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:17.629+0000 7f0a0e856640 1 -- 172.21.15.43:0/2536203219 shutdown_connections 2023-08-09T13:35:17.631 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:17.629+0000 7f0a0e856640 1 --2- 172.21.15.43:0/2536203219 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f0a08157ce0 0x7f0a080c0de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:35:17.631 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:17.629+0000 7f0a0e856640 1 --2- 172.21.15.43:0/2536203219 >> [v2:172.21.15.203:6832/830097474,v1:172.21.15.203:6833/830097474] conn(0x7f09ec06a050 0x7f09ec06c510 unknown :-1 s=CLOSED pgs=169 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:35:17.631 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:17.629+0000 7f0a0e856640 1 --2- 172.21.15.43:0/2536203219 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f0a080c8470 0x7f0a080c08a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:35:17.631 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:17.629+0000 7f0a0e856640 1 --2- 172.21.15.43:0/2536203219 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f0a0806f2c0 0x7f0a081f1700 unknown :-1 s=CLOSED pgs=318 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:35:17.632 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:17.629+0000 7f0a0e856640 1 -- 172.21.15.43:0/2536203219 >> 172.21.15.43:0/2536203219 conn(0x7f0a080af9f0 msgr2=0x7f0a080c3fc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-09T13:35:17.632 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:17.629+0000 7f0a0e856640 1 -- 172.21.15.43:0/2536203219 shutdown_connections 2023-08-09T13:35:17.632 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:17.629+0000 7f0a0e856640 1 -- 172.21.15.43:0/2536203219 wait complete. 2023-08-09T13:35:17.664 INFO:tasks.cephfs_test_runner:Starting test: test_concurrent_snap_creates (tasks.cephfs.test_snap_schedules.TestSnapSchedules) Test concurrent snap creates in same file-system without db issues 2023-08-09T13:35:17.665 INFO:tasks.cephfs_test_runner:test_concurrent_snap_creates (tasks.cephfs.test_snap_schedules.TestSnapSchedules) 2023-08-09T13:35:17.665 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Starting test tasks.cephfs.test_snap_schedules.TestSnapSchedules.test_concurrent_snap_creates' 2023-08-09T13:35:17.896 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:17.894+0000 7f66dbfff640 1 Processor -- start 2023-08-09T13:35:17.896 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:17.895+0000 7f66dbfff640 1 -- start start 2023-08-09T13:35:17.898 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:17.896+0000 7f66dbfff640 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7f66dc0c8570 0x7f66dc070ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:35:17.898 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:17.896+0000 7f66dbfff640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f66dc0c9d70 0x7f66dc15b210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:35:17.898 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:17.896+0000 7f66dbfff640 1 -- --> v1:172.21.15.203:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f66dc154cd0 con 0x7f66dc0c8950 2023-08-09T13:35:17.898 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:17.896+0000 7f66dbfff640 1 -- --> v2:172.21.15.203:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f66dc15b750 con 0x7f66dc0c8570 2023-08-09T13:35:17.899 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:17.896+0000 7f66dbfff640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f66dc06f4f0 con 0x7f66dc0c9d70 2023-08-09T13:35:17.899 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:17.897+0000 7f66d3fff640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f66dc0c9d70 0x7f66dc15b210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:35:17.900 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:17.897+0000 7f66d3fff640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f66dc0c9d70 0x7f66dc15b210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.203:3301/0 says I am v2:172.21.15.43:51444/0 (socket says 172.21.15.43:51444) 2023-08-09T13:35:17.900 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:17.897+0000 7f66d3fff640 1 -- 172.21.15.43:0/3883826648 learned_addr learned my addr 172.21.15.43:0/3883826648 (peer_addr_for_me v2:172.21.15.43:0/0) 2023-08-09T13:35:17.900 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:17.897+0000 7f66daffd640 1 --2- 172.21.15.43:0/3883826648 >> v2:172.21.15.203:3300/0 conn(0x7f66dc0c8570 0x7f66dc070ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:35:17.900 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:17.898+0000 7f66d3fff640 1 -- 172.21.15.43:0/3883826648 >> v2:172.21.15.203:3300/0 conn(0x7f66dc0c8570 msgr2=0x7f66dc070ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-09T13:35:17.900 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:17.898+0000 7f66d3fff640 1 --2- 172.21.15.43:0/3883826648 >> v2:172.21.15.203:3300/0 conn(0x7f66dc0c8570 0x7f66dc070ff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:35:17.901 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:17.898+0000 7f66d3fff640 1 -- 172.21.15.43:0/3883826648 >> v1:172.21.15.203:6789/0 conn(0x7f66dc0c8950 legacy=0x7f66dc06b9e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:35:17.901 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:17.898+0000 7f66d3fff640 1 -- 172.21.15.43:0/3883826648 --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f66dc06f7d0 con 0x7f66dc0c9d70 2023-08-09T13:35:17.902 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:17.899+0000 7f66d3fff640 1 --2- 172.21.15.43:0/3883826648 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f66dc0c9d70 0x7f66dc15b210 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f66cc009870 tx=0x7f66cc02b980 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ad4b23b50b00b044 server_cookie=0 in_seq=0 out_seq=0 2023-08-09T13:35:17.902 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:17.900+0000 7f66da7fc640 1 -- 172.21.15.43:0/3883826648 <== mon.2 v2:172.21.15.203:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f66cc030840 con 0x7f66dc0c9d70 2023-08-09T13:35:17.902 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:17.900+0000 7f66da7fc640 1 -- 172.21.15.43:0/3883826648 <== mon.2 v2:172.21.15.203:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f66cc0309a0 con 0x7f66dc0c9d70 2023-08-09T13:35:17.902 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:17.900+0000 7f66da7fc640 1 -- 172.21.15.43:0/3883826648 <== mon.2 v2:172.21.15.203:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f66cc030d30 con 0x7f66dc0c9d70 2023-08-09T13:35:17.903 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:17.901+0000 7f66dbfff640 1 -- 172.21.15.43:0/3883826648 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f66dc0c9d70 msgr2=0x7f66dc15b210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:35:17.903 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:17.901+0000 7f66dbfff640 1 --2- 172.21.15.43:0/3883826648 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f66dc0c9d70 0x7f66dc15b210 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f66cc009870 tx=0x7f66cc02b980 comp rx=0 tx=0).stop 2023-08-09T13:35:17.903 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:17.901+0000 7f66dbfff640 1 -- 172.21.15.43:0/3883826648 shutdown_connections 2023-08-09T13:35:17.904 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:17.901+0000 7f66dbfff640 1 --2- 172.21.15.43:0/3883826648 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f66dc0c9d70 0x7f66dc15b210 unknown :-1 s=CLOSED pgs=60 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:35:17.904 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:17.901+0000 7f66dbfff640 1 --2- 172.21.15.43:0/3883826648 >> v2:172.21.15.203:3300/0 conn(0x7f66dc0c8570 0x7f66dc070ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:35:17.904 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:17.901+0000 7f66dbfff640 1 -- 172.21.15.43:0/3883826648 >> 172.21.15.43:0/3883826648 conn(0x7f66dc0afaf0 msgr2=0x7f66dc0afed0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-09T13:35:17.904 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:17.901+0000 7f66dbfff640 1 -- 172.21.15.43:0/3883826648 shutdown_connections 2023-08-09T13:35:17.904 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:17.901+0000 7f66dbfff640 1 -- 172.21.15.43:0/3883826648 wait complete. 2023-08-09T13:35:17.905 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:17.903+0000 7f66dbfff640 1 Processor -- start 2023-08-09T13:35:17.905 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:17.903+0000 7f66dbfff640 1 -- start start 2023-08-09T13:35:17.906 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:17.905+0000 7f66dbfff640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f66dc0c8570 0x7f66dc14c4a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:35:17.906 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:17.905+0000 7f66dbfff640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f66dc0c8950 0x7f66dc0c2dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:35:17.906 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:17.905+0000 7f66dbfff640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f66dc0c9d70 0x7f66dc1534b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:35:17.907 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:17.905+0000 7f66dbfff640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f66dc0c1490 con 0x7f66dc0c8950 2023-08-09T13:35:17.907 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:17.905+0000 7f66dbfff640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f66dc14c9e0 con 0x7f66dc0c8570 2023-08-09T13:35:17.907 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:17.905+0000 7f66dbfff640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f66dc0c1a50 con 0x7f66dc0c9d70 2023-08-09T13:35:17.908 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:17.905+0000 7f66d3fff640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f66dc0c8950 0x7f66dc0c2dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:35:17.908 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:17.905+0000 7f66d3fff640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f66dc0c8950 0x7f66dc0c2dd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:42476/0 (socket says 172.21.15.43:42476) 2023-08-09T13:35:17.908 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:17.905+0000 7f66d3fff640 1 -- 172.21.15.43:0/3102856047 learned_addr learned my addr 172.21.15.43:0/3102856047 (peer_addr_for_me v2:172.21.15.43:0/0) 2023-08-09T13:35:17.908 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:17.905+0000 7f66db7fe640 1 --2- 172.21.15.43:0/3102856047 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f66dc0c9d70 0x7f66dc1534b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:35:17.908 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:17.905+0000 7f66daffd640 1 --2- 172.21.15.43:0/3102856047 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f66dc0c8570 0x7f66dc14c4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:35:17.909 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:17.906+0000 7f66d3fff640 1 -- 172.21.15.43:0/3102856047 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f66dc0c9d70 msgr2=0x7f66dc1534b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:35:17.909 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:17.906+0000 7f66d3fff640 1 --2- 172.21.15.43:0/3102856047 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f66dc0c9d70 0x7f66dc1534b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:35:17.909 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:17.906+0000 7f66d3fff640 1 -- 172.21.15.43:0/3102856047 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f66dc0c8570 msgr2=0x7f66dc14c4a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:35:17.909 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:17.906+0000 7f66d3fff640 1 --2- 172.21.15.43:0/3102856047 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f66dc0c8570 0x7f66dc14c4a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:35:17.909 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:17.906+0000 7f66d3fff640 1 -- 172.21.15.43:0/3102856047 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f66cc0094d0 con 0x7f66dc0c8950 2023-08-09T13:35:17.909 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:17.906+0000 7f66d3fff640 1 --2- 172.21.15.43:0/3102856047 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f66dc0c8950 0x7f66dc0c2dd0 secure :-1 s=READY pgs=319 cs=0 l=1 rev1=1 crypto rx=0x7f66cc000c00 tx=0x7f66cc004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-09T13:35:17.909 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:17.906+0000 7f66d8ff9640 1 -- 172.21.15.43:0/3102856047 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f66cc039c90 con 0x7f66dc0c8950 2023-08-09T13:35:17.910 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:17.906+0000 7f66d8ff9640 1 -- 172.21.15.43:0/3102856047 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f66cc039df0 con 0x7f66dc0c8950 2023-08-09T13:35:17.910 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:17.906+0000 7f66d8ff9640 1 -- 172.21.15.43:0/3102856047 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f66cc043640 con 0x7f66dc0c8950 2023-08-09T13:35:17.910 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:17.906+0000 7f66dbfff640 1 -- 172.21.15.43:0/3102856047 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f66dc0c1d30 con 0x7f66dc0c8950 2023-08-09T13:35:17.910 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:17.907+0000 7f66dbfff640 1 -- 172.21.15.43:0/3102856047 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f66dc0c22f0 con 0x7f66dc0c8950 2023-08-09T13:35:17.910 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:17.908+0000 7f66dbfff640 1 -- 172.21.15.43:0/3102856047 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f66dc0739f0 con 0x7f66dc0c8950 2023-08-09T13:35:17.911 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:17.909+0000 7f66d8ff9640 1 -- 172.21.15.43:0/3102856047 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 5) v1 ==== 88341+0+0 (secure 0 0 0) 0x7f66cc0437a0 con 0x7f66dc0c8950 2023-08-09T13:35:17.911 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:17.910+0000 7f66d8ff9640 1 --2- 172.21.15.43:0/3102856047 >> [v2:172.21.15.203:6832/830097474,v1:172.21.15.203:6833/830097474] conn(0x7f66bc06a050 0x7f66bc06c510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:35:17.911 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:17.910+0000 7f66d8ff9640 1 -- 172.21.15.43:0/3102856047 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 6871+0+0 (secure 0 0 0) 0x7f66cc03e070 con 0x7f66dc0c8950 2023-08-09T13:35:17.912 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:17.911+0000 7f66daffd640 1 --2- 172.21.15.43:0/3102856047 >> [v2:172.21.15.203:6832/830097474,v1:172.21.15.203:6833/830097474] conn(0x7f66bc06a050 0x7f66bc06c510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:35:17.913 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:17.912+0000 7f66daffd640 1 --2- 172.21.15.43:0/3102856047 >> [v2:172.21.15.203:6832/830097474,v1:172.21.15.203:6833/830097474] conn(0x7f66bc06a050 0x7f66bc06c510 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7f66dc1f5320 tx=0x7f66c4008040 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-09T13:35:17.916 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:17.915+0000 7f66d8ff9640 1 -- 172.21.15.43:0/3102856047 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178066 (secure 0 0 0) 0x7f66cc076c90 con 0x7f66dc0c8950 2023-08-09T13:35:18.177 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:18.176+0000 7f66dbfff640 1 -- 172.21.15.43:0/3102856047 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_snap_schedules.TestSnapSchedules.test_concurrent_snap_creates"]} v 0) v1 -- 0x7f66dc0cec70 con 0x7f66dc0c8950 2023-08-09T13:35:18.793 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:18.791+0000 7f66d8ff9640 1 -- 172.21.15.43:0/3102856047 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_snap_schedules.TestSnapSchedules.test_concurrent_snap_creates"]}]=0 v85) v1 ==== 161+0+0 (secure 0 0 0) 0x7f66cc07a2b0 con 0x7f66dc0c8950 2023-08-09T13:35:18.796 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:18.795+0000 7f66dbfff640 1 -- 172.21.15.43:0/3102856047 >> [v2:172.21.15.203:6832/830097474,v1:172.21.15.203:6833/830097474] conn(0x7f66bc06a050 msgr2=0x7f66bc06c510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:35:18.797 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:18.795+0000 7f66dbfff640 1 --2- 172.21.15.43:0/3102856047 >> [v2:172.21.15.203:6832/830097474,v1:172.21.15.203:6833/830097474] conn(0x7f66bc06a050 0x7f66bc06c510 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7f66dc1f5320 tx=0x7f66c4008040 comp rx=0 tx=0).stop 2023-08-09T13:35:18.797 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:18.795+0000 7f66dbfff640 1 -- 172.21.15.43:0/3102856047 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f66dc0c8950 msgr2=0x7f66dc0c2dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:35:18.797 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:18.795+0000 7f66dbfff640 1 --2- 172.21.15.43:0/3102856047 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f66dc0c8950 0x7f66dc0c2dd0 secure :-1 s=READY pgs=319 cs=0 l=1 rev1=1 crypto rx=0x7f66cc000c00 tx=0x7f66cc004730 comp rx=0 tx=0).stop 2023-08-09T13:35:18.797 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:18.795+0000 7f66dbfff640 1 -- 172.21.15.43:0/3102856047 shutdown_connections 2023-08-09T13:35:18.797 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:18.795+0000 7f66dbfff640 1 --2- 172.21.15.43:0/3102856047 >> [v2:172.21.15.203:6832/830097474,v1:172.21.15.203:6833/830097474] conn(0x7f66bc06a050 0x7f66bc06c510 unknown :-1 s=CLOSED pgs=170 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:35:18.797 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:18.795+0000 7f66dbfff640 1 --2- 172.21.15.43:0/3102856047 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f66dc0c9d70 0x7f66dc1534b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:35:18.798 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:18.795+0000 7f66dbfff640 1 --2- 172.21.15.43:0/3102856047 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f66dc0c8950 0x7f66dc0c2dd0 unknown :-1 s=CLOSED pgs=319 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:35:18.798 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:18.795+0000 7f66dbfff640 1 --2- 172.21.15.43:0/3102856047 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f66dc0c8570 0x7f66dc14c4a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:35:18.798 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:18.795+0000 7f66dbfff640 1 -- 172.21.15.43:0/3102856047 >> 172.21.15.43:0/3102856047 conn(0x7f66dc0afaf0 msgr2=0x7f66dc15c2c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-09T13:35:18.798 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:18.795+0000 7f66dbfff640 1 -- 172.21.15.43:0/3102856047 shutdown_connections 2023-08-09T13:35:18.798 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:18.795+0000 7f66dbfff640 1 -- 172.21.15.43:0/3102856047 wait complete. 2023-08-09T13:35:18.820 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-08-09T13:35:19.056 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:19.055+0000 7f1213a1f640 1 Processor -- start 2023-08-09T13:35:19.057 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:19.055+0000 7f1213a1f640 1 -- start start 2023-08-09T13:35:19.058 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:19.056+0000 7f1213a1f640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f120c0b2790 0x7f120c0d2ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:35:19.058 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:19.057+0000 7f1213a1f640 1 -- --> v1:172.21.15.43:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f120c0ce090 con 0x7f120c0b3050 2023-08-09T13:35:19.059 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:19.057+0000 7f1213a1f640 1 -- --> v1:172.21.15.203:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f120c0cd7f0 con 0x7f120c0b2b70 2023-08-09T13:35:19.059 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:19.057+0000 7f1213a1f640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f120c0b3950 con 0x7f120c0b2790 2023-08-09T13:35:19.060 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:19.057+0000 7f1211fbc640 1 --1- >> v1:172.21.15.43:6789/0 conn(0x7f120c0b3050 0x7f120c0b3f60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.43:6789/0 says I am v1:172.21.15.43:49146/0 (socket says 172.21.15.43:49146) 2023-08-09T13:35:19.060 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:19.057+0000 7f1211fbc640 1 -- 172.21.15.43:0/2787953193 learned_addr learned my addr 172.21.15.43:0/2787953193 (peer_addr_for_me v1:172.21.15.43:0/0) 2023-08-09T13:35:19.060 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:19.057+0000 7f12117bb640 1 --2- 172.21.15.43:0/2787953193 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f120c0b2790 0x7f120c0d2ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:35:19.060 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:19.058+0000 7f1203fff640 1 -- 172.21.15.43:0/2787953193 <== mon.0 v1:172.21.15.43:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 227426 0 0) 0x7f120c0ce090 con 0x7f120c0b3050 2023-08-09T13:35:19.060 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:19.058+0000 7f1203fff640 1 -- 172.21.15.43:0/2787953193 --> v1:172.21.15.43:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f11f8003520 con 0x7f120c0b3050 2023-08-09T13:35:19.060 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:19.058+0000 7f1203fff640 1 -- 172.21.15.43:0/2787953193 <== mon.1 v1:172.21.15.203:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2997427756 0 0) 0x7f120c0cd7f0 con 0x7f120c0b2b70 2023-08-09T13:35:19.061 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:19.058+0000 7f12117bb640 1 -- 172.21.15.43:0/2787953193 >> v1:172.21.15.203:6789/0 conn(0x7f120c0b2b70 legacy=0x7f120c0b5330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:35:19.061 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:19.058+0000 7f12117bb640 1 -- 172.21.15.43:0/2787953193 >> v1:172.21.15.43:6789/0 conn(0x7f120c0b3050 legacy=0x7f120c0b3f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:35:19.061 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:19.058+0000 7f12117bb640 1 -- 172.21.15.43:0/2787953193 --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f120c0cdad0 con 0x7f120c0b2790 2023-08-09T13:35:19.062 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:19.060+0000 7f12117bb640 1 --2- 172.21.15.43:0/2787953193 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f120c0b2790 0x7f120c0d2ed0 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f12080097d0 tx=0x7f120802b730 comp rx=0 tx=0).ready entity=mon.2 client_cookie=27ea2473da565020 server_cookie=0 in_seq=0 out_seq=0 2023-08-09T13:35:19.062 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:19.060+0000 7f1203fff640 1 -- 172.21.15.43:0/2787953193 <== mon.2 v2:172.21.15.203:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f12040019f0 con 0x7f120c0b2790 2023-08-09T13:35:19.062 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:19.060+0000 7f1203fff640 1 -- 172.21.15.43:0/2787953193 <== mon.2 v2:172.21.15.203:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f120802e050 con 0x7f120c0b2790 2023-08-09T13:35:19.062 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:19.060+0000 7f1203fff640 1 -- 172.21.15.43:0/2787953193 <== mon.2 v2:172.21.15.203:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1208030730 con 0x7f120c0b2790 2023-08-09T13:35:19.063 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:19.061+0000 7f1213a1f640 1 -- 172.21.15.43:0/2787953193 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f120c0b2790 msgr2=0x7f120c0d2ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:35:19.063 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:19.061+0000 7f1213a1f640 1 --2- 172.21.15.43:0/2787953193 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f120c0b2790 0x7f120c0d2ed0 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f12080097d0 tx=0x7f120802b730 comp rx=0 tx=0).stop 2023-08-09T13:35:19.063 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:19.061+0000 7f1213a1f640 1 -- 172.21.15.43:0/2787953193 shutdown_connections 2023-08-09T13:35:19.063 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:19.061+0000 7f1213a1f640 1 --2- 172.21.15.43:0/2787953193 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f120c0b2790 0x7f120c0d2ed0 unknown :-1 s=CLOSED pgs=61 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:35:19.063 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:19.061+0000 7f1213a1f640 1 -- 172.21.15.43:0/2787953193 >> 172.21.15.43:0/2787953193 conn(0x7f120c09b7b0 msgr2=0x7f120c0d24e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-09T13:35:19.064 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:19.061+0000 7f1213a1f640 1 -- 172.21.15.43:0/2787953193 shutdown_connections 2023-08-09T13:35:19.064 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:19.061+0000 7f1213a1f640 1 -- 172.21.15.43:0/2787953193 wait complete. 2023-08-09T13:35:19.064 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:19.063+0000 7f1213a1f640 1 Processor -- start 2023-08-09T13:35:19.064 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:19.063+0000 7f1213a1f640 1 -- start start 2023-08-09T13:35:19.065 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:19.064+0000 7f1213a1f640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f120c0b2790 0x7f120c0b00f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:35:19.066 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:19.064+0000 7f1213a1f640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f120c0b2b70 0x7f120c154320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:35:19.066 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:19.064+0000 7f1213a1f640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f120c0b3050 0x7f120c0cbf60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:35:19.066 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:19.064+0000 7f1213a1f640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f120c0b1020 con 0x7f120c0b2790 2023-08-09T13:35:19.067 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:19.064+0000 7f1213a1f640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f120c154860 con 0x7f120c0b2b70 2023-08-09T13:35:19.067 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:19.064+0000 7f1213a1f640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f120c0b0780 con 0x7f120c0b3050 2023-08-09T13:35:19.067 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:19.064+0000 7f12117bb640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f120c0b2790 0x7f120c0b00f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:35:19.068 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:19.064+0000 7f12117bb640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f120c0b2790 0x7f120c0b00f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:42482/0 (socket says 172.21.15.43:42482) 2023-08-09T13:35:19.068 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:19.064+0000 7f12117bb640 1 -- 172.21.15.43:0/2946775773 learned_addr learned my addr 172.21.15.43:0/2946775773 (peer_addr_for_me v2:172.21.15.43:0/0) 2023-08-09T13:35:19.071 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:19.065+0000 7f12117bb640 1 -- 172.21.15.43:0/2946775773 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f120c0b3050 msgr2=0x7f120c0cbf60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:35:19.071 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:19.065+0000 7f12117bb640 1 --2- 172.21.15.43:0/2946775773 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f120c0b3050 0x7f120c0cbf60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:35:19.072 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:19.065+0000 7f12117bb640 1 -- 172.21.15.43:0/2946775773 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f120c0b2b70 msgr2=0x7f120c154320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:35:19.072 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:19.065+0000 7f1210fba640 1 --2- 172.21.15.43:0/2946775773 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f120c0b2b70 0x7f120c154320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:35:19.072 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:19.065+0000 7f12117bb640 1 --2- 172.21.15.43:0/2946775773 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f120c0b2b70 0x7f120c154320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:35:19.073 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:19.065+0000 7f12117bb640 1 -- 172.21.15.43:0/2946775773 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1208009400 con 0x7f120c0b2790 2023-08-09T13:35:19.073 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:19.065+0000 7f1210fba640 1 --2- 172.21.15.43:0/2946775773 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f120c0b2b70 0x7f120c154320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-08-09T13:35:19.073 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:19.065+0000 7f12117bb640 1 --2- 172.21.15.43:0/2946775773 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f120c0b2790 0x7f120c0b00f0 secure :-1 s=READY pgs=321 cs=0 l=1 rev1=1 crypto rx=0x7f120802bd40 tx=0x7f12080046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-09T13:35:19.073 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:19.065+0000 7f1201ffb640 1 -- 172.21.15.43:0/2946775773 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f120803c8e0 con 0x7f120c0b2790 2023-08-09T13:35:19.074 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:19.065+0000 7f1201ffb640 1 -- 172.21.15.43:0/2946775773 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f120803ca60 con 0x7f120c0b2790 2023-08-09T13:35:19.074 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:19.065+0000 7f1201ffb640 1 -- 172.21.15.43:0/2946775773 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f120803ce10 con 0x7f120c0b2790 2023-08-09T13:35:19.074 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:19.065+0000 7f1213a1f640 1 -- 172.21.15.43:0/2946775773 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f120c0b0a60 con 0x7f120c0b2790 2023-08-09T13:35:19.074 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:19.066+0000 7f1213a1f640 1 -- 172.21.15.43:0/2946775773 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f120c0c61b0 con 0x7f120c0b2790 2023-08-09T13:35:19.075 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:19.067+0000 7f1213a1f640 1 -- 172.21.15.43:0/2946775773 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f120c0c5fb0 con 0x7f120c0b2790 2023-08-09T13:35:19.075 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:19.068+0000 7f1201ffb640 1 -- 172.21.15.43:0/2946775773 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 5) v1 ==== 88341+0+0 (secure 0 0 0) 0x7f1208039070 con 0x7f120c0b2790 2023-08-09T13:35:19.075 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:19.069+0000 7f1201ffb640 1 --2- 172.21.15.43:0/2946775773 >> [v2:172.21.15.203:6832/830097474,v1:172.21.15.203:6833/830097474] conn(0x7f11f006a050 0x7f11f006c510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:35:19.075 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:19.069+0000 7f1201ffb640 1 -- 172.21.15.43:0/2946775773 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 6871+0+0 (secure 0 0 0) 0x7f12080aa390 con 0x7f120c0b2790 2023-08-09T13:35:19.076 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:19.070+0000 7f1210fba640 1 --2- 172.21.15.43:0/2946775773 >> [v2:172.21.15.203:6832/830097474,v1:172.21.15.203:6833/830097474] conn(0x7f11f006a050 0x7f11f006c510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:35:19.077 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:19.071+0000 7f1210fba640 1 --2- 172.21.15.43:0/2946775773 >> [v2:172.21.15.203:6832/830097474,v1:172.21.15.203:6833/830097474] conn(0x7f11f006a050 0x7f11f006c510 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7f120c1f53f0 tx=0x7f11fc009040 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-09T13:35:19.077 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:19.074+0000 7f1201ffb640 1 -- 172.21.15.43:0/2946775773 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178066 (secure 0 0 0) 0x7f12080756b0 con 0x7f120c0b2790 2023-08-09T13:35:19.278 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:19.277+0000 7f1213a1f640 1 -- 172.21.15.43:0/2946775773 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f120c0739f0 con 0x7f120c0b2790 2023-08-09T13:35:19.282 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:19.280+0000 7f1201ffb640 1 -- 172.21.15.43:0/2946775773 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v26) v1 ==== 74+0+20146 (secure 0 0 0) 0x7f1208078cd0 con 0x7f120c0b2790 2023-08-09T13:35:19.283 INFO:teuthology.orchestra.run.smithi043.stdout: 2023-08-09T13:35:19.283 INFO:teuthology.orchestra.run.smithi043.stdout:{"epoch":26,"fsid":"daac87ee-1337-4250-a955-aea4e408a39b","created":"2023-08-09T13:33:41.701198+0000","modified":"2023-08-09T13:34:48.711911+0000","last_up_change":"2023-08-09T13:33:53.783453+0000","last_in_change":"2023-08-09T13:33:45.148904+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":5,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":"rbd","create_time":"2023-08-09T13:33:54.365137+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":".mgr","create_time":"2023-08-09T13:33:54.831803+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_metadata","create_time":"2023-08-09T13:34:00.207974+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data","create_time":"2023-08-09T13:34:01.614765+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":5,"pool_name":"cephfs_data_ec","create_time":"2023-08-09T13:34:05.239583+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"24","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"cbd49ecd-e896-4fbd-864e-5c37a829b165","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6800","nonce":502667335},{"type":"v1","addr":"172.21.15.43:6801","nonce":502667335}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6802","nonce":502667335},{"type":"v1","addr":"172.21.15.43:6803","nonce":502667335}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6806","nonce":502667335},{"type":"v1","addr":"172.21.15.43:6807","nonce":502667335}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6804","nonce":502667335},{"type":"v1","addr":"172.21.15.43:6805","nonce":502667335}]},"public_addr":"172.21.15.43:6801/502667335","cluster_addr":"172.21.15.43:6803/502667335","heartbeat_back_addr":"172.21.15.43:6807/502667335","heartbeat_front_addr":"172.21.15.43:6805/502667335","state":["exists","up"]},{"osd":1,"uuid":"75c7e0fe-1ec7-4fe8-a2f2-3661a65b8500","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6808","nonce":3289134610},{"type":"v1","addr":"172.21.15.43:6809","nonce":3289134610}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6810","nonce":3289134610},{"type":"v1","addr":"172.21.15.43:6812","nonce":3289134610}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6821","nonce":3289134610},{"type":"v1","addr":"172.21.15.43:6824","nonce":3289134610}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6815","nonce":3289134610},{"type":"v1","addr":"172.21.15.43:6818","nonce":3289134610}]},"public_addr":"172.21.15.43:6809/3289134610","cluster_addr":"172.21.15.43:6812/3289134610","heartbeat_back_addr":"172.21.15.43:6824/3289134610","heartbeat_front_addr":"172.21.15.43:6818/3289134610","state":["exists","up"]},{"osd":2,"uuid":"bb5c41e9-7f5f-4125-9b01-3bb8bfc2ea83","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6811","nonce":4079709736},{"type":"v1","addr":"172.21.15.43:6814","nonce":4079709736}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6817","nonce":4079709736},{"type":"v1","addr":"172.21.15.43:6820","nonce":4079709736}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6829","nonce":4079709736},{"type":"v1","addr":"172.21.15.43:6831","nonce":4079709736}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6823","nonce":4079709736},{"type":"v1","addr":"172.21.15.43:6826","nonce":4079709736}]},"public_addr":"172.21.15.43:6814/4079709736","cluster_addr":"172.21.15.43:6820/4079709736","heartbeat_back_addr":"172.21.15.43:6831/4079709736","heartbeat_front_addr":"172.21.15.43:6826/4079709736","state":["exists","up"]},{"osd":3,"uuid":"bfe4c8b4-694a-465c-87e6-2b2aab852613","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6813","nonce":1709538759},{"type":"v1","addr":"172.21.15.43:6816","nonce":1709538759}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6819","nonce":1709538759},{"type":"v1","addr":"172.21.15.43:6822","nonce":1709538759}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6828","nonce":1709538759},{"type":"v1","addr":"172.21.15.43:6830","nonce":1709538759}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6825","nonce":1709538759},{"type":"v1","addr":"172.21.15.43:6827","nonce":1709538759}]},"public_addr":"172.21.15.43:6816/1709538759","cluster_addr":"172.21.15.43:6822/1709538759","heartbeat_back_addr":"172.21.15.43:6830/1709538759","heartbeat_front_addr":"172.21.15.43:6827/1709538759","state":["exists","up"]},{"osd":4,"uuid":"fa2e8a77-aba1-4756-853e-0d174a5bdeba","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6800","nonce":4196866481},{"type":"v1","addr":"172.21.15.203:6801","nonce":4196866481}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6802","nonce":4196866481},{"type":"v1","addr":"172.21.15.203:6803","nonce":4196866481}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6806","nonce":4196866481},{"type":"v1","addr":"172.21.15.203:6807","nonce":4196866481}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6804","nonce":4196866481},{"type":"v1","addr":"172.21.15.203:6805","nonce":4196866481}]},"public_addr":"172.21.15.203:6801/4196866481","cluster_addr":"172.21.15.203:6803/4196866481","heartbeat_back_addr":"172.21.15.203:6807/4196866481","heartbeat_front_addr":"172.21.15.203:6805/4196866481","state":["exists","up"]},{"osd":5,"uuid":"81811e4b-4d24-4f51-a82c-f1d700a2a00d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6816","nonce":3059290081},{"type":"v1","addr":"172.21.15.203:6817","nonce":3059290081}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6818","nonce":3059290081},{"type":"v1","addr":"172.21.15.203:6819","nonce":3059290081}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6822","nonce":3059290081},{"type":"v1","addr":"172.21.15.203:6823","nonce":3059290081}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6820","nonce":3059290081},{"type":"v1","addr":"172.21.15.203:6821","nonce":3059290081}]},"public_addr":"172.21.15.203:6817/3059290081","cluster_addr":"172.21.15.203:6819/3059290081","heartbeat_back_addr":"172.21.15.203:6823/3059290081","heartbeat_front_addr":"172.21.15.203:6821/3059290081","state":["exists","up"]},{"osd":6,"uuid":"307bac4d-6a5b-443d-af75-56d695d7674f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6824","nonce":512155569},{"type":"v1","addr":"172.21.15.203:6825","nonce":512155569}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6826","nonce":512155569},{"type":"v1","addr":"172.21.15.203:6827","nonce":512155569}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6830","nonce":512155569},{"type":"v1","addr":"172.21.15.203:6831","nonce":512155569}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6828","nonce":512155569},{"type":"v1","addr":"172.21.15.203:6829","nonce":512155569}]},"public_addr":"172.21.15.203:6825/512155569","cluster_addr":"172.21.15.203:6827/512155569","heartbeat_back_addr":"172.21.15.203:6831/512155569","heartbeat_front_addr":"172.21.15.203:6829/512155569","state":["exists","up"]},{"osd":7,"uuid":"a95dc7ca-9f1b-4295-af70-68a22a2d7d5e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6808","nonce":3331450438},{"type":"v1","addr":"172.21.15.203:6809","nonce":3331450438}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6810","nonce":3331450438},{"type":"v1","addr":"172.21.15.203:6811","nonce":3331450438}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6814","nonce":3331450438},{"type":"v1","addr":"172.21.15.203:6815","nonce":3331450438}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6812","nonce":3331450438},{"type":"v1","addr":"172.21.15.203:6813","nonce":3331450438}]},"public_addr":"172.21.15.203:6809/3331450438","cluster_addr":"172.21.15.203:6811/3331450438","heartbeat_back_addr":"172.21.15.203:6815/3331450438","heartbeat_front_addr":"172.21.15.203:6813/3331450438","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-09T13:33:50.547652+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-09T13:33:50.589469+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-09T13:33:50.561724+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-09T13:33:50.599650+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-09T13:33:50.411855+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-09T13:33:50.219681+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-09T13:33:50.504664+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-09T13:33:50.441248+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"4.14","mappings":[{"from":1,"to":5}]},{"pgid":"4.29","mappings":[{"from":3,"to":2}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-08-09T13:35:19.285 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:19.283+0000 7f1213a1f640 1 -- 172.21.15.43:0/2946775773 >> [v2:172.21.15.203:6832/830097474,v1:172.21.15.203:6833/830097474] conn(0x7f11f006a050 msgr2=0x7f11f006c510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:35:19.285 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:19.283+0000 7f1213a1f640 1 --2- 172.21.15.43:0/2946775773 >> [v2:172.21.15.203:6832/830097474,v1:172.21.15.203:6833/830097474] conn(0x7f11f006a050 0x7f11f006c510 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7f120c1f53f0 tx=0x7f11fc009040 comp rx=0 tx=0).stop 2023-08-09T13:35:19.285 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:19.283+0000 7f1213a1f640 1 -- 172.21.15.43:0/2946775773 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f120c0b2790 msgr2=0x7f120c0b00f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:35:19.286 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:19.283+0000 7f1213a1f640 1 --2- 172.21.15.43:0/2946775773 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f120c0b2790 0x7f120c0b00f0 secure :-1 s=READY pgs=321 cs=0 l=1 rev1=1 crypto rx=0x7f120802bd40 tx=0x7f12080046c0 comp rx=0 tx=0).stop 2023-08-09T13:35:19.286 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:19.283+0000 7f1213a1f640 1 -- 172.21.15.43:0/2946775773 shutdown_connections 2023-08-09T13:35:19.286 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:19.283+0000 7f1213a1f640 1 --2- 172.21.15.43:0/2946775773 >> [v2:172.21.15.203:6832/830097474,v1:172.21.15.203:6833/830097474] conn(0x7f11f006a050 0x7f11f006c510 unknown :-1 s=CLOSED pgs=171 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:35:19.287 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:19.283+0000 7f1213a1f640 1 --2- 172.21.15.43:0/2946775773 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f120c0b3050 0x7f120c0cbf60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:35:19.287 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:19.283+0000 7f1213a1f640 1 --2- 172.21.15.43:0/2946775773 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f120c0b2b70 0x7f120c154320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:35:19.287 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:19.283+0000 7f1213a1f640 1 --2- 172.21.15.43:0/2946775773 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f120c0b2790 0x7f120c0b00f0 unknown :-1 s=CLOSED pgs=321 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:35:19.288 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:19.283+0000 7f1213a1f640 1 -- 172.21.15.43:0/2946775773 >> 172.21.15.43:0/2946775773 conn(0x7f120c09b7b0 msgr2=0x7f120c0b7860 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-09T13:35:19.288 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:19.283+0000 7f1213a1f640 1 -- 172.21.15.43:0/2946775773 shutdown_connections 2023-08-09T13:35:19.288 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:19.283+0000 7f1213a1f640 1 -- 172.21.15.43:0/2946775773 wait complete. 2023-08-09T13:35:19.304 INFO:tasks.cephfs.cephfs_test_case.ceph_manager:[{'pool': 1, 'pool_name': 'rbd', 'create_time': '2023-08-09T13:33:54.365137+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}, 'read_balance': {'score_acting': 2, 'score_stable': 2, 'optimal_score': 1, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': '.mgr', 'create_time': '2023-08-09T13:33:54.831803+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_metadata', 'create_time': '2023-08-09T13:34:00.207974+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data', 'create_time': '2023-08-09T13:34:01.614765+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 2, 'score_stable': 2, 'optimal_score': 1, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 5, 'pool_name': 'cephfs_data_ec', 'create_time': '2023-08-09T13:34:05.239583+0000', 'flags': 5, 'flags_names': 'hashpspool,ec_overwrites', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '24', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2023-08-09T13:35:19.305 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get rbd pg_num 2023-08-09T13:35:19.539 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:19.538+0000 7fa02708c640 1 Processor -- start 2023-08-09T13:35:19.540 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:19.538+0000 7fa02708c640 1 -- start start 2023-08-09T13:35:19.541 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:19.540+0000 7fa02708c640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fa0200b45f0 0x7fa0200cd6d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:35:19.541 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:19.540+0000 7fa02708c640 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7fa0200d04b0 0x7fa0200b59d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:35:19.541 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:19.540+0000 7fa02708c640 1 -- --> v1:172.21.15.203:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa0200b7bd0 con 0x7fa02015c550 2023-08-09T13:35:19.542 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:19.540+0000 7fa02708c640 1 -- --> v2:172.21.15.203:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa0200d1c00 con 0x7fa0200d04b0 2023-08-09T13:35:19.542 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:19.540+0000 7fa02708c640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa02015c930 con 0x7fa0200b45f0 2023-08-09T13:35:19.543 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:19.540+0000 7fa01ffff640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fa0200b45f0 0x7fa0200cd6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:35:19.543 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:19.540+0000 7fa025629640 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7fa0200d04b0 0x7fa0200b59d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:35:19.543 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:19.540+0000 7fa01ffff640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fa0200b45f0 0x7fa0200cd6d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.203:3301/0 says I am v2:172.21.15.43:51478/0 (socket says 172.21.15.43:51478) 2023-08-09T13:35:19.543 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:19.541+0000 7fa01ffff640 1 -- 172.21.15.43:0/538929494 learned_addr learned my addr 172.21.15.43:0/538929494 (peer_addr_for_me v2:172.21.15.43:0/0) 2023-08-09T13:35:19.543 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:19.541+0000 7fa01ffff640 1 -- 172.21.15.43:0/538929494 >> v2:172.21.15.203:3300/0 conn(0x7fa0200d04b0 msgr2=0x7fa0200b59d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-09T13:35:19.543 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:19.541+0000 7fa01ffff640 1 --2- 172.21.15.43:0/538929494 >> v2:172.21.15.203:3300/0 conn(0x7fa0200d04b0 0x7fa0200b59d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:35:19.544 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:19.541+0000 7fa01ffff640 1 -- 172.21.15.43:0/538929494 >> v1:172.21.15.203:6789/0 conn(0x7fa02015c550 legacy=0x7fa0200b3ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:35:19.544 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:19.541+0000 7fa01ffff640 1 -- 172.21.15.43:0/538929494 --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa0200b54f0 con 0x7fa0200b45f0 2023-08-09T13:35:19.544 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:19.543+0000 7fa01ffff640 1 --2- 172.21.15.43:0/538929494 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fa0200b45f0 0x7fa0200cd6d0 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7fa014009870 tx=0x7fa01402b980 comp rx=0 tx=0).ready entity=mon.2 client_cookie=cfee0305665c15dd server_cookie=0 in_seq=0 out_seq=0 2023-08-09T13:35:19.545 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:19.543+0000 7fa01f7fe640 1 -- 172.21.15.43:0/538929494 <== mon.2 v2:172.21.15.203:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa014030840 con 0x7fa0200b45f0 2023-08-09T13:35:19.545 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:19.543+0000 7fa01f7fe640 1 -- 172.21.15.43:0/538929494 <== mon.2 v2:172.21.15.203:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa0140309a0 con 0x7fa0200b45f0 2023-08-09T13:35:19.545 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:19.543+0000 7fa01f7fe640 1 -- 172.21.15.43:0/538929494 <== mon.2 v2:172.21.15.203:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa014030d30 con 0x7fa0200b45f0 2023-08-09T13:35:19.546 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:19.544+0000 7fa02708c640 1 -- 172.21.15.43:0/538929494 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fa0200b45f0 msgr2=0x7fa0200cd6d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:35:19.546 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:19.544+0000 7fa02708c640 1 --2- 172.21.15.43:0/538929494 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fa0200b45f0 0x7fa0200cd6d0 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7fa014009870 tx=0x7fa01402b980 comp rx=0 tx=0).stop 2023-08-09T13:35:19.546 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:19.544+0000 7fa02708c640 1 -- 172.21.15.43:0/538929494 shutdown_connections 2023-08-09T13:35:19.546 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:19.544+0000 7fa02708c640 1 --2- 172.21.15.43:0/538929494 >> v2:172.21.15.203:3300/0 conn(0x7fa0200d04b0 0x7fa0200b59d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:35:19.547 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:19.544+0000 7fa02708c640 1 --2- 172.21.15.43:0/538929494 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fa0200b45f0 0x7fa0200cd6d0 unknown :-1 s=CLOSED pgs=62 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:35:19.547 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:19.544+0000 7fa02708c640 1 -- 172.21.15.43:0/538929494 >> 172.21.15.43:0/538929494 conn(0x7fa020071520 msgr2=0x7fa0200ceee0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-09T13:35:19.547 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:19.544+0000 7fa02708c640 1 -- 172.21.15.43:0/538929494 shutdown_connections 2023-08-09T13:35:19.547 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:19.544+0000 7fa02708c640 1 -- 172.21.15.43:0/538929494 wait complete. 2023-08-09T13:35:19.548 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:19.546+0000 7fa02708c640 1 Processor -- start 2023-08-09T13:35:19.548 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:19.547+0000 7fa02708c640 1 -- start start 2023-08-09T13:35:19.549 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:19.548+0000 7fa02708c640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fa0200b45f0 0x7fa0201e5db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:35:19.550 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:19.548+0000 7fa02708c640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fa0200d04b0 0x7fa0201f4ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:35:19.550 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:19.548+0000 7fa02708c640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa02015c550 0x7fa0201f5410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:35:19.550 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:19.548+0000 7fa02708c640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa0200ca940 con 0x7fa02015c550 2023-08-09T13:35:19.550 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:19.548+0000 7fa02708c640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa0201e4d50 con 0x7fa0200d04b0 2023-08-09T13:35:19.550 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:19.548+0000 7fa02708c640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa0201e4ff0 con 0x7fa0200b45f0 2023-08-09T13:35:19.551 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:19.548+0000 7fa025629640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa02015c550 0x7fa0201f5410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:35:19.552 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:19.549+0000 7fa025629640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa02015c550 0x7fa0201f5410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:42486/0 (socket says 172.21.15.43:42486) 2023-08-09T13:35:19.552 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:19.549+0000 7fa025629640 1 -- 172.21.15.43:0/3050810392 learned_addr learned my addr 172.21.15.43:0/3050810392 (peer_addr_for_me v2:172.21.15.43:0/0) 2023-08-09T13:35:19.552 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:19.549+0000 7fa024e28640 1 --2- 172.21.15.43:0/3050810392 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fa0200b45f0 0x7fa0201e5db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:35:19.552 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:19.549+0000 7fa01ffff640 1 --2- 172.21.15.43:0/3050810392 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fa0200d04b0 0x7fa0201f4ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:35:19.552 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:19.549+0000 7fa025629640 1 -- 172.21.15.43:0/3050810392 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fa0200b45f0 msgr2=0x7fa0201e5db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:35:19.552 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:19.549+0000 7fa025629640 1 --2- 172.21.15.43:0/3050810392 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fa0200b45f0 0x7fa0201e5db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:35:19.552 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:19.549+0000 7fa025629640 1 -- 172.21.15.43:0/3050810392 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fa0200d04b0 msgr2=0x7fa0201f4ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:35:19.553 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:19.549+0000 7fa025629640 1 --2- 172.21.15.43:0/3050810392 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fa0200d04b0 0x7fa0201f4ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:35:19.553 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:19.549+0000 7fa025629640 1 -- 172.21.15.43:0/3050810392 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa0140094d0 con 0x7fa02015c550 2023-08-09T13:35:19.553 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:19.549+0000 7fa025629640 1 --2- 172.21.15.43:0/3050810392 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa02015c550 0x7fa0201f5410 secure :-1 s=READY pgs=322 cs=0 l=1 rev1=1 crypto rx=0x7fa01800e930 tx=0x7fa01800edf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-09T13:35:19.553 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:19.550+0000 7fa01d7fa640 1 -- 172.21.15.43:0/3050810392 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa018009690 con 0x7fa02015c550 2023-08-09T13:35:19.554 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:19.550+0000 7fa01d7fa640 1 -- 172.21.15.43:0/3050810392 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa01800c980 con 0x7fa02015c550 2023-08-09T13:35:19.554 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:19.550+0000 7fa01d7fa640 1 -- 172.21.15.43:0/3050810392 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa01800cd30 con 0x7fa02015c550 2023-08-09T13:35:19.554 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:19.550+0000 7fa02708c640 1 -- 172.21.15.43:0/3050810392 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa0201e52d0 con 0x7fa02015c550 2023-08-09T13:35:19.555 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:19.550+0000 7fa02708c640 1 -- 172.21.15.43:0/3050810392 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa0201e5890 con 0x7fa02015c550 2023-08-09T13:35:19.556 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:19.551+0000 7fa02708c640 1 -- 172.21.15.43:0/3050810392 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa0200739f0 con 0x7fa02015c550 2023-08-09T13:35:19.556 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:19.552+0000 7fa01d7fa640 1 -- 172.21.15.43:0/3050810392 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 5) v1 ==== 88341+0+0 (secure 0 0 0) 0x7fa018013470 con 0x7fa02015c550 2023-08-09T13:35:19.556 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:19.553+0000 7fa01d7fa640 1 --2- 172.21.15.43:0/3050810392 >> [v2:172.21.15.203:6832/830097474,v1:172.21.15.203:6833/830097474] conn(0x7fa00c06a050 0x7fa00c06c510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:35:19.557 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:19.553+0000 7fa01d7fa640 1 -- 172.21.15.43:0/3050810392 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 6871+0+0 (secure 0 0 0) 0x7fa018088ae0 con 0x7fa02015c550 2023-08-09T13:35:19.557 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:19.554+0000 7fa024e28640 1 --2- 172.21.15.43:0/3050810392 >> [v2:172.21.15.203:6832/830097474,v1:172.21.15.203:6833/830097474] conn(0x7fa00c06a050 0x7fa00c06c510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:35:19.558 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:19.555+0000 7fa024e28640 1 --2- 172.21.15.43:0/3050810392 >> [v2:172.21.15.203:6832/830097474,v1:172.21.15.203:6833/830097474] conn(0x7fa00c06a050 0x7fa00c06c510 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7fa010001e70 tx=0x7fa010009040 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-09T13:35:19.559 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:19.557+0000 7fa01d7fa640 1 -- 172.21.15.43:0/3050810392 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178066 (secure 0 0 0) 0x7fa018057a00 con 0x7fa02015c550 2023-08-09T13:35:19.752 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:19.750+0000 7fa02708c640 1 -- 172.21.15.43:0/3050810392 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"} v 0) v1 -- 0x7fa0201e5a80 con 0x7fa02015c550 2023-08-09T13:35:19.753 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:19.751+0000 7fa01d7fa640 1 -- 172.21.15.43:0/3050810392 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"}]=0 v26) v1 ==== 92+0+10 (secure 0 0 0) 0x7fa018057590 con 0x7fa02015c550 2023-08-09T13:35:19.753 INFO:teuthology.orchestra.run.smithi043.stdout:pg_num: 8 2023-08-09T13:35:19.756 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:19.754+0000 7fa02708c640 1 -- 172.21.15.43:0/3050810392 >> [v2:172.21.15.203:6832/830097474,v1:172.21.15.203:6833/830097474] conn(0x7fa00c06a050 msgr2=0x7fa00c06c510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:35:19.756 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:19.754+0000 7fa02708c640 1 --2- 172.21.15.43:0/3050810392 >> [v2:172.21.15.203:6832/830097474,v1:172.21.15.203:6833/830097474] conn(0x7fa00c06a050 0x7fa00c06c510 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7fa010001e70 tx=0x7fa010009040 comp rx=0 tx=0).stop 2023-08-09T13:35:19.756 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:19.754+0000 7fa02708c640 1 -- 172.21.15.43:0/3050810392 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa02015c550 msgr2=0x7fa0201f5410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:35:19.756 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:19.754+0000 7fa02708c640 1 --2- 172.21.15.43:0/3050810392 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa02015c550 0x7fa0201f5410 secure :-1 s=READY pgs=322 cs=0 l=1 rev1=1 crypto rx=0x7fa01800e930 tx=0x7fa01800edf0 comp rx=0 tx=0).stop 2023-08-09T13:35:19.756 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:19.754+0000 7fa02708c640 1 -- 172.21.15.43:0/3050810392 shutdown_connections 2023-08-09T13:35:19.757 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:19.754+0000 7fa02708c640 1 --2- 172.21.15.43:0/3050810392 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa02015c550 0x7fa0201f5410 unknown :-1 s=CLOSED pgs=322 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:35:19.757 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:19.754+0000 7fa02708c640 1 --2- 172.21.15.43:0/3050810392 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fa0200d04b0 0x7fa0201f4ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:35:19.757 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:19.754+0000 7fa02708c640 1 --2- 172.21.15.43:0/3050810392 >> [v2:172.21.15.203:6832/830097474,v1:172.21.15.203:6833/830097474] conn(0x7fa00c06a050 0x7fa00c06c510 unknown :-1 s=CLOSED pgs=172 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:35:19.757 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:19.754+0000 7fa02708c640 1 --2- 172.21.15.43:0/3050810392 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fa0200b45f0 0x7fa0201e5db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:35:19.757 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:19.754+0000 7fa02708c640 1 -- 172.21.15.43:0/3050810392 >> 172.21.15.43:0/3050810392 conn(0x7fa020071520 msgr2=0x7fa02015f370 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-09T13:35:19.757 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:19.754+0000 7fa02708c640 1 -- 172.21.15.43:0/3050810392 shutdown_connections 2023-08-09T13:35:19.757 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:19.755+0000 7fa02708c640 1 -- 172.21.15.43:0/3050810392 wait complete. 2023-08-09T13:35:19.776 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-08-09T13:35:20.008 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:20.007+0000 7f3c9f59e640 1 Processor -- start 2023-08-09T13:35:20.009 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:20.007+0000 7f3c9f59e640 1 -- start start 2023-08-09T13:35:20.010 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:20.008+0000 7f3c9f59e640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f3ca015c2d0 0x7f3ca00b7df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:35:20.010 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:20.009+0000 7f3c9f59e640 1 -- --> v1:172.21.15.43:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3ca00d16a0 con 0x7f3ca00d1b30 2023-08-09T13:35:20.010 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:20.009+0000 7f3c9f59e640 1 -- --> v1:172.21.15.203:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3ca00d1f70 con 0x7f3ca00cebf0 2023-08-09T13:35:20.010 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:20.009+0000 7f3c9f59e640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3ca00d0d70 con 0x7f3ca015c2d0 2023-08-09T13:35:20.012 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:20.009+0000 7f3c9dd9b640 1 --1- >> v1:172.21.15.43:6789/0 conn(0x7f3ca00d1b30 0x7f3ca00cd020 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.43:6789/0 says I am v1:172.21.15.43:49160/0 (socket says 172.21.15.43:49160) 2023-08-09T13:35:20.012 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:20.009+0000 7f3c9dd9b640 1 -- 172.21.15.43:0/2043082547 learned_addr learned my addr 172.21.15.43:0/2043082547 (peer_addr_for_me v1:172.21.15.43:0/0) 2023-08-09T13:35:20.013 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:20.009+0000 7f3c9e59c640 1 --2- 172.21.15.43:0/2043082547 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f3ca015c2d0 0x7f3ca00b7df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:35:20.013 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:20.009+0000 7f3c9d59a640 1 -- 172.21.15.43:0/2043082547 <== mon.0 v1:172.21.15.43:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 21370386 0 0) 0x7f3ca00d16a0 con 0x7f3ca00d1b30 2023-08-09T13:35:20.013 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:20.009+0000 7f3c9d59a640 1 -- 172.21.15.43:0/2043082547 --> v1:172.21.15.43:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f3c88003520 con 0x7f3ca00d1b30 2023-08-09T13:35:20.013 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:20.010+0000 7f3c9d59a640 1 -- 172.21.15.43:0/2043082547 <== mon.0 v1:172.21.15.43:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2840863147 0 0) 0x7f3c88003520 con 0x7f3ca00d1b30 2023-08-09T13:35:20.013 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:20.010+0000 7f3c9d59a640 1 -- 172.21.15.43:0/2043082547 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f3ca015c2d0 msgr2=0x7f3ca00b7df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-09T13:35:20.013 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:20.010+0000 7f3c9d59a640 1 --2- 172.21.15.43:0/2043082547 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f3ca015c2d0 0x7f3ca00b7df0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:35:20.013 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:20.010+0000 7f3c9d59a640 1 -- 172.21.15.43:0/2043082547 >> v1:172.21.15.203:6789/0 conn(0x7f3ca00cebf0 legacy=0x7f3ca00b5270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:35:20.014 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:20.010+0000 7f3c9d59a640 1 -- 172.21.15.43:0/2043082547 --> v1:172.21.15.43:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3ca00d1050 con 0x7f3ca00d1b30 2023-08-09T13:35:20.014 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:20.010+0000 7f3c9d59a640 1 -- 172.21.15.43:0/2043082547 <== mon.0 v1:172.21.15.43:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1285150740 0 0) 0x7f3c940019f0 con 0x7f3ca00d1b30 2023-08-09T13:35:20.014 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:20.010+0000 7f3c9d59a640 1 -- 172.21.15.43:0/2043082547 >> v1:172.21.15.43:6789/0 conn(0x7f3ca00d1b30 legacy=0x7f3ca00cd020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:35:20.014 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:20.010+0000 7f3c9d59a640 1 --2- 172.21.15.43:0/2043082547 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f3c880039a0 0x7f3c88003dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:35:20.014 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:20.010+0000 7f3c9d59a640 1 --2- 172.21.15.43:0/2043082547 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f3ca015c2d0 0x7f3c880047c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:35:20.014 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:20.010+0000 7f3c9d59a640 1 --2- 172.21.15.43:0/2043082547 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f3c88004d00 0x7f3c88005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:35:20.014 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:20.010+0000 7f3c9d59a640 1 -- 172.21.15.43:0/2043082547 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3ca00d0d70 con 0x7f3c88004d00 2023-08-09T13:35:20.015 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:20.010+0000 7f3c9d59a640 1 -- 172.21.15.43:0/2043082547 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3c8800bb10 con 0x7f3c880039a0 2023-08-09T13:35:20.015 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:20.010+0000 7f3c9d59a640 1 -- 172.21.15.43:0/2043082547 --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3c8800bca0 con 0x7f3ca015c2d0 2023-08-09T13:35:20.015 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:20.010+0000 7f3c9ed9d640 1 --2- 172.21.15.43:0/2043082547 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f3c88004d00 0x7f3c88005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:35:20.015 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:20.010+0000 7f3c9e59c640 1 --2- 172.21.15.43:0/2043082547 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f3c880039a0 0x7f3c88003dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:35:20.015 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:20.011+0000 7f3c9ed9d640 1 -- 172.21.15.43:0/2043082547 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f3ca015c2d0 msgr2=0x7f3c880047c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-09T13:35:20.015 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:20.011+0000 7f3c9ed9d640 1 --2- 172.21.15.43:0/2043082547 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f3ca015c2d0 0x7f3c880047c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:35:20.015 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:20.011+0000 7f3c9ed9d640 1 -- 172.21.15.43:0/2043082547 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f3c880039a0 msgr2=0x7f3c88003dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-09T13:35:20.016 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:20.011+0000 7f3c9ed9d640 1 --2- 172.21.15.43:0/2043082547 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f3c880039a0 0x7f3c88003dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:35:20.016 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:20.011+0000 7f3c9ed9d640 1 -- 172.21.15.43:0/2043082547 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3c88003520 con 0x7f3c88004d00 2023-08-09T13:35:20.016 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:20.011+0000 7f3c9e59c640 1 --2- 172.21.15.43:0/2043082547 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f3c880039a0 0x7f3c88003dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-08-09T13:35:20.023 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:20.012+0000 7f3c9ed9d640 1 --2- 172.21.15.43:0/2043082547 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f3c88004d00 0x7f3c88005180 secure :-1 s=READY pgs=324 cs=0 l=1 rev1=1 crypto rx=0x7f3c98001780 tx=0x7f3c880057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ba5398a7a34ae4e1 server_cookie=0 in_seq=0 out_seq=0 2023-08-09T13:35:20.023 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:20.012+0000 7f3c9d59a640 1 -- 172.21.15.43:0/2043082547 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3c980052d0 con 0x7f3c88004d00 2023-08-09T13:35:20.023 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:20.012+0000 7f3c9d59a640 1 -- 172.21.15.43:0/2043082547 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3c9802e050 con 0x7f3c88004d00 2023-08-09T13:35:20.023 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:20.012+0000 7f3c9d59a640 1 -- 172.21.15.43:0/2043082547 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3c980035f0 con 0x7f3c88004d00 2023-08-09T13:35:20.023 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:20.013+0000 7f3c9f59e640 1 -- 172.21.15.43:0/2043082547 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f3c88004d00 msgr2=0x7f3c88005180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:35:20.024 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:20.013+0000 7f3c9f59e640 1 --2- 172.21.15.43:0/2043082547 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f3c88004d00 0x7f3c88005180 secure :-1 s=READY pgs=324 cs=0 l=1 rev1=1 crypto rx=0x7f3c98001780 tx=0x7f3c880057f0 comp rx=0 tx=0).stop 2023-08-09T13:35:20.024 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:20.013+0000 7f3c9ed9d640 1 -- 172.21.15.43:0/2043082547 reap_dead start 2023-08-09T13:35:20.024 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:20.013+0000 7f3c9f59e640 1 -- 172.21.15.43:0/2043082547 shutdown_connections 2023-08-09T13:35:20.024 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:20.013+0000 7f3c9f59e640 1 -- 172.21.15.43:0/2043082547 >> 172.21.15.43:0/2043082547 conn(0x7f3ca006a230 msgr2=0x7f3ca006a640 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-09T13:35:20.024 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:20.013+0000 7f3c9f59e640 1 -- 172.21.15.43:0/2043082547 shutdown_connections 2023-08-09T13:35:20.024 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:20.013+0000 7f3c9f59e640 1 -- 172.21.15.43:0/2043082547 wait complete. 2023-08-09T13:35:20.025 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:20.015+0000 7f3c9f59e640 1 Processor -- start 2023-08-09T13:35:20.025 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:20.015+0000 7f3c9f59e640 1 -- start start 2023-08-09T13:35:20.025 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:20.016+0000 7f3c9f59e640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f3c88004d00 0x7f3ca0158e00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:35:20.025 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:20.016+0000 7f3c9f59e640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f3ca01f1940 0x7f3ca00c9cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:35:20.025 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:20.016+0000 7f3c9f59e640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f3ca0156650 0x7f3ca0156ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:35:20.025 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:20.016+0000 7f3c9f59e640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3ca00ca200 con 0x7f3ca01f1940 2023-08-09T13:35:20.025 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:20.016+0000 7f3c9f59e640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3ca00ca4a0 con 0x7f3ca0156650 2023-08-09T13:35:20.026 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:20.016+0000 7f3c9f59e640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3ca00ca780 con 0x7f3c88004d00 2023-08-09T13:35:20.026 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:20.016+0000 7f3c9dd9b640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f3ca01f1940 0x7f3ca00c9cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:35:20.026 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:20.016+0000 7f3c9dd9b640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f3ca01f1940 0x7f3ca00c9cc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:42518/0 (socket says 172.21.15.43:42518) 2023-08-09T13:35:20.026 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:20.016+0000 7f3c9dd9b640 1 -- 172.21.15.43:0/1657656951 learned_addr learned my addr 172.21.15.43:0/1657656951 (peer_addr_for_me v2:172.21.15.43:0/0) 2023-08-09T13:35:20.026 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:20.017+0000 7f3c9dd9b640 1 -- 172.21.15.43:0/1657656951 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f3c88004d00 msgr2=0x7f3ca0158e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:35:20.026 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:20.017+0000 7f3c9dd9b640 1 --2- 172.21.15.43:0/1657656951 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f3c88004d00 0x7f3ca0158e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:35:20.027 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:20.017+0000 7f3c9dd9b640 1 -- 172.21.15.43:0/1657656951 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f3ca0156650 msgr2=0x7f3ca0156ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:35:20.027 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:20.017+0000 7f3c9dd9b640 1 --2- 172.21.15.43:0/1657656951 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f3ca0156650 0x7f3ca0156ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:35:20.027 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:20.017+0000 7f3c9dd9b640 1 -- 172.21.15.43:0/1657656951 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3ca00caa60 con 0x7f3ca01f1940 2023-08-09T13:35:20.027 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:20.017+0000 7f3c9dd9b640 1 --2- 172.21.15.43:0/1657656951 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f3ca01f1940 0x7f3ca00c9cc0 secure :-1 s=READY pgs=325 cs=0 l=1 rev1=1 crypto rx=0x7f3c9400ca70 tx=0x7f3c9400cf30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-09T13:35:20.027 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:20.017+0000 7f3c8effd640 1 -- 172.21.15.43:0/1657656951 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3c9400a740 con 0x7f3ca01f1940 2023-08-09T13:35:20.027 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:20.017+0000 7f3c8effd640 1 -- 172.21.15.43:0/1657656951 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3c9400a8a0 con 0x7f3ca01f1940 2023-08-09T13:35:20.027 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:20.017+0000 7f3c8effd640 1 -- 172.21.15.43:0/1657656951 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3c9400ac90 con 0x7f3ca01f1940 2023-08-09T13:35:20.027 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:20.017+0000 7f3c9f59e640 1 -- 172.21.15.43:0/1657656951 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3ca00cad40 con 0x7f3ca01f1940 2023-08-09T13:35:20.028 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:20.017+0000 7f3c9f59e640 1 -- 172.21.15.43:0/1657656951 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3ca0157010 con 0x7f3ca01f1940 2023-08-09T13:35:20.028 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:20.019+0000 7f3c9f59e640 1 -- 172.21.15.43:0/1657656951 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3ca004e390 con 0x7f3ca01f1940 2023-08-09T13:35:20.028 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:20.020+0000 7f3c8effd640 1 -- 172.21.15.43:0/1657656951 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 5) v1 ==== 88341+0+0 (secure 0 0 0) 0x7f3c940070d0 con 0x7f3ca01f1940 2023-08-09T13:35:20.028 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:20.021+0000 7f3c8effd640 1 --2- 172.21.15.43:0/1657656951 >> [v2:172.21.15.203:6832/830097474,v1:172.21.15.203:6833/830097474] conn(0x7f3c8006a050 0x7f3c8006c510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:35:20.028 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:20.021+0000 7f3c8effd640 1 -- 172.21.15.43:0/1657656951 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 6871+0+0 (secure 0 0 0) 0x7f3c94088100 con 0x7f3ca01f1940 2023-08-09T13:35:20.028 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:20.021+0000 7f3c9e59c640 1 --2- 172.21.15.43:0/1657656951 >> [v2:172.21.15.203:6832/830097474,v1:172.21.15.203:6833/830097474] conn(0x7f3c8006a050 0x7f3c8006c510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:35:20.029 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:20.022+0000 7f3c9e59c640 1 --2- 172.21.15.43:0/1657656951 >> [v2:172.21.15.203:6832/830097474,v1:172.21.15.203:6833/830097474] conn(0x7f3c8006a050 0x7f3c8006c510 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7f3c90006fd0 tx=0x7f3c90009350 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-09T13:35:20.029 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:20.025+0000 7f3c8effd640 1 -- 172.21.15.43:0/1657656951 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178066 (secure 0 0 0) 0x7f3c94053440 con 0x7f3ca01f1940 2023-08-09T13:35:20.218 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:20.216+0000 7f3c9f59e640 1 -- 172.21.15.43:0/1657656951 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f3ca00d2f90 con 0x7f3ca01f1940 2023-08-09T13:35:20.219 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:20.217+0000 7f3c8effd640 1 -- 172.21.15.43:0/1657656951 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v26) v1 ==== 93+0+10 (secure 0 0 0) 0x7f3c94018070 con 0x7f3ca01f1940 2023-08-09T13:35:20.219 INFO:teuthology.orchestra.run.smithi043.stdout:pg_num: 1 2023-08-09T13:35:20.221 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:20.220+0000 7f3c9f59e640 1 -- 172.21.15.43:0/1657656951 >> [v2:172.21.15.203:6832/830097474,v1:172.21.15.203:6833/830097474] conn(0x7f3c8006a050 msgr2=0x7f3c8006c510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:35:20.221 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:20.220+0000 7f3c9f59e640 1 --2- 172.21.15.43:0/1657656951 >> [v2:172.21.15.203:6832/830097474,v1:172.21.15.203:6833/830097474] conn(0x7f3c8006a050 0x7f3c8006c510 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7f3c90006fd0 tx=0x7f3c90009350 comp rx=0 tx=0).stop 2023-08-09T13:35:20.221 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:20.220+0000 7f3c9f59e640 1 -- 172.21.15.43:0/1657656951 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f3ca01f1940 msgr2=0x7f3ca00c9cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:35:20.222 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:20.220+0000 7f3c9f59e640 1 --2- 172.21.15.43:0/1657656951 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f3ca01f1940 0x7f3ca00c9cc0 secure :-1 s=READY pgs=325 cs=0 l=1 rev1=1 crypto rx=0x7f3c9400ca70 tx=0x7f3c9400cf30 comp rx=0 tx=0).stop 2023-08-09T13:35:20.222 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:20.220+0000 7f3c9f59e640 1 -- 172.21.15.43:0/1657656951 shutdown_connections 2023-08-09T13:35:20.223 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:20.220+0000 7f3c9f59e640 1 --2- 172.21.15.43:0/1657656951 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f3ca0156650 0x7f3ca0156ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:35:20.223 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:20.220+0000 7f3c9f59e640 1 --2- 172.21.15.43:0/1657656951 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f3ca01f1940 0x7f3ca00c9cc0 unknown :-1 s=CLOSED pgs=325 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:35:20.223 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:20.220+0000 7f3c9f59e640 1 --2- 172.21.15.43:0/1657656951 >> [v2:172.21.15.203:6832/830097474,v1:172.21.15.203:6833/830097474] conn(0x7f3c8006a050 0x7f3c8006c510 unknown :-1 s=CLOSED pgs=173 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:35:20.223 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:20.220+0000 7f3c9f59e640 1 --2- 172.21.15.43:0/1657656951 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f3c88004d00 0x7f3ca0158e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:35:20.223 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:20.220+0000 7f3c9f59e640 1 -- 172.21.15.43:0/1657656951 >> 172.21.15.43:0/1657656951 conn(0x7f3ca006a230 msgr2=0x7f3ca00b5270 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-09T13:35:20.223 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:20.220+0000 7f3c9f59e640 1 -- 172.21.15.43:0/1657656951 shutdown_connections 2023-08-09T13:35:20.223 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:20.220+0000 7f3c9f59e640 1 -- 172.21.15.43:0/1657656951 wait complete. 2023-08-09T13:35:20.242 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-08-09T13:35:20.474 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:20.472+0000 7fb59322d640 1 Processor -- start 2023-08-09T13:35:20.474 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:20.472+0000 7fb59322d640 1 -- start start 2023-08-09T13:35:20.476 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:20.474+0000 7fb59322d640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fb58c15c630 0x7fb58c0abef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:35:20.476 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:20.474+0000 7fb59322d640 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7fb58c0cdc60 0x7fb58c15ca10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:35:20.476 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:20.474+0000 7fb59322d640 1 -- --> v1:172.21.15.203:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb58c0cca90 con 0x7fb58c0ccf20 2023-08-09T13:35:20.476 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:20.474+0000 7fb59322d640 1 -- --> v2:172.21.15.203:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb58c0cbc70 con 0x7fb58c0cdc60 2023-08-09T13:35:20.477 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:20.474+0000 7fb59322d640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb58c0cbe80 con 0x7fb58c15c630 2023-08-09T13:35:20.477 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:20.475+0000 7fb590fc9640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fb58c15c630 0x7fb58c0abef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:35:20.477 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:20.475+0000 7fb5917ca640 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7fb58c0cdc60 0x7fb58c15ca10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:35:20.478 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:20.475+0000 7fb5917ca640 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7fb58c0cdc60 0x7fb58c15ca10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.203:3300/0 says I am v2:172.21.15.43:58122/0 (socket says 172.21.15.43:58122) 2023-08-09T13:35:20.478 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:20.475+0000 7fb5917ca640 1 -- 172.21.15.43:0/4224430784 learned_addr learned my addr 172.21.15.43:0/4224430784 (peer_addr_for_me v2:172.21.15.43:0/0) 2023-08-09T13:35:20.478 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:20.476+0000 7fb590fc9640 1 -- 172.21.15.43:0/4224430784 >> v2:172.21.15.203:3300/0 conn(0x7fb58c0cdc60 msgr2=0x7fb58c15ca10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-09T13:35:20.478 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:20.476+0000 7fb590fc9640 1 --2- 172.21.15.43:0/4224430784 >> v2:172.21.15.203:3300/0 conn(0x7fb58c0cdc60 0x7fb58c15ca10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:35:20.479 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:20.477+0000 7fb590fc9640 1 -- 172.21.15.43:0/4224430784 >> v1:172.21.15.203:6789/0 conn(0x7fb58c0ccf20 legacy=0x7fb58c0cd550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:35:20.479 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:20.477+0000 7fb590fc9640 1 -- 172.21.15.43:0/4224430784 --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb58c0cc160 con 0x7fb58c15c630 2023-08-09T13:35:20.479 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:20.477+0000 7fb5917ca640 1 --2- 172.21.15.43:0/4224430784 >> v2:172.21.15.203:3300/0 conn(0x7fb58c0cdc60 0x7fb58c15ca10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-08-09T13:35:20.479 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:20.478+0000 7fb590fc9640 1 --2- 172.21.15.43:0/4224430784 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fb58c15c630 0x7fb58c0abef0 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7fb5800097a0 tx=0x7fb58002b8b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d4666d6dcfa0e6b server_cookie=0 in_seq=0 out_seq=0 2023-08-09T13:35:20.480 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:20.478+0000 7fb58b7fe640 1 -- 172.21.15.43:0/4224430784 <== mon.2 v2:172.21.15.203:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb58002d040 con 0x7fb58c15c630 2023-08-09T13:35:20.480 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:20.478+0000 7fb58b7fe640 1 -- 172.21.15.43:0/4224430784 <== mon.2 v2:172.21.15.203:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb580031810 con 0x7fb58c15c630 2023-08-09T13:35:20.480 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:20.478+0000 7fb58b7fe640 1 -- 172.21.15.43:0/4224430784 <== mon.2 v2:172.21.15.203:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb580031ba0 con 0x7fb58c15c630 2023-08-09T13:35:20.481 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:20.479+0000 7fb59322d640 1 -- 172.21.15.43:0/4224430784 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fb58c15c630 msgr2=0x7fb58c0abef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:35:20.481 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:20.479+0000 7fb59322d640 1 --2- 172.21.15.43:0/4224430784 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fb58c15c630 0x7fb58c0abef0 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7fb5800097a0 tx=0x7fb58002b8b0 comp rx=0 tx=0).stop 2023-08-09T13:35:20.481 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:20.479+0000 7fb59322d640 1 -- 172.21.15.43:0/4224430784 shutdown_connections 2023-08-09T13:35:20.481 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:20.479+0000 7fb59322d640 1 --2- 172.21.15.43:0/4224430784 >> v2:172.21.15.203:3300/0 conn(0x7fb58c0cdc60 0x7fb58c15ca10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:35:20.482 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:20.479+0000 7fb59322d640 1 --2- 172.21.15.43:0/4224430784 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fb58c15c630 0x7fb58c0abef0 unknown :-1 s=CLOSED pgs=63 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:35:20.482 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:20.479+0000 7fb59322d640 1 -- 172.21.15.43:0/4224430784 >> 172.21.15.43:0/4224430784 conn(0x7fb58c0ce0a0 msgr2=0x7fb58c0ce480 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-09T13:35:20.482 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:20.479+0000 7fb59322d640 1 -- 172.21.15.43:0/4224430784 shutdown_connections 2023-08-09T13:35:20.482 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:20.479+0000 7fb59322d640 1 -- 172.21.15.43:0/4224430784 wait complete. 2023-08-09T13:35:20.482 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:20.481+0000 7fb59322d640 1 Processor -- start 2023-08-09T13:35:20.483 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:20.481+0000 7fb59322d640 1 -- start start 2023-08-09T13:35:20.484 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:20.482+0000 7fb59322d640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fb58c0ccf20 0x7fb58c0c7940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:35:20.484 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:20.482+0000 7fb59322d640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb58c0cdc60 0x7fb58c159b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:35:20.484 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:20.482+0000 7fb59322d640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fb58c15c630 0x7fb58c06f440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:35:20.484 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:20.482+0000 7fb59322d640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb58c06fd20 con 0x7fb58c0cdc60 2023-08-09T13:35:20.484 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:20.482+0000 7fb59322d640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb58c070000 con 0x7fb58c15c630 2023-08-09T13:35:20.484 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:20.482+0000 7fb59322d640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb58c0c6e60 con 0x7fb58c0ccf20 2023-08-09T13:35:20.486 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:20.483+0000 7fb58bfff640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb58c0cdc60 0x7fb58c159b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:35:20.486 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:20.483+0000 7fb58bfff640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb58c0cdc60 0x7fb58c159b30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:42532/0 (socket says 172.21.15.43:42532) 2023-08-09T13:35:20.486 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:20.483+0000 7fb58bfff640 1 -- 172.21.15.43:0/2841718458 learned_addr learned my addr 172.21.15.43:0/2841718458 (peer_addr_for_me v2:172.21.15.43:0/0) 2023-08-09T13:35:20.486 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:20.483+0000 7fb590fc9640 1 --2- 172.21.15.43:0/2841718458 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fb58c0ccf20 0x7fb58c0c7940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:35:20.486 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:20.483+0000 7fb5917ca640 1 --2- 172.21.15.43:0/2841718458 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fb58c15c630 0x7fb58c06f440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:35:20.486 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:20.483+0000 7fb58bfff640 1 -- 172.21.15.43:0/2841718458 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fb58c0ccf20 msgr2=0x7fb58c0c7940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:35:20.486 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:20.483+0000 7fb58bfff640 1 --2- 172.21.15.43:0/2841718458 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fb58c0ccf20 0x7fb58c0c7940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:35:20.487 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:20.483+0000 7fb58bfff640 1 -- 172.21.15.43:0/2841718458 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fb58c15c630 msgr2=0x7fb58c06f440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:35:20.487 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:20.483+0000 7fb58bfff640 1 --2- 172.21.15.43:0/2841718458 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fb58c15c630 0x7fb58c06f440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:35:20.487 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:20.483+0000 7fb58bfff640 1 -- 172.21.15.43:0/2841718458 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb584009b30 con 0x7fb58c0cdc60 2023-08-09T13:35:20.487 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:20.484+0000 7fb58bfff640 1 --2- 172.21.15.43:0/2841718458 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb58c0cdc60 0x7fb58c159b30 secure :-1 s=READY pgs=326 cs=0 l=1 rev1=1 crypto rx=0x7fb57800c450 tx=0x7fb57800c910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-09T13:35:20.487 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:20.484+0000 7fb5897fa640 1 -- 172.21.15.43:0/2841718458 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb5780052d0 con 0x7fb58c0cdc60 2023-08-09T13:35:20.487 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:20.484+0000 7fb5897fa640 1 -- 172.21.15.43:0/2841718458 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb578005430 con 0x7fb58c0cdc60 2023-08-09T13:35:20.487 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:20.484+0000 7fb5897fa640 1 -- 172.21.15.43:0/2841718458 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb578012870 con 0x7fb58c0cdc60 2023-08-09T13:35:20.488 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:20.484+0000 7fb59322d640 1 -- 172.21.15.43:0/2841718458 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb580009400 con 0x7fb58c0cdc60 2023-08-09T13:35:20.488 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:20.484+0000 7fb59322d640 1 -- 172.21.15.43:0/2841718458 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb58c0c7420 con 0x7fb58c0cdc60 2023-08-09T13:35:20.488 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:20.485+0000 7fb59322d640 1 -- 172.21.15.43:0/2841718458 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb58c0c7640 con 0x7fb58c0cdc60 2023-08-09T13:35:20.488 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:20.486+0000 7fb5897fa640 1 -- 172.21.15.43:0/2841718458 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 5) v1 ==== 88341+0+0 (secure 0 0 0) 0x7fb5780129d0 con 0x7fb58c0cdc60 2023-08-09T13:35:20.489 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:20.487+0000 7fb5897fa640 1 --2- 172.21.15.43:0/2841718458 >> [v2:172.21.15.203:6832/830097474,v1:172.21.15.203:6833/830097474] conn(0x7fb57006a050 0x7fb57006c510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:35:20.489 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:20.487+0000 7fb5897fa640 1 -- 172.21.15.43:0/2841718458 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 6871+0+0 (secure 0 0 0) 0x7fb578088450 con 0x7fb58c0cdc60 2023-08-09T13:35:20.489 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:20.488+0000 7fb590fc9640 1 --2- 172.21.15.43:0/2841718458 >> [v2:172.21.15.203:6832/830097474,v1:172.21.15.203:6833/830097474] conn(0x7fb57006a050 0x7fb57006c510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:35:20.491 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:20.489+0000 7fb590fc9640 1 --2- 172.21.15.43:0/2841718458 >> [v2:172.21.15.203:6832/830097474,v1:172.21.15.203:6833/830097474] conn(0x7fb57006a050 0x7fb57006c510 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7fb580036000 tx=0x7fb58003c040 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-09T13:35:20.493 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:20.491+0000 7fb5897fa640 1 -- 172.21.15.43:0/2841718458 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178066 (secure 0 0 0) 0x7fb578053800 con 0x7fb58c0cdc60 2023-08-09T13:35:20.688 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:20.687+0000 7fb59322d640 1 -- 172.21.15.43:0/2841718458 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7fb58c06fe60 con 0x7fb58c0cdc60 2023-08-09T13:35:20.690 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:20.688+0000 7fb5897fa640 1 -- 172.21.15.43:0/2841718458 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v26) v1 ==== 104+0+11 (secure 0 0 0) 0x7fb58c06fe60 con 0x7fb58c0cdc60 2023-08-09T13:35:20.690 INFO:teuthology.orchestra.run.smithi043.stdout:pg_num: 32 2023-08-09T13:35:20.692 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:20.690+0000 7fb59322d640 1 -- 172.21.15.43:0/2841718458 >> [v2:172.21.15.203:6832/830097474,v1:172.21.15.203:6833/830097474] conn(0x7fb57006a050 msgr2=0x7fb57006c510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:35:20.692 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:20.690+0000 7fb59322d640 1 --2- 172.21.15.43:0/2841718458 >> [v2:172.21.15.203:6832/830097474,v1:172.21.15.203:6833/830097474] conn(0x7fb57006a050 0x7fb57006c510 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7fb580036000 tx=0x7fb58003c040 comp rx=0 tx=0).stop 2023-08-09T13:35:20.692 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:20.690+0000 7fb59322d640 1 -- 172.21.15.43:0/2841718458 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb58c0cdc60 msgr2=0x7fb58c159b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:35:20.692 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:20.690+0000 7fb59322d640 1 --2- 172.21.15.43:0/2841718458 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb58c0cdc60 0x7fb58c159b30 secure :-1 s=READY pgs=326 cs=0 l=1 rev1=1 crypto rx=0x7fb57800c450 tx=0x7fb57800c910 comp rx=0 tx=0).stop 2023-08-09T13:35:20.693 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:20.691+0000 7fb59322d640 1 -- 172.21.15.43:0/2841718458 shutdown_connections 2023-08-09T13:35:20.693 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:20.691+0000 7fb59322d640 1 --2- 172.21.15.43:0/2841718458 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fb58c15c630 0x7fb58c06f440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:35:20.693 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:20.691+0000 7fb59322d640 1 --2- 172.21.15.43:0/2841718458 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb58c0cdc60 0x7fb58c159b30 unknown :-1 s=CLOSED pgs=326 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:35:20.694 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:20.691+0000 7fb59322d640 1 --2- 172.21.15.43:0/2841718458 >> [v2:172.21.15.203:6832/830097474,v1:172.21.15.203:6833/830097474] conn(0x7fb57006a050 0x7fb57006c510 unknown :-1 s=CLOSED pgs=174 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:35:20.694 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:20.691+0000 7fb59322d640 1 --2- 172.21.15.43:0/2841718458 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fb58c0ccf20 0x7fb58c0c7940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:35:20.694 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:20.691+0000 7fb59322d640 1 -- 172.21.15.43:0/2841718458 >> 172.21.15.43:0/2841718458 conn(0x7fb58c0ce0a0 msgr2=0x7fb58c0cd340 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-09T13:35:20.694 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:20.691+0000 7fb59322d640 1 -- 172.21.15.43:0/2841718458 shutdown_connections 2023-08-09T13:35:20.694 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:20.691+0000 7fb59322d640 1 -- 172.21.15.43:0/2841718458 wait complete. 2023-08-09T13:35:20.712 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-08-09T13:35:20.947 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:20.945+0000 7ff54dd02640 1 Processor -- start 2023-08-09T13:35:20.948 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:20.945+0000 7ff54dd02640 1 -- start start 2023-08-09T13:35:20.949 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:20.946+0000 7ff54dd02640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7ff5480c1020 0x7ff548162360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:35:20.949 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:20.946+0000 7ff54dd02640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7ff548164940 0x7ff548164df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:35:20.949 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:20.946+0000 7ff54dd02640 1 -- --> v1:172.21.15.43:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff5480cc3b0 con 0x7ff548164560 2023-08-09T13:35:20.949 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:20.946+0000 7ff54dd02640 1 -- --> v2:172.21.15.43:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff5480ca6d0 con 0x7ff548164940 2023-08-09T13:35:20.950 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:20.947+0000 7ff54dd02640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff5480af2e0 con 0x7ff5480c1020 2023-08-09T13:35:20.950 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:20.947+0000 7ff54cd00640 1 --1- >> v1:172.21.15.43:6789/0 conn(0x7ff548164560 0x7ff5480c8420 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.43:6789/0 says I am v1:172.21.15.43:49166/0 (socket says 172.21.15.43:49166) 2023-08-09T13:35:20.950 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:20.947+0000 7ff54cd00640 1 -- 172.21.15.43:0/548804049 learned_addr learned my addr 172.21.15.43:0/548804049 (peer_addr_for_me v1:172.21.15.43:0/0) 2023-08-09T13:35:20.951 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:20.947+0000 7ff54d501640 1 --2- 172.21.15.43:0/548804049 >> v2:172.21.15.43:3300/0 conn(0x7ff548164940 0x7ff548164df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:35:20.951 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:20.947+0000 7ff547fff640 1 --2- 172.21.15.43:0/548804049 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7ff5480c1020 0x7ff548162360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:35:20.951 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:20.947+0000 7ff5477fe640 1 -- 172.21.15.43:0/548804049 <== mon.0 v1:172.21.15.43:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1717582691 0 0) 0x7ff5480cc3b0 con 0x7ff548164560 2023-08-09T13:35:20.951 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:20.947+0000 7ff5477fe640 1 -- 172.21.15.43:0/548804049 --> v1:172.21.15.43:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ff538003520 con 0x7ff548164560 2023-08-09T13:35:20.951 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:20.947+0000 7ff54d501640 1 -- 172.21.15.43:0/548804049 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7ff5480c1020 msgr2=0x7ff548162360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-09T13:35:20.952 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:20.947+0000 7ff54d501640 1 --2- 172.21.15.43:0/548804049 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7ff5480c1020 0x7ff548162360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:35:20.952 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:20.947+0000 7ff54d501640 1 -- 172.21.15.43:0/548804049 >> v1:172.21.15.43:6789/0 conn(0x7ff548164560 legacy=0x7ff5480c8420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:35:20.952 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:20.948+0000 7ff54d501640 1 -- 172.21.15.43:0/548804049 --> v2:172.21.15.43:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff5480cd790 con 0x7ff548164940 2023-08-09T13:35:20.952 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:20.949+0000 7ff54d501640 1 --2- 172.21.15.43:0/548804049 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff548164940 0x7ff548164df0 secure :-1 s=READY pgs=328 cs=0 l=1 rev1=1 crypto rx=0x7ff540009d90 tx=0x7ff54002bd90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bce531afe2b03bd3 server_cookie=0 in_seq=0 out_seq=0 2023-08-09T13:35:20.953 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:20.949+0000 7ff5477fe640 1 -- 172.21.15.43:0/548804049 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff54002cb40 con 0x7ff548164940 2023-08-09T13:35:20.953 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:20.949+0000 7ff5477fe640 1 -- 172.21.15.43:0/548804049 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff54002cca0 con 0x7ff548164940 2023-08-09T13:35:20.954 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:20.949+0000 7ff5477fe640 1 -- 172.21.15.43:0/548804049 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff540034640 con 0x7ff548164940 2023-08-09T13:35:20.954 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:20.949+0000 7ff54dd02640 1 -- 172.21.15.43:0/548804049 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff548164940 msgr2=0x7ff548164df0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:35:20.954 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:20.949+0000 7ff54dd02640 1 --2- 172.21.15.43:0/548804049 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff548164940 0x7ff548164df0 secure :-1 s=READY pgs=328 cs=0 l=1 rev1=1 crypto rx=0x7ff540009d90 tx=0x7ff54002bd90 comp rx=0 tx=0).stop 2023-08-09T13:35:20.954 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:20.950+0000 7ff54dd02640 1 -- 172.21.15.43:0/548804049 shutdown_connections 2023-08-09T13:35:20.954 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:20.950+0000 7ff54dd02640 1 --2- 172.21.15.43:0/548804049 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff548164940 0x7ff548164df0 unknown :-1 s=CLOSED pgs=328 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:35:20.954 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:20.950+0000 7ff54dd02640 1 --2- 172.21.15.43:0/548804049 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7ff5480c1020 0x7ff548162360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:35:20.954 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:20.950+0000 7ff54dd02640 1 -- 172.21.15.43:0/548804049 >> 172.21.15.43:0/548804049 conn(0x7ff5480cddd0 msgr2=0x7ff5480ce1e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-09T13:35:20.955 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:20.950+0000 7ff54dd02640 1 -- 172.21.15.43:0/548804049 shutdown_connections 2023-08-09T13:35:20.955 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:20.950+0000 7ff54dd02640 1 -- 172.21.15.43:0/548804049 wait complete. 2023-08-09T13:35:20.957 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:20.951+0000 7ff54dd02640 1 Processor -- start 2023-08-09T13:35:20.957 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:20.952+0000 7ff54dd02640 1 -- start start 2023-08-09T13:35:20.957 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:20.952+0000 7ff54dd02640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7ff5480c1020 0x7ff54815c120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:35:20.957 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:20.953+0000 7ff54dd02640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff548164560 0x7ff548156120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:35:20.957 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:20.953+0000 7ff54dd02640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7ff548164940 0x7ff548162150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:35:20.958 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:20.953+0000 7ff54dd02640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff54815bb50 con 0x7ff548164560 2023-08-09T13:35:20.958 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:20.953+0000 7ff54dd02640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff54815da70 con 0x7ff5480c1020 2023-08-09T13:35:20.958 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:20.953+0000 7ff54dd02640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff54815dd50 con 0x7ff548164940 2023-08-09T13:35:20.958 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:20.953+0000 7ff547fff640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff548164560 0x7ff548156120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:35:20.958 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:20.953+0000 7ff547fff640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff548164560 0x7ff548156120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:42542/0 (socket says 172.21.15.43:42542) 2023-08-09T13:35:20.958 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:20.953+0000 7ff547fff640 1 -- 172.21.15.43:0/3293328345 learned_addr learned my addr 172.21.15.43:0/3293328345 (peer_addr_for_me v2:172.21.15.43:0/0) 2023-08-09T13:35:20.958 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:20.953+0000 7ff54cd00640 1 --2- 172.21.15.43:0/3293328345 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7ff5480c1020 0x7ff54815c120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:35:20.959 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:20.953+0000 7ff54d501640 1 --2- 172.21.15.43:0/3293328345 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7ff548164940 0x7ff548162150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:35:20.959 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:20.953+0000 7ff547fff640 1 -- 172.21.15.43:0/3293328345 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7ff548164940 msgr2=0x7ff548162150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:35:20.959 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:20.954+0000 7ff547fff640 1 --2- 172.21.15.43:0/3293328345 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7ff548164940 0x7ff548162150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:35:20.959 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:20.954+0000 7ff547fff640 1 -- 172.21.15.43:0/3293328345 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7ff5480c1020 msgr2=0x7ff54815c120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:35:20.959 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:20.954+0000 7ff547fff640 1 --2- 172.21.15.43:0/3293328345 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7ff5480c1020 0x7ff54815c120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:35:20.959 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:20.954+0000 7ff547fff640 1 -- 172.21.15.43:0/3293328345 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff540009a70 con 0x7ff548164560 2023-08-09T13:35:20.959 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:20.954+0000 7ff547fff640 1 --2- 172.21.15.43:0/3293328345 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff548164560 0x7ff548156120 secure :-1 s=READY pgs=329 cs=0 l=1 rev1=1 crypto rx=0x7ff53400d5f0 tx=0x7ff53400dab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-09T13:35:20.960 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:20.954+0000 7ff5457fa640 1 -- 172.21.15.43:0/3293328345 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff5340125f0 con 0x7ff548164560 2023-08-09T13:35:20.960 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:20.954+0000 7ff5457fa640 1 -- 172.21.15.43:0/3293328345 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff534012750 con 0x7ff548164560 2023-08-09T13:35:20.960 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:20.954+0000 7ff5457fa640 1 -- 172.21.15.43:0/3293328345 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff534012b00 con 0x7ff548164560 2023-08-09T13:35:20.960 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:20.954+0000 7ff54dd02640 1 -- 172.21.15.43:0/3293328345 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff54815e030 con 0x7ff548164560 2023-08-09T13:35:20.960 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:20.954+0000 7ff54dd02640 1 -- 172.21.15.43:0/3293328345 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff548156660 con 0x7ff548164560 2023-08-09T13:35:20.961 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:20.956+0000 7ff54dd02640 1 -- 172.21.15.43:0/3293328345 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff5481567f0 con 0x7ff548164560 2023-08-09T13:35:20.961 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:20.957+0000 7ff5457fa640 1 -- 172.21.15.43:0/3293328345 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 5) v1 ==== 88341+0+0 (secure 0 0 0) 0x7ff534012c60 con 0x7ff548164560 2023-08-09T13:35:20.961 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:20.958+0000 7ff5457fa640 1 --2- 172.21.15.43:0/3293328345 >> [v2:172.21.15.203:6832/830097474,v1:172.21.15.203:6833/830097474] conn(0x7ff52c06a0a0 0x7ff52c06c560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:35:20.961 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:20.958+0000 7ff5457fa640 1 -- 172.21.15.43:0/3293328345 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 6871+0+0 (secure 0 0 0) 0x7ff534088d90 con 0x7ff548164560 2023-08-09T13:35:20.962 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:20.958+0000 7ff54cd00640 1 --2- 172.21.15.43:0/3293328345 >> [v2:172.21.15.203:6832/830097474,v1:172.21.15.203:6833/830097474] conn(0x7ff52c06a0a0 0x7ff52c06c560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:35:20.962 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:20.960+0000 7ff54cd00640 1 --2- 172.21.15.43:0/3293328345 >> [v2:172.21.15.203:6832/830097474,v1:172.21.15.203:6833/830097474] conn(0x7ff52c06a0a0 0x7ff52c06c560 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7ff53c0011e0 tx=0x7ff53c009040 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-09T13:35:20.964 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:20.962+0000 7ff5457fa640 1 -- 172.21.15.43:0/3293328345 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178066 (secure 0 0 0) 0x7ff534054130 con 0x7ff548164560 2023-08-09T13:35:21.161 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:21.159+0000 7ff54dd02640 1 -- 172.21.15.43:0/3293328345 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7ff54804e390 con 0x7ff548164560 2023-08-09T13:35:21.162 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:21.160+0000 7ff5457fa640 1 -- 172.21.15.43:0/3293328345 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v26) v1 ==== 100+0+11 (secure 0 0 0) 0x7ff534057750 con 0x7ff548164560 2023-08-09T13:35:21.163 INFO:teuthology.orchestra.run.smithi043.stdout:pg_num: 64 2023-08-09T13:35:21.165 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:21.162+0000 7ff54dd02640 1 -- 172.21.15.43:0/3293328345 >> [v2:172.21.15.203:6832/830097474,v1:172.21.15.203:6833/830097474] conn(0x7ff52c06a0a0 msgr2=0x7ff52c06c560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:35:21.165 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:21.162+0000 7ff54dd02640 1 --2- 172.21.15.43:0/3293328345 >> [v2:172.21.15.203:6832/830097474,v1:172.21.15.203:6833/830097474] conn(0x7ff52c06a0a0 0x7ff52c06c560 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7ff53c0011e0 tx=0x7ff53c009040 comp rx=0 tx=0).stop 2023-08-09T13:35:21.165 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:21.162+0000 7ff54dd02640 1 -- 172.21.15.43:0/3293328345 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff548164560 msgr2=0x7ff548156120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:35:21.165 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:21.162+0000 7ff54dd02640 1 --2- 172.21.15.43:0/3293328345 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff548164560 0x7ff548156120 secure :-1 s=READY pgs=329 cs=0 l=1 rev1=1 crypto rx=0x7ff53400d5f0 tx=0x7ff53400dab0 comp rx=0 tx=0).stop 2023-08-09T13:35:21.165 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:21.162+0000 7ff54dd02640 1 -- 172.21.15.43:0/3293328345 shutdown_connections 2023-08-09T13:35:21.165 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:21.162+0000 7ff54dd02640 1 --2- 172.21.15.43:0/3293328345 >> [v2:172.21.15.203:6832/830097474,v1:172.21.15.203:6833/830097474] conn(0x7ff52c06a0a0 0x7ff52c06c560 unknown :-1 s=CLOSED pgs=175 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:35:21.165 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:21.162+0000 7ff54dd02640 1 --2- 172.21.15.43:0/3293328345 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7ff548164940 0x7ff548162150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:35:21.166 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:21.162+0000 7ff54dd02640 1 --2- 172.21.15.43:0/3293328345 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff548164560 0x7ff548156120 unknown :-1 s=CLOSED pgs=329 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:35:21.166 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:21.162+0000 7ff54dd02640 1 --2- 172.21.15.43:0/3293328345 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7ff5480c1020 0x7ff54815c120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:35:21.166 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:21.162+0000 7ff54dd02640 1 -- 172.21.15.43:0/3293328345 >> 172.21.15.43:0/3293328345 conn(0x7ff5480cddd0 msgr2=0x7ff5480ce240 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-09T13:35:21.166 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:21.163+0000 7ff54dd02640 1 -- 172.21.15.43:0/3293328345 shutdown_connections 2023-08-09T13:35:21.167 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:21.163+0000 7ff54dd02640 1 -- 172.21.15.43:0/3293328345 wait complete. 2023-08-09T13:35:21.183 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2023-08-09T13:35:21.417 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:21.414+0000 7f33e0a77640 1 Processor -- start 2023-08-09T13:35:21.417 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:21.415+0000 7f33e0a77640 1 -- start start 2023-08-09T13:35:21.419 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:21.416+0000 7f33e0a77640 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7f33dc157ce0 0x7f33dc15b0f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:35:21.419 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:21.416+0000 7f33e0a77640 1 -- --> v1:172.21.15.43:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f33dc06b480 con 0x7f33dc0c8470 2023-08-09T13:35:21.419 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:21.416+0000 7f33e0a77640 1 -- --> v1:172.21.15.203:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f33dc071580 con 0x7f33dc06f2c0 2023-08-09T13:35:21.419 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:21.416+0000 7f33e0a77640 1 -- --> v2:172.21.15.203:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f33dc1580c0 con 0x7f33dc157ce0 2023-08-09T13:35:21.420 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:21.417+0000 7f33dbfff640 1 --1- >> v1:172.21.15.43:6789/0 conn(0x7f33dc0c8470 0x7f33dc0c8880 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.43:6789/0 says I am v1:172.21.15.43:49168/0 (socket says 172.21.15.43:49168) 2023-08-09T13:35:21.420 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:21.417+0000 7f33dbfff640 1 -- 172.21.15.43:0/2767625760 learned_addr learned my addr 172.21.15.43:0/2767625760 (peer_addr_for_me v1:172.21.15.43:0/0) 2023-08-09T13:35:21.421 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:21.417+0000 7f33db7fe640 1 --2- 172.21.15.43:0/2767625760 >> v2:172.21.15.203:3300/0 conn(0x7f33dc157ce0 0x7f33dc15b0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:35:21.421 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:21.417+0000 7f33da7fc640 1 -- 172.21.15.43:0/2767625760 <== mon.0 v1:172.21.15.43:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 18627679 0 0) 0x7f33dc06b480 con 0x7f33dc0c8470 2023-08-09T13:35:21.421 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:21.417+0000 7f33da7fc640 1 -- 172.21.15.43:0/2767625760 --> v1:172.21.15.43:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f33c4003520 con 0x7f33dc0c8470 2023-08-09T13:35:21.421 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:21.418+0000 7f33da7fc640 1 -- 172.21.15.43:0/2767625760 <== mon.0 v1:172.21.15.43:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 981845681 0 0) 0x7f33c4003520 con 0x7f33dc0c8470 2023-08-09T13:35:21.421 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:21.418+0000 7f33da7fc640 1 -- 172.21.15.43:0/2767625760 >> v2:172.21.15.203:3300/0 conn(0x7f33dc157ce0 msgr2=0x7f33dc15b0f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-09T13:35:21.421 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:21.418+0000 7f33da7fc640 1 --2- 172.21.15.43:0/2767625760 >> v2:172.21.15.203:3300/0 conn(0x7f33dc157ce0 0x7f33dc15b0f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:35:21.421 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:21.418+0000 7f33da7fc640 1 -- 172.21.15.43:0/2767625760 >> v1:172.21.15.203:6789/0 conn(0x7f33dc06f2c0 legacy=0x7f33dc06bc40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:35:21.422 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:21.418+0000 7f33da7fc640 1 -- 172.21.15.43:0/2767625760 --> v1:172.21.15.43:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f33dc06c350 con 0x7f33dc0c8470 2023-08-09T13:35:21.422 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:21.418+0000 7f33da7fc640 1 -- 172.21.15.43:0/2767625760 <== mon.0 v1:172.21.15.43:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1285150740 0 0) 0x7f33d40019f0 con 0x7f33dc0c8470 2023-08-09T13:35:21.422 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:21.418+0000 7f33da7fc640 1 -- 172.21.15.43:0/2767625760 >> v1:172.21.15.43:6789/0 conn(0x7f33dc0c8470 legacy=0x7f33dc0c8880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:35:21.422 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:21.418+0000 7f33da7fc640 1 --2- 172.21.15.43:0/2767625760 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f33c4003970 0x7f33c4003db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:35:21.422 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:21.418+0000 7f33da7fc640 1 --2- 172.21.15.43:0/2767625760 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f33c40047c0 0x7f33c4004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:35:21.422 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:21.418+0000 7f33da7fc640 1 --2- 172.21.15.43:0/2767625760 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f33c40051d0 0x7f33c400b980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:35:21.422 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:21.418+0000 7f33da7fc640 1 -- 172.21.15.43:0/2767625760 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f33dc1580c0 con 0x7f33c40047c0 2023-08-09T13:35:21.422 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:21.418+0000 7f33da7fc640 1 -- 172.21.15.43:0/2767625760 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f33c400c100 con 0x7f33c4003970 2023-08-09T13:35:21.423 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:21.418+0000 7f33da7fc640 1 -- 172.21.15.43:0/2767625760 --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f33c400c2c0 con 0x7f33c40051d0 2023-08-09T13:35:21.424 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:21.418+0000 7f33daffd640 1 --2- 172.21.15.43:0/2767625760 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f33c40047c0 0x7f33c4004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:35:21.424 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:21.419+0000 7f33db7fe640 1 --2- 172.21.15.43:0/2767625760 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f33c4003970 0x7f33c4003db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:35:21.425 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:21.419+0000 7f33dbfff640 1 --2- 172.21.15.43:0/2767625760 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f33c40051d0 0x7f33c400b980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:35:21.425 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:21.419+0000 7f33daffd640 1 -- 172.21.15.43:0/2767625760 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f33c40051d0 msgr2=0x7f33c400b980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-09T13:35:21.425 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:21.419+0000 7f33daffd640 1 --2- 172.21.15.43:0/2767625760 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f33c40051d0 0x7f33c400b980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:35:21.425 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:21.419+0000 7f33daffd640 1 -- 172.21.15.43:0/2767625760 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f33c4003970 msgr2=0x7f33c4003db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-09T13:35:21.425 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:21.419+0000 7f33daffd640 1 --2- 172.21.15.43:0/2767625760 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f33c4003970 0x7f33c4003db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:35:21.425 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:21.419+0000 7f33daffd640 1 -- 172.21.15.43:0/2767625760 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f33c4003520 con 0x7f33c40047c0 2023-08-09T13:35:21.425 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:21.419+0000 7f33dbfff640 1 --2- 172.21.15.43:0/2767625760 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f33c40051d0 0x7f33c400b980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-08-09T13:35:21.426 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:21.419+0000 7f33dbfff640 1 -- 172.21.15.43:0/2767625760 reap_dead start 2023-08-09T13:35:21.426 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:21.420+0000 7f33daffd640 1 --2- 172.21.15.43:0/2767625760 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f33c40047c0 0x7f33c4004c40 secure :-1 s=READY pgs=331 cs=0 l=1 rev1=1 crypto rx=0x7f33d0001750 tx=0x7f33c40057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7abe775ff22667e6 server_cookie=0 in_seq=0 out_seq=0 2023-08-09T13:35:21.426 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:21.420+0000 7f33da7fc640 1 -- 172.21.15.43:0/2767625760 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f33d00052d0 con 0x7f33c40047c0 2023-08-09T13:35:21.426 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:21.420+0000 7f33da7fc640 1 -- 172.21.15.43:0/2767625760 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f33d002e050 con 0x7f33c40047c0 2023-08-09T13:35:21.426 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:21.420+0000 7f33da7fc640 1 -- 172.21.15.43:0/2767625760 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f33d0003830 con 0x7f33c40047c0 2023-08-09T13:35:21.426 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:21.421+0000 7f33e0a77640 1 -- 172.21.15.43:0/2767625760 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f33c40047c0 msgr2=0x7f33c4004c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:35:21.426 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:21.421+0000 7f33e0a77640 1 --2- 172.21.15.43:0/2767625760 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f33c40047c0 0x7f33c4004c40 secure :-1 s=READY pgs=331 cs=0 l=1 rev1=1 crypto rx=0x7f33d0001750 tx=0x7f33c40057f0 comp rx=0 tx=0).stop 2023-08-09T13:35:21.426 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:21.421+0000 7f33e0a77640 1 -- 172.21.15.43:0/2767625760 shutdown_connections 2023-08-09T13:35:21.427 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:21.421+0000 7f33e0a77640 1 --2- 172.21.15.43:0/2767625760 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f33c40047c0 0x7f33c4004c40 unknown :-1 s=CLOSED pgs=331 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:35:21.427 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:21.421+0000 7f33e0a77640 1 -- 172.21.15.43:0/2767625760 >> 172.21.15.43:0/2767625760 conn(0x7f33dc0af9f0 msgr2=0x7f33dc0afdd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-09T13:35:21.427 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:21.421+0000 7f33e0a77640 1 -- 172.21.15.43:0/2767625760 shutdown_connections 2023-08-09T13:35:21.427 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:21.421+0000 7f33e0a77640 1 -- 172.21.15.43:0/2767625760 wait complete. 2023-08-09T13:35:21.429 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:21.423+0000 7f33e0a77640 1 Processor -- start 2023-08-09T13:35:21.430 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:21.423+0000 7f33e0a77640 1 -- start start 2023-08-09T13:35:21.430 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:21.424+0000 7f33e0a77640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f33c40047c0 0x7f33dc14f460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:35:21.430 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:21.424+0000 7f33e0a77640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f33dc0cd710 0x7f33dc0c4260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:35:21.430 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:21.424+0000 7f33e0a77640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f33dc0c1e40 0x7f33dc0c22c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:35:21.430 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:21.424+0000 7f33e0a77640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f33dc0c2800 con 0x7f33dc0cd710 2023-08-09T13:35:21.430 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:21.424+0000 7f33e0a77640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f33dc0cd2d0 con 0x7f33dc0c1e40 2023-08-09T13:35:21.430 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:21.424+0000 7f33e0a77640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f33dc0cc6c0 con 0x7f33c40047c0 2023-08-09T13:35:21.431 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:21.425+0000 7f33daffd640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f33dc0cd710 0x7f33dc0c4260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:35:21.431 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:21.425+0000 7f33daffd640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f33dc0cd710 0x7f33dc0c4260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:42554/0 (socket says 172.21.15.43:42554) 2023-08-09T13:35:21.431 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:21.425+0000 7f33daffd640 1 -- 172.21.15.43:0/3363908101 learned_addr learned my addr 172.21.15.43:0/3363908101 (peer_addr_for_me v2:172.21.15.43:0/0) 2023-08-09T13:35:21.431 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:21.425+0000 7f33dbfff640 1 --2- 172.21.15.43:0/3363908101 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f33dc0c1e40 0x7f33dc0c22c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:35:21.431 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:21.425+0000 7f33db7fe640 1 --2- 172.21.15.43:0/3363908101 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f33c40047c0 0x7f33dc14f460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:35:21.431 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:21.425+0000 7f33daffd640 1 -- 172.21.15.43:0/3363908101 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f33c40047c0 msgr2=0x7f33dc14f460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:35:21.431 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:21.425+0000 7f33daffd640 1 --2- 172.21.15.43:0/3363908101 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f33c40047c0 0x7f33dc14f460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:35:21.431 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:21.425+0000 7f33daffd640 1 -- 172.21.15.43:0/3363908101 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f33dc0c1e40 msgr2=0x7f33dc0c22c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:35:21.432 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:21.425+0000 7f33daffd640 1 --2- 172.21.15.43:0/3363908101 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f33dc0c1e40 0x7f33dc0c22c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:35:21.432 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:21.425+0000 7f33daffd640 1 -- 172.21.15.43:0/3363908101 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f33dc0cc9a0 con 0x7f33dc0cd710 2023-08-09T13:35:21.432 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:21.425+0000 7f33daffd640 1 --2- 172.21.15.43:0/3363908101 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f33dc0cd710 0x7f33dc0c4260 secure :-1 s=READY pgs=332 cs=0 l=1 rev1=1 crypto rx=0x7f33d002bc00 tx=0x7f33d0001120 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-09T13:35:21.432 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:21.426+0000 7f33c3fff640 1 -- 172.21.15.43:0/3363908101 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f33d00076c0 con 0x7f33dc0cd710 2023-08-09T13:35:21.432 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:21.426+0000 7f33c3fff640 1 -- 172.21.15.43:0/3363908101 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f33d0007a30 con 0x7f33dc0cd710 2023-08-09T13:35:21.432 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:21.426+0000 7f33c3fff640 1 -- 172.21.15.43:0/3363908101 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f33d0006be0 con 0x7f33dc0cd710 2023-08-09T13:35:21.432 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:21.426+0000 7f33e0a77640 1 -- 172.21.15.43:0/3363908101 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f33dc0ccc80 con 0x7f33dc0cd710 2023-08-09T13:35:21.432 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:21.426+0000 7f33e0a77640 1 -- 172.21.15.43:0/3363908101 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f33dc0c47a0 con 0x7f33dc0cd710 2023-08-09T13:35:21.433 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:21.427+0000 7f33e0a77640 1 -- 172.21.15.43:0/3363908101 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f33dc0cd410 con 0x7f33dc0cd710 2023-08-09T13:35:21.433 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:21.428+0000 7f33c3fff640 1 -- 172.21.15.43:0/3363908101 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 5) v1 ==== 88341+0+0 (secure 0 0 0) 0x7f33d0006d40 con 0x7f33dc0cd710 2023-08-09T13:35:21.433 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:21.429+0000 7f33c3fff640 1 --2- 172.21.15.43:0/3363908101 >> [v2:172.21.15.203:6832/830097474,v1:172.21.15.203:6833/830097474] conn(0x7f33c806a050 0x7f33c806c510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:35:21.434 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:21.429+0000 7f33c3fff640 1 -- 172.21.15.43:0/3363908101 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 6871+0+0 (secure 0 0 0) 0x7f33d00aa5f0 con 0x7f33dc0cd710 2023-08-09T13:35:21.434 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:21.430+0000 7f33db7fe640 1 --2- 172.21.15.43:0/3363908101 >> [v2:172.21.15.203:6832/830097474,v1:172.21.15.203:6833/830097474] conn(0x7f33c806a050 0x7f33c806c510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:35:21.434 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:21.431+0000 7f33db7fe640 1 --2- 172.21.15.43:0/3363908101 >> [v2:172.21.15.203:6832/830097474,v1:172.21.15.203:6833/830097474] conn(0x7f33c806a050 0x7f33c806c510 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7f33cc0099d0 tx=0x7f33cc008040 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-09T13:35:21.436 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:21.433+0000 7f33c3fff640 1 -- 172.21.15.43:0/3363908101 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178066 (secure 0 0 0) 0x7f33d00793a0 con 0x7f33dc0cd710 2023-08-09T13:35:21.630 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:21.628+0000 7f33e0a77640 1 -- 172.21.15.43:0/3363908101 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) v1 -- 0x7f33dc04e390 con 0x7f33dc0cd710 2023-08-09T13:35:21.631 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:21.629+0000 7f33c3fff640 1 -- 172.21.15.43:0/3363908101 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v26) v1 ==== 103+0+11 (secure 0 0 0) 0x7f33d0078f30 con 0x7f33dc0cd710 2023-08-09T13:35:21.631 INFO:teuthology.orchestra.run.smithi043.stdout:pg_num: 32 2023-08-09T13:35:21.634 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:21.631+0000 7f33e0a77640 1 -- 172.21.15.43:0/3363908101 >> [v2:172.21.15.203:6832/830097474,v1:172.21.15.203:6833/830097474] conn(0x7f33c806a050 msgr2=0x7f33c806c510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:35:21.634 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:21.631+0000 7f33e0a77640 1 --2- 172.21.15.43:0/3363908101 >> [v2:172.21.15.203:6832/830097474,v1:172.21.15.203:6833/830097474] conn(0x7f33c806a050 0x7f33c806c510 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7f33cc0099d0 tx=0x7f33cc008040 comp rx=0 tx=0).stop 2023-08-09T13:35:21.634 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:21.631+0000 7f33e0a77640 1 -- 172.21.15.43:0/3363908101 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f33dc0cd710 msgr2=0x7f33dc0c4260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:35:21.634 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:21.631+0000 7f33e0a77640 1 --2- 172.21.15.43:0/3363908101 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f33dc0cd710 0x7f33dc0c4260 secure :-1 s=READY pgs=332 cs=0 l=1 rev1=1 crypto rx=0x7f33d002bc00 tx=0x7f33d0001120 comp rx=0 tx=0).stop 2023-08-09T13:35:21.635 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:21.631+0000 7f33e0a77640 1 -- 172.21.15.43:0/3363908101 shutdown_connections 2023-08-09T13:35:21.635 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:21.631+0000 7f33e0a77640 1 --2- 172.21.15.43:0/3363908101 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f33dc0c1e40 0x7f33dc0c22c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:35:21.635 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:21.631+0000 7f33e0a77640 1 --2- 172.21.15.43:0/3363908101 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f33dc0cd710 0x7f33dc0c4260 unknown :-1 s=CLOSED pgs=332 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:35:21.635 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:21.631+0000 7f33e0a77640 1 --2- 172.21.15.43:0/3363908101 >> [v2:172.21.15.203:6832/830097474,v1:172.21.15.203:6833/830097474] conn(0x7f33c806a050 0x7f33c806c510 unknown :-1 s=CLOSED pgs=176 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:35:21.635 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:21.631+0000 7f33e0a77640 1 --2- 172.21.15.43:0/3363908101 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f33c40047c0 0x7f33dc14f460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:35:21.636 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:21.631+0000 7f33e0a77640 1 -- 172.21.15.43:0/3363908101 >> 172.21.15.43:0/3363908101 conn(0x7f33dc0af9f0 msgr2=0x7f33dc155c50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-09T13:35:21.636 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:21.632+0000 7f33e0a77640 1 -- 172.21.15.43:0/3363908101 shutdown_connections 2023-08-09T13:35:21.636 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:21.632+0000 7f33e0a77640 1 -- 172.21.15.43:0/3363908101 wait complete. 2023-08-09T13:35:21.654 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mon mon_allow_pool_delete True 2023-08-09T13:35:21.885 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:21.883+0000 7fdf0d953640 1 Processor -- start 2023-08-09T13:35:21.886 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:21.883+0000 7fdf0d953640 1 -- start start 2023-08-09T13:35:21.887 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:21.885+0000 7fdf0d953640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fdf0815f8a0 0x7fdf08070430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:35:21.888 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:21.885+0000 7fdf0d953640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7fdf08070970 0x7fdf080ade80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:35:21.888 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:21.885+0000 7fdf0d953640 1 -- --> v1:172.21.15.203:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fdf08070dd0 con 0x7fdf080ae3c0 2023-08-09T13:35:21.888 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:21.885+0000 7fdf0d953640 1 -- --> v2:172.21.15.43:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fdf0815fc80 con 0x7fdf08070970 2023-08-09T13:35:21.888 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:21.885+0000 7fdf0d953640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdf0806d800 con 0x7fdf0815f8a0 2023-08-09T13:35:21.889 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:21.885+0000 7fdefffff640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7fdf08070970 0x7fdf080ade80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:35:21.889 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:21.885+0000 7fdefffff640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7fdf08070970 0x7fdf080ade80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:42560/0 (socket says 172.21.15.43:42560) 2023-08-09T13:35:21.889 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:21.885+0000 7fdefffff640 1 -- 172.21.15.43:0/2686402605 learned_addr learned my addr 172.21.15.43:0/2686402605 (peer_addr_for_me v2:172.21.15.43:0/0) 2023-08-09T13:35:21.889 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:21.886+0000 7fdf0c951640 1 --2- 172.21.15.43:0/2686402605 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fdf0815f8a0 0x7fdf08070430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:35:21.889 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:21.886+0000 7fdefffff640 1 -- 172.21.15.43:0/2686402605 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fdf0815f8a0 msgr2=0x7fdf08070430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-09T13:35:21.890 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:21.886+0000 7fdefffff640 1 --2- 172.21.15.43:0/2686402605 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fdf0815f8a0 0x7fdf08070430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:35:21.890 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:21.886+0000 7fdefffff640 1 -- 172.21.15.43:0/2686402605 >> v1:172.21.15.203:6789/0 conn(0x7fdf080ae3c0 legacy=0x7fdf0815feb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-09T13:35:21.890 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:21.886+0000 7fdefffff640 1 -- 172.21.15.43:0/2686402605 --> v2:172.21.15.43:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdf0806dae0 con 0x7fdf08070970 2023-08-09T13:35:21.891 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:21.887+0000 7fdefffff640 1 --2- 172.21.15.43:0/2686402605 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fdf08070970 0x7fdf080ade80 secure :-1 s=READY pgs=333 cs=0 l=1 rev1=1 crypto rx=0x7fdef8009870 tx=0x7fdef802b860 comp rx=0 tx=0).ready entity=mon.0 client_cookie=99ed31345ec0801c server_cookie=0 in_seq=0 out_seq=0 2023-08-09T13:35:21.891 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:21.888+0000 7fdeff7fe640 1 -- 172.21.15.43:0/2686402605 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdef80317c0 con 0x7fdf08070970 2023-08-09T13:35:21.891 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:21.888+0000 7fdeff7fe640 1 -- 172.21.15.43:0/2686402605 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdef8031920 con 0x7fdf08070970 2023-08-09T13:35:21.892 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:21.888+0000 7fdeff7fe640 1 -- 172.21.15.43:0/2686402605 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdef8031cb0 con 0x7fdf08070970 2023-08-09T13:35:21.892 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:21.888+0000 7fdf0d953640 1 -- 172.21.15.43:0/2686402605 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fdf08070970 msgr2=0x7fdf080ade80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:35:21.892 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:21.888+0000 7fdf0d953640 1 --2- 172.21.15.43:0/2686402605 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fdf08070970 0x7fdf080ade80 secure :-1 s=READY pgs=333 cs=0 l=1 rev1=1 crypto rx=0x7fdef8009870 tx=0x7fdef802b860 comp rx=0 tx=0).stop 2023-08-09T13:35:21.892 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:21.889+0000 7fdf0d953640 1 -- 172.21.15.43:0/2686402605 shutdown_connections 2023-08-09T13:35:21.892 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:21.889+0000 7fdf0d953640 1 --2- 172.21.15.43:0/2686402605 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fdf08070970 0x7fdf080ade80 unknown :-1 s=CLOSED pgs=333 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:35:21.892 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:21.889+0000 7fdf0d953640 1 --2- 172.21.15.43:0/2686402605 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fdf0815f8a0 0x7fdf08070430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:35:21.892 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:21.889+0000 7fdf0d953640 1 -- 172.21.15.43:0/2686402605 >> 172.21.15.43:0/2686402605 conn(0x7fdf0806ce10 msgr2=0x7fdf080cef70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-09T13:35:21.892 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:21.889+0000 7fdf0d953640 1 -- 172.21.15.43:0/2686402605 shutdown_connections 2023-08-09T13:35:21.893 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:21.889+0000 7fdf0d953640 1 -- 172.21.15.43:0/2686402605 wait complete. 2023-08-09T13:35:21.893 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:21.891+0000 7fdf0d953640 1 Processor -- start 2023-08-09T13:35:21.893 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:21.891+0000 7fdf0d953640 1 -- start start 2023-08-09T13:35:21.894 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:21.892+0000 7fdf0d953640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fdf08070970 0x7fdf08160090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:35:21.894 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:21.892+0000 7fdf0d953640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fdf080ae3c0 0x7fdf081445e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:35:21.894 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:21.892+0000 7fdf0d953640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fdf0815f8a0 0x7fdf08144b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:35:21.895 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:21.892+0000 7fdf0d953640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdf08157640 con 0x7fdf0815f8a0 2023-08-09T13:35:21.895 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:21.892+0000 7fdf0d953640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdf08157360 con 0x7fdf08070970 2023-08-09T13:35:21.895 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:21.892+0000 7fdf0d953640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdf08152fb0 con 0x7fdf080ae3c0 2023-08-09T13:35:21.896 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:21.892+0000 7fdf0d152640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fdf0815f8a0 0x7fdf08144b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:35:21.896 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:21.892+0000 7fdf0d152640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fdf0815f8a0 0x7fdf08144b20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:42576/0 (socket says 172.21.15.43:42576) 2023-08-09T13:35:21.896 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:21.892+0000 7fdf0d152640 1 -- 172.21.15.43:0/2926955041 learned_addr learned my addr 172.21.15.43:0/2926955041 (peer_addr_for_me v2:172.21.15.43:0/0) 2023-08-09T13:35:21.897 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:21.893+0000 7fdf0c951640 1 --2- 172.21.15.43:0/2926955041 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fdf08070970 0x7fdf08160090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:35:21.897 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:21.893+0000 7fdefffff640 1 --2- 172.21.15.43:0/2926955041 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fdf080ae3c0 0x7fdf081445e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:35:21.897 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:21.893+0000 7fdf0d152640 1 -- 172.21.15.43:0/2926955041 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fdf080ae3c0 msgr2=0x7fdf081445e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:35:21.897 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:21.893+0000 7fdf0d152640 1 --2- 172.21.15.43:0/2926955041 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fdf080ae3c0 0x7fdf081445e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:35:21.897 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:21.893+0000 7fdf0d152640 1 -- 172.21.15.43:0/2926955041 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fdf08070970 msgr2=0x7fdf08160090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:35:21.897 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:21.893+0000 7fdf0d152640 1 --2- 172.21.15.43:0/2926955041 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fdf08070970 0x7fdf08160090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:35:21.897 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:21.893+0000 7fdf0d152640 1 -- 172.21.15.43:0/2926955041 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdef80094d0 con 0x7fdf0815f8a0 2023-08-09T13:35:21.897 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:21.893+0000 7fdf0d152640 1 --2- 172.21.15.43:0/2926955041 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fdf0815f8a0 0x7fdf08144b20 secure :-1 s=READY pgs=334 cs=0 l=1 rev1=1 crypto rx=0x7fdf0000c780 tx=0x7fdf0000cc40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-09T13:35:21.898 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:21.893+0000 7fdefd7fa640 1 -- 172.21.15.43:0/2926955041 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdf00004b60 con 0x7fdf0815f8a0 2023-08-09T13:35:21.898 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:21.893+0000 7fdefd7fa640 1 -- 172.21.15.43:0/2926955041 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdf0000ce90 con 0x7fdf0815f8a0 2023-08-09T13:35:21.898 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:21.893+0000 7fdefd7fa640 1 -- 172.21.15.43:0/2926955041 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdf0000a850 con 0x7fdf0815f8a0 2023-08-09T13:35:21.898 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:21.893+0000 7fdf0d953640 1 -- 172.21.15.43:0/2926955041 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdf08153290 con 0x7fdf0815f8a0 2023-08-09T13:35:21.898 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:21.894+0000 7fdf0d953640 1 -- 172.21.15.43:0/2926955041 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdf081605d0 con 0x7fdf0815f8a0 2023-08-09T13:35:21.899 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:21.895+0000 7fdf0d953640 1 -- 172.21.15.43:0/2926955041 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdf08152e10 con 0x7fdf0815f8a0 2023-08-09T13:35:21.899 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:21.896+0000 7fdefd7fa640 1 -- 172.21.15.43:0/2926955041 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 5) v1 ==== 88341+0+0 (secure 0 0 0) 0x7fdf0000a9b0 con 0x7fdf0815f8a0 2023-08-09T13:35:21.899 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:21.897+0000 7fdefd7fa640 1 --2- 172.21.15.43:0/2926955041 >> [v2:172.21.15.203:6832/830097474,v1:172.21.15.203:6833/830097474] conn(0x7fdef406a050 0x7fdef406c510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:35:21.899 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:21.897+0000 7fdefd7fa640 1 -- 172.21.15.43:0/2926955041 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 6871+0+0 (secure 0 0 0) 0x7fdf00088b30 con 0x7fdf0815f8a0 2023-08-09T13:35:21.900 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:21.897+0000 7fdf0c951640 1 --2- 172.21.15.43:0/2926955041 >> [v2:172.21.15.203:6832/830097474,v1:172.21.15.203:6833/830097474] conn(0x7fdef406a050 0x7fdef406c510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:35:21.901 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:21.899+0000 7fdf0c951640 1 --2- 172.21.15.43:0/2926955041 >> [v2:172.21.15.203:6832/830097474,v1:172.21.15.203:6833/830097474] conn(0x7fdef406a050 0x7fdef406c510 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7fdf04006fd0 tx=0x7fdf04008040 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-09T13:35:21.904 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:21.901+0000 7fdefd7fa640 1 -- 172.21.15.43:0/2926955041 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178066 (secure 0 0 0) 0x7fdf08152e10 con 0x7fdf0815f8a0 2023-08-09T13:35:22.101 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:22.099+0000 7fdf0d953640 1 -- 172.21.15.43:0/2926955041 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command([{prefix=config set, name=mon_allow_pool_delete}] v 0) v1 -- 0x7fdf081537b0 con 0x7fdf0815f8a0 2023-08-09T13:35:22.111 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:22.108+0000 7fdefd7fa640 1 -- 172.21.15.43:0/2926955041 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_allow_pool_delete}]=0 v10)=0 v10) v1 ==== 122+0+0 (secure 0 0 0) 0x7fdf081537b0 con 0x7fdf0815f8a0 2023-08-09T13:35:22.114 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:22.111+0000 7fdf0d953640 1 -- 172.21.15.43:0/2926955041 >> [v2:172.21.15.203:6832/830097474,v1:172.21.15.203:6833/830097474] conn(0x7fdef406a050 msgr2=0x7fdef406c510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:35:22.114 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:22.111+0000 7fdf0d953640 1 --2- 172.21.15.43:0/2926955041 >> [v2:172.21.15.203:6832/830097474,v1:172.21.15.203:6833/830097474] conn(0x7fdef406a050 0x7fdef406c510 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7fdf04006fd0 tx=0x7fdf04008040 comp rx=0 tx=0).stop 2023-08-09T13:35:22.114 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:22.111+0000 7fdf0d953640 1 -- 172.21.15.43:0/2926955041 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fdf0815f8a0 msgr2=0x7fdf08144b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:35:22.114 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:22.111+0000 7fdf0d953640 1 --2- 172.21.15.43:0/2926955041 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fdf0815f8a0 0x7fdf08144b20 secure :-1 s=READY pgs=334 cs=0 l=1 rev1=1 crypto rx=0x7fdf0000c780 tx=0x7fdf0000cc40 comp rx=0 tx=0).stop 2023-08-09T13:35:22.115 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:22.111+0000 7fdf0d953640 1 -- 172.21.15.43:0/2926955041 shutdown_connections 2023-08-09T13:35:22.115 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:22.111+0000 7fdf0d953640 1 --2- 172.21.15.43:0/2926955041 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fdf0815f8a0 0x7fdf08144b20 unknown :-1 s=CLOSED pgs=334 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:35:22.115 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:22.111+0000 7fdf0d953640 1 --2- 172.21.15.43:0/2926955041 >> [v2:172.21.15.203:6832/830097474,v1:172.21.15.203:6833/830097474] conn(0x7fdef406a050 0x7fdef406c510 unknown :-1 s=CLOSED pgs=177 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:35:22.115 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:22.111+0000 7fdf0d953640 1 --2- 172.21.15.43:0/2926955041 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fdf080ae3c0 0x7fdf081445e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:35:22.115 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:22.111+0000 7fdf0d953640 1 --2- 172.21.15.43:0/2926955041 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fdf08070970 0x7fdf08160090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:35:22.115 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:22.112+0000 7fdf0d953640 1 -- 172.21.15.43:0/2926955041 >> 172.21.15.43:0/2926955041 conn(0x7fdf0806ce10 msgr2=0x7fdf080b2380 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-09T13:35:22.115 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:22.112+0000 7fdf0d953640 1 -- 172.21.15.43:0/2926955041 shutdown_connections 2023-08-09T13:35:22.116 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:22.112+0000 7fdf0d953640 1 -- 172.21.15.43:0/2926955041 wait complete. 2023-08-09T13:35:22.135 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2023-08-09T13:35:22.136 DEBUG:teuthology.orchestra.run.smithi043:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2023-08-09T13:35:22.244 DEBUG:teuthology.orchestra.run.smithi203:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2023-08-09T13:35:22.369 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2023-08-09T13:35:22.370 DEBUG:teuthology.orchestra.run.smithi043:> set -ex 2023-08-09T13:35:22.370 DEBUG:teuthology.orchestra.run.smithi043:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-09T13:35:22.434 DEBUG:teuthology.orchestra.run.smithi043:> set -ex 2023-08-09T13:35:22.434 DEBUG:teuthology.orchestra.run.smithi043:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-09T13:35:22.494 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-08-09T13:35:22.717 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:22.715+0000 7f2d3cc8a640 1 Processor -- start 2023-08-09T13:35:22.718 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:22.715+0000 7f2d3cc8a640 1 -- start start 2023-08-09T13:35:22.719 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:22.717+0000 7f2d3cc8a640 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7f2d3815a350 0x7f2d380cfcf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:35:22.719 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:22.717+0000 7f2d3cc8a640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f2d380cb3d0 0x7f2d380b06c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:35:22.719 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:22.717+0000 7f2d3cc8a640 1 -- --> v1:172.21.15.43:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2d380d06e0 con 0x7f2d380d0230 2023-08-09T13:35:22.719 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:22.717+0000 7f2d3cc8a640 1 -- --> v2:172.21.15.43:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2d380cbfb0 con 0x7f2d380cb3d0 2023-08-09T13:35:22.720 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:22.717+0000 7f2d3cc8a640 1 -- --> v2:172.21.15.203:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2d380cb800 con 0x7f2d3815a350 2023-08-09T13:35:22.720 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:22.717+0000 7f2d37fff640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f2d380cb3d0 0x7f2d380b06c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:35:22.721 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:22.717+0000 7f2d37fff640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f2d380cb3d0 0x7f2d380b06c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:42580/0 (socket says 172.21.15.43:42580) 2023-08-09T13:35:22.721 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:22.717+0000 7f2d37fff640 1 -- 172.21.15.43:0/3505310815 learned_addr learned my addr 172.21.15.43:0/3505310815 (peer_addr_for_me v2:172.21.15.43:0/0) 2023-08-09T13:35:22.721 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:22.717+0000 7f2d377fe640 1 --2- 172.21.15.43:0/3505310815 >> v2:172.21.15.203:3300/0 conn(0x7f2d3815a350 0x7f2d380cfcf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:35:22.721 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:22.718+0000 7f2d37fff640 1 -- 172.21.15.43:0/3505310815 >> v2:172.21.15.203:3300/0 conn(0x7f2d3815a350 msgr2=0x7f2d380cfcf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-09T13:35:22.721 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:22.718+0000 7f2d37fff640 1 --2- 172.21.15.43:0/3505310815 >> v2:172.21.15.203:3300/0 conn(0x7f2d3815a350 0x7f2d380cfcf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:35:22.721 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:22.718+0000 7f2d37fff640 1 -- 172.21.15.43:0/3505310815 >> v1:172.21.15.43:6789/0 conn(0x7f2d380d0230 legacy=0x7f2d380cacc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:35:22.722 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:22.718+0000 7f2d37fff640 1 -- 172.21.15.43:0/3505310815 --> v2:172.21.15.43:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2d380cb9f0 con 0x7f2d380cb3d0 2023-08-09T13:35:22.722 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:22.718+0000 7f2d377fe640 1 --2- 172.21.15.43:0/3505310815 >> v2:172.21.15.203:3300/0 conn(0x7f2d3815a350 0x7f2d380cfcf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-08-09T13:35:22.723 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:22.719+0000 7f2d37fff640 1 --2- 172.21.15.43:0/3505310815 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f2d380cb3d0 0x7f2d380b06c0 secure :-1 s=READY pgs=336 cs=0 l=1 rev1=1 crypto rx=0x7f2d380d0df0 tx=0x7f2d3002bee0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=78032146caf0b6df server_cookie=0 in_seq=0 out_seq=0 2023-08-09T13:35:22.723 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:22.719+0000 7f2d367fc640 1 -- 172.21.15.43:0/3505310815 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2d3002ca80 con 0x7f2d380cb3d0 2023-08-09T13:35:22.723 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:22.719+0000 7f2d367fc640 1 -- 172.21.15.43:0/3505310815 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2d3002cbe0 con 0x7f2d380cb3d0 2023-08-09T13:35:22.723 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:22.719+0000 7f2d367fc640 1 -- 172.21.15.43:0/3505310815 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2d300343f0 con 0x7f2d380cb3d0 2023-08-09T13:35:22.723 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:22.720+0000 7f2d3cc8a640 1 -- 172.21.15.43:0/3505310815 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f2d380cb3d0 msgr2=0x7f2d380b06c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:35:22.723 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:22.720+0000 7f2d3cc8a640 1 --2- 172.21.15.43:0/3505310815 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f2d380cb3d0 0x7f2d380b06c0 secure :-1 s=READY pgs=336 cs=0 l=1 rev1=1 crypto rx=0x7f2d380d0df0 tx=0x7f2d3002bee0 comp rx=0 tx=0).stop 2023-08-09T13:35:22.724 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:22.720+0000 7f2d3cc8a640 1 -- 172.21.15.43:0/3505310815 shutdown_connections 2023-08-09T13:35:22.724 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:22.720+0000 7f2d3cc8a640 1 --2- 172.21.15.43:0/3505310815 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f2d380cb3d0 0x7f2d380b06c0 unknown :-1 s=CLOSED pgs=336 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:35:22.724 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:22.720+0000 7f2d3cc8a640 1 --2- 172.21.15.43:0/3505310815 >> v2:172.21.15.203:3300/0 conn(0x7f2d3815a350 0x7f2d380cfcf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:35:22.724 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:22.720+0000 7f2d3cc8a640 1 -- 172.21.15.43:0/3505310815 >> 172.21.15.43:0/3505310815 conn(0x7f2d38089fe0 msgr2=0x7f2d38099450 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-09T13:35:22.724 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:22.720+0000 7f2d3cc8a640 1 -- 172.21.15.43:0/3505310815 shutdown_connections 2023-08-09T13:35:22.724 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:22.720+0000 7f2d3cc8a640 1 -- 172.21.15.43:0/3505310815 wait complete. 2023-08-09T13:35:22.725 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:22.722+0000 7f2d3cc8a640 1 Processor -- start 2023-08-09T13:35:22.725 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:22.722+0000 7f2d3cc8a640 1 -- start start 2023-08-09T13:35:22.726 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:22.723+0000 7f2d3cc8a640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f2d380cb3d0 0x7f2d380c5cb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:35:22.726 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:22.723+0000 7f2d3cc8a640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f2d380d0230 0x7f2d3815d240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:35:22.726 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:22.723+0000 7f2d3cc8a640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f2d3815a350 0x7f2d380adf30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:35:22.727 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:22.723+0000 7f2d3cc8a640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2d380acc40 con 0x7f2d3815a350 2023-08-09T13:35:22.727 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:22.723+0000 7f2d3cc8a640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2d380adaa0 con 0x7f2d380cb3d0 2023-08-09T13:35:22.727 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:22.723+0000 7f2d3cc8a640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2d380ace90 con 0x7f2d380d0230 2023-08-09T13:35:22.727 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:22.723+0000 7f2d37fff640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f2d3815a350 0x7f2d380adf30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:35:22.727 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:22.723+0000 7f2d37fff640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f2d3815a350 0x7f2d380adf30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:42594/0 (socket says 172.21.15.43:42594) 2023-08-09T13:35:22.727 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:22.723+0000 7f2d37fff640 1 -- 172.21.15.43:0/2830619190 learned_addr learned my addr 172.21.15.43:0/2830619190 (peer_addr_for_me v2:172.21.15.43:0/0) 2023-08-09T13:35:22.727 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:22.724+0000 7f2d37fff640 1 -- 172.21.15.43:0/2830619190 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f2d380d0230 msgr2=0x7f2d3815d240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:35:22.728 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:22.724+0000 7f2d37fff640 1 --2- 172.21.15.43:0/2830619190 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f2d380d0230 0x7f2d3815d240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:35:22.728 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:22.724+0000 7f2d377fe640 1 --2- 172.21.15.43:0/2830619190 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f2d380cb3d0 0x7f2d380c5cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:35:22.728 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:22.724+0000 7f2d37fff640 1 -- 172.21.15.43:0/2830619190 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f2d380cb3d0 msgr2=0x7f2d380c5cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:35:22.728 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:22.724+0000 7f2d37fff640 1 --2- 172.21.15.43:0/2830619190 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f2d380cb3d0 0x7f2d380c5cb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:35:22.728 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:22.724+0000 7f2d37fff640 1 -- 172.21.15.43:0/2830619190 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2d30009b30 con 0x7f2d3815a350 2023-08-09T13:35:22.729 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:22.724+0000 7f2d37fff640 1 --2- 172.21.15.43:0/2830619190 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f2d3815a350 0x7f2d380adf30 secure :-1 s=READY pgs=337 cs=0 l=1 rev1=1 crypto rx=0x7f2d30009f40 tx=0x7f2d30004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-09T13:35:22.729 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:22.724+0000 7f2d2ffff640 1 -- 172.21.15.43:0/2830619190 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2d30009510 con 0x7f2d3815a350 2023-08-09T13:35:22.729 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:22.724+0000 7f2d2ffff640 1 -- 172.21.15.43:0/2830619190 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2d3003e450 con 0x7f2d3815a350 2023-08-09T13:35:22.730 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:22.724+0000 7f2d2ffff640 1 -- 172.21.15.43:0/2830619190 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2d3003e800 con 0x7f2d3815a350 2023-08-09T13:35:22.730 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:22.724+0000 7f2d3cc8a640 1 -- 172.21.15.43:0/2830619190 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2d380ad170 con 0x7f2d3815a350 2023-08-09T13:35:22.730 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:22.725+0000 7f2d3cc8a640 1 -- 172.21.15.43:0/2830619190 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2d380ad730 con 0x7f2d3815a350 2023-08-09T13:35:22.730 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:22.726+0000 7f2d3cc8a640 1 -- 172.21.15.43:0/2830619190 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2d380adc30 con 0x7f2d3815a350 2023-08-09T13:35:22.730 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:22.727+0000 7f2d2ffff640 1 -- 172.21.15.43:0/2830619190 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 5) v1 ==== 88341+0+0 (secure 0 0 0) 0x7f2d3002cd50 con 0x7f2d3815a350 2023-08-09T13:35:22.731 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:22.728+0000 7f2d2ffff640 1 --2- 172.21.15.43:0/2830619190 >> [v2:172.21.15.203:6832/830097474,v1:172.21.15.203:6833/830097474] conn(0x7f2d1806a080 0x7f2d1806c540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:35:22.731 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:22.728+0000 7f2d2ffff640 1 -- 172.21.15.43:0/2830619190 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 6871+0+0 (secure 0 0 0) 0x7f2d300ab480 con 0x7f2d3815a350 2023-08-09T13:35:22.731 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:22.728+0000 7f2d377fe640 1 --2- 172.21.15.43:0/2830619190 >> [v2:172.21.15.203:6832/830097474,v1:172.21.15.203:6833/830097474] conn(0x7f2d1806a080 0x7f2d1806c540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:35:22.732 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:22.730+0000 7f2d377fe640 1 --2- 172.21.15.43:0/2830619190 >> [v2:172.21.15.203:6832/830097474,v1:172.21.15.203:6833/830097474] conn(0x7f2d1806a080 0x7f2d1806c540 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7f2d2800b840 tx=0x7f2d28008040 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-09T13:35:22.734 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:22.732+0000 7f2d2ffff640 1 -- 172.21.15.43:0/2830619190 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178066 (secure 0 0 0) 0x7f2d300768e0 con 0x7f2d3815a350 2023-08-09T13:35:22.930 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:22.928+0000 7f2d3cc8a640 1 -- 172.21.15.43:0/2830619190 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f2d380739f0 con 0x7f2d3815a350 2023-08-09T13:35:22.934 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:22.931+0000 7f2d2ffff640 1 -- 172.21.15.43:0/2830619190 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v26) v1 ==== 74+0+20146 (secure 0 0 0) 0x7f2d30079f00 con 0x7f2d3815a350 2023-08-09T13:35:22.934 INFO:teuthology.orchestra.run.smithi043.stdout: 2023-08-09T13:35:22.935 INFO:teuthology.orchestra.run.smithi043.stdout:{"epoch":26,"fsid":"daac87ee-1337-4250-a955-aea4e408a39b","created":"2023-08-09T13:33:41.701198+0000","modified":"2023-08-09T13:34:48.711911+0000","last_up_change":"2023-08-09T13:33:53.783453+0000","last_in_change":"2023-08-09T13:33:45.148904+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":5,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":"rbd","create_time":"2023-08-09T13:33:54.365137+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":".mgr","create_time":"2023-08-09T13:33:54.831803+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_metadata","create_time":"2023-08-09T13:34:00.207974+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data","create_time":"2023-08-09T13:34:01.614765+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":5,"pool_name":"cephfs_data_ec","create_time":"2023-08-09T13:34:05.239583+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"24","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"cbd49ecd-e896-4fbd-864e-5c37a829b165","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6800","nonce":502667335},{"type":"v1","addr":"172.21.15.43:6801","nonce":502667335}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6802","nonce":502667335},{"type":"v1","addr":"172.21.15.43:6803","nonce":502667335}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6806","nonce":502667335},{"type":"v1","addr":"172.21.15.43:6807","nonce":502667335}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6804","nonce":502667335},{"type":"v1","addr":"172.21.15.43:6805","nonce":502667335}]},"public_addr":"172.21.15.43:6801/502667335","cluster_addr":"172.21.15.43:6803/502667335","heartbeat_back_addr":"172.21.15.43:6807/502667335","heartbeat_front_addr":"172.21.15.43:6805/502667335","state":["exists","up"]},{"osd":1,"uuid":"75c7e0fe-1ec7-4fe8-a2f2-3661a65b8500","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6808","nonce":3289134610},{"type":"v1","addr":"172.21.15.43:6809","nonce":3289134610}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6810","nonce":3289134610},{"type":"v1","addr":"172.21.15.43:6812","nonce":3289134610}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6821","nonce":3289134610},{"type":"v1","addr":"172.21.15.43:6824","nonce":3289134610}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6815","nonce":3289134610},{"type":"v1","addr":"172.21.15.43:6818","nonce":3289134610}]},"public_addr":"172.21.15.43:6809/3289134610","cluster_addr":"172.21.15.43:6812/3289134610","heartbeat_back_addr":"172.21.15.43:6824/3289134610","heartbeat_front_addr":"172.21.15.43:6818/3289134610","state":["exists","up"]},{"osd":2,"uuid":"bb5c41e9-7f5f-4125-9b01-3bb8bfc2ea83","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6811","nonce":4079709736},{"type":"v1","addr":"172.21.15.43:6814","nonce":4079709736}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6817","nonce":4079709736},{"type":"v1","addr":"172.21.15.43:6820","nonce":4079709736}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6829","nonce":4079709736},{"type":"v1","addr":"172.21.15.43:6831","nonce":4079709736}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6823","nonce":4079709736},{"type":"v1","addr":"172.21.15.43:6826","nonce":4079709736}]},"public_addr":"172.21.15.43:6814/4079709736","cluster_addr":"172.21.15.43:6820/4079709736","heartbeat_back_addr":"172.21.15.43:6831/4079709736","heartbeat_front_addr":"172.21.15.43:6826/4079709736","state":["exists","up"]},{"osd":3,"uuid":"bfe4c8b4-694a-465c-87e6-2b2aab852613","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6813","nonce":1709538759},{"type":"v1","addr":"172.21.15.43:6816","nonce":1709538759}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6819","nonce":1709538759},{"type":"v1","addr":"172.21.15.43:6822","nonce":1709538759}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6828","nonce":1709538759},{"type":"v1","addr":"172.21.15.43:6830","nonce":1709538759}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6825","nonce":1709538759},{"type":"v1","addr":"172.21.15.43:6827","nonce":1709538759}]},"public_addr":"172.21.15.43:6816/1709538759","cluster_addr":"172.21.15.43:6822/1709538759","heartbeat_back_addr":"172.21.15.43:6830/1709538759","heartbeat_front_addr":"172.21.15.43:6827/1709538759","state":["exists","up"]},{"osd":4,"uuid":"fa2e8a77-aba1-4756-853e-0d174a5bdeba","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6800","nonce":4196866481},{"type":"v1","addr":"172.21.15.203:6801","nonce":4196866481}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6802","nonce":4196866481},{"type":"v1","addr":"172.21.15.203:6803","nonce":4196866481}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6806","nonce":4196866481},{"type":"v1","addr":"172.21.15.203:6807","nonce":4196866481}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6804","nonce":4196866481},{"type":"v1","addr":"172.21.15.203:6805","nonce":4196866481}]},"public_addr":"172.21.15.203:6801/4196866481","cluster_addr":"172.21.15.203:6803/4196866481","heartbeat_back_addr":"172.21.15.203:6807/4196866481","heartbeat_front_addr":"172.21.15.203:6805/4196866481","state":["exists","up"]},{"osd":5,"uuid":"81811e4b-4d24-4f51-a82c-f1d700a2a00d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6816","nonce":3059290081},{"type":"v1","addr":"172.21.15.203:6817","nonce":3059290081}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6818","nonce":3059290081},{"type":"v1","addr":"172.21.15.203:6819","nonce":3059290081}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6822","nonce":3059290081},{"type":"v1","addr":"172.21.15.203:6823","nonce":3059290081}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6820","nonce":3059290081},{"type":"v1","addr":"172.21.15.203:6821","nonce":3059290081}]},"public_addr":"172.21.15.203:6817/3059290081","cluster_addr":"172.21.15.203:6819/3059290081","heartbeat_back_addr":"172.21.15.203:6823/3059290081","heartbeat_front_addr":"172.21.15.203:6821/3059290081","state":["exists","up"]},{"osd":6,"uuid":"307bac4d-6a5b-443d-af75-56d695d7674f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6824","nonce":512155569},{"type":"v1","addr":"172.21.15.203:6825","nonce":512155569}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6826","nonce":512155569},{"type":"v1","addr":"172.21.15.203:6827","nonce":512155569}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6830","nonce":512155569},{"type":"v1","addr":"172.21.15.203:6831","nonce":512155569}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6828","nonce":512155569},{"type":"v1","addr":"172.21.15.203:6829","nonce":512155569}]},"public_addr":"172.21.15.203:6825/512155569","cluster_addr":"172.21.15.203:6827/512155569","heartbeat_back_addr":"172.21.15.203:6831/512155569","heartbeat_front_addr":"172.21.15.203:6829/512155569","state":["exists","up"]},{"osd":7,"uuid":"a95dc7ca-9f1b-4295-af70-68a22a2d7d5e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6808","nonce":3331450438},{"type":"v1","addr":"172.21.15.203:6809","nonce":3331450438}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6810","nonce":3331450438},{"type":"v1","addr":"172.21.15.203:6811","nonce":3331450438}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6814","nonce":3331450438},{"type":"v1","addr":"172.21.15.203:6815","nonce":3331450438}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6812","nonce":3331450438},{"type":"v1","addr":"172.21.15.203:6813","nonce":3331450438}]},"public_addr":"172.21.15.203:6809/3331450438","cluster_addr":"172.21.15.203:6811/3331450438","heartbeat_back_addr":"172.21.15.203:6815/3331450438","heartbeat_front_addr":"172.21.15.203:6813/3331450438","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-09T13:33:50.547652+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-09T13:33:50.589469+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-09T13:33:50.561724+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-09T13:33:50.599650+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-09T13:33:50.411855+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-09T13:33:50.219681+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-09T13:33:50.504664+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-09T13:33:50.441248+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"4.14","mappings":[{"from":1,"to":5}]},{"pgid":"4.29","mappings":[{"from":3,"to":2}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-08-09T13:35:22.937 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:22.934+0000 7f2d3cc8a640 1 -- 172.21.15.43:0/2830619190 >> [v2:172.21.15.203:6832/830097474,v1:172.21.15.203:6833/830097474] conn(0x7f2d1806a080 msgr2=0x7f2d1806c540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:35:22.937 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:22.934+0000 7f2d3cc8a640 1 --2- 172.21.15.43:0/2830619190 >> [v2:172.21.15.203:6832/830097474,v1:172.21.15.203:6833/830097474] conn(0x7f2d1806a080 0x7f2d1806c540 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7f2d2800b840 tx=0x7f2d28008040 comp rx=0 tx=0).stop 2023-08-09T13:35:22.937 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:22.934+0000 7f2d3cc8a640 1 -- 172.21.15.43:0/2830619190 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f2d3815a350 msgr2=0x7f2d380adf30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:35:22.937 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:22.934+0000 7f2d3cc8a640 1 --2- 172.21.15.43:0/2830619190 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f2d3815a350 0x7f2d380adf30 secure :-1 s=READY pgs=337 cs=0 l=1 rev1=1 crypto rx=0x7f2d30009f40 tx=0x7f2d30004a40 comp rx=0 tx=0).stop 2023-08-09T13:35:22.937 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:22.934+0000 7f2d3cc8a640 1 -- 172.21.15.43:0/2830619190 shutdown_connections 2023-08-09T13:35:22.937 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:22.934+0000 7f2d3cc8a640 1 --2- 172.21.15.43:0/2830619190 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f2d3815a350 0x7f2d380adf30 unknown :-1 s=CLOSED pgs=337 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:35:22.938 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:22.934+0000 7f2d3cc8a640 1 --2- 172.21.15.43:0/2830619190 >> [v2:172.21.15.203:6832/830097474,v1:172.21.15.203:6833/830097474] conn(0x7f2d1806a080 0x7f2d1806c540 unknown :-1 s=CLOSED pgs=178 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:35:22.938 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:22.934+0000 7f2d3cc8a640 1 --2- 172.21.15.43:0/2830619190 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f2d380d0230 0x7f2d3815d240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:35:22.938 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:22.934+0000 7f2d3cc8a640 1 --2- 172.21.15.43:0/2830619190 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f2d380cb3d0 0x7f2d380c5cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:35:22.938 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:22.934+0000 7f2d3cc8a640 1 -- 172.21.15.43:0/2830619190 >> 172.21.15.43:0/2830619190 conn(0x7f2d38089fe0 msgr2=0x7f2d3806d710 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-09T13:35:22.938 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:22.934+0000 7f2d3cc8a640 1 -- 172.21.15.43:0/2830619190 shutdown_connections 2023-08-09T13:35:22.938 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:22.934+0000 7f2d3cc8a640 1 -- 172.21.15.43:0/2830619190 wait complete. 2023-08-09T13:35:22.957 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': 'rbd', 'create_time': '2023-08-09T13:33:54.365137+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}, 'read_balance': {'score_acting': 2, 'score_stable': 2, 'optimal_score': 1, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': '.mgr', 'create_time': '2023-08-09T13:33:54.831803+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_metadata', 'create_time': '2023-08-09T13:34:00.207974+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data', 'create_time': '2023-08-09T13:34:01.614765+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 2, 'score_stable': 2, 'optimal_score': 1, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 5, 'pool_name': 'cephfs_data_ec', 'create_time': '2023-08-09T13:34:05.239583+0000', 'flags': 5, 'flags_names': 'hashpspool,ec_overwrites', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '24', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2023-08-09T13:35:22.958 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get rbd pg_num 2023-08-09T13:35:23.189 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:23.186+0000 7f28493a4640 1 Processor -- start 2023-08-09T13:35:23.189 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:23.186+0000 7f28493a4640 1 -- start start 2023-08-09T13:35:23.191 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:23.188+0000 7f28493a4640 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7f2844164560 0x7f28440c8420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:35:23.191 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:23.188+0000 7f28493a4640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f28440c1020 0x7f2844162360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:35:23.191 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:23.188+0000 7f28493a4640 1 -- --> v1:172.21.15.203:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f28440cc3b0 con 0x7f2844164940 2023-08-09T13:35:23.191 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:23.188+0000 7f28493a4640 1 -- --> v2:172.21.15.43:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f28441654e0 con 0x7f28440c1020 2023-08-09T13:35:23.191 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:23.188+0000 7f28493a4640 1 -- --> v2:172.21.15.203:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f28440ca6d0 con 0x7f2844164560 2023-08-09T13:35:23.192 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:23.189+0000 7f283b7fe640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f28440c1020 0x7f2844162360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:35:23.192 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:23.189+0000 7f283b7fe640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f28440c1020 0x7f2844162360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:42604/0 (socket says 172.21.15.43:42604) 2023-08-09T13:35:23.192 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:23.189+0000 7f283b7fe640 1 -- 172.21.15.43:0/3818569948 learned_addr learned my addr 172.21.15.43:0/3818569948 (peer_addr_for_me v2:172.21.15.43:0/0) 2023-08-09T13:35:23.192 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:23.189+0000 7f2843fff640 1 --2- 172.21.15.43:0/3818569948 >> v2:172.21.15.203:3300/0 conn(0x7f2844164560 0x7f28440c8420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:35:23.193 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:23.189+0000 7f283b7fe640 1 -- 172.21.15.43:0/3818569948 >> v2:172.21.15.203:3300/0 conn(0x7f2844164560 msgr2=0x7f28440c8420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-09T13:35:23.193 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:23.189+0000 7f283b7fe640 1 --2- 172.21.15.43:0/3818569948 >> v2:172.21.15.203:3300/0 conn(0x7f2844164560 0x7f28440c8420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:35:23.193 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:23.189+0000 7f283b7fe640 1 -- 172.21.15.43:0/3818569948 >> v1:172.21.15.203:6789/0 conn(0x7f2844164940 legacy=0x7f2844164dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:35:23.193 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:23.189+0000 7f283b7fe640 1 -- 172.21.15.43:0/3818569948 --> v2:172.21.15.43:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f28440af2e0 con 0x7f28440c1020 2023-08-09T13:35:23.194 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:23.191+0000 7f283b7fe640 1 --2- 172.21.15.43:0/3818569948 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f28440c1020 0x7f2844162360 secure :-1 s=READY pgs=338 cs=0 l=1 rev1=1 crypto rx=0x7f28440cb920 tx=0x7f282c02b880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4525d4c658d220c2 server_cookie=0 in_seq=0 out_seq=0 2023-08-09T13:35:23.194 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:23.191+0000 7f28437fe640 1 -- 172.21.15.43:0/3818569948 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f282c0317e0 con 0x7f28440c1020 2023-08-09T13:35:23.194 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:23.191+0000 7f28437fe640 1 -- 172.21.15.43:0/3818569948 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f282c031940 con 0x7f28440c1020 2023-08-09T13:35:23.194 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:23.191+0000 7f28437fe640 1 -- 172.21.15.43:0/3818569948 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f282c031cd0 con 0x7f28440c1020 2023-08-09T13:35:23.194 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:23.192+0000 7f28493a4640 1 -- 172.21.15.43:0/3818569948 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f28440c1020 msgr2=0x7f2844162360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:35:23.194 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:23.192+0000 7f28493a4640 1 --2- 172.21.15.43:0/3818569948 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f28440c1020 0x7f2844162360 secure :-1 s=READY pgs=338 cs=0 l=1 rev1=1 crypto rx=0x7f28440cb920 tx=0x7f282c02b880 comp rx=0 tx=0).stop 2023-08-09T13:35:23.195 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:23.192+0000 7f28493a4640 1 -- 172.21.15.43:0/3818569948 shutdown_connections 2023-08-09T13:35:23.195 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:23.192+0000 7f28493a4640 1 --2- 172.21.15.43:0/3818569948 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f28440c1020 0x7f2844162360 unknown :-1 s=CLOSED pgs=338 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:35:23.195 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:23.192+0000 7f28493a4640 1 --2- 172.21.15.43:0/3818569948 >> v2:172.21.15.203:3300/0 conn(0x7f2844164560 0x7f28440c8420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:35:23.196 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:23.192+0000 7f28493a4640 1 -- 172.21.15.43:0/3818569948 >> 172.21.15.43:0/3818569948 conn(0x7f28440cddd0 msgr2=0x7f28440ce1e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-09T13:35:23.196 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:23.192+0000 7f28493a4640 1 -- 172.21.15.43:0/3818569948 shutdown_connections 2023-08-09T13:35:23.196 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:23.192+0000 7f28493a4640 1 -- 172.21.15.43:0/3818569948 wait complete. 2023-08-09T13:35:23.196 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:23.194+0000 7f28493a4640 1 Processor -- start 2023-08-09T13:35:23.197 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:23.194+0000 7f28493a4640 1 -- start start 2023-08-09T13:35:23.197 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:23.195+0000 7f28493a4640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f28440c1020 0x7f2844150370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:35:23.198 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:23.195+0000 7f28493a4640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f2844164560 0x7f28440c32e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:35:23.198 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:23.195+0000 7f28493a4640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f2844164940 0x7f28440c3820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:35:23.198 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:23.195+0000 7f28493a4640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f28440b1a50 con 0x7f2844164560 2023-08-09T13:35:23.198 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:23.195+0000 7f28493a4640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f28441508b0 con 0x7f2844164940 2023-08-09T13:35:23.198 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:23.195+0000 7f28493a4640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f28441ec0c0 con 0x7f28440c1020 2023-08-09T13:35:23.198 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:23.195+0000 7f283b7fe640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f2844164560 0x7f28440c32e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:35:23.199 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:23.195+0000 7f283b7fe640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f2844164560 0x7f28440c32e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:42612/0 (socket says 172.21.15.43:42612) 2023-08-09T13:35:23.199 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:23.195+0000 7f283b7fe640 1 -- 172.21.15.43:0/3850709029 learned_addr learned my addr 172.21.15.43:0/3850709029 (peer_addr_for_me v2:172.21.15.43:0/0) 2023-08-09T13:35:23.199 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:23.195+0000 7f2843fff640 1 --2- 172.21.15.43:0/3850709029 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f28440c1020 0x7f2844150370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:35:23.200 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:23.196+0000 7f2848ba3640 1 --2- 172.21.15.43:0/3850709029 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f2844164940 0x7f28440c3820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:35:23.200 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:23.196+0000 7f283b7fe640 1 -- 172.21.15.43:0/3850709029 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f28440c1020 msgr2=0x7f2844150370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:35:23.200 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:23.196+0000 7f283b7fe640 1 --2- 172.21.15.43:0/3850709029 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f28440c1020 0x7f2844150370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:35:23.201 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:23.196+0000 7f283b7fe640 1 -- 172.21.15.43:0/3850709029 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f2844164940 msgr2=0x7f28440c3820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:35:23.201 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:23.196+0000 7f283b7fe640 1 --2- 172.21.15.43:0/3850709029 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f2844164940 0x7f28440c3820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:35:23.201 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:23.196+0000 7f283b7fe640 1 -- 172.21.15.43:0/3850709029 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f282c0094d0 con 0x7f2844164560 2023-08-09T13:35:23.201 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:23.196+0000 7f2848ba3640 1 --2- 172.21.15.43:0/3850709029 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f2844164940 0x7f28440c3820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-08-09T13:35:23.201 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:23.196+0000 7f283b7fe640 1 --2- 172.21.15.43:0/3850709029 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f2844164560 0x7f28440c32e0 secure :-1 s=READY pgs=339 cs=0 l=1 rev1=1 crypto rx=0x7f282c000c00 tx=0x7f282c0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-09T13:35:23.201 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:23.196+0000 7f28417fa640 1 -- 172.21.15.43:0/3850709029 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f282c032880 con 0x7f2844164560 2023-08-09T13:35:23.201 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:23.196+0000 7f28417fa640 1 -- 172.21.15.43:0/3850709029 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f282c032a20 con 0x7f2844164560 2023-08-09T13:35:23.202 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:23.196+0000 7f28417fa640 1 -- 172.21.15.43:0/3850709029 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f282c032dd0 con 0x7f2844164560 2023-08-09T13:35:23.202 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:23.196+0000 7f28493a4640 1 -- 172.21.15.43:0/3850709029 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f28440b13d0 con 0x7f2844164560 2023-08-09T13:35:23.202 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:23.197+0000 7f28493a4640 1 -- 172.21.15.43:0/3850709029 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f28441ebde0 con 0x7f2844164560 2023-08-09T13:35:23.202 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:23.198+0000 7f28493a4640 1 -- 172.21.15.43:0/3850709029 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f28440b1b90 con 0x7f2844164560 2023-08-09T13:35:23.203 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:23.199+0000 7f28417fa640 1 -- 172.21.15.43:0/3850709029 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 5) v1 ==== 88341+0+0 (secure 0 0 0) 0x7f282c04e450 con 0x7f2844164560 2023-08-09T13:35:23.203 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:23.200+0000 7f28417fa640 1 --2- 172.21.15.43:0/3850709029 >> [v2:172.21.15.203:6832/830097474,v1:172.21.15.203:6833/830097474] conn(0x7f282406a050 0x7f282406c510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:35:23.203 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:23.200+0000 7f28417fa640 1 -- 172.21.15.43:0/3850709029 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 6871+0+0 (secure 0 0 0) 0x7f282c0aa9f0 con 0x7f2844164560 2023-08-09T13:35:23.203 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:23.200+0000 7f2843fff640 1 --2- 172.21.15.43:0/3850709029 >> [v2:172.21.15.203:6832/830097474,v1:172.21.15.203:6833/830097474] conn(0x7f282406a050 0x7f282406c510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:35:23.204 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:23.201+0000 7f2843fff640 1 --2- 172.21.15.43:0/3850709029 >> [v2:172.21.15.203:6832/830097474,v1:172.21.15.203:6833/830097474] conn(0x7f282406a050 0x7f282406c510 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7f2834009a30 tx=0x7f2834008040 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-09T13:35:23.206 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:23.204+0000 7f28417fa640 1 -- 172.21.15.43:0/3850709029 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178066 (secure 0 0 0) 0x7f28440b1b90 con 0x7f2844164560 2023-08-09T13:35:23.402 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:23.400+0000 7f28493a4640 1 -- 172.21.15.43:0/3850709029 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"} v 0) v1 -- 0x7f28440b1b90 con 0x7f2844164560 2023-08-09T13:35:23.403 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:23.400+0000 7f28417fa640 1 -- 172.21.15.43:0/3850709029 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"}]=0 v26) v1 ==== 92+0+10 (secure 0 0 0) 0x7f28440b1b90 con 0x7f2844164560 2023-08-09T13:35:23.403 INFO:teuthology.orchestra.run.smithi043.stdout:pg_num: 8 2023-08-09T13:35:23.405 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:23.403+0000 7f28493a4640 1 -- 172.21.15.43:0/3850709029 >> [v2:172.21.15.203:6832/830097474,v1:172.21.15.203:6833/830097474] conn(0x7f282406a050 msgr2=0x7f282406c510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:35:23.405 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:23.403+0000 7f28493a4640 1 --2- 172.21.15.43:0/3850709029 >> [v2:172.21.15.203:6832/830097474,v1:172.21.15.203:6833/830097474] conn(0x7f282406a050 0x7f282406c510 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7f2834009a30 tx=0x7f2834008040 comp rx=0 tx=0).stop 2023-08-09T13:35:23.405 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:23.403+0000 7f28493a4640 1 -- 172.21.15.43:0/3850709029 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f2844164560 msgr2=0x7f28440c32e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:35:23.406 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:23.403+0000 7f28493a4640 1 --2- 172.21.15.43:0/3850709029 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f2844164560 0x7f28440c32e0 secure :-1 s=READY pgs=339 cs=0 l=1 rev1=1 crypto rx=0x7f282c000c00 tx=0x7f282c0046c0 comp rx=0 tx=0).stop 2023-08-09T13:35:23.406 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:23.403+0000 7f28493a4640 1 -- 172.21.15.43:0/3850709029 shutdown_connections 2023-08-09T13:35:23.406 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:23.403+0000 7f28493a4640 1 --2- 172.21.15.43:0/3850709029 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f2844164940 0x7f28440c3820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:35:23.406 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:23.403+0000 7f28493a4640 1 --2- 172.21.15.43:0/3850709029 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f2844164560 0x7f28440c32e0 unknown :-1 s=CLOSED pgs=339 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:35:23.406 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:23.403+0000 7f28493a4640 1 --2- 172.21.15.43:0/3850709029 >> [v2:172.21.15.203:6832/830097474,v1:172.21.15.203:6833/830097474] conn(0x7f282406a050 0x7f282406c510 unknown :-1 s=CLOSED pgs=179 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:35:23.407 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:23.403+0000 7f28493a4640 1 --2- 172.21.15.43:0/3850709029 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f28440c1020 0x7f2844150370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:35:23.407 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:23.403+0000 7f28493a4640 1 -- 172.21.15.43:0/3850709029 >> 172.21.15.43:0/3850709029 conn(0x7f28440cddd0 msgr2=0x7f2844164dd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-09T13:35:23.407 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:23.403+0000 7f28493a4640 1 -- 172.21.15.43:0/3850709029 shutdown_connections 2023-08-09T13:35:23.407 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:23.403+0000 7f28493a4640 1 -- 172.21.15.43:0/3850709029 wait complete. 2023-08-09T13:35:23.425 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-08-09T13:35:23.655 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:23.652+0000 7f7157fff640 1 Processor -- start 2023-08-09T13:35:23.655 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:23.652+0000 7f7157fff640 1 -- start start 2023-08-09T13:35:23.656 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:23.654+0000 7f7157fff640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f71580c9ee0 0x7f715815a800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:35:23.657 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:23.654+0000 7f7157fff640 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7f71580c90a0 0x7f715815ad40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:35:23.657 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:23.654+0000 7f7157fff640 1 -- --> v1:172.21.15.43:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7158158280 con 0x7f7158157ea0 2023-08-09T13:35:23.657 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:23.654+0000 7f7157fff640 1 -- --> v2:172.21.15.203:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f715815b840 con 0x7f71580c90a0 2023-08-09T13:35:23.657 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:23.654+0000 7f7157fff640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f71580b3810 con 0x7f71580c9ee0 2023-08-09T13:35:23.659 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:23.654+0000 7f7156ffd640 1 --1- >> v1:172.21.15.43:6789/0 conn(0x7f7158157ea0 0x7f71580c97d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.43:6789/0 says I am v1:172.21.15.43:49180/0 (socket says 172.21.15.43:49180) 2023-08-09T13:35:23.659 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:23.654+0000 7f7156ffd640 1 -- 172.21.15.43:0/2618617574 learned_addr learned my addr 172.21.15.43:0/2618617574 (peer_addr_for_me v1:172.21.15.43:0/0) 2023-08-09T13:35:23.659 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:23.655+0000 7f71577fe640 1 --2- 172.21.15.43:0/2618617574 >> v2:172.21.15.203:3300/0 conn(0x7f71580c90a0 0x7f715815ad40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:35:23.660 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:23.655+0000 7f71567fc640 1 --2- 172.21.15.43:0/2618617574 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f71580c9ee0 0x7f715815a800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:35:23.660 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:23.655+0000 7f7155ffb640 1 -- 172.21.15.43:0/2618617574 <== mon.0 v1:172.21.15.43:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3067153842 0 0) 0x7f7148001110 con 0x7f7158157ea0 2023-08-09T13:35:23.660 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:23.655+0000 7f7155ffb640 1 -- 172.21.15.43:0/2618617574 --> v1:172.21.15.43:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f7140003520 con 0x7f7158157ea0 2023-08-09T13:35:23.660 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:23.655+0000 7f7155ffb640 1 -- 172.21.15.43:0/2618617574 <== mon.0 v1:172.21.15.43:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 643891129 0 0) 0x7f7140003520 con 0x7f7158157ea0 2023-08-09T13:35:23.660 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:23.655+0000 7f7155ffb640 1 -- 172.21.15.43:0/2618617574 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f71580c9ee0 msgr2=0x7f715815a800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-09T13:35:23.660 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:23.655+0000 7f7155ffb640 1 --2- 172.21.15.43:0/2618617574 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f71580c9ee0 0x7f715815a800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:35:23.660 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:23.655+0000 7f7155ffb640 1 -- 172.21.15.43:0/2618617574 >> v2:172.21.15.203:3300/0 conn(0x7f71580c90a0 msgr2=0x7f715815ad40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-09T13:35:23.661 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:23.655+0000 7f7155ffb640 1 --2- 172.21.15.43:0/2618617574 >> v2:172.21.15.203:3300/0 conn(0x7f71580c90a0 0x7f715815ad40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:35:23.661 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:23.655+0000 7f7155ffb640 1 -- 172.21.15.43:0/2618617574 --> v1:172.21.15.43:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f715815b560 con 0x7f7158157ea0 2023-08-09T13:35:23.661 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:23.655+0000 7f7155ffb640 1 -- 172.21.15.43:0/2618617574 <== mon.0 v1:172.21.15.43:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1285150740 0 0) 0x7f7148001800 con 0x7f7158157ea0 2023-08-09T13:35:23.661 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:23.655+0000 7f7155ffb640 1 -- 172.21.15.43:0/2618617574 >> v1:172.21.15.43:6789/0 conn(0x7f7158157ea0 legacy=0x7f71580c97d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:35:23.661 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:23.655+0000 7f7155ffb640 1 --2- 172.21.15.43:0/2618617574 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f7140003970 0x7f7140003d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:35:23.661 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:23.655+0000 7f7155ffb640 1 --2- 172.21.15.43:0/2618617574 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f71400047c0 0x7f7140004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:35:23.661 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:23.656+0000 7f7155ffb640 1 --2- 172.21.15.43:0/2618617574 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f71580c9ee0 0x7f7140005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:35:23.662 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:23.656+0000 7f7155ffb640 1 -- 172.21.15.43:0/2618617574 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f715815b840 con 0x7f7140003970 2023-08-09T13:35:23.662 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:23.656+0000 7f7155ffb640 1 -- 172.21.15.43:0/2618617574 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f71580b3810 con 0x7f71400047c0 2023-08-09T13:35:23.662 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:23.656+0000 7f7155ffb640 1 -- 172.21.15.43:0/2618617574 --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f714000b9b0 con 0x7f71580c9ee0 2023-08-09T13:35:23.662 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:23.656+0000 7f7156ffd640 1 --2- 172.21.15.43:0/2618617574 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f7140003970 0x7f7140003d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:35:23.662 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:23.656+0000 7f71567fc640 1 --2- 172.21.15.43:0/2618617574 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f71400047c0 0x7f7140004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:35:23.662 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:23.656+0000 7f71577fe640 1 --2- 172.21.15.43:0/2618617574 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f71580c9ee0 0x7f7140005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:35:23.662 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:23.656+0000 7f7156ffd640 1 -- 172.21.15.43:0/2618617574 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f71580c9ee0 msgr2=0x7f7140005180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-09T13:35:23.663 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:23.656+0000 7f7156ffd640 1 --2- 172.21.15.43:0/2618617574 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f71580c9ee0 0x7f7140005180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:35:23.663 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:23.656+0000 7f7156ffd640 1 -- 172.21.15.43:0/2618617574 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f71400047c0 msgr2=0x7f7140004c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-09T13:35:23.663 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:23.656+0000 7f7156ffd640 1 --2- 172.21.15.43:0/2618617574 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f71400047c0 0x7f7140004c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:35:23.663 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:23.656+0000 7f7156ffd640 1 -- 172.21.15.43:0/2618617574 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7140003520 con 0x7f7140003970 2023-08-09T13:35:23.663 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:23.656+0000 7f71567fc640 1 --2- 172.21.15.43:0/2618617574 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f71400047c0 0x7f7140004c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-08-09T13:35:23.665 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:23.657+0000 7f7156ffd640 1 --2- 172.21.15.43:0/2618617574 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f7140003970 0x7f7140003d90 secure :-1 s=READY pgs=341 cs=0 l=1 rev1=1 crypto rx=0x7f714800c210 tx=0x7f71400057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c5e6ed966057cad8 server_cookie=0 in_seq=0 out_seq=0 2023-08-09T13:35:23.665 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:23.658+0000 7f7155ffb640 1 -- 172.21.15.43:0/2618617574 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f714802cae0 con 0x7f7140003970 2023-08-09T13:35:23.665 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:23.658+0000 7f7155ffb640 1 -- 172.21.15.43:0/2618617574 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f714802cc60 con 0x7f7140003970 2023-08-09T13:35:23.666 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:23.658+0000 7f7155ffb640 1 -- 172.21.15.43:0/2618617574 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f71480058d0 con 0x7f7140003970 2023-08-09T13:35:23.666 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:23.658+0000 7f7157fff640 1 -- 172.21.15.43:0/2618617574 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f7140003970 msgr2=0x7f7140003d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:35:23.666 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:23.658+0000 7f7157fff640 1 --2- 172.21.15.43:0/2618617574 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f7140003970 0x7f7140003d90 secure :-1 s=READY pgs=341 cs=0 l=1 rev1=1 crypto rx=0x7f714800c210 tx=0x7f71400057f0 comp rx=0 tx=0).stop 2023-08-09T13:35:23.666 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:23.658+0000 7f71577fe640 1 -- 172.21.15.43:0/2618617574 reap_dead start 2023-08-09T13:35:23.666 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:23.659+0000 7f7157fff640 1 -- 172.21.15.43:0/2618617574 shutdown_connections 2023-08-09T13:35:23.666 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:23.659+0000 7f7157fff640 1 -- 172.21.15.43:0/2618617574 >> 172.21.15.43:0/2618617574 conn(0x7f71580cddd0 msgr2=0x7f71580ce1e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-09T13:35:23.666 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:23.659+0000 7f7157fff640 1 -- 172.21.15.43:0/2618617574 shutdown_connections 2023-08-09T13:35:23.667 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:23.659+0000 7f7157fff640 1 -- 172.21.15.43:0/2618617574 wait complete. 2023-08-09T13:35:23.667 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:23.661+0000 7f7157fff640 1 Processor -- start 2023-08-09T13:35:23.667 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:23.661+0000 7f7157fff640 1 -- start start 2023-08-09T13:35:23.667 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:23.662+0000 7f7157fff640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f7140003970 0x7f71580aeeb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:35:23.667 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:23.662+0000 7f7157fff640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f71580c76b0 0x7f71580c5a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:35:23.667 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:23.662+0000 7f7157fff640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f71580c5f80 0x7f71580ab8d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:35:23.667 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:23.662+0000 7f7157fff640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f71580c7e90 con 0x7f71580c5f80 2023-08-09T13:35:23.668 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:23.662+0000 7f7157fff640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f71580c5620 con 0x7f71580c76b0 2023-08-09T13:35:23.668 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:23.662+0000 7f7157fff640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f71580c7bb0 con 0x7f7140003970 2023-08-09T13:35:23.668 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:23.662+0000 7f71577fe640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f71580c5f80 0x7f71580ab8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:35:23.668 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:23.662+0000 7f71577fe640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f71580c5f80 0x7f71580ab8d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:42634/0 (socket says 172.21.15.43:42634) 2023-08-09T13:35:23.668 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:23.662+0000 7f71577fe640 1 -- 172.21.15.43:0/3312135288 learned_addr learned my addr 172.21.15.43:0/3312135288 (peer_addr_for_me v2:172.21.15.43:0/0) 2023-08-09T13:35:23.669 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:23.662+0000 7f71567fc640 1 --2- 172.21.15.43:0/3312135288 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f71580c76b0 0x7f71580c5a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:35:23.669 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:23.662+0000 7f7156ffd640 1 --2- 172.21.15.43:0/3312135288 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f7140003970 0x7f71580aeeb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:35:23.669 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:23.663+0000 7f71577fe640 1 -- 172.21.15.43:0/3312135288 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f7140003970 msgr2=0x7f71580aeeb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:35:23.669 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:23.663+0000 7f71577fe640 1 --2- 172.21.15.43:0/3312135288 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f7140003970 0x7f71580aeeb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:35:23.669 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:23.663+0000 7f71577fe640 1 -- 172.21.15.43:0/3312135288 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f71580c76b0 msgr2=0x7f71580c5a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:35:23.669 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:23.663+0000 7f71577fe640 1 --2- 172.21.15.43:0/3312135288 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f71580c76b0 0x7f71580c5a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:35:23.670 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:23.663+0000 7f71577fe640 1 -- 172.21.15.43:0/3312135288 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f71580c4fd0 con 0x7f71580c5f80 2023-08-09T13:35:23.670 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:23.663+0000 7f71567fc640 1 --2- 172.21.15.43:0/3312135288 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f71580c76b0 0x7f71580c5a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-08-09T13:35:23.671 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:23.663+0000 7f71577fe640 1 --2- 172.21.15.43:0/3312135288 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f71580c5f80 0x7f71580ab8d0 secure :-1 s=READY pgs=342 cs=0 l=1 rev1=1 crypto rx=0x7f715806d520 tx=0x7f715000f4a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-09T13:35:23.671 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:23.663+0000 7f713f7fe640 1 -- 172.21.15.43:0/3312135288 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7150017400 con 0x7f71580c5f80 2023-08-09T13:35:23.671 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:23.663+0000 7f713f7fe640 1 -- 172.21.15.43:0/3312135288 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7150011040 con 0x7f71580c5f80 2023-08-09T13:35:23.671 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:23.663+0000 7f713f7fe640 1 -- 172.21.15.43:0/3312135288 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f71500177c0 con 0x7f71580c5f80 2023-08-09T13:35:23.671 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:23.663+0000 7f7157fff640 1 -- 172.21.15.43:0/3312135288 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f71580c52b0 con 0x7f71580c5f80 2023-08-09T13:35:23.672 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:23.663+0000 7f7157fff640 1 -- 172.21.15.43:0/3312135288 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f71580abe10 con 0x7f71580c5f80 2023-08-09T13:35:23.672 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:23.665+0000 7f7157fff640 1 -- 172.21.15.43:0/3312135288 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f715804e390 con 0x7f71580c5f80 2023-08-09T13:35:23.672 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:23.666+0000 7f713f7fe640 1 -- 172.21.15.43:0/3312135288 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 5) v1 ==== 88341+0+0 (secure 0 0 0) 0x7f7150017920 con 0x7f71580c5f80 2023-08-09T13:35:23.672 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:23.667+0000 7f713f7fe640 1 --2- 172.21.15.43:0/3312135288 >> [v2:172.21.15.203:6832/830097474,v1:172.21.15.203:6833/830097474] conn(0x7f714406a080 0x7f714406c540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:35:23.673 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:23.667+0000 7f713f7fe640 1 -- 172.21.15.43:0/3312135288 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 6871+0+0 (secure 0 0 0) 0x7f7150088c90 con 0x7f71580c5f80 2023-08-09T13:35:23.673 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:23.668+0000 7f7156ffd640 1 --2- 172.21.15.43:0/3312135288 >> [v2:172.21.15.203:6832/830097474,v1:172.21.15.203:6833/830097474] conn(0x7f714406a080 0x7f714406c540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:35:23.673 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:23.670+0000 7f7156ffd640 1 --2- 172.21.15.43:0/3312135288 >> [v2:172.21.15.203:6832/830097474,v1:172.21.15.203:6833/830097474] conn(0x7f714406a080 0x7f714406c540 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7f7148034000 tx=0x7f7148007000 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-09T13:35:23.674 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:23.672+0000 7f713f7fe640 1 -- 172.21.15.43:0/3312135288 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178066 (secure 0 0 0) 0x7f7150054040 con 0x7f71580c5f80 2023-08-09T13:35:23.873 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:23.870+0000 7f7157fff640 1 -- 172.21.15.43:0/3312135288 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f71580713c0 con 0x7f71580c5f80 2023-08-09T13:35:23.873 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:23.871+0000 7f713f7fe640 1 -- 172.21.15.43:0/3312135288 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v26) v1 ==== 93+0+10 (secure 0 0 0) 0x7f7150057660 con 0x7f71580c5f80 2023-08-09T13:35:23.874 INFO:teuthology.orchestra.run.smithi043.stdout:pg_num: 1 2023-08-09T13:35:23.876 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:23.874+0000 7f7157fff640 1 -- 172.21.15.43:0/3312135288 >> [v2:172.21.15.203:6832/830097474,v1:172.21.15.203:6833/830097474] conn(0x7f714406a080 msgr2=0x7f714406c540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:35:23.876 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:23.874+0000 7f7157fff640 1 --2- 172.21.15.43:0/3312135288 >> [v2:172.21.15.203:6832/830097474,v1:172.21.15.203:6833/830097474] conn(0x7f714406a080 0x7f714406c540 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7f7148034000 tx=0x7f7148007000 comp rx=0 tx=0).stop 2023-08-09T13:35:23.877 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:23.874+0000 7f7157fff640 1 -- 172.21.15.43:0/3312135288 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f71580c5f80 msgr2=0x7f71580ab8d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:35:23.877 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:23.874+0000 7f7157fff640 1 --2- 172.21.15.43:0/3312135288 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f71580c5f80 0x7f71580ab8d0 secure :-1 s=READY pgs=342 cs=0 l=1 rev1=1 crypto rx=0x7f715806d520 tx=0x7f715000f4a0 comp rx=0 tx=0).stop 2023-08-09T13:35:23.877 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:23.874+0000 7f7157fff640 1 -- 172.21.15.43:0/3312135288 shutdown_connections 2023-08-09T13:35:23.878 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:23.874+0000 7f7157fff640 1 --2- 172.21.15.43:0/3312135288 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f71580c5f80 0x7f71580ab8d0 unknown :-1 s=CLOSED pgs=342 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:35:23.878 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:23.874+0000 7f7157fff640 1 --2- 172.21.15.43:0/3312135288 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f71580c76b0 0x7f71580c5a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:35:23.878 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:23.874+0000 7f7157fff640 1 --2- 172.21.15.43:0/3312135288 >> [v2:172.21.15.203:6832/830097474,v1:172.21.15.203:6833/830097474] conn(0x7f714406a080 0x7f714406c540 unknown :-1 s=CLOSED pgs=180 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:35:23.878 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:23.874+0000 7f7157fff640 1 --2- 172.21.15.43:0/3312135288 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f7140003970 0x7f71580aeeb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:35:23.878 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:23.874+0000 7f7157fff640 1 -- 172.21.15.43:0/3312135288 >> 172.21.15.43:0/3312135288 conn(0x7f71580cddd0 msgr2=0x7f71580cc5a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-09T13:35:23.878 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:23.874+0000 7f7157fff640 1 -- 172.21.15.43:0/3312135288 shutdown_connections 2023-08-09T13:35:23.878 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:23.874+0000 7f7157fff640 1 -- 172.21.15.43:0/3312135288 wait complete. 2023-08-09T13:35:23.897 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-08-09T13:35:24.136 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:24.134+0000 7f949bc01640 1 Processor -- start 2023-08-09T13:35:24.137 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:24.134+0000 7f949bc01640 1 -- start start 2023-08-09T13:35:24.138 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:24.136+0000 7f949bc01640 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7f949415a250 0x7f94940697c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:35:24.138 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:24.136+0000 7f949bc01640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f949415a630 0x7f9494069d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:35:24.139 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:24.136+0000 7f949bc01640 1 -- --> v1:172.21.15.203:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f94940716b0 con 0x7f94940cb750 2023-08-09T13:35:24.139 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:24.136+0000 7f949bc01640 1 -- --> v2:172.21.15.203:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9494070aa0 con 0x7f949415a250 2023-08-09T13:35:24.139 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:24.136+0000 7f949bc01640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9494070d80 con 0x7f949415a630 2023-08-09T13:35:24.139 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:24.137+0000 7f949999d640 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7f949415a250 0x7f94940697c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:35:24.140 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:24.137+0000 7f949999d640 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7f949415a250 0x7f94940697c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.203:3300/0 says I am v2:172.21.15.43:58226/0 (socket says 172.21.15.43:58226) 2023-08-09T13:35:24.140 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:24.137+0000 7f949919c640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f949415a630 0x7f9494069d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:35:24.140 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:24.137+0000 7f949999d640 1 -- 172.21.15.43:0/3942485144 learned_addr learned my addr 172.21.15.43:0/3942485144 (peer_addr_for_me v2:172.21.15.43:0/0) 2023-08-09T13:35:24.140 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:24.138+0000 7f949919c640 1 -- 172.21.15.43:0/3942485144 >> v2:172.21.15.203:3300/0 conn(0x7f949415a250 msgr2=0x7f94940697c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-09T13:35:24.141 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:24.138+0000 7f949919c640 1 --2- 172.21.15.43:0/3942485144 >> v2:172.21.15.203:3300/0 conn(0x7f949415a250 0x7f94940697c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:35:24.141 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:24.138+0000 7f949919c640 1 -- 172.21.15.43:0/3942485144 >> v1:172.21.15.203:6789/0 conn(0x7f94940cb750 legacy=0x7f949415e030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:35:24.141 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:24.138+0000 7f949919c640 1 -- 172.21.15.43:0/3942485144 --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9494071060 con 0x7f949415a630 2023-08-09T13:35:24.141 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:24.138+0000 7f949999d640 1 --2- 172.21.15.43:0/3942485144 >> v2:172.21.15.203:3300/0 conn(0x7f949415a250 0x7f94940697c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-08-09T13:35:24.141 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:24.139+0000 7f949919c640 1 --2- 172.21.15.43:0/3942485144 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f949415a630 0x7f9494069d00 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f94840097a0 tx=0x7f948402b940 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ac7abc3e29d0e208 server_cookie=0 in_seq=0 out_seq=0 2023-08-09T13:35:24.142 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:24.139+0000 7f949899b640 1 -- 172.21.15.43:0/3942485144 <== mon.2 v2:172.21.15.203:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9484030800 con 0x7f949415a630 2023-08-09T13:35:24.142 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:24.140+0000 7f949899b640 1 -- 172.21.15.43:0/3942485144 <== mon.2 v2:172.21.15.203:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9484030960 con 0x7f949415a630 2023-08-09T13:35:24.142 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:24.140+0000 7f949899b640 1 -- 172.21.15.43:0/3942485144 <== mon.2 v2:172.21.15.203:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9484030cf0 con 0x7f949415a630 2023-08-09T13:35:24.143 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:24.140+0000 7f949bc01640 1 -- 172.21.15.43:0/3942485144 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f949415a630 msgr2=0x7f9494069d00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:35:24.143 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:24.140+0000 7f949bc01640 1 --2- 172.21.15.43:0/3942485144 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f949415a630 0x7f9494069d00 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f94840097a0 tx=0x7f948402b940 comp rx=0 tx=0).stop 2023-08-09T13:35:24.143 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:24.140+0000 7f949bc01640 1 -- 172.21.15.43:0/3942485144 shutdown_connections 2023-08-09T13:35:24.144 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:24.140+0000 7f949bc01640 1 --2- 172.21.15.43:0/3942485144 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f949415a630 0x7f9494069d00 unknown :-1 s=CLOSED pgs=64 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:35:24.144 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:24.140+0000 7f949bc01640 1 --2- 172.21.15.43:0/3942485144 >> v2:172.21.15.203:3300/0 conn(0x7f949415a250 0x7f94940697c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:35:24.144 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:24.140+0000 7f949bc01640 1 -- 172.21.15.43:0/3942485144 >> 172.21.15.43:0/3942485144 conn(0x7f94940cbc80 msgr2=0x7f94940cc090 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-09T13:35:24.144 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:24.140+0000 7f949bc01640 1 -- 172.21.15.43:0/3942485144 shutdown_connections 2023-08-09T13:35:24.144 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:24.140+0000 7f949bc01640 1 -- 172.21.15.43:0/3942485144 wait complete. 2023-08-09T13:35:24.145 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:24.142+0000 7f949bc01640 1 Processor -- start 2023-08-09T13:35:24.145 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:24.142+0000 7f949bc01640 1 -- start start 2023-08-09T13:35:24.146 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:24.143+0000 7f949bc01640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f94940cb750 0x7f949414fee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:35:24.146 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:24.143+0000 7f949bc01640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f949415a250 0x7f94940c3ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:35:24.146 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:24.143+0000 7f949bc01640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f949415a630 0x7f94941f5600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:35:24.146 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:24.143+0000 7f949bc01640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f94940c96d0 con 0x7f949415a630 2023-08-09T13:35:24.146 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:24.143+0000 7f949bc01640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9494150420 con 0x7f949415a250 2023-08-09T13:35:24.147 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:24.143+0000 7f949bc01640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9494152330 con 0x7f94940cb750 2023-08-09T13:35:24.148 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:24.144+0000 7f949a19e640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f949415a630 0x7f94941f5600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:35:24.148 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:24.144+0000 7f949a19e640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f949415a630 0x7f94941f5600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:42644/0 (socket says 172.21.15.43:42644) 2023-08-09T13:35:24.148 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:24.144+0000 7f949a19e640 1 -- 172.21.15.43:0/3919423582 learned_addr learned my addr 172.21.15.43:0/3919423582 (peer_addr_for_me v2:172.21.15.43:0/0) 2023-08-09T13:35:24.148 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:24.144+0000 7f949919c640 1 --2- 172.21.15.43:0/3919423582 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f949415a250 0x7f94940c3ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:35:24.148 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:24.144+0000 7f949999d640 1 --2- 172.21.15.43:0/3919423582 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f94940cb750 0x7f949414fee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:35:24.149 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:24.144+0000 7f949a19e640 1 -- 172.21.15.43:0/3919423582 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f94940cb750 msgr2=0x7f949414fee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:35:24.149 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:24.144+0000 7f949a19e640 1 --2- 172.21.15.43:0/3919423582 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f94940cb750 0x7f949414fee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:35:24.149 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:24.144+0000 7f949a19e640 1 -- 172.21.15.43:0/3919423582 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f949415a250 msgr2=0x7f94940c3ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:35:24.149 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:24.144+0000 7f949a19e640 1 --2- 172.21.15.43:0/3919423582 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f949415a250 0x7f94940c3ae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:35:24.149 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:24.144+0000 7f949a19e640 1 -- 172.21.15.43:0/3919423582 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f94900094d0 con 0x7f949415a630 2023-08-09T13:35:24.150 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:24.145+0000 7f949a19e640 1 --2- 172.21.15.43:0/3919423582 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f949415a630 0x7f94941f5600 secure :-1 s=READY pgs=343 cs=0 l=1 rev1=1 crypto rx=0x7f948c00c7a0 tx=0x7f948c00cc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-09T13:35:24.150 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:24.145+0000 7f948a7fc640 1 -- 172.21.15.43:0/3919423582 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f948c003b70 con 0x7f949415a630 2023-08-09T13:35:24.150 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:24.145+0000 7f948a7fc640 1 -- 172.21.15.43:0/3919423582 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f948c003cf0 con 0x7f949415a630 2023-08-09T13:35:24.150 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:24.145+0000 7f948a7fc640 1 -- 172.21.15.43:0/3919423582 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f948c003ea0 con 0x7f949415a630 2023-08-09T13:35:24.150 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:24.145+0000 7f949bc01640 1 -- 172.21.15.43:0/3919423582 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9484009400 con 0x7f949415a630 2023-08-09T13:35:24.150 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:24.145+0000 7f949bc01640 1 -- 172.21.15.43:0/3919423582 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f94940c7dc0 con 0x7f949415a630 2023-08-09T13:35:24.151 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:24.146+0000 7f949bc01640 1 -- 172.21.15.43:0/3919423582 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f94941524a0 con 0x7f949415a630 2023-08-09T13:35:24.151 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:24.147+0000 7f948a7fc640 1 -- 172.21.15.43:0/3919423582 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 5) v1 ==== 88341+0+0 (secure 0 0 0) 0x7f948c00aa30 con 0x7f949415a630 2023-08-09T13:35:24.151 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:24.148+0000 7f948a7fc640 1 --2- 172.21.15.43:0/3919423582 >> [v2:172.21.15.203:6832/830097474,v1:172.21.15.203:6833/830097474] conn(0x7f947806a080 0x7f947806c540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:35:24.151 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:24.148+0000 7f948a7fc640 1 -- 172.21.15.43:0/3919423582 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 6871+0+0 (secure 0 0 0) 0x7f948c088710 con 0x7f949415a630 2023-08-09T13:35:24.152 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:24.149+0000 7f949999d640 1 --2- 172.21.15.43:0/3919423582 >> [v2:172.21.15.203:6832/830097474,v1:172.21.15.203:6833/830097474] conn(0x7f947806a080 0x7f947806c540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:35:24.154 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:24.152+0000 7f949999d640 1 --2- 172.21.15.43:0/3919423582 >> [v2:172.21.15.203:6832/830097474,v1:172.21.15.203:6833/830097474] conn(0x7f947806a080 0x7f947806c540 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7f9490009cf0 tx=0x7f9490008040 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-09T13:35:24.155 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:24.152+0000 7f948a7fc640 1 -- 172.21.15.43:0/3919423582 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178066 (secure 0 0 0) 0x7f948c053ae0 con 0x7f949415a630 2023-08-09T13:35:24.349 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:24.346+0000 7f949bc01640 1 -- 172.21.15.43:0/3919423582 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f94940739f0 con 0x7f949415a630 2023-08-09T13:35:24.349 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:24.347+0000 7f948a7fc640 1 -- 172.21.15.43:0/3919423582 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v26) v1 ==== 104+0+11 (secure 0 0 0) 0x7f948c057100 con 0x7f949415a630 2023-08-09T13:35:24.350 INFO:teuthology.orchestra.run.smithi043.stdout:pg_num: 32 2023-08-09T13:35:24.352 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:24.349+0000 7f949bc01640 1 -- 172.21.15.43:0/3919423582 >> [v2:172.21.15.203:6832/830097474,v1:172.21.15.203:6833/830097474] conn(0x7f947806a080 msgr2=0x7f947806c540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:35:24.352 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:24.349+0000 7f949bc01640 1 --2- 172.21.15.43:0/3919423582 >> [v2:172.21.15.203:6832/830097474,v1:172.21.15.203:6833/830097474] conn(0x7f947806a080 0x7f947806c540 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7f9490009cf0 tx=0x7f9490008040 comp rx=0 tx=0).stop 2023-08-09T13:35:24.353 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:24.349+0000 7f949bc01640 1 -- 172.21.15.43:0/3919423582 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f949415a630 msgr2=0x7f94941f5600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:35:24.353 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:24.349+0000 7f949bc01640 1 --2- 172.21.15.43:0/3919423582 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f949415a630 0x7f94941f5600 secure :-1 s=READY pgs=343 cs=0 l=1 rev1=1 crypto rx=0x7f948c00c7a0 tx=0x7f948c00cc60 comp rx=0 tx=0).stop 2023-08-09T13:35:24.353 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:24.350+0000 7f949bc01640 1 -- 172.21.15.43:0/3919423582 shutdown_connections 2023-08-09T13:35:24.353 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:24.350+0000 7f949bc01640 1 --2- 172.21.15.43:0/3919423582 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f949415a630 0x7f94941f5600 unknown :-1 s=CLOSED pgs=343 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:35:24.353 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:24.350+0000 7f949bc01640 1 --2- 172.21.15.43:0/3919423582 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f949415a250 0x7f94940c3ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:35:24.353 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:24.350+0000 7f949bc01640 1 --2- 172.21.15.43:0/3919423582 >> [v2:172.21.15.203:6832/830097474,v1:172.21.15.203:6833/830097474] conn(0x7f947806a080 0x7f947806c540 unknown :-1 s=CLOSED pgs=181 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:35:24.353 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:24.350+0000 7f949bc01640 1 --2- 172.21.15.43:0/3919423582 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f94940cb750 0x7f949414fee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:35:24.354 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:24.350+0000 7f949bc01640 1 -- 172.21.15.43:0/3919423582 >> 172.21.15.43:0/3919423582 conn(0x7f94940cbc80 msgr2=0x7f9494157840 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-09T13:35:24.354 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:24.350+0000 7f949bc01640 1 -- 172.21.15.43:0/3919423582 shutdown_connections 2023-08-09T13:35:24.354 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:24.350+0000 7f949bc01640 1 -- 172.21.15.43:0/3919423582 wait complete. 2023-08-09T13:35:24.373 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-08-09T13:35:24.607 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:24.605+0000 7fafc8512640 1 Processor -- start 2023-08-09T13:35:24.607 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:24.605+0000 7fafc8512640 1 -- start start 2023-08-09T13:35:24.609 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:24.606+0000 7fafc8512640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7fafc00b45f0 0x7fafc00cd6d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:35:24.609 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:24.606+0000 7fafc8512640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fafc00d04b0 0x7fafc00b59d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:35:24.609 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:24.607+0000 7fafc8512640 1 -- --> v1:172.21.15.203:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fafc00b7bd0 con 0x7fafc015c550 2023-08-09T13:35:24.609 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:24.607+0000 7fafc8512640 1 -- --> v2:172.21.15.43:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fafc00d1c00 con 0x7fafc00b45f0 2023-08-09T13:35:24.610 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:24.607+0000 7fafc8512640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fafc015c930 con 0x7fafc00d04b0 2023-08-09T13:35:24.610 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:24.607+0000 7fafc5aad640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7fafc00b45f0 0x7fafc00cd6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:35:24.610 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:24.607+0000 7fafc5aad640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7fafc00b45f0 0x7fafc00cd6d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:42654/0 (socket says 172.21.15.43:42654) 2023-08-09T13:35:24.611 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:24.607+0000 7fafc5aad640 1 -- 172.21.15.43:0/2563844362 learned_addr learned my addr 172.21.15.43:0/2563844362 (peer_addr_for_me v2:172.21.15.43:0/0) 2023-08-09T13:35:24.611 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:24.607+0000 7fafc6aaf640 1 --2- 172.21.15.43:0/2563844362 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fafc00d04b0 0x7fafc00b59d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:35:24.611 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:24.608+0000 7fafc5aad640 1 -- 172.21.15.43:0/2563844362 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fafc00d04b0 msgr2=0x7fafc00b59d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-09T13:35:24.611 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:24.608+0000 7fafc5aad640 1 --2- 172.21.15.43:0/2563844362 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fafc00d04b0 0x7fafc00b59d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:35:24.611 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:24.608+0000 7fafc5aad640 1 -- 172.21.15.43:0/2563844362 >> v1:172.21.15.203:6789/0 conn(0x7fafc015c550 legacy=0x7fafc00b3ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:35:24.611 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:24.608+0000 7fafc5aad640 1 -- 172.21.15.43:0/2563844362 --> v2:172.21.15.43:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fafc00b54f0 con 0x7fafc00b45f0 2023-08-09T13:35:24.612 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:24.609+0000 7fafc5aad640 1 --2- 172.21.15.43:0/2563844362 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fafc00b45f0 0x7fafc00cd6d0 secure :-1 s=READY pgs=344 cs=0 l=1 rev1=1 crypto rx=0x7fafb0009870 tx=0x7fafb002b980 comp rx=0 tx=0).ready entity=mon.0 client_cookie=41633bbd2e42454d server_cookie=0 in_seq=0 out_seq=0 2023-08-09T13:35:24.612 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:24.609+0000 7fafc52ac640 1 -- 172.21.15.43:0/2563844362 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fafb0030840 con 0x7fafc00b45f0 2023-08-09T13:35:24.612 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:24.609+0000 7fafc52ac640 1 -- 172.21.15.43:0/2563844362 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fafb00309a0 con 0x7fafc00b45f0 2023-08-09T13:35:24.612 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:24.609+0000 7fafc52ac640 1 -- 172.21.15.43:0/2563844362 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fafb0030d30 con 0x7fafc00b45f0 2023-08-09T13:35:24.613 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:24.610+0000 7fafc8512640 1 -- 172.21.15.43:0/2563844362 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fafc00b45f0 msgr2=0x7fafc00cd6d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:35:24.613 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:24.610+0000 7fafc8512640 1 --2- 172.21.15.43:0/2563844362 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fafc00b45f0 0x7fafc00cd6d0 secure :-1 s=READY pgs=344 cs=0 l=1 rev1=1 crypto rx=0x7fafb0009870 tx=0x7fafb002b980 comp rx=0 tx=0).stop 2023-08-09T13:35:24.613 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:24.610+0000 7fafc8512640 1 -- 172.21.15.43:0/2563844362 shutdown_connections 2023-08-09T13:35:24.613 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:24.610+0000 7fafc8512640 1 --2- 172.21.15.43:0/2563844362 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fafc00d04b0 0x7fafc00b59d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:35:24.613 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:24.610+0000 7fafc8512640 1 --2- 172.21.15.43:0/2563844362 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fafc00b45f0 0x7fafc00cd6d0 unknown :-1 s=CLOSED pgs=344 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:35:24.614 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:24.610+0000 7fafc8512640 1 -- 172.21.15.43:0/2563844362 >> 172.21.15.43:0/2563844362 conn(0x7fafc0071520 msgr2=0x7fafc00ceee0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-09T13:35:24.614 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:24.610+0000 7fafc8512640 1 -- 172.21.15.43:0/2563844362 shutdown_connections 2023-08-09T13:35:24.614 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:24.610+0000 7fafc8512640 1 -- 172.21.15.43:0/2563844362 wait complete. 2023-08-09T13:35:24.614 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:24.612+0000 7fafc8512640 1 Processor -- start 2023-08-09T13:35:24.615 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:24.612+0000 7fafc8512640 1 -- start start 2023-08-09T13:35:24.616 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:24.613+0000 7fafc8512640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fafc00b45f0 0x7fafc015b9e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:35:24.616 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:24.613+0000 7fafc8512640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fafc00d04b0 0x7fafc00cbef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:35:24.616 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:24.613+0000 7fafc8512640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fafc015c550 0x7fafc015a480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:35:24.616 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:24.613+0000 7fafc8512640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fafc01e5a10 con 0x7fafc00d04b0 2023-08-09T13:35:24.617 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:24.613+0000 7fafc8512640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fafc01e7ea0 con 0x7fafc00b45f0 2023-08-09T13:35:24.617 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:24.613+0000 7fafc8512640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fafc01e8180 con 0x7fafc015c550 2023-08-09T13:35:24.617 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:24.614+0000 7fafc5aad640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fafc00d04b0 0x7fafc00cbef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:35:24.617 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:24.614+0000 7fafc5aad640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fafc00d04b0 0x7fafc00cbef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:42668/0 (socket says 172.21.15.43:42668) 2023-08-09T13:35:24.617 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:24.614+0000 7fafc5aad640 1 -- 172.21.15.43:0/79598452 learned_addr learned my addr 172.21.15.43:0/79598452 (peer_addr_for_me v2:172.21.15.43:0/0) 2023-08-09T13:35:24.618 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:24.614+0000 7fafc62ae640 1 --2- 172.21.15.43:0/79598452 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fafc00b45f0 0x7fafc015b9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:35:24.618 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:24.614+0000 7fafc6aaf640 1 --2- 172.21.15.43:0/79598452 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fafc015c550 0x7fafc015a480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:35:24.618 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:24.614+0000 7fafc5aad640 1 -- 172.21.15.43:0/79598452 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fafc015c550 msgr2=0x7fafc015a480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:35:24.618 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:24.614+0000 7fafc5aad640 1 --2- 172.21.15.43:0/79598452 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fafc015c550 0x7fafc015a480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:35:24.619 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:24.614+0000 7fafc5aad640 1 -- 172.21.15.43:0/79598452 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fafc00b45f0 msgr2=0x7fafc015b9e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:35:24.619 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:24.614+0000 7fafc5aad640 1 --2- 172.21.15.43:0/79598452 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fafc00b45f0 0x7fafc015b9e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:35:24.619 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:24.614+0000 7fafc5aad640 1 -- 172.21.15.43:0/79598452 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fafb00094d0 con 0x7fafc00d04b0 2023-08-09T13:35:24.619 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:24.615+0000 7fafc5aad640 1 --2- 172.21.15.43:0/79598452 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fafc00d04b0 0x7fafc00cbef0 secure :-1 s=READY pgs=345 cs=0 l=1 rev1=1 crypto rx=0x7fafb0000c00 tx=0x7fafb0004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-09T13:35:24.619 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:24.615+0000 7fafb6ffd640 1 -- 172.21.15.43:0/79598452 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fafb0039c90 con 0x7fafc00d04b0 2023-08-09T13:35:24.619 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:24.615+0000 7fafb6ffd640 1 -- 172.21.15.43:0/79598452 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fafb0039df0 con 0x7fafc00d04b0 2023-08-09T13:35:24.619 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:24.615+0000 7fafb6ffd640 1 -- 172.21.15.43:0/79598452 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fafb0043640 con 0x7fafc00d04b0 2023-08-09T13:35:24.620 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:24.615+0000 7fafc8512640 1 -- 172.21.15.43:0/79598452 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fafc01e8460 con 0x7fafc00d04b0 2023-08-09T13:35:24.620 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:24.615+0000 7fafc8512640 1 -- 172.21.15.43:0/79598452 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fafc00a9110 con 0x7fafc00d04b0 2023-08-09T13:35:24.620 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:24.616+0000 7fafc8512640 1 -- 172.21.15.43:0/79598452 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fafc00739f0 con 0x7fafc00d04b0 2023-08-09T13:35:24.620 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:24.618+0000 7fafb6ffd640 1 -- 172.21.15.43:0/79598452 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 5) v1 ==== 88341+0+0 (secure 0 0 0) 0x7fafb00437a0 con 0x7fafc00d04b0 2023-08-09T13:35:24.621 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:24.619+0000 7fafb6ffd640 1 --2- 172.21.15.43:0/79598452 >> [v2:172.21.15.203:6832/830097474,v1:172.21.15.203:6833/830097474] conn(0x7fafa406a050 0x7fafa406c510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:35:24.622 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:24.619+0000 7fafb6ffd640 1 -- 172.21.15.43:0/79598452 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 6871+0+0 (secure 0 0 0) 0x7fafb003e070 con 0x7fafc00d04b0 2023-08-09T13:35:24.622 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:24.620+0000 7fafc62ae640 1 --2- 172.21.15.43:0/79598452 >> [v2:172.21.15.203:6832/830097474,v1:172.21.15.203:6833/830097474] conn(0x7fafa406a050 0x7fafa406c510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:35:24.623 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:24.621+0000 7fafc62ae640 1 --2- 172.21.15.43:0/79598452 >> [v2:172.21.15.203:6832/830097474,v1:172.21.15.203:6833/830097474] conn(0x7fafa406a050 0x7fafa406c510 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7fafbc0012e0 tx=0x7fafbc009040 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-09T13:35:24.626 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:24.623+0000 7fafb6ffd640 1 -- 172.21.15.43:0/79598452 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178066 (secure 0 0 0) 0x7fafb0076cd0 con 0x7fafc00d04b0 2023-08-09T13:35:24.821 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:24.818+0000 7fafc8512640 1 -- 172.21.15.43:0/79598452 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7fafc00d1300 con 0x7fafc00d04b0 2023-08-09T13:35:24.822 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:24.819+0000 7fafb6ffd640 1 -- 172.21.15.43:0/79598452 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v26) v1 ==== 100+0+11 (secure 0 0 0) 0x7fafb0034020 con 0x7fafc00d04b0 2023-08-09T13:35:24.822 INFO:teuthology.orchestra.run.smithi043.stdout:pg_num: 64 2023-08-09T13:35:24.825 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:24.822+0000 7fafc8512640 1 -- 172.21.15.43:0/79598452 >> [v2:172.21.15.203:6832/830097474,v1:172.21.15.203:6833/830097474] conn(0x7fafa406a050 msgr2=0x7fafa406c510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:35:24.825 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:24.822+0000 7fafc8512640 1 --2- 172.21.15.43:0/79598452 >> [v2:172.21.15.203:6832/830097474,v1:172.21.15.203:6833/830097474] conn(0x7fafa406a050 0x7fafa406c510 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7fafbc0012e0 tx=0x7fafbc009040 comp rx=0 tx=0).stop 2023-08-09T13:35:24.825 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:24.822+0000 7fafc8512640 1 -- 172.21.15.43:0/79598452 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fafc00d04b0 msgr2=0x7fafc00cbef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:35:24.825 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:24.822+0000 7fafc8512640 1 --2- 172.21.15.43:0/79598452 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fafc00d04b0 0x7fafc00cbef0 secure :-1 s=READY pgs=345 cs=0 l=1 rev1=1 crypto rx=0x7fafb0000c00 tx=0x7fafb0004730 comp rx=0 tx=0).stop 2023-08-09T13:35:24.826 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:24.822+0000 7fafc8512640 1 -- 172.21.15.43:0/79598452 shutdown_connections 2023-08-09T13:35:24.826 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:24.822+0000 7fafc8512640 1 --2- 172.21.15.43:0/79598452 >> [v2:172.21.15.203:6832/830097474,v1:172.21.15.203:6833/830097474] conn(0x7fafa406a050 0x7fafa406c510 unknown :-1 s=CLOSED pgs=182 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:35:24.826 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:24.822+0000 7fafc8512640 1 --2- 172.21.15.43:0/79598452 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fafc015c550 0x7fafc015a480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:35:24.826 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:24.822+0000 7fafc8512640 1 --2- 172.21.15.43:0/79598452 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fafc00d04b0 0x7fafc00cbef0 unknown :-1 s=CLOSED pgs=345 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:35:24.826 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:24.822+0000 7fafc8512640 1 --2- 172.21.15.43:0/79598452 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fafc00b45f0 0x7fafc015b9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:35:24.826 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:24.822+0000 7fafc8512640 1 -- 172.21.15.43:0/79598452 >> 172.21.15.43:0/79598452 conn(0x7fafc0071520 msgr2=0x7fafc015e240 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-09T13:35:24.826 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:24.823+0000 7fafc8512640 1 -- 172.21.15.43:0/79598452 shutdown_connections 2023-08-09T13:35:24.827 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:24.823+0000 7fafc8512640 1 -- 172.21.15.43:0/79598452 wait complete. 2023-08-09T13:35:24.843 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2023-08-09T13:35:25.098 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:25.095+0000 7fc307364640 1 Processor -- start 2023-08-09T13:35:25.098 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:25.096+0000 7fc307364640 1 -- start start 2023-08-09T13:35:25.100 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:25.097+0000 7fc307364640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7fc3000d0a70 0x7fc3000cd7f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:35:25.100 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:25.097+0000 7fc307364640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fc3000cdd30 0x7fc3000b3fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:35:25.100 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:25.097+0000 7fc307364640 1 -- --> v1:172.21.15.43:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc3000d0f70 con 0x7fc30015c680 2023-08-09T13:35:25.100 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:25.097+0000 7fc307364640 1 -- --> v2:172.21.15.43:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc3000ccd10 con 0x7fc3000d0a70 2023-08-09T13:35:25.101 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:25.097+0000 7fc307364640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc3000ccff0 con 0x7fc3000cdd30 2023-08-09T13:35:25.101 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:25.097+0000 7fc305100640 1 --1- >> v1:172.21.15.43:6789/0 conn(0x7fc30015c680 0x7fc3000ce760 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.43:6789/0 says I am v1:172.21.15.43:52190/0 (socket says 172.21.15.43:52190) 2023-08-09T13:35:25.101 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:25.097+0000 7fc305100640 1 -- 172.21.15.43:0/1668880745 learned_addr learned my addr 172.21.15.43:0/1668880745 (peer_addr_for_me v1:172.21.15.43:0/0) 2023-08-09T13:35:25.101 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:25.098+0000 7fc3048ff640 1 --2- 172.21.15.43:0/1668880745 >> v2:172.21.15.43:3300/0 conn(0x7fc3000d0a70 0x7fc3000cd7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:35:25.102 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:25.098+0000 7fc305901640 1 --2- 172.21.15.43:0/1668880745 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fc3000cdd30 0x7fc3000b3fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:35:25.102 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:25.098+0000 7fc3048ff640 1 -- 172.21.15.43:0/1668880745 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fc3000cdd30 msgr2=0x7fc3000b3fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-09T13:35:25.102 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:25.098+0000 7fc3048ff640 1 --2- 172.21.15.43:0/1668880745 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fc3000cdd30 0x7fc3000b3fa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:35:25.102 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:25.098+0000 7fc3048ff640 1 -- 172.21.15.43:0/1668880745 >> v1:172.21.15.43:6789/0 conn(0x7fc30015c680 legacy=0x7fc3000ce760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:35:25.102 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:25.098+0000 7fc3048ff640 1 -- 172.21.15.43:0/1668880745 --> v2:172.21.15.43:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc3000cd2d0 con 0x7fc3000d0a70 2023-08-09T13:35:25.103 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:25.100+0000 7fc3048ff640 1 --2- 172.21.15.43:0/1668880745 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc3000d0a70 0x7fc3000cd7f0 secure :-1 s=READY pgs=347 cs=0 l=1 rev1=1 crypto rx=0x7fc2f00097a0 tx=0x7fc2f002b8b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ac3740310dd45f01 server_cookie=0 in_seq=0 out_seq=0 2023-08-09T13:35:25.103 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:25.100+0000 7fc2f7fff640 1 -- 172.21.15.43:0/1668880745 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc2f002d040 con 0x7fc3000d0a70 2023-08-09T13:35:25.103 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:25.100+0000 7fc2f7fff640 1 -- 172.21.15.43:0/1668880745 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc2f0031810 con 0x7fc3000d0a70 2023-08-09T13:35:25.103 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:25.100+0000 7fc2f7fff640 1 -- 172.21.15.43:0/1668880745 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc2f0031ba0 con 0x7fc3000d0a70 2023-08-09T13:35:25.103 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:25.101+0000 7fc307364640 1 -- 172.21.15.43:0/1668880745 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc3000d0a70 msgr2=0x7fc3000cd7f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:35:25.104 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:25.101+0000 7fc307364640 1 --2- 172.21.15.43:0/1668880745 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc3000d0a70 0x7fc3000cd7f0 secure :-1 s=READY pgs=347 cs=0 l=1 rev1=1 crypto rx=0x7fc2f00097a0 tx=0x7fc2f002b8b0 comp rx=0 tx=0).stop 2023-08-09T13:35:25.104 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:25.101+0000 7fc307364640 1 -- 172.21.15.43:0/1668880745 shutdown_connections 2023-08-09T13:35:25.104 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:25.101+0000 7fc307364640 1 --2- 172.21.15.43:0/1668880745 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fc3000cdd30 0x7fc3000b3fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:35:25.105 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:25.101+0000 7fc307364640 1 --2- 172.21.15.43:0/1668880745 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc3000d0a70 0x7fc3000cd7f0 unknown :-1 s=CLOSED pgs=347 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:35:25.105 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:25.101+0000 7fc307364640 1 -- 172.21.15.43:0/1668880745 >> 172.21.15.43:0/1668880745 conn(0x7fc300071520 msgr2=0x7fc3000d2520 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-09T13:35:25.105 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:25.101+0000 7fc307364640 1 -- 172.21.15.43:0/1668880745 shutdown_connections 2023-08-09T13:35:25.105 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:25.101+0000 7fc307364640 1 -- 172.21.15.43:0/1668880745 wait complete. 2023-08-09T13:35:25.106 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:25.103+0000 7fc307364640 1 Processor -- start 2023-08-09T13:35:25.106 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:25.103+0000 7fc307364640 1 -- start start 2023-08-09T13:35:25.106 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:25.104+0000 7fc307364640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fc3000d0a70 0x7fc30015ee80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:35:25.107 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:25.104+0000 7fc307364640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fc30015c680 0x7fc3001f1690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:35:25.107 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:25.104+0000 7fc307364640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc3000ca640 0x7fc3000cf180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:35:25.107 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:25.104+0000 7fc307364640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc3000cb750 con 0x7fc3000ca640 2023-08-09T13:35:25.107 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:25.104+0000 7fc307364640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc3000cab40 con 0x7fc3000d0a70 2023-08-09T13:35:25.107 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:25.104+0000 7fc307364640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc3000cae20 con 0x7fc30015c680 2023-08-09T13:35:25.109 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:25.104+0000 7fc305901640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc3000ca640 0x7fc3000cf180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:35:25.109 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:25.105+0000 7fc305901640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc3000ca640 0x7fc3000cf180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:46916/0 (socket says 172.21.15.43:46916) 2023-08-09T13:35:25.109 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:25.105+0000 7fc305901640 1 -- 172.21.15.43:0/1496221125 learned_addr learned my addr 172.21.15.43:0/1496221125 (peer_addr_for_me v2:172.21.15.43:0/0) 2023-08-09T13:35:25.109 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:25.105+0000 7fc3048ff640 1 --2- 172.21.15.43:0/1496221125 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fc30015c680 0x7fc3001f1690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:35:25.109 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:25.105+0000 7fc305100640 1 --2- 172.21.15.43:0/1496221125 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fc3000d0a70 0x7fc30015ee80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:35:25.110 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:25.105+0000 7fc305901640 1 -- 172.21.15.43:0/1496221125 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fc30015c680 msgr2=0x7fc3001f1690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:35:25.110 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:25.105+0000 7fc305901640 1 --2- 172.21.15.43:0/1496221125 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fc30015c680 0x7fc3001f1690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:35:25.110 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:25.105+0000 7fc305901640 1 -- 172.21.15.43:0/1496221125 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fc3000d0a70 msgr2=0x7fc30015ee80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:35:25.110 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:25.105+0000 7fc305901640 1 --2- 172.21.15.43:0/1496221125 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fc3000d0a70 0x7fc30015ee80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:35:25.110 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:25.105+0000 7fc305901640 1 -- 172.21.15.43:0/1496221125 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc2f0009400 con 0x7fc3000ca640 2023-08-09T13:35:25.110 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:25.105+0000 7fc305901640 1 --2- 172.21.15.43:0/1496221125 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc3000ca640 0x7fc3000cf180 secure :-1 s=READY pgs=348 cs=0 l=1 rev1=1 crypto rx=0x7fc2f8007e30 tx=0x7fc2f800e4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-09T13:35:25.110 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:25.106+0000 7fc2f5ffb640 1 -- 172.21.15.43:0/1496221125 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc2f8014620 con 0x7fc3000ca640 2023-08-09T13:35:25.111 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:25.106+0000 7fc2f5ffb640 1 -- 172.21.15.43:0/1496221125 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc2f800ee90 con 0x7fc3000ca640 2023-08-09T13:35:25.111 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:25.106+0000 7fc2f5ffb640 1 -- 172.21.15.43:0/1496221125 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc2f80149c0 con 0x7fc3000ca640 2023-08-09T13:35:25.111 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:25.106+0000 7fc307364640 1 -- 172.21.15.43:0/1496221125 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc3000cb100 con 0x7fc3000ca640 2023-08-09T13:35:25.111 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:25.106+0000 7fc307364640 1 -- 172.21.15.43:0/1496221125 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc30015f3c0 con 0x7fc3000ca640 2023-08-09T13:35:25.112 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:25.107+0000 7fc307364640 1 -- 172.21.15.43:0/1496221125 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc3000739f0 con 0x7fc3000ca640 2023-08-09T13:35:25.112 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:25.108+0000 7fc2f5ffb640 1 -- 172.21.15.43:0/1496221125 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 5) v1 ==== 88341+0+0 (secure 0 0 0) 0x7fc2f8014b20 con 0x7fc3000ca640 2023-08-09T13:35:25.112 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:25.109+0000 7fc2f5ffb640 1 --2- 172.21.15.43:0/1496221125 >> [v2:172.21.15.203:6832/830097474,v1:172.21.15.203:6833/830097474] conn(0x7fc2e406a050 0x7fc2e406c510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:35:25.112 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:25.109+0000 7fc2f5ffb640 1 -- 172.21.15.43:0/1496221125 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 6871+0+0 (secure 0 0 0) 0x7fc2f8089280 con 0x7fc3000ca640 2023-08-09T13:35:25.112 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:25.110+0000 7fc305100640 1 --2- 172.21.15.43:0/1496221125 >> [v2:172.21.15.203:6832/830097474,v1:172.21.15.203:6833/830097474] conn(0x7fc2e406a050 0x7fc2e406c510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:35:25.113 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:25.111+0000 7fc305100640 1 --2- 172.21.15.43:0/1496221125 >> [v2:172.21.15.203:6832/830097474,v1:172.21.15.203:6833/830097474] conn(0x7fc2e406a050 0x7fc2e406c510 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7fc2fc005880 tx=0x7fc2fc009040 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-09T13:35:25.115 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:25.113+0000 7fc2f5ffb640 1 -- 172.21.15.43:0/1496221125 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178066 (secure 0 0 0) 0x7fc2f80545a0 con 0x7fc3000ca640 2023-08-09T13:35:25.309 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:25.306+0000 7fc307364640 1 -- 172.21.15.43:0/1496221125 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) v1 -- 0x7fc30015fe60 con 0x7fc3000ca640 2023-08-09T13:35:25.310 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:25.307+0000 7fc2f5ffb640 1 -- 172.21.15.43:0/1496221125 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v26) v1 ==== 103+0+11 (secure 0 0 0) 0x7fc2f8057bc0 con 0x7fc3000ca640 2023-08-09T13:35:25.310 INFO:teuthology.orchestra.run.smithi043.stdout:pg_num: 32 2023-08-09T13:35:25.313 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:25.310+0000 7fc307364640 1 -- 172.21.15.43:0/1496221125 >> [v2:172.21.15.203:6832/830097474,v1:172.21.15.203:6833/830097474] conn(0x7fc2e406a050 msgr2=0x7fc2e406c510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:35:25.313 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:25.310+0000 7fc307364640 1 --2- 172.21.15.43:0/1496221125 >> [v2:172.21.15.203:6832/830097474,v1:172.21.15.203:6833/830097474] conn(0x7fc2e406a050 0x7fc2e406c510 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7fc2fc005880 tx=0x7fc2fc009040 comp rx=0 tx=0).stop 2023-08-09T13:35:25.313 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:25.310+0000 7fc307364640 1 -- 172.21.15.43:0/1496221125 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc3000ca640 msgr2=0x7fc3000cf180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:35:25.314 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:25.310+0000 7fc307364640 1 --2- 172.21.15.43:0/1496221125 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc3000ca640 0x7fc3000cf180 secure :-1 s=READY pgs=348 cs=0 l=1 rev1=1 crypto rx=0x7fc2f8007e30 tx=0x7fc2f800e4b0 comp rx=0 tx=0).stop 2023-08-09T13:35:25.314 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:25.310+0000 7fc307364640 1 -- 172.21.15.43:0/1496221125 shutdown_connections 2023-08-09T13:35:25.314 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:25.310+0000 7fc307364640 1 --2- 172.21.15.43:0/1496221125 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc3000ca640 0x7fc3000cf180 unknown :-1 s=CLOSED pgs=348 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:35:25.314 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:25.310+0000 7fc307364640 1 --2- 172.21.15.43:0/1496221125 >> [v2:172.21.15.203:6832/830097474,v1:172.21.15.203:6833/830097474] conn(0x7fc2e406a050 0x7fc2e406c510 unknown :-1 s=CLOSED pgs=183 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:35:25.314 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:25.310+0000 7fc307364640 1 --2- 172.21.15.43:0/1496221125 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fc30015c680 0x7fc3001f1690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:35:25.314 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:25.310+0000 7fc307364640 1 --2- 172.21.15.43:0/1496221125 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fc3000d0a70 0x7fc30015ee80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:35:25.314 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:25.311+0000 7fc307364640 1 -- 172.21.15.43:0/1496221125 >> 172.21.15.43:0/1496221125 conn(0x7fc300071520 msgr2=0x7fc3000b7660 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-09T13:35:25.315 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:25.311+0000 7fc307364640 1 -- 172.21.15.43:0/1496221125 shutdown_connections 2023-08-09T13:35:25.315 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:25.311+0000 7fc307364640 1 -- 172.21.15.43:0/1496221125 wait complete. 2023-08-09T13:35:25.331 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2023-08-09T13:35:25.561 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:25.558+0000 7ff02dee0640 1 Processor -- start 2023-08-09T13:35:25.561 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:25.559+0000 7ff02dee0640 1 -- start start 2023-08-09T13:35:25.563 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:25.560+0000 7ff02dee0640 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7ff02815f7b0 0x7ff0280b4760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:35:25.563 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:25.560+0000 7ff02dee0640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7ff0280b1050 0x7ff0280ccfa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:35:25.563 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:25.560+0000 7ff02dee0640 1 -- --> v1:172.21.15.203:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff0280b7bf0 con 0x7ff0280cd4e0 2023-08-09T13:35:25.563 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:25.561+0000 7ff02dee0640 1 -- --> v2:172.21.15.43:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff0280cd970 con 0x7ff0280b1050 2023-08-09T13:35:25.564 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:25.561+0000 7ff02dee0640 1 -- --> v2:172.21.15.203:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff0280b7ed0 con 0x7ff02815f7b0 2023-08-09T13:35:25.564 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:25.561+0000 7ff027fff640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7ff0280b1050 0x7ff0280ccfa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:35:25.565 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:25.561+0000 7ff027fff640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7ff0280b1050 0x7ff0280ccfa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:46926/0 (socket says 172.21.15.43:46926) 2023-08-09T13:35:25.565 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:25.561+0000 7ff027fff640 1 -- 172.21.15.43:0/3988278057 learned_addr learned my addr 172.21.15.43:0/3988278057 (peer_addr_for_me v2:172.21.15.43:0/0) 2023-08-09T13:35:25.565 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:25.561+0000 7ff02cede640 1 --2- 172.21.15.43:0/3988278057 >> v2:172.21.15.203:3300/0 conn(0x7ff02815f7b0 0x7ff0280b4760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:35:25.565 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:25.561+0000 7ff027fff640 1 -- 172.21.15.43:0/3988278057 >> v2:172.21.15.203:3300/0 conn(0x7ff02815f7b0 msgr2=0x7ff0280b4760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-09T13:35:25.565 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:25.561+0000 7ff027fff640 1 --2- 172.21.15.43:0/3988278057 >> v2:172.21.15.203:3300/0 conn(0x7ff02815f7b0 0x7ff0280b4760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:35:25.565 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:25.561+0000 7ff027fff640 1 -- 172.21.15.43:0/3988278057 >> v1:172.21.15.203:6789/0 conn(0x7ff0280cd4e0 legacy=0x7ff0280ceae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-09T13:35:25.566 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:25.562+0000 7ff027fff640 1 -- 172.21.15.43:0/3988278057 --> v2:172.21.15.43:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff0280d0370 con 0x7ff0280b1050 2023-08-09T13:35:25.566 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:25.563+0000 7ff027fff640 1 --2- 172.21.15.43:0/3988278057 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff0280b1050 0x7ff0280ccfa0 secure :-1 s=READY pgs=349 cs=0 l=1 rev1=1 crypto rx=0x7ff02808aa40 tx=0x7ff01402b910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5c0d3fade36bd751 server_cookie=0 in_seq=0 out_seq=0 2023-08-09T13:35:25.567 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:25.563+0000 7ff0277fe640 1 -- 172.21.15.43:0/3988278057 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff014031870 con 0x7ff0280b1050 2023-08-09T13:35:25.567 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:25.563+0000 7ff0277fe640 1 -- 172.21.15.43:0/3988278057 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff0140319d0 con 0x7ff0280b1050 2023-08-09T13:35:25.567 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:25.563+0000 7ff0277fe640 1 -- 172.21.15.43:0/3988278057 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff014031d60 con 0x7ff0280b1050 2023-08-09T13:35:25.567 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:25.564+0000 7ff02dee0640 1 -- 172.21.15.43:0/3988278057 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff0280b1050 msgr2=0x7ff0280ccfa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:35:25.567 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:25.564+0000 7ff02dee0640 1 --2- 172.21.15.43:0/3988278057 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff0280b1050 0x7ff0280ccfa0 secure :-1 s=READY pgs=349 cs=0 l=1 rev1=1 crypto rx=0x7ff02808aa40 tx=0x7ff01402b910 comp rx=0 tx=0).stop 2023-08-09T13:35:25.567 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:25.564+0000 7ff02dee0640 1 -- 172.21.15.43:0/3988278057 shutdown_connections 2023-08-09T13:35:25.567 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:25.564+0000 7ff02dee0640 1 --2- 172.21.15.43:0/3988278057 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff0280b1050 0x7ff0280ccfa0 unknown :-1 s=CLOSED pgs=349 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:35:25.568 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:25.564+0000 7ff02dee0640 1 --2- 172.21.15.43:0/3988278057 >> v2:172.21.15.203:3300/0 conn(0x7ff02815f7b0 0x7ff0280b4760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:35:25.568 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:25.564+0000 7ff02dee0640 1 -- 172.21.15.43:0/3988278057 >> 172.21.15.43:0/3988278057 conn(0x7ff02806a2b0 msgr2=0x7ff02806a690 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-09T13:35:25.568 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:25.564+0000 7ff02dee0640 1 -- 172.21.15.43:0/3988278057 shutdown_connections 2023-08-09T13:35:25.568 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:25.564+0000 7ff02dee0640 1 -- 172.21.15.43:0/3988278057 wait complete. 2023-08-09T13:35:25.569 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:25.566+0000 7ff02dee0640 1 Processor -- start 2023-08-09T13:35:25.569 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:25.566+0000 7ff02dee0640 1 -- start start 2023-08-09T13:35:25.571 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:25.567+0000 7ff02dee0640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7ff0280b1050 0x7ff0281f5e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:35:25.572 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:25.567+0000 7ff02dee0640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff0280cd4e0 0x7ff0280cc3d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:35:25.572 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:25.567+0000 7ff02dee0640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7ff02815f7b0 0x7ff0280cc910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:35:25.572 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:25.567+0000 7ff02dee0640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff0280ccf70 con 0x7ff0280cd4e0 2023-08-09T13:35:25.572 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:25.567+0000 7ff02dee0640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff0281f5370 con 0x7ff0280b1050 2023-08-09T13:35:25.573 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:25.567+0000 7ff02dee0640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff0281f5650 con 0x7ff02815f7b0 2023-08-09T13:35:25.573 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:25.567+0000 7ff027fff640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff0280cd4e0 0x7ff0280cc3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:35:25.573 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:25.567+0000 7ff027fff640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff0280cd4e0 0x7ff0280cc3d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:46936/0 (socket says 172.21.15.43:46936) 2023-08-09T13:35:25.574 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:25.567+0000 7ff027fff640 1 -- 172.21.15.43:0/516351293 learned_addr learned my addr 172.21.15.43:0/516351293 (peer_addr_for_me v2:172.21.15.43:0/0) 2023-08-09T13:35:25.574 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:25.568+0000 7ff02cede640 1 --2- 172.21.15.43:0/516351293 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7ff0280b1050 0x7ff0281f5e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:35:25.574 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:25.568+0000 7ff02d6df640 1 --2- 172.21.15.43:0/516351293 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7ff02815f7b0 0x7ff0280cc910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:35:25.575 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:25.568+0000 7ff027fff640 1 -- 172.21.15.43:0/516351293 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7ff02815f7b0 msgr2=0x7ff0280cc910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:35:25.575 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:25.568+0000 7ff027fff640 1 --2- 172.21.15.43:0/516351293 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7ff02815f7b0 0x7ff0280cc910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:35:25.575 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:25.568+0000 7ff027fff640 1 -- 172.21.15.43:0/516351293 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7ff0280b1050 msgr2=0x7ff0281f5e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:35:25.575 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:25.568+0000 7ff027fff640 1 --2- 172.21.15.43:0/516351293 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7ff0280b1050 0x7ff0281f5e50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:35:25.576 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:25.568+0000 7ff027fff640 1 -- 172.21.15.43:0/516351293 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff0140094d0 con 0x7ff0280cd4e0 2023-08-09T13:35:25.576 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:25.568+0000 7ff027fff640 1 --2- 172.21.15.43:0/516351293 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff0280cd4e0 0x7ff0280cc3d0 secure :-1 s=READY pgs=350 cs=0 l=1 rev1=1 crypto rx=0x7ff014000c00 tx=0x7ff0140046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-09T13:35:25.576 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:25.568+0000 7ff0257fa640 1 -- 172.21.15.43:0/516351293 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff01403abe0 con 0x7ff0280cd4e0 2023-08-09T13:35:25.577 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:25.569+0000 7ff0257fa640 1 -- 172.21.15.43:0/516351293 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff01403ad40 con 0x7ff0280cd4e0 2023-08-09T13:35:25.577 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:25.569+0000 7ff0257fa640 1 -- 172.21.15.43:0/516351293 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff014044610 con 0x7ff0280cd4e0 2023-08-09T13:35:25.577 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:25.569+0000 7ff02dee0640 1 -- 172.21.15.43:0/516351293 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff0281f5930 con 0x7ff0280cd4e0 2023-08-09T13:35:25.577 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:25.569+0000 7ff02dee0640 1 -- 172.21.15.43:0/516351293 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff0280c7690 con 0x7ff0280cd4e0 2023-08-09T13:35:25.579 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:25.570+0000 7ff02dee0640 1 -- 172.21.15.43:0/516351293 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff0280739f0 con 0x7ff0280cd4e0 2023-08-09T13:35:25.579 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:25.571+0000 7ff0257fa640 1 -- 172.21.15.43:0/516351293 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 5) v1 ==== 88341+0+0 (secure 0 0 0) 0x7ff01404d450 con 0x7ff0280cd4e0 2023-08-09T13:35:25.579 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:25.572+0000 7ff0257fa640 1 --2- 172.21.15.43:0/516351293 >> [v2:172.21.15.203:6832/830097474,v1:172.21.15.203:6833/830097474] conn(0x7ff00c06a050 0x7ff00c06c510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:35:25.580 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:25.572+0000 7ff0257fa640 1 -- 172.21.15.43:0/516351293 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 6871+0+0 (secure 0 0 0) 0x7ff01402d040 con 0x7ff0280cd4e0 2023-08-09T13:35:25.580 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:25.572+0000 7ff02cede640 1 --2- 172.21.15.43:0/516351293 >> [v2:172.21.15.203:6832/830097474,v1:172.21.15.203:6833/830097474] conn(0x7ff00c06a050 0x7ff00c06c510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:35:25.580 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:25.574+0000 7ff02cede640 1 --2- 172.21.15.43:0/516351293 >> [v2:172.21.15.203:6832/830097474,v1:172.21.15.203:6833/830097474] conn(0x7ff00c06a050 0x7ff00c06c510 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7ff01c006fd0 tx=0x7ff01c008040 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-09T13:35:25.580 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:25.576+0000 7ff0257fa640 1 -- 172.21.15.43:0/516351293 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178066 (secure 0 0 0) 0x7ff014076330 con 0x7ff0280cd4e0 2023-08-09T13:35:25.789 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:25.786+0000 7ff02dee0640 1 -- 172.21.15.43:0/516351293 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7ff0280c7c50 con 0x7ff0280cd4e0 2023-08-09T13:35:25.790 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:25.787+0000 7ff0257fa640 1 -- 172.21.15.43:0/516351293 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v26) v1 ==== 80+0+0 (secure 0 0 0) 0x7ff014035020 con 0x7ff0280cd4e0 2023-08-09T13:35:25.790 INFO:teuthology.orchestra.run.smithi043.stderr:listed 0 entries 2023-08-09T13:35:25.793 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:25.790+0000 7ff02dee0640 1 -- 172.21.15.43:0/516351293 >> [v2:172.21.15.203:6832/830097474,v1:172.21.15.203:6833/830097474] conn(0x7ff00c06a050 msgr2=0x7ff00c06c510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:35:25.793 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:25.790+0000 7ff02dee0640 1 --2- 172.21.15.43:0/516351293 >> [v2:172.21.15.203:6832/830097474,v1:172.21.15.203:6833/830097474] conn(0x7ff00c06a050 0x7ff00c06c510 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7ff01c006fd0 tx=0x7ff01c008040 comp rx=0 tx=0).stop 2023-08-09T13:35:25.793 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:25.790+0000 7ff02dee0640 1 -- 172.21.15.43:0/516351293 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff0280cd4e0 msgr2=0x7ff0280cc3d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:35:25.793 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:25.790+0000 7ff02dee0640 1 --2- 172.21.15.43:0/516351293 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff0280cd4e0 0x7ff0280cc3d0 secure :-1 s=READY pgs=350 cs=0 l=1 rev1=1 crypto rx=0x7ff014000c00 tx=0x7ff0140046c0 comp rx=0 tx=0).stop 2023-08-09T13:35:25.794 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:25.790+0000 7ff02dee0640 1 -- 172.21.15.43:0/516351293 shutdown_connections 2023-08-09T13:35:25.794 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:25.790+0000 7ff02dee0640 1 --2- 172.21.15.43:0/516351293 >> [v2:172.21.15.203:6832/830097474,v1:172.21.15.203:6833/830097474] conn(0x7ff00c06a050 0x7ff00c06c510 unknown :-1 s=CLOSED pgs=184 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:35:25.794 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:25.790+0000 7ff02dee0640 1 --2- 172.21.15.43:0/516351293 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7ff02815f7b0 0x7ff0280cc910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:35:25.794 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:25.790+0000 7ff02dee0640 1 --2- 172.21.15.43:0/516351293 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff0280cd4e0 0x7ff0280cc3d0 unknown :-1 s=CLOSED pgs=350 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:35:25.794 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:25.790+0000 7ff02dee0640 1 --2- 172.21.15.43:0/516351293 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7ff0280b1050 0x7ff0281f5e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:35:25.794 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:25.790+0000 7ff02dee0640 1 -- 172.21.15.43:0/516351293 >> 172.21.15.43:0/516351293 conn(0x7ff02806a2b0 msgr2=0x7ff0280b3b70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-09T13:35:25.794 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:25.790+0000 7ff02dee0640 1 -- 172.21.15.43:0/516351293 shutdown_connections 2023-08-09T13:35:25.795 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:25.791+0000 7ff02dee0640 1 -- 172.21.15.43:0/516351293 wait complete. 2023-08-09T13:35:25.813 DEBUG:teuthology.orchestra.run.smithi043:> set -ex 2023-08-09T13:35:25.813 DEBUG:teuthology.orchestra.run.smithi043:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-09T13:35:25.867 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-08-09T13:35:26.098 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:26.095+0000 7f11cbe1f640 1 Processor -- start 2023-08-09T13:35:26.098 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:26.096+0000 7f11cbe1f640 1 -- start start 2023-08-09T13:35:26.100 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:26.097+0000 7f11cbe1f640 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7f11c40b2b70 0x7f11c40d2ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:35:26.100 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:26.097+0000 7f11cbe1f640 1 -- --> v1:172.21.15.43:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f11c40ce090 con 0x7f11c40b3050 2023-08-09T13:35:26.100 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:26.097+0000 7f11cbe1f640 1 -- --> v1:172.21.15.203:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f11c40cd7f0 con 0x7f11c40b2790 2023-08-09T13:35:26.100 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:26.097+0000 7f11cbe1f640 1 -- --> v2:172.21.15.203:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f11c40b3950 con 0x7f11c40b2b70 2023-08-09T13:35:26.101 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:26.098+0000 7f11ca3bc640 1 --1- >> v1:172.21.15.43:6789/0 conn(0x7f11c40b3050 0x7f11c40b3f60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.43:6789/0 says I am v1:172.21.15.43:52196/0 (socket says 172.21.15.43:52196) 2023-08-09T13:35:26.101 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:26.098+0000 7f11ca3bc640 1 -- 172.21.15.43:0/3527770837 learned_addr learned my addr 172.21.15.43:0/3527770837 (peer_addr_for_me v1:172.21.15.43:0/0) 2023-08-09T13:35:26.101 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:26.098+0000 7f11c93ba640 1 --2- 172.21.15.43:0/3527770837 >> v2:172.21.15.203:3300/0 conn(0x7f11c40b2b70 0x7f11c40d2ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:35:26.102 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:26.098+0000 7f11c8bb9640 1 -- 172.21.15.43:0/3527770837 <== mon.0 v1:172.21.15.43:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2667382140 0 0) 0x7f11c40ce090 con 0x7f11c40b3050 2023-08-09T13:35:26.102 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:26.098+0000 7f11c8bb9640 1 -- 172.21.15.43:0/3527770837 --> v1:172.21.15.43:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f11b0003520 con 0x7f11c40b3050 2023-08-09T13:35:26.102 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:26.099+0000 7f11c8bb9640 1 -- 172.21.15.43:0/3527770837 <== mon.0 v1:172.21.15.43:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1740412206 0 0) 0x7f11b0003520 con 0x7f11c40b3050 2023-08-09T13:35:26.102 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:26.099+0000 7f11c8bb9640 1 -- 172.21.15.43:0/3527770837 >> v2:172.21.15.203:3300/0 conn(0x7f11c40b2b70 msgr2=0x7f11c40d2ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-09T13:35:26.102 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:26.099+0000 7f11c8bb9640 1 --2- 172.21.15.43:0/3527770837 >> v2:172.21.15.203:3300/0 conn(0x7f11c40b2b70 0x7f11c40d2ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:35:26.102 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:26.099+0000 7f11c8bb9640 1 -- 172.21.15.43:0/3527770837 >> v1:172.21.15.203:6789/0 conn(0x7f11c40b2790 legacy=0x7f11c40b5330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:35:26.102 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:26.099+0000 7f11c8bb9640 1 -- 172.21.15.43:0/3527770837 --> v1:172.21.15.43:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f11c40cdad0 con 0x7f11c40b3050 2023-08-09T13:35:26.103 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:26.099+0000 7f11c8bb9640 1 -- 172.21.15.43:0/3527770837 <== mon.0 v1:172.21.15.43:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1285150740 0 0) 0x7f11bc001670 con 0x7f11c40b3050 2023-08-09T13:35:26.103 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:26.099+0000 7f11c8bb9640 1 -- 172.21.15.43:0/3527770837 >> v1:172.21.15.43:6789/0 conn(0x7f11c40b3050 legacy=0x7f11c40b3f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:35:26.103 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:26.099+0000 7f11c8bb9640 1 --2- 172.21.15.43:0/3527770837 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f11b00039e0 0x7f11b0003e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:35:26.103 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:26.099+0000 7f11c8bb9640 1 --2- 172.21.15.43:0/3527770837 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f11b00047c0 0x7f11b0004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:35:26.103 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:26.099+0000 7f11c8bb9640 1 --2- 172.21.15.43:0/3527770837 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f11b0005180 0x7f11b000b980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:35:26.103 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:26.099+0000 7f11c8bb9640 1 -- 172.21.15.43:0/3527770837 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f11c40b3950 con 0x7f11b00039e0 2023-08-09T13:35:26.103 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:26.099+0000 7f11c8bb9640 1 -- 172.21.15.43:0/3527770837 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f11b000c0b0 con 0x7f11b00047c0 2023-08-09T13:35:26.103 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:26.099+0000 7f11c8bb9640 1 -- 172.21.15.43:0/3527770837 --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f11b000c270 con 0x7f11b0005180 2023-08-09T13:35:26.105 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:26.099+0000 7f11c9bbb640 1 --2- 172.21.15.43:0/3527770837 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f11b00039e0 0x7f11b0003e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:35:26.105 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:26.100+0000 7f11ca3bc640 1 --2- 172.21.15.43:0/3527770837 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f11b0005180 0x7f11b000b980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:35:26.105 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:26.100+0000 7f11c93ba640 1 --2- 172.21.15.43:0/3527770837 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f11b00047c0 0x7f11b0004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:35:26.105 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:26.100+0000 7f11c9bbb640 1 -- 172.21.15.43:0/3527770837 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f11b0005180 msgr2=0x7f11b000b980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-09T13:35:26.105 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:26.100+0000 7f11c9bbb640 1 --2- 172.21.15.43:0/3527770837 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f11b0005180 0x7f11b000b980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:35:26.105 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:26.100+0000 7f11c9bbb640 1 -- 172.21.15.43:0/3527770837 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f11b00047c0 msgr2=0x7f11b0004c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-09T13:35:26.105 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:26.100+0000 7f11c9bbb640 1 --2- 172.21.15.43:0/3527770837 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f11b00047c0 0x7f11b0004c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:35:26.106 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:26.100+0000 7f11c9bbb640 1 -- 172.21.15.43:0/3527770837 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f11b0003520 con 0x7f11b00039e0 2023-08-09T13:35:26.106 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:26.100+0000 7f11ca3bc640 1 -- 172.21.15.43:0/3527770837 reap_dead start 2023-08-09T13:35:26.106 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:26.101+0000 7f11c9bbb640 1 --2- 172.21.15.43:0/3527770837 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f11b00039e0 0x7f11b0003e00 secure :-1 s=READY pgs=352 cs=0 l=1 rev1=1 crypto rx=0x7f11c0001e50 tx=0x7f11b00057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ef0a8e717ff508a4 server_cookie=0 in_seq=0 out_seq=0 2023-08-09T13:35:26.106 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:26.102+0000 7f11c8bb9640 1 -- 172.21.15.43:0/3527770837 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f11c0003b70 con 0x7f11b00039e0 2023-08-09T13:35:26.106 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:26.102+0000 7f11c8bb9640 1 -- 172.21.15.43:0/3527770837 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f11c0003cd0 con 0x7f11b00039e0 2023-08-09T13:35:26.106 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:26.102+0000 7f11c8bb9640 1 -- 172.21.15.43:0/3527770837 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f11c0005310 con 0x7f11b00039e0 2023-08-09T13:35:26.106 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:26.102+0000 7f11cbe1f640 1 -- 172.21.15.43:0/3527770837 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f11b00039e0 msgr2=0x7f11b0003e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:35:26.107 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:26.102+0000 7f11cbe1f640 1 --2- 172.21.15.43:0/3527770837 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f11b00039e0 0x7f11b0003e00 secure :-1 s=READY pgs=352 cs=0 l=1 rev1=1 crypto rx=0x7f11c0001e50 tx=0x7f11b00057f0 comp rx=0 tx=0).stop 2023-08-09T13:35:26.107 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:26.103+0000 7f11cbe1f640 1 -- 172.21.15.43:0/3527770837 shutdown_connections 2023-08-09T13:35:26.107 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:26.103+0000 7f11cbe1f640 1 --2- 172.21.15.43:0/3527770837 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f11b00039e0 0x7f11b0003e00 unknown :-1 s=CLOSED pgs=352 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:35:26.108 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:26.103+0000 7f11cbe1f640 1 -- 172.21.15.43:0/3527770837 >> 172.21.15.43:0/3527770837 conn(0x7f11c409b7b0 msgr2=0x7f11c40d24e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-09T13:35:26.108 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:26.103+0000 7f11cbe1f640 1 -- 172.21.15.43:0/3527770837 shutdown_connections 2023-08-09T13:35:26.108 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:26.103+0000 7f11cbe1f640 1 -- 172.21.15.43:0/3527770837 wait complete. 2023-08-09T13:35:26.108 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:26.104+0000 7f11cbe1f640 1 Processor -- start 2023-08-09T13:35:26.108 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:26.105+0000 7f11cbe1f640 1 -- start start 2023-08-09T13:35:26.110 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:26.106+0000 7f11cbe1f640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f11b00039e0 0x7f11c4152220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:35:26.110 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:26.106+0000 7f11cbe1f640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f11c40c8c30 0x7f11c40c64c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:35:26.110 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:26.106+0000 7f11cbe1f640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f11c40c6a00 0x7f11c41f53b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:35:26.110 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:26.106+0000 7f11cbe1f640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f11c40c6eb0 con 0x7f11b00039e0 2023-08-09T13:35:26.110 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:26.106+0000 7f11cbe1f640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f11c4153c30 con 0x7f11c40c6a00 2023-08-09T13:35:26.110 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:26.106+0000 7f11cbe1f640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f11c4152b90 con 0x7f11c40c8c30 2023-08-09T13:35:26.111 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:26.106+0000 7f11c9bbb640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f11b00039e0 0x7f11c4152220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:35:26.111 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:26.106+0000 7f11c9bbb640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f11b00039e0 0x7f11c4152220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:46952/0 (socket says 172.21.15.43:46952) 2023-08-09T13:35:26.111 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:26.106+0000 7f11c9bbb640 1 -- 172.21.15.43:0/1261748044 learned_addr learned my addr 172.21.15.43:0/1261748044 (peer_addr_for_me v2:172.21.15.43:0/0) 2023-08-09T13:35:26.111 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:26.106+0000 7f11c93ba640 1 --2- 172.21.15.43:0/1261748044 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f11c40c8c30 0x7f11c40c64c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:35:26.111 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:26.106+0000 7f11c9bbb640 1 -- 172.21.15.43:0/1261748044 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f11c40c8c30 msgr2=0x7f11c40c64c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:35:26.111 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:26.106+0000 7f11c9bbb640 1 --2- 172.21.15.43:0/1261748044 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f11c40c8c30 0x7f11c40c64c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:35:26.111 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:26.106+0000 7f11c9bbb640 1 -- 172.21.15.43:0/1261748044 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f11c40c6a00 msgr2=0x7f11c41f53b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:35:26.111 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:26.106+0000 7f11c9bbb640 1 --2- 172.21.15.43:0/1261748044 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f11c40c6a00 0x7f11c41f53b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:35:26.112 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:26.106+0000 7f11c9bbb640 1 -- 172.21.15.43:0/1261748044 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f11c4152e70 con 0x7f11b00039e0 2023-08-09T13:35:26.112 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:26.107+0000 7f11c9bbb640 1 --2- 172.21.15.43:0/1261748044 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f11b00039e0 0x7f11c4152220 secure :-1 s=READY pgs=353 cs=0 l=1 rev1=1 crypto rx=0x7f11c0000f80 tx=0x7f11c00014a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-09T13:35:26.112 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:26.107+0000 7f11ba7fc640 1 -- 172.21.15.43:0/1261748044 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f11c003e070 con 0x7f11b00039e0 2023-08-09T13:35:26.112 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:26.107+0000 7f11ba7fc640 1 -- 172.21.15.43:0/1261748044 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f11c002f440 con 0x7f11b00039e0 2023-08-09T13:35:26.112 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:26.107+0000 7f11ba7fc640 1 -- 172.21.15.43:0/1261748044 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f11c002f7d0 con 0x7f11b00039e0 2023-08-09T13:35:26.112 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:26.107+0000 7f11cbe1f640 1 -- 172.21.15.43:0/1261748044 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f11c4153150 con 0x7f11b00039e0 2023-08-09T13:35:26.112 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:26.107+0000 7f11cbe1f640 1 -- 172.21.15.43:0/1261748044 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f11c4153710 con 0x7f11b00039e0 2023-08-09T13:35:26.113 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:26.109+0000 7f11cbe1f640 1 -- 172.21.15.43:0/1261748044 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f11c40c6e50 con 0x7f11b00039e0 2023-08-09T13:35:26.113 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:26.109+0000 7f11ba7fc640 1 -- 172.21.15.43:0/1261748044 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 5) v1 ==== 88341+0+0 (secure 0 0 0) 0x7f11c002f930 con 0x7f11b00039e0 2023-08-09T13:35:26.113 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:26.110+0000 7f11ba7fc640 1 --2- 172.21.15.43:0/1261748044 >> [v2:172.21.15.203:6832/830097474,v1:172.21.15.203:6833/830097474] conn(0x7f11a806a080 0x7f11a806c540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:35:26.113 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:26.110+0000 7f11ba7fc640 1 -- 172.21.15.43:0/1261748044 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 6871+0+0 (secure 0 0 0) 0x7f11c00a9df0 con 0x7f11b00039e0 2023-08-09T13:35:26.114 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:26.111+0000 7f11c93ba640 1 --2- 172.21.15.43:0/1261748044 >> [v2:172.21.15.203:6832/830097474,v1:172.21.15.203:6833/830097474] conn(0x7f11a806a080 0x7f11a806c540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:35:26.114 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:26.112+0000 7f11c93ba640 1 --2- 172.21.15.43:0/1261748044 >> [v2:172.21.15.203:6832/830097474,v1:172.21.15.203:6833/830097474] conn(0x7f11a806a080 0x7f11a806c540 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7f11c4154fe0 tx=0x7f11b4008040 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-09T13:35:26.117 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:26.115+0000 7f11ba7fc640 1 -- 172.21.15.43:0/1261748044 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178066 (secure 0 0 0) 0x7f11c40c6e50 con 0x7f11b00039e0 2023-08-09T13:35:26.318 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:26.316+0000 7f11cbe1f640 1 -- 172.21.15.43:0/1261748044 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f11c40c6e50 con 0x7f11b00039e0 2023-08-09T13:35:26.320 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:26.317+0000 7f11ba7fc640 1 -- 172.21.15.43:0/1261748044 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v26) v1 ==== 74+0+20146 (secure 0 0 0) 0x7f11c40c6e50 con 0x7f11b00039e0 2023-08-09T13:35:26.320 INFO:teuthology.orchestra.run.smithi043.stdout: 2023-08-09T13:35:26.321 INFO:teuthology.orchestra.run.smithi043.stdout:{"epoch":26,"fsid":"daac87ee-1337-4250-a955-aea4e408a39b","created":"2023-08-09T13:33:41.701198+0000","modified":"2023-08-09T13:34:48.711911+0000","last_up_change":"2023-08-09T13:33:53.783453+0000","last_in_change":"2023-08-09T13:33:45.148904+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":5,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":"rbd","create_time":"2023-08-09T13:33:54.365137+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":".mgr","create_time":"2023-08-09T13:33:54.831803+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_metadata","create_time":"2023-08-09T13:34:00.207974+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data","create_time":"2023-08-09T13:34:01.614765+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":5,"pool_name":"cephfs_data_ec","create_time":"2023-08-09T13:34:05.239583+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"24","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"cbd49ecd-e896-4fbd-864e-5c37a829b165","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6800","nonce":502667335},{"type":"v1","addr":"172.21.15.43:6801","nonce":502667335}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6802","nonce":502667335},{"type":"v1","addr":"172.21.15.43:6803","nonce":502667335}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6806","nonce":502667335},{"type":"v1","addr":"172.21.15.43:6807","nonce":502667335}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6804","nonce":502667335},{"type":"v1","addr":"172.21.15.43:6805","nonce":502667335}]},"public_addr":"172.21.15.43:6801/502667335","cluster_addr":"172.21.15.43:6803/502667335","heartbeat_back_addr":"172.21.15.43:6807/502667335","heartbeat_front_addr":"172.21.15.43:6805/502667335","state":["exists","up"]},{"osd":1,"uuid":"75c7e0fe-1ec7-4fe8-a2f2-3661a65b8500","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6808","nonce":3289134610},{"type":"v1","addr":"172.21.15.43:6809","nonce":3289134610}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6810","nonce":3289134610},{"type":"v1","addr":"172.21.15.43:6812","nonce":3289134610}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6821","nonce":3289134610},{"type":"v1","addr":"172.21.15.43:6824","nonce":3289134610}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6815","nonce":3289134610},{"type":"v1","addr":"172.21.15.43:6818","nonce":3289134610}]},"public_addr":"172.21.15.43:6809/3289134610","cluster_addr":"172.21.15.43:6812/3289134610","heartbeat_back_addr":"172.21.15.43:6824/3289134610","heartbeat_front_addr":"172.21.15.43:6818/3289134610","state":["exists","up"]},{"osd":2,"uuid":"bb5c41e9-7f5f-4125-9b01-3bb8bfc2ea83","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6811","nonce":4079709736},{"type":"v1","addr":"172.21.15.43:6814","nonce":4079709736}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6817","nonce":4079709736},{"type":"v1","addr":"172.21.15.43:6820","nonce":4079709736}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6829","nonce":4079709736},{"type":"v1","addr":"172.21.15.43:6831","nonce":4079709736}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6823","nonce":4079709736},{"type":"v1","addr":"172.21.15.43:6826","nonce":4079709736}]},"public_addr":"172.21.15.43:6814/4079709736","cluster_addr":"172.21.15.43:6820/4079709736","heartbeat_back_addr":"172.21.15.43:6831/4079709736","heartbeat_front_addr":"172.21.15.43:6826/4079709736","state":["exists","up"]},{"osd":3,"uuid":"bfe4c8b4-694a-465c-87e6-2b2aab852613","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6813","nonce":1709538759},{"type":"v1","addr":"172.21.15.43:6816","nonce":1709538759}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6819","nonce":1709538759},{"type":"v1","addr":"172.21.15.43:6822","nonce":1709538759}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6828","nonce":1709538759},{"type":"v1","addr":"172.21.15.43:6830","nonce":1709538759}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6825","nonce":1709538759},{"type":"v1","addr":"172.21.15.43:6827","nonce":1709538759}]},"public_addr":"172.21.15.43:6816/1709538759","cluster_addr":"172.21.15.43:6822/1709538759","heartbeat_back_addr":"172.21.15.43:6830/1709538759","heartbeat_front_addr":"172.21.15.43:6827/1709538759","state":["exists","up"]},{"osd":4,"uuid":"fa2e8a77-aba1-4756-853e-0d174a5bdeba","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6800","nonce":4196866481},{"type":"v1","addr":"172.21.15.203:6801","nonce":4196866481}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6802","nonce":4196866481},{"type":"v1","addr":"172.21.15.203:6803","nonce":4196866481}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6806","nonce":4196866481},{"type":"v1","addr":"172.21.15.203:6807","nonce":4196866481}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6804","nonce":4196866481},{"type":"v1","addr":"172.21.15.203:6805","nonce":4196866481}]},"public_addr":"172.21.15.203:6801/4196866481","cluster_addr":"172.21.15.203:6803/4196866481","heartbeat_back_addr":"172.21.15.203:6807/4196866481","heartbeat_front_addr":"172.21.15.203:6805/4196866481","state":["exists","up"]},{"osd":5,"uuid":"81811e4b-4d24-4f51-a82c-f1d700a2a00d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6816","nonce":3059290081},{"type":"v1","addr":"172.21.15.203:6817","nonce":3059290081}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6818","nonce":3059290081},{"type":"v1","addr":"172.21.15.203:6819","nonce":3059290081}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6822","nonce":3059290081},{"type":"v1","addr":"172.21.15.203:6823","nonce":3059290081}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6820","nonce":3059290081},{"type":"v1","addr":"172.21.15.203:6821","nonce":3059290081}]},"public_addr":"172.21.15.203:6817/3059290081","cluster_addr":"172.21.15.203:6819/3059290081","heartbeat_back_addr":"172.21.15.203:6823/3059290081","heartbeat_front_addr":"172.21.15.203:6821/3059290081","state":["exists","up"]},{"osd":6,"uuid":"307bac4d-6a5b-443d-af75-56d695d7674f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6824","nonce":512155569},{"type":"v1","addr":"172.21.15.203:6825","nonce":512155569}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6826","nonce":512155569},{"type":"v1","addr":"172.21.15.203:6827","nonce":512155569}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6830","nonce":512155569},{"type":"v1","addr":"172.21.15.203:6831","nonce":512155569}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6828","nonce":512155569},{"type":"v1","addr":"172.21.15.203:6829","nonce":512155569}]},"public_addr":"172.21.15.203:6825/512155569","cluster_addr":"172.21.15.203:6827/512155569","heartbeat_back_addr":"172.21.15.203:6831/512155569","heartbeat_front_addr":"172.21.15.203:6829/512155569","state":["exists","up"]},{"osd":7,"uuid":"a95dc7ca-9f1b-4295-af70-68a22a2d7d5e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6808","nonce":3331450438},{"type":"v1","addr":"172.21.15.203:6809","nonce":3331450438}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6810","nonce":3331450438},{"type":"v1","addr":"172.21.15.203:6811","nonce":3331450438}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6814","nonce":3331450438},{"type":"v1","addr":"172.21.15.203:6815","nonce":3331450438}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6812","nonce":3331450438},{"type":"v1","addr":"172.21.15.203:6813","nonce":3331450438}]},"public_addr":"172.21.15.203:6809/3331450438","cluster_addr":"172.21.15.203:6811/3331450438","heartbeat_back_addr":"172.21.15.203:6815/3331450438","heartbeat_front_addr":"172.21.15.203:6813/3331450438","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-09T13:33:50.547652+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-09T13:33:50.589469+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-09T13:33:50.561724+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-09T13:33:50.599650+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-09T13:33:50.411855+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-09T13:33:50.219681+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-09T13:33:50.504664+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-09T13:33:50.441248+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"4.14","mappings":[{"from":1,"to":5}]},{"pgid":"4.29","mappings":[{"from":3,"to":2}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-08-09T13:35:26.323 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:26.320+0000 7f11cbe1f640 1 -- 172.21.15.43:0/1261748044 >> [v2:172.21.15.203:6832/830097474,v1:172.21.15.203:6833/830097474] conn(0x7f11a806a080 msgr2=0x7f11a806c540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:35:26.323 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:26.320+0000 7f11cbe1f640 1 --2- 172.21.15.43:0/1261748044 >> [v2:172.21.15.203:6832/830097474,v1:172.21.15.203:6833/830097474] conn(0x7f11a806a080 0x7f11a806c540 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7f11c4154fe0 tx=0x7f11b4008040 comp rx=0 tx=0).stop 2023-08-09T13:35:26.324 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:26.320+0000 7f11cbe1f640 1 -- 172.21.15.43:0/1261748044 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f11b00039e0 msgr2=0x7f11c4152220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:35:26.324 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:26.320+0000 7f11cbe1f640 1 --2- 172.21.15.43:0/1261748044 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f11b00039e0 0x7f11c4152220 secure :-1 s=READY pgs=353 cs=0 l=1 rev1=1 crypto rx=0x7f11c0000f80 tx=0x7f11c00014a0 comp rx=0 tx=0).stop 2023-08-09T13:35:26.324 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:26.321+0000 7f11cbe1f640 1 -- 172.21.15.43:0/1261748044 shutdown_connections 2023-08-09T13:35:26.324 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:26.321+0000 7f11cbe1f640 1 --2- 172.21.15.43:0/1261748044 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f11c40c6a00 0x7f11c41f53b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:35:26.324 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:26.321+0000 7f11cbe1f640 1 --2- 172.21.15.43:0/1261748044 >> [v2:172.21.15.203:6832/830097474,v1:172.21.15.203:6833/830097474] conn(0x7f11a806a080 0x7f11a806c540 unknown :-1 s=CLOSED pgs=185 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:35:26.324 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:26.321+0000 7f11cbe1f640 1 --2- 172.21.15.43:0/1261748044 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f11c40c8c30 0x7f11c40c64c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:35:26.324 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:26.321+0000 7f11cbe1f640 1 --2- 172.21.15.43:0/1261748044 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f11b00039e0 0x7f11c4152220 unknown :-1 s=CLOSED pgs=353 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:35:26.325 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:26.321+0000 7f11cbe1f640 1 -- 172.21.15.43:0/1261748044 >> 172.21.15.43:0/1261748044 conn(0x7f11c409b7b0 msgr2=0x7f11c4155cf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-09T13:35:26.325 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:26.321+0000 7f11cbe1f640 1 -- 172.21.15.43:0/1261748044 shutdown_connections 2023-08-09T13:35:26.325 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:26.321+0000 7f11cbe1f640 1 -- 172.21.15.43:0/1261748044 wait complete. 2023-08-09T13:35:26.342 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': 'rbd', 'create_time': '2023-08-09T13:33:54.365137+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}, 'read_balance': {'score_acting': 2, 'score_stable': 2, 'optimal_score': 1, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': '.mgr', 'create_time': '2023-08-09T13:33:54.831803+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_metadata', 'create_time': '2023-08-09T13:34:00.207974+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data', 'create_time': '2023-08-09T13:34:01.614765+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 2, 'score_stable': 2, 'optimal_score': 1, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 5, 'pool_name': 'cephfs_data_ec', 'create_time': '2023-08-09T13:34:05.239583+0000', 'flags': 5, 'flags_names': 'hashpspool,ec_overwrites', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '24', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2023-08-09T13:35:26.343 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get rbd pg_num 2023-08-09T13:35:26.575 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:26.572+0000 7f65f5144640 1 Processor -- start 2023-08-09T13:35:26.575 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:26.573+0000 7f65f5144640 1 -- start start 2023-08-09T13:35:26.577 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:26.574+0000 7f65f5144640 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7f65f0070ef0 0x7f65f00ab8d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:35:26.577 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:26.574+0000 7f65f5144640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f65f00cc0b0 0x7f65f006f310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:35:26.577 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:26.574+0000 7f65f5144640 1 -- --> v1:172.21.15.43:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f65f00acfa0 con 0x7f65f00abe10 2023-08-09T13:35:26.577 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:26.574+0000 7f65f5144640 1 -- --> v2:172.21.15.43:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f65f015c940 con 0x7f65f00cc0b0 2023-08-09T13:35:26.577 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:26.574+0000 7f65f5144640 1 -- --> v2:172.21.15.203:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f65f00ac390 con 0x7f65f0070ef0 2023-08-09T13:35:26.578 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:26.575+0000 7f65f4943640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f65f00cc0b0 0x7f65f006f310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:35:26.578 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:26.575+0000 7f65f4943640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f65f00cc0b0 0x7f65f006f310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:46958/0 (socket says 172.21.15.43:46958) 2023-08-09T13:35:26.578 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:26.575+0000 7f65f4943640 1 -- 172.21.15.43:0/44174215 learned_addr learned my addr 172.21.15.43:0/44174215 (peer_addr_for_me v2:172.21.15.43:0/0) 2023-08-09T13:35:26.578 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:26.575+0000 7f65effff640 1 --2- 172.21.15.43:0/44174215 >> v2:172.21.15.203:3300/0 conn(0x7f65f0070ef0 0x7f65f00ab8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:35:26.579 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:26.575+0000 7f65f4943640 1 -- 172.21.15.43:0/44174215 >> v2:172.21.15.203:3300/0 conn(0x7f65f0070ef0 msgr2=0x7f65f00ab8d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-09T13:35:26.579 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:26.575+0000 7f65f4943640 1 --2- 172.21.15.43:0/44174215 >> v2:172.21.15.203:3300/0 conn(0x7f65f0070ef0 0x7f65f00ab8d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:35:26.579 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:26.575+0000 7f65f4943640 1 -- 172.21.15.43:0/44174215 >> v1:172.21.15.43:6789/0 conn(0x7f65f00abe10 legacy=0x7f65f00cb9a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:35:26.579 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:26.575+0000 7f65f4943640 1 -- 172.21.15.43:0/44174215 --> v2:172.21.15.43:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f65f00ac670 con 0x7f65f00cc0b0 2023-08-09T13:35:26.580 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:26.577+0000 7f65f4943640 1 --2- 172.21.15.43:0/44174215 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f65f00cc0b0 0x7f65f006f310 secure :-1 s=READY pgs=355 cs=0 l=1 rev1=1 crypto rx=0x7f65f00cb920 tx=0x7f65e802c010 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ba2545e4618cedae server_cookie=0 in_seq=0 out_seq=0 2023-08-09T13:35:26.580 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:26.577+0000 7f65ef7fe640 1 -- 172.21.15.43:0/44174215 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f65e802cbb0 con 0x7f65f00cc0b0 2023-08-09T13:35:26.580 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:26.577+0000 7f65ef7fe640 1 -- 172.21.15.43:0/44174215 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f65e802cd10 con 0x7f65f00cc0b0 2023-08-09T13:35:26.580 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:26.577+0000 7f65ef7fe640 1 -- 172.21.15.43:0/44174215 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f65e8034610 con 0x7f65f00cc0b0 2023-08-09T13:35:26.581 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:26.578+0000 7f65f5144640 1 -- 172.21.15.43:0/44174215 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f65f00cc0b0 msgr2=0x7f65f006f310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:35:26.581 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:26.578+0000 7f65f5144640 1 --2- 172.21.15.43:0/44174215 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f65f00cc0b0 0x7f65f006f310 secure :-1 s=READY pgs=355 cs=0 l=1 rev1=1 crypto rx=0x7f65f00cb920 tx=0x7f65e802c010 comp rx=0 tx=0).stop 2023-08-09T13:35:26.581 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:26.578+0000 7f65f5144640 1 -- 172.21.15.43:0/44174215 shutdown_connections 2023-08-09T13:35:26.581 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:26.578+0000 7f65f5144640 1 --2- 172.21.15.43:0/44174215 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f65f00cc0b0 0x7f65f006f310 unknown :-1 s=CLOSED pgs=355 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:35:26.581 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:26.578+0000 7f65f5144640 1 --2- 172.21.15.43:0/44174215 >> v2:172.21.15.203:3300/0 conn(0x7f65f0070ef0 0x7f65f00ab8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:35:26.581 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:26.578+0000 7f65f5144640 1 -- 172.21.15.43:0/44174215 >> 172.21.15.43:0/44174215 conn(0x7f65f00cddd0 msgr2=0x7f65f00ce1e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-09T13:35:26.582 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:26.578+0000 7f65f5144640 1 -- 172.21.15.43:0/44174215 shutdown_connections 2023-08-09T13:35:26.582 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:26.578+0000 7f65f5144640 1 -- 172.21.15.43:0/44174215 wait complete. 2023-08-09T13:35:26.582 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:26.580+0000 7f65f5144640 1 Processor -- start 2023-08-09T13:35:26.582 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:26.580+0000 7f65f5144640 1 -- start start 2023-08-09T13:35:26.583 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:26.581+0000 7f65f5144640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f65f0070ef0 0x7f65f0154f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:35:26.584 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:26.581+0000 7f65f5144640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f65f00abe10 0x7f65f01f2240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:35:26.584 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:26.581+0000 7f65f5144640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f65f00cc0b0 0x7f65f00ae5f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:35:26.584 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:26.581+0000 7f65f5144640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f65f01554b0 con 0x7f65f00abe10 2023-08-09T13:35:26.584 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:26.581+0000 7f65f5144640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f65f015dc50 con 0x7f65f0070ef0 2023-08-09T13:35:26.584 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:26.581+0000 7f65f5144640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f65f015cbb0 con 0x7f65f00cc0b0 2023-08-09T13:35:26.585 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:26.581+0000 7f65e7fff640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f65f00abe10 0x7f65f01f2240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:35:26.586 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:26.582+0000 7f65e7fff640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f65f00abe10 0x7f65f01f2240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:46962/0 (socket says 172.21.15.43:46962) 2023-08-09T13:35:26.586 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:26.582+0000 7f65e7fff640 1 -- 172.21.15.43:0/386033947 learned_addr learned my addr 172.21.15.43:0/386033947 (peer_addr_for_me v2:172.21.15.43:0/0) 2023-08-09T13:35:26.586 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:26.582+0000 7f65f4943640 1 --2- 172.21.15.43:0/386033947 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f65f00cc0b0 0x7f65f00ae5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:35:26.586 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:26.582+0000 7f65effff640 1 --2- 172.21.15.43:0/386033947 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f65f0070ef0 0x7f65f0154f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:35:26.586 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:26.582+0000 7f65e7fff640 1 -- 172.21.15.43:0/386033947 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f65f00cc0b0 msgr2=0x7f65f00ae5f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:35:26.586 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:26.582+0000 7f65e7fff640 1 --2- 172.21.15.43:0/386033947 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f65f00cc0b0 0x7f65f00ae5f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:35:26.587 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:26.582+0000 7f65e7fff640 1 -- 172.21.15.43:0/386033947 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f65f0070ef0 msgr2=0x7f65f0154f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:35:26.587 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:26.582+0000 7f65e7fff640 1 --2- 172.21.15.43:0/386033947 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f65f0070ef0 0x7f65f0154f70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:35:26.587 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:26.582+0000 7f65e7fff640 1 -- 172.21.15.43:0/386033947 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f65e8009b30 con 0x7f65f00abe10 2023-08-09T13:35:26.587 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:26.582+0000 7f65f4943640 1 --2- 172.21.15.43:0/386033947 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f65f00cc0b0 0x7f65f00ae5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-08-09T13:35:26.587 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:26.582+0000 7f65e7fff640 1 --2- 172.21.15.43:0/386033947 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f65f00abe10 0x7f65f01f2240 secure :-1 s=READY pgs=356 cs=0 l=1 rev1=1 crypto rx=0x7f65d800c400 tx=0x7f65d800c8c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-09T13:35:26.587 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:26.583+0000 7f65ed7fa640 1 -- 172.21.15.43:0/386033947 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f65d80052d0 con 0x7f65f00abe10 2023-08-09T13:35:26.587 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:26.583+0000 7f65ed7fa640 1 -- 172.21.15.43:0/386033947 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f65d8005430 con 0x7f65f00abe10 2023-08-09T13:35:26.587 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:26.583+0000 7f65ed7fa640 1 -- 172.21.15.43:0/386033947 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f65d8003830 con 0x7f65f00abe10 2023-08-09T13:35:26.588 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:26.583+0000 7f65f5144640 1 -- 172.21.15.43:0/386033947 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f65f015ce90 con 0x7f65f00abe10 2023-08-09T13:35:26.588 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:26.583+0000 7f65f5144640 1 -- 172.21.15.43:0/386033947 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f65f015d450 con 0x7f65f00abe10 2023-08-09T13:35:26.588 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:26.584+0000 7f65f5144640 1 -- 172.21.15.43:0/386033947 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f65f004e390 con 0x7f65f00abe10 2023-08-09T13:35:26.588 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:26.585+0000 7f65ed7fa640 1 -- 172.21.15.43:0/386033947 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 5) v1 ==== 88341+0+0 (secure 0 0 0) 0x7f65d8003990 con 0x7f65f00abe10 2023-08-09T13:35:26.589 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:26.586+0000 7f65ed7fa640 1 --2- 172.21.15.43:0/386033947 >> [v2:172.21.15.203:6832/830097474,v1:172.21.15.203:6833/830097474] conn(0x7f65d006a050 0x7f65d006c510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:35:26.589 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:26.587+0000 7f65ed7fa640 1 -- 172.21.15.43:0/386033947 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 6871+0+0 (secure 0 0 0) 0x7f65d8087ed0 con 0x7f65f00abe10 2023-08-09T13:35:26.589 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:26.587+0000 7f65effff640 1 --2- 172.21.15.43:0/386033947 >> [v2:172.21.15.203:6832/830097474,v1:172.21.15.203:6833/830097474] conn(0x7f65d006a050 0x7f65d006c510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:35:26.590 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:26.588+0000 7f65effff640 1 --2- 172.21.15.43:0/386033947 >> [v2:172.21.15.203:6832/830097474,v1:172.21.15.203:6833/830097474] conn(0x7f65d006a050 0x7f65d006c510 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7f65e0006fd0 tx=0x7f65e0008040 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-09T13:35:26.593 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:26.591+0000 7f65ed7fa640 1 -- 172.21.15.43:0/386033947 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178066 (secure 0 0 0) 0x7f65d80531f0 con 0x7f65f00abe10 2023-08-09T13:35:26.788 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:26.786+0000 7f65f5144640 1 -- 172.21.15.43:0/386033947 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"} v 0) v1 -- 0x7f65f00c47b0 con 0x7f65f00abe10 2023-08-09T13:35:26.789 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:26.787+0000 7f65ed7fa640 1 -- 172.21.15.43:0/386033947 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"}]=0 v26) v1 ==== 92+0+10 (secure 0 0 0) 0x7f65d8017020 con 0x7f65f00abe10 2023-08-09T13:35:26.789 INFO:teuthology.orchestra.run.smithi043.stdout:pg_num: 8 2023-08-09T13:35:26.792 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:26.789+0000 7f65f5144640 1 -- 172.21.15.43:0/386033947 >> [v2:172.21.15.203:6832/830097474,v1:172.21.15.203:6833/830097474] conn(0x7f65d006a050 msgr2=0x7f65d006c510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:35:26.792 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:26.789+0000 7f65f5144640 1 --2- 172.21.15.43:0/386033947 >> [v2:172.21.15.203:6832/830097474,v1:172.21.15.203:6833/830097474] conn(0x7f65d006a050 0x7f65d006c510 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7f65e0006fd0 tx=0x7f65e0008040 comp rx=0 tx=0).stop 2023-08-09T13:35:26.792 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:26.789+0000 7f65f5144640 1 -- 172.21.15.43:0/386033947 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f65f00abe10 msgr2=0x7f65f01f2240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:35:26.792 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:26.789+0000 7f65f5144640 1 --2- 172.21.15.43:0/386033947 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f65f00abe10 0x7f65f01f2240 secure :-1 s=READY pgs=356 cs=0 l=1 rev1=1 crypto rx=0x7f65d800c400 tx=0x7f65d800c8c0 comp rx=0 tx=0).stop 2023-08-09T13:35:26.793 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:26.789+0000 7f65f5144640 1 -- 172.21.15.43:0/386033947 shutdown_connections 2023-08-09T13:35:26.793 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:26.789+0000 7f65f5144640 1 --2- 172.21.15.43:0/386033947 >> [v2:172.21.15.203:6832/830097474,v1:172.21.15.203:6833/830097474] conn(0x7f65d006a050 0x7f65d006c510 unknown :-1 s=CLOSED pgs=186 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:35:26.793 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:26.789+0000 7f65f5144640 1 --2- 172.21.15.43:0/386033947 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f65f00cc0b0 0x7f65f00ae5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:35:26.793 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:26.789+0000 7f65f5144640 1 --2- 172.21.15.43:0/386033947 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f65f00abe10 0x7f65f01f2240 unknown :-1 s=CLOSED pgs=356 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:35:26.793 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:26.789+0000 7f65f5144640 1 --2- 172.21.15.43:0/386033947 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f65f0070ef0 0x7f65f0154f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:35:26.793 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:26.789+0000 7f65f5144640 1 -- 172.21.15.43:0/386033947 >> 172.21.15.43:0/386033947 conn(0x7f65f00cddd0 msgr2=0x7f65f00700a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-09T13:35:26.793 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:26.790+0000 7f65f5144640 1 -- 172.21.15.43:0/386033947 shutdown_connections 2023-08-09T13:35:26.793 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:26.790+0000 7f65f5144640 1 -- 172.21.15.43:0/386033947 wait complete. 2023-08-09T13:35:26.812 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-08-09T13:35:27.049 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:27.047+0000 7fee3759e640 1 Processor -- start 2023-08-09T13:35:27.050 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:27.047+0000 7fee3759e640 1 -- start start 2023-08-09T13:35:27.051 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:27.048+0000 7fee3759e640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fee380cd630 0x7fee3806ed40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:35:27.051 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:27.049+0000 7fee3759e640 1 -- --> v1:172.21.15.43:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fee380cc6c0 con 0x7fee3815a1b0 2023-08-09T13:35:27.051 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:27.049+0000 7fee3759e640 1 -- --> v1:172.21.15.203:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fee3815dee0 con 0x7fee380b21e0 2023-08-09T13:35:27.051 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:27.049+0000 7fee3759e640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fee3815e1c0 con 0x7fee380cd630 2023-08-09T13:35:27.052 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:27.049+0000 7fee3659c640 1 --1- >> v1:172.21.15.43:6789/0 conn(0x7fee3815a1b0 0x7fee380b1ad0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.43:6789/0 says I am v1:172.21.15.43:52210/0 (socket says 172.21.15.43:52210) 2023-08-09T13:35:27.052 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:27.049+0000 7fee3659c640 1 -- 172.21.15.43:0/4122998222 learned_addr learned my addr 172.21.15.43:0/4122998222 (peer_addr_for_me v1:172.21.15.43:0/0) 2023-08-09T13:35:27.052 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:27.049+0000 7fee35d9b640 1 --2- 172.21.15.43:0/4122998222 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fee380cd630 0x7fee3806ed40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:35:27.052 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:27.050+0000 7fee3559a640 1 -- 172.21.15.43:0/4122998222 <== mon.0 v1:172.21.15.43:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3840067516 0 0) 0x7fee380cc6c0 con 0x7fee3815a1b0 2023-08-09T13:35:27.053 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:27.050+0000 7fee3559a640 1 -- 172.21.15.43:0/4122998222 --> v1:172.21.15.43:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fee20003520 con 0x7fee3815a1b0 2023-08-09T13:35:27.054 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:27.050+0000 7fee3559a640 1 -- 172.21.15.43:0/4122998222 <== mon.0 v1:172.21.15.43:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 554003864 0 0) 0x7fee20003520 con 0x7fee3815a1b0 2023-08-09T13:35:27.054 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:27.050+0000 7fee3559a640 1 -- 172.21.15.43:0/4122998222 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fee380cd630 msgr2=0x7fee3806ed40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-09T13:35:27.054 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:27.050+0000 7fee3559a640 1 --2- 172.21.15.43:0/4122998222 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fee380cd630 0x7fee3806ed40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:35:27.054 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:27.050+0000 7fee3559a640 1 -- 172.21.15.43:0/4122998222 >> v1:172.21.15.203:6789/0 conn(0x7fee380b21e0 legacy=0x7fee3806f280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:35:27.054 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:27.050+0000 7fee3559a640 1 -- 172.21.15.43:0/4122998222 --> v1:172.21.15.43:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fee3815e4a0 con 0x7fee3815a1b0 2023-08-09T13:35:27.055 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:27.050+0000 7fee3559a640 1 -- 172.21.15.43:0/4122998222 <== mon.0 v1:172.21.15.43:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1285150740 0 0) 0x7fee280019f0 con 0x7fee3815a1b0 2023-08-09T13:35:27.055 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:27.050+0000 7fee3559a640 1 -- 172.21.15.43:0/4122998222 >> v1:172.21.15.43:6789/0 conn(0x7fee3815a1b0 legacy=0x7fee380b1ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:35:27.055 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:27.050+0000 7fee3559a640 1 --2- 172.21.15.43:0/4122998222 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fee200039a0 0x7fee20003dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:35:27.055 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:27.050+0000 7fee3559a640 1 --2- 172.21.15.43:0/4122998222 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fee380cd630 0x7fee200047c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:35:27.055 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:27.050+0000 7fee3559a640 1 --2- 172.21.15.43:0/4122998222 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fee20004d00 0x7fee20005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:35:27.055 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:27.050+0000 7fee3559a640 1 -- 172.21.15.43:0/4122998222 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fee3815e1c0 con 0x7fee200039a0 2023-08-09T13:35:27.055 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:27.050+0000 7fee3559a640 1 -- 172.21.15.43:0/4122998222 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fee2000ba30 con 0x7fee20004d00 2023-08-09T13:35:27.056 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:27.050+0000 7fee3559a640 1 -- 172.21.15.43:0/4122998222 --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fee2000bbf0 con 0x7fee380cd630 2023-08-09T13:35:27.056 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:27.051+0000 7fee3659c640 1 --2- 172.21.15.43:0/4122998222 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fee200039a0 0x7fee20003dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:35:27.056 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:27.051+0000 7fee36d9d640 1 --2- 172.21.15.43:0/4122998222 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fee20004d00 0x7fee20005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:35:27.056 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:27.051+0000 7fee3659c640 1 -- 172.21.15.43:0/4122998222 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fee380cd630 msgr2=0x7fee200047c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-09T13:35:27.056 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:27.051+0000 7fee3659c640 1 --2- 172.21.15.43:0/4122998222 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fee380cd630 0x7fee200047c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:35:27.056 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:27.051+0000 7fee3659c640 1 -- 172.21.15.43:0/4122998222 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fee20004d00 msgr2=0x7fee20005180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-09T13:35:27.056 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:27.051+0000 7fee3659c640 1 --2- 172.21.15.43:0/4122998222 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fee20004d00 0x7fee20005180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:35:27.057 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:27.051+0000 7fee3659c640 1 -- 172.21.15.43:0/4122998222 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fee20003520 con 0x7fee200039a0 2023-08-09T13:35:27.057 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:27.051+0000 7fee36d9d640 1 --2- 172.21.15.43:0/4122998222 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fee20004d00 0x7fee20005180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-08-09T13:35:27.058 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:27.052+0000 7fee3659c640 1 --2- 172.21.15.43:0/4122998222 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fee200039a0 0x7fee20003dc0 secure :-1 s=READY pgs=358 cs=0 l=1 rev1=1 crypto rx=0x7fee2800a800 tx=0x7fee200057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8513cb5018c5092c server_cookie=0 in_seq=0 out_seq=0 2023-08-09T13:35:27.058 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:27.053+0000 7fee3559a640 1 -- 172.21.15.43:0/4122998222 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fee280072d0 con 0x7fee200039a0 2023-08-09T13:35:27.058 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:27.053+0000 7fee3559a640 1 -- 172.21.15.43:0/4122998222 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fee2802f070 con 0x7fee200039a0 2023-08-09T13:35:27.058 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:27.053+0000 7fee3559a640 1 -- 172.21.15.43:0/4122998222 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fee280076b0 con 0x7fee200039a0 2023-08-09T13:35:27.058 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:27.053+0000 7fee3759e640 1 -- 172.21.15.43:0/4122998222 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fee200039a0 msgr2=0x7fee20003dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:35:27.058 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:27.053+0000 7fee3759e640 1 --2- 172.21.15.43:0/4122998222 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fee200039a0 0x7fee20003dc0 secure :-1 s=READY pgs=358 cs=0 l=1 rev1=1 crypto rx=0x7fee2800a800 tx=0x7fee200057f0 comp rx=0 tx=0).stop 2023-08-09T13:35:27.059 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:27.053+0000 7fee36d9d640 1 -- 172.21.15.43:0/4122998222 reap_dead start 2023-08-09T13:35:27.059 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:27.053+0000 7fee3759e640 1 -- 172.21.15.43:0/4122998222 shutdown_connections 2023-08-09T13:35:27.059 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:27.053+0000 7fee3759e640 1 -- 172.21.15.43:0/4122998222 >> 172.21.15.43:0/4122998222 conn(0x7fee380ccad0 msgr2=0x7fee380cceb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-09T13:35:27.059 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:27.053+0000 7fee3759e640 1 -- 172.21.15.43:0/4122998222 shutdown_connections 2023-08-09T13:35:27.059 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:27.053+0000 7fee3759e640 1 -- 172.21.15.43:0/4122998222 wait complete. 2023-08-09T13:35:27.059 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:27.055+0000 7fee3759e640 1 Processor -- start 2023-08-09T13:35:27.059 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:27.056+0000 7fee3759e640 1 -- start start 2023-08-09T13:35:27.061 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:27.057+0000 7fee3759e640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fee200039a0 0x7fee380ae300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:35:27.061 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:27.057+0000 7fee3759e640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fee380ae840 0x7fee3815b7f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:35:27.061 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:27.057+0000 7fee3759e640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fee3815bd30 0x7fee3815aa80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:35:27.061 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:27.057+0000 7fee3759e640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fee3815c1c0 con 0x7fee3815bd30 2023-08-09T13:35:27.061 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:27.057+0000 7fee3759e640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fee380c6730 con 0x7fee200039a0 2023-08-09T13:35:27.061 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:27.057+0000 7fee3759e640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fee3806ed40 con 0x7fee380ae840 2023-08-09T13:35:27.062 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:27.057+0000 7fee36d9d640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fee3815bd30 0x7fee3815aa80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:35:27.062 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:27.057+0000 7fee36d9d640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fee3815bd30 0x7fee3815aa80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:46976/0 (socket says 172.21.15.43:46976) 2023-08-09T13:35:27.062 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:27.057+0000 7fee36d9d640 1 -- 172.21.15.43:0/2289109726 learned_addr learned my addr 172.21.15.43:0/2289109726 (peer_addr_for_me v2:172.21.15.43:0/0) 2023-08-09T13:35:27.062 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:27.057+0000 7fee3659c640 1 --2- 172.21.15.43:0/2289109726 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fee200039a0 0x7fee380ae300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:35:27.062 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:27.057+0000 7fee35d9b640 1 --2- 172.21.15.43:0/2289109726 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fee380ae840 0x7fee3815b7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:35:27.062 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:27.057+0000 7fee36d9d640 1 -- 172.21.15.43:0/2289109726 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fee380ae840 msgr2=0x7fee3815b7f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:35:27.062 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:27.057+0000 7fee36d9d640 1 --2- 172.21.15.43:0/2289109726 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fee380ae840 0x7fee3815b7f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:35:27.062 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:27.057+0000 7fee36d9d640 1 -- 172.21.15.43:0/2289109726 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fee200039a0 msgr2=0x7fee380ae300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:35:27.063 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:27.057+0000 7fee36d9d640 1 --2- 172.21.15.43:0/2289109726 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fee200039a0 0x7fee380ae300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:35:27.063 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:27.057+0000 7fee36d9d640 1 -- 172.21.15.43:0/2289109726 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fee3806f020 con 0x7fee3815bd30 2023-08-09T13:35:27.063 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:27.057+0000 7fee3659c640 1 --2- 172.21.15.43:0/2289109726 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fee200039a0 0x7fee380ae300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-08-09T13:35:27.063 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:27.058+0000 7fee36d9d640 1 --2- 172.21.15.43:0/2289109726 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fee3815bd30 0x7fee3815aa80 secure :-1 s=READY pgs=359 cs=0 l=1 rev1=1 crypto rx=0x7fee380cdb70 tx=0x7fee3000d750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-09T13:35:27.063 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:27.058+0000 7fee26ffd640 1 -- 172.21.15.43:0/2289109726 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fee3000b070 con 0x7fee3815bd30 2023-08-09T13:35:27.063 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:27.058+0000 7fee26ffd640 1 -- 172.21.15.43:0/2289109726 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fee30007030 con 0x7fee3815bd30 2023-08-09T13:35:27.063 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:27.058+0000 7fee26ffd640 1 -- 172.21.15.43:0/2289109726 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fee300073e0 con 0x7fee3815bd30 2023-08-09T13:35:27.063 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:27.058+0000 7fee3759e640 1 -- 172.21.15.43:0/2289109726 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fee3806f300 con 0x7fee3815bd30 2023-08-09T13:35:27.064 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:27.058+0000 7fee3759e640 1 -- 172.21.15.43:0/2289109726 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fee380c5a70 con 0x7fee3815bd30 2023-08-09T13:35:27.065 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:27.059+0000 7fee3759e640 1 -- 172.21.15.43:0/2289109726 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fee3815a7a0 con 0x7fee3815bd30 2023-08-09T13:35:27.065 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:27.060+0000 7fee26ffd640 1 -- 172.21.15.43:0/2289109726 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 5) v1 ==== 88341+0+0 (secure 0 0 0) 0x7fee30007540 con 0x7fee3815bd30 2023-08-09T13:35:27.065 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:27.061+0000 7fee26ffd640 1 --2- 172.21.15.43:0/2289109726 >> [v2:172.21.15.203:6832/830097474,v1:172.21.15.203:6833/830097474] conn(0x7fee1806a050 0x7fee1806c510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:35:27.065 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:27.061+0000 7fee26ffd640 1 -- 172.21.15.43:0/2289109726 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 6871+0+0 (secure 0 0 0) 0x7fee300883a0 con 0x7fee3815bd30 2023-08-09T13:35:27.065 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:27.062+0000 7fee3659c640 1 --2- 172.21.15.43:0/2289109726 >> [v2:172.21.15.203:6832/830097474,v1:172.21.15.203:6833/830097474] conn(0x7fee1806a050 0x7fee1806c510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:35:27.066 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:27.063+0000 7fee3659c640 1 --2- 172.21.15.43:0/2289109726 >> [v2:172.21.15.203:6832/830097474,v1:172.21.15.203:6833/830097474] conn(0x7fee1806a050 0x7fee1806c510 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7fee28034000 tx=0x7fee28007000 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-09T13:35:27.068 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:27.066+0000 7fee26ffd640 1 -- 172.21.15.43:0/2289109726 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178066 (secure 0 0 0) 0x7fee3815a7a0 con 0x7fee3815bd30 2023-08-09T13:35:27.271 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:27.268+0000 7fee3759e640 1 -- 172.21.15.43:0/2289109726 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7fee3815a7a0 con 0x7fee3815bd30 2023-08-09T13:35:27.272 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:27.269+0000 7fee26ffd640 1 -- 172.21.15.43:0/2289109726 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v26) v1 ==== 93+0+10 (secure 0 0 0) 0x7fee3815a7a0 con 0x7fee3815bd30 2023-08-09T13:35:27.272 INFO:teuthology.orchestra.run.smithi043.stdout:pg_num: 1 2023-08-09T13:35:27.275 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:27.272+0000 7fee3759e640 1 -- 172.21.15.43:0/2289109726 >> [v2:172.21.15.203:6832/830097474,v1:172.21.15.203:6833/830097474] conn(0x7fee1806a050 msgr2=0x7fee1806c510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:35:27.275 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:27.272+0000 7fee3759e640 1 --2- 172.21.15.43:0/2289109726 >> [v2:172.21.15.203:6832/830097474,v1:172.21.15.203:6833/830097474] conn(0x7fee1806a050 0x7fee1806c510 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7fee28034000 tx=0x7fee28007000 comp rx=0 tx=0).stop 2023-08-09T13:35:27.275 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:27.272+0000 7fee3759e640 1 -- 172.21.15.43:0/2289109726 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fee3815bd30 msgr2=0x7fee3815aa80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:35:27.275 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:27.272+0000 7fee3759e640 1 --2- 172.21.15.43:0/2289109726 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fee3815bd30 0x7fee3815aa80 secure :-1 s=READY pgs=359 cs=0 l=1 rev1=1 crypto rx=0x7fee380cdb70 tx=0x7fee3000d750 comp rx=0 tx=0).stop 2023-08-09T13:35:27.276 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:27.273+0000 7fee3759e640 1 -- 172.21.15.43:0/2289109726 shutdown_connections 2023-08-09T13:35:27.276 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:27.273+0000 7fee3759e640 1 --2- 172.21.15.43:0/2289109726 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fee3815bd30 0x7fee3815aa80 unknown :-1 s=CLOSED pgs=359 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:35:27.276 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:27.273+0000 7fee3759e640 1 --2- 172.21.15.43:0/2289109726 >> [v2:172.21.15.203:6832/830097474,v1:172.21.15.203:6833/830097474] conn(0x7fee1806a050 0x7fee1806c510 unknown :-1 s=CLOSED pgs=187 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:35:27.276 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:27.273+0000 7fee3759e640 1 --2- 172.21.15.43:0/2289109726 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fee380ae840 0x7fee3815b7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:35:27.277 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:27.273+0000 7fee3759e640 1 --2- 172.21.15.43:0/2289109726 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fee200039a0 0x7fee380ae300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:35:27.277 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:27.273+0000 7fee3759e640 1 -- 172.21.15.43:0/2289109726 >> 172.21.15.43:0/2289109726 conn(0x7fee380ccad0 msgr2=0x7fee3806c1a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-09T13:35:27.277 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:27.273+0000 7fee3759e640 1 -- 172.21.15.43:0/2289109726 shutdown_connections 2023-08-09T13:35:27.277 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:27.273+0000 7fee3759e640 1 -- 172.21.15.43:0/2289109726 wait complete. 2023-08-09T13:35:27.293 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-08-09T13:35:27.505 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:27.502+0000 7fbedf4ad640 1 Processor -- start 2023-08-09T13:35:27.505 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:27.502+0000 7fbedf4ad640 1 -- start start 2023-08-09T13:35:27.507 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:27.504+0000 7fbedf4ad640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7fbed8143c20 0x7fbed806f540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:35:27.507 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:27.504+0000 7fbedf4ad640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fbed806cfe0 0x7fbed80c8b50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:35:27.507 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:27.504+0000 7fbedf4ad640 1 -- --> v1:172.21.15.203:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbed80c9090 con 0x7fbed80c86f0 2023-08-09T13:35:27.507 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:27.504+0000 7fbedf4ad640 1 -- --> v2:172.21.15.43:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbed8144000 con 0x7fbed8143c20 2023-08-09T13:35:27.507 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:27.504+0000 7fbedf4ad640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbed806c440 con 0x7fbed806cfe0 2023-08-09T13:35:27.508 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:27.505+0000 7fbedd249640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7fbed8143c20 0x7fbed806f540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:35:27.508 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:27.505+0000 7fbedd249640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7fbed8143c20 0x7fbed806f540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:46990/0 (socket says 172.21.15.43:46990) 2023-08-09T13:35:27.509 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:27.505+0000 7fbedd249640 1 -- 172.21.15.43:0/3212528244 learned_addr learned my addr 172.21.15.43:0/3212528244 (peer_addr_for_me v2:172.21.15.43:0/0) 2023-08-09T13:35:27.509 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:27.505+0000 7fbedda4a640 1 --2- 172.21.15.43:0/3212528244 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fbed806cfe0 0x7fbed80c8b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:35:27.509 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:27.505+0000 7fbedd249640 1 -- 172.21.15.43:0/3212528244 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fbed806cfe0 msgr2=0x7fbed80c8b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-09T13:35:27.509 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:27.505+0000 7fbedd249640 1 --2- 172.21.15.43:0/3212528244 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fbed806cfe0 0x7fbed80c8b50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:35:27.509 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:27.505+0000 7fbedd249640 1 -- 172.21.15.43:0/3212528244 >> v1:172.21.15.203:6789/0 conn(0x7fbed80c86f0 legacy=0x7fbed806c8d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-09T13:35:27.509 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:27.505+0000 7fbedd249640 1 -- 172.21.15.43:0/3212528244 --> v2:172.21.15.43:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbed806b830 con 0x7fbed8143c20 2023-08-09T13:35:27.510 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:27.505+0000 7fbedda4a640 1 --2- 172.21.15.43:0/3212528244 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fbed806cfe0 0x7fbed80c8b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-08-09T13:35:27.511 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:27.507+0000 7fbedd249640 1 --2- 172.21.15.43:0/3212528244 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fbed8143c20 0x7fbed806f540 secure :-1 s=READY pgs=360 cs=0 l=1 rev1=1 crypto rx=0x7fbed4009870 tx=0x7fbed402b860 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1add3f81fb743ec9 server_cookie=0 in_seq=0 out_seq=0 2023-08-09T13:35:27.511 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:27.507+0000 7fbecffff640 1 -- 172.21.15.43:0/3212528244 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbed40317c0 con 0x7fbed8143c20 2023-08-09T13:35:27.511 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:27.507+0000 7fbecffff640 1 -- 172.21.15.43:0/3212528244 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbed4031920 con 0x7fbed8143c20 2023-08-09T13:35:27.511 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:27.507+0000 7fbecffff640 1 -- 172.21.15.43:0/3212528244 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbed4031cb0 con 0x7fbed8143c20 2023-08-09T13:35:27.511 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:27.508+0000 7fbedf4ad640 1 -- 172.21.15.43:0/3212528244 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fbed8143c20 msgr2=0x7fbed806f540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:35:27.512 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:27.508+0000 7fbedf4ad640 1 --2- 172.21.15.43:0/3212528244 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fbed8143c20 0x7fbed806f540 secure :-1 s=READY pgs=360 cs=0 l=1 rev1=1 crypto rx=0x7fbed4009870 tx=0x7fbed402b860 comp rx=0 tx=0).stop 2023-08-09T13:35:27.512 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:27.508+0000 7fbedf4ad640 1 -- 172.21.15.43:0/3212528244 shutdown_connections 2023-08-09T13:35:27.512 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:27.508+0000 7fbedf4ad640 1 --2- 172.21.15.43:0/3212528244 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fbed806cfe0 0x7fbed80c8b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:35:27.512 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:27.508+0000 7fbedf4ad640 1 --2- 172.21.15.43:0/3212528244 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fbed8143c20 0x7fbed806f540 unknown :-1 s=CLOSED pgs=360 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:35:27.512 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:27.508+0000 7fbedf4ad640 1 -- 172.21.15.43:0/3212528244 >> 172.21.15.43:0/3212528244 conn(0x7fbed80ce0a0 msgr2=0x7fbed80ce480 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-09T13:35:27.512 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:27.508+0000 7fbedf4ad640 1 -- 172.21.15.43:0/3212528244 shutdown_connections 2023-08-09T13:35:27.512 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:27.508+0000 7fbedf4ad640 1 -- 172.21.15.43:0/3212528244 wait complete. 2023-08-09T13:35:27.514 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:27.510+0000 7fbedf4ad640 1 Processor -- start 2023-08-09T13:35:27.514 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:27.510+0000 7fbedf4ad640 1 -- start start 2023-08-09T13:35:27.515 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:27.511+0000 7fbedf4ad640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fbed806cfe0 0x7fbed8159de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:35:27.515 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:27.511+0000 7fbedf4ad640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fbed80c86f0 0x7fbed80b3590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:35:27.515 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:27.511+0000 7fbedf4ad640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fbed8143c20 0x7fbed8141d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:35:27.515 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:27.511+0000 7fbedf4ad640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbed80b24f0 con 0x7fbed806cfe0 2023-08-09T13:35:27.516 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:27.511+0000 7fbedf4ad640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbed80b27d0 con 0x7fbed8143c20 2023-08-09T13:35:27.516 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:27.511+0000 7fbedf4ad640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbed80b2ab0 con 0x7fbed80c86f0 2023-08-09T13:35:27.517 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:27.511+0000 7fbedd249640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fbed806cfe0 0x7fbed8159de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:35:27.517 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:27.511+0000 7fbedd249640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fbed806cfe0 0x7fbed8159de0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:46998/0 (socket says 172.21.15.43:46998) 2023-08-09T13:35:27.517 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:27.511+0000 7fbedd249640 1 -- 172.21.15.43:0/1445124496 learned_addr learned my addr 172.21.15.43:0/1445124496 (peer_addr_for_me v2:172.21.15.43:0/0) 2023-08-09T13:35:27.517 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:27.511+0000 7fbedd249640 1 -- 172.21.15.43:0/1445124496 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fbed80c86f0 msgr2=0x7fbed80b3590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:35:27.517 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:27.511+0000 7fbedd249640 1 --2- 172.21.15.43:0/1445124496 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fbed80c86f0 0x7fbed80b3590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:35:27.517 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:27.511+0000 7fbedd249640 1 -- 172.21.15.43:0/1445124496 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fbed8143c20 msgr2=0x7fbed8141d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:35:27.518 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:27.511+0000 7fbedd249640 1 --2- 172.21.15.43:0/1445124496 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fbed8143c20 0x7fbed8141d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:35:27.518 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:27.511+0000 7fbedd249640 1 -- 172.21.15.43:0/1445124496 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbed40094d0 con 0x7fbed806cfe0 2023-08-09T13:35:27.518 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:27.512+0000 7fbedd249640 1 --2- 172.21.15.43:0/1445124496 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fbed806cfe0 0x7fbed8159de0 secure :-1 s=READY pgs=361 cs=0 l=1 rev1=1 crypto rx=0x7fbed4000c00 tx=0x7fbed40046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-09T13:35:27.519 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:27.512+0000 7fbecdffb640 1 -- 172.21.15.43:0/1445124496 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbed40329a0 con 0x7fbed806cfe0 2023-08-09T13:35:27.520 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:27.512+0000 7fbecdffb640 1 -- 172.21.15.43:0/1445124496 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbed4032b20 con 0x7fbed806cfe0 2023-08-09T13:35:27.520 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:27.512+0000 7fbecdffb640 1 -- 172.21.15.43:0/1445124496 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbed40453f0 con 0x7fbed806cfe0 2023-08-09T13:35:27.520 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:27.512+0000 7fbedf4ad640 1 -- 172.21.15.43:0/1445124496 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbed80b2d90 con 0x7fbed806cfe0 2023-08-09T13:35:27.520 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:27.512+0000 7fbedf4ad640 1 -- 172.21.15.43:0/1445124496 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbed80b3ad0 con 0x7fbed806cfe0 2023-08-09T13:35:27.520 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:27.514+0000 7fbedf4ad640 1 -- 172.21.15.43:0/1445124496 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbed80b32b0 con 0x7fbed806cfe0 2023-08-09T13:35:27.521 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:27.514+0000 7fbecdffb640 1 -- 172.21.15.43:0/1445124496 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 5) v1 ==== 88341+0+0 (secure 0 0 0) 0x7fbed404d3f0 con 0x7fbed806cfe0 2023-08-09T13:35:27.521 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:27.515+0000 7fbecdffb640 1 --2- 172.21.15.43:0/1445124496 >> [v2:172.21.15.203:6832/830097474,v1:172.21.15.203:6833/830097474] conn(0x7fbebc06a080 0x7fbebc06c540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:35:27.521 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:27.515+0000 7fbecdffb640 1 -- 172.21.15.43:0/1445124496 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 6871+0+0 (secure 0 0 0) 0x7fbed40aa860 con 0x7fbed806cfe0 2023-08-09T13:35:27.521 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:27.516+0000 7fbedca48640 1 --2- 172.21.15.43:0/1445124496 >> [v2:172.21.15.203:6832/830097474,v1:172.21.15.203:6833/830097474] conn(0x7fbebc06a080 0x7fbebc06c540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:35:27.521 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:27.517+0000 7fbedca48640 1 --2- 172.21.15.43:0/1445124496 >> [v2:172.21.15.203:6832/830097474,v1:172.21.15.203:6833/830097474] conn(0x7fbebc06a080 0x7fbebc06c540 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7fbed80b1f60 tx=0x7fbec80079b0 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-09T13:35:27.523 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:27.520+0000 7fbecdffb640 1 -- 172.21.15.43:0/1445124496 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178066 (secure 0 0 0) 0x7fbed80b32b0 con 0x7fbed806cfe0 2023-08-09T13:35:27.716 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:27.714+0000 7fbedf4ad640 1 -- 172.21.15.43:0/1445124496 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7fbed80b32b0 con 0x7fbed806cfe0 2023-08-09T13:35:27.718 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:27.715+0000 7fbecdffb640 1 -- 172.21.15.43:0/1445124496 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v26) v1 ==== 104+0+11 (secure 0 0 0) 0x7fbed80b32b0 con 0x7fbed806cfe0 2023-08-09T13:35:27.718 INFO:teuthology.orchestra.run.smithi043.stdout:pg_num: 32 2023-08-09T13:35:27.720 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:27.718+0000 7fbedf4ad640 1 -- 172.21.15.43:0/1445124496 >> [v2:172.21.15.203:6832/830097474,v1:172.21.15.203:6833/830097474] conn(0x7fbebc06a080 msgr2=0x7fbebc06c540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:35:27.721 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:27.718+0000 7fbedf4ad640 1 --2- 172.21.15.43:0/1445124496 >> [v2:172.21.15.203:6832/830097474,v1:172.21.15.203:6833/830097474] conn(0x7fbebc06a080 0x7fbebc06c540 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7fbed80b1f60 tx=0x7fbec80079b0 comp rx=0 tx=0).stop 2023-08-09T13:35:27.721 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:27.718+0000 7fbedf4ad640 1 -- 172.21.15.43:0/1445124496 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fbed806cfe0 msgr2=0x7fbed8159de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:35:27.721 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:27.718+0000 7fbedf4ad640 1 --2- 172.21.15.43:0/1445124496 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fbed806cfe0 0x7fbed8159de0 secure :-1 s=READY pgs=361 cs=0 l=1 rev1=1 crypto rx=0x7fbed4000c00 tx=0x7fbed40046c0 comp rx=0 tx=0).stop 2023-08-09T13:35:27.722 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:27.718+0000 7fbedf4ad640 1 -- 172.21.15.43:0/1445124496 shutdown_connections 2023-08-09T13:35:27.722 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:27.718+0000 7fbedf4ad640 1 --2- 172.21.15.43:0/1445124496 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fbed8143c20 0x7fbed8141d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:35:27.722 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:27.718+0000 7fbedf4ad640 1 --2- 172.21.15.43:0/1445124496 >> [v2:172.21.15.203:6832/830097474,v1:172.21.15.203:6833/830097474] conn(0x7fbebc06a080 0x7fbebc06c540 unknown :-1 s=CLOSED pgs=188 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:35:27.722 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:27.718+0000 7fbedf4ad640 1 --2- 172.21.15.43:0/1445124496 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fbed80c86f0 0x7fbed80b3590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:35:27.722 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:27.718+0000 7fbedf4ad640 1 --2- 172.21.15.43:0/1445124496 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fbed806cfe0 0x7fbed8159de0 unknown :-1 s=CLOSED pgs=361 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:35:27.722 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:27.718+0000 7fbedf4ad640 1 -- 172.21.15.43:0/1445124496 >> 172.21.15.43:0/1445124496 conn(0x7fbed80ce0a0 msgr2=0x7fbed80c4490 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-09T13:35:27.722 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:27.718+0000 7fbedf4ad640 1 -- 172.21.15.43:0/1445124496 shutdown_connections 2023-08-09T13:35:27.723 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:27.718+0000 7fbedf4ad640 1 -- 172.21.15.43:0/1445124496 wait complete. 2023-08-09T13:35:27.741 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-08-09T13:35:27.970 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:27.968+0000 7fe3affff640 1 Processor -- start 2023-08-09T13:35:27.971 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:27.968+0000 7fe3affff640 1 -- start start 2023-08-09T13:35:27.972 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:27.970+0000 7fe3affff640 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7fe3b015c2d0 0x7fe3b00b7df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:35:27.972 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:27.970+0000 7fe3affff640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fe3b00cebf0 0x7fe3b00cf080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:35:27.972 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:27.970+0000 7fe3affff640 1 -- --> v1:172.21.15.203:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe3b00d16a0 con 0x7fe3b00d1b30 2023-08-09T13:35:27.973 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:27.970+0000 7fe3affff640 1 -- --> v2:172.21.15.203:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe3b00cf5c0 con 0x7fe3b015c2d0 2023-08-09T13:35:27.973 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:27.970+0000 7fe3affff640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe3b00d1fa0 con 0x7fe3b00cebf0 2023-08-09T13:35:27.973 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:27.970+0000 7fe3af7fe640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fe3b00cebf0 0x7fe3b00cf080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:35:27.973 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:27.970+0000 7fe3aeffd640 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7fe3b015c2d0 0x7fe3b00b7df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:35:27.973 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:27.971+0000 7fe3aeffd640 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7fe3b015c2d0 0x7fe3b00b7df0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.203:3300/0 says I am v2:172.21.15.43:59690/0 (socket says 172.21.15.43:59690) 2023-08-09T13:35:27.974 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:27.971+0000 7fe3af7fe640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fe3b00cebf0 0x7fe3b00cf080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.203:3301/0 says I am v2:172.21.15.43:54656/0 (socket says 172.21.15.43:54656) 2023-08-09T13:35:27.974 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:27.971+0000 7fe3aeffd640 1 -- 172.21.15.43:0/1889029186 learned_addr learned my addr 172.21.15.43:0/1889029186 (peer_addr_for_me v2:172.21.15.43:0/0) 2023-08-09T13:35:27.974 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:27.971+0000 7fe3aeffd640 1 -- 172.21.15.43:0/1889029186 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fe3b00cebf0 msgr2=0x7fe3b00cf080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-09T13:35:27.974 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:27.971+0000 7fe3aeffd640 1 --2- 172.21.15.43:0/1889029186 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fe3b00cebf0 0x7fe3b00cf080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:35:27.975 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:27.971+0000 7fe3aeffd640 1 -- 172.21.15.43:0/1889029186 >> v1:172.21.15.203:6789/0 conn(0x7fe3b00d1b30 legacy=0x7fe3b00cd020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:35:27.975 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:27.971+0000 7fe3aeffd640 1 -- 172.21.15.43:0/1889029186 --> v2:172.21.15.203:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe3b00d0d70 con 0x7fe3b015c2d0 2023-08-09T13:35:27.975 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:27.973+0000 7fe3aeffd640 1 --2- 172.21.15.43:0/1889029186 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fe3b015c2d0 0x7fe3b00b7df0 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7fe3a0009870 tx=0x7fe3a002b980 comp rx=0 tx=0).ready entity=mon.1 client_cookie=9e8403cf601eb69f server_cookie=0 in_seq=0 out_seq=0 2023-08-09T13:35:27.975 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:27.973+0000 7fe3adffb640 1 -- 172.21.15.43:0/1889029186 <== mon.1 v2:172.21.15.203:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe3a0030840 con 0x7fe3b015c2d0 2023-08-09T13:35:27.976 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:27.973+0000 7fe3adffb640 1 -- 172.21.15.43:0/1889029186 <== mon.1 v2:172.21.15.203:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe3a00309a0 con 0x7fe3b015c2d0 2023-08-09T13:35:27.976 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:27.973+0000 7fe3adffb640 1 -- 172.21.15.43:0/1889029186 <== mon.1 v2:172.21.15.203:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe3a0030d30 con 0x7fe3b015c2d0 2023-08-09T13:35:27.977 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:27.974+0000 7fe3affff640 1 -- 172.21.15.43:0/1889029186 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fe3b015c2d0 msgr2=0x7fe3b00b7df0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:35:27.977 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:27.974+0000 7fe3affff640 1 --2- 172.21.15.43:0/1889029186 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fe3b015c2d0 0x7fe3b00b7df0 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7fe3a0009870 tx=0x7fe3a002b980 comp rx=0 tx=0).stop 2023-08-09T13:35:27.977 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:27.974+0000 7fe3affff640 1 -- 172.21.15.43:0/1889029186 shutdown_connections 2023-08-09T13:35:27.977 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:27.974+0000 7fe3affff640 1 --2- 172.21.15.43:0/1889029186 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fe3b00cebf0 0x7fe3b00cf080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:35:27.977 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:27.974+0000 7fe3affff640 1 --2- 172.21.15.43:0/1889029186 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fe3b015c2d0 0x7fe3b00b7df0 unknown :-1 s=CLOSED pgs=156 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:35:27.977 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:27.974+0000 7fe3affff640 1 -- 172.21.15.43:0/1889029186 >> 172.21.15.43:0/1889029186 conn(0x7fe3b006a230 msgr2=0x7fe3b006a640 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-09T13:35:27.977 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:27.974+0000 7fe3affff640 1 -- 172.21.15.43:0/1889029186 shutdown_connections 2023-08-09T13:35:27.978 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:27.974+0000 7fe3affff640 1 -- 172.21.15.43:0/1889029186 wait complete. 2023-08-09T13:35:27.978 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:27.976+0000 7fe3affff640 1 Processor -- start 2023-08-09T13:35:27.978 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:27.976+0000 7fe3affff640 1 -- start start 2023-08-09T13:35:27.980 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:27.977+0000 7fe3affff640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fe3b00cebf0 0x7fe3b015f360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:35:27.980 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:27.977+0000 7fe3affff640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fe3b00d1b30 0x7fe3b0154dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:35:27.980 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:27.977+0000 7fe3affff640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fe3b015c2d0 0x7fe3b01f4c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:35:27.980 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:27.977+0000 7fe3affff640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe3b01f2780 con 0x7fe3b015c2d0 2023-08-09T13:35:27.980 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:27.977+0000 7fe3affff640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe3b015f8a0 con 0x7fe3b00d1b30 2023-08-09T13:35:27.980 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:27.977+0000 7fe3affff640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe3b01f29d0 con 0x7fe3b00cebf0 2023-08-09T13:35:27.981 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:27.978+0000 7fe3af7fe640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fe3b015c2d0 0x7fe3b01f4c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:35:27.982 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:27.978+0000 7fe3af7fe640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fe3b015c2d0 0x7fe3b01f4c60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:47004/0 (socket says 172.21.15.43:47004) 2023-08-09T13:35:27.982 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:27.978+0000 7fe3af7fe640 1 -- 172.21.15.43:0/311120658 learned_addr learned my addr 172.21.15.43:0/311120658 (peer_addr_for_me v2:172.21.15.43:0/0) 2023-08-09T13:35:27.982 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:27.978+0000 7fe3ae7fc640 1 --2- 172.21.15.43:0/311120658 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fe3b00d1b30 0x7fe3b0154dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:35:27.982 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:27.978+0000 7fe3aeffd640 1 --2- 172.21.15.43:0/311120658 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fe3b00cebf0 0x7fe3b015f360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:35:27.982 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:27.978+0000 7fe3af7fe640 1 -- 172.21.15.43:0/311120658 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fe3b00cebf0 msgr2=0x7fe3b015f360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:35:27.982 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:27.978+0000 7fe3af7fe640 1 --2- 172.21.15.43:0/311120658 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fe3b00cebf0 0x7fe3b015f360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:35:27.982 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:27.978+0000 7fe3af7fe640 1 -- 172.21.15.43:0/311120658 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fe3b00d1b30 msgr2=0x7fe3b0154dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:35:27.983 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:27.978+0000 7fe3af7fe640 1 --2- 172.21.15.43:0/311120658 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fe3b00d1b30 0x7fe3b0154dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:35:27.983 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:27.978+0000 7fe3af7fe640 1 -- 172.21.15.43:0/311120658 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe3a00094d0 con 0x7fe3b015c2d0 2023-08-09T13:35:27.983 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:27.978+0000 7fe3af7fe640 1 --2- 172.21.15.43:0/311120658 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fe3b015c2d0 0x7fe3b01f4c60 secure :-1 s=READY pgs=362 cs=0 l=1 rev1=1 crypto rx=0x7fe3a8009d30 tx=0x7fe3a800c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-09T13:35:27.983 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:27.979+0000 7fe3977fe640 1 -- 172.21.15.43:0/311120658 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe3a8009ba0 con 0x7fe3b015c2d0 2023-08-09T13:35:27.983 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:27.979+0000 7fe3977fe640 1 -- 172.21.15.43:0/311120658 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe3a800f040 con 0x7fe3b015c2d0 2023-08-09T13:35:27.983 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:27.979+0000 7fe3977fe640 1 -- 172.21.15.43:0/311120658 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe3a80146c0 con 0x7fe3b015c2d0 2023-08-09T13:35:27.984 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:27.979+0000 7fe3affff640 1 -- 172.21.15.43:0/311120658 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe3b0155820 con 0x7fe3b015c2d0 2023-08-09T13:35:27.984 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:27.979+0000 7fe3affff640 1 -- 172.21.15.43:0/311120658 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe3b00ca5e0 con 0x7fe3b015c2d0 2023-08-09T13:35:27.984 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:27.980+0000 7fe3affff640 1 -- 172.21.15.43:0/311120658 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe3b004e390 con 0x7fe3b015c2d0 2023-08-09T13:35:27.985 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:27.981+0000 7fe3977fe640 1 -- 172.21.15.43:0/311120658 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 5) v1 ==== 88341+0+0 (secure 0 0 0) 0x7fe3a8014820 con 0x7fe3b015c2d0 2023-08-09T13:35:27.985 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:27.982+0000 7fe3977fe640 1 --2- 172.21.15.43:0/311120658 >> [v2:172.21.15.203:6832/830097474,v1:172.21.15.203:6833/830097474] conn(0x7fe39c06a050 0x7fe39c06c510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:35:27.985 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:27.982+0000 7fe3977fe640 1 -- 172.21.15.43:0/311120658 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 6871+0+0 (secure 0 0 0) 0x7fe3a8089450 con 0x7fe3b015c2d0 2023-08-09T13:35:27.986 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:27.983+0000 7fe3aeffd640 1 --2- 172.21.15.43:0/311120658 >> [v2:172.21.15.203:6832/830097474,v1:172.21.15.203:6833/830097474] conn(0x7fe39c06a050 0x7fe39c06c510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:35:27.987 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:27.985+0000 7fe3aeffd640 1 --2- 172.21.15.43:0/311120658 >> [v2:172.21.15.203:6832/830097474,v1:172.21.15.203:6833/830097474] conn(0x7fe39c06a050 0x7fe39c06c510 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7fe3a0036000 tx=0x7fe3a0008030 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-09T13:35:27.989 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:27.986+0000 7fe3977fe640 1 -- 172.21.15.43:0/311120658 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178066 (secure 0 0 0) 0x7fe3a8054800 con 0x7fe3b015c2d0 2023-08-09T13:35:28.186 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:28.183+0000 7fe3affff640 1 -- 172.21.15.43:0/311120658 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7fe3b00b1e50 con 0x7fe3b015c2d0 2023-08-09T13:35:28.186 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:28.184+0000 7fe3977fe640 1 -- 172.21.15.43:0/311120658 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v26) v1 ==== 100+0+11 (secure 0 0 0) 0x7fe3a8057e20 con 0x7fe3b015c2d0 2023-08-09T13:35:28.187 INFO:teuthology.orchestra.run.smithi043.stdout:pg_num: 64 2023-08-09T13:35:28.190 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:28.187+0000 7fe3affff640 1 -- 172.21.15.43:0/311120658 >> [v2:172.21.15.203:6832/830097474,v1:172.21.15.203:6833/830097474] conn(0x7fe39c06a050 msgr2=0x7fe39c06c510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:35:28.190 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:28.187+0000 7fe3affff640 1 --2- 172.21.15.43:0/311120658 >> [v2:172.21.15.203:6832/830097474,v1:172.21.15.203:6833/830097474] conn(0x7fe39c06a050 0x7fe39c06c510 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7fe3a0036000 tx=0x7fe3a0008030 comp rx=0 tx=0).stop 2023-08-09T13:35:28.190 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:28.187+0000 7fe3affff640 1 -- 172.21.15.43:0/311120658 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fe3b015c2d0 msgr2=0x7fe3b01f4c60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:35:28.190 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:28.187+0000 7fe3affff640 1 --2- 172.21.15.43:0/311120658 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fe3b015c2d0 0x7fe3b01f4c60 secure :-1 s=READY pgs=362 cs=0 l=1 rev1=1 crypto rx=0x7fe3a8009d30 tx=0x7fe3a800c490 comp rx=0 tx=0).stop 2023-08-09T13:35:28.190 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:28.187+0000 7fe3affff640 1 -- 172.21.15.43:0/311120658 shutdown_connections 2023-08-09T13:35:28.190 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:28.187+0000 7fe3affff640 1 --2- 172.21.15.43:0/311120658 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fe3b015c2d0 0x7fe3b01f4c60 unknown :-1 s=CLOSED pgs=362 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:35:28.190 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:28.187+0000 7fe3affff640 1 --2- 172.21.15.43:0/311120658 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fe3b00d1b30 0x7fe3b0154dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:35:28.191 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:28.187+0000 7fe3affff640 1 --2- 172.21.15.43:0/311120658 >> [v2:172.21.15.203:6832/830097474,v1:172.21.15.203:6833/830097474] conn(0x7fe39c06a050 0x7fe39c06c510 unknown :-1 s=CLOSED pgs=189 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:35:28.191 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:28.187+0000 7fe3affff640 1 --2- 172.21.15.43:0/311120658 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fe3b00cebf0 0x7fe3b015f360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:35:28.191 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:28.187+0000 7fe3affff640 1 -- 172.21.15.43:0/311120658 >> 172.21.15.43:0/311120658 conn(0x7fe3b006a230 msgr2=0x7fe3b00cca60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-09T13:35:28.191 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:28.187+0000 7fe3affff640 1 -- 172.21.15.43:0/311120658 shutdown_connections 2023-08-09T13:35:28.191 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:28.187+0000 7fe3affff640 1 -- 172.21.15.43:0/311120658 wait complete. 2023-08-09T13:35:28.208 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2023-08-09T13:35:28.441 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:28.439+0000 7fd56ffff640 1 Processor -- start 2023-08-09T13:35:28.442 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:28.439+0000 7fd56ffff640 1 -- start start 2023-08-09T13:35:28.443 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:28.440+0000 7fd56ffff640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fd570157ce0 0x7fd57015b0f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:35:28.443 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:28.440+0000 7fd56ffff640 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7fd5700c8470 0x7fd5700c88c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:35:28.443 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:28.440+0000 7fd56ffff640 1 -- --> v1:172.21.15.203:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd5700c8e00 con 0x7fd57006f2c0 2023-08-09T13:35:28.443 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:28.440+0000 7fd56ffff640 1 -- --> v2:172.21.15.203:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd5701580c0 con 0x7fd5700c8470 2023-08-09T13:35:28.444 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:28.440+0000 7fd56ffff640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd5700c92c0 con 0x7fd570157ce0 2023-08-09T13:35:28.445 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:28.441+0000 7fd56f7fe640 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7fd5700c8470 0x7fd5700c88c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:35:28.445 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:28.441+0000 7fd56effd640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fd570157ce0 0x7fd57015b0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:35:28.445 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:28.441+0000 7fd56effd640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fd570157ce0 0x7fd57015b0f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.203:3301/0 says I am v2:172.21.15.43:54678/0 (socket says 172.21.15.43:54678) 2023-08-09T13:35:28.445 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:28.441+0000 7fd56f7fe640 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7fd5700c8470 0x7fd5700c88c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.203:3300/0 says I am v2:172.21.15.43:59706/0 (socket says 172.21.15.43:59706) 2023-08-09T13:35:28.445 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:28.441+0000 7fd56effd640 1 -- 172.21.15.43:0/2559710072 learned_addr learned my addr 172.21.15.43:0/2559710072 (peer_addr_for_me v2:172.21.15.43:0/0) 2023-08-09T13:35:28.445 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:28.442+0000 7fd56effd640 1 -- 172.21.15.43:0/2559710072 >> v2:172.21.15.203:3300/0 conn(0x7fd5700c8470 msgr2=0x7fd5700c88c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-09T13:35:28.446 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:28.442+0000 7fd56effd640 1 --2- 172.21.15.43:0/2559710072 >> v2:172.21.15.203:3300/0 conn(0x7fd5700c8470 0x7fd5700c88c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:35:28.446 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:28.442+0000 7fd56effd640 1 -- 172.21.15.43:0/2559710072 >> v1:172.21.15.203:6789/0 conn(0x7fd57006f2c0 legacy=0x7fd57006bc40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:35:28.446 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:28.442+0000 7fd56effd640 1 -- 172.21.15.43:0/2559710072 --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd57006b480 con 0x7fd570157ce0 2023-08-09T13:35:28.447 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:28.443+0000 7fd56effd640 1 --2- 172.21.15.43:0/2559710072 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fd570157ce0 0x7fd57015b0f0 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7fd560009e50 tx=0x7fd56002bee0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=feb7347660d1a52a server_cookie=0 in_seq=0 out_seq=0 2023-08-09T13:35:28.447 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:28.444+0000 7fd56dffb640 1 -- 172.21.15.43:0/2559710072 <== mon.2 v2:172.21.15.203:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd56002ca80 con 0x7fd570157ce0 2023-08-09T13:35:28.447 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:28.444+0000 7fd56dffb640 1 -- 172.21.15.43:0/2559710072 <== mon.2 v2:172.21.15.203:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd56002cbe0 con 0x7fd570157ce0 2023-08-09T13:35:28.447 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:28.444+0000 7fd56dffb640 1 -- 172.21.15.43:0/2559710072 <== mon.2 v2:172.21.15.203:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd5600343f0 con 0x7fd570157ce0 2023-08-09T13:35:28.448 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:28.444+0000 7fd56ffff640 1 -- 172.21.15.43:0/2559710072 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fd570157ce0 msgr2=0x7fd57015b0f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:35:28.448 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:28.444+0000 7fd56ffff640 1 --2- 172.21.15.43:0/2559710072 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fd570157ce0 0x7fd57015b0f0 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7fd560009e50 tx=0x7fd56002bee0 comp rx=0 tx=0).stop 2023-08-09T13:35:28.448 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:28.444+0000 7fd56ffff640 1 -- 172.21.15.43:0/2559710072 shutdown_connections 2023-08-09T13:35:28.449 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:28.444+0000 7fd56ffff640 1 --2- 172.21.15.43:0/2559710072 >> v2:172.21.15.203:3300/0 conn(0x7fd5700c8470 0x7fd5700c88c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:35:28.449 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:28.444+0000 7fd56ffff640 1 --2- 172.21.15.43:0/2559710072 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fd570157ce0 0x7fd57015b0f0 unknown :-1 s=CLOSED pgs=65 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:35:28.449 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:28.444+0000 7fd56ffff640 1 -- 172.21.15.43:0/2559710072 >> 172.21.15.43:0/2559710072 conn(0x7fd5700af9f0 msgr2=0x7fd5700afdd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-09T13:35:28.449 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:28.445+0000 7fd56ffff640 1 -- 172.21.15.43:0/2559710072 shutdown_connections 2023-08-09T13:35:28.450 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:28.445+0000 7fd56ffff640 1 -- 172.21.15.43:0/2559710072 wait complete. 2023-08-09T13:35:28.453 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:28.446+0000 7fd56ffff640 1 Processor -- start 2023-08-09T13:35:28.453 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:28.446+0000 7fd56ffff640 1 -- start start 2023-08-09T13:35:28.454 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:28.447+0000 7fd56ffff640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fd57006f2c0 0x7fd570153410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:35:28.454 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:28.447+0000 7fd56ffff640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fd5700c8470 0x7fd5700c37b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:35:28.454 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:28.447+0000 7fd56ffff640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fd570157ce0 0x7fd5700c3cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:35:28.454 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:28.447+0000 7fd56ffff640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd5700c5420 con 0x7fd5700c8470 2023-08-09T13:35:28.454 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:28.447+0000 7fd56ffff640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd5700c3190 con 0x7fd570157ce0 2023-08-09T13:35:28.454 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:28.447+0000 7fd56ffff640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd5700cd360 con 0x7fd57006f2c0 2023-08-09T13:35:28.454 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:28.448+0000 7fd56e7fc640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fd5700c8470 0x7fd5700c37b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:35:28.455 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:28.448+0000 7fd56e7fc640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fd5700c8470 0x7fd5700c37b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:47012/0 (socket says 172.21.15.43:47012) 2023-08-09T13:35:28.455 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:28.448+0000 7fd56e7fc640 1 -- 172.21.15.43:0/264953270 learned_addr learned my addr 172.21.15.43:0/264953270 (peer_addr_for_me v2:172.21.15.43:0/0) 2023-08-09T13:35:28.455 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:28.448+0000 7fd56effd640 1 --2- 172.21.15.43:0/264953270 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fd57006f2c0 0x7fd570153410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:35:28.455 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:28.448+0000 7fd56f7fe640 1 --2- 172.21.15.43:0/264953270 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fd570157ce0 0x7fd5700c3cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:35:28.455 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:28.448+0000 7fd56e7fc640 1 -- 172.21.15.43:0/264953270 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fd57006f2c0 msgr2=0x7fd570153410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:35:28.455 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:28.448+0000 7fd56e7fc640 1 --2- 172.21.15.43:0/264953270 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fd57006f2c0 0x7fd570153410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:35:28.455 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:28.449+0000 7fd56e7fc640 1 -- 172.21.15.43:0/264953270 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fd570157ce0 msgr2=0x7fd5700c3cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:35:28.455 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:28.449+0000 7fd56e7fc640 1 --2- 172.21.15.43:0/264953270 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fd570157ce0 0x7fd5700c3cf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:35:28.456 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:28.449+0000 7fd56e7fc640 1 -- 172.21.15.43:0/264953270 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd560009b30 con 0x7fd5700c8470 2023-08-09T13:35:28.456 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:28.449+0000 7fd56f7fe640 1 --2- 172.21.15.43:0/264953270 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fd570157ce0 0x7fd5700c3cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-08-09T13:35:28.456 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:28.449+0000 7fd56e7fc640 1 --2- 172.21.15.43:0/264953270 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fd5700c8470 0x7fd5700c37b0 secure :-1 s=READY pgs=363 cs=0 l=1 rev1=1 crypto rx=0x7fd55800c400 tx=0x7fd55800c8c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-09T13:35:28.456 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:28.449+0000 7fd5677fe640 1 -- 172.21.15.43:0/264953270 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd5580052d0 con 0x7fd5700c8470 2023-08-09T13:35:28.456 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:28.449+0000 7fd5677fe640 1 -- 172.21.15.43:0/264953270 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd558005430 con 0x7fd5700c8470 2023-08-09T13:35:28.456 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:28.449+0000 7fd5677fe640 1 -- 172.21.15.43:0/264953270 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd558003830 con 0x7fd5700c8470 2023-08-09T13:35:28.456 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:28.449+0000 7fd56ffff640 1 -- 172.21.15.43:0/264953270 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd570153950 con 0x7fd5700c8470 2023-08-09T13:35:28.457 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:28.449+0000 7fd56ffff640 1 -- 172.21.15.43:0/264953270 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd5700ccb60 con 0x7fd5700c8470 2023-08-09T13:35:28.457 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:28.451+0000 7fd56ffff640 1 -- 172.21.15.43:0/264953270 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd57004e390 con 0x7fd5700c8470 2023-08-09T13:35:28.457 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:28.452+0000 7fd5677fe640 1 -- 172.21.15.43:0/264953270 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 5) v1 ==== 88341+0+0 (secure 0 0 0) 0x7fd558003990 con 0x7fd5700c8470 2023-08-09T13:35:28.458 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:28.452+0000 7fd5677fe640 1 --2- 172.21.15.43:0/264953270 >> [v2:172.21.15.203:6832/830097474,v1:172.21.15.203:6833/830097474] conn(0x7fd55006a050 0x7fd55006c510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:35:28.458 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:28.453+0000 7fd5677fe640 1 -- 172.21.15.43:0/264953270 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 6871+0+0 (secure 0 0 0) 0x7fd558087ba0 con 0x7fd5700c8470 2023-08-09T13:35:28.458 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:28.453+0000 7fd56effd640 1 --2- 172.21.15.43:0/264953270 >> [v2:172.21.15.203:6832/830097474,v1:172.21.15.203:6833/830097474] conn(0x7fd55006a050 0x7fd55006c510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:35:28.458 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:28.455+0000 7fd56effd640 1 --2- 172.21.15.43:0/264953270 >> [v2:172.21.15.203:6832/830097474,v1:172.21.15.203:6833/830097474] conn(0x7fd55006a050 0x7fd55006c510 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7fd560035000 tx=0x7fd560030000 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-09T13:35:28.459 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:28.457+0000 7fd5677fe640 1 -- 172.21.15.43:0/264953270 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178066 (secure 0 0 0) 0x7fd5580530a0 con 0x7fd5700c8470 2023-08-09T13:35:28.649 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:28.646+0000 7fd56ffff640 1 -- 172.21.15.43:0/264953270 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) v1 -- 0x7fd5700c8e00 con 0x7fd5700c8470 2023-08-09T13:35:28.649 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:28.647+0000 7fd5677fe640 1 -- 172.21.15.43:0/264953270 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v26) v1 ==== 103+0+11 (secure 0 0 0) 0x7fd558017020 con 0x7fd5700c8470 2023-08-09T13:35:28.649 INFO:teuthology.orchestra.run.smithi043.stdout:pg_num: 32 2023-08-09T13:35:28.652 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:28.649+0000 7fd56ffff640 1 -- 172.21.15.43:0/264953270 >> [v2:172.21.15.203:6832/830097474,v1:172.21.15.203:6833/830097474] conn(0x7fd55006a050 msgr2=0x7fd55006c510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:35:28.652 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:28.649+0000 7fd56ffff640 1 --2- 172.21.15.43:0/264953270 >> [v2:172.21.15.203:6832/830097474,v1:172.21.15.203:6833/830097474] conn(0x7fd55006a050 0x7fd55006c510 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7fd560035000 tx=0x7fd560030000 comp rx=0 tx=0).stop 2023-08-09T13:35:28.652 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:28.649+0000 7fd56ffff640 1 -- 172.21.15.43:0/264953270 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fd5700c8470 msgr2=0x7fd5700c37b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:35:28.653 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:28.649+0000 7fd56ffff640 1 --2- 172.21.15.43:0/264953270 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fd5700c8470 0x7fd5700c37b0 secure :-1 s=READY pgs=363 cs=0 l=1 rev1=1 crypto rx=0x7fd55800c400 tx=0x7fd55800c8c0 comp rx=0 tx=0).stop 2023-08-09T13:35:28.653 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:28.650+0000 7fd56ffff640 1 -- 172.21.15.43:0/264953270 shutdown_connections 2023-08-09T13:35:28.653 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:28.650+0000 7fd56ffff640 1 --2- 172.21.15.43:0/264953270 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fd570157ce0 0x7fd5700c3cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:35:28.653 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:28.650+0000 7fd56ffff640 1 --2- 172.21.15.43:0/264953270 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fd5700c8470 0x7fd5700c37b0 unknown :-1 s=CLOSED pgs=363 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:35:28.653 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:28.650+0000 7fd56ffff640 1 --2- 172.21.15.43:0/264953270 >> [v2:172.21.15.203:6832/830097474,v1:172.21.15.203:6833/830097474] conn(0x7fd55006a050 0x7fd55006c510 unknown :-1 s=CLOSED pgs=190 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:35:28.653 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:28.650+0000 7fd56ffff640 1 --2- 172.21.15.43:0/264953270 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fd57006f2c0 0x7fd570153410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:35:28.653 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:28.650+0000 7fd56ffff640 1 -- 172.21.15.43:0/264953270 >> 172.21.15.43:0/264953270 conn(0x7fd5700af9f0 msgr2=0x7fd5701599e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-09T13:35:28.654 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:28.650+0000 7fd56ffff640 1 -- 172.21.15.43:0/264953270 shutdown_connections 2023-08-09T13:35:28.654 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:28.650+0000 7fd56ffff640 1 -- 172.21.15.43:0/264953270 wait complete. 2023-08-09T13:35:28.672 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2023-08-09T13:35:28.908 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:28.905+0000 7fad0524c640 1 Processor -- start 2023-08-09T13:35:28.908 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:28.906+0000 7fad0524c640 1 -- start start 2023-08-09T13:35:28.910 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:28.907+0000 7fad0524c640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7fad00143c20 0x7fad0006f540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:35:28.910 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:28.908+0000 7fad0524c640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fad0006cfe0 0x7fad000c8b50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:35:28.910 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:28.908+0000 7fad0524c640 1 -- --> v1:172.21.15.203:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fad000c9090 con 0x7fad000c86f0 2023-08-09T13:35:28.911 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:28.908+0000 7fad0524c640 1 -- --> v2:172.21.15.43:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fad00144000 con 0x7fad00143c20 2023-08-09T13:35:28.911 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:28.908+0000 7fad0524c640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fad0006c440 con 0x7fad0006cfe0 2023-08-09T13:35:28.911 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:28.908+0000 7facfed9d640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7fad00143c20 0x7fad0006f540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:35:28.911 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:28.908+0000 7facfed9d640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7fad00143c20 0x7fad0006f540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:47028/0 (socket says 172.21.15.43:47028) 2023-08-09T13:35:28.911 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:28.908+0000 7facfed9d640 1 -- 172.21.15.43:0/200924901 learned_addr learned my addr 172.21.15.43:0/200924901 (peer_addr_for_me v2:172.21.15.43:0/0) 2023-08-09T13:35:28.912 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:28.908+0000 7facfed9d640 1 -- 172.21.15.43:0/200924901 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fad0006cfe0 msgr2=0x7fad000c8b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-09T13:35:28.912 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:28.908+0000 7facfed9d640 1 --2- 172.21.15.43:0/200924901 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fad0006cfe0 0x7fad000c8b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:35:28.912 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:28.908+0000 7facfed9d640 1 -- 172.21.15.43:0/200924901 >> v1:172.21.15.203:6789/0 conn(0x7fad000c86f0 legacy=0x7fad0006c8d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-09T13:35:28.912 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:28.908+0000 7facfed9d640 1 -- 172.21.15.43:0/200924901 --> v2:172.21.15.43:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fad0006b830 con 0x7fad00143c20 2023-08-09T13:35:28.912 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:28.909+0000 7facfed9d640 1 --2- 172.21.15.43:0/200924901 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fad00143c20 0x7fad0006f540 secure :-1 s=READY pgs=364 cs=0 l=1 rev1=1 crypto rx=0x7facf0009870 tx=0x7facf002b860 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c062ebedbb5610fb server_cookie=0 in_seq=0 out_seq=0 2023-08-09T13:35:28.913 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:28.910+0000 7facfe59c640 1 -- 172.21.15.43:0/200924901 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7facf00317c0 con 0x7fad00143c20 2023-08-09T13:35:28.913 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:28.910+0000 7facfe59c640 1 -- 172.21.15.43:0/200924901 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7facf0031920 con 0x7fad00143c20 2023-08-09T13:35:28.913 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:28.910+0000 7facfe59c640 1 -- 172.21.15.43:0/200924901 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7facf0031cb0 con 0x7fad00143c20 2023-08-09T13:35:28.914 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:28.910+0000 7fad0524c640 1 -- 172.21.15.43:0/200924901 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fad00143c20 msgr2=0x7fad0006f540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:35:28.914 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:28.910+0000 7fad0524c640 1 --2- 172.21.15.43:0/200924901 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fad00143c20 0x7fad0006f540 secure :-1 s=READY pgs=364 cs=0 l=1 rev1=1 crypto rx=0x7facf0009870 tx=0x7facf002b860 comp rx=0 tx=0).stop 2023-08-09T13:35:28.914 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:28.910+0000 7fad0524c640 1 -- 172.21.15.43:0/200924901 shutdown_connections 2023-08-09T13:35:28.914 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:28.910+0000 7fad0524c640 1 --2- 172.21.15.43:0/200924901 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fad0006cfe0 0x7fad000c8b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:35:28.914 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:28.910+0000 7fad0524c640 1 --2- 172.21.15.43:0/200924901 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fad00143c20 0x7fad0006f540 unknown :-1 s=CLOSED pgs=364 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:35:28.914 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:28.910+0000 7fad0524c640 1 -- 172.21.15.43:0/200924901 >> 172.21.15.43:0/200924901 conn(0x7fad000ce0a0 msgr2=0x7fad000ce480 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-09T13:35:28.914 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:28.911+0000 7fad0524c640 1 -- 172.21.15.43:0/200924901 shutdown_connections 2023-08-09T13:35:28.915 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:28.911+0000 7fad0524c640 1 -- 172.21.15.43:0/200924901 wait complete. 2023-08-09T13:35:28.915 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:28.912+0000 7fad0524c640 1 Processor -- start 2023-08-09T13:35:28.915 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:28.912+0000 7fad0524c640 1 -- start start 2023-08-09T13:35:28.916 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:28.913+0000 7fad0524c640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fad0006cfe0 0x7fad000b3890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:35:28.916 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:28.913+0000 7fad0524c640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fad000c86f0 0x7fad0015bdf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:35:28.916 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:28.914+0000 7fad0524c640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fad00143c20 0x7fad0013fee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:35:28.917 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:28.914+0000 7fad0524c640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fad0015c330 con 0x7fad000c86f0 2023-08-09T13:35:28.917 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:28.914+0000 7fad0524c640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fad0015d650 con 0x7fad0006cfe0 2023-08-09T13:35:28.917 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:28.914+0000 7fad0524c640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fad0015c5b0 con 0x7fad00143c20 2023-08-09T13:35:28.917 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:28.914+0000 7facf7fff640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fad000c86f0 0x7fad0015bdf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:35:28.917 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:28.914+0000 7facf7fff640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fad000c86f0 0x7fad0015bdf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:47032/0 (socket says 172.21.15.43:47032) 2023-08-09T13:35:28.917 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:28.914+0000 7facf7fff640 1 -- 172.21.15.43:0/2936789382 learned_addr learned my addr 172.21.15.43:0/2936789382 (peer_addr_for_me v2:172.21.15.43:0/0) 2023-08-09T13:35:28.918 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:28.914+0000 7facff59e640 1 --2- 172.21.15.43:0/2936789382 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fad00143c20 0x7fad0013fee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:35:28.918 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:28.914+0000 7facfed9d640 1 --2- 172.21.15.43:0/2936789382 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fad0006cfe0 0x7fad000b3890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:35:28.919 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:28.914+0000 7facf7fff640 1 -- 172.21.15.43:0/2936789382 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fad00143c20 msgr2=0x7fad0013fee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:35:28.919 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:28.914+0000 7facf7fff640 1 --2- 172.21.15.43:0/2936789382 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fad00143c20 0x7fad0013fee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:35:28.919 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:28.914+0000 7facf7fff640 1 -- 172.21.15.43:0/2936789382 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fad0006cfe0 msgr2=0x7fad000b3890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:35:28.919 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:28.914+0000 7facf7fff640 1 --2- 172.21.15.43:0/2936789382 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fad0006cfe0 0x7fad000b3890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:35:28.919 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:28.914+0000 7facf7fff640 1 -- 172.21.15.43:0/2936789382 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7facf00094d0 con 0x7fad000c86f0 2023-08-09T13:35:28.919 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:28.914+0000 7facfed9d640 1 --2- 172.21.15.43:0/2936789382 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fad0006cfe0 0x7fad000b3890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-08-09T13:35:28.919 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:28.915+0000 7facf7fff640 1 --2- 172.21.15.43:0/2936789382 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fad000c86f0 0x7fad0015bdf0 secure :-1 s=READY pgs=365 cs=0 l=1 rev1=1 crypto rx=0x7face800c410 tx=0x7face800c8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-09T13:35:28.919 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:28.915+0000 7facf77fe640 1 -- 172.21.15.43:0/2936789382 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7face80035f0 con 0x7fad000c86f0 2023-08-09T13:35:28.920 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:28.915+0000 7facf77fe640 1 -- 172.21.15.43:0/2936789382 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7face8003750 con 0x7fad000c86f0 2023-08-09T13:35:28.920 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:28.915+0000 7fad0524c640 1 -- 172.21.15.43:0/2936789382 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fad0015c890 con 0x7fad000c86f0 2023-08-09T13:35:28.920 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:28.915+0000 7facf77fe640 1 -- 172.21.15.43:0/2936789382 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7face8003b40 con 0x7fad000c86f0 2023-08-09T13:35:28.920 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:28.915+0000 7fad0524c640 1 -- 172.21.15.43:0/2936789382 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fad0015ce50 con 0x7fad000c86f0 2023-08-09T13:35:28.921 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:28.916+0000 7fad0524c640 1 -- 172.21.15.43:0/2936789382 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fad000739f0 con 0x7fad000c86f0 2023-08-09T13:35:28.921 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:28.917+0000 7facf77fe640 1 -- 172.21.15.43:0/2936789382 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 5) v1 ==== 88341+0+0 (secure 0 0 0) 0x7face8003ca0 con 0x7fad000c86f0 2023-08-09T13:35:28.921 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:28.918+0000 7facf77fe640 1 --2- 172.21.15.43:0/2936789382 >> [v2:172.21.15.203:6832/830097474,v1:172.21.15.203:6833/830097474] conn(0x7face006a050 0x7face006c510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:35:28.921 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:28.918+0000 7facf77fe640 1 -- 172.21.15.43:0/2936789382 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 6871+0+0 (secure 0 0 0) 0x7face8087fb0 con 0x7fad000c86f0 2023-08-09T13:35:28.921 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:28.919+0000 7facfed9d640 1 --2- 172.21.15.43:0/2936789382 >> [v2:172.21.15.203:6832/830097474,v1:172.21.15.203:6833/830097474] conn(0x7face006a050 0x7face006c510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:35:28.923 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:28.921+0000 7facfed9d640 1 --2- 172.21.15.43:0/2936789382 >> [v2:172.21.15.203:6832/830097474,v1:172.21.15.203:6833/830097474] conn(0x7face006a050 0x7face006c510 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7facf0036000 tx=0x7facf003d040 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-09T13:35:28.924 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:28.922+0000 7facf77fe640 1 -- 172.21.15.43:0/2936789382 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178066 (secure 0 0 0) 0x7face80532d0 con 0x7fad000c86f0 2023-08-09T13:35:28.967 DEBUG:teuthology.orchestra.run.smithi043:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-08-09T13:35:28.975 DEBUG:teuthology.orchestra.run.smithi203:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-08-09T13:35:29.132 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:29.130+0000 7fad0524c640 1 -- 172.21.15.43:0/2936789382 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7fad000cec90 con 0x7fad000c86f0 2023-08-09T13:35:29.133 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:29.131+0000 7facf77fe640 1 -- 172.21.15.43:0/2936789382 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v26) v1 ==== 80+0+0 (secure 0 0 0) 0x7face8017080 con 0x7fad000c86f0 2023-08-09T13:35:29.133 INFO:teuthology.orchestra.run.smithi043.stderr:listed 0 entries 2023-08-09T13:35:29.136 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:29.134+0000 7fad0524c640 1 -- 172.21.15.43:0/2936789382 >> [v2:172.21.15.203:6832/830097474,v1:172.21.15.203:6833/830097474] conn(0x7face006a050 msgr2=0x7face006c510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:35:29.136 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:29.134+0000 7fad0524c640 1 --2- 172.21.15.43:0/2936789382 >> [v2:172.21.15.203:6832/830097474,v1:172.21.15.203:6833/830097474] conn(0x7face006a050 0x7face006c510 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7facf0036000 tx=0x7facf003d040 comp rx=0 tx=0).stop 2023-08-09T13:35:29.137 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:29.134+0000 7fad0524c640 1 -- 172.21.15.43:0/2936789382 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fad000c86f0 msgr2=0x7fad0015bdf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:35:29.137 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:29.134+0000 7fad0524c640 1 --2- 172.21.15.43:0/2936789382 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fad000c86f0 0x7fad0015bdf0 secure :-1 s=READY pgs=365 cs=0 l=1 rev1=1 crypto rx=0x7face800c410 tx=0x7face800c8d0 comp rx=0 tx=0).stop 2023-08-09T13:35:29.137 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:29.134+0000 7fad0524c640 1 -- 172.21.15.43:0/2936789382 shutdown_connections 2023-08-09T13:35:29.137 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:29.134+0000 7fad0524c640 1 --2- 172.21.15.43:0/2936789382 >> [v2:172.21.15.203:6832/830097474,v1:172.21.15.203:6833/830097474] conn(0x7face006a050 0x7face006c510 unknown :-1 s=CLOSED pgs=191 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:35:29.137 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:29.134+0000 7fad0524c640 1 --2- 172.21.15.43:0/2936789382 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fad00143c20 0x7fad0013fee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:35:29.137 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:29.134+0000 7fad0524c640 1 --2- 172.21.15.43:0/2936789382 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fad000c86f0 0x7fad0015bdf0 unknown :-1 s=CLOSED pgs=365 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:35:29.138 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:29.134+0000 7fad0524c640 1 --2- 172.21.15.43:0/2936789382 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fad0006cfe0 0x7fad000b3890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:35:29.138 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:29.134+0000 7fad0524c640 1 -- 172.21.15.43:0/2936789382 >> 172.21.15.43:0/2936789382 conn(0x7fad000ce0a0 msgr2=0x7fad000cb3b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-09T13:35:29.138 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:29.134+0000 7fad0524c640 1 -- 172.21.15.43:0/2936789382 shutdown_connections 2023-08-09T13:35:29.138 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:29.134+0000 7fad0524c640 1 -- 172.21.15.43:0/2936789382 wait complete. 2023-08-09T13:35:29.156 INFO:tasks.cephfs.fuse_mount:Running fusermount -u on ubuntu@smithi043.front.sepia.ceph.com... 2023-08-09T13:35:29.157 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-09T13:35:29.157 DEBUG:teuthology.orchestra.run.smithi043:> sudo fusermount -u /home/ubuntu/cephtest/mnt.0 2023-08-09T13:35:29.303 INFO:teuthology.orchestra.run:waiting for 300 2023-08-09T13:35:29.304 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi043.front.sepia.ceph.com 2023-08-09T13:35:29.304 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-09T13:35:29.305 DEBUG:teuthology.orchestra.run.smithi043:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2023-08-09T13:35:29.367 DEBUG:teuthology.orchestra.run.smithi203:> set -ex 2023-08-09T13:35:29.368 DEBUG:teuthology.orchestra.run.smithi203:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-09T13:35:29.430 DEBUG:teuthology.orchestra.run.smithi203:> set -ex 2023-08-09T13:35:29.430 DEBUG:teuthology.orchestra.run.smithi203:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-09T13:35:29.485 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-08-09T13:35:29.722 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:29.720+0000 7f2e7359e640 1 Processor -- start 2023-08-09T13:35:29.723 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:29.720+0000 7f2e7359e640 1 -- start start 2023-08-09T13:35:29.724 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:29.721+0000 7f2e7359e640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f2e7415a6a0 0x7f2e740713b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:35:29.724 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:29.721+0000 7f2e7359e640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f2e740cbd90 0x7f2e740c8fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:35:29.725 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:29.721+0000 7f2e7359e640 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7f2e740c94f0 0x7f2e7415ad20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:35:29.725 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:29.721+0000 7f2e7359e640 1 -- --> v2:172.21.15.43:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2e740c9960 con 0x7f2e740cbd90 2023-08-09T13:35:29.725 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:29.721+0000 7f2e7359e640 1 -- --> v2:172.21.15.203:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2e740cdc70 con 0x7f2e740c94f0 2023-08-09T13:35:29.725 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:29.722+0000 7f2e7359e640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2e74070910 con 0x7f2e7415a6a0 2023-08-09T13:35:29.725 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:29.722+0000 7f2e71d9b640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f2e740cbd90 0x7f2e740c8fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:35:29.726 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:29.722+0000 7f2e71d9b640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f2e740cbd90 0x7f2e740c8fb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:47042/0 (socket says 172.21.15.43:47042) 2023-08-09T13:35:29.726 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:29.722+0000 7f2e71d9b640 1 -- 172.21.15.43:0/3154976132 learned_addr learned my addr 172.21.15.43:0/3154976132 (peer_addr_for_me v2:172.21.15.43:0/0) 2023-08-09T13:35:29.726 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:29.722+0000 7f2e7259c640 1 --2- 172.21.15.43:0/3154976132 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f2e7415a6a0 0x7f2e740713b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:35:29.727 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:29.722+0000 7f2e72d9d640 1 --2- 172.21.15.43:0/3154976132 >> v2:172.21.15.203:3300/0 conn(0x7f2e740c94f0 0x7f2e7415ad20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:35:29.727 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:29.722+0000 7f2e71d9b640 1 -- 172.21.15.43:0/3154976132 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f2e7415a6a0 msgr2=0x7f2e740713b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-09T13:35:29.727 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:29.722+0000 7f2e71d9b640 1 --2- 172.21.15.43:0/3154976132 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f2e7415a6a0 0x7f2e740713b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:35:29.727 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:29.722+0000 7f2e71d9b640 1 -- 172.21.15.43:0/3154976132 >> v2:172.21.15.203:3300/0 conn(0x7f2e740c94f0 msgr2=0x7f2e7415ad20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-09T13:35:29.727 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:29.722+0000 7f2e71d9b640 1 --2- 172.21.15.43:0/3154976132 >> v2:172.21.15.203:3300/0 conn(0x7f2e740c94f0 0x7f2e7415ad20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:35:29.727 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:29.722+0000 7f2e71d9b640 1 -- 172.21.15.43:0/3154976132 --> v2:172.21.15.43:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2e740c88c0 con 0x7f2e740cbd90 2023-08-09T13:35:29.728 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:29.724+0000 7f2e71d9b640 1 --2- 172.21.15.43:0/3154976132 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f2e740cbd90 0x7f2e740c8fb0 secure :-1 s=READY pgs=366 cs=0 l=1 rev1=1 crypto rx=0x7f2e7415dec0 tx=0x7f2e6802b880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=137615040ce374c5 server_cookie=0 in_seq=0 out_seq=0 2023-08-09T13:35:29.728 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:29.724+0000 7f2e7159a640 1 -- 172.21.15.43:0/3154976132 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2e680317e0 con 0x7f2e740cbd90 2023-08-09T13:35:29.728 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:29.724+0000 7f2e7159a640 1 -- 172.21.15.43:0/3154976132 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2e68031940 con 0x7f2e740cbd90 2023-08-09T13:35:29.728 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:29.724+0000 7f2e7159a640 1 -- 172.21.15.43:0/3154976132 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2e68031cd0 con 0x7f2e740cbd90 2023-08-09T13:35:29.729 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:29.725+0000 7f2e7359e640 1 -- 172.21.15.43:0/3154976132 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f2e740cbd90 msgr2=0x7f2e740c8fb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:35:29.729 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:29.725+0000 7f2e7359e640 1 --2- 172.21.15.43:0/3154976132 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f2e740cbd90 0x7f2e740c8fb0 secure :-1 s=READY pgs=366 cs=0 l=1 rev1=1 crypto rx=0x7f2e7415dec0 tx=0x7f2e6802b880 comp rx=0 tx=0).stop 2023-08-09T13:35:29.729 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:29.725+0000 7f2e7359e640 1 -- 172.21.15.43:0/3154976132 shutdown_connections 2023-08-09T13:35:29.729 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:29.725+0000 7f2e7359e640 1 --2- 172.21.15.43:0/3154976132 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f2e740cbd90 0x7f2e740c8fb0 unknown :-1 s=CLOSED pgs=366 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:35:29.729 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:29.725+0000 7f2e7359e640 1 --2- 172.21.15.43:0/3154976132 >> v2:172.21.15.203:3300/0 conn(0x7f2e740c94f0 0x7f2e7415ad20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:35:29.729 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:29.725+0000 7f2e7359e640 1 --2- 172.21.15.43:0/3154976132 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f2e7415a6a0 0x7f2e740713b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:35:29.730 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:29.725+0000 7f2e7359e640 1 -- 172.21.15.43:0/3154976132 >> 172.21.15.43:0/3154976132 conn(0x7f2e740cde90 msgr2=0x7f2e740ce2a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-09T13:35:29.730 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:29.725+0000 7f2e7359e640 1 -- 172.21.15.43:0/3154976132 shutdown_connections 2023-08-09T13:35:29.730 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:29.725+0000 7f2e7359e640 1 -- 172.21.15.43:0/3154976132 wait complete. 2023-08-09T13:35:29.731 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:29.727+0000 7f2e7359e640 1 Processor -- start 2023-08-09T13:35:29.731 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:29.727+0000 7f2e7359e640 1 -- start start 2023-08-09T13:35:29.731 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:29.728+0000 7f2e7359e640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f2e740c94f0 0x7f2e741591d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:35:29.732 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:29.728+0000 7f2e7359e640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f2e740cbd90 0x7f2e741589b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:35:29.732 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:29.728+0000 7f2e7359e640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f2e7415a6a0 0x7f2e740c6290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:35:29.732 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:29.728+0000 7f2e7359e640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2e740c4770 con 0x7f2e7415a6a0 2023-08-09T13:35:29.732 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:29.728+0000 7f2e7359e640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2e74159710 con 0x7f2e740c94f0 2023-08-09T13:35:29.732 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:29.728+0000 7f2e7359e640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2e740c5e70 con 0x7f2e740cbd90 2023-08-09T13:35:29.732 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:29.728+0000 7f2e72d9d640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f2e7415a6a0 0x7f2e740c6290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:35:29.733 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:29.728+0000 7f2e72d9d640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f2e7415a6a0 0x7f2e740c6290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:47058/0 (socket says 172.21.15.43:47058) 2023-08-09T13:35:29.733 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:29.728+0000 7f2e72d9d640 1 -- 172.21.15.43:0/3683102812 learned_addr learned my addr 172.21.15.43:0/3683102812 (peer_addr_for_me v2:172.21.15.43:0/0) 2023-08-09T13:35:29.733 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:29.729+0000 7f2e71d9b640 1 --2- 172.21.15.43:0/3683102812 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f2e740cbd90 0x7f2e741589b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:35:29.733 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:29.729+0000 7f2e7259c640 1 --2- 172.21.15.43:0/3683102812 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f2e740c94f0 0x7f2e741591d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:35:29.733 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:29.729+0000 7f2e72d9d640 1 -- 172.21.15.43:0/3683102812 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f2e740cbd90 msgr2=0x7f2e741589b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:35:29.733 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:29.729+0000 7f2e72d9d640 1 --2- 172.21.15.43:0/3683102812 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f2e740cbd90 0x7f2e741589b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:35:29.733 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:29.729+0000 7f2e72d9d640 1 -- 172.21.15.43:0/3683102812 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f2e740c94f0 msgr2=0x7f2e741591d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:35:29.734 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:29.729+0000 7f2e72d9d640 1 --2- 172.21.15.43:0/3683102812 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f2e740c94f0 0x7f2e741591d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:35:29.734 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:29.729+0000 7f2e72d9d640 1 -- 172.21.15.43:0/3683102812 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2e680094d0 con 0x7f2e7415a6a0 2023-08-09T13:35:29.734 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:29.729+0000 7f2e72d9d640 1 --2- 172.21.15.43:0/3683102812 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f2e7415a6a0 0x7f2e740c6290 secure :-1 s=READY pgs=367 cs=0 l=1 rev1=1 crypto rx=0x7f2e6c00cb80 tx=0x7f2e6c00a4c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-09T13:35:29.735 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:29.729+0000 7f2e62ffd640 1 -- 172.21.15.43:0/3683102812 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2e6c00ce70 con 0x7f2e7415a6a0 2023-08-09T13:35:29.735 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:29.729+0000 7f2e62ffd640 1 -- 172.21.15.43:0/3683102812 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2e6c00abd0 con 0x7f2e7415a6a0 2023-08-09T13:35:29.735 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:29.729+0000 7f2e62ffd640 1 -- 172.21.15.43:0/3683102812 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2e6c011410 con 0x7f2e7415a6a0 2023-08-09T13:35:29.735 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:29.729+0000 7f2e7359e640 1 -- 172.21.15.43:0/3683102812 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2e74098bd0 con 0x7f2e7415a6a0 2023-08-09T13:35:29.736 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:29.729+0000 7f2e7359e640 1 -- 172.21.15.43:0/3683102812 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2e74099190 con 0x7f2e7415a6a0 2023-08-09T13:35:29.736 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:29.731+0000 7f2e7359e640 1 -- 172.21.15.43:0/3683102812 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2e74099380 con 0x7f2e7415a6a0 2023-08-09T13:35:29.736 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:29.731+0000 7f2e62ffd640 1 -- 172.21.15.43:0/3683102812 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 5) v1 ==== 88341+0+0 (secure 0 0 0) 0x7f2e6c011570 con 0x7f2e7415a6a0 2023-08-09T13:35:29.737 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:29.733+0000 7f2e62ffd640 1 --2- 172.21.15.43:0/3683102812 >> [v2:172.21.15.203:6832/830097474,v1:172.21.15.203:6833/830097474] conn(0x7f2e5406a050 0x7f2e5406c510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:35:29.737 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:29.733+0000 7f2e62ffd640 1 -- 172.21.15.43:0/3683102812 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 6871+0+0 (secure 0 0 0) 0x7f2e6c088700 con 0x7f2e7415a6a0 2023-08-09T13:35:29.737 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:29.734+0000 7f2e7259c640 1 --2- 172.21.15.43:0/3683102812 >> [v2:172.21.15.203:6832/830097474,v1:172.21.15.203:6833/830097474] conn(0x7f2e5406a050 0x7f2e5406c510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:35:29.737 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:29.735+0000 7f2e7259c640 1 --2- 172.21.15.43:0/3683102812 >> [v2:172.21.15.203:6832/830097474,v1:172.21.15.203:6833/830097474] conn(0x7f2e5406a050 0x7f2e5406c510 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7f2e64006fd0 tx=0x7f2e64008040 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-09T13:35:29.739 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:29.737+0000 7f2e62ffd640 1 -- 172.21.15.43:0/3683102812 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178066 (secure 0 0 0) 0x7f2e6c053ab0 con 0x7f2e7415a6a0 2023-08-09T13:35:29.933 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:29.930+0000 7f2e7359e640 1 -- 172.21.15.43:0/3683102812 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f2e740739f0 con 0x7f2e7415a6a0 2023-08-09T13:35:29.936 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:29.934+0000 7f2e62ffd640 1 -- 172.21.15.43:0/3683102812 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v26) v1 ==== 74+0+20146 (secure 0 0 0) 0x7f2e6c0570d0 con 0x7f2e7415a6a0 2023-08-09T13:35:29.937 INFO:teuthology.orchestra.run.smithi043.stdout: 2023-08-09T13:35:29.937 INFO:teuthology.orchestra.run.smithi043.stdout:{"epoch":26,"fsid":"daac87ee-1337-4250-a955-aea4e408a39b","created":"2023-08-09T13:33:41.701198+0000","modified":"2023-08-09T13:34:48.711911+0000","last_up_change":"2023-08-09T13:33:53.783453+0000","last_in_change":"2023-08-09T13:33:45.148904+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":5,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":"rbd","create_time":"2023-08-09T13:33:54.365137+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":".mgr","create_time":"2023-08-09T13:33:54.831803+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_metadata","create_time":"2023-08-09T13:34:00.207974+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data","create_time":"2023-08-09T13:34:01.614765+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":5,"pool_name":"cephfs_data_ec","create_time":"2023-08-09T13:34:05.239583+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"24","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"cbd49ecd-e896-4fbd-864e-5c37a829b165","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6800","nonce":502667335},{"type":"v1","addr":"172.21.15.43:6801","nonce":502667335}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6802","nonce":502667335},{"type":"v1","addr":"172.21.15.43:6803","nonce":502667335}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6806","nonce":502667335},{"type":"v1","addr":"172.21.15.43:6807","nonce":502667335}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6804","nonce":502667335},{"type":"v1","addr":"172.21.15.43:6805","nonce":502667335}]},"public_addr":"172.21.15.43:6801/502667335","cluster_addr":"172.21.15.43:6803/502667335","heartbeat_back_addr":"172.21.15.43:6807/502667335","heartbeat_front_addr":"172.21.15.43:6805/502667335","state":["exists","up"]},{"osd":1,"uuid":"75c7e0fe-1ec7-4fe8-a2f2-3661a65b8500","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6808","nonce":3289134610},{"type":"v1","addr":"172.21.15.43:6809","nonce":3289134610}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6810","nonce":3289134610},{"type":"v1","addr":"172.21.15.43:6812","nonce":3289134610}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6821","nonce":3289134610},{"type":"v1","addr":"172.21.15.43:6824","nonce":3289134610}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6815","nonce":3289134610},{"type":"v1","addr":"172.21.15.43:6818","nonce":3289134610}]},"public_addr":"172.21.15.43:6809/3289134610","cluster_addr":"172.21.15.43:6812/3289134610","heartbeat_back_addr":"172.21.15.43:6824/3289134610","heartbeat_front_addr":"172.21.15.43:6818/3289134610","state":["exists","up"]},{"osd":2,"uuid":"bb5c41e9-7f5f-4125-9b01-3bb8bfc2ea83","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6811","nonce":4079709736},{"type":"v1","addr":"172.21.15.43:6814","nonce":4079709736}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6817","nonce":4079709736},{"type":"v1","addr":"172.21.15.43:6820","nonce":4079709736}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6829","nonce":4079709736},{"type":"v1","addr":"172.21.15.43:6831","nonce":4079709736}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6823","nonce":4079709736},{"type":"v1","addr":"172.21.15.43:6826","nonce":4079709736}]},"public_addr":"172.21.15.43:6814/4079709736","cluster_addr":"172.21.15.43:6820/4079709736","heartbeat_back_addr":"172.21.15.43:6831/4079709736","heartbeat_front_addr":"172.21.15.43:6826/4079709736","state":["exists","up"]},{"osd":3,"uuid":"bfe4c8b4-694a-465c-87e6-2b2aab852613","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6813","nonce":1709538759},{"type":"v1","addr":"172.21.15.43:6816","nonce":1709538759}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6819","nonce":1709538759},{"type":"v1","addr":"172.21.15.43:6822","nonce":1709538759}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6828","nonce":1709538759},{"type":"v1","addr":"172.21.15.43:6830","nonce":1709538759}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6825","nonce":1709538759},{"type":"v1","addr":"172.21.15.43:6827","nonce":1709538759}]},"public_addr":"172.21.15.43:6816/1709538759","cluster_addr":"172.21.15.43:6822/1709538759","heartbeat_back_addr":"172.21.15.43:6830/1709538759","heartbeat_front_addr":"172.21.15.43:6827/1709538759","state":["exists","up"]},{"osd":4,"uuid":"fa2e8a77-aba1-4756-853e-0d174a5bdeba","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6800","nonce":4196866481},{"type":"v1","addr":"172.21.15.203:6801","nonce":4196866481}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6802","nonce":4196866481},{"type":"v1","addr":"172.21.15.203:6803","nonce":4196866481}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6806","nonce":4196866481},{"type":"v1","addr":"172.21.15.203:6807","nonce":4196866481}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6804","nonce":4196866481},{"type":"v1","addr":"172.21.15.203:6805","nonce":4196866481}]},"public_addr":"172.21.15.203:6801/4196866481","cluster_addr":"172.21.15.203:6803/4196866481","heartbeat_back_addr":"172.21.15.203:6807/4196866481","heartbeat_front_addr":"172.21.15.203:6805/4196866481","state":["exists","up"]},{"osd":5,"uuid":"81811e4b-4d24-4f51-a82c-f1d700a2a00d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6816","nonce":3059290081},{"type":"v1","addr":"172.21.15.203:6817","nonce":3059290081}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6818","nonce":3059290081},{"type":"v1","addr":"172.21.15.203:6819","nonce":3059290081}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6822","nonce":3059290081},{"type":"v1","addr":"172.21.15.203:6823","nonce":3059290081}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6820","nonce":3059290081},{"type":"v1","addr":"172.21.15.203:6821","nonce":3059290081}]},"public_addr":"172.21.15.203:6817/3059290081","cluster_addr":"172.21.15.203:6819/3059290081","heartbeat_back_addr":"172.21.15.203:6823/3059290081","heartbeat_front_addr":"172.21.15.203:6821/3059290081","state":["exists","up"]},{"osd":6,"uuid":"307bac4d-6a5b-443d-af75-56d695d7674f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6824","nonce":512155569},{"type":"v1","addr":"172.21.15.203:6825","nonce":512155569}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6826","nonce":512155569},{"type":"v1","addr":"172.21.15.203:6827","nonce":512155569}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6830","nonce":512155569},{"type":"v1","addr":"172.21.15.203:6831","nonce":512155569}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6828","nonce":512155569},{"type":"v1","addr":"172.21.15.203:6829","nonce":512155569}]},"public_addr":"172.21.15.203:6825/512155569","cluster_addr":"172.21.15.203:6827/512155569","heartbeat_back_addr":"172.21.15.203:6831/512155569","heartbeat_front_addr":"172.21.15.203:6829/512155569","state":["exists","up"]},{"osd":7,"uuid":"a95dc7ca-9f1b-4295-af70-68a22a2d7d5e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6808","nonce":3331450438},{"type":"v1","addr":"172.21.15.203:6809","nonce":3331450438}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6810","nonce":3331450438},{"type":"v1","addr":"172.21.15.203:6811","nonce":3331450438}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6814","nonce":3331450438},{"type":"v1","addr":"172.21.15.203:6815","nonce":3331450438}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6812","nonce":3331450438},{"type":"v1","addr":"172.21.15.203:6813","nonce":3331450438}]},"public_addr":"172.21.15.203:6809/3331450438","cluster_addr":"172.21.15.203:6811/3331450438","heartbeat_back_addr":"172.21.15.203:6815/3331450438","heartbeat_front_addr":"172.21.15.203:6813/3331450438","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-09T13:33:50.547652+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-09T13:33:50.589469+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-09T13:33:50.561724+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-09T13:33:50.599650+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-09T13:33:50.411855+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-09T13:33:50.219681+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-09T13:33:50.504664+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-09T13:33:50.441248+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"4.14","mappings":[{"from":1,"to":5}]},{"pgid":"4.29","mappings":[{"from":3,"to":2}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-08-09T13:35:29.939 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:29.937+0000 7f2e7359e640 1 -- 172.21.15.43:0/3683102812 >> [v2:172.21.15.203:6832/830097474,v1:172.21.15.203:6833/830097474] conn(0x7f2e5406a050 msgr2=0x7f2e5406c510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:35:29.940 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:29.937+0000 7f2e7359e640 1 --2- 172.21.15.43:0/3683102812 >> [v2:172.21.15.203:6832/830097474,v1:172.21.15.203:6833/830097474] conn(0x7f2e5406a050 0x7f2e5406c510 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7f2e64006fd0 tx=0x7f2e64008040 comp rx=0 tx=0).stop 2023-08-09T13:35:29.940 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:29.937+0000 7f2e7359e640 1 -- 172.21.15.43:0/3683102812 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f2e7415a6a0 msgr2=0x7f2e740c6290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:35:29.940 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:29.937+0000 7f2e7359e640 1 --2- 172.21.15.43:0/3683102812 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f2e7415a6a0 0x7f2e740c6290 secure :-1 s=READY pgs=367 cs=0 l=1 rev1=1 crypto rx=0x7f2e6c00cb80 tx=0x7f2e6c00a4c0 comp rx=0 tx=0).stop 2023-08-09T13:35:29.940 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:29.937+0000 7f2e7359e640 1 -- 172.21.15.43:0/3683102812 shutdown_connections 2023-08-09T13:35:29.940 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:29.937+0000 7f2e7359e640 1 --2- 172.21.15.43:0/3683102812 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f2e7415a6a0 0x7f2e740c6290 unknown :-1 s=CLOSED pgs=367 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:35:29.940 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:29.937+0000 7f2e7359e640 1 --2- 172.21.15.43:0/3683102812 >> [v2:172.21.15.203:6832/830097474,v1:172.21.15.203:6833/830097474] conn(0x7f2e5406a050 0x7f2e5406c510 unknown :-1 s=CLOSED pgs=192 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:35:29.940 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:29.937+0000 7f2e7359e640 1 --2- 172.21.15.43:0/3683102812 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f2e740cbd90 0x7f2e741589b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:35:29.941 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:29.937+0000 7f2e7359e640 1 --2- 172.21.15.43:0/3683102812 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f2e740c94f0 0x7f2e741591d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:35:29.941 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:29.937+0000 7f2e7359e640 1 -- 172.21.15.43:0/3683102812 >> 172.21.15.43:0/3683102812 conn(0x7f2e740cde90 msgr2=0x7f2e740c9ae0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-09T13:35:29.941 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:29.937+0000 7f2e7359e640 1 -- 172.21.15.43:0/3683102812 shutdown_connections 2023-08-09T13:35:29.941 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:29.937+0000 7f2e7359e640 1 -- 172.21.15.43:0/3683102812 wait complete. 2023-08-09T13:35:29.958 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': 'rbd', 'create_time': '2023-08-09T13:33:54.365137+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}, 'read_balance': {'score_acting': 2, 'score_stable': 2, 'optimal_score': 1, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': '.mgr', 'create_time': '2023-08-09T13:33:54.831803+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_metadata', 'create_time': '2023-08-09T13:34:00.207974+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data', 'create_time': '2023-08-09T13:34:01.614765+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 2, 'score_stable': 2, 'optimal_score': 1, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 5, 'pool_name': 'cephfs_data_ec', 'create_time': '2023-08-09T13:34:05.239583+0000', 'flags': 5, 'flags_names': 'hashpspool,ec_overwrites', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '24', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2023-08-09T13:35:29.959 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get rbd pg_num 2023-08-09T13:35:30.191 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:30.189+0000 7fd891867640 1 Processor -- start 2023-08-09T13:35:30.192 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:30.189+0000 7fd891867640 1 -- start start 2023-08-09T13:35:30.193 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:30.190+0000 7fd891867640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fd88c15c550 0x7fd88c0cd6d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:35:30.193 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:30.190+0000 7fd891867640 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7fd88c0d04b0 0x7fd88c0b3ee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:35:30.193 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:30.190+0000 7fd891867640 1 -- --> v1:172.21.15.203:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd88c0b4870 con 0x7fd88c0b4420 2023-08-09T13:35:30.193 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:30.190+0000 7fd891867640 1 -- --> v2:172.21.15.203:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd88c0b7bd0 con 0x7fd88c0d04b0 2023-08-09T13:35:30.193 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:30.190+0000 7fd891867640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd88c0d1c00 con 0x7fd88c15c550 2023-08-09T13:35:30.194 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:30.191+0000 7fd88affd640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fd88c15c550 0x7fd88c0cd6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:35:30.194 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:30.191+0000 7fd88affd640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fd88c15c550 0x7fd88c0cd6d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.203:3301/0 says I am v2:172.21.15.43:54708/0 (socket says 172.21.15.43:54708) 2023-08-09T13:35:30.194 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:30.191+0000 7fd88affd640 1 -- 172.21.15.43:0/3518614459 learned_addr learned my addr 172.21.15.43:0/3518614459 (peer_addr_for_me v2:172.21.15.43:0/0) 2023-08-09T13:35:30.195 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:30.192+0000 7fd883fff640 1 --2- 172.21.15.43:0/3518614459 >> v2:172.21.15.203:3300/0 conn(0x7fd88c0d04b0 0x7fd88c0b3ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:35:30.195 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:30.192+0000 7fd88affd640 1 -- 172.21.15.43:0/3518614459 >> v2:172.21.15.203:3300/0 conn(0x7fd88c0d04b0 msgr2=0x7fd88c0b3ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-09T13:35:30.195 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:30.192+0000 7fd88affd640 1 --2- 172.21.15.43:0/3518614459 >> v2:172.21.15.203:3300/0 conn(0x7fd88c0d04b0 0x7fd88c0b3ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:35:30.195 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:30.192+0000 7fd88affd640 1 -- 172.21.15.43:0/3518614459 >> v1:172.21.15.203:6789/0 conn(0x7fd88c0b4420 legacy=0x7fd88c06a760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:35:30.196 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:30.192+0000 7fd88affd640 1 -- 172.21.15.43:0/3518614459 --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd88c15c930 con 0x7fd88c15c550 2023-08-09T13:35:30.196 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:30.194+0000 7fd88affd640 1 --2- 172.21.15.43:0/3518614459 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fd88c15c550 0x7fd88c0cd6d0 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7fd87c009870 tx=0x7fd87c02b980 comp rx=0 tx=0).ready entity=mon.2 client_cookie=50881c0a6db2d3bd server_cookie=0 in_seq=0 out_seq=0 2023-08-09T13:35:30.197 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:30.194+0000 7fd88a7fc640 1 -- 172.21.15.43:0/3518614459 <== mon.2 v2:172.21.15.203:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd87c030840 con 0x7fd88c15c550 2023-08-09T13:35:30.197 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:30.194+0000 7fd88a7fc640 1 -- 172.21.15.43:0/3518614459 <== mon.2 v2:172.21.15.203:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd87c0309a0 con 0x7fd88c15c550 2023-08-09T13:35:30.197 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:30.194+0000 7fd88a7fc640 1 -- 172.21.15.43:0/3518614459 <== mon.2 v2:172.21.15.203:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd87c030d30 con 0x7fd88c15c550 2023-08-09T13:35:30.198 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:30.195+0000 7fd891867640 1 -- 172.21.15.43:0/3518614459 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fd88c15c550 msgr2=0x7fd88c0cd6d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:35:30.198 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:30.195+0000 7fd891867640 1 --2- 172.21.15.43:0/3518614459 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fd88c15c550 0x7fd88c0cd6d0 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7fd87c009870 tx=0x7fd87c02b980 comp rx=0 tx=0).stop 2023-08-09T13:35:30.198 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:30.195+0000 7fd891867640 1 -- 172.21.15.43:0/3518614459 shutdown_connections 2023-08-09T13:35:30.198 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:30.195+0000 7fd891867640 1 --2- 172.21.15.43:0/3518614459 >> v2:172.21.15.203:3300/0 conn(0x7fd88c0d04b0 0x7fd88c0b3ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:35:30.198 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:30.195+0000 7fd891867640 1 --2- 172.21.15.43:0/3518614459 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fd88c15c550 0x7fd88c0cd6d0 unknown :-1 s=CLOSED pgs=66 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:35:30.198 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:30.195+0000 7fd891867640 1 -- 172.21.15.43:0/3518614459 >> 172.21.15.43:0/3518614459 conn(0x7fd88c071520 msgr2=0x7fd88c0ceee0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-09T13:35:30.198 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:30.195+0000 7fd891867640 1 -- 172.21.15.43:0/3518614459 shutdown_connections 2023-08-09T13:35:30.199 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:30.195+0000 7fd891867640 1 -- 172.21.15.43:0/3518614459 wait complete. 2023-08-09T13:35:30.199 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:30.197+0000 7fd891867640 1 Processor -- start 2023-08-09T13:35:30.200 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:30.197+0000 7fd891867640 1 -- start start 2023-08-09T13:35:30.200 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:30.198+0000 7fd891867640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fd88c0b4420 0x7fd88c14bae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:35:30.201 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:30.198+0000 7fd891867640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fd88c0d04b0 0x7fd88c0cb750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:35:30.201 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:30.198+0000 7fd891867640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fd88c15c550 0x7fd88c0ab7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:35:30.201 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:30.198+0000 7fd891867640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd88c0ca6b0 con 0x7fd88c0d04b0 2023-08-09T13:35:30.201 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:30.198+0000 7fd891867640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd88c0ca990 con 0x7fd88c15c550 2023-08-09T13:35:30.201 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:30.198+0000 7fd891867640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd88c0cac70 con 0x7fd88c0b4420 2023-08-09T13:35:30.202 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:30.199+0000 7fd883fff640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fd88c0d04b0 0x7fd88c0cb750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:35:30.202 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:30.199+0000 7fd883fff640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fd88c0d04b0 0x7fd88c0cb750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:47060/0 (socket says 172.21.15.43:47060) 2023-08-09T13:35:30.202 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:30.199+0000 7fd883fff640 1 -- 172.21.15.43:0/3615418169 learned_addr learned my addr 172.21.15.43:0/3615418169 (peer_addr_for_me v2:172.21.15.43:0/0) 2023-08-09T13:35:30.203 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:30.199+0000 7fd88affd640 1 --2- 172.21.15.43:0/3615418169 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fd88c0b4420 0x7fd88c14bae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:35:30.203 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:30.199+0000 7fd88b7fe640 1 --2- 172.21.15.43:0/3615418169 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fd88c15c550 0x7fd88c0ab7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:35:30.203 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:30.199+0000 7fd883fff640 1 -- 172.21.15.43:0/3615418169 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fd88c0b4420 msgr2=0x7fd88c14bae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:35:30.203 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:30.199+0000 7fd883fff640 1 --2- 172.21.15.43:0/3615418169 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fd88c0b4420 0x7fd88c14bae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:35:30.203 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:30.199+0000 7fd883fff640 1 -- 172.21.15.43:0/3615418169 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fd88c15c550 msgr2=0x7fd88c0ab7a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:35:30.203 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:30.199+0000 7fd883fff640 1 --2- 172.21.15.43:0/3615418169 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fd88c15c550 0x7fd88c0ab7a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:35:30.203 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:30.199+0000 7fd883fff640 1 -- 172.21.15.43:0/3615418169 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd87c0094d0 con 0x7fd88c0d04b0 2023-08-09T13:35:30.204 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:30.199+0000 7fd88b7fe640 1 --2- 172.21.15.43:0/3615418169 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fd88c15c550 0x7fd88c0ab7a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-08-09T13:35:30.204 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:30.200+0000 7fd883fff640 1 --2- 172.21.15.43:0/3615418169 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fd88c0d04b0 0x7fd88c0cb750 secure :-1 s=READY pgs=368 cs=0 l=1 rev1=1 crypto rx=0x7fd87400e9b0 tx=0x7fd87400ee70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-09T13:35:30.204 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:30.200+0000 7fd888ff9640 1 -- 172.21.15.43:0/3615418169 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd874009670 con 0x7fd88c0d04b0 2023-08-09T13:35:30.205 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:30.200+0000 7fd888ff9640 1 -- 172.21.15.43:0/3615418169 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd87400c9a0 con 0x7fd88c0d04b0 2023-08-09T13:35:30.205 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:30.200+0000 7fd891867640 1 -- 172.21.15.43:0/3615418169 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd88c0caf50 con 0x7fd88c0d04b0 2023-08-09T13:35:30.205 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:30.200+0000 7fd888ff9640 1 -- 172.21.15.43:0/3615418169 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd87400cd50 con 0x7fd88c0d04b0 2023-08-09T13:35:30.206 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:30.200+0000 7fd891867640 1 -- 172.21.15.43:0/3615418169 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd88c14c020 con 0x7fd88c0d04b0 2023-08-09T13:35:30.206 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:30.202+0000 7fd891867640 1 -- 172.21.15.43:0/3615418169 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd88c0739f0 con 0x7fd88c0d04b0 2023-08-09T13:35:30.206 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:30.203+0000 7fd888ff9640 1 -- 172.21.15.43:0/3615418169 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 5) v1 ==== 88341+0+0 (secure 0 0 0) 0x7fd874013470 con 0x7fd88c0d04b0 2023-08-09T13:35:30.207 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:30.204+0000 7fd888ff9640 1 --2- 172.21.15.43:0/3615418169 >> [v2:172.21.15.203:6832/830097474,v1:172.21.15.203:6833/830097474] conn(0x7fd86c06a0a0 0x7fd86c06c560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:35:30.208 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:30.205+0000 7fd888ff9640 1 -- 172.21.15.43:0/3615418169 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 6871+0+0 (secure 0 0 0) 0x7fd8740889d0 con 0x7fd88c0d04b0 2023-08-09T13:35:30.208 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:30.205+0000 7fd88affd640 1 --2- 172.21.15.43:0/3615418169 >> [v2:172.21.15.203:6832/830097474,v1:172.21.15.203:6833/830097474] conn(0x7fd86c06a0a0 0x7fd86c06c560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:35:30.209 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:30.207+0000 7fd88affd640 1 --2- 172.21.15.43:0/3615418169 >> [v2:172.21.15.203:6832/830097474,v1:172.21.15.203:6833/830097474] conn(0x7fd86c06a0a0 0x7fd86c06c560 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7fd87c036000 tx=0x7fd87c03b040 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-09T13:35:30.211 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:30.209+0000 7fd888ff9640 1 -- 172.21.15.43:0/3615418169 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178066 (secure 0 0 0) 0x7fd874053eb0 con 0x7fd88c0d04b0 2023-08-09T13:35:30.405 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:30.402+0000 7fd891867640 1 -- 172.21.15.43:0/3615418169 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"} v 0) v1 -- 0x7fd88c0cf2f0 con 0x7fd88c0d04b0 2023-08-09T13:35:30.406 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:30.403+0000 7fd888ff9640 1 -- 172.21.15.43:0/3615418169 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"}]=0 v26) v1 ==== 92+0+10 (secure 0 0 0) 0x7fd8740574d0 con 0x7fd88c0d04b0 2023-08-09T13:35:30.406 INFO:teuthology.orchestra.run.smithi043.stdout:pg_num: 8 2023-08-09T13:35:30.408 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:30.405+0000 7fd891867640 1 -- 172.21.15.43:0/3615418169 >> [v2:172.21.15.203:6832/830097474,v1:172.21.15.203:6833/830097474] conn(0x7fd86c06a0a0 msgr2=0x7fd86c06c560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:35:30.409 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:30.405+0000 7fd891867640 1 --2- 172.21.15.43:0/3615418169 >> [v2:172.21.15.203:6832/830097474,v1:172.21.15.203:6833/830097474] conn(0x7fd86c06a0a0 0x7fd86c06c560 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7fd87c036000 tx=0x7fd87c03b040 comp rx=0 tx=0).stop 2023-08-09T13:35:30.409 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:30.405+0000 7fd891867640 1 -- 172.21.15.43:0/3615418169 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fd88c0d04b0 msgr2=0x7fd88c0cb750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:35:30.409 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:30.405+0000 7fd891867640 1 --2- 172.21.15.43:0/3615418169 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fd88c0d04b0 0x7fd88c0cb750 secure :-1 s=READY pgs=368 cs=0 l=1 rev1=1 crypto rx=0x7fd87400e9b0 tx=0x7fd87400ee70 comp rx=0 tx=0).stop 2023-08-09T13:35:30.410 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:30.405+0000 7fd891867640 1 -- 172.21.15.43:0/3615418169 shutdown_connections 2023-08-09T13:35:30.410 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:30.405+0000 7fd891867640 1 --2- 172.21.15.43:0/3615418169 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fd88c15c550 0x7fd88c0ab7a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:35:30.410 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:30.405+0000 7fd891867640 1 --2- 172.21.15.43:0/3615418169 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fd88c0d04b0 0x7fd88c0cb750 unknown :-1 s=CLOSED pgs=368 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:35:30.410 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:30.405+0000 7fd891867640 1 --2- 172.21.15.43:0/3615418169 >> [v2:172.21.15.203:6832/830097474,v1:172.21.15.203:6833/830097474] conn(0x7fd86c06a0a0 0x7fd86c06c560 unknown :-1 s=CLOSED pgs=193 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:35:30.411 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:30.405+0000 7fd891867640 1 --2- 172.21.15.43:0/3615418169 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fd88c0b4420 0x7fd88c14bae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:35:30.411 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:30.405+0000 7fd891867640 1 -- 172.21.15.43:0/3615418169 >> 172.21.15.43:0/3615418169 conn(0x7fd88c071520 msgr2=0x7fd88c0cc940 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-09T13:35:30.411 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:30.406+0000 7fd891867640 1 -- 172.21.15.43:0/3615418169 shutdown_connections 2023-08-09T13:35:30.411 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:30.406+0000 7fd891867640 1 -- 172.21.15.43:0/3615418169 wait complete. 2023-08-09T13:35:30.427 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-08-09T13:35:30.653 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:30.650+0000 7fb287fff640 1 Processor -- start 2023-08-09T13:35:30.653 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:30.651+0000 7fb287fff640 1 -- start start 2023-08-09T13:35:30.660 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:30.652+0000 7fb287fff640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fb28815e2d0 0x7fb28815e6b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:35:30.660 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:30.652+0000 7fb287fff640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7fb2880b1530 0x7fb288069fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:35:30.660 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:30.652+0000 7fb287fff640 1 -- --> v1:172.21.15.203:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb28806b270 con 0x7fb28815a720 2023-08-09T13:35:30.660 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:30.652+0000 7fb287fff640 1 -- --> v2:172.21.15.43:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb28815ebf0 con 0x7fb2880b1530 2023-08-09T13:35:30.661 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:30.652+0000 7fb287fff640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb28815b010 con 0x7fb28815e2d0 2023-08-09T13:35:30.661 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:30.652+0000 7fb2877fe640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7fb2880b1530 0x7fb288069fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:35:30.661 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:30.652+0000 7fb2877fe640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7fb2880b1530 0x7fb288069fc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:47066/0 (socket says 172.21.15.43:47066) 2023-08-09T13:35:30.661 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:30.653+0000 7fb2877fe640 1 -- 172.21.15.43:0/4198815373 learned_addr learned my addr 172.21.15.43:0/4198815373 (peer_addr_for_me v2:172.21.15.43:0/0) 2023-08-09T13:35:30.661 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:30.653+0000 7fb286ffd640 1 --2- 172.21.15.43:0/4198815373 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fb28815e2d0 0x7fb28815e6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:35:30.661 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:30.653+0000 7fb2877fe640 1 -- 172.21.15.43:0/4198815373 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fb28815e2d0 msgr2=0x7fb28815e6b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-09T13:35:30.661 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:30.653+0000 7fb2877fe640 1 --2- 172.21.15.43:0/4198815373 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fb28815e2d0 0x7fb28815e6b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:35:30.661 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:30.653+0000 7fb2877fe640 1 -- 172.21.15.43:0/4198815373 >> v1:172.21.15.203:6789/0 conn(0x7fb28815a720 legacy=0x7fb2880b0e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:35:30.662 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:30.653+0000 7fb2877fe640 1 -- 172.21.15.43:0/4198815373 --> v2:172.21.15.43:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb28815b2f0 con 0x7fb2880b1530 2023-08-09T13:35:30.662 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:30.653+0000 7fb286ffd640 1 --2- 172.21.15.43:0/4198815373 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fb28815e2d0 0x7fb28815e6b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-08-09T13:35:30.662 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:30.654+0000 7fb2877fe640 1 --2- 172.21.15.43:0/4198815373 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb2880b1530 0x7fb288069fc0 secure :-1 s=READY pgs=369 cs=0 l=1 rev1=1 crypto rx=0x7fb280009e50 tx=0x7fb28002bf60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=91c7e1426c0e85cd server_cookie=0 in_seq=0 out_seq=0 2023-08-09T13:35:30.662 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:30.655+0000 7fb285ffb640 1 -- 172.21.15.43:0/4198815373 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb28002cb00 con 0x7fb2880b1530 2023-08-09T13:35:30.662 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:30.655+0000 7fb285ffb640 1 -- 172.21.15.43:0/4198815373 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb28002cc60 con 0x7fb2880b1530 2023-08-09T13:35:30.662 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:30.655+0000 7fb285ffb640 1 -- 172.21.15.43:0/4198815373 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb2800343f0 con 0x7fb2880b1530 2023-08-09T13:35:30.662 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:30.655+0000 7fb287fff640 1 -- 172.21.15.43:0/4198815373 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb2880b1530 msgr2=0x7fb288069fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:35:30.662 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:30.655+0000 7fb287fff640 1 --2- 172.21.15.43:0/4198815373 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb2880b1530 0x7fb288069fc0 secure :-1 s=READY pgs=369 cs=0 l=1 rev1=1 crypto rx=0x7fb280009e50 tx=0x7fb28002bf60 comp rx=0 tx=0).stop 2023-08-09T13:35:30.663 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:30.655+0000 7fb287fff640 1 -- 172.21.15.43:0/4198815373 shutdown_connections 2023-08-09T13:35:30.663 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:30.655+0000 7fb287fff640 1 --2- 172.21.15.43:0/4198815373 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb2880b1530 0x7fb288069fc0 unknown :-1 s=CLOSED pgs=369 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:35:30.663 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:30.655+0000 7fb287fff640 1 --2- 172.21.15.43:0/4198815373 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fb28815e2d0 0x7fb28815e6b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:35:30.663 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:30.655+0000 7fb287fff640 1 -- 172.21.15.43:0/4198815373 >> 172.21.15.43:0/4198815373 conn(0x7fb2880b4400 msgr2=0x7fb2880b47e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-09T13:35:30.663 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:30.655+0000 7fb287fff640 1 -- 172.21.15.43:0/4198815373 shutdown_connections 2023-08-09T13:35:30.663 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:30.655+0000 7fb287fff640 1 -- 172.21.15.43:0/4198815373 wait complete. 2023-08-09T13:35:30.663 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:30.657+0000 7fb287fff640 1 Processor -- start 2023-08-09T13:35:30.664 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:30.657+0000 7fb287fff640 1 -- start start 2023-08-09T13:35:30.665 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:30.658+0000 7fb287fff640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb2880b1530 0x7fb2880ade00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:35:30.665 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:30.658+0000 7fb287fff640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fb28815a720 0x7fb2880ce3b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:35:30.666 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:30.658+0000 7fb287fff640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fb28815e2d0 0x7fb28813b130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:35:30.666 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:30.658+0000 7fb287fff640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb2880ae340 con 0x7fb2880b1530 2023-08-09T13:35:30.666 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:30.658+0000 7fb287fff640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb288137e00 con 0x7fb28815a720 2023-08-09T13:35:30.666 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:30.658+0000 7fb287fff640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb288137280 con 0x7fb28815e2d0 2023-08-09T13:35:30.666 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:30.659+0000 7fb286ffd640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb2880b1530 0x7fb2880ade00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:35:30.666 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:30.659+0000 7fb286ffd640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb2880b1530 0x7fb2880ade00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:47074/0 (socket says 172.21.15.43:47074) 2023-08-09T13:35:30.666 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:30.659+0000 7fb286ffd640 1 -- 172.21.15.43:0/3572206900 learned_addr learned my addr 172.21.15.43:0/3572206900 (peer_addr_for_me v2:172.21.15.43:0/0) 2023-08-09T13:35:30.666 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:30.659+0000 7fb2877fe640 1 --2- 172.21.15.43:0/3572206900 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fb28815e2d0 0x7fb28813b130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:35:30.667 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:30.659+0000 7fb2867fc640 1 --2- 172.21.15.43:0/3572206900 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fb28815a720 0x7fb2880ce3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:35:30.667 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:30.659+0000 7fb286ffd640 1 -- 172.21.15.43:0/3572206900 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fb28815e2d0 msgr2=0x7fb28813b130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:35:30.667 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:30.659+0000 7fb286ffd640 1 --2- 172.21.15.43:0/3572206900 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fb28815e2d0 0x7fb28813b130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:35:30.667 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:30.659+0000 7fb286ffd640 1 -- 172.21.15.43:0/3572206900 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fb28815a720 msgr2=0x7fb2880ce3b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:35:30.667 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:30.659+0000 7fb286ffd640 1 --2- 172.21.15.43:0/3572206900 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fb28815a720 0x7fb2880ce3b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:35:30.667 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:30.659+0000 7fb286ffd640 1 -- 172.21.15.43:0/3572206900 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb280009b30 con 0x7fb2880b1530 2023-08-09T13:35:30.667 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:30.659+0000 7fb2877fe640 1 --2- 172.21.15.43:0/3572206900 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fb28815e2d0 0x7fb28813b130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-08-09T13:35:30.667 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:30.659+0000 7fb286ffd640 1 --2- 172.21.15.43:0/3572206900 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb2880b1530 0x7fb2880ade00 secure :-1 s=READY pgs=370 cs=0 l=1 rev1=1 crypto rx=0x7fb27800e730 tx=0x7fb27800ebf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-09T13:35:30.668 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:30.660+0000 7fb26f7fe640 1 -- 172.21.15.43:0/3572206900 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb278004230 con 0x7fb2880b1530 2023-08-09T13:35:30.668 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:30.660+0000 7fb26f7fe640 1 -- 172.21.15.43:0/3572206900 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb278004390 con 0x7fb2880b1530 2023-08-09T13:35:30.668 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:30.660+0000 7fb26f7fe640 1 -- 172.21.15.43:0/3572206900 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb27800c8f0 con 0x7fb2880b1530 2023-08-09T13:35:30.668 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:30.660+0000 7fb287fff640 1 -- 172.21.15.43:0/3572206900 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb288137560 con 0x7fb2880b1530 2023-08-09T13:35:30.668 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:30.660+0000 7fb287fff640 1 -- 172.21.15.43:0/3572206900 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb288137b20 con 0x7fb2880b1530 2023-08-09T13:35:30.668 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:30.661+0000 7fb287fff640 1 -- 172.21.15.43:0/3572206900 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb28804e390 con 0x7fb2880b1530 2023-08-09T13:35:30.668 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:30.662+0000 7fb26f7fe640 1 -- 172.21.15.43:0/3572206900 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 5) v1 ==== 88341+0+0 (secure 0 0 0) 0x7fb27800ca50 con 0x7fb2880b1530 2023-08-09T13:35:30.669 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:30.664+0000 7fb26f7fe640 1 --2- 172.21.15.43:0/3572206900 >> [v2:172.21.15.203:6832/830097474,v1:172.21.15.203:6833/830097474] conn(0x7fb27406a050 0x7fb27406c510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:35:30.669 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:30.664+0000 7fb26f7fe640 1 -- 172.21.15.43:0/3572206900 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 6871+0+0 (secure 0 0 0) 0x7fb278088580 con 0x7fb2880b1530 2023-08-09T13:35:30.669 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:30.665+0000 7fb2867fc640 1 --2- 172.21.15.43:0/3572206900 >> [v2:172.21.15.203:6832/830097474,v1:172.21.15.203:6833/830097474] conn(0x7fb27406a050 0x7fb27406c510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:35:30.670 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:30.666+0000 7fb2867fc640 1 --2- 172.21.15.43:0/3572206900 >> [v2:172.21.15.203:6832/830097474,v1:172.21.15.203:6833/830097474] conn(0x7fb27406a050 0x7fb27406c510 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7fb2880cff50 tx=0x7fb27c009040 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-09T13:35:30.671 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:30.669+0000 7fb26f7fe640 1 -- 172.21.15.43:0/3572206900 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178066 (secure 0 0 0) 0x7fb2780538a0 con 0x7fb2880b1530 2023-08-09T13:35:30.867 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:30.864+0000 7fb287fff640 1 -- 172.21.15.43:0/3572206900 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7fb2880b0690 con 0x7fb2880b1530 2023-08-09T13:35:30.868 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:30.865+0000 7fb26f7fe640 1 -- 172.21.15.43:0/3572206900 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v26) v1 ==== 93+0+10 (secure 0 0 0) 0x7fb278056ec0 con 0x7fb2880b1530 2023-08-09T13:35:30.868 INFO:teuthology.orchestra.run.smithi043.stdout:pg_num: 1 2023-08-09T13:35:30.870 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:30.867+0000 7fb287fff640 1 -- 172.21.15.43:0/3572206900 >> [v2:172.21.15.203:6832/830097474,v1:172.21.15.203:6833/830097474] conn(0x7fb27406a050 msgr2=0x7fb27406c510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:35:30.870 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:30.867+0000 7fb287fff640 1 --2- 172.21.15.43:0/3572206900 >> [v2:172.21.15.203:6832/830097474,v1:172.21.15.203:6833/830097474] conn(0x7fb27406a050 0x7fb27406c510 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7fb2880cff50 tx=0x7fb27c009040 comp rx=0 tx=0).stop 2023-08-09T13:35:30.870 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:30.868+0000 7fb287fff640 1 -- 172.21.15.43:0/3572206900 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb2880b1530 msgr2=0x7fb2880ade00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:35:30.870 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:30.868+0000 7fb287fff640 1 --2- 172.21.15.43:0/3572206900 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb2880b1530 0x7fb2880ade00 secure :-1 s=READY pgs=370 cs=0 l=1 rev1=1 crypto rx=0x7fb27800e730 tx=0x7fb27800ebf0 comp rx=0 tx=0).stop 2023-08-09T13:35:30.871 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:30.868+0000 7fb287fff640 1 -- 172.21.15.43:0/3572206900 shutdown_connections 2023-08-09T13:35:30.871 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:30.868+0000 7fb287fff640 1 --2- 172.21.15.43:0/3572206900 >> [v2:172.21.15.203:6832/830097474,v1:172.21.15.203:6833/830097474] conn(0x7fb27406a050 0x7fb27406c510 unknown :-1 s=CLOSED pgs=194 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:35:30.871 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:30.868+0000 7fb287fff640 1 --2- 172.21.15.43:0/3572206900 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fb28815e2d0 0x7fb28813b130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:35:30.871 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:30.868+0000 7fb287fff640 1 --2- 172.21.15.43:0/3572206900 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fb28815a720 0x7fb2880ce3b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:35:30.872 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:30.868+0000 7fb287fff640 1 --2- 172.21.15.43:0/3572206900 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb2880b1530 0x7fb2880ade00 unknown :-1 s=CLOSED pgs=370 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:35:30.872 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:30.868+0000 7fb287fff640 1 -- 172.21.15.43:0/3572206900 >> 172.21.15.43:0/3572206900 conn(0x7fb2880b4400 msgr2=0x7fb288150e90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-09T13:35:30.872 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:30.868+0000 7fb287fff640 1 -- 172.21.15.43:0/3572206900 shutdown_connections 2023-08-09T13:35:30.872 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:30.868+0000 7fb287fff640 1 -- 172.21.15.43:0/3572206900 wait complete. 2023-08-09T13:35:30.890 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-08-09T13:35:31.124 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:31.121+0000 7effcad54640 1 Processor -- start 2023-08-09T13:35:31.124 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:31.121+0000 7effcad54640 1 -- start start 2023-08-09T13:35:31.126 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:31.123+0000 7effcad54640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7effc40cdc60 0x7effc4167500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:35:31.126 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:31.123+0000 7effcad54640 1 -- --> v1:172.21.15.43:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7effc40cca90 con 0x7effc40ccf20 2023-08-09T13:35:31.126 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:31.123+0000 7effcad54640 1 -- --> v1:172.21.15.203:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7effc415da20 con 0x7effc415c630 2023-08-09T13:35:31.126 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:31.123+0000 7effcad54640 1 -- --> v2:172.21.15.43:3300/0 -- mon_getmap magic: 0 v1 -- 0x7effc40ac600 con 0x7effc40cdc60 2023-08-09T13:35:31.126 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:31.123+0000 7effc3fff640 1 --1- >> v1:172.21.15.43:6789/0 conn(0x7effc40ccf20 0x7effc40cd550 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.43:6789/0 says I am v1:172.21.15.43:52220/0 (socket says 172.21.15.43:52220) 2023-08-09T13:35:31.126 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:31.123+0000 7effc3fff640 1 -- 172.21.15.43:0/2366606179 learned_addr learned my addr 172.21.15.43:0/2366606179 (peer_addr_for_me v1:172.21.15.43:0/0) 2023-08-09T13:35:31.127 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:31.123+0000 7effc92f1640 1 --2- 172.21.15.43:0/2366606179 >> v2:172.21.15.43:3300/0 conn(0x7effc40cdc60 0x7effc4167500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:35:31.127 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:31.124+0000 7effc92f1640 1 -- 172.21.15.43:0/2366606179 >> v1:172.21.15.203:6789/0 conn(0x7effc415c630 legacy=0x7effc40abef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:35:31.127 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:31.124+0000 7effc92f1640 1 -- 172.21.15.43:0/2366606179 >> v1:172.21.15.43:6789/0 conn(0x7effc40ccf20 legacy=0x7effc40cd550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:35:31.127 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:31.124+0000 7effc92f1640 1 -- 172.21.15.43:0/2366606179 --> v2:172.21.15.43:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7effc40cbe80 con 0x7effc40cdc60 2023-08-09T13:35:31.128 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:31.125+0000 7effc92f1640 1 --2- 172.21.15.43:0/2366606179 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7effc40cdc60 0x7effc4167500 secure :-1 s=READY pgs=372 cs=0 l=1 rev1=1 crypto rx=0x7effbc009d90 tx=0x7effbc02bf50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=13d7edc3bb165caa server_cookie=0 in_seq=0 out_seq=0 2023-08-09T13:35:31.128 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:31.126+0000 7effc37fe640 1 -- 172.21.15.43:0/2366606179 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7effbc02cd00 con 0x7effc40cdc60 2023-08-09T13:35:31.129 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:31.126+0000 7effc37fe640 1 -- 172.21.15.43:0/2366606179 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7effbc02ce60 con 0x7effc40cdc60 2023-08-09T13:35:31.129 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:31.126+0000 7effc37fe640 1 -- 172.21.15.43:0/2366606179 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7effbc034660 con 0x7effc40cdc60 2023-08-09T13:35:31.129 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:31.126+0000 7effcad54640 1 -- 172.21.15.43:0/2366606179 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7effc40cdc60 msgr2=0x7effc4167500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:35:31.130 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:31.126+0000 7effcad54640 1 --2- 172.21.15.43:0/2366606179 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7effc40cdc60 0x7effc4167500 secure :-1 s=READY pgs=372 cs=0 l=1 rev1=1 crypto rx=0x7effbc009d90 tx=0x7effbc02bf50 comp rx=0 tx=0).stop 2023-08-09T13:35:31.130 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:31.127+0000 7effcad54640 1 -- 172.21.15.43:0/2366606179 shutdown_connections 2023-08-09T13:35:31.130 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:31.127+0000 7effcad54640 1 --2- 172.21.15.43:0/2366606179 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7effc40cdc60 0x7effc4167500 unknown :-1 s=CLOSED pgs=372 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:35:31.130 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:31.127+0000 7effcad54640 1 -- 172.21.15.43:0/2366606179 >> 172.21.15.43:0/2366606179 conn(0x7effc40ce0a0 msgr2=0x7effc40ce480 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-09T13:35:31.130 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:31.127+0000 7effcad54640 1 -- 172.21.15.43:0/2366606179 shutdown_connections 2023-08-09T13:35:31.130 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:31.127+0000 7effcad54640 1 -- 172.21.15.43:0/2366606179 wait complete. 2023-08-09T13:35:31.131 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:31.128+0000 7effcad54640 1 Processor -- start 2023-08-09T13:35:31.131 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:31.129+0000 7effcad54640 1 -- start start 2023-08-09T13:35:31.132 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:31.130+0000 7effcad54640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7effc40ccf20 0x7effc41f2a90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:35:31.132 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:31.130+0000 7effcad54640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7effc40cdc60 0x7effc40ae1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:35:31.133 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:31.130+0000 7effcad54640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7effc415c630 0x7effc415e5d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:35:31.133 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:31.130+0000 7effcad54640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7effc40aeaf0 con 0x7effc415c630 2023-08-09T13:35:31.133 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:31.130+0000 7effcad54640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7effc40aedd0 con 0x7effc40ccf20 2023-08-09T13:35:31.133 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:31.130+0000 7effcad54640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7effc40af0b0 con 0x7effc40cdc60 2023-08-09T13:35:31.133 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:31.130+0000 7effc92f1640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7effc415c630 0x7effc415e5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:35:31.133 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:31.130+0000 7effc92f1640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7effc415c630 0x7effc415e5d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:47092/0 (socket says 172.21.15.43:47092) 2023-08-09T13:35:31.133 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:31.130+0000 7effc92f1640 1 -- 172.21.15.43:0/2043202481 learned_addr learned my addr 172.21.15.43:0/2043202481 (peer_addr_for_me v2:172.21.15.43:0/0) 2023-08-09T13:35:31.134 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:31.130+0000 7effc3fff640 1 --2- 172.21.15.43:0/2043202481 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7effc40cdc60 0x7effc40ae1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:35:31.135 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:31.130+0000 7effc8af0640 1 --2- 172.21.15.43:0/2043202481 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7effc40ccf20 0x7effc41f2a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:35:31.135 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:31.131+0000 7effc92f1640 1 -- 172.21.15.43:0/2043202481 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7effc40cdc60 msgr2=0x7effc40ae1a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:35:31.135 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:31.131+0000 7effc92f1640 1 --2- 172.21.15.43:0/2043202481 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7effc40cdc60 0x7effc40ae1a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:35:31.135 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:31.131+0000 7effc92f1640 1 -- 172.21.15.43:0/2043202481 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7effc40ccf20 msgr2=0x7effc41f2a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:35:31.135 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:31.131+0000 7effc92f1640 1 --2- 172.21.15.43:0/2043202481 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7effc40ccf20 0x7effc41f2a90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:35:31.135 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:31.131+0000 7effc92f1640 1 -- 172.21.15.43:0/2043202481 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7effbc009a70 con 0x7effc415c630 2023-08-09T13:35:31.136 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:31.131+0000 7effc92f1640 1 --2- 172.21.15.43:0/2043202481 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7effc415c630 0x7effc415e5d0 secure :-1 s=READY pgs=373 cs=0 l=1 rev1=1 crypto rx=0x7effbc009d90 tx=0x7effbc004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-09T13:35:31.136 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:31.131+0000 7effc17fa640 1 -- 172.21.15.43:0/2043202481 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7effbc03d600 con 0x7effc415c630 2023-08-09T13:35:31.136 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:31.131+0000 7effc17fa640 1 -- 172.21.15.43:0/2043202481 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7effbc03d780 con 0x7effc415c630 2023-08-09T13:35:31.136 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:31.131+0000 7effc17fa640 1 -- 172.21.15.43:0/2043202481 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7effbc03db30 con 0x7effc415c630 2023-08-09T13:35:31.136 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:31.131+0000 7effcad54640 1 -- 172.21.15.43:0/2043202481 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7effc40af390 con 0x7effc415c630 2023-08-09T13:35:31.136 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:31.132+0000 7effcad54640 1 -- 172.21.15.43:0/2043202481 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7effc40ae6e0 con 0x7effc415c630 2023-08-09T13:35:31.137 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:31.133+0000 7effcad54640 1 -- 172.21.15.43:0/2043202481 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7effc40739f0 con 0x7effc415c630 2023-08-09T13:35:31.137 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:31.134+0000 7effc17fa640 1 -- 172.21.15.43:0/2043202481 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 5) v1 ==== 88341+0+0 (secure 0 0 0) 0x7effbc03b040 con 0x7effc415c630 2023-08-09T13:35:31.137 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:31.135+0000 7effc17fa640 1 --2- 172.21.15.43:0/2043202481 >> [v2:172.21.15.203:6832/830097474,v1:172.21.15.203:6833/830097474] conn(0x7effa806a050 0x7effa806c510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:35:31.138 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:31.135+0000 7effc17fa640 1 -- 172.21.15.43:0/2043202481 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 6871+0+0 (secure 0 0 0) 0x7effbc0aafd0 con 0x7effc415c630 2023-08-09T13:35:31.138 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:31.135+0000 7effc8af0640 1 --2- 172.21.15.43:0/2043202481 >> [v2:172.21.15.203:6832/830097474,v1:172.21.15.203:6833/830097474] conn(0x7effa806a050 0x7effa806c510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:35:31.141 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:31.137+0000 7effc8af0640 1 --2- 172.21.15.43:0/2043202481 >> [v2:172.21.15.203:6832/830097474,v1:172.21.15.203:6833/830097474] conn(0x7effa806a050 0x7effa806c510 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7effb8000f60 tx=0x7effb8009040 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-09T13:35:31.142 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:31.139+0000 7effc17fa640 1 -- 172.21.15.43:0/2043202481 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178066 (secure 0 0 0) 0x7effbc0763b0 con 0x7effc415c630 2023-08-09T13:35:31.347 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:31.344+0000 7effcad54640 1 -- 172.21.15.43:0/2043202481 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7effc40c7f80 con 0x7effc415c630 2023-08-09T13:35:31.347 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:31.345+0000 7effc17fa640 1 -- 172.21.15.43:0/2043202481 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v26) v1 ==== 104+0+11 (secure 0 0 0) 0x7effbc04e020 con 0x7effc415c630 2023-08-09T13:35:31.348 INFO:teuthology.orchestra.run.smithi043.stdout:pg_num: 32 2023-08-09T13:35:31.351 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:31.348+0000 7effcad54640 1 -- 172.21.15.43:0/2043202481 >> [v2:172.21.15.203:6832/830097474,v1:172.21.15.203:6833/830097474] conn(0x7effa806a050 msgr2=0x7effa806c510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:35:31.351 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:31.348+0000 7effcad54640 1 --2- 172.21.15.43:0/2043202481 >> [v2:172.21.15.203:6832/830097474,v1:172.21.15.203:6833/830097474] conn(0x7effa806a050 0x7effa806c510 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7effb8000f60 tx=0x7effb8009040 comp rx=0 tx=0).stop 2023-08-09T13:35:31.351 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:31.348+0000 7effcad54640 1 -- 172.21.15.43:0/2043202481 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7effc415c630 msgr2=0x7effc415e5d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:35:31.351 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:31.348+0000 7effcad54640 1 --2- 172.21.15.43:0/2043202481 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7effc415c630 0x7effc415e5d0 secure :-1 s=READY pgs=373 cs=0 l=1 rev1=1 crypto rx=0x7effbc009d90 tx=0x7effbc004a40 comp rx=0 tx=0).stop 2023-08-09T13:35:31.351 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:31.348+0000 7effcad54640 1 -- 172.21.15.43:0/2043202481 shutdown_connections 2023-08-09T13:35:31.351 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:31.348+0000 7effcad54640 1 --2- 172.21.15.43:0/2043202481 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7effc415c630 0x7effc415e5d0 unknown :-1 s=CLOSED pgs=373 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:35:31.351 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:31.348+0000 7effcad54640 1 --2- 172.21.15.43:0/2043202481 >> [v2:172.21.15.203:6832/830097474,v1:172.21.15.203:6833/830097474] conn(0x7effa806a050 0x7effa806c510 unknown :-1 s=CLOSED pgs=195 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:35:31.352 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:31.348+0000 7effcad54640 1 --2- 172.21.15.43:0/2043202481 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7effc40cdc60 0x7effc40ae1a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:35:31.352 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:31.348+0000 7effcad54640 1 --2- 172.21.15.43:0/2043202481 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7effc40ccf20 0x7effc41f2a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:35:31.352 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:31.348+0000 7effcad54640 1 -- 172.21.15.43:0/2043202481 >> 172.21.15.43:0/2043202481 conn(0x7effc40ce0a0 msgr2=0x7effc406b820 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-09T13:35:31.352 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:31.348+0000 7effcad54640 1 -- 172.21.15.43:0/2043202481 shutdown_connections 2023-08-09T13:35:31.352 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:31.348+0000 7effcad54640 1 -- 172.21.15.43:0/2043202481 wait complete. 2023-08-09T13:35:31.370 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-08-09T13:35:31.590 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:31.588+0000 7fca4b59e640 1 Processor -- start 2023-08-09T13:35:31.591 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:31.588+0000 7fca4b59e640 1 -- start start 2023-08-09T13:35:31.592 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:31.590+0000 7fca4b59e640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7fca4c0b21e0 0x7fca4c15d410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:35:31.593 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:31.590+0000 7fca4b59e640 1 -- --> v1:172.21.15.43:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fca4c0cdc20 con 0x7fca4c15c2f0 2023-08-09T13:35:31.593 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:31.590+0000 7fca4b59e640 1 -- --> v1:172.21.15.203:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fca4c06c570 con 0x7fca4c0cfa70 2023-08-09T13:35:31.593 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:31.590+0000 7fca4b59e640 1 -- --> v2:172.21.15.43:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fca4c15d950 con 0x7fca4c0b21e0 2023-08-09T13:35:31.593 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:31.590+0000 7fca4a59c640 1 --1- >> v1:172.21.15.43:6789/0 conn(0x7fca4c15c2f0 0x7fca4c0b1ad0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.43:6789/0 says I am v1:172.21.15.43:52232/0 (socket says 172.21.15.43:52232) 2023-08-09T13:35:31.593 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:31.590+0000 7fca4a59c640 1 -- 172.21.15.43:0/2193682198 learned_addr learned my addr 172.21.15.43:0/2193682198 (peer_addr_for_me v1:172.21.15.43:0/0) 2023-08-09T13:35:31.593 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:31.590+0000 7fca41d9b640 1 --2- 172.21.15.43:0/2193682198 >> v2:172.21.15.43:3300/0 conn(0x7fca4c0b21e0 0x7fca4c15d410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:35:31.594 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:31.591+0000 7fca41d9b640 1 -- 172.21.15.43:0/2193682198 >> v1:172.21.15.203:6789/0 conn(0x7fca4c0cfa70 legacy=0x7fca4c0cb7b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:35:31.594 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:31.591+0000 7fca41d9b640 1 -- 172.21.15.43:0/2193682198 >> v1:172.21.15.43:6789/0 conn(0x7fca4c15c2f0 legacy=0x7fca4c0b1ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:35:31.594 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:31.591+0000 7fca41d9b640 1 -- 172.21.15.43:0/2193682198 --> v2:172.21.15.43:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fca4c0cde70 con 0x7fca4c0b21e0 2023-08-09T13:35:31.595 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:31.592+0000 7fca41d9b640 1 --2- 172.21.15.43:0/2193682198 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fca4c0b21e0 0x7fca4c15d410 secure :-1 s=READY pgs=375 cs=0 l=1 rev1=1 crypto rx=0x7fca340097a0 tx=0x7fca3402b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8a7cad17549f273e server_cookie=0 in_seq=0 out_seq=0 2023-08-09T13:35:31.595 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:31.593+0000 7fca49d9b640 1 -- 172.21.15.43:0/2193682198 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fca34031830 con 0x7fca4c0b21e0 2023-08-09T13:35:31.596 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:31.593+0000 7fca49d9b640 1 -- 172.21.15.43:0/2193682198 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fca34031990 con 0x7fca4c0b21e0 2023-08-09T13:35:31.596 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:31.593+0000 7fca49d9b640 1 -- 172.21.15.43:0/2193682198 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fca34031d20 con 0x7fca4c0b21e0 2023-08-09T13:35:31.597 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:31.594+0000 7fca4b59e640 1 -- 172.21.15.43:0/2193682198 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fca4c0b21e0 msgr2=0x7fca4c15d410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:35:31.597 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:31.594+0000 7fca4b59e640 1 --2- 172.21.15.43:0/2193682198 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fca4c0b21e0 0x7fca4c15d410 secure :-1 s=READY pgs=375 cs=0 l=1 rev1=1 crypto rx=0x7fca340097a0 tx=0x7fca3402b8d0 comp rx=0 tx=0).stop 2023-08-09T13:35:31.597 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:31.594+0000 7fca4b59e640 1 -- 172.21.15.43:0/2193682198 shutdown_connections 2023-08-09T13:35:31.597 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:31.594+0000 7fca4b59e640 1 --2- 172.21.15.43:0/2193682198 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fca4c0b21e0 0x7fca4c15d410 unknown :-1 s=CLOSED pgs=375 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:35:31.597 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:31.594+0000 7fca4b59e640 1 -- 172.21.15.43:0/2193682198 >> 172.21.15.43:0/2193682198 conn(0x7fca4c0ccad0 msgr2=0x7fca4c0cceb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-09T13:35:31.597 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:31.594+0000 7fca4b59e640 1 -- 172.21.15.43:0/2193682198 shutdown_connections 2023-08-09T13:35:31.598 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:31.594+0000 7fca4b59e640 1 -- 172.21.15.43:0/2193682198 wait complete. 2023-08-09T13:35:31.598 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:31.595+0000 7fca4b59e640 1 Processor -- start 2023-08-09T13:35:31.598 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:31.596+0000 7fca4b59e640 1 -- start start 2023-08-09T13:35:31.599 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:31.597+0000 7fca4b59e640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fca4c0b21e0 0x7fca4c150bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:35:31.599 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:31.597+0000 7fca4b59e640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fca4c0cfa70 0x7fca4c0c36e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:35:31.600 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:31.597+0000 7fca4b59e640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fca4c15c2f0 0x7fca4c153d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:35:31.600 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:31.597+0000 7fca4b59e640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fca4c0c4cd0 con 0x7fca4c15c2f0 2023-08-09T13:35:31.600 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:31.597+0000 7fca4b59e640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fca4c151110 con 0x7fca4c0b21e0 2023-08-09T13:35:31.600 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:31.597+0000 7fca4b59e640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fca4c0c40c0 con 0x7fca4c0cfa70 2023-08-09T13:35:31.600 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:31.597+0000 7fca4ad9d640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fca4c15c2f0 0x7fca4c153d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:35:31.600 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:31.597+0000 7fca4ad9d640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fca4c15c2f0 0x7fca4c153d50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:47108/0 (socket says 172.21.15.43:47108) 2023-08-09T13:35:31.600 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:31.597+0000 7fca4ad9d640 1 -- 172.21.15.43:0/1997103718 learned_addr learned my addr 172.21.15.43:0/1997103718 (peer_addr_for_me v2:172.21.15.43:0/0) 2023-08-09T13:35:31.601 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:31.597+0000 7fca4a59c640 1 --2- 172.21.15.43:0/1997103718 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fca4c0b21e0 0x7fca4c150bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:35:31.602 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:31.597+0000 7fca4ad9d640 1 -- 172.21.15.43:0/1997103718 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fca4c0cfa70 msgr2=0x7fca4c0c36e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:35:31.602 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:31.597+0000 7fca4ad9d640 1 --2- 172.21.15.43:0/1997103718 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fca4c0cfa70 0x7fca4c0c36e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:35:31.602 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:31.597+0000 7fca4ad9d640 1 -- 172.21.15.43:0/1997103718 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fca4c0b21e0 msgr2=0x7fca4c150bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:35:31.602 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:31.597+0000 7fca4ad9d640 1 --2- 172.21.15.43:0/1997103718 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fca4c0b21e0 0x7fca4c150bd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:35:31.602 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:31.597+0000 7fca4ad9d640 1 -- 172.21.15.43:0/1997103718 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fca34009400 con 0x7fca4c15c2f0 2023-08-09T13:35:31.602 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:31.598+0000 7fca4ad9d640 1 --2- 172.21.15.43:0/1997103718 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fca4c15c2f0 0x7fca4c153d50 secure :-1 s=READY pgs=376 cs=0 l=1 rev1=1 crypto rx=0x7fca4400c7d0 tx=0x7fca4400cc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-09T13:35:31.602 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:31.598+0000 7fca437fe640 1 -- 172.21.15.43:0/1997103718 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fca44003b70 con 0x7fca4c15c2f0 2023-08-09T13:35:31.603 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:31.598+0000 7fca437fe640 1 -- 172.21.15.43:0/1997103718 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fca44003cf0 con 0x7fca4c15c2f0 2023-08-09T13:35:31.603 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:31.598+0000 7fca437fe640 1 -- 172.21.15.43:0/1997103718 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fca4400aa90 con 0x7fca4c15c2f0 2023-08-09T13:35:31.603 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:31.598+0000 7fca4b59e640 1 -- 172.21.15.43:0/1997103718 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fca4c0c43a0 con 0x7fca4c15c2f0 2023-08-09T13:35:31.603 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:31.598+0000 7fca4b59e640 1 -- 172.21.15.43:0/1997103718 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fca4c0c4960 con 0x7fca4c15c2f0 2023-08-09T13:35:31.603 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:31.599+0000 7fca4b59e640 1 -- 172.21.15.43:0/1997103718 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fca4c0c4e80 con 0x7fca4c15c2f0 2023-08-09T13:35:31.604 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:31.601+0000 7fca437fe640 1 -- 172.21.15.43:0/1997103718 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 5) v1 ==== 88341+0+0 (secure 0 0 0) 0x7fca4400abf0 con 0x7fca4c15c2f0 2023-08-09T13:35:31.604 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:31.602+0000 7fca437fe640 1 --2- 172.21.15.43:0/1997103718 >> [v2:172.21.15.203:6832/830097474,v1:172.21.15.203:6833/830097474] conn(0x7fca2c06a050 0x7fca2c06c510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:35:31.605 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:31.602+0000 7fca437fe640 1 -- 172.21.15.43:0/1997103718 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 6871+0+0 (secure 0 0 0) 0x7fca44088730 con 0x7fca4c15c2f0 2023-08-09T13:35:31.605 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:31.602+0000 7fca4a59c640 1 --2- 172.21.15.43:0/1997103718 >> [v2:172.21.15.203:6832/830097474,v1:172.21.15.203:6833/830097474] conn(0x7fca2c06a050 0x7fca2c06c510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:35:31.606 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:31.604+0000 7fca4a59c640 1 --2- 172.21.15.43:0/1997103718 >> [v2:172.21.15.203:6832/830097474,v1:172.21.15.203:6833/830097474] conn(0x7fca2c06a050 0x7fca2c06c510 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7fca3c005880 tx=0x7fca3c009040 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-09T13:35:31.608 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:31.606+0000 7fca437fe640 1 -- 172.21.15.43:0/1997103718 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178066 (secure 0 0 0) 0x7fca4c0c4e80 con 0x7fca4c15c2f0 2023-08-09T13:35:31.803 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:31.801+0000 7fca4b59e640 1 -- 172.21.15.43:0/1997103718 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7fca4c0c4e80 con 0x7fca4c15c2f0 2023-08-09T13:35:31.804 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:31.801+0000 7fca437fe640 1 -- 172.21.15.43:0/1997103718 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v26) v1 ==== 100+0+11 (secure 0 0 0) 0x7fca4c0c4e80 con 0x7fca4c15c2f0 2023-08-09T13:35:31.804 INFO:teuthology.orchestra.run.smithi043.stdout:pg_num: 64 2023-08-09T13:35:31.807 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:31.804+0000 7fca4b59e640 1 -- 172.21.15.43:0/1997103718 >> [v2:172.21.15.203:6832/830097474,v1:172.21.15.203:6833/830097474] conn(0x7fca2c06a050 msgr2=0x7fca2c06c510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:35:31.807 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:31.804+0000 7fca4b59e640 1 --2- 172.21.15.43:0/1997103718 >> [v2:172.21.15.203:6832/830097474,v1:172.21.15.203:6833/830097474] conn(0x7fca2c06a050 0x7fca2c06c510 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7fca3c005880 tx=0x7fca3c009040 comp rx=0 tx=0).stop 2023-08-09T13:35:31.807 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:31.804+0000 7fca4b59e640 1 -- 172.21.15.43:0/1997103718 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fca4c15c2f0 msgr2=0x7fca4c153d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:35:31.808 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:31.804+0000 7fca4b59e640 1 --2- 172.21.15.43:0/1997103718 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fca4c15c2f0 0x7fca4c153d50 secure :-1 s=READY pgs=376 cs=0 l=1 rev1=1 crypto rx=0x7fca4400c7d0 tx=0x7fca4400cc90 comp rx=0 tx=0).stop 2023-08-09T13:35:31.808 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:31.805+0000 7fca4b59e640 1 -- 172.21.15.43:0/1997103718 shutdown_connections 2023-08-09T13:35:31.808 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:31.805+0000 7fca4b59e640 1 --2- 172.21.15.43:0/1997103718 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fca4c15c2f0 0x7fca4c153d50 unknown :-1 s=CLOSED pgs=376 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:35:31.809 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:31.805+0000 7fca4b59e640 1 --2- 172.21.15.43:0/1997103718 >> [v2:172.21.15.203:6832/830097474,v1:172.21.15.203:6833/830097474] conn(0x7fca2c06a050 0x7fca2c06c510 unknown :-1 s=CLOSED pgs=196 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:35:31.809 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:31.805+0000 7fca4b59e640 1 --2- 172.21.15.43:0/1997103718 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fca4c0cfa70 0x7fca4c0c36e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:35:31.809 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:31.805+0000 7fca4b59e640 1 --2- 172.21.15.43:0/1997103718 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fca4c0b21e0 0x7fca4c150bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:35:31.809 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:31.805+0000 7fca4b59e640 1 -- 172.21.15.43:0/1997103718 >> 172.21.15.43:0/1997103718 conn(0x7fca4c0ccad0 msgr2=0x7fca4c06b7e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-09T13:35:31.809 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:31.805+0000 7fca4b59e640 1 -- 172.21.15.43:0/1997103718 shutdown_connections 2023-08-09T13:35:31.809 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:31.805+0000 7fca4b59e640 1 -- 172.21.15.43:0/1997103718 wait complete. 2023-08-09T13:35:31.827 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2023-08-09T13:35:32.051 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:32.048+0000 7f12f3f99640 1 Processor -- start 2023-08-09T13:35:32.051 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:32.048+0000 7f12f3f99640 1 -- start start 2023-08-09T13:35:32.052 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:32.050+0000 7f12f3f99640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f12ec15c550 0x7f12ec0d1a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:35:32.052 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:32.050+0000 7f12f3f99640 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7f12ec0cf550 0x7f12ec0b5460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:35:32.053 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:32.050+0000 7f12f3f99640 1 -- --> v1:172.21.15.43:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f12ec0d0700 con 0x7f12ec15c930 2023-08-09T13:35:32.053 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:32.050+0000 7f12f3f99640 1 -- --> v2:172.21.15.203:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f12ec0d09e0 con 0x7f12ec0cf550 2023-08-09T13:35:32.053 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:32.050+0000 7f12f3f99640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f12ec0d0cc0 con 0x7f12ec15c550 2023-08-09T13:35:32.054 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:32.050+0000 7f12f1534640 1 --1- >> v1:172.21.15.43:6789/0 conn(0x7f12ec15c930 0x7f12ec0cee40 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.43:6789/0 says I am v1:172.21.15.43:52234/0 (socket says 172.21.15.43:52234) 2023-08-09T13:35:32.054 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:32.050+0000 7f12f1534640 1 -- 172.21.15.43:0/1878705426 learned_addr learned my addr 172.21.15.43:0/1878705426 (peer_addr_for_me v1:172.21.15.43:0/0) 2023-08-09T13:35:32.054 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:32.051+0000 7f12f1d35640 1 --2- 172.21.15.43:0/1878705426 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f12ec15c550 0x7f12ec0d1a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:35:32.055 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:32.051+0000 7f12f2536640 1 --2- 172.21.15.43:0/1878705426 >> v2:172.21.15.203:3300/0 conn(0x7f12ec0cf550 0x7f12ec0b5460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:35:32.055 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:32.051+0000 7f12f0d33640 1 -- 172.21.15.43:0/1878705426 <== mon.0 v1:172.21.15.43:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 150401977 0 0) 0x7f12ec0d0700 con 0x7f12ec15c930 2023-08-09T13:35:32.055 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:32.051+0000 7f12f0d33640 1 -- 172.21.15.43:0/1878705426 --> v1:172.21.15.43:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f12d8003520 con 0x7f12ec15c930 2023-08-09T13:35:32.055 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:32.051+0000 7f12f0d33640 1 -- 172.21.15.43:0/1878705426 <== mon.0 v1:172.21.15.43:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3996833239 0 0) 0x7f12d8003520 con 0x7f12ec15c930 2023-08-09T13:35:32.055 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:32.051+0000 7f12f0d33640 1 -- 172.21.15.43:0/1878705426 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f12ec15c550 msgr2=0x7f12ec0d1a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-09T13:35:32.055 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:32.051+0000 7f12f0d33640 1 --2- 172.21.15.43:0/1878705426 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f12ec15c550 0x7f12ec0d1a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:35:32.055 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:32.051+0000 7f12f0d33640 1 -- 172.21.15.43:0/1878705426 >> v2:172.21.15.203:3300/0 conn(0x7f12ec0cf550 msgr2=0x7f12ec0b5460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-09T13:35:32.056 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:32.051+0000 7f12f0d33640 1 --2- 172.21.15.43:0/1878705426 >> v2:172.21.15.203:3300/0 conn(0x7f12ec0cf550 0x7f12ec0b5460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:35:32.056 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:32.051+0000 7f12f0d33640 1 -- 172.21.15.43:0/1878705426 --> v1:172.21.15.43:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f12ec0d0fa0 con 0x7f12ec15c930 2023-08-09T13:35:32.056 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:32.051+0000 7f12f0d33640 1 -- 172.21.15.43:0/1878705426 <== mon.0 v1:172.21.15.43:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1285150740 0 0) 0x7f12dc001670 con 0x7f12ec15c930 2023-08-09T13:35:32.056 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:32.051+0000 7f12f0d33640 1 -- 172.21.15.43:0/1878705426 >> v1:172.21.15.43:6789/0 conn(0x7f12ec15c930 legacy=0x7f12ec0cee40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:35:32.056 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:32.051+0000 7f12f0d33640 1 --2- 172.21.15.43:0/1878705426 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f12d80039e0 0x7f12d8003e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:35:32.056 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:32.051+0000 7f12f0d33640 1 --2- 172.21.15.43:0/1878705426 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f12ec15c550 0x7f12d80047c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:35:32.056 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:32.051+0000 7f12f0d33640 1 --2- 172.21.15.43:0/1878705426 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f12d8004d00 0x7f12d8005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:35:32.056 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:32.052+0000 7f12f0d33640 1 -- 172.21.15.43:0/1878705426 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f12ec0d09e0 con 0x7f12d80039e0 2023-08-09T13:35:32.057 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:32.052+0000 7f12f0d33640 1 -- 172.21.15.43:0/1878705426 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f12ec0d0cc0 con 0x7f12d8004d00 2023-08-09T13:35:32.057 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:32.052+0000 7f12f0d33640 1 -- 172.21.15.43:0/1878705426 --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f12d800ba00 con 0x7f12ec15c550 2023-08-09T13:35:32.057 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:32.052+0000 7f12f1d35640 1 --2- 172.21.15.43:0/1878705426 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f12d80039e0 0x7f12d8003e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:35:32.058 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:32.052+0000 7f12f1534640 1 --2- 172.21.15.43:0/1878705426 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f12ec15c550 0x7f12d80047c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:35:32.058 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:32.052+0000 7f12f1d35640 1 -- 172.21.15.43:0/1878705426 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f12ec15c550 msgr2=0x7f12d80047c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-09T13:35:32.059 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:32.052+0000 7f12f1d35640 1 --2- 172.21.15.43:0/1878705426 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f12ec15c550 0x7f12d80047c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:35:32.059 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:32.052+0000 7f12f2536640 1 --2- 172.21.15.43:0/1878705426 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f12d8004d00 0x7f12d8005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:35:32.059 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:32.052+0000 7f12f1d35640 1 -- 172.21.15.43:0/1878705426 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f12d8004d00 msgr2=0x7f12d8005180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-09T13:35:32.059 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:32.052+0000 7f12f1d35640 1 --2- 172.21.15.43:0/1878705426 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f12d8004d00 0x7f12d8005180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:35:32.059 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:32.052+0000 7f12f1d35640 1 -- 172.21.15.43:0/1878705426 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f12d8003520 con 0x7f12d80039e0 2023-08-09T13:35:32.059 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:32.052+0000 7f12f1534640 1 --2- 172.21.15.43:0/1878705426 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f12ec15c550 0x7f12d80047c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-08-09T13:35:32.059 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:32.053+0000 7f12f1d35640 1 --2- 172.21.15.43:0/1878705426 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f12d80039e0 0x7f12d8003e00 secure :-1 s=READY pgs=378 cs=0 l=1 rev1=1 crypto rx=0x7f12e8009ac0 tx=0x7f12d80057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8308ef2f72fc8289 server_cookie=0 in_seq=0 out_seq=0 2023-08-09T13:35:32.059 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:32.054+0000 7f12f0d33640 1 -- 172.21.15.43:0/1878705426 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f12e802c6b0 con 0x7f12d80039e0 2023-08-09T13:35:32.060 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:32.054+0000 7f12f0d33640 1 -- 172.21.15.43:0/1878705426 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f12e802c810 con 0x7f12d80039e0 2023-08-09T13:35:32.060 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:32.054+0000 7f12f0d33640 1 -- 172.21.15.43:0/1878705426 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f12e802cba0 con 0x7f12d80039e0 2023-08-09T13:35:32.060 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:32.054+0000 7f12f3f99640 1 -- 172.21.15.43:0/1878705426 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f12d80039e0 msgr2=0x7f12d8003e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:35:32.060 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:32.054+0000 7f12f3f99640 1 --2- 172.21.15.43:0/1878705426 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f12d80039e0 0x7f12d8003e00 secure :-1 s=READY pgs=378 cs=0 l=1 rev1=1 crypto rx=0x7f12e8009ac0 tx=0x7f12d80057f0 comp rx=0 tx=0).stop 2023-08-09T13:35:32.060 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:32.054+0000 7f12f2536640 1 -- 172.21.15.43:0/1878705426 reap_dead start 2023-08-09T13:35:32.060 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:32.054+0000 7f12f3f99640 1 -- 172.21.15.43:0/1878705426 shutdown_connections 2023-08-09T13:35:32.060 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:32.054+0000 7f12f3f99640 1 -- 172.21.15.43:0/1878705426 >> 172.21.15.43:0/1878705426 conn(0x7f12ec06a4e0 msgr2=0x7f12ec06a8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-09T13:35:32.061 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:32.054+0000 7f12f3f99640 1 -- 172.21.15.43:0/1878705426 shutdown_connections 2023-08-09T13:35:32.061 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:32.054+0000 7f12f3f99640 1 -- 172.21.15.43:0/1878705426 wait complete. 2023-08-09T13:35:32.062 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:32.056+0000 7f12f3f99640 1 Processor -- start 2023-08-09T13:35:32.062 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:32.057+0000 7f12f3f99640 1 -- start start 2023-08-09T13:35:32.062 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:32.057+0000 7f12f3f99640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f12d80039e0 0x7f12ec155290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:35:32.063 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:32.058+0000 7f12f3f99640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f12ec1559e0 0x7f12ec1560e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:35:32.063 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:32.058+0000 7f12f3f99640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f12ec15d9f0 0x7f12ec15de70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:35:32.063 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:32.058+0000 7f12f3f99640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f12ec15e3b0 con 0x7f12d80039e0 2023-08-09T13:35:32.063 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:32.058+0000 7f12f3f99640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f12ec0c9340 con 0x7f12ec1559e0 2023-08-09T13:35:32.063 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:32.058+0000 7f12f3f99640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f12ec1557d0 con 0x7f12ec15d9f0 2023-08-09T13:35:32.063 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:32.058+0000 7f12f1d35640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f12d80039e0 0x7f12ec155290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:35:32.063 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:32.058+0000 7f12f1d35640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f12d80039e0 0x7f12ec155290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:47122/0 (socket says 172.21.15.43:47122) 2023-08-09T13:35:32.063 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:32.058+0000 7f12f1d35640 1 -- 172.21.15.43:0/1931485383 learned_addr learned my addr 172.21.15.43:0/1931485383 (peer_addr_for_me v2:172.21.15.43:0/0) 2023-08-09T13:35:32.064 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:32.058+0000 7f12f1534640 1 --2- 172.21.15.43:0/1931485383 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f12ec1559e0 0x7f12ec1560e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:35:32.064 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:32.058+0000 7f12f1d35640 1 -- 172.21.15.43:0/1931485383 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f12ec15d9f0 msgr2=0x7f12ec15de70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:35:32.064 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:32.058+0000 7f12f1d35640 1 --2- 172.21.15.43:0/1931485383 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f12ec15d9f0 0x7f12ec15de70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:35:32.064 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:32.058+0000 7f12f1d35640 1 -- 172.21.15.43:0/1931485383 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f12ec1559e0 msgr2=0x7f12ec1560e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:35:32.064 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:32.058+0000 7f12f1d35640 1 --2- 172.21.15.43:0/1931485383 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f12ec1559e0 0x7f12ec1560e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:35:32.064 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:32.058+0000 7f12f1d35640 1 -- 172.21.15.43:0/1931485383 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f12ec156620 con 0x7f12d80039e0 2023-08-09T13:35:32.064 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:32.059+0000 7f12f1d35640 1 --2- 172.21.15.43:0/1931485383 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f12d80039e0 0x7f12ec155290 secure :-1 s=READY pgs=379 cs=0 l=1 rev1=1 crypto rx=0x7f12e802c250 tx=0x7f12e8004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-09T13:35:32.065 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:32.059+0000 7f12e27fc640 1 -- 172.21.15.43:0/1931485383 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f12e8003810 con 0x7f12d80039e0 2023-08-09T13:35:32.065 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:32.059+0000 7f12e27fc640 1 -- 172.21.15.43:0/1931485383 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f12e8003970 con 0x7f12d80039e0 2023-08-09T13:35:32.065 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:32.059+0000 7f12e27fc640 1 -- 172.21.15.43:0/1931485383 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f12e8003ce0 con 0x7f12d80039e0 2023-08-09T13:35:32.065 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:32.059+0000 7f12f3f99640 1 -- 172.21.15.43:0/1931485383 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f12ec0c8d80 con 0x7f12d80039e0 2023-08-09T13:35:32.065 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:32.059+0000 7f12f3f99640 1 -- 172.21.15.43:0/1931485383 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f12ec156b00 con 0x7f12d80039e0 2023-08-09T13:35:32.066 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:32.060+0000 7f12f3f99640 1 -- 172.21.15.43:0/1931485383 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f12ec156c90 con 0x7f12d80039e0 2023-08-09T13:35:32.066 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:32.061+0000 7f12e27fc640 1 -- 172.21.15.43:0/1931485383 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 5) v1 ==== 88341+0+0 (secure 0 0 0) 0x7f12e8034440 con 0x7f12d80039e0 2023-08-09T13:35:32.066 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:32.063+0000 7f12e27fc640 1 --2- 172.21.15.43:0/1931485383 >> [v2:172.21.15.203:6832/830097474,v1:172.21.15.203:6833/830097474] conn(0x7f12d006a050 0x7f12d006c510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:35:32.066 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:32.063+0000 7f12e27fc640 1 -- 172.21.15.43:0/1931485383 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 6871+0+0 (secure 0 0 0) 0x7f12e80aa880 con 0x7f12d80039e0 2023-08-09T13:35:32.066 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:32.064+0000 7f12f1534640 1 --2- 172.21.15.43:0/1931485383 >> [v2:172.21.15.203:6832/830097474,v1:172.21.15.203:6833/830097474] conn(0x7f12d006a050 0x7f12d006c510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:35:32.067 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:32.065+0000 7f12f1534640 1 --2- 172.21.15.43:0/1931485383 >> [v2:172.21.15.203:6832/830097474,v1:172.21.15.203:6833/830097474] conn(0x7f12d006a050 0x7f12d006c510 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7f12dc001200 tx=0x7f12dc009040 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-09T13:35:32.070 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:32.067+0000 7f12e27fc640 1 -- 172.21.15.43:0/1931485383 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178066 (secure 0 0 0) 0x7f12e8075d70 con 0x7f12d80039e0 2023-08-09T13:35:32.267 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:32.264+0000 7f12f3f99640 1 -- 172.21.15.43:0/1931485383 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) v1 -- 0x7f12ec0739f0 con 0x7f12d80039e0 2023-08-09T13:35:32.268 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:32.265+0000 7f12e27fc640 1 -- 172.21.15.43:0/1931485383 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v26) v1 ==== 103+0+11 (secure 0 0 0) 0x7f12e80044e0 con 0x7f12d80039e0 2023-08-09T13:35:32.268 INFO:teuthology.orchestra.run.smithi043.stdout:pg_num: 32 2023-08-09T13:35:32.270 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:32.268+0000 7f12f3f99640 1 -- 172.21.15.43:0/1931485383 >> [v2:172.21.15.203:6832/830097474,v1:172.21.15.203:6833/830097474] conn(0x7f12d006a050 msgr2=0x7f12d006c510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:35:32.271 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:32.268+0000 7f12f3f99640 1 --2- 172.21.15.43:0/1931485383 >> [v2:172.21.15.203:6832/830097474,v1:172.21.15.203:6833/830097474] conn(0x7f12d006a050 0x7f12d006c510 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7f12dc001200 tx=0x7f12dc009040 comp rx=0 tx=0).stop 2023-08-09T13:35:32.271 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:32.268+0000 7f12f3f99640 1 -- 172.21.15.43:0/1931485383 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f12d80039e0 msgr2=0x7f12ec155290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:35:32.271 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:32.268+0000 7f12f3f99640 1 --2- 172.21.15.43:0/1931485383 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f12d80039e0 0x7f12ec155290 secure :-1 s=READY pgs=379 cs=0 l=1 rev1=1 crypto rx=0x7f12e802c250 tx=0x7f12e8004730 comp rx=0 tx=0).stop 2023-08-09T13:35:32.271 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:32.268+0000 7f12f3f99640 1 -- 172.21.15.43:0/1931485383 shutdown_connections 2023-08-09T13:35:32.271 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:32.268+0000 7f12f3f99640 1 --2- 172.21.15.43:0/1931485383 >> [v2:172.21.15.203:6832/830097474,v1:172.21.15.203:6833/830097474] conn(0x7f12d006a050 0x7f12d006c510 unknown :-1 s=CLOSED pgs=197 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:35:32.271 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:32.268+0000 7f12f3f99640 1 --2- 172.21.15.43:0/1931485383 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f12ec15d9f0 0x7f12ec15de70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:35:32.271 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:32.268+0000 7f12f3f99640 1 --2- 172.21.15.43:0/1931485383 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f12ec1559e0 0x7f12ec1560e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:35:32.272 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:32.268+0000 7f12f3f99640 1 --2- 172.21.15.43:0/1931485383 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f12d80039e0 0x7f12ec155290 unknown :-1 s=CLOSED pgs=379 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:35:32.272 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:32.268+0000 7f12f3f99640 1 -- 172.21.15.43:0/1931485383 >> 172.21.15.43:0/1931485383 conn(0x7f12ec06a4e0 msgr2=0x7f12ec15ce40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-09T13:35:32.272 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:32.268+0000 7f12f3f99640 1 -- 172.21.15.43:0/1931485383 shutdown_connections 2023-08-09T13:35:32.272 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:32.268+0000 7f12f3f99640 1 -- 172.21.15.43:0/1931485383 wait complete. 2023-08-09T13:35:32.291 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2023-08-09T13:35:32.512 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:32.509+0000 7fe98fe2f640 1 Processor -- start 2023-08-09T13:35:32.512 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:32.509+0000 7fe98fe2f640 1 -- start start 2023-08-09T13:35:32.513 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:32.511+0000 7fe98fe2f640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7fe9880c86f0 0x7fe9880c8b50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:35:32.514 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:32.511+0000 7fe98fe2f640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fe98806c8d0 0x7fe98806cd40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:35:32.514 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:32.511+0000 7fe98fe2f640 1 -- --> v1:172.21.15.43:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe9880c9090 con 0x7fe988143c20 2023-08-09T13:35:32.514 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:32.511+0000 7fe98fe2f640 1 -- --> v2:172.21.15.43:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe988144000 con 0x7fe9880c86f0 2023-08-09T13:35:32.514 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:32.511+0000 7fe98fe2f640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe98806c440 con 0x7fe98806c8d0 2023-08-09T13:35:32.516 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:32.511+0000 7fe98d3ca640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7fe9880c86f0 0x7fe9880c8b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:35:32.517 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:32.511+0000 7fe98d3ca640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7fe9880c86f0 0x7fe9880c8b50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:47128/0 (socket says 172.21.15.43:47128) 2023-08-09T13:35:32.517 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:32.512+0000 7fe98d3ca640 1 -- 172.21.15.43:0/3758932891 learned_addr learned my addr 172.21.15.43:0/3758932891 (peer_addr_for_me v2:172.21.15.43:0/0) 2023-08-09T13:35:32.517 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:32.512+0000 7fe98e3cc640 1 --2- 172.21.15.43:0/3758932891 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fe98806c8d0 0x7fe98806cd40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:35:32.517 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:32.512+0000 7fe98d3ca640 1 -- 172.21.15.43:0/3758932891 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fe98806c8d0 msgr2=0x7fe98806cd40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-09T13:35:32.518 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:32.512+0000 7fe98d3ca640 1 --2- 172.21.15.43:0/3758932891 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fe98806c8d0 0x7fe98806cd40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:35:32.518 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:32.512+0000 7fe98d3ca640 1 -- 172.21.15.43:0/3758932891 >> v1:172.21.15.43:6789/0 conn(0x7fe988143c20 legacy=0x7fe98806f540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:35:32.518 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:32.512+0000 7fe98d3ca640 1 -- 172.21.15.43:0/3758932891 --> v2:172.21.15.43:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe98806fc50 con 0x7fe9880c86f0 2023-08-09T13:35:32.519 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:32.513+0000 7fe98d3ca640 1 --2- 172.21.15.43:0/3758932891 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fe9880c86f0 0x7fe9880c8b50 secure :-1 s=READY pgs=381 cs=0 l=1 rev1=1 crypto rx=0x7fe978009870 tx=0x7fe97802b8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1e7157c6ccae9567 server_cookie=0 in_seq=0 out_seq=0 2023-08-09T13:35:32.519 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:32.514+0000 7fe98cbc9640 1 -- 172.21.15.43:0/3758932891 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe978031640 con 0x7fe9880c86f0 2023-08-09T13:35:32.519 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:32.514+0000 7fe98cbc9640 1 -- 172.21.15.43:0/3758932891 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe9780317a0 con 0x7fe9880c86f0 2023-08-09T13:35:32.519 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:32.514+0000 7fe98cbc9640 1 -- 172.21.15.43:0/3758932891 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe978031b30 con 0x7fe9880c86f0 2023-08-09T13:35:32.523 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:32.514+0000 7fe98fe2f640 1 -- 172.21.15.43:0/3758932891 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fe9880c86f0 msgr2=0x7fe9880c8b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:35:32.524 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:32.514+0000 7fe98fe2f640 1 --2- 172.21.15.43:0/3758932891 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fe9880c86f0 0x7fe9880c8b50 secure :-1 s=READY pgs=381 cs=0 l=1 rev1=1 crypto rx=0x7fe978009870 tx=0x7fe97802b8f0 comp rx=0 tx=0).stop 2023-08-09T13:35:32.524 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:32.514+0000 7fe98fe2f640 1 -- 172.21.15.43:0/3758932891 shutdown_connections 2023-08-09T13:35:32.524 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:32.514+0000 7fe98fe2f640 1 --2- 172.21.15.43:0/3758932891 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fe98806c8d0 0x7fe98806cd40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:35:32.524 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:32.514+0000 7fe98fe2f640 1 --2- 172.21.15.43:0/3758932891 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fe9880c86f0 0x7fe9880c8b50 unknown :-1 s=CLOSED pgs=381 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:35:32.524 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:32.514+0000 7fe98fe2f640 1 -- 172.21.15.43:0/3758932891 >> 172.21.15.43:0/3758932891 conn(0x7fe9880ce0a0 msgr2=0x7fe9880ce480 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-09T13:35:32.525 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:32.515+0000 7fe98fe2f640 1 -- 172.21.15.43:0/3758932891 shutdown_connections 2023-08-09T13:35:32.525 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:32.515+0000 7fe98fe2f640 1 -- 172.21.15.43:0/3758932891 wait complete. 2023-08-09T13:35:32.525 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:32.516+0000 7fe98fe2f640 1 Processor -- start 2023-08-09T13:35:32.525 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:32.517+0000 7fe98fe2f640 1 -- start start 2023-08-09T13:35:32.525 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:32.517+0000 7fe98fe2f640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fe98806c8d0 0x7fe9880ab8c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:35:32.525 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:32.518+0000 7fe98fe2f640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fe9880c86f0 0x7fe9881421e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:35:32.525 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:32.518+0000 7fe98fe2f640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fe988143c20 0x7fe9880b1820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:35:32.526 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:32.518+0000 7fe98fe2f640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe988142720 con 0x7fe98806c8d0 2023-08-09T13:35:32.526 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:32.518+0000 7fe98fe2f640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe9880b1d60 con 0x7fe9880c86f0 2023-08-09T13:35:32.526 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:32.518+0000 7fe98fe2f640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe9880abea0 con 0x7fe988143c20 2023-08-09T13:35:32.526 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:32.518+0000 7fe98dbcb640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fe98806c8d0 0x7fe9880ab8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:35:32.526 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:32.518+0000 7fe98dbcb640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fe98806c8d0 0x7fe9880ab8c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:47134/0 (socket says 172.21.15.43:47134) 2023-08-09T13:35:32.526 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:32.518+0000 7fe98dbcb640 1 -- 172.21.15.43:0/2413508690 learned_addr learned my addr 172.21.15.43:0/2413508690 (peer_addr_for_me v2:172.21.15.43:0/0) 2023-08-09T13:35:32.526 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:32.518+0000 7fe98d3ca640 1 --2- 172.21.15.43:0/2413508690 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fe9880c86f0 0x7fe9881421e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:35:32.527 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:32.518+0000 7fe98e3cc640 1 --2- 172.21.15.43:0/2413508690 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fe988143c20 0x7fe9880b1820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:35:32.527 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:32.518+0000 7fe98dbcb640 1 -- 172.21.15.43:0/2413508690 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fe988143c20 msgr2=0x7fe9880b1820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:35:32.527 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:32.518+0000 7fe98dbcb640 1 --2- 172.21.15.43:0/2413508690 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fe988143c20 0x7fe9880b1820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:35:32.527 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:32.518+0000 7fe98dbcb640 1 -- 172.21.15.43:0/2413508690 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fe9880c86f0 msgr2=0x7fe9881421e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:35:32.527 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:32.518+0000 7fe98dbcb640 1 --2- 172.21.15.43:0/2413508690 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fe9880c86f0 0x7fe9881421e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:35:32.527 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:32.518+0000 7fe98dbcb640 1 -- 172.21.15.43:0/2413508690 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe9780094d0 con 0x7fe98806c8d0 2023-08-09T13:35:32.527 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:32.518+0000 7fe98e3cc640 1 --2- 172.21.15.43:0/2413508690 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fe988143c20 0x7fe9880b1820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-08-09T13:35:32.527 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:32.519+0000 7fe98dbcb640 1 --2- 172.21.15.43:0/2413508690 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fe98806c8d0 0x7fe9880ab8c0 secure :-1 s=READY pgs=382 cs=0 l=1 rev1=1 crypto rx=0x7fe98400c450 tx=0x7fe98400c910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-09T13:35:32.528 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:32.519+0000 7fe97e7fc640 1 -- 172.21.15.43:0/2413508690 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe9840052d0 con 0x7fe98806c8d0 2023-08-09T13:35:32.528 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:32.519+0000 7fe97e7fc640 1 -- 172.21.15.43:0/2413508690 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe984005430 con 0x7fe98806c8d0 2023-08-09T13:35:32.528 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:32.519+0000 7fe97e7fc640 1 -- 172.21.15.43:0/2413508690 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe984012870 con 0x7fe98806c8d0 2023-08-09T13:35:32.528 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:32.519+0000 7fe98fe2f640 1 -- 172.21.15.43:0/2413508690 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe9880ac180 con 0x7fe98806c8d0 2023-08-09T13:35:32.528 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:32.519+0000 7fe98fe2f640 1 -- 172.21.15.43:0/2413508690 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe9880ac740 con 0x7fe98806c8d0 2023-08-09T13:35:32.528 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:32.521+0000 7fe98fe2f640 1 -- 172.21.15.43:0/2413508690 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe9880739f0 con 0x7fe98806c8d0 2023-08-09T13:35:32.529 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:32.521+0000 7fe97e7fc640 1 -- 172.21.15.43:0/2413508690 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 5) v1 ==== 88341+0+0 (secure 0 0 0) 0x7fe9840129d0 con 0x7fe98806c8d0 2023-08-09T13:35:32.529 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:32.522+0000 7fe97e7fc640 1 --2- 172.21.15.43:0/2413508690 >> [v2:172.21.15.203:6832/830097474,v1:172.21.15.203:6833/830097474] conn(0x7fe96c06a080 0x7fe96c06c540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:35:32.529 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:32.522+0000 7fe97e7fc640 1 -- 172.21.15.43:0/2413508690 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 6871+0+0 (secure 0 0 0) 0x7fe984088340 con 0x7fe98806c8d0 2023-08-09T13:35:32.530 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:32.523+0000 7fe98d3ca640 1 --2- 172.21.15.43:0/2413508690 >> [v2:172.21.15.203:6832/830097474,v1:172.21.15.203:6833/830097474] conn(0x7fe96c06a080 0x7fe96c06c540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:35:32.530 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:32.524+0000 7fe98d3ca640 1 --2- 172.21.15.43:0/2413508690 >> [v2:172.21.15.203:6832/830097474,v1:172.21.15.203:6833/830097474] conn(0x7fe96c06a080 0x7fe96c06c540 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7fe978036000 tx=0x7fe97803c040 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-09T13:35:32.530 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:32.527+0000 7fe97e7fc640 1 -- 172.21.15.43:0/2413508690 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178066 (secure 0 0 0) 0x7fe984053790 con 0x7fe98806c8d0 2023-08-09T13:35:32.740 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:32.737+0000 7fe98fe2f640 1 -- 172.21.15.43:0/2413508690 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7fe9880c2f20 con 0x7fe98806c8d0 2023-08-09T13:35:32.740 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:32.738+0000 7fe97e7fc640 1 -- 172.21.15.43:0/2413508690 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v26) v1 ==== 80+0+0 (secure 0 0 0) 0x7fe984056db0 con 0x7fe98806c8d0 2023-08-09T13:35:32.741 INFO:teuthology.orchestra.run.smithi043.stderr:listed 0 entries 2023-08-09T13:35:32.743 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:32.741+0000 7fe98fe2f640 1 -- 172.21.15.43:0/2413508690 >> [v2:172.21.15.203:6832/830097474,v1:172.21.15.203:6833/830097474] conn(0x7fe96c06a080 msgr2=0x7fe96c06c540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:35:32.744 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:32.741+0000 7fe98fe2f640 1 --2- 172.21.15.43:0/2413508690 >> [v2:172.21.15.203:6832/830097474,v1:172.21.15.203:6833/830097474] conn(0x7fe96c06a080 0x7fe96c06c540 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7fe978036000 tx=0x7fe97803c040 comp rx=0 tx=0).stop 2023-08-09T13:35:32.744 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:32.741+0000 7fe98fe2f640 1 -- 172.21.15.43:0/2413508690 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fe98806c8d0 msgr2=0x7fe9880ab8c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:35:32.744 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:32.741+0000 7fe98fe2f640 1 --2- 172.21.15.43:0/2413508690 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fe98806c8d0 0x7fe9880ab8c0 secure :-1 s=READY pgs=382 cs=0 l=1 rev1=1 crypto rx=0x7fe98400c450 tx=0x7fe98400c910 comp rx=0 tx=0).stop 2023-08-09T13:35:32.744 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:32.741+0000 7fe98fe2f640 1 -- 172.21.15.43:0/2413508690 shutdown_connections 2023-08-09T13:35:32.744 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:32.741+0000 7fe98fe2f640 1 --2- 172.21.15.43:0/2413508690 >> [v2:172.21.15.203:6832/830097474,v1:172.21.15.203:6833/830097474] conn(0x7fe96c06a080 0x7fe96c06c540 unknown :-1 s=CLOSED pgs=198 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:35:32.744 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:32.741+0000 7fe98fe2f640 1 --2- 172.21.15.43:0/2413508690 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fe988143c20 0x7fe9880b1820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:35:32.744 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:32.741+0000 7fe98fe2f640 1 --2- 172.21.15.43:0/2413508690 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fe9880c86f0 0x7fe9881421e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:35:32.745 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:32.741+0000 7fe98fe2f640 1 --2- 172.21.15.43:0/2413508690 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fe98806c8d0 0x7fe9880ab8c0 unknown :-1 s=CLOSED pgs=382 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:35:32.745 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:32.741+0000 7fe98fe2f640 1 -- 172.21.15.43:0/2413508690 >> 172.21.15.43:0/2413508690 conn(0x7fe9880ce0a0 msgr2=0x7fe9880aede0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-09T13:35:32.745 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:32.741+0000 7fe98fe2f640 1 -- 172.21.15.43:0/2413508690 shutdown_connections 2023-08-09T13:35:32.745 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:32.741+0000 7fe98fe2f640 1 -- 172.21.15.43:0/2413508690 wait complete. 2023-08-09T13:35:32.763 DEBUG:teuthology.orchestra.run.smithi203:> set -ex 2023-08-09T13:35:32.763 DEBUG:teuthology.orchestra.run.smithi203:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-09T13:35:32.823 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-08-09T13:35:33.048 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:33.045+0000 7f31a6871640 1 Processor -- start 2023-08-09T13:35:33.048 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:33.045+0000 7f31a6871640 1 -- start start 2023-08-09T13:35:33.049 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:33.047+0000 7f31a6871640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f31a00ccf20 0x7f31a00cd550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:35:33.050 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:33.047+0000 7f31a6871640 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7f31a00cda90 0x7f31a015ca10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:35:33.050 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:33.047+0000 7f31a6871640 1 -- --> v1:172.21.15.43:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f31a00cca90 con 0x7f31a015c630 2023-08-09T13:35:33.050 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:33.047+0000 7f31a6871640 1 -- --> v2:172.21.15.43:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f31a00ac600 con 0x7f31a00ccf20 2023-08-09T13:35:33.050 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:33.047+0000 7f31a6871640 1 -- --> v2:172.21.15.203:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f31a00cbc70 con 0x7f31a00cda90 2023-08-09T13:35:33.051 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:33.048+0000 7f3197fff640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f31a00ccf20 0x7f31a00cd550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:35:33.051 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:33.048+0000 7f3197fff640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f31a00ccf20 0x7f31a00cd550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:47140/0 (socket says 172.21.15.43:47140) 2023-08-09T13:35:33.051 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:33.048+0000 7f3197fff640 1 -- 172.21.15.43:0/2960295665 learned_addr learned my addr 172.21.15.43:0/2960295665 (peer_addr_for_me v2:172.21.15.43:0/0) 2023-08-09T13:35:33.051 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:33.048+0000 7f31a4e0e640 1 --2- 172.21.15.43:0/2960295665 >> v2:172.21.15.203:3300/0 conn(0x7f31a00cda90 0x7f31a015ca10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:35:33.052 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:33.049+0000 7f3197fff640 1 -- 172.21.15.43:0/2960295665 >> v2:172.21.15.203:3300/0 conn(0x7f31a00cda90 msgr2=0x7f31a015ca10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-09T13:35:33.052 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:33.049+0000 7f3197fff640 1 --2- 172.21.15.43:0/2960295665 >> v2:172.21.15.203:3300/0 conn(0x7f31a00cda90 0x7f31a015ca10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:35:33.052 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:33.049+0000 7f3197fff640 1 -- 172.21.15.43:0/2960295665 >> v1:172.21.15.43:6789/0 conn(0x7f31a015c630 legacy=0x7f31a00abef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:35:33.052 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:33.049+0000 7f3197fff640 1 -- 172.21.15.43:0/2960295665 --> v2:172.21.15.43:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f31a00cdea0 con 0x7f31a00ccf20 2023-08-09T13:35:33.053 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:33.050+0000 7f3197fff640 1 --2- 172.21.15.43:0/2960295665 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f31a00ccf20 0x7f31a00cd550 secure :-1 s=READY pgs=384 cs=0 l=1 rev1=1 crypto rx=0x7f31a00ca9b0 tx=0x7f318802b9a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=65b8c449f3dc4bfa server_cookie=0 in_seq=0 out_seq=0 2023-08-09T13:35:33.053 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:33.050+0000 7f319f7fe640 1 -- 172.21.15.43:0/2960295665 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3188030860 con 0x7f31a00ccf20 2023-08-09T13:35:33.053 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:33.050+0000 7f319f7fe640 1 -- 172.21.15.43:0/2960295665 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f31880309c0 con 0x7f31a00ccf20 2023-08-09T13:35:33.053 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:33.050+0000 7f319f7fe640 1 -- 172.21.15.43:0/2960295665 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3188030d50 con 0x7f31a00ccf20 2023-08-09T13:35:33.054 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:33.051+0000 7f31a6871640 1 -- 172.21.15.43:0/2960295665 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f31a00ccf20 msgr2=0x7f31a00cd550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:35:33.054 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:33.051+0000 7f31a6871640 1 --2- 172.21.15.43:0/2960295665 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f31a00ccf20 0x7f31a00cd550 secure :-1 s=READY pgs=384 cs=0 l=1 rev1=1 crypto rx=0x7f31a00ca9b0 tx=0x7f318802b9a0 comp rx=0 tx=0).stop 2023-08-09T13:35:33.054 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:33.051+0000 7f31a6871640 1 -- 172.21.15.43:0/2960295665 shutdown_connections 2023-08-09T13:35:33.054 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:33.051+0000 7f31a6871640 1 --2- 172.21.15.43:0/2960295665 >> v2:172.21.15.203:3300/0 conn(0x7f31a00cda90 0x7f31a015ca10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:35:33.054 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:33.051+0000 7f31a6871640 1 --2- 172.21.15.43:0/2960295665 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f31a00ccf20 0x7f31a00cd550 unknown :-1 s=CLOSED pgs=384 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:35:33.055 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:33.051+0000 7f31a6871640 1 -- 172.21.15.43:0/2960295665 >> 172.21.15.43:0/2960295665 conn(0x7f31a00ce0a0 msgr2=0x7f31a00ce480 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-09T13:35:33.055 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:33.051+0000 7f31a6871640 1 -- 172.21.15.43:0/2960295665 shutdown_connections 2023-08-09T13:35:33.055 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:33.051+0000 7f31a6871640 1 -- 172.21.15.43:0/2960295665 wait complete. 2023-08-09T13:35:33.055 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:33.053+0000 7f31a6871640 1 Processor -- start 2023-08-09T13:35:33.055 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:33.053+0000 7f31a6871640 1 -- start start 2023-08-09T13:35:33.056 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:33.054+0000 7f31a6871640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f31a00ccf20 0x7f31a01f1ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:35:33.057 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:33.054+0000 7f31a6871640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f31a00cda90 0x7f31a00c7a90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:35:33.057 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:33.054+0000 7f31a6871640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f31a015c630 0x7f31a015f640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:35:33.057 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:33.054+0000 7f31a6871640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f31a01f2e90 con 0x7f31a00cda90 2023-08-09T13:35:33.057 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:33.054+0000 7f31a6871640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f31a01524b0 con 0x7f31a015c630 2023-08-09T13:35:33.057 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:33.054+0000 7f31a6871640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f31a00c7fd0 con 0x7f31a00ccf20 2023-08-09T13:35:33.057 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:33.054+0000 7f3197fff640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f31a00cda90 0x7f31a00c7a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:35:33.057 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:33.054+0000 7f3197fff640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f31a00cda90 0x7f31a00c7a90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:47150/0 (socket says 172.21.15.43:47150) 2023-08-09T13:35:33.058 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:33.054+0000 7f3197fff640 1 -- 172.21.15.43:0/3029681343 learned_addr learned my addr 172.21.15.43:0/3029681343 (peer_addr_for_me v2:172.21.15.43:0/0) 2023-08-09T13:35:33.058 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:33.055+0000 7f319ffff640 1 --2- 172.21.15.43:0/3029681343 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f31a00ccf20 0x7f31a01f1ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:35:33.059 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:33.055+0000 7f31a4e0e640 1 --2- 172.21.15.43:0/3029681343 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f31a015c630 0x7f31a015f640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:35:33.059 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:33.055+0000 7f3197fff640 1 -- 172.21.15.43:0/3029681343 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f31a00ccf20 msgr2=0x7f31a01f1ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:35:33.059 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:33.055+0000 7f3197fff640 1 --2- 172.21.15.43:0/3029681343 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f31a00ccf20 0x7f31a01f1ef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:35:33.059 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:33.055+0000 7f3197fff640 1 -- 172.21.15.43:0/3029681343 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f31a015c630 msgr2=0x7f31a015f640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:35:33.059 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:33.055+0000 7f3197fff640 1 --2- 172.21.15.43:0/3029681343 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f31a015c630 0x7f31a015f640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:35:33.059 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:33.055+0000 7f3197fff640 1 -- 172.21.15.43:0/3029681343 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f31880094d0 con 0x7f31a00cda90 2023-08-09T13:35:33.060 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:33.055+0000 7f319ffff640 1 --2- 172.21.15.43:0/3029681343 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f31a00ccf20 0x7f31a01f1ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-08-09T13:35:33.060 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:33.055+0000 7f31a4e0e640 1 --2- 172.21.15.43:0/3029681343 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f31a015c630 0x7f31a015f640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-08-09T13:35:33.060 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:33.055+0000 7f3197fff640 1 --2- 172.21.15.43:0/3029681343 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f31a00cda90 0x7f31a00c7a90 secure :-1 s=READY pgs=385 cs=0 l=1 rev1=1 crypto rx=0x7f3188000c00 tx=0x7f3188004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-09T13:35:33.060 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:33.055+0000 7f319d7fa640 1 -- 172.21.15.43:0/3029681343 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3188039c90 con 0x7f31a00cda90 2023-08-09T13:35:33.060 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:33.055+0000 7f319d7fa640 1 -- 172.21.15.43:0/3029681343 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3188039e10 con 0x7f31a00cda90 2023-08-09T13:35:33.060 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:33.055+0000 7f319d7fa640 1 -- 172.21.15.43:0/3029681343 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f31880446c0 con 0x7f31a00cda90 2023-08-09T13:35:33.060 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:33.055+0000 7f31a6871640 1 -- 172.21.15.43:0/3029681343 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f31a01f25f0 con 0x7f31a00cda90 2023-08-09T13:35:33.060 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:33.056+0000 7f31a6871640 1 -- 172.21.15.43:0/3029681343 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f31a01f2bb0 con 0x7f31a00cda90 2023-08-09T13:35:33.061 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:33.057+0000 7f31a6871640 1 -- 172.21.15.43:0/3029681343 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f31a00739f0 con 0x7f31a00cda90 2023-08-09T13:35:33.061 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:33.058+0000 7f319d7fa640 1 -- 172.21.15.43:0/3029681343 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 5) v1 ==== 88341+0+0 (secure 0 0 0) 0x7f318804d3b0 con 0x7f31a00cda90 2023-08-09T13:35:33.061 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:33.059+0000 7f319d7fa640 1 --2- 172.21.15.43:0/3029681343 >> [v2:172.21.15.203:6832/830097474,v1:172.21.15.203:6833/830097474] conn(0x7f318006a050 0x7f318006c510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:35:33.062 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:33.059+0000 7f319d7fa640 1 -- 172.21.15.43:0/3029681343 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 6871+0+0 (secure 0 0 0) 0x7f318803e070 con 0x7f31a00cda90 2023-08-09T13:35:33.062 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:33.060+0000 7f319ffff640 1 --2- 172.21.15.43:0/3029681343 >> [v2:172.21.15.203:6832/830097474,v1:172.21.15.203:6833/830097474] conn(0x7f318006a050 0x7f318006c510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:35:33.063 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:33.061+0000 7f319ffff640 1 --2- 172.21.15.43:0/3029681343 >> [v2:172.21.15.203:6832/830097474,v1:172.21.15.203:6833/830097474] conn(0x7f318006a050 0x7f318006c510 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7f3190000f60 tx=0x7f3190009040 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-09T13:35:33.066 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:33.064+0000 7f319d7fa640 1 -- 172.21.15.43:0/3029681343 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178066 (secure 0 0 0) 0x7f31880768b0 con 0x7f31a00cda90 2023-08-09T13:35:33.264 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:33.261+0000 7f31a6871640 1 -- 172.21.15.43:0/3029681343 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f31a0067be0 con 0x7f31a00cda90 2023-08-09T13:35:33.267 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:33.264+0000 7f319d7fa640 1 -- 172.21.15.43:0/3029681343 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v26) v1 ==== 74+0+20146 (secure 0 0 0) 0x7f3188079ed0 con 0x7f31a00cda90 2023-08-09T13:35:33.268 INFO:teuthology.orchestra.run.smithi043.stdout: 2023-08-09T13:35:33.268 INFO:teuthology.orchestra.run.smithi043.stdout:{"epoch":26,"fsid":"daac87ee-1337-4250-a955-aea4e408a39b","created":"2023-08-09T13:33:41.701198+0000","modified":"2023-08-09T13:34:48.711911+0000","last_up_change":"2023-08-09T13:33:53.783453+0000","last_in_change":"2023-08-09T13:33:45.148904+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":5,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":"rbd","create_time":"2023-08-09T13:33:54.365137+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":".mgr","create_time":"2023-08-09T13:33:54.831803+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_metadata","create_time":"2023-08-09T13:34:00.207974+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data","create_time":"2023-08-09T13:34:01.614765+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":5,"pool_name":"cephfs_data_ec","create_time":"2023-08-09T13:34:05.239583+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"24","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"cbd49ecd-e896-4fbd-864e-5c37a829b165","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6800","nonce":502667335},{"type":"v1","addr":"172.21.15.43:6801","nonce":502667335}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6802","nonce":502667335},{"type":"v1","addr":"172.21.15.43:6803","nonce":502667335}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6806","nonce":502667335},{"type":"v1","addr":"172.21.15.43:6807","nonce":502667335}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6804","nonce":502667335},{"type":"v1","addr":"172.21.15.43:6805","nonce":502667335}]},"public_addr":"172.21.15.43:6801/502667335","cluster_addr":"172.21.15.43:6803/502667335","heartbeat_back_addr":"172.21.15.43:6807/502667335","heartbeat_front_addr":"172.21.15.43:6805/502667335","state":["exists","up"]},{"osd":1,"uuid":"75c7e0fe-1ec7-4fe8-a2f2-3661a65b8500","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6808","nonce":3289134610},{"type":"v1","addr":"172.21.15.43:6809","nonce":3289134610}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6810","nonce":3289134610},{"type":"v1","addr":"172.21.15.43:6812","nonce":3289134610}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6821","nonce":3289134610},{"type":"v1","addr":"172.21.15.43:6824","nonce":3289134610}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6815","nonce":3289134610},{"type":"v1","addr":"172.21.15.43:6818","nonce":3289134610}]},"public_addr":"172.21.15.43:6809/3289134610","cluster_addr":"172.21.15.43:6812/3289134610","heartbeat_back_addr":"172.21.15.43:6824/3289134610","heartbeat_front_addr":"172.21.15.43:6818/3289134610","state":["exists","up"]},{"osd":2,"uuid":"bb5c41e9-7f5f-4125-9b01-3bb8bfc2ea83","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6811","nonce":4079709736},{"type":"v1","addr":"172.21.15.43:6814","nonce":4079709736}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6817","nonce":4079709736},{"type":"v1","addr":"172.21.15.43:6820","nonce":4079709736}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6829","nonce":4079709736},{"type":"v1","addr":"172.21.15.43:6831","nonce":4079709736}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6823","nonce":4079709736},{"type":"v1","addr":"172.21.15.43:6826","nonce":4079709736}]},"public_addr":"172.21.15.43:6814/4079709736","cluster_addr":"172.21.15.43:6820/4079709736","heartbeat_back_addr":"172.21.15.43:6831/4079709736","heartbeat_front_addr":"172.21.15.43:6826/4079709736","state":["exists","up"]},{"osd":3,"uuid":"bfe4c8b4-694a-465c-87e6-2b2aab852613","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6813","nonce":1709538759},{"type":"v1","addr":"172.21.15.43:6816","nonce":1709538759}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6819","nonce":1709538759},{"type":"v1","addr":"172.21.15.43:6822","nonce":1709538759}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6828","nonce":1709538759},{"type":"v1","addr":"172.21.15.43:6830","nonce":1709538759}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6825","nonce":1709538759},{"type":"v1","addr":"172.21.15.43:6827","nonce":1709538759}]},"public_addr":"172.21.15.43:6816/1709538759","cluster_addr":"172.21.15.43:6822/1709538759","heartbeat_back_addr":"172.21.15.43:6830/1709538759","heartbeat_front_addr":"172.21.15.43:6827/1709538759","state":["exists","up"]},{"osd":4,"uuid":"fa2e8a77-aba1-4756-853e-0d174a5bdeba","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6800","nonce":4196866481},{"type":"v1","addr":"172.21.15.203:6801","nonce":4196866481}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6802","nonce":4196866481},{"type":"v1","addr":"172.21.15.203:6803","nonce":4196866481}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6806","nonce":4196866481},{"type":"v1","addr":"172.21.15.203:6807","nonce":4196866481}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6804","nonce":4196866481},{"type":"v1","addr":"172.21.15.203:6805","nonce":4196866481}]},"public_addr":"172.21.15.203:6801/4196866481","cluster_addr":"172.21.15.203:6803/4196866481","heartbeat_back_addr":"172.21.15.203:6807/4196866481","heartbeat_front_addr":"172.21.15.203:6805/4196866481","state":["exists","up"]},{"osd":5,"uuid":"81811e4b-4d24-4f51-a82c-f1d700a2a00d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6816","nonce":3059290081},{"type":"v1","addr":"172.21.15.203:6817","nonce":3059290081}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6818","nonce":3059290081},{"type":"v1","addr":"172.21.15.203:6819","nonce":3059290081}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6822","nonce":3059290081},{"type":"v1","addr":"172.21.15.203:6823","nonce":3059290081}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6820","nonce":3059290081},{"type":"v1","addr":"172.21.15.203:6821","nonce":3059290081}]},"public_addr":"172.21.15.203:6817/3059290081","cluster_addr":"172.21.15.203:6819/3059290081","heartbeat_back_addr":"172.21.15.203:6823/3059290081","heartbeat_front_addr":"172.21.15.203:6821/3059290081","state":["exists","up"]},{"osd":6,"uuid":"307bac4d-6a5b-443d-af75-56d695d7674f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6824","nonce":512155569},{"type":"v1","addr":"172.21.15.203:6825","nonce":512155569}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6826","nonce":512155569},{"type":"v1","addr":"172.21.15.203:6827","nonce":512155569}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6830","nonce":512155569},{"type":"v1","addr":"172.21.15.203:6831","nonce":512155569}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6828","nonce":512155569},{"type":"v1","addr":"172.21.15.203:6829","nonce":512155569}]},"public_addr":"172.21.15.203:6825/512155569","cluster_addr":"172.21.15.203:6827/512155569","heartbeat_back_addr":"172.21.15.203:6831/512155569","heartbeat_front_addr":"172.21.15.203:6829/512155569","state":["exists","up"]},{"osd":7,"uuid":"a95dc7ca-9f1b-4295-af70-68a22a2d7d5e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6808","nonce":3331450438},{"type":"v1","addr":"172.21.15.203:6809","nonce":3331450438}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6810","nonce":3331450438},{"type":"v1","addr":"172.21.15.203:6811","nonce":3331450438}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6814","nonce":3331450438},{"type":"v1","addr":"172.21.15.203:6815","nonce":3331450438}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6812","nonce":3331450438},{"type":"v1","addr":"172.21.15.203:6813","nonce":3331450438}]},"public_addr":"172.21.15.203:6809/3331450438","cluster_addr":"172.21.15.203:6811/3331450438","heartbeat_back_addr":"172.21.15.203:6815/3331450438","heartbeat_front_addr":"172.21.15.203:6813/3331450438","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-09T13:33:50.547652+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-09T13:33:50.589469+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-09T13:33:50.561724+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-09T13:33:50.599650+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-09T13:33:50.411855+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-09T13:33:50.219681+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-09T13:33:50.504664+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-09T13:33:50.441248+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"4.14","mappings":[{"from":1,"to":5}]},{"pgid":"4.29","mappings":[{"from":3,"to":2}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-08-09T13:35:33.270 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:33.267+0000 7f31a6871640 1 -- 172.21.15.43:0/3029681343 >> [v2:172.21.15.203:6832/830097474,v1:172.21.15.203:6833/830097474] conn(0x7f318006a050 msgr2=0x7f318006c510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:35:33.270 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:33.267+0000 7f31a6871640 1 --2- 172.21.15.43:0/3029681343 >> [v2:172.21.15.203:6832/830097474,v1:172.21.15.203:6833/830097474] conn(0x7f318006a050 0x7f318006c510 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7f3190000f60 tx=0x7f3190009040 comp rx=0 tx=0).stop 2023-08-09T13:35:33.270 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:33.267+0000 7f31a6871640 1 -- 172.21.15.43:0/3029681343 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f31a00cda90 msgr2=0x7f31a00c7a90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:35:33.270 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:33.267+0000 7f31a6871640 1 --2- 172.21.15.43:0/3029681343 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f31a00cda90 0x7f31a00c7a90 secure :-1 s=READY pgs=385 cs=0 l=1 rev1=1 crypto rx=0x7f3188000c00 tx=0x7f3188004730 comp rx=0 tx=0).stop 2023-08-09T13:35:33.271 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:33.268+0000 7f31a6871640 1 -- 172.21.15.43:0/3029681343 shutdown_connections 2023-08-09T13:35:33.271 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:33.268+0000 7f31a6871640 1 --2- 172.21.15.43:0/3029681343 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f31a015c630 0x7f31a015f640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:35:33.271 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:33.268+0000 7f31a6871640 1 --2- 172.21.15.43:0/3029681343 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f31a00cda90 0x7f31a00c7a90 unknown :-1 s=CLOSED pgs=385 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:35:33.271 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:33.268+0000 7f31a6871640 1 --2- 172.21.15.43:0/3029681343 >> [v2:172.21.15.203:6832/830097474,v1:172.21.15.203:6833/830097474] conn(0x7f318006a050 0x7f318006c510 unknown :-1 s=CLOSED pgs=199 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:35:33.271 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:33.268+0000 7f31a6871640 1 --2- 172.21.15.43:0/3029681343 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f31a00ccf20 0x7f31a01f1ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:35:33.272 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:33.268+0000 7f31a6871640 1 -- 172.21.15.43:0/3029681343 >> 172.21.15.43:0/3029681343 conn(0x7f31a00ce0a0 msgr2=0x7f31a006c680 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-09T13:35:33.272 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:33.268+0000 7f31a6871640 1 -- 172.21.15.43:0/3029681343 shutdown_connections 2023-08-09T13:35:33.272 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:33.268+0000 7f31a6871640 1 -- 172.21.15.43:0/3029681343 wait complete. 2023-08-09T13:35:33.290 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': 'rbd', 'create_time': '2023-08-09T13:33:54.365137+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}, 'read_balance': {'score_acting': 2, 'score_stable': 2, 'optimal_score': 1, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': '.mgr', 'create_time': '2023-08-09T13:33:54.831803+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_metadata', 'create_time': '2023-08-09T13:34:00.207974+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data', 'create_time': '2023-08-09T13:34:01.614765+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 2, 'score_stable': 2, 'optimal_score': 1, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 5, 'pool_name': 'cephfs_data_ec', 'create_time': '2023-08-09T13:34:05.239583+0000', 'flags': 5, 'flags_names': 'hashpspool,ec_overwrites', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '24', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2023-08-09T13:35:33.291 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get rbd pg_num 2023-08-09T13:35:33.514 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:33.512+0000 7ff54084b640 1 Processor -- start 2023-08-09T13:35:33.515 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:33.512+0000 7ff54084b640 1 -- start start 2023-08-09T13:35:33.516 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:33.513+0000 7ff54084b640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7ff53c159400 0x7ff53c0cb9a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:35:33.516 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:33.513+0000 7ff54084b640 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7ff53c0c8900 0x7ff53c15a270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:35:33.516 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:33.513+0000 7ff54084b640 1 -- --> v1:172.21.15.43:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff53c15a7b0 con 0x7ff53c0c8e50 2023-08-09T13:35:33.516 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:33.513+0000 7ff54084b640 1 -- --> v2:172.21.15.203:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff53c070250 con 0x7ff53c0c8900 2023-08-09T13:35:33.517 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:33.513+0000 7ff54084b640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff53c0c86c0 con 0x7ff53c159400 2023-08-09T13:35:33.518 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:33.514+0000 7ff53bfff640 1 --1- >> v1:172.21.15.43:6789/0 conn(0x7ff53c0c8e50 0x7ff53c0706e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.43:6789/0 says I am v1:172.21.15.43:52254/0 (socket says 172.21.15.43:52254) 2023-08-09T13:35:33.518 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:33.514+0000 7ff53bfff640 1 -- 172.21.15.43:0/149870855 learned_addr learned my addr 172.21.15.43:0/149870855 (peer_addr_for_me v1:172.21.15.43:0/0) 2023-08-09T13:35:33.518 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:33.514+0000 7ff53b7fe640 1 --2- 172.21.15.43:0/149870855 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7ff53c159400 0x7ff53c0cb9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:35:33.518 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:33.514+0000 7ff53affd640 1 --2- 172.21.15.43:0/149870855 >> v2:172.21.15.203:3300/0 conn(0x7ff53c0c8900 0x7ff53c15a270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:35:33.518 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:33.514+0000 7ff53a7fc640 1 -- 172.21.15.43:0/149870855 <== mon.0 v1:172.21.15.43:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2639529390 0 0) 0x7ff534001490 con 0x7ff53c0c8e50 2023-08-09T13:35:33.518 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:33.514+0000 7ff53a7fc640 1 -- 172.21.15.43:0/149870855 --> v1:172.21.15.43:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ff524003520 con 0x7ff53c0c8e50 2023-08-09T13:35:33.519 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:33.515+0000 7ff53a7fc640 1 -- 172.21.15.43:0/149870855 <== mon.0 v1:172.21.15.43:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 725985115 0 0) 0x7ff524003520 con 0x7ff53c0c8e50 2023-08-09T13:35:33.519 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:33.515+0000 7ff53a7fc640 1 -- 172.21.15.43:0/149870855 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7ff53c159400 msgr2=0x7ff53c0cb9a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-09T13:35:33.519 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:33.515+0000 7ff53a7fc640 1 --2- 172.21.15.43:0/149870855 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7ff53c159400 0x7ff53c0cb9a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:35:33.519 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:33.515+0000 7ff53a7fc640 1 -- 172.21.15.43:0/149870855 >> v2:172.21.15.203:3300/0 conn(0x7ff53c0c8900 msgr2=0x7ff53c15a270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-09T13:35:33.519 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:33.515+0000 7ff53a7fc640 1 --2- 172.21.15.43:0/149870855 >> v2:172.21.15.203:3300/0 conn(0x7ff53c0c8900 0x7ff53c15a270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:35:33.519 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:33.515+0000 7ff53a7fc640 1 -- 172.21.15.43:0/149870855 --> v1:172.21.15.43:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff53c06fc60 con 0x7ff53c0c8e50 2023-08-09T13:35:33.519 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:33.515+0000 7ff53a7fc640 1 -- 172.21.15.43:0/149870855 <== mon.0 v1:172.21.15.43:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1285150740 0 0) 0x7ff534001b80 con 0x7ff53c0c8e50 2023-08-09T13:35:33.519 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:33.515+0000 7ff53a7fc640 1 -- 172.21.15.43:0/149870855 >> v1:172.21.15.43:6789/0 conn(0x7ff53c0c8e50 legacy=0x7ff53c0706e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:35:33.520 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:33.515+0000 7ff53a7fc640 1 --2- 172.21.15.43:0/149870855 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7ff524003970 0x7ff524003d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:35:33.520 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:33.515+0000 7ff53a7fc640 1 --2- 172.21.15.43:0/149870855 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7ff5240047c0 0x7ff524004c00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:35:33.520 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:33.515+0000 7ff53a7fc640 1 --2- 172.21.15.43:0/149870855 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff524005140 0x7ff52400b980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:35:33.520 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:33.515+0000 7ff53a7fc640 1 -- 172.21.15.43:0/149870855 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff53c070250 con 0x7ff524005140 2023-08-09T13:35:33.520 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:33.515+0000 7ff53a7fc640 1 -- 172.21.15.43:0/149870855 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff53c0c86c0 con 0x7ff524003970 2023-08-09T13:35:33.520 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:33.515+0000 7ff53a7fc640 1 -- 172.21.15.43:0/149870855 --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff52400bec0 con 0x7ff5240047c0 2023-08-09T13:35:33.520 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:33.515+0000 7ff53bfff640 1 --2- 172.21.15.43:0/149870855 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff524005140 0x7ff52400b980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:35:33.522 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:33.516+0000 7ff53b7fe640 1 --2- 172.21.15.43:0/149870855 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7ff524003970 0x7ff524003d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:35:33.522 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:33.516+0000 7ff53affd640 1 --2- 172.21.15.43:0/149870855 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7ff5240047c0 0x7ff524004c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:35:33.522 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:33.516+0000 7ff53bfff640 1 -- 172.21.15.43:0/149870855 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7ff5240047c0 msgr2=0x7ff524004c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-09T13:35:33.522 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:33.516+0000 7ff53bfff640 1 --2- 172.21.15.43:0/149870855 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7ff5240047c0 0x7ff524004c00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:35:33.523 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:33.516+0000 7ff53bfff640 1 -- 172.21.15.43:0/149870855 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7ff524003970 msgr2=0x7ff524003d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-09T13:35:33.523 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:33.516+0000 7ff53bfff640 1 --2- 172.21.15.43:0/149870855 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7ff524003970 0x7ff524003d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:35:33.523 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:33.516+0000 7ff53bfff640 1 -- 172.21.15.43:0/149870855 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff524003520 con 0x7ff524005140 2023-08-09T13:35:33.523 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:33.517+0000 7ff53bfff640 1 --2- 172.21.15.43:0/149870855 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff524005140 0x7ff52400b980 secure :-1 s=READY pgs=387 cs=0 l=1 rev1=1 crypto rx=0x7ff53400a830 tx=0x7ff5240057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a1b46c139177eb32 server_cookie=0 in_seq=0 out_seq=0 2023-08-09T13:35:33.523 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:33.518+0000 7ff53a7fc640 1 -- 172.21.15.43:0/149870855 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff534034630 con 0x7ff524005140 2023-08-09T13:35:33.523 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:33.518+0000 7ff53a7fc640 1 -- 172.21.15.43:0/149870855 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff53402f070 con 0x7ff524005140 2023-08-09T13:35:33.523 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:33.518+0000 7ff53a7fc640 1 -- 172.21.15.43:0/149870855 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff5340349f0 con 0x7ff524005140 2023-08-09T13:35:33.524 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:33.518+0000 7ff54084b640 1 -- 172.21.15.43:0/149870855 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff524005140 msgr2=0x7ff52400b980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:35:33.524 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:33.518+0000 7ff54084b640 1 --2- 172.21.15.43:0/149870855 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff524005140 0x7ff52400b980 secure :-1 s=READY pgs=387 cs=0 l=1 rev1=1 crypto rx=0x7ff53400a830 tx=0x7ff5240057f0 comp rx=0 tx=0).stop 2023-08-09T13:35:33.524 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:33.518+0000 7ff53bfff640 1 -- 172.21.15.43:0/149870855 reap_dead start 2023-08-09T13:35:33.524 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:33.518+0000 7ff54084b640 1 -- 172.21.15.43:0/149870855 shutdown_connections 2023-08-09T13:35:33.524 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:33.518+0000 7ff54084b640 1 -- 172.21.15.43:0/149870855 >> 172.21.15.43:0/149870855 conn(0x7ff53c0cddd0 msgr2=0x7ff53c0ce1e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-09T13:35:33.524 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:33.518+0000 7ff54084b640 1 -- 172.21.15.43:0/149870855 shutdown_connections 2023-08-09T13:35:33.524 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:33.519+0000 7ff54084b640 1 -- 172.21.15.43:0/149870855 wait complete. 2023-08-09T13:35:33.526 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:33.520+0000 7ff54084b640 1 Processor -- start 2023-08-09T13:35:33.526 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:33.520+0000 7ff54084b640 1 -- start start 2023-08-09T13:35:33.526 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:33.521+0000 7ff54084b640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff524005140 0x7ff53c144ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:35:33.526 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:33.521+0000 7ff54084b640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7ff53c0c1df0 0x7ff53c0cbc40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:35:33.526 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:33.521+0000 7ff54084b640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7ff53c151250 0x7ff53c0c2690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:35:33.527 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:33.521+0000 7ff54084b640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff53c0c22f0 con 0x7ff524005140 2023-08-09T13:35:33.527 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:33.521+0000 7ff54084b640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff53c143af0 con 0x7ff53c0c1df0 2023-08-09T13:35:33.527 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:33.521+0000 7ff54084b640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff53c144630 con 0x7ff53c151250 2023-08-09T13:35:33.527 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:33.522+0000 7ff53b7fe640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff524005140 0x7ff53c144ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:35:33.527 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:33.522+0000 7ff53b7fe640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff524005140 0x7ff53c144ac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:47170/0 (socket says 172.21.15.43:47170) 2023-08-09T13:35:33.527 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:33.522+0000 7ff53b7fe640 1 -- 172.21.15.43:0/1888652188 learned_addr learned my addr 172.21.15.43:0/1888652188 (peer_addr_for_me v2:172.21.15.43:0/0) 2023-08-09T13:35:33.527 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:33.522+0000 7ff53b7fe640 1 -- 172.21.15.43:0/1888652188 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7ff53c151250 msgr2=0x7ff53c0c2690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:35:33.527 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:33.522+0000 7ff53b7fe640 1 --2- 172.21.15.43:0/1888652188 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7ff53c151250 0x7ff53c0c2690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:35:33.528 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:33.522+0000 7ff53b7fe640 1 -- 172.21.15.43:0/1888652188 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7ff53c0c1df0 msgr2=0x7ff53c0cbc40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:35:33.528 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:33.522+0000 7ff53b7fe640 1 --2- 172.21.15.43:0/1888652188 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7ff53c0c1df0 0x7ff53c0cbc40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:35:33.528 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:33.522+0000 7ff53b7fe640 1 -- 172.21.15.43:0/1888652188 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff53c143d00 con 0x7ff524005140 2023-08-09T13:35:33.528 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:33.522+0000 7ff53b7fe640 1 --2- 172.21.15.43:0/1888652188 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff524005140 0x7ff53c144ac0 secure :-1 s=READY pgs=388 cs=0 l=1 rev1=1 crypto rx=0x7ff52c00f580 tx=0x7ff52c00fa40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-09T13:35:33.528 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:33.523+0000 7ff523fff640 1 -- 172.21.15.43:0/1888652188 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff52c014870 con 0x7ff524005140 2023-08-09T13:35:33.528 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:33.523+0000 7ff523fff640 1 -- 172.21.15.43:0/1888652188 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff52c0149d0 con 0x7ff524005140 2023-08-09T13:35:33.528 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:33.523+0000 7ff523fff640 1 -- 172.21.15.43:0/1888652188 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff52c014d80 con 0x7ff524005140 2023-08-09T13:35:33.529 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:33.523+0000 7ff54084b640 1 -- 172.21.15.43:0/1888652188 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff53c143fe0 con 0x7ff524005140 2023-08-09T13:35:33.529 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:33.523+0000 7ff54084b640 1 -- 172.21.15.43:0/1888652188 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff53c0cc180 con 0x7ff524005140 2023-08-09T13:35:33.529 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:33.525+0000 7ff54084b640 1 -- 172.21.15.43:0/1888652188 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff53c04e390 con 0x7ff524005140 2023-08-09T13:35:33.530 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:33.525+0000 7ff523fff640 1 -- 172.21.15.43:0/1888652188 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 5) v1 ==== 88341+0+0 (secure 0 0 0) 0x7ff52c015410 con 0x7ff524005140 2023-08-09T13:35:33.530 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:33.526+0000 7ff523fff640 1 --2- 172.21.15.43:0/1888652188 >> [v2:172.21.15.203:6832/830097474,v1:172.21.15.203:6833/830097474] conn(0x7ff52806a0d0 0x7ff52806c590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:35:33.530 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:33.526+0000 7ff523fff640 1 -- 172.21.15.43:0/1888652188 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 6871+0+0 (secure 0 0 0) 0x7ff52c089240 con 0x7ff524005140 2023-08-09T13:35:33.530 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:33.527+0000 7ff53affd640 1 --2- 172.21.15.43:0/1888652188 >> [v2:172.21.15.203:6832/830097474,v1:172.21.15.203:6833/830097474] conn(0x7ff52806a0d0 0x7ff52806c590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:35:33.530 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:33.528+0000 7ff53affd640 1 --2- 172.21.15.43:0/1888652188 >> [v2:172.21.15.203:6832/830097474,v1:172.21.15.203:6833/830097474] conn(0x7ff52806a0d0 0x7ff52806c590 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7ff530006e80 tx=0x7ff530006e10 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-09T13:35:33.533 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:33.530+0000 7ff523fff640 1 -- 172.21.15.43:0/1888652188 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178066 (secure 0 0 0) 0x7ff52c054690 con 0x7ff524005140 2023-08-09T13:35:33.726 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:33.724+0000 7ff54084b640 1 -- 172.21.15.43:0/1888652188 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"} v 0) v1 -- 0x7ff53c0cea30 con 0x7ff524005140 2023-08-09T13:35:33.727 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:33.725+0000 7ff523fff640 1 -- 172.21.15.43:0/1888652188 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"}]=0 v26) v1 ==== 92+0+10 (secure 0 0 0) 0x7ff52c057cb0 con 0x7ff524005140 2023-08-09T13:35:33.728 INFO:teuthology.orchestra.run.smithi043.stdout:pg_num: 8 2023-08-09T13:35:33.730 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:33.728+0000 7ff54084b640 1 -- 172.21.15.43:0/1888652188 >> [v2:172.21.15.203:6832/830097474,v1:172.21.15.203:6833/830097474] conn(0x7ff52806a0d0 msgr2=0x7ff52806c590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:35:33.730 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:33.728+0000 7ff54084b640 1 --2- 172.21.15.43:0/1888652188 >> [v2:172.21.15.203:6832/830097474,v1:172.21.15.203:6833/830097474] conn(0x7ff52806a0d0 0x7ff52806c590 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7ff530006e80 tx=0x7ff530006e10 comp rx=0 tx=0).stop 2023-08-09T13:35:33.731 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:33.728+0000 7ff54084b640 1 -- 172.21.15.43:0/1888652188 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff524005140 msgr2=0x7ff53c144ac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:35:33.731 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:33.728+0000 7ff54084b640 1 --2- 172.21.15.43:0/1888652188 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff524005140 0x7ff53c144ac0 secure :-1 s=READY pgs=388 cs=0 l=1 rev1=1 crypto rx=0x7ff52c00f580 tx=0x7ff52c00fa40 comp rx=0 tx=0).stop 2023-08-09T13:35:33.731 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:33.728+0000 7ff54084b640 1 -- 172.21.15.43:0/1888652188 shutdown_connections 2023-08-09T13:35:33.731 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:33.728+0000 7ff54084b640 1 --2- 172.21.15.43:0/1888652188 >> [v2:172.21.15.203:6832/830097474,v1:172.21.15.203:6833/830097474] conn(0x7ff52806a0d0 0x7ff52806c590 unknown :-1 s=CLOSED pgs=200 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:35:33.731 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:33.728+0000 7ff54084b640 1 --2- 172.21.15.43:0/1888652188 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7ff53c151250 0x7ff53c0c2690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:35:33.731 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:33.728+0000 7ff54084b640 1 --2- 172.21.15.43:0/1888652188 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7ff53c0c1df0 0x7ff53c0cbc40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:35:33.731 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:33.728+0000 7ff54084b640 1 --2- 172.21.15.43:0/1888652188 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff524005140 0x7ff53c144ac0 unknown :-1 s=CLOSED pgs=388 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:35:33.732 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:33.728+0000 7ff54084b640 1 -- 172.21.15.43:0/1888652188 >> 172.21.15.43:0/1888652188 conn(0x7ff53c0cddd0 msgr2=0x7ff53c06b810 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-09T13:35:33.732 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:33.728+0000 7ff54084b640 1 -- 172.21.15.43:0/1888652188 shutdown_connections 2023-08-09T13:35:33.732 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:33.728+0000 7ff54084b640 1 -- 172.21.15.43:0/1888652188 wait complete. 2023-08-09T13:35:33.749 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-08-09T13:35:33.983 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:33.980+0000 7fc9d438b640 1 Processor -- start 2023-08-09T13:35:34.315 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:33.980+0000 7fc9d438b640 1 -- start start 2023-08-09T13:35:34.319 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:33.982+0000 7fc9d438b640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7fc9cc15c620 0x7fc9cc0cceb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:35:34.320 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:33.982+0000 7fc9d438b640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fc9cc0cf510 0x7fc9cc0d0350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:35:34.320 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:33.982+0000 7fc9d438b640 1 -- --> v1:172.21.15.203:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc9cc0cd700 con 0x7fc9cc0b1260 2023-08-09T13:35:34.320 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:33.982+0000 7fc9d438b640 1 -- --> v2:172.21.15.43:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc9cc0b1b70 con 0x7fc9cc15c620 2023-08-09T13:35:34.320 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:33.982+0000 7fc9d438b640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc9cc0cd9e0 con 0x7fc9cc0cf510 2023-08-09T13:35:34.320 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:33.982+0000 7fc9d2127640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7fc9cc15c620 0x7fc9cc0cceb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:35:34.320 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:33.982+0000 7fc9d2127640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7fc9cc15c620 0x7fc9cc0cceb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:47174/0 (socket says 172.21.15.43:47174) 2023-08-09T13:35:34.321 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:33.982+0000 7fc9d2127640 1 -- 172.21.15.43:0/3473960151 learned_addr learned my addr 172.21.15.43:0/3473960151 (peer_addr_for_me v2:172.21.15.43:0/0) 2023-08-09T13:35:34.321 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:33.983+0000 7fc9d2928640 1 --2- 172.21.15.43:0/3473960151 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fc9cc0cf510 0x7fc9cc0d0350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:35:34.321 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:33.983+0000 7fc9d2127640 1 -- 172.21.15.43:0/3473960151 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fc9cc0cf510 msgr2=0x7fc9cc0d0350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-09T13:35:34.321 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:33.983+0000 7fc9d2127640 1 --2- 172.21.15.43:0/3473960151 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fc9cc0cf510 0x7fc9cc0d0350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:35:34.321 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:33.983+0000 7fc9d2127640 1 -- 172.21.15.43:0/3473960151 >> v1:172.21.15.203:6789/0 conn(0x7fc9cc0b1260 legacy=0x7fc9cc0cee00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-09T13:35:34.321 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:33.983+0000 7fc9d2127640 1 -- 172.21.15.43:0/3473960151 --> v2:172.21.15.43:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc9cc0cdcc0 con 0x7fc9cc15c620 2023-08-09T13:35:34.321 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:33.983+0000 7fc9d2928640 1 --2- 172.21.15.43:0/3473960151 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fc9cc0cf510 0x7fc9cc0d0350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-08-09T13:35:34.322 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:33.984+0000 7fc9d2127640 1 --2- 172.21.15.43:0/3473960151 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc9cc15c620 0x7fc9cc0cceb0 secure :-1 s=READY pgs=389 cs=0 l=1 rev1=1 crypto rx=0x7fc9cc08a6d0 tx=0x7fc9c802b790 comp rx=0 tx=0).ready entity=mon.0 client_cookie=191a7cbc4f7cdf64 server_cookie=0 in_seq=0 out_seq=0 2023-08-09T13:35:34.322 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:33.984+0000 7fc9d1125640 1 -- 172.21.15.43:0/3473960151 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc9c8004230 con 0x7fc9cc15c620 2023-08-09T13:35:34.322 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:33.985+0000 7fc9d1125640 1 -- 172.21.15.43:0/3473960151 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc9c8004390 con 0x7fc9cc15c620 2023-08-09T13:35:34.322 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:33.985+0000 7fc9d1125640 1 -- 172.21.15.43:0/3473960151 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc9c8003810 con 0x7fc9cc15c620 2023-08-09T13:35:34.322 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:33.985+0000 7fc9d438b640 1 -- 172.21.15.43:0/3473960151 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc9cc15c620 msgr2=0x7fc9cc0cceb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:35:34.322 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:33.985+0000 7fc9d438b640 1 --2- 172.21.15.43:0/3473960151 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc9cc15c620 0x7fc9cc0cceb0 secure :-1 s=READY pgs=389 cs=0 l=1 rev1=1 crypto rx=0x7fc9cc08a6d0 tx=0x7fc9c802b790 comp rx=0 tx=0).stop 2023-08-09T13:35:34.322 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:33.985+0000 7fc9d438b640 1 -- 172.21.15.43:0/3473960151 shutdown_connections 2023-08-09T13:35:34.323 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:33.985+0000 7fc9d438b640 1 --2- 172.21.15.43:0/3473960151 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fc9cc0cf510 0x7fc9cc0d0350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:35:34.323 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:33.985+0000 7fc9d438b640 1 --2- 172.21.15.43:0/3473960151 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc9cc15c620 0x7fc9cc0cceb0 unknown :-1 s=CLOSED pgs=389 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:35:34.323 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:33.985+0000 7fc9d438b640 1 -- 172.21.15.43:0/3473960151 >> 172.21.15.43:0/3473960151 conn(0x7fc9cc09b750 msgr2=0x7fc9cc0d2410 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-09T13:35:34.323 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:33.985+0000 7fc9d438b640 1 -- 172.21.15.43:0/3473960151 shutdown_connections 2023-08-09T13:35:34.323 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:33.985+0000 7fc9d438b640 1 -- 172.21.15.43:0/3473960151 wait complete. 2023-08-09T13:35:34.323 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:33.987+0000 7fc9d438b640 1 Processor -- start 2023-08-09T13:35:34.323 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:33.987+0000 7fc9d438b640 1 -- start start 2023-08-09T13:35:34.324 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:33.988+0000 7fc9d438b640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fc9cc0b1260 0x7fc9cc156510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:35:34.324 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:33.988+0000 7fc9d438b640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc9cc0cf510 0x7fc9cc0c9d60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:35:34.324 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:33.988+0000 7fc9d438b640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fc9cc15c620 0x7fc9cc155c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:35:34.324 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:33.988+0000 7fc9d438b640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc9cc0ca2a0 con 0x7fc9cc0cf510 2023-08-09T13:35:34.324 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:33.988+0000 7fc9d438b640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc9cc0b0940 con 0x7fc9cc0b1260 2023-08-09T13:35:34.324 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:33.988+0000 7fc9d438b640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc9cc0afb80 con 0x7fc9cc15c620 2023-08-09T13:35:34.324 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:33.988+0000 7fc9d1926640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc9cc0cf510 0x7fc9cc0c9d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:35:34.325 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:33.989+0000 7fc9d1926640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc9cc0cf510 0x7fc9cc0c9d60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:47190/0 (socket says 172.21.15.43:47190) 2023-08-09T13:35:34.325 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:33.989+0000 7fc9d1926640 1 -- 172.21.15.43:0/3983542330 learned_addr learned my addr 172.21.15.43:0/3983542330 (peer_addr_for_me v2:172.21.15.43:0/0) 2023-08-09T13:35:34.325 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:33.989+0000 7fc9d1926640 1 -- 172.21.15.43:0/3983542330 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fc9cc15c620 msgr2=0x7fc9cc155c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:35:34.325 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:33.989+0000 7fc9d1926640 1 --2- 172.21.15.43:0/3983542330 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fc9cc15c620 0x7fc9cc155c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:35:34.325 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:33.989+0000 7fc9d1926640 1 -- 172.21.15.43:0/3983542330 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fc9cc0b1260 msgr2=0x7fc9cc156510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:35:34.325 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:33.989+0000 7fc9d1926640 1 --2- 172.21.15.43:0/3983542330 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fc9cc0b1260 0x7fc9cc156510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:35:34.325 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:33.989+0000 7fc9d1926640 1 -- 172.21.15.43:0/3983542330 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc9c8009460 con 0x7fc9cc0cf510 2023-08-09T13:35:34.326 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:33.989+0000 7fc9d1926640 1 --2- 172.21.15.43:0/3983542330 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc9cc0cf510 0x7fc9cc0c9d60 secure :-1 s=READY pgs=390 cs=0 l=1 rev1=1 crypto rx=0x7fc9bc00b3f0 tx=0x7fc9bc00b8b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-09T13:35:34.326 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:33.990+0000 7fc9c2ffd640 1 -- 172.21.15.43:0/3983542330 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc9bc011830 con 0x7fc9cc0cf510 2023-08-09T13:35:34.326 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:33.990+0000 7fc9c2ffd640 1 -- 172.21.15.43:0/3983542330 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc9bc011990 con 0x7fc9cc0cf510 2023-08-09T13:35:34.326 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:33.990+0000 7fc9c2ffd640 1 -- 172.21.15.43:0/3983542330 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc9bc011d40 con 0x7fc9cc0cf510 2023-08-09T13:35:34.326 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:33.990+0000 7fc9d438b640 1 -- 172.21.15.43:0/3983542330 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc9cc0afe60 con 0x7fc9cc0cf510 2023-08-09T13:35:34.326 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:33.990+0000 7fc9d438b640 1 -- 172.21.15.43:0/3983542330 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc9cc0b0420 con 0x7fc9cc0cf510 2023-08-09T13:35:34.327 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:33.991+0000 7fc9d438b640 1 -- 172.21.15.43:0/3983542330 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc9cc0b0640 con 0x7fc9cc0cf510 2023-08-09T13:35:34.327 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:33.992+0000 7fc9c2ffd640 1 -- 172.21.15.43:0/3983542330 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 5) v1 ==== 88341+0+0 (secure 0 0 0) 0x7fc9bc01a400 con 0x7fc9cc0cf510 2023-08-09T13:35:34.327 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:33.994+0000 7fc9c2ffd640 1 --2- 172.21.15.43:0/3983542330 >> [v2:172.21.15.203:6832/830097474,v1:172.21.15.203:6833/830097474] conn(0x7fc9b006a050 0x7fc9b006c510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:35:34.327 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:33.994+0000 7fc9c2ffd640 1 -- 172.21.15.43:0/3983542330 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 6871+0+0 (secure 0 0 0) 0x7fc9bc0887d0 con 0x7fc9cc0cf510 2023-08-09T13:35:34.327 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:33.995+0000 7fc9d2127640 1 --2- 172.21.15.43:0/3983542330 >> [v2:172.21.15.203:6832/830097474,v1:172.21.15.203:6833/830097474] conn(0x7fc9b006a050 0x7fc9b006c510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:35:34.327 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:33.996+0000 7fc9d2127640 1 --2- 172.21.15.43:0/3983542330 >> [v2:172.21.15.203:6832/830097474,v1:172.21.15.203:6833/830097474] conn(0x7fc9b006a050 0x7fc9b006c510 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7fc9c8035000 tx=0x7fc9c8030000 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-09T13:35:34.327 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:33.998+0000 7fc9c2ffd640 1 -- 172.21.15.43:0/3983542330 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178066 (secure 0 0 0) 0x7fc9bc053af0 con 0x7fc9cc0cf510 2023-08-09T13:35:34.328 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:34.202+0000 7fc9d438b640 1 -- 172.21.15.43:0/3983542330 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7fc9cc0739f0 con 0x7fc9cc0cf510 2023-08-09T13:35:34.328 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:34.203+0000 7fc9c2ffd640 1 -- 172.21.15.43:0/3983542330 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v26) v1 ==== 93+0+10 (secure 0 0 0) 0x7fc9bc057110 con 0x7fc9cc0cf510 2023-08-09T13:35:34.328 INFO:teuthology.orchestra.run.smithi043.stdout:pg_num: 1 2023-08-09T13:35:34.330 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:34.206+0000 7fc9d438b640 1 -- 172.21.15.43:0/3983542330 >> [v2:172.21.15.203:6832/830097474,v1:172.21.15.203:6833/830097474] conn(0x7fc9b006a050 msgr2=0x7fc9b006c510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:35:34.330 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:34.206+0000 7fc9d438b640 1 --2- 172.21.15.43:0/3983542330 >> [v2:172.21.15.203:6832/830097474,v1:172.21.15.203:6833/830097474] conn(0x7fc9b006a050 0x7fc9b006c510 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7fc9c8035000 tx=0x7fc9c8030000 comp rx=0 tx=0).stop 2023-08-09T13:35:34.330 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:34.206+0000 7fc9d438b640 1 -- 172.21.15.43:0/3983542330 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc9cc0cf510 msgr2=0x7fc9cc0c9d60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:35:34.330 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:34.206+0000 7fc9d438b640 1 --2- 172.21.15.43:0/3983542330 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc9cc0cf510 0x7fc9cc0c9d60 secure :-1 s=READY pgs=390 cs=0 l=1 rev1=1 crypto rx=0x7fc9bc00b3f0 tx=0x7fc9bc00b8b0 comp rx=0 tx=0).stop 2023-08-09T13:35:34.330 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:34.206+0000 7fc9d438b640 1 -- 172.21.15.43:0/3983542330 shutdown_connections 2023-08-09T13:35:34.330 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:34.206+0000 7fc9d438b640 1 --2- 172.21.15.43:0/3983542330 >> [v2:172.21.15.203:6832/830097474,v1:172.21.15.203:6833/830097474] conn(0x7fc9b006a050 0x7fc9b006c510 unknown :-1 s=CLOSED pgs=201 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:35:34.331 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:34.206+0000 7fc9d438b640 1 --2- 172.21.15.43:0/3983542330 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fc9cc15c620 0x7fc9cc155c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:35:34.331 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:34.206+0000 7fc9d438b640 1 --2- 172.21.15.43:0/3983542330 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc9cc0cf510 0x7fc9cc0c9d60 unknown :-1 s=CLOSED pgs=390 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:35:34.331 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:34.206+0000 7fc9d438b640 1 --2- 172.21.15.43:0/3983542330 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fc9cc0b1260 0x7fc9cc156510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:35:34.331 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:34.206+0000 7fc9d438b640 1 -- 172.21.15.43:0/3983542330 >> 172.21.15.43:0/3983542330 conn(0x7fc9cc09b750 msgr2=0x7fc9cc0b1d00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-09T13:35:34.331 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:34.206+0000 7fc9d438b640 1 -- 172.21.15.43:0/3983542330 shutdown_connections 2023-08-09T13:35:34.331 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:34.206+0000 7fc9d438b640 1 -- 172.21.15.43:0/3983542330 wait complete. 2023-08-09T13:35:34.332 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-08-09T13:35:34.572 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:34.570+0000 7fa558196640 1 Processor -- start 2023-08-09T13:35:34.573 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:34.570+0000 7fa558196640 1 -- start start 2023-08-09T13:35:34.574 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:34.572+0000 7fa558196640 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7fa5500ccf20 0x7fa5500cd550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:35:34.575 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:34.572+0000 7fa558196640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fa5500cda90 0x7fa55015ca10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:35:34.575 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:34.572+0000 7fa558196640 1 -- --> v1:172.21.15.203:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa5500cdee0 con 0x7fa55015c630 2023-08-09T13:35:34.575 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:34.572+0000 7fa558196640 1 -- --> v2:172.21.15.203:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa5500ac600 con 0x7fa5500ccf20 2023-08-09T13:35:34.575 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:34.572+0000 7fa558196640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa5500cbc70 con 0x7fa5500cda90 2023-08-09T13:35:34.576 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:34.573+0000 7fa556733640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fa5500cda90 0x7fa55015ca10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:35:34.576 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:34.573+0000 7fa556733640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fa5500cda90 0x7fa55015ca10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.203:3301/0 says I am v2:172.21.15.43:54836/0 (socket says 172.21.15.43:54836) 2023-08-09T13:35:34.576 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:34.573+0000 7fa556733640 1 -- 172.21.15.43:0/2933816907 learned_addr learned my addr 172.21.15.43:0/2933816907 (peer_addr_for_me v2:172.21.15.43:0/0) 2023-08-09T13:35:34.576 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:34.573+0000 7fa555f32640 1 --1- 172.21.15.43:0/2933816907 >> v1:172.21.15.203:6789/0 conn(0x7fa55015c630 0x7fa5500abef0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.203:6789/0 says I am v1:172.21.15.43:35448/0 (socket says 172.21.15.43:35448) 2023-08-09T13:35:34.577 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:34.573+0000 7fa555731640 1 --2- 172.21.15.43:0/2933816907 >> v2:172.21.15.203:3300/0 conn(0x7fa5500ccf20 0x7fa5500cd550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:35:34.577 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:34.574+0000 7fa554f30640 1 -- 172.21.15.43:0/2933816907 <== mon.1 v1:172.21.15.203:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2288543617 0 0) 0x7fa54c001110 con 0x7fa55015c630 2023-08-09T13:35:34.578 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:34.574+0000 7fa554f30640 1 -- 172.21.15.43:0/2933816907 --> v1:172.21.15.203:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fa53c003520 con 0x7fa55015c630 2023-08-09T13:35:34.578 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:34.574+0000 7fa556733640 1 -- 172.21.15.43:0/2933816907 >> v2:172.21.15.203:3300/0 conn(0x7fa5500ccf20 msgr2=0x7fa5500cd550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-09T13:35:34.578 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:34.574+0000 7fa556733640 1 --2- 172.21.15.43:0/2933816907 >> v2:172.21.15.203:3300/0 conn(0x7fa5500ccf20 0x7fa5500cd550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:35:34.578 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:34.574+0000 7fa556733640 1 -- 172.21.15.43:0/2933816907 >> v1:172.21.15.203:6789/0 conn(0x7fa55015c630 legacy=0x7fa5500abef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:35:34.578 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:34.574+0000 7fa556733640 1 -- 172.21.15.43:0/2933816907 --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa5500cca90 con 0x7fa5500cda90 2023-08-09T13:35:34.578 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:34.575+0000 7fa556733640 1 --2- 172.21.15.43:0/2933816907 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fa5500cda90 0x7fa55015ca10 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7fa548009e50 tx=0x7fa54802bf60 comp rx=0 tx=0).ready entity=mon.2 client_cookie=3c2b845d2950dc8c server_cookie=0 in_seq=0 out_seq=0 2023-08-09T13:35:34.579 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:34.576+0000 7fa554f30640 1 -- 172.21.15.43:0/2933816907 <== mon.2 v2:172.21.15.203:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa54802cb00 con 0x7fa5500cda90 2023-08-09T13:35:34.579 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:34.576+0000 7fa554f30640 1 -- 172.21.15.43:0/2933816907 <== mon.2 v2:172.21.15.203:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa54802cc60 con 0x7fa5500cda90 2023-08-09T13:35:34.580 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:34.576+0000 7fa554f30640 1 -- 172.21.15.43:0/2933816907 <== mon.2 v2:172.21.15.203:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa5480343f0 con 0x7fa5500cda90 2023-08-09T13:35:34.580 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:34.577+0000 7fa558196640 1 -- 172.21.15.43:0/2933816907 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fa5500cda90 msgr2=0x7fa55015ca10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:35:34.580 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:34.577+0000 7fa558196640 1 --2- 172.21.15.43:0/2933816907 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fa5500cda90 0x7fa55015ca10 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7fa548009e50 tx=0x7fa54802bf60 comp rx=0 tx=0).stop 2023-08-09T13:35:34.580 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:34.577+0000 7fa558196640 1 -- 172.21.15.43:0/2933816907 shutdown_connections 2023-08-09T13:35:34.581 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:34.577+0000 7fa558196640 1 --2- 172.21.15.43:0/2933816907 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fa5500cda90 0x7fa55015ca10 unknown :-1 s=CLOSED pgs=67 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:35:34.581 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:34.577+0000 7fa558196640 1 --2- 172.21.15.43:0/2933816907 >> v2:172.21.15.203:3300/0 conn(0x7fa5500ccf20 0x7fa5500cd550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:35:34.581 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:34.577+0000 7fa558196640 1 -- 172.21.15.43:0/2933816907 >> 172.21.15.43:0/2933816907 conn(0x7fa5500ce0a0 msgr2=0x7fa5500ce480 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-09T13:35:34.581 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:34.577+0000 7fa558196640 1 -- 172.21.15.43:0/2933816907 shutdown_connections 2023-08-09T13:35:34.581 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:34.577+0000 7fa558196640 1 -- 172.21.15.43:0/2933816907 wait complete. 2023-08-09T13:35:34.582 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:34.579+0000 7fa558196640 1 Processor -- start 2023-08-09T13:35:34.582 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:34.580+0000 7fa558196640 1 -- start start 2023-08-09T13:35:34.584 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:34.581+0000 7fa558196640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa5500ccf20 0x7fa55015ca10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:35:34.584 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:34.581+0000 7fa558196640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fa5500cda90 0x7fa5501526b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:35:34.585 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:34.581+0000 7fa558196640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fa55015c630 0x7fa5500af4a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:35:34.585 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:34.581+0000 7fa558196640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa5500c62f0 con 0x7fa5500ccf20 2023-08-09T13:35:34.585 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:34.581+0000 7fa558196640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa55015cff0 con 0x7fa55015c630 2023-08-09T13:35:34.585 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:34.581+0000 7fa558196640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa55015d2d0 con 0x7fa5500cda90 2023-08-09T13:35:34.585 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:34.581+0000 7fa555f32640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa5500ccf20 0x7fa55015ca10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:35:34.585 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:34.581+0000 7fa555f32640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa5500ccf20 0x7fa55015ca10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:47194/0 (socket says 172.21.15.43:47194) 2023-08-09T13:35:34.586 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:34.581+0000 7fa555f32640 1 -- 172.21.15.43:0/3768974197 learned_addr learned my addr 172.21.15.43:0/3768974197 (peer_addr_for_me v2:172.21.15.43:0/0) 2023-08-09T13:35:34.586 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:34.582+0000 7fa555f32640 1 -- 172.21.15.43:0/3768974197 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fa5500cda90 msgr2=0x7fa5501526b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:35:34.586 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:34.582+0000 7fa555f32640 1 --2- 172.21.15.43:0/3768974197 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fa5500cda90 0x7fa5501526b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:35:34.586 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:34.582+0000 7fa556733640 1 --2- 172.21.15.43:0/3768974197 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fa55015c630 0x7fa5500af4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:35:34.586 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:34.582+0000 7fa555f32640 1 -- 172.21.15.43:0/3768974197 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fa55015c630 msgr2=0x7fa5500af4a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:35:34.586 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:34.582+0000 7fa555f32640 1 --2- 172.21.15.43:0/3768974197 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fa55015c630 0x7fa5500af4a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:35:34.587 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:34.582+0000 7fa555f32640 1 -- 172.21.15.43:0/3768974197 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa548009b30 con 0x7fa5500ccf20 2023-08-09T13:35:34.588 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:34.582+0000 7fa555f32640 1 --2- 172.21.15.43:0/3768974197 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa5500ccf20 0x7fa55015ca10 secure :-1 s=READY pgs=391 cs=0 l=1 rev1=1 crypto rx=0x7fa54c00c700 tx=0x7fa54c00cb00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-09T13:35:34.588 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:34.583+0000 7fa5467fc640 1 -- 172.21.15.43:0/3768974197 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa54c007480 con 0x7fa5500ccf20 2023-08-09T13:35:34.588 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:34.583+0000 7fa5467fc640 1 -- 172.21.15.43:0/3768974197 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa54c0075e0 con 0x7fa5500ccf20 2023-08-09T13:35:34.588 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:34.583+0000 7fa558196640 1 -- 172.21.15.43:0/3768974197 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa55015d5b0 con 0x7fa5500ccf20 2023-08-09T13:35:34.588 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:34.583+0000 7fa5467fc640 1 -- 172.21.15.43:0/3768974197 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa54c004a20 con 0x7fa5500ccf20 2023-08-09T13:35:34.589 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:34.583+0000 7fa558196640 1 -- 172.21.15.43:0/3768974197 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa5500af9e0 con 0x7fa5500ccf20 2023-08-09T13:35:34.589 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:34.584+0000 7fa558196640 1 -- 172.21.15.43:0/3768974197 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa5500739f0 con 0x7fa5500ccf20 2023-08-09T13:35:34.589 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:34.585+0000 7fa5467fc640 1 -- 172.21.15.43:0/3768974197 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 5) v1 ==== 88341+0+0 (secure 0 0 0) 0x7fa54c004b80 con 0x7fa5500ccf20 2023-08-09T13:35:34.589 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:34.586+0000 7fa5467fc640 1 --2- 172.21.15.43:0/3768974197 >> [v2:172.21.15.203:6832/830097474,v1:172.21.15.203:6833/830097474] conn(0x7fa53406a050 0x7fa53406c510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:35:34.590 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:34.586+0000 7fa5467fc640 1 -- 172.21.15.43:0/3768974197 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 6871+0+0 (secure 0 0 0) 0x7fa54c087c50 con 0x7fa5500ccf20 2023-08-09T13:35:34.590 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:34.587+0000 7fa555731640 1 --2- 172.21.15.43:0/3768974197 >> [v2:172.21.15.203:6832/830097474,v1:172.21.15.203:6833/830097474] conn(0x7fa53406a050 0x7fa53406c510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:35:34.590 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:34.588+0000 7fa555731640 1 --2- 172.21.15.43:0/3768974197 >> [v2:172.21.15.203:6832/830097474,v1:172.21.15.203:6833/830097474] conn(0x7fa53406a050 0x7fa53406c510 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7fa540006a90 tx=0x7fa540006a20 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-09T13:35:34.592 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:34.590+0000 7fa5467fc640 1 -- 172.21.15.43:0/3768974197 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178066 (secure 0 0 0) 0x7fa54c053000 con 0x7fa5500ccf20 2023-08-09T13:35:34.793 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:34.790+0000 7fa558196640 1 -- 172.21.15.43:0/3768974197 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7fa5500cdee0 con 0x7fa5500ccf20 2023-08-09T13:35:34.794 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:34.791+0000 7fa5467fc640 1 -- 172.21.15.43:0/3768974197 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v26) v1 ==== 104+0+11 (secure 0 0 0) 0x7fa5500cdee0 con 0x7fa5500ccf20 2023-08-09T13:35:34.794 INFO:teuthology.orchestra.run.smithi043.stdout:pg_num: 32 2023-08-09T13:35:34.796 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:34.794+0000 7fa558196640 1 -- 172.21.15.43:0/3768974197 >> [v2:172.21.15.203:6832/830097474,v1:172.21.15.203:6833/830097474] conn(0x7fa53406a050 msgr2=0x7fa53406c510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:35:34.797 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:34.794+0000 7fa558196640 1 --2- 172.21.15.43:0/3768974197 >> [v2:172.21.15.203:6832/830097474,v1:172.21.15.203:6833/830097474] conn(0x7fa53406a050 0x7fa53406c510 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7fa540006a90 tx=0x7fa540006a20 comp rx=0 tx=0).stop 2023-08-09T13:35:34.797 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:34.794+0000 7fa558196640 1 -- 172.21.15.43:0/3768974197 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa5500ccf20 msgr2=0x7fa55015ca10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:35:34.797 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:34.794+0000 7fa558196640 1 --2- 172.21.15.43:0/3768974197 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa5500ccf20 0x7fa55015ca10 secure :-1 s=READY pgs=391 cs=0 l=1 rev1=1 crypto rx=0x7fa54c00c700 tx=0x7fa54c00cb00 comp rx=0 tx=0).stop 2023-08-09T13:35:34.797 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:34.794+0000 7fa558196640 1 -- 172.21.15.43:0/3768974197 shutdown_connections 2023-08-09T13:35:34.797 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:34.794+0000 7fa558196640 1 --2- 172.21.15.43:0/3768974197 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fa55015c630 0x7fa5500af4a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:35:34.797 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:34.794+0000 7fa558196640 1 --2- 172.21.15.43:0/3768974197 >> [v2:172.21.15.203:6832/830097474,v1:172.21.15.203:6833/830097474] conn(0x7fa53406a050 0x7fa53406c510 unknown :-1 s=CLOSED pgs=202 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:35:34.797 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:34.794+0000 7fa558196640 1 --2- 172.21.15.43:0/3768974197 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fa5500cda90 0x7fa5501526b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:35:34.798 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:34.794+0000 7fa558196640 1 --2- 172.21.15.43:0/3768974197 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa5500ccf20 0x7fa55015ca10 unknown :-1 s=CLOSED pgs=391 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:35:34.798 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:34.794+0000 7fa558196640 1 -- 172.21.15.43:0/3768974197 >> 172.21.15.43:0/3768974197 conn(0x7fa5500ce0a0 msgr2=0x7fa55006c7d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-09T13:35:34.798 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:34.794+0000 7fa558196640 1 -- 172.21.15.43:0/3768974197 shutdown_connections 2023-08-09T13:35:34.798 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:34.794+0000 7fa558196640 1 -- 172.21.15.43:0/3768974197 wait complete. 2023-08-09T13:35:34.817 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-08-09T13:35:35.051 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:35.048+0000 7f67d266a640 1 Processor -- start 2023-08-09T13:35:35.052 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:35.049+0000 7f67d266a640 1 -- start start 2023-08-09T13:35:35.053 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:35.050+0000 7f67d266a640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f67cc15c550 0x7f67cc0cd6d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:35:35.053 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:35.051+0000 7f67d266a640 1 -- --> v1:172.21.15.43:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f67cc0b7bd0 con 0x7f67cc0b45f0 2023-08-09T13:35:35.053 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:35.051+0000 7f67d266a640 1 -- --> v1:172.21.15.203:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f67cc0d1c00 con 0x7f67cc0d04b0 2023-08-09T13:35:35.054 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:35.051+0000 7f67d266a640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f67cc15c930 con 0x7f67cc15c550 2023-08-09T13:35:35.055 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:35.051+0000 7f67d0c07640 1 --1- >> v1:172.21.15.43:6789/0 conn(0x7f67cc0b45f0 0x7f67cc0b59d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.43:6789/0 says I am v1:172.21.15.43:44850/0 (socket says 172.21.15.43:44850) 2023-08-09T13:35:35.055 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:35.051+0000 7f67d0c07640 1 -- 172.21.15.43:0/3585289441 learned_addr learned my addr 172.21.15.43:0/3585289441 (peer_addr_for_me v1:172.21.15.43:0/0) 2023-08-09T13:35:35.055 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:35.051+0000 7f67cbfff640 1 --2- 172.21.15.43:0/3585289441 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f67cc15c550 0x7f67cc0cd6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:35:35.055 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:35.052+0000 7f67caffd640 1 -- 172.21.15.43:0/3585289441 <== mon.0 v1:172.21.15.43:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2731283709 0 0) 0x7f67cc0b7bd0 con 0x7f67cc0b45f0 2023-08-09T13:35:35.055 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:35.052+0000 7f67caffd640 1 -- 172.21.15.43:0/3585289441 --> v1:172.21.15.43:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f67b4003520 con 0x7f67cc0b45f0 2023-08-09T13:35:35.056 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:35.052+0000 7f67caffd640 1 -- 172.21.15.43:0/3585289441 <== mon.0 v1:172.21.15.43:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2977831606 0 0) 0x7f67b4003520 con 0x7f67cc0b45f0 2023-08-09T13:35:35.056 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:35.052+0000 7f67caffd640 1 -- 172.21.15.43:0/3585289441 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f67cc15c550 msgr2=0x7f67cc0cd6d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-09T13:35:35.056 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:35.052+0000 7f67caffd640 1 --2- 172.21.15.43:0/3585289441 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f67cc15c550 0x7f67cc0cd6d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:35:35.056 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:35.052+0000 7f67caffd640 1 -- 172.21.15.43:0/3585289441 >> v1:172.21.15.203:6789/0 conn(0x7f67cc0d04b0 legacy=0x7f67cc0b3ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:35:35.056 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:35.052+0000 7f67caffd640 1 -- 172.21.15.43:0/3585289441 --> v1:172.21.15.43:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f67cc0b54f0 con 0x7f67cc0b45f0 2023-08-09T13:35:35.056 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:35.052+0000 7f67caffd640 1 -- 172.21.15.43:0/3585289441 <== mon.0 v1:172.21.15.43:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1285150740 0 0) 0x7f67c40019f0 con 0x7f67cc0b45f0 2023-08-09T13:35:35.056 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:35.052+0000 7f67caffd640 1 -- 172.21.15.43:0/3585289441 >> v1:172.21.15.43:6789/0 conn(0x7f67cc0b45f0 legacy=0x7f67cc0b59d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:35:35.057 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:35.052+0000 7f67caffd640 1 --2- 172.21.15.43:0/3585289441 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f67b40039a0 0x7f67b4003dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:35:35.057 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:35.052+0000 7f67caffd640 1 --2- 172.21.15.43:0/3585289441 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f67b40047c0 0x7f67b4004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:35:35.057 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:35.052+0000 7f67caffd640 1 --2- 172.21.15.43:0/3585289441 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f67cc15c550 0x7f67b4005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:35:35.057 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:35.052+0000 7f67caffd640 1 -- 172.21.15.43:0/3585289441 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f67cc15c930 con 0x7f67b40039a0 2023-08-09T13:35:35.057 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:35.052+0000 7f67caffd640 1 -- 172.21.15.43:0/3585289441 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f67b400ba30 con 0x7f67b40047c0 2023-08-09T13:35:35.057 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:35.052+0000 7f67caffd640 1 -- 172.21.15.43:0/3585289441 --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f67b400bbf0 con 0x7f67cc15c550 2023-08-09T13:35:35.057 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:35.053+0000 7f67cbfff640 1 --2- 172.21.15.43:0/3585289441 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f67b40039a0 0x7f67b4003dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:35:35.058 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:35.053+0000 7f67cbfff640 1 -- 172.21.15.43:0/3585289441 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f67cc15c550 msgr2=0x7f67b4005180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-09T13:35:35.059 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:35.053+0000 7f67d0c07640 1 --2- 172.21.15.43:0/3585289441 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f67cc15c550 0x7f67b4005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:35:35.059 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:35.053+0000 7f67cb7fe640 1 --2- 172.21.15.43:0/3585289441 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f67b40047c0 0x7f67b4004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:35:35.059 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:35.053+0000 7f67cbfff640 1 --2- 172.21.15.43:0/3585289441 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f67cc15c550 0x7f67b4005180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:35:35.059 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:35.053+0000 7f67cbfff640 1 -- 172.21.15.43:0/3585289441 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f67b40047c0 msgr2=0x7f67b4004c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-09T13:35:35.059 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:35.053+0000 7f67cbfff640 1 --2- 172.21.15.43:0/3585289441 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f67b40047c0 0x7f67b4004c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:35:35.059 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:35.053+0000 7f67cbfff640 1 -- 172.21.15.43:0/3585289441 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f67b4003520 con 0x7f67b40039a0 2023-08-09T13:35:35.060 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:35.053+0000 7f67cb7fe640 1 --2- 172.21.15.43:0/3585289441 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f67b40047c0 0x7f67b4004c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-08-09T13:35:35.060 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:35.054+0000 7f67cbfff640 1 --2- 172.21.15.43:0/3585289441 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f67b40039a0 0x7f67b4003dc0 secure :-1 s=READY pgs=393 cs=0 l=1 rev1=1 crypto rx=0x7f67bc009a70 tx=0x7f67b40057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f7c19d74318df173 server_cookie=0 in_seq=0 out_seq=0 2023-08-09T13:35:35.060 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:35.055+0000 7f67caffd640 1 -- 172.21.15.43:0/3585289441 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f67bc02c630 con 0x7f67b40039a0 2023-08-09T13:35:35.060 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:35.055+0000 7f67caffd640 1 -- 172.21.15.43:0/3585289441 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f67bc02c790 con 0x7f67b40039a0 2023-08-09T13:35:35.060 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:35.055+0000 7f67caffd640 1 -- 172.21.15.43:0/3585289441 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f67bc02cb40 con 0x7f67b40039a0 2023-08-09T13:35:35.060 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:35.055+0000 7f67d266a640 1 -- 172.21.15.43:0/3585289441 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f67b40039a0 msgr2=0x7f67b4003dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:35:35.060 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:35.055+0000 7f67d266a640 1 --2- 172.21.15.43:0/3585289441 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f67b40039a0 0x7f67b4003dc0 secure :-1 s=READY pgs=393 cs=0 l=1 rev1=1 crypto rx=0x7f67bc009a70 tx=0x7f67b40057f0 comp rx=0 tx=0).stop 2023-08-09T13:35:35.060 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:35.055+0000 7f67d0c07640 1 -- 172.21.15.43:0/3585289441 reap_dead start 2023-08-09T13:35:35.061 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:35.055+0000 7f67d266a640 1 -- 172.21.15.43:0/3585289441 shutdown_connections 2023-08-09T13:35:35.061 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:35.055+0000 7f67d266a640 1 -- 172.21.15.43:0/3585289441 >> 172.21.15.43:0/3585289441 conn(0x7f67cc071520 msgr2=0x7f67cc0ceee0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-09T13:35:35.061 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:35.056+0000 7f67d266a640 1 -- 172.21.15.43:0/3585289441 shutdown_connections 2023-08-09T13:35:35.061 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:35.056+0000 7f67d266a640 1 -- 172.21.15.43:0/3585289441 wait complete. 2023-08-09T13:35:35.062 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:35.057+0000 7f67d266a640 1 Processor -- start 2023-08-09T13:35:35.063 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:35.058+0000 7f67d266a640 1 -- start start 2023-08-09T13:35:35.063 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:35.059+0000 7f67d266a640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f67b40039a0 0x7f67cc15df70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:35:35.063 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:35.059+0000 7f67d266a640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f67cc0cabf0 0x7f67cc153980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:35:35.063 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:35.059+0000 7f67d266a640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f67cc15d080 0x7f67cc1567e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:35:35.063 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:35.059+0000 7f67d266a640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f67cc0cd420 con 0x7f67cc15d080 2023-08-09T13:35:35.063 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:35.059+0000 7f67d266a640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f67cc15e4b0 con 0x7f67cc0cabf0 2023-08-09T13:35:35.063 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:35.059+0000 7f67d266a640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f67cc153ec0 con 0x7f67b40039a0 2023-08-09T13:35:35.064 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:35.059+0000 7f67d0c07640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f67cc15d080 0x7f67cc1567e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:35:35.064 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:35.059+0000 7f67d0c07640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f67cc15d080 0x7f67cc1567e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:49852/0 (socket says 172.21.15.43:49852) 2023-08-09T13:35:35.064 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:35.059+0000 7f67d0c07640 1 -- 172.21.15.43:0/1970524618 learned_addr learned my addr 172.21.15.43:0/1970524618 (peer_addr_for_me v2:172.21.15.43:0/0) 2023-08-09T13:35:35.064 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:35.059+0000 7f67cbfff640 1 --2- 172.21.15.43:0/1970524618 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f67b40039a0 0x7f67cc15df70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:35:35.064 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:35.059+0000 7f67d0c07640 1 -- 172.21.15.43:0/1970524618 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f67b40039a0 msgr2=0x7f67cc15df70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:35:35.064 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:35.059+0000 7f67d0c07640 1 --2- 172.21.15.43:0/1970524618 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f67b40039a0 0x7f67cc15df70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:35:35.064 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:35.060+0000 7f67d0c07640 1 -- 172.21.15.43:0/1970524618 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f67cc0cabf0 msgr2=0x7f67cc153980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:35:35.065 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:35.060+0000 7f67d0c07640 1 --2- 172.21.15.43:0/1970524618 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f67cc0cabf0 0x7f67cc153980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:35:35.065 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:35.060+0000 7f67d0c07640 1 -- 172.21.15.43:0/1970524618 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f67cc15e710 con 0x7f67cc15d080 2023-08-09T13:35:35.065 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:35.060+0000 7f67d0c07640 1 --2- 172.21.15.43:0/1970524618 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f67cc15d080 0x7f67cc1567e0 secure :-1 s=READY pgs=394 cs=0 l=1 rev1=1 crypto rx=0x7f67cc0d01c0 tx=0x7f67c400b4a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-09T13:35:35.065 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:35.060+0000 7f67c8ff9640 1 -- 172.21.15.43:0/1970524618 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f67c400de90 con 0x7f67cc15d080 2023-08-09T13:35:35.065 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:35.060+0000 7f67c8ff9640 1 -- 172.21.15.43:0/1970524618 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f67c400bb80 con 0x7f67cc15d080 2023-08-09T13:35:35.065 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:35.060+0000 7f67c8ff9640 1 -- 172.21.15.43:0/1970524618 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f67c4007110 con 0x7f67cc15d080 2023-08-09T13:35:35.065 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:35.060+0000 7f67d266a640 1 -- 172.21.15.43:0/1970524618 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f67cc15e9f0 con 0x7f67cc15d080 2023-08-09T13:35:35.066 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:35.060+0000 7f67d266a640 1 -- 172.21.15.43:0/1970524618 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f67cc154560 con 0x7f67cc15d080 2023-08-09T13:35:35.066 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:35.062+0000 7f67d266a640 1 -- 172.21.15.43:0/1970524618 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f67cc0739f0 con 0x7f67cc15d080 2023-08-09T13:35:35.066 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:35.063+0000 7f67c8ff9640 1 -- 172.21.15.43:0/1970524618 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 5) v1 ==== 88341+0+0 (secure 0 0 0) 0x7f67c4007270 con 0x7f67cc15d080 2023-08-09T13:35:35.066 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:35.064+0000 7f67c8ff9640 1 --2- 172.21.15.43:0/1970524618 >> [v2:172.21.15.203:6832/830097474,v1:172.21.15.203:6833/830097474] conn(0x7f67b806a050 0x7f67b806c510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:35:35.067 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:35.064+0000 7f67c8ff9640 1 -- 172.21.15.43:0/1970524618 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 6871+0+0 (secure 0 0 0) 0x7f67c4088d20 con 0x7f67cc15d080 2023-08-09T13:35:35.067 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:35.064+0000 7f67cbfff640 1 --2- 172.21.15.43:0/1970524618 >> [v2:172.21.15.203:6832/830097474,v1:172.21.15.203:6833/830097474] conn(0x7f67b806a050 0x7f67b806c510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:35:35.068 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:35.066+0000 7f67cbfff640 1 --2- 172.21.15.43:0/1970524618 >> [v2:172.21.15.203:6832/830097474,v1:172.21.15.203:6833/830097474] conn(0x7f67b806a050 0x7f67b806c510 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7f67bc036000 tx=0x7f67bc030000 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-09T13:35:35.073 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:35.070+0000 7f67c8ff9640 1 -- 172.21.15.43:0/1970524618 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178066 (secure 0 0 0) 0x7f67c40540d0 con 0x7f67cc15d080 2023-08-09T13:35:35.275 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:35.273+0000 7f67d266a640 1 -- 172.21.15.43:0/1970524618 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f67cc0cd890 con 0x7f67cc15d080 2023-08-09T13:35:35.276 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:35.274+0000 7f67c8ff9640 1 -- 172.21.15.43:0/1970524618 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v26) v1 ==== 100+0+11 (secure 0 0 0) 0x7f67c40576f0 con 0x7f67cc15d080 2023-08-09T13:35:35.277 INFO:teuthology.orchestra.run.smithi043.stdout:pg_num: 64 2023-08-09T13:35:35.280 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:35.277+0000 7f67d266a640 1 -- 172.21.15.43:0/1970524618 >> [v2:172.21.15.203:6832/830097474,v1:172.21.15.203:6833/830097474] conn(0x7f67b806a050 msgr2=0x7f67b806c510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:35:35.280 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:35.277+0000 7f67d266a640 1 --2- 172.21.15.43:0/1970524618 >> [v2:172.21.15.203:6832/830097474,v1:172.21.15.203:6833/830097474] conn(0x7f67b806a050 0x7f67b806c510 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7f67bc036000 tx=0x7f67bc030000 comp rx=0 tx=0).stop 2023-08-09T13:35:35.280 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:35.277+0000 7f67d266a640 1 -- 172.21.15.43:0/1970524618 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f67cc15d080 msgr2=0x7f67cc1567e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:35:35.280 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:35.277+0000 7f67d266a640 1 --2- 172.21.15.43:0/1970524618 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f67cc15d080 0x7f67cc1567e0 secure :-1 s=READY pgs=394 cs=0 l=1 rev1=1 crypto rx=0x7f67cc0d01c0 tx=0x7f67c400b4a0 comp rx=0 tx=0).stop 2023-08-09T13:35:35.281 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:35.277+0000 7f67d266a640 1 -- 172.21.15.43:0/1970524618 shutdown_connections 2023-08-09T13:35:35.281 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:35.277+0000 7f67d266a640 1 --2- 172.21.15.43:0/1970524618 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f67cc15d080 0x7f67cc1567e0 unknown :-1 s=CLOSED pgs=394 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:35:35.281 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:35.277+0000 7f67d266a640 1 --2- 172.21.15.43:0/1970524618 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f67cc0cabf0 0x7f67cc153980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:35:35.281 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:35.277+0000 7f67d266a640 1 --2- 172.21.15.43:0/1970524618 >> [v2:172.21.15.203:6832/830097474,v1:172.21.15.203:6833/830097474] conn(0x7f67b806a050 0x7f67b806c510 unknown :-1 s=CLOSED pgs=203 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:35:35.281 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:35.277+0000 7f67d266a640 1 --2- 172.21.15.43:0/1970524618 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f67b40039a0 0x7f67cc15df70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:35:35.281 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:35.277+0000 7f67d266a640 1 -- 172.21.15.43:0/1970524618 >> 172.21.15.43:0/1970524618 conn(0x7f67cc071520 msgr2=0x7f67cc157640 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-09T13:35:35.281 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:35.278+0000 7f67d266a640 1 -- 172.21.15.43:0/1970524618 shutdown_connections 2023-08-09T13:35:35.282 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:35.278+0000 7f67d266a640 1 -- 172.21.15.43:0/1970524618 wait complete. 2023-08-09T13:35:35.301 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2023-08-09T13:35:35.535 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:35.533+0000 7f3701e8c640 1 Processor -- start 2023-08-09T13:35:35.536 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:35.533+0000 7f3701e8c640 1 -- start start 2023-08-09T13:35:35.538 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:35.535+0000 7f3701e8c640 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7f36fc0d0a70 0x7f36fc0cd7f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:35:35.538 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:35.535+0000 7f3701e8c640 1 -- --> v1:172.21.15.43:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f36fc0d0f70 con 0x7f36fc0cdd30 2023-08-09T13:35:35.538 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:35.535+0000 7f3701e8c640 1 -- --> v1:172.21.15.203:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f36fc0ccd10 con 0x7f36fc15c680 2023-08-09T13:35:35.538 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:35.535+0000 7f3701e8c640 1 -- --> v2:172.21.15.203:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f36fc0ccff0 con 0x7f36fc0d0a70 2023-08-09T13:35:35.538 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:35.535+0000 7f36fbfff640 1 --1- >> v1:172.21.15.43:6789/0 conn(0x7f36fc0cdd30 0x7f36fc0b3fa0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.43:6789/0 says I am v1:172.21.15.43:44862/0 (socket says 172.21.15.43:44862) 2023-08-09T13:35:35.538 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:35.535+0000 7f36fbfff640 1 -- 172.21.15.43:0/75227366 learned_addr learned my addr 172.21.15.43:0/75227366 (peer_addr_for_me v1:172.21.15.43:0/0) 2023-08-09T13:35:35.540 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:35.536+0000 7f36f2ffd640 1 --2- 172.21.15.43:0/75227366 >> v2:172.21.15.203:3300/0 conn(0x7f36fc0d0a70 0x7f36fc0cd7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:35:35.540 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:35.536+0000 7f36faffd640 1 -- 172.21.15.43:0/75227366 <== mon.0 v1:172.21.15.43:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1609341330 0 0) 0x7f36fc0d0f70 con 0x7f36fc0cdd30 2023-08-09T13:35:35.540 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:35.536+0000 7f36faffd640 1 -- 172.21.15.43:0/75227366 --> v1:172.21.15.43:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f36e8003520 con 0x7f36fc0cdd30 2023-08-09T13:35:35.540 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:35.536+0000 7f36faffd640 1 -- 172.21.15.43:0/75227366 <== mon.0 v1:172.21.15.43:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3877788200 0 0) 0x7f36e8003520 con 0x7f36fc0cdd30 2023-08-09T13:35:35.540 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:35.536+0000 7f36faffd640 1 -- 172.21.15.43:0/75227366 >> v2:172.21.15.203:3300/0 conn(0x7f36fc0d0a70 msgr2=0x7f36fc0cd7f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-09T13:35:35.540 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:35.536+0000 7f36faffd640 1 --2- 172.21.15.43:0/75227366 >> v2:172.21.15.203:3300/0 conn(0x7f36fc0d0a70 0x7f36fc0cd7f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:35:35.540 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:35.536+0000 7f36faffd640 1 -- 172.21.15.43:0/75227366 >> v1:172.21.15.203:6789/0 conn(0x7f36fc15c680 legacy=0x7f36fc0ce760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:35:35.541 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:35.536+0000 7f36faffd640 1 -- 172.21.15.43:0/75227366 --> v1:172.21.15.43:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f36fc0cd2d0 con 0x7f36fc0cdd30 2023-08-09T13:35:35.541 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:35.536+0000 7f36faffd640 1 -- 172.21.15.43:0/75227366 <== mon.0 v1:172.21.15.43:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1285150740 0 0) 0x7f36f40019f0 con 0x7f36fc0cdd30 2023-08-09T13:35:35.541 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:35.536+0000 7f36faffd640 1 -- 172.21.15.43:0/75227366 >> v1:172.21.15.43:6789/0 conn(0x7f36fc0cdd30 legacy=0x7f36fc0b3fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:35:35.541 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:35.536+0000 7f36faffd640 1 --2- 172.21.15.43:0/75227366 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f36e80039e0 0x7f36e8003e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:35:35.541 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:35.536+0000 7f36faffd640 1 --2- 172.21.15.43:0/75227366 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f36e80047c0 0x7f36e8004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:35:35.541 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:35.536+0000 7f36faffd640 1 --2- 172.21.15.43:0/75227366 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f36e8005180 0x7f36e800b980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:35:35.541 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:35.537+0000 7f36faffd640 1 -- 172.21.15.43:0/75227366 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f36fc0ccff0 con 0x7f36e80047c0 2023-08-09T13:35:35.542 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:35.537+0000 7f36faffd640 1 -- 172.21.15.43:0/75227366 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f36e800c0b0 con 0x7f36e8005180 2023-08-09T13:35:35.542 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:35.537+0000 7f36faffd640 1 -- 172.21.15.43:0/75227366 --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f36e800c270 con 0x7f36e80039e0 2023-08-09T13:35:35.542 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:35.537+0000 7f36f2ffd640 1 --2- 172.21.15.43:0/75227366 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f36e80047c0 0x7f36e8004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:35:35.542 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:35.537+0000 7f36f2ffd640 1 -- 172.21.15.43:0/75227366 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f36e80039e0 msgr2=0x7f36e8003e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-09T13:35:35.542 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:35.537+0000 7f36f2ffd640 1 --2- 172.21.15.43:0/75227366 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f36e80039e0 0x7f36e8003e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:35:35.542 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:35.537+0000 7f36f2ffd640 1 -- 172.21.15.43:0/75227366 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f36e8005180 msgr2=0x7f36e800b980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-09T13:35:35.542 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:35.537+0000 7f36f2ffd640 1 --2- 172.21.15.43:0/75227366 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f36e8005180 0x7f36e800b980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:35:35.542 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:35.537+0000 7f36f2ffd640 1 -- 172.21.15.43:0/75227366 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f36e8003520 con 0x7f36e80047c0 2023-08-09T13:35:35.543 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:35.537+0000 7f36fbfff640 1 -- 172.21.15.43:0/75227366 reap_dead start 2023-08-09T13:35:35.543 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:35.538+0000 7f36f2ffd640 1 --2- 172.21.15.43:0/75227366 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f36e80047c0 0x7f36e8004c40 secure :-1 s=READY pgs=396 cs=0 l=1 rev1=1 crypto rx=0x7f36e4009a90 tx=0x7f36e80057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3db873753a674313 server_cookie=0 in_seq=0 out_seq=0 2023-08-09T13:35:35.544 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:35.539+0000 7f36faffd640 1 -- 172.21.15.43:0/75227366 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f36e402c6b0 con 0x7f36e80047c0 2023-08-09T13:35:35.544 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:35.539+0000 7f36faffd640 1 -- 172.21.15.43:0/75227366 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f36e402c810 con 0x7f36e80047c0 2023-08-09T13:35:35.544 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:35.539+0000 7f36faffd640 1 -- 172.21.15.43:0/75227366 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f36e402cba0 con 0x7f36e80047c0 2023-08-09T13:35:35.544 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:35.539+0000 7f3701e8c640 1 -- 172.21.15.43:0/75227366 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f36e80047c0 msgr2=0x7f36e8004c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:35:35.544 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:35.539+0000 7f3701e8c640 1 --2- 172.21.15.43:0/75227366 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f36e80047c0 0x7f36e8004c40 secure :-1 s=READY pgs=396 cs=0 l=1 rev1=1 crypto rx=0x7f36e4009a90 tx=0x7f36e80057f0 comp rx=0 tx=0).stop 2023-08-09T13:35:35.544 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:35.539+0000 7f3701e8c640 1 -- 172.21.15.43:0/75227366 shutdown_connections 2023-08-09T13:35:35.545 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:35.539+0000 7f3701e8c640 1 --2- 172.21.15.43:0/75227366 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f36e80047c0 0x7f36e8004c40 unknown :-1 s=CLOSED pgs=396 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:35:35.545 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:35.539+0000 7f3701e8c640 1 -- 172.21.15.43:0/75227366 >> 172.21.15.43:0/75227366 conn(0x7f36fc071520 msgr2=0x7f36fc0d2520 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-09T13:35:35.545 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:35.540+0000 7f3701e8c640 1 -- 172.21.15.43:0/75227366 shutdown_connections 2023-08-09T13:35:35.545 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:35.540+0000 7f3701e8c640 1 -- 172.21.15.43:0/75227366 wait complete. 2023-08-09T13:35:35.545 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:35.541+0000 7f3701e8c640 1 Processor -- start 2023-08-09T13:35:35.546 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:35.541+0000 7f3701e8c640 1 -- start start 2023-08-09T13:35:35.547 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:35.542+0000 7f3701e8c640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f36e80047c0 0x7f36fc0c81a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:35:35.547 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:35.542+0000 7f3701e8c640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f36fc1f2300 0x7f36fc153e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:35:35.547 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:35.542+0000 7f3701e8c640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f36fc154380 0x7f36fc15eac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:35:35.547 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:35.542+0000 7f3701e8c640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f36fc0cc3d0 con 0x7f36fc1f2300 2023-08-09T13:35:35.547 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:35.542+0000 7f3701e8c640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f36fc154810 con 0x7f36fc154380 2023-08-09T13:35:35.547 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:35.542+0000 7f3701e8c640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f36fc0c86e0 con 0x7f36e80047c0 2023-08-09T13:35:35.547 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:35.543+0000 7f36f2ffd640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f36fc1f2300 0x7f36fc153e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:35:35.548 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:35.543+0000 7f36f2ffd640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f36fc1f2300 0x7f36fc153e40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:49882/0 (socket says 172.21.15.43:49882) 2023-08-09T13:35:35.548 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:35.543+0000 7f36f2ffd640 1 -- 172.21.15.43:0/2849868859 learned_addr learned my addr 172.21.15.43:0/2849868859 (peer_addr_for_me v2:172.21.15.43:0/0) 2023-08-09T13:35:35.548 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:35.543+0000 7f36fb7fe640 1 --2- 172.21.15.43:0/2849868859 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f36e80047c0 0x7f36fc0c81a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:35:35.548 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:35.543+0000 7f36f2ffd640 1 -- 172.21.15.43:0/2849868859 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f36e80047c0 msgr2=0x7f36fc0c81a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:35:35.548 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:35.543+0000 7f36f2ffd640 1 --2- 172.21.15.43:0/2849868859 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f36e80047c0 0x7f36fc0c81a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:35:35.548 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:35.543+0000 7f36f2ffd640 1 -- 172.21.15.43:0/2849868859 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f36fc154380 msgr2=0x7f36fc15eac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:35:35.548 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:35.543+0000 7f36f2ffd640 1 --2- 172.21.15.43:0/2849868859 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f36fc154380 0x7f36fc15eac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:35:35.548 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:35.543+0000 7f36f2ffd640 1 -- 172.21.15.43:0/2849868859 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f36fc15dd00 con 0x7f36fc1f2300 2023-08-09T13:35:35.549 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:35.544+0000 7f36f2ffd640 1 --2- 172.21.15.43:0/2849868859 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f36fc1f2300 0x7f36fc153e40 secure :-1 s=READY pgs=397 cs=0 l=1 rev1=1 crypto rx=0x7f36e4037040 tx=0x7f36e4004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-09T13:35:35.549 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:35.544+0000 7f36f8ff9640 1 -- 172.21.15.43:0/2849868859 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f36e4003810 con 0x7f36fc1f2300 2023-08-09T13:35:35.549 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:35.544+0000 7f36f8ff9640 1 -- 172.21.15.43:0/2849868859 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f36e4003970 con 0x7f36fc1f2300 2023-08-09T13:35:35.549 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:35.544+0000 7f36f8ff9640 1 -- 172.21.15.43:0/2849868859 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f36e4003d20 con 0x7f36fc1f2300 2023-08-09T13:35:35.549 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:35.544+0000 7f3701e8c640 1 -- 172.21.15.43:0/2849868859 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f36fc15dfe0 con 0x7f36fc1f2300 2023-08-09T13:35:35.549 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:35.544+0000 7f3701e8c640 1 -- 172.21.15.43:0/2849868859 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f36fc15e5a0 con 0x7f36fc1f2300 2023-08-09T13:35:35.550 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:35.545+0000 7f3701e8c640 1 -- 172.21.15.43:0/2849868859 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f36fc0739f0 con 0x7f36fc1f2300 2023-08-09T13:35:35.550 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:35.546+0000 7f36f8ff9640 1 -- 172.21.15.43:0/2849868859 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 5) v1 ==== 88341+0+0 (secure 0 0 0) 0x7f36e4040450 con 0x7f36fc1f2300 2023-08-09T13:35:35.550 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:35.547+0000 7f36f8ff9640 1 --2- 172.21.15.43:0/2849868859 >> [v2:172.21.15.203:6832/830097474,v1:172.21.15.203:6833/830097474] conn(0x7f36dc06a050 0x7f36dc06c510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:35:35.550 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:35.547+0000 7f36f8ff9640 1 -- 172.21.15.43:0/2849868859 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 6871+0+0 (secure 0 0 0) 0x7f36e40aa900 con 0x7f36fc1f2300 2023-08-09T13:35:35.551 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:35.548+0000 7f36fb7fe640 1 --2- 172.21.15.43:0/2849868859 >> [v2:172.21.15.203:6832/830097474,v1:172.21.15.203:6833/830097474] conn(0x7f36dc06a050 0x7f36dc06c510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:35:35.551 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:35.549+0000 7f36fb7fe640 1 --2- 172.21.15.43:0/2849868859 >> [v2:172.21.15.203:6832/830097474,v1:172.21.15.203:6833/830097474] conn(0x7f36dc06a050 0x7f36dc06c510 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7f36fc063390 tx=0x7f36ec009040 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-09T13:35:35.554 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:35.552+0000 7f36f8ff9640 1 -- 172.21.15.43:0/2849868859 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178066 (secure 0 0 0) 0x7f36e4075c20 con 0x7f36fc1f2300 2023-08-09T13:35:35.750 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:35.747+0000 7f3701e8c640 1 -- 172.21.15.43:0/2849868859 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) v1 -- 0x7f36fc15fe60 con 0x7f36fc1f2300 2023-08-09T13:35:35.751 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:35.748+0000 7f36f8ff9640 1 -- 172.21.15.43:0/2849868859 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v26) v1 ==== 103+0+11 (secure 0 0 0) 0x7f36e4079240 con 0x7f36fc1f2300 2023-08-09T13:35:35.751 INFO:teuthology.orchestra.run.smithi043.stdout:pg_num: 32 2023-08-09T13:35:35.754 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:35.751+0000 7f3701e8c640 1 -- 172.21.15.43:0/2849868859 >> [v2:172.21.15.203:6832/830097474,v1:172.21.15.203:6833/830097474] conn(0x7f36dc06a050 msgr2=0x7f36dc06c510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:35:35.754 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:35.751+0000 7f3701e8c640 1 --2- 172.21.15.43:0/2849868859 >> [v2:172.21.15.203:6832/830097474,v1:172.21.15.203:6833/830097474] conn(0x7f36dc06a050 0x7f36dc06c510 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7f36fc063390 tx=0x7f36ec009040 comp rx=0 tx=0).stop 2023-08-09T13:35:35.754 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:35.751+0000 7f3701e8c640 1 -- 172.21.15.43:0/2849868859 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f36fc1f2300 msgr2=0x7f36fc153e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:35:35.754 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:35.751+0000 7f3701e8c640 1 --2- 172.21.15.43:0/2849868859 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f36fc1f2300 0x7f36fc153e40 secure :-1 s=READY pgs=397 cs=0 l=1 rev1=1 crypto rx=0x7f36e4037040 tx=0x7f36e4004730 comp rx=0 tx=0).stop 2023-08-09T13:35:35.755 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:35.751+0000 7f3701e8c640 1 -- 172.21.15.43:0/2849868859 shutdown_connections 2023-08-09T13:35:35.755 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:35.751+0000 7f3701e8c640 1 --2- 172.21.15.43:0/2849868859 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f36fc154380 0x7f36fc15eac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:35:35.755 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:35.751+0000 7f3701e8c640 1 --2- 172.21.15.43:0/2849868859 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f36fc1f2300 0x7f36fc153e40 unknown :-1 s=CLOSED pgs=397 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:35:35.755 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:35.751+0000 7f3701e8c640 1 --2- 172.21.15.43:0/2849868859 >> [v2:172.21.15.203:6832/830097474,v1:172.21.15.203:6833/830097474] conn(0x7f36dc06a050 0x7f36dc06c510 unknown :-1 s=CLOSED pgs=204 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:35:35.755 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:35.751+0000 7f3701e8c640 1 --2- 172.21.15.43:0/2849868859 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f36e80047c0 0x7f36fc0c81a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:35:35.755 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:35.751+0000 7f3701e8c640 1 -- 172.21.15.43:0/2849868859 >> 172.21.15.43:0/2849868859 conn(0x7f36fc071520 msgr2=0x7f36fc0b5650 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-09T13:35:35.755 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:35.752+0000 7f3701e8c640 1 -- 172.21.15.43:0/2849868859 shutdown_connections 2023-08-09T13:35:35.756 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:35.752+0000 7f3701e8c640 1 -- 172.21.15.43:0/2849868859 wait complete. 2023-08-09T13:35:35.774 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2023-08-09T13:35:36.004 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:36.002+0000 7f6afc8ad640 1 Processor -- start 2023-08-09T13:35:36.004 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:36.002+0000 7f6afc8ad640 1 -- start start 2023-08-09T13:35:36.006 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:36.004+0000 7f6afc8ad640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f6af8143c20 0x7f6af806f540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:35:36.006 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:36.004+0000 7f6afc8ad640 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7f6af80c86f0 0x7f6af80c8b50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:35:36.006 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:36.004+0000 7f6afc8ad640 1 -- --> v1:172.21.15.43:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6af80c9090 con 0x7f6af806c8d0 2023-08-09T13:35:36.007 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:36.004+0000 7f6afc8ad640 1 -- --> v2:172.21.15.203:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6af8144000 con 0x7f6af80c86f0 2023-08-09T13:35:36.007 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:36.004+0000 7f6afc8ad640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6af806c440 con 0x7f6af8143c20 2023-08-09T13:35:36.007 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:36.004+0000 7f6af6d9d640 1 --1- >> v1:172.21.15.43:6789/0 conn(0x7f6af806c8d0 0x7f6af806cd60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.43:6789/0 says I am v1:172.21.15.43:44870/0 (socket says 172.21.15.43:44870) 2023-08-09T13:35:36.008 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:36.004+0000 7f6af6d9d640 1 -- 172.21.15.43:0/993131348 learned_addr learned my addr 172.21.15.43:0/993131348 (peer_addr_for_me v1:172.21.15.43:0/0) 2023-08-09T13:35:36.008 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:36.005+0000 7f6af659c640 1 --2- 172.21.15.43:0/993131348 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f6af8143c20 0x7f6af806f540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:35:36.008 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:36.005+0000 7f6af5d9b640 1 --2- 172.21.15.43:0/993131348 >> v2:172.21.15.203:3300/0 conn(0x7f6af80c86f0 0x7f6af80c8b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:35:36.008 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:36.005+0000 7f6af559a640 1 -- 172.21.15.43:0/993131348 <== mon.0 v1:172.21.15.43:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1204968467 0 0) 0x7f6af80c9090 con 0x7f6af806c8d0 2023-08-09T13:35:36.008 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:36.005+0000 7f6af559a640 1 -- 172.21.15.43:0/993131348 --> v1:172.21.15.43:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f6ae0003520 con 0x7f6af806c8d0 2023-08-09T13:35:36.009 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:36.005+0000 7f6af659c640 1 -- 172.21.15.43:0/993131348 >> v2:172.21.15.203:3300/0 conn(0x7f6af80c86f0 msgr2=0x7f6af80c8b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-09T13:35:36.010 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:36.005+0000 7f6af659c640 1 --2- 172.21.15.43:0/993131348 >> v2:172.21.15.203:3300/0 conn(0x7f6af80c86f0 0x7f6af80c8b50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:35:36.010 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:36.005+0000 7f6af659c640 1 -- 172.21.15.43:0/993131348 >> v1:172.21.15.43:6789/0 conn(0x7f6af806c8d0 legacy=0x7f6af806cd60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:35:36.010 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:36.005+0000 7f6af559a640 1 -- 172.21.15.43:0/993131348 <== mon.0 v1:172.21.15.43:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 9580826 0 0) 0x7f6ae0003520 con 0x7f6af806c8d0 2023-08-09T13:35:36.010 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:36.005+0000 7f6af659c640 1 -- 172.21.15.43:0/993131348 --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6af806b830 con 0x7f6af8143c20 2023-08-09T13:35:36.011 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:36.007+0000 7f6af659c640 1 --2- 172.21.15.43:0/993131348 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f6af8143c20 0x7f6af806f540 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f6ae8009d90 tx=0x7f6ae802bce0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=2cdb2b091e63d0c server_cookie=0 in_seq=0 out_seq=0 2023-08-09T13:35:36.011 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:36.008+0000 7f6af559a640 1 -- 172.21.15.43:0/993131348 <== mon.2 v2:172.21.15.203:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6af0001670 con 0x7f6af8143c20 2023-08-09T13:35:36.011 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:36.008+0000 7f6af559a640 1 -- 172.21.15.43:0/993131348 <== mon.2 v2:172.21.15.203:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6ae802c810 con 0x7f6af8143c20 2023-08-09T13:35:36.011 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:36.008+0000 7f6af559a640 1 -- 172.21.15.43:0/993131348 <== mon.2 v2:172.21.15.203:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6ae802cba0 con 0x7f6af8143c20 2023-08-09T13:35:36.011 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:36.008+0000 7f6afc8ad640 1 -- 172.21.15.43:0/993131348 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f6af8143c20 msgr2=0x7f6af806f540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:35:36.012 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:36.008+0000 7f6afc8ad640 1 --2- 172.21.15.43:0/993131348 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f6af8143c20 0x7f6af806f540 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f6ae8009d90 tx=0x7f6ae802bce0 comp rx=0 tx=0).stop 2023-08-09T13:35:36.012 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:36.009+0000 7f6afc8ad640 1 -- 172.21.15.43:0/993131348 shutdown_connections 2023-08-09T13:35:36.012 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:36.009+0000 7f6afc8ad640 1 --2- 172.21.15.43:0/993131348 >> v2:172.21.15.203:3300/0 conn(0x7f6af80c86f0 0x7f6af80c8b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:35:36.013 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:36.009+0000 7f6afc8ad640 1 --2- 172.21.15.43:0/993131348 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f6af8143c20 0x7f6af806f540 unknown :-1 s=CLOSED pgs=68 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:35:36.013 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:36.009+0000 7f6afc8ad640 1 -- 172.21.15.43:0/993131348 >> 172.21.15.43:0/993131348 conn(0x7f6af80ce0a0 msgr2=0x7f6af80ce480 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-09T13:35:36.013 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:36.009+0000 7f6afc8ad640 1 -- 172.21.15.43:0/993131348 shutdown_connections 2023-08-09T13:35:36.013 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:36.009+0000 7f6afc8ad640 1 -- 172.21.15.43:0/993131348 wait complete. 2023-08-09T13:35:36.013 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:36.011+0000 7f6afc8ad640 1 Processor -- start 2023-08-09T13:35:36.014 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:36.011+0000 7f6afc8ad640 1 -- start start 2023-08-09T13:35:36.015 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:36.012+0000 7f6afc8ad640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f6af806c8d0 0x7f6af80c7d60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:35:36.015 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:36.012+0000 7f6afc8ad640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f6af80c86f0 0x7f6af80c5170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:35:36.015 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:36.012+0000 7f6afc8ad640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f6af8143c20 0x7f6af80b21f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:35:36.015 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:36.012+0000 7f6afc8ad640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6af80c7940 con 0x7f6af80c86f0 2023-08-09T13:35:36.015 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:36.012+0000 7f6afc8ad640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6af80c6ad0 con 0x7f6af8143c20 2023-08-09T13:35:36.015 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:36.012+0000 7f6afc8ad640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6af80c56b0 con 0x7f6af806c8d0 2023-08-09T13:35:36.015 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:36.012+0000 7f6af5d9b640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f6af80c86f0 0x7f6af80c5170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:35:36.016 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:36.012+0000 7f6af5d9b640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f6af80c86f0 0x7f6af80c5170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:49884/0 (socket says 172.21.15.43:49884) 2023-08-09T13:35:36.016 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:36.012+0000 7f6af5d9b640 1 -- 172.21.15.43:0/1307931702 learned_addr learned my addr 172.21.15.43:0/1307931702 (peer_addr_for_me v2:172.21.15.43:0/0) 2023-08-09T13:35:36.017 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:36.013+0000 7f6af5d9b640 1 -- 172.21.15.43:0/1307931702 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f6af806c8d0 msgr2=0x7f6af80c7d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:35:36.017 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:36.013+0000 7f6af5d9b640 1 --2- 172.21.15.43:0/1307931702 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f6af806c8d0 0x7f6af80c7d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:35:36.017 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:36.013+0000 7f6af5d9b640 1 -- 172.21.15.43:0/1307931702 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f6af8143c20 msgr2=0x7f6af80b21f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:35:36.017 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:36.013+0000 7f6af5d9b640 1 --2- 172.21.15.43:0/1307931702 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f6af8143c20 0x7f6af80b21f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:35:36.017 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:36.013+0000 7f6af5d9b640 1 -- 172.21.15.43:0/1307931702 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6ae8009a70 con 0x7f6af80c86f0 2023-08-09T13:35:36.018 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:36.013+0000 7f6af5d9b640 1 --2- 172.21.15.43:0/1307931702 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f6af80c86f0 0x7f6af80c5170 secure :-1 s=READY pgs=399 cs=0 l=1 rev1=1 crypto rx=0x7f6aec00e9e0 tx=0x7f6aec00eea0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-09T13:35:36.018 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:36.013+0000 7f6ae6ffd640 1 -- 172.21.15.43:0/1307931702 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6aec009670 con 0x7f6af80c86f0 2023-08-09T13:35:36.018 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:36.013+0000 7f6ae6ffd640 1 -- 172.21.15.43:0/1307931702 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6aec00c970 con 0x7f6af80c86f0 2023-08-09T13:35:36.018 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:36.013+0000 7f6ae6ffd640 1 -- 172.21.15.43:0/1307931702 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6aec00cd20 con 0x7f6af80c86f0 2023-08-09T13:35:36.018 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:36.013+0000 7f6afc8ad640 1 -- 172.21.15.43:0/1307931702 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6af80c6d00 con 0x7f6af80c86f0 2023-08-09T13:35:36.018 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:36.014+0000 7f6afc8ad640 1 -- 172.21.15.43:0/1307931702 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6af80c7280 con 0x7f6af80c86f0 2023-08-09T13:35:36.018 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:36.015+0000 7f6afc8ad640 1 -- 172.21.15.43:0/1307931702 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6af80739f0 con 0x7f6af80c86f0 2023-08-09T13:35:36.019 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:36.016+0000 7f6ae6ffd640 1 -- 172.21.15.43:0/1307931702 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 5) v1 ==== 88341+0+0 (secure 0 0 0) 0x7f6aec013470 con 0x7f6af80c86f0 2023-08-09T13:35:36.019 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:36.017+0000 7f6ae6ffd640 1 --2- 172.21.15.43:0/1307931702 >> [v2:172.21.15.203:6832/830097474,v1:172.21.15.203:6833/830097474] conn(0x7f6ad806a050 0x7f6ad806c510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:35:36.019 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:36.017+0000 7f6ae6ffd640 1 -- 172.21.15.43:0/1307931702 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 6871+0+0 (secure 0 0 0) 0x7f6aec088a70 con 0x7f6af80c86f0 2023-08-09T13:35:36.020 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:36.018+0000 7f6af659c640 1 --2- 172.21.15.43:0/1307931702 >> [v2:172.21.15.203:6832/830097474,v1:172.21.15.203:6833/830097474] conn(0x7f6ad806a050 0x7f6ad806c510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:35:36.021 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:36.019+0000 7f6af659c640 1 --2- 172.21.15.43:0/1307931702 >> [v2:172.21.15.203:6832/830097474,v1:172.21.15.203:6833/830097474] conn(0x7f6ad806a050 0x7f6ad806c510 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7f6ae8036000 tx=0x7f6ae802e000 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-09T13:35:36.024 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:36.022+0000 7f6ae6ffd640 1 -- 172.21.15.43:0/1307931702 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178066 (secure 0 0 0) 0x7f6aec0578b0 con 0x7f6af80c86f0 2023-08-09T13:35:36.233 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:36.231+0000 7f6afc8ad640 1 -- 172.21.15.43:0/1307931702 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7f6af80c7620 con 0x7f6af80c86f0 2023-08-09T13:35:36.234 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:36.232+0000 7f6ae6ffd640 1 -- 172.21.15.43:0/1307931702 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v26) v1 ==== 80+0+0 (secure 0 0 0) 0x7f6aec057440 con 0x7f6af80c86f0 2023-08-09T13:35:36.235 INFO:teuthology.orchestra.run.smithi043.stderr:listed 0 entries 2023-08-09T13:35:36.237 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:36.234+0000 7f6afc8ad640 1 -- 172.21.15.43:0/1307931702 >> [v2:172.21.15.203:6832/830097474,v1:172.21.15.203:6833/830097474] conn(0x7f6ad806a050 msgr2=0x7f6ad806c510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:35:36.237 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:36.234+0000 7f6afc8ad640 1 --2- 172.21.15.43:0/1307931702 >> [v2:172.21.15.203:6832/830097474,v1:172.21.15.203:6833/830097474] conn(0x7f6ad806a050 0x7f6ad806c510 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7f6ae8036000 tx=0x7f6ae802e000 comp rx=0 tx=0).stop 2023-08-09T13:35:36.237 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:36.234+0000 7f6afc8ad640 1 -- 172.21.15.43:0/1307931702 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f6af80c86f0 msgr2=0x7f6af80c5170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:35:36.238 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:36.234+0000 7f6afc8ad640 1 --2- 172.21.15.43:0/1307931702 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f6af80c86f0 0x7f6af80c5170 secure :-1 s=READY pgs=399 cs=0 l=1 rev1=1 crypto rx=0x7f6aec00e9e0 tx=0x7f6aec00eea0 comp rx=0 tx=0).stop 2023-08-09T13:35:36.238 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:36.235+0000 7f6afc8ad640 1 -- 172.21.15.43:0/1307931702 shutdown_connections 2023-08-09T13:35:36.238 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:36.235+0000 7f6afc8ad640 1 --2- 172.21.15.43:0/1307931702 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f6af8143c20 0x7f6af80b21f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:35:36.238 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:36.235+0000 7f6afc8ad640 1 --2- 172.21.15.43:0/1307931702 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f6af80c86f0 0x7f6af80c5170 unknown :-1 s=CLOSED pgs=399 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:35:36.238 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:36.235+0000 7f6afc8ad640 1 --2- 172.21.15.43:0/1307931702 >> [v2:172.21.15.203:6832/830097474,v1:172.21.15.203:6833/830097474] conn(0x7f6ad806a050 0x7f6ad806c510 unknown :-1 s=CLOSED pgs=205 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:35:36.238 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:36.235+0000 7f6afc8ad640 1 --2- 172.21.15.43:0/1307931702 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f6af806c8d0 0x7f6af80c7d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:35:36.238 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:36.235+0000 7f6afc8ad640 1 -- 172.21.15.43:0/1307931702 >> 172.21.15.43:0/1307931702 conn(0x7f6af80ce0a0 msgr2=0x7f6af80c8b50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-09T13:35:36.238 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:36.235+0000 7f6afc8ad640 1 -- 172.21.15.43:0/1307931702 shutdown_connections 2023-08-09T13:35:36.239 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:36.235+0000 7f6afc8ad640 1 -- 172.21.15.43:0/1307931702 wait complete. 2023-08-09T13:35:36.256 INFO:tasks.cephfs.fuse_mount:Running fusermount -u on ubuntu@smithi203.front.sepia.ceph.com... 2023-08-09T13:35:36.257 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-09T13:35:36.257 DEBUG:teuthology.orchestra.run.smithi203:> sudo fusermount -u /home/ubuntu/cephtest/mnt.1 2023-08-09T13:35:36.363 INFO:teuthology.orchestra.run:waiting for 300 2023-08-09T13:35:36.364 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi203.front.sepia.ceph.com 2023-08-09T13:35:36.364 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-09T13:35:36.365 DEBUG:teuthology.orchestra.run.smithi203:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.1) 2023-08-09T13:35:36.430 DEBUG:teuthology.orchestra.run.smithi043:> set -ex 2023-08-09T13:35:36.431 DEBUG:teuthology.orchestra.run.smithi043:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-09T13:35:36.492 DEBUG:teuthology.orchestra.run.smithi043:> set -ex 2023-08-09T13:35:36.493 DEBUG:teuthology.orchestra.run.smithi043:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-09T13:35:36.555 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-08-09T13:35:36.784 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:36.781+0000 7fed9caa9640 1 Processor -- start 2023-08-09T13:35:36.784 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:36.781+0000 7fed9caa9640 1 -- start start 2023-08-09T13:35:36.785 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:36.783+0000 7fed9caa9640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fed980b2790 0x7fed980d2ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:35:36.785 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:36.783+0000 7fed9caa9640 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7fed980b3050 0x7fed980b3f60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:35:36.785 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:36.783+0000 7fed9caa9640 1 -- --> v1:172.21.15.43:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fed980ce090 con 0x7fed980b2b70 2023-08-09T13:35:36.786 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:36.783+0000 7fed9caa9640 1 -- --> v2:172.21.15.203:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fed980b3950 con 0x7fed980b3050 2023-08-09T13:35:36.786 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:36.783+0000 7fed9caa9640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fed980cd7f0 con 0x7fed980b2790 2023-08-09T13:35:36.788 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:36.783+0000 7fed8dd9b640 1 --1- >> v1:172.21.15.43:6789/0 conn(0x7fed980b2b70 0x7fed980b5330 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.43:6789/0 says I am v1:172.21.15.43:44886/0 (socket says 172.21.15.43:44886) 2023-08-09T13:35:36.788 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:36.783+0000 7fed8dd9b640 1 -- 172.21.15.43:0/352896230 learned_addr learned my addr 172.21.15.43:0/352896230 (peer_addr_for_me v1:172.21.15.43:0/0) 2023-08-09T13:35:36.788 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:36.784+0000 7fed9659c640 1 --2- 172.21.15.43:0/352896230 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fed980b2790 0x7fed980d2ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:35:36.788 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:36.784+0000 7fed96d9d640 1 --2- 172.21.15.43:0/352896230 >> v2:172.21.15.203:3300/0 conn(0x7fed980b3050 0x7fed980b3f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:35:36.788 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:36.784+0000 7fed95d9b640 1 -- 172.21.15.43:0/352896230 <== mon.0 v1:172.21.15.43:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3726709943 0 0) 0x7fed980ce090 con 0x7fed980b2b70 2023-08-09T13:35:36.789 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:36.784+0000 7fed95d9b640 1 -- 172.21.15.43:0/352896230 --> v1:172.21.15.43:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fed84003520 con 0x7fed980b2b70 2023-08-09T13:35:36.789 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:36.784+0000 7fed95d9b640 1 -- 172.21.15.43:0/352896230 <== mon.0 v1:172.21.15.43:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2899452668 0 0) 0x7fed84003520 con 0x7fed980b2b70 2023-08-09T13:35:36.789 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:36.784+0000 7fed95d9b640 1 -- 172.21.15.43:0/352896230 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fed980b2790 msgr2=0x7fed980d2ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-09T13:35:36.789 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:36.784+0000 7fed95d9b640 1 --2- 172.21.15.43:0/352896230 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fed980b2790 0x7fed980d2ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:35:36.789 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:36.784+0000 7fed95d9b640 1 -- 172.21.15.43:0/352896230 >> v2:172.21.15.203:3300/0 conn(0x7fed980b3050 msgr2=0x7fed980b3f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-09T13:35:36.789 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:36.784+0000 7fed95d9b640 1 --2- 172.21.15.43:0/352896230 >> v2:172.21.15.203:3300/0 conn(0x7fed980b3050 0x7fed980b3f60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:35:36.789 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:36.784+0000 7fed95d9b640 1 -- 172.21.15.43:0/352896230 --> v1:172.21.15.43:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fed980cdad0 con 0x7fed980b2b70 2023-08-09T13:35:36.790 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:36.784+0000 7fed95d9b640 1 -- 172.21.15.43:0/352896230 <== mon.0 v1:172.21.15.43:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1285150740 0 0) 0x7fed80001670 con 0x7fed980b2b70 2023-08-09T13:35:36.790 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:36.784+0000 7fed95d9b640 1 -- 172.21.15.43:0/352896230 >> v1:172.21.15.43:6789/0 conn(0x7fed980b2b70 legacy=0x7fed980b5330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:35:36.790 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:36.784+0000 7fed95d9b640 1 --2- 172.21.15.43:0/352896230 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fed840039e0 0x7fed84003e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:35:36.790 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:36.784+0000 7fed95d9b640 1 --2- 172.21.15.43:0/352896230 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fed840047c0 0x7fed84004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:35:36.790 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:36.784+0000 7fed95d9b640 1 --2- 172.21.15.43:0/352896230 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fed980b2790 0x7fed84005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:35:36.790 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:36.785+0000 7fed95d9b640 1 -- 172.21.15.43:0/352896230 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fed980b3950 con 0x7fed840039e0 2023-08-09T13:35:36.790 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:36.785+0000 7fed95d9b640 1 -- 172.21.15.43:0/352896230 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fed980cd7f0 con 0x7fed840047c0 2023-08-09T13:35:36.791 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:36.785+0000 7fed95d9b640 1 -- 172.21.15.43:0/352896230 --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fed8400ba00 con 0x7fed980b2790 2023-08-09T13:35:36.791 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:36.785+0000 7fed9659c640 1 --2- 172.21.15.43:0/352896230 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fed840039e0 0x7fed84003e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:35:36.791 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:36.785+0000 7fed9659c640 1 -- 172.21.15.43:0/352896230 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fed980b2790 msgr2=0x7fed84005180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-09T13:35:36.791 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:36.785+0000 7fed9659c640 1 --2- 172.21.15.43:0/352896230 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fed980b2790 0x7fed84005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:35:36.791 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:36.785+0000 7fed9659c640 1 -- 172.21.15.43:0/352896230 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fed840047c0 msgr2=0x7fed84004c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-09T13:35:36.791 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:36.785+0000 7fed9659c640 1 --2- 172.21.15.43:0/352896230 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fed840047c0 0x7fed84004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:35:36.791 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:36.785+0000 7fed9659c640 1 -- 172.21.15.43:0/352896230 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fed84003520 con 0x7fed840039e0 2023-08-09T13:35:36.792 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:36.787+0000 7fed9659c640 1 --2- 172.21.15.43:0/352896230 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fed840039e0 0x7fed84003e00 secure :-1 s=READY pgs=401 cs=0 l=1 rev1=1 crypto rx=0x7fed8800ba30 tx=0x7fed840057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=614d8f3e50b9cdf9 server_cookie=0 in_seq=0 out_seq=0 2023-08-09T13:35:36.793 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:36.787+0000 7fed95d9b640 1 -- 172.21.15.43:0/352896230 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fed88032680 con 0x7fed840039e0 2023-08-09T13:35:36.793 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:36.787+0000 7fed95d9b640 1 -- 172.21.15.43:0/352896230 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fed880327e0 con 0x7fed840039e0 2023-08-09T13:35:36.793 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:36.787+0000 7fed95d9b640 1 -- 172.21.15.43:0/352896230 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fed88032b70 con 0x7fed840039e0 2023-08-09T13:35:36.793 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:36.787+0000 7fed9caa9640 1 -- 172.21.15.43:0/352896230 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fed840039e0 msgr2=0x7fed84003e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:35:36.793 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:36.787+0000 7fed9caa9640 1 --2- 172.21.15.43:0/352896230 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fed840039e0 0x7fed84003e00 secure :-1 s=READY pgs=401 cs=0 l=1 rev1=1 crypto rx=0x7fed8800ba30 tx=0x7fed840057f0 comp rx=0 tx=0).stop 2023-08-09T13:35:36.793 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:36.787+0000 7fed96d9d640 1 -- 172.21.15.43:0/352896230 reap_dead start 2023-08-09T13:35:36.794 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:36.787+0000 7fed9caa9640 1 -- 172.21.15.43:0/352896230 shutdown_connections 2023-08-09T13:35:36.794 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:36.788+0000 7fed9caa9640 1 -- 172.21.15.43:0/352896230 >> 172.21.15.43:0/352896230 conn(0x7fed9809b7b0 msgr2=0x7fed980d24e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-09T13:35:36.794 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:36.788+0000 7fed9caa9640 1 -- 172.21.15.43:0/352896230 shutdown_connections 2023-08-09T13:35:36.794 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:36.788+0000 7fed9caa9640 1 -- 172.21.15.43:0/352896230 wait complete. 2023-08-09T13:35:36.794 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:36.789+0000 7fed9caa9640 1 Processor -- start 2023-08-09T13:35:36.794 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:36.790+0000 7fed9caa9640 1 -- start start 2023-08-09T13:35:36.796 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:36.790+0000 7fed9caa9640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fed840039e0 0x7fed9815ab00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:35:36.796 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:36.790+0000 7fed9caa9640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fed981f1bc0 0x7fed981f2040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:35:36.796 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:36.791+0000 7fed9caa9640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fed980c81e0 0x7fed980c8660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:35:36.796 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:36.791+0000 7fed9caa9640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fed980c8ba0 con 0x7fed981f1bc0 2023-08-09T13:35:36.796 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:36.791+0000 7fed9caa9640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fed980c95b0 con 0x7fed980c81e0 2023-08-09T13:35:36.797 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:36.791+0000 7fed9caa9640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fed981f26b0 con 0x7fed840039e0 2023-08-09T13:35:36.797 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:36.791+0000 7fed8dd9b640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fed981f1bc0 0x7fed981f2040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:35:36.797 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:36.791+0000 7fed8dd9b640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fed981f1bc0 0x7fed981f2040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:49906/0 (socket says 172.21.15.43:49906) 2023-08-09T13:35:36.797 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:36.791+0000 7fed8dd9b640 1 -- 172.21.15.43:0/31158548 learned_addr learned my addr 172.21.15.43:0/31158548 (peer_addr_for_me v2:172.21.15.43:0/0) 2023-08-09T13:35:36.797 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:36.791+0000 7fed96d9d640 1 --2- 172.21.15.43:0/31158548 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fed980c81e0 0x7fed980c8660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:35:36.797 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:36.791+0000 7fed9659c640 1 --2- 172.21.15.43:0/31158548 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fed840039e0 0x7fed9815ab00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:35:36.797 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:36.791+0000 7fed8dd9b640 1 -- 172.21.15.43:0/31158548 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fed840039e0 msgr2=0x7fed9815ab00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:35:36.798 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:36.791+0000 7fed8dd9b640 1 --2- 172.21.15.43:0/31158548 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fed840039e0 0x7fed9815ab00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:35:36.798 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:36.791+0000 7fed8dd9b640 1 -- 172.21.15.43:0/31158548 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fed980c81e0 msgr2=0x7fed980c8660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:35:36.798 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:36.791+0000 7fed8dd9b640 1 --2- 172.21.15.43:0/31158548 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fed980c81e0 0x7fed980c8660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:35:36.798 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:36.791+0000 7fed8dd9b640 1 -- 172.21.15.43:0/31158548 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fed981f2990 con 0x7fed981f1bc0 2023-08-09T13:35:36.798 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:36.792+0000 7fed8dd9b640 1 --2- 172.21.15.43:0/31158548 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fed981f1bc0 0x7fed981f2040 secure :-1 s=READY pgs=402 cs=0 l=1 rev1=1 crypto rx=0x7fed8000c730 tx=0x7fed8000cbf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-09T13:35:36.798 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:36.792+0000 7fed8f7fe640 1 -- 172.21.15.43:0/31158548 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fed800072d0 con 0x7fed981f1bc0 2023-08-09T13:35:36.798 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:36.792+0000 7fed8f7fe640 1 -- 172.21.15.43:0/31158548 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fed80007450 con 0x7fed981f1bc0 2023-08-09T13:35:36.798 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:36.792+0000 7fed9caa9640 1 -- 172.21.15.43:0/31158548 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fed980c9800 con 0x7fed981f1bc0 2023-08-09T13:35:36.799 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:36.792+0000 7fed8f7fe640 1 -- 172.21.15.43:0/31158548 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fed800047e0 con 0x7fed981f1bc0 2023-08-09T13:35:36.799 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:36.792+0000 7fed9caa9640 1 -- 172.21.15.43:0/31158548 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fed9815b320 con 0x7fed981f1bc0 2023-08-09T13:35:36.799 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:36.793+0000 7fed9caa9640 1 -- 172.21.15.43:0/31158548 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fed981f2bd0 con 0x7fed981f1bc0 2023-08-09T13:35:36.800 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:36.794+0000 7fed8f7fe640 1 -- 172.21.15.43:0/31158548 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 5) v1 ==== 88341+0+0 (secure 0 0 0) 0x7fed80004b70 con 0x7fed981f1bc0 2023-08-09T13:35:36.800 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:36.795+0000 7fed8f7fe640 1 --2- 172.21.15.43:0/31158548 >> [v2:172.21.15.203:6832/830097474,v1:172.21.15.203:6833/830097474] conn(0x7fed7806a050 0x7fed7806c510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:35:36.800 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:36.795+0000 7fed8f7fe640 1 -- 172.21.15.43:0/31158548 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 6871+0+0 (secure 0 0 0) 0x7fed80088180 con 0x7fed981f1bc0 2023-08-09T13:35:36.800 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:36.796+0000 7fed9659c640 1 --2- 172.21.15.43:0/31158548 >> [v2:172.21.15.203:6832/830097474,v1:172.21.15.203:6833/830097474] conn(0x7fed7806a050 0x7fed7806c510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:35:36.800 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:36.797+0000 7fed9659c640 1 --2- 172.21.15.43:0/31158548 >> [v2:172.21.15.203:6832/830097474,v1:172.21.15.203:6833/830097474] conn(0x7fed7806a050 0x7fed7806c510 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7fed88037000 tx=0x7fed88030000 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-09T13:35:36.802 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:36.800+0000 7fed8f7fe640 1 -- 172.21.15.43:0/31158548 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178066 (secure 0 0 0) 0x7fed800535d0 con 0x7fed981f1bc0 2023-08-09T13:35:36.996 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:36.994+0000 7fed9caa9640 1 -- 172.21.15.43:0/31158548 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fed980739f0 con 0x7fed981f1bc0 2023-08-09T13:35:36.998 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:36.996+0000 7fed8f7fe640 1 -- 172.21.15.43:0/31158548 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v26) v1 ==== 74+0+20146 (secure 0 0 0) 0x7fed80056bf0 con 0x7fed981f1bc0 2023-08-09T13:35:36.999 INFO:teuthology.orchestra.run.smithi043.stdout: 2023-08-09T13:35:36.999 INFO:teuthology.orchestra.run.smithi043.stdout:{"epoch":26,"fsid":"daac87ee-1337-4250-a955-aea4e408a39b","created":"2023-08-09T13:33:41.701198+0000","modified":"2023-08-09T13:34:48.711911+0000","last_up_change":"2023-08-09T13:33:53.783453+0000","last_in_change":"2023-08-09T13:33:45.148904+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":5,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":"rbd","create_time":"2023-08-09T13:33:54.365137+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":".mgr","create_time":"2023-08-09T13:33:54.831803+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_metadata","create_time":"2023-08-09T13:34:00.207974+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data","create_time":"2023-08-09T13:34:01.614765+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":5,"pool_name":"cephfs_data_ec","create_time":"2023-08-09T13:34:05.239583+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"24","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"cbd49ecd-e896-4fbd-864e-5c37a829b165","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6800","nonce":502667335},{"type":"v1","addr":"172.21.15.43:6801","nonce":502667335}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6802","nonce":502667335},{"type":"v1","addr":"172.21.15.43:6803","nonce":502667335}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6806","nonce":502667335},{"type":"v1","addr":"172.21.15.43:6807","nonce":502667335}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6804","nonce":502667335},{"type":"v1","addr":"172.21.15.43:6805","nonce":502667335}]},"public_addr":"172.21.15.43:6801/502667335","cluster_addr":"172.21.15.43:6803/502667335","heartbeat_back_addr":"172.21.15.43:6807/502667335","heartbeat_front_addr":"172.21.15.43:6805/502667335","state":["exists","up"]},{"osd":1,"uuid":"75c7e0fe-1ec7-4fe8-a2f2-3661a65b8500","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6808","nonce":3289134610},{"type":"v1","addr":"172.21.15.43:6809","nonce":3289134610}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6810","nonce":3289134610},{"type":"v1","addr":"172.21.15.43:6812","nonce":3289134610}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6821","nonce":3289134610},{"type":"v1","addr":"172.21.15.43:6824","nonce":3289134610}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6815","nonce":3289134610},{"type":"v1","addr":"172.21.15.43:6818","nonce":3289134610}]},"public_addr":"172.21.15.43:6809/3289134610","cluster_addr":"172.21.15.43:6812/3289134610","heartbeat_back_addr":"172.21.15.43:6824/3289134610","heartbeat_front_addr":"172.21.15.43:6818/3289134610","state":["exists","up"]},{"osd":2,"uuid":"bb5c41e9-7f5f-4125-9b01-3bb8bfc2ea83","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6811","nonce":4079709736},{"type":"v1","addr":"172.21.15.43:6814","nonce":4079709736}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6817","nonce":4079709736},{"type":"v1","addr":"172.21.15.43:6820","nonce":4079709736}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6829","nonce":4079709736},{"type":"v1","addr":"172.21.15.43:6831","nonce":4079709736}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6823","nonce":4079709736},{"type":"v1","addr":"172.21.15.43:6826","nonce":4079709736}]},"public_addr":"172.21.15.43:6814/4079709736","cluster_addr":"172.21.15.43:6820/4079709736","heartbeat_back_addr":"172.21.15.43:6831/4079709736","heartbeat_front_addr":"172.21.15.43:6826/4079709736","state":["exists","up"]},{"osd":3,"uuid":"bfe4c8b4-694a-465c-87e6-2b2aab852613","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6813","nonce":1709538759},{"type":"v1","addr":"172.21.15.43:6816","nonce":1709538759}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6819","nonce":1709538759},{"type":"v1","addr":"172.21.15.43:6822","nonce":1709538759}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6828","nonce":1709538759},{"type":"v1","addr":"172.21.15.43:6830","nonce":1709538759}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6825","nonce":1709538759},{"type":"v1","addr":"172.21.15.43:6827","nonce":1709538759}]},"public_addr":"172.21.15.43:6816/1709538759","cluster_addr":"172.21.15.43:6822/1709538759","heartbeat_back_addr":"172.21.15.43:6830/1709538759","heartbeat_front_addr":"172.21.15.43:6827/1709538759","state":["exists","up"]},{"osd":4,"uuid":"fa2e8a77-aba1-4756-853e-0d174a5bdeba","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6800","nonce":4196866481},{"type":"v1","addr":"172.21.15.203:6801","nonce":4196866481}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6802","nonce":4196866481},{"type":"v1","addr":"172.21.15.203:6803","nonce":4196866481}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6806","nonce":4196866481},{"type":"v1","addr":"172.21.15.203:6807","nonce":4196866481}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6804","nonce":4196866481},{"type":"v1","addr":"172.21.15.203:6805","nonce":4196866481}]},"public_addr":"172.21.15.203:6801/4196866481","cluster_addr":"172.21.15.203:6803/4196866481","heartbeat_back_addr":"172.21.15.203:6807/4196866481","heartbeat_front_addr":"172.21.15.203:6805/4196866481","state":["exists","up"]},{"osd":5,"uuid":"81811e4b-4d24-4f51-a82c-f1d700a2a00d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6816","nonce":3059290081},{"type":"v1","addr":"172.21.15.203:6817","nonce":3059290081}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6818","nonce":3059290081},{"type":"v1","addr":"172.21.15.203:6819","nonce":3059290081}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6822","nonce":3059290081},{"type":"v1","addr":"172.21.15.203:6823","nonce":3059290081}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6820","nonce":3059290081},{"type":"v1","addr":"172.21.15.203:6821","nonce":3059290081}]},"public_addr":"172.21.15.203:6817/3059290081","cluster_addr":"172.21.15.203:6819/3059290081","heartbeat_back_addr":"172.21.15.203:6823/3059290081","heartbeat_front_addr":"172.21.15.203:6821/3059290081","state":["exists","up"]},{"osd":6,"uuid":"307bac4d-6a5b-443d-af75-56d695d7674f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6824","nonce":512155569},{"type":"v1","addr":"172.21.15.203:6825","nonce":512155569}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6826","nonce":512155569},{"type":"v1","addr":"172.21.15.203:6827","nonce":512155569}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6830","nonce":512155569},{"type":"v1","addr":"172.21.15.203:6831","nonce":512155569}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6828","nonce":512155569},{"type":"v1","addr":"172.21.15.203:6829","nonce":512155569}]},"public_addr":"172.21.15.203:6825/512155569","cluster_addr":"172.21.15.203:6827/512155569","heartbeat_back_addr":"172.21.15.203:6831/512155569","heartbeat_front_addr":"172.21.15.203:6829/512155569","state":["exists","up"]},{"osd":7,"uuid":"a95dc7ca-9f1b-4295-af70-68a22a2d7d5e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6808","nonce":3331450438},{"type":"v1","addr":"172.21.15.203:6809","nonce":3331450438}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6810","nonce":3331450438},{"type":"v1","addr":"172.21.15.203:6811","nonce":3331450438}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6814","nonce":3331450438},{"type":"v1","addr":"172.21.15.203:6815","nonce":3331450438}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6812","nonce":3331450438},{"type":"v1","addr":"172.21.15.203:6813","nonce":3331450438}]},"public_addr":"172.21.15.203:6809/3331450438","cluster_addr":"172.21.15.203:6811/3331450438","heartbeat_back_addr":"172.21.15.203:6815/3331450438","heartbeat_front_addr":"172.21.15.203:6813/3331450438","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-09T13:33:50.547652+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-09T13:33:50.589469+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-09T13:33:50.561724+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-09T13:33:50.599650+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-09T13:33:50.411855+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-09T13:33:50.219681+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-09T13:33:50.504664+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-09T13:33:50.441248+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"4.14","mappings":[{"from":1,"to":5}]},{"pgid":"4.29","mappings":[{"from":3,"to":2}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-08-09T13:35:37.001 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:36.998+0000 7fed9caa9640 1 -- 172.21.15.43:0/31158548 >> [v2:172.21.15.203:6832/830097474,v1:172.21.15.203:6833/830097474] conn(0x7fed7806a050 msgr2=0x7fed7806c510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:35:37.002 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:36.998+0000 7fed9caa9640 1 --2- 172.21.15.43:0/31158548 >> [v2:172.21.15.203:6832/830097474,v1:172.21.15.203:6833/830097474] conn(0x7fed7806a050 0x7fed7806c510 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7fed88037000 tx=0x7fed88030000 comp rx=0 tx=0).stop 2023-08-09T13:35:37.002 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:36.998+0000 7fed9caa9640 1 -- 172.21.15.43:0/31158548 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fed981f1bc0 msgr2=0x7fed981f2040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:35:37.002 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:36.998+0000 7fed9caa9640 1 --2- 172.21.15.43:0/31158548 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fed981f1bc0 0x7fed981f2040 secure :-1 s=READY pgs=402 cs=0 l=1 rev1=1 crypto rx=0x7fed8000c730 tx=0x7fed8000cbf0 comp rx=0 tx=0).stop 2023-08-09T13:35:37.002 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:36.999+0000 7fed9caa9640 1 -- 172.21.15.43:0/31158548 shutdown_connections 2023-08-09T13:35:37.002 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:36.999+0000 7fed9caa9640 1 --2- 172.21.15.43:0/31158548 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fed980c81e0 0x7fed980c8660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:35:37.002 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:36.999+0000 7fed9caa9640 1 --2- 172.21.15.43:0/31158548 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fed981f1bc0 0x7fed981f2040 unknown :-1 s=CLOSED pgs=402 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:35:37.002 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:36.999+0000 7fed9caa9640 1 --2- 172.21.15.43:0/31158548 >> [v2:172.21.15.203:6832/830097474,v1:172.21.15.203:6833/830097474] conn(0x7fed7806a050 0x7fed7806c510 unknown :-1 s=CLOSED pgs=206 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:35:37.003 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:36.999+0000 7fed9caa9640 1 --2- 172.21.15.43:0/31158548 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fed840039e0 0x7fed9815ab00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:35:37.003 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:36.999+0000 7fed9caa9640 1 -- 172.21.15.43:0/31158548 >> 172.21.15.43:0/31158548 conn(0x7fed9809b7b0 msgr2=0x7fed9815ba50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-09T13:35:37.003 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:36.999+0000 7fed9caa9640 1 -- 172.21.15.43:0/31158548 shutdown_connections 2023-08-09T13:35:37.003 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:36.999+0000 7fed9caa9640 1 -- 172.21.15.43:0/31158548 wait complete. 2023-08-09T13:35:37.021 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': 'rbd', 'create_time': '2023-08-09T13:33:54.365137+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}, 'read_balance': {'score_acting': 2, 'score_stable': 2, 'optimal_score': 1, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': '.mgr', 'create_time': '2023-08-09T13:33:54.831803+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_metadata', 'create_time': '2023-08-09T13:34:00.207974+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data', 'create_time': '2023-08-09T13:34:01.614765+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 2, 'score_stable': 2, 'optimal_score': 1, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 5, 'pool_name': 'cephfs_data_ec', 'create_time': '2023-08-09T13:34:05.239583+0000', 'flags': 5, 'flags_names': 'hashpspool,ec_overwrites', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '24', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2023-08-09T13:35:37.022 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get rbd pg_num 2023-08-09T13:35:37.255 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:37.252+0000 7f464ffff640 1 Processor -- start 2023-08-09T13:35:37.255 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:37.252+0000 7f464ffff640 1 -- start start 2023-08-09T13:35:37.256 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:37.254+0000 7f464ffff640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f4650070ef0 0x7f46500ab8d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:35:37.257 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:37.254+0000 7f464ffff640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f46500abe10 0x7f46500cb9a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:35:37.257 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:37.254+0000 7f464ffff640 1 -- --> v1:172.21.15.43:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f46500acfa0 con 0x7f46500cbee0 2023-08-09T13:35:37.257 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:37.254+0000 7f464ffff640 1 -- --> v2:172.21.15.43:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f46500cc370 con 0x7f46500abe10 2023-08-09T13:35:37.257 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:37.254+0000 7f464ffff640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f465015c940 con 0x7f4650070ef0 2023-08-09T13:35:37.258 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:37.254+0000 7f464e7fc640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f46500abe10 0x7f46500cb9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:35:37.258 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:37.254+0000 7f464f7fe640 1 --1- >> v1:172.21.15.43:6789/0 conn(0x7f46500cbee0 0x7f4650141e50 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.43:6789/0 says I am v1:172.21.15.43:44894/0 (socket says 172.21.15.43:44894) 2023-08-09T13:35:37.258 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:37.254+0000 7f464f7fe640 1 -- 172.21.15.43:0/3916205699 learned_addr learned my addr 172.21.15.43:0/3916205699 (peer_addr_for_me v1:172.21.15.43:0/0) 2023-08-09T13:35:37.259 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:37.255+0000 7f464effd640 1 --2- 172.21.15.43:0/3916205699 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f4650070ef0 0x7f46500ab8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:35:37.259 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:37.255+0000 7f464e7fc640 1 -- 172.21.15.43:0/3916205699 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f4650070ef0 msgr2=0x7f46500ab8d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-09T13:35:37.259 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:37.255+0000 7f464e7fc640 1 --2- 172.21.15.43:0/3916205699 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f4650070ef0 0x7f46500ab8d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:35:37.259 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:37.255+0000 7f464e7fc640 1 -- 172.21.15.43:0/3916205699 >> v1:172.21.15.43:6789/0 conn(0x7f46500cbee0 legacy=0x7f4650141e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:35:37.259 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:37.255+0000 7f464e7fc640 1 -- 172.21.15.43:0/3916205699 --> v2:172.21.15.43:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f46500ac390 con 0x7f46500abe10 2023-08-09T13:35:37.260 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:37.256+0000 7f464e7fc640 1 --2- 172.21.15.43:0/3916205699 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f46500abe10 0x7f46500cb9a0 secure :-1 s=READY pgs=404 cs=0 l=1 rev1=1 crypto rx=0x7f46440097a0 tx=0x7f464402b940 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6e9471c5ed22ad5 server_cookie=0 in_seq=0 out_seq=0 2023-08-09T13:35:37.260 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:37.256+0000 7f464dffb640 1 -- 172.21.15.43:0/3916205699 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4644030800 con 0x7f46500abe10 2023-08-09T13:35:37.260 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:37.257+0000 7f464dffb640 1 -- 172.21.15.43:0/3916205699 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4644030960 con 0x7f46500abe10 2023-08-09T13:35:37.261 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:37.257+0000 7f464dffb640 1 -- 172.21.15.43:0/3916205699 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4644030cf0 con 0x7f46500abe10 2023-08-09T13:35:37.261 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:37.257+0000 7f464ffff640 1 -- 172.21.15.43:0/3916205699 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f46500abe10 msgr2=0x7f46500cb9a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:35:37.261 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:37.257+0000 7f464ffff640 1 --2- 172.21.15.43:0/3916205699 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f46500abe10 0x7f46500cb9a0 secure :-1 s=READY pgs=404 cs=0 l=1 rev1=1 crypto rx=0x7f46440097a0 tx=0x7f464402b940 comp rx=0 tx=0).stop 2023-08-09T13:35:37.261 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:37.257+0000 7f464ffff640 1 -- 172.21.15.43:0/3916205699 shutdown_connections 2023-08-09T13:35:37.261 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:37.257+0000 7f464ffff640 1 --2- 172.21.15.43:0/3916205699 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f46500abe10 0x7f46500cb9a0 unknown :-1 s=CLOSED pgs=404 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:35:37.261 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:37.257+0000 7f464ffff640 1 --2- 172.21.15.43:0/3916205699 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f4650070ef0 0x7f46500ab8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:35:37.261 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:37.257+0000 7f464ffff640 1 -- 172.21.15.43:0/3916205699 >> 172.21.15.43:0/3916205699 conn(0x7f46500cddd0 msgr2=0x7f46500ce1e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-09T13:35:37.262 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:37.257+0000 7f464ffff640 1 -- 172.21.15.43:0/3916205699 shutdown_connections 2023-08-09T13:35:37.262 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:37.257+0000 7f464ffff640 1 -- 172.21.15.43:0/3916205699 wait complete. 2023-08-09T13:35:37.262 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:37.259+0000 7f464ffff640 1 Processor -- start 2023-08-09T13:35:37.263 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:37.259+0000 7f464ffff640 1 -- start start 2023-08-09T13:35:37.263 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:37.260+0000 7f464ffff640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f4650070ef0 0x7f465014a1f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:35:37.263 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:37.260+0000 7f464ffff640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f46500abe10 0x7f4650146410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:35:37.263 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:37.260+0000 7f464ffff640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f46500cbee0 0x7f46500ca590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:35:37.263 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:37.260+0000 7f464ffff640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f465015cc90 con 0x7f46500cbee0 2023-08-09T13:35:37.263 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:37.260+0000 7f464ffff640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f465015cf50 con 0x7f4650070ef0 2023-08-09T13:35:37.264 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:37.260+0000 7f464ffff640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f465015d230 con 0x7f46500abe10 2023-08-09T13:35:37.265 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:37.261+0000 7f464f7fe640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f46500cbee0 0x7f46500ca590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:35:37.265 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:37.261+0000 7f464f7fe640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f46500cbee0 0x7f46500ca590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:49914/0 (socket says 172.21.15.43:49914) 2023-08-09T13:35:37.265 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:37.261+0000 7f464f7fe640 1 -- 172.21.15.43:0/1708798707 learned_addr learned my addr 172.21.15.43:0/1708798707 (peer_addr_for_me v2:172.21.15.43:0/0) 2023-08-09T13:35:37.265 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:37.261+0000 7f464e7fc640 1 --2- 172.21.15.43:0/1708798707 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f46500abe10 0x7f4650146410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:35:37.265 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:37.261+0000 7f464f7fe640 1 -- 172.21.15.43:0/1708798707 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f46500abe10 msgr2=0x7f4650146410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:35:37.266 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:37.261+0000 7f464f7fe640 1 --2- 172.21.15.43:0/1708798707 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f46500abe10 0x7f4650146410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:35:37.266 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:37.261+0000 7f464f7fe640 1 -- 172.21.15.43:0/1708798707 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f4650070ef0 msgr2=0x7f465014a1f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:35:37.266 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:37.261+0000 7f464f7fe640 1 --2- 172.21.15.43:0/1708798707 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f4650070ef0 0x7f465014a1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:35:37.266 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:37.261+0000 7f464f7fe640 1 -- 172.21.15.43:0/1708798707 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4644009400 con 0x7f46500cbee0 2023-08-09T13:35:37.266 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:37.261+0000 7f464f7fe640 1 --2- 172.21.15.43:0/1708798707 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f46500cbee0 0x7f46500ca590 secure :-1 s=READY pgs=405 cs=0 l=1 rev1=1 crypto rx=0x7f464800cbc0 tx=0x7f464800a430 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-09T13:35:37.266 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:37.262+0000 7f4637fff640 1 -- 172.21.15.43:0/1708798707 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f464800ad40 con 0x7f46500cbee0 2023-08-09T13:35:37.266 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:37.262+0000 7f4637fff640 1 -- 172.21.15.43:0/1708798707 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4648012400 con 0x7f46500cbee0 2023-08-09T13:35:37.267 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:37.262+0000 7f4637fff640 1 -- 172.21.15.43:0/1708798707 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f464800aea0 con 0x7f46500cbee0 2023-08-09T13:35:37.267 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:37.262+0000 7f464ffff640 1 -- 172.21.15.43:0/1708798707 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f465015d510 con 0x7f46500cbee0 2023-08-09T13:35:37.267 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:37.262+0000 7f464ffff640 1 -- 172.21.15.43:0/1708798707 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f46500caad0 con 0x7f46500cbee0 2023-08-09T13:35:37.267 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:37.263+0000 7f464ffff640 1 -- 172.21.15.43:0/1708798707 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f465015da30 con 0x7f46500cbee0 2023-08-09T13:35:37.268 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:37.264+0000 7f4637fff640 1 -- 172.21.15.43:0/1708798707 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 5) v1 ==== 88341+0+0 (secure 0 0 0) 0x7f4648007070 con 0x7f46500cbee0 2023-08-09T13:35:37.268 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:37.265+0000 7f4637fff640 1 --2- 172.21.15.43:0/1708798707 >> [v2:172.21.15.203:6832/830097474,v1:172.21.15.203:6833/830097474] conn(0x7f463c06a050 0x7f463c06c510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:35:37.268 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:37.265+0000 7f4637fff640 1 -- 172.21.15.43:0/1708798707 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 6871+0+0 (secure 0 0 0) 0x7f46480887d0 con 0x7f46500cbee0 2023-08-09T13:35:37.269 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:37.266+0000 7f464effd640 1 --2- 172.21.15.43:0/1708798707 >> [v2:172.21.15.203:6832/830097474,v1:172.21.15.203:6833/830097474] conn(0x7f463c06a050 0x7f463c06c510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:35:37.272 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:37.270+0000 7f4637fff640 1 -- 172.21.15.43:0/1708798707 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178066 (secure 0 0 0) 0x7f465015da30 con 0x7f46500cbee0 2023-08-09T13:35:37.472 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:37.469+0000 7f464ffff640 1 -- 172.21.15.43:0/1708798707 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"} v 0) v1 -- 0x7f465015da30 con 0x7f46500cbee0 2023-08-09T13:35:37.473 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:37.470+0000 7f4637fff640 1 -- 172.21.15.43:0/1708798707 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"}]=0 v26) v1 ==== 92+0+10 (secure 0 0 0) 0x7f465015da30 con 0x7f46500cbee0 2023-08-09T13:35:37.473 INFO:teuthology.orchestra.run.smithi043.stdout:pg_num: 8 2023-08-09T13:35:37.474 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:37.472+0000 7f464effd640 1 --2- 172.21.15.43:0/1708798707 >> [v2:172.21.15.203:6832/830097474,v1:172.21.15.203:6833/830097474] conn(0x7f463c06a050 0x7f463c06c510 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7f4640006fd0 tx=0x7f4640008040 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-09T13:35:37.476 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:37.473+0000 7f464ffff640 1 -- 172.21.15.43:0/1708798707 >> [v2:172.21.15.203:6832/830097474,v1:172.21.15.203:6833/830097474] conn(0x7f463c06a050 msgr2=0x7f463c06c510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:35:37.477 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:37.473+0000 7f464ffff640 1 --2- 172.21.15.43:0/1708798707 >> [v2:172.21.15.203:6832/830097474,v1:172.21.15.203:6833/830097474] conn(0x7f463c06a050 0x7f463c06c510 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7f4640006fd0 tx=0x7f4640008040 comp rx=0 tx=0).stop 2023-08-09T13:35:37.477 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:37.473+0000 7f464ffff640 1 -- 172.21.15.43:0/1708798707 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f46500cbee0 msgr2=0x7f46500ca590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:35:37.477 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:37.474+0000 7f464ffff640 1 --2- 172.21.15.43:0/1708798707 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f46500cbee0 0x7f46500ca590 secure :-1 s=READY pgs=405 cs=0 l=1 rev1=1 crypto rx=0x7f464800cbc0 tx=0x7f464800a430 comp rx=0 tx=0).stop 2023-08-09T13:35:37.477 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:37.474+0000 7f464ffff640 1 -- 172.21.15.43:0/1708798707 shutdown_connections 2023-08-09T13:35:37.477 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:37.474+0000 7f464ffff640 1 --2- 172.21.15.43:0/1708798707 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f46500cbee0 0x7f46500ca590 unknown :-1 s=CLOSED pgs=405 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:35:37.477 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:37.474+0000 7f464ffff640 1 --2- 172.21.15.43:0/1708798707 >> [v2:172.21.15.203:6832/830097474,v1:172.21.15.203:6833/830097474] conn(0x7f463c06a050 0x7f463c06c510 unknown :-1 s=CLOSED pgs=207 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:35:37.478 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:37.474+0000 7f464ffff640 1 --2- 172.21.15.43:0/1708798707 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f46500abe10 0x7f4650146410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:35:37.478 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:37.474+0000 7f464ffff640 1 --2- 172.21.15.43:0/1708798707 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f4650070ef0 0x7f465014a1f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:35:37.478 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:37.474+0000 7f464ffff640 1 -- 172.21.15.43:0/1708798707 >> 172.21.15.43:0/1708798707 conn(0x7f46500cddd0 msgr2=0x7f465015ffe0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-09T13:35:37.478 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:37.474+0000 7f464ffff640 1 -- 172.21.15.43:0/1708798707 shutdown_connections 2023-08-09T13:35:37.478 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:37.474+0000 7f464ffff640 1 -- 172.21.15.43:0/1708798707 wait complete. 2023-08-09T13:35:37.495 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-08-09T13:35:37.723 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:37.720+0000 7f72351bd640 1 Processor -- start 2023-08-09T13:35:37.723 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:37.720+0000 7f72351bd640 1 -- start start 2023-08-09T13:35:37.724 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:37.722+0000 7f72351bd640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f72300b7df0 0x7f72300d1b30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:35:37.724 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:37.722+0000 7f72351bd640 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7f72300cebf0 0x7f72300cf040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:35:37.725 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:37.722+0000 7f72351bd640 1 -- --> v1:172.21.15.43:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f72300cf580 con 0x7f723015c2d0 2023-08-09T13:35:37.725 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:37.722+0000 7f72351bd640 1 -- --> v2:172.21.15.43:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f72300b8230 con 0x7f72300b7df0 2023-08-09T13:35:37.725 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:37.722+0000 7f72351bd640 1 -- --> v2:172.21.15.203:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f72300d16a0 con 0x7f72300cebf0 2023-08-09T13:35:37.726 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:37.722+0000 7f722ffff640 1 --1- >> v1:172.21.15.43:6789/0 conn(0x7f723015c2d0 0x7f72300cd020 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.43:6789/0 says I am v1:172.21.15.43:44900/0 (socket says 172.21.15.43:44900) 2023-08-09T13:35:37.726 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:37.722+0000 7f722ffff640 1 -- 172.21.15.43:0/1046325217 learned_addr learned my addr 172.21.15.43:0/1046325217 (peer_addr_for_me v1:172.21.15.43:0/0) 2023-08-09T13:35:37.726 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:37.722+0000 7f72277fe640 1 --2- 172.21.15.43:0/1046325217 >> v2:172.21.15.43:3300/0 conn(0x7f72300b7df0 0x7f72300d1b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:35:37.726 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:37.723+0000 7f72349bc640 1 --2- 172.21.15.43:0/1046325217 >> v2:172.21.15.203:3300/0 conn(0x7f72300cebf0 0x7f72300cf040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:35:37.726 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:37.723+0000 7f72277fe640 1 -- 172.21.15.43:0/1046325217 >> v2:172.21.15.203:3300/0 conn(0x7f72300cebf0 msgr2=0x7f72300cf040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-09T13:35:37.726 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:37.723+0000 7f72277fe640 1 --2- 172.21.15.43:0/1046325217 >> v2:172.21.15.203:3300/0 conn(0x7f72300cebf0 0x7f72300cf040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:35:37.727 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:37.723+0000 7f72277fe640 1 -- 172.21.15.43:0/1046325217 >> v1:172.21.15.43:6789/0 conn(0x7f723015c2d0 legacy=0x7f72300cd020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:35:37.727 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:37.723+0000 7f72277fe640 1 -- 172.21.15.43:0/1046325217 --> v2:172.21.15.43:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f72300d0d70 con 0x7f72300b7df0 2023-08-09T13:35:37.727 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:37.725+0000 7f72277fe640 1 --2- 172.21.15.43:0/1046325217 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f72300b7df0 0x7f72300d1b30 secure :-1 s=READY pgs=407 cs=0 l=1 rev1=1 crypto rx=0x7f72300cfd60 tx=0x7f721802b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4e17726192504b62 server_cookie=0 in_seq=0 out_seq=0 2023-08-09T13:35:37.727 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:37.725+0000 7f722f7fe640 1 -- 172.21.15.43:0/1046325217 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7218031830 con 0x7f72300b7df0 2023-08-09T13:35:37.728 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:37.725+0000 7f722f7fe640 1 -- 172.21.15.43:0/1046325217 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7218031990 con 0x7f72300b7df0 2023-08-09T13:35:37.728 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:37.725+0000 7f722f7fe640 1 -- 172.21.15.43:0/1046325217 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7218031d20 con 0x7f72300b7df0 2023-08-09T13:35:37.728 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:37.726+0000 7f72351bd640 1 -- 172.21.15.43:0/1046325217 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f72300b7df0 msgr2=0x7f72300d1b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:35:37.729 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:37.726+0000 7f72351bd640 1 --2- 172.21.15.43:0/1046325217 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f72300b7df0 0x7f72300d1b30 secure :-1 s=READY pgs=407 cs=0 l=1 rev1=1 crypto rx=0x7f72300cfd60 tx=0x7f721802b8d0 comp rx=0 tx=0).stop 2023-08-09T13:35:37.729 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:37.726+0000 7f72351bd640 1 -- 172.21.15.43:0/1046325217 shutdown_connections 2023-08-09T13:35:37.729 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:37.726+0000 7f72351bd640 1 --2- 172.21.15.43:0/1046325217 >> v2:172.21.15.203:3300/0 conn(0x7f72300cebf0 0x7f72300cf040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:35:37.729 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:37.726+0000 7f72351bd640 1 --2- 172.21.15.43:0/1046325217 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f72300b7df0 0x7f72300d1b30 unknown :-1 s=CLOSED pgs=407 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:35:37.729 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:37.726+0000 7f72351bd640 1 -- 172.21.15.43:0/1046325217 >> 172.21.15.43:0/1046325217 conn(0x7f723006a230 msgr2=0x7f723006a640 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-09T13:35:37.729 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:37.726+0000 7f72351bd640 1 -- 172.21.15.43:0/1046325217 shutdown_connections 2023-08-09T13:35:37.729 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:37.726+0000 7f72351bd640 1 -- 172.21.15.43:0/1046325217 wait complete. 2023-08-09T13:35:37.730 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:37.728+0000 7f72351bd640 1 Processor -- start 2023-08-09T13:35:37.730 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:37.728+0000 7f72351bd640 1 -- start start 2023-08-09T13:35:37.731 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:37.729+0000 7f72351bd640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f72300b7df0 0x7f723014d630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:35:37.732 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:37.729+0000 7f72351bd640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f72300cebf0 0x7f723015ecf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:35:37.732 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:37.729+0000 7f72351bd640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f723015c2d0 0x7f723015f230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:35:37.732 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:37.729+0000 7f72351bd640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f723015cd80 con 0x7f723015c2d0 2023-08-09T13:35:37.732 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:37.729+0000 7f72351bd640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f723015c750 con 0x7f72300cebf0 2023-08-09T13:35:37.732 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:37.729+0000 7f72351bd640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7230156490 con 0x7f72300b7df0 2023-08-09T13:35:37.732 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:37.729+0000 7f72349bc640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f723015c2d0 0x7f723015f230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:35:37.732 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:37.729+0000 7f72349bc640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f723015c2d0 0x7f723015f230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:49934/0 (socket says 172.21.15.43:49934) 2023-08-09T13:35:37.733 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:37.729+0000 7f72349bc640 1 -- 172.21.15.43:0/431348758 learned_addr learned my addr 172.21.15.43:0/431348758 (peer_addr_for_me v2:172.21.15.43:0/0) 2023-08-09T13:35:37.734 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:37.730+0000 7f722ffff640 1 --2- 172.21.15.43:0/431348758 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f72300b7df0 0x7f723014d630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:35:37.734 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:37.730+0000 7f72349bc640 1 -- 172.21.15.43:0/431348758 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f72300b7df0 msgr2=0x7f723014d630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:35:37.734 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:37.730+0000 7f72277fe640 1 --2- 172.21.15.43:0/431348758 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f72300cebf0 0x7f723015ecf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:35:37.734 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:37.730+0000 7f72349bc640 1 --2- 172.21.15.43:0/431348758 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f72300b7df0 0x7f723014d630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:35:37.734 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:37.730+0000 7f72349bc640 1 -- 172.21.15.43:0/431348758 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f72300cebf0 msgr2=0x7f723015ecf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:35:37.735 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:37.730+0000 7f72349bc640 1 --2- 172.21.15.43:0/431348758 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f72300cebf0 0x7f723015ecf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:35:37.735 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:37.730+0000 7f72349bc640 1 -- 172.21.15.43:0/431348758 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7218009400 con 0x7f723015c2d0 2023-08-09T13:35:37.735 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:37.730+0000 7f722ffff640 1 --2- 172.21.15.43:0/431348758 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f72300b7df0 0x7f723014d630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-08-09T13:35:37.735 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:37.730+0000 7f72277fe640 1 --2- 172.21.15.43:0/431348758 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f72300cebf0 0x7f723015ecf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-08-09T13:35:37.735 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:37.730+0000 7f72349bc640 1 --2- 172.21.15.43:0/431348758 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f723015c2d0 0x7f723015f230 secure :-1 s=READY pgs=408 cs=0 l=1 rev1=1 crypto rx=0x7f7228007e40 tx=0x7f722800e4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-09T13:35:37.735 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:37.730+0000 7f722d7fa640 1 -- 172.21.15.43:0/431348758 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7228014620 con 0x7f723015c2d0 2023-08-09T13:35:37.735 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:37.730+0000 7f722d7fa640 1 -- 172.21.15.43:0/431348758 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f722800ee90 con 0x7f723015c2d0 2023-08-09T13:35:37.736 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:37.730+0000 7f722d7fa640 1 -- 172.21.15.43:0/431348758 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f72280149e0 con 0x7f723015c2d0 2023-08-09T13:35:37.736 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:37.731+0000 7f72351bd640 1 -- 172.21.15.43:0/431348758 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f723015ca10 con 0x7f723015c2d0 2023-08-09T13:35:37.736 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:37.731+0000 7f72351bd640 1 -- 172.21.15.43:0/431348758 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f723014db70 con 0x7f723015c2d0 2023-08-09T13:35:37.736 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:37.732+0000 7f72351bd640 1 -- 172.21.15.43:0/431348758 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f723015cef0 con 0x7f723015c2d0 2023-08-09T13:35:37.737 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:37.733+0000 7f722d7fa640 1 -- 172.21.15.43:0/431348758 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 5) v1 ==== 88341+0+0 (secure 0 0 0) 0x7f7228014b40 con 0x7f723015c2d0 2023-08-09T13:35:37.737 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:37.734+0000 7f722d7fa640 1 --2- 172.21.15.43:0/431348758 >> [v2:172.21.15.203:6832/830097474,v1:172.21.15.203:6833/830097474] conn(0x7f721006a080 0x7f721006c540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:35:37.737 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:37.734+0000 7f722d7fa640 1 -- 172.21.15.43:0/431348758 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 6871+0+0 (secure 0 0 0) 0x7f7228089330 con 0x7f723015c2d0 2023-08-09T13:35:37.737 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:37.735+0000 7f722ffff640 1 --2- 172.21.15.43:0/431348758 >> [v2:172.21.15.203:6832/830097474,v1:172.21.15.203:6833/830097474] conn(0x7f721006a080 0x7f721006c540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:35:37.738 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:37.736+0000 7f722ffff640 1 --2- 172.21.15.43:0/431348758 >> [v2:172.21.15.203:6832/830097474,v1:172.21.15.203:6833/830097474] conn(0x7f721006a080 0x7f721006c540 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7f7220007fd0 tx=0x7f7220009040 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-09T13:35:37.740 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:37.738+0000 7f722d7fa640 1 -- 172.21.15.43:0/431348758 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178066 (secure 0 0 0) 0x7f7228054650 con 0x7f723015c2d0 2023-08-09T13:35:37.940 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:37.937+0000 7f72351bd640 1 -- 172.21.15.43:0/431348758 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f723004e390 con 0x7f723015c2d0 2023-08-09T13:35:37.940 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:37.938+0000 7f722d7fa640 1 -- 172.21.15.43:0/431348758 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v26) v1 ==== 93+0+10 (secure 0 0 0) 0x7f7228057c70 con 0x7f723015c2d0 2023-08-09T13:35:37.941 INFO:teuthology.orchestra.run.smithi043.stdout:pg_num: 1 2023-08-09T13:35:37.943 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:37.940+0000 7f72351bd640 1 -- 172.21.15.43:0/431348758 >> [v2:172.21.15.203:6832/830097474,v1:172.21.15.203:6833/830097474] conn(0x7f721006a080 msgr2=0x7f721006c540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:35:37.944 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:37.940+0000 7f72351bd640 1 --2- 172.21.15.43:0/431348758 >> [v2:172.21.15.203:6832/830097474,v1:172.21.15.203:6833/830097474] conn(0x7f721006a080 0x7f721006c540 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7f7220007fd0 tx=0x7f7220009040 comp rx=0 tx=0).stop 2023-08-09T13:35:37.944 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:37.940+0000 7f72351bd640 1 -- 172.21.15.43:0/431348758 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f723015c2d0 msgr2=0x7f723015f230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:35:37.944 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:37.940+0000 7f72351bd640 1 --2- 172.21.15.43:0/431348758 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f723015c2d0 0x7f723015f230 secure :-1 s=READY pgs=408 cs=0 l=1 rev1=1 crypto rx=0x7f7228007e40 tx=0x7f722800e4b0 comp rx=0 tx=0).stop 2023-08-09T13:35:37.944 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:37.941+0000 7f72351bd640 1 -- 172.21.15.43:0/431348758 shutdown_connections 2023-08-09T13:35:37.944 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:37.941+0000 7f72351bd640 1 --2- 172.21.15.43:0/431348758 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f723015c2d0 0x7f723015f230 unknown :-1 s=CLOSED pgs=408 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:35:37.944 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:37.941+0000 7f72351bd640 1 --2- 172.21.15.43:0/431348758 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f72300cebf0 0x7f723015ecf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:35:37.944 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:37.941+0000 7f72351bd640 1 --2- 172.21.15.43:0/431348758 >> [v2:172.21.15.203:6832/830097474,v1:172.21.15.203:6833/830097474] conn(0x7f721006a080 0x7f721006c540 unknown :-1 s=CLOSED pgs=208 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:35:37.945 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:37.941+0000 7f72351bd640 1 --2- 172.21.15.43:0/431348758 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f72300b7df0 0x7f723014d630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:35:37.945 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:37.941+0000 7f72351bd640 1 -- 172.21.15.43:0/431348758 >> 172.21.15.43:0/431348758 conn(0x7f723006a230 msgr2=0x7f72300b25c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-09T13:35:37.945 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:37.941+0000 7f72351bd640 1 -- 172.21.15.43:0/431348758 shutdown_connections 2023-08-09T13:35:37.945 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:37.941+0000 7f72351bd640 1 -- 172.21.15.43:0/431348758 wait complete. 2023-08-09T13:35:37.963 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-08-09T13:35:38.191 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:38.188+0000 7f12b7fff640 1 Processor -- start 2023-08-09T13:35:38.191 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:38.188+0000 7f12b7fff640 1 -- start start 2023-08-09T13:35:38.192 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:38.189+0000 7f12b7fff640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f12b80b04e0 0x7f12b806ba60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:35:38.192 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:38.189+0000 7f12b7fff640 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7f12b80ca2a0 0x7f12b80ca710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:35:38.193 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:38.189+0000 7f12b7fff640 1 -- --> v1:172.21.15.203:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f12b80c8470 con 0x7f12b8160670 2023-08-09T13:35:38.193 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:38.190+0000 7f12b7fff640 1 -- --> v2:172.21.15.43:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f12b80c8a30 con 0x7f12b80b04e0 2023-08-09T13:35:38.193 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:38.190+0000 7f12b7fff640 1 -- --> v2:172.21.15.203:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f12b80cc580 con 0x7f12b80ca2a0 2023-08-09T13:35:38.193 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:38.190+0000 7f12affff640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f12b80b04e0 0x7f12b806ba60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:35:38.194 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:38.190+0000 7f12affff640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f12b80b04e0 0x7f12b806ba60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:49948/0 (socket says 172.21.15.43:49948) 2023-08-09T13:35:38.194 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:38.190+0000 7f12affff640 1 -- 172.21.15.43:0/466750717 learned_addr learned my addr 172.21.15.43:0/466750717 (peer_addr_for_me v2:172.21.15.43:0/0) 2023-08-09T13:35:38.194 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:38.190+0000 7f12affff640 1 -- 172.21.15.43:0/466750717 >> v2:172.21.15.203:3300/0 conn(0x7f12b80ca2a0 msgr2=0x7f12b80ca710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-09T13:35:38.195 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:38.190+0000 7f12affff640 1 --2- 172.21.15.43:0/466750717 >> v2:172.21.15.203:3300/0 conn(0x7f12b80ca2a0 0x7f12b80ca710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:35:38.195 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:38.190+0000 7f12affff640 1 -- 172.21.15.43:0/466750717 >> v1:172.21.15.203:6789/0 conn(0x7f12b8160670 legacy=0x7f12b806b350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-09T13:35:38.195 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:38.190+0000 7f12affff640 1 -- 172.21.15.43:0/466750717 --> v2:172.21.15.43:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f12b80cc860 con 0x7f12b80b04e0 2023-08-09T13:35:38.195 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:38.192+0000 7f12affff640 1 --2- 172.21.15.43:0/466750717 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f12b80b04e0 0x7f12b806ba60 secure :-1 s=READY pgs=409 cs=0 l=1 rev1=1 crypto rx=0x7f12b80c8420 tx=0x7f12a002b910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2deb8bdeceaa3ddd server_cookie=0 in_seq=0 out_seq=0 2023-08-09T13:35:38.197 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:38.194+0000 7f12b67fc640 1 -- 172.21.15.43:0/466750717 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f12a0031870 con 0x7f12b80b04e0 2023-08-09T13:35:38.197 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:38.195+0000 7f12b67fc640 1 -- 172.21.15.43:0/466750717 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f12a00319d0 con 0x7f12b80b04e0 2023-08-09T13:35:38.197 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:38.195+0000 7f12b67fc640 1 -- 172.21.15.43:0/466750717 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f12a0031d60 con 0x7f12b80b04e0 2023-08-09T13:35:38.198 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:38.195+0000 7f12b7fff640 1 -- 172.21.15.43:0/466750717 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f12b80b04e0 msgr2=0x7f12b806ba60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:35:38.198 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:38.195+0000 7f12b7fff640 1 --2- 172.21.15.43:0/466750717 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f12b80b04e0 0x7f12b806ba60 secure :-1 s=READY pgs=409 cs=0 l=1 rev1=1 crypto rx=0x7f12b80c8420 tx=0x7f12a002b910 comp rx=0 tx=0).stop 2023-08-09T13:35:38.198 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:38.195+0000 7f12b7fff640 1 -- 172.21.15.43:0/466750717 shutdown_connections 2023-08-09T13:35:38.198 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:38.195+0000 7f12b7fff640 1 --2- 172.21.15.43:0/466750717 >> v2:172.21.15.203:3300/0 conn(0x7f12b80ca2a0 0x7f12b80ca710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:35:38.199 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:38.195+0000 7f12b7fff640 1 --2- 172.21.15.43:0/466750717 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f12b80b04e0 0x7f12b806ba60 unknown :-1 s=CLOSED pgs=409 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:35:38.199 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:38.195+0000 7f12b7fff640 1 -- 172.21.15.43:0/466750717 >> 172.21.15.43:0/466750717 conn(0x7f12b80af9f0 msgr2=0x7f12b80afdd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-09T13:35:38.199 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:38.196+0000 7f12b7fff640 1 -- 172.21.15.43:0/466750717 shutdown_connections 2023-08-09T13:35:38.199 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:38.196+0000 7f12b7fff640 1 -- 172.21.15.43:0/466750717 wait complete. 2023-08-09T13:35:38.200 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:38.197+0000 7f12b7fff640 1 Processor -- start 2023-08-09T13:35:38.200 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:38.198+0000 7f12b7fff640 1 -- start start 2023-08-09T13:35:38.201 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:38.199+0000 7f12b7fff640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f12b80b04e0 0x7f12b815a710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:35:38.201 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:38.199+0000 7f12b7fff640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f12b80ca2a0 0x7f12b815d600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:35:38.202 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:38.199+0000 7f12b7fff640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f12b8160670 0x7f12b815ad30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:35:38.202 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:38.199+0000 7f12b7fff640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f12b815d170 con 0x7f12b80b04e0 2023-08-09T13:35:38.202 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:38.199+0000 7f12b7fff640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f12b80c4050 con 0x7f12b80ca2a0 2023-08-09T13:35:38.202 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:38.199+0000 7f12b7fff640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f12b80c42c0 con 0x7f12b8160670 2023-08-09T13:35:38.202 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:38.199+0000 7f12b6ffd640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f12b80b04e0 0x7f12b815a710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:35:38.202 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:38.199+0000 7f12b6ffd640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f12b80b04e0 0x7f12b815a710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:49960/0 (socket says 172.21.15.43:49960) 2023-08-09T13:35:38.202 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:38.199+0000 7f12b6ffd640 1 -- 172.21.15.43:0/1583814762 learned_addr learned my addr 172.21.15.43:0/1583814762 (peer_addr_for_me v2:172.21.15.43:0/0) 2023-08-09T13:35:38.203 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:38.199+0000 7f12affff640 1 --2- 172.21.15.43:0/1583814762 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f12b80ca2a0 0x7f12b815d600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:35:38.204 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:38.199+0000 7f12b77fe640 1 --2- 172.21.15.43:0/1583814762 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f12b8160670 0x7f12b815ad30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:35:38.204 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:38.199+0000 7f12b6ffd640 1 -- 172.21.15.43:0/1583814762 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f12b8160670 msgr2=0x7f12b815ad30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:35:38.204 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:38.200+0000 7f12b6ffd640 1 --2- 172.21.15.43:0/1583814762 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f12b8160670 0x7f12b815ad30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:35:38.204 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:38.200+0000 7f12b6ffd640 1 -- 172.21.15.43:0/1583814762 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f12b80ca2a0 msgr2=0x7f12b815d600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:35:38.204 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:38.200+0000 7f12b6ffd640 1 --2- 172.21.15.43:0/1583814762 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f12b80ca2a0 0x7f12b815d600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:35:38.204 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:38.200+0000 7f12b6ffd640 1 -- 172.21.15.43:0/1583814762 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f12a00094d0 con 0x7f12b80b04e0 2023-08-09T13:35:38.204 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:38.200+0000 7f12affff640 1 --2- 172.21.15.43:0/1583814762 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f12b80ca2a0 0x7f12b815d600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-08-09T13:35:38.205 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:38.200+0000 7f12b77fe640 1 --2- 172.21.15.43:0/1583814762 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f12b8160670 0x7f12b815ad30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-08-09T13:35:38.205 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:38.200+0000 7f12b6ffd640 1 --2- 172.21.15.43:0/1583814762 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f12b80b04e0 0x7f12b815a710 secure :-1 s=READY pgs=410 cs=0 l=1 rev1=1 crypto rx=0x7f12a800b420 tx=0x7f12a800b8e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-09T13:35:38.205 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:38.200+0000 7f12b4ff9640 1 -- 172.21.15.43:0/1583814762 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f12a8011830 con 0x7f12b80b04e0 2023-08-09T13:35:38.205 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:38.200+0000 7f12b4ff9640 1 -- 172.21.15.43:0/1583814762 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f12a8011990 con 0x7f12b80b04e0 2023-08-09T13:35:38.205 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:38.200+0000 7f12b7fff640 1 -- 172.21.15.43:0/1583814762 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f12b806f440 con 0x7f12b80b04e0 2023-08-09T13:35:38.205 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:38.200+0000 7f12b4ff9640 1 -- 172.21.15.43:0/1583814762 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f12a8011d40 con 0x7f12b80b04e0 2023-08-09T13:35:38.205 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:38.200+0000 7f12b7fff640 1 -- 172.21.15.43:0/1583814762 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f12b80b2e30 con 0x7f12b80b04e0 2023-08-09T13:35:38.206 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:38.202+0000 7f12b7fff640 1 -- 172.21.15.43:0/1583814762 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f12b804e390 con 0x7f12b80b04e0 2023-08-09T13:35:38.206 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:38.202+0000 7f12b4ff9640 1 -- 172.21.15.43:0/1583814762 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 5) v1 ==== 88341+0+0 (secure 0 0 0) 0x7f12a801b460 con 0x7f12b80b04e0 2023-08-09T13:35:38.206 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:38.203+0000 7f12b4ff9640 1 --2- 172.21.15.43:0/1583814762 >> [v2:172.21.15.203:6832/830097474,v1:172.21.15.203:6833/830097474] conn(0x7f129806a050 0x7f129806c510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:35:38.207 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:38.204+0000 7f12b4ff9640 1 -- 172.21.15.43:0/1583814762 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 6871+0+0 (secure 0 0 0) 0x7f12a80884a0 con 0x7f12b80b04e0 2023-08-09T13:35:38.207 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:38.204+0000 7f12affff640 1 --2- 172.21.15.43:0/1583814762 >> [v2:172.21.15.203:6832/830097474,v1:172.21.15.203:6833/830097474] conn(0x7f129806a050 0x7f129806c510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:35:38.208 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:38.205+0000 7f12affff640 1 --2- 172.21.15.43:0/1583814762 >> [v2:172.21.15.203:6832/830097474,v1:172.21.15.203:6833/830097474] conn(0x7f129806a050 0x7f129806c510 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7f12a0036000 tx=0x7f12a003c040 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-09T13:35:38.210 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:38.208+0000 7f12b4ff9640 1 -- 172.21.15.43:0/1583814762 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178066 (secure 0 0 0) 0x7f12a8053850 con 0x7f12b80b04e0 2023-08-09T13:35:38.403 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:38.400+0000 7f12b7fff640 1 -- 172.21.15.43:0/1583814762 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f12b80cb6e0 con 0x7f12b80b04e0 2023-08-09T13:35:38.404 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:38.402+0000 7f12b4ff9640 1 -- 172.21.15.43:0/1583814762 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v26) v1 ==== 104+0+11 (secure 0 0 0) 0x7f12a8056e70 con 0x7f12b80b04e0 2023-08-09T13:35:38.404 INFO:teuthology.orchestra.run.smithi043.stdout:pg_num: 32 2023-08-09T13:35:38.407 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:38.404+0000 7f12b7fff640 1 -- 172.21.15.43:0/1583814762 >> [v2:172.21.15.203:6832/830097474,v1:172.21.15.203:6833/830097474] conn(0x7f129806a050 msgr2=0x7f129806c510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:35:38.407 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:38.404+0000 7f12b7fff640 1 --2- 172.21.15.43:0/1583814762 >> [v2:172.21.15.203:6832/830097474,v1:172.21.15.203:6833/830097474] conn(0x7f129806a050 0x7f129806c510 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7f12a0036000 tx=0x7f12a003c040 comp rx=0 tx=0).stop 2023-08-09T13:35:38.407 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:38.404+0000 7f12b7fff640 1 -- 172.21.15.43:0/1583814762 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f12b80b04e0 msgr2=0x7f12b815a710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:35:38.408 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:38.404+0000 7f12b7fff640 1 --2- 172.21.15.43:0/1583814762 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f12b80b04e0 0x7f12b815a710 secure :-1 s=READY pgs=410 cs=0 l=1 rev1=1 crypto rx=0x7f12a800b420 tx=0x7f12a800b8e0 comp rx=0 tx=0).stop 2023-08-09T13:35:38.408 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:38.405+0000 7f12b7fff640 1 -- 172.21.15.43:0/1583814762 shutdown_connections 2023-08-09T13:35:38.408 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:38.405+0000 7f12b7fff640 1 --2- 172.21.15.43:0/1583814762 >> [v2:172.21.15.203:6832/830097474,v1:172.21.15.203:6833/830097474] conn(0x7f129806a050 0x7f129806c510 unknown :-1 s=CLOSED pgs=209 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:35:38.408 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:38.405+0000 7f12b7fff640 1 --2- 172.21.15.43:0/1583814762 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f12b8160670 0x7f12b815ad30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:35:38.408 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:38.405+0000 7f12b7fff640 1 --2- 172.21.15.43:0/1583814762 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f12b80ca2a0 0x7f12b815d600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:35:38.408 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:38.405+0000 7f12b7fff640 1 --2- 172.21.15.43:0/1583814762 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f12b80b04e0 0x7f12b815a710 unknown :-1 s=CLOSED pgs=410 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:35:38.408 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:38.405+0000 7f12b7fff640 1 -- 172.21.15.43:0/1583814762 >> 172.21.15.43:0/1583814762 conn(0x7f12b80af9f0 msgr2=0x7f12b8070f00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-09T13:35:38.409 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:38.405+0000 7f12b7fff640 1 -- 172.21.15.43:0/1583814762 shutdown_connections 2023-08-09T13:35:38.409 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:38.405+0000 7f12b7fff640 1 -- 172.21.15.43:0/1583814762 wait complete. 2023-08-09T13:35:38.428 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-08-09T13:35:38.663 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:38.661+0000 7fc25b406640 1 Processor -- start 2023-08-09T13:35:38.665 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:38.661+0000 7fc25b406640 1 -- start start 2023-08-09T13:35:38.667 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:38.663+0000 7fc25b406640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fc2540d04b0 0x7fc2540cd6d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:35:38.667 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:38.663+0000 7fc25b406640 1 -- --> v1:172.21.15.43:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc2540b7bd0 con 0x7fc25415c550 2023-08-09T13:35:38.667 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:38.663+0000 7fc25b406640 1 -- --> v1:172.21.15.203:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc2540d1c00 con 0x7fc2540b45f0 2023-08-09T13:35:38.667 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:38.663+0000 7fc25b406640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc25415c930 con 0x7fc2540d04b0 2023-08-09T13:35:38.667 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:38.663+0000 7fc2591a2640 1 --1- >> v1:172.21.15.43:6789/0 conn(0x7fc25415c550 0x7fc2540b3ee0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.43:6789/0 says I am v1:172.21.15.43:44908/0 (socket says 172.21.15.43:44908) 2023-08-09T13:35:38.667 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:38.663+0000 7fc2591a2640 1 -- 172.21.15.43:0/2217839805 learned_addr learned my addr 172.21.15.43:0/2217839805 (peer_addr_for_me v1:172.21.15.43:0/0) 2023-08-09T13:35:38.668 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:38.663+0000 7fc2599a3640 1 --2- 172.21.15.43:0/2217839805 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fc2540d04b0 0x7fc2540cd6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:35:38.668 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:38.663+0000 7fc24bfff640 1 -- 172.21.15.43:0/2217839805 <== mon.0 v1:172.21.15.43:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 283264727 0 0) 0x7fc2540b7bd0 con 0x7fc25415c550 2023-08-09T13:35:38.668 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:38.664+0000 7fc24bfff640 1 -- 172.21.15.43:0/2217839805 --> v1:172.21.15.43:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fc240003520 con 0x7fc25415c550 2023-08-09T13:35:38.668 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:38.664+0000 7fc24bfff640 1 -- 172.21.15.43:0/2217839805 <== mon.0 v1:172.21.15.43:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3254954233 0 0) 0x7fc240003520 con 0x7fc25415c550 2023-08-09T13:35:38.668 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:38.664+0000 7fc24bfff640 1 -- 172.21.15.43:0/2217839805 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fc2540d04b0 msgr2=0x7fc2540cd6d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-09T13:35:38.668 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:38.664+0000 7fc24bfff640 1 --2- 172.21.15.43:0/2217839805 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fc2540d04b0 0x7fc2540cd6d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:35:38.668 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:38.664+0000 7fc24bfff640 1 -- 172.21.15.43:0/2217839805 >> v1:172.21.15.203:6789/0 conn(0x7fc2540b45f0 legacy=0x7fc2540b59d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:35:38.668 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:38.664+0000 7fc24bfff640 1 -- 172.21.15.43:0/2217839805 --> v1:172.21.15.43:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc2540b54f0 con 0x7fc25415c550 2023-08-09T13:35:38.669 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:38.664+0000 7fc24bfff640 1 -- 172.21.15.43:0/2217839805 <== mon.0 v1:172.21.15.43:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1285150740 0 0) 0x7fc2500019f0 con 0x7fc25415c550 2023-08-09T13:35:38.669 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:38.664+0000 7fc24bfff640 1 -- 172.21.15.43:0/2217839805 >> v1:172.21.15.43:6789/0 conn(0x7fc25415c550 legacy=0x7fc2540b3ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:35:38.669 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:38.664+0000 7fc24bfff640 1 --2- 172.21.15.43:0/2217839805 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc2400039a0 0x7fc240003dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:35:38.669 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:38.664+0000 7fc24bfff640 1 --2- 172.21.15.43:0/2217839805 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fc2540d04b0 0x7fc2400047c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:35:38.669 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:38.664+0000 7fc24bfff640 1 --2- 172.21.15.43:0/2217839805 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fc240004d00 0x7fc240005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:35:38.669 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:38.664+0000 7fc24bfff640 1 -- 172.21.15.43:0/2217839805 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc25415c930 con 0x7fc2400039a0 2023-08-09T13:35:38.669 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:38.664+0000 7fc24bfff640 1 -- 172.21.15.43:0/2217839805 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc24000ba30 con 0x7fc240004d00 2023-08-09T13:35:38.670 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:38.664+0000 7fc24bfff640 1 -- 172.21.15.43:0/2217839805 --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc24000bbf0 con 0x7fc2540d04b0 2023-08-09T13:35:38.670 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:38.664+0000 7fc2591a2640 1 --2- 172.21.15.43:0/2217839805 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc2400039a0 0x7fc240003dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:35:38.671 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:38.665+0000 7fc2599a3640 1 --2- 172.21.15.43:0/2217839805 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fc240004d00 0x7fc240005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:35:38.671 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:38.665+0000 7fc2589a1640 1 --2- 172.21.15.43:0/2217839805 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fc2540d04b0 0x7fc2400047c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:35:38.671 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:38.665+0000 7fc2591a2640 1 -- 172.21.15.43:0/2217839805 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fc2540d04b0 msgr2=0x7fc2400047c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-09T13:35:38.671 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:38.665+0000 7fc2591a2640 1 --2- 172.21.15.43:0/2217839805 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fc2540d04b0 0x7fc2400047c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:35:38.671 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:38.665+0000 7fc2591a2640 1 -- 172.21.15.43:0/2217839805 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fc240004d00 msgr2=0x7fc240005180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-09T13:35:38.672 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:38.665+0000 7fc2591a2640 1 --2- 172.21.15.43:0/2217839805 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fc240004d00 0x7fc240005180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:35:38.672 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:38.665+0000 7fc2591a2640 1 -- 172.21.15.43:0/2217839805 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc240003520 con 0x7fc2400039a0 2023-08-09T13:35:38.672 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:38.665+0000 7fc2599a3640 1 --2- 172.21.15.43:0/2217839805 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fc240004d00 0x7fc240005180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-08-09T13:35:38.672 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:38.666+0000 7fc2591a2640 1 --2- 172.21.15.43:0/2217839805 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc2400039a0 0x7fc240003dc0 secure :-1 s=READY pgs=412 cs=0 l=1 rev1=1 crypto rx=0x7fc25000a7f0 tx=0x7fc2400057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=abcbfe1d0c0308cb server_cookie=0 in_seq=0 out_seq=0 2023-08-09T13:35:38.672 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:38.666+0000 7fc24bfff640 1 -- 172.21.15.43:0/2217839805 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc2500072d0 con 0x7fc2400039a0 2023-08-09T13:35:38.672 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:38.666+0000 7fc24bfff640 1 -- 172.21.15.43:0/2217839805 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc25002f070 con 0x7fc2400039a0 2023-08-09T13:35:38.672 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:38.666+0000 7fc24bfff640 1 -- 172.21.15.43:0/2217839805 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc2500076b0 con 0x7fc2400039a0 2023-08-09T13:35:38.673 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:38.667+0000 7fc25b406640 1 -- 172.21.15.43:0/2217839805 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc2400039a0 msgr2=0x7fc240003dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:35:38.673 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:38.667+0000 7fc25b406640 1 --2- 172.21.15.43:0/2217839805 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc2400039a0 0x7fc240003dc0 secure :-1 s=READY pgs=412 cs=0 l=1 rev1=1 crypto rx=0x7fc25000a7f0 tx=0x7fc2400057f0 comp rx=0 tx=0).stop 2023-08-09T13:35:38.673 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:38.667+0000 7fc2599a3640 1 -- 172.21.15.43:0/2217839805 reap_dead start 2023-08-09T13:35:38.673 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:38.667+0000 7fc25b406640 1 -- 172.21.15.43:0/2217839805 shutdown_connections 2023-08-09T13:35:38.673 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:38.667+0000 7fc25b406640 1 -- 172.21.15.43:0/2217839805 >> 172.21.15.43:0/2217839805 conn(0x7fc254071520 msgr2=0x7fc2540ceee0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-09T13:35:38.673 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:38.667+0000 7fc25b406640 1 -- 172.21.15.43:0/2217839805 shutdown_connections 2023-08-09T13:35:38.673 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:38.667+0000 7fc25b406640 1 -- 172.21.15.43:0/2217839805 wait complete. 2023-08-09T13:35:38.675 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:38.669+0000 7fc25b406640 1 Processor -- start 2023-08-09T13:35:38.675 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:38.669+0000 7fc25b406640 1 -- start start 2023-08-09T13:35:38.675 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:38.670+0000 7fc25b406640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc2400039a0 0x7fc254148e90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:35:38.676 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:38.670+0000 7fc25b406640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fc2540a9f50 0x7fc2540aa3d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:35:38.676 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:38.670+0000 7fc25b406640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fc2540a8dc0 0x7fc2540a9240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:35:38.676 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:38.670+0000 7fc25b406640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc2540a9dd0 con 0x7fc2400039a0 2023-08-09T13:35:38.676 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:38.670+0000 7fc25b406640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc2540aa910 con 0x7fc2540a8dc0 2023-08-09T13:35:38.676 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:38.670+0000 7fc25b406640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc25414cc60 con 0x7fc2540a9f50 2023-08-09T13:35:38.676 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:38.670+0000 7fc2591a2640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc2400039a0 0x7fc254148e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:35:38.676 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:38.670+0000 7fc2591a2640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc2400039a0 0x7fc254148e90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:49980/0 (socket says 172.21.15.43:49980) 2023-08-09T13:35:38.677 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:38.670+0000 7fc2591a2640 1 -- 172.21.15.43:0/2409741731 learned_addr learned my addr 172.21.15.43:0/2409741731 (peer_addr_for_me v2:172.21.15.43:0/0) 2023-08-09T13:35:38.677 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:38.671+0000 7fc2599a3640 1 --2- 172.21.15.43:0/2409741731 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fc2540a8dc0 0x7fc2540a9240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:35:38.677 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:38.671+0000 7fc2589a1640 1 --2- 172.21.15.43:0/2409741731 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fc2540a9f50 0x7fc2540aa3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:35:38.677 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:38.671+0000 7fc2591a2640 1 -- 172.21.15.43:0/2409741731 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fc2540a9f50 msgr2=0x7fc2540aa3d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:35:38.677 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:38.671+0000 7fc2591a2640 1 --2- 172.21.15.43:0/2409741731 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fc2540a9f50 0x7fc2540aa3d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:35:38.677 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:38.671+0000 7fc2591a2640 1 -- 172.21.15.43:0/2409741731 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fc2540a8dc0 msgr2=0x7fc2540a9240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:35:38.677 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:38.671+0000 7fc2591a2640 1 --2- 172.21.15.43:0/2409741731 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fc2540a8dc0 0x7fc2540a9240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:35:38.677 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:38.671+0000 7fc2591a2640 1 -- 172.21.15.43:0/2409741731 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc2540a9af0 con 0x7fc2400039a0 2023-08-09T13:35:38.678 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:38.671+0000 7fc2599a3640 1 --2- 172.21.15.43:0/2409741731 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fc2540a8dc0 0x7fc2540a9240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-08-09T13:35:38.678 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:38.671+0000 7fc2591a2640 1 --2- 172.21.15.43:0/2409741731 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc2400039a0 0x7fc254148e90 secure :-1 s=READY pgs=413 cs=0 l=1 rev1=1 crypto rx=0x7fc25000a7f0 tx=0x7fc2540b5680 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-09T13:35:38.678 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:38.671+0000 7fc249ffb640 1 -- 172.21.15.43:0/2409741731 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc250001250 con 0x7fc2400039a0 2023-08-09T13:35:38.678 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:38.672+0000 7fc249ffb640 1 -- 172.21.15.43:0/2409741731 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc2500072e0 con 0x7fc2400039a0 2023-08-09T13:35:38.678 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:38.672+0000 7fc249ffb640 1 -- 172.21.15.43:0/2409741731 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc250007670 con 0x7fc2400039a0 2023-08-09T13:35:38.678 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:38.672+0000 7fc25b406640 1 -- 172.21.15.43:0/2409741731 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc254149910 con 0x7fc2400039a0 2023-08-09T13:35:38.678 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:38.672+0000 7fc25b406640 1 -- 172.21.15.43:0/2409741731 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc254149ed0 con 0x7fc2400039a0 2023-08-09T13:35:38.678 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:38.673+0000 7fc25b406640 1 -- 172.21.15.43:0/2409741731 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc254149770 con 0x7fc2400039a0 2023-08-09T13:35:38.679 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:38.674+0000 7fc249ffb640 1 -- 172.21.15.43:0/2409741731 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 5) v1 ==== 88341+0+0 (secure 0 0 0) 0x7fc250031030 con 0x7fc2400039a0 2023-08-09T13:35:38.679 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:38.675+0000 7fc249ffb640 1 --2- 172.21.15.43:0/2409741731 >> [v2:172.21.15.203:6832/830097474,v1:172.21.15.203:6833/830097474] conn(0x7fc23806a080 0x7fc23806c540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:35:38.680 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:38.675+0000 7fc249ffb640 1 -- 172.21.15.43:0/2409741731 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 6871+0+0 (secure 0 0 0) 0x7fc2500aa240 con 0x7fc2400039a0 2023-08-09T13:35:38.680 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:38.676+0000 7fc2589a1640 1 --2- 172.21.15.43:0/2409741731 >> [v2:172.21.15.203:6832/830097474,v1:172.21.15.203:6833/830097474] conn(0x7fc23806a080 0x7fc23806c540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:35:38.680 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:38.677+0000 7fc2589a1640 1 --2- 172.21.15.43:0/2409741731 >> [v2:172.21.15.203:6832/830097474,v1:172.21.15.203:6833/830097474] conn(0x7fc23806a080 0x7fc23806c540 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7fc244007fd0 tx=0x7fc244009040 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-09T13:35:38.681 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:38.679+0000 7fc249ffb640 1 -- 172.21.15.43:0/2409741731 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178066 (secure 0 0 0) 0x7fc254149770 con 0x7fc2400039a0 2023-08-09T13:35:38.875 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:38.873+0000 7fc25b406640 1 -- 172.21.15.43:0/2409741731 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7fc254149770 con 0x7fc2400039a0 2023-08-09T13:35:38.877 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:38.874+0000 7fc249ffb640 1 -- 172.21.15.43:0/2409741731 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v26) v1 ==== 100+0+11 (secure 0 0 0) 0x7fc254149770 con 0x7fc2400039a0 2023-08-09T13:35:38.877 INFO:teuthology.orchestra.run.smithi043.stdout:pg_num: 64 2023-08-09T13:35:38.879 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:38.876+0000 7fc25b406640 1 -- 172.21.15.43:0/2409741731 >> [v2:172.21.15.203:6832/830097474,v1:172.21.15.203:6833/830097474] conn(0x7fc23806a080 msgr2=0x7fc23806c540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:35:38.879 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:38.876+0000 7fc25b406640 1 --2- 172.21.15.43:0/2409741731 >> [v2:172.21.15.203:6832/830097474,v1:172.21.15.203:6833/830097474] conn(0x7fc23806a080 0x7fc23806c540 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7fc244007fd0 tx=0x7fc244009040 comp rx=0 tx=0).stop 2023-08-09T13:35:38.879 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:38.876+0000 7fc25b406640 1 -- 172.21.15.43:0/2409741731 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc2400039a0 msgr2=0x7fc254148e90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:35:38.879 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:38.876+0000 7fc25b406640 1 --2- 172.21.15.43:0/2409741731 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc2400039a0 0x7fc254148e90 secure :-1 s=READY pgs=413 cs=0 l=1 rev1=1 crypto rx=0x7fc25000a7f0 tx=0x7fc2540b5680 comp rx=0 tx=0).stop 2023-08-09T13:35:38.880 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:38.877+0000 7fc25b406640 1 -- 172.21.15.43:0/2409741731 shutdown_connections 2023-08-09T13:35:38.880 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:38.877+0000 7fc25b406640 1 --2- 172.21.15.43:0/2409741731 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fc2540a8dc0 0x7fc2540a9240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:35:38.880 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:38.877+0000 7fc25b406640 1 --2- 172.21.15.43:0/2409741731 >> [v2:172.21.15.203:6832/830097474,v1:172.21.15.203:6833/830097474] conn(0x7fc23806a080 0x7fc23806c540 unknown :-1 s=CLOSED pgs=210 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:35:38.880 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:38.877+0000 7fc25b406640 1 --2- 172.21.15.43:0/2409741731 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fc2540a9f50 0x7fc2540aa3d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:35:38.880 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:38.877+0000 7fc25b406640 1 --2- 172.21.15.43:0/2409741731 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc2400039a0 0x7fc254148e90 unknown :-1 s=CLOSED pgs=413 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:35:38.880 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:38.877+0000 7fc25b406640 1 -- 172.21.15.43:0/2409741731 >> 172.21.15.43:0/2409741731 conn(0x7fc254071520 msgr2=0x7fc2540cd410 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-09T13:35:38.881 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:38.877+0000 7fc25b406640 1 -- 172.21.15.43:0/2409741731 shutdown_connections 2023-08-09T13:35:38.881 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:38.877+0000 7fc25b406640 1 -- 172.21.15.43:0/2409741731 wait complete. 2023-08-09T13:35:38.898 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2023-08-09T13:35:39.132 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:39.130+0000 7f90b4ae0640 1 Processor -- start 2023-08-09T13:35:39.133 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:39.130+0000 7f90b4ae0640 1 -- start start 2023-08-09T13:35:39.134 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:39.131+0000 7f90b4ae0640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f90b015c3a0 0x7f90b01608a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:35:39.134 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:39.132+0000 7f90b4ae0640 1 -- --> v1:172.21.15.43:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f90b0160de0 con 0x7f90b00b0f00 2023-08-09T13:35:39.134 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:39.132+0000 7f90b4ae0640 1 -- --> v1:172.21.15.203:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f90b00b7c00 con 0x7f90b00cd600 2023-08-09T13:35:39.134 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:39.132+0000 7f90b4ae0640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f90b00b81c0 con 0x7f90b015c3a0 2023-08-09T13:35:39.135 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:39.132+0000 7f90aeffd640 1 --1- >> v1:172.21.15.43:6789/0 conn(0x7f90b00b0f00 0x7f90b00ccef0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.43:6789/0 says I am v1:172.21.15.43:44910/0 (socket says 172.21.15.43:44910) 2023-08-09T13:35:39.136 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:39.132+0000 7f90aeffd640 1 -- 172.21.15.43:0/1131348103 learned_addr learned my addr 172.21.15.43:0/1131348103 (peer_addr_for_me v1:172.21.15.43:0/0) 2023-08-09T13:35:39.136 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:39.132+0000 7f90af7fe640 1 --2- 172.21.15.43:0/1131348103 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f90b015c3a0 0x7f90b01608a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:35:39.136 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:39.132+0000 7f90ae7fc640 1 -- 172.21.15.43:0/1131348103 <== mon.0 v1:172.21.15.43:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3674727526 0 0) 0x7f90b0160de0 con 0x7f90b00b0f00 2023-08-09T13:35:39.136 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:39.133+0000 7f90ae7fc640 1 -- 172.21.15.43:0/1131348103 --> v1:172.21.15.43:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f9098003520 con 0x7f90b00b0f00 2023-08-09T13:35:39.136 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:39.133+0000 7f90ae7fc640 1 -- 172.21.15.43:0/1131348103 <== mon.0 v1:172.21.15.43:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1473190356 0 0) 0x7f9098003520 con 0x7f90b00b0f00 2023-08-09T13:35:39.136 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:39.133+0000 7f90ae7fc640 1 -- 172.21.15.43:0/1131348103 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f90b015c3a0 msgr2=0x7f90b01608a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-09T13:35:39.136 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:39.133+0000 7f90ae7fc640 1 --2- 172.21.15.43:0/1131348103 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f90b015c3a0 0x7f90b01608a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:35:39.137 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:39.133+0000 7f90ae7fc640 1 -- 172.21.15.43:0/1131348103 >> v1:172.21.15.203:6789/0 conn(0x7f90b00cd600 legacy=0x7f90b00d0580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:35:39.137 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:39.133+0000 7f90ae7fc640 1 -- 172.21.15.43:0/1131348103 --> v1:172.21.15.43:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f90b00b7ee0 con 0x7f90b00b0f00 2023-08-09T13:35:39.137 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:39.133+0000 7f90ae7fc640 1 -- 172.21.15.43:0/1131348103 <== mon.0 v1:172.21.15.43:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1285150740 0 0) 0x7f90a40017f0 con 0x7f90b00b0f00 2023-08-09T13:35:39.137 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:39.133+0000 7f90ae7fc640 1 -- 172.21.15.43:0/1131348103 >> v1:172.21.15.43:6789/0 conn(0x7f90b00b0f00 legacy=0x7f90b00ccef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:35:39.137 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:39.133+0000 7f90ae7fc640 1 --2- 172.21.15.43:0/1131348103 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f90980039a0 0x7f9098003dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:35:39.137 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:39.133+0000 7f90ae7fc640 1 --2- 172.21.15.43:0/1131348103 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f90b015c3a0 0x7f90980047c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:35:39.137 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:39.133+0000 7f90ae7fc640 1 --2- 172.21.15.43:0/1131348103 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f9098004d00 0x7f9098005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:35:39.138 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:39.133+0000 7f90ae7fc640 1 -- 172.21.15.43:0/1131348103 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f90b00b81c0 con 0x7f90980039a0 2023-08-09T13:35:39.138 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:39.133+0000 7f90ae7fc640 1 -- 172.21.15.43:0/1131348103 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f909800bab0 con 0x7f9098004d00 2023-08-09T13:35:39.138 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:39.133+0000 7f90ae7fc640 1 -- 172.21.15.43:0/1131348103 --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f909800bc70 con 0x7f90b015c3a0 2023-08-09T13:35:39.139 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:39.133+0000 7f90af7fe640 1 --2- 172.21.15.43:0/1131348103 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f90980039a0 0x7f9098003dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:35:39.139 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:39.134+0000 7f90aeffd640 1 --2- 172.21.15.43:0/1131348103 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f90b015c3a0 0x7f90980047c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:35:39.140 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:39.134+0000 7f90affff640 1 --2- 172.21.15.43:0/1131348103 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f9098004d00 0x7f9098005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:35:39.140 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:39.134+0000 7f90af7fe640 1 -- 172.21.15.43:0/1131348103 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f90b015c3a0 msgr2=0x7f90980047c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-09T13:35:39.140 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:39.134+0000 7f90af7fe640 1 --2- 172.21.15.43:0/1131348103 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f90b015c3a0 0x7f90980047c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:35:39.140 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:39.134+0000 7f90af7fe640 1 -- 172.21.15.43:0/1131348103 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f9098004d00 msgr2=0x7f9098005180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-09T13:35:39.140 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:39.134+0000 7f90af7fe640 1 --2- 172.21.15.43:0/1131348103 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f9098004d00 0x7f9098005180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:35:39.140 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:39.134+0000 7f90af7fe640 1 -- 172.21.15.43:0/1131348103 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9098003520 con 0x7f90980039a0 2023-08-09T13:35:39.140 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:39.134+0000 7f90aeffd640 1 --2- 172.21.15.43:0/1131348103 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f90b015c3a0 0x7f90980047c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-08-09T13:35:39.141 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:39.134+0000 7f90affff640 1 --2- 172.21.15.43:0/1131348103 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f9098004d00 0x7f9098005180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-08-09T13:35:39.141 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:39.135+0000 7f90af7fe640 1 --2- 172.21.15.43:0/1131348103 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f90980039a0 0x7f9098003dc0 secure :-1 s=READY pgs=415 cs=0 l=1 rev1=1 crypto rx=0x7f90a0009fc0 tx=0x7f90980057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=81719f1f0f472c48 server_cookie=0 in_seq=0 out_seq=0 2023-08-09T13:35:39.141 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:39.135+0000 7f90ae7fc640 1 -- 172.21.15.43:0/1131348103 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f90a002cb40 con 0x7f90980039a0 2023-08-09T13:35:39.141 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:39.136+0000 7f90ae7fc640 1 -- 172.21.15.43:0/1131348103 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f90a000f040 con 0x7f90980039a0 2023-08-09T13:35:39.141 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:39.136+0000 7f90ae7fc640 1 -- 172.21.15.43:0/1131348103 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f90a0034440 con 0x7f90980039a0 2023-08-09T13:35:39.141 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:39.136+0000 7f90b4ae0640 1 -- 172.21.15.43:0/1131348103 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f90980039a0 msgr2=0x7f9098003dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:35:39.141 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:39.136+0000 7f90b4ae0640 1 --2- 172.21.15.43:0/1131348103 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f90980039a0 0x7f9098003dc0 secure :-1 s=READY pgs=415 cs=0 l=1 rev1=1 crypto rx=0x7f90a0009fc0 tx=0x7f90980057f0 comp rx=0 tx=0).stop 2023-08-09T13:35:39.142 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:39.136+0000 7f90affff640 1 -- 172.21.15.43:0/1131348103 reap_dead start 2023-08-09T13:35:39.142 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:39.136+0000 7f90b4ae0640 1 -- 172.21.15.43:0/1131348103 shutdown_connections 2023-08-09T13:35:39.142 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:39.136+0000 7f90b4ae0640 1 -- 172.21.15.43:0/1131348103 >> 172.21.15.43:0/1131348103 conn(0x7f90b006a210 msgr2=0x7f90b006a620 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-09T13:35:39.142 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:39.136+0000 7f90b4ae0640 1 -- 172.21.15.43:0/1131348103 shutdown_connections 2023-08-09T13:35:39.142 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:39.136+0000 7f90b4ae0640 1 -- 172.21.15.43:0/1131348103 wait complete. 2023-08-09T13:35:39.144 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:39.138+0000 7f90b4ae0640 1 Processor -- start 2023-08-09T13:35:39.144 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:39.138+0000 7f90b4ae0640 1 -- start start 2023-08-09T13:35:39.144 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:39.139+0000 7f90b4ae0640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f90980039a0 0x7f90b0153880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:35:39.144 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:39.139+0000 7f90b4ae0640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f90b0158aa0 0x7f90b00b2190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:35:39.144 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:39.139+0000 7f90b4ae0640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f90b0158f20 0x7f90b00b26d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:35:39.144 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:39.139+0000 7f90b4ae0640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f90b00cb470 con 0x7f90980039a0 2023-08-09T13:35:39.145 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:39.139+0000 7f90b4ae0640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f90b00cb750 con 0x7f90b0158aa0 2023-08-09T13:35:39.145 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:39.139+0000 7f90b4ae0640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f90b00cba30 con 0x7f90b0158f20 2023-08-09T13:35:39.145 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:39.140+0000 7f90af7fe640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f90980039a0 0x7f90b0153880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:35:39.145 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:39.140+0000 7f90af7fe640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f90980039a0 0x7f90b0153880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:50004/0 (socket says 172.21.15.43:50004) 2023-08-09T13:35:39.145 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:39.140+0000 7f90af7fe640 1 -- 172.21.15.43:0/3700054391 learned_addr learned my addr 172.21.15.43:0/3700054391 (peer_addr_for_me v2:172.21.15.43:0/0) 2023-08-09T13:35:39.145 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:39.140+0000 7f90aeffd640 1 --2- 172.21.15.43:0/3700054391 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f90b0158aa0 0x7f90b00b2190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:35:39.145 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:39.140+0000 7f90affff640 1 --2- 172.21.15.43:0/3700054391 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f90b0158f20 0x7f90b00b26d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:35:39.145 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:39.140+0000 7f90af7fe640 1 -- 172.21.15.43:0/3700054391 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f90b0158f20 msgr2=0x7f90b00b26d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:35:39.146 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:39.140+0000 7f90af7fe640 1 --2- 172.21.15.43:0/3700054391 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f90b0158f20 0x7f90b00b26d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:35:39.146 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:39.140+0000 7f90af7fe640 1 -- 172.21.15.43:0/3700054391 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f90b0158aa0 msgr2=0x7f90b00b2190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:35:39.146 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:39.140+0000 7f90af7fe640 1 --2- 172.21.15.43:0/3700054391 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f90b0158aa0 0x7f90b00b2190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:35:39.146 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:39.140+0000 7f90af7fe640 1 -- 172.21.15.43:0/3700054391 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f90b00cbd10 con 0x7f90980039a0 2023-08-09T13:35:39.146 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:39.141+0000 7f90af7fe640 1 --2- 172.21.15.43:0/3700054391 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f90980039a0 0x7f90b0153880 secure :-1 s=READY pgs=416 cs=0 l=1 rev1=1 crypto rx=0x7f90a0037040 tx=0x7f90a0009750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-09T13:35:39.146 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:39.141+0000 7f9097fff640 1 -- 172.21.15.43:0/3700054391 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f90a0034c40 con 0x7f90980039a0 2023-08-09T13:35:39.146 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:39.141+0000 7f9097fff640 1 -- 172.21.15.43:0/3700054391 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f90a0034da0 con 0x7f90980039a0 2023-08-09T13:35:39.146 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:39.141+0000 7f90b4ae0640 1 -- 172.21.15.43:0/3700054391 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f90b00cbff0 con 0x7f90980039a0 2023-08-09T13:35:39.147 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:39.141+0000 7f9097fff640 1 -- 172.21.15.43:0/3700054391 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f90a0003bd0 con 0x7f90980039a0 2023-08-09T13:35:39.147 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:39.141+0000 7f90b4ae0640 1 -- 172.21.15.43:0/3700054391 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f90b00cb090 con 0x7f90980039a0 2023-08-09T13:35:39.147 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:39.142+0000 7f90b4ae0640 1 -- 172.21.15.43:0/3700054391 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f90b004e390 con 0x7f90980039a0 2023-08-09T13:35:39.148 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:39.143+0000 7f9097fff640 1 -- 172.21.15.43:0/3700054391 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 5) v1 ==== 88341+0+0 (secure 0 0 0) 0x7f90a0003d30 con 0x7f90980039a0 2023-08-09T13:35:39.148 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:39.144+0000 7f9097fff640 1 --2- 172.21.15.43:0/3700054391 >> [v2:172.21.15.203:6832/830097474,v1:172.21.15.203:6833/830097474] conn(0x7f909c06a080 0x7f909c06c540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:35:39.148 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:39.144+0000 7f9097fff640 1 -- 172.21.15.43:0/3700054391 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 6871+0+0 (secure 0 0 0) 0x7f90a00aa6b0 con 0x7f90980039a0 2023-08-09T13:35:39.148 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:39.145+0000 7f90aeffd640 1 --2- 172.21.15.43:0/3700054391 >> [v2:172.21.15.203:6832/830097474,v1:172.21.15.203:6833/830097474] conn(0x7f909c06a080 0x7f909c06c540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:35:39.148 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:39.146+0000 7f90aeffd640 1 --2- 172.21.15.43:0/3700054391 >> [v2:172.21.15.203:6832/830097474,v1:172.21.15.203:6833/830097474] conn(0x7f909c06a080 0x7f909c06c540 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7f90b01599d0 tx=0x7f90a4009040 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-09T13:35:39.151 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:39.149+0000 7f9097fff640 1 -- 172.21.15.43:0/3700054391 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178066 (secure 0 0 0) 0x7f90a0075af0 con 0x7f90980039a0 2023-08-09T13:35:39.350 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:39.347+0000 7f90b4ae0640 1 -- 172.21.15.43:0/3700054391 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) v1 -- 0x7f90b00d18f0 con 0x7f90980039a0 2023-08-09T13:35:39.351 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:39.348+0000 7f9097fff640 1 -- 172.21.15.43:0/3700054391 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v26) v1 ==== 103+0+11 (secure 0 0 0) 0x7f90a0079110 con 0x7f90980039a0 2023-08-09T13:35:39.351 INFO:teuthology.orchestra.run.smithi043.stdout:pg_num: 32 2023-08-09T13:35:39.353 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:39.351+0000 7f90b4ae0640 1 -- 172.21.15.43:0/3700054391 >> [v2:172.21.15.203:6832/830097474,v1:172.21.15.203:6833/830097474] conn(0x7f909c06a080 msgr2=0x7f909c06c540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:35:39.353 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:39.351+0000 7f90b4ae0640 1 --2- 172.21.15.43:0/3700054391 >> [v2:172.21.15.203:6832/830097474,v1:172.21.15.203:6833/830097474] conn(0x7f909c06a080 0x7f909c06c540 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7f90b01599d0 tx=0x7f90a4009040 comp rx=0 tx=0).stop 2023-08-09T13:35:39.353 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:39.351+0000 7f90b4ae0640 1 -- 172.21.15.43:0/3700054391 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f90980039a0 msgr2=0x7f90b0153880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:35:39.354 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:39.351+0000 7f90b4ae0640 1 --2- 172.21.15.43:0/3700054391 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f90980039a0 0x7f90b0153880 secure :-1 s=READY pgs=416 cs=0 l=1 rev1=1 crypto rx=0x7f90a0037040 tx=0x7f90a0009750 comp rx=0 tx=0).stop 2023-08-09T13:35:39.354 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:39.351+0000 7f90b4ae0640 1 -- 172.21.15.43:0/3700054391 shutdown_connections 2023-08-09T13:35:39.354 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:39.351+0000 7f90b4ae0640 1 --2- 172.21.15.43:0/3700054391 >> [v2:172.21.15.203:6832/830097474,v1:172.21.15.203:6833/830097474] conn(0x7f909c06a080 0x7f909c06c540 unknown :-1 s=CLOSED pgs=211 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:35:39.355 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:39.351+0000 7f90b4ae0640 1 --2- 172.21.15.43:0/3700054391 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f90b0158f20 0x7f90b00b26d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:35:39.355 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:39.351+0000 7f90b4ae0640 1 --2- 172.21.15.43:0/3700054391 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f90b0158aa0 0x7f90b00b2190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:35:39.355 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:39.351+0000 7f90b4ae0640 1 --2- 172.21.15.43:0/3700054391 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f90980039a0 0x7f90b0153880 unknown :-1 s=CLOSED pgs=416 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:35:39.355 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:39.351+0000 7f90b4ae0640 1 -- 172.21.15.43:0/3700054391 >> 172.21.15.43:0/3700054391 conn(0x7f90b006a210 msgr2=0x7f90b015a330 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-09T13:35:39.355 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:39.351+0000 7f90b4ae0640 1 -- 172.21.15.43:0/3700054391 shutdown_connections 2023-08-09T13:35:39.355 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:39.351+0000 7f90b4ae0640 1 -- 172.21.15.43:0/3700054391 wait complete. 2023-08-09T13:35:39.373 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2023-08-09T13:35:39.609 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:39.606+0000 7f051100f640 1 Processor -- start 2023-08-09T13:35:39.609 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:39.606+0000 7f051100f640 1 -- start start 2023-08-09T13:35:39.610 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:39.608+0000 7f051100f640 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7f050c06d480 0x7f050c0c9360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:35:39.611 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:39.608+0000 7f051100f640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f050c0c98a0 0x7f050c0cba30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:35:39.611 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:39.608+0000 7f051100f640 1 -- --> v1:172.21.15.43:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f050c070c70 con 0x7f050c15c550 2023-08-09T13:35:39.611 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:39.608+0000 7f051100f640 1 -- --> v2:172.21.15.203:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f050c15c930 con 0x7f050c06d480 2023-08-09T13:35:39.611 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:39.608+0000 7f051100f640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f050c070060 con 0x7f050c0c98a0 2023-08-09T13:35:39.611 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:39.608+0000 7f050bfff640 1 --1- >> v1:172.21.15.43:6789/0 conn(0x7f050c15c550 0x7f050c06cd70 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.43:6789/0 says I am v1:172.21.15.43:44916/0 (socket says 172.21.15.43:44916) 2023-08-09T13:35:39.611 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:39.608+0000 7f050bfff640 1 -- 172.21.15.43:0/3964298412 learned_addr learned my addr 172.21.15.43:0/3964298412 (peer_addr_for_me v1:172.21.15.43:0/0) 2023-08-09T13:35:39.613 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:39.608+0000 7f050affd640 1 -- 172.21.15.43:0/3964298412 <== mon.0 v1:172.21.15.43:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1237587396 0 0) 0x7f050c070c70 con 0x7f050c15c550 2023-08-09T13:35:39.613 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:39.609+0000 7f050b7fe640 1 --2- 172.21.15.43:0/3964298412 >> v2:172.21.15.203:3300/0 conn(0x7f050c06d480 0x7f050c0c9360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:35:39.613 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:39.609+0000 7f050affd640 1 -- 172.21.15.43:0/3964298412 --> v1:172.21.15.43:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f04f4005730 con 0x7f050c15c550 2023-08-09T13:35:39.613 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:39.609+0000 7f051080e640 1 --2- 172.21.15.43:0/3964298412 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f050c0c98a0 0x7f050c0cba30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:35:39.614 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:39.609+0000 7f050affd640 1 -- 172.21.15.43:0/3964298412 <== mon.0 v1:172.21.15.43:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2868195181 0 0) 0x7f04f4005730 con 0x7f050c15c550 2023-08-09T13:35:39.614 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:39.609+0000 7f050affd640 1 -- 172.21.15.43:0/3964298412 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f050c0c98a0 msgr2=0x7f050c0cba30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-09T13:35:39.614 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:39.609+0000 7f050affd640 1 --2- 172.21.15.43:0/3964298412 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f050c0c98a0 0x7f050c0cba30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:35:39.614 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:39.609+0000 7f050affd640 1 -- 172.21.15.43:0/3964298412 >> v2:172.21.15.203:3300/0 conn(0x7f050c06d480 msgr2=0x7f050c0c9360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-09T13:35:39.614 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:39.609+0000 7f050affd640 1 --2- 172.21.15.43:0/3964298412 >> v2:172.21.15.203:3300/0 conn(0x7f050c06d480 0x7f050c0c9360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:35:39.615 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:39.609+0000 7f050affd640 1 -- 172.21.15.43:0/3964298412 --> v1:172.21.15.43:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f050c070340 con 0x7f050c15c550 2023-08-09T13:35:39.615 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:39.609+0000 7f050affd640 1 -- 172.21.15.43:0/3964298412 <== mon.0 v1:172.21.15.43:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1285150740 0 0) 0x7f04fc001670 con 0x7f050c15c550 2023-08-09T13:35:39.615 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:39.609+0000 7f051080e640 1 --2- 172.21.15.43:0/3964298412 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f050c0c98a0 0x7f050c0cba30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-08-09T13:35:39.615 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:39.609+0000 7f050affd640 1 -- 172.21.15.43:0/3964298412 >> v1:172.21.15.43:6789/0 conn(0x7f050c15c550 legacy=0x7f050c06cd70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:35:39.615 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:39.609+0000 7f050affd640 1 --2- 172.21.15.43:0/3964298412 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f050c0c98a0 0x7f04f4005b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:35:39.615 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:39.609+0000 7f050affd640 1 --2- 172.21.15.43:0/3964298412 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f04f40060c0 0x7f04f40069d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:35:39.615 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:39.609+0000 7f050affd640 1 --2- 172.21.15.43:0/3964298412 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f04f4006f10 0x7f04f4007390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:35:39.616 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:39.609+0000 7f050affd640 1 -- 172.21.15.43:0/3964298412 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f050c15c930 con 0x7f04f40060c0 2023-08-09T13:35:39.616 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:39.609+0000 7f050affd640 1 -- 172.21.15.43:0/3964298412 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f050c070060 con 0x7f04f4006f10 2023-08-09T13:35:39.616 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:39.609+0000 7f050affd640 1 -- 172.21.15.43:0/3964298412 --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f04f400dbc0 con 0x7f050c0c98a0 2023-08-09T13:35:39.616 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:39.610+0000 7f050b7fe640 1 --2- 172.21.15.43:0/3964298412 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f04f40060c0 0x7f04f40069d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:35:39.616 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:39.610+0000 7f050bfff640 1 --2- 172.21.15.43:0/3964298412 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f050c0c98a0 0x7f04f4005b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:35:39.616 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:39.610+0000 7f051080e640 1 --2- 172.21.15.43:0/3964298412 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f04f4006f10 0x7f04f4007390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:35:39.617 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:39.610+0000 7f050b7fe640 1 -- 172.21.15.43:0/3964298412 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f050c0c98a0 msgr2=0x7f04f4005b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-09T13:35:39.617 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:39.610+0000 7f050b7fe640 1 --2- 172.21.15.43:0/3964298412 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f050c0c98a0 0x7f04f4005b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:35:39.617 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:39.610+0000 7f050b7fe640 1 -- 172.21.15.43:0/3964298412 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f04f4006f10 msgr2=0x7f04f4007390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-09T13:35:39.617 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:39.610+0000 7f050b7fe640 1 --2- 172.21.15.43:0/3964298412 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f04f4006f10 0x7f04f4007390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:35:39.617 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:39.610+0000 7f050b7fe640 1 -- 172.21.15.43:0/3964298412 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f04f4005730 con 0x7f04f40060c0 2023-08-09T13:35:39.617 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:39.610+0000 7f051080e640 1 --2- 172.21.15.43:0/3964298412 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f04f4006f10 0x7f04f4007390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-08-09T13:35:39.619 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:39.611+0000 7f050b7fe640 1 --2- 172.21.15.43:0/3964298412 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f04f40060c0 0x7f04f40069d0 secure :-1 s=READY pgs=418 cs=0 l=1 rev1=1 crypto rx=0x7f0500009ac0 tx=0x7f04f4007a00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7e71ac23b7f3f85a server_cookie=0 in_seq=0 out_seq=0 2023-08-09T13:35:39.620 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:39.611+0000 7f050affd640 1 -- 172.21.15.43:0/3964298412 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f050002c710 con 0x7f04f40060c0 2023-08-09T13:35:39.620 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:39.611+0000 7f050affd640 1 -- 172.21.15.43:0/3964298412 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f050002c870 con 0x7f04f40060c0 2023-08-09T13:35:39.620 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:39.611+0000 7f050affd640 1 -- 172.21.15.43:0/3964298412 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f050002cc00 con 0x7f04f40060c0 2023-08-09T13:35:39.620 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:39.612+0000 7f051100f640 1 -- 172.21.15.43:0/3964298412 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f04f40060c0 msgr2=0x7f04f40069d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:35:39.620 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:39.612+0000 7f051100f640 1 --2- 172.21.15.43:0/3964298412 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f04f40060c0 0x7f04f40069d0 secure :-1 s=READY pgs=418 cs=0 l=1 rev1=1 crypto rx=0x7f0500009ac0 tx=0x7f04f4007a00 comp rx=0 tx=0).stop 2023-08-09T13:35:39.620 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:39.612+0000 7f051080e640 1 -- 172.21.15.43:0/3964298412 reap_dead start 2023-08-09T13:35:39.620 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:39.612+0000 7f051100f640 1 -- 172.21.15.43:0/3964298412 shutdown_connections 2023-08-09T13:35:39.621 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:39.612+0000 7f051100f640 1 -- 172.21.15.43:0/3964298412 >> 172.21.15.43:0/3964298412 conn(0x7f050c0af950 msgr2=0x7f050c0afd60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-09T13:35:39.621 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:39.612+0000 7f051100f640 1 -- 172.21.15.43:0/3964298412 shutdown_connections 2023-08-09T13:35:39.621 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:39.612+0000 7f051100f640 1 -- 172.21.15.43:0/3964298412 wait complete. 2023-08-09T13:35:39.621 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:39.614+0000 7f051100f640 1 Processor -- start 2023-08-09T13:35:39.621 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:39.614+0000 7f051100f640 1 -- start start 2023-08-09T13:35:39.621 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:39.615+0000 7f051100f640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f04f40060c0 0x7f050c0c2590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:35:39.621 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:39.615+0000 7f051100f640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f050c0c32a0 0x7f050c157380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:35:39.622 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:39.615+0000 7f051100f640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f050c1578c0 0x7f050c0b1ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:35:39.622 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:39.615+0000 7f051100f640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f050c159d90 con 0x7f04f40060c0 2023-08-09T13:35:39.622 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:39.615+0000 7f051100f640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f050c157d40 con 0x7f050c1578c0 2023-08-09T13:35:39.622 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:39.615+0000 7f051100f640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f050c0c4390 con 0x7f050c0c32a0 2023-08-09T13:35:39.622 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:39.615+0000 7f050bfff640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f04f40060c0 0x7f050c0c2590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:35:39.622 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:39.616+0000 7f050bfff640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f04f40060c0 0x7f050c0c2590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:50014/0 (socket says 172.21.15.43:50014) 2023-08-09T13:35:39.623 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:39.616+0000 7f050bfff640 1 -- 172.21.15.43:0/1213027698 learned_addr learned my addr 172.21.15.43:0/1213027698 (peer_addr_for_me v2:172.21.15.43:0/0) 2023-08-09T13:35:39.623 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:39.616+0000 7f050bfff640 1 -- 172.21.15.43:0/1213027698 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f050c0c32a0 msgr2=0x7f050c157380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:35:39.623 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:39.616+0000 7f050bfff640 1 --2- 172.21.15.43:0/1213027698 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f050c0c32a0 0x7f050c157380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:35:39.623 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:39.616+0000 7f050bfff640 1 -- 172.21.15.43:0/1213027698 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f050c1578c0 msgr2=0x7f050c0b1ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:35:39.623 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:39.616+0000 7f050bfff640 1 --2- 172.21.15.43:0/1213027698 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f050c1578c0 0x7f050c0b1ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:35:39.623 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:39.616+0000 7f050bfff640 1 -- 172.21.15.43:0/1213027698 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f050c1565a0 con 0x7f04f40060c0 2023-08-09T13:35:39.623 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:39.616+0000 7f050bfff640 1 --2- 172.21.15.43:0/1213027698 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f04f40060c0 0x7f050c0c2590 secure :-1 s=READY pgs=419 cs=0 l=1 rev1=1 crypto rx=0x7f04fc001eb0 tx=0x7f04fc00ed50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-09T13:35:39.624 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:39.616+0000 7f0508ff9640 1 -- 172.21.15.43:0/1213027698 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f04fc00c870 con 0x7f04f40060c0 2023-08-09T13:35:39.624 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:39.617+0000 7f0508ff9640 1 -- 172.21.15.43:0/1213027698 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f04fc00c9f0 con 0x7f04f40060c0 2023-08-09T13:35:39.624 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:39.617+0000 7f0508ff9640 1 -- 172.21.15.43:0/1213027698 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f04fc00cda0 con 0x7f04f40060c0 2023-08-09T13:35:39.624 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:39.617+0000 7f051100f640 1 -- 172.21.15.43:0/1213027698 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f050c0c3af0 con 0x7f04f40060c0 2023-08-09T13:35:39.624 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:39.617+0000 7f051100f640 1 -- 172.21.15.43:0/1213027698 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f050c0c40b0 con 0x7f04f40060c0 2023-08-09T13:35:39.625 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:39.618+0000 7f051100f640 1 -- 172.21.15.43:0/1213027698 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f050c0739f0 con 0x7f04f40060c0 2023-08-09T13:35:39.625 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:39.619+0000 7f0508ff9640 1 -- 172.21.15.43:0/1213027698 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 5) v1 ==== 88341+0+0 (secure 0 0 0) 0x7f04fc013420 con 0x7f04f40060c0 2023-08-09T13:35:39.625 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:39.620+0000 7f0508ff9640 1 --2- 172.21.15.43:0/1213027698 >> [v2:172.21.15.203:6832/830097474,v1:172.21.15.203:6833/830097474] conn(0x7f04f806a080 0x7f04f806c540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:35:39.626 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:39.620+0000 7f0508ff9640 1 -- 172.21.15.43:0/1213027698 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 6871+0+0 (secure 0 0 0) 0x7f04fc088640 con 0x7f04f40060c0 2023-08-09T13:35:39.626 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:39.620+0000 7f050b7fe640 1 --2- 172.21.15.43:0/1213027698 >> [v2:172.21.15.203:6832/830097474,v1:172.21.15.203:6833/830097474] conn(0x7f04f806a080 0x7f04f806c540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:35:39.626 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:39.622+0000 7f050b7fe640 1 --2- 172.21.15.43:0/1213027698 >> [v2:172.21.15.203:6832/830097474,v1:172.21.15.203:6833/830097474] conn(0x7f04f806a080 0x7f04f806c540 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7f0500036000 tx=0x7f050002e000 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-09T13:35:39.626 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:39.624+0000 7f0508ff9640 1 -- 172.21.15.43:0/1213027698 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178066 (secure 0 0 0) 0x7f04fc0539f0 con 0x7f04f40060c0 2023-08-09T13:35:39.834 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:39.832+0000 7f051100f640 1 -- 172.21.15.43:0/1213027698 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7f050c15bd70 con 0x7f04f40060c0 2023-08-09T13:35:39.835 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:39.833+0000 7f0508ff9640 1 -- 172.21.15.43:0/1213027698 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v26) v1 ==== 80+0+0 (secure 0 0 0) 0x7f04fc057010 con 0x7f04f40060c0 2023-08-09T13:35:39.836 INFO:teuthology.orchestra.run.smithi043.stderr:listed 0 entries 2023-08-09T13:35:39.838 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:39.836+0000 7f051100f640 1 -- 172.21.15.43:0/1213027698 >> [v2:172.21.15.203:6832/830097474,v1:172.21.15.203:6833/830097474] conn(0x7f04f806a080 msgr2=0x7f04f806c540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:35:39.838 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:39.836+0000 7f051100f640 1 --2- 172.21.15.43:0/1213027698 >> [v2:172.21.15.203:6832/830097474,v1:172.21.15.203:6833/830097474] conn(0x7f04f806a080 0x7f04f806c540 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7f0500036000 tx=0x7f050002e000 comp rx=0 tx=0).stop 2023-08-09T13:35:39.838 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:39.836+0000 7f051100f640 1 -- 172.21.15.43:0/1213027698 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f04f40060c0 msgr2=0x7f050c0c2590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:35:39.839 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:39.836+0000 7f051100f640 1 --2- 172.21.15.43:0/1213027698 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f04f40060c0 0x7f050c0c2590 secure :-1 s=READY pgs=419 cs=0 l=1 rev1=1 crypto rx=0x7f04fc001eb0 tx=0x7f04fc00ed50 comp rx=0 tx=0).stop 2023-08-09T13:35:39.839 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:39.836+0000 7f051100f640 1 -- 172.21.15.43:0/1213027698 shutdown_connections 2023-08-09T13:35:39.839 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:39.836+0000 7f051100f640 1 --2- 172.21.15.43:0/1213027698 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f050c1578c0 0x7f050c0b1ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:35:39.839 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:39.836+0000 7f051100f640 1 --2- 172.21.15.43:0/1213027698 >> [v2:172.21.15.203:6832/830097474,v1:172.21.15.203:6833/830097474] conn(0x7f04f806a080 0x7f04f806c540 unknown :-1 s=CLOSED pgs=212 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:35:39.839 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:39.836+0000 7f051100f640 1 --2- 172.21.15.43:0/1213027698 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f050c0c32a0 0x7f050c157380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:35:39.839 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:39.836+0000 7f051100f640 1 --2- 172.21.15.43:0/1213027698 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f04f40060c0 0x7f050c0c2590 unknown :-1 s=CLOSED pgs=419 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:35:39.839 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:39.836+0000 7f051100f640 1 -- 172.21.15.43:0/1213027698 >> 172.21.15.43:0/1213027698 conn(0x7f050c0af950 msgr2=0x7f050c0ad7c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-09T13:35:39.840 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:39.836+0000 7f051100f640 1 -- 172.21.15.43:0/1213027698 shutdown_connections 2023-08-09T13:35:39.840 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:39.836+0000 7f051100f640 1 -- 172.21.15.43:0/1213027698 wait complete. 2023-08-09T13:35:39.858 DEBUG:teuthology.orchestra.run.smithi043:> set -ex 2023-08-09T13:35:39.859 DEBUG:teuthology.orchestra.run.smithi043:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-09T13:35:39.913 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-08-09T13:35:40.142 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:40.140+0000 7fbe43e69640 1 Processor -- start 2023-08-09T13:35:40.142 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:40.140+0000 7fbe43e69640 1 -- start start 2023-08-09T13:35:40.144 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:40.141+0000 7fbe43e69640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7fbe3c143c20 0x7fbe3c06f540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:35:40.144 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:40.142+0000 7fbe43e69640 1 -- --> v1:172.21.15.43:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbe3c06c440 con 0x7fbe3c06cfe0 2023-08-09T13:35:40.144 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:40.142+0000 7fbe43e69640 1 -- --> v1:172.21.15.203:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbe3c06b830 con 0x7fbe3c0c86f0 2023-08-09T13:35:40.145 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:40.142+0000 7fbe43e69640 1 -- --> v2:172.21.15.43:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbe3c144000 con 0x7fbe3c143c20 2023-08-09T13:35:40.145 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:40.142+0000 7fbe42406640 1 --1- >> v1:172.21.15.43:6789/0 conn(0x7fbe3c06cfe0 0x7fbe3c0ccc00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.43:6789/0 says I am v1:172.21.15.43:44922/0 (socket says 172.21.15.43:44922) 2023-08-09T13:35:40.146 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:40.142+0000 7fbe42406640 1 -- 172.21.15.43:0/2159816955 learned_addr learned my addr 172.21.15.43:0/2159816955 (peer_addr_for_me v1:172.21.15.43:0/0) 2023-08-09T13:35:40.146 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:40.142+0000 7fbe41c05640 1 --2- 172.21.15.43:0/2159816955 >> v2:172.21.15.43:3300/0 conn(0x7fbe3c143c20 0x7fbe3c06f540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:35:40.146 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:40.142+0000 7fbe40c03640 1 -- 172.21.15.43:0/2159816955 <== mon.0 v1:172.21.15.43:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3750055544 0 0) 0x7fbe3c06c440 con 0x7fbe3c06cfe0 2023-08-09T13:35:40.146 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:40.142+0000 7fbe40c03640 1 -- 172.21.15.43:0/2159816955 --> v1:172.21.15.43:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fbe28003520 con 0x7fbe3c06cfe0 2023-08-09T13:35:40.146 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:40.143+0000 7fbe41c05640 1 -- 172.21.15.43:0/2159816955 >> v1:172.21.15.203:6789/0 conn(0x7fbe3c0c86f0 legacy=0x7fbe3c06c8d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-09T13:35:40.146 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:40.143+0000 7fbe41c05640 1 -- 172.21.15.43:0/2159816955 >> v1:172.21.15.43:6789/0 conn(0x7fbe3c06cfe0 legacy=0x7fbe3c0ccc00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:35:40.147 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:40.143+0000 7fbe41c05640 1 -- 172.21.15.43:0/2159816955 --> v2:172.21.15.43:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbe3c06bb10 con 0x7fbe3c143c20 2023-08-09T13:35:40.148 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:40.144+0000 7fbe41c05640 1 --2- 172.21.15.43:0/2159816955 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fbe3c143c20 0x7fbe3c06f540 secure :-1 s=READY pgs=421 cs=0 l=1 rev1=1 crypto rx=0x7fbe380097a0 tx=0x7fbe3802b7a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cae2ff9cce6d947 server_cookie=0 in_seq=0 out_seq=0 2023-08-09T13:35:40.148 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:40.144+0000 7fbe40c03640 1 -- 172.21.15.43:0/2159816955 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbe3802f050 con 0x7fbe3c143c20 2023-08-09T13:35:40.148 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:40.144+0000 7fbe40c03640 1 -- 172.21.15.43:0/2159816955 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbe38031700 con 0x7fbe3c143c20 2023-08-09T13:35:40.148 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:40.144+0000 7fbe40c03640 1 -- 172.21.15.43:0/2159816955 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbe38031a90 con 0x7fbe3c143c20 2023-08-09T13:35:40.148 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:40.145+0000 7fbe43e69640 1 -- 172.21.15.43:0/2159816955 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fbe3c143c20 msgr2=0x7fbe3c06f540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:35:40.148 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:40.145+0000 7fbe43e69640 1 --2- 172.21.15.43:0/2159816955 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fbe3c143c20 0x7fbe3c06f540 secure :-1 s=READY pgs=421 cs=0 l=1 rev1=1 crypto rx=0x7fbe380097a0 tx=0x7fbe3802b7a0 comp rx=0 tx=0).stop 2023-08-09T13:35:40.149 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:40.145+0000 7fbe43e69640 1 -- 172.21.15.43:0/2159816955 shutdown_connections 2023-08-09T13:35:40.149 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:40.145+0000 7fbe43e69640 1 --2- 172.21.15.43:0/2159816955 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fbe3c143c20 0x7fbe3c06f540 unknown :-1 s=CLOSED pgs=421 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:35:40.149 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:40.145+0000 7fbe43e69640 1 -- 172.21.15.43:0/2159816955 >> 172.21.15.43:0/2159816955 conn(0x7fbe3c0ce0a0 msgr2=0x7fbe3c0ce480 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-09T13:35:40.149 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:40.145+0000 7fbe43e69640 1 -- 172.21.15.43:0/2159816955 shutdown_connections 2023-08-09T13:35:40.149 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:40.145+0000 7fbe43e69640 1 -- 172.21.15.43:0/2159816955 wait complete. 2023-08-09T13:35:40.150 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:40.147+0000 7fbe43e69640 1 Processor -- start 2023-08-09T13:35:40.150 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:40.147+0000 7fbe43e69640 1 -- start start 2023-08-09T13:35:40.150 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:40.148+0000 7fbe43e69640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fbe3c06cfe0 0x7fbe3c130ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:35:40.150 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:40.148+0000 7fbe43e69640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fbe3c0c86f0 0x7fbe3c12f1f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:35:40.151 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:40.148+0000 7fbe43e69640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fbe3c143c20 0x7fbe3c15cc90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:35:40.151 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:40.148+0000 7fbe43e69640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbe3c1306c0 con 0x7fbe3c143c20 2023-08-09T13:35:40.151 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:40.148+0000 7fbe43e69640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbe3c131020 con 0x7fbe3c0c86f0 2023-08-09T13:35:40.151 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:40.148+0000 7fbe43e69640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbe3c12fa40 con 0x7fbe3c06cfe0 2023-08-09T13:35:40.152 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:40.148+0000 7fbe42406640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fbe3c143c20 0x7fbe3c15cc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:35:40.152 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:40.148+0000 7fbe42406640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fbe3c143c20 0x7fbe3c15cc90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:50022/0 (socket says 172.21.15.43:50022) 2023-08-09T13:35:40.153 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:40.148+0000 7fbe42406640 1 -- 172.21.15.43:0/1307854820 learned_addr learned my addr 172.21.15.43:0/1307854820 (peer_addr_for_me v2:172.21.15.43:0/0) 2023-08-09T13:35:40.153 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:40.148+0000 7fbe42406640 1 -- 172.21.15.43:0/1307854820 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fbe3c06cfe0 msgr2=0x7fbe3c130ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:35:40.153 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:40.148+0000 7fbe42406640 1 --2- 172.21.15.43:0/1307854820 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fbe3c06cfe0 0x7fbe3c130ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:35:40.153 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:40.148+0000 7fbe42406640 1 -- 172.21.15.43:0/1307854820 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fbe3c0c86f0 msgr2=0x7fbe3c12f1f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:35:40.154 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:40.148+0000 7fbe41404640 1 --2- 172.21.15.43:0/1307854820 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fbe3c0c86f0 0x7fbe3c12f1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:35:40.154 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:40.148+0000 7fbe42406640 1 --2- 172.21.15.43:0/1307854820 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fbe3c0c86f0 0x7fbe3c12f1f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:35:40.154 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:40.148+0000 7fbe42406640 1 -- 172.21.15.43:0/1307854820 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbe38009400 con 0x7fbe3c143c20 2023-08-09T13:35:40.154 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:40.149+0000 7fbe42406640 1 --2- 172.21.15.43:0/1307854820 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fbe3c143c20 0x7fbe3c15cc90 secure :-1 s=READY pgs=422 cs=0 l=1 rev1=1 crypto rx=0x7fbe3400caa0 tx=0x7fbe3400cf60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-09T13:35:40.154 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:40.149+0000 7fbe327fc640 1 -- 172.21.15.43:0/1307854820 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbe3400ac00 con 0x7fbe3c143c20 2023-08-09T13:35:40.154 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:40.149+0000 7fbe327fc640 1 -- 172.21.15.43:0/1307854820 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbe3400ad60 con 0x7fbe3c143c20 2023-08-09T13:35:40.154 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:40.149+0000 7fbe327fc640 1 -- 172.21.15.43:0/1307854820 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbe34012620 con 0x7fbe3c143c20 2023-08-09T13:35:40.155 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:40.149+0000 7fbe43e69640 1 -- 172.21.15.43:0/1307854820 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbe3c12fd20 con 0x7fbe3c143c20 2023-08-09T13:35:40.155 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:40.149+0000 7fbe43e69640 1 -- 172.21.15.43:0/1307854820 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbe3c1302e0 con 0x7fbe3c143c20 2023-08-09T13:35:40.155 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:40.150+0000 7fbe43e69640 1 -- 172.21.15.43:0/1307854820 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbe3c0739f0 con 0x7fbe3c143c20 2023-08-09T13:35:40.156 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:40.151+0000 7fbe327fc640 1 -- 172.21.15.43:0/1307854820 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 5) v1 ==== 88341+0+0 (secure 0 0 0) 0x7fbe34007070 con 0x7fbe3c143c20 2023-08-09T13:35:40.156 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:40.152+0000 7fbe327fc640 1 --2- 172.21.15.43:0/1307854820 >> [v2:172.21.15.203:6832/830097474,v1:172.21.15.203:6833/830097474] conn(0x7fbe2006a080 0x7fbe2006c540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:35:40.156 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:40.152+0000 7fbe327fc640 1 -- 172.21.15.43:0/1307854820 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 6871+0+0 (secure 0 0 0) 0x7fbe340888d0 con 0x7fbe3c143c20 2023-08-09T13:35:40.156 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:40.153+0000 7fbe41c05640 1 --2- 172.21.15.43:0/1307854820 >> [v2:172.21.15.203:6832/830097474,v1:172.21.15.203:6833/830097474] conn(0x7fbe2006a080 0x7fbe2006c540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:35:40.157 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:40.154+0000 7fbe41c05640 1 --2- 172.21.15.43:0/1307854820 >> [v2:172.21.15.203:6832/830097474,v1:172.21.15.203:6833/830097474] conn(0x7fbe2006a080 0x7fbe2006c540 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7fbe38035000 tx=0x7fbe38030000 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-09T13:35:40.159 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:40.156+0000 7fbe327fc640 1 -- 172.21.15.43:0/1307854820 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178066 (secure 0 0 0) 0x7fbe34053bf0 con 0x7fbe3c143c20 2023-08-09T13:35:40.363 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:40.361+0000 7fbe43e69640 1 -- 172.21.15.43:0/1307854820 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fbe3c06bdc0 con 0x7fbe3c143c20 2023-08-09T13:35:40.367 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:40.364+0000 7fbe327fc640 1 -- 172.21.15.43:0/1307854820 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v26) v1 ==== 74+0+20146 (secure 0 0 0) 0x7fbe34057210 con 0x7fbe3c143c20 2023-08-09T13:35:40.368 INFO:teuthology.orchestra.run.smithi043.stdout: 2023-08-09T13:35:40.368 INFO:teuthology.orchestra.run.smithi043.stdout:{"epoch":26,"fsid":"daac87ee-1337-4250-a955-aea4e408a39b","created":"2023-08-09T13:33:41.701198+0000","modified":"2023-08-09T13:34:48.711911+0000","last_up_change":"2023-08-09T13:33:53.783453+0000","last_in_change":"2023-08-09T13:33:45.148904+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":5,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":"rbd","create_time":"2023-08-09T13:33:54.365137+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":".mgr","create_time":"2023-08-09T13:33:54.831803+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_metadata","create_time":"2023-08-09T13:34:00.207974+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data","create_time":"2023-08-09T13:34:01.614765+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":5,"pool_name":"cephfs_data_ec","create_time":"2023-08-09T13:34:05.239583+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"24","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"cbd49ecd-e896-4fbd-864e-5c37a829b165","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6800","nonce":502667335},{"type":"v1","addr":"172.21.15.43:6801","nonce":502667335}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6802","nonce":502667335},{"type":"v1","addr":"172.21.15.43:6803","nonce":502667335}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6806","nonce":502667335},{"type":"v1","addr":"172.21.15.43:6807","nonce":502667335}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6804","nonce":502667335},{"type":"v1","addr":"172.21.15.43:6805","nonce":502667335}]},"public_addr":"172.21.15.43:6801/502667335","cluster_addr":"172.21.15.43:6803/502667335","heartbeat_back_addr":"172.21.15.43:6807/502667335","heartbeat_front_addr":"172.21.15.43:6805/502667335","state":["exists","up"]},{"osd":1,"uuid":"75c7e0fe-1ec7-4fe8-a2f2-3661a65b8500","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6808","nonce":3289134610},{"type":"v1","addr":"172.21.15.43:6809","nonce":3289134610}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6810","nonce":3289134610},{"type":"v1","addr":"172.21.15.43:6812","nonce":3289134610}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6821","nonce":3289134610},{"type":"v1","addr":"172.21.15.43:6824","nonce":3289134610}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6815","nonce":3289134610},{"type":"v1","addr":"172.21.15.43:6818","nonce":3289134610}]},"public_addr":"172.21.15.43:6809/3289134610","cluster_addr":"172.21.15.43:6812/3289134610","heartbeat_back_addr":"172.21.15.43:6824/3289134610","heartbeat_front_addr":"172.21.15.43:6818/3289134610","state":["exists","up"]},{"osd":2,"uuid":"bb5c41e9-7f5f-4125-9b01-3bb8bfc2ea83","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6811","nonce":4079709736},{"type":"v1","addr":"172.21.15.43:6814","nonce":4079709736}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6817","nonce":4079709736},{"type":"v1","addr":"172.21.15.43:6820","nonce":4079709736}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6829","nonce":4079709736},{"type":"v1","addr":"172.21.15.43:6831","nonce":4079709736}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6823","nonce":4079709736},{"type":"v1","addr":"172.21.15.43:6826","nonce":4079709736}]},"public_addr":"172.21.15.43:6814/4079709736","cluster_addr":"172.21.15.43:6820/4079709736","heartbeat_back_addr":"172.21.15.43:6831/4079709736","heartbeat_front_addr":"172.21.15.43:6826/4079709736","state":["exists","up"]},{"osd":3,"uuid":"bfe4c8b4-694a-465c-87e6-2b2aab852613","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6813","nonce":1709538759},{"type":"v1","addr":"172.21.15.43:6816","nonce":1709538759}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6819","nonce":1709538759},{"type":"v1","addr":"172.21.15.43:6822","nonce":1709538759}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6828","nonce":1709538759},{"type":"v1","addr":"172.21.15.43:6830","nonce":1709538759}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6825","nonce":1709538759},{"type":"v1","addr":"172.21.15.43:6827","nonce":1709538759}]},"public_addr":"172.21.15.43:6816/1709538759","cluster_addr":"172.21.15.43:6822/1709538759","heartbeat_back_addr":"172.21.15.43:6830/1709538759","heartbeat_front_addr":"172.21.15.43:6827/1709538759","state":["exists","up"]},{"osd":4,"uuid":"fa2e8a77-aba1-4756-853e-0d174a5bdeba","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6800","nonce":4196866481},{"type":"v1","addr":"172.21.15.203:6801","nonce":4196866481}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6802","nonce":4196866481},{"type":"v1","addr":"172.21.15.203:6803","nonce":4196866481}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6806","nonce":4196866481},{"type":"v1","addr":"172.21.15.203:6807","nonce":4196866481}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6804","nonce":4196866481},{"type":"v1","addr":"172.21.15.203:6805","nonce":4196866481}]},"public_addr":"172.21.15.203:6801/4196866481","cluster_addr":"172.21.15.203:6803/4196866481","heartbeat_back_addr":"172.21.15.203:6807/4196866481","heartbeat_front_addr":"172.21.15.203:6805/4196866481","state":["exists","up"]},{"osd":5,"uuid":"81811e4b-4d24-4f51-a82c-f1d700a2a00d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6816","nonce":3059290081},{"type":"v1","addr":"172.21.15.203:6817","nonce":3059290081}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6818","nonce":3059290081},{"type":"v1","addr":"172.21.15.203:6819","nonce":3059290081}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6822","nonce":3059290081},{"type":"v1","addr":"172.21.15.203:6823","nonce":3059290081}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6820","nonce":3059290081},{"type":"v1","addr":"172.21.15.203:6821","nonce":3059290081}]},"public_addr":"172.21.15.203:6817/3059290081","cluster_addr":"172.21.15.203:6819/3059290081","heartbeat_back_addr":"172.21.15.203:6823/3059290081","heartbeat_front_addr":"172.21.15.203:6821/3059290081","state":["exists","up"]},{"osd":6,"uuid":"307bac4d-6a5b-443d-af75-56d695d7674f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6824","nonce":512155569},{"type":"v1","addr":"172.21.15.203:6825","nonce":512155569}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6826","nonce":512155569},{"type":"v1","addr":"172.21.15.203:6827","nonce":512155569}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6830","nonce":512155569},{"type":"v1","addr":"172.21.15.203:6831","nonce":512155569}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6828","nonce":512155569},{"type":"v1","addr":"172.21.15.203:6829","nonce":512155569}]},"public_addr":"172.21.15.203:6825/512155569","cluster_addr":"172.21.15.203:6827/512155569","heartbeat_back_addr":"172.21.15.203:6831/512155569","heartbeat_front_addr":"172.21.15.203:6829/512155569","state":["exists","up"]},{"osd":7,"uuid":"a95dc7ca-9f1b-4295-af70-68a22a2d7d5e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6808","nonce":3331450438},{"type":"v1","addr":"172.21.15.203:6809","nonce":3331450438}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6810","nonce":3331450438},{"type":"v1","addr":"172.21.15.203:6811","nonce":3331450438}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6814","nonce":3331450438},{"type":"v1","addr":"172.21.15.203:6815","nonce":3331450438}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6812","nonce":3331450438},{"type":"v1","addr":"172.21.15.203:6813","nonce":3331450438}]},"public_addr":"172.21.15.203:6809/3331450438","cluster_addr":"172.21.15.203:6811/3331450438","heartbeat_back_addr":"172.21.15.203:6815/3331450438","heartbeat_front_addr":"172.21.15.203:6813/3331450438","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-09T13:33:50.547652+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-09T13:33:50.589469+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-09T13:33:50.561724+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-09T13:33:50.599650+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-09T13:33:50.411855+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-09T13:33:50.219681+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-09T13:33:50.504664+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-09T13:33:50.441248+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"4.14","mappings":[{"from":1,"to":5}]},{"pgid":"4.29","mappings":[{"from":3,"to":2}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-08-09T13:35:40.370 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:40.367+0000 7fbe43e69640 1 -- 172.21.15.43:0/1307854820 >> [v2:172.21.15.203:6832/830097474,v1:172.21.15.203:6833/830097474] conn(0x7fbe2006a080 msgr2=0x7fbe2006c540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:35:40.370 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:40.367+0000 7fbe43e69640 1 --2- 172.21.15.43:0/1307854820 >> [v2:172.21.15.203:6832/830097474,v1:172.21.15.203:6833/830097474] conn(0x7fbe2006a080 0x7fbe2006c540 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7fbe38035000 tx=0x7fbe38030000 comp rx=0 tx=0).stop 2023-08-09T13:35:40.370 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:40.367+0000 7fbe43e69640 1 -- 172.21.15.43:0/1307854820 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fbe3c143c20 msgr2=0x7fbe3c15cc90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:35:40.370 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:40.367+0000 7fbe43e69640 1 --2- 172.21.15.43:0/1307854820 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fbe3c143c20 0x7fbe3c15cc90 secure :-1 s=READY pgs=422 cs=0 l=1 rev1=1 crypto rx=0x7fbe3400caa0 tx=0x7fbe3400cf60 comp rx=0 tx=0).stop 2023-08-09T13:35:40.370 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:40.367+0000 7fbe43e69640 1 -- 172.21.15.43:0/1307854820 shutdown_connections 2023-08-09T13:35:40.370 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:40.367+0000 7fbe43e69640 1 --2- 172.21.15.43:0/1307854820 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fbe3c143c20 0x7fbe3c15cc90 unknown :-1 s=CLOSED pgs=422 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:35:40.371 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:40.367+0000 7fbe43e69640 1 --2- 172.21.15.43:0/1307854820 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fbe3c0c86f0 0x7fbe3c12f1f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:35:40.371 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:40.367+0000 7fbe43e69640 1 --2- 172.21.15.43:0/1307854820 >> [v2:172.21.15.203:6832/830097474,v1:172.21.15.203:6833/830097474] conn(0x7fbe2006a080 0x7fbe2006c540 unknown :-1 s=CLOSED pgs=213 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:35:40.371 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:40.367+0000 7fbe43e69640 1 --2- 172.21.15.43:0/1307854820 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fbe3c06cfe0 0x7fbe3c130ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:35:40.371 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:40.367+0000 7fbe43e69640 1 -- 172.21.15.43:0/1307854820 >> 172.21.15.43:0/1307854820 conn(0x7fbe3c0ce0a0 msgr2=0x7fbe3c0ae920 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-09T13:35:40.371 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:40.367+0000 7fbe43e69640 1 -- 172.21.15.43:0/1307854820 shutdown_connections 2023-08-09T13:35:40.372 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:40.367+0000 7fbe43e69640 1 -- 172.21.15.43:0/1307854820 wait complete. 2023-08-09T13:35:40.390 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': 'rbd', 'create_time': '2023-08-09T13:33:54.365137+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}, 'read_balance': {'score_acting': 2, 'score_stable': 2, 'optimal_score': 1, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': '.mgr', 'create_time': '2023-08-09T13:33:54.831803+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_metadata', 'create_time': '2023-08-09T13:34:00.207974+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data', 'create_time': '2023-08-09T13:34:01.614765+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 2, 'score_stable': 2, 'optimal_score': 1, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 5, 'pool_name': 'cephfs_data_ec', 'create_time': '2023-08-09T13:34:05.239583+0000', 'flags': 5, 'flags_names': 'hashpspool,ec_overwrites', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '24', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2023-08-09T13:35:40.392 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get rbd pg_num 2023-08-09T13:35:40.628 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:40.625+0000 7f3b68b5d640 1 Processor -- start 2023-08-09T13:35:40.628 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:40.625+0000 7f3b68b5d640 1 -- start start 2023-08-09T13:35:40.629 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:40.626+0000 7f3b68b5d640 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7f3b640b45d0 0x7f3b640ccd10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:35:40.629 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:40.627+0000 7f3b68b5d640 1 -- --> v1:172.21.15.43:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3b640cd3a0 con 0x7f3b64160dd0 2023-08-09T13:35:40.629 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:40.627+0000 7f3b68b5d640 1 -- --> v1:172.21.15.203:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3b640cd680 con 0x7f3b6415c7e0 2023-08-09T13:35:40.630 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:40.627+0000 7f3b68b5d640 1 -- --> v2:172.21.15.203:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3b640cdf20 con 0x7f3b640b45d0 2023-08-09T13:35:40.631 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:40.627+0000 7f3b62d9d640 1 --1- >> v1:172.21.15.43:6789/0 conn(0x7f3b64160dd0 0x7f3b640d0990 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.43:6789/0 says I am v1:172.21.15.43:44932/0 (socket says 172.21.15.43:44932) 2023-08-09T13:35:40.631 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:40.627+0000 7f3b62d9d640 1 -- 172.21.15.43:0/2927868251 learned_addr learned my addr 172.21.15.43:0/2927868251 (peer_addr_for_me v1:172.21.15.43:0/0) 2023-08-09T13:35:40.631 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:40.628+0000 7f3b59d9b640 1 --2- 172.21.15.43:0/2927868251 >> v2:172.21.15.203:3300/0 conn(0x7f3b640b45d0 0x7f3b640ccd10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:35:40.631 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:40.628+0000 7f3b61d9b640 1 -- 172.21.15.43:0/2927868251 <== mon.0 v1:172.21.15.43:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 550540936 0 0) 0x7f3b640cd3a0 con 0x7f3b64160dd0 2023-08-09T13:35:40.632 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:40.628+0000 7f3b61d9b640 1 -- 172.21.15.43:0/2927868251 --> v1:172.21.15.43:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f3b50003520 con 0x7f3b64160dd0 2023-08-09T13:35:40.632 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:40.628+0000 7f3b61d9b640 1 -- 172.21.15.43:0/2927868251 <== mon.0 v1:172.21.15.43:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3740565554 0 0) 0x7f3b50003520 con 0x7f3b64160dd0 2023-08-09T13:35:40.632 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:40.628+0000 7f3b61d9b640 1 -- 172.21.15.43:0/2927868251 >> v2:172.21.15.203:3300/0 conn(0x7f3b640b45d0 msgr2=0x7f3b640ccd10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-09T13:35:40.632 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:40.628+0000 7f3b61d9b640 1 --2- 172.21.15.43:0/2927868251 >> v2:172.21.15.203:3300/0 conn(0x7f3b640b45d0 0x7f3b640ccd10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:35:40.632 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:40.628+0000 7f3b61d9b640 1 -- 172.21.15.43:0/2927868251 >> v1:172.21.15.203:6789/0 conn(0x7f3b6415c7e0 legacy=0x7f3b640b3ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:35:40.632 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:40.628+0000 7f3b61d9b640 1 -- 172.21.15.43:0/2927868251 --> v1:172.21.15.43:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3b640cd960 con 0x7f3b64160dd0 2023-08-09T13:35:40.633 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:40.628+0000 7f3b61d9b640 1 -- 172.21.15.43:0/2927868251 <== mon.0 v1:172.21.15.43:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1285150740 0 0) 0x7f3b5c001670 con 0x7f3b64160dd0 2023-08-09T13:35:40.633 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:40.628+0000 7f3b61d9b640 1 -- 172.21.15.43:0/2927868251 >> v1:172.21.15.43:6789/0 conn(0x7f3b64160dd0 legacy=0x7f3b640d0990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:35:40.633 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:40.628+0000 7f3b61d9b640 1 --2- 172.21.15.43:0/2927868251 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f3b50003970 0x7f3b50003d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:35:40.633 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:40.628+0000 7f3b61d9b640 1 --2- 172.21.15.43:0/2927868251 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f3b500047c0 0x7f3b50004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:35:40.633 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:40.628+0000 7f3b61d9b640 1 --2- 172.21.15.43:0/2927868251 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f3b50005180 0x7f3b5000b980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:35:40.633 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:40.628+0000 7f3b61d9b640 1 -- 172.21.15.43:0/2927868251 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3b640cdf20 con 0x7f3b500047c0 2023-08-09T13:35:40.634 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:40.628+0000 7f3b61d9b640 1 -- 172.21.15.43:0/2927868251 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3b5000c020 con 0x7f3b50005180 2023-08-09T13:35:40.634 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:40.628+0000 7f3b61d9b640 1 -- 172.21.15.43:0/2927868251 --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3b5000c1e0 con 0x7f3b50003970 2023-08-09T13:35:40.634 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:40.628+0000 7f3b59d9b640 1 --2- 172.21.15.43:0/2927868251 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f3b500047c0 0x7f3b50004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:35:40.635 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:40.629+0000 7f3b62d9d640 1 --2- 172.21.15.43:0/2927868251 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f3b50005180 0x7f3b5000b980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:35:40.635 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:40.629+0000 7f3b6259c640 1 --2- 172.21.15.43:0/2927868251 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f3b50003970 0x7f3b50003d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:35:40.636 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:40.629+0000 7f3b59d9b640 1 -- 172.21.15.43:0/2927868251 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f3b50003970 msgr2=0x7f3b50003d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-09T13:35:40.636 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:40.629+0000 7f3b59d9b640 1 --2- 172.21.15.43:0/2927868251 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f3b50003970 0x7f3b50003d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:35:40.636 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:40.629+0000 7f3b59d9b640 1 -- 172.21.15.43:0/2927868251 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f3b50005180 msgr2=0x7f3b5000b980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-09T13:35:40.636 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:40.629+0000 7f3b59d9b640 1 --2- 172.21.15.43:0/2927868251 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f3b50005180 0x7f3b5000b980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:35:40.636 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:40.629+0000 7f3b59d9b640 1 -- 172.21.15.43:0/2927868251 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3b50003520 con 0x7f3b500047c0 2023-08-09T13:35:40.636 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:40.629+0000 7f3b62d9d640 1 -- 172.21.15.43:0/2927868251 reap_dead start 2023-08-09T13:35:40.636 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:40.629+0000 7f3b6259c640 1 --2- 172.21.15.43:0/2927868251 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f3b50003970 0x7f3b50003d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-08-09T13:35:40.637 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:40.630+0000 7f3b59d9b640 1 --2- 172.21.15.43:0/2927868251 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f3b500047c0 0x7f3b50004c40 secure :-1 s=READY pgs=424 cs=0 l=1 rev1=1 crypto rx=0x7f3b4c009ac0 tx=0x7f3b500057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2277545506a78ce server_cookie=0 in_seq=0 out_seq=0 2023-08-09T13:35:40.637 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:40.631+0000 7f3b61d9b640 1 -- 172.21.15.43:0/2927868251 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3b4c02c6b0 con 0x7f3b500047c0 2023-08-09T13:35:40.637 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:40.631+0000 7f3b61d9b640 1 -- 172.21.15.43:0/2927868251 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3b4c02c810 con 0x7f3b500047c0 2023-08-09T13:35:40.638 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:40.631+0000 7f3b61d9b640 1 -- 172.21.15.43:0/2927868251 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3b4c02cba0 con 0x7f3b500047c0 2023-08-09T13:35:40.638 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:40.631+0000 7f3b68b5d640 1 -- 172.21.15.43:0/2927868251 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f3b500047c0 msgr2=0x7f3b50004c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:35:40.638 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:40.631+0000 7f3b68b5d640 1 --2- 172.21.15.43:0/2927868251 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f3b500047c0 0x7f3b50004c40 secure :-1 s=READY pgs=424 cs=0 l=1 rev1=1 crypto rx=0x7f3b4c009ac0 tx=0x7f3b500057f0 comp rx=0 tx=0).stop 2023-08-09T13:35:40.638 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:40.631+0000 7f3b68b5d640 1 -- 172.21.15.43:0/2927868251 shutdown_connections 2023-08-09T13:35:40.639 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:40.631+0000 7f3b68b5d640 1 --2- 172.21.15.43:0/2927868251 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f3b500047c0 0x7f3b50004c40 unknown :-1 s=CLOSED pgs=424 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:35:40.639 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:40.631+0000 7f3b68b5d640 1 -- 172.21.15.43:0/2927868251 >> 172.21.15.43:0/2927868251 conn(0x7f3b640ceec0 msgr2=0x7f3b640cf2d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-09T13:35:40.639 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:40.631+0000 7f3b68b5d640 1 -- 172.21.15.43:0/2927868251 shutdown_connections 2023-08-09T13:35:40.640 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:40.632+0000 7f3b68b5d640 1 -- 172.21.15.43:0/2927868251 wait complete. 2023-08-09T13:35:40.642 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:40.633+0000 7f3b68b5d640 1 Processor -- start 2023-08-09T13:35:40.642 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:40.634+0000 7f3b68b5d640 1 -- start start 2023-08-09T13:35:40.642 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:40.635+0000 7f3b68b5d640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f3b500047c0 0x7f3b640b1e80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:35:40.642 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:40.635+0000 7f3b68b5d640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f3b641f2b40 0x7f3b640c8980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:35:40.642 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:40.635+0000 7f3b68b5d640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f3b640d0890 0x7f3b640d0d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:35:40.643 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:40.635+0000 7f3b68b5d640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3b640d1250 con 0x7f3b640d0890 2023-08-09T13:35:40.643 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:40.635+0000 7f3b68b5d640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3b641f2fc0 con 0x7f3b500047c0 2023-08-09T13:35:40.643 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:40.635+0000 7f3b68b5d640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3b640b2510 con 0x7f3b641f2b40 2023-08-09T13:35:40.643 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:40.635+0000 7f3b62d9d640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f3b640d0890 0x7f3b640d0d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:35:40.643 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:40.635+0000 7f3b62d9d640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f3b640d0890 0x7f3b640d0d10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:50040/0 (socket says 172.21.15.43:50040) 2023-08-09T13:35:40.644 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:40.635+0000 7f3b62d9d640 1 -- 172.21.15.43:0/1209009468 learned_addr learned my addr 172.21.15.43:0/1209009468 (peer_addr_for_me v2:172.21.15.43:0/0) 2023-08-09T13:35:40.644 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:40.635+0000 7f3b6259c640 1 --2- 172.21.15.43:0/1209009468 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f3b500047c0 0x7f3b640b1e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:35:40.644 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:40.635+0000 7f3b59d9b640 1 --2- 172.21.15.43:0/1209009468 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f3b641f2b40 0x7f3b640c8980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:35:40.644 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:40.635+0000 7f3b62d9d640 1 -- 172.21.15.43:0/1209009468 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f3b641f2b40 msgr2=0x7f3b640c8980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:35:40.644 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:40.635+0000 7f3b62d9d640 1 --2- 172.21.15.43:0/1209009468 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f3b641f2b40 0x7f3b640c8980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:35:40.644 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:40.635+0000 7f3b62d9d640 1 -- 172.21.15.43:0/1209009468 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f3b500047c0 msgr2=0x7f3b640b1e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:35:40.645 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:40.635+0000 7f3b62d9d640 1 --2- 172.21.15.43:0/1209009468 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f3b500047c0 0x7f3b640b1e80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:35:40.645 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:40.635+0000 7f3b62d9d640 1 -- 172.21.15.43:0/1209009468 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3b640b27f0 con 0x7f3b640d0890 2023-08-09T13:35:40.645 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:40.636+0000 7f3b62d9d640 1 --2- 172.21.15.43:0/1209009468 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f3b640d0890 0x7f3b640d0d10 secure :-1 s=READY pgs=425 cs=0 l=1 rev1=1 crypto rx=0x7f3b640cfdd0 tx=0x7f3b5c00e960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-09T13:35:40.645 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:40.636+0000 7f3b5b7fe640 1 -- 172.21.15.43:0/1209009468 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3b5c010510 con 0x7f3b640d0890 2023-08-09T13:35:40.645 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:40.636+0000 7f3b5b7fe640 1 -- 172.21.15.43:0/1209009468 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3b5c010670 con 0x7f3b640d0890 2023-08-09T13:35:40.645 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:40.636+0000 7f3b68b5d640 1 -- 172.21.15.43:0/1209009468 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3b640b2ad0 con 0x7f3b640d0890 2023-08-09T13:35:40.645 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:40.636+0000 7f3b5b7fe640 1 -- 172.21.15.43:0/1209009468 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3b5c010a20 con 0x7f3b640d0890 2023-08-09T13:35:40.646 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:40.636+0000 7f3b68b5d640 1 -- 172.21.15.43:0/1209009468 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3b640c8ec0 con 0x7f3b640d0890 2023-08-09T13:35:40.646 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:40.638+0000 7f3b68b5d640 1 -- 172.21.15.43:0/1209009468 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3b640739f0 con 0x7f3b640d0890 2023-08-09T13:35:40.646 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:40.638+0000 7f3b5b7fe640 1 -- 172.21.15.43:0/1209009468 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 5) v1 ==== 88341+0+0 (secure 0 0 0) 0x7f3b5c010b80 con 0x7f3b640d0890 2023-08-09T13:35:40.646 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:40.639+0000 7f3b5b7fe640 1 --2- 172.21.15.43:0/1209009468 >> [v2:172.21.15.203:6832/830097474,v1:172.21.15.203:6833/830097474] conn(0x7f3b4406a080 0x7f3b4406c540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:35:40.646 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:40.639+0000 7f3b5b7fe640 1 -- 172.21.15.43:0/1209009468 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 6871+0+0 (secure 0 0 0) 0x7f3b5c0880f0 con 0x7f3b640d0890 2023-08-09T13:35:40.647 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:40.640+0000 7f3b6259c640 1 --2- 172.21.15.43:0/1209009468 >> [v2:172.21.15.203:6832/830097474,v1:172.21.15.203:6833/830097474] conn(0x7f3b4406a080 0x7f3b4406c540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:35:40.647 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:40.641+0000 7f3b6259c640 1 --2- 172.21.15.43:0/1209009468 >> [v2:172.21.15.203:6832/830097474,v1:172.21.15.203:6833/830097474] conn(0x7f3b4406a080 0x7f3b4406c540 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7f3b54007fd0 tx=0x7f3b54009040 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-09T13:35:40.647 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:40.643+0000 7f3b5b7fe640 1 -- 172.21.15.43:0/1209009468 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178066 (secure 0 0 0) 0x7f3b5c053410 con 0x7f3b640d0890 2023-08-09T13:35:40.841 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:40.839+0000 7f3b68b5d640 1 -- 172.21.15.43:0/1209009468 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"} v 0) v1 -- 0x7f3b640d1840 con 0x7f3b640d0890 2023-08-09T13:35:40.842 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:40.839+0000 7f3b5b7fe640 1 -- 172.21.15.43:0/1209009468 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"}]=0 v26) v1 ==== 92+0+10 (secure 0 0 0) 0x7f3b5c0190a0 con 0x7f3b640d0890 2023-08-09T13:35:40.842 INFO:teuthology.orchestra.run.smithi043.stdout:pg_num: 8 2023-08-09T13:35:40.844 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:40.842+0000 7f3b68b5d640 1 -- 172.21.15.43:0/1209009468 >> [v2:172.21.15.203:6832/830097474,v1:172.21.15.203:6833/830097474] conn(0x7f3b4406a080 msgr2=0x7f3b4406c540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:35:40.844 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:40.842+0000 7f3b68b5d640 1 --2- 172.21.15.43:0/1209009468 >> [v2:172.21.15.203:6832/830097474,v1:172.21.15.203:6833/830097474] conn(0x7f3b4406a080 0x7f3b4406c540 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7f3b54007fd0 tx=0x7f3b54009040 comp rx=0 tx=0).stop 2023-08-09T13:35:40.845 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:40.842+0000 7f3b68b5d640 1 -- 172.21.15.43:0/1209009468 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f3b640d0890 msgr2=0x7f3b640d0d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:35:40.845 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:40.842+0000 7f3b68b5d640 1 --2- 172.21.15.43:0/1209009468 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f3b640d0890 0x7f3b640d0d10 secure :-1 s=READY pgs=425 cs=0 l=1 rev1=1 crypto rx=0x7f3b640cfdd0 tx=0x7f3b5c00e960 comp rx=0 tx=0).stop 2023-08-09T13:35:40.845 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:40.842+0000 7f3b68b5d640 1 -- 172.21.15.43:0/1209009468 shutdown_connections 2023-08-09T13:35:40.846 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:40.842+0000 7f3b68b5d640 1 --2- 172.21.15.43:0/1209009468 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f3b640d0890 0x7f3b640d0d10 unknown :-1 s=CLOSED pgs=425 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:35:40.846 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:40.842+0000 7f3b68b5d640 1 --2- 172.21.15.43:0/1209009468 >> [v2:172.21.15.203:6832/830097474,v1:172.21.15.203:6833/830097474] conn(0x7f3b4406a080 0x7f3b4406c540 unknown :-1 s=CLOSED pgs=214 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:35:40.846 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:40.842+0000 7f3b68b5d640 1 --2- 172.21.15.43:0/1209009468 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f3b641f2b40 0x7f3b640c8980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:35:40.846 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:40.842+0000 7f3b68b5d640 1 --2- 172.21.15.43:0/1209009468 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f3b500047c0 0x7f3b640b1e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:35:40.846 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:40.842+0000 7f3b68b5d640 1 -- 172.21.15.43:0/1209009468 >> 172.21.15.43:0/1209009468 conn(0x7f3b640ceec0 msgr2=0x7f3b6415add0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-09T13:35:40.846 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:40.842+0000 7f3b68b5d640 1 -- 172.21.15.43:0/1209009468 shutdown_connections 2023-08-09T13:35:40.846 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:40.843+0000 7f3b68b5d640 1 -- 172.21.15.43:0/1209009468 wait complete. 2023-08-09T13:35:40.864 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-08-09T13:35:41.090 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:41.087+0000 7f5094071640 1 Processor -- start 2023-08-09T13:35:41.090 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:41.087+0000 7f5094071640 1 -- start start 2023-08-09T13:35:41.092 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:41.089+0000 7f5094071640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f508c0cb2d0 0x7f508c15b380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:35:41.092 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:41.089+0000 7f5094071640 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7f508c0ce8d0 0x7f508c0c9f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:35:41.092 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:41.089+0000 7f5094071640 1 -- --> v1:172.21.15.43:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f508c15aef0 con 0x7f508c15df60 2023-08-09T13:35:41.092 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:41.089+0000 7f5094071640 1 -- --> v2:172.21.15.43:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f508c15a970 con 0x7f508c0cb2d0 2023-08-09T13:35:41.092 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:41.089+0000 7f5094071640 1 -- --> v2:172.21.15.203:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f508c15b8c0 con 0x7f508c0ce8d0 2023-08-09T13:35:41.093 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:41.090+0000 7f5091e0d640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f508c0cb2d0 0x7f508c15b380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:35:41.093 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:41.090+0000 7f509160c640 1 --1- >> v1:172.21.15.43:6789/0 conn(0x7f508c15df60 0x7f508c0ce1c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.43:6789/0 says I am v1:172.21.15.43:44940/0 (socket says 172.21.15.43:44940) 2023-08-09T13:35:41.094 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:41.090+0000 7f509160c640 1 -- 172.21.15.43:0/189551491 learned_addr learned my addr 172.21.15.43:0/189551491 (peer_addr_for_me v1:172.21.15.43:0/0) 2023-08-09T13:35:41.094 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:41.090+0000 7f509260e640 1 --2- 172.21.15.43:0/189551491 >> v2:172.21.15.203:3300/0 conn(0x7f508c0ce8d0 0x7f508c0c9f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:35:41.094 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:41.090+0000 7f5091e0d640 1 -- 172.21.15.43:0/189551491 >> v2:172.21.15.203:3300/0 conn(0x7f508c0ce8d0 msgr2=0x7f508c0c9f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-09T13:35:41.094 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:41.090+0000 7f5091e0d640 1 --2- 172.21.15.43:0/189551491 >> v2:172.21.15.203:3300/0 conn(0x7f508c0ce8d0 0x7f508c0c9f90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:35:41.094 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:41.090+0000 7f5091e0d640 1 -- 172.21.15.43:0/189551491 >> v1:172.21.15.43:6789/0 conn(0x7f508c15df60 legacy=0x7f508c0ce1c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:35:41.094 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:41.090+0000 7f5091e0d640 1 -- 172.21.15.43:0/189551491 --> v2:172.21.15.43:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f508c15e940 con 0x7f508c0cb2d0 2023-08-09T13:35:41.095 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:41.090+0000 7f5090e0b640 1 -- 172.21.15.43:0/189551491 <== mon.0 v1:172.21.15.43:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 155184385 0 0) 0x7f508c15aef0 con 0x7f508c15df60 2023-08-09T13:35:41.096 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:41.091+0000 7f5091e0d640 1 --2- 172.21.15.43:0/189551491 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f508c0cb2d0 0x7f508c15b380 secure :-1 s=READY pgs=427 cs=0 l=1 rev1=1 crypto rx=0x7f508c15de90 tx=0x7f508802b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6de1e28c5a3b1b12 server_cookie=0 in_seq=0 out_seq=0 2023-08-09T13:35:41.096 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:41.092+0000 7f5090e0b640 1 -- 172.21.15.43:0/189551491 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5088031830 con 0x7f508c0cb2d0 2023-08-09T13:35:41.096 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:41.092+0000 7f5090e0b640 1 -- 172.21.15.43:0/189551491 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5088031990 con 0x7f508c0cb2d0 2023-08-09T13:35:41.096 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:41.092+0000 7f5090e0b640 1 -- 172.21.15.43:0/189551491 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5088031d20 con 0x7f508c0cb2d0 2023-08-09T13:35:41.096 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:41.092+0000 7f5094071640 1 -- 172.21.15.43:0/189551491 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f508c0cb2d0 msgr2=0x7f508c15b380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:35:41.097 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:41.092+0000 7f5094071640 1 --2- 172.21.15.43:0/189551491 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f508c0cb2d0 0x7f508c15b380 secure :-1 s=READY pgs=427 cs=0 l=1 rev1=1 crypto rx=0x7f508c15de90 tx=0x7f508802b8d0 comp rx=0 tx=0).stop 2023-08-09T13:35:41.097 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:41.092+0000 7f5094071640 1 -- 172.21.15.43:0/189551491 shutdown_connections 2023-08-09T13:35:41.097 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:41.092+0000 7f5094071640 1 --2- 172.21.15.43:0/189551491 >> v2:172.21.15.203:3300/0 conn(0x7f508c0ce8d0 0x7f508c0c9f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:35:41.097 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:41.092+0000 7f5094071640 1 --2- 172.21.15.43:0/189551491 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f508c0cb2d0 0x7f508c15b380 unknown :-1 s=CLOSED pgs=427 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:35:41.097 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:41.092+0000 7f5094071640 1 -- 172.21.15.43:0/189551491 >> 172.21.15.43:0/189551491 conn(0x7f508c0aff50 msgr2=0x7f508c0b0360 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-09T13:35:41.097 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:41.092+0000 7f5094071640 1 -- 172.21.15.43:0/189551491 shutdown_connections 2023-08-09T13:35:41.098 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:41.092+0000 7f5094071640 1 -- 172.21.15.43:0/189551491 wait complete. 2023-08-09T13:35:41.100 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:41.094+0000 7f5094071640 1 Processor -- start 2023-08-09T13:35:41.100 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:41.094+0000 7f5094071640 1 -- start start 2023-08-09T13:35:41.100 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:41.095+0000 7f5094071640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f508c0cb2d0 0x7f508c0c3b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:35:41.100 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:41.095+0000 7f5094071640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f508c0ce8d0 0x7f508c13f4a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:35:41.100 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:41.095+0000 7f5094071640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f508c15df60 0x7f508c1556b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:35:41.101 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:41.095+0000 7f5094071640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f508c0c4070 con 0x7f508c15df60 2023-08-09T13:35:41.101 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:41.095+0000 7f5094071640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f508c13f9e0 con 0x7f508c0ce8d0 2023-08-09T13:35:41.101 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:41.095+0000 7f5094071640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f508c155bf0 con 0x7f508c0cb2d0 2023-08-09T13:35:41.101 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:41.096+0000 7f509260e640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f508c15df60 0x7f508c1556b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:35:41.101 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:41.096+0000 7f509260e640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f508c15df60 0x7f508c1556b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:50062/0 (socket says 172.21.15.43:50062) 2023-08-09T13:35:41.101 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:41.096+0000 7f509260e640 1 -- 172.21.15.43:0/3701353591 learned_addr learned my addr 172.21.15.43:0/3701353591 (peer_addr_for_me v2:172.21.15.43:0/0) 2023-08-09T13:35:41.102 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:41.096+0000 7f509160c640 1 --2- 172.21.15.43:0/3701353591 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f508c0ce8d0 0x7f508c13f4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:35:41.102 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:41.096+0000 7f5091e0d640 1 --2- 172.21.15.43:0/3701353591 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f508c0cb2d0 0x7f508c0c3b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:35:41.102 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:41.096+0000 7f509260e640 1 -- 172.21.15.43:0/3701353591 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f508c0cb2d0 msgr2=0x7f508c0c3b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:35:41.102 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:41.096+0000 7f509260e640 1 --2- 172.21.15.43:0/3701353591 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f508c0cb2d0 0x7f508c0c3b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:35:41.102 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:41.096+0000 7f509260e640 1 -- 172.21.15.43:0/3701353591 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f508c0ce8d0 msgr2=0x7f508c13f4a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:35:41.103 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:41.096+0000 7f509260e640 1 --2- 172.21.15.43:0/3701353591 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f508c0ce8d0 0x7f508c13f4a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:35:41.103 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:41.096+0000 7f509260e640 1 -- 172.21.15.43:0/3701353591 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5088009400 con 0x7f508c15df60 2023-08-09T13:35:41.103 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:41.096+0000 7f509160c640 1 --2- 172.21.15.43:0/3701353591 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f508c0ce8d0 0x7f508c13f4a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-08-09T13:35:41.103 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:41.097+0000 7f509260e640 1 --2- 172.21.15.43:0/3701353591 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f508c15df60 0x7f508c1556b0 secure :-1 s=READY pgs=428 cs=0 l=1 rev1=1 crypto rx=0x7f508400efc0 tx=0x7f508400c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-09T13:35:41.103 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:41.097+0000 7f50827fc640 1 -- 172.21.15.43:0/3701353591 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5084014480 con 0x7f508c15df60 2023-08-09T13:35:41.103 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:41.097+0000 7f50827fc640 1 -- 172.21.15.43:0/3701353591 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f508400f040 con 0x7f508c15df60 2023-08-09T13:35:41.103 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:41.097+0000 7f50827fc640 1 -- 172.21.15.43:0/3701353591 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5084014820 con 0x7f508c15df60 2023-08-09T13:35:41.104 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:41.097+0000 7f5094071640 1 -- 172.21.15.43:0/3701353591 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f508c154610 con 0x7f508c15df60 2023-08-09T13:35:41.104 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:41.097+0000 7f5094071640 1 -- 172.21.15.43:0/3701353591 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f508c154bd0 con 0x7f508c15df60 2023-08-09T13:35:41.105 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:41.098+0000 7f5094071640 1 -- 172.21.15.43:0/3701353591 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f508c0739f0 con 0x7f508c15df60 2023-08-09T13:35:41.105 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:41.099+0000 7f50827fc640 1 -- 172.21.15.43:0/3701353591 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 5) v1 ==== 88341+0+0 (secure 0 0 0) 0x7f5084014980 con 0x7f508c15df60 2023-08-09T13:35:41.105 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:41.101+0000 7f50827fc640 1 --2- 172.21.15.43:0/3701353591 >> [v2:172.21.15.203:6832/830097474,v1:172.21.15.203:6833/830097474] conn(0x7f507006a080 0x7f507006c540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:35:41.105 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:41.101+0000 7f50827fc640 1 -- 172.21.15.43:0/3701353591 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 6871+0+0 (secure 0 0 0) 0x7f5084089550 con 0x7f508c15df60 2023-08-09T13:35:41.105 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:41.102+0000 7f5091e0d640 1 --2- 172.21.15.43:0/3701353591 >> [v2:172.21.15.203:6832/830097474,v1:172.21.15.203:6833/830097474] conn(0x7f507006a080 0x7f507006c540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:35:41.106 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:41.103+0000 7f5091e0d640 1 --2- 172.21.15.43:0/3701353591 >> [v2:172.21.15.203:6832/830097474,v1:172.21.15.203:6833/830097474] conn(0x7f507006a080 0x7f507006c540 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7f5088036000 tx=0x7f5088008030 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-09T13:35:41.108 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:41.105+0000 7f50827fc640 1 -- 172.21.15.43:0/3701353591 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178066 (secure 0 0 0) 0x7f50840547e0 con 0x7f508c15df60 2023-08-09T13:35:41.302 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:41.299+0000 7f5094071640 1 -- 172.21.15.43:0/3701353591 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f508c0b04d0 con 0x7f508c15df60 2023-08-09T13:35:41.303 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:41.300+0000 7f50827fc640 1 -- 172.21.15.43:0/3701353591 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v26) v1 ==== 93+0+10 (secure 0 0 0) 0x7f50840180a0 con 0x7f508c15df60 2023-08-09T13:35:41.304 INFO:teuthology.orchestra.run.smithi043.stdout:pg_num: 1 2023-08-09T13:35:41.305 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:41.303+0000 7f5094071640 1 -- 172.21.15.43:0/3701353591 >> [v2:172.21.15.203:6832/830097474,v1:172.21.15.203:6833/830097474] conn(0x7f507006a080 msgr2=0x7f507006c540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:35:41.306 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:41.303+0000 7f5094071640 1 --2- 172.21.15.43:0/3701353591 >> [v2:172.21.15.203:6832/830097474,v1:172.21.15.203:6833/830097474] conn(0x7f507006a080 0x7f507006c540 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7f5088036000 tx=0x7f5088008030 comp rx=0 tx=0).stop 2023-08-09T13:35:41.306 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:41.303+0000 7f5094071640 1 -- 172.21.15.43:0/3701353591 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f508c15df60 msgr2=0x7f508c1556b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:35:41.306 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:41.303+0000 7f5094071640 1 --2- 172.21.15.43:0/3701353591 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f508c15df60 0x7f508c1556b0 secure :-1 s=READY pgs=428 cs=0 l=1 rev1=1 crypto rx=0x7f508400efc0 tx=0x7f508400c490 comp rx=0 tx=0).stop 2023-08-09T13:35:41.307 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:41.303+0000 7f5094071640 1 -- 172.21.15.43:0/3701353591 shutdown_connections 2023-08-09T13:35:41.307 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:41.303+0000 7f5094071640 1 --2- 172.21.15.43:0/3701353591 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f508c15df60 0x7f508c1556b0 unknown :-1 s=CLOSED pgs=428 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:35:41.307 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:41.303+0000 7f5094071640 1 --2- 172.21.15.43:0/3701353591 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f508c0ce8d0 0x7f508c13f4a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:35:41.307 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:41.303+0000 7f5094071640 1 --2- 172.21.15.43:0/3701353591 >> [v2:172.21.15.203:6832/830097474,v1:172.21.15.203:6833/830097474] conn(0x7f507006a080 0x7f507006c540 unknown :-1 s=CLOSED pgs=215 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:35:41.307 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:41.303+0000 7f5094071640 1 --2- 172.21.15.43:0/3701353591 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f508c0cb2d0 0x7f508c0c3b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:35:41.308 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:41.303+0000 7f5094071640 1 -- 172.21.15.43:0/3701353591 >> 172.21.15.43:0/3701353591 conn(0x7f508c0aff50 msgr2=0x7f508c15d150 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-09T13:35:41.308 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:41.304+0000 7f5094071640 1 -- 172.21.15.43:0/3701353591 shutdown_connections 2023-08-09T13:35:41.308 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:41.304+0000 7f5094071640 1 -- 172.21.15.43:0/3701353591 wait complete. 2023-08-09T13:35:41.324 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-08-09T13:35:41.560 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:41.558+0000 7fa28c7e3640 1 Processor -- start 2023-08-09T13:35:41.561 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:41.558+0000 7fa28c7e3640 1 -- start start 2023-08-09T13:35:41.563 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:41.560+0000 7fa28c7e3640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fa284160f20 0x7fa2840c3130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:35:41.563 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:41.560+0000 7fa28c7e3640 1 -- --> v1:172.21.15.43:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa2840c3670 con 0x7fa2840cc380 2023-08-09T13:35:41.563 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:41.560+0000 7fa28c7e3640 1 -- --> v1:172.21.15.203:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa2840c38a0 con 0x7fa28414a5c0 2023-08-09T13:35:41.563 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:41.560+0000 7fa28c7e3640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa2840c4420 con 0x7fa284160f20 2023-08-09T13:35:41.563 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:41.560+0000 7fa289d7e640 1 --1- >> v1:172.21.15.43:6789/0 conn(0x7fa2840cc380 0x7fa28406bb50 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.43:6789/0 says I am v1:172.21.15.43:44950/0 (socket says 172.21.15.43:44950) 2023-08-09T13:35:41.563 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:41.560+0000 7fa289d7e640 1 -- 172.21.15.43:0/1665753450 learned_addr learned my addr 172.21.15.43:0/1665753450 (peer_addr_for_me v1:172.21.15.43:0/0) 2023-08-09T13:35:41.565 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:41.561+0000 7fa28957d640 1 -- 172.21.15.43:0/1665753450 <== mon.0 v1:172.21.15.43:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 303881130 0 0) 0x7fa2840c3670 con 0x7fa2840cc380 2023-08-09T13:35:41.565 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:41.561+0000 7fa28957d640 1 -- 172.21.15.43:0/1665753450 --> v1:172.21.15.43:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fa270003520 con 0x7fa2840cc380 2023-08-09T13:35:41.565 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:41.561+0000 7fa28ad80640 1 --2- 172.21.15.43:0/1665753450 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fa284160f20 0x7fa2840c3130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:35:41.566 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:41.561+0000 7fa28957d640 1 -- 172.21.15.43:0/1665753450 <== mon.0 v1:172.21.15.43:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2181887296 0 0) 0x7fa270003520 con 0x7fa2840cc380 2023-08-09T13:35:41.566 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:41.561+0000 7fa28957d640 1 -- 172.21.15.43:0/1665753450 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fa284160f20 msgr2=0x7fa2840c3130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-09T13:35:41.566 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:41.561+0000 7fa28957d640 1 --2- 172.21.15.43:0/1665753450 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fa284160f20 0x7fa2840c3130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:35:41.566 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:41.561+0000 7fa28957d640 1 -- 172.21.15.43:0/1665753450 >> v1:172.21.15.203:6789/0 conn(0x7fa28414a5c0 legacy=0x7fa2840cbc70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:35:41.566 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:41.561+0000 7fa28957d640 1 -- 172.21.15.43:0/1665753450 --> v1:172.21.15.43:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa2840c3b80 con 0x7fa2840cc380 2023-08-09T13:35:41.566 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:41.561+0000 7fa28957d640 1 -- 172.21.15.43:0/1665753450 <== mon.0 v1:172.21.15.43:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1285150740 0 0) 0x7fa274001670 con 0x7fa2840cc380 2023-08-09T13:35:41.566 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:41.561+0000 7fa28957d640 1 -- 172.21.15.43:0/1665753450 >> v1:172.21.15.43:6789/0 conn(0x7fa2840cc380 legacy=0x7fa28406bb50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:35:41.567 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:41.561+0000 7fa28957d640 1 --2- 172.21.15.43:0/1665753450 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa270003a10 0x7fa270003e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:35:41.567 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:41.561+0000 7fa28957d640 1 --2- 172.21.15.43:0/1665753450 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fa2700047c0 0x7fa270004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:35:41.567 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:41.561+0000 7fa28957d640 1 --2- 172.21.15.43:0/1665753450 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fa284160f20 0x7fa27000b980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:35:41.567 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:41.561+0000 7fa28957d640 1 -- 172.21.15.43:0/1665753450 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa2840c4420 con 0x7fa270003a10 2023-08-09T13:35:41.568 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:41.561+0000 7fa28957d640 1 -- 172.21.15.43:0/1665753450 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa2700054e0 con 0x7fa2700047c0 2023-08-09T13:35:41.568 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:41.561+0000 7fa28957d640 1 -- 172.21.15.43:0/1665753450 --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa27000bec0 con 0x7fa284160f20 2023-08-09T13:35:41.568 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:41.562+0000 7fa28a57f640 1 --2- 172.21.15.43:0/1665753450 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa270003a10 0x7fa270003e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:35:41.568 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:41.562+0000 7fa289d7e640 1 --2- 172.21.15.43:0/1665753450 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fa2700047c0 0x7fa270004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:35:41.568 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:41.562+0000 7fa28a57f640 1 -- 172.21.15.43:0/1665753450 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fa284160f20 msgr2=0x7fa27000b980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-09T13:35:41.568 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:41.562+0000 7fa28a57f640 1 --2- 172.21.15.43:0/1665753450 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fa284160f20 0x7fa27000b980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:35:41.568 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:41.562+0000 7fa28a57f640 1 -- 172.21.15.43:0/1665753450 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fa2700047c0 msgr2=0x7fa270004c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-09T13:35:41.569 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:41.562+0000 7fa28a57f640 1 --2- 172.21.15.43:0/1665753450 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fa2700047c0 0x7fa270004c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:35:41.569 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:41.562+0000 7fa28a57f640 1 -- 172.21.15.43:0/1665753450 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa270003520 con 0x7fa270003a10 2023-08-09T13:35:41.571 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:41.563+0000 7fa28a57f640 1 --2- 172.21.15.43:0/1665753450 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa270003a10 0x7fa270003e30 secure :-1 s=READY pgs=430 cs=0 l=1 rev1=1 crypto rx=0x7fa280001750 tx=0x7fa2700057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=70f76524c046e7d1 server_cookie=0 in_seq=0 out_seq=0 2023-08-09T13:35:41.571 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:41.563+0000 7fa28957d640 1 -- 172.21.15.43:0/1665753450 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa2800052d0 con 0x7fa270003a10 2023-08-09T13:35:41.571 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:41.564+0000 7fa28957d640 1 -- 172.21.15.43:0/1665753450 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa28002e050 con 0x7fa270003a10 2023-08-09T13:35:41.571 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:41.564+0000 7fa28957d640 1 -- 172.21.15.43:0/1665753450 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa2800035f0 con 0x7fa270003a10 2023-08-09T13:35:41.571 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:41.564+0000 7fa28c7e3640 1 -- 172.21.15.43:0/1665753450 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa270003a10 msgr2=0x7fa270003e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:35:41.571 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:41.564+0000 7fa28c7e3640 1 --2- 172.21.15.43:0/1665753450 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa270003a10 0x7fa270003e30 secure :-1 s=READY pgs=430 cs=0 l=1 rev1=1 crypto rx=0x7fa280001750 tx=0x7fa2700057f0 comp rx=0 tx=0).stop 2023-08-09T13:35:41.571 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:41.564+0000 7fa28ad80640 1 -- 172.21.15.43:0/1665753450 reap_dead start 2023-08-09T13:35:41.572 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:41.564+0000 7fa28c7e3640 1 -- 172.21.15.43:0/1665753450 shutdown_connections 2023-08-09T13:35:41.572 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:41.564+0000 7fa28c7e3640 1 -- 172.21.15.43:0/1665753450 >> 172.21.15.43:0/1665753450 conn(0x7fa2840ce0a0 msgr2=0x7fa2840ce480 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-09T13:35:41.572 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:41.564+0000 7fa28c7e3640 1 -- 172.21.15.43:0/1665753450 shutdown_connections 2023-08-09T13:35:41.572 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:41.564+0000 7fa28c7e3640 1 -- 172.21.15.43:0/1665753450 wait complete. 2023-08-09T13:35:41.572 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:41.566+0000 7fa28c7e3640 1 Processor -- start 2023-08-09T13:35:41.572 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:41.566+0000 7fa28c7e3640 1 -- start start 2023-08-09T13:35:41.572 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:41.567+0000 7fa28c7e3640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fa270003a10 0x7fa284160590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:35:41.573 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:41.567+0000 7fa28c7e3640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fa2841620c0 0x7fa2841f6480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:35:41.573 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:41.567+0000 7fa28c7e3640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa284162540 0x7fa284163b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:35:41.573 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:41.567+0000 7fa28c7e3640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa284163700 con 0x7fa284162540 2023-08-09T13:35:41.573 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:41.567+0000 7fa28c7e3640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa284160ad0 con 0x7fa2841620c0 2023-08-09T13:35:41.573 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:41.567+0000 7fa28c7e3640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa284162a80 con 0x7fa270003a10 2023-08-09T13:35:41.574 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:41.568+0000 7fa28ad80640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa284162540 0x7fa284163b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:35:41.574 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:41.568+0000 7fa28ad80640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa284162540 0x7fa284163b20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:50074/0 (socket says 172.21.15.43:50074) 2023-08-09T13:35:41.574 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:41.568+0000 7fa28ad80640 1 -- 172.21.15.43:0/2041523238 learned_addr learned my addr 172.21.15.43:0/2041523238 (peer_addr_for_me v2:172.21.15.43:0/0) 2023-08-09T13:35:41.574 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:41.568+0000 7fa289d7e640 1 --2- 172.21.15.43:0/2041523238 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fa2841620c0 0x7fa2841f6480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:35:41.575 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:41.568+0000 7fa28ad80640 1 -- 172.21.15.43:0/2041523238 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fa270003a10 msgr2=0x7fa284160590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:35:41.575 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:41.568+0000 7fa28ad80640 1 --2- 172.21.15.43:0/2041523238 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fa270003a10 0x7fa284160590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:35:41.575 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:41.568+0000 7fa28ad80640 1 -- 172.21.15.43:0/2041523238 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fa2841620c0 msgr2=0x7fa2841f6480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:35:41.575 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:41.568+0000 7fa28ad80640 1 --2- 172.21.15.43:0/2041523238 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fa2841620c0 0x7fa2841f6480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:35:41.576 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:41.568+0000 7fa28ad80640 1 -- 172.21.15.43:0/2041523238 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa284162d60 con 0x7fa284162540 2023-08-09T13:35:41.576 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:41.568+0000 7fa28ad80640 1 --2- 172.21.15.43:0/2041523238 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa284162540 0x7fa284163b20 secure :-1 s=READY pgs=431 cs=0 l=1 rev1=1 crypto rx=0x7fa2840c49b0 tx=0x7fa27c00e4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-09T13:35:41.577 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:41.569+0000 7fa27affd640 1 -- 172.21.15.43:0/2041523238 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa27c00ed90 con 0x7fa284162540 2023-08-09T13:35:41.577 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:41.569+0000 7fa27affd640 1 -- 172.21.15.43:0/2041523238 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa27c014420 con 0x7fa284162540 2023-08-09T13:35:41.577 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:41.569+0000 7fa27affd640 1 -- 172.21.15.43:0/2041523238 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa27c0147d0 con 0x7fa284162540 2023-08-09T13:35:41.577 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:41.569+0000 7fa28c7e3640 1 -- 172.21.15.43:0/2041523238 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa284163040 con 0x7fa284162540 2023-08-09T13:35:41.577 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:41.569+0000 7fa28c7e3640 1 -- 172.21.15.43:0/2041523238 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa28406ff30 con 0x7fa284162540 2023-08-09T13:35:41.578 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:41.570+0000 7fa28c7e3640 1 -- 172.21.15.43:0/2041523238 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa2840700f0 con 0x7fa284162540 2023-08-09T13:35:41.578 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:41.571+0000 7fa27affd640 1 -- 172.21.15.43:0/2041523238 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 5) v1 ==== 88341+0+0 (secure 0 0 0) 0x7fa27c014930 con 0x7fa284162540 2023-08-09T13:35:41.578 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:41.572+0000 7fa27affd640 1 --2- 172.21.15.43:0/2041523238 >> [v2:172.21.15.203:6832/830097474,v1:172.21.15.203:6833/830097474] conn(0x7fa26806a080 0x7fa26806c540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:35:41.578 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:41.572+0000 7fa27affd640 1 -- 172.21.15.43:0/2041523238 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 6871+0+0 (secure 0 0 0) 0x7fa27c088d80 con 0x7fa284162540 2023-08-09T13:35:41.578 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:41.573+0000 7fa28a57f640 1 --2- 172.21.15.43:0/2041523238 >> [v2:172.21.15.203:6832/830097474,v1:172.21.15.203:6833/830097474] conn(0x7fa26806a080 0x7fa26806c540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:35:41.578 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:41.574+0000 7fa28a57f640 1 --2- 172.21.15.43:0/2041523238 >> [v2:172.21.15.203:6832/830097474,v1:172.21.15.203:6833/830097474] conn(0x7fa26806a080 0x7fa26806c540 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7fa280033000 tx=0x7fa28002f000 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-09T13:35:41.579 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:41.576+0000 7fa27affd640 1 -- 172.21.15.43:0/2041523238 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178066 (secure 0 0 0) 0x7fa27c0540a0 con 0x7fa284162540 2023-08-09T13:35:41.776 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:41.774+0000 7fa28c7e3640 1 -- 172.21.15.43:0/2041523238 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7fa2840c3e30 con 0x7fa284162540 2023-08-09T13:35:41.777 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:41.774+0000 7fa27affd640 1 -- 172.21.15.43:0/2041523238 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v26) v1 ==== 104+0+11 (secure 0 0 0) 0x7fa27c0576c0 con 0x7fa284162540 2023-08-09T13:35:41.777 INFO:teuthology.orchestra.run.smithi043.stdout:pg_num: 32 2023-08-09T13:35:41.780 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:41.777+0000 7fa28c7e3640 1 -- 172.21.15.43:0/2041523238 >> [v2:172.21.15.203:6832/830097474,v1:172.21.15.203:6833/830097474] conn(0x7fa26806a080 msgr2=0x7fa26806c540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:35:41.780 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:41.777+0000 7fa28c7e3640 1 --2- 172.21.15.43:0/2041523238 >> [v2:172.21.15.203:6832/830097474,v1:172.21.15.203:6833/830097474] conn(0x7fa26806a080 0x7fa26806c540 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7fa280033000 tx=0x7fa28002f000 comp rx=0 tx=0).stop 2023-08-09T13:35:41.780 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:41.777+0000 7fa28c7e3640 1 -- 172.21.15.43:0/2041523238 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa284162540 msgr2=0x7fa284163b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:35:41.781 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:41.777+0000 7fa28c7e3640 1 --2- 172.21.15.43:0/2041523238 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa284162540 0x7fa284163b20 secure :-1 s=READY pgs=431 cs=0 l=1 rev1=1 crypto rx=0x7fa2840c49b0 tx=0x7fa27c00e4b0 comp rx=0 tx=0).stop 2023-08-09T13:35:41.781 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:41.777+0000 7fa28c7e3640 1 -- 172.21.15.43:0/2041523238 shutdown_connections 2023-08-09T13:35:41.781 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:41.777+0000 7fa28c7e3640 1 --2- 172.21.15.43:0/2041523238 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa284162540 0x7fa284163b20 unknown :-1 s=CLOSED pgs=431 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:35:41.781 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:41.777+0000 7fa28c7e3640 1 --2- 172.21.15.43:0/2041523238 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fa2841620c0 0x7fa2841f6480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:35:41.781 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:41.777+0000 7fa28c7e3640 1 --2- 172.21.15.43:0/2041523238 >> [v2:172.21.15.203:6832/830097474,v1:172.21.15.203:6833/830097474] conn(0x7fa26806a080 0x7fa26806c540 unknown :-1 s=CLOSED pgs=216 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:35:41.781 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:41.777+0000 7fa28c7e3640 1 --2- 172.21.15.43:0/2041523238 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fa270003a10 0x7fa284160590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:35:41.781 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:41.777+0000 7fa28c7e3640 1 -- 172.21.15.43:0/2041523238 >> 172.21.15.43:0/2041523238 conn(0x7fa2840ce0a0 msgr2=0x7fa28406c4a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-09T13:35:41.783 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:41.778+0000 7fa28c7e3640 1 -- 172.21.15.43:0/2041523238 shutdown_connections 2023-08-09T13:35:41.783 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:41.778+0000 7fa28c7e3640 1 -- 172.21.15.43:0/2041523238 wait complete. 2023-08-09T13:35:41.801 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-08-09T13:35:42.037 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:42.035+0000 7fe346637640 1 Processor -- start 2023-08-09T13:35:42.038 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:42.035+0000 7fe346637640 1 -- start start 2023-08-09T13:35:42.039 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:42.036+0000 7fe346637640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7fe3400b7df0 0x7fe3400d1b30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:35:42.039 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:42.037+0000 7fe346637640 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7fe3400cebf0 0x7fe3400cf040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:35:42.039 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:42.037+0000 7fe346637640 1 -- --> v1:172.21.15.203:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe3400cf580 con 0x7fe34015c2d0 2023-08-09T13:35:42.040 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:42.037+0000 7fe346637640 1 -- --> v2:172.21.15.43:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe3400b8230 con 0x7fe3400b7df0 2023-08-09T13:35:42.040 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:42.037+0000 7fe346637640 1 -- --> v2:172.21.15.203:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe3400d16a0 con 0x7fe3400cebf0 2023-08-09T13:35:42.041 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:42.037+0000 7fe344e34640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7fe3400b7df0 0x7fe3400d1b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:35:42.041 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:42.037+0000 7fe344e34640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7fe3400b7df0 0x7fe3400d1b30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:50088/0 (socket says 172.21.15.43:50088) 2023-08-09T13:35:42.041 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:42.037+0000 7fe344e34640 1 -- 172.21.15.43:0/1911344269 learned_addr learned my addr 172.21.15.43:0/1911344269 (peer_addr_for_me v2:172.21.15.43:0/0) 2023-08-09T13:35:42.041 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:42.037+0000 7fe345e36640 1 --2- 172.21.15.43:0/1911344269 >> v2:172.21.15.203:3300/0 conn(0x7fe3400cebf0 0x7fe3400cf040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:35:42.041 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:42.038+0000 7fe344e34640 1 -- 172.21.15.43:0/1911344269 >> v2:172.21.15.203:3300/0 conn(0x7fe3400cebf0 msgr2=0x7fe3400cf040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-09T13:35:42.041 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:42.038+0000 7fe344e34640 1 --2- 172.21.15.43:0/1911344269 >> v2:172.21.15.203:3300/0 conn(0x7fe3400cebf0 0x7fe3400cf040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:35:42.041 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:42.038+0000 7fe344e34640 1 -- 172.21.15.43:0/1911344269 >> v1:172.21.15.203:6789/0 conn(0x7fe34015c2d0 legacy=0x7fe3400cd020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:35:42.042 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:42.038+0000 7fe344e34640 1 -- 172.21.15.43:0/1911344269 --> v2:172.21.15.43:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe3400d0d70 con 0x7fe3400b7df0 2023-08-09T13:35:42.043 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:42.039+0000 7fe344e34640 1 --2- 172.21.15.43:0/1911344269 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fe3400b7df0 0x7fe3400d1b30 secure :-1 s=READY pgs=432 cs=0 l=1 rev1=1 crypto rx=0x7fe3400cfd60 tx=0x7fe33002b9a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=eebe364458c48454 server_cookie=0 in_seq=0 out_seq=0 2023-08-09T13:35:42.043 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:42.039+0000 7fe337fff640 1 -- 172.21.15.43:0/1911344269 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe330030860 con 0x7fe3400b7df0 2023-08-09T13:35:42.043 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:42.039+0000 7fe337fff640 1 -- 172.21.15.43:0/1911344269 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe3300309c0 con 0x7fe3400b7df0 2023-08-09T13:35:42.043 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:42.039+0000 7fe337fff640 1 -- 172.21.15.43:0/1911344269 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe330030d50 con 0x7fe3400b7df0 2023-08-09T13:35:42.043 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:42.040+0000 7fe346637640 1 -- 172.21.15.43:0/1911344269 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fe3400b7df0 msgr2=0x7fe3400d1b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:35:42.043 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:42.040+0000 7fe346637640 1 --2- 172.21.15.43:0/1911344269 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fe3400b7df0 0x7fe3400d1b30 secure :-1 s=READY pgs=432 cs=0 l=1 rev1=1 crypto rx=0x7fe3400cfd60 tx=0x7fe33002b9a0 comp rx=0 tx=0).stop 2023-08-09T13:35:42.043 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:42.040+0000 7fe346637640 1 -- 172.21.15.43:0/1911344269 shutdown_connections 2023-08-09T13:35:42.044 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:42.040+0000 7fe346637640 1 --2- 172.21.15.43:0/1911344269 >> v2:172.21.15.203:3300/0 conn(0x7fe3400cebf0 0x7fe3400cf040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:35:42.044 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:42.040+0000 7fe346637640 1 --2- 172.21.15.43:0/1911344269 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fe3400b7df0 0x7fe3400d1b30 unknown :-1 s=CLOSED pgs=432 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:35:42.044 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:42.040+0000 7fe346637640 1 -- 172.21.15.43:0/1911344269 >> 172.21.15.43:0/1911344269 conn(0x7fe34006a230 msgr2=0x7fe34006a640 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-09T13:35:42.044 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:42.040+0000 7fe346637640 1 -- 172.21.15.43:0/1911344269 shutdown_connections 2023-08-09T13:35:42.044 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:42.040+0000 7fe346637640 1 -- 172.21.15.43:0/1911344269 wait complete. 2023-08-09T13:35:42.045 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:42.042+0000 7fe346637640 1 Processor -- start 2023-08-09T13:35:42.045 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:42.042+0000 7fe346637640 1 -- start start 2023-08-09T13:35:42.046 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:42.044+0000 7fe346637640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fe3400b7df0 0x7fe34014d630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:35:42.046 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:42.044+0000 7fe346637640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fe3400cebf0 0x7fe34015ecf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:35:42.047 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:42.044+0000 7fe346637640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fe34015c2d0 0x7fe34015f230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:35:42.047 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:42.044+0000 7fe346637640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe34015cd80 con 0x7fe34015c2d0 2023-08-09T13:35:42.047 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:42.044+0000 7fe346637640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe34015c750 con 0x7fe3400b7df0 2023-08-09T13:35:42.047 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:42.044+0000 7fe346637640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe340156490 con 0x7fe3400cebf0 2023-08-09T13:35:42.047 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:42.044+0000 7fe345e36640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fe34015c2d0 0x7fe34015f230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:35:42.047 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:42.044+0000 7fe345e36640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fe34015c2d0 0x7fe34015f230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:50096/0 (socket says 172.21.15.43:50096) 2023-08-09T13:35:42.047 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:42.044+0000 7fe345e36640 1 -- 172.21.15.43:0/1341964143 learned_addr learned my addr 172.21.15.43:0/1341964143 (peer_addr_for_me v2:172.21.15.43:0/0) 2023-08-09T13:35:42.048 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:42.044+0000 7fe345635640 1 --2- 172.21.15.43:0/1341964143 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fe3400b7df0 0x7fe34014d630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:35:42.049 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:42.045+0000 7fe345e36640 1 -- 172.21.15.43:0/1341964143 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fe3400cebf0 msgr2=0x7fe34015ecf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:35:42.049 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:42.045+0000 7fe345e36640 1 --2- 172.21.15.43:0/1341964143 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fe3400cebf0 0x7fe34015ecf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:35:42.049 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:42.045+0000 7fe345e36640 1 -- 172.21.15.43:0/1341964143 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fe3400b7df0 msgr2=0x7fe34014d630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:35:42.049 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:42.045+0000 7fe345e36640 1 --2- 172.21.15.43:0/1341964143 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fe3400b7df0 0x7fe34014d630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:35:42.049 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:42.045+0000 7fe345e36640 1 -- 172.21.15.43:0/1341964143 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe3300094d0 con 0x7fe34015c2d0 2023-08-09T13:35:42.049 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:42.045+0000 7fe345e36640 1 --2- 172.21.15.43:0/1341964143 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fe34015c2d0 0x7fe34015f230 secure :-1 s=READY pgs=433 cs=0 l=1 rev1=1 crypto rx=0x7fe33800d800 tx=0x7fe33800dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-09T13:35:42.049 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:42.045+0000 7fe335ffb640 1 -- 172.21.15.43:0/1341964143 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe33800b6c0 con 0x7fe34015c2d0 2023-08-09T13:35:42.050 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:42.045+0000 7fe335ffb640 1 -- 172.21.15.43:0/1341964143 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe33800b840 con 0x7fe34015c2d0 2023-08-09T13:35:42.050 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:42.045+0000 7fe346637640 1 -- 172.21.15.43:0/1341964143 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe34015ca10 con 0x7fe34015c2d0 2023-08-09T13:35:42.050 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:42.045+0000 7fe335ffb640 1 -- 172.21.15.43:0/1341964143 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe33800bbf0 con 0x7fe34015c2d0 2023-08-09T13:35:42.050 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:42.046+0000 7fe346637640 1 -- 172.21.15.43:0/1341964143 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe34014db70 con 0x7fe34015c2d0 2023-08-09T13:35:42.050 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:42.047+0000 7fe346637640 1 -- 172.21.15.43:0/1341964143 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe34015cef0 con 0x7fe34015c2d0 2023-08-09T13:35:42.051 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:42.048+0000 7fe335ffb640 1 -- 172.21.15.43:0/1341964143 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 5) v1 ==== 88341+0+0 (secure 0 0 0) 0x7fe33800bd50 con 0x7fe34015c2d0 2023-08-09T13:35:42.051 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:42.049+0000 7fe335ffb640 1 --2- 172.21.15.43:0/1341964143 >> [v2:172.21.15.203:6832/830097474,v1:172.21.15.203:6833/830097474] conn(0x7fe32406a080 0x7fe32406c540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:35:42.051 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:42.049+0000 7fe335ffb640 1 -- 172.21.15.43:0/1341964143 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 6871+0+0 (secure 0 0 0) 0x7fe338088e60 con 0x7fe34015c2d0 2023-08-09T13:35:42.052 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:42.050+0000 7fe345635640 1 --2- 172.21.15.43:0/1341964143 >> [v2:172.21.15.203:6832/830097474,v1:172.21.15.203:6833/830097474] conn(0x7fe32406a080 0x7fe32406c540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:35:42.053 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:42.051+0000 7fe345635640 1 --2- 172.21.15.43:0/1341964143 >> [v2:172.21.15.203:6832/830097474,v1:172.21.15.203:6833/830097474] conn(0x7fe32406a080 0x7fe32406c540 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7fe340153980 tx=0x7fe33c009040 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-09T13:35:42.056 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:42.054+0000 7fe335ffb640 1 -- 172.21.15.43:0/1341964143 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178066 (secure 0 0 0) 0x7fe338054180 con 0x7fe34015c2d0 2023-08-09T13:35:42.252 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:42.249+0000 7fe346637640 1 -- 172.21.15.43:0/1341964143 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7fe34004e390 con 0x7fe34015c2d0 2023-08-09T13:35:42.253 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:42.250+0000 7fe335ffb640 1 -- 172.21.15.43:0/1341964143 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v26) v1 ==== 100+0+11 (secure 0 0 0) 0x7fe33805be60 con 0x7fe34015c2d0 2023-08-09T13:35:42.253 INFO:teuthology.orchestra.run.smithi043.stdout:pg_num: 64 2023-08-09T13:35:42.256 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:42.253+0000 7fe346637640 1 -- 172.21.15.43:0/1341964143 >> [v2:172.21.15.203:6832/830097474,v1:172.21.15.203:6833/830097474] conn(0x7fe32406a080 msgr2=0x7fe32406c540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:35:42.256 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:42.253+0000 7fe346637640 1 --2- 172.21.15.43:0/1341964143 >> [v2:172.21.15.203:6832/830097474,v1:172.21.15.203:6833/830097474] conn(0x7fe32406a080 0x7fe32406c540 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7fe340153980 tx=0x7fe33c009040 comp rx=0 tx=0).stop 2023-08-09T13:35:42.256 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:42.253+0000 7fe346637640 1 -- 172.21.15.43:0/1341964143 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fe34015c2d0 msgr2=0x7fe34015f230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:35:42.256 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:42.253+0000 7fe346637640 1 --2- 172.21.15.43:0/1341964143 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fe34015c2d0 0x7fe34015f230 secure :-1 s=READY pgs=433 cs=0 l=1 rev1=1 crypto rx=0x7fe33800d800 tx=0x7fe33800dcc0 comp rx=0 tx=0).stop 2023-08-09T13:35:42.256 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:42.253+0000 7fe346637640 1 -- 172.21.15.43:0/1341964143 shutdown_connections 2023-08-09T13:35:42.256 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:42.253+0000 7fe346637640 1 --2- 172.21.15.43:0/1341964143 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fe34015c2d0 0x7fe34015f230 unknown :-1 s=CLOSED pgs=433 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:35:42.257 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:42.253+0000 7fe346637640 1 --2- 172.21.15.43:0/1341964143 >> [v2:172.21.15.203:6832/830097474,v1:172.21.15.203:6833/830097474] conn(0x7fe32406a080 0x7fe32406c540 unknown :-1 s=CLOSED pgs=217 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:35:42.257 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:42.253+0000 7fe346637640 1 --2- 172.21.15.43:0/1341964143 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fe3400cebf0 0x7fe34015ecf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:35:42.257 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:42.253+0000 7fe346637640 1 --2- 172.21.15.43:0/1341964143 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fe3400b7df0 0x7fe34014d630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:35:42.257 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:42.253+0000 7fe346637640 1 -- 172.21.15.43:0/1341964143 >> 172.21.15.43:0/1341964143 conn(0x7fe34006a230 msgr2=0x7fe3400b25c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-09T13:35:42.257 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:42.253+0000 7fe346637640 1 -- 172.21.15.43:0/1341964143 shutdown_connections 2023-08-09T13:35:42.257 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:42.254+0000 7fe346637640 1 -- 172.21.15.43:0/1341964143 wait complete. 2023-08-09T13:35:42.276 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2023-08-09T13:35:42.510 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:42.507+0000 7f3aaa147640 1 Processor -- start 2023-08-09T13:35:42.510 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:42.508+0000 7f3aaa147640 1 -- start start 2023-08-09T13:35:42.512 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:42.509+0000 7f3aaa147640 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7f3aa415c680 0x7f3aa40ce760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:35:42.512 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:42.509+0000 7f3aaa147640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f3aa40b3fa0 0x7f3aa40b43b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:35:42.512 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:42.509+0000 7f3aaa147640 1 -- --> v1:172.21.15.203:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3aa40b48f0 con 0x7f3aa40d0a70 2023-08-09T13:35:42.512 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:42.509+0000 7f3aaa147640 1 -- --> v2:172.21.15.43:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3aa40cdf00 con 0x7f3aa40b3fa0 2023-08-09T13:35:42.512 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:42.509+0000 7f3aaa147640 1 -- --> v2:172.21.15.203:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3aa40d0f70 con 0x7f3aa415c680 2023-08-09T13:35:42.513 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:42.510+0000 7f3aa3fff640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f3aa40b3fa0 0x7f3aa40b43b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:35:42.513 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:42.510+0000 7f3aa3fff640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f3aa40b3fa0 0x7f3aa40b43b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:50106/0 (socket says 172.21.15.43:50106) 2023-08-09T13:35:42.513 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:42.510+0000 7f3aa3fff640 1 -- 172.21.15.43:0/3245565896 learned_addr learned my addr 172.21.15.43:0/3245565896 (peer_addr_for_me v2:172.21.15.43:0/0) 2023-08-09T13:35:42.514 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:42.510+0000 7f3aa37fe640 1 --2- 172.21.15.43:0/3245565896 >> v2:172.21.15.203:3300/0 conn(0x7f3aa415c680 0x7f3aa40ce760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:35:42.514 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:42.510+0000 7f3aa3fff640 1 -- 172.21.15.43:0/3245565896 >> v2:172.21.15.203:3300/0 conn(0x7f3aa415c680 msgr2=0x7f3aa40ce760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-09T13:35:42.514 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:42.510+0000 7f3aa3fff640 1 --2- 172.21.15.43:0/3245565896 >> v2:172.21.15.203:3300/0 conn(0x7f3aa415c680 0x7f3aa40ce760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:35:42.514 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:42.510+0000 7f3aa3fff640 1 -- 172.21.15.43:0/3245565896 >> v1:172.21.15.203:6789/0 conn(0x7f3aa40d0a70 legacy=0x7f3aa40cd7f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-09T13:35:42.514 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:42.510+0000 7f3aa3fff640 1 -- 172.21.15.43:0/3245565896 --> v2:172.21.15.43:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3aa40ccd10 con 0x7f3aa40b3fa0 2023-08-09T13:35:42.514 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:42.510+0000 7f3aa37fe640 1 --2- 172.21.15.43:0/3245565896 >> v2:172.21.15.203:3300/0 conn(0x7f3aa415c680 0x7f3aa40ce760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-08-09T13:35:42.515 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:42.511+0000 7f3aa3fff640 1 --2- 172.21.15.43:0/3245565896 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f3aa40b3fa0 0x7f3aa40b43b0 secure :-1 s=READY pgs=434 cs=0 l=1 rev1=1 crypto rx=0x7f3aa406b7d0 tx=0x7f3a9c02bee0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=268955b2ea8e0992 server_cookie=0 in_seq=0 out_seq=0 2023-08-09T13:35:42.515 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:42.512+0000 7f3aa2ffd640 1 -- 172.21.15.43:0/3245565896 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3a9c02ca80 con 0x7f3aa40b3fa0 2023-08-09T13:35:42.516 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:42.512+0000 7f3aa2ffd640 1 -- 172.21.15.43:0/3245565896 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3a9c02cbe0 con 0x7f3aa40b3fa0 2023-08-09T13:35:42.516 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:42.512+0000 7f3aa2ffd640 1 -- 172.21.15.43:0/3245565896 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3a9c0343f0 con 0x7f3aa40b3fa0 2023-08-09T13:35:42.516 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:42.512+0000 7f3aaa147640 1 -- 172.21.15.43:0/3245565896 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f3aa40b3fa0 msgr2=0x7f3aa40b43b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:35:42.516 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:42.512+0000 7f3aaa147640 1 --2- 172.21.15.43:0/3245565896 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f3aa40b3fa0 0x7f3aa40b43b0 secure :-1 s=READY pgs=434 cs=0 l=1 rev1=1 crypto rx=0x7f3aa406b7d0 tx=0x7f3a9c02bee0 comp rx=0 tx=0).stop 2023-08-09T13:35:42.516 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:42.513+0000 7f3aaa147640 1 -- 172.21.15.43:0/3245565896 shutdown_connections 2023-08-09T13:35:42.516 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:42.513+0000 7f3aaa147640 1 --2- 172.21.15.43:0/3245565896 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f3aa40b3fa0 0x7f3aa40b43b0 unknown :-1 s=CLOSED pgs=434 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:35:42.516 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:42.513+0000 7f3aaa147640 1 --2- 172.21.15.43:0/3245565896 >> v2:172.21.15.203:3300/0 conn(0x7f3aa415c680 0x7f3aa40ce760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:35:42.517 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:42.513+0000 7f3aaa147640 1 -- 172.21.15.43:0/3245565896 >> 172.21.15.43:0/3245565896 conn(0x7f3aa4071520 msgr2=0x7f3aa40d2520 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-09T13:35:42.517 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:42.513+0000 7f3aaa147640 1 -- 172.21.15.43:0/3245565896 shutdown_connections 2023-08-09T13:35:42.517 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:42.513+0000 7f3aaa147640 1 -- 172.21.15.43:0/3245565896 wait complete. 2023-08-09T13:35:42.517 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:42.515+0000 7f3aaa147640 1 Processor -- start 2023-08-09T13:35:42.517 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:42.515+0000 7f3aaa147640 1 -- start start 2023-08-09T13:35:42.519 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:42.516+0000 7f3aaa147640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f3aa40b3fa0 0x7f3aa41504a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:35:42.519 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:42.516+0000 7f3aaa147640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f3aa40d0a70 0x7f3aa415e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:35:42.519 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:42.516+0000 7f3aaa147640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f3aa415c680 0x7f3aa40cb9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:35:42.519 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:42.516+0000 7f3aaa147640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3aa4151b10 con 0x7f3aa415c680 2023-08-09T13:35:42.519 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:42.516+0000 7f3aaa147640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3aa40c9920 con 0x7f3aa40d0a70 2023-08-09T13:35:42.519 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:42.516+0000 7f3aaa147640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3aa40c95f0 con 0x7f3aa40b3fa0 2023-08-09T13:35:42.519 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:42.516+0000 7f3aa3fff640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f3aa415c680 0x7f3aa40cb9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:35:42.520 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:42.516+0000 7f3aa3fff640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f3aa415c680 0x7f3aa40cb9c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:50118/0 (socket says 172.21.15.43:50118) 2023-08-09T13:35:42.520 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:42.516+0000 7f3aa3fff640 1 -- 172.21.15.43:0/3536545824 learned_addr learned my addr 172.21.15.43:0/3536545824 (peer_addr_for_me v2:172.21.15.43:0/0) 2023-08-09T13:35:42.521 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:42.517+0000 7f3aa37fe640 1 --2- 172.21.15.43:0/3536545824 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f3aa40b3fa0 0x7f3aa41504a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:35:42.521 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:42.517+0000 7f3a9bfff640 1 --2- 172.21.15.43:0/3536545824 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f3aa40d0a70 0x7f3aa415e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:35:42.521 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:42.517+0000 7f3aa3fff640 1 -- 172.21.15.43:0/3536545824 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f3aa40b3fa0 msgr2=0x7f3aa41504a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:35:42.521 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:42.517+0000 7f3aa3fff640 1 --2- 172.21.15.43:0/3536545824 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f3aa40b3fa0 0x7f3aa41504a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:35:42.521 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:42.517+0000 7f3aa3fff640 1 -- 172.21.15.43:0/3536545824 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f3aa40d0a70 msgr2=0x7f3aa415e2f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:35:42.522 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:42.517+0000 7f3aa3fff640 1 --2- 172.21.15.43:0/3536545824 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f3aa40d0a70 0x7f3aa415e2f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:35:42.522 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:42.517+0000 7f3aa3fff640 1 -- 172.21.15.43:0/3536545824 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3a9c009b30 con 0x7f3aa415c680 2023-08-09T13:35:42.522 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:42.517+0000 7f3a9bfff640 1 --2- 172.21.15.43:0/3536545824 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f3aa40d0a70 0x7f3aa415e2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-08-09T13:35:42.522 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:42.517+0000 7f3aa37fe640 1 --2- 172.21.15.43:0/3536545824 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f3aa40b3fa0 0x7f3aa41504a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-08-09T13:35:42.522 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:42.517+0000 7f3aa3fff640 1 --2- 172.21.15.43:0/3536545824 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f3aa415c680 0x7f3aa40cb9c0 secure :-1 s=READY pgs=435 cs=0 l=1 rev1=1 crypto rx=0x7f3a9c036040 tx=0x7f3a9c004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-09T13:35:42.522 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:42.518+0000 7f3aa0ff9640 1 -- 172.21.15.43:0/3536545824 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3a9c03e450 con 0x7f3aa415c680 2023-08-09T13:35:42.522 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:42.518+0000 7f3aa0ff9640 1 -- 172.21.15.43:0/3536545824 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3a9c03e5d0 con 0x7f3aa415c680 2023-08-09T13:35:42.522 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:42.518+0000 7f3aa0ff9640 1 -- 172.21.15.43:0/3536545824 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3a9c03e980 con 0x7f3aa415c680 2023-08-09T13:35:42.523 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:42.518+0000 7f3aaa147640 1 -- 172.21.15.43:0/3536545824 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3aa415ec60 con 0x7f3aa415c680 2023-08-09T13:35:42.524 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:42.518+0000 7f3aaa147640 1 -- 172.21.15.43:0/3536545824 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3aa415f220 con 0x7f3aa415c680 2023-08-09T13:35:42.524 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:42.519+0000 7f3aaa147640 1 -- 172.21.15.43:0/3536545824 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3aa415f440 con 0x7f3aa415c680 2023-08-09T13:35:42.524 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:42.520+0000 7f3aa0ff9640 1 -- 172.21.15.43:0/3536545824 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 5) v1 ==== 88341+0+0 (secure 0 0 0) 0x7f3a9c02cd50 con 0x7f3aa415c680 2023-08-09T13:35:42.525 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:42.521+0000 7f3aa0ff9640 1 --2- 172.21.15.43:0/3536545824 >> [v2:172.21.15.203:6832/830097474,v1:172.21.15.203:6833/830097474] conn(0x7f3a8406a050 0x7f3a8406c510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:35:42.525 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:42.521+0000 7f3aa0ff9640 1 -- 172.21.15.43:0/3536545824 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 6871+0+0 (secure 0 0 0) 0x7f3a9c0ab470 con 0x7f3aa415c680 2023-08-09T13:35:42.525 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:42.522+0000 7f3aa37fe640 1 --2- 172.21.15.43:0/3536545824 >> [v2:172.21.15.203:6832/830097474,v1:172.21.15.203:6833/830097474] conn(0x7f3a8406a050 0x7f3a8406c510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:35:42.525 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:42.523+0000 7f3aa37fe640 1 --2- 172.21.15.43:0/3536545824 >> [v2:172.21.15.203:6832/830097474,v1:172.21.15.203:6833/830097474] conn(0x7f3a8406a050 0x7f3a8406c510 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7f3a94006fd0 tx=0x7f3a94008040 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-09T13:35:42.528 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:42.525+0000 7f3aa0ff9640 1 -- 172.21.15.43:0/3536545824 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178066 (secure 0 0 0) 0x7f3aa415f440 con 0x7f3aa415c680 2023-08-09T13:35:42.722 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:42.719+0000 7f3aaa147640 1 -- 172.21.15.43:0/3536545824 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) v1 -- 0x7f3aa415f440 con 0x7f3aa415c680 2023-08-09T13:35:42.723 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:42.720+0000 7f3aa0ff9640 1 -- 172.21.15.43:0/3536545824 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v26) v1 ==== 103+0+11 (secure 0 0 0) 0x7f3a9c043020 con 0x7f3aa415c680 2023-08-09T13:35:42.723 INFO:teuthology.orchestra.run.smithi043.stdout:pg_num: 32 2023-08-09T13:35:42.726 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:42.724+0000 7f3aaa147640 1 -- 172.21.15.43:0/3536545824 >> [v2:172.21.15.203:6832/830097474,v1:172.21.15.203:6833/830097474] conn(0x7f3a8406a050 msgr2=0x7f3a8406c510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:35:42.727 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:42.724+0000 7f3aaa147640 1 --2- 172.21.15.43:0/3536545824 >> [v2:172.21.15.203:6832/830097474,v1:172.21.15.203:6833/830097474] conn(0x7f3a8406a050 0x7f3a8406c510 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7f3a94006fd0 tx=0x7f3a94008040 comp rx=0 tx=0).stop 2023-08-09T13:35:42.727 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:42.724+0000 7f3aaa147640 1 -- 172.21.15.43:0/3536545824 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f3aa415c680 msgr2=0x7f3aa40cb9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:35:42.727 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:42.724+0000 7f3aaa147640 1 --2- 172.21.15.43:0/3536545824 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f3aa415c680 0x7f3aa40cb9c0 secure :-1 s=READY pgs=435 cs=0 l=1 rev1=1 crypto rx=0x7f3a9c036040 tx=0x7f3a9c004a40 comp rx=0 tx=0).stop 2023-08-09T13:35:42.727 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:42.724+0000 7f3aaa147640 1 -- 172.21.15.43:0/3536545824 shutdown_connections 2023-08-09T13:35:42.727 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:42.724+0000 7f3aaa147640 1 --2- 172.21.15.43:0/3536545824 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f3aa415c680 0x7f3aa40cb9c0 unknown :-1 s=CLOSED pgs=435 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:35:42.727 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:42.724+0000 7f3aaa147640 1 --2- 172.21.15.43:0/3536545824 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f3aa40d0a70 0x7f3aa415e2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:35:42.728 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:42.724+0000 7f3aaa147640 1 --2- 172.21.15.43:0/3536545824 >> [v2:172.21.15.203:6832/830097474,v1:172.21.15.203:6833/830097474] conn(0x7f3a8406a050 0x7f3a8406c510 unknown :-1 s=CLOSED pgs=218 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:35:42.728 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:42.724+0000 7f3aaa147640 1 --2- 172.21.15.43:0/3536545824 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f3aa40b3fa0 0x7f3aa41504a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:35:42.728 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:42.724+0000 7f3aaa147640 1 -- 172.21.15.43:0/3536545824 >> 172.21.15.43:0/3536545824 conn(0x7f3aa4071520 msgr2=0x7f3aa40ce040 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-09T13:35:42.728 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:42.724+0000 7f3aaa147640 1 -- 172.21.15.43:0/3536545824 shutdown_connections 2023-08-09T13:35:42.728 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:42.724+0000 7f3aaa147640 1 -- 172.21.15.43:0/3536545824 wait complete. 2023-08-09T13:35:42.746 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2023-08-09T13:35:42.975 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:42.972+0000 7fa84fc8e640 1 Processor -- start 2023-08-09T13:35:42.975 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:42.972+0000 7fa84fc8e640 1 -- start start 2023-08-09T13:35:42.976 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:42.974+0000 7fa84fc8e640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fa8480ccf20 0x7fa8480cd550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:35:42.977 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:42.974+0000 7fa84fc8e640 1 -- --> v1:172.21.15.43:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa8480cca90 con 0x7fa84815c630 2023-08-09T13:35:42.977 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:42.974+0000 7fa84fc8e640 1 -- --> v1:172.21.15.203:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa8480cbe80 con 0x7fa8480cda90 2023-08-09T13:35:42.977 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:42.974+0000 7fa84fc8e640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa8480ac600 con 0x7fa8480ccf20 2023-08-09T13:35:42.978 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:42.975+0000 7fa84da2a640 1 --1- >> v1:172.21.15.43:6789/0 conn(0x7fa84815c630 0x7fa8480abef0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.43:6789/0 says I am v1:172.21.15.43:44954/0 (socket says 172.21.15.43:44954) 2023-08-09T13:35:42.978 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:42.975+0000 7fa84da2a640 1 -- 172.21.15.43:0/233344693 learned_addr learned my addr 172.21.15.43:0/233344693 (peer_addr_for_me v1:172.21.15.43:0/0) 2023-08-09T13:35:42.978 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:42.975+0000 7fa84d229640 1 --2- 172.21.15.43:0/233344693 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fa8480ccf20 0x7fa8480cd550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:35:42.978 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:42.975+0000 7fa84ca28640 1 -- 172.21.15.43:0/233344693 <== mon.0 v1:172.21.15.43:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3313396357 0 0) 0x7fa8480cca90 con 0x7fa84815c630 2023-08-09T13:35:42.978 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:42.975+0000 7fa84ca28640 1 -- 172.21.15.43:0/233344693 --> v1:172.21.15.43:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fa834003520 con 0x7fa84815c630 2023-08-09T13:35:42.980 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:42.976+0000 7fa84ca28640 1 -- 172.21.15.43:0/233344693 <== mon.0 v1:172.21.15.43:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 4004981327 0 0) 0x7fa834003520 con 0x7fa84815c630 2023-08-09T13:35:42.980 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:42.976+0000 7fa84ca28640 1 -- 172.21.15.43:0/233344693 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fa8480ccf20 msgr2=0x7fa8480cd550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-09T13:35:42.980 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:42.976+0000 7fa84ca28640 1 --2- 172.21.15.43:0/233344693 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fa8480ccf20 0x7fa8480cd550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:35:42.980 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:42.976+0000 7fa84ca28640 1 -- 172.21.15.43:0/233344693 >> v1:172.21.15.203:6789/0 conn(0x7fa8480cda90 legacy=0x7fa84815ca10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:35:42.980 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:42.976+0000 7fa84ca28640 1 -- 172.21.15.43:0/233344693 --> v1:172.21.15.43:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa8480cc160 con 0x7fa84815c630 2023-08-09T13:35:42.981 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:42.976+0000 7fa84ca28640 1 -- 172.21.15.43:0/233344693 <== mon.0 v1:172.21.15.43:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1285150740 0 0) 0x7fa844001670 con 0x7fa84815c630 2023-08-09T13:35:42.981 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:42.976+0000 7fa84ca28640 1 -- 172.21.15.43:0/233344693 >> v1:172.21.15.43:6789/0 conn(0x7fa84815c630 legacy=0x7fa8480abef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:35:42.981 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:42.976+0000 7fa84ca28640 1 --2- 172.21.15.43:0/233344693 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fa834003a10 0x7fa834003df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:35:42.981 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:42.976+0000 7fa84ca28640 1 --2- 172.21.15.43:0/233344693 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa8340047c0 0x7fa834004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:35:42.981 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:42.976+0000 7fa84ca28640 1 --2- 172.21.15.43:0/233344693 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fa834005180 0x7fa83400b980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:35:42.981 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:42.976+0000 7fa84ca28640 1 -- 172.21.15.43:0/233344693 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa8480ac600 con 0x7fa8340047c0 2023-08-09T13:35:42.981 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:42.976+0000 7fa84ca28640 1 -- 172.21.15.43:0/233344693 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa83400bfa0 con 0x7fa834005180 2023-08-09T13:35:42.982 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:42.976+0000 7fa84ca28640 1 -- 172.21.15.43:0/233344693 --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa83400c160 con 0x7fa834003a10 2023-08-09T13:35:42.982 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:42.976+0000 7fa84d229640 1 --2- 172.21.15.43:0/233344693 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fa8480ccf20 0x7fa8480cd550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-08-09T13:35:42.982 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:42.976+0000 7fa84d229640 1 --2- 172.21.15.43:0/233344693 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa8340047c0 0x7fa834004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:35:42.982 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:42.977+0000 7fa84e22b640 1 --2- 172.21.15.43:0/233344693 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fa834005180 0x7fa83400b980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:35:42.982 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:42.977+0000 7fa84da2a640 1 --2- 172.21.15.43:0/233344693 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fa834003a10 0x7fa834003df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:35:42.982 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:42.977+0000 7fa84d229640 1 -- 172.21.15.43:0/233344693 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fa834003a10 msgr2=0x7fa834003df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-09T13:35:42.982 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:42.977+0000 7fa84d229640 1 --2- 172.21.15.43:0/233344693 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fa834003a10 0x7fa834003df0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:35:42.983 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:42.977+0000 7fa84d229640 1 -- 172.21.15.43:0/233344693 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fa834005180 msgr2=0x7fa83400b980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-09T13:35:42.983 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:42.977+0000 7fa84d229640 1 --2- 172.21.15.43:0/233344693 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fa834005180 0x7fa83400b980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:35:42.983 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:42.977+0000 7fa84d229640 1 -- 172.21.15.43:0/233344693 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa834003520 con 0x7fa8340047c0 2023-08-09T13:35:42.985 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:42.978+0000 7fa84d229640 1 --2- 172.21.15.43:0/233344693 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa8340047c0 0x7fa834004c40 secure :-1 s=READY pgs=437 cs=0 l=1 rev1=1 crypto rx=0x7fa84815dcf0 tx=0x7fa8340057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=adce32d841a1f9e8 server_cookie=0 in_seq=0 out_seq=0 2023-08-09T13:35:42.985 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:42.978+0000 7fa84ca28640 1 -- 172.21.15.43:0/233344693 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa838004030 con 0x7fa8340047c0 2023-08-09T13:35:42.985 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:42.978+0000 7fa84ca28640 1 -- 172.21.15.43:0/233344693 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa838004190 con 0x7fa8340047c0 2023-08-09T13:35:42.985 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:42.978+0000 7fa84ca28640 1 -- 172.21.15.43:0/233344693 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa838004520 con 0x7fa8340047c0 2023-08-09T13:35:42.985 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:42.979+0000 7fa84fc8e640 1 -- 172.21.15.43:0/233344693 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa8340047c0 msgr2=0x7fa834004c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:35:42.985 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:42.979+0000 7fa84fc8e640 1 --2- 172.21.15.43:0/233344693 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa8340047c0 0x7fa834004c40 secure :-1 s=READY pgs=437 cs=0 l=1 rev1=1 crypto rx=0x7fa84815dcf0 tx=0x7fa8340057f0 comp rx=0 tx=0).stop 2023-08-09T13:35:42.986 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:42.979+0000 7fa84e22b640 1 -- 172.21.15.43:0/233344693 reap_dead start 2023-08-09T13:35:42.986 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:42.979+0000 7fa84fc8e640 1 -- 172.21.15.43:0/233344693 shutdown_connections 2023-08-09T13:35:42.986 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:42.979+0000 7fa84fc8e640 1 -- 172.21.15.43:0/233344693 >> 172.21.15.43:0/233344693 conn(0x7fa8480ce0a0 msgr2=0x7fa8480ce480 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-09T13:35:42.986 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:42.979+0000 7fa84fc8e640 1 -- 172.21.15.43:0/233344693 shutdown_connections 2023-08-09T13:35:42.986 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:42.979+0000 7fa84fc8e640 1 -- 172.21.15.43:0/233344693 wait complete. 2023-08-09T13:35:42.986 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:42.981+0000 7fa84fc8e640 1 Processor -- start 2023-08-09T13:35:42.986 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:42.981+0000 7fa84fc8e640 1 -- start start 2023-08-09T13:35:42.987 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:42.982+0000 7fa84fc8e640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fa8340047c0 0x7fa8480c5b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:35:42.987 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:42.982+0000 7fa84fc8e640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa84815d690 0x7fa8480c9480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:35:42.987 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:42.982+0000 7fa84fc8e640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fa8481f1260 0x7fa8481f16e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:35:42.987 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:42.982+0000 7fa84fc8e640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa8480c60c0 con 0x7fa84815d690 2023-08-09T13:35:42.987 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:42.982+0000 7fa84fc8e640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa8481f5720 con 0x7fa8481f1260 2023-08-09T13:35:42.987 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:42.982+0000 7fa84fc8e640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa8481f5a00 con 0x7fa8340047c0 2023-08-09T13:35:42.987 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:42.982+0000 7fa84d229640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa84815d690 0x7fa8480c9480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:35:42.988 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:42.982+0000 7fa84d229640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa84815d690 0x7fa8480c9480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:50130/0 (socket says 172.21.15.43:50130) 2023-08-09T13:35:42.988 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:42.982+0000 7fa84d229640 1 -- 172.21.15.43:0/292741151 learned_addr learned my addr 172.21.15.43:0/292741151 (peer_addr_for_me v2:172.21.15.43:0/0) 2023-08-09T13:35:42.989 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:42.983+0000 7fa84d229640 1 -- 172.21.15.43:0/292741151 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fa8340047c0 msgr2=0x7fa8480c5b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:35:42.989 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:42.983+0000 7fa84d229640 1 --2- 172.21.15.43:0/292741151 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fa8340047c0 0x7fa8480c5b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:35:42.989 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:42.983+0000 7fa84d229640 1 -- 172.21.15.43:0/292741151 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fa8481f1260 msgr2=0x7fa8481f16e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:35:42.989 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:42.983+0000 7fa84d229640 1 --2- 172.21.15.43:0/292741151 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fa8481f1260 0x7fa8481f16e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:35:42.989 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:42.983+0000 7fa84d229640 1 -- 172.21.15.43:0/292741151 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa8481f5ce0 con 0x7fa84815d690 2023-08-09T13:35:42.990 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:42.983+0000 7fa84d229640 1 --2- 172.21.15.43:0/292741151 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa84815d690 0x7fa8480c9480 secure :-1 s=READY pgs=438 cs=0 l=1 rev1=1 crypto rx=0x7fa838009bf0 tx=0x7fa838004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-09T13:35:42.990 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:42.983+0000 7fa83e7fc640 1 -- 172.21.15.43:0/292741151 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa83802cb20 con 0x7fa84815d690 2023-08-09T13:35:42.990 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:42.983+0000 7fa83e7fc640 1 -- 172.21.15.43:0/292741151 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa838005e90 con 0x7fa84815d690 2023-08-09T13:35:42.990 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:42.984+0000 7fa83e7fc640 1 -- 172.21.15.43:0/292741151 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa83802ce80 con 0x7fa84815d690 2023-08-09T13:35:42.990 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:42.984+0000 7fa84fc8e640 1 -- 172.21.15.43:0/292741151 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa8481f5fc0 con 0x7fa84815d690 2023-08-09T13:35:42.990 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:42.984+0000 7fa84fc8e640 1 -- 172.21.15.43:0/292741151 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa8481f62a0 con 0x7fa84815d690 2023-08-09T13:35:42.991 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:42.985+0000 7fa84fc8e640 1 -- 172.21.15.43:0/292741151 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa8480739f0 con 0x7fa84815d690 2023-08-09T13:35:42.991 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:42.986+0000 7fa83e7fc640 1 -- 172.21.15.43:0/292741151 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 5) v1 ==== 88341+0+0 (secure 0 0 0) 0x7fa838003610 con 0x7fa84815d690 2023-08-09T13:35:42.991 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:42.987+0000 7fa83e7fc640 1 --2- 172.21.15.43:0/292741151 >> [v2:172.21.15.203:6832/830097474,v1:172.21.15.203:6833/830097474] conn(0x7fa82c06a050 0x7fa82c06c510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:35:42.992 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:42.987+0000 7fa83e7fc640 1 -- 172.21.15.43:0/292741151 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 6871+0+0 (secure 0 0 0) 0x7fa8380aadc0 con 0x7fa84815d690 2023-08-09T13:35:42.992 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:42.988+0000 7fa84da2a640 1 --2- 172.21.15.43:0/292741151 >> [v2:172.21.15.203:6832/830097474,v1:172.21.15.203:6833/830097474] conn(0x7fa82c06a050 0x7fa82c06c510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:35:42.992 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:42.989+0000 7fa84da2a640 1 --2- 172.21.15.43:0/292741151 >> [v2:172.21.15.203:6832/830097474,v1:172.21.15.203:6833/830097474] conn(0x7fa82c06a050 0x7fa82c06c510 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7fa844001200 tx=0x7fa844000ed0 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-09T13:35:42.994 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:42.992+0000 7fa83e7fc640 1 -- 172.21.15.43:0/292741151 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178066 (secure 0 0 0) 0x7fa838076200 con 0x7fa84815d690 2023-08-09T13:35:43.206 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:43.203+0000 7fa84fc8e640 1 -- 172.21.15.43:0/292741151 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7fa8480cc410 con 0x7fa84815d690 2023-08-09T13:35:43.207 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:43.204+0000 7fa83e7fc640 1 -- 172.21.15.43:0/292741151 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v26) v1 ==== 80+0+0 (secure 0 0 0) 0x7fa83804c080 con 0x7fa84815d690 2023-08-09T13:35:43.207 INFO:teuthology.orchestra.run.smithi043.stderr:listed 0 entries 2023-08-09T13:35:43.210 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:43.207+0000 7fa84fc8e640 1 -- 172.21.15.43:0/292741151 >> [v2:172.21.15.203:6832/830097474,v1:172.21.15.203:6833/830097474] conn(0x7fa82c06a050 msgr2=0x7fa82c06c510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:35:43.210 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:43.207+0000 7fa84fc8e640 1 --2- 172.21.15.43:0/292741151 >> [v2:172.21.15.203:6832/830097474,v1:172.21.15.203:6833/830097474] conn(0x7fa82c06a050 0x7fa82c06c510 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7fa844001200 tx=0x7fa844000ed0 comp rx=0 tx=0).stop 2023-08-09T13:35:43.210 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:43.207+0000 7fa84fc8e640 1 -- 172.21.15.43:0/292741151 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa84815d690 msgr2=0x7fa8480c9480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:35:43.210 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:43.207+0000 7fa84fc8e640 1 --2- 172.21.15.43:0/292741151 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa84815d690 0x7fa8480c9480 secure :-1 s=READY pgs=438 cs=0 l=1 rev1=1 crypto rx=0x7fa838009bf0 tx=0x7fa838004730 comp rx=0 tx=0).stop 2023-08-09T13:35:43.210 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:43.207+0000 7fa84fc8e640 1 -- 172.21.15.43:0/292741151 shutdown_connections 2023-08-09T13:35:43.210 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:43.207+0000 7fa84fc8e640 1 --2- 172.21.15.43:0/292741151 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fa8481f1260 0x7fa8481f16e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:35:43.210 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:43.207+0000 7fa84fc8e640 1 --2- 172.21.15.43:0/292741151 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa84815d690 0x7fa8480c9480 unknown :-1 s=CLOSED pgs=438 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:35:43.211 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:43.207+0000 7fa84fc8e640 1 --2- 172.21.15.43:0/292741151 >> [v2:172.21.15.203:6832/830097474,v1:172.21.15.203:6833/830097474] conn(0x7fa82c06a050 0x7fa82c06c510 unknown :-1 s=CLOSED pgs=219 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:35:43.211 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:43.207+0000 7fa84fc8e640 1 --2- 172.21.15.43:0/292741151 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fa8340047c0 0x7fa8480c5b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:35:43.211 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:43.207+0000 7fa84fc8e640 1 -- 172.21.15.43:0/292741151 >> 172.21.15.43:0/292741151 conn(0x7fa8480ce0a0 msgr2=0x7fa8481605e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-09T13:35:43.211 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:43.207+0000 7fa84fc8e640 1 -- 172.21.15.43:0/292741151 shutdown_connections 2023-08-09T13:35:43.212 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:43.207+0000 7fa84fc8e640 1 -- 172.21.15.43:0/292741151 wait complete. 2023-08-09T13:35:43.229 INFO:tasks.cephfs.fuse_mount:Running fusermount -u on ubuntu@smithi043.front.sepia.ceph.com... 2023-08-09T13:35:43.229 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-09T13:35:43.229 DEBUG:teuthology.orchestra.run.smithi043:> sudo fusermount -u /home/ubuntu/cephtest/mnt.2 2023-08-09T13:35:43.330 INFO:teuthology.orchestra.run:waiting for 300 2023-08-09T13:35:43.331 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi043.front.sepia.ceph.com 2023-08-09T13:35:43.332 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-09T13:35:43.332 DEBUG:teuthology.orchestra.run.smithi043:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.2) 2023-08-09T13:35:43.387 DEBUG:teuthology.orchestra.run.smithi203:> set -ex 2023-08-09T13:35:43.388 DEBUG:teuthology.orchestra.run.smithi203:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-09T13:35:43.449 DEBUG:teuthology.orchestra.run.smithi203:> set -ex 2023-08-09T13:35:43.450 DEBUG:teuthology.orchestra.run.smithi203:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-09T13:35:43.503 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-08-09T13:35:43.741 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:43.739+0000 7f32750da640 1 Processor -- start 2023-08-09T13:35:43.742 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:43.739+0000 7f32750da640 1 -- start start 2023-08-09T13:35:43.743 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:43.741+0000 7f32750da640 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7f32700b2b70 0x7f32700d2ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:35:43.743 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:43.741+0000 7f32750da640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f32700b3050 0x7f32700b3f60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:35:43.744 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:43.741+0000 7f32750da640 1 -- --> v1:172.21.15.203:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f32700ce090 con 0x7f32700b2790 2023-08-09T13:35:43.744 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:43.741+0000 7f32750da640 1 -- --> v2:172.21.15.203:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f32700b3950 con 0x7f32700b2b70 2023-08-09T13:35:43.744 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:43.741+0000 7f32750da640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f32700cd7f0 con 0x7f32700b3050 2023-08-09T13:35:43.745 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:43.742+0000 7f326f59e640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f32700b3050 0x7f32700b3f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:35:43.745 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:43.742+0000 7f326f59e640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f32700b3050 0x7f32700b3f60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.203:3301/0 says I am v2:172.21.15.43:47844/0 (socket says 172.21.15.43:47844) 2023-08-09T13:35:43.745 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:43.742+0000 7f326f59e640 1 -- 172.21.15.43:0/1874781069 learned_addr learned my addr 172.21.15.43:0/1874781069 (peer_addr_for_me v2:172.21.15.43:0/0) 2023-08-09T13:35:43.745 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:43.742+0000 7f326e59c640 1 --2- 172.21.15.43:0/1874781069 >> v2:172.21.15.203:3300/0 conn(0x7f32700b2b70 0x7f32700d2ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:35:43.745 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:43.742+0000 7f326dd9b640 1 -- 172.21.15.43:0/1874781069 <== mon.1 v1:172.21.15.203:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1051258154 0 0) 0x7f32700ce090 con 0x7f32700b2790 2023-08-09T13:35:43.745 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:43.742+0000 7f326dd9b640 1 -- 172.21.15.43:0/1874781069 --> v1:172.21.15.203:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f3258003520 con 0x7f32700b2790 2023-08-09T13:35:43.746 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:43.743+0000 7f326f59e640 1 -- 172.21.15.43:0/1874781069 >> v2:172.21.15.203:3300/0 conn(0x7f32700b2b70 msgr2=0x7f32700d2ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-09T13:35:43.746 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:43.743+0000 7f326f59e640 1 --2- 172.21.15.43:0/1874781069 >> v2:172.21.15.203:3300/0 conn(0x7f32700b2b70 0x7f32700d2ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:35:43.746 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:43.743+0000 7f326f59e640 1 -- 172.21.15.43:0/1874781069 >> v1:172.21.15.203:6789/0 conn(0x7f32700b2790 legacy=0x7f32700b5330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:35:43.746 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:43.743+0000 7f326f59e640 1 -- 172.21.15.43:0/1874781069 --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f32700cdad0 con 0x7f32700b3050 2023-08-09T13:35:43.747 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:43.744+0000 7f326f59e640 1 --2- 172.21.15.43:0/1874781069 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f32700b3050 0x7f32700b3f60 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f3268009e50 tx=0x7f326802bf60 comp rx=0 tx=0).ready entity=mon.2 client_cookie=62569e5ec015cfee server_cookie=0 in_seq=0 out_seq=0 2023-08-09T13:35:43.747 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:43.744+0000 7f326dd9b640 1 -- 172.21.15.43:0/1874781069 <== mon.2 v2:172.21.15.203:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f326802cb00 con 0x7f32700b3050 2023-08-09T13:35:43.747 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:43.745+0000 7f326dd9b640 1 -- 172.21.15.43:0/1874781069 <== mon.2 v2:172.21.15.203:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f326802cc60 con 0x7f32700b3050 2023-08-09T13:35:43.747 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:43.745+0000 7f326dd9b640 1 -- 172.21.15.43:0/1874781069 <== mon.2 v2:172.21.15.203:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f32680343f0 con 0x7f32700b3050 2023-08-09T13:35:43.748 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:43.745+0000 7f32750da640 1 -- 172.21.15.43:0/1874781069 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f32700b3050 msgr2=0x7f32700b3f60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:35:43.748 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:43.745+0000 7f32750da640 1 --2- 172.21.15.43:0/1874781069 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f32700b3050 0x7f32700b3f60 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f3268009e50 tx=0x7f326802bf60 comp rx=0 tx=0).stop 2023-08-09T13:35:43.748 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:43.745+0000 7f32750da640 1 -- 172.21.15.43:0/1874781069 shutdown_connections 2023-08-09T13:35:43.749 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:43.745+0000 7f32750da640 1 --2- 172.21.15.43:0/1874781069 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f32700b3050 0x7f32700b3f60 unknown :-1 s=CLOSED pgs=69 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:35:43.749 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:43.745+0000 7f32750da640 1 --2- 172.21.15.43:0/1874781069 >> v2:172.21.15.203:3300/0 conn(0x7f32700b2b70 0x7f32700d2ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:35:43.749 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:43.745+0000 7f32750da640 1 -- 172.21.15.43:0/1874781069 >> 172.21.15.43:0/1874781069 conn(0x7f327009b7b0 msgr2=0x7f32700d24e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-09T13:35:43.749 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:43.745+0000 7f32750da640 1 -- 172.21.15.43:0/1874781069 shutdown_connections 2023-08-09T13:35:43.749 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:43.745+0000 7f32750da640 1 -- 172.21.15.43:0/1874781069 wait complete. 2023-08-09T13:35:43.749 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:43.747+0000 7f32750da640 1 Processor -- start 2023-08-09T13:35:43.750 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:43.747+0000 7f32750da640 1 -- start start 2023-08-09T13:35:43.751 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:43.748+0000 7f32750da640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f32700b2790 0x7f3270150960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:35:43.751 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:43.748+0000 7f32750da640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f32700b2b70 0x7f32700c8b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:35:43.751 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:43.748+0000 7f32750da640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f32700b3050 0x7f327006ae40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:35:43.751 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:43.748+0000 7f32750da640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f327015bf90 con 0x7f32700b2b70 2023-08-09T13:35:43.751 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:43.748+0000 7f32750da640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f32700c9650 con 0x7f32700b2790 2023-08-09T13:35:43.752 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:43.748+0000 7f32750da640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f32700c9930 con 0x7f32700b3050 2023-08-09T13:35:43.752 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:43.749+0000 7f326e59c640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f32700b2b70 0x7f32700c8b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:35:43.752 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:43.749+0000 7f326e59c640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f32700b2b70 0x7f32700c8b80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:50140/0 (socket says 172.21.15.43:50140) 2023-08-09T13:35:43.752 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:43.749+0000 7f326e59c640 1 -- 172.21.15.43:0/377276757 learned_addr learned my addr 172.21.15.43:0/377276757 (peer_addr_for_me v2:172.21.15.43:0/0) 2023-08-09T13:35:43.753 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:43.749+0000 7f326f59e640 1 --2- 172.21.15.43:0/377276757 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f32700b3050 0x7f327006ae40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:35:43.753 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:43.749+0000 7f326ed9d640 1 --2- 172.21.15.43:0/377276757 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f32700b2790 0x7f3270150960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:35:43.753 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:43.749+0000 7f326e59c640 1 -- 172.21.15.43:0/377276757 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f32700b3050 msgr2=0x7f327006ae40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:35:43.753 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:43.749+0000 7f326e59c640 1 --2- 172.21.15.43:0/377276757 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f32700b3050 0x7f327006ae40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:35:43.754 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:43.749+0000 7f326e59c640 1 -- 172.21.15.43:0/377276757 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f32700b2790 msgr2=0x7f3270150960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:35:43.754 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:43.749+0000 7f326e59c640 1 --2- 172.21.15.43:0/377276757 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f32700b2790 0x7f3270150960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:35:43.754 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:43.749+0000 7f326e59c640 1 -- 172.21.15.43:0/377276757 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3268009b30 con 0x7f32700b2b70 2023-08-09T13:35:43.754 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:43.749+0000 7f326ed9d640 1 --2- 172.21.15.43:0/377276757 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f32700b2790 0x7f3270150960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-08-09T13:35:43.754 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:43.749+0000 7f326e59c640 1 --2- 172.21.15.43:0/377276757 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f32700b2b70 0x7f32700c8b80 secure :-1 s=READY pgs=439 cs=0 l=1 rev1=1 crypto rx=0x7f326400e870 tx=0x7f326400ed30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-09T13:35:43.754 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:43.750+0000 7f32577fe640 1 -- 172.21.15.43:0/377276757 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f32640094f0 con 0x7f32700b2b70 2023-08-09T13:35:43.754 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:43.750+0000 7f32577fe640 1 -- 172.21.15.43:0/377276757 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f326400c940 con 0x7f32700b2b70 2023-08-09T13:35:43.754 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:43.750+0000 7f32577fe640 1 -- 172.21.15.43:0/377276757 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f326400ccf0 con 0x7f32700b2b70 2023-08-09T13:35:43.755 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:43.750+0000 7f32750da640 1 -- 172.21.15.43:0/377276757 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f32700c9c10 con 0x7f32700b2b70 2023-08-09T13:35:43.755 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:43.750+0000 7f32750da640 1 -- 172.21.15.43:0/377276757 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f32700ca1d0 con 0x7f32700b2b70 2023-08-09T13:35:43.756 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:43.751+0000 7f32750da640 1 -- 172.21.15.43:0/377276757 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f32700739f0 con 0x7f32700b2b70 2023-08-09T13:35:43.756 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:43.752+0000 7f32577fe640 1 -- 172.21.15.43:0/377276757 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 5) v1 ==== 88341+0+0 (secure 0 0 0) 0x7f3264013470 con 0x7f32700b2b70 2023-08-09T13:35:43.756 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:43.753+0000 7f32577fe640 1 --2- 172.21.15.43:0/377276757 >> [v2:172.21.15.203:6832/830097474,v1:172.21.15.203:6833/830097474] conn(0x7f325c06a080 0x7f325c06c540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:35:43.756 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:43.753+0000 7f32577fe640 1 -- 172.21.15.43:0/377276757 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 6871+0+0 (secure 0 0 0) 0x7f3264088aa0 con 0x7f32700b2b70 2023-08-09T13:35:43.757 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:43.754+0000 7f326ed9d640 1 --2- 172.21.15.43:0/377276757 >> [v2:172.21.15.203:6832/830097474,v1:172.21.15.203:6833/830097474] conn(0x7f325c06a080 0x7f325c06c540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:35:43.757 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:43.755+0000 7f326ed9d640 1 --2- 172.21.15.43:0/377276757 >> [v2:172.21.15.203:6832/830097474,v1:172.21.15.203:6833/830097474] conn(0x7f325c06a080 0x7f325c06c540 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7f32600011e0 tx=0x7f3260009040 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-09T13:35:43.759 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:43.757+0000 7f32577fe640 1 -- 172.21.15.43:0/377276757 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178066 (secure 0 0 0) 0x7f3264053d30 con 0x7f32700b2b70 2023-08-09T13:35:43.953 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:43.950+0000 7f32750da640 1 -- 172.21.15.43:0/377276757 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f32700b4b00 con 0x7f32700b2b70 2023-08-09T13:35:43.954 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:43.952+0000 7f32577fe640 1 -- 172.21.15.43:0/377276757 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v26) v1 ==== 74+0+20146 (secure 0 0 0) 0x7f3264021090 con 0x7f32700b2b70 2023-08-09T13:35:43.955 INFO:teuthology.orchestra.run.smithi043.stdout: 2023-08-09T13:35:43.955 INFO:teuthology.orchestra.run.smithi043.stdout:{"epoch":26,"fsid":"daac87ee-1337-4250-a955-aea4e408a39b","created":"2023-08-09T13:33:41.701198+0000","modified":"2023-08-09T13:34:48.711911+0000","last_up_change":"2023-08-09T13:33:53.783453+0000","last_in_change":"2023-08-09T13:33:45.148904+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":5,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":"rbd","create_time":"2023-08-09T13:33:54.365137+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":".mgr","create_time":"2023-08-09T13:33:54.831803+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_metadata","create_time":"2023-08-09T13:34:00.207974+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data","create_time":"2023-08-09T13:34:01.614765+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":5,"pool_name":"cephfs_data_ec","create_time":"2023-08-09T13:34:05.239583+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"24","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"cbd49ecd-e896-4fbd-864e-5c37a829b165","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6800","nonce":502667335},{"type":"v1","addr":"172.21.15.43:6801","nonce":502667335}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6802","nonce":502667335},{"type":"v1","addr":"172.21.15.43:6803","nonce":502667335}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6806","nonce":502667335},{"type":"v1","addr":"172.21.15.43:6807","nonce":502667335}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6804","nonce":502667335},{"type":"v1","addr":"172.21.15.43:6805","nonce":502667335}]},"public_addr":"172.21.15.43:6801/502667335","cluster_addr":"172.21.15.43:6803/502667335","heartbeat_back_addr":"172.21.15.43:6807/502667335","heartbeat_front_addr":"172.21.15.43:6805/502667335","state":["exists","up"]},{"osd":1,"uuid":"75c7e0fe-1ec7-4fe8-a2f2-3661a65b8500","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6808","nonce":3289134610},{"type":"v1","addr":"172.21.15.43:6809","nonce":3289134610}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6810","nonce":3289134610},{"type":"v1","addr":"172.21.15.43:6812","nonce":3289134610}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6821","nonce":3289134610},{"type":"v1","addr":"172.21.15.43:6824","nonce":3289134610}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6815","nonce":3289134610},{"type":"v1","addr":"172.21.15.43:6818","nonce":3289134610}]},"public_addr":"172.21.15.43:6809/3289134610","cluster_addr":"172.21.15.43:6812/3289134610","heartbeat_back_addr":"172.21.15.43:6824/3289134610","heartbeat_front_addr":"172.21.15.43:6818/3289134610","state":["exists","up"]},{"osd":2,"uuid":"bb5c41e9-7f5f-4125-9b01-3bb8bfc2ea83","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6811","nonce":4079709736},{"type":"v1","addr":"172.21.15.43:6814","nonce":4079709736}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6817","nonce":4079709736},{"type":"v1","addr":"172.21.15.43:6820","nonce":4079709736}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6829","nonce":4079709736},{"type":"v1","addr":"172.21.15.43:6831","nonce":4079709736}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6823","nonce":4079709736},{"type":"v1","addr":"172.21.15.43:6826","nonce":4079709736}]},"public_addr":"172.21.15.43:6814/4079709736","cluster_addr":"172.21.15.43:6820/4079709736","heartbeat_back_addr":"172.21.15.43:6831/4079709736","heartbeat_front_addr":"172.21.15.43:6826/4079709736","state":["exists","up"]},{"osd":3,"uuid":"bfe4c8b4-694a-465c-87e6-2b2aab852613","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6813","nonce":1709538759},{"type":"v1","addr":"172.21.15.43:6816","nonce":1709538759}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6819","nonce":1709538759},{"type":"v1","addr":"172.21.15.43:6822","nonce":1709538759}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6828","nonce":1709538759},{"type":"v1","addr":"172.21.15.43:6830","nonce":1709538759}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6825","nonce":1709538759},{"type":"v1","addr":"172.21.15.43:6827","nonce":1709538759}]},"public_addr":"172.21.15.43:6816/1709538759","cluster_addr":"172.21.15.43:6822/1709538759","heartbeat_back_addr":"172.21.15.43:6830/1709538759","heartbeat_front_addr":"172.21.15.43:6827/1709538759","state":["exists","up"]},{"osd":4,"uuid":"fa2e8a77-aba1-4756-853e-0d174a5bdeba","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6800","nonce":4196866481},{"type":"v1","addr":"172.21.15.203:6801","nonce":4196866481}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6802","nonce":4196866481},{"type":"v1","addr":"172.21.15.203:6803","nonce":4196866481}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6806","nonce":4196866481},{"type":"v1","addr":"172.21.15.203:6807","nonce":4196866481}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6804","nonce":4196866481},{"type":"v1","addr":"172.21.15.203:6805","nonce":4196866481}]},"public_addr":"172.21.15.203:6801/4196866481","cluster_addr":"172.21.15.203:6803/4196866481","heartbeat_back_addr":"172.21.15.203:6807/4196866481","heartbeat_front_addr":"172.21.15.203:6805/4196866481","state":["exists","up"]},{"osd":5,"uuid":"81811e4b-4d24-4f51-a82c-f1d700a2a00d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6816","nonce":3059290081},{"type":"v1","addr":"172.21.15.203:6817","nonce":3059290081}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6818","nonce":3059290081},{"type":"v1","addr":"172.21.15.203:6819","nonce":3059290081}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6822","nonce":3059290081},{"type":"v1","addr":"172.21.15.203:6823","nonce":3059290081}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6820","nonce":3059290081},{"type":"v1","addr":"172.21.15.203:6821","nonce":3059290081}]},"public_addr":"172.21.15.203:6817/3059290081","cluster_addr":"172.21.15.203:6819/3059290081","heartbeat_back_addr":"172.21.15.203:6823/3059290081","heartbeat_front_addr":"172.21.15.203:6821/3059290081","state":["exists","up"]},{"osd":6,"uuid":"307bac4d-6a5b-443d-af75-56d695d7674f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6824","nonce":512155569},{"type":"v1","addr":"172.21.15.203:6825","nonce":512155569}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6826","nonce":512155569},{"type":"v1","addr":"172.21.15.203:6827","nonce":512155569}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6830","nonce":512155569},{"type":"v1","addr":"172.21.15.203:6831","nonce":512155569}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6828","nonce":512155569},{"type":"v1","addr":"172.21.15.203:6829","nonce":512155569}]},"public_addr":"172.21.15.203:6825/512155569","cluster_addr":"172.21.15.203:6827/512155569","heartbeat_back_addr":"172.21.15.203:6831/512155569","heartbeat_front_addr":"172.21.15.203:6829/512155569","state":["exists","up"]},{"osd":7,"uuid":"a95dc7ca-9f1b-4295-af70-68a22a2d7d5e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6808","nonce":3331450438},{"type":"v1","addr":"172.21.15.203:6809","nonce":3331450438}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6810","nonce":3331450438},{"type":"v1","addr":"172.21.15.203:6811","nonce":3331450438}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6814","nonce":3331450438},{"type":"v1","addr":"172.21.15.203:6815","nonce":3331450438}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6812","nonce":3331450438},{"type":"v1","addr":"172.21.15.203:6813","nonce":3331450438}]},"public_addr":"172.21.15.203:6809/3331450438","cluster_addr":"172.21.15.203:6811/3331450438","heartbeat_back_addr":"172.21.15.203:6815/3331450438","heartbeat_front_addr":"172.21.15.203:6813/3331450438","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-09T13:33:50.547652+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-09T13:33:50.589469+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-09T13:33:50.561724+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-09T13:33:50.599650+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-09T13:33:50.411855+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-09T13:33:50.219681+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-09T13:33:50.504664+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-09T13:33:50.441248+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"4.14","mappings":[{"from":1,"to":5}]},{"pgid":"4.29","mappings":[{"from":3,"to":2}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-08-09T13:35:43.957 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:43.955+0000 7f32750da640 1 -- 172.21.15.43:0/377276757 >> [v2:172.21.15.203:6832/830097474,v1:172.21.15.203:6833/830097474] conn(0x7f325c06a080 msgr2=0x7f325c06c540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:35:43.958 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:43.955+0000 7f32750da640 1 --2- 172.21.15.43:0/377276757 >> [v2:172.21.15.203:6832/830097474,v1:172.21.15.203:6833/830097474] conn(0x7f325c06a080 0x7f325c06c540 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7f32600011e0 tx=0x7f3260009040 comp rx=0 tx=0).stop 2023-08-09T13:35:43.958 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:43.955+0000 7f32750da640 1 -- 172.21.15.43:0/377276757 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f32700b2b70 msgr2=0x7f32700c8b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:35:43.958 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:43.955+0000 7f32750da640 1 --2- 172.21.15.43:0/377276757 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f32700b2b70 0x7f32700c8b80 secure :-1 s=READY pgs=439 cs=0 l=1 rev1=1 crypto rx=0x7f326400e870 tx=0x7f326400ed30 comp rx=0 tx=0).stop 2023-08-09T13:35:43.958 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:43.955+0000 7f32750da640 1 -- 172.21.15.43:0/377276757 shutdown_connections 2023-08-09T13:35:43.958 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:43.955+0000 7f32750da640 1 --2- 172.21.15.43:0/377276757 >> [v2:172.21.15.203:6832/830097474,v1:172.21.15.203:6833/830097474] conn(0x7f325c06a080 0x7f325c06c540 unknown :-1 s=CLOSED pgs=220 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:35:43.958 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:43.955+0000 7f32750da640 1 --2- 172.21.15.43:0/377276757 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f32700b3050 0x7f327006ae40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:35:43.958 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:43.955+0000 7f32750da640 1 --2- 172.21.15.43:0/377276757 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f32700b2b70 0x7f32700c8b80 unknown :-1 s=CLOSED pgs=439 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:35:43.959 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:43.955+0000 7f32750da640 1 --2- 172.21.15.43:0/377276757 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f32700b2790 0x7f3270150960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:35:43.959 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:43.955+0000 7f32750da640 1 -- 172.21.15.43:0/377276757 >> 172.21.15.43:0/377276757 conn(0x7f327009b7b0 msgr2=0x7f32700ce500 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-09T13:35:43.959 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:43.955+0000 7f32750da640 1 -- 172.21.15.43:0/377276757 shutdown_connections 2023-08-09T13:35:43.959 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:43.955+0000 7f32750da640 1 -- 172.21.15.43:0/377276757 wait complete. 2023-08-09T13:35:43.977 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': 'rbd', 'create_time': '2023-08-09T13:33:54.365137+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}, 'read_balance': {'score_acting': 2, 'score_stable': 2, 'optimal_score': 1, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': '.mgr', 'create_time': '2023-08-09T13:33:54.831803+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_metadata', 'create_time': '2023-08-09T13:34:00.207974+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data', 'create_time': '2023-08-09T13:34:01.614765+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 2, 'score_stable': 2, 'optimal_score': 1, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 5, 'pool_name': 'cephfs_data_ec', 'create_time': '2023-08-09T13:34:05.239583+0000', 'flags': 5, 'flags_names': 'hashpspool,ec_overwrites', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '24', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2023-08-09T13:35:43.978 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get rbd pg_num 2023-08-09T13:35:44.208 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:44.205+0000 7f59baecc640 1 Processor -- start 2023-08-09T13:35:44.208 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:44.205+0000 7f59baecc640 1 -- start start 2023-08-09T13:35:44.213 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:44.207+0000 7f59baecc640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f59b40cb2d0 0x7f59b415b380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:35:44.213 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:44.207+0000 7f59baecc640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f59b415df60 0x7f59b40ce1c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:35:44.213 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:44.207+0000 7f59baecc640 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7f59b415e3c0 0x7f59b40ce700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:35:44.213 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:44.207+0000 7f59baecc640 1 -- --> v2:172.21.15.43:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f59b415a970 con 0x7f59b40cb2d0 2023-08-09T13:35:44.213 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:44.207+0000 7f59baecc640 1 -- --> v2:172.21.15.203:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f59b415aef0 con 0x7f59b415e3c0 2023-08-09T13:35:44.213 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:44.207+0000 7f59baecc640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f59b415b8c0 con 0x7f59b415df60 2023-08-09T13:35:44.214 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:44.207+0000 7f59b8c68640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f59b40cb2d0 0x7f59b415b380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:35:44.214 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:44.207+0000 7f59b8c68640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f59b40cb2d0 0x7f59b415b380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:50150/0 (socket says 172.21.15.43:50150) 2023-08-09T13:35:44.214 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:44.207+0000 7f59b8c68640 1 -- 172.21.15.43:0/2382739432 learned_addr learned my addr 172.21.15.43:0/2382739432 (peer_addr_for_me v2:172.21.15.43:0/0) 2023-08-09T13:35:44.214 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:44.208+0000 7f59b3fff640 1 --2- 172.21.15.43:0/2382739432 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f59b415df60 0x7f59b40ce1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:35:44.214 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:44.208+0000 7f59b8c68640 1 -- 172.21.15.43:0/2382739432 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f59b415df60 msgr2=0x7f59b40ce1c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-09T13:35:44.214 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:44.208+0000 7f59b9469640 1 --2- 172.21.15.43:0/2382739432 >> v2:172.21.15.203:3300/0 conn(0x7f59b415e3c0 0x7f59b40ce700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:35:44.214 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:44.208+0000 7f59b8c68640 1 --2- 172.21.15.43:0/2382739432 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f59b415df60 0x7f59b40ce1c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:35:44.214 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:44.208+0000 7f59b8c68640 1 -- 172.21.15.43:0/2382739432 >> v2:172.21.15.203:3300/0 conn(0x7f59b415e3c0 msgr2=0x7f59b40ce700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-09T13:35:44.215 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:44.208+0000 7f59b8c68640 1 --2- 172.21.15.43:0/2382739432 >> v2:172.21.15.203:3300/0 conn(0x7f59b415e3c0 0x7f59b40ce700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:35:44.215 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:44.208+0000 7f59b8c68640 1 -- 172.21.15.43:0/2382739432 --> v2:172.21.15.43:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f59b415e940 con 0x7f59b40cb2d0 2023-08-09T13:35:44.215 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:44.208+0000 7f59b3fff640 1 --2- 172.21.15.43:0/2382739432 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f59b415df60 0x7f59b40ce1c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-08-09T13:35:44.215 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:44.209+0000 7f59b8c68640 1 --2- 172.21.15.43:0/2382739432 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f59b40cb2d0 0x7f59b415b380 secure :-1 s=READY pgs=440 cs=0 l=1 rev1=1 crypto rx=0x7f59b4069aa0 tx=0x7f59a802b920 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c58fb84326d8e781 server_cookie=0 in_seq=0 out_seq=0 2023-08-09T13:35:44.215 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:44.210+0000 7f59b37fe640 1 -- 172.21.15.43:0/2382739432 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f59a8031880 con 0x7f59b40cb2d0 2023-08-09T13:35:44.215 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:44.210+0000 7f59b37fe640 1 -- 172.21.15.43:0/2382739432 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f59a80319e0 con 0x7f59b40cb2d0 2023-08-09T13:35:44.215 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:44.210+0000 7f59b37fe640 1 -- 172.21.15.43:0/2382739432 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f59a8031d70 con 0x7f59b40cb2d0 2023-08-09T13:35:44.216 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:44.210+0000 7f59baecc640 1 -- 172.21.15.43:0/2382739432 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f59b40cb2d0 msgr2=0x7f59b415b380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:35:44.216 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:44.210+0000 7f59baecc640 1 --2- 172.21.15.43:0/2382739432 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f59b40cb2d0 0x7f59b415b380 secure :-1 s=READY pgs=440 cs=0 l=1 rev1=1 crypto rx=0x7f59b4069aa0 tx=0x7f59a802b920 comp rx=0 tx=0).stop 2023-08-09T13:35:44.216 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:44.211+0000 7f59baecc640 1 -- 172.21.15.43:0/2382739432 shutdown_connections 2023-08-09T13:35:44.217 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:44.211+0000 7f59baecc640 1 --2- 172.21.15.43:0/2382739432 >> v2:172.21.15.203:3300/0 conn(0x7f59b415e3c0 0x7f59b40ce700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:35:44.217 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:44.211+0000 7f59baecc640 1 --2- 172.21.15.43:0/2382739432 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f59b415df60 0x7f59b40ce1c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:35:44.217 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:44.211+0000 7f59baecc640 1 --2- 172.21.15.43:0/2382739432 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f59b40cb2d0 0x7f59b415b380 unknown :-1 s=CLOSED pgs=440 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:35:44.217 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:44.211+0000 7f59baecc640 1 -- 172.21.15.43:0/2382739432 >> 172.21.15.43:0/2382739432 conn(0x7f59b40aff50 msgr2=0x7f59b40b0360 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-09T13:35:44.217 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:44.211+0000 7f59baecc640 1 -- 172.21.15.43:0/2382739432 shutdown_connections 2023-08-09T13:35:44.217 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:44.211+0000 7f59baecc640 1 -- 172.21.15.43:0/2382739432 wait complete. 2023-08-09T13:35:44.217 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:44.213+0000 7f59baecc640 1 Processor -- start 2023-08-09T13:35:44.218 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:44.213+0000 7f59baecc640 1 -- start start 2023-08-09T13:35:44.219 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:44.215+0000 7f59baecc640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f59b40cb2d0 0x7f59b40c8290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:35:44.219 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:44.215+0000 7f59baecc640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f59b415df60 0x7f59b415bb30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:35:44.219 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:44.215+0000 7f59baecc640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f59b415e3c0 0x7f59b415c070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:35:44.219 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:44.215+0000 7f59baecc640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f59b415c780 con 0x7f59b415df60 2023-08-09T13:35:44.219 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:44.215+0000 7f59baecc640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f59b415ca60 con 0x7f59b40cb2d0 2023-08-09T13:35:44.219 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:44.215+0000 7f59baecc640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f59b415cd40 con 0x7f59b415e3c0 2023-08-09T13:35:44.220 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:44.215+0000 7f59b3fff640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f59b415df60 0x7f59b415bb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:35:44.220 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:44.215+0000 7f59b3fff640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f59b415df60 0x7f59b415bb30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:50160/0 (socket says 172.21.15.43:50160) 2023-08-09T13:35:44.220 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:44.215+0000 7f59b3fff640 1 -- 172.21.15.43:0/4104940599 learned_addr learned my addr 172.21.15.43:0/4104940599 (peer_addr_for_me v2:172.21.15.43:0/0) 2023-08-09T13:35:44.220 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:44.216+0000 7f59b3fff640 1 -- 172.21.15.43:0/4104940599 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f59b415e3c0 msgr2=0x7f59b415c070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:35:44.220 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:44.216+0000 7f59b3fff640 1 --2- 172.21.15.43:0/4104940599 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f59b415e3c0 0x7f59b415c070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:35:44.220 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:44.216+0000 7f59b3fff640 1 -- 172.21.15.43:0/4104940599 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f59b40cb2d0 msgr2=0x7f59b40c8290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:35:44.221 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:44.216+0000 7f59b3fff640 1 --2- 172.21.15.43:0/4104940599 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f59b40cb2d0 0x7f59b40c8290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:35:44.221 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:44.216+0000 7f59b3fff640 1 -- 172.21.15.43:0/4104940599 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f59a80094d0 con 0x7f59b415df60 2023-08-09T13:35:44.221 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:44.216+0000 7f59b3fff640 1 --2- 172.21.15.43:0/4104940599 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f59b415df60 0x7f59b415bb30 secure :-1 s=READY pgs=441 cs=0 l=1 rev1=1 crypto rx=0x7f59a000c7c0 tx=0x7f59a000cc80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-09T13:35:44.221 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:44.216+0000 7f59b17fa640 1 -- 172.21.15.43:0/4104940599 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f59a0007a30 con 0x7f59b415df60 2023-08-09T13:35:44.221 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:44.216+0000 7f59b17fa640 1 -- 172.21.15.43:0/4104940599 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f59a0007b90 con 0x7f59b415df60 2023-08-09T13:35:44.221 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:44.216+0000 7f59b17fa640 1 -- 172.21.15.43:0/4104940599 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f59a000ce70 con 0x7f59b415df60 2023-08-09T13:35:44.221 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:44.216+0000 7f59baecc640 1 -- 172.21.15.43:0/4104940599 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f59b415d020 con 0x7f59b415df60 2023-08-09T13:35:44.221 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:44.216+0000 7f59baecc640 1 -- 172.21.15.43:0/4104940599 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f59b40c87d0 con 0x7f59b415df60 2023-08-09T13:35:44.222 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:44.218+0000 7f59baecc640 1 -- 172.21.15.43:0/4104940599 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f59b40739f0 con 0x7f59b415df60 2023-08-09T13:35:44.222 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:44.219+0000 7f59b17fa640 1 -- 172.21.15.43:0/4104940599 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 5) v1 ==== 88341+0+0 (secure 0 0 0) 0x7f59a00113f0 con 0x7f59b415df60 2023-08-09T13:35:44.223 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:44.220+0000 7f59b17fa640 1 --2- 172.21.15.43:0/4104940599 >> [v2:172.21.15.203:6832/830097474,v1:172.21.15.203:6833/830097474] conn(0x7f599806a080 0x7f599806c540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:35:44.223 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:44.220+0000 7f59b17fa640 1 -- 172.21.15.43:0/4104940599 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 6871+0+0 (secure 0 0 0) 0x7f59a0088970 con 0x7f59b415df60 2023-08-09T13:35:44.223 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:44.221+0000 7f59b8c68640 1 --2- 172.21.15.43:0/4104940599 >> [v2:172.21.15.203:6832/830097474,v1:172.21.15.203:6833/830097474] conn(0x7f599806a080 0x7f599806c540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:35:44.224 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:44.222+0000 7f59b8c68640 1 --2- 172.21.15.43:0/4104940599 >> [v2:172.21.15.203:6832/830097474,v1:172.21.15.203:6833/830097474] conn(0x7f599806a080 0x7f599806c540 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7f59a8036000 tx=0x7f59a8030000 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-09T13:35:44.227 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:44.224+0000 7f59b17fa640 1 -- 172.21.15.43:0/4104940599 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178066 (secure 0 0 0) 0x7f59a0053dc0 con 0x7f59b415df60 2023-08-09T13:35:44.420 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:44.417+0000 7f59baecc640 1 -- 172.21.15.43:0/4104940599 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"} v 0) v1 -- 0x7f59b4154df0 con 0x7f59b415df60 2023-08-09T13:35:44.426 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:44.418+0000 7f59b17fa640 1 -- 172.21.15.43:0/4104940599 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"}]=0 v26) v1 ==== 92+0+10 (secure 0 0 0) 0x7f59a00573e0 con 0x7f59b415df60 2023-08-09T13:35:44.427 INFO:teuthology.orchestra.run.smithi043.stdout:pg_num: 8 2023-08-09T13:35:44.428 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:44.421+0000 7f59baecc640 1 -- 172.21.15.43:0/4104940599 >> [v2:172.21.15.203:6832/830097474,v1:172.21.15.203:6833/830097474] conn(0x7f599806a080 msgr2=0x7f599806c540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:35:44.428 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:44.421+0000 7f59baecc640 1 --2- 172.21.15.43:0/4104940599 >> [v2:172.21.15.203:6832/830097474,v1:172.21.15.203:6833/830097474] conn(0x7f599806a080 0x7f599806c540 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7f59a8036000 tx=0x7f59a8030000 comp rx=0 tx=0).stop 2023-08-09T13:35:44.428 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:44.421+0000 7f59baecc640 1 -- 172.21.15.43:0/4104940599 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f59b415df60 msgr2=0x7f59b415bb30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:35:44.428 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:44.421+0000 7f59baecc640 1 --2- 172.21.15.43:0/4104940599 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f59b415df60 0x7f59b415bb30 secure :-1 s=READY pgs=441 cs=0 l=1 rev1=1 crypto rx=0x7f59a000c7c0 tx=0x7f59a000cc80 comp rx=0 tx=0).stop 2023-08-09T13:35:44.428 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:44.421+0000 7f59baecc640 1 -- 172.21.15.43:0/4104940599 shutdown_connections 2023-08-09T13:35:44.429 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:44.421+0000 7f59baecc640 1 --2- 172.21.15.43:0/4104940599 >> [v2:172.21.15.203:6832/830097474,v1:172.21.15.203:6833/830097474] conn(0x7f599806a080 0x7f599806c540 unknown :-1 s=CLOSED pgs=221 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:35:44.429 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:44.421+0000 7f59baecc640 1 --2- 172.21.15.43:0/4104940599 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f59b415e3c0 0x7f59b415c070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:35:44.429 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:44.421+0000 7f59baecc640 1 --2- 172.21.15.43:0/4104940599 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f59b415df60 0x7f59b415bb30 unknown :-1 s=CLOSED pgs=441 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:35:44.429 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:44.421+0000 7f59baecc640 1 --2- 172.21.15.43:0/4104940599 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f59b40cb2d0 0x7f59b40c8290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:35:44.429 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:44.421+0000 7f59baecc640 1 -- 172.21.15.43:0/4104940599 >> 172.21.15.43:0/4104940599 conn(0x7f59b40aff50 msgr2=0x7f59b40ce190 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-09T13:35:44.429 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:44.421+0000 7f59baecc640 1 -- 172.21.15.43:0/4104940599 shutdown_connections 2023-08-09T13:35:44.429 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:44.421+0000 7f59baecc640 1 -- 172.21.15.43:0/4104940599 wait complete. 2023-08-09T13:35:44.444 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-08-09T13:35:44.675 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:44.672+0000 7f9ab3396640 1 Processor -- start 2023-08-09T13:35:44.675 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:44.673+0000 7f9ab3396640 1 -- start start 2023-08-09T13:35:44.677 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:44.674+0000 7f9ab3396640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f9aac0cb2d0 0x7f9aac15b380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:35:44.677 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:44.675+0000 7f9ab3396640 1 -- --> v1:172.21.15.43:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9aac15aef0 con 0x7f9aac0ce8d0 2023-08-09T13:35:44.677 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:44.675+0000 7f9ab3396640 1 -- --> v1:172.21.15.203:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9aac15b8c0 con 0x7f9aac15df60 2023-08-09T13:35:44.677 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:44.675+0000 7f9ab3396640 1 -- --> v2:172.21.15.43:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9aac15a970 con 0x7f9aac0cb2d0 2023-08-09T13:35:44.678 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:44.675+0000 7f9ab1132640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f9aac0cb2d0 0x7f9aac15b380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:35:44.678 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:44.675+0000 7f9ab1933640 1 --1- >> v1:172.21.15.43:6789/0 conn(0x7f9aac0ce8d0 0x7f9aac0c9f90 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.43:6789/0 says I am v1:172.21.15.43:44970/0 (socket says 172.21.15.43:44970) 2023-08-09T13:35:44.679 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:44.675+0000 7f9ab1933640 1 -- 172.21.15.43:0/436888068 learned_addr learned my addr 172.21.15.43:0/436888068 (peer_addr_for_me v1:172.21.15.43:0/0) 2023-08-09T13:35:44.679 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:44.675+0000 7f9ab1132640 1 -- 172.21.15.43:0/436888068 >> v1:172.21.15.203:6789/0 conn(0x7f9aac15df60 legacy=0x7f9aac0ce1c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:35:44.679 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:44.675+0000 7f9ab1132640 1 -- 172.21.15.43:0/436888068 >> v1:172.21.15.43:6789/0 conn(0x7f9aac0ce8d0 legacy=0x7f9aac0c9f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:35:44.679 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:44.675+0000 7f9ab1132640 1 -- 172.21.15.43:0/436888068 --> v2:172.21.15.43:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9aac15e940 con 0x7f9aac0cb2d0 2023-08-09T13:35:44.680 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:44.677+0000 7f9ab1132640 1 --2- 172.21.15.43:0/436888068 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f9aac0cb2d0 0x7f9aac15b380 secure :-1 s=READY pgs=443 cs=0 l=1 rev1=1 crypto rx=0x7f9aa80097a0 tx=0x7f9aa802b960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9c0569b65eab5dd6 server_cookie=0 in_seq=0 out_seq=0 2023-08-09T13:35:44.680 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:44.677+0000 7f9aa3fff640 1 -- 172.21.15.43:0/436888068 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9aa8030610 con 0x7f9aac0cb2d0 2023-08-09T13:35:44.680 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:44.677+0000 7f9aa3fff640 1 -- 172.21.15.43:0/436888068 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9aa8030770 con 0x7f9aac0cb2d0 2023-08-09T13:35:44.681 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:44.677+0000 7f9aa3fff640 1 -- 172.21.15.43:0/436888068 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9aa8030b00 con 0x7f9aac0cb2d0 2023-08-09T13:35:44.681 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:44.677+0000 7f9ab3396640 1 -- 172.21.15.43:0/436888068 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f9aac0cb2d0 msgr2=0x7f9aac15b380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:35:44.681 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:44.677+0000 7f9ab3396640 1 --2- 172.21.15.43:0/436888068 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f9aac0cb2d0 0x7f9aac15b380 secure :-1 s=READY pgs=443 cs=0 l=1 rev1=1 crypto rx=0x7f9aa80097a0 tx=0x7f9aa802b960 comp rx=0 tx=0).stop 2023-08-09T13:35:44.681 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:44.678+0000 7f9ab3396640 1 -- 172.21.15.43:0/436888068 shutdown_connections 2023-08-09T13:35:44.681 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:44.678+0000 7f9ab3396640 1 --2- 172.21.15.43:0/436888068 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f9aac0cb2d0 0x7f9aac15b380 unknown :-1 s=CLOSED pgs=443 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:35:44.681 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:44.678+0000 7f9ab3396640 1 -- 172.21.15.43:0/436888068 >> 172.21.15.43:0/436888068 conn(0x7f9aac0aff50 msgr2=0x7f9aac0b0360 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-09T13:35:44.681 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:44.678+0000 7f9ab3396640 1 -- 172.21.15.43:0/436888068 shutdown_connections 2023-08-09T13:35:44.682 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:44.678+0000 7f9ab3396640 1 -- 172.21.15.43:0/436888068 wait complete. 2023-08-09T13:35:44.682 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:44.679+0000 7f9ab3396640 1 Processor -- start 2023-08-09T13:35:44.682 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:44.680+0000 7f9ab3396640 1 -- start start 2023-08-09T13:35:44.683 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:44.681+0000 7f9ab3396640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f9aac0cb2d0 0x7f9aac0c6020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:35:44.684 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:44.681+0000 7f9ab3396640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f9aac0ce8d0 0x7f9aac15c430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:35:44.684 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:44.681+0000 7f9ab3396640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f9aac15df60 0x7f9aac150400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:35:44.684 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:44.681+0000 7f9ab3396640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9aac0c7310 con 0x7f9aac0ce8d0 2023-08-09T13:35:44.684 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:44.681+0000 7f9ab3396640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9aac0c6690 con 0x7f9aac15df60 2023-08-09T13:35:44.684 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:44.681+0000 7f9ab3396640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9aac0c6970 con 0x7f9aac0cb2d0 2023-08-09T13:35:44.684 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:44.681+0000 7f9ab0931640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f9aac0ce8d0 0x7f9aac15c430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:35:44.684 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:44.681+0000 7f9ab0931640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f9aac0ce8d0 0x7f9aac15c430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:50176/0 (socket says 172.21.15.43:50176) 2023-08-09T13:35:44.685 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:44.681+0000 7f9ab0931640 1 -- 172.21.15.43:0/1350123536 learned_addr learned my addr 172.21.15.43:0/1350123536 (peer_addr_for_me v2:172.21.15.43:0/0) 2023-08-09T13:35:44.686 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:44.681+0000 7f9ab1132640 1 --2- 172.21.15.43:0/1350123536 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f9aac0cb2d0 0x7f9aac0c6020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:35:44.686 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:44.682+0000 7f9ab1933640 1 --2- 172.21.15.43:0/1350123536 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f9aac15df60 0x7f9aac150400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:35:44.687 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:44.682+0000 7f9ab0931640 1 -- 172.21.15.43:0/1350123536 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f9aac0cb2d0 msgr2=0x7f9aac0c6020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:35:44.687 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:44.682+0000 7f9ab0931640 1 --2- 172.21.15.43:0/1350123536 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f9aac0cb2d0 0x7f9aac0c6020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:35:44.687 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:44.682+0000 7f9ab0931640 1 -- 172.21.15.43:0/1350123536 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f9aac15df60 msgr2=0x7f9aac150400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:35:44.687 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:44.682+0000 7f9ab0931640 1 --2- 172.21.15.43:0/1350123536 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f9aac15df60 0x7f9aac150400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:35:44.687 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:44.682+0000 7f9ab0931640 1 -- 172.21.15.43:0/1350123536 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9aa8009400 con 0x7f9aac0ce8d0 2023-08-09T13:35:44.687 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:44.682+0000 7f9ab0931640 1 --2- 172.21.15.43:0/1350123536 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f9aac0ce8d0 0x7f9aac15c430 secure :-1 s=READY pgs=444 cs=0 l=1 rev1=1 crypto rx=0x7f9a9c00c400 tx=0x7f9a9c00c8c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-09T13:35:44.688 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:44.682+0000 7f9aa1ffb640 1 -- 172.21.15.43:0/1350123536 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9a9c0052d0 con 0x7f9aac0ce8d0 2023-08-09T13:35:44.688 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:44.682+0000 7f9aa1ffb640 1 -- 172.21.15.43:0/1350123536 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9a9c005430 con 0x7f9aac0ce8d0 2023-08-09T13:35:44.688 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:44.682+0000 7f9aa1ffb640 1 -- 172.21.15.43:0/1350123536 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9a9c003830 con 0x7f9aac0ce8d0 2023-08-09T13:35:44.688 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:44.682+0000 7f9ab3396640 1 -- 172.21.15.43:0/1350123536 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9aac0c6c50 con 0x7f9aac0ce8d0 2023-08-09T13:35:44.688 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:44.683+0000 7f9ab3396640 1 -- 172.21.15.43:0/1350123536 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9aac1516a0 con 0x7f9aac0ce8d0 2023-08-09T13:35:44.689 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:44.684+0000 7f9ab3396640 1 -- 172.21.15.43:0/1350123536 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9aac151830 con 0x7f9aac0ce8d0 2023-08-09T13:35:44.689 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:44.685+0000 7f9aa1ffb640 1 -- 172.21.15.43:0/1350123536 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 5) v1 ==== 88341+0+0 (secure 0 0 0) 0x7f9a9c003990 con 0x7f9aac0ce8d0 2023-08-09T13:35:44.689 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:44.686+0000 7f9aa1ffb640 1 --2- 172.21.15.43:0/1350123536 >> [v2:172.21.15.203:6832/830097474,v1:172.21.15.203:6833/830097474] conn(0x7f9a9006a050 0x7f9a9006c510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:35:44.689 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:44.687+0000 7f9aa1ffb640 1 -- 172.21.15.43:0/1350123536 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 6871+0+0 (secure 0 0 0) 0x7f9a9c087e60 con 0x7f9aac0ce8d0 2023-08-09T13:35:44.690 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:44.687+0000 7f9ab1132640 1 --2- 172.21.15.43:0/1350123536 >> [v2:172.21.15.203:6832/830097474,v1:172.21.15.203:6833/830097474] conn(0x7f9a9006a050 0x7f9a9006c510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:35:44.691 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:44.688+0000 7f9ab1132640 1 --2- 172.21.15.43:0/1350123536 >> [v2:172.21.15.203:6832/830097474,v1:172.21.15.203:6833/830097474] conn(0x7f9a9006a050 0x7f9a9006c510 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7f9aa8036000 tx=0x7f9aa8008030 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-09T13:35:44.695 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:44.693+0000 7f9aa1ffb640 1 -- 172.21.15.43:0/1350123536 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178066 (secure 0 0 0) 0x7f9aac151830 con 0x7f9aac0ce8d0 2023-08-09T13:35:44.889 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:44.887+0000 7f9ab3396640 1 -- 172.21.15.43:0/1350123536 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f9aac0c7450 con 0x7f9aac0ce8d0 2023-08-09T13:35:44.890 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:44.888+0000 7f9aa1ffb640 1 -- 172.21.15.43:0/1350123536 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v26) v1 ==== 93+0+10 (secure 0 0 0) 0x7f9a9c017020 con 0x7f9aac0ce8d0 2023-08-09T13:35:44.890 INFO:teuthology.orchestra.run.smithi043.stdout:pg_num: 1 2023-08-09T13:35:44.893 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:44.890+0000 7f9ab3396640 1 -- 172.21.15.43:0/1350123536 >> [v2:172.21.15.203:6832/830097474,v1:172.21.15.203:6833/830097474] conn(0x7f9a9006a050 msgr2=0x7f9a9006c510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:35:44.893 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:44.890+0000 7f9ab3396640 1 --2- 172.21.15.43:0/1350123536 >> [v2:172.21.15.203:6832/830097474,v1:172.21.15.203:6833/830097474] conn(0x7f9a9006a050 0x7f9a9006c510 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7f9aa8036000 tx=0x7f9aa8008030 comp rx=0 tx=0).stop 2023-08-09T13:35:44.894 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:44.890+0000 7f9ab3396640 1 -- 172.21.15.43:0/1350123536 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f9aac0ce8d0 msgr2=0x7f9aac15c430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:35:44.894 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:44.890+0000 7f9ab3396640 1 --2- 172.21.15.43:0/1350123536 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f9aac0ce8d0 0x7f9aac15c430 secure :-1 s=READY pgs=444 cs=0 l=1 rev1=1 crypto rx=0x7f9a9c00c400 tx=0x7f9a9c00c8c0 comp rx=0 tx=0).stop 2023-08-09T13:35:44.894 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:44.891+0000 7f9ab3396640 1 -- 172.21.15.43:0/1350123536 shutdown_connections 2023-08-09T13:35:44.894 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:44.891+0000 7f9ab3396640 1 --2- 172.21.15.43:0/1350123536 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f9aac15df60 0x7f9aac150400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:35:44.894 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:44.891+0000 7f9ab3396640 1 --2- 172.21.15.43:0/1350123536 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f9aac0ce8d0 0x7f9aac15c430 unknown :-1 s=CLOSED pgs=444 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:35:44.894 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:44.891+0000 7f9ab3396640 1 --2- 172.21.15.43:0/1350123536 >> [v2:172.21.15.203:6832/830097474,v1:172.21.15.203:6833/830097474] conn(0x7f9a9006a050 0x7f9a9006c510 unknown :-1 s=CLOSED pgs=222 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:35:44.894 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:44.891+0000 7f9ab3396640 1 --2- 172.21.15.43:0/1350123536 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f9aac0cb2d0 0x7f9aac0c6020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:35:44.894 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:44.891+0000 7f9ab3396640 1 -- 172.21.15.43:0/1350123536 >> 172.21.15.43:0/1350123536 conn(0x7f9aac0aff50 msgr2=0x7f9aac159e20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-09T13:35:44.895 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:44.891+0000 7f9ab3396640 1 -- 172.21.15.43:0/1350123536 shutdown_connections 2023-08-09T13:35:44.895 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:44.891+0000 7f9ab3396640 1 -- 172.21.15.43:0/1350123536 wait complete. 2023-08-09T13:35:44.911 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-08-09T13:35:45.142 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:45.140+0000 7fca7c805640 1 Processor -- start 2023-08-09T13:35:45.143 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:45.140+0000 7fca7c805640 1 -- start start 2023-08-09T13:35:45.144 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:45.141+0000 7fca7c805640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7fca780b0f00 0x7fca780ccef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:35:45.144 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:45.142+0000 7fca7c805640 1 -- --> v1:172.21.15.43:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fca780b7c00 con 0x7fca7815c3a0 2023-08-09T13:35:45.144 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:45.142+0000 7fca7c805640 1 -- --> v1:172.21.15.203:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fca780b7ee0 con 0x7fca780b1450 2023-08-09T13:35:45.145 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:45.142+0000 7fca7c805640 1 -- --> v2:172.21.15.43:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fca780b81c0 con 0x7fca780b0f00 2023-08-09T13:35:45.145 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:45.142+0000 7fca777fe640 1 --1- >> v1:172.21.15.43:6789/0 conn(0x7fca7815c3a0 0x7fca781608a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.43:6789/0 says I am v1:172.21.15.43:48534/0 (socket says 172.21.15.43:48534) 2023-08-09T13:35:45.145 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:45.142+0000 7fca777fe640 1 -- 172.21.15.43:0/2868601630 learned_addr learned my addr 172.21.15.43:0/2868601630 (peer_addr_for_me v1:172.21.15.43:0/0) 2023-08-09T13:35:45.145 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:45.142+0000 7fca76ffd640 1 --2- 172.21.15.43:0/2868601630 >> v2:172.21.15.43:3300/0 conn(0x7fca780b0f00 0x7fca780ccef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:35:45.146 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:45.142+0000 7fca767fc640 1 -- 172.21.15.43:0/2868601630 <== mon.0 v1:172.21.15.43:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3993657611 0 0) 0x7fca780b7c00 con 0x7fca7815c3a0 2023-08-09T13:35:45.146 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:45.142+0000 7fca767fc640 1 -- 172.21.15.43:0/2868601630 --> v1:172.21.15.43:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fca58003520 con 0x7fca7815c3a0 2023-08-09T13:35:45.146 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:45.143+0000 7fca76ffd640 1 -- 172.21.15.43:0/2868601630 >> v1:172.21.15.203:6789/0 conn(0x7fca780b1450 legacy=0x7fca780d0580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-09T13:35:45.146 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:45.143+0000 7fca76ffd640 1 -- 172.21.15.43:0/2868601630 >> v1:172.21.15.43:6789/0 conn(0x7fca7815c3a0 legacy=0x7fca781608a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:35:45.146 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:45.143+0000 7fca76ffd640 1 -- 172.21.15.43:0/2868601630 --> v2:172.21.15.43:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fca781600c0 con 0x7fca780b0f00 2023-08-09T13:35:45.147 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:45.144+0000 7fca76ffd640 1 --2- 172.21.15.43:0/2868601630 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fca780b0f00 0x7fca780ccef0 secure :-1 s=READY pgs=446 cs=0 l=1 rev1=1 crypto rx=0x7fca60009de0 tx=0x7fca6002be70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fe443f25ec0e89ef server_cookie=0 in_seq=0 out_seq=0 2023-08-09T13:35:45.147 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:45.144+0000 7fca767fc640 1 -- 172.21.15.43:0/2868601630 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fca6002cc20 con 0x7fca780b0f00 2023-08-09T13:35:45.147 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:45.144+0000 7fca767fc640 1 -- 172.21.15.43:0/2868601630 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fca6002cd80 con 0x7fca780b0f00 2023-08-09T13:35:45.147 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:45.144+0000 7fca767fc640 1 -- 172.21.15.43:0/2868601630 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fca60034640 con 0x7fca780b0f00 2023-08-09T13:35:45.149 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:45.145+0000 7fca7c805640 1 -- 172.21.15.43:0/2868601630 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fca780b0f00 msgr2=0x7fca780ccef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:35:45.149 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:45.145+0000 7fca7c805640 1 --2- 172.21.15.43:0/2868601630 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fca780b0f00 0x7fca780ccef0 secure :-1 s=READY pgs=446 cs=0 l=1 rev1=1 crypto rx=0x7fca60009de0 tx=0x7fca6002be70 comp rx=0 tx=0).stop 2023-08-09T13:35:45.149 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:45.145+0000 7fca7c805640 1 -- 172.21.15.43:0/2868601630 shutdown_connections 2023-08-09T13:35:45.149 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:45.145+0000 7fca7c805640 1 --2- 172.21.15.43:0/2868601630 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fca780b0f00 0x7fca780ccef0 unknown :-1 s=CLOSED pgs=446 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:35:45.149 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:45.145+0000 7fca7c805640 1 -- 172.21.15.43:0/2868601630 >> 172.21.15.43:0/2868601630 conn(0x7fca7806a210 msgr2=0x7fca7806a620 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-09T13:35:45.150 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:45.145+0000 7fca7c805640 1 -- 172.21.15.43:0/2868601630 shutdown_connections 2023-08-09T13:35:45.150 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:45.145+0000 7fca7c805640 1 -- 172.21.15.43:0/2868601630 wait complete. 2023-08-09T13:35:45.150 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:45.147+0000 7fca7c805640 1 Processor -- start 2023-08-09T13:35:45.150 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:45.148+0000 7fca7c805640 1 -- start start 2023-08-09T13:35:45.151 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:45.149+0000 7fca7c805640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fca780b0f00 0x7fca781f16e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:35:45.152 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:45.149+0000 7fca7c805640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fca780b1450 0x7fca781f43a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:35:45.152 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:45.149+0000 7fca7c805640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fca7815c3a0 0x7fca780c5ce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:35:45.152 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:45.149+0000 7fca7c805640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fca780c78a0 con 0x7fca780b0f00 2023-08-09T13:35:45.152 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:45.149+0000 7fca7c805640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fca780c65e0 con 0x7fca780b1450 2023-08-09T13:35:45.152 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:45.149+0000 7fca7c805640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fca780c68c0 con 0x7fca7815c3a0 2023-08-09T13:35:45.153 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:45.149+0000 7fca777fe640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fca780b0f00 0x7fca781f16e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:35:45.154 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:45.150+0000 7fca777fe640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fca780b0f00 0x7fca781f16e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:35474/0 (socket says 172.21.15.43:35474) 2023-08-09T13:35:45.154 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:45.150+0000 7fca777fe640 1 -- 172.21.15.43:0/803452772 learned_addr learned my addr 172.21.15.43:0/803452772 (peer_addr_for_me v2:172.21.15.43:0/0) 2023-08-09T13:35:45.154 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:45.150+0000 7fca77fff640 1 --2- 172.21.15.43:0/803452772 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fca7815c3a0 0x7fca780c5ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:35:45.154 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:45.150+0000 7fca76ffd640 1 --2- 172.21.15.43:0/803452772 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fca780b1450 0x7fca781f43a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:35:45.154 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:45.150+0000 7fca777fe640 1 -- 172.21.15.43:0/803452772 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fca7815c3a0 msgr2=0x7fca780c5ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:35:45.154 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:45.150+0000 7fca777fe640 1 --2- 172.21.15.43:0/803452772 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fca7815c3a0 0x7fca780c5ce0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:35:45.154 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:45.150+0000 7fca777fe640 1 -- 172.21.15.43:0/803452772 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fca780b1450 msgr2=0x7fca781f43a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:35:45.155 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:45.150+0000 7fca777fe640 1 --2- 172.21.15.43:0/803452772 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fca780b1450 0x7fca781f43a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:35:45.155 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:45.150+0000 7fca777fe640 1 -- 172.21.15.43:0/803452772 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fca60009a70 con 0x7fca780b0f00 2023-08-09T13:35:45.155 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:45.150+0000 7fca76ffd640 1 --2- 172.21.15.43:0/803452772 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fca780b1450 0x7fca781f43a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-08-09T13:35:45.155 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:45.150+0000 7fca777fe640 1 --2- 172.21.15.43:0/803452772 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fca780b0f00 0x7fca781f16e0 secure :-1 s=READY pgs=447 cs=0 l=1 rev1=1 crypto rx=0x7fca6800c430 tx=0x7fca6800c8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-09T13:35:45.155 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:45.151+0000 7fca6ffff640 1 -- 172.21.15.43:0/803452772 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fca68005ce0 con 0x7fca780b0f00 2023-08-09T13:35:45.155 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:45.151+0000 7fca6ffff640 1 -- 172.21.15.43:0/803452772 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fca68005e40 con 0x7fca780b0f00 2023-08-09T13:35:45.155 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:45.151+0000 7fca6ffff640 1 -- 172.21.15.43:0/803452772 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fca680072f0 con 0x7fca780b0f00 2023-08-09T13:35:45.155 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:45.151+0000 7fca7c805640 1 -- 172.21.15.43:0/803452772 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fca780c6ba0 con 0x7fca780b0f00 2023-08-09T13:35:45.156 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:45.151+0000 7fca7c805640 1 -- 172.21.15.43:0/803452772 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fca781f48e0 con 0x7fca780b0f00 2023-08-09T13:35:45.156 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:45.152+0000 7fca7c805640 1 -- 172.21.15.43:0/803452772 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fca7804e390 con 0x7fca780b0f00 2023-08-09T13:35:45.156 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:45.153+0000 7fca6ffff640 1 -- 172.21.15.43:0/803452772 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 5) v1 ==== 88341+0+0 (secure 0 0 0) 0x7fca68007450 con 0x7fca780b0f00 2023-08-09T13:35:45.157 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:45.154+0000 7fca6ffff640 1 --2- 172.21.15.43:0/803452772 >> [v2:172.21.15.203:6832/830097474,v1:172.21.15.203:6833/830097474] conn(0x7fca5c06a050 0x7fca5c06c510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:35:45.157 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:45.154+0000 7fca6ffff640 1 -- 172.21.15.43:0/803452772 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 6871+0+0 (secure 0 0 0) 0x7fca68087fe0 con 0x7fca780b0f00 2023-08-09T13:35:45.158 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:45.155+0000 7fca76ffd640 1 --2- 172.21.15.43:0/803452772 >> [v2:172.21.15.203:6832/830097474,v1:172.21.15.203:6833/830097474] conn(0x7fca5c06a050 0x7fca5c06c510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:35:45.159 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:45.157+0000 7fca76ffd640 1 --2- 172.21.15.43:0/803452772 >> [v2:172.21.15.203:6832/830097474,v1:172.21.15.203:6833/830097474] conn(0x7fca5c06a050 0x7fca5c06c510 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7fca60036000 tx=0x7fca60030000 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-09T13:35:45.161 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:45.159+0000 7fca6ffff640 1 -- 172.21.15.43:0/803452772 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178066 (secure 0 0 0) 0x7fca68056db0 con 0x7fca780b0f00 2023-08-09T13:35:45.354 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:45.351+0000 7fca7c805640 1 -- 172.21.15.43:0/803452772 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7fca780cfe50 con 0x7fca780b0f00 2023-08-09T13:35:45.355 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:45.352+0000 7fca6ffff640 1 -- 172.21.15.43:0/803452772 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v26) v1 ==== 104+0+11 (secure 0 0 0) 0x7fca68017020 con 0x7fca780b0f00 2023-08-09T13:35:45.355 INFO:teuthology.orchestra.run.smithi043.stdout:pg_num: 32 2023-08-09T13:35:45.357 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:45.355+0000 7fca7c805640 1 -- 172.21.15.43:0/803452772 >> [v2:172.21.15.203:6832/830097474,v1:172.21.15.203:6833/830097474] conn(0x7fca5c06a050 msgr2=0x7fca5c06c510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:35:45.358 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:45.355+0000 7fca7c805640 1 --2- 172.21.15.43:0/803452772 >> [v2:172.21.15.203:6832/830097474,v1:172.21.15.203:6833/830097474] conn(0x7fca5c06a050 0x7fca5c06c510 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7fca60036000 tx=0x7fca60030000 comp rx=0 tx=0).stop 2023-08-09T13:35:45.358 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:45.355+0000 7fca7c805640 1 -- 172.21.15.43:0/803452772 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fca780b0f00 msgr2=0x7fca781f16e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:35:45.358 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:45.355+0000 7fca7c805640 1 --2- 172.21.15.43:0/803452772 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fca780b0f00 0x7fca781f16e0 secure :-1 s=READY pgs=447 cs=0 l=1 rev1=1 crypto rx=0x7fca6800c430 tx=0x7fca6800c8f0 comp rx=0 tx=0).stop 2023-08-09T13:35:45.358 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:45.355+0000 7fca7c805640 1 -- 172.21.15.43:0/803452772 shutdown_connections 2023-08-09T13:35:45.358 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:45.355+0000 7fca7c805640 1 --2- 172.21.15.43:0/803452772 >> [v2:172.21.15.203:6832/830097474,v1:172.21.15.203:6833/830097474] conn(0x7fca5c06a050 0x7fca5c06c510 unknown :-1 s=CLOSED pgs=223 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:35:45.358 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:45.355+0000 7fca7c805640 1 --2- 172.21.15.43:0/803452772 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fca7815c3a0 0x7fca780c5ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:35:45.358 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:45.355+0000 7fca7c805640 1 --2- 172.21.15.43:0/803452772 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fca780b1450 0x7fca781f43a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:35:45.359 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:45.355+0000 7fca7c805640 1 --2- 172.21.15.43:0/803452772 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fca780b0f00 0x7fca781f16e0 unknown :-1 s=CLOSED pgs=447 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:35:45.359 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:45.355+0000 7fca7c805640 1 -- 172.21.15.43:0/803452772 >> 172.21.15.43:0/803452772 conn(0x7fca7806a210 msgr2=0x7fca780d09e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-09T13:35:45.359 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:45.355+0000 7fca7c805640 1 -- 172.21.15.43:0/803452772 shutdown_connections 2023-08-09T13:35:45.359 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:45.355+0000 7fca7c805640 1 -- 172.21.15.43:0/803452772 wait complete. 2023-08-09T13:35:45.376 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-08-09T13:35:45.612 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:45.609+0000 7f4e51122640 1 Processor -- start 2023-08-09T13:35:45.612 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:45.609+0000 7f4e51122640 1 -- start start 2023-08-09T13:35:45.613 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:45.611+0000 7f4e51122640 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7f4e4c0cf300 0x7f4e4c0d1b30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:35:45.614 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:45.611+0000 7f4e51122640 1 -- --> v1:172.21.15.43:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4e4c0d16a0 con 0x7f4e4c0b7df0 2023-08-09T13:35:45.614 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:45.611+0000 7f4e51122640 1 -- --> v1:172.21.15.203:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4e4c0d0d70 con 0x7f4e4c15c2d0 2023-08-09T13:35:45.614 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:45.611+0000 7f4e51122640 1 -- --> v2:172.21.15.203:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4e4c0b8230 con 0x7f4e4c0cf300 2023-08-09T13:35:45.615 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:45.611+0000 7f4e4b7fe640 1 --1- >> v1:172.21.15.43:6789/0 conn(0x7f4e4c0b7df0 0x7f4e4c0cebf0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.43:6789/0 says I am v1:172.21.15.43:48546/0 (socket says 172.21.15.43:48546) 2023-08-09T13:35:45.615 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:45.611+0000 7f4e4b7fe640 1 -- 172.21.15.43:0/4067355314 learned_addr learned my addr 172.21.15.43:0/4067355314 (peer_addr_for_me v1:172.21.15.43:0/0) 2023-08-09T13:35:45.615 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:45.612+0000 7f4e50921640 1 --2- 172.21.15.43:0/4067355314 >> v2:172.21.15.203:3300/0 conn(0x7f4e4c0cf300 0x7f4e4c0d1b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:35:45.615 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:45.612+0000 7f4e4affd640 1 -- 172.21.15.43:0/4067355314 <== mon.0 v1:172.21.15.43:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3279688171 0 0) 0x7f4e4c0d16a0 con 0x7f4e4c0b7df0 2023-08-09T13:35:45.616 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:45.612+0000 7f4e4affd640 1 -- 172.21.15.43:0/4067355314 --> v1:172.21.15.43:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f4e38003520 con 0x7f4e4c0b7df0 2023-08-09T13:35:45.616 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:45.612+0000 7f4e4affd640 1 -- 172.21.15.43:0/4067355314 <== mon.0 v1:172.21.15.43:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1851378607 0 0) 0x7f4e38003520 con 0x7f4e4c0b7df0 2023-08-09T13:35:45.616 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:45.612+0000 7f4e4affd640 1 -- 172.21.15.43:0/4067355314 >> v2:172.21.15.203:3300/0 conn(0x7f4e4c0cf300 msgr2=0x7f4e4c0d1b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-09T13:35:45.616 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:45.612+0000 7f4e4affd640 1 --2- 172.21.15.43:0/4067355314 >> v2:172.21.15.203:3300/0 conn(0x7f4e4c0cf300 0x7f4e4c0d1b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:35:45.616 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:45.612+0000 7f4e4affd640 1 -- 172.21.15.43:0/4067355314 >> v1:172.21.15.203:6789/0 conn(0x7f4e4c15c2d0 legacy=0x7f4e4c0cd020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:35:45.616 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:45.612+0000 7f4e4affd640 1 -- 172.21.15.43:0/4067355314 --> v1:172.21.15.43:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4e4c0d1050 con 0x7f4e4c0b7df0 2023-08-09T13:35:45.617 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:45.612+0000 7f4e4affd640 1 -- 172.21.15.43:0/4067355314 <== mon.0 v1:172.21.15.43:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1285150740 0 0) 0x7f4e34001670 con 0x7f4e4c0b7df0 2023-08-09T13:35:45.617 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:45.613+0000 7f4e4affd640 1 -- 172.21.15.43:0/4067355314 >> v1:172.21.15.43:6789/0 conn(0x7f4e4c0b7df0 legacy=0x7f4e4c0cebf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:35:45.617 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:45.613+0000 7f4e4affd640 1 --2- 172.21.15.43:0/4067355314 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f4e38003970 0x7f4e38003d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:35:45.617 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:45.613+0000 7f4e4affd640 1 --2- 172.21.15.43:0/4067355314 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f4e380047c0 0x7f4e38004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:35:45.617 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:45.613+0000 7f4e4affd640 1 --2- 172.21.15.43:0/4067355314 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f4e38005180 0x7f4e3800b980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:35:45.617 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:45.613+0000 7f4e4affd640 1 -- 172.21.15.43:0/4067355314 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4e3800bff0 con 0x7f4e38003970 2023-08-09T13:35:45.617 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:45.613+0000 7f4e4affd640 1 -- 172.21.15.43:0/4067355314 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4e3800c160 con 0x7f4e380047c0 2023-08-09T13:35:45.618 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:45.613+0000 7f4e4affd640 1 -- 172.21.15.43:0/4067355314 --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4e3800c320 con 0x7f4e38005180 2023-08-09T13:35:45.619 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:45.613+0000 7f4e4bfff640 1 --2- 172.21.15.43:0/4067355314 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f4e38003970 0x7f4e38003d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:35:45.619 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:45.613+0000 7f4e50921640 1 --2- 172.21.15.43:0/4067355314 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f4e38005180 0x7f4e3800b980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:35:45.619 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:45.613+0000 7f4e4b7fe640 1 --2- 172.21.15.43:0/4067355314 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f4e380047c0 0x7f4e38004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:35:45.620 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:45.613+0000 7f4e4bfff640 1 -- 172.21.15.43:0/4067355314 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f4e38005180 msgr2=0x7f4e3800b980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-09T13:35:45.620 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:45.613+0000 7f4e4bfff640 1 --2- 172.21.15.43:0/4067355314 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f4e38005180 0x7f4e3800b980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:35:45.620 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:45.613+0000 7f4e4bfff640 1 -- 172.21.15.43:0/4067355314 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f4e380047c0 msgr2=0x7f4e38004c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-09T13:35:45.620 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:45.614+0000 7f4e4bfff640 1 --2- 172.21.15.43:0/4067355314 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f4e380047c0 0x7f4e38004c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:35:45.620 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:45.614+0000 7f4e4bfff640 1 -- 172.21.15.43:0/4067355314 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4e38003520 con 0x7f4e38003970 2023-08-09T13:35:45.620 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:45.614+0000 7f4e4b7fe640 1 --2- 172.21.15.43:0/4067355314 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f4e380047c0 0x7f4e38004c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-08-09T13:35:45.621 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:45.614+0000 7f4e50921640 1 -- 172.21.15.43:0/4067355314 reap_dead start 2023-08-09T13:35:45.621 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:45.615+0000 7f4e4bfff640 1 --2- 172.21.15.43:0/4067355314 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f4e38003970 0x7f4e38003d90 secure :-1 s=READY pgs=449 cs=0 l=1 rev1=1 crypto rx=0x7f4e3c001e50 tx=0x7f4e380057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=690dd53af21e880f server_cookie=0 in_seq=0 out_seq=0 2023-08-09T13:35:45.621 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:45.615+0000 7f4e4affd640 1 -- 172.21.15.43:0/4067355314 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4e3c003b70 con 0x7f4e38003970 2023-08-09T13:35:45.621 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:45.615+0000 7f4e4affd640 1 -- 172.21.15.43:0/4067355314 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4e3c003cd0 con 0x7f4e38003970 2023-08-09T13:35:45.622 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:45.615+0000 7f4e4affd640 1 -- 172.21.15.43:0/4067355314 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4e3c005310 con 0x7f4e38003970 2023-08-09T13:35:45.622 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:45.615+0000 7f4e51122640 1 -- 172.21.15.43:0/4067355314 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f4e38003970 msgr2=0x7f4e38003d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:35:45.622 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:45.615+0000 7f4e51122640 1 --2- 172.21.15.43:0/4067355314 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f4e38003970 0x7f4e38003d90 secure :-1 s=READY pgs=449 cs=0 l=1 rev1=1 crypto rx=0x7f4e3c001e50 tx=0x7f4e380057f0 comp rx=0 tx=0).stop 2023-08-09T13:35:45.622 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:45.616+0000 7f4e51122640 1 -- 172.21.15.43:0/4067355314 shutdown_connections 2023-08-09T13:35:45.622 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:45.616+0000 7f4e51122640 1 --2- 172.21.15.43:0/4067355314 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f4e38003970 0x7f4e38003d90 unknown :-1 s=CLOSED pgs=449 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:35:45.623 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:45.616+0000 7f4e51122640 1 -- 172.21.15.43:0/4067355314 >> 172.21.15.43:0/4067355314 conn(0x7f4e4c06a230 msgr2=0x7f4e4c06a640 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-09T13:35:45.623 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:45.616+0000 7f4e51122640 1 -- 172.21.15.43:0/4067355314 shutdown_connections 2023-08-09T13:35:45.623 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:45.616+0000 7f4e51122640 1 -- 172.21.15.43:0/4067355314 wait complete. 2023-08-09T13:35:45.626 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:45.618+0000 7f4e51122640 1 Processor -- start 2023-08-09T13:35:45.627 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:45.618+0000 7f4e51122640 1 -- start start 2023-08-09T13:35:45.627 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:45.619+0000 7f4e51122640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f4e38003970 0x7f4e4c0a9550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:35:45.627 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:45.619+0000 7f4e51122640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f4e4c0b4430 0x7f4e4c158540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:35:45.627 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:45.619+0000 7f4e51122640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f4e4c0cb480 0x7f4e4c0cb900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:35:45.628 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:45.619+0000 7f4e51122640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4e4c159080 con 0x7f4e38003970 2023-08-09T13:35:45.628 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:45.619+0000 7f4e51122640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4e4c0cbe40 con 0x7f4e4c0b4430 2023-08-09T13:35:45.628 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:45.619+0000 7f4e51122640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4e4c0b2e50 con 0x7f4e4c0cb480 2023-08-09T13:35:45.629 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:45.619+0000 7f4e4bfff640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f4e38003970 0x7f4e4c0a9550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:35:45.629 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:45.619+0000 7f4e4bfff640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f4e38003970 0x7f4e4c0a9550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:35488/0 (socket says 172.21.15.43:35488) 2023-08-09T13:35:45.629 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:45.619+0000 7f4e4bfff640 1 -- 172.21.15.43:0/2253593967 learned_addr learned my addr 172.21.15.43:0/2253593967 (peer_addr_for_me v2:172.21.15.43:0/0) 2023-08-09T13:35:45.629 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:45.619+0000 7f4e4bfff640 1 -- 172.21.15.43:0/2253593967 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f4e4c0cb480 msgr2=0x7f4e4c0cb900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:35:45.630 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:45.619+0000 7f4e4bfff640 1 --2- 172.21.15.43:0/2253593967 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f4e4c0cb480 0x7f4e4c0cb900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:35:45.630 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:45.619+0000 7f4e4bfff640 1 -- 172.21.15.43:0/2253593967 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f4e4c0b4430 msgr2=0x7f4e4c158540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:35:45.630 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:45.619+0000 7f4e4bfff640 1 --2- 172.21.15.43:0/2253593967 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f4e4c0b4430 0x7f4e4c158540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:35:45.630 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:45.619+0000 7f4e4bfff640 1 -- 172.21.15.43:0/2253593967 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4e4c158a80 con 0x7f4e38003970 2023-08-09T13:35:45.631 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:45.620+0000 7f4e4bfff640 1 --2- 172.21.15.43:0/2253593967 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f4e38003970 0x7f4e4c0a9550 secure :-1 s=READY pgs=450 cs=0 l=1 rev1=1 crypto rx=0x7f4e3c02bf70 tx=0x7f4e3c0014a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-09T13:35:45.631 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:45.620+0000 7f4e48ff9640 1 -- 172.21.15.43:0/2253593967 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4e3c03e070 con 0x7f4e38003970 2023-08-09T13:35:45.631 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:45.620+0000 7f4e48ff9640 1 -- 172.21.15.43:0/2253593967 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4e3c003e40 con 0x7f4e38003970 2023-08-09T13:35:45.631 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:45.620+0000 7f4e48ff9640 1 -- 172.21.15.43:0/2253593967 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4e3c02f640 con 0x7f4e38003970 2023-08-09T13:35:45.631 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:45.620+0000 7f4e51122640 1 -- 172.21.15.43:0/2253593967 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4e4c0b30c0 con 0x7f4e38003970 2023-08-09T13:35:45.631 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:45.620+0000 7f4e51122640 1 -- 172.21.15.43:0/2253593967 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4e4c0b3680 con 0x7f4e38003970 2023-08-09T13:35:45.631 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:45.621+0000 7f4e51122640 1 -- 172.21.15.43:0/2253593967 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4e4c04e390 con 0x7f4e38003970 2023-08-09T13:35:45.632 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:45.622+0000 7f4e48ff9640 1 -- 172.21.15.43:0/2253593967 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 5) v1 ==== 88341+0+0 (secure 0 0 0) 0x7f4e3c02f7a0 con 0x7f4e38003970 2023-08-09T13:35:45.632 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:45.624+0000 7f4e48ff9640 1 --2- 172.21.15.43:0/2253593967 >> [v2:172.21.15.203:6832/830097474,v1:172.21.15.203:6833/830097474] conn(0x7f4e2c06a020 0x7f4e2c06c4e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:35:45.632 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:45.624+0000 7f4e48ff9640 1 -- 172.21.15.43:0/2253593967 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 6871+0+0 (secure 0 0 0) 0x7f4e3c0a9fe0 con 0x7f4e38003970 2023-08-09T13:35:45.633 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:45.625+0000 7f4e4b7fe640 1 --2- 172.21.15.43:0/2253593967 >> [v2:172.21.15.203:6832/830097474,v1:172.21.15.203:6833/830097474] conn(0x7f4e2c06a020 0x7f4e2c06c4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:35:45.633 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:45.626+0000 7f4e4b7fe640 1 --2- 172.21.15.43:0/2253593967 >> [v2:172.21.15.203:6832/830097474,v1:172.21.15.203:6833/830097474] conn(0x7f4e2c06a020 0x7f4e2c06c4e0 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7f4e4c0cf1c0 tx=0x7f4e34009040 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-09T13:35:45.633 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:45.628+0000 7f4e48ff9640 1 -- 172.21.15.43:0/2253593967 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178066 (secure 0 0 0) 0x7f4e3c0754a0 con 0x7f4e38003970 2023-08-09T13:35:45.825 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:45.822+0000 7f4e51122640 1 -- 172.21.15.43:0/2253593967 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f4e4c0c8bd0 con 0x7f4e38003970 2023-08-09T13:35:45.826 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:45.823+0000 7f4e48ff9640 1 -- 172.21.15.43:0/2253593967 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v26) v1 ==== 100+0+11 (secure 0 0 0) 0x7f4e3c047020 con 0x7f4e38003970 2023-08-09T13:35:45.826 INFO:teuthology.orchestra.run.smithi043.stdout:pg_num: 64 2023-08-09T13:35:45.828 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:45.826+0000 7f4e51122640 1 -- 172.21.15.43:0/2253593967 >> [v2:172.21.15.203:6832/830097474,v1:172.21.15.203:6833/830097474] conn(0x7f4e2c06a020 msgr2=0x7f4e2c06c4e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:35:45.829 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:45.826+0000 7f4e51122640 1 --2- 172.21.15.43:0/2253593967 >> [v2:172.21.15.203:6832/830097474,v1:172.21.15.203:6833/830097474] conn(0x7f4e2c06a020 0x7f4e2c06c4e0 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7f4e4c0cf1c0 tx=0x7f4e34009040 comp rx=0 tx=0).stop 2023-08-09T13:35:45.829 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:45.826+0000 7f4e51122640 1 -- 172.21.15.43:0/2253593967 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f4e38003970 msgr2=0x7f4e4c0a9550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:35:45.829 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:45.826+0000 7f4e51122640 1 --2- 172.21.15.43:0/2253593967 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f4e38003970 0x7f4e4c0a9550 secure :-1 s=READY pgs=450 cs=0 l=1 rev1=1 crypto rx=0x7f4e3c02bf70 tx=0x7f4e3c0014a0 comp rx=0 tx=0).stop 2023-08-09T13:35:45.829 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:45.826+0000 7f4e51122640 1 -- 172.21.15.43:0/2253593967 shutdown_connections 2023-08-09T13:35:45.829 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:45.826+0000 7f4e51122640 1 --2- 172.21.15.43:0/2253593967 >> [v2:172.21.15.203:6832/830097474,v1:172.21.15.203:6833/830097474] conn(0x7f4e2c06a020 0x7f4e2c06c4e0 unknown :-1 s=CLOSED pgs=224 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:35:45.829 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:45.826+0000 7f4e51122640 1 --2- 172.21.15.43:0/2253593967 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f4e4c0cb480 0x7f4e4c0cb900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:35:45.829 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:45.826+0000 7f4e51122640 1 --2- 172.21.15.43:0/2253593967 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f4e4c0b4430 0x7f4e4c158540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:35:45.830 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:45.826+0000 7f4e51122640 1 --2- 172.21.15.43:0/2253593967 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f4e38003970 0x7f4e4c0a9550 unknown :-1 s=CLOSED pgs=450 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:35:45.830 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:45.826+0000 7f4e51122640 1 -- 172.21.15.43:0/2253593967 >> 172.21.15.43:0/2253593967 conn(0x7f4e4c06a230 msgr2=0x7f4e4c15d750 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-09T13:35:45.830 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:45.826+0000 7f4e51122640 1 -- 172.21.15.43:0/2253593967 shutdown_connections 2023-08-09T13:35:45.830 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:45.826+0000 7f4e51122640 1 -- 172.21.15.43:0/2253593967 wait complete. 2023-08-09T13:35:45.849 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2023-08-09T13:35:46.078 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:46.075+0000 7fcfd600f640 1 Processor -- start 2023-08-09T13:35:46.078 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:46.075+0000 7fcfd600f640 1 -- start start 2023-08-09T13:35:46.079 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:46.077+0000 7fcfd600f640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7fcfd015c3a0 0x7fcfd01608a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:35:46.080 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:46.077+0000 7fcfd600f640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fcfd00b0f00 0x7fcfd00ccef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:35:46.080 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:46.077+0000 7fcfd600f640 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7fcfd00b1450 0x7fcfd00cd430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:35:46.080 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:46.077+0000 7fcfd600f640 1 -- --> v2:172.21.15.43:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fcfd00b81c0 con 0x7fcfd015c3a0 2023-08-09T13:35:46.080 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:46.077+0000 7fcfd600f640 1 -- --> v2:172.21.15.203:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fcfd0160de0 con 0x7fcfd00b1450 2023-08-09T13:35:46.080 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:46.077+0000 7fcfd600f640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcfd00b7c00 con 0x7fcfd00b0f00 2023-08-09T13:35:46.080 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:46.077+0000 7fcfd500d640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7fcfd015c3a0 0x7fcfd01608a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:35:46.081 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:46.077+0000 7fcfd500d640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7fcfd015c3a0 0x7fcfd01608a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:35490/0 (socket says 172.21.15.43:35490) 2023-08-09T13:35:46.081 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:46.077+0000 7fcfd500d640 1 -- 172.21.15.43:0/2989172391 learned_addr learned my addr 172.21.15.43:0/2989172391 (peer_addr_for_me v2:172.21.15.43:0/0) 2023-08-09T13:35:46.082 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:46.078+0000 7fcfd500d640 1 -- 172.21.15.43:0/2989172391 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fcfd00b0f00 msgr2=0x7fcfd00ccef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-09T13:35:46.082 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:46.078+0000 7fcfd480c640 1 --2- 172.21.15.43:0/2989172391 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fcfd00b0f00 0x7fcfd00ccef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:35:46.082 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:46.078+0000 7fcfd500d640 1 --2- 172.21.15.43:0/2989172391 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fcfd00b0f00 0x7fcfd00ccef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:35:46.082 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:46.078+0000 7fcfd500d640 1 -- 172.21.15.43:0/2989172391 >> v2:172.21.15.203:3300/0 conn(0x7fcfd00b1450 msgr2=0x7fcfd00cd430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-09T13:35:46.082 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:46.078+0000 7fcfd580e640 1 --2- 172.21.15.43:0/2989172391 >> v2:172.21.15.203:3300/0 conn(0x7fcfd00b1450 0x7fcfd00cd430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:35:46.082 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:46.078+0000 7fcfd500d640 1 --2- 172.21.15.43:0/2989172391 >> v2:172.21.15.203:3300/0 conn(0x7fcfd00b1450 0x7fcfd00cd430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:35:46.083 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:46.078+0000 7fcfd500d640 1 -- 172.21.15.43:0/2989172391 --> v2:172.21.15.43:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcfd00b7ee0 con 0x7fcfd015c3a0 2023-08-09T13:35:46.083 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:46.078+0000 7fcfd480c640 1 --2- 172.21.15.43:0/2989172391 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fcfd00b0f00 0x7fcfd00ccef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-08-09T13:35:46.083 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:46.079+0000 7fcfd500d640 1 --2- 172.21.15.43:0/2989172391 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fcfd015c3a0 0x7fcfd01608a0 secure :-1 s=READY pgs=451 cs=0 l=1 rev1=1 crypto rx=0x7fcfd00ceb10 tx=0x7fcfcc02b880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8d33b01e44129f0d server_cookie=0 in_seq=0 out_seq=0 2023-08-09T13:35:46.083 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:46.079+0000 7fcfc7fff640 1 -- 172.21.15.43:0/2989172391 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcfcc0317e0 con 0x7fcfd015c3a0 2023-08-09T13:35:46.083 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:46.079+0000 7fcfc7fff640 1 -- 172.21.15.43:0/2989172391 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcfcc031940 con 0x7fcfd015c3a0 2023-08-09T13:35:46.083 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:46.079+0000 7fcfc7fff640 1 -- 172.21.15.43:0/2989172391 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcfcc031cd0 con 0x7fcfd015c3a0 2023-08-09T13:35:46.084 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:46.080+0000 7fcfd600f640 1 -- 172.21.15.43:0/2989172391 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fcfd015c3a0 msgr2=0x7fcfd01608a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:35:46.084 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:46.080+0000 7fcfd600f640 1 --2- 172.21.15.43:0/2989172391 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fcfd015c3a0 0x7fcfd01608a0 secure :-1 s=READY pgs=451 cs=0 l=1 rev1=1 crypto rx=0x7fcfd00ceb10 tx=0x7fcfcc02b880 comp rx=0 tx=0).stop 2023-08-09T13:35:46.084 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:46.080+0000 7fcfd600f640 1 -- 172.21.15.43:0/2989172391 shutdown_connections 2023-08-09T13:35:46.085 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:46.080+0000 7fcfd600f640 1 --2- 172.21.15.43:0/2989172391 >> v2:172.21.15.203:3300/0 conn(0x7fcfd00b1450 0x7fcfd00cd430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:35:46.085 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:46.080+0000 7fcfd600f640 1 --2- 172.21.15.43:0/2989172391 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fcfd00b0f00 0x7fcfd00ccef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:35:46.085 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:46.080+0000 7fcfd600f640 1 --2- 172.21.15.43:0/2989172391 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fcfd015c3a0 0x7fcfd01608a0 unknown :-1 s=CLOSED pgs=451 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:35:46.085 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:46.080+0000 7fcfd600f640 1 -- 172.21.15.43:0/2989172391 >> 172.21.15.43:0/2989172391 conn(0x7fcfd006a210 msgr2=0x7fcfd006a620 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-09T13:35:46.085 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:46.080+0000 7fcfd600f640 1 -- 172.21.15.43:0/2989172391 shutdown_connections 2023-08-09T13:35:46.085 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:46.080+0000 7fcfd600f640 1 -- 172.21.15.43:0/2989172391 wait complete. 2023-08-09T13:35:46.086 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:46.083+0000 7fcfd600f640 1 Processor -- start 2023-08-09T13:35:46.086 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:46.083+0000 7fcfd600f640 1 -- start start 2023-08-09T13:35:46.087 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:46.085+0000 7fcfd600f640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fcfd00b0f00 0x7fcfd00b2470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:35:46.087 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:46.085+0000 7fcfd600f640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fcfd00b1450 0x7fcfd00b2f10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:35:46.088 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:46.085+0000 7fcfd600f640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fcfd015c3a0 0x7fcfd01f1340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:35:46.088 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:46.085+0000 7fcfd600f640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcfd00b3af0 con 0x7fcfd015c3a0 2023-08-09T13:35:46.088 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:46.085+0000 7fcfd600f640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcfd00b19d0 con 0x7fcfd00b1450 2023-08-09T13:35:46.088 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:46.085+0000 7fcfd600f640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcfd00b3810 con 0x7fcfd00b0f00 2023-08-09T13:35:46.088 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:46.085+0000 7fcfd580e640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fcfd015c3a0 0x7fcfd01f1340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:35:46.088 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:46.085+0000 7fcfd580e640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fcfd015c3a0 0x7fcfd01f1340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:35498/0 (socket says 172.21.15.43:35498) 2023-08-09T13:35:46.088 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:46.085+0000 7fcfd580e640 1 -- 172.21.15.43:0/88817585 learned_addr learned my addr 172.21.15.43:0/88817585 (peer_addr_for_me v2:172.21.15.43:0/0) 2023-08-09T13:35:46.089 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:46.085+0000 7fcfd500d640 1 --2- 172.21.15.43:0/88817585 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fcfd00b0f00 0x7fcfd00b2470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:35:46.090 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:46.085+0000 7fcfd480c640 1 --2- 172.21.15.43:0/88817585 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fcfd00b1450 0x7fcfd00b2f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:35:46.090 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:46.085+0000 7fcfd580e640 1 -- 172.21.15.43:0/88817585 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fcfd00b0f00 msgr2=0x7fcfd00b2470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:35:46.090 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:46.085+0000 7fcfd580e640 1 --2- 172.21.15.43:0/88817585 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fcfd00b0f00 0x7fcfd00b2470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:35:46.090 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:46.086+0000 7fcfd580e640 1 -- 172.21.15.43:0/88817585 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fcfd00b1450 msgr2=0x7fcfd00b2f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:35:46.090 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:46.086+0000 7fcfd580e640 1 --2- 172.21.15.43:0/88817585 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fcfd00b1450 0x7fcfd00b2f10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:35:46.091 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:46.086+0000 7fcfd580e640 1 -- 172.21.15.43:0/88817585 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcfcc0094d0 con 0x7fcfd015c3a0 2023-08-09T13:35:46.091 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:46.086+0000 7fcfd580e640 1 --2- 172.21.15.43:0/88817585 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fcfd015c3a0 0x7fcfd01f1340 secure :-1 s=READY pgs=452 cs=0 l=1 rev1=1 crypto rx=0x7fcfc800cc30 tx=0x7fcfc800a5c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-09T13:35:46.091 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:46.086+0000 7fcfc5ffb640 1 -- 172.21.15.43:0/88817585 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcfc800ce40 con 0x7fcfd015c3a0 2023-08-09T13:35:46.091 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:46.086+0000 7fcfc5ffb640 1 -- 172.21.15.43:0/88817585 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcfc8003990 con 0x7fcfd015c3a0 2023-08-09T13:35:46.091 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:46.086+0000 7fcfc5ffb640 1 -- 172.21.15.43:0/88817585 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcfc8003d20 con 0x7fcfd015c3a0 2023-08-09T13:35:46.091 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:46.086+0000 7fcfd600f640 1 -- 172.21.15.43:0/88817585 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcfd00b1c70 con 0x7fcfd015c3a0 2023-08-09T13:35:46.091 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:46.086+0000 7fcfd600f640 1 -- 172.21.15.43:0/88817585 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcfd00b3450 con 0x7fcfd015c3a0 2023-08-09T13:35:46.092 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:46.088+0000 7fcfd600f640 1 -- 172.21.15.43:0/88817585 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcfd00b1830 con 0x7fcfd015c3a0 2023-08-09T13:35:46.092 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:46.089+0000 7fcfc5ffb640 1 -- 172.21.15.43:0/88817585 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 5) v1 ==== 88341+0+0 (secure 0 0 0) 0x7fcfc8003e80 con 0x7fcfd015c3a0 2023-08-09T13:35:46.092 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:46.090+0000 7fcfc5ffb640 1 --2- 172.21.15.43:0/88817585 >> [v2:172.21.15.203:6832/830097474,v1:172.21.15.203:6833/830097474] conn(0x7fcfb406a050 0x7fcfb406c510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:35:46.092 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:46.090+0000 7fcfc5ffb640 1 -- 172.21.15.43:0/88817585 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 6871+0+0 (secure 0 0 0) 0x7fcfc8088bc0 con 0x7fcfd015c3a0 2023-08-09T13:35:46.093 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:46.090+0000 7fcfd500d640 1 --2- 172.21.15.43:0/88817585 >> [v2:172.21.15.203:6832/830097474,v1:172.21.15.203:6833/830097474] conn(0x7fcfb406a050 0x7fcfb406c510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:35:46.094 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:46.092+0000 7fcfd500d640 1 --2- 172.21.15.43:0/88817585 >> [v2:172.21.15.203:6832/830097474,v1:172.21.15.203:6833/830097474] conn(0x7fcfb406a050 0x7fcfb406c510 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7fcfcc036000 tx=0x7fcfcc03d040 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-09T13:35:46.097 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:46.095+0000 7fcfc5ffb640 1 -- 172.21.15.43:0/88817585 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178066 (secure 0 0 0) 0x7fcfd00b1830 con 0x7fcfd015c3a0 2023-08-09T13:35:46.299 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:46.296+0000 7fcfd600f640 1 -- 172.21.15.43:0/88817585 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) v1 -- 0x7fcfd00b1830 con 0x7fcfd015c3a0 2023-08-09T13:35:46.300 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:46.297+0000 7fcfc5ffb640 1 -- 172.21.15.43:0/88817585 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v26) v1 ==== 103+0+11 (secure 0 0 0) 0x7fcfd00b1830 con 0x7fcfd015c3a0 2023-08-09T13:35:46.300 INFO:teuthology.orchestra.run.smithi043.stdout:pg_num: 32 2023-08-09T13:35:46.303 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:46.300+0000 7fcfd600f640 1 -- 172.21.15.43:0/88817585 >> [v2:172.21.15.203:6832/830097474,v1:172.21.15.203:6833/830097474] conn(0x7fcfb406a050 msgr2=0x7fcfb406c510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:35:46.303 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:46.300+0000 7fcfd600f640 1 --2- 172.21.15.43:0/88817585 >> [v2:172.21.15.203:6832/830097474,v1:172.21.15.203:6833/830097474] conn(0x7fcfb406a050 0x7fcfb406c510 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7fcfcc036000 tx=0x7fcfcc03d040 comp rx=0 tx=0).stop 2023-08-09T13:35:46.303 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:46.300+0000 7fcfd600f640 1 -- 172.21.15.43:0/88817585 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fcfd015c3a0 msgr2=0x7fcfd01f1340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:35:46.303 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:46.300+0000 7fcfd600f640 1 --2- 172.21.15.43:0/88817585 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fcfd015c3a0 0x7fcfd01f1340 secure :-1 s=READY pgs=452 cs=0 l=1 rev1=1 crypto rx=0x7fcfc800cc30 tx=0x7fcfc800a5c0 comp rx=0 tx=0).stop 2023-08-09T13:35:46.303 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:46.300+0000 7fcfd600f640 1 -- 172.21.15.43:0/88817585 shutdown_connections 2023-08-09T13:35:46.303 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:46.300+0000 7fcfd600f640 1 --2- 172.21.15.43:0/88817585 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fcfd015c3a0 0x7fcfd01f1340 unknown :-1 s=CLOSED pgs=452 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:35:46.304 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:46.300+0000 7fcfd600f640 1 --2- 172.21.15.43:0/88817585 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fcfd00b1450 0x7fcfd00b2f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:35:46.304 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:46.300+0000 7fcfd600f640 1 --2- 172.21.15.43:0/88817585 >> [v2:172.21.15.203:6832/830097474,v1:172.21.15.203:6833/830097474] conn(0x7fcfb406a050 0x7fcfb406c510 unknown :-1 s=CLOSED pgs=225 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:35:46.304 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:46.300+0000 7fcfd600f640 1 --2- 172.21.15.43:0/88817585 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fcfd00b0f00 0x7fcfd00b2470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:35:46.304 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:46.300+0000 7fcfd600f640 1 -- 172.21.15.43:0/88817585 >> 172.21.15.43:0/88817585 conn(0x7fcfd006a210 msgr2=0x7fcfd00d09d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-09T13:35:46.304 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:46.300+0000 7fcfd600f640 1 -- 172.21.15.43:0/88817585 shutdown_connections 2023-08-09T13:35:46.304 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:46.300+0000 7fcfd600f640 1 -- 172.21.15.43:0/88817585 wait complete. 2023-08-09T13:35:46.322 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2023-08-09T13:35:46.554 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:46.551+0000 7f1a2c974640 1 Processor -- start 2023-08-09T13:35:46.554 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:46.552+0000 7f1a2c974640 1 -- start start 2023-08-09T13:35:46.556 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:46.553+0000 7f1a2c974640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f1a2815b460 0x7f1a2806b3d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:35:46.556 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:46.553+0000 7f1a2c974640 1 -- --> v1:172.21.15.43:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1a2806d770 con 0x7f1a28070ab0 2023-08-09T13:35:46.556 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:46.553+0000 7f1a2c974640 1 -- --> v1:172.21.15.203:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1a2806cf30 con 0x7f1a281546f0 2023-08-09T13:35:46.556 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:46.553+0000 7f1a2c974640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1a2806d1b0 con 0x7f1a2815b460 2023-08-09T13:35:46.557 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:46.554+0000 7f1a27fff640 1 --1- >> v1:172.21.15.43:6789/0 conn(0x7f1a28070ab0 0x7f1a280c84b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.43:6789/0 says I am v1:172.21.15.43:48552/0 (socket says 172.21.15.43:48552) 2023-08-09T13:35:46.558 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:46.554+0000 7f1a27fff640 1 -- 172.21.15.43:0/1874676706 learned_addr learned my addr 172.21.15.43:0/1874676706 (peer_addr_for_me v1:172.21.15.43:0/0) 2023-08-09T13:35:46.558 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:46.554+0000 7f1a1ffff640 1 --2- 172.21.15.43:0/1874676706 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f1a2815b460 0x7f1a2806b3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:35:46.558 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:46.554+0000 7f1a26ffd640 1 -- 172.21.15.43:0/1874676706 <== mon.0 v1:172.21.15.43:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2179381568 0 0) 0x7f1a20001490 con 0x7f1a28070ab0 2023-08-09T13:35:46.558 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:46.554+0000 7f1a26ffd640 1 -- 172.21.15.43:0/1874676706 --> v1:172.21.15.43:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f1a14003520 con 0x7f1a28070ab0 2023-08-09T13:35:46.558 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:46.555+0000 7f1a26ffd640 1 -- 172.21.15.43:0/1874676706 <== mon.1 v1:172.21.15.203:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4264263128 0 0) 0x7f1a2806cf30 con 0x7f1a281546f0 2023-08-09T13:35:46.558 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:46.555+0000 7f1a26ffd640 1 -- 172.21.15.43:0/1874676706 --> v1:172.21.15.203:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f1a20001490 con 0x7f1a281546f0 2023-08-09T13:35:46.558 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:46.555+0000 7f1a26ffd640 1 -- 172.21.15.43:0/1874676706 <== mon.0 v1:172.21.15.43:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2637619776 0 0) 0x7f1a14003520 con 0x7f1a28070ab0 2023-08-09T13:35:46.559 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:46.555+0000 7f1a26ffd640 1 -- 172.21.15.43:0/1874676706 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f1a2815b460 msgr2=0x7f1a2806b3d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-09T13:35:46.559 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:46.555+0000 7f1a26ffd640 1 --2- 172.21.15.43:0/1874676706 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f1a2815b460 0x7f1a2806b3d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:35:46.559 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:46.555+0000 7f1a26ffd640 1 -- 172.21.15.43:0/1874676706 >> v1:172.21.15.203:6789/0 conn(0x7f1a281546f0 legacy=0x7f1a2815e640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:35:46.559 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:46.555+0000 7f1a26ffd640 1 -- 172.21.15.43:0/1874676706 --> v1:172.21.15.43:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1a2806d490 con 0x7f1a28070ab0 2023-08-09T13:35:46.559 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:46.555+0000 7f1a26ffd640 1 -- 172.21.15.43:0/1874676706 <== mon.0 v1:172.21.15.43:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1285150740 0 0) 0x7f1a20001b80 con 0x7f1a28070ab0 2023-08-09T13:35:46.559 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:46.555+0000 7f1a26ffd640 1 -- 172.21.15.43:0/1874676706 >> v1:172.21.15.43:6789/0 conn(0x7f1a28070ab0 legacy=0x7f1a280c84b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:35:46.559 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:46.555+0000 7f1a26ffd640 1 --2- 172.21.15.43:0/1874676706 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f1a14003b30 0x7f1a14003f10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:35:46.559 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:46.555+0000 7f1a26ffd640 1 --2- 172.21.15.43:0/1874676706 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f1a140047c0 0x7f1a14004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:35:46.560 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:46.555+0000 7f1a26ffd640 1 --2- 172.21.15.43:0/1874676706 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f1a14005180 0x7f1a1400b980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:35:46.560 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:46.555+0000 7f1a26ffd640 1 -- 172.21.15.43:0/1874676706 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1a2806d1b0 con 0x7f1a140047c0 2023-08-09T13:35:46.560 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:46.555+0000 7f1a26ffd640 1 -- 172.21.15.43:0/1874676706 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1a1400bf70 con 0x7f1a14005180 2023-08-09T13:35:46.560 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:46.555+0000 7f1a26ffd640 1 -- 172.21.15.43:0/1874676706 --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1a1400c100 con 0x7f1a14003b30 2023-08-09T13:35:46.561 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:46.555+0000 7f1a1ffff640 1 --2- 172.21.15.43:0/1874676706 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f1a140047c0 0x7f1a14004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:35:46.562 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:46.556+0000 7f1a27fff640 1 --2- 172.21.15.43:0/1874676706 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f1a14005180 0x7f1a1400b980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:35:46.562 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:46.556+0000 7f1a277fe640 1 --2- 172.21.15.43:0/1874676706 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f1a14003b30 0x7f1a14003f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:35:46.562 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:46.556+0000 7f1a1ffff640 1 -- 172.21.15.43:0/1874676706 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f1a14003b30 msgr2=0x7f1a14003f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-09T13:35:46.562 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:46.556+0000 7f1a1ffff640 1 --2- 172.21.15.43:0/1874676706 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f1a14003b30 0x7f1a14003f10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:35:46.562 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:46.556+0000 7f1a1ffff640 1 -- 172.21.15.43:0/1874676706 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f1a14005180 msgr2=0x7f1a1400b980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-09T13:35:46.562 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:46.556+0000 7f1a1ffff640 1 --2- 172.21.15.43:0/1874676706 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f1a14005180 0x7f1a1400b980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:35:46.562 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:46.556+0000 7f1a1ffff640 1 -- 172.21.15.43:0/1874676706 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1a14003520 con 0x7f1a140047c0 2023-08-09T13:35:46.562 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:46.557+0000 7f1a1ffff640 1 --2- 172.21.15.43:0/1874676706 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f1a140047c0 0x7f1a14004c40 secure :-1 s=READY pgs=454 cs=0 l=1 rev1=1 crypto rx=0x7f1a10009af0 tx=0x7f1a140057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=49ff3565fc19f3fc server_cookie=0 in_seq=0 out_seq=0 2023-08-09T13:35:46.563 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:46.558+0000 7f1a26ffd640 1 -- 172.21.15.43:0/1874676706 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1a1002c6b0 con 0x7f1a140047c0 2023-08-09T13:35:46.563 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:46.558+0000 7f1a26ffd640 1 -- 172.21.15.43:0/1874676706 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1a1002c810 con 0x7f1a140047c0 2023-08-09T13:35:46.563 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:46.558+0000 7f1a26ffd640 1 -- 172.21.15.43:0/1874676706 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1a1002cba0 con 0x7f1a140047c0 2023-08-09T13:35:46.563 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:46.558+0000 7f1a2c974640 1 -- 172.21.15.43:0/1874676706 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f1a140047c0 msgr2=0x7f1a14004c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:35:46.563 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:46.558+0000 7f1a2c974640 1 --2- 172.21.15.43:0/1874676706 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f1a140047c0 0x7f1a14004c40 secure :-1 s=READY pgs=454 cs=0 l=1 rev1=1 crypto rx=0x7f1a10009af0 tx=0x7f1a140057f0 comp rx=0 tx=0).stop 2023-08-09T13:35:46.563 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:46.558+0000 7f1a27fff640 1 -- 172.21.15.43:0/1874676706 reap_dead start 2023-08-09T13:35:46.563 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:46.558+0000 7f1a2c974640 1 -- 172.21.15.43:0/1874676706 shutdown_connections 2023-08-09T13:35:46.564 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:46.558+0000 7f1a2c974640 1 -- 172.21.15.43:0/1874676706 >> 172.21.15.43:0/1874676706 conn(0x7f1a280af950 msgr2=0x7f1a280afd60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-09T13:35:46.564 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:46.558+0000 7f1a2c974640 1 -- 172.21.15.43:0/1874676706 shutdown_connections 2023-08-09T13:35:46.564 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:46.558+0000 7f1a2c974640 1 -- 172.21.15.43:0/1874676706 wait complete. 2023-08-09T13:35:46.565 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:46.560+0000 7f1a2c974640 1 Processor -- start 2023-08-09T13:35:46.565 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:46.561+0000 7f1a2c974640 1 -- start start 2023-08-09T13:35:46.565 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:46.562+0000 7f1a2c974640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f1a140047c0 0x7f1a28153a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:35:46.565 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:46.562+0000 7f1a2c974640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f1a2814e7e0 0x7f1a28151b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:35:46.565 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:46.562+0000 7f1a2c974640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f1a280c36f0 0x7f1a280c3b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:35:46.566 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:46.562+0000 7f1a2c974640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1a280c40b0 con 0x7f1a2814e7e0 2023-08-09T13:35:46.566 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:46.562+0000 7f1a2c974640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1a28152080 con 0x7f1a280c36f0 2023-08-09T13:35:46.566 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:46.562+0000 7f1a2c974640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1a28152350 con 0x7f1a140047c0 2023-08-09T13:35:46.566 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:46.562+0000 7f1a1ffff640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f1a2814e7e0 0x7f1a28151b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:35:46.566 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:46.562+0000 7f1a1ffff640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f1a2814e7e0 0x7f1a28151b40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:35512/0 (socket says 172.21.15.43:35512) 2023-08-09T13:35:46.566 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:46.562+0000 7f1a1ffff640 1 -- 172.21.15.43:0/2358503231 learned_addr learned my addr 172.21.15.43:0/2358503231 (peer_addr_for_me v2:172.21.15.43:0/0) 2023-08-09T13:35:46.566 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:46.562+0000 7f1a1ffff640 1 -- 172.21.15.43:0/2358503231 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f1a140047c0 msgr2=0x7f1a28153a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:35:46.566 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:46.562+0000 7f1a27fff640 1 --2- 172.21.15.43:0/2358503231 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f1a280c36f0 0x7f1a280c3b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:35:46.567 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:46.562+0000 7f1a1ffff640 1 --2- 172.21.15.43:0/2358503231 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f1a140047c0 0x7f1a28153a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:35:46.567 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:46.562+0000 7f1a1ffff640 1 -- 172.21.15.43:0/2358503231 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f1a280c36f0 msgr2=0x7f1a280c3b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:35:46.567 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:46.562+0000 7f1a1ffff640 1 --2- 172.21.15.43:0/2358503231 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f1a280c36f0 0x7f1a280c3b70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:35:46.567 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:46.562+0000 7f1a1ffff640 1 -- 172.21.15.43:0/2358503231 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1a28152630 con 0x7f1a2814e7e0 2023-08-09T13:35:46.568 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:46.563+0000 7f1a1ffff640 1 --2- 172.21.15.43:0/2358503231 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f1a2814e7e0 0x7f1a28151b40 secure :-1 s=READY pgs=455 cs=0 l=1 rev1=1 crypto rx=0x7f1a10037040 tx=0x7f1a10004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-09T13:35:46.568 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:46.563+0000 7f1a24ff9640 1 -- 172.21.15.43:0/2358503231 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1a10003810 con 0x7f1a2814e7e0 2023-08-09T13:35:46.568 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:46.563+0000 7f1a24ff9640 1 -- 172.21.15.43:0/2358503231 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1a10003970 con 0x7f1a2814e7e0 2023-08-09T13:35:46.568 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:46.563+0000 7f1a24ff9640 1 -- 172.21.15.43:0/2358503231 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1a10003ce0 con 0x7f1a2814e7e0 2023-08-09T13:35:46.568 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:46.564+0000 7f1a2c974640 1 -- 172.21.15.43:0/2358503231 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1a28152910 con 0x7f1a2814e7e0 2023-08-09T13:35:46.568 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:46.564+0000 7f1a2c974640 1 -- 172.21.15.43:0/2358503231 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1a28152bf0 con 0x7f1a2814e7e0 2023-08-09T13:35:46.568 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:46.565+0000 7f1a2c974640 1 -- 172.21.15.43:0/2358503231 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1a280739f0 con 0x7f1a2814e7e0 2023-08-09T13:35:46.569 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:46.566+0000 7f1a24ff9640 1 -- 172.21.15.43:0/2358503231 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 5) v1 ==== 88341+0+0 (secure 0 0 0) 0x7f1a10047450 con 0x7f1a2814e7e0 2023-08-09T13:35:46.569 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:46.567+0000 7f1a24ff9640 1 --2- 172.21.15.43:0/2358503231 >> [v2:172.21.15.203:6832/830097474,v1:172.21.15.203:6833/830097474] conn(0x7f1a0806a0d0 0x7f1a0806c590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:35:46.569 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:46.567+0000 7f1a24ff9640 1 -- 172.21.15.43:0/2358503231 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 6871+0+0 (secure 0 0 0) 0x7f1a100aaaa0 con 0x7f1a2814e7e0 2023-08-09T13:35:46.570 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:46.568+0000 7f1a277fe640 1 --2- 172.21.15.43:0/2358503231 >> [v2:172.21.15.203:6832/830097474,v1:172.21.15.203:6833/830097474] conn(0x7f1a0806a0d0 0x7f1a0806c590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:35:46.571 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:46.569+0000 7f1a277fe640 1 --2- 172.21.15.43:0/2358503231 >> [v2:172.21.15.203:6832/830097474,v1:172.21.15.203:6833/830097474] conn(0x7f1a0806a0d0 0x7f1a0806c590 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7f1a18000f60 tx=0x7f1a18009040 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-09T13:35:46.574 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:46.572+0000 7f1a24ff9640 1 -- 172.21.15.43:0/2358503231 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178066 (secure 0 0 0) 0x7f1a10075ef0 con 0x7f1a2814e7e0 2023-08-09T13:35:46.782 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:46.780+0000 7f1a2c974640 1 -- 172.21.15.43:0/2358503231 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7f1a280c9f80 con 0x7f1a2814e7e0 2023-08-09T13:35:46.783 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:46.781+0000 7f1a24ff9640 1 -- 172.21.15.43:0/2358503231 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v26) v1 ==== 80+0+0 (secure 0 0 0) 0x7f1a100044e0 con 0x7f1a2814e7e0 2023-08-09T13:35:46.783 INFO:teuthology.orchestra.run.smithi043.stderr:listed 0 entries 2023-08-09T13:35:46.787 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:46.784+0000 7f1a2c974640 1 -- 172.21.15.43:0/2358503231 >> [v2:172.21.15.203:6832/830097474,v1:172.21.15.203:6833/830097474] conn(0x7f1a0806a0d0 msgr2=0x7f1a0806c590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:35:46.787 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:46.784+0000 7f1a2c974640 1 --2- 172.21.15.43:0/2358503231 >> [v2:172.21.15.203:6832/830097474,v1:172.21.15.203:6833/830097474] conn(0x7f1a0806a0d0 0x7f1a0806c590 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7f1a18000f60 tx=0x7f1a18009040 comp rx=0 tx=0).stop 2023-08-09T13:35:46.787 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:46.784+0000 7f1a2c974640 1 -- 172.21.15.43:0/2358503231 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f1a2814e7e0 msgr2=0x7f1a28151b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:35:46.787 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:46.784+0000 7f1a2c974640 1 --2- 172.21.15.43:0/2358503231 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f1a2814e7e0 0x7f1a28151b40 secure :-1 s=READY pgs=455 cs=0 l=1 rev1=1 crypto rx=0x7f1a10037040 tx=0x7f1a10004730 comp rx=0 tx=0).stop 2023-08-09T13:35:46.787 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:46.784+0000 7f1a2c974640 1 -- 172.21.15.43:0/2358503231 shutdown_connections 2023-08-09T13:35:46.787 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:46.784+0000 7f1a2c974640 1 --2- 172.21.15.43:0/2358503231 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f1a280c36f0 0x7f1a280c3b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:35:46.788 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:46.784+0000 7f1a2c974640 1 --2- 172.21.15.43:0/2358503231 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f1a2814e7e0 0x7f1a28151b40 unknown :-1 s=CLOSED pgs=455 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:35:46.788 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:46.784+0000 7f1a2c974640 1 --2- 172.21.15.43:0/2358503231 >> [v2:172.21.15.203:6832/830097474,v1:172.21.15.203:6833/830097474] conn(0x7f1a0806a0d0 0x7f1a0806c590 unknown :-1 s=CLOSED pgs=226 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:35:46.788 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:46.784+0000 7f1a2c974640 1 --2- 172.21.15.43:0/2358503231 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f1a140047c0 0x7f1a28153a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:35:46.788 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:46.784+0000 7f1a2c974640 1 -- 172.21.15.43:0/2358503231 >> 172.21.15.43:0/2358503231 conn(0x7f1a280af950 msgr2=0x7f1a28067560 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-09T13:35:46.788 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:46.784+0000 7f1a2c974640 1 -- 172.21.15.43:0/2358503231 shutdown_connections 2023-08-09T13:35:46.788 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:46.784+0000 7f1a2c974640 1 -- 172.21.15.43:0/2358503231 wait complete. 2023-08-09T13:35:46.807 DEBUG:teuthology.orchestra.run.smithi203:> set -ex 2023-08-09T13:35:46.808 DEBUG:teuthology.orchestra.run.smithi203:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-09T13:35:46.869 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-08-09T13:35:47.103 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:47.100+0000 7f63fd825640 1 Processor -- start 2023-08-09T13:35:47.103 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:47.101+0000 7f63fd825640 1 -- start start 2023-08-09T13:35:47.105 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:47.102+0000 7f63fd825640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f63f815a350 0x7f63f80cfcf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:35:47.105 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:47.102+0000 7f63fd825640 1 -- --> v1:172.21.15.43:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f63f80d06e0 con 0x7f63f80cb3d0 2023-08-09T13:35:47.105 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:47.102+0000 7f63fd825640 1 -- --> v1:172.21.15.203:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f63f80cb9f0 con 0x7f63f80d0230 2023-08-09T13:35:47.105 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:47.102+0000 7f63fd825640 1 -- --> v2:172.21.15.43:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f63f80cbfb0 con 0x7f63f815a350 2023-08-09T13:35:47.106 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:47.103+0000 7f63fc823640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f63f815a350 0x7f63f80cfcf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:35:47.107 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:47.103+0000 7f63fc823640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f63f815a350 0x7f63f80cfcf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:35516/0 (socket says 172.21.15.43:35516) 2023-08-09T13:35:47.107 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:47.103+0000 7f63fc823640 1 -- 172.21.15.43:0/3736478833 learned_addr learned my addr 172.21.15.43:0/3736478833 (peer_addr_for_me v2:172.21.15.43:0/0) 2023-08-09T13:35:47.107 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:47.103+0000 7f63fc823640 1 -- 172.21.15.43:0/3736478833 >> v1:172.21.15.203:6789/0 conn(0x7f63f80d0230 legacy=0x7f63f80cacc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-09T13:35:47.108 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:47.103+0000 7f63fc823640 1 -- 172.21.15.43:0/3736478833 >> v1:172.21.15.43:6789/0 conn(0x7f63f80cb3d0 legacy=0x7f63f80b06c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-09T13:35:47.108 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:47.103+0000 7f63fc823640 1 -- 172.21.15.43:0/3736478833 --> v2:172.21.15.43:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f63f80cbcd0 con 0x7f63f815a350 2023-08-09T13:35:47.110 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:47.104+0000 7f63fc823640 1 --2- 172.21.15.43:0/3736478833 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f63f815a350 0x7f63f80cfcf0 secure :-1 s=READY pgs=457 cs=0 l=1 rev1=1 crypto rx=0x7f63ec009850 tx=0x7f63ec02b8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=57a3e6a0b4666e09 server_cookie=0 in_seq=0 out_seq=0 2023-08-09T13:35:47.110 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:47.105+0000 7f63f77fe640 1 -- 172.21.15.43:0/3736478833 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f63ec004230 con 0x7f63f815a350 2023-08-09T13:35:47.111 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:47.105+0000 7f63f77fe640 1 -- 172.21.15.43:0/3736478833 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f63ec004390 con 0x7f63f815a350 2023-08-09T13:35:47.111 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:47.105+0000 7f63f77fe640 1 -- 172.21.15.43:0/3736478833 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f63ec003810 con 0x7f63f815a350 2023-08-09T13:35:47.111 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:47.105+0000 7f63fd825640 1 -- 172.21.15.43:0/3736478833 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f63f815a350 msgr2=0x7f63f80cfcf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:35:47.111 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:47.105+0000 7f63fd825640 1 --2- 172.21.15.43:0/3736478833 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f63f815a350 0x7f63f80cfcf0 secure :-1 s=READY pgs=457 cs=0 l=1 rev1=1 crypto rx=0x7f63ec009850 tx=0x7f63ec02b8f0 comp rx=0 tx=0).stop 2023-08-09T13:35:47.112 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:47.106+0000 7f63fd825640 1 -- 172.21.15.43:0/3736478833 shutdown_connections 2023-08-09T13:35:47.112 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:47.106+0000 7f63fd825640 1 --2- 172.21.15.43:0/3736478833 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f63f815a350 0x7f63f80cfcf0 unknown :-1 s=CLOSED pgs=457 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:35:47.112 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:47.106+0000 7f63fd825640 1 -- 172.21.15.43:0/3736478833 >> 172.21.15.43:0/3736478833 conn(0x7f63f8089fe0 msgr2=0x7f63f8099450 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-09T13:35:47.112 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:47.106+0000 7f63fd825640 1 -- 172.21.15.43:0/3736478833 shutdown_connections 2023-08-09T13:35:47.113 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:47.106+0000 7f63fd825640 1 -- 172.21.15.43:0/3736478833 wait complete. 2023-08-09T13:35:47.113 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:47.107+0000 7f63fd825640 1 Processor -- start 2023-08-09T13:35:47.113 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:47.108+0000 7f63fd825640 1 -- start start 2023-08-09T13:35:47.114 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:47.108+0000 7f63fd825640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f63f80cb3d0 0x7f63f815a9d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:35:47.114 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:47.109+0000 7f63fd825640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f63f80d0230 0x7f63f81f5760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:35:47.115 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:47.109+0000 7f63fd825640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f63f815a350 0x7f63f80c9990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:35:47.115 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:47.109+0000 7f63fd825640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f63f81f5ca0 con 0x7f63f815a350 2023-08-09T13:35:47.115 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:47.109+0000 7f63fd825640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f63f80c9ed0 con 0x7f63f80d0230 2023-08-09T13:35:47.115 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:47.109+0000 7f63fd825640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f63f815b000 con 0x7f63f80cb3d0 2023-08-09T13:35:47.115 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:47.109+0000 7f63fd024640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f63f815a350 0x7f63f80c9990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:35:47.115 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:47.109+0000 7f63fd024640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f63f815a350 0x7f63f80c9990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:35532/0 (socket says 172.21.15.43:35532) 2023-08-09T13:35:47.115 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:47.109+0000 7f63fd024640 1 -- 172.21.15.43:0/3042174483 learned_addr learned my addr 172.21.15.43:0/3042174483 (peer_addr_for_me v2:172.21.15.43:0/0) 2023-08-09T13:35:47.116 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:47.109+0000 7f63f7fff640 1 --2- 172.21.15.43:0/3042174483 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f63f80d0230 0x7f63f81f5760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:35:47.116 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:47.109+0000 7f63fc823640 1 --2- 172.21.15.43:0/3042174483 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f63f80cb3d0 0x7f63f815a9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:35:47.116 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:47.109+0000 7f63fd024640 1 -- 172.21.15.43:0/3042174483 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f63f80cb3d0 msgr2=0x7f63f815a9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:35:47.116 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:47.109+0000 7f63fd024640 1 --2- 172.21.15.43:0/3042174483 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f63f80cb3d0 0x7f63f815a9d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:35:47.116 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:47.109+0000 7f63fd024640 1 -- 172.21.15.43:0/3042174483 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f63f80d0230 msgr2=0x7f63f81f5760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:35:47.116 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:47.109+0000 7f63fd024640 1 --2- 172.21.15.43:0/3042174483 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f63f80d0230 0x7f63f81f5760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:35:47.116 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:47.109+0000 7f63fd024640 1 -- 172.21.15.43:0/3042174483 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f63ec009460 con 0x7f63f815a350 2023-08-09T13:35:47.117 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:47.109+0000 7f63fc823640 1 --2- 172.21.15.43:0/3042174483 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f63f80cb3d0 0x7f63f815a9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-08-09T13:35:47.117 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:47.110+0000 7f63fd024640 1 --2- 172.21.15.43:0/3042174483 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f63f815a350 0x7f63f80c9990 secure :-1 s=READY pgs=458 cs=0 l=1 rev1=1 crypto rx=0x7f63f000c6b0 tx=0x7f63f000cb70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-09T13:35:47.117 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:47.110+0000 7f63f57fa640 1 -- 172.21.15.43:0/3042174483 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f63f0003b70 con 0x7f63f815a350 2023-08-09T13:35:47.117 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:47.110+0000 7f63fd825640 1 -- 172.21.15.43:0/3042174483 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f63f815b2e0 con 0x7f63f815a350 2023-08-09T13:35:47.117 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:47.110+0000 7f63f57fa640 1 -- 172.21.15.43:0/3042174483 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f63f0003cf0 con 0x7f63f815a350 2023-08-09T13:35:47.117 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:47.110+0000 7f63f57fa640 1 -- 172.21.15.43:0/3042174483 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f63f0003ea0 con 0x7f63f815a350 2023-08-09T13:35:47.118 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:47.110+0000 7f63fd825640 1 -- 172.21.15.43:0/3042174483 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f63f815b8a0 con 0x7f63f815a350 2023-08-09T13:35:47.118 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:47.112+0000 7f63fd825640 1 -- 172.21.15.43:0/3042174483 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f63f80739f0 con 0x7f63f815a350 2023-08-09T13:35:47.118 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:47.112+0000 7f63f57fa640 1 -- 172.21.15.43:0/3042174483 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 5) v1 ==== 88341+0+0 (secure 0 0 0) 0x7f63f0005310 con 0x7f63f815a350 2023-08-09T13:35:47.119 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:47.113+0000 7f63f57fa640 1 --2- 172.21.15.43:0/3042174483 >> [v2:172.21.15.203:6832/830097474,v1:172.21.15.203:6833/830097474] conn(0x7f63dc06a050 0x7f63dc06c510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:35:47.119 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:47.114+0000 7f63f57fa640 1 -- 172.21.15.43:0/3042174483 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 6871+0+0 (secure 0 0 0) 0x7f63f0088680 con 0x7f63f815a350 2023-08-09T13:35:47.119 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:47.114+0000 7f63fc823640 1 --2- 172.21.15.43:0/3042174483 >> [v2:172.21.15.203:6832/830097474,v1:172.21.15.203:6833/830097474] conn(0x7f63dc06a050 0x7f63dc06c510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:35:47.119 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:47.115+0000 7f63fc823640 1 --2- 172.21.15.43:0/3042174483 >> [v2:172.21.15.203:6832/830097474,v1:172.21.15.203:6833/830097474] conn(0x7f63dc06a050 0x7f63dc06c510 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7f63ec035000 tx=0x7f63ec008030 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-09T13:35:47.121 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:47.119+0000 7f63f57fa640 1 -- 172.21.15.43:0/3042174483 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178066 (secure 0 0 0) 0x7f63f0053a30 con 0x7f63f815a350 2023-08-09T13:35:47.325 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:47.322+0000 7f63fd825640 1 -- 172.21.15.43:0/3042174483 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f63f80caec0 con 0x7f63f815a350 2023-08-09T13:35:47.329 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:47.326+0000 7f63f57fa640 1 -- 172.21.15.43:0/3042174483 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v26) v1 ==== 74+0+20146 (secure 0 0 0) 0x7f63f0057050 con 0x7f63f815a350 2023-08-09T13:35:47.329 INFO:teuthology.orchestra.run.smithi043.stdout: 2023-08-09T13:35:47.330 INFO:teuthology.orchestra.run.smithi043.stdout:{"epoch":26,"fsid":"daac87ee-1337-4250-a955-aea4e408a39b","created":"2023-08-09T13:33:41.701198+0000","modified":"2023-08-09T13:34:48.711911+0000","last_up_change":"2023-08-09T13:33:53.783453+0000","last_in_change":"2023-08-09T13:33:45.148904+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":5,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":"rbd","create_time":"2023-08-09T13:33:54.365137+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":".mgr","create_time":"2023-08-09T13:33:54.831803+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_metadata","create_time":"2023-08-09T13:34:00.207974+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data","create_time":"2023-08-09T13:34:01.614765+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":5,"pool_name":"cephfs_data_ec","create_time":"2023-08-09T13:34:05.239583+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"24","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"cbd49ecd-e896-4fbd-864e-5c37a829b165","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6800","nonce":502667335},{"type":"v1","addr":"172.21.15.43:6801","nonce":502667335}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6802","nonce":502667335},{"type":"v1","addr":"172.21.15.43:6803","nonce":502667335}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6806","nonce":502667335},{"type":"v1","addr":"172.21.15.43:6807","nonce":502667335}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6804","nonce":502667335},{"type":"v1","addr":"172.21.15.43:6805","nonce":502667335}]},"public_addr":"172.21.15.43:6801/502667335","cluster_addr":"172.21.15.43:6803/502667335","heartbeat_back_addr":"172.21.15.43:6807/502667335","heartbeat_front_addr":"172.21.15.43:6805/502667335","state":["exists","up"]},{"osd":1,"uuid":"75c7e0fe-1ec7-4fe8-a2f2-3661a65b8500","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6808","nonce":3289134610},{"type":"v1","addr":"172.21.15.43:6809","nonce":3289134610}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6810","nonce":3289134610},{"type":"v1","addr":"172.21.15.43:6812","nonce":3289134610}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6821","nonce":3289134610},{"type":"v1","addr":"172.21.15.43:6824","nonce":3289134610}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6815","nonce":3289134610},{"type":"v1","addr":"172.21.15.43:6818","nonce":3289134610}]},"public_addr":"172.21.15.43:6809/3289134610","cluster_addr":"172.21.15.43:6812/3289134610","heartbeat_back_addr":"172.21.15.43:6824/3289134610","heartbeat_front_addr":"172.21.15.43:6818/3289134610","state":["exists","up"]},{"osd":2,"uuid":"bb5c41e9-7f5f-4125-9b01-3bb8bfc2ea83","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6811","nonce":4079709736},{"type":"v1","addr":"172.21.15.43:6814","nonce":4079709736}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6817","nonce":4079709736},{"type":"v1","addr":"172.21.15.43:6820","nonce":4079709736}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6829","nonce":4079709736},{"type":"v1","addr":"172.21.15.43:6831","nonce":4079709736}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6823","nonce":4079709736},{"type":"v1","addr":"172.21.15.43:6826","nonce":4079709736}]},"public_addr":"172.21.15.43:6814/4079709736","cluster_addr":"172.21.15.43:6820/4079709736","heartbeat_back_addr":"172.21.15.43:6831/4079709736","heartbeat_front_addr":"172.21.15.43:6826/4079709736","state":["exists","up"]},{"osd":3,"uuid":"bfe4c8b4-694a-465c-87e6-2b2aab852613","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6813","nonce":1709538759},{"type":"v1","addr":"172.21.15.43:6816","nonce":1709538759}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6819","nonce":1709538759},{"type":"v1","addr":"172.21.15.43:6822","nonce":1709538759}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6828","nonce":1709538759},{"type":"v1","addr":"172.21.15.43:6830","nonce":1709538759}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6825","nonce":1709538759},{"type":"v1","addr":"172.21.15.43:6827","nonce":1709538759}]},"public_addr":"172.21.15.43:6816/1709538759","cluster_addr":"172.21.15.43:6822/1709538759","heartbeat_back_addr":"172.21.15.43:6830/1709538759","heartbeat_front_addr":"172.21.15.43:6827/1709538759","state":["exists","up"]},{"osd":4,"uuid":"fa2e8a77-aba1-4756-853e-0d174a5bdeba","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6800","nonce":4196866481},{"type":"v1","addr":"172.21.15.203:6801","nonce":4196866481}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6802","nonce":4196866481},{"type":"v1","addr":"172.21.15.203:6803","nonce":4196866481}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6806","nonce":4196866481},{"type":"v1","addr":"172.21.15.203:6807","nonce":4196866481}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6804","nonce":4196866481},{"type":"v1","addr":"172.21.15.203:6805","nonce":4196866481}]},"public_addr":"172.21.15.203:6801/4196866481","cluster_addr":"172.21.15.203:6803/4196866481","heartbeat_back_addr":"172.21.15.203:6807/4196866481","heartbeat_front_addr":"172.21.15.203:6805/4196866481","state":["exists","up"]},{"osd":5,"uuid":"81811e4b-4d24-4f51-a82c-f1d700a2a00d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6816","nonce":3059290081},{"type":"v1","addr":"172.21.15.203:6817","nonce":3059290081}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6818","nonce":3059290081},{"type":"v1","addr":"172.21.15.203:6819","nonce":3059290081}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6822","nonce":3059290081},{"type":"v1","addr":"172.21.15.203:6823","nonce":3059290081}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6820","nonce":3059290081},{"type":"v1","addr":"172.21.15.203:6821","nonce":3059290081}]},"public_addr":"172.21.15.203:6817/3059290081","cluster_addr":"172.21.15.203:6819/3059290081","heartbeat_back_addr":"172.21.15.203:6823/3059290081","heartbeat_front_addr":"172.21.15.203:6821/3059290081","state":["exists","up"]},{"osd":6,"uuid":"307bac4d-6a5b-443d-af75-56d695d7674f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6824","nonce":512155569},{"type":"v1","addr":"172.21.15.203:6825","nonce":512155569}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6826","nonce":512155569},{"type":"v1","addr":"172.21.15.203:6827","nonce":512155569}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6830","nonce":512155569},{"type":"v1","addr":"172.21.15.203:6831","nonce":512155569}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6828","nonce":512155569},{"type":"v1","addr":"172.21.15.203:6829","nonce":512155569}]},"public_addr":"172.21.15.203:6825/512155569","cluster_addr":"172.21.15.203:6827/512155569","heartbeat_back_addr":"172.21.15.203:6831/512155569","heartbeat_front_addr":"172.21.15.203:6829/512155569","state":["exists","up"]},{"osd":7,"uuid":"a95dc7ca-9f1b-4295-af70-68a22a2d7d5e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6808","nonce":3331450438},{"type":"v1","addr":"172.21.15.203:6809","nonce":3331450438}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6810","nonce":3331450438},{"type":"v1","addr":"172.21.15.203:6811","nonce":3331450438}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6814","nonce":3331450438},{"type":"v1","addr":"172.21.15.203:6815","nonce":3331450438}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6812","nonce":3331450438},{"type":"v1","addr":"172.21.15.203:6813","nonce":3331450438}]},"public_addr":"172.21.15.203:6809/3331450438","cluster_addr":"172.21.15.203:6811/3331450438","heartbeat_back_addr":"172.21.15.203:6815/3331450438","heartbeat_front_addr":"172.21.15.203:6813/3331450438","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-09T13:33:50.547652+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-09T13:33:50.589469+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-09T13:33:50.561724+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-09T13:33:50.599650+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-09T13:33:50.411855+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-09T13:33:50.219681+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-09T13:33:50.504664+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-09T13:33:50.441248+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"4.14","mappings":[{"from":1,"to":5}]},{"pgid":"4.29","mappings":[{"from":3,"to":2}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-08-09T13:35:47.332 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:47.329+0000 7f63fd825640 1 -- 172.21.15.43:0/3042174483 >> [v2:172.21.15.203:6832/830097474,v1:172.21.15.203:6833/830097474] conn(0x7f63dc06a050 msgr2=0x7f63dc06c510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:35:47.332 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:47.329+0000 7f63fd825640 1 --2- 172.21.15.43:0/3042174483 >> [v2:172.21.15.203:6832/830097474,v1:172.21.15.203:6833/830097474] conn(0x7f63dc06a050 0x7f63dc06c510 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7f63ec035000 tx=0x7f63ec008030 comp rx=0 tx=0).stop 2023-08-09T13:35:47.332 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:47.329+0000 7f63fd825640 1 -- 172.21.15.43:0/3042174483 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f63f815a350 msgr2=0x7f63f80c9990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:35:47.332 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:47.329+0000 7f63fd825640 1 --2- 172.21.15.43:0/3042174483 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f63f815a350 0x7f63f80c9990 secure :-1 s=READY pgs=458 cs=0 l=1 rev1=1 crypto rx=0x7f63f000c6b0 tx=0x7f63f000cb70 comp rx=0 tx=0).stop 2023-08-09T13:35:47.333 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:47.329+0000 7f63fd825640 1 -- 172.21.15.43:0/3042174483 shutdown_connections 2023-08-09T13:35:47.333 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:47.329+0000 7f63fd825640 1 --2- 172.21.15.43:0/3042174483 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f63f815a350 0x7f63f80c9990 unknown :-1 s=CLOSED pgs=458 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:35:47.333 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:47.329+0000 7f63fd825640 1 --2- 172.21.15.43:0/3042174483 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f63f80d0230 0x7f63f81f5760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:35:47.333 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:47.329+0000 7f63fd825640 1 --2- 172.21.15.43:0/3042174483 >> [v2:172.21.15.203:6832/830097474,v1:172.21.15.203:6833/830097474] conn(0x7f63dc06a050 0x7f63dc06c510 unknown :-1 s=CLOSED pgs=227 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:35:47.333 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:47.329+0000 7f63fd825640 1 --2- 172.21.15.43:0/3042174483 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f63f80cb3d0 0x7f63f815a9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:35:47.333 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:47.329+0000 7f63fd825640 1 -- 172.21.15.43:0/3042174483 >> 172.21.15.43:0/3042174483 conn(0x7f63f8089fe0 msgr2=0x7f63f80cc3b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-09T13:35:47.333 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:47.329+0000 7f63fd825640 1 -- 172.21.15.43:0/3042174483 shutdown_connections 2023-08-09T13:35:47.334 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:47.329+0000 7f63fd825640 1 -- 172.21.15.43:0/3042174483 wait complete. 2023-08-09T13:35:47.352 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': 'rbd', 'create_time': '2023-08-09T13:33:54.365137+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}, 'read_balance': {'score_acting': 2, 'score_stable': 2, 'optimal_score': 1, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': '.mgr', 'create_time': '2023-08-09T13:33:54.831803+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_metadata', 'create_time': '2023-08-09T13:34:00.207974+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data', 'create_time': '2023-08-09T13:34:01.614765+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 2, 'score_stable': 2, 'optimal_score': 1, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 5, 'pool_name': 'cephfs_data_ec', 'create_time': '2023-08-09T13:34:05.239583+0000', 'flags': 5, 'flags_names': 'hashpspool,ec_overwrites', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '24', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2023-08-09T13:35:47.353 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get rbd pg_num 2023-08-09T13:35:47.583 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:47.580+0000 7f4a066ad640 1 Processor -- start 2023-08-09T13:35:47.583 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:47.581+0000 7f4a066ad640 1 -- start start 2023-08-09T13:35:47.585 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:47.582+0000 7f4a066ad640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f4a0015cc10 0x7f4a000b1b20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:35:47.585 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:47.582+0000 7f4a066ad640 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7f4a000b2060 0x7f4a00160710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:35:47.585 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:47.582+0000 7f4a066ad640 1 -- --> v1:172.21.15.43:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4a000b24c0 con 0x7f4a00160c50 2023-08-09T13:35:47.585 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:47.582+0000 7f4a066ad640 1 -- --> v2:172.21.15.203:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4a0015db00 con 0x7f4a000b2060 2023-08-09T13:35:47.585 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:47.582+0000 7f4a066ad640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4a00161480 con 0x7f4a0015cc10 2023-08-09T13:35:47.587 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:47.583+0000 7f4a05eac640 1 --1- >> v1:172.21.15.43:6789/0 conn(0x7f4a00160c50 0x7f4a0006ef60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.43:6789/0 says I am v1:172.21.15.43:48566/0 (socket says 172.21.15.43:48566) 2023-08-09T13:35:47.587 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:47.583+0000 7f4a05eac640 1 -- 172.21.15.43:0/3742666653 learned_addr learned my addr 172.21.15.43:0/3742666653 (peer_addr_for_me v1:172.21.15.43:0/0) 2023-08-09T13:35:47.587 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:47.583+0000 7f4a04eaa640 1 --2- 172.21.15.43:0/3742666653 >> v2:172.21.15.203:3300/0 conn(0x7f4a000b2060 0x7f4a00160710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:35:47.587 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:47.583+0000 7f4a056ab640 1 --2- 172.21.15.43:0/3742666653 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f4a0015cc10 0x7f4a000b1b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:35:47.588 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:47.583+0000 7f49f7fff640 1 -- 172.21.15.43:0/3742666653 <== mon.0 v1:172.21.15.43:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4136440439 0 0) 0x7f4a000b24c0 con 0x7f4a00160c50 2023-08-09T13:35:47.588 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:47.583+0000 7f49f7fff640 1 -- 172.21.15.43:0/3742666653 --> v1:172.21.15.43:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f49ec003520 con 0x7f4a00160c50 2023-08-09T13:35:47.588 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:47.584+0000 7f49f7fff640 1 -- 172.21.15.43:0/3742666653 <== mon.0 v1:172.21.15.43:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3380513645 0 0) 0x7f49ec003520 con 0x7f4a00160c50 2023-08-09T13:35:47.588 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:47.584+0000 7f49f7fff640 1 -- 172.21.15.43:0/3742666653 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f4a0015cc10 msgr2=0x7f4a000b1b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-09T13:35:47.588 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:47.584+0000 7f49f7fff640 1 --2- 172.21.15.43:0/3742666653 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f4a0015cc10 0x7f4a000b1b20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:35:47.588 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:47.584+0000 7f49f7fff640 1 -- 172.21.15.43:0/3742666653 >> v2:172.21.15.203:3300/0 conn(0x7f4a000b2060 msgr2=0x7f4a00160710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-09T13:35:47.588 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:47.584+0000 7f49f7fff640 1 --2- 172.21.15.43:0/3742666653 >> v2:172.21.15.203:3300/0 conn(0x7f4a000b2060 0x7f4a00160710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:35:47.589 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:47.584+0000 7f49f7fff640 1 -- 172.21.15.43:0/3742666653 --> v1:172.21.15.43:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4a0015d160 con 0x7f4a00160c50 2023-08-09T13:35:47.589 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:47.584+0000 7f49f7fff640 1 -- 172.21.15.43:0/3742666653 <== mon.0 v1:172.21.15.43:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1285150740 0 0) 0x7f49f80019f0 con 0x7f4a00160c50 2023-08-09T13:35:47.589 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:47.584+0000 7f49f7fff640 1 -- 172.21.15.43:0/3742666653 >> v1:172.21.15.43:6789/0 conn(0x7f4a00160c50 legacy=0x7f4a0006ef60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:35:47.589 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:47.584+0000 7f49f7fff640 1 --2- 172.21.15.43:0/3742666653 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f49ec003970 0x7f49ec003d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:35:47.589 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:47.584+0000 7f49f7fff640 1 --2- 172.21.15.43:0/3742666653 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f4a0015cc10 0x7f49ec0047c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:35:47.589 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:47.584+0000 7f49f7fff640 1 --2- 172.21.15.43:0/3742666653 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f49ec004d00 0x7f49ec005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:35:47.589 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:47.584+0000 7f49f7fff640 1 -- 172.21.15.43:0/3742666653 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4a0015db00 con 0x7f49ec003970 2023-08-09T13:35:47.590 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:47.584+0000 7f49f7fff640 1 -- 172.21.15.43:0/3742666653 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4a00161480 con 0x7f49ec004d00 2023-08-09T13:35:47.590 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:47.584+0000 7f49f7fff640 1 -- 172.21.15.43:0/3742666653 --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f49ec00ba00 con 0x7f4a0015cc10 2023-08-09T13:35:47.590 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:47.584+0000 7f4a056ab640 1 --2- 172.21.15.43:0/3742666653 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f49ec003970 0x7f49ec003d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:35:47.590 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:47.584+0000 7f4a04eaa640 1 --2- 172.21.15.43:0/3742666653 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f4a0015cc10 0x7f49ec0047c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:35:47.590 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:47.585+0000 7f4a056ab640 1 -- 172.21.15.43:0/3742666653 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f4a0015cc10 msgr2=0x7f49ec0047c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-09T13:35:47.590 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:47.584+0000 7f4a05eac640 1 --2- 172.21.15.43:0/3742666653 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f49ec004d00 0x7f49ec005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:35:47.590 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:47.585+0000 7f4a056ab640 1 --2- 172.21.15.43:0/3742666653 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f4a0015cc10 0x7f49ec0047c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:35:47.590 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:47.585+0000 7f4a056ab640 1 -- 172.21.15.43:0/3742666653 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f49ec004d00 msgr2=0x7f49ec005180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-09T13:35:47.591 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:47.585+0000 7f4a056ab640 1 --2- 172.21.15.43:0/3742666653 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f49ec004d00 0x7f49ec005180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:35:47.591 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:47.585+0000 7f4a056ab640 1 -- 172.21.15.43:0/3742666653 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f49ec003520 con 0x7f49ec003970 2023-08-09T13:35:47.591 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:47.585+0000 7f4a05eac640 1 --2- 172.21.15.43:0/3742666653 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f49ec004d00 0x7f49ec005180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-08-09T13:35:47.592 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:47.586+0000 7f4a056ab640 1 --2- 172.21.15.43:0/3742666653 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f49ec003970 0x7f49ec003d90 secure :-1 s=READY pgs=460 cs=0 l=1 rev1=1 crypto rx=0x7f49fc009ac0 tx=0x7f49ec0057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=62256efd869f7ac server_cookie=0 in_seq=0 out_seq=0 2023-08-09T13:35:47.592 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:47.586+0000 7f49f7fff640 1 -- 172.21.15.43:0/3742666653 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f49fc02c6b0 con 0x7f49ec003970 2023-08-09T13:35:47.592 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:47.586+0000 7f49f7fff640 1 -- 172.21.15.43:0/3742666653 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f49fc02c810 con 0x7f49ec003970 2023-08-09T13:35:47.593 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:47.586+0000 7f49f7fff640 1 -- 172.21.15.43:0/3742666653 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f49fc02cba0 con 0x7f49ec003970 2023-08-09T13:35:47.593 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:47.587+0000 7f4a066ad640 1 -- 172.21.15.43:0/3742666653 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f49ec003970 msgr2=0x7f49ec003d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:35:47.593 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:47.587+0000 7f4a066ad640 1 --2- 172.21.15.43:0/3742666653 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f49ec003970 0x7f49ec003d90 secure :-1 s=READY pgs=460 cs=0 l=1 rev1=1 crypto rx=0x7f49fc009ac0 tx=0x7f49ec0057f0 comp rx=0 tx=0).stop 2023-08-09T13:35:47.593 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:47.587+0000 7f4a05eac640 1 -- 172.21.15.43:0/3742666653 reap_dead start 2023-08-09T13:35:47.593 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:47.587+0000 7f4a066ad640 1 -- 172.21.15.43:0/3742666653 shutdown_connections 2023-08-09T13:35:47.593 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:47.587+0000 7f4a066ad640 1 -- 172.21.15.43:0/3742666653 >> 172.21.15.43:0/3742666653 conn(0x7f4a000d00a0 msgr2=0x7f4a000d0480 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-09T13:35:47.593 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:47.587+0000 7f4a066ad640 1 -- 172.21.15.43:0/3742666653 shutdown_connections 2023-08-09T13:35:47.594 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:47.587+0000 7f4a066ad640 1 -- 172.21.15.43:0/3742666653 wait complete. 2023-08-09T13:35:47.594 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:47.589+0000 7f4a066ad640 1 Processor -- start 2023-08-09T13:35:47.594 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:47.589+0000 7f4a066ad640 1 -- start start 2023-08-09T13:35:47.594 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:47.590+0000 7f4a066ad640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f49ec003970 0x7f4a001f2470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:35:47.594 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:47.590+0000 7f4a066ad640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f4a001f6350 0x7f4a000b56a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:35:47.594 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:47.590+0000 7f4a066ad640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f4a000b5be0 0x7f4a0006b160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:35:47.594 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:47.590+0000 7f4a066ad640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4a001f2ca0 con 0x7f49ec003970 2023-08-09T13:35:47.594 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:47.590+0000 7f4a066ad640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4a001f2f80 con 0x7f4a000b5be0 2023-08-09T13:35:47.595 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:47.590+0000 7f4a066ad640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4a001f3260 con 0x7f4a001f6350 2023-08-09T13:35:47.596 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:47.590+0000 7f4a056ab640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f49ec003970 0x7f4a001f2470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:35:47.596 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:47.590+0000 7f4a056ab640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f49ec003970 0x7f4a001f2470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:35542/0 (socket says 172.21.15.43:35542) 2023-08-09T13:35:47.596 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:47.590+0000 7f4a056ab640 1 -- 172.21.15.43:0/1353337992 learned_addr learned my addr 172.21.15.43:0/1353337992 (peer_addr_for_me v2:172.21.15.43:0/0) 2023-08-09T13:35:47.596 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:47.590+0000 7f4a04eaa640 1 --2- 172.21.15.43:0/1353337992 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f4a001f6350 0x7f4a000b56a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:35:47.596 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:47.590+0000 7f4a05eac640 1 --2- 172.21.15.43:0/1353337992 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f4a000b5be0 0x7f4a0006b160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:35:47.597 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:47.591+0000 7f4a056ab640 1 -- 172.21.15.43:0/1353337992 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f4a001f6350 msgr2=0x7f4a000b56a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:35:47.597 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:47.591+0000 7f4a056ab640 1 --2- 172.21.15.43:0/1353337992 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f4a001f6350 0x7f4a000b56a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:35:47.597 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:47.591+0000 7f4a056ab640 1 -- 172.21.15.43:0/1353337992 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f4a000b5be0 msgr2=0x7f4a0006b160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:35:47.597 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:47.591+0000 7f4a056ab640 1 --2- 172.21.15.43:0/1353337992 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f4a000b5be0 0x7f4a0006b160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:35:47.597 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:47.591+0000 7f4a056ab640 1 -- 172.21.15.43:0/1353337992 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4a000b5180 con 0x7f49ec003970 2023-08-09T13:35:47.597 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:47.591+0000 7f4a05eac640 1 --2- 172.21.15.43:0/1353337992 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f4a000b5be0 0x7f4a0006b160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-08-09T13:35:47.597 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:47.591+0000 7f4a056ab640 1 --2- 172.21.15.43:0/1353337992 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f49ec003970 0x7f4a001f2470 secure :-1 s=READY pgs=461 cs=0 l=1 rev1=1 crypto rx=0x7f49fc02c250 tx=0x7f49fc004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-09T13:35:47.597 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:47.591+0000 7f49f5ffb640 1 -- 172.21.15.43:0/1353337992 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f49fc003810 con 0x7f49ec003970 2023-08-09T13:35:47.598 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:47.591+0000 7f49f5ffb640 1 -- 172.21.15.43:0/1353337992 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f49fc003970 con 0x7f49ec003970 2023-08-09T13:35:47.598 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:47.591+0000 7f49f5ffb640 1 -- 172.21.15.43:0/1353337992 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f49fc003ce0 con 0x7f49ec003970 2023-08-09T13:35:47.598 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:47.591+0000 7f4a066ad640 1 -- 172.21.15.43:0/1353337992 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4a001f29b0 con 0x7f49ec003970 2023-08-09T13:35:47.598 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:47.591+0000 7f4a066ad640 1 -- 172.21.15.43:0/1353337992 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4a001f67d0 con 0x7f49ec003970 2023-08-09T13:35:47.598 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:47.593+0000 7f4a066ad640 1 -- 172.21.15.43:0/1353337992 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4a000b4fe0 con 0x7f49ec003970 2023-08-09T13:35:47.598 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:47.594+0000 7f49f5ffb640 1 -- 172.21.15.43:0/1353337992 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 5) v1 ==== 88341+0+0 (secure 0 0 0) 0x7f49fc034440 con 0x7f49ec003970 2023-08-09T13:35:47.599 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:47.594+0000 7f49f5ffb640 1 --2- 172.21.15.43:0/1353337992 >> [v2:172.21.15.203:6832/830097474,v1:172.21.15.203:6833/830097474] conn(0x7f49e406a050 0x7f49e406c510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:35:47.599 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:47.595+0000 7f49f5ffb640 1 -- 172.21.15.43:0/1353337992 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 6871+0+0 (secure 0 0 0) 0x7f49fc0aac80 con 0x7f49ec003970 2023-08-09T13:35:47.599 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:47.595+0000 7f4a04eaa640 1 --2- 172.21.15.43:0/1353337992 >> [v2:172.21.15.203:6832/830097474,v1:172.21.15.203:6833/830097474] conn(0x7f49e406a050 0x7f49e406c510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:35:47.599 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:47.596+0000 7f4a04eaa640 1 --2- 172.21.15.43:0/1353337992 >> [v2:172.21.15.203:6832/830097474,v1:172.21.15.203:6833/830097474] conn(0x7f49e406a050 0x7f49e406c510 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7f4a0006bd70 tx=0x7f49f0008040 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-09T13:35:47.601 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:47.599+0000 7f49f5ffb640 1 -- 172.21.15.43:0/1353337992 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178066 (secure 0 0 0) 0x7f4a000b4fe0 con 0x7f49ec003970 2023-08-09T13:35:47.794 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:47.791+0000 7f4a066ad640 1 -- 172.21.15.43:0/1353337992 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"} v 0) v1 -- 0x7f4a0015d700 con 0x7f49ec003970 2023-08-09T13:35:47.795 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:47.792+0000 7f49f5ffb640 1 -- 172.21.15.43:0/1353337992 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"}]=0 v26) v1 ==== 92+0+10 (secure 0 0 0) 0x7f49fc04c020 con 0x7f49ec003970 2023-08-09T13:35:47.795 INFO:teuthology.orchestra.run.smithi043.stdout:pg_num: 8 2023-08-09T13:35:47.798 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:47.795+0000 7f4a066ad640 1 -- 172.21.15.43:0/1353337992 >> [v2:172.21.15.203:6832/830097474,v1:172.21.15.203:6833/830097474] conn(0x7f49e406a050 msgr2=0x7f49e406c510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:35:47.798 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:47.795+0000 7f4a066ad640 1 --2- 172.21.15.43:0/1353337992 >> [v2:172.21.15.203:6832/830097474,v1:172.21.15.203:6833/830097474] conn(0x7f49e406a050 0x7f49e406c510 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7f4a0006bd70 tx=0x7f49f0008040 comp rx=0 tx=0).stop 2023-08-09T13:35:47.798 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:47.795+0000 7f4a066ad640 1 -- 172.21.15.43:0/1353337992 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f49ec003970 msgr2=0x7f4a001f2470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:35:47.798 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:47.795+0000 7f4a066ad640 1 --2- 172.21.15.43:0/1353337992 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f49ec003970 0x7f4a001f2470 secure :-1 s=READY pgs=461 cs=0 l=1 rev1=1 crypto rx=0x7f49fc02c250 tx=0x7f49fc004730 comp rx=0 tx=0).stop 2023-08-09T13:35:47.799 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:47.796+0000 7f4a066ad640 1 -- 172.21.15.43:0/1353337992 shutdown_connections 2023-08-09T13:35:47.799 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:47.796+0000 7f4a066ad640 1 --2- 172.21.15.43:0/1353337992 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f4a000b5be0 0x7f4a0006b160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:35:47.799 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:47.796+0000 7f4a066ad640 1 --2- 172.21.15.43:0/1353337992 >> [v2:172.21.15.203:6832/830097474,v1:172.21.15.203:6833/830097474] conn(0x7f49e406a050 0x7f49e406c510 unknown :-1 s=CLOSED pgs=228 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:35:47.799 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:47.796+0000 7f4a066ad640 1 --2- 172.21.15.43:0/1353337992 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f4a001f6350 0x7f4a000b56a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:35:47.800 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:47.796+0000 7f4a066ad640 1 --2- 172.21.15.43:0/1353337992 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f49ec003970 0x7f4a001f2470 unknown :-1 s=CLOSED pgs=461 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:35:47.800 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:47.796+0000 7f4a066ad640 1 -- 172.21.15.43:0/1353337992 >> 172.21.15.43:0/1353337992 conn(0x7f4a000d00a0 msgr2=0x7f4a0015bf10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-09T13:35:47.800 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:47.796+0000 7f4a066ad640 1 -- 172.21.15.43:0/1353337992 shutdown_connections 2023-08-09T13:35:47.800 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:47.796+0000 7f4a066ad640 1 -- 172.21.15.43:0/1353337992 wait complete. 2023-08-09T13:35:47.816 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-08-09T13:35:48.052 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:48.049+0000 7f5387fff640 1 Processor -- start 2023-08-09T13:35:48.052 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:48.050+0000 7f5387fff640 1 -- start start 2023-08-09T13:35:48.054 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:48.051+0000 7f5387fff640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f5388164560 0x7f53880c8420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:35:48.054 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:48.051+0000 7f5387fff640 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7f53880c1020 0x7f5388162360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:35:48.054 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:48.051+0000 7f5387fff640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f5388164940 0x7f5388164df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:35:48.054 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:48.051+0000 7f5387fff640 1 -- --> v2:172.21.15.43:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f53880ca6d0 con 0x7f5388164940 2023-08-09T13:35:48.054 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:48.051+0000 7f5387fff640 1 -- --> v2:172.21.15.203:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f53880cc3b0 con 0x7f53880c1020 2023-08-09T13:35:48.055 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:48.051+0000 7f5387fff640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f53880af2e0 con 0x7f5388164560 2023-08-09T13:35:48.055 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:48.052+0000 7f53877fe640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f5388164940 0x7f5388164df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:35:48.055 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:48.052+0000 7f53877fe640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f5388164940 0x7f5388164df0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:35558/0 (socket says 172.21.15.43:35558) 2023-08-09T13:35:48.056 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:48.052+0000 7f53877fe640 1 -- 172.21.15.43:0/2401167894 learned_addr learned my addr 172.21.15.43:0/2401167894 (peer_addr_for_me v2:172.21.15.43:0/0) 2023-08-09T13:35:48.056 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:48.052+0000 7f5386ffd640 1 --2- 172.21.15.43:0/2401167894 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f5388164560 0x7f53880c8420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:35:48.056 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:48.052+0000 7f53867fc640 1 --2- 172.21.15.43:0/2401167894 >> v2:172.21.15.203:3300/0 conn(0x7f53880c1020 0x7f5388162360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:35:48.056 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:48.052+0000 7f53877fe640 1 -- 172.21.15.43:0/2401167894 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f5388164560 msgr2=0x7f53880c8420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-09T13:35:48.056 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:48.052+0000 7f53877fe640 1 --2- 172.21.15.43:0/2401167894 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f5388164560 0x7f53880c8420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:35:48.056 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:48.052+0000 7f53877fe640 1 -- 172.21.15.43:0/2401167894 >> v2:172.21.15.203:3300/0 conn(0x7f53880c1020 msgr2=0x7f5388162360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-09T13:35:48.056 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:48.052+0000 7f53877fe640 1 --2- 172.21.15.43:0/2401167894 >> v2:172.21.15.203:3300/0 conn(0x7f53880c1020 0x7f5388162360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:35:48.057 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:48.052+0000 7f53877fe640 1 -- 172.21.15.43:0/2401167894 --> v2:172.21.15.43:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f53880cd790 con 0x7f5388164940 2023-08-09T13:35:48.057 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:48.054+0000 7f53877fe640 1 --2- 172.21.15.43:0/2401167894 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f5388164940 0x7f5388164df0 secure :-1 s=READY pgs=462 cs=0 l=1 rev1=1 crypto rx=0x7f538806a7c0 tx=0x7f538002bdb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6269f9532d39382f server_cookie=0 in_seq=0 out_seq=0 2023-08-09T13:35:48.058 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:48.054+0000 7f5385ffb640 1 -- 172.21.15.43:0/2401167894 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f538002c950 con 0x7f5388164940 2023-08-09T13:35:48.058 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:48.054+0000 7f5385ffb640 1 -- 172.21.15.43:0/2401167894 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f538002cab0 con 0x7f5388164940 2023-08-09T13:35:48.058 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:48.054+0000 7f5385ffb640 1 -- 172.21.15.43:0/2401167894 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f538002ce40 con 0x7f5388164940 2023-08-09T13:35:48.058 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:48.055+0000 7f5387fff640 1 -- 172.21.15.43:0/2401167894 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f5388164940 msgr2=0x7f5388164df0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:35:48.058 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:48.055+0000 7f5387fff640 1 --2- 172.21.15.43:0/2401167894 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f5388164940 0x7f5388164df0 secure :-1 s=READY pgs=462 cs=0 l=1 rev1=1 crypto rx=0x7f538806a7c0 tx=0x7f538002bdb0 comp rx=0 tx=0).stop 2023-08-09T13:35:48.058 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:48.055+0000 7f5387fff640 1 -- 172.21.15.43:0/2401167894 shutdown_connections 2023-08-09T13:35:48.058 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:48.055+0000 7f5387fff640 1 --2- 172.21.15.43:0/2401167894 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f5388164940 0x7f5388164df0 unknown :-1 s=CLOSED pgs=462 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:35:48.059 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:48.055+0000 7f5387fff640 1 --2- 172.21.15.43:0/2401167894 >> v2:172.21.15.203:3300/0 conn(0x7f53880c1020 0x7f5388162360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:35:48.059 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:48.055+0000 7f5387fff640 1 --2- 172.21.15.43:0/2401167894 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f5388164560 0x7f53880c8420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:35:48.059 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:48.055+0000 7f5387fff640 1 -- 172.21.15.43:0/2401167894 >> 172.21.15.43:0/2401167894 conn(0x7f53880cddd0 msgr2=0x7f53880ce1e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-09T13:35:48.059 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:48.055+0000 7f5387fff640 1 -- 172.21.15.43:0/2401167894 shutdown_connections 2023-08-09T13:35:48.059 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:48.055+0000 7f5387fff640 1 -- 172.21.15.43:0/2401167894 wait complete. 2023-08-09T13:35:48.060 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:48.057+0000 7f5387fff640 1 Processor -- start 2023-08-09T13:35:48.060 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:48.057+0000 7f5387fff640 1 -- start start 2023-08-09T13:35:48.061 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:48.058+0000 7f5387fff640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f53880c1020 0x7f538815d060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:35:48.061 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:48.058+0000 7f5387fff640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f5388164560 0x7f5388070260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:35:48.061 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:48.058+0000 7f5387fff640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f5388164940 0x7f53880707a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:35:48.061 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:48.058+0000 7f5387fff640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f53880c6330 con 0x7f5388164940 2023-08-09T13:35:48.061 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:48.058+0000 7f5387fff640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f538815e3a0 con 0x7f5388164560 2023-08-09T13:35:48.062 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:48.058+0000 7f5387fff640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f538815db00 con 0x7f53880c1020 2023-08-09T13:35:48.062 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:48.058+0000 7f53877fe640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f5388164940 0x7f53880707a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:35:48.062 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:48.058+0000 7f53877fe640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f5388164940 0x7f53880707a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:35560/0 (socket says 172.21.15.43:35560) 2023-08-09T13:35:48.062 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:48.058+0000 7f53877fe640 1 -- 172.21.15.43:0/1410053793 learned_addr learned my addr 172.21.15.43:0/1410053793 (peer_addr_for_me v2:172.21.15.43:0/0) 2023-08-09T13:35:48.062 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:48.058+0000 7f5386ffd640 1 --2- 172.21.15.43:0/1410053793 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f53880c1020 0x7f538815d060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:35:48.062 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:48.058+0000 7f53867fc640 1 --2- 172.21.15.43:0/1410053793 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f5388164560 0x7f5388070260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:35:48.062 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:48.058+0000 7f53877fe640 1 -- 172.21.15.43:0/1410053793 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f53880c1020 msgr2=0x7f538815d060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:35:48.062 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:48.058+0000 7f53877fe640 1 --2- 172.21.15.43:0/1410053793 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f53880c1020 0x7f538815d060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:35:48.063 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:48.058+0000 7f53877fe640 1 -- 172.21.15.43:0/1410053793 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f5388164560 msgr2=0x7f5388070260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:35:48.063 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:48.058+0000 7f53877fe640 1 --2- 172.21.15.43:0/1410053793 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f5388164560 0x7f5388070260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:35:48.063 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:48.059+0000 7f53877fe640 1 -- 172.21.15.43:0/1410053793 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5380009b30 con 0x7f5388164940 2023-08-09T13:35:48.064 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:48.059+0000 7f53877fe640 1 --2- 172.21.15.43:0/1410053793 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f5388164940 0x7f53880707a0 secure :-1 s=READY pgs=463 cs=0 l=1 rev1=1 crypto rx=0x7f5380036040 tx=0x7f5380004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-09T13:35:48.064 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:48.059+0000 7f536ffff640 1 -- 172.21.15.43:0/1410053793 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5380009510 con 0x7f5388164940 2023-08-09T13:35:48.064 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:48.059+0000 7f536ffff640 1 -- 172.21.15.43:0/1410053793 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f538003e450 con 0x7f5388164940 2023-08-09T13:35:48.064 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:48.059+0000 7f536ffff640 1 -- 172.21.15.43:0/1410053793 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f538003e7e0 con 0x7f5388164940 2023-08-09T13:35:48.064 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:48.059+0000 7f5387fff640 1 -- 172.21.15.43:0/1410053793 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f538815dde0 con 0x7f5388164940 2023-08-09T13:35:48.064 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:48.059+0000 7f5387fff640 1 -- 172.21.15.43:0/1410053793 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f53880c1c20 con 0x7f5388164940 2023-08-09T13:35:48.065 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:48.061+0000 7f5387fff640 1 -- 172.21.15.43:0/1410053793 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f53880cc840 con 0x7f5388164940 2023-08-09T13:35:48.065 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:48.061+0000 7f536ffff640 1 -- 172.21.15.43:0/1410053793 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 5) v1 ==== 88341+0+0 (secure 0 0 0) 0x7f538003e940 con 0x7f5388164940 2023-08-09T13:35:48.065 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:48.062+0000 7f536ffff640 1 --2- 172.21.15.43:0/1410053793 >> [v2:172.21.15.203:6832/830097474,v1:172.21.15.203:6833/830097474] conn(0x7f537406a050 0x7f537406c510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:35:48.065 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:48.062+0000 7f536ffff640 1 -- 172.21.15.43:0/1410053793 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 6871+0+0 (secure 0 0 0) 0x7f53800ab6b0 con 0x7f5388164940 2023-08-09T13:35:48.066 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:48.063+0000 7f5386ffd640 1 --2- 172.21.15.43:0/1410053793 >> [v2:172.21.15.203:6832/830097474,v1:172.21.15.203:6833/830097474] conn(0x7f537406a050 0x7f537406c510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:35:48.067 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:48.064+0000 7f5386ffd640 1 --2- 172.21.15.43:0/1410053793 >> [v2:172.21.15.203:6832/830097474,v1:172.21.15.203:6833/830097474] conn(0x7f537406a050 0x7f537406c510 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7f5378006fd0 tx=0x7f5378008040 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-09T13:35:48.069 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:48.067+0000 7f536ffff640 1 -- 172.21.15.43:0/1410053793 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178066 (secure 0 0 0) 0x7f5380076a60 con 0x7f5388164940 2023-08-09T13:35:48.268 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:48.265+0000 7f5387fff640 1 -- 172.21.15.43:0/1410053793 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f538804e390 con 0x7f5388164940 2023-08-09T13:35:48.269 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:48.266+0000 7f536ffff640 1 -- 172.21.15.43:0/1410053793 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v26) v1 ==== 93+0+10 (secure 0 0 0) 0x7f5380043020 con 0x7f5388164940 2023-08-09T13:35:48.269 INFO:teuthology.orchestra.run.smithi043.stdout:pg_num: 1 2023-08-09T13:35:48.271 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:48.269+0000 7f5387fff640 1 -- 172.21.15.43:0/1410053793 >> [v2:172.21.15.203:6832/830097474,v1:172.21.15.203:6833/830097474] conn(0x7f537406a050 msgr2=0x7f537406c510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:35:48.272 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:48.269+0000 7f5387fff640 1 --2- 172.21.15.43:0/1410053793 >> [v2:172.21.15.203:6832/830097474,v1:172.21.15.203:6833/830097474] conn(0x7f537406a050 0x7f537406c510 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7f5378006fd0 tx=0x7f5378008040 comp rx=0 tx=0).stop 2023-08-09T13:35:48.272 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:48.269+0000 7f5387fff640 1 -- 172.21.15.43:0/1410053793 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f5388164940 msgr2=0x7f53880707a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:35:48.272 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:48.269+0000 7f5387fff640 1 --2- 172.21.15.43:0/1410053793 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f5388164940 0x7f53880707a0 secure :-1 s=READY pgs=463 cs=0 l=1 rev1=1 crypto rx=0x7f5380036040 tx=0x7f5380004a40 comp rx=0 tx=0).stop 2023-08-09T13:35:48.272 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:48.269+0000 7f5387fff640 1 -- 172.21.15.43:0/1410053793 shutdown_connections 2023-08-09T13:35:48.272 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:48.269+0000 7f5387fff640 1 --2- 172.21.15.43:0/1410053793 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f5388164940 0x7f53880707a0 unknown :-1 s=CLOSED pgs=463 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:35:48.272 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:48.269+0000 7f5387fff640 1 --2- 172.21.15.43:0/1410053793 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f5388164560 0x7f5388070260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:35:48.272 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:48.269+0000 7f5387fff640 1 --2- 172.21.15.43:0/1410053793 >> [v2:172.21.15.203:6832/830097474,v1:172.21.15.203:6833/830097474] conn(0x7f537406a050 0x7f537406c510 unknown :-1 s=CLOSED pgs=229 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:35:48.273 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:48.269+0000 7f5387fff640 1 --2- 172.21.15.43:0/1410053793 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f53880c1020 0x7f538815d060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:35:48.273 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:48.269+0000 7f5387fff640 1 -- 172.21.15.43:0/1410053793 >> 172.21.15.43:0/1410053793 conn(0x7f53880cddd0 msgr2=0x7f5388162820 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-09T13:35:48.273 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:48.269+0000 7f5387fff640 1 -- 172.21.15.43:0/1410053793 shutdown_connections 2023-08-09T13:35:48.273 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:48.269+0000 7f5387fff640 1 -- 172.21.15.43:0/1410053793 wait complete. 2023-08-09T13:35:48.292 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-08-09T13:35:48.520 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:48.518+0000 7fe44359e640 1 Processor -- start 2023-08-09T13:35:48.521 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:48.518+0000 7fe44359e640 1 -- start start 2023-08-09T13:35:48.522 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:48.519+0000 7fe44359e640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fe4440b0f00 0x7fe4440ccef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:35:48.522 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:48.519+0000 7fe44359e640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7fe4440b1450 0x7fe4440cd430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:35:48.522 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:48.519+0000 7fe44359e640 1 -- --> v1:172.21.15.43:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe4440b7c00 con 0x7fe44415c3a0 2023-08-09T13:35:48.522 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:48.520+0000 7fe44359e640 1 -- --> v2:172.21.15.43:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe4440b81c0 con 0x7fe4440b1450 2023-08-09T13:35:48.523 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:48.520+0000 7fe44359e640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe4440b7ee0 con 0x7fe4440b0f00 2023-08-09T13:35:48.524 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:48.520+0000 7fe44259c640 1 --1- >> v1:172.21.15.43:6789/0 conn(0x7fe44415c3a0 0x7fe4441608a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.43:6789/0 says I am v1:172.21.15.43:48576/0 (socket says 172.21.15.43:48576) 2023-08-09T13:35:48.524 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:48.520+0000 7fe44259c640 1 -- 172.21.15.43:0/512001812 learned_addr learned my addr 172.21.15.43:0/512001812 (peer_addr_for_me v1:172.21.15.43:0/0) 2023-08-09T13:35:48.524 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:48.520+0000 7fe442d9d640 1 --2- 172.21.15.43:0/512001812 >> v2:172.21.15.43:3300/0 conn(0x7fe4440b1450 0x7fe4440cd430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:35:48.524 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:48.520+0000 7fe439d9b640 1 --2- 172.21.15.43:0/512001812 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fe4440b0f00 0x7fe4440ccef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:35:48.524 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:48.520+0000 7fe441d9b640 1 -- 172.21.15.43:0/512001812 <== mon.0 v1:172.21.15.43:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1773353642 0 0) 0x7fe4440b7c00 con 0x7fe44415c3a0 2023-08-09T13:35:48.524 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:48.521+0000 7fe441d9b640 1 -- 172.21.15.43:0/512001812 --> v1:172.21.15.43:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fe430003520 con 0x7fe44415c3a0 2023-08-09T13:35:48.525 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:48.521+0000 7fe442d9d640 1 -- 172.21.15.43:0/512001812 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fe4440b0f00 msgr2=0x7fe4440ccef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-09T13:35:48.525 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:48.521+0000 7fe442d9d640 1 --2- 172.21.15.43:0/512001812 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fe4440b0f00 0x7fe4440ccef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:35:48.525 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:48.521+0000 7fe442d9d640 1 -- 172.21.15.43:0/512001812 >> v1:172.21.15.43:6789/0 conn(0x7fe44415c3a0 legacy=0x7fe4441608a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:35:48.525 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:48.521+0000 7fe442d9d640 1 -- 172.21.15.43:0/512001812 --> v2:172.21.15.43:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe4441600c0 con 0x7fe4440b1450 2023-08-09T13:35:48.526 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:48.522+0000 7fe442d9d640 1 --2- 172.21.15.43:0/512001812 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fe4440b1450 0x7fe4440cd430 secure :-1 s=READY pgs=465 cs=0 l=1 rev1=1 crypto rx=0x7fe43c0097a0 tx=0x7fe43c02b820 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a74f5a6df839269b server_cookie=0 in_seq=0 out_seq=0 2023-08-09T13:35:48.527 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:48.522+0000 7fe441d9b640 1 -- 172.21.15.43:0/512001812 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe43c031780 con 0x7fe4440b1450 2023-08-09T13:35:48.527 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:48.522+0000 7fe441d9b640 1 -- 172.21.15.43:0/512001812 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe43c0318e0 con 0x7fe4440b1450 2023-08-09T13:35:48.527 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:48.522+0000 7fe441d9b640 1 -- 172.21.15.43:0/512001812 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe43c031c50 con 0x7fe4440b1450 2023-08-09T13:35:48.527 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:48.523+0000 7fe44359e640 1 -- 172.21.15.43:0/512001812 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fe4440b1450 msgr2=0x7fe4440cd430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:35:48.527 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:48.523+0000 7fe44359e640 1 --2- 172.21.15.43:0/512001812 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fe4440b1450 0x7fe4440cd430 secure :-1 s=READY pgs=465 cs=0 l=1 rev1=1 crypto rx=0x7fe43c0097a0 tx=0x7fe43c02b820 comp rx=0 tx=0).stop 2023-08-09T13:35:48.528 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:48.523+0000 7fe44359e640 1 -- 172.21.15.43:0/512001812 shutdown_connections 2023-08-09T13:35:48.528 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:48.523+0000 7fe44359e640 1 --2- 172.21.15.43:0/512001812 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fe4440b1450 0x7fe4440cd430 unknown :-1 s=CLOSED pgs=465 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:35:48.528 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:48.523+0000 7fe44359e640 1 --2- 172.21.15.43:0/512001812 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fe4440b0f00 0x7fe4440ccef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:35:48.528 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:48.523+0000 7fe44359e640 1 -- 172.21.15.43:0/512001812 >> 172.21.15.43:0/512001812 conn(0x7fe44406a210 msgr2=0x7fe44406a620 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-09T13:35:48.528 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:48.523+0000 7fe44359e640 1 -- 172.21.15.43:0/512001812 shutdown_connections 2023-08-09T13:35:48.528 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:48.523+0000 7fe44359e640 1 -- 172.21.15.43:0/512001812 wait complete. 2023-08-09T13:35:48.529 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:48.525+0000 7fe44359e640 1 Processor -- start 2023-08-09T13:35:48.529 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:48.525+0000 7fe44359e640 1 -- start start 2023-08-09T13:35:48.529 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:48.526+0000 7fe44359e640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fe4440b0f00 0x7fe444148160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:35:48.530 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:48.526+0000 7fe44359e640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fe4440b1450 0x7fe4440b2250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:35:48.530 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:48.526+0000 7fe44359e640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fe44415c3a0 0x7fe4441f5e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:35:48.530 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:48.526+0000 7fe44359e640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe4440cbed0 con 0x7fe4440b1450 2023-08-09T13:35:48.530 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:48.526+0000 7fe44359e640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe4440b2790 con 0x7fe44415c3a0 2023-08-09T13:35:48.530 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:48.526+0000 7fe44359e640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe4441f6350 con 0x7fe4440b0f00 2023-08-09T13:35:48.530 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:48.527+0000 7fe439d9b640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fe4440b1450 0x7fe4440b2250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:35:48.530 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:48.527+0000 7fe439d9b640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fe4440b1450 0x7fe4440b2250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:35580/0 (socket says 172.21.15.43:35580) 2023-08-09T13:35:48.530 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:48.527+0000 7fe439d9b640 1 -- 172.21.15.43:0/3933038084 learned_addr learned my addr 172.21.15.43:0/3933038084 (peer_addr_for_me v2:172.21.15.43:0/0) 2023-08-09T13:35:48.532 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:48.527+0000 7fe44259c640 1 --2- 172.21.15.43:0/3933038084 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fe4440b0f00 0x7fe444148160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:35:48.532 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:48.527+0000 7fe442d9d640 1 --2- 172.21.15.43:0/3933038084 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fe44415c3a0 0x7fe4441f5e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:35:48.532 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:48.527+0000 7fe439d9b640 1 -- 172.21.15.43:0/3933038084 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fe4440b0f00 msgr2=0x7fe444148160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:35:48.532 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:48.527+0000 7fe439d9b640 1 --2- 172.21.15.43:0/3933038084 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fe4440b0f00 0x7fe444148160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:35:48.532 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:48.527+0000 7fe439d9b640 1 -- 172.21.15.43:0/3933038084 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fe44415c3a0 msgr2=0x7fe4441f5e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:35:48.532 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:48.527+0000 7fe439d9b640 1 --2- 172.21.15.43:0/3933038084 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fe44415c3a0 0x7fe4441f5e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:35:48.533 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:48.527+0000 7fe439d9b640 1 -- 172.21.15.43:0/3933038084 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe43c009400 con 0x7fe4440b1450 2023-08-09T13:35:48.533 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:48.528+0000 7fe439d9b640 1 --2- 172.21.15.43:0/3933038084 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fe4440b1450 0x7fe4440b2250 secure :-1 s=READY pgs=466 cs=0 l=1 rev1=1 crypto rx=0x7fe42c00d800 tx=0x7fe42c00dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-09T13:35:48.533 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:48.528+0000 7fe43b7fe640 1 -- 172.21.15.43:0/3933038084 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe42c0096c0 con 0x7fe4440b1450 2023-08-09T13:35:48.533 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:48.528+0000 7fe43b7fe640 1 -- 172.21.15.43:0/3933038084 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe42c009820 con 0x7fe4440b1450 2023-08-09T13:35:48.533 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:48.528+0000 7fe43b7fe640 1 -- 172.21.15.43:0/3933038084 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe42c009bd0 con 0x7fe4440b1450 2023-08-09T13:35:48.533 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:48.528+0000 7fe44359e640 1 -- 172.21.15.43:0/3933038084 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe4440cb530 con 0x7fe4440b1450 2023-08-09T13:35:48.533 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:48.528+0000 7fe44359e640 1 -- 172.21.15.43:0/3933038084 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe4440cbaf0 con 0x7fe4440b1450 2023-08-09T13:35:48.534 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:48.529+0000 7fe44359e640 1 -- 172.21.15.43:0/3933038084 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe44415f450 con 0x7fe4440b1450 2023-08-09T13:35:48.534 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:48.530+0000 7fe43b7fe640 1 -- 172.21.15.43:0/3933038084 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 5) v1 ==== 88341+0+0 (secure 0 0 0) 0x7fe42c009d30 con 0x7fe4440b1450 2023-08-09T13:35:48.534 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:48.531+0000 7fe43b7fe640 1 --2- 172.21.15.43:0/3933038084 >> [v2:172.21.15.203:6832/830097474,v1:172.21.15.203:6833/830097474] conn(0x7fe42406a050 0x7fe42406c510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:35:48.534 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:48.531+0000 7fe43b7fe640 1 -- 172.21.15.43:0/3933038084 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 6871+0+0 (secure 0 0 0) 0x7fe42c0890b0 con 0x7fe4440b1450 2023-08-09T13:35:48.535 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:48.532+0000 7fe44259c640 1 --2- 172.21.15.43:0/3933038084 >> [v2:172.21.15.203:6832/830097474,v1:172.21.15.203:6833/830097474] conn(0x7fe42406a050 0x7fe42406c510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:35:48.535 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:48.533+0000 7fe44259c640 1 --2- 172.21.15.43:0/3933038084 >> [v2:172.21.15.203:6832/830097474,v1:172.21.15.203:6833/830097474] conn(0x7fe42406a050 0x7fe42406c510 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7fe434001560 tx=0x7fe434009040 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-09T13:35:48.538 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:48.535+0000 7fe43b7fe640 1 -- 172.21.15.43:0/3933038084 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178066 (secure 0 0 0) 0x7fe42c057ef0 con 0x7fe4440b1450 2023-08-09T13:35:48.733 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:48.731+0000 7fe44359e640 1 -- 172.21.15.43:0/3933038084 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7fe44404e390 con 0x7fe4440b1450 2023-08-09T13:35:48.734 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:48.732+0000 7fe43b7fe640 1 -- 172.21.15.43:0/3933038084 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v26) v1 ==== 104+0+11 (secure 0 0 0) 0x7fe42c057a80 con 0x7fe4440b1450 2023-08-09T13:35:48.735 INFO:teuthology.orchestra.run.smithi043.stdout:pg_num: 32 2023-08-09T13:35:48.737 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:48.734+0000 7fe44359e640 1 -- 172.21.15.43:0/3933038084 >> [v2:172.21.15.203:6832/830097474,v1:172.21.15.203:6833/830097474] conn(0x7fe42406a050 msgr2=0x7fe42406c510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:35:48.737 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:48.734+0000 7fe44359e640 1 --2- 172.21.15.43:0/3933038084 >> [v2:172.21.15.203:6832/830097474,v1:172.21.15.203:6833/830097474] conn(0x7fe42406a050 0x7fe42406c510 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7fe434001560 tx=0x7fe434009040 comp rx=0 tx=0).stop 2023-08-09T13:35:48.737 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:48.734+0000 7fe44359e640 1 -- 172.21.15.43:0/3933038084 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fe4440b1450 msgr2=0x7fe4440b2250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:35:48.738 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:48.734+0000 7fe44359e640 1 --2- 172.21.15.43:0/3933038084 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fe4440b1450 0x7fe4440b2250 secure :-1 s=READY pgs=466 cs=0 l=1 rev1=1 crypto rx=0x7fe42c00d800 tx=0x7fe42c00dcc0 comp rx=0 tx=0).stop 2023-08-09T13:35:48.738 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:48.734+0000 7fe44359e640 1 -- 172.21.15.43:0/3933038084 shutdown_connections 2023-08-09T13:35:48.738 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:48.734+0000 7fe44359e640 1 --2- 172.21.15.43:0/3933038084 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fe44415c3a0 0x7fe4441f5e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:35:48.738 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:48.734+0000 7fe44359e640 1 --2- 172.21.15.43:0/3933038084 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fe4440b1450 0x7fe4440b2250 unknown :-1 s=CLOSED pgs=466 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:35:48.738 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:48.734+0000 7fe44359e640 1 --2- 172.21.15.43:0/3933038084 >> [v2:172.21.15.203:6832/830097474,v1:172.21.15.203:6833/830097474] conn(0x7fe42406a050 0x7fe42406c510 unknown :-1 s=CLOSED pgs=230 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:35:48.738 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:48.734+0000 7fe44359e640 1 --2- 172.21.15.43:0/3933038084 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fe4440b0f00 0x7fe444148160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:35:48.738 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:48.734+0000 7fe44359e640 1 -- 172.21.15.43:0/3933038084 >> 172.21.15.43:0/3933038084 conn(0x7fe44406a210 msgr2=0x7fe4440c7cf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-09T13:35:48.738 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:48.735+0000 7fe44359e640 1 -- 172.21.15.43:0/3933038084 shutdown_connections 2023-08-09T13:35:48.739 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:48.735+0000 7fe44359e640 1 -- 172.21.15.43:0/3933038084 wait complete. 2023-08-09T13:35:48.757 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-08-09T13:35:48.987 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:48.984+0000 7fe271e41640 1 Processor -- start 2023-08-09T13:35:48.987 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:48.985+0000 7fe271e41640 1 -- start start 2023-08-09T13:35:48.988 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:48.986+0000 7fe271e41640 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7fe26c0c9560 0x7fe26c0cb9a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:35:48.989 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:48.986+0000 7fe271e41640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fe26c0c8900 0x7fe26c15a270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:35:48.989 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:48.986+0000 7fe271e41640 1 -- --> v1:172.21.15.203:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe26c15a7b0 con 0x7fe26c159400 2023-08-09T13:35:48.989 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:48.986+0000 7fe271e41640 1 -- --> v2:172.21.15.203:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe26c070250 con 0x7fe26c0c9560 2023-08-09T13:35:48.989 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:48.986+0000 7fe271e41640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe26c0c86c0 con 0x7fe26c0c8900 2023-08-09T13:35:48.990 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:48.987+0000 7fe271640640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fe26c0c8900 0x7fe26c15a270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:35:48.990 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:48.987+0000 7fe271640640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fe26c0c8900 0x7fe26c15a270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.203:3301/0 says I am v2:172.21.15.43:42100/0 (socket says 172.21.15.43:42100) 2023-08-09T13:35:48.990 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:48.987+0000 7fe271640640 1 -- 172.21.15.43:0/3603170666 learned_addr learned my addr 172.21.15.43:0/3603170666 (peer_addr_for_me v2:172.21.15.43:0/0) 2023-08-09T13:35:48.990 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:48.987+0000 7fe26bfff640 1 --2- 172.21.15.43:0/3603170666 >> v2:172.21.15.203:3300/0 conn(0x7fe26c0c9560 0x7fe26c0cb9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:35:48.991 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:48.988+0000 7fe271640640 1 -- 172.21.15.43:0/3603170666 >> v2:172.21.15.203:3300/0 conn(0x7fe26c0c9560 msgr2=0x7fe26c0cb9a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-09T13:35:48.991 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:48.988+0000 7fe271640640 1 --2- 172.21.15.43:0/3603170666 >> v2:172.21.15.203:3300/0 conn(0x7fe26c0c9560 0x7fe26c0cb9a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:35:48.991 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:48.988+0000 7fe271640640 1 -- 172.21.15.43:0/3603170666 >> v1:172.21.15.203:6789/0 conn(0x7fe26c159400 legacy=0x7fe26c0c8e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:35:48.991 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:48.988+0000 7fe271640640 1 -- 172.21.15.43:0/3603170666 --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe26c06fc60 con 0x7fe26c0c8900 2023-08-09T13:35:48.991 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:48.988+0000 7fe26bfff640 1 --2- 172.21.15.43:0/3603170666 >> v2:172.21.15.203:3300/0 conn(0x7fe26c0c9560 0x7fe26c0cb9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-08-09T13:35:48.992 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:48.989+0000 7fe271640640 1 --2- 172.21.15.43:0/3603170666 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fe26c0c8900 0x7fe26c15a270 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7fe264009e50 tx=0x7fe26402bee0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a0dd373a333756f6 server_cookie=0 in_seq=0 out_seq=0 2023-08-09T13:35:48.992 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:48.990+0000 7fe26b7fe640 1 -- 172.21.15.43:0/3603170666 <== mon.2 v2:172.21.15.203:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe26402ca80 con 0x7fe26c0c8900 2023-08-09T13:35:48.993 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:48.990+0000 7fe26b7fe640 1 -- 172.21.15.43:0/3603170666 <== mon.2 v2:172.21.15.203:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe26402cbe0 con 0x7fe26c0c8900 2023-08-09T13:35:48.993 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:48.990+0000 7fe26b7fe640 1 -- 172.21.15.43:0/3603170666 <== mon.2 v2:172.21.15.203:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe2640343f0 con 0x7fe26c0c8900 2023-08-09T13:35:48.993 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:48.991+0000 7fe271e41640 1 -- 172.21.15.43:0/3603170666 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fe26c0c8900 msgr2=0x7fe26c15a270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:35:48.994 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:48.991+0000 7fe271e41640 1 --2- 172.21.15.43:0/3603170666 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fe26c0c8900 0x7fe26c15a270 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7fe264009e50 tx=0x7fe26402bee0 comp rx=0 tx=0).stop 2023-08-09T13:35:48.994 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:48.991+0000 7fe271e41640 1 -- 172.21.15.43:0/3603170666 shutdown_connections 2023-08-09T13:35:48.994 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:48.991+0000 7fe271e41640 1 --2- 172.21.15.43:0/3603170666 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fe26c0c8900 0x7fe26c15a270 unknown :-1 s=CLOSED pgs=70 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:35:48.994 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:48.991+0000 7fe271e41640 1 --2- 172.21.15.43:0/3603170666 >> v2:172.21.15.203:3300/0 conn(0x7fe26c0c9560 0x7fe26c0cb9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:35:48.994 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:48.991+0000 7fe271e41640 1 -- 172.21.15.43:0/3603170666 >> 172.21.15.43:0/3603170666 conn(0x7fe26c0cddd0 msgr2=0x7fe26c0ce1e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-09T13:35:48.994 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:48.991+0000 7fe271e41640 1 -- 172.21.15.43:0/3603170666 shutdown_connections 2023-08-09T13:35:48.994 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:48.991+0000 7fe271e41640 1 -- 172.21.15.43:0/3603170666 wait complete. 2023-08-09T13:35:48.995 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:48.993+0000 7fe271e41640 1 Processor -- start 2023-08-09T13:35:48.995 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:48.993+0000 7fe271e41640 1 -- start start 2023-08-09T13:35:48.996 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:48.994+0000 7fe271e41640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fe26c0c8900 0x7fe26c15d260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:35:48.996 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:48.994+0000 7fe271e41640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fe26c0c9560 0x7fe26c157920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:35:48.996 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:48.994+0000 7fe271e41640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fe26c14e9c0 0x7fe26c157e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:35:48.997 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:48.994+0000 7fe271e41640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe26c15c1c0 con 0x7fe26c0c9560 2023-08-09T13:35:48.997 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:48.994+0000 7fe271e41640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe26c15c4a0 con 0x7fe26c14e9c0 2023-08-09T13:35:48.997 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:48.994+0000 7fe271e41640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe26c15c780 con 0x7fe26c0c8900 2023-08-09T13:35:48.998 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:48.994+0000 7fe26bfff640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fe26c0c9560 0x7fe26c157920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:35:48.998 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:48.994+0000 7fe26bfff640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fe26c0c9560 0x7fe26c157920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:35582/0 (socket says 172.21.15.43:35582) 2023-08-09T13:35:48.998 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:48.994+0000 7fe26bfff640 1 -- 172.21.15.43:0/3211372415 learned_addr learned my addr 172.21.15.43:0/3211372415 (peer_addr_for_me v2:172.21.15.43:0/0) 2023-08-09T13:35:48.998 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:48.994+0000 7fe270e3f640 1 --2- 172.21.15.43:0/3211372415 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fe26c0c8900 0x7fe26c15d260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:35:48.999 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:48.994+0000 7fe271640640 1 --2- 172.21.15.43:0/3211372415 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fe26c14e9c0 0x7fe26c157e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:35:48.999 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:48.995+0000 7fe26bfff640 1 -- 172.21.15.43:0/3211372415 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fe26c0c8900 msgr2=0x7fe26c15d260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:35:48.999 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:48.995+0000 7fe26bfff640 1 --2- 172.21.15.43:0/3211372415 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fe26c0c8900 0x7fe26c15d260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:35:48.999 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:48.995+0000 7fe26bfff640 1 -- 172.21.15.43:0/3211372415 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fe26c14e9c0 msgr2=0x7fe26c157e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:35:48.999 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:48.995+0000 7fe26bfff640 1 --2- 172.21.15.43:0/3211372415 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fe26c14e9c0 0x7fe26c157e60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:35:48.999 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:48.995+0000 7fe26bfff640 1 -- 172.21.15.43:0/3211372415 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe264009b30 con 0x7fe26c0c9560 2023-08-09T13:35:48.999 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:48.995+0000 7fe26bfff640 1 --2- 172.21.15.43:0/3211372415 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fe26c0c9560 0x7fe26c157920 secure :-1 s=READY pgs=467 cs=0 l=1 rev1=1 crypto rx=0x7fe25800e7e0 tx=0x7fe25800eca0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-09T13:35:49.000 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:48.995+0000 7fe2697fa640 1 -- 172.21.15.43:0/3211372415 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe258004030 con 0x7fe26c0c9560 2023-08-09T13:35:49.000 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:48.995+0000 7fe2697fa640 1 -- 172.21.15.43:0/3211372415 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe258004190 con 0x7fe26c0c9560 2023-08-09T13:35:49.000 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:48.995+0000 7fe2697fa640 1 -- 172.21.15.43:0/3211372415 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe258004520 con 0x7fe26c0c9560 2023-08-09T13:35:49.000 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:48.995+0000 7fe271e41640 1 -- 172.21.15.43:0/3211372415 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe26c15ca60 con 0x7fe26c0c9560 2023-08-09T13:35:49.000 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:48.995+0000 7fe271e41640 1 -- 172.21.15.43:0/3211372415 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe26c15be70 con 0x7fe26c0c9560 2023-08-09T13:35:49.001 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:48.997+0000 7fe271e41640 1 -- 172.21.15.43:0/3211372415 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe26c04e390 con 0x7fe26c0c9560 2023-08-09T13:35:49.001 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:48.997+0000 7fe2697fa640 1 -- 172.21.15.43:0/3211372415 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 5) v1 ==== 88341+0+0 (secure 0 0 0) 0x7fe25800c6d0 con 0x7fe26c0c9560 2023-08-09T13:35:49.001 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:48.998+0000 7fe2697fa640 1 --2- 172.21.15.43:0/3211372415 >> [v2:172.21.15.203:6832/830097474,v1:172.21.15.203:6833/830097474] conn(0x7fe25006a050 0x7fe25006c510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:35:49.001 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:48.999+0000 7fe2697fa640 1 -- 172.21.15.43:0/3211372415 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 6871+0+0 (secure 0 0 0) 0x7fe2580885d0 con 0x7fe26c0c9560 2023-08-09T13:35:49.002 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:48.999+0000 7fe270e3f640 1 --2- 172.21.15.43:0/3211372415 >> [v2:172.21.15.203:6832/830097474,v1:172.21.15.203:6833/830097474] conn(0x7fe25006a050 0x7fe25006c510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:35:49.003 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:49.001+0000 7fe270e3f640 1 --2- 172.21.15.43:0/3211372415 >> [v2:172.21.15.203:6832/830097474,v1:172.21.15.203:6833/830097474] conn(0x7fe25006a050 0x7fe25006c510 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7fe26c153040 tx=0x7fe260009040 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-09T13:35:49.005 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:49.002+0000 7fe2697fa640 1 -- 172.21.15.43:0/3211372415 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178066 (secure 0 0 0) 0x7fe258053980 con 0x7fe26c0c9560 2023-08-09T13:35:49.204 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:49.201+0000 7fe271e41640 1 -- 172.21.15.43:0/3211372415 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7fe26c0cea30 con 0x7fe26c0c9560 2023-08-09T13:35:49.205 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:49.202+0000 7fe2697fa640 1 -- 172.21.15.43:0/3211372415 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v26) v1 ==== 100+0+11 (secure 0 0 0) 0x7fe258056fa0 con 0x7fe26c0c9560 2023-08-09T13:35:49.205 INFO:teuthology.orchestra.run.smithi043.stdout:pg_num: 64 2023-08-09T13:35:49.208 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:49.205+0000 7fe271e41640 1 -- 172.21.15.43:0/3211372415 >> [v2:172.21.15.203:6832/830097474,v1:172.21.15.203:6833/830097474] conn(0x7fe25006a050 msgr2=0x7fe25006c510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:35:49.208 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:49.205+0000 7fe271e41640 1 --2- 172.21.15.43:0/3211372415 >> [v2:172.21.15.203:6832/830097474,v1:172.21.15.203:6833/830097474] conn(0x7fe25006a050 0x7fe25006c510 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7fe26c153040 tx=0x7fe260009040 comp rx=0 tx=0).stop 2023-08-09T13:35:49.208 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:49.205+0000 7fe271e41640 1 -- 172.21.15.43:0/3211372415 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fe26c0c9560 msgr2=0x7fe26c157920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:35:49.208 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:49.205+0000 7fe271e41640 1 --2- 172.21.15.43:0/3211372415 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fe26c0c9560 0x7fe26c157920 secure :-1 s=READY pgs=467 cs=0 l=1 rev1=1 crypto rx=0x7fe25800e7e0 tx=0x7fe25800eca0 comp rx=0 tx=0).stop 2023-08-09T13:35:49.208 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:49.205+0000 7fe271e41640 1 -- 172.21.15.43:0/3211372415 shutdown_connections 2023-08-09T13:35:49.208 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:49.205+0000 7fe271e41640 1 --2- 172.21.15.43:0/3211372415 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fe26c14e9c0 0x7fe26c157e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:35:49.209 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:49.205+0000 7fe271e41640 1 --2- 172.21.15.43:0/3211372415 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fe26c0c9560 0x7fe26c157920 unknown :-1 s=CLOSED pgs=467 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:35:49.209 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:49.205+0000 7fe271e41640 1 --2- 172.21.15.43:0/3211372415 >> [v2:172.21.15.203:6832/830097474,v1:172.21.15.203:6833/830097474] conn(0x7fe25006a050 0x7fe25006c510 unknown :-1 s=CLOSED pgs=231 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:35:49.209 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:49.205+0000 7fe271e41640 1 --2- 172.21.15.43:0/3211372415 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fe26c0c8900 0x7fe26c15d260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:35:49.209 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:49.205+0000 7fe271e41640 1 -- 172.21.15.43:0/3211372415 >> 172.21.15.43:0/3211372415 conn(0x7fe26c0cddd0 msgr2=0x7fe26c06cc60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-09T13:35:49.209 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:49.205+0000 7fe271e41640 1 -- 172.21.15.43:0/3211372415 shutdown_connections 2023-08-09T13:35:49.209 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:49.205+0000 7fe271e41640 1 -- 172.21.15.43:0/3211372415 wait complete. 2023-08-09T13:35:49.225 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2023-08-09T13:35:49.441 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:49.437+0000 7f84ba974640 1 Processor -- start 2023-08-09T13:35:49.441 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:49.438+0000 7f84ba974640 1 -- start start 2023-08-09T13:35:49.443 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:49.439+0000 7f84ba974640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f84b40b0f00 0x7f84b40ccef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:35:49.444 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:49.439+0000 7f84ba974640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f84b40b1450 0x7f84b40cd430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:35:49.445 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:49.439+0000 7f84ba974640 1 -- --> v1:172.21.15.43:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f84b40b7c00 con 0x7f84b415c3a0 2023-08-09T13:35:49.445 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:49.439+0000 7f84ba974640 1 -- --> v2:172.21.15.43:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f84b40b81c0 con 0x7f84b40b0f00 2023-08-09T13:35:49.445 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:49.439+0000 7f84ba974640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f84b40b7ee0 con 0x7f84b40b1450 2023-08-09T13:35:49.446 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:49.440+0000 7f84b9171640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f84b40b0f00 0x7f84b40ccef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:35:49.446 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:49.440+0000 7f84b9972640 1 --1- >> v1:172.21.15.43:6789/0 conn(0x7f84b415c3a0 0x7f84b41608a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.43:6789/0 says I am v1:172.21.15.43:48580/0 (socket says 172.21.15.43:48580) 2023-08-09T13:35:49.446 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:49.440+0000 7f84b9972640 1 -- 172.21.15.43:0/2424229038 learned_addr learned my addr 172.21.15.43:0/2424229038 (peer_addr_for_me v1:172.21.15.43:0/0) 2023-08-09T13:35:49.447 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:49.440+0000 7f84ba173640 1 --2- 172.21.15.43:0/2424229038 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f84b40b1450 0x7f84b40cd430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:35:49.447 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:49.440+0000 7f84b9171640 1 -- 172.21.15.43:0/2424229038 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f84b40b1450 msgr2=0x7f84b40cd430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-09T13:35:49.447 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:49.440+0000 7f84b9171640 1 --2- 172.21.15.43:0/2424229038 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f84b40b1450 0x7f84b40cd430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:35:49.448 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:49.440+0000 7f84b9171640 1 -- 172.21.15.43:0/2424229038 >> v1:172.21.15.43:6789/0 conn(0x7f84b415c3a0 legacy=0x7f84b41608a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:35:49.448 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:49.440+0000 7f84b9171640 1 -- 172.21.15.43:0/2424229038 --> v2:172.21.15.43:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f84b41600c0 con 0x7f84b40b0f00 2023-08-09T13:35:49.449 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:49.440+0000 7f84b8970640 1 -- 172.21.15.43:0/2424229038 <== mon.0 v1:172.21.15.43:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1679496168 0 0) 0x7f84b40b7c00 con 0x7f84b415c3a0 2023-08-09T13:35:49.454 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:49.441+0000 7f84b9171640 1 --2- 172.21.15.43:0/2424229038 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f84b40b0f00 0x7f84b40ccef0 secure :-1 s=READY pgs=469 cs=0 l=1 rev1=1 crypto rx=0x7f84a4009790 tx=0x7f84a402b810 comp rx=0 tx=0).ready entity=mon.0 client_cookie=10d0bf9e2e659092 server_cookie=0 in_seq=0 out_seq=0 2023-08-09T13:35:49.455 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:49.442+0000 7f84b8970640 1 -- 172.21.15.43:0/2424229038 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f84a4031770 con 0x7f84b40b0f00 2023-08-09T13:35:49.455 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:49.442+0000 7f84b8970640 1 -- 172.21.15.43:0/2424229038 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f84a40318d0 con 0x7f84b40b0f00 2023-08-09T13:35:49.456 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:49.442+0000 7f84b8970640 1 -- 172.21.15.43:0/2424229038 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f84a4031c60 con 0x7f84b40b0f00 2023-08-09T13:35:49.456 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:49.442+0000 7f84ba974640 1 -- 172.21.15.43:0/2424229038 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f84b40b0f00 msgr2=0x7f84b40ccef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:35:49.457 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:49.442+0000 7f84ba974640 1 --2- 172.21.15.43:0/2424229038 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f84b40b0f00 0x7f84b40ccef0 secure :-1 s=READY pgs=469 cs=0 l=1 rev1=1 crypto rx=0x7f84a4009790 tx=0x7f84a402b810 comp rx=0 tx=0).stop 2023-08-09T13:35:49.457 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:49.442+0000 7f84ba974640 1 -- 172.21.15.43:0/2424229038 shutdown_connections 2023-08-09T13:35:49.457 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:49.442+0000 7f84ba974640 1 --2- 172.21.15.43:0/2424229038 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f84b40b1450 0x7f84b40cd430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:35:49.458 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:49.442+0000 7f84ba974640 1 --2- 172.21.15.43:0/2424229038 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f84b40b0f00 0x7f84b40ccef0 unknown :-1 s=CLOSED pgs=469 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:35:49.458 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:49.442+0000 7f84ba974640 1 -- 172.21.15.43:0/2424229038 >> 172.21.15.43:0/2424229038 conn(0x7f84b406a210 msgr2=0x7f84b406a620 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-09T13:35:49.459 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:49.443+0000 7f84ba974640 1 -- 172.21.15.43:0/2424229038 shutdown_connections 2023-08-09T13:35:49.459 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:49.443+0000 7f84ba974640 1 -- 172.21.15.43:0/2424229038 wait complete. 2023-08-09T13:35:49.459 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:49.444+0000 7f84ba974640 1 Processor -- start 2023-08-09T13:35:49.460 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:49.444+0000 7f84ba974640 1 -- start start 2023-08-09T13:35:49.460 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:49.445+0000 7f84ba974640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f84b40b0f00 0x7f84b41511a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:35:49.460 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:49.445+0000 7f84ba974640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f84b40b1450 0x7f84b41f2650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:35:49.461 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:49.445+0000 7f84ba974640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f84b415c3a0 0x7f84b4150630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:35:49.461 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:49.445+0000 7f84ba974640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f84b41f2d50 con 0x7f84b415c3a0 2023-08-09T13:35:49.462 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:49.445+0000 7f84ba974640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f84b41f2b90 con 0x7f84b40b0f00 2023-08-09T13:35:49.462 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:49.445+0000 7f84ba974640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f84b4150b70 con 0x7f84b40b1450 2023-08-09T13:35:49.462 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:49.446+0000 7f84ba173640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f84b415c3a0 0x7f84b4150630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:35:49.463 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:49.446+0000 7f84ba173640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f84b415c3a0 0x7f84b4150630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:35600/0 (socket says 172.21.15.43:35600) 2023-08-09T13:35:49.463 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:49.446+0000 7f84ba173640 1 -- 172.21.15.43:0/1763803071 learned_addr learned my addr 172.21.15.43:0/1763803071 (peer_addr_for_me v2:172.21.15.43:0/0) 2023-08-09T13:35:49.463 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:49.446+0000 7f84ba173640 1 -- 172.21.15.43:0/1763803071 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f84b40b1450 msgr2=0x7f84b41f2650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:35:49.463 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:49.446+0000 7f84ba173640 1 --2- 172.21.15.43:0/1763803071 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f84b40b1450 0x7f84b41f2650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:35:49.464 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:49.446+0000 7f84ba173640 1 -- 172.21.15.43:0/1763803071 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f84b40b0f00 msgr2=0x7f84b41511a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:35:49.464 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:49.446+0000 7f84ba173640 1 --2- 172.21.15.43:0/1763803071 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f84b40b0f00 0x7f84b41511a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:35:49.464 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:49.446+0000 7f84ba173640 1 -- 172.21.15.43:0/1763803071 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f84a40093a0 con 0x7f84b415c3a0 2023-08-09T13:35:49.465 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:49.446+0000 7f84ba173640 1 --2- 172.21.15.43:0/1763803071 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f84b415c3a0 0x7f84b4150630 secure :-1 s=READY pgs=470 cs=0 l=1 rev1=1 crypto rx=0x7f84ac00d5d0 tx=0x7f84ac00da90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-09T13:35:49.465 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:49.447+0000 7f84aa7fc640 1 -- 172.21.15.43:0/1763803071 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f84ac0125f0 con 0x7f84b415c3a0 2023-08-09T13:35:49.465 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:49.447+0000 7f84aa7fc640 1 -- 172.21.15.43:0/1763803071 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f84ac012770 con 0x7f84b415c3a0 2023-08-09T13:35:49.465 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:49.447+0000 7f84ba974640 1 -- 172.21.15.43:0/1763803071 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f84b4151780 con 0x7f84b415c3a0 2023-08-09T13:35:49.466 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:49.447+0000 7f84aa7fc640 1 -- 172.21.15.43:0/1763803071 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f84ac012b20 con 0x7f84b415c3a0 2023-08-09T13:35:49.466 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:49.447+0000 7f84ba974640 1 -- 172.21.15.43:0/1763803071 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f84b4151d40 con 0x7f84b415c3a0 2023-08-09T13:35:49.467 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:49.448+0000 7f84ba974640 1 -- 172.21.15.43:0/1763803071 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f84b40cf670 con 0x7f84b415c3a0 2023-08-09T13:35:49.467 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:49.449+0000 7f84aa7fc640 1 -- 172.21.15.43:0/1763803071 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 5) v1 ==== 88341+0+0 (secure 0 0 0) 0x7f84ac012c80 con 0x7f84b415c3a0 2023-08-09T13:35:49.467 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:49.450+0000 7f84aa7fc640 1 --2- 172.21.15.43:0/1763803071 >> [v2:172.21.15.203:6832/830097474,v1:172.21.15.203:6833/830097474] conn(0x7f849806a050 0x7f849806c510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:35:49.468 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:49.450+0000 7f84aa7fc640 1 -- 172.21.15.43:0/1763803071 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 6871+0+0 (secure 0 0 0) 0x7f84ac088df0 con 0x7f84b415c3a0 2023-08-09T13:35:49.468 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:49.451+0000 7f84b9972640 1 --2- 172.21.15.43:0/1763803071 >> [v2:172.21.15.203:6832/830097474,v1:172.21.15.203:6833/830097474] conn(0x7f849806a050 0x7f849806c510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:35:49.469 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:49.452+0000 7f84b9972640 1 --2- 172.21.15.43:0/1763803071 >> [v2:172.21.15.203:6832/830097474,v1:172.21.15.203:6833/830097474] conn(0x7f849806a050 0x7f849806c510 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7f84b0005c00 tx=0x7f84b00015e0 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-09T13:35:49.469 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:49.455+0000 7f84aa7fc640 1 -- 172.21.15.43:0/1763803071 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178066 (secure 0 0 0) 0x7f84ac054120 con 0x7f84b415c3a0 2023-08-09T13:35:49.649 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:49.646+0000 7f84ba974640 1 -- 172.21.15.43:0/1763803071 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) v1 -- 0x7f84b404e390 con 0x7f84b415c3a0 2023-08-09T13:35:49.650 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:49.647+0000 7f84aa7fc640 1 -- 172.21.15.43:0/1763803071 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v26) v1 ==== 103+0+11 (secure 0 0 0) 0x7f84ac057740 con 0x7f84b415c3a0 2023-08-09T13:35:49.650 INFO:teuthology.orchestra.run.smithi043.stdout:pg_num: 32 2023-08-09T13:35:49.653 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:49.650+0000 7f84ba974640 1 -- 172.21.15.43:0/1763803071 >> [v2:172.21.15.203:6832/830097474,v1:172.21.15.203:6833/830097474] conn(0x7f849806a050 msgr2=0x7f849806c510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:35:49.653 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:49.650+0000 7f84ba974640 1 --2- 172.21.15.43:0/1763803071 >> [v2:172.21.15.203:6832/830097474,v1:172.21.15.203:6833/830097474] conn(0x7f849806a050 0x7f849806c510 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7f84b0005c00 tx=0x7f84b00015e0 comp rx=0 tx=0).stop 2023-08-09T13:35:49.653 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:49.650+0000 7f84ba974640 1 -- 172.21.15.43:0/1763803071 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f84b415c3a0 msgr2=0x7f84b4150630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:35:49.653 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:49.650+0000 7f84ba974640 1 --2- 172.21.15.43:0/1763803071 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f84b415c3a0 0x7f84b4150630 secure :-1 s=READY pgs=470 cs=0 l=1 rev1=1 crypto rx=0x7f84ac00d5d0 tx=0x7f84ac00da90 comp rx=0 tx=0).stop 2023-08-09T13:35:49.654 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:49.650+0000 7f84ba974640 1 -- 172.21.15.43:0/1763803071 shutdown_connections 2023-08-09T13:35:49.654 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:49.650+0000 7f84ba974640 1 --2- 172.21.15.43:0/1763803071 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f84b415c3a0 0x7f84b4150630 unknown :-1 s=CLOSED pgs=470 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:35:49.654 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:49.650+0000 7f84ba974640 1 --2- 172.21.15.43:0/1763803071 >> [v2:172.21.15.203:6832/830097474,v1:172.21.15.203:6833/830097474] conn(0x7f849806a050 0x7f849806c510 unknown :-1 s=CLOSED pgs=232 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:35:49.655 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:49.650+0000 7f84ba974640 1 --2- 172.21.15.43:0/1763803071 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f84b40b1450 0x7f84b41f2650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:35:49.655 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:49.650+0000 7f84ba974640 1 --2- 172.21.15.43:0/1763803071 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f84b40b0f00 0x7f84b41511a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:35:49.655 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:49.650+0000 7f84ba974640 1 -- 172.21.15.43:0/1763803071 >> 172.21.15.43:0/1763803071 conn(0x7f84b406a210 msgr2=0x7f84b415a840 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-09T13:35:49.656 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:49.651+0000 7f84ba974640 1 -- 172.21.15.43:0/1763803071 shutdown_connections 2023-08-09T13:35:49.656 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:49.651+0000 7f84ba974640 1 -- 172.21.15.43:0/1763803071 wait complete. 2023-08-09T13:35:49.673 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2023-08-09T13:35:49.900 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:49.898+0000 7f5ad1ccd640 1 Processor -- start 2023-08-09T13:35:49.901 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:49.898+0000 7f5ad1ccd640 1 -- start start 2023-08-09T13:35:49.901 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:49.899+0000 7f5ad1ccd640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f5acc15f7b0 0x7f5acc0b4760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:35:49.902 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:49.899+0000 7f5ad1ccd640 1 -- --> v1:172.21.15.43:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5acc0b7bf0 con 0x7f5acc0b1050 2023-08-09T13:35:49.902 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:49.899+0000 7f5ad1ccd640 1 -- --> v1:172.21.15.203:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5acc0b7ed0 con 0x7f5acc0cd6b0 2023-08-09T13:35:49.902 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:49.899+0000 7f5ad1ccd640 1 -- --> v2:172.21.15.43:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5acc0d0370 con 0x7f5acc15f7b0 2023-08-09T13:35:49.903 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:49.900+0000 7f5acbfff640 1 --1- >> v1:172.21.15.43:6789/0 conn(0x7f5acc0b1050 0x7f5acc0ccfa0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.43:6789/0 says I am v1:172.21.15.43:48582/0 (socket says 172.21.15.43:48582) 2023-08-09T13:35:49.903 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:49.900+0000 7f5ad0ccb640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f5acc15f7b0 0x7f5acc0b4760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:35:49.903 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:49.900+0000 7f5acbfff640 1 -- 172.21.15.43:0/2082889591 learned_addr learned my addr 172.21.15.43:0/2082889591 (peer_addr_for_me v1:172.21.15.43:0/0) 2023-08-09T13:35:49.903 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:49.900+0000 7f5ad0ccb640 1 -- 172.21.15.43:0/2082889591 >> v1:172.21.15.203:6789/0 conn(0x7f5acc0cd6b0 legacy=0x7f5acc0d0e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-09T13:35:49.903 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:49.900+0000 7f5ad0ccb640 1 -- 172.21.15.43:0/2082889591 >> v1:172.21.15.43:6789/0 conn(0x7f5acc0b1050 legacy=0x7f5acc0ccfa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:35:49.903 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:49.900+0000 7f5ad0ccb640 1 -- 172.21.15.43:0/2082889591 --> v2:172.21.15.43:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5acc0d0650 con 0x7f5acc15f7b0 2023-08-09T13:35:49.904 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:49.900+0000 7f5acb7fe640 1 -- 172.21.15.43:0/2082889591 <== mon.0 v1:172.21.15.43:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3407032683 0 0) 0x7f5acc0b7bf0 con 0x7f5acc0b1050 2023-08-09T13:35:49.904 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:49.902+0000 7f5ad0ccb640 1 --2- 172.21.15.43:0/2082889591 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f5acc15f7b0 0x7f5acc0b4760 secure :-1 s=READY pgs=472 cs=0 l=1 rev1=1 crypto rx=0x7f5abc009dc0 tx=0x7f5abc02be90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c231a4b689a657f0 server_cookie=0 in_seq=0 out_seq=0 2023-08-09T13:35:49.904 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:49.902+0000 7f5acb7fe640 1 -- 172.21.15.43:0/2082889591 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5abc02cc40 con 0x7f5acc15f7b0 2023-08-09T13:35:49.905 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:49.902+0000 7f5acb7fe640 1 -- 172.21.15.43:0/2082889591 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5abc02cda0 con 0x7f5acc15f7b0 2023-08-09T13:35:49.905 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:49.902+0000 7f5acb7fe640 1 -- 172.21.15.43:0/2082889591 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5abc034660 con 0x7f5acc15f7b0 2023-08-09T13:35:49.905 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:49.902+0000 7f5ad1ccd640 1 -- 172.21.15.43:0/2082889591 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f5acc15f7b0 msgr2=0x7f5acc0b4760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:35:49.906 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:49.902+0000 7f5ad1ccd640 1 --2- 172.21.15.43:0/2082889591 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f5acc15f7b0 0x7f5acc0b4760 secure :-1 s=READY pgs=472 cs=0 l=1 rev1=1 crypto rx=0x7f5abc009dc0 tx=0x7f5abc02be90 comp rx=0 tx=0).stop 2023-08-09T13:35:49.906 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:49.903+0000 7f5ad1ccd640 1 -- 172.21.15.43:0/2082889591 shutdown_connections 2023-08-09T13:35:49.906 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:49.903+0000 7f5ad1ccd640 1 --2- 172.21.15.43:0/2082889591 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f5acc15f7b0 0x7f5acc0b4760 unknown :-1 s=CLOSED pgs=472 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:35:49.906 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:49.903+0000 7f5ad1ccd640 1 -- 172.21.15.43:0/2082889591 >> 172.21.15.43:0/2082889591 conn(0x7f5acc06a2b0 msgr2=0x7f5acc06a690 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-09T13:35:49.906 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:49.903+0000 7f5ad1ccd640 1 -- 172.21.15.43:0/2082889591 shutdown_connections 2023-08-09T13:35:49.906 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:49.903+0000 7f5ad1ccd640 1 -- 172.21.15.43:0/2082889591 wait complete. 2023-08-09T13:35:49.907 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:49.904+0000 7f5ad1ccd640 1 Processor -- start 2023-08-09T13:35:49.907 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:49.905+0000 7f5ad1ccd640 1 -- start start 2023-08-09T13:35:49.908 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:49.906+0000 7f5ad1ccd640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f5acc0b1050 0x7f5acc1f4d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:35:49.908 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:49.906+0000 7f5ad1ccd640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f5acc0cd6b0 0x7f5acc0c9760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:35:49.909 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:49.906+0000 7f5ad1ccd640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f5acc15f7b0 0x7f5acc0c7c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:35:49.909 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:49.906+0000 7f5ad1ccd640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5acc0b2560 con 0x7f5acc0cd6b0 2023-08-09T13:35:49.909 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:49.906+0000 7f5ad1ccd640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5acc1f5b90 con 0x7f5acc15f7b0 2023-08-09T13:35:49.909 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:49.906+0000 7f5ad1ccd640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5acc1f52f0 con 0x7f5acc0b1050 2023-08-09T13:35:49.909 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:49.906+0000 7f5acbfff640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f5acc0cd6b0 0x7f5acc0c9760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:35:49.909 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:49.906+0000 7f5acbfff640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f5acc0cd6b0 0x7f5acc0c9760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:35618/0 (socket says 172.21.15.43:35618) 2023-08-09T13:35:49.909 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:49.906+0000 7f5acbfff640 1 -- 172.21.15.43:0/2036206972 learned_addr learned my addr 172.21.15.43:0/2036206972 (peer_addr_for_me v2:172.21.15.43:0/0) 2023-08-09T13:35:49.910 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:49.906+0000 7f5acbfff640 1 -- 172.21.15.43:0/2036206972 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f5acc0b1050 msgr2=0x7f5acc1f4d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:35:49.911 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:49.906+0000 7f5acbfff640 1 --2- 172.21.15.43:0/2036206972 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f5acc0b1050 0x7f5acc1f4d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:35:49.911 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:49.906+0000 7f5acbfff640 1 -- 172.21.15.43:0/2036206972 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f5acc15f7b0 msgr2=0x7f5acc0c7c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:35:49.911 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:49.906+0000 7f5acbfff640 1 --2- 172.21.15.43:0/2036206972 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f5acc15f7b0 0x7f5acc0c7c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:35:49.911 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:49.906+0000 7f5acbfff640 1 -- 172.21.15.43:0/2036206972 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5abc009a70 con 0x7f5acc0cd6b0 2023-08-09T13:35:49.911 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:49.907+0000 7f5acbfff640 1 --2- 172.21.15.43:0/2036206972 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f5acc0cd6b0 0x7f5acc0c9760 secure :-1 s=READY pgs=473 cs=0 l=1 rev1=1 crypto rx=0x7f5ac000c470 tx=0x7f5ac000c930 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-09T13:35:49.911 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:49.907+0000 7f5ac97fa640 1 -- 172.21.15.43:0/2036206972 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5ac00074e0 con 0x7f5acc0cd6b0 2023-08-09T13:35:49.911 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:49.907+0000 7f5ac97fa640 1 -- 172.21.15.43:0/2036206972 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5ac0007640 con 0x7f5acc0cd6b0 2023-08-09T13:35:49.912 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:49.907+0000 7f5ac97fa640 1 -- 172.21.15.43:0/2036206972 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5ac0004a20 con 0x7f5acc0cd6b0 2023-08-09T13:35:49.912 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:49.907+0000 7f5ad1ccd640 1 -- 172.21.15.43:0/2036206972 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5acc1f55d0 con 0x7f5acc0cd6b0 2023-08-09T13:35:49.912 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:49.907+0000 7f5ad1ccd640 1 -- 172.21.15.43:0/2036206972 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5acc0c9ca0 con 0x7f5acc0cd6b0 2023-08-09T13:35:49.912 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:49.908+0000 7f5ad1ccd640 1 -- 172.21.15.43:0/2036206972 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5acc0b26a0 con 0x7f5acc0cd6b0 2023-08-09T13:35:49.912 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:49.909+0000 7f5ac97fa640 1 -- 172.21.15.43:0/2036206972 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 5) v1 ==== 88341+0+0 (secure 0 0 0) 0x7f5ac0004b80 con 0x7f5acc0cd6b0 2023-08-09T13:35:49.913 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:49.910+0000 7f5ac97fa640 1 --2- 172.21.15.43:0/2036206972 >> [v2:172.21.15.203:6832/830097474,v1:172.21.15.203:6833/830097474] conn(0x7f5ab806a050 0x7f5ab806c510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:35:49.913 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:49.910+0000 7f5ac97fa640 1 -- 172.21.15.43:0/2036206972 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 6871+0+0 (secure 0 0 0) 0x7f5ac0087b30 con 0x7f5acc0cd6b0 2023-08-09T13:35:49.914 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:49.911+0000 7f5ad0ccb640 1 --2- 172.21.15.43:0/2036206972 >> [v2:172.21.15.203:6832/830097474,v1:172.21.15.203:6833/830097474] conn(0x7f5ab806a050 0x7f5ab806c510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:35:49.915 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:49.913+0000 7f5ad0ccb640 1 --2- 172.21.15.43:0/2036206972 >> [v2:172.21.15.203:6832/830097474,v1:172.21.15.203:6833/830097474] conn(0x7f5ab806a050 0x7f5ab806c510 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7f5abc036000 tx=0x7f5abc02e000 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-09T13:35:49.917 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:49.915+0000 7f5ac97fa640 1 -- 172.21.15.43:0/2036206972 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178066 (secure 0 0 0) 0x7f5acc0b26a0 con 0x7f5acc0cd6b0 2023-08-09T13:35:50.124 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:50.122+0000 7f5ad1ccd640 1 -- 172.21.15.43:0/2036206972 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7f5acc0b26a0 con 0x7f5acc0cd6b0 2023-08-09T13:35:50.125 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:50.122+0000 7f5ac97fa640 1 -- 172.21.15.43:0/2036206972 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v26) v1 ==== 80+0+0 (secure 0 0 0) 0x7f5acc0b26a0 con 0x7f5acc0cd6b0 2023-08-09T13:35:50.125 INFO:teuthology.orchestra.run.smithi043.stderr:listed 0 entries 2023-08-09T13:35:50.127 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:50.125+0000 7f5ad1ccd640 1 -- 172.21.15.43:0/2036206972 >> [v2:172.21.15.203:6832/830097474,v1:172.21.15.203:6833/830097474] conn(0x7f5ab806a050 msgr2=0x7f5ab806c510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:35:50.128 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:50.125+0000 7f5ad1ccd640 1 --2- 172.21.15.43:0/2036206972 >> [v2:172.21.15.203:6832/830097474,v1:172.21.15.203:6833/830097474] conn(0x7f5ab806a050 0x7f5ab806c510 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7f5abc036000 tx=0x7f5abc02e000 comp rx=0 tx=0).stop 2023-08-09T13:35:50.128 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:50.125+0000 7f5ad1ccd640 1 -- 172.21.15.43:0/2036206972 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f5acc0cd6b0 msgr2=0x7f5acc0c9760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:35:50.128 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:50.125+0000 7f5ad1ccd640 1 --2- 172.21.15.43:0/2036206972 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f5acc0cd6b0 0x7f5acc0c9760 secure :-1 s=READY pgs=473 cs=0 l=1 rev1=1 crypto rx=0x7f5ac000c470 tx=0x7f5ac000c930 comp rx=0 tx=0).stop 2023-08-09T13:35:50.128 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:50.125+0000 7f5ad1ccd640 1 -- 172.21.15.43:0/2036206972 shutdown_connections 2023-08-09T13:35:50.128 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:50.125+0000 7f5ad1ccd640 1 --2- 172.21.15.43:0/2036206972 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f5acc15f7b0 0x7f5acc0c7c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:35:50.129 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:50.125+0000 7f5ad1ccd640 1 --2- 172.21.15.43:0/2036206972 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f5acc0cd6b0 0x7f5acc0c9760 unknown :-1 s=CLOSED pgs=473 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:35:50.129 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:50.125+0000 7f5ad1ccd640 1 --2- 172.21.15.43:0/2036206972 >> [v2:172.21.15.203:6832/830097474,v1:172.21.15.203:6833/830097474] conn(0x7f5ab806a050 0x7f5ab806c510 unknown :-1 s=CLOSED pgs=233 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:35:50.129 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:50.125+0000 7f5ad1ccd640 1 --2- 172.21.15.43:0/2036206972 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f5acc0b1050 0x7f5acc1f4d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:35:50.129 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:50.125+0000 7f5ad1ccd640 1 -- 172.21.15.43:0/2036206972 >> 172.21.15.43:0/2036206972 conn(0x7f5acc06a2b0 msgr2=0x7f5acc0cf900 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-09T13:35:50.130 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:50.126+0000 7f5ad1ccd640 1 -- 172.21.15.43:0/2036206972 shutdown_connections 2023-08-09T13:35:50.130 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:50.126+0000 7f5ad1ccd640 1 -- 172.21.15.43:0/2036206972 wait complete. 2023-08-09T13:35:50.148 INFO:tasks.cephfs.fuse_mount:Running fusermount -u on ubuntu@smithi203.front.sepia.ceph.com... 2023-08-09T13:35:50.149 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-09T13:35:50.150 DEBUG:teuthology.orchestra.run.smithi203:> sudo fusermount -u /home/ubuntu/cephtest/mnt.3 2023-08-09T13:35:50.258 INFO:teuthology.orchestra.run:waiting for 300 2023-08-09T13:35:50.261 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi203.front.sepia.ceph.com 2023-08-09T13:35:50.261 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-09T13:35:50.262 DEBUG:teuthology.orchestra.run.smithi203:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.3) 2023-08-09T13:35:50.317 INFO:tasks.cephfs.cephfs_test_case:[, , , ] 2023-08-09T13:35:50.317 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-08-09T13:35:50.554 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:50.551+0000 7f8c4afa2640 1 Processor -- start 2023-08-09T13:35:50.555 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:50.552+0000 7f8c4afa2640 1 -- start start 2023-08-09T13:35:50.556 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:50.553+0000 7f8c4afa2640 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7f8c440b1050 0x7f8c440b4760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:35:50.556 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:50.553+0000 7f8c4afa2640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f8c440cd6b0 0x7f8c440d0e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:35:50.556 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:50.553+0000 7f8c4afa2640 1 -- --> v1:172.21.15.43:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8c440b7bf0 con 0x7f8c4415f7b0 2023-08-09T13:35:50.556 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:50.553+0000 7f8c4afa2640 1 -- --> v2:172.21.15.203:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8c440b7ed0 con 0x7f8c440b1050 2023-08-09T13:35:50.557 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:50.553+0000 7f8c4afa2640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8c440d0370 con 0x7f8c440cd6b0 2023-08-09T13:35:50.557 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:50.554+0000 7f8c49fa0640 1 --1- >> v1:172.21.15.43:6789/0 conn(0x7f8c4415f7b0 0x7f8c440ccfa0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.43:6789/0 says I am v1:172.21.15.43:48590/0 (socket says 172.21.15.43:48590) 2023-08-09T13:35:50.557 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:50.554+0000 7f8c49fa0640 1 -- 172.21.15.43:0/3307818172 learned_addr learned my addr 172.21.15.43:0/3307818172 (peer_addr_for_me v1:172.21.15.43:0/0) 2023-08-09T13:35:50.558 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:50.554+0000 7f8c4979f640 1 --2- 172.21.15.43:0/3307818172 >> v2:172.21.15.203:3300/0 conn(0x7f8c440b1050 0x7f8c440b4760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:35:50.559 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:50.554+0000 7f8c48f9e640 1 -- 172.21.15.43:0/3307818172 <== mon.0 v1:172.21.15.43:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3819178875 0 0) 0x7f8c440b7bf0 con 0x7f8c4415f7b0 2023-08-09T13:35:50.559 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:50.554+0000 7f8c4a7a1640 1 --2- 172.21.15.43:0/3307818172 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f8c440cd6b0 0x7f8c440d0e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:35:50.559 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:50.554+0000 7f8c48f9e640 1 -- 172.21.15.43:0/3307818172 --> v1:172.21.15.43:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f8c30003520 con 0x7f8c4415f7b0 2023-08-09T13:35:50.559 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:50.554+0000 7f8c48f9e640 1 -- 172.21.15.43:0/3307818172 <== mon.0 v1:172.21.15.43:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2499546009 0 0) 0x7f8c30003520 con 0x7f8c4415f7b0 2023-08-09T13:35:50.559 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:50.554+0000 7f8c48f9e640 1 -- 172.21.15.43:0/3307818172 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f8c440cd6b0 msgr2=0x7f8c440d0e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-09T13:35:50.559 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:50.554+0000 7f8c48f9e640 1 --2- 172.21.15.43:0/3307818172 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f8c440cd6b0 0x7f8c440d0e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:35:50.559 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:50.554+0000 7f8c48f9e640 1 -- 172.21.15.43:0/3307818172 >> v2:172.21.15.203:3300/0 conn(0x7f8c440b1050 msgr2=0x7f8c440b4760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-09T13:35:50.560 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:50.554+0000 7f8c48f9e640 1 --2- 172.21.15.43:0/3307818172 >> v2:172.21.15.203:3300/0 conn(0x7f8c440b1050 0x7f8c440b4760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:35:50.560 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:50.554+0000 7f8c48f9e640 1 -- 172.21.15.43:0/3307818172 --> v1:172.21.15.43:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8c440d0650 con 0x7f8c4415f7b0 2023-08-09T13:35:50.560 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:50.554+0000 7f8c48f9e640 1 -- 172.21.15.43:0/3307818172 <== mon.0 v1:172.21.15.43:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1285150740 0 0) 0x7f8c400019f0 con 0x7f8c4415f7b0 2023-08-09T13:35:50.560 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:50.554+0000 7f8c48f9e640 1 -- 172.21.15.43:0/3307818172 >> v1:172.21.15.43:6789/0 conn(0x7f8c4415f7b0 legacy=0x7f8c440ccfa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:35:50.560 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:50.554+0000 7f8c48f9e640 1 --2- 172.21.15.43:0/3307818172 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f8c30003970 0x7f8c30003d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:35:50.560 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:50.555+0000 7f8c48f9e640 1 --2- 172.21.15.43:0/3307818172 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f8c300047c0 0x7f8c30004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:35:50.560 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:50.555+0000 7f8c48f9e640 1 --2- 172.21.15.43:0/3307818172 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f8c30005180 0x7f8c3000b980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:35:50.561 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:50.555+0000 7f8c48f9e640 1 -- 172.21.15.43:0/3307818172 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8c440b7ed0 con 0x7f8c30005180 2023-08-09T13:35:50.561 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:50.555+0000 7f8c48f9e640 1 -- 172.21.15.43:0/3307818172 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8c440d0370 con 0x7f8c300047c0 2023-08-09T13:35:50.561 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:50.555+0000 7f8c48f9e640 1 -- 172.21.15.43:0/3307818172 --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8c3000bfc0 con 0x7f8c30003970 2023-08-09T13:35:50.561 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:50.555+0000 7f8c4a7a1640 1 --2- 172.21.15.43:0/3307818172 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f8c30005180 0x7f8c3000b980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:35:50.561 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:50.555+0000 7f8c49fa0640 1 --2- 172.21.15.43:0/3307818172 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f8c30003970 0x7f8c30003d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:35:50.561 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:50.555+0000 7f8c4979f640 1 --2- 172.21.15.43:0/3307818172 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f8c300047c0 0x7f8c30004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:35:50.561 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:50.555+0000 7f8c4a7a1640 1 -- 172.21.15.43:0/3307818172 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f8c30003970 msgr2=0x7f8c30003d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-09T13:35:50.562 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:50.556+0000 7f8c4a7a1640 1 --2- 172.21.15.43:0/3307818172 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f8c30003970 0x7f8c30003d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:35:50.562 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:50.556+0000 7f8c4a7a1640 1 -- 172.21.15.43:0/3307818172 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f8c300047c0 msgr2=0x7f8c30004c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-09T13:35:50.562 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:50.556+0000 7f8c4a7a1640 1 --2- 172.21.15.43:0/3307818172 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f8c300047c0 0x7f8c30004c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:35:50.562 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:50.556+0000 7f8c4a7a1640 1 -- 172.21.15.43:0/3307818172 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8c30003520 con 0x7f8c30005180 2023-08-09T13:35:50.562 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:50.556+0000 7f8c49fa0640 1 --2- 172.21.15.43:0/3307818172 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f8c30003970 0x7f8c30003d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-08-09T13:35:50.563 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:50.557+0000 7f8c4a7a1640 1 --2- 172.21.15.43:0/3307818172 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f8c30005180 0x7f8c3000b980 secure :-1 s=READY pgs=475 cs=0 l=1 rev1=1 crypto rx=0x7f8c3c00b930 tx=0x7f8c300057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f3d1f1986954abd1 server_cookie=0 in_seq=0 out_seq=0 2023-08-09T13:35:50.563 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:50.557+0000 7f8c48f9e640 1 -- 172.21.15.43:0/3307818172 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8c3c034610 con 0x7f8c30005180 2023-08-09T13:35:50.563 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:50.558+0000 7f8c48f9e640 1 -- 172.21.15.43:0/3307818172 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8c3c02e070 con 0x7f8c30005180 2023-08-09T13:35:50.564 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:50.558+0000 7f8c48f9e640 1 -- 172.21.15.43:0/3307818172 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8c3c034990 con 0x7f8c30005180 2023-08-09T13:35:50.564 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:50.558+0000 7f8c4afa2640 1 -- 172.21.15.43:0/3307818172 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f8c30005180 msgr2=0x7f8c3000b980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:35:50.564 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:50.558+0000 7f8c4afa2640 1 --2- 172.21.15.43:0/3307818172 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f8c30005180 0x7f8c3000b980 secure :-1 s=READY pgs=475 cs=0 l=1 rev1=1 crypto rx=0x7f8c3c00b930 tx=0x7f8c300057f0 comp rx=0 tx=0).stop 2023-08-09T13:35:50.564 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:50.558+0000 7f8c4a7a1640 1 -- 172.21.15.43:0/3307818172 reap_dead start 2023-08-09T13:35:50.564 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:50.558+0000 7f8c4afa2640 1 -- 172.21.15.43:0/3307818172 shutdown_connections 2023-08-09T13:35:50.564 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:50.558+0000 7f8c4afa2640 1 -- 172.21.15.43:0/3307818172 >> 172.21.15.43:0/3307818172 conn(0x7f8c4406a2b0 msgr2=0x7f8c4406a690 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-09T13:35:50.565 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:50.558+0000 7f8c4afa2640 1 -- 172.21.15.43:0/3307818172 shutdown_connections 2023-08-09T13:35:50.565 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:50.558+0000 7f8c4afa2640 1 -- 172.21.15.43:0/3307818172 wait complete. 2023-08-09T13:35:50.565 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:50.560+0000 7f8c4afa2640 1 Processor -- start 2023-08-09T13:35:50.565 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:50.560+0000 7f8c4afa2640 1 -- start start 2023-08-09T13:35:50.566 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:50.561+0000 7f8c4afa2640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f8c4415dbe0 0x7f8c441517d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:35:50.566 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:50.561+0000 7f8c4afa2640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f8c441f5140 0x7f8c441f1010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:35:50.567 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:50.561+0000 7f8c4afa2640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f8c4415dfc0 0x7f8c441f1550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:35:50.567 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:50.561+0000 7f8c4afa2640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8c4414e9e0 con 0x7f8c441f5140 2023-08-09T13:35:50.567 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:50.561+0000 7f8c4afa2640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8c44153600 con 0x7f8c4415dfc0 2023-08-09T13:35:50.567 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:50.561+0000 7f8c4afa2640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8c441f5640 con 0x7f8c4415dbe0 2023-08-09T13:35:50.567 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:50.561+0000 7f8c4979f640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f8c441f5140 0x7f8c441f1010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:35:50.567 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:50.562+0000 7f8c4979f640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f8c441f5140 0x7f8c441f1010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:35632/0 (socket says 172.21.15.43:35632) 2023-08-09T13:35:50.567 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:50.562+0000 7f8c4979f640 1 -- 172.21.15.43:0/3779377714 learned_addr learned my addr 172.21.15.43:0/3779377714 (peer_addr_for_me v2:172.21.15.43:0/0) 2023-08-09T13:35:50.568 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:50.562+0000 7f8c4979f640 1 -- 172.21.15.43:0/3779377714 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f8c4415dbe0 msgr2=0x7f8c441517d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:35:50.568 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:50.562+0000 7f8c4979f640 1 --2- 172.21.15.43:0/3779377714 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f8c4415dbe0 0x7f8c441517d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:35:50.568 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:50.562+0000 7f8c4a7a1640 1 --2- 172.21.15.43:0/3779377714 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f8c4415dfc0 0x7f8c441f1550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:35:50.568 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:50.562+0000 7f8c4979f640 1 -- 172.21.15.43:0/3779377714 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f8c4415dfc0 msgr2=0x7f8c441f1550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:35:50.568 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:50.562+0000 7f8c4979f640 1 --2- 172.21.15.43:0/3779377714 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f8c4415dfc0 0x7f8c441f1550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:35:50.568 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:50.562+0000 7f8c4979f640 1 -- 172.21.15.43:0/3779377714 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8c441f5920 con 0x7f8c441f5140 2023-08-09T13:35:50.568 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:50.562+0000 7f8c4979f640 1 --2- 172.21.15.43:0/3779377714 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f8c441f5140 0x7f8c441f1010 secure :-1 s=READY pgs=476 cs=0 l=1 rev1=1 crypto rx=0x7f8c3400f650 tx=0x7f8c3400fb10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-09T13:35:50.569 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:50.563+0000 7f8c3a7fc640 1 -- 172.21.15.43:0/3779377714 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8c34004020 con 0x7f8c441f5140 2023-08-09T13:35:50.569 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:50.563+0000 7f8c3a7fc640 1 -- 172.21.15.43:0/3779377714 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8c340041a0 con 0x7f8c441f5140 2023-08-09T13:35:50.569 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:50.563+0000 7f8c3a7fc640 1 -- 172.21.15.43:0/3779377714 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8c3400d740 con 0x7f8c441f5140 2023-08-09T13:35:50.569 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:50.563+0000 7f8c4afa2640 1 -- 172.21.15.43:0/3779377714 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8c441f5c00 con 0x7f8c441f5140 2023-08-09T13:35:50.569 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:50.563+0000 7f8c4afa2640 1 -- 172.21.15.43:0/3779377714 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8c44151d10 con 0x7f8c441f5140 2023-08-09T13:35:50.569 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:50.564+0000 7f8c4afa2640 1 -- 172.21.15.43:0/3779377714 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8c440739f0 con 0x7f8c441f5140 2023-08-09T13:35:50.570 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:50.565+0000 7f8c3a7fc640 1 -- 172.21.15.43:0/3779377714 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 5) v1 ==== 88341+0+0 (secure 0 0 0) 0x7f8c3400d8a0 con 0x7f8c441f5140 2023-08-09T13:35:50.570 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:50.566+0000 7f8c3a7fc640 1 --2- 172.21.15.43:0/3779377714 >> [v2:172.21.15.203:6832/830097474,v1:172.21.15.203:6833/830097474] conn(0x7f8c2806a050 0x7f8c2806c510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:35:50.570 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:50.566+0000 7f8c3a7fc640 1 -- 172.21.15.43:0/3779377714 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 6871+0+0 (secure 0 0 0) 0x7f8c34089000 con 0x7f8c441f5140 2023-08-09T13:35:50.570 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:50.567+0000 7f8c49fa0640 1 --2- 172.21.15.43:0/3779377714 >> [v2:172.21.15.203:6832/830097474,v1:172.21.15.203:6833/830097474] conn(0x7f8c2806a050 0x7f8c2806c510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:35:50.571 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:50.568+0000 7f8c49fa0640 1 --2- 172.21.15.43:0/3779377714 >> [v2:172.21.15.203:6832/830097474,v1:172.21.15.203:6833/830097474] conn(0x7f8c2806a050 0x7f8c2806c510 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7f8c40007230 tx=0x7f8c40002170 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-09T13:35:50.572 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:50.570+0000 7f8c3a7fc640 1 -- 172.21.15.43:0/3779377714 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178066 (secure 0 0 0) 0x7f8c34054320 con 0x7f8c441f5140 2023-08-09T13:35:50.808 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:50.805+0000 7f8c4afa2640 1 -- 172.21.15.43:0/3779377714 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f8c440b1b60 con 0x7f8c441f5140 2023-08-09T13:35:50.809 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:50.806+0000 7f8c3a7fc640 1 -- 172.21.15.43:0/3779377714 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 9 v9) v1 ==== 93+0+4874 (secure 0 0 0) 0x7f8c34057940 con 0x7f8c441f5140 2023-08-09T13:35:50.809 INFO:teuthology.orchestra.run.smithi043.stdout: 2023-08-09T13:35:50.810 INFO:teuthology.orchestra.run.smithi043.stdout:{"epoch":9,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4236,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.43:6833/3917539953","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6832","nonce":3917539953},{"type":"v1","addr":"172.21.15.43:6833","nonce":3917539953}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2},{"gid":4247,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.203:6837/2467794159","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6835","nonce":2467794159},{"type":"v1","addr":"172.21.15.203:6837","nonce":2467794159}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":4248,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.43:6835/1818380189","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6834","nonce":1818380189},{"type":"v1","addr":"172.21.15.43:6835","nonce":1818380189}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":9,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-08-09T13:34:02.575398+0000","modified":"2023-08-09T13:34:31.841945+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":4250},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4250":{"gid":4250,"name":"d","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.203:6836/798312463","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6834","nonce":798312463},{"type":"v1","addr":"172.21.15.203:6836","nonce":798312463}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[4,5],"metadata_pool":3,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-08-09T13:35:50.810 INFO:teuthology.orchestra.run.smithi043.stderr:dumped fsmap epoch 9 2023-08-09T13:35:50.813 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:50.810+0000 7f8c4afa2640 1 -- 172.21.15.43:0/3779377714 >> [v2:172.21.15.203:6832/830097474,v1:172.21.15.203:6833/830097474] conn(0x7f8c2806a050 msgr2=0x7f8c2806c510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:35:50.813 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:50.810+0000 7f8c4afa2640 1 --2- 172.21.15.43:0/3779377714 >> [v2:172.21.15.203:6832/830097474,v1:172.21.15.203:6833/830097474] conn(0x7f8c2806a050 0x7f8c2806c510 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7f8c40007230 tx=0x7f8c40002170 comp rx=0 tx=0).stop 2023-08-09T13:35:50.813 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:50.810+0000 7f8c4afa2640 1 -- 172.21.15.43:0/3779377714 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f8c441f5140 msgr2=0x7f8c441f1010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:35:50.813 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:50.810+0000 7f8c4afa2640 1 --2- 172.21.15.43:0/3779377714 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f8c441f5140 0x7f8c441f1010 secure :-1 s=READY pgs=476 cs=0 l=1 rev1=1 crypto rx=0x7f8c3400f650 tx=0x7f8c3400fb10 comp rx=0 tx=0).stop 2023-08-09T13:35:50.813 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:50.810+0000 7f8c4afa2640 1 -- 172.21.15.43:0/3779377714 shutdown_connections 2023-08-09T13:35:50.813 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:50.810+0000 7f8c4afa2640 1 --2- 172.21.15.43:0/3779377714 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f8c4415dfc0 0x7f8c441f1550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:35:50.814 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:50.810+0000 7f8c4afa2640 1 --2- 172.21.15.43:0/3779377714 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f8c441f5140 0x7f8c441f1010 unknown :-1 s=CLOSED pgs=476 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:35:50.814 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:50.810+0000 7f8c4afa2640 1 --2- 172.21.15.43:0/3779377714 >> [v2:172.21.15.203:6832/830097474,v1:172.21.15.203:6833/830097474] conn(0x7f8c2806a050 0x7f8c2806c510 unknown :-1 s=CLOSED pgs=234 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:35:50.814 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:50.810+0000 7f8c4afa2640 1 --2- 172.21.15.43:0/3779377714 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f8c4415dbe0 0x7f8c441517d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:35:50.814 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:50.810+0000 7f8c4afa2640 1 -- 172.21.15.43:0/3779377714 >> 172.21.15.43:0/3779377714 conn(0x7f8c4406a2b0 msgr2=0x7f8c440cf900 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-09T13:35:50.814 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:50.810+0000 7f8c4afa2640 1 -- 172.21.15.43:0/3779377714 shutdown_connections 2023-08-09T13:35:50.814 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:50.810+0000 7f8c4afa2640 1 -- 172.21.15.43:0/3779377714 wait complete. 2023-08-09T13:35:50.831 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-08-09T13:35:51.054 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:51.051+0000 7fa5d759e640 1 Processor -- start 2023-08-09T13:35:51.054 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:51.052+0000 7fa5d759e640 1 -- start start 2023-08-09T13:35:51.056 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:51.053+0000 7fa5d759e640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fa5d815c620 0x7fa5d80cfe70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:35:51.056 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:51.053+0000 7fa5d759e640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7fa5d80cf4f0 0x7fa5d80d07e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:35:51.056 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:51.053+0000 7fa5d759e640 1 -- --> v1:172.21.15.203:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa5d80b2330 con 0x7fa5d80d0d20 2023-08-09T13:35:51.056 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:51.053+0000 7fa5d759e640 1 -- --> v2:172.21.15.43:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa5d80d2ec0 con 0x7fa5d80cf4f0 2023-08-09T13:35:51.056 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:51.053+0000 7fa5d759e640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa5d80d0500 con 0x7fa5d815c620 2023-08-09T13:35:51.057 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:51.054+0000 7fa5cffff640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7fa5d80cf4f0 0x7fa5d80d07e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:35:51.057 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:51.054+0000 7fa5cffff640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7fa5d80cf4f0 0x7fa5d80d07e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:35646/0 (socket says 172.21.15.43:35646) 2023-08-09T13:35:51.057 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:51.054+0000 7fa5cffff640 1 -- 172.21.15.43:0/3253344316 learned_addr learned my addr 172.21.15.43:0/3253344316 (peer_addr_for_me v2:172.21.15.43:0/0) 2023-08-09T13:35:51.058 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:51.054+0000 7fa5d659c640 1 --2- 172.21.15.43:0/3253344316 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fa5d815c620 0x7fa5d80cfe70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:35:51.058 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:51.054+0000 7fa5cffff640 1 -- 172.21.15.43:0/3253344316 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fa5d815c620 msgr2=0x7fa5d80cfe70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-09T13:35:51.058 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:51.054+0000 7fa5cffff640 1 --2- 172.21.15.43:0/3253344316 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fa5d815c620 0x7fa5d80cfe70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:35:51.058 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:51.054+0000 7fa5cffff640 1 -- 172.21.15.43:0/3253344316 >> v1:172.21.15.203:6789/0 conn(0x7fa5d80d0d20 legacy=0x7fa5d80cce40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:35:51.058 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:51.054+0000 7fa5cffff640 1 -- 172.21.15.43:0/3253344316 --> v2:172.21.15.43:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa5d80b1a00 con 0x7fa5d80cf4f0 2023-08-09T13:35:51.059 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:51.056+0000 7fa5cffff640 1 --2- 172.21.15.43:0/3253344316 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa5d80cf4f0 0x7fa5d80d07e0 secure :-1 s=READY pgs=477 cs=0 l=1 rev1=1 crypto rx=0x7fa5c0009870 tx=0x7fa5c002b860 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7690a01247b97eb6 server_cookie=0 in_seq=0 out_seq=0 2023-08-09T13:35:51.059 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:51.056+0000 7fa5d5d9b640 1 -- 172.21.15.43:0/3253344316 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa5c00317c0 con 0x7fa5d80cf4f0 2023-08-09T13:35:51.059 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:51.056+0000 7fa5d5d9b640 1 -- 172.21.15.43:0/3253344316 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa5c0031920 con 0x7fa5d80cf4f0 2023-08-09T13:35:51.059 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:51.056+0000 7fa5d5d9b640 1 -- 172.21.15.43:0/3253344316 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa5c0031cb0 con 0x7fa5d80cf4f0 2023-08-09T13:35:51.060 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:51.056+0000 7fa5d759e640 1 -- 172.21.15.43:0/3253344316 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa5d80cf4f0 msgr2=0x7fa5d80d07e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:35:51.060 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:51.056+0000 7fa5d759e640 1 --2- 172.21.15.43:0/3253344316 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa5d80cf4f0 0x7fa5d80d07e0 secure :-1 s=READY pgs=477 cs=0 l=1 rev1=1 crypto rx=0x7fa5c0009870 tx=0x7fa5c002b860 comp rx=0 tx=0).stop 2023-08-09T13:35:51.060 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:51.057+0000 7fa5d759e640 1 -- 172.21.15.43:0/3253344316 shutdown_connections 2023-08-09T13:35:51.060 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:51.057+0000 7fa5d759e640 1 --2- 172.21.15.43:0/3253344316 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa5d80cf4f0 0x7fa5d80d07e0 unknown :-1 s=CLOSED pgs=477 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:35:51.060 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:51.057+0000 7fa5d759e640 1 --2- 172.21.15.43:0/3253344316 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fa5d815c620 0x7fa5d80cfe70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:35:51.060 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:51.057+0000 7fa5d759e640 1 -- 172.21.15.43:0/3253344316 >> 172.21.15.43:0/3253344316 conn(0x7fa5d809b650 msgr2=0x7fa5d80d22a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-09T13:35:51.060 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:51.057+0000 7fa5d759e640 1 -- 172.21.15.43:0/3253344316 shutdown_connections 2023-08-09T13:35:51.061 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:51.057+0000 7fa5d759e640 1 -- 172.21.15.43:0/3253344316 wait complete. 2023-08-09T13:35:51.061 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:51.059+0000 7fa5d759e640 1 Processor -- start 2023-08-09T13:35:51.061 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:51.059+0000 7fa5d759e640 1 -- start start 2023-08-09T13:35:51.062 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:51.060+0000 7fa5d759e640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa5d80cf4f0 0x7fa5d80c88c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:35:51.062 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:51.060+0000 7fa5d759e640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fa5d80d0d20 0x7fa5d81f1560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:35:51.062 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:51.060+0000 7fa5d759e640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fa5d815c620 0x7fa5d815eaf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:35:51.063 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:51.060+0000 7fa5d759e640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa5d80b74b0 con 0x7fa5d80cf4f0 2023-08-09T13:35:51.063 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:51.060+0000 7fa5d759e640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa5d80c62e0 con 0x7fa5d80d0d20 2023-08-09T13:35:51.063 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:51.060+0000 7fa5d759e640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa5d80b6ef0 con 0x7fa5d815c620 2023-08-09T13:35:51.063 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:51.060+0000 7fa5d659c640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa5d80cf4f0 0x7fa5d80c88c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:35:51.063 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:51.060+0000 7fa5d659c640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa5d80cf4f0 0x7fa5d80c88c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:35650/0 (socket says 172.21.15.43:35650) 2023-08-09T13:35:51.063 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:51.060+0000 7fa5d659c640 1 -- 172.21.15.43:0/976749306 learned_addr learned my addr 172.21.15.43:0/976749306 (peer_addr_for_me v2:172.21.15.43:0/0) 2023-08-09T13:35:51.064 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:51.060+0000 7fa5d6d9d640 1 --2- 172.21.15.43:0/976749306 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fa5d815c620 0x7fa5d815eaf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:35:51.064 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:51.060+0000 7fa5d659c640 1 -- 172.21.15.43:0/976749306 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fa5d815c620 msgr2=0x7fa5d815eaf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:35:51.065 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:51.060+0000 7fa5d659c640 1 --2- 172.21.15.43:0/976749306 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fa5d815c620 0x7fa5d815eaf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:35:51.065 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:51.060+0000 7fa5d659c640 1 -- 172.21.15.43:0/976749306 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fa5d80d0d20 msgr2=0x7fa5d81f1560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:35:51.065 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:51.060+0000 7fa5cffff640 1 --2- 172.21.15.43:0/976749306 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fa5d80d0d20 0x7fa5d81f1560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:35:51.065 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:51.061+0000 7fa5d659c640 1 --2- 172.21.15.43:0/976749306 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fa5d80d0d20 0x7fa5d81f1560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:35:51.065 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:51.061+0000 7fa5d659c640 1 -- 172.21.15.43:0/976749306 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa5c00094d0 con 0x7fa5d80cf4f0 2023-08-09T13:35:51.065 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:51.061+0000 7fa5d659c640 1 --2- 172.21.15.43:0/976749306 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa5d80cf4f0 0x7fa5d80c88c0 secure :-1 s=READY pgs=478 cs=0 l=1 rev1=1 crypto rx=0x7fa5c800d800 tx=0x7fa5c800dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-09T13:35:51.065 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:51.061+0000 7fa5ceffd640 1 -- 172.21.15.43:0/976749306 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa5c800b6c0 con 0x7fa5d80cf4f0 2023-08-09T13:35:51.066 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:51.061+0000 7fa5ceffd640 1 -- 172.21.15.43:0/976749306 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa5c800b820 con 0x7fa5d80cf4f0 2023-08-09T13:35:51.066 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:51.061+0000 7fa5ceffd640 1 -- 172.21.15.43:0/976749306 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa5c800bbd0 con 0x7fa5d80cf4f0 2023-08-09T13:35:51.066 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:51.061+0000 7fa5d759e640 1 -- 172.21.15.43:0/976749306 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa5d80b71d0 con 0x7fa5d80cf4f0 2023-08-09T13:35:51.066 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:51.061+0000 7fa5d759e640 1 -- 172.21.15.43:0/976749306 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa5d81f1aa0 con 0x7fa5d80cf4f0 2023-08-09T13:35:51.067 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:51.062+0000 7fa5d759e640 1 -- 172.21.15.43:0/976749306 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa5d80c5dd0 con 0x7fa5d80cf4f0 2023-08-09T13:35:51.067 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:51.063+0000 7fa5ceffd640 1 -- 172.21.15.43:0/976749306 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 5) v1 ==== 88341+0+0 (secure 0 0 0) 0x7fa5c800bd30 con 0x7fa5d80cf4f0 2023-08-09T13:35:51.067 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:51.064+0000 7fa5ceffd640 1 --2- 172.21.15.43:0/976749306 >> [v2:172.21.15.203:6832/830097474,v1:172.21.15.203:6833/830097474] conn(0x7fa5b806a050 0x7fa5b806c510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:35:51.067 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:51.064+0000 7fa5ceffd640 1 -- 172.21.15.43:0/976749306 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 6871+0+0 (secure 0 0 0) 0x7fa5c8088ea0 con 0x7fa5d80cf4f0 2023-08-09T13:35:51.067 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:51.065+0000 7fa5cffff640 1 --2- 172.21.15.43:0/976749306 >> [v2:172.21.15.203:6832/830097474,v1:172.21.15.203:6833/830097474] conn(0x7fa5b806a050 0x7fa5b806c510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:35:51.068 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:51.066+0000 7fa5cffff640 1 --2- 172.21.15.43:0/976749306 >> [v2:172.21.15.203:6832/830097474,v1:172.21.15.203:6833/830097474] conn(0x7fa5b806a050 0x7fa5b806c510 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7fa5c0036000 tx=0x7fa5c003d040 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-09T13:35:51.071 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:51.068+0000 7fa5ceffd640 1 -- 172.21.15.43:0/976749306 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178066 (secure 0 0 0) 0x7fa5d80c5dd0 con 0x7fa5d80cf4f0 2023-08-09T13:35:51.275 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:51.273+0000 7fa5d759e640 1 -- 172.21.15.43:0/976749306 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fa5d80c5dd0 con 0x7fa5d80cf4f0 2023-08-09T13:35:51.279 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:51.277+0000 7fa5ceffd640 1 -- 172.21.15.43:0/976749306 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v26) v1 ==== 74+0+20146 (secure 0 0 0) 0x7fa5d80c5dd0 con 0x7fa5d80cf4f0 2023-08-09T13:35:51.280 INFO:teuthology.orchestra.run.smithi043.stdout: 2023-08-09T13:35:51.280 INFO:teuthology.orchestra.run.smithi043.stdout:{"epoch":26,"fsid":"daac87ee-1337-4250-a955-aea4e408a39b","created":"2023-08-09T13:33:41.701198+0000","modified":"2023-08-09T13:34:48.711911+0000","last_up_change":"2023-08-09T13:33:53.783453+0000","last_in_change":"2023-08-09T13:33:45.148904+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":5,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":"rbd","create_time":"2023-08-09T13:33:54.365137+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":".mgr","create_time":"2023-08-09T13:33:54.831803+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_metadata","create_time":"2023-08-09T13:34:00.207974+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data","create_time":"2023-08-09T13:34:01.614765+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":5,"pool_name":"cephfs_data_ec","create_time":"2023-08-09T13:34:05.239583+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"24","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"cbd49ecd-e896-4fbd-864e-5c37a829b165","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6800","nonce":502667335},{"type":"v1","addr":"172.21.15.43:6801","nonce":502667335}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6802","nonce":502667335},{"type":"v1","addr":"172.21.15.43:6803","nonce":502667335}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6806","nonce":502667335},{"type":"v1","addr":"172.21.15.43:6807","nonce":502667335}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6804","nonce":502667335},{"type":"v1","addr":"172.21.15.43:6805","nonce":502667335}]},"public_addr":"172.21.15.43:6801/502667335","cluster_addr":"172.21.15.43:6803/502667335","heartbeat_back_addr":"172.21.15.43:6807/502667335","heartbeat_front_addr":"172.21.15.43:6805/502667335","state":["exists","up"]},{"osd":1,"uuid":"75c7e0fe-1ec7-4fe8-a2f2-3661a65b8500","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6808","nonce":3289134610},{"type":"v1","addr":"172.21.15.43:6809","nonce":3289134610}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6810","nonce":3289134610},{"type":"v1","addr":"172.21.15.43:6812","nonce":3289134610}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6821","nonce":3289134610},{"type":"v1","addr":"172.21.15.43:6824","nonce":3289134610}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6815","nonce":3289134610},{"type":"v1","addr":"172.21.15.43:6818","nonce":3289134610}]},"public_addr":"172.21.15.43:6809/3289134610","cluster_addr":"172.21.15.43:6812/3289134610","heartbeat_back_addr":"172.21.15.43:6824/3289134610","heartbeat_front_addr":"172.21.15.43:6818/3289134610","state":["exists","up"]},{"osd":2,"uuid":"bb5c41e9-7f5f-4125-9b01-3bb8bfc2ea83","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6811","nonce":4079709736},{"type":"v1","addr":"172.21.15.43:6814","nonce":4079709736}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6817","nonce":4079709736},{"type":"v1","addr":"172.21.15.43:6820","nonce":4079709736}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6829","nonce":4079709736},{"type":"v1","addr":"172.21.15.43:6831","nonce":4079709736}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6823","nonce":4079709736},{"type":"v1","addr":"172.21.15.43:6826","nonce":4079709736}]},"public_addr":"172.21.15.43:6814/4079709736","cluster_addr":"172.21.15.43:6820/4079709736","heartbeat_back_addr":"172.21.15.43:6831/4079709736","heartbeat_front_addr":"172.21.15.43:6826/4079709736","state":["exists","up"]},{"osd":3,"uuid":"bfe4c8b4-694a-465c-87e6-2b2aab852613","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6813","nonce":1709538759},{"type":"v1","addr":"172.21.15.43:6816","nonce":1709538759}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6819","nonce":1709538759},{"type":"v1","addr":"172.21.15.43:6822","nonce":1709538759}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6828","nonce":1709538759},{"type":"v1","addr":"172.21.15.43:6830","nonce":1709538759}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6825","nonce":1709538759},{"type":"v1","addr":"172.21.15.43:6827","nonce":1709538759}]},"public_addr":"172.21.15.43:6816/1709538759","cluster_addr":"172.21.15.43:6822/1709538759","heartbeat_back_addr":"172.21.15.43:6830/1709538759","heartbeat_front_addr":"172.21.15.43:6827/1709538759","state":["exists","up"]},{"osd":4,"uuid":"fa2e8a77-aba1-4756-853e-0d174a5bdeba","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6800","nonce":4196866481},{"type":"v1","addr":"172.21.15.203:6801","nonce":4196866481}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6802","nonce":4196866481},{"type":"v1","addr":"172.21.15.203:6803","nonce":4196866481}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6806","nonce":4196866481},{"type":"v1","addr":"172.21.15.203:6807","nonce":4196866481}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6804","nonce":4196866481},{"type":"v1","addr":"172.21.15.203:6805","nonce":4196866481}]},"public_addr":"172.21.15.203:6801/4196866481","cluster_addr":"172.21.15.203:6803/4196866481","heartbeat_back_addr":"172.21.15.203:6807/4196866481","heartbeat_front_addr":"172.21.15.203:6805/4196866481","state":["exists","up"]},{"osd":5,"uuid":"81811e4b-4d24-4f51-a82c-f1d700a2a00d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6816","nonce":3059290081},{"type":"v1","addr":"172.21.15.203:6817","nonce":3059290081}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6818","nonce":3059290081},{"type":"v1","addr":"172.21.15.203:6819","nonce":3059290081}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6822","nonce":3059290081},{"type":"v1","addr":"172.21.15.203:6823","nonce":3059290081}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6820","nonce":3059290081},{"type":"v1","addr":"172.21.15.203:6821","nonce":3059290081}]},"public_addr":"172.21.15.203:6817/3059290081","cluster_addr":"172.21.15.203:6819/3059290081","heartbeat_back_addr":"172.21.15.203:6823/3059290081","heartbeat_front_addr":"172.21.15.203:6821/3059290081","state":["exists","up"]},{"osd":6,"uuid":"307bac4d-6a5b-443d-af75-56d695d7674f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6824","nonce":512155569},{"type":"v1","addr":"172.21.15.203:6825","nonce":512155569}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6826","nonce":512155569},{"type":"v1","addr":"172.21.15.203:6827","nonce":512155569}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6830","nonce":512155569},{"type":"v1","addr":"172.21.15.203:6831","nonce":512155569}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6828","nonce":512155569},{"type":"v1","addr":"172.21.15.203:6829","nonce":512155569}]},"public_addr":"172.21.15.203:6825/512155569","cluster_addr":"172.21.15.203:6827/512155569","heartbeat_back_addr":"172.21.15.203:6831/512155569","heartbeat_front_addr":"172.21.15.203:6829/512155569","state":["exists","up"]},{"osd":7,"uuid":"a95dc7ca-9f1b-4295-af70-68a22a2d7d5e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6808","nonce":3331450438},{"type":"v1","addr":"172.21.15.203:6809","nonce":3331450438}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6810","nonce":3331450438},{"type":"v1","addr":"172.21.15.203:6811","nonce":3331450438}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6814","nonce":3331450438},{"type":"v1","addr":"172.21.15.203:6815","nonce":3331450438}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6812","nonce":3331450438},{"type":"v1","addr":"172.21.15.203:6813","nonce":3331450438}]},"public_addr":"172.21.15.203:6809/3331450438","cluster_addr":"172.21.15.203:6811/3331450438","heartbeat_back_addr":"172.21.15.203:6815/3331450438","heartbeat_front_addr":"172.21.15.203:6813/3331450438","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-09T13:33:50.547652+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-09T13:33:50.589469+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-09T13:33:50.561724+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-09T13:33:50.599650+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-09T13:33:50.411855+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-09T13:33:50.219681+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-09T13:33:50.504664+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-09T13:33:50.441248+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"4.14","mappings":[{"from":1,"to":5}]},{"pgid":"4.29","mappings":[{"from":3,"to":2}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-08-09T13:35:51.282 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:51.279+0000 7fa5d759e640 1 -- 172.21.15.43:0/976749306 >> [v2:172.21.15.203:6832/830097474,v1:172.21.15.203:6833/830097474] conn(0x7fa5b806a050 msgr2=0x7fa5b806c510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:35:51.282 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:51.279+0000 7fa5d759e640 1 --2- 172.21.15.43:0/976749306 >> [v2:172.21.15.203:6832/830097474,v1:172.21.15.203:6833/830097474] conn(0x7fa5b806a050 0x7fa5b806c510 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7fa5c0036000 tx=0x7fa5c003d040 comp rx=0 tx=0).stop 2023-08-09T13:35:51.282 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:51.279+0000 7fa5d759e640 1 -- 172.21.15.43:0/976749306 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa5d80cf4f0 msgr2=0x7fa5d80c88c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:35:51.282 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:51.279+0000 7fa5d759e640 1 --2- 172.21.15.43:0/976749306 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa5d80cf4f0 0x7fa5d80c88c0 secure :-1 s=READY pgs=478 cs=0 l=1 rev1=1 crypto rx=0x7fa5c800d800 tx=0x7fa5c800dcc0 comp rx=0 tx=0).stop 2023-08-09T13:35:51.282 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:51.279+0000 7fa5d759e640 1 -- 172.21.15.43:0/976749306 shutdown_connections 2023-08-09T13:35:51.282 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:51.279+0000 7fa5d759e640 1 --2- 172.21.15.43:0/976749306 >> [v2:172.21.15.203:6832/830097474,v1:172.21.15.203:6833/830097474] conn(0x7fa5b806a050 0x7fa5b806c510 unknown :-1 s=CLOSED pgs=235 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:35:51.282 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:51.279+0000 7fa5d759e640 1 --2- 172.21.15.43:0/976749306 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fa5d815c620 0x7fa5d815eaf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:35:51.283 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:51.279+0000 7fa5d759e640 1 --2- 172.21.15.43:0/976749306 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fa5d80d0d20 0x7fa5d81f1560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:35:51.283 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:51.279+0000 7fa5d759e640 1 --2- 172.21.15.43:0/976749306 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa5d80cf4f0 0x7fa5d80c88c0 unknown :-1 s=CLOSED pgs=478 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:35:51.283 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:51.279+0000 7fa5d759e640 1 -- 172.21.15.43:0/976749306 >> 172.21.15.43:0/976749306 conn(0x7fa5d809b650 msgr2=0x7fa5d80cce00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-09T13:35:51.283 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:51.279+0000 7fa5d759e640 1 -- 172.21.15.43:0/976749306 shutdown_connections 2023-08-09T13:35:51.283 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:51.279+0000 7fa5d759e640 1 -- 172.21.15.43:0/976749306 wait complete. 2023-08-09T13:35:51.301 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': 'rbd', 'create_time': '2023-08-09T13:33:54.365137+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}, 'read_balance': {'score_acting': 2, 'score_stable': 2, 'optimal_score': 1, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': '.mgr', 'create_time': '2023-08-09T13:33:54.831803+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_metadata', 'create_time': '2023-08-09T13:34:00.207974+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data', 'create_time': '2023-08-09T13:34:01.614765+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 2, 'score_stable': 2, 'optimal_score': 1, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 5, 'pool_name': 'cephfs_data_ec', 'create_time': '2023-08-09T13:34:05.239583+0000', 'flags': 5, 'flags_names': 'hashpspool,ec_overwrites', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '24', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2023-08-09T13:35:51.302 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get rbd pg_num 2023-08-09T13:35:51.532 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:51.530+0000 7f29782fa640 1 Processor -- start 2023-08-09T13:35:51.533 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:51.530+0000 7f29782fa640 1 -- start start 2023-08-09T13:35:51.534 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:51.531+0000 7f29782fa640 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7f29700d04b0 0x7f29700cd6d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:35:51.534 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:51.531+0000 7f29782fa640 1 -- --> v1:172.21.15.43:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f29700b7bd0 con 0x7f29700b45f0 2023-08-09T13:35:51.534 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:51.532+0000 7f29782fa640 1 -- --> v1:172.21.15.203:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f29700d1c00 con 0x7f297015c550 2023-08-09T13:35:51.534 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:51.532+0000 7f29782fa640 1 -- --> v2:172.21.15.203:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f297015c930 con 0x7f29700d04b0 2023-08-09T13:35:51.535 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:51.532+0000 7f2975895640 1 --1- >> v1:172.21.15.43:6789/0 conn(0x7f29700b45f0 0x7f29700b59d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.43:6789/0 says I am v1:172.21.15.43:48594/0 (socket says 172.21.15.43:48594) 2023-08-09T13:35:51.536 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:51.532+0000 7f2975895640 1 -- 172.21.15.43:0/1300613760 learned_addr learned my addr 172.21.15.43:0/1300613760 (peer_addr_for_me v1:172.21.15.43:0/0) 2023-08-09T13:35:51.536 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:51.532+0000 7f2976897640 1 --2- 172.21.15.43:0/1300613760 >> v2:172.21.15.203:3300/0 conn(0x7f29700d04b0 0x7f29700cd6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:35:51.536 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:51.533+0000 7f2975094640 1 -- 172.21.15.43:0/1300613760 <== mon.0 v1:172.21.15.43:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3256300313 0 0) 0x7f29700b7bd0 con 0x7f29700b45f0 2023-08-09T13:35:51.536 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:51.533+0000 7f2975094640 1 -- 172.21.15.43:0/1300613760 --> v1:172.21.15.43:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f295c003520 con 0x7f29700b45f0 2023-08-09T13:35:51.536 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:51.533+0000 7f2976897640 1 -- 172.21.15.43:0/1300613760 >> v1:172.21.15.203:6789/0 conn(0x7f297015c550 legacy=0x7f29700b3ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:35:51.536 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:51.533+0000 7f2976897640 1 -- 172.21.15.43:0/1300613760 >> v1:172.21.15.43:6789/0 conn(0x7f29700b45f0 legacy=0x7f29700b59d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:35:51.536 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:51.533+0000 7f2976897640 1 -- 172.21.15.43:0/1300613760 --> v2:172.21.15.203:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f29700b54f0 con 0x7f29700d04b0 2023-08-09T13:35:51.537 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:51.535+0000 7f2976897640 1 --2- 172.21.15.43:0/1300613760 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f29700d04b0 0x7f29700cd6d0 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7f2968009d90 tx=0x7f296802bf40 comp rx=0 tx=0).ready entity=mon.1 client_cookie=9b53b1c73a2ce50a server_cookie=0 in_seq=0 out_seq=0 2023-08-09T13:35:51.538 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:51.535+0000 7f2975094640 1 -- 172.21.15.43:0/1300613760 <== mon.1 v2:172.21.15.203:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f296802ccf0 con 0x7f29700d04b0 2023-08-09T13:35:51.538 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:51.535+0000 7f2975094640 1 -- 172.21.15.43:0/1300613760 <== mon.1 v2:172.21.15.203:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f296802ce50 con 0x7f29700d04b0 2023-08-09T13:35:51.538 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:51.535+0000 7f2975094640 1 -- 172.21.15.43:0/1300613760 <== mon.1 v2:172.21.15.203:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2968034660 con 0x7f29700d04b0 2023-08-09T13:35:51.539 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:51.536+0000 7f29782fa640 1 -- 172.21.15.43:0/1300613760 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f29700d04b0 msgr2=0x7f29700cd6d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:35:51.539 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:51.536+0000 7f29782fa640 1 --2- 172.21.15.43:0/1300613760 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f29700d04b0 0x7f29700cd6d0 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7f2968009d90 tx=0x7f296802bf40 comp rx=0 tx=0).stop 2023-08-09T13:35:51.539 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:51.536+0000 7f29782fa640 1 -- 172.21.15.43:0/1300613760 shutdown_connections 2023-08-09T13:35:51.539 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:51.536+0000 7f29782fa640 1 --2- 172.21.15.43:0/1300613760 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f29700d04b0 0x7f29700cd6d0 unknown :-1 s=CLOSED pgs=185 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:35:51.539 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:51.536+0000 7f29782fa640 1 -- 172.21.15.43:0/1300613760 >> 172.21.15.43:0/1300613760 conn(0x7f2970071520 msgr2=0x7f29700ceee0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-09T13:35:51.539 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:51.536+0000 7f29782fa640 1 -- 172.21.15.43:0/1300613760 shutdown_connections 2023-08-09T13:35:51.539 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:51.536+0000 7f29782fa640 1 -- 172.21.15.43:0/1300613760 wait complete. 2023-08-09T13:35:51.540 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:51.538+0000 7f29782fa640 1 Processor -- start 2023-08-09T13:35:51.540 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:51.538+0000 7f29782fa640 1 -- start start 2023-08-09T13:35:51.542 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:51.539+0000 7f29782fa640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f29700b45f0 0x7f29700a9a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:35:51.542 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:51.539+0000 7f29782fa640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f29700d04b0 0x7f2970150870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:35:51.542 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:51.539+0000 7f29782fa640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f297015c550 0x7f29700aa650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:35:51.542 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:51.539+0000 7f29782fa640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f29700ab0d0 con 0x7f29700d04b0 2023-08-09T13:35:51.542 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:51.539+0000 7f29782fa640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f29700ab3b0 con 0x7f29700b45f0 2023-08-09T13:35:51.542 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:51.539+0000 7f29782fa640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f29700a8f60 con 0x7f297015c550 2023-08-09T13:35:51.543 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:51.540+0000 7f2975895640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f29700d04b0 0x7f2970150870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:35:51.544 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:51.540+0000 7f2975895640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f29700d04b0 0x7f2970150870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:35652/0 (socket says 172.21.15.43:35652) 2023-08-09T13:35:51.544 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:51.540+0000 7f2975895640 1 -- 172.21.15.43:0/1379675979 learned_addr learned my addr 172.21.15.43:0/1379675979 (peer_addr_for_me v2:172.21.15.43:0/0) 2023-08-09T13:35:51.544 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:51.540+0000 7f2976096640 1 --2- 172.21.15.43:0/1379675979 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f29700b45f0 0x7f29700a9a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:35:51.544 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:51.540+0000 7f2976897640 1 --2- 172.21.15.43:0/1379675979 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f297015c550 0x7f29700aa650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:35:51.544 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:51.540+0000 7f2975895640 1 -- 172.21.15.43:0/1379675979 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f297015c550 msgr2=0x7f29700aa650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:35:51.544 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:51.540+0000 7f2975895640 1 --2- 172.21.15.43:0/1379675979 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f297015c550 0x7f29700aa650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:35:51.545 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:51.540+0000 7f2975895640 1 -- 172.21.15.43:0/1379675979 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f29700b45f0 msgr2=0x7f29700a9a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:35:51.545 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:51.540+0000 7f2975895640 1 --2- 172.21.15.43:0/1379675979 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f29700b45f0 0x7f29700a9a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:35:51.545 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:51.540+0000 7f2975895640 1 -- 172.21.15.43:0/1379675979 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2968009a70 con 0x7f29700d04b0 2023-08-09T13:35:51.545 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:51.541+0000 7f2975895640 1 --2- 172.21.15.43:0/1379675979 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f29700d04b0 0x7f2970150870 secure :-1 s=READY pgs=480 cs=0 l=1 rev1=1 crypto rx=0x7f296000c500 tx=0x7f296000c9c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-09T13:35:51.545 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:51.541+0000 7f2966ffd640 1 -- 172.21.15.43:0/1379675979 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2960005ce0 con 0x7f29700d04b0 2023-08-09T13:35:51.545 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:51.541+0000 7f2966ffd640 1 -- 172.21.15.43:0/1379675979 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2960005e40 con 0x7f29700d04b0 2023-08-09T13:35:51.545 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:51.541+0000 7f2966ffd640 1 -- 172.21.15.43:0/1379675979 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2960007330 con 0x7f29700d04b0 2023-08-09T13:35:51.545 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:51.541+0000 7f29782fa640 1 -- 172.21.15.43:0/1379675979 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f29700a9240 con 0x7f29700d04b0 2023-08-09T13:35:51.546 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:51.541+0000 7f29782fa640 1 -- 172.21.15.43:0/1379675979 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2970150db0 con 0x7f29700d04b0 2023-08-09T13:35:51.546 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:51.542+0000 7f29782fa640 1 -- 172.21.15.43:0/1379675979 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f29700a8dc0 con 0x7f29700d04b0 2023-08-09T13:35:51.546 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:51.543+0000 7f2966ffd640 1 -- 172.21.15.43:0/1379675979 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 5) v1 ==== 88341+0+0 (secure 0 0 0) 0x7f2960007490 con 0x7f29700d04b0 2023-08-09T13:35:51.547 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:51.544+0000 7f2966ffd640 1 --2- 172.21.15.43:0/1379675979 >> [v2:172.21.15.203:6832/830097474,v1:172.21.15.203:6833/830097474] conn(0x7f295406a080 0x7f295406c540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:35:51.547 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:51.544+0000 7f2966ffd640 1 -- 172.21.15.43:0/1379675979 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 6871+0+0 (secure 0 0 0) 0x7f2960087ee0 con 0x7f29700d04b0 2023-08-09T13:35:51.547 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:51.545+0000 7f2976096640 1 --2- 172.21.15.43:0/1379675979 >> [v2:172.21.15.203:6832/830097474,v1:172.21.15.203:6833/830097474] conn(0x7f295406a080 0x7f295406c540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:35:51.548 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:51.546+0000 7f2976096640 1 --2- 172.21.15.43:0/1379675979 >> [v2:172.21.15.203:6832/830097474,v1:172.21.15.203:6833/830097474] conn(0x7f295406a080 0x7f295406c540 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7f296c000f60 tx=0x7f296c009040 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-09T13:35:51.551 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:51.548+0000 7f2966ffd640 1 -- 172.21.15.43:0/1379675979 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178066 (secure 0 0 0) 0x7f29700a8dc0 con 0x7f29700d04b0 2023-08-09T13:35:51.746 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:51.743+0000 7f29782fa640 1 -- 172.21.15.43:0/1379675979 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"} v 0) v1 -- 0x7f29700a9760 con 0x7f29700d04b0 2023-08-09T13:35:51.746 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:51.744+0000 7f2966ffd640 1 -- 172.21.15.43:0/1379675979 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"}]=0 v26) v1 ==== 92+0+10 (secure 0 0 0) 0x7f29700a9760 con 0x7f29700d04b0 2023-08-09T13:35:51.747 INFO:teuthology.orchestra.run.smithi043.stdout:pg_num: 8 2023-08-09T13:35:51.750 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:51.747+0000 7f29782fa640 1 -- 172.21.15.43:0/1379675979 >> [v2:172.21.15.203:6832/830097474,v1:172.21.15.203:6833/830097474] conn(0x7f295406a080 msgr2=0x7f295406c540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:35:51.750 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:51.747+0000 7f29782fa640 1 --2- 172.21.15.43:0/1379675979 >> [v2:172.21.15.203:6832/830097474,v1:172.21.15.203:6833/830097474] conn(0x7f295406a080 0x7f295406c540 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7f296c000f60 tx=0x7f296c009040 comp rx=0 tx=0).stop 2023-08-09T13:35:51.750 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:51.747+0000 7f29782fa640 1 -- 172.21.15.43:0/1379675979 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f29700d04b0 msgr2=0x7f2970150870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:35:51.750 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:51.747+0000 7f29782fa640 1 --2- 172.21.15.43:0/1379675979 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f29700d04b0 0x7f2970150870 secure :-1 s=READY pgs=480 cs=0 l=1 rev1=1 crypto rx=0x7f296000c500 tx=0x7f296000c9c0 comp rx=0 tx=0).stop 2023-08-09T13:35:51.750 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:51.747+0000 7f29782fa640 1 -- 172.21.15.43:0/1379675979 shutdown_connections 2023-08-09T13:35:51.750 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:51.747+0000 7f29782fa640 1 --2- 172.21.15.43:0/1379675979 >> [v2:172.21.15.203:6832/830097474,v1:172.21.15.203:6833/830097474] conn(0x7f295406a080 0x7f295406c540 unknown :-1 s=CLOSED pgs=236 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:35:51.751 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:51.747+0000 7f29782fa640 1 --2- 172.21.15.43:0/1379675979 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f297015c550 0x7f29700aa650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:35:51.751 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:51.747+0000 7f29782fa640 1 --2- 172.21.15.43:0/1379675979 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f29700d04b0 0x7f2970150870 unknown :-1 s=CLOSED pgs=480 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:35:51.751 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:51.747+0000 7f29782fa640 1 --2- 172.21.15.43:0/1379675979 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f29700b45f0 0x7f29700a9a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:35:51.751 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:51.747+0000 7f29782fa640 1 -- 172.21.15.43:0/1379675979 >> 172.21.15.43:0/1379675979 conn(0x7f2970071520 msgr2=0x7f29700c8d90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-09T13:35:51.751 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:51.747+0000 7f29782fa640 1 -- 172.21.15.43:0/1379675979 shutdown_connections 2023-08-09T13:35:51.752 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:51.748+0000 7f29782fa640 1 -- 172.21.15.43:0/1379675979 wait complete. 2023-08-09T13:35:51.770 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-08-09T13:35:52.000 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:51.998+0000 7f3992e87640 1 Processor -- start 2023-08-09T13:35:52.001 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:51.998+0000 7f3992e87640 1 -- start start 2023-08-09T13:35:52.002 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:51.999+0000 7f3992e87640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f398c15a550 0x7f398c0c3d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:35:52.002 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:52.000+0000 7f3992e87640 1 -- --> v1:172.21.15.43:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f398c15e470 con 0x7f398c0cf790 2023-08-09T13:35:52.002 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:52.000+0000 7f3992e87640 1 -- --> v1:172.21.15.203:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f398c15e750 con 0x7f398c0ce850 2023-08-09T13:35:52.002 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:52.000+0000 7f3992e87640 1 -- --> v2:172.21.15.43:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f398c15ea30 con 0x7f398c15a550 2023-08-09T13:35:52.003 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:52.000+0000 7f398bfff640 1 --1- >> v1:172.21.15.43:6789/0 conn(0x7f398c0cf790 0x7f398c0ce140 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.43:6789/0 says I am v1:172.21.15.43:48610/0 (socket says 172.21.15.43:48610) 2023-08-09T13:35:52.003 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:52.000+0000 7f398bfff640 1 -- 172.21.15.43:0/2458303450 learned_addr learned my addr 172.21.15.43:0/2458303450 (peer_addr_for_me v1:172.21.15.43:0/0) 2023-08-09T13:35:52.003 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:52.000+0000 7f3990c23640 1 --2- 172.21.15.43:0/2458303450 >> v2:172.21.15.43:3300/0 conn(0x7f398c15a550 0x7f398c0c3d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:35:52.004 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:52.001+0000 7f3990c23640 1 -- 172.21.15.43:0/2458303450 >> v1:172.21.15.203:6789/0 conn(0x7f398c0ce850 legacy=0x7f398c06de00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-09T13:35:52.005 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:52.001+0000 7f3990c23640 1 -- 172.21.15.43:0/2458303450 >> v1:172.21.15.43:6789/0 conn(0x7f398c0cf790 legacy=0x7f398c0ce140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:35:52.005 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:52.001+0000 7f3990c23640 1 -- 172.21.15.43:0/2458303450 --> v2:172.21.15.43:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f398c15ed10 con 0x7f398c15a550 2023-08-09T13:35:52.005 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:52.001+0000 7f398b7fe640 1 -- 172.21.15.43:0/2458303450 <== mon.0 v1:172.21.15.43:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 411944089 0 0) 0x7f398c15e470 con 0x7f398c0cf790 2023-08-09T13:35:52.006 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:52.002+0000 7f3990c23640 1 --2- 172.21.15.43:0/2458303450 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f398c15a550 0x7f398c0c3d90 secure :-1 s=READY pgs=482 cs=0 l=1 rev1=1 crypto rx=0x7f397c0097a0 tx=0x7f397c02b840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ba66264d2723874b server_cookie=0 in_seq=0 out_seq=0 2023-08-09T13:35:52.006 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:52.002+0000 7f398b7fe640 1 -- 172.21.15.43:0/2458303450 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f397c02e070 con 0x7f398c15a550 2023-08-09T13:35:52.007 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:52.002+0000 7f398b7fe640 1 -- 172.21.15.43:0/2458303450 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f397c0315a0 con 0x7f398c15a550 2023-08-09T13:35:52.007 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:52.002+0000 7f398b7fe640 1 -- 172.21.15.43:0/2458303450 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f397c031930 con 0x7f398c15a550 2023-08-09T13:35:52.007 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:52.003+0000 7f3992e87640 1 -- 172.21.15.43:0/2458303450 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f398c15a550 msgr2=0x7f398c0c3d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:35:52.007 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:52.003+0000 7f3992e87640 1 --2- 172.21.15.43:0/2458303450 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f398c15a550 0x7f398c0c3d90 secure :-1 s=READY pgs=482 cs=0 l=1 rev1=1 crypto rx=0x7f397c0097a0 tx=0x7f397c02b840 comp rx=0 tx=0).stop 2023-08-09T13:35:52.007 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:52.003+0000 7f3992e87640 1 -- 172.21.15.43:0/2458303450 shutdown_connections 2023-08-09T13:35:52.007 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:52.003+0000 7f3992e87640 1 --2- 172.21.15.43:0/2458303450 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f398c15a550 0x7f398c0c3d90 unknown :-1 s=CLOSED pgs=482 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:35:52.007 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:52.003+0000 7f3992e87640 1 -- 172.21.15.43:0/2458303450 >> 172.21.15.43:0/2458303450 conn(0x7f398c0b1dd0 msgr2=0x7f398c0b21e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-09T13:35:52.008 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:52.003+0000 7f3992e87640 1 -- 172.21.15.43:0/2458303450 shutdown_connections 2023-08-09T13:35:52.008 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:52.003+0000 7f3992e87640 1 -- 172.21.15.43:0/2458303450 wait complete. 2023-08-09T13:35:52.008 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:52.005+0000 7f3992e87640 1 Processor -- start 2023-08-09T13:35:52.009 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:52.005+0000 7f3992e87640 1 -- start start 2023-08-09T13:35:52.009 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:52.006+0000 7f3992e87640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f398c0ce850 0x7f398c148530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:35:52.009 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:52.006+0000 7f3992e87640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f398c0cf790 0x7f398c0cc5e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:35:52.009 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:52.006+0000 7f3992e87640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f398c15a550 0x7f398c0ccb20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:35:52.010 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:52.006+0000 7f3992e87640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f398c1490b0 con 0x7f398c0ce850 2023-08-09T13:35:52.010 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:52.006+0000 7f3992e87640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f398c1492c0 con 0x7f398c15a550 2023-08-09T13:35:52.010 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:52.006+0000 7f3992e87640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f398c1495a0 con 0x7f398c0cf790 2023-08-09T13:35:52.011 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:52.006+0000 7f3990c23640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f398c0ce850 0x7f398c148530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:35:52.011 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:52.007+0000 7f3990c23640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f398c0ce850 0x7f398c148530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:35668/0 (socket says 172.21.15.43:35668) 2023-08-09T13:35:52.012 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:52.007+0000 7f3990c23640 1 -- 172.21.15.43:0/2364754242 learned_addr learned my addr 172.21.15.43:0/2364754242 (peer_addr_for_me v2:172.21.15.43:0/0) 2023-08-09T13:35:52.012 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:52.007+0000 7f398bfff640 1 --2- 172.21.15.43:0/2364754242 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f398c0cf790 0x7f398c0cc5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:35:52.012 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:52.007+0000 7f3990c23640 1 -- 172.21.15.43:0/2364754242 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f398c0cf790 msgr2=0x7f398c0cc5e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:35:52.012 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:52.007+0000 7f3990c23640 1 --2- 172.21.15.43:0/2364754242 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f398c0cf790 0x7f398c0cc5e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:35:52.012 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:52.007+0000 7f3990c23640 1 -- 172.21.15.43:0/2364754242 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f398c15a550 msgr2=0x7f398c0ccb20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:35:52.012 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:52.007+0000 7f3990c23640 1 --2- 172.21.15.43:0/2364754242 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f398c15a550 0x7f398c0ccb20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:35:52.013 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:52.007+0000 7f3990c23640 1 -- 172.21.15.43:0/2364754242 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f397c009400 con 0x7f398c0ce850 2023-08-09T13:35:52.013 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:52.007+0000 7f3990c23640 1 --2- 172.21.15.43:0/2364754242 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f398c0ce850 0x7f398c148530 secure :-1 s=READY pgs=483 cs=0 l=1 rev1=1 crypto rx=0x7f397c034040 tx=0x7f397c0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-09T13:35:52.013 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:52.008+0000 7f39897fa640 1 -- 172.21.15.43:0/2364754242 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f397c032980 con 0x7f398c0ce850 2023-08-09T13:35:52.013 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:52.008+0000 7f39897fa640 1 -- 172.21.15.43:0/2364754242 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f397c032ae0 con 0x7f398c0ce850 2023-08-09T13:35:52.013 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:52.008+0000 7f39897fa640 1 -- 172.21.15.43:0/2364754242 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f397c032e90 con 0x7f398c0ce850 2023-08-09T13:35:52.013 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:52.008+0000 7f3992e87640 1 -- 172.21.15.43:0/2364754242 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f398c149880 con 0x7f398c0ce850 2023-08-09T13:35:52.014 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:52.008+0000 7f3992e87640 1 -- 172.21.15.43:0/2364754242 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f398c149e40 con 0x7f398c0ce850 2023-08-09T13:35:52.014 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:52.009+0000 7f3992e87640 1 -- 172.21.15.43:0/2364754242 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f398c14a030 con 0x7f398c0ce850 2023-08-09T13:35:52.015 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:52.010+0000 7f39897fa640 1 -- 172.21.15.43:0/2364754242 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 5) v1 ==== 88341+0+0 (secure 0 0 0) 0x7f397c03b3f0 con 0x7f398c0ce850 2023-08-09T13:35:52.015 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:52.012+0000 7f39897fa640 1 --2- 172.21.15.43:0/2364754242 >> [v2:172.21.15.203:6832/830097474,v1:172.21.15.203:6833/830097474] conn(0x7f396c06a050 0x7f396c06c510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:35:52.015 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:52.012+0000 7f39897fa640 1 -- 172.21.15.43:0/2364754242 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 6871+0+0 (secure 0 0 0) 0x7f397c0aaaa0 con 0x7f398c0ce850 2023-08-09T13:35:52.015 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:52.012+0000 7f398bfff640 1 --2- 172.21.15.43:0/2364754242 >> [v2:172.21.15.203:6832/830097474,v1:172.21.15.203:6833/830097474] conn(0x7f396c06a050 0x7f396c06c510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:35:52.016 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:52.013+0000 7f398bfff640 1 --2- 172.21.15.43:0/2364754242 >> [v2:172.21.15.203:6832/830097474,v1:172.21.15.203:6833/830097474] conn(0x7f396c06a050 0x7f396c06c510 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7f398c0aead0 tx=0x7f3974009040 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-09T13:35:52.018 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:52.016+0000 7f39897fa640 1 -- 172.21.15.43:0/2364754242 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178066 (secure 0 0 0) 0x7f397c075dc0 con 0x7f398c0ce850 2023-08-09T13:35:52.218 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:52.215+0000 7f3992e87640 1 -- 172.21.15.43:0/2364754242 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f398c0739f0 con 0x7f398c0ce850 2023-08-09T13:35:52.218 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:52.216+0000 7f39897fa640 1 -- 172.21.15.43:0/2364754242 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v26) v1 ==== 93+0+10 (secure 0 0 0) 0x7f397c0793e0 con 0x7f398c0ce850 2023-08-09T13:35:52.219 INFO:teuthology.orchestra.run.smithi043.stdout:pg_num: 1 2023-08-09T13:35:52.221 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:52.218+0000 7f3992e87640 1 -- 172.21.15.43:0/2364754242 >> [v2:172.21.15.203:6832/830097474,v1:172.21.15.203:6833/830097474] conn(0x7f396c06a050 msgr2=0x7f396c06c510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:35:52.221 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:52.218+0000 7f3992e87640 1 --2- 172.21.15.43:0/2364754242 >> [v2:172.21.15.203:6832/830097474,v1:172.21.15.203:6833/830097474] conn(0x7f396c06a050 0x7f396c06c510 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7f398c0aead0 tx=0x7f3974009040 comp rx=0 tx=0).stop 2023-08-09T13:35:52.221 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:52.218+0000 7f3992e87640 1 -- 172.21.15.43:0/2364754242 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f398c0ce850 msgr2=0x7f398c148530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:35:52.221 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:52.218+0000 7f3992e87640 1 --2- 172.21.15.43:0/2364754242 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f398c0ce850 0x7f398c148530 secure :-1 s=READY pgs=483 cs=0 l=1 rev1=1 crypto rx=0x7f397c034040 tx=0x7f397c0046c0 comp rx=0 tx=0).stop 2023-08-09T13:35:52.221 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:52.218+0000 7f3992e87640 1 -- 172.21.15.43:0/2364754242 shutdown_connections 2023-08-09T13:35:52.222 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:52.218+0000 7f3992e87640 1 --2- 172.21.15.43:0/2364754242 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f398c15a550 0x7f398c0ccb20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:35:52.222 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:52.218+0000 7f3992e87640 1 --2- 172.21.15.43:0/2364754242 >> [v2:172.21.15.203:6832/830097474,v1:172.21.15.203:6833/830097474] conn(0x7f396c06a050 0x7f396c06c510 unknown :-1 s=CLOSED pgs=237 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:35:52.222 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:52.218+0000 7f3992e87640 1 --2- 172.21.15.43:0/2364754242 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f398c0cf790 0x7f398c0cc5e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:35:52.222 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:52.218+0000 7f3992e87640 1 --2- 172.21.15.43:0/2364754242 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f398c0ce850 0x7f398c148530 unknown :-1 s=CLOSED pgs=483 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:35:52.222 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:52.218+0000 7f3992e87640 1 -- 172.21.15.43:0/2364754242 >> 172.21.15.43:0/2364754242 conn(0x7f398c0b1dd0 msgr2=0x7f398c0ca660 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-09T13:35:52.222 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:52.219+0000 7f3992e87640 1 -- 172.21.15.43:0/2364754242 shutdown_connections 2023-08-09T13:35:52.223 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:52.219+0000 7f3992e87640 1 -- 172.21.15.43:0/2364754242 wait complete. 2023-08-09T13:35:52.241 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-08-09T13:35:52.475 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:52.473+0000 7fb2e995d640 1 Processor -- start 2023-08-09T13:35:52.476 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:52.473+0000 7fb2e995d640 1 -- start start 2023-08-09T13:35:52.477 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:52.475+0000 7fb2e995d640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7fb2e40cc380 0x7fb2e4160f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:35:52.477 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:52.475+0000 7fb2e995d640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fb2e40c3130 0x7fb2e406bb50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:35:52.478 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:52.475+0000 7fb2e995d640 1 -- --> v1:172.21.15.203:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb2e40c38a0 con 0x7fb2e414a5c0 2023-08-09T13:35:52.478 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:52.475+0000 7fb2e995d640 1 -- --> v2:172.21.15.43:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb2e40c4420 con 0x7fb2e40cc380 2023-08-09T13:35:52.478 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:52.475+0000 7fb2e995d640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb2e40c3b80 con 0x7fb2e40c3130 2023-08-09T13:35:52.479 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:52.475+0000 7fb2e27fc640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7fb2e40cc380 0x7fb2e4160f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:35:52.479 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:52.475+0000 7fb2e27fc640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7fb2e40cc380 0x7fb2e4160f20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:35676/0 (socket says 172.21.15.43:35676) 2023-08-09T13:35:52.479 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:52.475+0000 7fb2e27fc640 1 -- 172.21.15.43:0/1278806143 learned_addr learned my addr 172.21.15.43:0/1278806143 (peer_addr_for_me v2:172.21.15.43:0/0) 2023-08-09T13:35:52.479 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:52.476+0000 7fb2e37fe640 1 --2- 172.21.15.43:0/1278806143 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fb2e40c3130 0x7fb2e406bb50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:35:52.479 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:52.476+0000 7fb2e27fc640 1 -- 172.21.15.43:0/1278806143 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fb2e40c3130 msgr2=0x7fb2e406bb50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-09T13:35:52.479 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:52.476+0000 7fb2e27fc640 1 --2- 172.21.15.43:0/1278806143 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fb2e40c3130 0x7fb2e406bb50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:35:52.480 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:52.476+0000 7fb2e27fc640 1 -- 172.21.15.43:0/1278806143 >> v1:172.21.15.203:6789/0 conn(0x7fb2e414a5c0 legacy=0x7fb2e40cbc70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-09T13:35:52.480 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:52.476+0000 7fb2e27fc640 1 -- 172.21.15.43:0/1278806143 --> v2:172.21.15.43:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb2e40c3e60 con 0x7fb2e40cc380 2023-08-09T13:35:52.481 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:52.477+0000 7fb2e27fc640 1 --2- 172.21.15.43:0/1278806143 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb2e40cc380 0x7fb2e4160f20 secure :-1 s=READY pgs=484 cs=0 l=1 rev1=1 crypto rx=0x7fb2d8009870 tx=0x7fb2d802b8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3a6633e02e179f99 server_cookie=0 in_seq=0 out_seq=0 2023-08-09T13:35:52.481 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:52.477+0000 7fb2e1ffb640 1 -- 172.21.15.43:0/1278806143 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb2d8031640 con 0x7fb2e40cc380 2023-08-09T13:35:52.481 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:52.478+0000 7fb2e1ffb640 1 -- 172.21.15.43:0/1278806143 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb2d80317a0 con 0x7fb2e40cc380 2023-08-09T13:35:52.481 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:52.478+0000 7fb2e1ffb640 1 -- 172.21.15.43:0/1278806143 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb2d8031b30 con 0x7fb2e40cc380 2023-08-09T13:35:52.481 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:52.478+0000 7fb2e995d640 1 -- 172.21.15.43:0/1278806143 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb2e40cc380 msgr2=0x7fb2e4160f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:35:52.481 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:52.478+0000 7fb2e995d640 1 --2- 172.21.15.43:0/1278806143 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb2e40cc380 0x7fb2e4160f20 secure :-1 s=READY pgs=484 cs=0 l=1 rev1=1 crypto rx=0x7fb2d8009870 tx=0x7fb2d802b8f0 comp rx=0 tx=0).stop 2023-08-09T13:35:52.481 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:52.478+0000 7fb2e995d640 1 -- 172.21.15.43:0/1278806143 shutdown_connections 2023-08-09T13:35:52.482 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:52.478+0000 7fb2e995d640 1 --2- 172.21.15.43:0/1278806143 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fb2e40c3130 0x7fb2e406bb50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:35:52.482 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:52.478+0000 7fb2e995d640 1 --2- 172.21.15.43:0/1278806143 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb2e40cc380 0x7fb2e4160f20 unknown :-1 s=CLOSED pgs=484 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:35:52.482 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:52.478+0000 7fb2e995d640 1 -- 172.21.15.43:0/1278806143 >> 172.21.15.43:0/1278806143 conn(0x7fb2e40ce0a0 msgr2=0x7fb2e40ce480 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-09T13:35:52.482 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:52.478+0000 7fb2e995d640 1 -- 172.21.15.43:0/1278806143 shutdown_connections 2023-08-09T13:35:52.482 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:52.478+0000 7fb2e995d640 1 -- 172.21.15.43:0/1278806143 wait complete. 2023-08-09T13:35:52.483 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:52.480+0000 7fb2e995d640 1 Processor -- start 2023-08-09T13:35:52.483 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:52.480+0000 7fb2e995d640 1 -- start start 2023-08-09T13:35:52.484 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:52.481+0000 7fb2e995d640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fb2e40c3130 0x7fb2e40c6210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:35:52.484 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:52.481+0000 7fb2e995d640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb2e40cc380 0x7fb2e4160550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:35:52.484 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:52.481+0000 7fb2e995d640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fb2e414a5c0 0x7fb2e40ca9e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:35:52.484 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:52.481+0000 7fb2e995d640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb2e40b4100 con 0x7fb2e40cc380 2023-08-09T13:35:52.484 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:52.481+0000 7fb2e995d640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb2e4160a90 con 0x7fb2e40c3130 2023-08-09T13:35:52.484 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:52.481+0000 7fb2e995d640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb2e40c5170 con 0x7fb2e414a5c0 2023-08-09T13:35:52.485 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:52.482+0000 7fb2e27fc640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb2e40cc380 0x7fb2e4160550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:35:52.486 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:52.482+0000 7fb2e27fc640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb2e40cc380 0x7fb2e4160550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:35682/0 (socket says 172.21.15.43:35682) 2023-08-09T13:35:52.486 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:52.482+0000 7fb2e27fc640 1 -- 172.21.15.43:0/4136669722 learned_addr learned my addr 172.21.15.43:0/4136669722 (peer_addr_for_me v2:172.21.15.43:0/0) 2023-08-09T13:35:52.486 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:52.482+0000 7fb2e2ffd640 1 --2- 172.21.15.43:0/4136669722 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fb2e40c3130 0x7fb2e40c6210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:35:52.486 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:52.482+0000 7fb2e37fe640 1 --2- 172.21.15.43:0/4136669722 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fb2e414a5c0 0x7fb2e40ca9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:35:52.486 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:52.482+0000 7fb2e27fc640 1 -- 172.21.15.43:0/4136669722 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fb2e414a5c0 msgr2=0x7fb2e40ca9e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:35:52.486 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:52.482+0000 7fb2e27fc640 1 --2- 172.21.15.43:0/4136669722 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fb2e414a5c0 0x7fb2e40ca9e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:35:52.486 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:52.482+0000 7fb2e27fc640 1 -- 172.21.15.43:0/4136669722 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fb2e40c3130 msgr2=0x7fb2e40c6210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:35:52.487 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:52.482+0000 7fb2e27fc640 1 --2- 172.21.15.43:0/4136669722 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fb2e40c3130 0x7fb2e40c6210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:35:52.487 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:52.482+0000 7fb2e27fc640 1 -- 172.21.15.43:0/4136669722 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb2d80094d0 con 0x7fb2e40cc380 2023-08-09T13:35:52.487 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:52.483+0000 7fb2e27fc640 1 --2- 172.21.15.43:0/4136669722 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb2e40cc380 0x7fb2e4160550 secure :-1 s=READY pgs=485 cs=0 l=1 rev1=1 crypto rx=0x7fb2d8000c00 tx=0x7fb2d80046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-09T13:35:52.487 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:52.483+0000 7fb2cbfff640 1 -- 172.21.15.43:0/4136669722 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb2d803ac60 con 0x7fb2e40cc380 2023-08-09T13:35:52.487 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:52.483+0000 7fb2cbfff640 1 -- 172.21.15.43:0/4136669722 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb2d803ade0 con 0x7fb2e40cc380 2023-08-09T13:35:52.487 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:52.483+0000 7fb2cbfff640 1 -- 172.21.15.43:0/4136669722 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb2d8044630 con 0x7fb2e40cc380 2023-08-09T13:35:52.487 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:52.483+0000 7fb2e995d640 1 -- 172.21.15.43:0/4136669722 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb2e40c5450 con 0x7fb2e40cc380 2023-08-09T13:35:52.487 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:52.483+0000 7fb2e995d640 1 -- 172.21.15.43:0/4136669722 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb2e40c5a10 con 0x7fb2e40cc380 2023-08-09T13:35:52.488 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:52.484+0000 7fb2e995d640 1 -- 172.21.15.43:0/4136669722 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb2e40739f0 con 0x7fb2e40cc380 2023-08-09T13:35:52.488 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:52.485+0000 7fb2cbfff640 1 -- 172.21.15.43:0/4136669722 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 5) v1 ==== 88341+0+0 (secure 0 0 0) 0x7fb2d802f050 con 0x7fb2e40cc380 2023-08-09T13:35:52.489 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:52.486+0000 7fb2cbfff640 1 --2- 172.21.15.43:0/4136669722 >> [v2:172.21.15.203:6832/830097474,v1:172.21.15.203:6833/830097474] conn(0x7fb2d006a080 0x7fb2d006c540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:35:52.489 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:52.486+0000 7fb2cbfff640 1 -- 172.21.15.43:0/4136669722 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 6871+0+0 (secure 0 0 0) 0x7fb2d80aaae0 con 0x7fb2e40cc380 2023-08-09T13:35:52.489 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:52.487+0000 7fb2e2ffd640 1 --2- 172.21.15.43:0/4136669722 >> [v2:172.21.15.203:6832/830097474,v1:172.21.15.203:6833/830097474] conn(0x7fb2d006a080 0x7fb2d006c540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:35:52.490 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:52.488+0000 7fb2e2ffd640 1 --2- 172.21.15.43:0/4136669722 >> [v2:172.21.15.203:6832/830097474,v1:172.21.15.203:6833/830097474] conn(0x7fb2d006a080 0x7fb2d006c540 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7fb2d40012e0 tx=0x7fb2d4009040 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-09T13:35:52.492 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:52.490+0000 7fb2cbfff640 1 -- 172.21.15.43:0/4136669722 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178066 (secure 0 0 0) 0x7fb2d8075e00 con 0x7fb2e40cc380 2023-08-09T13:35:52.686 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:52.684+0000 7fb2e995d640 1 -- 172.21.15.43:0/4136669722 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7fb2e40cc830 con 0x7fb2e40cc380 2023-08-09T13:35:52.687 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:52.685+0000 7fb2cbfff640 1 -- 172.21.15.43:0/4136669722 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v26) v1 ==== 104+0+11 (secure 0 0 0) 0x7fb2d804d450 con 0x7fb2e40cc380 2023-08-09T13:35:52.688 INFO:teuthology.orchestra.run.smithi043.stdout:pg_num: 32 2023-08-09T13:35:52.690 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:52.687+0000 7fb2e995d640 1 -- 172.21.15.43:0/4136669722 >> [v2:172.21.15.203:6832/830097474,v1:172.21.15.203:6833/830097474] conn(0x7fb2d006a080 msgr2=0x7fb2d006c540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:35:52.690 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:52.687+0000 7fb2e995d640 1 --2- 172.21.15.43:0/4136669722 >> [v2:172.21.15.203:6832/830097474,v1:172.21.15.203:6833/830097474] conn(0x7fb2d006a080 0x7fb2d006c540 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7fb2d40012e0 tx=0x7fb2d4009040 comp rx=0 tx=0).stop 2023-08-09T13:35:52.691 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:52.687+0000 7fb2e995d640 1 -- 172.21.15.43:0/4136669722 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb2e40cc380 msgr2=0x7fb2e4160550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:35:52.691 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:52.687+0000 7fb2e995d640 1 --2- 172.21.15.43:0/4136669722 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb2e40cc380 0x7fb2e4160550 secure :-1 s=READY pgs=485 cs=0 l=1 rev1=1 crypto rx=0x7fb2d8000c00 tx=0x7fb2d80046c0 comp rx=0 tx=0).stop 2023-08-09T13:35:52.691 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:52.688+0000 7fb2e995d640 1 -- 172.21.15.43:0/4136669722 shutdown_connections 2023-08-09T13:35:52.691 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:52.688+0000 7fb2e995d640 1 --2- 172.21.15.43:0/4136669722 >> [v2:172.21.15.203:6832/830097474,v1:172.21.15.203:6833/830097474] conn(0x7fb2d006a080 0x7fb2d006c540 unknown :-1 s=CLOSED pgs=238 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:35:52.691 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:52.688+0000 7fb2e995d640 1 --2- 172.21.15.43:0/4136669722 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fb2e414a5c0 0x7fb2e40ca9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:35:52.691 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:52.688+0000 7fb2e995d640 1 --2- 172.21.15.43:0/4136669722 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb2e40cc380 0x7fb2e4160550 unknown :-1 s=CLOSED pgs=485 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:35:52.691 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:52.688+0000 7fb2e995d640 1 --2- 172.21.15.43:0/4136669722 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fb2e40c3130 0x7fb2e40c6210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:35:52.692 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:52.688+0000 7fb2e995d640 1 -- 172.21.15.43:0/4136669722 >> 172.21.15.43:0/4136669722 conn(0x7fb2e40ce0a0 msgr2=0x7fb2e40c4a20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-09T13:35:52.692 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:52.688+0000 7fb2e995d640 1 -- 172.21.15.43:0/4136669722 shutdown_connections 2023-08-09T13:35:52.692 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:52.688+0000 7fb2e995d640 1 -- 172.21.15.43:0/4136669722 wait complete. 2023-08-09T13:35:52.710 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-08-09T13:35:52.950 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:52.947+0000 7fef024ad640 1 Processor -- start 2023-08-09T13:35:52.950 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:52.947+0000 7fef024ad640 1 -- start start 2023-08-09T13:35:52.951 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:52.949+0000 7fef024ad640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7feefc15df60 0x7feefc0ce1c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:35:52.952 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:52.949+0000 7fef024ad640 1 -- --> v1:172.21.15.43:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7feefc15aef0 con 0x7feefc15e3d0 2023-08-09T13:35:52.952 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:52.949+0000 7fef024ad640 1 -- --> v1:172.21.15.203:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7feefc15e940 con 0x7feefc0cb2d0 2023-08-09T13:35:52.952 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:52.949+0000 7fef024ad640 1 -- --> v2:172.21.15.43:3300/0 -- mon_getmap magic: 0 v1 -- 0x7feefc15a970 con 0x7feefc15df60 2023-08-09T13:35:52.953 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:52.950+0000 7feef3fff640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7feefc15df60 0x7feefc0ce1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:35:52.953 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:52.950+0000 7fef00a4a640 1 --1- >> v1:172.21.15.43:6789/0 conn(0x7feefc15e3d0 0x7feefc0c9f90 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.43:6789/0 says I am v1:172.21.15.43:48620/0 (socket says 172.21.15.43:48620) 2023-08-09T13:35:52.953 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:52.950+0000 7fef00a4a640 1 -- 172.21.15.43:0/347484998 learned_addr learned my addr 172.21.15.43:0/347484998 (peer_addr_for_me v1:172.21.15.43:0/0) 2023-08-09T13:35:52.953 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:52.950+0000 7feef3fff640 1 -- 172.21.15.43:0/347484998 >> v1:172.21.15.203:6789/0 conn(0x7feefc0cb2d0 legacy=0x7feefc15b380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:35:52.953 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:52.950+0000 7feef3fff640 1 -- 172.21.15.43:0/347484998 >> v1:172.21.15.43:6789/0 conn(0x7feefc15e3d0 legacy=0x7feefc0c9f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:35:52.953 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:52.950+0000 7feef3fff640 1 -- 172.21.15.43:0/347484998 --> v2:172.21.15.43:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7feefc15ec20 con 0x7feefc15df60 2023-08-09T13:35:52.953 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:52.950+0000 7feefb7fe640 1 -- 172.21.15.43:0/347484998 <== mon.0 v1:172.21.15.43:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2715193566 0 0) 0x7feefc15aef0 con 0x7feefc15e3d0 2023-08-09T13:35:52.954 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:52.952+0000 7feef3fff640 1 --2- 172.21.15.43:0/347484998 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7feefc15df60 0x7feefc0ce1c0 secure :-1 s=READY pgs=487 cs=0 l=1 rev1=1 crypto rx=0x7feee40097a0 tx=0x7feee402b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=92deb259e9a5fe36 server_cookie=0 in_seq=0 out_seq=0 2023-08-09T13:35:52.954 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:52.952+0000 7feefb7fe640 1 -- 172.21.15.43:0/347484998 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feee4031830 con 0x7feefc15df60 2023-08-09T13:35:52.955 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:52.952+0000 7feefb7fe640 1 -- 172.21.15.43:0/347484998 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7feee4031990 con 0x7feefc15df60 2023-08-09T13:35:52.955 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:52.952+0000 7feefb7fe640 1 -- 172.21.15.43:0/347484998 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feee4031d20 con 0x7feefc15df60 2023-08-09T13:35:52.956 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:52.953+0000 7fef024ad640 1 -- 172.21.15.43:0/347484998 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7feefc15df60 msgr2=0x7feefc0ce1c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:35:52.956 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:52.953+0000 7fef024ad640 1 --2- 172.21.15.43:0/347484998 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7feefc15df60 0x7feefc0ce1c0 secure :-1 s=READY pgs=487 cs=0 l=1 rev1=1 crypto rx=0x7feee40097a0 tx=0x7feee402b8d0 comp rx=0 tx=0).stop 2023-08-09T13:35:52.956 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:52.953+0000 7fef024ad640 1 -- 172.21.15.43:0/347484998 shutdown_connections 2023-08-09T13:35:52.956 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:52.953+0000 7fef024ad640 1 --2- 172.21.15.43:0/347484998 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7feefc15df60 0x7feefc0ce1c0 unknown :-1 s=CLOSED pgs=487 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:35:52.956 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:52.953+0000 7fef024ad640 1 -- 172.21.15.43:0/347484998 >> 172.21.15.43:0/347484998 conn(0x7feefc0aff50 msgr2=0x7feefc0b0360 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-09T13:35:52.956 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:52.953+0000 7fef024ad640 1 -- 172.21.15.43:0/347484998 shutdown_connections 2023-08-09T13:35:52.956 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:52.953+0000 7fef024ad640 1 -- 172.21.15.43:0/347484998 wait complete. 2023-08-09T13:35:52.957 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:52.955+0000 7fef024ad640 1 Processor -- start 2023-08-09T13:35:52.957 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:52.955+0000 7fef024ad640 1 -- start start 2023-08-09T13:35:52.958 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:52.956+0000 7fef024ad640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7feefc0cb2d0 0x7feefc1f5220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:35:52.959 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:52.956+0000 7fef024ad640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7feefc15df60 0x7feefc148170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:35:52.959 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:52.956+0000 7fef024ad640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7feefc15e3d0 0x7feefc148bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:35:52.959 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:52.956+0000 7fef024ad640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feefc0c5930 con 0x7feefc15df60 2023-08-09T13:35:52.959 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:52.956+0000 7fef024ad640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feefc1f6150 con 0x7feefc0cb2d0 2023-08-09T13:35:52.959 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:52.956+0000 7fef024ad640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7feefc149900 con 0x7feefc15e3d0 2023-08-09T13:35:52.959 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:52.956+0000 7feef3fff640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7feefc15df60 0x7feefc148170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:35:52.959 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:52.956+0000 7feef3fff640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7feefc15df60 0x7feefc148170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:35714/0 (socket says 172.21.15.43:35714) 2023-08-09T13:35:52.960 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:52.956+0000 7feef3fff640 1 -- 172.21.15.43:0/2873544750 learned_addr learned my addr 172.21.15.43:0/2873544750 (peer_addr_for_me v2:172.21.15.43:0/0) 2023-08-09T13:35:52.961 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:52.957+0000 7feefbfff640 1 --2- 172.21.15.43:0/2873544750 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7feefc0cb2d0 0x7feefc1f5220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:35:52.961 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:52.957+0000 7fef00a4a640 1 --2- 172.21.15.43:0/2873544750 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7feefc15e3d0 0x7feefc148bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:35:52.961 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:52.957+0000 7feef3fff640 1 -- 172.21.15.43:0/2873544750 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7feefc15e3d0 msgr2=0x7feefc148bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:35:52.961 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:52.957+0000 7feef3fff640 1 --2- 172.21.15.43:0/2873544750 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7feefc15e3d0 0x7feefc148bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:35:52.961 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:52.957+0000 7feef3fff640 1 -- 172.21.15.43:0/2873544750 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7feefc0cb2d0 msgr2=0x7feefc1f5220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:35:52.962 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:52.957+0000 7feef3fff640 1 --2- 172.21.15.43:0/2873544750 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7feefc0cb2d0 0x7feefc1f5220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:35:52.962 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:52.957+0000 7feef3fff640 1 -- 172.21.15.43:0/2873544750 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7feee4009400 con 0x7feefc15df60 2023-08-09T13:35:52.962 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:52.957+0000 7fef00a4a640 1 --2- 172.21.15.43:0/2873544750 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7feefc15e3d0 0x7feefc148bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-08-09T13:35:52.962 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:52.957+0000 7feef3fff640 1 --2- 172.21.15.43:0/2873544750 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7feefc15df60 0x7feefc148170 secure :-1 s=READY pgs=488 cs=0 l=1 rev1=1 crypto rx=0x7feee40098c0 tx=0x7feee40046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-09T13:35:52.962 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:52.957+0000 7feef97fa640 1 -- 172.21.15.43:0/2873544750 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feee403ba70 con 0x7feefc15df60 2023-08-09T13:35:52.962 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:52.957+0000 7feef97fa640 1 -- 172.21.15.43:0/2873544750 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7feee403bbf0 con 0x7feefc15df60 2023-08-09T13:35:52.962 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:52.957+0000 7feef97fa640 1 -- 172.21.15.43:0/2873544750 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feee40453f0 con 0x7feefc15df60 2023-08-09T13:35:52.962 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:52.958+0000 7fef024ad640 1 -- 172.21.15.43:0/2873544750 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7feefc1f58b0 con 0x7feefc15df60 2023-08-09T13:35:52.963 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:52.958+0000 7fef024ad640 1 -- 172.21.15.43:0/2873544750 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7feefc1f5e70 con 0x7feefc15df60 2023-08-09T13:35:52.963 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:52.959+0000 7fef024ad640 1 -- 172.21.15.43:0/2873544750 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7feefc0739f0 con 0x7feefc15df60 2023-08-09T13:35:52.963 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:52.960+0000 7feef97fa640 1 -- 172.21.15.43:0/2873544750 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 5) v1 ==== 88341+0+0 (secure 0 0 0) 0x7feee404e450 con 0x7feefc15df60 2023-08-09T13:35:52.963 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:52.961+0000 7feef97fa640 1 --2- 172.21.15.43:0/2873544750 >> [v2:172.21.15.203:6832/830097474,v1:172.21.15.203:6833/830097474] conn(0x7feedc06a050 0x7feedc06c510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:35:52.964 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:52.961+0000 7feef97fa640 1 -- 172.21.15.43:0/2873544750 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 6871+0+0 (secure 0 0 0) 0x7feee40aabc0 con 0x7feefc15df60 2023-08-09T13:35:52.964 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:52.962+0000 7feefbfff640 1 --2- 172.21.15.43:0/2873544750 >> [v2:172.21.15.203:6832/830097474,v1:172.21.15.203:6833/830097474] conn(0x7feedc06a050 0x7feedc06c510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:35:52.965 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:52.963+0000 7feefbfff640 1 --2- 172.21.15.43:0/2873544750 >> [v2:172.21.15.203:6832/830097474,v1:172.21.15.203:6833/830097474] conn(0x7feedc06a050 0x7feedc06c510 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7feeec000f60 tx=0x7feeec009040 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-09T13:35:52.967 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:52.965+0000 7feef97fa640 1 -- 172.21.15.43:0/2873544750 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178066 (secure 0 0 0) 0x7feee4075f70 con 0x7feefc15df60 2023-08-09T13:35:53.161 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:53.159+0000 7fef024ad640 1 -- 172.21.15.43:0/2873544750 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7feefc15bf40 con 0x7feefc15df60 2023-08-09T13:35:53.162 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:53.159+0000 7feef97fa640 1 -- 172.21.15.43:0/2873544750 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v26) v1 ==== 100+0+11 (secure 0 0 0) 0x7feee4079590 con 0x7feefc15df60 2023-08-09T13:35:53.162 INFO:teuthology.orchestra.run.smithi043.stdout:pg_num: 64 2023-08-09T13:35:53.164 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:53.162+0000 7fef024ad640 1 -- 172.21.15.43:0/2873544750 >> [v2:172.21.15.203:6832/830097474,v1:172.21.15.203:6833/830097474] conn(0x7feedc06a050 msgr2=0x7feedc06c510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:35:53.164 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:53.162+0000 7fef024ad640 1 --2- 172.21.15.43:0/2873544750 >> [v2:172.21.15.203:6832/830097474,v1:172.21.15.203:6833/830097474] conn(0x7feedc06a050 0x7feedc06c510 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7feeec000f60 tx=0x7feeec009040 comp rx=0 tx=0).stop 2023-08-09T13:35:53.165 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:53.162+0000 7fef024ad640 1 -- 172.21.15.43:0/2873544750 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7feefc15df60 msgr2=0x7feefc148170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:35:53.165 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:53.162+0000 7fef024ad640 1 --2- 172.21.15.43:0/2873544750 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7feefc15df60 0x7feefc148170 secure :-1 s=READY pgs=488 cs=0 l=1 rev1=1 crypto rx=0x7feee40098c0 tx=0x7feee40046c0 comp rx=0 tx=0).stop 2023-08-09T13:35:53.165 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:53.162+0000 7fef024ad640 1 -- 172.21.15.43:0/2873544750 shutdown_connections 2023-08-09T13:35:53.166 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:53.162+0000 7fef024ad640 1 --2- 172.21.15.43:0/2873544750 >> [v2:172.21.15.203:6832/830097474,v1:172.21.15.203:6833/830097474] conn(0x7feedc06a050 0x7feedc06c510 unknown :-1 s=CLOSED pgs=239 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:35:53.166 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:53.162+0000 7fef024ad640 1 --2- 172.21.15.43:0/2873544750 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7feefc15e3d0 0x7feefc148bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:35:53.166 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:53.162+0000 7fef024ad640 1 --2- 172.21.15.43:0/2873544750 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7feefc15df60 0x7feefc148170 unknown :-1 s=CLOSED pgs=488 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:35:53.166 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:53.162+0000 7fef024ad640 1 --2- 172.21.15.43:0/2873544750 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7feefc0cb2d0 0x7feefc1f5220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:35:53.166 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:53.162+0000 7fef024ad640 1 -- 172.21.15.43:0/2873544750 >> 172.21.15.43:0/2873544750 conn(0x7feefc0aff50 msgr2=0x7feefc0ce190 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-09T13:35:53.166 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:53.162+0000 7fef024ad640 1 -- 172.21.15.43:0/2873544750 shutdown_connections 2023-08-09T13:35:53.166 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:53.163+0000 7fef024ad640 1 -- 172.21.15.43:0/2873544750 wait complete. 2023-08-09T13:35:53.183 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2023-08-09T13:35:53.411 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:53.409+0000 7efedffff640 1 Processor -- start 2023-08-09T13:35:53.412 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:53.409+0000 7efedffff640 1 -- start start 2023-08-09T13:35:53.413 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:53.410+0000 7efedffff640 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7efee015c3a0 0x7efee01608a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:35:53.413 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:53.410+0000 7efedffff640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7efee00cd600 0x7efee00d0580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:35:53.413 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:53.411+0000 7efedffff640 1 -- --> v1:172.21.15.203:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7efee0160de0 con 0x7efee00b0f00 2023-08-09T13:35:53.414 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:53.411+0000 7efedffff640 1 -- --> v2:172.21.15.203:3300/0 -- mon_getmap magic: 0 v1 -- 0x7efee00b81c0 con 0x7efee015c3a0 2023-08-09T13:35:53.414 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:53.411+0000 7efedffff640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7efee00b7c00 con 0x7efee00cd600 2023-08-09T13:35:53.415 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:53.411+0000 7efedf7fe640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7efee00cd600 0x7efee00d0580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:35:53.416 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:53.411+0000 7efedf7fe640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7efee00cd600 0x7efee00d0580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.203:3301/0 says I am v2:172.21.15.43:42268/0 (socket says 172.21.15.43:42268) 2023-08-09T13:35:53.416 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:53.411+0000 7efede7fc640 1 --1- >> v1:172.21.15.203:6789/0 conn(0x7efee00b0f00 0x7efee00ccef0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.203:6789/0 says I am v1:172.21.15.43:35360/0 (socket says 172.21.15.43:35360) 2023-08-09T13:35:53.416 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:53.411+0000 7efedf7fe640 1 -- 172.21.15.43:0/1592690978 learned_addr learned my addr 172.21.15.43:0/1592690978 (peer_addr_for_me v2:172.21.15.43:0/0) 2023-08-09T13:35:53.416 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:53.411+0000 7efedeffd640 1 --2- 172.21.15.43:0/1592690978 >> v2:172.21.15.203:3300/0 conn(0x7efee015c3a0 0x7efee01608a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:35:53.417 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:53.412+0000 7efeddffb640 1 -- 172.21.15.43:0/1592690978 <== mon.1 v1:172.21.15.203:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 180840688 0 0) 0x7efee0160de0 con 0x7efee00b0f00 2023-08-09T13:35:53.417 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:53.412+0000 7efeddffb640 1 -- 172.21.15.43:0/1592690978 --> v1:172.21.15.203:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7efecc003520 con 0x7efee00b0f00 2023-08-09T13:35:53.417 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:53.412+0000 7efedeffd640 1 -- 172.21.15.43:0/1592690978 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7efee00cd600 msgr2=0x7efee00d0580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-09T13:35:53.417 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:53.412+0000 7efedeffd640 1 --2- 172.21.15.43:0/1592690978 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7efee00cd600 0x7efee00d0580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:35:53.417 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:53.412+0000 7efedeffd640 1 -- 172.21.15.43:0/1592690978 >> v1:172.21.15.203:6789/0 conn(0x7efee00b0f00 legacy=0x7efee00ccef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:35:53.418 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:53.412+0000 7efedeffd640 1 -- 172.21.15.43:0/1592690978 --> v2:172.21.15.203:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efee00b7ee0 con 0x7efee015c3a0 2023-08-09T13:35:53.418 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:53.412+0000 7efedf7fe640 1 --2- 172.21.15.43:0/1592690978 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7efee00cd600 0x7efee00d0580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-08-09T13:35:53.419 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:53.414+0000 7efedeffd640 1 --2- 172.21.15.43:0/1592690978 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7efee015c3a0 0x7efee01608a0 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7efed0009d90 tx=0x7efed002be20 comp rx=0 tx=0).ready entity=mon.1 client_cookie=f1ebad8a87f1d3d2 server_cookie=0 in_seq=0 out_seq=0 2023-08-09T13:35:53.420 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:53.414+0000 7efeddffb640 1 -- 172.21.15.43:0/1592690978 <== mon.1 v2:172.21.15.203:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efed002cbd0 con 0x7efee015c3a0 2023-08-09T13:35:53.420 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:53.414+0000 7efeddffb640 1 -- 172.21.15.43:0/1592690978 <== mon.1 v2:172.21.15.203:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efed002cd30 con 0x7efee015c3a0 2023-08-09T13:35:53.420 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:53.414+0000 7efeddffb640 1 -- 172.21.15.43:0/1592690978 <== mon.1 v2:172.21.15.203:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efed002cea0 con 0x7efee015c3a0 2023-08-09T13:35:53.420 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:53.415+0000 7efedffff640 1 -- 172.21.15.43:0/1592690978 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7efee015c3a0 msgr2=0x7efee01608a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:35:53.421 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:53.415+0000 7efedffff640 1 --2- 172.21.15.43:0/1592690978 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7efee015c3a0 0x7efee01608a0 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7efed0009d90 tx=0x7efed002be20 comp rx=0 tx=0).stop 2023-08-09T13:35:53.421 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:53.415+0000 7efedffff640 1 -- 172.21.15.43:0/1592690978 shutdown_connections 2023-08-09T13:35:53.421 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:53.415+0000 7efedffff640 1 --2- 172.21.15.43:0/1592690978 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7efee00cd600 0x7efee00d0580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:35:53.421 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:53.415+0000 7efedffff640 1 --2- 172.21.15.43:0/1592690978 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7efee015c3a0 0x7efee01608a0 unknown :-1 s=CLOSED pgs=190 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:35:53.422 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:53.415+0000 7efedffff640 1 -- 172.21.15.43:0/1592690978 >> 172.21.15.43:0/1592690978 conn(0x7efee006a210 msgr2=0x7efee006a620 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-09T13:35:53.422 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:53.415+0000 7efedffff640 1 -- 172.21.15.43:0/1592690978 shutdown_connections 2023-08-09T13:35:53.422 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:53.415+0000 7efedffff640 1 -- 172.21.15.43:0/1592690978 wait complete. 2023-08-09T13:35:53.422 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:53.417+0000 7efedffff640 1 Processor -- start 2023-08-09T13:35:53.422 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:53.417+0000 7efedffff640 1 -- start start 2023-08-09T13:35:53.425 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:53.418+0000 7efedffff640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7efee00b0f00 0x7efee01f6000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:35:53.426 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:53.418+0000 7efedffff640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7efee00cd600 0x7efee00c7a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:35:53.426 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:53.418+0000 7efedffff640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7efee015c3a0 0x7efee00b1430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:35:53.426 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:53.418+0000 7efedffff640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efee00c9280 con 0x7efee015c3a0 2023-08-09T13:35:53.427 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:53.418+0000 7efedffff640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efee00ca070 con 0x7efee00cd600 2023-08-09T13:35:53.427 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:53.418+0000 7efedffff640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7efee01f6540 con 0x7efee00b0f00 2023-08-09T13:35:53.427 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:53.419+0000 7efedf7fe640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7efee015c3a0 0x7efee00b1430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:35:53.428 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:53.419+0000 7efedf7fe640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7efee015c3a0 0x7efee00b1430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:35730/0 (socket says 172.21.15.43:35730) 2023-08-09T13:35:53.428 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:53.419+0000 7efedf7fe640 1 -- 172.21.15.43:0/643333110 learned_addr learned my addr 172.21.15.43:0/643333110 (peer_addr_for_me v2:172.21.15.43:0/0) 2023-08-09T13:35:53.428 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:53.419+0000 7efedeffd640 1 --2- 172.21.15.43:0/643333110 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7efee00b0f00 0x7efee01f6000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:35:53.428 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:53.419+0000 7efede7fc640 1 --2- 172.21.15.43:0/643333110 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7efee00cd600 0x7efee00c7a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:35:53.429 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:53.419+0000 7efedf7fe640 1 -- 172.21.15.43:0/643333110 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7efee00b0f00 msgr2=0x7efee01f6000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:35:53.429 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:53.419+0000 7efedf7fe640 1 --2- 172.21.15.43:0/643333110 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7efee00b0f00 0x7efee01f6000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:35:53.429 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:53.419+0000 7efedf7fe640 1 -- 172.21.15.43:0/643333110 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7efee00cd600 msgr2=0x7efee00c7a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:35:53.430 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:53.419+0000 7efedf7fe640 1 --2- 172.21.15.43:0/643333110 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7efee00cd600 0x7efee00c7a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:35:53.430 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:53.419+0000 7efedf7fe640 1 -- 172.21.15.43:0/643333110 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efed80094d0 con 0x7efee015c3a0 2023-08-09T13:35:53.430 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:53.419+0000 7efedf7fe640 1 --2- 172.21.15.43:0/643333110 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7efee015c3a0 0x7efee00b1430 secure :-1 s=READY pgs=489 cs=0 l=1 rev1=1 crypto rx=0x7efed800e8b0 tx=0x7efed800ed70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-09T13:35:53.430 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:53.420+0000 7efed7fff640 1 -- 172.21.15.43:0/643333110 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efed800c8e0 con 0x7efee015c3a0 2023-08-09T13:35:53.431 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:53.420+0000 7efed7fff640 1 -- 172.21.15.43:0/643333110 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efed800ca60 con 0x7efee015c3a0 2023-08-09T13:35:53.431 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:53.420+0000 7efedffff640 1 -- 172.21.15.43:0/643333110 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7efed0009a70 con 0x7efee015c3a0 2023-08-09T13:35:53.431 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:53.420+0000 7efed7fff640 1 -- 172.21.15.43:0/643333110 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efed800ce10 con 0x7efee015c3a0 2023-08-09T13:35:53.431 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:53.420+0000 7efedffff640 1 -- 172.21.15.43:0/643333110 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7efee00c9740 con 0x7efee015c3a0 2023-08-09T13:35:53.431 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:53.421+0000 7efedffff640 1 -- 172.21.15.43:0/643333110 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7efee00ca200 con 0x7efee015c3a0 2023-08-09T13:35:53.432 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:53.422+0000 7efed7fff640 1 -- 172.21.15.43:0/643333110 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 5) v1 ==== 88341+0+0 (secure 0 0 0) 0x7efed8013470 con 0x7efee015c3a0 2023-08-09T13:35:53.432 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:53.423+0000 7efed7fff640 1 --2- 172.21.15.43:0/643333110 >> [v2:172.21.15.203:6832/830097474,v1:172.21.15.203:6833/830097474] conn(0x7efec006a080 0x7efec006c540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:35:53.432 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:53.423+0000 7efed7fff640 1 -- 172.21.15.43:0/643333110 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 6871+0+0 (secure 0 0 0) 0x7efed8088ae0 con 0x7efee015c3a0 2023-08-09T13:35:53.433 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:53.424+0000 7efedeffd640 1 --2- 172.21.15.43:0/643333110 >> [v2:172.21.15.203:6832/830097474,v1:172.21.15.203:6833/830097474] conn(0x7efec006a080 0x7efec006c540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:35:53.433 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:53.425+0000 7efedeffd640 1 --2- 172.21.15.43:0/643333110 >> [v2:172.21.15.203:6832/830097474,v1:172.21.15.203:6833/830097474] conn(0x7efec006a080 0x7efec006c540 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7efed0036000 tx=0x7efed0030000 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-09T13:35:53.434 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:53.427+0000 7efed7fff640 1 -- 172.21.15.43:0/643333110 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178066 (secure 0 0 0) 0x7efed8053e90 con 0x7efee015c3a0 2023-08-09T13:35:53.622 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:53.620+0000 7efedffff640 1 -- 172.21.15.43:0/643333110 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) v1 -- 0x7efee004e390 con 0x7efee015c3a0 2023-08-09T13:35:53.623 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:53.621+0000 7efed7fff640 1 -- 172.21.15.43:0/643333110 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v26) v1 ==== 103+0+11 (secure 0 0 0) 0x7efed80574b0 con 0x7efee015c3a0 2023-08-09T13:35:53.623 INFO:teuthology.orchestra.run.smithi043.stdout:pg_num: 32 2023-08-09T13:35:53.626 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:53.623+0000 7efedffff640 1 -- 172.21.15.43:0/643333110 >> [v2:172.21.15.203:6832/830097474,v1:172.21.15.203:6833/830097474] conn(0x7efec006a080 msgr2=0x7efec006c540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:35:53.627 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:53.623+0000 7efedffff640 1 --2- 172.21.15.43:0/643333110 >> [v2:172.21.15.203:6832/830097474,v1:172.21.15.203:6833/830097474] conn(0x7efec006a080 0x7efec006c540 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7efed0036000 tx=0x7efed0030000 comp rx=0 tx=0).stop 2023-08-09T13:35:53.627 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:53.623+0000 7efedffff640 1 -- 172.21.15.43:0/643333110 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7efee015c3a0 msgr2=0x7efee00b1430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:35:53.627 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:53.623+0000 7efedffff640 1 --2- 172.21.15.43:0/643333110 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7efee015c3a0 0x7efee00b1430 secure :-1 s=READY pgs=489 cs=0 l=1 rev1=1 crypto rx=0x7efed800e8b0 tx=0x7efed800ed70 comp rx=0 tx=0).stop 2023-08-09T13:35:53.627 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:53.624+0000 7efedffff640 1 -- 172.21.15.43:0/643333110 shutdown_connections 2023-08-09T13:35:53.627 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:53.624+0000 7efedffff640 1 --2- 172.21.15.43:0/643333110 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7efee015c3a0 0x7efee00b1430 unknown :-1 s=CLOSED pgs=489 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:35:53.627 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:53.624+0000 7efedffff640 1 --2- 172.21.15.43:0/643333110 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7efee00cd600 0x7efee00c7a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:35:53.627 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:53.624+0000 7efedffff640 1 --2- 172.21.15.43:0/643333110 >> [v2:172.21.15.203:6832/830097474,v1:172.21.15.203:6833/830097474] conn(0x7efec006a080 0x7efec006c540 unknown :-1 s=CLOSED pgs=240 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:35:53.628 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:53.624+0000 7efedffff640 1 --2- 172.21.15.43:0/643333110 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7efee00b0f00 0x7efee01f6000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:35:53.628 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:53.624+0000 7efedffff640 1 -- 172.21.15.43:0/643333110 >> 172.21.15.43:0/643333110 conn(0x7efee006a210 msgr2=0x7efee00c5d50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-09T13:35:53.628 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:53.624+0000 7efedffff640 1 -- 172.21.15.43:0/643333110 shutdown_connections 2023-08-09T13:35:53.628 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:53.624+0000 7efedffff640 1 -- 172.21.15.43:0/643333110 wait complete. 2023-08-09T13:35:53.647 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-08-09T13:35:53.871 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:53.869+0000 7f5966102640 1 Processor -- start 2023-08-09T13:35:53.871 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:53.869+0000 7f5966102640 1 -- start start 2023-08-09T13:35:53.873 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:53.870+0000 7f5966102640 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7f596015f7b0 0x7f59600b4760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:35:53.873 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:53.870+0000 7f5966102640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f59600b1050 0x7f59600ccfa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:35:53.873 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:53.870+0000 7f5966102640 1 -- --> v1:172.21.15.43:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f59600b7bf0 con 0x7f59600cd4e0 2023-08-09T13:35:53.873 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:53.870+0000 7f5966102640 1 -- --> v2:172.21.15.43:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f59600cd970 con 0x7f59600b1050 2023-08-09T13:35:53.873 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:53.870+0000 7f5966102640 1 -- --> v2:172.21.15.203:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f59600b7ed0 con 0x7f596015f7b0 2023-08-09T13:35:53.874 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:53.871+0000 7f5965901640 1 --1- >> v1:172.21.15.43:6789/0 conn(0x7f59600cd4e0 0x7f59600ceae0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.43:6789/0 says I am v1:172.21.15.43:48624/0 (socket says 172.21.15.43:48624) 2023-08-09T13:35:53.875 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:53.871+0000 7f5965901640 1 -- 172.21.15.43:0/3341569473 learned_addr learned my addr 172.21.15.43:0/3341569473 (peer_addr_for_me v1:172.21.15.43:0/0) 2023-08-09T13:35:53.875 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:53.871+0000 7f59648ff640 1 --2- 172.21.15.43:0/3341569473 >> v2:172.21.15.43:3300/0 conn(0x7f59600b1050 0x7f59600ccfa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:35:53.875 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:53.871+0000 7f5965100640 1 --2- 172.21.15.43:0/3341569473 >> v2:172.21.15.203:3300/0 conn(0x7f596015f7b0 0x7f59600b4760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:35:53.875 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:53.871+0000 7f5957fff640 1 -- 172.21.15.43:0/3341569473 <== mon.0 v1:172.21.15.43:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2125343543 0 0) 0x7f59600b7bf0 con 0x7f59600cd4e0 2023-08-09T13:35:53.875 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:53.872+0000 7f5957fff640 1 -- 172.21.15.43:0/3341569473 --> v1:172.21.15.43:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f594c003520 con 0x7f59600cd4e0 2023-08-09T13:35:53.875 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:53.872+0000 7f59648ff640 1 -- 172.21.15.43:0/3341569473 >> v2:172.21.15.203:3300/0 conn(0x7f596015f7b0 msgr2=0x7f59600b4760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-09T13:35:53.875 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:53.872+0000 7f59648ff640 1 --2- 172.21.15.43:0/3341569473 >> v2:172.21.15.203:3300/0 conn(0x7f596015f7b0 0x7f59600b4760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:35:53.876 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:53.872+0000 7f59648ff640 1 -- 172.21.15.43:0/3341569473 >> v1:172.21.15.43:6789/0 conn(0x7f59600cd4e0 legacy=0x7f59600ceae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:35:53.876 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:53.872+0000 7f59648ff640 1 -- 172.21.15.43:0/3341569473 --> v2:172.21.15.43:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f59600d0370 con 0x7f59600b1050 2023-08-09T13:35:53.877 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:53.873+0000 7f59648ff640 1 --2- 172.21.15.43:0/3341569473 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f59600b1050 0x7f59600ccfa0 secure :-1 s=READY pgs=491 cs=0 l=1 rev1=1 crypto rx=0x7f596008aa40 tx=0x7f595002b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b94241c1f4d0134c server_cookie=0 in_seq=0 out_seq=0 2023-08-09T13:35:53.877 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:53.873+0000 7f5957fff640 1 -- 172.21.15.43:0/3341569473 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5950031830 con 0x7f59600b1050 2023-08-09T13:35:53.877 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:53.873+0000 7f5957fff640 1 -- 172.21.15.43:0/3341569473 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5950031990 con 0x7f59600b1050 2023-08-09T13:35:53.877 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:53.873+0000 7f5957fff640 1 -- 172.21.15.43:0/3341569473 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5950031d20 con 0x7f59600b1050 2023-08-09T13:35:53.877 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:53.874+0000 7f5966102640 1 -- 172.21.15.43:0/3341569473 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f59600b1050 msgr2=0x7f59600ccfa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:35:53.878 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:53.874+0000 7f5966102640 1 --2- 172.21.15.43:0/3341569473 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f59600b1050 0x7f59600ccfa0 secure :-1 s=READY pgs=491 cs=0 l=1 rev1=1 crypto rx=0x7f596008aa40 tx=0x7f595002b8d0 comp rx=0 tx=0).stop 2023-08-09T13:35:53.878 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:53.874+0000 7f5966102640 1 -- 172.21.15.43:0/3341569473 shutdown_connections 2023-08-09T13:35:53.878 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:53.874+0000 7f5966102640 1 --2- 172.21.15.43:0/3341569473 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f59600b1050 0x7f59600ccfa0 unknown :-1 s=CLOSED pgs=491 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:35:53.878 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:53.874+0000 7f5966102640 1 --2- 172.21.15.43:0/3341569473 >> v2:172.21.15.203:3300/0 conn(0x7f596015f7b0 0x7f59600b4760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:35:53.878 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:53.874+0000 7f5966102640 1 -- 172.21.15.43:0/3341569473 >> 172.21.15.43:0/3341569473 conn(0x7f596006a2b0 msgr2=0x7f596006a690 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-09T13:35:53.878 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:53.874+0000 7f5966102640 1 -- 172.21.15.43:0/3341569473 shutdown_connections 2023-08-09T13:35:53.878 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:53.874+0000 7f5966102640 1 -- 172.21.15.43:0/3341569473 wait complete. 2023-08-09T13:35:53.879 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:53.876+0000 7f5966102640 1 Processor -- start 2023-08-09T13:35:53.879 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:53.876+0000 7f5966102640 1 -- start start 2023-08-09T13:35:53.880 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:53.877+0000 7f5966102640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f59600b1050 0x7f59600c72d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:35:53.880 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:53.877+0000 7f5966102640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f59600cd4e0 0x7f59600d2010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:35:53.880 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:53.877+0000 7f5966102640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f596015f7b0 0x7f59600c9480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:35:53.880 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:53.877+0000 7f5966102640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f59600c60a0 con 0x7f59600b1050 2023-08-09T13:35:53.881 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:53.877+0000 7f5966102640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f59600c6380 con 0x7f596015f7b0 2023-08-09T13:35:53.881 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:53.877+0000 7f5966102640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f59600c6660 con 0x7f59600cd4e0 2023-08-09T13:35:53.881 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:53.878+0000 7f5965100640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f59600b1050 0x7f59600c72d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:35:53.882 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:53.878+0000 7f5965100640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f59600b1050 0x7f59600c72d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:35748/0 (socket says 172.21.15.43:35748) 2023-08-09T13:35:53.882 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:53.878+0000 7f5965100640 1 -- 172.21.15.43:0/4233965538 learned_addr learned my addr 172.21.15.43:0/4233965538 (peer_addr_for_me v2:172.21.15.43:0/0) 2023-08-09T13:35:53.885 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:53.878+0000 7f59648ff640 1 --2- 172.21.15.43:0/4233965538 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f59600cd4e0 0x7f59600d2010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:35:53.886 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:53.878+0000 7f5965901640 1 --2- 172.21.15.43:0/4233965538 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f596015f7b0 0x7f59600c9480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:35:53.886 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:53.878+0000 7f5965100640 1 -- 172.21.15.43:0/4233965538 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f59600cd4e0 msgr2=0x7f59600d2010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:35:53.886 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:53.878+0000 7f5965100640 1 --2- 172.21.15.43:0/4233965538 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f59600cd4e0 0x7f59600d2010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:35:53.886 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:53.878+0000 7f5965100640 1 -- 172.21.15.43:0/4233965538 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f596015f7b0 msgr2=0x7f59600c9480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:35:53.887 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:53.878+0000 7f5965100640 1 --2- 172.21.15.43:0/4233965538 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f596015f7b0 0x7f59600c9480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:35:53.887 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:53.878+0000 7f5965100640 1 -- 172.21.15.43:0/4233965538 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5950009400 con 0x7f59600b1050 2023-08-09T13:35:53.887 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:53.878+0000 7f59648ff640 1 --2- 172.21.15.43:0/4233965538 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f59600cd4e0 0x7f59600d2010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-08-09T13:35:53.888 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:53.878+0000 7f5965901640 1 --2- 172.21.15.43:0/4233965538 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f596015f7b0 0x7f59600c9480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-08-09T13:35:53.888 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:53.879+0000 7f5965100640 1 --2- 172.21.15.43:0/4233965538 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f59600b1050 0x7f59600c72d0 secure :-1 s=READY pgs=492 cs=0 l=1 rev1=1 crypto rx=0x7f595c00e870 tx=0x7f595c00ed30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-09T13:35:53.888 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:53.879+0000 7f5955ffb640 1 -- 172.21.15.43:0/4233965538 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f595c0094f0 con 0x7f59600b1050 2023-08-09T13:35:53.888 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:53.879+0000 7f5955ffb640 1 -- 172.21.15.43:0/4233965538 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f595c00c940 con 0x7f59600b1050 2023-08-09T13:35:53.888 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:53.879+0000 7f5955ffb640 1 -- 172.21.15.43:0/4233965538 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f595c00ccf0 con 0x7f59600b1050 2023-08-09T13:35:53.889 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:53.879+0000 7f5966102640 1 -- 172.21.15.43:0/4233965538 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f59600c6940 con 0x7f59600b1050 2023-08-09T13:35:53.889 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:53.879+0000 7f5966102640 1 -- 172.21.15.43:0/4233965538 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f59600c5cc0 con 0x7f59600b1050 2023-08-09T13:35:53.889 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:53.880+0000 7f5966102640 1 -- 172.21.15.43:0/4233965538 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f59600739f0 con 0x7f59600b1050 2023-08-09T13:35:53.889 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:53.881+0000 7f5955ffb640 1 -- 172.21.15.43:0/4233965538 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 5) v1 ==== 88341+0+0 (secure 0 0 0) 0x7f595c013470 con 0x7f59600b1050 2023-08-09T13:35:53.890 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:53.882+0000 7f5955ffb640 1 --2- 172.21.15.43:0/4233965538 >> [v2:172.21.15.203:6832/830097474,v1:172.21.15.203:6833/830097474] conn(0x7f594406a050 0x7f594406c510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:35:53.890 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:53.882+0000 7f5955ffb640 1 -- 172.21.15.43:0/4233965538 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 6871+0+0 (secure 0 0 0) 0x7f595c088ab0 con 0x7f59600b1050 2023-08-09T13:35:53.890 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:53.883+0000 7f59648ff640 1 --2- 172.21.15.43:0/4233965538 >> [v2:172.21.15.203:6832/830097474,v1:172.21.15.203:6833/830097474] conn(0x7f594406a050 0x7f594406c510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:35:53.891 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:53.884+0000 7f59648ff640 1 --2- 172.21.15.43:0/4233965538 >> [v2:172.21.15.203:6832/830097474,v1:172.21.15.203:6833/830097474] conn(0x7f594406a050 0x7f594406c510 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7f5950036000 tx=0x7f5950008030 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-09T13:35:53.891 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:53.887+0000 7f5955ffb640 1 -- 172.21.15.43:0/4233965538 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178066 (secure 0 0 0) 0x7f595c053dd0 con 0x7f59600b1050 2023-08-09T13:35:54.122 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:54.119+0000 7f5966102640 1 -- 172.21.15.43:0/4233965538 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f59600cc010 con 0x7f59600b1050 2023-08-09T13:35:54.123 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:54.120+0000 7f5955ffb640 1 -- 172.21.15.43:0/4233965538 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 9 v9) v1 ==== 93+0+4874 (secure 0 0 0) 0x7f595c0573f0 con 0x7f59600b1050 2023-08-09T13:35:54.123 INFO:teuthology.orchestra.run.smithi043.stdout: 2023-08-09T13:35:54.124 INFO:teuthology.orchestra.run.smithi043.stdout:{"epoch":9,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4236,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.43:6833/3917539953","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6832","nonce":3917539953},{"type":"v1","addr":"172.21.15.43:6833","nonce":3917539953}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2},{"gid":4247,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.203:6837/2467794159","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6835","nonce":2467794159},{"type":"v1","addr":"172.21.15.203:6837","nonce":2467794159}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":4248,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.43:6835/1818380189","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6834","nonce":1818380189},{"type":"v1","addr":"172.21.15.43:6835","nonce":1818380189}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":9,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-08-09T13:34:02.575398+0000","modified":"2023-08-09T13:34:31.841945+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":4250},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4250":{"gid":4250,"name":"d","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.203:6836/798312463","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6834","nonce":798312463},{"type":"v1","addr":"172.21.15.203:6836","nonce":798312463}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[4,5],"metadata_pool":3,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-08-09T13:35:54.124 INFO:teuthology.orchestra.run.smithi043.stderr:dumped fsmap epoch 9 2023-08-09T13:35:54.125 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:54.123+0000 7f5966102640 1 -- 172.21.15.43:0/4233965538 >> [v2:172.21.15.203:6832/830097474,v1:172.21.15.203:6833/830097474] conn(0x7f594406a050 msgr2=0x7f594406c510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:35:54.126 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:54.123+0000 7f5966102640 1 --2- 172.21.15.43:0/4233965538 >> [v2:172.21.15.203:6832/830097474,v1:172.21.15.203:6833/830097474] conn(0x7f594406a050 0x7f594406c510 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7f5950036000 tx=0x7f5950008030 comp rx=0 tx=0).stop 2023-08-09T13:35:54.126 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:54.123+0000 7f5966102640 1 -- 172.21.15.43:0/4233965538 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f59600b1050 msgr2=0x7f59600c72d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:35:54.126 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:54.123+0000 7f5966102640 1 --2- 172.21.15.43:0/4233965538 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f59600b1050 0x7f59600c72d0 secure :-1 s=READY pgs=492 cs=0 l=1 rev1=1 crypto rx=0x7f595c00e870 tx=0x7f595c00ed30 comp rx=0 tx=0).stop 2023-08-09T13:35:54.126 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:54.123+0000 7f5966102640 1 -- 172.21.15.43:0/4233965538 shutdown_connections 2023-08-09T13:35:54.127 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:54.123+0000 7f5966102640 1 --2- 172.21.15.43:0/4233965538 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f596015f7b0 0x7f59600c9480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:35:54.127 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:54.123+0000 7f5966102640 1 --2- 172.21.15.43:0/4233965538 >> [v2:172.21.15.203:6832/830097474,v1:172.21.15.203:6833/830097474] conn(0x7f594406a050 0x7f594406c510 unknown :-1 s=CLOSED pgs=241 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:35:54.127 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:54.123+0000 7f5966102640 1 --2- 172.21.15.43:0/4233965538 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f59600cd4e0 0x7f59600d2010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:35:54.127 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:54.123+0000 7f5966102640 1 --2- 172.21.15.43:0/4233965538 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f59600b1050 0x7f59600c72d0 unknown :-1 s=CLOSED pgs=492 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:35:54.127 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:54.123+0000 7f5966102640 1 -- 172.21.15.43:0/4233965538 >> 172.21.15.43:0/4233965538 conn(0x7f596006a2b0 msgr2=0x7f596015aaf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-09T13:35:54.127 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:54.124+0000 7f5966102640 1 -- 172.21.15.43:0/4233965538 shutdown_connections 2023-08-09T13:35:54.127 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:54.124+0000 7f5966102640 1 -- 172.21.15.43:0/4233965538 wait complete. 2023-08-09T13:35:54.144 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-08-09T13:35:54.358 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:54.356+0000 7f7f0166c640 1 Processor -- start 2023-08-09T13:35:54.359 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:54.356+0000 7f7f0166c640 1 -- start start 2023-08-09T13:35:54.360 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:54.358+0000 7f7f0166c640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f7efc143c20 0x7f7efc06f540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:35:54.361 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:54.358+0000 7f7f0166c640 1 -- --> v1:172.21.15.43:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7efc06c440 con 0x7f7efc0c86f0 2023-08-09T13:35:54.361 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:54.358+0000 7f7f0166c640 1 -- --> v1:172.21.15.203:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7efc06b830 con 0x7f7efc06cfe0 2023-08-09T13:35:54.361 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:54.358+0000 7f7f0166c640 1 -- --> v2:172.21.15.43:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7efc144000 con 0x7f7efc143c20 2023-08-09T13:35:54.361 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:54.358+0000 7f7efa7fc640 1 --1- >> v1:172.21.15.43:6789/0 conn(0x7f7efc0c86f0 0x7f7efc06c8d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.43:6789/0 says I am v1:172.21.15.43:48628/0 (socket says 172.21.15.43:48628) 2023-08-09T13:35:54.362 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:54.358+0000 7f7efa7fc640 1 -- 172.21.15.43:0/188183474 learned_addr learned my addr 172.21.15.43:0/188183474 (peer_addr_for_me v1:172.21.15.43:0/0) 2023-08-09T13:35:54.362 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:54.359+0000 7f7efaffd640 1 --2- 172.21.15.43:0/188183474 >> v2:172.21.15.43:3300/0 conn(0x7f7efc143c20 0x7f7efc06f540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:35:54.362 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:54.359+0000 7f7ef9ffb640 1 -- 172.21.15.43:0/188183474 <== mon.0 v1:172.21.15.43:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3114064158 0 0) 0x7f7efc06c440 con 0x7f7efc0c86f0 2023-08-09T13:35:54.362 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:54.359+0000 7f7ef9ffb640 1 -- 172.21.15.43:0/188183474 --> v1:172.21.15.43:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f7ee4003520 con 0x7f7efc0c86f0 2023-08-09T13:35:54.363 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:54.359+0000 7f7efaffd640 1 -- 172.21.15.43:0/188183474 >> v1:172.21.15.203:6789/0 conn(0x7f7efc06cfe0 legacy=0x7f7efc0ccc00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:35:54.363 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:54.359+0000 7f7efaffd640 1 -- 172.21.15.43:0/188183474 >> v1:172.21.15.43:6789/0 conn(0x7f7efc0c86f0 legacy=0x7f7efc06c8d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:35:54.363 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:54.359+0000 7f7efaffd640 1 -- 172.21.15.43:0/188183474 --> v2:172.21.15.43:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7efc06bb10 con 0x7f7efc143c20 2023-08-09T13:35:54.363 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:54.360+0000 7f7efaffd640 1 --2- 172.21.15.43:0/188183474 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f7efc143c20 0x7f7efc06f540 secure :-1 s=READY pgs=494 cs=0 l=1 rev1=1 crypto rx=0x7f7eec0097a0 tx=0x7f7eec02b840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3722a9502ca9dea0 server_cookie=0 in_seq=0 out_seq=0 2023-08-09T13:35:54.364 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:54.361+0000 7f7ef9ffb640 1 -- 172.21.15.43:0/188183474 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7eec02e070 con 0x7f7efc143c20 2023-08-09T13:35:54.364 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:54.361+0000 7f7ef9ffb640 1 -- 172.21.15.43:0/188183474 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7eec0315a0 con 0x7f7efc143c20 2023-08-09T13:35:54.364 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:54.361+0000 7f7ef9ffb640 1 -- 172.21.15.43:0/188183474 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7eec031910 con 0x7f7efc143c20 2023-08-09T13:35:54.365 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:54.361+0000 7f7f0166c640 1 -- 172.21.15.43:0/188183474 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f7efc143c20 msgr2=0x7f7efc06f540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:35:54.365 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:54.361+0000 7f7f0166c640 1 --2- 172.21.15.43:0/188183474 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f7efc143c20 0x7f7efc06f540 secure :-1 s=READY pgs=494 cs=0 l=1 rev1=1 crypto rx=0x7f7eec0097a0 tx=0x7f7eec02b840 comp rx=0 tx=0).stop 2023-08-09T13:35:54.365 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:54.362+0000 7f7f0166c640 1 -- 172.21.15.43:0/188183474 shutdown_connections 2023-08-09T13:35:54.365 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:54.362+0000 7f7f0166c640 1 --2- 172.21.15.43:0/188183474 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f7efc143c20 0x7f7efc06f540 unknown :-1 s=CLOSED pgs=494 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:35:54.365 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:54.362+0000 7f7f0166c640 1 -- 172.21.15.43:0/188183474 >> 172.21.15.43:0/188183474 conn(0x7f7efc0ce0a0 msgr2=0x7f7efc0ce480 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-09T13:35:54.365 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:54.362+0000 7f7f0166c640 1 -- 172.21.15.43:0/188183474 shutdown_connections 2023-08-09T13:35:54.365 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:54.362+0000 7f7f0166c640 1 -- 172.21.15.43:0/188183474 wait complete. 2023-08-09T13:35:54.366 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:54.363+0000 7f7f0166c640 1 Processor -- start 2023-08-09T13:35:54.366 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:54.364+0000 7f7f0166c640 1 -- start start 2023-08-09T13:35:54.367 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:54.365+0000 7f7f0166c640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f7efc06cfe0 0x7f7efc141340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:35:54.367 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:54.365+0000 7f7f0166c640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f7efc0c86f0 0x7f7efc0b3640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:35:54.367 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:54.365+0000 7f7f0166c640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f7efc143c20 0x7f7efc0c5170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:35:54.368 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:54.365+0000 7f7f0166c640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7efc141880 con 0x7f7efc06cfe0 2023-08-09T13:35:54.368 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:54.365+0000 7f7f0166c640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7efc0b3b80 con 0x7f7efc0c86f0 2023-08-09T13:35:54.368 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:54.365+0000 7f7f0166c640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7efc0c7ff0 con 0x7f7efc143c20 2023-08-09T13:35:54.368 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:54.365+0000 7f7efaffd640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f7efc06cfe0 0x7f7efc141340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:35:54.368 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:54.365+0000 7f7efaffd640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f7efc06cfe0 0x7f7efc141340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:35768/0 (socket says 172.21.15.43:35768) 2023-08-09T13:35:54.368 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:54.365+0000 7f7efaffd640 1 -- 172.21.15.43:0/519545887 learned_addr learned my addr 172.21.15.43:0/519545887 (peer_addr_for_me v2:172.21.15.43:0/0) 2023-08-09T13:35:54.369 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:54.365+0000 7f7efaffd640 1 -- 172.21.15.43:0/519545887 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f7efc143c20 msgr2=0x7f7efc0c5170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:35:54.369 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:54.365+0000 7f7efaffd640 1 --2- 172.21.15.43:0/519545887 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f7efc143c20 0x7f7efc0c5170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:35:54.369 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:54.365+0000 7f7efaffd640 1 -- 172.21.15.43:0/519545887 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f7efc0c86f0 msgr2=0x7f7efc0b3640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:35:54.370 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:54.365+0000 7f7efaffd640 1 --2- 172.21.15.43:0/519545887 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f7efc0c86f0 0x7f7efc0b3640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:35:54.370 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:54.365+0000 7f7efaffd640 1 -- 172.21.15.43:0/519545887 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7eec009400 con 0x7f7efc06cfe0 2023-08-09T13:35:54.370 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:54.366+0000 7f7efaffd640 1 --2- 172.21.15.43:0/519545887 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f7efc06cfe0 0x7f7efc141340 secure :-1 s=READY pgs=495 cs=0 l=1 rev1=1 crypto rx=0x7f7eec0097a0 tx=0x7f7eec0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-09T13:35:54.370 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:54.366+0000 7f7ee37fe640 1 -- 172.21.15.43:0/519545887 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7eec004230 con 0x7f7efc06cfe0 2023-08-09T13:35:54.370 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:54.366+0000 7f7ee37fe640 1 -- 172.21.15.43:0/519545887 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7eec004390 con 0x7f7efc06cfe0 2023-08-09T13:35:54.370 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:54.366+0000 7f7ee37fe640 1 -- 172.21.15.43:0/519545887 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7eec003810 con 0x7f7efc06cfe0 2023-08-09T13:35:54.370 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:54.366+0000 7f7f0166c640 1 -- 172.21.15.43:0/519545887 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7efc140860 con 0x7f7efc06cfe0 2023-08-09T13:35:54.371 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:54.367+0000 7f7f0166c640 1 -- 172.21.15.43:0/519545887 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7efc140e20 con 0x7f7efc06cfe0 2023-08-09T13:35:54.371 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:54.368+0000 7f7f0166c640 1 -- 172.21.15.43:0/519545887 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7efc141040 con 0x7f7efc06cfe0 2023-08-09T13:35:54.371 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:54.368+0000 7f7ee37fe640 1 -- 172.21.15.43:0/519545887 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 5) v1 ==== 88341+0+0 (secure 0 0 0) 0x7f7eec003970 con 0x7f7efc06cfe0 2023-08-09T13:35:54.373 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:54.370+0000 7f7ee37fe640 1 --2- 172.21.15.43:0/519545887 >> [v2:172.21.15.203:6832/830097474,v1:172.21.15.203:6833/830097474] conn(0x7f7ee806a050 0x7f7ee806c510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:35:54.373 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:54.370+0000 7f7ee37fe640 1 -- 172.21.15.43:0/519545887 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 6871+0+0 (secure 0 0 0) 0x7f7eec0aaa50 con 0x7f7efc06cfe0 2023-08-09T13:35:54.373 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:54.371+0000 7f7efa7fc640 1 --2- 172.21.15.43:0/519545887 >> [v2:172.21.15.203:6832/830097474,v1:172.21.15.203:6833/830097474] conn(0x7f7ee806a050 0x7f7ee806c510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:35:54.375 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:54.372+0000 7f7efa7fc640 1 --2- 172.21.15.43:0/519545887 >> [v2:172.21.15.203:6832/830097474,v1:172.21.15.203:6833/830097474] conn(0x7f7ee806a050 0x7f7ee806c510 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7f7efc0c69d0 tx=0x7f7ef0007930 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-09T13:35:54.377 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:54.374+0000 7f7ee37fe640 1 -- 172.21.15.43:0/519545887 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178066 (secure 0 0 0) 0x7f7eec075e00 con 0x7f7efc06cfe0 2023-08-09T13:35:54.570 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:54.567+0000 7f7f0166c640 1 -- 172.21.15.43:0/519545887 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f7efc0739f0 con 0x7f7efc06cfe0 2023-08-09T13:35:54.574 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:54.571+0000 7f7ee37fe640 1 -- 172.21.15.43:0/519545887 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v26) v1 ==== 74+0+20146 (secure 0 0 0) 0x7f7eec044340 con 0x7f7efc06cfe0 2023-08-09T13:35:54.574 INFO:teuthology.orchestra.run.smithi043.stdout: 2023-08-09T13:35:54.575 INFO:teuthology.orchestra.run.smithi043.stdout:{"epoch":26,"fsid":"daac87ee-1337-4250-a955-aea4e408a39b","created":"2023-08-09T13:33:41.701198+0000","modified":"2023-08-09T13:34:48.711911+0000","last_up_change":"2023-08-09T13:33:53.783453+0000","last_in_change":"2023-08-09T13:33:45.148904+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":5,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":"rbd","create_time":"2023-08-09T13:33:54.365137+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":".mgr","create_time":"2023-08-09T13:33:54.831803+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_metadata","create_time":"2023-08-09T13:34:00.207974+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data","create_time":"2023-08-09T13:34:01.614765+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":5,"pool_name":"cephfs_data_ec","create_time":"2023-08-09T13:34:05.239583+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"24","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"cbd49ecd-e896-4fbd-864e-5c37a829b165","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6800","nonce":502667335},{"type":"v1","addr":"172.21.15.43:6801","nonce":502667335}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6802","nonce":502667335},{"type":"v1","addr":"172.21.15.43:6803","nonce":502667335}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6806","nonce":502667335},{"type":"v1","addr":"172.21.15.43:6807","nonce":502667335}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6804","nonce":502667335},{"type":"v1","addr":"172.21.15.43:6805","nonce":502667335}]},"public_addr":"172.21.15.43:6801/502667335","cluster_addr":"172.21.15.43:6803/502667335","heartbeat_back_addr":"172.21.15.43:6807/502667335","heartbeat_front_addr":"172.21.15.43:6805/502667335","state":["exists","up"]},{"osd":1,"uuid":"75c7e0fe-1ec7-4fe8-a2f2-3661a65b8500","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6808","nonce":3289134610},{"type":"v1","addr":"172.21.15.43:6809","nonce":3289134610}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6810","nonce":3289134610},{"type":"v1","addr":"172.21.15.43:6812","nonce":3289134610}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6821","nonce":3289134610},{"type":"v1","addr":"172.21.15.43:6824","nonce":3289134610}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6815","nonce":3289134610},{"type":"v1","addr":"172.21.15.43:6818","nonce":3289134610}]},"public_addr":"172.21.15.43:6809/3289134610","cluster_addr":"172.21.15.43:6812/3289134610","heartbeat_back_addr":"172.21.15.43:6824/3289134610","heartbeat_front_addr":"172.21.15.43:6818/3289134610","state":["exists","up"]},{"osd":2,"uuid":"bb5c41e9-7f5f-4125-9b01-3bb8bfc2ea83","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6811","nonce":4079709736},{"type":"v1","addr":"172.21.15.43:6814","nonce":4079709736}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6817","nonce":4079709736},{"type":"v1","addr":"172.21.15.43:6820","nonce":4079709736}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6829","nonce":4079709736},{"type":"v1","addr":"172.21.15.43:6831","nonce":4079709736}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6823","nonce":4079709736},{"type":"v1","addr":"172.21.15.43:6826","nonce":4079709736}]},"public_addr":"172.21.15.43:6814/4079709736","cluster_addr":"172.21.15.43:6820/4079709736","heartbeat_back_addr":"172.21.15.43:6831/4079709736","heartbeat_front_addr":"172.21.15.43:6826/4079709736","state":["exists","up"]},{"osd":3,"uuid":"bfe4c8b4-694a-465c-87e6-2b2aab852613","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6813","nonce":1709538759},{"type":"v1","addr":"172.21.15.43:6816","nonce":1709538759}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6819","nonce":1709538759},{"type":"v1","addr":"172.21.15.43:6822","nonce":1709538759}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6828","nonce":1709538759},{"type":"v1","addr":"172.21.15.43:6830","nonce":1709538759}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6825","nonce":1709538759},{"type":"v1","addr":"172.21.15.43:6827","nonce":1709538759}]},"public_addr":"172.21.15.43:6816/1709538759","cluster_addr":"172.21.15.43:6822/1709538759","heartbeat_back_addr":"172.21.15.43:6830/1709538759","heartbeat_front_addr":"172.21.15.43:6827/1709538759","state":["exists","up"]},{"osd":4,"uuid":"fa2e8a77-aba1-4756-853e-0d174a5bdeba","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6800","nonce":4196866481},{"type":"v1","addr":"172.21.15.203:6801","nonce":4196866481}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6802","nonce":4196866481},{"type":"v1","addr":"172.21.15.203:6803","nonce":4196866481}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6806","nonce":4196866481},{"type":"v1","addr":"172.21.15.203:6807","nonce":4196866481}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6804","nonce":4196866481},{"type":"v1","addr":"172.21.15.203:6805","nonce":4196866481}]},"public_addr":"172.21.15.203:6801/4196866481","cluster_addr":"172.21.15.203:6803/4196866481","heartbeat_back_addr":"172.21.15.203:6807/4196866481","heartbeat_front_addr":"172.21.15.203:6805/4196866481","state":["exists","up"]},{"osd":5,"uuid":"81811e4b-4d24-4f51-a82c-f1d700a2a00d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6816","nonce":3059290081},{"type":"v1","addr":"172.21.15.203:6817","nonce":3059290081}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6818","nonce":3059290081},{"type":"v1","addr":"172.21.15.203:6819","nonce":3059290081}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6822","nonce":3059290081},{"type":"v1","addr":"172.21.15.203:6823","nonce":3059290081}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6820","nonce":3059290081},{"type":"v1","addr":"172.21.15.203:6821","nonce":3059290081}]},"public_addr":"172.21.15.203:6817/3059290081","cluster_addr":"172.21.15.203:6819/3059290081","heartbeat_back_addr":"172.21.15.203:6823/3059290081","heartbeat_front_addr":"172.21.15.203:6821/3059290081","state":["exists","up"]},{"osd":6,"uuid":"307bac4d-6a5b-443d-af75-56d695d7674f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6824","nonce":512155569},{"type":"v1","addr":"172.21.15.203:6825","nonce":512155569}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6826","nonce":512155569},{"type":"v1","addr":"172.21.15.203:6827","nonce":512155569}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6830","nonce":512155569},{"type":"v1","addr":"172.21.15.203:6831","nonce":512155569}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6828","nonce":512155569},{"type":"v1","addr":"172.21.15.203:6829","nonce":512155569}]},"public_addr":"172.21.15.203:6825/512155569","cluster_addr":"172.21.15.203:6827/512155569","heartbeat_back_addr":"172.21.15.203:6831/512155569","heartbeat_front_addr":"172.21.15.203:6829/512155569","state":["exists","up"]},{"osd":7,"uuid":"a95dc7ca-9f1b-4295-af70-68a22a2d7d5e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6808","nonce":3331450438},{"type":"v1","addr":"172.21.15.203:6809","nonce":3331450438}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6810","nonce":3331450438},{"type":"v1","addr":"172.21.15.203:6811","nonce":3331450438}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6814","nonce":3331450438},{"type":"v1","addr":"172.21.15.203:6815","nonce":3331450438}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6812","nonce":3331450438},{"type":"v1","addr":"172.21.15.203:6813","nonce":3331450438}]},"public_addr":"172.21.15.203:6809/3331450438","cluster_addr":"172.21.15.203:6811/3331450438","heartbeat_back_addr":"172.21.15.203:6815/3331450438","heartbeat_front_addr":"172.21.15.203:6813/3331450438","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-09T13:33:50.547652+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-09T13:33:50.589469+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-09T13:33:50.561724+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-09T13:33:50.599650+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-09T13:33:50.411855+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-09T13:33:50.219681+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-09T13:33:50.504664+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-09T13:33:50.441248+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"4.14","mappings":[{"from":1,"to":5}]},{"pgid":"4.29","mappings":[{"from":3,"to":2}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-08-09T13:35:54.577 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:54.573+0000 7f7f0166c640 1 -- 172.21.15.43:0/519545887 >> [v2:172.21.15.203:6832/830097474,v1:172.21.15.203:6833/830097474] conn(0x7f7ee806a050 msgr2=0x7f7ee806c510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:35:54.577 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:54.573+0000 7f7f0166c640 1 --2- 172.21.15.43:0/519545887 >> [v2:172.21.15.203:6832/830097474,v1:172.21.15.203:6833/830097474] conn(0x7f7ee806a050 0x7f7ee806c510 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7f7efc0c69d0 tx=0x7f7ef0007930 comp rx=0 tx=0).stop 2023-08-09T13:35:54.577 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:54.573+0000 7f7f0166c640 1 -- 172.21.15.43:0/519545887 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f7efc06cfe0 msgr2=0x7f7efc141340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:35:54.577 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:54.573+0000 7f7f0166c640 1 --2- 172.21.15.43:0/519545887 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f7efc06cfe0 0x7f7efc141340 secure :-1 s=READY pgs=495 cs=0 l=1 rev1=1 crypto rx=0x7f7eec0097a0 tx=0x7f7eec0046c0 comp rx=0 tx=0).stop 2023-08-09T13:35:54.577 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:54.574+0000 7f7f0166c640 1 -- 172.21.15.43:0/519545887 shutdown_connections 2023-08-09T13:35:54.577 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:54.574+0000 7f7f0166c640 1 --2- 172.21.15.43:0/519545887 >> [v2:172.21.15.203:6832/830097474,v1:172.21.15.203:6833/830097474] conn(0x7f7ee806a050 0x7f7ee806c510 unknown :-1 s=CLOSED pgs=242 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:35:54.578 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:54.574+0000 7f7f0166c640 1 --2- 172.21.15.43:0/519545887 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f7efc143c20 0x7f7efc0c5170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:35:54.578 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:54.574+0000 7f7f0166c640 1 --2- 172.21.15.43:0/519545887 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f7efc0c86f0 0x7f7efc0b3640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:35:54.578 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:54.574+0000 7f7f0166c640 1 --2- 172.21.15.43:0/519545887 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f7efc06cfe0 0x7f7efc141340 unknown :-1 s=CLOSED pgs=495 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:35:54.578 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:54.574+0000 7f7f0166c640 1 -- 172.21.15.43:0/519545887 >> 172.21.15.43:0/519545887 conn(0x7f7efc0ce0a0 msgr2=0x7f7efc0ae5e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-09T13:35:54.579 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:54.574+0000 7f7f0166c640 1 -- 172.21.15.43:0/519545887 shutdown_connections 2023-08-09T13:35:54.579 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:54.574+0000 7f7f0166c640 1 -- 172.21.15.43:0/519545887 wait complete. 2023-08-09T13:35:54.596 INFO:tasks.cephfs.filesystem:Destroying file system cephfs and related pools 2023-08-09T13:35:54.596 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-08-09T13:35:54.818 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:54.816+0000 7f3d0985e640 1 Processor -- start 2023-08-09T13:35:54.819 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:54.816+0000 7f3d0985e640 1 -- start start 2023-08-09T13:35:54.820 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:54.818+0000 7f3d0985e640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f3d04142ca0 0x7f3d040c96e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:35:54.821 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:54.818+0000 7f3d0985e640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f3d0415ce40 0x7f3d040cce00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:35:54.821 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:54.818+0000 7f3d0985e640 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7f3d040cd340 0x7f3d040cbc70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:35:54.821 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:54.818+0000 7f3d0985e640 1 -- --> v2:172.21.15.43:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3d040cd7b0 con 0x7f3d0415ce40 2023-08-09T13:35:54.821 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:54.818+0000 7f3d0985e640 1 -- --> v2:172.21.15.203:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3d04143080 con 0x7f3d040cd340 2023-08-09T13:35:54.821 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:54.818+0000 7f3d0985e640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3d0406d400 con 0x7f3d04142ca0 2023-08-09T13:35:54.821 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:54.818+0000 7f3cfbfff640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f3d0415ce40 0x7f3d040cce00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:35:54.822 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:54.818+0000 7f3cfbfff640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f3d0415ce40 0x7f3d040cce00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:35780/0 (socket says 172.21.15.43:35780) 2023-08-09T13:35:54.822 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:54.818+0000 7f3cfbfff640 1 -- 172.21.15.43:0/1430135840 learned_addr learned my addr 172.21.15.43:0/1430135840 (peer_addr_for_me v2:172.21.15.43:0/0) 2023-08-09T13:35:54.822 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:54.818+0000 7f3d02ffd640 1 --2- 172.21.15.43:0/1430135840 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f3d04142ca0 0x7f3d040c96e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:35:54.823 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:54.819+0000 7f3d037fe640 1 --2- 172.21.15.43:0/1430135840 >> v2:172.21.15.203:3300/0 conn(0x7f3d040cd340 0x7f3d040cbc70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:35:54.823 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:54.819+0000 7f3cfbfff640 1 -- 172.21.15.43:0/1430135840 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f3d04142ca0 msgr2=0x7f3d040c96e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-09T13:35:54.823 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:54.819+0000 7f3cfbfff640 1 --2- 172.21.15.43:0/1430135840 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f3d04142ca0 0x7f3d040c96e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:35:54.823 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:54.819+0000 7f3cfbfff640 1 -- 172.21.15.43:0/1430135840 >> v2:172.21.15.203:3300/0 conn(0x7f3d040cd340 msgr2=0x7f3d040cbc70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-09T13:35:54.823 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:54.819+0000 7f3cfbfff640 1 --2- 172.21.15.43:0/1430135840 >> v2:172.21.15.203:3300/0 conn(0x7f3d040cd340 0x7f3d040cbc70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:35:54.823 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:54.819+0000 7f3cfbfff640 1 -- 172.21.15.43:0/1430135840 --> v2:172.21.15.43:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3d040c9d90 con 0x7f3d0415ce40 2023-08-09T13:35:54.823 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:54.820+0000 7f3cfbfff640 1 --2- 172.21.15.43:0/1430135840 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f3d0415ce40 0x7f3d040cce00 secure :-1 s=READY pgs=496 cs=0 l=1 rev1=1 crypto rx=0x7f3d040ccd10 tx=0x7f3cec02b8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=43e428c760d2eac2 server_cookie=0 in_seq=0 out_seq=0 2023-08-09T13:35:54.824 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:54.820+0000 7f3d027fc640 1 -- 172.21.15.43:0/1430135840 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3cec031640 con 0x7f3d0415ce40 2023-08-09T13:35:54.824 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:54.820+0000 7f3d027fc640 1 -- 172.21.15.43:0/1430135840 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3cec0317a0 con 0x7f3d0415ce40 2023-08-09T13:35:54.824 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:54.820+0000 7f3d027fc640 1 -- 172.21.15.43:0/1430135840 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3cec031b30 con 0x7f3d0415ce40 2023-08-09T13:35:54.825 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:54.821+0000 7f3d0985e640 1 -- 172.21.15.43:0/1430135840 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f3d0415ce40 msgr2=0x7f3d040cce00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:35:54.825 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:54.821+0000 7f3d0985e640 1 --2- 172.21.15.43:0/1430135840 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f3d0415ce40 0x7f3d040cce00 secure :-1 s=READY pgs=496 cs=0 l=1 rev1=1 crypto rx=0x7f3d040ccd10 tx=0x7f3cec02b8f0 comp rx=0 tx=0).stop 2023-08-09T13:35:54.825 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:54.821+0000 7f3d0985e640 1 -- 172.21.15.43:0/1430135840 shutdown_connections 2023-08-09T13:35:54.825 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:54.821+0000 7f3d0985e640 1 --2- 172.21.15.43:0/1430135840 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f3d0415ce40 0x7f3d040cce00 unknown :-1 s=CLOSED pgs=496 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:35:54.825 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:54.821+0000 7f3d0985e640 1 --2- 172.21.15.43:0/1430135840 >> v2:172.21.15.203:3300/0 conn(0x7f3d040cd340 0x7f3d040cbc70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:35:54.825 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:54.821+0000 7f3d0985e640 1 --2- 172.21.15.43:0/1430135840 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f3d04142ca0 0x7f3d040c96e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:35:54.825 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:54.821+0000 7f3d0985e640 1 -- 172.21.15.43:0/1430135840 >> 172.21.15.43:0/1430135840 conn(0x7f3d040ce0a0 msgr2=0x7f3d040ce480 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-09T13:35:54.826 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:54.821+0000 7f3d0985e640 1 -- 172.21.15.43:0/1430135840 shutdown_connections 2023-08-09T13:35:54.826 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:54.821+0000 7f3d0985e640 1 -- 172.21.15.43:0/1430135840 wait complete. 2023-08-09T13:35:54.827 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:54.823+0000 7f3d0985e640 1 Processor -- start 2023-08-09T13:35:54.827 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:54.823+0000 7f3d0985e640 1 -- start start 2023-08-09T13:35:54.827 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:54.824+0000 7f3d0985e640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f3d040cd340 0x7f3d04156b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:35:54.827 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:54.824+0000 7f3d0985e640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f3d04142ca0 0x7f3d04158420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:35:54.828 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:54.824+0000 7f3d0985e640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f3d0415ce40 0x7f3d04161930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:35:54.828 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:54.824+0000 7f3d0985e640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3d0415bdf0 con 0x7f3d04142ca0 2023-08-09T13:35:54.828 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:54.824+0000 7f3d0985e640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3d0415b4c0 con 0x7f3d040cd340 2023-08-09T13:35:54.828 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:54.824+0000 7f3d0985e640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3d0415b7a0 con 0x7f3d0415ce40 2023-08-09T13:35:54.828 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:54.824+0000 7f3cfbfff640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f3d04142ca0 0x7f3d04158420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:35:54.828 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:54.824+0000 7f3cfbfff640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f3d04142ca0 0x7f3d04158420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:35788/0 (socket says 172.21.15.43:35788) 2023-08-09T13:35:54.829 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:54.824+0000 7f3cfbfff640 1 -- 172.21.15.43:0/4019147493 learned_addr learned my addr 172.21.15.43:0/4019147493 (peer_addr_for_me v2:172.21.15.43:0/0) 2023-08-09T13:35:54.829 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:54.825+0000 7f3d02ffd640 1 --2- 172.21.15.43:0/4019147493 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f3d040cd340 0x7f3d04156b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:35:54.829 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:54.825+0000 7f3d037fe640 1 --2- 172.21.15.43:0/4019147493 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f3d0415ce40 0x7f3d04161930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:35:54.829 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:54.825+0000 7f3cfbfff640 1 -- 172.21.15.43:0/4019147493 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f3d0415ce40 msgr2=0x7f3d04161930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:35:54.829 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:54.825+0000 7f3cfbfff640 1 --2- 172.21.15.43:0/4019147493 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f3d0415ce40 0x7f3d04161930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:35:54.829 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:54.825+0000 7f3cfbfff640 1 -- 172.21.15.43:0/4019147493 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f3d040cd340 msgr2=0x7f3d04156b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:35:54.829 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:54.825+0000 7f3cfbfff640 1 --2- 172.21.15.43:0/4019147493 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f3d040cd340 0x7f3d04156b20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:35:54.829 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:54.825+0000 7f3cfbfff640 1 -- 172.21.15.43:0/4019147493 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3cec0094d0 con 0x7f3d04142ca0 2023-08-09T13:35:54.830 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:54.825+0000 7f3d02ffd640 1 --2- 172.21.15.43:0/4019147493 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f3d040cd340 0x7f3d04156b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-08-09T13:35:54.830 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:54.825+0000 7f3cfbfff640 1 --2- 172.21.15.43:0/4019147493 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f3d04142ca0 0x7f3d04158420 secure :-1 s=READY pgs=497 cs=0 l=1 rev1=1 crypto rx=0x7f3cec000c00 tx=0x7f3cec0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-09T13:35:54.831 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:54.825+0000 7f3d00ff9640 1 -- 172.21.15.43:0/4019147493 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3cec03ac60 con 0x7f3d04142ca0 2023-08-09T13:35:54.831 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:54.825+0000 7f3d00ff9640 1 -- 172.21.15.43:0/4019147493 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3cec03adc0 con 0x7f3d04142ca0 2023-08-09T13:35:54.831 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:54.825+0000 7f3d00ff9640 1 -- 172.21.15.43:0/4019147493 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3cec044630 con 0x7f3d04142ca0 2023-08-09T13:35:54.831 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:54.825+0000 7f3d0985e640 1 -- 172.21.15.43:0/4019147493 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3d0415ba80 con 0x7f3d04142ca0 2023-08-09T13:35:54.831 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:54.826+0000 7f3d0985e640 1 -- 172.21.15.43:0/4019147493 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3d0415c280 con 0x7f3d04142ca0 2023-08-09T13:35:54.831 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:54.827+0000 7f3d0985e640 1 -- 172.21.15.43:0/4019147493 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3d040739f0 con 0x7f3d04142ca0 2023-08-09T13:35:54.831 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:54.828+0000 7f3d00ff9640 1 -- 172.21.15.43:0/4019147493 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 5) v1 ==== 88341+0+0 (secure 0 0 0) 0x7f3cec02f050 con 0x7f3d04142ca0 2023-08-09T13:35:54.832 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:54.828+0000 7f3d00ff9640 1 --2- 172.21.15.43:0/4019147493 >> [v2:172.21.15.203:6832/830097474,v1:172.21.15.203:6833/830097474] conn(0x7f3ce406a050 0x7f3ce406c510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:35:54.832 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:54.829+0000 7f3d00ff9640 1 -- 172.21.15.43:0/4019147493 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 6871+0+0 (secure 0 0 0) 0x7f3cec0aa870 con 0x7f3d04142ca0 2023-08-09T13:35:54.832 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:54.829+0000 7f3d02ffd640 1 --2- 172.21.15.43:0/4019147493 >> [v2:172.21.15.203:6832/830097474,v1:172.21.15.203:6833/830097474] conn(0x7f3ce406a050 0x7f3ce406c510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:35:54.833 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:54.831+0000 7f3d02ffd640 1 --2- 172.21.15.43:0/4019147493 >> [v2:172.21.15.203:6832/830097474,v1:172.21.15.203:6833/830097474] conn(0x7f3ce406a050 0x7f3ce406c510 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7f3cf4006fd0 tx=0x7f3cf4008040 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-09T13:35:54.835 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:54.833+0000 7f3d00ff9640 1 -- 172.21.15.43:0/4019147493 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178066 (secure 0 0 0) 0x7f3cec075b90 con 0x7f3d04142ca0 2023-08-09T13:35:55.068 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:55.065+0000 7f3d0985e640 1 -- 172.21.15.43:0/4019147493 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f3d040cb700 con 0x7f3d04142ca0 2023-08-09T13:35:55.069 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:55.067+0000 7f3d00ff9640 1 -- 172.21.15.43:0/4019147493 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 9 v9) v1 ==== 93+0+4874 (secure 0 0 0) 0x7f3cec031e60 con 0x7f3d04142ca0 2023-08-09T13:35:55.069 INFO:teuthology.orchestra.run.smithi043.stdout: 2023-08-09T13:35:55.070 INFO:teuthology.orchestra.run.smithi043.stdout:{"epoch":9,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4236,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.43:6833/3917539953","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6832","nonce":3917539953},{"type":"v1","addr":"172.21.15.43:6833","nonce":3917539953}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2},{"gid":4247,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.203:6837/2467794159","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6835","nonce":2467794159},{"type":"v1","addr":"172.21.15.203:6837","nonce":2467794159}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":4248,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.43:6835/1818380189","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6834","nonce":1818380189},{"type":"v1","addr":"172.21.15.43:6835","nonce":1818380189}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":9,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-08-09T13:34:02.575398+0000","modified":"2023-08-09T13:34:31.841945+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":4250},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4250":{"gid":4250,"name":"d","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.203:6836/798312463","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6834","nonce":798312463},{"type":"v1","addr":"172.21.15.203:6836","nonce":798312463}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[4,5],"metadata_pool":3,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-08-09T13:35:55.070 INFO:teuthology.orchestra.run.smithi043.stderr:dumped fsmap epoch 9 2023-08-09T13:35:55.072 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:55.069+0000 7f3d0985e640 1 -- 172.21.15.43:0/4019147493 >> [v2:172.21.15.203:6832/830097474,v1:172.21.15.203:6833/830097474] conn(0x7f3ce406a050 msgr2=0x7f3ce406c510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:35:55.072 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:55.069+0000 7f3d0985e640 1 --2- 172.21.15.43:0/4019147493 >> [v2:172.21.15.203:6832/830097474,v1:172.21.15.203:6833/830097474] conn(0x7f3ce406a050 0x7f3ce406c510 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7f3cf4006fd0 tx=0x7f3cf4008040 comp rx=0 tx=0).stop 2023-08-09T13:35:55.072 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:55.069+0000 7f3d0985e640 1 -- 172.21.15.43:0/4019147493 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f3d04142ca0 msgr2=0x7f3d04158420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:35:55.072 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:55.069+0000 7f3d0985e640 1 --2- 172.21.15.43:0/4019147493 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f3d04142ca0 0x7f3d04158420 secure :-1 s=READY pgs=497 cs=0 l=1 rev1=1 crypto rx=0x7f3cec000c00 tx=0x7f3cec0046c0 comp rx=0 tx=0).stop 2023-08-09T13:35:55.072 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:55.069+0000 7f3d0985e640 1 -- 172.21.15.43:0/4019147493 shutdown_connections 2023-08-09T13:35:55.073 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:55.069+0000 7f3d0985e640 1 --2- 172.21.15.43:0/4019147493 >> [v2:172.21.15.203:6832/830097474,v1:172.21.15.203:6833/830097474] conn(0x7f3ce406a050 0x7f3ce406c510 unknown :-1 s=CLOSED pgs=243 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:35:55.073 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:55.069+0000 7f3d0985e640 1 --2- 172.21.15.43:0/4019147493 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f3d0415ce40 0x7f3d04161930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:35:55.073 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:55.069+0000 7f3d0985e640 1 --2- 172.21.15.43:0/4019147493 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f3d04142ca0 0x7f3d04158420 unknown :-1 s=CLOSED pgs=497 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:35:55.073 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:55.069+0000 7f3d0985e640 1 --2- 172.21.15.43:0/4019147493 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f3d040cd340 0x7f3d04156b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:35:55.073 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:55.069+0000 7f3d0985e640 1 -- 172.21.15.43:0/4019147493 >> 172.21.15.43:0/4019147493 conn(0x7f3d040ce0a0 msgr2=0x7f3d0406f8d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-09T13:35:55.073 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:55.070+0000 7f3d0985e640 1 -- 172.21.15.43:0/4019147493 shutdown_connections 2023-08-09T13:35:55.073 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:55.070+0000 7f3d0985e640 1 -- 172.21.15.43:0/4019147493 wait complete. 2023-08-09T13:35:55.092 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-08-09T13:35:55.316 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:55.314+0000 7ff7bd0d7640 1 Processor -- start 2023-08-09T13:35:55.317 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:55.314+0000 7ff7bd0d7640 1 -- start start 2023-08-09T13:35:55.318 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:55.315+0000 7ff7bd0d7640 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7ff7b80c96c0 0x7ff7b815ad20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:35:55.318 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:55.315+0000 7ff7bd0d7640 1 -- --> v1:172.21.15.43:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff7b80cdc70 con 0x7ff7b815a6a0 2023-08-09T13:35:55.318 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:55.315+0000 7ff7bd0d7640 1 -- --> v1:172.21.15.203:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff7b8070910 con 0x7ff7b80713b0 2023-08-09T13:35:55.318 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:55.315+0000 7ff7bd0d7640 1 -- --> v2:172.21.15.203:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff7b80c88c0 con 0x7ff7b80c96c0 2023-08-09T13:35:55.319 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:55.316+0000 7ff7b7fff640 1 --1- >> v1:172.21.15.43:6789/0 conn(0x7ff7b815a6a0 0x7ff7b80cbd90 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.43:6789/0 says I am v1:172.21.15.43:39106/0 (socket says 172.21.15.43:39106) 2023-08-09T13:35:55.320 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:55.316+0000 7ff7b7fff640 1 -- 172.21.15.43:0/1660991215 learned_addr learned my addr 172.21.15.43:0/1660991215 (peer_addr_for_me v1:172.21.15.43:0/0) 2023-08-09T13:35:55.320 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:55.316+0000 7ff7b77fe640 1 -- 172.21.15.43:0/1660991215 <== mon.0 v1:172.21.15.43:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 672487045 0 0) 0x7ff7b80cdc70 con 0x7ff7b815a6a0 2023-08-09T13:35:55.320 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:55.316+0000 7ff7b77fe640 1 -- 172.21.15.43:0/1660991215 --> v1:172.21.15.43:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ff7a4003520 con 0x7ff7b815a6a0 2023-08-09T13:35:55.320 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:55.316+0000 7ff7bc8d6640 1 --2- 172.21.15.43:0/1660991215 >> v2:172.21.15.203:3300/0 conn(0x7ff7b80c96c0 0x7ff7b815ad20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:35:55.320 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:55.317+0000 7ff7b77fe640 1 -- 172.21.15.43:0/1660991215 <== mon.0 v1:172.21.15.43:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2734045934 0 0) 0x7ff7a4003520 con 0x7ff7b815a6a0 2023-08-09T13:35:55.320 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:55.317+0000 7ff7b77fe640 1 -- 172.21.15.43:0/1660991215 >> v2:172.21.15.203:3300/0 conn(0x7ff7b80c96c0 msgr2=0x7ff7b815ad20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-09T13:35:55.321 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:55.317+0000 7ff7b77fe640 1 --2- 172.21.15.43:0/1660991215 >> v2:172.21.15.203:3300/0 conn(0x7ff7b80c96c0 0x7ff7b815ad20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:35:55.321 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:55.317+0000 7ff7b77fe640 1 -- 172.21.15.43:0/1660991215 >> v1:172.21.15.203:6789/0 conn(0x7ff7b80713b0 legacy=0x7ff7b80c8fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:35:55.321 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:55.317+0000 7ff7b77fe640 1 -- 172.21.15.43:0/1660991215 --> v1:172.21.15.43:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff7b8070bb0 con 0x7ff7b815a6a0 2023-08-09T13:35:55.321 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:55.317+0000 7ff7b77fe640 1 -- 172.21.15.43:0/1660991215 <== mon.0 v1:172.21.15.43:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1285150740 0 0) 0x7ff7a8001670 con 0x7ff7b815a6a0 2023-08-09T13:35:55.321 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:55.317+0000 7ff7b77fe640 1 -- 172.21.15.43:0/1660991215 >> v1:172.21.15.43:6789/0 conn(0x7ff7b815a6a0 legacy=0x7ff7b80cbd90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:35:55.321 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:55.317+0000 7ff7b77fe640 1 --2- 172.21.15.43:0/1660991215 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7ff7a4003970 0x7ff7a4003d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:35:55.321 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:55.317+0000 7ff7b77fe640 1 --2- 172.21.15.43:0/1660991215 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7ff7a40047c0 0x7ff7a4004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:35:55.321 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:55.317+0000 7ff7b77fe640 1 --2- 172.21.15.43:0/1660991215 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff7a40052a0 0x7ff7a400db90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:35:55.322 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:55.317+0000 7ff7b77fe640 1 -- 172.21.15.43:0/1660991215 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff7b80c88c0 con 0x7ff7a40052a0 2023-08-09T13:35:55.322 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:55.317+0000 7ff7b77fe640 1 -- 172.21.15.43:0/1660991215 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff7a400e380 con 0x7ff7a4003970 2023-08-09T13:35:55.322 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:55.317+0000 7ff7b77fe640 1 -- 172.21.15.43:0/1660991215 --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff7a400e540 con 0x7ff7a40047c0 2023-08-09T13:35:55.322 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:55.317+0000 7ff7bc8d6640 1 --2- 172.21.15.43:0/1660991215 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff7a40052a0 0x7ff7a400db90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:35:55.323 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:55.318+0000 7ff7b7fff640 1 --2- 172.21.15.43:0/1660991215 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7ff7a4003970 0x7ff7a4003d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:35:55.323 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:55.318+0000 7ff7affff640 1 --2- 172.21.15.43:0/1660991215 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7ff7a40047c0 0x7ff7a4004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:35:55.323 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:55.318+0000 7ff7bc8d6640 1 -- 172.21.15.43:0/1660991215 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7ff7a40047c0 msgr2=0x7ff7a4004c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-09T13:35:55.324 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:55.318+0000 7ff7bc8d6640 1 --2- 172.21.15.43:0/1660991215 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7ff7a40047c0 0x7ff7a4004c40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:35:55.324 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:55.318+0000 7ff7bc8d6640 1 -- 172.21.15.43:0/1660991215 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7ff7a4003970 msgr2=0x7ff7a4003d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-09T13:35:55.324 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:55.318+0000 7ff7bc8d6640 1 --2- 172.21.15.43:0/1660991215 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7ff7a4003970 0x7ff7a4003d90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:35:55.324 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:55.318+0000 7ff7bc8d6640 1 -- 172.21.15.43:0/1660991215 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff7a4003520 con 0x7ff7a40052a0 2023-08-09T13:35:55.324 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:55.319+0000 7ff7bc8d6640 1 -- 172.21.15.43:0/1660991215 reap_dead start 2023-08-09T13:35:55.324 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:55.319+0000 7ff7bc8d6640 1 --2- 172.21.15.43:0/1660991215 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff7a40052a0 0x7ff7a400db90 secure :-1 s=READY pgs=499 cs=0 l=1 rev1=1 crypto rx=0x7ff7b0007330 tx=0x7ff7a40057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c3aac4dbf21cd4ca server_cookie=0 in_seq=0 out_seq=0 2023-08-09T13:35:55.324 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:55.319+0000 7ff7b77fe640 1 -- 172.21.15.43:0/1660991215 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff7b002ccd0 con 0x7ff7a40052a0 2023-08-09T13:35:55.325 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:55.319+0000 7ff7b77fe640 1 -- 172.21.15.43:0/1660991215 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff7b002ce30 con 0x7ff7a40052a0 2023-08-09T13:35:55.325 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:55.319+0000 7ff7b77fe640 1 -- 172.21.15.43:0/1660991215 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff7b0003bd0 con 0x7ff7a40052a0 2023-08-09T13:35:55.325 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:55.319+0000 7ff7bd0d7640 1 -- 172.21.15.43:0/1660991215 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff7a40052a0 msgr2=0x7ff7a400db90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:35:55.325 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:55.319+0000 7ff7bd0d7640 1 --2- 172.21.15.43:0/1660991215 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff7a40052a0 0x7ff7a400db90 secure :-1 s=READY pgs=499 cs=0 l=1 rev1=1 crypto rx=0x7ff7b0007330 tx=0x7ff7a40057f0 comp rx=0 tx=0).stop 2023-08-09T13:35:55.325 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:55.320+0000 7ff7bd0d7640 1 -- 172.21.15.43:0/1660991215 shutdown_connections 2023-08-09T13:35:55.325 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:55.320+0000 7ff7bd0d7640 1 --2- 172.21.15.43:0/1660991215 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff7a40052a0 0x7ff7a400db90 unknown :-1 s=CLOSED pgs=499 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:35:55.325 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:55.320+0000 7ff7bd0d7640 1 -- 172.21.15.43:0/1660991215 >> 172.21.15.43:0/1660991215 conn(0x7ff7b80cde90 msgr2=0x7ff7b80ce2a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-09T13:35:55.326 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:55.320+0000 7ff7bd0d7640 1 -- 172.21.15.43:0/1660991215 shutdown_connections 2023-08-09T13:35:55.326 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:55.320+0000 7ff7bd0d7640 1 -- 172.21.15.43:0/1660991215 wait complete. 2023-08-09T13:35:55.327 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:55.322+0000 7ff7bd0d7640 1 Processor -- start 2023-08-09T13:35:55.327 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:55.322+0000 7ff7bd0d7640 1 -- start start 2023-08-09T13:35:55.327 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:55.323+0000 7ff7bd0d7640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff7a40052a0 0x7ff7b815d020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:35:55.327 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:55.323+0000 7ff7bd0d7640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7ff7b806b190 0x7ff7b8099380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:35:55.328 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:55.323+0000 7ff7bd0d7640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7ff7b80998c0 0x7ff7b80b1330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:35:55.328 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:55.323+0000 7ff7bd0d7640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff7b806c130 con 0x7ff7a40052a0 2023-08-09T13:35:55.328 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:55.323+0000 7ff7bd0d7640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff7b806bb70 con 0x7ff7b80998c0 2023-08-09T13:35:55.328 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:55.323+0000 7ff7bd0d7640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff7b806be50 con 0x7ff7b806b190 2023-08-09T13:35:55.328 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:55.323+0000 7ff7b7fff640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff7a40052a0 0x7ff7b815d020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:35:55.328 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:55.323+0000 7ff7b7fff640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff7a40052a0 0x7ff7b815d020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:45986/0 (socket says 172.21.15.43:45986) 2023-08-09T13:35:55.328 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:55.324+0000 7ff7b7fff640 1 -- 172.21.15.43:0/1325301256 learned_addr learned my addr 172.21.15.43:0/1325301256 (peer_addr_for_me v2:172.21.15.43:0/0) 2023-08-09T13:35:55.329 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:55.324+0000 7ff7b7fff640 1 -- 172.21.15.43:0/1325301256 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7ff7b806b190 msgr2=0x7ff7b8099380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:35:55.329 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:55.324+0000 7ff7b7fff640 1 --2- 172.21.15.43:0/1325301256 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7ff7b806b190 0x7ff7b8099380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:35:55.329 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:55.324+0000 7ff7b7fff640 1 -- 172.21.15.43:0/1325301256 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7ff7b80998c0 msgr2=0x7ff7b80b1330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:35:55.329 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:55.324+0000 7ff7b7fff640 1 --2- 172.21.15.43:0/1325301256 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7ff7b80998c0 0x7ff7b80b1330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:35:55.329 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:55.324+0000 7ff7b7fff640 1 -- 172.21.15.43:0/1325301256 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff7b8098b80 con 0x7ff7a40052a0 2023-08-09T13:35:55.329 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:55.324+0000 7ff7b7fff640 1 --2- 172.21.15.43:0/1325301256 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff7a40052a0 0x7ff7b815d020 secure :-1 s=READY pgs=500 cs=0 l=1 rev1=1 crypto rx=0x7ff7a8001200 tx=0x7ff7a800dc00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-09T13:35:55.329 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:55.324+0000 7ff7b57fa640 1 -- 172.21.15.43:0/1325301256 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff7a80074e0 con 0x7ff7a40052a0 2023-08-09T13:35:55.330 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:55.324+0000 7ff7b57fa640 1 -- 172.21.15.43:0/1325301256 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff7a8007640 con 0x7ff7a40052a0 2023-08-09T13:35:55.330 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:55.324+0000 7ff7b57fa640 1 -- 172.21.15.43:0/1325301256 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff7a8004a00 con 0x7ff7a40052a0 2023-08-09T13:35:55.330 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:55.325+0000 7ff7bd0d7640 1 -- 172.21.15.43:0/1325301256 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff7b8098e60 con 0x7ff7a40052a0 2023-08-09T13:35:55.330 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:55.325+0000 7ff7bd0d7640 1 -- 172.21.15.43:0/1325301256 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff7b815d740 con 0x7ff7a40052a0 2023-08-09T13:35:55.331 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:55.326+0000 7ff7bd0d7640 1 -- 172.21.15.43:0/1325301256 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff7b80990a0 con 0x7ff7a40052a0 2023-08-09T13:35:55.331 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:55.327+0000 7ff7b57fa640 1 -- 172.21.15.43:0/1325301256 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 5) v1 ==== 88341+0+0 (secure 0 0 0) 0x7ff7a8004b60 con 0x7ff7a40052a0 2023-08-09T13:35:55.331 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:55.328+0000 7ff7b57fa640 1 --2- 172.21.15.43:0/1325301256 >> [v2:172.21.15.203:6832/830097474,v1:172.21.15.203:6833/830097474] conn(0x7ff79806a080 0x7ff79806c540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:35:55.331 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:55.328+0000 7ff7b57fa640 1 -- 172.21.15.43:0/1325301256 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 6871+0+0 (secure 0 0 0) 0x7ff7a8088080 con 0x7ff7a40052a0 2023-08-09T13:35:55.331 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:55.329+0000 7ff7affff640 1 --2- 172.21.15.43:0/1325301256 >> [v2:172.21.15.203:6832/830097474,v1:172.21.15.203:6833/830097474] conn(0x7ff79806a080 0x7ff79806c540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:35:55.332 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:55.330+0000 7ff7affff640 1 --2- 172.21.15.43:0/1325301256 >> [v2:172.21.15.203:6832/830097474,v1:172.21.15.203:6833/830097474] conn(0x7ff79806a080 0x7ff79806c540 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7ff7b8157420 tx=0x7ff7a0009040 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-09T13:35:55.334 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:55.332+0000 7ff7b57fa640 1 -- 172.21.15.43:0/1325301256 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178066 (secure 0 0 0) 0x7ff7a80ba7b0 con 0x7ff7a40052a0 2023-08-09T13:35:55.565 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:55.562+0000 7ff7bd0d7640 1 -- 172.21.15.43:0/1325301256 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7ff7b815d740 con 0x7ff7a40052a0 2023-08-09T13:35:55.565 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:55.563+0000 7ff7b57fa640 1 -- 172.21.15.43:0/1325301256 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 9 v9) v1 ==== 93+0+4874 (secure 0 0 0) 0x7ff7b815d740 con 0x7ff7a40052a0 2023-08-09T13:35:55.566 INFO:teuthology.orchestra.run.smithi043.stdout: 2023-08-09T13:35:55.566 INFO:teuthology.orchestra.run.smithi043.stdout:{"epoch":9,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4236,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.43:6833/3917539953","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6832","nonce":3917539953},{"type":"v1","addr":"172.21.15.43:6833","nonce":3917539953}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2},{"gid":4247,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.203:6837/2467794159","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6835","nonce":2467794159},{"type":"v1","addr":"172.21.15.203:6837","nonce":2467794159}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":4248,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.43:6835/1818380189","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6834","nonce":1818380189},{"type":"v1","addr":"172.21.15.43:6835","nonce":1818380189}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":9,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-08-09T13:34:02.575398+0000","modified":"2023-08-09T13:34:31.841945+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":4250},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4250":{"gid":4250,"name":"d","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.203:6836/798312463","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6834","nonce":798312463},{"type":"v1","addr":"172.21.15.203:6836","nonce":798312463}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[4,5],"metadata_pool":3,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-08-09T13:35:55.566 INFO:teuthology.orchestra.run.smithi043.stderr:dumped fsmap epoch 9 2023-08-09T13:35:55.568 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:55.566+0000 7ff7bd0d7640 1 -- 172.21.15.43:0/1325301256 >> [v2:172.21.15.203:6832/830097474,v1:172.21.15.203:6833/830097474] conn(0x7ff79806a080 msgr2=0x7ff79806c540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:35:55.569 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:55.566+0000 7ff7bd0d7640 1 --2- 172.21.15.43:0/1325301256 >> [v2:172.21.15.203:6832/830097474,v1:172.21.15.203:6833/830097474] conn(0x7ff79806a080 0x7ff79806c540 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7ff7b8157420 tx=0x7ff7a0009040 comp rx=0 tx=0).stop 2023-08-09T13:35:55.569 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:55.566+0000 7ff7bd0d7640 1 -- 172.21.15.43:0/1325301256 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff7a40052a0 msgr2=0x7ff7b815d020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:35:55.569 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:55.566+0000 7ff7bd0d7640 1 --2- 172.21.15.43:0/1325301256 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff7a40052a0 0x7ff7b815d020 secure :-1 s=READY pgs=500 cs=0 l=1 rev1=1 crypto rx=0x7ff7a8001200 tx=0x7ff7a800dc00 comp rx=0 tx=0).stop 2023-08-09T13:35:55.569 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:55.566+0000 7ff7bd0d7640 1 -- 172.21.15.43:0/1325301256 shutdown_connections 2023-08-09T13:35:55.569 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:55.566+0000 7ff7bd0d7640 1 --2- 172.21.15.43:0/1325301256 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7ff7b80998c0 0x7ff7b80b1330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:35:55.569 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:55.566+0000 7ff7bd0d7640 1 --2- 172.21.15.43:0/1325301256 >> [v2:172.21.15.203:6832/830097474,v1:172.21.15.203:6833/830097474] conn(0x7ff79806a080 0x7ff79806c540 unknown :-1 s=CLOSED pgs=244 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:35:55.569 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:55.566+0000 7ff7bd0d7640 1 --2- 172.21.15.43:0/1325301256 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7ff7b806b190 0x7ff7b8099380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:35:55.570 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:55.566+0000 7ff7bd0d7640 1 --2- 172.21.15.43:0/1325301256 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff7a40052a0 0x7ff7b815d020 unknown :-1 s=CLOSED pgs=500 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:35:55.570 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:55.566+0000 7ff7bd0d7640 1 -- 172.21.15.43:0/1325301256 >> 172.21.15.43:0/1325301256 conn(0x7ff7b80cde90 msgr2=0x7ff7b80c9aa0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-09T13:35:55.570 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:55.566+0000 7ff7bd0d7640 1 -- 172.21.15.43:0/1325301256 shutdown_connections 2023-08-09T13:35:55.570 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:55.566+0000 7ff7bd0d7640 1 -- 172.21.15.43:0/1325301256 wait complete. 2023-08-09T13:35:55.586 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-08-09T13:35:55.806 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:55.803+0000 7fb45bfff640 1 Processor -- start 2023-08-09T13:35:55.806 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:55.804+0000 7fb45bfff640 1 -- start start 2023-08-09T13:35:55.808 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:55.805+0000 7fb45bfff640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fb45c0713b0 0x7fb45c0c8fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:35:55.808 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:55.805+0000 7fb45bfff640 1 -- --> v1:172.21.15.43:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb45c0cdc70 con 0x7fb45c0c94f0 2023-08-09T13:35:55.808 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:55.805+0000 7fb45bfff640 1 -- --> v1:172.21.15.203:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb45c070910 con 0x7fb45c15a6a0 2023-08-09T13:35:55.808 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:55.805+0000 7fb45bfff640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb45c0c9960 con 0x7fb45c0713b0 2023-08-09T13:35:55.809 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:55.806+0000 7fb45b7fe640 1 --1- >> v1:172.21.15.43:6789/0 conn(0x7fb45c0c94f0 0x7fb45c15ad20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.43:6789/0 says I am v1:172.21.15.43:39122/0 (socket says 172.21.15.43:39122) 2023-08-09T13:35:55.810 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:55.806+0000 7fb45b7fe640 1 -- 172.21.15.43:0/1791096220 learned_addr learned my addr 172.21.15.43:0/1791096220 (peer_addr_for_me v1:172.21.15.43:0/0) 2023-08-09T13:35:55.810 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:55.806+0000 7fb453fff640 1 --2- 172.21.15.43:0/1791096220 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fb45c0713b0 0x7fb45c0c8fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:35:55.810 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:55.806+0000 7fb45a7fc640 1 -- 172.21.15.43:0/1791096220 <== mon.0 v1:172.21.15.43:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 368235708 0 0) 0x7fb45c0cdc70 con 0x7fb45c0c94f0 2023-08-09T13:35:55.810 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:55.806+0000 7fb45a7fc640 1 -- 172.21.15.43:0/1791096220 --> v1:172.21.15.43:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fb448003520 con 0x7fb45c0c94f0 2023-08-09T13:35:55.810 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:55.807+0000 7fb45a7fc640 1 -- 172.21.15.43:0/1791096220 <== mon.0 v1:172.21.15.43:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 515965389 0 0) 0x7fb448003520 con 0x7fb45c0c94f0 2023-08-09T13:35:55.810 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:55.807+0000 7fb45a7fc640 1 -- 172.21.15.43:0/1791096220 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fb45c0713b0 msgr2=0x7fb45c0c8fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-09T13:35:55.811 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:55.807+0000 7fb45a7fc640 1 --2- 172.21.15.43:0/1791096220 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fb45c0713b0 0x7fb45c0c8fb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:35:55.811 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:55.807+0000 7fb45a7fc640 1 -- 172.21.15.43:0/1791096220 >> v1:172.21.15.203:6789/0 conn(0x7fb45c15a6a0 legacy=0x7fb45c0cbd90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:35:55.811 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:55.807+0000 7fb45a7fc640 1 -- 172.21.15.43:0/1791096220 --> v1:172.21.15.43:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb45c0c88c0 con 0x7fb45c0c94f0 2023-08-09T13:35:55.811 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:55.807+0000 7fb45a7fc640 1 -- 172.21.15.43:0/1791096220 <== mon.0 v1:172.21.15.43:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1285150740 0 0) 0x7fb4540019f0 con 0x7fb45c0c94f0 2023-08-09T13:35:55.811 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:55.807+0000 7fb45a7fc640 1 -- 172.21.15.43:0/1791096220 >> v1:172.21.15.43:6789/0 conn(0x7fb45c0c94f0 legacy=0x7fb45c15ad20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:35:55.811 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:55.807+0000 7fb45a7fc640 1 --2- 172.21.15.43:0/1791096220 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fb4480039a0 0x7fb448003d80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:35:55.811 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:55.807+0000 7fb45a7fc640 1 --2- 172.21.15.43:0/1791096220 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fb45c0713b0 0x7fb4480047c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:35:55.812 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:55.807+0000 7fb45a7fc640 1 --2- 172.21.15.43:0/1791096220 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb448004d00 0x7fb448005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:35:55.812 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:55.807+0000 7fb45a7fc640 1 -- 172.21.15.43:0/1791096220 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb45c0c9960 con 0x7fb448004d00 2023-08-09T13:35:55.812 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:55.807+0000 7fb45a7fc640 1 -- 172.21.15.43:0/1791096220 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb44800dc70 con 0x7fb4480039a0 2023-08-09T13:35:55.812 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:55.807+0000 7fb45a7fc640 1 -- 172.21.15.43:0/1791096220 --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb44800de30 con 0x7fb45c0713b0 2023-08-09T13:35:55.812 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:55.807+0000 7fb45b7fe640 1 --2- 172.21.15.43:0/1791096220 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb448004d00 0x7fb448005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:35:55.813 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:55.808+0000 7fb453fff640 1 --2- 172.21.15.43:0/1791096220 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fb45c0713b0 0x7fb4480047c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:35:55.813 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:55.808+0000 7fb45affd640 1 --2- 172.21.15.43:0/1791096220 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fb4480039a0 0x7fb448003d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:35:55.814 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:55.808+0000 7fb45b7fe640 1 -- 172.21.15.43:0/1791096220 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fb45c0713b0 msgr2=0x7fb4480047c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-09T13:35:55.814 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:55.808+0000 7fb45b7fe640 1 --2- 172.21.15.43:0/1791096220 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fb45c0713b0 0x7fb4480047c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:35:55.814 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:55.808+0000 7fb45b7fe640 1 -- 172.21.15.43:0/1791096220 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fb4480039a0 msgr2=0x7fb448003d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-09T13:35:55.814 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:55.808+0000 7fb45b7fe640 1 --2- 172.21.15.43:0/1791096220 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fb4480039a0 0x7fb448003d80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:35:55.814 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:55.808+0000 7fb45b7fe640 1 -- 172.21.15.43:0/1791096220 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb448003520 con 0x7fb448004d00 2023-08-09T13:35:55.814 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:55.809+0000 7fb45b7fe640 1 --2- 172.21.15.43:0/1791096220 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb448004d00 0x7fb448005180 secure :-1 s=READY pgs=502 cs=0 l=1 rev1=1 crypto rx=0x7fb45400a830 tx=0x7fb4480057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=80582ccdb86c6b18 server_cookie=0 in_seq=0 out_seq=0 2023-08-09T13:35:55.814 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:55.809+0000 7fb45a7fc640 1 -- 172.21.15.43:0/1791096220 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb45402cea0 con 0x7fb448004d00 2023-08-09T13:35:55.814 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:55.809+0000 7fb45a7fc640 1 -- 172.21.15.43:0/1791096220 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb45402f070 con 0x7fb448004d00 2023-08-09T13:35:55.815 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:55.809+0000 7fb45a7fc640 1 -- 172.21.15.43:0/1791096220 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb454034890 con 0x7fb448004d00 2023-08-09T13:35:55.815 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:55.810+0000 7fb45bfff640 1 -- 172.21.15.43:0/1791096220 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb448004d00 msgr2=0x7fb448005180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:35:55.815 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:55.810+0000 7fb45bfff640 1 --2- 172.21.15.43:0/1791096220 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb448004d00 0x7fb448005180 secure :-1 s=READY pgs=502 cs=0 l=1 rev1=1 crypto rx=0x7fb45400a830 tx=0x7fb4480057f0 comp rx=0 tx=0).stop 2023-08-09T13:35:55.815 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:55.810+0000 7fb45b7fe640 1 -- 172.21.15.43:0/1791096220 reap_dead start 2023-08-09T13:35:55.815 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:55.810+0000 7fb45bfff640 1 -- 172.21.15.43:0/1791096220 shutdown_connections 2023-08-09T13:35:55.815 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:55.810+0000 7fb45bfff640 1 -- 172.21.15.43:0/1791096220 >> 172.21.15.43:0/1791096220 conn(0x7fb45c0cde90 msgr2=0x7fb45c0ce2a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-09T13:35:55.815 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:55.810+0000 7fb45bfff640 1 -- 172.21.15.43:0/1791096220 shutdown_connections 2023-08-09T13:35:55.816 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:55.810+0000 7fb45bfff640 1 -- 172.21.15.43:0/1791096220 wait complete. 2023-08-09T13:35:55.817 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:55.812+0000 7fb45bfff640 1 Processor -- start 2023-08-09T13:35:55.817 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:55.812+0000 7fb45bfff640 1 -- start start 2023-08-09T13:35:55.817 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:55.813+0000 7fb45bfff640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fb448004d00 0x7fb45c098a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:35:55.817 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:55.813+0000 7fb45bfff640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fb45c15bdd0 0x7fb45c15c250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:35:55.817 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:55.813+0000 7fb45bfff640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb45c154180 0x7fb45c154600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:35:55.817 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:55.813+0000 7fb45bfff640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb45c15c790 con 0x7fb45c154180 2023-08-09T13:35:55.818 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:55.813+0000 7fb45bfff640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb45c1f54f0 con 0x7fb45c15bdd0 2023-08-09T13:35:55.818 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:55.813+0000 7fb45bfff640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb45c1f57d0 con 0x7fb448004d00 2023-08-09T13:35:55.818 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:55.814+0000 7fb45b7fe640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb45c154180 0x7fb45c154600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:35:55.818 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:55.814+0000 7fb45b7fe640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb45c154180 0x7fb45c154600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:46014/0 (socket says 172.21.15.43:46014) 2023-08-09T13:35:55.818 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:55.814+0000 7fb45b7fe640 1 -- 172.21.15.43:0/227995893 learned_addr learned my addr 172.21.15.43:0/227995893 (peer_addr_for_me v2:172.21.15.43:0/0) 2023-08-09T13:35:55.818 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:55.814+0000 7fb45affd640 1 --2- 172.21.15.43:0/227995893 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fb448004d00 0x7fb45c098a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:35:55.818 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:55.814+0000 7fb453fff640 1 --2- 172.21.15.43:0/227995893 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fb45c15bdd0 0x7fb45c15c250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:35:55.818 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:55.814+0000 7fb45b7fe640 1 -- 172.21.15.43:0/227995893 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fb448004d00 msgr2=0x7fb45c098a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:35:55.819 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:55.814+0000 7fb45b7fe640 1 --2- 172.21.15.43:0/227995893 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fb448004d00 0x7fb45c098a50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:35:55.819 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:55.814+0000 7fb45b7fe640 1 -- 172.21.15.43:0/227995893 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fb45c15bdd0 msgr2=0x7fb45c15c250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:35:55.819 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:55.814+0000 7fb45b7fe640 1 --2- 172.21.15.43:0/227995893 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fb45c15bdd0 0x7fb45c15c250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:35:55.819 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:55.814+0000 7fb45b7fe640 1 -- 172.21.15.43:0/227995893 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb45c1f5ab0 con 0x7fb45c154180 2023-08-09T13:35:55.819 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:55.814+0000 7fb45affd640 1 --2- 172.21.15.43:0/227995893 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fb448004d00 0x7fb45c098a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-08-09T13:35:55.819 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:55.815+0000 7fb45b7fe640 1 --2- 172.21.15.43:0/227995893 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb45c154180 0x7fb45c154600 secure :-1 s=READY pgs=503 cs=0 l=1 rev1=1 crypto rx=0x7fb454001580 tx=0x7fb454001d80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-09T13:35:55.820 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:55.815+0000 7fb4537fe640 1 -- 172.21.15.43:0/227995893 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb454034840 con 0x7fb45c154180 2023-08-09T13:35:55.820 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:55.815+0000 7fb4537fe640 1 -- 172.21.15.43:0/227995893 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb454001ae0 con 0x7fb45c154180 2023-08-09T13:35:55.820 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:55.815+0000 7fb4537fe640 1 -- 172.21.15.43:0/227995893 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb454034ba0 con 0x7fb45c154180 2023-08-09T13:35:55.821 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:55.815+0000 7fb45bfff640 1 -- 172.21.15.43:0/227995893 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb45c1f5d90 con 0x7fb45c154180 2023-08-09T13:35:55.821 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:55.815+0000 7fb45bfff640 1 -- 172.21.15.43:0/227995893 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb45c0990e0 con 0x7fb45c154180 2023-08-09T13:35:55.821 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:55.816+0000 7fb45bfff640 1 -- 172.21.15.43:0/227995893 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb45c0739f0 con 0x7fb45c154180 2023-08-09T13:35:55.821 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:55.817+0000 7fb4537fe640 1 -- 172.21.15.43:0/227995893 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 5) v1 ==== 88341+0+0 (secure 0 0 0) 0x7fb454031030 con 0x7fb45c154180 2023-08-09T13:35:55.821 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:55.818+0000 7fb4537fe640 1 --2- 172.21.15.43:0/227995893 >> [v2:172.21.15.203:6832/830097474,v1:172.21.15.203:6833/830097474] conn(0x7fb43c06a080 0x7fb43c06c540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:35:55.822 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:55.818+0000 7fb4537fe640 1 -- 172.21.15.43:0/227995893 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 6871+0+0 (secure 0 0 0) 0x7fb4540a9da0 con 0x7fb45c154180 2023-08-09T13:35:55.822 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:55.819+0000 7fb45affd640 1 --2- 172.21.15.43:0/227995893 >> [v2:172.21.15.203:6832/830097474,v1:172.21.15.203:6833/830097474] conn(0x7fb43c06a080 0x7fb43c06c540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:35:55.822 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:55.820+0000 7fb45affd640 1 --2- 172.21.15.43:0/227995893 >> [v2:172.21.15.203:6832/830097474,v1:172.21.15.203:6833/830097474] conn(0x7fb43c06a080 0x7fb43c06c540 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7fb44c001760 tx=0x7fb44c009040 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-09T13:35:55.824 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:55.822+0000 7fb4537fe640 1 -- 172.21.15.43:0/227995893 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178066 (secure 0 0 0) 0x7fb4540751e0 con 0x7fb45c154180 2023-08-09T13:35:56.020 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:56.017+0000 7fb45bfff640 1 -- 172.21.15.43:0/227995893 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fb45c0739f0 con 0x7fb45c154180 2023-08-09T13:35:56.022 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:56.020+0000 7fb4537fe640 1 -- 172.21.15.43:0/227995893 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v26) v1 ==== 74+0+20146 (secure 0 0 0) 0x7fb454045090 con 0x7fb45c154180 2023-08-09T13:35:56.023 INFO:teuthology.orchestra.run.smithi043.stdout: 2023-08-09T13:35:56.023 INFO:teuthology.orchestra.run.smithi043.stdout:{"epoch":26,"fsid":"daac87ee-1337-4250-a955-aea4e408a39b","created":"2023-08-09T13:33:41.701198+0000","modified":"2023-08-09T13:34:48.711911+0000","last_up_change":"2023-08-09T13:33:53.783453+0000","last_in_change":"2023-08-09T13:33:45.148904+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":5,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":"rbd","create_time":"2023-08-09T13:33:54.365137+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":".mgr","create_time":"2023-08-09T13:33:54.831803+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_metadata","create_time":"2023-08-09T13:34:00.207974+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data","create_time":"2023-08-09T13:34:01.614765+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":5,"pool_name":"cephfs_data_ec","create_time":"2023-08-09T13:34:05.239583+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"24","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"cbd49ecd-e896-4fbd-864e-5c37a829b165","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6800","nonce":502667335},{"type":"v1","addr":"172.21.15.43:6801","nonce":502667335}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6802","nonce":502667335},{"type":"v1","addr":"172.21.15.43:6803","nonce":502667335}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6806","nonce":502667335},{"type":"v1","addr":"172.21.15.43:6807","nonce":502667335}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6804","nonce":502667335},{"type":"v1","addr":"172.21.15.43:6805","nonce":502667335}]},"public_addr":"172.21.15.43:6801/502667335","cluster_addr":"172.21.15.43:6803/502667335","heartbeat_back_addr":"172.21.15.43:6807/502667335","heartbeat_front_addr":"172.21.15.43:6805/502667335","state":["exists","up"]},{"osd":1,"uuid":"75c7e0fe-1ec7-4fe8-a2f2-3661a65b8500","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6808","nonce":3289134610},{"type":"v1","addr":"172.21.15.43:6809","nonce":3289134610}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6810","nonce":3289134610},{"type":"v1","addr":"172.21.15.43:6812","nonce":3289134610}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6821","nonce":3289134610},{"type":"v1","addr":"172.21.15.43:6824","nonce":3289134610}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6815","nonce":3289134610},{"type":"v1","addr":"172.21.15.43:6818","nonce":3289134610}]},"public_addr":"172.21.15.43:6809/3289134610","cluster_addr":"172.21.15.43:6812/3289134610","heartbeat_back_addr":"172.21.15.43:6824/3289134610","heartbeat_front_addr":"172.21.15.43:6818/3289134610","state":["exists","up"]},{"osd":2,"uuid":"bb5c41e9-7f5f-4125-9b01-3bb8bfc2ea83","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6811","nonce":4079709736},{"type":"v1","addr":"172.21.15.43:6814","nonce":4079709736}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6817","nonce":4079709736},{"type":"v1","addr":"172.21.15.43:6820","nonce":4079709736}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6829","nonce":4079709736},{"type":"v1","addr":"172.21.15.43:6831","nonce":4079709736}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6823","nonce":4079709736},{"type":"v1","addr":"172.21.15.43:6826","nonce":4079709736}]},"public_addr":"172.21.15.43:6814/4079709736","cluster_addr":"172.21.15.43:6820/4079709736","heartbeat_back_addr":"172.21.15.43:6831/4079709736","heartbeat_front_addr":"172.21.15.43:6826/4079709736","state":["exists","up"]},{"osd":3,"uuid":"bfe4c8b4-694a-465c-87e6-2b2aab852613","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6813","nonce":1709538759},{"type":"v1","addr":"172.21.15.43:6816","nonce":1709538759}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6819","nonce":1709538759},{"type":"v1","addr":"172.21.15.43:6822","nonce":1709538759}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6828","nonce":1709538759},{"type":"v1","addr":"172.21.15.43:6830","nonce":1709538759}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6825","nonce":1709538759},{"type":"v1","addr":"172.21.15.43:6827","nonce":1709538759}]},"public_addr":"172.21.15.43:6816/1709538759","cluster_addr":"172.21.15.43:6822/1709538759","heartbeat_back_addr":"172.21.15.43:6830/1709538759","heartbeat_front_addr":"172.21.15.43:6827/1709538759","state":["exists","up"]},{"osd":4,"uuid":"fa2e8a77-aba1-4756-853e-0d174a5bdeba","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6800","nonce":4196866481},{"type":"v1","addr":"172.21.15.203:6801","nonce":4196866481}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6802","nonce":4196866481},{"type":"v1","addr":"172.21.15.203:6803","nonce":4196866481}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6806","nonce":4196866481},{"type":"v1","addr":"172.21.15.203:6807","nonce":4196866481}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6804","nonce":4196866481},{"type":"v1","addr":"172.21.15.203:6805","nonce":4196866481}]},"public_addr":"172.21.15.203:6801/4196866481","cluster_addr":"172.21.15.203:6803/4196866481","heartbeat_back_addr":"172.21.15.203:6807/4196866481","heartbeat_front_addr":"172.21.15.203:6805/4196866481","state":["exists","up"]},{"osd":5,"uuid":"81811e4b-4d24-4f51-a82c-f1d700a2a00d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6816","nonce":3059290081},{"type":"v1","addr":"172.21.15.203:6817","nonce":3059290081}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6818","nonce":3059290081},{"type":"v1","addr":"172.21.15.203:6819","nonce":3059290081}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6822","nonce":3059290081},{"type":"v1","addr":"172.21.15.203:6823","nonce":3059290081}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6820","nonce":3059290081},{"type":"v1","addr":"172.21.15.203:6821","nonce":3059290081}]},"public_addr":"172.21.15.203:6817/3059290081","cluster_addr":"172.21.15.203:6819/3059290081","heartbeat_back_addr":"172.21.15.203:6823/3059290081","heartbeat_front_addr":"172.21.15.203:6821/3059290081","state":["exists","up"]},{"osd":6,"uuid":"307bac4d-6a5b-443d-af75-56d695d7674f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6824","nonce":512155569},{"type":"v1","addr":"172.21.15.203:6825","nonce":512155569}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6826","nonce":512155569},{"type":"v1","addr":"172.21.15.203:6827","nonce":512155569}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6830","nonce":512155569},{"type":"v1","addr":"172.21.15.203:6831","nonce":512155569}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6828","nonce":512155569},{"type":"v1","addr":"172.21.15.203:6829","nonce":512155569}]},"public_addr":"172.21.15.203:6825/512155569","cluster_addr":"172.21.15.203:6827/512155569","heartbeat_back_addr":"172.21.15.203:6831/512155569","heartbeat_front_addr":"172.21.15.203:6829/512155569","state":["exists","up"]},{"osd":7,"uuid":"a95dc7ca-9f1b-4295-af70-68a22a2d7d5e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6808","nonce":3331450438},{"type":"v1","addr":"172.21.15.203:6809","nonce":3331450438}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6810","nonce":3331450438},{"type":"v1","addr":"172.21.15.203:6811","nonce":3331450438}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6814","nonce":3331450438},{"type":"v1","addr":"172.21.15.203:6815","nonce":3331450438}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6812","nonce":3331450438},{"type":"v1","addr":"172.21.15.203:6813","nonce":3331450438}]},"public_addr":"172.21.15.203:6809/3331450438","cluster_addr":"172.21.15.203:6811/3331450438","heartbeat_back_addr":"172.21.15.203:6815/3331450438","heartbeat_front_addr":"172.21.15.203:6813/3331450438","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-09T13:33:50.547652+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-09T13:33:50.589469+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-09T13:33:50.561724+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-09T13:33:50.599650+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-09T13:33:50.411855+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-09T13:33:50.219681+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-09T13:33:50.504664+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-09T13:33:50.441248+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"4.14","mappings":[{"from":1,"to":5}]},{"pgid":"4.29","mappings":[{"from":3,"to":2}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-08-09T13:35:56.025 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:56.023+0000 7fb45bfff640 1 -- 172.21.15.43:0/227995893 >> [v2:172.21.15.203:6832/830097474,v1:172.21.15.203:6833/830097474] conn(0x7fb43c06a080 msgr2=0x7fb43c06c540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:35:56.025 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:56.023+0000 7fb45bfff640 1 --2- 172.21.15.43:0/227995893 >> [v2:172.21.15.203:6832/830097474,v1:172.21.15.203:6833/830097474] conn(0x7fb43c06a080 0x7fb43c06c540 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7fb44c001760 tx=0x7fb44c009040 comp rx=0 tx=0).stop 2023-08-09T13:35:56.026 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:56.023+0000 7fb45bfff640 1 -- 172.21.15.43:0/227995893 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb45c154180 msgr2=0x7fb45c154600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:35:56.026 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:56.023+0000 7fb45bfff640 1 --2- 172.21.15.43:0/227995893 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb45c154180 0x7fb45c154600 secure :-1 s=READY pgs=503 cs=0 l=1 rev1=1 crypto rx=0x7fb454001580 tx=0x7fb454001d80 comp rx=0 tx=0).stop 2023-08-09T13:35:56.026 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:56.023+0000 7fb45bfff640 1 -- 172.21.15.43:0/227995893 shutdown_connections 2023-08-09T13:35:56.026 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:56.023+0000 7fb45bfff640 1 --2- 172.21.15.43:0/227995893 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb45c154180 0x7fb45c154600 unknown :-1 s=CLOSED pgs=503 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:35:56.026 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:56.023+0000 7fb45bfff640 1 --2- 172.21.15.43:0/227995893 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fb45c15bdd0 0x7fb45c15c250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:35:56.026 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:56.023+0000 7fb45bfff640 1 --2- 172.21.15.43:0/227995893 >> [v2:172.21.15.203:6832/830097474,v1:172.21.15.203:6833/830097474] conn(0x7fb43c06a080 0x7fb43c06c540 unknown :-1 s=CLOSED pgs=245 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:35:56.026 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:56.023+0000 7fb45bfff640 1 --2- 172.21.15.43:0/227995893 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fb448004d00 0x7fb45c098a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:35:56.027 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:56.023+0000 7fb45bfff640 1 -- 172.21.15.43:0/227995893 >> 172.21.15.43:0/227995893 conn(0x7fb45c0cde90 msgr2=0x7fb45c0c9aa0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-09T13:35:56.027 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:56.023+0000 7fb45bfff640 1 -- 172.21.15.43:0/227995893 shutdown_connections 2023-08-09T13:35:56.027 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:56.023+0000 7fb45bfff640 1 -- 172.21.15.43:0/227995893 wait complete. 2023-08-09T13:35:56.044 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2023-08-09T13:35:56.268 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:56.265+0000 7f6d71234640 1 Processor -- start 2023-08-09T13:35:56.268 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:56.265+0000 7f6d71234640 1 -- start start 2023-08-09T13:35:56.270 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:56.266+0000 7f6d71234640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f6d6c15c630 0x7f6d6c0abef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:35:56.271 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:56.266+0000 7f6d71234640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f6d6c0ccf20 0x7f6d6c0cd550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:35:56.271 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:56.266+0000 7f6d71234640 1 -- --> v1:172.21.15.43:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6d6c0cca90 con 0x7f6d6c0cda90 2023-08-09T13:35:56.271 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:56.266+0000 7f6d71234640 1 -- --> v2:172.21.15.43:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6d6c0cbc70 con 0x7f6d6c0ccf20 2023-08-09T13:35:56.272 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:56.266+0000 7f6d71234640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6d6c0cdf00 con 0x7f6d6c15c630 2023-08-09T13:35:56.272 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:56.267+0000 7f6d6b59e640 1 --1- >> v1:172.21.15.43:6789/0 conn(0x7f6d6c0cda90 0x7f6d6c0c2f20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.43:6789/0 says I am v1:172.21.15.43:39124/0 (socket says 172.21.15.43:39124) 2023-08-09T13:35:56.272 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:56.267+0000 7f6d6b59e640 1 -- 172.21.15.43:0/2113884692 learned_addr learned my addr 172.21.15.43:0/2113884692 (peer_addr_for_me v1:172.21.15.43:0/0) 2023-08-09T13:35:56.273 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:56.267+0000 7f6d6259c640 1 --2- 172.21.15.43:0/2113884692 >> v2:172.21.15.43:3300/0 conn(0x7f6d6c0ccf20 0x7f6d6c0cd550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:35:56.273 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:56.267+0000 7f6d6ad9d640 1 --2- 172.21.15.43:0/2113884692 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f6d6c15c630 0x7f6d6c0abef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:35:56.273 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:56.267+0000 7f6d6a59c640 1 -- 172.21.15.43:0/2113884692 <== mon.0 v1:172.21.15.43:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2678625713 0 0) 0x7f6d6c0cca90 con 0x7f6d6c0cda90 2023-08-09T13:35:56.273 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:56.267+0000 7f6d6a59c640 1 -- 172.21.15.43:0/2113884692 --> v1:172.21.15.43:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f6d58003520 con 0x7f6d6c0cda90 2023-08-09T13:35:56.274 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:56.267+0000 7f6d6259c640 1 -- 172.21.15.43:0/2113884692 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f6d6c15c630 msgr2=0x7f6d6c0abef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-09T13:35:56.274 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:56.267+0000 7f6d6259c640 1 --2- 172.21.15.43:0/2113884692 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f6d6c15c630 0x7f6d6c0abef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:35:56.274 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:56.267+0000 7f6d6259c640 1 -- 172.21.15.43:0/2113884692 >> v1:172.21.15.43:6789/0 conn(0x7f6d6c0cda90 legacy=0x7f6d6c0c2f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:35:56.274 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:56.268+0000 7f6d6259c640 1 -- 172.21.15.43:0/2113884692 --> v2:172.21.15.43:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6d6c15da20 con 0x7f6d6c0ccf20 2023-08-09T13:35:56.278 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:56.269+0000 7f6d6259c640 1 --2- 172.21.15.43:0/2113884692 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f6d6c0ccf20 0x7f6d6c0cd550 secure :-1 s=READY pgs=505 cs=0 l=1 rev1=1 crypto rx=0x7f6d540097a0 tx=0x7f6d5402b8b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=44403da49a1e4ba8 server_cookie=0 in_seq=0 out_seq=0 2023-08-09T13:35:56.278 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:56.269+0000 7f6d6a59c640 1 -- 172.21.15.43:0/2113884692 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6d5402d040 con 0x7f6d6c0ccf20 2023-08-09T13:35:56.279 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:56.269+0000 7f6d6a59c640 1 -- 172.21.15.43:0/2113884692 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6d54031810 con 0x7f6d6c0ccf20 2023-08-09T13:35:56.279 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:56.269+0000 7f6d6a59c640 1 -- 172.21.15.43:0/2113884692 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6d54031b80 con 0x7f6d6c0ccf20 2023-08-09T13:35:56.279 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:56.269+0000 7f6d71234640 1 -- 172.21.15.43:0/2113884692 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f6d6c0ccf20 msgr2=0x7f6d6c0cd550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:35:56.279 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:56.269+0000 7f6d71234640 1 --2- 172.21.15.43:0/2113884692 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f6d6c0ccf20 0x7f6d6c0cd550 secure :-1 s=READY pgs=505 cs=0 l=1 rev1=1 crypto rx=0x7f6d540097a0 tx=0x7f6d5402b8b0 comp rx=0 tx=0).stop 2023-08-09T13:35:56.280 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:56.270+0000 7f6d71234640 1 -- 172.21.15.43:0/2113884692 shutdown_connections 2023-08-09T13:35:56.280 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:56.270+0000 7f6d71234640 1 --2- 172.21.15.43:0/2113884692 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f6d6c0ccf20 0x7f6d6c0cd550 unknown :-1 s=CLOSED pgs=505 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:35:56.280 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:56.270+0000 7f6d71234640 1 --2- 172.21.15.43:0/2113884692 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f6d6c15c630 0x7f6d6c0abef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:35:56.281 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:56.270+0000 7f6d71234640 1 -- 172.21.15.43:0/2113884692 >> 172.21.15.43:0/2113884692 conn(0x7f6d6c0ce0a0 msgr2=0x7f6d6c0ce480 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-09T13:35:56.281 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:56.270+0000 7f6d71234640 1 -- 172.21.15.43:0/2113884692 shutdown_connections 2023-08-09T13:35:56.281 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:56.270+0000 7f6d71234640 1 -- 172.21.15.43:0/2113884692 wait complete. 2023-08-09T13:35:56.281 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:56.271+0000 7f6d71234640 1 Processor -- start 2023-08-09T13:35:56.282 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:56.272+0000 7f6d71234640 1 -- start start 2023-08-09T13:35:56.282 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:56.272+0000 7f6d71234640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f6d6c0ccf20 0x7f6d6c151f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:35:56.282 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:56.273+0000 7f6d71234640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f6d6c0cda90 0x7f6d6c15ebb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:35:56.282 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:56.273+0000 7f6d71234640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f6d6c15c630 0x7f6d6c0c6810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:35:56.283 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:56.273+0000 7f6d71234640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6d6c0af7c0 con 0x7f6d6c0cda90 2023-08-09T13:35:56.283 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:56.273+0000 7f6d71234640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6d6c0aebb0 con 0x7f6d6c15c630 2023-08-09T13:35:56.283 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:56.273+0000 7f6d71234640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6d6c0aee90 con 0x7f6d6c0ccf20 2023-08-09T13:35:56.283 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:56.273+0000 7f6d6259c640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f6d6c0cda90 0x7f6d6c15ebb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:35:56.284 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:56.273+0000 7f6d6259c640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f6d6c0cda90 0x7f6d6c15ebb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:46022/0 (socket says 172.21.15.43:46022) 2023-08-09T13:35:56.284 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:56.273+0000 7f6d6259c640 1 -- 172.21.15.43:0/1663332424 learned_addr learned my addr 172.21.15.43:0/1663332424 (peer_addr_for_me v2:172.21.15.43:0/0) 2023-08-09T13:35:56.284 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:56.273+0000 7f6d6ad9d640 1 --2- 172.21.15.43:0/1663332424 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f6d6c0ccf20 0x7f6d6c151f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:35:56.284 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:56.273+0000 7f6d6b59e640 1 --2- 172.21.15.43:0/1663332424 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f6d6c15c630 0x7f6d6c0c6810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:35:56.285 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:56.273+0000 7f6d6259c640 1 -- 172.21.15.43:0/1663332424 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f6d6c0ccf20 msgr2=0x7f6d6c151f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:35:56.285 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:56.273+0000 7f6d6259c640 1 --2- 172.21.15.43:0/1663332424 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f6d6c0ccf20 0x7f6d6c151f90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:35:56.285 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:56.273+0000 7f6d6259c640 1 -- 172.21.15.43:0/1663332424 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f6d6c15c630 msgr2=0x7f6d6c0c6810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:35:56.285 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:56.273+0000 7f6d6259c640 1 --2- 172.21.15.43:0/1663332424 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f6d6c15c630 0x7f6d6c0c6810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:35:56.286 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:56.273+0000 7f6d6259c640 1 -- 172.21.15.43:0/1663332424 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6d54009400 con 0x7f6d6c0cda90 2023-08-09T13:35:56.286 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:56.274+0000 7f6d6259c640 1 --2- 172.21.15.43:0/1663332424 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f6d6c0cda90 0x7f6d6c15ebb0 secure :-1 s=READY pgs=506 cs=0 l=1 rev1=1 crypto rx=0x7f6d54000c00 tx=0x7f6d540046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-09T13:35:56.286 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:56.274+0000 7f6d63fff640 1 -- 172.21.15.43:0/1663332424 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6d54006e10 con 0x7f6d6c0cda90 2023-08-09T13:35:56.286 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:56.274+0000 7f6d63fff640 1 -- 172.21.15.43:0/1663332424 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6d5403ad60 con 0x7f6d6c0cda90 2023-08-09T13:35:56.287 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:56.274+0000 7f6d63fff640 1 -- 172.21.15.43:0/1663332424 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6d54044620 con 0x7f6d6c0cda90 2023-08-09T13:35:56.287 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:56.274+0000 7f6d71234640 1 -- 172.21.15.43:0/1663332424 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6d6c0af170 con 0x7f6d6c0cda90 2023-08-09T13:35:56.287 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:56.274+0000 7f6d71234640 1 -- 172.21.15.43:0/1663332424 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6d6c1524d0 con 0x7f6d6c0cda90 2023-08-09T13:35:56.287 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:56.276+0000 7f6d71234640 1 -- 172.21.15.43:0/1663332424 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6d6c06cf50 con 0x7f6d6c0cda90 2023-08-09T13:35:56.289 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:56.277+0000 7f6d63fff640 1 -- 172.21.15.43:0/1663332424 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 5) v1 ==== 88341+0+0 (secure 0 0 0) 0x7f6d5404d440 con 0x7f6d6c0cda90 2023-08-09T13:35:56.289 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:56.277+0000 7f6d63fff640 1 --2- 172.21.15.43:0/1663332424 >> [v2:172.21.15.203:6832/830097474,v1:172.21.15.203:6833/830097474] conn(0x7f6d4c06a050 0x7f6d4c06c510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:35:56.289 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:56.278+0000 7f6d63fff640 1 -- 172.21.15.43:0/1663332424 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 6871+0+0 (secure 0 0 0) 0x7f6d540aac40 con 0x7f6d6c0cda90 2023-08-09T13:35:56.290 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:56.278+0000 7f6d6ad9d640 1 --2- 172.21.15.43:0/1663332424 >> [v2:172.21.15.203:6832/830097474,v1:172.21.15.203:6833/830097474] conn(0x7f6d4c06a050 0x7f6d4c06c510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:35:56.290 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:56.279+0000 7f6d6ad9d640 1 --2- 172.21.15.43:0/1663332424 >> [v2:172.21.15.203:6832/830097474,v1:172.21.15.203:6833/830097474] conn(0x7f6d4c06a050 0x7f6d4c06c510 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7f6d5c006fd0 tx=0x7f6d5c008040 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-09T13:35:56.290 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:56.282+0000 7f6d63fff640 1 -- 172.21.15.43:0/1663332424 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178066 (secure 0 0 0) 0x7f6d54076080 con 0x7f6d6c0cda90 2023-08-09T13:35:56.516 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:56.513+0000 7f6d71234640 1 -- 172.21.15.43:0/1663332424 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) v1 -- 0x7f6d6c0739f0 con 0x7f6d6c0cda90 2023-08-09T13:35:56.518 INFO:tasks.ceph.mon.a.smithi043.stderr:2023-08-09T13:35:56.515+0000 7f32867f3640 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2023-08-09T13:35:56.524 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:56.522+0000 7f6d63fff640 1 -- 172.21.15.43:0/1663332424 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v10) v1 ==== 161+0+0 (secure 0 0 0) 0x7f6d540796a0 con 0x7f6d6c0cda90 2023-08-09T13:35:56.525 INFO:teuthology.orchestra.run.smithi043.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2023-08-09T13:35:56.528 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:56.525+0000 7f6d71234640 1 -- 172.21.15.43:0/1663332424 >> [v2:172.21.15.203:6832/830097474,v1:172.21.15.203:6833/830097474] conn(0x7f6d4c06a050 msgr2=0x7f6d4c06c510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:35:56.528 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:56.525+0000 7f6d71234640 1 --2- 172.21.15.43:0/1663332424 >> [v2:172.21.15.203:6832/830097474,v1:172.21.15.203:6833/830097474] conn(0x7f6d4c06a050 0x7f6d4c06c510 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7f6d5c006fd0 tx=0x7f6d5c008040 comp rx=0 tx=0).stop 2023-08-09T13:35:56.528 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:56.526+0000 7f6d71234640 1 -- 172.21.15.43:0/1663332424 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f6d6c0cda90 msgr2=0x7f6d6c15ebb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:35:56.528 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:56.526+0000 7f6d71234640 1 --2- 172.21.15.43:0/1663332424 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f6d6c0cda90 0x7f6d6c15ebb0 secure :-1 s=READY pgs=506 cs=0 l=1 rev1=1 crypto rx=0x7f6d54000c00 tx=0x7f6d540046c0 comp rx=0 tx=0).stop 2023-08-09T13:35:56.529 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:56.526+0000 7f6d71234640 1 -- 172.21.15.43:0/1663332424 shutdown_connections 2023-08-09T13:35:56.529 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:56.526+0000 7f6d71234640 1 --2- 172.21.15.43:0/1663332424 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f6d6c15c630 0x7f6d6c0c6810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:35:56.529 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:56.526+0000 7f6d71234640 1 --2- 172.21.15.43:0/1663332424 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f6d6c0cda90 0x7f6d6c15ebb0 unknown :-1 s=CLOSED pgs=506 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:35:56.529 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:56.526+0000 7f6d71234640 1 --2- 172.21.15.43:0/1663332424 >> [v2:172.21.15.203:6832/830097474,v1:172.21.15.203:6833/830097474] conn(0x7f6d4c06a050 0x7f6d4c06c510 unknown :-1 s=CLOSED pgs=246 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:35:56.529 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:56.526+0000 7f6d71234640 1 --2- 172.21.15.43:0/1663332424 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f6d6c0ccf20 0x7f6d6c151f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:35:56.530 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:56.526+0000 7f6d71234640 1 -- 172.21.15.43:0/1663332424 >> 172.21.15.43:0/1663332424 conn(0x7f6d6c0ce0a0 msgr2=0x7f6d6c15bbe0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-09T13:35:56.530 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:56.527+0000 7f6d71234640 1 -- 172.21.15.43:0/1663332424 shutdown_connections 2023-08-09T13:35:56.530 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:56.527+0000 7f6d71234640 1 -- 172.21.15.43:0/1663332424 wait complete. 2023-08-09T13:35:56.548 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs rm cephfs --yes-i-really-mean-it 2023-08-09T13:35:56.559 INFO:tasks.ceph.mds.d.smithi203.stderr: -6122> 2023-08-09T13:34:07.117+0000 7ff9ca2d5640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2023-08-09T13:35:56.656 INFO:tasks.ceph.mds.d.smithi203.stdout:starting mds.d at 2023-08-09T13:35:56.778 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:56.776+0000 7f6d8081e640 1 Processor -- start 2023-08-09T13:35:56.778 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:56.776+0000 7f6d8081e640 1 -- start start 2023-08-09T13:35:56.780 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:56.777+0000 7f6d8081e640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f6d7c15c2d0 0x7f6d7c0b7df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:35:56.780 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:56.777+0000 7f6d8081e640 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7f6d7c0d1b30 0x7f6d7c0cd020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:35:56.780 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:56.777+0000 7f6d8081e640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f6d7c0cebf0 0x7f6d7c0cf040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:35:56.780 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:56.777+0000 7f6d8081e640 1 -- --> v2:172.21.15.43:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6d7c0cf580 con 0x7f6d7c0cebf0 2023-08-09T13:35:56.781 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:56.777+0000 7f6d8081e640 1 -- --> v2:172.21.15.203:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6d7c0d16a0 con 0x7f6d7c0d1b30 2023-08-09T13:35:56.781 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:56.777+0000 7f6d8081e640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6d7c0d1f70 con 0x7f6d7c15c2d0 2023-08-09T13:35:56.782 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:56.778+0000 7f6d7b59e640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f6d7c0cebf0 0x7f6d7c0cf040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:35:56.782 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:56.778+0000 7f6d7b59e640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f6d7c0cebf0 0x7f6d7c0cf040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:46026/0 (socket says 172.21.15.43:46026) 2023-08-09T13:35:56.782 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:56.778+0000 7f6d7b59e640 1 -- 172.21.15.43:0/3465885868 learned_addr learned my addr 172.21.15.43:0/3465885868 (peer_addr_for_me v2:172.21.15.43:0/0) 2023-08-09T13:35:56.782 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:56.778+0000 7f6d7ad9d640 1 --2- 172.21.15.43:0/3465885868 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f6d7c15c2d0 0x7f6d7c0b7df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:35:56.782 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:56.778+0000 7f6d7b59e640 1 -- 172.21.15.43:0/3465885868 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f6d7c15c2d0 msgr2=0x7f6d7c0b7df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-09T13:35:56.782 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:56.778+0000 7f6d7259c640 1 --2- 172.21.15.43:0/3465885868 >> v2:172.21.15.203:3300/0 conn(0x7f6d7c0d1b30 0x7f6d7c0cd020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:35:56.783 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:56.778+0000 7f6d7b59e640 1 --2- 172.21.15.43:0/3465885868 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f6d7c15c2d0 0x7f6d7c0b7df0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:35:56.783 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:56.778+0000 7f6d7b59e640 1 -- 172.21.15.43:0/3465885868 >> v2:172.21.15.203:3300/0 conn(0x7f6d7c0d1b30 msgr2=0x7f6d7c0cd020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-09T13:35:56.783 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:56.778+0000 7f6d7b59e640 1 --2- 172.21.15.43:0/3465885868 >> v2:172.21.15.203:3300/0 conn(0x7f6d7c0d1b30 0x7f6d7c0cd020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:35:56.783 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:56.778+0000 7f6d7b59e640 1 -- 172.21.15.43:0/3465885868 --> v2:172.21.15.43:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6d7c0cd560 con 0x7f6d7c0cebf0 2023-08-09T13:35:56.783 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:56.778+0000 7f6d7259c640 1 --2- 172.21.15.43:0/3465885868 >> v2:172.21.15.203:3300/0 conn(0x7f6d7c0d1b30 0x7f6d7c0cd020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-08-09T13:35:56.783 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:56.778+0000 7f6d7ad9d640 1 --2- 172.21.15.43:0/3465885868 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f6d7c15c2d0 0x7f6d7c0b7df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-08-09T13:35:56.783 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:56.780+0000 7f6d7b59e640 1 --2- 172.21.15.43:0/3465885868 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f6d7c0cebf0 0x7f6d7c0cf040 secure :-1 s=READY pgs=508 cs=0 l=1 rev1=1 crypto rx=0x7f6d7c0b4d70 tx=0x7f6d7402b920 comp rx=0 tx=0).ready entity=mon.0 client_cookie=18831b772cf471d7 server_cookie=0 in_seq=0 out_seq=0 2023-08-09T13:35:56.784 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:56.780+0000 7f6d7a59c640 1 -- 172.21.15.43:0/3465885868 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6d74031880 con 0x7f6d7c0cebf0 2023-08-09T13:35:56.784 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:56.780+0000 7f6d7a59c640 1 -- 172.21.15.43:0/3465885868 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6d740319e0 con 0x7f6d7c0cebf0 2023-08-09T13:35:56.785 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:56.780+0000 7f6d7a59c640 1 -- 172.21.15.43:0/3465885868 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6d74031d70 con 0x7f6d7c0cebf0 2023-08-09T13:35:56.785 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:56.780+0000 7f6d8081e640 1 -- 172.21.15.43:0/3465885868 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f6d7c0cebf0 msgr2=0x7f6d7c0cf040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:35:56.785 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:56.780+0000 7f6d8081e640 1 --2- 172.21.15.43:0/3465885868 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f6d7c0cebf0 0x7f6d7c0cf040 secure :-1 s=READY pgs=508 cs=0 l=1 rev1=1 crypto rx=0x7f6d7c0b4d70 tx=0x7f6d7402b920 comp rx=0 tx=0).stop 2023-08-09T13:35:56.785 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:56.781+0000 7f6d8081e640 1 -- 172.21.15.43:0/3465885868 shutdown_connections 2023-08-09T13:35:56.785 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:56.781+0000 7f6d8081e640 1 --2- 172.21.15.43:0/3465885868 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f6d7c0cebf0 0x7f6d7c0cf040 unknown :-1 s=CLOSED pgs=508 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:35:56.786 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:56.781+0000 7f6d8081e640 1 --2- 172.21.15.43:0/3465885868 >> v2:172.21.15.203:3300/0 conn(0x7f6d7c0d1b30 0x7f6d7c0cd020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:35:56.786 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:56.781+0000 7f6d8081e640 1 --2- 172.21.15.43:0/3465885868 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f6d7c15c2d0 0x7f6d7c0b7df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:35:56.786 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:56.781+0000 7f6d8081e640 1 -- 172.21.15.43:0/3465885868 >> 172.21.15.43:0/3465885868 conn(0x7f6d7c06a230 msgr2=0x7f6d7c06a640 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-09T13:35:56.786 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:56.781+0000 7f6d8081e640 1 -- 172.21.15.43:0/3465885868 shutdown_connections 2023-08-09T13:35:56.786 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:56.781+0000 7f6d8081e640 1 -- 172.21.15.43:0/3465885868 wait complete. 2023-08-09T13:35:56.788 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:56.782+0000 7f6d8081e640 1 Processor -- start 2023-08-09T13:35:56.788 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:56.783+0000 7f6d8081e640 1 -- start start 2023-08-09T13:35:56.788 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:56.784+0000 7f6d8081e640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f6d7c0cebf0 0x7f6d7c0b7290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:35:56.788 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:56.784+0000 7f6d8081e640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f6d7c0d1b30 0x7f6d7c15a990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:35:56.789 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:56.784+0000 7f6d8081e640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f6d7c15c2d0 0x7f6d7c0a9040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:35:56.789 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:56.784+0000 7f6d8081e640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6d7c15efe0 con 0x7f6d7c0d1b30 2023-08-09T13:35:56.789 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:56.784+0000 7f6d8081e640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6d7c15e110 con 0x7f6d7c15c2d0 2023-08-09T13:35:56.789 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:56.784+0000 7f6d8081e640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6d7c15aed0 con 0x7f6d7c0cebf0 2023-08-09T13:35:56.789 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:56.784+0000 7f6d7259c640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f6d7c0d1b30 0x7f6d7c15a990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:35:56.789 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:56.784+0000 7f6d7259c640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f6d7c0d1b30 0x7f6d7c15a990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:46030/0 (socket says 172.21.15.43:46030) 2023-08-09T13:35:56.789 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:56.784+0000 7f6d7259c640 1 -- 172.21.15.43:0/1914840838 learned_addr learned my addr 172.21.15.43:0/1914840838 (peer_addr_for_me v2:172.21.15.43:0/0) 2023-08-09T13:35:56.789 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:56.784+0000 7f6d7ad9d640 1 --2- 172.21.15.43:0/1914840838 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f6d7c0cebf0 0x7f6d7c0b7290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:35:56.790 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:56.784+0000 7f6d7259c640 1 -- 172.21.15.43:0/1914840838 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f6d7c0cebf0 msgr2=0x7f6d7c0b7290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:35:56.790 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:56.784+0000 7f6d7259c640 1 --2- 172.21.15.43:0/1914840838 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f6d7c0cebf0 0x7f6d7c0b7290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:35:56.790 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:56.784+0000 7f6d7259c640 1 -- 172.21.15.43:0/1914840838 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f6d7c15c2d0 msgr2=0x7f6d7c0a9040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:35:56.790 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:56.784+0000 7f6d7259c640 1 --2- 172.21.15.43:0/1914840838 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f6d7c15c2d0 0x7f6d7c0a9040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:35:56.790 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:56.784+0000 7f6d7259c640 1 -- 172.21.15.43:0/1914840838 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6d740094d0 con 0x7f6d7c0d1b30 2023-08-09T13:35:56.790 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:56.785+0000 7f6d7ad9d640 1 --2- 172.21.15.43:0/1914840838 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f6d7c0cebf0 0x7f6d7c0b7290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-08-09T13:35:56.790 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:56.785+0000 7f6d7259c640 1 --2- 172.21.15.43:0/1914840838 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f6d7c0d1b30 0x7f6d7c15a990 secure :-1 s=READY pgs=509 cs=0 l=1 rev1=1 crypto rx=0x7f6d6400c820 tx=0x7f6d6400cc20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-09T13:35:56.791 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:56.785+0000 7f6d73fff640 1 -- 172.21.15.43:0/1914840838 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6d64007a30 con 0x7f6d7c0d1b30 2023-08-09T13:35:56.791 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:56.785+0000 7f6d73fff640 1 -- 172.21.15.43:0/1914840838 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6d64007b90 con 0x7f6d7c0d1b30 2023-08-09T13:35:56.791 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:56.785+0000 7f6d73fff640 1 -- 172.21.15.43:0/1914840838 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6d6400ce70 con 0x7f6d7c0d1b30 2023-08-09T13:35:56.791 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:56.785+0000 7f6d8081e640 1 -- 172.21.15.43:0/1914840838 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6d7c15e360 con 0x7f6d7c0d1b30 2023-08-09T13:35:56.791 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:56.785+0000 7f6d8081e640 1 -- 172.21.15.43:0/1914840838 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6d7c15e920 con 0x7f6d7c0d1b30 2023-08-09T13:35:56.792 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:56.786+0000 7f6d8081e640 1 -- 172.21.15.43:0/1914840838 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6d7c04e390 con 0x7f6d7c0d1b30 2023-08-09T13:35:56.792 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:56.788+0000 7f6d73fff640 1 -- 172.21.15.43:0/1914840838 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 5) v1 ==== 88341+0+0 (secure 0 0 0) 0x7f6d640113f0 con 0x7f6d7c0d1b30 2023-08-09T13:35:56.792 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:56.789+0000 7f6d73fff640 1 --2- 172.21.15.43:0/1914840838 >> [v2:172.21.15.203:6832/830097474,v1:172.21.15.203:6833/830097474] conn(0x7f6d5c06a050 0x7f6d5c06c510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:35:56.792 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:56.789+0000 7f6d73fff640 1 -- 172.21.15.43:0/1914840838 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(27..27 src has 1..27) v4 ==== 6957+0+0 (secure 0 0 0) 0x7f6d64088b90 con 0x7f6d7c0d1b30 2023-08-09T13:35:56.793 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:56.790+0000 7f6d7ad9d640 1 --2- 172.21.15.43:0/1914840838 >> [v2:172.21.15.203:6832/830097474,v1:172.21.15.203:6833/830097474] conn(0x7f6d5c06a050 0x7f6d5c06c510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:35:56.793 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:56.791+0000 7f6d7ad9d640 1 --2- 172.21.15.43:0/1914840838 >> [v2:172.21.15.203:6832/830097474,v1:172.21.15.203:6833/830097474] conn(0x7f6d5c06a050 0x7f6d5c06c510 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7f6d6c007f70 tx=0x7f6d6c0074c0 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-09T13:35:56.796 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:56.794+0000 7f6d73fff640 1 -- 172.21.15.43:0/1914840838 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178066 (secure 0 0 0) 0x7f6d64053e60 con 0x7f6d7c0d1b30 2023-08-09T13:35:57.035 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:57.032+0000 7f6d8081e640 1 -- 172.21.15.43:0/1914840838 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} v 0) v1 -- 0x7f6d7c0cd890 con 0x7f6d7c0d1b30 2023-08-09T13:35:57.528 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:57.526+0000 7f6d73fff640 1 -- 172.21.15.43:0/1914840838 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]=0 v11) v1 ==== 104+0+0 (secure 0 0 0) 0x7f6d64057480 con 0x7f6d7c0d1b30 2023-08-09T13:35:57.532 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:57.529+0000 7f6d8081e640 1 -- 172.21.15.43:0/1914840838 >> [v2:172.21.15.203:6832/830097474,v1:172.21.15.203:6833/830097474] conn(0x7f6d5c06a050 msgr2=0x7f6d5c06c510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:35:57.532 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:57.529+0000 7f6d8081e640 1 --2- 172.21.15.43:0/1914840838 >> [v2:172.21.15.203:6832/830097474,v1:172.21.15.203:6833/830097474] conn(0x7f6d5c06a050 0x7f6d5c06c510 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7f6d6c007f70 tx=0x7f6d6c0074c0 comp rx=0 tx=0).stop 2023-08-09T13:35:57.532 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:57.529+0000 7f6d8081e640 1 -- 172.21.15.43:0/1914840838 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f6d7c0d1b30 msgr2=0x7f6d7c15a990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:35:57.533 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:57.529+0000 7f6d8081e640 1 --2- 172.21.15.43:0/1914840838 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f6d7c0d1b30 0x7f6d7c15a990 secure :-1 s=READY pgs=509 cs=0 l=1 rev1=1 crypto rx=0x7f6d6400c820 tx=0x7f6d6400cc20 comp rx=0 tx=0).stop 2023-08-09T13:35:57.533 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:57.529+0000 7f6d8081e640 1 -- 172.21.15.43:0/1914840838 shutdown_connections 2023-08-09T13:35:57.533 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:57.529+0000 7f6d8081e640 1 --2- 172.21.15.43:0/1914840838 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f6d7c15c2d0 0x7f6d7c0a9040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:35:57.533 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:57.529+0000 7f6d8081e640 1 --2- 172.21.15.43:0/1914840838 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f6d7c0d1b30 0x7f6d7c15a990 unknown :-1 s=CLOSED pgs=509 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:35:57.533 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:57.529+0000 7f6d8081e640 1 --2- 172.21.15.43:0/1914840838 >> [v2:172.21.15.203:6832/830097474,v1:172.21.15.203:6833/830097474] conn(0x7f6d5c06a050 0x7f6d5c06c510 unknown :-1 s=CLOSED pgs=247 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:35:57.533 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:57.529+0000 7f6d8081e640 1 --2- 172.21.15.43:0/1914840838 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f6d7c0cebf0 0x7f6d7c0b7290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:35:57.533 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:57.529+0000 7f6d8081e640 1 -- 172.21.15.43:0/1914840838 >> 172.21.15.43:0/1914840838 conn(0x7f6d7c06a230 msgr2=0x7f6d7c0b51a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-09T13:35:57.534 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:57.530+0000 7f6d8081e640 1 -- 172.21.15.43:0/1914840838 shutdown_connections 2023-08-09T13:35:57.534 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:57.530+0000 7f6d8081e640 1 -- 172.21.15.43:0/1914840838 wait complete. 2023-08-09T13:35:57.554 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_metadata cephfs_metadata --yes-i-really-really-mean-it 2023-08-09T13:35:57.796 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:57.793+0000 7fb70d237640 1 Processor -- start 2023-08-09T13:35:57.796 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:57.793+0000 7fb70d237640 1 -- start start 2023-08-09T13:35:57.797 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:57.795+0000 7fb70d237640 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7fb70806b400 0x7fb70806cae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:35:57.797 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:57.795+0000 7fb70d237640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7fb70815c8e0 0x7fb70815cd50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:35:57.797 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:57.795+0000 7fb70d237640 1 -- --> v1:172.21.15.43:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb7080ad260 con 0x7fb70815c4f0 2023-08-09T13:35:57.798 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:57.795+0000 7fb70d237640 1 -- --> v2:172.21.15.43:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb70806c280 con 0x7fb70815c8e0 2023-08-09T13:35:57.798 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:57.795+0000 7fb70d237640 1 -- --> v2:172.21.15.203:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb70815d290 con 0x7fb70806b400 2023-08-09T13:35:57.798 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:57.795+0000 7fb70ca36640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7fb70815c8e0 0x7fb70815cd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:35:57.798 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:57.795+0000 7fb70ca36640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7fb70815c8e0 0x7fb70815cd50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:46046/0 (socket says 172.21.15.43:46046) 2023-08-09T13:35:57.798 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:57.795+0000 7fb70ca36640 1 -- 172.21.15.43:0/3455132582 learned_addr learned my addr 172.21.15.43:0/3455132582 (peer_addr_for_me v2:172.21.15.43:0/0) 2023-08-09T13:35:57.799 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:57.795+0000 7fb70ca36640 1 -- 172.21.15.43:0/3455132582 >> v2:172.21.15.203:3300/0 conn(0x7fb70806b400 msgr2=0x7fb70806cae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-09T13:35:57.799 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:57.795+0000 7fb70ca36640 1 --2- 172.21.15.43:0/3455132582 >> v2:172.21.15.203:3300/0 conn(0x7fb70806b400 0x7fb70806cae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:35:57.799 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:57.795+0000 7fb70ca36640 1 -- 172.21.15.43:0/3455132582 >> v1:172.21.15.43:6789/0 conn(0x7fb70815c4f0 legacy=0x7fb708160590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:35:57.799 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:57.795+0000 7fb70ca36640 1 -- 172.21.15.43:0/3455132582 --> v2:172.21.15.43:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb70806b9e0 con 0x7fb70815c8e0 2023-08-09T13:35:57.799 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:57.797+0000 7fb70ca36640 1 --2- 172.21.15.43:0/3455132582 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb70815c8e0 0x7fb70815cd50 secure :-1 s=READY pgs=511 cs=0 l=1 rev1=1 crypto rx=0x7fb700009e50 tx=0x7fb70002bf80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5bd86b08cb55f069 server_cookie=0 in_seq=0 out_seq=0 2023-08-09T13:35:57.800 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:57.797+0000 7fb7077fe640 1 -- 172.21.15.43:0/3455132582 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb70002cb20 con 0x7fb70815c8e0 2023-08-09T13:35:57.800 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:57.797+0000 7fb7077fe640 1 -- 172.21.15.43:0/3455132582 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb70002cc80 con 0x7fb70815c8e0 2023-08-09T13:35:57.800 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:57.797+0000 7fb7077fe640 1 -- 172.21.15.43:0/3455132582 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb7000343f0 con 0x7fb70815c8e0 2023-08-09T13:35:57.801 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:57.797+0000 7fb70d237640 1 -- 172.21.15.43:0/3455132582 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb70815c8e0 msgr2=0x7fb70815cd50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:35:57.801 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:57.797+0000 7fb70d237640 1 --2- 172.21.15.43:0/3455132582 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb70815c8e0 0x7fb70815cd50 secure :-1 s=READY pgs=511 cs=0 l=1 rev1=1 crypto rx=0x7fb700009e50 tx=0x7fb70002bf80 comp rx=0 tx=0).stop 2023-08-09T13:35:57.801 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:57.798+0000 7fb70d237640 1 -- 172.21.15.43:0/3455132582 shutdown_connections 2023-08-09T13:35:57.801 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:57.798+0000 7fb70d237640 1 --2- 172.21.15.43:0/3455132582 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb70815c8e0 0x7fb70815cd50 unknown :-1 s=CLOSED pgs=511 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:35:57.801 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:57.798+0000 7fb70d237640 1 --2- 172.21.15.43:0/3455132582 >> v2:172.21.15.203:3300/0 conn(0x7fb70806b400 0x7fb70806cae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:35:57.801 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:57.798+0000 7fb70d237640 1 -- 172.21.15.43:0/3455132582 >> 172.21.15.43:0/3455132582 conn(0x7fb7080ca870 msgr2=0x7fb7080cac50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-09T13:35:57.801 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:57.798+0000 7fb70d237640 1 -- 172.21.15.43:0/3455132582 shutdown_connections 2023-08-09T13:35:57.802 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:57.798+0000 7fb70d237640 1 -- 172.21.15.43:0/3455132582 wait complete. 2023-08-09T13:35:57.802 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:57.800+0000 7fb70d237640 1 Processor -- start 2023-08-09T13:35:57.802 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:57.800+0000 7fb70d237640 1 -- start start 2023-08-09T13:35:57.803 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:57.801+0000 7fb70d237640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fb70806b400 0x7fb7080b3390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:35:57.804 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:57.801+0000 7fb70d237640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb70815c8e0 0x7fb7081495e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:35:57.804 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:57.801+0000 7fb70d237640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fb70806fd20 0x7fb7080701d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:35:57.804 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:57.801+0000 7fb70d237640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb708070710 con 0x7fb70815c8e0 2023-08-09T13:35:57.804 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:57.801+0000 7fb70d237640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb7080c6260 con 0x7fb70806b400 2023-08-09T13:35:57.804 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:57.801+0000 7fb70d237640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb7080b38d0 con 0x7fb70806fd20 2023-08-09T13:35:57.805 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:57.801+0000 7fb6fffff640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb70815c8e0 0x7fb7081495e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:35:57.806 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:57.802+0000 7fb6fffff640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb70815c8e0 0x7fb7081495e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:46056/0 (socket says 172.21.15.43:46056) 2023-08-09T13:35:57.806 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:57.802+0000 7fb6fffff640 1 -- 172.21.15.43:0/1201366364 learned_addr learned my addr 172.21.15.43:0/1201366364 (peer_addr_for_me v2:172.21.15.43:0/0) 2023-08-09T13:35:57.806 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:57.802+0000 7fb707fff640 1 --2- 172.21.15.43:0/1201366364 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fb70806b400 0x7fb7080b3390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:35:57.806 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:57.802+0000 7fb70ca36640 1 --2- 172.21.15.43:0/1201366364 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fb70806fd20 0x7fb7080701d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:35:57.806 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:57.802+0000 7fb6fffff640 1 -- 172.21.15.43:0/1201366364 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fb70806fd20 msgr2=0x7fb7080701d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:35:57.806 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:57.802+0000 7fb6fffff640 1 --2- 172.21.15.43:0/1201366364 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fb70806fd20 0x7fb7080701d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:35:57.806 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:57.802+0000 7fb6fffff640 1 -- 172.21.15.43:0/1201366364 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fb70806b400 msgr2=0x7fb7080b3390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:35:57.807 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:57.802+0000 7fb6fffff640 1 --2- 172.21.15.43:0/1201366364 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fb70806b400 0x7fb7080b3390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:35:57.807 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:57.802+0000 7fb6fffff640 1 -- 172.21.15.43:0/1201366364 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb700009b30 con 0x7fb70815c8e0 2023-08-09T13:35:57.807 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:57.802+0000 7fb6fffff640 1 --2- 172.21.15.43:0/1201366364 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb70815c8e0 0x7fb7081495e0 secure :-1 s=READY pgs=512 cs=0 l=1 rev1=1 crypto rx=0x7fb6f000b4a0 tx=0x7fb6f000b960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-09T13:35:57.807 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:57.803+0000 7fb7057fa640 1 -- 172.21.15.43:0/1201366364 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb6f0010610 con 0x7fb70815c8e0 2023-08-09T13:35:57.807 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:57.803+0000 7fb7057fa640 1 -- 172.21.15.43:0/1201366364 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb6f0010770 con 0x7fb70815c8e0 2023-08-09T13:35:57.807 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:57.803+0000 7fb7057fa640 1 -- 172.21.15.43:0/1201366364 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb6f0010b20 con 0x7fb70815c8e0 2023-08-09T13:35:57.807 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:57.803+0000 7fb70d237640 1 -- 172.21.15.43:0/1201366364 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb7080c5650 con 0x7fb70815c8e0 2023-08-09T13:35:57.807 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:57.803+0000 7fb70d237640 1 -- 172.21.15.43:0/1201366364 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb7080c5c10 con 0x7fb70815c8e0 2023-08-09T13:35:57.808 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:57.804+0000 7fb70d237640 1 -- 172.21.15.43:0/1201366364 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb7080c63f0 con 0x7fb70815c8e0 2023-08-09T13:35:57.808 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:57.805+0000 7fb7057fa640 1 -- 172.21.15.43:0/1201366364 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 5) v1 ==== 88341+0+0 (secure 0 0 0) 0x7fb6f0010c80 con 0x7fb70815c8e0 2023-08-09T13:35:57.809 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:57.806+0000 7fb7057fa640 1 --2- 172.21.15.43:0/1201366364 >> [v2:172.21.15.203:6832/830097474,v1:172.21.15.203:6833/830097474] conn(0x7fb6e806a080 0x7fb6e806c540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:35:57.809 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:57.806+0000 7fb7057fa640 1 -- 172.21.15.43:0/1201366364 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(27..27 src has 1..27) v4 ==== 6957+0+0 (secure 0 0 0) 0x7fb6f00885f0 con 0x7fb70815c8e0 2023-08-09T13:35:57.809 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:57.807+0000 7fb707fff640 1 --2- 172.21.15.43:0/1201366364 >> [v2:172.21.15.203:6832/830097474,v1:172.21.15.203:6833/830097474] conn(0x7fb6e806a080 0x7fb6e806c540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:35:57.810 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:57.808+0000 7fb707fff640 1 --2- 172.21.15.43:0/1201366364 >> [v2:172.21.15.203:6832/830097474,v1:172.21.15.203:6833/830097474] conn(0x7fb6e806a080 0x7fb6e806c540 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7fb6f8000f60 tx=0x7fb6f8009040 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-09T13:35:57.813 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:57.811+0000 7fb7057fa640 1 -- 172.21.15.43:0/1201366364 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178066 (secure 0 0 0) 0x7fb6f00538c0 con 0x7fb70815c8e0 2023-08-09T13:35:58.006 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:58.004+0000 7fb70d237640 1 -- 172.21.15.43:0/1201366364 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7fb7080739f0 con 0x7fb70815c8e0 2023-08-09T13:35:58.741 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:58.738+0000 7fb7057fa640 1 -- 172.21.15.43:0/1201366364 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_metadata' removed v28) v1 ==== 181+0+0 (secure 0 0 0) 0x7fb6f0056ee0 con 0x7fb70815c8e0 2023-08-09T13:35:58.741 INFO:teuthology.orchestra.run.smithi043.stderr:pool 'cephfs_metadata' removed 2023-08-09T13:35:58.749 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:58.747+0000 7fb70d237640 1 -- 172.21.15.43:0/1201366364 >> [v2:172.21.15.203:6832/830097474,v1:172.21.15.203:6833/830097474] conn(0x7fb6e806a080 msgr2=0x7fb6e806c540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:35:58.749 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:58.747+0000 7fb70d237640 1 --2- 172.21.15.43:0/1201366364 >> [v2:172.21.15.203:6832/830097474,v1:172.21.15.203:6833/830097474] conn(0x7fb6e806a080 0x7fb6e806c540 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7fb6f8000f60 tx=0x7fb6f8009040 comp rx=0 tx=0).stop 2023-08-09T13:35:58.749 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:58.747+0000 7fb70d237640 1 -- 172.21.15.43:0/1201366364 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb70815c8e0 msgr2=0x7fb7081495e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:35:58.750 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:58.747+0000 7fb70d237640 1 --2- 172.21.15.43:0/1201366364 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb70815c8e0 0x7fb7081495e0 secure :-1 s=READY pgs=512 cs=0 l=1 rev1=1 crypto rx=0x7fb6f000b4a0 tx=0x7fb6f000b960 comp rx=0 tx=0).stop 2023-08-09T13:35:58.751 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:58.749+0000 7fb70d237640 1 -- 172.21.15.43:0/1201366364 shutdown_connections 2023-08-09T13:35:58.751 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:58.749+0000 7fb70d237640 1 --2- 172.21.15.43:0/1201366364 >> [v2:172.21.15.203:6832/830097474,v1:172.21.15.203:6833/830097474] conn(0x7fb6e806a080 0x7fb6e806c540 unknown :-1 s=CLOSED pgs=249 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:35:58.752 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:58.749+0000 7fb70d237640 1 --2- 172.21.15.43:0/1201366364 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fb70806fd20 0x7fb7080701d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:35:58.752 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:58.749+0000 7fb70d237640 1 --2- 172.21.15.43:0/1201366364 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb70815c8e0 0x7fb7081495e0 unknown :-1 s=CLOSED pgs=512 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:35:58.752 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:58.749+0000 7fb70d237640 1 --2- 172.21.15.43:0/1201366364 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fb70806b400 0x7fb7080b3390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:35:58.752 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:58.749+0000 7fb70d237640 1 -- 172.21.15.43:0/1201366364 >> 172.21.15.43:0/1201366364 conn(0x7fb7080ca870 msgr2=0x7fb70806f230 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-09T13:35:58.752 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:58.749+0000 7fb70d237640 1 -- 172.21.15.43:0/1201366364 shutdown_connections 2023-08-09T13:35:58.753 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:58.750+0000 7fb70d237640 1 -- 172.21.15.43:0/1201366364 wait complete. 2023-08-09T13:35:58.777 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_data cephfs_data --yes-i-really-really-mean-it 2023-08-09T13:35:59.003 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:59.001+0000 7f809764c640 1 Processor -- start 2023-08-09T13:35:59.004 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:59.001+0000 7f809764c640 1 -- start start 2023-08-09T13:35:59.005 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:59.002+0000 7f809764c640 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7f80900b5b80 0x7f8090167190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:35:59.005 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:59.002+0000 7f809764c640 1 -- --> v1:172.21.15.43:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f80900d1890 con 0x7f809015c6f0 2023-08-09T13:35:59.005 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:59.002+0000 7f809764c640 1 -- --> v1:172.21.15.203:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f80900d0c80 con 0x7f80900b80a0 2023-08-09T13:35:59.005 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:59.002+0000 7f809764c640 1 -- --> v2:172.21.15.203:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f80900d2d30 con 0x7f80900b5b80 2023-08-09T13:35:59.005 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:59.002+0000 7f80953e8640 1 --1- >> v1:172.21.15.43:6789/0 conn(0x7f809015c6f0 0x7f80900d03a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.43:6789/0 says I am v1:172.21.15.43:39142/0 (socket says 172.21.15.43:39142) 2023-08-09T13:35:59.005 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:59.002+0000 7f80953e8640 1 -- 172.21.15.43:0/2842823619 learned_addr learned my addr 172.21.15.43:0/2842823619 (peer_addr_for_me v1:172.21.15.43:0/0) 2023-08-09T13:35:59.007 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:59.003+0000 7f808ffff640 1 -- 172.21.15.43:0/2842823619 <== mon.0 v1:172.21.15.43:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1730335227 0 0) 0x7f80900d1890 con 0x7f809015c6f0 2023-08-09T13:35:59.007 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:59.003+0000 7f808ffff640 1 -- 172.21.15.43:0/2842823619 --> v1:172.21.15.43:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f8078003520 con 0x7f809015c6f0 2023-08-09T13:35:59.008 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:59.003+0000 7f8095be9640 1 --2- 172.21.15.43:0/2842823619 >> v2:172.21.15.203:3300/0 conn(0x7f80900b5b80 0x7f8090167190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:35:59.008 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:59.003+0000 7f808ffff640 1 -- 172.21.15.43:0/2842823619 <== mon.0 v1:172.21.15.43:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2426073536 0 0) 0x7f8078003520 con 0x7f809015c6f0 2023-08-09T13:35:59.008 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:59.003+0000 7f808ffff640 1 -- 172.21.15.43:0/2842823619 >> v2:172.21.15.203:3300/0 conn(0x7f80900b5b80 msgr2=0x7f8090167190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-09T13:35:59.008 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:59.003+0000 7f808ffff640 1 --2- 172.21.15.43:0/2842823619 >> v2:172.21.15.203:3300/0 conn(0x7f80900b5b80 0x7f8090167190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:35:59.009 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:59.003+0000 7f808ffff640 1 -- 172.21.15.43:0/2842823619 >> v1:172.21.15.203:6789/0 conn(0x7f80900b80a0 legacy=0x7f80900b5470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:35:59.009 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:59.003+0000 7f808ffff640 1 -- 172.21.15.43:0/2842823619 --> v1:172.21.15.43:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f80900d0f60 con 0x7f809015c6f0 2023-08-09T13:35:59.009 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:59.003+0000 7f808ffff640 1 -- 172.21.15.43:0/2842823619 <== mon.0 v1:172.21.15.43:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1285150740 0 0) 0x7f80800019f0 con 0x7f809015c6f0 2023-08-09T13:35:59.009 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:59.003+0000 7f808ffff640 1 -- 172.21.15.43:0/2842823619 >> v1:172.21.15.43:6789/0 conn(0x7f809015c6f0 legacy=0x7f80900d03a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:35:59.009 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:59.003+0000 7f808ffff640 1 --2- 172.21.15.43:0/2842823619 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f80780039e0 0x7f8078003e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:35:59.009 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:59.003+0000 7f808ffff640 1 --2- 172.21.15.43:0/2842823619 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f80780047c0 0x7f8078004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:35:59.009 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:59.003+0000 7f808ffff640 1 --2- 172.21.15.43:0/2842823619 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f80780051f0 0x7f807800b980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:35:59.010 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:59.004+0000 7f808ffff640 1 -- 172.21.15.43:0/2842823619 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f80900d2d30 con 0x7f80780051f0 2023-08-09T13:35:59.010 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:59.004+0000 7f808ffff640 1 -- 172.21.15.43:0/2842823619 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f807800c150 con 0x7f80780039e0 2023-08-09T13:35:59.010 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:59.004+0000 7f808ffff640 1 -- 172.21.15.43:0/2842823619 --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f807800c310 con 0x7f80780047c0 2023-08-09T13:35:59.010 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:59.004+0000 7f8095be9640 1 --2- 172.21.15.43:0/2842823619 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f80780051f0 0x7f807800b980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:35:59.010 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:59.004+0000 7f8094be7640 1 --2- 172.21.15.43:0/2842823619 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f80780047c0 0x7f8078004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:35:59.010 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:59.004+0000 7f80953e8640 1 --2- 172.21.15.43:0/2842823619 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f80780039e0 0x7f8078003e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:35:59.011 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:59.004+0000 7f8095be9640 1 -- 172.21.15.43:0/2842823619 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f80780047c0 msgr2=0x7f8078004c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-09T13:35:59.011 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:59.004+0000 7f8095be9640 1 --2- 172.21.15.43:0/2842823619 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f80780047c0 0x7f8078004c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:35:59.011 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:59.004+0000 7f8095be9640 1 -- 172.21.15.43:0/2842823619 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f80780039e0 msgr2=0x7f8078003e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-09T13:35:59.011 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:59.004+0000 7f8095be9640 1 --2- 172.21.15.43:0/2842823619 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f80780039e0 0x7f8078003e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:35:59.011 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:59.004+0000 7f8095be9640 1 -- 172.21.15.43:0/2842823619 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8078003520 con 0x7f80780051f0 2023-08-09T13:35:59.011 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:59.004+0000 7f80953e8640 1 --2- 172.21.15.43:0/2842823619 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f80780039e0 0x7f8078003e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-08-09T13:35:59.014 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:59.005+0000 7f8095be9640 1 -- 172.21.15.43:0/2842823619 reap_dead start 2023-08-09T13:35:59.014 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:59.006+0000 7f8095be9640 1 --2- 172.21.15.43:0/2842823619 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f80780051f0 0x7f807800b980 secure :-1 s=READY pgs=514 cs=0 l=1 rev1=1 crypto rx=0x7f808800ba30 tx=0x7f80780057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ffe5fbf8a86f199a server_cookie=0 in_seq=0 out_seq=0 2023-08-09T13:35:59.014 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:59.006+0000 7f808ffff640 1 -- 172.21.15.43:0/2842823619 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8088004020 con 0x7f80780051f0 2023-08-09T13:35:59.015 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:59.006+0000 7f808ffff640 1 -- 172.21.15.43:0/2842823619 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f80880041c0 con 0x7f80780051f0 2023-08-09T13:35:59.015 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:59.006+0000 7f808ffff640 1 -- 172.21.15.43:0/2842823619 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8088003610 con 0x7f80780051f0 2023-08-09T13:35:59.015 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:59.006+0000 7f809764c640 1 -- 172.21.15.43:0/2842823619 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f80780051f0 msgr2=0x7f807800b980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:35:59.015 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:59.006+0000 7f809764c640 1 --2- 172.21.15.43:0/2842823619 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f80780051f0 0x7f807800b980 secure :-1 s=READY pgs=514 cs=0 l=1 rev1=1 crypto rx=0x7f808800ba30 tx=0x7f80780057f0 comp rx=0 tx=0).stop 2023-08-09T13:35:59.015 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:59.006+0000 7f809764c640 1 -- 172.21.15.43:0/2842823619 shutdown_connections 2023-08-09T13:35:59.016 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:59.006+0000 7f809764c640 1 --2- 172.21.15.43:0/2842823619 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f80780051f0 0x7f807800b980 unknown :-1 s=CLOSED pgs=514 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:35:59.016 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:59.006+0000 7f809764c640 1 -- 172.21.15.43:0/2842823619 >> 172.21.15.43:0/2842823619 conn(0x7f80900712e0 msgr2=0x7f80900716f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-09T13:35:59.016 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:59.007+0000 7f809764c640 1 -- 172.21.15.43:0/2842823619 shutdown_connections 2023-08-09T13:35:59.016 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:59.007+0000 7f809764c640 1 -- 172.21.15.43:0/2842823619 wait complete. 2023-08-09T13:35:59.016 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:59.008+0000 7f809764c640 1 Processor -- start 2023-08-09T13:35:59.016 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:59.009+0000 7f809764c640 1 -- start start 2023-08-09T13:35:59.016 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:59.010+0000 7f809764c640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f80780051f0 0x7f8090155480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:35:59.016 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:59.010+0000 7f809764c640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f809015b060 0x7f80901546e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:35:59.017 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:59.010+0000 7f809764c640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f80900b46b0 0x7f8090159210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:35:59.017 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:59.010+0000 7f809764c640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8090154c20 con 0x7f80780051f0 2023-08-09T13:35:59.017 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:59.010+0000 7f809764c640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f809015b620 con 0x7f809015b060 2023-08-09T13:35:59.017 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:59.010+0000 7f809764c640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f809015b900 con 0x7f80900b46b0 2023-08-09T13:35:59.018 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:59.010+0000 7f80953e8640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f80780051f0 0x7f8090155480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:35:59.018 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:59.010+0000 7f80953e8640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f80780051f0 0x7f8090155480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:46062/0 (socket says 172.21.15.43:46062) 2023-08-09T13:35:59.018 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:59.010+0000 7f80953e8640 1 -- 172.21.15.43:0/2891357790 learned_addr learned my addr 172.21.15.43:0/2891357790 (peer_addr_for_me v2:172.21.15.43:0/0) 2023-08-09T13:35:59.018 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:59.010+0000 7f8095be9640 1 --2- 172.21.15.43:0/2891357790 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f80900b46b0 0x7f8090159210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:35:59.018 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:59.010+0000 7f80953e8640 1 -- 172.21.15.43:0/2891357790 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f80900b46b0 msgr2=0x7f8090159210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:35:59.018 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:59.011+0000 7f8094be7640 1 --2- 172.21.15.43:0/2891357790 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f809015b060 0x7f80901546e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:35:59.018 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:59.011+0000 7f80953e8640 1 --2- 172.21.15.43:0/2891357790 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f80900b46b0 0x7f8090159210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:35:59.018 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:59.011+0000 7f80953e8640 1 -- 172.21.15.43:0/2891357790 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f809015b060 msgr2=0x7f80901546e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:35:59.019 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:59.011+0000 7f80953e8640 1 --2- 172.21.15.43:0/2891357790 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f809015b060 0x7f80901546e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:35:59.019 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:59.011+0000 7f80953e8640 1 -- 172.21.15.43:0/2891357790 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f809015bbe0 con 0x7f80780051f0 2023-08-09T13:35:59.019 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:59.011+0000 7f8095be9640 1 --2- 172.21.15.43:0/2891357790 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f80900b46b0 0x7f8090159210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-08-09T13:35:59.019 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:59.011+0000 7f80953e8640 1 --2- 172.21.15.43:0/2891357790 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f80780051f0 0x7f8090155480 secure :-1 s=READY pgs=515 cs=0 l=1 rev1=1 crypto rx=0x7f8080002170 tx=0x7f808000b4a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-09T13:35:59.019 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:59.011+0000 7f808dffb640 1 -- 172.21.15.43:0/2891357790 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8080004d60 con 0x7f80780051f0 2023-08-09T13:35:59.020 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:59.011+0000 7f808dffb640 1 -- 172.21.15.43:0/2891357790 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f80800070d0 con 0x7f80780051f0 2023-08-09T13:35:59.020 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:59.011+0000 7f808dffb640 1 -- 172.21.15.43:0/2891357790 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8080007440 con 0x7f80780051f0 2023-08-09T13:35:59.020 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:59.011+0000 7f809764c640 1 -- 172.21.15.43:0/2891357790 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f80900b2b50 con 0x7f80780051f0 2023-08-09T13:35:59.020 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:59.011+0000 7f809764c640 1 -- 172.21.15.43:0/2891357790 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f80900b4d10 con 0x7f80780051f0 2023-08-09T13:35:59.021 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:59.013+0000 7f809764c640 1 -- 172.21.15.43:0/2891357790 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f809015bd70 con 0x7f80780051f0 2023-08-09T13:35:59.021 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:59.013+0000 7f808dffb640 1 -- 172.21.15.43:0/2891357790 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 5) v1 ==== 88341+0+0 (secure 0 0 0) 0x7f80800075a0 con 0x7f80780051f0 2023-08-09T13:35:59.021 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:59.014+0000 7f808dffb640 1 --2- 172.21.15.43:0/2891357790 >> [v2:172.21.15.203:6832/830097474,v1:172.21.15.203:6833/830097474] conn(0x7f807c06a080 0x7f807c06c540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:35:59.021 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:59.015+0000 7f808dffb640 1 -- 172.21.15.43:0/2891357790 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(28..28 src has 1..28) v4 ==== 6519+0+0 (secure 0 0 0) 0x7f8080088820 con 0x7f80780051f0 2023-08-09T13:35:59.021 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:59.015+0000 7f8094be7640 1 --2- 172.21.15.43:0/2891357790 >> [v2:172.21.15.203:6832/830097474,v1:172.21.15.203:6833/830097474] conn(0x7f807c06a080 0x7f807c06c540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:35:59.022 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:59.016+0000 7f8094be7640 1 --2- 172.21.15.43:0/2891357790 >> [v2:172.21.15.203:6832/830097474,v1:172.21.15.203:6833/830097474] conn(0x7f807c06a080 0x7f807c06c540 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7f8084007fd0 tx=0x7f8084009040 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-09T13:35:59.022 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:59.019+0000 7f808dffb640 1 -- 172.21.15.43:0/2891357790 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178066 (secure 0 0 0) 0x7f8080053d30 con 0x7f80780051f0 2023-08-09T13:35:59.110 DEBUG:teuthology.orchestra.run.smithi043:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-08-09T13:35:59.119 DEBUG:teuthology.orchestra.run.smithi203:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-08-09T13:35:59.217 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:59.215+0000 7f809764c640 1 -- 172.21.15.43:0/2891357790 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7f80900739f0 con 0x7f80780051f0 2023-08-09T13:35:59.762 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:59.752+0000 7f808dffb640 1 -- 172.21.15.43:0/2891357790 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data' removed v29) v1 ==== 169+0+0 (secure 0 0 0) 0x7f8080057350 con 0x7f80780051f0 2023-08-09T13:35:59.763 INFO:teuthology.orchestra.run.smithi043.stderr:pool 'cephfs_data' removed 2023-08-09T13:35:59.768 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:59.765+0000 7f809764c640 1 -- 172.21.15.43:0/2891357790 >> [v2:172.21.15.203:6832/830097474,v1:172.21.15.203:6833/830097474] conn(0x7f807c06a080 msgr2=0x7f807c06c540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:35:59.768 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:59.765+0000 7f809764c640 1 --2- 172.21.15.43:0/2891357790 >> [v2:172.21.15.203:6832/830097474,v1:172.21.15.203:6833/830097474] conn(0x7f807c06a080 0x7f807c06c540 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7f8084007fd0 tx=0x7f8084009040 comp rx=0 tx=0).stop 2023-08-09T13:35:59.769 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:59.765+0000 7f809764c640 1 -- 172.21.15.43:0/2891357790 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f80780051f0 msgr2=0x7f8090155480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:35:59.769 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:59.765+0000 7f809764c640 1 --2- 172.21.15.43:0/2891357790 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f80780051f0 0x7f8090155480 secure :-1 s=READY pgs=515 cs=0 l=1 rev1=1 crypto rx=0x7f8080002170 tx=0x7f808000b4a0 comp rx=0 tx=0).stop 2023-08-09T13:35:59.771 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:59.766+0000 7f809764c640 1 -- 172.21.15.43:0/2891357790 shutdown_connections 2023-08-09T13:35:59.771 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:59.767+0000 7f809764c640 1 --2- 172.21.15.43:0/2891357790 >> [v2:172.21.15.203:6832/830097474,v1:172.21.15.203:6833/830097474] conn(0x7f807c06a080 0x7f807c06c540 unknown :-1 s=CLOSED pgs=250 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:35:59.771 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:59.768+0000 7f809764c640 1 --2- 172.21.15.43:0/2891357790 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f80900b46b0 0x7f8090159210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:35:59.771 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:59.768+0000 7f809764c640 1 --2- 172.21.15.43:0/2891357790 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f809015b060 0x7f80901546e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:35:59.771 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:59.768+0000 7f809764c640 1 --2- 172.21.15.43:0/2891357790 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f80780051f0 0x7f8090155480 unknown :-1 s=CLOSED pgs=515 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:35:59.772 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:59.768+0000 7f809764c640 1 -- 172.21.15.43:0/2891357790 >> 172.21.15.43:0/2891357790 conn(0x7f80900712e0 msgr2=0x7f80900716c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-09T13:35:59.772 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:59.769+0000 7f809764c640 1 -- 172.21.15.43:0/2891357790 shutdown_connections 2023-08-09T13:35:59.772 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:35:59.769+0000 7f809764c640 1 -- 172.21.15.43:0/2891357790 wait complete. 2023-08-09T13:35:59.796 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_data_ec cephfs_data_ec --yes-i-really-really-mean-it 2023-08-09T13:36:00.004 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:00.001+0000 7f6e5539f640 1 Processor -- start 2023-08-09T13:36:00.004 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:00.002+0000 7f6e5539f640 1 -- start start 2023-08-09T13:36:00.006 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:00.003+0000 7f6e5539f640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f6e5015c6f0 0x7f6e500d03a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:36:00.006 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:00.003+0000 7f6e5539f640 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7f6e500b80a0 0x7f6e500b5470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:36:00.006 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:00.003+0000 7f6e5539f640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f6e500b59b0 0x7f6e500b5e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:36:00.006 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:00.003+0000 7f6e5539f640 1 -- --> v2:172.21.15.43:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6e5015cad0 con 0x7f6e500b59b0 2023-08-09T13:36:00.006 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:00.003+0000 7f6e5539f640 1 -- --> v2:172.21.15.203:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6e500d1890 con 0x7f6e500b80a0 2023-08-09T13:36:00.007 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:00.003+0000 7f6e5539f640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6e500d0c80 con 0x7f6e5015c6f0 2023-08-09T13:36:00.008 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:00.004+0000 7f6e4f7fe640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f6e500b59b0 0x7f6e500b5e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:36:00.008 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:00.004+0000 7f6e4f7fe640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f6e500b59b0 0x7f6e500b5e40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:46070/0 (socket says 172.21.15.43:46070) 2023-08-09T13:36:00.008 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:00.004+0000 7f6e4f7fe640 1 -- 172.21.15.43:0/537765291 learned_addr learned my addr 172.21.15.43:0/537765291 (peer_addr_for_me v2:172.21.15.43:0/0) 2023-08-09T13:36:00.009 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:00.004+0000 7f6e4effd640 1 --2- 172.21.15.43:0/537765291 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f6e5015c6f0 0x7f6e500d03a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:36:00.009 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:00.004+0000 7f6e4e7fc640 1 --2- 172.21.15.43:0/537765291 >> v2:172.21.15.203:3300/0 conn(0x7f6e500b80a0 0x7f6e500b5470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:36:00.009 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:00.004+0000 7f6e4f7fe640 1 -- 172.21.15.43:0/537765291 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f6e5015c6f0 msgr2=0x7f6e500d03a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-09T13:36:00.009 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:00.004+0000 7f6e4f7fe640 1 --2- 172.21.15.43:0/537765291 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f6e5015c6f0 0x7f6e500d03a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:36:00.010 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:00.004+0000 7f6e4f7fe640 1 -- 172.21.15.43:0/537765291 >> v2:172.21.15.203:3300/0 conn(0x7f6e500b80a0 msgr2=0x7f6e500b5470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-09T13:36:00.010 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:00.004+0000 7f6e4f7fe640 1 --2- 172.21.15.43:0/537765291 >> v2:172.21.15.203:3300/0 conn(0x7f6e500b80a0 0x7f6e500b5470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:36:00.010 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:00.004+0000 7f6e4f7fe640 1 -- 172.21.15.43:0/537765291 --> v2:172.21.15.43:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6e500d0f60 con 0x7f6e500b59b0 2023-08-09T13:36:00.012 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:00.006+0000 7f6e4f7fe640 1 --2- 172.21.15.43:0/537765291 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f6e500b59b0 0x7f6e500b5e40 secure :-1 s=READY pgs=516 cs=0 l=1 rev1=1 crypto rx=0x7f6e5006b4f0 tx=0x7f6e4802bed0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=338e0c10173c64ce server_cookie=0 in_seq=0 out_seq=0 2023-08-09T13:36:00.012 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:00.006+0000 7f6e4dffb640 1 -- 172.21.15.43:0/537765291 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6e4802ca70 con 0x7f6e500b59b0 2023-08-09T13:36:00.013 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:00.006+0000 7f6e4dffb640 1 -- 172.21.15.43:0/537765291 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6e4802cbd0 con 0x7f6e500b59b0 2023-08-09T13:36:00.013 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:00.006+0000 7f6e4dffb640 1 -- 172.21.15.43:0/537765291 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6e480343f0 con 0x7f6e500b59b0 2023-08-09T13:36:00.013 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:00.007+0000 7f6e5539f640 1 -- 172.21.15.43:0/537765291 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f6e500b59b0 msgr2=0x7f6e500b5e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:36:00.013 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:00.007+0000 7f6e5539f640 1 --2- 172.21.15.43:0/537765291 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f6e500b59b0 0x7f6e500b5e40 secure :-1 s=READY pgs=516 cs=0 l=1 rev1=1 crypto rx=0x7f6e5006b4f0 tx=0x7f6e4802bed0 comp rx=0 tx=0).stop 2023-08-09T13:36:00.014 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:00.007+0000 7f6e5539f640 1 -- 172.21.15.43:0/537765291 shutdown_connections 2023-08-09T13:36:00.014 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:00.007+0000 7f6e5539f640 1 --2- 172.21.15.43:0/537765291 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f6e500b59b0 0x7f6e500b5e40 unknown :-1 s=CLOSED pgs=516 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:36:00.014 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:00.007+0000 7f6e5539f640 1 --2- 172.21.15.43:0/537765291 >> v2:172.21.15.203:3300/0 conn(0x7f6e500b80a0 0x7f6e500b5470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:36:00.014 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:00.007+0000 7f6e5539f640 1 --2- 172.21.15.43:0/537765291 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f6e5015c6f0 0x7f6e500d03a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:36:00.015 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:00.007+0000 7f6e5539f640 1 -- 172.21.15.43:0/537765291 >> 172.21.15.43:0/537765291 conn(0x7f6e500712e0 msgr2=0x7f6e500716f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-09T13:36:00.015 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:00.007+0000 7f6e5539f640 1 -- 172.21.15.43:0/537765291 shutdown_connections 2023-08-09T13:36:00.015 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:00.007+0000 7f6e5539f640 1 -- 172.21.15.43:0/537765291 wait complete. 2023-08-09T13:36:00.015 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:00.009+0000 7f6e5539f640 1 Processor -- start 2023-08-09T13:36:00.016 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:00.009+0000 7f6e5539f640 1 -- start start 2023-08-09T13:36:00.019 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:00.010+0000 7f6e5539f640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f6e500b59b0 0x7f6e500cb980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:36:00.019 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:00.010+0000 7f6e5539f640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f6e500b80a0 0x7f6e50157fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:36:00.019 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:00.010+0000 7f6e5539f640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f6e5015c6f0 0x7f6e501584f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:36:00.020 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:00.010+0000 7f6e5539f640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6e500cbec0 con 0x7f6e500b59b0 2023-08-09T13:36:00.020 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:00.010+0000 7f6e5539f640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6e500cb4f0 con 0x7f6e5015c6f0 2023-08-09T13:36:00.020 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:00.010+0000 7f6e5539f640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6e500c8b00 con 0x7f6e500b80a0 2023-08-09T13:36:00.020 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:00.010+0000 7f6e4effd640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f6e500b59b0 0x7f6e500cb980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:36:00.021 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:00.010+0000 7f6e4effd640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f6e500b59b0 0x7f6e500cb980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:46086/0 (socket says 172.21.15.43:46086) 2023-08-09T13:36:00.021 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:00.010+0000 7f6e4effd640 1 -- 172.21.15.43:0/940015589 learned_addr learned my addr 172.21.15.43:0/940015589 (peer_addr_for_me v2:172.21.15.43:0/0) 2023-08-09T13:36:00.021 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:00.011+0000 7f6e4effd640 1 -- 172.21.15.43:0/940015589 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f6e500b80a0 msgr2=0x7f6e50157fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:36:00.021 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:00.011+0000 7f6e4effd640 1 --2- 172.21.15.43:0/940015589 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f6e500b80a0 0x7f6e50157fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:36:00.022 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:00.011+0000 7f6e4effd640 1 -- 172.21.15.43:0/940015589 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f6e5015c6f0 msgr2=0x7f6e501584f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:36:00.022 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:00.011+0000 7f6e4effd640 1 --2- 172.21.15.43:0/940015589 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f6e5015c6f0 0x7f6e501584f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:36:00.022 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:00.011+0000 7f6e4effd640 1 -- 172.21.15.43:0/940015589 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6e48009b30 con 0x7f6e500b59b0 2023-08-09T13:36:00.022 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:00.011+0000 7f6e4effd640 1 --2- 172.21.15.43:0/940015589 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f6e500b59b0 0x7f6e500cb980 secure :-1 s=READY pgs=517 cs=0 l=1 rev1=1 crypto rx=0x7f6e4000d800 tx=0x7f6e4000dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-09T13:36:00.023 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:00.011+0000 7f6e377fe640 1 -- 172.21.15.43:0/940015589 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6e4000b6c0 con 0x7f6e500b59b0 2023-08-09T13:36:00.023 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:00.011+0000 7f6e377fe640 1 -- 172.21.15.43:0/940015589 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6e4000b820 con 0x7f6e500b59b0 2023-08-09T13:36:00.023 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:00.011+0000 7f6e377fe640 1 -- 172.21.15.43:0/940015589 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6e4000bbb0 con 0x7f6e500b59b0 2023-08-09T13:36:00.023 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:00.011+0000 7f6e5539f640 1 -- 172.21.15.43:0/940015589 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6e500ca8e0 con 0x7f6e500b59b0 2023-08-09T13:36:00.023 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:00.011+0000 7f6e5539f640 1 -- 172.21.15.43:0/940015589 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6e500caea0 con 0x7f6e500b59b0 2023-08-09T13:36:00.023 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:00.013+0000 7f6e5539f640 1 -- 172.21.15.43:0/940015589 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6e500739f0 con 0x7f6e500b59b0 2023-08-09T13:36:00.023 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:00.014+0000 7f6e377fe640 1 -- 172.21.15.43:0/940015589 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 5) v1 ==== 88341+0+0 (secure 0 0 0) 0x7f6e4000bd10 con 0x7f6e500b59b0 2023-08-09T13:36:00.023 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:00.014+0000 7f6e377fe640 1 --2- 172.21.15.43:0/940015589 >> [v2:172.21.15.203:6832/830097474,v1:172.21.15.203:6833/830097474] conn(0x7f6e3c06a050 0x7f6e3c06c510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:36:00.024 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:00.015+0000 7f6e377fe640 1 -- 172.21.15.43:0/940015589 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(29..29 src has 1..29) v4 ==== 6047+0+0 (secure 0 0 0) 0x7f6e40088ae0 con 0x7f6e500b59b0 2023-08-09T13:36:00.024 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:00.015+0000 7f6e4e7fc640 1 --2- 172.21.15.43:0/940015589 >> [v2:172.21.15.203:6832/830097474,v1:172.21.15.203:6833/830097474] conn(0x7f6e3c06a050 0x7f6e3c06c510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:36:00.024 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:00.017+0000 7f6e4e7fc640 1 --2- 172.21.15.43:0/940015589 >> [v2:172.21.15.203:6832/830097474,v1:172.21.15.203:6833/830097474] conn(0x7f6e3c06a050 0x7f6e3c06c510 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7f6e50152160 tx=0x7f6e44008040 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-09T13:36:00.024 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:00.019+0000 7f6e377fe640 1 -- 172.21.15.43:0/940015589 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178066 (secure 0 0 0) 0x7f6e400541e0 con 0x7f6e500b59b0 2023-08-09T13:36:00.216 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:00.213+0000 7f6e5539f640 1 -- 172.21.15.43:0/940015589 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data_ec", "pool2": "cephfs_data_ec", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7f6e50158a50 con 0x7f6e500b59b0 2023-08-09T13:36:00.766 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:00.761+0000 7f6e377fe640 1 -- 172.21.15.43:0/940015589 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data_ec", "pool2": "cephfs_data_ec", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data_ec' removed v30) v1 ==== 178+0+0 (secure 0 0 0) 0x7f6e40057800 con 0x7f6e500b59b0 2023-08-09T13:36:00.766 INFO:teuthology.orchestra.run.smithi043.stderr:pool 'cephfs_data_ec' removed 2023-08-09T13:36:00.772 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:00.769+0000 7f6e5539f640 1 -- 172.21.15.43:0/940015589 >> [v2:172.21.15.203:6832/830097474,v1:172.21.15.203:6833/830097474] conn(0x7f6e3c06a050 msgr2=0x7f6e3c06c510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:36:00.772 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:00.769+0000 7f6e5539f640 1 --2- 172.21.15.43:0/940015589 >> [v2:172.21.15.203:6832/830097474,v1:172.21.15.203:6833/830097474] conn(0x7f6e3c06a050 0x7f6e3c06c510 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7f6e50152160 tx=0x7f6e44008040 comp rx=0 tx=0).stop 2023-08-09T13:36:00.772 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:00.769+0000 7f6e5539f640 1 -- 172.21.15.43:0/940015589 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f6e500b59b0 msgr2=0x7f6e500cb980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:36:00.772 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:00.769+0000 7f6e5539f640 1 --2- 172.21.15.43:0/940015589 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f6e500b59b0 0x7f6e500cb980 secure :-1 s=READY pgs=517 cs=0 l=1 rev1=1 crypto rx=0x7f6e4000d800 tx=0x7f6e4000dcc0 comp rx=0 tx=0).stop 2023-08-09T13:36:00.773 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:00.770+0000 7f6e5539f640 1 -- 172.21.15.43:0/940015589 shutdown_connections 2023-08-09T13:36:00.773 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:00.770+0000 7f6e5539f640 1 --2- 172.21.15.43:0/940015589 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f6e5015c6f0 0x7f6e501584f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:36:00.773 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:00.770+0000 7f6e5539f640 1 --2- 172.21.15.43:0/940015589 >> [v2:172.21.15.203:6832/830097474,v1:172.21.15.203:6833/830097474] conn(0x7f6e3c06a050 0x7f6e3c06c510 unknown :-1 s=CLOSED pgs=251 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:36:00.773 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:00.770+0000 7f6e5539f640 1 --2- 172.21.15.43:0/940015589 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f6e500b80a0 0x7f6e50157fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:36:00.774 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:00.770+0000 7f6e5539f640 1 --2- 172.21.15.43:0/940015589 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f6e500b59b0 0x7f6e500cb980 unknown :-1 s=CLOSED pgs=517 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:36:00.774 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:00.770+0000 7f6e5539f640 1 -- 172.21.15.43:0/940015589 >> 172.21.15.43:0/940015589 conn(0x7f6e500712e0 msgr2=0x7f6e500b4720 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-09T13:36:00.774 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:00.770+0000 7f6e5539f640 1 -- 172.21.15.43:0/940015589 shutdown_connections 2023-08-09T13:36:00.774 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:00.771+0000 7f6e5539f640 1 -- 172.21.15.43:0/940015589 wait complete. 2023-08-09T13:36:00.797 INFO:tasks.ceph.mds.a:Restarting daemon 2023-08-09T13:36:00.797 INFO:tasks.ceph.mds.a:Stopping old one... 2023-08-09T13:36:00.797 DEBUG:tasks.ceph.mds.a:waiting for process to exit 2023-08-09T13:36:00.798 INFO:teuthology.orchestra.run:waiting for 300 2023-08-09T13:36:00.798 INFO:tasks.ceph.mds.c:Restarting daemon 2023-08-09T13:36:00.798 INFO:tasks.ceph.mds.c:Stopping old one... 2023-08-09T13:36:00.798 DEBUG:tasks.ceph.mds.c:waiting for process to exit 2023-08-09T13:36:00.799 INFO:teuthology.orchestra.run:waiting for 300 2023-08-09T13:36:00.799 INFO:tasks.ceph.mds.b:Restarting daemon 2023-08-09T13:36:00.799 INFO:tasks.ceph.mds.b:Stopping old one... 2023-08-09T13:36:00.799 DEBUG:tasks.ceph.mds.b:waiting for process to exit 2023-08-09T13:36:00.799 INFO:teuthology.orchestra.run:waiting for 300 2023-08-09T13:36:00.800 INFO:tasks.ceph.mds.d:Restarting daemon 2023-08-09T13:36:00.800 INFO:tasks.ceph.mds.d:Stopping old one... 2023-08-09T13:36:00.800 DEBUG:tasks.ceph.mds.d:waiting for process to exit 2023-08-09T13:36:00.800 INFO:teuthology.orchestra.run:waiting for 300 2023-08-09T13:36:00.825 INFO:tasks.ceph.mds.a:Stopped 2023-08-09T13:36:00.825 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i a 2023-08-09T13:36:00.826 INFO:tasks.ceph.mds.c:Stopped 2023-08-09T13:36:00.826 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i c 2023-08-09T13:36:00.829 INFO:tasks.ceph.mds.b:Stopped 2023-08-09T13:36:00.830 DEBUG:teuthology.orchestra.run.smithi203:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i b 2023-08-09T13:36:00.833 INFO:tasks.ceph.mds.d:Stopped 2023-08-09T13:36:00.833 DEBUG:teuthology.orchestra.run.smithi203:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i d 2023-08-09T13:36:00.873 INFO:tasks.ceph.mds.a:Started 2023-08-09T13:36:00.874 DEBUG:teuthology.parallel:result is None 2023-08-09T13:36:00.878 INFO:tasks.ceph.mds.b:Started 2023-08-09T13:36:00.878 DEBUG:teuthology.parallel:result is None 2023-08-09T13:36:00.880 INFO:tasks.ceph.mds.c:Started 2023-08-09T13:36:00.880 DEBUG:teuthology.parallel:result is None 2023-08-09T13:36:00.882 INFO:tasks.ceph.mds.d:Started 2023-08-09T13:36:00.883 DEBUG:teuthology.parallel:result is None 2023-08-09T13:36:00.883 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist clear 2023-08-09T13:36:01.037 INFO:tasks.ceph.mds.d.smithi203.stdout:starting mds.d at 2023-08-09T13:36:01.037 INFO:tasks.ceph.mds.b.smithi203.stdout:starting mds.b at 2023-08-09T13:36:01.043 INFO:tasks.ceph.mds.a.smithi043.stdout:starting mds.a at 2023-08-09T13:36:01.044 INFO:tasks.ceph.mds.c.smithi043.stdout:starting mds.c at 2023-08-09T13:36:01.082 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:01.080+0000 7f803deec640 1 Processor -- start 2023-08-09T13:36:01.082 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:01.080+0000 7f803deec640 1 -- start start 2023-08-09T13:36:01.083 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:01.081+0000 7f803deec640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f80380b3fa0 0x7f80380b43b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:36:01.084 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:01.081+0000 7f803deec640 1 -- --> v1:172.21.15.43:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f80380b48f0 con 0x7f803815c680 2023-08-09T13:36:01.084 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:01.081+0000 7f803deec640 1 -- --> v1:172.21.15.203:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f80380cdf00 con 0x7f80380d0a70 2023-08-09T13:36:01.084 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:01.081+0000 7f803deec640 1 -- --> v2:172.21.15.43:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f80380d0f70 con 0x7f80380b3fa0 2023-08-09T13:36:01.084 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:01.081+0000 7f80377fe640 1 --1- >> v1:172.21.15.43:6789/0 conn(0x7f803815c680 0x7f80380ce760 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.43:6789/0 says I am v1:172.21.15.43:39158/0 (socket says 172.21.15.43:39158) 2023-08-09T13:36:01.084 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:01.081+0000 7f80377fe640 1 -- 172.21.15.43:0/2850001570 learned_addr learned my addr 172.21.15.43:0/2850001570 (peer_addr_for_me v1:172.21.15.43:0/0) 2023-08-09T13:36:01.084 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:01.081+0000 7f8037fff640 1 --2- 172.21.15.43:0/2850001570 >> v2:172.21.15.43:3300/0 conn(0x7f80380b3fa0 0x7f80380b43b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:36:01.085 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:01.082+0000 7f8037fff640 1 -- 172.21.15.43:0/2850001570 >> v1:172.21.15.203:6789/0 conn(0x7f80380d0a70 legacy=0x7f80380cd7f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:36:01.085 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:01.082+0000 7f8037fff640 1 -- 172.21.15.43:0/2850001570 >> v1:172.21.15.43:6789/0 conn(0x7f803815c680 legacy=0x7f80380ce760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:36:01.085 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:01.082+0000 7f8037fff640 1 -- 172.21.15.43:0/2850001570 --> v2:172.21.15.43:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f80380ccd10 con 0x7f80380b3fa0 2023-08-09T13:36:01.085 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:01.082+0000 7f8036ffd640 1 -- 172.21.15.43:0/2850001570 <== mon.0 v1:172.21.15.43:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 453204907 0 0) 0x7f80380b48f0 con 0x7f803815c680 2023-08-09T13:36:01.086 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:01.083+0000 7f8037fff640 1 --2- 172.21.15.43:0/2850001570 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f80380b3fa0 0x7f80380b43b0 secure :-1 s=READY pgs=525 cs=0 l=1 rev1=1 crypto rx=0x7f80300097a0 tx=0x7f803002b960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c46d2d3b25b79e07 server_cookie=0 in_seq=0 out_seq=0 2023-08-09T13:36:01.087 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:01.083+0000 7f8036ffd640 1 -- 172.21.15.43:0/2850001570 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8030030610 con 0x7f80380b3fa0 2023-08-09T13:36:01.087 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:01.083+0000 7f8036ffd640 1 -- 172.21.15.43:0/2850001570 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8030030770 con 0x7f80380b3fa0 2023-08-09T13:36:01.087 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:01.083+0000 7f8036ffd640 1 -- 172.21.15.43:0/2850001570 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8030030b00 con 0x7f80380b3fa0 2023-08-09T13:36:01.087 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:01.084+0000 7f803deec640 1 -- 172.21.15.43:0/2850001570 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f80380b3fa0 msgr2=0x7f80380b43b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:36:01.087 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:01.084+0000 7f803deec640 1 --2- 172.21.15.43:0/2850001570 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f80380b3fa0 0x7f80380b43b0 secure :-1 s=READY pgs=525 cs=0 l=1 rev1=1 crypto rx=0x7f80300097a0 tx=0x7f803002b960 comp rx=0 tx=0).stop 2023-08-09T13:36:01.087 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:01.084+0000 7f803deec640 1 -- 172.21.15.43:0/2850001570 shutdown_connections 2023-08-09T13:36:01.087 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:01.084+0000 7f803deec640 1 --2- 172.21.15.43:0/2850001570 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f80380b3fa0 0x7f80380b43b0 unknown :-1 s=CLOSED pgs=525 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:36:01.088 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:01.084+0000 7f803deec640 1 -- 172.21.15.43:0/2850001570 >> 172.21.15.43:0/2850001570 conn(0x7f8038071520 msgr2=0x7f80380d2520 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-09T13:36:01.088 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:01.084+0000 7f803deec640 1 -- 172.21.15.43:0/2850001570 shutdown_connections 2023-08-09T13:36:01.088 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:01.084+0000 7f803deec640 1 -- 172.21.15.43:0/2850001570 wait complete. 2023-08-09T13:36:01.088 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:01.086+0000 7f803deec640 1 Processor -- start 2023-08-09T13:36:01.089 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:01.086+0000 7f803deec640 1 -- start start 2023-08-09T13:36:01.089 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:01.087+0000 7f803deec640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f80380b3fa0 0x7f803814d900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:36:01.089 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:01.087+0000 7f803deec640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f80380d0a70 0x7f8038156500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:36:01.090 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:01.087+0000 7f803deec640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f803815c680 0x7f8038156a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:36:01.090 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:01.087+0000 7f803deec640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f803815cb00 con 0x7f80380d0a70 2023-08-09T13:36:01.090 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:01.087+0000 7f803deec640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f803815cde0 con 0x7f80380b3fa0 2023-08-09T13:36:01.090 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:01.087+0000 7f803deec640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f803815d0c0 con 0x7f803815c680 2023-08-09T13:36:01.091 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:01.087+0000 7f802effd640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f80380d0a70 0x7f8038156500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:36:01.091 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:01.087+0000 7f802effd640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f80380d0a70 0x7f8038156500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:46124/0 (socket says 172.21.15.43:46124) 2023-08-09T13:36:01.091 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:01.087+0000 7f802effd640 1 -- 172.21.15.43:0/2720782950 learned_addr learned my addr 172.21.15.43:0/2720782950 (peer_addr_for_me v2:172.21.15.43:0/0) 2023-08-09T13:36:01.092 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:01.088+0000 7f8037fff640 1 --2- 172.21.15.43:0/2720782950 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f803815c680 0x7f8038156a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:36:01.092 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:01.088+0000 7f80377fe640 1 --2- 172.21.15.43:0/2720782950 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f80380b3fa0 0x7f803814d900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:36:01.092 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:01.088+0000 7f802effd640 1 -- 172.21.15.43:0/2720782950 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f803815c680 msgr2=0x7f8038156a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:36:01.092 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:01.088+0000 7f802effd640 1 --2- 172.21.15.43:0/2720782950 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f803815c680 0x7f8038156a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:36:01.092 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:01.088+0000 7f802effd640 1 -- 172.21.15.43:0/2720782950 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f80380b3fa0 msgr2=0x7f803814d900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:36:01.092 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:01.088+0000 7f802effd640 1 --2- 172.21.15.43:0/2720782950 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f80380b3fa0 0x7f803814d900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:36:01.092 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:01.088+0000 7f802effd640 1 -- 172.21.15.43:0/2720782950 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8030009400 con 0x7f80380d0a70 2023-08-09T13:36:01.093 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:01.088+0000 7f802effd640 1 --2- 172.21.15.43:0/2720782950 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f80380d0a70 0x7f8038156500 secure :-1 s=READY pgs=526 cs=0 l=1 rev1=1 crypto rx=0x7f802000c400 tx=0x7f802000c8c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-09T13:36:01.093 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:01.088+0000 7f8034ff9640 1 -- 172.21.15.43:0/2720782950 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f80200052d0 con 0x7f80380d0a70 2023-08-09T13:36:01.093 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:01.088+0000 7f8034ff9640 1 -- 172.21.15.43:0/2720782950 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8020005430 con 0x7f80380d0a70 2023-08-09T13:36:01.093 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:01.088+0000 7f8034ff9640 1 -- 172.21.15.43:0/2720782950 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8020003830 con 0x7f80380d0a70 2023-08-09T13:36:01.093 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:01.088+0000 7f803deec640 1 -- 172.21.15.43:0/2720782950 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f803815d3a0 con 0x7f80380d0a70 2023-08-09T13:36:01.093 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:01.089+0000 7f803deec640 1 -- 172.21.15.43:0/2720782950 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f803814de40 con 0x7f80380d0a70 2023-08-09T13:36:01.094 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:01.090+0000 7f803deec640 1 -- 172.21.15.43:0/2720782950 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f80380739f0 con 0x7f80380d0a70 2023-08-09T13:36:01.094 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:01.091+0000 7f8034ff9640 1 -- 172.21.15.43:0/2720782950 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 5) v1 ==== 88341+0+0 (secure 0 0 0) 0x7f8020003990 con 0x7f80380d0a70 2023-08-09T13:36:01.094 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:01.092+0000 7f8034ff9640 1 --2- 172.21.15.43:0/2720782950 >> [v2:172.21.15.203:6832/830097474,v1:172.21.15.203:6833/830097474] conn(0x7f801806a050 0x7f801806c510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:36:01.095 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:01.092+0000 7f8034ff9640 1 -- 172.21.15.43:0/2720782950 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(30..30 src has 1..30) v4 ==== 5616+0+0 (secure 0 0 0) 0x7f8020087a30 con 0x7f80380d0a70 2023-08-09T13:36:01.095 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:01.093+0000 7f80377fe640 1 --2- 172.21.15.43:0/2720782950 >> [v2:172.21.15.203:6832/830097474,v1:172.21.15.203:6833/830097474] conn(0x7f801806a050 0x7f801806c510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:36:01.096 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:01.094+0000 7f80377fe640 1 --2- 172.21.15.43:0/2720782950 >> [v2:172.21.15.203:6832/830097474,v1:172.21.15.203:6833/830097474] conn(0x7f801806a050 0x7f801806c510 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7f80280012e0 tx=0x7f8028005bf0 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-09T13:36:01.099 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:01.097+0000 7f8034ff9640 1 -- 172.21.15.43:0/2720782950 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178066 (secure 0 0 0) 0x7f8020053260 con 0x7f80380d0a70 2023-08-09T13:36:01.308 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:01.306+0000 7f803deec640 1 -- 172.21.15.43:0/2720782950 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "osd blocklist clear"} v 0) v1 -- 0x7f803815fe60 con 0x7f80380d0a70 2023-08-09T13:36:02.771 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:02.769+0000 7f8034ff9640 1 -- 172.21.15.43:0/2720782950 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist clear"}]=0 removed all blocklist entries v35) v1 ==== 97+0+0 (secure 0 0 0) 0x7f8020056880 con 0x7f80380d0a70 2023-08-09T13:36:02.772 INFO:teuthology.orchestra.run.smithi043.stderr: removed all blocklist entries 2023-08-09T13:36:02.775 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:02.772+0000 7f803deec640 1 -- 172.21.15.43:0/2720782950 >> [v2:172.21.15.203:6832/830097474,v1:172.21.15.203:6833/830097474] conn(0x7f801806a050 msgr2=0x7f801806c510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:36:02.775 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:02.772+0000 7f803deec640 1 --2- 172.21.15.43:0/2720782950 >> [v2:172.21.15.203:6832/830097474,v1:172.21.15.203:6833/830097474] conn(0x7f801806a050 0x7f801806c510 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7f80280012e0 tx=0x7f8028005bf0 comp rx=0 tx=0).stop 2023-08-09T13:36:02.775 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:02.772+0000 7f803deec640 1 -- 172.21.15.43:0/2720782950 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f80380d0a70 msgr2=0x7f8038156500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:36:02.775 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:02.772+0000 7f803deec640 1 --2- 172.21.15.43:0/2720782950 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f80380d0a70 0x7f8038156500 secure :-1 s=READY pgs=526 cs=0 l=1 rev1=1 crypto rx=0x7f802000c400 tx=0x7f802000c8c0 comp rx=0 tx=0).stop 2023-08-09T13:36:02.776 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:02.772+0000 7f803deec640 1 -- 172.21.15.43:0/2720782950 shutdown_connections 2023-08-09T13:36:02.776 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:02.772+0000 7f803deec640 1 --2- 172.21.15.43:0/2720782950 >> [v2:172.21.15.203:6832/830097474,v1:172.21.15.203:6833/830097474] conn(0x7f801806a050 0x7f801806c510 unknown :-1 s=CLOSED pgs=252 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:36:02.776 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:02.772+0000 7f803deec640 1 --2- 172.21.15.43:0/2720782950 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f803815c680 0x7f8038156a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:36:02.776 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:02.772+0000 7f803deec640 1 --2- 172.21.15.43:0/2720782950 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f80380d0a70 0x7f8038156500 unknown :-1 s=CLOSED pgs=526 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:36:02.776 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:02.772+0000 7f803deec640 1 --2- 172.21.15.43:0/2720782950 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f80380b3fa0 0x7f803814d900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:36:02.777 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:02.772+0000 7f803deec640 1 -- 172.21.15.43:0/2720782950 >> 172.21.15.43:0/2720782950 conn(0x7f8038071520 msgr2=0x7f803815b510 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-09T13:36:02.777 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:02.773+0000 7f803deec640 1 -- 172.21.15.43:0/2720782950 shutdown_connections 2023-08-09T13:36:02.777 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:02.773+0000 7f803deec640 1 -- 172.21.15.43:0/2720782950 wait complete. 2023-08-09T13:36:02.797 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth ls --format=json-pretty 2023-08-09T13:36:03.032 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:03.029+0000 7fdc4ad82640 1 Processor -- start 2023-08-09T13:36:03.032 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:03.030+0000 7fdc4ad82640 1 -- start start 2023-08-09T13:36:03.034 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:03.031+0000 7fdc4ad82640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fdc440c8b30 0x7fdc440ad340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:36:03.034 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:03.031+0000 7fdc4ad82640 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7fdc440acdf0 0x7fdc4415b2c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:36:03.034 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:03.031+0000 7fdc4ad82640 1 -- --> v1:172.21.15.203:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fdc440cbd90 con 0x7fdc440c8750 2023-08-09T13:36:03.034 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:03.032+0000 7fdc4ad82640 1 -- --> v2:172.21.15.203:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fdc4415e810 con 0x7fdc440acdf0 2023-08-09T13:36:03.035 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:03.032+0000 7fdc4ad82640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdc440c9f70 con 0x7fdc440c8b30 2023-08-09T13:36:03.035 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:03.032+0000 7fdc4957f640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fdc440c8b30 0x7fdc440ad340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:36:03.036 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:03.032+0000 7fdc4957f640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fdc440c8b30 0x7fdc440ad340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.203:3301/0 says I am v2:172.21.15.43:41334/0 (socket says 172.21.15.43:41334) 2023-08-09T13:36:03.036 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:03.032+0000 7fdc4957f640 1 -- 172.21.15.43:0/2903728843 learned_addr learned my addr 172.21.15.43:0/2903728843 (peer_addr_for_me v2:172.21.15.43:0/0) 2023-08-09T13:36:03.036 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:03.032+0000 7fdc4a581640 1 --2- 172.21.15.43:0/2903728843 >> v2:172.21.15.203:3300/0 conn(0x7fdc440acdf0 0x7fdc4415b2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:36:03.036 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:03.033+0000 7fdc4957f640 1 -- 172.21.15.43:0/2903728843 >> v2:172.21.15.203:3300/0 conn(0x7fdc440acdf0 msgr2=0x7fdc4415b2c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-09T13:36:03.036 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:03.033+0000 7fdc4957f640 1 --2- 172.21.15.43:0/2903728843 >> v2:172.21.15.203:3300/0 conn(0x7fdc440acdf0 0x7fdc4415b2c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:36:03.036 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:03.033+0000 7fdc4957f640 1 -- 172.21.15.43:0/2903728843 >> v1:172.21.15.203:6789/0 conn(0x7fdc440c8750 legacy=0x7fdc4406f410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:36:03.037 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:03.033+0000 7fdc4957f640 1 -- 172.21.15.43:0/2903728843 --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdc440ca2b0 con 0x7fdc440c8b30 2023-08-09T13:36:03.037 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:03.034+0000 7fdc4957f640 1 --2- 172.21.15.43:0/2903728843 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fdc440c8b30 0x7fdc440ad340 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7fdc34009870 tx=0x7fdc3402b8f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=94188c775efc247f server_cookie=0 in_seq=0 out_seq=0 2023-08-09T13:36:03.037 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:03.035+0000 7fdc48d7e640 1 -- 172.21.15.43:0/2903728843 <== mon.2 v2:172.21.15.203:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdc34031640 con 0x7fdc440c8b30 2023-08-09T13:36:03.038 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:03.035+0000 7fdc48d7e640 1 -- 172.21.15.43:0/2903728843 <== mon.2 v2:172.21.15.203:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdc340317a0 con 0x7fdc440c8b30 2023-08-09T13:36:03.038 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:03.035+0000 7fdc48d7e640 1 -- 172.21.15.43:0/2903728843 <== mon.2 v2:172.21.15.203:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdc34031b30 con 0x7fdc440c8b30 2023-08-09T13:36:03.038 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:03.035+0000 7fdc4ad82640 1 -- 172.21.15.43:0/2903728843 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fdc440c8b30 msgr2=0x7fdc440ad340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:36:03.039 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:03.035+0000 7fdc4ad82640 1 --2- 172.21.15.43:0/2903728843 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fdc440c8b30 0x7fdc440ad340 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7fdc34009870 tx=0x7fdc3402b8f0 comp rx=0 tx=0).stop 2023-08-09T13:36:03.039 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:03.036+0000 7fdc4ad82640 1 -- 172.21.15.43:0/2903728843 shutdown_connections 2023-08-09T13:36:03.039 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:03.036+0000 7fdc4ad82640 1 --2- 172.21.15.43:0/2903728843 >> v2:172.21.15.203:3300/0 conn(0x7fdc440acdf0 0x7fdc4415b2c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:36:03.039 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:03.036+0000 7fdc4ad82640 1 --2- 172.21.15.43:0/2903728843 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fdc440c8b30 0x7fdc440ad340 unknown :-1 s=CLOSED pgs=75 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:36:03.039 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:03.036+0000 7fdc4ad82640 1 -- 172.21.15.43:0/2903728843 >> 172.21.15.43:0/2903728843 conn(0x7fdc440cde90 msgr2=0x7fdc440ce2a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-09T13:36:03.039 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:03.036+0000 7fdc4ad82640 1 -- 172.21.15.43:0/2903728843 shutdown_connections 2023-08-09T13:36:03.040 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:03.036+0000 7fdc4ad82640 1 -- 172.21.15.43:0/2903728843 wait complete. 2023-08-09T13:36:03.040 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:03.037+0000 7fdc4ad82640 1 Processor -- start 2023-08-09T13:36:03.040 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:03.038+0000 7fdc4ad82640 1 -- start start 2023-08-09T13:36:03.041 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:03.039+0000 7fdc4ad82640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fdc440acdf0 0x7fdc4415a690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:36:03.041 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:03.039+0000 7fdc4ad82640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fdc440c8750 0x7fdc440c6830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:36:03.041 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:03.039+0000 7fdc4ad82640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fdc440c8b30 0x7fdc440c3fd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:36:03.041 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:03.039+0000 7fdc4ad82640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdc440c4510 con 0x7fdc440acdf0 2023-08-09T13:36:03.042 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:03.039+0000 7fdc4ad82640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdc440c25b0 con 0x7fdc440c8b30 2023-08-09T13:36:03.042 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:03.039+0000 7fdc4ad82640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdc440c5790 con 0x7fdc440c8750 2023-08-09T13:36:03.043 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:03.039+0000 7fdc49d80640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fdc440acdf0 0x7fdc4415a690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:36:03.043 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:03.039+0000 7fdc49d80640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fdc440acdf0 0x7fdc4415a690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:46128/0 (socket says 172.21.15.43:46128) 2023-08-09T13:36:03.043 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:03.039+0000 7fdc49d80640 1 -- 172.21.15.43:0/3683385622 learned_addr learned my addr 172.21.15.43:0/3683385622 (peer_addr_for_me v2:172.21.15.43:0/0) 2023-08-09T13:36:03.043 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:03.039+0000 7fdc4957f640 1 --2- 172.21.15.43:0/3683385622 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fdc440c8750 0x7fdc440c6830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:36:03.044 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:03.039+0000 7fdc4a581640 1 --2- 172.21.15.43:0/3683385622 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fdc440c8b30 0x7fdc440c3fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:36:03.044 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:03.039+0000 7fdc49d80640 1 -- 172.21.15.43:0/3683385622 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fdc440c8750 msgr2=0x7fdc440c6830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:36:03.044 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:03.039+0000 7fdc49d80640 1 --2- 172.21.15.43:0/3683385622 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fdc440c8750 0x7fdc440c6830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:36:03.044 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:03.040+0000 7fdc49d80640 1 -- 172.21.15.43:0/3683385622 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fdc440c8b30 msgr2=0x7fdc440c3fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:36:03.044 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:03.040+0000 7fdc49d80640 1 --2- 172.21.15.43:0/3683385622 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fdc440c8b30 0x7fdc440c3fd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:36:03.044 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:03.040+0000 7fdc49d80640 1 -- 172.21.15.43:0/3683385622 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdc340094d0 con 0x7fdc440acdf0 2023-08-09T13:36:03.044 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:03.040+0000 7fdc49d80640 1 --2- 172.21.15.43:0/3683385622 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fdc440acdf0 0x7fdc4415a690 secure :-1 s=READY pgs=527 cs=0 l=1 rev1=1 crypto rx=0x7fdc4000c870 tx=0x7fdc4000cd30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-09T13:36:03.045 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:03.040+0000 7fdc3a7fc640 1 -- 172.21.15.43:0/3683385622 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdc40003b70 con 0x7fdc440acdf0 2023-08-09T13:36:03.045 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:03.040+0000 7fdc3a7fc640 1 -- 172.21.15.43:0/3683385622 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdc40003cf0 con 0x7fdc440acdf0 2023-08-09T13:36:03.045 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:03.040+0000 7fdc3a7fc640 1 -- 172.21.15.43:0/3683385622 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdc4000aa90 con 0x7fdc440acdf0 2023-08-09T13:36:03.045 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:03.040+0000 7fdc4ad82640 1 -- 172.21.15.43:0/3683385622 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdc440c5a70 con 0x7fdc440acdf0 2023-08-09T13:36:03.045 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:03.040+0000 7fdc4ad82640 1 -- 172.21.15.43:0/3683385622 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdc440c6030 con 0x7fdc440acdf0 2023-08-09T13:36:03.046 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:03.042+0000 7fdc4ad82640 1 -- 172.21.15.43:0/3683385622 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdc440c6d70 con 0x7fdc440acdf0 2023-08-09T13:36:03.046 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:03.043+0000 7fdc3a7fc640 1 -- 172.21.15.43:0/3683385622 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 5) v1 ==== 88341+0+0 (secure 0 0 0) 0x7fdc4000abf0 con 0x7fdc440acdf0 2023-08-09T13:36:03.046 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:03.044+0000 7fdc3a7fc640 1 --2- 172.21.15.43:0/3683385622 >> [v2:172.21.15.203:6832/830097474,v1:172.21.15.203:6833/830097474] conn(0x7fdc2806a080 0x7fdc2806c540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:36:03.046 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:03.044+0000 7fdc3a7fc640 1 -- 172.21.15.43:0/3683385622 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(35..35 src has 1..35) v4 ==== 5530+0+0 (secure 0 0 0) 0x7fdc40088210 con 0x7fdc440acdf0 2023-08-09T13:36:03.047 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:03.044+0000 7fdc4957f640 1 --2- 172.21.15.43:0/3683385622 >> [v2:172.21.15.203:6832/830097474,v1:172.21.15.203:6833/830097474] conn(0x7fdc2806a080 0x7fdc2806c540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:36:03.048 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:03.046+0000 7fdc4957f640 1 --2- 172.21.15.43:0/3683385622 >> [v2:172.21.15.203:6832/830097474,v1:172.21.15.203:6833/830097474] conn(0x7fdc2806a080 0x7fdc2806c540 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7fdc34036000 tx=0x7fdc3403c040 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-09T13:36:03.051 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:03.048+0000 7fdc3a7fc640 1 -- 172.21.15.43:0/3683385622 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178066 (secure 0 0 0) 0x7fdc40053b00 con 0x7fdc440acdf0 2023-08-09T13:36:03.315 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:03.313+0000 7fdc4ad82640 1 -- 172.21.15.43:0/3683385622 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "auth ls", "format": "json-pretty"} v 0) v1 -- 0x7fdc440c9230 con 0x7fdc440acdf0 2023-08-09T13:36:03.316 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:03.314+0000 7fdc3a7fc640 1 -- 172.21.15.43:0/3683385622 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "auth ls", "format": "json-pretty"}]=0 v3) v1 ==== 80+0+6645 (secure 0 0 0) 0x7fdc40003e60 con 0x7fdc440acdf0 2023-08-09T13:36:03.317 INFO:teuthology.orchestra.run.smithi043.stdout: 2023-08-09T13:36:03.317 INFO:teuthology.orchestra.run.smithi043.stdout:{ 2023-08-09T13:36:03.317 INFO:teuthology.orchestra.run.smithi043.stdout: "auth_dump": [ 2023-08-09T13:36:03.317 INFO:teuthology.orchestra.run.smithi043.stdout: { 2023-08-09T13:36:03.317 INFO:teuthology.orchestra.run.smithi043.stdout: "entity": "mds.a", 2023-08-09T13:36:03.318 INFO:teuthology.orchestra.run.smithi043.stdout: "key": "AQCOldNkPu6GCRAAZecwctvMji0Zubb60KMlyQ==", 2023-08-09T13:36:03.318 INFO:teuthology.orchestra.run.smithi043.stdout: "caps": { 2023-08-09T13:36:03.318 INFO:teuthology.orchestra.run.smithi043.stdout: "mds": "allow", 2023-08-09T13:36:03.318 INFO:teuthology.orchestra.run.smithi043.stdout: "mgr": "allow *", 2023-08-09T13:36:03.318 INFO:teuthology.orchestra.run.smithi043.stdout: "mon": "allow *", 2023-08-09T13:36:03.318 INFO:teuthology.orchestra.run.smithi043.stdout: "osd": "allow *" 2023-08-09T13:36:03.318 INFO:teuthology.orchestra.run.smithi043.stdout: } 2023-08-09T13:36:03.319 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2023-08-09T13:36:03.319 INFO:teuthology.orchestra.run.smithi043.stdout: { 2023-08-09T13:36:03.319 INFO:teuthology.orchestra.run.smithi043.stdout: "entity": "mds.b", 2023-08-09T13:36:03.319 INFO:teuthology.orchestra.run.smithi043.stdout: "key": "AQCOldNkt2+5KxAAHcckB+S5NZHsF5z22StXmw==", 2023-08-09T13:36:03.319 INFO:teuthology.orchestra.run.smithi043.stdout: "caps": { 2023-08-09T13:36:03.319 INFO:teuthology.orchestra.run.smithi043.stdout: "mds": "allow", 2023-08-09T13:36:03.319 INFO:teuthology.orchestra.run.smithi043.stdout: "mgr": "allow *", 2023-08-09T13:36:03.320 INFO:teuthology.orchestra.run.smithi043.stdout: "mon": "allow *", 2023-08-09T13:36:03.320 INFO:teuthology.orchestra.run.smithi043.stdout: "osd": "allow *" 2023-08-09T13:36:03.320 INFO:teuthology.orchestra.run.smithi043.stdout: } 2023-08-09T13:36:03.320 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2023-08-09T13:36:03.320 INFO:teuthology.orchestra.run.smithi043.stdout: { 2023-08-09T13:36:03.321 INFO:teuthology.orchestra.run.smithi043.stdout: "entity": "mds.c", 2023-08-09T13:36:03.321 INFO:teuthology.orchestra.run.smithi043.stdout: "key": "AQCOldNk5V46GhAAQBhkvfjVSJCSHLS/YkhgCA==", 2023-08-09T13:36:03.321 INFO:teuthology.orchestra.run.smithi043.stdout: "caps": { 2023-08-09T13:36:03.321 INFO:teuthology.orchestra.run.smithi043.stdout: "mds": "allow", 2023-08-09T13:36:03.321 INFO:teuthology.orchestra.run.smithi043.stdout: "mgr": "allow *", 2023-08-09T13:36:03.322 INFO:teuthology.orchestra.run.smithi043.stdout: "mon": "allow *", 2023-08-09T13:36:03.322 INFO:teuthology.orchestra.run.smithi043.stdout: "osd": "allow *" 2023-08-09T13:36:03.322 INFO:teuthology.orchestra.run.smithi043.stdout: } 2023-08-09T13:36:03.322 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2023-08-09T13:36:03.323 INFO:teuthology.orchestra.run.smithi043.stdout: { 2023-08-09T13:36:03.323 INFO:teuthology.orchestra.run.smithi043.stdout: "entity": "mds.d", 2023-08-09T13:36:03.323 INFO:teuthology.orchestra.run.smithi043.stdout: "key": "AQCPldNkNBoTARAAMVW+rSyXQKaau6I90fUX9Q==", 2023-08-09T13:36:03.324 INFO:teuthology.orchestra.run.smithi043.stdout: "caps": { 2023-08-09T13:36:03.324 INFO:teuthology.orchestra.run.smithi043.stdout: "mds": "allow", 2023-08-09T13:36:03.324 INFO:teuthology.orchestra.run.smithi043.stdout: "mgr": "allow *", 2023-08-09T13:36:03.324 INFO:teuthology.orchestra.run.smithi043.stdout: "mon": "allow *", 2023-08-09T13:36:03.325 INFO:teuthology.orchestra.run.smithi043.stdout: "osd": "allow *" 2023-08-09T13:36:03.325 INFO:teuthology.orchestra.run.smithi043.stdout: } 2023-08-09T13:36:03.325 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2023-08-09T13:36:03.325 INFO:teuthology.orchestra.run.smithi043.stdout: { 2023-08-09T13:36:03.326 INFO:teuthology.orchestra.run.smithi043.stdout: "entity": "osd.0", 2023-08-09T13:36:03.326 INFO:teuthology.orchestra.run.smithi043.stdout: "key": "AQCRldNkltPlMhAAAt3rFvraCtI0aSqGvt6UxA==", 2023-08-09T13:36:03.326 INFO:teuthology.orchestra.run.smithi043.stdout: "caps": { 2023-08-09T13:36:03.327 INFO:teuthology.orchestra.run.smithi043.stdout: "mgr": "allow profile osd", 2023-08-09T13:36:03.327 INFO:teuthology.orchestra.run.smithi043.stdout: "mon": "allow profile osd", 2023-08-09T13:36:03.327 INFO:teuthology.orchestra.run.smithi043.stdout: "osd": "allow *" 2023-08-09T13:36:03.327 INFO:teuthology.orchestra.run.smithi043.stdout: } 2023-08-09T13:36:03.327 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2023-08-09T13:36:03.327 INFO:teuthology.orchestra.run.smithi043.stdout: { 2023-08-09T13:36:03.327 INFO:teuthology.orchestra.run.smithi043.stdout: "entity": "osd.1", 2023-08-09T13:36:03.327 INFO:teuthology.orchestra.run.smithi043.stdout: "key": "AQCVldNkzpKYEhAA+wen0kV/vUDLQztq3vhfOg==", 2023-08-09T13:36:03.328 INFO:teuthology.orchestra.run.smithi043.stdout: "caps": { 2023-08-09T13:36:03.328 INFO:teuthology.orchestra.run.smithi043.stdout: "mgr": "allow profile osd", 2023-08-09T13:36:03.328 INFO:teuthology.orchestra.run.smithi043.stdout: "mon": "allow profile osd", 2023-08-09T13:36:03.328 INFO:teuthology.orchestra.run.smithi043.stdout: "osd": "allow *" 2023-08-09T13:36:03.328 INFO:teuthology.orchestra.run.smithi043.stdout: } 2023-08-09T13:36:03.328 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2023-08-09T13:36:03.328 INFO:teuthology.orchestra.run.smithi043.stdout: { 2023-08-09T13:36:03.329 INFO:teuthology.orchestra.run.smithi043.stdout: "entity": "osd.2", 2023-08-09T13:36:03.329 INFO:teuthology.orchestra.run.smithi043.stdout: "key": "AQCYldNkYbwNHhAAQY8g9dldb1V9M2MM2SHKDg==", 2023-08-09T13:36:03.329 INFO:teuthology.orchestra.run.smithi043.stdout: "caps": { 2023-08-09T13:36:03.329 INFO:teuthology.orchestra.run.smithi043.stdout: "mgr": "allow profile osd", 2023-08-09T13:36:03.329 INFO:teuthology.orchestra.run.smithi043.stdout: "mon": "allow profile osd", 2023-08-09T13:36:03.329 INFO:teuthology.orchestra.run.smithi043.stdout: "osd": "allow *" 2023-08-09T13:36:03.329 INFO:teuthology.orchestra.run.smithi043.stdout: } 2023-08-09T13:36:03.330 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2023-08-09T13:36:03.330 INFO:teuthology.orchestra.run.smithi043.stdout: { 2023-08-09T13:36:03.330 INFO:teuthology.orchestra.run.smithi043.stdout: "entity": "osd.3", 2023-08-09T13:36:03.330 INFO:teuthology.orchestra.run.smithi043.stdout: "key": "AQCbldNkeyW6KRAAsd+kgqik2aD4U7PXwccWtA==", 2023-08-09T13:36:03.330 INFO:teuthology.orchestra.run.smithi043.stdout: "caps": { 2023-08-09T13:36:03.330 INFO:teuthology.orchestra.run.smithi043.stdout: "mgr": "allow profile osd", 2023-08-09T13:36:03.330 INFO:teuthology.orchestra.run.smithi043.stdout: "mon": "allow profile osd", 2023-08-09T13:36:03.330 INFO:teuthology.orchestra.run.smithi043.stdout: "osd": "allow *" 2023-08-09T13:36:03.331 INFO:teuthology.orchestra.run.smithi043.stdout: } 2023-08-09T13:36:03.331 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2023-08-09T13:36:03.331 INFO:teuthology.orchestra.run.smithi043.stdout: { 2023-08-09T13:36:03.331 INFO:teuthology.orchestra.run.smithi043.stdout: "entity": "osd.4", 2023-08-09T13:36:03.331 INFO:teuthology.orchestra.run.smithi043.stdout: "key": "AQCildNk80UkOBAAtX+4JGpOVcfZEHetJoeZKw==", 2023-08-09T13:36:03.331 INFO:teuthology.orchestra.run.smithi043.stdout: "caps": { 2023-08-09T13:36:03.331 INFO:teuthology.orchestra.run.smithi043.stdout: "mgr": "allow profile osd", 2023-08-09T13:36:03.331 INFO:teuthology.orchestra.run.smithi043.stdout: "mon": "allow profile osd", 2023-08-09T13:36:03.332 INFO:teuthology.orchestra.run.smithi043.stdout: "osd": "allow *" 2023-08-09T13:36:03.332 INFO:teuthology.orchestra.run.smithi043.stdout: } 2023-08-09T13:36:03.332 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2023-08-09T13:36:03.332 INFO:teuthology.orchestra.run.smithi043.stdout: { 2023-08-09T13:36:03.332 INFO:teuthology.orchestra.run.smithi043.stdout: "entity": "osd.5", 2023-08-09T13:36:03.332 INFO:teuthology.orchestra.run.smithi043.stdout: "key": "AQCmldNkCA9SBxAAZj4So6MRj2Eq1I9H9JzLDA==", 2023-08-09T13:36:03.332 INFO:teuthology.orchestra.run.smithi043.stdout: "caps": { 2023-08-09T13:36:03.333 INFO:teuthology.orchestra.run.smithi043.stdout: "mgr": "allow profile osd", 2023-08-09T13:36:03.333 INFO:teuthology.orchestra.run.smithi043.stdout: "mon": "allow profile osd", 2023-08-09T13:36:03.333 INFO:teuthology.orchestra.run.smithi043.stdout: "osd": "allow *" 2023-08-09T13:36:03.333 INFO:teuthology.orchestra.run.smithi043.stdout: } 2023-08-09T13:36:03.333 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2023-08-09T13:36:03.333 INFO:teuthology.orchestra.run.smithi043.stdout: { 2023-08-09T13:36:03.333 INFO:teuthology.orchestra.run.smithi043.stdout: "entity": "osd.6", 2023-08-09T13:36:03.334 INFO:teuthology.orchestra.run.smithi043.stdout: "key": "AQCpldNkaOwAExAAraV1tUwr9uU6GheZ+hvdqQ==", 2023-08-09T13:36:03.334 INFO:teuthology.orchestra.run.smithi043.stdout: "caps": { 2023-08-09T13:36:03.334 INFO:teuthology.orchestra.run.smithi043.stdout: "mgr": "allow profile osd", 2023-08-09T13:36:03.334 INFO:teuthology.orchestra.run.smithi043.stdout: "mon": "allow profile osd", 2023-08-09T13:36:03.334 INFO:teuthology.orchestra.run.smithi043.stdout: "osd": "allow *" 2023-08-09T13:36:03.334 INFO:teuthology.orchestra.run.smithi043.stdout: } 2023-08-09T13:36:03.334 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2023-08-09T13:36:03.335 INFO:teuthology.orchestra.run.smithi043.stdout: { 2023-08-09T13:36:03.335 INFO:teuthology.orchestra.run.smithi043.stdout: "entity": "osd.7", 2023-08-09T13:36:03.335 INFO:teuthology.orchestra.run.smithi043.stdout: "key": "AQCsldNk2/GwLBAAP42EXPivzLk44X8aMEVVCg==", 2023-08-09T13:36:03.335 INFO:teuthology.orchestra.run.smithi043.stdout: "caps": { 2023-08-09T13:36:03.335 INFO:teuthology.orchestra.run.smithi043.stdout: "mgr": "allow profile osd", 2023-08-09T13:36:03.335 INFO:teuthology.orchestra.run.smithi043.stdout: "mon": "allow profile osd", 2023-08-09T13:36:03.335 INFO:teuthology.orchestra.run.smithi043.stdout: "osd": "allow *" 2023-08-09T13:36:03.335 INFO:teuthology.orchestra.run.smithi043.stdout: } 2023-08-09T13:36:03.336 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2023-08-09T13:36:03.336 INFO:teuthology.orchestra.run.smithi043.stdout: { 2023-08-09T13:36:03.336 INFO:teuthology.orchestra.run.smithi043.stdout: "entity": "client.0", 2023-08-09T13:36:03.336 INFO:teuthology.orchestra.run.smithi043.stdout: "key": "AQCPldNkg8zrDxAAM/hnJaE6S9HU/fRwQeBV5w==", 2023-08-09T13:36:03.336 INFO:teuthology.orchestra.run.smithi043.stdout: "caps": { 2023-08-09T13:36:03.336 INFO:teuthology.orchestra.run.smithi043.stdout: "mds": "allow", 2023-08-09T13:36:03.336 INFO:teuthology.orchestra.run.smithi043.stdout: "mgr": "allow r", 2023-08-09T13:36:03.337 INFO:teuthology.orchestra.run.smithi043.stdout: "mon": "allow rw", 2023-08-09T13:36:03.337 INFO:teuthology.orchestra.run.smithi043.stdout: "osd": "allow rwx" 2023-08-09T13:36:03.337 INFO:teuthology.orchestra.run.smithi043.stdout: } 2023-08-09T13:36:03.337 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2023-08-09T13:36:03.337 INFO:teuthology.orchestra.run.smithi043.stdout: { 2023-08-09T13:36:03.337 INFO:teuthology.orchestra.run.smithi043.stdout: "entity": "client.1", 2023-08-09T13:36:03.337 INFO:teuthology.orchestra.run.smithi043.stdout: "key": "AQCPldNklfFxJxAA19+L78rW6jU1uOyjey6WNA==", 2023-08-09T13:36:03.337 INFO:teuthology.orchestra.run.smithi043.stdout: "caps": { 2023-08-09T13:36:03.338 INFO:teuthology.orchestra.run.smithi043.stdout: "mds": "allow", 2023-08-09T13:36:03.338 INFO:teuthology.orchestra.run.smithi043.stdout: "mgr": "allow r", 2023-08-09T13:36:03.338 INFO:teuthology.orchestra.run.smithi043.stdout: "mon": "allow rw", 2023-08-09T13:36:03.338 INFO:teuthology.orchestra.run.smithi043.stdout: "osd": "allow rwx" 2023-08-09T13:36:03.338 INFO:teuthology.orchestra.run.smithi043.stdout: } 2023-08-09T13:36:03.338 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2023-08-09T13:36:03.338 INFO:teuthology.orchestra.run.smithi043.stdout: { 2023-08-09T13:36:03.338 INFO:teuthology.orchestra.run.smithi043.stdout: "entity": "client.2", 2023-08-09T13:36:03.339 INFO:teuthology.orchestra.run.smithi043.stdout: "key": "AQCPldNkC2lgGxAAZ8qZboOCUaQlcpkPL1ozHA==", 2023-08-09T13:36:03.339 INFO:teuthology.orchestra.run.smithi043.stdout: "caps": { 2023-08-09T13:36:03.339 INFO:teuthology.orchestra.run.smithi043.stdout: "mds": "allow", 2023-08-09T13:36:03.339 INFO:teuthology.orchestra.run.smithi043.stdout: "mgr": "allow r", 2023-08-09T13:36:03.339 INFO:teuthology.orchestra.run.smithi043.stdout: "mon": "allow rw", 2023-08-09T13:36:03.339 INFO:teuthology.orchestra.run.smithi043.stdout: "osd": "allow rwx" 2023-08-09T13:36:03.339 INFO:teuthology.orchestra.run.smithi043.stdout: } 2023-08-09T13:36:03.339 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2023-08-09T13:36:03.340 INFO:teuthology.orchestra.run.smithi043.stdout: { 2023-08-09T13:36:03.340 INFO:teuthology.orchestra.run.smithi043.stdout: "entity": "client.3", 2023-08-09T13:36:03.340 INFO:teuthology.orchestra.run.smithi043.stdout: "key": "AQCPldNkl+rXMhAA9rf3KdnSYbvngqmLYnIGGA==", 2023-08-09T13:36:03.340 INFO:teuthology.orchestra.run.smithi043.stdout: "caps": { 2023-08-09T13:36:03.340 INFO:teuthology.orchestra.run.smithi043.stdout: "mds": "allow", 2023-08-09T13:36:03.340 INFO:teuthology.orchestra.run.smithi043.stdout: "mgr": "allow r", 2023-08-09T13:36:03.340 INFO:teuthology.orchestra.run.smithi043.stdout: "mon": "allow rw", 2023-08-09T13:36:03.341 INFO:teuthology.orchestra.run.smithi043.stdout: "osd": "allow rwx" 2023-08-09T13:36:03.341 INFO:teuthology.orchestra.run.smithi043.stdout: } 2023-08-09T13:36:03.341 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2023-08-09T13:36:03.341 INFO:teuthology.orchestra.run.smithi043.stdout: { 2023-08-09T13:36:03.341 INFO:teuthology.orchestra.run.smithi043.stdout: "entity": "client.admin", 2023-08-09T13:36:03.341 INFO:teuthology.orchestra.run.smithi043.stdout: "key": "AQCMldNkUae6NRAA2FBTr0QPBdKNDoe1WK5zyg==", 2023-08-09T13:36:03.341 INFO:teuthology.orchestra.run.smithi043.stdout: "caps": { 2023-08-09T13:36:03.342 INFO:teuthology.orchestra.run.smithi043.stdout: "mds": "allow *", 2023-08-09T13:36:03.342 INFO:teuthology.orchestra.run.smithi043.stdout: "mgr": "allow *", 2023-08-09T13:36:03.342 INFO:teuthology.orchestra.run.smithi043.stdout: "mon": "allow *", 2023-08-09T13:36:03.342 INFO:teuthology.orchestra.run.smithi043.stdout: "osd": "allow *" 2023-08-09T13:36:03.342 INFO:teuthology.orchestra.run.smithi043.stdout: } 2023-08-09T13:36:03.342 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2023-08-09T13:36:03.342 INFO:teuthology.orchestra.run.smithi043.stdout: { 2023-08-09T13:36:03.343 INFO:teuthology.orchestra.run.smithi043.stdout: "entity": "client.bootstrap-mds", 2023-08-09T13:36:03.343 INFO:teuthology.orchestra.run.smithi043.stdout: "key": "AQC1ldNkKdDTKRAAbfARKpiz7UuOMM/xcfjvsA==", 2023-08-09T13:36:03.343 INFO:teuthology.orchestra.run.smithi043.stdout: "caps": { 2023-08-09T13:36:03.343 INFO:teuthology.orchestra.run.smithi043.stdout: "mon": "allow profile bootstrap-mds" 2023-08-09T13:36:03.343 INFO:teuthology.orchestra.run.smithi043.stdout: } 2023-08-09T13:36:03.343 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2023-08-09T13:36:03.343 INFO:teuthology.orchestra.run.smithi043.stdout: { 2023-08-09T13:36:03.343 INFO:teuthology.orchestra.run.smithi043.stdout: "entity": "client.bootstrap-mgr", 2023-08-09T13:36:03.344 INFO:teuthology.orchestra.run.smithi043.stdout: "key": "AQC1ldNky9zTKRAAAGDDp9vnlceyhTnaK0tgrg==", 2023-08-09T13:36:03.344 INFO:teuthology.orchestra.run.smithi043.stdout: "caps": { 2023-08-09T13:36:03.344 INFO:teuthology.orchestra.run.smithi043.stdout: "mon": "allow profile bootstrap-mgr" 2023-08-09T13:36:03.344 INFO:teuthology.orchestra.run.smithi043.stdout: } 2023-08-09T13:36:03.344 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2023-08-09T13:36:03.344 INFO:teuthology.orchestra.run.smithi043.stdout: { 2023-08-09T13:36:03.344 INFO:teuthology.orchestra.run.smithi043.stdout: "entity": "client.bootstrap-osd", 2023-08-09T13:36:03.345 INFO:teuthology.orchestra.run.smithi043.stdout: "key": "AQC1ldNkXufTKRAA49D/lgJdcb2GGFAiiPSvIg==", 2023-08-09T13:36:03.345 INFO:teuthology.orchestra.run.smithi043.stdout: "caps": { 2023-08-09T13:36:03.345 INFO:teuthology.orchestra.run.smithi043.stdout: "mon": "allow profile bootstrap-osd" 2023-08-09T13:36:03.345 INFO:teuthology.orchestra.run.smithi043.stdout: } 2023-08-09T13:36:03.345 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2023-08-09T13:36:03.345 INFO:teuthology.orchestra.run.smithi043.stdout: { 2023-08-09T13:36:03.345 INFO:teuthology.orchestra.run.smithi043.stdout: "entity": "client.bootstrap-rbd", 2023-08-09T13:36:03.345 INFO:teuthology.orchestra.run.smithi043.stdout: "key": "AQC1ldNkUvDTKRAAFm7GCbZGsIFYRC5Tmn7TfQ==", 2023-08-09T13:36:03.346 INFO:teuthology.orchestra.run.smithi043.stdout: "caps": { 2023-08-09T13:36:03.346 INFO:teuthology.orchestra.run.smithi043.stdout: "mon": "allow profile bootstrap-rbd" 2023-08-09T13:36:03.346 INFO:teuthology.orchestra.run.smithi043.stdout: } 2023-08-09T13:36:03.346 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2023-08-09T13:36:03.346 INFO:teuthology.orchestra.run.smithi043.stdout: { 2023-08-09T13:36:03.346 INFO:teuthology.orchestra.run.smithi043.stdout: "entity": "client.bootstrap-rbd-mirror", 2023-08-09T13:36:03.346 INFO:teuthology.orchestra.run.smithi043.stdout: "key": "AQC1ldNkG/nTKRAAVMic8Z3UUIkclB+aeLZf+g==", 2023-08-09T13:36:03.346 INFO:teuthology.orchestra.run.smithi043.stdout: "caps": { 2023-08-09T13:36:03.347 INFO:teuthology.orchestra.run.smithi043.stdout: "mon": "allow profile bootstrap-rbd-mirror" 2023-08-09T13:36:03.347 INFO:teuthology.orchestra.run.smithi043.stdout: } 2023-08-09T13:36:03.347 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2023-08-09T13:36:03.347 INFO:teuthology.orchestra.run.smithi043.stdout: { 2023-08-09T13:36:03.347 INFO:teuthology.orchestra.run.smithi043.stdout: "entity": "client.bootstrap-rgw", 2023-08-09T13:36:03.347 INFO:teuthology.orchestra.run.smithi043.stdout: "key": "AQC1ldNkegLUKRAA52wSsxb0t/ZvtUbQmp8g3g==", 2023-08-09T13:36:03.347 INFO:teuthology.orchestra.run.smithi043.stdout: "caps": { 2023-08-09T13:36:03.348 INFO:teuthology.orchestra.run.smithi043.stdout: "mon": "allow profile bootstrap-rgw" 2023-08-09T13:36:03.348 INFO:teuthology.orchestra.run.smithi043.stdout: } 2023-08-09T13:36:03.348 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2023-08-09T13:36:03.348 INFO:teuthology.orchestra.run.smithi043.stdout: { 2023-08-09T13:36:03.348 INFO:teuthology.orchestra.run.smithi043.stdout: "entity": "mgr.x", 2023-08-09T13:36:03.348 INFO:teuthology.orchestra.run.smithi043.stdout: "key": "AQCNldNk2ufSNxAAU7msRkwLaoLZHTIsk0ck/g==", 2023-08-09T13:36:03.348 INFO:teuthology.orchestra.run.smithi043.stdout: "caps": { 2023-08-09T13:36:03.349 INFO:teuthology.orchestra.run.smithi043.stdout: "mds": "allow *", 2023-08-09T13:36:03.349 INFO:teuthology.orchestra.run.smithi043.stdout: "mon": "allow profile mgr", 2023-08-09T13:36:03.349 INFO:teuthology.orchestra.run.smithi043.stdout: "osd": "allow *" 2023-08-09T13:36:03.349 INFO:teuthology.orchestra.run.smithi043.stdout: } 2023-08-09T13:36:03.349 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2023-08-09T13:36:03.349 INFO:teuthology.orchestra.run.smithi043.stdout: { 2023-08-09T13:36:03.349 INFO:teuthology.orchestra.run.smithi043.stdout: "entity": "mgr.y", 2023-08-09T13:36:03.349 INFO:teuthology.orchestra.run.smithi043.stdout: "key": "AQCNldNk2YgzLBAAA11Q8uAMEgE/bUni+tXIKA==", 2023-08-09T13:36:03.350 INFO:teuthology.orchestra.run.smithi043.stdout: "caps": { 2023-08-09T13:36:03.350 INFO:teuthology.orchestra.run.smithi043.stdout: "mds": "allow *", 2023-08-09T13:36:03.350 INFO:teuthology.orchestra.run.smithi043.stdout: "mon": "allow profile mgr", 2023-08-09T13:36:03.350 INFO:teuthology.orchestra.run.smithi043.stdout: "osd": "allow *" 2023-08-09T13:36:03.350 INFO:teuthology.orchestra.run.smithi043.stdout: } 2023-08-09T13:36:03.350 INFO:teuthology.orchestra.run.smithi043.stdout: } 2023-08-09T13:36:03.350 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2023-08-09T13:36:03.350 INFO:teuthology.orchestra.run.smithi043.stdout:} 2023-08-09T13:36:03.352 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:03.317+0000 7fdc4ad82640 1 -- 172.21.15.43:0/3683385622 >> [v2:172.21.15.203:6832/830097474,v1:172.21.15.203:6833/830097474] conn(0x7fdc2806a080 msgr2=0x7fdc2806c540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:36:03.352 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:03.317+0000 7fdc4ad82640 1 --2- 172.21.15.43:0/3683385622 >> [v2:172.21.15.203:6832/830097474,v1:172.21.15.203:6833/830097474] conn(0x7fdc2806a080 0x7fdc2806c540 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7fdc34036000 tx=0x7fdc3403c040 comp rx=0 tx=0).stop 2023-08-09T13:36:03.352 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:03.317+0000 7fdc4ad82640 1 -- 172.21.15.43:0/3683385622 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fdc440acdf0 msgr2=0x7fdc4415a690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:36:03.352 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:03.317+0000 7fdc4ad82640 1 --2- 172.21.15.43:0/3683385622 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fdc440acdf0 0x7fdc4415a690 secure :-1 s=READY pgs=527 cs=0 l=1 rev1=1 crypto rx=0x7fdc4000c870 tx=0x7fdc4000cd30 comp rx=0 tx=0).stop 2023-08-09T13:36:03.352 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:03.317+0000 7fdc4ad82640 1 -- 172.21.15.43:0/3683385622 shutdown_connections 2023-08-09T13:36:03.353 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:03.317+0000 7fdc4ad82640 1 --2- 172.21.15.43:0/3683385622 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fdc440c8b30 0x7fdc440c3fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:36:03.353 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:03.317+0000 7fdc4ad82640 1 --2- 172.21.15.43:0/3683385622 >> [v2:172.21.15.203:6832/830097474,v1:172.21.15.203:6833/830097474] conn(0x7fdc2806a080 0x7fdc2806c540 unknown :-1 s=CLOSED pgs=257 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:36:03.353 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:03.317+0000 7fdc4ad82640 1 --2- 172.21.15.43:0/3683385622 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fdc440c8750 0x7fdc440c6830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:36:03.353 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:03.317+0000 7fdc4ad82640 1 --2- 172.21.15.43:0/3683385622 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fdc440acdf0 0x7fdc4415a690 unknown :-1 s=CLOSED pgs=527 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:36:03.353 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:03.317+0000 7fdc4ad82640 1 -- 172.21.15.43:0/3683385622 >> 172.21.15.43:0/3683385622 conn(0x7fdc440cde90 msgr2=0x7fdc440cd170 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-09T13:36:03.353 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:03.317+0000 7fdc4ad82640 1 -- 172.21.15.43:0/3683385622 shutdown_connections 2023-08-09T13:36:03.354 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:03.317+0000 7fdc4ad82640 1 -- 172.21.15.43:0/3683385622 wait complete. 2023-08-09T13:36:03.354 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth del client.bootstrap-mds 2023-08-09T13:36:03.582 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:03.579+0000 7fa2de140640 1 Processor -- start 2023-08-09T13:36:03.582 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:03.580+0000 7fa2de140640 1 -- start start 2023-08-09T13:36:03.583 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:03.581+0000 7fa2de140640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7fa2d8070ab0 0x7fa2d8070f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:36:03.583 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:03.581+0000 7fa2de140640 1 -- --> v1:172.21.15.43:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa2d806d770 con 0x7fa2d815b460 2023-08-09T13:36:03.583 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:03.581+0000 7fa2de140640 1 -- --> v1:172.21.15.203:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa2d806bae0 con 0x7fa2d81546f0 2023-08-09T13:36:03.584 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:03.581+0000 7fa2de140640 1 -- --> v2:172.21.15.43:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa2d8071480 con 0x7fa2d8070ab0 2023-08-09T13:36:03.584 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:03.581+0000 7fa2dd93f640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7fa2d8070ab0 0x7fa2d8070f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:36:03.584 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:03.581+0000 7fa2dd93f640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7fa2d8070ab0 0x7fa2d8070f40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:46142/0 (socket says 172.21.15.43:46142) 2023-08-09T13:36:03.585 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:03.581+0000 7fa2dd93f640 1 -- 172.21.15.43:0/1421008545 learned_addr learned my addr 172.21.15.43:0/1421008545 (peer_addr_for_me v2:172.21.15.43:0/0) 2023-08-09T13:36:03.585 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:03.582+0000 7fa2dd93f640 1 -- 172.21.15.43:0/1421008545 >> v1:172.21.15.203:6789/0 conn(0x7fa2d81546f0 legacy=0x7fa2d815e640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-09T13:36:03.585 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:03.582+0000 7fa2dd93f640 1 -- 172.21.15.43:0/1421008545 >> v1:172.21.15.43:6789/0 conn(0x7fa2d815b460 legacy=0x7fa2d806b3d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:36:03.585 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:03.582+0000 7fa2dd93f640 1 -- 172.21.15.43:0/1421008545 --> v2:172.21.15.43:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa2d806cf30 con 0x7fa2d8070ab0 2023-08-09T13:36:03.586 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:03.583+0000 7fa2dd93f640 1 --2- 172.21.15.43:0/1421008545 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa2d8070ab0 0x7fa2d8070f40 secure :-1 s=READY pgs=529 cs=0 l=1 rev1=1 crypto rx=0x7fa2d0009e80 tx=0x7fa2d002bec0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=55936b28ac74533b server_cookie=0 in_seq=0 out_seq=0 2023-08-09T13:36:03.586 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:03.583+0000 7fa2cffff640 1 -- 172.21.15.43:0/1421008545 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa2d002ca60 con 0x7fa2d8070ab0 2023-08-09T13:36:03.586 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:03.583+0000 7fa2cffff640 1 -- 172.21.15.43:0/1421008545 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa2d002cbc0 con 0x7fa2d8070ab0 2023-08-09T13:36:03.587 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:03.583+0000 7fa2cffff640 1 -- 172.21.15.43:0/1421008545 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa2d00343f0 con 0x7fa2d8070ab0 2023-08-09T13:36:03.587 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:03.583+0000 7fa2de140640 1 -- 172.21.15.43:0/1421008545 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa2d8070ab0 msgr2=0x7fa2d8070f40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:36:03.587 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:03.583+0000 7fa2de140640 1 --2- 172.21.15.43:0/1421008545 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa2d8070ab0 0x7fa2d8070f40 secure :-1 s=READY pgs=529 cs=0 l=1 rev1=1 crypto rx=0x7fa2d0009e80 tx=0x7fa2d002bec0 comp rx=0 tx=0).stop 2023-08-09T13:36:03.587 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:03.584+0000 7fa2de140640 1 -- 172.21.15.43:0/1421008545 shutdown_connections 2023-08-09T13:36:03.587 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:03.584+0000 7fa2de140640 1 --2- 172.21.15.43:0/1421008545 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa2d8070ab0 0x7fa2d8070f40 unknown :-1 s=CLOSED pgs=529 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:36:03.587 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:03.584+0000 7fa2de140640 1 -- 172.21.15.43:0/1421008545 >> 172.21.15.43:0/1421008545 conn(0x7fa2d80af950 msgr2=0x7fa2d80afd60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-09T13:36:03.587 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:03.584+0000 7fa2de140640 1 -- 172.21.15.43:0/1421008545 shutdown_connections 2023-08-09T13:36:03.588 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:03.584+0000 7fa2de140640 1 -- 172.21.15.43:0/1421008545 wait complete. 2023-08-09T13:36:03.588 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:03.585+0000 7fa2de140640 1 Processor -- start 2023-08-09T13:36:03.588 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:03.586+0000 7fa2de140640 1 -- start start 2023-08-09T13:36:03.589 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:03.587+0000 7fa2de140640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fa2d8070ab0 0x7fa2d814d150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:36:03.589 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:03.587+0000 7fa2de140640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fa2d81546f0 0x7fa2d81eddb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:36:03.589 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:03.587+0000 7fa2de140640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa2d80b3330 0x7fa2d814f750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:36:03.590 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:03.587+0000 7fa2de140640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa2d80c4fa0 con 0x7fa2d80b3330 2023-08-09T13:36:03.590 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:03.587+0000 7fa2de140640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa2d81ee2f0 con 0x7fa2d81546f0 2023-08-09T13:36:03.590 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:03.587+0000 7fa2de140640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa2d80c27b0 con 0x7fa2d8070ab0 2023-08-09T13:36:03.590 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:03.587+0000 7fa2dd93f640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa2d80b3330 0x7fa2d814f750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:36:03.590 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:03.587+0000 7fa2dd93f640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa2d80b3330 0x7fa2d814f750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:46152/0 (socket says 172.21.15.43:46152) 2023-08-09T13:36:03.590 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:03.587+0000 7fa2dd93f640 1 -- 172.21.15.43:0/4194010739 learned_addr learned my addr 172.21.15.43:0/4194010739 (peer_addr_for_me v2:172.21.15.43:0/0) 2023-08-09T13:36:03.591 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:03.587+0000 7fa2dd93f640 1 -- 172.21.15.43:0/4194010739 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fa2d8070ab0 msgr2=0x7fa2d814d150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:36:03.591 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:03.587+0000 7fa2dd13e640 1 --2- 172.21.15.43:0/4194010739 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fa2d8070ab0 0x7fa2d814d150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:36:03.592 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:03.587+0000 7fa2dd93f640 1 --2- 172.21.15.43:0/4194010739 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fa2d8070ab0 0x7fa2d814d150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:36:03.592 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:03.587+0000 7fa2dd93f640 1 -- 172.21.15.43:0/4194010739 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fa2d81546f0 msgr2=0x7fa2d81eddb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:36:03.592 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:03.587+0000 7fa2dd93f640 1 --2- 172.21.15.43:0/4194010739 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fa2d81546f0 0x7fa2d81eddb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:36:03.592 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:03.587+0000 7fa2dd93f640 1 -- 172.21.15.43:0/4194010739 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa2d0009b30 con 0x7fa2d80b3330 2023-08-09T13:36:03.592 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:03.588+0000 7fa2dd93f640 1 --2- 172.21.15.43:0/4194010739 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa2d80b3330 0x7fa2d814f750 secure :-1 s=READY pgs=530 cs=0 l=1 rev1=1 crypto rx=0x7fa2d0036040 tx=0x7fa2d0004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-09T13:36:03.592 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:03.588+0000 7fa2cdffb640 1 -- 172.21.15.43:0/4194010739 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa2d003e450 con 0x7fa2d80b3330 2023-08-09T13:36:03.592 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:03.588+0000 7fa2cdffb640 1 -- 172.21.15.43:0/4194010739 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa2d003e5d0 con 0x7fa2d80b3330 2023-08-09T13:36:03.593 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:03.588+0000 7fa2cdffb640 1 -- 172.21.15.43:0/4194010739 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa2d003e980 con 0x7fa2d80b3330 2023-08-09T13:36:03.593 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:03.588+0000 7fa2de140640 1 -- 172.21.15.43:0/4194010739 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa2d80b28d0 con 0x7fa2d80b3330 2023-08-09T13:36:03.593 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:03.588+0000 7fa2de140640 1 -- 172.21.15.43:0/4194010739 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa2d80c2440 con 0x7fa2d80b3330 2023-08-09T13:36:03.593 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:03.589+0000 7fa2de140640 1 -- 172.21.15.43:0/4194010739 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa2d81536e0 con 0x7fa2d80b3330 2023-08-09T13:36:03.594 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:03.590+0000 7fa2cdffb640 1 -- 172.21.15.43:0/4194010739 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 5) v1 ==== 88341+0+0 (secure 0 0 0) 0x7fa2d003eae0 con 0x7fa2d80b3330 2023-08-09T13:36:03.594 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:03.591+0000 7fa2cdffb640 1 --2- 172.21.15.43:0/4194010739 >> [v2:172.21.15.203:6832/830097474,v1:172.21.15.203:6833/830097474] conn(0x7fa2bc06a080 0x7fa2bc06c540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:36:03.594 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:03.591+0000 7fa2cdffb640 1 -- 172.21.15.43:0/4194010739 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(35..35 src has 1..35) v4 ==== 5530+0+0 (secure 0 0 0) 0x7fa2d00ab120 con 0x7fa2d80b3330 2023-08-09T13:36:03.594 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:03.592+0000 7fa2dd13e640 1 --2- 172.21.15.43:0/4194010739 >> [v2:172.21.15.203:6832/830097474,v1:172.21.15.203:6833/830097474] conn(0x7fa2bc06a080 0x7fa2bc06c540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:36:03.595 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:03.593+0000 7fa2dd13e640 1 --2- 172.21.15.43:0/4194010739 >> [v2:172.21.15.203:6832/830097474,v1:172.21.15.203:6833/830097474] conn(0x7fa2bc06a080 0x7fa2bc06c540 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7fa2d80b2680 tx=0x7fa2d4009040 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-09T13:36:03.597 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:03.595+0000 7fa2cdffb640 1 -- 172.21.15.43:0/4194010739 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178066 (secure 0 0 0) 0x7fa2d00768f0 con 0x7fa2d80b3330 2023-08-09T13:36:03.863 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:03.861+0000 7fa2de140640 1 -- 172.21.15.43:0/4194010739 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "auth del", "entity": "client.bootstrap-mds"} v 0) v1 -- 0x7fa2d806d240 con 0x7fa2d80b3330 2023-08-09T13:36:03.871 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:03.869+0000 7fa2cdffb640 1 -- 172.21.15.43:0/4194010739 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "auth del", "entity": "client.bootstrap-mds"}]=0 v4) v1 ==== 90+0+0 (secure 0 0 0) 0x7fa2d0079f10 con 0x7fa2d80b3330 2023-08-09T13:36:03.873 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:03.871+0000 7fa2de140640 1 -- 172.21.15.43:0/4194010739 >> [v2:172.21.15.203:6832/830097474,v1:172.21.15.203:6833/830097474] conn(0x7fa2bc06a080 msgr2=0x7fa2bc06c540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:36:03.874 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:03.871+0000 7fa2de140640 1 --2- 172.21.15.43:0/4194010739 >> [v2:172.21.15.203:6832/830097474,v1:172.21.15.203:6833/830097474] conn(0x7fa2bc06a080 0x7fa2bc06c540 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7fa2d80b2680 tx=0x7fa2d4009040 comp rx=0 tx=0).stop 2023-08-09T13:36:03.874 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:03.871+0000 7fa2de140640 1 -- 172.21.15.43:0/4194010739 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa2d80b3330 msgr2=0x7fa2d814f750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:36:03.874 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:03.871+0000 7fa2de140640 1 --2- 172.21.15.43:0/4194010739 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa2d80b3330 0x7fa2d814f750 secure :-1 s=READY pgs=530 cs=0 l=1 rev1=1 crypto rx=0x7fa2d0036040 tx=0x7fa2d0004a40 comp rx=0 tx=0).stop 2023-08-09T13:36:03.875 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:03.871+0000 7fa2de140640 1 -- 172.21.15.43:0/4194010739 shutdown_connections 2023-08-09T13:36:03.875 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:03.871+0000 7fa2de140640 1 --2- 172.21.15.43:0/4194010739 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa2d80b3330 0x7fa2d814f750 unknown :-1 s=CLOSED pgs=530 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:36:03.875 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:03.871+0000 7fa2de140640 1 --2- 172.21.15.43:0/4194010739 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fa2d81546f0 0x7fa2d81eddb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:36:03.875 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:03.871+0000 7fa2de140640 1 --2- 172.21.15.43:0/4194010739 >> [v2:172.21.15.203:6832/830097474,v1:172.21.15.203:6833/830097474] conn(0x7fa2bc06a080 0x7fa2bc06c540 unknown :-1 s=CLOSED pgs=258 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:36:03.875 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:03.871+0000 7fa2de140640 1 --2- 172.21.15.43:0/4194010739 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fa2d8070ab0 0x7fa2d814d150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:36:03.875 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:03.871+0000 7fa2de140640 1 -- 172.21.15.43:0/4194010739 >> 172.21.15.43:0/4194010739 conn(0x7fa2d80af950 msgr2=0x7fa2d806b3d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-09T13:36:03.875 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:03.872+0000 7fa2de140640 1 -- 172.21.15.43:0/4194010739 shutdown_connections 2023-08-09T13:36:03.876 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:03.872+0000 7fa2de140640 1 -- 172.21.15.43:0/4194010739 wait complete. 2023-08-09T13:36:03.893 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth del client.bootstrap-mgr 2023-08-09T13:36:04.129 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:04.126+0000 7fc619d69640 1 Processor -- start 2023-08-09T13:36:04.129 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:04.126+0000 7fc619d69640 1 -- start start 2023-08-09T13:36:04.130 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:04.128+0000 7fc619d69640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7fc61415c550 0x7fc61406cd70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:36:04.131 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:04.128+0000 7fc619d69640 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7fc6140c9a70 0x7fc6140cba30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:36:04.131 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:04.128+0000 7fc619d69640 1 -- --> v1:172.21.15.203:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc614070c70 con 0x7fc61406d2b0 2023-08-09T13:36:04.131 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:04.128+0000 7fc619d69640 1 -- --> v2:172.21.15.43:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc61406d740 con 0x7fc61415c550 2023-08-09T13:36:04.131 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:04.128+0000 7fc619d69640 1 -- --> v2:172.21.15.203:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc61415c930 con 0x7fc6140c9a70 2023-08-09T13:36:04.132 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:04.128+0000 7fc618d67640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7fc61415c550 0x7fc61406cd70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:36:04.132 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:04.128+0000 7fc618d67640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7fc61415c550 0x7fc61406cd70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:46162/0 (socket says 172.21.15.43:46162) 2023-08-09T13:36:04.132 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:04.128+0000 7fc618d67640 1 -- 172.21.15.43:0/134350234 learned_addr learned my addr 172.21.15.43:0/134350234 (peer_addr_for_me v2:172.21.15.43:0/0) 2023-08-09T13:36:04.132 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:04.129+0000 7fc618d67640 1 -- 172.21.15.43:0/134350234 >> v2:172.21.15.203:3300/0 conn(0x7fc6140c9a70 msgr2=0x7fc6140cba30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-09T13:36:04.132 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:04.129+0000 7fc618d67640 1 --2- 172.21.15.43:0/134350234 >> v2:172.21.15.203:3300/0 conn(0x7fc6140c9a70 0x7fc6140cba30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:36:04.133 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:04.129+0000 7fc618d67640 1 -- 172.21.15.43:0/134350234 >> v1:172.21.15.203:6789/0 conn(0x7fc61406d2b0 legacy=0x7fc6140c9360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-09T13:36:04.133 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:04.129+0000 7fc618d67640 1 -- 172.21.15.43:0/134350234 --> v2:172.21.15.43:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc614070060 con 0x7fc61415c550 2023-08-09T13:36:04.134 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:04.130+0000 7fc618d67640 1 --2- 172.21.15.43:0/134350234 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc61415c550 0x7fc61406cd70 secure :-1 s=READY pgs=531 cs=0 l=1 rev1=1 crypto rx=0x7fc6140b0740 tx=0x7fc60802b910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5051253cbd8411f8 server_cookie=0 in_seq=0 out_seq=0 2023-08-09T13:36:04.134 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:04.130+0000 7fc6137fe640 1 -- 172.21.15.43:0/134350234 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc608031870 con 0x7fc61415c550 2023-08-09T13:36:04.134 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:04.130+0000 7fc6137fe640 1 -- 172.21.15.43:0/134350234 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc6080319d0 con 0x7fc61415c550 2023-08-09T13:36:04.134 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:04.130+0000 7fc6137fe640 1 -- 172.21.15.43:0/134350234 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc608031d60 con 0x7fc61415c550 2023-08-09T13:36:04.134 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:04.131+0000 7fc619d69640 1 -- 172.21.15.43:0/134350234 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc61415c550 msgr2=0x7fc61406cd70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:36:04.134 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:04.131+0000 7fc619d69640 1 --2- 172.21.15.43:0/134350234 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc61415c550 0x7fc61406cd70 secure :-1 s=READY pgs=531 cs=0 l=1 rev1=1 crypto rx=0x7fc6140b0740 tx=0x7fc60802b910 comp rx=0 tx=0).stop 2023-08-09T13:36:04.135 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:04.131+0000 7fc619d69640 1 -- 172.21.15.43:0/134350234 shutdown_connections 2023-08-09T13:36:04.135 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:04.131+0000 7fc619d69640 1 --2- 172.21.15.43:0/134350234 >> v2:172.21.15.203:3300/0 conn(0x7fc6140c9a70 0x7fc6140cba30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:36:04.135 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:04.131+0000 7fc619d69640 1 --2- 172.21.15.43:0/134350234 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc61415c550 0x7fc61406cd70 unknown :-1 s=CLOSED pgs=531 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:36:04.135 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:04.131+0000 7fc619d69640 1 -- 172.21.15.43:0/134350234 >> 172.21.15.43:0/134350234 conn(0x7fc6140af950 msgr2=0x7fc6140afd60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-09T13:36:04.135 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:04.131+0000 7fc619d69640 1 -- 172.21.15.43:0/134350234 shutdown_connections 2023-08-09T13:36:04.135 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:04.131+0000 7fc619d69640 1 -- 172.21.15.43:0/134350234 wait complete. 2023-08-09T13:36:04.136 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:04.133+0000 7fc619d69640 1 Processor -- start 2023-08-09T13:36:04.136 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:04.133+0000 7fc619d69640 1 -- start start 2023-08-09T13:36:04.137 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:04.134+0000 7fc619d69640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc61406d2b0 0x7fc6140c69c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:36:04.137 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:04.134+0000 7fc619d69640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fc6140c9a70 0x7fc6140c4f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:36:04.137 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:04.134+0000 7fc619d69640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fc61415c550 0x7fc61414c850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:36:04.137 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:04.134+0000 7fc619d69640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc6140c6420 con 0x7fc61406d2b0 2023-08-09T13:36:04.137 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:04.134+0000 7fc619d69640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc6140c58a0 con 0x7fc61415c550 2023-08-09T13:36:04.137 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:04.134+0000 7fc619d69640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc6140c5b80 con 0x7fc6140c9a70 2023-08-09T13:36:04.137 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:04.134+0000 7fc618d67640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc61406d2b0 0x7fc6140c69c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:36:04.138 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:04.134+0000 7fc618d67640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc61406d2b0 0x7fc6140c69c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:46178/0 (socket says 172.21.15.43:46178) 2023-08-09T13:36:04.138 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:04.134+0000 7fc618d67640 1 -- 172.21.15.43:0/579978638 learned_addr learned my addr 172.21.15.43:0/579978638 (peer_addr_for_me v2:172.21.15.43:0/0) 2023-08-09T13:36:04.138 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:04.135+0000 7fc618d67640 1 -- 172.21.15.43:0/579978638 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fc6140c9a70 msgr2=0x7fc6140c4f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:36:04.139 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:04.135+0000 7fc618d67640 1 --2- 172.21.15.43:0/579978638 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fc6140c9a70 0x7fc6140c4f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:36:04.139 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:04.135+0000 7fc618d67640 1 -- 172.21.15.43:0/579978638 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fc61415c550 msgr2=0x7fc61414c850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:36:04.139 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:04.135+0000 7fc618d67640 1 --2- 172.21.15.43:0/579978638 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fc61415c550 0x7fc61414c850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:36:04.139 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:04.135+0000 7fc618d67640 1 -- 172.21.15.43:0/579978638 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc6080094d0 con 0x7fc61406d2b0 2023-08-09T13:36:04.139 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:04.135+0000 7fc618d67640 1 --2- 172.21.15.43:0/579978638 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc61406d2b0 0x7fc6140c69c0 secure :-1 s=READY pgs=532 cs=0 l=1 rev1=1 crypto rx=0x7fc608000c00 tx=0x7fc6080046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-09T13:36:04.139 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:04.135+0000 7fc6117fa640 1 -- 172.21.15.43:0/579978638 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc608031840 con 0x7fc61406d2b0 2023-08-09T13:36:04.139 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:04.135+0000 7fc6117fa640 1 -- 172.21.15.43:0/579978638 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc6080443f0 con 0x7fc61406d2b0 2023-08-09T13:36:04.140 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:04.135+0000 7fc6117fa640 1 -- 172.21.15.43:0/579978638 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc6080447c0 con 0x7fc61406d2b0 2023-08-09T13:36:04.140 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:04.135+0000 7fc619d69640 1 -- 172.21.15.43:0/579978638 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc6140c5e60 con 0x7fc61406d2b0 2023-08-09T13:36:04.140 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:04.136+0000 7fc619d69640 1 -- 172.21.15.43:0/579978638 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc6140c6f00 con 0x7fc61406d2b0 2023-08-09T13:36:04.140 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:04.137+0000 7fc619d69640 1 -- 172.21.15.43:0/579978638 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc6140739f0 con 0x7fc61406d2b0 2023-08-09T13:36:04.141 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:04.138+0000 7fc6117fa640 1 -- 172.21.15.43:0/579978638 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 5) v1 ==== 88341+0+0 (secure 0 0 0) 0x7fc60804d450 con 0x7fc61406d2b0 2023-08-09T13:36:04.141 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:04.138+0000 7fc6117fa640 1 --2- 172.21.15.43:0/579978638 >> [v2:172.21.15.203:6832/830097474,v1:172.21.15.203:6833/830097474] conn(0x7fc5f806a050 0x7fc5f806c510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:36:04.141 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:04.139+0000 7fc6117fa640 1 -- 172.21.15.43:0/579978638 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(35..35 src has 1..35) v4 ==== 5530+0+0 (secure 0 0 0) 0x7fc60802d040 con 0x7fc61406d2b0 2023-08-09T13:36:04.142 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:04.139+0000 7fc613fff640 1 --2- 172.21.15.43:0/579978638 >> [v2:172.21.15.203:6832/830097474,v1:172.21.15.203:6833/830097474] conn(0x7fc5f806a050 0x7fc5f806c510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:36:04.143 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:04.140+0000 7fc613fff640 1 --2- 172.21.15.43:0/579978638 >> [v2:172.21.15.203:6832/830097474,v1:172.21.15.203:6833/830097474] conn(0x7fc5f806a050 0x7fc5f806c510 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7fc614164ef0 tx=0x7fc600007990 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-09T13:36:04.145 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:04.143+0000 7fc6117fa640 1 -- 172.21.15.43:0/579978638 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178066 (secure 0 0 0) 0x7fc608076420 con 0x7fc61406d2b0 2023-08-09T13:36:04.409 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:04.407+0000 7fc619d69640 1 -- 172.21.15.43:0/579978638 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "auth del", "entity": "client.bootstrap-mgr"} v 0) v1 -- 0x7fc6140aec80 con 0x7fc61406d2b0 2023-08-09T13:36:04.417 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:04.415+0000 7fc6117fa640 1 -- 172.21.15.43:0/579978638 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "auth del", "entity": "client.bootstrap-mgr"}]=0 v5) v1 ==== 90+0+0 (secure 0 0 0) 0x7fc608079a40 con 0x7fc61406d2b0 2023-08-09T13:36:04.420 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:04.417+0000 7fc619d69640 1 -- 172.21.15.43:0/579978638 >> [v2:172.21.15.203:6832/830097474,v1:172.21.15.203:6833/830097474] conn(0x7fc5f806a050 msgr2=0x7fc5f806c510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:36:04.420 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:04.417+0000 7fc619d69640 1 --2- 172.21.15.43:0/579978638 >> [v2:172.21.15.203:6832/830097474,v1:172.21.15.203:6833/830097474] conn(0x7fc5f806a050 0x7fc5f806c510 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7fc614164ef0 tx=0x7fc600007990 comp rx=0 tx=0).stop 2023-08-09T13:36:04.420 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:04.417+0000 7fc619d69640 1 -- 172.21.15.43:0/579978638 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc61406d2b0 msgr2=0x7fc6140c69c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:36:04.420 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:04.417+0000 7fc619d69640 1 --2- 172.21.15.43:0/579978638 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc61406d2b0 0x7fc6140c69c0 secure :-1 s=READY pgs=532 cs=0 l=1 rev1=1 crypto rx=0x7fc608000c00 tx=0x7fc6080046c0 comp rx=0 tx=0).stop 2023-08-09T13:36:04.421 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:04.417+0000 7fc619d69640 1 -- 172.21.15.43:0/579978638 shutdown_connections 2023-08-09T13:36:04.421 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:04.417+0000 7fc619d69640 1 --2- 172.21.15.43:0/579978638 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fc61415c550 0x7fc61414c850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:36:04.421 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:04.417+0000 7fc619d69640 1 --2- 172.21.15.43:0/579978638 >> [v2:172.21.15.203:6832/830097474,v1:172.21.15.203:6833/830097474] conn(0x7fc5f806a050 0x7fc5f806c510 unknown :-1 s=CLOSED pgs=259 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:36:04.421 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:04.417+0000 7fc619d69640 1 --2- 172.21.15.43:0/579978638 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fc6140c9a70 0x7fc6140c4f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:36:04.421 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:04.417+0000 7fc619d69640 1 --2- 172.21.15.43:0/579978638 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc61406d2b0 0x7fc6140c69c0 unknown :-1 s=CLOSED pgs=532 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:36:04.421 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:04.417+0000 7fc619d69640 1 -- 172.21.15.43:0/579978638 >> 172.21.15.43:0/579978638 conn(0x7fc6140af950 msgr2=0x7fc6140af020 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-09T13:36:04.421 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:04.417+0000 7fc619d69640 1 -- 172.21.15.43:0/579978638 shutdown_connections 2023-08-09T13:36:04.421 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:04.418+0000 7fc619d69640 1 -- 172.21.15.43:0/579978638 wait complete. 2023-08-09T13:36:04.437 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth del client.bootstrap-osd 2023-08-09T13:36:04.668 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:04.666+0000 7fb1c1fa4640 1 Processor -- start 2023-08-09T13:36:04.669 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:04.666+0000 7fb1c1fa4640 1 -- start start 2023-08-09T13:36:04.671 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:04.667+0000 7fb1c1fa4640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7fb1bc15f7b0 0x7fb1bc0b4760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:36:04.671 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:04.667+0000 7fb1c1fa4640 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7fb1bc0b1050 0x7fb1bc0ccfa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:36:04.672 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:04.668+0000 7fb1c1fa4640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fb1bc0cd4e0 0x7fb1bc0ceae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:36:04.672 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:04.668+0000 7fb1c1fa4640 1 -- --> v2:172.21.15.43:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb1bc0cd970 con 0x7fb1bc15f7b0 2023-08-09T13:36:04.672 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:04.668+0000 7fb1c1fa4640 1 -- --> v2:172.21.15.203:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb1bc0b7bf0 con 0x7fb1bc0b1050 2023-08-09T13:36:04.672 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:04.668+0000 7fb1c1fa4640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb1bc0b7ed0 con 0x7fb1bc0cd4e0 2023-08-09T13:36:04.672 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:04.668+0000 7fb1c0fa2640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7fb1bc15f7b0 0x7fb1bc0b4760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:36:04.673 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:04.668+0000 7fb1c0fa2640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7fb1bc15f7b0 0x7fb1bc0b4760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:46180/0 (socket says 172.21.15.43:46180) 2023-08-09T13:36:04.673 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:04.668+0000 7fb1c0fa2640 1 -- 172.21.15.43:0/967828326 learned_addr learned my addr 172.21.15.43:0/967828326 (peer_addr_for_me v2:172.21.15.43:0/0) 2023-08-09T13:36:04.673 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:04.668+0000 7fb1c17a3640 1 --2- 172.21.15.43:0/967828326 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fb1bc0cd4e0 0x7fb1bc0ceae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:36:04.673 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:04.668+0000 7fb1b3fff640 1 --2- 172.21.15.43:0/967828326 >> v2:172.21.15.203:3300/0 conn(0x7fb1bc0b1050 0x7fb1bc0ccfa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:36:04.674 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:04.669+0000 7fb1c0fa2640 1 -- 172.21.15.43:0/967828326 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fb1bc0cd4e0 msgr2=0x7fb1bc0ceae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-09T13:36:04.674 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:04.669+0000 7fb1c0fa2640 1 --2- 172.21.15.43:0/967828326 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fb1bc0cd4e0 0x7fb1bc0ceae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:36:04.674 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:04.669+0000 7fb1c0fa2640 1 -- 172.21.15.43:0/967828326 >> v2:172.21.15.203:3300/0 conn(0x7fb1bc0b1050 msgr2=0x7fb1bc0ccfa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-09T13:36:04.674 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:04.669+0000 7fb1c0fa2640 1 --2- 172.21.15.43:0/967828326 >> v2:172.21.15.203:3300/0 conn(0x7fb1bc0b1050 0x7fb1bc0ccfa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:36:04.675 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:04.669+0000 7fb1c0fa2640 1 -- 172.21.15.43:0/967828326 --> v2:172.21.15.43:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb1bc0d0370 con 0x7fb1bc15f7b0 2023-08-09T13:36:04.675 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:04.669+0000 7fb1c17a3640 1 --2- 172.21.15.43:0/967828326 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fb1bc0cd4e0 0x7fb1bc0ceae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-08-09T13:36:04.676 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:04.670+0000 7fb1c0fa2640 1 --2- 172.21.15.43:0/967828326 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb1bc15f7b0 0x7fb1bc0b4760 secure :-1 s=READY pgs=533 cs=0 l=1 rev1=1 crypto rx=0x7fb1bc0ce9d0 tx=0x7fb1b802b920 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7befdac49049d352 server_cookie=0 in_seq=0 out_seq=0 2023-08-09T13:36:04.676 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:04.670+0000 7fb1b37fe640 1 -- 172.21.15.43:0/967828326 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb1b8031880 con 0x7fb1bc15f7b0 2023-08-09T13:36:04.676 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:04.670+0000 7fb1b37fe640 1 -- 172.21.15.43:0/967828326 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb1b80319e0 con 0x7fb1bc15f7b0 2023-08-09T13:36:04.676 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:04.670+0000 7fb1b37fe640 1 -- 172.21.15.43:0/967828326 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb1b8031d70 con 0x7fb1bc15f7b0 2023-08-09T13:36:04.677 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:04.671+0000 7fb1c1fa4640 1 -- 172.21.15.43:0/967828326 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb1bc15f7b0 msgr2=0x7fb1bc0b4760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:36:04.677 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:04.671+0000 7fb1c1fa4640 1 --2- 172.21.15.43:0/967828326 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb1bc15f7b0 0x7fb1bc0b4760 secure :-1 s=READY pgs=533 cs=0 l=1 rev1=1 crypto rx=0x7fb1bc0ce9d0 tx=0x7fb1b802b920 comp rx=0 tx=0).stop 2023-08-09T13:36:04.677 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:04.671+0000 7fb1c1fa4640 1 -- 172.21.15.43:0/967828326 shutdown_connections 2023-08-09T13:36:04.677 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:04.671+0000 7fb1c1fa4640 1 --2- 172.21.15.43:0/967828326 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fb1bc0cd4e0 0x7fb1bc0ceae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:36:04.677 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:04.671+0000 7fb1c1fa4640 1 --2- 172.21.15.43:0/967828326 >> v2:172.21.15.203:3300/0 conn(0x7fb1bc0b1050 0x7fb1bc0ccfa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:36:04.677 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:04.671+0000 7fb1c1fa4640 1 --2- 172.21.15.43:0/967828326 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb1bc15f7b0 0x7fb1bc0b4760 unknown :-1 s=CLOSED pgs=533 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:36:04.677 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:04.671+0000 7fb1c1fa4640 1 -- 172.21.15.43:0/967828326 >> 172.21.15.43:0/967828326 conn(0x7fb1bc06a2b0 msgr2=0x7fb1bc06a690 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-09T13:36:04.677 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:04.671+0000 7fb1c1fa4640 1 -- 172.21.15.43:0/967828326 shutdown_connections 2023-08-09T13:36:04.678 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:04.671+0000 7fb1c1fa4640 1 -- 172.21.15.43:0/967828326 wait complete. 2023-08-09T13:36:04.678 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:04.673+0000 7fb1c1fa4640 1 Processor -- start 2023-08-09T13:36:04.678 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:04.673+0000 7fb1c1fa4640 1 -- start start 2023-08-09T13:36:04.679 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:04.674+0000 7fb1c1fa4640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb1bc0b1050 0x7fb1bc153880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:36:04.680 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:04.674+0000 7fb1c1fa4640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fb1bc0cd4e0 0x7fb1bc15c290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:36:04.680 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:04.674+0000 7fb1c1fa4640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fb1bc15f7b0 0x7fb1bc0c6b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:36:04.680 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:04.674+0000 7fb1c1fa4640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb1bc15c7d0 con 0x7fb1bc0b1050 2023-08-09T13:36:04.680 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:04.674+0000 7fb1c1fa4640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb1bc0b2a90 con 0x7fb1bc15f7b0 2023-08-09T13:36:04.680 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:04.674+0000 7fb1c1fa4640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb1bc1f1b00 con 0x7fb1bc0cd4e0 2023-08-09T13:36:04.680 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:04.674+0000 7fb1c0fa2640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb1bc0b1050 0x7fb1bc153880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:36:04.681 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:04.674+0000 7fb1c0fa2640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb1bc0b1050 0x7fb1bc153880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:46192/0 (socket says 172.21.15.43:46192) 2023-08-09T13:36:04.681 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:04.674+0000 7fb1c0fa2640 1 -- 172.21.15.43:0/1734033781 learned_addr learned my addr 172.21.15.43:0/1734033781 (peer_addr_for_me v2:172.21.15.43:0/0) 2023-08-09T13:36:04.681 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:04.675+0000 7fb1b3fff640 1 --2- 172.21.15.43:0/1734033781 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fb1bc0cd4e0 0x7fb1bc15c290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:36:04.681 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:04.675+0000 7fb1c17a3640 1 --2- 172.21.15.43:0/1734033781 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fb1bc15f7b0 0x7fb1bc0c6b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:36:04.681 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:04.675+0000 7fb1c0fa2640 1 -- 172.21.15.43:0/1734033781 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fb1bc0cd4e0 msgr2=0x7fb1bc15c290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:36:04.681 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:04.675+0000 7fb1c0fa2640 1 --2- 172.21.15.43:0/1734033781 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fb1bc0cd4e0 0x7fb1bc15c290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:36:04.681 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:04.675+0000 7fb1c0fa2640 1 -- 172.21.15.43:0/1734033781 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fb1bc15f7b0 msgr2=0x7fb1bc0c6b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:36:04.682 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:04.675+0000 7fb1c0fa2640 1 --2- 172.21.15.43:0/1734033781 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fb1bc15f7b0 0x7fb1bc0c6b60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:36:04.682 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:04.675+0000 7fb1c0fa2640 1 -- 172.21.15.43:0/1734033781 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb1b80094d0 con 0x7fb1bc0b1050 2023-08-09T13:36:04.682 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:04.675+0000 7fb1b3fff640 1 --2- 172.21.15.43:0/1734033781 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fb1bc0cd4e0 0x7fb1bc15c290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-08-09T13:36:04.682 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:04.675+0000 7fb1c17a3640 1 --2- 172.21.15.43:0/1734033781 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fb1bc15f7b0 0x7fb1bc0c6b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-08-09T13:36:04.682 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:04.675+0000 7fb1c0fa2640 1 --2- 172.21.15.43:0/1734033781 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb1bc0b1050 0x7fb1bc153880 secure :-1 s=READY pgs=534 cs=0 l=1 rev1=1 crypto rx=0x7fb1b802b960 tx=0x7fb1b80046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-09T13:36:04.682 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:04.676+0000 7fb1b17fa640 1 -- 172.21.15.43:0/1734033781 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb1b8031870 con 0x7fb1bc0b1050 2023-08-09T13:36:04.682 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:04.676+0000 7fb1b17fa640 1 -- 172.21.15.43:0/1734033781 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb1b80443f0 con 0x7fb1bc0b1050 2023-08-09T13:36:04.682 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:04.676+0000 7fb1b17fa640 1 -- 172.21.15.43:0/1734033781 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb1b80447a0 con 0x7fb1bc0b1050 2023-08-09T13:36:04.683 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:04.676+0000 7fb1c1fa4640 1 -- 172.21.15.43:0/1734033781 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb1bc1f1de0 con 0x7fb1bc0b1050 2023-08-09T13:36:04.683 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:04.676+0000 7fb1c1fa4640 1 -- 172.21.15.43:0/1734033781 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb1bc1f1810 con 0x7fb1bc0b1050 2023-08-09T13:36:04.683 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:04.677+0000 7fb1c1fa4640 1 -- 172.21.15.43:0/1734033781 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb1bc1f1c40 con 0x7fb1bc0b1050 2023-08-09T13:36:04.683 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:04.678+0000 7fb1b17fa640 1 -- 172.21.15.43:0/1734033781 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 5) v1 ==== 88341+0+0 (secure 0 0 0) 0x7fb1b804d450 con 0x7fb1bc0b1050 2023-08-09T13:36:04.684 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:04.679+0000 7fb1b17fa640 1 --2- 172.21.15.43:0/1734033781 >> [v2:172.21.15.203:6832/830097474,v1:172.21.15.203:6833/830097474] conn(0x7fb1a806a050 0x7fb1a806c510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:36:04.684 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:04.679+0000 7fb1b17fa640 1 -- 172.21.15.43:0/1734033781 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(35..35 src has 1..35) v4 ==== 5530+0+0 (secure 0 0 0) 0x7fb1b80aabf0 con 0x7fb1bc0b1050 2023-08-09T13:36:04.684 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:04.680+0000 7fb1b3fff640 1 --2- 172.21.15.43:0/1734033781 >> [v2:172.21.15.203:6832/830097474,v1:172.21.15.203:6833/830097474] conn(0x7fb1a806a050 0x7fb1a806c510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:36:04.684 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:04.681+0000 7fb1b3fff640 1 --2- 172.21.15.43:0/1734033781 >> [v2:172.21.15.203:6832/830097474,v1:172.21.15.203:6833/830097474] conn(0x7fb1a806a050 0x7fb1a806c510 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7fb1bc0d2eb0 tx=0x7fb1ac008040 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-09T13:36:04.685 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:04.683+0000 7fb1b17fa640 1 -- 172.21.15.43:0/1734033781 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178066 (secure 0 0 0) 0x7fb1bc1f1c40 con 0x7fb1bc0b1050 2023-08-09T13:36:04.956 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:04.954+0000 7fb1c1fa4640 1 -- 172.21.15.43:0/1734033781 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "auth del", "entity": "client.bootstrap-osd"} v 0) v1 -- 0x7fb1bc1f1c40 con 0x7fb1bc0b1050 2023-08-09T13:36:04.964 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:04.961+0000 7fb1b17fa640 1 -- 172.21.15.43:0/1734033781 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "auth del", "entity": "client.bootstrap-osd"}]=0 v6) v1 ==== 90+0+0 (secure 0 0 0) 0x7fb1bc1f1c40 con 0x7fb1bc0b1050 2023-08-09T13:36:04.967 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:04.964+0000 7fb1c1fa4640 1 -- 172.21.15.43:0/1734033781 >> [v2:172.21.15.203:6832/830097474,v1:172.21.15.203:6833/830097474] conn(0x7fb1a806a050 msgr2=0x7fb1a806c510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:36:04.968 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:04.964+0000 7fb1c1fa4640 1 --2- 172.21.15.43:0/1734033781 >> [v2:172.21.15.203:6832/830097474,v1:172.21.15.203:6833/830097474] conn(0x7fb1a806a050 0x7fb1a806c510 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7fb1bc0d2eb0 tx=0x7fb1ac008040 comp rx=0 tx=0).stop 2023-08-09T13:36:04.968 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:04.964+0000 7fb1c1fa4640 1 -- 172.21.15.43:0/1734033781 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb1bc0b1050 msgr2=0x7fb1bc153880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:36:04.968 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:04.964+0000 7fb1c1fa4640 1 --2- 172.21.15.43:0/1734033781 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb1bc0b1050 0x7fb1bc153880 secure :-1 s=READY pgs=534 cs=0 l=1 rev1=1 crypto rx=0x7fb1b802b960 tx=0x7fb1b80046c0 comp rx=0 tx=0).stop 2023-08-09T13:36:04.968 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:04.965+0000 7fb1c1fa4640 1 -- 172.21.15.43:0/1734033781 shutdown_connections 2023-08-09T13:36:04.969 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:04.965+0000 7fb1c1fa4640 1 --2- 172.21.15.43:0/1734033781 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fb1bc15f7b0 0x7fb1bc0c6b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:36:04.969 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:04.965+0000 7fb1c1fa4640 1 --2- 172.21.15.43:0/1734033781 >> [v2:172.21.15.203:6832/830097474,v1:172.21.15.203:6833/830097474] conn(0x7fb1a806a050 0x7fb1a806c510 unknown :-1 s=CLOSED pgs=260 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:36:04.969 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:04.965+0000 7fb1c1fa4640 1 --2- 172.21.15.43:0/1734033781 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fb1bc0cd4e0 0x7fb1bc15c290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:36:04.970 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:04.965+0000 7fb1c1fa4640 1 --2- 172.21.15.43:0/1734033781 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb1bc0b1050 0x7fb1bc153880 unknown :-1 s=CLOSED pgs=534 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:36:04.970 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:04.965+0000 7fb1c1fa4640 1 -- 172.21.15.43:0/1734033781 >> 172.21.15.43:0/1734033781 conn(0x7fb1bc06a2b0 msgr2=0x7fb1bc0c8640 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-09T13:36:04.970 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:04.965+0000 7fb1c1fa4640 1 -- 172.21.15.43:0/1734033781 shutdown_connections 2023-08-09T13:36:04.970 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:04.965+0000 7fb1c1fa4640 1 -- 172.21.15.43:0/1734033781 wait complete. 2023-08-09T13:36:04.985 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth del client.bootstrap-rbd 2023-08-09T13:36:05.215 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:05.213+0000 7f0c59ac5640 1 Processor -- start 2023-08-09T13:36:05.215 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:05.213+0000 7f0c59ac5640 1 -- start start 2023-08-09T13:36:05.217 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:05.214+0000 7f0c59ac5640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f0c540c86f0 0x7f0c540c8b50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:36:05.218 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:05.215+0000 7f0c59ac5640 1 -- --> v1:172.21.15.43:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0c540c9090 con 0x7f0c5406c8d0 2023-08-09T13:36:05.218 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:05.215+0000 7f0c59ac5640 1 -- --> v1:172.21.15.203:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0c5406c440 con 0x7f0c54143c20 2023-08-09T13:36:05.218 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:05.215+0000 7f0c59ac5640 1 -- --> v2:172.21.15.43:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0c54144000 con 0x7f0c540c86f0 2023-08-09T13:36:05.218 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:05.215+0000 7f0c52ffd640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f0c540c86f0 0x7f0c540c8b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:36:05.219 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:05.215+0000 7f0c53fff640 1 --1- >> v1:172.21.15.43:6789/0 conn(0x7f0c5406c8d0 0x7f0c5406cd00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.43:6789/0 says I am v1:172.21.15.43:39424/0 (socket says 172.21.15.43:39424) 2023-08-09T13:36:05.219 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:05.215+0000 7f0c53fff640 1 -- 172.21.15.43:0/1680464779 learned_addr learned my addr 172.21.15.43:0/1680464779 (peer_addr_for_me v1:172.21.15.43:0/0) 2023-08-09T13:36:05.219 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:05.215+0000 7f0c52ffd640 1 --2- 172.21.15.43:0/1680464779 >> v2:172.21.15.43:3300/0 conn(0x7f0c540c86f0 0x7f0c540c8b50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:35790/0 (socket says 172.21.15.43:35790) 2023-08-09T13:36:05.220 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:05.215+0000 7f0c52ffd640 1 -- 172.21.15.43:0/1680464779 >> v1:172.21.15.203:6789/0 conn(0x7f0c54143c20 legacy=0x7f0c5406f540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-09T13:36:05.220 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:05.215+0000 7f0c52ffd640 1 -- 172.21.15.43:0/1680464779 >> v1:172.21.15.43:6789/0 conn(0x7f0c5406c8d0 legacy=0x7f0c5406cd00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:36:05.220 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:05.215+0000 7f0c52ffd640 1 -- 172.21.15.43:0/1680464779 --> v2:172.21.15.43:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0c5406fc50 con 0x7f0c540c86f0 2023-08-09T13:36:05.220 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:05.217+0000 7f0c52ffd640 1 --2- 172.21.15.43:0/1680464779 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f0c540c86f0 0x7f0c540c8b50 secure :-1 s=READY pgs=536 cs=0 l=1 rev1=1 crypto rx=0x7f0c480097a0 tx=0x7f0c4802b7a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=380b266f67e4e784 server_cookie=0 in_seq=0 out_seq=0 2023-08-09T13:36:05.220 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:05.217+0000 7f0c527fc640 1 -- 172.21.15.43:0/1680464779 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0c4802f050 con 0x7f0c540c86f0 2023-08-09T13:36:05.221 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:05.217+0000 7f0c527fc640 1 -- 172.21.15.43:0/1680464779 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0c48031700 con 0x7f0c540c86f0 2023-08-09T13:36:05.221 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:05.217+0000 7f0c527fc640 1 -- 172.21.15.43:0/1680464779 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0c48031a90 con 0x7f0c540c86f0 2023-08-09T13:36:05.221 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:05.218+0000 7f0c59ac5640 1 -- 172.21.15.43:0/1680464779 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f0c540c86f0 msgr2=0x7f0c540c8b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:36:05.222 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:05.218+0000 7f0c59ac5640 1 --2- 172.21.15.43:0/1680464779 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f0c540c86f0 0x7f0c540c8b50 secure :-1 s=READY pgs=536 cs=0 l=1 rev1=1 crypto rx=0x7f0c480097a0 tx=0x7f0c4802b7a0 comp rx=0 tx=0).stop 2023-08-09T13:36:05.222 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:05.218+0000 7f0c59ac5640 1 -- 172.21.15.43:0/1680464779 shutdown_connections 2023-08-09T13:36:05.222 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:05.218+0000 7f0c59ac5640 1 --2- 172.21.15.43:0/1680464779 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f0c540c86f0 0x7f0c540c8b50 unknown :-1 s=CLOSED pgs=536 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:36:05.222 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:05.218+0000 7f0c59ac5640 1 -- 172.21.15.43:0/1680464779 >> 172.21.15.43:0/1680464779 conn(0x7f0c540ce0a0 msgr2=0x7f0c540ce480 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-09T13:36:05.222 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:05.218+0000 7f0c59ac5640 1 -- 172.21.15.43:0/1680464779 shutdown_connections 2023-08-09T13:36:05.222 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:05.218+0000 7f0c59ac5640 1 -- 172.21.15.43:0/1680464779 wait complete. 2023-08-09T13:36:05.223 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:05.220+0000 7f0c59ac5640 1 Processor -- start 2023-08-09T13:36:05.223 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:05.221+0000 7f0c59ac5640 1 -- start start 2023-08-09T13:36:05.225 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:05.222+0000 7f0c59ac5640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f0c5406c8d0 0x7f0c5415cc90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:36:05.225 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:05.222+0000 7f0c59ac5640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f0c540c86f0 0x7f0c54134980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:36:05.225 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:05.222+0000 7f0c59ac5640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f0c54143c20 0x7f0c540b2f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:36:05.225 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:05.222+0000 7f0c59ac5640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0c540a8980 con 0x7f0c5406c8d0 2023-08-09T13:36:05.225 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:05.222+0000 7f0c59ac5640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0c541e3c00 con 0x7f0c54143c20 2023-08-09T13:36:05.225 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:05.222+0000 7f0c59ac5640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0c541e3ee0 con 0x7f0c540c86f0 2023-08-09T13:36:05.226 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:05.222+0000 7f0c537fe640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f0c5406c8d0 0x7f0c5415cc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:36:05.226 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:05.222+0000 7f0c537fe640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f0c5406c8d0 0x7f0c5415cc90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:35796/0 (socket says 172.21.15.43:35796) 2023-08-09T13:36:05.226 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:05.222+0000 7f0c537fe640 1 -- 172.21.15.43:0/4047519040 learned_addr learned my addr 172.21.15.43:0/4047519040 (peer_addr_for_me v2:172.21.15.43:0/0) 2023-08-09T13:36:05.227 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:05.223+0000 7f0c52ffd640 1 --2- 172.21.15.43:0/4047519040 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f0c540c86f0 0x7f0c54134980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:36:05.227 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:05.223+0000 7f0c537fe640 1 -- 172.21.15.43:0/4047519040 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f0c540c86f0 msgr2=0x7f0c54134980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:36:05.228 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:05.223+0000 7f0c537fe640 1 --2- 172.21.15.43:0/4047519040 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f0c540c86f0 0x7f0c54134980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:36:05.228 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:05.223+0000 7f0c53fff640 1 --2- 172.21.15.43:0/4047519040 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f0c54143c20 0x7f0c540b2f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:36:05.228 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:05.223+0000 7f0c537fe640 1 -- 172.21.15.43:0/4047519040 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f0c54143c20 msgr2=0x7f0c540b2f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:36:05.228 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:05.223+0000 7f0c537fe640 1 --2- 172.21.15.43:0/4047519040 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f0c54143c20 0x7f0c540b2f50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:36:05.228 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:05.223+0000 7f0c537fe640 1 -- 172.21.15.43:0/4047519040 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0c48009400 con 0x7f0c5406c8d0 2023-08-09T13:36:05.228 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:05.223+0000 7f0c53fff640 1 --2- 172.21.15.43:0/4047519040 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f0c54143c20 0x7f0c540b2f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-08-09T13:36:05.229 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:05.223+0000 7f0c537fe640 1 --2- 172.21.15.43:0/4047519040 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f0c5406c8d0 0x7f0c5415cc90 secure :-1 s=READY pgs=537 cs=0 l=1 rev1=1 crypto rx=0x7f0c4400c7c0 tx=0x7f0c4400cc80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-09T13:36:05.229 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:05.224+0000 7f0c3bfff640 1 -- 172.21.15.43:0/4047519040 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0c4400aa50 con 0x7f0c5406c8d0 2023-08-09T13:36:05.229 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:05.224+0000 7f0c3bfff640 1 -- 172.21.15.43:0/4047519040 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0c4400abb0 con 0x7f0c5406c8d0 2023-08-09T13:36:05.229 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:05.224+0000 7f0c3bfff640 1 -- 172.21.15.43:0/4047519040 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0c44012420 con 0x7f0c5406c8d0 2023-08-09T13:36:05.229 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:05.224+0000 7f0c59ac5640 1 -- 172.21.15.43:0/4047519040 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0c541e41c0 con 0x7f0c5406c8d0 2023-08-09T13:36:05.229 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:05.224+0000 7f0c59ac5640 1 -- 172.21.15.43:0/4047519040 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0c541e4780 con 0x7f0c5406c8d0 2023-08-09T13:36:05.229 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:05.225+0000 7f0c59ac5640 1 -- 172.21.15.43:0/4047519040 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0c540739f0 con 0x7f0c5406c8d0 2023-08-09T13:36:05.230 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:05.226+0000 7f0c3bfff640 1 -- 172.21.15.43:0/4047519040 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 5) v1 ==== 88341+0+0 (secure 0 0 0) 0x7f0c44012580 con 0x7f0c5406c8d0 2023-08-09T13:36:05.230 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:05.227+0000 7f0c3bfff640 1 --2- 172.21.15.43:0/4047519040 >> [v2:172.21.15.203:6832/830097474,v1:172.21.15.203:6833/830097474] conn(0x7f0c4006a050 0x7f0c4006c510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:36:05.230 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:05.227+0000 7f0c3bfff640 1 -- 172.21.15.43:0/4047519040 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(35..35 src has 1..35) v4 ==== 5530+0+0 (secure 0 0 0) 0x7f0c440883d0 con 0x7f0c5406c8d0 2023-08-09T13:36:05.230 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:05.228+0000 7f0c52ffd640 1 --2- 172.21.15.43:0/4047519040 >> [v2:172.21.15.203:6832/830097474,v1:172.21.15.203:6833/830097474] conn(0x7f0c4006a050 0x7f0c4006c510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:36:05.231 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:05.229+0000 7f0c52ffd640 1 --2- 172.21.15.43:0/4047519040 >> [v2:172.21.15.203:6832/830097474,v1:172.21.15.203:6833/830097474] conn(0x7f0c4006a050 0x7f0c4006c510 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7f0c48035000 tx=0x7f0c48008030 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-09T13:36:05.233 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:05.231+0000 7f0c3bfff640 1 -- 172.21.15.43:0/4047519040 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178066 (secure 0 0 0) 0x7f0c44053c30 con 0x7f0c5406c8d0 2023-08-09T13:36:05.497 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:05.494+0000 7f0c59ac5640 1 -- 172.21.15.43:0/4047519040 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "auth del", "entity": "client.bootstrap-rbd"} v 0) v1 -- 0x7f0c540cec90 con 0x7f0c5406c8d0 2023-08-09T13:36:05.505 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:05.502+0000 7f0c3bfff640 1 -- 172.21.15.43:0/4047519040 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "auth del", "entity": "client.bootstrap-rbd"}]=0 v7) v1 ==== 90+0+0 (secure 0 0 0) 0x7f0c44057250 con 0x7f0c5406c8d0 2023-08-09T13:36:05.508 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:05.505+0000 7f0c59ac5640 1 -- 172.21.15.43:0/4047519040 >> [v2:172.21.15.203:6832/830097474,v1:172.21.15.203:6833/830097474] conn(0x7f0c4006a050 msgr2=0x7f0c4006c510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:36:05.508 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:05.505+0000 7f0c59ac5640 1 --2- 172.21.15.43:0/4047519040 >> [v2:172.21.15.203:6832/830097474,v1:172.21.15.203:6833/830097474] conn(0x7f0c4006a050 0x7f0c4006c510 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7f0c48035000 tx=0x7f0c48008030 comp rx=0 tx=0).stop 2023-08-09T13:36:05.508 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:05.505+0000 7f0c59ac5640 1 -- 172.21.15.43:0/4047519040 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f0c5406c8d0 msgr2=0x7f0c5415cc90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:36:05.509 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:05.505+0000 7f0c59ac5640 1 --2- 172.21.15.43:0/4047519040 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f0c5406c8d0 0x7f0c5415cc90 secure :-1 s=READY pgs=537 cs=0 l=1 rev1=1 crypto rx=0x7f0c4400c7c0 tx=0x7f0c4400cc80 comp rx=0 tx=0).stop 2023-08-09T13:36:05.509 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:05.505+0000 7f0c59ac5640 1 -- 172.21.15.43:0/4047519040 shutdown_connections 2023-08-09T13:36:05.509 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:05.505+0000 7f0c59ac5640 1 --2- 172.21.15.43:0/4047519040 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f0c54143c20 0x7f0c540b2f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:36:05.510 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:05.505+0000 7f0c59ac5640 1 --2- 172.21.15.43:0/4047519040 >> [v2:172.21.15.203:6832/830097474,v1:172.21.15.203:6833/830097474] conn(0x7f0c4006a050 0x7f0c4006c510 unknown :-1 s=CLOSED pgs=261 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:36:05.510 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:05.505+0000 7f0c59ac5640 1 --2- 172.21.15.43:0/4047519040 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f0c540c86f0 0x7f0c54134980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:36:05.510 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:05.505+0000 7f0c59ac5640 1 --2- 172.21.15.43:0/4047519040 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f0c5406c8d0 0x7f0c5415cc90 unknown :-1 s=CLOSED pgs=537 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:36:05.510 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:05.505+0000 7f0c59ac5640 1 -- 172.21.15.43:0/4047519040 >> 172.21.15.43:0/4047519040 conn(0x7f0c540ce0a0 msgr2=0x7f0c540af080 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-09T13:36:05.511 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:05.505+0000 7f0c59ac5640 1 -- 172.21.15.43:0/4047519040 shutdown_connections 2023-08-09T13:36:05.511 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:05.505+0000 7f0c59ac5640 1 -- 172.21.15.43:0/4047519040 wait complete. 2023-08-09T13:36:05.527 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth del client.bootstrap-rbd-mirror 2023-08-09T13:36:05.763 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:05.760+0000 7f6f23fff640 1 Processor -- start 2023-08-09T13:36:05.763 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:05.760+0000 7f6f23fff640 1 -- start start 2023-08-09T13:36:05.764 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:05.762+0000 7f6f23fff640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f6f241546f0 0x7f6f2415e640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:36:05.765 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:05.762+0000 7f6f23fff640 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7f6f2415b460 0x7f6f2406b3d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:36:05.765 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:05.762+0000 7f6f23fff640 1 -- --> v1:172.21.15.203:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6f2406d770 con 0x7f6f24070ab0 2023-08-09T13:36:05.765 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:05.762+0000 7f6f23fff640 1 -- --> v2:172.21.15.203:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6f2406cf30 con 0x7f6f2415b460 2023-08-09T13:36:05.765 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:05.762+0000 7f6f23fff640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6f2406d1b0 con 0x7f6f241546f0 2023-08-09T13:36:05.766 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:05.763+0000 7f6f22ffd640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f6f241546f0 0x7f6f2415e640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:36:05.766 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:05.763+0000 7f6f22ffd640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f6f241546f0 0x7f6f2415e640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.203:3301/0 says I am v2:172.21.15.43:56228/0 (socket says 172.21.15.43:56228) 2023-08-09T13:36:05.766 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:05.763+0000 7f6f22ffd640 1 -- 172.21.15.43:0/3101127933 learned_addr learned my addr 172.21.15.43:0/3101127933 (peer_addr_for_me v2:172.21.15.43:0/0) 2023-08-09T13:36:05.766 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:05.763+0000 7f6f1bfff640 1 --2- 172.21.15.43:0/3101127933 >> v2:172.21.15.203:3300/0 conn(0x7f6f2415b460 0x7f6f2406b3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:36:05.767 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:05.764+0000 7f6f22ffd640 1 -- 172.21.15.43:0/3101127933 >> v2:172.21.15.203:3300/0 conn(0x7f6f2415b460 msgr2=0x7f6f2406b3d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-09T13:36:05.767 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:05.764+0000 7f6f22ffd640 1 --2- 172.21.15.43:0/3101127933 >> v2:172.21.15.203:3300/0 conn(0x7f6f2415b460 0x7f6f2406b3d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:36:05.767 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:05.764+0000 7f6f22ffd640 1 -- 172.21.15.43:0/3101127933 >> v1:172.21.15.203:6789/0 conn(0x7f6f24070ab0 legacy=0x7f6f240c84b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:36:05.768 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:05.764+0000 7f6f22ffd640 1 -- 172.21.15.43:0/3101127933 --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6f2406d490 con 0x7f6f241546f0 2023-08-09T13:36:05.768 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:05.764+0000 7f6f1bfff640 1 --2- 172.21.15.43:0/3101127933 >> v2:172.21.15.203:3300/0 conn(0x7f6f2415b460 0x7f6f2406b3d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-08-09T13:36:05.768 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:05.765+0000 7f6f22ffd640 1 --2- 172.21.15.43:0/3101127933 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f6f241546f0 0x7f6f2415e640 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f6f14009870 tx=0x7f6f1402b8f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f8f17eb889aadc52 server_cookie=0 in_seq=0 out_seq=0 2023-08-09T13:36:05.769 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:05.766+0000 7f6f227fc640 1 -- 172.21.15.43:0/3101127933 <== mon.2 v2:172.21.15.203:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6f14031640 con 0x7f6f241546f0 2023-08-09T13:36:05.769 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:05.766+0000 7f6f227fc640 1 -- 172.21.15.43:0/3101127933 <== mon.2 v2:172.21.15.203:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6f140317a0 con 0x7f6f241546f0 2023-08-09T13:36:05.769 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:05.766+0000 7f6f227fc640 1 -- 172.21.15.43:0/3101127933 <== mon.2 v2:172.21.15.203:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6f14031b30 con 0x7f6f241546f0 2023-08-09T13:36:05.769 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:05.766+0000 7f6f23fff640 1 -- 172.21.15.43:0/3101127933 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f6f241546f0 msgr2=0x7f6f2415e640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:36:05.769 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:05.766+0000 7f6f23fff640 1 --2- 172.21.15.43:0/3101127933 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f6f241546f0 0x7f6f2415e640 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f6f14009870 tx=0x7f6f1402b8f0 comp rx=0 tx=0).stop 2023-08-09T13:36:05.770 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:05.766+0000 7f6f23fff640 1 -- 172.21.15.43:0/3101127933 shutdown_connections 2023-08-09T13:36:05.770 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:05.766+0000 7f6f23fff640 1 --2- 172.21.15.43:0/3101127933 >> v2:172.21.15.203:3300/0 conn(0x7f6f2415b460 0x7f6f2406b3d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:36:05.770 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:05.766+0000 7f6f23fff640 1 --2- 172.21.15.43:0/3101127933 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f6f241546f0 0x7f6f2415e640 unknown :-1 s=CLOSED pgs=76 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:36:05.771 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:05.766+0000 7f6f23fff640 1 -- 172.21.15.43:0/3101127933 >> 172.21.15.43:0/3101127933 conn(0x7f6f240af950 msgr2=0x7f6f240afd60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-09T13:36:05.771 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:05.767+0000 7f6f23fff640 1 -- 172.21.15.43:0/3101127933 shutdown_connections 2023-08-09T13:36:05.771 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:05.767+0000 7f6f23fff640 1 -- 172.21.15.43:0/3101127933 wait complete. 2023-08-09T13:36:05.771 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:05.769+0000 7f6f23fff640 1 Processor -- start 2023-08-09T13:36:05.772 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:05.769+0000 7f6f23fff640 1 -- start start 2023-08-09T13:36:05.773 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:05.771+0000 7f6f23fff640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f6f24070ab0 0x7f6f241424f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:36:05.773 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:05.771+0000 7f6f23fff640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f6f241546f0 0x7f6f241f4f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:36:05.774 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:05.771+0000 7f6f23fff640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f6f2413f330 0x7f6f241f0fd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:36:05.774 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:05.771+0000 7f6f23fff640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6f2413eea0 con 0x7f6f24070ab0 2023-08-09T13:36:05.774 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:05.771+0000 7f6f23fff640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6f241f57b0 con 0x7f6f241546f0 2023-08-09T13:36:05.774 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:05.771+0000 7f6f23fff640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6f241f5a90 con 0x7f6f2413f330 2023-08-09T13:36:05.774 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:05.771+0000 7f6f22ffd640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f6f24070ab0 0x7f6f241424f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:36:05.774 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:05.771+0000 7f6f22ffd640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f6f24070ab0 0x7f6f241424f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:35806/0 (socket says 172.21.15.43:35806) 2023-08-09T13:36:05.774 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:05.771+0000 7f6f22ffd640 1 -- 172.21.15.43:0/3539917595 learned_addr learned my addr 172.21.15.43:0/3539917595 (peer_addr_for_me v2:172.21.15.43:0/0) 2023-08-09T13:36:05.776 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:05.771+0000 7f6f237fe640 1 --2- 172.21.15.43:0/3539917595 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f6f2413f330 0x7f6f241f0fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:36:05.776 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:05.772+0000 7f6f1bfff640 1 --2- 172.21.15.43:0/3539917595 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f6f241546f0 0x7f6f241f4f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:36:05.776 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:05.772+0000 7f6f22ffd640 1 -- 172.21.15.43:0/3539917595 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f6f2413f330 msgr2=0x7f6f241f0fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:36:05.776 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:05.772+0000 7f6f22ffd640 1 --2- 172.21.15.43:0/3539917595 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f6f2413f330 0x7f6f241f0fd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:36:05.776 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:05.772+0000 7f6f22ffd640 1 -- 172.21.15.43:0/3539917595 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f6f241546f0 msgr2=0x7f6f241f4f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:36:05.776 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:05.772+0000 7f6f22ffd640 1 --2- 172.21.15.43:0/3539917595 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f6f241546f0 0x7f6f241f4f80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:36:05.777 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:05.772+0000 7f6f22ffd640 1 -- 172.21.15.43:0/3539917595 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6f0c009400 con 0x7f6f24070ab0 2023-08-09T13:36:05.777 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:05.772+0000 7f6f1bfff640 1 --2- 172.21.15.43:0/3539917595 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f6f241546f0 0x7f6f241f4f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-08-09T13:36:05.777 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:05.772+0000 7f6f22ffd640 1 --2- 172.21.15.43:0/3539917595 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f6f24070ab0 0x7f6f241424f0 secure :-1 s=READY pgs=538 cs=0 l=1 rev1=1 crypto rx=0x7f6f14000c00 tx=0x7f6f140046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-09T13:36:05.777 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:05.772+0000 7f6f1b7fe640 1 -- 172.21.15.43:0/3539917595 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6f1403a930 con 0x7f6f24070ab0 2023-08-09T13:36:05.777 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:05.772+0000 7f6f1b7fe640 1 -- 172.21.15.43:0/3539917595 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6f1403aa90 con 0x7f6f24070ab0 2023-08-09T13:36:05.777 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:05.772+0000 7f6f1b7fe640 1 -- 172.21.15.43:0/3539917595 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6f1403ae40 con 0x7f6f24070ab0 2023-08-09T13:36:05.778 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:05.772+0000 7f6f23fff640 1 -- 172.21.15.43:0/3539917595 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6f140094d0 con 0x7f6f24070ab0 2023-08-09T13:36:05.778 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:05.773+0000 7f6f23fff640 1 -- 172.21.15.43:0/3539917595 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6f2413eb30 con 0x7f6f24070ab0 2023-08-09T13:36:05.778 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:05.774+0000 7f6f23fff640 1 -- 172.21.15.43:0/3539917595 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6f240739f0 con 0x7f6f24070ab0 2023-08-09T13:36:05.779 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:05.775+0000 7f6f1b7fe640 1 -- 172.21.15.43:0/3539917595 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 5) v1 ==== 88341+0+0 (secure 0 0 0) 0x7f6f1402f050 con 0x7f6f24070ab0 2023-08-09T13:36:05.779 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:05.776+0000 7f6f1b7fe640 1 --2- 172.21.15.43:0/3539917595 >> [v2:172.21.15.203:6832/830097474,v1:172.21.15.203:6833/830097474] conn(0x7f6f0406a080 0x7f6f0406c540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:36:05.779 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:05.776+0000 7f6f1b7fe640 1 -- 172.21.15.43:0/3539917595 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(35..35 src has 1..35) v4 ==== 5530+0+0 (secure 0 0 0) 0x7f6f140aa180 con 0x7f6f24070ab0 2023-08-09T13:36:05.779 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:05.777+0000 7f6f1bfff640 1 --2- 172.21.15.43:0/3539917595 >> [v2:172.21.15.203:6832/830097474,v1:172.21.15.203:6833/830097474] conn(0x7f6f0406a080 0x7f6f0406c540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:36:05.780 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:05.778+0000 7f6f1bfff640 1 --2- 172.21.15.43:0/3539917595 >> [v2:172.21.15.203:6832/830097474,v1:172.21.15.203:6833/830097474] conn(0x7f6f0406a080 0x7f6f0406c540 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7f6f240b3910 tx=0x7f6f0c008040 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-09T13:36:05.783 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:05.781+0000 7f6f1b7fe640 1 -- 172.21.15.43:0/3539917595 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178066 (secure 0 0 0) 0x7f6f140759f0 con 0x7f6f24070ab0 2023-08-09T13:36:06.048 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:06.045+0000 7f6f23fff640 1 -- 172.21.15.43:0/3539917595 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "auth del", "entity": "client.bootstrap-rbd-mirror"} v 0) v1 -- 0x7f6f2413ecc0 con 0x7f6f24070ab0 2023-08-09T13:36:06.056 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:06.053+0000 7f6f1b7fe640 1 -- 172.21.15.43:0/3539917595 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "auth del", "entity": "client.bootstrap-rbd-mirror"}]=0 v8) v1 ==== 97+0+0 (secure 0 0 0) 0x7f6f14079010 con 0x7f6f24070ab0 2023-08-09T13:36:06.058 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:06.055+0000 7f6f23fff640 1 -- 172.21.15.43:0/3539917595 >> [v2:172.21.15.203:6832/830097474,v1:172.21.15.203:6833/830097474] conn(0x7f6f0406a080 msgr2=0x7f6f0406c540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:36:06.058 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:06.055+0000 7f6f23fff640 1 --2- 172.21.15.43:0/3539917595 >> [v2:172.21.15.203:6832/830097474,v1:172.21.15.203:6833/830097474] conn(0x7f6f0406a080 0x7f6f0406c540 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7f6f240b3910 tx=0x7f6f0c008040 comp rx=0 tx=0).stop 2023-08-09T13:36:06.058 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:06.055+0000 7f6f23fff640 1 -- 172.21.15.43:0/3539917595 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f6f24070ab0 msgr2=0x7f6f241424f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:36:06.058 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:06.055+0000 7f6f23fff640 1 --2- 172.21.15.43:0/3539917595 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f6f24070ab0 0x7f6f241424f0 secure :-1 s=READY pgs=538 cs=0 l=1 rev1=1 crypto rx=0x7f6f14000c00 tx=0x7f6f140046c0 comp rx=0 tx=0).stop 2023-08-09T13:36:06.059 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:06.056+0000 7f6f23fff640 1 -- 172.21.15.43:0/3539917595 shutdown_connections 2023-08-09T13:36:06.059 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:06.056+0000 7f6f23fff640 1 --2- 172.21.15.43:0/3539917595 >> [v2:172.21.15.203:6832/830097474,v1:172.21.15.203:6833/830097474] conn(0x7f6f0406a080 0x7f6f0406c540 unknown :-1 s=CLOSED pgs=262 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:36:06.059 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:06.056+0000 7f6f23fff640 1 --2- 172.21.15.43:0/3539917595 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f6f2413f330 0x7f6f241f0fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:36:06.060 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:06.056+0000 7f6f23fff640 1 --2- 172.21.15.43:0/3539917595 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f6f241546f0 0x7f6f241f4f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:36:06.060 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:06.056+0000 7f6f23fff640 1 --2- 172.21.15.43:0/3539917595 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f6f24070ab0 0x7f6f241424f0 unknown :-1 s=CLOSED pgs=538 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:36:06.060 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:06.056+0000 7f6f23fff640 1 -- 172.21.15.43:0/3539917595 >> 172.21.15.43:0/3539917595 conn(0x7f6f240af950 msgr2=0x7f6f24070ee0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-09T13:36:06.060 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:06.056+0000 7f6f23fff640 1 -- 172.21.15.43:0/3539917595 shutdown_connections 2023-08-09T13:36:06.060 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:06.056+0000 7f6f23fff640 1 -- 172.21.15.43:0/3539917595 wait complete. 2023-08-09T13:36:06.076 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth del client.bootstrap-rgw 2023-08-09T13:36:06.301 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:06.298+0000 7f67270ad640 1 Processor -- start 2023-08-09T13:36:06.301 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:06.298+0000 7f67270ad640 1 -- start start 2023-08-09T13:36:06.302 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:06.300+0000 7f67270ad640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f6720143c20 0x7f672006f540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:36:06.303 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:06.300+0000 7f67270ad640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f67200c86f0 0x7f67200c8b50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:36:06.303 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:06.300+0000 7f67270ad640 1 -- --> v1:172.21.15.43:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f67200c9090 con 0x7f672006c8d0 2023-08-09T13:36:06.303 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:06.300+0000 7f67270ad640 1 -- --> v2:172.21.15.43:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6720144000 con 0x7f67200c86f0 2023-08-09T13:36:06.304 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:06.300+0000 7f67270ad640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f672006c440 con 0x7f6720143c20 2023-08-09T13:36:06.304 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:06.301+0000 7f671ffff640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f67200c86f0 0x7f67200c8b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:36:06.304 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:06.301+0000 7f671ffff640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f67200c86f0 0x7f67200c8b50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:35820/0 (socket says 172.21.15.43:35820) 2023-08-09T13:36:06.304 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:06.301+0000 7f671ffff640 1 -- 172.21.15.43:0/1484634468 learned_addr learned my addr 172.21.15.43:0/1484634468 (peer_addr_for_me v2:172.21.15.43:0/0) 2023-08-09T13:36:06.305 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:06.301+0000 7f6724e49640 1 --2- 172.21.15.43:0/1484634468 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f6720143c20 0x7f672006f540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:36:06.306 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:06.301+0000 7f671ffff640 1 -- 172.21.15.43:0/1484634468 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f6720143c20 msgr2=0x7f672006f540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-09T13:36:06.306 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:06.301+0000 7f671ffff640 1 --2- 172.21.15.43:0/1484634468 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f6720143c20 0x7f672006f540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:36:06.306 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:06.301+0000 7f671ffff640 1 -- 172.21.15.43:0/1484634468 >> v1:172.21.15.43:6789/0 conn(0x7f672006c8d0 legacy=0x7f672006cd60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:36:06.306 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:06.301+0000 7f671ffff640 1 -- 172.21.15.43:0/1484634468 --> v2:172.21.15.43:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f672006b830 con 0x7f67200c86f0 2023-08-09T13:36:06.306 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:06.301+0000 7f671f7fe640 1 -- 172.21.15.43:0/1484634468 <== mon.0 v1:172.21.15.43:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 228244669 0 0) 0x7f67200c9090 con 0x7f672006c8d0 2023-08-09T13:36:06.306 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:06.303+0000 7f671ffff640 1 --2- 172.21.15.43:0/1484634468 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f67200c86f0 0x7f67200c8b50 secure :-1 s=READY pgs=540 cs=0 l=1 rev1=1 crypto rx=0x7f670c009870 tx=0x7f670c02b8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8bc5b8e294c5b3e3 server_cookie=0 in_seq=0 out_seq=0 2023-08-09T13:36:06.306 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:06.303+0000 7f671f7fe640 1 -- 172.21.15.43:0/1484634468 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f670c031640 con 0x7f67200c86f0 2023-08-09T13:36:06.307 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:06.303+0000 7f671f7fe640 1 -- 172.21.15.43:0/1484634468 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f670c0317a0 con 0x7f67200c86f0 2023-08-09T13:36:06.307 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:06.303+0000 7f671f7fe640 1 -- 172.21.15.43:0/1484634468 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f670c031b30 con 0x7f67200c86f0 2023-08-09T13:36:06.307 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:06.303+0000 7f67270ad640 1 -- 172.21.15.43:0/1484634468 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f67200c86f0 msgr2=0x7f67200c8b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:36:06.308 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:06.303+0000 7f67270ad640 1 --2- 172.21.15.43:0/1484634468 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f67200c86f0 0x7f67200c8b50 secure :-1 s=READY pgs=540 cs=0 l=1 rev1=1 crypto rx=0x7f670c009870 tx=0x7f670c02b8f0 comp rx=0 tx=0).stop 2023-08-09T13:36:06.308 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:06.304+0000 7f67270ad640 1 -- 172.21.15.43:0/1484634468 shutdown_connections 2023-08-09T13:36:06.308 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:06.304+0000 7f67270ad640 1 --2- 172.21.15.43:0/1484634468 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f67200c86f0 0x7f67200c8b50 unknown :-1 s=CLOSED pgs=540 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:36:06.308 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:06.304+0000 7f67270ad640 1 --2- 172.21.15.43:0/1484634468 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f6720143c20 0x7f672006f540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:36:06.308 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:06.304+0000 7f67270ad640 1 -- 172.21.15.43:0/1484634468 >> 172.21.15.43:0/1484634468 conn(0x7f67200ce0a0 msgr2=0x7f67200ce480 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-09T13:36:06.308 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:06.304+0000 7f67270ad640 1 -- 172.21.15.43:0/1484634468 shutdown_connections 2023-08-09T13:36:06.309 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:06.304+0000 7f67270ad640 1 -- 172.21.15.43:0/1484634468 wait complete. 2023-08-09T13:36:06.310 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:06.306+0000 7f67270ad640 1 Processor -- start 2023-08-09T13:36:06.310 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:06.306+0000 7f67270ad640 1 -- start start 2023-08-09T13:36:06.310 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:06.307+0000 7f67270ad640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f672006c8d0 0x7f672015a9d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:36:06.310 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:06.307+0000 7f67270ad640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f67200c86f0 0x7f6720159cd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:36:06.310 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:06.307+0000 7f67270ad640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f6720143c20 0x7f672015c640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:36:06.310 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:06.307+0000 7f67270ad640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f672015be60 con 0x7f672006c8d0 2023-08-09T13:36:06.311 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:06.307+0000 7f67270ad640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f672015d8e0 con 0x7f6720143c20 2023-08-09T13:36:06.311 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:06.307+0000 7f67270ad640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f67200b1660 con 0x7f67200c86f0 2023-08-09T13:36:06.311 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:06.307+0000 7f6724e49640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f672006c8d0 0x7f672015a9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:36:06.311 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:06.307+0000 7f6724e49640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f672006c8d0 0x7f672015a9d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:35828/0 (socket says 172.21.15.43:35828) 2023-08-09T13:36:06.311 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:06.307+0000 7f6724e49640 1 -- 172.21.15.43:0/3773053907 learned_addr learned my addr 172.21.15.43:0/3773053907 (peer_addr_for_me v2:172.21.15.43:0/0) 2023-08-09T13:36:06.311 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:06.307+0000 7f6724e49640 1 -- 172.21.15.43:0/3773053907 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f67200c86f0 msgr2=0x7f6720159cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:36:06.311 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:06.307+0000 7f6724e49640 1 --2- 172.21.15.43:0/3773053907 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f67200c86f0 0x7f6720159cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:36:06.312 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:06.307+0000 7f6724e49640 1 -- 172.21.15.43:0/3773053907 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f6720143c20 msgr2=0x7f672015c640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:36:06.312 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:06.307+0000 7f6724e49640 1 --2- 172.21.15.43:0/3773053907 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f6720143c20 0x7f672015c640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:36:06.312 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:06.307+0000 7f6724e49640 1 -- 172.21.15.43:0/3773053907 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f670c0094d0 con 0x7f672006c8d0 2023-08-09T13:36:06.313 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:06.308+0000 7f6724e49640 1 --2- 172.21.15.43:0/3773053907 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f672006c8d0 0x7f672015a9d0 secure :-1 s=READY pgs=541 cs=0 l=1 rev1=1 crypto rx=0x7f6714007ef0 tx=0x7f671400e4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-09T13:36:06.313 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:06.308+0000 7f671d7fa640 1 -- 172.21.15.43:0/3773053907 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6714014620 con 0x7f672006c8d0 2023-08-09T13:36:06.313 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:06.308+0000 7f671d7fa640 1 -- 172.21.15.43:0/3773053907 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6714014780 con 0x7f672006c8d0 2023-08-09T13:36:06.313 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:06.308+0000 7f671d7fa640 1 -- 172.21.15.43:0/3773053907 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6714014b30 con 0x7f672006c8d0 2023-08-09T13:36:06.313 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:06.308+0000 7f67270ad640 1 -- 172.21.15.43:0/3773053907 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f67200b1940 con 0x7f672006c8d0 2023-08-09T13:36:06.313 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:06.308+0000 7f67270ad640 1 -- 172.21.15.43:0/3773053907 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f67200b1f00 con 0x7f672006c8d0 2023-08-09T13:36:06.314 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:06.310+0000 7f67270ad640 1 -- 172.21.15.43:0/3773053907 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f67200b1ad0 con 0x7f672006c8d0 2023-08-09T13:36:06.314 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:06.311+0000 7f671d7fa640 1 -- 172.21.15.43:0/3773053907 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 5) v1 ==== 88341+0+0 (secure 0 0 0) 0x7f6714014c90 con 0x7f672006c8d0 2023-08-09T13:36:06.314 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:06.311+0000 7f671d7fa640 1 --2- 172.21.15.43:0/3773053907 >> [v2:172.21.15.203:6832/830097474,v1:172.21.15.203:6833/830097474] conn(0x7f670406a050 0x7f670406c510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:36:06.314 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:06.312+0000 7f671d7fa640 1 -- 172.21.15.43:0/3773053907 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(35..35 src has 1..35) v4 ==== 5530+0+0 (secure 0 0 0) 0x7f6714088ef0 con 0x7f672006c8d0 2023-08-09T13:36:06.315 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:06.312+0000 7f671ffff640 1 --2- 172.21.15.43:0/3773053907 >> [v2:172.21.15.203:6832/830097474,v1:172.21.15.203:6833/830097474] conn(0x7f670406a050 0x7f670406c510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:36:06.316 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:06.314+0000 7f671ffff640 1 --2- 172.21.15.43:0/3773053907 >> [v2:172.21.15.203:6832/830097474,v1:172.21.15.203:6833/830097474] conn(0x7f670406a050 0x7f670406c510 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7f670c036000 tx=0x7f670c030000 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-09T13:36:06.318 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:06.316+0000 7f671d7fa640 1 -- 172.21.15.43:0/3773053907 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178066 (secure 0 0 0) 0x7f67140546c0 con 0x7f672006c8d0 2023-08-09T13:36:06.587 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:06.585+0000 7f67270ad640 1 -- 172.21.15.43:0/3773053907 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "auth del", "entity": "client.bootstrap-rgw"} v 0) v1 -- 0x7f67200739f0 con 0x7f672006c8d0 2023-08-09T13:36:06.594 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:06.591+0000 7f671d7fa640 1 -- 172.21.15.43:0/3773053907 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "auth del", "entity": "client.bootstrap-rgw"}]=0 v9) v1 ==== 90+0+0 (secure 0 0 0) 0x7f6714057ce0 con 0x7f672006c8d0 2023-08-09T13:36:06.597 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:06.594+0000 7f67270ad640 1 -- 172.21.15.43:0/3773053907 >> [v2:172.21.15.203:6832/830097474,v1:172.21.15.203:6833/830097474] conn(0x7f670406a050 msgr2=0x7f670406c510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:36:06.598 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:06.594+0000 7f67270ad640 1 --2- 172.21.15.43:0/3773053907 >> [v2:172.21.15.203:6832/830097474,v1:172.21.15.203:6833/830097474] conn(0x7f670406a050 0x7f670406c510 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7f670c036000 tx=0x7f670c030000 comp rx=0 tx=0).stop 2023-08-09T13:36:06.598 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:06.594+0000 7f67270ad640 1 -- 172.21.15.43:0/3773053907 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f672006c8d0 msgr2=0x7f672015a9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:36:06.598 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:06.594+0000 7f67270ad640 1 --2- 172.21.15.43:0/3773053907 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f672006c8d0 0x7f672015a9d0 secure :-1 s=READY pgs=541 cs=0 l=1 rev1=1 crypto rx=0x7f6714007ef0 tx=0x7f671400e4b0 comp rx=0 tx=0).stop 2023-08-09T13:36:06.599 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:06.594+0000 7f67270ad640 1 -- 172.21.15.43:0/3773053907 shutdown_connections 2023-08-09T13:36:06.599 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:06.594+0000 7f67270ad640 1 --2- 172.21.15.43:0/3773053907 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f6720143c20 0x7f672015c640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:36:06.599 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:06.594+0000 7f67270ad640 1 --2- 172.21.15.43:0/3773053907 >> [v2:172.21.15.203:6832/830097474,v1:172.21.15.203:6833/830097474] conn(0x7f670406a050 0x7f670406c510 unknown :-1 s=CLOSED pgs=263 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:36:06.600 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:06.594+0000 7f67270ad640 1 --2- 172.21.15.43:0/3773053907 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f67200c86f0 0x7f6720159cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:36:06.600 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:06.594+0000 7f67270ad640 1 --2- 172.21.15.43:0/3773053907 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f672006c8d0 0x7f672015a9d0 unknown :-1 s=CLOSED pgs=541 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:36:06.600 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:06.594+0000 7f67270ad640 1 -- 172.21.15.43:0/3773053907 >> 172.21.15.43:0/3773053907 conn(0x7f67200ce0a0 msgr2=0x7f6720070580 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-09T13:36:06.600 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:06.594+0000 7f67270ad640 1 -- 172.21.15.43:0/3773053907 shutdown_connections 2023-08-09T13:36:06.601 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:06.595+0000 7f67270ad640 1 -- 172.21.15.43:0/3773053907 wait complete. 2023-08-09T13:36:06.618 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-08-09T13:36:06.849 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:06.846+0000 7fcde98cf640 1 Processor -- start 2023-08-09T13:36:06.849 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:06.847+0000 7fcde98cf640 1 -- start start 2023-08-09T13:36:06.851 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:06.848+0000 7fcde98cf640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fcde415c620 0x7fcde40cfe70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:36:06.851 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:06.848+0000 7fcde98cf640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7fcde40cf4f0 0x7fcde40d07e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:36:06.851 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:06.848+0000 7fcde98cf640 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7fcde40d0d20 0x7fcde40cce40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:36:06.851 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:06.848+0000 7fcde98cf640 1 -- --> v2:172.21.15.43:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fcde40b2330 con 0x7fcde40cf4f0 2023-08-09T13:36:06.851 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:06.848+0000 7fcde98cf640 1 -- --> v2:172.21.15.203:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fcde40d2ec0 con 0x7fcde40d0d20 2023-08-09T13:36:06.852 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:06.848+0000 7fcde98cf640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcde40d0500 con 0x7fcde415c620 2023-08-09T13:36:06.852 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:06.849+0000 7fcde3fff640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7fcde40cf4f0 0x7fcde40d07e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:36:06.853 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:06.849+0000 7fcde3fff640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7fcde40cf4f0 0x7fcde40d07e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:35842/0 (socket says 172.21.15.43:35842) 2023-08-09T13:36:06.853 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:06.849+0000 7fcde3fff640 1 -- 172.21.15.43:0/1890867564 learned_addr learned my addr 172.21.15.43:0/1890867564 (peer_addr_for_me v2:172.21.15.43:0/0) 2023-08-09T13:36:06.853 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:06.849+0000 7fcde88cd640 1 --2- 172.21.15.43:0/1890867564 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fcde415c620 0x7fcde40cfe70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:36:06.853 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:06.849+0000 7fcde90ce640 1 --2- 172.21.15.43:0/1890867564 >> v2:172.21.15.203:3300/0 conn(0x7fcde40d0d20 0x7fcde40cce40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:36:06.853 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:06.849+0000 7fcde3fff640 1 -- 172.21.15.43:0/1890867564 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fcde415c620 msgr2=0x7fcde40cfe70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-09T13:36:06.853 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:06.849+0000 7fcde3fff640 1 --2- 172.21.15.43:0/1890867564 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fcde415c620 0x7fcde40cfe70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:36:06.853 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:06.849+0000 7fcde3fff640 1 -- 172.21.15.43:0/1890867564 >> v2:172.21.15.203:3300/0 conn(0x7fcde40d0d20 msgr2=0x7fcde40cce40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-09T13:36:06.854 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:06.849+0000 7fcde3fff640 1 --2- 172.21.15.43:0/1890867564 >> v2:172.21.15.203:3300/0 conn(0x7fcde40d0d20 0x7fcde40cce40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:36:06.854 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:06.849+0000 7fcde3fff640 1 -- 172.21.15.43:0/1890867564 --> v2:172.21.15.43:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcde40b1a00 con 0x7fcde40cf4f0 2023-08-09T13:36:06.854 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:06.849+0000 7fcde90ce640 1 --2- 172.21.15.43:0/1890867564 >> v2:172.21.15.203:3300/0 conn(0x7fcde40d0d20 0x7fcde40cce40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-08-09T13:36:06.855 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:06.851+0000 7fcde3fff640 1 --2- 172.21.15.43:0/1890867564 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fcde40cf4f0 0x7fcde40d07e0 secure :-1 s=READY pgs=542 cs=0 l=1 rev1=1 crypto rx=0x7fcde40d2230 tx=0x7fcdd802b870 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c308675a2d54468f server_cookie=0 in_seq=0 out_seq=0 2023-08-09T13:36:06.855 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:06.851+0000 7fcde37fe640 1 -- 172.21.15.43:0/1890867564 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcdd80317d0 con 0x7fcde40cf4f0 2023-08-09T13:36:06.855 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:06.851+0000 7fcde37fe640 1 -- 172.21.15.43:0/1890867564 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcdd8031930 con 0x7fcde40cf4f0 2023-08-09T13:36:06.855 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:06.851+0000 7fcde37fe640 1 -- 172.21.15.43:0/1890867564 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcdd8031cc0 con 0x7fcde40cf4f0 2023-08-09T13:36:06.855 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:06.851+0000 7fcde98cf640 1 -- 172.21.15.43:0/1890867564 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fcde40cf4f0 msgr2=0x7fcde40d07e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:36:06.855 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:06.851+0000 7fcde98cf640 1 --2- 172.21.15.43:0/1890867564 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fcde40cf4f0 0x7fcde40d07e0 secure :-1 s=READY pgs=542 cs=0 l=1 rev1=1 crypto rx=0x7fcde40d2230 tx=0x7fcdd802b870 comp rx=0 tx=0).stop 2023-08-09T13:36:06.856 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:06.852+0000 7fcde98cf640 1 -- 172.21.15.43:0/1890867564 shutdown_connections 2023-08-09T13:36:06.856 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:06.852+0000 7fcde98cf640 1 --2- 172.21.15.43:0/1890867564 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fcde40cf4f0 0x7fcde40d07e0 unknown :-1 s=CLOSED pgs=542 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:36:06.856 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:06.852+0000 7fcde98cf640 1 --2- 172.21.15.43:0/1890867564 >> v2:172.21.15.203:3300/0 conn(0x7fcde40d0d20 0x7fcde40cce40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:36:06.856 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:06.852+0000 7fcde98cf640 1 --2- 172.21.15.43:0/1890867564 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fcde415c620 0x7fcde40cfe70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:36:06.856 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:06.852+0000 7fcde98cf640 1 -- 172.21.15.43:0/1890867564 >> 172.21.15.43:0/1890867564 conn(0x7fcde409b650 msgr2=0x7fcde40d22a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-09T13:36:06.856 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:06.852+0000 7fcde98cf640 1 -- 172.21.15.43:0/1890867564 shutdown_connections 2023-08-09T13:36:06.856 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:06.852+0000 7fcde98cf640 1 -- 172.21.15.43:0/1890867564 wait complete. 2023-08-09T13:36:06.858 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:06.853+0000 7fcde98cf640 1 Processor -- start 2023-08-09T13:36:06.858 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:06.854+0000 7fcde98cf640 1 -- start start 2023-08-09T13:36:06.858 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:06.855+0000 7fcde98cf640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fcde40cf4f0 0x7fcde40c7e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:36:06.858 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:06.855+0000 7fcde98cf640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fcde40d0d20 0x7fcde40c5dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:36:06.858 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:06.855+0000 7fcde98cf640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fcde415c620 0x7fcde40b77b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:36:06.858 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:06.855+0000 7fcde98cf640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcde40cb640 con 0x7fcde40d0d20 2023-08-09T13:36:06.858 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:06.855+0000 7fcde98cf640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcde40b56d0 con 0x7fcde415c620 2023-08-09T13:36:06.859 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:06.855+0000 7fcde98cf640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcde40c6490 con 0x7fcde40cf4f0 2023-08-09T13:36:06.859 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:06.855+0000 7fcde3fff640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fcde40d0d20 0x7fcde40c5dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:36:06.859 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:06.855+0000 7fcde3fff640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fcde40d0d20 0x7fcde40c5dd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:35852/0 (socket says 172.21.15.43:35852) 2023-08-09T13:36:06.859 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:06.855+0000 7fcde3fff640 1 -- 172.21.15.43:0/742778316 learned_addr learned my addr 172.21.15.43:0/742778316 (peer_addr_for_me v2:172.21.15.43:0/0) 2023-08-09T13:36:06.859 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:06.855+0000 7fcde90ce640 1 --2- 172.21.15.43:0/742778316 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fcde415c620 0x7fcde40b77b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:36:06.859 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:06.855+0000 7fcde88cd640 1 --2- 172.21.15.43:0/742778316 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fcde40cf4f0 0x7fcde40c7e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:36:06.859 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:06.855+0000 7fcde3fff640 1 -- 172.21.15.43:0/742778316 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fcde40cf4f0 msgr2=0x7fcde40c7e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:36:06.859 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:06.855+0000 7fcde3fff640 1 --2- 172.21.15.43:0/742778316 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fcde40cf4f0 0x7fcde40c7e70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:36:06.860 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:06.855+0000 7fcde3fff640 1 -- 172.21.15.43:0/742778316 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fcde415c620 msgr2=0x7fcde40b77b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:36:06.860 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:06.855+0000 7fcde3fff640 1 --2- 172.21.15.43:0/742778316 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fcde415c620 0x7fcde40b77b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:36:06.860 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:06.855+0000 7fcde3fff640 1 -- 172.21.15.43:0/742778316 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcdd80094d0 con 0x7fcde40d0d20 2023-08-09T13:36:06.860 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:06.855+0000 7fcde90ce640 1 --2- 172.21.15.43:0/742778316 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fcde415c620 0x7fcde40b77b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-08-09T13:36:06.861 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:06.856+0000 7fcde3fff640 1 --2- 172.21.15.43:0/742778316 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fcde40d0d20 0x7fcde40c5dd0 secure :-1 s=READY pgs=543 cs=0 l=1 rev1=1 crypto rx=0x7fcdd8000c00 tx=0x7fcdd80046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-09T13:36:06.861 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:06.856+0000 7fcde17fa640 1 -- 172.21.15.43:0/742778316 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcdd8032870 con 0x7fcde40d0d20 2023-08-09T13:36:06.861 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:06.856+0000 7fcde17fa640 1 -- 172.21.15.43:0/742778316 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcdd80329f0 con 0x7fcde40d0d20 2023-08-09T13:36:06.861 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:06.856+0000 7fcde98cf640 1 -- 172.21.15.43:0/742778316 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcde40c6770 con 0x7fcde40d0d20 2023-08-09T13:36:06.861 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:06.856+0000 7fcde17fa640 1 -- 172.21.15.43:0/742778316 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcdd8032da0 con 0x7fcde40d0d20 2023-08-09T13:36:06.862 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:06.856+0000 7fcde98cf640 1 -- 172.21.15.43:0/742778316 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcde40c6d30 con 0x7fcde40d0d20 2023-08-09T13:36:06.862 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:06.857+0000 7fcde98cf640 1 -- 172.21.15.43:0/742778316 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcde40b5890 con 0x7fcde40d0d20 2023-08-09T13:36:06.862 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:06.858+0000 7fcde17fa640 1 -- 172.21.15.43:0/742778316 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 5) v1 ==== 88341+0+0 (secure 0 0 0) 0x7fcdd8030070 con 0x7fcde40d0d20 2023-08-09T13:36:06.862 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:06.860+0000 7fcde17fa640 1 --2- 172.21.15.43:0/742778316 >> [v2:172.21.15.203:6832/830097474,v1:172.21.15.203:6833/830097474] conn(0x7fcdd006a050 0x7fcdd006c510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:36:06.863 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:06.860+0000 7fcde17fa640 1 -- 172.21.15.43:0/742778316 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(35..35 src has 1..35) v4 ==== 5530+0+0 (secure 0 0 0) 0x7fcdd80a9fb0 con 0x7fcde40d0d20 2023-08-09T13:36:06.863 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:06.861+0000 7fcde88cd640 1 --2- 172.21.15.43:0/742778316 >> [v2:172.21.15.203:6832/830097474,v1:172.21.15.203:6833/830097474] conn(0x7fcdd006a050 0x7fcdd006c510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:36:06.864 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:06.862+0000 7fcde88cd640 1 --2- 172.21.15.43:0/742778316 >> [v2:172.21.15.203:6832/830097474,v1:172.21.15.203:6833/830097474] conn(0x7fcdd006a050 0x7fcdd006c510 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7fcde40cb220 tx=0x7fcdd400b040 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-09T13:36:06.866 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:06.864+0000 7fcde17fa640 1 -- 172.21.15.43:0/742778316 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178066 (secure 0 0 0) 0x7fcdd80758c0 con 0x7fcde40d0d20 2023-08-09T13:36:07.057 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:07.054+0000 7fcde98cf640 1 -- 172.21.15.43:0/742778316 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fcde40c6f20 con 0x7fcde40d0d20 2023-08-09T13:36:07.057 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:07.055+0000 7fcde17fa640 1 -- 172.21.15.43:0/742778316 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v35) v1 ==== 74+0+14774 (secure 0 0 0) 0x7fcdd8078ee0 con 0x7fcde40d0d20 2023-08-09T13:36:07.058 INFO:teuthology.orchestra.run.smithi043.stdout: 2023-08-09T13:36:07.058 INFO:teuthology.orchestra.run.smithi043.stdout:{"epoch":35,"fsid":"daac87ee-1337-4250-a955-aea4e408a39b","created":"2023-08-09T13:33:41.701198+0000","modified":"2023-08-09T13:36:02.761448+0000","last_up_change":"2023-08-09T13:33:53.783453+0000","last_in_change":"2023-08-09T13:33:45.148904+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":5,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":"rbd","create_time":"2023-08-09T13:33:54.365137+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":".mgr","create_time":"2023-08-09T13:33:54.831803+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"cbd49ecd-e896-4fbd-864e-5c37a829b165","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6800","nonce":502667335},{"type":"v1","addr":"172.21.15.43:6801","nonce":502667335}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6802","nonce":502667335},{"type":"v1","addr":"172.21.15.43:6803","nonce":502667335}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6806","nonce":502667335},{"type":"v1","addr":"172.21.15.43:6807","nonce":502667335}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6804","nonce":502667335},{"type":"v1","addr":"172.21.15.43:6805","nonce":502667335}]},"public_addr":"172.21.15.43:6801/502667335","cluster_addr":"172.21.15.43:6803/502667335","heartbeat_back_addr":"172.21.15.43:6807/502667335","heartbeat_front_addr":"172.21.15.43:6805/502667335","state":["exists","up"]},{"osd":1,"uuid":"75c7e0fe-1ec7-4fe8-a2f2-3661a65b8500","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6808","nonce":3289134610},{"type":"v1","addr":"172.21.15.43:6809","nonce":3289134610}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6810","nonce":3289134610},{"type":"v1","addr":"172.21.15.43:6812","nonce":3289134610}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6821","nonce":3289134610},{"type":"v1","addr":"172.21.15.43:6824","nonce":3289134610}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6815","nonce":3289134610},{"type":"v1","addr":"172.21.15.43:6818","nonce":3289134610}]},"public_addr":"172.21.15.43:6809/3289134610","cluster_addr":"172.21.15.43:6812/3289134610","heartbeat_back_addr":"172.21.15.43:6824/3289134610","heartbeat_front_addr":"172.21.15.43:6818/3289134610","state":["exists","up"]},{"osd":2,"uuid":"bb5c41e9-7f5f-4125-9b01-3bb8bfc2ea83","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6811","nonce":4079709736},{"type":"v1","addr":"172.21.15.43:6814","nonce":4079709736}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6817","nonce":4079709736},{"type":"v1","addr":"172.21.15.43:6820","nonce":4079709736}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6829","nonce":4079709736},{"type":"v1","addr":"172.21.15.43:6831","nonce":4079709736}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6823","nonce":4079709736},{"type":"v1","addr":"172.21.15.43:6826","nonce":4079709736}]},"public_addr":"172.21.15.43:6814/4079709736","cluster_addr":"172.21.15.43:6820/4079709736","heartbeat_back_addr":"172.21.15.43:6831/4079709736","heartbeat_front_addr":"172.21.15.43:6826/4079709736","state":["exists","up"]},{"osd":3,"uuid":"bfe4c8b4-694a-465c-87e6-2b2aab852613","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6813","nonce":1709538759},{"type":"v1","addr":"172.21.15.43:6816","nonce":1709538759}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6819","nonce":1709538759},{"type":"v1","addr":"172.21.15.43:6822","nonce":1709538759}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6828","nonce":1709538759},{"type":"v1","addr":"172.21.15.43:6830","nonce":1709538759}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6825","nonce":1709538759},{"type":"v1","addr":"172.21.15.43:6827","nonce":1709538759}]},"public_addr":"172.21.15.43:6816/1709538759","cluster_addr":"172.21.15.43:6822/1709538759","heartbeat_back_addr":"172.21.15.43:6830/1709538759","heartbeat_front_addr":"172.21.15.43:6827/1709538759","state":["exists","up"]},{"osd":4,"uuid":"fa2e8a77-aba1-4756-853e-0d174a5bdeba","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6800","nonce":4196866481},{"type":"v1","addr":"172.21.15.203:6801","nonce":4196866481}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6802","nonce":4196866481},{"type":"v1","addr":"172.21.15.203:6803","nonce":4196866481}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6806","nonce":4196866481},{"type":"v1","addr":"172.21.15.203:6807","nonce":4196866481}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6804","nonce":4196866481},{"type":"v1","addr":"172.21.15.203:6805","nonce":4196866481}]},"public_addr":"172.21.15.203:6801/4196866481","cluster_addr":"172.21.15.203:6803/4196866481","heartbeat_back_addr":"172.21.15.203:6807/4196866481","heartbeat_front_addr":"172.21.15.203:6805/4196866481","state":["exists","up"]},{"osd":5,"uuid":"81811e4b-4d24-4f51-a82c-f1d700a2a00d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6816","nonce":3059290081},{"type":"v1","addr":"172.21.15.203:6817","nonce":3059290081}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6818","nonce":3059290081},{"type":"v1","addr":"172.21.15.203:6819","nonce":3059290081}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6822","nonce":3059290081},{"type":"v1","addr":"172.21.15.203:6823","nonce":3059290081}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6820","nonce":3059290081},{"type":"v1","addr":"172.21.15.203:6821","nonce":3059290081}]},"public_addr":"172.21.15.203:6817/3059290081","cluster_addr":"172.21.15.203:6819/3059290081","heartbeat_back_addr":"172.21.15.203:6823/3059290081","heartbeat_front_addr":"172.21.15.203:6821/3059290081","state":["exists","up"]},{"osd":6,"uuid":"307bac4d-6a5b-443d-af75-56d695d7674f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6824","nonce":512155569},{"type":"v1","addr":"172.21.15.203:6825","nonce":512155569}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6826","nonce":512155569},{"type":"v1","addr":"172.21.15.203:6827","nonce":512155569}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6830","nonce":512155569},{"type":"v1","addr":"172.21.15.203:6831","nonce":512155569}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6828","nonce":512155569},{"type":"v1","addr":"172.21.15.203:6829","nonce":512155569}]},"public_addr":"172.21.15.203:6825/512155569","cluster_addr":"172.21.15.203:6827/512155569","heartbeat_back_addr":"172.21.15.203:6831/512155569","heartbeat_front_addr":"172.21.15.203:6829/512155569","state":["exists","up"]},{"osd":7,"uuid":"a95dc7ca-9f1b-4295-af70-68a22a2d7d5e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6808","nonce":3331450438},{"type":"v1","addr":"172.21.15.203:6809","nonce":3331450438}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6810","nonce":3331450438},{"type":"v1","addr":"172.21.15.203:6811","nonce":3331450438}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6814","nonce":3331450438},{"type":"v1","addr":"172.21.15.203:6815","nonce":3331450438}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6812","nonce":3331450438},{"type":"v1","addr":"172.21.15.203:6813","nonce":3331450438}]},"public_addr":"172.21.15.203:6809/3331450438","cluster_addr":"172.21.15.203:6811/3331450438","heartbeat_back_addr":"172.21.15.203:6815/3331450438","heartbeat_front_addr":"172.21.15.203:6813/3331450438","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-09T13:33:50.547652+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-09T13:33:50.589469+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-09T13:33:50.561724+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-09T13:33:50.599650+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-09T13:33:50.411855+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-09T13:33:50.219681+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-09T13:33:50.504664+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-09T13:33:50.441248+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-08-09T13:36:07.061 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:07.058+0000 7fcde98cf640 1 -- 172.21.15.43:0/742778316 >> [v2:172.21.15.203:6832/830097474,v1:172.21.15.203:6833/830097474] conn(0x7fcdd006a050 msgr2=0x7fcdd006c510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:36:07.061 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:07.058+0000 7fcde98cf640 1 --2- 172.21.15.43:0/742778316 >> [v2:172.21.15.203:6832/830097474,v1:172.21.15.203:6833/830097474] conn(0x7fcdd006a050 0x7fcdd006c510 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7fcde40cb220 tx=0x7fcdd400b040 comp rx=0 tx=0).stop 2023-08-09T13:36:07.062 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:07.058+0000 7fcde98cf640 1 -- 172.21.15.43:0/742778316 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fcde40d0d20 msgr2=0x7fcde40c5dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:36:07.062 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:07.058+0000 7fcde98cf640 1 --2- 172.21.15.43:0/742778316 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fcde40d0d20 0x7fcde40c5dd0 secure :-1 s=READY pgs=543 cs=0 l=1 rev1=1 crypto rx=0x7fcdd8000c00 tx=0x7fcdd80046c0 comp rx=0 tx=0).stop 2023-08-09T13:36:07.062 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:07.059+0000 7fcde98cf640 1 -- 172.21.15.43:0/742778316 shutdown_connections 2023-08-09T13:36:07.062 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:07.059+0000 7fcde98cf640 1 --2- 172.21.15.43:0/742778316 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fcde415c620 0x7fcde40b77b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:36:07.062 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:07.059+0000 7fcde98cf640 1 --2- 172.21.15.43:0/742778316 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fcde40d0d20 0x7fcde40c5dd0 unknown :-1 s=CLOSED pgs=543 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:36:07.062 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:07.059+0000 7fcde98cf640 1 --2- 172.21.15.43:0/742778316 >> [v2:172.21.15.203:6832/830097474,v1:172.21.15.203:6833/830097474] conn(0x7fcdd006a050 0x7fcdd006c510 unknown :-1 s=CLOSED pgs=264 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:36:07.062 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:07.059+0000 7fcde98cf640 1 --2- 172.21.15.43:0/742778316 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fcde40cf4f0 0x7fcde40c7e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:36:07.062 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:07.059+0000 7fcde98cf640 1 -- 172.21.15.43:0/742778316 >> 172.21.15.43:0/742778316 conn(0x7fcde409b650 msgr2=0x7fcde406a3c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-09T13:36:07.063 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:07.059+0000 7fcde98cf640 1 -- 172.21.15.43:0/742778316 shutdown_connections 2023-08-09T13:36:07.063 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:07.059+0000 7fcde98cf640 1 -- 172.21.15.43:0/742778316 wait complete. 2023-08-09T13:36:07.078 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': 'rbd', 'create_time': '2023-08-09T13:33:54.365137+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}, 'read_balance': {'score_acting': 2, 'score_stable': 2, 'optimal_score': 1, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': '.mgr', 'create_time': '2023-08-09T13:33:54.831803+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-08-09T13:36:07.079 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get rbd pg_num 2023-08-09T13:36:07.304 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:07.301+0000 7fdfc0c4b640 1 Processor -- start 2023-08-09T13:36:07.304 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:07.301+0000 7fdfc0c4b640 1 -- start start 2023-08-09T13:36:07.305 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:07.303+0000 7fdfc0c4b640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7fdfbc0c9ee0 0x7fdfbc15a800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:36:07.306 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:07.303+0000 7fdfc0c4b640 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7fdfbc0c90a0 0x7fdfbc15ad40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:36:07.306 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:07.303+0000 7fdfc0c4b640 1 -- --> v1:172.21.15.203:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fdfbc158280 con 0x7fdfbc157ea0 2023-08-09T13:36:07.306 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:07.303+0000 7fdfc0c4b640 1 -- --> v2:172.21.15.43:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fdfbc15b840 con 0x7fdfbc0c9ee0 2023-08-09T13:36:07.306 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:07.303+0000 7fdfc0c4b640 1 -- --> v2:172.21.15.203:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fdfbc0b3810 con 0x7fdfbc0c90a0 2023-08-09T13:36:07.307 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:07.304+0000 7fdfb3fff640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7fdfbc0c9ee0 0x7fdfbc15a800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:36:07.307 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:07.304+0000 7fdfb3fff640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7fdfbc0c9ee0 0x7fdfbc15a800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:35868/0 (socket says 172.21.15.43:35868) 2023-08-09T13:36:07.307 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:07.304+0000 7fdfb3fff640 1 -- 172.21.15.43:0/2349963414 learned_addr learned my addr 172.21.15.43:0/2349963414 (peer_addr_for_me v2:172.21.15.43:0/0) 2023-08-09T13:36:07.307 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:07.304+0000 7fdfb3fff640 1 -- 172.21.15.43:0/2349963414 >> v2:172.21.15.203:3300/0 conn(0x7fdfbc0c90a0 msgr2=0x7fdfbc15ad40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-09T13:36:07.308 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:07.304+0000 7fdfb3fff640 1 --2- 172.21.15.43:0/2349963414 >> v2:172.21.15.203:3300/0 conn(0x7fdfbc0c90a0 0x7fdfbc15ad40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:36:07.308 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:07.304+0000 7fdfb3fff640 1 -- 172.21.15.43:0/2349963414 >> v1:172.21.15.203:6789/0 conn(0x7fdfbc157ea0 legacy=0x7fdfbc0c97d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-09T13:36:07.308 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:07.304+0000 7fdfb3fff640 1 -- 172.21.15.43:0/2349963414 --> v2:172.21.15.43:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdfbc15b560 con 0x7fdfbc0c9ee0 2023-08-09T13:36:07.309 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:07.305+0000 7fdfb3fff640 1 --2- 172.21.15.43:0/2349963414 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fdfbc0c9ee0 0x7fdfbc15a800 secure :-1 s=READY pgs=544 cs=0 l=1 rev1=1 crypto rx=0x7fdfbc070190 tx=0x7fdfa402b880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e7c436d566cada16 server_cookie=0 in_seq=0 out_seq=0 2023-08-09T13:36:07.309 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:07.306+0000 7fdfbaffd640 1 -- 172.21.15.43:0/2349963414 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdfa40317e0 con 0x7fdfbc0c9ee0 2023-08-09T13:36:07.309 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:07.306+0000 7fdfbaffd640 1 -- 172.21.15.43:0/2349963414 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdfa4031940 con 0x7fdfbc0c9ee0 2023-08-09T13:36:07.309 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:07.306+0000 7fdfbaffd640 1 -- 172.21.15.43:0/2349963414 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdfa4031cd0 con 0x7fdfbc0c9ee0 2023-08-09T13:36:07.309 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:07.306+0000 7fdfc0c4b640 1 -- 172.21.15.43:0/2349963414 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fdfbc0c9ee0 msgr2=0x7fdfbc15a800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:36:07.310 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:07.306+0000 7fdfc0c4b640 1 --2- 172.21.15.43:0/2349963414 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fdfbc0c9ee0 0x7fdfbc15a800 secure :-1 s=READY pgs=544 cs=0 l=1 rev1=1 crypto rx=0x7fdfbc070190 tx=0x7fdfa402b880 comp rx=0 tx=0).stop 2023-08-09T13:36:07.310 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:07.306+0000 7fdfc0c4b640 1 -- 172.21.15.43:0/2349963414 shutdown_connections 2023-08-09T13:36:07.310 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:07.306+0000 7fdfc0c4b640 1 --2- 172.21.15.43:0/2349963414 >> v2:172.21.15.203:3300/0 conn(0x7fdfbc0c90a0 0x7fdfbc15ad40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:36:07.310 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:07.306+0000 7fdfc0c4b640 1 --2- 172.21.15.43:0/2349963414 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fdfbc0c9ee0 0x7fdfbc15a800 unknown :-1 s=CLOSED pgs=544 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:36:07.310 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:07.306+0000 7fdfc0c4b640 1 -- 172.21.15.43:0/2349963414 >> 172.21.15.43:0/2349963414 conn(0x7fdfbc0cddd0 msgr2=0x7fdfbc0ce1e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-09T13:36:07.310 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:07.307+0000 7fdfc0c4b640 1 -- 172.21.15.43:0/2349963414 shutdown_connections 2023-08-09T13:36:07.311 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:07.307+0000 7fdfc0c4b640 1 -- 172.21.15.43:0/2349963414 wait complete. 2023-08-09T13:36:07.311 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:07.308+0000 7fdfc0c4b640 1 Processor -- start 2023-08-09T13:36:07.311 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:07.309+0000 7fdfc0c4b640 1 -- start start 2023-08-09T13:36:07.312 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:07.310+0000 7fdfc0c4b640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fdfbc0c90a0 0x7fdfbc0cb6f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:36:07.312 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:07.310+0000 7fdfc0c4b640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fdfbc0c9ee0 0x7fdfbc06b3e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:36:07.313 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:07.310+0000 7fdfc0c4b640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fdfbc157ea0 0x7fdfbc06b920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:36:07.313 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:07.310+0000 7fdfc0c4b640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdfbc0cd880 con 0x7fdfbc0c9ee0 2023-08-09T13:36:07.313 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:07.310+0000 7fdfc0c4b640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdfbc0cd460 con 0x7fdfbc0c90a0 2023-08-09T13:36:07.313 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:07.310+0000 7fdfc0c4b640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdfbc0cc7e0 con 0x7fdfbc157ea0 2023-08-09T13:36:07.313 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:07.310+0000 7fdfb3fff640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fdfbc0c9ee0 0x7fdfbc06b3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:36:07.313 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:07.310+0000 7fdfb3fff640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fdfbc0c9ee0 0x7fdfbc06b3e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:35872/0 (socket says 172.21.15.43:35872) 2023-08-09T13:36:07.313 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:07.310+0000 7fdfb3fff640 1 -- 172.21.15.43:0/2776208885 learned_addr learned my addr 172.21.15.43:0/2776208885 (peer_addr_for_me v2:172.21.15.43:0/0) 2023-08-09T13:36:07.314 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:07.310+0000 7fdfbb7fe640 1 --2- 172.21.15.43:0/2776208885 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fdfbc0c90a0 0x7fdfbc0cb6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:36:07.315 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:07.310+0000 7fdfbbfff640 1 --2- 172.21.15.43:0/2776208885 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fdfbc157ea0 0x7fdfbc06b920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:36:07.315 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:07.310+0000 7fdfb3fff640 1 -- 172.21.15.43:0/2776208885 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fdfbc157ea0 msgr2=0x7fdfbc06b920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:36:07.315 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:07.310+0000 7fdfb3fff640 1 --2- 172.21.15.43:0/2776208885 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fdfbc157ea0 0x7fdfbc06b920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:36:07.315 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:07.310+0000 7fdfb3fff640 1 -- 172.21.15.43:0/2776208885 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fdfbc0c90a0 msgr2=0x7fdfbc0cb6f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:36:07.315 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:07.310+0000 7fdfb3fff640 1 --2- 172.21.15.43:0/2776208885 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fdfbc0c90a0 0x7fdfbc0cb6f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:36:07.316 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:07.310+0000 7fdfb3fff640 1 -- 172.21.15.43:0/2776208885 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdfa40094d0 con 0x7fdfbc0c9ee0 2023-08-09T13:36:07.316 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:07.311+0000 7fdfb3fff640 1 --2- 172.21.15.43:0/2776208885 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fdfbc0c9ee0 0x7fdfbc06b3e0 secure :-1 s=READY pgs=545 cs=0 l=1 rev1=1 crypto rx=0x7fdfa4000c00 tx=0x7fdfa40046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-09T13:36:07.316 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:07.311+0000 7fdfb8ff9640 1 -- 172.21.15.43:0/2776208885 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdfa4032880 con 0x7fdfbc0c9ee0 2023-08-09T13:36:07.316 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:07.311+0000 7fdfb8ff9640 1 -- 172.21.15.43:0/2776208885 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdfa4032a00 con 0x7fdfbc0c9ee0 2023-08-09T13:36:07.317 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:07.311+0000 7fdfb8ff9640 1 -- 172.21.15.43:0/2776208885 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdfa4032db0 con 0x7fdfbc0c9ee0 2023-08-09T13:36:07.317 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:07.311+0000 7fdfc0c4b640 1 -- 172.21.15.43:0/2776208885 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdfbc0ccac0 con 0x7fdfbc0c9ee0 2023-08-09T13:36:07.317 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:07.311+0000 7fdfc0c4b640 1 -- 172.21.15.43:0/2776208885 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdfbc0cd080 con 0x7fdfbc0c9ee0 2023-08-09T13:36:07.319 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:07.312+0000 7fdfc0c4b640 1 -- 172.21.15.43:0/2776208885 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdfbc0cd5a0 con 0x7fdfbc0c9ee0 2023-08-09T13:36:07.319 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:07.313+0000 7fdfb8ff9640 1 -- 172.21.15.43:0/2776208885 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 5) v1 ==== 88341+0+0 (secure 0 0 0) 0x7fdfa404d3f0 con 0x7fdfbc0c9ee0 2023-08-09T13:36:07.319 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:07.314+0000 7fdfb8ff9640 1 --2- 172.21.15.43:0/2776208885 >> [v2:172.21.15.203:6832/830097474,v1:172.21.15.203:6833/830097474] conn(0x7fdf9c06a050 0x7fdf9c06c510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:36:07.319 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:07.315+0000 7fdfb8ff9640 1 -- 172.21.15.43:0/2776208885 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(35..35 src has 1..35) v4 ==== 5530+0+0 (secure 0 0 0) 0x7fdfa40a9fe0 con 0x7fdfbc0c9ee0 2023-08-09T13:36:07.320 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:07.315+0000 7fdfbb7fe640 1 --2- 172.21.15.43:0/2776208885 >> [v2:172.21.15.203:6832/830097474,v1:172.21.15.203:6833/830097474] conn(0x7fdf9c06a050 0x7fdf9c06c510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:36:07.320 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:07.316+0000 7fdfbb7fe640 1 --2- 172.21.15.43:0/2776208885 >> [v2:172.21.15.203:6832/830097474,v1:172.21.15.203:6833/830097474] conn(0x7fdf9c06a050 0x7fdf9c06c510 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7fdfac006fd0 tx=0x7fdfac008040 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-09T13:36:07.321 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:07.319+0000 7fdfb8ff9640 1 -- 172.21.15.43:0/2776208885 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178066 (secure 0 0 0) 0x7fdfbc0cd5a0 con 0x7fdfbc0c9ee0 2023-08-09T13:36:07.514 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:07.512+0000 7fdfc0c4b640 1 -- 172.21.15.43:0/2776208885 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"} v 0) v1 -- 0x7fdfbc158280 con 0x7fdfbc0c9ee0 2023-08-09T13:36:07.515 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:07.513+0000 7fdfb8ff9640 1 -- 172.21.15.43:0/2776208885 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"}]=0 v35) v1 ==== 92+0+10 (secure 0 0 0) 0x7fdfbc158280 con 0x7fdfbc0c9ee0 2023-08-09T13:36:07.516 INFO:teuthology.orchestra.run.smithi043.stdout:pg_num: 8 2023-08-09T13:36:07.518 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:07.515+0000 7fdfc0c4b640 1 -- 172.21.15.43:0/2776208885 >> [v2:172.21.15.203:6832/830097474,v1:172.21.15.203:6833/830097474] conn(0x7fdf9c06a050 msgr2=0x7fdf9c06c510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:36:07.518 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:07.515+0000 7fdfc0c4b640 1 --2- 172.21.15.43:0/2776208885 >> [v2:172.21.15.203:6832/830097474,v1:172.21.15.203:6833/830097474] conn(0x7fdf9c06a050 0x7fdf9c06c510 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7fdfac006fd0 tx=0x7fdfac008040 comp rx=0 tx=0).stop 2023-08-09T13:36:07.519 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:07.515+0000 7fdfc0c4b640 1 -- 172.21.15.43:0/2776208885 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fdfbc0c9ee0 msgr2=0x7fdfbc06b3e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:36:07.519 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:07.515+0000 7fdfc0c4b640 1 --2- 172.21.15.43:0/2776208885 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fdfbc0c9ee0 0x7fdfbc06b3e0 secure :-1 s=READY pgs=545 cs=0 l=1 rev1=1 crypto rx=0x7fdfa4000c00 tx=0x7fdfa40046c0 comp rx=0 tx=0).stop 2023-08-09T13:36:07.519 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:07.515+0000 7fdfc0c4b640 1 -- 172.21.15.43:0/2776208885 shutdown_connections 2023-08-09T13:36:07.519 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:07.515+0000 7fdfc0c4b640 1 --2- 172.21.15.43:0/2776208885 >> [v2:172.21.15.203:6832/830097474,v1:172.21.15.203:6833/830097474] conn(0x7fdf9c06a050 0x7fdf9c06c510 unknown :-1 s=CLOSED pgs=265 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:36:07.520 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:07.515+0000 7fdfc0c4b640 1 --2- 172.21.15.43:0/2776208885 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fdfbc157ea0 0x7fdfbc06b920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:36:07.520 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:07.515+0000 7fdfc0c4b640 1 --2- 172.21.15.43:0/2776208885 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fdfbc0c9ee0 0x7fdfbc06b3e0 unknown :-1 s=CLOSED pgs=545 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:36:07.520 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:07.515+0000 7fdfc0c4b640 1 --2- 172.21.15.43:0/2776208885 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fdfbc0c90a0 0x7fdfbc0cb6f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:36:07.521 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:07.515+0000 7fdfc0c4b640 1 -- 172.21.15.43:0/2776208885 >> 172.21.15.43:0/2776208885 conn(0x7fdfbc0cddd0 msgr2=0x7fdfbc06cd00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-09T13:36:07.521 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:07.516+0000 7fdfc0c4b640 1 -- 172.21.15.43:0/2776208885 shutdown_connections 2023-08-09T13:36:07.521 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:07.516+0000 7fdfc0c4b640 1 -- 172.21.15.43:0/2776208885 wait complete. 2023-08-09T13:36:07.536 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-08-09T13:36:07.765 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:07.763+0000 7fd0f5491640 1 Processor -- start 2023-08-09T13:36:07.766 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:07.763+0000 7fd0f5491640 1 -- start start 2023-08-09T13:36:07.767 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:07.764+0000 7fd0f5491640 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7fd0f015c2d0 0x7fd0f00b7df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:36:07.768 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:07.764+0000 7fd0f5491640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7fd0f00d1b30 0x7fd0f00cd020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:36:07.768 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:07.764+0000 7fd0f5491640 1 -- --> v1:172.21.15.43:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd0f00d16a0 con 0x7fd0f00cebf0 2023-08-09T13:36:07.768 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:07.764+0000 7fd0f5491640 1 -- --> v2:172.21.15.43:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd0f00d1fa0 con 0x7fd0f00d1b30 2023-08-09T13:36:07.769 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:07.764+0000 7fd0f5491640 1 -- --> v2:172.21.15.203:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd0f00cd560 con 0x7fd0f015c2d0 2023-08-09T13:36:07.769 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:07.765+0000 7fd0ef7fe640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7fd0f00d1b30 0x7fd0f00cd020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:36:07.769 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:07.765+0000 7fd0ef7fe640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7fd0f00d1b30 0x7fd0f00cd020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:35886/0 (socket says 172.21.15.43:35886) 2023-08-09T13:36:07.769 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:07.765+0000 7fd0ef7fe640 1 -- 172.21.15.43:0/162389930 learned_addr learned my addr 172.21.15.43:0/162389930 (peer_addr_for_me v2:172.21.15.43:0/0) 2023-08-09T13:36:07.770 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:07.765+0000 7fd0effff640 1 --2- 172.21.15.43:0/162389930 >> v2:172.21.15.203:3300/0 conn(0x7fd0f015c2d0 0x7fd0f00b7df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:36:07.772 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:07.765+0000 7fd0ef7fe640 1 -- 172.21.15.43:0/162389930 >> v2:172.21.15.203:3300/0 conn(0x7fd0f015c2d0 msgr2=0x7fd0f00b7df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-09T13:36:07.772 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:07.765+0000 7fd0ef7fe640 1 --2- 172.21.15.43:0/162389930 >> v2:172.21.15.203:3300/0 conn(0x7fd0f015c2d0 0x7fd0f00b7df0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:36:07.772 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:07.765+0000 7fd0ef7fe640 1 -- 172.21.15.43:0/162389930 >> v1:172.21.15.43:6789/0 conn(0x7fd0f00cebf0 legacy=0x7fd0f00ceff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:36:07.772 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:07.765+0000 7fd0ef7fe640 1 -- 172.21.15.43:0/162389930 --> v2:172.21.15.43:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd0f00d0d70 con 0x7fd0f00d1b30 2023-08-09T13:36:07.772 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:07.767+0000 7fd0ef7fe640 1 --2- 172.21.15.43:0/162389930 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fd0f00d1b30 0x7fd0f00cd020 secure :-1 s=READY pgs=547 cs=0 l=1 rev1=1 crypto rx=0x7fd0f006ae00 tx=0x7fd0e402b9a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5c0f5386742d5d68 server_cookie=0 in_seq=0 out_seq=0 2023-08-09T13:36:07.773 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:07.767+0000 7fd0eeffd640 1 -- 172.21.15.43:0/162389930 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd0e4030860 con 0x7fd0f00d1b30 2023-08-09T13:36:07.773 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:07.767+0000 7fd0eeffd640 1 -- 172.21.15.43:0/162389930 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd0e40309c0 con 0x7fd0f00d1b30 2023-08-09T13:36:07.773 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:07.767+0000 7fd0eeffd640 1 -- 172.21.15.43:0/162389930 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd0e4030d50 con 0x7fd0f00d1b30 2023-08-09T13:36:07.773 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:07.767+0000 7fd0f5491640 1 -- 172.21.15.43:0/162389930 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fd0f00d1b30 msgr2=0x7fd0f00cd020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:36:07.773 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:07.767+0000 7fd0f5491640 1 --2- 172.21.15.43:0/162389930 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fd0f00d1b30 0x7fd0f00cd020 secure :-1 s=READY pgs=547 cs=0 l=1 rev1=1 crypto rx=0x7fd0f006ae00 tx=0x7fd0e402b9a0 comp rx=0 tx=0).stop 2023-08-09T13:36:07.773 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:07.767+0000 7fd0f5491640 1 -- 172.21.15.43:0/162389930 shutdown_connections 2023-08-09T13:36:07.773 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:07.767+0000 7fd0f5491640 1 --2- 172.21.15.43:0/162389930 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fd0f00d1b30 0x7fd0f00cd020 unknown :-1 s=CLOSED pgs=547 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:36:07.774 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:07.767+0000 7fd0f5491640 1 --2- 172.21.15.43:0/162389930 >> v2:172.21.15.203:3300/0 conn(0x7fd0f015c2d0 0x7fd0f00b7df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:36:07.774 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:07.767+0000 7fd0f5491640 1 -- 172.21.15.43:0/162389930 >> 172.21.15.43:0/162389930 conn(0x7fd0f006a230 msgr2=0x7fd0f006a640 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-09T13:36:07.774 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:07.768+0000 7fd0f5491640 1 -- 172.21.15.43:0/162389930 shutdown_connections 2023-08-09T13:36:07.774 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:07.768+0000 7fd0f5491640 1 -- 172.21.15.43:0/162389930 wait complete. 2023-08-09T13:36:07.774 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:07.769+0000 7fd0f5491640 1 Processor -- start 2023-08-09T13:36:07.774 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:07.769+0000 7fd0f5491640 1 -- start start 2023-08-09T13:36:07.778 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:07.770+0000 7fd0f5491640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fd0f00cebf0 0x7fd0f01f1020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:36:07.778 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:07.770+0000 7fd0f5491640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fd0f00d1b30 0x7fd0f00b6280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:36:07.778 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:07.770+0000 7fd0f5491640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fd0f015c2d0 0x7fd0f00b67c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:36:07.779 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:07.770+0000 7fd0f5491640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd0f014f010 con 0x7fd0f015c2d0 2023-08-09T13:36:07.779 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:07.770+0000 7fd0f5491640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd0f014c0f0 con 0x7fd0f00d1b30 2023-08-09T13:36:07.779 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:07.770+0000 7fd0f5491640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd0f015f430 con 0x7fd0f00cebf0 2023-08-09T13:36:07.779 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:07.771+0000 7fd0f4c90640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fd0f015c2d0 0x7fd0f00b67c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:36:07.780 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:07.771+0000 7fd0f4c90640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fd0f015c2d0 0x7fd0f00b67c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:35900/0 (socket says 172.21.15.43:35900) 2023-08-09T13:36:07.780 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:07.771+0000 7fd0f4c90640 1 -- 172.21.15.43:0/1718855335 learned_addr learned my addr 172.21.15.43:0/1718855335 (peer_addr_for_me v2:172.21.15.43:0/0) 2023-08-09T13:36:07.780 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:07.771+0000 7fd0effff640 1 --2- 172.21.15.43:0/1718855335 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fd0f00cebf0 0x7fd0f01f1020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:36:07.780 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:07.771+0000 7fd0ef7fe640 1 --2- 172.21.15.43:0/1718855335 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fd0f00d1b30 0x7fd0f00b6280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:36:07.781 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:07.771+0000 7fd0f4c90640 1 -- 172.21.15.43:0/1718855335 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fd0f00cebf0 msgr2=0x7fd0f01f1020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:36:07.781 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:07.771+0000 7fd0f4c90640 1 --2- 172.21.15.43:0/1718855335 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fd0f00cebf0 0x7fd0f01f1020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:36:07.781 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:07.771+0000 7fd0f4c90640 1 -- 172.21.15.43:0/1718855335 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fd0f00d1b30 msgr2=0x7fd0f00b6280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:36:07.781 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:07.771+0000 7fd0f4c90640 1 --2- 172.21.15.43:0/1718855335 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fd0f00d1b30 0x7fd0f00b6280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:36:07.782 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:07.771+0000 7fd0f4c90640 1 -- 172.21.15.43:0/1718855335 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd0e40094d0 con 0x7fd0f015c2d0 2023-08-09T13:36:07.782 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:07.771+0000 7fd0effff640 1 --2- 172.21.15.43:0/1718855335 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fd0f00cebf0 0x7fd0f01f1020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-08-09T13:36:07.782 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:07.771+0000 7fd0ef7fe640 1 --2- 172.21.15.43:0/1718855335 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fd0f00d1b30 0x7fd0f00b6280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-08-09T13:36:07.782 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:07.772+0000 7fd0f4c90640 1 --2- 172.21.15.43:0/1718855335 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fd0f015c2d0 0x7fd0f00b67c0 secure :-1 s=READY pgs=548 cs=0 l=1 rev1=1 crypto rx=0x7fd0e800c6b0 tx=0x7fd0e800cb70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-09T13:36:07.782 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:07.772+0000 7fd0ecff9640 1 -- 172.21.15.43:0/1718855335 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd0e8003b70 con 0x7fd0f015c2d0 2023-08-09T13:36:07.782 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:07.772+0000 7fd0ecff9640 1 -- 172.21.15.43:0/1718855335 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd0e8003cf0 con 0x7fd0f015c2d0 2023-08-09T13:36:07.783 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:07.772+0000 7fd0ecff9640 1 -- 172.21.15.43:0/1718855335 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd0e8003ea0 con 0x7fd0f015c2d0 2023-08-09T13:36:07.783 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:07.772+0000 7fd0f5491640 1 -- 172.21.15.43:0/1718855335 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd0f015f710 con 0x7fd0f015c2d0 2023-08-09T13:36:07.783 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:07.772+0000 7fd0f5491640 1 -- 172.21.15.43:0/1718855335 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd0f014c620 con 0x7fd0f015c2d0 2023-08-09T13:36:07.783 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:07.773+0000 7fd0f5491640 1 -- 172.21.15.43:0/1718855335 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd0f004e390 con 0x7fd0f015c2d0 2023-08-09T13:36:07.783 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:07.774+0000 7fd0ecff9640 1 -- 172.21.15.43:0/1718855335 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 5) v1 ==== 88341+0+0 (secure 0 0 0) 0x7fd0e8005310 con 0x7fd0f015c2d0 2023-08-09T13:36:07.783 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:07.776+0000 7fd0ecff9640 1 --2- 172.21.15.43:0/1718855335 >> [v2:172.21.15.203:6832/830097474,v1:172.21.15.203:6833/830097474] conn(0x7fd0dc06a050 0x7fd0dc06c510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:36:07.783 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:07.776+0000 7fd0ecff9640 1 -- 172.21.15.43:0/1718855335 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(35..35 src has 1..35) v4 ==== 5530+0+0 (secure 0 0 0) 0x7fd0e8088140 con 0x7fd0f015c2d0 2023-08-09T13:36:07.784 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:07.776+0000 7fd0effff640 1 --2- 172.21.15.43:0/1718855335 >> [v2:172.21.15.203:6832/830097474,v1:172.21.15.203:6833/830097474] conn(0x7fd0dc06a050 0x7fd0dc06c510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:36:07.784 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:07.777+0000 7fd0effff640 1 --2- 172.21.15.43:0/1718855335 >> [v2:172.21.15.203:6832/830097474,v1:172.21.15.203:6833/830097474] conn(0x7fd0dc06a050 0x7fd0dc06c510 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7fd0e0009ac0 tx=0x7fd0e0008040 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-09T13:36:07.784 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:07.780+0000 7fd0ecff9640 1 -- 172.21.15.43:0/1718855335 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178066 (secure 0 0 0) 0x7fd0e8053a30 con 0x7fd0f015c2d0 2023-08-09T13:36:07.984 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:07.981+0000 7fd0f5491640 1 -- 172.21.15.43:0/1718855335 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7fd0f00cfe00 con 0x7fd0f015c2d0 2023-08-09T13:36:07.985 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:07.982+0000 7fd0ecff9640 1 -- 172.21.15.43:0/1718855335 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v35) v1 ==== 93+0+10 (secure 0 0 0) 0x7fd0e8057050 con 0x7fd0f015c2d0 2023-08-09T13:36:07.985 INFO:teuthology.orchestra.run.smithi043.stdout:pg_num: 1 2023-08-09T13:36:07.988 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:07.985+0000 7fd0f5491640 1 -- 172.21.15.43:0/1718855335 >> [v2:172.21.15.203:6832/830097474,v1:172.21.15.203:6833/830097474] conn(0x7fd0dc06a050 msgr2=0x7fd0dc06c510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:36:07.989 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:07.985+0000 7fd0f5491640 1 --2- 172.21.15.43:0/1718855335 >> [v2:172.21.15.203:6832/830097474,v1:172.21.15.203:6833/830097474] conn(0x7fd0dc06a050 0x7fd0dc06c510 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7fd0e0009ac0 tx=0x7fd0e0008040 comp rx=0 tx=0).stop 2023-08-09T13:36:07.989 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:07.985+0000 7fd0f5491640 1 -- 172.21.15.43:0/1718855335 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fd0f015c2d0 msgr2=0x7fd0f00b67c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:36:07.989 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:07.985+0000 7fd0f5491640 1 --2- 172.21.15.43:0/1718855335 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fd0f015c2d0 0x7fd0f00b67c0 secure :-1 s=READY pgs=548 cs=0 l=1 rev1=1 crypto rx=0x7fd0e800c6b0 tx=0x7fd0e800cb70 comp rx=0 tx=0).stop 2023-08-09T13:36:07.989 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:07.986+0000 7fd0f5491640 1 -- 172.21.15.43:0/1718855335 shutdown_connections 2023-08-09T13:36:07.989 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:07.986+0000 7fd0f5491640 1 --2- 172.21.15.43:0/1718855335 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fd0f015c2d0 0x7fd0f00b67c0 unknown :-1 s=CLOSED pgs=548 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:36:07.989 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:07.986+0000 7fd0f5491640 1 --2- 172.21.15.43:0/1718855335 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fd0f00d1b30 0x7fd0f00b6280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:36:07.990 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:07.986+0000 7fd0f5491640 1 --2- 172.21.15.43:0/1718855335 >> [v2:172.21.15.203:6832/830097474,v1:172.21.15.203:6833/830097474] conn(0x7fd0dc06a050 0x7fd0dc06c510 unknown :-1 s=CLOSED pgs=266 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:36:07.990 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:07.986+0000 7fd0f5491640 1 --2- 172.21.15.43:0/1718855335 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fd0f00cebf0 0x7fd0f01f1020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:36:07.990 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:07.986+0000 7fd0f5491640 1 -- 172.21.15.43:0/1718855335 >> 172.21.15.43:0/1718855335 conn(0x7fd0f006a230 msgr2=0x7fd0f00c76d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-09T13:36:07.990 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:07.986+0000 7fd0f5491640 1 -- 172.21.15.43:0/1718855335 shutdown_connections 2023-08-09T13:36:07.990 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:07.986+0000 7fd0f5491640 1 -- 172.21.15.43:0/1718855335 wait complete. 2023-08-09T13:36:08.008 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --format=json-pretty osd lspools 2023-08-09T13:36:08.234 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:08.232+0000 7f223359e640 1 Processor -- start 2023-08-09T13:36:08.235 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:08.232+0000 7f223359e640 1 -- start start 2023-08-09T13:36:08.236 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:08.234+0000 7f223359e640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f22340c8b30 0x7f22340ad340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:36:08.237 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:08.234+0000 7f223359e640 1 -- --> v1:172.21.15.43:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f22340cbd90 con 0x7f22340acdf0 2023-08-09T13:36:08.237 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:08.234+0000 7f223359e640 1 -- --> v1:172.21.15.203:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f22340ca2b0 con 0x7f22340c8750 2023-08-09T13:36:08.237 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:08.234+0000 7f223359e640 1 -- --> v2:172.21.15.43:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f223415e810 con 0x7f22340c8b30 2023-08-09T13:36:08.237 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:08.234+0000 7f2231d9b640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f22340c8b30 0x7f22340ad340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:36:08.237 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:08.234+0000 7f2232d9d640 1 --1- >> v1:172.21.15.43:6789/0 conn(0x7f22340acdf0 0x7f2234070f00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.43:6789/0 says I am v1:172.21.15.43:39438/0 (socket says 172.21.15.43:39438) 2023-08-09T13:36:08.237 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:08.234+0000 7f2232d9d640 1 -- 172.21.15.43:0/605997213 learned_addr learned my addr 172.21.15.43:0/605997213 (peer_addr_for_me v1:172.21.15.43:0/0) 2023-08-09T13:36:08.238 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:08.235+0000 7f2231d9b640 1 -- 172.21.15.43:0/605997213 >> v1:172.21.15.203:6789/0 conn(0x7f22340c8750 legacy=0x7f223406f410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-09T13:36:08.238 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:08.235+0000 7f2231d9b640 1 -- 172.21.15.43:0/605997213 >> v1:172.21.15.43:6789/0 conn(0x7f22340acdf0 legacy=0x7f2234070f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:36:08.238 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:08.235+0000 7f2231d9b640 1 -- 172.21.15.43:0/605997213 --> v2:172.21.15.43:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f22340c9a20 con 0x7f22340c8b30 2023-08-09T13:36:08.238 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:08.235+0000 7f223159a640 1 -- 172.21.15.43:0/605997213 <== mon.0 v1:172.21.15.43:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2790672858 0 0) 0x7f22340cbd90 con 0x7f22340acdf0 2023-08-09T13:36:08.239 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:08.236+0000 7f2231d9b640 1 --2- 172.21.15.43:0/605997213 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f22340c8b30 0x7f22340ad340 secure :-1 s=READY pgs=550 cs=0 l=1 rev1=1 crypto rx=0x7f221c009d90 tx=0x7f221c02be20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f98428cbc118010c server_cookie=0 in_seq=0 out_seq=0 2023-08-09T13:36:08.239 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:08.236+0000 7f223159a640 1 -- 172.21.15.43:0/605997213 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f221c02cbd0 con 0x7f22340c8b30 2023-08-09T13:36:08.239 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:08.236+0000 7f223159a640 1 -- 172.21.15.43:0/605997213 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f221c02cd30 con 0x7f22340c8b30 2023-08-09T13:36:08.239 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:08.236+0000 7f223159a640 1 -- 172.21.15.43:0/605997213 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f221c02cea0 con 0x7f22340c8b30 2023-08-09T13:36:08.240 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:08.237+0000 7f223359e640 1 -- 172.21.15.43:0/605997213 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f22340c8b30 msgr2=0x7f22340ad340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:36:08.240 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:08.237+0000 7f223359e640 1 --2- 172.21.15.43:0/605997213 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f22340c8b30 0x7f22340ad340 secure :-1 s=READY pgs=550 cs=0 l=1 rev1=1 crypto rx=0x7f221c009d90 tx=0x7f221c02be20 comp rx=0 tx=0).stop 2023-08-09T13:36:08.240 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:08.237+0000 7f223359e640 1 -- 172.21.15.43:0/605997213 shutdown_connections 2023-08-09T13:36:08.241 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:08.237+0000 7f223359e640 1 --2- 172.21.15.43:0/605997213 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f22340c8b30 0x7f22340ad340 unknown :-1 s=CLOSED pgs=550 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:36:08.241 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:08.237+0000 7f223359e640 1 -- 172.21.15.43:0/605997213 >> 172.21.15.43:0/605997213 conn(0x7f22340cde90 msgr2=0x7f22340ce2a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-09T13:36:08.241 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:08.237+0000 7f223359e640 1 -- 172.21.15.43:0/605997213 shutdown_connections 2023-08-09T13:36:08.241 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:08.237+0000 7f223359e640 1 -- 172.21.15.43:0/605997213 wait complete. 2023-08-09T13:36:08.241 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:08.239+0000 7f223359e640 1 Processor -- start 2023-08-09T13:36:08.242 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:08.239+0000 7f223359e640 1 -- start start 2023-08-09T13:36:08.243 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:08.240+0000 7f223359e640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f22340acdf0 0x7f223415a4f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:36:08.243 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:08.240+0000 7f223359e640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f22340c8750 0x7f22340c6b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:36:08.243 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:08.240+0000 7f223359e640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f22340c8b30 0x7f22340abdd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:36:08.243 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:08.240+0000 7f223359e640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f22340c9f70 con 0x7f22340c8b30 2023-08-09T13:36:08.243 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:08.240+0000 7f223359e640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f22340c5e30 con 0x7f22340c8750 2023-08-09T13:36:08.244 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:08.240+0000 7f223359e640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f22340c4f60 con 0x7f22340acdf0 2023-08-09T13:36:08.244 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:08.241+0000 7f2232d9d640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f22340c8b30 0x7f22340abdd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:36:08.244 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:08.241+0000 7f2232d9d640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f22340c8b30 0x7f22340abdd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:35924/0 (socket says 172.21.15.43:35924) 2023-08-09T13:36:08.244 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:08.241+0000 7f2232d9d640 1 -- 172.21.15.43:0/3436254237 learned_addr learned my addr 172.21.15.43:0/3436254237 (peer_addr_for_me v2:172.21.15.43:0/0) 2023-08-09T13:36:08.245 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:08.241+0000 7f2231d9b640 1 --2- 172.21.15.43:0/3436254237 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f22340c8750 0x7f22340c6b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:36:08.245 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:08.241+0000 7f223259c640 1 --2- 172.21.15.43:0/3436254237 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f22340acdf0 0x7f223415a4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:36:08.245 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:08.241+0000 7f2232d9d640 1 -- 172.21.15.43:0/3436254237 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f22340acdf0 msgr2=0x7f223415a4f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:36:08.245 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:08.241+0000 7f2232d9d640 1 --2- 172.21.15.43:0/3436254237 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f22340acdf0 0x7f223415a4f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:36:08.246 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:08.241+0000 7f2232d9d640 1 -- 172.21.15.43:0/3436254237 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f22340c8750 msgr2=0x7f22340c6b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:36:08.246 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:08.241+0000 7f2232d9d640 1 --2- 172.21.15.43:0/3436254237 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f22340c8750 0x7f22340c6b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:36:08.246 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:08.241+0000 7f2232d9d640 1 -- 172.21.15.43:0/3436254237 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f221c009a70 con 0x7f22340c8b30 2023-08-09T13:36:08.246 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:08.242+0000 7f2232d9d640 1 --2- 172.21.15.43:0/3436254237 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f22340c8b30 0x7f22340abdd0 secure :-1 s=READY pgs=551 cs=0 l=1 rev1=1 crypto rx=0x7f222c00c450 tx=0x7f222c00c910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-09T13:36:08.246 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:08.242+0000 7f222affd640 1 -- 172.21.15.43:0/3436254237 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f222c0074e0 con 0x7f22340c8b30 2023-08-09T13:36:08.246 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:08.242+0000 7f222affd640 1 -- 172.21.15.43:0/3436254237 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f222c007660 con 0x7f22340c8b30 2023-08-09T13:36:08.246 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:08.242+0000 7f222affd640 1 -- 172.21.15.43:0/3436254237 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f222c004a20 con 0x7f22340c8b30 2023-08-09T13:36:08.247 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:08.242+0000 7f223359e640 1 -- 172.21.15.43:0/3436254237 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f223415aa30 con 0x7f22340c8b30 2023-08-09T13:36:08.247 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:08.242+0000 7f223359e640 1 -- 172.21.15.43:0/3436254237 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f22340c51b0 con 0x7f22340c8b30 2023-08-09T13:36:08.247 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:08.243+0000 7f223359e640 1 -- 172.21.15.43:0/3436254237 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f22340c5f70 con 0x7f22340c8b30 2023-08-09T13:36:08.247 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:08.244+0000 7f222affd640 1 -- 172.21.15.43:0/3436254237 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 5) v1 ==== 88341+0+0 (secure 0 0 0) 0x7f222c004b80 con 0x7f22340c8b30 2023-08-09T13:36:08.248 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:08.245+0000 7f222affd640 1 --2- 172.21.15.43:0/3436254237 >> [v2:172.21.15.203:6832/830097474,v1:172.21.15.203:6833/830097474] conn(0x7f221406a050 0x7f221406c510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:36:08.248 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:08.245+0000 7f222affd640 1 -- 172.21.15.43:0/3436254237 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(35..35 src has 1..35) v4 ==== 5530+0+0 (secure 0 0 0) 0x7f222c0874f0 con 0x7f22340c8b30 2023-08-09T13:36:08.248 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:08.246+0000 7f223259c640 1 --2- 172.21.15.43:0/3436254237 >> [v2:172.21.15.203:6832/830097474,v1:172.21.15.203:6833/830097474] conn(0x7f221406a050 0x7f221406c510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:36:08.249 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:08.247+0000 7f223259c640 1 --2- 172.21.15.43:0/3436254237 >> [v2:172.21.15.203:6832/830097474,v1:172.21.15.203:6833/830097474] conn(0x7f221406a050 0x7f221406c510 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7f2224004ea0 tx=0x7f2224009040 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-09T13:36:08.251 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:08.249+0000 7f222affd640 1 -- 172.21.15.43:0/3436254237 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178066 (secure 0 0 0) 0x7f222c052d50 con 0x7f22340c8b30 2023-08-09T13:36:08.453 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:08.450+0000 7f223359e640 1 -- 172.21.15.43:0/3436254237 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) v1 -- 0x7f22340c9230 con 0x7f22340c8b30 2023-08-09T13:36:08.454 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:08.451+0000 7f222affd640 1 -- 172.21.15.43:0/3436254237 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v35) v1 ==== 84+0+126 (secure 0 0 0) 0x7f222c056370 con 0x7f22340c8b30 2023-08-09T13:36:08.454 INFO:teuthology.orchestra.run.smithi043.stdout: 2023-08-09T13:36:08.454 INFO:teuthology.orchestra.run.smithi043.stdout:[ 2023-08-09T13:36:08.454 INFO:teuthology.orchestra.run.smithi043.stdout: { 2023-08-09T13:36:08.455 INFO:teuthology.orchestra.run.smithi043.stdout: "poolnum": 1, 2023-08-09T13:36:08.455 INFO:teuthology.orchestra.run.smithi043.stdout: "poolname": "rbd" 2023-08-09T13:36:08.455 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2023-08-09T13:36:08.456 INFO:teuthology.orchestra.run.smithi043.stdout: { 2023-08-09T13:36:08.456 INFO:teuthology.orchestra.run.smithi043.stdout: "poolnum": 2, 2023-08-09T13:36:08.456 INFO:teuthology.orchestra.run.smithi043.stdout: "poolname": ".mgr" 2023-08-09T13:36:08.457 INFO:teuthology.orchestra.run.smithi043.stdout: } 2023-08-09T13:36:08.457 INFO:teuthology.orchestra.run.smithi043.stdout:] 2023-08-09T13:36:08.458 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:08.454+0000 7f223359e640 1 -- 172.21.15.43:0/3436254237 >> [v2:172.21.15.203:6832/830097474,v1:172.21.15.203:6833/830097474] conn(0x7f221406a050 msgr2=0x7f221406c510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:36:08.458 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:08.454+0000 7f223359e640 1 --2- 172.21.15.43:0/3436254237 >> [v2:172.21.15.203:6832/830097474,v1:172.21.15.203:6833/830097474] conn(0x7f221406a050 0x7f221406c510 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7f2224004ea0 tx=0x7f2224009040 comp rx=0 tx=0).stop 2023-08-09T13:36:08.459 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:08.454+0000 7f223359e640 1 -- 172.21.15.43:0/3436254237 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f22340c8b30 msgr2=0x7f22340abdd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:36:08.459 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:08.454+0000 7f223359e640 1 --2- 172.21.15.43:0/3436254237 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f22340c8b30 0x7f22340abdd0 secure :-1 s=READY pgs=551 cs=0 l=1 rev1=1 crypto rx=0x7f222c00c450 tx=0x7f222c00c910 comp rx=0 tx=0).stop 2023-08-09T13:36:08.459 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:08.454+0000 7f223359e640 1 -- 172.21.15.43:0/3436254237 shutdown_connections 2023-08-09T13:36:08.459 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:08.454+0000 7f223359e640 1 --2- 172.21.15.43:0/3436254237 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f22340c8b30 0x7f22340abdd0 unknown :-1 s=CLOSED pgs=551 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:36:08.460 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:08.454+0000 7f223359e640 1 --2- 172.21.15.43:0/3436254237 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f22340c8750 0x7f22340c6b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:36:08.460 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:08.454+0000 7f223359e640 1 --2- 172.21.15.43:0/3436254237 >> [v2:172.21.15.203:6832/830097474,v1:172.21.15.203:6833/830097474] conn(0x7f221406a050 0x7f221406c510 unknown :-1 s=CLOSED pgs=267 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:36:08.460 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:08.454+0000 7f223359e640 1 --2- 172.21.15.43:0/3436254237 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f22340acdf0 0x7f223415a4f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:36:08.460 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:08.454+0000 7f223359e640 1 -- 172.21.15.43:0/3436254237 >> 172.21.15.43:0/3436254237 conn(0x7f22340cde90 msgr2=0x7f22340c0cc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-09T13:36:08.460 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:08.454+0000 7f223359e640 1 -- 172.21.15.43:0/3436254237 shutdown_connections 2023-08-09T13:36:08.461 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:08.454+0000 7f223359e640 1 -- 172.21.15.43:0/3436254237 wait complete. 2023-08-09T13:36:08.475 DEBUG:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2023-08-09T13:36:08.475 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_metadata --pg_num_min 64 2023-08-09T13:36:08.699 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:08.696+0000 7f73bf2d7640 1 Processor -- start 2023-08-09T13:36:08.700 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:08.697+0000 7f73bf2d7640 1 -- start start 2023-08-09T13:36:08.701 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:08.698+0000 7f73bf2d7640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f73b80cd3c0 0x7f73b80cec70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:36:08.701 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:08.698+0000 7f73bf2d7640 1 -- --> v1:172.21.15.43:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f73b80cf5e0 con 0x7f73b815c400 2023-08-09T13:36:08.701 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:08.698+0000 7f73bf2d7640 1 -- --> v1:172.21.15.203:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f73b80d13e0 con 0x7f73b80cf1b0 2023-08-09T13:36:08.701 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:08.698+0000 7f73bf2d7640 1 -- --> v2:172.21.15.43:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f73b815c7e0 con 0x7f73b80cd3c0 2023-08-09T13:36:08.702 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:08.699+0000 7f73be2d5640 1 --1- >> v1:172.21.15.43:6789/0 conn(0x7f73b815c400 0x7f73b80d0cd0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.43:6789/0 says I am v1:172.21.15.43:39452/0 (socket says 172.21.15.43:39452) 2023-08-09T13:36:08.702 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:08.699+0000 7f73be2d5640 1 -- 172.21.15.43:0/3060450712 learned_addr learned my addr 172.21.15.43:0/3060450712 (peer_addr_for_me v1:172.21.15.43:0/0) 2023-08-09T13:36:08.702 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:08.699+0000 7f73bdad4640 1 --2- 172.21.15.43:0/3060450712 >> v2:172.21.15.43:3300/0 conn(0x7f73b80cd3c0 0x7f73b80cec70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:36:08.702 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:08.699+0000 7f73bdad4640 1 -- 172.21.15.43:0/3060450712 >> v1:172.21.15.203:6789/0 conn(0x7f73b80cf1b0 legacy=0x7f73b80b5880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:36:08.702 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:08.699+0000 7f73bdad4640 1 -- 172.21.15.43:0/3060450712 >> v1:172.21.15.43:6789/0 conn(0x7f73b815c400 legacy=0x7f73b80d0cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:36:08.703 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:08.700+0000 7f73bdad4640 1 -- 172.21.15.43:0/3060450712 --> v2:172.21.15.43:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f73b80d1690 con 0x7f73b80cd3c0 2023-08-09T13:36:08.703 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:08.700+0000 7f73bd2d3640 1 -- 172.21.15.43:0/3060450712 <== mon.0 v1:172.21.15.43:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3202396410 0 0) 0x7f73b80cf5e0 con 0x7f73b815c400 2023-08-09T13:36:08.703 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:08.701+0000 7f73bdad4640 1 --2- 172.21.15.43:0/3060450712 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f73b80cd3c0 0x7f73b80cec70 secure :-1 s=READY pgs=553 cs=0 l=1 rev1=1 crypto rx=0x7f73a80097a0 tx=0x7f73a802b9f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e2cc0b0a78a54b4f server_cookie=0 in_seq=0 out_seq=0 2023-08-09T13:36:08.704 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:08.701+0000 7f73bd2d3640 1 -- 172.21.15.43:0/3060450712 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f73a802f810 con 0x7f73b80cd3c0 2023-08-09T13:36:08.704 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:08.701+0000 7f73bd2d3640 1 -- 172.21.15.43:0/3060450712 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f73a802f970 con 0x7f73b80cd3c0 2023-08-09T13:36:08.704 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:08.701+0000 7f73bd2d3640 1 -- 172.21.15.43:0/3060450712 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f73a802fd00 con 0x7f73b80cd3c0 2023-08-09T13:36:08.705 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:08.702+0000 7f73bf2d7640 1 -- 172.21.15.43:0/3060450712 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f73b80cd3c0 msgr2=0x7f73b80cec70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:36:08.705 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:08.702+0000 7f73bf2d7640 1 --2- 172.21.15.43:0/3060450712 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f73b80cd3c0 0x7f73b80cec70 secure :-1 s=READY pgs=553 cs=0 l=1 rev1=1 crypto rx=0x7f73a80097a0 tx=0x7f73a802b9f0 comp rx=0 tx=0).stop 2023-08-09T13:36:08.705 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:08.702+0000 7f73bf2d7640 1 -- 172.21.15.43:0/3060450712 shutdown_connections 2023-08-09T13:36:08.705 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:08.702+0000 7f73bf2d7640 1 --2- 172.21.15.43:0/3060450712 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f73b80cd3c0 0x7f73b80cec70 unknown :-1 s=CLOSED pgs=553 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:36:08.705 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:08.702+0000 7f73bf2d7640 1 -- 172.21.15.43:0/3060450712 >> 172.21.15.43:0/3060450712 conn(0x7f73b806a320 msgr2=0x7f73b806a730 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-09T13:36:08.705 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:08.702+0000 7f73bf2d7640 1 -- 172.21.15.43:0/3060450712 shutdown_connections 2023-08-09T13:36:08.705 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:08.702+0000 7f73bf2d7640 1 -- 172.21.15.43:0/3060450712 wait complete. 2023-08-09T13:36:08.706 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:08.704+0000 7f73bf2d7640 1 Processor -- start 2023-08-09T13:36:08.706 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:08.704+0000 7f73bf2d7640 1 -- start start 2023-08-09T13:36:08.707 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:08.705+0000 7f73bf2d7640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f73b80cd3c0 0x7f73b8152310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:36:08.708 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:08.705+0000 7f73bf2d7640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f73b80cf1b0 0x7f73b8152850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:36:08.708 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:08.705+0000 7f73bf2d7640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f73b815c400 0x7f73b8154430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:36:08.708 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:08.705+0000 7f73bf2d7640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f73b8152d90 con 0x7f73b815c400 2023-08-09T13:36:08.708 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:08.705+0000 7f73bf2d7640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f73b80ca3a0 con 0x7f73b80cf1b0 2023-08-09T13:36:08.708 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:08.705+0000 7f73bf2d7640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f73b8155200 con 0x7f73b80cd3c0 2023-08-09T13:36:08.708 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:08.705+0000 7f73bead6640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f73b815c400 0x7f73b8154430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:36:08.709 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:08.705+0000 7f73bead6640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f73b815c400 0x7f73b8154430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:35944/0 (socket says 172.21.15.43:35944) 2023-08-09T13:36:08.709 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:08.705+0000 7f73bead6640 1 -- 172.21.15.43:0/2128414648 learned_addr learned my addr 172.21.15.43:0/2128414648 (peer_addr_for_me v2:172.21.15.43:0/0) 2023-08-09T13:36:08.710 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:08.706+0000 7f73bdad4640 1 --2- 172.21.15.43:0/2128414648 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f73b80cf1b0 0x7f73b8152850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:36:08.710 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:08.706+0000 7f73be2d5640 1 --2- 172.21.15.43:0/2128414648 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f73b80cd3c0 0x7f73b8152310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:36:08.710 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:08.706+0000 7f73bead6640 1 -- 172.21.15.43:0/2128414648 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f73b80cd3c0 msgr2=0x7f73b8152310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:36:08.710 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:08.706+0000 7f73bead6640 1 --2- 172.21.15.43:0/2128414648 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f73b80cd3c0 0x7f73b8152310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:36:08.711 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:08.706+0000 7f73bead6640 1 -- 172.21.15.43:0/2128414648 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f73b80cf1b0 msgr2=0x7f73b8152850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:36:08.711 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:08.706+0000 7f73bead6640 1 --2- 172.21.15.43:0/2128414648 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f73b80cf1b0 0x7f73b8152850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:36:08.711 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:08.706+0000 7f73bead6640 1 -- 172.21.15.43:0/2128414648 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f73a8009400 con 0x7f73b815c400 2023-08-09T13:36:08.711 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:08.706+0000 7f73bdad4640 1 --2- 172.21.15.43:0/2128414648 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f73b80cf1b0 0x7f73b8152850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-08-09T13:36:08.711 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:08.706+0000 7f73bead6640 1 --2- 172.21.15.43:0/2128414648 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f73b815c400 0x7f73b8154430 secure :-1 s=READY pgs=554 cs=0 l=1 rev1=1 crypto rx=0x7f73b0001eb0 tx=0x7f73b000c890 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-09T13:36:08.712 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:08.706+0000 7f73aeffd640 1 -- 172.21.15.43:0/2128414648 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f73b00052d0 con 0x7f73b815c400 2023-08-09T13:36:08.712 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:08.706+0000 7f73aeffd640 1 -- 172.21.15.43:0/2128414648 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f73b0005430 con 0x7f73b815c400 2023-08-09T13:36:08.712 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:08.706+0000 7f73aeffd640 1 -- 172.21.15.43:0/2128414648 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f73b0003810 con 0x7f73b815c400 2023-08-09T13:36:08.712 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:08.707+0000 7f73bf2d7640 1 -- 172.21.15.43:0/2128414648 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f73b81554e0 con 0x7f73b815c400 2023-08-09T13:36:08.712 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:08.707+0000 7f73bf2d7640 1 -- 172.21.15.43:0/2128414648 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f73b80ca890 con 0x7f73b815c400 2023-08-09T13:36:08.713 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:08.708+0000 7f73bf2d7640 1 -- 172.21.15.43:0/2128414648 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f73b81556d0 con 0x7f73b815c400 2023-08-09T13:36:08.713 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:08.709+0000 7f73aeffd640 1 -- 172.21.15.43:0/2128414648 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 5) v1 ==== 88341+0+0 (secure 0 0 0) 0x7f73b0003970 con 0x7f73b815c400 2023-08-09T13:36:08.713 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:08.710+0000 7f73aeffd640 1 --2- 172.21.15.43:0/2128414648 >> [v2:172.21.15.203:6832/830097474,v1:172.21.15.203:6833/830097474] conn(0x7f739c06a050 0x7f739c06c510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:36:08.713 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:08.710+0000 7f73aeffd640 1 -- 172.21.15.43:0/2128414648 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(35..35 src has 1..35) v4 ==== 5530+0+0 (secure 0 0 0) 0x7f73b0087a00 con 0x7f73b815c400 2023-08-09T13:36:08.713 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:08.710+0000 7f73be2d5640 1 --2- 172.21.15.43:0/2128414648 >> [v2:172.21.15.203:6832/830097474,v1:172.21.15.203:6833/830097474] conn(0x7f739c06a050 0x7f739c06c510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:36:08.714 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:08.712+0000 7f73be2d5640 1 --2- 172.21.15.43:0/2128414648 >> [v2:172.21.15.203:6832/830097474,v1:172.21.15.203:6833/830097474] conn(0x7f739c06a050 0x7f739c06c510 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7f73b4007fd0 tx=0x7f73b4009040 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-09T13:36:08.717 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:08.714+0000 7f73aeffd640 1 -- 172.21.15.43:0/2128414648 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178066 (secure 0 0 0) 0x7f73b81556d0 con 0x7f73b815c400 2023-08-09T13:36:08.914 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:08.912+0000 7f73bf2d7640 1 -- 172.21.15.43:0/2128414648 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64} v 0) v1 -- 0x7f73b81556d0 con 0x7f73b815c400 2023-08-09T13:36:09.833 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:09.830+0000 7f73aeffd640 1 -- 172.21.15.43:0/2128414648 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]=0 pool 'cephfs_metadata' created v36) v1 ==== 138+0+0 (secure 0 0 0) 0x7f73b81556d0 con 0x7f73b815c400 2023-08-09T13:36:09.834 INFO:teuthology.orchestra.run.smithi043.stderr:pool 'cephfs_metadata' created 2023-08-09T13:36:09.838 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:09.836+0000 7f73bf2d7640 1 -- 172.21.15.43:0/2128414648 >> [v2:172.21.15.203:6832/830097474,v1:172.21.15.203:6833/830097474] conn(0x7f739c06a050 msgr2=0x7f739c06c510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:36:09.838 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:09.836+0000 7f73bf2d7640 1 --2- 172.21.15.43:0/2128414648 >> [v2:172.21.15.203:6832/830097474,v1:172.21.15.203:6833/830097474] conn(0x7f739c06a050 0x7f739c06c510 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7f73b4007fd0 tx=0x7f73b4009040 comp rx=0 tx=0).stop 2023-08-09T13:36:09.839 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:09.836+0000 7f73bf2d7640 1 -- 172.21.15.43:0/2128414648 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f73b815c400 msgr2=0x7f73b8154430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:36:09.839 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:09.836+0000 7f73bf2d7640 1 --2- 172.21.15.43:0/2128414648 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f73b815c400 0x7f73b8154430 secure :-1 s=READY pgs=554 cs=0 l=1 rev1=1 crypto rx=0x7f73b0001eb0 tx=0x7f73b000c890 comp rx=0 tx=0).stop 2023-08-09T13:36:09.839 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:09.836+0000 7f73bf2d7640 1 -- 172.21.15.43:0/2128414648 shutdown_connections 2023-08-09T13:36:09.840 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:09.836+0000 7f73bf2d7640 1 --2- 172.21.15.43:0/2128414648 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f73b815c400 0x7f73b8154430 unknown :-1 s=CLOSED pgs=554 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:36:09.840 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:09.836+0000 7f73bf2d7640 1 --2- 172.21.15.43:0/2128414648 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f73b80cf1b0 0x7f73b8152850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:36:09.840 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:09.836+0000 7f73bf2d7640 1 --2- 172.21.15.43:0/2128414648 >> [v2:172.21.15.203:6832/830097474,v1:172.21.15.203:6833/830097474] conn(0x7f739c06a050 0x7f739c06c510 unknown :-1 s=CLOSED pgs=268 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:36:09.840 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:09.836+0000 7f73bf2d7640 1 --2- 172.21.15.43:0/2128414648 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f73b80cd3c0 0x7f73b8152310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:36:09.840 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:09.836+0000 7f73bf2d7640 1 -- 172.21.15.43:0/2128414648 >> 172.21.15.43:0/2128414648 conn(0x7f73b806a320 msgr2=0x7f73b80b5780 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-09T13:36:09.841 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:09.838+0000 7f73bf2d7640 1 -- 172.21.15.43:0/2128414648 shutdown_connections 2023-08-09T13:36:09.841 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:09.838+0000 7f73bf2d7640 1 -- 172.21.15.43:0/2128414648 wait complete. 2023-08-09T13:36:09.865 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data 64 --pg_num_min 64 --target_size_ratio 0.9 2023-08-09T13:36:10.099 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:10.096+0000 7f6011bd6640 1 Processor -- start 2023-08-09T13:36:10.099 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:10.096+0000 7f6011bd6640 1 -- start start 2023-08-09T13:36:10.100 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:10.098+0000 7f6011bd6640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f600c157ea0 0x7f600c0c97d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:36:10.101 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:10.098+0000 7f6011bd6640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f600c0c90a0 0x7f600c0c9d10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:36:10.101 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:10.098+0000 7f6011bd6640 1 -- --> v1:172.21.15.43:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f600c158280 con 0x7f600c15a800 2023-08-09T13:36:10.101 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:10.098+0000 7f6011bd6640 1 -- --> v2:172.21.15.43:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f600c15b380 con 0x7f600c157ea0 2023-08-09T13:36:10.101 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:10.098+0000 7f6011bd6640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f600c15b840 con 0x7f600c0c90a0 2023-08-09T13:36:10.102 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:10.098+0000 7f6010bd4640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f600c157ea0 0x7f600c0c97d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:36:10.102 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:10.099+0000 7f6010bd4640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f600c157ea0 0x7f600c0c97d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:35958/0 (socket says 172.21.15.43:35958) 2023-08-09T13:36:10.102 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:10.099+0000 7f6010bd4640 1 -- 172.21.15.43:0/1077368661 learned_addr learned my addr 172.21.15.43:0/1077368661 (peer_addr_for_me v2:172.21.15.43:0/0) 2023-08-09T13:36:10.102 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:10.099+0000 7f600bfff640 1 --2- 172.21.15.43:0/1077368661 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f600c0c90a0 0x7f600c0c9d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:36:10.102 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:10.099+0000 7f6010bd4640 1 -- 172.21.15.43:0/1077368661 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f600c0c90a0 msgr2=0x7f600c0c9d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-09T13:36:10.103 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:10.099+0000 7f6010bd4640 1 --2- 172.21.15.43:0/1077368661 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f600c0c90a0 0x7f600c0c9d10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:36:10.103 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:10.099+0000 7f6010bd4640 1 -- 172.21.15.43:0/1077368661 >> v1:172.21.15.43:6789/0 conn(0x7f600c15a800 legacy=0x7f600c15ac70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:36:10.103 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:10.099+0000 7f6010bd4640 1 -- 172.21.15.43:0/1077368661 --> v2:172.21.15.43:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f600c15b560 con 0x7f600c157ea0 2023-08-09T13:36:10.104 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:10.100+0000 7f6010bd4640 1 --2- 172.21.15.43:0/1077368661 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f600c157ea0 0x7f600c0c97d0 secure :-1 s=READY pgs=556 cs=0 l=1 rev1=1 crypto rx=0x7f6000009870 tx=0x7f600002b8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=93f29f6b6bb87ae0 server_cookie=0 in_seq=0 out_seq=0 2023-08-09T13:36:10.104 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:10.101+0000 7f600b7fe640 1 -- 172.21.15.43:0/1077368661 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6000031640 con 0x7f600c157ea0 2023-08-09T13:36:10.104 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:10.101+0000 7f600b7fe640 1 -- 172.21.15.43:0/1077368661 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f60000317a0 con 0x7f600c157ea0 2023-08-09T13:36:10.105 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:10.101+0000 7f600b7fe640 1 -- 172.21.15.43:0/1077368661 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6000031b30 con 0x7f600c157ea0 2023-08-09T13:36:10.105 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:10.101+0000 7f6011bd6640 1 -- 172.21.15.43:0/1077368661 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f600c157ea0 msgr2=0x7f600c0c97d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:36:10.105 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:10.101+0000 7f6011bd6640 1 --2- 172.21.15.43:0/1077368661 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f600c157ea0 0x7f600c0c97d0 secure :-1 s=READY pgs=556 cs=0 l=1 rev1=1 crypto rx=0x7f6000009870 tx=0x7f600002b8f0 comp rx=0 tx=0).stop 2023-08-09T13:36:10.105 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:10.102+0000 7f6011bd6640 1 -- 172.21.15.43:0/1077368661 shutdown_connections 2023-08-09T13:36:10.105 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:10.102+0000 7f6011bd6640 1 --2- 172.21.15.43:0/1077368661 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f600c0c90a0 0x7f600c0c9d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:36:10.106 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:10.102+0000 7f6011bd6640 1 --2- 172.21.15.43:0/1077368661 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f600c157ea0 0x7f600c0c97d0 unknown :-1 s=CLOSED pgs=556 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:36:10.106 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:10.102+0000 7f6011bd6640 1 -- 172.21.15.43:0/1077368661 >> 172.21.15.43:0/1077368661 conn(0x7f600c0cddd0 msgr2=0x7f600c0ce1e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-09T13:36:10.106 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:10.102+0000 7f6011bd6640 1 -- 172.21.15.43:0/1077368661 shutdown_connections 2023-08-09T13:36:10.107 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:10.102+0000 7f6011bd6640 1 -- 172.21.15.43:0/1077368661 wait complete. 2023-08-09T13:36:10.110 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:10.103+0000 7f6011bd6640 1 Processor -- start 2023-08-09T13:36:10.110 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:10.104+0000 7f6011bd6640 1 -- start start 2023-08-09T13:36:10.111 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:10.105+0000 7f6011bd6640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f600c0c90a0 0x7f600c0c4220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:36:10.111 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:10.105+0000 7f6011bd6640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f600c157ea0 0x7f600c1efa50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:36:10.111 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:10.105+0000 7f6011bd6640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f600c15a800 0x7f600c0c5ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:36:10.112 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:10.105+0000 7f6011bd6640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f600c0c2370 con 0x7f600c0c90a0 2023-08-09T13:36:10.112 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:10.105+0000 7f6011bd6640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f600c0c1d30 con 0x7f600c15a800 2023-08-09T13:36:10.113 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:10.105+0000 7f6011bd6640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f600c0c6990 con 0x7f600c157ea0 2023-08-09T13:36:10.113 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:10.105+0000 7f6010bd4640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f600c0c90a0 0x7f600c0c4220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:36:10.113 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:10.105+0000 7f6010bd4640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f600c0c90a0 0x7f600c0c4220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:35964/0 (socket says 172.21.15.43:35964) 2023-08-09T13:36:10.113 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:10.105+0000 7f6010bd4640 1 -- 172.21.15.43:0/4045121958 learned_addr learned my addr 172.21.15.43:0/4045121958 (peer_addr_for_me v2:172.21.15.43:0/0) 2023-08-09T13:36:10.113 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:10.105+0000 7f6010bd4640 1 -- 172.21.15.43:0/4045121958 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f600c157ea0 msgr2=0x7f600c1efa50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:36:10.114 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:10.105+0000 7f600bfff640 1 --2- 172.21.15.43:0/4045121958 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f600c157ea0 0x7f600c1efa50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:36:10.114 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:10.105+0000 7f6010bd4640 1 --2- 172.21.15.43:0/4045121958 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f600c157ea0 0x7f600c1efa50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:36:10.114 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:10.105+0000 7f6010bd4640 1 -- 172.21.15.43:0/4045121958 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f600c15a800 msgr2=0x7f600c0c5ef0 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2023-08-09T13:36:10.114 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:10.105+0000 7f6010bd4640 1 --2- 172.21.15.43:0/4045121958 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f600c15a800 0x7f600c0c5ef0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:36:10.115 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:10.105+0000 7f6010bd4640 1 -- 172.21.15.43:0/4045121958 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f60000094d0 con 0x7f600c0c90a0 2023-08-09T13:36:10.115 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:10.106+0000 7f6010bd4640 1 --2- 172.21.15.43:0/4045121958 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f600c0c90a0 0x7f600c0c4220 secure :-1 s=READY pgs=557 cs=0 l=1 rev1=1 crypto rx=0x7f6000000c00 tx=0x7f60000046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-09T13:36:10.115 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:10.106+0000 7f60097fa640 1 -- 172.21.15.43:0/4045121958 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f600003ac60 con 0x7f600c0c90a0 2023-08-09T13:36:10.116 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:10.106+0000 7f60097fa640 1 -- 172.21.15.43:0/4045121958 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f600003adc0 con 0x7f600c0c90a0 2023-08-09T13:36:10.116 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:10.106+0000 7f60097fa640 1 -- 172.21.15.43:0/4045121958 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6000044630 con 0x7f600c0c90a0 2023-08-09T13:36:10.116 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:10.106+0000 7f6011bd6640 1 -- 172.21.15.43:0/4045121958 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f600c0c6c70 con 0x7f600c0c90a0 2023-08-09T13:36:10.116 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:10.106+0000 7f6011bd6640 1 -- 172.21.15.43:0/4045121958 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f600c0c4760 con 0x7f600c0c90a0 2023-08-09T13:36:10.116 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:10.108+0000 7f6011bd6640 1 -- 172.21.15.43:0/4045121958 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f600c04e390 con 0x7f600c0c90a0 2023-08-09T13:36:10.117 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:10.109+0000 7f60097fa640 1 -- 172.21.15.43:0/4045121958 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 5) v1 ==== 88341+0+0 (secure 0 0 0) 0x7f600002f050 con 0x7f600c0c90a0 2023-08-09T13:36:10.117 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:10.110+0000 7f60097fa640 1 --2- 172.21.15.43:0/4045121958 >> [v2:172.21.15.203:6832/830097474,v1:172.21.15.203:6833/830097474] conn(0x7f5ff006a050 0x7f5ff006c510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:36:10.118 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:10.111+0000 7f60097fa640 1 -- 172.21.15.43:0/4045121958 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(36..36 src has 1..36) v4 ==== 5900+0+0 (secure 0 0 0) 0x7f60000aa6e0 con 0x7f600c0c90a0 2023-08-09T13:36:10.118 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:10.111+0000 7f600bfff640 1 --2- 172.21.15.43:0/4045121958 >> [v2:172.21.15.203:6832/830097474,v1:172.21.15.203:6833/830097474] conn(0x7f5ff006a050 0x7f5ff006c510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:36:10.118 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:10.113+0000 7f600bfff640 1 --2- 172.21.15.43:0/4045121958 >> [v2:172.21.15.203:6832/830097474,v1:172.21.15.203:6833/830097474] conn(0x7f5ff006a050 0x7f5ff006c510 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7f600c1599a0 tx=0x7f5ff8008040 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-09T13:36:10.118 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:10.115+0000 7f60097fa640 1 -- 172.21.15.43:0/4045121958 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178066 (secure 0 0 0) 0x7f6000075fe0 con 0x7f600c0c90a0 2023-08-09T13:36:10.311 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:10.309+0000 7f6011bd6640 1 -- 172.21.15.43:0/4045121958 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9} v 0) v1 -- 0x7f600c071540 con 0x7f600c0c90a0 2023-08-09T13:36:10.849 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:10.845+0000 7f60097fa640 1 -- 172.21.15.43:0/4045121958 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]=0 pool 'cephfs_data' created v37) v1 ==== 170+0+0 (secure 0 0 0) 0x7f6000031e60 con 0x7f600c0c90a0 2023-08-09T13:36:10.850 INFO:teuthology.orchestra.run.smithi043.stderr:pool 'cephfs_data' created 2023-08-09T13:36:10.856 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:10.853+0000 7f6011bd6640 1 -- 172.21.15.43:0/4045121958 >> [v2:172.21.15.203:6832/830097474,v1:172.21.15.203:6833/830097474] conn(0x7f5ff006a050 msgr2=0x7f5ff006c510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:36:10.856 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:10.853+0000 7f6011bd6640 1 --2- 172.21.15.43:0/4045121958 >> [v2:172.21.15.203:6832/830097474,v1:172.21.15.203:6833/830097474] conn(0x7f5ff006a050 0x7f5ff006c510 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7f600c1599a0 tx=0x7f5ff8008040 comp rx=0 tx=0).stop 2023-08-09T13:36:10.856 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:10.853+0000 7f6011bd6640 1 -- 172.21.15.43:0/4045121958 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f600c0c90a0 msgr2=0x7f600c0c4220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:36:10.856 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:10.853+0000 7f6011bd6640 1 --2- 172.21.15.43:0/4045121958 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f600c0c90a0 0x7f600c0c4220 secure :-1 s=READY pgs=557 cs=0 l=1 rev1=1 crypto rx=0x7f6000000c00 tx=0x7f60000046c0 comp rx=0 tx=0).stop 2023-08-09T13:36:10.858 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:10.854+0000 7f6011bd6640 1 -- 172.21.15.43:0/4045121958 shutdown_connections 2023-08-09T13:36:10.858 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:10.854+0000 7f6011bd6640 1 --2- 172.21.15.43:0/4045121958 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f600c15a800 0x7f600c0c5ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:36:10.859 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:10.854+0000 7f6011bd6640 1 --2- 172.21.15.43:0/4045121958 >> [v2:172.21.15.203:6832/830097474,v1:172.21.15.203:6833/830097474] conn(0x7f5ff006a050 0x7f5ff006c510 unknown :-1 s=CLOSED pgs=269 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:36:10.859 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:10.854+0000 7f6011bd6640 1 --2- 172.21.15.43:0/4045121958 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f600c157ea0 0x7f600c1efa50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:36:10.859 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:10.854+0000 7f6011bd6640 1 --2- 172.21.15.43:0/4045121958 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f600c0c90a0 0x7f600c0c4220 secure :-1 s=CLOSED pgs=557 cs=0 l=1 rev1=1 crypto rx=0x7f6000000c00 tx=0x7f60000046c0 comp rx=0 tx=0).stop 2023-08-09T13:36:10.859 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:10.854+0000 7f6011bd6640 1 -- 172.21.15.43:0/4045121958 >> 172.21.15.43:0/4045121958 conn(0x7f600c0cddd0 msgr2=0x7f600c0c1040 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-09T13:36:10.860 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:10.858+0000 7f6011bd6640 1 -- 172.21.15.43:0/4045121958 shutdown_connections 2023-08-09T13:36:10.864 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:10.861+0000 7f6011bd6640 1 -- 172.21.15.43:0/4045121958 wait complete. 2023-08-09T13:36:10.888 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs new cephfs cephfs_metadata cephfs_data 2023-08-09T13:36:11.081 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:11.078+0000 7f005986a640 1 Processor -- start 2023-08-09T13:36:11.081 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:11.078+0000 7f005986a640 1 -- start start 2023-08-09T13:36:11.082 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:11.080+0000 7f005986a640 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7f005415c550 0x7f00540cd6d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:36:11.083 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:11.080+0000 7f005986a640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f00540b45f0 0x7f00540b59d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:36:11.083 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:11.080+0000 7f005986a640 1 -- --> v1:172.21.15.203:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f00540b7bd0 con 0x7f00540d04b0 2023-08-09T13:36:11.083 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:11.080+0000 7f005986a640 1 -- --> v2:172.21.15.203:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f00540d1c00 con 0x7f005415c550 2023-08-09T13:36:11.083 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:11.080+0000 7f005986a640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f005415c930 con 0x7f00540b45f0 2023-08-09T13:36:11.084 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:11.081+0000 7f00537fe640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f00540b45f0 0x7f00540b59d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:36:11.084 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:11.081+0000 7f00537fe640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f00540b45f0 0x7f00540b59d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.203:3301/0 says I am v2:172.21.15.43:56366/0 (socket says 172.21.15.43:56366) 2023-08-09T13:36:11.084 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:11.081+0000 7f00537fe640 1 -- 172.21.15.43:0/1690292072 learned_addr learned my addr 172.21.15.43:0/1690292072 (peer_addr_for_me v2:172.21.15.43:0/0) 2023-08-09T13:36:11.085 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:11.081+0000 7f0052ffd640 1 --2- 172.21.15.43:0/1690292072 >> v2:172.21.15.203:3300/0 conn(0x7f005415c550 0x7f00540cd6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:36:11.085 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:11.082+0000 7f00537fe640 1 -- 172.21.15.43:0/1690292072 >> v2:172.21.15.203:3300/0 conn(0x7f005415c550 msgr2=0x7f00540cd6d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-09T13:36:11.085 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:11.082+0000 7f00537fe640 1 --2- 172.21.15.43:0/1690292072 >> v2:172.21.15.203:3300/0 conn(0x7f005415c550 0x7f00540cd6d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:36:11.085 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:11.082+0000 7f00537fe640 1 -- 172.21.15.43:0/1690292072 >> v1:172.21.15.203:6789/0 conn(0x7f00540d04b0 legacy=0x7f00540b3ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:36:11.085 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:11.082+0000 7f00537fe640 1 -- 172.21.15.43:0/1690292072 --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f00540b54f0 con 0x7f00540b45f0 2023-08-09T13:36:11.085 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:11.082+0000 7f0051ffb640 1 -- 172.21.15.43:0/1690292072 <== mon.1 v1:172.21.15.203:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2571514891 0 0) 0x7f00540b7bd0 con 0x7f00540d04b0 2023-08-09T13:36:11.087 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:11.083+0000 7f00537fe640 1 --2- 172.21.15.43:0/1690292072 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f00540b45f0 0x7f00540b59d0 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f004c009e50 tx=0x7f004c02bed0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d44baec39851f943 server_cookie=0 in_seq=0 out_seq=0 2023-08-09T13:36:11.087 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:11.084+0000 7f0051ffb640 1 -- 172.21.15.43:0/1690292072 <== mon.2 v2:172.21.15.203:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f004c02ca70 con 0x7f00540b45f0 2023-08-09T13:36:11.087 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:11.084+0000 7f0051ffb640 1 -- 172.21.15.43:0/1690292072 <== mon.2 v2:172.21.15.203:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f004c02cbd0 con 0x7f00540b45f0 2023-08-09T13:36:11.087 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:11.084+0000 7f0051ffb640 1 -- 172.21.15.43:0/1690292072 <== mon.2 v2:172.21.15.203:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f004c0343f0 con 0x7f00540b45f0 2023-08-09T13:36:11.087 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:11.084+0000 7f005986a640 1 -- 172.21.15.43:0/1690292072 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f00540b45f0 msgr2=0x7f00540b59d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:36:11.088 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:11.084+0000 7f005986a640 1 --2- 172.21.15.43:0/1690292072 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f00540b45f0 0x7f00540b59d0 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f004c009e50 tx=0x7f004c02bed0 comp rx=0 tx=0).stop 2023-08-09T13:36:11.088 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:11.084+0000 7f005986a640 1 -- 172.21.15.43:0/1690292072 shutdown_connections 2023-08-09T13:36:11.088 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:11.084+0000 7f005986a640 1 --2- 172.21.15.43:0/1690292072 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f00540b45f0 0x7f00540b59d0 unknown :-1 s=CLOSED pgs=77 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:36:11.088 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:11.084+0000 7f005986a640 1 --2- 172.21.15.43:0/1690292072 >> v2:172.21.15.203:3300/0 conn(0x7f005415c550 0x7f00540cd6d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:36:11.088 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:11.084+0000 7f005986a640 1 -- 172.21.15.43:0/1690292072 >> 172.21.15.43:0/1690292072 conn(0x7f0054071520 msgr2=0x7f00540ceee0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-09T13:36:11.088 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:11.084+0000 7f005986a640 1 -- 172.21.15.43:0/1690292072 shutdown_connections 2023-08-09T13:36:11.089 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:11.085+0000 7f005986a640 1 -- 172.21.15.43:0/1690292072 wait complete. 2023-08-09T13:36:11.089 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:11.086+0000 7f005986a640 1 Processor -- start 2023-08-09T13:36:11.089 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:11.086+0000 7f005986a640 1 -- start start 2023-08-09T13:36:11.090 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:11.087+0000 7f005986a640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f00540b45f0 0x7f00540b34f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:36:11.091 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:11.087+0000 7f005986a640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f00540d04b0 0x7f00540a92d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:36:11.091 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:11.088+0000 7f005986a640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f005415c550 0x7f00540ac2c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:36:11.091 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:11.088+0000 7f005986a640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f00540ac800 con 0x7f005415c550 2023-08-09T13:36:11.091 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:11.088+0000 7f005986a640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f00540aaa70 con 0x7f00540b45f0 2023-08-09T13:36:11.091 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:11.088+0000 7f005986a640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f00540a99d0 con 0x7f00540d04b0 2023-08-09T13:36:11.092 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:11.088+0000 7f00537fe640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f005415c550 0x7f00540ac2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:36:11.092 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:11.088+0000 7f00537fe640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f005415c550 0x7f00540ac2c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:35980/0 (socket says 172.21.15.43:35980) 2023-08-09T13:36:11.092 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:11.088+0000 7f00537fe640 1 -- 172.21.15.43:0/286986221 learned_addr learned my addr 172.21.15.43:0/286986221 (peer_addr_for_me v2:172.21.15.43:0/0) 2023-08-09T13:36:11.092 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:11.088+0000 7f0052ffd640 1 --2- 172.21.15.43:0/286986221 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f00540b45f0 0x7f00540b34f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:36:11.093 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:11.088+0000 7f00527fc640 1 --2- 172.21.15.43:0/286986221 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f00540d04b0 0x7f00540a92d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:36:11.094 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:11.088+0000 7f00537fe640 1 -- 172.21.15.43:0/286986221 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f00540d04b0 msgr2=0x7f00540a92d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:36:11.094 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:11.088+0000 7f00537fe640 1 --2- 172.21.15.43:0/286986221 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f00540d04b0 0x7f00540a92d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:36:11.094 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:11.088+0000 7f00537fe640 1 -- 172.21.15.43:0/286986221 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f00540b45f0 msgr2=0x7f00540b34f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:36:11.094 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:11.088+0000 7f00537fe640 1 --2- 172.21.15.43:0/286986221 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f00540b45f0 0x7f00540b34f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:36:11.094 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:11.088+0000 7f00537fe640 1 -- 172.21.15.43:0/286986221 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f004c009b30 con 0x7f005415c550 2023-08-09T13:36:11.095 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:11.089+0000 7f00537fe640 1 --2- 172.21.15.43:0/286986221 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f005415c550 0x7f00540ac2c0 secure :-1 s=READY pgs=558 cs=0 l=1 rev1=1 crypto rx=0x7f004c036040 tx=0x7f004c004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-09T13:36:11.095 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:11.089+0000 7f003bfff640 1 -- 172.21.15.43:0/286986221 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f004c009510 con 0x7f005415c550 2023-08-09T13:36:11.095 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:11.089+0000 7f003bfff640 1 -- 172.21.15.43:0/286986221 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f004c03e450 con 0x7f005415c550 2023-08-09T13:36:11.095 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:11.089+0000 7f003bfff640 1 -- 172.21.15.43:0/286986221 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f004c03e5e0 con 0x7f005415c550 2023-08-09T13:36:11.095 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:11.089+0000 7f005986a640 1 -- 172.21.15.43:0/286986221 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f00540a9cb0 con 0x7f005415c550 2023-08-09T13:36:11.096 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:11.089+0000 7f005986a640 1 -- 172.21.15.43:0/286986221 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f00540aa270 con 0x7f005415c550 2023-08-09T13:36:11.096 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:11.090+0000 7f005986a640 1 -- 172.21.15.43:0/286986221 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f00540739f0 con 0x7f005415c550 2023-08-09T13:36:11.096 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:11.091+0000 7f003bfff640 1 -- 172.21.15.43:0/286986221 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 5) v1 ==== 88341+0+0 (secure 0 0 0) 0x7f004c03e740 con 0x7f005415c550 2023-08-09T13:36:11.097 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:11.092+0000 7f003bfff640 1 --2- 172.21.15.43:0/286986221 >> [v2:172.21.15.203:6832/830097474,v1:172.21.15.203:6833/830097474] conn(0x7f004006a050 0x7f004006c510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:36:11.097 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:11.092+0000 7f003bfff640 1 -- 172.21.15.43:0/286986221 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(37..37 src has 1..37) v4 ==== 6282+0+0 (secure 0 0 0) 0x7f004c0ab070 con 0x7f005415c550 2023-08-09T13:36:11.097 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:11.093+0000 7f0052ffd640 1 --2- 172.21.15.43:0/286986221 >> [v2:172.21.15.203:6832/830097474,v1:172.21.15.203:6833/830097474] conn(0x7f004006a050 0x7f004006c510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:36:11.098 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:11.094+0000 7f0052ffd640 1 --2- 172.21.15.43:0/286986221 >> [v2:172.21.15.203:6832/830097474,v1:172.21.15.203:6833/830097474] conn(0x7f004006a050 0x7f004006c510 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7f0044009a30 tx=0x7f0044008040 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-09T13:36:11.099 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:11.097+0000 7f003bfff640 1 -- 172.21.15.43:0/286986221 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178066 (secure 0 0 0) 0x7f004c0dd7b0 con 0x7f005415c550 2023-08-09T13:36:11.336 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:11.334+0000 7f005986a640 1 -- 172.21.15.43:0/286986221 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"} v 0) v1 -- 0x7f00540cf7f0 con 0x7f005415c550 2023-08-09T13:36:11.884 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:11.881+0000 7f003bfff640 1 -- 172.21.15.43:0/286986221 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]=0 new fs with metadata pool 6 and data pool 7 v13) v1 ==== 172+0+0 (secure 0 0 0) 0x7f004c0765e0 con 0x7f005415c550 2023-08-09T13:36:11.884 INFO:teuthology.orchestra.run.smithi043.stderr:new fs with metadata pool 6 and data pool 7 2023-08-09T13:36:11.887 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:11.884+0000 7f005986a640 1 -- 172.21.15.43:0/286986221 >> [v2:172.21.15.203:6832/830097474,v1:172.21.15.203:6833/830097474] conn(0x7f004006a050 msgr2=0x7f004006c510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:36:11.887 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:11.884+0000 7f005986a640 1 --2- 172.21.15.43:0/286986221 >> [v2:172.21.15.203:6832/830097474,v1:172.21.15.203:6833/830097474] conn(0x7f004006a050 0x7f004006c510 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7f0044009a30 tx=0x7f0044008040 comp rx=0 tx=0).stop 2023-08-09T13:36:11.888 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:11.884+0000 7f005986a640 1 -- 172.21.15.43:0/286986221 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f005415c550 msgr2=0x7f00540ac2c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:36:11.888 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:11.884+0000 7f005986a640 1 --2- 172.21.15.43:0/286986221 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f005415c550 0x7f00540ac2c0 secure :-1 s=READY pgs=558 cs=0 l=1 rev1=1 crypto rx=0x7f004c036040 tx=0x7f004c004a40 comp rx=0 tx=0).stop 2023-08-09T13:36:11.888 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:11.885+0000 7f005986a640 1 -- 172.21.15.43:0/286986221 shutdown_connections 2023-08-09T13:36:11.888 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:11.885+0000 7f005986a640 1 --2- 172.21.15.43:0/286986221 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f005415c550 0x7f00540ac2c0 unknown :-1 s=CLOSED pgs=558 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:36:11.888 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:11.885+0000 7f005986a640 1 --2- 172.21.15.43:0/286986221 >> [v2:172.21.15.203:6832/830097474,v1:172.21.15.203:6833/830097474] conn(0x7f004006a050 0x7f004006c510 unknown :-1 s=CLOSED pgs=270 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:36:11.888 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:11.885+0000 7f005986a640 1 --2- 172.21.15.43:0/286986221 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f00540d04b0 0x7f00540a92d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:36:11.889 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:11.885+0000 7f005986a640 1 --2- 172.21.15.43:0/286986221 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f00540b45f0 0x7f00540b34f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:36:11.889 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:11.885+0000 7f005986a640 1 -- 172.21.15.43:0/286986221 >> 172.21.15.43:0/286986221 conn(0x7f0054071520 msgr2=0x7f00540c8e00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-09T13:36:11.889 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:11.885+0000 7f005986a640 1 -- 172.21.15.43:0/286986221 shutdown_connections 2023-08-09T13:36:11.889 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:11.885+0000 7f005986a640 1 -- 172.21.15.43:0/286986221 wait complete. 2023-08-09T13:36:11.914 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-08-09T13:36:12.110 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:12.107+0000 7f9c08894640 1 Processor -- start 2023-08-09T13:36:12.110 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:12.107+0000 7f9c08894640 1 -- start start 2023-08-09T13:36:12.111 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:12.109+0000 7f9c08894640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f9c04143c20 0x7f9c0406f540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:36:12.111 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:12.109+0000 7f9c08894640 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7f9c040c86f0 0x7f9c040c8b50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:36:12.112 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:12.109+0000 7f9c08894640 1 -- --> v1:172.21.15.203:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9c040c9090 con 0x7f9c0406c8d0 2023-08-09T13:36:12.112 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:12.109+0000 7f9c08894640 1 -- --> v2:172.21.15.43:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9c04144000 con 0x7f9c04143c20 2023-08-09T13:36:12.112 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:12.109+0000 7f9c08894640 1 -- --> v2:172.21.15.203:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9c0406c440 con 0x7f9c040c86f0 2023-08-09T13:36:12.113 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:12.109+0000 7f9c0259c640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f9c04143c20 0x7f9c0406f540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:36:12.113 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:12.109+0000 7f9c0259c640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f9c04143c20 0x7f9c0406f540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:35986/0 (socket says 172.21.15.43:35986) 2023-08-09T13:36:12.113 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:12.109+0000 7f9c0259c640 1 -- 172.21.15.43:0/1323310411 learned_addr learned my addr 172.21.15.43:0/1323310411 (peer_addr_for_me v2:172.21.15.43:0/0) 2023-08-09T13:36:12.113 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:12.110+0000 7f9c0259c640 1 -- 172.21.15.43:0/1323310411 >> v2:172.21.15.203:3300/0 conn(0x7f9c040c86f0 msgr2=0x7f9c040c8b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-09T13:36:12.113 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:12.110+0000 7f9c0259c640 1 --2- 172.21.15.43:0/1323310411 >> v2:172.21.15.203:3300/0 conn(0x7f9c040c86f0 0x7f9c040c8b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:36:12.114 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:12.110+0000 7f9c0259c640 1 -- 172.21.15.43:0/1323310411 >> v1:172.21.15.203:6789/0 conn(0x7f9c0406c8d0 legacy=0x7f9c0406cd00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-09T13:36:12.114 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:12.110+0000 7f9c0259c640 1 -- 172.21.15.43:0/1323310411 --> v2:172.21.15.43:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9c0406b830 con 0x7f9c04143c20 2023-08-09T13:36:12.115 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:12.111+0000 7f9c0259c640 1 --2- 172.21.15.43:0/1323310411 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f9c04143c20 0x7f9c0406f540 secure :-1 s=READY pgs=559 cs=0 l=1 rev1=1 crypto rx=0x7f9c040c9b70 tx=0x7f9bf402b800 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e44570a084b86c2e server_cookie=0 in_seq=0 out_seq=0 2023-08-09T13:36:12.115 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:12.111+0000 7f9c01d9b640 1 -- 172.21.15.43:0/1323310411 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9bf4031760 con 0x7f9c04143c20 2023-08-09T13:36:12.115 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:12.111+0000 7f9c01d9b640 1 -- 172.21.15.43:0/1323310411 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9bf40318c0 con 0x7f9c04143c20 2023-08-09T13:36:12.115 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:12.111+0000 7f9c01d9b640 1 -- 172.21.15.43:0/1323310411 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9bf4031c50 con 0x7f9c04143c20 2023-08-09T13:36:12.116 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:12.112+0000 7f9c08894640 1 -- 172.21.15.43:0/1323310411 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f9c04143c20 msgr2=0x7f9c0406f540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:36:12.116 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:12.112+0000 7f9c08894640 1 --2- 172.21.15.43:0/1323310411 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f9c04143c20 0x7f9c0406f540 secure :-1 s=READY pgs=559 cs=0 l=1 rev1=1 crypto rx=0x7f9c040c9b70 tx=0x7f9bf402b800 comp rx=0 tx=0).stop 2023-08-09T13:36:12.116 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:12.112+0000 7f9c08894640 1 -- 172.21.15.43:0/1323310411 shutdown_connections 2023-08-09T13:36:12.116 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:12.112+0000 7f9c08894640 1 --2- 172.21.15.43:0/1323310411 >> v2:172.21.15.203:3300/0 conn(0x7f9c040c86f0 0x7f9c040c8b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:36:12.116 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:12.112+0000 7f9c08894640 1 --2- 172.21.15.43:0/1323310411 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f9c04143c20 0x7f9c0406f540 unknown :-1 s=CLOSED pgs=559 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:36:12.116 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:12.112+0000 7f9c08894640 1 -- 172.21.15.43:0/1323310411 >> 172.21.15.43:0/1323310411 conn(0x7f9c040ce0a0 msgr2=0x7f9c040ce480 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-09T13:36:12.117 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:12.112+0000 7f9c08894640 1 -- 172.21.15.43:0/1323310411 shutdown_connections 2023-08-09T13:36:12.117 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:12.112+0000 7f9c08894640 1 -- 172.21.15.43:0/1323310411 wait complete. 2023-08-09T13:36:12.118 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:12.114+0000 7f9c08894640 1 Processor -- start 2023-08-09T13:36:12.118 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:12.114+0000 7f9c08894640 1 -- start start 2023-08-09T13:36:12.118 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:12.115+0000 7f9c08894640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f9c0406c8d0 0x7f9c040b3350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:36:12.118 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:12.115+0000 7f9c08894640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f9c040c86f0 0x7f9c0413abb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:36:12.118 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:12.115+0000 7f9c08894640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f9c04143c20 0x7f9c04141560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:36:12.119 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:12.115+0000 7f9c08894640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9c040b2ec0 con 0x7f9c040c86f0 2023-08-09T13:36:12.119 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:12.115+0000 7f9c08894640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9c0413b0f0 con 0x7f9c04143c20 2023-08-09T13:36:12.119 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:12.115+0000 7f9c08894640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9c040b22b0 con 0x7f9c0406c8d0 2023-08-09T13:36:12.119 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:12.115+0000 7f9bfbfff640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f9c040c86f0 0x7f9c0413abb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:36:12.119 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:12.115+0000 7f9bfbfff640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f9c040c86f0 0x7f9c0413abb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:35990/0 (socket says 172.21.15.43:35990) 2023-08-09T13:36:12.119 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:12.115+0000 7f9bfbfff640 1 -- 172.21.15.43:0/1481055546 learned_addr learned my addr 172.21.15.43:0/1481055546 (peer_addr_for_me v2:172.21.15.43:0/0) 2023-08-09T13:36:12.120 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:12.116+0000 7f9bfbfff640 1 -- 172.21.15.43:0/1481055546 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f9c0406c8d0 msgr2=0x7f9c040b3350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:36:12.121 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:12.116+0000 7f9bfbfff640 1 --2- 172.21.15.43:0/1481055546 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f9c0406c8d0 0x7f9c040b3350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:36:12.121 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:12.116+0000 7f9c02d9d640 1 --2- 172.21.15.43:0/1481055546 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f9c04143c20 0x7f9c04141560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:36:12.121 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:12.116+0000 7f9bfbfff640 1 -- 172.21.15.43:0/1481055546 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f9c04143c20 msgr2=0x7f9c04141560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:36:12.121 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:12.116+0000 7f9bfbfff640 1 --2- 172.21.15.43:0/1481055546 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f9c04143c20 0x7f9c04141560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:36:12.121 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:12.116+0000 7f9bfbfff640 1 -- 172.21.15.43:0/1481055546 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9bf40094d0 con 0x7f9c040c86f0 2023-08-09T13:36:12.121 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:12.116+0000 7f9c02d9d640 1 --2- 172.21.15.43:0/1481055546 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f9c04143c20 0x7f9c04141560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-08-09T13:36:12.122 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:12.116+0000 7f9bfbfff640 1 --2- 172.21.15.43:0/1481055546 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f9c040c86f0 0x7f9c0413abb0 secure :-1 s=READY pgs=560 cs=0 l=1 rev1=1 crypto rx=0x7f9bec00b410 tx=0x7f9bec00b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-09T13:36:12.122 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:12.116+0000 7f9bfaffd640 1 -- 172.21.15.43:0/1481055546 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9bec011830 con 0x7f9c040c86f0 2023-08-09T13:36:12.122 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:12.116+0000 7f9bfaffd640 1 -- 172.21.15.43:0/1481055546 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9bec0119b0 con 0x7f9c040c86f0 2023-08-09T13:36:12.122 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:12.116+0000 7f9bfaffd640 1 -- 172.21.15.43:0/1481055546 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9bec011d60 con 0x7f9c040c86f0 2023-08-09T13:36:12.122 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:12.116+0000 7f9c08894640 1 -- 172.21.15.43:0/1481055546 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9c040b2590 con 0x7f9c040c86f0 2023-08-09T13:36:12.122 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:12.117+0000 7f9c08894640 1 -- 172.21.15.43:0/1481055546 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9c040b2b50 con 0x7f9c040c86f0 2023-08-09T13:36:12.123 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:12.118+0000 7f9c08894640 1 -- 172.21.15.43:0/1481055546 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9c040739f0 con 0x7f9c040c86f0 2023-08-09T13:36:12.123 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:12.119+0000 7f9bfaffd640 1 -- 172.21.15.43:0/1481055546 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 5) v1 ==== 88341+0+0 (secure 0 0 0) 0x7f9bec01a400 con 0x7f9c040c86f0 2023-08-09T13:36:12.123 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:12.120+0000 7f9bfaffd640 1 --2- 172.21.15.43:0/1481055546 >> [v2:172.21.15.203:6832/830097474,v1:172.21.15.203:6833/830097474] conn(0x7f9be406a080 0x7f9be406c540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:36:12.124 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:12.120+0000 7f9bfaffd640 1 -- 172.21.15.43:0/1481055546 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(38..38 src has 1..38) v4 ==== 6382+0+0 (secure 0 0 0) 0x7f9bec0888b0 con 0x7f9c040c86f0 2023-08-09T13:36:12.124 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:12.121+0000 7f9c0259c640 1 --2- 172.21.15.43:0/1481055546 >> [v2:172.21.15.203:6832/830097474,v1:172.21.15.203:6833/830097474] conn(0x7f9be406a080 0x7f9be406c540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:36:12.125 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:12.122+0000 7f9c0259c640 1 --2- 172.21.15.43:0/1481055546 >> [v2:172.21.15.203:6832/830097474,v1:172.21.15.203:6833/830097474] conn(0x7f9be406a080 0x7f9be406c540 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7f9bf4036000 tx=0x7f9bf4030000 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-09T13:36:12.126 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:12.124+0000 7f9bfaffd640 1 -- 172.21.15.43:0/1481055546 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178066 (secure 0 0 0) 0x7f9bec053dc0 con 0x7f9c040c86f0 2023-08-09T13:36:12.324 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:12.322+0000 7f9c08894640 1 -- 172.21.15.43:0/1481055546 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f9c040b3000 con 0x7f9c040c86f0 2023-08-09T13:36:12.328 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:12.325+0000 7f9bfaffd640 1 -- 172.21.15.43:0/1481055546 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v38) v1 ==== 74+0+18443 (secure 0 0 0) 0x7f9bec0573e0 con 0x7f9c040c86f0 2023-08-09T13:36:12.328 INFO:teuthology.orchestra.run.smithi043.stdout: 2023-08-09T13:36:12.329 INFO:teuthology.orchestra.run.smithi043.stdout:{"epoch":38,"fsid":"daac87ee-1337-4250-a955-aea4e408a39b","created":"2023-08-09T13:33:41.701198+0000","modified":"2023-08-09T13:36:11.336205+0000","last_up_change":"2023-08-09T13:33:53.783453+0000","last_in_change":"2023-08-09T13:33:45.148904+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":7,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":"rbd","create_time":"2023-08-09T13:33:54.365137+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":".mgr","create_time":"2023-08-09T13:33:54.831803+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":6,"pool_name":"cephfs_metadata","create_time":"2023-08-09T13:36:08.969993+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"38","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":7,"pool_name":"cephfs_data","create_time":"2023-08-09T13:36:10.366495+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"38","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"cbd49ecd-e896-4fbd-864e-5c37a829b165","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":37,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6800","nonce":502667335},{"type":"v1","addr":"172.21.15.43:6801","nonce":502667335}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6802","nonce":502667335},{"type":"v1","addr":"172.21.15.43:6803","nonce":502667335}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6806","nonce":502667335},{"type":"v1","addr":"172.21.15.43:6807","nonce":502667335}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6804","nonce":502667335},{"type":"v1","addr":"172.21.15.43:6805","nonce":502667335}]},"public_addr":"172.21.15.43:6801/502667335","cluster_addr":"172.21.15.43:6803/502667335","heartbeat_back_addr":"172.21.15.43:6807/502667335","heartbeat_front_addr":"172.21.15.43:6805/502667335","state":["exists","up"]},{"osd":1,"uuid":"75c7e0fe-1ec7-4fe8-a2f2-3661a65b8500","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":37,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6808","nonce":3289134610},{"type":"v1","addr":"172.21.15.43:6809","nonce":3289134610}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6810","nonce":3289134610},{"type":"v1","addr":"172.21.15.43:6812","nonce":3289134610}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6821","nonce":3289134610},{"type":"v1","addr":"172.21.15.43:6824","nonce":3289134610}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6815","nonce":3289134610},{"type":"v1","addr":"172.21.15.43:6818","nonce":3289134610}]},"public_addr":"172.21.15.43:6809/3289134610","cluster_addr":"172.21.15.43:6812/3289134610","heartbeat_back_addr":"172.21.15.43:6824/3289134610","heartbeat_front_addr":"172.21.15.43:6818/3289134610","state":["exists","up"]},{"osd":2,"uuid":"bb5c41e9-7f5f-4125-9b01-3bb8bfc2ea83","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":37,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6811","nonce":4079709736},{"type":"v1","addr":"172.21.15.43:6814","nonce":4079709736}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6817","nonce":4079709736},{"type":"v1","addr":"172.21.15.43:6820","nonce":4079709736}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6829","nonce":4079709736},{"type":"v1","addr":"172.21.15.43:6831","nonce":4079709736}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6823","nonce":4079709736},{"type":"v1","addr":"172.21.15.43:6826","nonce":4079709736}]},"public_addr":"172.21.15.43:6814/4079709736","cluster_addr":"172.21.15.43:6820/4079709736","heartbeat_back_addr":"172.21.15.43:6831/4079709736","heartbeat_front_addr":"172.21.15.43:6826/4079709736","state":["exists","up"]},{"osd":3,"uuid":"bfe4c8b4-694a-465c-87e6-2b2aab852613","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":37,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6813","nonce":1709538759},{"type":"v1","addr":"172.21.15.43:6816","nonce":1709538759}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6819","nonce":1709538759},{"type":"v1","addr":"172.21.15.43:6822","nonce":1709538759}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6828","nonce":1709538759},{"type":"v1","addr":"172.21.15.43:6830","nonce":1709538759}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6825","nonce":1709538759},{"type":"v1","addr":"172.21.15.43:6827","nonce":1709538759}]},"public_addr":"172.21.15.43:6816/1709538759","cluster_addr":"172.21.15.43:6822/1709538759","heartbeat_back_addr":"172.21.15.43:6830/1709538759","heartbeat_front_addr":"172.21.15.43:6827/1709538759","state":["exists","up"]},{"osd":4,"uuid":"fa2e8a77-aba1-4756-853e-0d174a5bdeba","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":37,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6800","nonce":4196866481},{"type":"v1","addr":"172.21.15.203:6801","nonce":4196866481}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6802","nonce":4196866481},{"type":"v1","addr":"172.21.15.203:6803","nonce":4196866481}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6806","nonce":4196866481},{"type":"v1","addr":"172.21.15.203:6807","nonce":4196866481}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6804","nonce":4196866481},{"type":"v1","addr":"172.21.15.203:6805","nonce":4196866481}]},"public_addr":"172.21.15.203:6801/4196866481","cluster_addr":"172.21.15.203:6803/4196866481","heartbeat_back_addr":"172.21.15.203:6807/4196866481","heartbeat_front_addr":"172.21.15.203:6805/4196866481","state":["exists","up"]},{"osd":5,"uuid":"81811e4b-4d24-4f51-a82c-f1d700a2a00d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":37,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6816","nonce":3059290081},{"type":"v1","addr":"172.21.15.203:6817","nonce":3059290081}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6818","nonce":3059290081},{"type":"v1","addr":"172.21.15.203:6819","nonce":3059290081}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6822","nonce":3059290081},{"type":"v1","addr":"172.21.15.203:6823","nonce":3059290081}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6820","nonce":3059290081},{"type":"v1","addr":"172.21.15.203:6821","nonce":3059290081}]},"public_addr":"172.21.15.203:6817/3059290081","cluster_addr":"172.21.15.203:6819/3059290081","heartbeat_back_addr":"172.21.15.203:6823/3059290081","heartbeat_front_addr":"172.21.15.203:6821/3059290081","state":["exists","up"]},{"osd":6,"uuid":"307bac4d-6a5b-443d-af75-56d695d7674f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":37,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6824","nonce":512155569},{"type":"v1","addr":"172.21.15.203:6825","nonce":512155569}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6826","nonce":512155569},{"type":"v1","addr":"172.21.15.203:6827","nonce":512155569}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6830","nonce":512155569},{"type":"v1","addr":"172.21.15.203:6831","nonce":512155569}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6828","nonce":512155569},{"type":"v1","addr":"172.21.15.203:6829","nonce":512155569}]},"public_addr":"172.21.15.203:6825/512155569","cluster_addr":"172.21.15.203:6827/512155569","heartbeat_back_addr":"172.21.15.203:6831/512155569","heartbeat_front_addr":"172.21.15.203:6829/512155569","state":["exists","up"]},{"osd":7,"uuid":"a95dc7ca-9f1b-4295-af70-68a22a2d7d5e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":37,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6808","nonce":3331450438},{"type":"v1","addr":"172.21.15.203:6809","nonce":3331450438}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6810","nonce":3331450438},{"type":"v1","addr":"172.21.15.203:6811","nonce":3331450438}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6814","nonce":3331450438},{"type":"v1","addr":"172.21.15.203:6815","nonce":3331450438}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6812","nonce":3331450438},{"type":"v1","addr":"172.21.15.203:6813","nonce":3331450438}]},"public_addr":"172.21.15.203:6809/3331450438","cluster_addr":"172.21.15.203:6811/3331450438","heartbeat_back_addr":"172.21.15.203:6815/3331450438","heartbeat_front_addr":"172.21.15.203:6813/3331450438","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-09T13:33:50.547652+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-09T13:33:50.589469+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-09T13:33:50.561724+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-09T13:33:50.599650+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-09T13:33:50.411855+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-09T13:33:50.219681+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-09T13:33:50.504664+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-09T13:33:50.441248+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-08-09T13:36:12.331 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:12.328+0000 7f9c08894640 1 -- 172.21.15.43:0/1481055546 >> [v2:172.21.15.203:6832/830097474,v1:172.21.15.203:6833/830097474] conn(0x7f9be406a080 msgr2=0x7f9be406c540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:36:12.331 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:12.328+0000 7f9c08894640 1 --2- 172.21.15.43:0/1481055546 >> [v2:172.21.15.203:6832/830097474,v1:172.21.15.203:6833/830097474] conn(0x7f9be406a080 0x7f9be406c540 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7f9bf4036000 tx=0x7f9bf4030000 comp rx=0 tx=0).stop 2023-08-09T13:36:12.331 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:12.328+0000 7f9c08894640 1 -- 172.21.15.43:0/1481055546 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f9c040c86f0 msgr2=0x7f9c0413abb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:36:12.331 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:12.328+0000 7f9c08894640 1 --2- 172.21.15.43:0/1481055546 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f9c040c86f0 0x7f9c0413abb0 secure :-1 s=READY pgs=560 cs=0 l=1 rev1=1 crypto rx=0x7f9bec00b410 tx=0x7f9bec00b8d0 comp rx=0 tx=0).stop 2023-08-09T13:36:12.331 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:12.328+0000 7f9c08894640 1 -- 172.21.15.43:0/1481055546 shutdown_connections 2023-08-09T13:36:12.332 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:12.328+0000 7f9c08894640 1 --2- 172.21.15.43:0/1481055546 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f9c04143c20 0x7f9c04141560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:36:12.332 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:12.328+0000 7f9c08894640 1 --2- 172.21.15.43:0/1481055546 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f9c040c86f0 0x7f9c0413abb0 unknown :-1 s=CLOSED pgs=560 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:36:12.332 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:12.328+0000 7f9c08894640 1 --2- 172.21.15.43:0/1481055546 >> [v2:172.21.15.203:6832/830097474,v1:172.21.15.203:6833/830097474] conn(0x7f9be406a080 0x7f9be406c540 unknown :-1 s=CLOSED pgs=271 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:36:12.332 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:12.328+0000 7f9c08894640 1 --2- 172.21.15.43:0/1481055546 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f9c0406c8d0 0x7f9c040b3350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:36:12.332 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:12.328+0000 7f9c08894640 1 -- 172.21.15.43:0/1481055546 >> 172.21.15.43:0/1481055546 conn(0x7f9c040ce0a0 msgr2=0x7f9c040cab20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-09T13:36:12.332 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:12.329+0000 7f9c08894640 1 -- 172.21.15.43:0/1481055546 shutdown_connections 2023-08-09T13:36:12.332 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:12.329+0000 7f9c08894640 1 -- 172.21.15.43:0/1481055546 wait complete. 2023-08-09T13:36:12.350 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-08-09T13:36:12.574 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:12.573+0000 7f15e7078640 1 Processor -- start 2023-08-09T13:36:12.575 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:12.573+0000 7f15e7078640 1 -- start start 2023-08-09T13:36:12.576 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:12.573+0000 7f15e7078640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f15e0160cb0 0x7f15e015c070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:36:12.576 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:12.573+0000 7f15e7078640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f15e0160760 0x7f15e015c5b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:36:12.577 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:12.573+0000 7f15e7078640 1 -- --> v1:172.21.15.43:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f15e00c8680 con 0x7f15e0160380 2023-08-09T13:36:12.577 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:12.573+0000 7f15e7078640 1 -- --> v2:172.21.15.43:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f15e00ca7d0 con 0x7f15e0160760 2023-08-09T13:36:12.577 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:12.573+0000 7f15e7078640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f15e00c8960 con 0x7f15e0160cb0 2023-08-09T13:36:12.577 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:12.574+0000 7f15e6076640 1 --1- >> v1:172.21.15.43:6789/0 conn(0x7f15e0160380 0x7f15e00ca0c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.43:6789/0 says I am v1:172.21.15.43:39460/0 (socket says 172.21.15.43:39460) 2023-08-09T13:36:12.577 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:12.574+0000 7f15e6076640 1 -- 172.21.15.43:0/2904419894 learned_addr learned my addr 172.21.15.43:0/2904419894 (peer_addr_for_me v1:172.21.15.43:0/0) 2023-08-09T13:36:12.577 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:12.574+0000 7f15e6877640 1 --2- 172.21.15.43:0/2904419894 >> v2:172.21.15.43:3300/0 conn(0x7f15e0160760 0x7f15e015c5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:36:12.578 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:12.574+0000 7f15e5875640 1 --2- 172.21.15.43:0/2904419894 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f15e0160cb0 0x7f15e015c070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:36:12.579 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:12.575+0000 7f15e5074640 1 -- 172.21.15.43:0/2904419894 <== mon.0 v1:172.21.15.43:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 175199069 0 0) 0x7f15e00c8680 con 0x7f15e0160380 2023-08-09T13:36:12.579 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:12.575+0000 7f15e6877640 1 -- 172.21.15.43:0/2904419894 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f15e0160cb0 msgr2=0x7f15e015c070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-09T13:36:12.579 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:12.575+0000 7f15e6877640 1 --2- 172.21.15.43:0/2904419894 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f15e0160cb0 0x7f15e015c070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:36:12.579 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:12.575+0000 7f15e6877640 1 -- 172.21.15.43:0/2904419894 >> v1:172.21.15.43:6789/0 conn(0x7f15e0160380 legacy=0x7f15e00ca0c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:36:12.579 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:12.575+0000 7f15e6877640 1 -- 172.21.15.43:0/2904419894 --> v2:172.21.15.43:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f15e00aaf00 con 0x7f15e0160760 2023-08-09T13:36:12.579 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:12.577+0000 7f15e6877640 1 --2- 172.21.15.43:0/2904419894 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f15e0160760 0x7f15e015c5b0 secure :-1 s=READY pgs=562 cs=0 l=1 rev1=1 crypto rx=0x7f15d8009d90 tx=0x7f15d802bf30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3dcf924b15ddaf07 server_cookie=0 in_seq=0 out_seq=0 2023-08-09T13:36:12.579 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:12.577+0000 7f15e5074640 1 -- 172.21.15.43:0/2904419894 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f15d802cce0 con 0x7f15e0160760 2023-08-09T13:36:12.580 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:12.577+0000 7f15e5074640 1 -- 172.21.15.43:0/2904419894 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f15d802ce40 con 0x7f15e0160760 2023-08-09T13:36:12.580 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:12.577+0000 7f15e5074640 1 -- 172.21.15.43:0/2904419894 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f15d8034660 con 0x7f15e0160760 2023-08-09T13:36:12.580 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:12.577+0000 7f15e7078640 1 -- 172.21.15.43:0/2904419894 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f15e0160760 msgr2=0x7f15e015c5b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:36:12.581 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:12.577+0000 7f15e7078640 1 --2- 172.21.15.43:0/2904419894 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f15e0160760 0x7f15e015c5b0 secure :-1 s=READY pgs=562 cs=0 l=1 rev1=1 crypto rx=0x7f15d8009d90 tx=0x7f15d802bf30 comp rx=0 tx=0).stop 2023-08-09T13:36:12.581 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:12.577+0000 7f15e7078640 1 -- 172.21.15.43:0/2904419894 shutdown_connections 2023-08-09T13:36:12.581 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:12.577+0000 7f15e7078640 1 --2- 172.21.15.43:0/2904419894 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f15e0160760 0x7f15e015c5b0 unknown :-1 s=CLOSED pgs=562 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:36:12.581 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:12.577+0000 7f15e7078640 1 --2- 172.21.15.43:0/2904419894 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f15e0160cb0 0x7f15e015c070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:36:12.581 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:12.577+0000 7f15e7078640 1 -- 172.21.15.43:0/2904419894 >> 172.21.15.43:0/2904419894 conn(0x7f15e00cde90 msgr2=0x7f15e00ce2a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-09T13:36:12.581 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:12.577+0000 7f15e7078640 1 -- 172.21.15.43:0/2904419894 shutdown_connections 2023-08-09T13:36:12.581 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:12.577+0000 7f15e7078640 1 -- 172.21.15.43:0/2904419894 wait complete. 2023-08-09T13:36:12.582 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:12.580+0000 7f15e7078640 1 Processor -- start 2023-08-09T13:36:12.583 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:12.580+0000 7f15e7078640 1 -- start start 2023-08-09T13:36:12.583 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:12.581+0000 7f15e7078640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f15e0160380 0x7f15e0148100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:36:12.583 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:12.581+0000 7f15e7078640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f15e0160760 0x7f15e009b4e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:36:12.583 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:12.581+0000 7f15e7078640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f15e0160cb0 0x7f15e0098ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:36:12.583 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:12.581+0000 7f15e7078640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f15e0145e10 con 0x7f15e0160380 2023-08-09T13:36:12.583 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:12.581+0000 7f15e7078640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f15e009ba20 con 0x7f15e0160760 2023-08-09T13:36:12.583 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:12.581+0000 7f15e7078640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f15e0146240 con 0x7f15e0160cb0 2023-08-09T13:36:12.584 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:12.581+0000 7f15e6076640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f15e0160380 0x7f15e0148100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:36:12.584 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:12.581+0000 7f15e6076640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f15e0160380 0x7f15e0148100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:36012/0 (socket says 172.21.15.43:36012) 2023-08-09T13:36:12.584 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:12.581+0000 7f15e6076640 1 -- 172.21.15.43:0/466132248 learned_addr learned my addr 172.21.15.43:0/466132248 (peer_addr_for_me v2:172.21.15.43:0/0) 2023-08-09T13:36:12.585 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:12.581+0000 7f15e6076640 1 -- 172.21.15.43:0/466132248 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f15e0160cb0 msgr2=0x7f15e0098ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:36:12.585 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:12.581+0000 7f15e6076640 1 --2- 172.21.15.43:0/466132248 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f15e0160cb0 0x7f15e0098ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:36:12.585 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:12.581+0000 7f15e6076640 1 -- 172.21.15.43:0/466132248 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f15e0160760 msgr2=0x7f15e009b4e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:36:12.585 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:12.581+0000 7f15e6076640 1 --2- 172.21.15.43:0/466132248 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f15e0160760 0x7f15e009b4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:36:12.585 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:12.581+0000 7f15e6076640 1 -- 172.21.15.43:0/466132248 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f15d8009a70 con 0x7f15e0160380 2023-08-09T13:36:12.585 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:12.581+0000 7f15e6076640 1 --2- 172.21.15.43:0/466132248 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f15e0160380 0x7f15e0148100 secure :-1 s=READY pgs=563 cs=0 l=1 rev1=1 crypto rx=0x7f15dc00c4d0 tx=0x7f15dc00c990 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-09T13:36:12.586 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:12.581+0000 7f15d6ffd640 1 -- 172.21.15.43:0/466132248 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f15dc0074e0 con 0x7f15e0160380 2023-08-09T13:36:12.586 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:12.581+0000 7f15d6ffd640 1 -- 172.21.15.43:0/466132248 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f15dc007660 con 0x7f15e0160380 2023-08-09T13:36:12.586 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:12.581+0000 7f15e7078640 1 -- 172.21.15.43:0/466132248 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f15e0099530 con 0x7f15e0160380 2023-08-09T13:36:12.586 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:12.581+0000 7f15d6ffd640 1 -- 172.21.15.43:0/466132248 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f15dc004a20 con 0x7f15e0160380 2023-08-09T13:36:12.586 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:12.581+0000 7f15e7078640 1 -- 172.21.15.43:0/466132248 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f15e0149090 con 0x7f15e0160380 2023-08-09T13:36:12.587 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:12.583+0000 7f15e7078640 1 -- 172.21.15.43:0/466132248 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f15e00739f0 con 0x7f15e0160380 2023-08-09T13:36:12.587 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:12.583+0000 7f15d6ffd640 1 -- 172.21.15.43:0/466132248 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 5) v1 ==== 88341+0+0 (secure 0 0 0) 0x7f15dc004b80 con 0x7f15e0160380 2023-08-09T13:36:12.587 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:12.584+0000 7f15d6ffd640 1 --2- 172.21.15.43:0/466132248 >> [v2:172.21.15.203:6832/830097474,v1:172.21.15.203:6833/830097474] conn(0x7f15c406a080 0x7f15c406c540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:36:12.587 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:12.584+0000 7f15d6ffd640 1 -- 172.21.15.43:0/466132248 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(38..38 src has 1..38) v4 ==== 6382+0+0 (secure 0 0 0) 0x7f15dc087b60 con 0x7f15e0160380 2023-08-09T13:36:12.588 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:12.585+0000 7f15e5875640 1 --2- 172.21.15.43:0/466132248 >> [v2:172.21.15.203:6832/830097474,v1:172.21.15.203:6833/830097474] conn(0x7f15c406a080 0x7f15c406c540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:36:12.588 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:12.586+0000 7f15e5875640 1 --2- 172.21.15.43:0/466132248 >> [v2:172.21.15.203:6832/830097474,v1:172.21.15.203:6833/830097474] conn(0x7f15c406a080 0x7f15c406c540 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7f15d0006a90 tx=0x7f15d0006a20 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-09T13:36:12.591 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:12.589+0000 7f15d6ffd640 1 -- 172.21.15.43:0/466132248 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178066 (secure 0 0 0) 0x7f15dc053110 con 0x7f15e0160380 2023-08-09T13:36:12.784 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:12.781+0000 7f15e7078640 1 -- 172.21.15.43:0/466132248 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f15e00cb440 con 0x7f15e0160380 2023-08-09T13:36:12.787 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:12.785+0000 7f15d6ffd640 1 -- 172.21.15.43:0/466132248 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v39) v1 ==== 74+0+18430 (secure 0 0 0) 0x7f15dc018090 con 0x7f15e0160380 2023-08-09T13:36:12.788 INFO:teuthology.orchestra.run.smithi043.stdout: 2023-08-09T13:36:12.788 INFO:teuthology.orchestra.run.smithi043.stdout:{"epoch":39,"fsid":"daac87ee-1337-4250-a955-aea4e408a39b","created":"2023-08-09T13:33:41.701198+0000","modified":"2023-08-09T13:36:12.734434+0000","last_up_change":"2023-08-09T13:33:53.783453+0000","last_in_change":"2023-08-09T13:33:45.148904+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":7,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":"rbd","create_time":"2023-08-09T13:33:54.365137+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":".mgr","create_time":"2023-08-09T13:33:54.831803+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":6,"pool_name":"cephfs_metadata","create_time":"2023-08-09T13:36:08.969993+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"38","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":7,"pool_name":"cephfs_data","create_time":"2023-08-09T13:36:10.366495+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"39","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"cbd49ecd-e896-4fbd-864e-5c37a829b165","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":37,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6800","nonce":502667335},{"type":"v1","addr":"172.21.15.43:6801","nonce":502667335}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6802","nonce":502667335},{"type":"v1","addr":"172.21.15.43:6803","nonce":502667335}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6806","nonce":502667335},{"type":"v1","addr":"172.21.15.43:6807","nonce":502667335}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6804","nonce":502667335},{"type":"v1","addr":"172.21.15.43:6805","nonce":502667335}]},"public_addr":"172.21.15.43:6801/502667335","cluster_addr":"172.21.15.43:6803/502667335","heartbeat_back_addr":"172.21.15.43:6807/502667335","heartbeat_front_addr":"172.21.15.43:6805/502667335","state":["exists","up"]},{"osd":1,"uuid":"75c7e0fe-1ec7-4fe8-a2f2-3661a65b8500","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":37,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6808","nonce":3289134610},{"type":"v1","addr":"172.21.15.43:6809","nonce":3289134610}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6810","nonce":3289134610},{"type":"v1","addr":"172.21.15.43:6812","nonce":3289134610}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6821","nonce":3289134610},{"type":"v1","addr":"172.21.15.43:6824","nonce":3289134610}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6815","nonce":3289134610},{"type":"v1","addr":"172.21.15.43:6818","nonce":3289134610}]},"public_addr":"172.21.15.43:6809/3289134610","cluster_addr":"172.21.15.43:6812/3289134610","heartbeat_back_addr":"172.21.15.43:6824/3289134610","heartbeat_front_addr":"172.21.15.43:6818/3289134610","state":["exists","up"]},{"osd":2,"uuid":"bb5c41e9-7f5f-4125-9b01-3bb8bfc2ea83","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":37,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6811","nonce":4079709736},{"type":"v1","addr":"172.21.15.43:6814","nonce":4079709736}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6817","nonce":4079709736},{"type":"v1","addr":"172.21.15.43:6820","nonce":4079709736}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6829","nonce":4079709736},{"type":"v1","addr":"172.21.15.43:6831","nonce":4079709736}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6823","nonce":4079709736},{"type":"v1","addr":"172.21.15.43:6826","nonce":4079709736}]},"public_addr":"172.21.15.43:6814/4079709736","cluster_addr":"172.21.15.43:6820/4079709736","heartbeat_back_addr":"172.21.15.43:6831/4079709736","heartbeat_front_addr":"172.21.15.43:6826/4079709736","state":["exists","up"]},{"osd":3,"uuid":"bfe4c8b4-694a-465c-87e6-2b2aab852613","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":37,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6813","nonce":1709538759},{"type":"v1","addr":"172.21.15.43:6816","nonce":1709538759}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6819","nonce":1709538759},{"type":"v1","addr":"172.21.15.43:6822","nonce":1709538759}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6828","nonce":1709538759},{"type":"v1","addr":"172.21.15.43:6830","nonce":1709538759}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6825","nonce":1709538759},{"type":"v1","addr":"172.21.15.43:6827","nonce":1709538759}]},"public_addr":"172.21.15.43:6816/1709538759","cluster_addr":"172.21.15.43:6822/1709538759","heartbeat_back_addr":"172.21.15.43:6830/1709538759","heartbeat_front_addr":"172.21.15.43:6827/1709538759","state":["exists","up"]},{"osd":4,"uuid":"fa2e8a77-aba1-4756-853e-0d174a5bdeba","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":37,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6800","nonce":4196866481},{"type":"v1","addr":"172.21.15.203:6801","nonce":4196866481}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6802","nonce":4196866481},{"type":"v1","addr":"172.21.15.203:6803","nonce":4196866481}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6806","nonce":4196866481},{"type":"v1","addr":"172.21.15.203:6807","nonce":4196866481}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6804","nonce":4196866481},{"type":"v1","addr":"172.21.15.203:6805","nonce":4196866481}]},"public_addr":"172.21.15.203:6801/4196866481","cluster_addr":"172.21.15.203:6803/4196866481","heartbeat_back_addr":"172.21.15.203:6807/4196866481","heartbeat_front_addr":"172.21.15.203:6805/4196866481","state":["exists","up"]},{"osd":5,"uuid":"81811e4b-4d24-4f51-a82c-f1d700a2a00d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":37,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6816","nonce":3059290081},{"type":"v1","addr":"172.21.15.203:6817","nonce":3059290081}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6818","nonce":3059290081},{"type":"v1","addr":"172.21.15.203:6819","nonce":3059290081}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6822","nonce":3059290081},{"type":"v1","addr":"172.21.15.203:6823","nonce":3059290081}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6820","nonce":3059290081},{"type":"v1","addr":"172.21.15.203:6821","nonce":3059290081}]},"public_addr":"172.21.15.203:6817/3059290081","cluster_addr":"172.21.15.203:6819/3059290081","heartbeat_back_addr":"172.21.15.203:6823/3059290081","heartbeat_front_addr":"172.21.15.203:6821/3059290081","state":["exists","up"]},{"osd":6,"uuid":"307bac4d-6a5b-443d-af75-56d695d7674f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":37,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6824","nonce":512155569},{"type":"v1","addr":"172.21.15.203:6825","nonce":512155569}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6826","nonce":512155569},{"type":"v1","addr":"172.21.15.203:6827","nonce":512155569}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6830","nonce":512155569},{"type":"v1","addr":"172.21.15.203:6831","nonce":512155569}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6828","nonce":512155569},{"type":"v1","addr":"172.21.15.203:6829","nonce":512155569}]},"public_addr":"172.21.15.203:6825/512155569","cluster_addr":"172.21.15.203:6827/512155569","heartbeat_back_addr":"172.21.15.203:6831/512155569","heartbeat_front_addr":"172.21.15.203:6829/512155569","state":["exists","up"]},{"osd":7,"uuid":"a95dc7ca-9f1b-4295-af70-68a22a2d7d5e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":37,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6808","nonce":3331450438},{"type":"v1","addr":"172.21.15.203:6809","nonce":3331450438}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6810","nonce":3331450438},{"type":"v1","addr":"172.21.15.203:6811","nonce":3331450438}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6814","nonce":3331450438},{"type":"v1","addr":"172.21.15.203:6815","nonce":3331450438}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6812","nonce":3331450438},{"type":"v1","addr":"172.21.15.203:6813","nonce":3331450438}]},"public_addr":"172.21.15.203:6809/3331450438","cluster_addr":"172.21.15.203:6811/3331450438","heartbeat_back_addr":"172.21.15.203:6815/3331450438","heartbeat_front_addr":"172.21.15.203:6813/3331450438","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-09T13:33:50.547652+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-09T13:33:50.589469+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-09T13:33:50.561724+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-09T13:33:50.599650+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-09T13:33:50.411855+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-09T13:33:50.219681+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-09T13:33:50.504664+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-09T13:33:50.441248+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-08-09T13:36:12.790 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:12.787+0000 7f15e7078640 1 -- 172.21.15.43:0/466132248 >> [v2:172.21.15.203:6832/830097474,v1:172.21.15.203:6833/830097474] conn(0x7f15c406a080 msgr2=0x7f15c406c540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:36:12.790 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:12.787+0000 7f15e7078640 1 --2- 172.21.15.43:0/466132248 >> [v2:172.21.15.203:6832/830097474,v1:172.21.15.203:6833/830097474] conn(0x7f15c406a080 0x7f15c406c540 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7f15d0006a90 tx=0x7f15d0006a20 comp rx=0 tx=0).stop 2023-08-09T13:36:12.790 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:12.787+0000 7f15e7078640 1 -- 172.21.15.43:0/466132248 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f15e0160380 msgr2=0x7f15e0148100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:36:12.790 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:12.787+0000 7f15e7078640 1 --2- 172.21.15.43:0/466132248 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f15e0160380 0x7f15e0148100 secure :-1 s=READY pgs=563 cs=0 l=1 rev1=1 crypto rx=0x7f15dc00c4d0 tx=0x7f15dc00c990 comp rx=0 tx=0).stop 2023-08-09T13:36:12.790 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:12.787+0000 7f15e7078640 1 -- 172.21.15.43:0/466132248 shutdown_connections 2023-08-09T13:36:12.791 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:12.787+0000 7f15e7078640 1 --2- 172.21.15.43:0/466132248 >> [v2:172.21.15.203:6832/830097474,v1:172.21.15.203:6833/830097474] conn(0x7f15c406a080 0x7f15c406c540 unknown :-1 s=CLOSED pgs=272 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:36:12.791 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:12.787+0000 7f15e7078640 1 --2- 172.21.15.43:0/466132248 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f15e0160cb0 0x7f15e0098ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:36:12.791 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:12.787+0000 7f15e7078640 1 --2- 172.21.15.43:0/466132248 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f15e0160760 0x7f15e009b4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:36:12.791 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:12.787+0000 7f15e7078640 1 --2- 172.21.15.43:0/466132248 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f15e0160380 0x7f15e0148100 unknown :-1 s=CLOSED pgs=563 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:36:12.791 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:12.787+0000 7f15e7078640 1 -- 172.21.15.43:0/466132248 >> 172.21.15.43:0/466132248 conn(0x7f15e00cde90 msgr2=0x7f15e00ca910 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-09T13:36:12.791 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:12.788+0000 7f15e7078640 1 -- 172.21.15.43:0/466132248 shutdown_connections 2023-08-09T13:36:12.791 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:12.788+0000 7f15e7078640 1 -- 172.21.15.43:0/466132248 wait complete. 2023-08-09T13:36:12.809 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs standby_count_wanted 0 2023-08-09T13:36:13.020 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:13.017+0000 7f9f6e2da640 1 Processor -- start 2023-08-09T13:36:13.021 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:13.018+0000 7f9f6e2da640 1 -- start start 2023-08-09T13:36:13.022 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:13.020+0000 7f9f6e2da640 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7f9f680ccf20 0x7f9f680cd550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:36:13.022 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:13.020+0000 7f9f6e2da640 1 -- --> v1:172.21.15.43:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9f680cca90 con 0x7f9f6815c630 2023-08-09T13:36:13.022 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:13.020+0000 7f9f6e2da640 1 -- --> v1:172.21.15.203:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9f680cdea0 con 0x7f9f680cda90 2023-08-09T13:36:13.023 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:13.020+0000 7f9f6e2da640 1 -- --> v2:172.21.15.203:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9f680ac600 con 0x7f9f680ccf20 2023-08-09T13:36:13.023 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:13.020+0000 7f9f67fff640 1 --1- >> v1:172.21.15.43:6789/0 conn(0x7f9f6815c630 0x7f9f680abef0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.43:6789/0 says I am v1:172.21.15.43:39476/0 (socket says 172.21.15.43:39476) 2023-08-09T13:36:13.023 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:13.020+0000 7f9f67fff640 1 -- 172.21.15.43:0/143964515 learned_addr learned my addr 172.21.15.43:0/143964515 (peer_addr_for_me v1:172.21.15.43:0/0) 2023-08-09T13:36:13.024 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:13.020+0000 7f9f677fe640 1 -- 172.21.15.43:0/143964515 <== mon.0 v1:172.21.15.43:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 174003296 0 0) 0x7f9f680cca90 con 0x7f9f6815c630 2023-08-09T13:36:13.024 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:13.020+0000 7f9f677fe640 1 -- 172.21.15.43:0/143964515 --> v1:172.21.15.43:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f9f54003520 con 0x7f9f6815c630 2023-08-09T13:36:13.025 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:13.020+0000 7f9f5f7fe640 1 --2- 172.21.15.43:0/143964515 >> v2:172.21.15.203:3300/0 conn(0x7f9f680ccf20 0x7f9f680cd550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:36:13.025 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:13.021+0000 7f9f677fe640 1 -- 172.21.15.43:0/143964515 <== mon.0 v1:172.21.15.43:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 593301813 0 0) 0x7f9f54003520 con 0x7f9f6815c630 2023-08-09T13:36:13.025 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:13.021+0000 7f9f677fe640 1 -- 172.21.15.43:0/143964515 >> v2:172.21.15.203:3300/0 conn(0x7f9f680ccf20 msgr2=0x7f9f680cd550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-09T13:36:13.025 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:13.021+0000 7f9f677fe640 1 --2- 172.21.15.43:0/143964515 >> v2:172.21.15.203:3300/0 conn(0x7f9f680ccf20 0x7f9f680cd550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:36:13.025 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:13.021+0000 7f9f677fe640 1 -- 172.21.15.43:0/143964515 >> v1:172.21.15.203:6789/0 conn(0x7f9f680cda90 legacy=0x7f9f6815ca10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-09T13:36:13.025 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:13.021+0000 7f9f677fe640 1 -- 172.21.15.43:0/143964515 --> v1:172.21.15.43:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9f680cbe80 con 0x7f9f6815c630 2023-08-09T13:36:13.025 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:13.021+0000 7f9f677fe640 1 -- 172.21.15.43:0/143964515 <== mon.0 v1:172.21.15.43:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1285150740 0 0) 0x7f9f580019f0 con 0x7f9f6815c630 2023-08-09T13:36:13.026 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:13.021+0000 7f9f677fe640 1 -- 172.21.15.43:0/143964515 >> v1:172.21.15.43:6789/0 conn(0x7f9f6815c630 legacy=0x7f9f680abef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:36:13.026 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:13.021+0000 7f9f677fe640 1 --2- 172.21.15.43:0/143964515 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f9f540039e0 0x7f9f54003e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:36:13.026 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:13.021+0000 7f9f677fe640 1 --2- 172.21.15.43:0/143964515 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f9f540047c0 0x7f9f54004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:36:13.026 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:13.021+0000 7f9f677fe640 1 --2- 172.21.15.43:0/143964515 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f9f540051f0 0x7f9f5400b980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:36:13.026 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:13.021+0000 7f9f677fe640 1 -- 172.21.15.43:0/143964515 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9f680ac600 con 0x7f9f540051f0 2023-08-09T13:36:13.026 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:13.021+0000 7f9f677fe640 1 -- 172.21.15.43:0/143964515 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9f5400c150 con 0x7f9f540047c0 2023-08-09T13:36:13.026 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:13.021+0000 7f9f677fe640 1 -- 172.21.15.43:0/143964515 --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9f5400c310 con 0x7f9f540039e0 2023-08-09T13:36:13.026 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:13.021+0000 7f9f6c877640 1 --2- 172.21.15.43:0/143964515 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f9f540051f0 0x7f9f5400b980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:36:13.027 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:13.022+0000 7f9f5f7fe640 1 --2- 172.21.15.43:0/143964515 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f9f540047c0 0x7f9f54004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:36:13.027 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:13.022+0000 7f9f67fff640 1 --2- 172.21.15.43:0/143964515 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f9f540039e0 0x7f9f54003e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:36:13.027 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:13.022+0000 7f9f6c877640 1 -- 172.21.15.43:0/143964515 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f9f540039e0 msgr2=0x7f9f54003e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-09T13:36:13.027 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:13.022+0000 7f9f6c877640 1 --2- 172.21.15.43:0/143964515 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f9f540039e0 0x7f9f54003e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:36:13.027 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:13.022+0000 7f9f6c877640 1 -- 172.21.15.43:0/143964515 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f9f540047c0 msgr2=0x7f9f54004c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-09T13:36:13.027 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:13.022+0000 7f9f6c877640 1 --2- 172.21.15.43:0/143964515 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f9f540047c0 0x7f9f54004c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:36:13.027 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:13.022+0000 7f9f6c877640 1 -- 172.21.15.43:0/143964515 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9f54003520 con 0x7f9f540051f0 2023-08-09T13:36:13.028 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:13.023+0000 7f9f6c877640 1 -- 172.21.15.43:0/143964515 reap_dead start 2023-08-09T13:36:13.029 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:13.023+0000 7f9f6c877640 1 --2- 172.21.15.43:0/143964515 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f9f540051f0 0x7f9f5400b980 secure :-1 s=READY pgs=565 cs=0 l=1 rev1=1 crypto rx=0x7f9f60001750 tx=0x7f9f540057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9eb76c123d552332 server_cookie=0 in_seq=0 out_seq=0 2023-08-09T13:36:13.029 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:13.023+0000 7f9f677fe640 1 -- 172.21.15.43:0/143964515 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9f600052d0 con 0x7f9f540051f0 2023-08-09T13:36:13.029 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:13.023+0000 7f9f677fe640 1 -- 172.21.15.43:0/143964515 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9f6002e050 con 0x7f9f540051f0 2023-08-09T13:36:13.029 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:13.023+0000 7f9f677fe640 1 -- 172.21.15.43:0/143964515 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9f60003830 con 0x7f9f540051f0 2023-08-09T13:36:13.029 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:13.024+0000 7f9f6e2da640 1 -- 172.21.15.43:0/143964515 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f9f540051f0 msgr2=0x7f9f5400b980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:36:13.029 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:13.024+0000 7f9f6e2da640 1 --2- 172.21.15.43:0/143964515 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f9f540051f0 0x7f9f5400b980 secure :-1 s=READY pgs=565 cs=0 l=1 rev1=1 crypto rx=0x7f9f60001750 tx=0x7f9f540057f0 comp rx=0 tx=0).stop 2023-08-09T13:36:13.030 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:13.024+0000 7f9f6e2da640 1 -- 172.21.15.43:0/143964515 shutdown_connections 2023-08-09T13:36:13.030 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:13.024+0000 7f9f6e2da640 1 --2- 172.21.15.43:0/143964515 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f9f540051f0 0x7f9f5400b980 unknown :-1 s=CLOSED pgs=565 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:36:13.030 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:13.024+0000 7f9f6e2da640 1 -- 172.21.15.43:0/143964515 >> 172.21.15.43:0/143964515 conn(0x7f9f680ce0a0 msgr2=0x7f9f680ce480 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-09T13:36:13.030 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:13.024+0000 7f9f6e2da640 1 -- 172.21.15.43:0/143964515 shutdown_connections 2023-08-09T13:36:13.030 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:13.024+0000 7f9f6e2da640 1 -- 172.21.15.43:0/143964515 wait complete. 2023-08-09T13:36:13.030 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:13.027+0000 7f9f6e2da640 1 Processor -- start 2023-08-09T13:36:13.030 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:13.027+0000 7f9f6e2da640 1 -- start start 2023-08-09T13:36:13.032 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:13.027+0000 7f9f6e2da640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f9f540051f0 0x7f9f680ae190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:36:13.032 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:13.027+0000 7f9f6e2da640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f9f68070ce0 0x7f9f68159180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:36:13.032 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:13.027+0000 7f9f6e2da640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f9f68071160 0x7f9f681596c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:36:13.032 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:13.027+0000 7f9f6e2da640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9f680ae6d0 con 0x7f9f540051f0 2023-08-09T13:36:13.032 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:13.027+0000 7f9f6e2da640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9f6806f8b0 con 0x7f9f68071160 2023-08-09T13:36:13.032 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:13.027+0000 7f9f6e2da640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9f6806fb90 con 0x7f9f68070ce0 2023-08-09T13:36:13.033 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:13.028+0000 7f9f67fff640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f9f540051f0 0x7f9f680ae190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:36:13.033 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:13.028+0000 7f9f67fff640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f9f540051f0 0x7f9f680ae190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:36042/0 (socket says 172.21.15.43:36042) 2023-08-09T13:36:13.033 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:13.028+0000 7f9f67fff640 1 -- 172.21.15.43:0/4215940090 learned_addr learned my addr 172.21.15.43:0/4215940090 (peer_addr_for_me v2:172.21.15.43:0/0) 2023-08-09T13:36:13.033 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:13.028+0000 7f9f67fff640 1 -- 172.21.15.43:0/4215940090 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f9f68070ce0 msgr2=0x7f9f68159180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:36:13.033 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:13.028+0000 7f9f5f7fe640 1 --2- 172.21.15.43:0/4215940090 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f9f68070ce0 0x7f9f68159180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:36:13.033 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:13.028+0000 7f9f6c877640 1 --2- 172.21.15.43:0/4215940090 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f9f68071160 0x7f9f681596c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:36:13.033 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:13.028+0000 7f9f67fff640 1 --2- 172.21.15.43:0/4215940090 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f9f68070ce0 0x7f9f68159180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:36:13.033 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:13.028+0000 7f9f67fff640 1 -- 172.21.15.43:0/4215940090 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f9f68071160 msgr2=0x7f9f681596c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:36:13.034 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:13.028+0000 7f9f67fff640 1 --2- 172.21.15.43:0/4215940090 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f9f68071160 0x7f9f681596c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:36:13.034 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:13.028+0000 7f9f67fff640 1 -- 172.21.15.43:0/4215940090 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9f6806fe70 con 0x7f9f540051f0 2023-08-09T13:36:13.034 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:13.028+0000 7f9f5f7fe640 1 --2- 172.21.15.43:0/4215940090 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f9f68070ce0 0x7f9f68159180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-08-09T13:36:13.034 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:13.028+0000 7f9f67fff640 1 --2- 172.21.15.43:0/4215940090 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f9f540051f0 0x7f9f680ae190 secure :-1 s=READY pgs=566 cs=0 l=1 rev1=1 crypto rx=0x7f9f58007a10 tx=0x7f9f5800e530 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-09T13:36:13.034 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:13.029+0000 7f9f657fa640 1 -- 172.21.15.43:0/4215940090 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9f5800ee50 con 0x7f9f540051f0 2023-08-09T13:36:13.034 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:13.029+0000 7f9f657fa640 1 -- 172.21.15.43:0/4215940090 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9f58013420 con 0x7f9f540051f0 2023-08-09T13:36:13.034 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:13.029+0000 7f9f657fa640 1 -- 172.21.15.43:0/4215940090 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9f580137d0 con 0x7f9f540051f0 2023-08-09T13:36:13.034 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:13.029+0000 7f9f6e2da640 1 -- 172.21.15.43:0/4215940090 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9f68070150 con 0x7f9f540051f0 2023-08-09T13:36:13.035 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:13.029+0000 7f9f6e2da640 1 -- 172.21.15.43:0/4215940090 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9f6806f440 con 0x7f9f540051f0 2023-08-09T13:36:13.035 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:13.030+0000 7f9f6e2da640 1 -- 172.21.15.43:0/4215940090 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9f68070670 con 0x7f9f540051f0 2023-08-09T13:36:13.035 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:13.031+0000 7f9f657fa640 1 -- 172.21.15.43:0/4215940090 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 5) v1 ==== 88341+0+0 (secure 0 0 0) 0x7f9f58013930 con 0x7f9f540051f0 2023-08-09T13:36:13.036 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:13.032+0000 7f9f657fa640 1 --2- 172.21.15.43:0/4215940090 >> [v2:172.21.15.203:6832/830097474,v1:172.21.15.203:6833/830097474] conn(0x7f9f4806a050 0x7f9f4806c510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:36:13.036 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:13.032+0000 7f9f657fa640 1 -- 172.21.15.43:0/4215940090 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(39..39 src has 1..39) v4 ==== 6382+0+0 (secure 0 0 0) 0x7f9f58088f60 con 0x7f9f540051f0 2023-08-09T13:36:13.036 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:13.033+0000 7f9f5f7fe640 1 --2- 172.21.15.43:0/4215940090 >> [v2:172.21.15.203:6832/830097474,v1:172.21.15.203:6833/830097474] conn(0x7f9f4806a050 0x7f9f4806c510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:36:13.036 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:13.034+0000 7f9f5f7fe640 1 --2- 172.21.15.43:0/4215940090 >> [v2:172.21.15.203:6832/830097474,v1:172.21.15.203:6833/830097474] conn(0x7f9f4806a050 0x7f9f4806c510 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7f9f681f5100 tx=0x7f9f50008040 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-09T13:36:13.038 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:13.036+0000 7f9f657fa640 1 -- 172.21.15.43:0/4215940090 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178066 (secure 0 0 0) 0x7f9f68070670 con 0x7f9f540051f0 2023-08-09T13:36:13.278 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:13.275+0000 7f9f6e2da640 1 -- 172.21.15.43:0/4215940090 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} v 0) v1 -- 0x7f9f68070670 con 0x7f9f540051f0 2023-08-09T13:36:13.910 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:13.908+0000 7f9f657fa640 1 -- 172.21.15.43:0/4215940090 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]=0 v16) v1 ==== 118+0+0 (secure 0 0 0) 0x7f9f68070670 con 0x7f9f540051f0 2023-08-09T13:36:13.913 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:13.911+0000 7f9f6e2da640 1 -- 172.21.15.43:0/4215940090 >> [v2:172.21.15.203:6832/830097474,v1:172.21.15.203:6833/830097474] conn(0x7f9f4806a050 msgr2=0x7f9f4806c510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:36:13.914 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:13.911+0000 7f9f6e2da640 1 --2- 172.21.15.43:0/4215940090 >> [v2:172.21.15.203:6832/830097474,v1:172.21.15.203:6833/830097474] conn(0x7f9f4806a050 0x7f9f4806c510 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7f9f681f5100 tx=0x7f9f50008040 comp rx=0 tx=0).stop 2023-08-09T13:36:13.914 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:13.911+0000 7f9f6e2da640 1 -- 172.21.15.43:0/4215940090 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f9f540051f0 msgr2=0x7f9f680ae190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:36:13.914 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:13.911+0000 7f9f6e2da640 1 --2- 172.21.15.43:0/4215940090 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f9f540051f0 0x7f9f680ae190 secure :-1 s=READY pgs=566 cs=0 l=1 rev1=1 crypto rx=0x7f9f58007a10 tx=0x7f9f5800e530 comp rx=0 tx=0).stop 2023-08-09T13:36:13.914 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:13.911+0000 7f9f6e2da640 1 -- 172.21.15.43:0/4215940090 shutdown_connections 2023-08-09T13:36:13.914 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:13.911+0000 7f9f6e2da640 1 --2- 172.21.15.43:0/4215940090 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f9f68071160 0x7f9f681596c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:36:13.914 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:13.911+0000 7f9f6e2da640 1 --2- 172.21.15.43:0/4215940090 >> [v2:172.21.15.203:6832/830097474,v1:172.21.15.203:6833/830097474] conn(0x7f9f4806a050 0x7f9f4806c510 unknown :-1 s=CLOSED pgs=273 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:36:13.915 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:13.911+0000 7f9f6e2da640 1 --2- 172.21.15.43:0/4215940090 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f9f68070ce0 0x7f9f68159180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:36:13.915 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:13.911+0000 7f9f6e2da640 1 --2- 172.21.15.43:0/4215940090 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f9f540051f0 0x7f9f680ae190 unknown :-1 s=CLOSED pgs=566 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:36:13.915 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:13.911+0000 7f9f6e2da640 1 -- 172.21.15.43:0/4215940090 >> 172.21.15.43:0/4215940090 conn(0x7f9f680ce0a0 msgr2=0x7f9f6815bc00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-09T13:36:13.915 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:13.911+0000 7f9f6e2da640 1 -- 172.21.15.43:0/4215940090 shutdown_connections 2023-08-09T13:36:13.915 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:13.911+0000 7f9f6e2da640 1 -- 172.21.15.43:0/4215940090 wait complete. 2023-08-09T13:36:13.935 DEBUG:tasks.cephfs.filesystem:fs_config: {} 2023-08-09T13:36:13.935 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs allow_standby_replay false 2023-08-09T13:36:14.170 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:14.168+0000 7f2b1fb0f640 1 Processor -- start 2023-08-09T13:36:14.170 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:14.169+0000 7f2b1fb0f640 1 -- start start 2023-08-09T13:36:14.171 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:14.170+0000 7f2b1fb0f640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f2b180c86f0 0x7f2b180c8b50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:36:14.171 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:14.170+0000 7f2b1fb0f640 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7f2b1806c8d0 0x7f2b1806cd60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:36:14.172 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:14.170+0000 7f2b1fb0f640 1 -- --> v1:172.21.15.203:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2b180c9090 con 0x7f2b18143c20 2023-08-09T13:36:14.172 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:14.170+0000 7f2b1fb0f640 1 -- --> v2:172.21.15.203:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2b18144000 con 0x7f2b1806c8d0 2023-08-09T13:36:14.172 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:14.170+0000 7f2b1fb0f640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2b1806c440 con 0x7f2b180c86f0 2023-08-09T13:36:14.173 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:14.171+0000 7f2b1d0aa640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f2b180c86f0 0x7f2b180c8b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:36:14.173 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:14.171+0000 7f2b1d0aa640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f2b180c86f0 0x7f2b180c8b50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.203:3301/0 says I am v2:172.21.15.43:56440/0 (socket says 172.21.15.43:56440) 2023-08-09T13:36:14.173 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:14.171+0000 7f2b1d0aa640 1 -- 172.21.15.43:0/1236137395 learned_addr learned my addr 172.21.15.43:0/1236137395 (peer_addr_for_me v2:172.21.15.43:0/0) 2023-08-09T13:36:14.173 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:14.171+0000 7f2b1e0ac640 1 --2- 172.21.15.43:0/1236137395 >> v2:172.21.15.203:3300/0 conn(0x7f2b1806c8d0 0x7f2b1806cd60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:36:14.173 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:14.172+0000 7f2b1d0aa640 1 -- 172.21.15.43:0/1236137395 >> v2:172.21.15.203:3300/0 conn(0x7f2b1806c8d0 msgr2=0x7f2b1806cd60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-09T13:36:14.174 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:14.172+0000 7f2b1d0aa640 1 --2- 172.21.15.43:0/1236137395 >> v2:172.21.15.203:3300/0 conn(0x7f2b1806c8d0 0x7f2b1806cd60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:36:14.174 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:14.172+0000 7f2b1d0aa640 1 -- 172.21.15.43:0/1236137395 >> v1:172.21.15.203:6789/0 conn(0x7f2b18143c20 legacy=0x7f2b1806f540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:36:14.174 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:14.172+0000 7f2b1d0aa640 1 -- 172.21.15.43:0/1236137395 --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2b1806fc50 con 0x7f2b180c86f0 2023-08-09T13:36:14.174 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:14.173+0000 7f2b1d0aa640 1 --2- 172.21.15.43:0/1236137395 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f2b180c86f0 0x7f2b180c8b50 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f2b08009870 tx=0x7f2b0802b8f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c395d8b4b7385ac9 server_cookie=0 in_seq=0 out_seq=0 2023-08-09T13:36:14.175 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:14.174+0000 7f2b1c8a9640 1 -- 172.21.15.43:0/1236137395 <== mon.2 v2:172.21.15.203:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2b08031640 con 0x7f2b180c86f0 2023-08-09T13:36:14.175 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:14.174+0000 7f2b1c8a9640 1 -- 172.21.15.43:0/1236137395 <== mon.2 v2:172.21.15.203:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2b080317a0 con 0x7f2b180c86f0 2023-08-09T13:36:14.175 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:14.174+0000 7f2b1c8a9640 1 -- 172.21.15.43:0/1236137395 <== mon.2 v2:172.21.15.203:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2b08031b30 con 0x7f2b180c86f0 2023-08-09T13:36:14.176 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:14.174+0000 7f2b1fb0f640 1 -- 172.21.15.43:0/1236137395 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f2b180c86f0 msgr2=0x7f2b180c8b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:36:14.176 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:14.174+0000 7f2b1fb0f640 1 --2- 172.21.15.43:0/1236137395 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f2b180c86f0 0x7f2b180c8b50 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f2b08009870 tx=0x7f2b0802b8f0 comp rx=0 tx=0).stop 2023-08-09T13:36:14.177 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:14.175+0000 7f2b1fb0f640 1 -- 172.21.15.43:0/1236137395 shutdown_connections 2023-08-09T13:36:14.177 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:14.175+0000 7f2b1fb0f640 1 --2- 172.21.15.43:0/1236137395 >> v2:172.21.15.203:3300/0 conn(0x7f2b1806c8d0 0x7f2b1806cd60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:36:14.177 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:14.175+0000 7f2b1fb0f640 1 --2- 172.21.15.43:0/1236137395 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f2b180c86f0 0x7f2b180c8b50 unknown :-1 s=CLOSED pgs=78 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:36:14.177 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:14.175+0000 7f2b1fb0f640 1 -- 172.21.15.43:0/1236137395 >> 172.21.15.43:0/1236137395 conn(0x7f2b180ce0a0 msgr2=0x7f2b180ce480 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-09T13:36:14.177 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:14.175+0000 7f2b1fb0f640 1 -- 172.21.15.43:0/1236137395 shutdown_connections 2023-08-09T13:36:14.177 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:14.175+0000 7f2b1fb0f640 1 -- 172.21.15.43:0/1236137395 wait complete. 2023-08-09T13:36:14.178 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:14.176+0000 7f2b1fb0f640 1 Processor -- start 2023-08-09T13:36:14.178 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:14.177+0000 7f2b1fb0f640 1 -- start start 2023-08-09T13:36:14.179 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:14.177+0000 7f2b1fb0f640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f2b1806c8d0 0x7f2b180c4820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:36:14.179 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:14.177+0000 7f2b1fb0f640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f2b180c86f0 0x7f2b180c7f40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:36:14.179 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:14.178+0000 7f2b1fb0f640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f2b18143c20 0x7f2b180c7070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:36:14.180 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:14.178+0000 7f2b1fb0f640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2b1813e370 con 0x7f2b18143c20 2023-08-09T13:36:14.180 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:14.178+0000 7f2b1fb0f640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2b180c8480 con 0x7f2b180c86f0 2023-08-09T13:36:14.180 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:14.178+0000 7f2b1fb0f640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2b180c62b0 con 0x7f2b1806c8d0 2023-08-09T13:36:14.180 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:14.178+0000 7f2b1e0ac640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f2b18143c20 0x7f2b180c7070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:36:14.180 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:14.178+0000 7f2b1e0ac640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f2b18143c20 0x7f2b180c7070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:36052/0 (socket says 172.21.15.43:36052) 2023-08-09T13:36:14.180 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:14.178+0000 7f2b1e0ac640 1 -- 172.21.15.43:0/1435277208 learned_addr learned my addr 172.21.15.43:0/1435277208 (peer_addr_for_me v2:172.21.15.43:0/0) 2023-08-09T13:36:14.181 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:14.178+0000 7f2b1e0ac640 1 -- 172.21.15.43:0/1435277208 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f2b1806c8d0 msgr2=0x7f2b180c4820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:36:14.181 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:14.178+0000 7f2b1d8ab640 1 --2- 172.21.15.43:0/1435277208 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f2b1806c8d0 0x7f2b180c4820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:36:14.182 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:14.178+0000 7f2b1d0aa640 1 --2- 172.21.15.43:0/1435277208 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f2b180c86f0 0x7f2b180c7f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:36:14.182 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:14.178+0000 7f2b1e0ac640 1 --2- 172.21.15.43:0/1435277208 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f2b1806c8d0 0x7f2b180c4820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:36:14.182 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:14.178+0000 7f2b1e0ac640 1 -- 172.21.15.43:0/1435277208 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f2b180c86f0 msgr2=0x7f2b180c7f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:36:14.182 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:14.178+0000 7f2b1e0ac640 1 --2- 172.21.15.43:0/1435277208 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f2b180c86f0 0x7f2b180c7f40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:36:14.182 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:14.178+0000 7f2b1e0ac640 1 -- 172.21.15.43:0/1435277208 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2b080094d0 con 0x7f2b18143c20 2023-08-09T13:36:14.182 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:14.178+0000 7f2b1d8ab640 1 --2- 172.21.15.43:0/1435277208 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f2b1806c8d0 0x7f2b180c4820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-08-09T13:36:14.182 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:14.179+0000 7f2b1e0ac640 1 --2- 172.21.15.43:0/1435277208 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f2b18143c20 0x7f2b180c7070 secure :-1 s=READY pgs=567 cs=0 l=1 rev1=1 crypto rx=0x7f2b1000e7f0 tx=0x7f2b1000ecb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-09T13:36:14.183 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:14.179+0000 7f2b0e7fc640 1 -- 172.21.15.43:0/1435277208 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2b10009460 con 0x7f2b18143c20 2023-08-09T13:36:14.183 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:14.179+0000 7f2b0e7fc640 1 -- 172.21.15.43:0/1435277208 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2b1000c730 con 0x7f2b18143c20 2023-08-09T13:36:14.183 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:14.179+0000 7f2b0e7fc640 1 -- 172.21.15.43:0/1435277208 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2b1000cae0 con 0x7f2b18143c20 2023-08-09T13:36:14.183 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:14.179+0000 7f2b1fb0f640 1 -- 172.21.15.43:0/1435277208 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2b180c6590 con 0x7f2b18143c20 2023-08-09T13:36:14.183 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:14.179+0000 7f2b1fb0f640 1 -- 172.21.15.43:0/1435277208 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2b180c6b50 con 0x7f2b18143c20 2023-08-09T13:36:14.184 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:14.180+0000 7f2b1fb0f640 1 -- 172.21.15.43:0/1435277208 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2b180c6d70 con 0x7f2b18143c20 2023-08-09T13:36:14.184 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:14.181+0000 7f2b0e7fc640 1 -- 172.21.15.43:0/1435277208 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 5) v1 ==== 88341+0+0 (secure 0 0 0) 0x7f2b1000cc40 con 0x7f2b18143c20 2023-08-09T13:36:14.184 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:14.182+0000 7f2b0e7fc640 1 --2- 172.21.15.43:0/1435277208 >> [v2:172.21.15.203:6832/830097474,v1:172.21.15.203:6833/830097474] conn(0x7f2afc06a050 0x7f2afc06c510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:36:14.184 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:14.182+0000 7f2b0e7fc640 1 -- 172.21.15.43:0/1435277208 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(39..39 src has 1..39) v4 ==== 6382+0+0 (secure 0 0 0) 0x7f2b10088e30 con 0x7f2b18143c20 2023-08-09T13:36:14.184 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:14.183+0000 7f2b1d8ab640 1 --2- 172.21.15.43:0/1435277208 >> [v2:172.21.15.203:6832/830097474,v1:172.21.15.203:6833/830097474] conn(0x7f2afc06a050 0x7f2afc06c510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:36:14.185 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:14.184+0000 7f2b1d8ab640 1 --2- 172.21.15.43:0/1435277208 >> [v2:172.21.15.203:6832/830097474,v1:172.21.15.203:6833/830097474] conn(0x7f2afc06a050 0x7f2afc06c510 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7f2b180b3950 tx=0x7f2b14009040 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-09T13:36:14.187 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:14.186+0000 7f2b0e7fc640 1 -- 172.21.15.43:0/1435277208 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178066 (secure 0 0 0) 0x7f2b100543d0 con 0x7f2b18143c20 2023-08-09T13:36:14.416 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:14.415+0000 7f2b1fb0f640 1 -- 172.21.15.43:0/1435277208 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"} v 0) v1 -- 0x7f2b180739f0 con 0x7f2b18143c20 2023-08-09T13:36:14.914 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:14.913+0000 7f2b0e7fc640 1 -- 172.21.15.43:0/1435277208 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"}]=0 v17) v1 ==== 122+0+0 (secure 0 0 0) 0x7f2b100210c0 con 0x7f2b18143c20 2023-08-09T13:36:14.918 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:14.917+0000 7f2b1fb0f640 1 -- 172.21.15.43:0/1435277208 >> [v2:172.21.15.203:6832/830097474,v1:172.21.15.203:6833/830097474] conn(0x7f2afc06a050 msgr2=0x7f2afc06c510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:36:14.919 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:14.917+0000 7f2b1fb0f640 1 --2- 172.21.15.43:0/1435277208 >> [v2:172.21.15.203:6832/830097474,v1:172.21.15.203:6833/830097474] conn(0x7f2afc06a050 0x7f2afc06c510 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7f2b180b3950 tx=0x7f2b14009040 comp rx=0 tx=0).stop 2023-08-09T13:36:14.919 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:14.917+0000 7f2b1fb0f640 1 -- 172.21.15.43:0/1435277208 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f2b18143c20 msgr2=0x7f2b180c7070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:36:14.919 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:14.917+0000 7f2b1fb0f640 1 --2- 172.21.15.43:0/1435277208 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f2b18143c20 0x7f2b180c7070 secure :-1 s=READY pgs=567 cs=0 l=1 rev1=1 crypto rx=0x7f2b1000e7f0 tx=0x7f2b1000ecb0 comp rx=0 tx=0).stop 2023-08-09T13:36:14.920 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:14.917+0000 7f2b1fb0f640 1 -- 172.21.15.43:0/1435277208 shutdown_connections 2023-08-09T13:36:14.920 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:14.917+0000 7f2b1fb0f640 1 --2- 172.21.15.43:0/1435277208 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f2b18143c20 0x7f2b180c7070 unknown :-1 s=CLOSED pgs=567 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:36:14.920 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:14.917+0000 7f2b1fb0f640 1 --2- 172.21.15.43:0/1435277208 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f2b180c86f0 0x7f2b180c7f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:36:14.920 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:14.917+0000 7f2b1fb0f640 1 --2- 172.21.15.43:0/1435277208 >> [v2:172.21.15.203:6832/830097474,v1:172.21.15.203:6833/830097474] conn(0x7f2afc06a050 0x7f2afc06c510 unknown :-1 s=CLOSED pgs=274 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:36:14.920 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:14.917+0000 7f2b1fb0f640 1 --2- 172.21.15.43:0/1435277208 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f2b1806c8d0 0x7f2b180c4820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:36:14.921 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:14.918+0000 7f2b1fb0f640 1 -- 172.21.15.43:0/1435277208 >> 172.21.15.43:0/1435277208 conn(0x7f2b180ce0a0 msgr2=0x7f2b1806fe40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-09T13:36:14.921 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:14.918+0000 7f2b1fb0f640 1 -- 172.21.15.43:0/1435277208 shutdown_connections 2023-08-09T13:36:14.921 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:14.918+0000 7f2b1fb0f640 1 -- 172.21.15.43:0/1435277208 wait complete. 2023-08-09T13:36:14.942 DEBUG:tasks.cephfs.filesystem:Not Creating any subvols for filesystem 'cephfs' 2023-08-09T13:36:14.942 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-08-09T13:36:15.164 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:15.163+0000 7f9abe996640 1 Processor -- start 2023-08-09T13:36:15.165 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:15.163+0000 7f9abe996640 1 -- start start 2023-08-09T13:36:15.166 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:15.165+0000 7f9abe996640 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7f9ab815c630 0x7f9ab80abef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:36:15.166 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:15.165+0000 7f9abe996640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f9ab80ccf20 0x7f9ab80cd550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:36:15.167 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:15.165+0000 7f9abe996640 1 -- --> v1:172.21.15.203:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9ab80cca90 con 0x7f9ab80cda90 2023-08-09T13:36:15.167 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:15.165+0000 7f9abe996640 1 -- --> v2:172.21.15.203:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9ab80cbc70 con 0x7f9ab815c630 2023-08-09T13:36:15.167 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:15.165+0000 7f9abe996640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9ab80cdf00 con 0x7f9ab80ccf20 2023-08-09T13:36:15.167 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:15.166+0000 7f9abcf33640 1 --1- >> v1:172.21.15.203:6789/0 conn(0x7f9ab80cda90 0x7f9ab80c2f20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.203:6789/0 says I am v1:172.21.15.43:45256/0 (socket says 172.21.15.43:45256) 2023-08-09T13:36:15.167 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:15.166+0000 7f9ab77fe640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f9ab80ccf20 0x7f9ab80cd550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:36:15.168 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:15.166+0000 7f9abcf33640 1 -- 172.21.15.43:0/3708632710 learned_addr learned my addr 172.21.15.43:0/3708632710 (peer_addr_for_me v1:172.21.15.43:0/0) 2023-08-09T13:36:15.168 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:15.166+0000 7f9ab7fff640 1 --2- 172.21.15.43:0/3708632710 >> v2:172.21.15.203:3300/0 conn(0x7f9ab815c630 0x7f9ab80abef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:36:15.169 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:15.167+0000 7f9ab6ffd640 1 -- 172.21.15.43:0/3708632710 <== mon.1 v1:172.21.15.203:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3401459983 0 0) 0x7f9ab80cca90 con 0x7f9ab80cda90 2023-08-09T13:36:15.169 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:15.167+0000 7f9ab6ffd640 1 -- 172.21.15.43:0/3708632710 --> v1:172.21.15.203:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f9aa0003520 con 0x7f9ab80cda90 2023-08-09T13:36:15.169 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:15.167+0000 7f9ab77fe640 1 -- 172.21.15.43:0/3708632710 >> v2:172.21.15.203:3300/0 conn(0x7f9ab815c630 msgr2=0x7f9ab80abef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-09T13:36:15.169 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:15.167+0000 7f9ab77fe640 1 --2- 172.21.15.43:0/3708632710 >> v2:172.21.15.203:3300/0 conn(0x7f9ab815c630 0x7f9ab80abef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:36:15.169 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:15.167+0000 7f9ab77fe640 1 -- 172.21.15.43:0/3708632710 >> v1:172.21.15.203:6789/0 conn(0x7f9ab80cda90 legacy=0x7f9ab80c2f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:36:15.169 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:15.167+0000 7f9ab77fe640 1 -- 172.21.15.43:0/3708632710 --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9ab815da20 con 0x7f9ab80ccf20 2023-08-09T13:36:15.169 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:15.167+0000 7f9ab7fff640 1 --2- 172.21.15.43:0/3708632710 >> v2:172.21.15.203:3300/0 conn(0x7f9ab815c630 0x7f9ab80abef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-08-09T13:36:15.170 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:15.169+0000 7f9ab77fe640 1 --2- 172.21.15.43:0/3708632710 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f9ab80ccf20 0x7f9ab80cd550 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f9aac0097a0 tx=0x7f9aac02b8b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=bad2f3f073d161d8 server_cookie=0 in_seq=0 out_seq=0 2023-08-09T13:36:15.170 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:15.169+0000 7f9ab6ffd640 1 -- 172.21.15.43:0/3708632710 <== mon.2 v2:172.21.15.203:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9aac02d040 con 0x7f9ab80ccf20 2023-08-09T13:36:15.171 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:15.169+0000 7f9ab6ffd640 1 -- 172.21.15.43:0/3708632710 <== mon.2 v2:172.21.15.203:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9aac031810 con 0x7f9ab80ccf20 2023-08-09T13:36:15.171 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:15.169+0000 7f9ab6ffd640 1 -- 172.21.15.43:0/3708632710 <== mon.2 v2:172.21.15.203:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9aac031ba0 con 0x7f9ab80ccf20 2023-08-09T13:36:15.171 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:15.170+0000 7f9abe996640 1 -- 172.21.15.43:0/3708632710 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f9ab80ccf20 msgr2=0x7f9ab80cd550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:36:15.172 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:15.170+0000 7f9abe996640 1 --2- 172.21.15.43:0/3708632710 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f9ab80ccf20 0x7f9ab80cd550 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f9aac0097a0 tx=0x7f9aac02b8b0 comp rx=0 tx=0).stop 2023-08-09T13:36:15.172 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:15.170+0000 7f9abe996640 1 -- 172.21.15.43:0/3708632710 shutdown_connections 2023-08-09T13:36:15.172 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:15.170+0000 7f9abe996640 1 --2- 172.21.15.43:0/3708632710 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f9ab80ccf20 0x7f9ab80cd550 unknown :-1 s=CLOSED pgs=79 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:36:15.172 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:15.170+0000 7f9abe996640 1 --2- 172.21.15.43:0/3708632710 >> v2:172.21.15.203:3300/0 conn(0x7f9ab815c630 0x7f9ab80abef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:36:15.172 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:15.170+0000 7f9abe996640 1 -- 172.21.15.43:0/3708632710 >> 172.21.15.43:0/3708632710 conn(0x7f9ab80ce0a0 msgr2=0x7f9ab80ce480 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-09T13:36:15.172 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:15.170+0000 7f9abe996640 1 -- 172.21.15.43:0/3708632710 shutdown_connections 2023-08-09T13:36:15.173 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:15.170+0000 7f9abe996640 1 -- 172.21.15.43:0/3708632710 wait complete. 2023-08-09T13:36:15.173 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:15.172+0000 7f9abe996640 1 Processor -- start 2023-08-09T13:36:15.173 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:15.172+0000 7f9abe996640 1 -- start start 2023-08-09T13:36:15.175 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:15.173+0000 7f9abe996640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f9ab80ccf20 0x7f9ab80c54c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:36:15.175 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:15.173+0000 7f9abe996640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f9ab80cda90 0x7f9ab80b3140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:36:15.175 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:15.173+0000 7f9abe996640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f9ab815c630 0x7f9ab8151240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:36:15.175 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:15.173+0000 7f9abe996640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9ab81f2700 con 0x7f9ab80ccf20 2023-08-09T13:36:15.175 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:15.173+0000 7f9abe996640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9ab80b3680 con 0x7f9ab815c630 2023-08-09T13:36:15.175 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:15.173+0000 7f9abe996640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9ab80b3bc0 con 0x7f9ab80cda90 2023-08-09T13:36:15.177 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:15.174+0000 7f9ab7fff640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f9ab80ccf20 0x7f9ab80c54c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:36:15.177 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:15.174+0000 7f9ab7fff640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f9ab80ccf20 0x7f9ab80c54c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:58060/0 (socket says 172.21.15.43:58060) 2023-08-09T13:36:15.177 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:15.174+0000 7f9ab7fff640 1 -- 172.21.15.43:0/973565949 learned_addr learned my addr 172.21.15.43:0/973565949 (peer_addr_for_me v2:172.21.15.43:0/0) 2023-08-09T13:36:15.177 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:15.174+0000 7f9abcf33640 1 --2- 172.21.15.43:0/973565949 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f9ab815c630 0x7f9ab8151240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:36:15.177 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:15.174+0000 7f9ab77fe640 1 --2- 172.21.15.43:0/973565949 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f9ab80cda90 0x7f9ab80b3140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:36:15.177 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:15.174+0000 7f9ab7fff640 1 -- 172.21.15.43:0/973565949 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f9ab80cda90 msgr2=0x7f9ab80b3140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:36:15.177 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:15.174+0000 7f9ab7fff640 1 --2- 172.21.15.43:0/973565949 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f9ab80cda90 0x7f9ab80b3140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:36:15.178 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:15.174+0000 7f9ab7fff640 1 -- 172.21.15.43:0/973565949 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f9ab815c630 msgr2=0x7f9ab8151240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:36:15.178 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:15.174+0000 7f9ab7fff640 1 --2- 172.21.15.43:0/973565949 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f9ab815c630 0x7f9ab8151240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:36:15.178 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:15.174+0000 7f9ab7fff640 1 -- 172.21.15.43:0/973565949 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9aa8009400 con 0x7f9ab80ccf20 2023-08-09T13:36:15.178 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:15.174+0000 7f9abcf33640 1 --2- 172.21.15.43:0/973565949 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f9ab815c630 0x7f9ab8151240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-08-09T13:36:15.178 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:15.175+0000 7f9ab7fff640 1 --2- 172.21.15.43:0/973565949 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f9ab80ccf20 0x7f9ab80c54c0 secure :-1 s=READY pgs=568 cs=0 l=1 rev1=1 crypto rx=0x7f9aa800ebe0 tx=0x7f9aa800c520 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-09T13:36:15.178 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:15.175+0000 7f9ab4ff9640 1 -- 172.21.15.43:0/973565949 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9aa8009760 con 0x7f9ab80ccf20 2023-08-09T13:36:15.178 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:15.175+0000 7f9ab4ff9640 1 -- 172.21.15.43:0/973565949 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9aa800cc30 con 0x7f9ab80ccf20 2023-08-09T13:36:15.178 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:15.175+0000 7f9abe996640 1 -- 172.21.15.43:0/973565949 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9aac009400 con 0x7f9ab80ccf20 2023-08-09T13:36:15.179 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:15.175+0000 7f9ab4ff9640 1 -- 172.21.15.43:0/973565949 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9aa800eee0 con 0x7f9ab80ccf20 2023-08-09T13:36:15.179 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:15.175+0000 7f9abe996640 1 -- 172.21.15.43:0/973565949 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9ab80b38e0 con 0x7f9ab80ccf20 2023-08-09T13:36:15.179 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:15.176+0000 7f9abe996640 1 -- 172.21.15.43:0/973565949 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9ab80739f0 con 0x7f9ab80ccf20 2023-08-09T13:36:15.179 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:15.177+0000 7f9ab4ff9640 1 -- 172.21.15.43:0/973565949 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 5) v1 ==== 88341+0+0 (secure 0 0 0) 0x7f9aa8013470 con 0x7f9ab80ccf20 2023-08-09T13:36:15.180 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:15.178+0000 7f9ab4ff9640 1 --2- 172.21.15.43:0/973565949 >> [v2:172.21.15.203:6832/830097474,v1:172.21.15.203:6833/830097474] conn(0x7f9aa406a050 0x7f9aa406c510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:36:15.180 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:15.178+0000 7f9ab4ff9640 1 -- 172.21.15.43:0/973565949 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(39..39 src has 1..39) v4 ==== 6382+0+0 (secure 0 0 0) 0x7f9aa8088ab0 con 0x7f9ab80ccf20 2023-08-09T13:36:15.180 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:15.179+0000 7f9ab77fe640 1 --2- 172.21.15.43:0/973565949 >> [v2:172.21.15.203:6832/830097474,v1:172.21.15.203:6833/830097474] conn(0x7f9aa406a050 0x7f9aa406c510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:36:15.181 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:15.180+0000 7f9ab77fe640 1 --2- 172.21.15.43:0/973565949 >> [v2:172.21.15.203:6832/830097474,v1:172.21.15.203:6833/830097474] conn(0x7f9aa406a050 0x7f9aa406c510 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7f9aac036000 tx=0x7f9aac03c040 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-09T13:36:15.184 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:15.183+0000 7f9ab4ff9640 1 -- 172.21.15.43:0/973565949 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178066 (secure 0 0 0) 0x7f9aa8057a50 con 0x7f9ab80ccf20 2023-08-09T13:36:15.414 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:15.412+0000 7f9abe996640 1 -- 172.21.15.43:0/973565949 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f9ab80b3fe0 con 0x7f9ab80ccf20 2023-08-09T13:36:15.415 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:15.414+0000 7f9ab4ff9640 1 -- 172.21.15.43:0/973565949 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 17 v17) v1 ==== 94+0+4878 (secure 0 0 0) 0x7f9aa80575e0 con 0x7f9ab80ccf20 2023-08-09T13:36:15.416 INFO:teuthology.orchestra.run.smithi043.stdout: 2023-08-09T13:36:15.416 INFO:teuthology.orchestra.run.smithi043.stdout:{"epoch":17,"default_fscid":2,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4919,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.203:6836/302214806","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6834","nonce":302214806},{"type":"v1","addr":"172.21.15.203:6836","nonce":302214806}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":12},{"gid":4922,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.203:6837/1677639100","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6835","nonce":1677639100},{"type":"v1","addr":"172.21.15.203:6837","nonce":1677639100}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":12},{"gid":5619,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.43:6834/2339480142","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6832","nonce":2339480142},{"type":"v1","addr":"172.21.15.43:6834","nonce":2339480142}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":12}],"filesystems":[{"mdsmap":{"epoch":17,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-08-09T13:36:11.336926+0000","modified":"2023-08-09T13:36:14.905554+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":5622},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5622":{"gid":5622,"name":"c","rank":0,"incarnation":13,"state":"up:active","state_seq":4,"addr":"172.21.15.43:6835/2019295969","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6833","nonce":2019295969},{"type":"v1","addr":"172.21.15.43:6835","nonce":2019295969}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[7],"metadata_pool":6,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":2}]} 2023-08-09T13:36:15.416 INFO:teuthology.orchestra.run.smithi043.stderr:dumped fsmap epoch 17 2023-08-09T13:36:15.418 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:15.416+0000 7f9abe996640 1 -- 172.21.15.43:0/973565949 >> [v2:172.21.15.203:6832/830097474,v1:172.21.15.203:6833/830097474] conn(0x7f9aa406a050 msgr2=0x7f9aa406c510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:36:15.418 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:15.416+0000 7f9abe996640 1 --2- 172.21.15.43:0/973565949 >> [v2:172.21.15.203:6832/830097474,v1:172.21.15.203:6833/830097474] conn(0x7f9aa406a050 0x7f9aa406c510 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7f9aac036000 tx=0x7f9aac03c040 comp rx=0 tx=0).stop 2023-08-09T13:36:15.418 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:15.416+0000 7f9abe996640 1 -- 172.21.15.43:0/973565949 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f9ab80ccf20 msgr2=0x7f9ab80c54c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:36:15.418 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:15.416+0000 7f9abe996640 1 --2- 172.21.15.43:0/973565949 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f9ab80ccf20 0x7f9ab80c54c0 secure :-1 s=READY pgs=568 cs=0 l=1 rev1=1 crypto rx=0x7f9aa800ebe0 tx=0x7f9aa800c520 comp rx=0 tx=0).stop 2023-08-09T13:36:15.419 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:15.417+0000 7f9abe996640 1 -- 172.21.15.43:0/973565949 shutdown_connections 2023-08-09T13:36:15.419 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:15.417+0000 7f9abe996640 1 --2- 172.21.15.43:0/973565949 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f9ab815c630 0x7f9ab8151240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:36:15.419 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:15.417+0000 7f9abe996640 1 --2- 172.21.15.43:0/973565949 >> [v2:172.21.15.203:6832/830097474,v1:172.21.15.203:6833/830097474] conn(0x7f9aa406a050 0x7f9aa406c510 unknown :-1 s=CLOSED pgs=275 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:36:15.419 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:15.417+0000 7f9abe996640 1 --2- 172.21.15.43:0/973565949 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f9ab80cda90 0x7f9ab80b3140 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:36:15.419 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:15.417+0000 7f9abe996640 1 --2- 172.21.15.43:0/973565949 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f9ab80ccf20 0x7f9ab80c54c0 unknown :-1 s=CLOSED pgs=568 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:36:15.419 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:15.417+0000 7f9abe996640 1 -- 172.21.15.43:0/973565949 >> 172.21.15.43:0/973565949 conn(0x7f9ab80ce0a0 msgr2=0x7f9ab815ba70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-09T13:36:15.420 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:15.417+0000 7f9abe996640 1 -- 172.21.15.43:0/973565949 shutdown_connections 2023-08-09T13:36:15.420 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:15.417+0000 7f9abe996640 1 -- 172.21.15.43:0/973565949 wait complete. 2023-08-09T13:36:15.438 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-08-09T13:36:15.667 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:15.665+0000 7f86ad129640 1 Processor -- start 2023-08-09T13:36:15.668 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:15.666+0000 7f86ad129640 1 -- start start 2023-08-09T13:36:15.669 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:15.667+0000 7f86ad129640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f86a815a6a0 0x7f86a80713b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:36:15.669 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:15.667+0000 7f86ad129640 1 -- --> v1:172.21.15.43:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f86a80cdc70 con 0x7f86a80cbd90 2023-08-09T13:36:15.669 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:15.667+0000 7f86ad129640 1 -- --> v1:172.21.15.203:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f86a8070910 con 0x7f86a80c96c0 2023-08-09T13:36:15.669 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:15.667+0000 7f86ad129640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f86a80c88c0 con 0x7f86a815a6a0 2023-08-09T13:36:15.671 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:15.668+0000 7f869ffff640 1 --1- >> v1:172.21.15.43:6789/0 conn(0x7f86a80cbd90 0x7f86a80c8fb0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.43:6789/0 says I am v1:172.21.15.43:35172/0 (socket says 172.21.15.43:35172) 2023-08-09T13:36:15.671 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:15.668+0000 7f869ffff640 1 -- 172.21.15.43:0/1952218627 learned_addr learned my addr 172.21.15.43:0/1952218627 (peer_addr_for_me v1:172.21.15.43:0/0) 2023-08-09T13:36:15.671 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:15.668+0000 7f86a7fff640 1 --2- 172.21.15.43:0/1952218627 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f86a815a6a0 0x7f86a80713b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:36:15.671 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:15.668+0000 7f86a77fe640 1 -- 172.21.15.43:0/1952218627 <== mon.0 v1:172.21.15.43:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 480028179 0 0) 0x7f86a80cdc70 con 0x7f86a80cbd90 2023-08-09T13:36:15.671 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:15.668+0000 7f86a77fe640 1 -- 172.21.15.43:0/1952218627 --> v1:172.21.15.43:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f8694003520 con 0x7f86a80cbd90 2023-08-09T13:36:15.671 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:15.669+0000 7f86a77fe640 1 -- 172.21.15.43:0/1952218627 <== mon.0 v1:172.21.15.43:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1803790310 0 0) 0x7f8694003520 con 0x7f86a80cbd90 2023-08-09T13:36:15.672 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:15.669+0000 7f86a77fe640 1 -- 172.21.15.43:0/1952218627 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f86a815a6a0 msgr2=0x7f86a80713b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-09T13:36:15.672 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:15.669+0000 7f86a77fe640 1 --2- 172.21.15.43:0/1952218627 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f86a815a6a0 0x7f86a80713b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:36:15.672 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:15.669+0000 7f86a77fe640 1 -- 172.21.15.43:0/1952218627 >> v1:172.21.15.203:6789/0 conn(0x7f86a80c96c0 legacy=0x7f86a815ad20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:36:15.672 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:15.669+0000 7f86a77fe640 1 -- 172.21.15.43:0/1952218627 --> v1:172.21.15.43:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f86a8070bb0 con 0x7f86a80cbd90 2023-08-09T13:36:15.672 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:15.669+0000 7f86a77fe640 1 -- 172.21.15.43:0/1952218627 <== mon.0 v1:172.21.15.43:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1285150740 0 0) 0x7f8690001670 con 0x7f86a80cbd90 2023-08-09T13:36:15.672 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:15.669+0000 7f86a77fe640 1 -- 172.21.15.43:0/1952218627 >> v1:172.21.15.43:6789/0 conn(0x7f86a80cbd90 legacy=0x7f86a80c8fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:36:15.672 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:15.669+0000 7f86a77fe640 1 --2- 172.21.15.43:0/1952218627 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f86a815a6a0 0x7f86940039a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:36:15.673 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:15.669+0000 7f86a77fe640 1 --2- 172.21.15.43:0/1952218627 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f8694003f20 0x7f86940047c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:36:15.673 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:15.669+0000 7f86a77fe640 1 --2- 172.21.15.43:0/1952218627 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f8694004d00 0x7f8694005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:36:15.673 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:15.669+0000 7f86a77fe640 1 -- 172.21.15.43:0/1952218627 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f86a80c88c0 con 0x7f8694003f20 2023-08-09T13:36:15.673 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:15.669+0000 7f86a77fe640 1 -- 172.21.15.43:0/1952218627 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f869400dd20 con 0x7f8694004d00 2023-08-09T13:36:15.673 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:15.669+0000 7f86a77fe640 1 -- 172.21.15.43:0/1952218627 --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f869400dee0 con 0x7f86a815a6a0 2023-08-09T13:36:15.673 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:15.669+0000 7f869ffff640 1 --2- 172.21.15.43:0/1952218627 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f8694003f20 0x7f86940047c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:36:15.674 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:15.670+0000 7f86a7fff640 1 --2- 172.21.15.43:0/1952218627 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f86a815a6a0 0x7f86940039a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:36:15.675 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:15.670+0000 7f86ac928640 1 --2- 172.21.15.43:0/1952218627 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f8694004d00 0x7f8694005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:36:15.675 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:15.670+0000 7f869ffff640 1 -- 172.21.15.43:0/1952218627 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f86a815a6a0 msgr2=0x7f86940039a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-09T13:36:15.675 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:15.670+0000 7f869ffff640 1 --2- 172.21.15.43:0/1952218627 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f86a815a6a0 0x7f86940039a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:36:15.675 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:15.670+0000 7f869ffff640 1 -- 172.21.15.43:0/1952218627 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f8694004d00 msgr2=0x7f8694005180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-09T13:36:15.675 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:15.670+0000 7f869ffff640 1 --2- 172.21.15.43:0/1952218627 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f8694004d00 0x7f8694005180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:36:15.675 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:15.670+0000 7f869ffff640 1 -- 172.21.15.43:0/1952218627 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8694003520 con 0x7f8694003f20 2023-08-09T13:36:15.675 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:15.671+0000 7f869ffff640 1 --2- 172.21.15.43:0/1952218627 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f8694003f20 0x7f86940047c0 secure :-1 s=READY pgs=570 cs=0 l=1 rev1=1 crypto rx=0x7f869000c240 tx=0x7f86940057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9665881aee018e9e server_cookie=0 in_seq=0 out_seq=0 2023-08-09T13:36:15.676 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:15.671+0000 7f86a77fe640 1 -- 172.21.15.43:0/1952218627 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8690005ad0 con 0x7f8694003f20 2023-08-09T13:36:15.676 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:15.671+0000 7f86a77fe640 1 -- 172.21.15.43:0/1952218627 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8690005c30 con 0x7f8694003f20 2023-08-09T13:36:15.676 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:15.671+0000 7f86a77fe640 1 -- 172.21.15.43:0/1952218627 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f869002c8b0 con 0x7f8694003f20 2023-08-09T13:36:15.676 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:15.672+0000 7f86ad129640 1 -- 172.21.15.43:0/1952218627 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f8694003f20 msgr2=0x7f86940047c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:36:15.676 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:15.672+0000 7f86ad129640 1 --2- 172.21.15.43:0/1952218627 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f8694003f20 0x7f86940047c0 secure :-1 s=READY pgs=570 cs=0 l=1 rev1=1 crypto rx=0x7f869000c240 tx=0x7f86940057f0 comp rx=0 tx=0).stop 2023-08-09T13:36:15.676 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:15.672+0000 7f86ac928640 1 -- 172.21.15.43:0/1952218627 reap_dead start 2023-08-09T13:36:15.677 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:15.672+0000 7f86ad129640 1 -- 172.21.15.43:0/1952218627 shutdown_connections 2023-08-09T13:36:15.677 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:15.672+0000 7f86ad129640 1 -- 172.21.15.43:0/1952218627 >> 172.21.15.43:0/1952218627 conn(0x7f86a80cde90 msgr2=0x7f86a80ce2a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-09T13:36:15.677 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:15.672+0000 7f86ad129640 1 -- 172.21.15.43:0/1952218627 shutdown_connections 2023-08-09T13:36:15.677 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:15.672+0000 7f86ad129640 1 -- 172.21.15.43:0/1952218627 wait complete. 2023-08-09T13:36:15.679 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:15.674+0000 7f86ad129640 1 Processor -- start 2023-08-09T13:36:15.679 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:15.674+0000 7f86ad129640 1 -- start start 2023-08-09T13:36:15.679 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:15.675+0000 7f86ad129640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f8694003f20 0x7f86a80c25b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:36:15.679 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:15.675+0000 7f86ad129640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f86a8153170 0x7f86a8155420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:36:15.679 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:15.675+0000 7f86ad129640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f86a8155960 0x7f86a8098a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:36:15.679 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:15.675+0000 7f86ad129640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f86a8154f90 con 0x7f86a8153170 2023-08-09T13:36:15.680 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:15.675+0000 7f86ad129640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f86a81540a0 con 0x7f86a8155960 2023-08-09T13:36:15.680 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:15.675+0000 7f86ad129640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f86a8154380 con 0x7f8694003f20 2023-08-09T13:36:15.680 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:15.676+0000 7f869ffff640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f86a8153170 0x7f86a8155420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:36:15.680 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:15.676+0000 7f869ffff640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f86a8153170 0x7f86a8155420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:58080/0 (socket says 172.21.15.43:58080) 2023-08-09T13:36:15.680 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:15.676+0000 7f869ffff640 1 -- 172.21.15.43:0/821765601 learned_addr learned my addr 172.21.15.43:0/821765601 (peer_addr_for_me v2:172.21.15.43:0/0) 2023-08-09T13:36:15.680 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:15.676+0000 7f86a7fff640 1 --2- 172.21.15.43:0/821765601 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f8694003f20 0x7f86a80c25b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:36:15.680 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:15.676+0000 7f86ac928640 1 --2- 172.21.15.43:0/821765601 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f86a8155960 0x7f86a8098a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:36:15.680 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:15.676+0000 7f869ffff640 1 -- 172.21.15.43:0/821765601 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f8694003f20 msgr2=0x7f86a80c25b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:36:15.681 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:15.676+0000 7f869ffff640 1 --2- 172.21.15.43:0/821765601 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f8694003f20 0x7f86a80c25b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:36:15.681 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:15.676+0000 7f869ffff640 1 -- 172.21.15.43:0/821765601 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f86a8155960 msgr2=0x7f86a8098a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:36:15.681 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:15.676+0000 7f869ffff640 1 --2- 172.21.15.43:0/821765601 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f86a8155960 0x7f86a8098a50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:36:15.681 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:15.676+0000 7f869ffff640 1 -- 172.21.15.43:0/821765601 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f86a8154660 con 0x7f86a8153170 2023-08-09T13:36:15.681 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:15.676+0000 7f869ffff640 1 --2- 172.21.15.43:0/821765601 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f86a8153170 0x7f86a8155420 secure :-1 s=READY pgs=571 cs=0 l=1 rev1=1 crypto rx=0x7f869002c450 tx=0x7f8690001a00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-09T13:36:15.681 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:15.677+0000 7f86a57fa640 1 -- 172.21.15.43:0/821765601 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f869002ccc0 con 0x7f86a8153170 2023-08-09T13:36:15.682 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:15.677+0000 7f86a57fa640 1 -- 172.21.15.43:0/821765601 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f869002ce20 con 0x7f86a8153170 2023-08-09T13:36:15.682 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:15.677+0000 7f86a57fa640 1 -- 172.21.15.43:0/821765601 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8690007270 con 0x7f86a8153170 2023-08-09T13:36:15.682 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:15.677+0000 7f86ad129640 1 -- 172.21.15.43:0/821765601 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f86a8154940 con 0x7f86a8153170 2023-08-09T13:36:15.682 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:15.677+0000 7f86ad129640 1 -- 172.21.15.43:0/821765601 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f86a80c2af0 con 0x7f86a8153170 2023-08-09T13:36:15.683 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:15.678+0000 7f86ad129640 1 -- 172.21.15.43:0/821765601 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f86a80c2cb0 con 0x7f86a8153170 2023-08-09T13:36:15.683 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:15.679+0000 7f86a57fa640 1 -- 172.21.15.43:0/821765601 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 5) v1 ==== 88341+0+0 (secure 0 0 0) 0x7f8690032070 con 0x7f86a8153170 2023-08-09T13:36:15.683 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:15.680+0000 7f86a57fa640 1 --2- 172.21.15.43:0/821765601 >> [v2:172.21.15.203:6832/830097474,v1:172.21.15.203:6833/830097474] conn(0x7f868806a050 0x7f868806c510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:36:15.683 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:15.681+0000 7f86a57fa640 1 -- 172.21.15.43:0/821765601 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(39..39 src has 1..39) v4 ==== 6382+0+0 (secure 0 0 0) 0x7f869003c040 con 0x7f86a8153170 2023-08-09T13:36:15.683 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:15.681+0000 7f86a7fff640 1 --2- 172.21.15.43:0/821765601 >> [v2:172.21.15.203:6832/830097474,v1:172.21.15.203:6833/830097474] conn(0x7f868806a050 0x7f868806c510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:36:15.684 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:15.682+0000 7f86a7fff640 1 --2- 172.21.15.43:0/821765601 >> [v2:172.21.15.203:6832/830097474,v1:172.21.15.203:6833/830097474] conn(0x7f868806a050 0x7f868806c510 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7f86a8153bc0 tx=0x7f8698008040 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-09T13:36:15.686 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:15.684+0000 7f86a57fa640 1 -- 172.21.15.43:0/821765601 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178066 (secure 0 0 0) 0x7f8690074df0 con 0x7f86a8153170 2023-08-09T13:36:15.879 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:15.877+0000 7f86ad129640 1 -- 172.21.15.43:0/821765601 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f86a80739f0 con 0x7f86a8153170 2023-08-09T13:36:15.882 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:15.881+0000 7f86a57fa640 1 -- 172.21.15.43:0/821765601 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v39) v1 ==== 74+0+18430 (secure 0 0 0) 0x7f8690078410 con 0x7f86a8153170 2023-08-09T13:36:15.883 INFO:teuthology.orchestra.run.smithi043.stdout: 2023-08-09T13:36:15.883 INFO:teuthology.orchestra.run.smithi043.stdout:{"epoch":39,"fsid":"daac87ee-1337-4250-a955-aea4e408a39b","created":"2023-08-09T13:33:41.701198+0000","modified":"2023-08-09T13:36:12.734434+0000","last_up_change":"2023-08-09T13:33:53.783453+0000","last_in_change":"2023-08-09T13:33:45.148904+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":7,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":"rbd","create_time":"2023-08-09T13:33:54.365137+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":".mgr","create_time":"2023-08-09T13:33:54.831803+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":6,"pool_name":"cephfs_metadata","create_time":"2023-08-09T13:36:08.969993+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"38","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":7,"pool_name":"cephfs_data","create_time":"2023-08-09T13:36:10.366495+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"39","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"cbd49ecd-e896-4fbd-864e-5c37a829b165","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":37,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6800","nonce":502667335},{"type":"v1","addr":"172.21.15.43:6801","nonce":502667335}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6802","nonce":502667335},{"type":"v1","addr":"172.21.15.43:6803","nonce":502667335}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6806","nonce":502667335},{"type":"v1","addr":"172.21.15.43:6807","nonce":502667335}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6804","nonce":502667335},{"type":"v1","addr":"172.21.15.43:6805","nonce":502667335}]},"public_addr":"172.21.15.43:6801/502667335","cluster_addr":"172.21.15.43:6803/502667335","heartbeat_back_addr":"172.21.15.43:6807/502667335","heartbeat_front_addr":"172.21.15.43:6805/502667335","state":["exists","up"]},{"osd":1,"uuid":"75c7e0fe-1ec7-4fe8-a2f2-3661a65b8500","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":37,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6808","nonce":3289134610},{"type":"v1","addr":"172.21.15.43:6809","nonce":3289134610}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6810","nonce":3289134610},{"type":"v1","addr":"172.21.15.43:6812","nonce":3289134610}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6821","nonce":3289134610},{"type":"v1","addr":"172.21.15.43:6824","nonce":3289134610}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6815","nonce":3289134610},{"type":"v1","addr":"172.21.15.43:6818","nonce":3289134610}]},"public_addr":"172.21.15.43:6809/3289134610","cluster_addr":"172.21.15.43:6812/3289134610","heartbeat_back_addr":"172.21.15.43:6824/3289134610","heartbeat_front_addr":"172.21.15.43:6818/3289134610","state":["exists","up"]},{"osd":2,"uuid":"bb5c41e9-7f5f-4125-9b01-3bb8bfc2ea83","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":37,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6811","nonce":4079709736},{"type":"v1","addr":"172.21.15.43:6814","nonce":4079709736}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6817","nonce":4079709736},{"type":"v1","addr":"172.21.15.43:6820","nonce":4079709736}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6829","nonce":4079709736},{"type":"v1","addr":"172.21.15.43:6831","nonce":4079709736}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6823","nonce":4079709736},{"type":"v1","addr":"172.21.15.43:6826","nonce":4079709736}]},"public_addr":"172.21.15.43:6814/4079709736","cluster_addr":"172.21.15.43:6820/4079709736","heartbeat_back_addr":"172.21.15.43:6831/4079709736","heartbeat_front_addr":"172.21.15.43:6826/4079709736","state":["exists","up"]},{"osd":3,"uuid":"bfe4c8b4-694a-465c-87e6-2b2aab852613","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":37,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6813","nonce":1709538759},{"type":"v1","addr":"172.21.15.43:6816","nonce":1709538759}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6819","nonce":1709538759},{"type":"v1","addr":"172.21.15.43:6822","nonce":1709538759}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6828","nonce":1709538759},{"type":"v1","addr":"172.21.15.43:6830","nonce":1709538759}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6825","nonce":1709538759},{"type":"v1","addr":"172.21.15.43:6827","nonce":1709538759}]},"public_addr":"172.21.15.43:6816/1709538759","cluster_addr":"172.21.15.43:6822/1709538759","heartbeat_back_addr":"172.21.15.43:6830/1709538759","heartbeat_front_addr":"172.21.15.43:6827/1709538759","state":["exists","up"]},{"osd":4,"uuid":"fa2e8a77-aba1-4756-853e-0d174a5bdeba","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":37,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6800","nonce":4196866481},{"type":"v1","addr":"172.21.15.203:6801","nonce":4196866481}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6802","nonce":4196866481},{"type":"v1","addr":"172.21.15.203:6803","nonce":4196866481}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6806","nonce":4196866481},{"type":"v1","addr":"172.21.15.203:6807","nonce":4196866481}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6804","nonce":4196866481},{"type":"v1","addr":"172.21.15.203:6805","nonce":4196866481}]},"public_addr":"172.21.15.203:6801/4196866481","cluster_addr":"172.21.15.203:6803/4196866481","heartbeat_back_addr":"172.21.15.203:6807/4196866481","heartbeat_front_addr":"172.21.15.203:6805/4196866481","state":["exists","up"]},{"osd":5,"uuid":"81811e4b-4d24-4f51-a82c-f1d700a2a00d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":37,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6816","nonce":3059290081},{"type":"v1","addr":"172.21.15.203:6817","nonce":3059290081}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6818","nonce":3059290081},{"type":"v1","addr":"172.21.15.203:6819","nonce":3059290081}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6822","nonce":3059290081},{"type":"v1","addr":"172.21.15.203:6823","nonce":3059290081}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6820","nonce":3059290081},{"type":"v1","addr":"172.21.15.203:6821","nonce":3059290081}]},"public_addr":"172.21.15.203:6817/3059290081","cluster_addr":"172.21.15.203:6819/3059290081","heartbeat_back_addr":"172.21.15.203:6823/3059290081","heartbeat_front_addr":"172.21.15.203:6821/3059290081","state":["exists","up"]},{"osd":6,"uuid":"307bac4d-6a5b-443d-af75-56d695d7674f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":37,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6824","nonce":512155569},{"type":"v1","addr":"172.21.15.203:6825","nonce":512155569}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6826","nonce":512155569},{"type":"v1","addr":"172.21.15.203:6827","nonce":512155569}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6830","nonce":512155569},{"type":"v1","addr":"172.21.15.203:6831","nonce":512155569}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6828","nonce":512155569},{"type":"v1","addr":"172.21.15.203:6829","nonce":512155569}]},"public_addr":"172.21.15.203:6825/512155569","cluster_addr":"172.21.15.203:6827/512155569","heartbeat_back_addr":"172.21.15.203:6831/512155569","heartbeat_front_addr":"172.21.15.203:6829/512155569","state":["exists","up"]},{"osd":7,"uuid":"a95dc7ca-9f1b-4295-af70-68a22a2d7d5e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":37,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6808","nonce":3331450438},{"type":"v1","addr":"172.21.15.203:6809","nonce":3331450438}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6810","nonce":3331450438},{"type":"v1","addr":"172.21.15.203:6811","nonce":3331450438}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6814","nonce":3331450438},{"type":"v1","addr":"172.21.15.203:6815","nonce":3331450438}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6812","nonce":3331450438},{"type":"v1","addr":"172.21.15.203:6813","nonce":3331450438}]},"public_addr":"172.21.15.203:6809/3331450438","cluster_addr":"172.21.15.203:6811/3331450438","heartbeat_back_addr":"172.21.15.203:6815/3331450438","heartbeat_front_addr":"172.21.15.203:6813/3331450438","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-09T13:33:50.547652+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-09T13:33:50.589469+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-09T13:33:50.561724+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-09T13:33:50.599650+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-09T13:33:50.411855+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-09T13:33:50.219681+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-09T13:33:50.504664+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-09T13:33:50.441248+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-08-09T13:36:15.885 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:15.883+0000 7f86ad129640 1 -- 172.21.15.43:0/821765601 >> [v2:172.21.15.203:6832/830097474,v1:172.21.15.203:6833/830097474] conn(0x7f868806a050 msgr2=0x7f868806c510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:36:15.885 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:15.883+0000 7f86ad129640 1 --2- 172.21.15.43:0/821765601 >> [v2:172.21.15.203:6832/830097474,v1:172.21.15.203:6833/830097474] conn(0x7f868806a050 0x7f868806c510 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7f86a8153bc0 tx=0x7f8698008040 comp rx=0 tx=0).stop 2023-08-09T13:36:15.885 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:15.883+0000 7f86ad129640 1 -- 172.21.15.43:0/821765601 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f86a8153170 msgr2=0x7f86a8155420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:36:15.885 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:15.883+0000 7f86ad129640 1 --2- 172.21.15.43:0/821765601 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f86a8153170 0x7f86a8155420 secure :-1 s=READY pgs=571 cs=0 l=1 rev1=1 crypto rx=0x7f869002c450 tx=0x7f8690001a00 comp rx=0 tx=0).stop 2023-08-09T13:36:15.885 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:15.883+0000 7f86ad129640 1 -- 172.21.15.43:0/821765601 shutdown_connections 2023-08-09T13:36:15.885 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:15.883+0000 7f86ad129640 1 --2- 172.21.15.43:0/821765601 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f86a8155960 0x7f86a8098a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:36:15.886 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:15.883+0000 7f86ad129640 1 --2- 172.21.15.43:0/821765601 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f86a8153170 0x7f86a8155420 unknown :-1 s=CLOSED pgs=571 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:36:15.886 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:15.883+0000 7f86ad129640 1 --2- 172.21.15.43:0/821765601 >> [v2:172.21.15.203:6832/830097474,v1:172.21.15.203:6833/830097474] conn(0x7f868806a050 0x7f868806c510 unknown :-1 s=CLOSED pgs=276 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:36:15.886 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:15.883+0000 7f86ad129640 1 --2- 172.21.15.43:0/821765601 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f8694003f20 0x7f86a80c25b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:36:15.886 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:15.883+0000 7f86ad129640 1 -- 172.21.15.43:0/821765601 >> 172.21.15.43:0/821765601 conn(0x7f86a80cde90 msgr2=0x7f86a815ab10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-09T13:36:15.886 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:15.883+0000 7f86ad129640 1 -- 172.21.15.43:0/821765601 shutdown_connections 2023-08-09T13:36:15.886 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:15.884+0000 7f86ad129640 1 -- 172.21.15.43:0/821765601 wait complete. 2023-08-09T13:36:15.905 INFO:tasks.cephfs.filesystem.ceph_manager:waiting for clean 2023-08-09T13:36:15.905 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-08-09T13:36:16.133 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:16.131+0000 7f79b078b640 1 Processor -- start 2023-08-09T13:36:16.133 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:16.131+0000 7f79b078b640 1 -- start start 2023-08-09T13:36:16.134 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:16.133+0000 7f79b078b640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f79a8143c20 0x7f79a806f540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:36:16.135 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:16.133+0000 7f79b078b640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f79a80c86f0 0x7f79a80c8b50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:36:16.135 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:16.133+0000 7f79b078b640 1 -- --> v1:172.21.15.43:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f79a80c9090 con 0x7f79a806c8d0 2023-08-09T13:36:16.135 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:16.133+0000 7f79b078b640 1 -- --> v2:172.21.15.43:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f79a8144000 con 0x7f79a8143c20 2023-08-09T13:36:16.135 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:16.133+0000 7f79b078b640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f79a806c440 con 0x7f79a80c86f0 2023-08-09T13:36:16.136 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:16.133+0000 7f79ae527640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f79a8143c20 0x7f79a806f540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:36:16.136 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:16.133+0000 7f79aed28640 1 --1- >> v1:172.21.15.43:6789/0 conn(0x7f79a806c8d0 0x7f79a806cd60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.43:6789/0 says I am v1:172.21.15.43:35184/0 (socket says 172.21.15.43:35184) 2023-08-09T13:36:16.136 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:16.133+0000 7f79aed28640 1 -- 172.21.15.43:0/976202621 learned_addr learned my addr 172.21.15.43:0/976202621 (peer_addr_for_me v1:172.21.15.43:0/0) 2023-08-09T13:36:16.136 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:16.134+0000 7f79add26640 1 --2- 172.21.15.43:0/976202621 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f79a80c86f0 0x7f79a80c8b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:36:16.137 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:16.134+0000 7f79ae527640 1 -- 172.21.15.43:0/976202621 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f79a80c86f0 msgr2=0x7f79a80c8b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-09T13:36:16.137 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:16.134+0000 7f79ae527640 1 --2- 172.21.15.43:0/976202621 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f79a80c86f0 0x7f79a80c8b50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:36:16.137 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:16.134+0000 7f79ae527640 1 -- 172.21.15.43:0/976202621 >> v1:172.21.15.43:6789/0 conn(0x7f79a806c8d0 legacy=0x7f79a806cd60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:36:16.137 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:16.134+0000 7f79ae527640 1 -- 172.21.15.43:0/976202621 --> v2:172.21.15.43:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f79a806b830 con 0x7f79a8143c20 2023-08-09T13:36:16.137 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:16.134+0000 7f79ad525640 1 -- 172.21.15.43:0/976202621 <== mon.0 v1:172.21.15.43:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 586342787 0 0) 0x7f79a80c9090 con 0x7f79a806c8d0 2023-08-09T13:36:16.138 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:16.135+0000 7f79ae527640 1 --2- 172.21.15.43:0/976202621 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f79a8143c20 0x7f79a806f540 secure :-1 s=READY pgs=573 cs=0 l=1 rev1=1 crypto rx=0x7f79a40097a0 tx=0x7f79a402b840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d4e96d7c7e83c65f server_cookie=0 in_seq=0 out_seq=0 2023-08-09T13:36:16.138 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:16.135+0000 7f79ad525640 1 -- 172.21.15.43:0/976202621 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f79a402e070 con 0x7f79a8143c20 2023-08-09T13:36:16.138 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:16.135+0000 7f79ad525640 1 -- 172.21.15.43:0/976202621 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f79a40315a0 con 0x7f79a8143c20 2023-08-09T13:36:16.138 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:16.135+0000 7f79ad525640 1 -- 172.21.15.43:0/976202621 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f79a4031930 con 0x7f79a8143c20 2023-08-09T13:36:16.139 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:16.136+0000 7f79b078b640 1 -- 172.21.15.43:0/976202621 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f79a8143c20 msgr2=0x7f79a806f540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:36:16.139 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:16.136+0000 7f79b078b640 1 --2- 172.21.15.43:0/976202621 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f79a8143c20 0x7f79a806f540 secure :-1 s=READY pgs=573 cs=0 l=1 rev1=1 crypto rx=0x7f79a40097a0 tx=0x7f79a402b840 comp rx=0 tx=0).stop 2023-08-09T13:36:16.139 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:16.136+0000 7f79b078b640 1 -- 172.21.15.43:0/976202621 shutdown_connections 2023-08-09T13:36:16.139 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:16.136+0000 7f79b078b640 1 --2- 172.21.15.43:0/976202621 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f79a80c86f0 0x7f79a80c8b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:36:16.139 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:16.136+0000 7f79b078b640 1 --2- 172.21.15.43:0/976202621 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f79a8143c20 0x7f79a806f540 unknown :-1 s=CLOSED pgs=573 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:36:16.139 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:16.136+0000 7f79b078b640 1 -- 172.21.15.43:0/976202621 >> 172.21.15.43:0/976202621 conn(0x7f79a80ce0a0 msgr2=0x7f79a80ce480 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-09T13:36:16.139 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:16.136+0000 7f79b078b640 1 -- 172.21.15.43:0/976202621 shutdown_connections 2023-08-09T13:36:16.140 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:16.136+0000 7f79b078b640 1 -- 172.21.15.43:0/976202621 wait complete. 2023-08-09T13:36:16.140 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:16.138+0000 7f79b078b640 1 Processor -- start 2023-08-09T13:36:16.140 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:16.138+0000 7f79b078b640 1 -- start start 2023-08-09T13:36:16.141 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:16.139+0000 7f79b078b640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f79a806c8d0 0x7f79a815c5d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:36:16.142 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:16.139+0000 7f79b078b640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f79a80c86f0 0x7f79a815cb10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:36:16.142 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:16.139+0000 7f79b078b640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f79a8143c20 0x7f79a815d050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:36:16.142 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:16.139+0000 7f79b078b640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f79a815a9d0 con 0x7f79a80c86f0 2023-08-09T13:36:16.142 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:16.139+0000 7f79b078b640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f79a80c7900 con 0x7f79a8143c20 2023-08-09T13:36:16.142 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:16.139+0000 7f79b078b640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f79a815ac00 con 0x7f79a806c8d0 2023-08-09T13:36:16.142 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:16.139+0000 7f79add26640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f79a80c86f0 0x7f79a815cb10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:36:16.142 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:16.139+0000 7f79add26640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f79a80c86f0 0x7f79a815cb10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:58092/0 (socket says 172.21.15.43:58092) 2023-08-09T13:36:16.142 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:16.139+0000 7f79add26640 1 -- 172.21.15.43:0/885381074 learned_addr learned my addr 172.21.15.43:0/885381074 (peer_addr_for_me v2:172.21.15.43:0/0) 2023-08-09T13:36:16.143 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:16.140+0000 7f79ae527640 1 --2- 172.21.15.43:0/885381074 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f79a806c8d0 0x7f79a815c5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:36:16.143 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:16.140+0000 7f79add26640 1 -- 172.21.15.43:0/885381074 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f79a806c8d0 msgr2=0x7f79a815c5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:36:16.143 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:16.140+0000 7f79add26640 1 --2- 172.21.15.43:0/885381074 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f79a806c8d0 0x7f79a815c5d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:36:16.143 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:16.140+0000 7f79add26640 1 -- 172.21.15.43:0/885381074 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f79a8143c20 msgr2=0x7f79a815d050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:36:16.143 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:16.140+0000 7f79add26640 1 --2- 172.21.15.43:0/885381074 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f79a8143c20 0x7f79a815d050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:36:16.143 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:16.140+0000 7f79add26640 1 -- 172.21.15.43:0/885381074 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f79a4009400 con 0x7f79a80c86f0 2023-08-09T13:36:16.143 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:16.140+0000 7f79ae527640 1 --2- 172.21.15.43:0/885381074 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f79a806c8d0 0x7f79a815c5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-08-09T13:36:16.144 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:16.140+0000 7f79add26640 1 --2- 172.21.15.43:0/885381074 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f79a80c86f0 0x7f79a815cb10 secure :-1 s=READY pgs=574 cs=0 l=1 rev1=1 crypto rx=0x7f799800d800 tx=0x7f799800dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-09T13:36:16.144 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:16.140+0000 7f799effd640 1 -- 172.21.15.43:0/885381074 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f799800b6c0 con 0x7f79a80c86f0 2023-08-09T13:36:16.145 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:16.140+0000 7f799effd640 1 -- 172.21.15.43:0/885381074 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f799800b820 con 0x7f79a80c86f0 2023-08-09T13:36:16.145 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:16.140+0000 7f799effd640 1 -- 172.21.15.43:0/885381074 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f799800bbb0 con 0x7f79a80c86f0 2023-08-09T13:36:16.145 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:16.141+0000 7f79b078b640 1 -- 172.21.15.43:0/885381074 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f79a80c7590 con 0x7f79a80c86f0 2023-08-09T13:36:16.145 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:16.141+0000 7f79b078b640 1 -- 172.21.15.43:0/885381074 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f79a80c71d0 con 0x7f79a80c86f0 2023-08-09T13:36:16.145 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:16.142+0000 7f79b078b640 1 -- 172.21.15.43:0/885381074 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f79a80c7ab0 con 0x7f79a80c86f0 2023-08-09T13:36:16.145 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:16.143+0000 7f799effd640 1 -- 172.21.15.43:0/885381074 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 5) v1 ==== 88341+0+0 (secure 0 0 0) 0x7f799800bd10 con 0x7f79a80c86f0 2023-08-09T13:36:16.146 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:16.144+0000 7f799effd640 1 --2- 172.21.15.43:0/885381074 >> [v2:172.21.15.203:6832/830097474,v1:172.21.15.203:6833/830097474] conn(0x7f798c06a050 0x7f798c06c510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:36:16.146 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:16.144+0000 7f799effd640 1 -- 172.21.15.43:0/885381074 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(39..39 src has 1..39) v4 ==== 6382+0+0 (secure 0 0 0) 0x7f7998088cd0 con 0x7f79a80c86f0 2023-08-09T13:36:16.146 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:16.145+0000 7f79ae527640 1 --2- 172.21.15.43:0/885381074 >> [v2:172.21.15.203:6832/830097474,v1:172.21.15.203:6833/830097474] conn(0x7f798c06a050 0x7f798c06c510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:36:16.147 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:16.146+0000 7f79ae527640 1 --2- 172.21.15.43:0/885381074 >> [v2:172.21.15.203:6832/830097474,v1:172.21.15.203:6833/830097474] conn(0x7f798c06a050 0x7f798c06c510 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7f79a4036000 tx=0x7f79a403d040 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-09T13:36:16.150 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:16.149+0000 7f799effd640 1 -- 172.21.15.43:0/885381074 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178066 (secure 0 0 0) 0x7f79a80c7ab0 con 0x7f79a80c86f0 2023-08-09T13:36:16.341 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:16.340+0000 7f79b078b640 1 -- 172.21.15.43:0/885381074 --> [v2:172.21.15.203:6832/830097474,v1:172.21.15.203:6833/830097474] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f79a806bdf0 con 0x7f798c06a050 2023-08-09T13:36:16.371 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:16.369+0000 7f799effd640 1 -- 172.21.15.43:0/885381074 <== mgr.4114 v2:172.21.15.203:6832/830097474 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+309248 (secure 0 0 0) 0x7f79a4031ae0 con 0x7f798c06a050 2023-08-09T13:36:16.372 INFO:teuthology.orchestra.run.smithi043.stdout: 2023-08-09T13:36:16.376 INFO:teuthology.orchestra.run.smithi043.stderr:dumped all 2023-08-09T13:36:16.378 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:16.377+0000 7f79b078b640 1 -- 172.21.15.43:0/885381074 >> [v2:172.21.15.203:6832/830097474,v1:172.21.15.203:6833/830097474] conn(0x7f798c06a050 msgr2=0x7f798c06c510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:36:16.378 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:16.377+0000 7f79b078b640 1 --2- 172.21.15.43:0/885381074 >> [v2:172.21.15.203:6832/830097474,v1:172.21.15.203:6833/830097474] conn(0x7f798c06a050 0x7f798c06c510 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7f79a4036000 tx=0x7f79a403d040 comp rx=0 tx=0).stop 2023-08-09T13:36:16.378 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:16.377+0000 7f79b078b640 1 -- 172.21.15.43:0/885381074 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f79a80c86f0 msgr2=0x7f79a815cb10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:36:16.379 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:16.377+0000 7f79b078b640 1 --2- 172.21.15.43:0/885381074 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f79a80c86f0 0x7f79a815cb10 secure :-1 s=READY pgs=574 cs=0 l=1 rev1=1 crypto rx=0x7f799800d800 tx=0x7f799800dcc0 comp rx=0 tx=0).stop 2023-08-09T13:36:16.379 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:16.377+0000 7f79b078b640 1 -- 172.21.15.43:0/885381074 shutdown_connections 2023-08-09T13:36:16.379 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:16.377+0000 7f79b078b640 1 --2- 172.21.15.43:0/885381074 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f79a8143c20 0x7f79a815d050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:36:16.379 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:16.377+0000 7f79b078b640 1 --2- 172.21.15.43:0/885381074 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f79a80c86f0 0x7f79a815cb10 unknown :-1 s=CLOSED pgs=574 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:36:16.380 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:16.377+0000 7f79b078b640 1 --2- 172.21.15.43:0/885381074 >> [v2:172.21.15.203:6832/830097474,v1:172.21.15.203:6833/830097474] conn(0x7f798c06a050 0x7f798c06c510 unknown :-1 s=CLOSED pgs=277 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:36:16.380 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:16.377+0000 7f79b078b640 1 --2- 172.21.15.43:0/885381074 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f79a806c8d0 0x7f79a815c5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:36:16.380 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:16.377+0000 7f79b078b640 1 -- 172.21.15.43:0/885381074 >> 172.21.15.43:0/885381074 conn(0x7f79a80ce0a0 msgr2=0x7f79a8070050 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-09T13:36:16.380 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:16.377+0000 7f79b078b640 1 -- 172.21.15.43:0/885381074 shutdown_connections 2023-08-09T13:36:16.380 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:16.377+0000 7f79b078b640 1 -- 172.21.15.43:0/885381074 wait complete. 2023-08-09T13:36:16.401 INFO:teuthology.orchestra.run.smithi043.stdout:{"pg_ready":true,"pg_map":{"version":103,"stamp":"2023-08-09T13:36:14.813906+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":23,"num_object_clones":0,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":203,"num_write_kb":2430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":155,"ondisk_log_size":155,"up":210,"acting":210,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":192,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":241392,"kb_used_data":4736,"kb_used_omap":13,"kb_used_meta":236530,"kb_avail":754733328,"statfs":{"total":773094113280,"available":772846927872,"internally_reserved":0,"allocated":4849664,"data_stored":2811674,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":14248,"internal_metadata":242206808},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":36,"apply_latency_ms":36,"commit_latency_ns":36000000,"apply_latency_ns":36000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.984386"},"pg_stats":[{"pgid":"7.3f","version":"0'0","reported_seq":11,"reported_epoch":39,"state":"active+clean","last_fresh":"2023-08-09T13:36:13.181426+0000","last_change":"2023-08-09T13:36:11.765333+0000","last_active":"2023-08-09T13:36:13.181426+0000","last_peered":"2023-08-09T13:36:13.181426+0000","last_clean":"2023-08-09T13:36:13.181426+0000","last_became_active":"2023-08-09T13:36:11.764961+0000","last_became_peered":"2023-08-09T13:36:11.764961+0000","last_unstale":"2023-08-09T13:36:13.181426+0000","last_undegraded":"2023-08-09T13:36:13.181426+0000","last_fullsized":"2023-08-09T13:36:13.181426+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:36:10.826860+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:36:10.826860+0000","last_clean_scrub_stamp":"2023-08-09T13:36:10.826860+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T21:17:28.886040+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.3e","version":"0'0","reported_seq":11,"reported_epoch":39,"state":"active+clean","last_fresh":"2023-08-09T13:36:13.180692+0000","last_change":"2023-08-09T13:36:11.769972+0000","last_active":"2023-08-09T13:36:13.180692+0000","last_peered":"2023-08-09T13:36:13.180692+0000","last_clean":"2023-08-09T13:36:13.180692+0000","last_became_active":"2023-08-09T13:36:11.769710+0000","last_became_peered":"2023-08-09T13:36:11.769710+0000","last_unstale":"2023-08-09T13:36:13.180692+0000","last_undegraded":"2023-08-09T13:36:13.180692+0000","last_fullsized":"2023-08-09T13:36:13.180692+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:36:10.826860+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:36:10.826860+0000","last_clean_scrub_stamp":"2023-08-09T13:36:10.826860+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T17:14:05.980609+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.3d","version":"0'0","reported_seq":11,"reported_epoch":39,"state":"active+clean","last_fresh":"2023-08-09T13:36:12.745747+0000","last_change":"2023-08-09T13:36:11.771122+0000","last_active":"2023-08-09T13:36:12.745747+0000","last_peered":"2023-08-09T13:36:12.745747+0000","last_clean":"2023-08-09T13:36:12.745747+0000","last_became_active":"2023-08-09T13:36:11.770819+0000","last_became_peered":"2023-08-09T13:36:11.770819+0000","last_unstale":"2023-08-09T13:36:12.745747+0000","last_undegraded":"2023-08-09T13:36:12.745747+0000","last_fullsized":"2023-08-09T13:36:12.745747+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:36:10.826860+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:36:10.826860+0000","last_clean_scrub_stamp":"2023-08-09T13:36:10.826860+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T20:43:03.944312+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.3c","version":"0'0","reported_seq":11,"reported_epoch":39,"state":"active+clean","last_fresh":"2023-08-09T13:36:13.180214+0000","last_change":"2023-08-09T13:36:11.778822+0000","last_active":"2023-08-09T13:36:13.180214+0000","last_peered":"2023-08-09T13:36:13.180214+0000","last_clean":"2023-08-09T13:36:13.180214+0000","last_became_active":"2023-08-09T13:36:11.778544+0000","last_became_peered":"2023-08-09T13:36:11.778544+0000","last_unstale":"2023-08-09T13:36:13.180214+0000","last_undegraded":"2023-08-09T13:36:13.180214+0000","last_fullsized":"2023-08-09T13:36:13.180214+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:36:10.826860+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:36:10.826860+0000","last_clean_scrub_stamp":"2023-08-09T13:36:10.826860+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-11T00:30:46.542672+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.3b","version":"0'0","reported_seq":11,"reported_epoch":39,"state":"active+clean","last_fresh":"2023-08-09T13:36:13.180730+0000","last_change":"2023-08-09T13:36:11.765844+0000","last_active":"2023-08-09T13:36:13.180730+0000","last_peered":"2023-08-09T13:36:13.180730+0000","last_clean":"2023-08-09T13:36:13.180730+0000","last_became_active":"2023-08-09T13:36:11.765565+0000","last_became_peered":"2023-08-09T13:36:11.765565+0000","last_unstale":"2023-08-09T13:36:13.180730+0000","last_undegraded":"2023-08-09T13:36:13.180730+0000","last_fullsized":"2023-08-09T13:36:13.180730+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:36:10.826860+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:36:10.826860+0000","last_clean_scrub_stamp":"2023-08-09T13:36:10.826860+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T17:14:43.547736+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.3a","version":"0'0","reported_seq":11,"reported_epoch":39,"state":"active+clean","last_fresh":"2023-08-09T13:36:13.182081+0000","last_change":"2023-08-09T13:36:11.768348+0000","last_active":"2023-08-09T13:36:13.182081+0000","last_peered":"2023-08-09T13:36:13.182081+0000","last_clean":"2023-08-09T13:36:13.182081+0000","last_became_active":"2023-08-09T13:36:11.768017+0000","last_became_peered":"2023-08-09T13:36:11.768017+0000","last_unstale":"2023-08-09T13:36:13.182081+0000","last_undegraded":"2023-08-09T13:36:13.182081+0000","last_fullsized":"2023-08-09T13:36:13.182081+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:36:10.826860+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:36:10.826860+0000","last_clean_scrub_stamp":"2023-08-09T13:36:10.826860+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T17:09:17.685613+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.39","version":"0'0","reported_seq":11,"reported_epoch":39,"state":"active+clean","last_fresh":"2023-08-09T13:36:13.179893+0000","last_change":"2023-08-09T13:36:11.770182+0000","last_active":"2023-08-09T13:36:13.179893+0000","last_peered":"2023-08-09T13:36:13.179893+0000","last_clean":"2023-08-09T13:36:13.179893+0000","last_became_active":"2023-08-09T13:36:11.769887+0000","last_became_peered":"2023-08-09T13:36:11.769887+0000","last_unstale":"2023-08-09T13:36:13.179893+0000","last_undegraded":"2023-08-09T13:36:13.179893+0000","last_fullsized":"2023-08-09T13:36:13.179893+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:36:10.826860+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:36:10.826860+0000","last_clean_scrub_stamp":"2023-08-09T13:36:10.826860+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T19:22:11.248322+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.38","version":"0'0","reported_seq":11,"reported_epoch":39,"state":"active+clean","last_fresh":"2023-08-09T13:36:13.178307+0000","last_change":"2023-08-09T13:36:11.778128+0000","last_active":"2023-08-09T13:36:13.178307+0000","last_peered":"2023-08-09T13:36:13.178307+0000","last_clean":"2023-08-09T13:36:13.178307+0000","last_became_active":"2023-08-09T13:36:11.777617+0000","last_became_peered":"2023-08-09T13:36:11.777617+0000","last_unstale":"2023-08-09T13:36:13.178307+0000","last_undegraded":"2023-08-09T13:36:13.178307+0000","last_fullsized":"2023-08-09T13:36:13.178307+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:36:10.826860+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:36:10.826860+0000","last_clean_scrub_stamp":"2023-08-09T13:36:10.826860+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T21:41:28.535846+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.37","version":"0'0","reported_seq":11,"reported_epoch":39,"state":"active+clean","last_fresh":"2023-08-09T13:36:13.180968+0000","last_change":"2023-08-09T13:36:11.774608+0000","last_active":"2023-08-09T13:36:13.180968+0000","last_peered":"2023-08-09T13:36:13.180968+0000","last_clean":"2023-08-09T13:36:13.180968+0000","last_became_active":"2023-08-09T13:36:11.774333+0000","last_became_peered":"2023-08-09T13:36:11.774333+0000","last_unstale":"2023-08-09T13:36:13.180968+0000","last_undegraded":"2023-08-09T13:36:13.180968+0000","last_fullsized":"2023-08-09T13:36:13.180968+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:36:10.826860+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:36:10.826860+0000","last_clean_scrub_stamp":"2023-08-09T13:36:10.826860+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T16:31:40.424973+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.36","version":"0'0","reported_seq":11,"reported_epoch":39,"state":"active+clean","last_fresh":"2023-08-09T13:36:13.180568+0000","last_change":"2023-08-09T13:36:11.780966+0000","last_active":"2023-08-09T13:36:13.180568+0000","last_peered":"2023-08-09T13:36:13.180568+0000","last_clean":"2023-08-09T13:36:13.180568+0000","last_became_active":"2023-08-09T13:36:11.780610+0000","last_became_peered":"2023-08-09T13:36:11.780610+0000","last_unstale":"2023-08-09T13:36:13.180568+0000","last_undegraded":"2023-08-09T13:36:13.180568+0000","last_fullsized":"2023-08-09T13:36:13.180568+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:36:10.826860+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:36:10.826860+0000","last_clean_scrub_stamp":"2023-08-09T13:36:10.826860+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T16:27:53.685325+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.35","version":"0'0","reported_seq":11,"reported_epoch":39,"state":"active+clean","last_fresh":"2023-08-09T13:36:13.180041+0000","last_change":"2023-08-09T13:36:11.769206+0000","last_active":"2023-08-09T13:36:13.180041+0000","last_peered":"2023-08-09T13:36:13.180041+0000","last_clean":"2023-08-09T13:36:13.180041+0000","last_became_active":"2023-08-09T13:36:11.768753+0000","last_became_peered":"2023-08-09T13:36:11.768753+0000","last_unstale":"2023-08-09T13:36:13.180041+0000","last_undegraded":"2023-08-09T13:36:13.180041+0000","last_fullsized":"2023-08-09T13:36:13.180041+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:36:10.826860+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:36:10.826860+0000","last_clean_scrub_stamp":"2023-08-09T13:36:10.826860+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T20:21:26.027735+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.34","version":"0'0","reported_seq":11,"reported_epoch":39,"state":"active+clean","last_fresh":"2023-08-09T13:36:13.180449+0000","last_change":"2023-08-09T13:36:11.768191+0000","last_active":"2023-08-09T13:36:13.180449+0000","last_peered":"2023-08-09T13:36:13.180449+0000","last_clean":"2023-08-09T13:36:13.180449+0000","last_became_active":"2023-08-09T13:36:11.767877+0000","last_became_peered":"2023-08-09T13:36:11.767877+0000","last_unstale":"2023-08-09T13:36:13.180449+0000","last_undegraded":"2023-08-09T13:36:13.180449+0000","last_fullsized":"2023-08-09T13:36:13.180449+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:36:10.826860+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:36:10.826860+0000","last_clean_scrub_stamp":"2023-08-09T13:36:10.826860+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T18:21:44.653185+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.33","version":"0'0","reported_seq":11,"reported_epoch":39,"state":"active+clean","last_fresh":"2023-08-09T13:36:13.181117+0000","last_change":"2023-08-09T13:36:11.768006+0000","last_active":"2023-08-09T13:36:13.181117+0000","last_peered":"2023-08-09T13:36:13.181117+0000","last_clean":"2023-08-09T13:36:13.181117+0000","last_became_active":"2023-08-09T13:36:11.767631+0000","last_became_peered":"2023-08-09T13:36:11.767631+0000","last_unstale":"2023-08-09T13:36:13.181117+0000","last_undegraded":"2023-08-09T13:36:13.181117+0000","last_fullsized":"2023-08-09T13:36:13.181117+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:36:10.826860+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:36:10.826860+0000","last_clean_scrub_stamp":"2023-08-09T13:36:10.826860+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T18:12:36.838776+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.32","version":"0'0","reported_seq":11,"reported_epoch":39,"state":"active+clean","last_fresh":"2023-08-09T13:36:13.179228+0000","last_change":"2023-08-09T13:36:11.766755+0000","last_active":"2023-08-09T13:36:13.179228+0000","last_peered":"2023-08-09T13:36:13.179228+0000","last_clean":"2023-08-09T13:36:13.179228+0000","last_became_active":"2023-08-09T13:36:11.766478+0000","last_became_peered":"2023-08-09T13:36:11.766478+0000","last_unstale":"2023-08-09T13:36:13.179228+0000","last_undegraded":"2023-08-09T13:36:13.179228+0000","last_fullsized":"2023-08-09T13:36:13.179228+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:36:10.826860+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:36:10.826860+0000","last_clean_scrub_stamp":"2023-08-09T13:36:10.826860+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T13:52:40.766251+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.31","version":"0'0","reported_seq":11,"reported_epoch":39,"state":"active+clean","last_fresh":"2023-08-09T13:36:13.182174+0000","last_change":"2023-08-09T13:36:11.766564+0000","last_active":"2023-08-09T13:36:13.182174+0000","last_peered":"2023-08-09T13:36:13.182174+0000","last_clean":"2023-08-09T13:36:13.182174+0000","last_became_active":"2023-08-09T13:36:11.766273+0000","last_became_peered":"2023-08-09T13:36:11.766273+0000","last_unstale":"2023-08-09T13:36:13.182174+0000","last_undegraded":"2023-08-09T13:36:13.182174+0000","last_fullsized":"2023-08-09T13:36:13.182174+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:36:10.826860+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:36:10.826860+0000","last_clean_scrub_stamp":"2023-08-09T13:36:10.826860+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T15:46:07.852527+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.30","version":"0'0","reported_seq":11,"reported_epoch":39,"state":"active+clean","last_fresh":"2023-08-09T13:36:12.745527+0000","last_change":"2023-08-09T13:36:11.779982+0000","last_active":"2023-08-09T13:36:12.745527+0000","last_peered":"2023-08-09T13:36:12.745527+0000","last_clean":"2023-08-09T13:36:12.745527+0000","last_became_active":"2023-08-09T13:36:11.779622+0000","last_became_peered":"2023-08-09T13:36:11.779622+0000","last_unstale":"2023-08-09T13:36:12.745527+0000","last_undegraded":"2023-08-09T13:36:12.745527+0000","last_fullsized":"2023-08-09T13:36:12.745527+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:36:10.826860+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:36:10.826860+0000","last_clean_scrub_stamp":"2023-08-09T13:36:10.826860+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T20:01:21.335013+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.2f","version":"0'0","reported_seq":11,"reported_epoch":39,"state":"active+clean","last_fresh":"2023-08-09T13:36:12.746947+0000","last_change":"2023-08-09T13:36:11.764541+0000","last_active":"2023-08-09T13:36:12.746947+0000","last_peered":"2023-08-09T13:36:12.746947+0000","last_clean":"2023-08-09T13:36:12.746947+0000","last_became_active":"2023-08-09T13:36:11.764200+0000","last_became_peered":"2023-08-09T13:36:11.764200+0000","last_unstale":"2023-08-09T13:36:12.746947+0000","last_undegraded":"2023-08-09T13:36:12.746947+0000","last_fullsized":"2023-08-09T13:36:12.746947+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:36:10.826860+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:36:10.826860+0000","last_clean_scrub_stamp":"2023-08-09T13:36:10.826860+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T21:47:36.132042+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.2e","version":"0'0","reported_seq":11,"reported_epoch":39,"state":"active+clean","last_fresh":"2023-08-09T13:36:12.746801+0000","last_change":"2023-08-09T13:36:11.769432+0000","last_active":"2023-08-09T13:36:12.746801+0000","last_peered":"2023-08-09T13:36:12.746801+0000","last_clean":"2023-08-09T13:36:12.746801+0000","last_became_active":"2023-08-09T13:36:11.768929+0000","last_became_peered":"2023-08-09T13:36:11.768929+0000","last_unstale":"2023-08-09T13:36:12.746801+0000","last_undegraded":"2023-08-09T13:36:12.746801+0000","last_fullsized":"2023-08-09T13:36:12.746801+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:36:10.826860+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:36:10.826860+0000","last_clean_scrub_stamp":"2023-08-09T13:36:10.826860+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T22:26:28.862983+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.2d","version":"0'0","reported_seq":11,"reported_epoch":39,"state":"active+clean","last_fresh":"2023-08-09T13:36:12.748003+0000","last_change":"2023-08-09T13:36:11.781277+0000","last_active":"2023-08-09T13:36:12.748003+0000","last_peered":"2023-08-09T13:36:12.748003+0000","last_clean":"2023-08-09T13:36:12.748003+0000","last_became_active":"2023-08-09T13:36:11.780880+0000","last_became_peered":"2023-08-09T13:36:11.780880+0000","last_unstale":"2023-08-09T13:36:12.748003+0000","last_undegraded":"2023-08-09T13:36:12.748003+0000","last_fullsized":"2023-08-09T13:36:12.748003+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:36:10.826860+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:36:10.826860+0000","last_clean_scrub_stamp":"2023-08-09T13:36:10.826860+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T15:14:19.445235+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.2c","version":"0'0","reported_seq":11,"reported_epoch":39,"state":"active+clean","last_fresh":"2023-08-09T13:36:13.180147+0000","last_change":"2023-08-09T13:36:11.769233+0000","last_active":"2023-08-09T13:36:13.180147+0000","last_peered":"2023-08-09T13:36:13.180147+0000","last_clean":"2023-08-09T13:36:13.180147+0000","last_became_active":"2023-08-09T13:36:11.768953+0000","last_became_peered":"2023-08-09T13:36:11.768953+0000","last_unstale":"2023-08-09T13:36:13.180147+0000","last_undegraded":"2023-08-09T13:36:13.180147+0000","last_fullsized":"2023-08-09T13:36:13.180147+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:36:10.826860+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:36:10.826860+0000","last_clean_scrub_stamp":"2023-08-09T13:36:10.826860+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T21:29:25.842401+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.2b","version":"0'0","reported_seq":11,"reported_epoch":39,"state":"active+clean","last_fresh":"2023-08-09T13:36:13.182610+0000","last_change":"2023-08-09T13:36:11.776832+0000","last_active":"2023-08-09T13:36:13.182610+0000","last_peered":"2023-08-09T13:36:13.182610+0000","last_clean":"2023-08-09T13:36:13.182610+0000","last_became_active":"2023-08-09T13:36:11.776502+0000","last_became_peered":"2023-08-09T13:36:11.776502+0000","last_unstale":"2023-08-09T13:36:13.182610+0000","last_undegraded":"2023-08-09T13:36:13.182610+0000","last_fullsized":"2023-08-09T13:36:13.182610+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:36:10.826860+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:36:10.826860+0000","last_clean_scrub_stamp":"2023-08-09T13:36:10.826860+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T14:36:24.296741+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.2a","version":"0'0","reported_seq":11,"reported_epoch":39,"state":"active+clean","last_fresh":"2023-08-09T13:36:13.179357+0000","last_change":"2023-08-09T13:36:11.780161+0000","last_active":"2023-08-09T13:36:13.179357+0000","last_peered":"2023-08-09T13:36:13.179357+0000","last_clean":"2023-08-09T13:36:13.179357+0000","last_became_active":"2023-08-09T13:36:11.779797+0000","last_became_peered":"2023-08-09T13:36:11.779797+0000","last_unstale":"2023-08-09T13:36:13.179357+0000","last_undegraded":"2023-08-09T13:36:13.179357+0000","last_fullsized":"2023-08-09T13:36:13.179357+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:36:10.826860+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:36:10.826860+0000","last_clean_scrub_stamp":"2023-08-09T13:36:10.826860+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T23:26:23.964084+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.29","version":"0'0","reported_seq":11,"reported_epoch":39,"state":"active+clean","last_fresh":"2023-08-09T13:36:12.746280+0000","last_change":"2023-08-09T13:36:11.773991+0000","last_active":"2023-08-09T13:36:12.746280+0000","last_peered":"2023-08-09T13:36:12.746280+0000","last_clean":"2023-08-09T13:36:12.746280+0000","last_became_active":"2023-08-09T13:36:11.773668+0000","last_became_peered":"2023-08-09T13:36:11.773668+0000","last_unstale":"2023-08-09T13:36:12.746280+0000","last_undegraded":"2023-08-09T13:36:12.746280+0000","last_fullsized":"2023-08-09T13:36:12.746280+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:36:10.826860+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:36:10.826860+0000","last_clean_scrub_stamp":"2023-08-09T13:36:10.826860+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T17:26:42.727199+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.28","version":"0'0","reported_seq":11,"reported_epoch":39,"state":"active+clean","last_fresh":"2023-08-09T13:36:12.745684+0000","last_change":"2023-08-09T13:36:11.771543+0000","last_active":"2023-08-09T13:36:12.745684+0000","last_peered":"2023-08-09T13:36:12.745684+0000","last_clean":"2023-08-09T13:36:12.745684+0000","last_became_active":"2023-08-09T13:36:11.771246+0000","last_became_peered":"2023-08-09T13:36:11.771246+0000","last_unstale":"2023-08-09T13:36:12.745684+0000","last_undegraded":"2023-08-09T13:36:12.745684+0000","last_fullsized":"2023-08-09T13:36:12.745684+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:36:10.826860+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:36:10.826860+0000","last_clean_scrub_stamp":"2023-08-09T13:36:10.826860+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T18:13:08.572943+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.27","version":"0'0","reported_seq":11,"reported_epoch":39,"state":"active+clean","last_fresh":"2023-08-09T13:36:12.748356+0000","last_change":"2023-08-09T13:36:11.774594+0000","last_active":"2023-08-09T13:36:12.748356+0000","last_peered":"2023-08-09T13:36:12.748356+0000","last_clean":"2023-08-09T13:36:12.748356+0000","last_became_active":"2023-08-09T13:36:11.774122+0000","last_became_peered":"2023-08-09T13:36:11.774122+0000","last_unstale":"2023-08-09T13:36:12.748356+0000","last_undegraded":"2023-08-09T13:36:12.748356+0000","last_fullsized":"2023-08-09T13:36:12.748356+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:36:10.826860+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:36:10.826860+0000","last_clean_scrub_stamp":"2023-08-09T13:36:10.826860+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T23:55:48.467147+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.26","version":"0'0","reported_seq":11,"reported_epoch":39,"state":"active+clean","last_fresh":"2023-08-09T13:36:13.181406+0000","last_change":"2023-08-09T13:36:11.766030+0000","last_active":"2023-08-09T13:36:13.181406+0000","last_peered":"2023-08-09T13:36:13.181406+0000","last_clean":"2023-08-09T13:36:13.181406+0000","last_became_active":"2023-08-09T13:36:11.764234+0000","last_became_peered":"2023-08-09T13:36:11.764234+0000","last_unstale":"2023-08-09T13:36:13.181406+0000","last_undegraded":"2023-08-09T13:36:13.181406+0000","last_fullsized":"2023-08-09T13:36:13.181406+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:36:10.826860+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:36:10.826860+0000","last_clean_scrub_stamp":"2023-08-09T13:36:10.826860+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T13:56:38.251585+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.25","version":"0'0","reported_seq":11,"reported_epoch":39,"state":"active+clean","last_fresh":"2023-08-09T13:36:12.746240+0000","last_change":"2023-08-09T13:36:11.764663+0000","last_active":"2023-08-09T13:36:12.746240+0000","last_peered":"2023-08-09T13:36:12.746240+0000","last_clean":"2023-08-09T13:36:12.746240+0000","last_became_active":"2023-08-09T13:36:11.764280+0000","last_became_peered":"2023-08-09T13:36:11.764280+0000","last_unstale":"2023-08-09T13:36:12.746240+0000","last_undegraded":"2023-08-09T13:36:12.746240+0000","last_fullsized":"2023-08-09T13:36:12.746240+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:36:10.826860+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:36:10.826860+0000","last_clean_scrub_stamp":"2023-08-09T13:36:10.826860+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T21:26:15.393102+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.24","version":"0'0","reported_seq":11,"reported_epoch":39,"state":"active+clean","last_fresh":"2023-08-09T13:36:13.180557+0000","last_change":"2023-08-09T13:36:11.768283+0000","last_active":"2023-08-09T13:36:13.180557+0000","last_peered":"2023-08-09T13:36:13.180557+0000","last_clean":"2023-08-09T13:36:13.180557+0000","last_became_active":"2023-08-09T13:36:11.767964+0000","last_became_peered":"2023-08-09T13:36:11.767964+0000","last_unstale":"2023-08-09T13:36:13.180557+0000","last_undegraded":"2023-08-09T13:36:13.180557+0000","last_fullsized":"2023-08-09T13:36:13.180557+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:36:10.826860+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:36:10.826860+0000","last_clean_scrub_stamp":"2023-08-09T13:36:10.826860+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T21:41:28.535846+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.23","version":"0'0","reported_seq":11,"reported_epoch":39,"state":"active+clean","last_fresh":"2023-08-09T13:36:12.747274+0000","last_change":"2023-08-09T13:36:11.773274+0000","last_active":"2023-08-09T13:36:12.747274+0000","last_peered":"2023-08-09T13:36:12.747274+0000","last_clean":"2023-08-09T13:36:12.747274+0000","last_became_active":"2023-08-09T13:36:11.772906+0000","last_became_peered":"2023-08-09T13:36:11.772906+0000","last_unstale":"2023-08-09T13:36:12.747274+0000","last_undegraded":"2023-08-09T13:36:12.747274+0000","last_fullsized":"2023-08-09T13:36:12.747274+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:36:10.826860+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:36:10.826860+0000","last_clean_scrub_stamp":"2023-08-09T13:36:10.826860+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T23:28:53.092968+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.22","version":"0'0","reported_seq":11,"reported_epoch":39,"state":"active+clean","last_fresh":"2023-08-09T13:36:12.744611+0000","last_change":"2023-08-09T13:36:11.763747+0000","last_active":"2023-08-09T13:36:12.744611+0000","last_peered":"2023-08-09T13:36:12.744611+0000","last_clean":"2023-08-09T13:36:12.744611+0000","last_became_active":"2023-08-09T13:36:11.763448+0000","last_became_peered":"2023-08-09T13:36:11.763448+0000","last_unstale":"2023-08-09T13:36:12.744611+0000","last_undegraded":"2023-08-09T13:36:12.744611+0000","last_fullsized":"2023-08-09T13:36:12.744611+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:36:10.826860+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:36:10.826860+0000","last_clean_scrub_stamp":"2023-08-09T13:36:10.826860+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T23:59:15.283205+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.21","version":"0'0","reported_seq":11,"reported_epoch":39,"state":"active+clean","last_fresh":"2023-08-09T13:36:13.182242+0000","last_change":"2023-08-09T13:36:11.773054+0000","last_active":"2023-08-09T13:36:13.182242+0000","last_peered":"2023-08-09T13:36:13.182242+0000","last_clean":"2023-08-09T13:36:13.182242+0000","last_became_active":"2023-08-09T13:36:11.772568+0000","last_became_peered":"2023-08-09T13:36:11.772568+0000","last_unstale":"2023-08-09T13:36:13.182242+0000","last_undegraded":"2023-08-09T13:36:13.182242+0000","last_fullsized":"2023-08-09T13:36:13.182242+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:36:10.826860+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:36:10.826860+0000","last_clean_scrub_stamp":"2023-08-09T13:36:10.826860+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T19:30:04.078025+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.20","version":"0'0","reported_seq":11,"reported_epoch":39,"state":"active+clean","last_fresh":"2023-08-09T13:36:13.179580+0000","last_change":"2023-08-09T13:36:11.778008+0000","last_active":"2023-08-09T13:36:13.179580+0000","last_peered":"2023-08-09T13:36:13.179580+0000","last_clean":"2023-08-09T13:36:13.179580+0000","last_became_active":"2023-08-09T13:36:11.777724+0000","last_became_peered":"2023-08-09T13:36:11.777724+0000","last_unstale":"2023-08-09T13:36:13.179580+0000","last_undegraded":"2023-08-09T13:36:13.179580+0000","last_fullsized":"2023-08-09T13:36:13.179580+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:36:10.826860+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:36:10.826860+0000","last_clean_scrub_stamp":"2023-08-09T13:36:10.826860+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T20:02:36.013365+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.1e","version":"0'0","reported_seq":11,"reported_epoch":39,"state":"active+clean","last_fresh":"2023-08-09T13:36:12.743763+0000","last_change":"2023-08-09T13:36:11.767917+0000","last_active":"2023-08-09T13:36:12.743763+0000","last_peered":"2023-08-09T13:36:12.743763+0000","last_clean":"2023-08-09T13:36:12.743763+0000","last_became_active":"2023-08-09T13:36:11.767617+0000","last_became_peered":"2023-08-09T13:36:11.767617+0000","last_unstale":"2023-08-09T13:36:12.743763+0000","last_undegraded":"2023-08-09T13:36:12.743763+0000","last_fullsized":"2023-08-09T13:36:12.743763+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:36:10.826860+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:36:10.826860+0000","last_clean_scrub_stamp":"2023-08-09T13:36:10.826860+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T23:39:34.789487+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1f","version":"38'4","reported_seq":17,"reported_epoch":39,"state":"active+clean","last_fresh":"2023-08-09T13:36:13.183924+0000","last_change":"2023-08-09T13:36:10.877706+0000","last_active":"2023-08-09T13:36:13.183924+0000","last_peered":"2023-08-09T13:36:13.183924+0000","last_clean":"2023-08-09T13:36:13.183924+0000","last_became_active":"2023-08-09T13:36:10.877403+0000","last_became_peered":"2023-08-09T13:36:10.877403+0000","last_unstale":"2023-08-09T13:36:13.183924+0000","last_undegraded":"2023-08-09T13:36:13.183924+0000","last_fullsized":"2023-08-09T13:36:13.183924+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:36:09.813580+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:36:09.813580+0000","last_clean_scrub_stamp":"2023-08-09T13:36:09.813580+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T19:23:49.018232+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.1f","version":"0'0","reported_seq":11,"reported_epoch":39,"state":"active+clean","last_fresh":"2023-08-09T13:36:12.747755+0000","last_change":"2023-08-09T13:36:11.774093+0000","last_active":"2023-08-09T13:36:12.747755+0000","last_peered":"2023-08-09T13:36:12.747755+0000","last_clean":"2023-08-09T13:36:12.747755+0000","last_became_active":"2023-08-09T13:36:11.773206+0000","last_became_peered":"2023-08-09T13:36:11.773206+0000","last_unstale":"2023-08-09T13:36:12.747755+0000","last_undegraded":"2023-08-09T13:36:12.747755+0000","last_fullsized":"2023-08-09T13:36:12.747755+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:36:10.826860+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:36:10.826860+0000","last_clean_scrub_stamp":"2023-08-09T13:36:10.826860+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T16:13:39.119652+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.1e","version":"38'1","reported_seq":14,"reported_epoch":39,"state":"active+clean","last_fresh":"2023-08-09T13:36:12.747024+0000","last_change":"2023-08-09T13:36:10.878238+0000","last_active":"2023-08-09T13:36:12.747024+0000","last_peered":"2023-08-09T13:36:12.747024+0000","last_clean":"2023-08-09T13:36:12.747024+0000","last_became_active":"2023-08-09T13:36:10.877926+0000","last_became_peered":"2023-08-09T13:36:10.877926+0000","last_unstale":"2023-08-09T13:36:12.747024+0000","last_undegraded":"2023-08-09T13:36:12.747024+0000","last_fullsized":"2023-08-09T13:36:12.747024+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:36:09.813580+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:36:09.813580+0000","last_clean_scrub_stamp":"2023-08-09T13:36:09.813580+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T19:34:51.030091+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.1c","version":"0'0","reported_seq":11,"reported_epoch":39,"state":"active+clean","last_fresh":"2023-08-09T13:36:13.180813+0000","last_change":"2023-08-09T13:36:11.764825+0000","last_active":"2023-08-09T13:36:13.180813+0000","last_peered":"2023-08-09T13:36:13.180813+0000","last_clean":"2023-08-09T13:36:13.180813+0000","last_became_active":"2023-08-09T13:36:11.764483+0000","last_became_peered":"2023-08-09T13:36:11.764483+0000","last_unstale":"2023-08-09T13:36:13.180813+0000","last_undegraded":"2023-08-09T13:36:13.180813+0000","last_fullsized":"2023-08-09T13:36:13.180813+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:36:10.826860+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:36:10.826860+0000","last_clean_scrub_stamp":"2023-08-09T13:36:10.826860+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T18:10:39.629916+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.1d","version":"38'2","reported_seq":15,"reported_epoch":39,"state":"active+clean","last_fresh":"2023-08-09T13:36:12.743758+0000","last_change":"2023-08-09T13:36:10.866053+0000","last_active":"2023-08-09T13:36:12.743758+0000","last_peered":"2023-08-09T13:36:12.743758+0000","last_clean":"2023-08-09T13:36:12.743758+0000","last_became_active":"2023-08-09T13:36:10.865665+0000","last_became_peered":"2023-08-09T13:36:10.865665+0000","last_unstale":"2023-08-09T13:36:12.743758+0000","last_undegraded":"2023-08-09T13:36:12.743758+0000","last_fullsized":"2023-08-09T13:36:12.743758+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:36:09.813580+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:36:09.813580+0000","last_clean_scrub_stamp":"2023-08-09T13:36:09.813580+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T16:18:22.253312+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.1d","version":"0'0","reported_seq":11,"reported_epoch":39,"state":"active+clean","last_fresh":"2023-08-09T13:36:12.743357+0000","last_change":"2023-08-09T13:36:11.778816+0000","last_active":"2023-08-09T13:36:12.743357+0000","last_peered":"2023-08-09T13:36:12.743357+0000","last_clean":"2023-08-09T13:36:12.743357+0000","last_became_active":"2023-08-09T13:36:11.778521+0000","last_became_peered":"2023-08-09T13:36:11.778521+0000","last_unstale":"2023-08-09T13:36:12.743357+0000","last_undegraded":"2023-08-09T13:36:12.743357+0000","last_fullsized":"2023-08-09T13:36:12.743357+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:36:10.826860+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:36:10.826860+0000","last_clean_scrub_stamp":"2023-08-09T13:36:10.826860+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T20:03:23.736139+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1c","version":"38'2","reported_seq":15,"reported_epoch":39,"state":"active+clean","last_fresh":"2023-08-09T13:36:13.181614+0000","last_change":"2023-08-09T13:36:10.876828+0000","last_active":"2023-08-09T13:36:13.181614+0000","last_peered":"2023-08-09T13:36:13.181614+0000","last_clean":"2023-08-09T13:36:13.181614+0000","last_became_active":"2023-08-09T13:36:10.876565+0000","last_became_peered":"2023-08-09T13:36:10.876565+0000","last_unstale":"2023-08-09T13:36:13.181614+0000","last_undegraded":"2023-08-09T13:36:13.181614+0000","last_fullsized":"2023-08-09T13:36:13.181614+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:36:09.813580+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:36:09.813580+0000","last_clean_scrub_stamp":"2023-08-09T13:36:09.813580+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T18:12:04.598648+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.1a","version":"0'0","reported_seq":11,"reported_epoch":39,"state":"active+clean","last_fresh":"2023-08-09T13:36:12.744691+0000","last_change":"2023-08-09T13:36:11.778261+0000","last_active":"2023-08-09T13:36:12.744691+0000","last_peered":"2023-08-09T13:36:12.744691+0000","last_clean":"2023-08-09T13:36:12.744691+0000","last_became_active":"2023-08-09T13:36:11.777965+0000","last_became_peered":"2023-08-09T13:36:11.777965+0000","last_unstale":"2023-08-09T13:36:12.744691+0000","last_undegraded":"2023-08-09T13:36:12.744691+0000","last_fullsized":"2023-08-09T13:36:12.744691+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:36:10.826860+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:36:10.826860+0000","last_clean_scrub_stamp":"2023-08-09T13:36:10.826860+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T21:44:16.043568+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.1b","version":"38'1","reported_seq":14,"reported_epoch":39,"state":"active+clean","last_fresh":"2023-08-09T13:36:13.182981+0000","last_change":"2023-08-09T13:36:10.868079+0000","last_active":"2023-08-09T13:36:13.182981+0000","last_peered":"2023-08-09T13:36:13.182981+0000","last_clean":"2023-08-09T13:36:13.182981+0000","last_became_active":"2023-08-09T13:36:10.867212+0000","last_became_peered":"2023-08-09T13:36:10.867212+0000","last_unstale":"2023-08-09T13:36:13.182981+0000","last_undegraded":"2023-08-09T13:36:13.182981+0000","last_fullsized":"2023-08-09T13:36:13.182981+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:36:09.813580+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:36:09.813580+0000","last_clean_scrub_stamp":"2023-08-09T13:36:09.813580+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T21:20:01.255654+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.1b","version":"0'0","reported_seq":11,"reported_epoch":39,"state":"active+clean","last_fresh":"2023-08-09T13:36:13.179592+0000","last_change":"2023-08-09T13:36:11.770442+0000","last_active":"2023-08-09T13:36:13.179592+0000","last_peered":"2023-08-09T13:36:13.179592+0000","last_clean":"2023-08-09T13:36:13.179592+0000","last_became_active":"2023-08-09T13:36:11.770115+0000","last_became_peered":"2023-08-09T13:36:11.770115+0000","last_unstale":"2023-08-09T13:36:13.179592+0000","last_undegraded":"2023-08-09T13:36:13.179592+0000","last_fullsized":"2023-08-09T13:36:13.179592+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:36:10.826860+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:36:10.826860+0000","last_clean_scrub_stamp":"2023-08-09T13:36:10.826860+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T23:29:32.450147+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.1a","version":"38'1","reported_seq":14,"reported_epoch":39,"state":"active+clean","last_fresh":"2023-08-09T13:36:12.744934+0000","last_change":"2023-08-09T13:36:10.868343+0000","last_active":"2023-08-09T13:36:12.744934+0000","last_peered":"2023-08-09T13:36:12.744934+0000","last_clean":"2023-08-09T13:36:12.744934+0000","last_became_active":"2023-08-09T13:36:10.868069+0000","last_became_peered":"2023-08-09T13:36:10.868069+0000","last_unstale":"2023-08-09T13:36:12.744934+0000","last_undegraded":"2023-08-09T13:36:12.744934+0000","last_fullsized":"2023-08-09T13:36:12.744934+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:36:09.813580+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:36:09.813580+0000","last_clean_scrub_stamp":"2023-08-09T13:36:09.813580+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T15:32:21.128658+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.18","version":"0'0","reported_seq":11,"reported_epoch":39,"state":"active+clean","last_fresh":"2023-08-09T13:36:13.178756+0000","last_change":"2023-08-09T13:36:11.777904+0000","last_active":"2023-08-09T13:36:13.178756+0000","last_peered":"2023-08-09T13:36:13.178756+0000","last_clean":"2023-08-09T13:36:13.178756+0000","last_became_active":"2023-08-09T13:36:11.777127+0000","last_became_peered":"2023-08-09T13:36:11.777127+0000","last_unstale":"2023-08-09T13:36:13.178756+0000","last_undegraded":"2023-08-09T13:36:13.178756+0000","last_fullsized":"2023-08-09T13:36:13.178756+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:36:10.826860+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:36:10.826860+0000","last_clean_scrub_stamp":"2023-08-09T13:36:10.826860+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T20:57:42.024261+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.19","version":"0'0","reported_seq":13,"reported_epoch":39,"state":"active+clean","last_fresh":"2023-08-09T13:36:13.179786+0000","last_change":"2023-08-09T13:36:10.859014+0000","last_active":"2023-08-09T13:36:13.179786+0000","last_peered":"2023-08-09T13:36:13.179786+0000","last_clean":"2023-08-09T13:36:13.179786+0000","last_became_active":"2023-08-09T13:36:10.858570+0000","last_became_peered":"2023-08-09T13:36:10.858570+0000","last_unstale":"2023-08-09T13:36:13.179786+0000","last_undegraded":"2023-08-09T13:36:13.179786+0000","last_fullsized":"2023-08-09T13:36:13.179786+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:36:09.813580+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:36:09.813580+0000","last_clean_scrub_stamp":"2023-08-09T13:36:09.813580+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T14:42:01.156986+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.19","version":"0'0","reported_seq":11,"reported_epoch":39,"state":"active+clean","last_fresh":"2023-08-09T13:36:12.745518+0000","last_change":"2023-08-09T13:36:11.780497+0000","last_active":"2023-08-09T13:36:12.745518+0000","last_peered":"2023-08-09T13:36:12.745518+0000","last_clean":"2023-08-09T13:36:12.745518+0000","last_became_active":"2023-08-09T13:36:11.780226+0000","last_became_peered":"2023-08-09T13:36:11.780226+0000","last_unstale":"2023-08-09T13:36:12.745518+0000","last_undegraded":"2023-08-09T13:36:12.745518+0000","last_fullsized":"2023-08-09T13:36:12.745518+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:36:10.826860+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:36:10.826860+0000","last_clean_scrub_stamp":"2023-08-09T13:36:10.826860+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T13:46:57.087145+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.18","version":"0'0","reported_seq":13,"reported_epoch":39,"state":"active+clean","last_fresh":"2023-08-09T13:36:13.178683+0000","last_change":"2023-08-09T13:36:10.856291+0000","last_active":"2023-08-09T13:36:13.178683+0000","last_peered":"2023-08-09T13:36:13.178683+0000","last_clean":"2023-08-09T13:36:13.178683+0000","last_became_active":"2023-08-09T13:36:10.855922+0000","last_became_peered":"2023-08-09T13:36:10.855922+0000","last_unstale":"2023-08-09T13:36:13.178683+0000","last_undegraded":"2023-08-09T13:36:13.178683+0000","last_fullsized":"2023-08-09T13:36:13.178683+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:36:09.813580+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:36:09.813580+0000","last_clean_scrub_stamp":"2023-08-09T13:36:09.813580+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T17:14:04.967329+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.16","version":"0'0","reported_seq":11,"reported_epoch":39,"state":"active+clean","last_fresh":"2023-08-09T13:36:12.746763+0000","last_change":"2023-08-09T13:36:11.769252+0000","last_active":"2023-08-09T13:36:12.746763+0000","last_peered":"2023-08-09T13:36:12.746763+0000","last_clean":"2023-08-09T13:36:12.746763+0000","last_became_active":"2023-08-09T13:36:11.768571+0000","last_became_peered":"2023-08-09T13:36:11.768571+0000","last_unstale":"2023-08-09T13:36:12.746763+0000","last_undegraded":"2023-08-09T13:36:12.746763+0000","last_fullsized":"2023-08-09T13:36:12.746763+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:36:10.826860+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:36:10.826860+0000","last_clean_scrub_stamp":"2023-08-09T13:36:10.826860+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T23:51:00.601318+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.17","version":"0'0","reported_seq":13,"reported_epoch":39,"state":"active+clean","last_fresh":"2023-08-09T13:36:12.747302+0000","last_change":"2023-08-09T13:36:10.870854+0000","last_active":"2023-08-09T13:36:12.747302+0000","last_peered":"2023-08-09T13:36:12.747302+0000","last_clean":"2023-08-09T13:36:12.747302+0000","last_became_active":"2023-08-09T13:36:10.870525+0000","last_became_peered":"2023-08-09T13:36:10.870525+0000","last_unstale":"2023-08-09T13:36:12.747302+0000","last_undegraded":"2023-08-09T13:36:12.747302+0000","last_fullsized":"2023-08-09T13:36:12.747302+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:36:09.813580+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:36:09.813580+0000","last_clean_scrub_stamp":"2023-08-09T13:36:09.813580+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T14:18:40.169070+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.17","version":"0'0","reported_seq":11,"reported_epoch":39,"state":"active+clean","last_fresh":"2023-08-09T13:36:13.181768+0000","last_change":"2023-08-09T13:36:11.759335+0000","last_active":"2023-08-09T13:36:13.181768+0000","last_peered":"2023-08-09T13:36:13.181768+0000","last_clean":"2023-08-09T13:36:13.181768+0000","last_became_active":"2023-08-09T13:36:11.758848+0000","last_became_peered":"2023-08-09T13:36:11.758848+0000","last_unstale":"2023-08-09T13:36:13.181768+0000","last_undegraded":"2023-08-09T13:36:13.181768+0000","last_fullsized":"2023-08-09T13:36:13.181768+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:36:10.826860+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:36:10.826860+0000","last_clean_scrub_stamp":"2023-08-09T13:36:10.826860+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T16:04:02.242978+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.16","version":"0'0","reported_seq":13,"reported_epoch":39,"state":"active+clean","last_fresh":"2023-08-09T13:36:13.180825+0000","last_change":"2023-08-09T13:36:10.866553+0000","last_active":"2023-08-09T13:36:13.180825+0000","last_peered":"2023-08-09T13:36:13.180825+0000","last_clean":"2023-08-09T13:36:13.180825+0000","last_became_active":"2023-08-09T13:36:10.866120+0000","last_became_peered":"2023-08-09T13:36:10.866120+0000","last_unstale":"2023-08-09T13:36:13.180825+0000","last_undegraded":"2023-08-09T13:36:13.180825+0000","last_fullsized":"2023-08-09T13:36:13.180825+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:36:09.813580+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:36:09.813580+0000","last_clean_scrub_stamp":"2023-08-09T13:36:09.813580+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T15:06:05.146016+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.14","version":"0'0","reported_seq":11,"reported_epoch":39,"state":"active+clean","last_fresh":"2023-08-09T13:36:13.183196+0000","last_change":"2023-08-09T13:36:11.772315+0000","last_active":"2023-08-09T13:36:13.183196+0000","last_peered":"2023-08-09T13:36:13.183196+0000","last_clean":"2023-08-09T13:36:13.183196+0000","last_became_active":"2023-08-09T13:36:11.772006+0000","last_became_peered":"2023-08-09T13:36:11.772006+0000","last_unstale":"2023-08-09T13:36:13.183196+0000","last_undegraded":"2023-08-09T13:36:13.183196+0000","last_fullsized":"2023-08-09T13:36:13.183196+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:36:10.826860+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:36:10.826860+0000","last_clean_scrub_stamp":"2023-08-09T13:36:10.826860+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T18:07:15.877189+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.15","version":"38'3","reported_seq":16,"reported_epoch":39,"state":"active+clean","last_fresh":"2023-08-09T13:36:13.181848+0000","last_change":"2023-08-09T13:36:10.875205+0000","last_active":"2023-08-09T13:36:13.181848+0000","last_peered":"2023-08-09T13:36:13.181848+0000","last_clean":"2023-08-09T13:36:13.181848+0000","last_became_active":"2023-08-09T13:36:10.874765+0000","last_became_peered":"2023-08-09T13:36:10.874765+0000","last_unstale":"2023-08-09T13:36:13.181848+0000","last_undegraded":"2023-08-09T13:36:13.181848+0000","last_fullsized":"2023-08-09T13:36:13.181848+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:36:09.813580+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:36:09.813580+0000","last_clean_scrub_stamp":"2023-08-09T13:36:09.813580+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T21:01:18.302601+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.15","version":"0'0","reported_seq":11,"reported_epoch":39,"state":"active+clean","last_fresh":"2023-08-09T13:36:13.181101+0000","last_change":"2023-08-09T13:36:11.778121+0000","last_active":"2023-08-09T13:36:13.181101+0000","last_peered":"2023-08-09T13:36:13.181101+0000","last_clean":"2023-08-09T13:36:13.181101+0000","last_became_active":"2023-08-09T13:36:11.777690+0000","last_became_peered":"2023-08-09T13:36:11.777690+0000","last_unstale":"2023-08-09T13:36:13.181101+0000","last_undegraded":"2023-08-09T13:36:13.181101+0000","last_fullsized":"2023-08-09T13:36:13.181101+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:36:10.826860+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:36:10.826860+0000","last_clean_scrub_stamp":"2023-08-09T13:36:10.826860+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T16:25:36.538091+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.14","version":"38'6","reported_seq":19,"reported_epoch":39,"state":"active+clean","last_fresh":"2023-08-09T13:36:13.180506+0000","last_change":"2023-08-09T13:36:10.870131+0000","last_active":"2023-08-09T13:36:13.180506+0000","last_peered":"2023-08-09T13:36:13.180506+0000","last_clean":"2023-08-09T13:36:13.180506+0000","last_became_active":"2023-08-09T13:36:10.869794+0000","last_became_peered":"2023-08-09T13:36:10.869794+0000","last_unstale":"2023-08-09T13:36:13.180506+0000","last_undegraded":"2023-08-09T13:36:13.180506+0000","last_fullsized":"2023-08-09T13:36:13.180506+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:36:09.813580+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:36:09.813580+0000","last_clean_scrub_stamp":"2023-08-09T13:36:09.813580+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T15:46:46.549814+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.12","version":"0'0","reported_seq":11,"reported_epoch":39,"state":"active+clean","last_fresh":"2023-08-09T13:36:12.746478+0000","last_change":"2023-08-09T13:36:11.779827+0000","last_active":"2023-08-09T13:36:12.746478+0000","last_peered":"2023-08-09T13:36:12.746478+0000","last_clean":"2023-08-09T13:36:12.746478+0000","last_became_active":"2023-08-09T13:36:11.778884+0000","last_became_peered":"2023-08-09T13:36:11.778884+0000","last_unstale":"2023-08-09T13:36:12.746478+0000","last_undegraded":"2023-08-09T13:36:12.746478+0000","last_fullsized":"2023-08-09T13:36:12.746478+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:36:10.826860+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:36:10.826860+0000","last_clean_scrub_stamp":"2023-08-09T13:36:10.826860+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T14:19:51.658754+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.13","version":"38'2","reported_seq":15,"reported_epoch":39,"state":"active+clean","last_fresh":"2023-08-09T13:36:13.183069+0000","last_change":"2023-08-09T13:36:10.868241+0000","last_active":"2023-08-09T13:36:13.183069+0000","last_peered":"2023-08-09T13:36:13.183069+0000","last_clean":"2023-08-09T13:36:13.183069+0000","last_became_active":"2023-08-09T13:36:10.867827+0000","last_became_peered":"2023-08-09T13:36:10.867827+0000","last_unstale":"2023-08-09T13:36:13.183069+0000","last_undegraded":"2023-08-09T13:36:13.183069+0000","last_fullsized":"2023-08-09T13:36:13.183069+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:36:09.813580+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:36:09.813580+0000","last_clean_scrub_stamp":"2023-08-09T13:36:09.813580+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T17:04:14.699579+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.13","version":"0'0","reported_seq":11,"reported_epoch":39,"state":"active+clean","last_fresh":"2023-08-09T13:36:13.179718+0000","last_change":"2023-08-09T13:36:11.756659+0000","last_active":"2023-08-09T13:36:13.179718+0000","last_peered":"2023-08-09T13:36:13.179718+0000","last_clean":"2023-08-09T13:36:13.179718+0000","last_became_active":"2023-08-09T13:36:11.756166+0000","last_became_peered":"2023-08-09T13:36:11.756166+0000","last_unstale":"2023-08-09T13:36:13.179718+0000","last_undegraded":"2023-08-09T13:36:13.179718+0000","last_fullsized":"2023-08-09T13:36:13.179718+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:36:10.826860+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:36:10.826860+0000","last_clean_scrub_stamp":"2023-08-09T13:36:10.826860+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T16:34:53.205983+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.12","version":"0'0","reported_seq":13,"reported_epoch":39,"state":"active+clean","last_fresh":"2023-08-09T13:36:13.181274+0000","last_change":"2023-08-09T13:36:10.865808+0000","last_active":"2023-08-09T13:36:13.181274+0000","last_peered":"2023-08-09T13:36:13.181274+0000","last_clean":"2023-08-09T13:36:13.181274+0000","last_became_active":"2023-08-09T13:36:10.865466+0000","last_became_peered":"2023-08-09T13:36:10.865466+0000","last_unstale":"2023-08-09T13:36:13.181274+0000","last_undegraded":"2023-08-09T13:36:13.181274+0000","last_fullsized":"2023-08-09T13:36:13.181274+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:36:09.813580+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:36:09.813580+0000","last_clean_scrub_stamp":"2023-08-09T13:36:09.813580+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T18:44:21.503311+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.10","version":"0'0","reported_seq":11,"reported_epoch":39,"state":"active+clean","last_fresh":"2023-08-09T13:36:13.181608+0000","last_change":"2023-08-09T13:36:11.771222+0000","last_active":"2023-08-09T13:36:13.181608+0000","last_peered":"2023-08-09T13:36:13.181608+0000","last_clean":"2023-08-09T13:36:13.181608+0000","last_became_active":"2023-08-09T13:36:11.770198+0000","last_became_peered":"2023-08-09T13:36:11.770198+0000","last_unstale":"2023-08-09T13:36:13.181608+0000","last_undegraded":"2023-08-09T13:36:13.181608+0000","last_fullsized":"2023-08-09T13:36:13.181608+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:36:10.826860+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:36:10.826860+0000","last_clean_scrub_stamp":"2023-08-09T13:36:10.826860+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T22:37:56.162796+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.11","version":"0'0","reported_seq":13,"reported_epoch":39,"state":"active+clean","last_fresh":"2023-08-09T13:36:13.182549+0000","last_change":"2023-08-09T13:36:10.862804+0000","last_active":"2023-08-09T13:36:13.182549+0000","last_peered":"2023-08-09T13:36:13.182549+0000","last_clean":"2023-08-09T13:36:13.182549+0000","last_became_active":"2023-08-09T13:36:10.862524+0000","last_became_peered":"2023-08-09T13:36:10.862524+0000","last_unstale":"2023-08-09T13:36:13.182549+0000","last_undegraded":"2023-08-09T13:36:13.182549+0000","last_fullsized":"2023-08-09T13:36:13.182549+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:36:09.813580+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:36:09.813580+0000","last_clean_scrub_stamp":"2023-08-09T13:36:09.813580+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T16:44:06.470566+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.11","version":"0'0","reported_seq":11,"reported_epoch":39,"state":"active+clean","last_fresh":"2023-08-09T13:36:13.181616+0000","last_change":"2023-08-09T13:36:11.777994+0000","last_active":"2023-08-09T13:36:13.181616+0000","last_peered":"2023-08-09T13:36:13.181616+0000","last_clean":"2023-08-09T13:36:13.181616+0000","last_became_active":"2023-08-09T13:36:11.777683+0000","last_became_peered":"2023-08-09T13:36:11.777683+0000","last_unstale":"2023-08-09T13:36:13.181616+0000","last_undegraded":"2023-08-09T13:36:13.181616+0000","last_fullsized":"2023-08-09T13:36:13.181616+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:36:10.826860+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:36:10.826860+0000","last_clean_scrub_stamp":"2023-08-09T13:36:10.826860+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T17:22:46.595006+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.10","version":"38'8","reported_seq":21,"reported_epoch":39,"state":"active+clean","last_fresh":"2023-08-09T13:36:13.179052+0000","last_change":"2023-08-09T13:36:10.849165+0000","last_active":"2023-08-09T13:36:13.179052+0000","last_peered":"2023-08-09T13:36:13.179052+0000","last_clean":"2023-08-09T13:36:13.179052+0000","last_became_active":"2023-08-09T13:36:10.848413+0000","last_became_peered":"2023-08-09T13:36:10.848413+0000","last_unstale":"2023-08-09T13:36:13.179052+0000","last_undegraded":"2023-08-09T13:36:13.179052+0000","last_fullsized":"2023-08-09T13:36:13.179052+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:36:09.813580+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:36:09.813580+0000","last_clean_scrub_stamp":"2023-08-09T13:36:09.813580+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-11T01:22:58.988197+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.e","version":"0'0","reported_seq":11,"reported_epoch":39,"state":"active+clean","last_fresh":"2023-08-09T13:36:13.181800+0000","last_change":"2023-08-09T13:36:11.764459+0000","last_active":"2023-08-09T13:36:13.181800+0000","last_peered":"2023-08-09T13:36:13.181800+0000","last_clean":"2023-08-09T13:36:13.181800+0000","last_became_active":"2023-08-09T13:36:11.764100+0000","last_became_peered":"2023-08-09T13:36:11.764100+0000","last_unstale":"2023-08-09T13:36:13.181800+0000","last_undegraded":"2023-08-09T13:36:13.181800+0000","last_fullsized":"2023-08-09T13:36:13.181800+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:36:10.826860+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:36:10.826860+0000","last_clean_scrub_stamp":"2023-08-09T13:36:10.826860+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T16:36:09.047515+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.f","version":"38'1","reported_seq":14,"reported_epoch":39,"state":"active+clean","last_fresh":"2023-08-09T13:36:13.179834+0000","last_change":"2023-08-09T13:36:10.863242+0000","last_active":"2023-08-09T13:36:13.179834+0000","last_peered":"2023-08-09T13:36:13.179834+0000","last_clean":"2023-08-09T13:36:13.179834+0000","last_became_active":"2023-08-09T13:36:10.862921+0000","last_became_peered":"2023-08-09T13:36:10.862921+0000","last_unstale":"2023-08-09T13:36:13.179834+0000","last_undegraded":"2023-08-09T13:36:13.179834+0000","last_fullsized":"2023-08-09T13:36:13.179834+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:36:09.813580+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:36:09.813580+0000","last_clean_scrub_stamp":"2023-08-09T13:36:09.813580+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T18:49:41.929234+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.f","version":"0'0","reported_seq":11,"reported_epoch":39,"state":"active+clean","last_fresh":"2023-08-09T13:36:13.181064+0000","last_change":"2023-08-09T13:36:11.764093+0000","last_active":"2023-08-09T13:36:13.181064+0000","last_peered":"2023-08-09T13:36:13.181064+0000","last_clean":"2023-08-09T13:36:13.181064+0000","last_became_active":"2023-08-09T13:36:11.763787+0000","last_became_peered":"2023-08-09T13:36:11.763787+0000","last_unstale":"2023-08-09T13:36:13.181064+0000","last_undegraded":"2023-08-09T13:36:13.181064+0000","last_fullsized":"2023-08-09T13:36:13.181064+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:36:10.826860+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:36:10.826860+0000","last_clean_scrub_stamp":"2023-08-09T13:36:10.826860+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T15:32:48.889127+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.e","version":"38'2","reported_seq":15,"reported_epoch":39,"state":"active+clean","last_fresh":"2023-08-09T13:36:12.747193+0000","last_change":"2023-08-09T13:36:10.867599+0000","last_active":"2023-08-09T13:36:12.747193+0000","last_peered":"2023-08-09T13:36:12.747193+0000","last_clean":"2023-08-09T13:36:12.747193+0000","last_became_active":"2023-08-09T13:36:10.867254+0000","last_became_peered":"2023-08-09T13:36:10.867254+0000","last_unstale":"2023-08-09T13:36:12.747193+0000","last_undegraded":"2023-08-09T13:36:12.747193+0000","last_fullsized":"2023-08-09T13:36:12.747193+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:36:09.813580+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:36:09.813580+0000","last_clean_scrub_stamp":"2023-08-09T13:36:09.813580+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T13:54:32.735702+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.c","version":"0'0","reported_seq":11,"reported_epoch":39,"state":"active+clean","last_fresh":"2023-08-09T13:36:12.747704+0000","last_change":"2023-08-09T13:36:11.776990+0000","last_active":"2023-08-09T13:36:12.747704+0000","last_peered":"2023-08-09T13:36:12.747704+0000","last_clean":"2023-08-09T13:36:12.747704+0000","last_became_active":"2023-08-09T13:36:11.776508+0000","last_became_peered":"2023-08-09T13:36:11.776508+0000","last_unstale":"2023-08-09T13:36:12.747704+0000","last_undegraded":"2023-08-09T13:36:12.747704+0000","last_fullsized":"2023-08-09T13:36:12.747704+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:36:10.826860+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:36:10.826860+0000","last_clean_scrub_stamp":"2023-08-09T13:36:10.826860+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T20:38:04.370278+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.d","version":"38'1","reported_seq":14,"reported_epoch":39,"state":"active+clean","last_fresh":"2023-08-09T13:36:13.181127+0000","last_change":"2023-08-09T13:36:10.860479+0000","last_active":"2023-08-09T13:36:13.181127+0000","last_peered":"2023-08-09T13:36:13.181127+0000","last_clean":"2023-08-09T13:36:13.181127+0000","last_became_active":"2023-08-09T13:36:10.860059+0000","last_became_peered":"2023-08-09T13:36:10.860059+0000","last_unstale":"2023-08-09T13:36:13.181127+0000","last_undegraded":"2023-08-09T13:36:13.181127+0000","last_fullsized":"2023-08-09T13:36:13.181127+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:36:09.813580+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:36:09.813580+0000","last_clean_scrub_stamp":"2023-08-09T13:36:09.813580+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-11T01:22:58.988197+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.d","version":"0'0","reported_seq":11,"reported_epoch":39,"state":"active+clean","last_fresh":"2023-08-09T13:36:12.747803+0000","last_change":"2023-08-09T13:36:11.782782+0000","last_active":"2023-08-09T13:36:12.747803+0000","last_peered":"2023-08-09T13:36:12.747803+0000","last_clean":"2023-08-09T13:36:12.747803+0000","last_became_active":"2023-08-09T13:36:11.782450+0000","last_became_peered":"2023-08-09T13:36:11.782450+0000","last_unstale":"2023-08-09T13:36:12.747803+0000","last_undegraded":"2023-08-09T13:36:12.747803+0000","last_fullsized":"2023-08-09T13:36:12.747803+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:36:10.826860+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:36:10.826860+0000","last_clean_scrub_stamp":"2023-08-09T13:36:10.826860+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T19:58:53.923099+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.c","version":"0'0","reported_seq":13,"reported_epoch":39,"state":"active+clean","last_fresh":"2023-08-09T13:36:13.183552+0000","last_change":"2023-08-09T13:36:10.877362+0000","last_active":"2023-08-09T13:36:13.183552+0000","last_peered":"2023-08-09T13:36:13.183552+0000","last_clean":"2023-08-09T13:36:13.183552+0000","last_became_active":"2023-08-09T13:36:10.877103+0000","last_became_peered":"2023-08-09T13:36:10.877103+0000","last_unstale":"2023-08-09T13:36:13.183552+0000","last_undegraded":"2023-08-09T13:36:13.183552+0000","last_fullsized":"2023-08-09T13:36:13.183552+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:36:09.813580+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:36:09.813580+0000","last_clean_scrub_stamp":"2023-08-09T13:36:09.813580+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T23:38:08.983084+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.a","version":"0'0","reported_seq":11,"reported_epoch":39,"state":"active+clean","last_fresh":"2023-08-09T13:36:13.182078+0000","last_change":"2023-08-09T13:36:11.770333+0000","last_active":"2023-08-09T13:36:13.182078+0000","last_peered":"2023-08-09T13:36:13.182078+0000","last_clean":"2023-08-09T13:36:13.182078+0000","last_became_active":"2023-08-09T13:36:11.770013+0000","last_became_peered":"2023-08-09T13:36:11.770013+0000","last_unstale":"2023-08-09T13:36:13.182078+0000","last_undegraded":"2023-08-09T13:36:13.182078+0000","last_fullsized":"2023-08-09T13:36:13.182078+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:36:10.826860+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:36:10.826860+0000","last_clean_scrub_stamp":"2023-08-09T13:36:10.826860+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T23:14:13.560864+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.b","version":"38'2","reported_seq":15,"reported_epoch":39,"state":"active+clean","last_fresh":"2023-08-09T13:36:13.183444+0000","last_change":"2023-08-09T13:36:10.867524+0000","last_active":"2023-08-09T13:36:13.183444+0000","last_peered":"2023-08-09T13:36:13.183444+0000","last_clean":"2023-08-09T13:36:13.183444+0000","last_became_active":"2023-08-09T13:36:10.866825+0000","last_became_peered":"2023-08-09T13:36:10.866825+0000","last_unstale":"2023-08-09T13:36:13.183444+0000","last_undegraded":"2023-08-09T13:36:13.183444+0000","last_fullsized":"2023-08-09T13:36:13.183444+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:36:09.813580+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:36:09.813580+0000","last_clean_scrub_stamp":"2023-08-09T13:36:09.813580+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T19:18:55.290314+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.b","version":"0'0","reported_seq":11,"reported_epoch":39,"state":"active+clean","last_fresh":"2023-08-09T13:36:12.743542+0000","last_change":"2023-08-09T13:36:11.766655+0000","last_active":"2023-08-09T13:36:12.743542+0000","last_peered":"2023-08-09T13:36:12.743542+0000","last_clean":"2023-08-09T13:36:12.743542+0000","last_became_active":"2023-08-09T13:36:11.766294+0000","last_became_peered":"2023-08-09T13:36:11.766294+0000","last_unstale":"2023-08-09T13:36:12.743542+0000","last_undegraded":"2023-08-09T13:36:12.743542+0000","last_fullsized":"2023-08-09T13:36:12.743542+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:36:10.826860+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:36:10.826860+0000","last_clean_scrub_stamp":"2023-08-09T13:36:10.826860+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T21:18:11.963195+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.a","version":"0'0","reported_seq":13,"reported_epoch":39,"state":"active+clean","last_fresh":"2023-08-09T13:36:13.179977+0000","last_change":"2023-08-09T13:36:10.877398+0000","last_active":"2023-08-09T13:36:13.179977+0000","last_peered":"2023-08-09T13:36:13.179977+0000","last_clean":"2023-08-09T13:36:13.179977+0000","last_became_active":"2023-08-09T13:36:10.877048+0000","last_became_peered":"2023-08-09T13:36:10.877048+0000","last_unstale":"2023-08-09T13:36:13.179977+0000","last_undegraded":"2023-08-09T13:36:13.179977+0000","last_fullsized":"2023-08-09T13:36:13.179977+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:36:09.813580+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:36:09.813580+0000","last_clean_scrub_stamp":"2023-08-09T13:36:09.813580+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T18:28:02.161530+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.8","version":"0'0","reported_seq":11,"reported_epoch":39,"state":"active+clean","last_fresh":"2023-08-09T13:36:13.179747+0000","last_change":"2023-08-09T13:36:11.765089+0000","last_active":"2023-08-09T13:36:13.179747+0000","last_peered":"2023-08-09T13:36:13.179747+0000","last_clean":"2023-08-09T13:36:13.179747+0000","last_became_active":"2023-08-09T13:36:11.763858+0000","last_became_peered":"2023-08-09T13:36:11.763858+0000","last_unstale":"2023-08-09T13:36:13.179747+0000","last_undegraded":"2023-08-09T13:36:13.179747+0000","last_fullsized":"2023-08-09T13:36:13.179747+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:36:10.826860+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:36:10.826860+0000","last_clean_scrub_stamp":"2023-08-09T13:36:10.826860+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T13:50:54.643230+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.9","version":"0'0","reported_seq":13,"reported_epoch":39,"state":"active+clean","last_fresh":"2023-08-09T13:36:13.178917+0000","last_change":"2023-08-09T13:36:10.866720+0000","last_active":"2023-08-09T13:36:13.178917+0000","last_peered":"2023-08-09T13:36:13.178917+0000","last_clean":"2023-08-09T13:36:13.178917+0000","last_became_active":"2023-08-09T13:36:10.866446+0000","last_became_peered":"2023-08-09T13:36:10.866446+0000","last_unstale":"2023-08-09T13:36:13.178917+0000","last_undegraded":"2023-08-09T13:36:13.178917+0000","last_fullsized":"2023-08-09T13:36:13.178917+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:36:09.813580+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:36:09.813580+0000","last_clean_scrub_stamp":"2023-08-09T13:36:09.813580+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T17:14:42.534456+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.9","version":"0'0","reported_seq":11,"reported_epoch":39,"state":"active+clean","last_fresh":"2023-08-09T13:36:13.179056+0000","last_change":"2023-08-09T13:36:11.774417+0000","last_active":"2023-08-09T13:36:13.179056+0000","last_peered":"2023-08-09T13:36:13.179056+0000","last_clean":"2023-08-09T13:36:13.179056+0000","last_became_active":"2023-08-09T13:36:11.771652+0000","last_became_peered":"2023-08-09T13:36:11.771652+0000","last_unstale":"2023-08-09T13:36:13.179056+0000","last_undegraded":"2023-08-09T13:36:13.179056+0000","last_fullsized":"2023-08-09T13:36:13.179056+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:36:10.826860+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:36:10.826860+0000","last_clean_scrub_stamp":"2023-08-09T13:36:10.826860+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T18:10:39.629916+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.8","version":"0'0","reported_seq":13,"reported_epoch":39,"state":"active+clean","last_fresh":"2023-08-09T13:36:13.180072+0000","last_change":"2023-08-09T13:36:10.866696+0000","last_active":"2023-08-09T13:36:13.180072+0000","last_peered":"2023-08-09T13:36:13.180072+0000","last_clean":"2023-08-09T13:36:13.180072+0000","last_became_active":"2023-08-09T13:36:10.866386+0000","last_became_peered":"2023-08-09T13:36:10.866386+0000","last_unstale":"2023-08-09T13:36:13.180072+0000","last_undegraded":"2023-08-09T13:36:13.180072+0000","last_fullsized":"2023-08-09T13:36:13.180072+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:36:09.813580+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:36:09.813580+0000","last_clean_scrub_stamp":"2023-08-09T13:36:09.813580+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T21:14:11.201990+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.1","version":"0'0","reported_seq":11,"reported_epoch":39,"state":"active+clean","last_fresh":"2023-08-09T13:36:12.744809+0000","last_change":"2023-08-09T13:36:11.766822+0000","last_active":"2023-08-09T13:36:12.744809+0000","last_peered":"2023-08-09T13:36:12.744809+0000","last_clean":"2023-08-09T13:36:12.744809+0000","last_became_active":"2023-08-09T13:36:11.766533+0000","last_became_peered":"2023-08-09T13:36:11.766533+0000","last_unstale":"2023-08-09T13:36:12.744809+0000","last_undegraded":"2023-08-09T13:36:12.744809+0000","last_fullsized":"2023-08-09T13:36:12.744809+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:36:10.826860+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:36:10.826860+0000","last_clean_scrub_stamp":"2023-08-09T13:36:10.826860+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T18:33:06.918646+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.0","version":"0'0","reported_seq":13,"reported_epoch":39,"state":"active+clean","last_fresh":"2023-08-09T13:36:13.179109+0000","last_change":"2023-08-09T13:36:10.865247+0000","last_active":"2023-08-09T13:36:13.179109+0000","last_peered":"2023-08-09T13:36:13.179109+0000","last_clean":"2023-08-09T13:36:13.179109+0000","last_became_active":"2023-08-09T13:36:10.864877+0000","last_became_peered":"2023-08-09T13:36:10.864877+0000","last_unstale":"2023-08-09T13:36:13.179109+0000","last_undegraded":"2023-08-09T13:36:13.179109+0000","last_fullsized":"2023-08-09T13:36:13.179109+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:36:09.813580+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:36:09.813580+0000","last_clean_scrub_stamp":"2023-08-09T13:36:09.813580+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T23:41:55.443735+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":50,"reported_epoch":39,"state":"active+clean","last_fresh":"2023-08-09T13:36:12.748224+0000","last_change":"2023-08-09T13:33:55.886389+0000","last_active":"2023-08-09T13:36:12.748224+0000","last_peered":"2023-08-09T13:36:12.748224+0000","last_clean":"2023-08-09T13:36:12.748224+0000","last_became_active":"2023-08-09T13:33:55.885638+0000","last_became_peered":"2023-08-09T13:33:55.885638+0000","last_unstale":"2023-08-09T13:36:12.748224+0000","last_undegraded":"2023-08-09T13:36:12.748224+0000","last_fullsized":"2023-08-09T13:36:12.748224+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:33:54.845869+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:33:54.845869+0000","last_clean_scrub_stamp":"2023-08-09T13:33:54.845869+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T21:22:21.904050+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.0","version":"0'0","reported_seq":11,"reported_epoch":39,"state":"active+clean","last_fresh":"2023-08-09T13:36:13.179673+0000","last_change":"2023-08-09T13:36:11.777749+0000","last_active":"2023-08-09T13:36:13.179673+0000","last_peered":"2023-08-09T13:36:13.179673+0000","last_clean":"2023-08-09T13:36:13.179673+0000","last_became_active":"2023-08-09T13:36:11.777465+0000","last_became_peered":"2023-08-09T13:36:11.777465+0000","last_unstale":"2023-08-09T13:36:13.179673+0000","last_undegraded":"2023-08-09T13:36:13.179673+0000","last_fullsized":"2023-08-09T13:36:13.179673+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:36:10.826860+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:36:10.826860+0000","last_clean_scrub_stamp":"2023-08-09T13:36:10.826860+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T23:06:21.128202+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.1","version":"0'0","reported_seq":13,"reported_epoch":39,"state":"active+clean","last_fresh":"2023-08-09T13:36:12.743107+0000","last_change":"2023-08-09T13:36:10.873994+0000","last_active":"2023-08-09T13:36:12.743107+0000","last_peered":"2023-08-09T13:36:12.743107+0000","last_clean":"2023-08-09T13:36:12.743107+0000","last_became_active":"2023-08-09T13:36:10.873698+0000","last_became_peered":"2023-08-09T13:36:10.873698+0000","last_unstale":"2023-08-09T13:36:12.743107+0000","last_undegraded":"2023-08-09T13:36:12.743107+0000","last_fullsized":"2023-08-09T13:36:12.743107+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:36:09.813580+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:36:09.813580+0000","last_clean_scrub_stamp":"2023-08-09T13:36:09.813580+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T14:32:43.462887+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":50,"reported_epoch":39,"state":"active+clean","last_fresh":"2023-08-09T13:36:12.747882+0000","last_change":"2023-08-09T13:33:56.401781+0000","last_active":"2023-08-09T13:36:12.747882+0000","last_peered":"2023-08-09T13:36:12.747882+0000","last_clean":"2023-08-09T13:36:12.747882+0000","last_became_active":"2023-08-09T13:33:56.400336+0000","last_became_peered":"2023-08-09T13:33:56.400336+0000","last_unstale":"2023-08-09T13:36:12.747882+0000","last_undegraded":"2023-08-09T13:36:12.747882+0000","last_fullsized":"2023-08-09T13:36:12.747882+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:33:54.845869+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:33:54.845869+0000","last_clean_scrub_stamp":"2023-08-09T13:33:54.845869+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-11T00:54:39.337743+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.3","version":"0'0","reported_seq":11,"reported_epoch":39,"state":"active+clean","last_fresh":"2023-08-09T13:36:13.182686+0000","last_change":"2023-08-09T13:36:11.771635+0000","last_active":"2023-08-09T13:36:13.182686+0000","last_peered":"2023-08-09T13:36:13.182686+0000","last_clean":"2023-08-09T13:36:13.182686+0000","last_became_active":"2023-08-09T13:36:11.771047+0000","last_became_peered":"2023-08-09T13:36:11.771047+0000","last_unstale":"2023-08-09T13:36:13.182686+0000","last_undegraded":"2023-08-09T13:36:13.182686+0000","last_fullsized":"2023-08-09T13:36:13.182686+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:36:10.826860+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:36:10.826860+0000","last_clean_scrub_stamp":"2023-08-09T13:36:10.826860+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T22:42:07.388189+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.2","version":"0'0","reported_seq":13,"reported_epoch":39,"state":"active+clean","last_fresh":"2023-08-09T13:36:12.745207+0000","last_change":"2023-08-09T13:36:10.869092+0000","last_active":"2023-08-09T13:36:12.745207+0000","last_peered":"2023-08-09T13:36:12.745207+0000","last_clean":"2023-08-09T13:36:12.745207+0000","last_became_active":"2023-08-09T13:36:10.868759+0000","last_became_peered":"2023-08-09T13:36:10.868759+0000","last_unstale":"2023-08-09T13:36:12.745207+0000","last_undegraded":"2023-08-09T13:36:12.745207+0000","last_fullsized":"2023-08-09T13:36:12.745207+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:36:09.813580+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:36:09.813580+0000","last_clean_scrub_stamp":"2023-08-09T13:36:09.813580+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T19:22:17.764271+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":54,"reported_epoch":39,"state":"active+clean","last_fresh":"2023-08-09T13:36:12.746148+0000","last_change":"2023-08-09T13:33:56.404236+0000","last_active":"2023-08-09T13:36:12.746148+0000","last_peered":"2023-08-09T13:36:12.746148+0000","last_clean":"2023-08-09T13:36:12.746148+0000","last_became_active":"2023-08-09T13:33:56.402485+0000","last_became_peered":"2023-08-09T13:33:56.402485+0000","last_unstale":"2023-08-09T13:36:12.746148+0000","last_undegraded":"2023-08-09T13:36:12.746148+0000","last_fullsized":"2023-08-09T13:36:12.746148+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:33:54.845869+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:33:54.845869+0000","last_clean_scrub_stamp":"2023-08-09T13:33:54.845869+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-11T00:39:23.589743+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.2","version":"0'0","reported_seq":11,"reported_epoch":39,"state":"active+clean","last_fresh":"2023-08-09T13:36:13.182480+0000","last_change":"2023-08-09T13:36:11.780136+0000","last_active":"2023-08-09T13:36:13.182480+0000","last_peered":"2023-08-09T13:36:13.182480+0000","last_clean":"2023-08-09T13:36:13.182480+0000","last_became_active":"2023-08-09T13:36:11.779856+0000","last_became_peered":"2023-08-09T13:36:11.779856+0000","last_unstale":"2023-08-09T13:36:13.182480+0000","last_undegraded":"2023-08-09T13:36:13.182480+0000","last_fullsized":"2023-08-09T13:36:13.182480+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:36:10.826860+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:36:10.826860+0000","last_clean_scrub_stamp":"2023-08-09T13:36:10.826860+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T18:55:31.620023+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.3","version":"0'0","reported_seq":13,"reported_epoch":39,"state":"active+clean","last_fresh":"2023-08-09T13:36:13.181606+0000","last_change":"2023-08-09T13:36:10.877771+0000","last_active":"2023-08-09T13:36:13.181606+0000","last_peered":"2023-08-09T13:36:13.181606+0000","last_clean":"2023-08-09T13:36:13.181606+0000","last_became_active":"2023-08-09T13:36:10.877493+0000","last_became_peered":"2023-08-09T13:36:10.877493+0000","last_unstale":"2023-08-09T13:36:13.181606+0000","last_undegraded":"2023-08-09T13:36:13.181606+0000","last_fullsized":"2023-08-09T13:36:13.181606+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:36:09.813580+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:36:09.813580+0000","last_clean_scrub_stamp":"2023-08-09T13:36:09.813580+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T20:42:30.032317+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":58,"reported_epoch":39,"state":"active+clean","last_fresh":"2023-08-09T13:36:13.183659+0000","last_change":"2023-08-09T13:33:55.889565+0000","last_active":"2023-08-09T13:36:13.183659+0000","last_peered":"2023-08-09T13:36:13.183659+0000","last_clean":"2023-08-09T13:36:13.183659+0000","last_became_active":"2023-08-09T13:33:55.889164+0000","last_became_peered":"2023-08-09T13:33:55.889164+0000","last_unstale":"2023-08-09T13:36:13.183659+0000","last_undegraded":"2023-08-09T13:36:13.183659+0000","last_fullsized":"2023-08-09T13:36:13.183659+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:33:54.845869+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:33:54.845869+0000","last_clean_scrub_stamp":"2023-08-09T13:33:54.845869+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T15:55:03.403580+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.5","version":"0'0","reported_seq":11,"reported_epoch":39,"state":"active+clean","last_fresh":"2023-08-09T13:36:13.181466+0000","last_change":"2023-08-09T13:36:11.765282+0000","last_active":"2023-08-09T13:36:13.181466+0000","last_peered":"2023-08-09T13:36:13.181466+0000","last_clean":"2023-08-09T13:36:13.181466+0000","last_became_active":"2023-08-09T13:36:11.764922+0000","last_became_peered":"2023-08-09T13:36:11.764922+0000","last_unstale":"2023-08-09T13:36:13.181466+0000","last_undegraded":"2023-08-09T13:36:13.181466+0000","last_fullsized":"2023-08-09T13:36:13.181466+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:36:10.826860+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:36:10.826860+0000","last_clean_scrub_stamp":"2023-08-09T13:36:10.826860+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T14:34:10.458179+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.4","version":"0'0","reported_seq":13,"reported_epoch":39,"state":"active+clean","last_fresh":"2023-08-09T13:36:12.743299+0000","last_change":"2023-08-09T13:36:10.869464+0000","last_active":"2023-08-09T13:36:12.743299+0000","last_peered":"2023-08-09T13:36:12.743299+0000","last_clean":"2023-08-09T13:36:12.743299+0000","last_became_active":"2023-08-09T13:36:10.869112+0000","last_became_peered":"2023-08-09T13:36:10.869112+0000","last_unstale":"2023-08-09T13:36:12.743299+0000","last_undegraded":"2023-08-09T13:36:12.743299+0000","last_fullsized":"2023-08-09T13:36:12.743299+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:36:09.813580+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:36:09.813580+0000","last_clean_scrub_stamp":"2023-08-09T13:36:09.813580+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T18:01:27.790776+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"16'117","reported_seq":200,"reported_epoch":39,"state":"active+clean","last_fresh":"2023-08-09T13:36:13.179633+0000","last_change":"2023-08-09T13:33:55.886345+0000","last_active":"2023-08-09T13:36:13.179633+0000","last_peered":"2023-08-09T13:36:13.179633+0000","last_clean":"2023-08-09T13:36:13.179633+0000","last_became_active":"2023-08-09T13:33:55.885781+0000","last_became_peered":"2023-08-09T13:33:55.885781+0000","last_unstale":"2023-08-09T13:36:13.179633+0000","last_undegraded":"2023-08-09T13:36:13.179633+0000","last_fullsized":"2023-08-09T13:36:13.179633+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:33:54.845869+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:33:54.845869+0000","last_clean_scrub_stamp":"2023-08-09T13:33:54.845869+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T15:21:07.169824+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.3","version":"0'0","reported_seq":52,"reported_epoch":39,"state":"active+clean","last_fresh":"2023-08-09T13:36:12.743662+0000","last_change":"2023-08-09T13:33:55.886749+0000","last_active":"2023-08-09T13:36:12.743662+0000","last_peered":"2023-08-09T13:36:12.743662+0000","last_clean":"2023-08-09T13:36:12.743662+0000","last_became_active":"2023-08-09T13:33:55.886116+0000","last_became_peered":"2023-08-09T13:33:55.886116+0000","last_unstale":"2023-08-09T13:36:12.743662+0000","last_undegraded":"2023-08-09T13:36:12.743662+0000","last_fullsized":"2023-08-09T13:36:12.743662+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:33:54.845869+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:33:54.845869+0000","last_clean_scrub_stamp":"2023-08-09T13:33:54.845869+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-11T01:21:30.839021+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.4","version":"0'0","reported_seq":11,"reported_epoch":39,"state":"active+clean","last_fresh":"2023-08-09T13:36:13.181549+0000","last_change":"2023-08-09T13:36:11.764825+0000","last_active":"2023-08-09T13:36:13.181549+0000","last_peered":"2023-08-09T13:36:13.181549+0000","last_clean":"2023-08-09T13:36:13.181549+0000","last_became_active":"2023-08-09T13:36:11.764519+0000","last_became_peered":"2023-08-09T13:36:11.764519+0000","last_unstale":"2023-08-09T13:36:13.181549+0000","last_undegraded":"2023-08-09T13:36:13.181549+0000","last_fullsized":"2023-08-09T13:36:13.181549+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:36:10.826860+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:36:10.826860+0000","last_clean_scrub_stamp":"2023-08-09T13:36:10.826860+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T22:18:23.522022+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.5","version":"0'0","reported_seq":13,"reported_epoch":39,"state":"active+clean","last_fresh":"2023-08-09T13:36:13.181888+0000","last_change":"2023-08-09T13:36:10.875065+0000","last_active":"2023-08-09T13:36:13.181888+0000","last_peered":"2023-08-09T13:36:13.181888+0000","last_clean":"2023-08-09T13:36:13.181888+0000","last_became_active":"2023-08-09T13:36:10.874358+0000","last_became_peered":"2023-08-09T13:36:10.874358+0000","last_unstale":"2023-08-09T13:36:13.181888+0000","last_undegraded":"2023-08-09T13:36:13.181888+0000","last_fullsized":"2023-08-09T13:36:13.181888+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:36:09.813580+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:36:09.813580+0000","last_clean_scrub_stamp":"2023-08-09T13:36:09.813580+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T22:46:33.401724+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.2","version":"0'0","reported_seq":50,"reported_epoch":39,"state":"active+clean","last_fresh":"2023-08-09T13:36:13.181307+0000","last_change":"2023-08-09T13:33:56.402794+0000","last_active":"2023-08-09T13:36:13.181307+0000","last_peered":"2023-08-09T13:36:13.181307+0000","last_clean":"2023-08-09T13:36:13.181307+0000","last_became_active":"2023-08-09T13:33:56.402038+0000","last_became_peered":"2023-08-09T13:33:56.402038+0000","last_unstale":"2023-08-09T13:36:13.181307+0000","last_undegraded":"2023-08-09T13:36:13.181307+0000","last_fullsized":"2023-08-09T13:36:13.181307+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:33:54.845869+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:33:54.845869+0000","last_clean_scrub_stamp":"2023-08-09T13:33:54.845869+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T15:54:41.238281+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.7","version":"0'0","reported_seq":11,"reported_epoch":39,"state":"active+clean","last_fresh":"2023-08-09T13:36:12.747913+0000","last_change":"2023-08-09T13:36:11.784020+0000","last_active":"2023-08-09T13:36:12.747913+0000","last_peered":"2023-08-09T13:36:12.747913+0000","last_clean":"2023-08-09T13:36:12.747913+0000","last_became_active":"2023-08-09T13:36:11.783758+0000","last_became_peered":"2023-08-09T13:36:11.783758+0000","last_unstale":"2023-08-09T13:36:12.747913+0000","last_undegraded":"2023-08-09T13:36:12.747913+0000","last_fullsized":"2023-08-09T13:36:12.747913+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:36:10.826860+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:36:10.826860+0000","last_clean_scrub_stamp":"2023-08-09T13:36:10.826860+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T23:43:23.933266+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.6","version":"0'0","reported_seq":13,"reported_epoch":39,"state":"active+clean","last_fresh":"2023-08-09T13:36:13.183415+0000","last_change":"2023-08-09T13:36:10.871425+0000","last_active":"2023-08-09T13:36:13.183415+0000","last_peered":"2023-08-09T13:36:13.183415+0000","last_clean":"2023-08-09T13:36:13.183415+0000","last_became_active":"2023-08-09T13:36:10.871105+0000","last_became_peered":"2023-08-09T13:36:10.871105+0000","last_unstale":"2023-08-09T13:36:13.183415+0000","last_undegraded":"2023-08-09T13:36:13.183415+0000","last_fullsized":"2023-08-09T13:36:13.183415+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:36:09.813580+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:36:09.813580+0000","last_clean_scrub_stamp":"2023-08-09T13:36:09.813580+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T18:30:24.843777+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":50,"reported_epoch":39,"state":"active+clean","last_fresh":"2023-08-09T13:36:13.179994+0000","last_change":"2023-08-09T13:33:56.401123+0000","last_active":"2023-08-09T13:36:13.179994+0000","last_peered":"2023-08-09T13:36:13.179994+0000","last_clean":"2023-08-09T13:36:13.179994+0000","last_became_active":"2023-08-09T13:33:56.399933+0000","last_became_peered":"2023-08-09T13:33:56.399933+0000","last_unstale":"2023-08-09T13:36:13.179994+0000","last_undegraded":"2023-08-09T13:36:13.179994+0000","last_fullsized":"2023-08-09T13:36:13.179994+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:33:54.845869+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:33:54.845869+0000","last_clean_scrub_stamp":"2023-08-09T13:33:54.845869+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T21:23:42.787477+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.6","version":"0'0","reported_seq":11,"reported_epoch":39,"state":"active+clean","last_fresh":"2023-08-09T13:36:13.180703+0000","last_change":"2023-08-09T13:36:11.762581+0000","last_active":"2023-08-09T13:36:13.180703+0000","last_peered":"2023-08-09T13:36:13.180703+0000","last_clean":"2023-08-09T13:36:13.180703+0000","last_became_active":"2023-08-09T13:36:11.762279+0000","last_became_peered":"2023-08-09T13:36:11.762279+0000","last_unstale":"2023-08-09T13:36:13.180703+0000","last_undegraded":"2023-08-09T13:36:13.180703+0000","last_fullsized":"2023-08-09T13:36:13.180703+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:36:10.826860+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:36:10.826860+0000","last_clean_scrub_stamp":"2023-08-09T13:36:10.826860+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T20:44:32.204365+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.7","version":"38'2","reported_seq":15,"reported_epoch":39,"state":"active+clean","last_fresh":"2023-08-09T13:36:13.180695+0000","last_change":"2023-08-09T13:36:10.865646+0000","last_active":"2023-08-09T13:36:13.180695+0000","last_peered":"2023-08-09T13:36:13.180695+0000","last_clean":"2023-08-09T13:36:13.180695+0000","last_became_active":"2023-08-09T13:36:10.865350+0000","last_became_peered":"2023-08-09T13:36:10.865350+0000","last_unstale":"2023-08-09T13:36:13.180695+0000","last_undegraded":"2023-08-09T13:36:13.180695+0000","last_fullsized":"2023-08-09T13:36:13.180695+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:36:09.813580+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:36:09.813580+0000","last_clean_scrub_stamp":"2023-08-09T13:36:09.813580+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T20:57:41.010981+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":50,"reported_epoch":39,"state":"active+clean","last_fresh":"2023-08-09T13:36:13.180307+0000","last_change":"2023-08-09T13:33:56.401406+0000","last_active":"2023-08-09T13:36:13.180307+0000","last_peered":"2023-08-09T13:36:13.180307+0000","last_clean":"2023-08-09T13:36:13.180307+0000","last_became_active":"2023-08-09T13:33:56.400487+0000","last_became_peered":"2023-08-09T13:33:56.400487+0000","last_unstale":"2023-08-09T13:36:13.180307+0000","last_undegraded":"2023-08-09T13:36:13.180307+0000","last_fullsized":"2023-08-09T13:36:13.180307+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:33:54.845869+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:33:54.845869+0000","last_clean_scrub_stamp":"2023-08-09T13:33:54.845869+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T14:48:49.894855+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":7,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":6,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":38,"ondisk_log_size":38,"up":64,"acting":64,"num_store_stats":8},{"poolid":2,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":8}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574878,"num_pgs":9,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32140,"kb_used_data":1024,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":94339700,"statfs":{"total":96636764160,"available":96603852800,"internally_reserved":0,"allocated":1048576,"data_stored":794063,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1781,"internal_metadata":31848715},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Aug 9 13:35:55 2023","interfaces":[{"interface":"back","average":{"1min":1.026,"5min":1.1859999999999999,"15min":1.212},"min":{"1min":0.61099999999999999,"5min":0.61099999999999999,"15min":0.61099999999999999},"max":{"1min":1.734,"5min":2.1709999999999998,"15min":2.1709999999999998},"last":1},{"interface":"front","average":{"1min":1.1180000000000001,"5min":1.3220000000000001,"15min":1.3560000000000001},"min":{"1min":0.74099999999999999,"5min":0.74099999999999999,"15min":0.74099999999999999},"max":{"1min":1.603,"5min":2.6240000000000001,"15min":2.6240000000000001},"last":0.76100000000000001}]},{"osd":1,"last update":"Wed Aug 9 13:35:55 2023","interfaces":[{"interface":"back","average":{"1min":1.119,"5min":1.718,"15min":1.8180000000000001},"min":{"1min":0.68500000000000005,"5min":0.68500000000000005,"15min":0.68500000000000005},"max":{"1min":1.7909999999999999,"5min":10.823,"15min":10.823},"last":0.85099999999999998},{"interface":"front","average":{"1min":1.2709999999999999,"5min":1.645,"15min":1.7070000000000001},"min":{"1min":0.73199999999999998,"5min":0.73199999999999998,"15min":0.73199999999999998},"max":{"1min":1.736,"5min":10.576000000000001,"15min":10.576000000000001},"last":1.0900000000000001}]},{"osd":2,"last update":"Wed Aug 9 13:35:55 2023","interfaces":[{"interface":"back","average":{"1min":1.0589999999999999,"5min":1.3220000000000001,"15min":1.3660000000000001},"min":{"1min":0.79300000000000004,"5min":0.78400000000000003,"15min":0.78400000000000003},"max":{"1min":1.4079999999999999,"5min":3.5270000000000001,"15min":3.5270000000000001},"last":1.2689999999999999},{"interface":"front","average":{"1min":1.319,"5min":1.4239999999999999,"15min":1.4419999999999999},"min":{"1min":0.83899999999999997,"5min":0.82099999999999995,"15min":0.82099999999999995},"max":{"1min":1.9019999999999999,"5min":3.1190000000000002,"15min":3.1190000000000002},"last":1.157}]},{"osd":3,"last update":"Wed Aug 9 13:35:55 2023","interfaces":[{"interface":"back","average":{"1min":1.1919999999999999,"5min":1.3169999999999999,"15min":1.3380000000000001},"min":{"1min":0.90900000000000003,"5min":0.82999999999999996,"15min":0.82999999999999996},"max":{"1min":1.542,"5min":2.3010000000000002,"15min":2.3010000000000002},"last":1.5449999999999999},{"interface":"front","average":{"1min":1.3979999999999999,"5min":1.5580000000000001,"15min":1.5840000000000001},"min":{"1min":0.93600000000000005,"5min":0.93600000000000005,"15min":0.93600000000000005},"max":{"1min":2.0139999999999998,"5min":3.2770000000000001,"15min":3.2770000000000001},"last":1.2410000000000001}]},{"osd":4,"last update":"Wed Aug 9 13:35:55 2023","interfaces":[{"interface":"back","average":{"1min":1.278,"5min":1.478,"15min":1.5109999999999999},"min":{"1min":0.72899999999999998,"5min":0.72899999999999998,"15min":0.72899999999999998},"max":{"1min":1.857,"5min":3.0209999999999999,"15min":3.0209999999999999},"last":1.204},{"interface":"front","average":{"1min":1.3320000000000001,"5min":1.5129999999999999,"15min":1.5429999999999999},"min":{"1min":0.56799999999999995,"5min":0.56799999999999995,"15min":0.56799999999999995},"max":{"1min":1.653,"5min":3.169,"15min":3.169},"last":1.6160000000000001}]},{"osd":5,"last update":"Wed Aug 9 13:35:55 2023","interfaces":[{"interface":"back","average":{"1min":1.4690000000000001,"5min":1.6459999999999999,"15min":1.6759999999999999},"min":{"1min":0.87,"5min":0.87,"15min":0.87},"max":{"1min":2.113,"5min":3.5830000000000002,"15min":3.5830000000000002},"last":1.3520000000000001},{"interface":"front","average":{"1min":1.468,"5min":1.613,"15min":1.637},"min":{"1min":0.69199999999999995,"5min":0.69199999999999995,"15min":0.69199999999999995},"max":{"1min":1.919,"5min":3.3439999999999999,"15min":3.3439999999999999},"last":1.3999999999999999}]},{"osd":6,"last update":"Wed Aug 9 13:35:55 2023","interfaces":[{"interface":"back","average":{"1min":1.5509999999999999,"5min":1.716,"15min":1.744},"min":{"1min":1.0549999999999999,"5min":1.0549999999999999,"15min":1.0549999999999999},"max":{"1min":2.1520000000000001,"5min":3.2189999999999999,"15min":3.2189999999999999},"last":1.518},{"interface":"front","average":{"1min":1.4199999999999999,"5min":1.6200000000000001,"15min":1.653},"min":{"1min":0.60799999999999998,"5min":0.60799999999999998,"15min":0.60799999999999998},"max":{"1min":1.7549999999999999,"5min":2.9590000000000001,"15min":2.9590000000000001},"last":1.73}]}]},{"osd":6,"up_from":13,"seq":55834574878,"num_pgs":32,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27472,"kb_used_data":444,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344368,"statfs":{"total":96636764160,"available":96608632832,"internally_reserved":0,"allocated":454656,"data_stored":202916,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1781,"internal_metadata":27654411},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Aug 9 13:35:58 2023","interfaces":[{"interface":"back","average":{"1min":1.109,"5min":1.149,"15min":1.1559999999999999},"min":{"1min":0.56599999999999995,"5min":0.56599999999999995,"15min":0.56599999999999995},"max":{"1min":1.776,"5min":1.776,"15min":1.776},"last":0.82799999999999996},{"interface":"front","average":{"1min":1.0880000000000001,"5min":1.0940000000000001,"15min":1.095},"min":{"1min":0.76000000000000001,"5min":0.72999999999999998,"15min":0.72999999999999998},"max":{"1min":1.4259999999999999,"5min":1.7509999999999999,"15min":1.7509999999999999},"last":1.026}]},{"osd":1,"last update":"Wed Aug 9 13:35:58 2023","interfaces":[{"interface":"back","average":{"1min":1.004,"5min":1.167,"15min":1.194},"min":{"1min":0.65500000000000003,"5min":0.65500000000000003,"15min":0.65500000000000003},"max":{"1min":1.5229999999999999,"5min":1.77,"15min":1.77},"last":0.97799999999999998},{"interface":"front","average":{"1min":1.135,"5min":1.1659999999999999,"15min":1.171},"min":{"1min":0.68100000000000005,"5min":0.68100000000000005,"15min":0.68100000000000005},"max":{"1min":1.4610000000000001,"5min":1.6120000000000001,"15min":1.6120000000000001},"last":1.2989999999999999}]},{"osd":2,"last update":"Wed Aug 9 13:35:58 2023","interfaces":[{"interface":"back","average":{"1min":1.1499999999999999,"5min":1.1970000000000001,"15min":1.2050000000000001},"min":{"1min":0.68700000000000006,"5min":0.68700000000000006,"15min":0.68700000000000006},"max":{"1min":1.6599999999999999,"5min":1.6599999999999999,"15min":1.6599999999999999},"last":1.639},{"interface":"front","average":{"1min":1.141,"5min":1.161,"15min":1.165},"min":{"1min":0.82699999999999996,"5min":0.59499999999999997,"15min":0.59499999999999997},"max":{"1min":1.5840000000000001,"5min":1.5840000000000001,"15min":1.5840000000000001},"last":1.1839999999999999}]},{"osd":3,"last update":"Wed Aug 9 13:35:58 2023","interfaces":[{"interface":"back","average":{"1min":1.2589999999999999,"5min":1.2250000000000001,"15min":1.2190000000000001},"min":{"1min":0.92400000000000004,"5min":0.73799999999999999,"15min":0.73799999999999999},"max":{"1min":1.6439999999999999,"5min":1.724,"15min":1.724},"last":1.4710000000000001},{"interface":"front","average":{"1min":1.1990000000000001,"5min":1.282,"15min":1.296},"min":{"1min":0.80200000000000005,"5min":0.79600000000000004,"15min":0.79600000000000004},"max":{"1min":1.48,"5min":1.915,"15min":1.915},"last":1.3759999999999999}]},{"osd":4,"last update":"Wed Aug 9 13:35:58 2023","interfaces":[{"interface":"back","average":{"1min":1.2569999999999999,"5min":1.385,"15min":1.407},"min":{"1min":0.79200000000000004,"5min":0.79200000000000004,"15min":0.79200000000000004},"max":{"1min":1.6910000000000001,"5min":2.1269999999999998,"15min":2.1269999999999998},"last":1.5309999999999999},{"interface":"front","average":{"1min":1.375,"5min":1.3720000000000001,"15min":1.3720000000000001},"min":{"1min":0.69099999999999995,"5min":0.69099999999999995,"15min":0.69099999999999995},"max":{"1min":1.748,"5min":1.8999999999999999,"15min":1.8999999999999999},"last":1.583}]},{"osd":5,"last update":"Wed Aug 9 13:35:58 2023","interfaces":[{"interface":"back","average":{"1min":1.278,"5min":1.379,"15min":1.3959999999999999},"min":{"1min":0.65400000000000003,"5min":0.65400000000000003,"15min":0.65400000000000003},"max":{"1min":1.698,"5min":2.0910000000000002,"15min":2.0910000000000002},"last":1.8180000000000001},{"interface":"front","average":{"1min":1.492,"5min":1.5960000000000001,"15min":1.613},"min":{"1min":1.054,"5min":1.054,"15min":1.054},"max":{"1min":1.9830000000000001,"5min":2.516,"15min":2.516},"last":1.8799999999999999}]},{"osd":7,"last update":"Wed Aug 9 13:35:58 2023","interfaces":[{"interface":"back","average":{"1min":1.5389999999999999,"5min":1.5620000000000001,"15min":1.5660000000000001},"min":{"1min":0.77600000000000002,"5min":0.77600000000000002,"15min":0.77600000000000002},"max":{"1min":1.8999999999999999,"5min":2.3570000000000002,"15min":2.3570000000000002},"last":1.782},{"interface":"front","average":{"1min":1.524,"5min":1.5960000000000001,"15min":1.6080000000000001},"min":{"1min":0.95699999999999996,"5min":0.95699999999999996,"15min":0.95699999999999996},"max":{"1min":1.909,"5min":2.2210000000000001,"15min":2.2210000000000001},"last":1.6830000000000001}]}]},{"osd":5,"up_from":13,"seq":55834574878,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27464,"kb_used_data":444,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344376,"statfs":{"total":96636764160,"available":96608641024,"internally_reserved":0,"allocated":454656,"data_stored":203675,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1781,"internal_metadata":27654411},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Aug 9 13:35:58 2023","interfaces":[{"interface":"back","average":{"1min":1.1419999999999999,"5min":1.117,"15min":1.113},"min":{"1min":0.67700000000000005,"5min":0.67700000000000005,"15min":0.67700000000000005},"max":{"1min":1.599,"5min":1.7130000000000001,"15min":1.7130000000000001},"last":1.0429999999999999},{"interface":"front","average":{"1min":1.0800000000000001,"5min":1.157,"15min":1.1699999999999999},"min":{"1min":0.79400000000000004,"5min":0.78200000000000003,"15min":0.78200000000000003},"max":{"1min":1.496,"5min":1.6859999999999999,"15min":1.6859999999999999},"last":1.1040000000000001}]},{"osd":1,"last update":"Wed Aug 9 13:35:58 2023","interfaces":[{"interface":"back","average":{"1min":1.0629999999999999,"5min":1.1399999999999999,"15min":1.153},"min":{"1min":0.748,"5min":0.748,"15min":0.748},"max":{"1min":1.546,"5min":1.599,"15min":1.599},"last":1.407},{"interface":"front","average":{"1min":1.179,"5min":1.2150000000000001,"15min":1.2210000000000001},"min":{"1min":0.75800000000000001,"5min":0.75800000000000001,"15min":0.75800000000000001},"max":{"1min":1.4790000000000001,"5min":1.5640000000000001,"15min":1.5640000000000001},"last":0.92600000000000005}]},{"osd":2,"last update":"Wed Aug 9 13:35:58 2023","interfaces":[{"interface":"back","average":{"1min":1.2110000000000001,"5min":1.159,"15min":1.1499999999999999},"min":{"1min":0.72299999999999998,"5min":0.72299999999999998,"15min":0.72299999999999998},"max":{"1min":1.899,"5min":1.899,"15min":1.899},"last":0.85799999999999998},{"interface":"front","average":{"1min":1.153,"5min":1.202,"15min":1.21},"min":{"1min":0.66800000000000004,"5min":0.66800000000000004,"15min":0.66800000000000004},"max":{"1min":1.663,"5min":1.6739999999999999,"15min":1.6739999999999999},"last":0.98299999999999998}]},{"osd":3,"last update":"Wed Aug 9 13:35:58 2023","interfaces":[{"interface":"back","average":{"1min":1.3540000000000001,"5min":1.266,"15min":1.252},"min":{"1min":1.0389999999999999,"5min":0.79000000000000004,"15min":0.79000000000000004},"max":{"1min":2.2690000000000001,"5min":2.2690000000000001,"15min":2.2690000000000001},"last":1.147},{"interface":"front","average":{"1min":1.3939999999999999,"5min":1.4099999999999999,"15min":1.413},"min":{"1min":0.96499999999999997,"5min":0.96499999999999997,"15min":0.96499999999999997},"max":{"1min":2.028,"5min":2.028,"15min":2.028},"last":1.365}]},{"osd":4,"last update":"Wed Aug 9 13:35:58 2023","interfaces":[{"interface":"back","average":{"1min":1.24,"5min":1.286,"15min":1.294},"min":{"1min":0.61399999999999999,"5min":0.61399999999999999,"15min":0.61399999999999999},"max":{"1min":2.121,"5min":2.121,"15min":2.121},"last":1.1859999999999999},{"interface":"front","average":{"1min":1.389,"5min":1.4430000000000001,"15min":1.452},"min":{"1min":0.58599999999999997,"5min":0.58599999999999997,"15min":0.58599999999999997},"max":{"1min":2.3679999999999999,"5min":2.3679999999999999,"15min":2.3679999999999999},"last":1.302}]},{"osd":6,"last update":"Wed Aug 9 13:35:58 2023","interfaces":[{"interface":"back","average":{"1min":1.4710000000000001,"5min":1.4730000000000001,"15min":1.4730000000000001},"min":{"1min":0.77700000000000002,"5min":0.77700000000000002,"15min":0.77700000000000002},"max":{"1min":2.1110000000000002,"5min":2.1110000000000002,"15min":2.1110000000000002},"last":1.238},{"interface":"front","average":{"1min":1.462,"5min":1.494,"15min":1.4990000000000001},"min":{"1min":0.71199999999999997,"5min":0.71199999999999997,"15min":0.71199999999999997},"max":{"1min":2.2090000000000001,"5min":2.2090000000000001,"15min":2.2090000000000001},"last":1.512}]},{"osd":7,"last update":"Wed Aug 9 13:35:58 2023","interfaces":[{"interface":"back","average":{"1min":1.5389999999999999,"5min":1.597,"15min":1.607},"min":{"1min":0.70899999999999996,"5min":0.70899999999999996,"15min":0.70899999999999996},"max":{"1min":2.4729999999999999,"5min":2.4729999999999999,"15min":2.4729999999999999},"last":1.4470000000000001},{"interface":"front","average":{"1min":1.673,"5min":1.6379999999999999,"15min":1.6319999999999999},"min":{"1min":0.93100000000000005,"5min":0.93100000000000005,"15min":0.93100000000000005},"max":{"1min":2.2069999999999999,"5min":2.2069999999999999,"15min":2.2069999999999999},"last":1.466}]}]},{"osd":4,"up_from":13,"seq":55834574878,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27480,"kb_used_data":452,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344360,"statfs":{"total":96636764160,"available":96608624640,"internally_reserved":0,"allocated":462848,"data_stored":205217,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1781,"internal_metadata":27654411},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Aug 9 13:35:57 2023","interfaces":[{"interface":"back","average":{"1min":1.073,"5min":1.0860000000000001,"15min":1.0880000000000001},"min":{"1min":0.67700000000000005,"5min":0.64200000000000002,"15min":0.64200000000000002},"max":{"1min":1.3680000000000001,"5min":1.5860000000000001,"15min":1.5860000000000001},"last":1.4239999999999999},{"interface":"front","average":{"1min":1.1759999999999999,"5min":1.196,"15min":1.2},"min":{"1min":0.71199999999999997,"5min":0.69999999999999996,"15min":0.69999999999999996},"max":{"1min":1.6240000000000001,"5min":1.7909999999999999,"15min":1.7909999999999999},"last":1.361}]},{"osd":1,"last update":"Wed Aug 9 13:35:57 2023","interfaces":[{"interface":"back","average":{"1min":1.123,"5min":1.2909999999999999,"15min":1.319},"min":{"1min":0.77100000000000002,"5min":0.77100000000000002,"15min":0.77100000000000002},"max":{"1min":1.601,"5min":1.615,"15min":1.615},"last":1.127},{"interface":"front","average":{"1min":1.1419999999999999,"5min":1.1659999999999999,"15min":1.1699999999999999},"min":{"1min":0.64900000000000002,"5min":0.64900000000000002,"15min":0.64900000000000002},"max":{"1min":1.577,"5min":1.6259999999999999,"15min":1.6259999999999999},"last":1.179}]},{"osd":2,"last update":"Wed Aug 9 13:35:57 2023","interfaces":[{"interface":"back","average":{"1min":1.224,"5min":1.222,"15min":1.222},"min":{"1min":0.89400000000000002,"5min":0.89400000000000002,"15min":0.89400000000000002},"max":{"1min":1.681,"5min":1.681,"15min":1.681},"last":1.0609999999999999},{"interface":"front","average":{"1min":1.272,"5min":1.401,"15min":1.423},"min":{"1min":0.74299999999999999,"5min":0.74299999999999999,"15min":0.74299999999999999},"max":{"1min":1.5660000000000001,"5min":1.948,"15min":1.948},"last":1.0009999999999999}]},{"osd":3,"last update":"Wed Aug 9 13:35:57 2023","interfaces":[{"interface":"back","average":{"1min":1.167,"5min":1.3080000000000001,"15min":1.3320000000000001},"min":{"1min":0.93400000000000005,"5min":0.88500000000000001,"15min":0.88500000000000001},"max":{"1min":1.48,"5min":2.3780000000000001,"15min":2.3780000000000001},"last":0.85099999999999998},{"interface":"front","average":{"1min":1.222,"5min":1.353,"15min":1.375},"min":{"1min":0.91000000000000003,"5min":0.73099999999999998,"15min":0.73099999999999998},"max":{"1min":1.6180000000000001,"5min":2.113,"15min":2.113},"last":0.93400000000000005}]},{"osd":5,"last update":"Wed Aug 9 13:35:57 2023","interfaces":[{"interface":"back","average":{"1min":1.345,"5min":1.409,"15min":1.419},"min":{"1min":0.86099999999999999,"5min":0.86099999999999999,"15min":0.86099999999999999},"max":{"1min":1.7070000000000001,"5min":2.2959999999999998,"15min":2.2959999999999998},"last":1.472},{"interface":"front","average":{"1min":1.375,"5min":1.343,"15min":1.3380000000000001},"min":{"1min":0.92800000000000005,"5min":0.92800000000000005,"15min":0.92800000000000005},"max":{"1min":1.7929999999999999,"5min":1.8859999999999999,"15min":1.8859999999999999},"last":1.3200000000000001}]},{"osd":6,"last update":"Wed Aug 9 13:35:57 2023","interfaces":[{"interface":"back","average":{"1min":1.4379999999999999,"5min":1.405,"15min":1.399},"min":{"1min":1.091,"5min":0.82499999999999996,"15min":0.82499999999999996},"max":{"1min":1.669,"5min":1.9379999999999999,"15min":1.9379999999999999},"last":1.272},{"interface":"front","average":{"1min":1.508,"5min":1.5449999999999999,"15min":1.5509999999999999},"min":{"1min":0.997,"5min":0.997,"15min":0.997},"max":{"1min":1.6879999999999999,"5min":2.1899999999999999,"15min":2.1899999999999999},"last":1.5900000000000001}]},{"osd":7,"last update":"Wed Aug 9 13:35:57 2023","interfaces":[{"interface":"back","average":{"1min":1.4630000000000001,"5min":1.51,"15min":1.518},"min":{"1min":1.232,"5min":1.1359999999999999,"15min":1.1359999999999999},"max":{"1min":1.6559999999999999,"5min":2.2210000000000001,"15min":2.2210000000000001},"last":1.397},{"interface":"front","average":{"1min":1.5449999999999999,"5min":1.5569999999999999,"15min":1.5589999999999999},"min":{"1min":1.3500000000000001,"5min":1.1679999999999999,"15min":1.1679999999999999},"max":{"1min":1.788,"5min":2.0489999999999999,"15min":2.0489999999999999},"last":1.542}]}]},{"osd":3,"up_from":13,"seq":55834574879,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31568,"kb_used_data":452,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":94340272,"statfs":{"total":96636764160,"available":96604438528,"internally_reserved":0,"allocated":462848,"data_stored":204243,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1781,"internal_metadata":31848715},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Aug 9 13:35:55 2023","interfaces":[{"interface":"back","average":{"1min":0.92500000000000004,"5min":0.93200000000000005,"15min":0.93300000000000005},"min":{"1min":0.38800000000000001,"5min":0.38800000000000001,"15min":0.38800000000000001},"max":{"1min":1.9199999999999999,"5min":1.944,"15min":1.944},"last":1.462},{"interface":"front","average":{"1min":0.96899999999999997,"5min":0.91700000000000004,"15min":0.90900000000000003},"min":{"1min":0.44600000000000001,"5min":0.40799999999999997,"15min":0.40799999999999997},"max":{"1min":2.2360000000000002,"5min":2.2360000000000002,"15min":2.2360000000000002},"last":0.92100000000000004}]},{"osd":1,"last update":"Wed Aug 9 13:35:55 2023","interfaces":[{"interface":"back","average":{"1min":0.875,"5min":0.86299999999999999,"15min":0.86099999999999999},"min":{"1min":0.51100000000000001,"5min":0.46200000000000002,"15min":0.46200000000000002},"max":{"1min":1.387,"5min":1.6919999999999999,"15min":1.6919999999999999},"last":0.80100000000000005},{"interface":"front","average":{"1min":1.024,"5min":1.0840000000000001,"15min":1.0940000000000001},"min":{"1min":0.42099999999999999,"5min":0.42099999999999999,"15min":0.42099999999999999},"max":{"1min":1.6439999999999999,"5min":2.3359999999999999,"15min":2.3359999999999999},"last":1.0509999999999999}]},{"osd":2,"last update":"Wed Aug 9 13:35:55 2023","interfaces":[{"interface":"back","average":{"1min":0.98299999999999998,"5min":0.91500000000000004,"15min":0.90300000000000002},"min":{"1min":0.53500000000000003,"5min":0.53500000000000003,"15min":0.53500000000000003},"max":{"1min":1.5069999999999999,"5min":1.5629999999999999,"15min":1.5629999999999999},"last":1.127},{"interface":"front","average":{"1min":1.109,"5min":1.208,"15min":1.224},"min":{"1min":0.52000000000000002,"5min":0.52000000000000002,"15min":0.52000000000000002},"max":{"1min":1.5409999999999999,"5min":2.1429999999999998,"15min":2.1429999999999998},"last":1.593}]},{"osd":4,"last update":"Wed Aug 9 13:35:55 2023","interfaces":[{"interface":"back","average":{"1min":1.228,"5min":1.2589999999999999,"15min":1.264},"min":{"1min":0.66500000000000004,"5min":0.66500000000000004,"15min":0.66500000000000004},"max":{"1min":1.7090000000000001,"5min":1.992,"15min":1.992},"last":1.3140000000000001},{"interface":"front","average":{"1min":1.2150000000000001,"5min":1.3460000000000001,"15min":1.3680000000000001},"min":{"1min":0.79400000000000004,"5min":0.79400000000000004,"15min":0.79400000000000004},"max":{"1min":1.869,"5min":3.0649999999999999,"15min":3.0649999999999999},"last":1.226}]},{"osd":5,"last update":"Wed Aug 9 13:35:55 2023","interfaces":[{"interface":"back","average":{"1min":1.323,"5min":1.3919999999999999,"15min":1.4039999999999999},"min":{"1min":0.73099999999999998,"5min":0.73099999999999998,"15min":0.73099999999999998},"max":{"1min":1.9570000000000001,"5min":2.3610000000000002,"15min":2.3610000000000002},"last":1.5580000000000001},{"interface":"front","average":{"1min":1.339,"5min":1.407,"15min":1.4179999999999999},"min":{"1min":0.81799999999999995,"5min":0.79100000000000004,"15min":0.79100000000000004},"max":{"1min":1.827,"5min":2.2639999999999998,"15min":2.2639999999999998},"last":1.7090000000000001}]},{"osd":6,"last update":"Wed Aug 9 13:35:55 2023","interfaces":[{"interface":"back","average":{"1min":1.323,"5min":1.371,"15min":1.379},"min":{"1min":0.86599999999999999,"5min":0.82299999999999995,"15min":0.82299999999999995},"max":{"1min":1.78,"5min":2.1629999999999998,"15min":2.1629999999999998},"last":1.516},{"interface":"front","average":{"1min":1.3080000000000001,"5min":1.452,"15min":1.476},"min":{"1min":0.68600000000000005,"5min":0.68600000000000005,"15min":0.68600000000000005},"max":{"1min":2.0169999999999999,"5min":3.0049999999999999,"15min":3.0049999999999999},"last":1.6279999999999999}]},{"osd":7,"last update":"Wed Aug 9 13:35:55 2023","interfaces":[{"interface":"back","average":{"1min":1.4390000000000001,"5min":1.5569999999999999,"15min":1.577},"min":{"1min":0.78800000000000003,"5min":0.78800000000000003,"15min":0.78800000000000003},"max":{"1min":1.9099999999999999,"5min":2.6120000000000001,"15min":2.6120000000000001},"last":1.8149999999999999},{"interface":"front","average":{"1min":1.3819999999999999,"5min":1.482,"15min":1.4990000000000001},"min":{"1min":0.61599999999999999,"5min":0.61599999999999999,"15min":0.61599999999999999},"max":{"1min":1.843,"5min":2.2989999999999999,"15min":2.2989999999999999},"last":1.766}]}]},{"osd":2,"up_from":13,"seq":55834574878,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31572,"kb_used_data":456,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":94340268,"statfs":{"total":96636764160,"available":96604434432,"internally_reserved":0,"allocated":466944,"data_stored":205239,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1781,"internal_metadata":31848715},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Aug 9 13:35:56 2023","interfaces":[{"interface":"back","average":{"1min":0.81399999999999995,"5min":0.90000000000000002,"15min":0.91400000000000003},"min":{"1min":0.41499999999999998,"5min":0.41499999999999998,"15min":0.41499999999999998},"max":{"1min":1.1930000000000001,"5min":1.542,"15min":1.542},"last":1.8380000000000001},{"interface":"front","average":{"1min":0.96199999999999997,"5min":1.0680000000000001,"15min":1.0860000000000001},"min":{"1min":0.51900000000000002,"5min":0.45300000000000001,"15min":0.45300000000000001},"max":{"1min":1.5309999999999999,"5min":1.7549999999999999,"15min":1.7549999999999999},"last":1.0289999999999999}]},{"osd":1,"last update":"Wed Aug 9 13:35:56 2023","interfaces":[{"interface":"back","average":{"1min":1.091,"5min":1.139,"15min":1.147},"min":{"1min":0.55400000000000005,"5min":0.54400000000000004,"15min":0.54400000000000004},"max":{"1min":1.6759999999999999,"5min":2.0249999999999999,"15min":2.0249999999999999},"last":0.96199999999999997},{"interface":"front","average":{"1min":1.1819999999999999,"5min":1.2310000000000001,"15min":1.2390000000000001},"min":{"1min":0.64600000000000002,"5min":0.52900000000000003,"15min":0.52900000000000003},"max":{"1min":1.925,"5min":1.9930000000000001,"15min":1.9930000000000001},"last":0.89400000000000002}]},{"osd":3,"last update":"Wed Aug 9 13:35:56 2023","interfaces":[{"interface":"back","average":{"1min":0.876,"5min":0.98299999999999998,"15min":1.0009999999999999},"min":{"1min":0.46800000000000003,"5min":0.42599999999999999,"15min":0.42599999999999999},"max":{"1min":1.524,"5min":1.635,"15min":1.635},"last":1.0640000000000001},{"interface":"front","average":{"1min":1.085,"5min":1.3360000000000001,"15min":1.3779999999999999},"min":{"1min":0.624,"5min":0.55300000000000005,"15min":0.55300000000000005},"max":{"1min":1.5700000000000001,"5min":3.024,"15min":3.024},"last":1.399}]},{"osd":4,"last update":"Wed Aug 9 13:35:56 2023","interfaces":[{"interface":"back","average":{"1min":1.3100000000000001,"5min":1.4790000000000001,"15min":1.5069999999999999},"min":{"1min":0.88,"5min":0.77900000000000003,"15min":0.77900000000000003},"max":{"1min":1.593,"5min":3.137,"15min":3.137},"last":1.6539999999999999},{"interface":"front","average":{"1min":1.347,"5min":1.431,"15min":1.4450000000000001},"min":{"1min":0.92600000000000005,"5min":0.79000000000000004,"15min":0.79000000000000004},"max":{"1min":1.756,"5min":2.5150000000000001,"15min":2.5150000000000001},"last":1.593}]},{"osd":5,"last update":"Wed Aug 9 13:35:56 2023","interfaces":[{"interface":"back","average":{"1min":1.347,"5min":1.458,"15min":1.476},"min":{"1min":0.94099999999999995,"5min":0.75900000000000001,"15min":0.75900000000000001},"max":{"1min":1.8540000000000001,"5min":2.3839999999999999,"15min":2.3839999999999999},"last":1.776},{"interface":"front","average":{"1min":1.254,"5min":1.4510000000000001,"15min":1.484},"min":{"1min":0.90600000000000003,"5min":0.90600000000000003,"15min":0.90600000000000003},"max":{"1min":1.6080000000000001,"5min":3.8599999999999999,"15min":3.8599999999999999},"last":1.282}]},{"osd":6,"last update":"Wed Aug 9 13:35:56 2023","interfaces":[{"interface":"back","average":{"1min":1.2509999999999999,"5min":1.389,"15min":1.4119999999999999},"min":{"1min":0.94199999999999995,"5min":0.80700000000000005,"15min":0.80700000000000005},"max":{"1min":1.659,"5min":2.476,"15min":2.476},"last":1.3620000000000001},{"interface":"front","average":{"1min":1.429,"5min":1.6120000000000001,"15min":1.6419999999999999},"min":{"1min":1.0269999999999999,"5min":1.0269999999999999,"15min":1.0269999999999999},"max":{"1min":1.798,"5min":3.5350000000000001,"15min":3.5350000000000001},"last":1.9419999999999999}]},{"osd":7,"last update":"Wed Aug 9 13:35:56 2023","interfaces":[{"interface":"back","average":{"1min":1.4790000000000001,"5min":1.7170000000000001,"15min":1.7569999999999999},"min":{"1min":0.90000000000000002,"5min":0.90000000000000002,"15min":0.90000000000000002},"max":{"1min":1.988,"5min":5.2969999999999997,"15min":5.2969999999999997},"last":1.899},{"interface":"front","average":{"1min":1.3300000000000001,"5min":1.617,"15min":1.665},"min":{"1min":0.94999999999999996,"5min":0.85899999999999999,"15min":0.85899999999999999},"max":{"1min":1.7230000000000001,"5min":5.4539999999999997,"15min":5.4539999999999997},"last":1.323}]}]},{"osd":1,"up_from":13,"seq":55834574878,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32140,"kb_used_data":1024,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":94339700,"statfs":{"total":96636764160,"available":96603852800,"internally_reserved":0,"allocated":1048576,"data_stored":794043,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1781,"internal_metadata":31848715},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Aug 9 13:35:57 2023","interfaces":[{"interface":"back","average":{"1min":0.80500000000000005,"5min":0.94899999999999995,"15min":0.97299999999999998},"min":{"1min":0.32400000000000001,"5min":0.32400000000000001,"15min":0.32400000000000001},"max":{"1min":1.369,"5min":2.7080000000000002,"15min":2.7080000000000002},"last":0.93000000000000005},{"interface":"front","average":{"1min":0.82999999999999996,"5min":1.02,"15min":1.052},"min":{"1min":0.42499999999999999,"5min":0.42499999999999999,"15min":0.42499999999999999},"max":{"1min":1.2769999999999999,"5min":2.6349999999999998,"15min":2.6349999999999998},"last":1.091}]},{"osd":2,"last update":"Wed Aug 9 13:35:57 2023","interfaces":[{"interface":"back","average":{"1min":0.86499999999999999,"5min":0.95199999999999996,"15min":0.96599999999999997},"min":{"1min":0.54200000000000004,"5min":0.54200000000000004,"15min":0.54200000000000004},"max":{"1min":1.2230000000000001,"5min":1.762,"15min":1.762},"last":1.0149999999999999},{"interface":"front","average":{"1min":0.93700000000000006,"5min":1.141,"15min":1.175},"min":{"1min":0.46500000000000002,"5min":0.46500000000000002,"15min":0.46500000000000002},"max":{"1min":1.7,"5min":2.2330000000000001,"15min":2.2330000000000001},"last":1.0469999999999999}]},{"osd":3,"last update":"Wed Aug 9 13:35:57 2023","interfaces":[{"interface":"back","average":{"1min":1.0309999999999999,"5min":1.177,"15min":1.2010000000000001},"min":{"1min":0.432,"5min":0.432,"15min":0.432},"max":{"1min":1.5820000000000001,"5min":1.5820000000000001,"15min":1.5820000000000001},"last":1.3169999999999999},{"interface":"front","average":{"1min":0.96399999999999997,"5min":1.216,"15min":1.258},"min":{"1min":0.44400000000000001,"5min":0.44400000000000001,"15min":0.44400000000000001},"max":{"1min":1.4570000000000001,"5min":3.3069999999999999,"15min":3.3069999999999999},"last":0.97299999999999998}]},{"osd":4,"last update":"Wed Aug 9 13:35:57 2023","interfaces":[{"interface":"back","average":{"1min":1.216,"5min":1.3380000000000001,"15min":1.3580000000000001},"min":{"1min":0.79000000000000004,"5min":0.79000000000000004,"15min":0.79000000000000004},"max":{"1min":1.569,"5min":2.504,"15min":2.504},"last":1.163},{"interface":"front","average":{"1min":1.2230000000000001,"5min":1.3440000000000001,"15min":1.3640000000000001},"min":{"1min":0.76400000000000001,"5min":0.76400000000000001,"15min":0.76400000000000001},"max":{"1min":1.7470000000000001,"5min":2.5099999999999998,"15min":2.5099999999999998},"last":1.1319999999999999}]},{"osd":5,"last update":"Wed Aug 9 13:35:57 2023","interfaces":[{"interface":"back","average":{"1min":1.252,"5min":1.3799999999999999,"15min":1.401},"min":{"1min":0.82699999999999996,"5min":0.82699999999999996,"15min":0.82699999999999996},"max":{"1min":2.1040000000000001,"5min":2.2570000000000001,"15min":2.2570000000000001},"last":1.425},{"interface":"front","average":{"1min":1.351,"5min":1.671,"15min":1.724},"min":{"1min":0.77700000000000002,"5min":0.77700000000000002,"15min":0.77700000000000002},"max":{"1min":2.024,"5min":4.9560000000000004,"15min":4.9560000000000004},"last":1.4770000000000001}]},{"osd":6,"last update":"Wed Aug 9 13:35:57 2023","interfaces":[{"interface":"back","average":{"1min":1.3149999999999999,"5min":1.484,"15min":1.512},"min":{"1min":0.78400000000000003,"5min":0.78400000000000003,"15min":0.78400000000000003},"max":{"1min":1.9810000000000001,"5min":3.0529999999999999,"15min":3.0529999999999999},"last":1.276},{"interface":"front","average":{"1min":1.288,"5min":1.5029999999999999,"15min":1.5389999999999999},"min":{"1min":0.79800000000000004,"5min":0.79800000000000004,"15min":0.79800000000000004},"max":{"1min":1.9410000000000001,"5min":2.7330000000000001,"15min":2.7330000000000001},"last":1.242}]},{"osd":7,"last update":"Wed Aug 9 13:35:57 2023","interfaces":[{"interface":"back","average":{"1min":1.3839999999999999,"5min":1.7330000000000001,"15min":1.7909999999999999},"min":{"1min":0.77100000000000002,"5min":0.77100000000000002,"15min":0.77100000000000002},"max":{"1min":2.1440000000000001,"5min":5.1360000000000001,"15min":5.1360000000000001},"last":1.6259999999999999},{"interface":"front","average":{"1min":1.371,"5min":1.5880000000000001,"15min":1.6240000000000001},"min":{"1min":0.75900000000000001,"5min":0.75900000000000001,"15min":0.75900000000000001},"max":{"1min":2.0720000000000001,"5min":3.194,"15min":3.194},"last":1.375}]}]},{"osd":0,"up_from":13,"seq":55834574878,"num_pgs":34,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31556,"kb_used_data":440,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":94340284,"statfs":{"total":96636764160,"available":96604450816,"internally_reserved":0,"allocated":450560,"data_stored":202278,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1781,"internal_metadata":31848715},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Wed Aug 9 13:36:00 2023","interfaces":[{"interface":"back","average":{"1min":0.88700000000000001,"5min":0.88,"15min":0.879},"min":{"1min":0.39600000000000002,"5min":0.39600000000000002,"15min":0.39600000000000002},"max":{"1min":1.4890000000000001,"5min":1.4890000000000001,"15min":1.4890000000000001},"last":1.1279999999999999},{"interface":"front","average":{"1min":1.0289999999999999,"5min":0.97499999999999998,"15min":0.96599999999999997},"min":{"1min":0.57399999999999995,"5min":0.53200000000000003,"15min":0.53200000000000003},"max":{"1min":1.581,"5min":1.581,"15min":1.581},"last":1.0509999999999999}]},{"osd":2,"last update":"Wed Aug 9 13:36:00 2023","interfaces":[{"interface":"back","average":{"1min":1.0269999999999999,"5min":1.071,"15min":1.079},"min":{"1min":0.44800000000000001,"5min":0.44800000000000001,"15min":0.44800000000000001},"max":{"1min":1.4990000000000001,"5min":1.7629999999999999,"15min":1.7629999999999999},"last":1.5569999999999999},{"interface":"front","average":{"1min":1.0589999999999999,"5min":1.1100000000000001,"15min":1.1180000000000001},"min":{"1min":0.52700000000000002,"5min":0.52700000000000002,"15min":0.52700000000000002},"max":{"1min":2.2519999999999998,"5min":2.2519999999999998,"15min":2.2519999999999998},"last":1.294}]},{"osd":3,"last update":"Wed Aug 9 13:36:00 2023","interfaces":[{"interface":"back","average":{"1min":1.0669999999999999,"5min":1.0569999999999999,"15min":1.0549999999999999},"min":{"1min":0.55200000000000005,"5min":0.55000000000000004,"15min":0.55000000000000004},"max":{"1min":2.5190000000000001,"5min":2.5649999999999999,"15min":2.5649999999999999},"last":0.76800000000000002},{"interface":"front","average":{"1min":1.242,"5min":1.2829999999999999,"15min":1.29},"min":{"1min":0.58799999999999997,"5min":0.58799999999999997,"15min":0.58799999999999997},"max":{"1min":2.4260000000000002,"5min":2.6819999999999999,"15min":2.6819999999999999},"last":1.46}]},{"osd":4,"last update":"Wed Aug 9 13:36:00 2023","interfaces":[{"interface":"back","average":{"1min":1.3160000000000001,"5min":1.3080000000000001,"15min":1.306},"min":{"1min":0.87,"5min":0.87,"15min":0.87},"max":{"1min":1.6870000000000001,"5min":1.6890000000000001,"15min":1.6890000000000001},"last":1.2430000000000001},{"interface":"front","average":{"1min":1.357,"5min":1.4370000000000001,"15min":1.45},"min":{"1min":0.74099999999999999,"5min":0.74099999999999999,"15min":0.74099999999999999},"max":{"1min":1.8959999999999999,"5min":2.181,"15min":2.181},"last":1.3260000000000001}]},{"osd":5,"last update":"Wed Aug 9 13:36:00 2023","interfaces":[{"interface":"back","average":{"1min":1.4259999999999999,"5min":1.4219999999999999,"15min":1.4219999999999999},"min":{"1min":0.93400000000000005,"5min":0.89200000000000002,"15min":0.89200000000000002},"max":{"1min":1.7150000000000001,"5min":2.085,"15min":2.085},"last":1.363},{"interface":"front","average":{"1min":1.365,"5min":1.3620000000000001,"15min":1.3620000000000001},"min":{"1min":0.79600000000000004,"5min":0.79600000000000004,"15min":0.79600000000000004},"max":{"1min":1.825,"5min":1.825,"15min":1.825},"last":1.3819999999999999}]},{"osd":6,"last update":"Wed Aug 9 13:36:00 2023","interfaces":[{"interface":"back","average":{"1min":1.47,"5min":1.546,"15min":1.5589999999999999},"min":{"1min":0.69799999999999995,"5min":0.69799999999999995,"15min":0.69799999999999995},"max":{"1min":1.867,"5min":2.2509999999999999,"15min":2.2509999999999999},"last":1.6719999999999999},{"interface":"front","average":{"1min":1.5269999999999999,"5min":1.4990000000000001,"15min":1.4950000000000001},"min":{"1min":1.083,"5min":0.96499999999999997,"15min":0.96499999999999997},"max":{"1min":1.998,"5min":2.2959999999999998,"15min":2.2959999999999998},"last":1.4239999999999999}]},{"osd":7,"last update":"Wed Aug 9 13:36:00 2023","interfaces":[{"interface":"back","average":{"1min":1.617,"5min":1.5780000000000001,"15min":1.5720000000000001},"min":{"1min":0.99199999999999999,"5min":0.99199999999999999,"15min":0.99199999999999999},"max":{"1min":2.464,"5min":2.464,"15min":2.464},"last":1.526},{"interface":"front","average":{"1min":1.569,"5min":1.589,"15min":1.5920000000000001},"min":{"1min":0.83199999999999996,"5min":0.83199999999999996,"15min":0.83199999999999996},"max":{"1min":2.0019999999999998,"5min":2.024,"15min":2.024},"last":1.605}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":34,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":70,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1634,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":638,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":70,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":672,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-08-09T13:36:16.409 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-08-09T13:36:16.680 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:16.678+0000 7f757d4ec640 1 Processor -- start 2023-08-09T13:36:16.680 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:16.678+0000 7f757d4ec640 1 -- start start 2023-08-09T13:36:16.682 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:16.680+0000 7f757d4ec640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f757815a340 0x7f75780cac60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:36:16.682 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:16.680+0000 7f757d4ec640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f75780ce620 0x7f75780ceab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:36:16.682 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:16.680+0000 7f757d4ec640 1 -- --> v1:172.21.15.203:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f757815a720 con 0x7f75780cb1a0 2023-08-09T13:36:16.683 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:16.680+0000 7f757d4ec640 1 -- --> v2:172.21.15.43:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f75780cd970 con 0x7f757815a340 2023-08-09T13:36:16.683 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:16.680+0000 7f757d4ec640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f75780b0190 con 0x7f75780ce620 2023-08-09T13:36:16.683 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:16.680+0000 7f7577fff640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f757815a340 0x7f75780cac60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:36:16.683 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:16.680+0000 7f7577fff640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f757815a340 0x7f75780cac60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:58106/0 (socket says 172.21.15.43:58106) 2023-08-09T13:36:16.684 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:16.680+0000 7f7577fff640 1 -- 172.21.15.43:0/694003026 learned_addr learned my addr 172.21.15.43:0/694003026 (peer_addr_for_me v2:172.21.15.43:0/0) 2023-08-09T13:36:16.686 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:16.681+0000 7f7577fff640 1 -- 172.21.15.43:0/694003026 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f75780ce620 msgr2=0x7f75780ceab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-09T13:36:16.687 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:16.681+0000 7f7577fff640 1 --2- 172.21.15.43:0/694003026 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f75780ce620 0x7f75780ceab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:36:16.687 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:16.681+0000 7f7577fff640 1 -- 172.21.15.43:0/694003026 >> v1:172.21.15.203:6789/0 conn(0x7f75780cb1a0 legacy=0x7f75780cc980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-09T13:36:16.687 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:16.681+0000 7f7577fff640 1 -- 172.21.15.43:0/694003026 --> v2:172.21.15.43:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f75780ceff0 con 0x7f757815a340 2023-08-09T13:36:16.687 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:16.682+0000 7f7577fff640 1 --2- 172.21.15.43:0/694003026 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f757815a340 0x7f75780cac60 secure :-1 s=READY pgs=575 cs=0 l=1 rev1=1 crypto rx=0x7f7568009e50 tx=0x7f756802bdc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2f4cc2adf3d0041d server_cookie=0 in_seq=0 out_seq=0 2023-08-09T13:36:16.688 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:16.682+0000 7f7576ffd640 1 -- 172.21.15.43:0/694003026 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f756802c960 con 0x7f757815a340 2023-08-09T13:36:16.688 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:16.682+0000 7f7576ffd640 1 -- 172.21.15.43:0/694003026 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f756802cac0 con 0x7f757815a340 2023-08-09T13:36:16.688 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:16.682+0000 7f7576ffd640 1 -- 172.21.15.43:0/694003026 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f756802ce50 con 0x7f757815a340 2023-08-09T13:36:16.689 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:16.683+0000 7f757d4ec640 1 -- 172.21.15.43:0/694003026 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f757815a340 msgr2=0x7f75780cac60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:36:16.689 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:16.683+0000 7f757d4ec640 1 --2- 172.21.15.43:0/694003026 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f757815a340 0x7f75780cac60 secure :-1 s=READY pgs=575 cs=0 l=1 rev1=1 crypto rx=0x7f7568009e50 tx=0x7f756802bdc0 comp rx=0 tx=0).stop 2023-08-09T13:36:16.689 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:16.683+0000 7f757d4ec640 1 -- 172.21.15.43:0/694003026 shutdown_connections 2023-08-09T13:36:16.689 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:16.683+0000 7f757d4ec640 1 --2- 172.21.15.43:0/694003026 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f75780ce620 0x7f75780ceab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:36:16.690 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:16.683+0000 7f757d4ec640 1 --2- 172.21.15.43:0/694003026 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f757815a340 0x7f75780cac60 unknown :-1 s=CLOSED pgs=575 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:36:16.690 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:16.683+0000 7f757d4ec640 1 -- 172.21.15.43:0/694003026 >> 172.21.15.43:0/694003026 conn(0x7f7578089fe0 msgr2=0x7f7578099470 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-09T13:36:16.690 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:16.683+0000 7f757d4ec640 1 -- 172.21.15.43:0/694003026 shutdown_connections 2023-08-09T13:36:16.690 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:16.683+0000 7f757d4ec640 1 -- 172.21.15.43:0/694003026 wait complete. 2023-08-09T13:36:16.691 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:16.685+0000 7f757d4ec640 1 Processor -- start 2023-08-09T13:36:16.691 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:16.685+0000 7f757d4ec640 1 -- start start 2023-08-09T13:36:16.694 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:16.686+0000 7f757d4ec640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f75780cb1a0 0x7f7578152a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:36:16.695 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:16.686+0000 7f757d4ec640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f75780ce620 0x7f75780c48a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:36:16.695 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:16.686+0000 7f757d4ec640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f757815a340 0x7f75780ade50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:36:16.695 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:16.686+0000 7f757d4ec640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f75780c5160 con 0x7f757815a340 2023-08-09T13:36:16.695 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:16.686+0000 7f757d4ec640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f757815bdd0 con 0x7f75780ce620 2023-08-09T13:36:16.696 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:16.686+0000 7f757d4ec640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f757815c0b0 con 0x7f75780cb1a0 2023-08-09T13:36:16.696 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:16.686+0000 7f757cceb640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f757815a340 0x7f75780ade50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:36:16.697 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:16.686+0000 7f757cceb640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f757815a340 0x7f75780ade50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:58120/0 (socket says 172.21.15.43:58120) 2023-08-09T13:36:16.697 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:16.686+0000 7f757cceb640 1 -- 172.21.15.43:0/1044930140 learned_addr learned my addr 172.21.15.43:0/1044930140 (peer_addr_for_me v2:172.21.15.43:0/0) 2023-08-09T13:36:16.697 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:16.687+0000 7f75777fe640 1 --2- 172.21.15.43:0/1044930140 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f75780ce620 0x7f75780c48a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:36:16.697 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:16.687+0000 7f7577fff640 1 --2- 172.21.15.43:0/1044930140 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f75780cb1a0 0x7f7578152a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:36:16.698 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:16.687+0000 7f757cceb640 1 -- 172.21.15.43:0/1044930140 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f75780cb1a0 msgr2=0x7f7578152a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:36:16.698 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:16.687+0000 7f757cceb640 1 --2- 172.21.15.43:0/1044930140 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f75780cb1a0 0x7f7578152a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:36:16.698 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:16.687+0000 7f757cceb640 1 -- 172.21.15.43:0/1044930140 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f75780ce620 msgr2=0x7f75780c48a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:36:16.698 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:16.687+0000 7f757cceb640 1 --2- 172.21.15.43:0/1044930140 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f75780ce620 0x7f75780c48a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:36:16.699 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:16.687+0000 7f757cceb640 1 -- 172.21.15.43:0/1044930140 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7568009b30 con 0x7f757815a340 2023-08-09T13:36:16.699 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:16.687+0000 7f75777fe640 1 --2- 172.21.15.43:0/1044930140 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f75780ce620 0x7f75780c48a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-08-09T13:36:16.699 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:16.687+0000 7f757cceb640 1 --2- 172.21.15.43:0/1044930140 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f757815a340 0x7f75780ade50 secure :-1 s=READY pgs=576 cs=0 l=1 rev1=1 crypto rx=0x7f7570009fd0 tx=0x7f757000b8e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-09T13:36:16.700 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:16.687+0000 7f7574ff9640 1 -- 172.21.15.43:0/1044930140 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7570011830 con 0x7f757815a340 2023-08-09T13:36:16.700 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:16.687+0000 7f7574ff9640 1 -- 172.21.15.43:0/1044930140 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7570011990 con 0x7f757815a340 2023-08-09T13:36:16.700 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:16.687+0000 7f757d4ec640 1 -- 172.21.15.43:0/1044930140 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f757815c390 con 0x7f757815a340 2023-08-09T13:36:16.700 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:16.687+0000 7f7574ff9640 1 -- 172.21.15.43:0/1044930140 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7570011d20 con 0x7f757815a340 2023-08-09T13:36:16.701 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:16.688+0000 7f757d4ec640 1 -- 172.21.15.43:0/1044930140 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f75780c4de0 con 0x7f757815a340 2023-08-09T13:36:16.701 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:16.689+0000 7f757d4ec640 1 -- 172.21.15.43:0/1044930140 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f75780c3a60 con 0x7f757815a340 2023-08-09T13:36:16.701 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:16.690+0000 7f7574ff9640 1 -- 172.21.15.43:0/1044930140 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 5) v1 ==== 88341+0+0 (secure 0 0 0) 0x7f757001b460 con 0x7f757815a340 2023-08-09T13:36:16.702 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:16.691+0000 7f7574ff9640 1 --2- 172.21.15.43:0/1044930140 >> [v2:172.21.15.203:6832/830097474,v1:172.21.15.203:6833/830097474] conn(0x7f756406a080 0x7f756406c540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:36:16.702 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:16.691+0000 7f7574ff9640 1 -- 172.21.15.43:0/1044930140 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(39..39 src has 1..39) v4 ==== 6382+0+0 (secure 0 0 0) 0x7f7570088470 con 0x7f757815a340 2023-08-09T13:36:16.702 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:16.691+0000 7f7577fff640 1 --2- 172.21.15.43:0/1044930140 >> [v2:172.21.15.203:6832/830097474,v1:172.21.15.203:6833/830097474] conn(0x7f756406a080 0x7f756406c540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:36:16.702 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:16.693+0000 7f7577fff640 1 --2- 172.21.15.43:0/1044930140 >> [v2:172.21.15.203:6832/830097474,v1:172.21.15.203:6833/830097474] conn(0x7f756406a080 0x7f756406c540 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7f7568035000 tx=0x7f7568030000 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-09T13:36:16.703 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:16.695+0000 7f7574ff9640 1 -- 172.21.15.43:0/1044930140 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178066 (secure 0 0 0) 0x7f7570053ab0 con 0x7f757815a340 2023-08-09T13:36:16.793 INFO:tasks.ceph.mds.c.smithi043.stderr:2023-08-09T13:36:16.791+0000 7f4cda0a3640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2023-08-09T13:36:16.899 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:16.897+0000 7f757d4ec640 1 -- 172.21.15.43:0/1044930140 --> [v2:172.21.15.203:6832/830097474,v1:172.21.15.203:6833/830097474] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f757815bf40 con 0x7f756406a080 2023-08-09T13:36:16.923 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:16.922+0000 7f7574ff9640 1 -- 172.21.15.43:0/1044930140 <== mgr.4114 v2:172.21.15.203:6832/830097474 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+309248 (secure 0 0 0) 0x7f757815bf40 con 0x7f756406a080 2023-08-09T13:36:16.929 INFO:teuthology.orchestra.run.smithi043.stdout: 2023-08-09T13:36:16.931 INFO:teuthology.orchestra.run.smithi043.stderr:dumped all 2023-08-09T13:36:16.934 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:16.927+0000 7f757d4ec640 1 -- 172.21.15.43:0/1044930140 >> [v2:172.21.15.203:6832/830097474,v1:172.21.15.203:6833/830097474] conn(0x7f756406a080 msgr2=0x7f756406c540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:36:16.934 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:16.927+0000 7f757d4ec640 1 --2- 172.21.15.43:0/1044930140 >> [v2:172.21.15.203:6832/830097474,v1:172.21.15.203:6833/830097474] conn(0x7f756406a080 0x7f756406c540 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7f7568035000 tx=0x7f7568030000 comp rx=0 tx=0).stop 2023-08-09T13:36:16.934 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:16.927+0000 7f757d4ec640 1 -- 172.21.15.43:0/1044930140 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f757815a340 msgr2=0x7f75780ade50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:36:16.934 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:16.927+0000 7f757d4ec640 1 --2- 172.21.15.43:0/1044930140 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f757815a340 0x7f75780ade50 secure :-1 s=READY pgs=576 cs=0 l=1 rev1=1 crypto rx=0x7f7570009fd0 tx=0x7f757000b8e0 comp rx=0 tx=0).stop 2023-08-09T13:36:16.934 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:16.928+0000 7f757d4ec640 1 -- 172.21.15.43:0/1044930140 shutdown_connections 2023-08-09T13:36:16.935 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:16.928+0000 7f757d4ec640 1 --2- 172.21.15.43:0/1044930140 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f757815a340 0x7f75780ade50 unknown :-1 s=CLOSED pgs=576 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:36:16.935 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:16.928+0000 7f757d4ec640 1 --2- 172.21.15.43:0/1044930140 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f75780ce620 0x7f75780c48a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:36:16.935 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:16.928+0000 7f757d4ec640 1 --2- 172.21.15.43:0/1044930140 >> [v2:172.21.15.203:6832/830097474,v1:172.21.15.203:6833/830097474] conn(0x7f756406a080 0x7f756406c540 unknown :-1 s=CLOSED pgs=278 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:36:16.936 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:16.928+0000 7f757d4ec640 1 --2- 172.21.15.43:0/1044930140 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f75780cb1a0 0x7f7578152a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:36:16.936 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:16.928+0000 7f757d4ec640 1 -- 172.21.15.43:0/1044930140 >> 172.21.15.43:0/1044930140 conn(0x7f7578089fe0 msgr2=0x7f757806d960 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-09T13:36:16.936 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:16.928+0000 7f757d4ec640 1 -- 172.21.15.43:0/1044930140 shutdown_connections 2023-08-09T13:36:16.936 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:16.928+0000 7f757d4ec640 1 -- 172.21.15.43:0/1044930140 wait complete. 2023-08-09T13:36:16.949 INFO:teuthology.orchestra.run.smithi043.stdout:{"pg_ready":true,"pg_map":{"version":104,"stamp":"2023-08-09T13:36:16.814779+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":23,"num_object_clones":0,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":203,"num_write_kb":2430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":155,"ondisk_log_size":155,"up":210,"acting":210,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":192,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":241392,"kb_used_data":4736,"kb_used_omap":13,"kb_used_meta":236530,"kb_avail":754733328,"statfs":{"total":773094113280,"available":772846927872,"internally_reserved":0,"allocated":4849664,"data_stored":2811674,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":14248,"internal_metadata":242206808},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":36,"apply_latency_ms":36,"commit_latency_ns":36000000,"apply_latency_ns":36000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.002005"},"pg_stats":[{"pgid":"7.3f","version":"0'0","reported_seq":11,"reported_epoch":39,"state":"active+clean","last_fresh":"2023-08-09T13:36:13.181426+0000","last_change":"2023-08-09T13:36:11.765333+0000","last_active":"2023-08-09T13:36:13.181426+0000","last_peered":"2023-08-09T13:36:13.181426+0000","last_clean":"2023-08-09T13:36:13.181426+0000","last_became_active":"2023-08-09T13:36:11.764961+0000","last_became_peered":"2023-08-09T13:36:11.764961+0000","last_unstale":"2023-08-09T13:36:13.181426+0000","last_undegraded":"2023-08-09T13:36:13.181426+0000","last_fullsized":"2023-08-09T13:36:13.181426+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:36:10.826860+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:36:10.826860+0000","last_clean_scrub_stamp":"2023-08-09T13:36:10.826860+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T21:17:28.886040+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.3e","version":"0'0","reported_seq":11,"reported_epoch":39,"state":"active+clean","last_fresh":"2023-08-09T13:36:13.180692+0000","last_change":"2023-08-09T13:36:11.769972+0000","last_active":"2023-08-09T13:36:13.180692+0000","last_peered":"2023-08-09T13:36:13.180692+0000","last_clean":"2023-08-09T13:36:13.180692+0000","last_became_active":"2023-08-09T13:36:11.769710+0000","last_became_peered":"2023-08-09T13:36:11.769710+0000","last_unstale":"2023-08-09T13:36:13.180692+0000","last_undegraded":"2023-08-09T13:36:13.180692+0000","last_fullsized":"2023-08-09T13:36:13.180692+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:36:10.826860+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:36:10.826860+0000","last_clean_scrub_stamp":"2023-08-09T13:36:10.826860+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T17:14:05.980609+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.3d","version":"0'0","reported_seq":11,"reported_epoch":39,"state":"active+clean","last_fresh":"2023-08-09T13:36:12.745747+0000","last_change":"2023-08-09T13:36:11.771122+0000","last_active":"2023-08-09T13:36:12.745747+0000","last_peered":"2023-08-09T13:36:12.745747+0000","last_clean":"2023-08-09T13:36:12.745747+0000","last_became_active":"2023-08-09T13:36:11.770819+0000","last_became_peered":"2023-08-09T13:36:11.770819+0000","last_unstale":"2023-08-09T13:36:12.745747+0000","last_undegraded":"2023-08-09T13:36:12.745747+0000","last_fullsized":"2023-08-09T13:36:12.745747+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:36:10.826860+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:36:10.826860+0000","last_clean_scrub_stamp":"2023-08-09T13:36:10.826860+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T20:43:03.944312+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.3c","version":"0'0","reported_seq":11,"reported_epoch":39,"state":"active+clean","last_fresh":"2023-08-09T13:36:13.180214+0000","last_change":"2023-08-09T13:36:11.778822+0000","last_active":"2023-08-09T13:36:13.180214+0000","last_peered":"2023-08-09T13:36:13.180214+0000","last_clean":"2023-08-09T13:36:13.180214+0000","last_became_active":"2023-08-09T13:36:11.778544+0000","last_became_peered":"2023-08-09T13:36:11.778544+0000","last_unstale":"2023-08-09T13:36:13.180214+0000","last_undegraded":"2023-08-09T13:36:13.180214+0000","last_fullsized":"2023-08-09T13:36:13.180214+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:36:10.826860+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:36:10.826860+0000","last_clean_scrub_stamp":"2023-08-09T13:36:10.826860+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-11T00:30:46.542672+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.3b","version":"0'0","reported_seq":11,"reported_epoch":39,"state":"active+clean","last_fresh":"2023-08-09T13:36:13.180730+0000","last_change":"2023-08-09T13:36:11.765844+0000","last_active":"2023-08-09T13:36:13.180730+0000","last_peered":"2023-08-09T13:36:13.180730+0000","last_clean":"2023-08-09T13:36:13.180730+0000","last_became_active":"2023-08-09T13:36:11.765565+0000","last_became_peered":"2023-08-09T13:36:11.765565+0000","last_unstale":"2023-08-09T13:36:13.180730+0000","last_undegraded":"2023-08-09T13:36:13.180730+0000","last_fullsized":"2023-08-09T13:36:13.180730+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:36:10.826860+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:36:10.826860+0000","last_clean_scrub_stamp":"2023-08-09T13:36:10.826860+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T17:14:43.547736+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.3a","version":"0'0","reported_seq":11,"reported_epoch":39,"state":"active+clean","last_fresh":"2023-08-09T13:36:13.182081+0000","last_change":"2023-08-09T13:36:11.768348+0000","last_active":"2023-08-09T13:36:13.182081+0000","last_peered":"2023-08-09T13:36:13.182081+0000","last_clean":"2023-08-09T13:36:13.182081+0000","last_became_active":"2023-08-09T13:36:11.768017+0000","last_became_peered":"2023-08-09T13:36:11.768017+0000","last_unstale":"2023-08-09T13:36:13.182081+0000","last_undegraded":"2023-08-09T13:36:13.182081+0000","last_fullsized":"2023-08-09T13:36:13.182081+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:36:10.826860+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:36:10.826860+0000","last_clean_scrub_stamp":"2023-08-09T13:36:10.826860+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T17:09:17.685613+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.39","version":"0'0","reported_seq":11,"reported_epoch":39,"state":"active+clean","last_fresh":"2023-08-09T13:36:13.179893+0000","last_change":"2023-08-09T13:36:11.770182+0000","last_active":"2023-08-09T13:36:13.179893+0000","last_peered":"2023-08-09T13:36:13.179893+0000","last_clean":"2023-08-09T13:36:13.179893+0000","last_became_active":"2023-08-09T13:36:11.769887+0000","last_became_peered":"2023-08-09T13:36:11.769887+0000","last_unstale":"2023-08-09T13:36:13.179893+0000","last_undegraded":"2023-08-09T13:36:13.179893+0000","last_fullsized":"2023-08-09T13:36:13.179893+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:36:10.826860+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:36:10.826860+0000","last_clean_scrub_stamp":"2023-08-09T13:36:10.826860+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T19:22:11.248322+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.38","version":"0'0","reported_seq":11,"reported_epoch":39,"state":"active+clean","last_fresh":"2023-08-09T13:36:13.178307+0000","last_change":"2023-08-09T13:36:11.778128+0000","last_active":"2023-08-09T13:36:13.178307+0000","last_peered":"2023-08-09T13:36:13.178307+0000","last_clean":"2023-08-09T13:36:13.178307+0000","last_became_active":"2023-08-09T13:36:11.777617+0000","last_became_peered":"2023-08-09T13:36:11.777617+0000","last_unstale":"2023-08-09T13:36:13.178307+0000","last_undegraded":"2023-08-09T13:36:13.178307+0000","last_fullsized":"2023-08-09T13:36:13.178307+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:36:10.826860+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:36:10.826860+0000","last_clean_scrub_stamp":"2023-08-09T13:36:10.826860+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T21:41:28.535846+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.37","version":"0'0","reported_seq":11,"reported_epoch":39,"state":"active+clean","last_fresh":"2023-08-09T13:36:13.180968+0000","last_change":"2023-08-09T13:36:11.774608+0000","last_active":"2023-08-09T13:36:13.180968+0000","last_peered":"2023-08-09T13:36:13.180968+0000","last_clean":"2023-08-09T13:36:13.180968+0000","last_became_active":"2023-08-09T13:36:11.774333+0000","last_became_peered":"2023-08-09T13:36:11.774333+0000","last_unstale":"2023-08-09T13:36:13.180968+0000","last_undegraded":"2023-08-09T13:36:13.180968+0000","last_fullsized":"2023-08-09T13:36:13.180968+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:36:10.826860+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:36:10.826860+0000","last_clean_scrub_stamp":"2023-08-09T13:36:10.826860+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T16:31:40.424973+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.36","version":"0'0","reported_seq":11,"reported_epoch":39,"state":"active+clean","last_fresh":"2023-08-09T13:36:13.180568+0000","last_change":"2023-08-09T13:36:11.780966+0000","last_active":"2023-08-09T13:36:13.180568+0000","last_peered":"2023-08-09T13:36:13.180568+0000","last_clean":"2023-08-09T13:36:13.180568+0000","last_became_active":"2023-08-09T13:36:11.780610+0000","last_became_peered":"2023-08-09T13:36:11.780610+0000","last_unstale":"2023-08-09T13:36:13.180568+0000","last_undegraded":"2023-08-09T13:36:13.180568+0000","last_fullsized":"2023-08-09T13:36:13.180568+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:36:10.826860+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:36:10.826860+0000","last_clean_scrub_stamp":"2023-08-09T13:36:10.826860+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T16:27:53.685325+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.35","version":"0'0","reported_seq":11,"reported_epoch":39,"state":"active+clean","last_fresh":"2023-08-09T13:36:13.180041+0000","last_change":"2023-08-09T13:36:11.769206+0000","last_active":"2023-08-09T13:36:13.180041+0000","last_peered":"2023-08-09T13:36:13.180041+0000","last_clean":"2023-08-09T13:36:13.180041+0000","last_became_active":"2023-08-09T13:36:11.768753+0000","last_became_peered":"2023-08-09T13:36:11.768753+0000","last_unstale":"2023-08-09T13:36:13.180041+0000","last_undegraded":"2023-08-09T13:36:13.180041+0000","last_fullsized":"2023-08-09T13:36:13.180041+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:36:10.826860+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:36:10.826860+0000","last_clean_scrub_stamp":"2023-08-09T13:36:10.826860+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T20:21:26.027735+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.34","version":"0'0","reported_seq":11,"reported_epoch":39,"state":"active+clean","last_fresh":"2023-08-09T13:36:13.180449+0000","last_change":"2023-08-09T13:36:11.768191+0000","last_active":"2023-08-09T13:36:13.180449+0000","last_peered":"2023-08-09T13:36:13.180449+0000","last_clean":"2023-08-09T13:36:13.180449+0000","last_became_active":"2023-08-09T13:36:11.767877+0000","last_became_peered":"2023-08-09T13:36:11.767877+0000","last_unstale":"2023-08-09T13:36:13.180449+0000","last_undegraded":"2023-08-09T13:36:13.180449+0000","last_fullsized":"2023-08-09T13:36:13.180449+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:36:10.826860+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:36:10.826860+0000","last_clean_scrub_stamp":"2023-08-09T13:36:10.826860+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T18:21:44.653185+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.33","version":"0'0","reported_seq":11,"reported_epoch":39,"state":"active+clean","last_fresh":"2023-08-09T13:36:13.181117+0000","last_change":"2023-08-09T13:36:11.768006+0000","last_active":"2023-08-09T13:36:13.181117+0000","last_peered":"2023-08-09T13:36:13.181117+0000","last_clean":"2023-08-09T13:36:13.181117+0000","last_became_active":"2023-08-09T13:36:11.767631+0000","last_became_peered":"2023-08-09T13:36:11.767631+0000","last_unstale":"2023-08-09T13:36:13.181117+0000","last_undegraded":"2023-08-09T13:36:13.181117+0000","last_fullsized":"2023-08-09T13:36:13.181117+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:36:10.826860+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:36:10.826860+0000","last_clean_scrub_stamp":"2023-08-09T13:36:10.826860+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T18:12:36.838776+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.32","version":"0'0","reported_seq":11,"reported_epoch":39,"state":"active+clean","last_fresh":"2023-08-09T13:36:13.179228+0000","last_change":"2023-08-09T13:36:11.766755+0000","last_active":"2023-08-09T13:36:13.179228+0000","last_peered":"2023-08-09T13:36:13.179228+0000","last_clean":"2023-08-09T13:36:13.179228+0000","last_became_active":"2023-08-09T13:36:11.766478+0000","last_became_peered":"2023-08-09T13:36:11.766478+0000","last_unstale":"2023-08-09T13:36:13.179228+0000","last_undegraded":"2023-08-09T13:36:13.179228+0000","last_fullsized":"2023-08-09T13:36:13.179228+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:36:10.826860+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:36:10.826860+0000","last_clean_scrub_stamp":"2023-08-09T13:36:10.826860+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T13:52:40.766251+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.31","version":"0'0","reported_seq":11,"reported_epoch":39,"state":"active+clean","last_fresh":"2023-08-09T13:36:13.182174+0000","last_change":"2023-08-09T13:36:11.766564+0000","last_active":"2023-08-09T13:36:13.182174+0000","last_peered":"2023-08-09T13:36:13.182174+0000","last_clean":"2023-08-09T13:36:13.182174+0000","last_became_active":"2023-08-09T13:36:11.766273+0000","last_became_peered":"2023-08-09T13:36:11.766273+0000","last_unstale":"2023-08-09T13:36:13.182174+0000","last_undegraded":"2023-08-09T13:36:13.182174+0000","last_fullsized":"2023-08-09T13:36:13.182174+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:36:10.826860+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:36:10.826860+0000","last_clean_scrub_stamp":"2023-08-09T13:36:10.826860+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T15:46:07.852527+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.30","version":"0'0","reported_seq":11,"reported_epoch":39,"state":"active+clean","last_fresh":"2023-08-09T13:36:12.745527+0000","last_change":"2023-08-09T13:36:11.779982+0000","last_active":"2023-08-09T13:36:12.745527+0000","last_peered":"2023-08-09T13:36:12.745527+0000","last_clean":"2023-08-09T13:36:12.745527+0000","last_became_active":"2023-08-09T13:36:11.779622+0000","last_became_peered":"2023-08-09T13:36:11.779622+0000","last_unstale":"2023-08-09T13:36:12.745527+0000","last_undegraded":"2023-08-09T13:36:12.745527+0000","last_fullsized":"2023-08-09T13:36:12.745527+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:36:10.826860+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:36:10.826860+0000","last_clean_scrub_stamp":"2023-08-09T13:36:10.826860+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T20:01:21.335013+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.2f","version":"0'0","reported_seq":11,"reported_epoch":39,"state":"active+clean","last_fresh":"2023-08-09T13:36:12.746947+0000","last_change":"2023-08-09T13:36:11.764541+0000","last_active":"2023-08-09T13:36:12.746947+0000","last_peered":"2023-08-09T13:36:12.746947+0000","last_clean":"2023-08-09T13:36:12.746947+0000","last_became_active":"2023-08-09T13:36:11.764200+0000","last_became_peered":"2023-08-09T13:36:11.764200+0000","last_unstale":"2023-08-09T13:36:12.746947+0000","last_undegraded":"2023-08-09T13:36:12.746947+0000","last_fullsized":"2023-08-09T13:36:12.746947+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:36:10.826860+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:36:10.826860+0000","last_clean_scrub_stamp":"2023-08-09T13:36:10.826860+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T21:47:36.132042+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.2e","version":"0'0","reported_seq":11,"reported_epoch":39,"state":"active+clean","last_fresh":"2023-08-09T13:36:12.746801+0000","last_change":"2023-08-09T13:36:11.769432+0000","last_active":"2023-08-09T13:36:12.746801+0000","last_peered":"2023-08-09T13:36:12.746801+0000","last_clean":"2023-08-09T13:36:12.746801+0000","last_became_active":"2023-08-09T13:36:11.768929+0000","last_became_peered":"2023-08-09T13:36:11.768929+0000","last_unstale":"2023-08-09T13:36:12.746801+0000","last_undegraded":"2023-08-09T13:36:12.746801+0000","last_fullsized":"2023-08-09T13:36:12.746801+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:36:10.826860+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:36:10.826860+0000","last_clean_scrub_stamp":"2023-08-09T13:36:10.826860+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T22:26:28.862983+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.2d","version":"0'0","reported_seq":11,"reported_epoch":39,"state":"active+clean","last_fresh":"2023-08-09T13:36:12.748003+0000","last_change":"2023-08-09T13:36:11.781277+0000","last_active":"2023-08-09T13:36:12.748003+0000","last_peered":"2023-08-09T13:36:12.748003+0000","last_clean":"2023-08-09T13:36:12.748003+0000","last_became_active":"2023-08-09T13:36:11.780880+0000","last_became_peered":"2023-08-09T13:36:11.780880+0000","last_unstale":"2023-08-09T13:36:12.748003+0000","last_undegraded":"2023-08-09T13:36:12.748003+0000","last_fullsized":"2023-08-09T13:36:12.748003+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:36:10.826860+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:36:10.826860+0000","last_clean_scrub_stamp":"2023-08-09T13:36:10.826860+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T15:14:19.445235+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.2c","version":"0'0","reported_seq":11,"reported_epoch":39,"state":"active+clean","last_fresh":"2023-08-09T13:36:13.180147+0000","last_change":"2023-08-09T13:36:11.769233+0000","last_active":"2023-08-09T13:36:13.180147+0000","last_peered":"2023-08-09T13:36:13.180147+0000","last_clean":"2023-08-09T13:36:13.180147+0000","last_became_active":"2023-08-09T13:36:11.768953+0000","last_became_peered":"2023-08-09T13:36:11.768953+0000","last_unstale":"2023-08-09T13:36:13.180147+0000","last_undegraded":"2023-08-09T13:36:13.180147+0000","last_fullsized":"2023-08-09T13:36:13.180147+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:36:10.826860+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:36:10.826860+0000","last_clean_scrub_stamp":"2023-08-09T13:36:10.826860+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T21:29:25.842401+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.2b","version":"0'0","reported_seq":11,"reported_epoch":39,"state":"active+clean","last_fresh":"2023-08-09T13:36:13.182610+0000","last_change":"2023-08-09T13:36:11.776832+0000","last_active":"2023-08-09T13:36:13.182610+0000","last_peered":"2023-08-09T13:36:13.182610+0000","last_clean":"2023-08-09T13:36:13.182610+0000","last_became_active":"2023-08-09T13:36:11.776502+0000","last_became_peered":"2023-08-09T13:36:11.776502+0000","last_unstale":"2023-08-09T13:36:13.182610+0000","last_undegraded":"2023-08-09T13:36:13.182610+0000","last_fullsized":"2023-08-09T13:36:13.182610+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:36:10.826860+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:36:10.826860+0000","last_clean_scrub_stamp":"2023-08-09T13:36:10.826860+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T14:36:24.296741+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.2a","version":"0'0","reported_seq":11,"reported_epoch":39,"state":"active+clean","last_fresh":"2023-08-09T13:36:13.179357+0000","last_change":"2023-08-09T13:36:11.780161+0000","last_active":"2023-08-09T13:36:13.179357+0000","last_peered":"2023-08-09T13:36:13.179357+0000","last_clean":"2023-08-09T13:36:13.179357+0000","last_became_active":"2023-08-09T13:36:11.779797+0000","last_became_peered":"2023-08-09T13:36:11.779797+0000","last_unstale":"2023-08-09T13:36:13.179357+0000","last_undegraded":"2023-08-09T13:36:13.179357+0000","last_fullsized":"2023-08-09T13:36:13.179357+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:36:10.826860+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:36:10.826860+0000","last_clean_scrub_stamp":"2023-08-09T13:36:10.826860+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T23:26:23.964084+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.29","version":"0'0","reported_seq":11,"reported_epoch":39,"state":"active+clean","last_fresh":"2023-08-09T13:36:12.746280+0000","last_change":"2023-08-09T13:36:11.773991+0000","last_active":"2023-08-09T13:36:12.746280+0000","last_peered":"2023-08-09T13:36:12.746280+0000","last_clean":"2023-08-09T13:36:12.746280+0000","last_became_active":"2023-08-09T13:36:11.773668+0000","last_became_peered":"2023-08-09T13:36:11.773668+0000","last_unstale":"2023-08-09T13:36:12.746280+0000","last_undegraded":"2023-08-09T13:36:12.746280+0000","last_fullsized":"2023-08-09T13:36:12.746280+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:36:10.826860+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:36:10.826860+0000","last_clean_scrub_stamp":"2023-08-09T13:36:10.826860+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T17:26:42.727199+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.28","version":"0'0","reported_seq":11,"reported_epoch":39,"state":"active+clean","last_fresh":"2023-08-09T13:36:12.745684+0000","last_change":"2023-08-09T13:36:11.771543+0000","last_active":"2023-08-09T13:36:12.745684+0000","last_peered":"2023-08-09T13:36:12.745684+0000","last_clean":"2023-08-09T13:36:12.745684+0000","last_became_active":"2023-08-09T13:36:11.771246+0000","last_became_peered":"2023-08-09T13:36:11.771246+0000","last_unstale":"2023-08-09T13:36:12.745684+0000","last_undegraded":"2023-08-09T13:36:12.745684+0000","last_fullsized":"2023-08-09T13:36:12.745684+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:36:10.826860+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:36:10.826860+0000","last_clean_scrub_stamp":"2023-08-09T13:36:10.826860+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T18:13:08.572943+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.27","version":"0'0","reported_seq":11,"reported_epoch":39,"state":"active+clean","last_fresh":"2023-08-09T13:36:12.748356+0000","last_change":"2023-08-09T13:36:11.774594+0000","last_active":"2023-08-09T13:36:12.748356+0000","last_peered":"2023-08-09T13:36:12.748356+0000","last_clean":"2023-08-09T13:36:12.748356+0000","last_became_active":"2023-08-09T13:36:11.774122+0000","last_became_peered":"2023-08-09T13:36:11.774122+0000","last_unstale":"2023-08-09T13:36:12.748356+0000","last_undegraded":"2023-08-09T13:36:12.748356+0000","last_fullsized":"2023-08-09T13:36:12.748356+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:36:10.826860+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:36:10.826860+0000","last_clean_scrub_stamp":"2023-08-09T13:36:10.826860+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T23:55:48.467147+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.26","version":"0'0","reported_seq":11,"reported_epoch":39,"state":"active+clean","last_fresh":"2023-08-09T13:36:13.181406+0000","last_change":"2023-08-09T13:36:11.766030+0000","last_active":"2023-08-09T13:36:13.181406+0000","last_peered":"2023-08-09T13:36:13.181406+0000","last_clean":"2023-08-09T13:36:13.181406+0000","last_became_active":"2023-08-09T13:36:11.764234+0000","last_became_peered":"2023-08-09T13:36:11.764234+0000","last_unstale":"2023-08-09T13:36:13.181406+0000","last_undegraded":"2023-08-09T13:36:13.181406+0000","last_fullsized":"2023-08-09T13:36:13.181406+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:36:10.826860+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:36:10.826860+0000","last_clean_scrub_stamp":"2023-08-09T13:36:10.826860+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T13:56:38.251585+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.25","version":"0'0","reported_seq":11,"reported_epoch":39,"state":"active+clean","last_fresh":"2023-08-09T13:36:12.746240+0000","last_change":"2023-08-09T13:36:11.764663+0000","last_active":"2023-08-09T13:36:12.746240+0000","last_peered":"2023-08-09T13:36:12.746240+0000","last_clean":"2023-08-09T13:36:12.746240+0000","last_became_active":"2023-08-09T13:36:11.764280+0000","last_became_peered":"2023-08-09T13:36:11.764280+0000","last_unstale":"2023-08-09T13:36:12.746240+0000","last_undegraded":"2023-08-09T13:36:12.746240+0000","last_fullsized":"2023-08-09T13:36:12.746240+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:36:10.826860+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:36:10.826860+0000","last_clean_scrub_stamp":"2023-08-09T13:36:10.826860+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T21:26:15.393102+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.24","version":"0'0","reported_seq":11,"reported_epoch":39,"state":"active+clean","last_fresh":"2023-08-09T13:36:13.180557+0000","last_change":"2023-08-09T13:36:11.768283+0000","last_active":"2023-08-09T13:36:13.180557+0000","last_peered":"2023-08-09T13:36:13.180557+0000","last_clean":"2023-08-09T13:36:13.180557+0000","last_became_active":"2023-08-09T13:36:11.767964+0000","last_became_peered":"2023-08-09T13:36:11.767964+0000","last_unstale":"2023-08-09T13:36:13.180557+0000","last_undegraded":"2023-08-09T13:36:13.180557+0000","last_fullsized":"2023-08-09T13:36:13.180557+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:36:10.826860+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:36:10.826860+0000","last_clean_scrub_stamp":"2023-08-09T13:36:10.826860+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T21:41:28.535846+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.23","version":"0'0","reported_seq":11,"reported_epoch":39,"state":"active+clean","last_fresh":"2023-08-09T13:36:12.747274+0000","last_change":"2023-08-09T13:36:11.773274+0000","last_active":"2023-08-09T13:36:12.747274+0000","last_peered":"2023-08-09T13:36:12.747274+0000","last_clean":"2023-08-09T13:36:12.747274+0000","last_became_active":"2023-08-09T13:36:11.772906+0000","last_became_peered":"2023-08-09T13:36:11.772906+0000","last_unstale":"2023-08-09T13:36:12.747274+0000","last_undegraded":"2023-08-09T13:36:12.747274+0000","last_fullsized":"2023-08-09T13:36:12.747274+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:36:10.826860+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:36:10.826860+0000","last_clean_scrub_stamp":"2023-08-09T13:36:10.826860+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T23:28:53.092968+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.22","version":"0'0","reported_seq":11,"reported_epoch":39,"state":"active+clean","last_fresh":"2023-08-09T13:36:12.744611+0000","last_change":"2023-08-09T13:36:11.763747+0000","last_active":"2023-08-09T13:36:12.744611+0000","last_peered":"2023-08-09T13:36:12.744611+0000","last_clean":"2023-08-09T13:36:12.744611+0000","last_became_active":"2023-08-09T13:36:11.763448+0000","last_became_peered":"2023-08-09T13:36:11.763448+0000","last_unstale":"2023-08-09T13:36:12.744611+0000","last_undegraded":"2023-08-09T13:36:12.744611+0000","last_fullsized":"2023-08-09T13:36:12.744611+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:36:10.826860+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:36:10.826860+0000","last_clean_scrub_stamp":"2023-08-09T13:36:10.826860+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T23:59:15.283205+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.21","version":"0'0","reported_seq":11,"reported_epoch":39,"state":"active+clean","last_fresh":"2023-08-09T13:36:13.182242+0000","last_change":"2023-08-09T13:36:11.773054+0000","last_active":"2023-08-09T13:36:13.182242+0000","last_peered":"2023-08-09T13:36:13.182242+0000","last_clean":"2023-08-09T13:36:13.182242+0000","last_became_active":"2023-08-09T13:36:11.772568+0000","last_became_peered":"2023-08-09T13:36:11.772568+0000","last_unstale":"2023-08-09T13:36:13.182242+0000","last_undegraded":"2023-08-09T13:36:13.182242+0000","last_fullsized":"2023-08-09T13:36:13.182242+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:36:10.826860+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:36:10.826860+0000","last_clean_scrub_stamp":"2023-08-09T13:36:10.826860+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T19:30:04.078025+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.20","version":"0'0","reported_seq":11,"reported_epoch":39,"state":"active+clean","last_fresh":"2023-08-09T13:36:13.179580+0000","last_change":"2023-08-09T13:36:11.778008+0000","last_active":"2023-08-09T13:36:13.179580+0000","last_peered":"2023-08-09T13:36:13.179580+0000","last_clean":"2023-08-09T13:36:13.179580+0000","last_became_active":"2023-08-09T13:36:11.777724+0000","last_became_peered":"2023-08-09T13:36:11.777724+0000","last_unstale":"2023-08-09T13:36:13.179580+0000","last_undegraded":"2023-08-09T13:36:13.179580+0000","last_fullsized":"2023-08-09T13:36:13.179580+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:36:10.826860+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:36:10.826860+0000","last_clean_scrub_stamp":"2023-08-09T13:36:10.826860+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T20:02:36.013365+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.1e","version":"0'0","reported_seq":11,"reported_epoch":39,"state":"active+clean","last_fresh":"2023-08-09T13:36:12.743763+0000","last_change":"2023-08-09T13:36:11.767917+0000","last_active":"2023-08-09T13:36:12.743763+0000","last_peered":"2023-08-09T13:36:12.743763+0000","last_clean":"2023-08-09T13:36:12.743763+0000","last_became_active":"2023-08-09T13:36:11.767617+0000","last_became_peered":"2023-08-09T13:36:11.767617+0000","last_unstale":"2023-08-09T13:36:12.743763+0000","last_undegraded":"2023-08-09T13:36:12.743763+0000","last_fullsized":"2023-08-09T13:36:12.743763+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:36:10.826860+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:36:10.826860+0000","last_clean_scrub_stamp":"2023-08-09T13:36:10.826860+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T23:39:34.789487+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1f","version":"38'4","reported_seq":17,"reported_epoch":39,"state":"active+clean","last_fresh":"2023-08-09T13:36:13.183924+0000","last_change":"2023-08-09T13:36:10.877706+0000","last_active":"2023-08-09T13:36:13.183924+0000","last_peered":"2023-08-09T13:36:13.183924+0000","last_clean":"2023-08-09T13:36:13.183924+0000","last_became_active":"2023-08-09T13:36:10.877403+0000","last_became_peered":"2023-08-09T13:36:10.877403+0000","last_unstale":"2023-08-09T13:36:13.183924+0000","last_undegraded":"2023-08-09T13:36:13.183924+0000","last_fullsized":"2023-08-09T13:36:13.183924+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:36:09.813580+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:36:09.813580+0000","last_clean_scrub_stamp":"2023-08-09T13:36:09.813580+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T19:23:49.018232+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.1f","version":"0'0","reported_seq":11,"reported_epoch":39,"state":"active+clean","last_fresh":"2023-08-09T13:36:12.747755+0000","last_change":"2023-08-09T13:36:11.774093+0000","last_active":"2023-08-09T13:36:12.747755+0000","last_peered":"2023-08-09T13:36:12.747755+0000","last_clean":"2023-08-09T13:36:12.747755+0000","last_became_active":"2023-08-09T13:36:11.773206+0000","last_became_peered":"2023-08-09T13:36:11.773206+0000","last_unstale":"2023-08-09T13:36:12.747755+0000","last_undegraded":"2023-08-09T13:36:12.747755+0000","last_fullsized":"2023-08-09T13:36:12.747755+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:36:10.826860+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:36:10.826860+0000","last_clean_scrub_stamp":"2023-08-09T13:36:10.826860+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T16:13:39.119652+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.1e","version":"38'1","reported_seq":14,"reported_epoch":39,"state":"active+clean","last_fresh":"2023-08-09T13:36:12.747024+0000","last_change":"2023-08-09T13:36:10.878238+0000","last_active":"2023-08-09T13:36:12.747024+0000","last_peered":"2023-08-09T13:36:12.747024+0000","last_clean":"2023-08-09T13:36:12.747024+0000","last_became_active":"2023-08-09T13:36:10.877926+0000","last_became_peered":"2023-08-09T13:36:10.877926+0000","last_unstale":"2023-08-09T13:36:12.747024+0000","last_undegraded":"2023-08-09T13:36:12.747024+0000","last_fullsized":"2023-08-09T13:36:12.747024+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:36:09.813580+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:36:09.813580+0000","last_clean_scrub_stamp":"2023-08-09T13:36:09.813580+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T19:34:51.030091+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.1c","version":"0'0","reported_seq":11,"reported_epoch":39,"state":"active+clean","last_fresh":"2023-08-09T13:36:13.180813+0000","last_change":"2023-08-09T13:36:11.764825+0000","last_active":"2023-08-09T13:36:13.180813+0000","last_peered":"2023-08-09T13:36:13.180813+0000","last_clean":"2023-08-09T13:36:13.180813+0000","last_became_active":"2023-08-09T13:36:11.764483+0000","last_became_peered":"2023-08-09T13:36:11.764483+0000","last_unstale":"2023-08-09T13:36:13.180813+0000","last_undegraded":"2023-08-09T13:36:13.180813+0000","last_fullsized":"2023-08-09T13:36:13.180813+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:36:10.826860+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:36:10.826860+0000","last_clean_scrub_stamp":"2023-08-09T13:36:10.826860+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T18:10:39.629916+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.1d","version":"38'2","reported_seq":15,"reported_epoch":39,"state":"active+clean","last_fresh":"2023-08-09T13:36:12.743758+0000","last_change":"2023-08-09T13:36:10.866053+0000","last_active":"2023-08-09T13:36:12.743758+0000","last_peered":"2023-08-09T13:36:12.743758+0000","last_clean":"2023-08-09T13:36:12.743758+0000","last_became_active":"2023-08-09T13:36:10.865665+0000","last_became_peered":"2023-08-09T13:36:10.865665+0000","last_unstale":"2023-08-09T13:36:12.743758+0000","last_undegraded":"2023-08-09T13:36:12.743758+0000","last_fullsized":"2023-08-09T13:36:12.743758+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:36:09.813580+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:36:09.813580+0000","last_clean_scrub_stamp":"2023-08-09T13:36:09.813580+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T16:18:22.253312+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.1d","version":"0'0","reported_seq":11,"reported_epoch":39,"state":"active+clean","last_fresh":"2023-08-09T13:36:12.743357+0000","last_change":"2023-08-09T13:36:11.778816+0000","last_active":"2023-08-09T13:36:12.743357+0000","last_peered":"2023-08-09T13:36:12.743357+0000","last_clean":"2023-08-09T13:36:12.743357+0000","last_became_active":"2023-08-09T13:36:11.778521+0000","last_became_peered":"2023-08-09T13:36:11.778521+0000","last_unstale":"2023-08-09T13:36:12.743357+0000","last_undegraded":"2023-08-09T13:36:12.743357+0000","last_fullsized":"2023-08-09T13:36:12.743357+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:36:10.826860+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:36:10.826860+0000","last_clean_scrub_stamp":"2023-08-09T13:36:10.826860+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T20:03:23.736139+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1c","version":"38'2","reported_seq":15,"reported_epoch":39,"state":"active+clean","last_fresh":"2023-08-09T13:36:13.181614+0000","last_change":"2023-08-09T13:36:10.876828+0000","last_active":"2023-08-09T13:36:13.181614+0000","last_peered":"2023-08-09T13:36:13.181614+0000","last_clean":"2023-08-09T13:36:13.181614+0000","last_became_active":"2023-08-09T13:36:10.876565+0000","last_became_peered":"2023-08-09T13:36:10.876565+0000","last_unstale":"2023-08-09T13:36:13.181614+0000","last_undegraded":"2023-08-09T13:36:13.181614+0000","last_fullsized":"2023-08-09T13:36:13.181614+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:36:09.813580+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:36:09.813580+0000","last_clean_scrub_stamp":"2023-08-09T13:36:09.813580+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T18:12:04.598648+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.1a","version":"0'0","reported_seq":11,"reported_epoch":39,"state":"active+clean","last_fresh":"2023-08-09T13:36:12.744691+0000","last_change":"2023-08-09T13:36:11.778261+0000","last_active":"2023-08-09T13:36:12.744691+0000","last_peered":"2023-08-09T13:36:12.744691+0000","last_clean":"2023-08-09T13:36:12.744691+0000","last_became_active":"2023-08-09T13:36:11.777965+0000","last_became_peered":"2023-08-09T13:36:11.777965+0000","last_unstale":"2023-08-09T13:36:12.744691+0000","last_undegraded":"2023-08-09T13:36:12.744691+0000","last_fullsized":"2023-08-09T13:36:12.744691+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:36:10.826860+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:36:10.826860+0000","last_clean_scrub_stamp":"2023-08-09T13:36:10.826860+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T21:44:16.043568+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.1b","version":"38'1","reported_seq":14,"reported_epoch":39,"state":"active+clean","last_fresh":"2023-08-09T13:36:13.182981+0000","last_change":"2023-08-09T13:36:10.868079+0000","last_active":"2023-08-09T13:36:13.182981+0000","last_peered":"2023-08-09T13:36:13.182981+0000","last_clean":"2023-08-09T13:36:13.182981+0000","last_became_active":"2023-08-09T13:36:10.867212+0000","last_became_peered":"2023-08-09T13:36:10.867212+0000","last_unstale":"2023-08-09T13:36:13.182981+0000","last_undegraded":"2023-08-09T13:36:13.182981+0000","last_fullsized":"2023-08-09T13:36:13.182981+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:36:09.813580+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:36:09.813580+0000","last_clean_scrub_stamp":"2023-08-09T13:36:09.813580+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T21:20:01.255654+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.1b","version":"0'0","reported_seq":11,"reported_epoch":39,"state":"active+clean","last_fresh":"2023-08-09T13:36:13.179592+0000","last_change":"2023-08-09T13:36:11.770442+0000","last_active":"2023-08-09T13:36:13.179592+0000","last_peered":"2023-08-09T13:36:13.179592+0000","last_clean":"2023-08-09T13:36:13.179592+0000","last_became_active":"2023-08-09T13:36:11.770115+0000","last_became_peered":"2023-08-09T13:36:11.770115+0000","last_unstale":"2023-08-09T13:36:13.179592+0000","last_undegraded":"2023-08-09T13:36:13.179592+0000","last_fullsized":"2023-08-09T13:36:13.179592+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:36:10.826860+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:36:10.826860+0000","last_clean_scrub_stamp":"2023-08-09T13:36:10.826860+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T23:29:32.450147+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.1a","version":"38'1","reported_seq":14,"reported_epoch":39,"state":"active+clean","last_fresh":"2023-08-09T13:36:12.744934+0000","last_change":"2023-08-09T13:36:10.868343+0000","last_active":"2023-08-09T13:36:12.744934+0000","last_peered":"2023-08-09T13:36:12.744934+0000","last_clean":"2023-08-09T13:36:12.744934+0000","last_became_active":"2023-08-09T13:36:10.868069+0000","last_became_peered":"2023-08-09T13:36:10.868069+0000","last_unstale":"2023-08-09T13:36:12.744934+0000","last_undegraded":"2023-08-09T13:36:12.744934+0000","last_fullsized":"2023-08-09T13:36:12.744934+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:36:09.813580+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:36:09.813580+0000","last_clean_scrub_stamp":"2023-08-09T13:36:09.813580+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T15:32:21.128658+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.18","version":"0'0","reported_seq":11,"reported_epoch":39,"state":"active+clean","last_fresh":"2023-08-09T13:36:13.178756+0000","last_change":"2023-08-09T13:36:11.777904+0000","last_active":"2023-08-09T13:36:13.178756+0000","last_peered":"2023-08-09T13:36:13.178756+0000","last_clean":"2023-08-09T13:36:13.178756+0000","last_became_active":"2023-08-09T13:36:11.777127+0000","last_became_peered":"2023-08-09T13:36:11.777127+0000","last_unstale":"2023-08-09T13:36:13.178756+0000","last_undegraded":"2023-08-09T13:36:13.178756+0000","last_fullsized":"2023-08-09T13:36:13.178756+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:36:10.826860+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:36:10.826860+0000","last_clean_scrub_stamp":"2023-08-09T13:36:10.826860+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T20:57:42.024261+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.19","version":"0'0","reported_seq":13,"reported_epoch":39,"state":"active+clean","last_fresh":"2023-08-09T13:36:13.179786+0000","last_change":"2023-08-09T13:36:10.859014+0000","last_active":"2023-08-09T13:36:13.179786+0000","last_peered":"2023-08-09T13:36:13.179786+0000","last_clean":"2023-08-09T13:36:13.179786+0000","last_became_active":"2023-08-09T13:36:10.858570+0000","last_became_peered":"2023-08-09T13:36:10.858570+0000","last_unstale":"2023-08-09T13:36:13.179786+0000","last_undegraded":"2023-08-09T13:36:13.179786+0000","last_fullsized":"2023-08-09T13:36:13.179786+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:36:09.813580+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:36:09.813580+0000","last_clean_scrub_stamp":"2023-08-09T13:36:09.813580+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T14:42:01.156986+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.19","version":"0'0","reported_seq":11,"reported_epoch":39,"state":"active+clean","last_fresh":"2023-08-09T13:36:12.745518+0000","last_change":"2023-08-09T13:36:11.780497+0000","last_active":"2023-08-09T13:36:12.745518+0000","last_peered":"2023-08-09T13:36:12.745518+0000","last_clean":"2023-08-09T13:36:12.745518+0000","last_became_active":"2023-08-09T13:36:11.780226+0000","last_became_peered":"2023-08-09T13:36:11.780226+0000","last_unstale":"2023-08-09T13:36:12.745518+0000","last_undegraded":"2023-08-09T13:36:12.745518+0000","last_fullsized":"2023-08-09T13:36:12.745518+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:36:10.826860+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:36:10.826860+0000","last_clean_scrub_stamp":"2023-08-09T13:36:10.826860+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T13:46:57.087145+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.18","version":"0'0","reported_seq":13,"reported_epoch":39,"state":"active+clean","last_fresh":"2023-08-09T13:36:13.178683+0000","last_change":"2023-08-09T13:36:10.856291+0000","last_active":"2023-08-09T13:36:13.178683+0000","last_peered":"2023-08-09T13:36:13.178683+0000","last_clean":"2023-08-09T13:36:13.178683+0000","last_became_active":"2023-08-09T13:36:10.855922+0000","last_became_peered":"2023-08-09T13:36:10.855922+0000","last_unstale":"2023-08-09T13:36:13.178683+0000","last_undegraded":"2023-08-09T13:36:13.178683+0000","last_fullsized":"2023-08-09T13:36:13.178683+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:36:09.813580+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:36:09.813580+0000","last_clean_scrub_stamp":"2023-08-09T13:36:09.813580+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T17:14:04.967329+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.16","version":"0'0","reported_seq":11,"reported_epoch":39,"state":"active+clean","last_fresh":"2023-08-09T13:36:12.746763+0000","last_change":"2023-08-09T13:36:11.769252+0000","last_active":"2023-08-09T13:36:12.746763+0000","last_peered":"2023-08-09T13:36:12.746763+0000","last_clean":"2023-08-09T13:36:12.746763+0000","last_became_active":"2023-08-09T13:36:11.768571+0000","last_became_peered":"2023-08-09T13:36:11.768571+0000","last_unstale":"2023-08-09T13:36:12.746763+0000","last_undegraded":"2023-08-09T13:36:12.746763+0000","last_fullsized":"2023-08-09T13:36:12.746763+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:36:10.826860+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:36:10.826860+0000","last_clean_scrub_stamp":"2023-08-09T13:36:10.826860+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T23:51:00.601318+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.17","version":"0'0","reported_seq":13,"reported_epoch":39,"state":"active+clean","last_fresh":"2023-08-09T13:36:12.747302+0000","last_change":"2023-08-09T13:36:10.870854+0000","last_active":"2023-08-09T13:36:12.747302+0000","last_peered":"2023-08-09T13:36:12.747302+0000","last_clean":"2023-08-09T13:36:12.747302+0000","last_became_active":"2023-08-09T13:36:10.870525+0000","last_became_peered":"2023-08-09T13:36:10.870525+0000","last_unstale":"2023-08-09T13:36:12.747302+0000","last_undegraded":"2023-08-09T13:36:12.747302+0000","last_fullsized":"2023-08-09T13:36:12.747302+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:36:09.813580+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:36:09.813580+0000","last_clean_scrub_stamp":"2023-08-09T13:36:09.813580+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T14:18:40.169070+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.17","version":"0'0","reported_seq":11,"reported_epoch":39,"state":"active+clean","last_fresh":"2023-08-09T13:36:13.181768+0000","last_change":"2023-08-09T13:36:11.759335+0000","last_active":"2023-08-09T13:36:13.181768+0000","last_peered":"2023-08-09T13:36:13.181768+0000","last_clean":"2023-08-09T13:36:13.181768+0000","last_became_active":"2023-08-09T13:36:11.758848+0000","last_became_peered":"2023-08-09T13:36:11.758848+0000","last_unstale":"2023-08-09T13:36:13.181768+0000","last_undegraded":"2023-08-09T13:36:13.181768+0000","last_fullsized":"2023-08-09T13:36:13.181768+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:36:10.826860+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:36:10.826860+0000","last_clean_scrub_stamp":"2023-08-09T13:36:10.826860+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T16:04:02.242978+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.16","version":"0'0","reported_seq":13,"reported_epoch":39,"state":"active+clean","last_fresh":"2023-08-09T13:36:13.180825+0000","last_change":"2023-08-09T13:36:10.866553+0000","last_active":"2023-08-09T13:36:13.180825+0000","last_peered":"2023-08-09T13:36:13.180825+0000","last_clean":"2023-08-09T13:36:13.180825+0000","last_became_active":"2023-08-09T13:36:10.866120+0000","last_became_peered":"2023-08-09T13:36:10.866120+0000","last_unstale":"2023-08-09T13:36:13.180825+0000","last_undegraded":"2023-08-09T13:36:13.180825+0000","last_fullsized":"2023-08-09T13:36:13.180825+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:36:09.813580+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:36:09.813580+0000","last_clean_scrub_stamp":"2023-08-09T13:36:09.813580+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T15:06:05.146016+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.14","version":"0'0","reported_seq":11,"reported_epoch":39,"state":"active+clean","last_fresh":"2023-08-09T13:36:13.183196+0000","last_change":"2023-08-09T13:36:11.772315+0000","last_active":"2023-08-09T13:36:13.183196+0000","last_peered":"2023-08-09T13:36:13.183196+0000","last_clean":"2023-08-09T13:36:13.183196+0000","last_became_active":"2023-08-09T13:36:11.772006+0000","last_became_peered":"2023-08-09T13:36:11.772006+0000","last_unstale":"2023-08-09T13:36:13.183196+0000","last_undegraded":"2023-08-09T13:36:13.183196+0000","last_fullsized":"2023-08-09T13:36:13.183196+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:36:10.826860+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:36:10.826860+0000","last_clean_scrub_stamp":"2023-08-09T13:36:10.826860+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T18:07:15.877189+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.15","version":"38'3","reported_seq":16,"reported_epoch":39,"state":"active+clean","last_fresh":"2023-08-09T13:36:13.181848+0000","last_change":"2023-08-09T13:36:10.875205+0000","last_active":"2023-08-09T13:36:13.181848+0000","last_peered":"2023-08-09T13:36:13.181848+0000","last_clean":"2023-08-09T13:36:13.181848+0000","last_became_active":"2023-08-09T13:36:10.874765+0000","last_became_peered":"2023-08-09T13:36:10.874765+0000","last_unstale":"2023-08-09T13:36:13.181848+0000","last_undegraded":"2023-08-09T13:36:13.181848+0000","last_fullsized":"2023-08-09T13:36:13.181848+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:36:09.813580+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:36:09.813580+0000","last_clean_scrub_stamp":"2023-08-09T13:36:09.813580+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T21:01:18.302601+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.15","version":"0'0","reported_seq":11,"reported_epoch":39,"state":"active+clean","last_fresh":"2023-08-09T13:36:13.181101+0000","last_change":"2023-08-09T13:36:11.778121+0000","last_active":"2023-08-09T13:36:13.181101+0000","last_peered":"2023-08-09T13:36:13.181101+0000","last_clean":"2023-08-09T13:36:13.181101+0000","last_became_active":"2023-08-09T13:36:11.777690+0000","last_became_peered":"2023-08-09T13:36:11.777690+0000","last_unstale":"2023-08-09T13:36:13.181101+0000","last_undegraded":"2023-08-09T13:36:13.181101+0000","last_fullsized":"2023-08-09T13:36:13.181101+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:36:10.826860+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:36:10.826860+0000","last_clean_scrub_stamp":"2023-08-09T13:36:10.826860+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T16:25:36.538091+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.14","version":"38'6","reported_seq":19,"reported_epoch":39,"state":"active+clean","last_fresh":"2023-08-09T13:36:13.180506+0000","last_change":"2023-08-09T13:36:10.870131+0000","last_active":"2023-08-09T13:36:13.180506+0000","last_peered":"2023-08-09T13:36:13.180506+0000","last_clean":"2023-08-09T13:36:13.180506+0000","last_became_active":"2023-08-09T13:36:10.869794+0000","last_became_peered":"2023-08-09T13:36:10.869794+0000","last_unstale":"2023-08-09T13:36:13.180506+0000","last_undegraded":"2023-08-09T13:36:13.180506+0000","last_fullsized":"2023-08-09T13:36:13.180506+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:36:09.813580+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:36:09.813580+0000","last_clean_scrub_stamp":"2023-08-09T13:36:09.813580+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T15:46:46.549814+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.12","version":"0'0","reported_seq":11,"reported_epoch":39,"state":"active+clean","last_fresh":"2023-08-09T13:36:12.746478+0000","last_change":"2023-08-09T13:36:11.779827+0000","last_active":"2023-08-09T13:36:12.746478+0000","last_peered":"2023-08-09T13:36:12.746478+0000","last_clean":"2023-08-09T13:36:12.746478+0000","last_became_active":"2023-08-09T13:36:11.778884+0000","last_became_peered":"2023-08-09T13:36:11.778884+0000","last_unstale":"2023-08-09T13:36:12.746478+0000","last_undegraded":"2023-08-09T13:36:12.746478+0000","last_fullsized":"2023-08-09T13:36:12.746478+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:36:10.826860+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:36:10.826860+0000","last_clean_scrub_stamp":"2023-08-09T13:36:10.826860+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T14:19:51.658754+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.13","version":"38'2","reported_seq":15,"reported_epoch":39,"state":"active+clean","last_fresh":"2023-08-09T13:36:13.183069+0000","last_change":"2023-08-09T13:36:10.868241+0000","last_active":"2023-08-09T13:36:13.183069+0000","last_peered":"2023-08-09T13:36:13.183069+0000","last_clean":"2023-08-09T13:36:13.183069+0000","last_became_active":"2023-08-09T13:36:10.867827+0000","last_became_peered":"2023-08-09T13:36:10.867827+0000","last_unstale":"2023-08-09T13:36:13.183069+0000","last_undegraded":"2023-08-09T13:36:13.183069+0000","last_fullsized":"2023-08-09T13:36:13.183069+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:36:09.813580+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:36:09.813580+0000","last_clean_scrub_stamp":"2023-08-09T13:36:09.813580+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T17:04:14.699579+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.13","version":"0'0","reported_seq":11,"reported_epoch":39,"state":"active+clean","last_fresh":"2023-08-09T13:36:13.179718+0000","last_change":"2023-08-09T13:36:11.756659+0000","last_active":"2023-08-09T13:36:13.179718+0000","last_peered":"2023-08-09T13:36:13.179718+0000","last_clean":"2023-08-09T13:36:13.179718+0000","last_became_active":"2023-08-09T13:36:11.756166+0000","last_became_peered":"2023-08-09T13:36:11.756166+0000","last_unstale":"2023-08-09T13:36:13.179718+0000","last_undegraded":"2023-08-09T13:36:13.179718+0000","last_fullsized":"2023-08-09T13:36:13.179718+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:36:10.826860+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:36:10.826860+0000","last_clean_scrub_stamp":"2023-08-09T13:36:10.826860+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T16:34:53.205983+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.12","version":"0'0","reported_seq":13,"reported_epoch":39,"state":"active+clean","last_fresh":"2023-08-09T13:36:13.181274+0000","last_change":"2023-08-09T13:36:10.865808+0000","last_active":"2023-08-09T13:36:13.181274+0000","last_peered":"2023-08-09T13:36:13.181274+0000","last_clean":"2023-08-09T13:36:13.181274+0000","last_became_active":"2023-08-09T13:36:10.865466+0000","last_became_peered":"2023-08-09T13:36:10.865466+0000","last_unstale":"2023-08-09T13:36:13.181274+0000","last_undegraded":"2023-08-09T13:36:13.181274+0000","last_fullsized":"2023-08-09T13:36:13.181274+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:36:09.813580+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:36:09.813580+0000","last_clean_scrub_stamp":"2023-08-09T13:36:09.813580+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T18:44:21.503311+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.10","version":"0'0","reported_seq":11,"reported_epoch":39,"state":"active+clean","last_fresh":"2023-08-09T13:36:13.181608+0000","last_change":"2023-08-09T13:36:11.771222+0000","last_active":"2023-08-09T13:36:13.181608+0000","last_peered":"2023-08-09T13:36:13.181608+0000","last_clean":"2023-08-09T13:36:13.181608+0000","last_became_active":"2023-08-09T13:36:11.770198+0000","last_became_peered":"2023-08-09T13:36:11.770198+0000","last_unstale":"2023-08-09T13:36:13.181608+0000","last_undegraded":"2023-08-09T13:36:13.181608+0000","last_fullsized":"2023-08-09T13:36:13.181608+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:36:10.826860+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:36:10.826860+0000","last_clean_scrub_stamp":"2023-08-09T13:36:10.826860+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T22:37:56.162796+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.11","version":"0'0","reported_seq":13,"reported_epoch":39,"state":"active+clean","last_fresh":"2023-08-09T13:36:13.182549+0000","last_change":"2023-08-09T13:36:10.862804+0000","last_active":"2023-08-09T13:36:13.182549+0000","last_peered":"2023-08-09T13:36:13.182549+0000","last_clean":"2023-08-09T13:36:13.182549+0000","last_became_active":"2023-08-09T13:36:10.862524+0000","last_became_peered":"2023-08-09T13:36:10.862524+0000","last_unstale":"2023-08-09T13:36:13.182549+0000","last_undegraded":"2023-08-09T13:36:13.182549+0000","last_fullsized":"2023-08-09T13:36:13.182549+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:36:09.813580+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:36:09.813580+0000","last_clean_scrub_stamp":"2023-08-09T13:36:09.813580+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T16:44:06.470566+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.11","version":"0'0","reported_seq":11,"reported_epoch":39,"state":"active+clean","last_fresh":"2023-08-09T13:36:13.181616+0000","last_change":"2023-08-09T13:36:11.777994+0000","last_active":"2023-08-09T13:36:13.181616+0000","last_peered":"2023-08-09T13:36:13.181616+0000","last_clean":"2023-08-09T13:36:13.181616+0000","last_became_active":"2023-08-09T13:36:11.777683+0000","last_became_peered":"2023-08-09T13:36:11.777683+0000","last_unstale":"2023-08-09T13:36:13.181616+0000","last_undegraded":"2023-08-09T13:36:13.181616+0000","last_fullsized":"2023-08-09T13:36:13.181616+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:36:10.826860+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:36:10.826860+0000","last_clean_scrub_stamp":"2023-08-09T13:36:10.826860+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T17:22:46.595006+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.10","version":"38'8","reported_seq":21,"reported_epoch":39,"state":"active+clean","last_fresh":"2023-08-09T13:36:13.179052+0000","last_change":"2023-08-09T13:36:10.849165+0000","last_active":"2023-08-09T13:36:13.179052+0000","last_peered":"2023-08-09T13:36:13.179052+0000","last_clean":"2023-08-09T13:36:13.179052+0000","last_became_active":"2023-08-09T13:36:10.848413+0000","last_became_peered":"2023-08-09T13:36:10.848413+0000","last_unstale":"2023-08-09T13:36:13.179052+0000","last_undegraded":"2023-08-09T13:36:13.179052+0000","last_fullsized":"2023-08-09T13:36:13.179052+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:36:09.813580+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:36:09.813580+0000","last_clean_scrub_stamp":"2023-08-09T13:36:09.813580+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-11T01:22:58.988197+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.e","version":"0'0","reported_seq":11,"reported_epoch":39,"state":"active+clean","last_fresh":"2023-08-09T13:36:13.181800+0000","last_change":"2023-08-09T13:36:11.764459+0000","last_active":"2023-08-09T13:36:13.181800+0000","last_peered":"2023-08-09T13:36:13.181800+0000","last_clean":"2023-08-09T13:36:13.181800+0000","last_became_active":"2023-08-09T13:36:11.764100+0000","last_became_peered":"2023-08-09T13:36:11.764100+0000","last_unstale":"2023-08-09T13:36:13.181800+0000","last_undegraded":"2023-08-09T13:36:13.181800+0000","last_fullsized":"2023-08-09T13:36:13.181800+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:36:10.826860+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:36:10.826860+0000","last_clean_scrub_stamp":"2023-08-09T13:36:10.826860+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T16:36:09.047515+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.f","version":"38'1","reported_seq":14,"reported_epoch":39,"state":"active+clean","last_fresh":"2023-08-09T13:36:13.179834+0000","last_change":"2023-08-09T13:36:10.863242+0000","last_active":"2023-08-09T13:36:13.179834+0000","last_peered":"2023-08-09T13:36:13.179834+0000","last_clean":"2023-08-09T13:36:13.179834+0000","last_became_active":"2023-08-09T13:36:10.862921+0000","last_became_peered":"2023-08-09T13:36:10.862921+0000","last_unstale":"2023-08-09T13:36:13.179834+0000","last_undegraded":"2023-08-09T13:36:13.179834+0000","last_fullsized":"2023-08-09T13:36:13.179834+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:36:09.813580+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:36:09.813580+0000","last_clean_scrub_stamp":"2023-08-09T13:36:09.813580+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T18:49:41.929234+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.f","version":"0'0","reported_seq":11,"reported_epoch":39,"state":"active+clean","last_fresh":"2023-08-09T13:36:13.181064+0000","last_change":"2023-08-09T13:36:11.764093+0000","last_active":"2023-08-09T13:36:13.181064+0000","last_peered":"2023-08-09T13:36:13.181064+0000","last_clean":"2023-08-09T13:36:13.181064+0000","last_became_active":"2023-08-09T13:36:11.763787+0000","last_became_peered":"2023-08-09T13:36:11.763787+0000","last_unstale":"2023-08-09T13:36:13.181064+0000","last_undegraded":"2023-08-09T13:36:13.181064+0000","last_fullsized":"2023-08-09T13:36:13.181064+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:36:10.826860+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:36:10.826860+0000","last_clean_scrub_stamp":"2023-08-09T13:36:10.826860+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T15:32:48.889127+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.e","version":"38'2","reported_seq":15,"reported_epoch":39,"state":"active+clean","last_fresh":"2023-08-09T13:36:12.747193+0000","last_change":"2023-08-09T13:36:10.867599+0000","last_active":"2023-08-09T13:36:12.747193+0000","last_peered":"2023-08-09T13:36:12.747193+0000","last_clean":"2023-08-09T13:36:12.747193+0000","last_became_active":"2023-08-09T13:36:10.867254+0000","last_became_peered":"2023-08-09T13:36:10.867254+0000","last_unstale":"2023-08-09T13:36:12.747193+0000","last_undegraded":"2023-08-09T13:36:12.747193+0000","last_fullsized":"2023-08-09T13:36:12.747193+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:36:09.813580+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:36:09.813580+0000","last_clean_scrub_stamp":"2023-08-09T13:36:09.813580+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T13:54:32.735702+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.c","version":"0'0","reported_seq":11,"reported_epoch":39,"state":"active+clean","last_fresh":"2023-08-09T13:36:12.747704+0000","last_change":"2023-08-09T13:36:11.776990+0000","last_active":"2023-08-09T13:36:12.747704+0000","last_peered":"2023-08-09T13:36:12.747704+0000","last_clean":"2023-08-09T13:36:12.747704+0000","last_became_active":"2023-08-09T13:36:11.776508+0000","last_became_peered":"2023-08-09T13:36:11.776508+0000","last_unstale":"2023-08-09T13:36:12.747704+0000","last_undegraded":"2023-08-09T13:36:12.747704+0000","last_fullsized":"2023-08-09T13:36:12.747704+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:36:10.826860+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:36:10.826860+0000","last_clean_scrub_stamp":"2023-08-09T13:36:10.826860+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T20:38:04.370278+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.d","version":"38'1","reported_seq":14,"reported_epoch":39,"state":"active+clean","last_fresh":"2023-08-09T13:36:13.181127+0000","last_change":"2023-08-09T13:36:10.860479+0000","last_active":"2023-08-09T13:36:13.181127+0000","last_peered":"2023-08-09T13:36:13.181127+0000","last_clean":"2023-08-09T13:36:13.181127+0000","last_became_active":"2023-08-09T13:36:10.860059+0000","last_became_peered":"2023-08-09T13:36:10.860059+0000","last_unstale":"2023-08-09T13:36:13.181127+0000","last_undegraded":"2023-08-09T13:36:13.181127+0000","last_fullsized":"2023-08-09T13:36:13.181127+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:36:09.813580+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:36:09.813580+0000","last_clean_scrub_stamp":"2023-08-09T13:36:09.813580+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-11T01:22:58.988197+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.d","version":"0'0","reported_seq":11,"reported_epoch":39,"state":"active+clean","last_fresh":"2023-08-09T13:36:12.747803+0000","last_change":"2023-08-09T13:36:11.782782+0000","last_active":"2023-08-09T13:36:12.747803+0000","last_peered":"2023-08-09T13:36:12.747803+0000","last_clean":"2023-08-09T13:36:12.747803+0000","last_became_active":"2023-08-09T13:36:11.782450+0000","last_became_peered":"2023-08-09T13:36:11.782450+0000","last_unstale":"2023-08-09T13:36:12.747803+0000","last_undegraded":"2023-08-09T13:36:12.747803+0000","last_fullsized":"2023-08-09T13:36:12.747803+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:36:10.826860+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:36:10.826860+0000","last_clean_scrub_stamp":"2023-08-09T13:36:10.826860+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T19:58:53.923099+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.c","version":"0'0","reported_seq":13,"reported_epoch":39,"state":"active+clean","last_fresh":"2023-08-09T13:36:13.183552+0000","last_change":"2023-08-09T13:36:10.877362+0000","last_active":"2023-08-09T13:36:13.183552+0000","last_peered":"2023-08-09T13:36:13.183552+0000","last_clean":"2023-08-09T13:36:13.183552+0000","last_became_active":"2023-08-09T13:36:10.877103+0000","last_became_peered":"2023-08-09T13:36:10.877103+0000","last_unstale":"2023-08-09T13:36:13.183552+0000","last_undegraded":"2023-08-09T13:36:13.183552+0000","last_fullsized":"2023-08-09T13:36:13.183552+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:36:09.813580+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:36:09.813580+0000","last_clean_scrub_stamp":"2023-08-09T13:36:09.813580+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T23:38:08.983084+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.a","version":"0'0","reported_seq":11,"reported_epoch":39,"state":"active+clean","last_fresh":"2023-08-09T13:36:13.182078+0000","last_change":"2023-08-09T13:36:11.770333+0000","last_active":"2023-08-09T13:36:13.182078+0000","last_peered":"2023-08-09T13:36:13.182078+0000","last_clean":"2023-08-09T13:36:13.182078+0000","last_became_active":"2023-08-09T13:36:11.770013+0000","last_became_peered":"2023-08-09T13:36:11.770013+0000","last_unstale":"2023-08-09T13:36:13.182078+0000","last_undegraded":"2023-08-09T13:36:13.182078+0000","last_fullsized":"2023-08-09T13:36:13.182078+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:36:10.826860+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:36:10.826860+0000","last_clean_scrub_stamp":"2023-08-09T13:36:10.826860+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T23:14:13.560864+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.b","version":"38'2","reported_seq":15,"reported_epoch":39,"state":"active+clean","last_fresh":"2023-08-09T13:36:13.183444+0000","last_change":"2023-08-09T13:36:10.867524+0000","last_active":"2023-08-09T13:36:13.183444+0000","last_peered":"2023-08-09T13:36:13.183444+0000","last_clean":"2023-08-09T13:36:13.183444+0000","last_became_active":"2023-08-09T13:36:10.866825+0000","last_became_peered":"2023-08-09T13:36:10.866825+0000","last_unstale":"2023-08-09T13:36:13.183444+0000","last_undegraded":"2023-08-09T13:36:13.183444+0000","last_fullsized":"2023-08-09T13:36:13.183444+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:36:09.813580+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:36:09.813580+0000","last_clean_scrub_stamp":"2023-08-09T13:36:09.813580+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T19:18:55.290314+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.b","version":"0'0","reported_seq":11,"reported_epoch":39,"state":"active+clean","last_fresh":"2023-08-09T13:36:12.743542+0000","last_change":"2023-08-09T13:36:11.766655+0000","last_active":"2023-08-09T13:36:12.743542+0000","last_peered":"2023-08-09T13:36:12.743542+0000","last_clean":"2023-08-09T13:36:12.743542+0000","last_became_active":"2023-08-09T13:36:11.766294+0000","last_became_peered":"2023-08-09T13:36:11.766294+0000","last_unstale":"2023-08-09T13:36:12.743542+0000","last_undegraded":"2023-08-09T13:36:12.743542+0000","last_fullsized":"2023-08-09T13:36:12.743542+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:36:10.826860+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:36:10.826860+0000","last_clean_scrub_stamp":"2023-08-09T13:36:10.826860+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T21:18:11.963195+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.a","version":"0'0","reported_seq":13,"reported_epoch":39,"state":"active+clean","last_fresh":"2023-08-09T13:36:13.179977+0000","last_change":"2023-08-09T13:36:10.877398+0000","last_active":"2023-08-09T13:36:13.179977+0000","last_peered":"2023-08-09T13:36:13.179977+0000","last_clean":"2023-08-09T13:36:13.179977+0000","last_became_active":"2023-08-09T13:36:10.877048+0000","last_became_peered":"2023-08-09T13:36:10.877048+0000","last_unstale":"2023-08-09T13:36:13.179977+0000","last_undegraded":"2023-08-09T13:36:13.179977+0000","last_fullsized":"2023-08-09T13:36:13.179977+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:36:09.813580+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:36:09.813580+0000","last_clean_scrub_stamp":"2023-08-09T13:36:09.813580+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T18:28:02.161530+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.8","version":"0'0","reported_seq":11,"reported_epoch":39,"state":"active+clean","last_fresh":"2023-08-09T13:36:13.179747+0000","last_change":"2023-08-09T13:36:11.765089+0000","last_active":"2023-08-09T13:36:13.179747+0000","last_peered":"2023-08-09T13:36:13.179747+0000","last_clean":"2023-08-09T13:36:13.179747+0000","last_became_active":"2023-08-09T13:36:11.763858+0000","last_became_peered":"2023-08-09T13:36:11.763858+0000","last_unstale":"2023-08-09T13:36:13.179747+0000","last_undegraded":"2023-08-09T13:36:13.179747+0000","last_fullsized":"2023-08-09T13:36:13.179747+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:36:10.826860+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:36:10.826860+0000","last_clean_scrub_stamp":"2023-08-09T13:36:10.826860+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T13:50:54.643230+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.9","version":"0'0","reported_seq":13,"reported_epoch":39,"state":"active+clean","last_fresh":"2023-08-09T13:36:13.178917+0000","last_change":"2023-08-09T13:36:10.866720+0000","last_active":"2023-08-09T13:36:13.178917+0000","last_peered":"2023-08-09T13:36:13.178917+0000","last_clean":"2023-08-09T13:36:13.178917+0000","last_became_active":"2023-08-09T13:36:10.866446+0000","last_became_peered":"2023-08-09T13:36:10.866446+0000","last_unstale":"2023-08-09T13:36:13.178917+0000","last_undegraded":"2023-08-09T13:36:13.178917+0000","last_fullsized":"2023-08-09T13:36:13.178917+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:36:09.813580+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:36:09.813580+0000","last_clean_scrub_stamp":"2023-08-09T13:36:09.813580+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T17:14:42.534456+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.9","version":"0'0","reported_seq":11,"reported_epoch":39,"state":"active+clean","last_fresh":"2023-08-09T13:36:13.179056+0000","last_change":"2023-08-09T13:36:11.774417+0000","last_active":"2023-08-09T13:36:13.179056+0000","last_peered":"2023-08-09T13:36:13.179056+0000","last_clean":"2023-08-09T13:36:13.179056+0000","last_became_active":"2023-08-09T13:36:11.771652+0000","last_became_peered":"2023-08-09T13:36:11.771652+0000","last_unstale":"2023-08-09T13:36:13.179056+0000","last_undegraded":"2023-08-09T13:36:13.179056+0000","last_fullsized":"2023-08-09T13:36:13.179056+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:36:10.826860+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:36:10.826860+0000","last_clean_scrub_stamp":"2023-08-09T13:36:10.826860+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T18:10:39.629916+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.8","version":"0'0","reported_seq":13,"reported_epoch":39,"state":"active+clean","last_fresh":"2023-08-09T13:36:13.180072+0000","last_change":"2023-08-09T13:36:10.866696+0000","last_active":"2023-08-09T13:36:13.180072+0000","last_peered":"2023-08-09T13:36:13.180072+0000","last_clean":"2023-08-09T13:36:13.180072+0000","last_became_active":"2023-08-09T13:36:10.866386+0000","last_became_peered":"2023-08-09T13:36:10.866386+0000","last_unstale":"2023-08-09T13:36:13.180072+0000","last_undegraded":"2023-08-09T13:36:13.180072+0000","last_fullsized":"2023-08-09T13:36:13.180072+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:36:09.813580+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:36:09.813580+0000","last_clean_scrub_stamp":"2023-08-09T13:36:09.813580+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T21:14:11.201990+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.1","version":"0'0","reported_seq":11,"reported_epoch":39,"state":"active+clean","last_fresh":"2023-08-09T13:36:12.744809+0000","last_change":"2023-08-09T13:36:11.766822+0000","last_active":"2023-08-09T13:36:12.744809+0000","last_peered":"2023-08-09T13:36:12.744809+0000","last_clean":"2023-08-09T13:36:12.744809+0000","last_became_active":"2023-08-09T13:36:11.766533+0000","last_became_peered":"2023-08-09T13:36:11.766533+0000","last_unstale":"2023-08-09T13:36:12.744809+0000","last_undegraded":"2023-08-09T13:36:12.744809+0000","last_fullsized":"2023-08-09T13:36:12.744809+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:36:10.826860+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:36:10.826860+0000","last_clean_scrub_stamp":"2023-08-09T13:36:10.826860+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T18:33:06.918646+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.0","version":"0'0","reported_seq":13,"reported_epoch":39,"state":"active+clean","last_fresh":"2023-08-09T13:36:13.179109+0000","last_change":"2023-08-09T13:36:10.865247+0000","last_active":"2023-08-09T13:36:13.179109+0000","last_peered":"2023-08-09T13:36:13.179109+0000","last_clean":"2023-08-09T13:36:13.179109+0000","last_became_active":"2023-08-09T13:36:10.864877+0000","last_became_peered":"2023-08-09T13:36:10.864877+0000","last_unstale":"2023-08-09T13:36:13.179109+0000","last_undegraded":"2023-08-09T13:36:13.179109+0000","last_fullsized":"2023-08-09T13:36:13.179109+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:36:09.813580+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:36:09.813580+0000","last_clean_scrub_stamp":"2023-08-09T13:36:09.813580+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T23:41:55.443735+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":50,"reported_epoch":39,"state":"active+clean","last_fresh":"2023-08-09T13:36:12.748224+0000","last_change":"2023-08-09T13:33:55.886389+0000","last_active":"2023-08-09T13:36:12.748224+0000","last_peered":"2023-08-09T13:36:12.748224+0000","last_clean":"2023-08-09T13:36:12.748224+0000","last_became_active":"2023-08-09T13:33:55.885638+0000","last_became_peered":"2023-08-09T13:33:55.885638+0000","last_unstale":"2023-08-09T13:36:12.748224+0000","last_undegraded":"2023-08-09T13:36:12.748224+0000","last_fullsized":"2023-08-09T13:36:12.748224+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:33:54.845869+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:33:54.845869+0000","last_clean_scrub_stamp":"2023-08-09T13:33:54.845869+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T21:22:21.904050+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.0","version":"0'0","reported_seq":11,"reported_epoch":39,"state":"active+clean","last_fresh":"2023-08-09T13:36:13.179673+0000","last_change":"2023-08-09T13:36:11.777749+0000","last_active":"2023-08-09T13:36:13.179673+0000","last_peered":"2023-08-09T13:36:13.179673+0000","last_clean":"2023-08-09T13:36:13.179673+0000","last_became_active":"2023-08-09T13:36:11.777465+0000","last_became_peered":"2023-08-09T13:36:11.777465+0000","last_unstale":"2023-08-09T13:36:13.179673+0000","last_undegraded":"2023-08-09T13:36:13.179673+0000","last_fullsized":"2023-08-09T13:36:13.179673+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:36:10.826860+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:36:10.826860+0000","last_clean_scrub_stamp":"2023-08-09T13:36:10.826860+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T23:06:21.128202+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.1","version":"0'0","reported_seq":13,"reported_epoch":39,"state":"active+clean","last_fresh":"2023-08-09T13:36:12.743107+0000","last_change":"2023-08-09T13:36:10.873994+0000","last_active":"2023-08-09T13:36:12.743107+0000","last_peered":"2023-08-09T13:36:12.743107+0000","last_clean":"2023-08-09T13:36:12.743107+0000","last_became_active":"2023-08-09T13:36:10.873698+0000","last_became_peered":"2023-08-09T13:36:10.873698+0000","last_unstale":"2023-08-09T13:36:12.743107+0000","last_undegraded":"2023-08-09T13:36:12.743107+0000","last_fullsized":"2023-08-09T13:36:12.743107+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:36:09.813580+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:36:09.813580+0000","last_clean_scrub_stamp":"2023-08-09T13:36:09.813580+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T14:32:43.462887+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":50,"reported_epoch":39,"state":"active+clean","last_fresh":"2023-08-09T13:36:12.747882+0000","last_change":"2023-08-09T13:33:56.401781+0000","last_active":"2023-08-09T13:36:12.747882+0000","last_peered":"2023-08-09T13:36:12.747882+0000","last_clean":"2023-08-09T13:36:12.747882+0000","last_became_active":"2023-08-09T13:33:56.400336+0000","last_became_peered":"2023-08-09T13:33:56.400336+0000","last_unstale":"2023-08-09T13:36:12.747882+0000","last_undegraded":"2023-08-09T13:36:12.747882+0000","last_fullsized":"2023-08-09T13:36:12.747882+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:33:54.845869+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:33:54.845869+0000","last_clean_scrub_stamp":"2023-08-09T13:33:54.845869+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-11T00:54:39.337743+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.3","version":"0'0","reported_seq":11,"reported_epoch":39,"state":"active+clean","last_fresh":"2023-08-09T13:36:13.182686+0000","last_change":"2023-08-09T13:36:11.771635+0000","last_active":"2023-08-09T13:36:13.182686+0000","last_peered":"2023-08-09T13:36:13.182686+0000","last_clean":"2023-08-09T13:36:13.182686+0000","last_became_active":"2023-08-09T13:36:11.771047+0000","last_became_peered":"2023-08-09T13:36:11.771047+0000","last_unstale":"2023-08-09T13:36:13.182686+0000","last_undegraded":"2023-08-09T13:36:13.182686+0000","last_fullsized":"2023-08-09T13:36:13.182686+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:36:10.826860+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:36:10.826860+0000","last_clean_scrub_stamp":"2023-08-09T13:36:10.826860+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T22:42:07.388189+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.2","version":"0'0","reported_seq":13,"reported_epoch":39,"state":"active+clean","last_fresh":"2023-08-09T13:36:12.745207+0000","last_change":"2023-08-09T13:36:10.869092+0000","last_active":"2023-08-09T13:36:12.745207+0000","last_peered":"2023-08-09T13:36:12.745207+0000","last_clean":"2023-08-09T13:36:12.745207+0000","last_became_active":"2023-08-09T13:36:10.868759+0000","last_became_peered":"2023-08-09T13:36:10.868759+0000","last_unstale":"2023-08-09T13:36:12.745207+0000","last_undegraded":"2023-08-09T13:36:12.745207+0000","last_fullsized":"2023-08-09T13:36:12.745207+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:36:09.813580+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:36:09.813580+0000","last_clean_scrub_stamp":"2023-08-09T13:36:09.813580+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T19:22:17.764271+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":54,"reported_epoch":39,"state":"active+clean","last_fresh":"2023-08-09T13:36:12.746148+0000","last_change":"2023-08-09T13:33:56.404236+0000","last_active":"2023-08-09T13:36:12.746148+0000","last_peered":"2023-08-09T13:36:12.746148+0000","last_clean":"2023-08-09T13:36:12.746148+0000","last_became_active":"2023-08-09T13:33:56.402485+0000","last_became_peered":"2023-08-09T13:33:56.402485+0000","last_unstale":"2023-08-09T13:36:12.746148+0000","last_undegraded":"2023-08-09T13:36:12.746148+0000","last_fullsized":"2023-08-09T13:36:12.746148+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:33:54.845869+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:33:54.845869+0000","last_clean_scrub_stamp":"2023-08-09T13:33:54.845869+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-11T00:39:23.589743+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.2","version":"0'0","reported_seq":11,"reported_epoch":39,"state":"active+clean","last_fresh":"2023-08-09T13:36:13.182480+0000","last_change":"2023-08-09T13:36:11.780136+0000","last_active":"2023-08-09T13:36:13.182480+0000","last_peered":"2023-08-09T13:36:13.182480+0000","last_clean":"2023-08-09T13:36:13.182480+0000","last_became_active":"2023-08-09T13:36:11.779856+0000","last_became_peered":"2023-08-09T13:36:11.779856+0000","last_unstale":"2023-08-09T13:36:13.182480+0000","last_undegraded":"2023-08-09T13:36:13.182480+0000","last_fullsized":"2023-08-09T13:36:13.182480+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:36:10.826860+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:36:10.826860+0000","last_clean_scrub_stamp":"2023-08-09T13:36:10.826860+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T18:55:31.620023+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.3","version":"0'0","reported_seq":13,"reported_epoch":39,"state":"active+clean","last_fresh":"2023-08-09T13:36:13.181606+0000","last_change":"2023-08-09T13:36:10.877771+0000","last_active":"2023-08-09T13:36:13.181606+0000","last_peered":"2023-08-09T13:36:13.181606+0000","last_clean":"2023-08-09T13:36:13.181606+0000","last_became_active":"2023-08-09T13:36:10.877493+0000","last_became_peered":"2023-08-09T13:36:10.877493+0000","last_unstale":"2023-08-09T13:36:13.181606+0000","last_undegraded":"2023-08-09T13:36:13.181606+0000","last_fullsized":"2023-08-09T13:36:13.181606+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:36:09.813580+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:36:09.813580+0000","last_clean_scrub_stamp":"2023-08-09T13:36:09.813580+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T20:42:30.032317+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":58,"reported_epoch":39,"state":"active+clean","last_fresh":"2023-08-09T13:36:13.183659+0000","last_change":"2023-08-09T13:33:55.889565+0000","last_active":"2023-08-09T13:36:13.183659+0000","last_peered":"2023-08-09T13:36:13.183659+0000","last_clean":"2023-08-09T13:36:13.183659+0000","last_became_active":"2023-08-09T13:33:55.889164+0000","last_became_peered":"2023-08-09T13:33:55.889164+0000","last_unstale":"2023-08-09T13:36:13.183659+0000","last_undegraded":"2023-08-09T13:36:13.183659+0000","last_fullsized":"2023-08-09T13:36:13.183659+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:33:54.845869+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:33:54.845869+0000","last_clean_scrub_stamp":"2023-08-09T13:33:54.845869+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T15:55:03.403580+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.5","version":"0'0","reported_seq":11,"reported_epoch":39,"state":"active+clean","last_fresh":"2023-08-09T13:36:13.181466+0000","last_change":"2023-08-09T13:36:11.765282+0000","last_active":"2023-08-09T13:36:13.181466+0000","last_peered":"2023-08-09T13:36:13.181466+0000","last_clean":"2023-08-09T13:36:13.181466+0000","last_became_active":"2023-08-09T13:36:11.764922+0000","last_became_peered":"2023-08-09T13:36:11.764922+0000","last_unstale":"2023-08-09T13:36:13.181466+0000","last_undegraded":"2023-08-09T13:36:13.181466+0000","last_fullsized":"2023-08-09T13:36:13.181466+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:36:10.826860+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:36:10.826860+0000","last_clean_scrub_stamp":"2023-08-09T13:36:10.826860+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T14:34:10.458179+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.4","version":"0'0","reported_seq":13,"reported_epoch":39,"state":"active+clean","last_fresh":"2023-08-09T13:36:12.743299+0000","last_change":"2023-08-09T13:36:10.869464+0000","last_active":"2023-08-09T13:36:12.743299+0000","last_peered":"2023-08-09T13:36:12.743299+0000","last_clean":"2023-08-09T13:36:12.743299+0000","last_became_active":"2023-08-09T13:36:10.869112+0000","last_became_peered":"2023-08-09T13:36:10.869112+0000","last_unstale":"2023-08-09T13:36:12.743299+0000","last_undegraded":"2023-08-09T13:36:12.743299+0000","last_fullsized":"2023-08-09T13:36:12.743299+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:36:09.813580+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:36:09.813580+0000","last_clean_scrub_stamp":"2023-08-09T13:36:09.813580+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T18:01:27.790776+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"16'117","reported_seq":200,"reported_epoch":39,"state":"active+clean","last_fresh":"2023-08-09T13:36:13.179633+0000","last_change":"2023-08-09T13:33:55.886345+0000","last_active":"2023-08-09T13:36:13.179633+0000","last_peered":"2023-08-09T13:36:13.179633+0000","last_clean":"2023-08-09T13:36:13.179633+0000","last_became_active":"2023-08-09T13:33:55.885781+0000","last_became_peered":"2023-08-09T13:33:55.885781+0000","last_unstale":"2023-08-09T13:36:13.179633+0000","last_undegraded":"2023-08-09T13:36:13.179633+0000","last_fullsized":"2023-08-09T13:36:13.179633+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:33:54.845869+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:33:54.845869+0000","last_clean_scrub_stamp":"2023-08-09T13:33:54.845869+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T15:21:07.169824+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.3","version":"0'0","reported_seq":52,"reported_epoch":39,"state":"active+clean","last_fresh":"2023-08-09T13:36:12.743662+0000","last_change":"2023-08-09T13:33:55.886749+0000","last_active":"2023-08-09T13:36:12.743662+0000","last_peered":"2023-08-09T13:36:12.743662+0000","last_clean":"2023-08-09T13:36:12.743662+0000","last_became_active":"2023-08-09T13:33:55.886116+0000","last_became_peered":"2023-08-09T13:33:55.886116+0000","last_unstale":"2023-08-09T13:36:12.743662+0000","last_undegraded":"2023-08-09T13:36:12.743662+0000","last_fullsized":"2023-08-09T13:36:12.743662+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:33:54.845869+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:33:54.845869+0000","last_clean_scrub_stamp":"2023-08-09T13:33:54.845869+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-11T01:21:30.839021+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.4","version":"0'0","reported_seq":11,"reported_epoch":39,"state":"active+clean","last_fresh":"2023-08-09T13:36:13.181549+0000","last_change":"2023-08-09T13:36:11.764825+0000","last_active":"2023-08-09T13:36:13.181549+0000","last_peered":"2023-08-09T13:36:13.181549+0000","last_clean":"2023-08-09T13:36:13.181549+0000","last_became_active":"2023-08-09T13:36:11.764519+0000","last_became_peered":"2023-08-09T13:36:11.764519+0000","last_unstale":"2023-08-09T13:36:13.181549+0000","last_undegraded":"2023-08-09T13:36:13.181549+0000","last_fullsized":"2023-08-09T13:36:13.181549+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:36:10.826860+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:36:10.826860+0000","last_clean_scrub_stamp":"2023-08-09T13:36:10.826860+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T22:18:23.522022+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.5","version":"0'0","reported_seq":13,"reported_epoch":39,"state":"active+clean","last_fresh":"2023-08-09T13:36:13.181888+0000","last_change":"2023-08-09T13:36:10.875065+0000","last_active":"2023-08-09T13:36:13.181888+0000","last_peered":"2023-08-09T13:36:13.181888+0000","last_clean":"2023-08-09T13:36:13.181888+0000","last_became_active":"2023-08-09T13:36:10.874358+0000","last_became_peered":"2023-08-09T13:36:10.874358+0000","last_unstale":"2023-08-09T13:36:13.181888+0000","last_undegraded":"2023-08-09T13:36:13.181888+0000","last_fullsized":"2023-08-09T13:36:13.181888+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:36:09.813580+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:36:09.813580+0000","last_clean_scrub_stamp":"2023-08-09T13:36:09.813580+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T22:46:33.401724+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.2","version":"0'0","reported_seq":50,"reported_epoch":39,"state":"active+clean","last_fresh":"2023-08-09T13:36:13.181307+0000","last_change":"2023-08-09T13:33:56.402794+0000","last_active":"2023-08-09T13:36:13.181307+0000","last_peered":"2023-08-09T13:36:13.181307+0000","last_clean":"2023-08-09T13:36:13.181307+0000","last_became_active":"2023-08-09T13:33:56.402038+0000","last_became_peered":"2023-08-09T13:33:56.402038+0000","last_unstale":"2023-08-09T13:36:13.181307+0000","last_undegraded":"2023-08-09T13:36:13.181307+0000","last_fullsized":"2023-08-09T13:36:13.181307+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:33:54.845869+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:33:54.845869+0000","last_clean_scrub_stamp":"2023-08-09T13:33:54.845869+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T15:54:41.238281+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.7","version":"0'0","reported_seq":11,"reported_epoch":39,"state":"active+clean","last_fresh":"2023-08-09T13:36:12.747913+0000","last_change":"2023-08-09T13:36:11.784020+0000","last_active":"2023-08-09T13:36:12.747913+0000","last_peered":"2023-08-09T13:36:12.747913+0000","last_clean":"2023-08-09T13:36:12.747913+0000","last_became_active":"2023-08-09T13:36:11.783758+0000","last_became_peered":"2023-08-09T13:36:11.783758+0000","last_unstale":"2023-08-09T13:36:12.747913+0000","last_undegraded":"2023-08-09T13:36:12.747913+0000","last_fullsized":"2023-08-09T13:36:12.747913+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:36:10.826860+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:36:10.826860+0000","last_clean_scrub_stamp":"2023-08-09T13:36:10.826860+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T23:43:23.933266+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.6","version":"0'0","reported_seq":13,"reported_epoch":39,"state":"active+clean","last_fresh":"2023-08-09T13:36:13.183415+0000","last_change":"2023-08-09T13:36:10.871425+0000","last_active":"2023-08-09T13:36:13.183415+0000","last_peered":"2023-08-09T13:36:13.183415+0000","last_clean":"2023-08-09T13:36:13.183415+0000","last_became_active":"2023-08-09T13:36:10.871105+0000","last_became_peered":"2023-08-09T13:36:10.871105+0000","last_unstale":"2023-08-09T13:36:13.183415+0000","last_undegraded":"2023-08-09T13:36:13.183415+0000","last_fullsized":"2023-08-09T13:36:13.183415+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:36:09.813580+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:36:09.813580+0000","last_clean_scrub_stamp":"2023-08-09T13:36:09.813580+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T18:30:24.843777+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":50,"reported_epoch":39,"state":"active+clean","last_fresh":"2023-08-09T13:36:13.179994+0000","last_change":"2023-08-09T13:33:56.401123+0000","last_active":"2023-08-09T13:36:13.179994+0000","last_peered":"2023-08-09T13:36:13.179994+0000","last_clean":"2023-08-09T13:36:13.179994+0000","last_became_active":"2023-08-09T13:33:56.399933+0000","last_became_peered":"2023-08-09T13:33:56.399933+0000","last_unstale":"2023-08-09T13:36:13.179994+0000","last_undegraded":"2023-08-09T13:36:13.179994+0000","last_fullsized":"2023-08-09T13:36:13.179994+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:33:54.845869+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:33:54.845869+0000","last_clean_scrub_stamp":"2023-08-09T13:33:54.845869+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T21:23:42.787477+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.6","version":"0'0","reported_seq":11,"reported_epoch":39,"state":"active+clean","last_fresh":"2023-08-09T13:36:13.180703+0000","last_change":"2023-08-09T13:36:11.762581+0000","last_active":"2023-08-09T13:36:13.180703+0000","last_peered":"2023-08-09T13:36:13.180703+0000","last_clean":"2023-08-09T13:36:13.180703+0000","last_became_active":"2023-08-09T13:36:11.762279+0000","last_became_peered":"2023-08-09T13:36:11.762279+0000","last_unstale":"2023-08-09T13:36:13.180703+0000","last_undegraded":"2023-08-09T13:36:13.180703+0000","last_fullsized":"2023-08-09T13:36:13.180703+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:36:10.826860+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:36:10.826860+0000","last_clean_scrub_stamp":"2023-08-09T13:36:10.826860+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T20:44:32.204365+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.7","version":"38'2","reported_seq":15,"reported_epoch":39,"state":"active+clean","last_fresh":"2023-08-09T13:36:13.180695+0000","last_change":"2023-08-09T13:36:10.865646+0000","last_active":"2023-08-09T13:36:13.180695+0000","last_peered":"2023-08-09T13:36:13.180695+0000","last_clean":"2023-08-09T13:36:13.180695+0000","last_became_active":"2023-08-09T13:36:10.865350+0000","last_became_peered":"2023-08-09T13:36:10.865350+0000","last_unstale":"2023-08-09T13:36:13.180695+0000","last_undegraded":"2023-08-09T13:36:13.180695+0000","last_fullsized":"2023-08-09T13:36:13.180695+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:36:09.813580+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:36:09.813580+0000","last_clean_scrub_stamp":"2023-08-09T13:36:09.813580+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T20:57:41.010981+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":50,"reported_epoch":39,"state":"active+clean","last_fresh":"2023-08-09T13:36:13.180307+0000","last_change":"2023-08-09T13:33:56.401406+0000","last_active":"2023-08-09T13:36:13.180307+0000","last_peered":"2023-08-09T13:36:13.180307+0000","last_clean":"2023-08-09T13:36:13.180307+0000","last_became_active":"2023-08-09T13:33:56.400487+0000","last_became_peered":"2023-08-09T13:33:56.400487+0000","last_unstale":"2023-08-09T13:36:13.180307+0000","last_undegraded":"2023-08-09T13:36:13.180307+0000","last_fullsized":"2023-08-09T13:36:13.180307+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:33:54.845869+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:33:54.845869+0000","last_clean_scrub_stamp":"2023-08-09T13:33:54.845869+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T14:48:49.894855+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":7,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":6,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":38,"ondisk_log_size":38,"up":64,"acting":64,"num_store_stats":8},{"poolid":2,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":8}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574878,"num_pgs":9,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32140,"kb_used_data":1024,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":94339700,"statfs":{"total":96636764160,"available":96603852800,"internally_reserved":0,"allocated":1048576,"data_stored":794063,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1781,"internal_metadata":31848715},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Aug 9 13:35:55 2023","interfaces":[{"interface":"back","average":{"1min":1.026,"5min":1.1859999999999999,"15min":1.212},"min":{"1min":0.61099999999999999,"5min":0.61099999999999999,"15min":0.61099999999999999},"max":{"1min":1.734,"5min":2.1709999999999998,"15min":2.1709999999999998},"last":1},{"interface":"front","average":{"1min":1.1180000000000001,"5min":1.3220000000000001,"15min":1.3560000000000001},"min":{"1min":0.74099999999999999,"5min":0.74099999999999999,"15min":0.74099999999999999},"max":{"1min":1.603,"5min":2.6240000000000001,"15min":2.6240000000000001},"last":0.76100000000000001}]},{"osd":1,"last update":"Wed Aug 9 13:35:55 2023","interfaces":[{"interface":"back","average":{"1min":1.119,"5min":1.718,"15min":1.8180000000000001},"min":{"1min":0.68500000000000005,"5min":0.68500000000000005,"15min":0.68500000000000005},"max":{"1min":1.7909999999999999,"5min":10.823,"15min":10.823},"last":0.85099999999999998},{"interface":"front","average":{"1min":1.2709999999999999,"5min":1.645,"15min":1.7070000000000001},"min":{"1min":0.73199999999999998,"5min":0.73199999999999998,"15min":0.73199999999999998},"max":{"1min":1.736,"5min":10.576000000000001,"15min":10.576000000000001},"last":1.0900000000000001}]},{"osd":2,"last update":"Wed Aug 9 13:35:55 2023","interfaces":[{"interface":"back","average":{"1min":1.0589999999999999,"5min":1.3220000000000001,"15min":1.3660000000000001},"min":{"1min":0.79300000000000004,"5min":0.78400000000000003,"15min":0.78400000000000003},"max":{"1min":1.4079999999999999,"5min":3.5270000000000001,"15min":3.5270000000000001},"last":1.2689999999999999},{"interface":"front","average":{"1min":1.319,"5min":1.4239999999999999,"15min":1.4419999999999999},"min":{"1min":0.83899999999999997,"5min":0.82099999999999995,"15min":0.82099999999999995},"max":{"1min":1.9019999999999999,"5min":3.1190000000000002,"15min":3.1190000000000002},"last":1.157}]},{"osd":3,"last update":"Wed Aug 9 13:35:55 2023","interfaces":[{"interface":"back","average":{"1min":1.1919999999999999,"5min":1.3169999999999999,"15min":1.3380000000000001},"min":{"1min":0.90900000000000003,"5min":0.82999999999999996,"15min":0.82999999999999996},"max":{"1min":1.542,"5min":2.3010000000000002,"15min":2.3010000000000002},"last":1.5449999999999999},{"interface":"front","average":{"1min":1.3979999999999999,"5min":1.5580000000000001,"15min":1.5840000000000001},"min":{"1min":0.93600000000000005,"5min":0.93600000000000005,"15min":0.93600000000000005},"max":{"1min":2.0139999999999998,"5min":3.2770000000000001,"15min":3.2770000000000001},"last":1.2410000000000001}]},{"osd":4,"last update":"Wed Aug 9 13:35:55 2023","interfaces":[{"interface":"back","average":{"1min":1.278,"5min":1.478,"15min":1.5109999999999999},"min":{"1min":0.72899999999999998,"5min":0.72899999999999998,"15min":0.72899999999999998},"max":{"1min":1.857,"5min":3.0209999999999999,"15min":3.0209999999999999},"last":1.204},{"interface":"front","average":{"1min":1.3320000000000001,"5min":1.5129999999999999,"15min":1.5429999999999999},"min":{"1min":0.56799999999999995,"5min":0.56799999999999995,"15min":0.56799999999999995},"max":{"1min":1.653,"5min":3.169,"15min":3.169},"last":1.6160000000000001}]},{"osd":5,"last update":"Wed Aug 9 13:35:55 2023","interfaces":[{"interface":"back","average":{"1min":1.4690000000000001,"5min":1.6459999999999999,"15min":1.6759999999999999},"min":{"1min":0.87,"5min":0.87,"15min":0.87},"max":{"1min":2.113,"5min":3.5830000000000002,"15min":3.5830000000000002},"last":1.3520000000000001},{"interface":"front","average":{"1min":1.468,"5min":1.613,"15min":1.637},"min":{"1min":0.69199999999999995,"5min":0.69199999999999995,"15min":0.69199999999999995},"max":{"1min":1.919,"5min":3.3439999999999999,"15min":3.3439999999999999},"last":1.3999999999999999}]},{"osd":6,"last update":"Wed Aug 9 13:35:55 2023","interfaces":[{"interface":"back","average":{"1min":1.5509999999999999,"5min":1.716,"15min":1.744},"min":{"1min":1.0549999999999999,"5min":1.0549999999999999,"15min":1.0549999999999999},"max":{"1min":2.1520000000000001,"5min":3.2189999999999999,"15min":3.2189999999999999},"last":1.518},{"interface":"front","average":{"1min":1.4199999999999999,"5min":1.6200000000000001,"15min":1.653},"min":{"1min":0.60799999999999998,"5min":0.60799999999999998,"15min":0.60799999999999998},"max":{"1min":1.7549999999999999,"5min":2.9590000000000001,"15min":2.9590000000000001},"last":1.73}]}]},{"osd":6,"up_from":13,"seq":55834574878,"num_pgs":32,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27472,"kb_used_data":444,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344368,"statfs":{"total":96636764160,"available":96608632832,"internally_reserved":0,"allocated":454656,"data_stored":202916,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1781,"internal_metadata":27654411},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Aug 9 13:35:58 2023","interfaces":[{"interface":"back","average":{"1min":1.109,"5min":1.149,"15min":1.1559999999999999},"min":{"1min":0.56599999999999995,"5min":0.56599999999999995,"15min":0.56599999999999995},"max":{"1min":1.776,"5min":1.776,"15min":1.776},"last":0.82799999999999996},{"interface":"front","average":{"1min":1.0880000000000001,"5min":1.0940000000000001,"15min":1.095},"min":{"1min":0.76000000000000001,"5min":0.72999999999999998,"15min":0.72999999999999998},"max":{"1min":1.4259999999999999,"5min":1.7509999999999999,"15min":1.7509999999999999},"last":1.026}]},{"osd":1,"last update":"Wed Aug 9 13:35:58 2023","interfaces":[{"interface":"back","average":{"1min":1.004,"5min":1.167,"15min":1.194},"min":{"1min":0.65500000000000003,"5min":0.65500000000000003,"15min":0.65500000000000003},"max":{"1min":1.5229999999999999,"5min":1.77,"15min":1.77},"last":0.97799999999999998},{"interface":"front","average":{"1min":1.135,"5min":1.1659999999999999,"15min":1.171},"min":{"1min":0.68100000000000005,"5min":0.68100000000000005,"15min":0.68100000000000005},"max":{"1min":1.4610000000000001,"5min":1.6120000000000001,"15min":1.6120000000000001},"last":1.2989999999999999}]},{"osd":2,"last update":"Wed Aug 9 13:35:58 2023","interfaces":[{"interface":"back","average":{"1min":1.1499999999999999,"5min":1.1970000000000001,"15min":1.2050000000000001},"min":{"1min":0.68700000000000006,"5min":0.68700000000000006,"15min":0.68700000000000006},"max":{"1min":1.6599999999999999,"5min":1.6599999999999999,"15min":1.6599999999999999},"last":1.639},{"interface":"front","average":{"1min":1.141,"5min":1.161,"15min":1.165},"min":{"1min":0.82699999999999996,"5min":0.59499999999999997,"15min":0.59499999999999997},"max":{"1min":1.5840000000000001,"5min":1.5840000000000001,"15min":1.5840000000000001},"last":1.1839999999999999}]},{"osd":3,"last update":"Wed Aug 9 13:35:58 2023","interfaces":[{"interface":"back","average":{"1min":1.2589999999999999,"5min":1.2250000000000001,"15min":1.2190000000000001},"min":{"1min":0.92400000000000004,"5min":0.73799999999999999,"15min":0.73799999999999999},"max":{"1min":1.6439999999999999,"5min":1.724,"15min":1.724},"last":1.4710000000000001},{"interface":"front","average":{"1min":1.1990000000000001,"5min":1.282,"15min":1.296},"min":{"1min":0.80200000000000005,"5min":0.79600000000000004,"15min":0.79600000000000004},"max":{"1min":1.48,"5min":1.915,"15min":1.915},"last":1.3759999999999999}]},{"osd":4,"last update":"Wed Aug 9 13:35:58 2023","interfaces":[{"interface":"back","average":{"1min":1.2569999999999999,"5min":1.385,"15min":1.407},"min":{"1min":0.79200000000000004,"5min":0.79200000000000004,"15min":0.79200000000000004},"max":{"1min":1.6910000000000001,"5min":2.1269999999999998,"15min":2.1269999999999998},"last":1.5309999999999999},{"interface":"front","average":{"1min":1.375,"5min":1.3720000000000001,"15min":1.3720000000000001},"min":{"1min":0.69099999999999995,"5min":0.69099999999999995,"15min":0.69099999999999995},"max":{"1min":1.748,"5min":1.8999999999999999,"15min":1.8999999999999999},"last":1.583}]},{"osd":5,"last update":"Wed Aug 9 13:35:58 2023","interfaces":[{"interface":"back","average":{"1min":1.278,"5min":1.379,"15min":1.3959999999999999},"min":{"1min":0.65400000000000003,"5min":0.65400000000000003,"15min":0.65400000000000003},"max":{"1min":1.698,"5min":2.0910000000000002,"15min":2.0910000000000002},"last":1.8180000000000001},{"interface":"front","average":{"1min":1.492,"5min":1.5960000000000001,"15min":1.613},"min":{"1min":1.054,"5min":1.054,"15min":1.054},"max":{"1min":1.9830000000000001,"5min":2.516,"15min":2.516},"last":1.8799999999999999}]},{"osd":7,"last update":"Wed Aug 9 13:35:58 2023","interfaces":[{"interface":"back","average":{"1min":1.5389999999999999,"5min":1.5620000000000001,"15min":1.5660000000000001},"min":{"1min":0.77600000000000002,"5min":0.77600000000000002,"15min":0.77600000000000002},"max":{"1min":1.8999999999999999,"5min":2.3570000000000002,"15min":2.3570000000000002},"last":1.782},{"interface":"front","average":{"1min":1.524,"5min":1.5960000000000001,"15min":1.6080000000000001},"min":{"1min":0.95699999999999996,"5min":0.95699999999999996,"15min":0.95699999999999996},"max":{"1min":1.909,"5min":2.2210000000000001,"15min":2.2210000000000001},"last":1.6830000000000001}]}]},{"osd":5,"up_from":13,"seq":55834574878,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27464,"kb_used_data":444,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344376,"statfs":{"total":96636764160,"available":96608641024,"internally_reserved":0,"allocated":454656,"data_stored":203675,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1781,"internal_metadata":27654411},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Aug 9 13:35:58 2023","interfaces":[{"interface":"back","average":{"1min":1.1419999999999999,"5min":1.117,"15min":1.113},"min":{"1min":0.67700000000000005,"5min":0.67700000000000005,"15min":0.67700000000000005},"max":{"1min":1.599,"5min":1.7130000000000001,"15min":1.7130000000000001},"last":1.0429999999999999},{"interface":"front","average":{"1min":1.0800000000000001,"5min":1.157,"15min":1.1699999999999999},"min":{"1min":0.79400000000000004,"5min":0.78200000000000003,"15min":0.78200000000000003},"max":{"1min":1.496,"5min":1.6859999999999999,"15min":1.6859999999999999},"last":1.1040000000000001}]},{"osd":1,"last update":"Wed Aug 9 13:35:58 2023","interfaces":[{"interface":"back","average":{"1min":1.0629999999999999,"5min":1.1399999999999999,"15min":1.153},"min":{"1min":0.748,"5min":0.748,"15min":0.748},"max":{"1min":1.546,"5min":1.599,"15min":1.599},"last":1.407},{"interface":"front","average":{"1min":1.179,"5min":1.2150000000000001,"15min":1.2210000000000001},"min":{"1min":0.75800000000000001,"5min":0.75800000000000001,"15min":0.75800000000000001},"max":{"1min":1.4790000000000001,"5min":1.5640000000000001,"15min":1.5640000000000001},"last":0.92600000000000005}]},{"osd":2,"last update":"Wed Aug 9 13:35:58 2023","interfaces":[{"interface":"back","average":{"1min":1.2110000000000001,"5min":1.159,"15min":1.1499999999999999},"min":{"1min":0.72299999999999998,"5min":0.72299999999999998,"15min":0.72299999999999998},"max":{"1min":1.899,"5min":1.899,"15min":1.899},"last":0.85799999999999998},{"interface":"front","average":{"1min":1.153,"5min":1.202,"15min":1.21},"min":{"1min":0.66800000000000004,"5min":0.66800000000000004,"15min":0.66800000000000004},"max":{"1min":1.663,"5min":1.6739999999999999,"15min":1.6739999999999999},"last":0.98299999999999998}]},{"osd":3,"last update":"Wed Aug 9 13:35:58 2023","interfaces":[{"interface":"back","average":{"1min":1.3540000000000001,"5min":1.266,"15min":1.252},"min":{"1min":1.0389999999999999,"5min":0.79000000000000004,"15min":0.79000000000000004},"max":{"1min":2.2690000000000001,"5min":2.2690000000000001,"15min":2.2690000000000001},"last":1.147},{"interface":"front","average":{"1min":1.3939999999999999,"5min":1.4099999999999999,"15min":1.413},"min":{"1min":0.96499999999999997,"5min":0.96499999999999997,"15min":0.96499999999999997},"max":{"1min":2.028,"5min":2.028,"15min":2.028},"last":1.365}]},{"osd":4,"last update":"Wed Aug 9 13:35:58 2023","interfaces":[{"interface":"back","average":{"1min":1.24,"5min":1.286,"15min":1.294},"min":{"1min":0.61399999999999999,"5min":0.61399999999999999,"15min":0.61399999999999999},"max":{"1min":2.121,"5min":2.121,"15min":2.121},"last":1.1859999999999999},{"interface":"front","average":{"1min":1.389,"5min":1.4430000000000001,"15min":1.452},"min":{"1min":0.58599999999999997,"5min":0.58599999999999997,"15min":0.58599999999999997},"max":{"1min":2.3679999999999999,"5min":2.3679999999999999,"15min":2.3679999999999999},"last":1.302}]},{"osd":6,"last update":"Wed Aug 9 13:35:58 2023","interfaces":[{"interface":"back","average":{"1min":1.4710000000000001,"5min":1.4730000000000001,"15min":1.4730000000000001},"min":{"1min":0.77700000000000002,"5min":0.77700000000000002,"15min":0.77700000000000002},"max":{"1min":2.1110000000000002,"5min":2.1110000000000002,"15min":2.1110000000000002},"last":1.238},{"interface":"front","average":{"1min":1.462,"5min":1.494,"15min":1.4990000000000001},"min":{"1min":0.71199999999999997,"5min":0.71199999999999997,"15min":0.71199999999999997},"max":{"1min":2.2090000000000001,"5min":2.2090000000000001,"15min":2.2090000000000001},"last":1.512}]},{"osd":7,"last update":"Wed Aug 9 13:35:58 2023","interfaces":[{"interface":"back","average":{"1min":1.5389999999999999,"5min":1.597,"15min":1.607},"min":{"1min":0.70899999999999996,"5min":0.70899999999999996,"15min":0.70899999999999996},"max":{"1min":2.4729999999999999,"5min":2.4729999999999999,"15min":2.4729999999999999},"last":1.4470000000000001},{"interface":"front","average":{"1min":1.673,"5min":1.6379999999999999,"15min":1.6319999999999999},"min":{"1min":0.93100000000000005,"5min":0.93100000000000005,"15min":0.93100000000000005},"max":{"1min":2.2069999999999999,"5min":2.2069999999999999,"15min":2.2069999999999999},"last":1.466}]}]},{"osd":4,"up_from":13,"seq":55834574878,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27480,"kb_used_data":452,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344360,"statfs":{"total":96636764160,"available":96608624640,"internally_reserved":0,"allocated":462848,"data_stored":205217,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1781,"internal_metadata":27654411},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Aug 9 13:35:57 2023","interfaces":[{"interface":"back","average":{"1min":1.073,"5min":1.0860000000000001,"15min":1.0880000000000001},"min":{"1min":0.67700000000000005,"5min":0.64200000000000002,"15min":0.64200000000000002},"max":{"1min":1.3680000000000001,"5min":1.5860000000000001,"15min":1.5860000000000001},"last":1.4239999999999999},{"interface":"front","average":{"1min":1.1759999999999999,"5min":1.196,"15min":1.2},"min":{"1min":0.71199999999999997,"5min":0.69999999999999996,"15min":0.69999999999999996},"max":{"1min":1.6240000000000001,"5min":1.7909999999999999,"15min":1.7909999999999999},"last":1.361}]},{"osd":1,"last update":"Wed Aug 9 13:35:57 2023","interfaces":[{"interface":"back","average":{"1min":1.123,"5min":1.2909999999999999,"15min":1.319},"min":{"1min":0.77100000000000002,"5min":0.77100000000000002,"15min":0.77100000000000002},"max":{"1min":1.601,"5min":1.615,"15min":1.615},"last":1.127},{"interface":"front","average":{"1min":1.1419999999999999,"5min":1.1659999999999999,"15min":1.1699999999999999},"min":{"1min":0.64900000000000002,"5min":0.64900000000000002,"15min":0.64900000000000002},"max":{"1min":1.577,"5min":1.6259999999999999,"15min":1.6259999999999999},"last":1.179}]},{"osd":2,"last update":"Wed Aug 9 13:35:57 2023","interfaces":[{"interface":"back","average":{"1min":1.224,"5min":1.222,"15min":1.222},"min":{"1min":0.89400000000000002,"5min":0.89400000000000002,"15min":0.89400000000000002},"max":{"1min":1.681,"5min":1.681,"15min":1.681},"last":1.0609999999999999},{"interface":"front","average":{"1min":1.272,"5min":1.401,"15min":1.423},"min":{"1min":0.74299999999999999,"5min":0.74299999999999999,"15min":0.74299999999999999},"max":{"1min":1.5660000000000001,"5min":1.948,"15min":1.948},"last":1.0009999999999999}]},{"osd":3,"last update":"Wed Aug 9 13:35:57 2023","interfaces":[{"interface":"back","average":{"1min":1.167,"5min":1.3080000000000001,"15min":1.3320000000000001},"min":{"1min":0.93400000000000005,"5min":0.88500000000000001,"15min":0.88500000000000001},"max":{"1min":1.48,"5min":2.3780000000000001,"15min":2.3780000000000001},"last":0.85099999999999998},{"interface":"front","average":{"1min":1.222,"5min":1.353,"15min":1.375},"min":{"1min":0.91000000000000003,"5min":0.73099999999999998,"15min":0.73099999999999998},"max":{"1min":1.6180000000000001,"5min":2.113,"15min":2.113},"last":0.93400000000000005}]},{"osd":5,"last update":"Wed Aug 9 13:35:57 2023","interfaces":[{"interface":"back","average":{"1min":1.345,"5min":1.409,"15min":1.419},"min":{"1min":0.86099999999999999,"5min":0.86099999999999999,"15min":0.86099999999999999},"max":{"1min":1.7070000000000001,"5min":2.2959999999999998,"15min":2.2959999999999998},"last":1.472},{"interface":"front","average":{"1min":1.375,"5min":1.343,"15min":1.3380000000000001},"min":{"1min":0.92800000000000005,"5min":0.92800000000000005,"15min":0.92800000000000005},"max":{"1min":1.7929999999999999,"5min":1.8859999999999999,"15min":1.8859999999999999},"last":1.3200000000000001}]},{"osd":6,"last update":"Wed Aug 9 13:35:57 2023","interfaces":[{"interface":"back","average":{"1min":1.4379999999999999,"5min":1.405,"15min":1.399},"min":{"1min":1.091,"5min":0.82499999999999996,"15min":0.82499999999999996},"max":{"1min":1.669,"5min":1.9379999999999999,"15min":1.9379999999999999},"last":1.272},{"interface":"front","average":{"1min":1.508,"5min":1.5449999999999999,"15min":1.5509999999999999},"min":{"1min":0.997,"5min":0.997,"15min":0.997},"max":{"1min":1.6879999999999999,"5min":2.1899999999999999,"15min":2.1899999999999999},"last":1.5900000000000001}]},{"osd":7,"last update":"Wed Aug 9 13:35:57 2023","interfaces":[{"interface":"back","average":{"1min":1.4630000000000001,"5min":1.51,"15min":1.518},"min":{"1min":1.232,"5min":1.1359999999999999,"15min":1.1359999999999999},"max":{"1min":1.6559999999999999,"5min":2.2210000000000001,"15min":2.2210000000000001},"last":1.397},{"interface":"front","average":{"1min":1.5449999999999999,"5min":1.5569999999999999,"15min":1.5589999999999999},"min":{"1min":1.3500000000000001,"5min":1.1679999999999999,"15min":1.1679999999999999},"max":{"1min":1.788,"5min":2.0489999999999999,"15min":2.0489999999999999},"last":1.542}]}]},{"osd":3,"up_from":13,"seq":55834574879,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31568,"kb_used_data":452,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":94340272,"statfs":{"total":96636764160,"available":96604438528,"internally_reserved":0,"allocated":462848,"data_stored":204243,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1781,"internal_metadata":31848715},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Aug 9 13:35:55 2023","interfaces":[{"interface":"back","average":{"1min":0.92500000000000004,"5min":0.93200000000000005,"15min":0.93300000000000005},"min":{"1min":0.38800000000000001,"5min":0.38800000000000001,"15min":0.38800000000000001},"max":{"1min":1.9199999999999999,"5min":1.944,"15min":1.944},"last":1.462},{"interface":"front","average":{"1min":0.96899999999999997,"5min":0.91700000000000004,"15min":0.90900000000000003},"min":{"1min":0.44600000000000001,"5min":0.40799999999999997,"15min":0.40799999999999997},"max":{"1min":2.2360000000000002,"5min":2.2360000000000002,"15min":2.2360000000000002},"last":0.92100000000000004}]},{"osd":1,"last update":"Wed Aug 9 13:35:55 2023","interfaces":[{"interface":"back","average":{"1min":0.875,"5min":0.86299999999999999,"15min":0.86099999999999999},"min":{"1min":0.51100000000000001,"5min":0.46200000000000002,"15min":0.46200000000000002},"max":{"1min":1.387,"5min":1.6919999999999999,"15min":1.6919999999999999},"last":0.80100000000000005},{"interface":"front","average":{"1min":1.024,"5min":1.0840000000000001,"15min":1.0940000000000001},"min":{"1min":0.42099999999999999,"5min":0.42099999999999999,"15min":0.42099999999999999},"max":{"1min":1.6439999999999999,"5min":2.3359999999999999,"15min":2.3359999999999999},"last":1.0509999999999999}]},{"osd":2,"last update":"Wed Aug 9 13:35:55 2023","interfaces":[{"interface":"back","average":{"1min":0.98299999999999998,"5min":0.91500000000000004,"15min":0.90300000000000002},"min":{"1min":0.53500000000000003,"5min":0.53500000000000003,"15min":0.53500000000000003},"max":{"1min":1.5069999999999999,"5min":1.5629999999999999,"15min":1.5629999999999999},"last":1.127},{"interface":"front","average":{"1min":1.109,"5min":1.208,"15min":1.224},"min":{"1min":0.52000000000000002,"5min":0.52000000000000002,"15min":0.52000000000000002},"max":{"1min":1.5409999999999999,"5min":2.1429999999999998,"15min":2.1429999999999998},"last":1.593}]},{"osd":4,"last update":"Wed Aug 9 13:35:55 2023","interfaces":[{"interface":"back","average":{"1min":1.228,"5min":1.2589999999999999,"15min":1.264},"min":{"1min":0.66500000000000004,"5min":0.66500000000000004,"15min":0.66500000000000004},"max":{"1min":1.7090000000000001,"5min":1.992,"15min":1.992},"last":1.3140000000000001},{"interface":"front","average":{"1min":1.2150000000000001,"5min":1.3460000000000001,"15min":1.3680000000000001},"min":{"1min":0.79400000000000004,"5min":0.79400000000000004,"15min":0.79400000000000004},"max":{"1min":1.869,"5min":3.0649999999999999,"15min":3.0649999999999999},"last":1.226}]},{"osd":5,"last update":"Wed Aug 9 13:35:55 2023","interfaces":[{"interface":"back","average":{"1min":1.323,"5min":1.3919999999999999,"15min":1.4039999999999999},"min":{"1min":0.73099999999999998,"5min":0.73099999999999998,"15min":0.73099999999999998},"max":{"1min":1.9570000000000001,"5min":2.3610000000000002,"15min":2.3610000000000002},"last":1.5580000000000001},{"interface":"front","average":{"1min":1.339,"5min":1.407,"15min":1.4179999999999999},"min":{"1min":0.81799999999999995,"5min":0.79100000000000004,"15min":0.79100000000000004},"max":{"1min":1.827,"5min":2.2639999999999998,"15min":2.2639999999999998},"last":1.7090000000000001}]},{"osd":6,"last update":"Wed Aug 9 13:35:55 2023","interfaces":[{"interface":"back","average":{"1min":1.323,"5min":1.371,"15min":1.379},"min":{"1min":0.86599999999999999,"5min":0.82299999999999995,"15min":0.82299999999999995},"max":{"1min":1.78,"5min":2.1629999999999998,"15min":2.1629999999999998},"last":1.516},{"interface":"front","average":{"1min":1.3080000000000001,"5min":1.452,"15min":1.476},"min":{"1min":0.68600000000000005,"5min":0.68600000000000005,"15min":0.68600000000000005},"max":{"1min":2.0169999999999999,"5min":3.0049999999999999,"15min":3.0049999999999999},"last":1.6279999999999999}]},{"osd":7,"last update":"Wed Aug 9 13:35:55 2023","interfaces":[{"interface":"back","average":{"1min":1.4390000000000001,"5min":1.5569999999999999,"15min":1.577},"min":{"1min":0.78800000000000003,"5min":0.78800000000000003,"15min":0.78800000000000003},"max":{"1min":1.9099999999999999,"5min":2.6120000000000001,"15min":2.6120000000000001},"last":1.8149999999999999},{"interface":"front","average":{"1min":1.3819999999999999,"5min":1.482,"15min":1.4990000000000001},"min":{"1min":0.61599999999999999,"5min":0.61599999999999999,"15min":0.61599999999999999},"max":{"1min":1.843,"5min":2.2989999999999999,"15min":2.2989999999999999},"last":1.766}]}]},{"osd":2,"up_from":13,"seq":55834574878,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31572,"kb_used_data":456,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":94340268,"statfs":{"total":96636764160,"available":96604434432,"internally_reserved":0,"allocated":466944,"data_stored":205239,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1781,"internal_metadata":31848715},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Aug 9 13:35:56 2023","interfaces":[{"interface":"back","average":{"1min":0.81399999999999995,"5min":0.90000000000000002,"15min":0.91400000000000003},"min":{"1min":0.41499999999999998,"5min":0.41499999999999998,"15min":0.41499999999999998},"max":{"1min":1.1930000000000001,"5min":1.542,"15min":1.542},"last":1.8380000000000001},{"interface":"front","average":{"1min":0.96199999999999997,"5min":1.0680000000000001,"15min":1.0860000000000001},"min":{"1min":0.51900000000000002,"5min":0.45300000000000001,"15min":0.45300000000000001},"max":{"1min":1.5309999999999999,"5min":1.7549999999999999,"15min":1.7549999999999999},"last":1.0289999999999999}]},{"osd":1,"last update":"Wed Aug 9 13:35:56 2023","interfaces":[{"interface":"back","average":{"1min":1.091,"5min":1.139,"15min":1.147},"min":{"1min":0.55400000000000005,"5min":0.54400000000000004,"15min":0.54400000000000004},"max":{"1min":1.6759999999999999,"5min":2.0249999999999999,"15min":2.0249999999999999},"last":0.96199999999999997},{"interface":"front","average":{"1min":1.1819999999999999,"5min":1.2310000000000001,"15min":1.2390000000000001},"min":{"1min":0.64600000000000002,"5min":0.52900000000000003,"15min":0.52900000000000003},"max":{"1min":1.925,"5min":1.9930000000000001,"15min":1.9930000000000001},"last":0.89400000000000002}]},{"osd":3,"last update":"Wed Aug 9 13:35:56 2023","interfaces":[{"interface":"back","average":{"1min":0.876,"5min":0.98299999999999998,"15min":1.0009999999999999},"min":{"1min":0.46800000000000003,"5min":0.42599999999999999,"15min":0.42599999999999999},"max":{"1min":1.524,"5min":1.635,"15min":1.635},"last":1.0640000000000001},{"interface":"front","average":{"1min":1.085,"5min":1.3360000000000001,"15min":1.3779999999999999},"min":{"1min":0.624,"5min":0.55300000000000005,"15min":0.55300000000000005},"max":{"1min":1.5700000000000001,"5min":3.024,"15min":3.024},"last":1.399}]},{"osd":4,"last update":"Wed Aug 9 13:35:56 2023","interfaces":[{"interface":"back","average":{"1min":1.3100000000000001,"5min":1.4790000000000001,"15min":1.5069999999999999},"min":{"1min":0.88,"5min":0.77900000000000003,"15min":0.77900000000000003},"max":{"1min":1.593,"5min":3.137,"15min":3.137},"last":1.6539999999999999},{"interface":"front","average":{"1min":1.347,"5min":1.431,"15min":1.4450000000000001},"min":{"1min":0.92600000000000005,"5min":0.79000000000000004,"15min":0.79000000000000004},"max":{"1min":1.756,"5min":2.5150000000000001,"15min":2.5150000000000001},"last":1.593}]},{"osd":5,"last update":"Wed Aug 9 13:35:56 2023","interfaces":[{"interface":"back","average":{"1min":1.347,"5min":1.458,"15min":1.476},"min":{"1min":0.94099999999999995,"5min":0.75900000000000001,"15min":0.75900000000000001},"max":{"1min":1.8540000000000001,"5min":2.3839999999999999,"15min":2.3839999999999999},"last":1.776},{"interface":"front","average":{"1min":1.254,"5min":1.4510000000000001,"15min":1.484},"min":{"1min":0.90600000000000003,"5min":0.90600000000000003,"15min":0.90600000000000003},"max":{"1min":1.6080000000000001,"5min":3.8599999999999999,"15min":3.8599999999999999},"last":1.282}]},{"osd":6,"last update":"Wed Aug 9 13:35:56 2023","interfaces":[{"interface":"back","average":{"1min":1.2509999999999999,"5min":1.389,"15min":1.4119999999999999},"min":{"1min":0.94199999999999995,"5min":0.80700000000000005,"15min":0.80700000000000005},"max":{"1min":1.659,"5min":2.476,"15min":2.476},"last":1.3620000000000001},{"interface":"front","average":{"1min":1.429,"5min":1.6120000000000001,"15min":1.6419999999999999},"min":{"1min":1.0269999999999999,"5min":1.0269999999999999,"15min":1.0269999999999999},"max":{"1min":1.798,"5min":3.5350000000000001,"15min":3.5350000000000001},"last":1.9419999999999999}]},{"osd":7,"last update":"Wed Aug 9 13:35:56 2023","interfaces":[{"interface":"back","average":{"1min":1.4790000000000001,"5min":1.7170000000000001,"15min":1.7569999999999999},"min":{"1min":0.90000000000000002,"5min":0.90000000000000002,"15min":0.90000000000000002},"max":{"1min":1.988,"5min":5.2969999999999997,"15min":5.2969999999999997},"last":1.899},{"interface":"front","average":{"1min":1.3300000000000001,"5min":1.617,"15min":1.665},"min":{"1min":0.94999999999999996,"5min":0.85899999999999999,"15min":0.85899999999999999},"max":{"1min":1.7230000000000001,"5min":5.4539999999999997,"15min":5.4539999999999997},"last":1.323}]}]},{"osd":1,"up_from":13,"seq":55834574878,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32140,"kb_used_data":1024,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":94339700,"statfs":{"total":96636764160,"available":96603852800,"internally_reserved":0,"allocated":1048576,"data_stored":794043,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1781,"internal_metadata":31848715},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Aug 9 13:35:57 2023","interfaces":[{"interface":"back","average":{"1min":0.80500000000000005,"5min":0.94899999999999995,"15min":0.97299999999999998},"min":{"1min":0.32400000000000001,"5min":0.32400000000000001,"15min":0.32400000000000001},"max":{"1min":1.369,"5min":2.7080000000000002,"15min":2.7080000000000002},"last":0.93000000000000005},{"interface":"front","average":{"1min":0.82999999999999996,"5min":1.02,"15min":1.052},"min":{"1min":0.42499999999999999,"5min":0.42499999999999999,"15min":0.42499999999999999},"max":{"1min":1.2769999999999999,"5min":2.6349999999999998,"15min":2.6349999999999998},"last":1.091}]},{"osd":2,"last update":"Wed Aug 9 13:35:57 2023","interfaces":[{"interface":"back","average":{"1min":0.86499999999999999,"5min":0.95199999999999996,"15min":0.96599999999999997},"min":{"1min":0.54200000000000004,"5min":0.54200000000000004,"15min":0.54200000000000004},"max":{"1min":1.2230000000000001,"5min":1.762,"15min":1.762},"last":1.0149999999999999},{"interface":"front","average":{"1min":0.93700000000000006,"5min":1.141,"15min":1.175},"min":{"1min":0.46500000000000002,"5min":0.46500000000000002,"15min":0.46500000000000002},"max":{"1min":1.7,"5min":2.2330000000000001,"15min":2.2330000000000001},"last":1.0469999999999999}]},{"osd":3,"last update":"Wed Aug 9 13:35:57 2023","interfaces":[{"interface":"back","average":{"1min":1.0309999999999999,"5min":1.177,"15min":1.2010000000000001},"min":{"1min":0.432,"5min":0.432,"15min":0.432},"max":{"1min":1.5820000000000001,"5min":1.5820000000000001,"15min":1.5820000000000001},"last":1.3169999999999999},{"interface":"front","average":{"1min":0.96399999999999997,"5min":1.216,"15min":1.258},"min":{"1min":0.44400000000000001,"5min":0.44400000000000001,"15min":0.44400000000000001},"max":{"1min":1.4570000000000001,"5min":3.3069999999999999,"15min":3.3069999999999999},"last":0.97299999999999998}]},{"osd":4,"last update":"Wed Aug 9 13:35:57 2023","interfaces":[{"interface":"back","average":{"1min":1.216,"5min":1.3380000000000001,"15min":1.3580000000000001},"min":{"1min":0.79000000000000004,"5min":0.79000000000000004,"15min":0.79000000000000004},"max":{"1min":1.569,"5min":2.504,"15min":2.504},"last":1.163},{"interface":"front","average":{"1min":1.2230000000000001,"5min":1.3440000000000001,"15min":1.3640000000000001},"min":{"1min":0.76400000000000001,"5min":0.76400000000000001,"15min":0.76400000000000001},"max":{"1min":1.7470000000000001,"5min":2.5099999999999998,"15min":2.5099999999999998},"last":1.1319999999999999}]},{"osd":5,"last update":"Wed Aug 9 13:35:57 2023","interfaces":[{"interface":"back","average":{"1min":1.252,"5min":1.3799999999999999,"15min":1.401},"min":{"1min":0.82699999999999996,"5min":0.82699999999999996,"15min":0.82699999999999996},"max":{"1min":2.1040000000000001,"5min":2.2570000000000001,"15min":2.2570000000000001},"last":1.425},{"interface":"front","average":{"1min":1.351,"5min":1.671,"15min":1.724},"min":{"1min":0.77700000000000002,"5min":0.77700000000000002,"15min":0.77700000000000002},"max":{"1min":2.024,"5min":4.9560000000000004,"15min":4.9560000000000004},"last":1.4770000000000001}]},{"osd":6,"last update":"Wed Aug 9 13:35:57 2023","interfaces":[{"interface":"back","average":{"1min":1.3149999999999999,"5min":1.484,"15min":1.512},"min":{"1min":0.78400000000000003,"5min":0.78400000000000003,"15min":0.78400000000000003},"max":{"1min":1.9810000000000001,"5min":3.0529999999999999,"15min":3.0529999999999999},"last":1.276},{"interface":"front","average":{"1min":1.288,"5min":1.5029999999999999,"15min":1.5389999999999999},"min":{"1min":0.79800000000000004,"5min":0.79800000000000004,"15min":0.79800000000000004},"max":{"1min":1.9410000000000001,"5min":2.7330000000000001,"15min":2.7330000000000001},"last":1.242}]},{"osd":7,"last update":"Wed Aug 9 13:35:57 2023","interfaces":[{"interface":"back","average":{"1min":1.3839999999999999,"5min":1.7330000000000001,"15min":1.7909999999999999},"min":{"1min":0.77100000000000002,"5min":0.77100000000000002,"15min":0.77100000000000002},"max":{"1min":2.1440000000000001,"5min":5.1360000000000001,"15min":5.1360000000000001},"last":1.6259999999999999},{"interface":"front","average":{"1min":1.371,"5min":1.5880000000000001,"15min":1.6240000000000001},"min":{"1min":0.75900000000000001,"5min":0.75900000000000001,"15min":0.75900000000000001},"max":{"1min":2.0720000000000001,"5min":3.194,"15min":3.194},"last":1.375}]}]},{"osd":0,"up_from":13,"seq":55834574878,"num_pgs":34,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31556,"kb_used_data":440,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":94340284,"statfs":{"total":96636764160,"available":96604450816,"internally_reserved":0,"allocated":450560,"data_stored":202278,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1781,"internal_metadata":31848715},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Wed Aug 9 13:36:00 2023","interfaces":[{"interface":"back","average":{"1min":0.88700000000000001,"5min":0.88,"15min":0.879},"min":{"1min":0.39600000000000002,"5min":0.39600000000000002,"15min":0.39600000000000002},"max":{"1min":1.4890000000000001,"5min":1.4890000000000001,"15min":1.4890000000000001},"last":1.1279999999999999},{"interface":"front","average":{"1min":1.0289999999999999,"5min":0.97499999999999998,"15min":0.96599999999999997},"min":{"1min":0.57399999999999995,"5min":0.53200000000000003,"15min":0.53200000000000003},"max":{"1min":1.581,"5min":1.581,"15min":1.581},"last":1.0509999999999999}]},{"osd":2,"last update":"Wed Aug 9 13:36:00 2023","interfaces":[{"interface":"back","average":{"1min":1.0269999999999999,"5min":1.071,"15min":1.079},"min":{"1min":0.44800000000000001,"5min":0.44800000000000001,"15min":0.44800000000000001},"max":{"1min":1.4990000000000001,"5min":1.7629999999999999,"15min":1.7629999999999999},"last":1.5569999999999999},{"interface":"front","average":{"1min":1.0589999999999999,"5min":1.1100000000000001,"15min":1.1180000000000001},"min":{"1min":0.52700000000000002,"5min":0.52700000000000002,"15min":0.52700000000000002},"max":{"1min":2.2519999999999998,"5min":2.2519999999999998,"15min":2.2519999999999998},"last":1.294}]},{"osd":3,"last update":"Wed Aug 9 13:36:00 2023","interfaces":[{"interface":"back","average":{"1min":1.0669999999999999,"5min":1.0569999999999999,"15min":1.0549999999999999},"min":{"1min":0.55200000000000005,"5min":0.55000000000000004,"15min":0.55000000000000004},"max":{"1min":2.5190000000000001,"5min":2.5649999999999999,"15min":2.5649999999999999},"last":0.76800000000000002},{"interface":"front","average":{"1min":1.242,"5min":1.2829999999999999,"15min":1.29},"min":{"1min":0.58799999999999997,"5min":0.58799999999999997,"15min":0.58799999999999997},"max":{"1min":2.4260000000000002,"5min":2.6819999999999999,"15min":2.6819999999999999},"last":1.46}]},{"osd":4,"last update":"Wed Aug 9 13:36:00 2023","interfaces":[{"interface":"back","average":{"1min":1.3160000000000001,"5min":1.3080000000000001,"15min":1.306},"min":{"1min":0.87,"5min":0.87,"15min":0.87},"max":{"1min":1.6870000000000001,"5min":1.6890000000000001,"15min":1.6890000000000001},"last":1.2430000000000001},{"interface":"front","average":{"1min":1.357,"5min":1.4370000000000001,"15min":1.45},"min":{"1min":0.74099999999999999,"5min":0.74099999999999999,"15min":0.74099999999999999},"max":{"1min":1.8959999999999999,"5min":2.181,"15min":2.181},"last":1.3260000000000001}]},{"osd":5,"last update":"Wed Aug 9 13:36:00 2023","interfaces":[{"interface":"back","average":{"1min":1.4259999999999999,"5min":1.4219999999999999,"15min":1.4219999999999999},"min":{"1min":0.93400000000000005,"5min":0.89200000000000002,"15min":0.89200000000000002},"max":{"1min":1.7150000000000001,"5min":2.085,"15min":2.085},"last":1.363},{"interface":"front","average":{"1min":1.365,"5min":1.3620000000000001,"15min":1.3620000000000001},"min":{"1min":0.79600000000000004,"5min":0.79600000000000004,"15min":0.79600000000000004},"max":{"1min":1.825,"5min":1.825,"15min":1.825},"last":1.3819999999999999}]},{"osd":6,"last update":"Wed Aug 9 13:36:00 2023","interfaces":[{"interface":"back","average":{"1min":1.47,"5min":1.546,"15min":1.5589999999999999},"min":{"1min":0.69799999999999995,"5min":0.69799999999999995,"15min":0.69799999999999995},"max":{"1min":1.867,"5min":2.2509999999999999,"15min":2.2509999999999999},"last":1.6719999999999999},{"interface":"front","average":{"1min":1.5269999999999999,"5min":1.4990000000000001,"15min":1.4950000000000001},"min":{"1min":1.083,"5min":0.96499999999999997,"15min":0.96499999999999997},"max":{"1min":1.998,"5min":2.2959999999999998,"15min":2.2959999999999998},"last":1.4239999999999999}]},{"osd":7,"last update":"Wed Aug 9 13:36:00 2023","interfaces":[{"interface":"back","average":{"1min":1.617,"5min":1.5780000000000001,"15min":1.5720000000000001},"min":{"1min":0.99199999999999999,"5min":0.99199999999999999,"15min":0.99199999999999999},"max":{"1min":2.464,"5min":2.464,"15min":2.464},"last":1.526},{"interface":"front","average":{"1min":1.569,"5min":1.589,"15min":1.5920000000000001},"min":{"1min":0.83199999999999996,"5min":0.83199999999999996,"15min":0.83199999999999996},"max":{"1min":2.0019999999999998,"5min":2.024,"15min":2.024},"last":1.605}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":34,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":70,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1634,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":638,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":70,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":672,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-08-09T13:36:16.957 INFO:tasks.cephfs.filesystem.ceph_manager:clean! 2023-08-09T13:36:16.958 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth caps client.0 mon 'allow r' osd 'allow rw tag cephfs data=cephfs' mds allow 2023-08-09T13:36:17.223 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:17.221+0000 7ff4db0f3640 1 Processor -- start 2023-08-09T13:36:17.224 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:17.222+0000 7ff4db0f3640 1 -- start start 2023-08-09T13:36:17.225 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:17.223+0000 7ff4db0f3640 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7ff4d40c90a0 0x7ff4d415af10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:36:17.225 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:17.223+0000 7ff4db0f3640 1 -- --> v1:172.21.15.43:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff4d4158280 con 0x7ff4d40c9ee0 2023-08-09T13:36:17.225 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:17.223+0000 7ff4db0f3640 1 -- --> v1:172.21.15.203:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff4d415b560 con 0x7ff4d4157ea0 2023-08-09T13:36:17.225 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:17.223+0000 7ff4db0f3640 1 -- --> v2:172.21.15.203:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff4d415b840 con 0x7ff4d40c90a0 2023-08-09T13:36:17.227 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:17.224+0000 7ff4d98f0640 1 --1- >> v1:172.21.15.43:6789/0 conn(0x7ff4d40c9ee0 0x7ff4d415a800 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.43:6789/0 says I am v1:172.21.15.43:35192/0 (socket says 172.21.15.43:35192) 2023-08-09T13:36:17.227 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:17.224+0000 7ff4d98f0640 1 -- 172.21.15.43:0/900098279 learned_addr learned my addr 172.21.15.43:0/900098279 (peer_addr_for_me v1:172.21.15.43:0/0) 2023-08-09T13:36:17.228 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:17.224+0000 7ff4da8f2640 1 --2- 172.21.15.43:0/900098279 >> v2:172.21.15.203:3300/0 conn(0x7ff4d40c90a0 0x7ff4d415af10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:36:17.228 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:17.224+0000 7ff4d90ef640 1 -- 172.21.15.43:0/900098279 <== mon.0 v1:172.21.15.43:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3825714289 0 0) 0x7ff4c4001110 con 0x7ff4d40c9ee0 2023-08-09T13:36:17.228 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:17.224+0000 7ff4d90ef640 1 -- 172.21.15.43:0/900098279 --> v1:172.21.15.43:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ff4c0003520 con 0x7ff4d40c9ee0 2023-08-09T13:36:17.228 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:17.225+0000 7ff4d90ef640 1 -- 172.21.15.43:0/900098279 <== mon.0 v1:172.21.15.43:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1220075302 0 0) 0x7ff4c0003520 con 0x7ff4d40c9ee0 2023-08-09T13:36:17.228 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:17.225+0000 7ff4d90ef640 1 -- 172.21.15.43:0/900098279 >> v2:172.21.15.203:3300/0 conn(0x7ff4d40c90a0 msgr2=0x7ff4d415af10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-09T13:36:17.228 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:17.225+0000 7ff4d90ef640 1 --2- 172.21.15.43:0/900098279 >> v2:172.21.15.203:3300/0 conn(0x7ff4d40c90a0 0x7ff4d415af10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:36:17.228 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:17.225+0000 7ff4d90ef640 1 -- 172.21.15.43:0/900098279 >> v1:172.21.15.203:6789/0 conn(0x7ff4d4157ea0 legacy=0x7ff4d40c97d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:36:17.230 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:17.225+0000 7ff4d90ef640 1 -- 172.21.15.43:0/900098279 --> v1:172.21.15.43:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff4d40c85c0 con 0x7ff4d40c9ee0 2023-08-09T13:36:17.230 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:17.225+0000 7ff4d90ef640 1 -- 172.21.15.43:0/900098279 <== mon.0 v1:172.21.15.43:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1285150740 0 0) 0x7ff4c4001800 con 0x7ff4d40c9ee0 2023-08-09T13:36:17.230 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:17.225+0000 7ff4d90ef640 1 -- 172.21.15.43:0/900098279 >> v1:172.21.15.43:6789/0 conn(0x7ff4d40c9ee0 legacy=0x7ff4d415a800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:36:17.230 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:17.225+0000 7ff4d90ef640 1 --2- 172.21.15.43:0/900098279 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff4c0003970 0x7ff4c0003d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:36:17.230 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:17.225+0000 7ff4d90ef640 1 --2- 172.21.15.43:0/900098279 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7ff4c00047c0 0x7ff4c0004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:36:17.231 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:17.225+0000 7ff4d90ef640 1 --2- 172.21.15.43:0/900098279 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7ff4c0005180 0x7ff4c000b980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:36:17.231 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:17.225+0000 7ff4d90ef640 1 -- 172.21.15.43:0/900098279 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff4d415b840 con 0x7ff4c0003970 2023-08-09T13:36:17.231 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:17.225+0000 7ff4d90ef640 1 -- 172.21.15.43:0/900098279 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff4c000c020 con 0x7ff4c0005180 2023-08-09T13:36:17.231 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:17.225+0000 7ff4d90ef640 1 -- 172.21.15.43:0/900098279 --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff4c000c1e0 con 0x7ff4c00047c0 2023-08-09T13:36:17.231 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:17.225+0000 7ff4da0f1640 1 --2- 172.21.15.43:0/900098279 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff4c0003970 0x7ff4c0003d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:36:17.231 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:17.226+0000 7ff4da8f2640 1 --2- 172.21.15.43:0/900098279 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7ff4c0005180 0x7ff4c000b980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:36:17.231 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:17.226+0000 7ff4da0f1640 1 -- 172.21.15.43:0/900098279 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7ff4c00047c0 msgr2=0x7ff4c0004c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-09T13:36:17.232 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:17.226+0000 7ff4d98f0640 1 --2- 172.21.15.43:0/900098279 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7ff4c00047c0 0x7ff4c0004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:36:17.232 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:17.226+0000 7ff4da0f1640 1 --2- 172.21.15.43:0/900098279 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7ff4c00047c0 0x7ff4c0004c40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:36:17.232 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:17.226+0000 7ff4da0f1640 1 -- 172.21.15.43:0/900098279 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7ff4c0005180 msgr2=0x7ff4c000b980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-09T13:36:17.232 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:17.226+0000 7ff4da0f1640 1 --2- 172.21.15.43:0/900098279 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7ff4c0005180 0x7ff4c000b980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:36:17.232 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:17.226+0000 7ff4da0f1640 1 -- 172.21.15.43:0/900098279 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff4c0003520 con 0x7ff4c0003970 2023-08-09T13:36:17.232 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:17.226+0000 7ff4da8f2640 1 --2- 172.21.15.43:0/900098279 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7ff4c0005180 0x7ff4c000b980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-08-09T13:36:17.232 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:17.226+0000 7ff4da8f2640 1 -- 172.21.15.43:0/900098279 reap_dead start 2023-08-09T13:36:17.235 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:17.227+0000 7ff4da0f1640 1 --2- 172.21.15.43:0/900098279 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff4c0003970 0x7ff4c0003d90 secure :-1 s=READY pgs=578 cs=0 l=1 rev1=1 crypto rx=0x7ff4d0001750 tx=0x7ff4c00057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=deb1ce9c5c2343a8 server_cookie=0 in_seq=0 out_seq=0 2023-08-09T13:36:17.235 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:17.227+0000 7ff4d90ef640 1 -- 172.21.15.43:0/900098279 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff4d00052d0 con 0x7ff4c0003970 2023-08-09T13:36:17.235 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:17.227+0000 7ff4d90ef640 1 -- 172.21.15.43:0/900098279 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff4d002e050 con 0x7ff4c0003970 2023-08-09T13:36:17.235 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:17.227+0000 7ff4d90ef640 1 -- 172.21.15.43:0/900098279 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff4d0003830 con 0x7ff4c0003970 2023-08-09T13:36:17.235 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:17.228+0000 7ff4db0f3640 1 -- 172.21.15.43:0/900098279 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff4c0003970 msgr2=0x7ff4c0003d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:36:17.235 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:17.228+0000 7ff4db0f3640 1 --2- 172.21.15.43:0/900098279 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff4c0003970 0x7ff4c0003d90 secure :-1 s=READY pgs=578 cs=0 l=1 rev1=1 crypto rx=0x7ff4d0001750 tx=0x7ff4c00057f0 comp rx=0 tx=0).stop 2023-08-09T13:36:17.235 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:17.228+0000 7ff4db0f3640 1 -- 172.21.15.43:0/900098279 shutdown_connections 2023-08-09T13:36:17.236 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:17.228+0000 7ff4db0f3640 1 --2- 172.21.15.43:0/900098279 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff4c0003970 0x7ff4c0003d90 unknown :-1 s=CLOSED pgs=578 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:36:17.236 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:17.228+0000 7ff4db0f3640 1 -- 172.21.15.43:0/900098279 >> 172.21.15.43:0/900098279 conn(0x7ff4d40cddd0 msgr2=0x7ff4d40ce1e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-09T13:36:17.236 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:17.228+0000 7ff4db0f3640 1 -- 172.21.15.43:0/900098279 shutdown_connections 2023-08-09T13:36:17.236 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:17.228+0000 7ff4db0f3640 1 -- 172.21.15.43:0/900098279 wait complete. 2023-08-09T13:36:17.236 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:17.230+0000 7ff4db0f3640 1 Processor -- start 2023-08-09T13:36:17.236 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:17.230+0000 7ff4db0f3640 1 -- start start 2023-08-09T13:36:17.236 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:17.232+0000 7ff4db0f3640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7ff4c0003970 0x7ff4d40b2300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:36:17.237 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:17.232+0000 7ff4db0f3640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7ff4d40b09f0 0x7ff4d40c32d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:36:17.237 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:17.232+0000 7ff4db0f3640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff4d414cdf0 0x7ff4d4159470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:36:17.237 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:17.232+0000 7ff4db0f3640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff4d41f0830 con 0x7ff4d414cdf0 2023-08-09T13:36:17.237 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:17.232+0000 7ff4db0f3640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff4d41f0a10 con 0x7ff4d40b09f0 2023-08-09T13:36:17.237 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:17.232+0000 7ff4db0f3640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff4d40b0f60 con 0x7ff4c0003970 2023-08-09T13:36:17.237 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:17.232+0000 7ff4da8f2640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff4d414cdf0 0x7ff4d4159470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:36:17.237 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:17.232+0000 7ff4da8f2640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff4d414cdf0 0x7ff4d4159470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:58138/0 (socket says 172.21.15.43:58138) 2023-08-09T13:36:17.237 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:17.232+0000 7ff4da8f2640 1 -- 172.21.15.43:0/3339201121 learned_addr learned my addr 172.21.15.43:0/3339201121 (peer_addr_for_me v2:172.21.15.43:0/0) 2023-08-09T13:36:17.238 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:17.232+0000 7ff4da0f1640 1 --2- 172.21.15.43:0/3339201121 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7ff4c0003970 0x7ff4d40b2300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:36:17.238 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:17.232+0000 7ff4d98f0640 1 --2- 172.21.15.43:0/3339201121 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7ff4d40b09f0 0x7ff4d40c32d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:36:17.238 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:17.232+0000 7ff4da8f2640 1 -- 172.21.15.43:0/3339201121 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7ff4c0003970 msgr2=0x7ff4d40b2300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:36:17.238 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:17.232+0000 7ff4da8f2640 1 --2- 172.21.15.43:0/3339201121 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7ff4c0003970 0x7ff4d40b2300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:36:17.238 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:17.232+0000 7ff4da8f2640 1 -- 172.21.15.43:0/3339201121 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7ff4d40b09f0 msgr2=0x7ff4d40c32d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:36:17.238 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:17.232+0000 7ff4da8f2640 1 --2- 172.21.15.43:0/3339201121 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7ff4d40b09f0 0x7ff4d40c32d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:36:17.238 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:17.232+0000 7ff4da8f2640 1 -- 172.21.15.43:0/3339201121 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff4d40b1240 con 0x7ff4d414cdf0 2023-08-09T13:36:17.239 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:17.233+0000 7ff4da8f2640 1 --2- 172.21.15.43:0/3339201121 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff4d414cdf0 0x7ff4d4159470 secure :-1 s=READY pgs=579 cs=0 l=1 rev1=1 crypto rx=0x7ff4d40ca790 tx=0x7ff4cc00f520 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-09T13:36:17.239 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:17.233+0000 7ff4caffd640 1 -- 172.21.15.43:0/3339201121 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff4cc016420 con 0x7ff4d414cdf0 2023-08-09T13:36:17.239 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:17.233+0000 7ff4caffd640 1 -- 172.21.15.43:0/3339201121 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff4cc016580 con 0x7ff4d414cdf0 2023-08-09T13:36:17.239 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:17.233+0000 7ff4caffd640 1 -- 172.21.15.43:0/3339201121 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff4cc0168f0 con 0x7ff4d414cdf0 2023-08-09T13:36:17.239 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:17.233+0000 7ff4db0f3640 1 -- 172.21.15.43:0/3339201121 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff4d40b1520 con 0x7ff4d414cdf0 2023-08-09T13:36:17.239 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:17.233+0000 7ff4db0f3640 1 -- 172.21.15.43:0/3339201121 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff4d40b2840 con 0x7ff4d414cdf0 2023-08-09T13:36:17.240 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:17.235+0000 7ff4db0f3640 1 -- 172.21.15.43:0/3339201121 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff4d40b2a00 con 0x7ff4d414cdf0 2023-08-09T13:36:17.240 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:17.235+0000 7ff4caffd640 1 -- 172.21.15.43:0/3339201121 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 5) v1 ==== 88341+0+0 (secure 0 0 0) 0x7ff4cc016a50 con 0x7ff4d414cdf0 2023-08-09T13:36:17.240 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:17.236+0000 7ff4caffd640 1 --2- 172.21.15.43:0/3339201121 >> [v2:172.21.15.203:6832/830097474,v1:172.21.15.203:6833/830097474] conn(0x7ff4b806a070 0x7ff4b806c530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:36:17.240 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:17.236+0000 7ff4caffd640 1 -- 172.21.15.43:0/3339201121 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(39..39 src has 1..39) v4 ==== 6382+0+0 (secure 0 0 0) 0x7ff4cc0893c0 con 0x7ff4d414cdf0 2023-08-09T13:36:17.241 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:17.237+0000 7ff4da0f1640 1 --2- 172.21.15.43:0/3339201121 >> [v2:172.21.15.203:6832/830097474,v1:172.21.15.203:6833/830097474] conn(0x7ff4b806a070 0x7ff4b806c530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:36:17.241 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:17.238+0000 7ff4da0f1640 1 --2- 172.21.15.43:0/3339201121 >> [v2:172.21.15.203:6832/830097474,v1:172.21.15.203:6833/830097474] conn(0x7ff4b806a070 0x7ff4b806c530 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7ff4d0033000 tx=0x7ff4d0009030 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-09T13:36:17.242 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:17.241+0000 7ff4caffd640 1 -- 172.21.15.43:0/3339201121 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178066 (secure 0 0 0) 0x7ff4d40b2a00 con 0x7ff4d414cdf0 2023-08-09T13:36:17.506 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:17.504+0000 7ff4db0f3640 1 -- 172.21.15.43:0/3339201121 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]} v 0) v1 -- 0x7ff4d40b2a00 con 0x7ff4d414cdf0 2023-08-09T13:36:17.514 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:17.512+0000 7ff4caffd640 1 -- 172.21.15.43:0/3339201121 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]}]=0 updated caps for client.0 v10) v1 ==== 190+0+147 (secure 0 0 0) 0x7ff4d40b2a00 con 0x7ff4d414cdf0 2023-08-09T13:36:17.514 INFO:teuthology.orchestra.run.smithi043.stdout:[client.0] 2023-08-09T13:36:17.514 INFO:teuthology.orchestra.run.smithi043.stdout: key = AQCPldNkg8zrDxAAM/hnJaE6S9HU/fRwQeBV5w== 2023-08-09T13:36:17.514 INFO:teuthology.orchestra.run.smithi043.stdout: caps mds = "allow" 2023-08-09T13:36:17.515 INFO:teuthology.orchestra.run.smithi043.stdout: caps mon = "allow r" 2023-08-09T13:36:17.515 INFO:teuthology.orchestra.run.smithi043.stdout: caps osd = "allow rw tag cephfs data=cephfs" 2023-08-09T13:36:17.515 INFO:teuthology.orchestra.run.smithi043.stderr:updated caps for client.0 2023-08-09T13:36:17.517 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:17.515+0000 7ff4db0f3640 1 -- 172.21.15.43:0/3339201121 >> [v2:172.21.15.203:6832/830097474,v1:172.21.15.203:6833/830097474] conn(0x7ff4b806a070 msgr2=0x7ff4b806c530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:36:17.518 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:17.515+0000 7ff4db0f3640 1 --2- 172.21.15.43:0/3339201121 >> [v2:172.21.15.203:6832/830097474,v1:172.21.15.203:6833/830097474] conn(0x7ff4b806a070 0x7ff4b806c530 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7ff4d0033000 tx=0x7ff4d0009030 comp rx=0 tx=0).stop 2023-08-09T13:36:17.518 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:17.515+0000 7ff4db0f3640 1 -- 172.21.15.43:0/3339201121 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff4d414cdf0 msgr2=0x7ff4d4159470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:36:17.518 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:17.515+0000 7ff4db0f3640 1 --2- 172.21.15.43:0/3339201121 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff4d414cdf0 0x7ff4d4159470 secure :-1 s=READY pgs=579 cs=0 l=1 rev1=1 crypto rx=0x7ff4d40ca790 tx=0x7ff4cc00f520 comp rx=0 tx=0).stop 2023-08-09T13:36:17.518 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:17.516+0000 7ff4db0f3640 1 -- 172.21.15.43:0/3339201121 shutdown_connections 2023-08-09T13:36:17.518 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:17.516+0000 7ff4db0f3640 1 --2- 172.21.15.43:0/3339201121 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff4d414cdf0 0x7ff4d4159470 unknown :-1 s=CLOSED pgs=579 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:36:17.518 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:17.516+0000 7ff4db0f3640 1 --2- 172.21.15.43:0/3339201121 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7ff4d40b09f0 0x7ff4d40c32d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:36:17.518 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:17.516+0000 7ff4db0f3640 1 --2- 172.21.15.43:0/3339201121 >> [v2:172.21.15.203:6832/830097474,v1:172.21.15.203:6833/830097474] conn(0x7ff4b806a070 0x7ff4b806c530 unknown :-1 s=CLOSED pgs=279 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:36:17.519 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:17.516+0000 7ff4db0f3640 1 --2- 172.21.15.43:0/3339201121 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7ff4c0003970 0x7ff4d40b2300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:36:17.519 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:17.516+0000 7ff4db0f3640 1 -- 172.21.15.43:0/3339201121 >> 172.21.15.43:0/3339201121 conn(0x7ff4d40cddd0 msgr2=0x7ff4d415a690 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-09T13:36:17.519 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:17.516+0000 7ff4db0f3640 1 -- 172.21.15.43:0/3339201121 shutdown_connections 2023-08-09T13:36:17.519 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:17.516+0000 7ff4db0f3640 1 -- 172.21.15.43:0/3339201121 wait complete. 2023-08-09T13:36:17.537 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-08-09T13:36:17.766 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:17.765+0000 7f1c61742640 1 Processor -- start 2023-08-09T13:36:17.767 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:17.765+0000 7f1c61742640 1 -- start start 2023-08-09T13:36:17.768 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:17.767+0000 7f1c61742640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f1c5c0c86f0 0x7f1c5c0c8b50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:36:17.768 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:17.767+0000 7f1c61742640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f1c5c06c8d0 0x7f1c5c06cd40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:36:17.769 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:17.767+0000 7f1c61742640 1 -- --> v1:172.21.15.43:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1c5c0c9090 con 0x7f1c5c143c20 2023-08-09T13:36:17.769 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:17.767+0000 7f1c61742640 1 -- --> v2:172.21.15.43:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1c5c144000 con 0x7f1c5c0c86f0 2023-08-09T13:36:17.769 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:17.767+0000 7f1c61742640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1c5c06c440 con 0x7f1c5c06c8d0 2023-08-09T13:36:17.770 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:17.767+0000 7f1c5a7fc640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f1c5c0c86f0 0x7f1c5c0c8b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:36:17.770 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:17.767+0000 7f1c5a7fc640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f1c5c0c86f0 0x7f1c5c0c8b50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:58140/0 (socket says 172.21.15.43:58140) 2023-08-09T13:36:17.770 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:17.767+0000 7f1c5a7fc640 1 -- 172.21.15.43:0/3899504878 learned_addr learned my addr 172.21.15.43:0/3899504878 (peer_addr_for_me v2:172.21.15.43:0/0) 2023-08-09T13:36:17.770 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:17.768+0000 7f1c5b7fe640 1 --2- 172.21.15.43:0/3899504878 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f1c5c06c8d0 0x7f1c5c06cd40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:36:17.770 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:17.768+0000 7f1c5a7fc640 1 -- 172.21.15.43:0/3899504878 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f1c5c06c8d0 msgr2=0x7f1c5c06cd40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-09T13:36:17.770 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:17.768+0000 7f1c5a7fc640 1 --2- 172.21.15.43:0/3899504878 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f1c5c06c8d0 0x7f1c5c06cd40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:36:17.770 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:17.768+0000 7f1c5a7fc640 1 -- 172.21.15.43:0/3899504878 >> v1:172.21.15.43:6789/0 conn(0x7f1c5c143c20 legacy=0x7f1c5c06f540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:36:17.771 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:17.768+0000 7f1c5a7fc640 1 -- 172.21.15.43:0/3899504878 --> v2:172.21.15.43:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1c5c06fc50 con 0x7f1c5c0c86f0 2023-08-09T13:36:17.771 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:17.768+0000 7f1c5b7fe640 1 --2- 172.21.15.43:0/3899504878 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f1c5c06c8d0 0x7f1c5c06cd40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-08-09T13:36:17.772 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:17.769+0000 7f1c5a7fc640 1 --2- 172.21.15.43:0/3899504878 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f1c5c0c86f0 0x7f1c5c0c8b50 secure :-1 s=READY pgs=581 cs=0 l=1 rev1=1 crypto rx=0x7f1c50009870 tx=0x7f1c5002b8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9b7788841b6baed1 server_cookie=0 in_seq=0 out_seq=0 2023-08-09T13:36:17.772 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:17.769+0000 7f1c59ffb640 1 -- 172.21.15.43:0/3899504878 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1c50031640 con 0x7f1c5c0c86f0 2023-08-09T13:36:17.772 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:17.769+0000 7f1c59ffb640 1 -- 172.21.15.43:0/3899504878 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1c500317a0 con 0x7f1c5c0c86f0 2023-08-09T13:36:17.773 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:17.769+0000 7f1c59ffb640 1 -- 172.21.15.43:0/3899504878 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1c50031b30 con 0x7f1c5c0c86f0 2023-08-09T13:36:17.773 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:17.770+0000 7f1c61742640 1 -- 172.21.15.43:0/3899504878 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f1c5c0c86f0 msgr2=0x7f1c5c0c8b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:36:17.773 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:17.770+0000 7f1c61742640 1 --2- 172.21.15.43:0/3899504878 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f1c5c0c86f0 0x7f1c5c0c8b50 secure :-1 s=READY pgs=581 cs=0 l=1 rev1=1 crypto rx=0x7f1c50009870 tx=0x7f1c5002b8f0 comp rx=0 tx=0).stop 2023-08-09T13:36:17.774 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:17.770+0000 7f1c61742640 1 -- 172.21.15.43:0/3899504878 shutdown_connections 2023-08-09T13:36:17.774 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:17.770+0000 7f1c61742640 1 --2- 172.21.15.43:0/3899504878 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f1c5c06c8d0 0x7f1c5c06cd40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:36:17.774 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:17.770+0000 7f1c61742640 1 --2- 172.21.15.43:0/3899504878 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f1c5c0c86f0 0x7f1c5c0c8b50 unknown :-1 s=CLOSED pgs=581 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:36:17.774 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:17.770+0000 7f1c61742640 1 -- 172.21.15.43:0/3899504878 >> 172.21.15.43:0/3899504878 conn(0x7f1c5c0ce0a0 msgr2=0x7f1c5c0ce480 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-09T13:36:17.775 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:17.770+0000 7f1c61742640 1 -- 172.21.15.43:0/3899504878 shutdown_connections 2023-08-09T13:36:17.775 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:17.770+0000 7f1c61742640 1 -- 172.21.15.43:0/3899504878 wait complete. 2023-08-09T13:36:17.777 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:17.772+0000 7f1c61742640 1 Processor -- start 2023-08-09T13:36:17.778 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:17.772+0000 7f1c61742640 1 -- start start 2023-08-09T13:36:17.778 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:17.773+0000 7f1c61742640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f1c5c06c8d0 0x7f1c5c15d480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:36:17.778 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:17.773+0000 7f1c61742640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f1c5c0c86f0 0x7f1c5c141740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:36:17.778 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:17.773+0000 7f1c61742640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f1c5c143c20 0x7f1c5c141c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:36:17.779 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:17.773+0000 7f1c61742640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1c5c0b1700 con 0x7f1c5c06c8d0 2023-08-09T13:36:17.779 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:17.773+0000 7f1c61742640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1c5c142b50 con 0x7f1c5c0c86f0 2023-08-09T13:36:17.779 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:17.773+0000 7f1c61742640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1c5c0b10f0 con 0x7f1c5c143c20 2023-08-09T13:36:17.779 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:17.773+0000 7f1c5affd640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f1c5c06c8d0 0x7f1c5c15d480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:36:17.780 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:17.773+0000 7f1c5affd640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f1c5c06c8d0 0x7f1c5c15d480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:58154/0 (socket says 172.21.15.43:58154) 2023-08-09T13:36:17.780 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:17.773+0000 7f1c5affd640 1 -- 172.21.15.43:0/2439175088 learned_addr learned my addr 172.21.15.43:0/2439175088 (peer_addr_for_me v2:172.21.15.43:0/0) 2023-08-09T13:36:17.780 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:17.774+0000 7f1c5affd640 1 -- 172.21.15.43:0/2439175088 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f1c5c143c20 msgr2=0x7f1c5c141c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:36:17.780 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:17.774+0000 7f1c5affd640 1 --2- 172.21.15.43:0/2439175088 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f1c5c143c20 0x7f1c5c141c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:36:17.781 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:17.774+0000 7f1c5affd640 1 -- 172.21.15.43:0/2439175088 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f1c5c0c86f0 msgr2=0x7f1c5c141740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:36:17.781 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:17.774+0000 7f1c5affd640 1 --2- 172.21.15.43:0/2439175088 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f1c5c0c86f0 0x7f1c5c141740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:36:17.781 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:17.774+0000 7f1c5affd640 1 -- 172.21.15.43:0/2439175088 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1c500094d0 con 0x7f1c5c06c8d0 2023-08-09T13:36:17.781 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:17.774+0000 7f1c5affd640 1 --2- 172.21.15.43:0/2439175088 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f1c5c06c8d0 0x7f1c5c15d480 secure :-1 s=READY pgs=582 cs=0 l=1 rev1=1 crypto rx=0x7f1c4c00c450 tx=0x7f1c4c00c910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-09T13:36:17.782 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:17.774+0000 7f1c437fe640 1 -- 172.21.15.43:0/2439175088 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1c4c0052d0 con 0x7f1c5c06c8d0 2023-08-09T13:36:17.782 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:17.774+0000 7f1c437fe640 1 -- 172.21.15.43:0/2439175088 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1c4c005430 con 0x7f1c5c06c8d0 2023-08-09T13:36:17.782 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:17.774+0000 7f1c61742640 1 -- 172.21.15.43:0/2439175088 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1c5c0b1910 con 0x7f1c5c06c8d0 2023-08-09T13:36:17.782 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:17.774+0000 7f1c437fe640 1 -- 172.21.15.43:0/2439175088 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1c4c012870 con 0x7f1c5c06c8d0 2023-08-09T13:36:17.782 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:17.775+0000 7f1c61742640 1 -- 172.21.15.43:0/2439175088 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1c5c0b0d80 con 0x7f1c5c06c8d0 2023-08-09T13:36:17.782 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:17.776+0000 7f1c61742640 1 -- 172.21.15.43:0/2439175088 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1c5c0b0f10 con 0x7f1c5c06c8d0 2023-08-09T13:36:17.783 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:17.777+0000 7f1c437fe640 1 -- 172.21.15.43:0/2439175088 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 5) v1 ==== 88341+0+0 (secure 0 0 0) 0x7f1c4c0129d0 con 0x7f1c5c06c8d0 2023-08-09T13:36:17.784 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:17.777+0000 7f1c437fe640 1 --2- 172.21.15.43:0/2439175088 >> [v2:172.21.15.203:6832/830097474,v1:172.21.15.203:6833/830097474] conn(0x7f1c4806a050 0x7f1c4806c510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:36:17.784 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:17.778+0000 7f1c437fe640 1 -- 172.21.15.43:0/2439175088 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(39..39 src has 1..39) v4 ==== 6382+0+0 (secure 0 0 0) 0x7f1c4c088220 con 0x7f1c5c06c8d0 2023-08-09T13:36:17.784 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:17.778+0000 7f1c5a7fc640 1 --2- 172.21.15.43:0/2439175088 >> [v2:172.21.15.203:6832/830097474,v1:172.21.15.203:6833/830097474] conn(0x7f1c4806a050 0x7f1c4806c510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:36:17.784 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:17.779+0000 7f1c5a7fc640 1 --2- 172.21.15.43:0/2439175088 >> [v2:172.21.15.203:6832/830097474,v1:172.21.15.203:6833/830097474] conn(0x7f1c4806a050 0x7f1c4806c510 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7f1c50036000 tx=0x7f1c50030000 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-09T13:36:17.784 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:17.782+0000 7f1c437fe640 1 -- 172.21.15.43:0/2439175088 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178066 (secure 0 0 0) 0x7f1c4c0537d0 con 0x7f1c5c06c8d0 2023-08-09T13:36:18.018 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:18.016+0000 7f1c61742640 1 -- 172.21.15.43:0/2439175088 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f1c5c0739f0 con 0x7f1c5c06c8d0 2023-08-09T13:36:18.019 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:18.018+0000 7f1c437fe640 1 -- 172.21.15.43:0/2439175088 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 17 v17) v1 ==== 94+0+4878 (secure 0 0 0) 0x7f1c4c056df0 con 0x7f1c5c06c8d0 2023-08-09T13:36:18.019 INFO:teuthology.orchestra.run.smithi043.stdout: 2023-08-09T13:36:18.020 INFO:teuthology.orchestra.run.smithi043.stdout:{"epoch":17,"default_fscid":2,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4919,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.203:6836/302214806","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6834","nonce":302214806},{"type":"v1","addr":"172.21.15.203:6836","nonce":302214806}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":12},{"gid":4922,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.203:6837/1677639100","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6835","nonce":1677639100},{"type":"v1","addr":"172.21.15.203:6837","nonce":1677639100}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":12},{"gid":5619,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.43:6834/2339480142","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6832","nonce":2339480142},{"type":"v1","addr":"172.21.15.43:6834","nonce":2339480142}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":12}],"filesystems":[{"mdsmap":{"epoch":17,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-08-09T13:36:11.336926+0000","modified":"2023-08-09T13:36:14.905554+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":5622},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5622":{"gid":5622,"name":"c","rank":0,"incarnation":13,"state":"up:active","state_seq":4,"addr":"172.21.15.43:6835/2019295969","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6833","nonce":2019295969},{"type":"v1","addr":"172.21.15.43:6835","nonce":2019295969}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[7],"metadata_pool":6,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":2}]} 2023-08-09T13:36:18.020 INFO:teuthology.orchestra.run.smithi043.stderr:dumped fsmap epoch 17 2023-08-09T13:36:18.021 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:18.020+0000 7f1c61742640 1 -- 172.21.15.43:0/2439175088 >> [v2:172.21.15.203:6832/830097474,v1:172.21.15.203:6833/830097474] conn(0x7f1c4806a050 msgr2=0x7f1c4806c510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:36:18.021 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:18.020+0000 7f1c61742640 1 --2- 172.21.15.43:0/2439175088 >> [v2:172.21.15.203:6832/830097474,v1:172.21.15.203:6833/830097474] conn(0x7f1c4806a050 0x7f1c4806c510 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7f1c50036000 tx=0x7f1c50030000 comp rx=0 tx=0).stop 2023-08-09T13:36:18.021 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:18.020+0000 7f1c61742640 1 -- 172.21.15.43:0/2439175088 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f1c5c06c8d0 msgr2=0x7f1c5c15d480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:36:18.022 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:18.020+0000 7f1c61742640 1 --2- 172.21.15.43:0/2439175088 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f1c5c06c8d0 0x7f1c5c15d480 secure :-1 s=READY pgs=582 cs=0 l=1 rev1=1 crypto rx=0x7f1c4c00c450 tx=0x7f1c4c00c910 comp rx=0 tx=0).stop 2023-08-09T13:36:18.022 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:18.020+0000 7f1c61742640 1 -- 172.21.15.43:0/2439175088 shutdown_connections 2023-08-09T13:36:18.022 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:18.020+0000 7f1c61742640 1 --2- 172.21.15.43:0/2439175088 >> [v2:172.21.15.203:6832/830097474,v1:172.21.15.203:6833/830097474] conn(0x7f1c4806a050 0x7f1c4806c510 unknown :-1 s=CLOSED pgs=280 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:36:18.023 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:18.020+0000 7f1c61742640 1 --2- 172.21.15.43:0/2439175088 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f1c5c143c20 0x7f1c5c141c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:36:18.023 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:18.020+0000 7f1c61742640 1 --2- 172.21.15.43:0/2439175088 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f1c5c0c86f0 0x7f1c5c141740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:36:18.023 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:18.020+0000 7f1c61742640 1 --2- 172.21.15.43:0/2439175088 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f1c5c06c8d0 0x7f1c5c15d480 unknown :-1 s=CLOSED pgs=582 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:36:18.023 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:18.020+0000 7f1c61742640 1 -- 172.21.15.43:0/2439175088 >> 172.21.15.43:0/2439175088 conn(0x7f1c5c0ce0a0 msgr2=0x7f1c5c0676b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-09T13:36:18.023 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:18.020+0000 7f1c61742640 1 -- 172.21.15.43:0/2439175088 shutdown_connections 2023-08-09T13:36:18.023 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:18.020+0000 7f1c61742640 1 -- 172.21.15.43:0/2439175088 wait complete. 2023-08-09T13:36:18.444 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 17, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2023-08-09T13:36:11.336926+0000', 'modified': '2023-08-09T13:36:14.905554+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 5622}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_5622': {'gid': 5622, 'name': 'c', 'rank': 0, 'incarnation': 13, 'state': 'up:active', 'state_seq': 4, 'addr': '172.21.15.43:6835/2019295969', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.43:6833', 'nonce': 2019295969}, {'type': 'v1', 'addr': '172.21.15.43:6835', 'nonce': 2019295969}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138322906710015, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}}, 'data_pools': [7], 'metadata_pool': 6, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0} 2023-08-09T13:36:18.444 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2023-08-09T13:36:18.445 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.c status 2023-08-09T13:36:18.677 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:18.675+0000 7f509404d640 1 Processor -- start 2023-08-09T13:36:18.677 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:18.675+0000 7f509404d640 1 -- start start 2023-08-09T13:36:18.678 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:18.677+0000 7f509404d640 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7f508c15c680 0x7f508c0ce760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:36:18.679 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:18.677+0000 7f509404d640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f508c0b3fa0 0x7f508c0b43b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:36:18.679 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:18.677+0000 7f509404d640 1 -- --> v1:172.21.15.203:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f508c0b48f0 con 0x7f508c0d0a70 2023-08-09T13:36:18.679 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:18.677+0000 7f509404d640 1 -- --> v2:172.21.15.43:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f508c0cdf00 con 0x7f508c0b3fa0 2023-08-09T13:36:18.679 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:18.677+0000 7f509404d640 1 -- --> v2:172.21.15.203:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f508c0d0f70 con 0x7f508c15c680 2023-08-09T13:36:18.680 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:18.678+0000 7f50925ea640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f508c0b3fa0 0x7f508c0b43b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:36:18.680 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:18.678+0000 7f50925ea640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f508c0b3fa0 0x7f508c0b43b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:58170/0 (socket says 172.21.15.43:58170) 2023-08-09T13:36:18.680 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:18.678+0000 7f50925ea640 1 -- 172.21.15.43:0/348499820 learned_addr learned my addr 172.21.15.43:0/348499820 (peer_addr_for_me v2:172.21.15.43:0/0) 2023-08-09T13:36:18.680 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:18.678+0000 7f5091de9640 1 --2- 172.21.15.43:0/348499820 >> v2:172.21.15.203:3300/0 conn(0x7f508c15c680 0x7f508c0ce760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:36:18.681 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:18.678+0000 7f50925ea640 1 -- 172.21.15.43:0/348499820 >> v2:172.21.15.203:3300/0 conn(0x7f508c15c680 msgr2=0x7f508c0ce760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-09T13:36:18.681 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:18.678+0000 7f50925ea640 1 --2- 172.21.15.43:0/348499820 >> v2:172.21.15.203:3300/0 conn(0x7f508c15c680 0x7f508c0ce760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:36:18.681 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:18.678+0000 7f50925ea640 1 -- 172.21.15.43:0/348499820 >> v1:172.21.15.203:6789/0 conn(0x7f508c0d0a70 legacy=0x7f508c0cd7f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-09T13:36:18.681 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:18.678+0000 7f50925ea640 1 -- 172.21.15.43:0/348499820 --> v2:172.21.15.43:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f508c0ccd10 con 0x7f508c0b3fa0 2023-08-09T13:36:18.681 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:18.678+0000 7f5091de9640 1 --2- 172.21.15.43:0/348499820 >> v2:172.21.15.203:3300/0 conn(0x7f508c15c680 0x7f508c0ce760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-08-09T13:36:18.682 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:18.679+0000 7f50925ea640 1 --2- 172.21.15.43:0/348499820 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f508c0b3fa0 0x7f508c0b43b0 secure :-1 s=READY pgs=583 cs=0 l=1 rev1=1 crypto rx=0x7f508c06b7d0 tx=0x7f508402bf70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8aa20c6bff3534f7 server_cookie=0 in_seq=0 out_seq=0 2023-08-09T13:36:18.682 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:18.680+0000 7f5090de7640 1 -- 172.21.15.43:0/348499820 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f508402cb10 con 0x7f508c0b3fa0 2023-08-09T13:36:18.682 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:18.680+0000 7f5090de7640 1 -- 172.21.15.43:0/348499820 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f508402cc70 con 0x7f508c0b3fa0 2023-08-09T13:36:18.683 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:18.680+0000 7f5090de7640 1 -- 172.21.15.43:0/348499820 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f50840343f0 con 0x7f508c0b3fa0 2023-08-09T13:36:18.683 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:18.681+0000 7f509404d640 1 -- 172.21.15.43:0/348499820 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f508c0b3fa0 msgr2=0x7f508c0b43b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:36:18.683 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:18.681+0000 7f509404d640 1 --2- 172.21.15.43:0/348499820 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f508c0b3fa0 0x7f508c0b43b0 secure :-1 s=READY pgs=583 cs=0 l=1 rev1=1 crypto rx=0x7f508c06b7d0 tx=0x7f508402bf70 comp rx=0 tx=0).stop 2023-08-09T13:36:18.683 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:18.681+0000 7f509404d640 1 -- 172.21.15.43:0/348499820 shutdown_connections 2023-08-09T13:36:18.684 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:18.681+0000 7f509404d640 1 --2- 172.21.15.43:0/348499820 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f508c0b3fa0 0x7f508c0b43b0 unknown :-1 s=CLOSED pgs=583 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:36:18.684 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:18.681+0000 7f509404d640 1 --2- 172.21.15.43:0/348499820 >> v2:172.21.15.203:3300/0 conn(0x7f508c15c680 0x7f508c0ce760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:36:18.684 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:18.681+0000 7f509404d640 1 -- 172.21.15.43:0/348499820 >> 172.21.15.43:0/348499820 conn(0x7f508c071520 msgr2=0x7f508c0d2520 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-09T13:36:18.684 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:18.681+0000 7f509404d640 1 -- 172.21.15.43:0/348499820 shutdown_connections 2023-08-09T13:36:18.684 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:18.681+0000 7f509404d640 1 -- 172.21.15.43:0/348499820 wait complete. 2023-08-09T13:36:18.685 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:18.683+0000 7f509404d640 1 Processor -- start 2023-08-09T13:36:18.685 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:18.683+0000 7f509404d640 1 -- start start 2023-08-09T13:36:18.686 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:18.684+0000 7f509404d640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f508c0b3fa0 0x7f508c149930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:36:18.686 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:18.684+0000 7f509404d640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f508c0d0a70 0x7f508c0cb9a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:36:18.686 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:18.684+0000 7f509404d640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f508c15c680 0x7f508c0ca000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:36:18.686 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:18.684+0000 7f509404d640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f508c149e70 con 0x7f508c15c680 2023-08-09T13:36:18.686 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:18.684+0000 7f509404d640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f508c0ca900 con 0x7f508c0d0a70 2023-08-09T13:36:18.686 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:18.684+0000 7f509404d640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f508c0cabe0 con 0x7f508c0b3fa0 2023-08-09T13:36:18.687 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:18.684+0000 7f50925ea640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f508c15c680 0x7f508c0ca000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:36:18.687 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:18.684+0000 7f50925ea640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f508c15c680 0x7f508c0ca000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:58176/0 (socket says 172.21.15.43:58176) 2023-08-09T13:36:18.687 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:18.684+0000 7f50925ea640 1 -- 172.21.15.43:0/4194340286 learned_addr learned my addr 172.21.15.43:0/4194340286 (peer_addr_for_me v2:172.21.15.43:0/0) 2023-08-09T13:36:18.688 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:18.685+0000 7f5091de9640 1 --2- 172.21.15.43:0/4194340286 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f508c0b3fa0 0x7f508c149930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:36:18.688 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:18.685+0000 7f50925ea640 1 -- 172.21.15.43:0/4194340286 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f508c0b3fa0 msgr2=0x7f508c149930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:36:18.688 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:18.685+0000 7f50925ea640 1 --2- 172.21.15.43:0/4194340286 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f508c0b3fa0 0x7f508c149930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:36:18.688 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:18.685+0000 7f50915e8640 1 --2- 172.21.15.43:0/4194340286 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f508c0d0a70 0x7f508c0cb9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:36:18.688 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:18.685+0000 7f50925ea640 1 -- 172.21.15.43:0/4194340286 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f508c0d0a70 msgr2=0x7f508c0cb9a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:36:18.689 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:18.685+0000 7f50925ea640 1 --2- 172.21.15.43:0/4194340286 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f508c0d0a70 0x7f508c0cb9a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:36:18.689 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:18.685+0000 7f50925ea640 1 -- 172.21.15.43:0/4194340286 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5084009b30 con 0x7f508c15c680 2023-08-09T13:36:18.689 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:18.685+0000 7f50925ea640 1 --2- 172.21.15.43:0/4194340286 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f508c15c680 0x7f508c0ca000 secure :-1 s=READY pgs=584 cs=0 l=1 rev1=1 crypto rx=0x7f5084036040 tx=0x7f5084004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-09T13:36:18.689 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:18.685+0000 7f50827fc640 1 -- 172.21.15.43:0/4194340286 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5084009510 con 0x7f508c15c680 2023-08-09T13:36:18.689 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:18.685+0000 7f50827fc640 1 -- 172.21.15.43:0/4194340286 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f508403e450 con 0x7f508c15c680 2023-08-09T13:36:18.689 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:18.685+0000 7f50827fc640 1 -- 172.21.15.43:0/4194340286 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f508403e800 con 0x7f508c15c680 2023-08-09T13:36:18.689 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:18.685+0000 7f509404d640 1 -- 172.21.15.43:0/4194340286 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f508c0caec0 con 0x7f508c15c680 2023-08-09T13:36:18.690 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:18.686+0000 7f509404d640 1 -- 172.21.15.43:0/4194340286 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f508c0cb480 con 0x7f508c15c680 2023-08-09T13:36:18.690 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:18.688+0000 7f50827fc640 1 -- 172.21.15.43:0/4194340286 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 5) v1 ==== 88341+0+0 (secure 0 0 0) 0x7f508403e960 con 0x7f508c15c680 2023-08-09T13:36:18.690 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:18.689+0000 7f50827fc640 1 --2- 172.21.15.43:0/4194340286 >> [v2:172.21.15.203:6832/830097474,v1:172.21.15.203:6833/830097474] conn(0x7f507006a050 0x7f507006c510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:36:18.690 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:18.689+0000 7f50827fc640 1 -- 172.21.15.43:0/4194340286 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(39..39 src has 1..39) v4 ==== 6382+0+0 (secure 0 0 0) 0x7f50840ab620 con 0x7f508c15c680 2023-08-09T13:36:18.691 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:18.689+0000 7f5091de9640 1 --2- 172.21.15.43:0/4194340286 >> [v2:172.21.15.203:6832/830097474,v1:172.21.15.203:6833/830097474] conn(0x7f507006a050 0x7f507006c510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:36:18.691 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:18.690+0000 7f5096bbf740 1 Processor -- start 2023-08-09T13:36:18.691 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:18.690+0000 7f5096bbf740 1 -- start start 2023-08-09T13:36:18.692 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:18.690+0000 7f5091de9640 1 --2- 172.21.15.43:0/4194340286 >> [v2:172.21.15.203:6832/830097474,v1:172.21.15.203:6833/830097474] conn(0x7f507006a050 0x7f507006c510 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7f508800b840 tx=0x7f5088008040 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-09T13:36:18.692 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:18.691+0000 7f5096bbf740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x557de6ff5c50 0x557de6ff6030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:36:18.693 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:18.691+0000 7f5096bbf740 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x557de6fc5bd0 0x557de6ff2ec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:36:18.693 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:18.691+0000 7f5096bbf740 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x557de6fc97d0 0x557de6ff3400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:36:18.693 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:18.691+0000 7f5096bbf740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x557de6f378f0 con 0x557de6ff5c50 2023-08-09T13:36:18.693 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:18.691+0000 7f5096bbf740 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x557de6f34fe0 con 0x557de6fc5bd0 2023-08-09T13:36:18.693 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:18.691+0000 7f5096bbf740 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x557de6f308d0 con 0x557de6fc97d0 2023-08-09T13:36:18.693 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:18.691+0000 7f5091de9640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x557de6ff5c50 0x557de6ff6030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:36:18.693 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:18.691+0000 7f5091de9640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x557de6ff5c50 0x557de6ff6030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:58178/0 (socket says 172.21.15.43:58178) 2023-08-09T13:36:18.693 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:18.691+0000 7f5091de9640 1 -- 172.21.15.43:0/1606720538 learned_addr learned my addr 172.21.15.43:0/1606720538 (peer_addr_for_me v2:172.21.15.43:0/0) 2023-08-09T13:36:18.695 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:18.691+0000 7f5091de9640 1 -- 172.21.15.43:0/1606720538 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x557de6fc97d0 msgr2=0x557de6ff3400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-09T13:36:18.695 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:18.691+0000 7f5091de9640 1 --2- 172.21.15.43:0/1606720538 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x557de6fc97d0 0x557de6ff3400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:36:18.695 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:18.691+0000 7f50915e8640 1 --2- 172.21.15.43:0/1606720538 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x557de6fc5bd0 0x557de6ff2ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:36:18.695 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:18.691+0000 7f5091de9640 1 -- 172.21.15.43:0/1606720538 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x557de6fc5bd0 msgr2=0x557de6ff2ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-09T13:36:18.695 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:18.691+0000 7f5091de9640 1 --2- 172.21.15.43:0/1606720538 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x557de6fc5bd0 0x557de6ff2ec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:36:18.696 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:18.691+0000 7f5091de9640 1 -- 172.21.15.43:0/1606720538 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x557de6ee9ac0 con 0x557de6ff5c50 2023-08-09T13:36:18.696 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:18.692+0000 7f5091de9640 1 --2- 172.21.15.43:0/1606720538 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x557de6ff5c50 0x557de6ff6030 secure :-1 s=READY pgs=585 cs=0 l=1 rev1=1 crypto rx=0x7f508800db60 tx=0x7f5088010000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7a5c7c52b25c2ec0 server_cookie=0 in_seq=0 out_seq=0 2023-08-09T13:36:18.696 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:18.692+0000 7f506f7fe640 1 -- 172.21.15.43:0/1606720538 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f508800b050 con 0x557de6ff5c50 2023-08-09T13:36:18.696 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:18.692+0000 7f506f7fe640 1 -- 172.21.15.43:0/1606720538 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5088017070 con 0x557de6ff5c50 2023-08-09T13:36:18.696 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:18.692+0000 7f506f7fe640 1 -- 172.21.15.43:0/1606720538 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5088012de0 con 0x557de6ff5c50 2023-08-09T13:36:18.696 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:18.693+0000 7f5096bbf740 1 -- 172.21.15.43:0/1606720538 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x557de6ff5c50 msgr2=0x557de6ff6030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:36:18.696 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:18.693+0000 7f5096bbf740 1 --2- 172.21.15.43:0/1606720538 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x557de6ff5c50 0x557de6ff6030 secure :-1 s=READY pgs=585 cs=0 l=1 rev1=1 crypto rx=0x7f508800db60 tx=0x7f5088010000 comp rx=0 tx=0).stop 2023-08-09T13:36:18.696 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:18.693+0000 7f5096bbf740 1 -- 172.21.15.43:0/1606720538 shutdown_connections 2023-08-09T13:36:18.697 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:18.693+0000 7f5096bbf740 1 --2- 172.21.15.43:0/1606720538 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x557de6fc97d0 0x557de6ff3400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:36:18.697 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:18.693+0000 7f5096bbf740 1 --2- 172.21.15.43:0/1606720538 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x557de6fc5bd0 0x557de6ff2ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:36:18.697 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:18.693+0000 7f5096bbf740 1 --2- 172.21.15.43:0/1606720538 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x557de6ff5c50 0x557de6ff6030 unknown :-1 s=CLOSED pgs=585 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:36:18.697 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:18.693+0000 7f5096bbf740 1 -- 172.21.15.43:0/1606720538 >> 172.21.15.43:0/1606720538 conn(0x557de6fc7ee0 msgr2=0x557de7010a60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-09T13:36:18.697 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:18.693+0000 7f5096bbf740 1 -- 172.21.15.43:0/1606720538 shutdown_connections 2023-08-09T13:36:18.697 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:18.693+0000 7f5096bbf740 1 -- 172.21.15.43:0/1606720538 wait complete. 2023-08-09T13:36:18.697 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:18.694+0000 7f5096bbf740 1 -- start start 2023-08-09T13:36:18.700 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:18.694+0000 7f5096bbf740 1 Processor -- start 2023-08-09T13:36:18.700 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:18.695+0000 7f5096bbf740 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x557de6fc97d0 0x557de6fed930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:36:18.700 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:18.695+0000 7f5096bbf740 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x557de6ff5c50 0x557de6ff75e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:36:18.701 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:18.695+0000 7f5096bbf740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x557de6ff7b20 0x557de6ff1a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:36:18.701 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:18.695+0000 7f5096bbf740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x557de6ed0d40 con 0x557de6ff7b20 2023-08-09T13:36:18.701 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:18.695+0000 7f5096bbf740 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x557de6ed09e0 con 0x557de6ff5c50 2023-08-09T13:36:18.701 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:18.695+0000 7f5096bbf740 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x557de6c96f20 con 0x557de6fc97d0 2023-08-09T13:36:18.701 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:18.695+0000 7f5096bbf740 10 client.0 _ll_register_callbacks cb 0x557de6ee9160 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-08-09T13:36:18.701 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:18.695+0000 7f50925ea640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x557de6ff7b20 0x557de6ff1a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:36:18.701 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:18.695+0000 7f50925ea640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x557de6ff7b20 0x557de6ff1a70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:58194/0 (socket says 172.21.15.43:58194) 2023-08-09T13:36:18.702 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:18.695+0000 7f50925ea640 1 -- 172.21.15.43:0/2656037652 learned_addr learned my addr 172.21.15.43:0/2656037652 (peer_addr_for_me v2:172.21.15.43:0/0) 2023-08-09T13:36:18.702 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:18.696+0000 7f50915e8640 1 --2- 172.21.15.43:0/2656037652 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x557de6ff5c50 0x557de6ff75e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:36:18.702 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:18.696+0000 7f50925ea640 1 -- 172.21.15.43:0/2656037652 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x557de6fc97d0 msgr2=0x557de6fed930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-09T13:36:18.702 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:18.696+0000 7f5091de9640 1 --2- 172.21.15.43:0/2656037652 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x557de6fc97d0 0x557de6fed930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:36:18.702 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:18.696+0000 7f50925ea640 1 --2- 172.21.15.43:0/2656037652 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x557de6fc97d0 0x557de6fed930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:36:18.702 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:18.696+0000 7f50925ea640 1 -- 172.21.15.43:0/2656037652 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x557de6ff5c50 msgr2=0x557de6ff75e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-09T13:36:18.702 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:18.696+0000 7f50925ea640 1 --2- 172.21.15.43:0/2656037652 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x557de6ff5c50 0x557de6ff75e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:36:18.702 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:18.696+0000 7f50925ea640 1 -- 172.21.15.43:0/2656037652 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f508800d840 con 0x557de6ff7b20 2023-08-09T13:36:18.703 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:18.696+0000 7f5091de9640 1 --2- 172.21.15.43:0/2656037652 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x557de6fc97d0 0x557de6fed930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-08-09T13:36:18.703 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:18.696+0000 7f50915e8640 1 --2- 172.21.15.43:0/2656037652 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x557de6ff5c50 0x557de6ff75e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-08-09T13:36:18.703 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:18.696+0000 7f50925ea640 1 --2- 172.21.15.43:0/2656037652 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x557de6ff7b20 0x557de6ff1a70 secure :-1 s=READY pgs=586 cs=0 l=1 rev1=1 crypto rx=0x7f508402bfb0 tx=0x7f508407e790 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e9b47cd66a119b5d server_cookie=0 in_seq=0 out_seq=0 2023-08-09T13:36:18.703 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:18.696+0000 7f506cff9640 10 client.0 ms_handle_connect on v2:172.21.15.43:3300/0 2023-08-09T13:36:18.703 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:18.696+0000 7f506cff9640 1 -- 172.21.15.43:0/2656037652 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5084082390 con 0x557de6ff7b20 2023-08-09T13:36:18.703 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:18.696+0000 7f506cff9640 1 -- 172.21.15.43:0/2656037652 <== mon.0 v2:172.21.15.43:3300/0 2 ==== osd_map(39..39 src has 1..39) v4 ==== 6382+0+0 (secure 0 0 0) 0x7f50840767b0 con 0x557de6ff7b20 2023-08-09T13:36:18.703 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:18.696+0000 7f5096bbf740 1 -- 172.21.15.43:0/2656037652 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x557de6f08500 con 0x557de6ff7b20 2023-08-09T13:36:18.703 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:18.697+0000 7f506cff9640 1 -- 172.21.15.43:0/2656037652 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_get_version_reply(handle=1 version=17) v2 ==== 24+0+0 (secure 0 0 0) 0x7f5084076970 con 0x557de6ff7b20 2023-08-09T13:36:18.704 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:18.697+0000 7f5096bbf740 10 client.5802 fetch_fsmap learned FSMap version 17 2023-08-09T13:36:18.704 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:18.697+0000 7f5096bbf740 1 -- 172.21.15.43:0/2656037652 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,fsmap=17,monmap=2+}) v3 -- 0x557de6f2ef50 con 0x557de6ff7b20 2023-08-09T13:36:18.704 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:18.697+0000 7f506cff9640 1 -- 172.21.15.43:0/2656037652 <== mon.0 v2:172.21.15.43:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x557de6f08500 con 0x557de6ff7b20 2023-08-09T13:36:18.704 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:18.697+0000 7f506cff9640 1 -- 172.21.15.43:0/2656037652 <== mon.0 v2:172.21.15.43:3300/0 5 ==== fsmap(e 17) v1 ==== 3294+0+0 (secure 0 0 0) 0x7f508404f020 con 0x557de6ff7b20 2023-08-09T13:36:18.704 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:18.697+0000 7f5096bbf740 10 client.5802 fetch_fsmap finished waiting for FSMap version 17 2023-08-09T13:36:18.704 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:18.697+0000 7f5096bbf740 10 client.5802 resolve_mds: resolved name 'c' to daemon mds.c 2023-08-09T13:36:18.704 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:18.697+0000 7f5096bbf740 20 client.5802 populate_metadata read hostname 'smithi043' 2023-08-09T13:36:18.705 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:18.697+0000 7f5096bbf740 1 --2- 172.21.15.43:0/2656037652 >> [v2:172.21.15.43:6833/2019295969,v1:172.21.15.43:6835/2019295969] conn(0x557de7024300 0x557de70246e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:36:18.705 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:18.697+0000 7f5096bbf740 4 client.5802 mds_command: new command op to 5622 tid=0[{"prefix": "get_command_descriptions"}] 2023-08-09T13:36:18.705 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:18.697+0000 7f5096bbf740 1 -- 172.21.15.43:0/2656037652 --> [v2:172.21.15.43:6833/2019295969,v1:172.21.15.43:6835/2019295969] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x557de6f05bc0 con 0x557de7024300 2023-08-09T13:36:18.705 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:18.697+0000 7f5091de9640 1 --2- 172.21.15.43:0/2656037652 >> [v2:172.21.15.43:6833/2019295969,v1:172.21.15.43:6835/2019295969] conn(0x557de7024300 0x557de70246e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:36:18.705 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:18.698+0000 7f5091de9640 1 --2- 172.21.15.43:0/2656037652 >> [v2:172.21.15.43:6833/2019295969,v1:172.21.15.43:6835/2019295969] conn(0x557de7024300 0x557de70246e0 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c3666ac6e441c9cc server_cookie=253a05457378c013 in_seq=0 out_seq=0 2023-08-09T13:36:18.705 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:18.698+0000 7f506cff9640 10 client.5802 ms_handle_connect on v2:172.21.15.43:6833/2019295969 2023-08-09T13:36:18.706 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:18.699+0000 7f506cff9640 1 -- 172.21.15.43:0/2656037652 <== mds.0 v2:172.21.15.43:6833/2019295969 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18017 (crc 0 0 0) 0x557de6f05bc0 con 0x557de7024300 2023-08-09T13:36:18.706 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:18.699+0000 7f506cff9640 10 client.5802 handle_command_reply: tid=0 2023-08-09T13:36:18.706 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:18.699+0000 7f5096bbf740 1 client.5802 shutdown 2023-08-09T13:36:18.707 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:18.699+0000 7f5096bbf740 1 -- 172.21.15.43:0/2656037652 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x557de6ff7b20 msgr2=0x557de6ff1a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:36:18.707 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:18.699+0000 7f5096bbf740 1 --2- 172.21.15.43:0/2656037652 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x557de6ff7b20 0x557de6ff1a70 secure :-1 s=READY pgs=586 cs=0 l=1 rev1=1 crypto rx=0x7f508402bfb0 tx=0x7f508407e790 comp rx=0 tx=0).stop 2023-08-09T13:36:18.707 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:18.699+0000 7f5096bbf740 1 -- 172.21.15.43:0/2656037652 shutdown_connections 2023-08-09T13:36:18.707 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:18.699+0000 7f5096bbf740 1 --2- 172.21.15.43:0/2656037652 >> [v2:172.21.15.43:6833/2019295969,v1:172.21.15.43:6835/2019295969] conn(0x557de7024300 0x557de70246e0 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:36:18.707 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:18.699+0000 7f5096bbf740 1 --2- 172.21.15.43:0/2656037652 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x557de6ff7b20 0x557de6ff1a70 unknown :-1 s=CLOSED pgs=586 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:36:18.707 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:18.699+0000 7f5096bbf740 1 --2- 172.21.15.43:0/2656037652 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x557de6ff5c50 0x557de6ff75e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:36:18.708 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:18.699+0000 7f5096bbf740 1 --2- 172.21.15.43:0/2656037652 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x557de6fc97d0 0x557de6fed930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:36:18.708 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:18.699+0000 7f506cff9640 0 client.5802 ms_handle_reset on v2:172.21.15.43:6833/2019295969 2023-08-09T13:36:18.708 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:18.699+0000 7f5096bbf740 1 -- 172.21.15.43:0/2656037652 >> 172.21.15.43:0/2656037652 conn(0x557de6fc5bd0 msgr2=0x557de6fca050 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-09T13:36:18.708 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:18.699+0000 7f5096bbf740 1 -- 172.21.15.43:0/2656037652 shutdown_connections 2023-08-09T13:36:18.708 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:18.699+0000 7f5096bbf740 1 -- 172.21.15.43:0/2656037652 wait complete. 2023-08-09T13:36:18.708 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:18.700+0000 7f5096bbf740 20 client.5802 trim_cache size 0 max 16384 2023-08-09T13:36:18.718 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:18.717+0000 7f5096bbf740 1 Processor -- start 2023-08-09T13:36:18.718 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:18.717+0000 7f5096bbf740 1 -- start start 2023-08-09T13:36:18.719 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:18.718+0000 7f5096bbf740 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x557de6fc97d0 0x557de6fcbc30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:36:18.719 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:18.718+0000 7f5096bbf740 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x557de6ff5c50 0x557de6ff7560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:36:18.719 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:18.718+0000 7f5096bbf740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x557de6ff7b20 0x557de70201f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:36:18.719 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:18.718+0000 7f5096bbf740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x557de6fcc950 con 0x557de6ff7b20 2023-08-09T13:36:18.719 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:18.718+0000 7f5096bbf740 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x557de6feceb0 con 0x557de6ff5c50 2023-08-09T13:36:18.720 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:18.718+0000 7f5096bbf740 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x557de6f27870 con 0x557de6fc97d0 2023-08-09T13:36:18.721 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:18.718+0000 7f50925ea640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x557de6ff7b20 0x557de70201f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:36:18.721 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:18.718+0000 7f50925ea640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x557de6ff7b20 0x557de70201f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:58198/0 (socket says 172.21.15.43:58198) 2023-08-09T13:36:18.721 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:18.718+0000 7f50925ea640 1 -- 172.21.15.43:0/702733817 learned_addr learned my addr 172.21.15.43:0/702733817 (peer_addr_for_me v2:172.21.15.43:0/0) 2023-08-09T13:36:18.721 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:18.718+0000 7f50925ea640 1 -- 172.21.15.43:0/702733817 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x557de6fc97d0 msgr2=0x557de6fcbc30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-09T13:36:18.721 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:18.718+0000 7f50925ea640 1 --2- 172.21.15.43:0/702733817 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x557de6fc97d0 0x557de6fcbc30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:36:18.722 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:18.718+0000 7f50925ea640 1 -- 172.21.15.43:0/702733817 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x557de6ff5c50 msgr2=0x557de6ff7560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-09T13:36:18.722 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:18.718+0000 7f50925ea640 1 --2- 172.21.15.43:0/702733817 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x557de6ff5c50 0x557de6ff7560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:36:18.722 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:18.718+0000 7f50925ea640 1 -- 172.21.15.43:0/702733817 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x557de6f62300 con 0x557de6ff7b20 2023-08-09T13:36:18.722 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:18.719+0000 7f50925ea640 1 --2- 172.21.15.43:0/702733817 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x557de6ff7b20 0x557de70201f0 secure :-1 s=READY pgs=587 cs=0 l=1 rev1=1 crypto rx=0x7f508407a6c0 tx=0x7f5084080000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=37a8e31983086e43 server_cookie=0 in_seq=0 out_seq=0 2023-08-09T13:36:18.722 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:18.719+0000 7f506ffff640 1 -- 172.21.15.43:0/702733817 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5084082390 con 0x557de6ff7b20 2023-08-09T13:36:18.722 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:18.719+0000 7f506ffff640 1 -- 172.21.15.43:0/702733817 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5084082620 con 0x557de6ff7b20 2023-08-09T13:36:18.722 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:18.719+0000 7f506ffff640 1 -- 172.21.15.43:0/702733817 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f50840769b0 con 0x557de6ff7b20 2023-08-09T13:36:18.723 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:18.719+0000 7f5096bbf740 1 -- 172.21.15.43:0/702733817 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x557de6ff7b20 msgr2=0x557de70201f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:36:18.723 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:18.719+0000 7f5096bbf740 1 --2- 172.21.15.43:0/702733817 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x557de6ff7b20 0x557de70201f0 secure :-1 s=READY pgs=587 cs=0 l=1 rev1=1 crypto rx=0x7f508407a6c0 tx=0x7f5084080000 comp rx=0 tx=0).stop 2023-08-09T13:36:18.723 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:18.720+0000 7f5096bbf740 1 -- 172.21.15.43:0/702733817 shutdown_connections 2023-08-09T13:36:18.723 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:18.720+0000 7f5096bbf740 1 --2- 172.21.15.43:0/702733817 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x557de6ff7b20 0x557de70201f0 unknown :-1 s=CLOSED pgs=587 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:36:18.723 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:18.720+0000 7f5096bbf740 1 --2- 172.21.15.43:0/702733817 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x557de6ff5c50 0x557de6ff7560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:36:18.723 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:18.720+0000 7f5096bbf740 1 --2- 172.21.15.43:0/702733817 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x557de6fc97d0 0x557de6fcbc30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:36:18.723 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:18.720+0000 7f5096bbf740 1 -- 172.21.15.43:0/702733817 >> 172.21.15.43:0/702733817 conn(0x557de6fc5bd0 msgr2=0x557de6fc1400 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-09T13:36:18.723 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:18.720+0000 7f5096bbf740 1 -- 172.21.15.43:0/702733817 shutdown_connections 2023-08-09T13:36:18.724 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:18.720+0000 7f5096bbf740 1 -- 172.21.15.43:0/702733817 wait complete. 2023-08-09T13:36:18.726 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:18.720+0000 7f5096bbf740 1 -- start start 2023-08-09T13:36:18.726 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:18.720+0000 7f5096bbf740 1 Processor -- start 2023-08-09T13:36:18.726 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:18.721+0000 7f5096bbf740 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x557de6fc97d0 0x557de6fea210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:36:18.726 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:18.721+0000 7f5096bbf740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x557de6ff5c50 0x557de6ff6030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:36:18.726 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:18.721+0000 7f5096bbf740 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x557de6ff7b20 0x557de6fed830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:36:18.727 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:18.722+0000 7f5096bbf740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x557de6ff6d50 con 0x557de6ff5c50 2023-08-09T13:36:18.727 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:18.722+0000 7f5096bbf740 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x557de6fca650 con 0x557de6fc97d0 2023-08-09T13:36:18.727 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:18.722+0000 7f5096bbf740 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x557de6ff2490 con 0x557de6ff7b20 2023-08-09T13:36:18.727 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:18.722+0000 7f5096bbf740 10 client.0 _ll_register_callbacks cb 0x7f508407a5b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-08-09T13:36:18.727 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:18.722+0000 7f50915e8640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x557de6ff5c50 0x557de6ff6030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:36:18.727 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:18.722+0000 7f50915e8640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x557de6ff5c50 0x557de6ff6030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:58204/0 (socket says 172.21.15.43:58204) 2023-08-09T13:36:18.727 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:18.722+0000 7f50915e8640 1 -- 172.21.15.43:0/931157314 learned_addr learned my addr 172.21.15.43:0/931157314 (peer_addr_for_me v2:172.21.15.43:0/0) 2023-08-09T13:36:18.727 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:18.722+0000 7f50925ea640 1 --2- 172.21.15.43:0/931157314 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x557de6ff7b20 0x557de6fed830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:36:18.728 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:18.722+0000 7f5091de9640 1 --2- 172.21.15.43:0/931157314 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x557de6fc97d0 0x557de6fea210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:36:18.728 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:18.722+0000 7f50915e8640 1 -- 172.21.15.43:0/931157314 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x557de6ff7b20 msgr2=0x557de6fed830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-09T13:36:18.728 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:18.722+0000 7f50915e8640 1 --2- 172.21.15.43:0/931157314 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x557de6ff7b20 0x557de6fed830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:36:18.728 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:18.722+0000 7f50915e8640 1 -- 172.21.15.43:0/931157314 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x557de6fc97d0 msgr2=0x557de6fea210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-09T13:36:18.728 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:18.722+0000 7f50915e8640 1 --2- 172.21.15.43:0/931157314 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x557de6fc97d0 0x557de6fea210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:36:18.728 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:18.722+0000 7f50915e8640 1 -- 172.21.15.43:0/931157314 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x557de6f2ef50 con 0x557de6ff5c50 2023-08-09T13:36:18.729 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:18.723+0000 7f50915e8640 1 --2- 172.21.15.43:0/931157314 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x557de6ff5c50 0x557de6ff6030 secure :-1 s=READY pgs=588 cs=0 l=1 rev1=1 crypto rx=0x7f507c000f60 tx=0x7f507c00f600 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a926357f7270bef server_cookie=0 in_seq=0 out_seq=0 2023-08-09T13:36:18.729 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:18.723+0000 7f506cff9640 10 client.0 ms_handle_connect on v2:172.21.15.43:3300/0 2023-08-09T13:36:18.729 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:18.723+0000 7f506cff9640 1 -- 172.21.15.43:0/931157314 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f507c0052d0 con 0x557de6ff5c50 2023-08-09T13:36:18.729 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:18.723+0000 7f506cff9640 1 -- 172.21.15.43:0/931157314 <== mon.0 v2:172.21.15.43:3300/0 2 ==== osd_map(39..39 src has 1..39) v4 ==== 6382+0+0 (secure 0 0 0) 0x7f507c00c050 con 0x557de6ff5c50 2023-08-09T13:36:18.729 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:18.723+0000 7f5096bbf740 1 -- 172.21.15.43:0/931157314 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x557de6f298b0 con 0x557de6ff5c50 2023-08-09T13:36:18.729 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:18.723+0000 7f506cff9640 1 -- 172.21.15.43:0/931157314 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_get_version_reply(handle=1 version=17) v2 ==== 24+0+0 (secure 0 0 0) 0x7f507c001d60 con 0x557de6ff5c50 2023-08-09T13:36:18.729 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:18.723+0000 7f5096bbf740 10 client.5808 fetch_fsmap learned FSMap version 17 2023-08-09T13:36:18.730 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:18.723+0000 7f5096bbf740 1 -- 172.21.15.43:0/931157314 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,fsmap=17,monmap=2+}) v3 -- 0x557de6fccb20 con 0x557de6ff5c50 2023-08-09T13:36:18.730 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:18.724+0000 7f506cff9640 1 -- 172.21.15.43:0/931157314 <== mon.0 v2:172.21.15.43:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x557de6f298b0 con 0x557de6ff5c50 2023-08-09T13:36:18.730 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:18.724+0000 7f506cff9640 1 -- 172.21.15.43:0/931157314 <== mon.0 v2:172.21.15.43:3300/0 5 ==== fsmap(e 17) v1 ==== 3294+0+0 (secure 0 0 0) 0x7f507c007020 con 0x557de6ff5c50 2023-08-09T13:36:18.730 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:18.724+0000 7f5096bbf740 10 client.5808 fetch_fsmap finished waiting for FSMap version 17 2023-08-09T13:36:18.730 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:18.724+0000 7f5096bbf740 10 client.5808 resolve_mds: resolved name 'c' to daemon mds.c 2023-08-09T13:36:18.730 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:18.724+0000 7f5096bbf740 20 client.5808 populate_metadata read hostname 'smithi043' 2023-08-09T13:36:18.731 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:18.724+0000 7f5096bbf740 1 --2- 172.21.15.43:0/931157314 >> [v2:172.21.15.43:6833/2019295969,v1:172.21.15.43:6835/2019295969] conn(0x557de6fedd70 0x557de701ee60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:36:18.731 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:18.724+0000 7f5096bbf740 4 client.5808 mds_command: new command op to 5622 tid=0[{"prefix": "status"}] 2023-08-09T13:36:18.731 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:18.724+0000 7f5096bbf740 1 -- 172.21.15.43:0/931157314 --> [v2:172.21.15.43:6833/2019295969,v1:172.21.15.43:6835/2019295969] -- command(tid 0: {"prefix": "status"}) v1 -- 0x557de6be05e0 con 0x557de6fedd70 2023-08-09T13:36:18.731 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:18.724+0000 7f5091de9640 1 --2- 172.21.15.43:0/931157314 >> [v2:172.21.15.43:6833/2019295969,v1:172.21.15.43:6835/2019295969] conn(0x557de6fedd70 0x557de701ee60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:36:18.731 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:18.725+0000 7f5091de9640 1 --2- 172.21.15.43:0/931157314 >> [v2:172.21.15.43:6833/2019295969,v1:172.21.15.43:6835/2019295969] conn(0x557de6fedd70 0x557de701ee60 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=576781e8bb67289b server_cookie=1b4045ef445e2356 in_seq=0 out_seq=0 2023-08-09T13:36:18.731 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:18.725+0000 7f506cff9640 10 client.5808 ms_handle_connect on v2:172.21.15.43:6833/2019295969 2023-08-09T13:36:18.732 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:18.725+0000 7f506cff9640 1 -- 172.21.15.43:0/931157314 <== mds.0 v2:172.21.15.43:6833/2019295969 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+331 (crc 0 0 0) 0x557de6be05e0 con 0x557de6fedd70 2023-08-09T13:36:18.732 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:18.725+0000 7f506cff9640 10 client.5808 handle_command_reply: tid=0 2023-08-09T13:36:18.733 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:18.725+0000 7f5096bbf740 1 client.5808 shutdown 2023-08-09T13:36:18.733 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:18.725+0000 7f5096bbf740 1 -- 172.21.15.43:0/931157314 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x557de6ff5c50 msgr2=0x557de6ff6030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:36:18.733 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:18.725+0000 7f5096bbf740 1 --2- 172.21.15.43:0/931157314 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x557de6ff5c50 0x557de6ff6030 secure :-1 s=READY pgs=588 cs=0 l=1 rev1=1 crypto rx=0x7f507c000f60 tx=0x7f507c00f600 comp rx=0 tx=0).stop 2023-08-09T13:36:18.733 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:18.725+0000 7f5096bbf740 1 -- 172.21.15.43:0/931157314 shutdown_connections 2023-08-09T13:36:18.733 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:18.725+0000 7f5096bbf740 1 --2- 172.21.15.43:0/931157314 >> [v2:172.21.15.43:6833/2019295969,v1:172.21.15.43:6835/2019295969] conn(0x557de6fedd70 0x557de701ee60 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:36:18.733 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:18.725+0000 7f5096bbf740 1 --2- 172.21.15.43:0/931157314 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x557de6ff7b20 0x557de6fed830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:36:18.733 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:18.725+0000 7f5096bbf740 1 --2- 172.21.15.43:0/931157314 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x557de6ff5c50 0x557de6ff6030 unknown :-1 s=CLOSED pgs=588 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:36:18.734 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:18.725+0000 7f5096bbf740 1 --2- 172.21.15.43:0/931157314 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x557de6fc97d0 0x557de6fea210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:36:18.734 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:18.725+0000 7f506cff9640 0 client.5808 ms_handle_reset on v2:172.21.15.43:6833/2019295969 2023-08-09T13:36:18.734 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:18.725+0000 7f5096bbf740 1 -- 172.21.15.43:0/931157314 >> 172.21.15.43:0/931157314 conn(0x557de6fc5bd0 msgr2=0x557de702a710 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-09T13:36:18.734 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:18.725+0000 7f5096bbf740 1 -- 172.21.15.43:0/931157314 shutdown_connections 2023-08-09T13:36:18.734 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:18.725+0000 7f5096bbf740 1 -- 172.21.15.43:0/931157314 wait complete. 2023-08-09T13:36:18.734 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:18.726+0000 7f5096bbf740 20 client.5808 trim_cache size 0 max 16384 2023-08-09T13:36:18.734 INFO:teuthology.orchestra.run.smithi043.stdout:{ 2023-08-09T13:36:18.735 INFO:teuthology.orchestra.run.smithi043.stdout: "cluster_fsid": "daac87ee-1337-4250-a955-aea4e408a39b", 2023-08-09T13:36:18.735 INFO:teuthology.orchestra.run.smithi043.stdout: "whoami": 0, 2023-08-09T13:36:18.735 INFO:teuthology.orchestra.run.smithi043.stdout: "id": 5622, 2023-08-09T13:36:18.735 INFO:teuthology.orchestra.run.smithi043.stdout: "want_state": "up:active", 2023-08-09T13:36:18.735 INFO:teuthology.orchestra.run.smithi043.stdout: "state": "up:active", 2023-08-09T13:36:18.735 INFO:teuthology.orchestra.run.smithi043.stdout: "fs_name": "cephfs", 2023-08-09T13:36:18.736 INFO:teuthology.orchestra.run.smithi043.stdout: "rank_uptime": 6.8432866729999997, 2023-08-09T13:36:18.736 INFO:teuthology.orchestra.run.smithi043.stdout: "mdsmap_epoch": 17, 2023-08-09T13:36:18.736 INFO:teuthology.orchestra.run.smithi043.stdout: "osdmap_epoch": 39, 2023-08-09T13:36:18.736 INFO:teuthology.orchestra.run.smithi043.stdout: "osdmap_epoch_barrier": 39, 2023-08-09T13:36:18.736 INFO:teuthology.orchestra.run.smithi043.stdout: "uptime": 17.676651536000001 2023-08-09T13:36:18.737 INFO:teuthology.orchestra.run.smithi043.stdout:} 2023-08-09T13:36:18.737 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:18.726+0000 7f509404d640 1 -- 172.21.15.43:0/4194340286 >> [v2:172.21.15.203:6832/830097474,v1:172.21.15.203:6833/830097474] conn(0x7f507006a050 msgr2=0x7f507006c510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:36:18.737 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:18.726+0000 7f509404d640 1 --2- 172.21.15.43:0/4194340286 >> [v2:172.21.15.203:6832/830097474,v1:172.21.15.203:6833/830097474] conn(0x7f507006a050 0x7f507006c510 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7f508800b840 tx=0x7f5088008040 comp rx=0 tx=0).stop 2023-08-09T13:36:18.738 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:18.726+0000 7f509404d640 1 -- 172.21.15.43:0/4194340286 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f508c15c680 msgr2=0x7f508c0ca000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:36:18.738 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:18.726+0000 7f509404d640 1 --2- 172.21.15.43:0/4194340286 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f508c15c680 0x7f508c0ca000 secure :-1 s=READY pgs=584 cs=0 l=1 rev1=1 crypto rx=0x7f5084036040 tx=0x7f5084004a40 comp rx=0 tx=0).stop 2023-08-09T13:36:18.738 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:18.726+0000 7f509404d640 1 -- 172.21.15.43:0/4194340286 shutdown_connections 2023-08-09T13:36:18.738 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:18.726+0000 7f509404d640 1 --2- 172.21.15.43:0/4194340286 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f508c15c680 0x7f508c0ca000 unknown :-1 s=CLOSED pgs=584 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:36:18.738 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:18.726+0000 7f509404d640 1 --2- 172.21.15.43:0/4194340286 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f508c0d0a70 0x7f508c0cb9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:36:18.738 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:18.726+0000 7f509404d640 1 --2- 172.21.15.43:0/4194340286 >> [v2:172.21.15.203:6832/830097474,v1:172.21.15.203:6833/830097474] conn(0x7f507006a050 0x7f507006c510 unknown :-1 s=CLOSED pgs=281 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:36:18.738 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:18.726+0000 7f509404d640 1 --2- 172.21.15.43:0/4194340286 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f508c0b3fa0 0x7f508c149930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:36:18.739 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:18.726+0000 7f509404d640 1 -- 172.21.15.43:0/4194340286 >> 172.21.15.43:0/4194340286 conn(0x7f508c071520 msgr2=0x7f508c157700 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-09T13:36:18.739 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:18.726+0000 7f509404d640 1 -- 172.21.15.43:0/4194340286 shutdown_connections 2023-08-09T13:36:18.739 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:18.726+0000 7f509404d640 1 -- 172.21.15.43:0/4194340286 wait complete. 2023-08-09T13:36:18.745 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2023-08-09T13:36:18.966 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:18.965+0000 7f4953f4e640 1 Processor -- start 2023-08-09T13:36:18.966 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:18.965+0000 7f4953f4e640 1 -- start start 2023-08-09T13:36:18.968 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:18.967+0000 7f4953f4e640 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7f494c143c20 0x7f494c06f540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:36:18.968 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:18.967+0000 7f4953f4e640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f494c06cfe0 0x7f494c0c8b50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:36:18.968 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:18.967+0000 7f4953f4e640 1 -- --> v1:172.21.15.203:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f494c0c9090 con 0x7f494c0c86f0 2023-08-09T13:36:18.969 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:18.967+0000 7f4953f4e640 1 -- --> v2:172.21.15.203:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f494c144000 con 0x7f494c143c20 2023-08-09T13:36:18.969 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:18.967+0000 7f4953f4e640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f494c06c440 con 0x7f494c06cfe0 2023-08-09T13:36:18.969 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:18.968+0000 7f49524eb640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f494c06cfe0 0x7f494c0c8b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:36:18.969 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:18.968+0000 7f49524eb640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f494c06cfe0 0x7f494c0c8b50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.203:3301/0 says I am v2:172.21.15.43:35334/0 (socket says 172.21.15.43:35334) 2023-08-09T13:36:18.970 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:18.968+0000 7f49524eb640 1 -- 172.21.15.43:0/1293493328 learned_addr learned my addr 172.21.15.43:0/1293493328 (peer_addr_for_me v2:172.21.15.43:0/0) 2023-08-09T13:36:18.970 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:18.968+0000 7f4951cea640 1 --2- 172.21.15.43:0/1293493328 >> v2:172.21.15.203:3300/0 conn(0x7f494c143c20 0x7f494c06f540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:36:18.970 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:18.968+0000 7f4951cea640 1 -- 172.21.15.43:0/1293493328 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f494c06cfe0 msgr2=0x7f494c0c8b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-09T13:36:18.970 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:18.968+0000 7f4951cea640 1 --2- 172.21.15.43:0/1293493328 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f494c06cfe0 0x7f494c0c8b50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:36:18.971 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:18.969+0000 7f4951cea640 1 -- 172.21.15.43:0/1293493328 >> v1:172.21.15.203:6789/0 conn(0x7f494c0c86f0 legacy=0x7f494c06c8d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:36:18.971 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:18.969+0000 7f4951cea640 1 -- 172.21.15.43:0/1293493328 --> v2:172.21.15.203:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f494c06b830 con 0x7f494c143c20 2023-08-09T13:36:18.971 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:18.969+0000 7f49524eb640 1 --2- 172.21.15.43:0/1293493328 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f494c06cfe0 0x7f494c0c8b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-08-09T13:36:18.971 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:18.969+0000 7f4950ce8640 1 -- 172.21.15.43:0/1293493328 <== mon.1 v1:172.21.15.203:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2750023924 0 0) 0x7f494c0c9090 con 0x7f494c0c86f0 2023-08-09T13:36:18.972 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:18.970+0000 7f4951cea640 1 --2- 172.21.15.43:0/1293493328 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f494c143c20 0x7f494c06f540 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7f4948009d90 tx=0x7f494802be20 comp rx=0 tx=0).ready entity=mon.1 client_cookie=73fab759524d8b95 server_cookie=0 in_seq=0 out_seq=0 2023-08-09T13:36:18.972 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:18.970+0000 7f4950ce8640 1 -- 172.21.15.43:0/1293493328 <== mon.1 v2:172.21.15.203:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f494802cbd0 con 0x7f494c143c20 2023-08-09T13:36:18.972 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:18.970+0000 7f4950ce8640 1 -- 172.21.15.43:0/1293493328 <== mon.1 v2:172.21.15.203:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f494802cd30 con 0x7f494c143c20 2023-08-09T13:36:18.972 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:18.970+0000 7f4950ce8640 1 -- 172.21.15.43:0/1293493328 <== mon.1 v2:172.21.15.203:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f494802cea0 con 0x7f494c143c20 2023-08-09T13:36:18.973 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:18.971+0000 7f4953f4e640 1 -- 172.21.15.43:0/1293493328 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f494c143c20 msgr2=0x7f494c06f540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:36:18.973 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:18.971+0000 7f4953f4e640 1 --2- 172.21.15.43:0/1293493328 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f494c143c20 0x7f494c06f540 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7f4948009d90 tx=0x7f494802be20 comp rx=0 tx=0).stop 2023-08-09T13:36:18.973 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:18.971+0000 7f4953f4e640 1 -- 172.21.15.43:0/1293493328 shutdown_connections 2023-08-09T13:36:18.973 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:18.971+0000 7f4953f4e640 1 --2- 172.21.15.43:0/1293493328 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f494c06cfe0 0x7f494c0c8b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:36:18.974 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:18.971+0000 7f4953f4e640 1 --2- 172.21.15.43:0/1293493328 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f494c143c20 0x7f494c06f540 unknown :-1 s=CLOSED pgs=217 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:36:18.974 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:18.971+0000 7f4953f4e640 1 -- 172.21.15.43:0/1293493328 >> 172.21.15.43:0/1293493328 conn(0x7f494c0ce0a0 msgr2=0x7f494c0ce480 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-09T13:36:18.974 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:18.971+0000 7f4953f4e640 1 -- 172.21.15.43:0/1293493328 shutdown_connections 2023-08-09T13:36:18.974 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:18.971+0000 7f4953f4e640 1 -- 172.21.15.43:0/1293493328 wait complete. 2023-08-09T13:36:18.974 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:18.973+0000 7f4953f4e640 1 Processor -- start 2023-08-09T13:36:18.975 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:18.973+0000 7f4953f4e640 1 -- start start 2023-08-09T13:36:18.976 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:18.974+0000 7f4953f4e640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f494c06cfe0 0x7f494c140010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:36:18.976 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:18.974+0000 7f4953f4e640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f494c0c86f0 0x7f494c15b130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:36:18.976 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:18.974+0000 7f4953f4e640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f494c143c20 0x7f494c15b670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:36:18.976 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:18.974+0000 7f4953f4e640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f494c140550 con 0x7f494c06cfe0 2023-08-09T13:36:18.976 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:18.974+0000 7f4953f4e640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f494c142540 con 0x7f494c143c20 2023-08-09T13:36:18.976 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:18.974+0000 7f4953f4e640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f494c142260 con 0x7f494c0c86f0 2023-08-09T13:36:18.977 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:18.975+0000 7f4951cea640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f494c06cfe0 0x7f494c140010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:36:18.978 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:18.975+0000 7f4951cea640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f494c06cfe0 0x7f494c140010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:58214/0 (socket says 172.21.15.43:58214) 2023-08-09T13:36:18.978 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:18.975+0000 7f4951cea640 1 -- 172.21.15.43:0/1660068100 learned_addr learned my addr 172.21.15.43:0/1660068100 (peer_addr_for_me v2:172.21.15.43:0/0) 2023-08-09T13:36:18.978 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:18.975+0000 7f49514e9640 1 --2- 172.21.15.43:0/1660068100 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f494c0c86f0 0x7f494c15b130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:36:18.978 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:18.975+0000 7f49524eb640 1 --2- 172.21.15.43:0/1660068100 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f494c143c20 0x7f494c15b670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:36:18.978 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:18.975+0000 7f4951cea640 1 -- 172.21.15.43:0/1660068100 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f494c0c86f0 msgr2=0x7f494c15b130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:36:18.978 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:18.975+0000 7f4951cea640 1 --2- 172.21.15.43:0/1660068100 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f494c0c86f0 0x7f494c15b130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:36:18.978 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:18.975+0000 7f4951cea640 1 -- 172.21.15.43:0/1660068100 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f494c143c20 msgr2=0x7f494c15b670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:36:18.979 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:18.975+0000 7f4951cea640 1 --2- 172.21.15.43:0/1660068100 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f494c143c20 0x7f494c15b670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:36:18.979 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:18.975+0000 7f4951cea640 1 -- 172.21.15.43:0/1660068100 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f49440094d0 con 0x7f494c06cfe0 2023-08-09T13:36:18.979 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:18.976+0000 7f4951cea640 1 --2- 172.21.15.43:0/1660068100 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f494c06cfe0 0x7f494c140010 secure :-1 s=READY pgs=589 cs=0 l=1 rev1=1 crypto rx=0x7f4948009d90 tx=0x7f4948004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-09T13:36:18.979 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:18.976+0000 7f49427fc640 1 -- 172.21.15.43:0/1660068100 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f494802cea0 con 0x7f494c06cfe0 2023-08-09T13:36:18.979 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:18.976+0000 7f49427fc640 1 -- 172.21.15.43:0/1660068100 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f494803e630 con 0x7f494c06cfe0 2023-08-09T13:36:18.979 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:18.976+0000 7f4953f4e640 1 -- 172.21.15.43:0/1660068100 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4948009a70 con 0x7f494c06cfe0 2023-08-09T13:36:18.979 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:18.976+0000 7f49427fc640 1 -- 172.21.15.43:0/1660068100 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f494803e9e0 con 0x7f494c06cfe0 2023-08-09T13:36:18.980 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:18.976+0000 7f4953f4e640 1 -- 172.21.15.43:0/1660068100 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f494c0b1ee0 con 0x7f494c06cfe0 2023-08-09T13:36:18.980 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:18.977+0000 7f4953f4e640 1 -- 172.21.15.43:0/1660068100 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f494c0739f0 con 0x7f494c06cfe0 2023-08-09T13:36:18.980 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:18.978+0000 7f49427fc640 1 -- 172.21.15.43:0/1660068100 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 5) v1 ==== 88341+0+0 (secure 0 0 0) 0x7f494803b050 con 0x7f494c06cfe0 2023-08-09T13:36:18.981 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:18.979+0000 7f49427fc640 1 --2- 172.21.15.43:0/1660068100 >> [v2:172.21.15.203:6832/830097474,v1:172.21.15.203:6833/830097474] conn(0x7f493006a080 0x7f493006c540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:36:18.981 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:18.979+0000 7f49427fc640 1 -- 172.21.15.43:0/1660068100 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(39..39 src has 1..39) v4 ==== 6382+0+0 (secure 0 0 0) 0x7f49480aa850 con 0x7f494c06cfe0 2023-08-09T13:36:18.981 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:18.980+0000 7f49514e9640 1 --2- 172.21.15.43:0/1660068100 >> [v2:172.21.15.203:6832/830097474,v1:172.21.15.203:6833/830097474] conn(0x7f493006a080 0x7f493006c540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:36:18.982 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:18.981+0000 7f49514e9640 1 --2- 172.21.15.43:0/1660068100 >> [v2:172.21.15.203:6832/830097474,v1:172.21.15.203:6833/830097474] conn(0x7f493006a080 0x7f493006c540 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7f493c007fd0 tx=0x7f493c009040 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-09T13:36:18.985 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:18.984+0000 7f49427fc640 1 -- 172.21.15.43:0/1660068100 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178066 (secure 0 0 0) 0x7f4948075d70 con 0x7f494c06cfe0 2023-08-09T13:36:19.215 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:19.214+0000 7f4953f4e640 1 -- 172.21.15.43:0/1660068100 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7f494c0cec90 con 0x7f494c06cfe0 2023-08-09T13:36:19.216 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:19.215+0000 7f49427fc640 1 -- 172.21.15.43:0/1660068100 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v17) v1 ==== 53+0+73 (secure 0 0 0) 0x7f49480a8e60 con 0x7f494c06cfe0 2023-08-09T13:36:19.217 INFO:teuthology.orchestra.run.smithi043.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2023-08-09T13:36:19.219 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:19.218+0000 7f4953f4e640 1 -- 172.21.15.43:0/1660068100 >> [v2:172.21.15.203:6832/830097474,v1:172.21.15.203:6833/830097474] conn(0x7f493006a080 msgr2=0x7f493006c540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:36:19.219 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:19.218+0000 7f4953f4e640 1 --2- 172.21.15.43:0/1660068100 >> [v2:172.21.15.203:6832/830097474,v1:172.21.15.203:6833/830097474] conn(0x7f493006a080 0x7f493006c540 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7f493c007fd0 tx=0x7f493c009040 comp rx=0 tx=0).stop 2023-08-09T13:36:19.220 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:19.218+0000 7f4953f4e640 1 -- 172.21.15.43:0/1660068100 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f494c06cfe0 msgr2=0x7f494c140010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:36:19.220 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:19.218+0000 7f4953f4e640 1 --2- 172.21.15.43:0/1660068100 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f494c06cfe0 0x7f494c140010 secure :-1 s=READY pgs=589 cs=0 l=1 rev1=1 crypto rx=0x7f4948009d90 tx=0x7f4948004a40 comp rx=0 tx=0).stop 2023-08-09T13:36:19.220 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:19.218+0000 7f4953f4e640 1 -- 172.21.15.43:0/1660068100 shutdown_connections 2023-08-09T13:36:19.220 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:19.218+0000 7f4953f4e640 1 --2- 172.21.15.43:0/1660068100 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f494c143c20 0x7f494c15b670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:36:19.220 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:19.218+0000 7f4953f4e640 1 --2- 172.21.15.43:0/1660068100 >> [v2:172.21.15.203:6832/830097474,v1:172.21.15.203:6833/830097474] conn(0x7f493006a080 0x7f493006c540 unknown :-1 s=CLOSED pgs=282 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:36:19.220 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:19.218+0000 7f4953f4e640 1 --2- 172.21.15.43:0/1660068100 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f494c0c86f0 0x7f494c15b130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:36:19.220 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:19.218+0000 7f4953f4e640 1 --2- 172.21.15.43:0/1660068100 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f494c06cfe0 0x7f494c140010 unknown :-1 s=CLOSED pgs=589 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:36:19.220 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:19.218+0000 7f4953f4e640 1 -- 172.21.15.43:0/1660068100 >> 172.21.15.43:0/1660068100 conn(0x7f494c0ce0a0 msgr2=0x7f494c0adc20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-09T13:36:19.221 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:19.218+0000 7f4953f4e640 1 -- 172.21.15.43:0/1660068100 shutdown_connections 2023-08-09T13:36:19.221 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:19.218+0000 7f4953f4e640 1 -- 172.21.15.43:0/1660068100 wait complete. 2023-08-09T13:36:19.236 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2023-08-09T13:36:19.237 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2023-08-09T13:36:19.237 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi043.front.sepia.ceph.com 2023-08-09T13:36:19.237 INFO:tasks.cephfs.mount:self.client.name = client.0 2023-08-09T13:36:19.238 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2023-08-09T13:36:19.238 INFO:tasks.cephfs.mount:self.cephfs_name = None 2023-08-09T13:36:19.238 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2023-08-09T13:36:19.238 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2023-08-09T13:36:19.239 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2023-08-09T13:36:19.239 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-09T13:36:19.239 DEBUG:teuthology.orchestra.run.smithi043:> ip addr 2023-08-09T13:36:19.292 INFO:teuthology.orchestra.run.smithi043.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2023-08-09T13:36:19.292 INFO:teuthology.orchestra.run.smithi043.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2023-08-09T13:36:19.292 INFO:teuthology.orchestra.run.smithi043.stdout: inet 127.0.0.1/8 scope host lo 2023-08-09T13:36:19.293 INFO:teuthology.orchestra.run.smithi043.stdout: valid_lft forever preferred_lft forever 2023-08-09T13:36:19.293 INFO:teuthology.orchestra.run.smithi043.stdout: inet6 ::1/128 scope host 2023-08-09T13:36:19.293 INFO:teuthology.orchestra.run.smithi043.stdout: valid_lft forever preferred_lft forever 2023-08-09T13:36:19.293 INFO:teuthology.orchestra.run.smithi043.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-08-09T13:36:19.293 INFO:teuthology.orchestra.run.smithi043.stdout: link/ether 0c:c4:7a:6b:fd:9c brd ff:ff:ff:ff:ff:ff 2023-08-09T13:36:19.293 INFO:teuthology.orchestra.run.smithi043.stdout: altname enp7s0f0 2023-08-09T13:36:19.293 INFO:teuthology.orchestra.run.smithi043.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-08-09T13:36:19.294 INFO:teuthology.orchestra.run.smithi043.stdout: link/ether 0c:c4:7a:6b:fd:9d brd ff:ff:ff:ff:ff:ff 2023-08-09T13:36:19.294 INFO:teuthology.orchestra.run.smithi043.stdout: altname enp7s0f1 2023-08-09T13:36:19.294 INFO:teuthology.orchestra.run.smithi043.stdout:4: ens1f0: mtu 1500 qdisc mq state UP group default qlen 1000 2023-08-09T13:36:19.294 INFO:teuthology.orchestra.run.smithi043.stdout: link/ether 0c:c4:7a:bd:13:58 brd ff:ff:ff:ff:ff:ff 2023-08-09T13:36:19.294 INFO:teuthology.orchestra.run.smithi043.stdout: altname enp1s0f0 2023-08-09T13:36:19.294 INFO:teuthology.orchestra.run.smithi043.stdout: inet 172.21.15.43/20 brd 172.21.15.255 scope global dynamic noprefixroute ens1f0 2023-08-09T13:36:19.294 INFO:teuthology.orchestra.run.smithi043.stdout: valid_lft 42124sec preferred_lft 42124sec 2023-08-09T13:36:19.294 INFO:teuthology.orchestra.run.smithi043.stdout: inet6 fe80::ec4:7aff:febd:1358/64 scope link 2023-08-09T13:36:19.295 INFO:teuthology.orchestra.run.smithi043.stdout: valid_lft forever preferred_lft forever 2023-08-09T13:36:19.295 INFO:teuthology.orchestra.run.smithi043.stdout:5: ens1f1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-08-09T13:36:19.295 INFO:teuthology.orchestra.run.smithi043.stdout: link/ether 0c:c4:7a:bd:13:59 brd ff:ff:ff:ff:ff:ff 2023-08-09T13:36:19.295 INFO:teuthology.orchestra.run.smithi043.stdout: altname enp1s0f1 2023-08-09T13:36:19.295 INFO:teuthology.orchestra.run.smithi043.stdout:8: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2023-08-09T13:36:19.295 INFO:teuthology.orchestra.run.smithi043.stdout: link/ether 56:43:50:19:f4:0c brd ff:ff:ff:ff:ff:ff 2023-08-09T13:36:19.295 INFO:teuthology.orchestra.run.smithi043.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2023-08-09T13:36:19.295 INFO:teuthology.orchestra.run.smithi043.stdout: valid_lft forever preferred_lft forever 2023-08-09T13:36:19.296 INFO:teuthology.orchestra.run.smithi043.stdout: inet6 fe80::e494:65ff:fe92:a182/64 scope link 2023-08-09T13:36:19.296 INFO:teuthology.orchestra.run.smithi043.stdout: valid_lft forever preferred_lft forever 2023-08-09T13:36:19.296 INFO:teuthology.orchestra.run.smithi043.stdout:9: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2023-08-09T13:36:19.296 INFO:teuthology.orchestra.run.smithi043.stdout: link/ether 7e:93:26:fa:07:8f brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.0 2023-08-09T13:36:19.296 INFO:teuthology.orchestra.run.smithi043.stdout: inet6 fe80::7c93:26ff:fefa:78f/64 scope link 2023-08-09T13:36:19.296 INFO:teuthology.orchestra.run.smithi043.stdout: valid_lft forever preferred_lft forever 2023-08-09T13:36:19.296 INFO:teuthology.orchestra.run.smithi043.stdout:10: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2023-08-09T13:36:19.297 INFO:teuthology.orchestra.run.smithi043.stdout: link/ether 56:43:50:19:f4:0c brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.2 2023-08-09T13:36:19.297 INFO:teuthology.orchestra.run.smithi043.stdout: inet6 fe80::5443:50ff:fe19:f40c/64 scope link 2023-08-09T13:36:19.297 INFO:teuthology.orchestra.run.smithi043.stdout: valid_lft forever preferred_lft forever 2023-08-09T13:36:19.298 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-09T13:36:19.298 DEBUG:teuthology.orchestra.run.smithi043:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2023-08-09T13:36:19.391 INFO:teuthology.orchestra.run.smithi043.stdout:1 2023-08-09T13:36:19.395 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-09T13:36:19.395 DEBUG:teuthology.orchestra.run.smithi043:> route 2023-08-09T13:36:19.450 INFO:teuthology.orchestra.run.smithi043.stdout:Kernel IP routing table 2023-08-09T13:36:19.450 INFO:teuthology.orchestra.run.smithi043.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2023-08-09T13:36:19.450 INFO:teuthology.orchestra.run.smithi043.stdout:default _gateway 0.0.0.0 UG 100 0 0 ens1f0 2023-08-09T13:36:19.451 INFO:teuthology.orchestra.run.smithi043.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 ens1f0 2023-08-09T13:36:19.451 INFO:teuthology.orchestra.run.smithi043.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2023-08-09T13:36:19.452 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-09T13:36:19.452 DEBUG:teuthology.orchestra.run.smithi043:> (cd / && exec bash -c ' 2023-08-09T13:36:19.452 DEBUG:teuthology.orchestra.run.smithi043:> set -e 2023-08-09T13:36:19.453 DEBUG:teuthology.orchestra.run.smithi043:> sudo iptables -A FORWARD -o ens1f0 -i ceph-brx -j ACCEPT 2023-08-09T13:36:19.453 DEBUG:teuthology.orchestra.run.smithi043:> sudo iptables -A FORWARD -i ens1f0 -o ceph-brx -j ACCEPT 2023-08-09T13:36:19.453 DEBUG:teuthology.orchestra.run.smithi043:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o ens1f0 -j MASQUERADE 2023-08-09T13:36:19.454 DEBUG:teuthology.orchestra.run.smithi043:> ') 2023-08-09T13:36:19.655 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-09T13:36:19.656 DEBUG:teuthology.orchestra.run.smithi043:> ip netns list 2023-08-09T13:36:19.711 INFO:teuthology.orchestra.run.smithi043.stdout:ceph-ns--home-ubuntu-cephtest-mnt.2 (id: 1) 2023-08-09T13:36:19.711 INFO:teuthology.orchestra.run.smithi043.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2023-08-09T13:36:19.712 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2023-08-09T13:36:19.713 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-09T13:36:19.713 DEBUG:teuthology.orchestra.run.smithi043:> sudo ip link set brx.0 up 2023-08-09T13:36:19.820 INFO:tasks.cephfs.fuse_mount:Client client.0 config is {} 2023-08-09T13:36:19.820 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-08-09T13:36:19.821 DEBUG:teuthology.orchestra.run.smithi043:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2023-08-09T13:36:19.880 INFO:teuthology.orchestra.run.smithi043.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2023-08-09T13:36:19.880 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-08-09T13:36:19.881 DEBUG:teuthology.orchestra.run.smithi043:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2023-08-09T13:36:19.935 DEBUG:teuthology.orchestra.run.smithi043:> sudo modprobe fuse 2023-08-09T13:36:20.033 INFO:teuthology.orchestra.run:Running command with timeout 30 2023-08-09T13:36:20.034 DEBUG:teuthology.orchestra.run.smithi043:> sudo mount -t fusectl /sys/fs/fuse/connections /sys/fs/fuse/connections 2023-08-09T13:36:20.137 INFO:teuthology.orchestra.run.smithi043.stderr:mount: /sys/fs/fuse/connections: /sys/fs/fuse/connections already mounted or mount point busy. 2023-08-09T13:36:20.140 DEBUG:teuthology.orchestra.run:got remote process result: 32 2023-08-09T13:36:20.141 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-09T13:36:20.141 DEBUG:teuthology.orchestra.run.smithi043:> ls /sys/fs/fuse/connections 2023-08-09T13:36:20.201 INFO:tasks.cephfs.fuse_mount:Pre-mount connections: [] 2023-08-09T13:36:20.201 DEBUG:teuthology.orchestra.run.smithi043:> (cd /home/ubuntu/cephtest && exec sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-fuse -f --admin-socket '/var/run/ceph/$cluster-$name.$pid.asok' /home/ubuntu/cephtest/mnt.0 --id 0) 2023-08-09T13:36:20.207 DEBUG:teuthology.orchestra.run.smithi043:> sudo modprobe fuse 2023-08-09T13:36:20.293 INFO:teuthology.orchestra.run:Running command with timeout 30 2023-08-09T13:36:20.293 DEBUG:teuthology.orchestra.run.smithi043:> sudo mount -t fusectl /sys/fs/fuse/connections /sys/fs/fuse/connections 2023-08-09T13:36:20.369 INFO:teuthology.orchestra.run.smithi043.stderr:mount: /sys/fs/fuse/connections: /sys/fs/fuse/connections already mounted or mount point busy. 2023-08-09T13:36:20.372 DEBUG:teuthology.orchestra.run:got remote process result: 32 2023-08-09T13:36:20.373 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-09T13:36:20.373 DEBUG:teuthology.orchestra.run.smithi043:> ls /sys/fs/fuse/connections 2023-08-09T13:36:20.413 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi043.stderr:2023-08-09T13:36:20.411+0000 7f7a9860b480 -1 init, newargv = 0x560e7d1c6460 newargc=15 2023-08-09T13:36:20.413 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi043.stderr:ceph-fuse[105196]: starting ceph client 2023-08-09T13:36:20.422 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi043.stderr:ceph-fuse[105196]: starting fuse 2023-08-09T13:36:21.521 DEBUG:teuthology.orchestra.run.smithi043:> sudo modprobe fuse 2023-08-09T13:36:21.636 INFO:teuthology.orchestra.run:Running command with timeout 30 2023-08-09T13:36:21.637 DEBUG:teuthology.orchestra.run.smithi043:> sudo mount -t fusectl /sys/fs/fuse/connections /sys/fs/fuse/connections 2023-08-09T13:36:21.747 INFO:teuthology.orchestra.run.smithi043.stderr:mount: /sys/fs/fuse/connections: /sys/fs/fuse/connections already mounted or mount point busy. 2023-08-09T13:36:21.751 DEBUG:teuthology.orchestra.run:got remote process result: 32 2023-08-09T13:36:21.752 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-09T13:36:21.752 DEBUG:teuthology.orchestra.run.smithi043:> ls /sys/fs/fuse/connections 2023-08-09T13:36:21.810 INFO:teuthology.orchestra.run.smithi043.stdout:40 2023-08-09T13:36:21.811 INFO:tasks.cephfs.fuse_mount:Post-mount connections: [40] 2023-08-09T13:36:21.811 DEBUG:teuthology.orchestra.run.smithi043:> sudo stdin-killer -- python3 -c ' 2023-08-09T13:36:21.811 DEBUG:teuthology.orchestra.run.smithi043:> import glob 2023-08-09T13:36:21.812 DEBUG:teuthology.orchestra.run.smithi043:> import re 2023-08-09T13:36:21.812 DEBUG:teuthology.orchestra.run.smithi043:> import os 2023-08-09T13:36:21.812 DEBUG:teuthology.orchestra.run.smithi043:> import subprocess 2023-08-09T13:36:21.813 DEBUG:teuthology.orchestra.run.smithi043:> 2023-08-09T13:36:21.813 DEBUG:teuthology.orchestra.run.smithi043:> def _find_admin_socket(client_name): 2023-08-09T13:36:21.813 DEBUG:teuthology.orchestra.run.smithi043:> asok_path = "/var/run/ceph/ceph-client.0.*.asok" 2023-08-09T13:36:21.814 DEBUG:teuthology.orchestra.run.smithi043:> files = glob.glob(asok_path) 2023-08-09T13:36:21.814 DEBUG:teuthology.orchestra.run.smithi043:> mountpoint = "/home/ubuntu/cephtest/mnt.0" 2023-08-09T13:36:21.815 DEBUG:teuthology.orchestra.run.smithi043:> 2023-08-09T13:36:21.815 DEBUG:teuthology.orchestra.run.smithi043:> # Given a non-glob path, it better be there 2023-08-09T13:36:21.815 DEBUG:teuthology.orchestra.run.smithi043:> if "*" not in asok_path: 2023-08-09T13:36:21.816 DEBUG:teuthology.orchestra.run.smithi043:> assert(len(files) == 1) 2023-08-09T13:36:21.816 DEBUG:teuthology.orchestra.run.smithi043:> return files[0] 2023-08-09T13:36:21.816 DEBUG:teuthology.orchestra.run.smithi043:> 2023-08-09T13:36:21.817 DEBUG:teuthology.orchestra.run.smithi043:> for f in files: 2023-08-09T13:36:21.817 DEBUG:teuthology.orchestra.run.smithi043:> pid = re.match(".*\.(\d+)\.asok$", f).group(1) 2023-08-09T13:36:21.817 DEBUG:teuthology.orchestra.run.smithi043:> if os.path.exists("/proc/{0}".format(pid)): 2023-08-09T13:36:21.818 DEBUG:teuthology.orchestra.run.smithi043:> with open("/proc/{0}/cmdline".format(pid), '"'"'r'"'"') as proc_f: 2023-08-09T13:36:21.818 DEBUG:teuthology.orchestra.run.smithi043:> contents = proc_f.read() 2023-08-09T13:36:21.818 DEBUG:teuthology.orchestra.run.smithi043:> if mountpoint in contents: 2023-08-09T13:36:21.819 DEBUG:teuthology.orchestra.run.smithi043:> return f 2023-08-09T13:36:21.819 DEBUG:teuthology.orchestra.run.smithi043:> raise RuntimeError("Client socket {0} not found".format(client_name)) 2023-08-09T13:36:21.819 DEBUG:teuthology.orchestra.run.smithi043:> 2023-08-09T13:36:21.819 DEBUG:teuthology.orchestra.run.smithi043:> print(_find_admin_socket("client.0")) 2023-08-09T13:36:21.820 DEBUG:teuthology.orchestra.run.smithi043:> ' 2023-08-09T13:36:22.015 INFO:teuthology.orchestra.run.smithi043.stdout:/var/run/ceph/ceph-client.0.105196.asok 2023-08-09T13:36:22.020 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:22 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2023-08-09T13:36:22.034 INFO:tasks.cephfs.fuse_mount:Found client admin socket at /var/run/ceph/ceph-client.0.105196.asok 2023-08-09T13:36:22.034 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-09T13:36:22.035 DEBUG:teuthology.orchestra.run.smithi043:> sudo ceph --admin-daemon /var/run/ceph/ceph-client.0.105196.asok status 2023-08-09T13:36:22.288 INFO:teuthology.orchestra.run.smithi043.stdout:{ 2023-08-09T13:36:22.288 INFO:teuthology.orchestra.run.smithi043.stdout: "metadata": { 2023-08-09T13:36:22.288 INFO:teuthology.orchestra.run.smithi043.stdout: "ceph_sha1": "668a86f232c9454b2ce5928d1432a29a72964d85", 2023-08-09T13:36:22.288 INFO:teuthology.orchestra.run.smithi043.stdout: "ceph_version": "ceph version 18.0.0-5389-g668a86f2 (668a86f232c9454b2ce5928d1432a29a72964d85) reef (dev)", 2023-08-09T13:36:22.288 INFO:teuthology.orchestra.run.smithi043.stdout: "entity_id": "0", 2023-08-09T13:36:22.289 INFO:teuthology.orchestra.run.smithi043.stdout: "hostname": "smithi043", 2023-08-09T13:36:22.289 INFO:teuthology.orchestra.run.smithi043.stdout: "mount_point": "/home/ubuntu/cephtest/mnt.0", 2023-08-09T13:36:22.289 INFO:teuthology.orchestra.run.smithi043.stdout: "pid": "105196", 2023-08-09T13:36:22.289 INFO:teuthology.orchestra.run.smithi043.stdout: "root": "/" 2023-08-09T13:36:22.289 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2023-08-09T13:36:22.289 INFO:teuthology.orchestra.run.smithi043.stdout: "dentry_count": 0, 2023-08-09T13:36:22.289 INFO:teuthology.orchestra.run.smithi043.stdout: "dentry_pinned_count": 0, 2023-08-09T13:36:22.290 INFO:teuthology.orchestra.run.smithi043.stdout: "id": 5817, 2023-08-09T13:36:22.290 INFO:teuthology.orchestra.run.smithi043.stdout: "inst": { 2023-08-09T13:36:22.290 INFO:teuthology.orchestra.run.smithi043.stdout: "name": { 2023-08-09T13:36:22.290 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "client", 2023-08-09T13:36:22.290 INFO:teuthology.orchestra.run.smithi043.stdout: "num": 5817 2023-08-09T13:36:22.290 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2023-08-09T13:36:22.290 INFO:teuthology.orchestra.run.smithi043.stdout: "addr": { 2023-08-09T13:36:22.290 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "v1", 2023-08-09T13:36:22.291 INFO:teuthology.orchestra.run.smithi043.stdout: "addr": "192.168.0.1:0", 2023-08-09T13:36:22.291 INFO:teuthology.orchestra.run.smithi043.stdout: "nonce": 3343918528 2023-08-09T13:36:22.291 INFO:teuthology.orchestra.run.smithi043.stdout: } 2023-08-09T13:36:22.291 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2023-08-09T13:36:22.291 INFO:teuthology.orchestra.run.smithi043.stdout: "addr": { 2023-08-09T13:36:22.291 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "v1", 2023-08-09T13:36:22.291 INFO:teuthology.orchestra.run.smithi043.stdout: "addr": "192.168.0.1:0", 2023-08-09T13:36:22.291 INFO:teuthology.orchestra.run.smithi043.stdout: "nonce": 3343918528 2023-08-09T13:36:22.292 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2023-08-09T13:36:22.292 INFO:teuthology.orchestra.run.smithi043.stdout: "inst_str": "client.5817 192.168.0.1:0/3343918528", 2023-08-09T13:36:22.292 INFO:teuthology.orchestra.run.smithi043.stdout: "addr_str": "192.168.0.1:0/3343918528", 2023-08-09T13:36:22.292 INFO:teuthology.orchestra.run.smithi043.stdout: "inode_count": 1, 2023-08-09T13:36:22.292 INFO:teuthology.orchestra.run.smithi043.stdout: "mds_epoch": 17, 2023-08-09T13:36:22.292 INFO:teuthology.orchestra.run.smithi043.stdout: "osd_epoch": 39, 2023-08-09T13:36:22.292 INFO:teuthology.orchestra.run.smithi043.stdout: "osd_epoch_barrier": 0, 2023-08-09T13:36:22.293 INFO:teuthology.orchestra.run.smithi043.stdout: "blocklisted": false, 2023-08-09T13:36:22.293 INFO:teuthology.orchestra.run.smithi043.stdout: "fs_name": "cephfs" 2023-08-09T13:36:22.293 INFO:teuthology.orchestra.run.smithi043.stdout:} 2023-08-09T13:36:22.303 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-09T13:36:22.304 DEBUG:teuthology.orchestra.run.smithi043:> stat --file-system '--printf=%T 2023-08-09T13:36:22.304 DEBUG:teuthology.orchestra.run.smithi043:> ' -- /home/ubuntu/cephtest/mnt.0 2023-08-09T13:36:22.361 INFO:teuthology.orchestra.run.smithi043.stdout:fuseblk 2023-08-09T13:36:22.362 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.0 2023-08-09T13:36:22.362 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-09T13:36:22.362 DEBUG:teuthology.orchestra.run.smithi043:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2023-08-09T13:36:22.464 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs volume ls 2023-08-09T13:36:22.702 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:22.701+0000 7fee82691640 1 Processor -- start 2023-08-09T13:36:22.703 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:22.701+0000 7fee82691640 1 -- start start 2023-08-09T13:36:22.704 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:22.702+0000 7fee82691640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fee7c15f7b0 0x7fee7c0b4760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:36:22.704 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:22.703+0000 7fee82691640 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7fee7c0b1050 0x7fee7c0ccfa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:36:22.704 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:22.703+0000 7fee82691640 1 -- --> v1:172.21.15.43:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fee7c0cd930 con 0x7fee7c0cd4e0 2023-08-09T13:36:22.704 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:22.703+0000 7fee82691640 1 -- --> v2:172.21.15.203:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fee7c0b7bf0 con 0x7fee7c0b1050 2023-08-09T13:36:22.705 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:22.703+0000 7fee82691640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fee7c0b7ed0 con 0x7fee7c15f7b0 2023-08-09T13:36:22.706 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:22.703+0000 7fee81e90640 1 --1- >> v1:172.21.15.43:6789/0 conn(0x7fee7c0cd4e0 0x7fee7c0ceae0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.43:6789/0 says I am v1:172.21.15.43:35204/0 (socket says 172.21.15.43:35204) 2023-08-09T13:36:22.706 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:22.703+0000 7fee81e90640 1 -- 172.21.15.43:0/196501196 learned_addr learned my addr 172.21.15.43:0/196501196 (peer_addr_for_me v1:172.21.15.43:0/0) 2023-08-09T13:36:22.706 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:22.703+0000 7fee8168f640 1 --2- 172.21.15.43:0/196501196 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fee7c15f7b0 0x7fee7c0b4760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:36:22.707 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:22.703+0000 7fee80e8e640 1 --2- 172.21.15.43:0/196501196 >> v2:172.21.15.203:3300/0 conn(0x7fee7c0b1050 0x7fee7c0ccfa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:36:22.707 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:22.704+0000 7fee73fff640 1 -- 172.21.15.43:0/196501196 <== mon.0 v1:172.21.15.43:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 812876458 0 0) 0x7fee74001490 con 0x7fee7c0cd4e0 2023-08-09T13:36:22.707 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:22.704+0000 7fee73fff640 1 -- 172.21.15.43:0/196501196 --> v1:172.21.15.43:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fee68003520 con 0x7fee7c0cd4e0 2023-08-09T13:36:22.707 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:22.704+0000 7fee73fff640 1 -- 172.21.15.43:0/196501196 <== mon.0 v1:172.21.15.43:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3310705088 0 0) 0x7fee68003520 con 0x7fee7c0cd4e0 2023-08-09T13:36:22.707 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:22.704+0000 7fee73fff640 1 -- 172.21.15.43:0/196501196 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fee7c15f7b0 msgr2=0x7fee7c0b4760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-09T13:36:22.707 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:22.704+0000 7fee73fff640 1 --2- 172.21.15.43:0/196501196 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fee7c15f7b0 0x7fee7c0b4760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:36:22.708 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:22.704+0000 7fee73fff640 1 -- 172.21.15.43:0/196501196 >> v2:172.21.15.203:3300/0 conn(0x7fee7c0b1050 msgr2=0x7fee7c0ccfa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-09T13:36:22.708 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:22.704+0000 7fee73fff640 1 --2- 172.21.15.43:0/196501196 >> v2:172.21.15.203:3300/0 conn(0x7fee7c0b1050 0x7fee7c0ccfa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:36:22.708 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:22.704+0000 7fee73fff640 1 -- 172.21.15.43:0/196501196 --> v1:172.21.15.43:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fee7c0d0370 con 0x7fee7c0cd4e0 2023-08-09T13:36:22.708 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:22.704+0000 7fee73fff640 1 -- 172.21.15.43:0/196501196 <== mon.0 v1:172.21.15.43:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1285150740 0 0) 0x7fee74001b80 con 0x7fee7c0cd4e0 2023-08-09T13:36:22.708 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:22.704+0000 7fee73fff640 1 -- 172.21.15.43:0/196501196 >> v1:172.21.15.43:6789/0 conn(0x7fee7c0cd4e0 legacy=0x7fee7c0ceae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:36:22.708 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:22.704+0000 7fee73fff640 1 --2- 172.21.15.43:0/196501196 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fee7c15f7b0 0x7fee68003970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:36:22.709 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:22.704+0000 7fee73fff640 1 --2- 172.21.15.43:0/196501196 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fee68003ef0 0x7fee680047c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:36:22.709 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:22.704+0000 7fee73fff640 1 --2- 172.21.15.43:0/196501196 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fee68004d00 0x7fee68005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:36:22.709 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:22.704+0000 7fee73fff640 1 -- 172.21.15.43:0/196501196 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fee7c0b7bf0 con 0x7fee68003ef0 2023-08-09T13:36:22.709 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:22.704+0000 7fee73fff640 1 -- 172.21.15.43:0/196501196 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fee7c0b7ed0 con 0x7fee68004d00 2023-08-09T13:36:22.709 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:22.704+0000 7fee73fff640 1 -- 172.21.15.43:0/196501196 --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fee6800b980 con 0x7fee7c15f7b0 2023-08-09T13:36:22.709 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:22.704+0000 7fee80e8e640 1 --2- 172.21.15.43:0/196501196 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fee68003ef0 0x7fee680047c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:36:22.709 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:22.705+0000 7fee8168f640 1 --2- 172.21.15.43:0/196501196 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fee7c15f7b0 0x7fee68003970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:36:22.710 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:22.705+0000 7fee80e8e640 1 -- 172.21.15.43:0/196501196 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fee7c15f7b0 msgr2=0x7fee68003970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-09T13:36:22.710 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:22.705+0000 7fee80e8e640 1 --2- 172.21.15.43:0/196501196 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fee7c15f7b0 0x7fee68003970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:36:22.710 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:22.705+0000 7fee80e8e640 1 -- 172.21.15.43:0/196501196 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fee68004d00 msgr2=0x7fee68005180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-09T13:36:22.710 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:22.705+0000 7fee80e8e640 1 --2- 172.21.15.43:0/196501196 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fee68004d00 0x7fee68005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:36:22.710 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:22.705+0000 7fee80e8e640 1 -- 172.21.15.43:0/196501196 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fee68003520 con 0x7fee68003ef0 2023-08-09T13:36:22.711 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:22.706+0000 7fee80e8e640 1 --2- 172.21.15.43:0/196501196 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fee68003ef0 0x7fee680047c0 secure :-1 s=READY pgs=593 cs=0 l=1 rev1=1 crypto rx=0x7fee6c009ac0 tx=0x7fee680057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c0f69b7df8e0536 server_cookie=0 in_seq=0 out_seq=0 2023-08-09T13:36:22.712 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:22.706+0000 7fee73fff640 1 -- 172.21.15.43:0/196501196 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fee6c02c6d0 con 0x7fee68003ef0 2023-08-09T13:36:22.712 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:22.706+0000 7fee73fff640 1 -- 172.21.15.43:0/196501196 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fee6c02c830 con 0x7fee68003ef0 2023-08-09T13:36:22.712 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:22.706+0000 7fee73fff640 1 -- 172.21.15.43:0/196501196 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fee6c02cbc0 con 0x7fee68003ef0 2023-08-09T13:36:22.712 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:22.707+0000 7fee82691640 1 -- 172.21.15.43:0/196501196 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fee68003ef0 msgr2=0x7fee680047c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:36:22.712 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:22.707+0000 7fee82691640 1 --2- 172.21.15.43:0/196501196 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fee68003ef0 0x7fee680047c0 secure :-1 s=READY pgs=593 cs=0 l=1 rev1=1 crypto rx=0x7fee6c009ac0 tx=0x7fee680057f0 comp rx=0 tx=0).stop 2023-08-09T13:36:22.712 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:22.707+0000 7fee81e90640 1 -- 172.21.15.43:0/196501196 reap_dead start 2023-08-09T13:36:22.712 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:22.707+0000 7fee82691640 1 -- 172.21.15.43:0/196501196 shutdown_connections 2023-08-09T13:36:22.713 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:22.707+0000 7fee82691640 1 -- 172.21.15.43:0/196501196 >> 172.21.15.43:0/196501196 conn(0x7fee7c06a2b0 msgr2=0x7fee7c06a690 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-09T13:36:22.713 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:22.707+0000 7fee82691640 1 -- 172.21.15.43:0/196501196 shutdown_connections 2023-08-09T13:36:22.713 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:22.707+0000 7fee82691640 1 -- 172.21.15.43:0/196501196 wait complete. 2023-08-09T13:36:22.713 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:22.709+0000 7fee82691640 1 Processor -- start 2023-08-09T13:36:22.713 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:22.709+0000 7fee82691640 1 -- start start 2023-08-09T13:36:22.713 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:22.710+0000 7fee82691640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fee68003ef0 0x7fee7c0c8570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:36:22.713 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:22.710+0000 7fee82691640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fee7c153bc0 0x7fee7c1587a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:36:22.713 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:22.710+0000 7fee82691640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fee7c158ce0 0x7fee7c1f2610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:36:22.714 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:22.710+0000 7fee82691640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fee7c159190 con 0x7fee68003ef0 2023-08-09T13:36:22.714 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:22.710+0000 7fee82691640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fee7c1f2120 con 0x7fee7c158ce0 2023-08-09T13:36:22.714 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:22.710+0000 7fee82691640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fee7c1540c0 con 0x7fee7c153bc0 2023-08-09T13:36:22.714 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:22.710+0000 7fee8168f640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fee68003ef0 0x7fee7c0c8570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:36:22.714 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:22.710+0000 7fee8168f640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fee68003ef0 0x7fee7c0c8570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:58236/0 (socket says 172.21.15.43:58236) 2023-08-09T13:36:22.714 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:22.710+0000 7fee8168f640 1 -- 172.21.15.43:0/3634444688 learned_addr learned my addr 172.21.15.43:0/3634444688 (peer_addr_for_me v2:172.21.15.43:0/0) 2023-08-09T13:36:22.715 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:22.711+0000 7fee8168f640 1 -- 172.21.15.43:0/3634444688 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fee7c153bc0 msgr2=0x7fee7c1587a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:36:22.715 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:22.711+0000 7fee8168f640 1 --2- 172.21.15.43:0/3634444688 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fee7c153bc0 0x7fee7c1587a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:36:22.716 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:22.711+0000 7fee8168f640 1 -- 172.21.15.43:0/3634444688 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fee7c158ce0 msgr2=0x7fee7c1f2610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:36:22.716 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:22.711+0000 7fee8168f640 1 --2- 172.21.15.43:0/3634444688 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fee7c158ce0 0x7fee7c1f2610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:36:22.716 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:22.711+0000 7fee8168f640 1 -- 172.21.15.43:0/3634444688 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fee7c1543a0 con 0x7fee68003ef0 2023-08-09T13:36:22.716 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:22.711+0000 7fee8168f640 1 --2- 172.21.15.43:0/3634444688 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fee68003ef0 0x7fee7c0c8570 secure :-1 s=READY pgs=594 cs=0 l=1 rev1=1 crypto rx=0x7fee78010860 tx=0x7fee78010d20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-09T13:36:22.716 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:22.711+0000 7fee71ffb640 1 -- 172.21.15.43:0/3634444688 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fee78009610 con 0x7fee68003ef0 2023-08-09T13:36:22.716 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:22.711+0000 7fee71ffb640 1 -- 172.21.15.43:0/3634444688 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fee7800eb60 con 0x7fee68003ef0 2023-08-09T13:36:22.716 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:22.711+0000 7fee71ffb640 1 -- 172.21.15.43:0/3634444688 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fee7801c400 con 0x7fee68003ef0 2023-08-09T13:36:22.717 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:22.711+0000 7fee82691640 1 -- 172.21.15.43:0/3634444688 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fee7c154680 con 0x7fee68003ef0 2023-08-09T13:36:22.717 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:22.712+0000 7fee82691640 1 -- 172.21.15.43:0/3634444688 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fee7c0c8ab0 con 0x7fee68003ef0 2023-08-09T13:36:22.717 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:22.713+0000 7fee82691640 1 -- 172.21.15.43:0/3634444688 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fee7c15fb90 con 0x7fee68003ef0 2023-08-09T13:36:22.717 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:22.714+0000 7fee71ffb640 1 -- 172.21.15.43:0/3634444688 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 5) v1 ==== 88341+0+0 (secure 0 0 0) 0x7fee7801c560 con 0x7fee68003ef0 2023-08-09T13:36:22.717 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:22.715+0000 7fee71ffb640 1 --2- 172.21.15.43:0/3634444688 >> [v2:172.21.15.203:6832/830097474,v1:172.21.15.203:6833/830097474] conn(0x7fee6006a050 0x7fee6006c510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:36:22.718 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:22.715+0000 7fee71ffb640 1 -- 172.21.15.43:0/3634444688 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(39..39 src has 1..39) v4 ==== 6382+0+0 (secure 0 0 0) 0x7fee78088bc0 con 0x7fee68003ef0 2023-08-09T13:36:22.718 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:22.715+0000 7fee80e8e640 1 --2- 172.21.15.43:0/3634444688 >> [v2:172.21.15.203:6832/830097474,v1:172.21.15.203:6833/830097474] conn(0x7fee6006a050 0x7fee6006c510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:36:22.718 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:22.717+0000 7fee80e8e640 1 --2- 172.21.15.43:0/3634444688 >> [v2:172.21.15.203:6832/830097474,v1:172.21.15.203:6833/830097474] conn(0x7fee6006a050 0x7fee6006c510 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7fee6c036000 tx=0x7fee6c02e000 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-09T13:36:22.720 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:22.719+0000 7fee71ffb640 1 -- 172.21.15.43:0/3634444688 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178066 (secure 0 0 0) 0x7fee78054220 con 0x7fee68003ef0 2023-08-09T13:36:22.943 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:22.941+0000 7fee82691640 1 -- 172.21.15.43:0/3634444688 --> [v2:172.21.15.203:6832/830097474,v1:172.21.15.203:6833/830097474] -- mgr_command(tid 0: {"prefix": "fs volume ls", "target": ["mon-mgr", ""]}) v1 -- 0x7fee7c0c8ca0 con 0x7fee6006a050 2023-08-09T13:36:22.947 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:22.945+0000 7fee71ffb640 1 -- 172.21.15.43:0/3634444688 <== mgr.4114 v2:172.21.15.203:6832/830097474 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+41 (secure 0 0 0) 0x7fee7c0c8ca0 con 0x7fee6006a050 2023-08-09T13:36:22.947 INFO:teuthology.orchestra.run.smithi043.stdout:[ 2023-08-09T13:36:22.948 INFO:teuthology.orchestra.run.smithi043.stdout: { 2023-08-09T13:36:22.948 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "cephfs" 2023-08-09T13:36:22.948 INFO:teuthology.orchestra.run.smithi043.stdout: } 2023-08-09T13:36:22.948 INFO:teuthology.orchestra.run.smithi043.stdout:] 2023-08-09T13:36:22.949 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:22.948+0000 7fee82691640 1 -- 172.21.15.43:0/3634444688 >> [v2:172.21.15.203:6832/830097474,v1:172.21.15.203:6833/830097474] conn(0x7fee6006a050 msgr2=0x7fee6006c510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:36:22.950 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:22.948+0000 7fee82691640 1 --2- 172.21.15.43:0/3634444688 >> [v2:172.21.15.203:6832/830097474,v1:172.21.15.203:6833/830097474] conn(0x7fee6006a050 0x7fee6006c510 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7fee6c036000 tx=0x7fee6c02e000 comp rx=0 tx=0).stop 2023-08-09T13:36:22.950 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:22.948+0000 7fee82691640 1 -- 172.21.15.43:0/3634444688 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fee68003ef0 msgr2=0x7fee7c0c8570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:36:22.950 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:22.948+0000 7fee82691640 1 --2- 172.21.15.43:0/3634444688 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fee68003ef0 0x7fee7c0c8570 secure :-1 s=READY pgs=594 cs=0 l=1 rev1=1 crypto rx=0x7fee78010860 tx=0x7fee78010d20 comp rx=0 tx=0).stop 2023-08-09T13:36:22.951 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:22.948+0000 7fee82691640 1 -- 172.21.15.43:0/3634444688 shutdown_connections 2023-08-09T13:36:22.951 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:22.948+0000 7fee82691640 1 --2- 172.21.15.43:0/3634444688 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fee7c158ce0 0x7fee7c1f2610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:36:22.951 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:22.948+0000 7fee82691640 1 --2- 172.21.15.43:0/3634444688 >> [v2:172.21.15.203:6832/830097474,v1:172.21.15.203:6833/830097474] conn(0x7fee6006a050 0x7fee6006c510 unknown :-1 s=CLOSED pgs=283 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:36:22.951 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:22.948+0000 7fee82691640 1 --2- 172.21.15.43:0/3634444688 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fee7c153bc0 0x7fee7c1587a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:36:22.951 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:22.948+0000 7fee82691640 1 --2- 172.21.15.43:0/3634444688 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fee68003ef0 0x7fee7c0c8570 unknown :-1 s=CLOSED pgs=594 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:36:22.951 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:22.948+0000 7fee82691640 1 -- 172.21.15.43:0/3634444688 >> 172.21.15.43:0/3634444688 conn(0x7fee7c06a2b0 msgr2=0x7fee7c0d0d50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-09T13:36:22.951 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:22.949+0000 7fee82691640 1 -- 172.21.15.43:0/3634444688 shutdown_connections 2023-08-09T13:36:22.952 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:22.949+0000 7fee82691640 1 -- 172.21.15.43:0/3634444688 wait complete. 2023-08-09T13:36:22.969 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph mgr module enable snap_schedule 2023-08-09T13:36:23.199 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:23.197+0000 7f193c5d6640 1 Processor -- start 2023-08-09T13:36:23.199 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:23.197+0000 7f193c5d6640 1 -- start start 2023-08-09T13:36:23.201 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:23.199+0000 7f193c5d6640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f1934143c20 0x7f193406f540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:36:23.201 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:23.199+0000 7f193c5d6640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f193406cfe0 0x7f19340c8b50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:36:23.201 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:23.199+0000 7f193c5d6640 1 -- --> v1:172.21.15.203:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f19340c9090 con 0x7f19340c86f0 2023-08-09T13:36:23.201 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:23.199+0000 7f193c5d6640 1 -- --> v2:172.21.15.43:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1934144000 con 0x7f1934143c20 2023-08-09T13:36:23.201 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:23.199+0000 7f193c5d6640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f193406c440 con 0x7f193406cfe0 2023-08-09T13:36:23.202 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:23.199+0000 7f193a372640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f1934143c20 0x7f193406f540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:36:23.202 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:23.200+0000 7f193a372640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f1934143c20 0x7f193406f540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:58246/0 (socket says 172.21.15.43:58246) 2023-08-09T13:36:23.202 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:23.200+0000 7f193a372640 1 -- 172.21.15.43:0/87613162 learned_addr learned my addr 172.21.15.43:0/87613162 (peer_addr_for_me v2:172.21.15.43:0/0) 2023-08-09T13:36:23.203 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:23.200+0000 7f193a372640 1 -- 172.21.15.43:0/87613162 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f193406cfe0 msgr2=0x7f19340c8b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-09T13:36:23.203 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:23.200+0000 7f193ab73640 1 --2- 172.21.15.43:0/87613162 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f193406cfe0 0x7f19340c8b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:36:23.203 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:23.200+0000 7f193a372640 1 --2- 172.21.15.43:0/87613162 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f193406cfe0 0x7f19340c8b50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:36:23.203 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:23.200+0000 7f193a372640 1 -- 172.21.15.43:0/87613162 >> v1:172.21.15.203:6789/0 conn(0x7f19340c86f0 legacy=0x7f193406c8d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-09T13:36:23.203 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:23.200+0000 7f193a372640 1 -- 172.21.15.43:0/87613162 --> v2:172.21.15.43:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f193406b830 con 0x7f1934143c20 2023-08-09T13:36:23.203 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:23.200+0000 7f193ab73640 1 --2- 172.21.15.43:0/87613162 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f193406cfe0 0x7f19340c8b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-08-09T13:36:23.204 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:23.201+0000 7f193a372640 1 --2- 172.21.15.43:0/87613162 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f1934143c20 0x7f193406f540 secure :-1 s=READY pgs=595 cs=0 l=1 rev1=1 crypto rx=0x7f1930009870 tx=0x7f193002b860 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ee3e1e7e279310ef server_cookie=0 in_seq=0 out_seq=0 2023-08-09T13:36:23.204 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:23.202+0000 7f1939370640 1 -- 172.21.15.43:0/87613162 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f19300317c0 con 0x7f1934143c20 2023-08-09T13:36:23.205 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:23.202+0000 7f1939370640 1 -- 172.21.15.43:0/87613162 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1930031920 con 0x7f1934143c20 2023-08-09T13:36:23.205 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:23.202+0000 7f1939370640 1 -- 172.21.15.43:0/87613162 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1930031cb0 con 0x7f1934143c20 2023-08-09T13:36:23.205 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:23.202+0000 7f193c5d6640 1 -- 172.21.15.43:0/87613162 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f1934143c20 msgr2=0x7f193406f540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:36:23.205 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:23.202+0000 7f193c5d6640 1 --2- 172.21.15.43:0/87613162 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f1934143c20 0x7f193406f540 secure :-1 s=READY pgs=595 cs=0 l=1 rev1=1 crypto rx=0x7f1930009870 tx=0x7f193002b860 comp rx=0 tx=0).stop 2023-08-09T13:36:23.205 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:23.203+0000 7f193c5d6640 1 -- 172.21.15.43:0/87613162 shutdown_connections 2023-08-09T13:36:23.205 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:23.203+0000 7f193c5d6640 1 --2- 172.21.15.43:0/87613162 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f193406cfe0 0x7f19340c8b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:36:23.205 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:23.203+0000 7f193c5d6640 1 --2- 172.21.15.43:0/87613162 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f1934143c20 0x7f193406f540 unknown :-1 s=CLOSED pgs=595 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:36:23.206 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:23.203+0000 7f193c5d6640 1 -- 172.21.15.43:0/87613162 >> 172.21.15.43:0/87613162 conn(0x7f19340ce0a0 msgr2=0x7f19340ce480 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-09T13:36:23.206 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:23.203+0000 7f193c5d6640 1 -- 172.21.15.43:0/87613162 shutdown_connections 2023-08-09T13:36:23.206 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:23.203+0000 7f193c5d6640 1 -- 172.21.15.43:0/87613162 wait complete. 2023-08-09T13:36:23.206 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:23.205+0000 7f193c5d6640 1 Processor -- start 2023-08-09T13:36:23.207 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:23.205+0000 7f193c5d6640 1 -- start start 2023-08-09T13:36:23.208 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:23.207+0000 7f193c5d6640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f193406cfe0 0x7f19340c70b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:36:23.208 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:23.207+0000 7f193c5d6640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f19340c86f0 0x7f19340b3570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:36:23.209 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:23.207+0000 7f193c5d6640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f1934143c20 0x7f19340b3ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:36:23.209 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:23.207+0000 7f193c5d6640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f19340b3110 con 0x7f1934143c20 2023-08-09T13:36:23.209 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:23.207+0000 7f193c5d6640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f19340b2070 con 0x7f193406cfe0 2023-08-09T13:36:23.209 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:23.207+0000 7f193c5d6640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f19340b2350 con 0x7f19340c86f0 2023-08-09T13:36:23.209 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:23.207+0000 7f193ab73640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f1934143c20 0x7f19340b3ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:36:23.209 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:23.207+0000 7f193ab73640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f1934143c20 0x7f19340b3ab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:58252/0 (socket says 172.21.15.43:58252) 2023-08-09T13:36:23.209 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:23.207+0000 7f193ab73640 1 -- 172.21.15.43:0/4274190387 learned_addr learned my addr 172.21.15.43:0/4274190387 (peer_addr_for_me v2:172.21.15.43:0/0) 2023-08-09T13:36:23.210 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:23.207+0000 7f193a372640 1 --2- 172.21.15.43:0/4274190387 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f193406cfe0 0x7f19340c70b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:36:23.211 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:23.207+0000 7f1939b71640 1 --2- 172.21.15.43:0/4274190387 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f19340c86f0 0x7f19340b3570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:36:23.211 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:23.208+0000 7f193ab73640 1 -- 172.21.15.43:0/4274190387 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f19340c86f0 msgr2=0x7f19340b3570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:36:23.211 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:23.208+0000 7f193ab73640 1 --2- 172.21.15.43:0/4274190387 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f19340c86f0 0x7f19340b3570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:36:23.211 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:23.208+0000 7f193ab73640 1 -- 172.21.15.43:0/4274190387 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f193406cfe0 msgr2=0x7f19340c70b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:36:23.211 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:23.208+0000 7f193ab73640 1 --2- 172.21.15.43:0/4274190387 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f193406cfe0 0x7f19340c70b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:36:23.211 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:23.208+0000 7f193ab73640 1 -- 172.21.15.43:0/4274190387 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f19300094d0 con 0x7f1934143c20 2023-08-09T13:36:23.211 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:23.208+0000 7f1939b71640 1 --2- 172.21.15.43:0/4274190387 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f19340c86f0 0x7f19340b3570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-08-09T13:36:23.211 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:23.208+0000 7f193ab73640 1 --2- 172.21.15.43:0/4274190387 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f1934143c20 0x7f19340b3ab0 secure :-1 s=READY pgs=596 cs=0 l=1 rev1=1 crypto rx=0x7f192c00cca0 tx=0x7f192c00a650 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-09T13:36:23.212 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:23.208+0000 7f192affd640 1 -- 172.21.15.43:0/4274190387 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f192c00f680 con 0x7f1934143c20 2023-08-09T13:36:23.212 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:23.208+0000 7f192affd640 1 -- 172.21.15.43:0/4274190387 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f192c00f800 con 0x7f1934143c20 2023-08-09T13:36:23.212 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:23.208+0000 7f192affd640 1 -- 172.21.15.43:0/4274190387 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f192c00fbb0 con 0x7f1934143c20 2023-08-09T13:36:23.212 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:23.208+0000 7f193c5d6640 1 -- 172.21.15.43:0/4274190387 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f19340b2630 con 0x7f1934143c20 2023-08-09T13:36:23.212 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:23.209+0000 7f193c5d6640 1 -- 172.21.15.43:0/4274190387 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f19340b2bf0 con 0x7f1934143c20 2023-08-09T13:36:23.213 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:23.210+0000 7f193c5d6640 1 -- 172.21.15.43:0/4274190387 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f19340b2e10 con 0x7f1934143c20 2023-08-09T13:36:23.213 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:23.211+0000 7f192affd640 1 -- 172.21.15.43:0/4274190387 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 5) v1 ==== 88341+0+0 (secure 0 0 0) 0x7f192c00b040 con 0x7f1934143c20 2023-08-09T13:36:23.213 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:23.212+0000 7f192affd640 1 --2- 172.21.15.43:0/4274190387 >> [v2:172.21.15.203:6832/830097474,v1:172.21.15.203:6833/830097474] conn(0x7f191806a050 0x7f191806c510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:36:23.213 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:23.212+0000 7f192affd640 1 -- 172.21.15.43:0/4274190387 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(39..39 src has 1..39) v4 ==== 6382+0+0 (secure 0 0 0) 0x7f192c0890e0 con 0x7f1934143c20 2023-08-09T13:36:23.214 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:23.213+0000 7f193a372640 1 --2- 172.21.15.43:0/4274190387 >> [v2:172.21.15.203:6832/830097474,v1:172.21.15.203:6833/830097474] conn(0x7f191806a050 0x7f191806c510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:36:23.215 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:23.214+0000 7f193a372640 1 --2- 172.21.15.43:0/4274190387 >> [v2:172.21.15.203:6832/830097474,v1:172.21.15.203:6833/830097474] conn(0x7f191806a050 0x7f191806c510 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7f1930036000 tx=0x7f193003d040 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-09T13:36:23.218 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:23.217+0000 7f192affd640 1 -- 172.21.15.43:0/4274190387 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178066 (secure 0 0 0) 0x7f192c054680 con 0x7f1934143c20 2023-08-09T13:36:23.466 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:23.465+0000 7f193c5d6640 1 -- 172.21.15.43:0/4274190387 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "mgr module enable", "module": "snap_schedule"} v 0) v1 -- 0x7f19340739f0 con 0x7f1934143c20 2023-08-09T13:36:23.897 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:23.896+0000 7f192affd640 1 -- 172.21.15.43:0/4274190387 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mgrmap(e 6) v1 ==== 88358+0+0 (secure 0 0 0) 0x7f192c00fd10 con 0x7f1934143c20 2023-08-09T13:36:23.902 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:23.900+0000 7f192affd640 1 -- 172.21.15.43:0/4274190387 <== mon.0 v2:172.21.15.43:3300/0 8 ==== mon_command_ack([{"prefix": "mgr module enable", "module": "snap_schedule"}]=0 v6) v1 ==== 92+0+0 (secure 0 0 0) 0x7f192c054680 con 0x7f1934143c20 2023-08-09T13:36:23.905 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:23.904+0000 7f193c5d6640 1 -- 172.21.15.43:0/4274190387 >> [v2:172.21.15.203:6832/830097474,v1:172.21.15.203:6833/830097474] conn(0x7f191806a050 msgr2=0x7f191806c510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:36:23.906 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:23.904+0000 7f193c5d6640 1 --2- 172.21.15.43:0/4274190387 >> [v2:172.21.15.203:6832/830097474,v1:172.21.15.203:6833/830097474] conn(0x7f191806a050 0x7f191806c510 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7f1930036000 tx=0x7f193003d040 comp rx=0 tx=0).stop 2023-08-09T13:36:23.906 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:23.904+0000 7f193c5d6640 1 -- 172.21.15.43:0/4274190387 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f1934143c20 msgr2=0x7f19340b3ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:36:23.906 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:23.904+0000 7f193c5d6640 1 --2- 172.21.15.43:0/4274190387 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f1934143c20 0x7f19340b3ab0 secure :-1 s=READY pgs=596 cs=0 l=1 rev1=1 crypto rx=0x7f192c00cca0 tx=0x7f192c00a650 comp rx=0 tx=0).stop 2023-08-09T13:36:23.906 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:23.904+0000 7f193c5d6640 1 -- 172.21.15.43:0/4274190387 shutdown_connections 2023-08-09T13:36:23.906 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:23.904+0000 7f193c5d6640 1 --2- 172.21.15.43:0/4274190387 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f1934143c20 0x7f19340b3ab0 unknown :-1 s=CLOSED pgs=596 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:36:23.906 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:23.904+0000 7f193c5d6640 1 --2- 172.21.15.43:0/4274190387 >> [v2:172.21.15.203:6832/830097474,v1:172.21.15.203:6833/830097474] conn(0x7f191806a050 0x7f191806c510 unknown :-1 s=CLOSED pgs=284 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:36:23.907 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:23.904+0000 7f193c5d6640 1 --2- 172.21.15.43:0/4274190387 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f19340c86f0 0x7f19340b3570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:36:23.907 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:23.904+0000 7f193c5d6640 1 --2- 172.21.15.43:0/4274190387 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f193406cfe0 0x7f19340c70b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:36:23.907 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:23.904+0000 7f193c5d6640 1 -- 172.21.15.43:0/4274190387 >> 172.21.15.43:0/4274190387 conn(0x7f19340ce0a0 msgr2=0x7f19340cab20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-09T13:36:23.907 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:23.904+0000 7f193c5d6640 1 -- 172.21.15.43:0/4274190387 shutdown_connections 2023-08-09T13:36:23.907 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:23.904+0000 7f193c5d6640 1 -- 172.21.15.43:0/4274190387 wait complete. 2023-08-09T13:36:23.933 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mgr mgr/snap_schedule/allow_m_granularity True 2023-08-09T13:36:24.121 INFO:tasks.ceph.mgr.y.smithi043.stderr:2023-08-09T13:36:24.120+0000 7f0522250040 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2023-08-09T13:36:24.141 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:24.139+0000 7fb0b8776640 1 Processor -- start 2023-08-09T13:36:24.141 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:24.139+0000 7fb0b8776640 1 -- start start 2023-08-09T13:36:24.143 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:24.141+0000 7fb0b8776640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7fb0b00b34a0 0x7fb0b00b22d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:36:24.143 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:24.141+0000 7fb0b8776640 1 -- --> v1:172.21.15.43:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb0b00ce340 con 0x7fb0b01587a0 2023-08-09T13:36:24.143 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:24.141+0000 7fb0b8776640 1 -- --> v1:172.21.15.203:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb0b015ca10 con 0x7fb0b015c470 2023-08-09T13:36:24.143 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:24.141+0000 7fb0b8776640 1 -- --> v2:172.21.15.43:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb0b00b2810 con 0x7fb0b00b34a0 2023-08-09T13:36:24.144 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:24.142+0000 7fb0b6d13640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7fb0b00b34a0 0x7fb0b00b22d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:36:24.144 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:24.142+0000 7fb0b6d13640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7fb0b00b34a0 0x7fb0b00b22d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:58268/0 (socket says 172.21.15.43:58268) 2023-08-09T13:36:24.144 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:24.142+0000 7fb0b6d13640 1 -- 172.21.15.43:0/4133641311 learned_addr learned my addr 172.21.15.43:0/4133641311 (peer_addr_for_me v2:172.21.15.43:0/0) 2023-08-09T13:36:24.144 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:24.142+0000 7fb0b6d13640 1 -- 172.21.15.43:0/4133641311 >> v1:172.21.15.203:6789/0 conn(0x7fb0b015c470 legacy=0x7fb0b00b2d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-09T13:36:24.144 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:24.142+0000 7fb0b6d13640 1 -- 172.21.15.43:0/4133641311 >> v1:172.21.15.43:6789/0 conn(0x7fb0b01587a0 legacy=0x7fb0b00cb980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:36:24.145 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:24.142+0000 7fb0b6d13640 1 -- 172.21.15.43:0/4133641311 --> v2:172.21.15.43:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb0b015ccf0 con 0x7fb0b00b34a0 2023-08-09T13:36:24.145 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:24.143+0000 7fb0b6d13640 1 --2- 172.21.15.43:0/4133641311 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb0b00b34a0 0x7fb0b00b22d0 secure :-1 s=READY pgs=602 cs=0 l=1 rev1=1 crypto rx=0x7fb0a800bb00 tx=0x7fb0a802df30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ccd4a255848c0fcf server_cookie=0 in_seq=0 out_seq=0 2023-08-09T13:36:24.145 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:24.144+0000 7fb0b5510640 1 -- 172.21.15.43:0/4133641311 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb0a802ece0 con 0x7fb0b00b34a0 2023-08-09T13:36:24.146 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:24.144+0000 7fb0b5510640 1 -- 172.21.15.43:0/4133641311 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb0a8032070 con 0x7fb0b00b34a0 2023-08-09T13:36:24.146 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:24.144+0000 7fb0b5510640 1 -- 172.21.15.43:0/4133641311 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb0a8038620 con 0x7fb0b00b34a0 2023-08-09T13:36:24.146 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:24.144+0000 7fb0b8776640 1 -- 172.21.15.43:0/4133641311 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb0b00b34a0 msgr2=0x7fb0b00b22d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:36:24.147 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:24.144+0000 7fb0b8776640 1 --2- 172.21.15.43:0/4133641311 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb0b00b34a0 0x7fb0b00b22d0 secure :-1 s=READY pgs=602 cs=0 l=1 rev1=1 crypto rx=0x7fb0a800bb00 tx=0x7fb0a802df30 comp rx=0 tx=0).stop 2023-08-09T13:36:24.147 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:24.145+0000 7fb0b8776640 1 -- 172.21.15.43:0/4133641311 shutdown_connections 2023-08-09T13:36:24.147 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:24.145+0000 7fb0b8776640 1 --2- 172.21.15.43:0/4133641311 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb0b00b34a0 0x7fb0b00b22d0 unknown :-1 s=CLOSED pgs=602 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:36:24.147 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:24.145+0000 7fb0b8776640 1 -- 172.21.15.43:0/4133641311 >> 172.21.15.43:0/4133641311 conn(0x7fb0b00697c0 msgr2=0x7fb0b0069ba0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-09T13:36:24.147 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:24.145+0000 7fb0b8776640 1 -- 172.21.15.43:0/4133641311 shutdown_connections 2023-08-09T13:36:24.147 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:24.145+0000 7fb0b8776640 1 -- 172.21.15.43:0/4133641311 wait complete. 2023-08-09T13:36:24.151 INFO:tasks.ceph.mgr.x.smithi203.stderr:2023-08-09T13:36:24.147+0000 7f58fd03b040 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2023-08-09T13:36:24.154 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:24.146+0000 7fb0b8776640 1 Processor -- start 2023-08-09T13:36:24.154 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:24.147+0000 7fb0b8776640 1 -- start start 2023-08-09T13:36:24.155 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:24.147+0000 7fb0b8776640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fb0b00b34a0 0x7fb0b01579a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:36:24.155 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:24.148+0000 7fb0b8776640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb0b01587a0 0x7fb0b01e3430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:36:24.156 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:24.148+0000 7fb0b8776640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fb0b015c470 0x7fb0b01e3970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:36:24.157 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:24.148+0000 7fb0b8776640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb0b0159d90 con 0x7fb0b01587a0 2023-08-09T13:36:24.158 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:24.148+0000 7fb0b8776640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb0b015a070 con 0x7fb0b015c470 2023-08-09T13:36:24.158 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:24.148+0000 7fb0b8776640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb0b01559a0 con 0x7fb0b00b34a0 2023-08-09T13:36:24.159 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:24.148+0000 7fb0b5d11640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb0b01587a0 0x7fb0b01e3430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:36:24.159 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:24.148+0000 7fb0b5d11640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb0b01587a0 0x7fb0b01e3430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:58270/0 (socket says 172.21.15.43:58270) 2023-08-09T13:36:24.160 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:24.148+0000 7fb0b5d11640 1 -- 172.21.15.43:0/3370670323 learned_addr learned my addr 172.21.15.43:0/3370670323 (peer_addr_for_me v2:172.21.15.43:0/0) 2023-08-09T13:36:24.161 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:24.148+0000 7fb0b6512640 1 --2- 172.21.15.43:0/3370670323 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fb0b00b34a0 0x7fb0b01579a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:36:24.162 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:24.148+0000 7fb0b5d11640 1 -- 172.21.15.43:0/3370670323 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fb0b00b34a0 msgr2=0x7fb0b01579a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:36:24.163 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:24.148+0000 7fb0b5d11640 1 --2- 172.21.15.43:0/3370670323 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fb0b00b34a0 0x7fb0b01579a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:36:24.163 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:24.149+0000 7fb0b6d13640 1 --2- 172.21.15.43:0/3370670323 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fb0b015c470 0x7fb0b01e3970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:36:24.164 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:24.149+0000 7fb0b5d11640 1 -- 172.21.15.43:0/3370670323 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fb0b015c470 msgr2=0x7fb0b01e3970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:36:24.164 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:24.149+0000 7fb0b5d11640 1 --2- 172.21.15.43:0/3370670323 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fb0b015c470 0x7fb0b01e3970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:36:24.164 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:24.149+0000 7fb0b5d11640 1 -- 172.21.15.43:0/3370670323 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb0a800b760 con 0x7fb0b01587a0 2023-08-09T13:36:24.165 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:24.149+0000 7fb0b6512640 1 --2- 172.21.15.43:0/3370670323 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fb0b00b34a0 0x7fb0b01579a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-08-09T13:36:24.165 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:24.149+0000 7fb0b6d13640 1 --2- 172.21.15.43:0/3370670323 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fb0b015c470 0x7fb0b01e3970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-08-09T13:36:24.165 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:24.149+0000 7fb0b5d11640 1 --2- 172.21.15.43:0/3370670323 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb0b01587a0 0x7fb0b01e3430 secure :-1 s=READY pgs=603 cs=0 l=1 rev1=1 crypto rx=0x7fb0a000b420 tx=0x7fb0a000b8e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-09T13:36:24.165 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:24.149+0000 7fb0a6ffd640 1 -- 172.21.15.43:0/3370670323 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb0a0011830 con 0x7fb0b01587a0 2023-08-09T13:36:24.166 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:24.149+0000 7fb0a6ffd640 1 -- 172.21.15.43:0/3370670323 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb0a0011990 con 0x7fb0b01587a0 2023-08-09T13:36:24.166 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:24.149+0000 7fb0a6ffd640 1 -- 172.21.15.43:0/3370670323 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb0a0011d40 con 0x7fb0b01587a0 2023-08-09T13:36:24.166 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:24.149+0000 7fb0b8776640 1 -- 172.21.15.43:0/3370670323 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb0b0155c80 con 0x7fb0b01587a0 2023-08-09T13:36:24.166 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:24.150+0000 7fb0b8776640 1 -- 172.21.15.43:0/3370670323 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb0b0157ee0 con 0x7fb0b01587a0 2023-08-09T13:36:24.167 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:24.151+0000 7fb0b8776640 1 -- 172.21.15.43:0/3370670323 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb0b01561a0 con 0x7fb0b01587a0 2023-08-09T13:36:24.167 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:24.153+0000 7fb0a6ffd640 1 -- 172.21.15.43:0/3370670323 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 6) v1 ==== 88358+0+0 (secure 0 0 0) 0x7fb0a001b460 con 0x7fb0b01587a0 2023-08-09T13:36:24.168 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:24.154+0000 7fb0a6ffd640 1 --2- 172.21.15.43:0/3370670323 >> [v2:172.21.15.203:6832/830097474,v1:172.21.15.203:6833/830097474] conn(0x7fb09406a0d0 0x7fb09406c590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:36:24.168 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:24.154+0000 7fb0a6ffd640 1 -- 172.21.15.43:0/3370670323 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(39..39 src has 1..39) v4 ==== 6382+0+0 (secure 0 0 0) 0x7fb0a00882e0 con 0x7fb0b01587a0 2023-08-09T13:36:24.168 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:24.155+0000 7fb0b6512640 1 -- 172.21.15.43:0/3370670323 >> [v2:172.21.15.203:6832/830097474,v1:172.21.15.203:6833/830097474] conn(0x7fb09406a0d0 msgr2=0x7fb09406c590 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.203:6832/830097474 2023-08-09T13:36:24.169 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:24.155+0000 7fb0b6512640 1 --2- 172.21.15.43:0/3370670323 >> [v2:172.21.15.203:6832/830097474,v1:172.21.15.203:6833/830097474] conn(0x7fb09406a0d0 0x7fb09406c590 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-08-09T13:36:24.169 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:24.159+0000 7fb0a6ffd640 1 -- 172.21.15.43:0/3370670323 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178066 (secure 0 0 0) 0x7fb0a00537f0 con 0x7fb0b01587a0 2023-08-09T13:36:24.357 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:24.356+0000 7fb0b6512640 1 -- 172.21.15.43:0/3370670323 >> [v2:172.21.15.203:6832/830097474,v1:172.21.15.203:6833/830097474] conn(0x7fb09406a0d0 msgr2=0x7fb09406c590 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.203:6832/830097474 2023-08-09T13:36:24.358 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:24.356+0000 7fb0b6512640 1 --2- 172.21.15.43:0/3370670323 >> [v2:172.21.15.203:6832/830097474,v1:172.21.15.203:6833/830097474] conn(0x7fb09406a0d0 0x7fb09406c590 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2023-08-09T13:36:24.358 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:24.357+0000 7fb0b8776640 1 -- 172.21.15.43:0/3370670323 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command([{prefix=config set, name=mgr/snap_schedule/allow_m_granularity}] v 0) v1 -- 0x7fb0b006a7d0 con 0x7fb0b01587a0 2023-08-09T13:36:24.365 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:24.363+0000 7fb0a6ffd640 1 -- 172.21.15.43:0/3370670323 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mgr/snap_schedule/allow_m_granularity}]=0 v11)=0 v11) v1 ==== 138+0+0 (secure 0 0 0) 0x7fb0a0056e10 con 0x7fb0b01587a0 2023-08-09T13:36:24.368 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:24.366+0000 7fb0b8776640 1 -- 172.21.15.43:0/3370670323 >> [v2:172.21.15.203:6832/830097474,v1:172.21.15.203:6833/830097474] conn(0x7fb09406a0d0 msgr2=0x7fb09406c590 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-08-09T13:36:24.368 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:24.366+0000 7fb0b8776640 1 --2- 172.21.15.43:0/3370670323 >> [v2:172.21.15.203:6832/830097474,v1:172.21.15.203:6833/830097474] conn(0x7fb09406a0d0 0x7fb09406c590 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:36:24.368 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:24.366+0000 7fb0b8776640 1 -- 172.21.15.43:0/3370670323 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb0b01587a0 msgr2=0x7fb0b01e3430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:36:24.368 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:24.366+0000 7fb0b8776640 1 --2- 172.21.15.43:0/3370670323 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb0b01587a0 0x7fb0b01e3430 secure :-1 s=READY pgs=603 cs=0 l=1 rev1=1 crypto rx=0x7fb0a000b420 tx=0x7fb0a000b8e0 comp rx=0 tx=0).stop 2023-08-09T13:36:24.368 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:24.366+0000 7fb0b8776640 1 -- 172.21.15.43:0/3370670323 shutdown_connections 2023-08-09T13:36:24.369 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:24.366+0000 7fb0b8776640 1 --2- 172.21.15.43:0/3370670323 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fb0b015c470 0x7fb0b01e3970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:36:24.369 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:24.366+0000 7fb0b8776640 1 --2- 172.21.15.43:0/3370670323 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb0b01587a0 0x7fb0b01e3430 unknown :-1 s=CLOSED pgs=603 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:36:24.369 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:24.366+0000 7fb0b8776640 1 --2- 172.21.15.43:0/3370670323 >> [v2:172.21.15.203:6832/830097474,v1:172.21.15.203:6833/830097474] conn(0x7fb09406a0d0 0x7fb09406c590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:36:24.369 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:24.366+0000 7fb0b8776640 1 --2- 172.21.15.43:0/3370670323 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fb0b00b34a0 0x7fb0b01579a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:36:24.369 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:24.366+0000 7fb0b8776640 1 -- 172.21.15.43:0/3370670323 >> 172.21.15.43:0/3370670323 conn(0x7fb0b00697c0 msgr2=0x7fb0b00ccbd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-09T13:36:24.369 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:24.366+0000 7fb0b8776640 1 -- 172.21.15.43:0/3370670323 shutdown_connections 2023-08-09T13:36:24.370 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:24.366+0000 7fb0b8776640 1 -- 172.21.15.43:0/3370670323 wait complete. 2023-08-09T13:36:24.387 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mgr mgr/snap_schedule/dump_on_update True 2023-08-09T13:36:24.603 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:24.601+0000 7f8ac759e640 1 Processor -- start 2023-08-09T13:36:24.603 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:24.602+0000 7f8ac759e640 1 -- start start 2023-08-09T13:36:24.605 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:24.603+0000 7f8ac759e640 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7f8ac815a1b0 0x7f8ac80b1ad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:36:24.605 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:24.603+0000 7f8ac759e640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f8ac815e9c0 0x7f8ac80b2010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:36:24.605 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:24.603+0000 7f8ac759e640 1 -- --> v1:172.21.15.43:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8ac80cc6c0 con 0x7f8ac80cd630 2023-08-09T13:36:24.605 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:24.603+0000 7f8ac759e640 1 -- --> v2:172.21.15.203:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8ac815dee0 con 0x7f8ac815a1b0 2023-08-09T13:36:24.606 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:24.603+0000 7f8ac759e640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8ac815e1c0 con 0x7f8ac815e9c0 2023-08-09T13:36:24.606 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:24.604+0000 7f8ac5d9b640 1 --1- >> v1:172.21.15.43:6789/0 conn(0x7f8ac80cd630 0x7f8ac80cdc20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.43:6789/0 says I am v1:172.21.15.43:35226/0 (socket says 172.21.15.43:35226) 2023-08-09T13:36:24.606 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:24.604+0000 7f8ac5d9b640 1 -- 172.21.15.43:0/2945534475 learned_addr learned my addr 172.21.15.43:0/2945534475 (peer_addr_for_me v1:172.21.15.43:0/0) 2023-08-09T13:36:24.608 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:24.604+0000 7f8ac659c640 1 --2- 172.21.15.43:0/2945534475 >> v2:172.21.15.203:3300/0 conn(0x7f8ac815a1b0 0x7f8ac80b1ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:36:24.608 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:24.604+0000 7f8ac6d9d640 1 --2- 172.21.15.43:0/2945534475 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f8ac815e9c0 0x7f8ac80b2010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:36:24.609 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:24.604+0000 7f8ac559a640 1 -- 172.21.15.43:0/2945534475 <== mon.0 v1:172.21.15.43:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 691873063 0 0) 0x7f8ac80cc6c0 con 0x7f8ac80cd630 2023-08-09T13:36:24.609 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:24.604+0000 7f8ac559a640 1 -- 172.21.15.43:0/2945534475 --> v1:172.21.15.43:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f8ab0003520 con 0x7f8ac80cd630 2023-08-09T13:36:24.609 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:24.604+0000 7f8ac559a640 1 -- 172.21.15.43:0/2945534475 <== mon.0 v1:172.21.15.43:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 480940343 0 0) 0x7f8ab0003520 con 0x7f8ac80cd630 2023-08-09T13:36:24.609 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:24.604+0000 7f8ac559a640 1 -- 172.21.15.43:0/2945534475 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f8ac815e9c0 msgr2=0x7f8ac80b2010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-09T13:36:24.609 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:24.605+0000 7f8ac559a640 1 --2- 172.21.15.43:0/2945534475 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f8ac815e9c0 0x7f8ac80b2010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:36:24.609 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:24.605+0000 7f8ac559a640 1 -- 172.21.15.43:0/2945534475 >> v2:172.21.15.203:3300/0 conn(0x7f8ac815a1b0 msgr2=0x7f8ac80b1ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-09T13:36:24.609 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:24.605+0000 7f8ac559a640 1 --2- 172.21.15.43:0/2945534475 >> v2:172.21.15.203:3300/0 conn(0x7f8ac815a1b0 0x7f8ac80b1ad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:36:24.610 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:24.605+0000 7f8ac559a640 1 -- 172.21.15.43:0/2945534475 --> v1:172.21.15.43:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8ac815e4a0 con 0x7f8ac80cd630 2023-08-09T13:36:24.610 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:24.605+0000 7f8ac559a640 1 -- 172.21.15.43:0/2945534475 <== mon.0 v1:172.21.15.43:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1285150740 0 0) 0x7f8abc001670 con 0x7f8ac80cd630 2023-08-09T13:36:24.610 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:24.605+0000 7f8ac559a640 1 -- 172.21.15.43:0/2945534475 >> v1:172.21.15.43:6789/0 conn(0x7f8ac80cd630 legacy=0x7f8ac80cdc20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:36:24.610 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:24.605+0000 7f8ac559a640 1 --2- 172.21.15.43:0/2945534475 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f8ab0003970 0x7f8ab0003d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:36:24.610 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:24.605+0000 7f8ac559a640 1 --2- 172.21.15.43:0/2945534475 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f8ac815e9c0 0x7f8ab00047c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:36:24.610 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:24.605+0000 7f8ac559a640 1 --2- 172.21.15.43:0/2945534475 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f8ab0004d00 0x7f8ab0005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:36:24.610 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:24.605+0000 7f8ac559a640 1 -- 172.21.15.43:0/2945534475 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8ac815dee0 con 0x7f8ab0004d00 2023-08-09T13:36:24.610 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:24.605+0000 7f8ac559a640 1 -- 172.21.15.43:0/2945534475 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8ac815e1c0 con 0x7f8ab0003970 2023-08-09T13:36:24.611 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:24.605+0000 7f8ac559a640 1 -- 172.21.15.43:0/2945534475 --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8ab000b980 con 0x7f8ac815e9c0 2023-08-09T13:36:24.611 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:24.605+0000 7f8ac6d9d640 1 --2- 172.21.15.43:0/2945534475 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f8ab0004d00 0x7f8ab0005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:36:24.611 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:24.605+0000 7f8ac5d9b640 1 --2- 172.21.15.43:0/2945534475 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f8ac815e9c0 0x7f8ab00047c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:36:24.611 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:24.605+0000 7f8ac659c640 1 --2- 172.21.15.43:0/2945534475 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f8ab0003970 0x7f8ab0003d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:36:24.611 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:24.605+0000 7f8ac6d9d640 1 -- 172.21.15.43:0/2945534475 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f8ac815e9c0 msgr2=0x7f8ab00047c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-09T13:36:24.611 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:24.605+0000 7f8ac6d9d640 1 --2- 172.21.15.43:0/2945534475 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f8ac815e9c0 0x7f8ab00047c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:36:24.611 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:24.605+0000 7f8ac6d9d640 1 -- 172.21.15.43:0/2945534475 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f8ab0003970 msgr2=0x7f8ab0003d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-09T13:36:24.611 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:24.605+0000 7f8ac6d9d640 1 --2- 172.21.15.43:0/2945534475 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f8ab0003970 0x7f8ab0003d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:36:24.612 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:24.605+0000 7f8ac6d9d640 1 -- 172.21.15.43:0/2945534475 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8ab0003520 con 0x7f8ab0004d00 2023-08-09T13:36:24.612 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:24.605+0000 7f8ac659c640 1 --2- 172.21.15.43:0/2945534475 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f8ab0003970 0x7f8ab0003d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-08-09T13:36:24.612 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:24.607+0000 7f8ac6d9d640 1 --2- 172.21.15.43:0/2945534475 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f8ab0004d00 0x7f8ab0005180 secure :-1 s=READY pgs=605 cs=0 l=1 rev1=1 crypto rx=0x7f8ac0009fc0 tx=0x7f8ab00057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=24c998b3cb6aa76b server_cookie=0 in_seq=0 out_seq=0 2023-08-09T13:36:24.612 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:24.607+0000 7f8ac559a640 1 -- 172.21.15.43:0/2945534475 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8ac002cbd0 con 0x7f8ab0004d00 2023-08-09T13:36:24.612 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:24.607+0000 7f8ac559a640 1 -- 172.21.15.43:0/2945534475 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8ac000f040 con 0x7f8ab0004d00 2023-08-09T13:36:24.612 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:24.607+0000 7f8ac559a640 1 -- 172.21.15.43:0/2945534475 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8ac0034440 con 0x7f8ab0004d00 2023-08-09T13:36:24.614 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:24.607+0000 7f8ac759e640 1 -- 172.21.15.43:0/2945534475 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f8ab0004d00 msgr2=0x7f8ab0005180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:36:24.614 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:24.607+0000 7f8ac759e640 1 --2- 172.21.15.43:0/2945534475 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f8ab0004d00 0x7f8ab0005180 secure :-1 s=READY pgs=605 cs=0 l=1 rev1=1 crypto rx=0x7f8ac0009fc0 tx=0x7f8ab00057f0 comp rx=0 tx=0).stop 2023-08-09T13:36:24.614 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:24.607+0000 7f8ac6d9d640 1 -- 172.21.15.43:0/2945534475 reap_dead start 2023-08-09T13:36:24.615 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:24.608+0000 7f8ac759e640 1 -- 172.21.15.43:0/2945534475 shutdown_connections 2023-08-09T13:36:24.615 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:24.608+0000 7f8ac759e640 1 -- 172.21.15.43:0/2945534475 >> 172.21.15.43:0/2945534475 conn(0x7f8ac80ccad0 msgr2=0x7f8ac80cceb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-09T13:36:24.615 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:24.608+0000 7f8ac759e640 1 -- 172.21.15.43:0/2945534475 shutdown_connections 2023-08-09T13:36:24.615 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:24.608+0000 7f8ac759e640 1 -- 172.21.15.43:0/2945534475 wait complete. 2023-08-09T13:36:24.615 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:24.609+0000 7f8ac759e640 1 Processor -- start 2023-08-09T13:36:24.615 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:24.610+0000 7f8ac759e640 1 -- start start 2023-08-09T13:36:24.615 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:24.611+0000 7f8ac759e640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f8ab0004d00 0x7f8ac80cf8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:36:24.615 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:24.611+0000 7f8ac759e640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f8ac814e5d0 0x7f8ac80c75c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:36:24.616 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:24.611+0000 7f8ac759e640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f8ac80c7b00 0x7f8ac80cded0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:36:24.616 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:24.611+0000 7f8ac759e640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8ac80c7f80 con 0x7f8ac80c7b00 2023-08-09T13:36:24.616 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:24.611+0000 7f8ac759e640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8ac80cfe30 con 0x7f8ac814e5d0 2023-08-09T13:36:24.616 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:24.611+0000 7f8ac759e640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8ac815aa80 con 0x7f8ab0004d00 2023-08-09T13:36:24.616 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:24.611+0000 7f8ac6d9d640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f8ac80c7b00 0x7f8ac80cded0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:36:24.616 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:24.611+0000 7f8ac6d9d640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f8ac80c7b00 0x7f8ac80cded0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:58282/0 (socket says 172.21.15.43:58282) 2023-08-09T13:36:24.616 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:24.611+0000 7f8ac6d9d640 1 -- 172.21.15.43:0/1567052738 learned_addr learned my addr 172.21.15.43:0/1567052738 (peer_addr_for_me v2:172.21.15.43:0/0) 2023-08-09T13:36:24.617 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:24.611+0000 7f8ac659c640 1 --2- 172.21.15.43:0/1567052738 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f8ab0004d00 0x7f8ac80cf8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:36:24.617 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:24.611+0000 7f8ac5d9b640 1 --2- 172.21.15.43:0/1567052738 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f8ac814e5d0 0x7f8ac80c75c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:36:24.617 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:24.611+0000 7f8ac6d9d640 1 -- 172.21.15.43:0/1567052738 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f8ab0004d00 msgr2=0x7f8ac80cf8f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:36:24.617 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:24.611+0000 7f8ac6d9d640 1 --2- 172.21.15.43:0/1567052738 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f8ab0004d00 0x7f8ac80cf8f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:36:24.617 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:24.611+0000 7f8ac6d9d640 1 -- 172.21.15.43:0/1567052738 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f8ac814e5d0 msgr2=0x7f8ac80c75c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:36:24.617 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:24.611+0000 7f8ac6d9d640 1 --2- 172.21.15.43:0/1567052738 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f8ac814e5d0 0x7f8ac80c75c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:36:24.617 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:24.612+0000 7f8ac6d9d640 1 -- 172.21.15.43:0/1567052738 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8ac815ad60 con 0x7f8ac80c7b00 2023-08-09T13:36:24.617 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:24.612+0000 7f8ac659c640 1 --2- 172.21.15.43:0/1567052738 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f8ab0004d00 0x7f8ac80cf8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-08-09T13:36:24.618 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:24.612+0000 7f8ac6d9d640 1 --2- 172.21.15.43:0/1567052738 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f8ac80c7b00 0x7f8ac80cded0 secure :-1 s=READY pgs=606 cs=0 l=1 rev1=1 crypto rx=0x7f8ac000e230 tx=0x7f8ac0009750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-09T13:36:24.618 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:24.612+0000 7f8ab6ffd640 1 -- 172.21.15.43:0/1567052738 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8ac0034cd0 con 0x7f8ac80c7b00 2023-08-09T13:36:24.618 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:24.612+0000 7f8ab6ffd640 1 -- 172.21.15.43:0/1567052738 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8ac0034e30 con 0x7f8ac80c7b00 2023-08-09T13:36:24.618 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:24.612+0000 7f8ab6ffd640 1 -- 172.21.15.43:0/1567052738 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8ac003e660 con 0x7f8ac80c7b00 2023-08-09T13:36:24.618 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:24.612+0000 7f8ac759e640 1 -- 172.21.15.43:0/1567052738 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8ac815b040 con 0x7f8ac80c7b00 2023-08-09T13:36:24.618 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:24.612+0000 7f8ac759e640 1 -- 172.21.15.43:0/1567052738 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8ac815b600 con 0x7f8ac80c7b00 2023-08-09T13:36:24.619 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:24.614+0000 7f8ac759e640 1 -- 172.21.15.43:0/1567052738 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8ac804e390 con 0x7f8ac80c7b00 2023-08-09T13:36:24.619 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:24.614+0000 7f8ab6ffd640 1 -- 172.21.15.43:0/1567052738 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 6) v1 ==== 88358+0+0 (secure 0 0 0) 0x7f8ac003e7c0 con 0x7f8ac80c7b00 2023-08-09T13:36:24.619 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:24.616+0000 7f8ab6ffd640 1 --2- 172.21.15.43:0/1567052738 >> [v2:172.21.15.203:6832/830097474,v1:172.21.15.203:6833/830097474] conn(0x7f8aa806a060 0x7f8aa806c520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:36:24.619 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:24.616+0000 7f8ab6ffd640 1 -- 172.21.15.43:0/1567052738 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(39..39 src has 1..39) v4 ==== 6382+0+0 (secure 0 0 0) 0x7f8ac00aaa90 con 0x7f8ac80c7b00 2023-08-09T13:36:24.620 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:24.616+0000 7f8ac659c640 1 -- 172.21.15.43:0/1567052738 >> [v2:172.21.15.203:6832/830097474,v1:172.21.15.203:6833/830097474] conn(0x7f8aa806a060 msgr2=0x7f8aa806c520 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.203:6832/830097474 2023-08-09T13:36:24.620 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:24.616+0000 7f8ac659c640 1 --2- 172.21.15.43:0/1567052738 >> [v2:172.21.15.203:6832/830097474,v1:172.21.15.203:6833/830097474] conn(0x7f8aa806a060 0x7f8aa806c520 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-08-09T13:36:24.621 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:24.620+0000 7f8ab6ffd640 1 -- 172.21.15.43:0/1567052738 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178066 (secure 0 0 0) 0x7f8ac0076030 con 0x7f8ac80c7b00 2023-08-09T13:36:24.814 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:24.812+0000 7f8ac759e640 1 -- 172.21.15.43:0/1567052738 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command([{prefix=config set, name=mgr/snap_schedule/dump_on_update}] v 0) v1 -- 0x7f8ac806c170 con 0x7f8ac80c7b00 2023-08-09T13:36:24.818 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:24.817+0000 7f8ac659c640 1 -- 172.21.15.43:0/1567052738 >> [v2:172.21.15.203:6832/830097474,v1:172.21.15.203:6833/830097474] conn(0x7f8aa806a060 msgr2=0x7f8aa806c520 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.203:6832/830097474 2023-08-09T13:36:24.819 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:24.817+0000 7f8ac659c640 1 --2- 172.21.15.43:0/1567052738 >> [v2:172.21.15.203:6832/830097474,v1:172.21.15.203:6833/830097474] conn(0x7f8aa806a060 0x7f8aa806c520 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2023-08-09T13:36:24.822 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:24.821+0000 7f8ab6ffd640 1 -- 172.21.15.43:0/1567052738 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mgr/snap_schedule/dump_on_update}]=0 v12)=0 v12) v1 ==== 133+0+0 (secure 0 0 0) 0x7f8ac0079650 con 0x7f8ac80c7b00 2023-08-09T13:36:24.825 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:24.824+0000 7f8ac759e640 1 -- 172.21.15.43:0/1567052738 >> [v2:172.21.15.203:6832/830097474,v1:172.21.15.203:6833/830097474] conn(0x7f8aa806a060 msgr2=0x7f8aa806c520 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-08-09T13:36:24.825 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:24.824+0000 7f8ac759e640 1 --2- 172.21.15.43:0/1567052738 >> [v2:172.21.15.203:6832/830097474,v1:172.21.15.203:6833/830097474] conn(0x7f8aa806a060 0x7f8aa806c520 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:36:24.826 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:24.824+0000 7f8ac759e640 1 -- 172.21.15.43:0/1567052738 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f8ac80c7b00 msgr2=0x7f8ac80cded0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:36:24.826 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:24.824+0000 7f8ac759e640 1 --2- 172.21.15.43:0/1567052738 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f8ac80c7b00 0x7f8ac80cded0 secure :-1 s=READY pgs=606 cs=0 l=1 rev1=1 crypto rx=0x7f8ac000e230 tx=0x7f8ac0009750 comp rx=0 tx=0).stop 2023-08-09T13:36:24.826 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:24.824+0000 7f8ac759e640 1 -- 172.21.15.43:0/1567052738 shutdown_connections 2023-08-09T13:36:24.827 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:24.824+0000 7f8ac759e640 1 --2- 172.21.15.43:0/1567052738 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f8ac80c7b00 0x7f8ac80cded0 unknown :-1 s=CLOSED pgs=606 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:36:24.827 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:24.824+0000 7f8ac759e640 1 --2- 172.21.15.43:0/1567052738 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f8ac814e5d0 0x7f8ac80c75c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:36:24.827 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:24.824+0000 7f8ac759e640 1 --2- 172.21.15.43:0/1567052738 >> [v2:172.21.15.203:6832/830097474,v1:172.21.15.203:6833/830097474] conn(0x7f8aa806a060 0x7f8aa806c520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:36:24.827 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:24.824+0000 7f8ac759e640 1 --2- 172.21.15.43:0/1567052738 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f8ab0004d00 0x7f8ac80cf8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:36:24.827 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:24.824+0000 7f8ac759e640 1 -- 172.21.15.43:0/1567052738 >> 172.21.15.43:0/1567052738 conn(0x7f8ac80ccad0 msgr2=0x7f8ac80b0c70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-09T13:36:24.827 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:24.824+0000 7f8ac759e640 1 -- 172.21.15.43:0/1567052738 shutdown_connections 2023-08-09T13:36:24.827 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:24.824+0000 7f8ac759e640 1 -- 172.21.15.43:0/1567052738 wait complete. 2023-08-09T13:36:24.844 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-09T13:36:24.844 DEBUG:teuthology.orchestra.run.smithi043:> (cd /home/ubuntu/cephtest/mnt.0 && exec mkdir -p snap_test_dir1) 2023-08-09T13:36:24.854 INFO:tasks.ceph.mgr.x.smithi203.stderr:2023-08-09T13:36:24.852+0000 7f58fd03b040 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2023-08-09T13:36:24.881 INFO:tasks.ceph.mgr.y.smithi043.stderr:2023-08-09T13:36:24.879+0000 7f0522250040 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2023-08-09T13:36:24.914 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-09T13:36:24.914 DEBUG:teuthology.orchestra.run.smithi043:> (cd /home/ubuntu/cephtest/mnt.0 && exec mkdir -p snap_test_dir1/dir0) 2023-08-09T13:36:24.976 INFO:tasks.ceph.mgr.x.smithi203.stderr:2023-08-09T13:36:24.973+0000 7f58fd03b040 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2023-08-09T13:36:25.016 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs snap-schedule add --fs cephfs /snap_test_dir1/dir0 1M 2023-08-09T13:36:25.022 INFO:tasks.ceph.mgr.y.smithi043.stderr:2023-08-09T13:36:25.017+0000 7f0522250040 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2023-08-09T13:36:25.032 INFO:tasks.ceph.mgr.x.smithi203.stderr:2023-08-09T13:36:25.030+0000 7f58fd03b040 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2023-08-09T13:36:25.081 INFO:tasks.ceph.mgr.y.smithi043.stderr:2023-08-09T13:36:25.078+0000 7f0522250040 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2023-08-09T13:36:25.083 INFO:tasks.ceph.mgr.x.smithi203.stderr:2023-08-09T13:36:25.080+0000 7f58fd03b040 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2023-08-09T13:36:25.133 INFO:tasks.ceph.mgr.y.smithi043.stderr:2023-08-09T13:36:25.131+0000 7f0522250040 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2023-08-09T13:36:25.229 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:25.227+0000 7efeed471640 1 Processor -- start 2023-08-09T13:36:25.229 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:25.228+0000 7efeed471640 1 -- start start 2023-08-09T13:36:25.231 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:25.229+0000 7efeed471640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7efee80b5b80 0x7efee8167190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:36:25.231 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:25.229+0000 7efeed471640 1 -- --> v1:172.21.15.43:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7efee80d1890 con 0x7efee80b80a0 2023-08-09T13:36:25.231 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:25.229+0000 7efeed471640 1 -- --> v1:172.21.15.203:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7efee80d0c80 con 0x7efee815c6f0 2023-08-09T13:36:25.232 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:25.229+0000 7efeed471640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7efee80d2d30 con 0x7efee80b5b80 2023-08-09T13:36:25.233 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:25.230+0000 7efee67fc640 1 --1- >> v1:172.21.15.43:6789/0 conn(0x7efee80b80a0 0x7efee80b5470 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.43:6789/0 says I am v1:172.21.15.43:33326/0 (socket says 172.21.15.43:33326) 2023-08-09T13:36:25.234 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:25.230+0000 7efee67fc640 1 -- 172.21.15.43:0/1730091967 learned_addr learned my addr 172.21.15.43:0/1730091967 (peer_addr_for_me v1:172.21.15.43:0/0) 2023-08-09T13:36:25.234 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:25.230+0000 7efee77fe640 1 --2- 172.21.15.43:0/1730091967 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7efee80b5b80 0x7efee8167190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:36:25.234 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:25.230+0000 7efee5ffb640 1 -- 172.21.15.43:0/1730091967 <== mon.0 v1:172.21.15.43:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1915717516 0 0) 0x7efee80d1890 con 0x7efee80b80a0 2023-08-09T13:36:25.234 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:25.230+0000 7efee5ffb640 1 -- 172.21.15.43:0/1730091967 --> v1:172.21.15.43:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7efed0003520 con 0x7efee80b80a0 2023-08-09T13:36:25.234 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:25.231+0000 7efee5ffb640 1 -- 172.21.15.43:0/1730091967 <== mon.0 v1:172.21.15.43:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 647064602 0 0) 0x7efed0003520 con 0x7efee80b80a0 2023-08-09T13:36:25.234 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:25.231+0000 7efee5ffb640 1 -- 172.21.15.43:0/1730091967 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7efee80b5b80 msgr2=0x7efee8167190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-09T13:36:25.235 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:25.231+0000 7efee5ffb640 1 --2- 172.21.15.43:0/1730091967 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7efee80b5b80 0x7efee8167190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:36:25.235 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:25.231+0000 7efee5ffb640 1 -- 172.21.15.43:0/1730091967 >> v1:172.21.15.203:6789/0 conn(0x7efee815c6f0 legacy=0x7efee80d03a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:36:25.235 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:25.231+0000 7efee5ffb640 1 -- 172.21.15.43:0/1730091967 --> v1:172.21.15.43:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efee80d0f60 con 0x7efee80b80a0 2023-08-09T13:36:25.235 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:25.231+0000 7efee5ffb640 1 -- 172.21.15.43:0/1730091967 <== mon.0 v1:172.21.15.43:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1285150740 0 0) 0x7efedc001670 con 0x7efee80b80a0 2023-08-09T13:36:25.235 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:25.231+0000 7efee5ffb640 1 -- 172.21.15.43:0/1730091967 >> v1:172.21.15.43:6789/0 conn(0x7efee80b80a0 legacy=0x7efee80b5470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:36:25.235 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:25.231+0000 7efee5ffb640 1 --2- 172.21.15.43:0/1730091967 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7efed0003a10 0x7efed0003e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:36:25.235 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:25.231+0000 7efee5ffb640 1 --2- 172.21.15.43:0/1730091967 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7efed00047c0 0x7efed0004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:36:25.236 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:25.231+0000 7efee5ffb640 1 --2- 172.21.15.43:0/1730091967 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7efee80b5b80 0x7efed0005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:36:25.236 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:25.231+0000 7efee5ffb640 1 -- 172.21.15.43:0/1730091967 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efee80d2d30 con 0x7efed0003a10 2023-08-09T13:36:25.236 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:25.231+0000 7efee5ffb640 1 -- 172.21.15.43:0/1730091967 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efed000bab0 con 0x7efed00047c0 2023-08-09T13:36:25.236 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:25.231+0000 7efee5ffb640 1 -- 172.21.15.43:0/1730091967 --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7efed000bc70 con 0x7efee80b5b80 2023-08-09T13:36:25.236 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:25.231+0000 7efee6ffd640 1 --2- 172.21.15.43:0/1730091967 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7efed0003a10 0x7efed0003e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:36:25.236 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:25.231+0000 7efee77fe640 1 --2- 172.21.15.43:0/1730091967 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7efee80b5b80 0x7efed0005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:36:25.237 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:25.231+0000 7efee6ffd640 1 -- 172.21.15.43:0/1730091967 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7efee80b5b80 msgr2=0x7efed0005180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-09T13:36:25.237 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:25.231+0000 7efee6ffd640 1 --2- 172.21.15.43:0/1730091967 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7efee80b5b80 0x7efed0005180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:36:25.237 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:25.231+0000 7efee6ffd640 1 -- 172.21.15.43:0/1730091967 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7efed00047c0 msgr2=0x7efed0004c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-09T13:36:25.237 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:25.231+0000 7efee6ffd640 1 --2- 172.21.15.43:0/1730091967 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7efed00047c0 0x7efed0004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:36:25.237 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:25.231+0000 7efee6ffd640 1 -- 172.21.15.43:0/1730091967 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efed0003520 con 0x7efed0003a10 2023-08-09T13:36:25.239 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:25.232+0000 7efee6ffd640 1 --2- 172.21.15.43:0/1730091967 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7efed0003a10 0x7efed0003e30 secure :-1 s=READY pgs=608 cs=0 l=1 rev1=1 crypto rx=0x7efed8001750 tx=0x7efed00057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2a25d9998d4b2167 server_cookie=0 in_seq=0 out_seq=0 2023-08-09T13:36:25.239 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:25.233+0000 7efee5ffb640 1 -- 172.21.15.43:0/1730091967 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efed80052d0 con 0x7efed0003a10 2023-08-09T13:36:25.240 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:25.233+0000 7efee5ffb640 1 -- 172.21.15.43:0/1730091967 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efed802e050 con 0x7efed0003a10 2023-08-09T13:36:25.240 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:25.233+0000 7efee5ffb640 1 -- 172.21.15.43:0/1730091967 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efed80035f0 con 0x7efed0003a10 2023-08-09T13:36:25.240 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:25.233+0000 7efeed471640 1 -- 172.21.15.43:0/1730091967 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7efed0003a10 msgr2=0x7efed0003e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:36:25.240 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:25.233+0000 7efeed471640 1 --2- 172.21.15.43:0/1730091967 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7efed0003a10 0x7efed0003e30 secure :-1 s=READY pgs=608 cs=0 l=1 rev1=1 crypto rx=0x7efed8001750 tx=0x7efed00057f0 comp rx=0 tx=0).stop 2023-08-09T13:36:25.240 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:25.233+0000 7efee77fe640 1 -- 172.21.15.43:0/1730091967 reap_dead start 2023-08-09T13:36:25.240 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:25.233+0000 7efeed471640 1 -- 172.21.15.43:0/1730091967 shutdown_connections 2023-08-09T13:36:25.240 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:25.233+0000 7efeed471640 1 -- 172.21.15.43:0/1730091967 >> 172.21.15.43:0/1730091967 conn(0x7efee80712e0 msgr2=0x7efee80716f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-09T13:36:25.241 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:25.233+0000 7efeed471640 1 -- 172.21.15.43:0/1730091967 shutdown_connections 2023-08-09T13:36:25.241 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:25.234+0000 7efeed471640 1 -- 172.21.15.43:0/1730091967 wait complete. 2023-08-09T13:36:25.241 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:25.235+0000 7efeed471640 1 Processor -- start 2023-08-09T13:36:25.241 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:25.235+0000 7efeed471640 1 -- start start 2023-08-09T13:36:25.241 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:25.236+0000 7efeed471640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7efed0003a10 0x7efee8156970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:36:25.241 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:25.236+0000 7efeed471640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7efee8156eb0 0x7efee81465c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:36:25.241 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:25.236+0000 7efeed471640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7efee8146b00 0x7efee8146f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:36:25.241 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:25.236+0000 7efeed471640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efee81474c0 con 0x7efee8156eb0 2023-08-09T13:36:25.242 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:25.236+0000 7efeed471640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efee80c88a0 con 0x7efed0003a10 2023-08-09T13:36:25.242 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:25.236+0000 7efeed471640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7efee80cb1b0 con 0x7efee8146b00 2023-08-09T13:36:25.242 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:25.237+0000 7efee67fc640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7efee8156eb0 0x7efee81465c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:36:25.242 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:25.237+0000 7efee67fc640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7efee8156eb0 0x7efee81465c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:49378/0 (socket says 172.21.15.43:49378) 2023-08-09T13:36:25.242 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:25.237+0000 7efee67fc640 1 -- 172.21.15.43:0/858786148 learned_addr learned my addr 172.21.15.43:0/858786148 (peer_addr_for_me v2:172.21.15.43:0/0) 2023-08-09T13:36:25.242 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:25.237+0000 7efee77fe640 1 --2- 172.21.15.43:0/858786148 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7efee8146b00 0x7efee8146f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:36:25.242 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:25.237+0000 7efee6ffd640 1 --2- 172.21.15.43:0/858786148 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7efed0003a10 0x7efee8156970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:36:25.243 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:25.237+0000 7efee67fc640 1 -- 172.21.15.43:0/858786148 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7efee8146b00 msgr2=0x7efee8146f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:36:25.243 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:25.237+0000 7efee67fc640 1 --2- 172.21.15.43:0/858786148 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7efee8146b00 0x7efee8146f80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:36:25.243 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:25.237+0000 7efee67fc640 1 -- 172.21.15.43:0/858786148 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7efed0003a10 msgr2=0x7efee8156970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:36:25.243 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:25.237+0000 7efee67fc640 1 --2- 172.21.15.43:0/858786148 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7efed0003a10 0x7efee8156970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:36:25.243 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:25.237+0000 7efee67fc640 1 -- 172.21.15.43:0/858786148 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efee8157330 con 0x7efee8156eb0 2023-08-09T13:36:25.243 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:25.238+0000 7efee67fc640 1 --2- 172.21.15.43:0/858786148 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7efee8156eb0 0x7efee81465c0 secure :-1 s=READY pgs=609 cs=0 l=1 rev1=1 crypto rx=0x7efedc00c700 tx=0x7efedc00cbc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-09T13:36:25.243 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:25.238+0000 7efecf7fe640 1 -- 172.21.15.43:0/858786148 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efedc0072d0 con 0x7efee8156eb0 2023-08-09T13:36:25.243 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:25.238+0000 7efecf7fe640 1 -- 172.21.15.43:0/858786148 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efedc007430 con 0x7efee8156eb0 2023-08-09T13:36:25.244 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:25.238+0000 7efecf7fe640 1 -- 172.21.15.43:0/858786148 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efedc0047e0 con 0x7efee8156eb0 2023-08-09T13:36:25.244 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:25.238+0000 7efeed471640 1 -- 172.21.15.43:0/858786148 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7efee80caed0 con 0x7efee8156eb0 2023-08-09T13:36:25.244 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:25.238+0000 7efeed471640 1 -- 172.21.15.43:0/858786148 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7efee81449a0 con 0x7efee8156eb0 2023-08-09T13:36:25.245 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:25.239+0000 7efeed471640 1 -- 172.21.15.43:0/858786148 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7efee80739f0 con 0x7efee8156eb0 2023-08-09T13:36:25.245 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:25.241+0000 7efecf7fe640 1 -- 172.21.15.43:0/858786148 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 6) v1 ==== 88358+0+0 (secure 0 0 0) 0x7efedc004940 con 0x7efee8156eb0 2023-08-09T13:36:25.245 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:25.242+0000 7efecf7fe640 1 --2- 172.21.15.43:0/858786148 >> [v2:172.21.15.203:6832/830097474,v1:172.21.15.203:6833/830097474] conn(0x7efed406a0d0 0x7efed406c590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:36:25.245 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:25.242+0000 7efecf7fe640 1 -- 172.21.15.43:0/858786148 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(39..39 src has 1..39) v4 ==== 6382+0+0 (secure 0 0 0) 0x7efedc087ae0 con 0x7efee8156eb0 2023-08-09T13:36:25.245 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:25.242+0000 7efee6ffd640 1 -- 172.21.15.43:0/858786148 >> [v2:172.21.15.203:6832/830097474,v1:172.21.15.203:6833/830097474] conn(0x7efed406a0d0 msgr2=0x7efed406c590 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.203:6832/830097474 2023-08-09T13:36:25.245 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:25.242+0000 7efee6ffd640 1 --2- 172.21.15.43:0/858786148 >> [v2:172.21.15.203:6832/830097474,v1:172.21.15.203:6833/830097474] conn(0x7efed406a0d0 0x7efed406c590 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-08-09T13:36:25.247 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:25.246+0000 7efecf7fe640 1 -- 172.21.15.43:0/858786148 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178066 (secure 0 0 0) 0x7efedc053080 con 0x7efee8156eb0 2023-08-09T13:36:25.446 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:25.444+0000 7efee6ffd640 1 -- 172.21.15.43:0/858786148 >> [v2:172.21.15.203:6832/830097474,v1:172.21.15.203:6833/830097474] conn(0x7efed406a0d0 msgr2=0x7efed406c590 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.203:6832/830097474 2023-08-09T13:36:25.447 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:25.444+0000 7efee6ffd640 1 --2- 172.21.15.43:0/858786148 >> [v2:172.21.15.203:6832/830097474,v1:172.21.15.203:6833/830097474] conn(0x7efed406a0d0 0x7efed406c590 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2023-08-09T13:36:25.463 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:25.462+0000 7efeed471640 1 -- 172.21.15.43:0/858786148 --> [v2:172.21.15.203:6832/830097474,v1:172.21.15.203:6833/830097474] -- mgr_command(tid 0: {"prefix": "fs snap-schedule add", "fs": "cephfs", "path": "/snap_test_dir1/dir0", "snap_schedule": "1M", "target": ["mon-mgr", ""]}) v1 -- 0x7efee80d1240 con 0x7efed406a0d0 2023-08-09T13:36:25.772 INFO:tasks.ceph.mgr.x.smithi203.stderr:2023-08-09T13:36:25.770+0000 7f58fd03b040 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2023-08-09T13:36:25.834 INFO:tasks.ceph.mgr.x.smithi203.stderr:2023-08-09T13:36:25.831+0000 7f58fd03b040 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2023-08-09T13:36:25.847 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:25.845+0000 7efee6ffd640 1 -- 172.21.15.43:0/858786148 >> [v2:172.21.15.203:6832/830097474,v1:172.21.15.203:6833/830097474] conn(0x7efed406a0d0 msgr2=0x7efed406c590 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.203:6832/830097474 2023-08-09T13:36:25.847 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:25.845+0000 7efee6ffd640 1 --2- 172.21.15.43:0/858786148 >> [v2:172.21.15.203:6832/830097474,v1:172.21.15.203:6833/830097474] conn(0x7efed406a0d0 0x7efed406c590 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2023-08-09T13:36:25.866 INFO:tasks.ceph.mgr.y.smithi043.stderr:2023-08-09T13:36:25.865+0000 7f0522250040 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2023-08-09T13:36:25.885 INFO:tasks.ceph.mgr.x.smithi203.stderr:2023-08-09T13:36:25.882+0000 7f58fd03b040 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2023-08-09T13:36:25.931 INFO:tasks.ceph.mgr.y.smithi043.stderr:2023-08-09T13:36:25.929+0000 7f0522250040 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2023-08-09T13:36:25.983 INFO:tasks.ceph.mgr.y.smithi043.stderr:2023-08-09T13:36:25.981+0000 7f0522250040 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2023-08-09T13:36:26.003 INFO:tasks.ceph.mgr.x.smithi203.stderr:2023-08-09T13:36:26.000+0000 7f58fd03b040 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2023-08-09T13:36:26.108 INFO:tasks.ceph.mgr.y.smithi043.stderr:2023-08-09T13:36:26.106+0000 7f0522250040 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2023-08-09T13:36:26.161 INFO:tasks.ceph.mgr.x.smithi203.stderr:2023-08-09T13:36:26.158+0000 7f58fd03b040 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2023-08-09T13:36:26.268 INFO:tasks.ceph.mgr.x.smithi203.stderr:2023-08-09T13:36:26.266+0000 7f58fd03b040 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2023-08-09T13:36:26.270 INFO:tasks.ceph.mgr.y.smithi043.stderr:2023-08-09T13:36:26.269+0000 7f0522250040 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2023-08-09T13:36:26.317 INFO:tasks.ceph.mgr.x.smithi203.stderr:2023-08-09T13:36:26.315+0000 7f58fd03b040 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2023-08-09T13:36:26.382 INFO:tasks.ceph.mgr.y.smithi043.stderr:2023-08-09T13:36:26.381+0000 7f0522250040 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2023-08-09T13:36:26.434 INFO:tasks.ceph.mgr.y.smithi043.stderr:2023-08-09T13:36:26.432+0000 7f0522250040 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2023-08-09T13:36:26.476 INFO:tasks.ceph.mgr.x.smithi203.stderr:2023-08-09T13:36:26.473+0000 7f58fd03b040 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2023-08-09T13:36:26.601 INFO:tasks.ceph.mgr.y.smithi043.stderr:2023-08-09T13:36:26.599+0000 7f0522250040 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2023-08-09T13:36:26.648 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:26.646+0000 7efee6ffd640 1 -- 172.21.15.43:0/858786148 >> [v2:172.21.15.203:6832/830097474,v1:172.21.15.203:6833/830097474] conn(0x7efed406a0d0 msgr2=0x7efed406c590 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.203:6832/830097474 2023-08-09T13:36:26.648 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:26.646+0000 7efee6ffd640 1 --2- 172.21.15.43:0/858786148 >> [v2:172.21.15.203:6832/830097474,v1:172.21.15.203:6833/830097474] conn(0x7efed406a0d0 0x7efed406c590 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2023-08-09T13:36:26.866 INFO:tasks.ceph.mgr.x.smithi203.stderr:2023-08-09T13:36:26.864+0000 7f58fd03b040 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2023-08-09T13:36:27.007 INFO:tasks.ceph.mgr.y.smithi043.stderr:2023-08-09T13:36:27.006+0000 7f0522250040 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2023-08-09T13:36:27.089 INFO:tasks.ceph.mgr.x.smithi203.stderr:2023-08-09T13:36:27.087+0000 7f58fd03b040 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2023-08-09T13:36:27.146 INFO:tasks.ceph.mgr.x.smithi203.stderr:2023-08-09T13:36:27.143+0000 7f58fd03b040 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2023-08-09T13:36:27.195 INFO:tasks.ceph.mgr.x.smithi203.stderr:2023-08-09T13:36:27.193+0000 7f58fd03b040 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2023-08-09T13:36:27.238 INFO:tasks.ceph.mgr.y.smithi043.stderr:2023-08-09T13:36:27.237+0000 7f0522250040 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2023-08-09T13:36:27.247 INFO:tasks.ceph.mgr.x.smithi203.stderr:2023-08-09T13:36:27.245+0000 7f58fd03b040 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2023-08-09T13:36:27.298 INFO:tasks.ceph.mgr.y.smithi043.stderr:2023-08-09T13:36:27.296+0000 7f0522250040 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2023-08-09T13:36:27.351 INFO:tasks.ceph.mgr.y.smithi043.stderr:2023-08-09T13:36:27.349+0000 7f0522250040 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2023-08-09T13:36:27.405 INFO:tasks.ceph.mgr.y.smithi043.stderr:2023-08-09T13:36:27.403+0000 7f0522250040 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2023-08-09T13:36:27.541 INFO:tasks.ceph.mgr.x.smithi203.stderr:2023-08-09T13:36:27.539+0000 7f58fd03b040 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2023-08-09T13:36:27.599 INFO:tasks.ceph.mgr.x.smithi203.stderr:2023-08-09T13:36:27.596+0000 7f58fd03b040 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2023-08-09T13:36:27.650 INFO:tasks.ceph.mgr.x.smithi203.stderr:2023-08-09T13:36:27.648+0000 7f58fd03b040 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2023-08-09T13:36:27.715 INFO:tasks.ceph.mgr.y.smithi043.stderr:2023-08-09T13:36:27.713+0000 7f0522250040 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2023-08-09T13:36:27.779 INFO:tasks.ceph.mgr.y.smithi043.stderr:2023-08-09T13:36:27.778+0000 7f0522250040 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2023-08-09T13:36:27.814 INFO:tasks.ceph.mgr.x.smithi203.stderr:2023-08-09T13:36:27.812+0000 7f58fd03b040 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2023-08-09T13:36:27.835 INFO:tasks.ceph.mgr.y.smithi043.stderr:2023-08-09T13:36:27.833+0000 7f0522250040 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2023-08-09T13:36:27.865 INFO:tasks.ceph.mgr.x.smithi203.stderr:2023-08-09T13:36:27.862+0000 7f58fd03b040 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2023-08-09T13:36:27.918 INFO:tasks.ceph.mgr.x.smithi203.stderr:2023-08-09T13:36:27.915+0000 7f58fd03b040 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2023-08-09T13:36:27.968 INFO:tasks.ceph.mgr.x.smithi203.stderr:2023-08-09T13:36:27.966+0000 7f58fd03b040 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2023-08-09T13:36:27.993 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:27.992+0000 7efecf7fe640 1 -- 172.21.15.43:0/858786148 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mgrmap(e 7) v1 ==== 88069+0+0 (secure 0 0 0) 0x7efedc053450 con 0x7efee8156eb0 2023-08-09T13:36:27.994 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:27.993+0000 7efecf7fe640 1 -- 172.21.15.43:0/858786148 >> [v2:172.21.15.203:6832/830097474,v1:172.21.15.203:6833/830097474] conn(0x7efed406a0d0 msgr2=0x7efed406c590 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-08-09T13:36:27.994 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:27.993+0000 7efecf7fe640 1 --2- 172.21.15.43:0/858786148 >> [v2:172.21.15.203:6832/830097474,v1:172.21.15.203:6833/830097474] conn(0x7efed406a0d0 0x7efed406c590 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:36:28.010 INFO:tasks.ceph.mgr.y.smithi043.stderr:2023-08-09T13:36:28.008+0000 7f0522250040 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2023-08-09T13:36:28.067 INFO:tasks.ceph.mgr.y.smithi043.stderr:2023-08-09T13:36:28.066+0000 7f0522250040 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2023-08-09T13:36:28.136 INFO:tasks.ceph.mgr.y.smithi043.stderr:2023-08-09T13:36:28.135+0000 7f0522250040 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2023-08-09T13:36:28.161 INFO:tasks.ceph.mgr.x.smithi203.stderr:2023-08-09T13:36:28.159+0000 7f587cb21640 -1 client.0 error registering admin socket command: (17) File exists 2023-08-09T13:36:28.162 INFO:tasks.ceph.mgr.x.smithi203.stderr:2023-08-09T13:36:28.159+0000 7f587cb21640 -1 client.0 error registering admin socket command: (17) File exists 2023-08-09T13:36:28.162 INFO:tasks.ceph.mgr.x.smithi203.stderr:2023-08-09T13:36:28.159+0000 7f587cb21640 -1 client.0 error registering admin socket command: (17) File exists 2023-08-09T13:36:28.162 INFO:tasks.ceph.mgr.x.smithi203.stderr:2023-08-09T13:36:28.159+0000 7f587cb21640 -1 client.0 error registering admin socket command: (17) File exists 2023-08-09T13:36:28.162 INFO:tasks.ceph.mgr.x.smithi203.stderr:2023-08-09T13:36:28.159+0000 7f587cb21640 -1 client.0 error registering admin socket command: (17) File exists 2023-08-09T13:36:28.162 INFO:tasks.ceph.mgr.x.smithi203.stderr:2023-08-09T13:36:28.159+0000 7f5879b1b640 -1 client.0 error registering admin socket command: (17) File exists 2023-08-09T13:36:28.162 INFO:tasks.ceph.mgr.x.smithi203.stderr:2023-08-09T13:36:28.159+0000 7f5879b1b640 -1 client.0 error registering admin socket command: (17) File exists 2023-08-09T13:36:28.162 INFO:tasks.ceph.mgr.x.smithi203.stderr:2023-08-09T13:36:28.159+0000 7f5879b1b640 -1 client.0 error registering admin socket command: (17) File exists 2023-08-09T13:36:28.163 INFO:tasks.ceph.mgr.x.smithi203.stderr:2023-08-09T13:36:28.159+0000 7f5879b1b640 -1 client.0 error registering admin socket command: (17) File exists 2023-08-09T13:36:28.163 INFO:tasks.ceph.mgr.x.smithi203.stderr:2023-08-09T13:36:28.159+0000 7f5879b1b640 -1 client.0 error registering admin socket command: (17) File exists 2023-08-09T13:36:28.195 INFO:tasks.ceph.mgr.y.smithi043.stderr:2023-08-09T13:36:28.193+0000 7f0522250040 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2023-08-09T13:36:29.006 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:29.005+0000 7efecf7fe640 1 -- 172.21.15.43:0/858786148 <== mon.0 v2:172.21.15.43:3300/0 8 ==== mgrmap(e 8) v1 ==== 88196+0+0 (secure 0 0 0) 0x7efedc075420 con 0x7efee8156eb0 2023-08-09T13:36:29.007 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:29.006+0000 7efecf7fe640 1 --2- 172.21.15.43:0/858786148 >> [v2:172.21.15.203:6832/998005540,v1:172.21.15.203:6833/998005540] conn(0x7efed401c930 0x7efed4074b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:36:29.008 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:29.006+0000 7efecf7fe640 1 -- 172.21.15.43:0/858786148 --> [v2:172.21.15.203:6832/998005540,v1:172.21.15.203:6833/998005540] -- mgr_command(tid 0: {"prefix": "fs snap-schedule add", "fs": "cephfs", "path": "/snap_test_dir1/dir0", "snap_schedule": "1M", "target": ["mon-mgr", ""]}) v1 -- 0x7efed4036180 con 0x7efed401c930 2023-08-09T13:36:29.009 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:29.008+0000 7efee6ffd640 1 --2- 172.21.15.43:0/858786148 >> [v2:172.21.15.203:6832/998005540,v1:172.21.15.203:6833/998005540] conn(0x7efed401c930 0x7efed4074b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:36:29.014 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:29.013+0000 7efee6ffd640 1 --2- 172.21.15.43:0/858786148 >> [v2:172.21.15.203:6832/998005540,v1:172.21.15.203:6833/998005540] conn(0x7efed401c930 0x7efed4074b40 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7efed8033000 tx=0x7efed8009030 comp rx=0 tx=0).ready entity=mgr.5003 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-09T13:36:29.186 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:29.184+0000 7efecf7fe640 1 -- 172.21.15.43:0/858786148 <== mgr.5003 v2:172.21.15.203:6832/998005540 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+43 (secure 0 0 0) 0x7efed80054d0 con 0x7efed401c930 2023-08-09T13:36:29.186 INFO:teuthology.orchestra.run.smithi043.stdout:Schedule set for path /snap_test_dir1/dir0 2023-08-09T13:36:29.189 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:29.187+0000 7efeed471640 1 -- 172.21.15.43:0/858786148 >> [v2:172.21.15.203:6832/998005540,v1:172.21.15.203:6833/998005540] conn(0x7efed401c930 msgr2=0x7efed4074b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:36:29.189 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:29.187+0000 7efeed471640 1 --2- 172.21.15.43:0/858786148 >> [v2:172.21.15.203:6832/998005540,v1:172.21.15.203:6833/998005540] conn(0x7efed401c930 0x7efed4074b40 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7efed8033000 tx=0x7efed8009030 comp rx=0 tx=0).stop 2023-08-09T13:36:29.189 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:29.187+0000 7efeed471640 1 -- 172.21.15.43:0/858786148 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7efee8156eb0 msgr2=0x7efee81465c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:36:29.189 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:29.187+0000 7efeed471640 1 --2- 172.21.15.43:0/858786148 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7efee8156eb0 0x7efee81465c0 secure :-1 s=READY pgs=609 cs=0 l=1 rev1=1 crypto rx=0x7efedc00c700 tx=0x7efedc00cbc0 comp rx=0 tx=0).stop 2023-08-09T13:36:29.190 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:29.188+0000 7efee77fe640 1 -- 172.21.15.43:0/858786148 reap_dead start 2023-08-09T13:36:29.190 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:29.188+0000 7efeed471640 1 -- 172.21.15.43:0/858786148 shutdown_connections 2023-08-09T13:36:29.190 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:29.188+0000 7efeed471640 1 -- 172.21.15.43:0/858786148 >> 172.21.15.43:0/858786148 conn(0x7efee80712e0 msgr2=0x7efee80ccc60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-09T13:36:29.190 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:29.188+0000 7efeed471640 1 -- 172.21.15.43:0/858786148 shutdown_connections 2023-08-09T13:36:29.191 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:29.188+0000 7efeed471640 1 -- 172.21.15.43:0/858786148 wait complete. 2023-08-09T13:36:29.212 DEBUG:tasks.cephfs.test_snap_schedules:res=Schedule set for path /snap_test_dir1/dir0 2023-08-09T13:36:29.212 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-09T13:36:29.212 DEBUG:teuthology.orchestra.run.smithi043:> (cd /home/ubuntu/cephtest/mnt.0 && exec mkdir -p snap_test_dir1/dir1) 2023-08-09T13:36:29.276 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs snap-schedule add --fs cephfs /snap_test_dir1/dir1 1M 2023-08-09T13:36:29.326 DEBUG:teuthology.orchestra.run.smithi043:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-08-09T13:36:29.332 DEBUG:teuthology.orchestra.run.smithi203:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-08-09T13:36:29.514 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:29.512+0000 7fb34f502640 1 Processor -- start 2023-08-09T13:36:29.514 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:29.512+0000 7fb34f502640 1 -- start start 2023-08-09T13:36:29.515 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:29.514+0000 7fb34f502640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fb3480cc8e0 0x7fb348158760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:36:29.516 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:29.514+0000 7fb34f502640 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7fb34806f330 0x7fb34806f7c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:36:29.516 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:29.514+0000 7fb34f502640 1 -- --> v1:172.21.15.203:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb3480c9390 con 0x7fb348071200 2023-08-09T13:36:29.516 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:29.514+0000 7fb34f502640 1 -- --> v2:172.21.15.203:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb3480715e0 con 0x7fb34806f330 2023-08-09T13:36:29.516 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:29.514+0000 7fb34f502640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb3480afff0 con 0x7fb3480cc8e0 2023-08-09T13:36:29.517 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:29.515+0000 7fb34ed01640 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7fb34806f330 0x7fb34806f7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:36:29.517 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:29.515+0000 7fb34e500640 1 --1- >> v1:172.21.15.203:6789/0 conn(0x7fb348071200 0x7fb3480af8e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.203:6789/0 says I am v1:172.21.15.43:49066/0 (socket says 172.21.15.43:49066) 2023-08-09T13:36:29.517 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:29.515+0000 7fb34e500640 1 -- 172.21.15.43:0/3592548930 learned_addr learned my addr 172.21.15.43:0/3592548930 (peer_addr_for_me v1:172.21.15.43:0/0) 2023-08-09T13:36:29.517 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:29.515+0000 7fb34dcff640 1 --2- 172.21.15.43:0/3592548930 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fb3480cc8e0 0x7fb348158760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:36:29.518 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:29.516+0000 7fb34dcff640 1 -- 172.21.15.43:0/3592548930 >> v2:172.21.15.203:3300/0 conn(0x7fb34806f330 msgr2=0x7fb34806f7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-09T13:36:29.518 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:29.516+0000 7fb34dcff640 1 --2- 172.21.15.43:0/3592548930 >> v2:172.21.15.203:3300/0 conn(0x7fb34806f330 0x7fb34806f7c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:36:29.518 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:29.516+0000 7fb34dcff640 1 -- 172.21.15.43:0/3592548930 >> v1:172.21.15.203:6789/0 conn(0x7fb348071200 legacy=0x7fb3480af8e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:36:29.518 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:29.516+0000 7fb34dcff640 1 -- 172.21.15.43:0/3592548930 --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb348158ca0 con 0x7fb3480cc8e0 2023-08-09T13:36:29.518 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:29.517+0000 7fb34dcff640 1 --2- 172.21.15.43:0/3592548930 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fb3480cc8e0 0x7fb348158760 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7fb3380097a0 tx=0x7fb33802b8d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=607f9779af66a0d9 server_cookie=0 in_seq=0 out_seq=0 2023-08-09T13:36:29.519 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:29.518+0000 7fb34d4fe640 1 -- 172.21.15.43:0/3592548930 <== mon.2 v2:172.21.15.203:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb338031830 con 0x7fb3480cc8e0 2023-08-09T13:36:29.519 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:29.518+0000 7fb34d4fe640 1 -- 172.21.15.43:0/3592548930 <== mon.2 v2:172.21.15.203:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb338031990 con 0x7fb3480cc8e0 2023-08-09T13:36:29.519 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:29.518+0000 7fb34d4fe640 1 -- 172.21.15.43:0/3592548930 <== mon.2 v2:172.21.15.203:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb338031d20 con 0x7fb3480cc8e0 2023-08-09T13:36:29.520 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:29.518+0000 7fb34f502640 1 -- 172.21.15.43:0/3592548930 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fb3480cc8e0 msgr2=0x7fb348158760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:36:29.520 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:29.518+0000 7fb34f502640 1 --2- 172.21.15.43:0/3592548930 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fb3480cc8e0 0x7fb348158760 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7fb3380097a0 tx=0x7fb33802b8d0 comp rx=0 tx=0).stop 2023-08-09T13:36:29.521 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:29.518+0000 7fb34f502640 1 -- 172.21.15.43:0/3592548930 shutdown_connections 2023-08-09T13:36:29.521 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:29.518+0000 7fb34f502640 1 --2- 172.21.15.43:0/3592548930 >> v2:172.21.15.203:3300/0 conn(0x7fb34806f330 0x7fb34806f7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:36:29.521 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:29.518+0000 7fb34f502640 1 --2- 172.21.15.43:0/3592548930 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fb3480cc8e0 0x7fb348158760 unknown :-1 s=CLOSED pgs=84 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:36:29.521 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:29.518+0000 7fb34f502640 1 -- 172.21.15.43:0/3592548930 >> 172.21.15.43:0/3592548930 conn(0x7fb3480ca870 msgr2=0x7fb3480cac50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-09T13:36:29.521 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:29.519+0000 7fb34f502640 1 -- 172.21.15.43:0/3592548930 shutdown_connections 2023-08-09T13:36:29.521 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:29.519+0000 7fb34f502640 1 -- 172.21.15.43:0/3592548930 wait complete. 2023-08-09T13:36:29.522 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:29.520+0000 7fb34f502640 1 Processor -- start 2023-08-09T13:36:29.522 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:29.521+0000 7fb34f502640 1 -- start start 2023-08-09T13:36:29.523 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:29.521+0000 7fb34f502640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fb34806f330 0x7fb3481517a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:36:29.523 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:29.521+0000 7fb34f502640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fb348071200 0x7fb3480c76d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:36:29.523 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:29.522+0000 7fb34f502640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb3480cc8e0 0x7fb34814f660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:36:29.523 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:29.522+0000 7fb34f502640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb348150a40 con 0x7fb3480cc8e0 2023-08-09T13:36:29.523 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:29.522+0000 7fb34f502640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb3480c2e20 con 0x7fb348071200 2023-08-09T13:36:29.524 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:29.522+0000 7fb34f502640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb3480c2830 con 0x7fb34806f330 2023-08-09T13:36:29.524 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:29.522+0000 7fb34ed01640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb3480cc8e0 0x7fb34814f660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:36:29.525 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:29.522+0000 7fb34e500640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fb34806f330 0x7fb3481517a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:36:29.525 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:29.522+0000 7fb34dcff640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fb348071200 0x7fb3480c76d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:36:29.525 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:29.522+0000 7fb34ed01640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb3480cc8e0 0x7fb34814f660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:49390/0 (socket says 172.21.15.43:49390) 2023-08-09T13:36:29.525 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:29.522+0000 7fb34ed01640 1 -- 172.21.15.43:0/3097559036 learned_addr learned my addr 172.21.15.43:0/3097559036 (peer_addr_for_me v2:172.21.15.43:0/0) 2023-08-09T13:36:29.525 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:29.523+0000 7fb34ed01640 1 -- 172.21.15.43:0/3097559036 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fb34806f330 msgr2=0x7fb3481517a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:36:29.525 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:29.523+0000 7fb34ed01640 1 --2- 172.21.15.43:0/3097559036 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fb34806f330 0x7fb3481517a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:36:29.525 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:29.523+0000 7fb34ed01640 1 -- 172.21.15.43:0/3097559036 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fb348071200 msgr2=0x7fb3480c76d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:36:29.526 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:29.523+0000 7fb34ed01640 1 --2- 172.21.15.43:0/3097559036 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fb348071200 0x7fb3480c76d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:36:29.526 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:29.523+0000 7fb34ed01640 1 -- 172.21.15.43:0/3097559036 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb338009400 con 0x7fb3480cc8e0 2023-08-09T13:36:29.526 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:29.523+0000 7fb34ed01640 1 --2- 172.21.15.43:0/3097559036 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb3480cc8e0 0x7fb34814f660 secure :-1 s=READY pgs=610 cs=0 l=1 rev1=1 crypto rx=0x7fb34000efc0 tx=0x7fb34000c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-09T13:36:29.526 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:29.523+0000 7fb33effd640 1 -- 172.21.15.43:0/3097559036 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb340009ca0 con 0x7fb3480cc8e0 2023-08-09T13:36:29.527 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:29.524+0000 7fb33effd640 1 -- 172.21.15.43:0/3097559036 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb34000f040 con 0x7fb3480cc8e0 2023-08-09T13:36:29.527 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:29.524+0000 7fb33effd640 1 -- 172.21.15.43:0/3097559036 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb3400146c0 con 0x7fb3480cc8e0 2023-08-09T13:36:29.527 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:29.524+0000 7fb34f502640 1 -- 172.21.15.43:0/3097559036 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb348150200 con 0x7fb3480cc8e0 2023-08-09T13:36:29.527 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:29.524+0000 7fb34f502640 1 -- 172.21.15.43:0/3097559036 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb348150760 con 0x7fb3480cc8e0 2023-08-09T13:36:29.527 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:29.525+0000 7fb34f502640 1 -- 172.21.15.43:0/3097559036 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb3480c2f60 con 0x7fb3480cc8e0 2023-08-09T13:36:29.528 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:29.526+0000 7fb33effd640 1 -- 172.21.15.43:0/3097559036 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 8) v1 ==== 88196+0+0 (secure 0 0 0) 0x7fb340014820 con 0x7fb3480cc8e0 2023-08-09T13:36:29.529 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:29.528+0000 7fb33effd640 1 --2- 172.21.15.43:0/3097559036 >> [v2:172.21.15.203:6832/998005540,v1:172.21.15.203:6833/998005540] conn(0x7fb32c069ef0 0x7fb32c06c3b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:36:29.529 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:29.528+0000 7fb33effd640 1 -- 172.21.15.43:0/3097559036 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 6640+0+0 (secure 0 0 0) 0x7fb3400892f0 con 0x7fb3480cc8e0 2023-08-09T13:36:29.529 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:29.528+0000 7fb34e500640 1 --2- 172.21.15.43:0/3097559036 >> [v2:172.21.15.203:6832/998005540,v1:172.21.15.203:6833/998005540] conn(0x7fb32c069ef0 0x7fb32c06c3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:36:29.531 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:29.529+0000 7fb34e500640 1 --2- 172.21.15.43:0/3097559036 >> [v2:172.21.15.203:6832/998005540,v1:172.21.15.203:6833/998005540] conn(0x7fb32c069ef0 0x7fb32c06c3b0 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7fb344005880 tx=0x7fb344009040 comp rx=0 tx=0).ready entity=mgr.5003 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-09T13:36:29.534 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:29.533+0000 7fb33effd640 1 -- 172.21.15.43:0/3097559036 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178066 (secure 0 0 0) 0x7fb3480c2f60 con 0x7fb3480cc8e0 2023-08-09T13:36:29.749 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:29.748+0000 7fb34f502640 1 -- 172.21.15.43:0/3097559036 --> [v2:172.21.15.203:6832/998005540,v1:172.21.15.203:6833/998005540] -- mgr_command(tid 0: {"prefix": "fs snap-schedule add", "fs": "cephfs", "path": "/snap_test_dir1/dir1", "snap_schedule": "1M", "target": ["mon-mgr", ""]}) v1 -- 0x7fb34806b580 con 0x7fb32c069ef0 2023-08-09T13:36:29.909 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:29.907+0000 7fb33effd640 1 -- 172.21.15.43:0/3097559036 <== mgr.5003 v2:172.21.15.203:6832/998005540 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+43 (secure 0 0 0) 0x7fb34400f820 con 0x7fb32c069ef0 2023-08-09T13:36:29.909 INFO:teuthology.orchestra.run.smithi043.stdout:Schedule set for path /snap_test_dir1/dir1 2023-08-09T13:36:29.913 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:29.911+0000 7fb34f502640 1 -- 172.21.15.43:0/3097559036 >> [v2:172.21.15.203:6832/998005540,v1:172.21.15.203:6833/998005540] conn(0x7fb32c069ef0 msgr2=0x7fb32c06c3b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:36:29.913 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:29.911+0000 7fb34f502640 1 --2- 172.21.15.43:0/3097559036 >> [v2:172.21.15.203:6832/998005540,v1:172.21.15.203:6833/998005540] conn(0x7fb32c069ef0 0x7fb32c06c3b0 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7fb344005880 tx=0x7fb344009040 comp rx=0 tx=0).stop 2023-08-09T13:36:29.913 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:29.911+0000 7fb34f502640 1 -- 172.21.15.43:0/3097559036 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb3480cc8e0 msgr2=0x7fb34814f660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:36:29.914 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:29.911+0000 7fb34f502640 1 --2- 172.21.15.43:0/3097559036 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb3480cc8e0 0x7fb34814f660 secure :-1 s=READY pgs=610 cs=0 l=1 rev1=1 crypto rx=0x7fb34000efc0 tx=0x7fb34000c490 comp rx=0 tx=0).stop 2023-08-09T13:36:29.914 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:29.911+0000 7fb34f502640 1 -- 172.21.15.43:0/3097559036 shutdown_connections 2023-08-09T13:36:29.914 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:29.911+0000 7fb34f502640 1 --2- 172.21.15.43:0/3097559036 >> [v2:172.21.15.203:6832/998005540,v1:172.21.15.203:6833/998005540] conn(0x7fb32c069ef0 0x7fb32c06c3b0 unknown :-1 s=CLOSED pgs=23 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:36:29.914 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:29.911+0000 7fb34f502640 1 --2- 172.21.15.43:0/3097559036 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb3480cc8e0 0x7fb34814f660 unknown :-1 s=CLOSED pgs=610 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:36:29.914 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:29.911+0000 7fb34f502640 1 --2- 172.21.15.43:0/3097559036 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fb348071200 0x7fb3480c76d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:36:29.914 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:29.911+0000 7fb34f502640 1 --2- 172.21.15.43:0/3097559036 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fb34806f330 0x7fb3481517a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:36:29.914 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:29.911+0000 7fb34f502640 1 -- 172.21.15.43:0/3097559036 >> 172.21.15.43:0/3097559036 conn(0x7fb3480ca870 msgr2=0x7fb3480c8460 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-09T13:36:29.915 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:29.911+0000 7fb34f502640 1 -- 172.21.15.43:0/3097559036 shutdown_connections 2023-08-09T13:36:29.915 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:29.912+0000 7fb34f502640 1 -- 172.21.15.43:0/3097559036 wait complete. 2023-08-09T13:36:29.936 DEBUG:tasks.cephfs.test_snap_schedules:res=Schedule set for path /snap_test_dir1/dir1 2023-08-09T13:36:29.937 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-09T13:36:29.937 DEBUG:teuthology.orchestra.run.smithi043:> (cd /home/ubuntu/cephtest/mnt.0 && exec mkdir -p snap_test_dir1/dir2) 2023-08-09T13:36:30.001 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs snap-schedule add --fs cephfs /snap_test_dir1/dir2 1M 2023-08-09T13:36:30.244 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:30.242+0000 7f22989d2640 1 Processor -- start 2023-08-09T13:36:30.244 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:30.242+0000 7f22989d2640 1 -- start start 2023-08-09T13:36:30.245 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:30.244+0000 7f22989d2640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f22940aff80 0x7f22940c9780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:36:30.245 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:30.244+0000 7f22989d2640 1 -- --> v1:172.21.15.43:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f229415dd30 con 0x7f229415ea60 2023-08-09T13:36:30.246 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:30.244+0000 7f22989d2640 1 -- --> v1:172.21.15.203:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2294071490 con 0x7f229415a330 2023-08-09T13:36:30.246 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:30.244+0000 7f22989d2640 1 -- --> v2:172.21.15.43:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f22940c9cc0 con 0x7f22940aff80 2023-08-09T13:36:30.246 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:30.244+0000 7f2292ffd640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f22940aff80 0x7f22940c9780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:36:30.246 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:30.244+0000 7f2292ffd640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f22940aff80 0x7f22940c9780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:49406/0 (socket says 172.21.15.43:49406) 2023-08-09T13:36:30.247 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:30.244+0000 7f2292ffd640 1 -- 172.21.15.43:0/4278263961 learned_addr learned my addr 172.21.15.43:0/4278263961 (peer_addr_for_me v2:172.21.15.43:0/0) 2023-08-09T13:36:30.247 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:30.245+0000 7f2292ffd640 1 -- 172.21.15.43:0/4278263961 >> v1:172.21.15.203:6789/0 conn(0x7f229415a330 legacy=0x7f22940af870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-09T13:36:30.247 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:30.245+0000 7f2292ffd640 1 -- 172.21.15.43:0/4278263961 >> v1:172.21.15.43:6789/0 conn(0x7f229415ea60 legacy=0x7f229406b400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:36:30.247 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:30.245+0000 7f2292ffd640 1 -- 172.21.15.43:0/4278263961 --> v2:172.21.15.43:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f229415df80 con 0x7f22940aff80 2023-08-09T13:36:30.247 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:30.246+0000 7f2292ffd640 1 --2- 172.21.15.43:0/4278263961 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f22940aff80 0x7f22940c9780 secure :-1 s=READY pgs=612 cs=0 l=1 rev1=1 crypto rx=0x7f227c009870 tx=0x7f227c02b990 comp rx=0 tx=0).ready entity=mon.0 client_cookie=73b95f104f3cab38 server_cookie=0 in_seq=0 out_seq=0 2023-08-09T13:36:30.248 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:30.247+0000 7f22927fc640 1 -- 172.21.15.43:0/4278263961 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f227c030850 con 0x7f22940aff80 2023-08-09T13:36:30.248 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:30.247+0000 7f22927fc640 1 -- 172.21.15.43:0/4278263961 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f227c0309b0 con 0x7f22940aff80 2023-08-09T13:36:30.248 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:30.247+0000 7f22927fc640 1 -- 172.21.15.43:0/4278263961 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f227c030d40 con 0x7f22940aff80 2023-08-09T13:36:30.249 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:30.247+0000 7f22989d2640 1 -- 172.21.15.43:0/4278263961 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f22940aff80 msgr2=0x7f22940c9780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:36:30.249 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:30.247+0000 7f22989d2640 1 --2- 172.21.15.43:0/4278263961 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f22940aff80 0x7f22940c9780 secure :-1 s=READY pgs=612 cs=0 l=1 rev1=1 crypto rx=0x7f227c009870 tx=0x7f227c02b990 comp rx=0 tx=0).stop 2023-08-09T13:36:30.249 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:30.247+0000 7f22989d2640 1 -- 172.21.15.43:0/4278263961 shutdown_connections 2023-08-09T13:36:30.250 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:30.247+0000 7f22989d2640 1 --2- 172.21.15.43:0/4278263961 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f22940aff80 0x7f22940c9780 unknown :-1 s=CLOSED pgs=612 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:36:30.250 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:30.247+0000 7f22989d2640 1 -- 172.21.15.43:0/4278263961 >> 172.21.15.43:0/4278263961 conn(0x7f22940ca870 msgr2=0x7f22940cac50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-09T13:36:30.250 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:30.247+0000 7f22989d2640 1 -- 172.21.15.43:0/4278263961 shutdown_connections 2023-08-09T13:36:30.250 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:30.248+0000 7f22989d2640 1 -- 172.21.15.43:0/4278263961 wait complete. 2023-08-09T13:36:30.251 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:30.249+0000 7f22989d2640 1 Processor -- start 2023-08-09T13:36:30.251 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:30.249+0000 7f22989d2640 1 -- start start 2023-08-09T13:36:30.252 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:30.250+0000 7f22989d2640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f22940aff80 0x7f22940add40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:36:30.252 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:30.250+0000 7f22989d2640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f229415a330 0x7f2294150460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:36:30.252 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:30.250+0000 7f22989d2640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f229415ea60 0x7f22941509a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:36:30.252 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:30.250+0000 7f22989d2640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2294150040 con 0x7f22940aff80 2023-08-09T13:36:30.253 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:30.250+0000 7f22989d2640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2294140c40 con 0x7f229415a330 2023-08-09T13:36:30.253 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:30.250+0000 7f22989d2640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f229413fe80 con 0x7f229415ea60 2023-08-09T13:36:30.254 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:30.251+0000 7f22937fe640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f22940aff80 0x7f22940add40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:36:30.254 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:30.251+0000 7f22937fe640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f22940aff80 0x7f22940add40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:49416/0 (socket says 172.21.15.43:49416) 2023-08-09T13:36:30.254 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:30.251+0000 7f22937fe640 1 -- 172.21.15.43:0/3477936113 learned_addr learned my addr 172.21.15.43:0/3477936113 (peer_addr_for_me v2:172.21.15.43:0/0) 2023-08-09T13:36:30.254 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:30.251+0000 7f22937fe640 1 -- 172.21.15.43:0/3477936113 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f229415ea60 msgr2=0x7f22941509a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:36:30.255 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:30.251+0000 7f22937fe640 1 --2- 172.21.15.43:0/3477936113 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f229415ea60 0x7f22941509a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:36:30.255 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:30.251+0000 7f22937fe640 1 -- 172.21.15.43:0/3477936113 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f229415a330 msgr2=0x7f2294150460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:36:30.255 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:30.251+0000 7f22937fe640 1 --2- 172.21.15.43:0/3477936113 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f229415a330 0x7f2294150460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:36:30.255 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:30.251+0000 7f22937fe640 1 -- 172.21.15.43:0/3477936113 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f227c0094d0 con 0x7f22940aff80 2023-08-09T13:36:30.255 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:30.251+0000 7f22937fe640 1 --2- 172.21.15.43:0/3477936113 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f22940aff80 0x7f22940add40 secure :-1 s=READY pgs=613 cs=0 l=1 rev1=1 crypto rx=0x7f228400c430 tx=0x7f228400c8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-09T13:36:30.255 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:30.252+0000 7f228bfff640 1 -- 172.21.15.43:0/3477936113 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2284011790 con 0x7f22940aff80 2023-08-09T13:36:30.255 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:30.252+0000 7f228bfff640 1 -- 172.21.15.43:0/3477936113 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f22840118f0 con 0x7f22940aff80 2023-08-09T13:36:30.255 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:30.252+0000 7f228bfff640 1 -- 172.21.15.43:0/3477936113 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2284011ce0 con 0x7f22940aff80 2023-08-09T13:36:30.256 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:30.252+0000 7f22989d2640 1 -- 172.21.15.43:0/3477936113 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2294140160 con 0x7f22940aff80 2023-08-09T13:36:30.256 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:30.252+0000 7f22989d2640 1 -- 172.21.15.43:0/3477936113 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2294140720 con 0x7f22940aff80 2023-08-09T13:36:30.256 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:30.253+0000 7f22989d2640 1 -- 172.21.15.43:0/3477936113 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f22940739f0 con 0x7f22940aff80 2023-08-09T13:36:30.257 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:30.254+0000 7f228bfff640 1 -- 172.21.15.43:0/3477936113 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 9) v1 ==== 88461+0+0 (secure 0 0 0) 0x7f228401c460 con 0x7f22940aff80 2023-08-09T13:36:30.257 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:30.255+0000 7f228bfff640 1 --2- 172.21.15.43:0/3477936113 >> [v2:172.21.15.203:6832/998005540,v1:172.21.15.203:6833/998005540] conn(0x7f227406a1f0 0x7f227406c6b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:36:30.257 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:30.255+0000 7f228bfff640 1 -- 172.21.15.43:0/3477936113 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 6640+0+0 (secure 0 0 0) 0x7f2284088940 con 0x7f22940aff80 2023-08-09T13:36:30.257 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:30.256+0000 7f2292ffd640 1 --2- 172.21.15.43:0/3477936113 >> [v2:172.21.15.203:6832/998005540,v1:172.21.15.203:6833/998005540] conn(0x7f227406a1f0 0x7f227406c6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:36:30.258 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:30.257+0000 7f2292ffd640 1 --2- 172.21.15.43:0/3477936113 >> [v2:172.21.15.203:6832/998005540,v1:172.21.15.203:6833/998005540] conn(0x7f227406a1f0 0x7f227406c6b0 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7f227c036000 tx=0x7f227c03b040 comp rx=0 tx=0).ready entity=mgr.5003 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-09T13:36:30.261 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:30.260+0000 7f228bfff640 1 -- 172.21.15.43:0/3477936113 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178066 (secure 0 0 0) 0x7f2284053ce0 con 0x7f22940aff80 2023-08-09T13:36:30.475 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:30.473+0000 7f22989d2640 1 -- 172.21.15.43:0/3477936113 --> [v2:172.21.15.203:6832/998005540,v1:172.21.15.203:6833/998005540] -- mgr_command(tid 0: {"prefix": "fs snap-schedule add", "fs": "cephfs", "path": "/snap_test_dir1/dir2", "snap_schedule": "1M", "target": ["mon-mgr", ""]}) v1 -- 0x7f229415e540 con 0x7f227406a1f0 2023-08-09T13:36:30.644 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:30.642+0000 7f228bfff640 1 -- 172.21.15.43:0/3477936113 <== mgr.5003 v2:172.21.15.203:6832/998005540 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+43 (secure 0 0 0) 0x7f227c030ba0 con 0x7f227406a1f0 2023-08-09T13:36:30.645 INFO:teuthology.orchestra.run.smithi043.stdout:Schedule set for path /snap_test_dir1/dir2 2023-08-09T13:36:30.647 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:30.645+0000 7f22989d2640 1 -- 172.21.15.43:0/3477936113 >> [v2:172.21.15.203:6832/998005540,v1:172.21.15.203:6833/998005540] conn(0x7f227406a1f0 msgr2=0x7f227406c6b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:36:30.647 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:30.645+0000 7f22989d2640 1 --2- 172.21.15.43:0/3477936113 >> [v2:172.21.15.203:6832/998005540,v1:172.21.15.203:6833/998005540] conn(0x7f227406a1f0 0x7f227406c6b0 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7f227c036000 tx=0x7f227c03b040 comp rx=0 tx=0).stop 2023-08-09T13:36:30.647 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:30.645+0000 7f22989d2640 1 -- 172.21.15.43:0/3477936113 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f22940aff80 msgr2=0x7f22940add40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:36:30.648 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:30.645+0000 7f22989d2640 1 --2- 172.21.15.43:0/3477936113 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f22940aff80 0x7f22940add40 secure :-1 s=READY pgs=613 cs=0 l=1 rev1=1 crypto rx=0x7f228400c430 tx=0x7f228400c8f0 comp rx=0 tx=0).stop 2023-08-09T13:36:30.648 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:30.645+0000 7f22989d2640 1 -- 172.21.15.43:0/3477936113 shutdown_connections 2023-08-09T13:36:30.648 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:30.645+0000 7f22989d2640 1 --2- 172.21.15.43:0/3477936113 >> [v2:172.21.15.203:6832/998005540,v1:172.21.15.203:6833/998005540] conn(0x7f227406a1f0 0x7f227406c6b0 unknown :-1 s=CLOSED pgs=25 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:36:30.648 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:30.645+0000 7f22989d2640 1 --2- 172.21.15.43:0/3477936113 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f229415ea60 0x7f22941509a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:36:30.648 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:30.645+0000 7f22989d2640 1 --2- 172.21.15.43:0/3477936113 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f229415a330 0x7f2294150460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:36:30.648 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:30.645+0000 7f22989d2640 1 --2- 172.21.15.43:0/3477936113 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f22940aff80 0x7f22940add40 unknown :-1 s=CLOSED pgs=613 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:36:30.648 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:30.645+0000 7f22989d2640 1 -- 172.21.15.43:0/3477936113 >> 172.21.15.43:0/3477936113 conn(0x7f22940ca870 msgr2=0x7f2294158060 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-09T13:36:30.649 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:30.646+0000 7f22989d2640 1 -- 172.21.15.43:0/3477936113 shutdown_connections 2023-08-09T13:36:30.649 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:30.646+0000 7f22989d2640 1 -- 172.21.15.43:0/3477936113 wait complete. 2023-08-09T13:36:30.669 DEBUG:tasks.cephfs.test_snap_schedules:res=Schedule set for path /snap_test_dir1/dir2 2023-08-09T13:36:30.669 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-09T13:36:30.670 DEBUG:teuthology.orchestra.run.smithi043:> (cd /home/ubuntu/cephtest/mnt.0 && exec mkdir -p snap_test_dir1/dir3) 2023-08-09T13:36:30.729 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs snap-schedule add --fs cephfs /snap_test_dir1/dir3 1M 2023-08-09T13:36:30.949 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:30.948+0000 7fb3badab640 1 Processor -- start 2023-08-09T13:36:30.950 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:30.948+0000 7fb3badab640 1 -- start start 2023-08-09T13:36:30.951 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:30.950+0000 7fb3badab640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7fb3b40cc240 0x7fb3b406a8d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:36:30.951 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:30.950+0000 7fb3badab640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fb3b4160d30 0x7fb3b4160120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:36:30.952 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:30.950+0000 7fb3badab640 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7fb3b40cf560 0x7fb3b4160660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:36:30.952 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:30.950+0000 7fb3badab640 1 -- --> v2:172.21.15.43:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb3b40cf0d0 con 0x7fb3b40cc240 2023-08-09T13:36:30.952 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:30.950+0000 7fb3badab640 1 -- --> v2:172.21.15.203:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb3b40ce7a0 con 0x7fb3b40cf560 2023-08-09T13:36:30.952 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:30.950+0000 7fb3badab640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb3b40cea80 con 0x7fb3b4160d30 2023-08-09T13:36:30.953 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:30.950+0000 7fb3b8b47640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7fb3b40cc240 0x7fb3b406a8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:36:30.953 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:30.950+0000 7fb3b8b47640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7fb3b40cc240 0x7fb3b406a8d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:49430/0 (socket says 172.21.15.43:49430) 2023-08-09T13:36:30.953 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:30.950+0000 7fb3b8b47640 1 -- 172.21.15.43:0/747979469 learned_addr learned my addr 172.21.15.43:0/747979469 (peer_addr_for_me v2:172.21.15.43:0/0) 2023-08-09T13:36:30.953 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:30.951+0000 7fb3b8b47640 1 -- 172.21.15.43:0/747979469 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fb3b4160d30 msgr2=0x7fb3b4160120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-09T13:36:30.953 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:30.951+0000 7fb3b8b47640 1 --2- 172.21.15.43:0/747979469 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fb3b4160d30 0x7fb3b4160120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:36:30.954 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:30.951+0000 7fb3b8b47640 1 -- 172.21.15.43:0/747979469 >> v2:172.21.15.203:3300/0 conn(0x7fb3b40cf560 msgr2=0x7fb3b4160660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-09T13:36:30.954 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:30.951+0000 7fb3b9348640 1 --2- 172.21.15.43:0/747979469 >> v2:172.21.15.203:3300/0 conn(0x7fb3b40cf560 0x7fb3b4160660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:36:30.954 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:30.951+0000 7fb3b8b47640 1 --2- 172.21.15.43:0/747979469 >> v2:172.21.15.203:3300/0 conn(0x7fb3b40cf560 0x7fb3b4160660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:36:30.954 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:30.951+0000 7fb3b8b47640 1 -- 172.21.15.43:0/747979469 --> v2:172.21.15.43:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb3b40ced60 con 0x7fb3b40cc240 2023-08-09T13:36:30.954 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:30.951+0000 7fb3b9348640 1 --2- 172.21.15.43:0/747979469 >> v2:172.21.15.203:3300/0 conn(0x7fb3b40cf560 0x7fb3b4160660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-08-09T13:36:30.955 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:30.952+0000 7fb3b8b47640 1 --2- 172.21.15.43:0/747979469 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb3b40cc240 0x7fb3b406a8d0 secure :-1 s=READY pgs=614 cs=0 l=1 rev1=1 crypto rx=0x7fb3a4009ea0 tx=0x7fb3a402bf20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=96716c8c8757abf9 server_cookie=0 in_seq=0 out_seq=0 2023-08-09T13:36:30.955 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:30.952+0000 7fb3b37fe640 1 -- 172.21.15.43:0/747979469 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb3a402cac0 con 0x7fb3b40cc240 2023-08-09T13:36:30.955 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:30.952+0000 7fb3b37fe640 1 -- 172.21.15.43:0/747979469 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb3a402cc20 con 0x7fb3b40cc240 2023-08-09T13:36:30.955 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:30.952+0000 7fb3b37fe640 1 -- 172.21.15.43:0/747979469 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb3a40343f0 con 0x7fb3b40cc240 2023-08-09T13:36:30.956 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:30.953+0000 7fb3badab640 1 -- 172.21.15.43:0/747979469 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb3b40cc240 msgr2=0x7fb3b406a8d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:36:30.956 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:30.953+0000 7fb3badab640 1 --2- 172.21.15.43:0/747979469 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb3b40cc240 0x7fb3b406a8d0 secure :-1 s=READY pgs=614 cs=0 l=1 rev1=1 crypto rx=0x7fb3a4009ea0 tx=0x7fb3a402bf20 comp rx=0 tx=0).stop 2023-08-09T13:36:30.956 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:30.953+0000 7fb3badab640 1 -- 172.21.15.43:0/747979469 shutdown_connections 2023-08-09T13:36:30.956 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:30.953+0000 7fb3badab640 1 --2- 172.21.15.43:0/747979469 >> v2:172.21.15.203:3300/0 conn(0x7fb3b40cf560 0x7fb3b4160660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:36:30.956 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:30.953+0000 7fb3badab640 1 --2- 172.21.15.43:0/747979469 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fb3b4160d30 0x7fb3b4160120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:36:30.956 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:30.953+0000 7fb3badab640 1 --2- 172.21.15.43:0/747979469 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb3b40cc240 0x7fb3b406a8d0 unknown :-1 s=CLOSED pgs=614 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:36:30.956 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:30.953+0000 7fb3badab640 1 -- 172.21.15.43:0/747979469 >> 172.21.15.43:0/747979469 conn(0x7fb3b4099600 msgr2=0x7fb3b40d02c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-09T13:36:30.957 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:30.953+0000 7fb3badab640 1 -- 172.21.15.43:0/747979469 shutdown_connections 2023-08-09T13:36:30.957 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:30.953+0000 7fb3badab640 1 -- 172.21.15.43:0/747979469 wait complete. 2023-08-09T13:36:30.957 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:30.955+0000 7fb3badab640 1 Processor -- start 2023-08-09T13:36:30.957 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:30.955+0000 7fb3badab640 1 -- start start 2023-08-09T13:36:30.958 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:30.956+0000 7fb3badab640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb3b40cc240 0x7fb3b41f27c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:36:30.958 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:30.956+0000 7fb3badab640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fb3b40cf560 0x7fb3b41f2ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:36:30.958 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:30.956+0000 7fb3badab640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fb3b4160d30 0x7fb3b40b2ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:36:30.959 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:30.956+0000 7fb3badab640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb3b40b1db0 con 0x7fb3b40cc240 2023-08-09T13:36:30.959 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:30.956+0000 7fb3badab640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb3b41f2d00 con 0x7fb3b40cf560 2023-08-09T13:36:30.959 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:30.956+0000 7fb3badab640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb3b40c5e50 con 0x7fb3b4160d30 2023-08-09T13:36:30.959 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:30.957+0000 7fb3b8b47640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb3b40cc240 0x7fb3b41f27c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:36:30.959 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:30.957+0000 7fb3b8b47640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb3b40cc240 0x7fb3b41f27c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:49438/0 (socket says 172.21.15.43:49438) 2023-08-09T13:36:30.959 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:30.957+0000 7fb3b8b47640 1 -- 172.21.15.43:0/290732639 learned_addr learned my addr 172.21.15.43:0/290732639 (peer_addr_for_me v2:172.21.15.43:0/0) 2023-08-09T13:36:30.960 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:30.957+0000 7fb3b9348640 1 --2- 172.21.15.43:0/290732639 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fb3b4160d30 0x7fb3b40b2ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:36:30.961 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:30.957+0000 7fb3b3fff640 1 --2- 172.21.15.43:0/290732639 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fb3b40cf560 0x7fb3b41f2ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:36:30.961 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:30.957+0000 7fb3b8b47640 1 -- 172.21.15.43:0/290732639 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fb3b4160d30 msgr2=0x7fb3b40b2ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:36:30.961 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:30.957+0000 7fb3b8b47640 1 --2- 172.21.15.43:0/290732639 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fb3b4160d30 0x7fb3b40b2ae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:36:30.961 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:30.957+0000 7fb3b8b47640 1 -- 172.21.15.43:0/290732639 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fb3b40cf560 msgr2=0x7fb3b41f2ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:36:30.961 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:30.957+0000 7fb3b8b47640 1 --2- 172.21.15.43:0/290732639 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fb3b40cf560 0x7fb3b41f2ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:36:30.961 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:30.957+0000 7fb3b8b47640 1 -- 172.21.15.43:0/290732639 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb3a4009b30 con 0x7fb3b40cc240 2023-08-09T13:36:30.961 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:30.957+0000 7fb3b9348640 1 --2- 172.21.15.43:0/290732639 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fb3b4160d30 0x7fb3b40b2ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-08-09T13:36:30.962 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:30.958+0000 7fb3b8b47640 1 --2- 172.21.15.43:0/290732639 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb3b40cc240 0x7fb3b41f27c0 secure :-1 s=READY pgs=615 cs=0 l=1 rev1=1 crypto rx=0x7fb3a4009ea0 tx=0x7fb3a4004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-09T13:36:30.962 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:30.958+0000 7fb3b17fa640 1 -- 172.21.15.43:0/290732639 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb3a403e450 con 0x7fb3b40cc240 2023-08-09T13:36:30.962 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:30.958+0000 7fb3b17fa640 1 -- 172.21.15.43:0/290732639 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb3a403e5f0 con 0x7fb3b40cc240 2023-08-09T13:36:30.962 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:30.958+0000 7fb3b17fa640 1 -- 172.21.15.43:0/290732639 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb3a403e9a0 con 0x7fb3b40cc240 2023-08-09T13:36:30.962 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:30.958+0000 7fb3badab640 1 -- 172.21.15.43:0/290732639 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb3b40cc7a0 con 0x7fb3b40cc240 2023-08-09T13:36:30.962 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:30.958+0000 7fb3badab640 1 -- 172.21.15.43:0/290732639 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb3b40b22e0 con 0x7fb3b40cc240 2023-08-09T13:36:30.963 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:30.959+0000 7fb3badab640 1 -- 172.21.15.43:0/290732639 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb3b40739f0 con 0x7fb3b40cc240 2023-08-09T13:36:30.963 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:30.960+0000 7fb3b17fa640 1 -- 172.21.15.43:0/290732639 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 9) v1 ==== 88461+0+0 (secure 0 0 0) 0x7fb3a403eb00 con 0x7fb3b40cc240 2023-08-09T13:36:30.963 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:30.961+0000 7fb3b17fa640 1 --2- 172.21.15.43:0/290732639 >> [v2:172.21.15.203:6832/998005540,v1:172.21.15.203:6833/998005540] conn(0x7fb3a006a1f0 0x7fb3a006c6b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:36:30.963 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:30.961+0000 7fb3b17fa640 1 -- 172.21.15.43:0/290732639 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 6640+0+0 (secure 0 0 0) 0x7fb3a40ab670 con 0x7fb3b40cc240 2023-08-09T13:36:30.964 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:30.962+0000 7fb3b3fff640 1 --2- 172.21.15.43:0/290732639 >> [v2:172.21.15.203:6832/998005540,v1:172.21.15.203:6833/998005540] conn(0x7fb3a006a1f0 0x7fb3a006c6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:36:30.964 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:30.963+0000 7fb3b3fff640 1 --2- 172.21.15.43:0/290732639 >> [v2:172.21.15.203:6832/998005540,v1:172.21.15.203:6833/998005540] conn(0x7fb3a006a1f0 0x7fb3a006c6b0 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7fb3b40b17d0 tx=0x7fb3a8008040 comp rx=0 tx=0).ready entity=mgr.5003 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-09T13:36:30.967 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:30.966+0000 7fb3b17fa640 1 -- 172.21.15.43:0/290732639 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178066 (secure 0 0 0) 0x7fb3a4076990 con 0x7fb3b40cc240 2023-08-09T13:36:31.191 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:31.189+0000 7fb3badab640 1 -- 172.21.15.43:0/290732639 --> [v2:172.21.15.203:6832/998005540,v1:172.21.15.203:6833/998005540] -- mgr_command(tid 0: {"prefix": "fs snap-schedule add", "fs": "cephfs", "path": "/snap_test_dir1/dir3", "snap_schedule": "1M", "target": ["mon-mgr", ""]}) v1 -- 0x7fb3b40cc620 con 0x7fb3a006a1f0 2023-08-09T13:36:31.348 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:31.347+0000 7fb3b17fa640 1 -- 172.21.15.43:0/290732639 <== mgr.5003 v2:172.21.15.203:6832/998005540 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+43 (secure 0 0 0) 0x7fb3a800bdc0 con 0x7fb3a006a1f0 2023-08-09T13:36:31.349 INFO:teuthology.orchestra.run.smithi043.stdout:Schedule set for path /snap_test_dir1/dir3 2023-08-09T13:36:31.352 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:31.350+0000 7fb3badab640 1 -- 172.21.15.43:0/290732639 >> [v2:172.21.15.203:6832/998005540,v1:172.21.15.203:6833/998005540] conn(0x7fb3a006a1f0 msgr2=0x7fb3a006c6b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:36:31.352 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:31.350+0000 7fb3badab640 1 --2- 172.21.15.43:0/290732639 >> [v2:172.21.15.203:6832/998005540,v1:172.21.15.203:6833/998005540] conn(0x7fb3a006a1f0 0x7fb3a006c6b0 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7fb3b40b17d0 tx=0x7fb3a8008040 comp rx=0 tx=0).stop 2023-08-09T13:36:31.352 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:31.350+0000 7fb3badab640 1 -- 172.21.15.43:0/290732639 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb3b40cc240 msgr2=0x7fb3b41f27c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:36:31.352 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:31.350+0000 7fb3badab640 1 --2- 172.21.15.43:0/290732639 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb3b40cc240 0x7fb3b41f27c0 secure :-1 s=READY pgs=615 cs=0 l=1 rev1=1 crypto rx=0x7fb3a4009ea0 tx=0x7fb3a4004a40 comp rx=0 tx=0).stop 2023-08-09T13:36:31.353 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:31.350+0000 7fb3badab640 1 -- 172.21.15.43:0/290732639 shutdown_connections 2023-08-09T13:36:31.353 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:31.350+0000 7fb3badab640 1 --2- 172.21.15.43:0/290732639 >> [v2:172.21.15.203:6832/998005540,v1:172.21.15.203:6833/998005540] conn(0x7fb3a006a1f0 0x7fb3a006c6b0 unknown :-1 s=CLOSED pgs=26 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:36:31.353 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:31.350+0000 7fb3badab640 1 --2- 172.21.15.43:0/290732639 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fb3b4160d30 0x7fb3b40b2ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:36:31.353 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:31.350+0000 7fb3badab640 1 --2- 172.21.15.43:0/290732639 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fb3b40cf560 0x7fb3b41f2ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:36:31.353 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:31.350+0000 7fb3badab640 1 --2- 172.21.15.43:0/290732639 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb3b40cc240 0x7fb3b41f27c0 unknown :-1 s=CLOSED pgs=615 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:36:31.353 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:31.350+0000 7fb3badab640 1 -- 172.21.15.43:0/290732639 >> 172.21.15.43:0/290732639 conn(0x7fb3b4099600 msgr2=0x7fb3b406f140 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-09T13:36:31.353 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:31.351+0000 7fb3badab640 1 -- 172.21.15.43:0/290732639 shutdown_connections 2023-08-09T13:36:31.354 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:31.351+0000 7fb3badab640 1 -- 172.21.15.43:0/290732639 wait complete. 2023-08-09T13:36:31.373 DEBUG:tasks.cephfs.test_snap_schedules:res=Schedule set for path /snap_test_dir1/dir3 2023-08-09T13:36:31.373 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-09T13:36:31.374 DEBUG:teuthology.orchestra.run.smithi043:> (cd /home/ubuntu/cephtest/mnt.0 && exec mkdir -p snap_test_dir1/dir4) 2023-08-09T13:36:31.438 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs snap-schedule add --fs cephfs /snap_test_dir1/dir4 1M 2023-08-09T13:36:31.674 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:31.672+0000 7f22cdebf640 1 Processor -- start 2023-08-09T13:36:31.674 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:31.672+0000 7f22cdebf640 1 -- start start 2023-08-09T13:36:31.675 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:31.674+0000 7f22cdebf640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f22c8069a30 0x7f22c8159760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:36:31.676 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:31.674+0000 7f22cdebf640 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7f22c815da10 0x7f22c80ce4c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:36:31.676 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:31.674+0000 7f22cdebf640 1 -- --> v1:172.21.15.203:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f22c80ca410 con 0x7f22c815d5b0 2023-08-09T13:36:31.676 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:31.674+0000 7f22cdebf640 1 -- --> v2:172.21.15.203:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f22c815ce80 con 0x7f22c815da10 2023-08-09T13:36:31.676 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:31.674+0000 7f22cdebf640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f22c815a050 con 0x7f22c8069a30 2023-08-09T13:36:31.677 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:31.675+0000 7f22c77fe640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f22c8069a30 0x7f22c8159760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:36:31.677 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:31.675+0000 7f22c77fe640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f22c8069a30 0x7f22c8159760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.203:3301/0 says I am v2:172.21.15.43:45048/0 (socket says 172.21.15.43:45048) 2023-08-09T13:36:31.677 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:31.675+0000 7f22c7fff640 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7f22c815da10 0x7f22c80ce4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:36:31.677 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:31.675+0000 7f22c77fe640 1 -- 172.21.15.43:0/749836889 learned_addr learned my addr 172.21.15.43:0/749836889 (peer_addr_for_me v2:172.21.15.43:0/0) 2023-08-09T13:36:31.678 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:31.676+0000 7f22c7fff640 1 -- 172.21.15.43:0/749836889 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f22c8069a30 msgr2=0x7f22c8159760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-09T13:36:31.678 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:31.676+0000 7f22c7fff640 1 --2- 172.21.15.43:0/749836889 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f22c8069a30 0x7f22c8159760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:36:31.678 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:31.676+0000 7f22c7fff640 1 -- 172.21.15.43:0/749836889 >> v1:172.21.15.203:6789/0 conn(0x7f22c815d5b0 legacy=0x7f22c8069e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:36:31.678 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:31.676+0000 7f22c7fff640 1 -- 172.21.15.43:0/749836889 --> v2:172.21.15.203:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f22c815a330 con 0x7f22c815da10 2023-08-09T13:36:31.679 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:31.678+0000 7f22c7fff640 1 --2- 172.21.15.43:0/749836889 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f22c815da10 0x7f22c80ce4c0 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7f22c00097a0 tx=0x7f22c002b820 comp rx=0 tx=0).ready entity=mon.1 client_cookie=9a74a82ad7c90768 server_cookie=0 in_seq=0 out_seq=0 2023-08-09T13:36:31.680 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:31.679+0000 7f22c6ffd640 1 -- 172.21.15.43:0/749836889 <== mon.1 v2:172.21.15.203:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f22c0031780 con 0x7f22c815da10 2023-08-09T13:36:31.680 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:31.679+0000 7f22c6ffd640 1 -- 172.21.15.43:0/749836889 <== mon.1 v2:172.21.15.203:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f22c00318e0 con 0x7f22c815da10 2023-08-09T13:36:31.681 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:31.679+0000 7f22c6ffd640 1 -- 172.21.15.43:0/749836889 <== mon.1 v2:172.21.15.203:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f22c0031c70 con 0x7f22c815da10 2023-08-09T13:36:31.681 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:31.679+0000 7f22cdebf640 1 -- 172.21.15.43:0/749836889 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f22c815da10 msgr2=0x7f22c80ce4c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:36:31.681 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:31.679+0000 7f22cdebf640 1 --2- 172.21.15.43:0/749836889 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f22c815da10 0x7f22c80ce4c0 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7f22c00097a0 tx=0x7f22c002b820 comp rx=0 tx=0).stop 2023-08-09T13:36:31.682 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:31.680+0000 7f22cdebf640 1 -- 172.21.15.43:0/749836889 shutdown_connections 2023-08-09T13:36:31.682 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:31.680+0000 7f22cdebf640 1 --2- 172.21.15.43:0/749836889 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f22c815da10 0x7f22c80ce4c0 unknown :-1 s=CLOSED pgs=238 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:36:31.682 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:31.680+0000 7f22cdebf640 1 --2- 172.21.15.43:0/749836889 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f22c8069a30 0x7f22c8159760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:36:31.682 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:31.680+0000 7f22cdebf640 1 -- 172.21.15.43:0/749836889 >> 172.21.15.43:0/749836889 conn(0x7f22c806b990 msgr2=0x7f22c806bda0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-09T13:36:31.682 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:31.680+0000 7f22cdebf640 1 -- 172.21.15.43:0/749836889 shutdown_connections 2023-08-09T13:36:31.682 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:31.680+0000 7f22cdebf640 1 -- 172.21.15.43:0/749836889 wait complete. 2023-08-09T13:36:31.683 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:31.682+0000 7f22cdebf640 1 Processor -- start 2023-08-09T13:36:31.683 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:31.682+0000 7f22cdebf640 1 -- start start 2023-08-09T13:36:31.684 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:31.683+0000 7f22cdebf640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f22c8069a30 0x7f22c80a9690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:36:31.685 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:31.683+0000 7f22cdebf640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f22c815d5b0 0x7f22c81f53b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:36:31.685 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:31.683+0000 7f22cdebf640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f22c815da10 0x7f22c81f58f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:36:31.685 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:31.683+0000 7f22cdebf640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f22c80a9bd0 con 0x7f22c815da10 2023-08-09T13:36:31.685 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:31.683+0000 7f22cdebf640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f22c80a9de0 con 0x7f22c815d5b0 2023-08-09T13:36:31.685 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:31.683+0000 7f22cdebf640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f22c80aa0c0 con 0x7f22c8069a30 2023-08-09T13:36:31.685 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:31.683+0000 7f22c7fff640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f22c815da10 0x7f22c81f58f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:36:31.686 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:31.683+0000 7f22c7fff640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f22c815da10 0x7f22c81f58f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:49446/0 (socket says 172.21.15.43:49446) 2023-08-09T13:36:31.686 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:31.683+0000 7f22c7fff640 1 -- 172.21.15.43:0/3426000335 learned_addr learned my addr 172.21.15.43:0/3426000335 (peer_addr_for_me v2:172.21.15.43:0/0) 2023-08-09T13:36:31.687 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:31.684+0000 7f22c77fe640 1 --2- 172.21.15.43:0/3426000335 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f22c8069a30 0x7f22c80a9690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:36:31.687 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:31.684+0000 7f22beffd640 1 --2- 172.21.15.43:0/3426000335 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f22c815d5b0 0x7f22c81f53b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:36:31.687 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:31.684+0000 7f22c7fff640 1 -- 172.21.15.43:0/3426000335 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f22c8069a30 msgr2=0x7f22c80a9690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:36:31.687 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:31.684+0000 7f22c7fff640 1 --2- 172.21.15.43:0/3426000335 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f22c8069a30 0x7f22c80a9690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:36:31.687 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:31.684+0000 7f22c7fff640 1 -- 172.21.15.43:0/3426000335 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f22c815d5b0 msgr2=0x7f22c81f53b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:36:31.687 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:31.684+0000 7f22c7fff640 1 --2- 172.21.15.43:0/3426000335 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f22c815d5b0 0x7f22c81f53b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:36:31.687 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:31.684+0000 7f22c7fff640 1 -- 172.21.15.43:0/3426000335 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f22c0009400 con 0x7f22c815da10 2023-08-09T13:36:31.688 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:31.684+0000 7f22c7fff640 1 --2- 172.21.15.43:0/3426000335 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f22c815da10 0x7f22c81f58f0 secure :-1 s=READY pgs=616 cs=0 l=1 rev1=1 crypto rx=0x7f22c0034040 tx=0x7f22c00046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-09T13:36:31.688 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:31.684+0000 7f22c4ff9640 1 -- 172.21.15.43:0/3426000335 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f22c0032820 con 0x7f22c815da10 2023-08-09T13:36:31.688 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:31.685+0000 7f22c4ff9640 1 -- 172.21.15.43:0/3426000335 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f22c0032980 con 0x7f22c815da10 2023-08-09T13:36:31.688 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:31.685+0000 7f22c4ff9640 1 -- 172.21.15.43:0/3426000335 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f22c0032d30 con 0x7f22c815da10 2023-08-09T13:36:31.688 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:31.685+0000 7f22cdebf640 1 -- 172.21.15.43:0/3426000335 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f22c80aa3a0 con 0x7f22c815da10 2023-08-09T13:36:31.688 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:31.685+0000 7f22cdebf640 1 -- 172.21.15.43:0/3426000335 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f22c80aa960 con 0x7f22c815da10 2023-08-09T13:36:31.689 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:31.686+0000 7f22cdebf640 1 -- 172.21.15.43:0/3426000335 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f22c806bd70 con 0x7f22c815da10 2023-08-09T13:36:31.689 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:31.687+0000 7f22c4ff9640 1 -- 172.21.15.43:0/3426000335 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 9) v1 ==== 88461+0+0 (secure 0 0 0) 0x7f22c004e450 con 0x7f22c815da10 2023-08-09T13:36:31.689 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:31.688+0000 7f22c4ff9640 1 --2- 172.21.15.43:0/3426000335 >> [v2:172.21.15.203:6832/998005540,v1:172.21.15.203:6833/998005540] conn(0x7f22a806a1c0 0x7f22a806c680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:36:31.689 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:31.688+0000 7f22c4ff9640 1 -- 172.21.15.43:0/3426000335 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 6640+0+0 (secure 0 0 0) 0x7f22c00aab20 con 0x7f22c815da10 2023-08-09T13:36:31.690 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:31.688+0000 7f22c77fe640 1 --2- 172.21.15.43:0/3426000335 >> [v2:172.21.15.203:6832/998005540,v1:172.21.15.203:6833/998005540] conn(0x7f22a806a1c0 0x7f22a806c680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:36:31.691 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:31.690+0000 7f22c77fe640 1 --2- 172.21.15.43:0/3426000335 >> [v2:172.21.15.203:6832/998005540,v1:172.21.15.203:6833/998005540] conn(0x7f22a806a1c0 0x7f22a806c680 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7f22b8009ce0 tx=0x7f22b800b040 comp rx=0 tx=0).ready entity=mgr.5003 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-09T13:36:31.694 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:31.692+0000 7f22c4ff9640 1 -- 172.21.15.43:0/3426000335 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178066 (secure 0 0 0) 0x7f22c0075d30 con 0x7f22c815da10 2023-08-09T13:36:31.913 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:31.911+0000 7f22cdebf640 1 -- 172.21.15.43:0/3426000335 --> [v2:172.21.15.203:6832/998005540,v1:172.21.15.203:6833/998005540] -- mgr_command(tid 0: {"prefix": "fs snap-schedule add", "fs": "cephfs", "path": "/snap_test_dir1/dir4", "snap_schedule": "1M", "target": ["mon-mgr", ""]}) v1 -- 0x7f22c815d090 con 0x7f22a806a1c0 2023-08-09T13:36:32.070 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:32.068+0000 7f22c4ff9640 1 -- 172.21.15.43:0/3426000335 <== mgr.5003 v2:172.21.15.203:6832/998005540 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+43 (secure 0 0 0) 0x7f22b800f040 con 0x7f22a806a1c0 2023-08-09T13:36:32.070 INFO:teuthology.orchestra.run.smithi043.stdout:Schedule set for path /snap_test_dir1/dir4 2023-08-09T13:36:32.073 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:32.071+0000 7f22cdebf640 1 -- 172.21.15.43:0/3426000335 >> [v2:172.21.15.203:6832/998005540,v1:172.21.15.203:6833/998005540] conn(0x7f22a806a1c0 msgr2=0x7f22a806c680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:36:32.073 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:32.071+0000 7f22cdebf640 1 --2- 172.21.15.43:0/3426000335 >> [v2:172.21.15.203:6832/998005540,v1:172.21.15.203:6833/998005540] conn(0x7f22a806a1c0 0x7f22a806c680 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7f22b8009ce0 tx=0x7f22b800b040 comp rx=0 tx=0).stop 2023-08-09T13:36:32.073 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:32.071+0000 7f22cdebf640 1 -- 172.21.15.43:0/3426000335 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f22c815da10 msgr2=0x7f22c81f58f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:36:32.073 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:32.071+0000 7f22cdebf640 1 --2- 172.21.15.43:0/3426000335 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f22c815da10 0x7f22c81f58f0 secure :-1 s=READY pgs=616 cs=0 l=1 rev1=1 crypto rx=0x7f22c0034040 tx=0x7f22c00046c0 comp rx=0 tx=0).stop 2023-08-09T13:36:32.074 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:32.071+0000 7f22cdebf640 1 -- 172.21.15.43:0/3426000335 shutdown_connections 2023-08-09T13:36:32.074 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:32.071+0000 7f22cdebf640 1 --2- 172.21.15.43:0/3426000335 >> [v2:172.21.15.203:6832/998005540,v1:172.21.15.203:6833/998005540] conn(0x7f22a806a1c0 0x7f22a806c680 unknown :-1 s=CLOSED pgs=27 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:36:32.074 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:32.071+0000 7f22cdebf640 1 --2- 172.21.15.43:0/3426000335 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f22c815da10 0x7f22c81f58f0 unknown :-1 s=CLOSED pgs=616 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:36:32.074 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:32.071+0000 7f22cdebf640 1 --2- 172.21.15.43:0/3426000335 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f22c815d5b0 0x7f22c81f53b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:36:32.075 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:32.071+0000 7f22cdebf640 1 --2- 172.21.15.43:0/3426000335 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f22c8069a30 0x7f22c80a9690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:36:32.075 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:32.071+0000 7f22cdebf640 1 -- 172.21.15.43:0/3426000335 >> 172.21.15.43:0/3426000335 conn(0x7f22c806b990 msgr2=0x7f22c815c210 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-09T13:36:32.075 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:32.072+0000 7f22cdebf640 1 -- 172.21.15.43:0/3426000335 shutdown_connections 2023-08-09T13:36:32.075 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:32.072+0000 7f22cdebf640 1 -- 172.21.15.43:0/3426000335 wait complete. 2023-08-09T13:36:32.094 DEBUG:tasks.cephfs.test_snap_schedules:res=Schedule set for path /snap_test_dir1/dir4 2023-08-09T13:36:32.095 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-09T13:36:32.095 DEBUG:teuthology.orchestra.run.smithi043:> (cd /home/ubuntu/cephtest/mnt.0 && exec mkdir -p snap_test_dir1/dir5) 2023-08-09T13:36:32.155 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs snap-schedule add --fs cephfs /snap_test_dir1/dir5 1M 2023-08-09T13:36:32.390 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:32.388+0000 7f5afe6c5640 1 Processor -- start 2023-08-09T13:36:32.390 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:32.389+0000 7f5afe6c5640 1 -- start start 2023-08-09T13:36:32.392 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:32.390+0000 7f5afe6c5640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f5af80b80a0 0x7f5af80b5470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:36:32.392 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:32.390+0000 7f5afe6c5640 1 -- --> v1:172.21.15.43:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5af80d1890 con 0x7f5af80b59b0 2023-08-09T13:36:32.392 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:32.390+0000 7f5afe6c5640 1 -- --> v1:172.21.15.203:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5af80d0c80 con 0x7f5af815c6f0 2023-08-09T13:36:32.392 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:32.390+0000 7f5afe6c5640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5af815cad0 con 0x7f5af80b80a0 2023-08-09T13:36:32.393 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:32.391+0000 7f5afcc62640 1 --1- >> v1:172.21.15.43:6789/0 conn(0x7f5af80b59b0 0x7f5af80d1d20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.43:6789/0 says I am v1:172.21.15.43:33340/0 (socket says 172.21.15.43:33340) 2023-08-09T13:36:32.393 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:32.391+0000 7f5afcc62640 1 -- 172.21.15.43:0/156243514 learned_addr learned my addr 172.21.15.43:0/156243514 (peer_addr_for_me v1:172.21.15.43:0/0) 2023-08-09T13:36:32.394 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:32.391+0000 7f5af77fe640 1 --2- 172.21.15.43:0/156243514 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f5af80b80a0 0x7f5af80b5470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:36:32.394 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:32.391+0000 7f5af6ffd640 1 -- 172.21.15.43:0/156243514 <== mon.0 v1:172.21.15.43:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 889106887 0 0) 0x7f5af80d1890 con 0x7f5af80b59b0 2023-08-09T13:36:32.394 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:32.391+0000 7f5af6ffd640 1 -- 172.21.15.43:0/156243514 --> v1:172.21.15.43:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f5ae0003520 con 0x7f5af80b59b0 2023-08-09T13:36:32.394 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:32.392+0000 7f5af6ffd640 1 -- 172.21.15.43:0/156243514 <== mon.0 v1:172.21.15.43:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2572716721 0 0) 0x7f5ae0003520 con 0x7f5af80b59b0 2023-08-09T13:36:32.394 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:32.392+0000 7f5af6ffd640 1 -- 172.21.15.43:0/156243514 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f5af80b80a0 msgr2=0x7f5af80b5470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-09T13:36:32.394 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:32.392+0000 7f5af6ffd640 1 --2- 172.21.15.43:0/156243514 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f5af80b80a0 0x7f5af80b5470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:36:32.395 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:32.392+0000 7f5af6ffd640 1 -- 172.21.15.43:0/156243514 >> v1:172.21.15.203:6789/0 conn(0x7f5af815c6f0 legacy=0x7f5af80d03a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:36:32.395 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:32.392+0000 7f5af6ffd640 1 -- 172.21.15.43:0/156243514 --> v1:172.21.15.43:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5af80d0f60 con 0x7f5af80b59b0 2023-08-09T13:36:32.395 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:32.392+0000 7f5af6ffd640 1 -- 172.21.15.43:0/156243514 <== mon.0 v1:172.21.15.43:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1285150740 0 0) 0x7f5af0001670 con 0x7f5af80b59b0 2023-08-09T13:36:32.395 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:32.392+0000 7f5af6ffd640 1 -- 172.21.15.43:0/156243514 >> v1:172.21.15.43:6789/0 conn(0x7f5af80b59b0 legacy=0x7f5af80d1d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:36:32.395 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:32.392+0000 7f5af6ffd640 1 --2- 172.21.15.43:0/156243514 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f5af80b80a0 0x7f5ae0003a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:36:32.395 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:32.392+0000 7f5af6ffd640 1 --2- 172.21.15.43:0/156243514 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f5ae0003f90 0x7f5ae00047c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:36:32.395 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:32.392+0000 7f5af6ffd640 1 --2- 172.21.15.43:0/156243514 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f5ae0004d00 0x7f5ae0005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:36:32.397 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:32.392+0000 7f5af6ffd640 1 -- 172.21.15.43:0/156243514 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5af815cad0 con 0x7f5ae0004d00 2023-08-09T13:36:32.397 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:32.392+0000 7f5af6ffd640 1 -- 172.21.15.43:0/156243514 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5ae000bae0 con 0x7f5ae0003f90 2023-08-09T13:36:32.397 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:32.392+0000 7f5af6ffd640 1 -- 172.21.15.43:0/156243514 --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5ae000bca0 con 0x7f5af80b80a0 2023-08-09T13:36:32.398 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:32.392+0000 7f5afcc62640 1 --2- 172.21.15.43:0/156243514 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f5ae0004d00 0x7f5ae0005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:36:32.398 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:32.393+0000 7f5afcc62640 1 -- 172.21.15.43:0/156243514 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f5af80b80a0 msgr2=0x7f5ae0003a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-09T13:36:32.398 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:32.393+0000 7f5afcc62640 1 --2- 172.21.15.43:0/156243514 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f5af80b80a0 0x7f5ae0003a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:36:32.399 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:32.393+0000 7f5afcc62640 1 -- 172.21.15.43:0/156243514 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f5ae0003f90 msgr2=0x7f5ae00047c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-09T13:36:32.399 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:32.393+0000 7f5afcc62640 1 --2- 172.21.15.43:0/156243514 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f5ae0003f90 0x7f5ae00047c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:36:32.399 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:32.393+0000 7f5afcc62640 1 -- 172.21.15.43:0/156243514 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5ae0003520 con 0x7f5ae0004d00 2023-08-09T13:36:32.399 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:32.394+0000 7f5afcc62640 1 --2- 172.21.15.43:0/156243514 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f5ae0004d00 0x7f5ae0005180 secure :-1 s=READY pgs=618 cs=0 l=1 rev1=1 crypto rx=0x7f5af000c240 tx=0x7f5ae00057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=214f05d810d5dde7 server_cookie=0 in_seq=0 out_seq=0 2023-08-09T13:36:32.400 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:32.394+0000 7f5af6ffd640 1 -- 172.21.15.43:0/156243514 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5af0005ad0 con 0x7f5ae0004d00 2023-08-09T13:36:32.400 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:32.394+0000 7f5af6ffd640 1 -- 172.21.15.43:0/156243514 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5af0005c30 con 0x7f5ae0004d00 2023-08-09T13:36:32.400 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:32.394+0000 7f5af6ffd640 1 -- 172.21.15.43:0/156243514 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5af002caf0 con 0x7f5ae0004d00 2023-08-09T13:36:32.400 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:32.395+0000 7f5afe6c5640 1 -- 172.21.15.43:0/156243514 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f5ae0004d00 msgr2=0x7f5ae0005180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:36:32.401 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:32.395+0000 7f5afe6c5640 1 --2- 172.21.15.43:0/156243514 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f5ae0004d00 0x7f5ae0005180 secure :-1 s=READY pgs=618 cs=0 l=1 rev1=1 crypto rx=0x7f5af000c240 tx=0x7f5ae00057f0 comp rx=0 tx=0).stop 2023-08-09T13:36:32.401 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:32.395+0000 7f5afcc62640 1 -- 172.21.15.43:0/156243514 reap_dead start 2023-08-09T13:36:32.401 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:32.395+0000 7f5afe6c5640 1 -- 172.21.15.43:0/156243514 shutdown_connections 2023-08-09T13:36:32.402 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:32.395+0000 7f5afe6c5640 1 -- 172.21.15.43:0/156243514 >> 172.21.15.43:0/156243514 conn(0x7f5af80712e0 msgr2=0x7f5af80716f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-09T13:36:32.402 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:32.395+0000 7f5afe6c5640 1 -- 172.21.15.43:0/156243514 shutdown_connections 2023-08-09T13:36:32.402 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:32.395+0000 7f5afe6c5640 1 -- 172.21.15.43:0/156243514 wait complete. 2023-08-09T13:36:32.405 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:32.397+0000 7f5afe6c5640 1 Processor -- start 2023-08-09T13:36:32.405 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:32.397+0000 7f5afe6c5640 1 -- start start 2023-08-09T13:36:32.405 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:32.398+0000 7f5afe6c5640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f5ae0004d00 0x7f5af8152a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:36:32.405 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:32.398+0000 7f5afe6c5640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f5af81586c0 0x7f5af8151900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:36:32.405 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:32.398+0000 7f5afe6c5640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f5af81545b0 0x7f5af8154a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:36:32.405 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:32.398+0000 7f5afe6c5640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5af80cb1c0 con 0x7f5af81586c0 2023-08-09T13:36:32.406 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:32.398+0000 7f5afe6c5640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5af80cb4a0 con 0x7f5ae0004d00 2023-08-09T13:36:32.406 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:32.398+0000 7f5afe6c5640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5af80cb780 con 0x7f5af81545b0 2023-08-09T13:36:32.406 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:32.399+0000 7f5af77fe640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f5af81586c0 0x7f5af8151900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:36:32.406 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:32.399+0000 7f5af77fe640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f5af81586c0 0x7f5af8151900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:49468/0 (socket says 172.21.15.43:49468) 2023-08-09T13:36:32.406 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:32.399+0000 7f5af77fe640 1 -- 172.21.15.43:0/1962767843 learned_addr learned my addr 172.21.15.43:0/1962767843 (peer_addr_for_me v2:172.21.15.43:0/0) 2023-08-09T13:36:32.406 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:32.399+0000 7f5af7fff640 1 --2- 172.21.15.43:0/1962767843 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f5ae0004d00 0x7f5af8152a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:36:32.406 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:32.399+0000 7f5afcc62640 1 --2- 172.21.15.43:0/1962767843 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f5af81545b0 0x7f5af8154a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:36:32.406 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:32.399+0000 7f5af77fe640 1 -- 172.21.15.43:0/1962767843 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f5af81545b0 msgr2=0x7f5af8154a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:36:32.407 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:32.399+0000 7f5af77fe640 1 --2- 172.21.15.43:0/1962767843 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f5af81545b0 0x7f5af8154a30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:36:32.407 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:32.399+0000 7f5af77fe640 1 -- 172.21.15.43:0/1962767843 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f5ae0004d00 msgr2=0x7f5af8152a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:36:32.407 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:32.399+0000 7f5af77fe640 1 --2- 172.21.15.43:0/1962767843 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f5ae0004d00 0x7f5af8152a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:36:32.407 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:32.399+0000 7f5af77fe640 1 -- 172.21.15.43:0/1962767843 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5af80cba60 con 0x7f5af81586c0 2023-08-09T13:36:32.407 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:32.399+0000 7f5af77fe640 1 --2- 172.21.15.43:0/1962767843 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f5af81586c0 0x7f5af8151900 secure :-1 s=READY pgs=619 cs=0 l=1 rev1=1 crypto rx=0x7f5aec00e780 tx=0x7f5aec00ec40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-09T13:36:32.407 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:32.400+0000 7f5af4ff9640 1 -- 172.21.15.43:0/1962767843 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5aec00ee90 con 0x7f5af81586c0 2023-08-09T13:36:32.408 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:32.400+0000 7f5af4ff9640 1 -- 172.21.15.43:0/1962767843 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5aec00c940 con 0x7f5af81586c0 2023-08-09T13:36:32.408 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:32.400+0000 7f5af4ff9640 1 -- 172.21.15.43:0/1962767843 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5aec00ccf0 con 0x7f5af81586c0 2023-08-09T13:36:32.408 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:32.400+0000 7f5afe6c5640 1 -- 172.21.15.43:0/1962767843 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5af8151e40 con 0x7f5af81586c0 2023-08-09T13:36:32.408 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:32.400+0000 7f5afe6c5640 1 -- 172.21.15.43:0/1962767843 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5af8158420 con 0x7f5af81586c0 2023-08-09T13:36:32.408 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:32.401+0000 7f5afe6c5640 1 -- 172.21.15.43:0/1962767843 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5af80739f0 con 0x7f5af81586c0 2023-08-09T13:36:32.409 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:32.402+0000 7f5af4ff9640 1 -- 172.21.15.43:0/1962767843 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 9) v1 ==== 88461+0+0 (secure 0 0 0) 0x7f5aec013470 con 0x7f5af81586c0 2023-08-09T13:36:32.409 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:32.403+0000 7f5af4ff9640 1 --2- 172.21.15.43:0/1962767843 >> [v2:172.21.15.203:6832/998005540,v1:172.21.15.203:6833/998005540] conn(0x7f5ae406a1f0 0x7f5ae406c6b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:36:32.409 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:32.403+0000 7f5af4ff9640 1 -- 172.21.15.43:0/1962767843 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 6640+0+0 (secure 0 0 0) 0x7f5aec088a20 con 0x7f5af81586c0 2023-08-09T13:36:32.409 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:32.404+0000 7f5af7fff640 1 --2- 172.21.15.43:0/1962767843 >> [v2:172.21.15.203:6832/998005540,v1:172.21.15.203:6833/998005540] conn(0x7f5ae406a1f0 0x7f5ae406c6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:36:32.410 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:32.405+0000 7f5af7fff640 1 --2- 172.21.15.43:0/1962767843 >> [v2:172.21.15.203:6832/998005540,v1:172.21.15.203:6833/998005540] conn(0x7f5ae406a1f0 0x7f5ae406c6b0 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7f5ae8001d80 tx=0x7f5ae8009040 comp rx=0 tx=0).ready entity=mgr.5003 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-09T13:36:32.410 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:32.408+0000 7f5af4ff9640 1 -- 172.21.15.43:0/1962767843 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178066 (secure 0 0 0) 0x7f5aec053dd0 con 0x7f5af81586c0 2023-08-09T13:36:32.623 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:32.621+0000 7f5afe6c5640 1 -- 172.21.15.43:0/1962767843 --> [v2:172.21.15.203:6832/998005540,v1:172.21.15.203:6833/998005540] -- mgr_command(tid 0: {"prefix": "fs snap-schedule add", "fs": "cephfs", "path": "/snap_test_dir1/dir5", "snap_schedule": "1M", "target": ["mon-mgr", ""]}) v1 -- 0x7f5af80d1520 con 0x7f5ae406a1f0 2023-08-09T13:36:32.766 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:32.764+0000 7f5af4ff9640 1 -- 172.21.15.43:0/1962767843 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mgrmap(e 10) v1 ==== 88517+0+0 (secure 0 0 0) 0x7f5aec0541a0 con 0x7f5af81586c0 2023-08-09T13:36:32.798 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:32.796+0000 7f5af4ff9640 1 -- 172.21.15.43:0/1962767843 <== mgr.5003 v2:172.21.15.203:6832/998005540 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+43 (secure 0 0 0) 0x7f5ae8003970 con 0x7f5ae406a1f0 2023-08-09T13:36:32.798 INFO:teuthology.orchestra.run.smithi043.stdout:Schedule set for path /snap_test_dir1/dir5 2023-08-09T13:36:32.801 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:32.800+0000 7f5afe6c5640 1 -- 172.21.15.43:0/1962767843 >> [v2:172.21.15.203:6832/998005540,v1:172.21.15.203:6833/998005540] conn(0x7f5ae406a1f0 msgr2=0x7f5ae406c6b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:36:32.801 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:32.800+0000 7f5afe6c5640 1 --2- 172.21.15.43:0/1962767843 >> [v2:172.21.15.203:6832/998005540,v1:172.21.15.203:6833/998005540] conn(0x7f5ae406a1f0 0x7f5ae406c6b0 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7f5ae8001d80 tx=0x7f5ae8009040 comp rx=0 tx=0).stop 2023-08-09T13:36:32.802 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:32.800+0000 7f5afe6c5640 1 -- 172.21.15.43:0/1962767843 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f5af81586c0 msgr2=0x7f5af8151900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:36:32.802 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:32.800+0000 7f5afe6c5640 1 --2- 172.21.15.43:0/1962767843 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f5af81586c0 0x7f5af8151900 secure :-1 s=READY pgs=619 cs=0 l=1 rev1=1 crypto rx=0x7f5aec00e780 tx=0x7f5aec00ec40 comp rx=0 tx=0).stop 2023-08-09T13:36:32.802 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:32.800+0000 7f5afe6c5640 1 -- 172.21.15.43:0/1962767843 shutdown_connections 2023-08-09T13:36:32.802 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:32.800+0000 7f5afe6c5640 1 --2- 172.21.15.43:0/1962767843 >> [v2:172.21.15.203:6832/998005540,v1:172.21.15.203:6833/998005540] conn(0x7f5ae406a1f0 0x7f5ae406c6b0 unknown :-1 s=CLOSED pgs=28 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:36:32.803 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:32.800+0000 7f5afe6c5640 1 --2- 172.21.15.43:0/1962767843 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f5af81545b0 0x7f5af8154a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:36:32.803 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:32.800+0000 7f5afe6c5640 1 --2- 172.21.15.43:0/1962767843 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f5af81586c0 0x7f5af8151900 unknown :-1 s=CLOSED pgs=619 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:36:32.803 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:32.800+0000 7f5afe6c5640 1 --2- 172.21.15.43:0/1962767843 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f5ae0004d00 0x7f5af8152a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:36:32.803 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:32.800+0000 7f5afe6c5640 1 -- 172.21.15.43:0/1962767843 >> 172.21.15.43:0/1962767843 conn(0x7f5af80712e0 msgr2=0x7f5af80cce60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-09T13:36:32.803 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:32.800+0000 7f5afe6c5640 1 -- 172.21.15.43:0/1962767843 shutdown_connections 2023-08-09T13:36:32.803 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:32.800+0000 7f5afe6c5640 1 -- 172.21.15.43:0/1962767843 wait complete. 2023-08-09T13:36:32.823 DEBUG:tasks.cephfs.test_snap_schedules:res=Schedule set for path /snap_test_dir1/dir5 2023-08-09T13:36:32.824 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-09T13:36:32.824 DEBUG:teuthology.orchestra.run.smithi043:> (cd /home/ubuntu/cephtest/mnt.0 && exec mkdir -p snap_test_dir1/dir6) 2023-08-09T13:36:32.882 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs snap-schedule add --fs cephfs /snap_test_dir1/dir6 1M 2023-08-09T13:36:33.114 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:33.113+0000 7fc1c08cb640 1 Processor -- start 2023-08-09T13:36:33.115 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:33.113+0000 7fc1c08cb640 1 -- start start 2023-08-09T13:36:33.116 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:33.115+0000 7fc1c08cb640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7fc1bc15c400 0x7fc1bc0d0cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:36:33.117 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:33.115+0000 7fc1c08cb640 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7fc1bc0d1210 0x7fc1bc0cec70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:36:33.117 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:33.115+0000 7fc1c08cb640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fc1bc0cd3c0 0x7fc1bc0cf1b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:36:33.117 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:33.115+0000 7fc1c08cb640 1 -- --> v2:172.21.15.43:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc1bc15c7e0 con 0x7fc1bc15c400 2023-08-09T13:36:33.117 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:33.115+0000 7fc1c08cb640 1 -- --> v2:172.21.15.203:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc1bc0d1690 con 0x7fc1bc0d1210 2023-08-09T13:36:33.117 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:33.115+0000 7fc1c08cb640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc1bc0d1970 con 0x7fc1bc0cd3c0 2023-08-09T13:36:33.117 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:33.115+0000 7fc1bad9d640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7fc1bc15c400 0x7fc1bc0d0cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:36:33.117 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:33.115+0000 7fc1bad9d640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7fc1bc15c400 0x7fc1bc0d0cd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:49476/0 (socket says 172.21.15.43:49476) 2023-08-09T13:36:33.118 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:33.115+0000 7fc1bad9d640 1 -- 172.21.15.43:0/1043568573 learned_addr learned my addr 172.21.15.43:0/1043568573 (peer_addr_for_me v2:172.21.15.43:0/0) 2023-08-09T13:36:33.118 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:33.116+0000 7fc1ba59c640 1 --2- 172.21.15.43:0/1043568573 >> v2:172.21.15.203:3300/0 conn(0x7fc1bc0d1210 0x7fc1bc0cec70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:36:33.119 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:33.116+0000 7fc1bb59e640 1 --2- 172.21.15.43:0/1043568573 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fc1bc0cd3c0 0x7fc1bc0cf1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:36:33.119 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:33.116+0000 7fc1bad9d640 1 -- 172.21.15.43:0/1043568573 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fc1bc0cd3c0 msgr2=0x7fc1bc0cf1b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-09T13:36:33.119 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:33.116+0000 7fc1bad9d640 1 --2- 172.21.15.43:0/1043568573 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fc1bc0cd3c0 0x7fc1bc0cf1b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:36:33.119 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:33.116+0000 7fc1bad9d640 1 -- 172.21.15.43:0/1043568573 >> v2:172.21.15.203:3300/0 conn(0x7fc1bc0d1210 msgr2=0x7fc1bc0cec70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-09T13:36:33.119 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:33.116+0000 7fc1bad9d640 1 --2- 172.21.15.43:0/1043568573 >> v2:172.21.15.203:3300/0 conn(0x7fc1bc0d1210 0x7fc1bc0cec70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:36:33.119 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:33.116+0000 7fc1bad9d640 1 -- 172.21.15.43:0/1043568573 --> v2:172.21.15.43:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc1bc0d1c50 con 0x7fc1bc15c400 2023-08-09T13:36:33.119 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:33.116+0000 7fc1bb59e640 1 --2- 172.21.15.43:0/1043568573 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fc1bc0cd3c0 0x7fc1bc0cf1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-08-09T13:36:33.120 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:33.117+0000 7fc1bad9d640 1 --2- 172.21.15.43:0/1043568573 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc1bc15c400 0x7fc1bc0d0cd0 secure :-1 s=READY pgs=620 cs=0 l=1 rev1=1 crypto rx=0x7fc1bc069dd0 tx=0x7fc1ac02bf60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=aacd503e6f6be9f5 server_cookie=0 in_seq=0 out_seq=0 2023-08-09T13:36:33.120 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:33.117+0000 7fc1b9d9b640 1 -- 172.21.15.43:0/1043568573 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc1ac02cb00 con 0x7fc1bc15c400 2023-08-09T13:36:33.120 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:33.117+0000 7fc1b9d9b640 1 -- 172.21.15.43:0/1043568573 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc1ac02cc60 con 0x7fc1bc15c400 2023-08-09T13:36:33.120 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:33.117+0000 7fc1b9d9b640 1 -- 172.21.15.43:0/1043568573 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc1ac0343f0 con 0x7fc1bc15c400 2023-08-09T13:36:33.121 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:33.118+0000 7fc1c08cb640 1 -- 172.21.15.43:0/1043568573 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc1bc15c400 msgr2=0x7fc1bc0d0cd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:36:33.121 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:33.118+0000 7fc1c08cb640 1 --2- 172.21.15.43:0/1043568573 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc1bc15c400 0x7fc1bc0d0cd0 secure :-1 s=READY pgs=620 cs=0 l=1 rev1=1 crypto rx=0x7fc1bc069dd0 tx=0x7fc1ac02bf60 comp rx=0 tx=0).stop 2023-08-09T13:36:33.121 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:33.118+0000 7fc1c08cb640 1 -- 172.21.15.43:0/1043568573 shutdown_connections 2023-08-09T13:36:33.121 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:33.118+0000 7fc1c08cb640 1 --2- 172.21.15.43:0/1043568573 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fc1bc0cd3c0 0x7fc1bc0cf1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:36:33.121 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:33.118+0000 7fc1c08cb640 1 --2- 172.21.15.43:0/1043568573 >> v2:172.21.15.203:3300/0 conn(0x7fc1bc0d1210 0x7fc1bc0cec70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:36:33.122 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:33.118+0000 7fc1c08cb640 1 --2- 172.21.15.43:0/1043568573 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc1bc15c400 0x7fc1bc0d0cd0 unknown :-1 s=CLOSED pgs=620 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:36:33.122 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:33.118+0000 7fc1c08cb640 1 -- 172.21.15.43:0/1043568573 >> 172.21.15.43:0/1043568573 conn(0x7fc1bc06a320 msgr2=0x7fc1bc06a730 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-09T13:36:33.122 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:33.118+0000 7fc1c08cb640 1 -- 172.21.15.43:0/1043568573 shutdown_connections 2023-08-09T13:36:33.122 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:33.118+0000 7fc1c08cb640 1 -- 172.21.15.43:0/1043568573 wait complete. 2023-08-09T13:36:33.123 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:33.120+0000 7fc1c08cb640 1 Processor -- start 2023-08-09T13:36:33.123 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:33.120+0000 7fc1c08cb640 1 -- start start 2023-08-09T13:36:33.123 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:33.121+0000 7fc1c08cb640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fc1bc0cd3c0 0x7fc1bc155fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:36:33.123 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:33.121+0000 7fc1c08cb640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fc1bc0d1210 0x7fc1bc15f390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:36:33.124 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:33.121+0000 7fc1c08cb640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc1bc15c400 0x7fc1bc1592f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:36:33.124 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:33.121+0000 7fc1c08cb640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc1bc156500 con 0x7fc1bc15c400 2023-08-09T13:36:33.124 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:33.121+0000 7fc1c08cb640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc1bc15cfe0 con 0x7fc1bc0cd3c0 2023-08-09T13:36:33.124 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:33.121+0000 7fc1c08cb640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc1bc15f8d0 con 0x7fc1bc0d1210 2023-08-09T13:36:33.124 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:33.122+0000 7fc1bb59e640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc1bc15c400 0x7fc1bc1592f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:36:33.124 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:33.122+0000 7fc1bb59e640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc1bc15c400 0x7fc1bc1592f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:49492/0 (socket says 172.21.15.43:49492) 2023-08-09T13:36:33.124 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:33.122+0000 7fc1bb59e640 1 -- 172.21.15.43:0/1335380876 learned_addr learned my addr 172.21.15.43:0/1335380876 (peer_addr_for_me v2:172.21.15.43:0/0) 2023-08-09T13:36:33.125 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:33.122+0000 7fc1bad9d640 1 --2- 172.21.15.43:0/1335380876 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fc1bc0cd3c0 0x7fc1bc155fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:36:33.126 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:33.122+0000 7fc1ba59c640 1 --2- 172.21.15.43:0/1335380876 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fc1bc0d1210 0x7fc1bc15f390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:36:33.126 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:33.122+0000 7fc1bb59e640 1 -- 172.21.15.43:0/1335380876 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fc1bc0d1210 msgr2=0x7fc1bc15f390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:36:33.126 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:33.122+0000 7fc1bb59e640 1 --2- 172.21.15.43:0/1335380876 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fc1bc0d1210 0x7fc1bc15f390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:36:33.126 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:33.122+0000 7fc1bb59e640 1 -- 172.21.15.43:0/1335380876 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fc1bc0cd3c0 msgr2=0x7fc1bc155fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:36:33.126 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:33.122+0000 7fc1bb59e640 1 --2- 172.21.15.43:0/1335380876 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fc1bc0cd3c0 0x7fc1bc155fc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:36:33.126 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:33.122+0000 7fc1bb59e640 1 -- 172.21.15.43:0/1335380876 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc1ac009b30 con 0x7fc1bc15c400 2023-08-09T13:36:33.127 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:33.122+0000 7fc1bb59e640 1 --2- 172.21.15.43:0/1335380876 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc1bc15c400 0x7fc1bc1592f0 secure :-1 s=READY pgs=621 cs=0 l=1 rev1=1 crypto rx=0x7fc1b400c7f0 tx=0x7fc1b400ccb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-09T13:36:33.127 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:33.123+0000 7fc1a37fe640 1 -- 172.21.15.43:0/1335380876 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc1b4007820 con 0x7fc1bc15c400 2023-08-09T13:36:33.127 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:33.123+0000 7fc1a37fe640 1 -- 172.21.15.43:0/1335380876 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc1b4007980 con 0x7fc1bc15c400 2023-08-09T13:36:33.127 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:33.123+0000 7fc1c08cb640 1 -- 172.21.15.43:0/1335380876 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc1bc15cd00 con 0x7fc1bc15c400 2023-08-09T13:36:33.127 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:33.123+0000 7fc1a37fe640 1 -- 172.21.15.43:0/1335380876 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc1b4007d30 con 0x7fc1bc15c400 2023-08-09T13:36:33.127 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:33.123+0000 7fc1c08cb640 1 -- 172.21.15.43:0/1335380876 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc1bc0c9740 con 0x7fc1bc15c400 2023-08-09T13:36:33.127 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:33.124+0000 7fc1c08cb640 1 -- 172.21.15.43:0/1335380876 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc1bc0739f0 con 0x7fc1bc15c400 2023-08-09T13:36:33.128 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:33.125+0000 7fc1a37fe640 1 -- 172.21.15.43:0/1335380876 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 10) v1 ==== 88517+0+0 (secure 0 0 0) 0x7fc1b40113f0 con 0x7fc1bc15c400 2023-08-09T13:36:33.128 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:33.126+0000 7fc1a37fe640 1 --2- 172.21.15.43:0/1335380876 >> [v2:172.21.15.203:6832/998005540,v1:172.21.15.203:6833/998005540] conn(0x7fc1a806a250 0x7fc1a806c710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:36:33.128 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:33.126+0000 7fc1a37fe640 1 -- 172.21.15.43:0/1335380876 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 6640+0+0 (secure 0 0 0) 0x7fc1b4089020 con 0x7fc1bc15c400 2023-08-09T13:36:33.129 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:33.127+0000 7fc1bad9d640 1 --2- 172.21.15.43:0/1335380876 >> [v2:172.21.15.203:6832/998005540,v1:172.21.15.203:6833/998005540] conn(0x7fc1a806a250 0x7fc1a806c710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:36:33.129 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:33.128+0000 7fc1bad9d640 1 --2- 172.21.15.43:0/1335380876 >> [v2:172.21.15.203:6832/998005540,v1:172.21.15.203:6833/998005540] conn(0x7fc1a806a250 0x7fc1a806c710 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7fc1ac035000 tx=0x7fc1ac030000 comp rx=0 tx=0).ready entity=mgr.5003 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-09T13:36:33.132 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:33.130+0000 7fc1a37fe640 1 -- 172.21.15.43:0/1335380876 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178066 (secure 0 0 0) 0x7fc1b40540c0 con 0x7fc1bc15c400 2023-08-09T13:36:33.350 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:33.349+0000 7fc1c08cb640 1 -- 172.21.15.43:0/1335380876 --> [v2:172.21.15.203:6832/998005540,v1:172.21.15.203:6833/998005540] -- mgr_command(tid 0: {"prefix": "fs snap-schedule add", "fs": "cephfs", "path": "/snap_test_dir1/dir6", "snap_schedule": "1M", "target": ["mon-mgr", ""]}) v1 -- 0x7fc1bc0b52f0 con 0x7fc1a806a250 2023-08-09T13:36:33.511 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:33.509+0000 7fc1a37fe640 1 -- 172.21.15.43:0/1335380876 <== mgr.5003 v2:172.21.15.203:6832/998005540 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+43 (secure 0 0 0) 0x7fc1ac0440d0 con 0x7fc1a806a250 2023-08-09T13:36:33.511 INFO:teuthology.orchestra.run.smithi043.stdout:Schedule set for path /snap_test_dir1/dir6 2023-08-09T13:36:33.514 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:33.512+0000 7fc1c08cb640 1 -- 172.21.15.43:0/1335380876 >> [v2:172.21.15.203:6832/998005540,v1:172.21.15.203:6833/998005540] conn(0x7fc1a806a250 msgr2=0x7fc1a806c710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:36:33.514 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:33.512+0000 7fc1c08cb640 1 --2- 172.21.15.43:0/1335380876 >> [v2:172.21.15.203:6832/998005540,v1:172.21.15.203:6833/998005540] conn(0x7fc1a806a250 0x7fc1a806c710 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7fc1ac035000 tx=0x7fc1ac030000 comp rx=0 tx=0).stop 2023-08-09T13:36:33.515 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:33.512+0000 7fc1c08cb640 1 -- 172.21.15.43:0/1335380876 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc1bc15c400 msgr2=0x7fc1bc1592f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:36:33.515 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:33.512+0000 7fc1c08cb640 1 --2- 172.21.15.43:0/1335380876 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc1bc15c400 0x7fc1bc1592f0 secure :-1 s=READY pgs=621 cs=0 l=1 rev1=1 crypto rx=0x7fc1b400c7f0 tx=0x7fc1b400ccb0 comp rx=0 tx=0).stop 2023-08-09T13:36:33.515 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:33.513+0000 7fc1c08cb640 1 -- 172.21.15.43:0/1335380876 shutdown_connections 2023-08-09T13:36:33.515 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:33.513+0000 7fc1c08cb640 1 --2- 172.21.15.43:0/1335380876 >> [v2:172.21.15.203:6832/998005540,v1:172.21.15.203:6833/998005540] conn(0x7fc1a806a250 0x7fc1a806c710 unknown :-1 s=CLOSED pgs=29 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:36:33.515 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:33.513+0000 7fc1c08cb640 1 --2- 172.21.15.43:0/1335380876 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc1bc15c400 0x7fc1bc1592f0 unknown :-1 s=CLOSED pgs=621 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:36:33.515 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:33.513+0000 7fc1c08cb640 1 --2- 172.21.15.43:0/1335380876 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fc1bc0d1210 0x7fc1bc15f390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:36:33.516 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:33.513+0000 7fc1c08cb640 1 --2- 172.21.15.43:0/1335380876 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fc1bc0cd3c0 0x7fc1bc155fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:36:33.516 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:33.513+0000 7fc1c08cb640 1 -- 172.21.15.43:0/1335380876 >> 172.21.15.43:0/1335380876 conn(0x7fc1bc06a320 msgr2=0x7fc1bc0b3380 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-09T13:36:33.516 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:33.513+0000 7fc1c08cb640 1 -- 172.21.15.43:0/1335380876 shutdown_connections 2023-08-09T13:36:33.516 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:33.513+0000 7fc1c08cb640 1 -- 172.21.15.43:0/1335380876 wait complete. 2023-08-09T13:36:33.534 DEBUG:tasks.cephfs.test_snap_schedules:res=Schedule set for path /snap_test_dir1/dir6 2023-08-09T13:36:33.535 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-09T13:36:33.535 DEBUG:teuthology.orchestra.run.smithi043:> (cd /home/ubuntu/cephtest/mnt.0 && exec mkdir -p snap_test_dir1/dir7) 2023-08-09T13:36:33.598 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs snap-schedule add --fs cephfs /snap_test_dir1/dir7 1M 2023-08-09T13:36:33.835 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:33.833+0000 7f38ea4f3640 1 Processor -- start 2023-08-09T13:36:33.835 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:33.834+0000 7f38ea4f3640 1 -- start start 2023-08-09T13:36:33.837 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:33.835+0000 7f38ea4f3640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f38e4069a30 0x7f38e4159760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:36:33.837 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:33.835+0000 7f38ea4f3640 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7f38e415d5b0 0x7f38e4069e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:36:33.837 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:33.835+0000 7f38ea4f3640 1 -- --> v1:172.21.15.43:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f38e40ca410 con 0x7f38e406a350 2023-08-09T13:36:33.837 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:33.835+0000 7f38ea4f3640 1 -- --> v2:172.21.15.43:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f38e415ce80 con 0x7f38e4069a30 2023-08-09T13:36:33.837 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:33.835+0000 7f38ea4f3640 1 -- --> v2:172.21.15.203:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f38e415a050 con 0x7f38e415d5b0 2023-08-09T13:36:33.838 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:33.836+0000 7f38e3fff640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f38e4069a30 0x7f38e4159760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:36:33.838 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:33.836+0000 7f38e8a90640 1 --1- >> v1:172.21.15.43:6789/0 conn(0x7f38e406a350 0x7f38e40ce4c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.43:6789/0 says I am v1:172.21.15.43:33356/0 (socket says 172.21.15.43:33356) 2023-08-09T13:36:33.839 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:33.836+0000 7f38e8a90640 1 -- 172.21.15.43:0/3515873912 learned_addr learned my addr 172.21.15.43:0/3515873912 (peer_addr_for_me v1:172.21.15.43:0/0) 2023-08-09T13:36:33.839 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:33.836+0000 7f38dbfff640 1 --2- 172.21.15.43:0/3515873912 >> v2:172.21.15.203:3300/0 conn(0x7f38e415d5b0 0x7f38e4069e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:36:33.839 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:33.836+0000 7f38e37fe640 1 -- 172.21.15.43:0/3515873912 <== mon.0 v1:172.21.15.43:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1893581486 0 0) 0x7f38e40ca410 con 0x7f38e406a350 2023-08-09T13:36:33.839 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:33.836+0000 7f38e37fe640 1 -- 172.21.15.43:0/3515873912 --> v1:172.21.15.43:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f38d0005730 con 0x7f38e406a350 2023-08-09T13:36:33.839 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:33.836+0000 7f38e3fff640 1 -- 172.21.15.43:0/3515873912 >> v2:172.21.15.203:3300/0 conn(0x7f38e415d5b0 msgr2=0x7f38e4069e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-09T13:36:33.839 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:33.836+0000 7f38e3fff640 1 --2- 172.21.15.43:0/3515873912 >> v2:172.21.15.203:3300/0 conn(0x7f38e415d5b0 0x7f38e4069e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:36:33.839 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:33.836+0000 7f38e3fff640 1 -- 172.21.15.43:0/3515873912 >> v1:172.21.15.43:6789/0 conn(0x7f38e406a350 legacy=0x7f38e40ce4c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:36:33.840 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:33.836+0000 7f38e3fff640 1 -- 172.21.15.43:0/3515873912 --> v2:172.21.15.43:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f38e415a330 con 0x7f38e4069a30 2023-08-09T13:36:33.840 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:33.836+0000 7f38dbfff640 1 --2- 172.21.15.43:0/3515873912 >> v2:172.21.15.203:3300/0 conn(0x7f38e415d5b0 0x7f38e4069e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-08-09T13:36:33.841 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:33.838+0000 7f38e3fff640 1 --2- 172.21.15.43:0/3515873912 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f38e4069a30 0x7f38e4159760 secure :-1 s=READY pgs=623 cs=0 l=1 rev1=1 crypto rx=0x7f38d40097a0 tx=0x7f38d402b7b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b5fb8950d1fcec46 server_cookie=0 in_seq=0 out_seq=0 2023-08-09T13:36:33.841 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:33.838+0000 7f38e37fe640 1 -- 172.21.15.43:0/3515873912 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f38d4031710 con 0x7f38e4069a30 2023-08-09T13:36:33.841 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:33.838+0000 7f38e37fe640 1 -- 172.21.15.43:0/3515873912 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f38d402f050 con 0x7f38e4069a30 2023-08-09T13:36:33.841 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:33.838+0000 7f38e37fe640 1 -- 172.21.15.43:0/3515873912 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f38d4031a90 con 0x7f38e4069a30 2023-08-09T13:36:33.841 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:33.838+0000 7f38ea4f3640 1 -- 172.21.15.43:0/3515873912 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f38e4069a30 msgr2=0x7f38e4159760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:36:33.842 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:33.838+0000 7f38ea4f3640 1 --2- 172.21.15.43:0/3515873912 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f38e4069a30 0x7f38e4159760 secure :-1 s=READY pgs=623 cs=0 l=1 rev1=1 crypto rx=0x7f38d40097a0 tx=0x7f38d402b7b0 comp rx=0 tx=0).stop 2023-08-09T13:36:33.842 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:33.839+0000 7f38ea4f3640 1 -- 172.21.15.43:0/3515873912 shutdown_connections 2023-08-09T13:36:33.842 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:33.839+0000 7f38ea4f3640 1 --2- 172.21.15.43:0/3515873912 >> v2:172.21.15.203:3300/0 conn(0x7f38e415d5b0 0x7f38e4069e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:36:33.842 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:33.839+0000 7f38ea4f3640 1 --2- 172.21.15.43:0/3515873912 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f38e4069a30 0x7f38e4159760 unknown :-1 s=CLOSED pgs=623 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:36:33.842 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:33.839+0000 7f38ea4f3640 1 -- 172.21.15.43:0/3515873912 >> 172.21.15.43:0/3515873912 conn(0x7f38e406b990 msgr2=0x7f38e406bda0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-09T13:36:33.842 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:33.839+0000 7f38ea4f3640 1 -- 172.21.15.43:0/3515873912 shutdown_connections 2023-08-09T13:36:33.842 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:33.839+0000 7f38ea4f3640 1 -- 172.21.15.43:0/3515873912 wait complete. 2023-08-09T13:36:33.843 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:33.840+0000 7f38ea4f3640 1 Processor -- start 2023-08-09T13:36:33.844 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:33.841+0000 7f38ea4f3640 1 -- start start 2023-08-09T13:36:33.844 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:33.842+0000 7f38ea4f3640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f38e4069a30 0x7f38e41f05b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:36:33.844 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:33.842+0000 7f38ea4f3640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f38e406a350 0x7f38e40afff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:36:33.844 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:33.842+0000 7f38ea4f3640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f38e415d5b0 0x7f38e40b0c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:36:33.844 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:33.842+0000 7f38ea4f3640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f38e40b0a50 con 0x7f38e4069a30 2023-08-09T13:36:33.844 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:33.842+0000 7f38ea4f3640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f38e41f0af0 con 0x7f38e415d5b0 2023-08-09T13:36:33.844 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:33.842+0000 7f38ea4f3640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f38e40c3450 con 0x7f38e406a350 2023-08-09T13:36:33.845 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:33.842+0000 7f38e3fff640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f38e4069a30 0x7f38e41f05b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:36:33.845 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:33.842+0000 7f38e3fff640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f38e4069a30 0x7f38e41f05b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:49520/0 (socket says 172.21.15.43:49520) 2023-08-09T13:36:33.845 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:33.842+0000 7f38e3fff640 1 -- 172.21.15.43:0/110489359 learned_addr learned my addr 172.21.15.43:0/110489359 (peer_addr_for_me v2:172.21.15.43:0/0) 2023-08-09T13:36:33.846 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:33.842+0000 7f38e8a90640 1 --2- 172.21.15.43:0/110489359 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f38e415d5b0 0x7f38e40b0c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:36:33.846 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:33.842+0000 7f38e3fff640 1 -- 172.21.15.43:0/110489359 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f38e406a350 msgr2=0x7f38e40afff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:36:33.846 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:33.842+0000 7f38e3fff640 1 --2- 172.21.15.43:0/110489359 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f38e406a350 0x7f38e40afff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:36:33.847 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:33.842+0000 7f38e3fff640 1 -- 172.21.15.43:0/110489359 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f38e415d5b0 msgr2=0x7f38e40b0c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:36:33.847 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:33.842+0000 7f38e3fff640 1 --2- 172.21.15.43:0/110489359 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f38e415d5b0 0x7f38e40b0c20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:36:33.847 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:33.842+0000 7f38e3fff640 1 -- 172.21.15.43:0/110489359 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f38d4009400 con 0x7f38e4069a30 2023-08-09T13:36:33.847 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:33.843+0000 7f38e3fff640 1 --2- 172.21.15.43:0/110489359 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f38e4069a30 0x7f38e41f05b0 secure :-1 s=READY pgs=624 cs=0 l=1 rev1=1 crypto rx=0x7f38d40098c0 tx=0x7f38d40046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-09T13:36:33.847 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:33.843+0000 7f38e17fa640 1 -- 172.21.15.43:0/110489359 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f38d403c8d0 con 0x7f38e4069a30 2023-08-09T13:36:33.847 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:33.843+0000 7f38e17fa640 1 -- 172.21.15.43:0/110489359 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f38d403ca70 con 0x7f38e4069a30 2023-08-09T13:36:33.847 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:33.843+0000 7f38e17fa640 1 -- 172.21.15.43:0/110489359 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f38d403ce20 con 0x7f38e4069a30 2023-08-09T13:36:33.848 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:33.843+0000 7f38ea4f3640 1 -- 172.21.15.43:0/110489359 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f38e40c3730 con 0x7f38e4069a30 2023-08-09T13:36:33.848 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:33.843+0000 7f38ea4f3640 1 -- 172.21.15.43:0/110489359 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f38e40c3cf0 con 0x7f38e4069a30 2023-08-09T13:36:33.848 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:33.845+0000 7f38ea4f3640 1 -- 172.21.15.43:0/110489359 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f38e40739f0 con 0x7f38e4069a30 2023-08-09T13:36:33.849 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:33.845+0000 7f38e17fa640 1 -- 172.21.15.43:0/110489359 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 10) v1 ==== 88517+0+0 (secure 0 0 0) 0x7f38d403a040 con 0x7f38e4069a30 2023-08-09T13:36:33.849 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:33.847+0000 7f38e17fa640 1 --2- 172.21.15.43:0/110489359 >> [v2:172.21.15.203:6832/998005540,v1:172.21.15.203:6833/998005540] conn(0x7f38c406a250 0x7f38c406c710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:36:33.849 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:33.847+0000 7f38e17fa640 1 -- 172.21.15.43:0/110489359 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 6640+0+0 (secure 0 0 0) 0x7f38d40aa500 con 0x7f38e4069a30 2023-08-09T13:36:33.849 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:33.848+0000 7f38dbfff640 1 --2- 172.21.15.43:0/110489359 >> [v2:172.21.15.203:6832/998005540,v1:172.21.15.203:6833/998005540] conn(0x7f38c406a250 0x7f38c406c710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:36:33.850 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:33.849+0000 7f38dbfff640 1 --2- 172.21.15.43:0/110489359 >> [v2:172.21.15.203:6832/998005540,v1:172.21.15.203:6833/998005540] conn(0x7f38c406a250 0x7f38c406c710 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0x7f38cc006e80 tx=0x7f38cc008040 comp rx=0 tx=0).ready entity=mgr.5003 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-09T13:36:33.853 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:33.852+0000 7f38e17fa640 1 -- 172.21.15.43:0/110489359 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178066 (secure 0 0 0) 0x7f38d4075630 con 0x7f38e4069a30 2023-08-09T13:36:34.068 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:34.067+0000 7f38ea4f3640 1 -- 172.21.15.43:0/110489359 --> [v2:172.21.15.203:6832/998005540,v1:172.21.15.203:6833/998005540] -- mgr_command(tid 0: {"prefix": "fs snap-schedule add", "fs": "cephfs", "path": "/snap_test_dir1/dir7", "snap_schedule": "1M", "target": ["mon-mgr", ""]}) v1 -- 0x7f38e415d090 con 0x7f38c406a250 2023-08-09T13:36:34.224 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:34.222+0000 7f38e17fa640 1 -- 172.21.15.43:0/110489359 <== mgr.5003 v2:172.21.15.203:6832/998005540 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+43 (secure 0 0 0) 0x7f38cc00be60 con 0x7f38c406a250 2023-08-09T13:36:34.225 INFO:teuthology.orchestra.run.smithi043.stdout:Schedule set for path /snap_test_dir1/dir7 2023-08-09T13:36:34.227 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:34.225+0000 7f38ea4f3640 1 -- 172.21.15.43:0/110489359 >> [v2:172.21.15.203:6832/998005540,v1:172.21.15.203:6833/998005540] conn(0x7f38c406a250 msgr2=0x7f38c406c710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:36:34.227 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:34.225+0000 7f38ea4f3640 1 --2- 172.21.15.43:0/110489359 >> [v2:172.21.15.203:6832/998005540,v1:172.21.15.203:6833/998005540] conn(0x7f38c406a250 0x7f38c406c710 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0x7f38cc006e80 tx=0x7f38cc008040 comp rx=0 tx=0).stop 2023-08-09T13:36:34.227 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:34.225+0000 7f38ea4f3640 1 -- 172.21.15.43:0/110489359 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f38e4069a30 msgr2=0x7f38e41f05b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:36:34.228 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:34.225+0000 7f38ea4f3640 1 --2- 172.21.15.43:0/110489359 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f38e4069a30 0x7f38e41f05b0 secure :-1 s=READY pgs=624 cs=0 l=1 rev1=1 crypto rx=0x7f38d40098c0 tx=0x7f38d40046c0 comp rx=0 tx=0).stop 2023-08-09T13:36:34.228 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:34.226+0000 7f38ea4f3640 1 -- 172.21.15.43:0/110489359 shutdown_connections 2023-08-09T13:36:34.228 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:34.226+0000 7f38ea4f3640 1 --2- 172.21.15.43:0/110489359 >> [v2:172.21.15.203:6832/998005540,v1:172.21.15.203:6833/998005540] conn(0x7f38c406a250 0x7f38c406c710 unknown :-1 s=CLOSED pgs=30 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:36:34.228 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:34.226+0000 7f38ea4f3640 1 --2- 172.21.15.43:0/110489359 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f38e415d5b0 0x7f38e40b0c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:36:34.228 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:34.226+0000 7f38ea4f3640 1 --2- 172.21.15.43:0/110489359 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f38e406a350 0x7f38e40afff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:36:34.228 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:34.226+0000 7f38ea4f3640 1 --2- 172.21.15.43:0/110489359 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f38e4069a30 0x7f38e41f05b0 unknown :-1 s=CLOSED pgs=624 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:36:34.229 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:34.226+0000 7f38ea4f3640 1 -- 172.21.15.43:0/110489359 >> 172.21.15.43:0/110489359 conn(0x7f38e406b990 msgr2=0x7f38e406bd70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-09T13:36:34.229 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:34.226+0000 7f38ea4f3640 1 -- 172.21.15.43:0/110489359 shutdown_connections 2023-08-09T13:36:34.229 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:34.226+0000 7f38ea4f3640 1 -- 172.21.15.43:0/110489359 wait complete. 2023-08-09T13:36:34.248 DEBUG:tasks.cephfs.test_snap_schedules:res=Schedule set for path /snap_test_dir1/dir7 2023-08-09T13:36:34.248 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-09T13:36:34.248 DEBUG:teuthology.orchestra.run.smithi043:> (cd /home/ubuntu/cephtest/mnt.0 && exec mkdir -p snap_test_dir1/dir8) 2023-08-09T13:36:34.312 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs snap-schedule add --fs cephfs /snap_test_dir1/dir8 1M 2023-08-09T13:36:34.550 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:34.549+0000 7f55fd483640 1 Processor -- start 2023-08-09T13:36:34.551 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:34.549+0000 7f55fd483640 1 -- start start 2023-08-09T13:36:34.552 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:34.551+0000 7f55fd483640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f55f80b80a0 0x7f55f80b5470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:36:34.552 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:34.551+0000 7f55fd483640 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7f55f80b59b0 0x7f55f80b5e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:36:34.553 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:34.551+0000 7f55fd483640 1 -- --> v1:172.21.15.203:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f55f80d1890 con 0x7f55f815c6f0 2023-08-09T13:36:34.553 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:34.551+0000 7f55fd483640 1 -- --> v2:172.21.15.43:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f55f80d2d30 con 0x7f55f80b80a0 2023-08-09T13:36:34.553 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:34.551+0000 7f55fd483640 1 -- --> v2:172.21.15.203:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f55f815cad0 con 0x7f55f80b59b0 2023-08-09T13:36:34.554 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:34.551+0000 7f55f67fc640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f55f80b80a0 0x7f55f80b5470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:36:34.554 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:34.551+0000 7f55f67fc640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f55f80b80a0 0x7f55f80b5470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:49522/0 (socket says 172.21.15.43:49522) 2023-08-09T13:36:34.554 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:34.551+0000 7f55f67fc640 1 -- 172.21.15.43:0/3177917131 learned_addr learned my addr 172.21.15.43:0/3177917131 (peer_addr_for_me v2:172.21.15.43:0/0) 2023-08-09T13:36:34.554 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:34.552+0000 7f55f77fe640 1 --2- 172.21.15.43:0/3177917131 >> v2:172.21.15.203:3300/0 conn(0x7f55f80b59b0 0x7f55f80b5e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:36:34.554 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:34.552+0000 7f55f67fc640 1 -- 172.21.15.43:0/3177917131 >> v2:172.21.15.203:3300/0 conn(0x7f55f80b59b0 msgr2=0x7f55f80b5e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-09T13:36:34.554 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:34.552+0000 7f55f67fc640 1 --2- 172.21.15.43:0/3177917131 >> v2:172.21.15.203:3300/0 conn(0x7f55f80b59b0 0x7f55f80b5e40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:36:34.555 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:34.552+0000 7f55f67fc640 1 -- 172.21.15.43:0/3177917131 >> v1:172.21.15.203:6789/0 conn(0x7f55f815c6f0 legacy=0x7f55f80d03a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-09T13:36:34.555 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:34.552+0000 7f55f67fc640 1 -- 172.21.15.43:0/3177917131 --> v2:172.21.15.43:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f55f80d0c80 con 0x7f55f80b80a0 2023-08-09T13:36:34.555 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:34.553+0000 7f55f67fc640 1 --2- 172.21.15.43:0/3177917131 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f55f80b80a0 0x7f55f80b5470 secure :-1 s=READY pgs=625 cs=0 l=1 rev1=1 crypto rx=0x7f55f80cf4a0 tx=0x7f55e002b9a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fa8b80504f001be7 server_cookie=0 in_seq=0 out_seq=0 2023-08-09T13:36:34.555 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:34.553+0000 7f55f5ffb640 1 -- 172.21.15.43:0/3177917131 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f55e0030860 con 0x7f55f80b80a0 2023-08-09T13:36:34.556 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:34.554+0000 7f55f5ffb640 1 -- 172.21.15.43:0/3177917131 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f55e00309c0 con 0x7f55f80b80a0 2023-08-09T13:36:34.556 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:34.554+0000 7f55f5ffb640 1 -- 172.21.15.43:0/3177917131 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f55e0030d50 con 0x7f55f80b80a0 2023-08-09T13:36:34.556 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:34.554+0000 7f55fd483640 1 -- 172.21.15.43:0/3177917131 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f55f80b80a0 msgr2=0x7f55f80b5470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:36:34.556 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:34.554+0000 7f55fd483640 1 --2- 172.21.15.43:0/3177917131 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f55f80b80a0 0x7f55f80b5470 secure :-1 s=READY pgs=625 cs=0 l=1 rev1=1 crypto rx=0x7f55f80cf4a0 tx=0x7f55e002b9a0 comp rx=0 tx=0).stop 2023-08-09T13:36:34.557 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:34.554+0000 7f55fd483640 1 -- 172.21.15.43:0/3177917131 shutdown_connections 2023-08-09T13:36:34.557 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:34.554+0000 7f55fd483640 1 --2- 172.21.15.43:0/3177917131 >> v2:172.21.15.203:3300/0 conn(0x7f55f80b59b0 0x7f55f80b5e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:36:34.558 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:34.554+0000 7f55fd483640 1 --2- 172.21.15.43:0/3177917131 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f55f80b80a0 0x7f55f80b5470 unknown :-1 s=CLOSED pgs=625 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:36:34.558 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:34.554+0000 7f55fd483640 1 -- 172.21.15.43:0/3177917131 >> 172.21.15.43:0/3177917131 conn(0x7f55f80712e0 msgr2=0x7f55f80716f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-09T13:36:34.558 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:34.554+0000 7f55fd483640 1 -- 172.21.15.43:0/3177917131 shutdown_connections 2023-08-09T13:36:34.558 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:34.554+0000 7f55fd483640 1 -- 172.21.15.43:0/3177917131 wait complete. 2023-08-09T13:36:34.560 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:34.556+0000 7f55fd483640 1 Processor -- start 2023-08-09T13:36:34.560 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:34.556+0000 7f55fd483640 1 -- start start 2023-08-09T13:36:34.560 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:34.557+0000 7f55fd483640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f55f80b59b0 0x7f55f80b3cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:36:34.560 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:34.557+0000 7f55fd483640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f55f80b80a0 0x7f55f80d1d20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:36:34.561 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:34.557+0000 7f55fd483640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f55f815c6f0 0x7f55f80d2260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:36:34.561 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:34.558+0000 7f55fd483640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f55f814f010 con 0x7f55f80b80a0 2023-08-09T13:36:34.561 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:34.558+0000 7f55fd483640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f55f81f5e20 con 0x7f55f815c6f0 2023-08-09T13:36:34.561 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:34.558+0000 7f55fd483640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f55f80d2910 con 0x7f55f80b59b0 2023-08-09T13:36:34.561 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:34.558+0000 7f55f67fc640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f55f80b80a0 0x7f55f80d1d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:36:34.561 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:34.558+0000 7f55f67fc640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f55f80b80a0 0x7f55f80d1d20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:49538/0 (socket says 172.21.15.43:49538) 2023-08-09T13:36:34.561 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:34.558+0000 7f55f67fc640 1 -- 172.21.15.43:0/2748788313 learned_addr learned my addr 172.21.15.43:0/2748788313 (peer_addr_for_me v2:172.21.15.43:0/0) 2023-08-09T13:36:34.562 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:34.558+0000 7f55f6ffd640 1 --2- 172.21.15.43:0/2748788313 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f55f80b59b0 0x7f55f80b3cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:36:34.562 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:34.558+0000 7f55f77fe640 1 --2- 172.21.15.43:0/2748788313 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f55f815c6f0 0x7f55f80d2260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:36:34.562 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:34.558+0000 7f55f67fc640 1 -- 172.21.15.43:0/2748788313 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f55f80b59b0 msgr2=0x7f55f80b3cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:36:34.562 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:34.558+0000 7f55f67fc640 1 --2- 172.21.15.43:0/2748788313 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f55f80b59b0 0x7f55f80b3cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:36:34.562 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:34.558+0000 7f55f67fc640 1 -- 172.21.15.43:0/2748788313 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f55f815c6f0 msgr2=0x7f55f80d2260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:36:34.562 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:34.558+0000 7f55f67fc640 1 --2- 172.21.15.43:0/2748788313 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f55f815c6f0 0x7f55f80d2260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:36:34.563 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:34.559+0000 7f55f67fc640 1 -- 172.21.15.43:0/2748788313 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f55e00094d0 con 0x7f55f80b80a0 2023-08-09T13:36:34.563 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:34.559+0000 7f55f77fe640 1 --2- 172.21.15.43:0/2748788313 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f55f815c6f0 0x7f55f80d2260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-08-09T13:36:34.564 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:34.559+0000 7f55f67fc640 1 --2- 172.21.15.43:0/2748788313 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f55f80b80a0 0x7f55f80d1d20 secure :-1 s=READY pgs=626 cs=0 l=1 rev1=1 crypto rx=0x7f55e002b9e0 tx=0x7f55e0004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-09T13:36:34.564 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:34.559+0000 7f55ef7fe640 1 -- 172.21.15.43:0/2748788313 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f55e0030850 con 0x7f55f80b80a0 2023-08-09T13:36:34.564 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:34.559+0000 7f55ef7fe640 1 -- 172.21.15.43:0/2748788313 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f55e0043400 con 0x7f55f80b80a0 2023-08-09T13:36:34.564 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:34.559+0000 7f55ef7fe640 1 -- 172.21.15.43:0/2748788313 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f55e00437b0 con 0x7f55f80b80a0 2023-08-09T13:36:34.564 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:34.559+0000 7f55fd483640 1 -- 172.21.15.43:0/2748788313 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f55f814e670 con 0x7f55f80b80a0 2023-08-09T13:36:34.564 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:34.560+0000 7f55fd483640 1 -- 172.21.15.43:0/2748788313 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f55f814ec30 con 0x7f55f80b80a0 2023-08-09T13:36:34.565 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:34.561+0000 7f55fd483640 1 -- 172.21.15.43:0/2748788313 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f55f80739f0 con 0x7f55f80b80a0 2023-08-09T13:36:34.565 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:34.562+0000 7f55ef7fe640 1 -- 172.21.15.43:0/2748788313 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 10) v1 ==== 88517+0+0 (secure 0 0 0) 0x7f55e0043910 con 0x7f55f80b80a0 2023-08-09T13:36:34.565 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:34.563+0000 7f55ef7fe640 1 --2- 172.21.15.43:0/2748788313 >> [v2:172.21.15.203:6832/998005540,v1:172.21.15.203:6833/998005540] conn(0x7f55d806a250 0x7f55d806c710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:36:34.565 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:34.563+0000 7f55ef7fe640 1 -- 172.21.15.43:0/2748788313 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 6640+0+0 (secure 0 0 0) 0x7f55e003e070 con 0x7f55f80b80a0 2023-08-09T13:36:34.566 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:34.564+0000 7f55f6ffd640 1 --2- 172.21.15.43:0/2748788313 >> [v2:172.21.15.203:6832/998005540,v1:172.21.15.203:6833/998005540] conn(0x7f55d806a250 0x7f55d806c710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:36:34.566 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:34.565+0000 7f55f6ffd640 1 --2- 172.21.15.43:0/2748788313 >> [v2:172.21.15.203:6832/998005540,v1:172.21.15.203:6833/998005540] conn(0x7f55d806a250 0x7f55d806c710 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0x7f55e80012e0 tx=0x7f55e8009040 comp rx=0 tx=0).ready entity=mgr.5003 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-09T13:36:34.569 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:34.568+0000 7f55ef7fe640 1 -- 172.21.15.43:0/2748788313 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178066 (secure 0 0 0) 0x7f55e0077130 con 0x7f55f80b80a0 2023-08-09T13:36:34.786 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:34.784+0000 7f55fd483640 1 -- 172.21.15.43:0/2748788313 --> [v2:172.21.15.203:6832/998005540,v1:172.21.15.203:6833/998005540] -- mgr_command(tid 0: {"prefix": "fs snap-schedule add", "fs": "cephfs", "path": "/snap_test_dir1/dir8", "snap_schedule": "1M", "target": ["mon-mgr", ""]}) v1 -- 0x7f55f80d1240 con 0x7f55d806a250 2023-08-09T13:36:34.947 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:34.946+0000 7f55ef7fe640 1 -- 172.21.15.43:0/2748788313 <== mgr.5003 v2:172.21.15.203:6832/998005540 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+43 (secure 0 0 0) 0x7f55e8003970 con 0x7f55d806a250 2023-08-09T13:36:34.948 INFO:teuthology.orchestra.run.smithi043.stdout:Schedule set for path /snap_test_dir1/dir8 2023-08-09T13:36:34.951 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:34.950+0000 7f55fd483640 1 -- 172.21.15.43:0/2748788313 >> [v2:172.21.15.203:6832/998005540,v1:172.21.15.203:6833/998005540] conn(0x7f55d806a250 msgr2=0x7f55d806c710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:36:34.952 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:34.950+0000 7f55fd483640 1 --2- 172.21.15.43:0/2748788313 >> [v2:172.21.15.203:6832/998005540,v1:172.21.15.203:6833/998005540] conn(0x7f55d806a250 0x7f55d806c710 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0x7f55e80012e0 tx=0x7f55e8009040 comp rx=0 tx=0).stop 2023-08-09T13:36:34.952 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:34.950+0000 7f55fd483640 1 -- 172.21.15.43:0/2748788313 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f55f80b80a0 msgr2=0x7f55f80d1d20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:36:34.952 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:34.950+0000 7f55fd483640 1 --2- 172.21.15.43:0/2748788313 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f55f80b80a0 0x7f55f80d1d20 secure :-1 s=READY pgs=626 cs=0 l=1 rev1=1 crypto rx=0x7f55e002b9e0 tx=0x7f55e0004730 comp rx=0 tx=0).stop 2023-08-09T13:36:34.952 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:34.950+0000 7f55fd483640 1 -- 172.21.15.43:0/2748788313 shutdown_connections 2023-08-09T13:36:34.952 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:34.950+0000 7f55fd483640 1 --2- 172.21.15.43:0/2748788313 >> [v2:172.21.15.203:6832/998005540,v1:172.21.15.203:6833/998005540] conn(0x7f55d806a250 0x7f55d806c710 unknown :-1 s=CLOSED pgs=31 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:36:34.952 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:34.950+0000 7f55fd483640 1 --2- 172.21.15.43:0/2748788313 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f55f815c6f0 0x7f55f80d2260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:36:34.953 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:34.950+0000 7f55fd483640 1 --2- 172.21.15.43:0/2748788313 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f55f80b80a0 0x7f55f80d1d20 unknown :-1 s=CLOSED pgs=626 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:36:34.953 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:34.950+0000 7f55fd483640 1 --2- 172.21.15.43:0/2748788313 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f55f80b59b0 0x7f55f80b3cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:36:34.953 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:34.950+0000 7f55fd483640 1 -- 172.21.15.43:0/2748788313 >> 172.21.15.43:0/2748788313 conn(0x7f55f80712e0 msgr2=0x7f55f80b6420 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-09T13:36:34.953 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:34.950+0000 7f55fd483640 1 -- 172.21.15.43:0/2748788313 shutdown_connections 2023-08-09T13:36:34.953 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:34.950+0000 7f55fd483640 1 -- 172.21.15.43:0/2748788313 wait complete. 2023-08-09T13:36:34.973 DEBUG:tasks.cephfs.test_snap_schedules:res=Schedule set for path /snap_test_dir1/dir8 2023-08-09T13:36:34.974 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-09T13:36:34.974 DEBUG:teuthology.orchestra.run.smithi043:> (cd /home/ubuntu/cephtest/mnt.0 && exec mkdir -p snap_test_dir1/dir9) 2023-08-09T13:36:35.036 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs snap-schedule add --fs cephfs /snap_test_dir1/dir9 1M 2023-08-09T13:36:35.253 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:35.252+0000 7f5be7660640 1 Processor -- start 2023-08-09T13:36:35.254 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:35.252+0000 7f5be7660640 1 -- start start 2023-08-09T13:36:35.255 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:35.253+0000 7f5be7660640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f5be015c6f0 0x7f5be00d03a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:36:35.256 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:35.253+0000 7f5be7660640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f5be00b5b80 0x7f5be00d1d20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:36:35.256 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:35.253+0000 7f5be7660640 1 -- --> v1:172.21.15.43:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5be00d1890 con 0x7f5be00b80a0 2023-08-09T13:36:35.256 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:35.253+0000 7f5be7660640 1 -- --> v2:172.21.15.43:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5be015cad0 con 0x7f5be00b5b80 2023-08-09T13:36:35.257 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:35.253+0000 7f5be7660640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5be00d0c80 con 0x7f5be015c6f0 2023-08-09T13:36:35.257 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:35.253+0000 7f5be5bfd640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f5be00b5b80 0x7f5be00d1d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:36:35.257 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:35.253+0000 7f5be5bfd640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f5be00b5b80 0x7f5be00d1d20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:45504/0 (socket says 172.21.15.43:45504) 2023-08-09T13:36:35.257 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:35.253+0000 7f5be5bfd640 1 -- 172.21.15.43:0/1187771176 learned_addr learned my addr 172.21.15.43:0/1187771176 (peer_addr_for_me v2:172.21.15.43:0/0) 2023-08-09T13:36:35.258 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:35.254+0000 7f5be53fc640 1 --2- 172.21.15.43:0/1187771176 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f5be015c6f0 0x7f5be00d03a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:36:35.258 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:35.254+0000 7f5be5bfd640 1 -- 172.21.15.43:0/1187771176 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f5be015c6f0 msgr2=0x7f5be00d03a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-09T13:36:35.258 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:35.254+0000 7f5be5bfd640 1 --2- 172.21.15.43:0/1187771176 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f5be015c6f0 0x7f5be00d03a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:36:35.258 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:35.254+0000 7f5be5bfd640 1 -- 172.21.15.43:0/1187771176 >> v1:172.21.15.43:6789/0 conn(0x7f5be00b80a0 legacy=0x7f5be00b5470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:36:35.259 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:35.254+0000 7f5be5bfd640 1 -- 172.21.15.43:0/1187771176 --> v2:172.21.15.43:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5be00d0f60 con 0x7f5be00b5b80 2023-08-09T13:36:35.263 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:35.255+0000 7f5be5bfd640 1 --2- 172.21.15.43:0/1187771176 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f5be00b5b80 0x7f5be00d1d20 secure :-1 s=READY pgs=628 cs=0 l=1 rev1=1 crypto rx=0x7f5bd8009870 tx=0x7f5bd802b980 comp rx=0 tx=0).ready entity=mon.0 client_cookie=94e20763b320b32b server_cookie=0 in_seq=0 out_seq=0 2023-08-09T13:36:35.263 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:35.255+0000 7f5bd7fff640 1 -- 172.21.15.43:0/1187771176 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5bd8030840 con 0x7f5be00b5b80 2023-08-09T13:36:35.263 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:35.255+0000 7f5bd7fff640 1 -- 172.21.15.43:0/1187771176 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5bd80309a0 con 0x7f5be00b5b80 2023-08-09T13:36:35.264 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:35.256+0000 7f5bd7fff640 1 -- 172.21.15.43:0/1187771176 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5bd8030d30 con 0x7f5be00b5b80 2023-08-09T13:36:35.264 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:35.256+0000 7f5be7660640 1 -- 172.21.15.43:0/1187771176 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f5be00b5b80 msgr2=0x7f5be00d1d20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:36:35.264 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:35.256+0000 7f5be7660640 1 --2- 172.21.15.43:0/1187771176 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f5be00b5b80 0x7f5be00d1d20 secure :-1 s=READY pgs=628 cs=0 l=1 rev1=1 crypto rx=0x7f5bd8009870 tx=0x7f5bd802b980 comp rx=0 tx=0).stop 2023-08-09T13:36:35.264 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:35.256+0000 7f5be7660640 1 -- 172.21.15.43:0/1187771176 shutdown_connections 2023-08-09T13:36:35.265 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:35.256+0000 7f5be7660640 1 --2- 172.21.15.43:0/1187771176 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f5be00b5b80 0x7f5be00d1d20 unknown :-1 s=CLOSED pgs=628 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:36:35.265 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:35.256+0000 7f5be7660640 1 --2- 172.21.15.43:0/1187771176 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f5be015c6f0 0x7f5be00d03a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:36:35.265 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:35.256+0000 7f5be7660640 1 -- 172.21.15.43:0/1187771176 >> 172.21.15.43:0/1187771176 conn(0x7f5be00712e0 msgr2=0x7f5be00716f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-09T13:36:35.265 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:35.256+0000 7f5be7660640 1 -- 172.21.15.43:0/1187771176 shutdown_connections 2023-08-09T13:36:35.266 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:35.256+0000 7f5be7660640 1 -- 172.21.15.43:0/1187771176 wait complete. 2023-08-09T13:36:35.266 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:35.258+0000 7f5be7660640 1 Processor -- start 2023-08-09T13:36:35.266 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:35.258+0000 7f5be7660640 1 -- start start 2023-08-09T13:36:35.267 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:35.259+0000 7f5be7660640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f5be00b5b80 0x7f5be014fa70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:36:35.267 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:35.259+0000 7f5be7660640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f5be00b80a0 0x7f5be00cb7d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:36:35.267 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:35.259+0000 7f5be7660640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f5be015c6f0 0x7f5be015d320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:36:35.267 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:35.259+0000 7f5be7660640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5be0152d20 con 0x7f5be015c6f0 2023-08-09T13:36:35.268 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:35.259+0000 7f5be7660640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5be014ffb0 con 0x7f5be00b5b80 2023-08-09T13:36:35.268 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:35.259+0000 7f5be7660640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5be00cc1b0 con 0x7f5be00b80a0 2023-08-09T13:36:35.268 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:35.259+0000 7f5be5bfd640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f5be015c6f0 0x7f5be015d320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:36:35.268 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:35.260+0000 7f5be5bfd640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f5be015c6f0 0x7f5be015d320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:45506/0 (socket says 172.21.15.43:45506) 2023-08-09T13:36:35.269 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:35.260+0000 7f5be5bfd640 1 -- 172.21.15.43:0/650636922 learned_addr learned my addr 172.21.15.43:0/650636922 (peer_addr_for_me v2:172.21.15.43:0/0) 2023-08-09T13:36:35.269 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:35.260+0000 7f5be4bfb640 1 --2- 172.21.15.43:0/650636922 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f5be00b80a0 0x7f5be00cb7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:36:35.269 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:35.260+0000 7f5be53fc640 1 --2- 172.21.15.43:0/650636922 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f5be00b5b80 0x7f5be014fa70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:36:35.269 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:35.260+0000 7f5be5bfd640 1 -- 172.21.15.43:0/650636922 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f5be00b80a0 msgr2=0x7f5be00cb7d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:36:35.270 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:35.260+0000 7f5be5bfd640 1 --2- 172.21.15.43:0/650636922 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f5be00b80a0 0x7f5be00cb7d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:36:35.270 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:35.260+0000 7f5be5bfd640 1 -- 172.21.15.43:0/650636922 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f5be00b5b80 msgr2=0x7f5be014fa70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:36:35.270 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:35.260+0000 7f5be5bfd640 1 --2- 172.21.15.43:0/650636922 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f5be00b5b80 0x7f5be014fa70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:36:35.270 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:35.260+0000 7f5be5bfd640 1 -- 172.21.15.43:0/650636922 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5bd80094d0 con 0x7f5be015c6f0 2023-08-09T13:36:35.271 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:35.261+0000 7f5be5bfd640 1 --2- 172.21.15.43:0/650636922 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f5be015c6f0 0x7f5be015d320 secure :-1 s=READY pgs=629 cs=0 l=1 rev1=1 crypto rx=0x7f5bd802b9c0 tx=0x7f5bd8004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-09T13:36:35.271 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:35.261+0000 7f5bd5ffb640 1 -- 172.21.15.43:0/650636922 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5bd8039c90 con 0x7f5be015c6f0 2023-08-09T13:36:35.271 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:35.261+0000 7f5bd5ffb640 1 -- 172.21.15.43:0/650636922 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5bd8039e10 con 0x7f5be015c6f0 2023-08-09T13:36:35.271 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:35.261+0000 7f5bd5ffb640 1 -- 172.21.15.43:0/650636922 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5bd80446c0 con 0x7f5be015c6f0 2023-08-09T13:36:35.272 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:35.261+0000 7f5be7660640 1 -- 172.21.15.43:0/650636922 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5be0152380 con 0x7f5be015c6f0 2023-08-09T13:36:35.272 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:35.261+0000 7f5be7660640 1 -- 172.21.15.43:0/650636922 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5be0152940 con 0x7f5be015c6f0 2023-08-09T13:36:35.273 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:35.262+0000 7f5be7660640 1 -- 172.21.15.43:0/650636922 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5be00739f0 con 0x7f5be015c6f0 2023-08-09T13:36:35.274 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:35.263+0000 7f5bd5ffb640 1 -- 172.21.15.43:0/650636922 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 10) v1 ==== 88517+0+0 (secure 0 0 0) 0x7f5bd804d3b0 con 0x7f5be015c6f0 2023-08-09T13:36:35.274 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:35.265+0000 7f5bd5ffb640 1 --2- 172.21.15.43:0/650636922 >> [v2:172.21.15.203:6832/998005540,v1:172.21.15.203:6833/998005540] conn(0x7f5bc406a250 0x7f5bc406c710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:36:35.274 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:35.265+0000 7f5bd5ffb640 1 -- 172.21.15.43:0/650636922 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 6640+0+0 (secure 0 0 0) 0x7f5bd803e070 con 0x7f5be015c6f0 2023-08-09T13:36:35.275 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:35.265+0000 7f5be53fc640 1 --2- 172.21.15.43:0/650636922 >> [v2:172.21.15.203:6832/998005540,v1:172.21.15.203:6833/998005540] conn(0x7f5bc406a250 0x7f5bc406c710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:36:35.275 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:35.267+0000 7f5be53fc640 1 --2- 172.21.15.43:0/650636922 >> [v2:172.21.15.203:6832/998005540,v1:172.21.15.203:6833/998005540] conn(0x7f5bc406a250 0x7f5bc406c710 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0x7f5bdc0099c0 tx=0x7f5bdc003990 comp rx=0 tx=0).ready entity=mgr.5003 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-09T13:36:35.275 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:35.269+0000 7f5bd5ffb640 1 -- 172.21.15.43:0/650636922 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178066 (secure 0 0 0) 0x7f5bd8076a40 con 0x7f5be015c6f0 2023-08-09T13:36:35.486 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:35.484+0000 7f5be7660640 1 -- 172.21.15.43:0/650636922 --> [v2:172.21.15.203:6832/998005540,v1:172.21.15.203:6833/998005540] -- mgr_command(tid 0: {"prefix": "fs snap-schedule add", "fs": "cephfs", "path": "/snap_test_dir1/dir9", "snap_schedule": "1M", "target": ["mon-mgr", ""]}) v1 -- 0x7f5be00d1520 con 0x7f5bc406a250 2023-08-09T13:36:35.651 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:35.649+0000 7f5bd5ffb640 1 -- 172.21.15.43:0/650636922 <== mgr.5003 v2:172.21.15.203:6832/998005540 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+43 (secure 0 0 0) 0x7f5bdc0046c0 con 0x7f5bc406a250 2023-08-09T13:36:35.651 INFO:teuthology.orchestra.run.smithi043.stdout:Schedule set for path /snap_test_dir1/dir9 2023-08-09T13:36:35.654 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:35.652+0000 7f5be7660640 1 -- 172.21.15.43:0/650636922 >> [v2:172.21.15.203:6832/998005540,v1:172.21.15.203:6833/998005540] conn(0x7f5bc406a250 msgr2=0x7f5bc406c710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:36:35.654 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:35.652+0000 7f5be7660640 1 --2- 172.21.15.43:0/650636922 >> [v2:172.21.15.203:6832/998005540,v1:172.21.15.203:6833/998005540] conn(0x7f5bc406a250 0x7f5bc406c710 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0x7f5bdc0099c0 tx=0x7f5bdc003990 comp rx=0 tx=0).stop 2023-08-09T13:36:35.655 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:35.652+0000 7f5be7660640 1 -- 172.21.15.43:0/650636922 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f5be015c6f0 msgr2=0x7f5be015d320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:36:35.655 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:35.652+0000 7f5be7660640 1 --2- 172.21.15.43:0/650636922 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f5be015c6f0 0x7f5be015d320 secure :-1 s=READY pgs=629 cs=0 l=1 rev1=1 crypto rx=0x7f5bd802b9c0 tx=0x7f5bd8004730 comp rx=0 tx=0).stop 2023-08-09T13:36:35.655 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:35.653+0000 7f5be7660640 1 -- 172.21.15.43:0/650636922 shutdown_connections 2023-08-09T13:36:35.655 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:35.653+0000 7f5be7660640 1 --2- 172.21.15.43:0/650636922 >> [v2:172.21.15.203:6832/998005540,v1:172.21.15.203:6833/998005540] conn(0x7f5bc406a250 0x7f5bc406c710 unknown :-1 s=CLOSED pgs=32 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:36:35.655 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:35.653+0000 7f5be7660640 1 --2- 172.21.15.43:0/650636922 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f5be015c6f0 0x7f5be015d320 unknown :-1 s=CLOSED pgs=629 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:36:35.655 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:35.653+0000 7f5be7660640 1 --2- 172.21.15.43:0/650636922 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f5be00b80a0 0x7f5be00cb7d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:36:35.655 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:35.653+0000 7f5be7660640 1 --2- 172.21.15.43:0/650636922 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f5be00b5b80 0x7f5be014fa70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:36:35.656 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:35.653+0000 7f5be7660640 1 -- 172.21.15.43:0/650636922 >> 172.21.15.43:0/650636922 conn(0x7f5be00712e0 msgr2=0x7f5be00b5ff0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-09T13:36:35.656 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:35.653+0000 7f5be7660640 1 -- 172.21.15.43:0/650636922 shutdown_connections 2023-08-09T13:36:35.656 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:35.653+0000 7f5be7660640 1 -- 172.21.15.43:0/650636922 wait complete. 2023-08-09T13:36:35.677 DEBUG:tasks.cephfs.test_snap_schedules:res=Schedule set for path /snap_test_dir1/dir9 2023-08-09T13:36:35.677 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs snap-schedule activate --fs cephfs /snap_test_dir1/dir0 1M 2023-08-09T13:36:35.924 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:35.923+0000 7f900e234640 1 Processor -- start 2023-08-09T13:36:35.924 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:35.923+0000 7f900e234640 1 -- start start 2023-08-09T13:36:35.926 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:35.924+0000 7f900e234640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f900815fc70 0x7f90080b10c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:36:35.926 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:35.924+0000 7f900e234640 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7f90080d3020 0x7f90080af0a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:36:35.926 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:35.924+0000 7f900e234640 1 -- --> v1:172.21.15.203:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f90080b0c30 con 0x7f90080d00b0 2023-08-09T13:36:35.926 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:35.924+0000 7f900e234640 1 -- --> v2:172.21.15.203:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f90080b0300 con 0x7f90080d3020 2023-08-09T13:36:35.927 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:35.924+0000 7f900e234640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f90080b05e0 con 0x7f900815fc70 2023-08-09T13:36:35.927 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:35.925+0000 7f90077fe640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f900815fc70 0x7f90080b10c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:36:35.927 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:35.925+0000 7f9007fff640 1 --1- >> v1:172.21.15.203:6789/0 conn(0x7f90080d00b0 0x7f90080af5e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.203:6789/0 says I am v1:172.21.15.43:36888/0 (socket says 172.21.15.43:36888) 2023-08-09T13:36:35.928 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:35.925+0000 7f9007fff640 1 -- 172.21.15.43:0/2211122836 learned_addr learned my addr 172.21.15.43:0/2211122836 (peer_addr_for_me v1:172.21.15.43:0/0) 2023-08-09T13:36:35.928 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:35.926+0000 7f8ffeffd640 1 --2- 172.21.15.43:0/2211122836 >> v2:172.21.15.203:3300/0 conn(0x7f90080d3020 0x7f90080af0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:36:35.928 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:35.926+0000 7f90077fe640 1 -- 172.21.15.43:0/2211122836 >> v2:172.21.15.203:3300/0 conn(0x7f90080d3020 msgr2=0x7f90080af0a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-09T13:36:35.929 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:35.926+0000 7f90077fe640 1 --2- 172.21.15.43:0/2211122836 >> v2:172.21.15.203:3300/0 conn(0x7f90080d3020 0x7f90080af0a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:36:35.929 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:35.926+0000 7f90077fe640 1 -- 172.21.15.43:0/2211122836 >> v1:172.21.15.203:6789/0 conn(0x7f90080d00b0 legacy=0x7f90080af5e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:36:35.929 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:35.926+0000 7f90077fe640 1 -- 172.21.15.43:0/2211122836 --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f90080b08c0 con 0x7f900815fc70 2023-08-09T13:36:35.929 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:35.928+0000 7f90077fe640 1 --2- 172.21.15.43:0/2211122836 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f900815fc70 0x7f90080b10c0 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f8ff8009de0 tx=0x7f8ff802bee0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=6b1202212a4d369c server_cookie=0 in_seq=0 out_seq=0 2023-08-09T13:36:35.930 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:35.928+0000 7f9006ffd640 1 -- 172.21.15.43:0/2211122836 <== mon.2 v2:172.21.15.203:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8ff802cc90 con 0x7f900815fc70 2023-08-09T13:36:35.930 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:35.929+0000 7f9006ffd640 1 -- 172.21.15.43:0/2211122836 <== mon.2 v2:172.21.15.203:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8ff802cdf0 con 0x7f900815fc70 2023-08-09T13:36:35.930 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:35.929+0000 7f9006ffd640 1 -- 172.21.15.43:0/2211122836 <== mon.2 v2:172.21.15.203:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8ff8034660 con 0x7f900815fc70 2023-08-09T13:36:35.931 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:35.929+0000 7f900e234640 1 -- 172.21.15.43:0/2211122836 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f900815fc70 msgr2=0x7f90080b10c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:36:35.931 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:35.929+0000 7f900e234640 1 --2- 172.21.15.43:0/2211122836 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f900815fc70 0x7f90080b10c0 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f8ff8009de0 tx=0x7f8ff802bee0 comp rx=0 tx=0).stop 2023-08-09T13:36:35.931 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:35.929+0000 7f900e234640 1 -- 172.21.15.43:0/2211122836 shutdown_connections 2023-08-09T13:36:35.931 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:35.929+0000 7f900e234640 1 --2- 172.21.15.43:0/2211122836 >> v2:172.21.15.203:3300/0 conn(0x7f90080d3020 0x7f90080af0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:36:35.932 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:35.929+0000 7f900e234640 1 --2- 172.21.15.43:0/2211122836 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f900815fc70 0x7f90080b10c0 unknown :-1 s=CLOSED pgs=86 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:36:35.932 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:35.929+0000 7f900e234640 1 -- 172.21.15.43:0/2211122836 >> 172.21.15.43:0/2211122836 conn(0x7f900809b7b0 msgr2=0x7f90080d2470 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-09T13:36:35.932 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:35.929+0000 7f900e234640 1 -- 172.21.15.43:0/2211122836 shutdown_connections 2023-08-09T13:36:35.932 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:35.930+0000 7f900e234640 1 -- 172.21.15.43:0/2211122836 wait complete. 2023-08-09T13:36:35.933 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:35.931+0000 7f900e234640 1 Processor -- start 2023-08-09T13:36:35.933 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:35.932+0000 7f900e234640 1 -- start start 2023-08-09T13:36:35.934 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:35.933+0000 7f900e234640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f90080d00b0 0x7f90080b7e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:36:35.934 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:35.933+0000 7f900e234640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f900815fc70 0x7f90080ce9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:36:35.935 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:35.933+0000 7f900e234640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f90080cef00 0x7f90080c9640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:36:35.935 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:35.933+0000 7f900e234640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f90080cf9b0 con 0x7f90080cef00 2023-08-09T13:36:35.935 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:35.933+0000 7f900e234640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f90080cf3b0 con 0x7f90080d00b0 2023-08-09T13:36:35.935 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:35.933+0000 7f900e234640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f90080cafd0 con 0x7f900815fc70 2023-08-09T13:36:35.935 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:35.933+0000 7f9007fff640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f90080cef00 0x7f90080c9640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:36:35.935 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:35.933+0000 7f9007fff640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f90080cef00 0x7f90080c9640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:45518/0 (socket says 172.21.15.43:45518) 2023-08-09T13:36:35.935 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:35.933+0000 7f9007fff640 1 -- 172.21.15.43:0/757036083 learned_addr learned my addr 172.21.15.43:0/757036083 (peer_addr_for_me v2:172.21.15.43:0/0) 2023-08-09T13:36:35.937 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:35.933+0000 7f8ffeffd640 1 --2- 172.21.15.43:0/757036083 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f900815fc70 0x7f90080ce9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:36:35.937 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:35.933+0000 7f90077fe640 1 --2- 172.21.15.43:0/757036083 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f90080d00b0 0x7f90080b7e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:36:35.937 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:35.934+0000 7f9007fff640 1 -- 172.21.15.43:0/757036083 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f900815fc70 msgr2=0x7f90080ce9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:36:35.937 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:35.934+0000 7f9007fff640 1 --2- 172.21.15.43:0/757036083 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f900815fc70 0x7f90080ce9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:36:35.937 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:35.934+0000 7f9007fff640 1 -- 172.21.15.43:0/757036083 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f90080d00b0 msgr2=0x7f90080b7e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:36:35.938 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:35.934+0000 7f9007fff640 1 --2- 172.21.15.43:0/757036083 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f90080d00b0 0x7f90080b7e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:36:35.938 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:35.934+0000 7f9007fff640 1 -- 172.21.15.43:0/757036083 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8ff8009a70 con 0x7f90080cef00 2023-08-09T13:36:35.938 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:35.934+0000 7f8ffeffd640 1 --2- 172.21.15.43:0/757036083 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f900815fc70 0x7f90080ce9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-08-09T13:36:35.938 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:35.934+0000 7f90077fe640 1 --2- 172.21.15.43:0/757036083 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f90080d00b0 0x7f90080b7e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-08-09T13:36:35.938 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:35.934+0000 7f9007fff640 1 --2- 172.21.15.43:0/757036083 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f90080cef00 0x7f90080c9640 secure :-1 s=READY pgs=630 cs=0 l=1 rev1=1 crypto rx=0x7f900000c430 tx=0x7f900000c8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-09T13:36:35.938 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:35.934+0000 7f9004ff9640 1 -- 172.21.15.43:0/757036083 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9000005ce0 con 0x7f90080cef00 2023-08-09T13:36:35.939 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:35.934+0000 7f9004ff9640 1 -- 172.21.15.43:0/757036083 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9000005e40 con 0x7f90080cef00 2023-08-09T13:36:35.939 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:35.934+0000 7f900e234640 1 -- 172.21.15.43:0/757036083 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f90080b8380 con 0x7f90080cef00 2023-08-09T13:36:35.939 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:35.935+0000 7f9004ff9640 1 -- 172.21.15.43:0/757036083 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f90000072d0 con 0x7f90080cef00 2023-08-09T13:36:35.939 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:35.935+0000 7f900e234640 1 -- 172.21.15.43:0/757036083 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f90080ca6a0 con 0x7f90080cef00 2023-08-09T13:36:35.939 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:35.936+0000 7f900e234640 1 -- 172.21.15.43:0/757036083 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f90080b5f10 con 0x7f90080cef00 2023-08-09T13:36:35.940 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:35.937+0000 7f9004ff9640 1 -- 172.21.15.43:0/757036083 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 10) v1 ==== 88517+0+0 (secure 0 0 0) 0x7f9000007430 con 0x7f90080cef00 2023-08-09T13:36:35.940 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:35.938+0000 7f9004ff9640 1 --2- 172.21.15.43:0/757036083 >> [v2:172.21.15.203:6832/998005540,v1:172.21.15.203:6833/998005540] conn(0x7f8fe806a250 0x7f8fe806c710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:36:35.940 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:35.938+0000 7f9004ff9640 1 -- 172.21.15.43:0/757036083 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 6640+0+0 (secure 0 0 0) 0x7f9000087ff0 con 0x7f90080cef00 2023-08-09T13:36:35.940 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:35.938+0000 7f90077fe640 1 --2- 172.21.15.43:0/757036083 >> [v2:172.21.15.203:6832/998005540,v1:172.21.15.203:6833/998005540] conn(0x7f8fe806a250 0x7f8fe806c710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:36:35.941 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:35.940+0000 7f90077fe640 1 --2- 172.21.15.43:0/757036083 >> [v2:172.21.15.203:6832/998005540,v1:172.21.15.203:6833/998005540] conn(0x7f8fe806a250 0x7f8fe806c710 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0x7f8ff8036000 tx=0x7f8ff8030000 comp rx=0 tx=0).ready entity=mgr.5003 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-09T13:36:35.944 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:35.943+0000 7f9004ff9640 1 -- 172.21.15.43:0/757036083 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178066 (secure 0 0 0) 0x7f90000532d0 con 0x7f90080cef00 2023-08-09T13:36:36.171 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:36.169+0000 7f900e234640 1 -- 172.21.15.43:0/757036083 --> [v2:172.21.15.203:6832/998005540,v1:172.21.15.203:6833/998005540] -- mgr_command(tid 0: {"prefix": "fs snap-schedule activate", "fs": "cephfs", "path": "/snap_test_dir1/dir0", "repeat": "1M", "target": ["mon-mgr", ""]}) v1 -- 0x7f9008073a60 con 0x7f8fe806a250 2023-08-09T13:36:36.301 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:36.299+0000 7f9004ff9640 1 -- 172.21.15.43:0/757036083 <== mgr.5003 v2:172.21.15.203:6832/998005540 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+49 (secure 0 0 0) 0x7f9008073a60 con 0x7f8fe806a250 2023-08-09T13:36:36.301 INFO:teuthology.orchestra.run.smithi043.stdout:Schedule activated for path /snap_test_dir1/dir0 2023-08-09T13:36:36.304 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:36.303+0000 7f900e234640 1 -- 172.21.15.43:0/757036083 >> [v2:172.21.15.203:6832/998005540,v1:172.21.15.203:6833/998005540] conn(0x7f8fe806a250 msgr2=0x7f8fe806c710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:36:36.305 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:36.303+0000 7f900e234640 1 --2- 172.21.15.43:0/757036083 >> [v2:172.21.15.203:6832/998005540,v1:172.21.15.203:6833/998005540] conn(0x7f8fe806a250 0x7f8fe806c710 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0x7f8ff8036000 tx=0x7f8ff8030000 comp rx=0 tx=0).stop 2023-08-09T13:36:36.305 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:36.303+0000 7f900e234640 1 -- 172.21.15.43:0/757036083 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f90080cef00 msgr2=0x7f90080c9640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:36:36.305 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:36.303+0000 7f900e234640 1 --2- 172.21.15.43:0/757036083 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f90080cef00 0x7f90080c9640 secure :-1 s=READY pgs=630 cs=0 l=1 rev1=1 crypto rx=0x7f900000c430 tx=0x7f900000c8f0 comp rx=0 tx=0).stop 2023-08-09T13:36:36.305 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:36.303+0000 7f900e234640 1 -- 172.21.15.43:0/757036083 shutdown_connections 2023-08-09T13:36:36.305 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:36.303+0000 7f900e234640 1 --2- 172.21.15.43:0/757036083 >> [v2:172.21.15.203:6832/998005540,v1:172.21.15.203:6833/998005540] conn(0x7f8fe806a250 0x7f8fe806c710 unknown :-1 s=CLOSED pgs=33 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:36:36.306 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:36.303+0000 7f900e234640 1 --2- 172.21.15.43:0/757036083 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f90080cef00 0x7f90080c9640 unknown :-1 s=CLOSED pgs=630 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:36:36.306 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:36.303+0000 7f900e234640 1 --2- 172.21.15.43:0/757036083 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f900815fc70 0x7f90080ce9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:36:36.306 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:36.303+0000 7f900e234640 1 --2- 172.21.15.43:0/757036083 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f90080d00b0 0x7f90080b7e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:36:36.306 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:36.303+0000 7f900e234640 1 -- 172.21.15.43:0/757036083 >> 172.21.15.43:0/757036083 conn(0x7f900809b7b0 msgr2=0x7f900815db90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-09T13:36:36.306 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:36.303+0000 7f900e234640 1 -- 172.21.15.43:0/757036083 shutdown_connections 2023-08-09T13:36:36.306 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:36.303+0000 7f900e234640 1 -- 172.21.15.43:0/757036083 wait complete. 2023-08-09T13:36:36.326 DEBUG:tasks.cephfs.test_snap_schedules:res=Schedule activated for path /snap_test_dir1/dir0 2023-08-09T13:36:36.326 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs snap-schedule activate --fs cephfs /snap_test_dir1/dir1 1M 2023-08-09T13:36:36.566 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:36.565+0000 7f1bcdf42640 1 Processor -- start 2023-08-09T13:36:36.567 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:36.565+0000 7f1bcdf42640 1 -- start start 2023-08-09T13:36:36.568 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:36.567+0000 7f1bcdf42640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f1bc815a560 0x7f1bc80d09d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:36:36.568 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:36.567+0000 7f1bcdf42640 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7f1bc80d0f10 0x7f1bc80cccd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:36:36.568 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:36.567+0000 7f1bcdf42640 1 -- --> v1:172.21.15.43:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1bc80cbde0 con 0x7f1bc80cd210 2023-08-09T13:36:36.568 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:36.567+0000 7f1bcdf42640 1 -- --> v2:172.21.15.203:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1bc80cd6c0 con 0x7f1bc80d0f10 2023-08-09T13:36:36.569 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:36.567+0000 7f1bcdf42640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1bc815a940 con 0x7f1bc815a560 2023-08-09T13:36:36.570 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:36.567+0000 7f1bc7fff640 1 --1- >> v1:172.21.15.43:6789/0 conn(0x7f1bc80cd210 0x7f1bc80b0f30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.43:6789/0 says I am v1:172.21.15.43:58102/0 (socket says 172.21.15.43:58102) 2023-08-09T13:36:36.570 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:36.567+0000 7f1bc7fff640 1 -- 172.21.15.43:0/3425439642 learned_addr learned my addr 172.21.15.43:0/3425439642 (peer_addr_for_me v1:172.21.15.43:0/0) 2023-08-09T13:36:36.570 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:36.567+0000 7f1bc77fe640 1 --2- 172.21.15.43:0/3425439642 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f1bc815a560 0x7f1bc80d09d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:36:36.570 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:36.568+0000 7f1bc6ffd640 1 --2- 172.21.15.43:0/3425439642 >> v2:172.21.15.203:3300/0 conn(0x7f1bc80d0f10 0x7f1bc80cccd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:36:36.571 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:36.568+0000 7f1bc67fc640 1 -- 172.21.15.43:0/3425439642 <== mon.0 v1:172.21.15.43:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 28967373 0 0) 0x7f1bc80cbde0 con 0x7f1bc80cd210 2023-08-09T13:36:36.571 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:36.568+0000 7f1bc67fc640 1 -- 172.21.15.43:0/3425439642 --> v1:172.21.15.43:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f1bb0003520 con 0x7f1bc80cd210 2023-08-09T13:36:36.571 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:36.568+0000 7f1bc67fc640 1 -- 172.21.15.43:0/3425439642 <== mon.0 v1:172.21.15.43:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2317612285 0 0) 0x7f1bb0003520 con 0x7f1bc80cd210 2023-08-09T13:36:36.571 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:36.568+0000 7f1bc67fc640 1 -- 172.21.15.43:0/3425439642 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f1bc815a560 msgr2=0x7f1bc80d09d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-09T13:36:36.571 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:36.568+0000 7f1bc67fc640 1 --2- 172.21.15.43:0/3425439642 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f1bc815a560 0x7f1bc80d09d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:36:36.571 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:36.568+0000 7f1bc67fc640 1 -- 172.21.15.43:0/3425439642 >> v2:172.21.15.203:3300/0 conn(0x7f1bc80d0f10 msgr2=0x7f1bc80cccd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-09T13:36:36.571 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:36.568+0000 7f1bc67fc640 1 --2- 172.21.15.43:0/3425439642 >> v2:172.21.15.203:3300/0 conn(0x7f1bc80d0f10 0x7f1bc80cccd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:36:36.572 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:36.568+0000 7f1bc67fc640 1 -- 172.21.15.43:0/3425439642 --> v1:172.21.15.43:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1bc80cb1d0 con 0x7f1bc80cd210 2023-08-09T13:36:36.572 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:36.568+0000 7f1bc67fc640 1 -- 172.21.15.43:0/3425439642 <== mon.0 v1:172.21.15.43:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1285150740 0 0) 0x7f1bc0001670 con 0x7f1bc80cd210 2023-08-09T13:36:36.572 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:36.568+0000 7f1bc67fc640 1 -- 172.21.15.43:0/3425439642 >> v1:172.21.15.43:6789/0 conn(0x7f1bc80cd210 legacy=0x7f1bc80b0f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:36:36.572 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:36.568+0000 7f1bc67fc640 1 --2- 172.21.15.43:0/3425439642 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f1bb00039e0 0x7f1bb0003e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:36:36.572 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:36.568+0000 7f1bc67fc640 1 --2- 172.21.15.43:0/3425439642 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f1bb00047c0 0x7f1bb0004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:36:36.573 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:36.568+0000 7f1bc67fc640 1 --2- 172.21.15.43:0/3425439642 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f1bc815a560 0x7f1bb0005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:36:36.573 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:36.568+0000 7f1bc67fc640 1 -- 172.21.15.43:0/3425439642 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1bc80cd6c0 con 0x7f1bb00047c0 2023-08-09T13:36:36.573 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:36.568+0000 7f1bc67fc640 1 -- 172.21.15.43:0/3425439642 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1bb000b980 con 0x7f1bb00039e0 2023-08-09T13:36:36.573 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:36.568+0000 7f1bc67fc640 1 -- 172.21.15.43:0/3425439642 --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1bb000bac0 con 0x7f1bc815a560 2023-08-09T13:36:36.573 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:36.569+0000 7f1bc6ffd640 1 --2- 172.21.15.43:0/3425439642 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f1bb00047c0 0x7f1bb0004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:36:36.575 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:36.569+0000 7f1bc77fe640 1 --2- 172.21.15.43:0/3425439642 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f1bb00039e0 0x7f1bb0003e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:36:36.575 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:36.569+0000 7f1bc7fff640 1 --2- 172.21.15.43:0/3425439642 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f1bc815a560 0x7f1bb0005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:36:36.575 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:36.569+0000 7f1bc6ffd640 1 -- 172.21.15.43:0/3425439642 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f1bc815a560 msgr2=0x7f1bb0005180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-09T13:36:36.575 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:36.569+0000 7f1bc6ffd640 1 --2- 172.21.15.43:0/3425439642 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f1bc815a560 0x7f1bb0005180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:36:36.575 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:36.569+0000 7f1bc6ffd640 1 -- 172.21.15.43:0/3425439642 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f1bb00039e0 msgr2=0x7f1bb0003e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-09T13:36:36.575 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:36.569+0000 7f1bc6ffd640 1 --2- 172.21.15.43:0/3425439642 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f1bb00039e0 0x7f1bb0003e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:36:36.576 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:36.569+0000 7f1bc6ffd640 1 -- 172.21.15.43:0/3425439642 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1bb0003520 con 0x7f1bb00047c0 2023-08-09T13:36:36.576 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:36.569+0000 7f1bc77fe640 1 --2- 172.21.15.43:0/3425439642 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f1bb00039e0 0x7f1bb0003e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-08-09T13:36:36.576 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:36.570+0000 7f1bc6ffd640 1 --2- 172.21.15.43:0/3425439642 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f1bb00047c0 0x7f1bb0004c40 secure :-1 s=READY pgs=632 cs=0 l=1 rev1=1 crypto rx=0x7f1bbc009af0 tx=0x7f1bb00057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cac8bdbc5721ae7c server_cookie=0 in_seq=0 out_seq=0 2023-08-09T13:36:36.576 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:36.571+0000 7f1bc67fc640 1 -- 172.21.15.43:0/3425439642 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1bbc02c710 con 0x7f1bb00047c0 2023-08-09T13:36:36.576 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:36.571+0000 7f1bc67fc640 1 -- 172.21.15.43:0/3425439642 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1bbc02c870 con 0x7f1bb00047c0 2023-08-09T13:36:36.576 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:36.571+0000 7f1bc67fc640 1 -- 172.21.15.43:0/3425439642 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1bbc02cc00 con 0x7f1bb00047c0 2023-08-09T13:36:36.577 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:36.571+0000 7f1bcdf42640 1 -- 172.21.15.43:0/3425439642 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f1bb00047c0 msgr2=0x7f1bb0004c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:36:36.577 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:36.571+0000 7f1bcdf42640 1 --2- 172.21.15.43:0/3425439642 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f1bb00047c0 0x7f1bb0004c40 secure :-1 s=READY pgs=632 cs=0 l=1 rev1=1 crypto rx=0x7f1bbc009af0 tx=0x7f1bb00057f0 comp rx=0 tx=0).stop 2023-08-09T13:36:36.577 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:36.571+0000 7f1bc7fff640 1 -- 172.21.15.43:0/3425439642 reap_dead start 2023-08-09T13:36:36.577 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:36.571+0000 7f1bcdf42640 1 -- 172.21.15.43:0/3425439642 shutdown_connections 2023-08-09T13:36:36.577 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:36.571+0000 7f1bcdf42640 1 -- 172.21.15.43:0/3425439642 >> 172.21.15.43:0/3425439642 conn(0x7f1bc8099600 msgr2=0x7f1bc80d02c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-09T13:36:36.577 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:36.572+0000 7f1bcdf42640 1 -- 172.21.15.43:0/3425439642 shutdown_connections 2023-08-09T13:36:36.577 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:36.572+0000 7f1bcdf42640 1 -- 172.21.15.43:0/3425439642 wait complete. 2023-08-09T13:36:36.578 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:36.573+0000 7f1bcdf42640 1 Processor -- start 2023-08-09T13:36:36.578 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:36.574+0000 7f1bcdf42640 1 -- start start 2023-08-09T13:36:36.579 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:36.574+0000 7f1bcdf42640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f1bc815d140 0x7f1bc809a070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:36:36.579 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:36.574+0000 7f1bcdf42640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f1bc815d520 0x7f1bc8148f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:36:36.580 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:36.574+0000 7f1bcdf42640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f1bc81f3c40 0x7f1bc80c8aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:36:36.580 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:36.574+0000 7f1bcdf42640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1bc81497a0 con 0x7f1bc815d140 2023-08-09T13:36:36.580 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:36.574+0000 7f1bcdf42640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1bc8149a80 con 0x7f1bc815d520 2023-08-09T13:36:36.580 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:36.574+0000 7f1bcdf42640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1bc8149d60 con 0x7f1bc81f3c40 2023-08-09T13:36:36.580 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:36.575+0000 7f1bc77fe640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f1bc815d140 0x7f1bc809a070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:36:36.580 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:36.575+0000 7f1bc77fe640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f1bc815d140 0x7f1bc809a070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:45536/0 (socket says 172.21.15.43:45536) 2023-08-09T13:36:36.581 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:36.575+0000 7f1bc77fe640 1 -- 172.21.15.43:0/185184618 learned_addr learned my addr 172.21.15.43:0/185184618 (peer_addr_for_me v2:172.21.15.43:0/0) 2023-08-09T13:36:36.581 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:36.575+0000 7f1bc6ffd640 1 --2- 172.21.15.43:0/185184618 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f1bc815d520 0x7f1bc8148f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:36:36.581 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:36.575+0000 7f1bc7fff640 1 --2- 172.21.15.43:0/185184618 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f1bc81f3c40 0x7f1bc80c8aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:36:36.581 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:36.575+0000 7f1bc77fe640 1 -- 172.21.15.43:0/185184618 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f1bc81f3c40 msgr2=0x7f1bc80c8aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:36:36.581 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:36.575+0000 7f1bc77fe640 1 --2- 172.21.15.43:0/185184618 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f1bc81f3c40 0x7f1bc80c8aa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:36:36.581 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:36.575+0000 7f1bc77fe640 1 -- 172.21.15.43:0/185184618 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f1bc815d520 msgr2=0x7f1bc8148f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:36:36.582 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:36.575+0000 7f1bc77fe640 1 --2- 172.21.15.43:0/185184618 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f1bc815d520 0x7f1bc8148f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:36:36.582 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:36.575+0000 7f1bc77fe640 1 -- 172.21.15.43:0/185184618 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1bc809cf30 con 0x7f1bc815d140 2023-08-09T13:36:36.582 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:36.575+0000 7f1bc6ffd640 1 --2- 172.21.15.43:0/185184618 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f1bc815d520 0x7f1bc8148f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-08-09T13:36:36.582 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:36.576+0000 7f1bc77fe640 1 --2- 172.21.15.43:0/185184618 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f1bc815d140 0x7f1bc809a070 secure :-1 s=READY pgs=633 cs=0 l=1 rev1=1 crypto rx=0x7f1bb800f350 tx=0x7f1bb800f810 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-09T13:36:36.582 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:36.576+0000 7f1baffff640 1 -- 172.21.15.43:0/185184618 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1bb8003b90 con 0x7f1bc815d140 2023-08-09T13:36:36.582 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:36.576+0000 7f1baffff640 1 -- 172.21.15.43:0/185184618 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1bb8011040 con 0x7f1bc815d140 2023-08-09T13:36:36.582 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:36.576+0000 7f1baffff640 1 -- 172.21.15.43:0/185184618 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1bb800fea0 con 0x7f1bc815d140 2023-08-09T13:36:36.583 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:36.576+0000 7f1bcdf42640 1 -- 172.21.15.43:0/185184618 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1bc809a5b0 con 0x7f1bc815d140 2023-08-09T13:36:36.583 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:36.576+0000 7f1bcdf42640 1 -- 172.21.15.43:0/185184618 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1bc809cb50 con 0x7f1bc815d140 2023-08-09T13:36:36.583 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:36.577+0000 7f1bcdf42640 1 -- 172.21.15.43:0/185184618 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1bc809cd40 con 0x7f1bc815d140 2023-08-09T13:36:36.583 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:36.578+0000 7f1baffff640 1 -- 172.21.15.43:0/185184618 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 10) v1 ==== 88517+0+0 (secure 0 0 0) 0x7f1bb8004020 con 0x7f1bc815d140 2023-08-09T13:36:36.584 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:36.579+0000 7f1baffff640 1 --2- 172.21.15.43:0/185184618 >> [v2:172.21.15.203:6832/998005540,v1:172.21.15.203:6833/998005540] conn(0x7f1bb406a280 0x7f1bb406c740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:36:36.584 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:36.579+0000 7f1baffff640 1 -- 172.21.15.43:0/185184618 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 6640+0+0 (secure 0 0 0) 0x7f1bb8088fe0 con 0x7f1bc815d140 2023-08-09T13:36:36.584 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:36.580+0000 7f1bc6ffd640 1 --2- 172.21.15.43:0/185184618 >> [v2:172.21.15.203:6832/998005540,v1:172.21.15.203:6833/998005540] conn(0x7f1bb406a280 0x7f1bb406c740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:36:36.584 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:36.581+0000 7f1bc6ffd640 1 --2- 172.21.15.43:0/185184618 >> [v2:172.21.15.203:6832/998005540,v1:172.21.15.203:6833/998005540] conn(0x7f1bb406a280 0x7f1bb406c740 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0x7f1bbc036000 tx=0x7f1bbc030000 comp rx=0 tx=0).ready entity=mgr.5003 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-09T13:36:36.585 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:36.584+0000 7f1baffff640 1 -- 172.21.15.43:0/185184618 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178066 (secure 0 0 0) 0x7f1bb8054160 con 0x7f1bc815d140 2023-08-09T13:36:36.803 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:36.802+0000 7f1bcdf42640 1 -- 172.21.15.43:0/185184618 --> [v2:172.21.15.203:6832/998005540,v1:172.21.15.203:6833/998005540] -- mgr_command(tid 0: {"prefix": "fs snap-schedule activate", "fs": "cephfs", "path": "/snap_test_dir1/dir1", "repeat": "1M", "target": ["mon-mgr", ""]}) v1 -- 0x7f1bc8149bf0 con 0x7f1bb406a280 2023-08-09T13:36:36.931 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:36.930+0000 7f1baffff640 1 -- 172.21.15.43:0/185184618 <== mgr.5003 v2:172.21.15.203:6832/998005540 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+49 (secure 0 0 0) 0x7f1bc8149bf0 con 0x7f1bb406a280 2023-08-09T13:36:36.932 INFO:teuthology.orchestra.run.smithi043.stdout:Schedule activated for path /snap_test_dir1/dir1 2023-08-09T13:36:36.935 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:36.933+0000 7f1bcdf42640 1 -- 172.21.15.43:0/185184618 >> [v2:172.21.15.203:6832/998005540,v1:172.21.15.203:6833/998005540] conn(0x7f1bb406a280 msgr2=0x7f1bb406c740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:36:36.936 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:36.933+0000 7f1bcdf42640 1 --2- 172.21.15.43:0/185184618 >> [v2:172.21.15.203:6832/998005540,v1:172.21.15.203:6833/998005540] conn(0x7f1bb406a280 0x7f1bb406c740 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0x7f1bbc036000 tx=0x7f1bbc030000 comp rx=0 tx=0).stop 2023-08-09T13:36:36.936 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:36.933+0000 7f1bcdf42640 1 -- 172.21.15.43:0/185184618 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f1bc815d140 msgr2=0x7f1bc809a070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:36:36.936 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:36.933+0000 7f1bcdf42640 1 --2- 172.21.15.43:0/185184618 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f1bc815d140 0x7f1bc809a070 secure :-1 s=READY pgs=633 cs=0 l=1 rev1=1 crypto rx=0x7f1bb800f350 tx=0x7f1bb800f810 comp rx=0 tx=0).stop 2023-08-09T13:36:36.936 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:36.934+0000 7f1bcdf42640 1 -- 172.21.15.43:0/185184618 shutdown_connections 2023-08-09T13:36:36.936 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:36.934+0000 7f1bcdf42640 1 --2- 172.21.15.43:0/185184618 >> [v2:172.21.15.203:6832/998005540,v1:172.21.15.203:6833/998005540] conn(0x7f1bb406a280 0x7f1bb406c740 unknown :-1 s=CLOSED pgs=34 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:36:36.936 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:36.934+0000 7f1bcdf42640 1 --2- 172.21.15.43:0/185184618 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f1bc81f3c40 0x7f1bc80c8aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:36:36.937 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:36.934+0000 7f1bcdf42640 1 --2- 172.21.15.43:0/185184618 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f1bc815d520 0x7f1bc8148f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:36:36.937 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:36.934+0000 7f1bcdf42640 1 --2- 172.21.15.43:0/185184618 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f1bc815d140 0x7f1bc809a070 unknown :-1 s=CLOSED pgs=633 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:36:36.937 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:36.934+0000 7f1bcdf42640 1 -- 172.21.15.43:0/185184618 >> 172.21.15.43:0/185184618 conn(0x7f1bc8099600 msgr2=0x7f1bc80b53f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-09T13:36:36.937 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:36.934+0000 7f1bcdf42640 1 -- 172.21.15.43:0/185184618 shutdown_connections 2023-08-09T13:36:36.937 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:36.934+0000 7f1bcdf42640 1 -- 172.21.15.43:0/185184618 wait complete. 2023-08-09T13:36:36.957 DEBUG:tasks.cephfs.test_snap_schedules:res=Schedule activated for path /snap_test_dir1/dir1 2023-08-09T13:36:36.958 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs snap-schedule activate --fs cephfs /snap_test_dir1/dir2 1M 2023-08-09T13:36:37.188 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:37.187+0000 7f5efdd02640 1 Processor -- start 2023-08-09T13:36:37.189 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:37.187+0000 7f5efdd02640 1 -- start start 2023-08-09T13:36:37.190 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:37.189+0000 7f5efdd02640 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7f5ef80cd3c0 0x7f5ef80cec70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:36:37.191 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:37.189+0000 7f5efdd02640 1 -- --> v1:172.21.15.43:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5ef80cf5e0 con 0x7f5ef80cf1b0 2023-08-09T13:36:37.191 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:37.189+0000 7f5efdd02640 1 -- --> v1:172.21.15.203:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5ef80d13e0 con 0x7f5ef815c400 2023-08-09T13:36:37.191 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:37.189+0000 7f5efdd02640 1 -- --> v2:172.21.15.203:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5ef815c7e0 con 0x7f5ef80cd3c0 2023-08-09T13:36:37.192 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:37.189+0000 7f5efd501640 1 --1- >> v1:172.21.15.43:6789/0 conn(0x7f5ef80cf1b0 0x7f5ef80b5880 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.43:6789/0 says I am v1:172.21.15.43:58104/0 (socket says 172.21.15.43:58104) 2023-08-09T13:36:37.192 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:37.189+0000 7f5efd501640 1 -- 172.21.15.43:0/2945975997 learned_addr learned my addr 172.21.15.43:0/2945975997 (peer_addr_for_me v1:172.21.15.43:0/0) 2023-08-09T13:36:37.192 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:37.190+0000 7f5eeffff640 1 --2- 172.21.15.43:0/2945975997 >> v2:172.21.15.203:3300/0 conn(0x7f5ef80cd3c0 0x7f5ef80cec70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:36:37.192 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:37.190+0000 7f5eef7fe640 1 -- 172.21.15.43:0/2945975997 <== mon.0 v1:172.21.15.43:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 591125621 0 0) 0x7f5ef80cf5e0 con 0x7f5ef80cf1b0 2023-08-09T13:36:37.193 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:37.190+0000 7f5eef7fe640 1 -- 172.21.15.43:0/2945975997 --> v1:172.21.15.43:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f5ee0003520 con 0x7f5ef80cf1b0 2023-08-09T13:36:37.193 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:37.190+0000 7f5eef7fe640 1 -- 172.21.15.43:0/2945975997 <== mon.0 v1:172.21.15.43:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 996336500 0 0) 0x7f5ee0003520 con 0x7f5ef80cf1b0 2023-08-09T13:36:37.193 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:37.190+0000 7f5eef7fe640 1 -- 172.21.15.43:0/2945975997 >> v2:172.21.15.203:3300/0 conn(0x7f5ef80cd3c0 msgr2=0x7f5ef80cec70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-09T13:36:37.193 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:37.190+0000 7f5eef7fe640 1 --2- 172.21.15.43:0/2945975997 >> v2:172.21.15.203:3300/0 conn(0x7f5ef80cd3c0 0x7f5ef80cec70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:36:37.193 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:37.190+0000 7f5eef7fe640 1 -- 172.21.15.43:0/2945975997 >> v1:172.21.15.203:6789/0 conn(0x7f5ef815c400 legacy=0x7f5ef80d0cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:36:37.193 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:37.190+0000 7f5eef7fe640 1 -- 172.21.15.43:0/2945975997 --> v1:172.21.15.43:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5ef80d1690 con 0x7f5ef80cf1b0 2023-08-09T13:36:37.193 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:37.190+0000 7f5eef7fe640 1 -- 172.21.15.43:0/2945975997 <== mon.0 v1:172.21.15.43:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1285150740 0 0) 0x7f5ef0001670 con 0x7f5ef80cf1b0 2023-08-09T13:36:37.194 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:37.190+0000 7f5eef7fe640 1 -- 172.21.15.43:0/2945975997 >> v1:172.21.15.43:6789/0 conn(0x7f5ef80cf1b0 legacy=0x7f5ef80b5880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:36:37.194 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:37.191+0000 7f5eef7fe640 1 --2- 172.21.15.43:0/2945975997 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f5ee0003970 0x7f5ee0003d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:36:37.194 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:37.191+0000 7f5eef7fe640 1 --2- 172.21.15.43:0/2945975997 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f5ee00047c0 0x7f5ee0004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:36:37.194 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:37.191+0000 7f5eef7fe640 1 --2- 172.21.15.43:0/2945975997 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f5ee0005180 0x7f5ee000b980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:36:37.194 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:37.191+0000 7f5eef7fe640 1 -- 172.21.15.43:0/2945975997 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5ef815c7e0 con 0x7f5ee0005180 2023-08-09T13:36:37.194 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:37.191+0000 7f5eef7fe640 1 -- 172.21.15.43:0/2945975997 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5ee000c030 con 0x7f5ee0003970 2023-08-09T13:36:37.194 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:37.191+0000 7f5eef7fe640 1 -- 172.21.15.43:0/2945975997 --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5ee000c1f0 con 0x7f5ee00047c0 2023-08-09T13:36:37.196 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:37.191+0000 7f5efd501640 1 --2- 172.21.15.43:0/2945975997 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f5ee0005180 0x7f5ee000b980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:36:37.196 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:37.191+0000 7f5efcd00640 1 --2- 172.21.15.43:0/2945975997 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f5ee0003970 0x7f5ee0003d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:36:37.196 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:37.191+0000 7f5eeffff640 1 --2- 172.21.15.43:0/2945975997 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f5ee00047c0 0x7f5ee0004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:36:37.196 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:37.191+0000 7f5efd501640 1 -- 172.21.15.43:0/2945975997 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f5ee00047c0 msgr2=0x7f5ee0004c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-09T13:36:37.196 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:37.191+0000 7f5efd501640 1 --2- 172.21.15.43:0/2945975997 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f5ee00047c0 0x7f5ee0004c40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:36:37.197 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:37.191+0000 7f5efd501640 1 -- 172.21.15.43:0/2945975997 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f5ee0003970 msgr2=0x7f5ee0003d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-09T13:36:37.197 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:37.191+0000 7f5efd501640 1 --2- 172.21.15.43:0/2945975997 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f5ee0003970 0x7f5ee0003d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:36:37.197 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:37.192+0000 7f5efd501640 1 -- 172.21.15.43:0/2945975997 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5ee0003520 con 0x7f5ee0005180 2023-08-09T13:36:37.197 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:37.192+0000 7f5efd501640 1 -- 172.21.15.43:0/2945975997 reap_dead start 2023-08-09T13:36:37.197 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:37.193+0000 7f5efd501640 1 --2- 172.21.15.43:0/2945975997 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f5ee0005180 0x7f5ee000b980 secure :-1 s=READY pgs=635 cs=0 l=1 rev1=1 crypto rx=0x7f5ef000c240 tx=0x7f5ee00057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3d56afb216f7e351 server_cookie=0 in_seq=0 out_seq=0 2023-08-09T13:36:37.197 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:37.193+0000 7f5eef7fe640 1 -- 172.21.15.43:0/2945975997 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5ef0005ad0 con 0x7f5ee0005180 2023-08-09T13:36:37.197 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:37.193+0000 7f5eef7fe640 1 -- 172.21.15.43:0/2945975997 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5ef0005c70 con 0x7f5ee0005180 2023-08-09T13:36:37.197 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:37.193+0000 7f5eef7fe640 1 -- 172.21.15.43:0/2945975997 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5ef002c8b0 con 0x7f5ee0005180 2023-08-09T13:36:37.198 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:37.193+0000 7f5efdd02640 1 -- 172.21.15.43:0/2945975997 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f5ee0005180 msgr2=0x7f5ee000b980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:36:37.198 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:37.193+0000 7f5efdd02640 1 --2- 172.21.15.43:0/2945975997 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f5ee0005180 0x7f5ee000b980 secure :-1 s=READY pgs=635 cs=0 l=1 rev1=1 crypto rx=0x7f5ef000c240 tx=0x7f5ee00057f0 comp rx=0 tx=0).stop 2023-08-09T13:36:37.198 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:37.194+0000 7f5efdd02640 1 -- 172.21.15.43:0/2945975997 shutdown_connections 2023-08-09T13:36:37.198 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:37.194+0000 7f5efdd02640 1 --2- 172.21.15.43:0/2945975997 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f5ee0005180 0x7f5ee000b980 unknown :-1 s=CLOSED pgs=635 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:36:37.198 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:37.194+0000 7f5efdd02640 1 -- 172.21.15.43:0/2945975997 >> 172.21.15.43:0/2945975997 conn(0x7f5ef806a320 msgr2=0x7f5ef806a730 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-09T13:36:37.198 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:37.194+0000 7f5efdd02640 1 -- 172.21.15.43:0/2945975997 shutdown_connections 2023-08-09T13:36:37.198 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:37.194+0000 7f5efdd02640 1 -- 172.21.15.43:0/2945975997 wait complete. 2023-08-09T13:36:37.200 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:37.195+0000 7f5efdd02640 1 Processor -- start 2023-08-09T13:36:37.200 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:37.196+0000 7f5efdd02640 1 -- start start 2023-08-09T13:36:37.200 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:37.197+0000 7f5efdd02640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f5ee0005180 0x7f5ef80cb040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:36:37.200 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:37.197+0000 7f5efdd02640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f5ef80b18e0 0x7f5ef8155e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:36:37.201 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:37.197+0000 7f5efdd02640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f5ef80c9350 0x7f5ef815e4b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:36:37.201 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:37.197+0000 7f5efdd02640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5ef8159cd0 con 0x7f5ef80b18e0 2023-08-09T13:36:37.201 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:37.197+0000 7f5efdd02640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5ef80cb580 con 0x7f5ef80c9350 2023-08-09T13:36:37.201 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:37.197+0000 7f5efdd02640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5ef80cb820 con 0x7f5ee0005180 2023-08-09T13:36:37.201 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:37.197+0000 7f5eeffff640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f5ef80b18e0 0x7f5ef8155e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:36:37.201 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:37.197+0000 7f5eeffff640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f5ef80b18e0 0x7f5ef8155e70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:45546/0 (socket says 172.21.15.43:45546) 2023-08-09T13:36:37.201 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:37.197+0000 7f5eeffff640 1 -- 172.21.15.43:0/3942707008 learned_addr learned my addr 172.21.15.43:0/3942707008 (peer_addr_for_me v2:172.21.15.43:0/0) 2023-08-09T13:36:37.202 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:37.197+0000 7f5efd501640 1 --2- 172.21.15.43:0/3942707008 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f5ef80c9350 0x7f5ef815e4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:36:37.202 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:37.197+0000 7f5efcd00640 1 --2- 172.21.15.43:0/3942707008 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f5ee0005180 0x7f5ef80cb040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:36:37.202 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:37.197+0000 7f5eeffff640 1 -- 172.21.15.43:0/3942707008 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f5ee0005180 msgr2=0x7f5ef80cb040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:36:37.202 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:37.197+0000 7f5eeffff640 1 --2- 172.21.15.43:0/3942707008 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f5ee0005180 0x7f5ef80cb040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:36:37.202 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:37.198+0000 7f5eeffff640 1 -- 172.21.15.43:0/3942707008 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f5ef80c9350 msgr2=0x7f5ef815e4b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:36:37.202 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:37.198+0000 7f5eeffff640 1 --2- 172.21.15.43:0/3942707008 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f5ef80c9350 0x7f5ef815e4b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:36:37.202 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:37.198+0000 7f5eeffff640 1 -- 172.21.15.43:0/3942707008 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5ef80cbb00 con 0x7f5ef80b18e0 2023-08-09T13:36:37.202 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:37.198+0000 7f5eeffff640 1 --2- 172.21.15.43:0/3942707008 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f5ef80b18e0 0x7f5ef8155e70 secure :-1 s=READY pgs=636 cs=0 l=1 rev1=1 crypto rx=0x7f5ee8010860 tx=0x7f5ee8010d20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-09T13:36:37.203 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:37.198+0000 7f5eed7fa640 1 -- 172.21.15.43:0/3942707008 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5ee8009610 con 0x7f5ef80b18e0 2023-08-09T13:36:37.203 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:37.198+0000 7f5eed7fa640 1 -- 172.21.15.43:0/3942707008 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5ee800eb60 con 0x7f5ef80b18e0 2023-08-09T13:36:37.203 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:37.198+0000 7f5eed7fa640 1 -- 172.21.15.43:0/3942707008 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5ee801c400 con 0x7f5ef80b18e0 2023-08-09T13:36:37.203 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:37.198+0000 7f5efdd02640 1 -- 172.21.15.43:0/3942707008 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5ef80cbde0 con 0x7f5ef80b18e0 2023-08-09T13:36:37.203 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:37.198+0000 7f5efdd02640 1 -- 172.21.15.43:0/3942707008 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5ef81563b0 con 0x7f5ef80b18e0 2023-08-09T13:36:37.204 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:37.199+0000 7f5efdd02640 1 -- 172.21.15.43:0/3942707008 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5ef80739f0 con 0x7f5ef80b18e0 2023-08-09T13:36:37.204 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:37.201+0000 7f5eed7fa640 1 -- 172.21.15.43:0/3942707008 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 10) v1 ==== 88517+0+0 (secure 0 0 0) 0x7f5ee801c560 con 0x7f5ef80b18e0 2023-08-09T13:36:37.204 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:37.202+0000 7f5eed7fa640 1 --2- 172.21.15.43:0/3942707008 >> [v2:172.21.15.203:6832/998005540,v1:172.21.15.203:6833/998005540] conn(0x7f5ee406a250 0x7f5ee406c710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:36:37.204 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:37.203+0000 7f5eed7fa640 1 -- 172.21.15.43:0/3942707008 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 6640+0+0 (secure 0 0 0) 0x7f5ee80890e0 con 0x7f5ef80b18e0 2023-08-09T13:36:37.205 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:37.203+0000 7f5efcd00640 1 --2- 172.21.15.43:0/3942707008 >> [v2:172.21.15.203:6832/998005540,v1:172.21.15.203:6833/998005540] conn(0x7f5ee406a250 0x7f5ee406c710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:36:37.206 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:37.204+0000 7f5efcd00640 1 --2- 172.21.15.43:0/3942707008 >> [v2:172.21.15.203:6832/998005540,v1:172.21.15.203:6833/998005540] conn(0x7f5ee406a250 0x7f5ee406c710 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7f5ef81e50a0 tx=0x7f5ef4009040 comp rx=0 tx=0).ready entity=mgr.5003 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-09T13:36:37.209 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:37.207+0000 7f5eed7fa640 1 -- 172.21.15.43:0/3942707008 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178066 (secure 0 0 0) 0x7f5ee80543d0 con 0x7f5ef80b18e0 2023-08-09T13:36:37.426 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:37.424+0000 7f5efdd02640 1 -- 172.21.15.43:0/3942707008 --> [v2:172.21.15.203:6832/998005540,v1:172.21.15.203:6833/998005540] -- mgr_command(tid 0: {"prefix": "fs snap-schedule activate", "fs": "cephfs", "path": "/snap_test_dir1/dir2", "repeat": "1M", "target": ["mon-mgr", ""]}) v1 -- 0x7f5ef80d1c50 con 0x7f5ee406a250 2023-08-09T13:36:37.555 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:37.553+0000 7f5eed7fa640 1 -- 172.21.15.43:0/3942707008 <== mgr.5003 v2:172.21.15.203:6832/998005540 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+49 (secure 0 0 0) 0x7f5ef400d050 con 0x7f5ee406a250 2023-08-09T13:36:37.555 INFO:teuthology.orchestra.run.smithi043.stdout:Schedule activated for path /snap_test_dir1/dir2 2023-08-09T13:36:37.559 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:37.558+0000 7f5efdd02640 1 -- 172.21.15.43:0/3942707008 >> [v2:172.21.15.203:6832/998005540,v1:172.21.15.203:6833/998005540] conn(0x7f5ee406a250 msgr2=0x7f5ee406c710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:36:37.560 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:37.558+0000 7f5efdd02640 1 --2- 172.21.15.43:0/3942707008 >> [v2:172.21.15.203:6832/998005540,v1:172.21.15.203:6833/998005540] conn(0x7f5ee406a250 0x7f5ee406c710 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7f5ef81e50a0 tx=0x7f5ef4009040 comp rx=0 tx=0).stop 2023-08-09T13:36:37.560 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:37.558+0000 7f5efdd02640 1 -- 172.21.15.43:0/3942707008 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f5ef80b18e0 msgr2=0x7f5ef8155e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:36:37.560 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:37.558+0000 7f5efdd02640 1 --2- 172.21.15.43:0/3942707008 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f5ef80b18e0 0x7f5ef8155e70 secure :-1 s=READY pgs=636 cs=0 l=1 rev1=1 crypto rx=0x7f5ee8010860 tx=0x7f5ee8010d20 comp rx=0 tx=0).stop 2023-08-09T13:36:37.560 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:37.558+0000 7f5efdd02640 1 -- 172.21.15.43:0/3942707008 shutdown_connections 2023-08-09T13:36:37.560 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:37.558+0000 7f5efdd02640 1 --2- 172.21.15.43:0/3942707008 >> [v2:172.21.15.203:6832/998005540,v1:172.21.15.203:6833/998005540] conn(0x7f5ee406a250 0x7f5ee406c710 unknown :-1 s=CLOSED pgs=35 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:36:37.560 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:37.558+0000 7f5efdd02640 1 --2- 172.21.15.43:0/3942707008 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f5ef80c9350 0x7f5ef815e4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:36:37.561 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:37.558+0000 7f5efdd02640 1 --2- 172.21.15.43:0/3942707008 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f5ef80b18e0 0x7f5ef8155e70 unknown :-1 s=CLOSED pgs=636 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:36:37.561 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:37.558+0000 7f5efdd02640 1 --2- 172.21.15.43:0/3942707008 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f5ee0005180 0x7f5ef80cb040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:36:37.561 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:37.558+0000 7f5efdd02640 1 -- 172.21.15.43:0/3942707008 >> 172.21.15.43:0/3942707008 conn(0x7f5ef806a320 msgr2=0x7f5ef815b410 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-09T13:36:37.561 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:37.558+0000 7f5efdd02640 1 -- 172.21.15.43:0/3942707008 shutdown_connections 2023-08-09T13:36:37.562 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:37.558+0000 7f5efdd02640 1 -- 172.21.15.43:0/3942707008 wait complete. 2023-08-09T13:36:37.581 DEBUG:tasks.cephfs.test_snap_schedules:res=Schedule activated for path /snap_test_dir1/dir2 2023-08-09T13:36:37.581 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs snap-schedule activate --fs cephfs /snap_test_dir1/dir3 1M 2023-08-09T13:36:37.825 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:37.824+0000 7fbfa0613640 1 Processor -- start 2023-08-09T13:36:37.826 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:37.824+0000 7fbfa0613640 1 -- start start 2023-08-09T13:36:37.827 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:37.826+0000 7fbfa0613640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7fbf98153590 0x7fbf980cc3f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:36:37.827 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:37.826+0000 7fbfa0613640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fbf980cebd0 0x7fbf980ca540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:36:37.828 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:37.826+0000 7fbfa0613640 1 -- --> v1:172.21.15.43:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbf980c9510 con 0x7fbf980cbd70 2023-08-09T13:36:37.828 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:37.826+0000 7fbfa0613640 1 -- --> v2:172.21.15.43:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbf980c86f0 con 0x7fbf98153590 2023-08-09T13:36:37.828 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:37.826+0000 7fbfa0613640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbf980c99a0 con 0x7fbf980cebd0 2023-08-09T13:36:37.829 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:37.826+0000 7fbf9e3af640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7fbf98153590 0x7fbf980cc3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:36:37.829 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:37.826+0000 7fbf9e3af640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7fbf98153590 0x7fbf980cc3f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:45562/0 (socket says 172.21.15.43:45562) 2023-08-09T13:36:37.829 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:37.826+0000 7fbf9e3af640 1 -- 172.21.15.43:0/933298620 learned_addr learned my addr 172.21.15.43:0/933298620 (peer_addr_for_me v2:172.21.15.43:0/0) 2023-08-09T13:36:37.829 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:37.827+0000 7fbf9ebb0640 1 --2- 172.21.15.43:0/933298620 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fbf980cebd0 0x7fbf980ca540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:36:37.829 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:37.827+0000 7fbf9e3af640 1 -- 172.21.15.43:0/933298620 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fbf980cebd0 msgr2=0x7fbf980ca540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-09T13:36:37.829 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:37.827+0000 7fbf9e3af640 1 --2- 172.21.15.43:0/933298620 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fbf980cebd0 0x7fbf980ca540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:36:37.830 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:37.827+0000 7fbf9e3af640 1 -- 172.21.15.43:0/933298620 >> v1:172.21.15.43:6789/0 conn(0x7fbf980cbd70 legacy=0x7fbf980ce4c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:36:37.830 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:37.827+0000 7fbf9e3af640 1 -- 172.21.15.43:0/933298620 --> v2:172.21.15.43:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbf980c8900 con 0x7fbf98153590 2023-08-09T13:36:37.830 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:37.827+0000 7fbf9ebb0640 1 --2- 172.21.15.43:0/933298620 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fbf980cebd0 0x7fbf980ca540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-08-09T13:36:37.831 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:37.828+0000 7fbf9e3af640 1 --2- 172.21.15.43:0/933298620 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fbf98153590 0x7fbf980cc3f0 secure :-1 s=READY pgs=638 cs=0 l=1 rev1=1 crypto rx=0x7fbf94009870 tx=0x7fbf9402b980 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5e1265f11b7d7367 server_cookie=0 in_seq=0 out_seq=0 2023-08-09T13:36:37.831 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:37.829+0000 7fbf9d3ad640 1 -- 172.21.15.43:0/933298620 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbf94030840 con 0x7fbf98153590 2023-08-09T13:36:37.831 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:37.829+0000 7fbf9d3ad640 1 -- 172.21.15.43:0/933298620 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbf940309a0 con 0x7fbf98153590 2023-08-09T13:36:37.831 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:37.829+0000 7fbf9d3ad640 1 -- 172.21.15.43:0/933298620 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbf94030d30 con 0x7fbf98153590 2023-08-09T13:36:37.831 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:37.829+0000 7fbfa0613640 1 -- 172.21.15.43:0/933298620 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fbf98153590 msgr2=0x7fbf980cc3f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:36:37.832 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:37.829+0000 7fbfa0613640 1 --2- 172.21.15.43:0/933298620 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fbf98153590 0x7fbf980cc3f0 secure :-1 s=READY pgs=638 cs=0 l=1 rev1=1 crypto rx=0x7fbf94009870 tx=0x7fbf9402b980 comp rx=0 tx=0).stop 2023-08-09T13:36:37.832 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:37.829+0000 7fbfa0613640 1 -- 172.21.15.43:0/933298620 shutdown_connections 2023-08-09T13:36:37.832 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:37.829+0000 7fbfa0613640 1 --2- 172.21.15.43:0/933298620 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fbf980cebd0 0x7fbf980ca540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:36:37.832 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:37.829+0000 7fbfa0613640 1 --2- 172.21.15.43:0/933298620 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fbf98153590 0x7fbf980cc3f0 unknown :-1 s=CLOSED pgs=638 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:36:37.832 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:37.829+0000 7fbfa0613640 1 -- 172.21.15.43:0/933298620 >> 172.21.15.43:0/933298620 conn(0x7fbf9806b990 msgr2=0x7fbf9806bda0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-09T13:36:37.832 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:37.829+0000 7fbfa0613640 1 -- 172.21.15.43:0/933298620 shutdown_connections 2023-08-09T13:36:37.832 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:37.829+0000 7fbfa0613640 1 -- 172.21.15.43:0/933298620 wait complete. 2023-08-09T13:36:37.833 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:37.831+0000 7fbfa0613640 1 Processor -- start 2023-08-09T13:36:37.833 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:37.831+0000 7fbfa0613640 1 -- start start 2023-08-09T13:36:37.834 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:37.832+0000 7fbfa0613640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fbf980cbd70 0x7fbf980c79e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:36:37.834 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:37.832+0000 7fbfa0613640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fbf980cebd0 0x7fbf98153970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:36:37.834 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:37.832+0000 7fbfa0613640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fbf98153590 0x7fbf98153eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:36:37.834 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:37.832+0000 7fbfa0613640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbf980c1ba0 con 0x7fbf980cebd0 2023-08-09T13:36:37.834 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:37.832+0000 7fbfa0613640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbf980c7f20 con 0x7fbf980cbd70 2023-08-09T13:36:37.835 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:37.832+0000 7fbfa0613640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbf980c1df0 con 0x7fbf98153590 2023-08-09T13:36:37.835 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:37.833+0000 7fbf9dbae640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fbf980cebd0 0x7fbf98153970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:36:37.835 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:37.833+0000 7fbf9dbae640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fbf980cebd0 0x7fbf98153970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:45576/0 (socket says 172.21.15.43:45576) 2023-08-09T13:36:37.835 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:37.833+0000 7fbf9dbae640 1 -- 172.21.15.43:0/1961139332 learned_addr learned my addr 172.21.15.43:0/1961139332 (peer_addr_for_me v2:172.21.15.43:0/0) 2023-08-09T13:36:37.836 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:37.833+0000 7fbf9ebb0640 1 --2- 172.21.15.43:0/1961139332 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fbf98153590 0x7fbf98153eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:36:37.836 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:37.833+0000 7fbf9e3af640 1 --2- 172.21.15.43:0/1961139332 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fbf980cbd70 0x7fbf980c79e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:36:37.836 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:37.833+0000 7fbf9dbae640 1 -- 172.21.15.43:0/1961139332 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fbf98153590 msgr2=0x7fbf98153eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:36:37.837 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:37.833+0000 7fbf9dbae640 1 --2- 172.21.15.43:0/1961139332 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fbf98153590 0x7fbf98153eb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:36:37.837 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:37.833+0000 7fbf9dbae640 1 -- 172.21.15.43:0/1961139332 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fbf980cbd70 msgr2=0x7fbf980c79e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:36:37.837 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:37.833+0000 7fbf9dbae640 1 --2- 172.21.15.43:0/1961139332 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fbf980cbd70 0x7fbf980c79e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:36:37.837 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:37.833+0000 7fbf9dbae640 1 -- 172.21.15.43:0/1961139332 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbf940094d0 con 0x7fbf980cebd0 2023-08-09T13:36:37.837 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:37.833+0000 7fbf9e3af640 1 --2- 172.21.15.43:0/1961139332 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fbf980cbd70 0x7fbf980c79e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-08-09T13:36:37.837 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:37.833+0000 7fbf9ebb0640 1 --2- 172.21.15.43:0/1961139332 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fbf98153590 0x7fbf98153eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-08-09T13:36:37.837 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:37.833+0000 7fbf9dbae640 1 --2- 172.21.15.43:0/1961139332 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fbf980cebd0 0x7fbf98153970 secure :-1 s=READY pgs=639 cs=0 l=1 rev1=1 crypto rx=0x7fbf8800c450 tx=0x7fbf8800c910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-09T13:36:37.838 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:37.834+0000 7fbf8effd640 1 -- 172.21.15.43:0/1961139332 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbf880052d0 con 0x7fbf980cebd0 2023-08-09T13:36:37.838 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:37.834+0000 7fbf8effd640 1 -- 172.21.15.43:0/1961139332 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbf88005430 con 0x7fbf980cebd0 2023-08-09T13:36:37.838 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:37.834+0000 7fbf8effd640 1 -- 172.21.15.43:0/1961139332 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbf88012870 con 0x7fbf980cebd0 2023-08-09T13:36:37.838 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:37.834+0000 7fbfa0613640 1 -- 172.21.15.43:0/1961139332 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbf980c20d0 con 0x7fbf980cebd0 2023-08-09T13:36:37.838 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:37.834+0000 7fbfa0613640 1 -- 172.21.15.43:0/1961139332 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbf980c2690 con 0x7fbf980cebd0 2023-08-09T13:36:37.838 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:37.835+0000 7fbfa0613640 1 -- 172.21.15.43:0/1961139332 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbf980739f0 con 0x7fbf980cebd0 2023-08-09T13:36:37.839 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:37.836+0000 7fbf8effd640 1 -- 172.21.15.43:0/1961139332 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 10) v1 ==== 88517+0+0 (secure 0 0 0) 0x7fbf880129d0 con 0x7fbf980cebd0 2023-08-09T13:36:37.839 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:37.837+0000 7fbf8effd640 1 --2- 172.21.15.43:0/1961139332 >> [v2:172.21.15.203:6832/998005540,v1:172.21.15.203:6833/998005540] conn(0x7fbf7c06a250 0x7fbf7c06c710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:36:37.839 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:37.837+0000 7fbf8effd640 1 -- 172.21.15.43:0/1961139332 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 6640+0+0 (secure 0 0 0) 0x7fbf88088690 con 0x7fbf980cebd0 2023-08-09T13:36:37.839 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:37.838+0000 7fbf9e3af640 1 --2- 172.21.15.43:0/1961139332 >> [v2:172.21.15.203:6832/998005540,v1:172.21.15.203:6833/998005540] conn(0x7fbf7c06a250 0x7fbf7c06c710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:36:37.840 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:37.839+0000 7fbf9e3af640 1 --2- 172.21.15.43:0/1961139332 >> [v2:172.21.15.203:6832/998005540,v1:172.21.15.203:6833/998005540] conn(0x7fbf7c06a250 0x7fbf7c06c710 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7fbf94036000 tx=0x7fbf9403b040 comp rx=0 tx=0).ready entity=mgr.5003 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-09T13:36:37.842 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:37.841+0000 7fbf8effd640 1 -- 172.21.15.43:0/1961139332 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178066 (secure 0 0 0) 0x7fbf880538d0 con 0x7fbf980cebd0 2023-08-09T13:36:38.061 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:38.059+0000 7fbfa0613640 1 -- 172.21.15.43:0/1961139332 --> [v2:172.21.15.203:6832/998005540,v1:172.21.15.203:6833/998005540] -- mgr_command(tid 0: {"prefix": "fs snap-schedule activate", "fs": "cephfs", "path": "/snap_test_dir1/dir3", "repeat": "1M", "target": ["mon-mgr", ""]}) v1 -- 0x7fbf980c8ec0 con 0x7fbf7c06a250 2023-08-09T13:36:38.183 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:38.181+0000 7fbf8effd640 1 -- 172.21.15.43:0/1961139332 <== mgr.5003 v2:172.21.15.203:6832/998005540 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+49 (secure 0 0 0) 0x7fbf94030cd0 con 0x7fbf7c06a250 2023-08-09T13:36:38.183 INFO:teuthology.orchestra.run.smithi043.stdout:Schedule activated for path /snap_test_dir1/dir3 2023-08-09T13:36:38.186 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:38.184+0000 7fbfa0613640 1 -- 172.21.15.43:0/1961139332 >> [v2:172.21.15.203:6832/998005540,v1:172.21.15.203:6833/998005540] conn(0x7fbf7c06a250 msgr2=0x7fbf7c06c710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:36:38.186 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:38.184+0000 7fbfa0613640 1 --2- 172.21.15.43:0/1961139332 >> [v2:172.21.15.203:6832/998005540,v1:172.21.15.203:6833/998005540] conn(0x7fbf7c06a250 0x7fbf7c06c710 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7fbf94036000 tx=0x7fbf9403b040 comp rx=0 tx=0).stop 2023-08-09T13:36:38.187 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:38.184+0000 7fbfa0613640 1 -- 172.21.15.43:0/1961139332 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fbf980cebd0 msgr2=0x7fbf98153970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:36:38.187 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:38.184+0000 7fbfa0613640 1 --2- 172.21.15.43:0/1961139332 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fbf980cebd0 0x7fbf98153970 secure :-1 s=READY pgs=639 cs=0 l=1 rev1=1 crypto rx=0x7fbf8800c450 tx=0x7fbf8800c910 comp rx=0 tx=0).stop 2023-08-09T13:36:38.187 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:38.185+0000 7fbfa0613640 1 -- 172.21.15.43:0/1961139332 shutdown_connections 2023-08-09T13:36:38.187 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:38.185+0000 7fbfa0613640 1 --2- 172.21.15.43:0/1961139332 >> [v2:172.21.15.203:6832/998005540,v1:172.21.15.203:6833/998005540] conn(0x7fbf7c06a250 0x7fbf7c06c710 unknown :-1 s=CLOSED pgs=36 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:36:38.187 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:38.185+0000 7fbfa0613640 1 --2- 172.21.15.43:0/1961139332 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fbf98153590 0x7fbf98153eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:36:38.187 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:38.185+0000 7fbfa0613640 1 --2- 172.21.15.43:0/1961139332 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fbf980cebd0 0x7fbf98153970 unknown :-1 s=CLOSED pgs=639 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:36:38.187 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:38.185+0000 7fbfa0613640 1 --2- 172.21.15.43:0/1961139332 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fbf980cbd70 0x7fbf980c79e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:36:38.188 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:38.185+0000 7fbfa0613640 1 -- 172.21.15.43:0/1961139332 >> 172.21.15.43:0/1961139332 conn(0x7fbf9806b990 msgr2=0x7fbf9815dd20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-09T13:36:38.188 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:38.185+0000 7fbfa0613640 1 -- 172.21.15.43:0/1961139332 shutdown_connections 2023-08-09T13:36:38.188 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:38.185+0000 7fbfa0613640 1 -- 172.21.15.43:0/1961139332 wait complete. 2023-08-09T13:36:38.208 DEBUG:tasks.cephfs.test_snap_schedules:res=Schedule activated for path /snap_test_dir1/dir3 2023-08-09T13:36:38.209 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs snap-schedule activate --fs cephfs /snap_test_dir1/dir4 1M 2023-08-09T13:36:38.437 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:38.436+0000 7f6641b02640 1 Processor -- start 2023-08-09T13:36:38.438 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:38.436+0000 7f6641b02640 1 -- start start 2023-08-09T13:36:38.439 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:38.437+0000 7f6641b02640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f663c06b400 0x7f663c06cae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:36:38.439 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:38.437+0000 7f6641b02640 1 -- --> v1:172.21.15.43:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f663c0ad260 con 0x7f663c15c4f0 2023-08-09T13:36:38.439 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:38.438+0000 7f6641b02640 1 -- --> v1:172.21.15.203:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f663c06b9e0 con 0x7f663c15c8e0 2023-08-09T13:36:38.439 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:38.438+0000 7f6641b02640 1 -- --> v2:172.21.15.43:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f663c06c280 con 0x7f663c06b400 2023-08-09T13:36:38.440 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:38.438+0000 7f663bfff640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f663c06b400 0x7f663c06cae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:36:38.440 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:38.438+0000 7f663bfff640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f663c06b400 0x7f663c06cae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:45586/0 (socket says 172.21.15.43:45586) 2023-08-09T13:36:38.440 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:38.438+0000 7f663bfff640 1 -- 172.21.15.43:0/4114126198 learned_addr learned my addr 172.21.15.43:0/4114126198 (peer_addr_for_me v2:172.21.15.43:0/0) 2023-08-09T13:36:38.441 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:38.438+0000 7f663bfff640 1 -- 172.21.15.43:0/4114126198 >> v1:172.21.15.203:6789/0 conn(0x7f663c15c8e0 legacy=0x7f663c0af870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-09T13:36:38.441 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:38.438+0000 7f663bfff640 1 -- 172.21.15.43:0/4114126198 >> v1:172.21.15.43:6789/0 conn(0x7f663c15c4f0 legacy=0x7f663c160590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-09T13:36:38.441 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:38.438+0000 7f663bfff640 1 -- 172.21.15.43:0/4114126198 --> v2:172.21.15.43:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f663c06bcc0 con 0x7f663c06b400 2023-08-09T13:36:38.441 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:38.440+0000 7f663bfff640 1 --2- 172.21.15.43:0/4114126198 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f663c06b400 0x7f663c06cae0 secure :-1 s=READY pgs=641 cs=0 l=1 rev1=1 crypto rx=0x7f66280098a0 tx=0x7f662802b8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b90f868e705a9bb5 server_cookie=0 in_seq=0 out_seq=0 2023-08-09T13:36:38.442 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:38.440+0000 7f663b7fe640 1 -- 172.21.15.43:0/4114126198 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6628031640 con 0x7f663c06b400 2023-08-09T13:36:38.442 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:38.440+0000 7f663b7fe640 1 -- 172.21.15.43:0/4114126198 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f66280317a0 con 0x7f663c06b400 2023-08-09T13:36:38.442 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:38.440+0000 7f663b7fe640 1 -- 172.21.15.43:0/4114126198 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6628031b30 con 0x7f663c06b400 2023-08-09T13:36:38.443 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:38.441+0000 7f6641b02640 1 -- 172.21.15.43:0/4114126198 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f663c06b400 msgr2=0x7f663c06cae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:36:38.443 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:38.441+0000 7f6641b02640 1 --2- 172.21.15.43:0/4114126198 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f663c06b400 0x7f663c06cae0 secure :-1 s=READY pgs=641 cs=0 l=1 rev1=1 crypto rx=0x7f66280098a0 tx=0x7f662802b8f0 comp rx=0 tx=0).stop 2023-08-09T13:36:38.443 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:38.441+0000 7f6641b02640 1 -- 172.21.15.43:0/4114126198 shutdown_connections 2023-08-09T13:36:38.443 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:38.441+0000 7f6641b02640 1 --2- 172.21.15.43:0/4114126198 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f663c06b400 0x7f663c06cae0 unknown :-1 s=CLOSED pgs=641 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:36:38.443 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:38.441+0000 7f6641b02640 1 -- 172.21.15.43:0/4114126198 >> 172.21.15.43:0/4114126198 conn(0x7f663c0ca870 msgr2=0x7f663c0cac50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-09T13:36:38.443 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:38.441+0000 7f6641b02640 1 -- 172.21.15.43:0/4114126198 shutdown_connections 2023-08-09T13:36:38.444 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:38.441+0000 7f6641b02640 1 -- 172.21.15.43:0/4114126198 wait complete. 2023-08-09T13:36:38.444 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:38.443+0000 7f6641b02640 1 Processor -- start 2023-08-09T13:36:38.444 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:38.443+0000 7f6641b02640 1 -- start start 2023-08-09T13:36:38.446 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:38.444+0000 7f6641b02640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f663c06b400 0x7f663c0ae1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:36:38.446 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:38.444+0000 7f6641b02640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f663c15c8e0 0x7f663c1539d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:36:38.446 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:38.444+0000 7f6641b02640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f663c1580c0 0x7f663c0ac2e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:36:38.446 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:38.444+0000 7f6641b02640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f663c0ae6e0 con 0x7f663c15c8e0 2023-08-09T13:36:38.446 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:38.444+0000 7f6641b02640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f663c153f10 con 0x7f663c06b400 2023-08-09T13:36:38.446 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:38.444+0000 7f6641b02640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f663c158690 con 0x7f663c1580c0 2023-08-09T13:36:38.448 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:38.445+0000 7f663bfff640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f663c15c8e0 0x7f663c1539d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:36:38.448 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:38.445+0000 7f663bfff640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f663c15c8e0 0x7f663c1539d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:45598/0 (socket says 172.21.15.43:45598) 2023-08-09T13:36:38.448 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:38.445+0000 7f663bfff640 1 -- 172.21.15.43:0/2842308855 learned_addr learned my addr 172.21.15.43:0/2842308855 (peer_addr_for_me v2:172.21.15.43:0/0) 2023-08-09T13:36:38.448 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:38.445+0000 7f6641301640 1 --2- 172.21.15.43:0/2842308855 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f663c1580c0 0x7f663c0ac2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:36:38.448 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:38.445+0000 7f6640b00640 1 --2- 172.21.15.43:0/2842308855 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f663c06b400 0x7f663c0ae1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:36:38.448 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:38.445+0000 7f663bfff640 1 -- 172.21.15.43:0/2842308855 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f663c1580c0 msgr2=0x7f663c0ac2e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:36:38.448 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:38.445+0000 7f663bfff640 1 --2- 172.21.15.43:0/2842308855 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f663c1580c0 0x7f663c0ac2e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:36:38.449 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:38.445+0000 7f663bfff640 1 -- 172.21.15.43:0/2842308855 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f663c06b400 msgr2=0x7f663c0ae1a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:36:38.449 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:38.445+0000 7f663bfff640 1 --2- 172.21.15.43:0/2842308855 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f663c06b400 0x7f663c0ae1a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:36:38.449 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:38.445+0000 7f663bfff640 1 -- 172.21.15.43:0/2842308855 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f66280094d0 con 0x7f663c15c8e0 2023-08-09T13:36:38.449 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:38.446+0000 7f663bfff640 1 --2- 172.21.15.43:0/2842308855 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f663c15c8e0 0x7f663c1539d0 secure :-1 s=READY pgs=642 cs=0 l=1 rev1=1 crypto rx=0x7f6628000c00 tx=0x7f66280046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-09T13:36:38.449 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:38.446+0000 7f66397fa640 1 -- 172.21.15.43:0/2842308855 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f662803ac60 con 0x7f663c15c8e0 2023-08-09T13:36:38.449 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:38.446+0000 7f66397fa640 1 -- 172.21.15.43:0/2842308855 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f662803adc0 con 0x7f663c15c8e0 2023-08-09T13:36:38.449 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:38.446+0000 7f66397fa640 1 -- 172.21.15.43:0/2842308855 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6628044630 con 0x7f663c15c8e0 2023-08-09T13:36:38.450 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:38.446+0000 7f6641b02640 1 -- 172.21.15.43:0/2842308855 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f663c158970 con 0x7f663c15c8e0 2023-08-09T13:36:38.450 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:38.446+0000 7f6641b02640 1 -- 172.21.15.43:0/2842308855 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f663c158f30 con 0x7f663c15c8e0 2023-08-09T13:36:38.450 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:38.447+0000 7f6641b02640 1 -- 172.21.15.43:0/2842308855 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f663c0739f0 con 0x7f663c15c8e0 2023-08-09T13:36:38.450 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:38.448+0000 7f66397fa640 1 -- 172.21.15.43:0/2842308855 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 10) v1 ==== 88517+0+0 (secure 0 0 0) 0x7f662802f050 con 0x7f663c15c8e0 2023-08-09T13:36:38.451 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:38.449+0000 7f66397fa640 1 --2- 172.21.15.43:0/2842308855 >> [v2:172.21.15.203:6832/998005540,v1:172.21.15.203:6833/998005540] conn(0x7f662006a250 0x7f662006c710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:36:38.451 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:38.449+0000 7f66397fa640 1 -- 172.21.15.43:0/2842308855 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 6640+0+0 (secure 0 0 0) 0x7f66280aace0 con 0x7f663c15c8e0 2023-08-09T13:36:38.451 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:38.450+0000 7f6640b00640 1 --2- 172.21.15.43:0/2842308855 >> [v2:172.21.15.203:6832/998005540,v1:172.21.15.203:6833/998005540] conn(0x7f662006a250 0x7f662006c710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:36:38.452 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:38.451+0000 7f6640b00640 1 --2- 172.21.15.43:0/2842308855 >> [v2:172.21.15.203:6832/998005540,v1:172.21.15.203:6833/998005540] conn(0x7f662006a250 0x7f662006c710 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7f6630004ea0 tx=0x7f6630009040 comp rx=0 tx=0).ready entity=mgr.5003 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-09T13:36:38.455 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:38.453+0000 7f66397fa640 1 -- 172.21.15.43:0/2842308855 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178066 (secure 0 0 0) 0x7f6628075d80 con 0x7f663c15c8e0 2023-08-09T13:36:38.673 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:38.671+0000 7f6641b02640 1 -- 172.21.15.43:0/2842308855 --> [v2:172.21.15.203:6832/998005540,v1:172.21.15.203:6833/998005540] -- mgr_command(tid 0: {"prefix": "fs snap-schedule activate", "fs": "cephfs", "path": "/snap_test_dir1/dir4", "repeat": "1M", "target": ["mon-mgr", ""]}) v1 -- 0x7f663c160070 con 0x7f662006a250 2023-08-09T13:36:38.795 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:38.793+0000 7f66397fa640 1 -- 172.21.15.43:0/2842308855 <== mgr.5003 v2:172.21.15.203:6832/998005540 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+49 (secure 0 0 0) 0x7f6630005420 con 0x7f662006a250 2023-08-09T13:36:38.795 INFO:teuthology.orchestra.run.smithi043.stdout:Schedule activated for path /snap_test_dir1/dir4 2023-08-09T13:36:38.798 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:38.796+0000 7f6641b02640 1 -- 172.21.15.43:0/2842308855 >> [v2:172.21.15.203:6832/998005540,v1:172.21.15.203:6833/998005540] conn(0x7f662006a250 msgr2=0x7f662006c710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:36:38.798 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:38.796+0000 7f6641b02640 1 --2- 172.21.15.43:0/2842308855 >> [v2:172.21.15.203:6832/998005540,v1:172.21.15.203:6833/998005540] conn(0x7f662006a250 0x7f662006c710 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7f6630004ea0 tx=0x7f6630009040 comp rx=0 tx=0).stop 2023-08-09T13:36:38.799 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:38.797+0000 7f6641b02640 1 -- 172.21.15.43:0/2842308855 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f663c15c8e0 msgr2=0x7f663c1539d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:36:38.799 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:38.797+0000 7f6641b02640 1 --2- 172.21.15.43:0/2842308855 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f663c15c8e0 0x7f663c1539d0 secure :-1 s=READY pgs=642 cs=0 l=1 rev1=1 crypto rx=0x7f6628000c00 tx=0x7f66280046c0 comp rx=0 tx=0).stop 2023-08-09T13:36:38.799 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:38.797+0000 7f6641b02640 1 -- 172.21.15.43:0/2842308855 shutdown_connections 2023-08-09T13:36:38.799 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:38.797+0000 7f6641b02640 1 --2- 172.21.15.43:0/2842308855 >> [v2:172.21.15.203:6832/998005540,v1:172.21.15.203:6833/998005540] conn(0x7f662006a250 0x7f662006c710 unknown :-1 s=CLOSED pgs=37 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:36:38.799 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:38.797+0000 7f6641b02640 1 --2- 172.21.15.43:0/2842308855 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f663c1580c0 0x7f663c0ac2e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:36:38.799 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:38.797+0000 7f6641b02640 1 --2- 172.21.15.43:0/2842308855 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f663c15c8e0 0x7f663c1539d0 unknown :-1 s=CLOSED pgs=642 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:36:38.799 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:38.797+0000 7f6641b02640 1 --2- 172.21.15.43:0/2842308855 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f663c06b400 0x7f663c0ae1a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:36:38.800 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:38.797+0000 7f6641b02640 1 -- 172.21.15.43:0/2842308855 >> 172.21.15.43:0/2842308855 conn(0x7f663c0ca870 msgr2=0x7f663c0ab8d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-09T13:36:38.800 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:38.797+0000 7f6641b02640 1 -- 172.21.15.43:0/2842308855 shutdown_connections 2023-08-09T13:36:38.800 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:38.797+0000 7f6641b02640 1 -- 172.21.15.43:0/2842308855 wait complete. 2023-08-09T13:36:38.821 DEBUG:tasks.cephfs.test_snap_schedules:res=Schedule activated for path /snap_test_dir1/dir4 2023-08-09T13:36:38.821 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs snap-schedule activate --fs cephfs /snap_test_dir1/dir5 1M 2023-08-09T13:36:39.069 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:39.067+0000 7f10abfff640 1 Processor -- start 2023-08-09T13:36:39.069 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:39.067+0000 7f10abfff640 1 -- start start 2023-08-09T13:36:39.070 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:39.069+0000 7f10abfff640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f10ac15c720 0x7f10ac0cd850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:36:39.070 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:39.069+0000 7f10abfff640 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7f10ac0d0900 0x7f10ac0cec60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:36:39.071 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:39.069+0000 7f10abfff640 1 -- --> v1:172.21.15.43:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f10ac0b5660 con 0x7f10ac0cf1a0 2023-08-09T13:36:39.071 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:39.069+0000 7f10abfff640 1 -- --> v2:172.21.15.203:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f10ac15cb00 con 0x7f10ac0d0900 2023-08-09T13:36:39.071 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:39.069+0000 7f10abfff640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f10ac0cf630 con 0x7f10ac15c720 2023-08-09T13:36:39.072 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:39.069+0000 7f10ab7fe640 1 --1- >> v1:172.21.15.43:6789/0 conn(0x7f10ac0cf1a0 0x7f10ac0b3cd0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.43:6789/0 says I am v1:172.21.15.43:58138/0 (socket says 172.21.15.43:58138) 2023-08-09T13:36:39.073 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:39.069+0000 7f10ab7fe640 1 -- 172.21.15.43:0/2325924409 learned_addr learned my addr 172.21.15.43:0/2325924409 (peer_addr_for_me v1:172.21.15.43:0/0) 2023-08-09T13:36:39.073 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:39.070+0000 7f10aaffd640 1 --2- 172.21.15.43:0/2325924409 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f10ac15c720 0x7f10ac0cd850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:36:39.073 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:39.070+0000 7f10aa7fc640 1 --2- 172.21.15.43:0/2325924409 >> v2:172.21.15.203:3300/0 conn(0x7f10ac0d0900 0x7f10ac0cec60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:36:39.073 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:39.070+0000 7f10a9ffb640 1 -- 172.21.15.43:0/2325924409 <== mon.0 v1:172.21.15.43:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2074283870 0 0) 0x7f10ac0b5660 con 0x7f10ac0cf1a0 2023-08-09T13:36:39.073 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:39.070+0000 7f10a9ffb640 1 -- 172.21.15.43:0/2325924409 --> v1:172.21.15.43:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f1094003520 con 0x7f10ac0cf1a0 2023-08-09T13:36:39.073 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:39.070+0000 7f10a9ffb640 1 -- 172.21.15.43:0/2325924409 <== mon.0 v1:172.21.15.43:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1516616431 0 0) 0x7f1094003520 con 0x7f10ac0cf1a0 2023-08-09T13:36:39.073 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:39.070+0000 7f10a9ffb640 1 -- 172.21.15.43:0/2325924409 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f10ac15c720 msgr2=0x7f10ac0cd850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-09T13:36:39.074 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:39.070+0000 7f10a9ffb640 1 --2- 172.21.15.43:0/2325924409 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f10ac15c720 0x7f10ac0cd850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:36:39.074 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:39.070+0000 7f10a9ffb640 1 -- 172.21.15.43:0/2325924409 >> v2:172.21.15.203:3300/0 conn(0x7f10ac0d0900 msgr2=0x7f10ac0cec60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-09T13:36:39.074 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:39.070+0000 7f10a9ffb640 1 --2- 172.21.15.43:0/2325924409 >> v2:172.21.15.203:3300/0 conn(0x7f10ac0d0900 0x7f10ac0cec60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:36:39.074 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:39.070+0000 7f10a9ffb640 1 -- 172.21.15.43:0/2325924409 --> v1:172.21.15.43:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f10ac0b5900 con 0x7f10ac0cf1a0 2023-08-09T13:36:39.074 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:39.070+0000 7f10a9ffb640 1 -- 172.21.15.43:0/2325924409 <== mon.0 v1:172.21.15.43:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1285150740 0 0) 0x7f10a40019f0 con 0x7f10ac0cf1a0 2023-08-09T13:36:39.074 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:39.070+0000 7f10a9ffb640 1 -- 172.21.15.43:0/2325924409 >> v1:172.21.15.43:6789/0 conn(0x7f10ac0cf1a0 legacy=0x7f10ac0b3cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:36:39.074 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:39.070+0000 7f10a9ffb640 1 --2- 172.21.15.43:0/2325924409 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f10ac15c720 0x7f1094003970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:36:39.075 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:39.070+0000 7f10a9ffb640 1 --2- 172.21.15.43:0/2325924409 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f1094003ef0 0x7f10940047c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:36:39.075 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:39.070+0000 7f10a9ffb640 1 --2- 172.21.15.43:0/2325924409 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f1094004d00 0x7f1094005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:36:39.075 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:39.071+0000 7f10a9ffb640 1 -- 172.21.15.43:0/2325924409 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f10ac15cb00 con 0x7f1094003ef0 2023-08-09T13:36:39.075 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:39.071+0000 7f10a9ffb640 1 -- 172.21.15.43:0/2325924409 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f10ac0cf630 con 0x7f1094004d00 2023-08-09T13:36:39.075 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:39.071+0000 7f10a9ffb640 1 -- 172.21.15.43:0/2325924409 --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f109400b980 con 0x7f10ac15c720 2023-08-09T13:36:39.075 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:39.071+0000 7f10aa7fc640 1 --2- 172.21.15.43:0/2325924409 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f1094003ef0 0x7f10940047c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:36:39.076 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:39.071+0000 7f10aaffd640 1 --2- 172.21.15.43:0/2325924409 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f10ac15c720 0x7f1094003970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:36:39.076 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:39.071+0000 7f10aa7fc640 1 -- 172.21.15.43:0/2325924409 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f10ac15c720 msgr2=0x7f1094003970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-09T13:36:39.076 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:39.071+0000 7f10aa7fc640 1 --2- 172.21.15.43:0/2325924409 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f10ac15c720 0x7f1094003970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:36:39.076 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:39.071+0000 7f10aa7fc640 1 -- 172.21.15.43:0/2325924409 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f1094004d00 msgr2=0x7f1094005180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-09T13:36:39.076 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:39.071+0000 7f10aa7fc640 1 --2- 172.21.15.43:0/2325924409 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f1094004d00 0x7f1094005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:36:39.076 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:39.071+0000 7f10aa7fc640 1 -- 172.21.15.43:0/2325924409 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1094003520 con 0x7f1094003ef0 2023-08-09T13:36:39.077 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:39.072+0000 7f10aa7fc640 1 --2- 172.21.15.43:0/2325924409 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f1094003ef0 0x7f10940047c0 secure :-1 s=READY pgs=644 cs=0 l=1 rev1=1 crypto rx=0x7f10a0009ac0 tx=0x7f10940057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2e70cfd426f64237 server_cookie=0 in_seq=0 out_seq=0 2023-08-09T13:36:39.077 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:39.073+0000 7f10a9ffb640 1 -- 172.21.15.43:0/2325924409 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f10a002c6d0 con 0x7f1094003ef0 2023-08-09T13:36:39.078 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:39.073+0000 7f10a9ffb640 1 -- 172.21.15.43:0/2325924409 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f10a002c830 con 0x7f1094003ef0 2023-08-09T13:36:39.078 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:39.073+0000 7f10a9ffb640 1 -- 172.21.15.43:0/2325924409 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f10a002cbc0 con 0x7f1094003ef0 2023-08-09T13:36:39.078 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:39.073+0000 7f10abfff640 1 -- 172.21.15.43:0/2325924409 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f1094003ef0 msgr2=0x7f10940047c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:36:39.078 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:39.073+0000 7f10abfff640 1 --2- 172.21.15.43:0/2325924409 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f1094003ef0 0x7f10940047c0 secure :-1 s=READY pgs=644 cs=0 l=1 rev1=1 crypto rx=0x7f10a0009ac0 tx=0x7f10940057f0 comp rx=0 tx=0).stop 2023-08-09T13:36:39.078 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:39.073+0000 7f10ab7fe640 1 -- 172.21.15.43:0/2325924409 reap_dead start 2023-08-09T13:36:39.079 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:39.073+0000 7f10abfff640 1 -- 172.21.15.43:0/2325924409 shutdown_connections 2023-08-09T13:36:39.079 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:39.073+0000 7f10abfff640 1 -- 172.21.15.43:0/2325924409 >> 172.21.15.43:0/2325924409 conn(0x7f10ac06a2a0 msgr2=0x7f10ac06a680 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-09T13:36:39.079 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:39.073+0000 7f10abfff640 1 -- 172.21.15.43:0/2325924409 shutdown_connections 2023-08-09T13:36:39.079 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:39.073+0000 7f10abfff640 1 -- 172.21.15.43:0/2325924409 wait complete. 2023-08-09T13:36:39.079 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:39.075+0000 7f10abfff640 1 Processor -- start 2023-08-09T13:36:39.079 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:39.076+0000 7f10abfff640 1 -- start start 2023-08-09T13:36:39.081 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:39.077+0000 7f10abfff640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f1094003ef0 0x7f10ac0c5cd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:36:39.081 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:39.077+0000 7f10abfff640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f10ac1f5040 0x7f10ac1f54c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:36:39.081 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:39.077+0000 7f10abfff640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f10ac1f5a00 0x7f10ac0ca8a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:36:39.081 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:39.077+0000 7f10abfff640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f10ac1f5eb0 con 0x7f1094003ef0 2023-08-09T13:36:39.081 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:39.077+0000 7f10abfff640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f10ac0c8c80 con 0x7f10ac1f5040 2023-08-09T13:36:39.081 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:39.077+0000 7f10abfff640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f10ac0c83e0 con 0x7f10ac1f5a00 2023-08-09T13:36:39.081 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:39.077+0000 7f10aaffd640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f1094003ef0 0x7f10ac0c5cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:36:39.082 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:39.077+0000 7f10aaffd640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f1094003ef0 0x7f10ac0c5cd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:45610/0 (socket says 172.21.15.43:45610) 2023-08-09T13:36:39.082 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:39.077+0000 7f10aaffd640 1 -- 172.21.15.43:0/1604015472 learned_addr learned my addr 172.21.15.43:0/1604015472 (peer_addr_for_me v2:172.21.15.43:0/0) 2023-08-09T13:36:39.082 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:39.077+0000 7f10aaffd640 1 -- 172.21.15.43:0/1604015472 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f10ac1f5a00 msgr2=0x7f10ac0ca8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:36:39.082 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:39.077+0000 7f10aaffd640 1 --2- 172.21.15.43:0/1604015472 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f10ac1f5a00 0x7f10ac0ca8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:36:39.082 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:39.077+0000 7f10aaffd640 1 -- 172.21.15.43:0/1604015472 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f10ac1f5040 msgr2=0x7f10ac1f54c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:36:39.082 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:39.077+0000 7f10aaffd640 1 --2- 172.21.15.43:0/1604015472 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f10ac1f5040 0x7f10ac1f54c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:36:39.083 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:39.077+0000 7f10aaffd640 1 -- 172.21.15.43:0/1604015472 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f10ac0c86c0 con 0x7f1094003ef0 2023-08-09T13:36:39.083 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:39.078+0000 7f10aaffd640 1 --2- 172.21.15.43:0/1604015472 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f1094003ef0 0x7f10ac0c5cd0 secure :-1 s=READY pgs=645 cs=0 l=1 rev1=1 crypto rx=0x7f109c011600 tx=0x7f109c011ac0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-09T13:36:39.083 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:39.078+0000 7f1093fff640 1 -- 172.21.15.43:0/1604015472 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f109c00f920 con 0x7f1094003ef0 2023-08-09T13:36:39.083 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:39.078+0000 7f1093fff640 1 -- 172.21.15.43:0/1604015472 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f109c00fa80 con 0x7f1094003ef0 2023-08-09T13:36:39.083 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:39.078+0000 7f1093fff640 1 -- 172.21.15.43:0/1604015472 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f109c00fdf0 con 0x7f1094003ef0 2023-08-09T13:36:39.084 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:39.078+0000 7f10abfff640 1 -- 172.21.15.43:0/1604015472 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f10ac0c89a0 con 0x7f1094003ef0 2023-08-09T13:36:39.084 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:39.078+0000 7f10abfff640 1 -- 172.21.15.43:0/1604015472 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f10ac0c6210 con 0x7f1094003ef0 2023-08-09T13:36:39.084 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:39.079+0000 7f10abfff640 1 -- 172.21.15.43:0/1604015472 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f10ac0739f0 con 0x7f1094003ef0 2023-08-09T13:36:39.085 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:39.080+0000 7f1093fff640 1 -- 172.21.15.43:0/1604015472 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 10) v1 ==== 88517+0+0 (secure 0 0 0) 0x7f109c015460 con 0x7f1094003ef0 2023-08-09T13:36:39.085 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:39.082+0000 7f1093fff640 1 --2- 172.21.15.43:0/1604015472 >> [v2:172.21.15.203:6832/998005540,v1:172.21.15.203:6833/998005540] conn(0x7f109806a280 0x7f109806c740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:36:39.085 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:39.082+0000 7f1093fff640 1 -- 172.21.15.43:0/1604015472 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 6640+0+0 (secure 0 0 0) 0x7f109c088fd0 con 0x7f1094003ef0 2023-08-09T13:36:39.085 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:39.082+0000 7f10aa7fc640 1 --2- 172.21.15.43:0/1604015472 >> [v2:172.21.15.203:6832/998005540,v1:172.21.15.203:6833/998005540] conn(0x7f109806a280 0x7f109806c740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:36:39.085 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:39.084+0000 7f10aa7fc640 1 --2- 172.21.15.43:0/1604015472 >> [v2:172.21.15.203:6832/998005540,v1:172.21.15.203:6833/998005540] conn(0x7f109806a280 0x7f109806c740 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7f10a0036000 tx=0x7f10a002e000 comp rx=0 tx=0).ready entity=mgr.5003 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-09T13:36:39.087 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:39.086+0000 7f1093fff640 1 -- 172.21.15.43:0/1604015472 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178066 (secure 0 0 0) 0x7f109c054180 con 0x7f1094003ef0 2023-08-09T13:36:39.305 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:39.304+0000 7f10abfff640 1 -- 172.21.15.43:0/1604015472 --> [v2:172.21.15.203:6832/998005540,v1:172.21.15.203:6833/998005540] -- mgr_command(tid 0: {"prefix": "fs snap-schedule activate", "fs": "cephfs", "path": "/snap_test_dir1/dir5", "repeat": "1M", "target": ["mon-mgr", ""]}) v1 -- 0x7f10ac0c6430 con 0x7f109806a280 2023-08-09T13:36:39.425 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:39.423+0000 7f1093fff640 1 -- 172.21.15.43:0/1604015472 <== mgr.5003 v2:172.21.15.203:6832/998005540 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+49 (secure 0 0 0) 0x7f10a00040c0 con 0x7f109806a280 2023-08-09T13:36:39.425 INFO:teuthology.orchestra.run.smithi043.stdout:Schedule activated for path /snap_test_dir1/dir5 2023-08-09T13:36:39.428 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:39.427+0000 7f10abfff640 1 -- 172.21.15.43:0/1604015472 >> [v2:172.21.15.203:6832/998005540,v1:172.21.15.203:6833/998005540] conn(0x7f109806a280 msgr2=0x7f109806c740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:36:39.429 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:39.427+0000 7f10abfff640 1 --2- 172.21.15.43:0/1604015472 >> [v2:172.21.15.203:6832/998005540,v1:172.21.15.203:6833/998005540] conn(0x7f109806a280 0x7f109806c740 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7f10a0036000 tx=0x7f10a002e000 comp rx=0 tx=0).stop 2023-08-09T13:36:39.429 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:39.427+0000 7f10abfff640 1 -- 172.21.15.43:0/1604015472 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f1094003ef0 msgr2=0x7f10ac0c5cd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:36:39.429 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:39.427+0000 7f10abfff640 1 --2- 172.21.15.43:0/1604015472 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f1094003ef0 0x7f10ac0c5cd0 secure :-1 s=READY pgs=645 cs=0 l=1 rev1=1 crypto rx=0x7f109c011600 tx=0x7f109c011ac0 comp rx=0 tx=0).stop 2023-08-09T13:36:39.429 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:39.427+0000 7f10abfff640 1 -- 172.21.15.43:0/1604015472 shutdown_connections 2023-08-09T13:36:39.429 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:39.427+0000 7f10abfff640 1 --2- 172.21.15.43:0/1604015472 >> [v2:172.21.15.203:6832/998005540,v1:172.21.15.203:6833/998005540] conn(0x7f109806a280 0x7f109806c740 unknown :-1 s=CLOSED pgs=38 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:36:39.429 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:39.427+0000 7f10abfff640 1 --2- 172.21.15.43:0/1604015472 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f10ac1f5a00 0x7f10ac0ca8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:36:39.430 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:39.427+0000 7f10abfff640 1 --2- 172.21.15.43:0/1604015472 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f10ac1f5040 0x7f10ac1f54c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:36:39.430 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:39.427+0000 7f10abfff640 1 --2- 172.21.15.43:0/1604015472 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f1094003ef0 0x7f10ac0c5cd0 unknown :-1 s=CLOSED pgs=645 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:36:39.430 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:39.427+0000 7f10abfff640 1 -- 172.21.15.43:0/1604015472 >> 172.21.15.43:0/1604015472 conn(0x7f10ac06a2a0 msgr2=0x7f10ac0b7c40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-09T13:36:39.430 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:39.427+0000 7f10abfff640 1 -- 172.21.15.43:0/1604015472 shutdown_connections 2023-08-09T13:36:39.430 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:39.427+0000 7f10abfff640 1 -- 172.21.15.43:0/1604015472 wait complete. 2023-08-09T13:36:39.451 DEBUG:tasks.cephfs.test_snap_schedules:res=Schedule activated for path /snap_test_dir1/dir5 2023-08-09T13:36:39.451 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs snap-schedule activate --fs cephfs /snap_test_dir1/dir6 1M 2023-08-09T13:36:39.683 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:39.682+0000 7fe567547640 1 Processor -- start 2023-08-09T13:36:39.684 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:39.682+0000 7fe567547640 1 -- start start 2023-08-09T13:36:39.685 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:39.683+0000 7fe567547640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fe56006fb70 0x7fe5600caab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:36:39.685 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:39.683+0000 7fe567547640 1 -- --> v1:172.21.15.43:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe56015b9b0 con 0x7fe5600ce4c0 2023-08-09T13:36:39.685 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:39.683+0000 7fe567547640 1 -- --> v1:172.21.15.203:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe56015ae10 con 0x7fe5600ca370 2023-08-09T13:36:39.685 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:39.684+0000 7fe567547640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe560154e70 con 0x7fe56006fb70 2023-08-09T13:36:39.687 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:39.684+0000 7fe565ae4640 1 --1- >> v1:172.21.15.43:6789/0 conn(0x7fe5600ce4c0 0x7fe5600c86f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.43:6789/0 says I am v1:172.21.15.43:58148/0 (socket says 172.21.15.43:58148) 2023-08-09T13:36:39.687 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:39.684+0000 7fe565ae4640 1 -- 172.21.15.43:0/3094257969 learned_addr learned my addr 172.21.15.43:0/3094257969 (peer_addr_for_me v1:172.21.15.43:0/0) 2023-08-09T13:36:39.688 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:39.684+0000 7fe5652e3640 1 --2- 172.21.15.43:0/3094257969 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fe56006fb70 0x7fe5600caab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:36:39.688 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:39.684+0000 7fe557fff640 1 -- 172.21.15.43:0/3094257969 <== mon.0 v1:172.21.15.43:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 679007780 0 0) 0x7fe558001490 con 0x7fe5600ce4c0 2023-08-09T13:36:39.688 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:39.684+0000 7fe557fff640 1 -- 172.21.15.43:0/3094257969 --> v1:172.21.15.43:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fe54c003520 con 0x7fe5600ce4c0 2023-08-09T13:36:39.688 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:39.685+0000 7fe557fff640 1 -- 172.21.15.43:0/3094257969 <== mon.0 v1:172.21.15.43:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1024812341 0 0) 0x7fe54c003520 con 0x7fe5600ce4c0 2023-08-09T13:36:39.688 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:39.685+0000 7fe557fff640 1 -- 172.21.15.43:0/3094257969 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fe56006fb70 msgr2=0x7fe5600caab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-09T13:36:39.688 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:39.685+0000 7fe557fff640 1 --2- 172.21.15.43:0/3094257969 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fe56006fb70 0x7fe5600caab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:36:39.689 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:39.685+0000 7fe557fff640 1 -- 172.21.15.43:0/3094257969 >> v1:172.21.15.203:6789/0 conn(0x7fe5600ca370 legacy=0x7fe56015b2a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:36:39.689 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:39.685+0000 7fe557fff640 1 -- 172.21.15.43:0/3094257969 --> v1:172.21.15.43:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe56006ff50 con 0x7fe5600ce4c0 2023-08-09T13:36:39.689 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:39.685+0000 7fe557fff640 1 -- 172.21.15.43:0/3094257969 <== mon.0 v1:172.21.15.43:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1285150740 0 0) 0x7fe558001b80 con 0x7fe5600ce4c0 2023-08-09T13:36:39.689 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:39.685+0000 7fe557fff640 1 -- 172.21.15.43:0/3094257969 >> v1:172.21.15.43:6789/0 conn(0x7fe5600ce4c0 legacy=0x7fe5600c86f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:36:39.689 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:39.685+0000 7fe557fff640 1 --2- 172.21.15.43:0/3094257969 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fe56006fb70 0x7fe54c003a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:36:39.689 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:39.685+0000 7fe557fff640 1 --2- 172.21.15.43:0/3094257969 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fe54c003f90 0x7fe54c0047c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:36:39.689 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:39.685+0000 7fe557fff640 1 --2- 172.21.15.43:0/3094257969 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fe54c004d00 0x7fe54c005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:36:39.690 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:39.685+0000 7fe557fff640 1 -- 172.21.15.43:0/3094257969 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe560154e70 con 0x7fe54c004d00 2023-08-09T13:36:39.690 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:39.685+0000 7fe557fff640 1 -- 172.21.15.43:0/3094257969 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe54c00bae0 con 0x7fe54c003f90 2023-08-09T13:36:39.690 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:39.685+0000 7fe557fff640 1 -- 172.21.15.43:0/3094257969 --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe54c00bca0 con 0x7fe56006fb70 2023-08-09T13:36:39.690 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:39.685+0000 7fe565ae4640 1 --2- 172.21.15.43:0/3094257969 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fe54c004d00 0x7fe54c005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:36:39.690 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:39.686+0000 7fe5652e3640 1 --2- 172.21.15.43:0/3094257969 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fe56006fb70 0x7fe54c003a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:36:39.690 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:39.686+0000 7fe565ae4640 1 -- 172.21.15.43:0/3094257969 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fe56006fb70 msgr2=0x7fe54c003a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-09T13:36:39.690 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:39.686+0000 7fe565ae4640 1 --2- 172.21.15.43:0/3094257969 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fe56006fb70 0x7fe54c003a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:36:39.691 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:39.686+0000 7fe564ae2640 1 --2- 172.21.15.43:0/3094257969 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fe54c003f90 0x7fe54c0047c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:36:39.691 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:39.686+0000 7fe565ae4640 1 -- 172.21.15.43:0/3094257969 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fe54c003f90 msgr2=0x7fe54c0047c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-09T13:36:39.691 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:39.686+0000 7fe565ae4640 1 --2- 172.21.15.43:0/3094257969 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fe54c003f90 0x7fe54c0047c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:36:39.691 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:39.686+0000 7fe565ae4640 1 -- 172.21.15.43:0/3094257969 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe54c003520 con 0x7fe54c004d00 2023-08-09T13:36:39.691 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:39.686+0000 7fe564ae2640 1 --2- 172.21.15.43:0/3094257969 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fe54c003f90 0x7fe54c0047c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-08-09T13:36:39.692 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:39.687+0000 7fe565ae4640 1 --2- 172.21.15.43:0/3094257969 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fe54c004d00 0x7fe54c005180 secure :-1 s=READY pgs=647 cs=0 l=1 rev1=1 crypto rx=0x7fe55800a7e0 tx=0x7fe54c0057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=28325c3663bf37c9 server_cookie=0 in_seq=0 out_seq=0 2023-08-09T13:36:39.693 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:39.687+0000 7fe557fff640 1 -- 172.21.15.43:0/3094257969 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe558034630 con 0x7fe54c004d00 2023-08-09T13:36:39.693 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:39.687+0000 7fe557fff640 1 -- 172.21.15.43:0/3094257969 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe55802f070 con 0x7fe54c004d00 2023-08-09T13:36:39.693 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:39.687+0000 7fe557fff640 1 -- 172.21.15.43:0/3094257969 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe5580349f0 con 0x7fe54c004d00 2023-08-09T13:36:39.693 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:39.688+0000 7fe567547640 1 -- 172.21.15.43:0/3094257969 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fe54c004d00 msgr2=0x7fe54c005180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:36:39.693 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:39.688+0000 7fe567547640 1 --2- 172.21.15.43:0/3094257969 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fe54c004d00 0x7fe54c005180 secure :-1 s=READY pgs=647 cs=0 l=1 rev1=1 crypto rx=0x7fe55800a7e0 tx=0x7fe54c0057f0 comp rx=0 tx=0).stop 2023-08-09T13:36:39.693 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:39.688+0000 7fe565ae4640 1 -- 172.21.15.43:0/3094257969 reap_dead start 2023-08-09T13:36:39.694 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:39.688+0000 7fe567547640 1 -- 172.21.15.43:0/3094257969 shutdown_connections 2023-08-09T13:36:39.694 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:39.688+0000 7fe567547640 1 -- 172.21.15.43:0/3094257969 >> 172.21.15.43:0/3094257969 conn(0x7fe56006b990 msgr2=0x7fe56006bda0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-09T13:36:39.694 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:39.688+0000 7fe567547640 1 -- 172.21.15.43:0/3094257969 shutdown_connections 2023-08-09T13:36:39.694 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:39.688+0000 7fe567547640 1 -- 172.21.15.43:0/3094257969 wait complete. 2023-08-09T13:36:39.694 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:39.690+0000 7fe567547640 1 Processor -- start 2023-08-09T13:36:39.694 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:39.690+0000 7fe567547640 1 -- start start 2023-08-09T13:36:39.694 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:39.691+0000 7fe567547640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fe54c004d00 0x7fe560145640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:36:39.695 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:39.691+0000 7fe567547640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fe560159420 0x7fe5601598a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:36:39.695 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:39.691+0000 7fe567547640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fe5600cbd50 0x7fe5600cc1d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:36:39.695 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:39.691+0000 7fe567547640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe560159de0 con 0x7fe5600cbd50 2023-08-09T13:36:39.695 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:39.691+0000 7fe567547640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe5600afc80 con 0x7fe560159420 2023-08-09T13:36:39.695 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:39.691+0000 7fe567547640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe5600c7e70 con 0x7fe54c004d00 2023-08-09T13:36:39.696 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:39.691+0000 7fe565ae4640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fe5600cbd50 0x7fe5600cc1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:36:39.697 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:39.691+0000 7fe565ae4640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fe5600cbd50 0x7fe5600cc1d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:45620/0 (socket says 172.21.15.43:45620) 2023-08-09T13:36:39.697 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:39.691+0000 7fe565ae4640 1 -- 172.21.15.43:0/2859496636 learned_addr learned my addr 172.21.15.43:0/2859496636 (peer_addr_for_me v2:172.21.15.43:0/0) 2023-08-09T13:36:39.697 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:39.692+0000 7fe5652e3640 1 --2- 172.21.15.43:0/2859496636 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fe54c004d00 0x7fe560145640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:36:39.697 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:39.692+0000 7fe564ae2640 1 --2- 172.21.15.43:0/2859496636 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fe560159420 0x7fe5601598a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:36:39.697 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:39.692+0000 7fe565ae4640 1 -- 172.21.15.43:0/2859496636 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fe54c004d00 msgr2=0x7fe560145640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:36:39.697 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:39.692+0000 7fe565ae4640 1 --2- 172.21.15.43:0/2859496636 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fe54c004d00 0x7fe560145640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:36:39.698 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:39.692+0000 7fe565ae4640 1 -- 172.21.15.43:0/2859496636 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fe560159420 msgr2=0x7fe5601598a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:36:39.698 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:39.692+0000 7fe565ae4640 1 --2- 172.21.15.43:0/2859496636 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fe560159420 0x7fe5601598a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:36:39.698 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:39.692+0000 7fe565ae4640 1 -- 172.21.15.43:0/2859496636 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe5600aff40 con 0x7fe5600cbd50 2023-08-09T13:36:39.698 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:39.692+0000 7fe5652e3640 1 --2- 172.21.15.43:0/2859496636 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fe54c004d00 0x7fe560145640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-08-09T13:36:39.698 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:39.692+0000 7fe564ae2640 1 --2- 172.21.15.43:0/2859496636 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fe560159420 0x7fe5601598a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-08-09T13:36:39.698 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:39.692+0000 7fe565ae4640 1 --2- 172.21.15.43:0/2859496636 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fe5600cbd50 0x7fe5600cc1d0 secure :-1 s=READY pgs=648 cs=0 l=1 rev1=1 crypto rx=0x7fe55800a0b0 tx=0x7fe56015a990 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-09T13:36:39.698 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:39.693+0000 7fe555ffb640 1 -- 172.21.15.43:0/2859496636 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe558042070 con 0x7fe5600cbd50 2023-08-09T13:36:39.699 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:39.693+0000 7fe555ffb640 1 -- 172.21.15.43:0/2859496636 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe558001490 con 0x7fe5600cbd50 2023-08-09T13:36:39.699 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:39.693+0000 7fe555ffb640 1 -- 172.21.15.43:0/2859496636 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe558004d60 con 0x7fe5600cbd50 2023-08-09T13:36:39.699 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:39.693+0000 7fe567547640 1 -- 172.21.15.43:0/2859496636 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe5600b0220 con 0x7fe5600cbd50 2023-08-09T13:36:39.699 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:39.693+0000 7fe567547640 1 -- 172.21.15.43:0/2859496636 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe560154210 con 0x7fe5600cbd50 2023-08-09T13:36:39.699 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:39.694+0000 7fe567547640 1 -- 172.21.15.43:0/2859496636 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe560154210 con 0x7fe5600cbd50 2023-08-09T13:36:39.699 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:39.695+0000 7fe555ffb640 1 -- 172.21.15.43:0/2859496636 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 10) v1 ==== 88517+0+0 (secure 0 0 0) 0x7fe558031030 con 0x7fe5600cbd50 2023-08-09T13:36:39.700 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:39.696+0000 7fe555ffb640 1 --2- 172.21.15.43:0/2859496636 >> [v2:172.21.15.203:6832/998005540,v1:172.21.15.203:6833/998005540] conn(0x7fe54406a250 0x7fe54406c710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:36:39.700 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:39.696+0000 7fe555ffb640 1 -- 172.21.15.43:0/2859496636 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 6640+0+0 (secure 0 0 0) 0x7fe5580aa090 con 0x7fe5600cbd50 2023-08-09T13:36:39.700 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:39.697+0000 7fe5652e3640 1 --2- 172.21.15.43:0/2859496636 >> [v2:172.21.15.203:6832/998005540,v1:172.21.15.203:6833/998005540] conn(0x7fe54406a250 0x7fe54406c710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:36:39.700 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:39.698+0000 7fe5652e3640 1 --2- 172.21.15.43:0/2859496636 >> [v2:172.21.15.203:6832/998005540,v1:172.21.15.203:6833/998005540] conn(0x7fe54406a250 0x7fe54406c710 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7fe55c009a00 tx=0x7fe55c008040 comp rx=0 tx=0).ready entity=mgr.5003 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-09T13:36:39.702 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:39.701+0000 7fe555ffb640 1 -- 172.21.15.43:0/2859496636 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178066 (secure 0 0 0) 0x7fe5580750b0 con 0x7fe5600cbd50 2023-08-09T13:36:39.920 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:39.919+0000 7fe567547640 1 -- 172.21.15.43:0/2859496636 --> [v2:172.21.15.203:6832/998005540,v1:172.21.15.203:6833/998005540] -- mgr_command(tid 0: {"prefix": "fs snap-schedule activate", "fs": "cephfs", "path": "/snap_test_dir1/dir6", "repeat": "1M", "target": ["mon-mgr", ""]}) v1 -- 0x7fe5600cb600 con 0x7fe54406a250 2023-08-09T13:36:40.044 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:40.042+0000 7fe555ffb640 1 -- 172.21.15.43:0/2859496636 <== mgr.5003 v2:172.21.15.203:6832/998005540 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+49 (secure 0 0 0) 0x7fe55c012950 con 0x7fe54406a250 2023-08-09T13:36:40.045 INFO:teuthology.orchestra.run.smithi043.stdout:Schedule activated for path /snap_test_dir1/dir6 2023-08-09T13:36:40.048 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:40.046+0000 7fe567547640 1 -- 172.21.15.43:0/2859496636 >> [v2:172.21.15.203:6832/998005540,v1:172.21.15.203:6833/998005540] conn(0x7fe54406a250 msgr2=0x7fe54406c710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:36:40.048 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:40.046+0000 7fe567547640 1 --2- 172.21.15.43:0/2859496636 >> [v2:172.21.15.203:6832/998005540,v1:172.21.15.203:6833/998005540] conn(0x7fe54406a250 0x7fe54406c710 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7fe55c009a00 tx=0x7fe55c008040 comp rx=0 tx=0).stop 2023-08-09T13:36:40.049 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:40.046+0000 7fe567547640 1 -- 172.21.15.43:0/2859496636 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fe5600cbd50 msgr2=0x7fe5600cc1d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:36:40.049 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:40.046+0000 7fe567547640 1 --2- 172.21.15.43:0/2859496636 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fe5600cbd50 0x7fe5600cc1d0 secure :-1 s=READY pgs=648 cs=0 l=1 rev1=1 crypto rx=0x7fe55800a0b0 tx=0x7fe56015a990 comp rx=0 tx=0).stop 2023-08-09T13:36:40.049 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:40.047+0000 7fe567547640 1 -- 172.21.15.43:0/2859496636 shutdown_connections 2023-08-09T13:36:40.049 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:40.047+0000 7fe567547640 1 --2- 172.21.15.43:0/2859496636 >> [v2:172.21.15.203:6832/998005540,v1:172.21.15.203:6833/998005540] conn(0x7fe54406a250 0x7fe54406c710 unknown :-1 s=CLOSED pgs=39 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:36:40.049 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:40.047+0000 7fe567547640 1 --2- 172.21.15.43:0/2859496636 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fe5600cbd50 0x7fe5600cc1d0 unknown :-1 s=CLOSED pgs=648 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:36:40.049 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:40.047+0000 7fe567547640 1 --2- 172.21.15.43:0/2859496636 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fe560159420 0x7fe5601598a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:36:40.049 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:40.047+0000 7fe567547640 1 --2- 172.21.15.43:0/2859496636 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fe54c004d00 0x7fe560145640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:36:40.050 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:40.047+0000 7fe567547640 1 -- 172.21.15.43:0/2859496636 >> 172.21.15.43:0/2859496636 conn(0x7fe56006b990 msgr2=0x7fe56006bda0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-09T13:36:40.050 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:40.047+0000 7fe567547640 1 -- 172.21.15.43:0/2859496636 shutdown_connections 2023-08-09T13:36:40.050 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:40.047+0000 7fe567547640 1 -- 172.21.15.43:0/2859496636 wait complete. 2023-08-09T13:36:40.070 DEBUG:tasks.cephfs.test_snap_schedules:res=Schedule activated for path /snap_test_dir1/dir6 2023-08-09T13:36:40.070 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs snap-schedule activate --fs cephfs /snap_test_dir1/dir7 1M 2023-08-09T13:36:40.297 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:40.295+0000 7f2ff100b640 1 Processor -- start 2023-08-09T13:36:40.297 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:40.296+0000 7f2ff100b640 1 -- start start 2023-08-09T13:36:40.299 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:40.297+0000 7f2ff100b640 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7f2fec06fb70 0x7f2fec0caab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:36:40.299 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:40.297+0000 7f2ff100b640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f2fec0ca370 0x7f2fec15b2a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:36:40.299 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:40.297+0000 7f2ff100b640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f2fec0ce4c0 0x7f2fec0ce950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:36:40.300 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:40.297+0000 7f2ff100b640 1 -- --> v2:172.21.15.43:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2fec0cee90 con 0x7f2fec0ca370 2023-08-09T13:36:40.300 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:40.297+0000 7f2ff100b640 1 -- --> v2:172.21.15.203:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2fec15ae10 con 0x7f2fec06fb70 2023-08-09T13:36:40.300 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:40.297+0000 7f2ff100b640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2fec154e70 con 0x7f2fec0ce4c0 2023-08-09T13:36:40.300 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:40.298+0000 7f2fea59c640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f2fec0ca370 0x7f2fec15b2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:36:40.300 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:40.298+0000 7f2fea59c640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f2fec0ca370 0x7f2fec15b2a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:45630/0 (socket says 172.21.15.43:45630) 2023-08-09T13:36:40.300 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:40.298+0000 7f2fea59c640 1 -- 172.21.15.43:0/2179547452 learned_addr learned my addr 172.21.15.43:0/2179547452 (peer_addr_for_me v2:172.21.15.43:0/0) 2023-08-09T13:36:40.301 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:40.298+0000 7f2fead9d640 1 --2- 172.21.15.43:0/2179547452 >> v2:172.21.15.203:3300/0 conn(0x7f2fec06fb70 0x7f2fec0caab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:36:40.301 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:40.298+0000 7f2feb59e640 1 --2- 172.21.15.43:0/2179547452 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f2fec0ce4c0 0x7f2fec0ce950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:36:40.302 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:40.298+0000 7f2fea59c640 1 -- 172.21.15.43:0/2179547452 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f2fec0ce4c0 msgr2=0x7f2fec0ce950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-09T13:36:40.302 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:40.298+0000 7f2fea59c640 1 --2- 172.21.15.43:0/2179547452 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f2fec0ce4c0 0x7f2fec0ce950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:36:40.302 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:40.298+0000 7f2fea59c640 1 -- 172.21.15.43:0/2179547452 >> v2:172.21.15.203:3300/0 conn(0x7f2fec06fb70 msgr2=0x7f2fec0caab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-09T13:36:40.302 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:40.298+0000 7f2fea59c640 1 --2- 172.21.15.43:0/2179547452 >> v2:172.21.15.203:3300/0 conn(0x7f2fec06fb70 0x7f2fec0caab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:36:40.302 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:40.298+0000 7f2fea59c640 1 -- 172.21.15.43:0/2179547452 --> v2:172.21.15.43:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2fec06ff50 con 0x7f2fec0ca370 2023-08-09T13:36:40.302 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:40.298+0000 7f2feb59e640 1 --2- 172.21.15.43:0/2179547452 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f2fec0ce4c0 0x7f2fec0ce950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-08-09T13:36:40.302 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:40.298+0000 7f2fead9d640 1 --2- 172.21.15.43:0/2179547452 >> v2:172.21.15.203:3300/0 conn(0x7f2fec06fb70 0x7f2fec0caab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-08-09T13:36:40.303 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:40.299+0000 7f2fea59c640 1 --2- 172.21.15.43:0/2179547452 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f2fec0ca370 0x7f2fec15b2a0 secure :-1 s=READY pgs=649 cs=0 l=1 rev1=1 crypto rx=0x7f2fec0cbb80 tx=0x7f2fd402b910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=385a73b4935b8cc7 server_cookie=0 in_seq=0 out_seq=0 2023-08-09T13:36:40.303 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:40.300+0000 7f2fe9d9b640 1 -- 172.21.15.43:0/2179547452 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2fd4031870 con 0x7f2fec0ca370 2023-08-09T13:36:40.303 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:40.300+0000 7f2fe9d9b640 1 -- 172.21.15.43:0/2179547452 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2fd40319d0 con 0x7f2fec0ca370 2023-08-09T13:36:40.303 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:40.300+0000 7f2fe9d9b640 1 -- 172.21.15.43:0/2179547452 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2fd4031d60 con 0x7f2fec0ca370 2023-08-09T13:36:40.304 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:40.301+0000 7f2ff100b640 1 -- 172.21.15.43:0/2179547452 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f2fec0ca370 msgr2=0x7f2fec15b2a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:36:40.304 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:40.301+0000 7f2ff100b640 1 --2- 172.21.15.43:0/2179547452 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f2fec0ca370 0x7f2fec15b2a0 secure :-1 s=READY pgs=649 cs=0 l=1 rev1=1 crypto rx=0x7f2fec0cbb80 tx=0x7f2fd402b910 comp rx=0 tx=0).stop 2023-08-09T13:36:40.304 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:40.301+0000 7f2ff100b640 1 -- 172.21.15.43:0/2179547452 shutdown_connections 2023-08-09T13:36:40.304 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:40.301+0000 7f2ff100b640 1 --2- 172.21.15.43:0/2179547452 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f2fec0ca370 0x7f2fec15b2a0 unknown :-1 s=CLOSED pgs=649 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:36:40.304 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:40.301+0000 7f2ff100b640 1 --2- 172.21.15.43:0/2179547452 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f2fec0ce4c0 0x7f2fec0ce950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:36:40.304 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:40.301+0000 7f2ff100b640 1 --2- 172.21.15.43:0/2179547452 >> v2:172.21.15.203:3300/0 conn(0x7f2fec06fb70 0x7f2fec0caab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:36:40.305 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:40.301+0000 7f2ff100b640 1 -- 172.21.15.43:0/2179547452 >> 172.21.15.43:0/2179547452 conn(0x7f2fec06b990 msgr2=0x7f2fec06bda0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-09T13:36:40.305 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:40.301+0000 7f2ff100b640 1 -- 172.21.15.43:0/2179547452 shutdown_connections 2023-08-09T13:36:40.305 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:40.301+0000 7f2ff100b640 1 -- 172.21.15.43:0/2179547452 wait complete. 2023-08-09T13:36:40.305 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:40.303+0000 7f2ff100b640 1 Processor -- start 2023-08-09T13:36:40.306 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:40.304+0000 7f2ff100b640 1 -- start start 2023-08-09T13:36:40.306 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:40.305+0000 7f2ff100b640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f2fec06fb70 0x7f2fec150540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:36:40.307 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:40.305+0000 7f2ff100b640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f2fec0ca370 0x7f2fec0c1bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:36:40.307 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:40.305+0000 7f2ff100b640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f2fec0ce4c0 0x7f2fec1f5e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:36:40.307 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:40.305+0000 7f2ff100b640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2fec0b1d10 con 0x7f2fec06fb70 2023-08-09T13:36:40.307 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:40.305+0000 7f2ff100b640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2fec150a80 con 0x7f2fec0ce4c0 2023-08-09T13:36:40.307 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:40.305+0000 7f2ff100b640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2fec0c4630 con 0x7f2fec0ca370 2023-08-09T13:36:40.307 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:40.305+0000 7f2fead9d640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f2fec06fb70 0x7f2fec150540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:36:40.308 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:40.305+0000 7f2fead9d640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f2fec06fb70 0x7f2fec150540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:45642/0 (socket says 172.21.15.43:45642) 2023-08-09T13:36:40.308 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:40.305+0000 7f2fead9d640 1 -- 172.21.15.43:0/2425258967 learned_addr learned my addr 172.21.15.43:0/2425258967 (peer_addr_for_me v2:172.21.15.43:0/0) 2023-08-09T13:36:40.309 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:40.306+0000 7f2fea59c640 1 --2- 172.21.15.43:0/2425258967 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f2fec0ca370 0x7f2fec0c1bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:36:40.309 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:40.306+0000 7f2feb59e640 1 --2- 172.21.15.43:0/2425258967 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f2fec0ce4c0 0x7f2fec1f5e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:36:40.309 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:40.306+0000 7f2fead9d640 1 -- 172.21.15.43:0/2425258967 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f2fec0ca370 msgr2=0x7f2fec0c1bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:36:40.309 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:40.306+0000 7f2fead9d640 1 --2- 172.21.15.43:0/2425258967 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f2fec0ca370 0x7f2fec0c1bd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:36:40.310 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:40.306+0000 7f2fead9d640 1 -- 172.21.15.43:0/2425258967 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f2fec0ce4c0 msgr2=0x7f2fec1f5e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:36:40.310 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:40.306+0000 7f2fead9d640 1 --2- 172.21.15.43:0/2425258967 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f2fec0ce4c0 0x7f2fec1f5e70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:36:40.310 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:40.306+0000 7f2fead9d640 1 -- 172.21.15.43:0/2425258967 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2fd40094d0 con 0x7f2fec06fb70 2023-08-09T13:36:40.310 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:40.306+0000 7f2fea59c640 1 --2- 172.21.15.43:0/2425258967 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f2fec0ca370 0x7f2fec0c1bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-08-09T13:36:40.310 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:40.306+0000 7f2fead9d640 1 --2- 172.21.15.43:0/2425258967 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f2fec06fb70 0x7f2fec150540 secure :-1 s=READY pgs=650 cs=0 l=1 rev1=1 crypto rx=0x7f2fdc00c7c0 tx=0x7f2fdc00cc80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-09T13:36:40.310 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:40.307+0000 7f2fe37fe640 1 -- 172.21.15.43:0/2425258967 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2fdc007a30 con 0x7f2fec06fb70 2023-08-09T13:36:40.310 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:40.307+0000 7f2fe37fe640 1 -- 172.21.15.43:0/2425258967 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2fdc007b90 con 0x7f2fec06fb70 2023-08-09T13:36:40.310 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:40.307+0000 7f2fe37fe640 1 -- 172.21.15.43:0/2425258967 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2fdc00ce70 con 0x7f2fec06fb70 2023-08-09T13:36:40.311 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:40.307+0000 7f2ff100b640 1 -- 172.21.15.43:0/2425258967 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2fec0c4910 con 0x7f2fec06fb70 2023-08-09T13:36:40.311 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:40.307+0000 7f2ff100b640 1 -- 172.21.15.43:0/2425258967 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2fec0c4ed0 con 0x7f2fec06fb70 2023-08-09T13:36:40.311 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:40.308+0000 7f2ff100b640 1 -- 172.21.15.43:0/2425258967 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2fec0739f0 con 0x7f2fec06fb70 2023-08-09T13:36:40.312 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:40.309+0000 7f2fe37fe640 1 -- 172.21.15.43:0/2425258967 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 10) v1 ==== 88517+0+0 (secure 0 0 0) 0x7f2fdc0113f0 con 0x7f2fec06fb70 2023-08-09T13:36:40.312 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:40.310+0000 7f2fe37fe640 1 --2- 172.21.15.43:0/2425258967 >> [v2:172.21.15.203:6832/998005540,v1:172.21.15.203:6833/998005540] conn(0x7f2fcc06a250 0x7f2fcc06c710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:36:40.312 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:40.310+0000 7f2fe37fe640 1 -- 172.21.15.43:0/2425258967 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 6640+0+0 (secure 0 0 0) 0x7f2fdc088e50 con 0x7f2fec06fb70 2023-08-09T13:36:40.312 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:40.311+0000 7f2fea59c640 1 --2- 172.21.15.43:0/2425258967 >> [v2:172.21.15.203:6832/998005540,v1:172.21.15.203:6833/998005540] conn(0x7f2fcc06a250 0x7f2fcc06c710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:36:40.313 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:40.312+0000 7f2fea59c640 1 --2- 172.21.15.43:0/2425258967 >> [v2:172.21.15.203:6832/998005540,v1:172.21.15.203:6833/998005540] conn(0x7f2fcc06a250 0x7f2fcc06c710 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7f2fd4036000 tx=0x7f2fd403c040 comp rx=0 tx=0).ready entity=mgr.5003 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-09T13:36:40.316 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:40.315+0000 7f2fe37fe640 1 -- 172.21.15.43:0/2425258967 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178066 (secure 0 0 0) 0x7f2fdc053ef0 con 0x7f2fec06fb70 2023-08-09T13:36:40.536 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:40.534+0000 7f2ff100b640 1 -- 172.21.15.43:0/2425258967 --> [v2:172.21.15.203:6832/998005540,v1:172.21.15.203:6833/998005540] -- mgr_command(tid 0: {"prefix": "fs snap-schedule activate", "fs": "cephfs", "path": "/snap_test_dir1/dir7", "repeat": "1M", "target": ["mon-mgr", ""]}) v1 -- 0x7f2fec0cb600 con 0x7f2fcc06a250 2023-08-09T13:36:40.660 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:40.659+0000 7f2fe37fe640 1 -- 172.21.15.43:0/2425258967 <== mgr.5003 v2:172.21.15.203:6832/998005540 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+49 (secure 0 0 0) 0x7f2fd403a420 con 0x7f2fcc06a250 2023-08-09T13:36:40.665 INFO:teuthology.orchestra.run.smithi043.stdout:Schedule activated for path /snap_test_dir1/dir7 2023-08-09T13:36:40.666 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:40.663+0000 7f2ff100b640 1 -- 172.21.15.43:0/2425258967 >> [v2:172.21.15.203:6832/998005540,v1:172.21.15.203:6833/998005540] conn(0x7f2fcc06a250 msgr2=0x7f2fcc06c710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:36:40.666 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:40.663+0000 7f2ff100b640 1 --2- 172.21.15.43:0/2425258967 >> [v2:172.21.15.203:6832/998005540,v1:172.21.15.203:6833/998005540] conn(0x7f2fcc06a250 0x7f2fcc06c710 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7f2fd4036000 tx=0x7f2fd403c040 comp rx=0 tx=0).stop 2023-08-09T13:36:40.666 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:40.663+0000 7f2ff100b640 1 -- 172.21.15.43:0/2425258967 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f2fec06fb70 msgr2=0x7f2fec150540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:36:40.666 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:40.663+0000 7f2ff100b640 1 --2- 172.21.15.43:0/2425258967 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f2fec06fb70 0x7f2fec150540 secure :-1 s=READY pgs=650 cs=0 l=1 rev1=1 crypto rx=0x7f2fdc00c7c0 tx=0x7f2fdc00cc80 comp rx=0 tx=0).stop 2023-08-09T13:36:40.666 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:40.663+0000 7f2ff100b640 1 -- 172.21.15.43:0/2425258967 shutdown_connections 2023-08-09T13:36:40.666 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:40.663+0000 7f2ff100b640 1 --2- 172.21.15.43:0/2425258967 >> [v2:172.21.15.203:6832/998005540,v1:172.21.15.203:6833/998005540] conn(0x7f2fcc06a250 0x7f2fcc06c710 unknown :-1 s=CLOSED pgs=40 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:36:40.667 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:40.663+0000 7f2ff100b640 1 --2- 172.21.15.43:0/2425258967 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f2fec0ce4c0 0x7f2fec1f5e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:36:40.667 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:40.663+0000 7f2ff100b640 1 --2- 172.21.15.43:0/2425258967 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f2fec0ca370 0x7f2fec0c1bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:36:40.667 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:40.663+0000 7f2ff100b640 1 --2- 172.21.15.43:0/2425258967 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f2fec06fb70 0x7f2fec150540 unknown :-1 s=CLOSED pgs=650 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:36:40.667 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:40.663+0000 7f2ff100b640 1 -- 172.21.15.43:0/2425258967 >> 172.21.15.43:0/2425258967 conn(0x7f2fec06b990 msgr2=0x7f2fec06bd70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-09T13:36:40.667 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:40.663+0000 7f2ff100b640 1 -- 172.21.15.43:0/2425258967 shutdown_connections 2023-08-09T13:36:40.667 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:40.663+0000 7f2ff100b640 1 -- 172.21.15.43:0/2425258967 wait complete. 2023-08-09T13:36:40.687 DEBUG:tasks.cephfs.test_snap_schedules:res=Schedule activated for path /snap_test_dir1/dir7 2023-08-09T13:36:40.688 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs snap-schedule activate --fs cephfs /snap_test_dir1/dir8 1M 2023-08-09T13:36:40.924 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:40.923+0000 7fc9c2564640 1 Processor -- start 2023-08-09T13:36:40.924 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:40.923+0000 7fc9c2564640 1 -- start start 2023-08-09T13:36:40.926 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:40.924+0000 7fc9c2564640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fc9bc0cbd70 0x7fc9bc0ce4c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:36:40.926 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:40.924+0000 7fc9c2564640 1 -- --> v1:172.21.15.43:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc9bc0cee50 con 0x7fc9bc153590 2023-08-09T13:36:40.926 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:40.925+0000 7fc9c2564640 1 -- --> v1:172.21.15.203:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc9bc0c9510 con 0x7fc9bc0cea00 2023-08-09T13:36:40.926 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:40.925+0000 7fc9c2564640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc9bc0ccb00 con 0x7fc9bc0cbd70 2023-08-09T13:36:40.927 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:40.925+0000 7fc9bbfff640 1 --1- >> v1:172.21.15.43:6789/0 conn(0x7fc9bc153590 0x7fc9bc0cc3f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.43:6789/0 says I am v1:172.21.15.43:58154/0 (socket says 172.21.15.43:58154) 2023-08-09T13:36:40.927 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:40.925+0000 7fc9bbfff640 1 -- 172.21.15.43:0/48281003 learned_addr learned my addr 172.21.15.43:0/48281003 (peer_addr_for_me v1:172.21.15.43:0/0) 2023-08-09T13:36:40.928 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:40.925+0000 7fc9bb7fe640 1 --2- 172.21.15.43:0/48281003 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fc9bc0cbd70 0x7fc9bc0ce4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:36:40.928 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:40.925+0000 7fc9baffd640 1 -- 172.21.15.43:0/48281003 <== mon.0 v1:172.21.15.43:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4206120088 0 0) 0x7fc9ac001490 con 0x7fc9bc153590 2023-08-09T13:36:40.929 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:40.925+0000 7fc9baffd640 1 -- 172.21.15.43:0/48281003 --> v1:172.21.15.43:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fc9a8003520 con 0x7fc9bc153590 2023-08-09T13:36:40.929 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:40.926+0000 7fc9baffd640 1 -- 172.21.15.43:0/48281003 <== mon.0 v1:172.21.15.43:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2517788940 0 0) 0x7fc9a8003520 con 0x7fc9bc153590 2023-08-09T13:36:40.929 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:40.926+0000 7fc9baffd640 1 -- 172.21.15.43:0/48281003 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fc9bc0cbd70 msgr2=0x7fc9bc0ce4c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-09T13:36:40.929 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:40.926+0000 7fc9baffd640 1 --2- 172.21.15.43:0/48281003 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fc9bc0cbd70 0x7fc9bc0ce4c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:36:40.929 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:40.926+0000 7fc9baffd640 1 -- 172.21.15.43:0/48281003 >> v1:172.21.15.203:6789/0 conn(0x7fc9bc0cea00 legacy=0x7fc9bc0ca540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:36:40.929 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:40.926+0000 7fc9baffd640 1 -- 172.21.15.43:0/48281003 --> v1:172.21.15.43:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc9bc0c8900 con 0x7fc9bc153590 2023-08-09T13:36:40.930 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:40.926+0000 7fc9baffd640 1 -- 172.21.15.43:0/48281003 <== mon.0 v1:172.21.15.43:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1285150740 0 0) 0x7fc9ac001b80 con 0x7fc9bc153590 2023-08-09T13:36:40.930 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:40.926+0000 7fc9baffd640 1 -- 172.21.15.43:0/48281003 >> v1:172.21.15.43:6789/0 conn(0x7fc9bc153590 legacy=0x7fc9bc0cc3f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:36:40.930 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:40.926+0000 7fc9baffd640 1 --2- 172.21.15.43:0/48281003 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fc9a8003a10 0x7fc9a8003e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:36:40.930 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:40.926+0000 7fc9baffd640 1 --2- 172.21.15.43:0/48281003 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fc9bc0cbd70 0x7fc9a80047c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:36:40.930 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:40.926+0000 7fc9baffd640 1 --2- 172.21.15.43:0/48281003 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc9a8004d00 0x7fc9a8005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:36:40.930 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:40.926+0000 7fc9baffd640 1 -- 172.21.15.43:0/48281003 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc9bc0ccb00 con 0x7fc9a8004d00 2023-08-09T13:36:40.931 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:40.926+0000 7fc9baffd640 1 -- 172.21.15.43:0/48281003 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc9a800ba90 con 0x7fc9a8003a10 2023-08-09T13:36:40.931 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:40.926+0000 7fc9baffd640 1 -- 172.21.15.43:0/48281003 --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc9a800bc50 con 0x7fc9bc0cbd70 2023-08-09T13:36:40.931 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:40.926+0000 7fc9c0b01640 1 --2- 172.21.15.43:0/48281003 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc9a8004d00 0x7fc9a8005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:36:40.931 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:40.926+0000 7fc9bbfff640 1 --2- 172.21.15.43:0/48281003 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fc9a8003a10 0x7fc9a8003e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:36:40.931 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:40.927+0000 7fc9bb7fe640 1 --2- 172.21.15.43:0/48281003 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fc9bc0cbd70 0x7fc9a80047c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:36:40.931 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:40.927+0000 7fc9c0b01640 1 -- 172.21.15.43:0/48281003 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fc9bc0cbd70 msgr2=0x7fc9a80047c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-09T13:36:40.931 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:40.927+0000 7fc9c0b01640 1 --2- 172.21.15.43:0/48281003 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fc9bc0cbd70 0x7fc9a80047c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:36:40.931 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:40.927+0000 7fc9c0b01640 1 -- 172.21.15.43:0/48281003 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fc9a8003a10 msgr2=0x7fc9a8003e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-09T13:36:40.932 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:40.927+0000 7fc9c0b01640 1 --2- 172.21.15.43:0/48281003 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fc9a8003a10 0x7fc9a8003e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:36:40.932 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:40.927+0000 7fc9c0b01640 1 -- 172.21.15.43:0/48281003 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc9bc0c8900 con 0x7fc9a8004d00 2023-08-09T13:36:40.933 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:40.928+0000 7fc9c0b01640 1 --2- 172.21.15.43:0/48281003 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc9a8004d00 0x7fc9a8005180 secure :-1 s=READY pgs=652 cs=0 l=1 rev1=1 crypto rx=0x7fc9b4001720 tx=0x7fc9a80057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=93286597b5ac886e server_cookie=0 in_seq=0 out_seq=0 2023-08-09T13:36:40.933 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:40.928+0000 7fc9baffd640 1 -- 172.21.15.43:0/48281003 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc9b40052d0 con 0x7fc9a8004d00 2023-08-09T13:36:40.933 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:40.928+0000 7fc9baffd640 1 -- 172.21.15.43:0/48281003 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc9b402e050 con 0x7fc9a8004d00 2023-08-09T13:36:40.933 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:40.928+0000 7fc9baffd640 1 -- 172.21.15.43:0/48281003 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc9b40035f0 con 0x7fc9a8004d00 2023-08-09T13:36:40.933 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:40.929+0000 7fc9c2564640 1 -- 172.21.15.43:0/48281003 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc9a8004d00 msgr2=0x7fc9a8005180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:36:40.934 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:40.929+0000 7fc9c2564640 1 --2- 172.21.15.43:0/48281003 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc9a8004d00 0x7fc9a8005180 secure :-1 s=READY pgs=652 cs=0 l=1 rev1=1 crypto rx=0x7fc9b4001720 tx=0x7fc9a80057f0 comp rx=0 tx=0).stop 2023-08-09T13:36:40.934 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:40.929+0000 7fc9c0b01640 1 -- 172.21.15.43:0/48281003 reap_dead start 2023-08-09T13:36:40.934 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:40.929+0000 7fc9c2564640 1 -- 172.21.15.43:0/48281003 shutdown_connections 2023-08-09T13:36:40.934 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:40.929+0000 7fc9c2564640 1 -- 172.21.15.43:0/48281003 >> 172.21.15.43:0/48281003 conn(0x7fc9bc06b990 msgr2=0x7fc9bc06bda0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-09T13:36:40.934 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:40.929+0000 7fc9c2564640 1 -- 172.21.15.43:0/48281003 shutdown_connections 2023-08-09T13:36:40.934 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:40.929+0000 7fc9c2564640 1 -- 172.21.15.43:0/48281003 wait complete. 2023-08-09T13:36:40.934 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:40.931+0000 7fc9c2564640 1 Processor -- start 2023-08-09T13:36:40.934 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:40.931+0000 7fc9c2564640 1 -- start start 2023-08-09T13:36:40.936 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:40.932+0000 7fc9c2564640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc9a8004d00 0x7fc9bc0c4250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:36:40.936 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:40.932+0000 7fc9c2564640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fc9bc14d750 0x7fc9bc0c2660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:36:40.936 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:40.932+0000 7fc9c2564640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fc9bc0c2ba0 0x7fc9bc15d920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:36:40.936 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:40.932+0000 7fc9c2564640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc9bc158b90 con 0x7fc9a8004d00 2023-08-09T13:36:40.936 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:40.932+0000 7fc9c2564640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc9bc14dbb0 con 0x7fc9bc14d750 2023-08-09T13:36:40.937 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:40.932+0000 7fc9c2564640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc9bc0c4850 con 0x7fc9bc0c2ba0 2023-08-09T13:36:40.937 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:40.932+0000 7fc9bbfff640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc9a8004d00 0x7fc9bc0c4250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:36:40.937 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:40.932+0000 7fc9bbfff640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc9a8004d00 0x7fc9bc0c4250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:45660/0 (socket says 172.21.15.43:45660) 2023-08-09T13:36:40.937 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:40.932+0000 7fc9bbfff640 1 -- 172.21.15.43:0/583387873 learned_addr learned my addr 172.21.15.43:0/583387873 (peer_addr_for_me v2:172.21.15.43:0/0) 2023-08-09T13:36:40.937 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:40.933+0000 7fc9bbfff640 1 -- 172.21.15.43:0/583387873 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fc9bc0c2ba0 msgr2=0x7fc9bc15d920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:36:40.937 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:40.933+0000 7fc9bbfff640 1 --2- 172.21.15.43:0/583387873 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fc9bc0c2ba0 0x7fc9bc15d920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:36:40.937 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:40.933+0000 7fc9bbfff640 1 -- 172.21.15.43:0/583387873 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fc9bc14d750 msgr2=0x7fc9bc0c2660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:36:40.938 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:40.933+0000 7fc9bbfff640 1 --2- 172.21.15.43:0/583387873 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fc9bc14d750 0x7fc9bc0c2660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:36:40.938 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:40.933+0000 7fc9bbfff640 1 -- 172.21.15.43:0/583387873 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc9bc0c4b30 con 0x7fc9a8004d00 2023-08-09T13:36:40.938 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:40.933+0000 7fc9bbfff640 1 --2- 172.21.15.43:0/583387873 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc9a8004d00 0x7fc9bc0c4250 secure :-1 s=READY pgs=653 cs=0 l=1 rev1=1 crypto rx=0x7fc9ac002180 tx=0x7fc9ac00e530 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-09T13:36:40.938 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:40.933+0000 7fc9b8ff9640 1 -- 172.21.15.43:0/583387873 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc9ac00ee30 con 0x7fc9a8004d00 2023-08-09T13:36:40.938 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:40.933+0000 7fc9b8ff9640 1 -- 172.21.15.43:0/583387873 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc9ac013420 con 0x7fc9a8004d00 2023-08-09T13:36:40.938 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:40.933+0000 7fc9c2564640 1 -- 172.21.15.43:0/583387873 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc9bc0c4e10 con 0x7fc9a8004d00 2023-08-09T13:36:40.938 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:40.933+0000 7fc9b8ff9640 1 -- 172.21.15.43:0/583387873 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc9ac0137b0 con 0x7fc9a8004d00 2023-08-09T13:36:40.939 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:40.933+0000 7fc9c2564640 1 -- 172.21.15.43:0/583387873 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc9bc15de60 con 0x7fc9a8004d00 2023-08-09T13:36:40.939 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:40.934+0000 7fc9c2564640 1 -- 172.21.15.43:0/583387873 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc9bc0739f0 con 0x7fc9a8004d00 2023-08-09T13:36:40.939 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:40.935+0000 7fc9b8ff9640 1 -- 172.21.15.43:0/583387873 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 10) v1 ==== 88517+0+0 (secure 0 0 0) 0x7fc9ac013910 con 0x7fc9a8004d00 2023-08-09T13:36:40.940 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:40.936+0000 7fc9b8ff9640 1 --2- 172.21.15.43:0/583387873 >> [v2:172.21.15.203:6832/998005540,v1:172.21.15.203:6833/998005540] conn(0x7fc99c06a250 0x7fc99c06c710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:36:40.940 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:40.936+0000 7fc9b8ff9640 1 -- 172.21.15.43:0/583387873 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 6640+0+0 (secure 0 0 0) 0x7fc9ac089210 con 0x7fc9a8004d00 2023-08-09T13:36:40.940 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:40.937+0000 7fc9bb7fe640 1 --2- 172.21.15.43:0/583387873 >> [v2:172.21.15.203:6832/998005540,v1:172.21.15.203:6833/998005540] conn(0x7fc99c06a250 0x7fc99c06c710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:36:40.940 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:40.938+0000 7fc9bb7fe640 1 --2- 172.21.15.43:0/583387873 >> [v2:172.21.15.203:6832/998005540,v1:172.21.15.203:6833/998005540] conn(0x7fc99c06a250 0x7fc99c06c710 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7fc9bc1505c0 tx=0x7fc9a4008040 comp rx=0 tx=0).ready entity=mgr.5003 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-09T13:36:40.942 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:40.941+0000 7fc9b8ff9640 1 -- 172.21.15.43:0/583387873 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178066 (secure 0 0 0) 0x7fc9ac054440 con 0x7fc9a8004d00 2023-08-09T13:36:41.159 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:41.157+0000 7fc9c2564640 1 -- 172.21.15.43:0/583387873 --> [v2:172.21.15.203:6832/998005540,v1:172.21.15.203:6833/998005540] -- mgr_command(tid 0: {"prefix": "fs snap-schedule activate", "fs": "cephfs", "path": "/snap_test_dir1/dir8", "repeat": "1M", "target": ["mon-mgr", ""]}) v1 -- 0x7fc9bc0c8be0 con 0x7fc99c06a250 2023-08-09T13:36:41.281 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:41.280+0000 7fc9b8ff9640 1 -- 172.21.15.43:0/583387873 <== mgr.5003 v2:172.21.15.203:6832/998005540 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+49 (secure 0 0 0) 0x7fc9a40095c0 con 0x7fc99c06a250 2023-08-09T13:36:41.282 INFO:teuthology.orchestra.run.smithi043.stdout:Schedule activated for path /snap_test_dir1/dir8 2023-08-09T13:36:41.284 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:41.283+0000 7fc9c2564640 1 -- 172.21.15.43:0/583387873 >> [v2:172.21.15.203:6832/998005540,v1:172.21.15.203:6833/998005540] conn(0x7fc99c06a250 msgr2=0x7fc99c06c710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:36:41.285 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:41.283+0000 7fc9c2564640 1 --2- 172.21.15.43:0/583387873 >> [v2:172.21.15.203:6832/998005540,v1:172.21.15.203:6833/998005540] conn(0x7fc99c06a250 0x7fc99c06c710 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7fc9bc1505c0 tx=0x7fc9a4008040 comp rx=0 tx=0).stop 2023-08-09T13:36:41.285 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:41.283+0000 7fc9c2564640 1 -- 172.21.15.43:0/583387873 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc9a8004d00 msgr2=0x7fc9bc0c4250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:36:41.285 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:41.283+0000 7fc9c2564640 1 --2- 172.21.15.43:0/583387873 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc9a8004d00 0x7fc9bc0c4250 secure :-1 s=READY pgs=653 cs=0 l=1 rev1=1 crypto rx=0x7fc9ac002180 tx=0x7fc9ac00e530 comp rx=0 tx=0).stop 2023-08-09T13:36:41.285 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:41.283+0000 7fc9c2564640 1 -- 172.21.15.43:0/583387873 shutdown_connections 2023-08-09T13:36:41.285 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:41.283+0000 7fc9c2564640 1 --2- 172.21.15.43:0/583387873 >> [v2:172.21.15.203:6832/998005540,v1:172.21.15.203:6833/998005540] conn(0x7fc99c06a250 0x7fc99c06c710 unknown :-1 s=CLOSED pgs=41 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:36:41.286 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:41.283+0000 7fc9c2564640 1 --2- 172.21.15.43:0/583387873 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fc9bc0c2ba0 0x7fc9bc15d920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:36:41.286 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:41.283+0000 7fc9c2564640 1 --2- 172.21.15.43:0/583387873 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fc9bc14d750 0x7fc9bc0c2660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:36:41.286 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:41.283+0000 7fc9c2564640 1 --2- 172.21.15.43:0/583387873 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc9a8004d00 0x7fc9bc0c4250 unknown :-1 s=CLOSED pgs=653 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:36:41.286 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:41.283+0000 7fc9c2564640 1 -- 172.21.15.43:0/583387873 >> 172.21.15.43:0/583387873 conn(0x7fc9bc06b990 msgr2=0x7fc9bc0706c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-09T13:36:41.286 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:41.283+0000 7fc9c2564640 1 -- 172.21.15.43:0/583387873 shutdown_connections 2023-08-09T13:36:41.286 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:41.283+0000 7fc9c2564640 1 -- 172.21.15.43:0/583387873 wait complete. 2023-08-09T13:36:41.306 DEBUG:tasks.cephfs.test_snap_schedules:res=Schedule activated for path /snap_test_dir1/dir8 2023-08-09T13:36:41.307 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs snap-schedule activate --fs cephfs /snap_test_dir1/dir9 1M 2023-08-09T13:36:41.543 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:41.542+0000 7f2b87c87640 1 Processor -- start 2023-08-09T13:36:41.544 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:41.542+0000 7f2b87c87640 1 -- start start 2023-08-09T13:36:41.545 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:41.543+0000 7f2b87c87640 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7f2b80069a30 0x7f2b80159760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:36:41.545 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:41.544+0000 7f2b87c87640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f2b8015d5b0 0x7f2b80069e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:36:41.546 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:41.544+0000 7f2b87c87640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f2b8006a350 0x7f2b800ce4c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:36:41.546 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:41.544+0000 7f2b87c87640 1 -- --> v2:172.21.15.43:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2b8015ce80 con 0x7f2b8015d5b0 2023-08-09T13:36:41.546 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:41.544+0000 7f2b87c87640 1 -- --> v2:172.21.15.203:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2b800ca410 con 0x7f2b80069a30 2023-08-09T13:36:41.546 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:41.544+0000 7f2b87c87640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2b8015a050 con 0x7f2b8006a350 2023-08-09T13:36:41.546 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:41.544+0000 7f2b85222640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f2b8015d5b0 0x7f2b80069e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:36:41.546 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:41.544+0000 7f2b85222640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f2b8015d5b0 0x7f2b80069e10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:45666/0 (socket says 172.21.15.43:45666) 2023-08-09T13:36:41.546 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:41.544+0000 7f2b85222640 1 -- 172.21.15.43:0/2563803236 learned_addr learned my addr 172.21.15.43:0/2563803236 (peer_addr_for_me v2:172.21.15.43:0/0) 2023-08-09T13:36:41.547 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:41.544+0000 7f2b85a23640 1 --2- 172.21.15.43:0/2563803236 >> v2:172.21.15.203:3300/0 conn(0x7f2b80069a30 0x7f2b80159760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:36:41.548 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:41.544+0000 7f2b86224640 1 --2- 172.21.15.43:0/2563803236 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f2b8006a350 0x7f2b800ce4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:36:41.548 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:41.544+0000 7f2b85222640 1 -- 172.21.15.43:0/2563803236 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f2b8006a350 msgr2=0x7f2b800ce4c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-09T13:36:41.548 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:41.544+0000 7f2b85222640 1 --2- 172.21.15.43:0/2563803236 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f2b8006a350 0x7f2b800ce4c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:36:41.548 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:41.544+0000 7f2b85222640 1 -- 172.21.15.43:0/2563803236 >> v2:172.21.15.203:3300/0 conn(0x7f2b80069a30 msgr2=0x7f2b80159760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-09T13:36:41.548 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:41.544+0000 7f2b85222640 1 --2- 172.21.15.43:0/2563803236 >> v2:172.21.15.203:3300/0 conn(0x7f2b80069a30 0x7f2b80159760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:36:41.548 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:41.545+0000 7f2b85222640 1 -- 172.21.15.43:0/2563803236 --> v2:172.21.15.43:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2b8015a330 con 0x7f2b8015d5b0 2023-08-09T13:36:41.548 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:41.546+0000 7f2b85222640 1 --2- 172.21.15.43:0/2563803236 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f2b8015d5b0 0x7f2b80069e10 secure :-1 s=READY pgs=654 cs=0 l=1 rev1=1 crypto rx=0x7f2b7000bb10 tx=0x7f2b7002e030 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fbc94911c897299c server_cookie=0 in_seq=0 out_seq=0 2023-08-09T13:36:41.549 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:41.546+0000 7f2b84a21640 1 -- 172.21.15.43:0/2563803236 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2b7002ede0 con 0x7f2b8015d5b0 2023-08-09T13:36:41.549 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:41.546+0000 7f2b84a21640 1 -- 172.21.15.43:0/2563803236 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2b70036440 con 0x7f2b8015d5b0 2023-08-09T13:36:41.549 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:41.546+0000 7f2b84a21640 1 -- 172.21.15.43:0/2563803236 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2b700367d0 con 0x7f2b8015d5b0 2023-08-09T13:36:41.549 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:41.547+0000 7f2b87c87640 1 -- 172.21.15.43:0/2563803236 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f2b8015d5b0 msgr2=0x7f2b80069e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:36:41.550 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:41.547+0000 7f2b87c87640 1 --2- 172.21.15.43:0/2563803236 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f2b8015d5b0 0x7f2b80069e10 secure :-1 s=READY pgs=654 cs=0 l=1 rev1=1 crypto rx=0x7f2b7000bb10 tx=0x7f2b7002e030 comp rx=0 tx=0).stop 2023-08-09T13:36:41.550 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:41.547+0000 7f2b87c87640 1 -- 172.21.15.43:0/2563803236 shutdown_connections 2023-08-09T13:36:41.550 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:41.547+0000 7f2b87c87640 1 --2- 172.21.15.43:0/2563803236 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f2b8015d5b0 0x7f2b80069e10 unknown :-1 s=CLOSED pgs=654 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:36:41.550 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:41.547+0000 7f2b87c87640 1 --2- 172.21.15.43:0/2563803236 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f2b8006a350 0x7f2b800ce4c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:36:41.550 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:41.547+0000 7f2b87c87640 1 --2- 172.21.15.43:0/2563803236 >> v2:172.21.15.203:3300/0 conn(0x7f2b80069a30 0x7f2b80159760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:36:41.550 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:41.547+0000 7f2b87c87640 1 -- 172.21.15.43:0/2563803236 >> 172.21.15.43:0/2563803236 conn(0x7f2b8006b990 msgr2=0x7f2b8006bda0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-09T13:36:41.551 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:41.547+0000 7f2b87c87640 1 -- 172.21.15.43:0/2563803236 shutdown_connections 2023-08-09T13:36:41.551 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:41.547+0000 7f2b87c87640 1 -- 172.21.15.43:0/2563803236 wait complete. 2023-08-09T13:36:41.551 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:41.549+0000 7f2b87c87640 1 Processor -- start 2023-08-09T13:36:41.552 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:41.549+0000 7f2b87c87640 1 -- start start 2023-08-09T13:36:41.552 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:41.550+0000 7f2b87c87640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f2b80069a30 0x7f2b800c7dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:36:41.552 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:41.550+0000 7f2b87c87640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f2b8006a350 0x7f2b801f4e80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:36:41.552 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:41.550+0000 7f2b87c87640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f2b8015d5b0 0x7f2b801f53c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:36:41.552 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:41.550+0000 7f2b87c87640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2b801e3190 con 0x7f2b8015d5b0 2023-08-09T13:36:41.552 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:41.550+0000 7f2b87c87640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2b801f5900 con 0x7f2b8006a350 2023-08-09T13:36:41.552 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:41.550+0000 7f2b87c87640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2b80145740 con 0x7f2b80069a30 2023-08-09T13:36:41.553 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:41.550+0000 7f2b86224640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f2b8015d5b0 0x7f2b801f53c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:36:41.554 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:41.550+0000 7f2b86224640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f2b8015d5b0 0x7f2b801f53c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:45670/0 (socket says 172.21.15.43:45670) 2023-08-09T13:36:41.554 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:41.551+0000 7f2b86224640 1 -- 172.21.15.43:0/4113790223 learned_addr learned my addr 172.21.15.43:0/4113790223 (peer_addr_for_me v2:172.21.15.43:0/0) 2023-08-09T13:36:41.554 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:41.551+0000 7f2b85222640 1 --2- 172.21.15.43:0/4113790223 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f2b8006a350 0x7f2b801f4e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:36:41.554 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:41.551+0000 7f2b85a23640 1 --2- 172.21.15.43:0/4113790223 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f2b80069a30 0x7f2b800c7dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:36:41.554 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:41.551+0000 7f2b86224640 1 -- 172.21.15.43:0/4113790223 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f2b80069a30 msgr2=0x7f2b800c7dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:36:41.554 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:41.551+0000 7f2b86224640 1 --2- 172.21.15.43:0/4113790223 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f2b80069a30 0x7f2b800c7dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:36:41.554 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:41.551+0000 7f2b86224640 1 -- 172.21.15.43:0/4113790223 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f2b8006a350 msgr2=0x7f2b801f4e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:36:41.555 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:41.551+0000 7f2b86224640 1 --2- 172.21.15.43:0/4113790223 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f2b8006a350 0x7f2b801f4e80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:36:41.555 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:41.551+0000 7f2b86224640 1 -- 172.21.15.43:0/4113790223 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2b7000b770 con 0x7f2b8015d5b0 2023-08-09T13:36:41.555 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:41.551+0000 7f2b86224640 1 --2- 172.21.15.43:0/4113790223 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f2b8015d5b0 0x7f2b801f53c0 secure :-1 s=READY pgs=655 cs=0 l=1 rev1=1 crypto rx=0x7f2b7800c5f0 tx=0x7f2b7800cab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-09T13:36:41.555 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:41.552+0000 7f2b767fc640 1 -- 172.21.15.43:0/4113790223 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2b78004220 con 0x7f2b8015d5b0 2023-08-09T13:36:41.555 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:41.552+0000 7f2b767fc640 1 -- 172.21.15.43:0/4113790223 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2b78004380 con 0x7f2b8015d5b0 2023-08-09T13:36:41.555 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:41.552+0000 7f2b767fc640 1 -- 172.21.15.43:0/4113790223 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2b78003850 con 0x7f2b8015d5b0 2023-08-09T13:36:41.556 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:41.552+0000 7f2b87c87640 1 -- 172.21.15.43:0/4113790223 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2b80145990 con 0x7f2b8015d5b0 2023-08-09T13:36:41.556 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:41.552+0000 7f2b87c87640 1 -- 172.21.15.43:0/4113790223 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2b80145f50 con 0x7f2b8015d5b0 2023-08-09T13:36:41.556 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:41.553+0000 7f2b87c87640 1 -- 172.21.15.43:0/4113790223 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2b8006a170 con 0x7f2b8015d5b0 2023-08-09T13:36:41.557 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:41.554+0000 7f2b767fc640 1 -- 172.21.15.43:0/4113790223 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 10) v1 ==== 88517+0+0 (secure 0 0 0) 0x7f2b780039b0 con 0x7f2b8015d5b0 2023-08-09T13:36:41.557 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:41.555+0000 7f2b767fc640 1 --2- 172.21.15.43:0/4113790223 >> [v2:172.21.15.203:6832/998005540,v1:172.21.15.203:6833/998005540] conn(0x7f2b6406a250 0x7f2b6406c710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:36:41.557 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:41.555+0000 7f2b767fc640 1 -- 172.21.15.43:0/4113790223 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 6640+0+0 (secure 0 0 0) 0x7f2b78088930 con 0x7f2b8015d5b0 2023-08-09T13:36:41.557 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:41.556+0000 7f2b85a23640 1 --2- 172.21.15.43:0/4113790223 >> [v2:172.21.15.203:6832/998005540,v1:172.21.15.203:6833/998005540] conn(0x7f2b6406a250 0x7f2b6406c710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:36:41.558 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:41.557+0000 7f2b85a23640 1 --2- 172.21.15.43:0/4113790223 >> [v2:172.21.15.203:6832/998005540,v1:172.21.15.203:6833/998005540] conn(0x7f2b6406a250 0x7f2b6406c710 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7f2b7c0070b0 tx=0x7f2b7c007040 comp rx=0 tx=0).ready entity=mgr.5003 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-09T13:36:41.560 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:41.559+0000 7f2b767fc640 1 -- 172.21.15.43:0/4113790223 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178066 (secure 0 0 0) 0x7f2b780539e0 con 0x7f2b8015d5b0 2023-08-09T13:36:41.780 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:41.779+0000 7f2b87c87640 1 -- 172.21.15.43:0/4113790223 --> [v2:172.21.15.203:6832/998005540,v1:172.21.15.203:6833/998005540] -- mgr_command(tid 0: {"prefix": "fs snap-schedule activate", "fs": "cephfs", "path": "/snap_test_dir1/dir9", "repeat": "1M", "target": ["mon-mgr", ""]}) v1 -- 0x7f2b8015d090 con 0x7f2b6406a250 2023-08-09T13:36:41.903 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:41.901+0000 7f2b767fc640 1 -- 172.21.15.43:0/4113790223 <== mgr.5003 v2:172.21.15.203:6832/998005540 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+49 (secure 0 0 0) 0x7f2b7c013070 con 0x7f2b6406a250 2023-08-09T13:36:41.903 INFO:teuthology.orchestra.run.smithi043.stdout:Schedule activated for path /snap_test_dir1/dir9 2023-08-09T13:36:41.906 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:41.905+0000 7f2b87c87640 1 -- 172.21.15.43:0/4113790223 >> [v2:172.21.15.203:6832/998005540,v1:172.21.15.203:6833/998005540] conn(0x7f2b6406a250 msgr2=0x7f2b6406c710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:36:41.907 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:41.905+0000 7f2b87c87640 1 --2- 172.21.15.43:0/4113790223 >> [v2:172.21.15.203:6832/998005540,v1:172.21.15.203:6833/998005540] conn(0x7f2b6406a250 0x7f2b6406c710 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7f2b7c0070b0 tx=0x7f2b7c007040 comp rx=0 tx=0).stop 2023-08-09T13:36:41.907 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:41.905+0000 7f2b87c87640 1 -- 172.21.15.43:0/4113790223 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f2b8015d5b0 msgr2=0x7f2b801f53c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:36:41.907 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:41.905+0000 7f2b87c87640 1 --2- 172.21.15.43:0/4113790223 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f2b8015d5b0 0x7f2b801f53c0 secure :-1 s=READY pgs=655 cs=0 l=1 rev1=1 crypto rx=0x7f2b7800c5f0 tx=0x7f2b7800cab0 comp rx=0 tx=0).stop 2023-08-09T13:36:41.907 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:41.905+0000 7f2b87c87640 1 -- 172.21.15.43:0/4113790223 shutdown_connections 2023-08-09T13:36:41.907 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:41.905+0000 7f2b87c87640 1 --2- 172.21.15.43:0/4113790223 >> [v2:172.21.15.203:6832/998005540,v1:172.21.15.203:6833/998005540] conn(0x7f2b6406a250 0x7f2b6406c710 unknown :-1 s=CLOSED pgs=42 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:36:41.907 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:41.905+0000 7f2b87c87640 1 --2- 172.21.15.43:0/4113790223 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f2b8015d5b0 0x7f2b801f53c0 unknown :-1 s=CLOSED pgs=655 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:36:41.908 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:41.905+0000 7f2b87c87640 1 --2- 172.21.15.43:0/4113790223 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f2b8006a350 0x7f2b801f4e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:36:41.908 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:41.905+0000 7f2b87c87640 1 --2- 172.21.15.43:0/4113790223 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f2b80069a30 0x7f2b800c7dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:36:41.908 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:41.905+0000 7f2b87c87640 1 -- 172.21.15.43:0/4113790223 >> 172.21.15.43:0/4113790223 conn(0x7f2b8006b990 msgr2=0x7f2b8015bfa0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-09T13:36:41.908 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:41.905+0000 7f2b87c87640 1 -- 172.21.15.43:0/4113790223 shutdown_connections 2023-08-09T13:36:41.908 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:36:41.905+0000 7f2b87c87640 1 -- 172.21.15.43:0/4113790223 wait complete. 2023-08-09T13:36:41.929 DEBUG:tasks.cephfs.test_snap_schedules:res=Schedule activated for path /snap_test_dir1/dir9 2023-08-09T13:36:59.476 DEBUG:teuthology.orchestra.run.smithi043:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-08-09T13:36:59.485 DEBUG:teuthology.orchestra.run.smithi203:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-08-09T13:37:00.066 INFO:tasks.ceph.mgr.x.smithi203.stderr:2023-08-09T13:37:00.063+0000 7f5865a73640 -1 client.0 error registering admin socket command: (17) File exists 2023-08-09T13:37:00.066 INFO:tasks.ceph.mgr.x.smithi203.stderr:2023-08-09T13:37:00.063+0000 7f5865a73640 -1 client.0 error registering admin socket command: (17) File exists 2023-08-09T13:37:00.066 INFO:tasks.ceph.mgr.x.smithi203.stderr:2023-08-09T13:37:00.063+0000 7f5865a73640 -1 client.0 error registering admin socket command: (17) File exists 2023-08-09T13:37:00.066 INFO:tasks.ceph.mgr.x.smithi203.stderr:2023-08-09T13:37:00.063+0000 7f5865a73640 -1 client.0 error registering admin socket command: (17) File exists 2023-08-09T13:37:00.066 INFO:tasks.ceph.mgr.x.smithi203.stderr:2023-08-09T13:37:00.063+0000 7f5865a73640 -1 client.0 error registering admin socket command: (17) File exists 2023-08-09T13:37:29.648 DEBUG:teuthology.orchestra.run.smithi043:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-08-09T13:37:29.658 DEBUG:teuthology.orchestra.run.smithi203:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-08-09T13:37:59.799 DEBUG:teuthology.orchestra.run.smithi043:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-08-09T13:37:59.809 DEBUG:teuthology.orchestra.run.smithi203:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-08-09T13:38:29.955 DEBUG:teuthology.orchestra.run.smithi043:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-08-09T13:38:29.965 DEBUG:teuthology.orchestra.run.smithi203:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-08-09T13:39:00.122 DEBUG:teuthology.orchestra.run.smithi043:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-08-09T13:39:00.133 DEBUG:teuthology.orchestra.run.smithi203:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-08-09T13:39:30.265 DEBUG:teuthology.orchestra.run.smithi043:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-08-09T13:39:30.275 DEBUG:teuthology.orchestra.run.smithi203:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-08-09T13:40:00.407 DEBUG:teuthology.orchestra.run.smithi043:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-08-09T13:40:00.417 DEBUG:teuthology.orchestra.run.smithi203:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-08-09T13:40:00.530 INFO:tasks.ceph.mgr.x.smithi203.stderr:2023-08-09T13:40:00.528+0000 7f5867276640 -1 client.0 error registering admin socket command: (17) File exists 2023-08-09T13:40:00.530 INFO:tasks.ceph.mgr.x.smithi203.stderr:2023-08-09T13:40:00.528+0000 7f5867276640 -1 client.0 error registering admin socket command: (17) File exists 2023-08-09T13:40:00.530 INFO:tasks.ceph.mgr.x.smithi203.stderr:2023-08-09T13:40:00.528+0000 7f5867276640 -1 client.0 error registering admin socket command: (17) File exists 2023-08-09T13:40:00.531 INFO:tasks.ceph.mgr.x.smithi203.stderr:2023-08-09T13:40:00.528+0000 7f5867276640 -1 client.0 error registering admin socket command: (17) File exists 2023-08-09T13:40:00.531 INFO:tasks.ceph.mgr.x.smithi203.stderr:2023-08-09T13:40:00.528+0000 7f5867276640 -1 client.0 error registering admin socket command: (17) File exists 2023-08-09T13:40:30.558 DEBUG:teuthology.orchestra.run.smithi043:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-08-09T13:40:30.568 DEBUG:teuthology.orchestra.run.smithi203:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-08-09T13:41:00.711 DEBUG:teuthology.orchestra.run.smithi043:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-08-09T13:41:00.724 DEBUG:teuthology.orchestra.run.smithi203:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-08-09T13:41:30.866 DEBUG:teuthology.orchestra.run.smithi043:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-08-09T13:41:30.876 DEBUG:teuthology.orchestra.run.smithi203:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-08-09T13:42:01.035 DEBUG:teuthology.orchestra.run.smithi043:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-08-09T13:42:01.044 DEBUG:teuthology.orchestra.run.smithi203:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-08-09T13:42:31.193 DEBUG:teuthology.orchestra.run.smithi043:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-08-09T13:42:31.204 DEBUG:teuthology.orchestra.run.smithi203:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-08-09T13:43:01.364 DEBUG:teuthology.orchestra.run.smithi043:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-08-09T13:43:01.375 DEBUG:teuthology.orchestra.run.smithi203:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-08-09T13:43:31.514 DEBUG:teuthology.orchestra.run.smithi043:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-08-09T13:43:31.524 DEBUG:teuthology.orchestra.run.smithi203:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-08-09T13:44:01.680 DEBUG:teuthology.orchestra.run.smithi043:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-08-09T13:44:01.690 DEBUG:teuthology.orchestra.run.smithi203:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-08-09T13:44:31.836 DEBUG:teuthology.orchestra.run.smithi043:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-08-09T13:44:31.846 DEBUG:teuthology.orchestra.run.smithi203:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-08-09T13:45:02.026 DEBUG:teuthology.orchestra.run.smithi043:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-08-09T13:45:02.035 DEBUG:teuthology.orchestra.run.smithi203:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-08-09T13:45:32.167 DEBUG:teuthology.orchestra.run.smithi043:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-08-09T13:45:32.177 DEBUG:teuthology.orchestra.run.smithi203:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-08-09T13:46:02.323 DEBUG:teuthology.orchestra.run.smithi043:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-08-09T13:46:02.333 DEBUG:teuthology.orchestra.run.smithi203:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-08-09T13:46:32.480 DEBUG:teuthology.orchestra.run.smithi043:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-08-09T13:46:32.489 DEBUG:teuthology.orchestra.run.smithi203:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-08-09T13:47:02.650 DEBUG:teuthology.orchestra.run.smithi043:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-08-09T13:47:02.660 DEBUG:teuthology.orchestra.run.smithi203:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-08-09T13:47:21.931 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs snap-schedule deactivate --fs cephfs /snap_test_dir1/dir0 1M 2023-08-09T13:47:22.176 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:22.174+0000 7f8414571640 1 Processor -- start 2023-08-09T13:47:22.177 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:22.174+0000 7f8414571640 1 -- start start 2023-08-09T13:47:22.178 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:22.176+0000 7f8414571640 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7f840c06f4a0 0x7f840c0ce4c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:47:22.178 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:22.176+0000 7f8414571640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f840c0cea00 0x7f840c0cc610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:47:22.178 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:22.176+0000 7f8414571640 1 -- --> v1:172.21.15.43:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f840c0cc180 con 0x7f840c0c8f50 2023-08-09T13:47:22.178 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:22.176+0000 7f8414571640 1 -- --> v2:172.21.15.203:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f840c0cee90 con 0x7f840c06f4a0 2023-08-09T13:47:22.179 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:22.176+0000 7f8414571640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f840c06f900 con 0x7f840c0cea00 2023-08-09T13:47:22.179 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:22.176+0000 7f841230d640 1 --1- >> v1:172.21.15.43:6789/0 conn(0x7f840c0c8f50 0x7f840c164560 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.43:6789/0 says I am v1:172.21.15.43:59560/0 (socket says 172.21.15.43:59560) 2023-08-09T13:47:22.179 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:22.176+0000 7f841230d640 1 -- 172.21.15.43:0/184999566 learned_addr learned my addr 172.21.15.43:0/184999566 (peer_addr_for_me v1:172.21.15.43:0/0) 2023-08-09T13:47:22.180 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:22.177+0000 7f8412b0e640 1 --2- 172.21.15.43:0/184999566 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f840c0cea00 0x7f840c0cc610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:47:22.180 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:22.177+0000 7f841130b640 1 -- 172.21.15.43:0/184999566 <== mon.0 v1:172.21.15.43:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1452506625 0 0) 0x7f840c0cc180 con 0x7f840c0c8f50 2023-08-09T13:47:22.180 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:22.177+0000 7f841130b640 1 -- 172.21.15.43:0/184999566 --> v1:172.21.15.43:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f83f8003520 con 0x7f840c0c8f50 2023-08-09T13:47:22.180 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:22.177+0000 7f8411b0c640 1 --2- 172.21.15.43:0/184999566 >> v2:172.21.15.203:3300/0 conn(0x7f840c06f4a0 0x7f840c0ce4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:47:22.180 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:22.177+0000 7f841130b640 1 -- 172.21.15.43:0/184999566 <== mon.0 v1:172.21.15.43:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 800530333 0 0) 0x7f83f8003520 con 0x7f840c0c8f50 2023-08-09T13:47:22.180 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:22.177+0000 7f841130b640 1 -- 172.21.15.43:0/184999566 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f840c0cea00 msgr2=0x7f840c0cc610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-09T13:47:22.180 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:22.177+0000 7f841130b640 1 --2- 172.21.15.43:0/184999566 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f840c0cea00 0x7f840c0cc610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:47:22.181 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:22.177+0000 7f841130b640 1 -- 172.21.15.43:0/184999566 >> v2:172.21.15.203:3300/0 conn(0x7f840c06f4a0 msgr2=0x7f840c0ce4c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-09T13:47:22.181 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:22.177+0000 7f841130b640 1 --2- 172.21.15.43:0/184999566 >> v2:172.21.15.203:3300/0 conn(0x7f840c06f4a0 0x7f840c0ce4c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:47:22.181 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:22.177+0000 7f841130b640 1 -- 172.21.15.43:0/184999566 --> v1:172.21.15.43:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f840c164fe0 con 0x7f840c0c8f50 2023-08-09T13:47:22.182 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:22.177+0000 7f841130b640 1 -- 172.21.15.43:0/184999566 <== mon.0 v1:172.21.15.43:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1285150740 0 0) 0x7f8408001670 con 0x7f840c0c8f50 2023-08-09T13:47:22.182 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:22.177+0000 7f841130b640 1 -- 172.21.15.43:0/184999566 >> v1:172.21.15.43:6789/0 conn(0x7f840c0c8f50 legacy=0x7f840c164560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:47:22.182 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:22.177+0000 7f841130b640 1 --2- 172.21.15.43:0/184999566 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f83f80039e0 0x7f83f8003e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:47:22.182 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:22.177+0000 7f841130b640 1 --2- 172.21.15.43:0/184999566 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f840c0cea00 0x7f83f80047c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:47:22.182 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:22.177+0000 7f841130b640 1 --2- 172.21.15.43:0/184999566 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f83f8004d00 0x7f83f8005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:47:22.182 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:22.177+0000 7f841130b640 1 -- 172.21.15.43:0/184999566 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f840c0cee90 con 0x7f83f8004d00 2023-08-09T13:47:22.182 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:22.177+0000 7f841130b640 1 -- 172.21.15.43:0/184999566 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f840c06f900 con 0x7f83f80039e0 2023-08-09T13:47:22.183 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:22.177+0000 7f841130b640 1 -- 172.21.15.43:0/184999566 --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f83f800ba80 con 0x7f840c0cea00 2023-08-09T13:47:22.183 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:22.178+0000 7f8412b0e640 1 --2- 172.21.15.43:0/184999566 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f83f8004d00 0x7f83f8005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:47:22.183 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:22.178+0000 7f841230d640 1 --2- 172.21.15.43:0/184999566 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f83f80039e0 0x7f83f8003e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:47:22.183 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:22.178+0000 7f8412b0e640 1 -- 172.21.15.43:0/184999566 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f840c0cea00 msgr2=0x7f83f80047c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-09T13:47:22.183 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:22.178+0000 7f8412b0e640 1 --2- 172.21.15.43:0/184999566 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f840c0cea00 0x7f83f80047c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:47:22.183 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:22.178+0000 7f8412b0e640 1 -- 172.21.15.43:0/184999566 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f83f80039e0 msgr2=0x7f83f8003e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-09T13:47:22.183 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:22.178+0000 7f8412b0e640 1 --2- 172.21.15.43:0/184999566 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f83f80039e0 0x7f83f8003e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:47:22.183 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:22.178+0000 7f8412b0e640 1 -- 172.21.15.43:0/184999566 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f83f8003520 con 0x7f83f8004d00 2023-08-09T13:47:22.184 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:22.179+0000 7f8412b0e640 1 --2- 172.21.15.43:0/184999566 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f83f8004d00 0x7f83f8005180 secure :-1 s=READY pgs=657 cs=0 l=1 rev1=1 crypto rx=0x7f8404009fc0 tx=0x7f83f80057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d6a9b8ede6b4ab7a server_cookie=0 in_seq=0 out_seq=0 2023-08-09T13:47:22.185 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:22.180+0000 7f841130b640 1 -- 172.21.15.43:0/184999566 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f840402cc40 con 0x7f83f8004d00 2023-08-09T13:47:22.185 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:22.180+0000 7f841130b640 1 -- 172.21.15.43:0/184999566 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f840400f040 con 0x7f83f8004d00 2023-08-09T13:47:22.185 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:22.180+0000 7f841130b640 1 -- 172.21.15.43:0/184999566 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8404034440 con 0x7f83f8004d00 2023-08-09T13:47:22.185 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:22.180+0000 7f8414571640 1 -- 172.21.15.43:0/184999566 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f83f8004d00 msgr2=0x7f83f8005180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:47:22.185 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:22.180+0000 7f8414571640 1 --2- 172.21.15.43:0/184999566 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f83f8004d00 0x7f83f8005180 secure :-1 s=READY pgs=657 cs=0 l=1 rev1=1 crypto rx=0x7f8404009fc0 tx=0x7f83f80057f0 comp rx=0 tx=0).stop 2023-08-09T13:47:22.185 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:22.180+0000 7f8412b0e640 1 -- 172.21.15.43:0/184999566 reap_dead start 2023-08-09T13:47:22.185 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:22.180+0000 7f8414571640 1 -- 172.21.15.43:0/184999566 shutdown_connections 2023-08-09T13:47:22.186 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:22.180+0000 7f8414571640 1 -- 172.21.15.43:0/184999566 >> 172.21.15.43:0/184999566 conn(0x7f840c06b990 msgr2=0x7f840c06bda0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-09T13:47:22.186 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:22.180+0000 7f8414571640 1 -- 172.21.15.43:0/184999566 shutdown_connections 2023-08-09T13:47:22.186 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:22.180+0000 7f8414571640 1 -- 172.21.15.43:0/184999566 wait complete. 2023-08-09T13:47:22.186 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:22.183+0000 7f8414571640 1 Processor -- start 2023-08-09T13:47:22.186 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:22.183+0000 7f8414571640 1 -- start start 2023-08-09T13:47:22.188 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:22.184+0000 7f8414571640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f83f8004d00 0x7f840c0c2530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:47:22.188 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:22.184+0000 7f8414571640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f840c0c4a60 0x7f840c0b3550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:47:22.188 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:22.184+0000 7f8414571640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f840c0aedd0 0x7f840c0af250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:47:22.188 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:22.184+0000 7f8414571640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f840c0c2ba0 con 0x7f840c0aedd0 2023-08-09T13:47:22.188 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:22.184+0000 7f8414571640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f840c1f9440 con 0x7f83f8004d00 2023-08-09T13:47:22.188 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:22.184+0000 7f8414571640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f840c0c4ee0 con 0x7f840c0c4a60 2023-08-09T13:47:22.188 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:22.184+0000 7f8412b0e640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f840c0aedd0 0x7f840c0af250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:47:22.189 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:22.184+0000 7f8412b0e640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f840c0aedd0 0x7f840c0af250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:57490/0 (socket says 172.21.15.43:57490) 2023-08-09T13:47:22.189 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:22.184+0000 7f8412b0e640 1 -- 172.21.15.43:0/518679471 learned_addr learned my addr 172.21.15.43:0/518679471 (peer_addr_for_me v2:172.21.15.43:0/0) 2023-08-09T13:47:22.189 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:22.184+0000 7f841230d640 1 --2- 172.21.15.43:0/518679471 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f83f8004d00 0x7f840c0c2530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:47:22.189 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:22.184+0000 7f8411b0c640 1 --2- 172.21.15.43:0/518679471 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f840c0c4a60 0x7f840c0b3550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:47:22.189 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:22.184+0000 7f8412b0e640 1 -- 172.21.15.43:0/518679471 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f840c0c4a60 msgr2=0x7f840c0b3550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:47:22.189 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:22.184+0000 7f8412b0e640 1 --2- 172.21.15.43:0/518679471 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f840c0c4a60 0x7f840c0b3550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:47:22.189 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:22.184+0000 7f8412b0e640 1 -- 172.21.15.43:0/518679471 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f83f8004d00 msgr2=0x7f840c0c2530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:47:22.189 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:22.184+0000 7f8412b0e640 1 --2- 172.21.15.43:0/518679471 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f83f8004d00 0x7f840c0c2530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:47:22.190 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:22.184+0000 7f8412b0e640 1 -- 172.21.15.43:0/518679471 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f840c0af790 con 0x7f840c0aedd0 2023-08-09T13:47:22.190 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:22.185+0000 7f8412b0e640 1 --2- 172.21.15.43:0/518679471 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f840c0aedd0 0x7f840c0af250 secure :-1 s=READY pgs=658 cs=0 l=1 rev1=1 crypto rx=0x7f8404037040 tx=0x7f8404009750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-09T13:47:22.190 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:22.185+0000 7f8402ffd640 1 -- 172.21.15.43:0/518679471 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8404034ad0 con 0x7f840c0aedd0 2023-08-09T13:47:22.190 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:22.185+0000 7f8402ffd640 1 -- 172.21.15.43:0/518679471 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8404034c30 con 0x7f840c0aedd0 2023-08-09T13:47:22.190 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:22.185+0000 7f8402ffd640 1 -- 172.21.15.43:0/518679471 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8404003990 con 0x7f840c0aedd0 2023-08-09T13:47:22.190 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:22.185+0000 7f8414571640 1 -- 172.21.15.43:0/518679471 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f840c1f9650 con 0x7f840c0aedd0 2023-08-09T13:47:22.190 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:22.185+0000 7f8414571640 1 -- 172.21.15.43:0/518679471 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f840c1f9c10 con 0x7f840c0aedd0 2023-08-09T13:47:22.191 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:22.186+0000 7f8414571640 1 -- 172.21.15.43:0/518679471 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f840c0739f0 con 0x7f840c0aedd0 2023-08-09T13:47:22.191 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:22.187+0000 7f8402ffd640 1 -- 172.21.15.43:0/518679471 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 13) v1 ==== 88529+0+0 (secure 0 0 0) 0x7f8404047460 con 0x7f840c0aedd0 2023-08-09T13:47:22.192 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:22.188+0000 7f8402ffd640 1 --2- 172.21.15.43:0/518679471 >> [v2:172.21.15.203:6832/998005540,v1:172.21.15.203:6833/998005540] conn(0x7f83f006a250 0x7f83f006c710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:47:22.192 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:22.188+0000 7f8402ffd640 1 -- 172.21.15.43:0/518679471 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(42..42 src has 1..42) v4 ==== 6698+0+0 (secure 0 0 0) 0x7f84040aaa60 con 0x7f840c0aedd0 2023-08-09T13:47:22.192 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:22.189+0000 7f841230d640 1 --2- 172.21.15.43:0/518679471 >> [v2:172.21.15.203:6832/998005540,v1:172.21.15.203:6833/998005540] conn(0x7f83f006a250 0x7f83f006c710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:47:22.192 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:22.190+0000 7f841230d640 1 --2- 172.21.15.43:0/518679471 >> [v2:172.21.15.203:6832/998005540,v1:172.21.15.203:6833/998005540] conn(0x7f83f006a250 0x7f83f006c710 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7f8408001eb0 tx=0x7f8408009040 comp rx=0 tx=0).ready entity=mgr.5003 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-09T13:47:22.196 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:22.193+0000 7f8402ffd640 1 -- 172.21.15.43:0/518679471 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178066 (secure 0 0 0) 0x7f8404075c40 con 0x7f840c0aedd0 2023-08-09T13:47:22.412 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:22.410+0000 7f8414571640 1 -- 172.21.15.43:0/518679471 --> [v2:172.21.15.203:6832/998005540,v1:172.21.15.203:6833/998005540] -- mgr_command(tid 0: {"prefix": "fs snap-schedule deactivate", "fs": "cephfs", "path": "/snap_test_dir1/dir0", "repeat": "1M", "target": ["mon-mgr", ""]}) v1 -- 0x7f840c1652c0 con 0x7f83f006a250 2023-08-09T13:47:22.585 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:22.582+0000 7f8402ffd640 1 -- 172.21.15.43:0/518679471 <== mgr.5003 v2:172.21.15.203:6832/998005540 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+51 (secure 0 0 0) 0x7f8408000ed0 con 0x7f83f006a250 2023-08-09T13:47:22.586 INFO:teuthology.orchestra.run.smithi043.stdout:Schedule deactivated for path /snap_test_dir1/dir0 2023-08-09T13:47:22.589 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:22.586+0000 7f8414571640 1 -- 172.21.15.43:0/518679471 >> [v2:172.21.15.203:6832/998005540,v1:172.21.15.203:6833/998005540] conn(0x7f83f006a250 msgr2=0x7f83f006c710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:47:22.589 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:22.586+0000 7f8414571640 1 --2- 172.21.15.43:0/518679471 >> [v2:172.21.15.203:6832/998005540,v1:172.21.15.203:6833/998005540] conn(0x7f83f006a250 0x7f83f006c710 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7f8408001eb0 tx=0x7f8408009040 comp rx=0 tx=0).stop 2023-08-09T13:47:22.589 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:22.586+0000 7f8414571640 1 -- 172.21.15.43:0/518679471 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f840c0aedd0 msgr2=0x7f840c0af250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:47:22.589 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:22.586+0000 7f8414571640 1 --2- 172.21.15.43:0/518679471 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f840c0aedd0 0x7f840c0af250 secure :-1 s=READY pgs=658 cs=0 l=1 rev1=1 crypto rx=0x7f8404037040 tx=0x7f8404009750 comp rx=0 tx=0).stop 2023-08-09T13:47:22.589 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:22.586+0000 7f8414571640 1 -- 172.21.15.43:0/518679471 shutdown_connections 2023-08-09T13:47:22.589 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:22.586+0000 7f8414571640 1 --2- 172.21.15.43:0/518679471 >> [v2:172.21.15.203:6832/998005540,v1:172.21.15.203:6833/998005540] conn(0x7f83f006a250 0x7f83f006c710 unknown :-1 s=CLOSED pgs=43 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:47:22.590 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:22.586+0000 7f8414571640 1 --2- 172.21.15.43:0/518679471 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f840c0aedd0 0x7f840c0af250 unknown :-1 s=CLOSED pgs=658 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:47:22.590 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:22.586+0000 7f8414571640 1 --2- 172.21.15.43:0/518679471 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f840c0c4a60 0x7f840c0b3550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:47:22.590 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:22.586+0000 7f8414571640 1 --2- 172.21.15.43:0/518679471 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f83f8004d00 0x7f840c0c2530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:47:22.590 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:22.586+0000 7f8414571640 1 -- 172.21.15.43:0/518679471 >> 172.21.15.43:0/518679471 conn(0x7f840c06b990 msgr2=0x7f840c162f20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-09T13:47:22.590 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:22.587+0000 7f8414571640 1 -- 172.21.15.43:0/518679471 shutdown_connections 2023-08-09T13:47:22.591 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:22.587+0000 7f8414571640 1 -- 172.21.15.43:0/518679471 wait complete. 2023-08-09T13:47:22.608 DEBUG:tasks.cephfs.test_snap_schedules:res=Schedule deactivated for path /snap_test_dir1/dir0 2023-08-09T13:47:22.609 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs snap-schedule deactivate --fs cephfs /snap_test_dir1/dir1 1M 2023-08-09T13:47:22.841 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:22.839+0000 7efe13fff640 1 Processor -- start 2023-08-09T13:47:22.842 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:22.839+0000 7efe13fff640 1 -- start start 2023-08-09T13:47:22.843 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:22.841+0000 7efe13fff640 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7efe140cb9c0 0x7efe140ac270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:47:22.843 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:22.841+0000 7efe13fff640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7efe1415ca40 0x7efe1406b400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:47:22.843 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:22.841+0000 7efe13fff640 1 -- --> v1:172.21.15.43:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7efe140ac7b0 con 0x7efe140ad1f0 2023-08-09T13:47:22.844 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:22.841+0000 7efe13fff640 1 -- --> v2:172.21.15.203:3300/0 -- mon_getmap magic: 0 v1 -- 0x7efe1415d9a0 con 0x7efe140cb9c0 2023-08-09T13:47:22.844 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:22.841+0000 7efe13fff640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7efe140abe50 con 0x7efe1415ca40 2023-08-09T13:47:22.846 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:22.841+0000 7efe127fc640 1 --1- >> v1:172.21.15.43:6789/0 conn(0x7efe140ad1f0 0x7efe140cbda0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.43:6789/0 says I am v1:172.21.15.43:59564/0 (socket says 172.21.15.43:59564) 2023-08-09T13:47:22.846 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:22.841+0000 7efe127fc640 1 -- 172.21.15.43:0/1480357763 learned_addr learned my addr 172.21.15.43:0/1480357763 (peer_addr_for_me v1:172.21.15.43:0/0) 2023-08-09T13:47:22.846 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:22.842+0000 7efe12ffd640 1 --2- 172.21.15.43:0/1480357763 >> v2:172.21.15.203:3300/0 conn(0x7efe140cb9c0 0x7efe140ac270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:47:22.846 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:22.842+0000 7efe11ffb640 1 -- 172.21.15.43:0/1480357763 <== mon.0 v1:172.21.15.43:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2799176918 0 0) 0x7efe140ac7b0 con 0x7efe140ad1f0 2023-08-09T13:47:22.847 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:22.842+0000 7efe137fe640 1 --2- 172.21.15.43:0/1480357763 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7efe1415ca40 0x7efe1406b400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:47:22.847 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:22.842+0000 7efe11ffb640 1 -- 172.21.15.43:0/1480357763 --> v1:172.21.15.43:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7efdfc003520 con 0x7efe140ad1f0 2023-08-09T13:47:22.847 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:22.842+0000 7efe11ffb640 1 -- 172.21.15.43:0/1480357763 <== mon.0 v1:172.21.15.43:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2180148056 0 0) 0x7efdfc003520 con 0x7efe140ad1f0 2023-08-09T13:47:22.847 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:22.842+0000 7efe11ffb640 1 -- 172.21.15.43:0/1480357763 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7efe1415ca40 msgr2=0x7efe1406b400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-09T13:47:22.847 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:22.842+0000 7efe11ffb640 1 --2- 172.21.15.43:0/1480357763 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7efe1415ca40 0x7efe1406b400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:47:22.847 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:22.842+0000 7efe11ffb640 1 -- 172.21.15.43:0/1480357763 >> v2:172.21.15.203:3300/0 conn(0x7efe140cb9c0 msgr2=0x7efe140ac270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-09T13:47:22.847 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:22.842+0000 7efe11ffb640 1 --2- 172.21.15.43:0/1480357763 >> v2:172.21.15.203:3300/0 conn(0x7efe140cb9c0 0x7efe140ac270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:47:22.848 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:22.842+0000 7efe11ffb640 1 -- 172.21.15.43:0/1480357763 --> v1:172.21.15.43:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efe1415d3e0 con 0x7efe140ad1f0 2023-08-09T13:47:22.848 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:22.842+0000 7efe11ffb640 1 -- 172.21.15.43:0/1480357763 <== mon.0 v1:172.21.15.43:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1285150740 0 0) 0x7efe08001670 con 0x7efe140ad1f0 2023-08-09T13:47:22.848 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:22.842+0000 7efe11ffb640 1 -- 172.21.15.43:0/1480357763 >> v1:172.21.15.43:6789/0 conn(0x7efe140ad1f0 legacy=0x7efe140cbda0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:47:22.848 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:22.842+0000 7efe11ffb640 1 --2- 172.21.15.43:0/1480357763 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7efe1415ca40 0x7efdfc0039a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:47:22.848 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:22.842+0000 7efe11ffb640 1 --2- 172.21.15.43:0/1480357763 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7efdfc003f20 0x7efdfc0047c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:47:22.848 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:22.842+0000 7efe11ffb640 1 --2- 172.21.15.43:0/1480357763 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7efdfc004d00 0x7efdfc005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:47:22.848 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:22.842+0000 7efe11ffb640 1 -- 172.21.15.43:0/1480357763 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efe1415d9a0 con 0x7efdfc003f20 2023-08-09T13:47:22.849 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:22.842+0000 7efe11ffb640 1 -- 172.21.15.43:0/1480357763 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efe140abe50 con 0x7efdfc004d00 2023-08-09T13:47:22.849 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:22.842+0000 7efe11ffb640 1 -- 172.21.15.43:0/1480357763 --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7efdfc00ba00 con 0x7efe1415ca40 2023-08-09T13:47:22.849 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:22.843+0000 7efe127fc640 1 --2- 172.21.15.43:0/1480357763 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7efdfc003f20 0x7efdfc0047c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:47:22.849 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:22.843+0000 7efe12ffd640 1 --2- 172.21.15.43:0/1480357763 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7efe1415ca40 0x7efdfc0039a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:47:22.850 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:22.843+0000 7efe137fe640 1 --2- 172.21.15.43:0/1480357763 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7efdfc004d00 0x7efdfc005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:47:22.850 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:22.843+0000 7efe127fc640 1 -- 172.21.15.43:0/1480357763 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7efe1415ca40 msgr2=0x7efdfc0039a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-09T13:47:22.850 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:22.843+0000 7efe127fc640 1 --2- 172.21.15.43:0/1480357763 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7efe1415ca40 0x7efdfc0039a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:47:22.850 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:22.843+0000 7efe127fc640 1 -- 172.21.15.43:0/1480357763 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7efdfc004d00 msgr2=0x7efdfc005180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-09T13:47:22.850 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:22.843+0000 7efe127fc640 1 --2- 172.21.15.43:0/1480357763 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7efdfc004d00 0x7efdfc005180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:47:22.851 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:22.843+0000 7efe127fc640 1 -- 172.21.15.43:0/1480357763 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efdfc003520 con 0x7efdfc003f20 2023-08-09T13:47:22.851 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:22.843+0000 7efe12ffd640 1 --2- 172.21.15.43:0/1480357763 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7efe1415ca40 0x7efdfc0039a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-08-09T13:47:22.856 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:22.844+0000 7efe127fc640 1 --2- 172.21.15.43:0/1480357763 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7efdfc003f20 0x7efdfc0047c0 secure :-1 s=READY pgs=660 cs=0 l=1 rev1=1 crypto rx=0x7efe0800c240 tx=0x7efdfc0057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d8011db6828a684f server_cookie=0 in_seq=0 out_seq=0 2023-08-09T13:47:22.856 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:22.845+0000 7efe11ffb640 1 -- 172.21.15.43:0/1480357763 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efe08005ad0 con 0x7efdfc003f20 2023-08-09T13:47:22.856 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:22.845+0000 7efe11ffb640 1 -- 172.21.15.43:0/1480357763 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efe08005c50 con 0x7efdfc003f20 2023-08-09T13:47:22.856 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:22.845+0000 7efe11ffb640 1 -- 172.21.15.43:0/1480357763 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efe0802c8b0 con 0x7efdfc003f20 2023-08-09T13:47:22.857 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:22.845+0000 7efe13fff640 1 -- 172.21.15.43:0/1480357763 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7efdfc003f20 msgr2=0x7efdfc0047c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:47:22.857 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:22.845+0000 7efe13fff640 1 --2- 172.21.15.43:0/1480357763 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7efdfc003f20 0x7efdfc0047c0 secure :-1 s=READY pgs=660 cs=0 l=1 rev1=1 crypto rx=0x7efe0800c240 tx=0x7efdfc0057f0 comp rx=0 tx=0).stop 2023-08-09T13:47:22.857 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:22.845+0000 7efe137fe640 1 -- 172.21.15.43:0/1480357763 reap_dead start 2023-08-09T13:47:22.857 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:22.845+0000 7efe13fff640 1 -- 172.21.15.43:0/1480357763 shutdown_connections 2023-08-09T13:47:22.857 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:22.845+0000 7efe13fff640 1 -- 172.21.15.43:0/1480357763 >> 172.21.15.43:0/1480357763 conn(0x7efe140ca870 msgr2=0x7efe140cac80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-09T13:47:22.857 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:22.845+0000 7efe13fff640 1 -- 172.21.15.43:0/1480357763 shutdown_connections 2023-08-09T13:47:22.857 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:22.845+0000 7efe13fff640 1 -- 172.21.15.43:0/1480357763 wait complete. 2023-08-09T13:47:22.858 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:22.848+0000 7efe13fff640 1 Processor -- start 2023-08-09T13:47:22.858 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:22.848+0000 7efe13fff640 1 -- start start 2023-08-09T13:47:22.858 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:22.849+0000 7efe13fff640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7efdfc003f20 0x7efe14148660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:47:22.858 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:22.849+0000 7efe13fff640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7efe140c49d0 0x7efe140c1750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:47:22.858 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:22.849+0000 7efe13fff640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7efe141f22f0 0x7efe141f2770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:47:22.858 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:22.849+0000 7efe13fff640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efe14158470 con 0x7efe140c49d0 2023-08-09T13:47:22.858 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:22.849+0000 7efe13fff640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efe140c1f80 con 0x7efe141f22f0 2023-08-09T13:47:22.859 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:22.849+0000 7efe13fff640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7efe14157ad0 con 0x7efdfc003f20 2023-08-09T13:47:22.859 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:22.849+0000 7efe127fc640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7efe140c49d0 0x7efe140c1750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:47:22.859 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:22.849+0000 7efe127fc640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7efe140c49d0 0x7efe140c1750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:57514/0 (socket says 172.21.15.43:57514) 2023-08-09T13:47:22.859 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:22.849+0000 7efe127fc640 1 -- 172.21.15.43:0/2451253382 learned_addr learned my addr 172.21.15.43:0/2451253382 (peer_addr_for_me v2:172.21.15.43:0/0) 2023-08-09T13:47:22.859 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:22.850+0000 7efe12ffd640 1 --2- 172.21.15.43:0/2451253382 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7efdfc003f20 0x7efe14148660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:47:22.859 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:22.850+0000 7efe137fe640 1 --2- 172.21.15.43:0/2451253382 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7efe141f22f0 0x7efe141f2770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:47:22.859 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:22.850+0000 7efe127fc640 1 -- 172.21.15.43:0/2451253382 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7efdfc003f20 msgr2=0x7efe14148660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:47:22.860 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:22.850+0000 7efe127fc640 1 --2- 172.21.15.43:0/2451253382 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7efdfc003f20 0x7efe14148660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:47:22.860 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:22.850+0000 7efe127fc640 1 -- 172.21.15.43:0/2451253382 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7efe141f22f0 msgr2=0x7efe141f2770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:47:22.860 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:22.850+0000 7efe127fc640 1 --2- 172.21.15.43:0/2451253382 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7efe141f22f0 0x7efe141f2770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:47:22.860 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:22.850+0000 7efe127fc640 1 -- 172.21.15.43:0/2451253382 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efe14157db0 con 0x7efe140c49d0 2023-08-09T13:47:22.860 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:22.850+0000 7efe127fc640 1 --2- 172.21.15.43:0/2451253382 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7efe140c49d0 0x7efe140c1750 secure :-1 s=READY pgs=661 cs=0 l=1 rev1=1 crypto rx=0x7efe0800c330 tx=0x7efe08001a00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-09T13:47:22.860 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:22.850+0000 7efdfb7fe640 1 -- 172.21.15.43:0/2451253382 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efe0803c050 con 0x7efe140c49d0 2023-08-09T13:47:22.861 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:22.851+0000 7efdfb7fe640 1 -- 172.21.15.43:0/2451253382 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efe0803a030 con 0x7efe140c49d0 2023-08-09T13:47:22.861 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:22.851+0000 7efdfb7fe640 1 -- 172.21.15.43:0/2451253382 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efe08007070 con 0x7efe140c49d0 2023-08-09T13:47:22.861 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:22.851+0000 7efe13fff640 1 -- 172.21.15.43:0/2451253382 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7efe14158090 con 0x7efe140c49d0 2023-08-09T13:47:22.861 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:22.851+0000 7efe13fff640 1 -- 172.21.15.43:0/2451253382 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7efe14158890 con 0x7efe140c49d0 2023-08-09T13:47:22.861 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:22.852+0000 7efe13fff640 1 -- 172.21.15.43:0/2451253382 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7efe141586e0 con 0x7efe140c49d0 2023-08-09T13:47:22.861 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:22.854+0000 7efdfb7fe640 1 -- 172.21.15.43:0/2451253382 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 13) v1 ==== 88529+0+0 (secure 0 0 0) 0x7efe08032070 con 0x7efe140c49d0 2023-08-09T13:47:22.862 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:22.854+0000 7efdfb7fe640 1 --2- 172.21.15.43:0/2451253382 >> [v2:172.21.15.203:6832/998005540,v1:172.21.15.203:6833/998005540] conn(0x7efe0006a250 0x7efe0006c710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:47:22.862 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:22.855+0000 7efdfb7fe640 1 -- 172.21.15.43:0/2451253382 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(42..42 src has 1..42) v4 ==== 6698+0+0 (secure 0 0 0) 0x7efe080aa1d0 con 0x7efe140c49d0 2023-08-09T13:47:22.862 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:22.855+0000 7efe12ffd640 1 --2- 172.21.15.43:0/2451253382 >> [v2:172.21.15.203:6832/998005540,v1:172.21.15.203:6833/998005540] conn(0x7efe0006a250 0x7efe0006c710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:47:22.863 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:22.856+0000 7efe12ffd640 1 --2- 172.21.15.43:0/2451253382 >> [v2:172.21.15.203:6832/998005540,v1:172.21.15.203:6833/998005540] conn(0x7efe0006a250 0x7efe0006c710 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7efe04007520 tx=0x7efe0400b040 comp rx=0 tx=0).ready entity=mgr.5003 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-09T13:47:22.863 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:22.859+0000 7efdfb7fe640 1 -- 172.21.15.43:0/2451253382 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178066 (secure 0 0 0) 0x7efe080753b0 con 0x7efe140c49d0 2023-08-09T13:47:23.083 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:23.080+0000 7efe13fff640 1 -- 172.21.15.43:0/2451253382 --> [v2:172.21.15.203:6832/998005540,v1:172.21.15.203:6833/998005540] -- mgr_command(tid 0: {"prefix": "fs snap-schedule deactivate", "fs": "cephfs", "path": "/snap_test_dir1/dir1", "repeat": "1M", "target": ["mon-mgr", ""]}) v1 -- 0x7efe14157c40 con 0x7efe0006a250 2023-08-09T13:47:23.255 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:23.253+0000 7efdfb7fe640 1 -- 172.21.15.43:0/2451253382 <== mgr.5003 v2:172.21.15.203:6832/998005540 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+51 (secure 0 0 0) 0x7efe14157c40 con 0x7efe0006a250 2023-08-09T13:47:23.255 INFO:teuthology.orchestra.run.smithi043.stdout:Schedule deactivated for path /snap_test_dir1/dir1 2023-08-09T13:47:23.258 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:23.257+0000 7efe13fff640 1 -- 172.21.15.43:0/2451253382 >> [v2:172.21.15.203:6832/998005540,v1:172.21.15.203:6833/998005540] conn(0x7efe0006a250 msgr2=0x7efe0006c710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:47:23.258 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:23.257+0000 7efe13fff640 1 --2- 172.21.15.43:0/2451253382 >> [v2:172.21.15.203:6832/998005540,v1:172.21.15.203:6833/998005540] conn(0x7efe0006a250 0x7efe0006c710 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7efe04007520 tx=0x7efe0400b040 comp rx=0 tx=0).stop 2023-08-09T13:47:23.258 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:23.257+0000 7efe13fff640 1 -- 172.21.15.43:0/2451253382 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7efe140c49d0 msgr2=0x7efe140c1750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:47:23.258 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:23.257+0000 7efe13fff640 1 --2- 172.21.15.43:0/2451253382 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7efe140c49d0 0x7efe140c1750 secure :-1 s=READY pgs=661 cs=0 l=1 rev1=1 crypto rx=0x7efe0800c330 tx=0x7efe08001a00 comp rx=0 tx=0).stop 2023-08-09T13:47:23.259 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:23.257+0000 7efe13fff640 1 -- 172.21.15.43:0/2451253382 shutdown_connections 2023-08-09T13:47:23.259 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:23.257+0000 7efe13fff640 1 --2- 172.21.15.43:0/2451253382 >> [v2:172.21.15.203:6832/998005540,v1:172.21.15.203:6833/998005540] conn(0x7efe0006a250 0x7efe0006c710 unknown :-1 s=CLOSED pgs=44 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:47:23.259 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:23.257+0000 7efe13fff640 1 --2- 172.21.15.43:0/2451253382 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7efe141f22f0 0x7efe141f2770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:47:23.259 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:23.257+0000 7efe13fff640 1 --2- 172.21.15.43:0/2451253382 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7efe140c49d0 0x7efe140c1750 unknown :-1 s=CLOSED pgs=661 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:47:23.259 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:23.257+0000 7efe13fff640 1 --2- 172.21.15.43:0/2451253382 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7efdfc003f20 0x7efe14148660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:47:23.259 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:23.257+0000 7efe13fff640 1 -- 172.21.15.43:0/2451253382 >> 172.21.15.43:0/2451253382 conn(0x7efe140ca870 msgr2=0x7efe1406b400 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-09T13:47:23.260 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:23.257+0000 7efe13fff640 1 -- 172.21.15.43:0/2451253382 shutdown_connections 2023-08-09T13:47:23.260 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:23.257+0000 7efe13fff640 1 -- 172.21.15.43:0/2451253382 wait complete. 2023-08-09T13:47:23.280 DEBUG:tasks.cephfs.test_snap_schedules:res=Schedule deactivated for path /snap_test_dir1/dir1 2023-08-09T13:47:23.281 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs snap-schedule deactivate --fs cephfs /snap_test_dir1/dir2 1M 2023-08-09T13:47:23.515 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:23.514+0000 7f0961cbf640 1 Processor -- start 2023-08-09T13:47:23.517 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:23.514+0000 7f0961cbf640 1 -- start start 2023-08-09T13:47:23.518 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:23.516+0000 7f0961cbf640 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7f095c15a560 0x7f095c0d09d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:47:23.518 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:23.516+0000 7f0961cbf640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f095c0d0f10 0x7f095c0cccd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:47:23.518 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:23.516+0000 7f0961cbf640 1 -- --> v1:172.21.15.43:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f095c0cd660 con 0x7f095c0cd210 2023-08-09T13:47:23.518 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:23.516+0000 7f0961cbf640 1 -- --> v2:172.21.15.43:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f095c15a940 con 0x7f095c0d0f10 2023-08-09T13:47:23.518 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:23.516+0000 7f0961cbf640 1 -- --> v2:172.21.15.203:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f095c0cbde0 con 0x7f095c15a560 2023-08-09T13:47:23.519 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:23.516+0000 7f0952ffd640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f095c0d0f10 0x7f095c0cccd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:47:23.519 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:23.516+0000 7f0952ffd640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f095c0d0f10 0x7f095c0cccd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:57520/0 (socket says 172.21.15.43:57520) 2023-08-09T13:47:23.519 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:23.516+0000 7f0952ffd640 1 -- 172.21.15.43:0/226601362 learned_addr learned my addr 172.21.15.43:0/226601362 (peer_addr_for_me v2:172.21.15.43:0/0) 2023-08-09T13:47:23.519 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:23.516+0000 7f095bfff640 1 --1- >> v1:172.21.15.43:6789/0 conn(0x7f095c0cd210 0x7f095c0b0f30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.43:6789/0 says I am v1:172.21.15.43:59566/0 (socket says 172.21.15.43:59566) 2023-08-09T13:47:23.519 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:23.516+0000 7f095b7fe640 1 --2- 172.21.15.43:0/226601362 >> v2:172.21.15.203:3300/0 conn(0x7f095c15a560 0x7f095c0d09d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:47:23.520 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:23.517+0000 7f0952ffd640 1 -- 172.21.15.43:0/226601362 >> v2:172.21.15.203:3300/0 conn(0x7f095c15a560 msgr2=0x7f095c0d09d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-09T13:47:23.520 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:23.517+0000 7f0952ffd640 1 --2- 172.21.15.43:0/226601362 >> v2:172.21.15.203:3300/0 conn(0x7f095c15a560 0x7f095c0d09d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:47:23.520 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:23.517+0000 7f0952ffd640 1 -- 172.21.15.43:0/226601362 >> v1:172.21.15.43:6789/0 conn(0x7f095c0cd210 legacy=0x7f095c0b0f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:47:23.520 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:23.517+0000 7f0952ffd640 1 -- 172.21.15.43:0/226601362 --> v2:172.21.15.43:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f095c0cb1d0 con 0x7f095c0d0f10 2023-08-09T13:47:23.520 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:23.518+0000 7f0952ffd640 1 --2- 172.21.15.43:0/226601362 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f095c0d0f10 0x7f095c0cccd0 secure :-1 s=READY pgs=663 cs=0 l=1 rev1=1 crypto rx=0x7f095c06ffb0 tx=0x7f094402b9a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=201a7c77d559c545 server_cookie=0 in_seq=0 out_seq=0 2023-08-09T13:47:23.520 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:23.518+0000 7f095affd640 1 -- 172.21.15.43:0/226601362 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0944030860 con 0x7f095c0d0f10 2023-08-09T13:47:23.521 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:23.518+0000 7f095affd640 1 -- 172.21.15.43:0/226601362 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f09440309c0 con 0x7f095c0d0f10 2023-08-09T13:47:23.521 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:23.518+0000 7f095affd640 1 -- 172.21.15.43:0/226601362 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0944030d50 con 0x7f095c0d0f10 2023-08-09T13:47:23.521 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:23.519+0000 7f0961cbf640 1 -- 172.21.15.43:0/226601362 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f095c0d0f10 msgr2=0x7f095c0cccd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:47:23.522 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:23.519+0000 7f0961cbf640 1 --2- 172.21.15.43:0/226601362 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f095c0d0f10 0x7f095c0cccd0 secure :-1 s=READY pgs=663 cs=0 l=1 rev1=1 crypto rx=0x7f095c06ffb0 tx=0x7f094402b9a0 comp rx=0 tx=0).stop 2023-08-09T13:47:23.522 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:23.519+0000 7f0961cbf640 1 -- 172.21.15.43:0/226601362 shutdown_connections 2023-08-09T13:47:23.522 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:23.519+0000 7f0961cbf640 1 --2- 172.21.15.43:0/226601362 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f095c0d0f10 0x7f095c0cccd0 unknown :-1 s=CLOSED pgs=663 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:47:23.522 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:23.519+0000 7f0961cbf640 1 --2- 172.21.15.43:0/226601362 >> v2:172.21.15.203:3300/0 conn(0x7f095c15a560 0x7f095c0d09d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:47:23.522 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:23.519+0000 7f0961cbf640 1 -- 172.21.15.43:0/226601362 >> 172.21.15.43:0/226601362 conn(0x7f095c099600 msgr2=0x7f095c0d02c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-09T13:47:23.522 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:23.519+0000 7f0961cbf640 1 -- 172.21.15.43:0/226601362 shutdown_connections 2023-08-09T13:47:23.523 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:23.519+0000 7f0961cbf640 1 -- 172.21.15.43:0/226601362 wait complete. 2023-08-09T13:47:23.523 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:23.521+0000 7f0961cbf640 1 Processor -- start 2023-08-09T13:47:23.523 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:23.521+0000 7f0961cbf640 1 -- start start 2023-08-09T13:47:23.524 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:23.522+0000 7f0961cbf640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f095c0cd210 0x7f095c0c7da0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:47:23.524 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:23.522+0000 7f0961cbf640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f095c0d0f10 0x7f095c156120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:47:23.524 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:23.522+0000 7f0961cbf640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f095c15a560 0x7f095c06b970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:47:23.524 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:23.522+0000 7f0961cbf640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f095c156660 con 0x7f095c0cd210 2023-08-09T13:47:23.525 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:23.522+0000 7f0961cbf640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f095c0c8f90 con 0x7f095c15a560 2023-08-09T13:47:23.525 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:23.522+0000 7f0961cbf640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f095c0c9420 con 0x7f095c0d0f10 2023-08-09T13:47:23.525 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:23.523+0000 7f095b7fe640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f095c0cd210 0x7f095c0c7da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:47:23.525 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:23.523+0000 7f095b7fe640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f095c0cd210 0x7f095c0c7da0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:57526/0 (socket says 172.21.15.43:57526) 2023-08-09T13:47:23.525 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:23.523+0000 7f095b7fe640 1 -- 172.21.15.43:0/2884919782 learned_addr learned my addr 172.21.15.43:0/2884919782 (peer_addr_for_me v2:172.21.15.43:0/0) 2023-08-09T13:47:23.527 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:23.523+0000 7f0952ffd640 1 --2- 172.21.15.43:0/2884919782 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f095c0d0f10 0x7f095c156120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:47:23.528 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:23.523+0000 7f095bfff640 1 --2- 172.21.15.43:0/2884919782 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f095c15a560 0x7f095c06b970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:47:23.528 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:23.523+0000 7f095b7fe640 1 -- 172.21.15.43:0/2884919782 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f095c0d0f10 msgr2=0x7f095c156120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:47:23.528 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:23.523+0000 7f095b7fe640 1 --2- 172.21.15.43:0/2884919782 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f095c0d0f10 0x7f095c156120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:47:23.529 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:23.523+0000 7f095b7fe640 1 -- 172.21.15.43:0/2884919782 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f095c15a560 msgr2=0x7f095c06b970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:47:23.529 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:23.523+0000 7f095b7fe640 1 --2- 172.21.15.43:0/2884919782 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f095c15a560 0x7f095c06b970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:47:23.529 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:23.523+0000 7f095b7fe640 1 -- 172.21.15.43:0/2884919782 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f09440094d0 con 0x7f095c0cd210 2023-08-09T13:47:23.529 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:23.524+0000 7f095b7fe640 1 --2- 172.21.15.43:0/2884919782 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f095c0cd210 0x7f095c0c7da0 secure :-1 s=READY pgs=664 cs=0 l=1 rev1=1 crypto rx=0x7f094c007ef0 tx=0x7f094c00e4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-09T13:47:23.530 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:23.524+0000 7f0958ff9640 1 -- 172.21.15.43:0/2884919782 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f094c014620 con 0x7f095c0cd210 2023-08-09T13:47:23.530 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:23.524+0000 7f0958ff9640 1 -- 172.21.15.43:0/2884919782 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f094c00ee90 con 0x7f095c0cd210 2023-08-09T13:47:23.530 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:23.524+0000 7f0958ff9640 1 -- 172.21.15.43:0/2884919782 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f094c0149c0 con 0x7f095c0cd210 2023-08-09T13:47:23.530 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:23.524+0000 7f0961cbf640 1 -- 172.21.15.43:0/2884919782 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f095c06beb0 con 0x7f095c0cd210 2023-08-09T13:47:23.531 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:23.524+0000 7f0961cbf640 1 -- 172.21.15.43:0/2884919782 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f095c0c8660 con 0x7f095c0cd210 2023-08-09T13:47:23.531 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:23.525+0000 7f0961cbf640 1 -- 172.21.15.43:0/2884919782 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f095c0739f0 con 0x7f095c0cd210 2023-08-09T13:47:23.532 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:23.526+0000 7f0958ff9640 1 -- 172.21.15.43:0/2884919782 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 13) v1 ==== 88529+0+0 (secure 0 0 0) 0x7f094c014b20 con 0x7f095c0cd210 2023-08-09T13:47:23.532 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:23.527+0000 7f0958ff9640 1 --2- 172.21.15.43:0/2884919782 >> [v2:172.21.15.203:6832/998005540,v1:172.21.15.203:6833/998005540] conn(0x7f093c06a250 0x7f093c06c710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:47:23.533 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:23.527+0000 7f0958ff9640 1 -- 172.21.15.43:0/2884919782 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(42..42 src has 1..42) v4 ==== 6698+0+0 (secure 0 0 0) 0x7f094c0895f0 con 0x7f095c0cd210 2023-08-09T13:47:23.533 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:23.528+0000 7f0952ffd640 1 --2- 172.21.15.43:0/2884919782 >> [v2:172.21.15.203:6832/998005540,v1:172.21.15.203:6833/998005540] conn(0x7f093c06a250 0x7f093c06c710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:47:23.533 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:23.529+0000 7f0952ffd640 1 --2- 172.21.15.43:0/2884919782 >> [v2:172.21.15.203:6832/998005540,v1:172.21.15.203:6833/998005540] conn(0x7f093c06a250 0x7f093c06c710 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7f0944036000 tx=0x7f0944008030 comp rx=0 tx=0).ready entity=mgr.5003 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-09T13:47:23.534 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:23.532+0000 7f0958ff9640 1 -- 172.21.15.43:0/2884919782 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178066 (secure 0 0 0) 0x7f094c054740 con 0x7f095c0cd210 2023-08-09T13:47:23.752 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:23.750+0000 7f0961cbf640 1 -- 172.21.15.43:0/2884919782 --> [v2:172.21.15.203:6832/998005540,v1:172.21.15.203:6833/998005540] -- mgr_command(tid 0: {"prefix": "fs snap-schedule deactivate", "fs": "cephfs", "path": "/snap_test_dir1/dir2", "repeat": "1M", "target": ["mon-mgr", ""]}) v1 -- 0x7f095c0cb790 con 0x7f093c06a250 2023-08-09T13:47:23.922 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:23.921+0000 7f0958ff9640 1 -- 172.21.15.43:0/2884919782 <== mgr.5003 v2:172.21.15.203:6832/998005540 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+51 (secure 0 0 0) 0x7f0944030bb0 con 0x7f093c06a250 2023-08-09T13:47:23.923 INFO:teuthology.orchestra.run.smithi043.stdout:Schedule deactivated for path /snap_test_dir1/dir2 2023-08-09T13:47:23.927 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:23.924+0000 7f0961cbf640 1 -- 172.21.15.43:0/2884919782 >> [v2:172.21.15.203:6832/998005540,v1:172.21.15.203:6833/998005540] conn(0x7f093c06a250 msgr2=0x7f093c06c710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:47:23.927 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:23.924+0000 7f0961cbf640 1 --2- 172.21.15.43:0/2884919782 >> [v2:172.21.15.203:6832/998005540,v1:172.21.15.203:6833/998005540] conn(0x7f093c06a250 0x7f093c06c710 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7f0944036000 tx=0x7f0944008030 comp rx=0 tx=0).stop 2023-08-09T13:47:23.927 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:23.924+0000 7f0961cbf640 1 -- 172.21.15.43:0/2884919782 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f095c0cd210 msgr2=0x7f095c0c7da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:47:23.928 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:23.924+0000 7f0961cbf640 1 --2- 172.21.15.43:0/2884919782 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f095c0cd210 0x7f095c0c7da0 secure :-1 s=READY pgs=664 cs=0 l=1 rev1=1 crypto rx=0x7f094c007ef0 tx=0x7f094c00e4b0 comp rx=0 tx=0).stop 2023-08-09T13:47:23.928 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:23.924+0000 7f0961cbf640 1 -- 172.21.15.43:0/2884919782 shutdown_connections 2023-08-09T13:47:23.928 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:23.924+0000 7f0961cbf640 1 --2- 172.21.15.43:0/2884919782 >> [v2:172.21.15.203:6832/998005540,v1:172.21.15.203:6833/998005540] conn(0x7f093c06a250 0x7f093c06c710 unknown :-1 s=CLOSED pgs=45 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:47:23.929 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:23.924+0000 7f0961cbf640 1 --2- 172.21.15.43:0/2884919782 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f095c15a560 0x7f095c06b970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:47:23.929 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:23.924+0000 7f0961cbf640 1 --2- 172.21.15.43:0/2884919782 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f095c0d0f10 0x7f095c156120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:47:23.929 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:23.924+0000 7f0961cbf640 1 --2- 172.21.15.43:0/2884919782 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f095c0cd210 0x7f095c0c7da0 unknown :-1 s=CLOSED pgs=664 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:47:23.930 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:23.924+0000 7f0961cbf640 1 -- 172.21.15.43:0/2884919782 >> 172.21.15.43:0/2884919782 conn(0x7f095c099600 msgr2=0x7f095c06ee80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-09T13:47:23.930 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:23.925+0000 7f0961cbf640 1 -- 172.21.15.43:0/2884919782 shutdown_connections 2023-08-09T13:47:23.930 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:23.925+0000 7f0961cbf640 1 -- 172.21.15.43:0/2884919782 wait complete. 2023-08-09T13:47:23.948 DEBUG:tasks.cephfs.test_snap_schedules:res=Schedule deactivated for path /snap_test_dir1/dir2 2023-08-09T13:47:23.948 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs snap-schedule deactivate --fs cephfs /snap_test_dir1/dir3 1M 2023-08-09T13:47:24.186 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:24.184+0000 7f6e0143f640 1 Processor -- start 2023-08-09T13:47:24.186 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:24.184+0000 7f6e0143f640 1 -- start start 2023-08-09T13:47:24.188 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:24.186+0000 7f6e0143f640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f6dfc06fb70 0x7f6dfc0caab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:47:24.188 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:24.186+0000 7f6e0143f640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f6dfc0ca370 0x7f6dfc15b2a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:47:24.189 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:24.186+0000 7f6e0143f640 1 -- --> v1:172.21.15.43:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6dfc15ae10 con 0x7f6dfc0ce4c0 2023-08-09T13:47:24.189 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:24.186+0000 7f6e0143f640 1 -- --> v2:172.21.15.43:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6dfc154e70 con 0x7f6dfc06fb70 2023-08-09T13:47:24.189 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:24.186+0000 7f6e0143f640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6dfc06ff50 con 0x7f6dfc0ca370 2023-08-09T13:47:24.189 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:24.186+0000 7f6dfaffd640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f6dfc06fb70 0x7f6dfc0caab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:47:24.190 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:24.186+0000 7f6dfaffd640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f6dfc06fb70 0x7f6dfc0caab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:57532/0 (socket says 172.21.15.43:57532) 2023-08-09T13:47:24.190 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:24.186+0000 7f6dfaffd640 1 -- 172.21.15.43:0/2347252296 learned_addr learned my addr 172.21.15.43:0/2347252296 (peer_addr_for_me v2:172.21.15.43:0/0) 2023-08-09T13:47:24.190 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:24.186+0000 7f6dfb7fe640 1 --1- >> v1:172.21.15.43:6789/0 conn(0x7f6dfc0ce4c0 0x7f6dfc0c86f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.43:6789/0 says I am v1:172.21.15.43:59582/0 (socket says 172.21.15.43:59582) 2023-08-09T13:47:24.191 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:24.187+0000 7f6dfa7fc640 1 --2- 172.21.15.43:0/2347252296 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f6dfc0ca370 0x7f6dfc15b2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:47:24.193 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:24.187+0000 7f6dfaffd640 1 -- 172.21.15.43:0/2347252296 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f6dfc0ca370 msgr2=0x7f6dfc15b2a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-09T13:47:24.193 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:24.187+0000 7f6dfaffd640 1 --2- 172.21.15.43:0/2347252296 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f6dfc0ca370 0x7f6dfc15b2a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:47:24.193 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:24.187+0000 7f6dfaffd640 1 -- 172.21.15.43:0/2347252296 >> v1:172.21.15.43:6789/0 conn(0x7f6dfc0ce4c0 legacy=0x7f6dfc0c86f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:47:24.194 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:24.187+0000 7f6dfaffd640 1 -- 172.21.15.43:0/2347252296 --> v2:172.21.15.43:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6dfc0cb320 con 0x7f6dfc06fb70 2023-08-09T13:47:24.194 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:24.187+0000 7f6df9ffb640 1 -- 172.21.15.43:0/2347252296 <== mon.0 v1:172.21.15.43:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1824577985 0 0) 0x7f6dfc15ae10 con 0x7f6dfc0ce4c0 2023-08-09T13:47:24.194 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:24.188+0000 7f6dfaffd640 1 --2- 172.21.15.43:0/2347252296 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f6dfc06fb70 0x7f6dfc0caab0 secure :-1 s=READY pgs=666 cs=0 l=1 rev1=1 crypto rx=0x7f6dec009870 tx=0x7f6dec02b980 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a228e088cab9747b server_cookie=0 in_seq=0 out_seq=0 2023-08-09T13:47:24.195 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:24.188+0000 7f6df9ffb640 1 -- 172.21.15.43:0/2347252296 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6dec030840 con 0x7f6dfc06fb70 2023-08-09T13:47:24.195 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:24.189+0000 7f6df9ffb640 1 -- 172.21.15.43:0/2347252296 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6dec0309a0 con 0x7f6dfc06fb70 2023-08-09T13:47:24.195 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:24.189+0000 7f6df9ffb640 1 -- 172.21.15.43:0/2347252296 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6dec030d30 con 0x7f6dfc06fb70 2023-08-09T13:47:24.195 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:24.189+0000 7f6e0143f640 1 -- 172.21.15.43:0/2347252296 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f6dfc06fb70 msgr2=0x7f6dfc0caab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:47:24.195 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:24.189+0000 7f6e0143f640 1 --2- 172.21.15.43:0/2347252296 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f6dfc06fb70 0x7f6dfc0caab0 secure :-1 s=READY pgs=666 cs=0 l=1 rev1=1 crypto rx=0x7f6dec009870 tx=0x7f6dec02b980 comp rx=0 tx=0).stop 2023-08-09T13:47:24.196 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:24.189+0000 7f6e0143f640 1 -- 172.21.15.43:0/2347252296 shutdown_connections 2023-08-09T13:47:24.196 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:24.189+0000 7f6e0143f640 1 --2- 172.21.15.43:0/2347252296 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f6dfc0ca370 0x7f6dfc15b2a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:47:24.196 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:24.189+0000 7f6e0143f640 1 --2- 172.21.15.43:0/2347252296 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f6dfc06fb70 0x7f6dfc0caab0 unknown :-1 s=CLOSED pgs=666 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:47:24.197 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:24.189+0000 7f6e0143f640 1 -- 172.21.15.43:0/2347252296 >> 172.21.15.43:0/2347252296 conn(0x7f6dfc06b990 msgr2=0x7f6dfc06bda0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-09T13:47:24.197 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:24.189+0000 7f6e0143f640 1 -- 172.21.15.43:0/2347252296 shutdown_connections 2023-08-09T13:47:24.197 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:24.189+0000 7f6e0143f640 1 -- 172.21.15.43:0/2347252296 wait complete. 2023-08-09T13:47:24.197 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:24.191+0000 7f6e0143f640 1 Processor -- start 2023-08-09T13:47:24.198 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:24.191+0000 7f6e0143f640 1 -- start start 2023-08-09T13:47:24.201 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:24.192+0000 7f6e0143f640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f6dfc06fb70 0x7f6dfc0aeae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:47:24.201 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:24.192+0000 7f6e0143f640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f6dfc0ca370 0x7f6dfc153690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:47:24.201 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:24.192+0000 7f6e0143f640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f6dfc0ce4c0 0x7f6dfc155050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:47:24.202 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:24.192+0000 7f6e0143f640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6dfc153bd0 con 0x7f6dfc0ca370 2023-08-09T13:47:24.202 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:24.192+0000 7f6e0143f640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6dfc0c3410 con 0x7f6dfc06fb70 2023-08-09T13:47:24.202 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:24.192+0000 7f6e0143f640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6dfc157880 con 0x7f6dfc0ce4c0 2023-08-09T13:47:24.203 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:24.193+0000 7f6dfa7fc640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f6dfc0ca370 0x7f6dfc153690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:47:24.203 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:24.193+0000 7f6dfa7fc640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f6dfc0ca370 0x7f6dfc153690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:57540/0 (socket says 172.21.15.43:57540) 2023-08-09T13:47:24.203 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:24.193+0000 7f6dfa7fc640 1 -- 172.21.15.43:0/2272105723 learned_addr learned my addr 172.21.15.43:0/2272105723 (peer_addr_for_me v2:172.21.15.43:0/0) 2023-08-09T13:47:24.203 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:24.193+0000 7f6dfb7fe640 1 --2- 172.21.15.43:0/2272105723 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f6dfc0ce4c0 0x7f6dfc155050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:47:24.204 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:24.193+0000 7f6dfa7fc640 1 -- 172.21.15.43:0/2272105723 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f6dfc0ce4c0 msgr2=0x7f6dfc155050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:47:24.204 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:24.193+0000 7f6dfa7fc640 1 --2- 172.21.15.43:0/2272105723 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f6dfc0ce4c0 0x7f6dfc155050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:47:24.204 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:24.193+0000 7f6dfa7fc640 1 -- 172.21.15.43:0/2272105723 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f6dfc06fb70 msgr2=0x7f6dfc0aeae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:47:24.204 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:24.193+0000 7f6dfa7fc640 1 --2- 172.21.15.43:0/2272105723 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f6dfc06fb70 0x7f6dfc0aeae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:47:24.205 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:24.193+0000 7f6dfa7fc640 1 -- 172.21.15.43:0/2272105723 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6dec0094d0 con 0x7f6dfc0ca370 2023-08-09T13:47:24.205 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:24.193+0000 7f6dfb7fe640 1 --2- 172.21.15.43:0/2272105723 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f6dfc0ce4c0 0x7f6dfc155050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-08-09T13:47:24.205 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:24.193+0000 7f6dfa7fc640 1 --2- 172.21.15.43:0/2272105723 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f6dfc0ca370 0x7f6dfc153690 secure :-1 s=READY pgs=667 cs=0 l=1 rev1=1 crypto rx=0x7f6df000d800 tx=0x7f6df000dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-09T13:47:24.205 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:24.194+0000 7f6de37fe640 1 -- 172.21.15.43:0/2272105723 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6df000b6c0 con 0x7f6dfc0ca370 2023-08-09T13:47:24.206 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:24.194+0000 7f6de37fe640 1 -- 172.21.15.43:0/2272105723 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6df000b820 con 0x7f6dfc0ca370 2023-08-09T13:47:24.206 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:24.194+0000 7f6de37fe640 1 -- 172.21.15.43:0/2272105723 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6df000bbb0 con 0x7f6dfc0ca370 2023-08-09T13:47:24.206 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:24.194+0000 7f6e0143f640 1 -- 172.21.15.43:0/2272105723 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6dfc157b60 con 0x7f6dfc0ca370 2023-08-09T13:47:24.207 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:24.194+0000 7f6e0143f640 1 -- 172.21.15.43:0/2272105723 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6dfc0c30a0 con 0x7f6dfc0ca370 2023-08-09T13:47:24.207 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:24.195+0000 7f6e0143f640 1 -- 172.21.15.43:0/2272105723 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6dfc0c3260 con 0x7f6dfc0ca370 2023-08-09T13:47:24.207 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:24.196+0000 7f6de37fe640 1 -- 172.21.15.43:0/2272105723 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 13) v1 ==== 88529+0+0 (secure 0 0 0) 0x7f6df000bd10 con 0x7f6dfc0ca370 2023-08-09T13:47:24.207 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:24.197+0000 7f6de37fe640 1 --2- 172.21.15.43:0/2272105723 >> [v2:172.21.15.203:6832/998005540,v1:172.21.15.203:6833/998005540] conn(0x7f6de806a250 0x7f6de806c710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:47:24.208 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:24.197+0000 7f6de37fe640 1 -- 172.21.15.43:0/2272105723 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(42..42 src has 1..42) v4 ==== 6698+0+0 (secure 0 0 0) 0x7f6df0089170 con 0x7f6dfc0ca370 2023-08-09T13:47:24.208 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:24.198+0000 7f6dfaffd640 1 --2- 172.21.15.43:0/2272105723 >> [v2:172.21.15.203:6832/998005540,v1:172.21.15.203:6833/998005540] conn(0x7f6de806a250 0x7f6de806c710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:47:24.208 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:24.199+0000 7f6dfaffd640 1 --2- 172.21.15.43:0/2272105723 >> [v2:172.21.15.203:6832/998005540,v1:172.21.15.203:6833/998005540] conn(0x7f6de806a250 0x7f6de806c710 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7f6dec036000 tx=0x7f6dec03b040 comp rx=0 tx=0).ready entity=mgr.5003 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-09T13:47:24.209 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:24.202+0000 7f6de37fe640 1 -- 172.21.15.43:0/2272105723 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178066 (secure 0 0 0) 0x7f6df00542c0 con 0x7f6dfc0ca370 2023-08-09T13:47:24.421 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:24.419+0000 7f6e0143f640 1 -- 172.21.15.43:0/2272105723 --> [v2:172.21.15.203:6832/998005540,v1:172.21.15.203:6833/998005540] -- mgr_command(tid 0: {"prefix": "fs snap-schedule deactivate", "fs": "cephfs", "path": "/snap_test_dir1/dir3", "repeat": "1M", "target": ["mon-mgr", ""]}) v1 -- 0x7f6dfc0c3580 con 0x7f6de806a250 2023-08-09T13:47:24.589 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:24.588+0000 7f6de37fe640 1 -- 172.21.15.43:0/2272105723 <== mgr.5003 v2:172.21.15.203:6832/998005540 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+51 (secure 0 0 0) 0x7f6dfc0c3580 con 0x7f6de806a250 2023-08-09T13:47:24.590 INFO:teuthology.orchestra.run.smithi043.stdout:Schedule deactivated for path /snap_test_dir1/dir3 2023-08-09T13:47:24.592 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:24.591+0000 7f6e0143f640 1 -- 172.21.15.43:0/2272105723 >> [v2:172.21.15.203:6832/998005540,v1:172.21.15.203:6833/998005540] conn(0x7f6de806a250 msgr2=0x7f6de806c710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:47:24.593 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:24.591+0000 7f6e0143f640 1 --2- 172.21.15.43:0/2272105723 >> [v2:172.21.15.203:6832/998005540,v1:172.21.15.203:6833/998005540] conn(0x7f6de806a250 0x7f6de806c710 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7f6dec036000 tx=0x7f6dec03b040 comp rx=0 tx=0).stop 2023-08-09T13:47:24.593 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:24.591+0000 7f6e0143f640 1 -- 172.21.15.43:0/2272105723 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f6dfc0ca370 msgr2=0x7f6dfc153690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:47:24.594 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:24.591+0000 7f6e0143f640 1 --2- 172.21.15.43:0/2272105723 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f6dfc0ca370 0x7f6dfc153690 secure :-1 s=READY pgs=667 cs=0 l=1 rev1=1 crypto rx=0x7f6df000d800 tx=0x7f6df000dcc0 comp rx=0 tx=0).stop 2023-08-09T13:47:24.594 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:24.591+0000 7f6e0143f640 1 -- 172.21.15.43:0/2272105723 shutdown_connections 2023-08-09T13:47:24.594 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:24.591+0000 7f6e0143f640 1 --2- 172.21.15.43:0/2272105723 >> [v2:172.21.15.203:6832/998005540,v1:172.21.15.203:6833/998005540] conn(0x7f6de806a250 0x7f6de806c710 unknown :-1 s=CLOSED pgs=46 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:47:24.594 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:24.591+0000 7f6e0143f640 1 --2- 172.21.15.43:0/2272105723 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f6dfc0ce4c0 0x7f6dfc155050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:47:24.594 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:24.591+0000 7f6e0143f640 1 --2- 172.21.15.43:0/2272105723 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f6dfc0ca370 0x7f6dfc153690 unknown :-1 s=CLOSED pgs=667 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:47:24.594 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:24.591+0000 7f6e0143f640 1 --2- 172.21.15.43:0/2272105723 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f6dfc06fb70 0x7f6dfc0aeae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:47:24.594 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:24.591+0000 7f6e0143f640 1 -- 172.21.15.43:0/2272105723 >> 172.21.15.43:0/2272105723 conn(0x7f6dfc06b990 msgr2=0x7f6dfc0cc9c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-09T13:47:24.595 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:24.591+0000 7f6e0143f640 1 -- 172.21.15.43:0/2272105723 shutdown_connections 2023-08-09T13:47:24.595 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:24.592+0000 7f6e0143f640 1 -- 172.21.15.43:0/2272105723 wait complete. 2023-08-09T13:47:24.614 DEBUG:tasks.cephfs.test_snap_schedules:res=Schedule deactivated for path /snap_test_dir1/dir3 2023-08-09T13:47:24.614 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs snap-schedule deactivate --fs cephfs /snap_test_dir1/dir4 1M 2023-08-09T13:47:24.846 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:24.844+0000 7fcfa5b76640 1 Processor -- start 2023-08-09T13:47:24.846 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:24.844+0000 7fcfa5b76640 1 -- start start 2023-08-09T13:47:24.847 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:24.846+0000 7fcfa5b76640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fcfa006d560 0x7fcfa00cd9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:47:24.848 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:24.846+0000 7fcfa5b76640 1 -- --> v1:172.21.15.43:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fcfa015cf30 con 0x7fcfa0158440 2023-08-09T13:47:24.848 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:24.846+0000 7fcfa5b76640 1 -- --> v1:172.21.15.203:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fcfa00c8900 con 0x7fcfa00cb170 2023-08-09T13:47:24.848 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:24.846+0000 7fcfa5b76640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcfa00c86f0 con 0x7fcfa006d560 2023-08-09T13:47:24.849 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:24.846+0000 7fcf9f7fe640 1 --1- >> v1:172.21.15.43:6789/0 conn(0x7fcfa0158440 0x7fcfa00caa60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.43:6789/0 says I am v1:172.21.15.43:59594/0 (socket says 172.21.15.43:59594) 2023-08-09T13:47:24.850 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:24.846+0000 7fcf9f7fe640 1 -- 172.21.15.43:0/2219572002 learned_addr learned my addr 172.21.15.43:0/2219572002 (peer_addr_for_me v1:172.21.15.43:0/0) 2023-08-09T13:47:24.850 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:24.847+0000 7fcf9effd640 1 --2- 172.21.15.43:0/2219572002 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fcfa006d560 0x7fcfa00cd9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:47:24.850 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:24.847+0000 7fcf9e7fc640 1 -- 172.21.15.43:0/2219572002 <== mon.0 v1:172.21.15.43:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1610853848 0 0) 0x7fcfa015cf30 con 0x7fcfa0158440 2023-08-09T13:47:24.850 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:24.847+0000 7fcf9e7fc640 1 -- 172.21.15.43:0/2219572002 --> v1:172.21.15.43:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fcf88003520 con 0x7fcfa0158440 2023-08-09T13:47:24.850 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:24.847+0000 7fcf9e7fc640 1 -- 172.21.15.43:0/2219572002 <== mon.0 v1:172.21.15.43:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1083445376 0 0) 0x7fcf88003520 con 0x7fcfa0158440 2023-08-09T13:47:24.850 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:24.847+0000 7fcf9e7fc640 1 -- 172.21.15.43:0/2219572002 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fcfa006d560 msgr2=0x7fcfa00cd9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-09T13:47:24.850 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:24.847+0000 7fcf9e7fc640 1 --2- 172.21.15.43:0/2219572002 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fcfa006d560 0x7fcfa00cd9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:47:24.851 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:24.847+0000 7fcf9e7fc640 1 -- 172.21.15.43:0/2219572002 >> v1:172.21.15.203:6789/0 conn(0x7fcfa00cb170 legacy=0x7fcfa00cc030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:47:24.851 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:24.847+0000 7fcf9e7fc640 1 -- 172.21.15.43:0/2219572002 --> v1:172.21.15.43:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcfa00c8be0 con 0x7fcfa0158440 2023-08-09T13:47:24.851 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:24.847+0000 7fcf9e7fc640 1 -- 172.21.15.43:0/2219572002 <== mon.0 v1:172.21.15.43:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1285150740 0 0) 0x7fcf90001670 con 0x7fcfa0158440 2023-08-09T13:47:24.851 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:24.847+0000 7fcf9e7fc640 1 -- 172.21.15.43:0/2219572002 >> v1:172.21.15.43:6789/0 conn(0x7fcfa0158440 legacy=0x7fcfa00caa60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:47:24.851 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:24.848+0000 7fcf9e7fc640 1 --2- 172.21.15.43:0/2219572002 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fcf88003a10 0x7fcf88003e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:47:24.851 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:24.848+0000 7fcf9e7fc640 1 --2- 172.21.15.43:0/2219572002 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fcf880047c0 0x7fcf88004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:47:24.851 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:24.848+0000 7fcf9e7fc640 1 --2- 172.21.15.43:0/2219572002 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fcfa006d560 0x7fcf88005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:47:24.852 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:24.848+0000 7fcf9e7fc640 1 -- 172.21.15.43:0/2219572002 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcfa00c86f0 con 0x7fcf88003a10 2023-08-09T13:47:24.852 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:24.848+0000 7fcf9e7fc640 1 -- 172.21.15.43:0/2219572002 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcf8800ba80 con 0x7fcf880047c0 2023-08-09T13:47:24.852 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:24.848+0000 7fcf9e7fc640 1 -- 172.21.15.43:0/2219572002 --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcf8800bc40 con 0x7fcfa006d560 2023-08-09T13:47:24.852 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:24.848+0000 7fcf9f7fe640 1 --2- 172.21.15.43:0/2219572002 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fcf88003a10 0x7fcf88003e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:47:24.852 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:24.848+0000 7fcf9effd640 1 --2- 172.21.15.43:0/2219572002 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fcf880047c0 0x7fcf88004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:47:24.852 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:24.848+0000 7fcf9ffff640 1 --2- 172.21.15.43:0/2219572002 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fcfa006d560 0x7fcf88005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:47:24.853 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:24.848+0000 7fcf9f7fe640 1 -- 172.21.15.43:0/2219572002 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fcfa006d560 msgr2=0x7fcf88005180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-09T13:47:24.853 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:24.848+0000 7fcf9f7fe640 1 --2- 172.21.15.43:0/2219572002 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fcfa006d560 0x7fcf88005180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:47:24.854 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:24.848+0000 7fcf9f7fe640 1 -- 172.21.15.43:0/2219572002 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fcf880047c0 msgr2=0x7fcf88004c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-09T13:47:24.854 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:24.848+0000 7fcf9f7fe640 1 --2- 172.21.15.43:0/2219572002 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fcf880047c0 0x7fcf88004c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:47:24.854 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:24.848+0000 7fcf9f7fe640 1 -- 172.21.15.43:0/2219572002 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcf88003520 con 0x7fcf88003a10 2023-08-09T13:47:24.854 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:24.850+0000 7fcf9f7fe640 1 --2- 172.21.15.43:0/2219572002 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fcf88003a10 0x7fcf88003e30 secure :-1 s=READY pgs=669 cs=0 l=1 rev1=1 crypto rx=0x7fcf9000c240 tx=0x7fcf880057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e29c190d61f49f56 server_cookie=0 in_seq=0 out_seq=0 2023-08-09T13:47:24.854 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:24.850+0000 7fcf9e7fc640 1 -- 172.21.15.43:0/2219572002 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcf90005ad0 con 0x7fcf88003a10 2023-08-09T13:47:24.854 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:24.850+0000 7fcf9e7fc640 1 -- 172.21.15.43:0/2219572002 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcf90005c50 con 0x7fcf88003a10 2023-08-09T13:47:24.854 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:24.850+0000 7fcf9e7fc640 1 -- 172.21.15.43:0/2219572002 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcf9002c8b0 con 0x7fcf88003a10 2023-08-09T13:47:24.855 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:24.850+0000 7fcfa5b76640 1 -- 172.21.15.43:0/2219572002 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fcf88003a10 msgr2=0x7fcf88003e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:47:24.855 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:24.850+0000 7fcfa5b76640 1 --2- 172.21.15.43:0/2219572002 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fcf88003a10 0x7fcf88003e30 secure :-1 s=READY pgs=669 cs=0 l=1 rev1=1 crypto rx=0x7fcf9000c240 tx=0x7fcf880057f0 comp rx=0 tx=0).stop 2023-08-09T13:47:24.855 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:24.851+0000 7fcf9ffff640 1 -- 172.21.15.43:0/2219572002 reap_dead start 2023-08-09T13:47:24.855 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:24.851+0000 7fcfa5b76640 1 -- 172.21.15.43:0/2219572002 shutdown_connections 2023-08-09T13:47:24.855 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:24.851+0000 7fcfa5b76640 1 -- 172.21.15.43:0/2219572002 >> 172.21.15.43:0/2219572002 conn(0x7fcfa006b990 msgr2=0x7fcfa006bda0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-09T13:47:24.855 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:24.851+0000 7fcfa5b76640 1 -- 172.21.15.43:0/2219572002 shutdown_connections 2023-08-09T13:47:24.855 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:24.851+0000 7fcfa5b76640 1 -- 172.21.15.43:0/2219572002 wait complete. 2023-08-09T13:47:24.857 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:24.853+0000 7fcfa5b76640 1 Processor -- start 2023-08-09T13:47:24.857 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:24.853+0000 7fcfa5b76640 1 -- start start 2023-08-09T13:47:24.857 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:24.854+0000 7fcfa5b76640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fcf88003a10 0x7fcfa00accc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:47:24.857 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:24.854+0000 7fcfa5b76640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fcfa00cc7f0 0x7fcfa014bdf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:47:24.858 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:24.854+0000 7fcfa5b76640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fcfa0156b10 0x7fcfa014c330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:47:24.858 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:24.854+0000 7fcfa5b76640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcfa0155450 con 0x7fcfa0156b10 2023-08-09T13:47:24.858 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:24.854+0000 7fcfa5b76640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcfa0143a20 con 0x7fcfa00cc7f0 2023-08-09T13:47:24.858 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:24.854+0000 7fcfa5b76640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcfa0142770 con 0x7fcf88003a10 2023-08-09T13:47:24.858 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:24.854+0000 7fcf9ffff640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fcfa0156b10 0x7fcfa014c330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:47:24.858 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:24.854+0000 7fcf9ffff640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fcfa0156b10 0x7fcfa014c330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:57552/0 (socket says 172.21.15.43:57552) 2023-08-09T13:47:24.858 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:24.854+0000 7fcf9ffff640 1 -- 172.21.15.43:0/322620151 learned_addr learned my addr 172.21.15.43:0/322620151 (peer_addr_for_me v2:172.21.15.43:0/0) 2023-08-09T13:47:24.858 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:24.855+0000 7fcf9f7fe640 1 --2- 172.21.15.43:0/322620151 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fcf88003a10 0x7fcfa00accc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:47:24.859 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:24.855+0000 7fcf9ffff640 1 -- 172.21.15.43:0/322620151 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fcf88003a10 msgr2=0x7fcfa00accc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:47:24.859 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:24.855+0000 7fcf9effd640 1 --2- 172.21.15.43:0/322620151 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fcfa00cc7f0 0x7fcfa014bdf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:47:24.859 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:24.855+0000 7fcf9ffff640 1 --2- 172.21.15.43:0/322620151 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fcf88003a10 0x7fcfa00accc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:47:24.859 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:24.855+0000 7fcf9ffff640 1 -- 172.21.15.43:0/322620151 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fcfa00cc7f0 msgr2=0x7fcfa014bdf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:47:24.859 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:24.855+0000 7fcf9ffff640 1 --2- 172.21.15.43:0/322620151 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fcfa00cc7f0 0x7fcfa014bdf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:47:24.859 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:24.855+0000 7fcf9ffff640 1 -- 172.21.15.43:0/322620151 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcfa01556a0 con 0x7fcfa0156b10 2023-08-09T13:47:24.859 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:24.855+0000 7fcf9f7fe640 1 --2- 172.21.15.43:0/322620151 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fcf88003a10 0x7fcfa00accc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-08-09T13:47:24.860 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:24.855+0000 7fcf9ffff640 1 --2- 172.21.15.43:0/322620151 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fcfa0156b10 0x7fcfa014c330 secure :-1 s=READY pgs=670 cs=0 l=1 rev1=1 crypto rx=0x7fcfa00b0c10 tx=0x7fcf9800ec10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-09T13:47:24.860 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:24.855+0000 7fcf87fff640 1 -- 172.21.15.43:0/322620151 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcf98003970 con 0x7fcfa0156b10 2023-08-09T13:47:24.860 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:24.855+0000 7fcf87fff640 1 -- 172.21.15.43:0/322620151 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcf98003ad0 con 0x7fcfa0156b10 2023-08-09T13:47:24.860 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:24.855+0000 7fcf87fff640 1 -- 172.21.15.43:0/322620151 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcf98003e80 con 0x7fcfa0156b10 2023-08-09T13:47:24.860 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:24.855+0000 7fcfa5b76640 1 -- 172.21.15.43:0/322620151 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcfa0155980 con 0x7fcfa0156b10 2023-08-09T13:47:24.860 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:24.856+0000 7fcfa5b76640 1 -- 172.21.15.43:0/322620151 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcfa0142cf0 con 0x7fcfa0156b10 2023-08-09T13:47:24.861 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:24.857+0000 7fcfa5b76640 1 -- 172.21.15.43:0/322620151 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcfa00739f0 con 0x7fcfa0156b10 2023-08-09T13:47:24.861 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:24.858+0000 7fcf87fff640 1 -- 172.21.15.43:0/322620151 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 13) v1 ==== 88529+0+0 (secure 0 0 0) 0x7fcf980050d0 con 0x7fcfa0156b10 2023-08-09T13:47:24.861 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:24.858+0000 7fcf87fff640 1 --2- 172.21.15.43:0/322620151 >> [v2:172.21.15.203:6832/998005540,v1:172.21.15.203:6833/998005540] conn(0x7fcf8c06a250 0x7fcf8c06c710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:47:24.861 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:24.859+0000 7fcf87fff640 1 -- 172.21.15.43:0/322620151 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(42..42 src has 1..42) v4 ==== 6698+0+0 (secure 0 0 0) 0x7fcf98088a40 con 0x7fcfa0156b10 2023-08-09T13:47:24.862 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:24.859+0000 7fcf9f7fe640 1 --2- 172.21.15.43:0/322620151 >> [v2:172.21.15.203:6832/998005540,v1:172.21.15.203:6833/998005540] conn(0x7fcf8c06a250 0x7fcf8c06c710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:47:24.862 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:24.860+0000 7fcf9f7fe640 1 --2- 172.21.15.43:0/322620151 >> [v2:172.21.15.203:6832/998005540,v1:172.21.15.203:6833/998005540] conn(0x7fcf8c06a250 0x7fcf8c06c710 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7fcf90034000 tx=0x7fcf90009030 comp rx=0 tx=0).ready entity=mgr.5003 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-09T13:47:24.864 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:24.863+0000 7fcf87fff640 1 -- 172.21.15.43:0/322620151 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178066 (secure 0 0 0) 0x7fcf98053b90 con 0x7fcfa0156b10 2023-08-09T13:47:25.083 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:25.081+0000 7fcfa5b76640 1 -- 172.21.15.43:0/322620151 --> [v2:172.21.15.203:6832/998005540,v1:172.21.15.203:6833/998005540] -- mgr_command(tid 0: {"prefix": "fs snap-schedule deactivate", "fs": "cephfs", "path": "/snap_test_dir1/dir4", "repeat": "1M", "target": ["mon-mgr", ""]}) v1 -- 0x7fcfa015cbe0 con 0x7fcf8c06a250 2023-08-09T13:47:25.263 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:25.262+0000 7fcf87fff640 1 -- 172.21.15.43:0/322620151 <== mgr.5003 v2:172.21.15.203:6832/998005540 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+51 (secure 0 0 0) 0x7fcf90001430 con 0x7fcf8c06a250 2023-08-09T13:47:25.264 INFO:teuthology.orchestra.run.smithi043.stdout:Schedule deactivated for path /snap_test_dir1/dir4 2023-08-09T13:47:25.266 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:25.265+0000 7fcfa5b76640 1 -- 172.21.15.43:0/322620151 >> [v2:172.21.15.203:6832/998005540,v1:172.21.15.203:6833/998005540] conn(0x7fcf8c06a250 msgr2=0x7fcf8c06c710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:47:25.267 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:25.265+0000 7fcfa5b76640 1 --2- 172.21.15.43:0/322620151 >> [v2:172.21.15.203:6832/998005540,v1:172.21.15.203:6833/998005540] conn(0x7fcf8c06a250 0x7fcf8c06c710 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7fcf90034000 tx=0x7fcf90009030 comp rx=0 tx=0).stop 2023-08-09T13:47:25.267 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:25.265+0000 7fcfa5b76640 1 -- 172.21.15.43:0/322620151 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fcfa0156b10 msgr2=0x7fcfa014c330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:47:25.267 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:25.265+0000 7fcfa5b76640 1 --2- 172.21.15.43:0/322620151 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fcfa0156b10 0x7fcfa014c330 secure :-1 s=READY pgs=670 cs=0 l=1 rev1=1 crypto rx=0x7fcfa00b0c10 tx=0x7fcf9800ec10 comp rx=0 tx=0).stop 2023-08-09T13:47:25.267 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:25.265+0000 7fcfa5b76640 1 -- 172.21.15.43:0/322620151 shutdown_connections 2023-08-09T13:47:25.268 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:25.265+0000 7fcfa5b76640 1 --2- 172.21.15.43:0/322620151 >> [v2:172.21.15.203:6832/998005540,v1:172.21.15.203:6833/998005540] conn(0x7fcf8c06a250 0x7fcf8c06c710 unknown :-1 s=CLOSED pgs=47 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:47:25.268 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:25.265+0000 7fcfa5b76640 1 --2- 172.21.15.43:0/322620151 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fcfa0156b10 0x7fcfa014c330 unknown :-1 s=CLOSED pgs=670 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:47:25.268 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:25.265+0000 7fcfa5b76640 1 --2- 172.21.15.43:0/322620151 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fcfa00cc7f0 0x7fcfa014bdf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:47:25.268 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:25.265+0000 7fcfa5b76640 1 --2- 172.21.15.43:0/322620151 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fcf88003a10 0x7fcfa00accc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:47:25.268 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:25.265+0000 7fcfa5b76640 1 -- 172.21.15.43:0/322620151 >> 172.21.15.43:0/322620151 conn(0x7fcfa006b990 msgr2=0x7fcfa0149ad0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-09T13:47:25.268 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:25.266+0000 7fcfa5b76640 1 -- 172.21.15.43:0/322620151 shutdown_connections 2023-08-09T13:47:25.268 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:25.266+0000 7fcfa5b76640 1 -- 172.21.15.43:0/322620151 wait complete. 2023-08-09T13:47:25.286 DEBUG:tasks.cephfs.test_snap_schedules:res=Schedule deactivated for path /snap_test_dir1/dir4 2023-08-09T13:47:25.287 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs snap-schedule deactivate --fs cephfs /snap_test_dir1/dir5 1M 2023-08-09T13:47:25.525 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:25.524+0000 7f6fc0b64640 1 Processor -- start 2023-08-09T13:47:25.526 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:25.524+0000 7f6fc0b64640 1 -- start start 2023-08-09T13:47:25.527 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:25.526+0000 7f6fc0b64640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f6fbc071360 0x7f6fbc158fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:47:25.527 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:25.526+0000 7f6fc0b64640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f6fbc15c610 0x7f6fbc0c9330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:47:25.528 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:25.526+0000 7f6fc0b64640 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7f6fbc0c9870 0x7f6fbc0cc7f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:47:25.528 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:25.526+0000 7f6fc0b64640 1 -- --> v2:172.21.15.43:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6fbc0c8ea0 con 0x7f6fbc15c610 2023-08-09T13:47:25.528 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:25.526+0000 7f6fc0b64640 1 -- --> v2:172.21.15.203:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6fbc071740 con 0x7f6fbc0c9870 2023-08-09T13:47:25.528 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:25.526+0000 7f6fc0b64640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6fbc159900 con 0x7f6fbc071360 2023-08-09T13:47:25.529 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:25.526+0000 7f6fb1d9b640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f6fbc15c610 0x7f6fbc0c9330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:47:25.529 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:25.526+0000 7f6fb1d9b640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f6fbc15c610 0x7f6fbc0c9330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:53008/0 (socket says 172.21.15.43:53008) 2023-08-09T13:47:25.529 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:25.526+0000 7f6fb1d9b640 1 -- 172.21.15.43:0/1989101553 learned_addr learned my addr 172.21.15.43:0/1989101553 (peer_addr_for_me v2:172.21.15.43:0/0) 2023-08-09T13:47:25.529 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:25.527+0000 7f6fba59c640 1 --2- 172.21.15.43:0/1989101553 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f6fbc071360 0x7f6fbc158fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:47:25.529 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:25.527+0000 7f6fbad9d640 1 --2- 172.21.15.43:0/1989101553 >> v2:172.21.15.203:3300/0 conn(0x7f6fbc0c9870 0x7f6fbc0cc7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:47:25.530 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:25.527+0000 7f6fb1d9b640 1 -- 172.21.15.43:0/1989101553 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f6fbc071360 msgr2=0x7f6fbc158fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-09T13:47:25.530 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:25.527+0000 7f6fb1d9b640 1 --2- 172.21.15.43:0/1989101553 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f6fbc071360 0x7f6fbc158fb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:47:25.530 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:25.527+0000 7f6fb1d9b640 1 -- 172.21.15.43:0/1989101553 >> v2:172.21.15.203:3300/0 conn(0x7f6fbc0c9870 msgr2=0x7f6fbc0cc7f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-09T13:47:25.530 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:25.527+0000 7f6fb1d9b640 1 --2- 172.21.15.43:0/1989101553 >> v2:172.21.15.203:3300/0 conn(0x7f6fbc0c9870 0x7f6fbc0cc7f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:47:25.530 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:25.527+0000 7f6fb1d9b640 1 -- 172.21.15.43:0/1989101553 --> v2:172.21.15.43:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6fbc159be0 con 0x7f6fbc15c610 2023-08-09T13:47:25.531 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:25.528+0000 7f6fb1d9b640 1 --2- 172.21.15.43:0/1989101553 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f6fbc15c610 0x7f6fbc0c9330 secure :-1 s=READY pgs=671 cs=0 l=1 rev1=1 crypto rx=0x7f6fbc15d650 tx=0x7f6fa402e0e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d04324768f16e2c server_cookie=0 in_seq=0 out_seq=0 2023-08-09T13:47:25.531 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:25.529+0000 7f6fb9d9b640 1 -- 172.21.15.43:0/1989101553 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6fa402ee90 con 0x7f6fbc15c610 2023-08-09T13:47:25.531 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:25.529+0000 7f6fb9d9b640 1 -- 172.21.15.43:0/1989101553 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6fa4036440 con 0x7f6fbc15c610 2023-08-09T13:47:25.532 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:25.529+0000 7f6fb9d9b640 1 -- 172.21.15.43:0/1989101553 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6fa40367d0 con 0x7f6fbc15c610 2023-08-09T13:47:25.532 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:25.530+0000 7f6fc0b64640 1 -- 172.21.15.43:0/1989101553 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f6fbc15c610 msgr2=0x7f6fbc0c9330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:47:25.532 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:25.530+0000 7f6fc0b64640 1 --2- 172.21.15.43:0/1989101553 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f6fbc15c610 0x7f6fbc0c9330 secure :-1 s=READY pgs=671 cs=0 l=1 rev1=1 crypto rx=0x7f6fbc15d650 tx=0x7f6fa402e0e0 comp rx=0 tx=0).stop 2023-08-09T13:47:25.532 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:25.530+0000 7f6fc0b64640 1 -- 172.21.15.43:0/1989101553 shutdown_connections 2023-08-09T13:47:25.532 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:25.530+0000 7f6fc0b64640 1 --2- 172.21.15.43:0/1989101553 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f6fbc15c610 0x7f6fbc0c9330 unknown :-1 s=CLOSED pgs=671 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:47:25.532 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:25.530+0000 7f6fc0b64640 1 --2- 172.21.15.43:0/1989101553 >> v2:172.21.15.203:3300/0 conn(0x7f6fbc0c9870 0x7f6fbc0cc7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:47:25.532 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:25.530+0000 7f6fc0b64640 1 --2- 172.21.15.43:0/1989101553 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f6fbc071360 0x7f6fbc158fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:47:25.532 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:25.530+0000 7f6fc0b64640 1 -- 172.21.15.43:0/1989101553 >> 172.21.15.43:0/1989101553 conn(0x7f6fbc06b990 msgr2=0x7f6fbc06bda0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-09T13:47:25.533 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:25.530+0000 7f6fc0b64640 1 -- 172.21.15.43:0/1989101553 shutdown_connections 2023-08-09T13:47:25.533 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:25.530+0000 7f6fc0b64640 1 -- 172.21.15.43:0/1989101553 wait complete. 2023-08-09T13:47:25.533 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:25.532+0000 7f6fc0b64640 1 Processor -- start 2023-08-09T13:47:25.534 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:25.532+0000 7f6fc0b64640 1 -- start start 2023-08-09T13:47:25.535 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:25.533+0000 7f6fc0b64640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f6fbc071360 0x7f6fbc14be30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:47:25.535 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:25.533+0000 7f6fc0b64640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f6fbc0c9870 0x7f6fbc0c53e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:47:25.535 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:25.533+0000 7f6fc0b64640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f6fbc15c610 0x7f6fbc150610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:47:25.535 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:25.533+0000 7f6fc0b64640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6fbc0c7280 con 0x7f6fbc071360 2023-08-09T13:47:25.535 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:25.533+0000 7f6fc0b64640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6fbc0c7520 con 0x7f6fbc0c9870 2023-08-09T13:47:25.535 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:25.533+0000 7f6fc0b64640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6fbc0c7800 con 0x7f6fbc15c610 2023-08-09T13:47:25.537 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:25.534+0000 7f6fba59c640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f6fbc071360 0x7f6fbc14be30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:47:25.537 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:25.534+0000 7f6fba59c640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f6fbc071360 0x7f6fbc14be30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:53020/0 (socket says 172.21.15.43:53020) 2023-08-09T13:47:25.537 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:25.534+0000 7f6fba59c640 1 -- 172.21.15.43:0/691479691 learned_addr learned my addr 172.21.15.43:0/691479691 (peer_addr_for_me v2:172.21.15.43:0/0) 2023-08-09T13:47:25.537 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:25.534+0000 7f6fb1d9b640 1 --2- 172.21.15.43:0/691479691 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f6fbc0c9870 0x7f6fbc0c53e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:47:25.537 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:25.534+0000 7f6fbad9d640 1 --2- 172.21.15.43:0/691479691 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f6fbc15c610 0x7f6fbc150610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:47:25.537 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:25.534+0000 7f6fba59c640 1 -- 172.21.15.43:0/691479691 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f6fbc15c610 msgr2=0x7f6fbc150610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:47:25.537 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:25.534+0000 7f6fba59c640 1 --2- 172.21.15.43:0/691479691 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f6fbc15c610 0x7f6fbc150610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:47:25.538 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:25.534+0000 7f6fba59c640 1 -- 172.21.15.43:0/691479691 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f6fbc0c9870 msgr2=0x7f6fbc0c53e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:47:25.538 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:25.534+0000 7f6fba59c640 1 --2- 172.21.15.43:0/691479691 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f6fbc0c9870 0x7f6fbc0c53e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:47:25.538 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:25.534+0000 7f6fba59c640 1 -- 172.21.15.43:0/691479691 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6fa400b770 con 0x7f6fbc071360 2023-08-09T13:47:25.538 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:25.534+0000 7f6fb1d9b640 1 --2- 172.21.15.43:0/691479691 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f6fbc0c9870 0x7f6fbc0c53e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-08-09T13:47:25.538 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:25.535+0000 7f6fba59c640 1 --2- 172.21.15.43:0/691479691 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f6fbc071360 0x7f6fbc14be30 secure :-1 s=READY pgs=672 cs=0 l=1 rev1=1 crypto rx=0x7f6fac00eed0 tx=0x7f6fac00b480 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-09T13:47:25.538 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:25.535+0000 7f6fb37fe640 1 -- 172.21.15.43:0/691479691 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6fac010040 con 0x7f6fbc071360 2023-08-09T13:47:25.538 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:25.535+0000 7f6fb37fe640 1 -- 172.21.15.43:0/691479691 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6fac009e80 con 0x7f6fbc071360 2023-08-09T13:47:25.538 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:25.535+0000 7f6fb37fe640 1 -- 172.21.15.43:0/691479691 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6fac014620 con 0x7f6fbc071360 2023-08-09T13:47:25.539 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:25.535+0000 7f6fc0b64640 1 -- 172.21.15.43:0/691479691 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6fbc0c7ae0 con 0x7f6fbc071360 2023-08-09T13:47:25.539 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:25.535+0000 7f6fc0b64640 1 -- 172.21.15.43:0/691479691 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6fbc14c4d0 con 0x7f6fbc071360 2023-08-09T13:47:25.539 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:25.536+0000 7f6fc0b64640 1 -- 172.21.15.43:0/691479691 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6fbc0739f0 con 0x7f6fbc071360 2023-08-09T13:47:25.540 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:25.537+0000 7f6fb37fe640 1 -- 172.21.15.43:0/691479691 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 13) v1 ==== 88529+0+0 (secure 0 0 0) 0x7f6fac014780 con 0x7f6fbc071360 2023-08-09T13:47:25.540 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:25.538+0000 7f6fb37fe640 1 --2- 172.21.15.43:0/691479691 >> [v2:172.21.15.203:6832/998005540,v1:172.21.15.203:6833/998005540] conn(0x7f6f9c06a250 0x7f6f9c06c710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:47:25.540 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:25.538+0000 7f6fb37fe640 1 -- 172.21.15.43:0/691479691 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(42..42 src has 1..42) v4 ==== 6698+0+0 (secure 0 0 0) 0x7f6fac089430 con 0x7f6fbc071360 2023-08-09T13:47:25.540 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:25.539+0000 7f6fb1d9b640 1 --2- 172.21.15.43:0/691479691 >> [v2:172.21.15.203:6832/998005540,v1:172.21.15.203:6833/998005540] conn(0x7f6f9c06a250 0x7f6f9c06c710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:47:25.541 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:25.540+0000 7f6fb1d9b640 1 --2- 172.21.15.43:0/691479691 >> [v2:172.21.15.203:6832/998005540,v1:172.21.15.203:6833/998005540] conn(0x7f6f9c06a250 0x7f6f9c06c710 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7f6fa4038000 tx=0x7f6fa4008040 comp rx=0 tx=0).ready entity=mgr.5003 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-09T13:47:25.544 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:25.543+0000 7f6fb37fe640 1 -- 172.21.15.43:0/691479691 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178066 (secure 0 0 0) 0x7f6fac054500 con 0x7f6fbc071360 2023-08-09T13:47:25.763 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:25.761+0000 7f6fc0b64640 1 -- 172.21.15.43:0/691479691 --> [v2:172.21.15.203:6832/998005540,v1:172.21.15.203:6833/998005540] -- mgr_command(tid 0: {"prefix": "fs snap-schedule deactivate", "fs": "cephfs", "path": "/snap_test_dir1/dir5", "repeat": "1M", "target": ["mon-mgr", ""]}) v1 -- 0x7f6fbc0c8b30 con 0x7f6f9c06a250 2023-08-09T13:47:25.938 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:25.937+0000 7f6fb37fe640 1 -- 172.21.15.43:0/691479691 <== mgr.5003 v2:172.21.15.203:6832/998005540 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+51 (secure 0 0 0) 0x7f6fa40369a0 con 0x7f6f9c06a250 2023-08-09T13:47:25.939 INFO:teuthology.orchestra.run.smithi043.stdout:Schedule deactivated for path /snap_test_dir1/dir5 2023-08-09T13:47:25.942 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:25.940+0000 7f6fc0b64640 1 -- 172.21.15.43:0/691479691 >> [v2:172.21.15.203:6832/998005540,v1:172.21.15.203:6833/998005540] conn(0x7f6f9c06a250 msgr2=0x7f6f9c06c710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:47:25.942 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:25.940+0000 7f6fc0b64640 1 --2- 172.21.15.43:0/691479691 >> [v2:172.21.15.203:6832/998005540,v1:172.21.15.203:6833/998005540] conn(0x7f6f9c06a250 0x7f6f9c06c710 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7f6fa4038000 tx=0x7f6fa4008040 comp rx=0 tx=0).stop 2023-08-09T13:47:25.942 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:25.940+0000 7f6fc0b64640 1 -- 172.21.15.43:0/691479691 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f6fbc071360 msgr2=0x7f6fbc14be30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:47:25.942 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:25.940+0000 7f6fc0b64640 1 --2- 172.21.15.43:0/691479691 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f6fbc071360 0x7f6fbc14be30 secure :-1 s=READY pgs=672 cs=0 l=1 rev1=1 crypto rx=0x7f6fac00eed0 tx=0x7f6fac00b480 comp rx=0 tx=0).stop 2023-08-09T13:47:25.943 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:25.940+0000 7f6fc0b64640 1 -- 172.21.15.43:0/691479691 shutdown_connections 2023-08-09T13:47:25.943 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:25.940+0000 7f6fc0b64640 1 --2- 172.21.15.43:0/691479691 >> [v2:172.21.15.203:6832/998005540,v1:172.21.15.203:6833/998005540] conn(0x7f6f9c06a250 0x7f6f9c06c710 unknown :-1 s=CLOSED pgs=48 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:47:25.943 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:25.940+0000 7f6fc0b64640 1 --2- 172.21.15.43:0/691479691 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f6fbc15c610 0x7f6fbc150610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:47:25.943 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:25.940+0000 7f6fc0b64640 1 --2- 172.21.15.43:0/691479691 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f6fbc0c9870 0x7f6fbc0c53e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:47:25.943 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:25.940+0000 7f6fc0b64640 1 --2- 172.21.15.43:0/691479691 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f6fbc071360 0x7f6fbc14be30 unknown :-1 s=CLOSED pgs=672 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:47:25.943 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:25.941+0000 7f6fc0b64640 1 -- 172.21.15.43:0/691479691 >> 172.21.15.43:0/691479691 conn(0x7f6fbc06b990 msgr2=0x7f6fbc153400 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-09T13:47:25.944 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:25.941+0000 7f6fc0b64640 1 -- 172.21.15.43:0/691479691 shutdown_connections 2023-08-09T13:47:25.944 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:25.941+0000 7f6fc0b64640 1 -- 172.21.15.43:0/691479691 wait complete. 2023-08-09T13:47:25.965 DEBUG:tasks.cephfs.test_snap_schedules:res=Schedule deactivated for path /snap_test_dir1/dir5 2023-08-09T13:47:25.966 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs snap-schedule deactivate --fs cephfs /snap_test_dir1/dir6 1M 2023-08-09T13:47:26.202 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:26.200+0000 7ff148ef3640 1 Processor -- start 2023-08-09T13:47:26.202 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:26.201+0000 7ff148ef3640 1 -- start start 2023-08-09T13:47:26.204 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:26.202+0000 7ff148ef3640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7ff14415c720 0x7ff1440cd850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:47:26.205 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:26.202+0000 7ff148ef3640 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7ff1440d0900 0x7ff1440cec60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:47:26.205 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:26.202+0000 7ff148ef3640 1 -- --> v1:172.21.15.43:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff1440cf5c0 con 0x7ff1440cf1a0 2023-08-09T13:47:26.205 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:26.202+0000 7ff148ef3640 1 -- --> v2:172.21.15.43:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff14415cb00 con 0x7ff14415c720 2023-08-09T13:47:26.205 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:26.202+0000 7ff148ef3640 1 -- --> v2:172.21.15.203:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff1440d1960 con 0x7ff1440d0900 2023-08-09T13:47:26.206 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:26.203+0000 7ff143fff640 1 --1- >> v1:172.21.15.43:6789/0 conn(0x7ff1440cf1a0 0x7ff1440b3cd0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.43:6789/0 says I am v1:172.21.15.43:49536/0 (socket says 172.21.15.43:49536) 2023-08-09T13:47:26.207 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:26.203+0000 7ff143fff640 1 -- 172.21.15.43:0/3328728339 learned_addr learned my addr 172.21.15.43:0/3328728339 (peer_addr_for_me v1:172.21.15.43:0/0) 2023-08-09T13:47:26.207 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:26.203+0000 7ff1437fe640 1 --2- 172.21.15.43:0/3328728339 >> v2:172.21.15.43:3300/0 conn(0x7ff14415c720 0x7ff1440cd850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:47:26.207 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:26.203+0000 7ff142ffd640 1 --2- 172.21.15.43:0/3328728339 >> v2:172.21.15.203:3300/0 conn(0x7ff1440d0900 0x7ff1440cec60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:47:26.207 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:26.203+0000 7ff1427fc640 1 -- 172.21.15.43:0/3328728339 <== mon.0 v1:172.21.15.43:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4073679119 0 0) 0x7ff1440cf5c0 con 0x7ff1440cf1a0 2023-08-09T13:47:26.207 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:26.203+0000 7ff1427fc640 1 -- 172.21.15.43:0/3328728339 --> v1:172.21.15.43:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ff130003520 con 0x7ff1440cf1a0 2023-08-09T13:47:26.207 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:26.204+0000 7ff1427fc640 1 -- 172.21.15.43:0/3328728339 <== mon.0 v1:172.21.15.43:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3979706141 0 0) 0x7ff130003520 con 0x7ff1440cf1a0 2023-08-09T13:47:26.208 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:26.204+0000 7ff1437fe640 1 -- 172.21.15.43:0/3328728339 >> v2:172.21.15.203:3300/0 conn(0x7ff1440d0900 msgr2=0x7ff1440cec60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-09T13:47:26.208 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:26.204+0000 7ff1437fe640 1 --2- 172.21.15.43:0/3328728339 >> v2:172.21.15.203:3300/0 conn(0x7ff1440d0900 0x7ff1440cec60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:47:26.208 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:26.204+0000 7ff1437fe640 1 -- 172.21.15.43:0/3328728339 >> v1:172.21.15.43:6789/0 conn(0x7ff1440cf1a0 legacy=0x7ff1440b3cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:47:26.208 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:26.204+0000 7ff1437fe640 1 -- 172.21.15.43:0/3328728339 --> v2:172.21.15.43:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff1440b5660 con 0x7ff14415c720 2023-08-09T13:47:26.208 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:26.205+0000 7ff1437fe640 1 --2- 172.21.15.43:0/3328728339 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff14415c720 0x7ff1440cd850 secure :-1 s=READY pgs=674 cs=0 l=1 rev1=1 crypto rx=0x7ff1440cd150 tx=0x7ff13402b790 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6fdd6f88a616d1f2 server_cookie=0 in_seq=0 out_seq=0 2023-08-09T13:47:26.208 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:26.205+0000 7ff1427fc640 1 -- 172.21.15.43:0/3328728339 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff13c0019f0 con 0x7ff14415c720 2023-08-09T13:47:26.208 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:26.205+0000 7ff1427fc640 1 -- 172.21.15.43:0/3328728339 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff13402e050 con 0x7ff14415c720 2023-08-09T13:47:26.209 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:26.205+0000 7ff1427fc640 1 -- 172.21.15.43:0/3328728339 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff134030730 con 0x7ff14415c720 2023-08-09T13:47:26.209 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:26.206+0000 7ff148ef3640 1 -- 172.21.15.43:0/3328728339 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff14415c720 msgr2=0x7ff1440cd850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:47:26.210 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:26.206+0000 7ff148ef3640 1 --2- 172.21.15.43:0/3328728339 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff14415c720 0x7ff1440cd850 secure :-1 s=READY pgs=674 cs=0 l=1 rev1=1 crypto rx=0x7ff1440cd150 tx=0x7ff13402b790 comp rx=0 tx=0).stop 2023-08-09T13:47:26.210 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:26.206+0000 7ff148ef3640 1 -- 172.21.15.43:0/3328728339 shutdown_connections 2023-08-09T13:47:26.210 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:26.206+0000 7ff148ef3640 1 --2- 172.21.15.43:0/3328728339 >> v2:172.21.15.203:3300/0 conn(0x7ff1440d0900 0x7ff1440cec60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:47:26.210 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:26.206+0000 7ff148ef3640 1 --2- 172.21.15.43:0/3328728339 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff14415c720 0x7ff1440cd850 unknown :-1 s=CLOSED pgs=674 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:47:26.210 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:26.206+0000 7ff148ef3640 1 -- 172.21.15.43:0/3328728339 >> 172.21.15.43:0/3328728339 conn(0x7ff14406a2a0 msgr2=0x7ff14406a680 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-09T13:47:26.210 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:26.206+0000 7ff148ef3640 1 -- 172.21.15.43:0/3328728339 shutdown_connections 2023-08-09T13:47:26.210 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:26.206+0000 7ff148ef3640 1 -- 172.21.15.43:0/3328728339 wait complete. 2023-08-09T13:47:26.211 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:26.208+0000 7ff148ef3640 1 Processor -- start 2023-08-09T13:47:26.211 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:26.208+0000 7ff148ef3640 1 -- start start 2023-08-09T13:47:26.212 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:26.209+0000 7ff148ef3640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff1440cf1a0 0x7ff144156a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:47:26.212 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:26.209+0000 7ff148ef3640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7ff1440d0900 0x7ff1440c5cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:47:26.212 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:26.209+0000 7ff148ef3640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7ff14415c720 0x7ff1440c74f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:47:26.213 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:26.209+0000 7ff148ef3640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff14415d0e0 con 0x7ff1440cf1a0 2023-08-09T13:47:26.213 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:26.209+0000 7ff148ef3640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff1440b22b0 con 0x7ff14415c720 2023-08-09T13:47:26.213 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:26.209+0000 7ff148ef3640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff1440c6d70 con 0x7ff1440d0900 2023-08-09T13:47:26.213 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:26.209+0000 7ff1437fe640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff1440cf1a0 0x7ff144156a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:47:26.213 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:26.209+0000 7ff1437fe640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff1440cf1a0 0x7ff144156a50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:53036/0 (socket says 172.21.15.43:53036) 2023-08-09T13:47:26.213 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:26.209+0000 7ff1437fe640 1 -- 172.21.15.43:0/3245617197 learned_addr learned my addr 172.21.15.43:0/3245617197 (peer_addr_for_me v2:172.21.15.43:0/0) 2023-08-09T13:47:26.213 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:26.210+0000 7ff143fff640 1 --2- 172.21.15.43:0/3245617197 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7ff14415c720 0x7ff1440c74f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:47:26.213 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:26.210+0000 7ff142ffd640 1 --2- 172.21.15.43:0/3245617197 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7ff1440d0900 0x7ff1440c5cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:47:26.214 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:26.210+0000 7ff1437fe640 1 -- 172.21.15.43:0/3245617197 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7ff1440d0900 msgr2=0x7ff1440c5cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:47:26.214 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:26.210+0000 7ff1437fe640 1 --2- 172.21.15.43:0/3245617197 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7ff1440d0900 0x7ff1440c5cf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:47:26.214 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:26.210+0000 7ff1437fe640 1 -- 172.21.15.43:0/3245617197 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7ff14415c720 msgr2=0x7ff1440c74f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:47:26.214 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:26.210+0000 7ff1437fe640 1 --2- 172.21.15.43:0/3245617197 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7ff14415c720 0x7ff1440c74f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:47:26.214 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:26.210+0000 7ff1437fe640 1 -- 172.21.15.43:0/3245617197 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff134009400 con 0x7ff1440cf1a0 2023-08-09T13:47:26.214 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:26.210+0000 7ff143fff640 1 --2- 172.21.15.43:0/3245617197 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7ff14415c720 0x7ff1440c74f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-08-09T13:47:26.214 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:26.210+0000 7ff142ffd640 1 --2- 172.21.15.43:0/3245617197 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7ff1440d0900 0x7ff1440c5cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-08-09T13:47:26.215 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:26.210+0000 7ff1437fe640 1 --2- 172.21.15.43:0/3245617197 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff1440cf1a0 0x7ff144156a50 secure :-1 s=READY pgs=675 cs=0 l=1 rev1=1 crypto rx=0x7ff134000c00 tx=0x7ff1340046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-09T13:47:26.215 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:26.211+0000 7ff13bfff640 1 -- 172.21.15.43:0/3245617197 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff13403b810 con 0x7ff1440cf1a0 2023-08-09T13:47:26.215 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:26.211+0000 7ff13bfff640 1 -- 172.21.15.43:0/3245617197 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff13403b970 con 0x7ff1440cf1a0 2023-08-09T13:47:26.215 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:26.211+0000 7ff13bfff640 1 -- 172.21.15.43:0/3245617197 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff13403bd20 con 0x7ff1440cf1a0 2023-08-09T13:47:26.215 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:26.211+0000 7ff148ef3640 1 -- 172.21.15.43:0/3245617197 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff1440c67b0 con 0x7ff1440cf1a0 2023-08-09T13:47:26.215 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:26.211+0000 7ff148ef3640 1 -- 172.21.15.43:0/3245617197 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff14415cd70 con 0x7ff1440cf1a0 2023-08-09T13:47:26.216 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:26.212+0000 7ff148ef3640 1 -- 172.21.15.43:0/3245617197 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff14415cf00 con 0x7ff1440cf1a0 2023-08-09T13:47:26.216 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:26.213+0000 7ff13bfff640 1 -- 172.21.15.43:0/3245617197 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 13) v1 ==== 88529+0+0 (secure 0 0 0) 0x7ff134039040 con 0x7ff1440cf1a0 2023-08-09T13:47:26.216 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:26.214+0000 7ff13bfff640 1 --2- 172.21.15.43:0/3245617197 >> [v2:172.21.15.203:6832/998005540,v1:172.21.15.203:6833/998005540] conn(0x7ff12406a250 0x7ff12406c710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:47:26.217 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:26.214+0000 7ff13bfff640 1 -- 172.21.15.43:0/3245617197 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(42..42 src has 1..42) v4 ==== 6698+0+0 (secure 0 0 0) 0x7ff1340aa2d0 con 0x7ff1440cf1a0 2023-08-09T13:47:26.217 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:26.215+0000 7ff142ffd640 1 --2- 172.21.15.43:0/3245617197 >> [v2:172.21.15.203:6832/998005540,v1:172.21.15.203:6833/998005540] conn(0x7ff12406a250 0x7ff12406c710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:47:26.217 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:26.216+0000 7ff142ffd640 1 --2- 172.21.15.43:0/3245617197 >> [v2:172.21.15.203:6832/998005540,v1:172.21.15.203:6833/998005540] conn(0x7ff12406a250 0x7ff12406c710 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7ff1441f4df0 tx=0x7ff12c008040 comp rx=0 tx=0).ready entity=mgr.5003 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-09T13:47:26.219 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:26.218+0000 7ff13bfff640 1 -- 172.21.15.43:0/3245617197 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178066 (secure 0 0 0) 0x7ff134075440 con 0x7ff1440cf1a0 2023-08-09T13:47:26.440 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:26.439+0000 7ff148ef3640 1 -- 172.21.15.43:0/3245617197 --> [v2:172.21.15.203:6832/998005540,v1:172.21.15.203:6833/998005540] -- mgr_command(tid 0: {"prefix": "fs snap-schedule deactivate", "fs": "cephfs", "path": "/snap_test_dir1/dir6", "repeat": "1M", "target": ["mon-mgr", ""]}) v1 -- 0x7ff1440d10c0 con 0x7ff12406a250 2023-08-09T13:47:26.609 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:26.608+0000 7ff13bfff640 1 -- 172.21.15.43:0/3245617197 <== mgr.5003 v2:172.21.15.203:6832/998005540 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+51 (secure 0 0 0) 0x7ff12c010900 con 0x7ff12406a250 2023-08-09T13:47:26.610 INFO:teuthology.orchestra.run.smithi043.stdout:Schedule deactivated for path /snap_test_dir1/dir6 2023-08-09T13:47:26.613 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:26.612+0000 7ff148ef3640 1 -- 172.21.15.43:0/3245617197 >> [v2:172.21.15.203:6832/998005540,v1:172.21.15.203:6833/998005540] conn(0x7ff12406a250 msgr2=0x7ff12406c710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:47:26.613 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:26.612+0000 7ff148ef3640 1 --2- 172.21.15.43:0/3245617197 >> [v2:172.21.15.203:6832/998005540,v1:172.21.15.203:6833/998005540] conn(0x7ff12406a250 0x7ff12406c710 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7ff1441f4df0 tx=0x7ff12c008040 comp rx=0 tx=0).stop 2023-08-09T13:47:26.614 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:26.612+0000 7ff148ef3640 1 -- 172.21.15.43:0/3245617197 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff1440cf1a0 msgr2=0x7ff144156a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:47:26.614 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:26.612+0000 7ff148ef3640 1 --2- 172.21.15.43:0/3245617197 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff1440cf1a0 0x7ff144156a50 secure :-1 s=READY pgs=675 cs=0 l=1 rev1=1 crypto rx=0x7ff134000c00 tx=0x7ff1340046c0 comp rx=0 tx=0).stop 2023-08-09T13:47:26.614 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:26.612+0000 7ff148ef3640 1 -- 172.21.15.43:0/3245617197 shutdown_connections 2023-08-09T13:47:26.614 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:26.612+0000 7ff148ef3640 1 --2- 172.21.15.43:0/3245617197 >> [v2:172.21.15.203:6832/998005540,v1:172.21.15.203:6833/998005540] conn(0x7ff12406a250 0x7ff12406c710 unknown :-1 s=CLOSED pgs=49 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:47:26.614 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:26.612+0000 7ff148ef3640 1 --2- 172.21.15.43:0/3245617197 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7ff14415c720 0x7ff1440c74f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:47:26.614 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:26.612+0000 7ff148ef3640 1 --2- 172.21.15.43:0/3245617197 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7ff1440d0900 0x7ff1440c5cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:47:26.614 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:26.612+0000 7ff148ef3640 1 --2- 172.21.15.43:0/3245617197 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff1440cf1a0 0x7ff144156a50 unknown :-1 s=CLOSED pgs=675 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:47:26.615 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:26.612+0000 7ff148ef3640 1 -- 172.21.15.43:0/3245617197 >> 172.21.15.43:0/3245617197 conn(0x7ff14406a2a0 msgr2=0x7ff1440cfd90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-09T13:47:26.615 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:26.612+0000 7ff148ef3640 1 -- 172.21.15.43:0/3245617197 shutdown_connections 2023-08-09T13:47:26.615 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:26.612+0000 7ff148ef3640 1 -- 172.21.15.43:0/3245617197 wait complete. 2023-08-09T13:47:26.633 DEBUG:tasks.cephfs.test_snap_schedules:res=Schedule deactivated for path /snap_test_dir1/dir6 2023-08-09T13:47:26.634 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs snap-schedule deactivate --fs cephfs /snap_test_dir1/dir7 1M 2023-08-09T13:47:26.866 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:26.865+0000 7f960eb22640 1 Processor -- start 2023-08-09T13:47:26.867 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:26.865+0000 7f960eb22640 1 -- start start 2023-08-09T13:47:26.868 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:26.867+0000 7f960eb22640 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7f960815c720 0x7f96080cd850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:47:26.869 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:26.867+0000 7f960eb22640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f96080cf370 0x7f96080b5de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:47:26.869 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:26.867+0000 7f960eb22640 1 -- --> v1:172.21.15.43:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f96080b5660 con 0x7f96080d0900 2023-08-09T13:47:26.869 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:26.867+0000 7f960eb22640 1 -- --> v2:172.21.15.43:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f960815cb00 con 0x7f96080cf370 2023-08-09T13:47:26.869 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:26.867+0000 7f960eb22640 1 -- --> v2:172.21.15.203:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f96080d1960 con 0x7f960815c720 2023-08-09T13:47:26.869 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:26.867+0000 7f960d31f640 1 --1- >> v1:172.21.15.43:6789/0 conn(0x7f96080d0900 0x7f96080cec60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.43:6789/0 says I am v1:172.21.15.43:49540/0 (socket says 172.21.15.43:49540) 2023-08-09T13:47:26.869 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:26.867+0000 7f960d31f640 1 -- 172.21.15.43:0/1865433985 learned_addr learned my addr 172.21.15.43:0/1865433985 (peer_addr_for_me v1:172.21.15.43:0/0) 2023-08-09T13:47:26.870 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:26.867+0000 7f960e321640 1 --2- 172.21.15.43:0/1865433985 >> v2:172.21.15.43:3300/0 conn(0x7f96080cf370 0x7f96080b5de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:47:26.870 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:26.867+0000 7f960db20640 1 --2- 172.21.15.43:0/1865433985 >> v2:172.21.15.203:3300/0 conn(0x7f960815c720 0x7f96080cd850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:47:26.871 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:26.868+0000 7f960e321640 1 -- 172.21.15.43:0/1865433985 >> v2:172.21.15.203:3300/0 conn(0x7f960815c720 msgr2=0x7f96080cd850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-09T13:47:26.871 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:26.868+0000 7f960e321640 1 --2- 172.21.15.43:0/1865433985 >> v2:172.21.15.203:3300/0 conn(0x7f960815c720 0x7f96080cd850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:47:26.871 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:26.868+0000 7f960e321640 1 -- 172.21.15.43:0/1865433985 >> v1:172.21.15.43:6789/0 conn(0x7f96080d0900 legacy=0x7f96080cec60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:47:26.871 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:26.868+0000 7f960e321640 1 -- 172.21.15.43:0/1865433985 --> v2:172.21.15.43:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f96080b5900 con 0x7f96080cf370 2023-08-09T13:47:26.872 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:26.868+0000 7f960cb1e640 1 -- 172.21.15.43:0/1865433985 <== mon.0 v1:172.21.15.43:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1542152214 0 0) 0x7f96080b5660 con 0x7f96080d0900 2023-08-09T13:47:26.872 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:26.869+0000 7f960e321640 1 --2- 172.21.15.43:0/1865433985 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f96080cf370 0x7f96080b5de0 secure :-1 s=READY pgs=677 cs=0 l=1 rev1=1 crypto rx=0x7f96080b4de0 tx=0x7f960002b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1c36a5989e8987d2 server_cookie=0 in_seq=0 out_seq=0 2023-08-09T13:47:26.872 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:26.869+0000 7f960cb1e640 1 -- 172.21.15.43:0/1865433985 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9600031830 con 0x7f96080cf370 2023-08-09T13:47:26.872 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:26.869+0000 7f960cb1e640 1 -- 172.21.15.43:0/1865433985 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9600031990 con 0x7f96080cf370 2023-08-09T13:47:26.872 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:26.869+0000 7f960cb1e640 1 -- 172.21.15.43:0/1865433985 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9600031d20 con 0x7f96080cf370 2023-08-09T13:47:26.874 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:26.870+0000 7f960eb22640 1 -- 172.21.15.43:0/1865433985 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f96080cf370 msgr2=0x7f96080b5de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:47:26.875 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:26.870+0000 7f960eb22640 1 --2- 172.21.15.43:0/1865433985 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f96080cf370 0x7f96080b5de0 secure :-1 s=READY pgs=677 cs=0 l=1 rev1=1 crypto rx=0x7f96080b4de0 tx=0x7f960002b8d0 comp rx=0 tx=0).stop 2023-08-09T13:47:26.875 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:26.870+0000 7f960eb22640 1 -- 172.21.15.43:0/1865433985 shutdown_connections 2023-08-09T13:47:26.875 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:26.870+0000 7f960eb22640 1 --2- 172.21.15.43:0/1865433985 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f96080cf370 0x7f96080b5de0 unknown :-1 s=CLOSED pgs=677 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:47:26.875 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:26.870+0000 7f960eb22640 1 --2- 172.21.15.43:0/1865433985 >> v2:172.21.15.203:3300/0 conn(0x7f960815c720 0x7f96080cd850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:47:26.875 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:26.870+0000 7f960eb22640 1 -- 172.21.15.43:0/1865433985 >> 172.21.15.43:0/1865433985 conn(0x7f960806a2a0 msgr2=0x7f960806a680 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-09T13:47:26.875 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:26.870+0000 7f960eb22640 1 -- 172.21.15.43:0/1865433985 shutdown_connections 2023-08-09T13:47:26.876 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:26.870+0000 7f960eb22640 1 -- 172.21.15.43:0/1865433985 wait complete. 2023-08-09T13:47:26.876 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:26.872+0000 7f960eb22640 1 Processor -- start 2023-08-09T13:47:26.876 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:26.872+0000 7f960eb22640 1 -- start start 2023-08-09T13:47:26.876 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:26.873+0000 7f960eb22640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f96080cf370 0x7f960814b6f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:47:26.876 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:26.873+0000 7f960eb22640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f96080d0900 0x7f960806a680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:47:26.876 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:26.873+0000 7f960eb22640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f960815c720 0x7f960806abc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:47:26.876 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:26.873+0000 7f960eb22640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f96081f5e00 con 0x7f96080d0900 2023-08-09T13:47:26.877 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:26.873+0000 7f960eb22640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f960814c1f0 con 0x7f96080cf370 2023-08-09T13:47:26.877 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:26.873+0000 7f960eb22640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f960814c4d0 con 0x7f960815c720 2023-08-09T13:47:26.877 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:26.873+0000 7f960d31f640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f96080d0900 0x7f960806a680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:47:26.877 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:26.873+0000 7f960d31f640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f96080d0900 0x7f960806a680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:53048/0 (socket says 172.21.15.43:53048) 2023-08-09T13:47:26.877 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:26.873+0000 7f960d31f640 1 -- 172.21.15.43:0/1216808243 learned_addr learned my addr 172.21.15.43:0/1216808243 (peer_addr_for_me v2:172.21.15.43:0/0) 2023-08-09T13:47:26.878 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:26.874+0000 7f960db20640 1 --2- 172.21.15.43:0/1216808243 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f96080cf370 0x7f960814b6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:47:26.878 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:26.874+0000 7f960e321640 1 --2- 172.21.15.43:0/1216808243 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f960815c720 0x7f960806abc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:47:26.879 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:26.874+0000 7f960d31f640 1 -- 172.21.15.43:0/1216808243 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f960815c720 msgr2=0x7f960806abc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:47:26.879 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:26.874+0000 7f960d31f640 1 --2- 172.21.15.43:0/1216808243 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f960815c720 0x7f960806abc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:47:26.879 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:26.874+0000 7f960d31f640 1 -- 172.21.15.43:0/1216808243 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f96080cf370 msgr2=0x7f960814b6f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:47:26.879 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:26.874+0000 7f960d31f640 1 --2- 172.21.15.43:0/1216808243 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f96080cf370 0x7f960814b6f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:47:26.879 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:26.874+0000 7f960d31f640 1 -- 172.21.15.43:0/1216808243 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9600009400 con 0x7f96080d0900 2023-08-09T13:47:26.879 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:26.874+0000 7f960e321640 1 --2- 172.21.15.43:0/1216808243 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f960815c720 0x7f960806abc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-08-09T13:47:26.879 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:26.874+0000 7f960d31f640 1 --2- 172.21.15.43:0/1216808243 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f96080d0900 0x7f960806a680 secure :-1 s=READY pgs=678 cs=0 l=1 rev1=1 crypto rx=0x7f95f400c4d0 tx=0x7f95f400c990 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-09T13:47:26.880 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:26.874+0000 7f96067fc640 1 -- 172.21.15.43:0/1216808243 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f95f40074e0 con 0x7f96080d0900 2023-08-09T13:47:26.880 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:26.874+0000 7f96067fc640 1 -- 172.21.15.43:0/1216808243 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f95f4007660 con 0x7f96080d0900 2023-08-09T13:47:26.880 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:26.874+0000 7f96067fc640 1 -- 172.21.15.43:0/1216808243 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f95f4004a20 con 0x7f96080d0900 2023-08-09T13:47:26.880 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:26.875+0000 7f960eb22640 1 -- 172.21.15.43:0/1216808243 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f960814c7b0 con 0x7f96080d0900 2023-08-09T13:47:26.880 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:26.875+0000 7f960eb22640 1 -- 172.21.15.43:0/1216808243 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f960814ceb0 con 0x7f96080d0900 2023-08-09T13:47:26.880 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:26.876+0000 7f960eb22640 1 -- 172.21.15.43:0/1216808243 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f96080739f0 con 0x7f96080d0900 2023-08-09T13:47:26.880 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:26.877+0000 7f96067fc640 1 -- 172.21.15.43:0/1216808243 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 13) v1 ==== 88529+0+0 (secure 0 0 0) 0x7f95f4004b80 con 0x7f96080d0900 2023-08-09T13:47:26.881 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:26.878+0000 7f96067fc640 1 --2- 172.21.15.43:0/1216808243 >> [v2:172.21.15.203:6832/998005540,v1:172.21.15.203:6833/998005540] conn(0x7f95ec06a250 0x7f95ec06c710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:47:26.881 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:26.878+0000 7f96067fc640 1 -- 172.21.15.43:0/1216808243 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(42..42 src has 1..42) v4 ==== 6698+0+0 (secure 0 0 0) 0x7f95f4087d40 con 0x7f96080d0900 2023-08-09T13:47:26.881 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:26.878+0000 7f960db20640 1 --2- 172.21.15.43:0/1216808243 >> [v2:172.21.15.203:6832/998005540,v1:172.21.15.203:6833/998005540] conn(0x7f95ec06a250 0x7f95ec06c710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:47:26.881 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:26.880+0000 7f960db20640 1 --2- 172.21.15.43:0/1216808243 >> [v2:172.21.15.203:6832/998005540,v1:172.21.15.203:6833/998005540] conn(0x7f95ec06a250 0x7f95ec06c710 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7f95fc0070b0 tx=0x7f95fc007040 comp rx=0 tx=0).ready entity=mgr.5003 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-09T13:47:26.883 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:26.882+0000 7f96067fc640 1 -- 172.21.15.43:0/1216808243 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178066 (secure 0 0 0) 0x7f95f4052f20 con 0x7f96080d0900 2023-08-09T13:47:27.105 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:27.103+0000 7f960eb22640 1 -- 172.21.15.43:0/1216808243 --> [v2:172.21.15.203:6832/998005540,v1:172.21.15.203:6833/998005540] -- mgr_command(tid 0: {"prefix": "fs snap-schedule deactivate", "fs": "cephfs", "path": "/snap_test_dir1/dir7", "repeat": "1M", "target": ["mon-mgr", ""]}) v1 -- 0x7f96080d13a0 con 0x7f95ec06a250 2023-08-09T13:47:27.282 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:27.281+0000 7f96067fc640 1 -- 172.21.15.43:0/1216808243 <== mgr.5003 v2:172.21.15.203:6832/998005540 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+51 (secure 0 0 0) 0x7f95fc007bc0 con 0x7f95ec06a250 2023-08-09T13:47:27.283 INFO:teuthology.orchestra.run.smithi043.stdout:Schedule deactivated for path /snap_test_dir1/dir7 2023-08-09T13:47:27.286 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:27.284+0000 7f960eb22640 1 -- 172.21.15.43:0/1216808243 >> [v2:172.21.15.203:6832/998005540,v1:172.21.15.203:6833/998005540] conn(0x7f95ec06a250 msgr2=0x7f95ec06c710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:47:27.286 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:27.284+0000 7f960eb22640 1 --2- 172.21.15.43:0/1216808243 >> [v2:172.21.15.203:6832/998005540,v1:172.21.15.203:6833/998005540] conn(0x7f95ec06a250 0x7f95ec06c710 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7f95fc0070b0 tx=0x7f95fc007040 comp rx=0 tx=0).stop 2023-08-09T13:47:27.286 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:27.284+0000 7f960eb22640 1 -- 172.21.15.43:0/1216808243 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f96080d0900 msgr2=0x7f960806a680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:47:27.286 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:27.284+0000 7f960eb22640 1 --2- 172.21.15.43:0/1216808243 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f96080d0900 0x7f960806a680 secure :-1 s=READY pgs=678 cs=0 l=1 rev1=1 crypto rx=0x7f95f400c4d0 tx=0x7f95f400c990 comp rx=0 tx=0).stop 2023-08-09T13:47:27.287 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:27.285+0000 7f960eb22640 1 -- 172.21.15.43:0/1216808243 shutdown_connections 2023-08-09T13:47:27.287 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:27.285+0000 7f960eb22640 1 --2- 172.21.15.43:0/1216808243 >> [v2:172.21.15.203:6832/998005540,v1:172.21.15.203:6833/998005540] conn(0x7f95ec06a250 0x7f95ec06c710 unknown :-1 s=CLOSED pgs=50 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:47:27.287 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:27.285+0000 7f960eb22640 1 --2- 172.21.15.43:0/1216808243 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f960815c720 0x7f960806abc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:47:27.287 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:27.285+0000 7f960eb22640 1 --2- 172.21.15.43:0/1216808243 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f96080d0900 0x7f960806a680 unknown :-1 s=CLOSED pgs=678 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:47:27.287 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:27.285+0000 7f960eb22640 1 --2- 172.21.15.43:0/1216808243 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f96080cf370 0x7f960814b6f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:47:27.287 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:27.285+0000 7f960eb22640 1 -- 172.21.15.43:0/1216808243 >> 172.21.15.43:0/1216808243 conn(0x7f960806a2a0 msgr2=0x7f960815cc40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-09T13:47:27.288 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:27.285+0000 7f960eb22640 1 -- 172.21.15.43:0/1216808243 shutdown_connections 2023-08-09T13:47:27.288 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:27.285+0000 7f960eb22640 1 -- 172.21.15.43:0/1216808243 wait complete. 2023-08-09T13:47:27.308 DEBUG:tasks.cephfs.test_snap_schedules:res=Schedule deactivated for path /snap_test_dir1/dir7 2023-08-09T13:47:27.309 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs snap-schedule deactivate --fs cephfs /snap_test_dir1/dir8 1M 2023-08-09T13:47:27.547 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:27.546+0000 7f8a60d9b640 1 Processor -- start 2023-08-09T13:47:27.548 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:27.546+0000 7f8a60d9b640 1 -- start start 2023-08-09T13:47:27.552 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:27.548+0000 7f8a60d9b640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f8a5c15c720 0x7f8a5c0cd850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:47:27.553 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:27.548+0000 7f8a60d9b640 1 -- --> v1:172.21.15.43:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8a5c0b5660 con 0x7f8a5c0cf370 2023-08-09T13:47:27.553 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:27.548+0000 7f8a60d9b640 1 -- --> v1:172.21.15.203:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8a5c0b5900 con 0x7f8a5c0d0900 2023-08-09T13:47:27.553 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:27.548+0000 7f8a60d9b640 1 -- --> v2:172.21.15.43:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8a5c15cb00 con 0x7f8a5c15c720 2023-08-09T13:47:27.553 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:27.548+0000 7f8a5b7fe640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f8a5c15c720 0x7f8a5c0cd850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:47:27.554 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:27.548+0000 7f8a5b7fe640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f8a5c15c720 0x7f8a5c0cd850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:53050/0 (socket says 172.21.15.43:53050) 2023-08-09T13:47:27.554 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:27.548+0000 7f8a5b7fe640 1 -- 172.21.15.43:0/3105817776 learned_addr learned my addr 172.21.15.43:0/3105817776 (peer_addr_for_me v2:172.21.15.43:0/0) 2023-08-09T13:47:27.554 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:27.549+0000 7f8a5b7fe640 1 -- 172.21.15.43:0/3105817776 >> v1:172.21.15.203:6789/0 conn(0x7f8a5c0d0900 legacy=0x7f8a5c0cec60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-09T13:47:27.554 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:27.549+0000 7f8a5b7fe640 1 -- 172.21.15.43:0/3105817776 >> v1:172.21.15.43:6789/0 conn(0x7f8a5c0cf370 legacy=0x7f8a5c0b5de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:47:27.554 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:27.549+0000 7f8a5b7fe640 1 -- 172.21.15.43:0/3105817776 --> v2:172.21.15.43:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8a5c0d10c0 con 0x7f8a5c15c720 2023-08-09T13:47:27.555 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:27.550+0000 7f8a5b7fe640 1 --2- 172.21.15.43:0/3105817776 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f8a5c15c720 0x7f8a5c0cd850 secure :-1 s=READY pgs=680 cs=0 l=1 rev1=1 crypto rx=0x7f8a4c009e50 tx=0x7f8a4c02bf70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e1b0734e880639d4 server_cookie=0 in_seq=0 out_seq=0 2023-08-09T13:47:27.555 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:27.550+0000 7f8a5affd640 1 -- 172.21.15.43:0/3105817776 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8a4c02cb10 con 0x7f8a5c15c720 2023-08-09T13:47:27.555 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:27.550+0000 7f8a5affd640 1 -- 172.21.15.43:0/3105817776 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8a4c02cc70 con 0x7f8a5c15c720 2023-08-09T13:47:27.555 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:27.550+0000 7f8a5affd640 1 -- 172.21.15.43:0/3105817776 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8a4c0343f0 con 0x7f8a5c15c720 2023-08-09T13:47:27.555 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:27.551+0000 7f8a60d9b640 1 -- 172.21.15.43:0/3105817776 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f8a5c15c720 msgr2=0x7f8a5c0cd850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:47:27.556 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:27.551+0000 7f8a60d9b640 1 --2- 172.21.15.43:0/3105817776 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f8a5c15c720 0x7f8a5c0cd850 secure :-1 s=READY pgs=680 cs=0 l=1 rev1=1 crypto rx=0x7f8a4c009e50 tx=0x7f8a4c02bf70 comp rx=0 tx=0).stop 2023-08-09T13:47:27.556 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:27.551+0000 7f8a60d9b640 1 -- 172.21.15.43:0/3105817776 shutdown_connections 2023-08-09T13:47:27.556 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:27.551+0000 7f8a60d9b640 1 --2- 172.21.15.43:0/3105817776 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f8a5c15c720 0x7f8a5c0cd850 unknown :-1 s=CLOSED pgs=680 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:47:27.556 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:27.551+0000 7f8a60d9b640 1 -- 172.21.15.43:0/3105817776 >> 172.21.15.43:0/3105817776 conn(0x7f8a5c06a2a0 msgr2=0x7f8a5c06a680 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-09T13:47:27.556 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:27.551+0000 7f8a60d9b640 1 -- 172.21.15.43:0/3105817776 shutdown_connections 2023-08-09T13:47:27.556 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:27.551+0000 7f8a60d9b640 1 -- 172.21.15.43:0/3105817776 wait complete. 2023-08-09T13:47:27.558 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:27.553+0000 7f8a60d9b640 1 Processor -- start 2023-08-09T13:47:27.558 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:27.553+0000 7f8a60d9b640 1 -- start start 2023-08-09T13:47:27.559 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:27.555+0000 7f8a60d9b640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f8a5c0cf370 0x7f8a5c0ca2e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:47:27.559 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:27.555+0000 7f8a60d9b640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f8a5c0d0900 0x7f8a5c0c73c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:47:27.559 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:27.555+0000 7f8a60d9b640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f8a5c15c720 0x7f8a5c0b7470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:47:27.559 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:27.555+0000 7f8a60d9b640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8a5c0b3390 con 0x7f8a5c15c720 2023-08-09T13:47:27.559 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:27.555+0000 7f8a60d9b640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8a5c14eaf0 con 0x7f8a5c0cf370 2023-08-09T13:47:27.559 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:27.555+0000 7f8a60d9b640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8a5c0ca9c0 con 0x7f8a5c0d0900 2023-08-09T13:47:27.559 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:27.555+0000 7f8a5bfff640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f8a5c15c720 0x7f8a5c0b7470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:47:27.560 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:27.555+0000 7f8a5bfff640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f8a5c15c720 0x7f8a5c0b7470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:53058/0 (socket says 172.21.15.43:53058) 2023-08-09T13:47:27.560 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:27.555+0000 7f8a5bfff640 1 -- 172.21.15.43:0/2247826050 learned_addr learned my addr 172.21.15.43:0/2247826050 (peer_addr_for_me v2:172.21.15.43:0/0) 2023-08-09T13:47:27.560 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:27.555+0000 7f8a5b7fe640 1 --2- 172.21.15.43:0/2247826050 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f8a5c0cf370 0x7f8a5c0ca2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:47:27.560 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:27.556+0000 7f8a52ffd640 1 --2- 172.21.15.43:0/2247826050 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f8a5c0d0900 0x7f8a5c0c73c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:47:27.560 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:27.556+0000 7f8a5bfff640 1 -- 172.21.15.43:0/2247826050 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f8a5c0d0900 msgr2=0x7f8a5c0c73c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:47:27.560 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:27.556+0000 7f8a5bfff640 1 --2- 172.21.15.43:0/2247826050 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f8a5c0d0900 0x7f8a5c0c73c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:47:27.560 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:27.556+0000 7f8a5bfff640 1 -- 172.21.15.43:0/2247826050 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f8a5c0cf370 msgr2=0x7f8a5c0ca2e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:47:27.561 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:27.556+0000 7f8a5bfff640 1 --2- 172.21.15.43:0/2247826050 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f8a5c0cf370 0x7f8a5c0ca2e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:47:27.561 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:27.556+0000 7f8a5bfff640 1 -- 172.21.15.43:0/2247826050 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8a4c009b30 con 0x7f8a5c15c720 2023-08-09T13:47:27.561 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:27.556+0000 7f8a5b7fe640 1 --2- 172.21.15.43:0/2247826050 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f8a5c0cf370 0x7f8a5c0ca2e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-08-09T13:47:27.561 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:27.556+0000 7f8a5bfff640 1 --2- 172.21.15.43:0/2247826050 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f8a5c15c720 0x7f8a5c0b7470 secure :-1 s=READY pgs=681 cs=0 l=1 rev1=1 crypto rx=0x7f8a5400c400 tx=0x7f8a5400c8c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-09T13:47:27.562 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:27.556+0000 7f8a58ff9640 1 -- 172.21.15.43:0/2247826050 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8a540052d0 con 0x7f8a5c15c720 2023-08-09T13:47:27.562 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:27.556+0000 7f8a58ff9640 1 -- 172.21.15.43:0/2247826050 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8a54005430 con 0x7f8a5c15c720 2023-08-09T13:47:27.562 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:27.556+0000 7f8a58ff9640 1 -- 172.21.15.43:0/2247826050 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8a54003810 con 0x7f8a5c15c720 2023-08-09T13:47:27.562 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:27.557+0000 7f8a60d9b640 1 -- 172.21.15.43:0/2247826050 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8a5c0caca0 con 0x7f8a5c15c720 2023-08-09T13:47:27.562 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:27.557+0000 7f8a60d9b640 1 -- 172.21.15.43:0/2247826050 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8a5c0cb260 con 0x7f8a5c15c720 2023-08-09T13:47:27.563 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:27.558+0000 7f8a60d9b640 1 -- 172.21.15.43:0/2247826050 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8a5c0b30f0 con 0x7f8a5c15c720 2023-08-09T13:47:27.563 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:27.559+0000 7f8a58ff9640 1 -- 172.21.15.43:0/2247826050 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 13) v1 ==== 88529+0+0 (secure 0 0 0) 0x7f8a54003970 con 0x7f8a5c15c720 2023-08-09T13:47:27.564 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:27.560+0000 7f8a58ff9640 1 --2- 172.21.15.43:0/2247826050 >> [v2:172.21.15.203:6832/998005540,v1:172.21.15.203:6833/998005540] conn(0x7f8a3c06a250 0x7f8a3c06c710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:47:27.564 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:27.560+0000 7f8a58ff9640 1 -- 172.21.15.43:0/2247826050 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(42..42 src has 1..42) v4 ==== 6698+0+0 (secure 0 0 0) 0x7f8a54088150 con 0x7f8a5c15c720 2023-08-09T13:47:27.564 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:27.561+0000 7f8a5b7fe640 1 --2- 172.21.15.43:0/2247826050 >> [v2:172.21.15.203:6832/998005540,v1:172.21.15.203:6833/998005540] conn(0x7f8a3c06a250 0x7f8a3c06c710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:47:27.564 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:27.562+0000 7f8a5b7fe640 1 --2- 172.21.15.43:0/2247826050 >> [v2:172.21.15.203:6832/998005540,v1:172.21.15.203:6833/998005540] conn(0x7f8a3c06a250 0x7f8a3c06c710 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7f8a4c035000 tx=0x7f8a4c030000 comp rx=0 tx=0).ready entity=mgr.5003 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-09T13:47:27.569 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:27.568+0000 7f8a58ff9640 1 -- 172.21.15.43:0/2247826050 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178066 (secure 0 0 0) 0x7f8a5c0b30f0 con 0x7f8a5c15c720 2023-08-09T13:47:27.787 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:27.786+0000 7f8a60d9b640 1 -- 172.21.15.43:0/2247826050 --> [v2:172.21.15.203:6832/998005540,v1:172.21.15.203:6833/998005540] -- mgr_command(tid 0: {"prefix": "fs snap-schedule deactivate", "fs": "cephfs", "path": "/snap_test_dir1/dir8", "repeat": "1M", "target": ["mon-mgr", ""]}) v1 -- 0x7f8a5c0d13a0 con 0x7f8a3c06a250 2023-08-09T13:47:27.968 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:27.966+0000 7f8a58ff9640 1 -- 172.21.15.43:0/2247826050 <== mgr.5003 v2:172.21.15.203:6832/998005540 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+51 (secure 0 0 0) 0x7f8a4c0440d0 con 0x7f8a3c06a250 2023-08-09T13:47:27.968 INFO:teuthology.orchestra.run.smithi043.stdout:Schedule deactivated for path /snap_test_dir1/dir8 2023-08-09T13:47:27.971 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:27.969+0000 7f8a60d9b640 1 -- 172.21.15.43:0/2247826050 >> [v2:172.21.15.203:6832/998005540,v1:172.21.15.203:6833/998005540] conn(0x7f8a3c06a250 msgr2=0x7f8a3c06c710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:47:27.971 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:27.969+0000 7f8a60d9b640 1 --2- 172.21.15.43:0/2247826050 >> [v2:172.21.15.203:6832/998005540,v1:172.21.15.203:6833/998005540] conn(0x7f8a3c06a250 0x7f8a3c06c710 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7f8a4c035000 tx=0x7f8a4c030000 comp rx=0 tx=0).stop 2023-08-09T13:47:27.971 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:27.969+0000 7f8a60d9b640 1 -- 172.21.15.43:0/2247826050 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f8a5c15c720 msgr2=0x7f8a5c0b7470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:47:27.971 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:27.969+0000 7f8a60d9b640 1 --2- 172.21.15.43:0/2247826050 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f8a5c15c720 0x7f8a5c0b7470 secure :-1 s=READY pgs=681 cs=0 l=1 rev1=1 crypto rx=0x7f8a5400c400 tx=0x7f8a5400c8c0 comp rx=0 tx=0).stop 2023-08-09T13:47:27.972 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:27.970+0000 7f8a60d9b640 1 -- 172.21.15.43:0/2247826050 shutdown_connections 2023-08-09T13:47:27.972 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:27.970+0000 7f8a60d9b640 1 --2- 172.21.15.43:0/2247826050 >> [v2:172.21.15.203:6832/998005540,v1:172.21.15.203:6833/998005540] conn(0x7f8a3c06a250 0x7f8a3c06c710 unknown :-1 s=CLOSED pgs=51 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:47:27.972 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:27.970+0000 7f8a60d9b640 1 --2- 172.21.15.43:0/2247826050 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f8a5c15c720 0x7f8a5c0b7470 unknown :-1 s=CLOSED pgs=681 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:47:27.972 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:27.970+0000 7f8a60d9b640 1 --2- 172.21.15.43:0/2247826050 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f8a5c0d0900 0x7f8a5c0c73c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:47:27.973 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:27.970+0000 7f8a60d9b640 1 --2- 172.21.15.43:0/2247826050 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f8a5c0cf370 0x7f8a5c0ca2e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:47:27.973 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:27.970+0000 7f8a60d9b640 1 -- 172.21.15.43:0/2247826050 >> 172.21.15.43:0/2247826050 conn(0x7f8a5c06a2a0 msgr2=0x7f8a5c0cd2b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-09T13:47:27.973 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:27.970+0000 7f8a60d9b640 1 -- 172.21.15.43:0/2247826050 shutdown_connections 2023-08-09T13:47:27.973 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:27.970+0000 7f8a60d9b640 1 -- 172.21.15.43:0/2247826050 wait complete. 2023-08-09T13:47:27.993 DEBUG:tasks.cephfs.test_snap_schedules:res=Schedule deactivated for path /snap_test_dir1/dir8 2023-08-09T13:47:27.993 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs snap-schedule deactivate --fs cephfs /snap_test_dir1/dir9 1M 2023-08-09T13:47:28.228 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:28.227+0000 7f1204004640 1 Processor -- start 2023-08-09T13:47:28.229 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:28.227+0000 7f1204004640 1 -- start start 2023-08-09T13:47:28.230 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:28.229+0000 7f1204004640 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7f11fc069a30 0x7f11fc159760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:47:28.230 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:28.229+0000 7f1204004640 1 -- --> v1:172.21.15.43:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f11fc0ca410 con 0x7f11fc15da40 2023-08-09T13:47:28.230 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:28.229+0000 7f1204004640 1 -- --> v1:172.21.15.203:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f11fc15a050 con 0x7f11fc15d5b0 2023-08-09T13:47:28.231 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:28.229+0000 7f1204004640 1 -- --> v2:172.21.15.203:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f11fc15ce80 con 0x7f11fc069a30 2023-08-09T13:47:28.232 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:28.229+0000 7f12025a1640 1 --1- >> v1:172.21.15.43:6789/0 conn(0x7f11fc15da40 0x7f11fc0ce4c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.43:6789/0 says I am v1:172.21.15.43:49556/0 (socket says 172.21.15.43:49556) 2023-08-09T13:47:28.232 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:28.229+0000 7f12025a1640 1 -- 172.21.15.43:0/4204781841 learned_addr learned my addr 172.21.15.43:0/4204781841 (peer_addr_for_me v1:172.21.15.43:0/0) 2023-08-09T13:47:28.232 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:28.230+0000 7f1201da0640 1 --2- 172.21.15.43:0/4204781841 >> v2:172.21.15.203:3300/0 conn(0x7f11fc069a30 0x7f11fc159760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:47:28.232 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:28.230+0000 7f1200d9e640 1 -- 172.21.15.43:0/4204781841 <== mon.0 v1:172.21.15.43:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 816581864 0 0) 0x7f11fc0ca410 con 0x7f11fc15da40 2023-08-09T13:47:28.232 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:28.230+0000 7f1200d9e640 1 -- 172.21.15.43:0/4204781841 --> v1:172.21.15.43:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f11e8005730 con 0x7f11fc15da40 2023-08-09T13:47:28.233 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:28.230+0000 7f1200d9e640 1 -- 172.21.15.43:0/4204781841 <== mon.0 v1:172.21.15.43:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2709368927 0 0) 0x7f11e8005730 con 0x7f11fc15da40 2023-08-09T13:47:28.233 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:28.230+0000 7f1200d9e640 1 -- 172.21.15.43:0/4204781841 >> v2:172.21.15.203:3300/0 conn(0x7f11fc069a30 msgr2=0x7f11fc159760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-09T13:47:28.233 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:28.230+0000 7f1200d9e640 1 --2- 172.21.15.43:0/4204781841 >> v2:172.21.15.203:3300/0 conn(0x7f11fc069a30 0x7f11fc159760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:47:28.233 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:28.230+0000 7f1200d9e640 1 -- 172.21.15.43:0/4204781841 >> v1:172.21.15.203:6789/0 conn(0x7f11fc15d5b0 legacy=0x7f11fc069e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-09T13:47:28.233 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:28.230+0000 7f1200d9e640 1 -- 172.21.15.43:0/4204781841 --> v1:172.21.15.43:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f11fc15a330 con 0x7f11fc15da40 2023-08-09T13:47:28.233 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:28.230+0000 7f1200d9e640 1 -- 172.21.15.43:0/4204781841 <== mon.0 v1:172.21.15.43:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1285150740 0 0) 0x7f11f40019f0 con 0x7f11fc15da40 2023-08-09T13:47:28.233 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:28.230+0000 7f1200d9e640 1 -- 172.21.15.43:0/4204781841 >> v1:172.21.15.43:6789/0 conn(0x7f11fc15da40 legacy=0x7f11fc0ce4c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:47:28.233 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:28.230+0000 7f1200d9e640 1 --2- 172.21.15.43:0/4204781841 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f11e8005bf0 0x7f11e8006010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:47:28.234 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:28.230+0000 7f1200d9e640 1 --2- 172.21.15.43:0/4204781841 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f11e80069d0 0x7f11e8006e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:47:28.234 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:28.231+0000 7f1200d9e640 1 --2- 172.21.15.43:0/4204781841 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f11e8007400 0x7f11e800db90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:47:28.234 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:28.231+0000 7f1200d9e640 1 -- 172.21.15.43:0/4204781841 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f11fc15ce80 con 0x7f11e8007400 2023-08-09T13:47:28.234 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:28.231+0000 7f1200d9e640 1 -- 172.21.15.43:0/4204781841 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f11e800e360 con 0x7f11e8005bf0 2023-08-09T13:47:28.234 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:28.231+0000 7f1200d9e640 1 -- 172.21.15.43:0/4204781841 --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f11e800e520 con 0x7f11e80069d0 2023-08-09T13:47:28.235 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:28.231+0000 7f12025a1640 1 --2- 172.21.15.43:0/4204781841 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f11e8007400 0x7f11e800db90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:47:28.236 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:28.231+0000 7f1201da0640 1 --2- 172.21.15.43:0/4204781841 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f11e8005bf0 0x7f11e8006010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:47:28.236 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:28.231+0000 7f120159f640 1 --2- 172.21.15.43:0/4204781841 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f11e80069d0 0x7f11e8006e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:47:28.236 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:28.231+0000 7f12025a1640 1 -- 172.21.15.43:0/4204781841 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f11e80069d0 msgr2=0x7f11e8006e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-09T13:47:28.236 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:28.231+0000 7f12025a1640 1 --2- 172.21.15.43:0/4204781841 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f11e80069d0 0x7f11e8006e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:47:28.236 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:28.231+0000 7f12025a1640 1 -- 172.21.15.43:0/4204781841 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f11e8005bf0 msgr2=0x7f11e8006010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-09T13:47:28.237 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:28.231+0000 7f12025a1640 1 --2- 172.21.15.43:0/4204781841 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f11e8005bf0 0x7f11e8006010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:47:28.237 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:28.231+0000 7f12025a1640 1 -- 172.21.15.43:0/4204781841 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f11e8005730 con 0x7f11e8007400 2023-08-09T13:47:28.237 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:28.231+0000 7f1201da0640 1 --2- 172.21.15.43:0/4204781841 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f11e8005bf0 0x7f11e8006010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-08-09T13:47:28.238 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:28.231+0000 7f120159f640 1 --2- 172.21.15.43:0/4204781841 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f11e80069d0 0x7f11e8006e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-08-09T13:47:28.238 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:28.232+0000 7f12025a1640 1 -- 172.21.15.43:0/4204781841 reap_dead start 2023-08-09T13:47:28.238 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:28.233+0000 7f12025a1640 1 --2- 172.21.15.43:0/4204781841 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f11e8007400 0x7f11e800db90 secure :-1 s=READY pgs=683 cs=0 l=1 rev1=1 crypto rx=0x7f11f400a800 tx=0x7f11e8007a00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=100bfd4746131f48 server_cookie=0 in_seq=0 out_seq=0 2023-08-09T13:47:28.238 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:28.233+0000 7f1200d9e640 1 -- 172.21.15.43:0/4204781841 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f11f40072d0 con 0x7f11e8007400 2023-08-09T13:47:28.238 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:28.233+0000 7f1200d9e640 1 -- 172.21.15.43:0/4204781841 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f11f402f070 con 0x7f11e8007400 2023-08-09T13:47:28.238 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:28.233+0000 7f1200d9e640 1 -- 172.21.15.43:0/4204781841 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f11f4007690 con 0x7f11e8007400 2023-08-09T13:47:28.238 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:28.233+0000 7f1204004640 1 -- 172.21.15.43:0/4204781841 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f11e8007400 msgr2=0x7f11e800db90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:47:28.239 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:28.233+0000 7f1204004640 1 --2- 172.21.15.43:0/4204781841 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f11e8007400 0x7f11e800db90 secure :-1 s=READY pgs=683 cs=0 l=1 rev1=1 crypto rx=0x7f11f400a800 tx=0x7f11e8007a00 comp rx=0 tx=0).stop 2023-08-09T13:47:28.239 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:28.234+0000 7f1204004640 1 -- 172.21.15.43:0/4204781841 shutdown_connections 2023-08-09T13:47:28.239 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:28.234+0000 7f1204004640 1 --2- 172.21.15.43:0/4204781841 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f11e8007400 0x7f11e800db90 unknown :-1 s=CLOSED pgs=683 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:47:28.239 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:28.234+0000 7f1204004640 1 -- 172.21.15.43:0/4204781841 >> 172.21.15.43:0/4204781841 conn(0x7f11fc06b990 msgr2=0x7f11fc06bda0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-09T13:47:28.239 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:28.234+0000 7f1204004640 1 -- 172.21.15.43:0/4204781841 shutdown_connections 2023-08-09T13:47:28.239 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:28.234+0000 7f1204004640 1 -- 172.21.15.43:0/4204781841 wait complete. 2023-08-09T13:47:28.241 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:28.235+0000 7f1204004640 1 Processor -- start 2023-08-09T13:47:28.241 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:28.236+0000 7f1204004640 1 -- start start 2023-08-09T13:47:28.241 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:28.237+0000 7f1204004640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f11e8007400 0x7f11fc1536e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:47:28.242 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:28.237+0000 7f1204004640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f11fc0c6840 0x7f11fc0c7cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:47:28.242 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:28.237+0000 7f1204004640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f11fc0c8200 0x7f11fc1f0f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:47:28.242 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:28.237+0000 7f1204004640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f11fc1f1ec0 con 0x7f11fc0c6840 2023-08-09T13:47:28.242 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:28.237+0000 7f1204004640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f11fc153c20 con 0x7f11fc0c8200 2023-08-09T13:47:28.242 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:28.237+0000 7f1204004640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f11fc0c57a0 con 0x7f11e8007400 2023-08-09T13:47:28.242 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:28.237+0000 7f120159f640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f11fc0c6840 0x7f11fc0c7cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:47:28.242 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:28.237+0000 7f120159f640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f11fc0c6840 0x7f11fc0c7cc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:53088/0 (socket says 172.21.15.43:53088) 2023-08-09T13:47:28.243 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:28.237+0000 7f120159f640 1 -- 172.21.15.43:0/1420272916 learned_addr learned my addr 172.21.15.43:0/1420272916 (peer_addr_for_me v2:172.21.15.43:0/0) 2023-08-09T13:47:28.243 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:28.237+0000 7f1201da0640 1 --2- 172.21.15.43:0/1420272916 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f11e8007400 0x7f11fc1536e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:47:28.243 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:28.237+0000 7f12025a1640 1 --2- 172.21.15.43:0/1420272916 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f11fc0c8200 0x7f11fc1f0f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:47:28.243 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:28.237+0000 7f120159f640 1 -- 172.21.15.43:0/1420272916 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f11e8007400 msgr2=0x7f11fc1536e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:47:28.243 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:28.237+0000 7f120159f640 1 --2- 172.21.15.43:0/1420272916 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f11e8007400 0x7f11fc1536e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:47:28.243 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:28.237+0000 7f120159f640 1 -- 172.21.15.43:0/1420272916 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f11fc0c8200 msgr2=0x7f11fc1f0f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:47:28.244 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:28.237+0000 7f120159f640 1 --2- 172.21.15.43:0/1420272916 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f11fc0c8200 0x7f11fc1f0f70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:47:28.244 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:28.237+0000 7f120159f640 1 -- 172.21.15.43:0/1420272916 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f11fc0c5a80 con 0x7f11fc0c6840 2023-08-09T13:47:28.244 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:28.238+0000 7f120159f640 1 --2- 172.21.15.43:0/1420272916 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f11fc0c6840 0x7f11fc0c7cc0 secure :-1 s=READY pgs=684 cs=0 l=1 rev1=1 crypto rx=0x7f11ec00d770 tx=0x7f11ec00dc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-09T13:47:28.244 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:28.238+0000 7f11f27fc640 1 -- 172.21.15.43:0/1420272916 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f11ec005270 con 0x7f11fc0c6840 2023-08-09T13:47:28.244 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:28.238+0000 7f11f27fc640 1 -- 172.21.15.43:0/1420272916 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f11ec0053d0 con 0x7f11fc0c6840 2023-08-09T13:47:28.245 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:28.238+0000 7f11f27fc640 1 -- 172.21.15.43:0/1420272916 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f11ec00b8f0 con 0x7f11fc0c6840 2023-08-09T13:47:28.245 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:28.238+0000 7f1204004640 1 -- 172.21.15.43:0/1420272916 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f11fc0c5d60 con 0x7f11fc0c6840 2023-08-09T13:47:28.245 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:28.238+0000 7f1204004640 1 -- 172.21.15.43:0/1420272916 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f11fc0c6320 con 0x7f11fc0c6840 2023-08-09T13:47:28.245 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:28.239+0000 7f1204004640 1 -- 172.21.15.43:0/1420272916 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f11fc0c64b0 con 0x7f11fc0c6840 2023-08-09T13:47:28.246 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:28.240+0000 7f11f27fc640 1 -- 172.21.15.43:0/1420272916 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 13) v1 ==== 88529+0+0 (secure 0 0 0) 0x7f11ec00ba50 con 0x7f11fc0c6840 2023-08-09T13:47:28.246 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:28.241+0000 7f11f27fc640 1 --2- 172.21.15.43:0/1420272916 >> [v2:172.21.15.203:6832/998005540,v1:172.21.15.203:6833/998005540] conn(0x7f11e006a250 0x7f11e006c710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:47:28.246 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:28.241+0000 7f11f27fc640 1 -- 172.21.15.43:0/1420272916 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(42..42 src has 1..42) v4 ==== 6698+0+0 (secure 0 0 0) 0x7f11ec088c70 con 0x7f11fc0c6840 2023-08-09T13:47:28.246 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:28.242+0000 7f1201da0640 1 --2- 172.21.15.43:0/1420272916 >> [v2:172.21.15.203:6832/998005540,v1:172.21.15.203:6833/998005540] conn(0x7f11e006a250 0x7f11e006c710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:47:28.246 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:28.243+0000 7f1201da0640 1 --2- 172.21.15.43:0/1420272916 >> [v2:172.21.15.203:6832/998005540,v1:172.21.15.203:6833/998005540] conn(0x7f11e006a250 0x7f11e006c710 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7f11f8009aa0 tx=0x7f11f8008040 comp rx=0 tx=0).ready entity=mgr.5003 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-09T13:47:28.247 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:28.245+0000 7f11f27fc640 1 -- 172.21.15.43:0/1420272916 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178066 (secure 0 0 0) 0x7f11ec053d50 con 0x7f11fc0c6840 2023-08-09T13:47:28.466 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:28.464+0000 7f1204004640 1 -- 172.21.15.43:0/1420272916 --> [v2:172.21.15.203:6832/998005540,v1:172.21.15.203:6833/998005540] -- mgr_command(tid 0: {"prefix": "fs snap-schedule deactivate", "fs": "cephfs", "path": "/snap_test_dir1/dir9", "repeat": "1M", "target": ["mon-mgr", ""]}) v1 -- 0x7f11fc15d090 con 0x7f11e006a250 2023-08-09T13:47:28.648 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:28.646+0000 7f11f27fc640 1 -- 172.21.15.43:0/1420272916 <== mgr.5003 v2:172.21.15.203:6832/998005540 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+51 (secure 0 0 0) 0x7f11f80129b0 con 0x7f11e006a250 2023-08-09T13:47:28.649 INFO:teuthology.orchestra.run.smithi043.stdout:Schedule deactivated for path /snap_test_dir1/dir9 2023-08-09T13:47:28.652 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:28.650+0000 7f1204004640 1 -- 172.21.15.43:0/1420272916 >> [v2:172.21.15.203:6832/998005540,v1:172.21.15.203:6833/998005540] conn(0x7f11e006a250 msgr2=0x7f11e006c710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:47:28.652 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:28.650+0000 7f1204004640 1 --2- 172.21.15.43:0/1420272916 >> [v2:172.21.15.203:6832/998005540,v1:172.21.15.203:6833/998005540] conn(0x7f11e006a250 0x7f11e006c710 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7f11f8009aa0 tx=0x7f11f8008040 comp rx=0 tx=0).stop 2023-08-09T13:47:28.652 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:28.650+0000 7f1204004640 1 -- 172.21.15.43:0/1420272916 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f11fc0c6840 msgr2=0x7f11fc0c7cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:47:28.652 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:28.650+0000 7f1204004640 1 --2- 172.21.15.43:0/1420272916 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f11fc0c6840 0x7f11fc0c7cc0 secure :-1 s=READY pgs=684 cs=0 l=1 rev1=1 crypto rx=0x7f11ec00d770 tx=0x7f11ec00dc30 comp rx=0 tx=0).stop 2023-08-09T13:47:28.653 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:28.650+0000 7f1204004640 1 -- 172.21.15.43:0/1420272916 shutdown_connections 2023-08-09T13:47:28.653 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:28.650+0000 7f1204004640 1 --2- 172.21.15.43:0/1420272916 >> [v2:172.21.15.203:6832/998005540,v1:172.21.15.203:6833/998005540] conn(0x7f11e006a250 0x7f11e006c710 unknown :-1 s=CLOSED pgs=52 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:47:28.653 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:28.650+0000 7f1204004640 1 --2- 172.21.15.43:0/1420272916 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f11fc0c8200 0x7f11fc1f0f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:47:28.653 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:28.650+0000 7f1204004640 1 --2- 172.21.15.43:0/1420272916 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f11fc0c6840 0x7f11fc0c7cc0 unknown :-1 s=CLOSED pgs=684 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:47:28.653 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:28.650+0000 7f1204004640 1 --2- 172.21.15.43:0/1420272916 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f11e8007400 0x7f11fc1536e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:47:28.653 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:28.650+0000 7f1204004640 1 -- 172.21.15.43:0/1420272916 >> 172.21.15.43:0/1420272916 conn(0x7f11fc06b990 msgr2=0x7f11fc0ce490 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-09T13:47:28.654 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:28.651+0000 7f1204004640 1 -- 172.21.15.43:0/1420272916 shutdown_connections 2023-08-09T13:47:28.654 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:28.651+0000 7f1204004640 1 -- 172.21.15.43:0/1420272916 wait complete. 2023-08-09T13:47:28.676 DEBUG:tasks.cephfs.test_snap_schedules:res=Schedule deactivated for path /snap_test_dir1/dir9 2023-08-09T13:47:28.676 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-09T13:47:28.677 DEBUG:teuthology.orchestra.run.smithi043:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/dir0/.snap) 2023-08-09T13:47:28.752 INFO:teuthology.orchestra.run.smithi043.stdout:scheduled-2023-08-09-13_37_00_UTC 2023-08-09T13:47:28.753 INFO:teuthology.orchestra.run.smithi043.stdout:scheduled-2023-08-09-13_38_01_UTC 2023-08-09T13:47:28.753 INFO:teuthology.orchestra.run.smithi043.stdout:scheduled-2023-08-09-13_39_01_UTC 2023-08-09T13:47:28.753 INFO:teuthology.orchestra.run.smithi043.stdout:scheduled-2023-08-09-13_40_00_UTC 2023-08-09T13:47:28.753 INFO:teuthology.orchestra.run.smithi043.stdout:scheduled-2023-08-09-13_41_00_UTC 2023-08-09T13:47:28.753 INFO:teuthology.orchestra.run.smithi043.stdout:scheduled-2023-08-09-13_42_00_UTC 2023-08-09T13:47:28.753 INFO:teuthology.orchestra.run.smithi043.stdout:scheduled-2023-08-09-13_43_00_UTC 2023-08-09T13:47:28.754 INFO:teuthology.orchestra.run.smithi043.stdout:scheduled-2023-08-09-13_44_01_UTC 2023-08-09T13:47:28.754 INFO:teuthology.orchestra.run.smithi043.stdout:scheduled-2023-08-09-13_45_01_UTC 2023-08-09T13:47:28.754 INFO:teuthology.orchestra.run.smithi043.stdout:scheduled-2023-08-09-13_46_01_UTC 2023-08-09T13:47:28.754 INFO:teuthology.orchestra.run.smithi043.stdout:scheduled-2023-08-09-13_47_01_UTC 2023-08-09T13:47:28.754 DEBUG:tasks.cephfs.test_snap_schedules:snapshots: ['scheduled-2023-08-09-13_37_00_UTC', 'scheduled-2023-08-09-13_38_01_UTC', 'scheduled-2023-08-09-13_39_01_UTC', 'scheduled-2023-08-09-13_40_00_UTC', 'scheduled-2023-08-09-13_41_00_UTC', 'scheduled-2023-08-09-13_42_00_UTC', 'scheduled-2023-08-09-13_43_00_UTC', 'scheduled-2023-08-09-13_44_01_UTC', 'scheduled-2023-08-09-13_45_01_UTC', 'scheduled-2023-08-09-13_46_01_UTC', 'scheduled-2023-08-09-13_47_01_UTC'] 2023-08-09T13:47:28.755 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs snap-schedule status --fs cephfs --format json /snap_test_dir1/dir0 2023-08-09T13:47:28.992 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:28.990+0000 7f3d6b07c640 1 Processor -- start 2023-08-09T13:47:28.992 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:28.991+0000 7f3d6b07c640 1 -- start start 2023-08-09T13:47:28.994 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:28.992+0000 7f3d6b07c640 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7f3d6406fb70 0x7f3d640caab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:47:28.994 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:28.993+0000 7f3d6b07c640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f3d640ce4c0 0x7f3d640ce950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:47:28.994 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:28.993+0000 7f3d6b07c640 1 -- --> v1:172.21.15.43:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3d6415b9b0 con 0x7f3d640ca370 2023-08-09T13:47:28.994 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:28.993+0000 7f3d6b07c640 1 -- --> v2:172.21.15.43:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3d640cee90 con 0x7f3d640ce4c0 2023-08-09T13:47:28.995 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:28.993+0000 7f3d6b07c640 1 -- --> v2:172.21.15.203:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3d6415ae10 con 0x7f3d6406fb70 2023-08-09T13:47:28.995 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:28.993+0000 7f3d63fff640 1 --1- >> v1:172.21.15.43:6789/0 conn(0x7f3d640ca370 0x7f3d6415b2a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.43:6789/0 says I am v1:172.21.15.43:49570/0 (socket says 172.21.15.43:49570) 2023-08-09T13:47:28.996 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:28.993+0000 7f3d63fff640 1 -- 172.21.15.43:0/1111957329 learned_addr learned my addr 172.21.15.43:0/1111957329 (peer_addr_for_me v1:172.21.15.43:0/0) 2023-08-09T13:47:28.996 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:28.993+0000 7f3d69619640 1 --2- 172.21.15.43:0/1111957329 >> v2:172.21.15.43:3300/0 conn(0x7f3d640ce4c0 0x7f3d640ce950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:47:28.996 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:28.993+0000 7f3d68e18640 1 --2- 172.21.15.43:0/1111957329 >> v2:172.21.15.203:3300/0 conn(0x7f3d6406fb70 0x7f3d640caab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:47:28.996 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:28.994+0000 7f3d69619640 1 -- 172.21.15.43:0/1111957329 >> v2:172.21.15.203:3300/0 conn(0x7f3d6406fb70 msgr2=0x7f3d640caab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-09T13:47:28.996 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:28.994+0000 7f3d69619640 1 --2- 172.21.15.43:0/1111957329 >> v2:172.21.15.203:3300/0 conn(0x7f3d6406fb70 0x7f3d640caab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:47:28.996 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:28.994+0000 7f3d69619640 1 -- 172.21.15.43:0/1111957329 >> v1:172.21.15.43:6789/0 conn(0x7f3d640ca370 legacy=0x7f3d6415b2a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:47:28.997 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:28.994+0000 7f3d69619640 1 -- 172.21.15.43:0/1111957329 --> v2:172.21.15.43:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3d64154e70 con 0x7f3d640ce4c0 2023-08-09T13:47:28.997 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:28.995+0000 7f3d69619640 1 --2- 172.21.15.43:0/1111957329 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f3d640ce4c0 0x7f3d640ce950 secure :-1 s=READY pgs=686 cs=0 l=1 rev1=1 crypto rx=0x7f3d640cbb80 tx=0x7f3d5c02bd00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e54d32ab4941e3c5 server_cookie=0 in_seq=0 out_seq=0 2023-08-09T13:47:28.998 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:28.995+0000 7f3d637fe640 1 -- 172.21.15.43:0/1111957329 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3d5c02ca30 con 0x7f3d640ce4c0 2023-08-09T13:47:28.998 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:28.995+0000 7f3d637fe640 1 -- 172.21.15.43:0/1111957329 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3d5c02cb90 con 0x7f3d640ce4c0 2023-08-09T13:47:28.998 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:28.995+0000 7f3d637fe640 1 -- 172.21.15.43:0/1111957329 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3d5c034440 con 0x7f3d640ce4c0 2023-08-09T13:47:28.998 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:28.996+0000 7f3d6b07c640 1 -- 172.21.15.43:0/1111957329 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f3d640ce4c0 msgr2=0x7f3d640ce950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:47:28.998 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:28.996+0000 7f3d6b07c640 1 --2- 172.21.15.43:0/1111957329 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f3d640ce4c0 0x7f3d640ce950 secure :-1 s=READY pgs=686 cs=0 l=1 rev1=1 crypto rx=0x7f3d640cbb80 tx=0x7f3d5c02bd00 comp rx=0 tx=0).stop 2023-08-09T13:47:28.998 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:28.996+0000 7f3d6b07c640 1 -- 172.21.15.43:0/1111957329 shutdown_connections 2023-08-09T13:47:28.999 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:28.996+0000 7f3d6b07c640 1 --2- 172.21.15.43:0/1111957329 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f3d640ce4c0 0x7f3d640ce950 unknown :-1 s=CLOSED pgs=686 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:47:28.999 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:28.996+0000 7f3d6b07c640 1 --2- 172.21.15.43:0/1111957329 >> v2:172.21.15.203:3300/0 conn(0x7f3d6406fb70 0x7f3d640caab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:47:28.999 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:28.996+0000 7f3d6b07c640 1 -- 172.21.15.43:0/1111957329 >> 172.21.15.43:0/1111957329 conn(0x7f3d6406b990 msgr2=0x7f3d6406bda0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-09T13:47:28.999 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:28.996+0000 7f3d6b07c640 1 -- 172.21.15.43:0/1111957329 shutdown_connections 2023-08-09T13:47:28.999 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:28.996+0000 7f3d6b07c640 1 -- 172.21.15.43:0/1111957329 wait complete. 2023-08-09T13:47:29.000 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:28.998+0000 7f3d6b07c640 1 Processor -- start 2023-08-09T13:47:29.000 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:28.998+0000 7f3d6b07c640 1 -- start start 2023-08-09T13:47:29.001 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:28.999+0000 7f3d6b07c640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f3d6406fb70 0x7f3d640c2d70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:47:29.001 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:28.999+0000 7f3d6b07c640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f3d640ca370 0x7f3d640b26a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:47:29.001 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:28.999+0000 7f3d6b07c640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f3d640ce4c0 0x7f3d64154610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:47:29.001 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:28.999+0000 7f3d6b07c640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3d640c39c0 con 0x7f3d640ce4c0 2023-08-09T13:47:29.002 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:28.999+0000 7f3d6b07c640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3d640c51e0 con 0x7f3d6406fb70 2023-08-09T13:47:29.002 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:28.999+0000 7f3d6b07c640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3d640c20e0 con 0x7f3d640ca370 2023-08-09T13:47:29.002 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:28.999+0000 7f3d69619640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f3d640ce4c0 0x7f3d64154610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:47:29.002 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:28.999+0000 7f3d69619640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f3d640ce4c0 0x7f3d64154610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:53112/0 (socket says 172.21.15.43:53112) 2023-08-09T13:47:29.002 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:28.999+0000 7f3d69619640 1 -- 172.21.15.43:0/3343495267 learned_addr learned my addr 172.21.15.43:0/3343495267 (peer_addr_for_me v2:172.21.15.43:0/0) 2023-08-09T13:47:29.002 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:29.000+0000 7f3d68e18640 1 --2- 172.21.15.43:0/3343495267 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f3d6406fb70 0x7f3d640c2d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:47:29.002 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:29.000+0000 7f3d69619640 1 -- 172.21.15.43:0/3343495267 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f3d640ca370 msgr2=0x7f3d640b26a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:47:29.003 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:29.000+0000 7f3d69619640 1 --2- 172.21.15.43:0/3343495267 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f3d640ca370 0x7f3d640b26a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:47:29.003 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:29.000+0000 7f3d69619640 1 -- 172.21.15.43:0/3343495267 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f3d6406fb70 msgr2=0x7f3d640c2d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:47:29.003 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:29.000+0000 7f3d69619640 1 --2- 172.21.15.43:0/3343495267 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f3d6406fb70 0x7f3d640c2d70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:47:29.003 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:29.000+0000 7f3d69619640 1 -- 172.21.15.43:0/3343495267 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3d5c009a70 con 0x7f3d640ce4c0 2023-08-09T13:47:29.004 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:29.000+0000 7f3d69619640 1 --2- 172.21.15.43:0/3343495267 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f3d640ce4c0 0x7f3d64154610 secure :-1 s=READY pgs=687 cs=0 l=1 rev1=1 crypto rx=0x7f3d5c009e80 tx=0x7f3d5c004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-09T13:47:29.004 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:29.000+0000 7f3d617fa640 1 -- 172.21.15.43:0/3343495267 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3d5c040650 con 0x7f3d640ce4c0 2023-08-09T13:47:29.004 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:29.000+0000 7f3d617fa640 1 -- 172.21.15.43:0/3343495267 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3d5c0407d0 con 0x7f3d640ce4c0 2023-08-09T13:47:29.004 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:29.000+0000 7f3d617fa640 1 -- 172.21.15.43:0/3343495267 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3d5c040b80 con 0x7f3d640ce4c0 2023-08-09T13:47:29.005 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:29.000+0000 7f3d6b07c640 1 -- 172.21.15.43:0/3343495267 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3d640b2be0 con 0x7f3d640ce4c0 2023-08-09T13:47:29.005 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:29.001+0000 7f3d6b07c640 1 -- 172.21.15.43:0/3343495267 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3d640c3400 con 0x7f3d640ce4c0 2023-08-09T13:47:29.005 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:29.001+0000 7f3d6b07c640 1 -- 172.21.15.43:0/3343495267 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3d640c2250 con 0x7f3d640ce4c0 2023-08-09T13:47:29.005 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:29.003+0000 7f3d617fa640 1 -- 172.21.15.43:0/3343495267 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 13) v1 ==== 88529+0+0 (secure 0 0 0) 0x7f3d5c040ce0 con 0x7f3d640ce4c0 2023-08-09T13:47:29.005 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:29.003+0000 7f3d617fa640 1 --2- 172.21.15.43:0/3343495267 >> [v2:172.21.15.203:6832/998005540,v1:172.21.15.203:6833/998005540] conn(0x7f3d4806a280 0x7f3d4806c740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:47:29.006 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:29.004+0000 7f3d617fa640 1 -- 172.21.15.43:0/3343495267 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(42..42 src has 1..42) v4 ==== 6698+0+0 (secure 0 0 0) 0x7f3d5c0aba40 con 0x7f3d640ce4c0 2023-08-09T13:47:29.006 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:29.004+0000 7f3d68e18640 1 --2- 172.21.15.43:0/3343495267 >> [v2:172.21.15.203:6832/998005540,v1:172.21.15.203:6833/998005540] conn(0x7f3d4806a280 0x7f3d4806c740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:47:29.006 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:29.005+0000 7f3d68e18640 1 --2- 172.21.15.43:0/3343495267 >> [v2:172.21.15.203:6832/998005540,v1:172.21.15.203:6833/998005540] conn(0x7f3d4806a280 0x7f3d4806c740 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f3d58006fd0 tx=0x7f3d58008040 comp rx=0 tx=0).ready entity=mgr.5003 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-09T13:47:29.009 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:29.008+0000 7f3d617fa640 1 -- 172.21.15.43:0/3343495267 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178066 (secure 0 0 0) 0x7f3d5c076b90 con 0x7f3d640ce4c0 2023-08-09T13:47:29.244 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:29.242+0000 7f3d6b07c640 1 -- 172.21.15.43:0/3343495267 --> [v2:172.21.15.203:6832/998005540,v1:172.21.15.203:6833/998005540] -- mgr_command(tid 0: {"prefix": "fs snap-schedule status", "fs": "cephfs", "path": "/snap_test_dir1/dir0", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f3d640cb600 con 0x7f3d4806a280 2023-08-09T13:47:29.262 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:29.260+0000 7f3d617fa640 1 -- 172.21.15.43:0/3343495267 <== mgr.5003 v2:172.21.15.203:6832/998005540 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+345 (secure 0 0 0) 0x7f3d58009d60 con 0x7f3d4806a280 2023-08-09T13:47:29.262 INFO:teuthology.orchestra.run.smithi043.stdout: 2023-08-09T13:47:29.263 INFO:teuthology.orchestra.run.smithi043.stdout:[{"fs": "cephfs", "subvol": null, "path": "/snap_test_dir1/dir0", "rel_path": "/snap_test_dir1/dir0", "schedule": "1M", "retention": {}, "start": "2023-08-09T00:00:00", "created": "2023-08-09T13:36:29", "first": "2023-08-09T13:37:00", "last": "2023-08-09T13:47:01", "last_pruned": null, "created_count": 11, "pruned_count": 0, "active": false}] 2023-08-09T13:47:29.265 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:29.263+0000 7f3d6b07c640 1 -- 172.21.15.43:0/3343495267 >> [v2:172.21.15.203:6832/998005540,v1:172.21.15.203:6833/998005540] conn(0x7f3d4806a280 msgr2=0x7f3d4806c740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:47:29.265 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:29.263+0000 7f3d6b07c640 1 --2- 172.21.15.43:0/3343495267 >> [v2:172.21.15.203:6832/998005540,v1:172.21.15.203:6833/998005540] conn(0x7f3d4806a280 0x7f3d4806c740 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f3d58006fd0 tx=0x7f3d58008040 comp rx=0 tx=0).stop 2023-08-09T13:47:29.265 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:29.263+0000 7f3d6b07c640 1 -- 172.21.15.43:0/3343495267 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f3d640ce4c0 msgr2=0x7f3d64154610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:47:29.265 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:29.263+0000 7f3d6b07c640 1 --2- 172.21.15.43:0/3343495267 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f3d640ce4c0 0x7f3d64154610 secure :-1 s=READY pgs=687 cs=0 l=1 rev1=1 crypto rx=0x7f3d5c009e80 tx=0x7f3d5c004a40 comp rx=0 tx=0).stop 2023-08-09T13:47:29.266 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:29.263+0000 7f3d6b07c640 1 -- 172.21.15.43:0/3343495267 shutdown_connections 2023-08-09T13:47:29.266 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:29.263+0000 7f3d6b07c640 1 --2- 172.21.15.43:0/3343495267 >> [v2:172.21.15.203:6832/998005540,v1:172.21.15.203:6833/998005540] conn(0x7f3d4806a280 0x7f3d4806c740 unknown :-1 s=CLOSED pgs=53 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:47:29.266 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:29.263+0000 7f3d6b07c640 1 --2- 172.21.15.43:0/3343495267 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f3d640ce4c0 0x7f3d64154610 unknown :-1 s=CLOSED pgs=687 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:47:29.266 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:29.263+0000 7f3d6b07c640 1 --2- 172.21.15.43:0/3343495267 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f3d640ca370 0x7f3d640b26a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:47:29.266 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:29.263+0000 7f3d6b07c640 1 --2- 172.21.15.43:0/3343495267 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f3d6406fb70 0x7f3d640c2d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:47:29.266 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:29.263+0000 7f3d6b07c640 1 -- 172.21.15.43:0/3343495267 >> 172.21.15.43:0/3343495267 conn(0x7f3d6406b990 msgr2=0x7f3d640b0400 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-09T13:47:29.267 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:29.264+0000 7f3d6b07c640 1 -- 172.21.15.43:0/3343495267 shutdown_connections 2023-08-09T13:47:29.267 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:29.264+0000 7f3d6b07c640 1 -- 172.21.15.43:0/3343495267 wait complete. 2023-08-09T13:47:29.284 DEBUG:tasks.cephfs.test_snap_schedules:res= [{"fs": "cephfs", "subvol": null, "path": "/snap_test_dir1/dir0", "rel_path": "/snap_test_dir1/dir0", "schedule": "1M", "retention": {}, "start": "2023-08-09T00:00:00", "created": "2023-08-09T13:36:29", "first": "2023-08-09T13:37:00", "last": "2023-08-09T13:47:01", "last_pruned": null, "created_count": 11, "pruned_count": 0, "active": false}] 2023-08-09T13:47:29.285 DEBUG:tasks.cephfs.test_snap_schedules:json_res: {'fs': 'cephfs', 'subvol': None, 'path': '/snap_test_dir1/dir0', 'rel_path': '/snap_test_dir1/dir0', 'schedule': '1M', 'retention': {}, 'start': '2023-08-09T00:00:00', 'created': '2023-08-09T13:36:29', 'first': '2023-08-09T13:37:00', 'last': '2023-08-09T13:47:01', 'last_pruned': None, 'created_count': 11, 'pruned_count': 0, 'active': False} 2023-08-09T13:47:29.285 DEBUG:tasks.cephfs.test_snap_schedules:fs_count: 11 2023-08-09T13:47:29.286 DEBUG:tasks.cephfs.test_snap_schedules:db_count: 11 2023-08-09T13:47:29.286 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-09T13:47:29.286 DEBUG:teuthology.orchestra.run.smithi043:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/dir1/.snap) 2023-08-09T13:47:29.354 INFO:teuthology.orchestra.run.smithi043.stdout:scheduled-2023-08-09-13_37_01_UTC 2023-08-09T13:47:29.354 INFO:teuthology.orchestra.run.smithi043.stdout:scheduled-2023-08-09-13_38_01_UTC 2023-08-09T13:47:29.355 INFO:teuthology.orchestra.run.smithi043.stdout:scheduled-2023-08-09-13_39_00_UTC 2023-08-09T13:47:29.355 INFO:teuthology.orchestra.run.smithi043.stdout:scheduled-2023-08-09-13_40_00_UTC 2023-08-09T13:47:29.355 INFO:teuthology.orchestra.run.smithi043.stdout:scheduled-2023-08-09-13_41_00_UTC 2023-08-09T13:47:29.355 INFO:teuthology.orchestra.run.smithi043.stdout:scheduled-2023-08-09-13_42_01_UTC 2023-08-09T13:47:29.355 INFO:teuthology.orchestra.run.smithi043.stdout:scheduled-2023-08-09-13_43_01_UTC 2023-08-09T13:47:29.355 INFO:teuthology.orchestra.run.smithi043.stdout:scheduled-2023-08-09-13_44_01_UTC 2023-08-09T13:47:29.356 INFO:teuthology.orchestra.run.smithi043.stdout:scheduled-2023-08-09-13_45_01_UTC 2023-08-09T13:47:29.356 INFO:teuthology.orchestra.run.smithi043.stdout:scheduled-2023-08-09-13_46_01_UTC 2023-08-09T13:47:29.356 INFO:teuthology.orchestra.run.smithi043.stdout:scheduled-2023-08-09-13_47_01_UTC 2023-08-09T13:47:29.357 DEBUG:tasks.cephfs.test_snap_schedules:snapshots: ['scheduled-2023-08-09-13_37_01_UTC', 'scheduled-2023-08-09-13_38_01_UTC', 'scheduled-2023-08-09-13_39_00_UTC', 'scheduled-2023-08-09-13_40_00_UTC', 'scheduled-2023-08-09-13_41_00_UTC', 'scheduled-2023-08-09-13_42_01_UTC', 'scheduled-2023-08-09-13_43_01_UTC', 'scheduled-2023-08-09-13_44_01_UTC', 'scheduled-2023-08-09-13_45_01_UTC', 'scheduled-2023-08-09-13_46_01_UTC', 'scheduled-2023-08-09-13_47_01_UTC'] 2023-08-09T13:47:29.357 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs snap-schedule status --fs cephfs --format json /snap_test_dir1/dir1 2023-08-09T13:47:29.594 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:29.592+0000 7f8151c6c640 1 Processor -- start 2023-08-09T13:47:29.594 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:29.592+0000 7f8151c6c640 1 -- start start 2023-08-09T13:47:29.596 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:29.594+0000 7f8151c6c640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f814c06fb70 0x7f814c0caab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:47:29.596 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:29.594+0000 7f8151c6c640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f814c0ce4c0 0x7f814c0ce950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:47:29.596 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:29.594+0000 7f8151c6c640 1 -- --> v1:172.21.15.43:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f814c15b9b0 con 0x7f814c0ca370 2023-08-09T13:47:29.596 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:29.594+0000 7f8151c6c640 1 -- --> v2:172.21.15.43:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f814c0cee90 con 0x7f814c06fb70 2023-08-09T13:47:29.597 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:29.594+0000 7f8151c6c640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f814c15ae10 con 0x7f814c0ce4c0 2023-08-09T13:47:29.597 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:29.594+0000 7f814b7fe640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f814c06fb70 0x7f814c0caab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:47:29.597 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:29.594+0000 7f8143fff640 1 --1- >> v1:172.21.15.43:6789/0 conn(0x7f814c0ca370 0x7f814c15b2a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.43:6789/0 says I am v1:172.21.15.43:49586/0 (socket says 172.21.15.43:49586) 2023-08-09T13:47:29.597 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:29.594+0000 7f8143fff640 1 -- 172.21.15.43:0/2630557870 learned_addr learned my addr 172.21.15.43:0/2630557870 (peer_addr_for_me v1:172.21.15.43:0/0) 2023-08-09T13:47:29.597 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:29.594+0000 7f814b7fe640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f814c06fb70 0x7f814c0caab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:53124/0 (socket says 172.21.15.43:53124) 2023-08-09T13:47:29.597 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:29.595+0000 7f814bfff640 1 --2- 172.21.15.43:0/2630557870 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f814c0ce4c0 0x7f814c0ce950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:47:29.598 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:29.595+0000 7f814b7fe640 1 -- 172.21.15.43:0/2630557870 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f814c0ce4c0 msgr2=0x7f814c0ce950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-09T13:47:29.598 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:29.595+0000 7f814b7fe640 1 --2- 172.21.15.43:0/2630557870 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f814c0ce4c0 0x7f814c0ce950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:47:29.598 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:29.595+0000 7f814b7fe640 1 -- 172.21.15.43:0/2630557870 >> v1:172.21.15.43:6789/0 conn(0x7f814c0ca370 legacy=0x7f814c15b2a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:47:29.598 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:29.595+0000 7f814b7fe640 1 -- 172.21.15.43:0/2630557870 --> v2:172.21.15.43:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f814c154e70 con 0x7f814c06fb70 2023-08-09T13:47:29.599 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:29.596+0000 7f814b7fe640 1 --2- 172.21.15.43:0/2630557870 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f814c06fb70 0x7f814c0caab0 secure :-1 s=READY pgs=689 cs=0 l=1 rev1=1 crypto rx=0x7f813c0097a0 tx=0x7f813c02b8b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f8d57277a74a5011 server_cookie=0 in_seq=0 out_seq=0 2023-08-09T13:47:29.599 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:29.597+0000 7f814affd640 1 -- 172.21.15.43:0/2630557870 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f813c02d040 con 0x7f814c06fb70 2023-08-09T13:47:29.599 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:29.597+0000 7f814affd640 1 -- 172.21.15.43:0/2630557870 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f813c031810 con 0x7f814c06fb70 2023-08-09T13:47:29.599 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:29.597+0000 7f814affd640 1 -- 172.21.15.43:0/2630557870 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f813c031ba0 con 0x7f814c06fb70 2023-08-09T13:47:29.600 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:29.597+0000 7f8151c6c640 1 -- 172.21.15.43:0/2630557870 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f814c06fb70 msgr2=0x7f814c0caab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:47:29.600 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:29.597+0000 7f8151c6c640 1 --2- 172.21.15.43:0/2630557870 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f814c06fb70 0x7f814c0caab0 secure :-1 s=READY pgs=689 cs=0 l=1 rev1=1 crypto rx=0x7f813c0097a0 tx=0x7f813c02b8b0 comp rx=0 tx=0).stop 2023-08-09T13:47:29.600 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:29.597+0000 7f8151c6c640 1 -- 172.21.15.43:0/2630557870 shutdown_connections 2023-08-09T13:47:29.600 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:29.597+0000 7f8151c6c640 1 --2- 172.21.15.43:0/2630557870 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f814c0ce4c0 0x7f814c0ce950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:47:29.600 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:29.597+0000 7f8151c6c640 1 --2- 172.21.15.43:0/2630557870 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f814c06fb70 0x7f814c0caab0 unknown :-1 s=CLOSED pgs=689 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:47:29.601 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:29.597+0000 7f8151c6c640 1 -- 172.21.15.43:0/2630557870 >> 172.21.15.43:0/2630557870 conn(0x7f814c06b990 msgr2=0x7f814c06bda0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-09T13:47:29.601 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:29.597+0000 7f8151c6c640 1 -- 172.21.15.43:0/2630557870 shutdown_connections 2023-08-09T13:47:29.601 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:29.597+0000 7f8151c6c640 1 -- 172.21.15.43:0/2630557870 wait complete. 2023-08-09T13:47:29.601 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:29.599+0000 7f8151c6c640 1 Processor -- start 2023-08-09T13:47:29.602 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:29.599+0000 7f8151c6c640 1 -- start start 2023-08-09T13:47:29.602 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:29.600+0000 7f8151c6c640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f814c06fb70 0x7f814c0c1bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:47:29.602 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:29.600+0000 7f8151c6c640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f814c0ca370 0x7f814c0b1330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:47:29.602 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:29.600+0000 7f8151c6c640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f814c0ce4c0 0x7f814c0b0490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:47:29.603 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:29.600+0000 7f8151c6c640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f814c0b2e50 con 0x7f814c06fb70 2023-08-09T13:47:29.603 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:29.600+0000 7f8151c6c640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f814c153e20 con 0x7f814c0ca370 2023-08-09T13:47:29.604 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:29.600+0000 7f8151c6c640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f814c0b25f0 con 0x7f814c0ce4c0 2023-08-09T13:47:29.604 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:29.601+0000 7f814b7fe640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f814c06fb70 0x7f814c0c1bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:47:29.604 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:29.601+0000 7f814b7fe640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f814c06fb70 0x7f814c0c1bd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:53128/0 (socket says 172.21.15.43:53128) 2023-08-09T13:47:29.604 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:29.601+0000 7f814b7fe640 1 -- 172.21.15.43:0/768749989 learned_addr learned my addr 172.21.15.43:0/768749989 (peer_addr_for_me v2:172.21.15.43:0/0) 2023-08-09T13:47:29.604 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:29.601+0000 7f814bfff640 1 --2- 172.21.15.43:0/768749989 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f814c0ce4c0 0x7f814c0b0490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:47:29.604 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:29.601+0000 7f8143fff640 1 --2- 172.21.15.43:0/768749989 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f814c0ca370 0x7f814c0b1330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:47:29.604 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:29.601+0000 7f814b7fe640 1 -- 172.21.15.43:0/768749989 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f814c0ce4c0 msgr2=0x7f814c0b0490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:47:29.604 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:29.601+0000 7f814b7fe640 1 --2- 172.21.15.43:0/768749989 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f814c0ce4c0 0x7f814c0b0490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:47:29.605 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:29.601+0000 7f814b7fe640 1 -- 172.21.15.43:0/768749989 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f814c0ca370 msgr2=0x7f814c0b1330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:47:29.605 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:29.601+0000 7f814b7fe640 1 --2- 172.21.15.43:0/768749989 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f814c0ca370 0x7f814c0b1330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:47:29.605 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:29.601+0000 7f814b7fe640 1 -- 172.21.15.43:0/768749989 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f813c009400 con 0x7f814c06fb70 2023-08-09T13:47:29.605 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:29.601+0000 7f8143fff640 1 --2- 172.21.15.43:0/768749989 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f814c0ca370 0x7f814c0b1330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-08-09T13:47:29.605 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:29.602+0000 7f814b7fe640 1 --2- 172.21.15.43:0/768749989 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f814c06fb70 0x7f814c0c1bd0 secure :-1 s=READY pgs=690 cs=0 l=1 rev1=1 crypto rx=0x7f813c0098c0 tx=0x7f813c0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-09T13:47:29.605 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:29.602+0000 7f8148ff9640 1 -- 172.21.15.43:0/768749989 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f813c03ad60 con 0x7f814c06fb70 2023-08-09T13:47:29.605 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:29.602+0000 7f8148ff9640 1 -- 172.21.15.43:0/768749989 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f813c044400 con 0x7f814c06fb70 2023-08-09T13:47:29.606 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:29.602+0000 7f8148ff9640 1 -- 172.21.15.43:0/768749989 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f813c0447b0 con 0x7f814c06fb70 2023-08-09T13:47:29.606 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:29.602+0000 7f8151c6c640 1 -- 172.21.15.43:0/768749989 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f814c0c3730 con 0x7f814c06fb70 2023-08-09T13:47:29.606 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:29.602+0000 7f8151c6c640 1 -- 172.21.15.43:0/768749989 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f814c0aeb90 con 0x7f814c06fb70 2023-08-09T13:47:29.607 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:29.603+0000 7f8151c6c640 1 -- 172.21.15.43:0/768749989 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f814c0b1920 con 0x7f814c06fb70 2023-08-09T13:47:29.607 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:29.604+0000 7f8148ff9640 1 -- 172.21.15.43:0/768749989 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 13) v1 ==== 88529+0+0 (secure 0 0 0) 0x7f813c04d440 con 0x7f814c06fb70 2023-08-09T13:47:29.607 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:29.605+0000 7f8148ff9640 1 --2- 172.21.15.43:0/768749989 >> [v2:172.21.15.203:6832/998005540,v1:172.21.15.203:6833/998005540] conn(0x7f812c06a250 0x7f812c06c710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:47:29.607 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:29.605+0000 7f8148ff9640 1 -- 172.21.15.43:0/768749989 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(42..42 src has 1..42) v4 ==== 6698+0+0 (secure 0 0 0) 0x7f813c0ab0f0 con 0x7f814c06fb70 2023-08-09T13:47:29.607 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:29.606+0000 7f8143fff640 1 --2- 172.21.15.43:0/768749989 >> [v2:172.21.15.203:6832/998005540,v1:172.21.15.203:6833/998005540] conn(0x7f812c06a250 0x7f812c06c710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:47:29.608 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:29.607+0000 7f8143fff640 1 --2- 172.21.15.43:0/768749989 >> [v2:172.21.15.203:6832/998005540,v1:172.21.15.203:6833/998005540] conn(0x7f812c06a250 0x7f812c06c710 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f814c0b10b0 tx=0x7f8134009040 comp rx=0 tx=0).ready entity=mgr.5003 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-09T13:47:29.610 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:29.609+0000 7f8148ff9640 1 -- 172.21.15.43:0/768749989 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178066 (secure 0 0 0) 0x7f813c0762e0 con 0x7f814c06fb70 2023-08-09T13:47:29.824 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:29.823+0000 7f8151c6c640 1 -- 172.21.15.43:0/768749989 --> [v2:172.21.15.203:6832/998005540,v1:172.21.15.203:6833/998005540] -- mgr_command(tid 0: {"prefix": "fs snap-schedule status", "fs": "cephfs", "path": "/snap_test_dir1/dir1", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f814c0cb320 con 0x7f812c06a250 2023-08-09T13:47:29.845 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:29.843+0000 7f8148ff9640 1 -- 172.21.15.43:0/768749989 <== mgr.5003 v2:172.21.15.203:6832/998005540 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+345 (secure 0 0 0) 0x7f81340013d0 con 0x7f812c06a250 2023-08-09T13:47:29.845 INFO:teuthology.orchestra.run.smithi043.stdout: 2023-08-09T13:47:29.846 INFO:teuthology.orchestra.run.smithi043.stdout:[{"fs": "cephfs", "subvol": null, "path": "/snap_test_dir1/dir1", "rel_path": "/snap_test_dir1/dir1", "schedule": "1M", "retention": {}, "start": "2023-08-09T00:00:00", "created": "2023-08-09T13:36:29", "first": "2023-08-09T13:37:01", "last": "2023-08-09T13:47:01", "last_pruned": null, "created_count": 11, "pruned_count": 0, "active": false}] 2023-08-09T13:47:29.847 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:29.846+0000 7f8151c6c640 1 -- 172.21.15.43:0/768749989 >> [v2:172.21.15.203:6832/998005540,v1:172.21.15.203:6833/998005540] conn(0x7f812c06a250 msgr2=0x7f812c06c710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:47:29.848 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:29.846+0000 7f8151c6c640 1 --2- 172.21.15.43:0/768749989 >> [v2:172.21.15.203:6832/998005540,v1:172.21.15.203:6833/998005540] conn(0x7f812c06a250 0x7f812c06c710 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f814c0b10b0 tx=0x7f8134009040 comp rx=0 tx=0).stop 2023-08-09T13:47:29.848 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:29.846+0000 7f8151c6c640 1 -- 172.21.15.43:0/768749989 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f814c06fb70 msgr2=0x7f814c0c1bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:47:29.848 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:29.846+0000 7f8151c6c640 1 --2- 172.21.15.43:0/768749989 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f814c06fb70 0x7f814c0c1bd0 secure :-1 s=READY pgs=690 cs=0 l=1 rev1=1 crypto rx=0x7f813c0098c0 tx=0x7f813c0046c0 comp rx=0 tx=0).stop 2023-08-09T13:47:29.849 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:29.846+0000 7f8151c6c640 1 -- 172.21.15.43:0/768749989 shutdown_connections 2023-08-09T13:47:29.849 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:29.846+0000 7f8151c6c640 1 --2- 172.21.15.43:0/768749989 >> [v2:172.21.15.203:6832/998005540,v1:172.21.15.203:6833/998005540] conn(0x7f812c06a250 0x7f812c06c710 unknown :-1 s=CLOSED pgs=54 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:47:29.849 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:29.846+0000 7f8151c6c640 1 --2- 172.21.15.43:0/768749989 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f814c0ce4c0 0x7f814c0b0490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:47:29.849 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:29.846+0000 7f8151c6c640 1 --2- 172.21.15.43:0/768749989 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f814c0ca370 0x7f814c0b1330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:47:29.849 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:29.846+0000 7f8151c6c640 1 --2- 172.21.15.43:0/768749989 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f814c06fb70 0x7f814c0c1bd0 unknown :-1 s=CLOSED pgs=690 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:47:29.849 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:29.846+0000 7f8151c6c640 1 -- 172.21.15.43:0/768749989 >> 172.21.15.43:0/768749989 conn(0x7f814c06b990 msgr2=0x7f814c06bd70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-09T13:47:29.849 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:29.846+0000 7f8151c6c640 1 -- 172.21.15.43:0/768749989 shutdown_connections 2023-08-09T13:47:29.850 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:29.846+0000 7f8151c6c640 1 -- 172.21.15.43:0/768749989 wait complete. 2023-08-09T13:47:29.866 DEBUG:tasks.cephfs.test_snap_schedules:res= [{"fs": "cephfs", "subvol": null, "path": "/snap_test_dir1/dir1", "rel_path": "/snap_test_dir1/dir1", "schedule": "1M", "retention": {}, "start": "2023-08-09T00:00:00", "created": "2023-08-09T13:36:29", "first": "2023-08-09T13:37:01", "last": "2023-08-09T13:47:01", "last_pruned": null, "created_count": 11, "pruned_count": 0, "active": false}] 2023-08-09T13:47:29.866 DEBUG:tasks.cephfs.test_snap_schedules:json_res: {'fs': 'cephfs', 'subvol': None, 'path': '/snap_test_dir1/dir1', 'rel_path': '/snap_test_dir1/dir1', 'schedule': '1M', 'retention': {}, 'start': '2023-08-09T00:00:00', 'created': '2023-08-09T13:36:29', 'first': '2023-08-09T13:37:01', 'last': '2023-08-09T13:47:01', 'last_pruned': None, 'created_count': 11, 'pruned_count': 0, 'active': False} 2023-08-09T13:47:29.867 DEBUG:tasks.cephfs.test_snap_schedules:fs_count: 11 2023-08-09T13:47:29.867 DEBUG:tasks.cephfs.test_snap_schedules:db_count: 11 2023-08-09T13:47:29.867 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-09T13:47:29.867 DEBUG:teuthology.orchestra.run.smithi043:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/dir2/.snap) 2023-08-09T13:47:29.937 INFO:teuthology.orchestra.run.smithi043.stdout:scheduled-2023-08-09-13_37_00_UTC 2023-08-09T13:47:29.937 INFO:teuthology.orchestra.run.smithi043.stdout:scheduled-2023-08-09-13_38_00_UTC 2023-08-09T13:47:29.937 INFO:teuthology.orchestra.run.smithi043.stdout:scheduled-2023-08-09-13_39_00_UTC 2023-08-09T13:47:29.938 INFO:teuthology.orchestra.run.smithi043.stdout:scheduled-2023-08-09-13_40_00_UTC 2023-08-09T13:47:29.938 INFO:teuthology.orchestra.run.smithi043.stdout:scheduled-2023-08-09-13_41_00_UTC 2023-08-09T13:47:29.938 INFO:teuthology.orchestra.run.smithi043.stdout:scheduled-2023-08-09-13_42_00_UTC 2023-08-09T13:47:29.938 INFO:teuthology.orchestra.run.smithi043.stdout:scheduled-2023-08-09-13_43_00_UTC 2023-08-09T13:47:29.938 INFO:teuthology.orchestra.run.smithi043.stdout:scheduled-2023-08-09-13_44_00_UTC 2023-08-09T13:47:29.938 INFO:teuthology.orchestra.run.smithi043.stdout:scheduled-2023-08-09-13_45_00_UTC 2023-08-09T13:47:29.939 INFO:teuthology.orchestra.run.smithi043.stdout:scheduled-2023-08-09-13_46_00_UTC 2023-08-09T13:47:29.939 INFO:teuthology.orchestra.run.smithi043.stdout:scheduled-2023-08-09-13_47_00_UTC 2023-08-09T13:47:29.940 DEBUG:tasks.cephfs.test_snap_schedules:snapshots: ['scheduled-2023-08-09-13_37_00_UTC', 'scheduled-2023-08-09-13_38_00_UTC', 'scheduled-2023-08-09-13_39_00_UTC', 'scheduled-2023-08-09-13_40_00_UTC', 'scheduled-2023-08-09-13_41_00_UTC', 'scheduled-2023-08-09-13_42_00_UTC', 'scheduled-2023-08-09-13_43_00_UTC', 'scheduled-2023-08-09-13_44_00_UTC', 'scheduled-2023-08-09-13_45_00_UTC', 'scheduled-2023-08-09-13_46_00_UTC', 'scheduled-2023-08-09-13_47_00_UTC'] 2023-08-09T13:47:29.940 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs snap-schedule status --fs cephfs --format json /snap_test_dir1/dir2 2023-08-09T13:47:30.180 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:30.178+0000 7f484fb76640 1 Processor -- start 2023-08-09T13:47:30.180 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:30.179+0000 7f484fb76640 1 -- start start 2023-08-09T13:47:30.182 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:30.180+0000 7f484fb76640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f484815a560 0x7f48480d09d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:47:30.182 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:30.180+0000 7f484fb76640 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7f48480d0f10 0x7f48480cccd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:47:30.183 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:30.180+0000 7f484fb76640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f48480cd210 0x7f48480b0f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:47:30.183 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:30.180+0000 7f484fb76640 1 -- --> v2:172.21.15.43:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f48480cd6c0 con 0x7f48480cd210 2023-08-09T13:47:30.183 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:30.180+0000 7f484fb76640 1 -- --> v2:172.21.15.203:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f484815a940 con 0x7f48480d0f10 2023-08-09T13:47:30.183 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:30.180+0000 7f484fb76640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f48480cbde0 con 0x7f484815a560 2023-08-09T13:47:30.183 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:30.181+0000 7f484e113640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f48480cd210 0x7f48480b0f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:47:30.184 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:30.181+0000 7f484e113640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f48480cd210 0x7f48480b0f30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:53138/0 (socket says 172.21.15.43:53138) 2023-08-09T13:47:30.184 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:30.181+0000 7f484e113640 1 -- 172.21.15.43:0/2736434632 learned_addr learned my addr 172.21.15.43:0/2736434632 (peer_addr_for_me v2:172.21.15.43:0/0) 2023-08-09T13:47:30.184 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:30.181+0000 7f484d912640 1 --2- 172.21.15.43:0/2736434632 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f484815a560 0x7f48480d09d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:47:30.185 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:30.181+0000 7f484e113640 1 -- 172.21.15.43:0/2736434632 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f484815a560 msgr2=0x7f48480d09d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-09T13:47:30.185 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:30.181+0000 7f484e113640 1 --2- 172.21.15.43:0/2736434632 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f484815a560 0x7f48480d09d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:47:30.185 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:30.181+0000 7f484e113640 1 -- 172.21.15.43:0/2736434632 >> v2:172.21.15.203:3300/0 conn(0x7f48480d0f10 msgr2=0x7f48480cccd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-09T13:47:30.185 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:30.181+0000 7f484d111640 1 --2- 172.21.15.43:0/2736434632 >> v2:172.21.15.203:3300/0 conn(0x7f48480d0f10 0x7f48480cccd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:47:30.186 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:30.181+0000 7f484e113640 1 --2- 172.21.15.43:0/2736434632 >> v2:172.21.15.203:3300/0 conn(0x7f48480d0f10 0x7f48480cccd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:47:30.186 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:30.181+0000 7f484e113640 1 -- 172.21.15.43:0/2736434632 --> v2:172.21.15.43:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f48480cb1d0 con 0x7f48480cd210 2023-08-09T13:47:30.186 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:30.181+0000 7f484d111640 1 --2- 172.21.15.43:0/2736434632 >> v2:172.21.15.203:3300/0 conn(0x7f48480d0f10 0x7f48480cccd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-08-09T13:47:30.186 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:30.183+0000 7f484e113640 1 --2- 172.21.15.43:0/2736434632 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f48480cd210 0x7f48480b0f30 secure :-1 s=READY pgs=691 cs=0 l=1 rev1=1 crypto rx=0x7f4848071380 tx=0x7f484002b910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5deb9f0ab8818b1 server_cookie=0 in_seq=0 out_seq=0 2023-08-09T13:47:30.186 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:30.183+0000 7f484c910640 1 -- 172.21.15.43:0/2736434632 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4840031870 con 0x7f48480cd210 2023-08-09T13:47:30.187 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:30.183+0000 7f484c910640 1 -- 172.21.15.43:0/2736434632 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f48400319d0 con 0x7f48480cd210 2023-08-09T13:47:30.187 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:30.183+0000 7f484c910640 1 -- 172.21.15.43:0/2736434632 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4840031d60 con 0x7f48480cd210 2023-08-09T13:47:30.187 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:30.184+0000 7f484fb76640 1 -- 172.21.15.43:0/2736434632 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f48480cd210 msgr2=0x7f48480b0f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:47:30.187 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:30.184+0000 7f484fb76640 1 --2- 172.21.15.43:0/2736434632 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f48480cd210 0x7f48480b0f30 secure :-1 s=READY pgs=691 cs=0 l=1 rev1=1 crypto rx=0x7f4848071380 tx=0x7f484002b910 comp rx=0 tx=0).stop 2023-08-09T13:47:30.188 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:30.184+0000 7f484fb76640 1 -- 172.21.15.43:0/2736434632 shutdown_connections 2023-08-09T13:47:30.188 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:30.184+0000 7f484fb76640 1 --2- 172.21.15.43:0/2736434632 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f48480cd210 0x7f48480b0f30 unknown :-1 s=CLOSED pgs=691 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:47:30.188 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:30.184+0000 7f484fb76640 1 --2- 172.21.15.43:0/2736434632 >> v2:172.21.15.203:3300/0 conn(0x7f48480d0f10 0x7f48480cccd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:47:30.188 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:30.184+0000 7f484fb76640 1 --2- 172.21.15.43:0/2736434632 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f484815a560 0x7f48480d09d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:47:30.189 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:30.184+0000 7f484fb76640 1 -- 172.21.15.43:0/2736434632 >> 172.21.15.43:0/2736434632 conn(0x7f4848099600 msgr2=0x7f48480d02c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-09T13:47:30.189 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:30.184+0000 7f484fb76640 1 -- 172.21.15.43:0/2736434632 shutdown_connections 2023-08-09T13:47:30.189 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:30.184+0000 7f484fb76640 1 -- 172.21.15.43:0/2736434632 wait complete. 2023-08-09T13:47:30.189 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:30.186+0000 7f484fb76640 1 Processor -- start 2023-08-09T13:47:30.189 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:30.186+0000 7f484fb76640 1 -- start start 2023-08-09T13:47:30.191 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:30.187+0000 7f484fb76640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f48480cd210 0x7f4848142e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:47:30.191 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:30.187+0000 7f484fb76640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f48480d0f10 0x7f4848141520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:47:30.191 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:30.187+0000 7f484fb76640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f484815a560 0x7f48480c5cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:47:30.191 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:30.187+0000 7f484fb76640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4848143350 con 0x7f484815a560 2023-08-09T13:47:30.191 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:30.187+0000 7f484fb76640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4848141d70 con 0x7f48480d0f10 2023-08-09T13:47:30.191 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:30.187+0000 7f484fb76640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4848142050 con 0x7f48480cd210 2023-08-09T13:47:30.192 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:30.187+0000 7f484e113640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f484815a560 0x7f48480c5cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:47:30.192 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:30.187+0000 7f484e113640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f484815a560 0x7f48480c5cf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:53152/0 (socket says 172.21.15.43:53152) 2023-08-09T13:47:30.192 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:30.187+0000 7f484e113640 1 -- 172.21.15.43:0/835771228 learned_addr learned my addr 172.21.15.43:0/835771228 (peer_addr_for_me v2:172.21.15.43:0/0) 2023-08-09T13:47:30.192 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:30.188+0000 7f484d912640 1 --2- 172.21.15.43:0/835771228 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f48480cd210 0x7f4848142e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:47:30.192 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:30.188+0000 7f484d111640 1 --2- 172.21.15.43:0/835771228 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f48480d0f10 0x7f4848141520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:47:30.193 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:30.188+0000 7f484e113640 1 -- 172.21.15.43:0/835771228 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f48480cd210 msgr2=0x7f4848142e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:47:30.193 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:30.188+0000 7f484e113640 1 --2- 172.21.15.43:0/835771228 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f48480cd210 0x7f4848142e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:47:30.193 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:30.188+0000 7f484e113640 1 -- 172.21.15.43:0/835771228 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f48480d0f10 msgr2=0x7f4848141520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:47:30.193 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:30.188+0000 7f484e113640 1 --2- 172.21.15.43:0/835771228 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f48480d0f10 0x7f4848141520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:47:30.193 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:30.188+0000 7f484e113640 1 -- 172.21.15.43:0/835771228 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f48400094d0 con 0x7f484815a560 2023-08-09T13:47:30.194 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:30.188+0000 7f484d912640 1 --2- 172.21.15.43:0/835771228 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f48480cd210 0x7f4848142e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-08-09T13:47:30.194 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:30.188+0000 7f484d111640 1 --2- 172.21.15.43:0/835771228 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f48480d0f10 0x7f4848141520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-08-09T13:47:30.194 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:30.188+0000 7f484e113640 1 --2- 172.21.15.43:0/835771228 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f484815a560 0x7f48480c5cf0 secure :-1 s=READY pgs=692 cs=0 l=1 rev1=1 crypto rx=0x7f48400090b0 tx=0x7f48400046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-09T13:47:30.194 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:30.189+0000 7f483e7fc640 1 -- 172.21.15.43:0/835771228 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f484003ac60 con 0x7f484815a560 2023-08-09T13:47:30.195 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:30.189+0000 7f483e7fc640 1 -- 172.21.15.43:0/835771228 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f484003adc0 con 0x7f484815a560 2023-08-09T13:47:30.195 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:30.189+0000 7f483e7fc640 1 -- 172.21.15.43:0/835771228 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4840044630 con 0x7f484815a560 2023-08-09T13:47:30.195 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:30.189+0000 7f484fb76640 1 -- 172.21.15.43:0/835771228 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4848142330 con 0x7f484815a560 2023-08-09T13:47:30.195 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:30.189+0000 7f484fb76640 1 -- 172.21.15.43:0/835771228 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f48481428f0 con 0x7f484815a560 2023-08-09T13:47:30.197 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:30.190+0000 7f484fb76640 1 -- 172.21.15.43:0/835771228 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4848142ae0 con 0x7f484815a560 2023-08-09T13:47:30.197 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:30.191+0000 7f483e7fc640 1 -- 172.21.15.43:0/835771228 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 13) v1 ==== 88529+0+0 (secure 0 0 0) 0x7f484004d450 con 0x7f484815a560 2023-08-09T13:47:30.197 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:30.192+0000 7f483e7fc640 1 --2- 172.21.15.43:0/835771228 >> [v2:172.21.15.203:6832/998005540,v1:172.21.15.203:6833/998005540] conn(0x7f482c06a250 0x7f482c06c710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:47:30.198 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:30.192+0000 7f483e7fc640 1 -- 172.21.15.43:0/835771228 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(42..42 src has 1..42) v4 ==== 6698+0+0 (secure 0 0 0) 0x7f484002d040 con 0x7f484815a560 2023-08-09T13:47:30.198 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:30.193+0000 7f484d912640 1 --2- 172.21.15.43:0/835771228 >> [v2:172.21.15.203:6832/998005540,v1:172.21.15.203:6833/998005540] conn(0x7f482c06a250 0x7f482c06c710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:47:30.198 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:30.194+0000 7f484d912640 1 --2- 172.21.15.43:0/835771228 >> [v2:172.21.15.203:6832/998005540,v1:172.21.15.203:6833/998005540] conn(0x7f482c06a250 0x7f482c06c710 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f4844009fa0 tx=0x7f484400b040 comp rx=0 tx=0).ready entity=mgr.5003 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-09T13:47:30.199 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:30.196+0000 7f483e7fc640 1 -- 172.21.15.43:0/835771228 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178066 (secure 0 0 0) 0x7f48400763a0 con 0x7f484815a560 2023-08-09T13:47:30.416 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:30.414+0000 7f484fb76640 1 -- 172.21.15.43:0/835771228 --> [v2:172.21.15.203:6832/998005540,v1:172.21.15.203:6833/998005540] -- mgr_command(tid 0: {"prefix": "fs snap-schedule status", "fs": "cephfs", "path": "/snap_test_dir1/dir2", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f48480cb790 con 0x7f482c06a250 2023-08-09T13:47:30.435 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:30.434+0000 7f483e7fc640 1 -- 172.21.15.43:0/835771228 <== mgr.5003 v2:172.21.15.203:6832/998005540 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+345 (secure 0 0 0) 0x7f4844009740 con 0x7f482c06a250 2023-08-09T13:47:30.436 INFO:teuthology.orchestra.run.smithi043.stdout: 2023-08-09T13:47:30.436 INFO:teuthology.orchestra.run.smithi043.stdout:[{"fs": "cephfs", "subvol": null, "path": "/snap_test_dir1/dir2", "rel_path": "/snap_test_dir1/dir2", "schedule": "1M", "retention": {}, "start": "2023-08-09T00:00:00", "created": "2023-08-09T13:36:30", "first": "2023-08-09T13:37:00", "last": "2023-08-09T13:47:00", "last_pruned": null, "created_count": 11, "pruned_count": 0, "active": false}] 2023-08-09T13:47:30.438 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:30.437+0000 7f484fb76640 1 -- 172.21.15.43:0/835771228 >> [v2:172.21.15.203:6832/998005540,v1:172.21.15.203:6833/998005540] conn(0x7f482c06a250 msgr2=0x7f482c06c710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:47:30.439 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:30.437+0000 7f484fb76640 1 --2- 172.21.15.43:0/835771228 >> [v2:172.21.15.203:6832/998005540,v1:172.21.15.203:6833/998005540] conn(0x7f482c06a250 0x7f482c06c710 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f4844009fa0 tx=0x7f484400b040 comp rx=0 tx=0).stop 2023-08-09T13:47:30.439 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:30.437+0000 7f484fb76640 1 -- 172.21.15.43:0/835771228 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f484815a560 msgr2=0x7f48480c5cf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:47:30.439 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:30.437+0000 7f484fb76640 1 --2- 172.21.15.43:0/835771228 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f484815a560 0x7f48480c5cf0 secure :-1 s=READY pgs=692 cs=0 l=1 rev1=1 crypto rx=0x7f48400090b0 tx=0x7f48400046c0 comp rx=0 tx=0).stop 2023-08-09T13:47:30.439 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:30.437+0000 7f484fb76640 1 -- 172.21.15.43:0/835771228 shutdown_connections 2023-08-09T13:47:30.439 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:30.437+0000 7f484fb76640 1 --2- 172.21.15.43:0/835771228 >> [v2:172.21.15.203:6832/998005540,v1:172.21.15.203:6833/998005540] conn(0x7f482c06a250 0x7f482c06c710 unknown :-1 s=CLOSED pgs=55 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:47:30.439 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:30.437+0000 7f484fb76640 1 --2- 172.21.15.43:0/835771228 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f484815a560 0x7f48480c5cf0 unknown :-1 s=CLOSED pgs=692 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:47:30.440 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:30.437+0000 7f484fb76640 1 --2- 172.21.15.43:0/835771228 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f48480d0f10 0x7f4848141520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:47:30.440 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:30.437+0000 7f484fb76640 1 --2- 172.21.15.43:0/835771228 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f48480cd210 0x7f4848142e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:47:30.440 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:30.437+0000 7f484fb76640 1 -- 172.21.15.43:0/835771228 >> 172.21.15.43:0/835771228 conn(0x7f4848099600 msgr2=0x7f484806c300 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-09T13:47:30.440 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:30.437+0000 7f484fb76640 1 -- 172.21.15.43:0/835771228 shutdown_connections 2023-08-09T13:47:30.441 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:30.437+0000 7f484fb76640 1 -- 172.21.15.43:0/835771228 wait complete. 2023-08-09T13:47:30.459 DEBUG:tasks.cephfs.test_snap_schedules:res= [{"fs": "cephfs", "subvol": null, "path": "/snap_test_dir1/dir2", "rel_path": "/snap_test_dir1/dir2", "schedule": "1M", "retention": {}, "start": "2023-08-09T00:00:00", "created": "2023-08-09T13:36:30", "first": "2023-08-09T13:37:00", "last": "2023-08-09T13:47:00", "last_pruned": null, "created_count": 11, "pruned_count": 0, "active": false}] 2023-08-09T13:47:30.460 DEBUG:tasks.cephfs.test_snap_schedules:json_res: {'fs': 'cephfs', 'subvol': None, 'path': '/snap_test_dir1/dir2', 'rel_path': '/snap_test_dir1/dir2', 'schedule': '1M', 'retention': {}, 'start': '2023-08-09T00:00:00', 'created': '2023-08-09T13:36:30', 'first': '2023-08-09T13:37:00', 'last': '2023-08-09T13:47:00', 'last_pruned': None, 'created_count': 11, 'pruned_count': 0, 'active': False} 2023-08-09T13:47:30.461 DEBUG:tasks.cephfs.test_snap_schedules:fs_count: 11 2023-08-09T13:47:30.462 DEBUG:tasks.cephfs.test_snap_schedules:db_count: 11 2023-08-09T13:47:30.462 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-09T13:47:30.463 DEBUG:teuthology.orchestra.run.smithi043:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/dir3/.snap) 2023-08-09T13:47:30.529 INFO:teuthology.orchestra.run.smithi043.stdout:scheduled-2023-08-09-13_37_00_UTC 2023-08-09T13:47:30.530 INFO:teuthology.orchestra.run.smithi043.stdout:scheduled-2023-08-09-13_38_00_UTC 2023-08-09T13:47:30.530 INFO:teuthology.orchestra.run.smithi043.stdout:scheduled-2023-08-09-13_39_00_UTC 2023-08-09T13:47:30.530 INFO:teuthology.orchestra.run.smithi043.stdout:scheduled-2023-08-09-13_40_00_UTC 2023-08-09T13:47:30.531 INFO:teuthology.orchestra.run.smithi043.stdout:scheduled-2023-08-09-13_41_00_UTC 2023-08-09T13:47:30.531 INFO:teuthology.orchestra.run.smithi043.stdout:scheduled-2023-08-09-13_42_00_UTC 2023-08-09T13:47:30.531 INFO:teuthology.orchestra.run.smithi043.stdout:scheduled-2023-08-09-13_43_01_UTC 2023-08-09T13:47:30.531 INFO:teuthology.orchestra.run.smithi043.stdout:scheduled-2023-08-09-13_44_01_UTC 2023-08-09T13:47:30.532 INFO:teuthology.orchestra.run.smithi043.stdout:scheduled-2023-08-09-13_45_01_UTC 2023-08-09T13:47:30.532 INFO:teuthology.orchestra.run.smithi043.stdout:scheduled-2023-08-09-13_46_00_UTC 2023-08-09T13:47:30.532 INFO:teuthology.orchestra.run.smithi043.stdout:scheduled-2023-08-09-13_47_00_UTC 2023-08-09T13:47:30.534 DEBUG:tasks.cephfs.test_snap_schedules:snapshots: ['scheduled-2023-08-09-13_37_00_UTC', 'scheduled-2023-08-09-13_38_00_UTC', 'scheduled-2023-08-09-13_39_00_UTC', 'scheduled-2023-08-09-13_40_00_UTC', 'scheduled-2023-08-09-13_41_00_UTC', 'scheduled-2023-08-09-13_42_00_UTC', 'scheduled-2023-08-09-13_43_01_UTC', 'scheduled-2023-08-09-13_44_01_UTC', 'scheduled-2023-08-09-13_45_01_UTC', 'scheduled-2023-08-09-13_46_00_UTC', 'scheduled-2023-08-09-13_47_00_UTC'] 2023-08-09T13:47:30.535 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs snap-schedule status --fs cephfs --format json /snap_test_dir1/dir3 2023-08-09T13:47:30.773 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:30.771+0000 7f8bdfc1f640 1 Processor -- start 2023-08-09T13:47:30.773 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:30.772+0000 7f8bdfc1f640 1 -- start start 2023-08-09T13:47:30.775 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:30.773+0000 7f8bdfc1f640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f8bd8153590 0x7f8bd80cc3f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:47:30.775 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:30.773+0000 7f8bdfc1f640 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7f8bd80cebd0 0x7f8bd80ca540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:47:30.775 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:30.773+0000 7f8bdfc1f640 1 -- --> v1:172.21.15.203:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8bd80c9510 con 0x7f8bd80cbd70 2023-08-09T13:47:30.775 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:30.774+0000 7f8bdfc1f640 1 -- --> v2:172.21.15.203:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8bd80c86f0 con 0x7f8bd80cebd0 2023-08-09T13:47:30.775 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:30.774+0000 7f8bdfc1f640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8bd80c99a0 con 0x7f8bd8153590 2023-08-09T13:47:30.776 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:30.774+0000 7f8bdd9bb640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f8bd8153590 0x7f8bd80cc3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:47:30.776 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:30.774+0000 7f8bdd9bb640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f8bd8153590 0x7f8bd80cc3f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.203:3301/0 says I am v2:172.21.15.43:41868/0 (socket says 172.21.15.43:41868) 2023-08-09T13:47:30.777 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:30.774+0000 7f8bdd9bb640 1 -- 172.21.15.43:0/2474118668 learned_addr learned my addr 172.21.15.43:0/2474118668 (peer_addr_for_me v2:172.21.15.43:0/0) 2023-08-09T13:47:30.777 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:30.774+0000 7f8bde1bc640 1 --2- 172.21.15.43:0/2474118668 >> v2:172.21.15.203:3300/0 conn(0x7f8bd80cebd0 0x7f8bd80ca540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:47:30.777 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:30.775+0000 7f8bdd9bb640 1 -- 172.21.15.43:0/2474118668 >> v2:172.21.15.203:3300/0 conn(0x7f8bd80cebd0 msgr2=0x7f8bd80ca540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-09T13:47:30.777 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:30.775+0000 7f8bdd9bb640 1 --2- 172.21.15.43:0/2474118668 >> v2:172.21.15.203:3300/0 conn(0x7f8bd80cebd0 0x7f8bd80ca540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:47:30.777 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:30.775+0000 7f8bdd9bb640 1 -- 172.21.15.43:0/2474118668 >> v1:172.21.15.203:6789/0 conn(0x7f8bd80cbd70 legacy=0x7f8bd80ce4c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:47:30.777 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:30.775+0000 7f8bdd9bb640 1 -- 172.21.15.43:0/2474118668 --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8bd80c8900 con 0x7f8bd8153590 2023-08-09T13:47:30.778 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:30.777+0000 7f8bdd9bb640 1 --2- 172.21.15.43:0/2474118668 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f8bd8153590 0x7f8bd80cc3f0 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f8bd4009870 tx=0x7f8bd402b980 comp rx=0 tx=0).ready entity=mon.2 client_cookie=20c42e0bdc45d6bb server_cookie=0 in_seq=0 out_seq=0 2023-08-09T13:47:30.779 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:30.777+0000 7f8bdc9b9640 1 -- 172.21.15.43:0/2474118668 <== mon.2 v2:172.21.15.203:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8bd4030840 con 0x7f8bd8153590 2023-08-09T13:47:30.779 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:30.778+0000 7f8bdc9b9640 1 -- 172.21.15.43:0/2474118668 <== mon.2 v2:172.21.15.203:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8bd40309a0 con 0x7f8bd8153590 2023-08-09T13:47:30.779 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:30.778+0000 7f8bdc9b9640 1 -- 172.21.15.43:0/2474118668 <== mon.2 v2:172.21.15.203:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8bd4030d30 con 0x7f8bd8153590 2023-08-09T13:47:30.780 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:30.778+0000 7f8bdfc1f640 1 -- 172.21.15.43:0/2474118668 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f8bd8153590 msgr2=0x7f8bd80cc3f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:47:30.780 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:30.778+0000 7f8bdfc1f640 1 --2- 172.21.15.43:0/2474118668 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f8bd8153590 0x7f8bd80cc3f0 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f8bd4009870 tx=0x7f8bd402b980 comp rx=0 tx=0).stop 2023-08-09T13:47:30.780 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:30.778+0000 7f8bdfc1f640 1 -- 172.21.15.43:0/2474118668 shutdown_connections 2023-08-09T13:47:30.780 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:30.778+0000 7f8bdfc1f640 1 --2- 172.21.15.43:0/2474118668 >> v2:172.21.15.203:3300/0 conn(0x7f8bd80cebd0 0x7f8bd80ca540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:47:30.781 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:30.778+0000 7f8bdfc1f640 1 --2- 172.21.15.43:0/2474118668 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f8bd8153590 0x7f8bd80cc3f0 unknown :-1 s=CLOSED pgs=90 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:47:30.781 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:30.778+0000 7f8bdfc1f640 1 -- 172.21.15.43:0/2474118668 >> 172.21.15.43:0/2474118668 conn(0x7f8bd806b990 msgr2=0x7f8bd806bda0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-09T13:47:30.781 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:30.779+0000 7f8bdfc1f640 1 -- 172.21.15.43:0/2474118668 shutdown_connections 2023-08-09T13:47:30.781 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:30.779+0000 7f8bdfc1f640 1 -- 172.21.15.43:0/2474118668 wait complete. 2023-08-09T13:47:30.782 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:30.781+0000 7f8bdfc1f640 1 Processor -- start 2023-08-09T13:47:30.782 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:30.781+0000 7f8bdfc1f640 1 -- start start 2023-08-09T13:47:30.784 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:30.782+0000 7f8bdfc1f640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f8bd80cbd70 0x7f8bd814fcb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:47:30.784 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:30.782+0000 7f8bdfc1f640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f8bd80cebd0 0x7f8bd80c06e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:47:30.784 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:30.782+0000 7f8bdfc1f640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f8bd8153590 0x7f8bd81f2080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:47:30.785 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:30.782+0000 7f8bdfc1f640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8bd80c0c20 con 0x7f8bd80cebd0 2023-08-09T13:47:30.785 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:30.782+0000 7f8bdfc1f640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8bd81502e0 con 0x7f8bd80cbd70 2023-08-09T13:47:30.785 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:30.782+0000 7f8bdfc1f640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8bd80c69c0 con 0x7f8bd8153590 2023-08-09T13:47:30.785 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:30.783+0000 7f8bdd1ba640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f8bd80cebd0 0x7f8bd80c06e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:47:30.785 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:30.783+0000 7f8bdd1ba640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f8bd80cebd0 0x7f8bd80c06e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:53168/0 (socket says 172.21.15.43:53168) 2023-08-09T13:47:30.785 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:30.783+0000 7f8bdd1ba640 1 -- 172.21.15.43:0/4281329034 learned_addr learned my addr 172.21.15.43:0/4281329034 (peer_addr_for_me v2:172.21.15.43:0/0) 2023-08-09T13:47:30.786 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:30.783+0000 7f8bdd9bb640 1 --2- 172.21.15.43:0/4281329034 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f8bd80cbd70 0x7f8bd814fcb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:47:30.786 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:30.783+0000 7f8bde1bc640 1 --2- 172.21.15.43:0/4281329034 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f8bd8153590 0x7f8bd81f2080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:47:30.787 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:30.783+0000 7f8bdd1ba640 1 -- 172.21.15.43:0/4281329034 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f8bd8153590 msgr2=0x7f8bd81f2080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:47:30.787 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:30.783+0000 7f8bdd1ba640 1 --2- 172.21.15.43:0/4281329034 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f8bd8153590 0x7f8bd81f2080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:47:30.787 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:30.783+0000 7f8bdd1ba640 1 -- 172.21.15.43:0/4281329034 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f8bd80cbd70 msgr2=0x7f8bd814fcb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:47:30.787 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:30.783+0000 7f8bdd1ba640 1 --2- 172.21.15.43:0/4281329034 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f8bd80cbd70 0x7f8bd814fcb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:47:30.787 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:30.783+0000 7f8bdd1ba640 1 -- 172.21.15.43:0/4281329034 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8bd40094d0 con 0x7f8bd80cebd0 2023-08-09T13:47:30.787 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:30.784+0000 7f8bdd1ba640 1 --2- 172.21.15.43:0/4281329034 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f8bd80cebd0 0x7f8bd80c06e0 secure :-1 s=READY pgs=693 cs=0 l=1 rev1=1 crypto rx=0x7f8bc800c400 tx=0x7f8bc800c8c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-09T13:47:30.788 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:30.784+0000 7f8bce7fc640 1 -- 172.21.15.43:0/4281329034 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8bc80052d0 con 0x7f8bd80cebd0 2023-08-09T13:47:30.788 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:30.784+0000 7f8bce7fc640 1 -- 172.21.15.43:0/4281329034 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8bc8005450 con 0x7f8bd80cebd0 2023-08-09T13:47:30.788 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:30.784+0000 7f8bce7fc640 1 -- 172.21.15.43:0/4281329034 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8bc8007260 con 0x7f8bd80cebd0 2023-08-09T13:47:30.788 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:30.784+0000 7f8bdfc1f640 1 -- 172.21.15.43:0/4281329034 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8bd80c6ca0 con 0x7f8bd80cebd0 2023-08-09T13:47:30.788 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:30.784+0000 7f8bdfc1f640 1 -- 172.21.15.43:0/4281329034 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8bd80c7260 con 0x7f8bd80cebd0 2023-08-09T13:47:30.788 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:30.785+0000 7f8bdfc1f640 1 -- 172.21.15.43:0/4281329034 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8bd80c9180 con 0x7f8bd80cebd0 2023-08-09T13:47:30.789 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:30.786+0000 7f8bce7fc640 1 -- 172.21.15.43:0/4281329034 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 13) v1 ==== 88529+0+0 (secure 0 0 0) 0x7f8bc80073c0 con 0x7f8bd80cebd0 2023-08-09T13:47:30.789 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:30.787+0000 7f8bce7fc640 1 --2- 172.21.15.43:0/4281329034 >> [v2:172.21.15.203:6832/998005540,v1:172.21.15.203:6833/998005540] conn(0x7f8bbc06a250 0x7f8bbc06c710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:47:30.789 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:30.788+0000 7f8bce7fc640 1 -- 172.21.15.43:0/4281329034 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(42..42 src has 1..42) v4 ==== 6698+0+0 (secure 0 0 0) 0x7f8bc8087fe0 con 0x7f8bd80cebd0 2023-08-09T13:47:30.789 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:30.788+0000 7f8bdd9bb640 1 --2- 172.21.15.43:0/4281329034 >> [v2:172.21.15.203:6832/998005540,v1:172.21.15.203:6833/998005540] conn(0x7f8bbc06a250 0x7f8bbc06c710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:47:30.790 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:30.789+0000 7f8bdd9bb640 1 --2- 172.21.15.43:0/4281329034 >> [v2:172.21.15.203:6832/998005540,v1:172.21.15.203:6833/998005540] conn(0x7f8bbc06a250 0x7f8bbc06c710 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f8bd4036000 tx=0x7f8bd403b040 comp rx=0 tx=0).ready entity=mgr.5003 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-09T13:47:30.793 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:30.792+0000 7f8bce7fc640 1 -- 172.21.15.43:0/4281329034 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178066 (secure 0 0 0) 0x7f8bd80c9180 con 0x7f8bd80cebd0 2023-08-09T13:47:31.009 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:31.007+0000 7f8bdfc1f640 1 -- 172.21.15.43:0/4281329034 --> [v2:172.21.15.203:6832/998005540,v1:172.21.15.203:6833/998005540] -- mgr_command(tid 0: {"prefix": "fs snap-schedule status", "fs": "cephfs", "path": "/snap_test_dir1/dir3", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f8bd80c8ec0 con 0x7f8bbc06a250 2023-08-09T13:47:31.029 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:31.028+0000 7f8bce7fc640 1 -- 172.21.15.43:0/4281329034 <== mgr.5003 v2:172.21.15.203:6832/998005540 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+345 (secure 0 0 0) 0x7f8bd4030c80 con 0x7f8bbc06a250 2023-08-09T13:47:31.030 INFO:teuthology.orchestra.run.smithi043.stdout: 2023-08-09T13:47:31.030 INFO:teuthology.orchestra.run.smithi043.stdout:[{"fs": "cephfs", "subvol": null, "path": "/snap_test_dir1/dir3", "rel_path": "/snap_test_dir1/dir3", "schedule": "1M", "retention": {}, "start": "2023-08-09T00:00:00", "created": "2023-08-09T13:36:31", "first": "2023-08-09T13:37:00", "last": "2023-08-09T13:47:00", "last_pruned": null, "created_count": 11, "pruned_count": 0, "active": false}] 2023-08-09T13:47:31.031 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:31.030+0000 7f8bdfc1f640 1 -- 172.21.15.43:0/4281329034 >> [v2:172.21.15.203:6832/998005540,v1:172.21.15.203:6833/998005540] conn(0x7f8bbc06a250 msgr2=0x7f8bbc06c710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:47:31.032 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:31.030+0000 7f8bdfc1f640 1 --2- 172.21.15.43:0/4281329034 >> [v2:172.21.15.203:6832/998005540,v1:172.21.15.203:6833/998005540] conn(0x7f8bbc06a250 0x7f8bbc06c710 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f8bd4036000 tx=0x7f8bd403b040 comp rx=0 tx=0).stop 2023-08-09T13:47:31.032 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:31.030+0000 7f8bdfc1f640 1 -- 172.21.15.43:0/4281329034 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f8bd80cebd0 msgr2=0x7f8bd80c06e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:47:31.032 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:31.030+0000 7f8bdfc1f640 1 --2- 172.21.15.43:0/4281329034 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f8bd80cebd0 0x7f8bd80c06e0 secure :-1 s=READY pgs=693 cs=0 l=1 rev1=1 crypto rx=0x7f8bc800c400 tx=0x7f8bc800c8c0 comp rx=0 tx=0).stop 2023-08-09T13:47:31.033 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:31.031+0000 7f8bdfc1f640 1 -- 172.21.15.43:0/4281329034 shutdown_connections 2023-08-09T13:47:31.033 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:31.031+0000 7f8bdfc1f640 1 --2- 172.21.15.43:0/4281329034 >> [v2:172.21.15.203:6832/998005540,v1:172.21.15.203:6833/998005540] conn(0x7f8bbc06a250 0x7f8bbc06c710 unknown :-1 s=CLOSED pgs=56 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:47:31.033 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:31.031+0000 7f8bdfc1f640 1 --2- 172.21.15.43:0/4281329034 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f8bd8153590 0x7f8bd81f2080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:47:31.033 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:31.031+0000 7f8bdfc1f640 1 --2- 172.21.15.43:0/4281329034 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f8bd80cebd0 0x7f8bd80c06e0 unknown :-1 s=CLOSED pgs=693 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:47:31.033 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:31.031+0000 7f8bdfc1f640 1 --2- 172.21.15.43:0/4281329034 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f8bd80cbd70 0x7f8bd814fcb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:47:31.033 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:31.031+0000 7f8bdfc1f640 1 -- 172.21.15.43:0/4281329034 >> 172.21.15.43:0/4281329034 conn(0x7f8bd806b990 msgr2=0x7f8bd815dd20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-09T13:47:31.034 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:31.031+0000 7f8bdfc1f640 1 -- 172.21.15.43:0/4281329034 shutdown_connections 2023-08-09T13:47:31.034 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:31.031+0000 7f8bdfc1f640 1 -- 172.21.15.43:0/4281329034 wait complete. 2023-08-09T13:47:31.053 DEBUG:tasks.cephfs.test_snap_schedules:res= [{"fs": "cephfs", "subvol": null, "path": "/snap_test_dir1/dir3", "rel_path": "/snap_test_dir1/dir3", "schedule": "1M", "retention": {}, "start": "2023-08-09T00:00:00", "created": "2023-08-09T13:36:31", "first": "2023-08-09T13:37:00", "last": "2023-08-09T13:47:00", "last_pruned": null, "created_count": 11, "pruned_count": 0, "active": false}] 2023-08-09T13:47:31.053 DEBUG:tasks.cephfs.test_snap_schedules:json_res: {'fs': 'cephfs', 'subvol': None, 'path': '/snap_test_dir1/dir3', 'rel_path': '/snap_test_dir1/dir3', 'schedule': '1M', 'retention': {}, 'start': '2023-08-09T00:00:00', 'created': '2023-08-09T13:36:31', 'first': '2023-08-09T13:37:00', 'last': '2023-08-09T13:47:00', 'last_pruned': None, 'created_count': 11, 'pruned_count': 0, 'active': False} 2023-08-09T13:47:31.054 DEBUG:tasks.cephfs.test_snap_schedules:fs_count: 11 2023-08-09T13:47:31.054 DEBUG:tasks.cephfs.test_snap_schedules:db_count: 11 2023-08-09T13:47:31.054 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-09T13:47:31.054 DEBUG:teuthology.orchestra.run.smithi043:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/dir4/.snap) 2023-08-09T13:47:31.117 INFO:teuthology.orchestra.run.smithi043.stdout:scheduled-2023-08-09-13_37_01_UTC 2023-08-09T13:47:31.118 INFO:teuthology.orchestra.run.smithi043.stdout:scheduled-2023-08-09-13_38_00_UTC 2023-08-09T13:47:31.118 INFO:teuthology.orchestra.run.smithi043.stdout:scheduled-2023-08-09-13_39_00_UTC 2023-08-09T13:47:31.118 INFO:teuthology.orchestra.run.smithi043.stdout:scheduled-2023-08-09-13_40_00_UTC 2023-08-09T13:47:31.118 INFO:teuthology.orchestra.run.smithi043.stdout:scheduled-2023-08-09-13_41_00_UTC 2023-08-09T13:47:31.118 INFO:teuthology.orchestra.run.smithi043.stdout:scheduled-2023-08-09-13_42_00_UTC 2023-08-09T13:47:31.118 INFO:teuthology.orchestra.run.smithi043.stdout:scheduled-2023-08-09-13_43_00_UTC 2023-08-09T13:47:31.118 INFO:teuthology.orchestra.run.smithi043.stdout:scheduled-2023-08-09-13_44_00_UTC 2023-08-09T13:47:31.119 INFO:teuthology.orchestra.run.smithi043.stdout:scheduled-2023-08-09-13_45_00_UTC 2023-08-09T13:47:31.119 INFO:teuthology.orchestra.run.smithi043.stdout:scheduled-2023-08-09-13_46_00_UTC 2023-08-09T13:47:31.119 INFO:teuthology.orchestra.run.smithi043.stdout:scheduled-2023-08-09-13_47_00_UTC 2023-08-09T13:47:31.120 DEBUG:tasks.cephfs.test_snap_schedules:snapshots: ['scheduled-2023-08-09-13_37_01_UTC', 'scheduled-2023-08-09-13_38_00_UTC', 'scheduled-2023-08-09-13_39_00_UTC', 'scheduled-2023-08-09-13_40_00_UTC', 'scheduled-2023-08-09-13_41_00_UTC', 'scheduled-2023-08-09-13_42_00_UTC', 'scheduled-2023-08-09-13_43_00_UTC', 'scheduled-2023-08-09-13_44_00_UTC', 'scheduled-2023-08-09-13_45_00_UTC', 'scheduled-2023-08-09-13_46_00_UTC', 'scheduled-2023-08-09-13_47_00_UTC'] 2023-08-09T13:47:31.120 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs snap-schedule status --fs cephfs --format json /snap_test_dir1/dir4 2023-08-09T13:47:31.348 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:31.347+0000 7f0563924640 1 Processor -- start 2023-08-09T13:47:31.348 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:31.347+0000 7f0563924640 1 -- start start 2023-08-09T13:47:31.350 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:31.348+0000 7f0563924640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f055c0b80a0 0x7f055c0b5470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:47:31.350 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:31.349+0000 7f0563924640 1 -- --> v1:172.21.15.43:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f055c0d1890 con 0x7f055c15c6f0 2023-08-09T13:47:31.350 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:31.349+0000 7f0563924640 1 -- --> v1:172.21.15.203:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f055c0d0c80 con 0x7f055c0b59b0 2023-08-09T13:47:31.350 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:31.349+0000 7f0563924640 1 -- --> v2:172.21.15.43:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f055c15cad0 con 0x7f055c0b80a0 2023-08-09T13:47:31.351 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:31.349+0000 7f0560ebf640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f055c0b80a0 0x7f055c0b5470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:47:31.351 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:31.349+0000 7f05616c0640 1 --1- >> v1:172.21.15.43:6789/0 conn(0x7f055c15c6f0 0x7f055c0d03a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.43:6789/0 says I am v1:172.21.15.43:49590/0 (socket says 172.21.15.43:49590) 2023-08-09T13:47:31.351 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:31.349+0000 7f05616c0640 1 -- 172.21.15.43:0/3291882961 learned_addr learned my addr 172.21.15.43:0/3291882961 (peer_addr_for_me v1:172.21.15.43:0/0) 2023-08-09T13:47:31.351 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:31.349+0000 7f0560ebf640 1 -- 172.21.15.43:0/3291882961 >> v1:172.21.15.203:6789/0 conn(0x7f055c0b59b0 legacy=0x7f055c0d1d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-09T13:47:31.351 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:31.349+0000 7f0560ebf640 1 -- 172.21.15.43:0/3291882961 >> v1:172.21.15.43:6789/0 conn(0x7f055c15c6f0 legacy=0x7f055c0d03a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:47:31.352 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:31.349+0000 7f0560ebf640 1 -- 172.21.15.43:0/3291882961 --> v2:172.21.15.43:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f055c0d0f60 con 0x7f055c0b80a0 2023-08-09T13:47:31.352 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:31.350+0000 7f0560ebf640 1 --2- 172.21.15.43:0/3291882961 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f055c0b80a0 0x7f055c0b5470 secure :-1 s=READY pgs=695 cs=0 l=1 rev1=1 crypto rx=0x7f054c0097a0 tx=0x7f054c02b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=42fd3961f190c5fe server_cookie=0 in_seq=0 out_seq=0 2023-08-09T13:47:31.352 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:31.351+0000 7f0553fff640 1 -- 172.21.15.43:0/3291882961 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f054c031830 con 0x7f055c0b80a0 2023-08-09T13:47:31.353 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:31.351+0000 7f0553fff640 1 -- 172.21.15.43:0/3291882961 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f054c031990 con 0x7f055c0b80a0 2023-08-09T13:47:31.353 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:31.351+0000 7f0553fff640 1 -- 172.21.15.43:0/3291882961 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f054c031d20 con 0x7f055c0b80a0 2023-08-09T13:47:31.353 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:31.351+0000 7f0563924640 1 -- 172.21.15.43:0/3291882961 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f055c0b80a0 msgr2=0x7f055c0b5470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:47:31.354 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:31.351+0000 7f0563924640 1 --2- 172.21.15.43:0/3291882961 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f055c0b80a0 0x7f055c0b5470 secure :-1 s=READY pgs=695 cs=0 l=1 rev1=1 crypto rx=0x7f054c0097a0 tx=0x7f054c02b8d0 comp rx=0 tx=0).stop 2023-08-09T13:47:31.354 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:31.352+0000 7f0563924640 1 -- 172.21.15.43:0/3291882961 shutdown_connections 2023-08-09T13:47:31.354 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:31.352+0000 7f0563924640 1 --2- 172.21.15.43:0/3291882961 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f055c0b80a0 0x7f055c0b5470 unknown :-1 s=CLOSED pgs=695 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:47:31.354 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:31.352+0000 7f0563924640 1 -- 172.21.15.43:0/3291882961 >> 172.21.15.43:0/3291882961 conn(0x7f055c0712e0 msgr2=0x7f055c0716f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-09T13:47:31.354 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:31.352+0000 7f0563924640 1 -- 172.21.15.43:0/3291882961 shutdown_connections 2023-08-09T13:47:31.354 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:31.352+0000 7f0563924640 1 -- 172.21.15.43:0/3291882961 wait complete. 2023-08-09T13:47:31.355 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:31.354+0000 7f0563924640 1 Processor -- start 2023-08-09T13:47:31.356 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:31.354+0000 7f0563924640 1 -- start start 2023-08-09T13:47:31.357 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:31.356+0000 7f0563924640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f055c0b59b0 0x7f055c14a8c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:47:31.358 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:31.356+0000 7f0563924640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f055c0b80a0 0x7f055c146c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:47:31.358 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:31.356+0000 7f0563924640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f055c15c6f0 0x7f055c147140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:47:31.358 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:31.356+0000 7f0563924640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f055c0a8ff0 con 0x7f055c0b80a0 2023-08-09T13:47:31.358 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:31.356+0000 7f0563924640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f055c15ab20 con 0x7f055c15c6f0 2023-08-09T13:47:31.358 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:31.356+0000 7f0563924640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f055c144c80 con 0x7f055c0b59b0 2023-08-09T13:47:31.358 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:31.356+0000 7f0560ebf640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f055c0b80a0 0x7f055c146c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:47:31.358 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:31.356+0000 7f0560ebf640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f055c0b80a0 0x7f055c146c00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:53190/0 (socket says 172.21.15.43:53190) 2023-08-09T13:47:31.359 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:31.356+0000 7f0560ebf640 1 -- 172.21.15.43:0/135477140 learned_addr learned my addr 172.21.15.43:0/135477140 (peer_addr_for_me v2:172.21.15.43:0/0) 2023-08-09T13:47:31.359 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:31.357+0000 7f05616c0640 1 --2- 172.21.15.43:0/135477140 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f055c0b59b0 0x7f055c14a8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:47:31.360 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:31.357+0000 7f0560ebf640 1 -- 172.21.15.43:0/135477140 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f055c0b59b0 msgr2=0x7f055c14a8c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:47:31.360 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:31.357+0000 7f0560ebf640 1 --2- 172.21.15.43:0/135477140 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f055c0b59b0 0x7f055c14a8c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:47:31.360 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:31.357+0000 7f0561ec1640 1 --2- 172.21.15.43:0/135477140 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f055c15c6f0 0x7f055c147140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:47:31.360 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:31.357+0000 7f0560ebf640 1 -- 172.21.15.43:0/135477140 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f055c15c6f0 msgr2=0x7f055c147140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:47:31.360 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:31.357+0000 7f0560ebf640 1 --2- 172.21.15.43:0/135477140 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f055c15c6f0 0x7f055c147140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:47:31.360 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:31.357+0000 7f0560ebf640 1 -- 172.21.15.43:0/135477140 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f054c009400 con 0x7f055c0b80a0 2023-08-09T13:47:31.361 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:31.357+0000 7f0560ebf640 1 --2- 172.21.15.43:0/135477140 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f055c0b80a0 0x7f055c146c00 secure :-1 s=READY pgs=696 cs=0 l=1 rev1=1 crypto rx=0x7f054c02bee0 tx=0x7f054c0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-09T13:47:31.361 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:31.357+0000 7f0551ffb640 1 -- 172.21.15.43:0/135477140 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f054c03ba70 con 0x7f055c0b80a0 2023-08-09T13:47:31.361 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:31.357+0000 7f0551ffb640 1 -- 172.21.15.43:0/135477140 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f054c03bbf0 con 0x7f055c0b80a0 2023-08-09T13:47:31.361 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:31.357+0000 7f0551ffb640 1 -- 172.21.15.43:0/135477140 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f054c0433f0 con 0x7f055c0b80a0 2023-08-09T13:47:31.361 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:31.358+0000 7f0563924640 1 -- 172.21.15.43:0/135477140 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f055c144f60 con 0x7f055c0b80a0 2023-08-09T13:47:31.361 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:31.358+0000 7f0563924640 1 -- 172.21.15.43:0/135477140 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f055c14ae00 con 0x7f055c0b80a0 2023-08-09T13:47:31.362 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:31.359+0000 7f0563924640 1 -- 172.21.15.43:0/135477140 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f055c14af90 con 0x7f055c0b80a0 2023-08-09T13:47:31.362 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:31.360+0000 7f0551ffb640 1 -- 172.21.15.43:0/135477140 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 13) v1 ==== 88529+0+0 (secure 0 0 0) 0x7f054c04c3b0 con 0x7f055c0b80a0 2023-08-09T13:47:31.362 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:31.361+0000 7f0551ffb640 1 --2- 172.21.15.43:0/135477140 >> [v2:172.21.15.203:6832/998005540,v1:172.21.15.203:6833/998005540] conn(0x7f054006a220 0x7f054006c6e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:47:31.363 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:31.361+0000 7f0551ffb640 1 -- 172.21.15.43:0/135477140 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(42..42 src has 1..42) v4 ==== 6698+0+0 (secure 0 0 0) 0x7f054c0aae70 con 0x7f055c0b80a0 2023-08-09T13:47:31.363 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:31.362+0000 7f05616c0640 1 --2- 172.21.15.43:0/135477140 >> [v2:172.21.15.203:6832/998005540,v1:172.21.15.203:6833/998005540] conn(0x7f054006a220 0x7f054006c6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:47:31.364 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:31.363+0000 7f05616c0640 1 --2- 172.21.15.43:0/135477140 >> [v2:172.21.15.203:6832/998005540,v1:172.21.15.203:6833/998005540] conn(0x7f054006a220 0x7f054006c6e0 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f0558005880 tx=0x7f0558009040 comp rx=0 tx=0).ready entity=mgr.5003 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-09T13:47:31.367 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:31.365+0000 7f0551ffb640 1 -- 172.21.15.43:0/135477140 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178066 (secure 0 0 0) 0x7f054c075fd0 con 0x7f055c0b80a0 2023-08-09T13:47:31.582 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:31.580+0000 7f0563924640 1 -- 172.21.15.43:0/135477140 --> [v2:172.21.15.203:6832/998005540,v1:172.21.15.203:6833/998005540] -- mgr_command(tid 0: {"prefix": "fs snap-schedule status", "fs": "cephfs", "path": "/snap_test_dir1/dir4", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f055c0d1520 con 0x7f054006a220 2023-08-09T13:47:31.602 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:31.601+0000 7f0551ffb640 1 -- 172.21.15.43:0/135477140 <== mgr.5003 v2:172.21.15.203:6832/998005540 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+345 (secure 0 0 0) 0x7f05580058b0 con 0x7f054006a220 2023-08-09T13:47:31.603 INFO:teuthology.orchestra.run.smithi043.stdout: 2023-08-09T13:47:31.603 INFO:teuthology.orchestra.run.smithi043.stdout:[{"fs": "cephfs", "subvol": null, "path": "/snap_test_dir1/dir4", "rel_path": "/snap_test_dir1/dir4", "schedule": "1M", "retention": {}, "start": "2023-08-09T00:00:00", "created": "2023-08-09T13:36:31", "first": "2023-08-09T13:37:01", "last": "2023-08-09T13:47:00", "last_pruned": null, "created_count": 11, "pruned_count": 0, "active": false}] 2023-08-09T13:47:31.605 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:31.604+0000 7f0563924640 1 -- 172.21.15.43:0/135477140 >> [v2:172.21.15.203:6832/998005540,v1:172.21.15.203:6833/998005540] conn(0x7f054006a220 msgr2=0x7f054006c6e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:47:31.606 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:31.604+0000 7f0563924640 1 --2- 172.21.15.43:0/135477140 >> [v2:172.21.15.203:6832/998005540,v1:172.21.15.203:6833/998005540] conn(0x7f054006a220 0x7f054006c6e0 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f0558005880 tx=0x7f0558009040 comp rx=0 tx=0).stop 2023-08-09T13:47:31.606 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:31.604+0000 7f0563924640 1 -- 172.21.15.43:0/135477140 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f055c0b80a0 msgr2=0x7f055c146c00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:47:31.606 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:31.604+0000 7f0563924640 1 --2- 172.21.15.43:0/135477140 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f055c0b80a0 0x7f055c146c00 secure :-1 s=READY pgs=696 cs=0 l=1 rev1=1 crypto rx=0x7f054c02bee0 tx=0x7f054c0046c0 comp rx=0 tx=0).stop 2023-08-09T13:47:31.606 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:31.604+0000 7f0563924640 1 -- 172.21.15.43:0/135477140 shutdown_connections 2023-08-09T13:47:31.606 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:31.604+0000 7f0563924640 1 --2- 172.21.15.43:0/135477140 >> [v2:172.21.15.203:6832/998005540,v1:172.21.15.203:6833/998005540] conn(0x7f054006a220 0x7f054006c6e0 unknown :-1 s=CLOSED pgs=57 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:47:31.606 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:31.604+0000 7f0563924640 1 --2- 172.21.15.43:0/135477140 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f055c15c6f0 0x7f055c147140 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:47:31.607 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:31.604+0000 7f0563924640 1 --2- 172.21.15.43:0/135477140 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f055c0b80a0 0x7f055c146c00 unknown :-1 s=CLOSED pgs=696 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:47:31.607 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:31.604+0000 7f0563924640 1 --2- 172.21.15.43:0/135477140 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f055c0b59b0 0x7f055c14a8c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:47:31.607 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:31.604+0000 7f0563924640 1 -- 172.21.15.43:0/135477140 >> 172.21.15.43:0/135477140 conn(0x7f055c0712e0 msgr2=0x7f055c15eaf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-09T13:47:31.607 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:31.604+0000 7f0563924640 1 -- 172.21.15.43:0/135477140 shutdown_connections 2023-08-09T13:47:31.608 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:31.604+0000 7f0563924640 1 -- 172.21.15.43:0/135477140 wait complete. 2023-08-09T13:47:31.626 DEBUG:tasks.cephfs.test_snap_schedules:res= [{"fs": "cephfs", "subvol": null, "path": "/snap_test_dir1/dir4", "rel_path": "/snap_test_dir1/dir4", "schedule": "1M", "retention": {}, "start": "2023-08-09T00:00:00", "created": "2023-08-09T13:36:31", "first": "2023-08-09T13:37:01", "last": "2023-08-09T13:47:00", "last_pruned": null, "created_count": 11, "pruned_count": 0, "active": false}] 2023-08-09T13:47:31.627 DEBUG:tasks.cephfs.test_snap_schedules:json_res: {'fs': 'cephfs', 'subvol': None, 'path': '/snap_test_dir1/dir4', 'rel_path': '/snap_test_dir1/dir4', 'schedule': '1M', 'retention': {}, 'start': '2023-08-09T00:00:00', 'created': '2023-08-09T13:36:31', 'first': '2023-08-09T13:37:01', 'last': '2023-08-09T13:47:00', 'last_pruned': None, 'created_count': 11, 'pruned_count': 0, 'active': False} 2023-08-09T13:47:31.627 DEBUG:tasks.cephfs.test_snap_schedules:fs_count: 11 2023-08-09T13:47:31.627 DEBUG:tasks.cephfs.test_snap_schedules:db_count: 11 2023-08-09T13:47:31.628 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-09T13:47:31.628 DEBUG:teuthology.orchestra.run.smithi043:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/dir5/.snap) 2023-08-09T13:47:31.698 INFO:teuthology.orchestra.run.smithi043.stdout:scheduled-2023-08-09-13_37_00_UTC 2023-08-09T13:47:31.698 INFO:teuthology.orchestra.run.smithi043.stdout:scheduled-2023-08-09-13_38_00_UTC 2023-08-09T13:47:31.698 INFO:teuthology.orchestra.run.smithi043.stdout:scheduled-2023-08-09-13_39_00_UTC 2023-08-09T13:47:31.698 INFO:teuthology.orchestra.run.smithi043.stdout:scheduled-2023-08-09-13_40_00_UTC 2023-08-09T13:47:31.698 INFO:teuthology.orchestra.run.smithi043.stdout:scheduled-2023-08-09-13_41_00_UTC 2023-08-09T13:47:31.698 INFO:teuthology.orchestra.run.smithi043.stdout:scheduled-2023-08-09-13_42_00_UTC 2023-08-09T13:47:31.699 INFO:teuthology.orchestra.run.smithi043.stdout:scheduled-2023-08-09-13_43_00_UTC 2023-08-09T13:47:31.699 INFO:teuthology.orchestra.run.smithi043.stdout:scheduled-2023-08-09-13_44_00_UTC 2023-08-09T13:47:31.699 INFO:teuthology.orchestra.run.smithi043.stdout:scheduled-2023-08-09-13_45_01_UTC 2023-08-09T13:47:31.699 INFO:teuthology.orchestra.run.smithi043.stdout:scheduled-2023-08-09-13_46_01_UTC 2023-08-09T13:47:31.699 INFO:teuthology.orchestra.run.smithi043.stdout:scheduled-2023-08-09-13_47_00_UTC 2023-08-09T13:47:31.700 DEBUG:tasks.cephfs.test_snap_schedules:snapshots: ['scheduled-2023-08-09-13_37_00_UTC', 'scheduled-2023-08-09-13_38_00_UTC', 'scheduled-2023-08-09-13_39_00_UTC', 'scheduled-2023-08-09-13_40_00_UTC', 'scheduled-2023-08-09-13_41_00_UTC', 'scheduled-2023-08-09-13_42_00_UTC', 'scheduled-2023-08-09-13_43_00_UTC', 'scheduled-2023-08-09-13_44_00_UTC', 'scheduled-2023-08-09-13_45_01_UTC', 'scheduled-2023-08-09-13_46_01_UTC', 'scheduled-2023-08-09-13_47_00_UTC'] 2023-08-09T13:47:31.700 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs snap-schedule status --fs cephfs --format json /snap_test_dir1/dir5 2023-08-09T13:47:31.943 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:31.942+0000 7f7235e1f640 1 Processor -- start 2023-08-09T13:47:31.944 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:31.942+0000 7f7235e1f640 1 -- start start 2023-08-09T13:47:31.945 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:31.943+0000 7f7235e1f640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f723015d5b0 0x7f7230069e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:47:31.945 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:31.943+0000 7f7235e1f640 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7f723006a350 0x7f72300ce4c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:47:31.945 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:31.943+0000 7f7235e1f640 1 -- --> v1:172.21.15.203:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f72300ca410 con 0x7f7230069a30 2023-08-09T13:47:31.945 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:31.943+0000 7f7235e1f640 1 -- --> v2:172.21.15.203:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7230159e70 con 0x7f723006a350 2023-08-09T13:47:31.946 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:31.943+0000 7f7235e1f640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f723015ce80 con 0x7f723015d5b0 2023-08-09T13:47:31.946 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:31.944+0000 7f7227fff640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f723015d5b0 0x7f7230069e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:47:31.947 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:31.944+0000 7f7227fff640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f723015d5b0 0x7f7230069e10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.203:3301/0 says I am v2:172.21.15.43:41906/0 (socket says 172.21.15.43:41906) 2023-08-09T13:47:31.947 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:31.944+0000 7f7227fff640 1 -- 172.21.15.43:0/4051587044 learned_addr learned my addr 172.21.15.43:0/4051587044 (peer_addr_for_me v2:172.21.15.43:0/0) 2023-08-09T13:47:31.947 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:31.944+0000 7f722ffff640 1 --2- 172.21.15.43:0/4051587044 >> v2:172.21.15.203:3300/0 conn(0x7f723006a350 0x7f72300ce4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:47:31.947 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:31.945+0000 7f7227fff640 1 -- 172.21.15.43:0/4051587044 >> v2:172.21.15.203:3300/0 conn(0x7f723006a350 msgr2=0x7f72300ce4c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-09T13:47:31.947 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:31.945+0000 7f7227fff640 1 --2- 172.21.15.43:0/4051587044 >> v2:172.21.15.203:3300/0 conn(0x7f723006a350 0x7f72300ce4c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:47:31.947 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:31.945+0000 7f7227fff640 1 -- 172.21.15.43:0/4051587044 >> v1:172.21.15.203:6789/0 conn(0x7f7230069a30 legacy=0x7f7230159760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:47:31.947 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:31.945+0000 7f7227fff640 1 -- 172.21.15.43:0/4051587044 --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f723015a050 con 0x7f723015d5b0 2023-08-09T13:47:31.948 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:31.946+0000 7f7227fff640 1 --2- 172.21.15.43:0/4051587044 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f723015d5b0 0x7f7230069e10 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f721800bb10 tx=0x7f721802e0c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=1ba0842d2009dba5 server_cookie=0 in_seq=0 out_seq=0 2023-08-09T13:47:31.948 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:31.947+0000 7f722effd640 1 -- 172.21.15.43:0/4051587044 <== mon.2 v2:172.21.15.203:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f721802ee70 con 0x7f723015d5b0 2023-08-09T13:47:31.948 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:31.947+0000 7f722effd640 1 -- 172.21.15.43:0/4051587044 <== mon.2 v2:172.21.15.203:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7218036440 con 0x7f723015d5b0 2023-08-09T13:47:31.949 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:31.947+0000 7f722effd640 1 -- 172.21.15.43:0/4051587044 <== mon.2 v2:172.21.15.203:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f72180367d0 con 0x7f723015d5b0 2023-08-09T13:47:31.949 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:31.947+0000 7f7235e1f640 1 -- 172.21.15.43:0/4051587044 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f723015d5b0 msgr2=0x7f7230069e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:47:31.949 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:31.947+0000 7f7235e1f640 1 --2- 172.21.15.43:0/4051587044 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f723015d5b0 0x7f7230069e10 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f721800bb10 tx=0x7f721802e0c0 comp rx=0 tx=0).stop 2023-08-09T13:47:31.950 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:31.947+0000 7f7235e1f640 1 -- 172.21.15.43:0/4051587044 shutdown_connections 2023-08-09T13:47:31.950 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:31.947+0000 7f7235e1f640 1 --2- 172.21.15.43:0/4051587044 >> v2:172.21.15.203:3300/0 conn(0x7f723006a350 0x7f72300ce4c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:47:31.950 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:31.947+0000 7f7235e1f640 1 --2- 172.21.15.43:0/4051587044 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f723015d5b0 0x7f7230069e10 unknown :-1 s=CLOSED pgs=91 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:47:31.950 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:31.947+0000 7f7235e1f640 1 -- 172.21.15.43:0/4051587044 >> 172.21.15.43:0/4051587044 conn(0x7f723006b990 msgr2=0x7f723006bda0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-09T13:47:31.950 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:31.948+0000 7f7235e1f640 1 -- 172.21.15.43:0/4051587044 shutdown_connections 2023-08-09T13:47:31.950 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:31.948+0000 7f7235e1f640 1 -- 172.21.15.43:0/4051587044 wait complete. 2023-08-09T13:47:31.951 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:31.949+0000 7f7235e1f640 1 Processor -- start 2023-08-09T13:47:31.951 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:31.949+0000 7f7235e1f640 1 -- start start 2023-08-09T13:47:31.952 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:31.950+0000 7f7235e1f640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f7230069a30 0x7f72300c0a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:47:31.952 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:31.950+0000 7f7235e1f640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f723006a350 0x7f72300c3da0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:47:31.952 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:31.950+0000 7f7235e1f640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f723015d5b0 0x7f72300c42e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:47:31.952 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:31.950+0000 7f7235e1f640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f723014a1a0 con 0x7f723015d5b0 2023-08-09T13:47:31.953 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:31.950+0000 7f7235e1f640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f723014a480 con 0x7f7230069a30 2023-08-09T13:47:31.953 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:31.950+0000 7f7235e1f640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f723014a760 con 0x7f723006a350 2023-08-09T13:47:31.953 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:31.951+0000 7f722ffff640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f723015d5b0 0x7f72300c42e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:47:31.953 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:31.951+0000 7f722ffff640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f723015d5b0 0x7f72300c42e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:53206/0 (socket says 172.21.15.43:53206) 2023-08-09T13:47:31.953 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:31.951+0000 7f722ffff640 1 -- 172.21.15.43:0/3111193637 learned_addr learned my addr 172.21.15.43:0/3111193637 (peer_addr_for_me v2:172.21.15.43:0/0) 2023-08-09T13:47:31.955 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:31.951+0000 7f7227fff640 1 --2- 172.21.15.43:0/3111193637 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f723006a350 0x7f72300c3da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:47:31.956 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:31.951+0000 7f722f7fe640 1 --2- 172.21.15.43:0/3111193637 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f7230069a30 0x7f72300c0a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:47:31.956 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:31.951+0000 7f722ffff640 1 -- 172.21.15.43:0/3111193637 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f723006a350 msgr2=0x7f72300c3da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:47:31.956 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:31.951+0000 7f722ffff640 1 --2- 172.21.15.43:0/3111193637 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f723006a350 0x7f72300c3da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:47:31.956 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:31.951+0000 7f722ffff640 1 -- 172.21.15.43:0/3111193637 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f7230069a30 msgr2=0x7f72300c0a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:47:31.956 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:31.951+0000 7f722ffff640 1 --2- 172.21.15.43:0/3111193637 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f7230069a30 0x7f72300c0a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:47:31.957 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:31.951+0000 7f722ffff640 1 -- 172.21.15.43:0/3111193637 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f721800b770 con 0x7f723015d5b0 2023-08-09T13:47:31.957 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:31.951+0000 7f722f7fe640 1 --2- 172.21.15.43:0/3111193637 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f7230069a30 0x7f72300c0a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-08-09T13:47:31.957 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:31.952+0000 7f722ffff640 1 --2- 172.21.15.43:0/3111193637 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f723015d5b0 0x7f72300c42e0 secure :-1 s=READY pgs=697 cs=0 l=1 rev1=1 crypto rx=0x7f722800efc0 tx=0x7f722800c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-09T13:47:31.957 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:31.952+0000 7f722cff9640 1 -- 172.21.15.43:0/3111193637 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7228009ca0 con 0x7f723015d5b0 2023-08-09T13:47:31.958 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:31.952+0000 7f722cff9640 1 -- 172.21.15.43:0/3111193637 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f722800f040 con 0x7f723015d5b0 2023-08-09T13:47:31.958 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:31.952+0000 7f722cff9640 1 -- 172.21.15.43:0/3111193637 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f72280146c0 con 0x7f723015d5b0 2023-08-09T13:47:31.958 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:31.952+0000 7f7235e1f640 1 -- 172.21.15.43:0/3111193637 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f723014aa40 con 0x7f723015d5b0 2023-08-09T13:47:31.958 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:31.952+0000 7f7235e1f640 1 -- 172.21.15.43:0/3111193637 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f72300c0f60 con 0x7f723015d5b0 2023-08-09T13:47:31.958 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:31.953+0000 7f7235e1f640 1 -- 172.21.15.43:0/3111193637 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f723014af60 con 0x7f723015d5b0 2023-08-09T13:47:31.959 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:31.954+0000 7f722cff9640 1 -- 172.21.15.43:0/3111193637 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 13) v1 ==== 88529+0+0 (secure 0 0 0) 0x7f7228014820 con 0x7f723015d5b0 2023-08-09T13:47:31.959 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:31.955+0000 7f722cff9640 1 --2- 172.21.15.43:0/3111193637 >> [v2:172.21.15.203:6832/998005540,v1:172.21.15.203:6833/998005540] conn(0x7f721006a280 0x7f721006c740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:47:31.959 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:31.955+0000 7f722cff9640 1 -- 172.21.15.43:0/3111193637 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(42..42 src has 1..42) v4 ==== 6698+0+0 (secure 0 0 0) 0x7f7228089bc0 con 0x7f723015d5b0 2023-08-09T13:47:31.960 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:31.956+0000 7f722f7fe640 1 --2- 172.21.15.43:0/3111193637 >> [v2:172.21.15.203:6832/998005540,v1:172.21.15.203:6833/998005540] conn(0x7f721006a280 0x7f721006c740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:47:31.960 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:31.957+0000 7f722f7fe640 1 --2- 172.21.15.43:0/3111193637 >> [v2:172.21.15.203:6832/998005540,v1:172.21.15.203:6833/998005540] conn(0x7f721006a280 0x7f721006c740 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f72300ca1d0 tx=0x7f7220009040 comp rx=0 tx=0).ready entity=mgr.5003 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-09T13:47:31.960 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:31.959+0000 7f722cff9640 1 -- 172.21.15.43:0/3111193637 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178066 (secure 0 0 0) 0x7f723014af60 con 0x7f723015d5b0 2023-08-09T13:47:32.175 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:32.174+0000 7f7235e1f640 1 -- 172.21.15.43:0/3111193637 --> [v2:172.21.15.203:6832/998005540,v1:172.21.15.203:6833/998005540] -- mgr_command(tid 0: {"prefix": "fs snap-schedule status", "fs": "cephfs", "path": "/snap_test_dir1/dir5", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f723014a5f0 con 0x7f721006a280 2023-08-09T13:47:32.196 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:32.194+0000 7f722cff9640 1 -- 172.21.15.43:0/3111193637 <== mgr.5003 v2:172.21.15.203:6832/998005540 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+345 (secure 0 0 0) 0x7f723014a5f0 con 0x7f721006a280 2023-08-09T13:47:32.196 INFO:teuthology.orchestra.run.smithi043.stdout: 2023-08-09T13:47:32.197 INFO:teuthology.orchestra.run.smithi043.stdout:[{"fs": "cephfs", "subvol": null, "path": "/snap_test_dir1/dir5", "rel_path": "/snap_test_dir1/dir5", "schedule": "1M", "retention": {}, "start": "2023-08-09T00:00:00", "created": "2023-08-09T13:36:32", "first": "2023-08-09T13:37:00", "last": "2023-08-09T13:47:00", "last_pruned": null, "created_count": 11, "pruned_count": 0, "active": false}] 2023-08-09T13:47:32.198 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:32.197+0000 7f7235e1f640 1 -- 172.21.15.43:0/3111193637 >> [v2:172.21.15.203:6832/998005540,v1:172.21.15.203:6833/998005540] conn(0x7f721006a280 msgr2=0x7f721006c740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:47:32.198 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:32.197+0000 7f7235e1f640 1 --2- 172.21.15.43:0/3111193637 >> [v2:172.21.15.203:6832/998005540,v1:172.21.15.203:6833/998005540] conn(0x7f721006a280 0x7f721006c740 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f72300ca1d0 tx=0x7f7220009040 comp rx=0 tx=0).stop 2023-08-09T13:47:32.199 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:32.197+0000 7f7235e1f640 1 -- 172.21.15.43:0/3111193637 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f723015d5b0 msgr2=0x7f72300c42e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:47:32.199 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:32.197+0000 7f7235e1f640 1 --2- 172.21.15.43:0/3111193637 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f723015d5b0 0x7f72300c42e0 secure :-1 s=READY pgs=697 cs=0 l=1 rev1=1 crypto rx=0x7f722800efc0 tx=0x7f722800c490 comp rx=0 tx=0).stop 2023-08-09T13:47:32.199 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:32.197+0000 7f7235e1f640 1 -- 172.21.15.43:0/3111193637 shutdown_connections 2023-08-09T13:47:32.200 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:32.197+0000 7f7235e1f640 1 --2- 172.21.15.43:0/3111193637 >> [v2:172.21.15.203:6832/998005540,v1:172.21.15.203:6833/998005540] conn(0x7f721006a280 0x7f721006c740 unknown :-1 s=CLOSED pgs=58 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:47:32.200 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:32.197+0000 7f7235e1f640 1 --2- 172.21.15.43:0/3111193637 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f723015d5b0 0x7f72300c42e0 unknown :-1 s=CLOSED pgs=697 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:47:32.200 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:32.197+0000 7f7235e1f640 1 --2- 172.21.15.43:0/3111193637 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f723006a350 0x7f72300c3da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:47:32.200 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:32.197+0000 7f7235e1f640 1 --2- 172.21.15.43:0/3111193637 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f7230069a30 0x7f72300c0a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:47:32.200 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:32.197+0000 7f7235e1f640 1 -- 172.21.15.43:0/3111193637 >> 172.21.15.43:0/3111193637 conn(0x7f723006b990 msgr2=0x7f72300adb40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-09T13:47:32.200 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:32.198+0000 7f7235e1f640 1 -- 172.21.15.43:0/3111193637 shutdown_connections 2023-08-09T13:47:32.200 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:32.198+0000 7f7235e1f640 1 -- 172.21.15.43:0/3111193637 wait complete. 2023-08-09T13:47:32.217 DEBUG:tasks.cephfs.test_snap_schedules:res= [{"fs": "cephfs", "subvol": null, "path": "/snap_test_dir1/dir5", "rel_path": "/snap_test_dir1/dir5", "schedule": "1M", "retention": {}, "start": "2023-08-09T00:00:00", "created": "2023-08-09T13:36:32", "first": "2023-08-09T13:37:00", "last": "2023-08-09T13:47:00", "last_pruned": null, "created_count": 11, "pruned_count": 0, "active": false}] 2023-08-09T13:47:32.218 DEBUG:tasks.cephfs.test_snap_schedules:json_res: {'fs': 'cephfs', 'subvol': None, 'path': '/snap_test_dir1/dir5', 'rel_path': '/snap_test_dir1/dir5', 'schedule': '1M', 'retention': {}, 'start': '2023-08-09T00:00:00', 'created': '2023-08-09T13:36:32', 'first': '2023-08-09T13:37:00', 'last': '2023-08-09T13:47:00', 'last_pruned': None, 'created_count': 11, 'pruned_count': 0, 'active': False} 2023-08-09T13:47:32.218 DEBUG:tasks.cephfs.test_snap_schedules:fs_count: 11 2023-08-09T13:47:32.218 DEBUG:tasks.cephfs.test_snap_schedules:db_count: 11 2023-08-09T13:47:32.219 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-09T13:47:32.219 DEBUG:teuthology.orchestra.run.smithi043:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/dir6/.snap) 2023-08-09T13:47:32.282 INFO:teuthology.orchestra.run.smithi043.stdout:scheduled-2023-08-09-13_37_00_UTC 2023-08-09T13:47:32.283 INFO:teuthology.orchestra.run.smithi043.stdout:scheduled-2023-08-09-13_38_00_UTC 2023-08-09T13:47:32.283 INFO:teuthology.orchestra.run.smithi043.stdout:scheduled-2023-08-09-13_39_00_UTC 2023-08-09T13:47:32.283 INFO:teuthology.orchestra.run.smithi043.stdout:scheduled-2023-08-09-13_40_00_UTC 2023-08-09T13:47:32.284 INFO:teuthology.orchestra.run.smithi043.stdout:scheduled-2023-08-09-13_41_01_UTC 2023-08-09T13:47:32.284 INFO:teuthology.orchestra.run.smithi043.stdout:scheduled-2023-08-09-13_42_01_UTC 2023-08-09T13:47:32.284 INFO:teuthology.orchestra.run.smithi043.stdout:scheduled-2023-08-09-13_43_01_UTC 2023-08-09T13:47:32.285 INFO:teuthology.orchestra.run.smithi043.stdout:scheduled-2023-08-09-13_44_00_UTC 2023-08-09T13:47:32.285 INFO:teuthology.orchestra.run.smithi043.stdout:scheduled-2023-08-09-13_45_00_UTC 2023-08-09T13:47:32.285 INFO:teuthology.orchestra.run.smithi043.stdout:scheduled-2023-08-09-13_46_00_UTC 2023-08-09T13:47:32.285 INFO:teuthology.orchestra.run.smithi043.stdout:scheduled-2023-08-09-13_47_00_UTC 2023-08-09T13:47:32.287 DEBUG:tasks.cephfs.test_snap_schedules:snapshots: ['scheduled-2023-08-09-13_37_00_UTC', 'scheduled-2023-08-09-13_38_00_UTC', 'scheduled-2023-08-09-13_39_00_UTC', 'scheduled-2023-08-09-13_40_00_UTC', 'scheduled-2023-08-09-13_41_01_UTC', 'scheduled-2023-08-09-13_42_01_UTC', 'scheduled-2023-08-09-13_43_01_UTC', 'scheduled-2023-08-09-13_44_00_UTC', 'scheduled-2023-08-09-13_45_00_UTC', 'scheduled-2023-08-09-13_46_00_UTC', 'scheduled-2023-08-09-13_47_00_UTC'] 2023-08-09T13:47:32.288 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs snap-schedule status --fs cephfs --format json /snap_test_dir1/dir6 2023-08-09T13:47:32.519 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:32.518+0000 7f34e9d84640 1 Processor -- start 2023-08-09T13:47:32.520 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:32.518+0000 7f34e9d84640 1 -- start start 2023-08-09T13:47:32.521 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:32.519+0000 7f34e9d84640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f34e40d3020 0x7f34e40b10c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:47:32.521 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:32.520+0000 7f34e9d84640 1 -- --> v1:172.21.15.43:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f34e40b0c30 con 0x7f34e415fc70 2023-08-09T13:47:32.521 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:32.520+0000 7f34e9d84640 1 -- --> v1:172.21.15.203:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f34e40b0300 con 0x7f34e40d00b0 2023-08-09T13:47:32.521 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:32.520+0000 7f34e9d84640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f34e40b05e0 con 0x7f34e40d3020 2023-08-09T13:47:32.523 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:32.520+0000 7f34e37fe640 1 --1- >> v1:172.21.15.43:6789/0 conn(0x7f34e415fc70 0x7f34e40af0a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.43:6789/0 says I am v1:172.21.15.43:49606/0 (socket says 172.21.15.43:49606) 2023-08-09T13:47:32.523 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:32.520+0000 7f34e37fe640 1 -- 172.21.15.43:0/2914445233 learned_addr learned my addr 172.21.15.43:0/2914445233 (peer_addr_for_me v1:172.21.15.43:0/0) 2023-08-09T13:47:32.523 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:32.520+0000 7f34e2ffd640 1 --2- 172.21.15.43:0/2914445233 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f34e40d3020 0x7f34e40b10c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:47:32.524 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:32.520+0000 7f34e27fc640 1 -- 172.21.15.43:0/2914445233 <== mon.0 v1:172.21.15.43:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3922353792 0 0) 0x7f34e40b0c30 con 0x7f34e415fc70 2023-08-09T13:47:32.524 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:32.521+0000 7f34e27fc640 1 -- 172.21.15.43:0/2914445233 --> v1:172.21.15.43:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f34cc003520 con 0x7f34e415fc70 2023-08-09T13:47:32.524 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:32.521+0000 7f34e27fc640 1 -- 172.21.15.43:0/2914445233 <== mon.0 v1:172.21.15.43:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1437606561 0 0) 0x7f34cc003520 con 0x7f34e415fc70 2023-08-09T13:47:32.524 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:32.521+0000 7f34e27fc640 1 -- 172.21.15.43:0/2914445233 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f34e40d3020 msgr2=0x7f34e40b10c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-09T13:47:32.525 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:32.521+0000 7f34e27fc640 1 --2- 172.21.15.43:0/2914445233 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f34e40d3020 0x7f34e40b10c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:47:32.525 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:32.521+0000 7f34e27fc640 1 -- 172.21.15.43:0/2914445233 >> v1:172.21.15.203:6789/0 conn(0x7f34e40d00b0 legacy=0x7f34e40af7b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:47:32.525 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:32.521+0000 7f34e27fc640 1 -- 172.21.15.43:0/2914445233 --> v1:172.21.15.43:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f34e40b08c0 con 0x7f34e415fc70 2023-08-09T13:47:32.525 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:32.521+0000 7f34e27fc640 1 -- 172.21.15.43:0/2914445233 <== mon.0 v1:172.21.15.43:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1285150740 0 0) 0x7f34d4001b70 con 0x7f34e415fc70 2023-08-09T13:47:32.526 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:32.521+0000 7f34e27fc640 1 -- 172.21.15.43:0/2914445233 >> v1:172.21.15.43:6789/0 conn(0x7f34e415fc70 legacy=0x7f34e40af0a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:47:32.526 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:32.521+0000 7f34e27fc640 1 --2- 172.21.15.43:0/2914445233 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f34cc003a10 0x7f34cc003e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:47:32.526 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:32.521+0000 7f34e27fc640 1 --2- 172.21.15.43:0/2914445233 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f34cc0047c0 0x7f34cc004c00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:47:32.526 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:32.521+0000 7f34e27fc640 1 --2- 172.21.15.43:0/2914445233 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f34cc005140 0x7f34cc00b980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:47:32.527 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:32.521+0000 7f34e27fc640 1 -- 172.21.15.43:0/2914445233 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f34e40b05e0 con 0x7f34cc003a10 2023-08-09T13:47:32.527 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:32.521+0000 7f34e27fc640 1 -- 172.21.15.43:0/2914445233 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f34cc00bec0 con 0x7f34cc005140 2023-08-09T13:47:32.527 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:32.521+0000 7f34e27fc640 1 -- 172.21.15.43:0/2914445233 --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f34cc00c050 con 0x7f34cc0047c0 2023-08-09T13:47:32.527 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:32.521+0000 7f34e37fe640 1 --2- 172.21.15.43:0/2914445233 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f34cc003a10 0x7f34cc003e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:47:32.527 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:32.522+0000 7f34e2ffd640 1 --2- 172.21.15.43:0/2914445233 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f34cc0047c0 0x7f34cc004c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:47:32.530 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:32.522+0000 7f34e3fff640 1 --2- 172.21.15.43:0/2914445233 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f34cc005140 0x7f34cc00b980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:47:32.530 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:32.522+0000 7f34e37fe640 1 -- 172.21.15.43:0/2914445233 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f34cc0047c0 msgr2=0x7f34cc004c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-09T13:47:32.530 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:32.522+0000 7f34e37fe640 1 --2- 172.21.15.43:0/2914445233 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f34cc0047c0 0x7f34cc004c00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:47:32.530 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:32.522+0000 7f34e37fe640 1 -- 172.21.15.43:0/2914445233 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f34cc005140 msgr2=0x7f34cc00b980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-09T13:47:32.530 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:32.522+0000 7f34e37fe640 1 --2- 172.21.15.43:0/2914445233 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f34cc005140 0x7f34cc00b980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:47:32.530 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:32.522+0000 7f34e37fe640 1 -- 172.21.15.43:0/2914445233 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f34cc003520 con 0x7f34cc003a10 2023-08-09T13:47:32.531 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:32.522+0000 7f34e3fff640 1 --2- 172.21.15.43:0/2914445233 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f34cc005140 0x7f34cc00b980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-08-09T13:47:32.531 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:32.522+0000 7f34e2ffd640 1 --2- 172.21.15.43:0/2914445233 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f34cc0047c0 0x7f34cc004c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-08-09T13:47:32.531 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:32.523+0000 7f34e37fe640 1 --2- 172.21.15.43:0/2914445233 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f34cc003a10 0x7f34cc003e30 secure :-1 s=READY pgs=699 cs=0 l=1 rev1=1 crypto rx=0x7f34d400c550 tx=0x7f34cc0057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=79ec917b4cd60582 server_cookie=0 in_seq=0 out_seq=0 2023-08-09T13:47:32.531 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:32.523+0000 7f34e27fc640 1 -- 172.21.15.43:0/2914445233 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f34d402ce40 con 0x7f34cc003a10 2023-08-09T13:47:32.531 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:32.524+0000 7f34e27fc640 1 -- 172.21.15.43:0/2914445233 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f34d4005b20 con 0x7f34cc003a10 2023-08-09T13:47:32.531 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:32.524+0000 7f34e27fc640 1 -- 172.21.15.43:0/2914445233 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f34d4007070 con 0x7f34cc003a10 2023-08-09T13:47:32.531 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:32.524+0000 7f34e9d84640 1 -- 172.21.15.43:0/2914445233 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f34cc003a10 msgr2=0x7f34cc003e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:47:32.532 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:32.524+0000 7f34e9d84640 1 --2- 172.21.15.43:0/2914445233 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f34cc003a10 0x7f34cc003e30 secure :-1 s=READY pgs=699 cs=0 l=1 rev1=1 crypto rx=0x7f34d400c550 tx=0x7f34cc0057f0 comp rx=0 tx=0).stop 2023-08-09T13:47:32.532 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:32.524+0000 7f34e3fff640 1 -- 172.21.15.43:0/2914445233 reap_dead start 2023-08-09T13:47:32.532 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:32.524+0000 7f34e9d84640 1 -- 172.21.15.43:0/2914445233 shutdown_connections 2023-08-09T13:47:32.532 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:32.524+0000 7f34e9d84640 1 -- 172.21.15.43:0/2914445233 >> 172.21.15.43:0/2914445233 conn(0x7f34e409b7b0 msgr2=0x7f34e40d2470 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-09T13:47:32.532 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:32.524+0000 7f34e9d84640 1 -- 172.21.15.43:0/2914445233 shutdown_connections 2023-08-09T13:47:32.532 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:32.524+0000 7f34e9d84640 1 -- 172.21.15.43:0/2914445233 wait complete. 2023-08-09T13:47:32.533 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:32.526+0000 7f34e9d84640 1 Processor -- start 2023-08-09T13:47:32.533 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:32.526+0000 7f34e9d84640 1 -- start start 2023-08-09T13:47:32.533 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:32.527+0000 7f34e9d84640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f34cc003a10 0x7f34e415b910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:47:32.533 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:32.527+0000 7f34e9d84640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f34e40ced60 0x7f34e40b60b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:47:32.533 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:32.527+0000 7f34e9d84640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f34e40ca8d0 0x7f34e40b1a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:47:32.533 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:32.527+0000 7f34e9d84640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f34e40cd180 con 0x7f34e40ced60 2023-08-09T13:47:32.533 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:32.527+0000 7f34e9d84640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f34e40b69b0 con 0x7f34e40ca8d0 2023-08-09T13:47:32.533 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:32.527+0000 7f34e9d84640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f34e40b6c90 con 0x7f34cc003a10 2023-08-09T13:47:32.534 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:32.528+0000 7f34e2ffd640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f34e40ced60 0x7f34e40b60b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:47:32.534 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:32.528+0000 7f34e2ffd640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f34e40ced60 0x7f34e40b60b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:53226/0 (socket says 172.21.15.43:53226) 2023-08-09T13:47:32.534 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:32.528+0000 7f34e2ffd640 1 -- 172.21.15.43:0/4223378474 learned_addr learned my addr 172.21.15.43:0/4223378474 (peer_addr_for_me v2:172.21.15.43:0/0) 2023-08-09T13:47:32.534 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:32.528+0000 7f34e37fe640 1 --2- 172.21.15.43:0/4223378474 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f34cc003a10 0x7f34e415b910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:47:32.534 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:32.528+0000 7f34e3fff640 1 --2- 172.21.15.43:0/4223378474 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f34e40ca8d0 0x7f34e40b1a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:47:32.534 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:32.528+0000 7f34e2ffd640 1 -- 172.21.15.43:0/4223378474 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f34cc003a10 msgr2=0x7f34e415b910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:47:32.534 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:32.528+0000 7f34e2ffd640 1 --2- 172.21.15.43:0/4223378474 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f34cc003a10 0x7f34e415b910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:47:32.535 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:32.528+0000 7f34e2ffd640 1 -- 172.21.15.43:0/4223378474 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f34e40ca8d0 msgr2=0x7f34e40b1a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:47:32.535 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:32.528+0000 7f34e2ffd640 1 --2- 172.21.15.43:0/4223378474 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f34e40ca8d0 0x7f34e40b1a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:47:32.535 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:32.528+0000 7f34e2ffd640 1 -- 172.21.15.43:0/4223378474 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f34e40b6f70 con 0x7f34e40ced60 2023-08-09T13:47:32.535 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:32.528+0000 7f34e37fe640 1 --2- 172.21.15.43:0/4223378474 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f34cc003a10 0x7f34e415b910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-08-09T13:47:32.535 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:32.528+0000 7f34e3fff640 1 --2- 172.21.15.43:0/4223378474 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f34e40ca8d0 0x7f34e40b1a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-08-09T13:47:32.535 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:32.528+0000 7f34e2ffd640 1 --2- 172.21.15.43:0/4223378474 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f34e40ced60 0x7f34e40b60b0 secure :-1 s=READY pgs=700 cs=0 l=1 rev1=1 crypto rx=0x7f34d80116d0 tx=0x7f34d8011b90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-09T13:47:32.536 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:32.529+0000 7f34cbfff640 1 -- 172.21.15.43:0/4223378474 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f34d800f9c0 con 0x7f34e40ced60 2023-08-09T13:47:32.537 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:32.529+0000 7f34cbfff640 1 -- 172.21.15.43:0/4223378474 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f34d800fb20 con 0x7f34e40ced60 2023-08-09T13:47:32.537 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:32.529+0000 7f34e9d84640 1 -- 172.21.15.43:0/4223378474 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f34e40cce10 con 0x7f34e40ced60 2023-08-09T13:47:32.537 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:32.529+0000 7f34cbfff640 1 -- 172.21.15.43:0/4223378474 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f34d800fed0 con 0x7f34e40ced60 2023-08-09T13:47:32.537 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:32.529+0000 7f34e9d84640 1 -- 172.21.15.43:0/4223378474 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f34e40b65f0 con 0x7f34e40ced60 2023-08-09T13:47:32.537 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:32.530+0000 7f34e9d84640 1 -- 172.21.15.43:0/4223378474 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f34e40739f0 con 0x7f34e40ced60 2023-08-09T13:47:32.537 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:32.531+0000 7f34cbfff640 1 -- 172.21.15.43:0/4223378474 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 13) v1 ==== 88529+0+0 (secure 0 0 0) 0x7f34d8015460 con 0x7f34e40ced60 2023-08-09T13:47:32.537 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:32.532+0000 7f34cbfff640 1 --2- 172.21.15.43:0/4223378474 >> [v2:172.21.15.203:6832/998005540,v1:172.21.15.203:6833/998005540] conn(0x7f34d006a280 0x7f34d006c740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:47:32.538 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:32.532+0000 7f34cbfff640 1 -- 172.21.15.43:0/4223378474 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(42..42 src has 1..42) v4 ==== 6698+0+0 (secure 0 0 0) 0x7f34d8089460 con 0x7f34e40ced60 2023-08-09T13:47:32.538 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:32.533+0000 7f34e37fe640 1 --2- 172.21.15.43:0/4223378474 >> [v2:172.21.15.203:6832/998005540,v1:172.21.15.203:6833/998005540] conn(0x7f34d006a280 0x7f34d006c740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:47:32.538 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:32.534+0000 7f34e37fe640 1 --2- 172.21.15.43:0/4223378474 >> [v2:172.21.15.203:6832/998005540,v1:172.21.15.203:6833/998005540] conn(0x7f34d006a280 0x7f34d006c740 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f34d400c520 tx=0x7f34d4037040 comp rx=0 tx=0).ready entity=mgr.5003 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-09T13:47:32.538 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:32.537+0000 7f34cbfff640 1 -- 172.21.15.43:0/4223378474 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178066 (secure 0 0 0) 0x7f34d8054710 con 0x7f34e40ced60 2023-08-09T13:47:32.756 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:32.755+0000 7f34e9d84640 1 -- 172.21.15.43:0/4223378474 --> [v2:172.21.15.203:6832/998005540,v1:172.21.15.203:6833/998005540] -- mgr_command(tid 0: {"prefix": "fs snap-schedule status", "fs": "cephfs", "path": "/snap_test_dir1/dir6", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f34e4160050 con 0x7f34d006a280 2023-08-09T13:47:32.777 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:32.776+0000 7f34cbfff640 1 -- 172.21.15.43:0/4223378474 <== mgr.5003 v2:172.21.15.203:6832/998005540 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+345 (secure 0 0 0) 0x7f34d4001930 con 0x7f34d006a280 2023-08-09T13:47:32.778 INFO:teuthology.orchestra.run.smithi043.stdout: 2023-08-09T13:47:32.778 INFO:teuthology.orchestra.run.smithi043.stdout:[{"fs": "cephfs", "subvol": null, "path": "/snap_test_dir1/dir6", "rel_path": "/snap_test_dir1/dir6", "schedule": "1M", "retention": {}, "start": "2023-08-09T00:00:00", "created": "2023-08-09T13:36:33", "first": "2023-08-09T13:37:00", "last": "2023-08-09T13:47:00", "last_pruned": null, "created_count": 11, "pruned_count": 0, "active": false}] 2023-08-09T13:47:32.780 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:32.779+0000 7f34e9d84640 1 -- 172.21.15.43:0/4223378474 >> [v2:172.21.15.203:6832/998005540,v1:172.21.15.203:6833/998005540] conn(0x7f34d006a280 msgr2=0x7f34d006c740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:47:32.781 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:32.779+0000 7f34e9d84640 1 --2- 172.21.15.43:0/4223378474 >> [v2:172.21.15.203:6832/998005540,v1:172.21.15.203:6833/998005540] conn(0x7f34d006a280 0x7f34d006c740 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f34d400c520 tx=0x7f34d4037040 comp rx=0 tx=0).stop 2023-08-09T13:47:32.781 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:32.779+0000 7f34e9d84640 1 -- 172.21.15.43:0/4223378474 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f34e40ced60 msgr2=0x7f34e40b60b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:47:32.781 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:32.779+0000 7f34e9d84640 1 --2- 172.21.15.43:0/4223378474 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f34e40ced60 0x7f34e40b60b0 secure :-1 s=READY pgs=700 cs=0 l=1 rev1=1 crypto rx=0x7f34d80116d0 tx=0x7f34d8011b90 comp rx=0 tx=0).stop 2023-08-09T13:47:32.781 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:32.779+0000 7f34e9d84640 1 -- 172.21.15.43:0/4223378474 shutdown_connections 2023-08-09T13:47:32.781 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:32.779+0000 7f34e9d84640 1 --2- 172.21.15.43:0/4223378474 >> [v2:172.21.15.203:6832/998005540,v1:172.21.15.203:6833/998005540] conn(0x7f34d006a280 0x7f34d006c740 unknown :-1 s=CLOSED pgs=59 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:47:32.781 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:32.779+0000 7f34e9d84640 1 --2- 172.21.15.43:0/4223378474 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f34e40ca8d0 0x7f34e40b1a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:47:32.781 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:32.779+0000 7f34e9d84640 1 --2- 172.21.15.43:0/4223378474 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f34e40ced60 0x7f34e40b60b0 unknown :-1 s=CLOSED pgs=700 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:47:32.782 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:32.779+0000 7f34e9d84640 1 --2- 172.21.15.43:0/4223378474 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f34cc003a10 0x7f34e415b910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:47:32.782 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:32.779+0000 7f34e9d84640 1 -- 172.21.15.43:0/4223378474 >> 172.21.15.43:0/4223378474 conn(0x7f34e409b7b0 msgr2=0x7f34e40b2f50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-09T13:47:32.782 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:32.779+0000 7f34e9d84640 1 -- 172.21.15.43:0/4223378474 shutdown_connections 2023-08-09T13:47:32.782 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:32.779+0000 7f34e9d84640 1 -- 172.21.15.43:0/4223378474 wait complete. 2023-08-09T13:47:32.794 DEBUG:teuthology.orchestra.run.smithi043:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-08-09T13:47:32.799 DEBUG:teuthology.orchestra.run.smithi203:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-08-09T13:47:32.801 DEBUG:tasks.cephfs.test_snap_schedules:res= [{"fs": "cephfs", "subvol": null, "path": "/snap_test_dir1/dir6", "rel_path": "/snap_test_dir1/dir6", "schedule": "1M", "retention": {}, "start": "2023-08-09T00:00:00", "created": "2023-08-09T13:36:33", "first": "2023-08-09T13:37:00", "last": "2023-08-09T13:47:00", "last_pruned": null, "created_count": 11, "pruned_count": 0, "active": false}] 2023-08-09T13:47:32.801 DEBUG:tasks.cephfs.test_snap_schedules:json_res: {'fs': 'cephfs', 'subvol': None, 'path': '/snap_test_dir1/dir6', 'rel_path': '/snap_test_dir1/dir6', 'schedule': '1M', 'retention': {}, 'start': '2023-08-09T00:00:00', 'created': '2023-08-09T13:36:33', 'first': '2023-08-09T13:37:00', 'last': '2023-08-09T13:47:00', 'last_pruned': None, 'created_count': 11, 'pruned_count': 0, 'active': False} 2023-08-09T13:47:32.802 DEBUG:tasks.cephfs.test_snap_schedules:fs_count: 11 2023-08-09T13:47:32.802 DEBUG:tasks.cephfs.test_snap_schedules:db_count: 11 2023-08-09T13:47:32.802 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-09T13:47:32.802 DEBUG:teuthology.orchestra.run.smithi043:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/dir7/.snap) 2023-08-09T13:47:32.892 INFO:teuthology.orchestra.run.smithi043.stdout:scheduled-2023-08-09-13_37_00_UTC 2023-08-09T13:47:32.893 INFO:teuthology.orchestra.run.smithi043.stdout:scheduled-2023-08-09-13_38_00_UTC 2023-08-09T13:47:32.893 INFO:teuthology.orchestra.run.smithi043.stdout:scheduled-2023-08-09-13_39_01_UTC 2023-08-09T13:47:32.893 INFO:teuthology.orchestra.run.smithi043.stdout:scheduled-2023-08-09-13_40_01_UTC 2023-08-09T13:47:32.893 INFO:teuthology.orchestra.run.smithi043.stdout:scheduled-2023-08-09-13_41_00_UTC 2023-08-09T13:47:32.893 INFO:teuthology.orchestra.run.smithi043.stdout:scheduled-2023-08-09-13_42_00_UTC 2023-08-09T13:47:32.893 INFO:teuthology.orchestra.run.smithi043.stdout:scheduled-2023-08-09-13_43_00_UTC 2023-08-09T13:47:32.893 INFO:teuthology.orchestra.run.smithi043.stdout:scheduled-2023-08-09-13_44_00_UTC 2023-08-09T13:47:32.894 INFO:teuthology.orchestra.run.smithi043.stdout:scheduled-2023-08-09-13_45_00_UTC 2023-08-09T13:47:32.894 INFO:teuthology.orchestra.run.smithi043.stdout:scheduled-2023-08-09-13_46_00_UTC 2023-08-09T13:47:32.894 INFO:teuthology.orchestra.run.smithi043.stdout:scheduled-2023-08-09-13_47_01_UTC 2023-08-09T13:47:32.895 DEBUG:tasks.cephfs.test_snap_schedules:snapshots: ['scheduled-2023-08-09-13_37_00_UTC', 'scheduled-2023-08-09-13_38_00_UTC', 'scheduled-2023-08-09-13_39_01_UTC', 'scheduled-2023-08-09-13_40_01_UTC', 'scheduled-2023-08-09-13_41_00_UTC', 'scheduled-2023-08-09-13_42_00_UTC', 'scheduled-2023-08-09-13_43_00_UTC', 'scheduled-2023-08-09-13_44_00_UTC', 'scheduled-2023-08-09-13_45_00_UTC', 'scheduled-2023-08-09-13_46_00_UTC', 'scheduled-2023-08-09-13_47_01_UTC'] 2023-08-09T13:47:32.895 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs snap-schedule status --fs cephfs --format json /snap_test_dir1/dir7 2023-08-09T13:47:33.120 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:33.118+0000 7fd29abd2640 1 Processor -- start 2023-08-09T13:47:33.120 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:33.119+0000 7fd29abd2640 1 -- start start 2023-08-09T13:47:33.121 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:33.120+0000 7fd29abd2640 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7fd29415a330 0x7fd2940c9780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:47:33.121 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:33.120+0000 7fd29abd2640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fd2940aff80 0x7fd294070d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:47:33.121 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:33.120+0000 7fd29abd2640 1 -- --> v1:172.21.15.203:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd29415dd30 con 0x7fd29415ea60 2023-08-09T13:47:33.122 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:33.120+0000 7fd29abd2640 1 -- --> v2:172.21.15.203:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd2940c9cc0 con 0x7fd29415a330 2023-08-09T13:47:33.122 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:33.120+0000 7fd29abd2640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd294071240 con 0x7fd2940aff80 2023-08-09T13:47:33.122 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:33.121+0000 7fd29a3d1640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fd2940aff80 0x7fd294070d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:47:33.123 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:33.121+0000 7fd299bd0640 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7fd29415a330 0x7fd2940c9780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:47:33.123 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:33.121+0000 7fd29a3d1640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fd2940aff80 0x7fd294070d00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.203:3301/0 says I am v2:172.21.15.43:41950/0 (socket says 172.21.15.43:41950) 2023-08-09T13:47:33.123 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:33.121+0000 7fd29a3d1640 1 -- 172.21.15.43:0/1001593526 learned_addr learned my addr 172.21.15.43:0/1001593526 (peer_addr_for_me v2:172.21.15.43:0/0) 2023-08-09T13:47:33.123 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:33.121+0000 7fd29a3d1640 1 -- 172.21.15.43:0/1001593526 >> v2:172.21.15.203:3300/0 conn(0x7fd29415a330 msgr2=0x7fd2940c9780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-09T13:47:33.124 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:33.121+0000 7fd29a3d1640 1 --2- 172.21.15.43:0/1001593526 >> v2:172.21.15.203:3300/0 conn(0x7fd29415a330 0x7fd2940c9780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:47:33.124 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:33.121+0000 7fd29a3d1640 1 -- 172.21.15.43:0/1001593526 >> v1:172.21.15.203:6789/0 conn(0x7fd29415ea60 legacy=0x7fd2940af870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:47:33.124 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:33.121+0000 7fd29a3d1640 1 -- 172.21.15.43:0/1001593526 --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd294071490 con 0x7fd2940aff80 2023-08-09T13:47:33.124 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:33.123+0000 7fd29a3d1640 1 --2- 172.21.15.43:0/1001593526 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fd2940aff80 0x7fd294070d00 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7fd28c009e50 tx=0x7fd28c02bf60 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e6fd055dbee62499 server_cookie=0 in_seq=0 out_seq=0 2023-08-09T13:47:33.125 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:33.123+0000 7fd298bce640 1 -- 172.21.15.43:0/1001593526 <== mon.2 v2:172.21.15.203:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd28c02cb00 con 0x7fd2940aff80 2023-08-09T13:47:33.125 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:33.124+0000 7fd298bce640 1 -- 172.21.15.43:0/1001593526 <== mon.2 v2:172.21.15.203:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd28c02cc60 con 0x7fd2940aff80 2023-08-09T13:47:33.125 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:33.124+0000 7fd298bce640 1 -- 172.21.15.43:0/1001593526 <== mon.2 v2:172.21.15.203:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd28c0343f0 con 0x7fd2940aff80 2023-08-09T13:47:33.126 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:33.124+0000 7fd29abd2640 1 -- 172.21.15.43:0/1001593526 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fd2940aff80 msgr2=0x7fd294070d00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:47:33.126 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:33.124+0000 7fd29abd2640 1 --2- 172.21.15.43:0/1001593526 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fd2940aff80 0x7fd294070d00 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7fd28c009e50 tx=0x7fd28c02bf60 comp rx=0 tx=0).stop 2023-08-09T13:47:33.126 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:33.124+0000 7fd29abd2640 1 -- 172.21.15.43:0/1001593526 shutdown_connections 2023-08-09T13:47:33.126 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:33.124+0000 7fd29abd2640 1 --2- 172.21.15.43:0/1001593526 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fd2940aff80 0x7fd294070d00 unknown :-1 s=CLOSED pgs=92 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:47:33.126 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:33.124+0000 7fd29abd2640 1 --2- 172.21.15.43:0/1001593526 >> v2:172.21.15.203:3300/0 conn(0x7fd29415a330 0x7fd2940c9780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:47:33.127 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:33.124+0000 7fd29abd2640 1 -- 172.21.15.43:0/1001593526 >> 172.21.15.43:0/1001593526 conn(0x7fd2940ca870 msgr2=0x7fd2940cac50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-09T13:47:33.127 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:33.124+0000 7fd29abd2640 1 -- 172.21.15.43:0/1001593526 shutdown_connections 2023-08-09T13:47:33.127 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:33.125+0000 7fd29abd2640 1 -- 172.21.15.43:0/1001593526 wait complete. 2023-08-09T13:47:33.127 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:33.126+0000 7fd29abd2640 1 Processor -- start 2023-08-09T13:47:33.128 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:33.126+0000 7fd29abd2640 1 -- start start 2023-08-09T13:47:33.129 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:33.127+0000 7fd29abd2640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fd2940aff80 0x7fd2940c3b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:47:33.129 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:33.127+0000 7fd29abd2640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fd29415a330 0x7fd294148e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:47:33.129 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:33.127+0000 7fd29abd2640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fd29415ea60 0x7fd29414fab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:47:33.129 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:33.127+0000 7fd29abd2640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd29414ed80 con 0x7fd2940aff80 2023-08-09T13:47:33.129 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:33.127+0000 7fd29abd2640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd29414f060 con 0x7fd29415a330 2023-08-09T13:47:33.130 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:33.127+0000 7fd29abd2640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd29414f340 con 0x7fd29415ea60 2023-08-09T13:47:33.130 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:33.128+0000 7fd299bd0640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fd2940aff80 0x7fd2940c3b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:47:33.130 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:33.128+0000 7fd299bd0640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fd2940aff80 0x7fd2940c3b00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:53228/0 (socket says 172.21.15.43:53228) 2023-08-09T13:47:33.130 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:33.128+0000 7fd299bd0640 1 -- 172.21.15.43:0/1406329970 learned_addr learned my addr 172.21.15.43:0/1406329970 (peer_addr_for_me v2:172.21.15.43:0/0) 2023-08-09T13:47:33.131 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:33.128+0000 7fd29a3d1640 1 --2- 172.21.15.43:0/1406329970 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fd29415ea60 0x7fd29414fab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:47:33.131 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:33.128+0000 7fd2993cf640 1 --2- 172.21.15.43:0/1406329970 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fd29415a330 0x7fd294148e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:47:33.131 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:33.128+0000 7fd299bd0640 1 -- 172.21.15.43:0/1406329970 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fd29415ea60 msgr2=0x7fd29414fab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:47:33.131 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:33.128+0000 7fd299bd0640 1 --2- 172.21.15.43:0/1406329970 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fd29415ea60 0x7fd29414fab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:47:33.132 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:33.128+0000 7fd299bd0640 1 -- 172.21.15.43:0/1406329970 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fd29415a330 msgr2=0x7fd294148e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:47:33.132 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:33.128+0000 7fd299bd0640 1 --2- 172.21.15.43:0/1406329970 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fd29415a330 0x7fd294148e60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:47:33.132 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:33.128+0000 7fd299bd0640 1 -- 172.21.15.43:0/1406329970 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd28c009b30 con 0x7fd2940aff80 2023-08-09T13:47:33.132 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:33.128+0000 7fd2993cf640 1 --2- 172.21.15.43:0/1406329970 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fd29415a330 0x7fd294148e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-08-09T13:47:33.132 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:33.129+0000 7fd299bd0640 1 --2- 172.21.15.43:0/1406329970 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fd2940aff80 0x7fd2940c3b00 secure :-1 s=READY pgs=701 cs=0 l=1 rev1=1 crypto rx=0x7fd29000e9b0 tx=0x7fd29000ee70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-09T13:47:33.132 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:33.129+0000 7fd28a7fc640 1 -- 172.21.15.43:0/1406329970 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd290009670 con 0x7fd2940aff80 2023-08-09T13:47:33.133 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:33.129+0000 7fd28a7fc640 1 -- 172.21.15.43:0/1406329970 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd29000c9a0 con 0x7fd2940aff80 2023-08-09T13:47:33.133 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:33.129+0000 7fd28a7fc640 1 -- 172.21.15.43:0/1406329970 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd29000cd50 con 0x7fd2940aff80 2023-08-09T13:47:33.133 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:33.129+0000 7fd29abd2640 1 -- 172.21.15.43:0/1406329970 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd2940b0ab0 con 0x7fd2940aff80 2023-08-09T13:47:33.133 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:33.129+0000 7fd29abd2640 1 -- 172.21.15.43:0/1406329970 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd2940c4040 con 0x7fd2940aff80 2023-08-09T13:47:33.133 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:33.130+0000 7fd29abd2640 1 -- 172.21.15.43:0/1406329970 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd2940ca690 con 0x7fd2940aff80 2023-08-09T13:47:33.134 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:33.131+0000 7fd28a7fc640 1 -- 172.21.15.43:0/1406329970 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 13) v1 ==== 88529+0+0 (secure 0 0 0) 0x7fd290013470 con 0x7fd2940aff80 2023-08-09T13:47:33.134 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:33.132+0000 7fd28a7fc640 1 --2- 172.21.15.43:0/1406329970 >> [v2:172.21.15.203:6832/998005540,v1:172.21.15.203:6833/998005540] conn(0x7fd27806a250 0x7fd27806c710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:47:33.134 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:33.132+0000 7fd28a7fc640 1 -- 172.21.15.43:0/1406329970 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(42..42 src has 1..42) v4 ==== 6698+0+0 (secure 0 0 0) 0x7fd290088e60 con 0x7fd2940aff80 2023-08-09T13:47:33.134 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:33.133+0000 7fd2993cf640 1 --2- 172.21.15.43:0/1406329970 >> [v2:172.21.15.203:6832/998005540,v1:172.21.15.203:6833/998005540] conn(0x7fd27806a250 0x7fd27806c710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:47:33.136 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:33.134+0000 7fd2993cf640 1 --2- 172.21.15.43:0/1406329970 >> [v2:172.21.15.203:6832/998005540,v1:172.21.15.203:6833/998005540] conn(0x7fd27806a250 0x7fd27806c710 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7fd29414f5a0 tx=0x7fd284009040 comp rx=0 tx=0).ready entity=mgr.5003 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-09T13:47:33.138 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:33.137+0000 7fd28a7fc640 1 -- 172.21.15.43:0/1406329970 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178066 (secure 0 0 0) 0x7fd290054040 con 0x7fd2940aff80 2023-08-09T13:47:33.356 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:33.354+0000 7fd29abd2640 1 -- 172.21.15.43:0/1406329970 --> [v2:172.21.15.203:6832/998005540,v1:172.21.15.203:6833/998005540] -- mgr_command(tid 0: {"prefix": "fs snap-schedule status", "fs": "cephfs", "path": "/snap_test_dir1/dir7", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7fd29415e260 con 0x7fd27806a250 2023-08-09T13:47:33.375 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:33.374+0000 7fd28a7fc640 1 -- 172.21.15.43:0/1406329970 <== mgr.5003 v2:172.21.15.203:6832/998005540 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+345 (secure 0 0 0) 0x7fd284001d30 con 0x7fd27806a250 2023-08-09T13:47:33.376 INFO:teuthology.orchestra.run.smithi043.stdout: 2023-08-09T13:47:33.376 INFO:teuthology.orchestra.run.smithi043.stdout:[{"fs": "cephfs", "subvol": null, "path": "/snap_test_dir1/dir7", "rel_path": "/snap_test_dir1/dir7", "schedule": "1M", "retention": {}, "start": "2023-08-09T00:00:00", "created": "2023-08-09T13:36:34", "first": "2023-08-09T13:37:00", "last": "2023-08-09T13:47:01", "last_pruned": null, "created_count": 11, "pruned_count": 0, "active": false}] 2023-08-09T13:47:33.378 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:33.377+0000 7fd29abd2640 1 -- 172.21.15.43:0/1406329970 >> [v2:172.21.15.203:6832/998005540,v1:172.21.15.203:6833/998005540] conn(0x7fd27806a250 msgr2=0x7fd27806c710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:47:33.379 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:33.377+0000 7fd29abd2640 1 --2- 172.21.15.43:0/1406329970 >> [v2:172.21.15.203:6832/998005540,v1:172.21.15.203:6833/998005540] conn(0x7fd27806a250 0x7fd27806c710 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7fd29414f5a0 tx=0x7fd284009040 comp rx=0 tx=0).stop 2023-08-09T13:47:33.379 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:33.377+0000 7fd29abd2640 1 -- 172.21.15.43:0/1406329970 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fd2940aff80 msgr2=0x7fd2940c3b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:47:33.379 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:33.377+0000 7fd29abd2640 1 --2- 172.21.15.43:0/1406329970 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fd2940aff80 0x7fd2940c3b00 secure :-1 s=READY pgs=701 cs=0 l=1 rev1=1 crypto rx=0x7fd29000e9b0 tx=0x7fd29000ee70 comp rx=0 tx=0).stop 2023-08-09T13:47:33.379 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:33.377+0000 7fd29abd2640 1 -- 172.21.15.43:0/1406329970 shutdown_connections 2023-08-09T13:47:33.379 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:33.377+0000 7fd29abd2640 1 --2- 172.21.15.43:0/1406329970 >> [v2:172.21.15.203:6832/998005540,v1:172.21.15.203:6833/998005540] conn(0x7fd27806a250 0x7fd27806c710 unknown :-1 s=CLOSED pgs=60 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:47:33.379 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:33.377+0000 7fd29abd2640 1 --2- 172.21.15.43:0/1406329970 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fd29415ea60 0x7fd29414fab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:47:33.380 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:33.377+0000 7fd29abd2640 1 --2- 172.21.15.43:0/1406329970 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fd29415a330 0x7fd294148e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:47:33.380 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:33.377+0000 7fd29abd2640 1 --2- 172.21.15.43:0/1406329970 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fd2940aff80 0x7fd2940c3b00 unknown :-1 s=CLOSED pgs=701 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:47:33.380 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:33.377+0000 7fd29abd2640 1 -- 172.21.15.43:0/1406329970 >> 172.21.15.43:0/1406329970 conn(0x7fd2940ca870 msgr2=0x7fd2941595e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-09T13:47:33.380 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:33.377+0000 7fd29abd2640 1 -- 172.21.15.43:0/1406329970 shutdown_connections 2023-08-09T13:47:33.380 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:33.377+0000 7fd29abd2640 1 -- 172.21.15.43:0/1406329970 wait complete. 2023-08-09T13:47:33.398 DEBUG:tasks.cephfs.test_snap_schedules:res= [{"fs": "cephfs", "subvol": null, "path": "/snap_test_dir1/dir7", "rel_path": "/snap_test_dir1/dir7", "schedule": "1M", "retention": {}, "start": "2023-08-09T00:00:00", "created": "2023-08-09T13:36:34", "first": "2023-08-09T13:37:00", "last": "2023-08-09T13:47:01", "last_pruned": null, "created_count": 11, "pruned_count": 0, "active": false}] 2023-08-09T13:47:33.399 DEBUG:tasks.cephfs.test_snap_schedules:json_res: {'fs': 'cephfs', 'subvol': None, 'path': '/snap_test_dir1/dir7', 'rel_path': '/snap_test_dir1/dir7', 'schedule': '1M', 'retention': {}, 'start': '2023-08-09T00:00:00', 'created': '2023-08-09T13:36:34', 'first': '2023-08-09T13:37:00', 'last': '2023-08-09T13:47:01', 'last_pruned': None, 'created_count': 11, 'pruned_count': 0, 'active': False} 2023-08-09T13:47:33.399 DEBUG:tasks.cephfs.test_snap_schedules:fs_count: 11 2023-08-09T13:47:33.399 DEBUG:tasks.cephfs.test_snap_schedules:db_count: 11 2023-08-09T13:47:33.399 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-09T13:47:33.400 DEBUG:teuthology.orchestra.run.smithi043:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/dir8/.snap) 2023-08-09T13:47:33.465 INFO:teuthology.orchestra.run.smithi043.stdout:scheduled-2023-08-09-13_37_00_UTC 2023-08-09T13:47:33.465 INFO:teuthology.orchestra.run.smithi043.stdout:scheduled-2023-08-09-13_38_00_UTC 2023-08-09T13:47:33.465 INFO:teuthology.orchestra.run.smithi043.stdout:scheduled-2023-08-09-13_39_00_UTC 2023-08-09T13:47:33.465 INFO:teuthology.orchestra.run.smithi043.stdout:scheduled-2023-08-09-13_40_01_UTC 2023-08-09T13:47:33.465 INFO:teuthology.orchestra.run.smithi043.stdout:scheduled-2023-08-09-13_41_01_UTC 2023-08-09T13:47:33.466 INFO:teuthology.orchestra.run.smithi043.stdout:scheduled-2023-08-09-13_42_01_UTC 2023-08-09T13:47:33.466 INFO:teuthology.orchestra.run.smithi043.stdout:scheduled-2023-08-09-13_43_01_UTC 2023-08-09T13:47:33.466 INFO:teuthology.orchestra.run.smithi043.stdout:scheduled-2023-08-09-13_44_00_UTC 2023-08-09T13:47:33.466 INFO:teuthology.orchestra.run.smithi043.stdout:scheduled-2023-08-09-13_45_00_UTC 2023-08-09T13:47:33.466 INFO:teuthology.orchestra.run.smithi043.stdout:scheduled-2023-08-09-13_46_01_UTC 2023-08-09T13:47:33.466 INFO:teuthology.orchestra.run.smithi043.stdout:scheduled-2023-08-09-13_47_00_UTC 2023-08-09T13:47:33.467 DEBUG:tasks.cephfs.test_snap_schedules:snapshots: ['scheduled-2023-08-09-13_37_00_UTC', 'scheduled-2023-08-09-13_38_00_UTC', 'scheduled-2023-08-09-13_39_00_UTC', 'scheduled-2023-08-09-13_40_01_UTC', 'scheduled-2023-08-09-13_41_01_UTC', 'scheduled-2023-08-09-13_42_01_UTC', 'scheduled-2023-08-09-13_43_01_UTC', 'scheduled-2023-08-09-13_44_00_UTC', 'scheduled-2023-08-09-13_45_00_UTC', 'scheduled-2023-08-09-13_46_01_UTC', 'scheduled-2023-08-09-13_47_00_UTC'] 2023-08-09T13:47:33.468 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs snap-schedule status --fs cephfs --format json /snap_test_dir1/dir8 2023-08-09T13:47:33.696 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:33.694+0000 7f83f6b96640 1 Processor -- start 2023-08-09T13:47:33.697 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:33.695+0000 7f83f6b96640 1 -- start start 2023-08-09T13:47:33.698 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:33.697+0000 7f83f6b96640 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7f83f00c8f50 0x7f83f006f4a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:47:33.698 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:33.697+0000 7f83f6b96640 1 -- --> v1:172.21.15.43:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f83f00cc180 con 0x7f83f0164560 2023-08-09T13:47:33.698 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:33.697+0000 7f83f6b96640 1 -- --> v1:172.21.15.203:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f83f0164fe0 con 0x7f83f00cebd0 2023-08-09T13:47:33.699 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:33.697+0000 7f83f6b96640 1 -- --> v2:172.21.15.203:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f83f01652c0 con 0x7f83f00c8f50 2023-08-09T13:47:33.700 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:33.697+0000 7f83effff640 1 --1- >> v1:172.21.15.43:6789/0 conn(0x7f83f0164560 0x7f83f00ce4c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.43:6789/0 says I am v1:172.21.15.43:49618/0 (socket says 172.21.15.43:49618) 2023-08-09T13:47:33.700 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:33.697+0000 7f83effff640 1 -- 172.21.15.43:0/971081708 learned_addr learned my addr 172.21.15.43:0/971081708 (peer_addr_for_me v1:172.21.15.43:0/0) 2023-08-09T13:47:33.700 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:33.697+0000 7f83ef7fe640 1 -- 172.21.15.43:0/971081708 <== mon.0 v1:172.21.15.43:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2133134406 0 0) 0x7f83f00cc180 con 0x7f83f0164560 2023-08-09T13:47:33.700 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:33.698+0000 7f83ef7fe640 1 -- 172.21.15.43:0/971081708 --> v1:172.21.15.43:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f83e0003520 con 0x7f83f0164560 2023-08-09T13:47:33.701 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:33.698+0000 7f83f4932640 1 --2- 172.21.15.43:0/971081708 >> v2:172.21.15.203:3300/0 conn(0x7f83f00c8f50 0x7f83f006f4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:47:33.701 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:33.698+0000 7f83ef7fe640 1 -- 172.21.15.43:0/971081708 <== mon.0 v1:172.21.15.43:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3173999047 0 0) 0x7f83e0003520 con 0x7f83f0164560 2023-08-09T13:47:33.701 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:33.698+0000 7f83ef7fe640 1 -- 172.21.15.43:0/971081708 >> v2:172.21.15.203:3300/0 conn(0x7f83f00c8f50 msgr2=0x7f83f006f4a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-09T13:47:33.701 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:33.698+0000 7f83ef7fe640 1 --2- 172.21.15.43:0/971081708 >> v2:172.21.15.203:3300/0 conn(0x7f83f00c8f50 0x7f83f006f4a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:47:33.701 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:33.698+0000 7f83ef7fe640 1 -- 172.21.15.43:0/971081708 >> v1:172.21.15.203:6789/0 conn(0x7f83f00cebd0 legacy=0x7f83f00cc610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-09T13:47:33.701 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:33.698+0000 7f83ef7fe640 1 -- 172.21.15.43:0/971081708 --> v1:172.21.15.43:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f83f00cbe10 con 0x7f83f0164560 2023-08-09T13:47:33.701 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:33.698+0000 7f83ef7fe640 1 -- 172.21.15.43:0/971081708 <== mon.0 v1:172.21.15.43:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1285150740 0 0) 0x7f83dc001670 con 0x7f83f0164560 2023-08-09T13:47:33.701 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:33.698+0000 7f83ef7fe640 1 -- 172.21.15.43:0/971081708 >> v1:172.21.15.43:6789/0 conn(0x7f83f0164560 legacy=0x7f83f00ce4c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:47:33.702 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:33.698+0000 7f83ef7fe640 1 --2- 172.21.15.43:0/971081708 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f83e00039e0 0x7f83e0003de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:47:33.702 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:33.698+0000 7f83ef7fe640 1 --2- 172.21.15.43:0/971081708 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f83e00047c0 0x7f83e0004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:47:33.702 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:33.698+0000 7f83ef7fe640 1 --2- 172.21.15.43:0/971081708 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f83e00052a0 0x7f83e000b980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:47:33.702 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:33.698+0000 7f83ef7fe640 1 -- 172.21.15.43:0/971081708 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f83f01652c0 con 0x7f83e00052a0 2023-08-09T13:47:33.702 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:33.698+0000 7f83ef7fe640 1 -- 172.21.15.43:0/971081708 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f83e000c1c0 con 0x7f83e00039e0 2023-08-09T13:47:33.702 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:33.698+0000 7f83ef7fe640 1 -- 172.21.15.43:0/971081708 --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f83e000c380 con 0x7f83e00047c0 2023-08-09T13:47:33.702 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:33.698+0000 7f83f5133640 1 --2- 172.21.15.43:0/971081708 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f83e00052a0 0x7f83e000b980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:47:33.704 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:33.699+0000 7f83effff640 1 --2- 172.21.15.43:0/971081708 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f83e00047c0 0x7f83e0004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:47:33.704 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:33.699+0000 7f83f5133640 1 -- 172.21.15.43:0/971081708 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f83e00047c0 msgr2=0x7f83e0004c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-09T13:47:33.704 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:33.699+0000 7f83f5133640 1 --2- 172.21.15.43:0/971081708 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f83e00047c0 0x7f83e0004c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:47:33.704 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:33.699+0000 7f83f4932640 1 --2- 172.21.15.43:0/971081708 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f83e00039e0 0x7f83e0003de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:47:33.704 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:33.699+0000 7f83f5133640 1 -- 172.21.15.43:0/971081708 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f83e00039e0 msgr2=0x7f83e0003de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-09T13:47:33.705 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:33.699+0000 7f83f5133640 1 --2- 172.21.15.43:0/971081708 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f83e00039e0 0x7f83e0003de0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:47:33.705 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:33.699+0000 7f83f5133640 1 -- 172.21.15.43:0/971081708 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f83e0003520 con 0x7f83e00052a0 2023-08-09T13:47:33.705 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:33.700+0000 7f83f5133640 1 -- 172.21.15.43:0/971081708 reap_dead start 2023-08-09T13:47:33.705 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:33.700+0000 7f83f5133640 1 --2- 172.21.15.43:0/971081708 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f83e00052a0 0x7f83e000b980 secure :-1 s=READY pgs=703 cs=0 l=1 rev1=1 crypto rx=0x7f83f00cdc00 tx=0x7f83e00057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=36be8ce3d52e215c server_cookie=0 in_seq=0 out_seq=0 2023-08-09T13:47:33.705 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:33.701+0000 7f83ef7fe640 1 -- 172.21.15.43:0/971081708 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f83e8006d50 con 0x7f83e00052a0 2023-08-09T13:47:33.706 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:33.701+0000 7f83ef7fe640 1 -- 172.21.15.43:0/971081708 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f83e8007020 con 0x7f83e00052a0 2023-08-09T13:47:33.706 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:33.701+0000 7f83ef7fe640 1 -- 172.21.15.43:0/971081708 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f83e80073d0 con 0x7f83e00052a0 2023-08-09T13:47:33.706 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:33.701+0000 7f83f6b96640 1 -- 172.21.15.43:0/971081708 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f83e00052a0 msgr2=0x7f83e000b980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:47:33.706 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:33.701+0000 7f83f6b96640 1 --2- 172.21.15.43:0/971081708 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f83e00052a0 0x7f83e000b980 secure :-1 s=READY pgs=703 cs=0 l=1 rev1=1 crypto rx=0x7f83f00cdc00 tx=0x7f83e00057f0 comp rx=0 tx=0).stop 2023-08-09T13:47:33.706 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:33.701+0000 7f83f6b96640 1 -- 172.21.15.43:0/971081708 shutdown_connections 2023-08-09T13:47:33.706 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:33.701+0000 7f83f6b96640 1 --2- 172.21.15.43:0/971081708 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f83e00052a0 0x7f83e000b980 unknown :-1 s=CLOSED pgs=703 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:47:33.706 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:33.701+0000 7f83f6b96640 1 -- 172.21.15.43:0/971081708 >> 172.21.15.43:0/971081708 conn(0x7f83f006b990 msgr2=0x7f83f006bda0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-09T13:47:33.706 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:33.701+0000 7f83f6b96640 1 -- 172.21.15.43:0/971081708 shutdown_connections 2023-08-09T13:47:33.707 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:33.701+0000 7f83f6b96640 1 -- 172.21.15.43:0/971081708 wait complete. 2023-08-09T13:47:33.709 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:33.703+0000 7f83f6b96640 1 Processor -- start 2023-08-09T13:47:33.709 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:33.703+0000 7f83f6b96640 1 -- start start 2023-08-09T13:47:33.709 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:33.704+0000 7f83f6b96640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f83e00052a0 0x7f83f00b3030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:47:33.710 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:33.704+0000 7f83f6b96640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f83f015d770 0x7f83f00b1480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:47:33.710 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:33.704+0000 7f83f6b96640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f83f01634a0 0x7f83f00c6d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:47:33.710 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:33.704+0000 7f83f6b96640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f83f00b0cc0 con 0x7f83f01634a0 2023-08-09T13:47:33.710 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:33.704+0000 7f83f6b96640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f83f00b36c0 con 0x7f83e00052a0 2023-08-09T13:47:33.710 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:33.704+0000 7f83f6b96640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f83f00b39a0 con 0x7f83f015d770 2023-08-09T13:47:33.710 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:33.705+0000 7f83f5133640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f83f01634a0 0x7f83f00c6d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:47:33.711 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:33.705+0000 7f83f5133640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f83f01634a0 0x7f83f00c6d00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:53236/0 (socket says 172.21.15.43:53236) 2023-08-09T13:47:33.711 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:33.705+0000 7f83f5133640 1 -- 172.21.15.43:0/2256136176 learned_addr learned my addr 172.21.15.43:0/2256136176 (peer_addr_for_me v2:172.21.15.43:0/0) 2023-08-09T13:47:33.711 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:33.705+0000 7f83effff640 1 --2- 172.21.15.43:0/2256136176 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f83f015d770 0x7f83f00b1480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:47:33.711 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:33.705+0000 7f83f4932640 1 --2- 172.21.15.43:0/2256136176 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f83e00052a0 0x7f83f00b3030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:47:33.711 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:33.705+0000 7f83f5133640 1 -- 172.21.15.43:0/2256136176 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f83f015d770 msgr2=0x7f83f00b1480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:47:33.711 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:33.705+0000 7f83f5133640 1 --2- 172.21.15.43:0/2256136176 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f83f015d770 0x7f83f00b1480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:47:33.712 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:33.705+0000 7f83f5133640 1 -- 172.21.15.43:0/2256136176 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f83e00052a0 msgr2=0x7f83f00b3030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:47:33.712 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:33.705+0000 7f83f5133640 1 --2- 172.21.15.43:0/2256136176 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f83e00052a0 0x7f83f00b3030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:47:33.712 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:33.705+0000 7f83f5133640 1 -- 172.21.15.43:0/2256136176 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f83f00b3c80 con 0x7f83f01634a0 2023-08-09T13:47:33.713 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:33.705+0000 7f83f4932640 1 --2- 172.21.15.43:0/2256136176 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f83e00052a0 0x7f83f00b3030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-08-09T13:47:33.713 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:33.706+0000 7f83f5133640 1 --2- 172.21.15.43:0/2256136176 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f83f01634a0 0x7f83f00c6d00 secure :-1 s=READY pgs=704 cs=0 l=1 rev1=1 crypto rx=0x7f83e802bf50 tx=0x7f83e8003a70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-09T13:47:33.713 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:33.706+0000 7f83ed7fa640 1 -- 172.21.15.43:0/2256136176 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f83e803e070 con 0x7f83f01634a0 2023-08-09T13:47:33.713 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:33.706+0000 7f83ed7fa640 1 -- 172.21.15.43:0/2256136176 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f83e80079a0 con 0x7f83f01634a0 2023-08-09T13:47:33.714 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:33.706+0000 7f83ed7fa640 1 -- 172.21.15.43:0/2256136176 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f83e8042630 con 0x7f83f01634a0 2023-08-09T13:47:33.714 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:33.706+0000 7f83f6b96640 1 -- 172.21.15.43:0/2256136176 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f83f00b0f60 con 0x7f83f01634a0 2023-08-09T13:47:33.714 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:33.706+0000 7f83f6b96640 1 -- 172.21.15.43:0/2256136176 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f83f00b19c0 con 0x7f83f01634a0 2023-08-09T13:47:33.714 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:33.707+0000 7f83f6b96640 1 -- 172.21.15.43:0/2256136176 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f83f00b1150 con 0x7f83f01634a0 2023-08-09T13:47:33.716 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:33.708+0000 7f83ed7fa640 1 -- 172.21.15.43:0/2256136176 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 13) v1 ==== 88529+0+0 (secure 0 0 0) 0x7f83e8038400 con 0x7f83f01634a0 2023-08-09T13:47:33.716 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:33.709+0000 7f83ed7fa640 1 --2- 172.21.15.43:0/2256136176 >> [v2:172.21.15.203:6832/998005540,v1:172.21.15.203:6833/998005540] conn(0x7f83d406a280 0x7f83d406c740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:47:33.716 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:33.709+0000 7f83ed7fa640 1 -- 172.21.15.43:0/2256136176 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(42..42 src has 1..42) v4 ==== 6698+0+0 (secure 0 0 0) 0x7f83e80aa0f0 con 0x7f83f01634a0 2023-08-09T13:47:33.717 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:33.710+0000 7f83f4932640 1 --2- 172.21.15.43:0/2256136176 >> [v2:172.21.15.203:6832/998005540,v1:172.21.15.203:6833/998005540] conn(0x7f83d406a280 0x7f83d406c740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:47:33.717 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:33.711+0000 7f83f4932640 1 --2- 172.21.15.43:0/2256136176 >> [v2:172.21.15.203:6832/998005540,v1:172.21.15.203:6833/998005540] conn(0x7f83d406a280 0x7f83d406c740 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f83f0160800 tx=0x7f83e4008040 comp rx=0 tx=0).ready entity=mgr.5003 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-09T13:47:33.717 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:33.713+0000 7f83ed7fa640 1 -- 172.21.15.43:0/2256136176 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178066 (secure 0 0 0) 0x7f83e8075280 con 0x7f83f01634a0 2023-08-09T13:47:33.932 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:33.931+0000 7f83f6b96640 1 -- 172.21.15.43:0/2256136176 --> [v2:172.21.15.203:6832/998005540,v1:172.21.15.203:6833/998005540] -- mgr_command(tid 0: {"prefix": "fs snap-schedule status", "fs": "cephfs", "path": "/snap_test_dir1/dir8", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f83f006a500 con 0x7f83d406a280 2023-08-09T13:47:33.951 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:33.950+0000 7f83ed7fa640 1 -- 172.21.15.43:0/2256136176 <== mgr.5003 v2:172.21.15.203:6832/998005540 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+345 (secure 0 0 0) 0x7f83e4009180 con 0x7f83d406a280 2023-08-09T13:47:33.952 INFO:teuthology.orchestra.run.smithi043.stdout: 2023-08-09T13:47:33.952 INFO:teuthology.orchestra.run.smithi043.stdout:[{"fs": "cephfs", "subvol": null, "path": "/snap_test_dir1/dir8", "rel_path": "/snap_test_dir1/dir8", "schedule": "1M", "retention": {}, "start": "2023-08-09T00:00:00", "created": "2023-08-09T13:36:34", "first": "2023-08-09T13:37:00", "last": "2023-08-09T13:47:00", "last_pruned": null, "created_count": 11, "pruned_count": 0, "active": false}] 2023-08-09T13:47:33.953 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:33.952+0000 7f83f6b96640 1 -- 172.21.15.43:0/2256136176 >> [v2:172.21.15.203:6832/998005540,v1:172.21.15.203:6833/998005540] conn(0x7f83d406a280 msgr2=0x7f83d406c740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:47:33.954 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:33.952+0000 7f83f6b96640 1 --2- 172.21.15.43:0/2256136176 >> [v2:172.21.15.203:6832/998005540,v1:172.21.15.203:6833/998005540] conn(0x7f83d406a280 0x7f83d406c740 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f83f0160800 tx=0x7f83e4008040 comp rx=0 tx=0).stop 2023-08-09T13:47:33.954 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:33.952+0000 7f83f6b96640 1 -- 172.21.15.43:0/2256136176 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f83f01634a0 msgr2=0x7f83f00c6d00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:47:33.954 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:33.952+0000 7f83f6b96640 1 --2- 172.21.15.43:0/2256136176 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f83f01634a0 0x7f83f00c6d00 secure :-1 s=READY pgs=704 cs=0 l=1 rev1=1 crypto rx=0x7f83e802bf50 tx=0x7f83e8003a70 comp rx=0 tx=0).stop 2023-08-09T13:47:33.955 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:33.952+0000 7f83f6b96640 1 -- 172.21.15.43:0/2256136176 shutdown_connections 2023-08-09T13:47:33.955 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:33.952+0000 7f83f6b96640 1 --2- 172.21.15.43:0/2256136176 >> [v2:172.21.15.203:6832/998005540,v1:172.21.15.203:6833/998005540] conn(0x7f83d406a280 0x7f83d406c740 unknown :-1 s=CLOSED pgs=61 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:47:33.955 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:33.952+0000 7f83f6b96640 1 --2- 172.21.15.43:0/2256136176 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f83f01634a0 0x7f83f00c6d00 unknown :-1 s=CLOSED pgs=704 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:47:33.955 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:33.952+0000 7f83f6b96640 1 --2- 172.21.15.43:0/2256136176 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f83f015d770 0x7f83f00b1480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:47:33.955 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:33.952+0000 7f83f6b96640 1 --2- 172.21.15.43:0/2256136176 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f83e00052a0 0x7f83f00b3030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:47:33.955 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:33.952+0000 7f83f6b96640 1 -- 172.21.15.43:0/2256136176 >> 172.21.15.43:0/2256136176 conn(0x7f83f006b990 msgr2=0x7f83f00aff60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-09T13:47:33.955 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:33.953+0000 7f83f6b96640 1 -- 172.21.15.43:0/2256136176 shutdown_connections 2023-08-09T13:47:33.956 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:33.953+0000 7f83f6b96640 1 -- 172.21.15.43:0/2256136176 wait complete. 2023-08-09T13:47:33.973 DEBUG:tasks.cephfs.test_snap_schedules:res= [{"fs": "cephfs", "subvol": null, "path": "/snap_test_dir1/dir8", "rel_path": "/snap_test_dir1/dir8", "schedule": "1M", "retention": {}, "start": "2023-08-09T00:00:00", "created": "2023-08-09T13:36:34", "first": "2023-08-09T13:37:00", "last": "2023-08-09T13:47:00", "last_pruned": null, "created_count": 11, "pruned_count": 0, "active": false}] 2023-08-09T13:47:33.973 DEBUG:tasks.cephfs.test_snap_schedules:json_res: {'fs': 'cephfs', 'subvol': None, 'path': '/snap_test_dir1/dir8', 'rel_path': '/snap_test_dir1/dir8', 'schedule': '1M', 'retention': {}, 'start': '2023-08-09T00:00:00', 'created': '2023-08-09T13:36:34', 'first': '2023-08-09T13:37:00', 'last': '2023-08-09T13:47:00', 'last_pruned': None, 'created_count': 11, 'pruned_count': 0, 'active': False} 2023-08-09T13:47:33.973 DEBUG:tasks.cephfs.test_snap_schedules:fs_count: 11 2023-08-09T13:47:33.974 DEBUG:tasks.cephfs.test_snap_schedules:db_count: 11 2023-08-09T13:47:33.974 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-09T13:47:33.974 DEBUG:teuthology.orchestra.run.smithi043:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/dir9/.snap) 2023-08-09T13:47:34.045 INFO:teuthology.orchestra.run.smithi043.stdout:scheduled-2023-08-09-13_37_01_UTC 2023-08-09T13:47:34.046 INFO:teuthology.orchestra.run.smithi043.stdout:scheduled-2023-08-09-13_38_00_UTC 2023-08-09T13:47:34.046 INFO:teuthology.orchestra.run.smithi043.stdout:scheduled-2023-08-09-13_39_00_UTC 2023-08-09T13:47:34.046 INFO:teuthology.orchestra.run.smithi043.stdout:scheduled-2023-08-09-13_40_00_UTC 2023-08-09T13:47:34.046 INFO:teuthology.orchestra.run.smithi043.stdout:scheduled-2023-08-09-13_41_00_UTC 2023-08-09T13:47:34.046 INFO:teuthology.orchestra.run.smithi043.stdout:scheduled-2023-08-09-13_42_00_UTC 2023-08-09T13:47:34.046 INFO:teuthology.orchestra.run.smithi043.stdout:scheduled-2023-08-09-13_43_00_UTC 2023-08-09T13:47:34.046 INFO:teuthology.orchestra.run.smithi043.stdout:scheduled-2023-08-09-13_44_00_UTC 2023-08-09T13:47:34.047 INFO:teuthology.orchestra.run.smithi043.stdout:scheduled-2023-08-09-13_45_00_UTC 2023-08-09T13:47:34.047 INFO:teuthology.orchestra.run.smithi043.stdout:scheduled-2023-08-09-13_46_00_UTC 2023-08-09T13:47:34.047 INFO:teuthology.orchestra.run.smithi043.stdout:scheduled-2023-08-09-13_47_00_UTC 2023-08-09T13:47:34.048 DEBUG:tasks.cephfs.test_snap_schedules:snapshots: ['scheduled-2023-08-09-13_37_01_UTC', 'scheduled-2023-08-09-13_38_00_UTC', 'scheduled-2023-08-09-13_39_00_UTC', 'scheduled-2023-08-09-13_40_00_UTC', 'scheduled-2023-08-09-13_41_00_UTC', 'scheduled-2023-08-09-13_42_00_UTC', 'scheduled-2023-08-09-13_43_00_UTC', 'scheduled-2023-08-09-13_44_00_UTC', 'scheduled-2023-08-09-13_45_00_UTC', 'scheduled-2023-08-09-13_46_00_UTC', 'scheduled-2023-08-09-13_47_00_UTC'] 2023-08-09T13:47:34.049 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs snap-schedule status --fs cephfs --format json /snap_test_dir1/dir9 2023-08-09T13:47:34.290 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:34.289+0000 7f260f59e640 1 Processor -- start 2023-08-09T13:47:34.291 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:34.289+0000 7f260f59e640 1 -- start start 2023-08-09T13:47:34.292 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:34.291+0000 7f260f59e640 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7f261015a140 0x7f261006ed40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:47:34.292 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:34.291+0000 7f260f59e640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f261006c0d0 0x7f26100cf900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:47:34.292 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:34.291+0000 7f260f59e640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f26100b1ad0 0x7f26100b1f60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:47:34.292 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:34.291+0000 7f260f59e640 1 -- --> v2:172.21.15.43:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f26100b24a0 con 0x7f261006c0d0 2023-08-09T13:47:34.293 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:34.291+0000 7f260f59e640 1 -- --> v2:172.21.15.203:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f26100cfe40 con 0x7f261015a140 2023-08-09T13:47:34.293 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:34.291+0000 7f260f59e640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f26100ce2d0 con 0x7f26100b1ad0 2023-08-09T13:47:34.293 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:34.291+0000 7f2605d9b640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f261006c0d0 0x7f26100cf900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:47:34.294 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:34.291+0000 7f2605d9b640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f261006c0d0 0x7f26100cf900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:53240/0 (socket says 172.21.15.43:53240) 2023-08-09T13:47:34.294 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:34.291+0000 7f2605d9b640 1 -- 172.21.15.43:0/4164839545 learned_addr learned my addr 172.21.15.43:0/4164839545 (peer_addr_for_me v2:172.21.15.43:0/0) 2023-08-09T13:47:34.294 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:34.292+0000 7f260ed9d640 1 --2- 172.21.15.43:0/4164839545 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f26100b1ad0 0x7f26100b1f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:47:34.294 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:34.292+0000 7f260e59c640 1 --2- 172.21.15.43:0/4164839545 >> v2:172.21.15.203:3300/0 conn(0x7f261015a140 0x7f261006ed40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:47:34.294 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:34.292+0000 7f2605d9b640 1 -- 172.21.15.43:0/4164839545 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f26100b1ad0 msgr2=0x7f26100b1f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-09T13:47:34.294 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:34.292+0000 7f2605d9b640 1 --2- 172.21.15.43:0/4164839545 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f26100b1ad0 0x7f26100b1f60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:47:34.295 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:34.292+0000 7f2605d9b640 1 -- 172.21.15.43:0/4164839545 >> v2:172.21.15.203:3300/0 conn(0x7f261015a140 msgr2=0x7f261006ed40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-09T13:47:34.295 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:34.292+0000 7f2605d9b640 1 --2- 172.21.15.43:0/4164839545 >> v2:172.21.15.203:3300/0 conn(0x7f261015a140 0x7f261006ed40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:47:34.295 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:34.292+0000 7f2605d9b640 1 -- 172.21.15.43:0/4164839545 --> v2:172.21.15.43:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f261006c8e0 con 0x7f261006c0d0 2023-08-09T13:47:34.296 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:34.293+0000 7f2605d9b640 1 --2- 172.21.15.43:0/4164839545 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f261006c0d0 0x7f26100cf900 secure :-1 s=READY pgs=705 cs=0 l=1 rev1=1 crypto rx=0x7f26100cffb0 tx=0x7f25f802b880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cdfa5ff6297e02e1 server_cookie=0 in_seq=0 out_seq=0 2023-08-09T13:47:34.296 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:34.294+0000 7f260dd9b640 1 -- 172.21.15.43:0/4164839545 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f25f80317e0 con 0x7f261006c0d0 2023-08-09T13:47:34.296 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:34.294+0000 7f260dd9b640 1 -- 172.21.15.43:0/4164839545 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f25f8031940 con 0x7f261006c0d0 2023-08-09T13:47:34.296 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:34.294+0000 7f260dd9b640 1 -- 172.21.15.43:0/4164839545 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f25f8031cd0 con 0x7f261006c0d0 2023-08-09T13:47:34.297 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:34.294+0000 7f260f59e640 1 -- 172.21.15.43:0/4164839545 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f261006c0d0 msgr2=0x7f26100cf900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:47:34.297 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:34.294+0000 7f260f59e640 1 --2- 172.21.15.43:0/4164839545 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f261006c0d0 0x7f26100cf900 secure :-1 s=READY pgs=705 cs=0 l=1 rev1=1 crypto rx=0x7f26100cffb0 tx=0x7f25f802b880 comp rx=0 tx=0).stop 2023-08-09T13:47:34.297 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:34.294+0000 7f260f59e640 1 -- 172.21.15.43:0/4164839545 shutdown_connections 2023-08-09T13:47:34.297 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:34.294+0000 7f260f59e640 1 --2- 172.21.15.43:0/4164839545 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f261006c0d0 0x7f26100cf900 unknown :-1 s=CLOSED pgs=705 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:47:34.297 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:34.294+0000 7f260f59e640 1 --2- 172.21.15.43:0/4164839545 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f26100b1ad0 0x7f26100b1f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:47:34.297 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:34.294+0000 7f260f59e640 1 --2- 172.21.15.43:0/4164839545 >> v2:172.21.15.203:3300/0 conn(0x7f261015a140 0x7f261006ed40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:47:34.297 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:34.294+0000 7f260f59e640 1 -- 172.21.15.43:0/4164839545 >> 172.21.15.43:0/4164839545 conn(0x7f26100ccad0 msgr2=0x7f26100cceb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-09T13:47:34.297 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:34.294+0000 7f260f59e640 1 -- 172.21.15.43:0/4164839545 shutdown_connections 2023-08-09T13:47:34.298 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:34.294+0000 7f260f59e640 1 -- 172.21.15.43:0/4164839545 wait complete. 2023-08-09T13:47:34.298 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:34.296+0000 7f260f59e640 1 Processor -- start 2023-08-09T13:47:34.298 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:34.296+0000 7f260f59e640 1 -- start start 2023-08-09T13:47:34.299 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:34.297+0000 7f260f59e640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f261006c0d0 0x7f261014e6f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:47:34.299 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:34.297+0000 7f260f59e640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f26100b1ad0 0x7f2610155cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:47:34.299 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:34.297+0000 7f260f59e640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f261015a140 0x7f261015b740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:47:34.299 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:34.297+0000 7f260f59e640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2610155050 con 0x7f261006c0d0 2023-08-09T13:47:34.300 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:34.297+0000 7f260f59e640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f261015a6e0 con 0x7f261015a140 2023-08-09T13:47:34.300 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:34.297+0000 7f260f59e640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f261014ec30 con 0x7f26100b1ad0 2023-08-09T13:47:34.300 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:34.298+0000 7f260e59c640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f261006c0d0 0x7f261014e6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:47:34.300 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:34.298+0000 7f260e59c640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f261006c0d0 0x7f261014e6f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:53256/0 (socket says 172.21.15.43:53256) 2023-08-09T13:47:34.300 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:34.298+0000 7f260e59c640 1 -- 172.21.15.43:0/279881634 learned_addr learned my addr 172.21.15.43:0/279881634 (peer_addr_for_me v2:172.21.15.43:0/0) 2023-08-09T13:47:34.301 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:34.298+0000 7f2605d9b640 1 --2- 172.21.15.43:0/279881634 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f26100b1ad0 0x7f2610155cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:47:34.302 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:34.298+0000 7f260ed9d640 1 --2- 172.21.15.43:0/279881634 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f261015a140 0x7f261015b740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:47:34.302 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:34.298+0000 7f260e59c640 1 -- 172.21.15.43:0/279881634 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f26100b1ad0 msgr2=0x7f2610155cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:47:34.302 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:34.298+0000 7f260e59c640 1 --2- 172.21.15.43:0/279881634 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f26100b1ad0 0x7f2610155cf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:47:34.302 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:34.298+0000 7f260e59c640 1 -- 172.21.15.43:0/279881634 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f261015a140 msgr2=0x7f261015b740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:47:34.302 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:34.298+0000 7f260e59c640 1 --2- 172.21.15.43:0/279881634 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f261015a140 0x7f261015b740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:47:34.302 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:34.298+0000 7f260e59c640 1 -- 172.21.15.43:0/279881634 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f25f80094d0 con 0x7f261006c0d0 2023-08-09T13:47:34.303 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:34.298+0000 7f2605d9b640 1 --2- 172.21.15.43:0/279881634 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f26100b1ad0 0x7f2610155cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-08-09T13:47:34.303 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:34.298+0000 7f260ed9d640 1 --2- 172.21.15.43:0/279881634 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f261015a140 0x7f261015b740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-08-09T13:47:34.303 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:34.299+0000 7f260e59c640 1 --2- 172.21.15.43:0/279881634 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f261006c0d0 0x7f261014e6f0 secure :-1 s=READY pgs=706 cs=0 l=1 rev1=1 crypto rx=0x7f260000d800 tx=0x7f260000dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-09T13:47:34.303 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:34.299+0000 7f26077fe640 1 -- 172.21.15.43:0/279881634 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f260000b6c0 con 0x7f261006c0d0 2023-08-09T13:47:34.303 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:34.299+0000 7f26077fe640 1 -- 172.21.15.43:0/279881634 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f260000b820 con 0x7f261006c0d0 2023-08-09T13:47:34.304 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:34.299+0000 7f26077fe640 1 -- 172.21.15.43:0/279881634 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f260000bbb0 con 0x7f261006c0d0 2023-08-09T13:47:34.304 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:34.299+0000 7f260f59e640 1 -- 172.21.15.43:0/279881634 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f261015a980 con 0x7f261006c0d0 2023-08-09T13:47:34.304 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:34.299+0000 7f260f59e640 1 -- 172.21.15.43:0/279881634 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f261015af40 con 0x7f261006c0d0 2023-08-09T13:47:34.304 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:34.300+0000 7f260f59e640 1 -- 172.21.15.43:0/279881634 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f26100739f0 con 0x7f261006c0d0 2023-08-09T13:47:34.305 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:34.301+0000 7f26077fe640 1 -- 172.21.15.43:0/279881634 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 13) v1 ==== 88529+0+0 (secure 0 0 0) 0x7f260000bd10 con 0x7f261006c0d0 2023-08-09T13:47:34.305 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:34.302+0000 7f26077fe640 1 --2- 172.21.15.43:0/279881634 >> [v2:172.21.15.203:6832/998005540,v1:172.21.15.203:6833/998005540] conn(0x7f25f006a250 0x7f25f006c710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:47:34.305 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:34.302+0000 7f26077fe640 1 -- 172.21.15.43:0/279881634 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(42..42 src has 1..42) v4 ==== 6698+0+0 (secure 0 0 0) 0x7f2600089140 con 0x7f261006c0d0 2023-08-09T13:47:34.305 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:34.303+0000 7f2605d9b640 1 --2- 172.21.15.43:0/279881634 >> [v2:172.21.15.203:6832/998005540,v1:172.21.15.203:6833/998005540] conn(0x7f25f006a250 0x7f25f006c710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:47:34.305 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:34.304+0000 7f2605d9b640 1 --2- 172.21.15.43:0/279881634 >> [v2:172.21.15.203:6832/998005540,v1:172.21.15.203:6833/998005540] conn(0x7f25f006a250 0x7f25f006c710 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f25f8036000 tx=0x7f25f803d040 comp rx=0 tx=0).ready entity=mgr.5003 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-09T13:47:34.308 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:34.306+0000 7f26077fe640 1 -- 172.21.15.43:0/279881634 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178066 (secure 0 0 0) 0x7f2600054320 con 0x7f261006c0d0 2023-08-09T13:47:34.522 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:34.521+0000 7f260f59e640 1 -- 172.21.15.43:0/279881634 --> [v2:172.21.15.203:6832/998005540,v1:172.21.15.203:6833/998005540] -- mgr_command(tid 0: {"prefix": "fs snap-schedule status", "fs": "cephfs", "path": "/snap_test_dir1/dir9", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f2610160b00 con 0x7f25f006a250 2023-08-09T13:47:34.543 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:34.541+0000 7f26077fe640 1 -- 172.21.15.43:0/279881634 <== mgr.5003 v2:172.21.15.203:6832/998005540 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+345 (secure 0 0 0) 0x7f25f8031c20 con 0x7f25f006a250 2023-08-09T13:47:34.543 INFO:teuthology.orchestra.run.smithi043.stdout: 2023-08-09T13:47:34.543 INFO:teuthology.orchestra.run.smithi043.stdout:[{"fs": "cephfs", "subvol": null, "path": "/snap_test_dir1/dir9", "rel_path": "/snap_test_dir1/dir9", "schedule": "1M", "retention": {}, "start": "2023-08-09T00:00:00", "created": "2023-08-09T13:36:35", "first": "2023-08-09T13:37:01", "last": "2023-08-09T13:47:00", "last_pruned": null, "created_count": 11, "pruned_count": 0, "active": false}] 2023-08-09T13:47:34.545 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:34.544+0000 7f260f59e640 1 -- 172.21.15.43:0/279881634 >> [v2:172.21.15.203:6832/998005540,v1:172.21.15.203:6833/998005540] conn(0x7f25f006a250 msgr2=0x7f25f006c710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:47:34.546 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:34.544+0000 7f260f59e640 1 --2- 172.21.15.43:0/279881634 >> [v2:172.21.15.203:6832/998005540,v1:172.21.15.203:6833/998005540] conn(0x7f25f006a250 0x7f25f006c710 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f25f8036000 tx=0x7f25f803d040 comp rx=0 tx=0).stop 2023-08-09T13:47:34.546 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:34.544+0000 7f260f59e640 1 -- 172.21.15.43:0/279881634 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f261006c0d0 msgr2=0x7f261014e6f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:47:34.546 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:34.544+0000 7f260f59e640 1 --2- 172.21.15.43:0/279881634 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f261006c0d0 0x7f261014e6f0 secure :-1 s=READY pgs=706 cs=0 l=1 rev1=1 crypto rx=0x7f260000d800 tx=0x7f260000dcc0 comp rx=0 tx=0).stop 2023-08-09T13:47:34.546 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:34.544+0000 7f260f59e640 1 -- 172.21.15.43:0/279881634 shutdown_connections 2023-08-09T13:47:34.546 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:34.544+0000 7f260f59e640 1 --2- 172.21.15.43:0/279881634 >> [v2:172.21.15.203:6832/998005540,v1:172.21.15.203:6833/998005540] conn(0x7f25f006a250 0x7f25f006c710 unknown :-1 s=CLOSED pgs=62 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:47:34.547 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:34.544+0000 7f260f59e640 1 --2- 172.21.15.43:0/279881634 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f261015a140 0x7f261015b740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:47:34.547 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:34.544+0000 7f260f59e640 1 --2- 172.21.15.43:0/279881634 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f26100b1ad0 0x7f2610155cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:47:34.547 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:34.544+0000 7f260f59e640 1 --2- 172.21.15.43:0/279881634 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f261006c0d0 0x7f261014e6f0 unknown :-1 s=CLOSED pgs=706 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:47:34.547 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:34.544+0000 7f260f59e640 1 -- 172.21.15.43:0/279881634 >> 172.21.15.43:0/279881634 conn(0x7f26100ccad0 msgr2=0x7f26101597c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-09T13:47:34.547 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:34.544+0000 7f260f59e640 1 -- 172.21.15.43:0/279881634 shutdown_connections 2023-08-09T13:47:34.547 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:34.544+0000 7f260f59e640 1 -- 172.21.15.43:0/279881634 wait complete. 2023-08-09T13:47:34.564 DEBUG:tasks.cephfs.test_snap_schedules:res= [{"fs": "cephfs", "subvol": null, "path": "/snap_test_dir1/dir9", "rel_path": "/snap_test_dir1/dir9", "schedule": "1M", "retention": {}, "start": "2023-08-09T00:00:00", "created": "2023-08-09T13:36:35", "first": "2023-08-09T13:37:01", "last": "2023-08-09T13:47:00", "last_pruned": null, "created_count": 11, "pruned_count": 0, "active": false}] 2023-08-09T13:47:34.564 DEBUG:tasks.cephfs.test_snap_schedules:json_res: {'fs': 'cephfs', 'subvol': None, 'path': '/snap_test_dir1/dir9', 'rel_path': '/snap_test_dir1/dir9', 'schedule': '1M', 'retention': {}, 'start': '2023-08-09T00:00:00', 'created': '2023-08-09T13:36:35', 'first': '2023-08-09T13:37:01', 'last': '2023-08-09T13:47:00', 'last_pruned': None, 'created_count': 11, 'pruned_count': 0, 'active': False} 2023-08-09T13:47:34.565 DEBUG:tasks.cephfs.test_snap_schedules:fs_count: 11 2023-08-09T13:47:34.565 DEBUG:tasks.cephfs.test_snap_schedules:db_count: 11 2023-08-09T13:47:34.565 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs snap-schedule remove --fs cephfs /snap_test_dir1/dir0 1M 2023-08-09T13:47:34.784 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:34.782+0000 7f77fd95b640 1 Processor -- start 2023-08-09T13:47:34.785 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:34.783+0000 7f77fd95b640 1 -- start start 2023-08-09T13:47:34.788 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:34.784+0000 7f77fd95b640 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7f77f806fb70 0x7f77f80caab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:47:34.788 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:34.784+0000 7f77fd95b640 1 -- --> v1:172.21.15.43:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f77f815b9b0 con 0x7f77f80ca370 2023-08-09T13:47:34.788 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:34.784+0000 7f77fd95b640 1 -- --> v1:172.21.15.203:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f77f815ae10 con 0x7f77f80ce4c0 2023-08-09T13:47:34.789 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:34.784+0000 7f77fd95b640 1 -- --> v2:172.21.15.203:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f77f8154e70 con 0x7f77f806fb70 2023-08-09T13:47:34.789 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:34.784+0000 7f77f67fc640 1 --1- >> v1:172.21.15.43:6789/0 conn(0x7f77f80ca370 0x7f77f815b2a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.43:6789/0 says I am v1:172.21.15.43:49632/0 (socket says 172.21.15.43:49632) 2023-08-09T13:47:34.789 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:34.784+0000 7f77f67fc640 1 -- 172.21.15.43:0/4130159487 learned_addr learned my addr 172.21.15.43:0/4130159487 (peer_addr_for_me v1:172.21.15.43:0/0) 2023-08-09T13:47:34.789 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:34.785+0000 7f77f5ffb640 1 -- 172.21.15.43:0/4130159487 <== mon.0 v1:172.21.15.43:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3247756891 0 0) 0x7f77ec001110 con 0x7f77f80ca370 2023-08-09T13:47:34.790 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:34.785+0000 7f77f5ffb640 1 -- 172.21.15.43:0/4130159487 --> v1:172.21.15.43:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f77e0003520 con 0x7f77f80ca370 2023-08-09T13:47:34.790 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:34.785+0000 7f77f6ffd640 1 --2- 172.21.15.43:0/4130159487 >> v2:172.21.15.203:3300/0 conn(0x7f77f806fb70 0x7f77f80caab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:47:34.791 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:34.785+0000 7f77f5ffb640 1 -- 172.21.15.43:0/4130159487 <== mon.0 v1:172.21.15.43:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 722519387 0 0) 0x7f77e0003520 con 0x7f77f80ca370 2023-08-09T13:47:34.791 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:34.785+0000 7f77f5ffb640 1 -- 172.21.15.43:0/4130159487 >> v2:172.21.15.203:3300/0 conn(0x7f77f806fb70 msgr2=0x7f77f80caab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-09T13:47:34.792 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:34.785+0000 7f77f5ffb640 1 --2- 172.21.15.43:0/4130159487 >> v2:172.21.15.203:3300/0 conn(0x7f77f806fb70 0x7f77f80caab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:47:34.792 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:34.785+0000 7f77f5ffb640 1 -- 172.21.15.43:0/4130159487 >> v1:172.21.15.203:6789/0 conn(0x7f77f80ce4c0 legacy=0x7f77f80c86f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-09T13:47:34.792 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:34.785+0000 7f77f5ffb640 1 -- 172.21.15.43:0/4130159487 --> v1:172.21.15.43:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f77f806ff50 con 0x7f77f80ca370 2023-08-09T13:47:34.792 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:34.785+0000 7f77f5ffb640 1 -- 172.21.15.43:0/4130159487 <== mon.0 v1:172.21.15.43:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1285150740 0 0) 0x7f77ec001800 con 0x7f77f80ca370 2023-08-09T13:47:34.793 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:34.785+0000 7f77f5ffb640 1 -- 172.21.15.43:0/4130159487 >> v1:172.21.15.43:6789/0 conn(0x7f77f80ca370 legacy=0x7f77f815b2a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:47:34.793 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:34.785+0000 7f77f5ffb640 1 --2- 172.21.15.43:0/4130159487 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f77e00039e0 0x7f77e0003de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:47:34.793 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:34.785+0000 7f77f5ffb640 1 --2- 172.21.15.43:0/4130159487 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f77e00047c0 0x7f77e0004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:47:34.793 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:34.785+0000 7f77f5ffb640 1 --2- 172.21.15.43:0/4130159487 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f77e0005180 0x7f77e000b980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:47:34.794 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:34.785+0000 7f77f5ffb640 1 -- 172.21.15.43:0/4130159487 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f77f8154e70 con 0x7f77e00047c0 2023-08-09T13:47:34.794 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:34.785+0000 7f77f5ffb640 1 -- 172.21.15.43:0/4130159487 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f77e000c080 con 0x7f77e00039e0 2023-08-09T13:47:34.794 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:34.785+0000 7f77f5ffb640 1 -- 172.21.15.43:0/4130159487 --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f77e000c240 con 0x7f77e0005180 2023-08-09T13:47:34.795 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:34.786+0000 7f77f67fc640 1 --2- 172.21.15.43:0/4130159487 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f77e00047c0 0x7f77e0004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:47:34.795 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:34.786+0000 7f77f67fc640 1 -- 172.21.15.43:0/4130159487 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f77e0005180 msgr2=0x7f77e000b980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-09T13:47:34.795 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:34.786+0000 7f77f67fc640 1 --2- 172.21.15.43:0/4130159487 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f77e0005180 0x7f77e000b980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:47:34.795 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:34.786+0000 7f77f67fc640 1 -- 172.21.15.43:0/4130159487 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f77e00039e0 msgr2=0x7f77e0003de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-09T13:47:34.795 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:34.786+0000 7f77f67fc640 1 --2- 172.21.15.43:0/4130159487 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f77e00039e0 0x7f77e0003de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:47:34.796 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:34.786+0000 7f77f67fc640 1 -- 172.21.15.43:0/4130159487 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f77e0003520 con 0x7f77e00047c0 2023-08-09T13:47:34.796 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:34.786+0000 7f77f77fe640 1 -- 172.21.15.43:0/4130159487 reap_dead start 2023-08-09T13:47:34.800 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:34.787+0000 7f77f67fc640 1 --2- 172.21.15.43:0/4130159487 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f77e00047c0 0x7f77e0004c40 secure :-1 s=READY pgs=708 cs=0 l=1 rev1=1 crypto rx=0x7f77ec00c240 tx=0x7f77e00057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=26e96212aca67c0f server_cookie=0 in_seq=0 out_seq=0 2023-08-09T13:47:34.800 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:34.787+0000 7f77f5ffb640 1 -- 172.21.15.43:0/4130159487 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f77ec005ad0 con 0x7f77e00047c0 2023-08-09T13:47:34.801 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:34.787+0000 7f77f5ffb640 1 -- 172.21.15.43:0/4130159487 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f77ec005c30 con 0x7f77e00047c0 2023-08-09T13:47:34.801 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:34.787+0000 7f77f5ffb640 1 -- 172.21.15.43:0/4130159487 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f77ec02c8b0 con 0x7f77e00047c0 2023-08-09T13:47:34.801 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:34.788+0000 7f77fd95b640 1 -- 172.21.15.43:0/4130159487 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f77e00047c0 msgr2=0x7f77e0004c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:47:34.802 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:34.788+0000 7f77fd95b640 1 --2- 172.21.15.43:0/4130159487 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f77e00047c0 0x7f77e0004c40 secure :-1 s=READY pgs=708 cs=0 l=1 rev1=1 crypto rx=0x7f77ec00c240 tx=0x7f77e00057f0 comp rx=0 tx=0).stop 2023-08-09T13:47:34.802 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:34.788+0000 7f77fd95b640 1 -- 172.21.15.43:0/4130159487 shutdown_connections 2023-08-09T13:47:34.802 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:34.788+0000 7f77fd95b640 1 --2- 172.21.15.43:0/4130159487 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f77e00047c0 0x7f77e0004c40 unknown :-1 s=CLOSED pgs=708 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:47:34.802 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:34.788+0000 7f77fd95b640 1 -- 172.21.15.43:0/4130159487 >> 172.21.15.43:0/4130159487 conn(0x7f77f806b990 msgr2=0x7f77f806bda0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-09T13:47:34.803 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:34.788+0000 7f77fd95b640 1 -- 172.21.15.43:0/4130159487 shutdown_connections 2023-08-09T13:47:34.803 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:34.788+0000 7f77fd95b640 1 -- 172.21.15.43:0/4130159487 wait complete. 2023-08-09T13:47:34.803 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:34.790+0000 7f77fd95b640 1 Processor -- start 2023-08-09T13:47:34.803 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:34.790+0000 7f77fd95b640 1 -- start start 2023-08-09T13:47:34.804 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:34.791+0000 7f77fd95b640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f77e00047c0 0x7f77f814b580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:47:34.804 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:34.791+0000 7f77fd95b640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f77f80c2fc0 0x7f77f80c37d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:47:34.804 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:34.791+0000 7f77fd95b640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f77f80c3d10 0x7f77f80b1f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:47:34.804 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:34.791+0000 7f77fd95b640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f77f814bac0 con 0x7f77e00047c0 2023-08-09T13:47:34.805 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:34.791+0000 7f77fd95b640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f77f80c1f20 con 0x7f77f80c3d10 2023-08-09T13:47:34.805 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:34.791+0000 7f77fd95b640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f77f80c2200 con 0x7f77f80c2fc0 2023-08-09T13:47:34.805 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:34.791+0000 7f77f6ffd640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f77e00047c0 0x7f77f814b580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:47:34.805 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:34.791+0000 7f77f6ffd640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f77e00047c0 0x7f77f814b580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:53276/0 (socket says 172.21.15.43:53276) 2023-08-09T13:47:34.806 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:34.792+0000 7f77f6ffd640 1 -- 172.21.15.43:0/1197068200 learned_addr learned my addr 172.21.15.43:0/1197068200 (peer_addr_for_me v2:172.21.15.43:0/0) 2023-08-09T13:47:34.806 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:34.792+0000 7f77f6ffd640 1 -- 172.21.15.43:0/1197068200 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f77f80c2fc0 msgr2=0x7f77f80c37d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:47:34.806 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:34.792+0000 7f77f6ffd640 1 --2- 172.21.15.43:0/1197068200 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f77f80c2fc0 0x7f77f80c37d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:47:34.806 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:34.792+0000 7f77f6ffd640 1 -- 172.21.15.43:0/1197068200 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f77f80c3d10 msgr2=0x7f77f80b1f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:47:34.807 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:34.792+0000 7f77f6ffd640 1 --2- 172.21.15.43:0/1197068200 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f77f80c3d10 0x7f77f80b1f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:47:34.807 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:34.792+0000 7f77f6ffd640 1 -- 172.21.15.43:0/1197068200 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f77f80c24e0 con 0x7f77e00047c0 2023-08-09T13:47:34.807 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:34.792+0000 7f77f6ffd640 1 --2- 172.21.15.43:0/1197068200 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f77e00047c0 0x7f77f814b580 secure :-1 s=READY pgs=709 cs=0 l=1 rev1=1 crypto rx=0x7f77e800e860 tx=0x7f77e800ed20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-09T13:47:34.807 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:34.793+0000 7f77dffff640 1 -- 172.21.15.43:0/1197068200 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f77e8009600 con 0x7f77e00047c0 2023-08-09T13:47:34.808 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:34.793+0000 7f77dffff640 1 -- 172.21.15.43:0/1197068200 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f77e800c940 con 0x7f77e00047c0 2023-08-09T13:47:34.808 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:34.793+0000 7f77dffff640 1 -- 172.21.15.43:0/1197068200 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f77e800ccd0 con 0x7f77e00047c0 2023-08-09T13:47:34.808 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:34.793+0000 7f77fd95b640 1 -- 172.21.15.43:0/1197068200 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f77f80c27c0 con 0x7f77e00047c0 2023-08-09T13:47:34.809 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:34.793+0000 7f77fd95b640 1 -- 172.21.15.43:0/1197068200 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f77f80c1bd0 con 0x7f77e00047c0 2023-08-09T13:47:34.809 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:34.794+0000 7f77fd95b640 1 -- 172.21.15.43:0/1197068200 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f77f80739f0 con 0x7f77e00047c0 2023-08-09T13:47:34.809 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:34.795+0000 7f77dffff640 1 -- 172.21.15.43:0/1197068200 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 13) v1 ==== 88529+0+0 (secure 0 0 0) 0x7f77e8013470 con 0x7f77e00047c0 2023-08-09T13:47:34.809 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:34.796+0000 7f77dffff640 1 --2- 172.21.15.43:0/1197068200 >> [v2:172.21.15.203:6832/998005540,v1:172.21.15.203:6833/998005540] conn(0x7f77e406a250 0x7f77e406c710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:47:34.809 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:34.796+0000 7f77dffff640 1 -- 172.21.15.43:0/1197068200 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(42..42 src has 1..42) v4 ==== 6698+0+0 (secure 0 0 0) 0x7f77e8088e70 con 0x7f77e00047c0 2023-08-09T13:47:34.809 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:34.796+0000 7f77f67fc640 1 --2- 172.21.15.43:0/1197068200 >> [v2:172.21.15.203:6832/998005540,v1:172.21.15.203:6833/998005540] conn(0x7f77e406a250 0x7f77e406c710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:47:34.810 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:34.798+0000 7f77f67fc640 1 --2- 172.21.15.43:0/1197068200 >> [v2:172.21.15.203:6832/998005540,v1:172.21.15.203:6833/998005540] conn(0x7f77e406a250 0x7f77e406c710 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f77ec034000 tx=0x7f77ec039040 comp rx=0 tx=0).ready entity=mgr.5003 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-09T13:47:34.810 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:34.800+0000 7f77dffff640 1 -- 172.21.15.43:0/1197068200 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178066 (secure 0 0 0) 0x7f77e8053fc0 con 0x7f77e00047c0 2023-08-09T13:47:35.016 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:35.014+0000 7f77fd95b640 1 -- 172.21.15.43:0/1197068200 --> [v2:172.21.15.203:6832/998005540,v1:172.21.15.203:6833/998005540] -- mgr_command(tid 0: {"prefix": "fs snap-schedule remove", "fs": "cephfs", "path": "/snap_test_dir1/dir0", "repeat": "1M", "target": ["mon-mgr", ""]}) v1 -- 0x7f77f80c4160 con 0x7f77e406a250 2023-08-09T13:47:35.220 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:35.219+0000 7f77dffff640 1 -- 172.21.15.43:0/1197068200 <== mgr.5003 v2:172.21.15.203:6832/998005540 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+47 (secure 0 0 0) 0x7f77f80c4160 con 0x7f77e406a250 2023-08-09T13:47:35.221 INFO:teuthology.orchestra.run.smithi043.stdout:Schedule removed for path /snap_test_dir1/dir0 2023-08-09T13:47:35.224 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:35.222+0000 7f77fd95b640 1 -- 172.21.15.43:0/1197068200 >> [v2:172.21.15.203:6832/998005540,v1:172.21.15.203:6833/998005540] conn(0x7f77e406a250 msgr2=0x7f77e406c710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:47:35.225 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:35.223+0000 7f77fd95b640 1 --2- 172.21.15.43:0/1197068200 >> [v2:172.21.15.203:6832/998005540,v1:172.21.15.203:6833/998005540] conn(0x7f77e406a250 0x7f77e406c710 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f77ec034000 tx=0x7f77ec039040 comp rx=0 tx=0).stop 2023-08-09T13:47:35.225 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:35.223+0000 7f77fd95b640 1 -- 172.21.15.43:0/1197068200 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f77e00047c0 msgr2=0x7f77f814b580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:47:35.225 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:35.223+0000 7f77fd95b640 1 --2- 172.21.15.43:0/1197068200 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f77e00047c0 0x7f77f814b580 secure :-1 s=READY pgs=709 cs=0 l=1 rev1=1 crypto rx=0x7f77e800e860 tx=0x7f77e800ed20 comp rx=0 tx=0).stop 2023-08-09T13:47:35.225 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:35.223+0000 7f77fd95b640 1 -- 172.21.15.43:0/1197068200 shutdown_connections 2023-08-09T13:47:35.225 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:35.223+0000 7f77fd95b640 1 --2- 172.21.15.43:0/1197068200 >> [v2:172.21.15.203:6832/998005540,v1:172.21.15.203:6833/998005540] conn(0x7f77e406a250 0x7f77e406c710 unknown :-1 s=CLOSED pgs=63 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:47:35.225 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:35.223+0000 7f77fd95b640 1 --2- 172.21.15.43:0/1197068200 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f77f80c3d10 0x7f77f80b1f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:47:35.226 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:35.223+0000 7f77fd95b640 1 --2- 172.21.15.43:0/1197068200 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f77f80c2fc0 0x7f77f80c37d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:47:35.226 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:35.223+0000 7f77fd95b640 1 --2- 172.21.15.43:0/1197068200 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f77e00047c0 0x7f77f814b580 unknown :-1 s=CLOSED pgs=709 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:47:35.226 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:35.223+0000 7f77fd95b640 1 -- 172.21.15.43:0/1197068200 >> 172.21.15.43:0/1197068200 conn(0x7f77f806b990 msgr2=0x7f77f8159680 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-09T13:47:35.226 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:35.223+0000 7f77fd95b640 1 -- 172.21.15.43:0/1197068200 shutdown_connections 2023-08-09T13:47:35.226 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:35.223+0000 7f77fd95b640 1 -- 172.21.15.43:0/1197068200 wait complete. 2023-08-09T13:47:35.246 DEBUG:tasks.cephfs.test_snap_schedules:res=Schedule removed for path /snap_test_dir1/dir0 2023-08-09T13:47:35.246 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-09T13:47:35.247 DEBUG:teuthology.orchestra.run.smithi043:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/dir0/.snap) 2023-08-09T13:47:35.316 INFO:teuthology.orchestra.run.smithi043.stdout:scheduled-2023-08-09-13_37_00_UTC 2023-08-09T13:47:35.316 INFO:teuthology.orchestra.run.smithi043.stdout:scheduled-2023-08-09-13_38_01_UTC 2023-08-09T13:47:35.316 INFO:teuthology.orchestra.run.smithi043.stdout:scheduled-2023-08-09-13_39_01_UTC 2023-08-09T13:47:35.316 INFO:teuthology.orchestra.run.smithi043.stdout:scheduled-2023-08-09-13_40_00_UTC 2023-08-09T13:47:35.316 INFO:teuthology.orchestra.run.smithi043.stdout:scheduled-2023-08-09-13_41_00_UTC 2023-08-09T13:47:35.316 INFO:teuthology.orchestra.run.smithi043.stdout:scheduled-2023-08-09-13_42_00_UTC 2023-08-09T13:47:35.317 INFO:teuthology.orchestra.run.smithi043.stdout:scheduled-2023-08-09-13_43_00_UTC 2023-08-09T13:47:35.317 INFO:teuthology.orchestra.run.smithi043.stdout:scheduled-2023-08-09-13_44_01_UTC 2023-08-09T13:47:35.317 INFO:teuthology.orchestra.run.smithi043.stdout:scheduled-2023-08-09-13_45_01_UTC 2023-08-09T13:47:35.317 INFO:teuthology.orchestra.run.smithi043.stdout:scheduled-2023-08-09-13_46_01_UTC 2023-08-09T13:47:35.317 INFO:teuthology.orchestra.run.smithi043.stdout:scheduled-2023-08-09-13_47_01_UTC 2023-08-09T13:47:35.318 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/dir0/.snap/scheduled-2023-08-09-13_37_00_UTC 2023-08-09T13:47:35.319 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-09T13:47:35.319 DEBUG:teuthology.orchestra.run.smithi043:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/dir0/.snap/scheduled-2023-08-09-13_37_00_UTC) 2023-08-09T13:47:35.401 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/dir0/.snap/scheduled-2023-08-09-13_38_01_UTC 2023-08-09T13:47:35.402 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-09T13:47:35.402 DEBUG:teuthology.orchestra.run.smithi043:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/dir0/.snap/scheduled-2023-08-09-13_38_01_UTC) 2023-08-09T13:47:35.488 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/dir0/.snap/scheduled-2023-08-09-13_39_01_UTC 2023-08-09T13:47:35.488 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-09T13:47:35.489 DEBUG:teuthology.orchestra.run.smithi043:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/dir0/.snap/scheduled-2023-08-09-13_39_01_UTC) 2023-08-09T13:47:35.570 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/dir0/.snap/scheduled-2023-08-09-13_40_00_UTC 2023-08-09T13:47:35.570 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-09T13:47:35.571 DEBUG:teuthology.orchestra.run.smithi043:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/dir0/.snap/scheduled-2023-08-09-13_40_00_UTC) 2023-08-09T13:47:35.653 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/dir0/.snap/scheduled-2023-08-09-13_41_00_UTC 2023-08-09T13:47:35.653 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-09T13:47:35.653 DEBUG:teuthology.orchestra.run.smithi043:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/dir0/.snap/scheduled-2023-08-09-13_41_00_UTC) 2023-08-09T13:47:35.735 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/dir0/.snap/scheduled-2023-08-09-13_42_00_UTC 2023-08-09T13:47:35.736 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-09T13:47:35.736 DEBUG:teuthology.orchestra.run.smithi043:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/dir0/.snap/scheduled-2023-08-09-13_42_00_UTC) 2023-08-09T13:47:35.820 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/dir0/.snap/scheduled-2023-08-09-13_43_00_UTC 2023-08-09T13:47:35.821 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-09T13:47:35.821 DEBUG:teuthology.orchestra.run.smithi043:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/dir0/.snap/scheduled-2023-08-09-13_43_00_UTC) 2023-08-09T13:47:35.900 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/dir0/.snap/scheduled-2023-08-09-13_44_01_UTC 2023-08-09T13:47:35.900 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-09T13:47:35.900 DEBUG:teuthology.orchestra.run.smithi043:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/dir0/.snap/scheduled-2023-08-09-13_44_01_UTC) 2023-08-09T13:47:35.976 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/dir0/.snap/scheduled-2023-08-09-13_45_01_UTC 2023-08-09T13:47:35.977 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-09T13:47:35.977 DEBUG:teuthology.orchestra.run.smithi043:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/dir0/.snap/scheduled-2023-08-09-13_45_01_UTC) 2023-08-09T13:47:36.059 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/dir0/.snap/scheduled-2023-08-09-13_46_01_UTC 2023-08-09T13:47:36.060 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-09T13:47:36.060 DEBUG:teuthology.orchestra.run.smithi043:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/dir0/.snap/scheduled-2023-08-09-13_46_01_UTC) 2023-08-09T13:47:36.187 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/dir0/.snap/scheduled-2023-08-09-13_47_01_UTC 2023-08-09T13:47:36.187 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-09T13:47:36.187 DEBUG:teuthology.orchestra.run.smithi043:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/dir0/.snap/scheduled-2023-08-09-13_47_01_UTC) 2023-08-09T13:47:36.253 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-09T13:47:36.253 DEBUG:teuthology.orchestra.run.smithi043:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/dir0) 2023-08-09T13:47:36.300 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs snap-schedule remove --fs cephfs /snap_test_dir1/dir1 1M 2023-08-09T13:47:36.517 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:36.516+0000 7f6df51fc640 1 Processor -- start 2023-08-09T13:47:36.518 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:36.516+0000 7f6df51fc640 1 -- start start 2023-08-09T13:47:36.519 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:36.518+0000 7f6df51fc640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f6df00c99b0 0x7f6df015d3c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:47:36.519 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:36.518+0000 7f6df51fc640 1 -- --> v1:172.21.15.43:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6df0159c40 con 0x7f6df00c86c0 2023-08-09T13:47:36.520 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:36.518+0000 7f6df51fc640 1 -- --> v1:172.21.15.203:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6df0159ea0 con 0x7f6df006c510 2023-08-09T13:47:36.520 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:36.518+0000 7f6df51fc640 1 -- --> v2:172.21.15.43:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6df0157000 con 0x7f6df00c99b0 2023-08-09T13:47:36.520 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:36.518+0000 7f6deffff640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f6df00c99b0 0x7f6df015d3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:47:36.521 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:36.518+0000 7f6de7fff640 1 --1- >> v1:172.21.15.43:6789/0 conn(0x7f6df00c86c0 0x7f6df0159530 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.43:6789/0 says I am v1:172.21.15.43:58208/0 (socket says 172.21.15.43:58208) 2023-08-09T13:47:36.521 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:36.518+0000 7f6deffff640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f6df00c99b0 0x7f6df015d3c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:40266/0 (socket says 172.21.15.43:40266) 2023-08-09T13:47:36.521 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:36.518+0000 7f6de7fff640 1 -- 172.21.15.43:0/2179694569 learned_addr learned my addr 172.21.15.43:0/2179694569 (peer_addr_for_me v1:172.21.15.43:0/0) 2023-08-09T13:47:36.521 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:36.519+0000 7f6deffff640 1 -- 172.21.15.43:0/2179694569 >> v1:172.21.15.203:6789/0 conn(0x7f6df006c510 legacy=0x7f6df006c920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-09T13:47:36.521 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:36.519+0000 7f6deffff640 1 -- 172.21.15.43:0/2179694569 >> v1:172.21.15.43:6789/0 conn(0x7f6df00c86c0 legacy=0x7f6df0159530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:47:36.521 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:36.519+0000 7f6deffff640 1 -- 172.21.15.43:0/2179694569 --> v2:172.21.15.43:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6df015a180 con 0x7f6df00c99b0 2023-08-09T13:47:36.521 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:36.519+0000 7f6def7fe640 1 -- 172.21.15.43:0/2179694569 <== mon.0 v1:172.21.15.43:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1401348089 0 0) 0x7f6df0159c40 con 0x7f6df00c86c0 2023-08-09T13:47:36.522 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:36.520+0000 7f6deffff640 1 --2- 172.21.15.43:0/2179694569 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f6df00c99b0 0x7f6df015d3c0 secure :-1 s=READY pgs=711 cs=0 l=1 rev1=1 crypto rx=0x7f6de00097a0 tx=0x7f6de002b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a92dfb196ecb5c99 server_cookie=0 in_seq=0 out_seq=0 2023-08-09T13:47:36.522 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:36.520+0000 7f6def7fe640 1 -- 172.21.15.43:0/2179694569 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6de0031830 con 0x7f6df00c99b0 2023-08-09T13:47:36.522 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:36.520+0000 7f6def7fe640 1 -- 172.21.15.43:0/2179694569 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6de0031990 con 0x7f6df00c99b0 2023-08-09T13:47:36.522 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:36.520+0000 7f6def7fe640 1 -- 172.21.15.43:0/2179694569 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6de0031d20 con 0x7f6df00c99b0 2023-08-09T13:47:36.523 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:36.521+0000 7f6df51fc640 1 -- 172.21.15.43:0/2179694569 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f6df00c99b0 msgr2=0x7f6df015d3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:47:36.524 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:36.521+0000 7f6df51fc640 1 --2- 172.21.15.43:0/2179694569 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f6df00c99b0 0x7f6df015d3c0 secure :-1 s=READY pgs=711 cs=0 l=1 rev1=1 crypto rx=0x7f6de00097a0 tx=0x7f6de002b8d0 comp rx=0 tx=0).stop 2023-08-09T13:47:36.524 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:36.521+0000 7f6df51fc640 1 -- 172.21.15.43:0/2179694569 shutdown_connections 2023-08-09T13:47:36.524 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:36.521+0000 7f6df51fc640 1 --2- 172.21.15.43:0/2179694569 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f6df00c99b0 0x7f6df015d3c0 unknown :-1 s=CLOSED pgs=711 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:47:36.524 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:36.521+0000 7f6df51fc640 1 -- 172.21.15.43:0/2179694569 >> 172.21.15.43:0/2179694569 conn(0x7f6df00ca870 msgr2=0x7f6df00cac50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-09T13:47:36.524 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:36.521+0000 7f6df51fc640 1 -- 172.21.15.43:0/2179694569 shutdown_connections 2023-08-09T13:47:36.524 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:36.521+0000 7f6df51fc640 1 -- 172.21.15.43:0/2179694569 wait complete. 2023-08-09T13:47:36.525 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:36.523+0000 7f6df51fc640 1 Processor -- start 2023-08-09T13:47:36.525 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:36.523+0000 7f6df51fc640 1 -- start start 2023-08-09T13:47:36.526 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:36.524+0000 7f6df51fc640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f6df006c510 0x7f6df0146520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:47:36.526 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:36.524+0000 7f6df51fc640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f6df00c86c0 0x7f6df01f5d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:47:36.526 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:36.524+0000 7f6df51fc640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f6df00c99b0 0x7f6df00ac810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:47:36.526 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:36.524+0000 7f6df51fc640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6df00ae8a0 con 0x7f6df006c510 2023-08-09T13:47:36.527 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:36.524+0000 7f6df51fc640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6df00aeb40 con 0x7f6df00c86c0 2023-08-09T13:47:36.527 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:36.524+0000 7f6df51fc640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6df00aee20 con 0x7f6df00c99b0 2023-08-09T13:47:36.527 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:36.525+0000 7f6deffff640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f6df006c510 0x7f6df0146520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:47:36.527 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:36.525+0000 7f6deffff640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f6df006c510 0x7f6df0146520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:40268/0 (socket says 172.21.15.43:40268) 2023-08-09T13:47:36.527 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:36.525+0000 7f6deffff640 1 -- 172.21.15.43:0/1159546386 learned_addr learned my addr 172.21.15.43:0/1159546386 (peer_addr_for_me v2:172.21.15.43:0/0) 2023-08-09T13:47:36.528 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:36.525+0000 7f6de7fff640 1 --2- 172.21.15.43:0/1159546386 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f6df00c86c0 0x7f6df01f5d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:47:36.529 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:36.525+0000 7f6df49fb640 1 --2- 172.21.15.43:0/1159546386 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f6df00c99b0 0x7f6df00ac810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:47:36.529 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:36.525+0000 7f6deffff640 1 -- 172.21.15.43:0/1159546386 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f6df00c99b0 msgr2=0x7f6df00ac810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:47:36.529 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:36.525+0000 7f6deffff640 1 --2- 172.21.15.43:0/1159546386 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f6df00c99b0 0x7f6df00ac810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:47:36.529 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:36.525+0000 7f6deffff640 1 -- 172.21.15.43:0/1159546386 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f6df00c86c0 msgr2=0x7f6df01f5d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:47:36.529 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:36.525+0000 7f6deffff640 1 --2- 172.21.15.43:0/1159546386 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f6df00c86c0 0x7f6df01f5d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:47:36.529 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:36.525+0000 7f6deffff640 1 -- 172.21.15.43:0/1159546386 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6de0009400 con 0x7f6df006c510 2023-08-09T13:47:36.529 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:36.525+0000 7f6df49fb640 1 --2- 172.21.15.43:0/1159546386 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f6df00c99b0 0x7f6df00ac810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-08-09T13:47:36.530 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:36.525+0000 7f6deffff640 1 --2- 172.21.15.43:0/1159546386 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f6df006c510 0x7f6df0146520 secure :-1 s=READY pgs=712 cs=0 l=1 rev1=1 crypto rx=0x7f6de00097a0 tx=0x7f6de00046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-09T13:47:36.530 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:36.526+0000 7f6ded7fa640 1 -- 172.21.15.43:0/1159546386 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6de00317d0 con 0x7f6df006c510 2023-08-09T13:47:36.530 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:36.526+0000 7f6ded7fa640 1 -- 172.21.15.43:0/1159546386 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6de0004030 con 0x7f6df006c510 2023-08-09T13:47:36.530 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:36.526+0000 7f6ded7fa640 1 -- 172.21.15.43:0/1159546386 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6de00043e0 con 0x7f6df006c510 2023-08-09T13:47:36.530 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:36.526+0000 7f6df51fc640 1 -- 172.21.15.43:0/1159546386 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6df00af100 con 0x7f6df006c510 2023-08-09T13:47:36.530 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:36.526+0000 7f6df51fc640 1 -- 172.21.15.43:0/1159546386 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6df00c4ec0 con 0x7f6df006c510 2023-08-09T13:47:36.530 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:36.527+0000 7f6df51fc640 1 -- 172.21.15.43:0/1159546386 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6df00af620 con 0x7f6df006c510 2023-08-09T13:47:36.531 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:36.528+0000 7f6ded7fa640 1 -- 172.21.15.43:0/1159546386 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 13) v1 ==== 88529+0+0 (secure 0 0 0) 0x7f6de0043070 con 0x7f6df006c510 2023-08-09T13:47:36.531 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:36.529+0000 7f6ded7fa640 1 --2- 172.21.15.43:0/1159546386 >> [v2:172.21.15.203:6832/998005540,v1:172.21.15.203:6833/998005540] conn(0x7f6dd006a250 0x7f6dd006c710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:47:36.531 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:36.529+0000 7f6ded7fa640 1 -- 172.21.15.43:0/1159546386 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(43..43 src has 1..43) v4 ==== 7010+0+0 (secure 0 0 0) 0x7f6de00aac50 con 0x7f6df006c510 2023-08-09T13:47:36.531 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:36.530+0000 7f6de7fff640 1 --2- 172.21.15.43:0/1159546386 >> [v2:172.21.15.203:6832/998005540,v1:172.21.15.203:6833/998005540] conn(0x7f6dd006a250 0x7f6dd006c710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:47:36.532 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:36.531+0000 7f6de7fff640 1 --2- 172.21.15.43:0/1159546386 >> [v2:172.21.15.203:6832/998005540,v1:172.21.15.203:6833/998005540] conn(0x7f6dd006a250 0x7f6dd006c710 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f6df0070cf0 tx=0x7f6dd8007c00 comp rx=0 tx=0).ready entity=mgr.5003 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-09T13:47:36.534 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:36.533+0000 7f6ded7fa640 1 -- 172.21.15.43:0/1159546386 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178066 (secure 0 0 0) 0x7f6df00af620 con 0x7f6df006c510 2023-08-09T13:47:36.748 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:36.746+0000 7f6df51fc640 1 -- 172.21.15.43:0/1159546386 --> [v2:172.21.15.203:6832/998005540,v1:172.21.15.203:6833/998005540] -- mgr_command(tid 0: {"prefix": "fs snap-schedule remove", "fs": "cephfs", "path": "/snap_test_dir1/dir1", "repeat": "1M", "target": ["mon-mgr", ""]}) v1 -- 0x7f6df015cbc0 con 0x7f6dd006a250 2023-08-09T13:47:36.953 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:36.952+0000 7f6ded7fa640 1 -- 172.21.15.43:0/1159546386 <== mgr.5003 v2:172.21.15.203:6832/998005540 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+47 (secure 0 0 0) 0x7f6dd8007270 con 0x7f6dd006a250 2023-08-09T13:47:36.954 INFO:teuthology.orchestra.run.smithi043.stdout:Schedule removed for path /snap_test_dir1/dir1 2023-08-09T13:47:36.957 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:36.955+0000 7f6df51fc640 1 -- 172.21.15.43:0/1159546386 >> [v2:172.21.15.203:6832/998005540,v1:172.21.15.203:6833/998005540] conn(0x7f6dd006a250 msgr2=0x7f6dd006c710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:47:36.957 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:36.955+0000 7f6df51fc640 1 --2- 172.21.15.43:0/1159546386 >> [v2:172.21.15.203:6832/998005540,v1:172.21.15.203:6833/998005540] conn(0x7f6dd006a250 0x7f6dd006c710 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f6df0070cf0 tx=0x7f6dd8007c00 comp rx=0 tx=0).stop 2023-08-09T13:47:36.957 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:36.955+0000 7f6df51fc640 1 -- 172.21.15.43:0/1159546386 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f6df006c510 msgr2=0x7f6df0146520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:47:36.957 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:36.955+0000 7f6df51fc640 1 --2- 172.21.15.43:0/1159546386 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f6df006c510 0x7f6df0146520 secure :-1 s=READY pgs=712 cs=0 l=1 rev1=1 crypto rx=0x7f6de00097a0 tx=0x7f6de00046c0 comp rx=0 tx=0).stop 2023-08-09T13:47:36.958 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:36.955+0000 7f6df51fc640 1 -- 172.21.15.43:0/1159546386 shutdown_connections 2023-08-09T13:47:36.958 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:36.955+0000 7f6df51fc640 1 --2- 172.21.15.43:0/1159546386 >> [v2:172.21.15.203:6832/998005540,v1:172.21.15.203:6833/998005540] conn(0x7f6dd006a250 0x7f6dd006c710 unknown :-1 s=CLOSED pgs=64 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:47:36.958 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:36.955+0000 7f6df51fc640 1 --2- 172.21.15.43:0/1159546386 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f6df00c99b0 0x7f6df00ac810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:47:36.958 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:36.955+0000 7f6df51fc640 1 --2- 172.21.15.43:0/1159546386 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f6df00c86c0 0x7f6df01f5d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:47:36.958 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:36.955+0000 7f6df51fc640 1 --2- 172.21.15.43:0/1159546386 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f6df006c510 0x7f6df0146520 unknown :-1 s=CLOSED pgs=712 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:47:36.958 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:36.955+0000 7f6df51fc640 1 -- 172.21.15.43:0/1159546386 >> 172.21.15.43:0/1159546386 conn(0x7f6df00ca870 msgr2=0x7f6df006fb90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-09T13:47:36.958 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:36.955+0000 7f6df51fc640 1 -- 172.21.15.43:0/1159546386 shutdown_connections 2023-08-09T13:47:36.959 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:36.956+0000 7f6df51fc640 1 -- 172.21.15.43:0/1159546386 wait complete. 2023-08-09T13:47:36.977 DEBUG:tasks.cephfs.test_snap_schedules:res=Schedule removed for path /snap_test_dir1/dir1 2023-08-09T13:47:36.978 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-09T13:47:36.978 DEBUG:teuthology.orchestra.run.smithi043:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/dir1/.snap) 2023-08-09T13:47:37.047 INFO:teuthology.orchestra.run.smithi043.stdout:scheduled-2023-08-09-13_37_01_UTC 2023-08-09T13:47:37.048 INFO:teuthology.orchestra.run.smithi043.stdout:scheduled-2023-08-09-13_38_01_UTC 2023-08-09T13:47:37.048 INFO:teuthology.orchestra.run.smithi043.stdout:scheduled-2023-08-09-13_39_00_UTC 2023-08-09T13:47:37.048 INFO:teuthology.orchestra.run.smithi043.stdout:scheduled-2023-08-09-13_40_00_UTC 2023-08-09T13:47:37.048 INFO:teuthology.orchestra.run.smithi043.stdout:scheduled-2023-08-09-13_41_00_UTC 2023-08-09T13:47:37.049 INFO:teuthology.orchestra.run.smithi043.stdout:scheduled-2023-08-09-13_42_01_UTC 2023-08-09T13:47:37.049 INFO:teuthology.orchestra.run.smithi043.stdout:scheduled-2023-08-09-13_43_01_UTC 2023-08-09T13:47:37.049 INFO:teuthology.orchestra.run.smithi043.stdout:scheduled-2023-08-09-13_44_01_UTC 2023-08-09T13:47:37.049 INFO:teuthology.orchestra.run.smithi043.stdout:scheduled-2023-08-09-13_45_01_UTC 2023-08-09T13:47:37.049 INFO:teuthology.orchestra.run.smithi043.stdout:scheduled-2023-08-09-13_46_01_UTC 2023-08-09T13:47:37.049 INFO:teuthology.orchestra.run.smithi043.stdout:scheduled-2023-08-09-13_47_01_UTC 2023-08-09T13:47:37.050 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/dir1/.snap/scheduled-2023-08-09-13_37_01_UTC 2023-08-09T13:47:37.050 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-09T13:47:37.051 DEBUG:teuthology.orchestra.run.smithi043:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/dir1/.snap/scheduled-2023-08-09-13_37_01_UTC) 2023-08-09T13:47:37.124 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/dir1/.snap/scheduled-2023-08-09-13_38_01_UTC 2023-08-09T13:47:37.124 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-09T13:47:37.125 DEBUG:teuthology.orchestra.run.smithi043:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/dir1/.snap/scheduled-2023-08-09-13_38_01_UTC) 2023-08-09T13:47:37.185 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/dir1/.snap/scheduled-2023-08-09-13_39_00_UTC 2023-08-09T13:47:37.186 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-09T13:47:37.186 DEBUG:teuthology.orchestra.run.smithi043:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/dir1/.snap/scheduled-2023-08-09-13_39_00_UTC) 2023-08-09T13:47:37.251 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/dir1/.snap/scheduled-2023-08-09-13_40_00_UTC 2023-08-09T13:47:37.251 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-09T13:47:37.252 DEBUG:teuthology.orchestra.run.smithi043:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/dir1/.snap/scheduled-2023-08-09-13_40_00_UTC) 2023-08-09T13:47:37.325 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/dir1/.snap/scheduled-2023-08-09-13_41_00_UTC 2023-08-09T13:47:37.325 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-09T13:47:37.326 DEBUG:teuthology.orchestra.run.smithi043:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/dir1/.snap/scheduled-2023-08-09-13_41_00_UTC) 2023-08-09T13:47:37.409 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/dir1/.snap/scheduled-2023-08-09-13_42_01_UTC 2023-08-09T13:47:37.410 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-09T13:47:37.410 DEBUG:teuthology.orchestra.run.smithi043:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/dir1/.snap/scheduled-2023-08-09-13_42_01_UTC) 2023-08-09T13:47:37.485 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/dir1/.snap/scheduled-2023-08-09-13_43_01_UTC 2023-08-09T13:47:37.486 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-09T13:47:37.486 DEBUG:teuthology.orchestra.run.smithi043:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/dir1/.snap/scheduled-2023-08-09-13_43_01_UTC) 2023-08-09T13:47:37.562 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/dir1/.snap/scheduled-2023-08-09-13_44_01_UTC 2023-08-09T13:47:37.562 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-09T13:47:37.563 DEBUG:teuthology.orchestra.run.smithi043:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/dir1/.snap/scheduled-2023-08-09-13_44_01_UTC) 2023-08-09T13:47:37.636 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/dir1/.snap/scheduled-2023-08-09-13_45_01_UTC 2023-08-09T13:47:37.637 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-09T13:47:37.637 DEBUG:teuthology.orchestra.run.smithi043:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/dir1/.snap/scheduled-2023-08-09-13_45_01_UTC) 2023-08-09T13:47:37.714 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/dir1/.snap/scheduled-2023-08-09-13_46_01_UTC 2023-08-09T13:47:37.715 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-09T13:47:37.715 DEBUG:teuthology.orchestra.run.smithi043:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/dir1/.snap/scheduled-2023-08-09-13_46_01_UTC) 2023-08-09T13:47:37.793 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/dir1/.snap/scheduled-2023-08-09-13_47_01_UTC 2023-08-09T13:47:37.794 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-09T13:47:37.795 DEBUG:teuthology.orchestra.run.smithi043:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/dir1/.snap/scheduled-2023-08-09-13_47_01_UTC) 2023-08-09T13:47:37.877 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-09T13:47:37.878 DEBUG:teuthology.orchestra.run.smithi043:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/dir1) 2023-08-09T13:47:37.948 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs snap-schedule remove --fs cephfs /snap_test_dir1/dir2 1M 2023-08-09T13:47:38.197 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:38.196+0000 7fc67ed42640 1 Processor -- start 2023-08-09T13:47:38.198 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:38.196+0000 7fc67ed42640 1 -- start start 2023-08-09T13:47:38.200 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:38.198+0000 7fc67ed42640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fc6780ca370 0x7fc67815b2a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:47:38.200 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:38.198+0000 7fc67ed42640 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7fc6780ce4c0 0x7fc6780ce950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:47:38.200 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:38.198+0000 7fc67ed42640 1 -- --> v1:172.21.15.43:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc67815ae10 con 0x7fc67806fb70 2023-08-09T13:47:38.200 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:38.198+0000 7fc67ed42640 1 -- --> v2:172.21.15.203:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc6780cee90 con 0x7fc6780ce4c0 2023-08-09T13:47:38.200 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:38.198+0000 7fc67ed42640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc678154e70 con 0x7fc6780ca370 2023-08-09T13:47:38.200 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:38.198+0000 7fc67cade640 1 --1- >> v1:172.21.15.43:6789/0 conn(0x7fc67806fb70 0x7fc6780caab0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.43:6789/0 says I am v1:172.21.15.43:58210/0 (socket says 172.21.15.43:58210) 2023-08-09T13:47:38.201 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:38.198+0000 7fc67cade640 1 -- 172.21.15.43:0/4206939647 learned_addr learned my addr 172.21.15.43:0/4206939647 (peer_addr_for_me v1:172.21.15.43:0/0) 2023-08-09T13:47:38.202 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:38.199+0000 7fc677fff640 1 --2- 172.21.15.43:0/4206939647 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fc6780ca370 0x7fc67815b2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:47:38.202 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:38.199+0000 7fc67d2df640 1 --2- 172.21.15.43:0/4206939647 >> v2:172.21.15.203:3300/0 conn(0x7fc6780ce4c0 0x7fc6780ce950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:47:38.202 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:38.199+0000 7fc6777fe640 1 -- 172.21.15.43:0/4206939647 <== mon.0 v1:172.21.15.43:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3576412316 0 0) 0x7fc67815ae10 con 0x7fc67806fb70 2023-08-09T13:47:38.203 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:38.199+0000 7fc6777fe640 1 -- 172.21.15.43:0/4206939647 --> v1:172.21.15.43:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fc660003520 con 0x7fc67806fb70 2023-08-09T13:47:38.203 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:38.199+0000 7fc6777fe640 1 -- 172.21.15.43:0/4206939647 <== mon.0 v1:172.21.15.43:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2579654557 0 0) 0x7fc660003520 con 0x7fc67806fb70 2023-08-09T13:47:38.203 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:38.199+0000 7fc6777fe640 1 -- 172.21.15.43:0/4206939647 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fc6780ca370 msgr2=0x7fc67815b2a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-09T13:47:38.203 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:38.199+0000 7fc6777fe640 1 --2- 172.21.15.43:0/4206939647 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fc6780ca370 0x7fc67815b2a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:47:38.203 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:38.199+0000 7fc6777fe640 1 -- 172.21.15.43:0/4206939647 >> v2:172.21.15.203:3300/0 conn(0x7fc6780ce4c0 msgr2=0x7fc6780ce950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-09T13:47:38.203 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:38.199+0000 7fc6777fe640 1 --2- 172.21.15.43:0/4206939647 >> v2:172.21.15.203:3300/0 conn(0x7fc6780ce4c0 0x7fc6780ce950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:47:38.203 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:38.199+0000 7fc6777fe640 1 -- 172.21.15.43:0/4206939647 --> v1:172.21.15.43:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc67806ff50 con 0x7fc67806fb70 2023-08-09T13:47:38.204 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:38.199+0000 7fc6777fe640 1 -- 172.21.15.43:0/4206939647 <== mon.0 v1:172.21.15.43:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1285150740 0 0) 0x7fc668001670 con 0x7fc67806fb70 2023-08-09T13:47:38.204 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:38.199+0000 7fc6777fe640 1 -- 172.21.15.43:0/4206939647 >> v1:172.21.15.43:6789/0 conn(0x7fc67806fb70 legacy=0x7fc6780caab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:47:38.204 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:38.199+0000 7fc6777fe640 1 --2- 172.21.15.43:0/4206939647 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fc6780ca370 0x7fc6600039e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:47:38.204 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:38.199+0000 7fc6777fe640 1 --2- 172.21.15.43:0/4206939647 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc660003f60 0x7fc6600047c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:47:38.204 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:38.199+0000 7fc6777fe640 1 --2- 172.21.15.43:0/4206939647 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fc660004d00 0x7fc660005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:47:38.204 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:38.199+0000 7fc6777fe640 1 -- 172.21.15.43:0/4206939647 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc6780cee90 con 0x7fc660003f60 2023-08-09T13:47:38.204 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:38.199+0000 7fc6777fe640 1 -- 172.21.15.43:0/4206939647 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc678154e70 con 0x7fc660004d00 2023-08-09T13:47:38.205 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:38.199+0000 7fc6777fe640 1 -- 172.21.15.43:0/4206939647 --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc66000b9b0 con 0x7fc6780ca370 2023-08-09T13:47:38.205 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:38.200+0000 7fc677fff640 1 --2- 172.21.15.43:0/4206939647 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc660003f60 0x7fc6600047c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:47:38.205 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:38.200+0000 7fc67cade640 1 --2- 172.21.15.43:0/4206939647 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fc6780ca370 0x7fc6600039e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:47:38.205 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:38.200+0000 7fc67d2df640 1 --2- 172.21.15.43:0/4206939647 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fc660004d00 0x7fc660005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:47:38.205 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:38.200+0000 7fc677fff640 1 -- 172.21.15.43:0/4206939647 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fc6780ca370 msgr2=0x7fc6600039e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-09T13:47:38.205 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:38.200+0000 7fc677fff640 1 --2- 172.21.15.43:0/4206939647 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fc6780ca370 0x7fc6600039e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:47:38.205 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:38.200+0000 7fc677fff640 1 -- 172.21.15.43:0/4206939647 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fc660004d00 msgr2=0x7fc660005180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-09T13:47:38.206 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:38.200+0000 7fc677fff640 1 --2- 172.21.15.43:0/4206939647 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fc660004d00 0x7fc660005180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:47:38.206 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:38.200+0000 7fc677fff640 1 -- 172.21.15.43:0/4206939647 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc660003520 con 0x7fc660003f60 2023-08-09T13:47:38.206 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:38.200+0000 7fc67d2df640 1 --2- 172.21.15.43:0/4206939647 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fc660004d00 0x7fc660005180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-08-09T13:47:38.206 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:38.200+0000 7fc67cade640 1 --2- 172.21.15.43:0/4206939647 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fc6780ca370 0x7fc6600039e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-08-09T13:47:38.208 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:38.201+0000 7fc677fff640 1 --2- 172.21.15.43:0/4206939647 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc660003f60 0x7fc6600047c0 secure :-1 s=READY pgs=714 cs=0 l=1 rev1=1 crypto rx=0x7fc66c009fc0 tx=0x7fc6600057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d4f6c0f3d4140d78 server_cookie=0 in_seq=0 out_seq=0 2023-08-09T13:47:38.208 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:38.202+0000 7fc6777fe640 1 -- 172.21.15.43:0/4206939647 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc66c02cbf0 con 0x7fc660003f60 2023-08-09T13:47:38.208 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:38.202+0000 7fc6777fe640 1 -- 172.21.15.43:0/4206939647 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc66c00f040 con 0x7fc660003f60 2023-08-09T13:47:38.208 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:38.202+0000 7fc6777fe640 1 -- 172.21.15.43:0/4206939647 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc66c034440 con 0x7fc660003f60 2023-08-09T13:47:38.208 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:38.202+0000 7fc67ed42640 1 -- 172.21.15.43:0/4206939647 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc660003f60 msgr2=0x7fc6600047c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:47:38.209 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:38.202+0000 7fc67ed42640 1 --2- 172.21.15.43:0/4206939647 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc660003f60 0x7fc6600047c0 secure :-1 s=READY pgs=714 cs=0 l=1 rev1=1 crypto rx=0x7fc66c009fc0 tx=0x7fc6600057f0 comp rx=0 tx=0).stop 2023-08-09T13:47:38.209 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:38.202+0000 7fc67d2df640 1 -- 172.21.15.43:0/4206939647 reap_dead start 2023-08-09T13:47:38.209 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:38.202+0000 7fc67ed42640 1 -- 172.21.15.43:0/4206939647 shutdown_connections 2023-08-09T13:47:38.209 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:38.202+0000 7fc67ed42640 1 -- 172.21.15.43:0/4206939647 >> 172.21.15.43:0/4206939647 conn(0x7fc67806b990 msgr2=0x7fc67806bda0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-09T13:47:38.209 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:38.202+0000 7fc67ed42640 1 -- 172.21.15.43:0/4206939647 shutdown_connections 2023-08-09T13:47:38.209 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:38.202+0000 7fc67ed42640 1 -- 172.21.15.43:0/4206939647 wait complete. 2023-08-09T13:47:38.209 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:38.204+0000 7fc67ed42640 1 Processor -- start 2023-08-09T13:47:38.209 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:38.204+0000 7fc67ed42640 1 -- start start 2023-08-09T13:47:38.210 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:38.205+0000 7fc67ed42640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fc660003f60 0x7fc6781465c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:47:38.210 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:38.205+0000 7fc67ed42640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc6780c8230 0x7fc6780b1c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:47:38.210 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:38.205+0000 7fc67ed42640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fc6780c5170 0x7fc6780c5600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:47:38.210 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:38.205+0000 7fc67ed42640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc6780c5b40 con 0x7fc6780c8230 2023-08-09T13:47:38.210 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:38.205+0000 7fc67ed42640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc678146b00 con 0x7fc660003f60 2023-08-09T13:47:38.210 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:38.205+0000 7fc67ed42640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc6780b3680 con 0x7fc6780c5170 2023-08-09T13:47:38.211 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:38.206+0000 7fc677fff640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc6780c8230 0x7fc6780b1c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:47:38.211 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:38.206+0000 7fc677fff640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc6780c8230 0x7fc6780b1c70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:40278/0 (socket says 172.21.15.43:40278) 2023-08-09T13:47:38.211 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:38.206+0000 7fc677fff640 1 -- 172.21.15.43:0/2067051188 learned_addr learned my addr 172.21.15.43:0/2067051188 (peer_addr_for_me v2:172.21.15.43:0/0) 2023-08-09T13:47:38.211 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:38.206+0000 7fc67cade640 1 --2- 172.21.15.43:0/2067051188 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fc660003f60 0x7fc6781465c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:47:38.211 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:38.206+0000 7fc67d2df640 1 --2- 172.21.15.43:0/2067051188 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fc6780c5170 0x7fc6780c5600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:47:38.211 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:38.206+0000 7fc677fff640 1 -- 172.21.15.43:0/2067051188 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fc6780c5170 msgr2=0x7fc6780c5600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:47:38.211 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:38.206+0000 7fc677fff640 1 --2- 172.21.15.43:0/2067051188 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fc6780c5170 0x7fc6780c5600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:47:38.211 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:38.206+0000 7fc677fff640 1 -- 172.21.15.43:0/2067051188 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fc660003f60 msgr2=0x7fc6781465c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:47:38.212 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:38.206+0000 7fc677fff640 1 --2- 172.21.15.43:0/2067051188 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fc660003f60 0x7fc6781465c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:47:38.212 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:38.206+0000 7fc677fff640 1 -- 172.21.15.43:0/2067051188 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc6780b31f0 con 0x7fc6780c8230 2023-08-09T13:47:38.212 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:38.206+0000 7fc67cade640 1 --2- 172.21.15.43:0/2067051188 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fc660003f60 0x7fc6781465c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-08-09T13:47:38.212 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:38.207+0000 7fc677fff640 1 --2- 172.21.15.43:0/2067051188 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc6780c8230 0x7fc6780b1c70 secure :-1 s=READY pgs=715 cs=0 l=1 rev1=1 crypto rx=0x7fc66c02c660 tx=0x7fc66c009750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-09T13:47:38.213 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:38.207+0000 7fc6757fa640 1 -- 172.21.15.43:0/2067051188 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc66c034bd0 con 0x7fc6780c8230 2023-08-09T13:47:38.213 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:38.207+0000 7fc6757fa640 1 -- 172.21.15.43:0/2067051188 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc66c034d50 con 0x7fc6780c8230 2023-08-09T13:47:38.213 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:38.207+0000 7fc6757fa640 1 -- 172.21.15.43:0/2067051188 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc66c003bd0 con 0x7fc6780c8230 2023-08-09T13:47:38.214 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:38.207+0000 7fc67ed42640 1 -- 172.21.15.43:0/2067051188 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc6780b25e0 con 0x7fc6780c8230 2023-08-09T13:47:38.214 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:38.207+0000 7fc67ed42640 1 -- 172.21.15.43:0/2067051188 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc6780b2ba0 con 0x7fc6780c8230 2023-08-09T13:47:38.214 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:38.208+0000 7fc67ed42640 1 -- 172.21.15.43:0/2067051188 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc6780b3380 con 0x7fc6780c8230 2023-08-09T13:47:38.214 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:38.209+0000 7fc6757fa640 1 -- 172.21.15.43:0/2067051188 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 13) v1 ==== 88529+0+0 (secure 0 0 0) 0x7fc66c040450 con 0x7fc6780c8230 2023-08-09T13:47:38.214 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:38.210+0000 7fc6757fa640 1 --2- 172.21.15.43:0/2067051188 >> [v2:172.21.15.203:6832/998005540,v1:172.21.15.203:6833/998005540] conn(0x7fc66406a250 0x7fc66406c710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:47:38.214 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:38.210+0000 7fc6757fa640 1 -- 172.21.15.43:0/2067051188 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(45..45 src has 1..45) v4 ==== 6962+0+0 (secure 0 0 0) 0x7fc66c0ab520 con 0x7fc6780c8230 2023-08-09T13:47:38.214 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:38.211+0000 7fc67cade640 1 --2- 172.21.15.43:0/2067051188 >> [v2:172.21.15.203:6832/998005540,v1:172.21.15.203:6833/998005540] conn(0x7fc66406a250 0x7fc66406c710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:47:38.215 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:38.212+0000 7fc67cade640 1 --2- 172.21.15.43:0/2067051188 >> [v2:172.21.15.203:6832/998005540,v1:172.21.15.203:6833/998005540] conn(0x7fc66406a250 0x7fc66406c710 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7fc6680018e0 tx=0x7fc668009040 comp rx=0 tx=0).ready entity=mgr.5003 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-09T13:47:38.216 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:38.215+0000 7fc6757fa640 1 -- 172.21.15.43:0/2067051188 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178066 (secure 0 0 0) 0x7fc66c076560 con 0x7fc6780c8230 2023-08-09T13:47:38.452 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:38.450+0000 7fc67ed42640 1 -- 172.21.15.43:0/2067051188 --> [v2:172.21.15.203:6832/998005540,v1:172.21.15.203:6833/998005540] -- mgr_command(tid 0: {"prefix": "fs snap-schedule remove", "fs": "cephfs", "path": "/snap_test_dir1/dir2", "repeat": "1M", "target": ["mon-mgr", ""]}) v1 -- 0x7fc6780cb600 con 0x7fc66406a250 2023-08-09T13:47:38.650 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:38.649+0000 7fc6757fa640 1 -- 172.21.15.43:0/2067051188 <== mgr.5003 v2:172.21.15.203:6832/998005540 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+47 (secure 0 0 0) 0x7fc66800d040 con 0x7fc66406a250 2023-08-09T13:47:38.651 INFO:teuthology.orchestra.run.smithi043.stdout:Schedule removed for path /snap_test_dir1/dir2 2023-08-09T13:47:38.653 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:38.652+0000 7fc67ed42640 1 -- 172.21.15.43:0/2067051188 >> [v2:172.21.15.203:6832/998005540,v1:172.21.15.203:6833/998005540] conn(0x7fc66406a250 msgr2=0x7fc66406c710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:47:38.654 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:38.652+0000 7fc67ed42640 1 --2- 172.21.15.43:0/2067051188 >> [v2:172.21.15.203:6832/998005540,v1:172.21.15.203:6833/998005540] conn(0x7fc66406a250 0x7fc66406c710 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7fc6680018e0 tx=0x7fc668009040 comp rx=0 tx=0).stop 2023-08-09T13:47:38.654 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:38.652+0000 7fc67ed42640 1 -- 172.21.15.43:0/2067051188 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc6780c8230 msgr2=0x7fc6780b1c70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:47:38.654 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:38.652+0000 7fc67ed42640 1 --2- 172.21.15.43:0/2067051188 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc6780c8230 0x7fc6780b1c70 secure :-1 s=READY pgs=715 cs=0 l=1 rev1=1 crypto rx=0x7fc66c02c660 tx=0x7fc66c009750 comp rx=0 tx=0).stop 2023-08-09T13:47:38.654 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:38.652+0000 7fc67ed42640 1 -- 172.21.15.43:0/2067051188 shutdown_connections 2023-08-09T13:47:38.655 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:38.652+0000 7fc67ed42640 1 --2- 172.21.15.43:0/2067051188 >> [v2:172.21.15.203:6832/998005540,v1:172.21.15.203:6833/998005540] conn(0x7fc66406a250 0x7fc66406c710 unknown :-1 s=CLOSED pgs=65 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:47:38.655 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:38.652+0000 7fc67ed42640 1 --2- 172.21.15.43:0/2067051188 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fc6780c5170 0x7fc6780c5600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:47:38.655 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:38.652+0000 7fc67ed42640 1 --2- 172.21.15.43:0/2067051188 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc6780c8230 0x7fc6780b1c70 unknown :-1 s=CLOSED pgs=715 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:47:38.655 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:38.652+0000 7fc67ed42640 1 --2- 172.21.15.43:0/2067051188 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fc660003f60 0x7fc6781465c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:47:38.655 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:38.652+0000 7fc67ed42640 1 -- 172.21.15.43:0/2067051188 >> 172.21.15.43:0/2067051188 conn(0x7fc67806b990 msgr2=0x7fc67806bda0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-09T13:47:38.655 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:38.652+0000 7fc67ed42640 1 -- 172.21.15.43:0/2067051188 shutdown_connections 2023-08-09T13:47:38.655 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:38.652+0000 7fc67ed42640 1 -- 172.21.15.43:0/2067051188 wait complete. 2023-08-09T13:47:38.673 DEBUG:tasks.cephfs.test_snap_schedules:res=Schedule removed for path /snap_test_dir1/dir2 2023-08-09T13:47:38.674 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-09T13:47:38.675 DEBUG:teuthology.orchestra.run.smithi043:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/dir2/.snap) 2023-08-09T13:47:38.754 INFO:teuthology.orchestra.run.smithi043.stdout:scheduled-2023-08-09-13_37_00_UTC 2023-08-09T13:47:38.755 INFO:teuthology.orchestra.run.smithi043.stdout:scheduled-2023-08-09-13_38_00_UTC 2023-08-09T13:47:38.755 INFO:teuthology.orchestra.run.smithi043.stdout:scheduled-2023-08-09-13_39_00_UTC 2023-08-09T13:47:38.755 INFO:teuthology.orchestra.run.smithi043.stdout:scheduled-2023-08-09-13_40_00_UTC 2023-08-09T13:47:38.755 INFO:teuthology.orchestra.run.smithi043.stdout:scheduled-2023-08-09-13_41_00_UTC 2023-08-09T13:47:38.755 INFO:teuthology.orchestra.run.smithi043.stdout:scheduled-2023-08-09-13_42_00_UTC 2023-08-09T13:47:38.755 INFO:teuthology.orchestra.run.smithi043.stdout:scheduled-2023-08-09-13_43_00_UTC 2023-08-09T13:47:38.756 INFO:teuthology.orchestra.run.smithi043.stdout:scheduled-2023-08-09-13_44_00_UTC 2023-08-09T13:47:38.756 INFO:teuthology.orchestra.run.smithi043.stdout:scheduled-2023-08-09-13_45_00_UTC 2023-08-09T13:47:38.756 INFO:teuthology.orchestra.run.smithi043.stdout:scheduled-2023-08-09-13_46_00_UTC 2023-08-09T13:47:38.756 INFO:teuthology.orchestra.run.smithi043.stdout:scheduled-2023-08-09-13_47_00_UTC 2023-08-09T13:47:38.757 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/dir2/.snap/scheduled-2023-08-09-13_37_00_UTC 2023-08-09T13:47:38.757 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-09T13:47:38.758 DEBUG:teuthology.orchestra.run.smithi043:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/dir2/.snap/scheduled-2023-08-09-13_37_00_UTC) 2023-08-09T13:47:38.834 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/dir2/.snap/scheduled-2023-08-09-13_38_00_UTC 2023-08-09T13:47:38.835 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-09T13:47:38.835 DEBUG:teuthology.orchestra.run.smithi043:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/dir2/.snap/scheduled-2023-08-09-13_38_00_UTC) 2023-08-09T13:47:38.916 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/dir2/.snap/scheduled-2023-08-09-13_39_00_UTC 2023-08-09T13:47:38.917 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-09T13:47:38.917 DEBUG:teuthology.orchestra.run.smithi043:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/dir2/.snap/scheduled-2023-08-09-13_39_00_UTC) 2023-08-09T13:47:38.998 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/dir2/.snap/scheduled-2023-08-09-13_40_00_UTC 2023-08-09T13:47:38.999 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-09T13:47:38.999 DEBUG:teuthology.orchestra.run.smithi043:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/dir2/.snap/scheduled-2023-08-09-13_40_00_UTC) 2023-08-09T13:47:39.081 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/dir2/.snap/scheduled-2023-08-09-13_41_00_UTC 2023-08-09T13:47:39.081 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-09T13:47:39.082 DEBUG:teuthology.orchestra.run.smithi043:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/dir2/.snap/scheduled-2023-08-09-13_41_00_UTC) 2023-08-09T13:47:39.171 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/dir2/.snap/scheduled-2023-08-09-13_42_00_UTC 2023-08-09T13:47:39.172 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-09T13:47:39.172 DEBUG:teuthology.orchestra.run.smithi043:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/dir2/.snap/scheduled-2023-08-09-13_42_00_UTC) 2023-08-09T13:47:39.229 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/dir2/.snap/scheduled-2023-08-09-13_43_00_UTC 2023-08-09T13:47:39.229 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-09T13:47:39.229 DEBUG:teuthology.orchestra.run.smithi043:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/dir2/.snap/scheduled-2023-08-09-13_43_00_UTC) 2023-08-09T13:47:39.294 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/dir2/.snap/scheduled-2023-08-09-13_44_00_UTC 2023-08-09T13:47:39.295 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-09T13:47:39.295 DEBUG:teuthology.orchestra.run.smithi043:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/dir2/.snap/scheduled-2023-08-09-13_44_00_UTC) 2023-08-09T13:47:39.369 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/dir2/.snap/scheduled-2023-08-09-13_45_00_UTC 2023-08-09T13:47:39.370 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-09T13:47:39.371 DEBUG:teuthology.orchestra.run.smithi043:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/dir2/.snap/scheduled-2023-08-09-13_45_00_UTC) 2023-08-09T13:47:39.450 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/dir2/.snap/scheduled-2023-08-09-13_46_00_UTC 2023-08-09T13:47:39.450 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-09T13:47:39.451 DEBUG:teuthology.orchestra.run.smithi043:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/dir2/.snap/scheduled-2023-08-09-13_46_00_UTC) 2023-08-09T13:47:39.539 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/dir2/.snap/scheduled-2023-08-09-13_47_00_UTC 2023-08-09T13:47:39.671 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-09T13:47:39.672 DEBUG:teuthology.orchestra.run.smithi043:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/dir2/.snap/scheduled-2023-08-09-13_47_00_UTC) 2023-08-09T13:47:39.756 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-09T13:47:39.757 DEBUG:teuthology.orchestra.run.smithi043:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/dir2) 2023-08-09T13:47:39.831 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs snap-schedule remove --fs cephfs /snap_test_dir1/dir3 1M 2023-08-09T13:47:40.035 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:40.034+0000 7f2ec9404640 1 Processor -- start 2023-08-09T13:47:40.036 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:40.034+0000 7f2ec9404640 1 -- start start 2023-08-09T13:47:40.037 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:40.035+0000 7f2ec9404640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f2ec4069a30 0x7f2ec4159760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:47:40.037 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:40.035+0000 7f2ec9404640 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7f2ec415d5b0 0x7f2ec4069e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:47:40.037 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:40.035+0000 7f2ec9404640 1 -- --> v1:172.21.15.203:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2ec40ca410 con 0x7f2ec406a350 2023-08-09T13:47:40.037 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:40.035+0000 7f2ec9404640 1 -- --> v2:172.21.15.203:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2ec415ce80 con 0x7f2ec415d5b0 2023-08-09T13:47:40.038 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:40.036+0000 7f2ec9404640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2ec415a050 con 0x7f2ec4069a30 2023-08-09T13:47:40.038 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:40.036+0000 7f2ec2ffd640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f2ec4069a30 0x7f2ec4159760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:47:40.039 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:40.036+0000 7f2ec2ffd640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f2ec4069a30 0x7f2ec4159760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.203:3301/0 says I am v2:172.21.15.43:56718/0 (socket says 172.21.15.43:56718) 2023-08-09T13:47:40.039 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:40.036+0000 7f2ec2ffd640 1 -- 172.21.15.43:0/1311963936 learned_addr learned my addr 172.21.15.43:0/1311963936 (peer_addr_for_me v2:172.21.15.43:0/0) 2023-08-09T13:47:40.039 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:40.036+0000 7f2ec27fc640 1 --2- 172.21.15.43:0/1311963936 >> v2:172.21.15.203:3300/0 conn(0x7f2ec415d5b0 0x7f2ec4069e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:47:40.039 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:40.037+0000 7f2ec2ffd640 1 -- 172.21.15.43:0/1311963936 >> v2:172.21.15.203:3300/0 conn(0x7f2ec415d5b0 msgr2=0x7f2ec4069e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-09T13:47:40.039 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:40.037+0000 7f2ec2ffd640 1 --2- 172.21.15.43:0/1311963936 >> v2:172.21.15.203:3300/0 conn(0x7f2ec415d5b0 0x7f2ec4069e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:47:40.039 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:40.037+0000 7f2ec2ffd640 1 -- 172.21.15.43:0/1311963936 >> v1:172.21.15.203:6789/0 conn(0x7f2ec406a350 legacy=0x7f2ec40ce4c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-09T13:47:40.040 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:40.037+0000 7f2ec2ffd640 1 -- 172.21.15.43:0/1311963936 --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2ec415a330 con 0x7f2ec4069a30 2023-08-09T13:47:40.040 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:40.039+0000 7f2ec2ffd640 1 --2- 172.21.15.43:0/1311963936 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f2ec4069a30 0x7f2ec4159760 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f2eb4009e50 tx=0x7f2eb402bed0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=24d9095bb687300b server_cookie=0 in_seq=0 out_seq=0 2023-08-09T13:47:40.041 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:40.039+0000 7f2ec1ffb640 1 -- 172.21.15.43:0/1311963936 <== mon.2 v2:172.21.15.203:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2eb402ca70 con 0x7f2ec4069a30 2023-08-09T13:47:40.041 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:40.039+0000 7f2ec1ffb640 1 -- 172.21.15.43:0/1311963936 <== mon.2 v2:172.21.15.203:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2eb402cbd0 con 0x7f2ec4069a30 2023-08-09T13:47:40.041 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:40.039+0000 7f2ec1ffb640 1 -- 172.21.15.43:0/1311963936 <== mon.2 v2:172.21.15.203:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2eb40343f0 con 0x7f2ec4069a30 2023-08-09T13:47:40.042 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:40.040+0000 7f2ec9404640 1 -- 172.21.15.43:0/1311963936 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f2ec4069a30 msgr2=0x7f2ec4159760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:47:40.042 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:40.040+0000 7f2ec9404640 1 --2- 172.21.15.43:0/1311963936 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f2ec4069a30 0x7f2ec4159760 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f2eb4009e50 tx=0x7f2eb402bed0 comp rx=0 tx=0).stop 2023-08-09T13:47:40.042 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:40.040+0000 7f2ec9404640 1 -- 172.21.15.43:0/1311963936 shutdown_connections 2023-08-09T13:47:40.042 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:40.040+0000 7f2ec9404640 1 --2- 172.21.15.43:0/1311963936 >> v2:172.21.15.203:3300/0 conn(0x7f2ec415d5b0 0x7f2ec4069e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:47:40.042 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:40.040+0000 7f2ec9404640 1 --2- 172.21.15.43:0/1311963936 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f2ec4069a30 0x7f2ec4159760 unknown :-1 s=CLOSED pgs=93 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:47:40.042 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:40.040+0000 7f2ec9404640 1 -- 172.21.15.43:0/1311963936 >> 172.21.15.43:0/1311963936 conn(0x7f2ec406b990 msgr2=0x7f2ec406bda0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-09T13:47:40.043 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:40.040+0000 7f2ec9404640 1 -- 172.21.15.43:0/1311963936 shutdown_connections 2023-08-09T13:47:40.043 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:40.040+0000 7f2ec9404640 1 -- 172.21.15.43:0/1311963936 wait complete. 2023-08-09T13:47:40.043 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:40.042+0000 7f2ec9404640 1 Processor -- start 2023-08-09T13:47:40.043 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:40.042+0000 7f2ec9404640 1 -- start start 2023-08-09T13:47:40.045 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:40.043+0000 7f2ec9404640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f2ec4069a30 0x7f2ec41f2590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:47:40.045 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:40.043+0000 7f2ec9404640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f2ec406a350 0x7f2ec40abcb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:47:40.045 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:40.043+0000 7f2ec9404640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f2ec415d5b0 0x7f2ec40ac1f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:47:40.045 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:40.043+0000 7f2ec9404640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2ec40aac90 con 0x7f2ec4069a30 2023-08-09T13:47:40.045 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:40.043+0000 7f2ec9404640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2ec40aa010 con 0x7f2ec415d5b0 2023-08-09T13:47:40.045 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:40.043+0000 7f2ec9404640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2ec40aa2f0 con 0x7f2ec406a350 2023-08-09T13:47:40.045 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:40.043+0000 7f2ec2ffd640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f2ec4069a30 0x7f2ec41f2590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:47:40.046 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:40.043+0000 7f2ec2ffd640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f2ec4069a30 0x7f2ec41f2590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:40290/0 (socket says 172.21.15.43:40290) 2023-08-09T13:47:40.046 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:40.043+0000 7f2ec2ffd640 1 -- 172.21.15.43:0/28971124 learned_addr learned my addr 172.21.15.43:0/28971124 (peer_addr_for_me v2:172.21.15.43:0/0) 2023-08-09T13:47:40.047 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:40.044+0000 7f2ec27fc640 1 --2- 172.21.15.43:0/28971124 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f2ec406a350 0x7f2ec40abcb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:47:40.047 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:40.044+0000 7f2ec2ffd640 1 -- 172.21.15.43:0/28971124 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f2ec406a350 msgr2=0x7f2ec40abcb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:47:40.047 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:40.044+0000 7f2ec37fe640 1 --2- 172.21.15.43:0/28971124 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f2ec415d5b0 0x7f2ec40ac1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:47:40.047 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:40.044+0000 7f2ec2ffd640 1 --2- 172.21.15.43:0/28971124 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f2ec406a350 0x7f2ec40abcb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:47:40.047 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:40.044+0000 7f2ec2ffd640 1 -- 172.21.15.43:0/28971124 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f2ec415d5b0 msgr2=0x7f2ec40ac1f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:47:40.047 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:40.044+0000 7f2ec2ffd640 1 --2- 172.21.15.43:0/28971124 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f2ec415d5b0 0x7f2ec40ac1f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:47:40.048 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:40.044+0000 7f2ec2ffd640 1 -- 172.21.15.43:0/28971124 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2eb4009b30 con 0x7f2ec4069a30 2023-08-09T13:47:40.048 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:40.044+0000 7f2ec37fe640 1 --2- 172.21.15.43:0/28971124 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f2ec415d5b0 0x7f2ec40ac1f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-08-09T13:47:40.048 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:40.044+0000 7f2ec2ffd640 1 --2- 172.21.15.43:0/28971124 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f2ec4069a30 0x7f2ec41f2590 secure :-1 s=READY pgs=716 cs=0 l=1 rev1=1 crypto rx=0x7f2eb4000f80 tx=0x7f2eb4004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-09T13:47:40.048 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:40.045+0000 7f2eab7fe640 1 -- 172.21.15.43:0/28971124 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2eb4009510 con 0x7f2ec4069a30 2023-08-09T13:47:40.048 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:40.045+0000 7f2eab7fe640 1 -- 172.21.15.43:0/28971124 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2eb403e450 con 0x7f2ec4069a30 2023-08-09T13:47:40.048 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:40.045+0000 7f2eab7fe640 1 -- 172.21.15.43:0/28971124 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2eb403e800 con 0x7f2ec4069a30 2023-08-09T13:47:40.049 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:40.045+0000 7f2ec9404640 1 -- 172.21.15.43:0/28971124 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2ec40aa5d0 con 0x7f2ec4069a30 2023-08-09T13:47:40.049 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:40.045+0000 7f2ec9404640 1 -- 172.21.15.43:0/28971124 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2ec41f2ad0 con 0x7f2ec4069a30 2023-08-09T13:47:40.049 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:40.046+0000 7f2ec9404640 1 -- 172.21.15.43:0/28971124 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2ec40660f0 con 0x7f2ec4069a30 2023-08-09T13:47:40.049 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:40.047+0000 7f2eab7fe640 1 -- 172.21.15.43:0/28971124 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 13) v1 ==== 88529+0+0 (secure 0 0 0) 0x7f2eb403e960 con 0x7f2ec4069a30 2023-08-09T13:47:40.050 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:40.049+0000 7f2eab7fe640 1 --2- 172.21.15.43:0/28971124 >> [v2:172.21.15.203:6832/998005540,v1:172.21.15.203:6833/998005540] conn(0x7f2eb006a250 0x7f2eb006c710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:47:40.050 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:40.049+0000 7f2eab7fe640 1 -- 172.21.15.43:0/28971124 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(46..46 src has 1..46) v4 ==== 7138+0+0 (secure 0 0 0) 0x7f2eb40abb70 con 0x7f2ec4069a30 2023-08-09T13:47:40.051 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:40.050+0000 7f2ec27fc640 1 --2- 172.21.15.43:0/28971124 >> [v2:172.21.15.203:6832/998005540,v1:172.21.15.203:6833/998005540] conn(0x7f2eb006a250 0x7f2eb006c710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:47:40.052 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:40.051+0000 7f2ec27fc640 1 --2- 172.21.15.43:0/28971124 >> [v2:172.21.15.203:6832/998005540,v1:172.21.15.203:6833/998005540] conn(0x7f2eb006a250 0x7f2eb006c710 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f2eb8009a30 tx=0x7f2eb8008040 comp rx=0 tx=0).ready entity=mgr.5003 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-09T13:47:40.055 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:40.054+0000 7f2eab7fe640 1 -- 172.21.15.43:0/28971124 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178066 (secure 0 0 0) 0x7f2eb4076c30 con 0x7f2ec4069a30 2023-08-09T13:47:40.322 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:40.320+0000 7f2ec9404640 1 -- 172.21.15.43:0/28971124 --> [v2:172.21.15.203:6832/998005540,v1:172.21.15.203:6833/998005540] -- mgr_command(tid 0: {"prefix": "fs snap-schedule remove", "fs": "cephfs", "path": "/snap_test_dir1/dir3", "repeat": "1M", "target": ["mon-mgr", ""]}) v1 -- 0x7f2ec415d090 con 0x7f2eb006a250 2023-08-09T13:47:40.500 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:40.498+0000 7f2eab7fe640 1 -- 172.21.15.43:0/28971124 <== mgr.5003 v2:172.21.15.203:6832/998005540 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+47 (secure 0 0 0) 0x7f2eb80119a0 con 0x7f2eb006a250 2023-08-09T13:47:40.500 INFO:teuthology.orchestra.run.smithi043.stdout:Schedule removed for path /snap_test_dir1/dir3 2023-08-09T13:47:40.503 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:40.502+0000 7f2ec9404640 1 -- 172.21.15.43:0/28971124 >> [v2:172.21.15.203:6832/998005540,v1:172.21.15.203:6833/998005540] conn(0x7f2eb006a250 msgr2=0x7f2eb006c710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:47:40.504 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:40.502+0000 7f2ec9404640 1 --2- 172.21.15.43:0/28971124 >> [v2:172.21.15.203:6832/998005540,v1:172.21.15.203:6833/998005540] conn(0x7f2eb006a250 0x7f2eb006c710 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f2eb8009a30 tx=0x7f2eb8008040 comp rx=0 tx=0).stop 2023-08-09T13:47:40.504 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:40.502+0000 7f2ec9404640 1 -- 172.21.15.43:0/28971124 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f2ec4069a30 msgr2=0x7f2ec41f2590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:47:40.504 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:40.502+0000 7f2ec9404640 1 --2- 172.21.15.43:0/28971124 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f2ec4069a30 0x7f2ec41f2590 secure :-1 s=READY pgs=716 cs=0 l=1 rev1=1 crypto rx=0x7f2eb4000f80 tx=0x7f2eb4004a40 comp rx=0 tx=0).stop 2023-08-09T13:47:40.504 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:40.502+0000 7f2ec9404640 1 -- 172.21.15.43:0/28971124 shutdown_connections 2023-08-09T13:47:40.505 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:40.502+0000 7f2ec9404640 1 --2- 172.21.15.43:0/28971124 >> [v2:172.21.15.203:6832/998005540,v1:172.21.15.203:6833/998005540] conn(0x7f2eb006a250 0x7f2eb006c710 unknown :-1 s=CLOSED pgs=66 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:47:40.505 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:40.502+0000 7f2ec9404640 1 --2- 172.21.15.43:0/28971124 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f2ec415d5b0 0x7f2ec40ac1f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:47:40.505 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:40.502+0000 7f2ec9404640 1 --2- 172.21.15.43:0/28971124 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f2ec406a350 0x7f2ec40abcb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:47:40.505 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:40.502+0000 7f2ec9404640 1 --2- 172.21.15.43:0/28971124 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f2ec4069a30 0x7f2ec41f2590 unknown :-1 s=CLOSED pgs=716 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:47:40.506 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:40.502+0000 7f2ec9404640 1 -- 172.21.15.43:0/28971124 >> 172.21.15.43:0/28971124 conn(0x7f2ec406b990 msgr2=0x7f2ec40affa0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-09T13:47:40.506 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:40.502+0000 7f2ec9404640 1 -- 172.21.15.43:0/28971124 shutdown_connections 2023-08-09T13:47:40.506 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:40.502+0000 7f2ec9404640 1 -- 172.21.15.43:0/28971124 wait complete. 2023-08-09T13:47:40.524 DEBUG:tasks.cephfs.test_snap_schedules:res=Schedule removed for path /snap_test_dir1/dir3 2023-08-09T13:47:40.524 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-09T13:47:40.525 DEBUG:teuthology.orchestra.run.smithi043:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/dir3/.snap) 2023-08-09T13:47:40.596 INFO:teuthology.orchestra.run.smithi043.stdout:scheduled-2023-08-09-13_37_00_UTC 2023-08-09T13:47:40.597 INFO:teuthology.orchestra.run.smithi043.stdout:scheduled-2023-08-09-13_38_00_UTC 2023-08-09T13:47:40.597 INFO:teuthology.orchestra.run.smithi043.stdout:scheduled-2023-08-09-13_39_00_UTC 2023-08-09T13:47:40.597 INFO:teuthology.orchestra.run.smithi043.stdout:scheduled-2023-08-09-13_40_00_UTC 2023-08-09T13:47:40.598 INFO:teuthology.orchestra.run.smithi043.stdout:scheduled-2023-08-09-13_41_00_UTC 2023-08-09T13:47:40.598 INFO:teuthology.orchestra.run.smithi043.stdout:scheduled-2023-08-09-13_42_00_UTC 2023-08-09T13:47:40.598 INFO:teuthology.orchestra.run.smithi043.stdout:scheduled-2023-08-09-13_43_01_UTC 2023-08-09T13:47:40.598 INFO:teuthology.orchestra.run.smithi043.stdout:scheduled-2023-08-09-13_44_01_UTC 2023-08-09T13:47:40.599 INFO:teuthology.orchestra.run.smithi043.stdout:scheduled-2023-08-09-13_45_01_UTC 2023-08-09T13:47:40.599 INFO:teuthology.orchestra.run.smithi043.stdout:scheduled-2023-08-09-13_46_00_UTC 2023-08-09T13:47:40.599 INFO:teuthology.orchestra.run.smithi043.stdout:scheduled-2023-08-09-13_47_00_UTC 2023-08-09T13:47:40.601 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/dir3/.snap/scheduled-2023-08-09-13_37_00_UTC 2023-08-09T13:47:40.601 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-09T13:47:40.601 DEBUG:teuthology.orchestra.run.smithi043:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/dir3/.snap/scheduled-2023-08-09-13_37_00_UTC) 2023-08-09T13:47:40.675 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/dir3/.snap/scheduled-2023-08-09-13_38_00_UTC 2023-08-09T13:47:40.676 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-09T13:47:40.676 DEBUG:teuthology.orchestra.run.smithi043:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/dir3/.snap/scheduled-2023-08-09-13_38_00_UTC) 2023-08-09T13:47:40.756 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/dir3/.snap/scheduled-2023-08-09-13_39_00_UTC 2023-08-09T13:47:40.757 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-09T13:47:40.757 DEBUG:teuthology.orchestra.run.smithi043:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/dir3/.snap/scheduled-2023-08-09-13_39_00_UTC) 2023-08-09T13:47:40.835 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/dir3/.snap/scheduled-2023-08-09-13_40_00_UTC 2023-08-09T13:47:40.836 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-09T13:47:40.836 DEBUG:teuthology.orchestra.run.smithi043:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/dir3/.snap/scheduled-2023-08-09-13_40_00_UTC) 2023-08-09T13:47:40.911 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/dir3/.snap/scheduled-2023-08-09-13_41_00_UTC 2023-08-09T13:47:40.911 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-09T13:47:40.911 DEBUG:teuthology.orchestra.run.smithi043:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/dir3/.snap/scheduled-2023-08-09-13_41_00_UTC) 2023-08-09T13:47:40.992 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/dir3/.snap/scheduled-2023-08-09-13_42_00_UTC 2023-08-09T13:47:40.992 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-09T13:47:40.993 DEBUG:teuthology.orchestra.run.smithi043:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/dir3/.snap/scheduled-2023-08-09-13_42_00_UTC) 2023-08-09T13:47:41.071 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/dir3/.snap/scheduled-2023-08-09-13_43_01_UTC 2023-08-09T13:47:41.072 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-09T13:47:41.072 DEBUG:teuthology.orchestra.run.smithi043:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/dir3/.snap/scheduled-2023-08-09-13_43_01_UTC) 2023-08-09T13:47:41.194 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/dir3/.snap/scheduled-2023-08-09-13_44_01_UTC 2023-08-09T13:47:41.194 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-09T13:47:41.194 DEBUG:teuthology.orchestra.run.smithi043:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/dir3/.snap/scheduled-2023-08-09-13_44_01_UTC) 2023-08-09T13:47:41.263 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/dir3/.snap/scheduled-2023-08-09-13_45_01_UTC 2023-08-09T13:47:41.263 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-09T13:47:41.264 DEBUG:teuthology.orchestra.run.smithi043:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/dir3/.snap/scheduled-2023-08-09-13_45_01_UTC) 2023-08-09T13:47:41.316 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/dir3/.snap/scheduled-2023-08-09-13_46_00_UTC 2023-08-09T13:47:41.317 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-09T13:47:41.317 DEBUG:teuthology.orchestra.run.smithi043:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/dir3/.snap/scheduled-2023-08-09-13_46_00_UTC) 2023-08-09T13:47:41.373 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/dir3/.snap/scheduled-2023-08-09-13_47_00_UTC 2023-08-09T13:47:41.373 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-09T13:47:41.374 DEBUG:teuthology.orchestra.run.smithi043:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/dir3/.snap/scheduled-2023-08-09-13_47_00_UTC) 2023-08-09T13:47:41.443 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-09T13:47:41.444 DEBUG:teuthology.orchestra.run.smithi043:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/dir3) 2023-08-09T13:47:41.511 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs snap-schedule remove --fs cephfs /snap_test_dir1/dir4 1M 2023-08-09T13:47:41.790 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:41.789+0000 7fab415d2640 1 Processor -- start 2023-08-09T13:47:41.790 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:41.789+0000 7fab415d2640 1 -- start start 2023-08-09T13:47:41.791 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:41.790+0000 7fab415d2640 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7fab3c15c6f0 0x7fab3c0d03a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:47:41.792 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:41.790+0000 7fab415d2640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fab3c0b5b80 0x7fab3c0d1d20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:47:41.792 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:41.790+0000 7fab415d2640 1 -- --> v1:172.21.15.43:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fab3c0d1890 con 0x7fab3c0b80a0 2023-08-09T13:47:41.792 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:41.790+0000 7fab415d2640 1 -- --> v2:172.21.15.203:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fab3c15cad0 con 0x7fab3c15c6f0 2023-08-09T13:47:41.792 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:41.790+0000 7fab415d2640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fab3c0d0c80 con 0x7fab3c0b5b80 2023-08-09T13:47:41.792 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:41.790+0000 7fab33fff640 1 --1- >> v1:172.21.15.43:6789/0 conn(0x7fab3c0b80a0 0x7fab3c0b5470 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.43:6789/0 says I am v1:172.21.15.43:58224/0 (socket says 172.21.15.43:58224) 2023-08-09T13:47:41.792 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:41.790+0000 7fab33fff640 1 -- 172.21.15.43:0/3240526286 learned_addr learned my addr 172.21.15.43:0/3240526286 (peer_addr_for_me v1:172.21.15.43:0/0) 2023-08-09T13:47:41.794 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:41.791+0000 7fab3affd640 1 --2- 172.21.15.43:0/3240526286 >> v2:172.21.15.203:3300/0 conn(0x7fab3c15c6f0 0x7fab3c0d03a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:47:41.794 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:41.791+0000 7fab3b7fe640 1 --2- 172.21.15.43:0/3240526286 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fab3c0b5b80 0x7fab3c0d1d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:47:41.794 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:41.791+0000 7fab3a7fc640 1 -- 172.21.15.43:0/3240526286 <== mon.0 v1:172.21.15.43:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2736832551 0 0) 0x7fab3c0d1890 con 0x7fab3c0b80a0 2023-08-09T13:47:41.794 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:41.791+0000 7fab3a7fc640 1 -- 172.21.15.43:0/3240526286 --> v1:172.21.15.43:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fab28003520 con 0x7fab3c0b80a0 2023-08-09T13:47:41.794 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:41.791+0000 7fab3a7fc640 1 -- 172.21.15.43:0/3240526286 <== mon.0 v1:172.21.15.43:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1335738548 0 0) 0x7fab28003520 con 0x7fab3c0b80a0 2023-08-09T13:47:41.795 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:41.791+0000 7fab3a7fc640 1 -- 172.21.15.43:0/3240526286 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fab3c0b5b80 msgr2=0x7fab3c0d1d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-09T13:47:41.795 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:41.791+0000 7fab3a7fc640 1 --2- 172.21.15.43:0/3240526286 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fab3c0b5b80 0x7fab3c0d1d20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:47:41.795 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:41.791+0000 7fab3a7fc640 1 -- 172.21.15.43:0/3240526286 >> v2:172.21.15.203:3300/0 conn(0x7fab3c15c6f0 msgr2=0x7fab3c0d03a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-09T13:47:41.795 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:41.791+0000 7fab3a7fc640 1 --2- 172.21.15.43:0/3240526286 >> v2:172.21.15.203:3300/0 conn(0x7fab3c15c6f0 0x7fab3c0d03a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:47:41.795 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:41.791+0000 7fab3a7fc640 1 -- 172.21.15.43:0/3240526286 --> v1:172.21.15.43:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fab3c0d0f60 con 0x7fab3c0b80a0 2023-08-09T13:47:41.795 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:41.791+0000 7fab3a7fc640 1 -- 172.21.15.43:0/3240526286 <== mon.0 v1:172.21.15.43:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1285150740 0 0) 0x7fab24001670 con 0x7fab3c0b80a0 2023-08-09T13:47:41.795 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:41.791+0000 7fab3a7fc640 1 -- 172.21.15.43:0/3240526286 >> v1:172.21.15.43:6789/0 conn(0x7fab3c0b80a0 legacy=0x7fab3c0b5470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:47:41.796 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:41.791+0000 7fab3a7fc640 1 --2- 172.21.15.43:0/3240526286 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fab280039e0 0x7fab28003e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:47:41.796 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:41.791+0000 7fab3a7fc640 1 --2- 172.21.15.43:0/3240526286 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fab280047c0 0x7fab28004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:47:41.796 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:41.791+0000 7fab3a7fc640 1 --2- 172.21.15.43:0/3240526286 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fab3c0b5b80 0x7fab28005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:47:41.796 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:41.791+0000 7fab3a7fc640 1 -- 172.21.15.43:0/3240526286 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fab3c15cad0 con 0x7fab280047c0 2023-08-09T13:47:41.796 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:41.791+0000 7fab3a7fc640 1 -- 172.21.15.43:0/3240526286 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fab3c0d0c80 con 0x7fab280039e0 2023-08-09T13:47:41.796 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:41.791+0000 7fab3a7fc640 1 -- 172.21.15.43:0/3240526286 --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fab2800b980 con 0x7fab3c0b5b80 2023-08-09T13:47:41.796 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:41.792+0000 7fab33fff640 1 --2- 172.21.15.43:0/3240526286 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fab280047c0 0x7fab28004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:47:41.797 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:41.792+0000 7fab3affd640 1 --2- 172.21.15.43:0/3240526286 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fab280039e0 0x7fab28003e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:47:41.797 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:41.792+0000 7fab3b7fe640 1 --2- 172.21.15.43:0/3240526286 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fab3c0b5b80 0x7fab28005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:47:41.797 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:41.792+0000 7fab3affd640 1 -- 172.21.15.43:0/3240526286 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fab3c0b5b80 msgr2=0x7fab28005180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-09T13:47:41.797 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:41.792+0000 7fab3affd640 1 --2- 172.21.15.43:0/3240526286 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fab3c0b5b80 0x7fab28005180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:47:41.797 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:41.792+0000 7fab3affd640 1 -- 172.21.15.43:0/3240526286 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fab280047c0 msgr2=0x7fab28004c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-09T13:47:41.797 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:41.792+0000 7fab3affd640 1 --2- 172.21.15.43:0/3240526286 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fab280047c0 0x7fab28004c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:47:41.797 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:41.792+0000 7fab3affd640 1 -- 172.21.15.43:0/3240526286 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fab28003520 con 0x7fab280039e0 2023-08-09T13:47:41.797 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:41.792+0000 7fab33fff640 1 --2- 172.21.15.43:0/3240526286 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fab280047c0 0x7fab28004c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-08-09T13:47:41.799 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:41.793+0000 7fab3affd640 1 --2- 172.21.15.43:0/3240526286 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fab280039e0 0x7fab28003e00 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7fab2c009ac0 tx=0x7fab280057f0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=f05e411736915e57 server_cookie=0 in_seq=0 out_seq=0 2023-08-09T13:47:41.800 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:41.794+0000 7fab3a7fc640 1 -- 172.21.15.43:0/3240526286 <== mon.1 v2:172.21.15.203:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fab2c02c710 con 0x7fab280039e0 2023-08-09T13:47:41.800 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:41.794+0000 7fab3a7fc640 1 -- 172.21.15.43:0/3240526286 <== mon.1 v2:172.21.15.203:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fab2c02c870 con 0x7fab280039e0 2023-08-09T13:47:41.800 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:41.794+0000 7fab3a7fc640 1 -- 172.21.15.43:0/3240526286 <== mon.1 v2:172.21.15.203:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fab2c02cc00 con 0x7fab280039e0 2023-08-09T13:47:41.801 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:41.794+0000 7fab415d2640 1 -- 172.21.15.43:0/3240526286 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fab280039e0 msgr2=0x7fab28003e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:47:41.801 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:41.794+0000 7fab415d2640 1 --2- 172.21.15.43:0/3240526286 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fab280039e0 0x7fab28003e00 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7fab2c009ac0 tx=0x7fab280057f0 comp rx=0 tx=0).stop 2023-08-09T13:47:41.801 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:41.794+0000 7fab3b7fe640 1 -- 172.21.15.43:0/3240526286 reap_dead start 2023-08-09T13:47:41.801 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:41.794+0000 7fab415d2640 1 -- 172.21.15.43:0/3240526286 shutdown_connections 2023-08-09T13:47:41.802 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:41.794+0000 7fab415d2640 1 -- 172.21.15.43:0/3240526286 >> 172.21.15.43:0/3240526286 conn(0x7fab3c0712e0 msgr2=0x7fab3c0716f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-09T13:47:41.802 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:41.794+0000 7fab415d2640 1 -- 172.21.15.43:0/3240526286 shutdown_connections 2023-08-09T13:47:41.802 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:41.794+0000 7fab415d2640 1 -- 172.21.15.43:0/3240526286 wait complete. 2023-08-09T13:47:41.802 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:41.796+0000 7fab415d2640 1 Processor -- start 2023-08-09T13:47:41.802 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:41.796+0000 7fab415d2640 1 -- start start 2023-08-09T13:47:41.802 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:41.797+0000 7fab415d2640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fab280039e0 0x7fab3c157420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:47:41.802 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:41.797+0000 7fab415d2640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fab3c0c7eb0 0x7fab3c1537e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:47:41.803 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:41.797+0000 7fab415d2640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fab3c1f17d0 0x7fab3c1f1c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:47:41.803 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:41.797+0000 7fab415d2640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fab3c157a00 con 0x7fab3c1f17d0 2023-08-09T13:47:41.803 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:41.797+0000 7fab415d2640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fab3c157ce0 con 0x7fab280039e0 2023-08-09T13:47:41.803 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:41.797+0000 7fab415d2640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fab3c157fc0 con 0x7fab3c0c7eb0 2023-08-09T13:47:41.803 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:41.798+0000 7fab3b7fe640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fab3c1f17d0 0x7fab3c1f1c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:47:41.803 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:41.798+0000 7fab3affd640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fab280039e0 0x7fab3c157420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:47:41.803 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:41.798+0000 7fab3b7fe640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fab3c1f17d0 0x7fab3c1f1c50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:40308/0 (socket says 172.21.15.43:40308) 2023-08-09T13:47:41.803 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:41.798+0000 7fab3b7fe640 1 -- 172.21.15.43:0/2893177004 learned_addr learned my addr 172.21.15.43:0/2893177004 (peer_addr_for_me v2:172.21.15.43:0/0) 2023-08-09T13:47:41.804 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:41.798+0000 7fab33fff640 1 --2- 172.21.15.43:0/2893177004 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fab3c0c7eb0 0x7fab3c1537e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:47:41.804 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:41.798+0000 7fab3b7fe640 1 -- 172.21.15.43:0/2893177004 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fab3c0c7eb0 msgr2=0x7fab3c1537e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:47:41.804 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:41.798+0000 7fab3b7fe640 1 --2- 172.21.15.43:0/2893177004 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fab3c0c7eb0 0x7fab3c1537e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:47:41.804 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:41.798+0000 7fab3b7fe640 1 -- 172.21.15.43:0/2893177004 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fab280039e0 msgr2=0x7fab3c157420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:47:41.804 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:41.798+0000 7fab3b7fe640 1 --2- 172.21.15.43:0/2893177004 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fab280039e0 0x7fab3c157420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:47:41.804 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:41.798+0000 7fab3b7fe640 1 -- 172.21.15.43:0/2893177004 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fab3c1582a0 con 0x7fab3c1f17d0 2023-08-09T13:47:41.804 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:41.798+0000 7fab33fff640 1 --2- 172.21.15.43:0/2893177004 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fab3c0c7eb0 0x7fab3c1537e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-08-09T13:47:41.806 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:41.798+0000 7fab3b7fe640 1 --2- 172.21.15.43:0/2893177004 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fab3c1f17d0 0x7fab3c1f1c50 secure :-1 s=READY pgs=718 cs=0 l=1 rev1=1 crypto rx=0x7fab3c0cf800 tx=0x7fab3400f580 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-09T13:47:41.806 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:41.799+0000 7fab337fe640 1 -- 172.21.15.43:0/2893177004 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fab3400e040 con 0x7fab3c1f17d0 2023-08-09T13:47:41.806 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:41.799+0000 7fab337fe640 1 -- 172.21.15.43:0/2893177004 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fab3400f040 con 0x7fab3c1f17d0 2023-08-09T13:47:41.806 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:41.799+0000 7fab337fe640 1 -- 172.21.15.43:0/2893177004 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fab34015430 con 0x7fab3c1f17d0 2023-08-09T13:47:41.806 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:41.799+0000 7fab415d2640 1 -- 172.21.15.43:0/2893177004 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fab3c158580 con 0x7fab3c1f17d0 2023-08-09T13:47:41.806 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:41.799+0000 7fab415d2640 1 -- 172.21.15.43:0/2893177004 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fab3c0c8310 con 0x7fab3c1f17d0 2023-08-09T13:47:41.807 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:41.800+0000 7fab415d2640 1 -- 172.21.15.43:0/2893177004 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fab3c0739f0 con 0x7fab3c1f17d0 2023-08-09T13:47:41.807 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:41.801+0000 7fab337fe640 1 -- 172.21.15.43:0/2893177004 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 13) v1 ==== 88529+0+0 (secure 0 0 0) 0x7fab34015590 con 0x7fab3c1f17d0 2023-08-09T13:47:41.807 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:41.802+0000 7fab337fe640 1 --2- 172.21.15.43:0/2893177004 >> [v2:172.21.15.203:6832/998005540,v1:172.21.15.203:6833/998005540] conn(0x7fab1c06a280 0x7fab1c06c740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:47:41.807 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:41.802+0000 7fab337fe640 1 -- 172.21.15.43:0/2893177004 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(48..48 src has 1..48) v4 ==== 7362+0+0 (secure 0 0 0) 0x7fab34089480 con 0x7fab3c1f17d0 2023-08-09T13:47:41.807 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:41.803+0000 7fab3affd640 1 --2- 172.21.15.43:0/2893177004 >> [v2:172.21.15.203:6832/998005540,v1:172.21.15.203:6833/998005540] conn(0x7fab1c06a280 0x7fab1c06c740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:47:41.807 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:41.803+0000 7fab3affd640 1 --2- 172.21.15.43:0/2893177004 >> [v2:172.21.15.203:6832/998005540,v1:172.21.15.203:6833/998005540] conn(0x7fab1c06a280 0x7fab1c06c740 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7fab2c009ac0 tx=0x7fab2c030000 comp rx=0 tx=0).ready entity=mgr.5003 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-09T13:47:41.808 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:41.806+0000 7fab337fe640 1 -- 172.21.15.43:0/2893177004 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178066 (secure 0 0 0) 0x7fab34054230 con 0x7fab3c1f17d0 2023-08-09T13:47:42.037 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:42.036+0000 7fab415d2640 1 -- 172.21.15.43:0/2893177004 --> [v2:172.21.15.203:6832/998005540,v1:172.21.15.203:6833/998005540] -- mgr_command(tid 0: {"prefix": "fs snap-schedule remove", "fs": "cephfs", "path": "/snap_test_dir1/dir4", "repeat": "1M", "target": ["mon-mgr", ""]}) v1 -- 0x7fab3c0d1520 con 0x7fab1c06a280 2023-08-09T13:47:42.261 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:42.260+0000 7fab337fe640 1 -- 172.21.15.43:0/2893177004 <== mgr.5003 v2:172.21.15.203:6832/998005540 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+47 (secure 0 0 0) 0x7fab2c02ca60 con 0x7fab1c06a280 2023-08-09T13:47:42.262 INFO:teuthology.orchestra.run.smithi043.stdout:Schedule removed for path /snap_test_dir1/dir4 2023-08-09T13:47:42.265 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:42.263+0000 7fab415d2640 1 -- 172.21.15.43:0/2893177004 >> [v2:172.21.15.203:6832/998005540,v1:172.21.15.203:6833/998005540] conn(0x7fab1c06a280 msgr2=0x7fab1c06c740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:47:42.265 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:42.264+0000 7fab415d2640 1 --2- 172.21.15.43:0/2893177004 >> [v2:172.21.15.203:6832/998005540,v1:172.21.15.203:6833/998005540] conn(0x7fab1c06a280 0x7fab1c06c740 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7fab2c009ac0 tx=0x7fab2c030000 comp rx=0 tx=0).stop 2023-08-09T13:47:42.265 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:42.264+0000 7fab415d2640 1 -- 172.21.15.43:0/2893177004 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fab3c1f17d0 msgr2=0x7fab3c1f1c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:47:42.266 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:42.264+0000 7fab415d2640 1 --2- 172.21.15.43:0/2893177004 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fab3c1f17d0 0x7fab3c1f1c50 secure :-1 s=READY pgs=718 cs=0 l=1 rev1=1 crypto rx=0x7fab3c0cf800 tx=0x7fab3400f580 comp rx=0 tx=0).stop 2023-08-09T13:47:42.266 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:42.264+0000 7fab415d2640 1 -- 172.21.15.43:0/2893177004 shutdown_connections 2023-08-09T13:47:42.266 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:42.264+0000 7fab415d2640 1 --2- 172.21.15.43:0/2893177004 >> [v2:172.21.15.203:6832/998005540,v1:172.21.15.203:6833/998005540] conn(0x7fab1c06a280 0x7fab1c06c740 unknown :-1 s=CLOSED pgs=67 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:47:42.266 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:42.264+0000 7fab415d2640 1 --2- 172.21.15.43:0/2893177004 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fab3c1f17d0 0x7fab3c1f1c50 unknown :-1 s=CLOSED pgs=718 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:47:42.267 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:42.264+0000 7fab415d2640 1 --2- 172.21.15.43:0/2893177004 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fab3c0c7eb0 0x7fab3c1537e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:47:42.267 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:42.264+0000 7fab415d2640 1 --2- 172.21.15.43:0/2893177004 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fab280039e0 0x7fab3c157420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:47:42.267 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:42.264+0000 7fab415d2640 1 -- 172.21.15.43:0/2893177004 >> 172.21.15.43:0/2893177004 conn(0x7fab3c0712e0 msgr2=0x7fab3c15d660 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-09T13:47:42.267 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:42.264+0000 7fab415d2640 1 -- 172.21.15.43:0/2893177004 shutdown_connections 2023-08-09T13:47:42.267 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:42.264+0000 7fab415d2640 1 -- 172.21.15.43:0/2893177004 wait complete. 2023-08-09T13:47:42.286 DEBUG:tasks.cephfs.test_snap_schedules:res=Schedule removed for path /snap_test_dir1/dir4 2023-08-09T13:47:42.286 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-09T13:47:42.286 DEBUG:teuthology.orchestra.run.smithi043:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/dir4/.snap) 2023-08-09T13:47:42.353 INFO:teuthology.orchestra.run.smithi043.stdout:scheduled-2023-08-09-13_37_01_UTC 2023-08-09T13:47:42.354 INFO:teuthology.orchestra.run.smithi043.stdout:scheduled-2023-08-09-13_38_00_UTC 2023-08-09T13:47:42.354 INFO:teuthology.orchestra.run.smithi043.stdout:scheduled-2023-08-09-13_39_00_UTC 2023-08-09T13:47:42.354 INFO:teuthology.orchestra.run.smithi043.stdout:scheduled-2023-08-09-13_40_00_UTC 2023-08-09T13:47:42.354 INFO:teuthology.orchestra.run.smithi043.stdout:scheduled-2023-08-09-13_41_00_UTC 2023-08-09T13:47:42.354 INFO:teuthology.orchestra.run.smithi043.stdout:scheduled-2023-08-09-13_42_00_UTC 2023-08-09T13:47:42.354 INFO:teuthology.orchestra.run.smithi043.stdout:scheduled-2023-08-09-13_43_00_UTC 2023-08-09T13:47:42.354 INFO:teuthology.orchestra.run.smithi043.stdout:scheduled-2023-08-09-13_44_00_UTC 2023-08-09T13:47:42.355 INFO:teuthology.orchestra.run.smithi043.stdout:scheduled-2023-08-09-13_45_00_UTC 2023-08-09T13:47:42.355 INFO:teuthology.orchestra.run.smithi043.stdout:scheduled-2023-08-09-13_46_00_UTC 2023-08-09T13:47:42.355 INFO:teuthology.orchestra.run.smithi043.stdout:scheduled-2023-08-09-13_47_00_UTC 2023-08-09T13:47:42.356 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/dir4/.snap/scheduled-2023-08-09-13_37_01_UTC 2023-08-09T13:47:42.356 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-09T13:47:42.356 DEBUG:teuthology.orchestra.run.smithi043:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/dir4/.snap/scheduled-2023-08-09-13_37_01_UTC) 2023-08-09T13:47:42.432 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/dir4/.snap/scheduled-2023-08-09-13_38_00_UTC 2023-08-09T13:47:42.432 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-09T13:47:42.432 DEBUG:teuthology.orchestra.run.smithi043:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/dir4/.snap/scheduled-2023-08-09-13_38_00_UTC) 2023-08-09T13:47:42.511 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/dir4/.snap/scheduled-2023-08-09-13_39_00_UTC 2023-08-09T13:47:42.512 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-09T13:47:42.512 DEBUG:teuthology.orchestra.run.smithi043:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/dir4/.snap/scheduled-2023-08-09-13_39_00_UTC) 2023-08-09T13:47:42.593 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/dir4/.snap/scheduled-2023-08-09-13_40_00_UTC 2023-08-09T13:47:42.594 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-09T13:47:42.594 DEBUG:teuthology.orchestra.run.smithi043:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/dir4/.snap/scheduled-2023-08-09-13_40_00_UTC) 2023-08-09T13:47:42.675 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/dir4/.snap/scheduled-2023-08-09-13_41_00_UTC 2023-08-09T13:47:42.675 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-09T13:47:42.675 DEBUG:teuthology.orchestra.run.smithi043:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/dir4/.snap/scheduled-2023-08-09-13_41_00_UTC) 2023-08-09T13:47:42.754 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/dir4/.snap/scheduled-2023-08-09-13_42_00_UTC 2023-08-09T13:47:42.754 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-09T13:47:42.755 DEBUG:teuthology.orchestra.run.smithi043:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/dir4/.snap/scheduled-2023-08-09-13_42_00_UTC) 2023-08-09T13:47:42.835 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/dir4/.snap/scheduled-2023-08-09-13_43_00_UTC 2023-08-09T13:47:42.835 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-09T13:47:42.836 DEBUG:teuthology.orchestra.run.smithi043:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/dir4/.snap/scheduled-2023-08-09-13_43_00_UTC) 2023-08-09T13:47:42.911 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/dir4/.snap/scheduled-2023-08-09-13_44_00_UTC 2023-08-09T13:47:42.911 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-09T13:47:42.911 DEBUG:teuthology.orchestra.run.smithi043:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/dir4/.snap/scheduled-2023-08-09-13_44_00_UTC) 2023-08-09T13:47:42.990 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/dir4/.snap/scheduled-2023-08-09-13_45_00_UTC 2023-08-09T13:47:42.991 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-09T13:47:42.991 DEBUG:teuthology.orchestra.run.smithi043:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/dir4/.snap/scheduled-2023-08-09-13_45_00_UTC) 2023-08-09T13:47:43.072 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/dir4/.snap/scheduled-2023-08-09-13_46_00_UTC 2023-08-09T13:47:43.073 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-09T13:47:43.073 DEBUG:teuthology.orchestra.run.smithi043:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/dir4/.snap/scheduled-2023-08-09-13_46_00_UTC) 2023-08-09T13:47:43.144 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/dir4/.snap/scheduled-2023-08-09-13_47_00_UTC 2023-08-09T13:47:43.144 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-09T13:47:43.144 DEBUG:teuthology.orchestra.run.smithi043:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/dir4/.snap/scheduled-2023-08-09-13_47_00_UTC) 2023-08-09T13:47:43.222 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-09T13:47:43.222 DEBUG:teuthology.orchestra.run.smithi043:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/dir4) 2023-08-09T13:47:43.283 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs snap-schedule remove --fs cephfs /snap_test_dir1/dir5 1M 2023-08-09T13:47:43.494 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:43.492+0000 7f0dcffff640 1 Processor -- start 2023-08-09T13:47:43.495 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:43.493+0000 7f0dcffff640 1 -- start start 2023-08-09T13:47:43.498 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:43.495+0000 7f0dcffff640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f0dd00c5950 0x7f0dd0070d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:47:43.499 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:43.495+0000 7f0dcffff640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f0dd0145750 0x7f0dd006d7a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:47:43.499 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:43.495+0000 7f0dcffff640 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7f0dd006dce0 0x7f0dd00697c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:47:43.499 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:43.495+0000 7f0dcffff640 1 -- --> v2:172.21.15.43:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0dd00cceb0 con 0x7f0dd00c5950 2023-08-09T13:47:43.499 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:43.495+0000 7f0dcffff640 1 -- --> v2:172.21.15.203:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0dd0145510 con 0x7f0dd006dce0 2023-08-09T13:47:43.500 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:43.495+0000 7f0dcffff640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0dd006e9a0 con 0x7f0dd0145750 2023-08-09T13:47:43.500 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:43.495+0000 7f0dceffd640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f0dd00c5950 0x7f0dd0070d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:47:43.500 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:43.496+0000 7f0dceffd640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f0dd00c5950 0x7f0dd0070d30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:40312/0 (socket says 172.21.15.43:40312) 2023-08-09T13:47:43.501 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:43.496+0000 7f0dceffd640 1 -- 172.21.15.43:0/695491741 learned_addr learned my addr 172.21.15.43:0/695491741 (peer_addr_for_me v2:172.21.15.43:0/0) 2023-08-09T13:47:43.501 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:43.496+0000 7f0dcf7fe640 1 --2- 172.21.15.43:0/695491741 >> v2:172.21.15.203:3300/0 conn(0x7f0dd006dce0 0x7f0dd00697c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:47:43.501 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:43.496+0000 7f0dce7fc640 1 --2- 172.21.15.43:0/695491741 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f0dd0145750 0x7f0dd006d7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:47:43.501 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:43.496+0000 7f0dceffd640 1 -- 172.21.15.43:0/695491741 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f0dd0145750 msgr2=0x7f0dd006d7a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-09T13:47:43.502 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:43.496+0000 7f0dceffd640 1 --2- 172.21.15.43:0/695491741 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f0dd0145750 0x7f0dd006d7a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:47:43.502 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:43.496+0000 7f0dceffd640 1 -- 172.21.15.43:0/695491741 >> v2:172.21.15.203:3300/0 conn(0x7f0dd006dce0 msgr2=0x7f0dd00697c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-09T13:47:43.502 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:43.496+0000 7f0dceffd640 1 --2- 172.21.15.43:0/695491741 >> v2:172.21.15.203:3300/0 conn(0x7f0dd006dce0 0x7f0dd00697c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:47:43.502 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:43.496+0000 7f0dceffd640 1 -- 172.21.15.43:0/695491741 --> v2:172.21.15.43:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0dd006ec60 con 0x7f0dd00c5950 2023-08-09T13:47:43.502 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:43.496+0000 7f0dcf7fe640 1 --2- 172.21.15.43:0/695491741 >> v2:172.21.15.203:3300/0 conn(0x7f0dd006dce0 0x7f0dd00697c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-08-09T13:47:43.503 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:43.496+0000 7f0dce7fc640 1 --2- 172.21.15.43:0/695491741 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f0dd0145750 0x7f0dd006d7a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-08-09T13:47:43.506 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:43.498+0000 7f0dceffd640 1 --2- 172.21.15.43:0/695491741 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f0dd00c5950 0x7f0dd0070d30 secure :-1 s=READY pgs=719 cs=0 l=1 rev1=1 crypto rx=0x7f0dd00cee60 tx=0x7f0dc002b910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=eff65febf051467f server_cookie=0 in_seq=0 out_seq=0 2023-08-09T13:47:43.506 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:43.498+0000 7f0dcdffb640 1 -- 172.21.15.43:0/695491741 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0dc0031870 con 0x7f0dd00c5950 2023-08-09T13:47:43.506 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:43.498+0000 7f0dcdffb640 1 -- 172.21.15.43:0/695491741 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0dc00319d0 con 0x7f0dd00c5950 2023-08-09T13:47:43.507 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:43.498+0000 7f0dcdffb640 1 -- 172.21.15.43:0/695491741 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0dc0031d60 con 0x7f0dd00c5950 2023-08-09T13:47:43.507 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:43.499+0000 7f0dcffff640 1 -- 172.21.15.43:0/695491741 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f0dd00c5950 msgr2=0x7f0dd0070d30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:47:43.507 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:43.499+0000 7f0dcffff640 1 --2- 172.21.15.43:0/695491741 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f0dd00c5950 0x7f0dd0070d30 secure :-1 s=READY pgs=719 cs=0 l=1 rev1=1 crypto rx=0x7f0dd00cee60 tx=0x7f0dc002b910 comp rx=0 tx=0).stop 2023-08-09T13:47:43.507 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:43.499+0000 7f0dcffff640 1 -- 172.21.15.43:0/695491741 shutdown_connections 2023-08-09T13:47:43.507 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:43.499+0000 7f0dcffff640 1 --2- 172.21.15.43:0/695491741 >> v2:172.21.15.203:3300/0 conn(0x7f0dd006dce0 0x7f0dd00697c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:47:43.507 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:43.499+0000 7f0dcffff640 1 --2- 172.21.15.43:0/695491741 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f0dd0145750 0x7f0dd006d7a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:47:43.507 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:43.499+0000 7f0dcffff640 1 --2- 172.21.15.43:0/695491741 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f0dd00c5950 0x7f0dd0070d30 unknown :-1 s=CLOSED pgs=719 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:47:43.507 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:43.499+0000 7f0dcffff640 1 -- 172.21.15.43:0/695491741 >> 172.21.15.43:0/695491741 conn(0x7f0dd00cba30 msgr2=0x7f0dd00cbe40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-09T13:47:43.508 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:43.499+0000 7f0dcffff640 1 -- 172.21.15.43:0/695491741 shutdown_connections 2023-08-09T13:47:43.508 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:43.499+0000 7f0dcffff640 1 -- 172.21.15.43:0/695491741 wait complete. 2023-08-09T13:47:43.508 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:43.502+0000 7f0dcffff640 1 Processor -- start 2023-08-09T13:47:43.508 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:43.502+0000 7f0dcffff640 1 -- start start 2023-08-09T13:47:43.508 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:43.503+0000 7f0dcffff640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f0dd006dce0 0x7f0dd00b2c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:47:43.508 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:43.503+0000 7f0dcffff640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f0dd00c5950 0x7f0dd00acb80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:47:43.508 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:43.503+0000 7f0dcffff640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f0dd0145750 0x7f0dd015ba60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:47:43.509 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:43.503+0000 7f0dcffff640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0dd015ca10 con 0x7f0dd0145750 2023-08-09T13:47:43.509 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:43.503+0000 7f0dcffff640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0dd00b49f0 con 0x7f0dd00c5950 2023-08-09T13:47:43.509 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:43.503+0000 7f0dcffff640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0dd015ccb0 con 0x7f0dd006dce0 2023-08-09T13:47:43.509 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:43.503+0000 7f0dcf7fe640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f0dd0145750 0x7f0dd015ba60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:47:43.509 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:43.503+0000 7f0dcf7fe640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f0dd0145750 0x7f0dd015ba60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:40328/0 (socket says 172.21.15.43:40328) 2023-08-09T13:47:43.509 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:43.503+0000 7f0dcf7fe640 1 -- 172.21.15.43:0/1044184673 learned_addr learned my addr 172.21.15.43:0/1044184673 (peer_addr_for_me v2:172.21.15.43:0/0) 2023-08-09T13:47:43.509 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:43.504+0000 7f0dcf7fe640 1 -- 172.21.15.43:0/1044184673 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f0dd006dce0 msgr2=0x7f0dd00b2c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:47:43.510 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:43.504+0000 7f0dceffd640 1 --2- 172.21.15.43:0/1044184673 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f0dd006dce0 0x7f0dd00b2c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:47:43.510 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:43.504+0000 7f0dcf7fe640 1 --2- 172.21.15.43:0/1044184673 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f0dd006dce0 0x7f0dd00b2c50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:47:43.510 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:43.504+0000 7f0dcf7fe640 1 -- 172.21.15.43:0/1044184673 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f0dd00c5950 msgr2=0x7f0dd00acb80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:47:43.510 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:43.504+0000 7f0dcf7fe640 1 --2- 172.21.15.43:0/1044184673 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f0dd00c5950 0x7f0dd00acb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:47:43.510 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:43.504+0000 7f0dcf7fe640 1 -- 172.21.15.43:0/1044184673 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0dc00094d0 con 0x7f0dd0145750 2023-08-09T13:47:43.510 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:43.504+0000 7f0dcf7fe640 1 --2- 172.21.15.43:0/1044184673 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f0dd0145750 0x7f0dd015ba60 secure :-1 s=READY pgs=720 cs=0 l=1 rev1=1 crypto rx=0x7f0dc800cd10 tx=0x7f0dc800a650 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-09T13:47:43.510 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:43.504+0000 7f0db77fe640 1 -- 172.21.15.43:0/1044184673 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0dc800f680 con 0x7f0dd0145750 2023-08-09T13:47:43.511 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:43.504+0000 7f0db77fe640 1 -- 172.21.15.43:0/1044184673 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0dc800f800 con 0x7f0dd0145750 2023-08-09T13:47:43.511 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:43.504+0000 7f0db77fe640 1 -- 172.21.15.43:0/1044184673 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0dc800fbb0 con 0x7f0dd0145750 2023-08-09T13:47:43.511 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:43.505+0000 7f0dcffff640 1 -- 172.21.15.43:0/1044184673 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0dd015cf90 con 0x7f0dd0145750 2023-08-09T13:47:43.511 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:43.505+0000 7f0dcffff640 1 -- 172.21.15.43:0/1044184673 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0dd00ad0c0 con 0x7f0dd0145750 2023-08-09T13:47:43.512 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:43.506+0000 7f0dcffff640 1 -- 172.21.15.43:0/1044184673 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0dd00739f0 con 0x7f0dd0145750 2023-08-09T13:47:43.512 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:43.508+0000 7f0db77fe640 1 -- 172.21.15.43:0/1044184673 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 13) v1 ==== 88529+0+0 (secure 0 0 0) 0x7f0dc800b040 con 0x7f0dd0145750 2023-08-09T13:47:43.512 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:43.509+0000 7f0db77fe640 1 --2- 172.21.15.43:0/1044184673 >> [v2:172.21.15.203:6832/998005540,v1:172.21.15.203:6833/998005540] conn(0x7f0dbc06a250 0x7f0dbc06c710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:47:43.512 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:43.509+0000 7f0db77fe640 1 -- 172.21.15.43:0/1044184673 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(50..50 src has 1..50) v4 ==== 7282+0+0 (secure 0 0 0) 0x7f0dc8089810 con 0x7f0dd0145750 2023-08-09T13:47:43.512 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:43.510+0000 7f0dceffd640 1 --2- 172.21.15.43:0/1044184673 >> [v2:172.21.15.203:6832/998005540,v1:172.21.15.203:6833/998005540] conn(0x7f0dbc06a250 0x7f0dbc06c710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:47:43.513 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:43.512+0000 7f0dceffd640 1 --2- 172.21.15.43:0/1044184673 >> [v2:172.21.15.203:6832/998005540,v1:172.21.15.203:6833/998005540] conn(0x7f0dbc06a250 0x7f0dbc06c710 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f0dc0036000 tx=0x7f0dc003c040 comp rx=0 tx=0).ready entity=mgr.5003 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-09T13:47:43.515 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:43.514+0000 7f0db77fe640 1 -- 172.21.15.43:0/1044184673 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178066 (secure 0 0 0) 0x7f0dc8054730 con 0x7f0dd0145750 2023-08-09T13:47:43.735 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:43.734+0000 7f0dcffff640 1 -- 172.21.15.43:0/1044184673 --> [v2:172.21.15.203:6832/998005540,v1:172.21.15.203:6833/998005540] -- mgr_command(tid 0: {"prefix": "fs snap-schedule remove", "fs": "cephfs", "path": "/snap_test_dir1/dir5", "repeat": "1M", "target": ["mon-mgr", ""]}) v1 -- 0x7f0dd00cd370 con 0x7f0dbc06a250 2023-08-09T13:47:43.938 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:43.937+0000 7f0db77fe640 1 -- 172.21.15.43:0/1044184673 <== mgr.5003 v2:172.21.15.203:6832/998005540 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+47 (secure 0 0 0) 0x7f0dc003a420 con 0x7f0dbc06a250 2023-08-09T13:47:43.939 INFO:teuthology.orchestra.run.smithi043.stdout:Schedule removed for path /snap_test_dir1/dir5 2023-08-09T13:47:43.942 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:43.941+0000 7f0dcffff640 1 -- 172.21.15.43:0/1044184673 >> [v2:172.21.15.203:6832/998005540,v1:172.21.15.203:6833/998005540] conn(0x7f0dbc06a250 msgr2=0x7f0dbc06c710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:47:43.942 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:43.941+0000 7f0dcffff640 1 --2- 172.21.15.43:0/1044184673 >> [v2:172.21.15.203:6832/998005540,v1:172.21.15.203:6833/998005540] conn(0x7f0dbc06a250 0x7f0dbc06c710 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f0dc0036000 tx=0x7f0dc003c040 comp rx=0 tx=0).stop 2023-08-09T13:47:43.942 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:43.941+0000 7f0dcffff640 1 -- 172.21.15.43:0/1044184673 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f0dd0145750 msgr2=0x7f0dd015ba60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:47:43.943 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:43.941+0000 7f0dcffff640 1 --2- 172.21.15.43:0/1044184673 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f0dd0145750 0x7f0dd015ba60 secure :-1 s=READY pgs=720 cs=0 l=1 rev1=1 crypto rx=0x7f0dc800cd10 tx=0x7f0dc800a650 comp rx=0 tx=0).stop 2023-08-09T13:47:43.943 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:43.941+0000 7f0dcffff640 1 -- 172.21.15.43:0/1044184673 shutdown_connections 2023-08-09T13:47:43.944 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:43.941+0000 7f0dcffff640 1 --2- 172.21.15.43:0/1044184673 >> [v2:172.21.15.203:6832/998005540,v1:172.21.15.203:6833/998005540] conn(0x7f0dbc06a250 0x7f0dbc06c710 unknown :-1 s=CLOSED pgs=68 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:47:43.944 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:43.941+0000 7f0dcffff640 1 --2- 172.21.15.43:0/1044184673 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f0dd0145750 0x7f0dd015ba60 unknown :-1 s=CLOSED pgs=720 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:47:43.944 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:43.941+0000 7f0dcffff640 1 --2- 172.21.15.43:0/1044184673 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f0dd00c5950 0x7f0dd00acb80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:47:43.945 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:43.941+0000 7f0dcffff640 1 --2- 172.21.15.43:0/1044184673 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f0dd006dce0 0x7f0dd00b2c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:47:43.945 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:43.941+0000 7f0dcffff640 1 -- 172.21.15.43:0/1044184673 >> 172.21.15.43:0/1044184673 conn(0x7f0dd00cba30 msgr2=0x7f0dd0144600 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-09T13:47:43.945 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:43.941+0000 7f0dcffff640 1 -- 172.21.15.43:0/1044184673 shutdown_connections 2023-08-09T13:47:43.945 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:43.941+0000 7f0dcffff640 1 -- 172.21.15.43:0/1044184673 wait complete. 2023-08-09T13:47:43.962 DEBUG:tasks.cephfs.test_snap_schedules:res=Schedule removed for path /snap_test_dir1/dir5 2023-08-09T13:47:43.962 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-09T13:47:43.962 DEBUG:teuthology.orchestra.run.smithi043:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/dir5/.snap) 2023-08-09T13:47:44.066 INFO:teuthology.orchestra.run.smithi043.stdout:scheduled-2023-08-09-13_37_00_UTC 2023-08-09T13:47:44.066 INFO:teuthology.orchestra.run.smithi043.stdout:scheduled-2023-08-09-13_38_00_UTC 2023-08-09T13:47:44.066 INFO:teuthology.orchestra.run.smithi043.stdout:scheduled-2023-08-09-13_39_00_UTC 2023-08-09T13:47:44.067 INFO:teuthology.orchestra.run.smithi043.stdout:scheduled-2023-08-09-13_40_00_UTC 2023-08-09T13:47:44.067 INFO:teuthology.orchestra.run.smithi043.stdout:scheduled-2023-08-09-13_41_00_UTC 2023-08-09T13:47:44.067 INFO:teuthology.orchestra.run.smithi043.stdout:scheduled-2023-08-09-13_42_00_UTC 2023-08-09T13:47:44.067 INFO:teuthology.orchestra.run.smithi043.stdout:scheduled-2023-08-09-13_43_00_UTC 2023-08-09T13:47:44.068 INFO:teuthology.orchestra.run.smithi043.stdout:scheduled-2023-08-09-13_44_00_UTC 2023-08-09T13:47:44.068 INFO:teuthology.orchestra.run.smithi043.stdout:scheduled-2023-08-09-13_45_01_UTC 2023-08-09T13:47:44.068 INFO:teuthology.orchestra.run.smithi043.stdout:scheduled-2023-08-09-13_46_01_UTC 2023-08-09T13:47:44.069 INFO:teuthology.orchestra.run.smithi043.stdout:scheduled-2023-08-09-13_47_00_UTC 2023-08-09T13:47:44.070 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/dir5/.snap/scheduled-2023-08-09-13_37_00_UTC 2023-08-09T13:47:44.071 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-09T13:47:44.071 DEBUG:teuthology.orchestra.run.smithi043:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/dir5/.snap/scheduled-2023-08-09-13_37_00_UTC) 2023-08-09T13:47:44.137 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/dir5/.snap/scheduled-2023-08-09-13_38_00_UTC 2023-08-09T13:47:44.137 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-09T13:47:44.138 DEBUG:teuthology.orchestra.run.smithi043:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/dir5/.snap/scheduled-2023-08-09-13_38_00_UTC) 2023-08-09T13:47:44.194 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/dir5/.snap/scheduled-2023-08-09-13_39_00_UTC 2023-08-09T13:47:44.194 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-09T13:47:44.195 DEBUG:teuthology.orchestra.run.smithi043:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/dir5/.snap/scheduled-2023-08-09-13_39_00_UTC) 2023-08-09T13:47:44.256 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/dir5/.snap/scheduled-2023-08-09-13_40_00_UTC 2023-08-09T13:47:44.256 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-09T13:47:44.256 DEBUG:teuthology.orchestra.run.smithi043:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/dir5/.snap/scheduled-2023-08-09-13_40_00_UTC) 2023-08-09T13:47:44.324 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/dir5/.snap/scheduled-2023-08-09-13_41_00_UTC 2023-08-09T13:47:44.325 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-09T13:47:44.325 DEBUG:teuthology.orchestra.run.smithi043:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/dir5/.snap/scheduled-2023-08-09-13_41_00_UTC) 2023-08-09T13:47:44.432 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/dir5/.snap/scheduled-2023-08-09-13_42_00_UTC 2023-08-09T13:47:44.433 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-09T13:47:44.433 DEBUG:teuthology.orchestra.run.smithi043:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/dir5/.snap/scheduled-2023-08-09-13_42_00_UTC) 2023-08-09T13:47:44.484 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/dir5/.snap/scheduled-2023-08-09-13_43_00_UTC 2023-08-09T13:47:44.484 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-09T13:47:44.485 DEBUG:teuthology.orchestra.run.smithi043:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/dir5/.snap/scheduled-2023-08-09-13_43_00_UTC) 2023-08-09T13:47:44.544 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/dir5/.snap/scheduled-2023-08-09-13_44_00_UTC 2023-08-09T13:47:44.545 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-09T13:47:44.546 DEBUG:teuthology.orchestra.run.smithi043:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/dir5/.snap/scheduled-2023-08-09-13_44_00_UTC) 2023-08-09T13:47:44.620 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/dir5/.snap/scheduled-2023-08-09-13_45_01_UTC 2023-08-09T13:47:44.621 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-09T13:47:44.622 DEBUG:teuthology.orchestra.run.smithi043:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/dir5/.snap/scheduled-2023-08-09-13_45_01_UTC) 2023-08-09T13:47:44.695 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/dir5/.snap/scheduled-2023-08-09-13_46_01_UTC 2023-08-09T13:47:44.695 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-09T13:47:44.695 DEBUG:teuthology.orchestra.run.smithi043:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/dir5/.snap/scheduled-2023-08-09-13_46_01_UTC) 2023-08-09T13:47:44.777 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/dir5/.snap/scheduled-2023-08-09-13_47_00_UTC 2023-08-09T13:47:44.777 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-09T13:47:44.778 DEBUG:teuthology.orchestra.run.smithi043:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/dir5/.snap/scheduled-2023-08-09-13_47_00_UTC) 2023-08-09T13:47:44.860 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-09T13:47:44.861 DEBUG:teuthology.orchestra.run.smithi043:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/dir5) 2023-08-09T13:47:44.927 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs snap-schedule remove --fs cephfs /snap_test_dir1/dir6 1M 2023-08-09T13:47:45.169 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:45.168+0000 7f41d8c47640 1 Processor -- start 2023-08-09T13:47:45.169 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:45.168+0000 7f41d8c47640 1 -- start start 2023-08-09T13:47:45.171 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:45.169+0000 7f41d8c47640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f41d40cf370 0x7f41d4167220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:47:45.171 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:45.169+0000 7f41d8c47640 1 -- --> v1:172.21.15.43:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f41d40cdce0 con 0x7f41d40cd850 2023-08-09T13:47:45.171 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:45.169+0000 7f41d8c47640 1 -- --> v1:172.21.15.203:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f41d40b5660 con 0x7f41d415c720 2023-08-09T13:47:45.171 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:45.169+0000 7f41d8c47640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f41d415cb00 con 0x7f41d40cf370 2023-08-09T13:47:45.172 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:45.170+0000 7f41d2ffd640 1 --1- >> v1:172.21.15.43:6789/0 conn(0x7f41d40cd850 0x7f41d40cec60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.43:6789/0 says I am v1:172.21.15.43:39020/0 (socket says 172.21.15.43:39020) 2023-08-09T13:47:45.172 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:45.170+0000 7f41d2ffd640 1 -- 172.21.15.43:0/74272859 learned_addr learned my addr 172.21.15.43:0/74272859 (peer_addr_for_me v1:172.21.15.43:0/0) 2023-08-09T13:47:45.172 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:45.170+0000 7f41d3fff640 1 --2- 172.21.15.43:0/74272859 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f41d40cf370 0x7f41d4167220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:47:45.172 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:45.170+0000 7f41d27fc640 1 -- 172.21.15.43:0/74272859 <== mon.0 v1:172.21.15.43:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4061277295 0 0) 0x7f41d40cdce0 con 0x7f41d40cd850 2023-08-09T13:47:45.172 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:45.171+0000 7f41d27fc640 1 -- 172.21.15.43:0/74272859 --> v1:172.21.15.43:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f41bc003520 con 0x7f41d40cd850 2023-08-09T13:47:45.173 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:45.171+0000 7f41d27fc640 1 -- 172.21.15.43:0/74272859 <== mon.1 v1:172.21.15.203:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2838235933 0 0) 0x7f41d40b5660 con 0x7f41d415c720 2023-08-09T13:47:45.173 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:45.171+0000 7f41d3fff640 1 -- 172.21.15.43:0/74272859 >> v1:172.21.15.203:6789/0 conn(0x7f41d415c720 legacy=0x7f41d40d0900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:47:45.173 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:45.171+0000 7f41d3fff640 1 -- 172.21.15.43:0/74272859 >> v1:172.21.15.43:6789/0 conn(0x7f41d40cd850 legacy=0x7f41d40cec60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:47:45.173 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:45.171+0000 7f41d3fff640 1 -- 172.21.15.43:0/74272859 --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f41d40b5900 con 0x7f41d40cf370 2023-08-09T13:47:45.174 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:45.173+0000 7f41d3fff640 1 --2- 172.21.15.43:0/74272859 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f41d40cf370 0x7f41d4167220 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7f41cc009dc0 tx=0x7f41cc02bf10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=6cbd90c96fe4ae88 server_cookie=0 in_seq=0 out_seq=0 2023-08-09T13:47:45.175 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:45.173+0000 7f41d27fc640 1 -- 172.21.15.43:0/74272859 <== mon.2 v2:172.21.15.203:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f41cc02ccc0 con 0x7f41d40cf370 2023-08-09T13:47:45.175 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:45.174+0000 7f41d27fc640 1 -- 172.21.15.43:0/74272859 <== mon.2 v2:172.21.15.203:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f41cc02ce20 con 0x7f41d40cf370 2023-08-09T13:47:45.175 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:45.174+0000 7f41d27fc640 1 -- 172.21.15.43:0/74272859 <== mon.2 v2:172.21.15.203:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f41cc034660 con 0x7f41d40cf370 2023-08-09T13:47:45.176 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:45.174+0000 7f41d8c47640 1 -- 172.21.15.43:0/74272859 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f41d40cf370 msgr2=0x7f41d4167220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:47:45.176 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:45.174+0000 7f41d8c47640 1 --2- 172.21.15.43:0/74272859 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f41d40cf370 0x7f41d4167220 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7f41cc009dc0 tx=0x7f41cc02bf10 comp rx=0 tx=0).stop 2023-08-09T13:47:45.176 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:45.174+0000 7f41d8c47640 1 -- 172.21.15.43:0/74272859 shutdown_connections 2023-08-09T13:47:45.176 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:45.174+0000 7f41d8c47640 1 --2- 172.21.15.43:0/74272859 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f41d40cf370 0x7f41d4167220 unknown :-1 s=CLOSED pgs=94 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:47:45.176 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:45.174+0000 7f41d8c47640 1 -- 172.21.15.43:0/74272859 >> 172.21.15.43:0/74272859 conn(0x7f41d406a2a0 msgr2=0x7f41d406a680 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-09T13:47:45.177 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:45.174+0000 7f41d8c47640 1 -- 172.21.15.43:0/74272859 shutdown_connections 2023-08-09T13:47:45.177 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:45.174+0000 7f41d8c47640 1 -- 172.21.15.43:0/74272859 wait complete. 2023-08-09T13:47:45.177 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:45.176+0000 7f41d8c47640 1 Processor -- start 2023-08-09T13:47:45.178 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:45.176+0000 7f41d8c47640 1 -- start start 2023-08-09T13:47:45.179 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:45.177+0000 7f41d8c47640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f41d40cd850 0x7f41d415e230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:47:45.179 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:45.177+0000 7f41d8c47640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f41d40cf370 0x7f41d415ea90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:47:45.179 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:45.177+0000 7f41d8c47640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f41d415c720 0x7f41d40c6c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:47:45.179 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:45.178+0000 7f41d8c47640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f41d4156fe0 con 0x7f41d415c720 2023-08-09T13:47:45.180 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:45.178+0000 7f41d8c47640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f41d40c5e90 con 0x7f41d40cf370 2023-08-09T13:47:45.180 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:45.178+0000 7f41d8c47640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f41d40c6170 con 0x7f41d40cd850 2023-08-09T13:47:45.180 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:45.178+0000 7f41d3fff640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f41d415c720 0x7f41d40c6c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:47:45.180 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:45.178+0000 7f41d3fff640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f41d415c720 0x7f41d40c6c50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:55764/0 (socket says 172.21.15.43:55764) 2023-08-09T13:47:45.180 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:45.178+0000 7f41d3fff640 1 -- 172.21.15.43:0/3864259282 learned_addr learned my addr 172.21.15.43:0/3864259282 (peer_addr_for_me v2:172.21.15.43:0/0) 2023-08-09T13:47:45.181 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:45.178+0000 7f41d37fe640 1 --2- 172.21.15.43:0/3864259282 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f41d40cd850 0x7f41d415e230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:47:45.181 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:45.178+0000 7f41d2ffd640 1 --2- 172.21.15.43:0/3864259282 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f41d40cf370 0x7f41d415ea90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:47:45.182 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:45.178+0000 7f41d3fff640 1 -- 172.21.15.43:0/3864259282 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f41d40cd850 msgr2=0x7f41d415e230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:47:45.182 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:45.178+0000 7f41d3fff640 1 --2- 172.21.15.43:0/3864259282 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f41d40cd850 0x7f41d415e230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:47:45.182 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:45.178+0000 7f41d3fff640 1 -- 172.21.15.43:0/3864259282 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f41d40cf370 msgr2=0x7f41d415ea90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:47:45.182 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:45.178+0000 7f41d3fff640 1 --2- 172.21.15.43:0/3864259282 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f41d40cf370 0x7f41d415ea90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:47:45.182 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:45.178+0000 7f41d3fff640 1 -- 172.21.15.43:0/3864259282 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f41cc009a70 con 0x7f41d415c720 2023-08-09T13:47:45.182 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:45.178+0000 7f41d2ffd640 1 --2- 172.21.15.43:0/3864259282 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f41d40cf370 0x7f41d415ea90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-08-09T13:47:45.182 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:45.179+0000 7f41d3fff640 1 --2- 172.21.15.43:0/3864259282 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f41d415c720 0x7f41d40c6c50 secure :-1 s=READY pgs=722 cs=0 l=1 rev1=1 crypto rx=0x7f41cc009ee0 tx=0x7f41cc004ab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-09T13:47:45.182 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:45.179+0000 7f41bbfff640 1 -- 172.21.15.43:0/3864259282 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f41cc040650 con 0x7f41d415c720 2023-08-09T13:47:45.183 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:45.179+0000 7f41bbfff640 1 -- 172.21.15.43:0/3864259282 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f41cc0407b0 con 0x7f41d415c720 2023-08-09T13:47:45.183 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:45.179+0000 7f41bbfff640 1 -- 172.21.15.43:0/3864259282 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f41cc040b60 con 0x7f41d415c720 2023-08-09T13:47:45.183 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:45.179+0000 7f41d8c47640 1 -- 172.21.15.43:0/3864259282 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f41d40c6450 con 0x7f41d415c720 2023-08-09T13:47:45.183 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:45.179+0000 7f41d8c47640 1 -- 172.21.15.43:0/3864259282 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f41d41f2360 con 0x7f41d415c720 2023-08-09T13:47:45.184 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:45.180+0000 7f41d8c47640 1 -- 172.21.15.43:0/3864259282 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f41d40739f0 con 0x7f41d415c720 2023-08-09T13:47:45.184 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:45.181+0000 7f41bbfff640 1 -- 172.21.15.43:0/3864259282 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 13) v1 ==== 88529+0+0 (secure 0 0 0) 0x7f41cc040cc0 con 0x7f41d415c720 2023-08-09T13:47:45.184 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:45.183+0000 7f41bbfff640 1 --2- 172.21.15.43:0/3864259282 >> [v2:172.21.15.203:6832/998005540,v1:172.21.15.203:6833/998005540] conn(0x7f41c006a250 0x7f41c006c710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:47:45.184 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:45.183+0000 7f41bbfff640 1 -- 172.21.15.43:0/3864259282 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(52..52 src has 1..52) v4 ==== 7362+0+0 (secure 0 0 0) 0x7f41cc0abb90 con 0x7f41d415c720 2023-08-09T13:47:45.185 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:45.184+0000 7f41d37fe640 1 --2- 172.21.15.43:0/3864259282 >> [v2:172.21.15.203:6832/998005540,v1:172.21.15.203:6833/998005540] conn(0x7f41c006a250 0x7f41c006c710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:47:45.186 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:45.185+0000 7f41d37fe640 1 --2- 172.21.15.43:0/3864259282 >> [v2:172.21.15.203:6832/998005540,v1:172.21.15.203:6833/998005540] conn(0x7f41c006a250 0x7f41c006c710 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f41c4000f60 tx=0x7f41c4009040 comp rx=0 tx=0).ready entity=mgr.5003 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-09T13:47:45.188 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:45.187+0000 7f41bbfff640 1 -- 172.21.15.43:0/3864259282 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178066 (secure 0 0 0) 0x7f41cc076a40 con 0x7f41d415c720 2023-08-09T13:47:45.434 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:45.433+0000 7f41d8c47640 1 -- 172.21.15.43:0/3864259282 --> [v2:172.21.15.203:6832/998005540,v1:172.21.15.203:6833/998005540] -- mgr_command(tid 0: {"prefix": "fs snap-schedule remove", "fs": "cephfs", "path": "/snap_test_dir1/dir6", "repeat": "1M", "target": ["mon-mgr", ""]}) v1 -- 0x7f41d40c6000 con 0x7f41c006a250 2023-08-09T13:47:45.637 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:45.636+0000 7f41bbfff640 1 -- 172.21.15.43:0/3864259282 <== mgr.5003 v2:172.21.15.203:6832/998005540 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+47 (secure 0 0 0) 0x7f41d40c6000 con 0x7f41c006a250 2023-08-09T13:47:45.638 INFO:teuthology.orchestra.run.smithi043.stdout:Schedule removed for path /snap_test_dir1/dir6 2023-08-09T13:47:45.640 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:45.639+0000 7f41d8c47640 1 -- 172.21.15.43:0/3864259282 >> [v2:172.21.15.203:6832/998005540,v1:172.21.15.203:6833/998005540] conn(0x7f41c006a250 msgr2=0x7f41c006c710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:47:45.641 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:45.639+0000 7f41d8c47640 1 --2- 172.21.15.43:0/3864259282 >> [v2:172.21.15.203:6832/998005540,v1:172.21.15.203:6833/998005540] conn(0x7f41c006a250 0x7f41c006c710 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f41c4000f60 tx=0x7f41c4009040 comp rx=0 tx=0).stop 2023-08-09T13:47:45.641 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:45.639+0000 7f41d8c47640 1 -- 172.21.15.43:0/3864259282 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f41d415c720 msgr2=0x7f41d40c6c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:47:45.641 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:45.639+0000 7f41d8c47640 1 --2- 172.21.15.43:0/3864259282 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f41d415c720 0x7f41d40c6c50 secure :-1 s=READY pgs=722 cs=0 l=1 rev1=1 crypto rx=0x7f41cc009ee0 tx=0x7f41cc004ab0 comp rx=0 tx=0).stop 2023-08-09T13:47:45.641 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:45.639+0000 7f41d8c47640 1 -- 172.21.15.43:0/3864259282 shutdown_connections 2023-08-09T13:47:45.642 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:45.639+0000 7f41d8c47640 1 --2- 172.21.15.43:0/3864259282 >> [v2:172.21.15.203:6832/998005540,v1:172.21.15.203:6833/998005540] conn(0x7f41c006a250 0x7f41c006c710 unknown :-1 s=CLOSED pgs=69 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:47:45.642 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:45.639+0000 7f41d8c47640 1 --2- 172.21.15.43:0/3864259282 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f41d415c720 0x7f41d40c6c50 unknown :-1 s=CLOSED pgs=722 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:47:45.642 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:45.639+0000 7f41d8c47640 1 --2- 172.21.15.43:0/3864259282 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f41d40cf370 0x7f41d415ea90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:47:45.642 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:45.639+0000 7f41d8c47640 1 --2- 172.21.15.43:0/3864259282 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f41d40cd850 0x7f41d415e230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:47:45.642 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:45.639+0000 7f41d8c47640 1 -- 172.21.15.43:0/3864259282 >> 172.21.15.43:0/3864259282 conn(0x7f41d406a2a0 msgr2=0x7f41d415b9c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-09T13:47:45.642 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:45.640+0000 7f41d8c47640 1 -- 172.21.15.43:0/3864259282 shutdown_connections 2023-08-09T13:47:45.643 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:45.640+0000 7f41d8c47640 1 -- 172.21.15.43:0/3864259282 wait complete. 2023-08-09T13:47:45.662 DEBUG:tasks.cephfs.test_snap_schedules:res=Schedule removed for path /snap_test_dir1/dir6 2023-08-09T13:47:45.662 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-09T13:47:45.663 DEBUG:teuthology.orchestra.run.smithi043:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/dir6/.snap) 2023-08-09T13:47:45.745 INFO:teuthology.orchestra.run.smithi043.stdout:scheduled-2023-08-09-13_37_00_UTC 2023-08-09T13:47:45.745 INFO:teuthology.orchestra.run.smithi043.stdout:scheduled-2023-08-09-13_38_00_UTC 2023-08-09T13:47:45.746 INFO:teuthology.orchestra.run.smithi043.stdout:scheduled-2023-08-09-13_39_00_UTC 2023-08-09T13:47:45.746 INFO:teuthology.orchestra.run.smithi043.stdout:scheduled-2023-08-09-13_40_00_UTC 2023-08-09T13:47:45.746 INFO:teuthology.orchestra.run.smithi043.stdout:scheduled-2023-08-09-13_41_01_UTC 2023-08-09T13:47:45.746 INFO:teuthology.orchestra.run.smithi043.stdout:scheduled-2023-08-09-13_42_01_UTC 2023-08-09T13:47:45.746 INFO:teuthology.orchestra.run.smithi043.stdout:scheduled-2023-08-09-13_43_01_UTC 2023-08-09T13:47:45.746 INFO:teuthology.orchestra.run.smithi043.stdout:scheduled-2023-08-09-13_44_00_UTC 2023-08-09T13:47:45.746 INFO:teuthology.orchestra.run.smithi043.stdout:scheduled-2023-08-09-13_45_00_UTC 2023-08-09T13:47:45.747 INFO:teuthology.orchestra.run.smithi043.stdout:scheduled-2023-08-09-13_46_00_UTC 2023-08-09T13:47:45.747 INFO:teuthology.orchestra.run.smithi043.stdout:scheduled-2023-08-09-13_47_00_UTC 2023-08-09T13:47:45.748 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/dir6/.snap/scheduled-2023-08-09-13_37_00_UTC 2023-08-09T13:47:45.748 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-09T13:47:45.748 DEBUG:teuthology.orchestra.run.smithi043:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/dir6/.snap/scheduled-2023-08-09-13_37_00_UTC) 2023-08-09T13:47:45.836 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/dir6/.snap/scheduled-2023-08-09-13_38_00_UTC 2023-08-09T13:47:45.837 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-09T13:47:45.837 DEBUG:teuthology.orchestra.run.smithi043:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/dir6/.snap/scheduled-2023-08-09-13_38_00_UTC) 2023-08-09T13:47:45.916 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/dir6/.snap/scheduled-2023-08-09-13_39_00_UTC 2023-08-09T13:47:45.917 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-09T13:47:45.917 DEBUG:teuthology.orchestra.run.smithi043:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/dir6/.snap/scheduled-2023-08-09-13_39_00_UTC) 2023-08-09T13:47:46.001 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/dir6/.snap/scheduled-2023-08-09-13_40_00_UTC 2023-08-09T13:47:46.002 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-09T13:47:46.002 DEBUG:teuthology.orchestra.run.smithi043:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/dir6/.snap/scheduled-2023-08-09-13_40_00_UTC) 2023-08-09T13:47:46.097 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/dir6/.snap/scheduled-2023-08-09-13_41_01_UTC 2023-08-09T13:47:46.097 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-09T13:47:46.098 DEBUG:teuthology.orchestra.run.smithi043:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/dir6/.snap/scheduled-2023-08-09-13_41_01_UTC) 2023-08-09T13:47:46.161 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/dir6/.snap/scheduled-2023-08-09-13_42_01_UTC 2023-08-09T13:47:46.162 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-09T13:47:46.162 DEBUG:teuthology.orchestra.run.smithi043:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/dir6/.snap/scheduled-2023-08-09-13_42_01_UTC) 2023-08-09T13:47:46.244 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/dir6/.snap/scheduled-2023-08-09-13_43_01_UTC 2023-08-09T13:47:46.245 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-09T13:47:46.245 DEBUG:teuthology.orchestra.run.smithi043:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/dir6/.snap/scheduled-2023-08-09-13_43_01_UTC) 2023-08-09T13:47:46.301 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/dir6/.snap/scheduled-2023-08-09-13_44_00_UTC 2023-08-09T13:47:46.302 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-09T13:47:46.302 DEBUG:teuthology.orchestra.run.smithi043:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/dir6/.snap/scheduled-2023-08-09-13_44_00_UTC) 2023-08-09T13:47:46.366 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/dir6/.snap/scheduled-2023-08-09-13_45_00_UTC 2023-08-09T13:47:46.366 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-09T13:47:46.367 DEBUG:teuthology.orchestra.run.smithi043:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/dir6/.snap/scheduled-2023-08-09-13_45_00_UTC) 2023-08-09T13:47:46.435 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/dir6/.snap/scheduled-2023-08-09-13_46_00_UTC 2023-08-09T13:47:46.435 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-09T13:47:46.435 DEBUG:teuthology.orchestra.run.smithi043:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/dir6/.snap/scheduled-2023-08-09-13_46_00_UTC) 2023-08-09T13:47:46.513 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/dir6/.snap/scheduled-2023-08-09-13_47_00_UTC 2023-08-09T13:47:46.514 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-09T13:47:46.514 DEBUG:teuthology.orchestra.run.smithi043:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/dir6/.snap/scheduled-2023-08-09-13_47_00_UTC) 2023-08-09T13:47:46.604 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-09T13:47:46.605 DEBUG:teuthology.orchestra.run.smithi043:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/dir6) 2023-08-09T13:47:46.652 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs snap-schedule remove --fs cephfs /snap_test_dir1/dir7 1M 2023-08-09T13:47:46.920 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:46.918+0000 7fb5618d9640 1 Processor -- start 2023-08-09T13:47:46.920 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:46.919+0000 7fb5618d9640 1 -- start start 2023-08-09T13:47:46.922 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:46.920+0000 7fb5618d9640 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7fb55c0cccd0 0x7fb55c0cd180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:47:46.922 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:46.920+0000 7fb5618d9640 1 -- --> v1:172.21.15.43:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb55c0cbde0 con 0x7fb55c15a560 2023-08-09T13:47:46.922 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:46.920+0000 7fb5618d9640 1 -- --> v1:172.21.15.203:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb55c0cb1d0 con 0x7fb55c0b0f30 2023-08-09T13:47:46.922 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:46.920+0000 7fb5618d9640 1 -- --> v2:172.21.15.203:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb55c0cd6c0 con 0x7fb55c0cccd0 2023-08-09T13:47:46.923 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:46.921+0000 7fb55affd640 1 --1- >> v1:172.21.15.43:6789/0 conn(0x7fb55c15a560 0x7fb55c0d09d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.43:6789/0 says I am v1:172.21.15.43:39032/0 (socket says 172.21.15.43:39032) 2023-08-09T13:47:46.923 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:46.921+0000 7fb55affd640 1 -- 172.21.15.43:0/1064581092 learned_addr learned my addr 172.21.15.43:0/1064581092 (peer_addr_for_me v1:172.21.15.43:0/0) 2023-08-09T13:47:46.923 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:46.921+0000 7fb55a7fc640 1 --2- 172.21.15.43:0/1064581092 >> v2:172.21.15.203:3300/0 conn(0x7fb55c0cccd0 0x7fb55c0cd180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:47:46.923 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:46.921+0000 7fb559ffb640 1 -- 172.21.15.43:0/1064581092 <== mon.0 v1:172.21.15.43:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4237918666 0 0) 0x7fb55c0cbde0 con 0x7fb55c15a560 2023-08-09T13:47:46.923 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:46.922+0000 7fb559ffb640 1 -- 172.21.15.43:0/1064581092 --> v1:172.21.15.43:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fb544003520 con 0x7fb55c15a560 2023-08-09T13:47:46.924 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:46.922+0000 7fb559ffb640 1 -- 172.21.15.43:0/1064581092 <== mon.0 v1:172.21.15.43:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3781819228 0 0) 0x7fb544003520 con 0x7fb55c15a560 2023-08-09T13:47:46.925 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:46.922+0000 7fb559ffb640 1 -- 172.21.15.43:0/1064581092 >> v2:172.21.15.203:3300/0 conn(0x7fb55c0cccd0 msgr2=0x7fb55c0cd180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-09T13:47:46.925 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:46.922+0000 7fb559ffb640 1 --2- 172.21.15.43:0/1064581092 >> v2:172.21.15.203:3300/0 conn(0x7fb55c0cccd0 0x7fb55c0cd180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:47:46.925 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:46.922+0000 7fb559ffb640 1 -- 172.21.15.43:0/1064581092 >> v1:172.21.15.203:6789/0 conn(0x7fb55c0b0f30 legacy=0x7fb55c0b1380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:47:46.925 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:46.922+0000 7fb559ffb640 1 -- 172.21.15.43:0/1064581092 --> v1:172.21.15.43:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb55c0cb4b0 con 0x7fb55c15a560 2023-08-09T13:47:46.925 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:46.922+0000 7fb559ffb640 1 -- 172.21.15.43:0/1064581092 <== mon.0 v1:172.21.15.43:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1285150740 0 0) 0x7fb54c001b70 con 0x7fb55c15a560 2023-08-09T13:47:46.925 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:46.922+0000 7fb559ffb640 1 -- 172.21.15.43:0/1064581092 >> v1:172.21.15.43:6789/0 conn(0x7fb55c15a560 legacy=0x7fb55c0d09d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:47:46.926 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:46.922+0000 7fb559ffb640 1 --2- 172.21.15.43:0/1064581092 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fb5440039e0 0x7fb544003e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:47:46.926 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:46.922+0000 7fb559ffb640 1 --2- 172.21.15.43:0/1064581092 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb5440047c0 0x7fb544004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:47:46.926 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:46.922+0000 7fb559ffb640 1 --2- 172.21.15.43:0/1064581092 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fb544005180 0x7fb54400b980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:47:46.926 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:46.922+0000 7fb559ffb640 1 -- 172.21.15.43:0/1064581092 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb55c0cd6c0 con 0x7fb5440047c0 2023-08-09T13:47:46.926 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:46.922+0000 7fb559ffb640 1 -- 172.21.15.43:0/1064581092 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb54400c0b0 con 0x7fb544005180 2023-08-09T13:47:46.926 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:46.922+0000 7fb559ffb640 1 -- 172.21.15.43:0/1064581092 --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb54400c270 con 0x7fb5440039e0 2023-08-09T13:47:46.926 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:46.922+0000 7fb55a7fc640 1 --2- 172.21.15.43:0/1064581092 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb5440047c0 0x7fb544004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:47:46.926 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:46.923+0000 7fb55affd640 1 --2- 172.21.15.43:0/1064581092 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fb5440039e0 0x7fb544003e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:47:46.927 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:46.923+0000 7fb55a7fc640 1 -- 172.21.15.43:0/1064581092 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fb5440039e0 msgr2=0x7fb544003e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-09T13:47:46.927 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:46.923+0000 7fb55a7fc640 1 --2- 172.21.15.43:0/1064581092 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fb5440039e0 0x7fb544003e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:47:46.927 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:46.923+0000 7fb55a7fc640 1 -- 172.21.15.43:0/1064581092 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fb544005180 msgr2=0x7fb54400b980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-09T13:47:46.927 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:46.923+0000 7fb55a7fc640 1 --2- 172.21.15.43:0/1064581092 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fb544005180 0x7fb54400b980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:47:46.927 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:46.923+0000 7fb55a7fc640 1 -- 172.21.15.43:0/1064581092 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb544003520 con 0x7fb5440047c0 2023-08-09T13:47:46.927 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:46.923+0000 7fb55b7fe640 1 -- 172.21.15.43:0/1064581092 reap_dead start 2023-08-09T13:47:46.928 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:46.924+0000 7fb55a7fc640 1 --2- 172.21.15.43:0/1064581092 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb5440047c0 0x7fb544004c40 secure :-1 s=READY pgs=724 cs=0 l=1 rev1=1 crypto rx=0x7fb550009ac0 tx=0x7fb5440057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cab1f68fa8a01462 server_cookie=0 in_seq=0 out_seq=0 2023-08-09T13:47:46.928 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:46.924+0000 7fb559ffb640 1 -- 172.21.15.43:0/1064581092 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb55002c710 con 0x7fb5440047c0 2023-08-09T13:47:46.928 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:46.925+0000 7fb559ffb640 1 -- 172.21.15.43:0/1064581092 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb55002c870 con 0x7fb5440047c0 2023-08-09T13:47:46.929 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:46.925+0000 7fb559ffb640 1 -- 172.21.15.43:0/1064581092 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb55002cc00 con 0x7fb5440047c0 2023-08-09T13:47:46.929 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:46.925+0000 7fb5618d9640 1 -- 172.21.15.43:0/1064581092 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb5440047c0 msgr2=0x7fb544004c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:47:46.929 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:46.925+0000 7fb5618d9640 1 --2- 172.21.15.43:0/1064581092 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb5440047c0 0x7fb544004c40 secure :-1 s=READY pgs=724 cs=0 l=1 rev1=1 crypto rx=0x7fb550009ac0 tx=0x7fb5440057f0 comp rx=0 tx=0).stop 2023-08-09T13:47:46.929 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:46.925+0000 7fb5618d9640 1 -- 172.21.15.43:0/1064581092 shutdown_connections 2023-08-09T13:47:46.929 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:46.925+0000 7fb5618d9640 1 --2- 172.21.15.43:0/1064581092 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb5440047c0 0x7fb544004c40 unknown :-1 s=CLOSED pgs=724 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:47:46.929 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:46.925+0000 7fb5618d9640 1 -- 172.21.15.43:0/1064581092 >> 172.21.15.43:0/1064581092 conn(0x7fb55c099600 msgr2=0x7fb55c0d02c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-09T13:47:46.929 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:46.925+0000 7fb5618d9640 1 -- 172.21.15.43:0/1064581092 shutdown_connections 2023-08-09T13:47:46.929 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:46.925+0000 7fb5618d9640 1 -- 172.21.15.43:0/1064581092 wait complete. 2023-08-09T13:47:46.931 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:46.927+0000 7fb5618d9640 1 Processor -- start 2023-08-09T13:47:46.931 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:46.927+0000 7fb5618d9640 1 -- start start 2023-08-09T13:47:46.931 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:46.928+0000 7fb5618d9640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fb5440047c0 0x7fb55c0c3d70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:47:46.931 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:46.928+0000 7fb5618d9640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fb55c15d1c0 0x7fb55c0c5710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:47:46.931 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:46.928+0000 7fb5618d9640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb55c0c5c50 0x7fb55c0c6100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:47:46.931 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:46.928+0000 7fb5618d9640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb55c0c32e0 con 0x7fb55c0c5c50 2023-08-09T13:47:46.932 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:46.928+0000 7fb5618d9640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb55c06b410 con 0x7fb55c15d1c0 2023-08-09T13:47:46.932 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:46.928+0000 7fb5618d9640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb55c15d9a0 con 0x7fb5440047c0 2023-08-09T13:47:46.932 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:46.929+0000 7fb55b7fe640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb55c0c5c50 0x7fb55c0c6100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:47:46.932 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:46.929+0000 7fb55b7fe640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb55c0c5c50 0x7fb55c0c6100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:55774/0 (socket says 172.21.15.43:55774) 2023-08-09T13:47:46.932 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:46.929+0000 7fb55b7fe640 1 -- 172.21.15.43:0/2578120957 learned_addr learned my addr 172.21.15.43:0/2578120957 (peer_addr_for_me v2:172.21.15.43:0/0) 2023-08-09T13:47:46.932 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:46.929+0000 7fb55a7fc640 1 --2- 172.21.15.43:0/2578120957 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fb55c15d1c0 0x7fb55c0c5710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:47:46.932 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:46.929+0000 7fb55affd640 1 --2- 172.21.15.43:0/2578120957 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fb5440047c0 0x7fb55c0c3d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:47:46.933 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:46.929+0000 7fb55b7fe640 1 -- 172.21.15.43:0/2578120957 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fb5440047c0 msgr2=0x7fb55c0c3d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:47:46.933 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:46.929+0000 7fb55b7fe640 1 --2- 172.21.15.43:0/2578120957 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fb5440047c0 0x7fb55c0c3d70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:47:46.933 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:46.929+0000 7fb55b7fe640 1 -- 172.21.15.43:0/2578120957 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fb55c15d1c0 msgr2=0x7fb55c0c5710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:47:46.933 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:46.929+0000 7fb55b7fe640 1 --2- 172.21.15.43:0/2578120957 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fb55c15d1c0 0x7fb55c0c5710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:47:46.933 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:46.929+0000 7fb55b7fe640 1 -- 172.21.15.43:0/2578120957 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb55c06c190 con 0x7fb55c0c5c50 2023-08-09T13:47:46.933 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:46.930+0000 7fb55b7fe640 1 --2- 172.21.15.43:0/2578120957 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb55c0c5c50 0x7fb55c0c6100 secure :-1 s=READY pgs=725 cs=0 l=1 rev1=1 crypto rx=0x7fb55c071ed0 tx=0x7fb55400c820 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-09T13:47:46.934 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:46.930+0000 7fb543fff640 1 -- 172.21.15.43:0/2578120957 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb5540072d0 con 0x7fb55c0c5c50 2023-08-09T13:47:46.934 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:46.930+0000 7fb543fff640 1 -- 172.21.15.43:0/2578120957 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb554007450 con 0x7fb55c0c5c50 2023-08-09T13:47:46.934 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:46.930+0000 7fb543fff640 1 -- 172.21.15.43:0/2578120957 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb5540057c0 con 0x7fb55c0c5c50 2023-08-09T13:47:46.934 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:46.930+0000 7fb5618d9640 1 -- 172.21.15.43:0/2578120957 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb55c06b6b0 con 0x7fb55c0c5c50 2023-08-09T13:47:46.935 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:46.930+0000 7fb5618d9640 1 -- 172.21.15.43:0/2578120957 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb55c06bc70 con 0x7fb55c0c5c50 2023-08-09T13:47:46.935 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:46.931+0000 7fb5618d9640 1 -- 172.21.15.43:0/2578120957 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb55c0cad20 con 0x7fb55c0c5c50 2023-08-09T13:47:46.935 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:46.932+0000 7fb543fff640 1 -- 172.21.15.43:0/2578120957 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 13) v1 ==== 88529+0+0 (secure 0 0 0) 0x7fb554005920 con 0x7fb55c0c5c50 2023-08-09T13:47:46.935 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:46.934+0000 7fb543fff640 1 --2- 172.21.15.43:0/2578120957 >> [v2:172.21.15.203:6832/998005540,v1:172.21.15.203:6833/998005540] conn(0x7fb54806a280 0x7fb54806c740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:47:46.936 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:46.934+0000 7fb543fff640 1 -- 172.21.15.43:0/2578120957 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(53..53 src has 1..53) v4 ==== 7442+0+0 (secure 0 0 0) 0x7fb554087fd0 con 0x7fb55c0c5c50 2023-08-09T13:47:46.936 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:46.934+0000 7fb55affd640 1 --2- 172.21.15.43:0/2578120957 >> [v2:172.21.15.203:6832/998005540,v1:172.21.15.203:6833/998005540] conn(0x7fb54806a280 0x7fb54806c740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:47:46.937 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:46.935+0000 7fb55affd640 1 --2- 172.21.15.43:0/2578120957 >> [v2:172.21.15.203:6832/998005540,v1:172.21.15.203:6833/998005540] conn(0x7fb54806a280 0x7fb54806c740 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7fb55c06c3e0 tx=0x7fb54c0012f0 comp rx=0 tx=0).ready entity=mgr.5003 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-09T13:47:46.940 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:46.938+0000 7fb543fff640 1 -- 172.21.15.43:0/2578120957 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178066 (secure 0 0 0) 0x7fb554052f70 con 0x7fb55c0c5c50 2023-08-09T13:47:47.171 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:47.170+0000 7fb5618d9640 1 -- 172.21.15.43:0/2578120957 --> [v2:172.21.15.203:6832/998005540,v1:172.21.15.203:6833/998005540] -- mgr_command(tid 0: {"prefix": "fs snap-schedule remove", "fs": "cephfs", "path": "/snap_test_dir1/dir7", "repeat": "1M", "target": ["mon-mgr", ""]}) v1 -- 0x7fb55c0cba70 con 0x7fb54806a280 2023-08-09T13:47:47.383 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:47.381+0000 7fb543fff640 1 -- 172.21.15.43:0/2578120957 <== mgr.5003 v2:172.21.15.203:6832/998005540 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+47 (secure 0 0 0) 0x7fb54c005e20 con 0x7fb54806a280 2023-08-09T13:47:47.383 INFO:teuthology.orchestra.run.smithi043.stdout:Schedule removed for path /snap_test_dir1/dir7 2023-08-09T13:47:47.386 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:47.385+0000 7fb5618d9640 1 -- 172.21.15.43:0/2578120957 >> [v2:172.21.15.203:6832/998005540,v1:172.21.15.203:6833/998005540] conn(0x7fb54806a280 msgr2=0x7fb54806c740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:47:47.386 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:47.385+0000 7fb5618d9640 1 --2- 172.21.15.43:0/2578120957 >> [v2:172.21.15.203:6832/998005540,v1:172.21.15.203:6833/998005540] conn(0x7fb54806a280 0x7fb54806c740 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7fb55c06c3e0 tx=0x7fb54c0012f0 comp rx=0 tx=0).stop 2023-08-09T13:47:47.386 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:47.385+0000 7fb5618d9640 1 -- 172.21.15.43:0/2578120957 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb55c0c5c50 msgr2=0x7fb55c0c6100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:47:47.386 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:47.385+0000 7fb5618d9640 1 --2- 172.21.15.43:0/2578120957 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb55c0c5c50 0x7fb55c0c6100 secure :-1 s=READY pgs=725 cs=0 l=1 rev1=1 crypto rx=0x7fb55c071ed0 tx=0x7fb55400c820 comp rx=0 tx=0).stop 2023-08-09T13:47:47.387 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:47.385+0000 7fb5618d9640 1 -- 172.21.15.43:0/2578120957 shutdown_connections 2023-08-09T13:47:47.387 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:47.385+0000 7fb5618d9640 1 --2- 172.21.15.43:0/2578120957 >> [v2:172.21.15.203:6832/998005540,v1:172.21.15.203:6833/998005540] conn(0x7fb54806a280 0x7fb54806c740 unknown :-1 s=CLOSED pgs=70 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:47:47.387 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:47.385+0000 7fb5618d9640 1 --2- 172.21.15.43:0/2578120957 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb55c0c5c50 0x7fb55c0c6100 unknown :-1 s=CLOSED pgs=725 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:47:47.388 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:47.385+0000 7fb5618d9640 1 --2- 172.21.15.43:0/2578120957 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fb55c15d1c0 0x7fb55c0c5710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:47:47.388 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:47.385+0000 7fb5618d9640 1 --2- 172.21.15.43:0/2578120957 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fb5440047c0 0x7fb55c0c3d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:47:47.388 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:47.385+0000 7fb5618d9640 1 -- 172.21.15.43:0/2578120957 >> 172.21.15.43:0/2578120957 conn(0x7fb55c099600 msgr2=0x7fb55c0b39f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-09T13:47:47.388 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:47.385+0000 7fb5618d9640 1 -- 172.21.15.43:0/2578120957 shutdown_connections 2023-08-09T13:47:47.388 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:47.385+0000 7fb5618d9640 1 -- 172.21.15.43:0/2578120957 wait complete. 2023-08-09T13:47:47.409 DEBUG:tasks.cephfs.test_snap_schedules:res=Schedule removed for path /snap_test_dir1/dir7 2023-08-09T13:47:47.409 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-09T13:47:47.410 DEBUG:teuthology.orchestra.run.smithi043:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/dir7/.snap) 2023-08-09T13:47:47.482 INFO:teuthology.orchestra.run.smithi043.stdout:scheduled-2023-08-09-13_37_00_UTC 2023-08-09T13:47:47.482 INFO:teuthology.orchestra.run.smithi043.stdout:scheduled-2023-08-09-13_38_00_UTC 2023-08-09T13:47:47.482 INFO:teuthology.orchestra.run.smithi043.stdout:scheduled-2023-08-09-13_39_01_UTC 2023-08-09T13:47:47.482 INFO:teuthology.orchestra.run.smithi043.stdout:scheduled-2023-08-09-13_40_01_UTC 2023-08-09T13:47:47.483 INFO:teuthology.orchestra.run.smithi043.stdout:scheduled-2023-08-09-13_41_00_UTC 2023-08-09T13:47:47.483 INFO:teuthology.orchestra.run.smithi043.stdout:scheduled-2023-08-09-13_42_00_UTC 2023-08-09T13:47:47.483 INFO:teuthology.orchestra.run.smithi043.stdout:scheduled-2023-08-09-13_43_00_UTC 2023-08-09T13:47:47.483 INFO:teuthology.orchestra.run.smithi043.stdout:scheduled-2023-08-09-13_44_00_UTC 2023-08-09T13:47:47.483 INFO:teuthology.orchestra.run.smithi043.stdout:scheduled-2023-08-09-13_45_00_UTC 2023-08-09T13:47:47.483 INFO:teuthology.orchestra.run.smithi043.stdout:scheduled-2023-08-09-13_46_00_UTC 2023-08-09T13:47:47.483 INFO:teuthology.orchestra.run.smithi043.stdout:scheduled-2023-08-09-13_47_01_UTC 2023-08-09T13:47:47.484 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/dir7/.snap/scheduled-2023-08-09-13_37_00_UTC 2023-08-09T13:47:47.485 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-09T13:47:47.485 DEBUG:teuthology.orchestra.run.smithi043:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/dir7/.snap/scheduled-2023-08-09-13_37_00_UTC) 2023-08-09T13:47:47.553 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/dir7/.snap/scheduled-2023-08-09-13_38_00_UTC 2023-08-09T13:47:47.554 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-09T13:47:47.554 DEBUG:teuthology.orchestra.run.smithi043:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/dir7/.snap/scheduled-2023-08-09-13_38_00_UTC) 2023-08-09T13:47:47.626 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/dir7/.snap/scheduled-2023-08-09-13_39_01_UTC 2023-08-09T13:47:47.626 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-09T13:47:47.627 DEBUG:teuthology.orchestra.run.smithi043:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/dir7/.snap/scheduled-2023-08-09-13_39_01_UTC) 2023-08-09T13:47:47.707 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/dir7/.snap/scheduled-2023-08-09-13_40_01_UTC 2023-08-09T13:47:47.708 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-09T13:47:47.708 DEBUG:teuthology.orchestra.run.smithi043:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/dir7/.snap/scheduled-2023-08-09-13_40_01_UTC) 2023-08-09T13:47:47.788 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/dir7/.snap/scheduled-2023-08-09-13_41_00_UTC 2023-08-09T13:47:47.789 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-09T13:47:47.789 DEBUG:teuthology.orchestra.run.smithi043:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/dir7/.snap/scheduled-2023-08-09-13_41_00_UTC) 2023-08-09T13:47:47.866 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/dir7/.snap/scheduled-2023-08-09-13_42_00_UTC 2023-08-09T13:47:47.867 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-09T13:47:47.867 DEBUG:teuthology.orchestra.run.smithi043:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/dir7/.snap/scheduled-2023-08-09-13_42_00_UTC) 2023-08-09T13:47:47.951 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/dir7/.snap/scheduled-2023-08-09-13_43_00_UTC 2023-08-09T13:47:47.951 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-09T13:47:47.952 DEBUG:teuthology.orchestra.run.smithi043:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/dir7/.snap/scheduled-2023-08-09-13_43_00_UTC) 2023-08-09T13:47:48.045 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/dir7/.snap/scheduled-2023-08-09-13_44_00_UTC 2023-08-09T13:47:48.046 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-09T13:47:48.046 DEBUG:teuthology.orchestra.run.smithi043:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/dir7/.snap/scheduled-2023-08-09-13_44_00_UTC) 2023-08-09T13:47:48.110 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/dir7/.snap/scheduled-2023-08-09-13_45_00_UTC 2023-08-09T13:47:48.111 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-09T13:47:48.111 DEBUG:teuthology.orchestra.run.smithi043:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/dir7/.snap/scheduled-2023-08-09-13_45_00_UTC) 2023-08-09T13:47:48.173 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/dir7/.snap/scheduled-2023-08-09-13_46_00_UTC 2023-08-09T13:47:48.174 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-09T13:47:48.174 DEBUG:teuthology.orchestra.run.smithi043:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/dir7/.snap/scheduled-2023-08-09-13_46_00_UTC) 2023-08-09T13:47:48.244 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/dir7/.snap/scheduled-2023-08-09-13_47_01_UTC 2023-08-09T13:47:48.244 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-09T13:47:48.244 DEBUG:teuthology.orchestra.run.smithi043:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/dir7/.snap/scheduled-2023-08-09-13_47_01_UTC) 2023-08-09T13:47:48.331 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-09T13:47:48.332 DEBUG:teuthology.orchestra.run.smithi043:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/dir7) 2023-08-09T13:47:48.406 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs snap-schedule remove --fs cephfs /snap_test_dir1/dir8 1M 2023-08-09T13:47:48.601 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:48.599+0000 7f3c74396640 1 Processor -- start 2023-08-09T13:47:48.601 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:48.599+0000 7f3c74396640 1 -- start start 2023-08-09T13:47:48.603 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:48.601+0000 7f3c74396640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f3c6c15da40 0x7f3c6c0ce4c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:47:48.603 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:48.601+0000 7f3c74396640 1 -- --> v1:172.21.15.43:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3c6c0ca410 con 0x7f3c6c15d5b0 2023-08-09T13:47:48.603 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:48.601+0000 7f3c74396640 1 -- --> v1:172.21.15.203:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3c6c15a050 con 0x7f3c6c069a30 2023-08-09T13:47:48.603 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:48.601+0000 7f3c74396640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3c6c159e70 con 0x7f3c6c15da40 2023-08-09T13:47:48.603 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:48.601+0000 7f3c71931640 1 --1- >> v1:172.21.15.43:6789/0 conn(0x7f3c6c15d5b0 0x7f3c6c069e10 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.43:6789/0 says I am v1:172.21.15.43:39038/0 (socket says 172.21.15.43:39038) 2023-08-09T13:47:48.603 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:48.601+0000 7f3c71931640 1 -- 172.21.15.43:0/3400099810 learned_addr learned my addr 172.21.15.43:0/3400099810 (peer_addr_for_me v1:172.21.15.43:0/0) 2023-08-09T13:47:48.605 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:48.602+0000 7f3c72933640 1 --2- 172.21.15.43:0/3400099810 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f3c6c15da40 0x7f3c6c0ce4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:47:48.605 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:48.602+0000 7f3c71130640 1 -- 172.21.15.43:0/3400099810 <== mon.0 v1:172.21.15.43:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1670350931 0 0) 0x7f3c6c0ca410 con 0x7f3c6c15d5b0 2023-08-09T13:47:48.605 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:48.602+0000 7f3c71130640 1 -- 172.21.15.43:0/3400099810 --> v1:172.21.15.43:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f3c58005730 con 0x7f3c6c15d5b0 2023-08-09T13:47:48.605 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:48.602+0000 7f3c71130640 1 -- 172.21.15.43:0/3400099810 <== mon.0 v1:172.21.15.43:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2874856654 0 0) 0x7f3c58005730 con 0x7f3c6c15d5b0 2023-08-09T13:47:48.605 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:48.602+0000 7f3c71130640 1 -- 172.21.15.43:0/3400099810 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f3c6c15da40 msgr2=0x7f3c6c0ce4c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-09T13:47:48.605 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:48.602+0000 7f3c71130640 1 --2- 172.21.15.43:0/3400099810 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f3c6c15da40 0x7f3c6c0ce4c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:47:48.606 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:48.602+0000 7f3c71130640 1 -- 172.21.15.43:0/3400099810 >> v1:172.21.15.203:6789/0 conn(0x7f3c6c069a30 legacy=0x7f3c6c159760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:47:48.606 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:48.602+0000 7f3c71130640 1 -- 172.21.15.43:0/3400099810 --> v1:172.21.15.43:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3c6c15a330 con 0x7f3c6c15d5b0 2023-08-09T13:47:48.606 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:48.602+0000 7f3c71130640 1 -- 172.21.15.43:0/3400099810 <== mon.0 v1:172.21.15.43:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1285150740 0 0) 0x7f3c5c001670 con 0x7f3c6c15d5b0 2023-08-09T13:47:48.606 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:48.602+0000 7f3c71130640 1 -- 172.21.15.43:0/3400099810 >> v1:172.21.15.43:6789/0 conn(0x7f3c6c15d5b0 legacy=0x7f3c6c069e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:47:48.606 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:48.602+0000 7f3c71130640 1 --2- 172.21.15.43:0/3400099810 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f3c6c15da40 0x7f3c58005c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:47:48.606 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:48.602+0000 7f3c71130640 1 --2- 172.21.15.43:0/3400099810 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f3c580061a0 0x7f3c580069d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:47:48.607 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:48.602+0000 7f3c71130640 1 --2- 172.21.15.43:0/3400099810 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f3c58006f10 0x7f3c58007390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:47:48.607 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:48.602+0000 7f3c71130640 1 -- 172.21.15.43:0/3400099810 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3c6c159e70 con 0x7f3c58006f10 2023-08-09T13:47:48.607 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:48.603+0000 7f3c71130640 1 -- 172.21.15.43:0/3400099810 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3c5800dd80 con 0x7f3c580061a0 2023-08-09T13:47:48.607 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:48.603+0000 7f3c71130640 1 -- 172.21.15.43:0/3400099810 --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3c5800df40 con 0x7f3c6c15da40 2023-08-09T13:47:48.607 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:48.603+0000 7f3c72933640 1 --2- 172.21.15.43:0/3400099810 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f3c58006f10 0x7f3c58007390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:47:48.607 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:48.603+0000 7f3c72132640 1 --2- 172.21.15.43:0/3400099810 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f3c6c15da40 0x7f3c58005c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:47:48.607 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:48.603+0000 7f3c71931640 1 --2- 172.21.15.43:0/3400099810 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f3c580061a0 0x7f3c580069d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:47:48.608 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:48.603+0000 7f3c72933640 1 -- 172.21.15.43:0/3400099810 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f3c6c15da40 msgr2=0x7f3c58005c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-09T13:47:48.608 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:48.603+0000 7f3c72933640 1 --2- 172.21.15.43:0/3400099810 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f3c6c15da40 0x7f3c58005c20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:47:48.608 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:48.603+0000 7f3c72933640 1 -- 172.21.15.43:0/3400099810 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f3c580061a0 msgr2=0x7f3c580069d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-09T13:47:48.608 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:48.603+0000 7f3c72933640 1 --2- 172.21.15.43:0/3400099810 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f3c580061a0 0x7f3c580069d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:47:48.608 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:48.603+0000 7f3c72933640 1 -- 172.21.15.43:0/3400099810 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3c58005730 con 0x7f3c58006f10 2023-08-09T13:47:48.609 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:48.604+0000 7f3c72933640 1 --2- 172.21.15.43:0/3400099810 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f3c58006f10 0x7f3c58007390 secure :-1 s=READY pgs=727 cs=0 l=1 rev1=1 crypto rx=0x7f3c64009fc0 tx=0x7f3c58007a00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b06a199149c39ab5 server_cookie=0 in_seq=0 out_seq=0 2023-08-09T13:47:48.609 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:48.605+0000 7f3c71130640 1 -- 172.21.15.43:0/3400099810 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3c6402cc20 con 0x7f3c58006f10 2023-08-09T13:47:48.609 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:48.605+0000 7f3c71130640 1 -- 172.21.15.43:0/3400099810 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3c6400f040 con 0x7f3c58006f10 2023-08-09T13:47:48.610 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:48.605+0000 7f3c71130640 1 -- 172.21.15.43:0/3400099810 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3c64034440 con 0x7f3c58006f10 2023-08-09T13:47:48.610 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:48.605+0000 7f3c74396640 1 -- 172.21.15.43:0/3400099810 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f3c58006f10 msgr2=0x7f3c58007390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:47:48.610 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:48.605+0000 7f3c74396640 1 --2- 172.21.15.43:0/3400099810 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f3c58006f10 0x7f3c58007390 secure :-1 s=READY pgs=727 cs=0 l=1 rev1=1 crypto rx=0x7f3c64009fc0 tx=0x7f3c58007a00 comp rx=0 tx=0).stop 2023-08-09T13:47:48.610 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:48.605+0000 7f3c72933640 1 -- 172.21.15.43:0/3400099810 reap_dead start 2023-08-09T13:47:48.610 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:48.605+0000 7f3c74396640 1 -- 172.21.15.43:0/3400099810 shutdown_connections 2023-08-09T13:47:48.610 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:48.605+0000 7f3c74396640 1 -- 172.21.15.43:0/3400099810 >> 172.21.15.43:0/3400099810 conn(0x7f3c6c06b990 msgr2=0x7f3c6c06bda0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-09T13:47:48.610 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:48.606+0000 7f3c74396640 1 -- 172.21.15.43:0/3400099810 shutdown_connections 2023-08-09T13:47:48.611 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:48.606+0000 7f3c74396640 1 -- 172.21.15.43:0/3400099810 wait complete. 2023-08-09T13:47:48.611 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:48.607+0000 7f3c74396640 1 Processor -- start 2023-08-09T13:47:48.611 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:48.608+0000 7f3c74396640 1 -- start start 2023-08-09T13:47:48.612 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:48.609+0000 7f3c74396640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f3c58006f10 0x7f3c6c156b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:47:48.612 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:48.609+0000 7f3c74396640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f3c6c156430 0x7f3c6c0ab880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:47:48.612 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:48.609+0000 7f3c74396640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f3c6c0abdc0 0x7f3c6c0c7190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:47:48.613 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:48.609+0000 7f3c74396640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3c6c0c6d70 con 0x7f3c58006f10 2023-08-09T13:47:48.613 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:48.609+0000 7f3c74396640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3c6c0aa420 con 0x7f3c6c156430 2023-08-09T13:47:48.613 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:48.609+0000 7f3c74396640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3c6c0a8190 con 0x7f3c6c0abdc0 2023-08-09T13:47:48.613 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:48.609+0000 7f3c72132640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f3c58006f10 0x7f3c6c156b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:47:48.613 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:48.609+0000 7f3c72132640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f3c58006f10 0x7f3c6c156b50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:55784/0 (socket says 172.21.15.43:55784) 2023-08-09T13:47:48.613 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:48.609+0000 7f3c72132640 1 -- 172.21.15.43:0/2354160538 learned_addr learned my addr 172.21.15.43:0/2354160538 (peer_addr_for_me v2:172.21.15.43:0/0) 2023-08-09T13:47:48.613 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:48.609+0000 7f3c72132640 1 -- 172.21.15.43:0/2354160538 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f3c6c0abdc0 msgr2=0x7f3c6c0c7190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:47:48.614 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:48.609+0000 7f3c72132640 1 --2- 172.21.15.43:0/2354160538 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f3c6c0abdc0 0x7f3c6c0c7190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:47:48.614 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:48.609+0000 7f3c72132640 1 -- 172.21.15.43:0/2354160538 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f3c6c156430 msgr2=0x7f3c6c0ab880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:47:48.614 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:48.609+0000 7f3c72132640 1 --2- 172.21.15.43:0/2354160538 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f3c6c156430 0x7f3c6c0ab880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:47:48.614 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:48.609+0000 7f3c72132640 1 -- 172.21.15.43:0/2354160538 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3c6c0a8470 con 0x7f3c58006f10 2023-08-09T13:47:48.614 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:48.610+0000 7f3c72132640 1 --2- 172.21.15.43:0/2354160538 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f3c58006f10 0x7f3c6c156b50 secure :-1 s=READY pgs=728 cs=0 l=1 rev1=1 crypto rx=0x7f3c6800e550 tx=0x7f3c6800ea10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-09T13:47:48.614 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:48.610+0000 7f3c62ffd640 1 -- 172.21.15.43:0/2354160538 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3c680052e0 con 0x7f3c58006f10 2023-08-09T13:47:48.614 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:48.610+0000 7f3c62ffd640 1 -- 172.21.15.43:0/2354160538 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3c68005440 con 0x7f3c58006f10 2023-08-09T13:47:48.615 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:48.610+0000 7f3c74396640 1 -- 172.21.15.43:0/2354160538 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3c6c0a8750 con 0x7f3c58006f10 2023-08-09T13:47:48.615 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:48.610+0000 7f3c62ffd640 1 -- 172.21.15.43:0/2354160538 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3c68003810 con 0x7f3c58006f10 2023-08-09T13:47:48.615 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:48.610+0000 7f3c74396640 1 -- 172.21.15.43:0/2354160538 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3c6c157250 con 0x7f3c58006f10 2023-08-09T13:47:48.615 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:48.611+0000 7f3c74396640 1 -- 172.21.15.43:0/2354160538 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3c6c0739f0 con 0x7f3c58006f10 2023-08-09T13:47:48.616 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:48.613+0000 7f3c62ffd640 1 -- 172.21.15.43:0/2354160538 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 13) v1 ==== 88529+0+0 (secure 0 0 0) 0x7f3c68003970 con 0x7f3c58006f10 2023-08-09T13:47:48.616 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:48.615+0000 7f3c62ffd640 1 --2- 172.21.15.43:0/2354160538 >> [v2:172.21.15.203:6832/998005540,v1:172.21.15.203:6833/998005540] conn(0x7f3c5006a250 0x7f3c5006c710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:47:48.616 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:48.615+0000 7f3c62ffd640 1 -- 172.21.15.43:0/2354160538 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(55..55 src has 1..55) v4 ==== 7282+0+0 (secure 0 0 0) 0x7f3c68089570 con 0x7f3c58006f10 2023-08-09T13:47:48.617 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:48.615+0000 7f3c71931640 1 --2- 172.21.15.43:0/2354160538 >> [v2:172.21.15.203:6832/998005540,v1:172.21.15.203:6833/998005540] conn(0x7f3c5006a250 0x7f3c5006c710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:47:48.618 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:48.617+0000 7f3c71931640 1 --2- 172.21.15.43:0/2354160538 >> [v2:172.21.15.203:6832/998005540,v1:172.21.15.203:6833/998005540] conn(0x7f3c5006a250 0x7f3c5006c710 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f3c5c001eb0 tx=0x7f3c5c009040 comp rx=0 tx=0).ready entity=mgr.5003 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-09T13:47:48.619 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:48.618+0000 7f3c62ffd640 1 -- 172.21.15.43:0/2354160538 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178066 (secure 0 0 0) 0x7f3c68054580 con 0x7f3c58006f10 2023-08-09T13:47:48.851 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:48.849+0000 7f3c74396640 1 -- 172.21.15.43:0/2354160538 --> [v2:172.21.15.203:6832/998005540,v1:172.21.15.203:6833/998005540] -- mgr_command(tid 0: {"prefix": "fs snap-schedule remove", "fs": "cephfs", "path": "/snap_test_dir1/dir8", "repeat": "1M", "target": ["mon-mgr", ""]}) v1 -- 0x7f3c6c157440 con 0x7f3c5006a250 2023-08-09T13:47:49.077 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:49.076+0000 7f3c62ffd640 1 -- 172.21.15.43:0/2354160538 <== mgr.5003 v2:172.21.15.203:6832/998005540 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+47 (secure 0 0 0) 0x7f3c6c157440 con 0x7f3c5006a250 2023-08-09T13:47:49.078 INFO:teuthology.orchestra.run.smithi043.stdout:Schedule removed for path /snap_test_dir1/dir8 2023-08-09T13:47:49.081 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:49.080+0000 7f3c74396640 1 -- 172.21.15.43:0/2354160538 >> [v2:172.21.15.203:6832/998005540,v1:172.21.15.203:6833/998005540] conn(0x7f3c5006a250 msgr2=0x7f3c5006c710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:47:49.082 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:49.080+0000 7f3c74396640 1 --2- 172.21.15.43:0/2354160538 >> [v2:172.21.15.203:6832/998005540,v1:172.21.15.203:6833/998005540] conn(0x7f3c5006a250 0x7f3c5006c710 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f3c5c001eb0 tx=0x7f3c5c009040 comp rx=0 tx=0).stop 2023-08-09T13:47:49.082 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:49.080+0000 7f3c74396640 1 -- 172.21.15.43:0/2354160538 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f3c58006f10 msgr2=0x7f3c6c156b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:47:49.082 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:49.080+0000 7f3c74396640 1 --2- 172.21.15.43:0/2354160538 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f3c58006f10 0x7f3c6c156b50 secure :-1 s=READY pgs=728 cs=0 l=1 rev1=1 crypto rx=0x7f3c6800e550 tx=0x7f3c6800ea10 comp rx=0 tx=0).stop 2023-08-09T13:47:49.082 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:49.081+0000 7f3c74396640 1 -- 172.21.15.43:0/2354160538 shutdown_connections 2023-08-09T13:47:49.083 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:49.081+0000 7f3c74396640 1 --2- 172.21.15.43:0/2354160538 >> [v2:172.21.15.203:6832/998005540,v1:172.21.15.203:6833/998005540] conn(0x7f3c5006a250 0x7f3c5006c710 unknown :-1 s=CLOSED pgs=71 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:47:49.083 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:49.081+0000 7f3c74396640 1 --2- 172.21.15.43:0/2354160538 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f3c6c0abdc0 0x7f3c6c0c7190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:47:49.083 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:49.081+0000 7f3c74396640 1 --2- 172.21.15.43:0/2354160538 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f3c6c156430 0x7f3c6c0ab880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:47:49.083 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:49.081+0000 7f3c74396640 1 --2- 172.21.15.43:0/2354160538 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f3c58006f10 0x7f3c6c156b50 unknown :-1 s=CLOSED pgs=728 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:47:49.083 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:49.081+0000 7f3c74396640 1 -- 172.21.15.43:0/2354160538 >> 172.21.15.43:0/2354160538 conn(0x7f3c6c06b990 msgr2=0x7f3c6c0c4160 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-09T13:47:49.083 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:49.081+0000 7f3c74396640 1 -- 172.21.15.43:0/2354160538 shutdown_connections 2023-08-09T13:47:49.083 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:49.081+0000 7f3c74396640 1 -- 172.21.15.43:0/2354160538 wait complete. 2023-08-09T13:47:49.103 DEBUG:tasks.cephfs.test_snap_schedules:res=Schedule removed for path /snap_test_dir1/dir8 2023-08-09T13:47:49.104 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-09T13:47:49.104 DEBUG:teuthology.orchestra.run.smithi043:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/dir8/.snap) 2023-08-09T13:47:49.179 INFO:teuthology.orchestra.run.smithi043.stdout:scheduled-2023-08-09-13_37_00_UTC 2023-08-09T13:47:49.179 INFO:teuthology.orchestra.run.smithi043.stdout:scheduled-2023-08-09-13_38_00_UTC 2023-08-09T13:47:49.179 INFO:teuthology.orchestra.run.smithi043.stdout:scheduled-2023-08-09-13_39_00_UTC 2023-08-09T13:47:49.179 INFO:teuthology.orchestra.run.smithi043.stdout:scheduled-2023-08-09-13_40_01_UTC 2023-08-09T13:47:49.179 INFO:teuthology.orchestra.run.smithi043.stdout:scheduled-2023-08-09-13_41_01_UTC 2023-08-09T13:47:49.179 INFO:teuthology.orchestra.run.smithi043.stdout:scheduled-2023-08-09-13_42_01_UTC 2023-08-09T13:47:49.180 INFO:teuthology.orchestra.run.smithi043.stdout:scheduled-2023-08-09-13_43_01_UTC 2023-08-09T13:47:49.180 INFO:teuthology.orchestra.run.smithi043.stdout:scheduled-2023-08-09-13_44_00_UTC 2023-08-09T13:47:49.180 INFO:teuthology.orchestra.run.smithi043.stdout:scheduled-2023-08-09-13_45_00_UTC 2023-08-09T13:47:49.180 INFO:teuthology.orchestra.run.smithi043.stdout:scheduled-2023-08-09-13_46_01_UTC 2023-08-09T13:47:49.180 INFO:teuthology.orchestra.run.smithi043.stdout:scheduled-2023-08-09-13_47_00_UTC 2023-08-09T13:47:49.181 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/dir8/.snap/scheduled-2023-08-09-13_37_00_UTC 2023-08-09T13:47:49.181 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-09T13:47:49.182 DEBUG:teuthology.orchestra.run.smithi043:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/dir8/.snap/scheduled-2023-08-09-13_37_00_UTC) 2023-08-09T13:47:49.239 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/dir8/.snap/scheduled-2023-08-09-13_38_00_UTC 2023-08-09T13:47:49.240 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-09T13:47:49.240 DEBUG:teuthology.orchestra.run.smithi043:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/dir8/.snap/scheduled-2023-08-09-13_38_00_UTC) 2023-08-09T13:47:49.298 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/dir8/.snap/scheduled-2023-08-09-13_39_00_UTC 2023-08-09T13:47:49.298 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-09T13:47:49.298 DEBUG:teuthology.orchestra.run.smithi043:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/dir8/.snap/scheduled-2023-08-09-13_39_00_UTC) 2023-08-09T13:47:49.367 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/dir8/.snap/scheduled-2023-08-09-13_40_01_UTC 2023-08-09T13:47:49.368 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-09T13:47:49.368 DEBUG:teuthology.orchestra.run.smithi043:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/dir8/.snap/scheduled-2023-08-09-13_40_01_UTC) 2023-08-09T13:47:49.445 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/dir8/.snap/scheduled-2023-08-09-13_41_01_UTC 2023-08-09T13:47:49.446 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-09T13:47:49.446 DEBUG:teuthology.orchestra.run.smithi043:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/dir8/.snap/scheduled-2023-08-09-13_41_01_UTC) 2023-08-09T13:47:49.518 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/dir8/.snap/scheduled-2023-08-09-13_42_01_UTC 2023-08-09T13:47:49.519 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-09T13:47:49.519 DEBUG:teuthology.orchestra.run.smithi043:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/dir8/.snap/scheduled-2023-08-09-13_42_01_UTC) 2023-08-09T13:47:49.582 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/dir8/.snap/scheduled-2023-08-09-13_43_01_UTC 2023-08-09T13:47:49.583 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-09T13:47:49.583 DEBUG:teuthology.orchestra.run.smithi043:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/dir8/.snap/scheduled-2023-08-09-13_43_01_UTC) 2023-08-09T13:47:49.658 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/dir8/.snap/scheduled-2023-08-09-13_44_00_UTC 2023-08-09T13:47:49.659 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-09T13:47:49.659 DEBUG:teuthology.orchestra.run.smithi043:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/dir8/.snap/scheduled-2023-08-09-13_44_00_UTC) 2023-08-09T13:47:49.745 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/dir8/.snap/scheduled-2023-08-09-13_45_00_UTC 2023-08-09T13:47:49.746 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-09T13:47:49.746 DEBUG:teuthology.orchestra.run.smithi043:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/dir8/.snap/scheduled-2023-08-09-13_45_00_UTC) 2023-08-09T13:47:49.824 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/dir8/.snap/scheduled-2023-08-09-13_46_01_UTC 2023-08-09T13:47:49.824 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-09T13:47:49.825 DEBUG:teuthology.orchestra.run.smithi043:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/dir8/.snap/scheduled-2023-08-09-13_46_01_UTC) 2023-08-09T13:47:49.898 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/dir8/.snap/scheduled-2023-08-09-13_47_00_UTC 2023-08-09T13:47:49.898 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-09T13:47:49.899 DEBUG:teuthology.orchestra.run.smithi043:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/dir8/.snap/scheduled-2023-08-09-13_47_00_UTC) 2023-08-09T13:47:49.972 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-09T13:47:49.973 DEBUG:teuthology.orchestra.run.smithi043:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/dir8) 2023-08-09T13:47:50.044 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs snap-schedule remove --fs cephfs /snap_test_dir1/dir9 1M 2023-08-09T13:47:50.276 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:50.275+0000 7fbcf306a640 1 Processor -- start 2023-08-09T13:47:50.277 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:50.275+0000 7fbcf306a640 1 -- start start 2023-08-09T13:47:50.278 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:50.277+0000 7fbcf306a640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7fbcec15a2c0 0x7fbcec06a1d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:47:50.278 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:50.277+0000 7fbcf306a640 1 -- --> v1:172.21.15.43:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbcec069d40 con 0x7fbcec0c27e0 2023-08-09T13:47:50.279 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:50.277+0000 7fbcf306a640 1 -- --> v1:172.21.15.203:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbcec06a710 con 0x7fbcec0cc900 2023-08-09T13:47:50.279 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:50.277+0000 7fbcf306a640 1 -- --> v2:172.21.15.43:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbcec0697c0 con 0x7fbcec15a2c0 2023-08-09T13:47:50.279 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:50.277+0000 7fbcf0e06640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7fbcec15a2c0 0x7fbcec06a1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:47:50.280 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:50.277+0000 7fbcf0e06640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7fbcec15a2c0 0x7fbcec06a1d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:55786/0 (socket says 172.21.15.43:55786) 2023-08-09T13:47:50.280 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:50.277+0000 7fbcf0e06640 1 -- 172.21.15.43:0/3154867624 learned_addr learned my addr 172.21.15.43:0/3154867624 (peer_addr_for_me v2:172.21.15.43:0/0) 2023-08-09T13:47:50.280 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:50.277+0000 7fbcf1607640 1 --1- 172.21.15.43:0/3154867624 >> v1:172.21.15.43:6789/0 conn(0x7fbcec0c27e0 0x7fbcec0706e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.43:6789/0 says I am v1:172.21.15.43:39042/0 (socket says 172.21.15.43:39042) 2023-08-09T13:47:50.280 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:50.278+0000 7fbcf0e06640 1 -- 172.21.15.43:0/3154867624 >> v1:172.21.15.203:6789/0 conn(0x7fbcec0cc900 legacy=0x7fbcec06dbc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-09T13:47:50.280 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:50.278+0000 7fbcf0e06640 1 -- 172.21.15.43:0/3154867624 >> v1:172.21.15.43:6789/0 conn(0x7fbcec0c27e0 legacy=0x7fbcec0706e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:47:50.280 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:50.278+0000 7fbcf0e06640 1 -- 172.21.15.43:0/3154867624 --> v2:172.21.15.43:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbcec0cb770 con 0x7fbcec15a2c0 2023-08-09T13:47:50.281 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:50.279+0000 7fbcf0e06640 1 --2- 172.21.15.43:0/3154867624 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fbcec15a2c0 0x7fbcec06a1d0 secure :-1 s=READY pgs=730 cs=0 l=1 rev1=1 crypto rx=0x7fbce0009870 tx=0x7fbce002ba30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3ef413cd98b0e76c server_cookie=0 in_seq=0 out_seq=0 2023-08-09T13:47:50.282 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:50.279+0000 7fbceb7fe640 1 -- 172.21.15.43:0/3154867624 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbce002f850 con 0x7fbcec15a2c0 2023-08-09T13:47:50.282 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:50.279+0000 7fbceb7fe640 1 -- 172.21.15.43:0/3154867624 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbce002f9b0 con 0x7fbcec15a2c0 2023-08-09T13:47:50.282 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:50.279+0000 7fbceb7fe640 1 -- 172.21.15.43:0/3154867624 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbce002fd40 con 0x7fbcec15a2c0 2023-08-09T13:47:50.282 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:50.280+0000 7fbcf306a640 1 -- 172.21.15.43:0/3154867624 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fbcec15a2c0 msgr2=0x7fbcec06a1d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:47:50.282 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:50.280+0000 7fbcf306a640 1 --2- 172.21.15.43:0/3154867624 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fbcec15a2c0 0x7fbcec06a1d0 secure :-1 s=READY pgs=730 cs=0 l=1 rev1=1 crypto rx=0x7fbce0009870 tx=0x7fbce002ba30 comp rx=0 tx=0).stop 2023-08-09T13:47:50.282 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:50.280+0000 7fbcf306a640 1 -- 172.21.15.43:0/3154867624 shutdown_connections 2023-08-09T13:47:50.283 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:50.280+0000 7fbcf306a640 1 --2- 172.21.15.43:0/3154867624 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fbcec15a2c0 0x7fbcec06a1d0 unknown :-1 s=CLOSED pgs=730 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:47:50.283 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:50.280+0000 7fbcf306a640 1 -- 172.21.15.43:0/3154867624 >> 172.21.15.43:0/3154867624 conn(0x7fbcec0cbbd0 msgr2=0x7fbcec0cbfe0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-09T13:47:50.283 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:50.280+0000 7fbcf306a640 1 -- 172.21.15.43:0/3154867624 shutdown_connections 2023-08-09T13:47:50.283 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:50.280+0000 7fbcf306a640 1 -- 172.21.15.43:0/3154867624 wait complete. 2023-08-09T13:47:50.283 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:50.282+0000 7fbcf306a640 1 Processor -- start 2023-08-09T13:47:50.284 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:50.282+0000 7fbcf306a640 1 -- start start 2023-08-09T13:47:50.284 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:50.283+0000 7fbcf306a640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fbcec0c27e0 0x7fbcec0ce330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:47:50.285 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:50.283+0000 7fbcf306a640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fbcec0cc900 0x7fbcec0c6220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:47:50.285 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:50.283+0000 7fbcf306a640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fbcec15a2c0 0x7fbcec0c6760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:47:50.285 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:50.283+0000 7fbcf306a640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbcec0c6ca0 con 0x7fbcec0c27e0 2023-08-09T13:47:50.285 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:50.283+0000 7fbcf306a640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbcec0ce870 con 0x7fbcec15a2c0 2023-08-09T13:47:50.285 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:50.283+0000 7fbcf306a640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbcec0cd290 con 0x7fbcec0cc900 2023-08-09T13:47:50.285 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:50.283+0000 7fbcf0e06640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fbcec0c27e0 0x7fbcec0ce330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:47:50.287 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:50.283+0000 7fbcf0e06640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fbcec0c27e0 0x7fbcec0ce330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:55794/0 (socket says 172.21.15.43:55794) 2023-08-09T13:47:50.287 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:50.283+0000 7fbcf0e06640 1 -- 172.21.15.43:0/2536433987 learned_addr learned my addr 172.21.15.43:0/2536433987 (peer_addr_for_me v2:172.21.15.43:0/0) 2023-08-09T13:47:50.287 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:50.284+0000 7fbcf1607640 1 --2- 172.21.15.43:0/2536433987 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fbcec15a2c0 0x7fbcec0c6760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:47:50.287 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:50.284+0000 7fbcebfff640 1 --2- 172.21.15.43:0/2536433987 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fbcec0cc900 0x7fbcec0c6220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:47:50.287 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:50.284+0000 7fbcf0e06640 1 -- 172.21.15.43:0/2536433987 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fbcec0cc900 msgr2=0x7fbcec0c6220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:47:50.287 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:50.284+0000 7fbcf0e06640 1 --2- 172.21.15.43:0/2536433987 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fbcec0cc900 0x7fbcec0c6220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:47:50.287 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:50.284+0000 7fbcf0e06640 1 -- 172.21.15.43:0/2536433987 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fbcec15a2c0 msgr2=0x7fbcec0c6760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:47:50.288 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:50.284+0000 7fbcf0e06640 1 --2- 172.21.15.43:0/2536433987 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fbcec15a2c0 0x7fbcec0c6760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:47:50.288 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:50.284+0000 7fbcf0e06640 1 -- 172.21.15.43:0/2536433987 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbce00094d0 con 0x7fbcec0c27e0 2023-08-09T13:47:50.288 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:50.284+0000 7fbcebfff640 1 --2- 172.21.15.43:0/2536433987 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fbcec0cc900 0x7fbcec0c6220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-08-09T13:47:50.288 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:50.284+0000 7fbcf0e06640 1 --2- 172.21.15.43:0/2536433987 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fbcec0c27e0 0x7fbcec0ce330 secure :-1 s=READY pgs=731 cs=0 l=1 rev1=1 crypto rx=0x7fbce00097c0 tx=0x7fbce0004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-09T13:47:50.288 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:50.285+0000 7fbce97fa640 1 -- 172.21.15.43:0/2536433987 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbce0041400 con 0x7fbcec0c27e0 2023-08-09T13:47:50.288 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:50.285+0000 7fbce97fa640 1 -- 172.21.15.43:0/2536433987 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbce0041560 con 0x7fbcec0c27e0 2023-08-09T13:47:50.288 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:50.285+0000 7fbce97fa640 1 -- 172.21.15.43:0/2536433987 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbce0041910 con 0x7fbcec0c27e0 2023-08-09T13:47:50.289 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:50.285+0000 7fbcf306a640 1 -- 172.21.15.43:0/2536433987 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbcec0cd570 con 0x7fbcec0c27e0 2023-08-09T13:47:50.289 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:50.285+0000 7fbcf306a640 1 -- 172.21.15.43:0/2536433987 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbcec0cdb30 con 0x7fbcec0c27e0 2023-08-09T13:47:50.289 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:50.286+0000 7fbcf306a640 1 -- 172.21.15.43:0/2536433987 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbcec0739f0 con 0x7fbcec0c27e0 2023-08-09T13:47:50.290 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:50.287+0000 7fbce97fa640 1 -- 172.21.15.43:0/2536433987 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 13) v1 ==== 88529+0+0 (secure 0 0 0) 0x7fbce0041a70 con 0x7fbcec0c27e0 2023-08-09T13:47:50.290 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:50.288+0000 7fbce97fa640 1 --2- 172.21.15.43:0/2536433987 >> [v2:172.21.15.203:6832/998005540,v1:172.21.15.203:6833/998005540] conn(0x7fbcd006a250 0x7fbcd006c710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:47:50.290 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:50.288+0000 7fbce97fa640 1 -- 172.21.15.43:0/2536433987 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(57..57 src has 1..57) v4 ==== 7298+0+0 (secure 0 0 0) 0x7fbce003c070 con 0x7fbcec0c27e0 2023-08-09T13:47:50.290 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:50.289+0000 7fbcebfff640 1 --2- 172.21.15.43:0/2536433987 >> [v2:172.21.15.203:6832/998005540,v1:172.21.15.203:6833/998005540] conn(0x7fbcd006a250 0x7fbcd006c710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:47:50.291 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:50.290+0000 7fbcebfff640 1 --2- 172.21.15.43:0/2536433987 >> [v2:172.21.15.203:6832/998005540,v1:172.21.15.203:6833/998005540] conn(0x7fbcd006a250 0x7fbcd006c710 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7fbcec0c9d80 tx=0x7fbcd8009040 comp rx=0 tx=0).ready entity=mgr.5003 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-09T13:47:50.293 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:50.292+0000 7fbce97fa640 1 -- 172.21.15.43:0/2536433987 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178066 (secure 0 0 0) 0x7fbce0076c70 con 0x7fbcec0c27e0 2023-08-09T13:47:50.512 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:50.510+0000 7fbcf306a640 1 -- 172.21.15.43:0/2536433987 --> [v2:172.21.15.203:6832/998005540,v1:172.21.15.203:6833/998005540] -- mgr_command(tid 0: {"prefix": "fs snap-schedule remove", "fs": "cephfs", "path": "/snap_test_dir1/dir9", "repeat": "1M", "target": ["mon-mgr", ""]}) v1 -- 0x7fbcec0699d0 con 0x7fbcd006a250 2023-08-09T13:47:50.716 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:50.714+0000 7fbce97fa640 1 -- 172.21.15.43:0/2536433987 <== mgr.5003 v2:172.21.15.203:6832/998005540 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+47 (secure 0 0 0) 0x7fbcd8005030 con 0x7fbcd006a250 2023-08-09T13:47:50.716 INFO:teuthology.orchestra.run.smithi043.stdout:Schedule removed for path /snap_test_dir1/dir9 2023-08-09T13:47:50.719 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:50.718+0000 7fbcf306a640 1 -- 172.21.15.43:0/2536433987 >> [v2:172.21.15.203:6832/998005540,v1:172.21.15.203:6833/998005540] conn(0x7fbcd006a250 msgr2=0x7fbcd006c710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:47:50.720 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:50.718+0000 7fbcf306a640 1 --2- 172.21.15.43:0/2536433987 >> [v2:172.21.15.203:6832/998005540,v1:172.21.15.203:6833/998005540] conn(0x7fbcd006a250 0x7fbcd006c710 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7fbcec0c9d80 tx=0x7fbcd8009040 comp rx=0 tx=0).stop 2023-08-09T13:47:50.720 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:50.718+0000 7fbcf306a640 1 -- 172.21.15.43:0/2536433987 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fbcec0c27e0 msgr2=0x7fbcec0ce330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:47:50.720 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:50.718+0000 7fbcf306a640 1 --2- 172.21.15.43:0/2536433987 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fbcec0c27e0 0x7fbcec0ce330 secure :-1 s=READY pgs=731 cs=0 l=1 rev1=1 crypto rx=0x7fbce00097c0 tx=0x7fbce0004730 comp rx=0 tx=0).stop 2023-08-09T13:47:50.720 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:50.718+0000 7fbcf306a640 1 -- 172.21.15.43:0/2536433987 shutdown_connections 2023-08-09T13:47:50.720 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:50.718+0000 7fbcf306a640 1 --2- 172.21.15.43:0/2536433987 >> [v2:172.21.15.203:6832/998005540,v1:172.21.15.203:6833/998005540] conn(0x7fbcd006a250 0x7fbcd006c710 unknown :-1 s=CLOSED pgs=72 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:47:50.720 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:50.718+0000 7fbcf306a640 1 --2- 172.21.15.43:0/2536433987 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fbcec15a2c0 0x7fbcec0c6760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:47:50.721 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:50.718+0000 7fbcf306a640 1 --2- 172.21.15.43:0/2536433987 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fbcec0cc900 0x7fbcec0c6220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:47:50.721 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:50.718+0000 7fbcf306a640 1 --2- 172.21.15.43:0/2536433987 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fbcec0c27e0 0x7fbcec0ce330 unknown :-1 s=CLOSED pgs=731 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:47:50.721 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:50.718+0000 7fbcf306a640 1 -- 172.21.15.43:0/2536433987 >> 172.21.15.43:0/2536433987 conn(0x7fbcec0cbbd0 msgr2=0x7fbcec0c4d50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-09T13:47:50.721 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:50.718+0000 7fbcf306a640 1 -- 172.21.15.43:0/2536433987 shutdown_connections 2023-08-09T13:47:50.721 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:50.719+0000 7fbcf306a640 1 -- 172.21.15.43:0/2536433987 wait complete. 2023-08-09T13:47:50.742 DEBUG:tasks.cephfs.test_snap_schedules:res=Schedule removed for path /snap_test_dir1/dir9 2023-08-09T13:47:50.742 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-09T13:47:50.743 DEBUG:teuthology.orchestra.run.smithi043:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/dir9/.snap) 2023-08-09T13:47:50.819 INFO:teuthology.orchestra.run.smithi043.stdout:scheduled-2023-08-09-13_37_01_UTC 2023-08-09T13:47:50.819 INFO:teuthology.orchestra.run.smithi043.stdout:scheduled-2023-08-09-13_38_00_UTC 2023-08-09T13:47:50.819 INFO:teuthology.orchestra.run.smithi043.stdout:scheduled-2023-08-09-13_39_00_UTC 2023-08-09T13:47:50.819 INFO:teuthology.orchestra.run.smithi043.stdout:scheduled-2023-08-09-13_40_00_UTC 2023-08-09T13:47:50.819 INFO:teuthology.orchestra.run.smithi043.stdout:scheduled-2023-08-09-13_41_00_UTC 2023-08-09T13:47:50.820 INFO:teuthology.orchestra.run.smithi043.stdout:scheduled-2023-08-09-13_42_00_UTC 2023-08-09T13:47:50.820 INFO:teuthology.orchestra.run.smithi043.stdout:scheduled-2023-08-09-13_43_00_UTC 2023-08-09T13:47:50.820 INFO:teuthology.orchestra.run.smithi043.stdout:scheduled-2023-08-09-13_44_00_UTC 2023-08-09T13:47:50.820 INFO:teuthology.orchestra.run.smithi043.stdout:scheduled-2023-08-09-13_45_00_UTC 2023-08-09T13:47:50.821 INFO:teuthology.orchestra.run.smithi043.stdout:scheduled-2023-08-09-13_46_00_UTC 2023-08-09T13:47:50.821 INFO:teuthology.orchestra.run.smithi043.stdout:scheduled-2023-08-09-13_47_00_UTC 2023-08-09T13:47:50.822 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/dir9/.snap/scheduled-2023-08-09-13_37_01_UTC 2023-08-09T13:47:50.822 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-09T13:47:50.822 DEBUG:teuthology.orchestra.run.smithi043:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/dir9/.snap/scheduled-2023-08-09-13_37_01_UTC) 2023-08-09T13:47:50.897 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/dir9/.snap/scheduled-2023-08-09-13_38_00_UTC 2023-08-09T13:47:50.898 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-09T13:47:50.898 DEBUG:teuthology.orchestra.run.smithi043:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/dir9/.snap/scheduled-2023-08-09-13_38_00_UTC) 2023-08-09T13:47:50.971 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/dir9/.snap/scheduled-2023-08-09-13_39_00_UTC 2023-08-09T13:47:50.972 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-09T13:47:50.972 DEBUG:teuthology.orchestra.run.smithi043:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/dir9/.snap/scheduled-2023-08-09-13_39_00_UTC) 2023-08-09T13:47:51.053 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/dir9/.snap/scheduled-2023-08-09-13_40_00_UTC 2023-08-09T13:47:51.053 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-09T13:47:51.053 DEBUG:teuthology.orchestra.run.smithi043:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/dir9/.snap/scheduled-2023-08-09-13_40_00_UTC) 2023-08-09T13:47:51.190 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/dir9/.snap/scheduled-2023-08-09-13_41_00_UTC 2023-08-09T13:47:51.191 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-09T13:47:51.191 DEBUG:teuthology.orchestra.run.smithi043:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/dir9/.snap/scheduled-2023-08-09-13_41_00_UTC) 2023-08-09T13:47:51.244 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/dir9/.snap/scheduled-2023-08-09-13_42_00_UTC 2023-08-09T13:47:51.245 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-09T13:47:51.245 DEBUG:teuthology.orchestra.run.smithi043:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/dir9/.snap/scheduled-2023-08-09-13_42_00_UTC) 2023-08-09T13:47:51.300 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/dir9/.snap/scheduled-2023-08-09-13_43_00_UTC 2023-08-09T13:47:51.300 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-09T13:47:51.301 DEBUG:teuthology.orchestra.run.smithi043:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/dir9/.snap/scheduled-2023-08-09-13_43_00_UTC) 2023-08-09T13:47:51.359 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/dir9/.snap/scheduled-2023-08-09-13_44_00_UTC 2023-08-09T13:47:51.359 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-09T13:47:51.360 DEBUG:teuthology.orchestra.run.smithi043:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/dir9/.snap/scheduled-2023-08-09-13_44_00_UTC) 2023-08-09T13:47:51.433 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/dir9/.snap/scheduled-2023-08-09-13_45_00_UTC 2023-08-09T13:47:51.433 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-09T13:47:51.433 DEBUG:teuthology.orchestra.run.smithi043:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/dir9/.snap/scheduled-2023-08-09-13_45_00_UTC) 2023-08-09T13:47:51.509 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/dir9/.snap/scheduled-2023-08-09-13_46_00_UTC 2023-08-09T13:47:51.510 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-09T13:47:51.510 DEBUG:teuthology.orchestra.run.smithi043:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/dir9/.snap/scheduled-2023-08-09-13_46_00_UTC) 2023-08-09T13:47:51.593 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/dir9/.snap/scheduled-2023-08-09-13_47_00_UTC 2023-08-09T13:47:51.594 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-09T13:47:51.594 DEBUG:teuthology.orchestra.run.smithi043:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/dir9/.snap/scheduled-2023-08-09-13_47_00_UTC) 2023-08-09T13:47:51.673 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-09T13:47:51.674 DEBUG:teuthology.orchestra.run.smithi043:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/dir9) 2023-08-09T13:47:51.744 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph mgr module disable snap_schedule 2023-08-09T13:47:51.975 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:51.974+0000 7fa1ffaad640 1 Processor -- start 2023-08-09T13:47:51.976 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:51.974+0000 7fa1ffaad640 1 -- start start 2023-08-09T13:47:51.977 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:51.976+0000 7fa1ffaad640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fa1f8143c20 0x7fa1f806f540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:47:51.977 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:51.976+0000 7fa1ffaad640 1 -- --> v1:172.21.15.43:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa1f806c440 con 0x7fa1f806cfe0 2023-08-09T13:47:51.977 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:51.976+0000 7fa1ffaad640 1 -- --> v1:172.21.15.203:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa1f806b830 con 0x7fa1f80c86f0 2023-08-09T13:47:51.978 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:51.976+0000 7fa1ffaad640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa1f8144000 con 0x7fa1f8143c20 2023-08-09T13:47:51.978 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:51.976+0000 7fa1fe04a640 1 --1- >> v1:172.21.15.43:6789/0 conn(0x7fa1f806cfe0 0x7fa1f80ccc00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.43:6789/0 says I am v1:172.21.15.43:39058/0 (socket says 172.21.15.43:39058) 2023-08-09T13:47:51.978 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:51.976+0000 7fa1fe04a640 1 -- 172.21.15.43:0/1794345859 learned_addr learned my addr 172.21.15.43:0/1794345859 (peer_addr_for_me v1:172.21.15.43:0/0) 2023-08-09T13:47:51.978 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:51.977+0000 7fa1fd849640 1 --2- 172.21.15.43:0/1794345859 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fa1f8143c20 0x7fa1f806f540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:47:51.979 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:51.977+0000 7fa1fc847640 1 -- 172.21.15.43:0/1794345859 <== mon.0 v1:172.21.15.43:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2326574342 0 0) 0x7fa1f806c440 con 0x7fa1f806cfe0 2023-08-09T13:47:51.979 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:51.977+0000 7fa1fc847640 1 -- 172.21.15.43:0/1794345859 --> v1:172.21.15.43:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fa1e4003520 con 0x7fa1f806cfe0 2023-08-09T13:47:51.980 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:51.977+0000 7fa1fc847640 1 -- 172.21.15.43:0/1794345859 <== mon.0 v1:172.21.15.43:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3224635940 0 0) 0x7fa1e4003520 con 0x7fa1f806cfe0 2023-08-09T13:47:51.980 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:51.977+0000 7fa1fc847640 1 -- 172.21.15.43:0/1794345859 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fa1f8143c20 msgr2=0x7fa1f806f540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-09T13:47:51.980 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:51.977+0000 7fa1fc847640 1 --2- 172.21.15.43:0/1794345859 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fa1f8143c20 0x7fa1f806f540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:47:51.980 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:51.977+0000 7fa1fc847640 1 -- 172.21.15.43:0/1794345859 >> v1:172.21.15.203:6789/0 conn(0x7fa1f80c86f0 legacy=0x7fa1f806c8d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:47:51.980 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:51.977+0000 7fa1fc847640 1 -- 172.21.15.43:0/1794345859 --> v1:172.21.15.43:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa1f806bb10 con 0x7fa1f806cfe0 2023-08-09T13:47:51.981 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:51.977+0000 7fa1fc847640 1 -- 172.21.15.43:0/1794345859 <== mon.0 v1:172.21.15.43:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1285150740 0 0) 0x7fa1f00019f0 con 0x7fa1f806cfe0 2023-08-09T13:47:51.981 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:51.977+0000 7fa1fc847640 1 -- 172.21.15.43:0/1794345859 >> v1:172.21.15.43:6789/0 conn(0x7fa1f806cfe0 legacy=0x7fa1f80ccc00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:47:51.981 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:51.977+0000 7fa1fc847640 1 --2- 172.21.15.43:0/1794345859 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa1e4003a10 0x7fa1e4003e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:47:51.981 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:51.977+0000 7fa1fc847640 1 --2- 172.21.15.43:0/1794345859 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fa1e40047c0 0x7fa1e4004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:47:51.981 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:51.977+0000 7fa1fc847640 1 --2- 172.21.15.43:0/1794345859 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fa1f8143c20 0x7fa1e4005230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:47:51.981 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:51.977+0000 7fa1fc847640 1 -- 172.21.15.43:0/1794345859 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa1f8144000 con 0x7fa1e4003a10 2023-08-09T13:47:51.981 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:51.977+0000 7fa1fc847640 1 -- 172.21.15.43:0/1794345859 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa1e400bb40 con 0x7fa1e40047c0 2023-08-09T13:47:51.982 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:51.977+0000 7fa1fc847640 1 -- 172.21.15.43:0/1794345859 --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa1e400bd00 con 0x7fa1f8143c20 2023-08-09T13:47:51.982 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:51.978+0000 7fa1fd849640 1 --2- 172.21.15.43:0/1794345859 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa1e4003a10 0x7fa1e4003e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:47:51.982 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:51.978+0000 7fa1fe04a640 1 --2- 172.21.15.43:0/1794345859 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fa1f8143c20 0x7fa1e4005230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:47:51.982 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:51.978+0000 7fa1fd849640 1 -- 172.21.15.43:0/1794345859 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fa1f8143c20 msgr2=0x7fa1e4005230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-09T13:47:51.982 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:51.978+0000 7fa1fd849640 1 --2- 172.21.15.43:0/1794345859 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fa1f8143c20 0x7fa1e4005230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:47:51.982 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:51.978+0000 7fa1fd849640 1 -- 172.21.15.43:0/1794345859 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fa1e40047c0 msgr2=0x7fa1e4004c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-09T13:47:51.983 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:51.978+0000 7fa1fd849640 1 --2- 172.21.15.43:0/1794345859 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fa1e40047c0 0x7fa1e4004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:47:51.983 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:51.978+0000 7fa1fd849640 1 -- 172.21.15.43:0/1794345859 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa1e4003520 con 0x7fa1e4003a10 2023-08-09T13:47:51.984 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:51.980+0000 7fa1fd849640 1 --2- 172.21.15.43:0/1794345859 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa1e4003a10 0x7fa1e4003e30 secure :-1 s=READY pgs=733 cs=0 l=1 rev1=1 crypto rx=0x7fa1f4009a90 tx=0x7fa1e40057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1578d2090867a947 server_cookie=0 in_seq=0 out_seq=0 2023-08-09T13:47:51.984 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:51.980+0000 7fa1fc847640 1 -- 172.21.15.43:0/1794345859 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa1f402c680 con 0x7fa1e4003a10 2023-08-09T13:47:51.984 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:51.980+0000 7fa1fc847640 1 -- 172.21.15.43:0/1794345859 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa1f402c7e0 con 0x7fa1e4003a10 2023-08-09T13:47:51.984 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:51.980+0000 7fa1fc847640 1 -- 172.21.15.43:0/1794345859 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa1f402cb70 con 0x7fa1e4003a10 2023-08-09T13:47:51.984 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:51.980+0000 7fa1ffaad640 1 -- 172.21.15.43:0/1794345859 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa1e4003a10 msgr2=0x7fa1e4003e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:47:51.984 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:51.980+0000 7fa1ffaad640 1 --2- 172.21.15.43:0/1794345859 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa1e4003a10 0x7fa1e4003e30 secure :-1 s=READY pgs=733 cs=0 l=1 rev1=1 crypto rx=0x7fa1f4009a90 tx=0x7fa1e40057f0 comp rx=0 tx=0).stop 2023-08-09T13:47:51.985 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:51.981+0000 7fa1fe04a640 1 -- 172.21.15.43:0/1794345859 reap_dead start 2023-08-09T13:47:51.985 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:51.981+0000 7fa1ffaad640 1 -- 172.21.15.43:0/1794345859 shutdown_connections 2023-08-09T13:47:51.985 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:51.981+0000 7fa1ffaad640 1 -- 172.21.15.43:0/1794345859 >> 172.21.15.43:0/1794345859 conn(0x7fa1f80ce0a0 msgr2=0x7fa1f80ce480 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-09T13:47:51.988 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:51.981+0000 7fa1ffaad640 1 -- 172.21.15.43:0/1794345859 shutdown_connections 2023-08-09T13:47:51.988 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:51.981+0000 7fa1ffaad640 1 -- 172.21.15.43:0/1794345859 wait complete. 2023-08-09T13:47:51.988 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:51.983+0000 7fa1ffaad640 1 Processor -- start 2023-08-09T13:47:51.991 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:51.983+0000 7fa1ffaad640 1 -- start start 2023-08-09T13:47:51.991 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:51.984+0000 7fa1ffaad640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa1e4003a10 0x7fa1f81f2d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:47:51.991 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:51.984+0000 7fa1ffaad640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fa1f815cc90 0x7fa1f80c38c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:47:51.991 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:51.984+0000 7fa1ffaad640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fa1f80c3e00 0x7fa1f80705b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:47:51.991 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:51.984+0000 7fa1ffaad640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa1f80b1dd0 con 0x7fa1e4003a10 2023-08-09T13:47:51.991 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:51.984+0000 7fa1ffaad640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa1f80b20b0 con 0x7fa1f815cc90 2023-08-09T13:47:51.992 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:51.984+0000 7fa1ffaad640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa1f80b2390 con 0x7fa1f80c3e00 2023-08-09T13:47:51.992 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:51.984+0000 7fa1fd849640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa1e4003a10 0x7fa1f81f2d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:47:51.992 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:51.984+0000 7fa1fd849640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa1e4003a10 0x7fa1f81f2d90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:55812/0 (socket says 172.21.15.43:55812) 2023-08-09T13:47:51.992 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:51.984+0000 7fa1fd849640 1 -- 172.21.15.43:0/2878796656 learned_addr learned my addr 172.21.15.43:0/2878796656 (peer_addr_for_me v2:172.21.15.43:0/0) 2023-08-09T13:47:51.992 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:51.985+0000 7fa1fd849640 1 -- 172.21.15.43:0/2878796656 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fa1f80c3e00 msgr2=0x7fa1f80705b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:47:51.992 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:51.985+0000 7fa1fd849640 1 --2- 172.21.15.43:0/2878796656 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fa1f80c3e00 0x7fa1f80705b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:47:51.992 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:51.985+0000 7fa1fd048640 1 --2- 172.21.15.43:0/2878796656 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fa1f815cc90 0x7fa1f80c38c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:47:51.993 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:51.985+0000 7fa1fd849640 1 -- 172.21.15.43:0/2878796656 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fa1f815cc90 msgr2=0x7fa1f80c38c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:47:51.993 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:51.985+0000 7fa1fd849640 1 --2- 172.21.15.43:0/2878796656 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fa1f815cc90 0x7fa1f80c38c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:47:51.993 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:51.985+0000 7fa1fd849640 1 -- 172.21.15.43:0/2878796656 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa1f80b2670 con 0x7fa1e4003a10 2023-08-09T13:47:51.993 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:51.985+0000 7fa1fd048640 1 --2- 172.21.15.43:0/2878796656 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fa1f815cc90 0x7fa1f80c38c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-08-09T13:47:51.993 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:51.985+0000 7fa1fd849640 1 --2- 172.21.15.43:0/2878796656 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa1e4003a10 0x7fa1f81f2d90 secure :-1 s=READY pgs=734 cs=0 l=1 rev1=1 crypto rx=0x7fa1f4009a90 tx=0x7fa1f4004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-09T13:47:51.993 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:51.985+0000 7fa1ee7fc640 1 -- 172.21.15.43:0/2878796656 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa1f402dea0 con 0x7fa1e4003a10 2023-08-09T13:47:51.993 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:51.985+0000 7fa1ee7fc640 1 -- 172.21.15.43:0/2878796656 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa1f4003810 con 0x7fa1e4003a10 2023-08-09T13:47:51.993 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:51.985+0000 7fa1ee7fc640 1 -- 172.21.15.43:0/2878796656 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa1f4003ba0 con 0x7fa1e4003a10 2023-08-09T13:47:51.994 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:51.985+0000 7fa1ffaad640 1 -- 172.21.15.43:0/2878796656 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa1f80b2950 con 0x7fa1e4003a10 2023-08-09T13:47:51.994 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:51.986+0000 7fa1ffaad640 1 -- 172.21.15.43:0/2878796656 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa1f815d110 con 0x7fa1e4003a10 2023-08-09T13:47:51.994 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:51.987+0000 7fa1ffaad640 1 -- 172.21.15.43:0/2878796656 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa1f80739f0 con 0x7fa1e4003a10 2023-08-09T13:47:51.994 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:51.988+0000 7fa1ee7fc640 1 -- 172.21.15.43:0/2878796656 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 13) v1 ==== 88529+0+0 (secure 0 0 0) 0x7fa1f4003d00 con 0x7fa1e4003a10 2023-08-09T13:47:51.994 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:51.988+0000 7fa1ee7fc640 1 --2- 172.21.15.43:0/2878796656 >> [v2:172.21.15.203:6832/998005540,v1:172.21.15.203:6833/998005540] conn(0x7fa1dc06a280 0x7fa1dc06c740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:47:51.994 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:51.989+0000 7fa1ee7fc640 1 -- 172.21.15.43:0/2878796656 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(58..58 src has 1..58) v4 ==== 7410+0+0 (secure 0 0 0) 0x7fa1f40aacc0 con 0x7fa1e4003a10 2023-08-09T13:47:51.994 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:51.989+0000 7fa1fd048640 1 --2- 172.21.15.43:0/2878796656 >> [v2:172.21.15.203:6832/998005540,v1:172.21.15.203:6833/998005540] conn(0x7fa1dc06a280 0x7fa1dc06c740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:47:51.995 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:51.990+0000 7fa1fd048640 1 --2- 172.21.15.43:0/2878796656 >> [v2:172.21.15.203:6832/998005540,v1:172.21.15.203:6833/998005540] conn(0x7fa1dc06a280 0x7fa1dc06c740 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7fa1f80cd9e0 tx=0x7fa1e8009040 comp rx=0 tx=0).ready entity=mgr.5003 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-09T13:47:51.995 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:51.993+0000 7fa1ee7fc640 1 -- 172.21.15.43:0/2878796656 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178066 (secure 0 0 0) 0x7fa1f4075be0 con 0x7fa1e4003a10 2023-08-09T13:47:52.266 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:52.264+0000 7fa1ffaad640 1 -- 172.21.15.43:0/2878796656 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "mgr module disable", "module": "snap_schedule"} v 0) v1 -- 0x7fa1f80cec90 con 0x7fa1e4003a10 2023-08-09T13:47:53.081 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:53.080+0000 7fa1ee7fc640 1 -- 172.21.15.43:0/2878796656 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mgrmap(e 14) v1 ==== 88512+0+0 (secure 0 0 0) 0x7fa1f4075fb0 con 0x7fa1e4003a10 2023-08-09T13:47:53.082 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:53.080+0000 7fa1ee7fc640 1 -- 172.21.15.43:0/2878796656 <== mon.0 v2:172.21.15.43:3300/0 8 ==== mon_command_ack([{"prefix": "mgr module disable", "module": "snap_schedule"}]=0 v14) v1 ==== 93+0+0 (secure 0 0 0) 0x7fa1f4079670 con 0x7fa1e4003a10 2023-08-09T13:47:53.087 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:53.085+0000 7fa1ffaad640 1 -- 172.21.15.43:0/2878796656 >> [v2:172.21.15.203:6832/998005540,v1:172.21.15.203:6833/998005540] conn(0x7fa1dc06a280 msgr2=0x7fa1dc06c740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:47:53.088 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:53.085+0000 7fa1ffaad640 1 --2- 172.21.15.43:0/2878796656 >> [v2:172.21.15.203:6832/998005540,v1:172.21.15.203:6833/998005540] conn(0x7fa1dc06a280 0x7fa1dc06c740 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7fa1f80cd9e0 tx=0x7fa1e8009040 comp rx=0 tx=0).stop 2023-08-09T13:47:53.088 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:53.085+0000 7fa1ffaad640 1 -- 172.21.15.43:0/2878796656 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa1e4003a10 msgr2=0x7fa1f81f2d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:47:53.088 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:53.085+0000 7fa1ffaad640 1 --2- 172.21.15.43:0/2878796656 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa1e4003a10 0x7fa1f81f2d90 secure :-1 s=READY pgs=734 cs=0 l=1 rev1=1 crypto rx=0x7fa1f4009a90 tx=0x7fa1f4004730 comp rx=0 tx=0).stop 2023-08-09T13:47:53.088 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:53.086+0000 7fa1ffaad640 1 -- 172.21.15.43:0/2878796656 shutdown_connections 2023-08-09T13:47:53.089 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:53.086+0000 7fa1ffaad640 1 --2- 172.21.15.43:0/2878796656 >> [v2:172.21.15.203:6832/998005540,v1:172.21.15.203:6833/998005540] conn(0x7fa1dc06a280 0x7fa1dc06c740 unknown :-1 s=CLOSED pgs=73 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:47:53.089 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:53.086+0000 7fa1ffaad640 1 --2- 172.21.15.43:0/2878796656 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fa1f80c3e00 0x7fa1f80705b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:47:53.089 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:53.086+0000 7fa1ffaad640 1 --2- 172.21.15.43:0/2878796656 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fa1f815cc90 0x7fa1f80c38c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:47:53.089 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:53.086+0000 7fa1ffaad640 1 --2- 172.21.15.43:0/2878796656 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa1e4003a10 0x7fa1f81f2d90 unknown :-1 s=CLOSED pgs=734 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:47:53.089 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:53.086+0000 7fa1ffaad640 1 -- 172.21.15.43:0/2878796656 >> 172.21.15.43:0/2878796656 conn(0x7fa1f80ce0a0 msgr2=0x7fa1f80c7330 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-09T13:47:53.089 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:53.086+0000 7fa1ffaad640 1 -- 172.21.15.43:0/2878796656 shutdown_connections 2023-08-09T13:47:53.089 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:53.086+0000 7fa1ffaad640 1 -- 172.21.15.43:0/2878796656 wait complete. 2023-08-09T13:47:53.109 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2023-08-09T13:47:53.110 DEBUG:teuthology.orchestra.run.smithi043:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2023-08-09T13:47:53.223 DEBUG:teuthology.orchestra.run.smithi203:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2023-08-09T13:47:53.304 INFO:tasks.ceph.mgr.x.smithi203.stderr:2023-08-09T13:47:53.302+0000 7fb1cbc55040 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2023-08-09T13:47:53.318 INFO:tasks.ceph.mgr.y.smithi043.stderr:2023-08-09T13:47:53.316+0000 7fac17030040 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2023-08-09T13:47:53.319 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2023-08-09T13:47:53.319 DEBUG:teuthology.orchestra.run.smithi043:> set -ex 2023-08-09T13:47:53.320 DEBUG:teuthology.orchestra.run.smithi043:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-09T13:47:53.380 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-08-09T13:47:53.608 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:53.606+0000 7fed1e184640 1 Processor -- start 2023-08-09T13:47:53.608 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:53.606+0000 7fed1e184640 1 -- start start 2023-08-09T13:47:53.610 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:53.608+0000 7fed1e184640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fed180b2b70 0x7fed180d2ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:47:53.610 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:53.608+0000 7fed1e184640 1 -- --> v1:172.21.15.43:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fed180ce090 con 0x7fed180b3050 2023-08-09T13:47:53.610 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:53.608+0000 7fed1e184640 1 -- --> v1:172.21.15.203:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fed180cd7f0 con 0x7fed180b2790 2023-08-09T13:47:53.610 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:53.608+0000 7fed1e184640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fed180b3950 con 0x7fed180b2b70 2023-08-09T13:47:53.610 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:53.608+0000 7fed17fff640 1 --1- >> v1:172.21.15.43:6789/0 conn(0x7fed180b3050 0x7fed180b3f60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.43:6789/0 says I am v1:172.21.15.43:39066/0 (socket says 172.21.15.43:39066) 2023-08-09T13:47:53.611 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:53.608+0000 7fed17fff640 1 -- 172.21.15.43:0/3518460128 learned_addr learned my addr 172.21.15.43:0/3518460128 (peer_addr_for_me v1:172.21.15.43:0/0) 2023-08-09T13:47:53.612 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:53.609+0000 7fed0ffff640 1 --2- 172.21.15.43:0/3518460128 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fed180b2b70 0x7fed180d2ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:47:53.612 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:53.609+0000 7fed16ffd640 1 -- 172.21.15.43:0/3518460128 <== mon.0 v1:172.21.15.43:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1771623970 0 0) 0x7fed180ce090 con 0x7fed180b3050 2023-08-09T13:47:53.613 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:53.609+0000 7fed16ffd640 1 -- 172.21.15.43:0/3518460128 --> v1:172.21.15.43:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fed04003520 con 0x7fed180b3050 2023-08-09T13:47:53.613 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:53.609+0000 7fed16ffd640 1 -- 172.21.15.43:0/3518460128 <== mon.1 v1:172.21.15.203:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1974861702 0 0) 0x7fed180cd7f0 con 0x7fed180b2790 2023-08-09T13:47:53.613 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:53.609+0000 7fed16ffd640 1 -- 172.21.15.43:0/3518460128 --> v1:172.21.15.203:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fed180ce090 con 0x7fed180b2790 2023-08-09T13:47:53.613 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:53.609+0000 7fed16ffd640 1 -- 172.21.15.43:0/3518460128 <== mon.0 v1:172.21.15.43:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3049034785 0 0) 0x7fed04003520 con 0x7fed180b3050 2023-08-09T13:47:53.613 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:53.609+0000 7fed16ffd640 1 -- 172.21.15.43:0/3518460128 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fed180b2b70 msgr2=0x7fed180d2ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-09T13:47:53.613 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:53.609+0000 7fed16ffd640 1 --2- 172.21.15.43:0/3518460128 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fed180b2b70 0x7fed180d2ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:47:53.614 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:53.609+0000 7fed16ffd640 1 -- 172.21.15.43:0/3518460128 >> v1:172.21.15.203:6789/0 conn(0x7fed180b2790 legacy=0x7fed180b5330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:47:53.614 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:53.609+0000 7fed16ffd640 1 -- 172.21.15.43:0/3518460128 --> v1:172.21.15.43:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fed180cdad0 con 0x7fed180b3050 2023-08-09T13:47:53.614 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:53.609+0000 7fed16ffd640 1 -- 172.21.15.43:0/3518460128 <== mon.0 v1:172.21.15.43:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1285150740 0 0) 0x7fed100019f0 con 0x7fed180b3050 2023-08-09T13:47:53.614 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:53.609+0000 7fed16ffd640 1 -- 172.21.15.43:0/3518460128 >> v1:172.21.15.43:6789/0 conn(0x7fed180b3050 legacy=0x7fed180b3f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:47:53.614 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:53.609+0000 7fed16ffd640 1 --2- 172.21.15.43:0/3518460128 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fed04003ba0 0x7fed04003fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:47:53.614 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:53.609+0000 7fed16ffd640 1 --2- 172.21.15.43:0/3518460128 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fed180b2b70 0x7fed040047c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:47:53.614 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:53.610+0000 7fed16ffd640 1 --2- 172.21.15.43:0/3518460128 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fed04004d00 0x7fed04005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:47:53.615 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:53.610+0000 7fed16ffd640 1 -- 172.21.15.43:0/3518460128 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fed180b3950 con 0x7fed04004d00 2023-08-09T13:47:53.615 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:53.610+0000 7fed16ffd640 1 -- 172.21.15.43:0/3518460128 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fed0400ba60 con 0x7fed04003ba0 2023-08-09T13:47:53.615 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:53.610+0000 7fed16ffd640 1 -- 172.21.15.43:0/3518460128 --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fed0400bc20 con 0x7fed180b2b70 2023-08-09T13:47:53.615 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:53.610+0000 7fed17fff640 1 --2- 172.21.15.43:0/3518460128 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fed04004d00 0x7fed04005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:47:53.615 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:53.610+0000 7fed177fe640 1 --2- 172.21.15.43:0/3518460128 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fed04003ba0 0x7fed04003fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:47:53.615 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:53.610+0000 7fed0ffff640 1 --2- 172.21.15.43:0/3518460128 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fed180b2b70 0x7fed040047c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:47:53.615 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:53.610+0000 7fed17fff640 1 -- 172.21.15.43:0/3518460128 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fed180b2b70 msgr2=0x7fed040047c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-09T13:47:53.615 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:53.610+0000 7fed17fff640 1 --2- 172.21.15.43:0/3518460128 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fed180b2b70 0x7fed040047c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:47:53.616 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:53.610+0000 7fed17fff640 1 -- 172.21.15.43:0/3518460128 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fed04003ba0 msgr2=0x7fed04003fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-09T13:47:53.616 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:53.610+0000 7fed17fff640 1 --2- 172.21.15.43:0/3518460128 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fed04003ba0 0x7fed04003fa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:47:53.616 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:53.610+0000 7fed17fff640 1 -- 172.21.15.43:0/3518460128 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fed04003520 con 0x7fed04004d00 2023-08-09T13:47:53.616 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:53.611+0000 7fed0ffff640 1 --2- 172.21.15.43:0/3518460128 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fed180b2b70 0x7fed040047c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-08-09T13:47:53.617 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:53.612+0000 7fed17fff640 1 --2- 172.21.15.43:0/3518460128 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fed04004d00 0x7fed04005180 secure :-1 s=READY pgs=738 cs=0 l=1 rev1=1 crypto rx=0x7fed10002230 tx=0x7fed040057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a64431fcf54d1ea server_cookie=0 in_seq=0 out_seq=0 2023-08-09T13:47:53.617 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:53.613+0000 7fed16ffd640 1 -- 172.21.15.43:0/3518460128 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fed10034630 con 0x7fed04004d00 2023-08-09T13:47:53.617 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:53.613+0000 7fed16ffd640 1 -- 172.21.15.43:0/3518460128 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fed1002f070 con 0x7fed04004d00 2023-08-09T13:47:53.618 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:53.613+0000 7fed16ffd640 1 -- 172.21.15.43:0/3518460128 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fed100349f0 con 0x7fed04004d00 2023-08-09T13:47:53.618 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:53.613+0000 7fed1e184640 1 -- 172.21.15.43:0/3518460128 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fed04004d00 msgr2=0x7fed04005180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:47:53.618 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:53.613+0000 7fed1e184640 1 --2- 172.21.15.43:0/3518460128 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fed04004d00 0x7fed04005180 secure :-1 s=READY pgs=738 cs=0 l=1 rev1=1 crypto rx=0x7fed10002230 tx=0x7fed040057f0 comp rx=0 tx=0).stop 2023-08-09T13:47:53.618 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:53.613+0000 7fed17fff640 1 -- 172.21.15.43:0/3518460128 reap_dead start 2023-08-09T13:47:53.618 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:53.613+0000 7fed1e184640 1 -- 172.21.15.43:0/3518460128 shutdown_connections 2023-08-09T13:47:53.618 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:53.613+0000 7fed1e184640 1 -- 172.21.15.43:0/3518460128 >> 172.21.15.43:0/3518460128 conn(0x7fed1809b7b0 msgr2=0x7fed180d24e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-09T13:47:53.618 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:53.613+0000 7fed1e184640 1 -- 172.21.15.43:0/3518460128 shutdown_connections 2023-08-09T13:47:53.619 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:53.614+0000 7fed1e184640 1 -- 172.21.15.43:0/3518460128 wait complete. 2023-08-09T13:47:53.619 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:53.615+0000 7fed1e184640 1 Processor -- start 2023-08-09T13:47:53.619 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:53.615+0000 7fed1e184640 1 -- start start 2023-08-09T13:47:53.620 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:53.616+0000 7fed1e184640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fed04004d00 0x7fed180b1ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:47:53.620 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:53.616+0000 7fed1e184640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fed180c95b0 0x7fed1815af50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:47:53.621 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:53.616+0000 7fed1e184640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fed1815c580 0x7fed180cee70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:47:53.621 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:53.616+0000 7fed1e184640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fed180ccc10 con 0x7fed180c95b0 2023-08-09T13:47:53.621 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:53.616+0000 7fed1e184640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fed180c9d20 con 0x7fed04004d00 2023-08-09T13:47:53.621 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:53.616+0000 7fed1e184640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fed180ca000 con 0x7fed1815c580 2023-08-09T13:47:53.621 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:53.617+0000 7fed0ffff640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fed180c95b0 0x7fed1815af50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:47:53.621 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:53.617+0000 7fed0ffff640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fed180c95b0 0x7fed1815af50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:55834/0 (socket says 172.21.15.43:55834) 2023-08-09T13:47:53.621 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:53.617+0000 7fed0ffff640 1 -- 172.21.15.43:0/3490512542 learned_addr learned my addr 172.21.15.43:0/3490512542 (peer_addr_for_me v2:172.21.15.43:0/0) 2023-08-09T13:47:53.621 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:53.617+0000 7fed177fe640 1 --2- 172.21.15.43:0/3490512542 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fed04004d00 0x7fed180b1ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:47:53.622 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:53.617+0000 7fed17fff640 1 --2- 172.21.15.43:0/3490512542 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fed1815c580 0x7fed180cee70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:47:53.622 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:53.617+0000 7fed0ffff640 1 -- 172.21.15.43:0/3490512542 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fed1815c580 msgr2=0x7fed180cee70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:47:53.622 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:53.617+0000 7fed0ffff640 1 --2- 172.21.15.43:0/3490512542 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fed1815c580 0x7fed180cee70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:47:53.622 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:53.617+0000 7fed0ffff640 1 -- 172.21.15.43:0/3490512542 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fed04004d00 msgr2=0x7fed180b1ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:47:53.622 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:53.617+0000 7fed0ffff640 1 --2- 172.21.15.43:0/3490512542 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fed04004d00 0x7fed180b1ad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:47:53.622 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:53.617+0000 7fed0ffff640 1 -- 172.21.15.43:0/3490512542 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fed180ca2e0 con 0x7fed180c95b0 2023-08-09T13:47:53.622 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:53.617+0000 7fed0ffff640 1 --2- 172.21.15.43:0/3490512542 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fed180c95b0 0x7fed1815af50 secure :-1 s=READY pgs=739 cs=0 l=1 rev1=1 crypto rx=0x7fed000108b0 tx=0x7fed00010d70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-09T13:47:53.623 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:53.618+0000 7fed14ff9640 1 -- 172.21.15.43:0/3490512542 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fed0000eb60 con 0x7fed180c95b0 2023-08-09T13:47:53.623 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:53.618+0000 7fed14ff9640 1 -- 172.21.15.43:0/3490512542 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fed0000ecc0 con 0x7fed180c95b0 2023-08-09T13:47:53.623 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:53.618+0000 7fed14ff9640 1 -- 172.21.15.43:0/3490512542 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fed0001c400 con 0x7fed180c95b0 2023-08-09T13:47:53.623 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:53.618+0000 7fed1e184640 1 -- 172.21.15.43:0/3490512542 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fed180ca5c0 con 0x7fed180c95b0 2023-08-09T13:47:53.623 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:53.618+0000 7fed1e184640 1 -- 172.21.15.43:0/3490512542 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fed180c9a30 con 0x7fed180c95b0 2023-08-09T13:47:53.623 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:53.619+0000 7fed1e184640 1 -- 172.21.15.43:0/3490512542 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fed180739f0 con 0x7fed180c95b0 2023-08-09T13:47:53.624 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:53.620+0000 7fed14ff9640 1 -- 172.21.15.43:0/3490512542 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 14) v1 ==== 88512+0+0 (secure 0 0 0) 0x7fed0001c560 con 0x7fed180c95b0 2023-08-09T13:47:53.624 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:53.621+0000 7fed14ff9640 1 --2- 172.21.15.43:0/3490512542 >> [v2:172.21.15.203:6832/998005540,v1:172.21.15.203:6833/998005540] conn(0x7fecf806a230 0x7fecf806c6f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:47:53.624 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:53.621+0000 7fed14ff9640 1 -- 172.21.15.43:0/3490512542 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(60..60 src has 1..60) v4 ==== 7298+0+0 (secure 0 0 0) 0x7fed00089360 con 0x7fed180c95b0 2023-08-09T13:47:53.624 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:53.621+0000 7fed177fe640 1 -- 172.21.15.43:0/3490512542 >> [v2:172.21.15.203:6832/998005540,v1:172.21.15.203:6833/998005540] conn(0x7fecf806a230 msgr2=0x7fecf806c6f0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.203:6832/998005540 2023-08-09T13:47:53.625 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:53.622+0000 7fed177fe640 1 --2- 172.21.15.43:0/3490512542 >> [v2:172.21.15.203:6832/998005540,v1:172.21.15.203:6833/998005540] conn(0x7fecf806a230 0x7fecf806c6f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-08-09T13:47:53.627 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:53.626+0000 7fed14ff9640 1 -- 172.21.15.43:0/3490512542 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178066 (secure 0 0 0) 0x7fed00054390 con 0x7fed180c95b0 2023-08-09T13:47:53.825 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:53.823+0000 7fed177fe640 1 -- 172.21.15.43:0/3490512542 >> [v2:172.21.15.203:6832/998005540,v1:172.21.15.203:6833/998005540] conn(0x7fecf806a230 msgr2=0x7fecf806c6f0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.203:6832/998005540 2023-08-09T13:47:53.825 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:53.823+0000 7fed177fe640 1 --2- 172.21.15.43:0/3490512542 >> [v2:172.21.15.203:6832/998005540,v1:172.21.15.203:6833/998005540] conn(0x7fecf806a230 0x7fecf806c6f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2023-08-09T13:47:53.826 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:53.825+0000 7fed1e184640 1 -- 172.21.15.43:0/3490512542 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fed180d1e00 con 0x7fed180c95b0 2023-08-09T13:47:53.828 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:53.827+0000 7fed14ff9640 1 -- 172.21.15.43:0/3490512542 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v60) v1 ==== 74+0+19833 (secure 0 0 0) 0x7fed000579b0 con 0x7fed180c95b0 2023-08-09T13:47:53.829 INFO:teuthology.orchestra.run.smithi043.stdout: 2023-08-09T13:47:53.829 INFO:teuthology.orchestra.run.smithi043.stdout:{"epoch":60,"fsid":"daac87ee-1337-4250-a955-aea4e408a39b","created":"2023-08-09T13:33:41.701198+0000","modified":"2023-08-09T13:47:53.017118+0000","last_up_change":"2023-08-09T13:33:53.783453+0000","last_in_change":"2023-08-09T13:33:45.148904+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":7,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":"rbd","create_time":"2023-08-09T13:33:54.365137+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":".mgr","create_time":"2023-08-09T13:33:54.831803+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":6,"pool_name":"cephfs_metadata","create_time":"2023-08-09T13:36:08.969993+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"38","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":7,"pool_name":"cephfs_data","create_time":"2023-08-09T13:36:10.366495+0000","flags":8193,"flags_names":"hashpspool,selfmanaged_snaps","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"60","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":220,"snap_epoch":60,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"cbd49ecd-e896-4fbd-864e-5c37a829b165","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":37,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6800","nonce":502667335},{"type":"v1","addr":"172.21.15.43:6801","nonce":502667335}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6802","nonce":502667335},{"type":"v1","addr":"172.21.15.43:6803","nonce":502667335}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6806","nonce":502667335},{"type":"v1","addr":"172.21.15.43:6807","nonce":502667335}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6804","nonce":502667335},{"type":"v1","addr":"172.21.15.43:6805","nonce":502667335}]},"public_addr":"172.21.15.43:6801/502667335","cluster_addr":"172.21.15.43:6803/502667335","heartbeat_back_addr":"172.21.15.43:6807/502667335","heartbeat_front_addr":"172.21.15.43:6805/502667335","state":["exists","up"]},{"osd":1,"uuid":"75c7e0fe-1ec7-4fe8-a2f2-3661a65b8500","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":37,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6808","nonce":3289134610},{"type":"v1","addr":"172.21.15.43:6809","nonce":3289134610}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6810","nonce":3289134610},{"type":"v1","addr":"172.21.15.43:6812","nonce":3289134610}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6821","nonce":3289134610},{"type":"v1","addr":"172.21.15.43:6824","nonce":3289134610}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6815","nonce":3289134610},{"type":"v1","addr":"172.21.15.43:6818","nonce":3289134610}]},"public_addr":"172.21.15.43:6809/3289134610","cluster_addr":"172.21.15.43:6812/3289134610","heartbeat_back_addr":"172.21.15.43:6824/3289134610","heartbeat_front_addr":"172.21.15.43:6818/3289134610","state":["exists","up"]},{"osd":2,"uuid":"bb5c41e9-7f5f-4125-9b01-3bb8bfc2ea83","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":37,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6811","nonce":4079709736},{"type":"v1","addr":"172.21.15.43:6814","nonce":4079709736}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6817","nonce":4079709736},{"type":"v1","addr":"172.21.15.43:6820","nonce":4079709736}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6829","nonce":4079709736},{"type":"v1","addr":"172.21.15.43:6831","nonce":4079709736}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6823","nonce":4079709736},{"type":"v1","addr":"172.21.15.43:6826","nonce":4079709736}]},"public_addr":"172.21.15.43:6814/4079709736","cluster_addr":"172.21.15.43:6820/4079709736","heartbeat_back_addr":"172.21.15.43:6831/4079709736","heartbeat_front_addr":"172.21.15.43:6826/4079709736","state":["exists","up"]},{"osd":3,"uuid":"bfe4c8b4-694a-465c-87e6-2b2aab852613","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":37,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6813","nonce":1709538759},{"type":"v1","addr":"172.21.15.43:6816","nonce":1709538759}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6819","nonce":1709538759},{"type":"v1","addr":"172.21.15.43:6822","nonce":1709538759}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6828","nonce":1709538759},{"type":"v1","addr":"172.21.15.43:6830","nonce":1709538759}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6825","nonce":1709538759},{"type":"v1","addr":"172.21.15.43:6827","nonce":1709538759}]},"public_addr":"172.21.15.43:6816/1709538759","cluster_addr":"172.21.15.43:6822/1709538759","heartbeat_back_addr":"172.21.15.43:6830/1709538759","heartbeat_front_addr":"172.21.15.43:6827/1709538759","state":["exists","up"]},{"osd":4,"uuid":"fa2e8a77-aba1-4756-853e-0d174a5bdeba","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":41,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6800","nonce":4196866481},{"type":"v1","addr":"172.21.15.203:6801","nonce":4196866481}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6802","nonce":4196866481},{"type":"v1","addr":"172.21.15.203:6803","nonce":4196866481}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6806","nonce":4196866481},{"type":"v1","addr":"172.21.15.203:6807","nonce":4196866481}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6804","nonce":4196866481},{"type":"v1","addr":"172.21.15.203:6805","nonce":4196866481}]},"public_addr":"172.21.15.203:6801/4196866481","cluster_addr":"172.21.15.203:6803/4196866481","heartbeat_back_addr":"172.21.15.203:6807/4196866481","heartbeat_front_addr":"172.21.15.203:6805/4196866481","state":["exists","up"]},{"osd":5,"uuid":"81811e4b-4d24-4f51-a82c-f1d700a2a00d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":41,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6816","nonce":3059290081},{"type":"v1","addr":"172.21.15.203:6817","nonce":3059290081}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6818","nonce":3059290081},{"type":"v1","addr":"172.21.15.203:6819","nonce":3059290081}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6822","nonce":3059290081},{"type":"v1","addr":"172.21.15.203:6823","nonce":3059290081}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6820","nonce":3059290081},{"type":"v1","addr":"172.21.15.203:6821","nonce":3059290081}]},"public_addr":"172.21.15.203:6817/3059290081","cluster_addr":"172.21.15.203:6819/3059290081","heartbeat_back_addr":"172.21.15.203:6823/3059290081","heartbeat_front_addr":"172.21.15.203:6821/3059290081","state":["exists","up"]},{"osd":6,"uuid":"307bac4d-6a5b-443d-af75-56d695d7674f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":37,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6824","nonce":512155569},{"type":"v1","addr":"172.21.15.203:6825","nonce":512155569}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6826","nonce":512155569},{"type":"v1","addr":"172.21.15.203:6827","nonce":512155569}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6830","nonce":512155569},{"type":"v1","addr":"172.21.15.203:6831","nonce":512155569}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6828","nonce":512155569},{"type":"v1","addr":"172.21.15.203:6829","nonce":512155569}]},"public_addr":"172.21.15.203:6825/512155569","cluster_addr":"172.21.15.203:6827/512155569","heartbeat_back_addr":"172.21.15.203:6831/512155569","heartbeat_front_addr":"172.21.15.203:6829/512155569","state":["exists","up"]},{"osd":7,"uuid":"a95dc7ca-9f1b-4295-af70-68a22a2d7d5e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":37,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6808","nonce":3331450438},{"type":"v1","addr":"172.21.15.203:6809","nonce":3331450438}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6810","nonce":3331450438},{"type":"v1","addr":"172.21.15.203:6811","nonce":3331450438}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6814","nonce":3331450438},{"type":"v1","addr":"172.21.15.203:6815","nonce":3331450438}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6812","nonce":3331450438},{"type":"v1","addr":"172.21.15.203:6813","nonce":3331450438}]},"public_addr":"172.21.15.203:6809/3331450438","cluster_addr":"172.21.15.203:6811/3331450438","heartbeat_back_addr":"172.21.15.203:6815/3331450438","heartbeat_front_addr":"172.21.15.203:6813/3331450438","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-09T13:33:50.547652+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-09T13:33:50.589469+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-09T13:33:50.561724+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-09T13:33:50.599650+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-09T13:33:50.411855+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-09T13:33:50.219681+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-09T13:33:50.504664+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-09T13:33:50.441248+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"7.2a","mappings":[{"from":0,"to":5}]},{"pgid":"7.3c","mappings":[{"from":0,"to":4}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.203:0/890377927":"2023-08-10T13:36:27.970754+0000","172.21.15.203:0/1761968483":"2023-08-10T13:36:27.970754+0000","172.21.15.203:0/2439593041":"2023-08-10T13:36:27.970754+0000","172.21.15.203:6833/830097474":"2023-08-10T13:36:27.970754+0000","172.21.15.203:6832/830097474":"2023-08-10T13:36:27.970754+0000","172.21.15.203:0/3309257000":"2023-08-10T13:36:27.970754+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":7,"snaps":[{"begin":4,"length":1},{"begin":8,"length":1},{"begin":10,"length":1},{"begin":15,"length":1},{"begin":18,"length":2},{"begin":25,"length":1},{"begin":27,"length":1},{"begin":31,"length":1},{"begin":34,"length":1},{"begin":40,"length":2},{"begin":48,"length":3},{"begin":55,"length":1},{"begin":57,"length":1},{"begin":59,"length":1},{"begin":62,"length":1},{"begin":65,"length":1},{"begin":68,"length":1},{"begin":72,"length":1},{"begin":74,"length":1},{"begin":77,"length":1},{"begin":83,"length":2},{"begin":87,"length":1},{"begin":95,"length":2},{"begin":98,"length":1},{"begin":107,"length":2},{"begin":111,"length":1},{"begin":188,"length":33}]}],"new_removed_snaps":[{"pool":7,"snaps":[{"begin":27,"length":1},{"begin":34,"length":1},{"begin":49,"length":1},{"begin":57,"length":1},{"begin":65,"length":1},{"begin":72,"length":1},{"begin":83,"length":1},{"begin":96,"length":1},{"begin":213,"length":8}]}],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-08-09T13:47:53.831 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:53.830+0000 7fed1e184640 1 -- 172.21.15.43:0/3490512542 >> [v2:172.21.15.203:6832/998005540,v1:172.21.15.203:6833/998005540] conn(0x7fecf806a230 msgr2=0x7fecf806c6f0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-08-09T13:47:53.831 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:53.830+0000 7fed1e184640 1 --2- 172.21.15.43:0/3490512542 >> [v2:172.21.15.203:6832/998005540,v1:172.21.15.203:6833/998005540] conn(0x7fecf806a230 0x7fecf806c6f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:47:53.832 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:53.830+0000 7fed1e184640 1 -- 172.21.15.43:0/3490512542 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fed180c95b0 msgr2=0x7fed1815af50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:47:53.832 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:53.830+0000 7fed1e184640 1 --2- 172.21.15.43:0/3490512542 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fed180c95b0 0x7fed1815af50 secure :-1 s=READY pgs=739 cs=0 l=1 rev1=1 crypto rx=0x7fed000108b0 tx=0x7fed00010d70 comp rx=0 tx=0).stop 2023-08-09T13:47:53.832 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:53.830+0000 7fed1e184640 1 -- 172.21.15.43:0/3490512542 shutdown_connections 2023-08-09T13:47:53.832 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:53.830+0000 7fed1e184640 1 --2- 172.21.15.43:0/3490512542 >> [v2:172.21.15.203:6832/998005540,v1:172.21.15.203:6833/998005540] conn(0x7fecf806a230 0x7fecf806c6f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:47:53.832 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:53.830+0000 7fed1e184640 1 --2- 172.21.15.43:0/3490512542 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fed1815c580 0x7fed180cee70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:47:53.832 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:53.830+0000 7fed1e184640 1 --2- 172.21.15.43:0/3490512542 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fed180c95b0 0x7fed1815af50 unknown :-1 s=CLOSED pgs=739 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:47:53.832 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:53.830+0000 7fed1e184640 1 --2- 172.21.15.43:0/3490512542 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fed04004d00 0x7fed180b1ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:47:53.833 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:53.830+0000 7fed1e184640 1 -- 172.21.15.43:0/3490512542 >> 172.21.15.43:0/3490512542 conn(0x7fed1809b7b0 msgr2=0x7fed1806a800 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-09T13:47:53.833 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:53.830+0000 7fed1e184640 1 -- 172.21.15.43:0/3490512542 shutdown_connections 2023-08-09T13:47:53.833 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:53.830+0000 7fed1e184640 1 -- 172.21.15.43:0/3490512542 wait complete. 2023-08-09T13:47:53.848 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': 'rbd', 'create_time': '2023-08-09T13:33:54.365137+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}, 'read_balance': {'score_acting': 2, 'score_stable': 2, 'optimal_score': 1, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': '.mgr', 'create_time': '2023-08-09T13:33:54.831803+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 6, 'pool_name': 'cephfs_metadata', 'create_time': '2023-08-09T13:36:08.969993+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '38', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.75, 'score_stable': 1.75, 'optimal_score': 1, 'raw_score_acting': 1.75, 'raw_score_stable': 1.75, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 7, 'pool_name': 'cephfs_data', 'create_time': '2023-08-09T13:36:10.366495+0000', 'flags': 8193, 'flags_names': 'hashpspool,selfmanaged_snaps', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '60', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 220, 'snap_epoch': 60, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-08-09T13:47:53.849 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get rbd pg_num 2023-08-09T13:47:54.051 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:54.049+0000 7fa994ddd640 1 Processor -- start 2023-08-09T13:47:54.051 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:54.049+0000 7fa994ddd640 1 -- start start 2023-08-09T13:47:54.052 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:54.051+0000 7fa994ddd640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fa9900b7df0 0x7fa9900d1b30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:47:54.053 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:54.051+0000 7fa994ddd640 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7fa9900cebf0 0x7fa9900cf060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:47:54.053 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:54.051+0000 7fa994ddd640 1 -- --> v1:172.21.15.203:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa9900d16a0 con 0x7fa99015c2d0 2023-08-09T13:47:54.053 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:54.051+0000 7fa994ddd640 1 -- --> v2:172.21.15.203:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa9900b8230 con 0x7fa9900cebf0 2023-08-09T13:47:54.053 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:54.051+0000 7fa994ddd640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa9900cf5a0 con 0x7fa9900b7df0 2023-08-09T13:47:54.054 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:54.052+0000 7fa98ffff640 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7fa9900cebf0 0x7fa9900cf060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:47:54.054 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:54.052+0000 7fa98ffff640 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7fa9900cebf0 0x7fa9900cf060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.203:3300/0 says I am v2:172.21.15.43:41904/0 (socket says 172.21.15.43:41904) 2023-08-09T13:47:54.054 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:54.052+0000 7fa98ffff640 1 -- 172.21.15.43:0/4027674657 learned_addr learned my addr 172.21.15.43:0/4027674657 (peer_addr_for_me v2:172.21.15.43:0/0) 2023-08-09T13:47:54.054 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:54.052+0000 7fa98effd640 1 --2- 172.21.15.43:0/4027674657 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fa9900b7df0 0x7fa9900d1b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:47:54.054 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:54.053+0000 7fa98e7fc640 1 -- 172.21.15.43:0/4027674657 <== mon.1 v1:172.21.15.203:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1954864425 0 0) 0x7fa9900d16a0 con 0x7fa99015c2d0 2023-08-09T13:47:54.055 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:54.053+0000 7fa98ffff640 1 -- 172.21.15.43:0/4027674657 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fa9900b7df0 msgr2=0x7fa9900d1b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-09T13:47:54.055 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:54.053+0000 7fa98ffff640 1 --2- 172.21.15.43:0/4027674657 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fa9900b7df0 0x7fa9900d1b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:47:54.055 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:54.053+0000 7fa98ffff640 1 -- 172.21.15.43:0/4027674657 >> v1:172.21.15.203:6789/0 conn(0x7fa99015c2d0 legacy=0x7fa9900cd020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:47:54.055 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:54.053+0000 7fa98ffff640 1 -- 172.21.15.43:0/4027674657 --> v2:172.21.15.203:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa9900d0d70 con 0x7fa9900cebf0 2023-08-09T13:47:54.055 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:54.053+0000 7fa98effd640 1 --2- 172.21.15.43:0/4027674657 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fa9900b7df0 0x7fa9900d1b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-08-09T13:47:54.056 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:54.055+0000 7fa98ffff640 1 --2- 172.21.15.43:0/4027674657 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fa9900cebf0 0x7fa9900cf060 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7fa988009e50 tx=0x7fa98802bf60 comp rx=0 tx=0).ready entity=mon.1 client_cookie=bf6ae66308ce54cd server_cookie=0 in_seq=0 out_seq=0 2023-08-09T13:47:54.056 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:54.055+0000 7fa98e7fc640 1 -- 172.21.15.43:0/4027674657 <== mon.1 v2:172.21.15.203:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa98802cb00 con 0x7fa9900cebf0 2023-08-09T13:47:54.057 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:54.055+0000 7fa98e7fc640 1 -- 172.21.15.43:0/4027674657 <== mon.1 v2:172.21.15.203:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa98802cc60 con 0x7fa9900cebf0 2023-08-09T13:47:54.057 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:54.055+0000 7fa98e7fc640 1 -- 172.21.15.43:0/4027674657 <== mon.1 v2:172.21.15.203:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa9880343f0 con 0x7fa9900cebf0 2023-08-09T13:47:54.057 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:54.056+0000 7fa994ddd640 1 -- 172.21.15.43:0/4027674657 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fa9900cebf0 msgr2=0x7fa9900cf060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:47:54.058 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:54.056+0000 7fa994ddd640 1 --2- 172.21.15.43:0/4027674657 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fa9900cebf0 0x7fa9900cf060 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7fa988009e50 tx=0x7fa98802bf60 comp rx=0 tx=0).stop 2023-08-09T13:47:54.058 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:54.056+0000 7fa994ddd640 1 -- 172.21.15.43:0/4027674657 shutdown_connections 2023-08-09T13:47:54.058 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:54.056+0000 7fa994ddd640 1 --2- 172.21.15.43:0/4027674657 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fa9900cebf0 0x7fa9900cf060 unknown :-1 s=CLOSED pgs=269 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:47:54.058 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:54.056+0000 7fa994ddd640 1 --2- 172.21.15.43:0/4027674657 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fa9900b7df0 0x7fa9900d1b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:47:54.058 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:54.056+0000 7fa994ddd640 1 -- 172.21.15.43:0/4027674657 >> 172.21.15.43:0/4027674657 conn(0x7fa99006a230 msgr2=0x7fa99006a640 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-09T13:47:54.058 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:54.056+0000 7fa994ddd640 1 -- 172.21.15.43:0/4027674657 shutdown_connections 2023-08-09T13:47:54.058 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:54.056+0000 7fa994ddd640 1 -- 172.21.15.43:0/4027674657 wait complete. 2023-08-09T13:47:54.059 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:54.058+0000 7fa994ddd640 1 Processor -- start 2023-08-09T13:47:54.060 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:54.058+0000 7fa994ddd640 1 -- start start 2023-08-09T13:47:54.060 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:54.059+0000 7fa994ddd640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fa9900b7df0 0x7fa9900cb880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:47:54.061 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:54.059+0000 7fa994ddd640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fa9900cebf0 0x7fa9900b6020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:47:54.061 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:54.059+0000 7fa994ddd640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa99015c2d0 0x7fa9901f0c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:47:54.061 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:54.059+0000 7fa994ddd640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa9900cc0b0 con 0x7fa99015c2d0 2023-08-09T13:47:54.061 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:54.059+0000 7fa994ddd640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa99015dbf0 con 0x7fa9900cebf0 2023-08-09T13:47:54.061 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:54.059+0000 7fa994ddd640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa99015ded0 con 0x7fa9900b7df0 2023-08-09T13:47:54.062 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:54.060+0000 7fa98ffff640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa99015c2d0 0x7fa9901f0c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:47:54.063 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:54.060+0000 7fa98effd640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fa9900cebf0 0x7fa9900b6020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:47:54.063 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:54.060+0000 7fa98ffff640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa99015c2d0 0x7fa9901f0c70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:55848/0 (socket says 172.21.15.43:55848) 2023-08-09T13:47:54.063 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:54.060+0000 7fa98ffff640 1 -- 172.21.15.43:0/3890573342 learned_addr learned my addr 172.21.15.43:0/3890573342 (peer_addr_for_me v2:172.21.15.43:0/0) 2023-08-09T13:47:54.063 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:54.060+0000 7fa98f7fe640 1 --2- 172.21.15.43:0/3890573342 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fa9900b7df0 0x7fa9900cb880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:47:54.063 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:54.060+0000 7fa98ffff640 1 -- 172.21.15.43:0/3890573342 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fa9900b7df0 msgr2=0x7fa9900cb880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:47:54.063 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:54.060+0000 7fa98ffff640 1 --2- 172.21.15.43:0/3890573342 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fa9900b7df0 0x7fa9900cb880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:47:54.063 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:54.060+0000 7fa98ffff640 1 -- 172.21.15.43:0/3890573342 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fa9900cebf0 msgr2=0x7fa9900b6020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:47:54.064 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:54.060+0000 7fa98ffff640 1 --2- 172.21.15.43:0/3890573342 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fa9900cebf0 0x7fa9900b6020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:47:54.064 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:54.060+0000 7fa98ffff640 1 -- 172.21.15.43:0/3890573342 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa984009400 con 0x7fa99015c2d0 2023-08-09T13:47:54.064 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:54.061+0000 7fa98ffff640 1 --2- 172.21.15.43:0/3890573342 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa99015c2d0 0x7fa9901f0c70 secure :-1 s=READY pgs=740 cs=0 l=1 rev1=1 crypto rx=0x7fa988036040 tx=0x7fa988004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-09T13:47:54.064 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:54.061+0000 7fa977fff640 1 -- 172.21.15.43:0/3890573342 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa988009510 con 0x7fa99015c2d0 2023-08-09T13:47:54.064 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:54.061+0000 7fa977fff640 1 -- 172.21.15.43:0/3890573342 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa98803e450 con 0x7fa99015c2d0 2023-08-09T13:47:54.064 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:54.061+0000 7fa994ddd640 1 -- 172.21.15.43:0/3890573342 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa988009b30 con 0x7fa99015c2d0 2023-08-09T13:47:54.064 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:54.061+0000 7fa977fff640 1 -- 172.21.15.43:0/3890573342 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa98803e5e0 con 0x7fa99015c2d0 2023-08-09T13:47:54.065 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:54.061+0000 7fa994ddd640 1 -- 172.21.15.43:0/3890573342 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa99015e490 con 0x7fa99015c2d0 2023-08-09T13:47:54.065 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:54.063+0000 7fa994ddd640 1 -- 172.21.15.43:0/3890573342 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa99004e390 con 0x7fa99015c2d0 2023-08-09T13:47:54.066 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:54.064+0000 7fa977fff640 1 -- 172.21.15.43:0/3890573342 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 14) v1 ==== 88512+0+0 (secure 0 0 0) 0x7fa98803e740 con 0x7fa99015c2d0 2023-08-09T13:47:54.067 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:54.066+0000 7fa977fff640 1 --2- 172.21.15.43:0/3890573342 >> [v2:172.21.15.203:6832/998005540,v1:172.21.15.203:6833/998005540] conn(0x7fa97c06a200 0x7fa97c06c6c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:47:54.067 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:54.066+0000 7fa977fff640 1 -- 172.21.15.43:0/3890573342 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(60..60 src has 1..60) v4 ==== 7298+0+0 (secure 0 0 0) 0x7fa9880ab5e0 con 0x7fa99015c2d0 2023-08-09T13:47:54.068 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:54.066+0000 7fa98f7fe640 1 -- 172.21.15.43:0/3890573342 >> [v2:172.21.15.203:6832/998005540,v1:172.21.15.203:6833/998005540] conn(0x7fa97c06a200 msgr2=0x7fa97c06c6c0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.203:6832/998005540 2023-08-09T13:47:54.068 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:54.066+0000 7fa98f7fe640 1 --2- 172.21.15.43:0/3890573342 >> [v2:172.21.15.203:6832/998005540,v1:172.21.15.203:6833/998005540] conn(0x7fa97c06a200 0x7fa97c06c6c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-08-09T13:47:54.072 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:54.070+0000 7fa977fff640 1 -- 172.21.15.43:0/3890573342 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178066 (secure 0 0 0) 0x7fa988076570 con 0x7fa99015c2d0 2023-08-09T13:47:54.072 INFO:tasks.ceph.mgr.x.smithi203.stderr:2023-08-09T13:47:54.071+0000 7fb1cbc55040 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2023-08-09T13:47:54.128 INFO:tasks.ceph.mgr.y.smithi043.stderr:2023-08-09T13:47:54.126+0000 7fac17030040 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2023-08-09T13:47:54.205 INFO:tasks.ceph.mgr.x.smithi203.stderr:2023-08-09T13:47:54.204+0000 7fb1cbc55040 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2023-08-09T13:47:54.252 INFO:tasks.ceph.mgr.y.smithi043.stderr:2023-08-09T13:47:54.251+0000 7fac17030040 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2023-08-09T13:47:54.265 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:54.263+0000 7fa994ddd640 1 -- 172.21.15.43:0/3890573342 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"} v 0) v1 -- 0x7fa9900d0770 con 0x7fa99015c2d0 2023-08-09T13:47:54.266 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:54.264+0000 7fa977fff640 1 -- 172.21.15.43:0/3890573342 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"}]=0 v60) v1 ==== 92+0+10 (secure 0 0 0) 0x7fa988079b90 con 0x7fa99015c2d0 2023-08-09T13:47:54.266 INFO:teuthology.orchestra.run.smithi043.stdout:pg_num: 8 2023-08-09T13:47:54.267 INFO:tasks.ceph.mgr.x.smithi203.stderr:2023-08-09T13:47:54.266+0000 7fb1cbc55040 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2023-08-09T13:47:54.269 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:54.267+0000 7fa994ddd640 1 -- 172.21.15.43:0/3890573342 >> [v2:172.21.15.203:6832/998005540,v1:172.21.15.203:6833/998005540] conn(0x7fa97c06a200 msgr2=0x7fa97c06c6c0 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2023-08-09T13:47:54.269 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:54.267+0000 7fa98f7fe640 1 -- 172.21.15.43:0/3890573342 >> [v2:172.21.15.203:6832/998005540,v1:172.21.15.203:6833/998005540] conn(0x7fa97c06a200 msgr2=0x7fa97c06c6c0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.203:6832/998005540 2023-08-09T13:47:54.269 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:54.267+0000 7fa98f7fe640 1 --2- 172.21.15.43:0/3890573342 >> [v2:172.21.15.203:6832/998005540,v1:172.21.15.203:6833/998005540] conn(0x7fa97c06a200 0x7fa97c06c6c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2023-08-09T13:47:54.269 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:54.267+0000 7fa994ddd640 1 --2- 172.21.15.43:0/3890573342 >> [v2:172.21.15.203:6832/998005540,v1:172.21.15.203:6833/998005540] conn(0x7fa97c06a200 0x7fa97c06c6c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:47:54.269 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:54.267+0000 7fa994ddd640 1 -- 172.21.15.43:0/3890573342 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa99015c2d0 msgr2=0x7fa9901f0c70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:47:54.270 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:54.267+0000 7fa994ddd640 1 --2- 172.21.15.43:0/3890573342 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa99015c2d0 0x7fa9901f0c70 secure :-1 s=READY pgs=740 cs=0 l=1 rev1=1 crypto rx=0x7fa988036040 tx=0x7fa988004a40 comp rx=0 tx=0).stop 2023-08-09T13:47:54.270 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:54.267+0000 7fa994ddd640 1 -- 172.21.15.43:0/3890573342 shutdown_connections 2023-08-09T13:47:54.270 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:54.267+0000 7fa994ddd640 1 --2- 172.21.15.43:0/3890573342 >> [v2:172.21.15.203:6832/998005540,v1:172.21.15.203:6833/998005540] conn(0x7fa97c06a200 0x7fa97c06c6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:47:54.270 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:54.267+0000 7fa994ddd640 1 --2- 172.21.15.43:0/3890573342 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa99015c2d0 0x7fa9901f0c70 unknown :-1 s=CLOSED pgs=740 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:47:54.270 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:54.267+0000 7fa994ddd640 1 --2- 172.21.15.43:0/3890573342 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fa9900cebf0 0x7fa9900b6020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:47:54.270 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:54.267+0000 7fa994ddd640 1 --2- 172.21.15.43:0/3890573342 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fa9900b7df0 0x7fa9900cb880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:47:54.270 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:54.267+0000 7fa994ddd640 1 -- 172.21.15.43:0/3890573342 >> 172.21.15.43:0/3890573342 conn(0x7fa99006a230 msgr2=0x7fa9900cfd90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-09T13:47:54.271 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:54.267+0000 7fa994ddd640 1 -- 172.21.15.43:0/3890573342 shutdown_connections 2023-08-09T13:47:54.271 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:54.267+0000 7fa994ddd640 1 -- 172.21.15.43:0/3890573342 wait complete. 2023-08-09T13:47:54.288 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-08-09T13:47:54.315 INFO:tasks.ceph.mgr.y.smithi043.stderr:2023-08-09T13:47:54.314+0000 7fac17030040 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2023-08-09T13:47:54.316 INFO:tasks.ceph.mgr.x.smithi203.stderr:2023-08-09T13:47:54.315+0000 7fb1cbc55040 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2023-08-09T13:47:54.370 INFO:tasks.ceph.mgr.y.smithi043.stderr:2023-08-09T13:47:54.369+0000 7fac17030040 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2023-08-09T13:47:54.497 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:54.495+0000 7ff028c2f640 1 Processor -- start 2023-08-09T13:47:54.497 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:54.496+0000 7ff028c2f640 1 -- start start 2023-08-09T13:47:54.499 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:54.497+0000 7ff028c2f640 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7ff0240ce850 0x7ff0240c3d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:47:54.499 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:54.497+0000 7ff028c2f640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7ff0240cf790 0x7ff0240cd630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:47:54.499 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:54.498+0000 7ff028c2f640 1 -- --> v1:172.21.15.203:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff02415e470 con 0x7ff02415a550 2023-08-09T13:47:54.499 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:54.498+0000 7ff028c2f640 1 -- --> v2:172.21.15.203:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff02415e750 con 0x7ff0240ce850 2023-08-09T13:47:54.499 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:54.498+0000 7ff028c2f640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff02415ea30 con 0x7ff0240cf790 2023-08-09T13:47:54.500 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:54.498+0000 7ff021d9b640 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7ff0240ce850 0x7ff0240c3d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:47:54.500 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:54.498+0000 7ff021d9b640 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7ff0240ce850 0x7ff0240c3d90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.203:3300/0 says I am v2:172.21.15.43:41920/0 (socket says 172.21.15.43:41920) 2023-08-09T13:47:54.501 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:54.498+0000 7ff021d9b640 1 -- 172.21.15.43:0/2193105811 learned_addr learned my addr 172.21.15.43:0/2193105811 (peer_addr_for_me v2:172.21.15.43:0/0) 2023-08-09T13:47:54.501 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:54.499+0000 7ff022d9d640 1 --2- 172.21.15.43:0/2193105811 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7ff0240cf790 0x7ff0240cd630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:47:54.501 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:54.499+0000 7ff021d9b640 1 -- 172.21.15.43:0/2193105811 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7ff0240cf790 msgr2=0x7ff0240cd630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-09T13:47:54.501 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:54.499+0000 7ff021d9b640 1 --2- 172.21.15.43:0/2193105811 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7ff0240cf790 0x7ff0240cd630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:47:54.501 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:54.499+0000 7ff021d9b640 1 -- 172.21.15.43:0/2193105811 >> v1:172.21.15.203:6789/0 conn(0x7ff02415a550 legacy=0x7ff0240ce140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:47:54.501 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:54.499+0000 7ff021d9b640 1 -- 172.21.15.43:0/2193105811 --> v2:172.21.15.203:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff02415ed10 con 0x7ff0240ce850 2023-08-09T13:47:54.502 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:54.500+0000 7ff021d9b640 1 --2- 172.21.15.43:0/2193105811 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7ff0240ce850 0x7ff0240c3d90 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7ff018009870 tx=0x7ff01802b860 comp rx=0 tx=0).ready entity=mon.1 client_cookie=9bc02769d9b90df2 server_cookie=0 in_seq=0 out_seq=0 2023-08-09T13:47:54.502 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:54.501+0000 7ff02159a640 1 -- 172.21.15.43:0/2193105811 <== mon.1 v2:172.21.15.203:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff0180317c0 con 0x7ff0240ce850 2023-08-09T13:47:54.502 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:54.501+0000 7ff02159a640 1 -- 172.21.15.43:0/2193105811 <== mon.1 v2:172.21.15.203:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff018031920 con 0x7ff0240ce850 2023-08-09T13:47:54.502 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:54.501+0000 7ff02159a640 1 -- 172.21.15.43:0/2193105811 <== mon.1 v2:172.21.15.203:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff018031cb0 con 0x7ff0240ce850 2023-08-09T13:47:54.503 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:54.501+0000 7ff028c2f640 1 -- 172.21.15.43:0/2193105811 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7ff0240ce850 msgr2=0x7ff0240c3d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:47:54.503 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:54.501+0000 7ff028c2f640 1 --2- 172.21.15.43:0/2193105811 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7ff0240ce850 0x7ff0240c3d90 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7ff018009870 tx=0x7ff01802b860 comp rx=0 tx=0).stop 2023-08-09T13:47:54.503 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:54.502+0000 7ff028c2f640 1 -- 172.21.15.43:0/2193105811 shutdown_connections 2023-08-09T13:47:54.504 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:54.502+0000 7ff028c2f640 1 --2- 172.21.15.43:0/2193105811 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7ff0240cf790 0x7ff0240cd630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:47:54.504 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:54.502+0000 7ff028c2f640 1 --2- 172.21.15.43:0/2193105811 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7ff0240ce850 0x7ff0240c3d90 unknown :-1 s=CLOSED pgs=271 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:47:54.504 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:54.502+0000 7ff028c2f640 1 -- 172.21.15.43:0/2193105811 >> 172.21.15.43:0/2193105811 conn(0x7ff0240b1dd0 msgr2=0x7ff0240b21e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-09T13:47:54.504 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:54.502+0000 7ff028c2f640 1 -- 172.21.15.43:0/2193105811 shutdown_connections 2023-08-09T13:47:54.504 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:54.502+0000 7ff028c2f640 1 -- 172.21.15.43:0/2193105811 wait complete. 2023-08-09T13:47:54.505 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:54.503+0000 7ff028c2f640 1 Processor -- start 2023-08-09T13:47:54.505 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:54.504+0000 7ff028c2f640 1 -- start start 2023-08-09T13:47:54.506 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:54.504+0000 7ff028c2f640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff0240ce850 0x7ff0240c8360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:47:54.506 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:54.504+0000 7ff028c2f640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7ff0240cf790 0x7ff0240c3850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:47:54.506 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:54.505+0000 7ff028c2f640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7ff02415a550 0x7ff0241ef940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:47:54.506 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:54.505+0000 7ff028c2f640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff0240c9630 con 0x7ff0240ce850 2023-08-09T13:47:54.507 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:54.505+0000 7ff028c2f640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff024158ef0 con 0x7ff02415a550 2023-08-09T13:47:54.507 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:54.505+0000 7ff028c2f640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff0240c89b0 con 0x7ff0240cf790 2023-08-09T13:47:54.507 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:54.505+0000 7ff02259c640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff0240ce850 0x7ff0240c8360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:47:54.507 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:54.505+0000 7ff02259c640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff0240ce850 0x7ff0240c8360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:55862/0 (socket says 172.21.15.43:55862) 2023-08-09T13:47:54.507 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:54.505+0000 7ff02259c640 1 -- 172.21.15.43:0/1372708904 learned_addr learned my addr 172.21.15.43:0/1372708904 (peer_addr_for_me v2:172.21.15.43:0/0) 2023-08-09T13:47:54.508 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:54.505+0000 7ff022d9d640 1 --2- 172.21.15.43:0/1372708904 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7ff02415a550 0x7ff0241ef940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:47:54.509 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:54.505+0000 7ff02259c640 1 -- 172.21.15.43:0/1372708904 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7ff0240cf790 msgr2=0x7ff0240c3850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:47:54.509 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:54.505+0000 7ff021d9b640 1 --2- 172.21.15.43:0/1372708904 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7ff0240cf790 0x7ff0240c3850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:47:54.509 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:54.505+0000 7ff02259c640 1 --2- 172.21.15.43:0/1372708904 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7ff0240cf790 0x7ff0240c3850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:47:54.509 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:54.505+0000 7ff02259c640 1 -- 172.21.15.43:0/1372708904 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7ff02415a550 msgr2=0x7ff0241ef940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:47:54.509 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:54.505+0000 7ff02259c640 1 --2- 172.21.15.43:0/1372708904 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7ff02415a550 0x7ff0241ef940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:47:54.509 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:54.505+0000 7ff02259c640 1 -- 172.21.15.43:0/1372708904 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff0180094d0 con 0x7ff0240ce850 2023-08-09T13:47:54.509 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:54.505+0000 7ff022d9d640 1 --2- 172.21.15.43:0/1372708904 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7ff02415a550 0x7ff0241ef940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-08-09T13:47:54.509 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:54.505+0000 7ff021d9b640 1 --2- 172.21.15.43:0/1372708904 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7ff0240cf790 0x7ff0240c3850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-08-09T13:47:54.510 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:54.506+0000 7ff02259c640 1 --2- 172.21.15.43:0/1372708904 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff0240ce850 0x7ff0240c8360 secure :-1 s=READY pgs=741 cs=0 l=1 rev1=1 crypto rx=0x7ff01400c450 tx=0x7ff01400c910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-09T13:47:54.510 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:54.506+0000 7ff012ffd640 1 -- 172.21.15.43:0/1372708904 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff0140052d0 con 0x7ff0240ce850 2023-08-09T13:47:54.510 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:54.506+0000 7ff012ffd640 1 -- 172.21.15.43:0/1372708904 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff014005430 con 0x7ff0240ce850 2023-08-09T13:47:54.510 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:54.506+0000 7ff012ffd640 1 -- 172.21.15.43:0/1372708904 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff014012870 con 0x7ff0240ce850 2023-08-09T13:47:54.510 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:54.506+0000 7ff028c2f640 1 -- 172.21.15.43:0/1372708904 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff0240c8c90 con 0x7ff0240ce850 2023-08-09T13:47:54.510 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:54.506+0000 7ff028c2f640 1 -- 172.21.15.43:0/1372708904 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff0240c9250 con 0x7ff0240ce850 2023-08-09T13:47:54.510 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:54.507+0000 7ff028c2f640 1 -- 172.21.15.43:0/1372708904 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff0240739f0 con 0x7ff0240ce850 2023-08-09T13:47:54.511 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:54.508+0000 7ff012ffd640 1 -- 172.21.15.43:0/1372708904 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 14) v1 ==== 88512+0+0 (secure 0 0 0) 0x7ff0140129d0 con 0x7ff0240ce850 2023-08-09T13:47:54.511 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:54.509+0000 7ff012ffd640 1 --2- 172.21.15.43:0/1372708904 >> [v2:172.21.15.203:6832/998005540,v1:172.21.15.203:6833/998005540] conn(0x7ff00406a200 0x7ff00406c6c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:47:54.511 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:54.509+0000 7ff012ffd640 1 -- 172.21.15.43:0/1372708904 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(60..60 src has 1..60) v4 ==== 7298+0+0 (secure 0 0 0) 0x7ff0140888a0 con 0x7ff0240ce850 2023-08-09T13:47:54.512 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:54.509+0000 7ff021d9b640 1 -- 172.21.15.43:0/1372708904 >> [v2:172.21.15.203:6832/998005540,v1:172.21.15.203:6833/998005540] conn(0x7ff00406a200 msgr2=0x7ff00406c6c0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.203:6832/998005540 2023-08-09T13:47:54.512 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:54.509+0000 7ff021d9b640 1 --2- 172.21.15.43:0/1372708904 >> [v2:172.21.15.203:6832/998005540,v1:172.21.15.203:6833/998005540] conn(0x7ff00406a200 0x7ff00406c6c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-08-09T13:47:54.518 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:54.517+0000 7ff012ffd640 1 -- 172.21.15.43:0/1372708904 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178066 (secure 0 0 0) 0x7ff014053840 con 0x7ff0240ce850 2023-08-09T13:47:54.709 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:54.708+0000 7ff028c2f640 1 -- 172.21.15.43:0/1372708904 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7ff0240cdad0 con 0x7ff0240ce850 2023-08-09T13:47:54.710 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:54.709+0000 7ff012ffd640 1 -- 172.21.15.43:0/1372708904 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v60) v1 ==== 93+0+10 (secure 0 0 0) 0x7ff014056e60 con 0x7ff0240ce850 2023-08-09T13:47:54.710 INFO:teuthology.orchestra.run.smithi043.stdout:pg_num: 1 2023-08-09T13:47:54.711 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:54.710+0000 7ff021d9b640 1 -- 172.21.15.43:0/1372708904 >> [v2:172.21.15.203:6832/998005540,v1:172.21.15.203:6833/998005540] conn(0x7ff00406a200 msgr2=0x7ff00406c6c0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.203:6832/998005540 2023-08-09T13:47:54.711 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:54.710+0000 7ff021d9b640 1 --2- 172.21.15.43:0/1372708904 >> [v2:172.21.15.203:6832/998005540,v1:172.21.15.203:6833/998005540] conn(0x7ff00406a200 0x7ff00406c6c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2023-08-09T13:47:54.713 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:54.711+0000 7ff028c2f640 1 -- 172.21.15.43:0/1372708904 >> [v2:172.21.15.203:6832/998005540,v1:172.21.15.203:6833/998005540] conn(0x7ff00406a200 msgr2=0x7ff00406c6c0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-08-09T13:47:54.713 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:54.711+0000 7ff028c2f640 1 --2- 172.21.15.43:0/1372708904 >> [v2:172.21.15.203:6832/998005540,v1:172.21.15.203:6833/998005540] conn(0x7ff00406a200 0x7ff00406c6c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:47:54.713 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:54.711+0000 7ff028c2f640 1 -- 172.21.15.43:0/1372708904 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff0240ce850 msgr2=0x7ff0240c8360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:47:54.713 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:54.711+0000 7ff028c2f640 1 --2- 172.21.15.43:0/1372708904 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff0240ce850 0x7ff0240c8360 secure :-1 s=READY pgs=741 cs=0 l=1 rev1=1 crypto rx=0x7ff01400c450 tx=0x7ff01400c910 comp rx=0 tx=0).stop 2023-08-09T13:47:54.713 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:54.711+0000 7ff028c2f640 1 -- 172.21.15.43:0/1372708904 shutdown_connections 2023-08-09T13:47:54.713 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:54.711+0000 7ff028c2f640 1 --2- 172.21.15.43:0/1372708904 >> [v2:172.21.15.203:6832/998005540,v1:172.21.15.203:6833/998005540] conn(0x7ff00406a200 0x7ff00406c6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:47:54.714 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:54.711+0000 7ff028c2f640 1 --2- 172.21.15.43:0/1372708904 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7ff02415a550 0x7ff0241ef940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:47:54.714 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:54.711+0000 7ff028c2f640 1 --2- 172.21.15.43:0/1372708904 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7ff0240cf790 0x7ff0240c3850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:47:54.714 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:54.711+0000 7ff028c2f640 1 --2- 172.21.15.43:0/1372708904 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff0240ce850 0x7ff0240c8360 unknown :-1 s=CLOSED pgs=741 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:47:54.714 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:54.711+0000 7ff028c2f640 1 -- 172.21.15.43:0/1372708904 >> 172.21.15.43:0/1372708904 conn(0x7ff0240b1dd0 msgr2=0x7ff0240cff80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-09T13:47:54.714 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:54.712+0000 7ff028c2f640 1 -- 172.21.15.43:0/1372708904 shutdown_connections 2023-08-09T13:47:54.714 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:54.712+0000 7ff028c2f640 1 -- 172.21.15.43:0/1372708904 wait complete. 2023-08-09T13:47:54.730 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-08-09T13:47:54.986 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:54.984+0000 7f0fdac06640 1 Processor -- start 2023-08-09T13:47:54.986 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:54.985+0000 7f0fdac06640 1 -- start start 2023-08-09T13:47:54.987 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:54.986+0000 7f0fdac06640 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7f0fd40af6f0 0x7f0fd406a8d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:47:54.988 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:54.986+0000 7f0fdac06640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f0fd40cf760 0x7f0fd406dcb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:47:54.988 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:54.986+0000 7f0fdac06640 1 -- --> v1:172.21.15.203:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0fd40af1a0 con 0x7f0fd406b140 2023-08-09T13:47:54.988 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:54.986+0000 7f0fdac06640 1 -- --> v2:172.21.15.203:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0fd406be10 con 0x7f0fd40af6f0 2023-08-09T13:47:54.988 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:54.986+0000 7f0fdac06640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0fd40cdc20 con 0x7f0fd40cf760 2023-08-09T13:47:54.989 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:54.987+0000 7f0fd3fff640 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7f0fd40af6f0 0x7f0fd406a8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:47:54.989 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:54.987+0000 7f0fd3fff640 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7f0fd40af6f0 0x7f0fd406a8d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.203:3300/0 says I am v2:172.21.15.43:47888/0 (socket says 172.21.15.43:47888) 2023-08-09T13:47:54.989 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:54.987+0000 7f0fd91a3640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f0fd40cf760 0x7f0fd406dcb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:47:54.989 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:54.987+0000 7f0fd3fff640 1 -- 172.21.15.43:0/2489335701 learned_addr learned my addr 172.21.15.43:0/2489335701 (peer_addr_for_me v2:172.21.15.43:0/0) 2023-08-09T13:47:54.990 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:54.988+0000 7f0fd3fff640 1 -- 172.21.15.43:0/2489335701 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f0fd40cf760 msgr2=0x7f0fd406dcb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-09T13:47:54.990 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:54.988+0000 7f0fd3fff640 1 --2- 172.21.15.43:0/2489335701 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f0fd40cf760 0x7f0fd406dcb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:47:54.990 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:54.988+0000 7f0fd3fff640 1 -- 172.21.15.43:0/2489335701 >> v1:172.21.15.203:6789/0 conn(0x7f0fd406b140 legacy=0x7f0fd406b520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:47:54.990 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:54.988+0000 7f0fd3fff640 1 -- 172.21.15.43:0/2489335701 --> v2:172.21.15.203:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0fd4160100 con 0x7f0fd40af6f0 2023-08-09T13:47:54.991 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:54.989+0000 7f0fd3fff640 1 --2- 172.21.15.43:0/2489335701 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f0fd40af6f0 0x7f0fd406a8d0 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7f0fc0009870 tx=0x7f0fc002b980 comp rx=0 tx=0).ready entity=mon.1 client_cookie=5a5b17e53b90bc1f server_cookie=0 in_seq=0 out_seq=0 2023-08-09T13:47:54.991 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:54.989+0000 7f0fd37fe640 1 -- 172.21.15.43:0/2489335701 <== mon.1 v2:172.21.15.203:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0fc0030840 con 0x7f0fd40af6f0 2023-08-09T13:47:54.991 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:54.990+0000 7f0fd37fe640 1 -- 172.21.15.43:0/2489335701 <== mon.1 v2:172.21.15.203:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0fc00309a0 con 0x7f0fd40af6f0 2023-08-09T13:47:54.991 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:54.990+0000 7f0fd37fe640 1 -- 172.21.15.43:0/2489335701 <== mon.1 v2:172.21.15.203:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0fc0030d30 con 0x7f0fd40af6f0 2023-08-09T13:47:54.992 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:54.990+0000 7f0fdac06640 1 -- 172.21.15.43:0/2489335701 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f0fd40af6f0 msgr2=0x7f0fd406a8d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:47:54.992 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:54.990+0000 7f0fdac06640 1 --2- 172.21.15.43:0/2489335701 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f0fd40af6f0 0x7f0fd406a8d0 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7f0fc0009870 tx=0x7f0fc002b980 comp rx=0 tx=0).stop 2023-08-09T13:47:54.992 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:54.990+0000 7f0fdac06640 1 -- 172.21.15.43:0/2489335701 shutdown_connections 2023-08-09T13:47:54.992 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:54.990+0000 7f0fdac06640 1 --2- 172.21.15.43:0/2489335701 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f0fd40cf760 0x7f0fd406dcb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:47:54.992 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:54.990+0000 7f0fdac06640 1 --2- 172.21.15.43:0/2489335701 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f0fd40af6f0 0x7f0fd406a8d0 unknown :-1 s=CLOSED pgs=273 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:47:54.993 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:54.990+0000 7f0fdac06640 1 -- 172.21.15.43:0/2489335701 >> 172.21.15.43:0/2489335701 conn(0x7f0fd40ccd80 msgr2=0x7f0fd40cd190 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-09T13:47:54.993 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:54.990+0000 7f0fdac06640 1 -- 172.21.15.43:0/2489335701 shutdown_connections 2023-08-09T13:47:54.993 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:54.990+0000 7f0fdac06640 1 -- 172.21.15.43:0/2489335701 wait complete. 2023-08-09T13:47:54.993 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:54.992+0000 7f0fdac06640 1 Processor -- start 2023-08-09T13:47:54.994 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:54.992+0000 7f0fdac06640 1 -- start start 2023-08-09T13:47:54.994 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:54.993+0000 7f0fdac06640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f0fd406b140 0x7f0fd40b5a90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:47:54.995 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:54.993+0000 7f0fdac06640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f0fd40af6f0 0x7f0fd41f1d20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:47:54.995 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:54.993+0000 7f0fdac06640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f0fd40cf760 0x7f0fd415bd10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:47:54.995 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:54.993+0000 7f0fdac06640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0fd41f2260 con 0x7f0fd40af6f0 2023-08-09T13:47:54.995 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:54.993+0000 7f0fdac06640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0fd415c250 con 0x7f0fd406b140 2023-08-09T13:47:54.996 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:54.993+0000 7f0fdac06640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0fd41f2490 con 0x7f0fd40cf760 2023-08-09T13:47:54.997 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:54.994+0000 7f0fd3fff640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f0fd40af6f0 0x7f0fd41f1d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:47:54.997 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:54.994+0000 7f0fd3fff640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f0fd40af6f0 0x7f0fd41f1d20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:57192/0 (socket says 172.21.15.43:57192) 2023-08-09T13:47:54.997 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:54.994+0000 7f0fd3fff640 1 -- 172.21.15.43:0/1750100348 learned_addr learned my addr 172.21.15.43:0/1750100348 (peer_addr_for_me v2:172.21.15.43:0/0) 2023-08-09T13:47:54.997 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:54.994+0000 7f0fd89a2640 1 --2- 172.21.15.43:0/1750100348 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f0fd406b140 0x7f0fd40b5a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:47:54.997 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:54.994+0000 7f0fd91a3640 1 --2- 172.21.15.43:0/1750100348 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f0fd40cf760 0x7f0fd415bd10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:47:54.998 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:54.994+0000 7f0fd3fff640 1 -- 172.21.15.43:0/1750100348 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f0fd40cf760 msgr2=0x7f0fd415bd10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:47:54.998 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:54.994+0000 7f0fd3fff640 1 --2- 172.21.15.43:0/1750100348 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f0fd40cf760 0x7f0fd415bd10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:47:54.998 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:54.994+0000 7f0fd3fff640 1 -- 172.21.15.43:0/1750100348 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f0fd406b140 msgr2=0x7f0fd40b5a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:47:54.998 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:54.994+0000 7f0fd3fff640 1 --2- 172.21.15.43:0/1750100348 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f0fd406b140 0x7f0fd40b5a90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:47:54.998 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:54.994+0000 7f0fd3fff640 1 -- 172.21.15.43:0/1750100348 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0fc00094d0 con 0x7f0fd40af6f0 2023-08-09T13:47:54.998 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:54.994+0000 7f0fd3fff640 1 --2- 172.21.15.43:0/1750100348 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f0fd40af6f0 0x7f0fd41f1d20 secure :-1 s=READY pgs=742 cs=0 l=1 rev1=1 crypto rx=0x7f0fc0000c00 tx=0x7f0fc0004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-09T13:47:54.998 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:54.995+0000 7f0fd17fa640 1 -- 172.21.15.43:0/1750100348 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0fc0039c90 con 0x7f0fd40af6f0 2023-08-09T13:47:54.998 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:54.995+0000 7f0fd17fa640 1 -- 172.21.15.43:0/1750100348 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0fc0039df0 con 0x7f0fd40af6f0 2023-08-09T13:47:54.999 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:54.995+0000 7f0fd17fa640 1 -- 172.21.15.43:0/1750100348 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0fc0043640 con 0x7f0fd40af6f0 2023-08-09T13:47:54.999 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:54.995+0000 7f0fdac06640 1 -- 172.21.15.43:0/1750100348 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0fd41f2770 con 0x7f0fd40af6f0 2023-08-09T13:47:54.999 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:54.995+0000 7f0fdac06640 1 -- 172.21.15.43:0/1750100348 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0fd41f5470 con 0x7f0fd40af6f0 2023-08-09T13:47:55.000 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:54.996+0000 7f0fdac06640 1 -- 172.21.15.43:0/1750100348 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0fd41f2c90 con 0x7f0fd40af6f0 2023-08-09T13:47:55.000 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:54.997+0000 7f0fd17fa640 1 -- 172.21.15.43:0/1750100348 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 14) v1 ==== 88512+0+0 (secure 0 0 0) 0x7f0fc00437a0 con 0x7f0fd40af6f0 2023-08-09T13:47:55.000 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:54.998+0000 7f0fd17fa640 1 --2- 172.21.15.43:0/1750100348 >> [v2:172.21.15.203:6832/998005540,v1:172.21.15.203:6833/998005540] conn(0x7f0fb806a200 0x7f0fb806c6c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:47:55.000 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:54.998+0000 7f0fd17fa640 1 -- 172.21.15.43:0/1750100348 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(60..60 src has 1..60) v4 ==== 7298+0+0 (secure 0 0 0) 0x7f0fc003e070 con 0x7f0fd40af6f0 2023-08-09T13:47:55.000 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:54.998+0000 7f0fd89a2640 1 -- 172.21.15.43:0/1750100348 >> [v2:172.21.15.203:6832/998005540,v1:172.21.15.203:6833/998005540] conn(0x7f0fb806a200 msgr2=0x7f0fb806c6c0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.203:6832/998005540 2023-08-09T13:47:55.000 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:54.998+0000 7f0fd89a2640 1 --2- 172.21.15.43:0/1750100348 >> [v2:172.21.15.203:6832/998005540,v1:172.21.15.203:6833/998005540] conn(0x7f0fb806a200 0x7f0fb806c6c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-08-09T13:47:55.006 INFO:tasks.ceph.mgr.x.smithi203.stderr:2023-08-09T13:47:55.005+0000 7fb1cbc55040 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2023-08-09T13:47:55.007 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:55.006+0000 7f0fd17fa640 1 -- 172.21.15.43:0/1750100348 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178066 (secure 0 0 0) 0x7f0fd41f2c90 con 0x7f0fd40af6f0 2023-08-09T13:47:55.067 INFO:tasks.ceph.mgr.x.smithi203.stderr:2023-08-09T13:47:55.066+0000 7fb1cbc55040 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2023-08-09T13:47:55.105 INFO:tasks.ceph.mgr.y.smithi043.stderr:2023-08-09T13:47:55.103+0000 7fac17030040 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2023-08-09T13:47:55.118 INFO:tasks.ceph.mgr.x.smithi203.stderr:2023-08-09T13:47:55.117+0000 7fb1cbc55040 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2023-08-09T13:47:55.169 INFO:tasks.ceph.mgr.y.smithi043.stderr:2023-08-09T13:47:55.167+0000 7fac17030040 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2023-08-09T13:47:55.197 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:55.196+0000 7f0fdac06640 1 -- 172.21.15.43:0/1750100348 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f0fd41f52d0 con 0x7f0fd40af6f0 2023-08-09T13:47:55.198 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:55.197+0000 7f0fd17fa640 1 -- 172.21.15.43:0/1750100348 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v60) v1 ==== 104+0+11 (secure 0 0 0) 0x7f0fd41f52d0 con 0x7f0fd40af6f0 2023-08-09T13:47:55.198 INFO:teuthology.orchestra.run.smithi043.stdout:pg_num: 32 2023-08-09T13:47:55.200 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:55.199+0000 7f0fd89a2640 1 -- 172.21.15.43:0/1750100348 >> [v2:172.21.15.203:6832/998005540,v1:172.21.15.203:6833/998005540] conn(0x7f0fb806a200 msgr2=0x7f0fb806c6c0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.203:6832/998005540 2023-08-09T13:47:55.200 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:55.199+0000 7f0fd89a2640 1 --2- 172.21.15.43:0/1750100348 >> [v2:172.21.15.203:6832/998005540,v1:172.21.15.203:6833/998005540] conn(0x7f0fb806a200 0x7f0fb806c6c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2023-08-09T13:47:55.201 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:55.199+0000 7f0fdac06640 1 -- 172.21.15.43:0/1750100348 >> [v2:172.21.15.203:6832/998005540,v1:172.21.15.203:6833/998005540] conn(0x7f0fb806a200 msgr2=0x7f0fb806c6c0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-08-09T13:47:55.201 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:55.199+0000 7f0fdac06640 1 --2- 172.21.15.43:0/1750100348 >> [v2:172.21.15.203:6832/998005540,v1:172.21.15.203:6833/998005540] conn(0x7f0fb806a200 0x7f0fb806c6c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:47:55.201 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:55.199+0000 7f0fdac06640 1 -- 172.21.15.43:0/1750100348 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f0fd40af6f0 msgr2=0x7f0fd41f1d20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:47:55.202 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:55.199+0000 7f0fdac06640 1 --2- 172.21.15.43:0/1750100348 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f0fd40af6f0 0x7f0fd41f1d20 secure :-1 s=READY pgs=742 cs=0 l=1 rev1=1 crypto rx=0x7f0fc0000c00 tx=0x7f0fc0004730 comp rx=0 tx=0).stop 2023-08-09T13:47:55.202 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:55.199+0000 7f0fdac06640 1 -- 172.21.15.43:0/1750100348 shutdown_connections 2023-08-09T13:47:55.202 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:55.199+0000 7f0fdac06640 1 --2- 172.21.15.43:0/1750100348 >> [v2:172.21.15.203:6832/998005540,v1:172.21.15.203:6833/998005540] conn(0x7f0fb806a200 0x7f0fb806c6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:47:55.202 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:55.199+0000 7f0fdac06640 1 --2- 172.21.15.43:0/1750100348 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f0fd40cf760 0x7f0fd415bd10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:47:55.202 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:55.199+0000 7f0fdac06640 1 --2- 172.21.15.43:0/1750100348 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f0fd40af6f0 0x7f0fd41f1d20 unknown :-1 s=CLOSED pgs=742 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:47:55.202 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:55.199+0000 7f0fdac06640 1 --2- 172.21.15.43:0/1750100348 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f0fd406b140 0x7f0fd40b5a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:47:55.202 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:55.199+0000 7f0fdac06640 1 -- 172.21.15.43:0/1750100348 >> 172.21.15.43:0/1750100348 conn(0x7f0fd40ccd80 msgr2=0x7f0fd40cc5a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-09T13:47:55.203 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:55.199+0000 7f0fdac06640 1 -- 172.21.15.43:0/1750100348 shutdown_connections 2023-08-09T13:47:55.203 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:55.200+0000 7f0fdac06640 1 -- 172.21.15.43:0/1750100348 wait complete. 2023-08-09T13:47:55.218 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-08-09T13:47:55.224 INFO:tasks.ceph.mgr.y.smithi043.stderr:2023-08-09T13:47:55.223+0000 7fac17030040 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2023-08-09T13:47:55.235 INFO:tasks.ceph.mgr.x.smithi203.stderr:2023-08-09T13:47:55.234+0000 7fb1cbc55040 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2023-08-09T13:47:55.350 INFO:tasks.ceph.mgr.y.smithi043.stderr:2023-08-09T13:47:55.349+0000 7fac17030040 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2023-08-09T13:47:55.391 INFO:tasks.ceph.mgr.x.smithi203.stderr:2023-08-09T13:47:55.389+0000 7fb1cbc55040 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2023-08-09T13:47:55.440 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:55.438+0000 7f4c4ac7f640 1 Processor -- start 2023-08-09T13:47:55.440 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:55.438+0000 7f4c4ac7f640 1 -- start start 2023-08-09T13:47:55.441 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:55.440+0000 7f4c4ac7f640 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7f4c440ce310 0x7f4c4415b050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:47:55.442 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:55.440+0000 7f4c4ac7f640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f4c440ceb20 0x7f4c440cef20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:47:55.442 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:55.440+0000 7f4c4ac7f640 1 -- --> v1:172.21.15.43:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4c440cf460 con 0x7f4c4415e260 2023-08-09T13:47:55.442 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:55.440+0000 7f4c4ac7f640 1 -- --> v2:172.21.15.43:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4c4415b590 con 0x7f4c440ceb20 2023-08-09T13:47:55.442 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:55.440+0000 7f4c4ac7f640 1 -- --> v2:172.21.15.203:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4c440cd270 con 0x7f4c440ce310 2023-08-09T13:47:55.442 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:55.440+0000 7f4c49c7d640 1 --1- >> v1:172.21.15.43:6789/0 conn(0x7f4c4415e260 0x7f4c4415e640 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.43:6789/0 says I am v1:172.21.15.43:36710/0 (socket says 172.21.15.43:36710) 2023-08-09T13:47:55.442 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:55.440+0000 7f4c49c7d640 1 -- 172.21.15.43:0/2819789300 learned_addr learned my addr 172.21.15.43:0/2819789300 (peer_addr_for_me v1:172.21.15.43:0/0) 2023-08-09T13:47:55.442 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:55.440+0000 7f4c4a47e640 1 --2- 172.21.15.43:0/2819789300 >> v2:172.21.15.43:3300/0 conn(0x7f4c440ceb20 0x7f4c440cef20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:47:55.443 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:55.441+0000 7f4c4947c640 1 --2- 172.21.15.43:0/2819789300 >> v2:172.21.15.203:3300/0 conn(0x7f4c440ce310 0x7f4c4415b050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:47:55.443 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:55.441+0000 7f4c48c7b640 1 -- 172.21.15.43:0/2819789300 <== mon.0 v1:172.21.15.43:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1448221199 0 0) 0x7f4c440cf460 con 0x7f4c4415e260 2023-08-09T13:47:55.443 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:55.441+0000 7f4c48c7b640 1 -- 172.21.15.43:0/2819789300 --> v1:172.21.15.43:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f4c30003520 con 0x7f4c4415e260 2023-08-09T13:47:55.444 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:55.441+0000 7f4c4a47e640 1 -- 172.21.15.43:0/2819789300 >> v2:172.21.15.203:3300/0 conn(0x7f4c440ce310 msgr2=0x7f4c4415b050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-09T13:47:55.444 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:55.441+0000 7f4c4a47e640 1 --2- 172.21.15.43:0/2819789300 >> v2:172.21.15.203:3300/0 conn(0x7f4c440ce310 0x7f4c4415b050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:47:55.444 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:55.441+0000 7f4c4a47e640 1 -- 172.21.15.43:0/2819789300 >> v1:172.21.15.43:6789/0 conn(0x7f4c4415e260 legacy=0x7f4c4415e640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:47:55.444 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:55.441+0000 7f4c4a47e640 1 -- 172.21.15.43:0/2819789300 --> v2:172.21.15.43:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4c440cd550 con 0x7f4c440ceb20 2023-08-09T13:47:55.444 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:55.442+0000 7f4c4a47e640 1 --2- 172.21.15.43:0/2819789300 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f4c440ceb20 0x7f4c440cef20 secure :-1 s=READY pgs=744 cs=0 l=1 rev1=1 crypto rx=0x7f4c44070e20 tx=0x7f4c3c02bed0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bc61d40303c7a04a server_cookie=0 in_seq=0 out_seq=0 2023-08-09T13:47:55.445 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:55.442+0000 7f4c48c7b640 1 -- 172.21.15.43:0/2819789300 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4c3c02cc80 con 0x7f4c440ceb20 2023-08-09T13:47:55.445 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:55.442+0000 7f4c48c7b640 1 -- 172.21.15.43:0/2819789300 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4c3c02cde0 con 0x7f4c440ceb20 2023-08-09T13:47:55.445 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:55.442+0000 7f4c48c7b640 1 -- 172.21.15.43:0/2819789300 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4c3c034640 con 0x7f4c440ceb20 2023-08-09T13:47:55.445 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:55.443+0000 7f4c4ac7f640 1 -- 172.21.15.43:0/2819789300 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f4c440ceb20 msgr2=0x7f4c440cef20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:47:55.446 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:55.443+0000 7f4c4ac7f640 1 --2- 172.21.15.43:0/2819789300 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f4c440ceb20 0x7f4c440cef20 secure :-1 s=READY pgs=744 cs=0 l=1 rev1=1 crypto rx=0x7f4c44070e20 tx=0x7f4c3c02bed0 comp rx=0 tx=0).stop 2023-08-09T13:47:55.446 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:55.443+0000 7f4c4ac7f640 1 -- 172.21.15.43:0/2819789300 shutdown_connections 2023-08-09T13:47:55.446 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:55.443+0000 7f4c4ac7f640 1 --2- 172.21.15.43:0/2819789300 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f4c440ceb20 0x7f4c440cef20 unknown :-1 s=CLOSED pgs=744 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:47:55.446 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:55.443+0000 7f4c4ac7f640 1 --2- 172.21.15.43:0/2819789300 >> v2:172.21.15.203:3300/0 conn(0x7f4c440ce310 0x7f4c4415b050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:47:55.446 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:55.443+0000 7f4c4ac7f640 1 -- 172.21.15.43:0/2819789300 >> 172.21.15.43:0/2819789300 conn(0x7f4c44069be0 msgr2=0x7f4c44069ff0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-09T13:47:55.446 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:55.443+0000 7f4c4ac7f640 1 -- 172.21.15.43:0/2819789300 shutdown_connections 2023-08-09T13:47:55.446 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:55.443+0000 7f4c4ac7f640 1 -- 172.21.15.43:0/2819789300 wait complete. 2023-08-09T13:47:55.447 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:55.445+0000 7f4c4ac7f640 1 Processor -- start 2023-08-09T13:47:55.447 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:55.445+0000 7f4c4ac7f640 1 -- start start 2023-08-09T13:47:55.448 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:55.446+0000 7f4c4ac7f640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f4c440ceb20 0x7f4c441436f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:47:55.448 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:55.446+0000 7f4c4ac7f640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f4c4415e260 0x7f4c44154400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:47:55.448 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:55.446+0000 7f4c4ac7f640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f4c4413b0f0 0x7f4c440b2540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:47:55.448 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:55.446+0000 7f4c4ac7f640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4c44155760 con 0x7f4c4415e260 2023-08-09T13:47:55.448 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:55.446+0000 7f4c4ac7f640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4c44156630 con 0x7f4c440ceb20 2023-08-09T13:47:55.448 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:55.446+0000 7f4c4ac7f640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4c4413b580 con 0x7f4c4413b0f0 2023-08-09T13:47:55.448 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:55.446+0000 7f4c4947c640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f4c4415e260 0x7f4c44154400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:47:55.449 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:55.446+0000 7f4c4947c640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f4c4415e260 0x7f4c44154400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:57208/0 (socket says 172.21.15.43:57208) 2023-08-09T13:47:55.449 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:55.446+0000 7f4c4947c640 1 -- 172.21.15.43:0/508843393 learned_addr learned my addr 172.21.15.43:0/508843393 (peer_addr_for_me v2:172.21.15.43:0/0) 2023-08-09T13:47:55.450 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:55.447+0000 7f4c49c7d640 1 --2- 172.21.15.43:0/508843393 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f4c440ceb20 0x7f4c441436f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:47:55.450 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:55.447+0000 7f4c4a47e640 1 --2- 172.21.15.43:0/508843393 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f4c4413b0f0 0x7f4c440b2540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:47:55.450 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:55.447+0000 7f4c4947c640 1 -- 172.21.15.43:0/508843393 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f4c4413b0f0 msgr2=0x7f4c440b2540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:47:55.450 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:55.447+0000 7f4c4947c640 1 --2- 172.21.15.43:0/508843393 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f4c4413b0f0 0x7f4c440b2540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:47:55.450 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:55.447+0000 7f4c4947c640 1 -- 172.21.15.43:0/508843393 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f4c440ceb20 msgr2=0x7f4c441436f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:47:55.451 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:55.447+0000 7f4c4947c640 1 --2- 172.21.15.43:0/508843393 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f4c440ceb20 0x7f4c441436f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:47:55.451 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:55.447+0000 7f4c4947c640 1 -- 172.21.15.43:0/508843393 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4c3c009a70 con 0x7f4c4415e260 2023-08-09T13:47:55.451 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:55.447+0000 7f4c4a47e640 1 --2- 172.21.15.43:0/508843393 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f4c4413b0f0 0x7f4c440b2540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-08-09T13:47:55.451 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:55.447+0000 7f4c4947c640 1 --2- 172.21.15.43:0/508843393 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f4c4415e260 0x7f4c44154400 secure :-1 s=READY pgs=745 cs=0 l=1 rev1=1 crypto rx=0x7f4c3400d830 tx=0x7f4c3400dcf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-09T13:47:55.451 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:55.447+0000 7f4c3a7fc640 1 -- 172.21.15.43:0/508843393 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4c3400b6c0 con 0x7f4c4415e260 2023-08-09T13:47:55.451 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:55.447+0000 7f4c3a7fc640 1 -- 172.21.15.43:0/508843393 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4c3400b820 con 0x7f4c4415e260 2023-08-09T13:47:55.451 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:55.447+0000 7f4c3a7fc640 1 -- 172.21.15.43:0/508843393 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4c3400bbb0 con 0x7f4c4415e260 2023-08-09T13:47:55.451 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:55.448+0000 7f4c4ac7f640 1 -- 172.21.15.43:0/508843393 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4c44154ae0 con 0x7f4c4415e260 2023-08-09T13:47:55.452 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:55.448+0000 7f4c4ac7f640 1 -- 172.21.15.43:0/508843393 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4c441550a0 con 0x7f4c4415e260 2023-08-09T13:47:55.452 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:55.449+0000 7f4c4ac7f640 1 -- 172.21.15.43:0/508843393 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4c4404e390 con 0x7f4c4415e260 2023-08-09T13:47:55.452 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:55.451+0000 7f4c3a7fc640 1 -- 172.21.15.43:0/508843393 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 14) v1 ==== 88512+0+0 (secure 0 0 0) 0x7f4c3401e460 con 0x7f4c4415e260 2023-08-09T13:47:55.453 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:55.452+0000 7f4c3a7fc640 1 --2- 172.21.15.43:0/508843393 >> [v2:172.21.15.203:6832/998005540,v1:172.21.15.203:6833/998005540] conn(0x7f4c2806a230 0x7f4c2806c6f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:47:55.453 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:55.452+0000 7f4c3a7fc640 1 -- 172.21.15.43:0/508843393 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(60..60 src has 1..60) v4 ==== 7298+0+0 (secure 0 0 0) 0x7f4c340894d0 con 0x7f4c4415e260 2023-08-09T13:47:55.453 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:55.452+0000 7f4c49c7d640 1 -- 172.21.15.43:0/508843393 >> [v2:172.21.15.203:6832/998005540,v1:172.21.15.203:6833/998005540] conn(0x7f4c2806a230 msgr2=0x7f4c2806c6f0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.203:6832/998005540 2023-08-09T13:47:55.454 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:55.452+0000 7f4c49c7d640 1 --2- 172.21.15.43:0/508843393 >> [v2:172.21.15.203:6832/998005540,v1:172.21.15.203:6833/998005540] conn(0x7f4c2806a230 0x7f4c2806c6f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-08-09T13:47:55.457 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:55.456+0000 7f4c3a7fc640 1 -- 172.21.15.43:0/508843393 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178066 (secure 0 0 0) 0x7f4c340544d0 con 0x7f4c4415e260 2023-08-09T13:47:55.499 INFO:tasks.ceph.mgr.x.smithi203.stderr:2023-08-09T13:47:55.498+0000 7fb1cbc55040 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2023-08-09T13:47:55.520 INFO:tasks.ceph.mgr.y.smithi043.stderr:2023-08-09T13:47:55.518+0000 7fac17030040 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2023-08-09T13:47:55.548 INFO:tasks.ceph.mgr.x.smithi203.stderr:2023-08-09T13:47:55.547+0000 7fb1cbc55040 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2023-08-09T13:47:55.634 INFO:tasks.ceph.mgr.y.smithi043.stderr:2023-08-09T13:47:55.632+0000 7fac17030040 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2023-08-09T13:47:55.653 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:55.650+0000 7f4c4ac7f640 1 -- 172.21.15.43:0/508843393 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f4c4406bd30 con 0x7f4c4415e260 2023-08-09T13:47:55.653 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:55.651+0000 7f4c3a7fc640 1 -- 172.21.15.43:0/508843393 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v60) v1 ==== 100+0+11 (secure 0 0 0) 0x7f4c34057af0 con 0x7f4c4415e260 2023-08-09T13:47:55.654 INFO:teuthology.orchestra.run.smithi043.stdout:pg_num: 64 2023-08-09T13:47:55.654 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:55.652+0000 7f4c49c7d640 1 -- 172.21.15.43:0/508843393 >> [v2:172.21.15.203:6832/998005540,v1:172.21.15.203:6833/998005540] conn(0x7f4c2806a230 msgr2=0x7f4c2806c6f0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.203:6832/998005540 2023-08-09T13:47:55.655 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:55.652+0000 7f4c49c7d640 1 --2- 172.21.15.43:0/508843393 >> [v2:172.21.15.203:6832/998005540,v1:172.21.15.203:6833/998005540] conn(0x7f4c2806a230 0x7f4c2806c6f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2023-08-09T13:47:55.656 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:55.653+0000 7f4c4ac7f640 1 -- 172.21.15.43:0/508843393 >> [v2:172.21.15.203:6832/998005540,v1:172.21.15.203:6833/998005540] conn(0x7f4c2806a230 msgr2=0x7f4c2806c6f0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-08-09T13:47:55.656 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:55.653+0000 7f4c4ac7f640 1 --2- 172.21.15.43:0/508843393 >> [v2:172.21.15.203:6832/998005540,v1:172.21.15.203:6833/998005540] conn(0x7f4c2806a230 0x7f4c2806c6f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:47:55.656 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:55.653+0000 7f4c4ac7f640 1 -- 172.21.15.43:0/508843393 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f4c4415e260 msgr2=0x7f4c44154400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:47:55.657 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:55.653+0000 7f4c4ac7f640 1 --2- 172.21.15.43:0/508843393 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f4c4415e260 0x7f4c44154400 secure :-1 s=READY pgs=745 cs=0 l=1 rev1=1 crypto rx=0x7f4c3400d830 tx=0x7f4c3400dcf0 comp rx=0 tx=0).stop 2023-08-09T13:47:55.657 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:55.654+0000 7f4c4ac7f640 1 -- 172.21.15.43:0/508843393 shutdown_connections 2023-08-09T13:47:55.657 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:55.654+0000 7f4c4ac7f640 1 --2- 172.21.15.43:0/508843393 >> [v2:172.21.15.203:6832/998005540,v1:172.21.15.203:6833/998005540] conn(0x7f4c2806a230 0x7f4c2806c6f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:47:55.657 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:55.654+0000 7f4c4ac7f640 1 --2- 172.21.15.43:0/508843393 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f4c4413b0f0 0x7f4c440b2540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:47:55.657 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:55.654+0000 7f4c4ac7f640 1 --2- 172.21.15.43:0/508843393 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f4c4415e260 0x7f4c44154400 unknown :-1 s=CLOSED pgs=745 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:47:55.657 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:55.654+0000 7f4c4ac7f640 1 --2- 172.21.15.43:0/508843393 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f4c440ceb20 0x7f4c441436f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:47:55.657 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:55.654+0000 7f4c4ac7f640 1 -- 172.21.15.43:0/508843393 >> 172.21.15.43:0/508843393 conn(0x7f4c44069be0 msgr2=0x7f4c441576c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-09T13:47:55.658 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:55.654+0000 7f4c4ac7f640 1 -- 172.21.15.43:0/508843393 shutdown_connections 2023-08-09T13:47:55.658 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:55.654+0000 7f4c4ac7f640 1 -- 172.21.15.43:0/508843393 wait complete. 2023-08-09T13:47:55.673 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2023-08-09T13:47:55.687 INFO:tasks.ceph.mgr.y.smithi043.stderr:2023-08-09T13:47:55.685+0000 7fac17030040 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2023-08-09T13:47:55.706 INFO:tasks.ceph.mgr.x.smithi203.stderr:2023-08-09T13:47:55.704+0000 7fb1cbc55040 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2023-08-09T13:47:55.863 INFO:tasks.ceph.mgr.y.smithi043.stderr:2023-08-09T13:47:55.860+0000 7fac17030040 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2023-08-09T13:47:55.913 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:55.911+0000 7fbabd7c8640 1 Processor -- start 2023-08-09T13:47:55.913 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:55.911+0000 7fbabd7c8640 1 -- start start 2023-08-09T13:47:55.914 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:55.912+0000 7fbabd7c8640 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7fbab806ed20 0x7fbab815cb20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:47:55.914 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:55.912+0000 7fbabd7c8640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fbab80697c0 0x7fbab806dbc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:47:55.915 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:55.912+0000 7fbabd7c8640 1 -- --> v1:172.21.15.203:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbab815d060 con 0x7fbab80ccdd0 2023-08-09T13:47:55.915 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:55.912+0000 7fbabd7c8640 1 -- --> v2:172.21.15.203:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbab815db30 con 0x7fbab806ed20 2023-08-09T13:47:55.915 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:55.912+0000 7fbabd7c8640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbab8070910 con 0x7fbab80697c0 2023-08-09T13:47:55.915 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:55.913+0000 7fbab6ffd640 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7fbab806ed20 0x7fbab815cb20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:47:55.915 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:55.913+0000 7fbaae7fc640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fbab80697c0 0x7fbab806dbc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:47:55.916 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:55.913+0000 7fbab6ffd640 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7fbab806ed20 0x7fbab815cb20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.203:3300/0 says I am v2:172.21.15.43:47938/0 (socket says 172.21.15.43:47938) 2023-08-09T13:47:55.916 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:55.913+0000 7fbab6ffd640 1 -- 172.21.15.43:0/1516338099 learned_addr learned my addr 172.21.15.43:0/1516338099 (peer_addr_for_me v2:172.21.15.43:0/0) 2023-08-09T13:47:55.916 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:55.914+0000 7fbaae7fc640 1 -- 172.21.15.43:0/1516338099 >> v2:172.21.15.203:3300/0 conn(0x7fbab806ed20 msgr2=0x7fbab815cb20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-09T13:47:55.917 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:55.914+0000 7fbaae7fc640 1 --2- 172.21.15.43:0/1516338099 >> v2:172.21.15.203:3300/0 conn(0x7fbab806ed20 0x7fbab815cb20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:47:55.917 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:55.914+0000 7fbaae7fc640 1 -- 172.21.15.43:0/1516338099 >> v1:172.21.15.203:6789/0 conn(0x7fbab80ccdd0 legacy=0x7fbab8070da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:47:55.917 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:55.914+0000 7fbaae7fc640 1 -- 172.21.15.43:0/1516338099 --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbab806f100 con 0x7fbab80697c0 2023-08-09T13:47:55.917 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:55.915+0000 7fbaae7fc640 1 --2- 172.21.15.43:0/1516338099 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fbab80697c0 0x7fbab806dbc0 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7fbab806f290 tx=0x7fbaa002b810 comp rx=0 tx=0).ready entity=mon.2 client_cookie=86ab1cb67d2a05ef server_cookie=0 in_seq=0 out_seq=0 2023-08-09T13:47:55.918 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:55.915+0000 7fbab67fc640 1 -- 172.21.15.43:0/1516338099 <== mon.2 v2:172.21.15.203:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbaa0031770 con 0x7fbab80697c0 2023-08-09T13:47:55.918 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:55.915+0000 7fbab67fc640 1 -- 172.21.15.43:0/1516338099 <== mon.2 v2:172.21.15.203:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbaa00318d0 con 0x7fbab80697c0 2023-08-09T13:47:55.918 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:55.915+0000 7fbab67fc640 1 -- 172.21.15.43:0/1516338099 <== mon.2 v2:172.21.15.203:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbaa0031c60 con 0x7fbab80697c0 2023-08-09T13:47:55.919 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:55.916+0000 7fbabd7c8640 1 -- 172.21.15.43:0/1516338099 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fbab80697c0 msgr2=0x7fbab806dbc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:47:55.919 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:55.916+0000 7fbabd7c8640 1 --2- 172.21.15.43:0/1516338099 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fbab80697c0 0x7fbab806dbc0 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7fbab806f290 tx=0x7fbaa002b810 comp rx=0 tx=0).stop 2023-08-09T13:47:55.919 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:55.916+0000 7fbabd7c8640 1 -- 172.21.15.43:0/1516338099 shutdown_connections 2023-08-09T13:47:55.919 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:55.916+0000 7fbabd7c8640 1 --2- 172.21.15.43:0/1516338099 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fbab80697c0 0x7fbab806dbc0 unknown :-1 s=CLOSED pgs=95 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:47:55.919 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:55.916+0000 7fbabd7c8640 1 --2- 172.21.15.43:0/1516338099 >> v2:172.21.15.203:3300/0 conn(0x7fbab806ed20 0x7fbab815cb20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:47:55.919 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:55.916+0000 7fbabd7c8640 1 -- 172.21.15.43:0/1516338099 >> 172.21.15.43:0/1516338099 conn(0x7fbab80cbc80 msgr2=0x7fbab80cc090 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-09T13:47:55.919 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:55.916+0000 7fbabd7c8640 1 -- 172.21.15.43:0/1516338099 shutdown_connections 2023-08-09T13:47:55.920 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:55.916+0000 7fbabd7c8640 1 -- 172.21.15.43:0/1516338099 wait complete. 2023-08-09T13:47:55.920 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:55.918+0000 7fbabd7c8640 1 Processor -- start 2023-08-09T13:47:55.920 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:55.918+0000 7fbabd7c8640 1 -- start start 2023-08-09T13:47:55.921 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:55.919+0000 7fbabd7c8640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fbab80697c0 0x7fbab8158510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:47:55.921 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:55.919+0000 7fbabd7c8640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fbab806ed20 0x7fbab80aca70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:47:55.922 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:55.919+0000 7fbabd7c8640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fbab80ccdd0 0x7fbab81f1610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:47:55.922 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:55.919+0000 7fbabd7c8640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbab80acfb0 con 0x7fbab806ed20 2023-08-09T13:47:55.922 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:55.919+0000 7fbabd7c8640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbab80ad280 con 0x7fbab80ccdd0 2023-08-09T13:47:55.922 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:55.919+0000 7fbabd7c8640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbab80ad560 con 0x7fbab80697c0 2023-08-09T13:47:55.923 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:55.919+0000 7fbaae7fc640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fbab806ed20 0x7fbab80aca70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:47:55.923 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:55.919+0000 7fbaae7fc640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fbab806ed20 0x7fbab80aca70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:57218/0 (socket says 172.21.15.43:57218) 2023-08-09T13:47:55.923 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:55.919+0000 7fbaae7fc640 1 -- 172.21.15.43:0/2878247747 learned_addr learned my addr 172.21.15.43:0/2878247747 (peer_addr_for_me v2:172.21.15.43:0/0) 2023-08-09T13:47:55.924 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:55.920+0000 7fbab6ffd640 1 --2- 172.21.15.43:0/2878247747 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fbab80697c0 0x7fbab8158510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:47:55.924 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:55.920+0000 7fbab77fe640 1 --2- 172.21.15.43:0/2878247747 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fbab80ccdd0 0x7fbab81f1610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:47:55.924 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:55.920+0000 7fbaae7fc640 1 -- 172.21.15.43:0/2878247747 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fbab80697c0 msgr2=0x7fbab8158510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:47:55.924 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:55.920+0000 7fbaae7fc640 1 --2- 172.21.15.43:0/2878247747 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fbab80697c0 0x7fbab8158510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:47:55.924 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:55.920+0000 7fbaae7fc640 1 -- 172.21.15.43:0/2878247747 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fbab80ccdd0 msgr2=0x7fbab81f1610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:47:55.924 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:55.920+0000 7fbaae7fc640 1 --2- 172.21.15.43:0/2878247747 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fbab80ccdd0 0x7fbab81f1610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:47:55.924 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:55.920+0000 7fbaae7fc640 1 -- 172.21.15.43:0/2878247747 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbaa0009400 con 0x7fbab806ed20 2023-08-09T13:47:55.924 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:55.920+0000 7fbaae7fc640 1 --2- 172.21.15.43:0/2878247747 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fbab806ed20 0x7fbab80aca70 secure :-1 s=READY pgs=746 cs=0 l=1 rev1=1 crypto rx=0x7fbaa0009890 tx=0x7fbaa00046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-09T13:47:55.925 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:55.921+0000 7fbaaffff640 1 -- 172.21.15.43:0/2878247747 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbaa0032810 con 0x7fbab806ed20 2023-08-09T13:47:55.925 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:55.921+0000 7fbaaffff640 1 -- 172.21.15.43:0/2878247747 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbaa0032990 con 0x7fbab806ed20 2023-08-09T13:47:55.925 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:55.921+0000 7fbabd7c8640 1 -- 172.21.15.43:0/2878247747 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbab80ad840 con 0x7fbab806ed20 2023-08-09T13:47:55.925 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:55.921+0000 7fbaaffff640 1 -- 172.21.15.43:0/2878247747 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbaa0032d40 con 0x7fbab806ed20 2023-08-09T13:47:55.925 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:55.921+0000 7fbabd7c8640 1 -- 172.21.15.43:0/2878247747 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbab80ade00 con 0x7fbab806ed20 2023-08-09T13:47:55.926 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:55.922+0000 7fbabd7c8640 1 -- 172.21.15.43:0/2878247747 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbab80adff0 con 0x7fbab806ed20 2023-08-09T13:47:55.926 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:55.923+0000 7fbaaffff640 1 -- 172.21.15.43:0/2878247747 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 14) v1 ==== 88512+0+0 (secure 0 0 0) 0x7fbaa004d3f0 con 0x7fbab806ed20 2023-08-09T13:47:55.927 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:55.924+0000 7fbaaffff640 1 --2- 172.21.15.43:0/2878247747 >> [v2:172.21.15.203:6832/998005540,v1:172.21.15.203:6833/998005540] conn(0x7fba9806a200 0x7fba9806c6c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:47:55.927 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:55.924+0000 7fbaaffff640 1 -- 172.21.15.43:0/2878247747 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(60..60 src has 1..60) v4 ==== 7298+0+0 (secure 0 0 0) 0x7fbaa00aa960 con 0x7fbab806ed20 2023-08-09T13:47:55.927 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:55.924+0000 7fbab6ffd640 1 -- 172.21.15.43:0/2878247747 >> [v2:172.21.15.203:6832/998005540,v1:172.21.15.203:6833/998005540] conn(0x7fba9806a200 msgr2=0x7fba9806c6c0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.203:6832/998005540 2023-08-09T13:47:55.927 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:55.925+0000 7fbab6ffd640 1 --2- 172.21.15.43:0/2878247747 >> [v2:172.21.15.203:6832/998005540,v1:172.21.15.203:6833/998005540] conn(0x7fba9806a200 0x7fba9806c6c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-08-09T13:47:55.935 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:55.933+0000 7fbaaffff640 1 -- 172.21.15.43:0/2878247747 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178066 (secure 0 0 0) 0x7fbaa00757d0 con 0x7fbab806ed20 2023-08-09T13:47:56.096 INFO:tasks.ceph.mgr.x.smithi203.stderr:2023-08-09T13:47:56.095+0000 7fb1cbc55040 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2023-08-09T13:47:56.128 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:56.126+0000 7fbab6ffd640 1 -- 172.21.15.43:0/2878247747 >> [v2:172.21.15.203:6832/998005540,v1:172.21.15.203:6833/998005540] conn(0x7fba9806a200 msgr2=0x7fba9806c6c0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.203:6832/998005540 2023-08-09T13:47:56.129 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:56.126+0000 7fbab6ffd640 1 --2- 172.21.15.43:0/2878247747 >> [v2:172.21.15.203:6832/998005540,v1:172.21.15.203:6833/998005540] conn(0x7fba9806a200 0x7fba9806c6c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2023-08-09T13:47:56.146 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:56.143+0000 7fbabd7c8640 1 -- 172.21.15.43:0/2878247747 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7fbab80cefe0 con 0x7fbab806ed20 2023-08-09T13:47:56.146 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:56.144+0000 7fbaaffff640 1 -- 172.21.15.43:0/2878247747 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 6 entries v60) v1 ==== 80+0+357 (secure 0 0 0) 0x7fbaa0078df0 con 0x7fbab806ed20 2023-08-09T13:47:56.147 INFO:teuthology.orchestra.run.smithi043.stdout:172.21.15.203:0/890377927 2023-08-10T13:36:27.970754+0000 2023-08-09T13:47:56.147 INFO:teuthology.orchestra.run.smithi043.stdout:172.21.15.203:0/1761968483 2023-08-10T13:36:27.970754+0000 2023-08-09T13:47:56.147 INFO:teuthology.orchestra.run.smithi043.stdout:172.21.15.203:0/2439593041 2023-08-10T13:36:27.970754+0000 2023-08-09T13:47:56.147 INFO:teuthology.orchestra.run.smithi043.stdout:172.21.15.203:6833/830097474 2023-08-10T13:36:27.970754+0000 2023-08-09T13:47:56.147 INFO:teuthology.orchestra.run.smithi043.stdout:172.21.15.203:6832/830097474 2023-08-10T13:36:27.970754+0000 2023-08-09T13:47:56.147 INFO:teuthology.orchestra.run.smithi043.stdout:172.21.15.203:0/3309257000 2023-08-10T13:36:27.970754+0000 2023-08-09T13:47:56.148 INFO:teuthology.orchestra.run.smithi043.stderr:listed 6 entries 2023-08-09T13:47:56.149 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:56.146+0000 7fbabd7c8640 1 -- 172.21.15.43:0/2878247747 >> [v2:172.21.15.203:6832/998005540,v1:172.21.15.203:6833/998005540] conn(0x7fba9806a200 msgr2=0x7fba9806c6c0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-08-09T13:47:56.149 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:56.146+0000 7fbabd7c8640 1 --2- 172.21.15.43:0/2878247747 >> [v2:172.21.15.203:6832/998005540,v1:172.21.15.203:6833/998005540] conn(0x7fba9806a200 0x7fba9806c6c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:47:56.149 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:56.146+0000 7fbabd7c8640 1 -- 172.21.15.43:0/2878247747 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fbab806ed20 msgr2=0x7fbab80aca70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:47:56.149 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:56.146+0000 7fbabd7c8640 1 --2- 172.21.15.43:0/2878247747 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fbab806ed20 0x7fbab80aca70 secure :-1 s=READY pgs=746 cs=0 l=1 rev1=1 crypto rx=0x7fbaa0009890 tx=0x7fbaa00046c0 comp rx=0 tx=0).stop 2023-08-09T13:47:56.150 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:56.146+0000 7fbabd7c8640 1 -- 172.21.15.43:0/2878247747 shutdown_connections 2023-08-09T13:47:56.150 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:56.146+0000 7fbabd7c8640 1 --2- 172.21.15.43:0/2878247747 >> [v2:172.21.15.203:6832/998005540,v1:172.21.15.203:6833/998005540] conn(0x7fba9806a200 0x7fba9806c6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:47:56.150 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:56.146+0000 7fbabd7c8640 1 --2- 172.21.15.43:0/2878247747 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fbab80ccdd0 0x7fbab81f1610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:47:56.150 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:56.146+0000 7fbabd7c8640 1 --2- 172.21.15.43:0/2878247747 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fbab806ed20 0x7fbab80aca70 unknown :-1 s=CLOSED pgs=746 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:47:56.150 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:56.146+0000 7fbabd7c8640 1 --2- 172.21.15.43:0/2878247747 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fbab80697c0 0x7fbab8158510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:47:56.150 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:56.146+0000 7fbabd7c8640 1 -- 172.21.15.43:0/2878247747 >> 172.21.15.43:0/2878247747 conn(0x7fbab80cbc80 msgr2=0x7fbab815e810 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-09T13:47:56.150 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:56.147+0000 7fbabd7c8640 1 -- 172.21.15.43:0/2878247747 shutdown_connections 2023-08-09T13:47:56.151 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:56.147+0000 7fbabd7c8640 1 -- 172.21.15.43:0/2878247747 wait complete. 2023-08-09T13:47:56.168 INFO:tasks.cephfs.fuse_mount:Running fusermount -u on ubuntu@smithi043.front.sepia.ceph.com... 2023-08-09T13:47:56.168 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-09T13:47:56.169 DEBUG:teuthology.orchestra.run.smithi043:> sudo fusermount -u /home/ubuntu/cephtest/mnt.0 2023-08-09T13:47:56.251 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi043.stderr:ceph-fuse[105196]: fuse finished with error 0 and tester_r 0 2023-08-09T13:47:56.256 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi043.front.sepia.ceph.com 2023-08-09T13:47:56.256 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-09T13:47:56.257 DEBUG:teuthology.orchestra.run.smithi043:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2023-08-09T13:47:56.280 INFO:tasks.ceph.mgr.y.smithi043.stderr:2023-08-09T13:47:56.277+0000 7fac17030040 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2023-08-09T13:47:56.317 INFO:tasks.ceph.mgr.x.smithi203.stderr:2023-08-09T13:47:56.316+0000 7fb1cbc55040 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2023-08-09T13:47:56.373 INFO:tasks.ceph.mgr.x.smithi203.stderr:2023-08-09T13:47:56.372+0000 7fb1cbc55040 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2023-08-09T13:47:56.423 INFO:tasks.ceph.mgr.x.smithi203.stderr:2023-08-09T13:47:56.422+0000 7fb1cbc55040 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2023-08-09T13:47:56.436 DEBUG:teuthology.orchestra.run.smithi203:> set -ex 2023-08-09T13:47:56.437 DEBUG:teuthology.orchestra.run.smithi203:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-09T13:47:56.478 INFO:tasks.ceph.mgr.x.smithi203.stderr:2023-08-09T13:47:56.476+0000 7fb1cbc55040 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2023-08-09T13:47:56.492 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4053648,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497576k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=27,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248784k,nr_inodes=812196,mode=700,uid=1000,gid=1275 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497576k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2023-08-09T13:47:56.493 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi203.front.sepia.ceph.com 2023-08-09T13:47:56.493 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-09T13:47:56.493 DEBUG:teuthology.orchestra.run.smithi203:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.1) 2023-08-09T13:47:56.518 INFO:tasks.ceph.mgr.y.smithi043.stderr:2023-08-09T13:47:56.516+0000 7fac17030040 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2023-08-09T13:47:56.578 INFO:tasks.ceph.mgr.y.smithi043.stderr:2023-08-09T13:47:56.575+0000 7fac17030040 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2023-08-09T13:47:56.586 INFO:teuthology.orchestra.run.smithi203.stderr:rmdir: failed to remove '/home/ubuntu/cephtest/mnt.1': No such file or directory 2023-08-09T13:47:56.587 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-08-09T13:47:56.588 DEBUG:teuthology.orchestra.run.smithi043:> set -ex 2023-08-09T13:47:56.588 DEBUG:teuthology.orchestra.run.smithi043:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-09T13:47:56.630 INFO:tasks.ceph.mgr.y.smithi043.stderr:2023-08-09T13:47:56.628+0000 7fac17030040 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2023-08-09T13:47:56.645 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4054441,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6498844k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=27,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3249420k,nr_inodes=812355,mode=700,uid=1000,gid=1275 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6498844k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2023-08-09T13:47:56.645 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi043.front.sepia.ceph.com 2023-08-09T13:47:56.646 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-09T13:47:56.646 DEBUG:teuthology.orchestra.run.smithi043:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.2) 2023-08-09T13:47:56.685 INFO:tasks.ceph.mgr.y.smithi043.stderr:2023-08-09T13:47:56.683+0000 7fac17030040 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2023-08-09T13:47:56.739 INFO:teuthology.orchestra.run.smithi043.stderr:rmdir: failed to remove '/home/ubuntu/cephtest/mnt.2': No such file or directory 2023-08-09T13:47:56.739 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-08-09T13:47:56.740 DEBUG:teuthology.orchestra.run.smithi203:> set -ex 2023-08-09T13:47:56.741 DEBUG:teuthology.orchestra.run.smithi203:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-09T13:47:56.781 INFO:tasks.ceph.mgr.x.smithi203.stderr:2023-08-09T13:47:56.779+0000 7fb1cbc55040 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2023-08-09T13:47:56.796 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4053648,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497576k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=27,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248784k,nr_inodes=812196,mode=700,uid=1000,gid=1275 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497576k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2023-08-09T13:47:56.796 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi203.front.sepia.ceph.com 2023-08-09T13:47:56.797 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-09T13:47:56.797 DEBUG:teuthology.orchestra.run.smithi203:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.3) 2023-08-09T13:47:56.845 INFO:tasks.ceph.mgr.x.smithi203.stderr:2023-08-09T13:47:56.839+0000 7fb1cbc55040 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2023-08-09T13:47:56.890 INFO:teuthology.orchestra.run.smithi203.stderr:rmdir: failed to remove '/home/ubuntu/cephtest/mnt.3': No such file or directory 2023-08-09T13:47:56.891 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-08-09T13:47:56.892 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-08-09T13:47:56.895 INFO:tasks.ceph.mgr.x.smithi203.stderr:2023-08-09T13:47:56.894+0000 7fb1cbc55040 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2023-08-09T13:47:56.994 INFO:tasks.ceph.mgr.y.smithi043.stderr:2023-08-09T13:47:56.992+0000 7fac17030040 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2023-08-09T13:47:57.056 INFO:tasks.ceph.mgr.x.smithi203.stderr:2023-08-09T13:47:57.055+0000 7fb1cbc55040 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2023-08-09T13:47:57.057 INFO:tasks.ceph.mgr.y.smithi043.stderr:2023-08-09T13:47:57.054+0000 7fac17030040 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2023-08-09T13:47:57.107 INFO:tasks.ceph.mgr.x.smithi203.stderr:2023-08-09T13:47:57.106+0000 7fb1cbc55040 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2023-08-09T13:47:57.108 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:57.105+0000 7f22ebcba640 1 Processor -- start 2023-08-09T13:47:57.108 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:57.105+0000 7f22ebcba640 1 -- start start 2023-08-09T13:47:57.109 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:57.107+0000 7f22ebcba640 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7f22e40b2b70 0x7f22e40d2ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:47:57.109 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:57.107+0000 7f22ebcba640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f22e40b3050 0x7f22e40b3f60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:47:57.109 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:57.107+0000 7f22ebcba640 1 -- --> v1:172.21.15.203:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f22e40ce090 con 0x7f22e40b2790 2023-08-09T13:47:57.110 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:57.107+0000 7f22ebcba640 1 -- --> v2:172.21.15.43:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f22e40b3950 con 0x7f22e40b3050 2023-08-09T13:47:57.110 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:57.107+0000 7f22ebcba640 1 -- --> v2:172.21.15.203:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f22e40cd7f0 con 0x7f22e40b2b70 2023-08-09T13:47:57.110 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:57.107+0000 7f22ea257640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f22e40b3050 0x7f22e40b3f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:47:57.111 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:57.107+0000 7f22ea257640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f22e40b3050 0x7f22e40b3f60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:57220/0 (socket says 172.21.15.43:57220) 2023-08-09T13:47:57.111 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:57.107+0000 7f22ea257640 1 -- 172.21.15.43:0/573148104 learned_addr learned my addr 172.21.15.43:0/573148104 (peer_addr_for_me v2:172.21.15.43:0/0) 2023-08-09T13:47:57.111 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:57.107+0000 7f22e9255640 1 --2- 172.21.15.43:0/573148104 >> v2:172.21.15.203:3300/0 conn(0x7f22e40b2b70 0x7f22e40d2ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:47:57.111 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:57.108+0000 7f22ea257640 1 -- 172.21.15.43:0/573148104 >> v2:172.21.15.203:3300/0 conn(0x7f22e40b2b70 msgr2=0x7f22e40d2ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-09T13:47:57.111 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:57.108+0000 7f22ea257640 1 --2- 172.21.15.43:0/573148104 >> v2:172.21.15.203:3300/0 conn(0x7f22e40b2b70 0x7f22e40d2ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:47:57.112 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:57.108+0000 7f22ea257640 1 -- 172.21.15.43:0/573148104 >> v1:172.21.15.203:6789/0 conn(0x7f22e40b2790 legacy=0x7f22e40b5330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-09T13:47:57.112 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:57.108+0000 7f22ea257640 1 -- 172.21.15.43:0/573148104 --> v2:172.21.15.43:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f22e40cdad0 con 0x7f22e40b3050 2023-08-09T13:47:57.112 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:57.109+0000 7f22ea257640 1 --2- 172.21.15.43:0/573148104 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f22e40b3050 0x7f22e40b3f60 secure :-1 s=READY pgs=747 cs=0 l=1 rev1=1 crypto rx=0x7f22e4066af0 tx=0x7f22dc02bee0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=437a32a4093ddb38 server_cookie=0 in_seq=0 out_seq=0 2023-08-09T13:47:57.113 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:57.109+0000 7f22e8a54640 1 -- 172.21.15.43:0/573148104 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f22dc02ca80 con 0x7f22e40b3050 2023-08-09T13:47:57.113 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:57.110+0000 7f22e8a54640 1 -- 172.21.15.43:0/573148104 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f22dc02cbe0 con 0x7f22e40b3050 2023-08-09T13:47:57.113 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:57.110+0000 7f22e8a54640 1 -- 172.21.15.43:0/573148104 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f22dc0343f0 con 0x7f22e40b3050 2023-08-09T13:47:57.113 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:57.110+0000 7f22ebcba640 1 -- 172.21.15.43:0/573148104 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f22e40b3050 msgr2=0x7f22e40b3f60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:47:57.113 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:57.110+0000 7f22ebcba640 1 --2- 172.21.15.43:0/573148104 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f22e40b3050 0x7f22e40b3f60 secure :-1 s=READY pgs=747 cs=0 l=1 rev1=1 crypto rx=0x7f22e4066af0 tx=0x7f22dc02bee0 comp rx=0 tx=0).stop 2023-08-09T13:47:57.114 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:57.110+0000 7f22ebcba640 1 -- 172.21.15.43:0/573148104 shutdown_connections 2023-08-09T13:47:57.114 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:57.110+0000 7f22ebcba640 1 --2- 172.21.15.43:0/573148104 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f22e40b3050 0x7f22e40b3f60 unknown :-1 s=CLOSED pgs=747 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:47:57.114 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:57.110+0000 7f22ebcba640 1 --2- 172.21.15.43:0/573148104 >> v2:172.21.15.203:3300/0 conn(0x7f22e40b2b70 0x7f22e40d2ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:47:57.114 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:57.110+0000 7f22ebcba640 1 -- 172.21.15.43:0/573148104 >> 172.21.15.43:0/573148104 conn(0x7f22e409b7b0 msgr2=0x7f22e40d24e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-09T13:47:57.114 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:57.111+0000 7f22ebcba640 1 -- 172.21.15.43:0/573148104 shutdown_connections 2023-08-09T13:47:57.115 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:57.111+0000 7f22ebcba640 1 -- 172.21.15.43:0/573148104 wait complete. 2023-08-09T13:47:57.115 INFO:tasks.ceph.mgr.y.smithi043.stderr:2023-08-09T13:47:57.111+0000 7fac17030040 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2023-08-09T13:47:57.115 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:57.113+0000 7f22ebcba640 1 Processor -- start 2023-08-09T13:47:57.115 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:57.113+0000 7f22ebcba640 1 -- start start 2023-08-09T13:47:57.116 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:57.114+0000 7f22ebcba640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f22e40b2790 0x7f22e4148720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:47:57.116 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:57.114+0000 7f22ebcba640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f22e40b2b70 0x7f22e40c9c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:47:57.117 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:57.114+0000 7f22ebcba640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f22e40b3050 0x7f22e406b9b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:47:57.117 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:57.114+0000 7f22ebcba640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f22e4148c60 con 0x7f22e40b3050 2023-08-09T13:47:57.117 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:57.114+0000 7f22ebcba640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f22e41f2db0 con 0x7f22e40b2b70 2023-08-09T13:47:57.117 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:57.114+0000 7f22ebcba640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f22e41f21a0 con 0x7f22e40b2790 2023-08-09T13:47:57.118 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:57.114+0000 7f22ea257640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f22e40b3050 0x7f22e406b9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:47:57.119 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:57.114+0000 7f22ea257640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f22e40b3050 0x7f22e406b9b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:57234/0 (socket says 172.21.15.43:57234) 2023-08-09T13:47:57.119 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:57.114+0000 7f22ea257640 1 -- 172.21.15.43:0/2157349403 learned_addr learned my addr 172.21.15.43:0/2157349403 (peer_addr_for_me v2:172.21.15.43:0/0) 2023-08-09T13:47:57.119 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:57.114+0000 7f22e9255640 1 --2- 172.21.15.43:0/2157349403 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f22e40b2b70 0x7f22e40c9c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:47:57.119 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:57.115+0000 7f22ea257640 1 -- 172.21.15.43:0/2157349403 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f22e40b2790 msgr2=0x7f22e4148720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:47:57.119 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:57.115+0000 7f22e9a56640 1 --2- 172.21.15.43:0/2157349403 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f22e40b2790 0x7f22e4148720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:47:57.119 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:57.115+0000 7f22ea257640 1 --2- 172.21.15.43:0/2157349403 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f22e40b2790 0x7f22e4148720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:47:57.119 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:57.115+0000 7f22ea257640 1 -- 172.21.15.43:0/2157349403 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f22e40b2b70 msgr2=0x7f22e40c9c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:47:57.120 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:57.115+0000 7f22ea257640 1 --2- 172.21.15.43:0/2157349403 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f22e40b2b70 0x7f22e40c9c70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:47:57.120 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:57.115+0000 7f22ea257640 1 -- 172.21.15.43:0/2157349403 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f22dc009b30 con 0x7f22e40b3050 2023-08-09T13:47:57.120 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:57.115+0000 7f22e9a56640 1 --2- 172.21.15.43:0/2157349403 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f22e40b2790 0x7f22e4148720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-08-09T13:47:57.120 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:57.115+0000 7f22ea257640 1 --2- 172.21.15.43:0/2157349403 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f22e40b3050 0x7f22e406b9b0 secure :-1 s=READY pgs=748 cs=0 l=1 rev1=1 crypto rx=0x7f22dc036040 tx=0x7f22dc004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-09T13:47:57.120 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:57.115+0000 7f22da7fc640 1 -- 172.21.15.43:0/2157349403 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f22dc009510 con 0x7f22e40b3050 2023-08-09T13:47:57.120 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:57.115+0000 7f22da7fc640 1 -- 172.21.15.43:0/2157349403 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f22dc03e450 con 0x7f22e40b3050 2023-08-09T13:47:57.120 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:57.115+0000 7f22da7fc640 1 -- 172.21.15.43:0/2157349403 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f22dc03e800 con 0x7f22e40b3050 2023-08-09T13:47:57.121 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:57.115+0000 7f22ebcba640 1 -- 172.21.15.43:0/2157349403 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f22e41f2480 con 0x7f22e40b3050 2023-08-09T13:47:57.121 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:57.116+0000 7f22ebcba640 1 -- 172.21.15.43:0/2157349403 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f22e41f2a40 con 0x7f22e40b3050 2023-08-09T13:47:57.121 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:57.117+0000 7f22ebcba640 1 -- 172.21.15.43:0/2157349403 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f22e41f2f40 con 0x7f22e40b3050 2023-08-09T13:47:57.121 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:57.118+0000 7f22da7fc640 1 -- 172.21.15.43:0/2157349403 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 14) v1 ==== 88512+0+0 (secure 0 0 0) 0x7f22dc02cd50 con 0x7f22e40b3050 2023-08-09T13:47:57.122 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:57.120+0000 7f22da7fc640 1 --2- 172.21.15.43:0/2157349403 >> [v2:172.21.15.203:6832/998005540,v1:172.21.15.203:6833/998005540] conn(0x7f22c806a200 0x7f22c806c6c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:47:57.122 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:57.120+0000 7f22da7fc640 1 -- 172.21.15.43:0/2157349403 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(60..60 src has 1..60) v4 ==== 7298+0+0 (secure 0 0 0) 0x7f22dc0ab940 con 0x7f22e40b3050 2023-08-09T13:47:57.122 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:57.120+0000 7f22e9a56640 1 -- 172.21.15.43:0/2157349403 >> [v2:172.21.15.203:6832/998005540,v1:172.21.15.203:6833/998005540] conn(0x7f22c806a200 msgr2=0x7f22c806c6c0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.203:6832/998005540 2023-08-09T13:47:57.123 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:57.120+0000 7f22e9a56640 1 --2- 172.21.15.43:0/2157349403 >> [v2:172.21.15.203:6832/998005540,v1:172.21.15.203:6833/998005540] conn(0x7f22c806a200 0x7f22c806c6c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-08-09T13:47:57.127 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:57.125+0000 7f22da7fc640 1 -- 172.21.15.43:0/2157349403 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178066 (secure 0 0 0) 0x7f22dc076950 con 0x7f22e40b3050 2023-08-09T13:47:57.162 INFO:tasks.ceph.mgr.x.smithi203.stderr:2023-08-09T13:47:57.160+0000 7fb1cbc55040 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2023-08-09T13:47:57.212 INFO:tasks.ceph.mgr.x.smithi203.stderr:2023-08-09T13:47:57.211+0000 7fb1cbc55040 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2023-08-09T13:47:57.237 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:57.234+0000 7f22da7fc640 1 -- 172.21.15.43:0/2157349403 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mgrmap(e 15) v1 ==== 88052+0+0 (secure 0 0 0) 0x7f22dc076d20 con 0x7f22e40b3050 2023-08-09T13:47:57.237 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:57.235+0000 7f22da7fc640 1 -- 172.21.15.43:0/2157349403 >> [v2:172.21.15.203:6832/998005540,v1:172.21.15.203:6833/998005540] conn(0x7f22c806a200 msgr2=0x7f22c806c6c0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-08-09T13:47:57.237 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:57.235+0000 7f22da7fc640 1 --2- 172.21.15.43:0/2157349403 >> [v2:172.21.15.203:6832/998005540,v1:172.21.15.203:6833/998005540] conn(0x7f22c806a200 0x7f22c806c6c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:47:57.295 INFO:tasks.ceph.mgr.y.smithi043.stderr:2023-08-09T13:47:57.292+0000 7fac17030040 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2023-08-09T13:47:57.304 INFO:tasks.ceph.mgr.x.smithi203.stderr:2023-08-09T13:47:57.303+0000 7fb150f34640 -1 client.0 error registering admin socket command: (17) File exists 2023-08-09T13:47:57.304 INFO:tasks.ceph.mgr.x.smithi203.stderr:2023-08-09T13:47:57.303+0000 7fb150f34640 -1 client.0 error registering admin socket command: (17) File exists 2023-08-09T13:47:57.304 INFO:tasks.ceph.mgr.x.smithi203.stderr:2023-08-09T13:47:57.303+0000 7fb150f34640 -1 client.0 error registering admin socket command: (17) File exists 2023-08-09T13:47:57.305 INFO:tasks.ceph.mgr.x.smithi203.stderr:2023-08-09T13:47:57.303+0000 7fb150f34640 -1 client.0 error registering admin socket command: (17) File exists 2023-08-09T13:47:57.305 INFO:tasks.ceph.mgr.x.smithi203.stderr:2023-08-09T13:47:57.303+0000 7fb150f34640 -1 client.0 error registering admin socket command: (17) File exists 2023-08-09T13:47:57.305 INFO:tasks.ceph.mgr.x.smithi203.stderr:2023-08-09T13:47:57.303+0000 7fb14f731640 -1 client.0 error registering admin socket command: (17) File exists 2023-08-09T13:47:57.305 INFO:tasks.ceph.mgr.x.smithi203.stderr:2023-08-09T13:47:57.303+0000 7fb14f731640 -1 client.0 error registering admin socket command: (17) File exists 2023-08-09T13:47:57.305 INFO:tasks.ceph.mgr.x.smithi203.stderr:2023-08-09T13:47:57.303+0000 7fb14f731640 -1 client.0 error registering admin socket command: (17) File exists 2023-08-09T13:47:57.305 INFO:tasks.ceph.mgr.x.smithi203.stderr:2023-08-09T13:47:57.303+0000 7fb14f731640 -1 client.0 error registering admin socket command: (17) File exists 2023-08-09T13:47:57.305 INFO:tasks.ceph.mgr.x.smithi203.stderr:2023-08-09T13:47:57.303+0000 7fb14f731640 -1 client.0 error registering admin socket command: (17) File exists 2023-08-09T13:47:57.349 INFO:tasks.ceph.mgr.y.smithi043.stderr:2023-08-09T13:47:57.347+0000 7fac17030040 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2023-08-09T13:47:57.377 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:57.375+0000 7f22ebcba640 1 -- 172.21.15.43:0/2157349403 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f22e40660b0 con 0x7f22e40b3050 2023-08-09T13:47:57.379 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:57.376+0000 7f22da7fc640 1 -- 172.21.15.43:0/2157349403 <== mon.0 v2:172.21.15.43:3300/0 8 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 17 v17) v1 ==== 94+0+4878 (secure 0 0 0) 0x7f22dc07dc40 con 0x7f22e40b3050 2023-08-09T13:47:57.379 INFO:teuthology.orchestra.run.smithi043.stdout: 2023-08-09T13:47:57.380 INFO:teuthology.orchestra.run.smithi043.stdout:{"epoch":17,"default_fscid":2,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4919,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.203:6836/302214806","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6834","nonce":302214806},{"type":"v1","addr":"172.21.15.203:6836","nonce":302214806}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":12},{"gid":4922,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.203:6837/1677639100","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6835","nonce":1677639100},{"type":"v1","addr":"172.21.15.203:6837","nonce":1677639100}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":12},{"gid":5619,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.43:6834/2339480142","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6832","nonce":2339480142},{"type":"v1","addr":"172.21.15.43:6834","nonce":2339480142}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":12}],"filesystems":[{"mdsmap":{"epoch":17,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-08-09T13:36:11.336926+0000","modified":"2023-08-09T13:36:14.905554+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":5622},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5622":{"gid":5622,"name":"c","rank":0,"incarnation":13,"state":"up:active","state_seq":4,"addr":"172.21.15.43:6835/2019295969","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6833","nonce":2019295969},{"type":"v1","addr":"172.21.15.43:6835","nonce":2019295969}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[7],"metadata_pool":6,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":2}]} 2023-08-09T13:47:57.380 INFO:teuthology.orchestra.run.smithi043.stderr:dumped fsmap epoch 17 2023-08-09T13:47:57.382 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:57.379+0000 7f22ebcba640 1 -- 172.21.15.43:0/2157349403 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f22e40b3050 msgr2=0x7f22e406b9b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:47:57.382 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:57.379+0000 7f22ebcba640 1 --2- 172.21.15.43:0/2157349403 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f22e40b3050 0x7f22e406b9b0 secure :-1 s=READY pgs=748 cs=0 l=1 rev1=1 crypto rx=0x7f22dc036040 tx=0x7f22dc004a40 comp rx=0 tx=0).stop 2023-08-09T13:47:57.382 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:57.379+0000 7f22ebcba640 1 -- 172.21.15.43:0/2157349403 shutdown_connections 2023-08-09T13:47:57.382 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:57.379+0000 7f22ebcba640 1 --2- 172.21.15.43:0/2157349403 >> [v2:172.21.15.203:6832/998005540,v1:172.21.15.203:6833/998005540] conn(0x7f22c806a200 0x7f22c806c6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:47:57.382 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:57.379+0000 7f22ebcba640 1 --2- 172.21.15.43:0/2157349403 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f22e40b3050 0x7f22e406b9b0 unknown :-1 s=CLOSED pgs=748 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:47:57.382 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:57.379+0000 7f22ebcba640 1 --2- 172.21.15.43:0/2157349403 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f22e40b2b70 0x7f22e40c9c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:47:57.383 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:57.379+0000 7f22ebcba640 1 --2- 172.21.15.43:0/2157349403 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f22e40b2790 0x7f22e4148720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:47:57.383 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:57.379+0000 7f22ebcba640 1 -- 172.21.15.43:0/2157349403 >> 172.21.15.43:0/2157349403 conn(0x7f22e409b7b0 msgr2=0x7f22e40b5100 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-09T13:47:57.383 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:57.379+0000 7f22ebcba640 1 -- 172.21.15.43:0/2157349403 shutdown_connections 2023-08-09T13:47:57.383 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:57.379+0000 7f22ebcba640 1 -- 172.21.15.43:0/2157349403 wait complete. 2023-08-09T13:47:57.398 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-08-09T13:47:57.408 INFO:tasks.ceph.mgr.y.smithi043.stderr:2023-08-09T13:47:57.406+0000 7fac17030040 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2023-08-09T13:47:57.465 INFO:tasks.ceph.mgr.y.smithi043.stderr:2023-08-09T13:47:57.463+0000 7fac17030040 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2023-08-09T13:47:57.616 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:57.613+0000 7fa91dcc0640 1 Processor -- start 2023-08-09T13:47:57.616 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:57.614+0000 7fa91dcc0640 1 -- start start 2023-08-09T13:47:57.618 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:57.615+0000 7fa91dcc0640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7fa918153600 0x7fa9180ca900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:47:57.618 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:57.615+0000 7fa91dcc0640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fa91806dba0 0x7fa918070240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:47:57.618 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:57.616+0000 7fa91dcc0640 1 -- --> v1:172.21.15.43:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa9180caf70 con 0x7fa91815a110 2023-08-09T13:47:57.618 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:57.616+0000 7fa91dcc0640 1 -- --> v2:172.21.15.43:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa91806fdb0 con 0x7fa918153600 2023-08-09T13:47:57.618 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:57.616+0000 7fa91dcc0640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa9180c8ef0 con 0x7fa91806dba0 2023-08-09T13:47:57.619 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:57.616+0000 7fa91ccbe640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7fa918153600 0x7fa9180ca900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:47:57.619 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:57.616+0000 7fa917fff640 1 --1- >> v1:172.21.15.43:6789/0 conn(0x7fa91815a110 0x7fa91806d490 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.43:6789/0 says I am v1:172.21.15.43:36726/0 (socket says 172.21.15.43:36726) 2023-08-09T13:47:57.620 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:57.616+0000 7fa917fff640 1 -- 172.21.15.43:0/3208722252 learned_addr learned my addr 172.21.15.43:0/3208722252 (peer_addr_for_me v1:172.21.15.43:0/0) 2023-08-09T13:47:57.620 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:57.616+0000 7fa91d4bf640 1 --2- 172.21.15.43:0/3208722252 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fa91806dba0 0x7fa918070240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:47:57.620 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:57.617+0000 7fa91ccbe640 1 -- 172.21.15.43:0/3208722252 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fa91806dba0 msgr2=0x7fa918070240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-09T13:47:57.620 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:57.617+0000 7fa91ccbe640 1 --2- 172.21.15.43:0/3208722252 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fa91806dba0 0x7fa918070240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:47:57.620 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:57.617+0000 7fa91ccbe640 1 -- 172.21.15.43:0/3208722252 >> v1:172.21.15.43:6789/0 conn(0x7fa91815a110 legacy=0x7fa91806d490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:47:57.620 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:57.617+0000 7fa9177fe640 1 -- 172.21.15.43:0/3208722252 <== mon.0 v1:172.21.15.43:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3753997484 0 0) 0x7fa9180caf70 con 0x7fa91815a110 2023-08-09T13:47:57.620 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:57.617+0000 7fa91ccbe640 1 -- 172.21.15.43:0/3208722252 --> v2:172.21.15.43:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa9180c91d0 con 0x7fa918153600 2023-08-09T13:47:57.621 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:57.617+0000 7fa91d4bf640 1 --2- 172.21.15.43:0/3208722252 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fa91806dba0 0x7fa918070240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-08-09T13:47:57.622 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:57.618+0000 7fa91ccbe640 1 --2- 172.21.15.43:0/3208722252 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa918153600 0x7fa9180ca900 secure :-1 s=READY pgs=753 cs=0 l=1 rev1=1 crypto rx=0x7fa90c0097a0 tx=0x7fa90c02d9a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=38125231955ea5d server_cookie=0 in_seq=0 out_seq=0 2023-08-09T13:47:57.622 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:57.618+0000 7fa9177fe640 1 -- 172.21.15.43:0/3208722252 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa90c032860 con 0x7fa918153600 2023-08-09T13:47:57.622 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:57.618+0000 7fa9177fe640 1 -- 172.21.15.43:0/3208722252 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa90c0329c0 con 0x7fa918153600 2023-08-09T13:47:57.622 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:57.618+0000 7fa9177fe640 1 -- 172.21.15.43:0/3208722252 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa90c032d50 con 0x7fa918153600 2023-08-09T13:47:57.622 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:57.619+0000 7fa91dcc0640 1 -- 172.21.15.43:0/3208722252 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa918153600 msgr2=0x7fa9180ca900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:47:57.622 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:57.619+0000 7fa91dcc0640 1 --2- 172.21.15.43:0/3208722252 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa918153600 0x7fa9180ca900 secure :-1 s=READY pgs=753 cs=0 l=1 rev1=1 crypto rx=0x7fa90c0097a0 tx=0x7fa90c02d9a0 comp rx=0 tx=0).stop 2023-08-09T13:47:57.622 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:57.619+0000 7fa91dcc0640 1 -- 172.21.15.43:0/3208722252 shutdown_connections 2023-08-09T13:47:57.623 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:57.619+0000 7fa91dcc0640 1 --2- 172.21.15.43:0/3208722252 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fa91806dba0 0x7fa918070240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:47:57.623 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:57.619+0000 7fa91dcc0640 1 --2- 172.21.15.43:0/3208722252 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa918153600 0x7fa9180ca900 unknown :-1 s=CLOSED pgs=753 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:47:57.623 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:57.619+0000 7fa91dcc0640 1 -- 172.21.15.43:0/3208722252 >> 172.21.15.43:0/3208722252 conn(0x7fa91806be30 msgr2=0x7fa91806c210 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-09T13:47:57.623 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:57.619+0000 7fa91dcc0640 1 -- 172.21.15.43:0/3208722252 shutdown_connections 2023-08-09T13:47:57.623 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:57.619+0000 7fa91dcc0640 1 -- 172.21.15.43:0/3208722252 wait complete. 2023-08-09T13:47:57.624 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:57.621+0000 7fa91dcc0640 1 Processor -- start 2023-08-09T13:47:57.624 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:57.621+0000 7fa91dcc0640 1 -- start start 2023-08-09T13:47:57.625 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:57.622+0000 7fa91dcc0640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa91806dba0 0x7fa9181463a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:47:57.625 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:57.622+0000 7fa91dcc0640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fa918153600 0x7fa9180c6570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:47:57.625 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:57.622+0000 7fa91dcc0640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fa91815a110 0x7fa9180c4b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:47:57.625 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:57.622+0000 7fa91dcc0640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa9180c7530 con 0x7fa91806dba0 2023-08-09T13:47:57.625 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:57.622+0000 7fa91dcc0640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa9180c7810 con 0x7fa918153600 2023-08-09T13:47:57.626 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:57.622+0000 7fa91dcc0640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa9180c7af0 con 0x7fa91815a110 2023-08-09T13:47:57.626 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:57.622+0000 7fa91ccbe640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa91806dba0 0x7fa9181463a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:47:57.626 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:57.622+0000 7fa91ccbe640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa91806dba0 0x7fa9181463a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:57248/0 (socket says 172.21.15.43:57248) 2023-08-09T13:47:57.626 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:57.622+0000 7fa91ccbe640 1 -- 172.21.15.43:0/3066150150 learned_addr learned my addr 172.21.15.43:0/3066150150 (peer_addr_for_me v2:172.21.15.43:0/0) 2023-08-09T13:47:57.626 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:57.622+0000 7fa91ccbe640 1 -- 172.21.15.43:0/3066150150 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fa91815a110 msgr2=0x7fa9180c4b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:47:57.626 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:57.622+0000 7fa91ccbe640 1 --2- 172.21.15.43:0/3066150150 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fa91815a110 0x7fa9180c4b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:47:57.626 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:57.622+0000 7fa91ccbe640 1 -- 172.21.15.43:0/3066150150 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fa918153600 msgr2=0x7fa9180c6570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:47:57.627 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:57.622+0000 7fa91ccbe640 1 --2- 172.21.15.43:0/3066150150 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fa918153600 0x7fa9180c6570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:47:57.627 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:57.623+0000 7fa91ccbe640 1 -- 172.21.15.43:0/3066150150 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa90c009400 con 0x7fa91806dba0 2023-08-09T13:47:57.627 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:57.623+0000 7fa91ccbe640 1 --2- 172.21.15.43:0/3066150150 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa91806dba0 0x7fa9181463a0 secure :-1 s=READY pgs=754 cs=0 l=1 rev1=1 crypto rx=0x7fa90c02dfb0 tx=0x7fa90c004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-09T13:47:57.628 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:57.623+0000 7fa9157fa640 1 -- 172.21.15.43:0/3066150150 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa90c040040 con 0x7fa91806dba0 2023-08-09T13:47:57.628 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:57.623+0000 7fa9157fa640 1 -- 172.21.15.43:0/3066150150 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa90c032800 con 0x7fa91806dba0 2023-08-09T13:47:57.628 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:57.623+0000 7fa9157fa640 1 -- 172.21.15.43:0/3066150150 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa90c046630 con 0x7fa91806dba0 2023-08-09T13:47:57.628 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:57.623+0000 7fa91dcc0640 1 -- 172.21.15.43:0/3066150150 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa9180c7dd0 con 0x7fa91806dba0 2023-08-09T13:47:57.628 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:57.623+0000 7fa91dcc0640 1 -- 172.21.15.43:0/3066150150 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa9181468e0 con 0x7fa91806dba0 2023-08-09T13:47:57.628 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:57.624+0000 7fa91dcc0640 1 -- 172.21.15.43:0/3066150150 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa9180c82f0 con 0x7fa91806dba0 2023-08-09T13:47:57.628 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:57.625+0000 7fa9157fa640 1 -- 172.21.15.43:0/3066150150 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 15) v1 ==== 88052+0+0 (secure 0 0 0) 0x7fa90c0313b0 con 0x7fa91806dba0 2023-08-09T13:47:57.629 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:57.627+0000 7fa9157fa640 1 -- 172.21.15.43:0/3066150150 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(61..61 src has 1..61) v4 ==== 7529+0+0 (secure 0 0 0) 0x7fa90c0acda0 con 0x7fa91806dba0 2023-08-09T13:47:57.634 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:57.632+0000 7fa9157fa640 1 -- 172.21.15.43:0/3066150150 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178066 (secure 0 0 0) 0x7fa9180c82f0 con 0x7fa91806dba0 2023-08-09T13:47:57.830 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:57.827+0000 7fa91dcc0640 1 -- 172.21.15.43:0/3066150150 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fa9180c82f0 con 0x7fa91806dba0 2023-08-09T13:47:57.831 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:57.829+0000 7fa9157fa640 1 -- 172.21.15.43:0/3066150150 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v61) v1 ==== 74+0+20164 (secure 0 0 0) 0x7fa9180c82f0 con 0x7fa91806dba0 2023-08-09T13:47:57.832 INFO:teuthology.orchestra.run.smithi043.stdout: 2023-08-09T13:47:57.832 INFO:teuthology.orchestra.run.smithi043.stdout:{"epoch":61,"fsid":"daac87ee-1337-4250-a955-aea4e408a39b","created":"2023-08-09T13:33:41.701198+0000","modified":"2023-08-09T13:47:57.215924+0000","last_up_change":"2023-08-09T13:33:53.783453+0000","last_in_change":"2023-08-09T13:33:45.148904+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":7,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":"rbd","create_time":"2023-08-09T13:33:54.365137+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":".mgr","create_time":"2023-08-09T13:33:54.831803+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":6,"pool_name":"cephfs_metadata","create_time":"2023-08-09T13:36:08.969993+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"38","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":7,"pool_name":"cephfs_data","create_time":"2023-08-09T13:36:10.366495+0000","flags":8193,"flags_names":"hashpspool,selfmanaged_snaps","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"60","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":220,"snap_epoch":60,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"cbd49ecd-e896-4fbd-864e-5c37a829b165","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":37,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6800","nonce":502667335},{"type":"v1","addr":"172.21.15.43:6801","nonce":502667335}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6802","nonce":502667335},{"type":"v1","addr":"172.21.15.43:6803","nonce":502667335}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6806","nonce":502667335},{"type":"v1","addr":"172.21.15.43:6807","nonce":502667335}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6804","nonce":502667335},{"type":"v1","addr":"172.21.15.43:6805","nonce":502667335}]},"public_addr":"172.21.15.43:6801/502667335","cluster_addr":"172.21.15.43:6803/502667335","heartbeat_back_addr":"172.21.15.43:6807/502667335","heartbeat_front_addr":"172.21.15.43:6805/502667335","state":["exists","up"]},{"osd":1,"uuid":"75c7e0fe-1ec7-4fe8-a2f2-3661a65b8500","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":37,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6808","nonce":3289134610},{"type":"v1","addr":"172.21.15.43:6809","nonce":3289134610}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6810","nonce":3289134610},{"type":"v1","addr":"172.21.15.43:6812","nonce":3289134610}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6821","nonce":3289134610},{"type":"v1","addr":"172.21.15.43:6824","nonce":3289134610}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6815","nonce":3289134610},{"type":"v1","addr":"172.21.15.43:6818","nonce":3289134610}]},"public_addr":"172.21.15.43:6809/3289134610","cluster_addr":"172.21.15.43:6812/3289134610","heartbeat_back_addr":"172.21.15.43:6824/3289134610","heartbeat_front_addr":"172.21.15.43:6818/3289134610","state":["exists","up"]},{"osd":2,"uuid":"bb5c41e9-7f5f-4125-9b01-3bb8bfc2ea83","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":37,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6811","nonce":4079709736},{"type":"v1","addr":"172.21.15.43:6814","nonce":4079709736}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6817","nonce":4079709736},{"type":"v1","addr":"172.21.15.43:6820","nonce":4079709736}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6829","nonce":4079709736},{"type":"v1","addr":"172.21.15.43:6831","nonce":4079709736}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6823","nonce":4079709736},{"type":"v1","addr":"172.21.15.43:6826","nonce":4079709736}]},"public_addr":"172.21.15.43:6814/4079709736","cluster_addr":"172.21.15.43:6820/4079709736","heartbeat_back_addr":"172.21.15.43:6831/4079709736","heartbeat_front_addr":"172.21.15.43:6826/4079709736","state":["exists","up"]},{"osd":3,"uuid":"bfe4c8b4-694a-465c-87e6-2b2aab852613","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":37,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6813","nonce":1709538759},{"type":"v1","addr":"172.21.15.43:6816","nonce":1709538759}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6819","nonce":1709538759},{"type":"v1","addr":"172.21.15.43:6822","nonce":1709538759}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6828","nonce":1709538759},{"type":"v1","addr":"172.21.15.43:6830","nonce":1709538759}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6825","nonce":1709538759},{"type":"v1","addr":"172.21.15.43:6827","nonce":1709538759}]},"public_addr":"172.21.15.43:6816/1709538759","cluster_addr":"172.21.15.43:6822/1709538759","heartbeat_back_addr":"172.21.15.43:6830/1709538759","heartbeat_front_addr":"172.21.15.43:6827/1709538759","state":["exists","up"]},{"osd":4,"uuid":"fa2e8a77-aba1-4756-853e-0d174a5bdeba","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":41,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6800","nonce":4196866481},{"type":"v1","addr":"172.21.15.203:6801","nonce":4196866481}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6802","nonce":4196866481},{"type":"v1","addr":"172.21.15.203:6803","nonce":4196866481}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6806","nonce":4196866481},{"type":"v1","addr":"172.21.15.203:6807","nonce":4196866481}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6804","nonce":4196866481},{"type":"v1","addr":"172.21.15.203:6805","nonce":4196866481}]},"public_addr":"172.21.15.203:6801/4196866481","cluster_addr":"172.21.15.203:6803/4196866481","heartbeat_back_addr":"172.21.15.203:6807/4196866481","heartbeat_front_addr":"172.21.15.203:6805/4196866481","state":["exists","up"]},{"osd":5,"uuid":"81811e4b-4d24-4f51-a82c-f1d700a2a00d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":41,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6816","nonce":3059290081},{"type":"v1","addr":"172.21.15.203:6817","nonce":3059290081}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6818","nonce":3059290081},{"type":"v1","addr":"172.21.15.203:6819","nonce":3059290081}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6822","nonce":3059290081},{"type":"v1","addr":"172.21.15.203:6823","nonce":3059290081}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6820","nonce":3059290081},{"type":"v1","addr":"172.21.15.203:6821","nonce":3059290081}]},"public_addr":"172.21.15.203:6817/3059290081","cluster_addr":"172.21.15.203:6819/3059290081","heartbeat_back_addr":"172.21.15.203:6823/3059290081","heartbeat_front_addr":"172.21.15.203:6821/3059290081","state":["exists","up"]},{"osd":6,"uuid":"307bac4d-6a5b-443d-af75-56d695d7674f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":37,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6824","nonce":512155569},{"type":"v1","addr":"172.21.15.203:6825","nonce":512155569}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6826","nonce":512155569},{"type":"v1","addr":"172.21.15.203:6827","nonce":512155569}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6830","nonce":512155569},{"type":"v1","addr":"172.21.15.203:6831","nonce":512155569}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6828","nonce":512155569},{"type":"v1","addr":"172.21.15.203:6829","nonce":512155569}]},"public_addr":"172.21.15.203:6825/512155569","cluster_addr":"172.21.15.203:6827/512155569","heartbeat_back_addr":"172.21.15.203:6831/512155569","heartbeat_front_addr":"172.21.15.203:6829/512155569","state":["exists","up"]},{"osd":7,"uuid":"a95dc7ca-9f1b-4295-af70-68a22a2d7d5e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":37,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6808","nonce":3331450438},{"type":"v1","addr":"172.21.15.203:6809","nonce":3331450438}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6810","nonce":3331450438},{"type":"v1","addr":"172.21.15.203:6811","nonce":3331450438}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6814","nonce":3331450438},{"type":"v1","addr":"172.21.15.203:6815","nonce":3331450438}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6812","nonce":3331450438},{"type":"v1","addr":"172.21.15.203:6813","nonce":3331450438}]},"public_addr":"172.21.15.203:6809/3331450438","cluster_addr":"172.21.15.203:6811/3331450438","heartbeat_back_addr":"172.21.15.203:6815/3331450438","heartbeat_front_addr":"172.21.15.203:6813/3331450438","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-09T13:33:50.547652+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-09T13:33:50.589469+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-09T13:33:50.561724+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-09T13:33:50.599650+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-09T13:33:50.411855+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-09T13:33:50.219681+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-09T13:33:50.504664+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-09T13:33:50.441248+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"7.2a","mappings":[{"from":0,"to":5}]},{"pgid":"7.3c","mappings":[{"from":0,"to":4}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.203:0/830217892":"2023-08-10T13:47:57.215872+0000","172.21.15.203:0/1493361813":"2023-08-10T13:47:57.215872+0000","172.21.15.203:0/3309257000":"2023-08-10T13:36:27.970754+0000","172.21.15.203:0/2444066336":"2023-08-10T13:47:57.215872+0000","172.21.15.203:6832/830097474":"2023-08-10T13:36:27.970754+0000","172.21.15.203:6833/830097474":"2023-08-10T13:36:27.970754+0000","172.21.15.203:0/2439593041":"2023-08-10T13:36:27.970754+0000","172.21.15.203:0/2454266731":"2023-08-10T13:47:57.215872+0000","172.21.15.203:0/1761968483":"2023-08-10T13:36:27.970754+0000","172.21.15.203:0/890377927":"2023-08-10T13:36:27.970754+0000","172.21.15.203:0/560220444":"2023-08-10T13:47:57.215872+0000","172.21.15.203:6832/998005540":"2023-08-10T13:47:57.215872+0000","172.21.15.203:6833/998005540":"2023-08-10T13:47:57.215872+0000","172.21.15.203:0/315780925":"2023-08-10T13:47:57.215872+0000","172.21.15.203:0/1813604691":"2023-08-10T13:47:57.215872+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":7,"snaps":[{"begin":4,"length":1},{"begin":8,"length":1},{"begin":10,"length":1},{"begin":15,"length":1},{"begin":18,"length":2},{"begin":25,"length":1},{"begin":27,"length":1},{"begin":31,"length":1},{"begin":34,"length":1},{"begin":40,"length":2},{"begin":48,"length":3},{"begin":55,"length":1},{"begin":57,"length":1},{"begin":59,"length":1},{"begin":62,"length":1},{"begin":65,"length":1},{"begin":68,"length":1},{"begin":72,"length":1},{"begin":74,"length":1},{"begin":77,"length":1},{"begin":83,"length":2},{"begin":87,"length":1},{"begin":95,"length":2},{"begin":98,"length":1},{"begin":107,"length":2},{"begin":111,"length":1},{"begin":188,"length":33}]}],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-08-09T13:47:57.834 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:57.831+0000 7fa91dcc0640 1 -- 172.21.15.43:0/3066150150 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa91806dba0 msgr2=0x7fa9181463a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:47:57.834 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:57.831+0000 7fa91dcc0640 1 --2- 172.21.15.43:0/3066150150 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa91806dba0 0x7fa9181463a0 secure :-1 s=READY pgs=754 cs=0 l=1 rev1=1 crypto rx=0x7fa90c02dfb0 tx=0x7fa90c004730 comp rx=0 tx=0).stop 2023-08-09T13:47:57.834 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:57.832+0000 7fa91dcc0640 1 -- 172.21.15.43:0/3066150150 shutdown_connections 2023-08-09T13:47:57.834 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:57.832+0000 7fa91dcc0640 1 --2- 172.21.15.43:0/3066150150 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fa91815a110 0x7fa9180c4b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:47:57.834 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:57.832+0000 7fa91dcc0640 1 --2- 172.21.15.43:0/3066150150 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fa918153600 0x7fa9180c6570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:47:57.835 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:57.832+0000 7fa91dcc0640 1 --2- 172.21.15.43:0/3066150150 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa91806dba0 0x7fa9181463a0 unknown :-1 s=CLOSED pgs=754 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:47:57.835 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:57.832+0000 7fa91dcc0640 1 -- 172.21.15.43:0/3066150150 >> 172.21.15.43:0/3066150150 conn(0x7fa91806be30 msgr2=0x7fa91806d490 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-09T13:47:57.835 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:57.832+0000 7fa91dcc0640 1 -- 172.21.15.43:0/3066150150 shutdown_connections 2023-08-09T13:47:57.835 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:57.832+0000 7fa91dcc0640 1 -- 172.21.15.43:0/3066150150 wait complete. 2023-08-09T13:47:57.854 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': 'rbd', 'create_time': '2023-08-09T13:33:54.365137+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}, 'read_balance': {'score_acting': 2, 'score_stable': 2, 'optimal_score': 1, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': '.mgr', 'create_time': '2023-08-09T13:33:54.831803+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 6, 'pool_name': 'cephfs_metadata', 'create_time': '2023-08-09T13:36:08.969993+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '38', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.75, 'score_stable': 1.75, 'optimal_score': 1, 'raw_score_acting': 1.75, 'raw_score_stable': 1.75, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 7, 'pool_name': 'cephfs_data', 'create_time': '2023-08-09T13:36:10.366495+0000', 'flags': 8193, 'flags_names': 'hashpspool,selfmanaged_snaps', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '60', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 220, 'snap_epoch': 60, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-08-09T13:47:57.854 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get rbd pg_num 2023-08-09T13:47:58.132 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:58.130+0000 7f4048e2c640 1 Processor -- start 2023-08-09T13:47:58.133 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:58.130+0000 7f4048e2c640 1 -- start start 2023-08-09T13:47:58.134 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:58.131+0000 7f4048e2c640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f40440c3540 0x7f404406a8d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:47:58.134 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:58.131+0000 7f4048e2c640 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7f40440b21e0 0x7f4044161890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:47:58.134 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:58.131+0000 7f4048e2c640 1 -- --> v1:172.21.15.203:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4044161dd0 con 0x7f40440cd7f0 2023-08-09T13:47:58.134 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:58.131+0000 7f4048e2c640 1 -- --> v2:172.21.15.43:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f40440ce380 con 0x7f40440c3540 2023-08-09T13:47:58.134 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:58.131+0000 7f4048e2c640 1 -- --> v2:172.21.15.203:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f40440c3920 con 0x7f40440b21e0 2023-08-09T13:47:58.135 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:58.132+0000 7f40437fe640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f40440c3540 0x7f404406a8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:47:58.135 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:58.132+0000 7f40437fe640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f40440c3540 0x7f404406a8d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:57254/0 (socket says 172.21.15.43:57254) 2023-08-09T13:47:58.135 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:58.132+0000 7f40437fe640 1 -- 172.21.15.43:0/1772260698 learned_addr learned my addr 172.21.15.43:0/1772260698 (peer_addr_for_me v2:172.21.15.43:0/0) 2023-08-09T13:47:58.136 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:58.132+0000 7f4043fff640 1 --2- 172.21.15.43:0/1772260698 >> v2:172.21.15.203:3300/0 conn(0x7f40440b21e0 0x7f4044161890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:47:58.136 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:58.132+0000 7f40437fe640 1 -- 172.21.15.43:0/1772260698 >> v2:172.21.15.203:3300/0 conn(0x7f40440b21e0 msgr2=0x7f4044161890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-09T13:47:58.136 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:58.132+0000 7f40437fe640 1 --2- 172.21.15.43:0/1772260698 >> v2:172.21.15.203:3300/0 conn(0x7f40440b21e0 0x7f4044161890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:47:58.136 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:58.132+0000 7f40437fe640 1 -- 172.21.15.43:0/1772260698 >> v1:172.21.15.203:6789/0 conn(0x7f40440cd7f0 legacy=0x7f40440b1ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-09T13:47:58.136 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:58.132+0000 7f40437fe640 1 -- 172.21.15.43:0/1772260698 --> v2:172.21.15.43:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f404406c300 con 0x7f40440c3540 2023-08-09T13:47:58.137 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:58.133+0000 7f40437fe640 1 --2- 172.21.15.43:0/1772260698 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f40440c3540 0x7f404406a8d0 secure :-1 s=READY pgs=755 cs=0 l=1 rev1=1 crypto rx=0x7f40440b2bc0 tx=0x7f403402b880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=21b2d99ec30fa92e server_cookie=0 in_seq=0 out_seq=0 2023-08-09T13:47:58.137 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:58.134+0000 7f4042ffd640 1 -- 172.21.15.43:0/1772260698 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f40340317e0 con 0x7f40440c3540 2023-08-09T13:47:58.137 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:58.134+0000 7f4042ffd640 1 -- 172.21.15.43:0/1772260698 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4034031940 con 0x7f40440c3540 2023-08-09T13:47:58.138 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:58.134+0000 7f4042ffd640 1 -- 172.21.15.43:0/1772260698 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4034031cd0 con 0x7f40440c3540 2023-08-09T13:47:58.138 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:58.134+0000 7f4048e2c640 1 -- 172.21.15.43:0/1772260698 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f40440c3540 msgr2=0x7f404406a8d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:47:58.138 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:58.134+0000 7f4048e2c640 1 --2- 172.21.15.43:0/1772260698 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f40440c3540 0x7f404406a8d0 secure :-1 s=READY pgs=755 cs=0 l=1 rev1=1 crypto rx=0x7f40440b2bc0 tx=0x7f403402b880 comp rx=0 tx=0).stop 2023-08-09T13:47:58.138 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:58.134+0000 7f4048e2c640 1 -- 172.21.15.43:0/1772260698 shutdown_connections 2023-08-09T13:47:58.138 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:58.134+0000 7f4048e2c640 1 --2- 172.21.15.43:0/1772260698 >> v2:172.21.15.203:3300/0 conn(0x7f40440b21e0 0x7f4044161890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:47:58.138 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:58.134+0000 7f4048e2c640 1 --2- 172.21.15.43:0/1772260698 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f40440c3540 0x7f404406a8d0 unknown :-1 s=CLOSED pgs=755 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:47:58.138 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:58.134+0000 7f4048e2c640 1 -- 172.21.15.43:0/1772260698 >> 172.21.15.43:0/1772260698 conn(0x7f40440ccad0 msgr2=0x7f40440cceb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-09T13:47:58.139 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:58.134+0000 7f4048e2c640 1 -- 172.21.15.43:0/1772260698 shutdown_connections 2023-08-09T13:47:58.139 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:58.135+0000 7f4048e2c640 1 -- 172.21.15.43:0/1772260698 wait complete. 2023-08-09T13:47:58.139 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:58.136+0000 7f4048e2c640 1 Processor -- start 2023-08-09T13:47:58.139 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:58.136+0000 7f4048e2c640 1 -- start start 2023-08-09T13:47:58.141 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:58.137+0000 7f4048e2c640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f40440b21e0 0x7f404406f960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:47:58.141 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:58.137+0000 7f4048e2c640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f40440c3540 0x7f40440bece0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:47:58.141 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:58.137+0000 7f4048e2c640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f40440cd7f0 0x7f404415f170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:47:58.141 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:58.138+0000 7f4048e2c640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f40440bf8c0 con 0x7f40440c3540 2023-08-09T13:47:58.141 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:58.138+0000 7f4048e2c640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f404415f6b0 con 0x7f40440b21e0 2023-08-09T13:47:58.142 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:58.138+0000 7f4048e2c640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f40440c0d50 con 0x7f40440cd7f0 2023-08-09T13:47:58.142 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:58.138+0000 7f403affd640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f40440c3540 0x7f40440bece0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:47:58.142 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:58.138+0000 7f403affd640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f40440c3540 0x7f40440bece0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:57268/0 (socket says 172.21.15.43:57268) 2023-08-09T13:47:58.142 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:58.138+0000 7f403affd640 1 -- 172.21.15.43:0/889543435 learned_addr learned my addr 172.21.15.43:0/889543435 (peer_addr_for_me v2:172.21.15.43:0/0) 2023-08-09T13:47:58.142 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:58.138+0000 7f40437fe640 1 --2- 172.21.15.43:0/889543435 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f40440b21e0 0x7f404406f960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:47:58.142 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:58.138+0000 7f4043fff640 1 --2- 172.21.15.43:0/889543435 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f40440cd7f0 0x7f404415f170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:47:58.142 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:58.138+0000 7f403affd640 1 -- 172.21.15.43:0/889543435 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f40440cd7f0 msgr2=0x7f404415f170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:47:58.143 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:58.138+0000 7f403affd640 1 --2- 172.21.15.43:0/889543435 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f40440cd7f0 0x7f404415f170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:47:58.143 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:58.138+0000 7f403affd640 1 -- 172.21.15.43:0/889543435 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f40440b21e0 msgr2=0x7f404406f960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:47:58.143 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:58.138+0000 7f403affd640 1 --2- 172.21.15.43:0/889543435 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f40440b21e0 0x7f404406f960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:47:58.143 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:58.138+0000 7f403affd640 1 -- 172.21.15.43:0/889543435 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f40340094d0 con 0x7f40440c3540 2023-08-09T13:47:58.143 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:58.139+0000 7f403affd640 1 --2- 172.21.15.43:0/889543435 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f40440c3540 0x7f40440bece0 secure :-1 s=READY pgs=756 cs=0 l=1 rev1=1 crypto rx=0x7f402c00b3f0 tx=0x7f402c00b8b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-09T13:47:58.144 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:58.139+0000 7f4040ff9640 1 -- 172.21.15.43:0/889543435 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f402c011830 con 0x7f40440c3540 2023-08-09T13:47:58.144 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:58.139+0000 7f4040ff9640 1 -- 172.21.15.43:0/889543435 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f402c011990 con 0x7f40440c3540 2023-08-09T13:47:58.144 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:58.139+0000 7f4040ff9640 1 -- 172.21.15.43:0/889543435 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f402c011d20 con 0x7f40440c3540 2023-08-09T13:47:58.144 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:58.139+0000 7f4048e2c640 1 -- 172.21.15.43:0/889543435 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f40440bf300 con 0x7f40440c3540 2023-08-09T13:47:58.145 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:58.139+0000 7f4048e2c640 1 -- 172.21.15.43:0/889543435 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f40440c0ff0 con 0x7f40440c3540 2023-08-09T13:47:58.145 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:58.140+0000 7f4048e2c640 1 -- 172.21.15.43:0/889543435 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f404404e390 con 0x7f40440c3540 2023-08-09T13:47:58.145 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:58.141+0000 7f4040ff9640 1 -- 172.21.15.43:0/889543435 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 15) v1 ==== 88052+0+0 (secure 0 0 0) 0x7f402c01a400 con 0x7f40440c3540 2023-08-09T13:47:58.145 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:58.142+0000 7f4040ff9640 1 -- 172.21.15.43:0/889543435 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(61..61 src has 1..61) v4 ==== 7529+0+0 (secure 0 0 0) 0x7f402c088790 con 0x7f40440c3540 2023-08-09T13:47:58.148 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:58.146+0000 7f4040ff9640 1 -- 172.21.15.43:0/889543435 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178066 (secure 0 0 0) 0x7f402c053c20 con 0x7f40440c3540 2023-08-09T13:47:58.256 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:58.253+0000 7f4040ff9640 1 -- 172.21.15.43:0/889543435 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mgrmap(e 16) v1 ==== 88179+0+0 (secure 0 0 0) 0x7f402c0536d0 con 0x7f40440c3540 2023-08-09T13:47:58.257 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:58.254+0000 7f4040ff9640 1 --2- 172.21.15.43:0/889543435 >> [v2:172.21.15.203:6832/272917756,v1:172.21.15.203:6833/272917756] conn(0x7f40240709d0 0x7f4024072dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:47:58.258 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:58.256+0000 7f40437fe640 1 --2- 172.21.15.43:0/889543435 >> [v2:172.21.15.203:6832/272917756,v1:172.21.15.203:6833/272917756] conn(0x7f40240709d0 0x7f4024072dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:47:58.263 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:58.261+0000 7f40437fe640 1 --2- 172.21.15.43:0/889543435 >> [v2:172.21.15.203:6832/272917756,v1:172.21.15.203:6833/272917756] conn(0x7f40240709d0 0x7f4024072dc0 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7f4034036000 tx=0x7f403403d040 comp rx=0 tx=0).ready entity=mgr.5533 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-09T13:47:58.363 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:58.361+0000 7f4048e2c640 1 -- 172.21.15.43:0/889543435 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"} v 0) v1 -- 0x7f4044066bd0 con 0x7f40440c3540 2023-08-09T13:47:58.364 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:58.361+0000 7f4040ff9640 1 -- 172.21.15.43:0/889543435 <== mon.0 v2:172.21.15.43:3300/0 8 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"}]=0 v62) v1 ==== 92+0+10 (secure 0 0 0) 0x7f402c06fdc0 con 0x7f40440c3540 2023-08-09T13:47:58.364 INFO:teuthology.orchestra.run.smithi043.stdout:pg_num: 8 2023-08-09T13:47:58.367 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:58.364+0000 7f4048e2c640 1 -- 172.21.15.43:0/889543435 >> [v2:172.21.15.203:6832/272917756,v1:172.21.15.203:6833/272917756] conn(0x7f40240709d0 msgr2=0x7f4024072dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:47:58.367 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:58.364+0000 7f4048e2c640 1 --2- 172.21.15.43:0/889543435 >> [v2:172.21.15.203:6832/272917756,v1:172.21.15.203:6833/272917756] conn(0x7f40240709d0 0x7f4024072dc0 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7f4034036000 tx=0x7f403403d040 comp rx=0 tx=0).stop 2023-08-09T13:47:58.367 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:58.364+0000 7f4048e2c640 1 -- 172.21.15.43:0/889543435 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f40440c3540 msgr2=0x7f40440bece0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:47:58.367 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:58.364+0000 7f4048e2c640 1 --2- 172.21.15.43:0/889543435 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f40440c3540 0x7f40440bece0 secure :-1 s=READY pgs=756 cs=0 l=1 rev1=1 crypto rx=0x7f402c00b3f0 tx=0x7f402c00b8b0 comp rx=0 tx=0).stop 2023-08-09T13:47:58.367 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:58.364+0000 7f4048e2c640 1 -- 172.21.15.43:0/889543435 shutdown_connections 2023-08-09T13:47:58.367 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:58.364+0000 7f4048e2c640 1 --2- 172.21.15.43:0/889543435 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f40440cd7f0 0x7f404415f170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:47:58.368 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:58.364+0000 7f4048e2c640 1 --2- 172.21.15.43:0/889543435 >> [v2:172.21.15.203:6832/272917756,v1:172.21.15.203:6833/272917756] conn(0x7f40240709d0 0x7f4024072dc0 unknown :-1 s=CLOSED pgs=6 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:47:58.368 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:58.364+0000 7f4048e2c640 1 --2- 172.21.15.43:0/889543435 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f40440c3540 0x7f40440bece0 unknown :-1 s=CLOSED pgs=756 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:47:58.368 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:58.364+0000 7f4048e2c640 1 --2- 172.21.15.43:0/889543435 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f40440b21e0 0x7f404406f960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:47:58.368 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:58.364+0000 7f4048e2c640 1 -- 172.21.15.43:0/889543435 >> 172.21.15.43:0/889543435 conn(0x7f40440ccad0 msgr2=0x7f40440cd0c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-09T13:47:58.368 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:58.364+0000 7f4048e2c640 1 -- 172.21.15.43:0/889543435 shutdown_connections 2023-08-09T13:47:58.368 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:58.364+0000 7f4048e2c640 1 -- 172.21.15.43:0/889543435 wait complete. 2023-08-09T13:47:58.387 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-08-09T13:47:58.601 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:58.599+0000 7fd4bde8e640 1 Processor -- start 2023-08-09T13:47:58.601 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:58.599+0000 7fd4bde8e640 1 -- start start 2023-08-09T13:47:58.603 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:58.600+0000 7fd4bde8e640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7fd4b815c550 0x7fd4b80cd6d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:47:58.603 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:58.601+0000 7fd4bde8e640 1 -- --> v1:172.21.15.43:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd4b80b7bd0 con 0x7fd4b80b45f0 2023-08-09T13:47:58.603 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:58.601+0000 7fd4bde8e640 1 -- --> v1:172.21.15.203:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd4b80d1c00 con 0x7fd4b80d04b0 2023-08-09T13:47:58.603 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:58.601+0000 7fd4bde8e640 1 -- --> v2:172.21.15.43:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd4b815c930 con 0x7fd4b815c550 2023-08-09T13:47:58.604 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:58.601+0000 7fd4b77fe640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7fd4b815c550 0x7fd4b80cd6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:47:58.604 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:58.601+0000 7fd4b77fe640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7fd4b815c550 0x7fd4b80cd6d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:57284/0 (socket says 172.21.15.43:57284) 2023-08-09T13:47:58.604 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:58.601+0000 7fd4b77fe640 1 -- 172.21.15.43:0/495853707 learned_addr learned my addr 172.21.15.43:0/495853707 (peer_addr_for_me v2:172.21.15.43:0/0) 2023-08-09T13:47:58.604 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:58.601+0000 7fd4b7fff640 1 --1- 172.21.15.43:0/495853707 >> v1:172.21.15.43:6789/0 conn(0x7fd4b80b45f0 0x7fd4b80b59d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.43:6789/0 says I am v1:172.21.15.43:36730/0 (socket says 172.21.15.43:36730) 2023-08-09T13:47:58.605 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:58.601+0000 7fd4b77fe640 1 -- 172.21.15.43:0/495853707 >> v1:172.21.15.203:6789/0 conn(0x7fd4b80d04b0 legacy=0x7fd4b80b3ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-09T13:47:58.605 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:58.602+0000 7fd4b77fe640 1 -- 172.21.15.43:0/495853707 >> v1:172.21.15.43:6789/0 conn(0x7fd4b80b45f0 legacy=0x7fd4b80b59d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:47:58.605 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:58.602+0000 7fd4b77fe640 1 -- 172.21.15.43:0/495853707 --> v2:172.21.15.43:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd4b80b54f0 con 0x7fd4b815c550 2023-08-09T13:47:58.605 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:58.603+0000 7fd4b77fe640 1 --2- 172.21.15.43:0/495853707 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fd4b815c550 0x7fd4b80cd6d0 secure :-1 s=READY pgs=758 cs=0 l=1 rev1=1 crypto rx=0x7fd4a8009e50 tx=0x7fd4a802bee0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1da35b5928f7946d server_cookie=0 in_seq=0 out_seq=0 2023-08-09T13:47:58.606 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:58.603+0000 7fd4b67fc640 1 -- 172.21.15.43:0/495853707 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd4a802ca80 con 0x7fd4b815c550 2023-08-09T13:47:58.606 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:58.603+0000 7fd4b67fc640 1 -- 172.21.15.43:0/495853707 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd4a802cbe0 con 0x7fd4b815c550 2023-08-09T13:47:58.606 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:58.603+0000 7fd4b67fc640 1 -- 172.21.15.43:0/495853707 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd4a80343f0 con 0x7fd4b815c550 2023-08-09T13:47:58.607 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:58.604+0000 7fd4bde8e640 1 -- 172.21.15.43:0/495853707 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fd4b815c550 msgr2=0x7fd4b80cd6d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:47:58.607 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:58.604+0000 7fd4bde8e640 1 --2- 172.21.15.43:0/495853707 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fd4b815c550 0x7fd4b80cd6d0 secure :-1 s=READY pgs=758 cs=0 l=1 rev1=1 crypto rx=0x7fd4a8009e50 tx=0x7fd4a802bee0 comp rx=0 tx=0).stop 2023-08-09T13:47:58.607 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:58.604+0000 7fd4bde8e640 1 -- 172.21.15.43:0/495853707 shutdown_connections 2023-08-09T13:47:58.607 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:58.604+0000 7fd4bde8e640 1 --2- 172.21.15.43:0/495853707 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fd4b815c550 0x7fd4b80cd6d0 unknown :-1 s=CLOSED pgs=758 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:47:58.607 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:58.604+0000 7fd4bde8e640 1 -- 172.21.15.43:0/495853707 >> 172.21.15.43:0/495853707 conn(0x7fd4b8071520 msgr2=0x7fd4b80ceee0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-09T13:47:58.607 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:58.604+0000 7fd4bde8e640 1 -- 172.21.15.43:0/495853707 shutdown_connections 2023-08-09T13:47:58.608 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:58.604+0000 7fd4bde8e640 1 -- 172.21.15.43:0/495853707 wait complete. 2023-08-09T13:47:58.608 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:58.606+0000 7fd4bde8e640 1 Processor -- start 2023-08-09T13:47:58.609 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:58.606+0000 7fd4bde8e640 1 -- start start 2023-08-09T13:47:58.610 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:58.608+0000 7fd4bde8e640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fd4b80b45f0 0x7fd4b814b1f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:47:58.610 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:58.608+0000 7fd4bde8e640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fd4b80d04b0 0x7fd4b815dc70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:47:58.610 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:58.608+0000 7fd4bde8e640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fd4b815c550 0x7fd4b815e1b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:47:58.611 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:58.608+0000 7fd4bde8e640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd4b814eb70 con 0x7fd4b80b45f0 2023-08-09T13:47:58.611 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:58.608+0000 7fd4bde8e640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd4b81f19d0 con 0x7fd4b80d04b0 2023-08-09T13:47:58.611 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:58.608+0000 7fd4bde8e640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd4b814e890 con 0x7fd4b815c550 2023-08-09T13:47:58.611 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:58.608+0000 7fd4b77fe640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fd4b80b45f0 0x7fd4b814b1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:47:58.611 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:58.608+0000 7fd4b77fe640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fd4b80b45f0 0x7fd4b814b1f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:57294/0 (socket says 172.21.15.43:57294) 2023-08-09T13:47:58.611 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:58.608+0000 7fd4b77fe640 1 -- 172.21.15.43:0/513009588 learned_addr learned my addr 172.21.15.43:0/513009588 (peer_addr_for_me v2:172.21.15.43:0/0) 2023-08-09T13:47:58.612 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:58.608+0000 7fd4b6ffd640 1 --2- 172.21.15.43:0/513009588 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fd4b80d04b0 0x7fd4b815dc70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:47:58.612 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:58.608+0000 7fd4b77fe640 1 -- 172.21.15.43:0/513009588 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fd4b815c550 msgr2=0x7fd4b815e1b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:47:58.613 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:58.608+0000 7fd4b77fe640 1 --2- 172.21.15.43:0/513009588 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fd4b815c550 0x7fd4b815e1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:47:58.613 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:58.608+0000 7fd4b77fe640 1 -- 172.21.15.43:0/513009588 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fd4b80d04b0 msgr2=0x7fd4b815dc70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:47:58.613 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:58.608+0000 7fd4b77fe640 1 --2- 172.21.15.43:0/513009588 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fd4b80d04b0 0x7fd4b815dc70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:47:58.613 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:58.608+0000 7fd4b77fe640 1 -- 172.21.15.43:0/513009588 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd4a8009b30 con 0x7fd4b80b45f0 2023-08-09T13:47:58.613 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:58.609+0000 7fd4b77fe640 1 --2- 172.21.15.43:0/513009588 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fd4b80b45f0 0x7fd4b814b1f0 secure :-1 s=READY pgs=759 cs=0 l=1 rev1=1 crypto rx=0x7fd4a8009e50 tx=0x7fd4a8004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-09T13:47:58.613 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:58.609+0000 7fd49ffff640 1 -- 172.21.15.43:0/513009588 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd4a803e450 con 0x7fd4b80b45f0 2023-08-09T13:47:58.613 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:58.609+0000 7fd49ffff640 1 -- 172.21.15.43:0/513009588 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd4a803e5f0 con 0x7fd4b80b45f0 2023-08-09T13:47:58.614 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:58.609+0000 7fd49ffff640 1 -- 172.21.15.43:0/513009588 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd4a803e9a0 con 0x7fd4b80b45f0 2023-08-09T13:47:58.614 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:58.609+0000 7fd4bde8e640 1 -- 172.21.15.43:0/513009588 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd4b81f1be0 con 0x7fd4b80b45f0 2023-08-09T13:47:58.614 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:58.609+0000 7fd4bde8e640 1 -- 172.21.15.43:0/513009588 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd4b81f21a0 con 0x7fd4b80b45f0 2023-08-09T13:47:58.614 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:58.610+0000 7fd4bde8e640 1 -- 172.21.15.43:0/513009588 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd4b80739f0 con 0x7fd4b80b45f0 2023-08-09T13:47:58.614 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:58.612+0000 7fd49ffff640 1 -- 172.21.15.43:0/513009588 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 16) v1 ==== 88179+0+0 (secure 0 0 0) 0x7fd4a802cd50 con 0x7fd4b80b45f0 2023-08-09T13:47:58.615 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:58.612+0000 7fd49ffff640 1 --2- 172.21.15.43:0/513009588 >> [v2:172.21.15.203:6832/272917756,v1:172.21.15.203:6833/272917756] conn(0x7fd4a4069ea0 0x7fd4a406c360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:47:58.615 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:58.613+0000 7fd49ffff640 1 -- 172.21.15.43:0/513009588 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(62..62 src has 1..62) v4 ==== 7589+0+0 (secure 0 0 0) 0x7fd4a80ab4a0 con 0x7fd4b80b45f0 2023-08-09T13:47:58.615 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:58.613+0000 7fd4b6ffd640 1 --2- 172.21.15.43:0/513009588 >> [v2:172.21.15.203:6832/272917756,v1:172.21.15.203:6833/272917756] conn(0x7fd4a4069ea0 0x7fd4a406c360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:47:58.616 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:58.614+0000 7fd4b6ffd640 1 --2- 172.21.15.43:0/513009588 >> [v2:172.21.15.203:6832/272917756,v1:172.21.15.203:6833/272917756] conn(0x7fd4a4069ea0 0x7fd4a406c360 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7fd4b80cc510 tx=0x7fd4ac009040 comp rx=0 tx=0).ready entity=mgr.5533 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-09T13:47:58.619 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:58.617+0000 7fd49ffff640 1 -- 172.21.15.43:0/513009588 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178066 (secure 0 0 0) 0x7fd4a8076920 con 0x7fd4b80b45f0 2023-08-09T13:47:58.814 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:58.812+0000 7fd4bde8e640 1 -- 172.21.15.43:0/513009588 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7fd4b8067560 con 0x7fd4b80b45f0 2023-08-09T13:47:58.815 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:58.813+0000 7fd49ffff640 1 -- 172.21.15.43:0/513009588 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v62) v1 ==== 93+0+10 (secure 0 0 0) 0x7fd4a8079f40 con 0x7fd4b80b45f0 2023-08-09T13:47:58.815 INFO:teuthology.orchestra.run.smithi043.stdout:pg_num: 1 2023-08-09T13:47:58.818 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:58.815+0000 7fd4bde8e640 1 -- 172.21.15.43:0/513009588 >> [v2:172.21.15.203:6832/272917756,v1:172.21.15.203:6833/272917756] conn(0x7fd4a4069ea0 msgr2=0x7fd4a406c360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:47:58.818 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:58.815+0000 7fd4bde8e640 1 --2- 172.21.15.43:0/513009588 >> [v2:172.21.15.203:6832/272917756,v1:172.21.15.203:6833/272917756] conn(0x7fd4a4069ea0 0x7fd4a406c360 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7fd4b80cc510 tx=0x7fd4ac009040 comp rx=0 tx=0).stop 2023-08-09T13:47:58.818 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:58.815+0000 7fd4bde8e640 1 -- 172.21.15.43:0/513009588 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fd4b80b45f0 msgr2=0x7fd4b814b1f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:47:58.818 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:58.815+0000 7fd4bde8e640 1 --2- 172.21.15.43:0/513009588 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fd4b80b45f0 0x7fd4b814b1f0 secure :-1 s=READY pgs=759 cs=0 l=1 rev1=1 crypto rx=0x7fd4a8009e50 tx=0x7fd4a8004a40 comp rx=0 tx=0).stop 2023-08-09T13:47:58.819 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:58.815+0000 7fd4bde8e640 1 -- 172.21.15.43:0/513009588 shutdown_connections 2023-08-09T13:47:58.819 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:58.815+0000 7fd4bde8e640 1 --2- 172.21.15.43:0/513009588 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fd4b815c550 0x7fd4b815e1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:47:58.819 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:58.815+0000 7fd4bde8e640 1 --2- 172.21.15.43:0/513009588 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fd4b80d04b0 0x7fd4b815dc70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:47:58.819 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:58.815+0000 7fd4bde8e640 1 --2- 172.21.15.43:0/513009588 >> [v2:172.21.15.203:6832/272917756,v1:172.21.15.203:6833/272917756] conn(0x7fd4a4069ea0 0x7fd4a406c360 unknown :-1 s=CLOSED pgs=22 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:47:58.819 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:58.815+0000 7fd4bde8e640 1 --2- 172.21.15.43:0/513009588 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fd4b80b45f0 0x7fd4b814b1f0 unknown :-1 s=CLOSED pgs=759 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:47:58.819 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:58.816+0000 7fd4bde8e640 1 -- 172.21.15.43:0/513009588 >> 172.21.15.43:0/513009588 conn(0x7fd4b8071520 msgr2=0x7fd4b80d2440 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-09T13:47:58.819 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:58.816+0000 7fd4bde8e640 1 -- 172.21.15.43:0/513009588 shutdown_connections 2023-08-09T13:47:58.820 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:58.816+0000 7fd4bde8e640 1 -- 172.21.15.43:0/513009588 wait complete. 2023-08-09T13:47:58.837 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-08-09T13:47:59.068 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:59.065+0000 7ff5e359e640 1 Processor -- start 2023-08-09T13:47:59.068 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:59.066+0000 7ff5e359e640 1 -- start start 2023-08-09T13:47:59.069 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:59.067+0000 7ff5e359e640 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7ff5e415c3a0 0x7ff5e41608a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:47:59.069 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:59.067+0000 7ff5e359e640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7ff5e40b0f00 0x7ff5e40ccef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:47:59.070 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:59.067+0000 7ff5e359e640 1 -- --> v1:172.21.15.43:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff5e4160de0 con 0x7ff5e40b1450 2023-08-09T13:47:59.070 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:59.067+0000 7ff5e359e640 1 -- --> v2:172.21.15.43:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff5e40b81c0 con 0x7ff5e40b0f00 2023-08-09T13:47:59.070 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:59.067+0000 7ff5e359e640 1 -- --> v2:172.21.15.203:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff5e40b7c00 con 0x7ff5e415c3a0 2023-08-09T13:47:59.071 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:59.067+0000 7ff5e1d9b640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7ff5e40b0f00 0x7ff5e40ccef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:47:59.071 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:59.067+0000 7ff5e1d9b640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7ff5e40b0f00 0x7ff5e40ccef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:57296/0 (socket says 172.21.15.43:57296) 2023-08-09T13:47:59.071 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:59.067+0000 7ff5e1d9b640 1 -- 172.21.15.43:0/1268837061 learned_addr learned my addr 172.21.15.43:0/1268837061 (peer_addr_for_me v2:172.21.15.43:0/0) 2023-08-09T13:47:59.071 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:59.068+0000 7ff5e259c640 1 --2- 172.21.15.43:0/1268837061 >> v2:172.21.15.203:3300/0 conn(0x7ff5e415c3a0 0x7ff5e41608a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:47:59.072 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:59.068+0000 7ff5e1d9b640 1 -- 172.21.15.43:0/1268837061 >> v2:172.21.15.203:3300/0 conn(0x7ff5e415c3a0 msgr2=0x7ff5e41608a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-09T13:47:59.072 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:59.068+0000 7ff5e1d9b640 1 --2- 172.21.15.43:0/1268837061 >> v2:172.21.15.203:3300/0 conn(0x7ff5e415c3a0 0x7ff5e41608a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:47:59.072 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:59.068+0000 7ff5e1d9b640 1 -- 172.21.15.43:0/1268837061 >> v1:172.21.15.43:6789/0 conn(0x7ff5e40b1450 legacy=0x7ff5e40d0580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:47:59.072 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:59.068+0000 7ff5e1d9b640 1 -- 172.21.15.43:0/1268837061 --> v2:172.21.15.43:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff5e40b7ee0 con 0x7ff5e40b0f00 2023-08-09T13:47:59.072 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:59.068+0000 7ff5e159a640 1 -- 172.21.15.43:0/1268837061 <== mon.0 v1:172.21.15.43:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3462221014 0 0) 0x7ff5e4160de0 con 0x7ff5e40b1450 2023-08-09T13:47:59.073 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:59.069+0000 7ff5e1d9b640 1 --2- 172.21.15.43:0/1268837061 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff5e40b0f00 0x7ff5e40ccef0 secure :-1 s=READY pgs=761 cs=0 l=1 rev1=1 crypto rx=0x7ff5e40cfd30 tx=0x7ff5d402b8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6fb108dde0b7e4fc server_cookie=0 in_seq=0 out_seq=0 2023-08-09T13:47:59.073 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:59.069+0000 7ff5e159a640 1 -- 172.21.15.43:0/1268837061 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff5d4004230 con 0x7ff5e40b0f00 2023-08-09T13:47:59.074 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:59.069+0000 7ff5e159a640 1 -- 172.21.15.43:0/1268837061 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff5d4004390 con 0x7ff5e40b0f00 2023-08-09T13:47:59.074 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:59.069+0000 7ff5e159a640 1 -- 172.21.15.43:0/1268837061 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff5d4003810 con 0x7ff5e40b0f00 2023-08-09T13:47:59.074 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:59.070+0000 7ff5e359e640 1 -- 172.21.15.43:0/1268837061 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff5e40b0f00 msgr2=0x7ff5e40ccef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:47:59.074 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:59.070+0000 7ff5e359e640 1 --2- 172.21.15.43:0/1268837061 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff5e40b0f00 0x7ff5e40ccef0 secure :-1 s=READY pgs=761 cs=0 l=1 rev1=1 crypto rx=0x7ff5e40cfd30 tx=0x7ff5d402b8f0 comp rx=0 tx=0).stop 2023-08-09T13:47:59.074 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:59.070+0000 7ff5e359e640 1 -- 172.21.15.43:0/1268837061 shutdown_connections 2023-08-09T13:47:59.074 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:59.070+0000 7ff5e359e640 1 --2- 172.21.15.43:0/1268837061 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff5e40b0f00 0x7ff5e40ccef0 unknown :-1 s=CLOSED pgs=761 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:47:59.074 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:59.070+0000 7ff5e359e640 1 --2- 172.21.15.43:0/1268837061 >> v2:172.21.15.203:3300/0 conn(0x7ff5e415c3a0 0x7ff5e41608a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:47:59.075 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:59.070+0000 7ff5e359e640 1 -- 172.21.15.43:0/1268837061 >> 172.21.15.43:0/1268837061 conn(0x7ff5e406a210 msgr2=0x7ff5e406a620 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-09T13:47:59.075 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:59.070+0000 7ff5e359e640 1 -- 172.21.15.43:0/1268837061 shutdown_connections 2023-08-09T13:47:59.075 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:59.070+0000 7ff5e359e640 1 -- 172.21.15.43:0/1268837061 wait complete. 2023-08-09T13:47:59.075 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:59.072+0000 7ff5e359e640 1 Processor -- start 2023-08-09T13:47:59.075 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:59.072+0000 7ff5e359e640 1 -- start start 2023-08-09T13:47:59.076 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:59.073+0000 7ff5e359e640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff5e40b0f00 0x7ff5e4152de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:47:59.077 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:59.073+0000 7ff5e359e640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7ff5e40b1450 0x7ff5e4158dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:47:59.077 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:59.073+0000 7ff5e359e640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7ff5e415c3a0 0x7ff5e4159310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:47:59.077 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:59.073+0000 7ff5e359e640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff5e4153750 con 0x7ff5e40b0f00 2023-08-09T13:47:59.077 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:59.073+0000 7ff5e359e640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff5e4153a30 con 0x7ff5e40b1450 2023-08-09T13:47:59.077 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:59.073+0000 7ff5e359e640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff5e4153d10 con 0x7ff5e415c3a0 2023-08-09T13:47:59.077 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:59.074+0000 7ff5e259c640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff5e40b0f00 0x7ff5e4152de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:47:59.078 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:59.074+0000 7ff5e259c640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff5e40b0f00 0x7ff5e4152de0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:57310/0 (socket says 172.21.15.43:57310) 2023-08-09T13:47:59.078 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:59.074+0000 7ff5e259c640 1 -- 172.21.15.43:0/2197064885 learned_addr learned my addr 172.21.15.43:0/2197064885 (peer_addr_for_me v2:172.21.15.43:0/0) 2023-08-09T13:47:59.078 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:59.074+0000 7ff5e259c640 1 -- 172.21.15.43:0/2197064885 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7ff5e415c3a0 msgr2=0x7ff5e4159310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:47:59.078 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:59.074+0000 7ff5e1d9b640 1 --2- 172.21.15.43:0/2197064885 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7ff5e40b1450 0x7ff5e4158dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:47:59.078 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:59.074+0000 7ff5e259c640 1 --2- 172.21.15.43:0/2197064885 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7ff5e415c3a0 0x7ff5e4159310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:47:59.078 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:59.074+0000 7ff5e259c640 1 -- 172.21.15.43:0/2197064885 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7ff5e40b1450 msgr2=0x7ff5e4158dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:47:59.078 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:59.074+0000 7ff5e259c640 1 --2- 172.21.15.43:0/2197064885 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7ff5e40b1450 0x7ff5e4158dd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:47:59.078 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:59.074+0000 7ff5e259c640 1 -- 172.21.15.43:0/2197064885 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff5d4009460 con 0x7ff5e40b0f00 2023-08-09T13:47:59.079 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:59.074+0000 7ff5e259c640 1 --2- 172.21.15.43:0/2197064885 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff5e40b0f00 0x7ff5e4152de0 secure :-1 s=READY pgs=762 cs=0 l=1 rev1=1 crypto rx=0x7ff5cc00d5f0 tx=0x7ff5cc00dab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-09T13:47:59.080 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:59.075+0000 7ff5daffd640 1 -- 172.21.15.43:0/2197064885 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff5cc0125f0 con 0x7ff5e40b0f00 2023-08-09T13:47:59.080 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:59.075+0000 7ff5daffd640 1 -- 172.21.15.43:0/2197064885 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff5cc012750 con 0x7ff5e40b0f00 2023-08-09T13:47:59.080 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:59.075+0000 7ff5daffd640 1 -- 172.21.15.43:0/2197064885 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff5cc012b00 con 0x7ff5e40b0f00 2023-08-09T13:47:59.080 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:59.075+0000 7ff5e359e640 1 -- 172.21.15.43:0/2197064885 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff5e4153ff0 con 0x7ff5e40b0f00 2023-08-09T13:47:59.080 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:59.075+0000 7ff5e359e640 1 -- 172.21.15.43:0/2197064885 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff5e41f1a60 con 0x7ff5e40b0f00 2023-08-09T13:47:59.080 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:59.076+0000 7ff5e359e640 1 -- 172.21.15.43:0/2197064885 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff5e404e390 con 0x7ff5e40b0f00 2023-08-09T13:47:59.080 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:59.077+0000 7ff5daffd640 1 -- 172.21.15.43:0/2197064885 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 16) v1 ==== 88179+0+0 (secure 0 0 0) 0x7ff5cc012c60 con 0x7ff5e40b0f00 2023-08-09T13:47:59.081 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:59.078+0000 7ff5daffd640 1 --2- 172.21.15.43:0/2197064885 >> [v2:172.21.15.203:6832/272917756,v1:172.21.15.203:6833/272917756] conn(0x7ff5c4069ed0 0x7ff5c406c390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:47:59.081 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:59.078+0000 7ff5daffd640 1 -- 172.21.15.43:0/2197064885 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(62..62 src has 1..62) v4 ==== 7589+0+0 (secure 0 0 0) 0x7ff5cc088d00 con 0x7ff5e40b0f00 2023-08-09T13:47:59.081 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:59.079+0000 7ff5e1d9b640 1 --2- 172.21.15.43:0/2197064885 >> [v2:172.21.15.203:6832/272917756,v1:172.21.15.203:6833/272917756] conn(0x7ff5c4069ed0 0x7ff5c406c390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:47:59.082 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:59.080+0000 7ff5e1d9b640 1 --2- 172.21.15.43:0/2197064885 >> [v2:172.21.15.203:6832/272917756,v1:172.21.15.203:6833/272917756] conn(0x7ff5c4069ed0 0x7ff5c406c390 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7ff5d4035000 tx=0x7ff5d4008030 comp rx=0 tx=0).ready entity=mgr.5533 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-09T13:47:59.084 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:59.082+0000 7ff5daffd640 1 -- 172.21.15.43:0/2197064885 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178066 (secure 0 0 0) 0x7ff5cc054060 con 0x7ff5e40b0f00 2023-08-09T13:47:59.268 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:59.266+0000 7ff5daffd640 1 -- 172.21.15.43:0/2197064885 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mgrmap(e 17) v1 ==== 88387+0+0 (secure 0 0 0) 0x7ff5cc054430 con 0x7ff5e40b0f00 2023-08-09T13:47:59.293 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:59.290+0000 7ff5e359e640 1 -- 172.21.15.43:0/2197064885 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7ff5e40cc790 con 0x7ff5e40b0f00 2023-08-09T13:47:59.294 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:59.291+0000 7ff5daffd640 1 -- 172.21.15.43:0/2197064885 <== mon.0 v2:172.21.15.43:3300/0 8 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v62) v1 ==== 104+0+11 (secure 0 0 0) 0x7ff5cc057af0 con 0x7ff5e40b0f00 2023-08-09T13:47:59.294 INFO:teuthology.orchestra.run.smithi043.stdout:pg_num: 32 2023-08-09T13:47:59.296 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:59.293+0000 7ff5e359e640 1 -- 172.21.15.43:0/2197064885 >> [v2:172.21.15.203:6832/272917756,v1:172.21.15.203:6833/272917756] conn(0x7ff5c4069ed0 msgr2=0x7ff5c406c390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:47:59.297 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:59.293+0000 7ff5e359e640 1 --2- 172.21.15.43:0/2197064885 >> [v2:172.21.15.203:6832/272917756,v1:172.21.15.203:6833/272917756] conn(0x7ff5c4069ed0 0x7ff5c406c390 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7ff5d4035000 tx=0x7ff5d4008030 comp rx=0 tx=0).stop 2023-08-09T13:47:59.297 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:59.293+0000 7ff5e359e640 1 -- 172.21.15.43:0/2197064885 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff5e40b0f00 msgr2=0x7ff5e4152de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:47:59.297 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:59.293+0000 7ff5e359e640 1 --2- 172.21.15.43:0/2197064885 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff5e40b0f00 0x7ff5e4152de0 secure :-1 s=READY pgs=762 cs=0 l=1 rev1=1 crypto rx=0x7ff5cc00d5f0 tx=0x7ff5cc00dab0 comp rx=0 tx=0).stop 2023-08-09T13:47:59.297 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:59.294+0000 7ff5e359e640 1 -- 172.21.15.43:0/2197064885 shutdown_connections 2023-08-09T13:47:59.297 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:59.294+0000 7ff5e359e640 1 --2- 172.21.15.43:0/2197064885 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7ff5e415c3a0 0x7ff5e4159310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:47:59.297 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:59.294+0000 7ff5e359e640 1 --2- 172.21.15.43:0/2197064885 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7ff5e40b1450 0x7ff5e4158dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:47:59.297 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:59.294+0000 7ff5e359e640 1 --2- 172.21.15.43:0/2197064885 >> [v2:172.21.15.203:6832/272917756,v1:172.21.15.203:6833/272917756] conn(0x7ff5c4069ed0 0x7ff5c406c390 unknown :-1 s=CLOSED pgs=23 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:47:59.298 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:59.294+0000 7ff5e359e640 1 --2- 172.21.15.43:0/2197064885 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff5e40b0f00 0x7ff5e4152de0 unknown :-1 s=CLOSED pgs=762 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:47:59.298 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:59.294+0000 7ff5e359e640 1 -- 172.21.15.43:0/2197064885 >> 172.21.15.43:0/2197064885 conn(0x7ff5e406a210 msgr2=0x7ff5e40b6d60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-09T13:47:59.298 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:59.294+0000 7ff5e359e640 1 -- 172.21.15.43:0/2197064885 shutdown_connections 2023-08-09T13:47:59.298 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:59.294+0000 7ff5e359e640 1 -- 172.21.15.43:0/2197064885 wait complete. 2023-08-09T13:47:59.316 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-08-09T13:47:59.544 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:59.542+0000 7f805dc9f640 1 Processor -- start 2023-08-09T13:47:59.545 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:59.542+0000 7f805dc9f640 1 -- start start 2023-08-09T13:47:59.545 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:59.543+0000 7f805dc9f640 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7f8058157ea0 0x7f80580c97d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:47:59.546 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:59.543+0000 7f805dc9f640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f805815af10 0x7f80580c9d10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:47:59.546 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:59.543+0000 7f805dc9f640 1 -- --> v1:172.21.15.43:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8058158280 con 0x7f80580c90a0 2023-08-09T13:47:59.546 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:59.543+0000 7f805dc9f640 1 -- --> v2:172.21.15.203:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f805815b840 con 0x7f8058157ea0 2023-08-09T13:47:59.546 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:59.543+0000 7f805dc9f640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f80580b3810 con 0x7f805815af10 2023-08-09T13:47:59.547 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:59.543+0000 7f8057fff640 1 --1- >> v1:172.21.15.43:6789/0 conn(0x7f80580c90a0 0x7f805815a800 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.43:6789/0 says I am v1:172.21.15.43:36750/0 (socket says 172.21.15.43:36750) 2023-08-09T13:47:59.547 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:59.543+0000 7f8057fff640 1 -- 172.21.15.43:0/1844023249 learned_addr learned my addr 172.21.15.43:0/1844023249 (peer_addr_for_me v1:172.21.15.43:0/0) 2023-08-09T13:47:59.548 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:59.544+0000 7f805d49e640 1 --2- 172.21.15.43:0/1844023249 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f805815af10 0x7f80580c9d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:47:59.548 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:59.544+0000 7f805cc9d640 1 --2- 172.21.15.43:0/1844023249 >> v2:172.21.15.203:3300/0 conn(0x7f8058157ea0 0x7f80580c97d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:47:59.548 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:59.544+0000 7f80577fe640 1 -- 172.21.15.43:0/1844023249 <== mon.0 v1:172.21.15.43:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 967267276 0 0) 0x7f8044001110 con 0x7f80580c90a0 2023-08-09T13:47:59.548 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:59.544+0000 7f80577fe640 1 -- 172.21.15.43:0/1844023249 --> v1:172.21.15.43:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f8048003520 con 0x7f80580c90a0 2023-08-09T13:47:59.548 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:59.544+0000 7f80577fe640 1 -- 172.21.15.43:0/1844023249 <== mon.0 v1:172.21.15.43:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1933962552 0 0) 0x7f8048003520 con 0x7f80580c90a0 2023-08-09T13:47:59.548 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:59.544+0000 7f80577fe640 1 -- 172.21.15.43:0/1844023249 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f805815af10 msgr2=0x7f80580c9d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-09T13:47:59.548 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:59.544+0000 7f80577fe640 1 --2- 172.21.15.43:0/1844023249 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f805815af10 0x7f80580c9d10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:47:59.549 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:59.544+0000 7f80577fe640 1 -- 172.21.15.43:0/1844023249 >> v2:172.21.15.203:3300/0 conn(0x7f8058157ea0 msgr2=0x7f80580c97d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-09T13:47:59.549 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:59.544+0000 7f80577fe640 1 --2- 172.21.15.43:0/1844023249 >> v2:172.21.15.203:3300/0 conn(0x7f8058157ea0 0x7f80580c97d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:47:59.549 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:59.544+0000 7f80577fe640 1 -- 172.21.15.43:0/1844023249 --> v1:172.21.15.43:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f805815b560 con 0x7f80580c90a0 2023-08-09T13:47:59.549 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:59.544+0000 7f80577fe640 1 -- 172.21.15.43:0/1844023249 <== mon.0 v1:172.21.15.43:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1285150740 0 0) 0x7f8044001800 con 0x7f80580c90a0 2023-08-09T13:47:59.549 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:59.544+0000 7f80577fe640 1 -- 172.21.15.43:0/1844023249 >> v1:172.21.15.43:6789/0 conn(0x7f80580c90a0 legacy=0x7f805815a800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:47:59.549 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:59.544+0000 7f80577fe640 1 --2- 172.21.15.43:0/1844023249 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f805815af10 0x7f80480039a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:47:59.549 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:59.544+0000 7f80577fe640 1 --2- 172.21.15.43:0/1844023249 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f8048003f20 0x7f80480047c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:47:59.549 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:59.545+0000 7f80577fe640 1 --2- 172.21.15.43:0/1844023249 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f8048004d00 0x7f8048005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:47:59.550 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:59.545+0000 7f80577fe640 1 -- 172.21.15.43:0/1844023249 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f805815b840 con 0x7f8048003f20 2023-08-09T13:47:59.550 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:59.545+0000 7f80577fe640 1 -- 172.21.15.43:0/1844023249 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f80580b3810 con 0x7f8048004d00 2023-08-09T13:47:59.550 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:59.545+0000 7f80577fe640 1 -- 172.21.15.43:0/1844023249 --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f804800fec0 con 0x7f805815af10 2023-08-09T13:47:59.550 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:59.545+0000 7f8057fff640 1 --2- 172.21.15.43:0/1844023249 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f8048003f20 0x7f80480047c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:47:59.550 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:59.545+0000 7f805cc9d640 1 --2- 172.21.15.43:0/1844023249 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f805815af10 0x7f80480039a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:47:59.551 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:59.545+0000 7f805d49e640 1 --2- 172.21.15.43:0/1844023249 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f8048004d00 0x7f8048005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:47:59.551 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:59.545+0000 7f8057fff640 1 -- 172.21.15.43:0/1844023249 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f805815af10 msgr2=0x7f80480039a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-09T13:47:59.552 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:59.545+0000 7f8057fff640 1 --2- 172.21.15.43:0/1844023249 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f805815af10 0x7f80480039a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:47:59.552 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:59.545+0000 7f8057fff640 1 -- 172.21.15.43:0/1844023249 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f8048004d00 msgr2=0x7f8048005180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-09T13:47:59.552 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:59.545+0000 7f8057fff640 1 --2- 172.21.15.43:0/1844023249 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f8048004d00 0x7f8048005180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:47:59.552 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:59.545+0000 7f8057fff640 1 -- 172.21.15.43:0/1844023249 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8048003520 con 0x7f8048003f20 2023-08-09T13:47:59.552 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:59.546+0000 7f8057fff640 1 --2- 172.21.15.43:0/1844023249 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f8048003f20 0x7f80480047c0 secure :-1 s=READY pgs=764 cs=0 l=1 rev1=1 crypto rx=0x7f804400c240 tx=0x7f80480057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1d2009cb504656da server_cookie=0 in_seq=0 out_seq=0 2023-08-09T13:47:59.552 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:59.547+0000 7f80577fe640 1 -- 172.21.15.43:0/1844023249 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8044005ad0 con 0x7f8048003f20 2023-08-09T13:47:59.552 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:59.547+0000 7f80577fe640 1 -- 172.21.15.43:0/1844023249 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8044005c50 con 0x7f8048003f20 2023-08-09T13:47:59.553 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:59.547+0000 7f80577fe640 1 -- 172.21.15.43:0/1844023249 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f804402c8b0 con 0x7f8048003f20 2023-08-09T13:47:59.553 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:59.547+0000 7f805dc9f640 1 -- 172.21.15.43:0/1844023249 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f8048003f20 msgr2=0x7f80480047c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:47:59.553 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:59.547+0000 7f805dc9f640 1 --2- 172.21.15.43:0/1844023249 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f8048003f20 0x7f80480047c0 secure :-1 s=READY pgs=764 cs=0 l=1 rev1=1 crypto rx=0x7f804400c240 tx=0x7f80480057f0 comp rx=0 tx=0).stop 2023-08-09T13:47:59.553 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:59.547+0000 7f805d49e640 1 -- 172.21.15.43:0/1844023249 reap_dead start 2023-08-09T13:47:59.553 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:59.548+0000 7f805dc9f640 1 -- 172.21.15.43:0/1844023249 shutdown_connections 2023-08-09T13:47:59.553 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:59.548+0000 7f805dc9f640 1 -- 172.21.15.43:0/1844023249 >> 172.21.15.43:0/1844023249 conn(0x7f80580cddd0 msgr2=0x7f80580ce1e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-09T13:47:59.553 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:59.548+0000 7f805dc9f640 1 -- 172.21.15.43:0/1844023249 shutdown_connections 2023-08-09T13:47:59.553 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:59.548+0000 7f805dc9f640 1 -- 172.21.15.43:0/1844023249 wait complete. 2023-08-09T13:47:59.555 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:59.550+0000 7f805dc9f640 1 Processor -- start 2023-08-09T13:47:59.555 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:59.550+0000 7f805dc9f640 1 -- start start 2023-08-09T13:47:59.555 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:59.551+0000 7f805dc9f640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f8048003f20 0x7f80580c43e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:47:59.556 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:59.551+0000 7f805dc9f640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f80580c1900 0x7f80580c1d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:47:59.556 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:59.551+0000 7f805dc9f640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f80581f0d80 0x7f80581f1230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:47:59.556 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:59.551+0000 7f805dc9f640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f80581f1ab0 con 0x7f8048003f20 2023-08-09T13:47:59.556 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:59.551+0000 7f805dc9f640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f80580c22c0 con 0x7f80580c1900 2023-08-09T13:47:59.556 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:59.551+0000 7f805dc9f640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8058144cb0 con 0x7f80581f0d80 2023-08-09T13:47:59.556 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:59.551+0000 7f805cc9d640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f8048003f20 0x7f80580c43e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:47:59.556 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:59.551+0000 7f805cc9d640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f8048003f20 0x7f80580c43e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:57336/0 (socket says 172.21.15.43:57336) 2023-08-09T13:47:59.557 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:59.551+0000 7f805cc9d640 1 -- 172.21.15.43:0/60147083 learned_addr learned my addr 172.21.15.43:0/60147083 (peer_addr_for_me v2:172.21.15.43:0/0) 2023-08-09T13:47:59.557 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:59.551+0000 7f805cc9d640 1 -- 172.21.15.43:0/60147083 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f80581f0d80 msgr2=0x7f80581f1230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:47:59.557 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:59.551+0000 7f805cc9d640 1 --2- 172.21.15.43:0/60147083 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f80581f0d80 0x7f80581f1230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:47:59.557 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:59.551+0000 7f805cc9d640 1 -- 172.21.15.43:0/60147083 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f80580c1900 msgr2=0x7f80580c1d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:47:59.557 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:59.551+0000 7f805cc9d640 1 --2- 172.21.15.43:0/60147083 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f80580c1900 0x7f80580c1d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:47:59.557 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:59.551+0000 7f805cc9d640 1 -- 172.21.15.43:0/60147083 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f80581ed4b0 con 0x7f8048003f20 2023-08-09T13:47:59.557 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:59.552+0000 7f805cc9d640 1 --2- 172.21.15.43:0/60147083 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f8048003f20 0x7f80580c43e0 secure :-1 s=READY pgs=765 cs=0 l=1 rev1=1 crypto rx=0x7f804c00f5b0 tx=0x7f804c00fa70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-09T13:47:59.557 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:59.552+0000 7f80557fa640 1 -- 172.21.15.43:0/60147083 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f804c014870 con 0x7f8048003f20 2023-08-09T13:47:59.558 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:59.552+0000 7f80557fa640 1 -- 172.21.15.43:0/60147083 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f804c0149d0 con 0x7f8048003f20 2023-08-09T13:47:59.558 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:59.552+0000 7f80557fa640 1 -- 172.21.15.43:0/60147083 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f804c014d80 con 0x7f8048003f20 2023-08-09T13:47:59.558 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:59.552+0000 7f805dc9f640 1 -- 172.21.15.43:0/60147083 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f80581ed700 con 0x7f8048003f20 2023-08-09T13:47:59.558 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:59.552+0000 7f805dc9f640 1 -- 172.21.15.43:0/60147083 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f80581edcc0 con 0x7f8048003f20 2023-08-09T13:47:59.559 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:59.553+0000 7f805dc9f640 1 -- 172.21.15.43:0/60147083 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f805804e390 con 0x7f8048003f20 2023-08-09T13:47:59.559 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:59.554+0000 7f80557fa640 1 -- 172.21.15.43:0/60147083 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 17) v1 ==== 88387+0+0 (secure 0 0 0) 0x7f804c015410 con 0x7f8048003f20 2023-08-09T13:47:59.559 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:59.555+0000 7f80557fa640 1 --2- 172.21.15.43:0/60147083 >> [v2:172.21.15.203:6832/272917756,v1:172.21.15.203:6833/272917756] conn(0x7f803c06a110 0x7f803c06c5d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:47:59.559 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:59.556+0000 7f80557fa640 1 -- 172.21.15.43:0/60147083 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(62..62 src has 1..62) v4 ==== 7589+0+0 (secure 0 0 0) 0x7f804c0894b0 con 0x7f8048003f20 2023-08-09T13:47:59.559 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:59.556+0000 7f8057fff640 1 --2- 172.21.15.43:0/60147083 >> [v2:172.21.15.203:6832/272917756,v1:172.21.15.203:6833/272917756] conn(0x7f803c06a110 0x7f803c06c5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:47:59.560 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:59.557+0000 7f8057fff640 1 --2- 172.21.15.43:0/60147083 >> [v2:172.21.15.203:6832/272917756,v1:172.21.15.203:6833/272917756] conn(0x7f803c06a110 0x7f803c06c5d0 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7f804400c240 tx=0x7f8044007000 comp rx=0 tx=0).ready entity=mgr.5533 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-09T13:47:59.562 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:59.560+0000 7f80557fa640 1 -- 172.21.15.43:0/60147083 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178066 (secure 0 0 0) 0x7f804c0545b0 con 0x7f8048003f20 2023-08-09T13:47:59.755 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:59.752+0000 7f805dc9f640 1 -- 172.21.15.43:0/60147083 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f805815d1a0 con 0x7f8048003f20 2023-08-09T13:47:59.756 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:59.753+0000 7f80557fa640 1 -- 172.21.15.43:0/60147083 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v62) v1 ==== 100+0+11 (secure 0 0 0) 0x7f804c057bd0 con 0x7f8048003f20 2023-08-09T13:47:59.756 INFO:teuthology.orchestra.run.smithi043.stdout:pg_num: 64 2023-08-09T13:47:59.759 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:59.756+0000 7f805dc9f640 1 -- 172.21.15.43:0/60147083 >> [v2:172.21.15.203:6832/272917756,v1:172.21.15.203:6833/272917756] conn(0x7f803c06a110 msgr2=0x7f803c06c5d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:47:59.759 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:59.756+0000 7f805dc9f640 1 --2- 172.21.15.43:0/60147083 >> [v2:172.21.15.203:6832/272917756,v1:172.21.15.203:6833/272917756] conn(0x7f803c06a110 0x7f803c06c5d0 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7f804400c240 tx=0x7f8044007000 comp rx=0 tx=0).stop 2023-08-09T13:47:59.759 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:59.756+0000 7f805dc9f640 1 -- 172.21.15.43:0/60147083 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f8048003f20 msgr2=0x7f80580c43e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:47:59.759 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:59.756+0000 7f805dc9f640 1 --2- 172.21.15.43:0/60147083 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f8048003f20 0x7f80580c43e0 secure :-1 s=READY pgs=765 cs=0 l=1 rev1=1 crypto rx=0x7f804c00f5b0 tx=0x7f804c00fa70 comp rx=0 tx=0).stop 2023-08-09T13:47:59.759 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:59.756+0000 7f805dc9f640 1 -- 172.21.15.43:0/60147083 shutdown_connections 2023-08-09T13:47:59.759 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:59.756+0000 7f805dc9f640 1 --2- 172.21.15.43:0/60147083 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f80581f0d80 0x7f80581f1230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:47:59.760 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:59.756+0000 7f805dc9f640 1 --2- 172.21.15.43:0/60147083 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f80580c1900 0x7f80580c1d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:47:59.760 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:59.756+0000 7f805dc9f640 1 --2- 172.21.15.43:0/60147083 >> [v2:172.21.15.203:6832/272917756,v1:172.21.15.203:6833/272917756] conn(0x7f803c06a110 0x7f803c06c5d0 unknown :-1 s=CLOSED pgs=25 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:47:59.760 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:59.756+0000 7f805dc9f640 1 --2- 172.21.15.43:0/60147083 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f8048003f20 0x7f80580c43e0 unknown :-1 s=CLOSED pgs=765 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:47:59.760 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:59.756+0000 7f805dc9f640 1 -- 172.21.15.43:0/60147083 >> 172.21.15.43:0/60147083 conn(0x7f80580cddd0 msgr2=0x7f8058070ba0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-09T13:47:59.761 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:59.757+0000 7f805dc9f640 1 -- 172.21.15.43:0/60147083 shutdown_connections 2023-08-09T13:47:59.761 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:47:59.757+0000 7f805dc9f640 1 -- 172.21.15.43:0/60147083 wait complete. 2023-08-09T13:47:59.779 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-08-09T13:48:00.004 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:00.002+0000 7f48bd2b0640 1 Processor -- start 2023-08-09T13:48:00.005 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:00.002+0000 7f48bd2b0640 1 -- start start 2023-08-09T13:48:00.005 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:00.003+0000 7f48bd2b0640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f48b80cd6b0 0x7f48b80b4760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:48:00.006 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:00.003+0000 7f48bd2b0640 1 -- --> v1:172.21.15.43:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f48b80b7bf0 con 0x7f48b815f7b0 2023-08-09T13:48:00.006 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:00.003+0000 7f48bd2b0640 1 -- --> v1:172.21.15.203:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f48b80b7ed0 con 0x7f48b80b1050 2023-08-09T13:48:00.006 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:00.003+0000 7f48bd2b0640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f48b80d0370 con 0x7f48b80cd6b0 2023-08-09T13:48:00.006 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:00.003+0000 7f48b7fff640 1 --1- >> v1:172.21.15.43:6789/0 conn(0x7f48b815f7b0 0x7f48b80ccfa0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.43:6789/0 says I am v1:172.21.15.43:36762/0 (socket says 172.21.15.43:36762) 2023-08-09T13:48:00.007 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:00.003+0000 7f48b7fff640 1 -- 172.21.15.43:0/1388688351 learned_addr learned my addr 172.21.15.43:0/1388688351 (peer_addr_for_me v1:172.21.15.43:0/0) 2023-08-09T13:48:00.008 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:00.004+0000 7f48bcaaf640 1 --2- 172.21.15.43:0/1388688351 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f48b80cd6b0 0x7f48b80b4760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:48:00.008 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:00.004+0000 7f48b6ffd640 1 -- 172.21.15.43:0/1388688351 <== mon.0 v1:172.21.15.43:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2811408000 0 0) 0x7f48b80b7bf0 con 0x7f48b815f7b0 2023-08-09T13:48:00.008 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:00.004+0000 7f48b6ffd640 1 -- 172.21.15.43:0/1388688351 --> v1:172.21.15.43:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f48a4003520 con 0x7f48b815f7b0 2023-08-09T13:48:00.008 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:00.004+0000 7f48b6ffd640 1 -- 172.21.15.43:0/1388688351 <== mon.0 v1:172.21.15.43:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 878023861 0 0) 0x7f48a4003520 con 0x7f48b815f7b0 2023-08-09T13:48:00.009 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:00.004+0000 7f48b6ffd640 1 -- 172.21.15.43:0/1388688351 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f48b80cd6b0 msgr2=0x7f48b80b4760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-09T13:48:00.009 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:00.004+0000 7f48b6ffd640 1 --2- 172.21.15.43:0/1388688351 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f48b80cd6b0 0x7f48b80b4760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:48:00.009 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:00.004+0000 7f48b6ffd640 1 -- 172.21.15.43:0/1388688351 >> v1:172.21.15.203:6789/0 conn(0x7f48b80b1050 legacy=0x7f48b80d0e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:48:00.009 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:00.004+0000 7f48b6ffd640 1 -- 172.21.15.43:0/1388688351 --> v1:172.21.15.43:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f48b80d0650 con 0x7f48b815f7b0 2023-08-09T13:48:00.009 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:00.004+0000 7f48b6ffd640 1 -- 172.21.15.43:0/1388688351 <== mon.0 v1:172.21.15.43:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1285150740 0 0) 0x7f48a8001670 con 0x7f48b815f7b0 2023-08-09T13:48:00.009 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:00.004+0000 7f48b6ffd640 1 -- 172.21.15.43:0/1388688351 >> v1:172.21.15.43:6789/0 conn(0x7f48b815f7b0 legacy=0x7f48b80ccfa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:48:00.009 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:00.005+0000 7f48b6ffd640 1 --2- 172.21.15.43:0/1388688351 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f48a40039a0 0x7f48a4003dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:48:00.010 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:00.005+0000 7f48b6ffd640 1 --2- 172.21.15.43:0/1388688351 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f48a4004390 0x7f48a4004860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:48:00.010 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:00.005+0000 7f48b6ffd640 1 --2- 172.21.15.43:0/1388688351 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f48a4005060 0x7f48a400b980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:48:00.010 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:00.005+0000 7f48b6ffd640 1 -- 172.21.15.43:0/1388688351 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f48b80d0370 con 0x7f48a4005060 2023-08-09T13:48:00.010 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:00.005+0000 7f48b6ffd640 1 -- 172.21.15.43:0/1388688351 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f48a400bf70 con 0x7f48a4004390 2023-08-09T13:48:00.010 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:00.005+0000 7f48b6ffd640 1 -- 172.21.15.43:0/1388688351 --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f48a400c130 con 0x7f48a40039a0 2023-08-09T13:48:00.010 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:00.005+0000 7f48bcaaf640 1 --2- 172.21.15.43:0/1388688351 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f48b80cd6b0 0x7f48b80b4760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-08-09T13:48:00.010 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:00.005+0000 7f48bcaaf640 1 --2- 172.21.15.43:0/1388688351 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f48a4005060 0x7f48a400b980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:48:00.010 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:00.005+0000 7f48b77fe640 1 --2- 172.21.15.43:0/1388688351 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f48a4004390 0x7f48a4004860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:48:00.011 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:00.005+0000 7f48bcaaf640 1 -- 172.21.15.43:0/1388688351 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f48a40039a0 msgr2=0x7f48a4003dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-09T13:48:00.011 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:00.005+0000 7f48bcaaf640 1 --2- 172.21.15.43:0/1388688351 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f48a40039a0 0x7f48a4003dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:48:00.011 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:00.005+0000 7f48bcaaf640 1 -- 172.21.15.43:0/1388688351 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f48a4004390 msgr2=0x7f48a4004860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-09T13:48:00.011 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:00.005+0000 7f48bcaaf640 1 --2- 172.21.15.43:0/1388688351 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f48a4004390 0x7f48a4004860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:48:00.011 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:00.005+0000 7f48bcaaf640 1 -- 172.21.15.43:0/1388688351 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f48a4003520 con 0x7f48a4005060 2023-08-09T13:48:00.012 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:00.007+0000 7f48bcaaf640 1 --2- 172.21.15.43:0/1388688351 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f48a4005060 0x7f48a400b980 secure :-1 s=READY pgs=767 cs=0 l=1 rev1=1 crypto rx=0x7f48b000e1f0 tx=0x7f48a40057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e5d6a080442fa2e4 server_cookie=0 in_seq=0 out_seq=0 2023-08-09T13:48:00.012 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:00.007+0000 7f48b6ffd640 1 -- 172.21.15.43:0/1388688351 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f48b002cce0 con 0x7f48a4005060 2023-08-09T13:48:00.012 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:00.007+0000 7f48b6ffd640 1 -- 172.21.15.43:0/1388688351 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f48b000f040 con 0x7f48a4005060 2023-08-09T13:48:00.013 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:00.007+0000 7f48b6ffd640 1 -- 172.21.15.43:0/1388688351 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f48b0003bb0 con 0x7f48a4005060 2023-08-09T13:48:00.013 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:00.007+0000 7f48bd2b0640 1 -- 172.21.15.43:0/1388688351 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f48a4005060 msgr2=0x7f48a400b980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:48:00.013 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:00.007+0000 7f48bd2b0640 1 --2- 172.21.15.43:0/1388688351 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f48a4005060 0x7f48a400b980 secure :-1 s=READY pgs=767 cs=0 l=1 rev1=1 crypto rx=0x7f48b000e1f0 tx=0x7f48a40057f0 comp rx=0 tx=0).stop 2023-08-09T13:48:00.013 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:00.007+0000 7f48bcaaf640 1 -- 172.21.15.43:0/1388688351 reap_dead start 2023-08-09T13:48:00.013 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:00.008+0000 7f48bd2b0640 1 -- 172.21.15.43:0/1388688351 shutdown_connections 2023-08-09T13:48:00.013 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:00.008+0000 7f48bd2b0640 1 -- 172.21.15.43:0/1388688351 >> 172.21.15.43:0/1388688351 conn(0x7f48b806a2b0 msgr2=0x7f48b806a690 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-09T13:48:00.013 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:00.008+0000 7f48bd2b0640 1 -- 172.21.15.43:0/1388688351 shutdown_connections 2023-08-09T13:48:00.014 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:00.008+0000 7f48bd2b0640 1 -- 172.21.15.43:0/1388688351 wait complete. 2023-08-09T13:48:00.014 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:00.009+0000 7f48bd2b0640 1 Processor -- start 2023-08-09T13:48:00.014 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:00.010+0000 7f48bd2b0640 1 -- start start 2023-08-09T13:48:00.015 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:00.011+0000 7f48bd2b0640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f48a4005060 0x7f48b8153970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:48:00.015 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:00.011+0000 7f48bd2b0640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f48b8151cb0 0x7f48b80b2bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:48:00.016 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:00.011+0000 7f48bd2b0640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f48b80b3110 0x7f48b8156230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:48:00.016 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:00.011+0000 7f48bd2b0640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f48b8153550 con 0x7f48b80b3110 2023-08-09T13:48:00.016 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:00.011+0000 7f48bd2b0640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f48b80b35c0 con 0x7f48b8151cb0 2023-08-09T13:48:00.016 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:00.011+0000 7f48bd2b0640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f48b8152c40 con 0x7f48a4005060 2023-08-09T13:48:00.016 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:00.011+0000 7f48bcaaf640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f48b80b3110 0x7f48b8156230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:48:00.016 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:00.011+0000 7f48bcaaf640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f48b80b3110 0x7f48b8156230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:57352/0 (socket says 172.21.15.43:57352) 2023-08-09T13:48:00.016 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:00.011+0000 7f48bcaaf640 1 -- 172.21.15.43:0/2807555445 learned_addr learned my addr 172.21.15.43:0/2807555445 (peer_addr_for_me v2:172.21.15.43:0/0) 2023-08-09T13:48:00.017 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:00.011+0000 7f48b7fff640 1 --2- 172.21.15.43:0/2807555445 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f48a4005060 0x7f48b8153970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:48:00.017 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:00.011+0000 7f48bcaaf640 1 -- 172.21.15.43:0/2807555445 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f48a4005060 msgr2=0x7f48b8153970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:48:00.017 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:00.011+0000 7f48b77fe640 1 --2- 172.21.15.43:0/2807555445 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f48b8151cb0 0x7f48b80b2bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:48:00.017 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:00.011+0000 7f48bcaaf640 1 --2- 172.21.15.43:0/2807555445 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f48a4005060 0x7f48b8153970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:48:00.017 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:00.011+0000 7f48bcaaf640 1 -- 172.21.15.43:0/2807555445 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f48b8151cb0 msgr2=0x7f48b80b2bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:48:00.017 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:00.011+0000 7f48bcaaf640 1 --2- 172.21.15.43:0/2807555445 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f48b8151cb0 0x7f48b80b2bd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:48:00.017 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:00.011+0000 7f48bcaaf640 1 -- 172.21.15.43:0/2807555445 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f48b8156f80 con 0x7f48b80b3110 2023-08-09T13:48:00.017 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:00.011+0000 7f48b7fff640 1 --2- 172.21.15.43:0/2807555445 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f48a4005060 0x7f48b8153970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-08-09T13:48:00.018 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:00.012+0000 7f48bcaaf640 1 --2- 172.21.15.43:0/2807555445 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f48b80b3110 0x7f48b8156230 secure :-1 s=READY pgs=768 cs=0 l=1 rev1=1 crypto rx=0x7f48b0036040 tx=0x7f48b0009750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-09T13:48:00.018 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:00.012+0000 7f48b4ff9640 1 -- 172.21.15.43:0/2807555445 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f48b0004240 con 0x7f48b80b3110 2023-08-09T13:48:00.018 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:00.012+0000 7f48b4ff9640 1 -- 172.21.15.43:0/2807555445 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f48b00043c0 con 0x7f48b80b3110 2023-08-09T13:48:00.018 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:00.012+0000 7f48b4ff9640 1 -- 172.21.15.43:0/2807555445 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f48b0004770 con 0x7f48b80b3110 2023-08-09T13:48:00.018 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:00.012+0000 7f48bd2b0640 1 -- 172.21.15.43:0/2807555445 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f48b8152e90 con 0x7f48b80b3110 2023-08-09T13:48:00.018 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:00.012+0000 7f48bd2b0640 1 -- 172.21.15.43:0/2807555445 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f48b8152110 con 0x7f48b80b3110 2023-08-09T13:48:00.019 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:00.014+0000 7f48bd2b0640 1 -- 172.21.15.43:0/2807555445 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f48b8072ae0 con 0x7f48b80b3110 2023-08-09T13:48:00.019 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:00.014+0000 7f48b4ff9640 1 -- 172.21.15.43:0/2807555445 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 17) v1 ==== 88387+0+0 (secure 0 0 0) 0x7f48b004b460 con 0x7f48b80b3110 2023-08-09T13:48:00.019 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:00.015+0000 7f48b4ff9640 1 --2- 172.21.15.43:0/2807555445 >> [v2:172.21.15.203:6832/272917756,v1:172.21.15.203:6833/272917756] conn(0x7f489806a0e0 0x7f489806c5a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:48:00.019 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:00.015+0000 7f48b4ff9640 1 -- 172.21.15.43:0/2807555445 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(62..62 src has 1..62) v4 ==== 7589+0+0 (secure 0 0 0) 0x7f48b00ab5f0 con 0x7f48b80b3110 2023-08-09T13:48:00.020 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:00.016+0000 7f48b7fff640 1 --2- 172.21.15.43:0/2807555445 >> [v2:172.21.15.203:6832/272917756,v1:172.21.15.203:6833/272917756] conn(0x7f489806a0e0 0x7f489806c5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:48:00.020 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:00.017+0000 7f48b7fff640 1 --2- 172.21.15.43:0/2807555445 >> [v2:172.21.15.203:6832/272917756,v1:172.21.15.203:6833/272917756] conn(0x7f489806a0e0 0x7f489806c5a0 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7f48a8001eb0 tx=0x7f48a8009040 comp rx=0 tx=0).ready entity=mgr.5533 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-09T13:48:00.022 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:00.020+0000 7f48b4ff9640 1 -- 172.21.15.43:0/2807555445 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178066 (secure 0 0 0) 0x7f48b8072ae0 con 0x7f48b80b3110 2023-08-09T13:48:00.258 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:00.255+0000 7f48bd2b0640 1 -- 172.21.15.43:0/2807555445 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f48b8153690 con 0x7f48b80b3110 2023-08-09T13:48:00.259 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:00.256+0000 7f48b4ff9640 1 -- 172.21.15.43:0/2807555445 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 17 v17) v1 ==== 94+0+4878 (secure 0 0 0) 0x7f48b8153690 con 0x7f48b80b3110 2023-08-09T13:48:00.259 INFO:teuthology.orchestra.run.smithi043.stdout: 2023-08-09T13:48:00.259 INFO:teuthology.orchestra.run.smithi043.stdout:{"epoch":17,"default_fscid":2,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4919,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.203:6836/302214806","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6834","nonce":302214806},{"type":"v1","addr":"172.21.15.203:6836","nonce":302214806}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":12},{"gid":4922,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.203:6837/1677639100","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6835","nonce":1677639100},{"type":"v1","addr":"172.21.15.203:6837","nonce":1677639100}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":12},{"gid":5619,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.43:6834/2339480142","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6832","nonce":2339480142},{"type":"v1","addr":"172.21.15.43:6834","nonce":2339480142}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":12}],"filesystems":[{"mdsmap":{"epoch":17,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-08-09T13:36:11.336926+0000","modified":"2023-08-09T13:36:14.905554+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":5622},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5622":{"gid":5622,"name":"c","rank":0,"incarnation":13,"state":"up:active","state_seq":4,"addr":"172.21.15.43:6835/2019295969","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6833","nonce":2019295969},{"type":"v1","addr":"172.21.15.43:6835","nonce":2019295969}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[7],"metadata_pool":6,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":2}]} 2023-08-09T13:48:00.260 INFO:teuthology.orchestra.run.smithi043.stderr:dumped fsmap epoch 17 2023-08-09T13:48:00.261 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:00.259+0000 7f48bd2b0640 1 -- 172.21.15.43:0/2807555445 >> [v2:172.21.15.203:6832/272917756,v1:172.21.15.203:6833/272917756] conn(0x7f489806a0e0 msgr2=0x7f489806c5a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:48:00.261 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:00.259+0000 7f48bd2b0640 1 --2- 172.21.15.43:0/2807555445 >> [v2:172.21.15.203:6832/272917756,v1:172.21.15.203:6833/272917756] conn(0x7f489806a0e0 0x7f489806c5a0 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7f48a8001eb0 tx=0x7f48a8009040 comp rx=0 tx=0).stop 2023-08-09T13:48:00.262 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:00.259+0000 7f48bd2b0640 1 -- 172.21.15.43:0/2807555445 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f48b80b3110 msgr2=0x7f48b8156230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:48:00.262 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:00.259+0000 7f48bd2b0640 1 --2- 172.21.15.43:0/2807555445 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f48b80b3110 0x7f48b8156230 secure :-1 s=READY pgs=768 cs=0 l=1 rev1=1 crypto rx=0x7f48b0036040 tx=0x7f48b0009750 comp rx=0 tx=0).stop 2023-08-09T13:48:00.262 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:00.259+0000 7f48bd2b0640 1 -- 172.21.15.43:0/2807555445 shutdown_connections 2023-08-09T13:48:00.263 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:00.259+0000 7f48bd2b0640 1 --2- 172.21.15.43:0/2807555445 >> [v2:172.21.15.203:6832/272917756,v1:172.21.15.203:6833/272917756] conn(0x7f489806a0e0 0x7f489806c5a0 unknown :-1 s=CLOSED pgs=26 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:48:00.263 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:00.259+0000 7f48bd2b0640 1 --2- 172.21.15.43:0/2807555445 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f48b80b3110 0x7f48b8156230 unknown :-1 s=CLOSED pgs=768 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:48:00.263 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:00.259+0000 7f48bd2b0640 1 --2- 172.21.15.43:0/2807555445 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f48b8151cb0 0x7f48b80b2bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:48:00.263 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:00.259+0000 7f48bd2b0640 1 --2- 172.21.15.43:0/2807555445 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f48a4005060 0x7f48b8153970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:48:00.263 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:00.259+0000 7f48bd2b0640 1 -- 172.21.15.43:0/2807555445 >> 172.21.15.43:0/2807555445 conn(0x7f48b806a2b0 msgr2=0x7f48b80d14d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-09T13:48:00.264 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:00.260+0000 7f48bd2b0640 1 -- 172.21.15.43:0/2807555445 shutdown_connections 2023-08-09T13:48:00.264 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:00.260+0000 7f48bd2b0640 1 -- 172.21.15.43:0/2807555445 wait complete. 2023-08-09T13:48:00.282 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-08-09T13:48:00.505 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:00.502+0000 7efe634cc640 1 Processor -- start 2023-08-09T13:48:00.505 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:00.502+0000 7efe634cc640 1 -- start start 2023-08-09T13:48:00.506 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:00.504+0000 7efe634cc640 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7efe5c15ce40 0x7efe5c0c96e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:48:00.506 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:00.504+0000 7efe634cc640 1 -- --> v1:172.21.15.43:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7efe5c143080 con 0x7efe5c0cd510 2023-08-09T13:48:00.507 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:00.504+0000 7efe634cc640 1 -- --> v1:172.21.15.203:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7efe5c06d400 con 0x7efe5c142ca0 2023-08-09T13:48:00.507 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:00.504+0000 7efe634cc640 1 -- --> v2:172.21.15.203:3300/0 -- mon_getmap magic: 0 v1 -- 0x7efe5c0c9d90 con 0x7efe5c15ce40 2023-08-09T13:48:00.508 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:00.504+0000 7efe60a67640 1 --1- >> v1:172.21.15.43:6789/0 conn(0x7efe5c0cd510 0x7efe5c0c3290 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.43:6789/0 says I am v1:172.21.15.43:36768/0 (socket says 172.21.15.43:36768) 2023-08-09T13:48:00.508 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:00.504+0000 7efe60a67640 1 -- 172.21.15.43:0/1936207529 learned_addr learned my addr 172.21.15.43:0/1936207529 (peer_addr_for_me v1:172.21.15.43:0/0) 2023-08-09T13:48:00.508 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:00.505+0000 7efe61a69640 1 --2- 172.21.15.43:0/1936207529 >> v2:172.21.15.203:3300/0 conn(0x7efe5c15ce40 0x7efe5c0c96e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:48:00.509 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:00.505+0000 7efe53fff640 1 -- 172.21.15.43:0/1936207529 <== mon.0 v1:172.21.15.43:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 319083869 0 0) 0x7efe5c143080 con 0x7efe5c0cd510 2023-08-09T13:48:00.509 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:00.505+0000 7efe53fff640 1 -- 172.21.15.43:0/1936207529 --> v1:172.21.15.43:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7efe48003520 con 0x7efe5c0cd510 2023-08-09T13:48:00.509 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:00.505+0000 7efe53fff640 1 -- 172.21.15.43:0/1936207529 <== mon.0 v1:172.21.15.43:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3150146130 0 0) 0x7efe48003520 con 0x7efe5c0cd510 2023-08-09T13:48:00.509 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:00.505+0000 7efe53fff640 1 -- 172.21.15.43:0/1936207529 >> v2:172.21.15.203:3300/0 conn(0x7efe5c15ce40 msgr2=0x7efe5c0c96e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-09T13:48:00.509 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:00.505+0000 7efe53fff640 1 --2- 172.21.15.43:0/1936207529 >> v2:172.21.15.203:3300/0 conn(0x7efe5c15ce40 0x7efe5c0c96e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:48:00.509 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:00.505+0000 7efe53fff640 1 -- 172.21.15.43:0/1936207529 >> v1:172.21.15.203:6789/0 conn(0x7efe5c142ca0 legacy=0x7efe5c0cce00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:48:00.509 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:00.505+0000 7efe53fff640 1 -- 172.21.15.43:0/1936207529 --> v1:172.21.15.43:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efe5c0ca070 con 0x7efe5c0cd510 2023-08-09T13:48:00.510 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:00.505+0000 7efe53fff640 1 -- 172.21.15.43:0/1936207529 <== mon.0 v1:172.21.15.43:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1285150740 0 0) 0x7efe4c001670 con 0x7efe5c0cd510 2023-08-09T13:48:00.510 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:00.505+0000 7efe53fff640 1 -- 172.21.15.43:0/1936207529 >> v1:172.21.15.43:6789/0 conn(0x7efe5c0cd510 legacy=0x7efe5c0c3290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:48:00.510 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:00.505+0000 7efe53fff640 1 --2- 172.21.15.43:0/1936207529 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7efe480039e0 0x7efe48003e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:48:00.510 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:00.506+0000 7efe53fff640 1 --2- 172.21.15.43:0/1936207529 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7efe480047c0 0x7efe48004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:48:00.510 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:00.506+0000 7efe53fff640 1 --2- 172.21.15.43:0/1936207529 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7efe480051f0 0x7efe4800b980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:48:00.510 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:00.506+0000 7efe53fff640 1 -- 172.21.15.43:0/1936207529 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efe5c0c9d90 con 0x7efe480051f0 2023-08-09T13:48:00.510 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:00.506+0000 7efe53fff640 1 -- 172.21.15.43:0/1936207529 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efe4800c150 con 0x7efe480047c0 2023-08-09T13:48:00.510 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:00.506+0000 7efe53fff640 1 -- 172.21.15.43:0/1936207529 --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7efe4800c310 con 0x7efe480039e0 2023-08-09T13:48:00.511 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:00.506+0000 7efe61a69640 1 --2- 172.21.15.43:0/1936207529 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7efe480051f0 0x7efe4800b980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:48:00.512 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:00.506+0000 7efe61268640 1 --2- 172.21.15.43:0/1936207529 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7efe480039e0 0x7efe48003e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:48:00.512 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:00.506+0000 7efe60a67640 1 --2- 172.21.15.43:0/1936207529 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7efe480047c0 0x7efe48004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:48:00.512 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:00.506+0000 7efe61a69640 1 -- 172.21.15.43:0/1936207529 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7efe480039e0 msgr2=0x7efe48003e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-09T13:48:00.512 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:00.506+0000 7efe61a69640 1 --2- 172.21.15.43:0/1936207529 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7efe480039e0 0x7efe48003e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:48:00.513 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:00.506+0000 7efe61a69640 1 -- 172.21.15.43:0/1936207529 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7efe480047c0 msgr2=0x7efe48004c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-09T13:48:00.513 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:00.506+0000 7efe61a69640 1 --2- 172.21.15.43:0/1936207529 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7efe480047c0 0x7efe48004c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:48:00.513 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:00.506+0000 7efe61a69640 1 -- 172.21.15.43:0/1936207529 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efe48003520 con 0x7efe480051f0 2023-08-09T13:48:00.513 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:00.507+0000 7efe61a69640 1 -- 172.21.15.43:0/1936207529 reap_dead start 2023-08-09T13:48:00.513 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:00.508+0000 7efe61a69640 1 --2- 172.21.15.43:0/1936207529 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7efe480051f0 0x7efe4800b980 secure :-1 s=READY pgs=770 cs=0 l=1 rev1=1 crypto rx=0x7efe54007360 tx=0x7efe480057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5e700a6421b5d605 server_cookie=0 in_seq=0 out_seq=0 2023-08-09T13:48:00.513 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:00.508+0000 7efe53fff640 1 -- 172.21.15.43:0/1936207529 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efe5402ccf0 con 0x7efe480051f0 2023-08-09T13:48:00.513 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:00.508+0000 7efe53fff640 1 -- 172.21.15.43:0/1936207529 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efe5402ce50 con 0x7efe480051f0 2023-08-09T13:48:00.513 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:00.508+0000 7efe53fff640 1 -- 172.21.15.43:0/1936207529 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efe54003bd0 con 0x7efe480051f0 2023-08-09T13:48:00.514 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:00.508+0000 7efe634cc640 1 -- 172.21.15.43:0/1936207529 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7efe480051f0 msgr2=0x7efe4800b980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:48:00.514 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:00.508+0000 7efe634cc640 1 --2- 172.21.15.43:0/1936207529 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7efe480051f0 0x7efe4800b980 secure :-1 s=READY pgs=770 cs=0 l=1 rev1=1 crypto rx=0x7efe54007360 tx=0x7efe480057f0 comp rx=0 tx=0).stop 2023-08-09T13:48:00.514 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:00.509+0000 7efe634cc640 1 -- 172.21.15.43:0/1936207529 shutdown_connections 2023-08-09T13:48:00.514 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:00.509+0000 7efe634cc640 1 --2- 172.21.15.43:0/1936207529 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7efe480051f0 0x7efe4800b980 unknown :-1 s=CLOSED pgs=770 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:48:00.514 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:00.509+0000 7efe634cc640 1 -- 172.21.15.43:0/1936207529 >> 172.21.15.43:0/1936207529 conn(0x7efe5c0ce0a0 msgr2=0x7efe5c0ce480 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-09T13:48:00.514 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:00.509+0000 7efe634cc640 1 -- 172.21.15.43:0/1936207529 shutdown_connections 2023-08-09T13:48:00.514 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:00.509+0000 7efe634cc640 1 -- 172.21.15.43:0/1936207529 wait complete. 2023-08-09T13:48:00.516 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:00.511+0000 7efe634cc640 1 Processor -- start 2023-08-09T13:48:00.516 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:00.511+0000 7efe634cc640 1 -- start start 2023-08-09T13:48:00.516 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:00.512+0000 7efe634cc640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7efe480051f0 0x7efe5c0c6690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:48:00.516 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:00.512+0000 7efe634cc640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7efe5c0c7d10 0x7efe5c13cdd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:48:00.517 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:00.512+0000 7efe634cc640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7efe5c0b2170 0x7efe5c0c3190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:48:00.517 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:00.512+0000 7efe634cc640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efe5c13d310 con 0x7efe5c0c7d10 2023-08-09T13:48:00.517 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:00.512+0000 7efe634cc640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efe5c0c6c70 con 0x7efe5c0b2170 2023-08-09T13:48:00.517 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:00.512+0000 7efe634cc640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7efe5c0c6f50 con 0x7efe480051f0 2023-08-09T13:48:00.517 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:00.512+0000 7efe60a67640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7efe5c0c7d10 0x7efe5c13cdd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:48:00.517 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:00.512+0000 7efe60a67640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7efe5c0c7d10 0x7efe5c13cdd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:57356/0 (socket says 172.21.15.43:57356) 2023-08-09T13:48:00.517 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:00.512+0000 7efe60a67640 1 -- 172.21.15.43:0/3915359189 learned_addr learned my addr 172.21.15.43:0/3915359189 (peer_addr_for_me v2:172.21.15.43:0/0) 2023-08-09T13:48:00.517 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:00.513+0000 7efe61268640 1 --2- 172.21.15.43:0/3915359189 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7efe480051f0 0x7efe5c0c6690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:48:00.518 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:00.513+0000 7efe60a67640 1 -- 172.21.15.43:0/3915359189 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7efe480051f0 msgr2=0x7efe5c0c6690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:48:00.518 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:00.513+0000 7efe60a67640 1 --2- 172.21.15.43:0/3915359189 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7efe480051f0 0x7efe5c0c6690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:48:00.518 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:00.513+0000 7efe61a69640 1 --2- 172.21.15.43:0/3915359189 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7efe5c0b2170 0x7efe5c0c3190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:48:00.518 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:00.513+0000 7efe60a67640 1 -- 172.21.15.43:0/3915359189 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7efe5c0b2170 msgr2=0x7efe5c0c3190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:48:00.518 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:00.513+0000 7efe60a67640 1 --2- 172.21.15.43:0/3915359189 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7efe5c0b2170 0x7efe5c0c3190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:48:00.518 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:00.513+0000 7efe60a67640 1 -- 172.21.15.43:0/3915359189 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efe5c0c7230 con 0x7efe5c0c7d10 2023-08-09T13:48:00.518 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:00.513+0000 7efe60a67640 1 --2- 172.21.15.43:0/3915359189 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7efe5c0c7d10 0x7efe5c13cdd0 secure :-1 s=READY pgs=771 cs=0 l=1 rev1=1 crypto rx=0x7efe4c00e730 tx=0x7efe4c00ebf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-09T13:48:00.519 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:00.513+0000 7efe51ffb640 1 -- 172.21.15.43:0/3915359189 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efe4c00c6c0 con 0x7efe5c0c7d10 2023-08-09T13:48:00.519 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:00.513+0000 7efe51ffb640 1 -- 172.21.15.43:0/3915359189 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efe4c00c840 con 0x7efe5c0c7d10 2023-08-09T13:48:00.519 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:00.513+0000 7efe51ffb640 1 -- 172.21.15.43:0/3915359189 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efe4c00cbf0 con 0x7efe5c0c7d10 2023-08-09T13:48:00.519 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:00.513+0000 7efe634cc640 1 -- 172.21.15.43:0/3915359189 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7efe5c0c7510 con 0x7efe5c0c7d10 2023-08-09T13:48:00.519 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:00.514+0000 7efe634cc640 1 -- 172.21.15.43:0/3915359189 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7efe5c0b25f0 con 0x7efe5c0c7d10 2023-08-09T13:48:00.520 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:00.515+0000 7efe634cc640 1 -- 172.21.15.43:0/3915359189 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7efe5c0c7a30 con 0x7efe5c0c7d10 2023-08-09T13:48:00.520 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:00.516+0000 7efe51ffb640 1 -- 172.21.15.43:0/3915359189 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 17) v1 ==== 88387+0+0 (secure 0 0 0) 0x7efe4c01d460 con 0x7efe5c0c7d10 2023-08-09T13:48:00.520 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:00.517+0000 7efe51ffb640 1 --2- 172.21.15.43:0/3915359189 >> [v2:172.21.15.203:6832/272917756,v1:172.21.15.203:6833/272917756] conn(0x7efe4006a110 0x7efe4006c5d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:48:00.520 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:00.517+0000 7efe51ffb640 1 -- 172.21.15.43:0/3915359189 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(62..62 src has 1..62) v4 ==== 7589+0+0 (secure 0 0 0) 0x7efe4c088c80 con 0x7efe5c0c7d10 2023-08-09T13:48:00.520 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:00.517+0000 7efe61268640 1 --2- 172.21.15.43:0/3915359189 >> [v2:172.21.15.203:6832/272917756,v1:172.21.15.203:6833/272917756] conn(0x7efe4006a110 0x7efe4006c5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:48:00.521 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:00.519+0000 7efe61268640 1 --2- 172.21.15.43:0/3915359189 >> [v2:172.21.15.203:6832/272917756,v1:172.21.15.203:6833/272917756] conn(0x7efe4006a110 0x7efe4006c5d0 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7efe58007fd0 tx=0x7efe58009040 comp rx=0 tx=0).ready entity=mgr.5533 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-09T13:48:00.523 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:00.521+0000 7efe51ffb640 1 -- 172.21.15.43:0/3915359189 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178066 (secure 0 0 0) 0x7efe5c0c7a30 con 0x7efe5c0c7d10 2023-08-09T13:48:00.718 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:00.715+0000 7efe634cc640 1 -- 172.21.15.43:0/3915359189 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7efe5c0c7a30 con 0x7efe5c0c7d10 2023-08-09T13:48:00.720 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:00.718+0000 7efe51ffb640 1 -- 172.21.15.43:0/3915359189 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v62) v1 ==== 74+0+20259 (secure 0 0 0) 0x7efe5c0c7a30 con 0x7efe5c0c7d10 2023-08-09T13:48:00.721 INFO:teuthology.orchestra.run.smithi043.stdout: 2023-08-09T13:48:00.721 INFO:teuthology.orchestra.run.smithi043.stdout:{"epoch":62,"fsid":"daac87ee-1337-4250-a955-aea4e408a39b","created":"2023-08-09T13:33:41.701198+0000","modified":"2023-08-09T13:47:58.221531+0000","last_up_change":"2023-08-09T13:33:53.783453+0000","last_in_change":"2023-08-09T13:33:45.148904+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":7,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":"rbd","create_time":"2023-08-09T13:33:54.365137+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":".mgr","create_time":"2023-08-09T13:33:54.831803+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":6,"pool_name":"cephfs_metadata","create_time":"2023-08-09T13:36:08.969993+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"38","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":7,"pool_name":"cephfs_data","create_time":"2023-08-09T13:36:10.366495+0000","flags":8193,"flags_names":"hashpspool,selfmanaged_snaps","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"62","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":221,"snap_epoch":62,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"cbd49ecd-e896-4fbd-864e-5c37a829b165","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":37,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6800","nonce":502667335},{"type":"v1","addr":"172.21.15.43:6801","nonce":502667335}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6802","nonce":502667335},{"type":"v1","addr":"172.21.15.43:6803","nonce":502667335}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6806","nonce":502667335},{"type":"v1","addr":"172.21.15.43:6807","nonce":502667335}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6804","nonce":502667335},{"type":"v1","addr":"172.21.15.43:6805","nonce":502667335}]},"public_addr":"172.21.15.43:6801/502667335","cluster_addr":"172.21.15.43:6803/502667335","heartbeat_back_addr":"172.21.15.43:6807/502667335","heartbeat_front_addr":"172.21.15.43:6805/502667335","state":["exists","up"]},{"osd":1,"uuid":"75c7e0fe-1ec7-4fe8-a2f2-3661a65b8500","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":37,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6808","nonce":3289134610},{"type":"v1","addr":"172.21.15.43:6809","nonce":3289134610}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6810","nonce":3289134610},{"type":"v1","addr":"172.21.15.43:6812","nonce":3289134610}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6821","nonce":3289134610},{"type":"v1","addr":"172.21.15.43:6824","nonce":3289134610}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6815","nonce":3289134610},{"type":"v1","addr":"172.21.15.43:6818","nonce":3289134610}]},"public_addr":"172.21.15.43:6809/3289134610","cluster_addr":"172.21.15.43:6812/3289134610","heartbeat_back_addr":"172.21.15.43:6824/3289134610","heartbeat_front_addr":"172.21.15.43:6818/3289134610","state":["exists","up"]},{"osd":2,"uuid":"bb5c41e9-7f5f-4125-9b01-3bb8bfc2ea83","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":37,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6811","nonce":4079709736},{"type":"v1","addr":"172.21.15.43:6814","nonce":4079709736}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6817","nonce":4079709736},{"type":"v1","addr":"172.21.15.43:6820","nonce":4079709736}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6829","nonce":4079709736},{"type":"v1","addr":"172.21.15.43:6831","nonce":4079709736}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6823","nonce":4079709736},{"type":"v1","addr":"172.21.15.43:6826","nonce":4079709736}]},"public_addr":"172.21.15.43:6814/4079709736","cluster_addr":"172.21.15.43:6820/4079709736","heartbeat_back_addr":"172.21.15.43:6831/4079709736","heartbeat_front_addr":"172.21.15.43:6826/4079709736","state":["exists","up"]},{"osd":3,"uuid":"bfe4c8b4-694a-465c-87e6-2b2aab852613","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":37,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6813","nonce":1709538759},{"type":"v1","addr":"172.21.15.43:6816","nonce":1709538759}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6819","nonce":1709538759},{"type":"v1","addr":"172.21.15.43:6822","nonce":1709538759}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6828","nonce":1709538759},{"type":"v1","addr":"172.21.15.43:6830","nonce":1709538759}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6825","nonce":1709538759},{"type":"v1","addr":"172.21.15.43:6827","nonce":1709538759}]},"public_addr":"172.21.15.43:6816/1709538759","cluster_addr":"172.21.15.43:6822/1709538759","heartbeat_back_addr":"172.21.15.43:6830/1709538759","heartbeat_front_addr":"172.21.15.43:6827/1709538759","state":["exists","up"]},{"osd":4,"uuid":"fa2e8a77-aba1-4756-853e-0d174a5bdeba","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":41,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6800","nonce":4196866481},{"type":"v1","addr":"172.21.15.203:6801","nonce":4196866481}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6802","nonce":4196866481},{"type":"v1","addr":"172.21.15.203:6803","nonce":4196866481}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6806","nonce":4196866481},{"type":"v1","addr":"172.21.15.203:6807","nonce":4196866481}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6804","nonce":4196866481},{"type":"v1","addr":"172.21.15.203:6805","nonce":4196866481}]},"public_addr":"172.21.15.203:6801/4196866481","cluster_addr":"172.21.15.203:6803/4196866481","heartbeat_back_addr":"172.21.15.203:6807/4196866481","heartbeat_front_addr":"172.21.15.203:6805/4196866481","state":["exists","up"]},{"osd":5,"uuid":"81811e4b-4d24-4f51-a82c-f1d700a2a00d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":41,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6816","nonce":3059290081},{"type":"v1","addr":"172.21.15.203:6817","nonce":3059290081}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6818","nonce":3059290081},{"type":"v1","addr":"172.21.15.203:6819","nonce":3059290081}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6822","nonce":3059290081},{"type":"v1","addr":"172.21.15.203:6823","nonce":3059290081}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6820","nonce":3059290081},{"type":"v1","addr":"172.21.15.203:6821","nonce":3059290081}]},"public_addr":"172.21.15.203:6817/3059290081","cluster_addr":"172.21.15.203:6819/3059290081","heartbeat_back_addr":"172.21.15.203:6823/3059290081","heartbeat_front_addr":"172.21.15.203:6821/3059290081","state":["exists","up"]},{"osd":6,"uuid":"307bac4d-6a5b-443d-af75-56d695d7674f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":37,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6824","nonce":512155569},{"type":"v1","addr":"172.21.15.203:6825","nonce":512155569}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6826","nonce":512155569},{"type":"v1","addr":"172.21.15.203:6827","nonce":512155569}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6830","nonce":512155569},{"type":"v1","addr":"172.21.15.203:6831","nonce":512155569}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6828","nonce":512155569},{"type":"v1","addr":"172.21.15.203:6829","nonce":512155569}]},"public_addr":"172.21.15.203:6825/512155569","cluster_addr":"172.21.15.203:6827/512155569","heartbeat_back_addr":"172.21.15.203:6831/512155569","heartbeat_front_addr":"172.21.15.203:6829/512155569","state":["exists","up"]},{"osd":7,"uuid":"a95dc7ca-9f1b-4295-af70-68a22a2d7d5e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":37,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6808","nonce":3331450438},{"type":"v1","addr":"172.21.15.203:6809","nonce":3331450438}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6810","nonce":3331450438},{"type":"v1","addr":"172.21.15.203:6811","nonce":3331450438}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6814","nonce":3331450438},{"type":"v1","addr":"172.21.15.203:6815","nonce":3331450438}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6812","nonce":3331450438},{"type":"v1","addr":"172.21.15.203:6813","nonce":3331450438}]},"public_addr":"172.21.15.203:6809/3331450438","cluster_addr":"172.21.15.203:6811/3331450438","heartbeat_back_addr":"172.21.15.203:6815/3331450438","heartbeat_front_addr":"172.21.15.203:6813/3331450438","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-09T13:33:50.547652+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-09T13:33:50.589469+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-09T13:33:50.561724+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-09T13:33:50.599650+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-09T13:33:50.411855+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-09T13:33:50.219681+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-09T13:33:50.504664+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-09T13:33:50.441248+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"7.2a","mappings":[{"from":0,"to":5}]},{"pgid":"7.3c","mappings":[{"from":0,"to":4}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.203:0/830217892":"2023-08-10T13:47:57.215872+0000","172.21.15.203:0/1493361813":"2023-08-10T13:47:57.215872+0000","172.21.15.203:0/3309257000":"2023-08-10T13:36:27.970754+0000","172.21.15.203:0/2444066336":"2023-08-10T13:47:57.215872+0000","172.21.15.203:6832/830097474":"2023-08-10T13:36:27.970754+0000","172.21.15.203:6833/830097474":"2023-08-10T13:36:27.970754+0000","172.21.15.203:0/2439593041":"2023-08-10T13:36:27.970754+0000","172.21.15.203:0/2454266731":"2023-08-10T13:47:57.215872+0000","172.21.15.203:0/1761968483":"2023-08-10T13:36:27.970754+0000","172.21.15.203:0/890377927":"2023-08-10T13:36:27.970754+0000","172.21.15.203:0/560220444":"2023-08-10T13:47:57.215872+0000","172.21.15.203:6832/998005540":"2023-08-10T13:47:57.215872+0000","172.21.15.203:6833/998005540":"2023-08-10T13:47:57.215872+0000","172.21.15.203:0/315780925":"2023-08-10T13:47:57.215872+0000","172.21.15.203:0/1813604691":"2023-08-10T13:47:57.215872+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":7,"snaps":[{"begin":4,"length":1},{"begin":8,"length":1},{"begin":10,"length":1},{"begin":15,"length":1},{"begin":18,"length":2},{"begin":25,"length":1},{"begin":27,"length":1},{"begin":31,"length":1},{"begin":34,"length":1},{"begin":40,"length":2},{"begin":48,"length":3},{"begin":55,"length":1},{"begin":57,"length":1},{"begin":59,"length":1},{"begin":62,"length":1},{"begin":65,"length":1},{"begin":68,"length":1},{"begin":72,"length":1},{"begin":74,"length":1},{"begin":77,"length":1},{"begin":83,"length":2},{"begin":87,"length":1},{"begin":95,"length":2},{"begin":98,"length":1},{"begin":104,"length":1},{"begin":107,"length":2},{"begin":111,"length":1},{"begin":188,"length":34}]}],"new_removed_snaps":[{"pool":7,"snaps":[{"begin":104,"length":1},{"begin":221,"length":1}]}],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-08-09T13:48:00.723 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:00.720+0000 7efe634cc640 1 -- 172.21.15.43:0/3915359189 >> [v2:172.21.15.203:6832/272917756,v1:172.21.15.203:6833/272917756] conn(0x7efe4006a110 msgr2=0x7efe4006c5d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:48:00.723 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:00.720+0000 7efe634cc640 1 --2- 172.21.15.43:0/3915359189 >> [v2:172.21.15.203:6832/272917756,v1:172.21.15.203:6833/272917756] conn(0x7efe4006a110 0x7efe4006c5d0 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7efe58007fd0 tx=0x7efe58009040 comp rx=0 tx=0).stop 2023-08-09T13:48:00.723 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:00.720+0000 7efe634cc640 1 -- 172.21.15.43:0/3915359189 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7efe5c0c7d10 msgr2=0x7efe5c13cdd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:48:00.723 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:00.720+0000 7efe634cc640 1 --2- 172.21.15.43:0/3915359189 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7efe5c0c7d10 0x7efe5c13cdd0 secure :-1 s=READY pgs=771 cs=0 l=1 rev1=1 crypto rx=0x7efe4c00e730 tx=0x7efe4c00ebf0 comp rx=0 tx=0).stop 2023-08-09T13:48:00.724 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:00.720+0000 7efe634cc640 1 -- 172.21.15.43:0/3915359189 shutdown_connections 2023-08-09T13:48:00.724 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:00.720+0000 7efe634cc640 1 --2- 172.21.15.43:0/3915359189 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7efe5c0b2170 0x7efe5c0c3190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:48:00.724 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:00.720+0000 7efe634cc640 1 --2- 172.21.15.43:0/3915359189 >> [v2:172.21.15.203:6832/272917756,v1:172.21.15.203:6833/272917756] conn(0x7efe4006a110 0x7efe4006c5d0 unknown :-1 s=CLOSED pgs=27 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:48:00.724 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:00.720+0000 7efe634cc640 1 --2- 172.21.15.43:0/3915359189 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7efe5c0c7d10 0x7efe5c13cdd0 unknown :-1 s=CLOSED pgs=771 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:48:00.724 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:00.720+0000 7efe634cc640 1 --2- 172.21.15.43:0/3915359189 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7efe480051f0 0x7efe5c0c6690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:48:00.724 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:00.720+0000 7efe634cc640 1 -- 172.21.15.43:0/3915359189 >> 172.21.15.43:0/3915359189 conn(0x7efe5c0ce0a0 msgr2=0x7efe5c0cce00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-09T13:48:00.725 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:00.721+0000 7efe634cc640 1 -- 172.21.15.43:0/3915359189 shutdown_connections 2023-08-09T13:48:00.725 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:00.721+0000 7efe634cc640 1 -- 172.21.15.43:0/3915359189 wait complete. 2023-08-09T13:48:00.745 INFO:tasks.cephfs.filesystem:Destroying file system cephfs and related pools 2023-08-09T13:48:00.745 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-08-09T13:48:00.964 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:00.961+0000 7fe0aea9f640 1 Processor -- start 2023-08-09T13:48:00.964 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:00.961+0000 7fe0aea9f640 1 -- start start 2023-08-09T13:48:00.965 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:00.963+0000 7fe0aea9f640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7fe0a815a6a0 0x7fe0a80713b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:48:00.965 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:00.963+0000 7fe0aea9f640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fe0a80cbd90 0x7fe0a80c8fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:48:00.966 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:00.963+0000 7fe0aea9f640 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7fe0a80c94f0 0x7fe0a815ad20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:48:00.966 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:00.963+0000 7fe0aea9f640 1 -- --> v2:172.21.15.43:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe0a80c9960 con 0x7fe0a815a6a0 2023-08-09T13:48:00.966 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:00.963+0000 7fe0aea9f640 1 -- --> v2:172.21.15.203:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe0a80cdc70 con 0x7fe0a80c94f0 2023-08-09T13:48:00.966 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:00.963+0000 7fe0aea9f640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe0a8070910 con 0x7fe0a80cbd90 2023-08-09T13:48:00.967 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:00.963+0000 7fe0ada9d640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7fe0a815a6a0 0x7fe0a80713b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:48:00.967 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:00.963+0000 7fe0ada9d640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7fe0a815a6a0 0x7fe0a80713b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:57368/0 (socket says 172.21.15.43:57368) 2023-08-09T13:48:00.967 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:00.963+0000 7fe0ada9d640 1 -- 172.21.15.43:0/2001251803 learned_addr learned my addr 172.21.15.43:0/2001251803 (peer_addr_for_me v2:172.21.15.43:0/0) 2023-08-09T13:48:00.967 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:00.964+0000 7fe0ad29c640 1 --2- 172.21.15.43:0/2001251803 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fe0a80cbd90 0x7fe0a80c8fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:48:00.967 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:00.964+0000 7fe0ae29e640 1 --2- 172.21.15.43:0/2001251803 >> v2:172.21.15.203:3300/0 conn(0x7fe0a80c94f0 0x7fe0a815ad20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:48:00.967 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:00.964+0000 7fe0ada9d640 1 -- 172.21.15.43:0/2001251803 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fe0a80cbd90 msgr2=0x7fe0a80c8fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-09T13:48:00.968 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:00.964+0000 7fe0ada9d640 1 --2- 172.21.15.43:0/2001251803 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fe0a80cbd90 0x7fe0a80c8fb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:48:00.968 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:00.964+0000 7fe0ada9d640 1 -- 172.21.15.43:0/2001251803 >> v2:172.21.15.203:3300/0 conn(0x7fe0a80c94f0 msgr2=0x7fe0a815ad20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-09T13:48:00.968 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:00.964+0000 7fe0ada9d640 1 --2- 172.21.15.43:0/2001251803 >> v2:172.21.15.203:3300/0 conn(0x7fe0a80c94f0 0x7fe0a815ad20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:48:00.968 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:00.964+0000 7fe0ada9d640 1 -- 172.21.15.43:0/2001251803 --> v2:172.21.15.43:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe0a80c88c0 con 0x7fe0a815a6a0 2023-08-09T13:48:00.969 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:00.965+0000 7fe0ada9d640 1 --2- 172.21.15.43:0/2001251803 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fe0a815a6a0 0x7fe0a80713b0 secure :-1 s=READY pgs=772 cs=0 l=1 rev1=1 crypto rx=0x7fe0a815dec0 tx=0x7fe0a402e050 comp rx=0 tx=0).ready entity=mon.0 client_cookie=928db150deb41649 server_cookie=0 in_seq=0 out_seq=0 2023-08-09T13:48:00.969 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:00.965+0000 7fe0aca9b640 1 -- 172.21.15.43:0/2001251803 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe0a402e9f0 con 0x7fe0a815a6a0 2023-08-09T13:48:00.969 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:00.965+0000 7fe0aca9b640 1 -- 172.21.15.43:0/2001251803 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe0a402eb50 con 0x7fe0a815a6a0 2023-08-09T13:48:00.969 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:00.965+0000 7fe0aca9b640 1 -- 172.21.15.43:0/2001251803 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe0a4009600 con 0x7fe0a815a6a0 2023-08-09T13:48:00.970 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:00.966+0000 7fe0aea9f640 1 -- 172.21.15.43:0/2001251803 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fe0a815a6a0 msgr2=0x7fe0a80713b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:48:00.970 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:00.966+0000 7fe0aea9f640 1 --2- 172.21.15.43:0/2001251803 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fe0a815a6a0 0x7fe0a80713b0 secure :-1 s=READY pgs=772 cs=0 l=1 rev1=1 crypto rx=0x7fe0a815dec0 tx=0x7fe0a402e050 comp rx=0 tx=0).stop 2023-08-09T13:48:00.970 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:00.966+0000 7fe0aea9f640 1 -- 172.21.15.43:0/2001251803 shutdown_connections 2023-08-09T13:48:00.970 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:00.966+0000 7fe0aea9f640 1 --2- 172.21.15.43:0/2001251803 >> v2:172.21.15.203:3300/0 conn(0x7fe0a80c94f0 0x7fe0a815ad20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:48:00.970 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:00.966+0000 7fe0aea9f640 1 --2- 172.21.15.43:0/2001251803 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fe0a80cbd90 0x7fe0a80c8fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:48:00.970 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:00.966+0000 7fe0aea9f640 1 --2- 172.21.15.43:0/2001251803 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fe0a815a6a0 0x7fe0a80713b0 unknown :-1 s=CLOSED pgs=772 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:48:00.970 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:00.966+0000 7fe0aea9f640 1 -- 172.21.15.43:0/2001251803 >> 172.21.15.43:0/2001251803 conn(0x7fe0a80cde90 msgr2=0x7fe0a80ce2a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-09T13:48:00.970 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:00.966+0000 7fe0aea9f640 1 -- 172.21.15.43:0/2001251803 shutdown_connections 2023-08-09T13:48:00.971 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:00.966+0000 7fe0aea9f640 1 -- 172.21.15.43:0/2001251803 wait complete. 2023-08-09T13:48:00.971 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:00.968+0000 7fe0aea9f640 1 Processor -- start 2023-08-09T13:48:00.971 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:00.968+0000 7fe0aea9f640 1 -- start start 2023-08-09T13:48:00.972 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:00.969+0000 7fe0aea9f640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fe0a80c94f0 0x7fe0a8099410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:48:00.972 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:00.969+0000 7fe0aea9f640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fe0a80cbd90 0x7fe0a8158ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:48:00.973 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:00.969+0000 7fe0aea9f640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fe0a815a6a0 0x7fe0a80c6440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:48:00.973 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:00.969+0000 7fe0aea9f640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe0a809a850 con 0x7fe0a80cbd90 2023-08-09T13:48:00.973 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:00.969+0000 7fe0aea9f640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe0a8099c40 con 0x7fe0a80c94f0 2023-08-09T13:48:00.973 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:00.969+0000 7fe0aea9f640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe0a8099f20 con 0x7fe0a815a6a0 2023-08-09T13:48:00.973 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:00.969+0000 7fe0ad29c640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fe0a80cbd90 0x7fe0a8158ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:48:00.973 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:00.969+0000 7fe0ad29c640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fe0a80cbd90 0x7fe0a8158ee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:57376/0 (socket says 172.21.15.43:57376) 2023-08-09T13:48:00.973 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:00.969+0000 7fe0ad29c640 1 -- 172.21.15.43:0/1267000526 learned_addr learned my addr 172.21.15.43:0/1267000526 (peer_addr_for_me v2:172.21.15.43:0/0) 2023-08-09T13:48:00.974 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:00.970+0000 7fe0ae29e640 1 --2- 172.21.15.43:0/1267000526 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fe0a815a6a0 0x7fe0a80c6440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:48:00.974 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:00.970+0000 7fe0ada9d640 1 --2- 172.21.15.43:0/1267000526 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fe0a80c94f0 0x7fe0a8099410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:48:00.974 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:00.970+0000 7fe0ad29c640 1 -- 172.21.15.43:0/1267000526 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fe0a815a6a0 msgr2=0x7fe0a80c6440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:48:00.974 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:00.970+0000 7fe0ad29c640 1 --2- 172.21.15.43:0/1267000526 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fe0a815a6a0 0x7fe0a80c6440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:48:00.974 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:00.970+0000 7fe0ad29c640 1 -- 172.21.15.43:0/1267000526 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fe0a80c94f0 msgr2=0x7fe0a8099410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:48:00.974 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:00.970+0000 7fe0ad29c640 1 --2- 172.21.15.43:0/1267000526 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fe0a80c94f0 0x7fe0a8099410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:48:00.974 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:00.970+0000 7fe0ad29c640 1 -- 172.21.15.43:0/1267000526 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe0a400bd40 con 0x7fe0a80cbd90 2023-08-09T13:48:00.975 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:00.970+0000 7fe0ad29c640 1 --2- 172.21.15.43:0/1267000526 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fe0a80cbd90 0x7fe0a8158ee0 secure :-1 s=READY pgs=773 cs=0 l=1 rev1=1 crypto rx=0x7fe09800d5f0 tx=0x7fe09800dab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-09T13:48:00.975 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:00.971+0000 7fe09e7fc640 1 -- 172.21.15.43:0/1267000526 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe0980125f0 con 0x7fe0a80cbd90 2023-08-09T13:48:00.975 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:00.971+0000 7fe09e7fc640 1 -- 172.21.15.43:0/1267000526 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe098012750 con 0x7fe0a80cbd90 2023-08-09T13:48:00.976 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:00.971+0000 7fe0aea9f640 1 -- 172.21.15.43:0/1267000526 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe0a809a200 con 0x7fe0a80cbd90 2023-08-09T13:48:00.976 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:00.971+0000 7fe09e7fc640 1 -- 172.21.15.43:0/1267000526 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe098012b00 con 0x7fe0a80cbd90 2023-08-09T13:48:00.976 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:00.971+0000 7fe0aea9f640 1 -- 172.21.15.43:0/1267000526 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe0a8099950 con 0x7fe0a80cbd90 2023-08-09T13:48:00.976 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:00.972+0000 7fe0aea9f640 1 -- 172.21.15.43:0/1267000526 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe0a80b3be0 con 0x7fe0a80cbd90 2023-08-09T13:48:00.976 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:00.973+0000 7fe09e7fc640 1 -- 172.21.15.43:0/1267000526 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 17) v1 ==== 88387+0+0 (secure 0 0 0) 0x7fe098012c60 con 0x7fe0a80cbd90 2023-08-09T13:48:00.977 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:00.974+0000 7fe09e7fc640 1 --2- 172.21.15.43:0/1267000526 >> [v2:172.21.15.203:6832/272917756,v1:172.21.15.203:6833/272917756] conn(0x7fe08c06a0e0 0x7fe08c06c5a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:48:00.977 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:00.974+0000 7fe09e7fc640 1 -- 172.21.15.43:0/1267000526 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(62..62 src has 1..62) v4 ==== 7589+0+0 (secure 0 0 0) 0x7fe098089150 con 0x7fe0a80cbd90 2023-08-09T13:48:00.977 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:00.975+0000 7fe0ada9d640 1 --2- 172.21.15.43:0/1267000526 >> [v2:172.21.15.203:6832/272917756,v1:172.21.15.203:6833/272917756] conn(0x7fe08c06a0e0 0x7fe08c06c5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:48:00.978 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:00.976+0000 7fe0ada9d640 1 --2- 172.21.15.43:0/1267000526 >> [v2:172.21.15.203:6832/272917756,v1:172.21.15.203:6833/272917756] conn(0x7fe08c06a0e0 0x7fe08c06c5a0 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7fe0a4037000 tx=0x7fe0a4032000 comp rx=0 tx=0).ready entity=mgr.5533 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-09T13:48:00.980 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:00.978+0000 7fe09e7fc640 1 -- 172.21.15.43:0/1267000526 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178066 (secure 0 0 0) 0x7fe098054150 con 0x7fe0a80cbd90 2023-08-09T13:48:01.216 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:01.214+0000 7fe0aea9f640 1 -- 172.21.15.43:0/1267000526 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fe0a80739f0 con 0x7fe0a80cbd90 2023-08-09T13:48:01.217 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:01.215+0000 7fe09e7fc640 1 -- 172.21.15.43:0/1267000526 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 17 v17) v1 ==== 94+0+4878 (secure 0 0 0) 0x7fe098057770 con 0x7fe0a80cbd90 2023-08-09T13:48:01.218 INFO:teuthology.orchestra.run.smithi043.stdout: 2023-08-09T13:48:01.218 INFO:teuthology.orchestra.run.smithi043.stdout:{"epoch":17,"default_fscid":2,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4919,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.203:6836/302214806","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6834","nonce":302214806},{"type":"v1","addr":"172.21.15.203:6836","nonce":302214806}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":12},{"gid":4922,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.203:6837/1677639100","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6835","nonce":1677639100},{"type":"v1","addr":"172.21.15.203:6837","nonce":1677639100}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":12},{"gid":5619,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.43:6834/2339480142","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6832","nonce":2339480142},{"type":"v1","addr":"172.21.15.43:6834","nonce":2339480142}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":12}],"filesystems":[{"mdsmap":{"epoch":17,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-08-09T13:36:11.336926+0000","modified":"2023-08-09T13:36:14.905554+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":5622},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5622":{"gid":5622,"name":"c","rank":0,"incarnation":13,"state":"up:active","state_seq":4,"addr":"172.21.15.43:6835/2019295969","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6833","nonce":2019295969},{"type":"v1","addr":"172.21.15.43:6835","nonce":2019295969}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[7],"metadata_pool":6,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":2}]} 2023-08-09T13:48:01.218 INFO:teuthology.orchestra.run.smithi043.stderr:dumped fsmap epoch 17 2023-08-09T13:48:01.220 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:01.218+0000 7fe0aea9f640 1 -- 172.21.15.43:0/1267000526 >> [v2:172.21.15.203:6832/272917756,v1:172.21.15.203:6833/272917756] conn(0x7fe08c06a0e0 msgr2=0x7fe08c06c5a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:48:01.220 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:01.218+0000 7fe0aea9f640 1 --2- 172.21.15.43:0/1267000526 >> [v2:172.21.15.203:6832/272917756,v1:172.21.15.203:6833/272917756] conn(0x7fe08c06a0e0 0x7fe08c06c5a0 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7fe0a4037000 tx=0x7fe0a4032000 comp rx=0 tx=0).stop 2023-08-09T13:48:01.221 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:01.218+0000 7fe0aea9f640 1 -- 172.21.15.43:0/1267000526 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fe0a80cbd90 msgr2=0x7fe0a8158ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:48:01.221 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:01.218+0000 7fe0aea9f640 1 --2- 172.21.15.43:0/1267000526 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fe0a80cbd90 0x7fe0a8158ee0 secure :-1 s=READY pgs=773 cs=0 l=1 rev1=1 crypto rx=0x7fe09800d5f0 tx=0x7fe09800dab0 comp rx=0 tx=0).stop 2023-08-09T13:48:01.221 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:01.218+0000 7fe0aea9f640 1 -- 172.21.15.43:0/1267000526 shutdown_connections 2023-08-09T13:48:01.221 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:01.218+0000 7fe0aea9f640 1 --2- 172.21.15.43:0/1267000526 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fe0a815a6a0 0x7fe0a80c6440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:48:01.221 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:01.218+0000 7fe0aea9f640 1 --2- 172.21.15.43:0/1267000526 >> [v2:172.21.15.203:6832/272917756,v1:172.21.15.203:6833/272917756] conn(0x7fe08c06a0e0 0x7fe08c06c5a0 unknown :-1 s=CLOSED pgs=28 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:48:01.221 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:01.218+0000 7fe0aea9f640 1 --2- 172.21.15.43:0/1267000526 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fe0a80cbd90 0x7fe0a8158ee0 unknown :-1 s=CLOSED pgs=773 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:48:01.221 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:01.218+0000 7fe0aea9f640 1 --2- 172.21.15.43:0/1267000526 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fe0a80c94f0 0x7fe0a8099410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:48:01.222 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:01.218+0000 7fe0aea9f640 1 -- 172.21.15.43:0/1267000526 >> 172.21.15.43:0/1267000526 conn(0x7fe0a80cde90 msgr2=0x7fe0a815ab80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-09T13:48:01.222 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:01.218+0000 7fe0aea9f640 1 -- 172.21.15.43:0/1267000526 shutdown_connections 2023-08-09T13:48:01.222 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:01.218+0000 7fe0aea9f640 1 -- 172.21.15.43:0/1267000526 wait complete. 2023-08-09T13:48:01.239 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-08-09T13:48:01.467 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:01.465+0000 7fcb910cf640 1 Processor -- start 2023-08-09T13:48:01.468 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:01.465+0000 7fcb910cf640 1 -- start start 2023-08-09T13:48:01.469 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:01.467+0000 7fcb910cf640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7fcb8c15c4c0 0x7fcb8c0afd20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:48:01.469 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:01.467+0000 7fcb910cf640 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7fcb8c06b960 0x7fcb8c0ce440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:48:01.469 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:01.467+0000 7fcb910cf640 1 -- --> v1:172.21.15.203:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fcb8c0b0fc0 con 0x7fcb8c06ab70 2023-08-09T13:48:01.469 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:01.467+0000 7fcb910cf640 1 -- --> v2:172.21.15.43:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fcb8c0af8e0 con 0x7fcb8c15c4c0 2023-08-09T13:48:01.470 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:01.467+0000 7fcb910cf640 1 -- --> v2:172.21.15.203:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fcb8c0b03b0 con 0x7fcb8c06b960 2023-08-09T13:48:01.470 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:01.467+0000 7fcb8bfff640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7fcb8c15c4c0 0x7fcb8c0afd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:48:01.471 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:01.467+0000 7fcb8bfff640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7fcb8c15c4c0 0x7fcb8c0afd20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:57382/0 (socket says 172.21.15.43:57382) 2023-08-09T13:48:01.471 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:01.467+0000 7fcb8bfff640 1 -- 172.21.15.43:0/969903882 learned_addr learned my addr 172.21.15.43:0/969903882 (peer_addr_for_me v2:172.21.15.43:0/0) 2023-08-09T13:48:01.471 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:01.468+0000 7fcb8bfff640 1 -- 172.21.15.43:0/969903882 >> v2:172.21.15.203:3300/0 conn(0x7fcb8c06b960 msgr2=0x7fcb8c0ce440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-09T13:48:01.471 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:01.468+0000 7fcb908ce640 1 --2- 172.21.15.43:0/969903882 >> v2:172.21.15.203:3300/0 conn(0x7fcb8c06b960 0x7fcb8c0ce440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:48:01.471 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:01.468+0000 7fcb8bfff640 1 --2- 172.21.15.43:0/969903882 >> v2:172.21.15.203:3300/0 conn(0x7fcb8c06b960 0x7fcb8c0ce440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:48:01.471 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:01.468+0000 7fcb8bfff640 1 -- 172.21.15.43:0/969903882 >> v1:172.21.15.203:6789/0 conn(0x7fcb8c06ab70 legacy=0x7fcb8c06b250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:48:01.471 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:01.468+0000 7fcb8bfff640 1 -- 172.21.15.43:0/969903882 --> v2:172.21.15.43:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcb8c0b0690 con 0x7fcb8c15c4c0 2023-08-09T13:48:01.472 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:01.468+0000 7fcb908ce640 1 --2- 172.21.15.43:0/969903882 >> v2:172.21.15.203:3300/0 conn(0x7fcb8c06b960 0x7fcb8c0ce440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-08-09T13:48:01.472 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:01.469+0000 7fcb8bfff640 1 --2- 172.21.15.43:0/969903882 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fcb8c15c4c0 0x7fcb8c0afd20 secure :-1 s=READY pgs=774 cs=0 l=1 rev1=1 crypto rx=0x7fcb8c0ca690 tx=0x7fcb7c02bfc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=55b3b5c6bbf8bf08 server_cookie=0 in_seq=0 out_seq=0 2023-08-09T13:48:01.473 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:01.469+0000 7fcb8affd640 1 -- 172.21.15.43:0/969903882 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcb7c02cb60 con 0x7fcb8c15c4c0 2023-08-09T13:48:01.473 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:01.469+0000 7fcb8affd640 1 -- 172.21.15.43:0/969903882 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcb7c02ccc0 con 0x7fcb8c15c4c0 2023-08-09T13:48:01.473 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:01.469+0000 7fcb8affd640 1 -- 172.21.15.43:0/969903882 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcb7c034610 con 0x7fcb8c15c4c0 2023-08-09T13:48:01.473 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:01.470+0000 7fcb910cf640 1 -- 172.21.15.43:0/969903882 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fcb8c15c4c0 msgr2=0x7fcb8c0afd20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:48:01.473 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:01.470+0000 7fcb910cf640 1 --2- 172.21.15.43:0/969903882 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fcb8c15c4c0 0x7fcb8c0afd20 secure :-1 s=READY pgs=774 cs=0 l=1 rev1=1 crypto rx=0x7fcb8c0ca690 tx=0x7fcb7c02bfc0 comp rx=0 tx=0).stop 2023-08-09T13:48:01.473 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:01.470+0000 7fcb910cf640 1 -- 172.21.15.43:0/969903882 shutdown_connections 2023-08-09T13:48:01.474 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:01.470+0000 7fcb910cf640 1 --2- 172.21.15.43:0/969903882 >> v2:172.21.15.203:3300/0 conn(0x7fcb8c06b960 0x7fcb8c0ce440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:48:01.474 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:01.470+0000 7fcb910cf640 1 --2- 172.21.15.43:0/969903882 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fcb8c15c4c0 0x7fcb8c0afd20 unknown :-1 s=CLOSED pgs=774 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:48:01.474 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:01.470+0000 7fcb910cf640 1 -- 172.21.15.43:0/969903882 >> 172.21.15.43:0/969903882 conn(0x7fcb8c089fe0 msgr2=0x7fcb8c099450 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-09T13:48:01.474 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:01.470+0000 7fcb910cf640 1 -- 172.21.15.43:0/969903882 shutdown_connections 2023-08-09T13:48:01.474 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:01.470+0000 7fcb910cf640 1 -- 172.21.15.43:0/969903882 wait complete. 2023-08-09T13:48:01.475 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:01.472+0000 7fcb910cf640 1 Processor -- start 2023-08-09T13:48:01.475 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:01.472+0000 7fcb910cf640 1 -- start start 2023-08-09T13:48:01.476 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:01.473+0000 7fcb910cf640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fcb8c06ab70 0x7fcb8c0ca6e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:48:01.476 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:01.473+0000 7fcb910cf640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fcb8c06b960 0x7fcb8c1e69b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:48:01.476 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:01.473+0000 7fcb910cf640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fcb8c15c4c0 0x7fcb8c0b54c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:48:01.476 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:01.473+0000 7fcb910cf640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcb8c0cacc0 con 0x7fcb8c06ab70 2023-08-09T13:48:01.476 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:01.473+0000 7fcb910cf640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcb8c0cafa0 con 0x7fcb8c06b960 2023-08-09T13:48:01.476 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:01.473+0000 7fcb910cf640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcb8c0cb280 con 0x7fcb8c15c4c0 2023-08-09T13:48:01.478 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:01.474+0000 7fcb8bfff640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fcb8c06ab70 0x7fcb8c0ca6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:48:01.478 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:01.474+0000 7fcb8bfff640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fcb8c06ab70 0x7fcb8c0ca6e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:57390/0 (socket says 172.21.15.43:57390) 2023-08-09T13:48:01.478 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:01.474+0000 7fcb8bfff640 1 -- 172.21.15.43:0/2029726272 learned_addr learned my addr 172.21.15.43:0/2029726272 (peer_addr_for_me v2:172.21.15.43:0/0) 2023-08-09T13:48:01.478 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:01.474+0000 7fcb8bfff640 1 -- 172.21.15.43:0/2029726272 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fcb8c15c4c0 msgr2=0x7fcb8c0b54c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:48:01.478 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:01.474+0000 7fcb908ce640 1 --2- 172.21.15.43:0/2029726272 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fcb8c15c4c0 0x7fcb8c0b54c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:48:01.478 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:01.474+0000 7fcb8b7fe640 1 --2- 172.21.15.43:0/2029726272 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fcb8c06b960 0x7fcb8c1e69b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:48:01.478 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:01.474+0000 7fcb8bfff640 1 --2- 172.21.15.43:0/2029726272 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fcb8c15c4c0 0x7fcb8c0b54c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:48:01.479 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:01.474+0000 7fcb8bfff640 1 -- 172.21.15.43:0/2029726272 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fcb8c06b960 msgr2=0x7fcb8c1e69b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:48:01.479 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:01.474+0000 7fcb8bfff640 1 --2- 172.21.15.43:0/2029726272 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fcb8c06b960 0x7fcb8c1e69b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:48:01.479 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:01.474+0000 7fcb8bfff640 1 -- 172.21.15.43:0/2029726272 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcb7c009b30 con 0x7fcb8c06ab70 2023-08-09T13:48:01.479 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:01.474+0000 7fcb908ce640 1 --2- 172.21.15.43:0/2029726272 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fcb8c15c4c0 0x7fcb8c0b54c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-08-09T13:48:01.479 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:01.474+0000 7fcb8bfff640 1 --2- 172.21.15.43:0/2029726272 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fcb8c06ab70 0x7fcb8c0ca6e0 secure :-1 s=READY pgs=775 cs=0 l=1 rev1=1 crypto rx=0x7fcb7c000f80 tx=0x7fcb7c004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-09T13:48:01.479 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:01.475+0000 7fcb88ff9640 1 -- 172.21.15.43:0/2029726272 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcb7c03e450 con 0x7fcb8c06ab70 2023-08-09T13:48:01.479 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:01.475+0000 7fcb88ff9640 1 -- 172.21.15.43:0/2029726272 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcb7c03e5f0 con 0x7fcb8c06ab70 2023-08-09T13:48:01.480 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:01.475+0000 7fcb88ff9640 1 -- 172.21.15.43:0/2029726272 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcb7c03e9a0 con 0x7fcb8c06ab70 2023-08-09T13:48:01.480 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:01.475+0000 7fcb910cf640 1 -- 172.21.15.43:0/2029726272 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcb8c0cb560 con 0x7fcb8c06ab70 2023-08-09T13:48:01.480 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:01.475+0000 7fcb910cf640 1 -- 172.21.15.43:0/2029726272 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcb8c1e6ef0 con 0x7fcb8c06ab70 2023-08-09T13:48:01.480 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:01.476+0000 7fcb910cf640 1 -- 172.21.15.43:0/2029726272 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcb8c0b19a0 con 0x7fcb8c06ab70 2023-08-09T13:48:01.481 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:01.477+0000 7fcb88ff9640 1 -- 172.21.15.43:0/2029726272 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 18) v1 ==== 88443+0+0 (secure 0 0 0) 0x7fcb7c03eb00 con 0x7fcb8c06ab70 2023-08-09T13:48:01.481 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:01.478+0000 7fcb88ff9640 1 --2- 172.21.15.43:0/2029726272 >> [v2:172.21.15.203:6832/272917756,v1:172.21.15.203:6833/272917756] conn(0x7fcb6c06a170 0x7fcb6c06c630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:48:01.481 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:01.478+0000 7fcb88ff9640 1 -- 172.21.15.43:0/2029726272 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(62..62 src has 1..62) v4 ==== 7589+0+0 (secure 0 0 0) 0x7fcb7c0abb80 con 0x7fcb8c06ab70 2023-08-09T13:48:01.481 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:01.479+0000 7fcb8b7fe640 1 --2- 172.21.15.43:0/2029726272 >> [v2:172.21.15.203:6832/272917756,v1:172.21.15.203:6833/272917756] conn(0x7fcb6c06a170 0x7fcb6c06c630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:48:01.483 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:01.480+0000 7fcb8b7fe640 1 --2- 172.21.15.43:0/2029726272 >> [v2:172.21.15.203:6832/272917756,v1:172.21.15.203:6833/272917756] conn(0x7fcb6c06a170 0x7fcb6c06c630 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7fcb8c09df90 tx=0x7fcb74009040 comp rx=0 tx=0).ready entity=mgr.5533 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-09T13:48:01.487 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:01.485+0000 7fcb88ff9640 1 -- 172.21.15.43:0/2029726272 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178066 (secure 0 0 0) 0x7fcb7c076d10 con 0x7fcb8c06ab70 2023-08-09T13:48:01.720 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:01.717+0000 7fcb910cf640 1 -- 172.21.15.43:0/2029726272 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fcb8c0739f0 con 0x7fcb8c06ab70 2023-08-09T13:48:01.721 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:01.719+0000 7fcb88ff9640 1 -- 172.21.15.43:0/2029726272 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 17 v17) v1 ==== 94+0+4878 (secure 0 0 0) 0x7fcb7c03edb0 con 0x7fcb8c06ab70 2023-08-09T13:48:01.721 INFO:teuthology.orchestra.run.smithi043.stdout: 2023-08-09T13:48:01.722 INFO:teuthology.orchestra.run.smithi043.stdout:{"epoch":17,"default_fscid":2,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4919,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.203:6836/302214806","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6834","nonce":302214806},{"type":"v1","addr":"172.21.15.203:6836","nonce":302214806}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":12},{"gid":4922,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.203:6837/1677639100","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6835","nonce":1677639100},{"type":"v1","addr":"172.21.15.203:6837","nonce":1677639100}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":12},{"gid":5619,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.43:6834/2339480142","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6832","nonce":2339480142},{"type":"v1","addr":"172.21.15.43:6834","nonce":2339480142}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":12}],"filesystems":[{"mdsmap":{"epoch":17,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-08-09T13:36:11.336926+0000","modified":"2023-08-09T13:36:14.905554+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":5622},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5622":{"gid":5622,"name":"c","rank":0,"incarnation":13,"state":"up:active","state_seq":4,"addr":"172.21.15.43:6835/2019295969","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6833","nonce":2019295969},{"type":"v1","addr":"172.21.15.43:6835","nonce":2019295969}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[7],"metadata_pool":6,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":2}]} 2023-08-09T13:48:01.722 INFO:teuthology.orchestra.run.smithi043.stderr:dumped fsmap epoch 17 2023-08-09T13:48:01.723 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:01.721+0000 7fcb910cf640 1 -- 172.21.15.43:0/2029726272 >> [v2:172.21.15.203:6832/272917756,v1:172.21.15.203:6833/272917756] conn(0x7fcb6c06a170 msgr2=0x7fcb6c06c630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:48:01.723 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:01.721+0000 7fcb910cf640 1 --2- 172.21.15.43:0/2029726272 >> [v2:172.21.15.203:6832/272917756,v1:172.21.15.203:6833/272917756] conn(0x7fcb6c06a170 0x7fcb6c06c630 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7fcb8c09df90 tx=0x7fcb74009040 comp rx=0 tx=0).stop 2023-08-09T13:48:01.724 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:01.721+0000 7fcb910cf640 1 -- 172.21.15.43:0/2029726272 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fcb8c06ab70 msgr2=0x7fcb8c0ca6e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:48:01.724 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:01.721+0000 7fcb910cf640 1 --2- 172.21.15.43:0/2029726272 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fcb8c06ab70 0x7fcb8c0ca6e0 secure :-1 s=READY pgs=775 cs=0 l=1 rev1=1 crypto rx=0x7fcb7c000f80 tx=0x7fcb7c004a40 comp rx=0 tx=0).stop 2023-08-09T13:48:01.724 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:01.721+0000 7fcb910cf640 1 -- 172.21.15.43:0/2029726272 shutdown_connections 2023-08-09T13:48:01.725 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:01.721+0000 7fcb910cf640 1 --2- 172.21.15.43:0/2029726272 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fcb8c15c4c0 0x7fcb8c0b54c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:48:01.725 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:01.721+0000 7fcb910cf640 1 --2- 172.21.15.43:0/2029726272 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fcb8c06b960 0x7fcb8c1e69b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:48:01.725 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:01.721+0000 7fcb910cf640 1 --2- 172.21.15.43:0/2029726272 >> [v2:172.21.15.203:6832/272917756,v1:172.21.15.203:6833/272917756] conn(0x7fcb6c06a170 0x7fcb6c06c630 unknown :-1 s=CLOSED pgs=29 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:48:01.725 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:01.721+0000 7fcb910cf640 1 --2- 172.21.15.43:0/2029726272 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fcb8c06ab70 0x7fcb8c0ca6e0 unknown :-1 s=CLOSED pgs=775 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:48:01.725 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:01.721+0000 7fcb910cf640 1 -- 172.21.15.43:0/2029726272 >> 172.21.15.43:0/2029726272 conn(0x7fcb8c089fe0 msgr2=0x7fcb8c0d0140 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-09T13:48:01.725 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:01.722+0000 7fcb910cf640 1 -- 172.21.15.43:0/2029726272 shutdown_connections 2023-08-09T13:48:01.725 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:01.722+0000 7fcb910cf640 1 -- 172.21.15.43:0/2029726272 wait complete. 2023-08-09T13:48:01.742 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-08-09T13:48:01.964 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:01.961+0000 7f43ca021640 1 Processor -- start 2023-08-09T13:48:01.964 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:01.962+0000 7f43ca021640 1 -- start start 2023-08-09T13:48:01.965 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:01.963+0000 7f43ca021640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f43c40c8750 0x7f43c40ad340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:48:01.965 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:01.963+0000 7f43ca021640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f43c40acdf0 0x7f43c415b2c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:48:01.966 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:01.963+0000 7f43ca021640 1 -- --> v1:172.21.15.43:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f43c40cbd90 con 0x7f43c40c8b30 2023-08-09T13:48:01.966 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:01.963+0000 7f43ca021640 1 -- --> v2:172.21.15.43:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f43c415e810 con 0x7f43c40c8750 2023-08-09T13:48:01.966 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:01.963+0000 7f43ca021640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f43c40c9f70 con 0x7f43c40acdf0 2023-08-09T13:48:01.967 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:01.963+0000 7f43c901f640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f43c40c8750 0x7f43c40ad340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:48:01.967 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:01.963+0000 7f43c881e640 1 --1- >> v1:172.21.15.43:6789/0 conn(0x7f43c40c8b30 0x7f43c406f410 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.43:6789/0 says I am v1:172.21.15.43:36778/0 (socket says 172.21.15.43:36778) 2023-08-09T13:48:01.967 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:01.963+0000 7f43c901f640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f43c40c8750 0x7f43c40ad340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:57392/0 (socket says 172.21.15.43:57392) 2023-08-09T13:48:01.967 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:01.963+0000 7f43c881e640 1 -- 172.21.15.43:0/1492387272 learned_addr learned my addr 172.21.15.43:0/1492387272 (peer_addr_for_me v1:172.21.15.43:0/0) 2023-08-09T13:48:01.967 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:01.964+0000 7f43c9820640 1 --2- 172.21.15.43:0/1492387272 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f43c40acdf0 0x7f43c415b2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:48:01.967 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:01.964+0000 7f43c901f640 1 -- 172.21.15.43:0/1492387272 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f43c40acdf0 msgr2=0x7f43c415b2c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-09T13:48:01.968 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:01.964+0000 7f43c901f640 1 --2- 172.21.15.43:0/1492387272 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f43c40acdf0 0x7f43c415b2c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:48:01.968 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:01.964+0000 7f43c901f640 1 -- 172.21.15.43:0/1492387272 >> v1:172.21.15.43:6789/0 conn(0x7f43c40c8b30 legacy=0x7f43c406f410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:48:01.968 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:01.964+0000 7f43c901f640 1 -- 172.21.15.43:0/1492387272 --> v2:172.21.15.43:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f43c40ca2b0 con 0x7f43c40c8750 2023-08-09T13:48:01.969 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:01.965+0000 7f43c901f640 1 --2- 172.21.15.43:0/1492387272 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f43c40c8750 0x7f43c40ad340 secure :-1 s=READY pgs=777 cs=0 l=1 rev1=1 crypto rx=0x7f43c0009d90 tx=0x7f43c002be10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=de900c5a0bf95672 server_cookie=0 in_seq=0 out_seq=0 2023-08-09T13:48:01.969 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:01.965+0000 7f43bbfff640 1 -- 172.21.15.43:0/1492387272 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f43c002cbc0 con 0x7f43c40c8750 2023-08-09T13:48:01.969 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:01.965+0000 7f43bbfff640 1 -- 172.21.15.43:0/1492387272 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f43c002cd20 con 0x7f43c40c8750 2023-08-09T13:48:01.969 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:01.965+0000 7f43bbfff640 1 -- 172.21.15.43:0/1492387272 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f43c0034660 con 0x7f43c40c8750 2023-08-09T13:48:01.969 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:01.966+0000 7f43ca021640 1 -- 172.21.15.43:0/1492387272 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f43c40c8750 msgr2=0x7f43c40ad340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:48:01.969 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:01.966+0000 7f43ca021640 1 --2- 172.21.15.43:0/1492387272 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f43c40c8750 0x7f43c40ad340 secure :-1 s=READY pgs=777 cs=0 l=1 rev1=1 crypto rx=0x7f43c0009d90 tx=0x7f43c002be10 comp rx=0 tx=0).stop 2023-08-09T13:48:01.970 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:01.966+0000 7f43ca021640 1 -- 172.21.15.43:0/1492387272 shutdown_connections 2023-08-09T13:48:01.970 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:01.966+0000 7f43ca021640 1 --2- 172.21.15.43:0/1492387272 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f43c40acdf0 0x7f43c415b2c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:48:01.970 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:01.966+0000 7f43ca021640 1 --2- 172.21.15.43:0/1492387272 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f43c40c8750 0x7f43c40ad340 unknown :-1 s=CLOSED pgs=777 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:48:01.970 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:01.966+0000 7f43ca021640 1 -- 172.21.15.43:0/1492387272 >> 172.21.15.43:0/1492387272 conn(0x7f43c40cde90 msgr2=0x7f43c40ce2a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-09T13:48:01.970 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:01.966+0000 7f43ca021640 1 -- 172.21.15.43:0/1492387272 shutdown_connections 2023-08-09T13:48:01.970 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:01.966+0000 7f43ca021640 1 -- 172.21.15.43:0/1492387272 wait complete. 2023-08-09T13:48:01.971 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:01.968+0000 7f43ca021640 1 Processor -- start 2023-08-09T13:48:01.971 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:01.968+0000 7f43ca021640 1 -- start start 2023-08-09T13:48:01.972 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:01.969+0000 7f43ca021640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f43c40acdf0 0x7f43c409af10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:48:01.972 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:01.969+0000 7f43ca021640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f43c40c8750 0x7f43c415b9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:48:01.972 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:01.969+0000 7f43ca021640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f43c40c8b30 0x7f43c415bf30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:48:01.972 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:01.969+0000 7f43ca021640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f43c40c57b0 con 0x7f43c40c8750 2023-08-09T13:48:01.972 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:01.969+0000 7f43ca021640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f43c415a9e0 con 0x7f43c40acdf0 2023-08-09T13:48:01.973 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:01.969+0000 7f43ca021640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f43c409b450 con 0x7f43c40c8b30 2023-08-09T13:48:01.973 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:01.970+0000 7f43c881e640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f43c40c8750 0x7f43c415b9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:48:01.973 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:01.970+0000 7f43c881e640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f43c40c8750 0x7f43c415b9f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:57396/0 (socket says 172.21.15.43:57396) 2023-08-09T13:48:01.973 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:01.970+0000 7f43c881e640 1 -- 172.21.15.43:0/2025664357 learned_addr learned my addr 172.21.15.43:0/2025664357 (peer_addr_for_me v2:172.21.15.43:0/0) 2023-08-09T13:48:01.973 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:01.970+0000 7f43c9820640 1 --2- 172.21.15.43:0/2025664357 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f43c40c8b30 0x7f43c415bf30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:48:01.973 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:01.970+0000 7f43c901f640 1 --2- 172.21.15.43:0/2025664357 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f43c40acdf0 0x7f43c409af10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:48:01.974 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:01.970+0000 7f43c881e640 1 -- 172.21.15.43:0/2025664357 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f43c40c8b30 msgr2=0x7f43c415bf30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:48:01.974 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:01.970+0000 7f43c881e640 1 --2- 172.21.15.43:0/2025664357 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f43c40c8b30 0x7f43c415bf30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:48:01.975 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:01.970+0000 7f43c881e640 1 -- 172.21.15.43:0/2025664357 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f43c40acdf0 msgr2=0x7f43c409af10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:48:01.975 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:01.970+0000 7f43c881e640 1 --2- 172.21.15.43:0/2025664357 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f43c40acdf0 0x7f43c409af10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:48:01.975 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:01.970+0000 7f43c881e640 1 -- 172.21.15.43:0/2025664357 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f43c0009a70 con 0x7f43c40c8750 2023-08-09T13:48:01.975 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:01.970+0000 7f43c881e640 1 --2- 172.21.15.43:0/2025664357 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f43c40c8750 0x7f43c415b9f0 secure :-1 s=READY pgs=778 cs=0 l=1 rev1=1 crypto rx=0x7f43b400c4b0 tx=0x7f43b400c970 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-09T13:48:01.975 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:01.971+0000 7f43b9ffb640 1 -- 172.21.15.43:0/2025664357 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f43b4005ce0 con 0x7f43c40c8750 2023-08-09T13:48:01.975 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:01.971+0000 7f43b9ffb640 1 -- 172.21.15.43:0/2025664357 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f43b4005e40 con 0x7f43c40c8750 2023-08-09T13:48:01.975 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:01.971+0000 7f43b9ffb640 1 -- 172.21.15.43:0/2025664357 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f43b4007270 con 0x7f43c40c8750 2023-08-09T13:48:01.975 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:01.971+0000 7f43ca021640 1 -- 172.21.15.43:0/2025664357 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f43c40c54d0 con 0x7f43c40c8750 2023-08-09T13:48:01.976 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:01.971+0000 7f43ca021640 1 -- 172.21.15.43:0/2025664357 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f43c40c27d0 con 0x7f43c40c8750 2023-08-09T13:48:01.976 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:01.972+0000 7f43ca021640 1 -- 172.21.15.43:0/2025664357 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f43c40c2320 con 0x7f43c40c8750 2023-08-09T13:48:01.976 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:01.973+0000 7f43b9ffb640 1 -- 172.21.15.43:0/2025664357 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 18) v1 ==== 88443+0+0 (secure 0 0 0) 0x7f43b40073d0 con 0x7f43c40c8750 2023-08-09T13:48:01.977 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:01.974+0000 7f43b9ffb640 1 --2- 172.21.15.43:0/2025664357 >> [v2:172.21.15.203:6832/272917756,v1:172.21.15.203:6833/272917756] conn(0x7f43a806a170 0x7f43a806c630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:48:01.977 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:01.974+0000 7f43b9ffb640 1 -- 172.21.15.43:0/2025664357 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(62..62 src has 1..62) v4 ==== 7589+0+0 (secure 0 0 0) 0x7f43b40880c0 con 0x7f43c40c8750 2023-08-09T13:48:01.977 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:01.975+0000 7f43c901f640 1 --2- 172.21.15.43:0/2025664357 >> [v2:172.21.15.203:6832/272917756,v1:172.21.15.203:6833/272917756] conn(0x7f43a806a170 0x7f43a806c630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:48:01.978 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:01.976+0000 7f43c901f640 1 --2- 172.21.15.43:0/2025664357 >> [v2:172.21.15.203:6832/272917756,v1:172.21.15.203:6833/272917756] conn(0x7f43a806a170 0x7f43a806c630 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0x7f43c0036000 tx=0x7f43c0030000 comp rx=0 tx=0).ready entity=mgr.5533 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-09T13:48:01.981 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:01.979+0000 7f43b9ffb640 1 -- 172.21.15.43:0/2025664357 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178066 (secure 0 0 0) 0x7f43b40531e0 con 0x7f43c40c8750 2023-08-09T13:48:02.174 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:02.171+0000 7f43ca021640 1 -- 172.21.15.43:0/2025664357 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f43c40c9230 con 0x7f43c40c8750 2023-08-09T13:48:02.176 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:02.173+0000 7f43b9ffb640 1 -- 172.21.15.43:0/2025664357 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v63) v1 ==== 74+0+20234 (secure 0 0 0) 0x7f43b4056800 con 0x7f43c40c8750 2023-08-09T13:48:02.177 INFO:teuthology.orchestra.run.smithi043.stdout: 2023-08-09T13:48:02.177 INFO:teuthology.orchestra.run.smithi043.stdout:{"epoch":63,"fsid":"daac87ee-1337-4250-a955-aea4e408a39b","created":"2023-08-09T13:33:41.701198+0000","modified":"2023-08-09T13:48:02.018053+0000","last_up_change":"2023-08-09T13:33:53.783453+0000","last_in_change":"2023-08-09T13:33:45.148904+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":7,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":"rbd","create_time":"2023-08-09T13:33:54.365137+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":".mgr","create_time":"2023-08-09T13:33:54.831803+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":6,"pool_name":"cephfs_metadata","create_time":"2023-08-09T13:36:08.969993+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"38","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":7,"pool_name":"cephfs_data","create_time":"2023-08-09T13:36:10.366495+0000","flags":8193,"flags_names":"hashpspool,selfmanaged_snaps","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"62","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":221,"snap_epoch":62,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"cbd49ecd-e896-4fbd-864e-5c37a829b165","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":37,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6800","nonce":502667335},{"type":"v1","addr":"172.21.15.43:6801","nonce":502667335}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6802","nonce":502667335},{"type":"v1","addr":"172.21.15.43:6803","nonce":502667335}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6806","nonce":502667335},{"type":"v1","addr":"172.21.15.43:6807","nonce":502667335}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6804","nonce":502667335},{"type":"v1","addr":"172.21.15.43:6805","nonce":502667335}]},"public_addr":"172.21.15.43:6801/502667335","cluster_addr":"172.21.15.43:6803/502667335","heartbeat_back_addr":"172.21.15.43:6807/502667335","heartbeat_front_addr":"172.21.15.43:6805/502667335","state":["exists","up"]},{"osd":1,"uuid":"75c7e0fe-1ec7-4fe8-a2f2-3661a65b8500","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":37,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6808","nonce":3289134610},{"type":"v1","addr":"172.21.15.43:6809","nonce":3289134610}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6810","nonce":3289134610},{"type":"v1","addr":"172.21.15.43:6812","nonce":3289134610}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6821","nonce":3289134610},{"type":"v1","addr":"172.21.15.43:6824","nonce":3289134610}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6815","nonce":3289134610},{"type":"v1","addr":"172.21.15.43:6818","nonce":3289134610}]},"public_addr":"172.21.15.43:6809/3289134610","cluster_addr":"172.21.15.43:6812/3289134610","heartbeat_back_addr":"172.21.15.43:6824/3289134610","heartbeat_front_addr":"172.21.15.43:6818/3289134610","state":["exists","up"]},{"osd":2,"uuid":"bb5c41e9-7f5f-4125-9b01-3bb8bfc2ea83","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":37,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6811","nonce":4079709736},{"type":"v1","addr":"172.21.15.43:6814","nonce":4079709736}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6817","nonce":4079709736},{"type":"v1","addr":"172.21.15.43:6820","nonce":4079709736}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6829","nonce":4079709736},{"type":"v1","addr":"172.21.15.43:6831","nonce":4079709736}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6823","nonce":4079709736},{"type":"v1","addr":"172.21.15.43:6826","nonce":4079709736}]},"public_addr":"172.21.15.43:6814/4079709736","cluster_addr":"172.21.15.43:6820/4079709736","heartbeat_back_addr":"172.21.15.43:6831/4079709736","heartbeat_front_addr":"172.21.15.43:6826/4079709736","state":["exists","up"]},{"osd":3,"uuid":"bfe4c8b4-694a-465c-87e6-2b2aab852613","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":37,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6813","nonce":1709538759},{"type":"v1","addr":"172.21.15.43:6816","nonce":1709538759}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6819","nonce":1709538759},{"type":"v1","addr":"172.21.15.43:6822","nonce":1709538759}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6828","nonce":1709538759},{"type":"v1","addr":"172.21.15.43:6830","nonce":1709538759}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6825","nonce":1709538759},{"type":"v1","addr":"172.21.15.43:6827","nonce":1709538759}]},"public_addr":"172.21.15.43:6816/1709538759","cluster_addr":"172.21.15.43:6822/1709538759","heartbeat_back_addr":"172.21.15.43:6830/1709538759","heartbeat_front_addr":"172.21.15.43:6827/1709538759","state":["exists","up"]},{"osd":4,"uuid":"fa2e8a77-aba1-4756-853e-0d174a5bdeba","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":41,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6800","nonce":4196866481},{"type":"v1","addr":"172.21.15.203:6801","nonce":4196866481}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6802","nonce":4196866481},{"type":"v1","addr":"172.21.15.203:6803","nonce":4196866481}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6806","nonce":4196866481},{"type":"v1","addr":"172.21.15.203:6807","nonce":4196866481}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6804","nonce":4196866481},{"type":"v1","addr":"172.21.15.203:6805","nonce":4196866481}]},"public_addr":"172.21.15.203:6801/4196866481","cluster_addr":"172.21.15.203:6803/4196866481","heartbeat_back_addr":"172.21.15.203:6807/4196866481","heartbeat_front_addr":"172.21.15.203:6805/4196866481","state":["exists","up"]},{"osd":5,"uuid":"81811e4b-4d24-4f51-a82c-f1d700a2a00d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":41,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6816","nonce":3059290081},{"type":"v1","addr":"172.21.15.203:6817","nonce":3059290081}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6818","nonce":3059290081},{"type":"v1","addr":"172.21.15.203:6819","nonce":3059290081}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6822","nonce":3059290081},{"type":"v1","addr":"172.21.15.203:6823","nonce":3059290081}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6820","nonce":3059290081},{"type":"v1","addr":"172.21.15.203:6821","nonce":3059290081}]},"public_addr":"172.21.15.203:6817/3059290081","cluster_addr":"172.21.15.203:6819/3059290081","heartbeat_back_addr":"172.21.15.203:6823/3059290081","heartbeat_front_addr":"172.21.15.203:6821/3059290081","state":["exists","up"]},{"osd":6,"uuid":"307bac4d-6a5b-443d-af75-56d695d7674f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":37,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6824","nonce":512155569},{"type":"v1","addr":"172.21.15.203:6825","nonce":512155569}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6826","nonce":512155569},{"type":"v1","addr":"172.21.15.203:6827","nonce":512155569}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6830","nonce":512155569},{"type":"v1","addr":"172.21.15.203:6831","nonce":512155569}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6828","nonce":512155569},{"type":"v1","addr":"172.21.15.203:6829","nonce":512155569}]},"public_addr":"172.21.15.203:6825/512155569","cluster_addr":"172.21.15.203:6827/512155569","heartbeat_back_addr":"172.21.15.203:6831/512155569","heartbeat_front_addr":"172.21.15.203:6829/512155569","state":["exists","up"]},{"osd":7,"uuid":"a95dc7ca-9f1b-4295-af70-68a22a2d7d5e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":37,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6808","nonce":3331450438},{"type":"v1","addr":"172.21.15.203:6809","nonce":3331450438}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6810","nonce":3331450438},{"type":"v1","addr":"172.21.15.203:6811","nonce":3331450438}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6814","nonce":3331450438},{"type":"v1","addr":"172.21.15.203:6815","nonce":3331450438}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6812","nonce":3331450438},{"type":"v1","addr":"172.21.15.203:6813","nonce":3331450438}]},"public_addr":"172.21.15.203:6809/3331450438","cluster_addr":"172.21.15.203:6811/3331450438","heartbeat_back_addr":"172.21.15.203:6815/3331450438","heartbeat_front_addr":"172.21.15.203:6813/3331450438","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-09T13:33:50.547652+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-09T13:33:50.589469+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-09T13:33:50.561724+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-09T13:33:50.599650+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-09T13:33:50.411855+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-09T13:33:50.219681+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-09T13:33:50.504664+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-09T13:33:50.441248+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"7.2a","mappings":[{"from":0,"to":5}]},{"pgid":"7.3c","mappings":[{"from":0,"to":4}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.203:0/830217892":"2023-08-10T13:47:57.215872+0000","172.21.15.203:0/1493361813":"2023-08-10T13:47:57.215872+0000","172.21.15.203:0/3309257000":"2023-08-10T13:36:27.970754+0000","172.21.15.203:0/2444066336":"2023-08-10T13:47:57.215872+0000","172.21.15.203:6832/830097474":"2023-08-10T13:36:27.970754+0000","172.21.15.203:6833/830097474":"2023-08-10T13:36:27.970754+0000","172.21.15.203:0/2439593041":"2023-08-10T13:36:27.970754+0000","172.21.15.203:0/2454266731":"2023-08-10T13:47:57.215872+0000","172.21.15.203:0/1761968483":"2023-08-10T13:36:27.970754+0000","172.21.15.203:0/890377927":"2023-08-10T13:36:27.970754+0000","172.21.15.203:0/560220444":"2023-08-10T13:47:57.215872+0000","172.21.15.203:6832/998005540":"2023-08-10T13:47:57.215872+0000","172.21.15.203:6833/998005540":"2023-08-10T13:47:57.215872+0000","172.21.15.203:0/315780925":"2023-08-10T13:47:57.215872+0000","172.21.15.203:0/1813604691":"2023-08-10T13:47:57.215872+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":7,"snaps":[{"begin":104,"length":1},{"begin":221,"length":1}]}],"new_removed_snaps":[],"new_purged_snaps":[{"pool":7,"snaps":[{"begin":4,"length":1},{"begin":8,"length":1},{"begin":10,"length":1},{"begin":15,"length":1},{"begin":18,"length":2},{"begin":25,"length":1},{"begin":27,"length":1},{"begin":31,"length":1},{"begin":34,"length":1},{"begin":40,"length":2},{"begin":48,"length":3},{"begin":55,"length":1},{"begin":57,"length":1},{"begin":59,"length":1},{"begin":62,"length":1},{"begin":65,"length":1},{"begin":68,"length":1},{"begin":72,"length":1},{"begin":74,"length":1},{"begin":77,"length":1},{"begin":83,"length":2},{"begin":87,"length":1},{"begin":95,"length":2},{"begin":98,"length":1},{"begin":107,"length":2},{"begin":111,"length":1},{"begin":188,"length":33}]}],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-08-09T13:48:02.178 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:02.176+0000 7f43ca021640 1 -- 172.21.15.43:0/2025664357 >> [v2:172.21.15.203:6832/272917756,v1:172.21.15.203:6833/272917756] conn(0x7f43a806a170 msgr2=0x7f43a806c630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:48:02.179 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:02.176+0000 7f43ca021640 1 --2- 172.21.15.43:0/2025664357 >> [v2:172.21.15.203:6832/272917756,v1:172.21.15.203:6833/272917756] conn(0x7f43a806a170 0x7f43a806c630 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0x7f43c0036000 tx=0x7f43c0030000 comp rx=0 tx=0).stop 2023-08-09T13:48:02.179 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:02.176+0000 7f43ca021640 1 -- 172.21.15.43:0/2025664357 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f43c40c8750 msgr2=0x7f43c415b9f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:48:02.179 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:02.176+0000 7f43ca021640 1 --2- 172.21.15.43:0/2025664357 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f43c40c8750 0x7f43c415b9f0 secure :-1 s=READY pgs=778 cs=0 l=1 rev1=1 crypto rx=0x7f43b400c4b0 tx=0x7f43b400c970 comp rx=0 tx=0).stop 2023-08-09T13:48:02.179 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:02.176+0000 7f43ca021640 1 -- 172.21.15.43:0/2025664357 shutdown_connections 2023-08-09T13:48:02.179 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:02.176+0000 7f43ca021640 1 --2- 172.21.15.43:0/2025664357 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f43c40c8b30 0x7f43c415bf30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:48:02.179 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:02.176+0000 7f43ca021640 1 --2- 172.21.15.43:0/2025664357 >> [v2:172.21.15.203:6832/272917756,v1:172.21.15.203:6833/272917756] conn(0x7f43a806a170 0x7f43a806c630 unknown :-1 s=CLOSED pgs=30 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:48:02.179 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:02.176+0000 7f43ca021640 1 --2- 172.21.15.43:0/2025664357 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f43c40c8750 0x7f43c415b9f0 unknown :-1 s=CLOSED pgs=778 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:48:02.180 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:02.176+0000 7f43ca021640 1 --2- 172.21.15.43:0/2025664357 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f43c40acdf0 0x7f43c409af10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:48:02.180 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:02.176+0000 7f43ca021640 1 -- 172.21.15.43:0/2025664357 >> 172.21.15.43:0/2025664357 conn(0x7f43c40cde90 msgr2=0x7f43c40af070 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-09T13:48:02.180 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:02.176+0000 7f43ca021640 1 -- 172.21.15.43:0/2025664357 shutdown_connections 2023-08-09T13:48:02.180 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:02.176+0000 7f43ca021640 1 -- 172.21.15.43:0/2025664357 wait complete. 2023-08-09T13:48:02.198 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2023-08-09T13:48:02.415 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:02.413+0000 7f5199ffc640 1 Processor -- start 2023-08-09T13:48:02.416 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:02.413+0000 7f5199ffc640 1 -- start start 2023-08-09T13:48:02.417 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:02.415+0000 7f5199ffc640 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7f51940c8670 0x7f519415c8e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:48:02.417 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:02.415+0000 7f5199ffc640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f51940abbb0 0x7f51940ac000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:48:02.418 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:02.415+0000 7f5199ffc640 1 -- --> v1:172.21.15.43:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5194070730 con 0x7f519406f3d0 2023-08-09T13:48:02.418 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:02.415+0000 7f5199ffc640 1 -- --> v2:172.21.15.43:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5194156e60 con 0x7f51940abbb0 2023-08-09T13:48:02.418 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:02.415+0000 7f5199ffc640 1 -- --> v2:172.21.15.203:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5194071620 con 0x7f51940c8670 2023-08-09T13:48:02.419 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:02.415+0000 7f51997fb640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f51940abbb0 0x7f51940ac000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:48:02.419 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:02.415+0000 7f51997fb640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f51940abbb0 0x7f51940ac000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:57412/0 (socket says 172.21.15.43:57412) 2023-08-09T13:48:02.419 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:02.415+0000 7f51997fb640 1 -- 172.21.15.43:0/227109167 learned_addr learned my addr 172.21.15.43:0/227109167 (peer_addr_for_me v2:172.21.15.43:0/0) 2023-08-09T13:48:02.419 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:02.416+0000 7f5198ffa640 1 --2- 172.21.15.43:0/227109167 >> v2:172.21.15.203:3300/0 conn(0x7f51940c8670 0x7f519415c8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:48:02.419 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:02.416+0000 7f51997fb640 1 -- 172.21.15.43:0/227109167 >> v2:172.21.15.203:3300/0 conn(0x7f51940c8670 msgr2=0x7f519415c8e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-09T13:48:02.419 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:02.416+0000 7f51997fb640 1 --2- 172.21.15.43:0/227109167 >> v2:172.21.15.203:3300/0 conn(0x7f51940c8670 0x7f519415c8e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:48:02.419 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:02.416+0000 7f51997fb640 1 -- 172.21.15.43:0/227109167 >> v1:172.21.15.43:6789/0 conn(0x7f519406f3d0 legacy=0x7f51940c8a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:48:02.420 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:02.416+0000 7f51997fb640 1 -- 172.21.15.43:0/227109167 --> v2:172.21.15.43:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f519406fb20 con 0x7f51940abbb0 2023-08-09T13:48:02.420 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:02.416+0000 7f51937fe640 1 -- 172.21.15.43:0/227109167 <== mon.0 v1:172.21.15.43:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1156370414 0 0) 0x7f5194070730 con 0x7f519406f3d0 2023-08-09T13:48:02.421 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:02.417+0000 7f51997fb640 1 --2- 172.21.15.43:0/227109167 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f51940abbb0 0x7f51940ac000 secure :-1 s=READY pgs=780 cs=0 l=1 rev1=1 crypto rx=0x7f519406a860 tx=0x7f518c02be50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e35fd21dc23b9064 server_cookie=0 in_seq=0 out_seq=0 2023-08-09T13:48:02.421 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:02.417+0000 7f51937fe640 1 -- 172.21.15.43:0/227109167 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f518c02c9f0 con 0x7f51940abbb0 2023-08-09T13:48:02.421 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:02.417+0000 7f51937fe640 1 -- 172.21.15.43:0/227109167 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f518c02cb50 con 0x7f51940abbb0 2023-08-09T13:48:02.421 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:02.417+0000 7f51937fe640 1 -- 172.21.15.43:0/227109167 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f518c0343f0 con 0x7f51940abbb0 2023-08-09T13:48:02.421 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:02.418+0000 7f5199ffc640 1 -- 172.21.15.43:0/227109167 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f51940abbb0 msgr2=0x7f51940ac000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:48:02.421 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:02.418+0000 7f5199ffc640 1 --2- 172.21.15.43:0/227109167 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f51940abbb0 0x7f51940ac000 secure :-1 s=READY pgs=780 cs=0 l=1 rev1=1 crypto rx=0x7f519406a860 tx=0x7f518c02be50 comp rx=0 tx=0).stop 2023-08-09T13:48:02.422 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:02.418+0000 7f5199ffc640 1 -- 172.21.15.43:0/227109167 shutdown_connections 2023-08-09T13:48:02.422 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:02.418+0000 7f5199ffc640 1 --2- 172.21.15.43:0/227109167 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f51940abbb0 0x7f51940ac000 unknown :-1 s=CLOSED pgs=780 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:48:02.422 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:02.418+0000 7f5199ffc640 1 --2- 172.21.15.43:0/227109167 >> v2:172.21.15.203:3300/0 conn(0x7f51940c8670 0x7f519415c8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:48:02.422 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:02.418+0000 7f5199ffc640 1 -- 172.21.15.43:0/227109167 >> 172.21.15.43:0/227109167 conn(0x7f51940af950 msgr2=0x7f51940afd60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-09T13:48:02.422 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:02.418+0000 7f5199ffc640 1 -- 172.21.15.43:0/227109167 shutdown_connections 2023-08-09T13:48:02.422 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:02.418+0000 7f5199ffc640 1 -- 172.21.15.43:0/227109167 wait complete. 2023-08-09T13:48:02.423 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:02.420+0000 7f5199ffc640 1 Processor -- start 2023-08-09T13:48:02.423 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:02.420+0000 7f5199ffc640 1 -- start start 2023-08-09T13:48:02.424 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:02.421+0000 7f5199ffc640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f519406f3d0 0x7f51940b30c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:48:02.424 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:02.421+0000 7f5199ffc640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f51940abbb0 0x7f51940b0ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:48:02.424 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:02.421+0000 7f5199ffc640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f51940c8670 0x7f51940b1410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:48:02.424 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:02.421+0000 7f5199ffc640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5194152fb0 con 0x7f51940abbb0 2023-08-09T13:48:02.424 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:02.421+0000 7f5199ffc640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f51940b1950 con 0x7f519406f3d0 2023-08-09T13:48:02.424 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:02.421+0000 7f5199ffc640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5194141c80 con 0x7f51940c8670 2023-08-09T13:48:02.424 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:02.421+0000 7f5193fff640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f51940abbb0 0x7f51940b0ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:48:02.425 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:02.421+0000 7f5193fff640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f51940abbb0 0x7f51940b0ed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:57420/0 (socket says 172.21.15.43:57420) 2023-08-09T13:48:02.425 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:02.421+0000 7f5193fff640 1 -- 172.21.15.43:0/1559086222 learned_addr learned my addr 172.21.15.43:0/1559086222 (peer_addr_for_me v2:172.21.15.43:0/0) 2023-08-09T13:48:02.426 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:02.421+0000 7f5193fff640 1 -- 172.21.15.43:0/1559086222 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f51940c8670 msgr2=0x7f51940b1410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:48:02.426 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:02.421+0000 7f5193fff640 1 --2- 172.21.15.43:0/1559086222 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f51940c8670 0x7f51940b1410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:48:02.426 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:02.421+0000 7f5193fff640 1 -- 172.21.15.43:0/1559086222 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f519406f3d0 msgr2=0x7f51940b30c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:48:02.426 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:02.421+0000 7f5193fff640 1 --2- 172.21.15.43:0/1559086222 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f519406f3d0 0x7f51940b30c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:48:02.426 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:02.421+0000 7f5193fff640 1 -- 172.21.15.43:0/1559086222 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f518c009b30 con 0x7f51940abbb0 2023-08-09T13:48:02.426 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:02.422+0000 7f5193fff640 1 --2- 172.21.15.43:0/1559086222 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f51940abbb0 0x7f51940b0ed0 secure :-1 s=READY pgs=781 cs=0 l=1 rev1=1 crypto rx=0x7f518000c440 tx=0x7f518000c900 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-09T13:48:02.426 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:02.422+0000 7f51917fa640 1 -- 172.21.15.43:0/1559086222 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5180005480 con 0x7f51940abbb0 2023-08-09T13:48:02.427 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:02.422+0000 7f51917fa640 1 -- 172.21.15.43:0/1559086222 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f51800035f0 con 0x7f51940abbb0 2023-08-09T13:48:02.427 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:02.422+0000 7f51917fa640 1 -- 172.21.15.43:0/1559086222 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f51800039a0 con 0x7f51940abbb0 2023-08-09T13:48:02.427 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:02.422+0000 7f5199ffc640 1 -- 172.21.15.43:0/1559086222 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f51941f5bd0 con 0x7f51940abbb0 2023-08-09T13:48:02.427 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:02.422+0000 7f5199ffc640 1 -- 172.21.15.43:0/1559086222 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5194153470 con 0x7f51940abbb0 2023-08-09T13:48:02.427 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:02.423+0000 7f5199ffc640 1 -- 172.21.15.43:0/1559086222 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f51940739f0 con 0x7f51940abbb0 2023-08-09T13:48:02.428 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:02.424+0000 7f51917fa640 1 -- 172.21.15.43:0/1559086222 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 18) v1 ==== 88443+0+0 (secure 0 0 0) 0x7f5180003b00 con 0x7f51940abbb0 2023-08-09T13:48:02.428 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:02.425+0000 7f51917fa640 1 --2- 172.21.15.43:0/1559086222 >> [v2:172.21.15.203:6832/272917756,v1:172.21.15.203:6833/272917756] conn(0x7f517806a170 0x7f517806c630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:48:02.428 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:02.425+0000 7f51917fa640 1 -- 172.21.15.43:0/1559086222 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(63..63 src has 1..63) v4 ==== 7573+0+0 (secure 0 0 0) 0x7f51800882e0 con 0x7f51940abbb0 2023-08-09T13:48:02.428 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:02.426+0000 7f5198ffa640 1 --2- 172.21.15.43:0/1559086222 >> [v2:172.21.15.203:6832/272917756,v1:172.21.15.203:6833/272917756] conn(0x7f517806a170 0x7f517806c630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:48:02.429 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:02.427+0000 7f5198ffa640 1 --2- 172.21.15.43:0/1559086222 >> [v2:172.21.15.203:6832/272917756,v1:172.21.15.203:6833/272917756] conn(0x7f517806a170 0x7f517806c630 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0x7f51880098f0 tx=0x7f5188008040 comp rx=0 tx=0).ready entity=mgr.5533 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-09T13:48:02.432 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:02.430+0000 7f51917fa640 1 -- 172.21.15.43:0/1559086222 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178066 (secure 0 0 0) 0x7f5180053230 con 0x7f51940abbb0 2023-08-09T13:48:02.674 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:02.671+0000 7f5199ffc640 1 -- 172.21.15.43:0/1559086222 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) v1 -- 0x7f51940adb90 con 0x7f51940abbb0 2023-08-09T13:48:02.675 INFO:tasks.ceph.mon.a.smithi043.stderr:2023-08-09T13:48:02.672+0000 7f32867f3640 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2023-08-09T13:48:02.686 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:02.683+0000 7f51917fa640 1 -- 172.21.15.43:0/1559086222 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v18) v1 ==== 161+0+0 (secure 0 0 0) 0x7f5180056850 con 0x7f51940abbb0 2023-08-09T13:48:02.686 INFO:teuthology.orchestra.run.smithi043.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2023-08-09T13:48:02.689 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:02.687+0000 7f5199ffc640 1 -- 172.21.15.43:0/1559086222 >> [v2:172.21.15.203:6832/272917756,v1:172.21.15.203:6833/272917756] conn(0x7f517806a170 msgr2=0x7f517806c630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:48:02.689 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:02.687+0000 7f5199ffc640 1 --2- 172.21.15.43:0/1559086222 >> [v2:172.21.15.203:6832/272917756,v1:172.21.15.203:6833/272917756] conn(0x7f517806a170 0x7f517806c630 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0x7f51880098f0 tx=0x7f5188008040 comp rx=0 tx=0).stop 2023-08-09T13:48:02.689 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:02.687+0000 7f5199ffc640 1 -- 172.21.15.43:0/1559086222 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f51940abbb0 msgr2=0x7f51940b0ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:48:02.690 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:02.687+0000 7f5199ffc640 1 --2- 172.21.15.43:0/1559086222 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f51940abbb0 0x7f51940b0ed0 secure :-1 s=READY pgs=781 cs=0 l=1 rev1=1 crypto rx=0x7f518000c440 tx=0x7f518000c900 comp rx=0 tx=0).stop 2023-08-09T13:48:02.690 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:02.687+0000 7f5199ffc640 1 -- 172.21.15.43:0/1559086222 shutdown_connections 2023-08-09T13:48:02.690 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:02.687+0000 7f5199ffc640 1 --2- 172.21.15.43:0/1559086222 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f51940c8670 0x7f51940b1410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:48:02.691 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:02.687+0000 7f5199ffc640 1 --2- 172.21.15.43:0/1559086222 >> [v2:172.21.15.203:6832/272917756,v1:172.21.15.203:6833/272917756] conn(0x7f517806a170 0x7f517806c630 unknown :-1 s=CLOSED pgs=31 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:48:02.691 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:02.687+0000 7f5199ffc640 1 --2- 172.21.15.43:0/1559086222 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f51940abbb0 0x7f51940b0ed0 unknown :-1 s=CLOSED pgs=781 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:48:02.691 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:02.687+0000 7f5199ffc640 1 --2- 172.21.15.43:0/1559086222 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f519406f3d0 0x7f51940b30c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:48:02.691 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:02.687+0000 7f5199ffc640 1 -- 172.21.15.43:0/1559086222 >> 172.21.15.43:0/1559086222 conn(0x7f51940af950 msgr2=0x7f51940ae0b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-09T13:48:02.691 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:02.687+0000 7f5199ffc640 1 -- 172.21.15.43:0/1559086222 shutdown_connections 2023-08-09T13:48:02.691 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:02.687+0000 7f5199ffc640 1 -- 172.21.15.43:0/1559086222 wait complete. 2023-08-09T13:48:02.707 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs rm cephfs --yes-i-really-mean-it 2023-08-09T13:48:02.782 INFO:tasks.ceph.mds.c.smithi043.stdout:starting mds.c at 2023-08-09T13:48:02.899 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:02.897+0000 7f60d885d640 1 Processor -- start 2023-08-09T13:48:02.900 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:02.897+0000 7f60d885d640 1 -- start start 2023-08-09T13:48:02.901 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:02.898+0000 7f60d885d640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f60d4070ef0 0x7f60d40ab8d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:48:02.901 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:02.898+0000 7f60d885d640 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7f60d40cc0b0 0x7f60d406f310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:48:02.901 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:02.898+0000 7f60d885d640 1 -- --> v1:172.21.15.43:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f60d40acfa0 con 0x7f60d40abe10 2023-08-09T13:48:02.901 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:02.898+0000 7f60d885d640 1 -- --> v2:172.21.15.203:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f60d415c940 con 0x7f60d40cc0b0 2023-08-09T13:48:02.902 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:02.898+0000 7f60d885d640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f60d40ac390 con 0x7f60d4070ef0 2023-08-09T13:48:02.905 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:02.899+0000 7f60d2ffd640 1 --1- >> v1:172.21.15.43:6789/0 conn(0x7f60d40abe10 0x7f60d40cb9a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.43:6789/0 says I am v1:172.21.15.43:36794/0 (socket says 172.21.15.43:36794) 2023-08-09T13:48:02.906 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:02.899+0000 7f60d2ffd640 1 -- 172.21.15.43:0/2581549616 learned_addr learned my addr 172.21.15.43:0/2581549616 (peer_addr_for_me v1:172.21.15.43:0/0) 2023-08-09T13:48:02.906 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:02.899+0000 7f60d27fc640 1 -- 172.21.15.43:0/2581549616 <== mon.0 v1:172.21.15.43:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4116914020 0 0) 0x7f60d40acfa0 con 0x7f60d40abe10 2023-08-09T13:48:02.906 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:02.899+0000 7f60d27fc640 1 -- 172.21.15.43:0/2581549616 --> v1:172.21.15.43:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f60bc003520 con 0x7f60d40abe10 2023-08-09T13:48:02.907 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:02.899+0000 7f60d3fff640 1 --2- 172.21.15.43:0/2581549616 >> v2:172.21.15.203:3300/0 conn(0x7f60d40cc0b0 0x7f60d406f310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:48:02.907 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:02.899+0000 7f60d37fe640 1 --2- 172.21.15.43:0/2581549616 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f60d4070ef0 0x7f60d40ab8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:48:02.908 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:02.899+0000 7f60d27fc640 1 -- 172.21.15.43:0/2581549616 <== mon.0 v1:172.21.15.43:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1130710573 0 0) 0x7f60bc003520 con 0x7f60d40abe10 2023-08-09T13:48:02.908 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:02.899+0000 7f60d27fc640 1 -- 172.21.15.43:0/2581549616 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f60d4070ef0 msgr2=0x7f60d40ab8d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-09T13:48:02.908 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:02.899+0000 7f60d27fc640 1 --2- 172.21.15.43:0/2581549616 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f60d4070ef0 0x7f60d40ab8d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:48:02.908 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:02.899+0000 7f60d27fc640 1 -- 172.21.15.43:0/2581549616 >> v2:172.21.15.203:3300/0 conn(0x7f60d40cc0b0 msgr2=0x7f60d406f310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-09T13:48:02.909 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:02.899+0000 7f60d27fc640 1 --2- 172.21.15.43:0/2581549616 >> v2:172.21.15.203:3300/0 conn(0x7f60d40cc0b0 0x7f60d406f310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:48:02.909 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:02.899+0000 7f60d27fc640 1 -- 172.21.15.43:0/2581549616 --> v1:172.21.15.43:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f60d40ac670 con 0x7f60d40abe10 2023-08-09T13:48:02.909 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:02.900+0000 7f60d27fc640 1 -- 172.21.15.43:0/2581549616 <== mon.0 v1:172.21.15.43:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1285150740 0 0) 0x7f60c8001670 con 0x7f60d40abe10 2023-08-09T13:48:02.909 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:02.900+0000 7f60d27fc640 1 -- 172.21.15.43:0/2581549616 >> v1:172.21.15.43:6789/0 conn(0x7f60d40abe10 legacy=0x7f60d40cb9a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:48:02.910 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:02.900+0000 7f60d27fc640 1 --2- 172.21.15.43:0/2581549616 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f60bc003970 0x7f60bc003d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:48:02.910 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:02.900+0000 7f60d27fc640 1 --2- 172.21.15.43:0/2581549616 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f60d4070ef0 0x7f60bc0047c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:48:02.910 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:02.900+0000 7f60d27fc640 1 --2- 172.21.15.43:0/2581549616 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f60bc004d00 0x7f60bc005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:48:02.910 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:02.900+0000 7f60d27fc640 1 -- 172.21.15.43:0/2581549616 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f60d415c940 con 0x7f60bc003970 2023-08-09T13:48:02.911 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:02.900+0000 7f60d27fc640 1 -- 172.21.15.43:0/2581549616 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f60d40ac390 con 0x7f60bc004d00 2023-08-09T13:48:02.911 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:02.900+0000 7f60d27fc640 1 -- 172.21.15.43:0/2581549616 --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f60bc00bba0 con 0x7f60d4070ef0 2023-08-09T13:48:02.911 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:02.900+0000 7f60d37fe640 1 --2- 172.21.15.43:0/2581549616 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f60bc003970 0x7f60bc003d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:48:02.912 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:02.900+0000 7f60d37fe640 1 -- 172.21.15.43:0/2581549616 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f60d4070ef0 msgr2=0x7f60bc0047c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-09T13:48:02.912 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:02.900+0000 7f60d37fe640 1 --2- 172.21.15.43:0/2581549616 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f60d4070ef0 0x7f60bc0047c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:48:02.912 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:02.900+0000 7f60d37fe640 1 -- 172.21.15.43:0/2581549616 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f60bc004d00 msgr2=0x7f60bc005180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-09T13:48:02.912 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:02.900+0000 7f60d37fe640 1 --2- 172.21.15.43:0/2581549616 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f60bc004d00 0x7f60bc005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:48:02.912 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:02.900+0000 7f60d37fe640 1 -- 172.21.15.43:0/2581549616 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f60bc003520 con 0x7f60bc003970 2023-08-09T13:48:02.913 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:02.902+0000 7f60d37fe640 1 --2- 172.21.15.43:0/2581549616 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f60bc003970 0x7f60bc003d90 secure :-1 s=READY pgs=786 cs=0 l=1 rev1=1 crypto rx=0x7f60c400a9a0 tx=0x7f60bc0057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3f4c8f467973421b server_cookie=0 in_seq=0 out_seq=0 2023-08-09T13:48:02.913 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:02.902+0000 7f60d27fc640 1 -- 172.21.15.43:0/2581549616 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f60c4031430 con 0x7f60bc003970 2023-08-09T13:48:02.913 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:02.902+0000 7f60d27fc640 1 -- 172.21.15.43:0/2581549616 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f60c4031590 con 0x7f60bc003970 2023-08-09T13:48:02.913 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:02.902+0000 7f60d27fc640 1 -- 172.21.15.43:0/2581549616 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f60c4031940 con 0x7f60bc003970 2023-08-09T13:48:02.914 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:02.902+0000 7f60d885d640 1 -- 172.21.15.43:0/2581549616 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f60bc003970 msgr2=0x7f60bc003d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:48:02.914 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:02.902+0000 7f60d885d640 1 --2- 172.21.15.43:0/2581549616 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f60bc003970 0x7f60bc003d90 secure :-1 s=READY pgs=786 cs=0 l=1 rev1=1 crypto rx=0x7f60c400a9a0 tx=0x7f60bc0057f0 comp rx=0 tx=0).stop 2023-08-09T13:48:02.914 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:02.902+0000 7f60d3fff640 1 -- 172.21.15.43:0/2581549616 reap_dead start 2023-08-09T13:48:02.914 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:02.903+0000 7f60d885d640 1 -- 172.21.15.43:0/2581549616 shutdown_connections 2023-08-09T13:48:02.914 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:02.903+0000 7f60d885d640 1 -- 172.21.15.43:0/2581549616 >> 172.21.15.43:0/2581549616 conn(0x7f60d40cddd0 msgr2=0x7f60d40ce1e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-09T13:48:02.915 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:02.903+0000 7f60d885d640 1 -- 172.21.15.43:0/2581549616 shutdown_connections 2023-08-09T13:48:02.915 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:02.903+0000 7f60d885d640 1 -- 172.21.15.43:0/2581549616 wait complete. 2023-08-09T13:48:02.918 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:02.904+0000 7f60d885d640 1 Processor -- start 2023-08-09T13:48:02.918 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:02.905+0000 7f60d885d640 1 -- start start 2023-08-09T13:48:02.918 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:02.905+0000 7f60d885d640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f60bc003970 0x7f60d4145220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:48:02.919 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:02.905+0000 7f60d885d640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f60d40b36a0 0x7f60d4157020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:48:02.919 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:02.905+0000 7f60d885d640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f60d40b28a0 0x7f60d40b2d20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:48:02.919 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:02.905+0000 7f60d885d640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f60d40b3260 con 0x7f60bc003970 2023-08-09T13:48:02.919 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:02.905+0000 7f60d885d640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f60d4157560 con 0x7f60d40b36a0 2023-08-09T13:48:02.920 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:02.905+0000 7f60d885d640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f60d4145900 con 0x7f60d40b28a0 2023-08-09T13:48:02.920 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:02.906+0000 7f60d37fe640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f60bc003970 0x7f60d4145220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:48:02.920 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:02.906+0000 7f60d37fe640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f60bc003970 0x7f60d4145220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:57462/0 (socket says 172.21.15.43:57462) 2023-08-09T13:48:02.920 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:02.906+0000 7f60d37fe640 1 -- 172.21.15.43:0/2299755390 learned_addr learned my addr 172.21.15.43:0/2299755390 (peer_addr_for_me v2:172.21.15.43:0/0) 2023-08-09T13:48:02.921 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:02.906+0000 7f60d37fe640 1 -- 172.21.15.43:0/2299755390 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f60d40b28a0 msgr2=0x7f60d40b2d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:48:02.921 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:02.906+0000 7f60d37fe640 1 --2- 172.21.15.43:0/2299755390 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f60d40b28a0 0x7f60d40b2d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:48:02.921 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:02.906+0000 7f60d37fe640 1 -- 172.21.15.43:0/2299755390 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f60d40b36a0 msgr2=0x7f60d4157020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:48:02.921 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:02.906+0000 7f60d37fe640 1 --2- 172.21.15.43:0/2299755390 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f60d40b36a0 0x7f60d4157020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:48:02.922 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:02.906+0000 7f60d37fe640 1 -- 172.21.15.43:0/2299755390 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f60d4145be0 con 0x7f60bc003970 2023-08-09T13:48:02.922 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:02.907+0000 7f60d37fe640 1 --2- 172.21.15.43:0/2299755390 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f60bc003970 0x7f60d4145220 secure :-1 s=READY pgs=787 cs=0 l=1 rev1=1 crypto rx=0x7f60c400a120 tx=0x7f60c4004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-09T13:48:02.922 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:02.907+0000 7f60bbfff640 1 -- 172.21.15.43:0/2299755390 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f60c403f070 con 0x7f60bc003970 2023-08-09T13:48:02.923 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:02.907+0000 7f60bbfff640 1 -- 172.21.15.43:0/2299755390 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f60c403a7d0 con 0x7f60bc003970 2023-08-09T13:48:02.923 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:02.907+0000 7f60bbfff640 1 -- 172.21.15.43:0/2299755390 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f60c403ab40 con 0x7f60bc003970 2023-08-09T13:48:02.923 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:02.907+0000 7f60d885d640 1 -- 172.21.15.43:0/2299755390 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f60d4145ec0 con 0x7f60bc003970 2023-08-09T13:48:02.923 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:02.907+0000 7f60d885d640 1 -- 172.21.15.43:0/2299755390 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f60d415ce90 con 0x7f60bc003970 2023-08-09T13:48:02.924 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:02.908+0000 7f60d885d640 1 -- 172.21.15.43:0/2299755390 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f60d406ce90 con 0x7f60bc003970 2023-08-09T13:48:02.924 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:02.909+0000 7f60bbfff640 1 -- 172.21.15.43:0/2299755390 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 18) v1 ==== 88443+0+0 (secure 0 0 0) 0x7f60c4043070 con 0x7f60bc003970 2023-08-09T13:48:02.924 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:02.910+0000 7f60bbfff640 1 --2- 172.21.15.43:0/2299755390 >> [v2:172.21.15.203:6832/272917756,v1:172.21.15.203:6833/272917756] conn(0x7f60c006a170 0x7f60c006c630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:48:02.924 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:02.910+0000 7f60bbfff640 1 -- 172.21.15.43:0/2299755390 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(64..64 src has 1..64) v4 ==== 7215+0+0 (secure 0 0 0) 0x7f60c40aa4b0 con 0x7f60bc003970 2023-08-09T13:48:02.925 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:02.911+0000 7f60d2ffd640 1 --2- 172.21.15.43:0/2299755390 >> [v2:172.21.15.203:6832/272917756,v1:172.21.15.203:6833/272917756] conn(0x7f60c006a170 0x7f60c006c630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:48:02.925 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:02.912+0000 7f60d2ffd640 1 --2- 172.21.15.43:0/2299755390 >> [v2:172.21.15.203:6832/272917756,v1:172.21.15.203:6833/272917756] conn(0x7f60c006a170 0x7f60c006c630 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0x7f60d406b930 tx=0x7f60c8001480 comp rx=0 tx=0).ready entity=mgr.5533 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-09T13:48:02.925 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:02.914+0000 7f60bbfff640 1 -- 172.21.15.43:0/2299755390 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178066 (secure 0 0 0) 0x7f60c4078f90 con 0x7f60bc003970 2023-08-09T13:48:02.925 DEBUG:teuthology.orchestra.run.smithi043:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-08-09T13:48:02.932 DEBUG:teuthology.orchestra.run.smithi203:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-08-09T13:48:03.140 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:03.137+0000 7f60d885d640 1 -- 172.21.15.43:0/2299755390 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} v 0) v1 -- 0x7f60d404e390 con 0x7f60bc003970 2023-08-09T13:48:03.687 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:03.684+0000 7f60bbfff640 1 -- 172.21.15.43:0/2299755390 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]=0 v19) v1 ==== 104+0+0 (secure 0 0 0) 0x7f60c4078b20 con 0x7f60bc003970 2023-08-09T13:48:03.692 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:03.689+0000 7f60d885d640 1 -- 172.21.15.43:0/2299755390 >> [v2:172.21.15.203:6832/272917756,v1:172.21.15.203:6833/272917756] conn(0x7f60c006a170 msgr2=0x7f60c006c630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:48:03.692 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:03.689+0000 7f60d885d640 1 --2- 172.21.15.43:0/2299755390 >> [v2:172.21.15.203:6832/272917756,v1:172.21.15.203:6833/272917756] conn(0x7f60c006a170 0x7f60c006c630 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0x7f60d406b930 tx=0x7f60c8001480 comp rx=0 tx=0).stop 2023-08-09T13:48:03.692 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:03.689+0000 7f60d885d640 1 -- 172.21.15.43:0/2299755390 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f60bc003970 msgr2=0x7f60d4145220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:48:03.692 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:03.689+0000 7f60d885d640 1 --2- 172.21.15.43:0/2299755390 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f60bc003970 0x7f60d4145220 secure :-1 s=READY pgs=787 cs=0 l=1 rev1=1 crypto rx=0x7f60c400a120 tx=0x7f60c4004730 comp rx=0 tx=0).stop 2023-08-09T13:48:03.693 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:03.689+0000 7f60d885d640 1 -- 172.21.15.43:0/2299755390 shutdown_connections 2023-08-09T13:48:03.693 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:03.689+0000 7f60d885d640 1 --2- 172.21.15.43:0/2299755390 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f60d40b28a0 0x7f60d40b2d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:48:03.693 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:03.689+0000 7f60d885d640 1 --2- 172.21.15.43:0/2299755390 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f60d40b36a0 0x7f60d4157020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:48:03.693 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:03.689+0000 7f60d885d640 1 --2- 172.21.15.43:0/2299755390 >> [v2:172.21.15.203:6832/272917756,v1:172.21.15.203:6833/272917756] conn(0x7f60c006a170 0x7f60c006c630 unknown :-1 s=CLOSED pgs=32 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:48:03.693 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:03.689+0000 7f60d885d640 1 --2- 172.21.15.43:0/2299755390 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f60bc003970 0x7f60d4145220 unknown :-1 s=CLOSED pgs=787 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:48:03.693 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:03.689+0000 7f60d885d640 1 -- 172.21.15.43:0/2299755390 >> 172.21.15.43:0/2299755390 conn(0x7f60d40cddd0 msgr2=0x7f60d40b09f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-09T13:48:03.694 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:03.690+0000 7f60d885d640 1 -- 172.21.15.43:0/2299755390 shutdown_connections 2023-08-09T13:48:03.694 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:03.690+0000 7f60d885d640 1 -- 172.21.15.43:0/2299755390 wait complete. 2023-08-09T13:48:03.712 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_metadata cephfs_metadata --yes-i-really-really-mean-it 2023-08-09T13:48:03.946 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:03.943+0000 7ff07a6b0640 1 Processor -- start 2023-08-09T13:48:03.946 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:03.943+0000 7ff07a6b0640 1 -- start start 2023-08-09T13:48:03.947 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:03.945+0000 7ff07a6b0640 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7ff0740cf370 0x7ff074167220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:48:03.948 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:03.945+0000 7ff07a6b0640 1 -- --> v1:172.21.15.43:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff0740cdce0 con 0x7ff07415c720 2023-08-09T13:48:03.948 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:03.945+0000 7ff07a6b0640 1 -- --> v1:172.21.15.203:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff0740b5660 con 0x7ff0740cd850 2023-08-09T13:48:03.948 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:03.945+0000 7ff07a6b0640 1 -- --> v2:172.21.15.203:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff07415cb00 con 0x7ff0740cf370 2023-08-09T13:48:03.948 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:03.945+0000 7ff0796ae640 1 --1- >> v1:172.21.15.43:6789/0 conn(0x7ff07415c720 0x7ff0740d0900 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.43:6789/0 says I am v1:172.21.15.43:36810/0 (socket says 172.21.15.43:36810) 2023-08-09T13:48:03.949 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:03.946+0000 7ff0796ae640 1 -- 172.21.15.43:0/2516972716 learned_addr learned my addr 172.21.15.43:0/2516972716 (peer_addr_for_me v1:172.21.15.43:0/0) 2023-08-09T13:48:03.949 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:03.946+0000 7ff079eaf640 1 --2- 172.21.15.43:0/2516972716 >> v2:172.21.15.203:3300/0 conn(0x7ff0740cf370 0x7ff074167220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:48:03.949 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:03.946+0000 7ff06bfff640 1 -- 172.21.15.43:0/2516972716 <== mon.0 v1:172.21.15.43:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2586072395 0 0) 0x7ff0740cdce0 con 0x7ff07415c720 2023-08-09T13:48:03.949 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:03.946+0000 7ff06bfff640 1 -- 172.21.15.43:0/2516972716 --> v1:172.21.15.43:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ff060003520 con 0x7ff07415c720 2023-08-09T13:48:03.950 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:03.946+0000 7ff06bfff640 1 -- 172.21.15.43:0/2516972716 <== mon.0 v1:172.21.15.43:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2531782152 0 0) 0x7ff060003520 con 0x7ff07415c720 2023-08-09T13:48:03.950 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:03.946+0000 7ff06bfff640 1 -- 172.21.15.43:0/2516972716 >> v2:172.21.15.203:3300/0 conn(0x7ff0740cf370 msgr2=0x7ff074167220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-09T13:48:03.951 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:03.946+0000 7ff06bfff640 1 --2- 172.21.15.43:0/2516972716 >> v2:172.21.15.203:3300/0 conn(0x7ff0740cf370 0x7ff074167220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:48:03.951 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:03.946+0000 7ff06bfff640 1 -- 172.21.15.43:0/2516972716 >> v1:172.21.15.203:6789/0 conn(0x7ff0740cd850 legacy=0x7ff0740cec60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:48:03.951 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:03.946+0000 7ff06bfff640 1 -- 172.21.15.43:0/2516972716 --> v1:172.21.15.43:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff0740b5900 con 0x7ff07415c720 2023-08-09T13:48:03.951 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:03.947+0000 7ff06bfff640 1 -- 172.21.15.43:0/2516972716 <== mon.0 v1:172.21.15.43:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1285150740 0 0) 0x7ff070001670 con 0x7ff07415c720 2023-08-09T13:48:03.951 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:03.947+0000 7ff06bfff640 1 -- 172.21.15.43:0/2516972716 >> v1:172.21.15.43:6789/0 conn(0x7ff07415c720 legacy=0x7ff0740d0900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:48:03.951 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:03.947+0000 7ff06bfff640 1 --2- 172.21.15.43:0/2516972716 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7ff060003970 0x7ff060003d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:48:03.952 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:03.947+0000 7ff06bfff640 1 --2- 172.21.15.43:0/2516972716 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff0600047c0 0x7ff060004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:48:03.952 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:03.947+0000 7ff06bfff640 1 --2- 172.21.15.43:0/2516972716 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7ff060005180 0x7ff06000b980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:48:03.952 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:03.947+0000 7ff06bfff640 1 -- 172.21.15.43:0/2516972716 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff07415cb00 con 0x7ff0600047c0 2023-08-09T13:48:03.952 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:03.947+0000 7ff06bfff640 1 -- 172.21.15.43:0/2516972716 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff06000c020 con 0x7ff060003970 2023-08-09T13:48:03.952 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:03.947+0000 7ff06bfff640 1 -- 172.21.15.43:0/2516972716 --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff06000c1e0 con 0x7ff060005180 2023-08-09T13:48:03.952 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:03.947+0000 7ff078ead640 1 --2- 172.21.15.43:0/2516972716 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff0600047c0 0x7ff060004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:48:03.952 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:03.947+0000 7ff078ead640 1 -- 172.21.15.43:0/2516972716 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7ff060005180 msgr2=0x7ff06000b980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-09T13:48:03.952 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:03.947+0000 7ff078ead640 1 --2- 172.21.15.43:0/2516972716 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7ff060005180 0x7ff06000b980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:48:03.953 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:03.947+0000 7ff078ead640 1 -- 172.21.15.43:0/2516972716 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7ff060003970 msgr2=0x7ff060003d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-09T13:48:03.953 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:03.947+0000 7ff078ead640 1 --2- 172.21.15.43:0/2516972716 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7ff060003970 0x7ff060003d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:48:03.953 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:03.947+0000 7ff078ead640 1 -- 172.21.15.43:0/2516972716 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff060003520 con 0x7ff0600047c0 2023-08-09T13:48:03.953 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:03.947+0000 7ff079eaf640 1 -- 172.21.15.43:0/2516972716 reap_dead start 2023-08-09T13:48:03.954 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:03.948+0000 7ff078ead640 1 --2- 172.21.15.43:0/2516972716 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff0600047c0 0x7ff060004c40 secure :-1 s=READY pgs=789 cs=0 l=1 rev1=1 crypto rx=0x7ff064001750 tx=0x7ff0600057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7d7bb937958cfb26 server_cookie=0 in_seq=0 out_seq=0 2023-08-09T13:48:03.954 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:03.949+0000 7ff06bfff640 1 -- 172.21.15.43:0/2516972716 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff0640052d0 con 0x7ff0600047c0 2023-08-09T13:48:03.954 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:03.949+0000 7ff06bfff640 1 -- 172.21.15.43:0/2516972716 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff06402e050 con 0x7ff0600047c0 2023-08-09T13:48:03.955 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:03.949+0000 7ff06bfff640 1 -- 172.21.15.43:0/2516972716 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff064003830 con 0x7ff0600047c0 2023-08-09T13:48:03.955 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:03.949+0000 7ff07a6b0640 1 -- 172.21.15.43:0/2516972716 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff0600047c0 msgr2=0x7ff060004c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:48:03.955 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:03.949+0000 7ff07a6b0640 1 --2- 172.21.15.43:0/2516972716 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff0600047c0 0x7ff060004c40 secure :-1 s=READY pgs=789 cs=0 l=1 rev1=1 crypto rx=0x7ff064001750 tx=0x7ff0600057f0 comp rx=0 tx=0).stop 2023-08-09T13:48:03.955 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:03.949+0000 7ff07a6b0640 1 -- 172.21.15.43:0/2516972716 shutdown_connections 2023-08-09T13:48:03.955 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:03.949+0000 7ff07a6b0640 1 --2- 172.21.15.43:0/2516972716 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff0600047c0 0x7ff060004c40 unknown :-1 s=CLOSED pgs=789 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:48:03.955 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:03.949+0000 7ff07a6b0640 1 -- 172.21.15.43:0/2516972716 >> 172.21.15.43:0/2516972716 conn(0x7ff07406a2a0 msgr2=0x7ff07406a680 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-09T13:48:03.955 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:03.949+0000 7ff07a6b0640 1 -- 172.21.15.43:0/2516972716 shutdown_connections 2023-08-09T13:48:03.956 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:03.949+0000 7ff07a6b0640 1 -- 172.21.15.43:0/2516972716 wait complete. 2023-08-09T13:48:03.956 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:03.951+0000 7ff07a6b0640 1 Processor -- start 2023-08-09T13:48:03.956 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:03.951+0000 7ff07a6b0640 1 -- start start 2023-08-09T13:48:03.957 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:03.952+0000 7ff07a6b0640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7ff0600047c0 0x7ff0740b31c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:48:03.958 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:03.952+0000 7ff07a6b0640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff0740b1bb0 0x7ff0740ca840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:48:03.958 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:03.952+0000 7ff07a6b0640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7ff074152820 0x7ff074149360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:48:03.958 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:03.952+0000 7ff07a6b0640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff0740b2d30 con 0x7ff0740b1bb0 2023-08-09T13:48:03.958 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:03.952+0000 7ff07a6b0640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff0740b2120 con 0x7ff074152820 2023-08-09T13:48:03.958 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:03.952+0000 7ff07a6b0640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff0740b2400 con 0x7ff0600047c0 2023-08-09T13:48:03.958 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:03.953+0000 7ff078ead640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff0740b1bb0 0x7ff0740ca840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:48:03.958 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:03.953+0000 7ff078ead640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff0740b1bb0 0x7ff0740ca840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:57480/0 (socket says 172.21.15.43:57480) 2023-08-09T13:48:03.959 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:03.953+0000 7ff078ead640 1 -- 172.21.15.43:0/3688580878 learned_addr learned my addr 172.21.15.43:0/3688580878 (peer_addr_for_me v2:172.21.15.43:0/0) 2023-08-09T13:48:03.959 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:03.953+0000 7ff0796ae640 1 --2- 172.21.15.43:0/3688580878 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7ff0600047c0 0x7ff0740b31c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:48:03.959 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:03.953+0000 7ff078ead640 1 -- 172.21.15.43:0/3688580878 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7ff0600047c0 msgr2=0x7ff0740b31c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:48:03.959 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:03.953+0000 7ff078ead640 1 --2- 172.21.15.43:0/3688580878 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7ff0600047c0 0x7ff0740b31c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:48:03.959 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:03.953+0000 7ff078ead640 1 -- 172.21.15.43:0/3688580878 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7ff074152820 msgr2=0x7ff074149360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:48:03.959 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:03.953+0000 7ff078ead640 1 --2- 172.21.15.43:0/3688580878 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7ff074152820 0x7ff074149360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:48:03.959 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:03.953+0000 7ff078ead640 1 -- 172.21.15.43:0/3688580878 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff0740b26e0 con 0x7ff0740b1bb0 2023-08-09T13:48:03.959 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:03.953+0000 7ff078ead640 1 --2- 172.21.15.43:0/3688580878 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff0740b1bb0 0x7ff0740ca840 secure :-1 s=READY pgs=790 cs=0 l=1 rev1=1 crypto rx=0x7ff06402bc10 tx=0x7ff064001120 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-09T13:48:03.960 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:03.954+0000 7ff069ffb640 1 -- 172.21.15.43:0/3688580878 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff064007a30 con 0x7ff0740b1bb0 2023-08-09T13:48:03.960 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:03.954+0000 7ff069ffb640 1 -- 172.21.15.43:0/3688580878 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff064007640 con 0x7ff0740b1bb0 2023-08-09T13:48:03.960 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:03.954+0000 7ff069ffb640 1 -- 172.21.15.43:0/3688580878 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff064006be0 con 0x7ff0740b1bb0 2023-08-09T13:48:03.960 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:03.954+0000 7ff07a6b0640 1 -- 172.21.15.43:0/3688580878 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff0740b29c0 con 0x7ff0740b1bb0 2023-08-09T13:48:03.960 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:03.954+0000 7ff07a6b0640 1 -- 172.21.15.43:0/3688580878 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff074152f00 con 0x7ff0740b1bb0 2023-08-09T13:48:03.960 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:03.955+0000 7ff07a6b0640 1 -- 172.21.15.43:0/3688580878 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff0741530f0 con 0x7ff0740b1bb0 2023-08-09T13:48:03.961 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:03.956+0000 7ff069ffb640 1 -- 172.21.15.43:0/3688580878 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 18) v1 ==== 88443+0+0 (secure 0 0 0) 0x7ff064006d40 con 0x7ff0740b1bb0 2023-08-09T13:48:03.961 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:03.957+0000 7ff069ffb640 1 --2- 172.21.15.43:0/3688580878 >> [v2:172.21.15.203:6832/272917756,v1:172.21.15.203:6833/272917756] conn(0x7ff05806a170 0x7ff05806c630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:48:03.961 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:03.957+0000 7ff069ffb640 1 -- 172.21.15.43:0/3688580878 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(64..64 src has 1..64) v4 ==== 7215+0+0 (secure 0 0 0) 0x7ff0640aa920 con 0x7ff0740b1bb0 2023-08-09T13:48:03.961 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:03.958+0000 7ff0796ae640 1 --2- 172.21.15.43:0/3688580878 >> [v2:172.21.15.203:6832/272917756,v1:172.21.15.203:6833/272917756] conn(0x7ff05806a170 0x7ff05806c630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:48:03.962 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:03.959+0000 7ff0796ae640 1 --2- 172.21.15.43:0/3688580878 >> [v2:172.21.15.203:6832/272917756,v1:172.21.15.203:6833/272917756] conn(0x7ff05806a170 0x7ff05806c630 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0x7ff070001eb0 tx=0x7ff070009040 comp rx=0 tx=0).ready entity=mgr.5533 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-09T13:48:03.964 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:03.962+0000 7ff069ffb640 1 -- 172.21.15.43:0/3688580878 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178066 (secure 0 0 0) 0x7ff064079510 con 0x7ff0740b1bb0 2023-08-09T13:48:04.160 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:04.157+0000 7ff07a6b0640 1 -- 172.21.15.43:0/3688580878 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7ff0740cd670 con 0x7ff0740b1bb0 2023-08-09T13:48:04.702 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:04.698+0000 7ff069ffb640 1 -- 172.21.15.43:0/3688580878 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_metadata' removed v65) v1 ==== 181+0+0 (secure 0 0 0) 0x7ff0640790a0 con 0x7ff0740b1bb0 2023-08-09T13:48:04.703 INFO:teuthology.orchestra.run.smithi043.stderr:pool 'cephfs_metadata' removed 2023-08-09T13:48:04.708 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:04.706+0000 7ff07a6b0640 1 -- 172.21.15.43:0/3688580878 >> [v2:172.21.15.203:6832/272917756,v1:172.21.15.203:6833/272917756] conn(0x7ff05806a170 msgr2=0x7ff05806c630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:48:04.708 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:04.706+0000 7ff07a6b0640 1 --2- 172.21.15.43:0/3688580878 >> [v2:172.21.15.203:6832/272917756,v1:172.21.15.203:6833/272917756] conn(0x7ff05806a170 0x7ff05806c630 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0x7ff070001eb0 tx=0x7ff070009040 comp rx=0 tx=0).stop 2023-08-09T13:48:04.709 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:04.706+0000 7ff07a6b0640 1 -- 172.21.15.43:0/3688580878 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff0740b1bb0 msgr2=0x7ff0740ca840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:48:04.709 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:04.706+0000 7ff07a6b0640 1 --2- 172.21.15.43:0/3688580878 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff0740b1bb0 0x7ff0740ca840 secure :-1 s=READY pgs=790 cs=0 l=1 rev1=1 crypto rx=0x7ff06402bc10 tx=0x7ff064001120 comp rx=0 tx=0).stop 2023-08-09T13:48:04.709 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:04.707+0000 7ff07a6b0640 1 -- 172.21.15.43:0/3688580878 shutdown_connections 2023-08-09T13:48:04.709 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:04.707+0000 7ff07a6b0640 1 --2- 172.21.15.43:0/3688580878 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7ff074152820 0x7ff074149360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:48:04.710 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:04.707+0000 7ff07a6b0640 1 --2- 172.21.15.43:0/3688580878 >> [v2:172.21.15.203:6832/272917756,v1:172.21.15.203:6833/272917756] conn(0x7ff05806a170 0x7ff05806c630 unknown :-1 s=CLOSED pgs=34 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:48:04.710 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:04.707+0000 7ff07a6b0640 1 --2- 172.21.15.43:0/3688580878 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff0740b1bb0 0x7ff0740ca840 unknown :-1 s=CLOSED pgs=790 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:48:04.710 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:04.707+0000 7ff07a6b0640 1 --2- 172.21.15.43:0/3688580878 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7ff0600047c0 0x7ff0740b31c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:48:04.710 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:04.707+0000 7ff07a6b0640 1 -- 172.21.15.43:0/3688580878 >> 172.21.15.43:0/3688580878 conn(0x7ff07406a2a0 msgr2=0x7ff0740d1ae0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-09T13:48:04.710 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:04.707+0000 7ff07a6b0640 1 -- 172.21.15.43:0/3688580878 shutdown_connections 2023-08-09T13:48:04.711 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:04.708+0000 7ff07a6b0640 1 -- 172.21.15.43:0/3688580878 wait complete. 2023-08-09T13:48:04.734 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_data cephfs_data --yes-i-really-really-mean-it 2023-08-09T13:48:04.962 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:04.959+0000 7f77f55e1640 1 Processor -- start 2023-08-09T13:48:04.963 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:04.960+0000 7f77f55e1640 1 -- start start 2023-08-09T13:48:04.964 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:04.961+0000 7f77f55e1640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f77f015c6f0 0x7f77f00d03a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:48:04.964 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:04.961+0000 7f77f55e1640 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7f77f00b80a0 0x7f77f00b5470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:48:04.964 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:04.961+0000 7f77f55e1640 1 -- --> v1:172.21.15.43:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f77f00d1890 con 0x7f77f00b59b0 2023-08-09T13:48:04.964 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:04.961+0000 7f77f55e1640 1 -- --> v2:172.21.15.203:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f77f00d2d30 con 0x7f77f00b80a0 2023-08-09T13:48:04.965 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:04.961+0000 7f77f55e1640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f77f015cad0 con 0x7f77f015c6f0 2023-08-09T13:48:04.965 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:04.962+0000 7f77ef7fe640 1 --1- >> v1:172.21.15.43:6789/0 conn(0x7f77f00b59b0 0x7f77f00b3e60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.43:6789/0 says I am v1:172.21.15.43:41644/0 (socket says 172.21.15.43:41644) 2023-08-09T13:48:04.965 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:04.962+0000 7f77ef7fe640 1 -- 172.21.15.43:0/886018349 learned_addr learned my addr 172.21.15.43:0/886018349 (peer_addr_for_me v1:172.21.15.43:0/0) 2023-08-09T13:48:04.966 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:04.962+0000 7f77eeffd640 1 --2- 172.21.15.43:0/886018349 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f77f015c6f0 0x7f77f00d03a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:48:04.966 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:04.962+0000 7f77ee7fc640 1 --2- 172.21.15.43:0/886018349 >> v2:172.21.15.203:3300/0 conn(0x7f77f00b80a0 0x7f77f00b5470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:48:04.967 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:04.962+0000 7f77edffb640 1 -- 172.21.15.43:0/886018349 <== mon.0 v1:172.21.15.43:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2710661400 0 0) 0x7f77f00d1890 con 0x7f77f00b59b0 2023-08-09T13:48:04.967 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:04.962+0000 7f77edffb640 1 -- 172.21.15.43:0/886018349 --> v1:172.21.15.43:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f77d8003520 con 0x7f77f00b59b0 2023-08-09T13:48:04.967 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:04.963+0000 7f77edffb640 1 -- 172.21.15.43:0/886018349 <== mon.0 v1:172.21.15.43:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1578263372 0 0) 0x7f77d8003520 con 0x7f77f00b59b0 2023-08-09T13:48:04.967 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:04.963+0000 7f77edffb640 1 -- 172.21.15.43:0/886018349 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f77f015c6f0 msgr2=0x7f77f00d03a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-09T13:48:04.967 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:04.963+0000 7f77edffb640 1 --2- 172.21.15.43:0/886018349 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f77f015c6f0 0x7f77f00d03a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:48:04.967 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:04.963+0000 7f77edffb640 1 -- 172.21.15.43:0/886018349 >> v2:172.21.15.203:3300/0 conn(0x7f77f00b80a0 msgr2=0x7f77f00b5470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-09T13:48:04.967 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:04.963+0000 7f77edffb640 1 --2- 172.21.15.43:0/886018349 >> v2:172.21.15.203:3300/0 conn(0x7f77f00b80a0 0x7f77f00b5470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:48:04.968 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:04.963+0000 7f77edffb640 1 -- 172.21.15.43:0/886018349 --> v1:172.21.15.43:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f77f00d0c80 con 0x7f77f00b59b0 2023-08-09T13:48:04.968 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:04.963+0000 7f77edffb640 1 -- 172.21.15.43:0/886018349 <== mon.0 v1:172.21.15.43:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1285150740 0 0) 0x7f77e80019f0 con 0x7f77f00b59b0 2023-08-09T13:48:04.968 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:04.963+0000 7f77edffb640 1 -- 172.21.15.43:0/886018349 >> v1:172.21.15.43:6789/0 conn(0x7f77f00b59b0 legacy=0x7f77f00b3e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:48:04.968 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:04.963+0000 7f77edffb640 1 --2- 172.21.15.43:0/886018349 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f77d80039e0 0x7f77d8003e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:48:04.968 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:04.963+0000 7f77edffb640 1 --2- 172.21.15.43:0/886018349 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f77d80047c0 0x7f77d8004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:48:04.968 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:04.963+0000 7f77edffb640 1 --2- 172.21.15.43:0/886018349 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f77f015c6f0 0x7f77d8005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:48:04.968 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:04.963+0000 7f77edffb640 1 -- 172.21.15.43:0/886018349 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f77f00d2d30 con 0x7f77d80039e0 2023-08-09T13:48:04.968 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:04.963+0000 7f77edffb640 1 -- 172.21.15.43:0/886018349 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f77f015cad0 con 0x7f77d80047c0 2023-08-09T13:48:04.969 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:04.963+0000 7f77edffb640 1 -- 172.21.15.43:0/886018349 --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f77d800b980 con 0x7f77f015c6f0 2023-08-09T13:48:04.969 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:04.963+0000 7f77eeffd640 1 --2- 172.21.15.43:0/886018349 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f77d80039e0 0x7f77d8003e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:48:04.969 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:04.964+0000 7f77eeffd640 1 -- 172.21.15.43:0/886018349 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f77f015c6f0 msgr2=0x7f77d8005180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-09T13:48:04.969 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:04.964+0000 7f77eeffd640 1 --2- 172.21.15.43:0/886018349 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f77f015c6f0 0x7f77d8005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:48:04.969 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:04.964+0000 7f77eeffd640 1 -- 172.21.15.43:0/886018349 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f77d80047c0 msgr2=0x7f77d8004c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-09T13:48:04.969 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:04.964+0000 7f77eeffd640 1 --2- 172.21.15.43:0/886018349 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f77d80047c0 0x7f77d8004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:48:04.969 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:04.964+0000 7f77eeffd640 1 -- 172.21.15.43:0/886018349 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f77d8003520 con 0x7f77d80039e0 2023-08-09T13:48:04.970 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:04.965+0000 7f77eeffd640 1 --2- 172.21.15.43:0/886018349 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f77d80039e0 0x7f77d8003e00 secure :-1 s=READY pgs=792 cs=0 l=1 rev1=1 crypto rx=0x7f77e0009af0 tx=0x7f77d80057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=111b41d864932548 server_cookie=0 in_seq=0 out_seq=0 2023-08-09T13:48:04.971 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:04.965+0000 7f77edffb640 1 -- 172.21.15.43:0/886018349 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f77e002c710 con 0x7f77d80039e0 2023-08-09T13:48:04.971 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:04.965+0000 7f77edffb640 1 -- 172.21.15.43:0/886018349 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f77e002c870 con 0x7f77d80039e0 2023-08-09T13:48:04.971 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:04.965+0000 7f77edffb640 1 -- 172.21.15.43:0/886018349 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f77e002cc00 con 0x7f77d80039e0 2023-08-09T13:48:04.971 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:04.965+0000 7f77f55e1640 1 -- 172.21.15.43:0/886018349 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f77d80039e0 msgr2=0x7f77d8003e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:48:04.971 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:04.965+0000 7f77f55e1640 1 --2- 172.21.15.43:0/886018349 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f77d80039e0 0x7f77d8003e00 secure :-1 s=READY pgs=792 cs=0 l=1 rev1=1 crypto rx=0x7f77e0009af0 tx=0x7f77d80057f0 comp rx=0 tx=0).stop 2023-08-09T13:48:04.971 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:04.966+0000 7f77ef7fe640 1 -- 172.21.15.43:0/886018349 reap_dead start 2023-08-09T13:48:04.972 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:04.966+0000 7f77f55e1640 1 -- 172.21.15.43:0/886018349 shutdown_connections 2023-08-09T13:48:04.972 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:04.966+0000 7f77f55e1640 1 -- 172.21.15.43:0/886018349 >> 172.21.15.43:0/886018349 conn(0x7f77f00712e0 msgr2=0x7f77f00716f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-09T13:48:04.972 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:04.966+0000 7f77f55e1640 1 -- 172.21.15.43:0/886018349 shutdown_connections 2023-08-09T13:48:04.972 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:04.966+0000 7f77f55e1640 1 -- 172.21.15.43:0/886018349 wait complete. 2023-08-09T13:48:04.972 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:04.968+0000 7f77f55e1640 1 Processor -- start 2023-08-09T13:48:04.972 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:04.968+0000 7f77f55e1640 1 -- start start 2023-08-09T13:48:04.973 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:04.969+0000 7f77f55e1640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f77d80039e0 0x7f77f014a180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:48:04.974 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:04.969+0000 7f77f55e1640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f77f01f1ad0 0x7f77f01f1f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:48:04.974 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:04.969+0000 7f77f55e1640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f77f00a9000 0x7f77f00a9490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:48:04.974 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:04.969+0000 7f77f55e1640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f77f00b4b60 con 0x7f77d80039e0 2023-08-09T13:48:04.974 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:04.969+0000 7f77f55e1640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f77f01f2490 con 0x7f77f00a9000 2023-08-09T13:48:04.974 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:04.969+0000 7f77f55e1640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f77f014a6c0 con 0x7f77f01f1ad0 2023-08-09T13:48:04.974 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:04.969+0000 7f77eeffd640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f77d80039e0 0x7f77f014a180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:48:04.974 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:04.969+0000 7f77eeffd640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f77d80039e0 0x7f77f014a180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:35022/0 (socket says 172.21.15.43:35022) 2023-08-09T13:48:04.975 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:04.969+0000 7f77eeffd640 1 -- 172.21.15.43:0/1621775052 learned_addr learned my addr 172.21.15.43:0/1621775052 (peer_addr_for_me v2:172.21.15.43:0/0) 2023-08-09T13:48:04.975 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:04.969+0000 7f77eeffd640 1 -- 172.21.15.43:0/1621775052 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f77f01f1ad0 msgr2=0x7f77f01f1f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:48:04.975 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:04.969+0000 7f77eeffd640 1 --2- 172.21.15.43:0/1621775052 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f77f01f1ad0 0x7f77f01f1f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:48:04.975 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:04.970+0000 7f77eeffd640 1 -- 172.21.15.43:0/1621775052 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f77f00a9000 msgr2=0x7f77f00a9490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:48:04.975 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:04.970+0000 7f77eeffd640 1 --2- 172.21.15.43:0/1621775052 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f77f00a9000 0x7f77f00a9490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:48:04.975 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:04.970+0000 7f77eeffd640 1 -- 172.21.15.43:0/1621775052 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f77f00a9d70 con 0x7f77d80039e0 2023-08-09T13:48:04.975 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:04.970+0000 7f77eeffd640 1 --2- 172.21.15.43:0/1621775052 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f77d80039e0 0x7f77f014a180 secure :-1 s=READY pgs=793 cs=0 l=1 rev1=1 crypto rx=0x7f77e002c2b0 tx=0x7f77e0004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-09T13:48:04.975 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:04.970+0000 7f77d77fe640 1 -- 172.21.15.43:0/1621775052 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f77e0003810 con 0x7f77d80039e0 2023-08-09T13:48:04.976 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:04.970+0000 7f77d77fe640 1 -- 172.21.15.43:0/1621775052 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f77e002de90 con 0x7f77d80039e0 2023-08-09T13:48:04.976 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:04.970+0000 7f77d77fe640 1 -- 172.21.15.43:0/1621775052 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f77e0003b70 con 0x7f77d80039e0 2023-08-09T13:48:04.976 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:04.970+0000 7f77f55e1640 1 -- 172.21.15.43:0/1621775052 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f77f00aa050 con 0x7f77d80039e0 2023-08-09T13:48:04.976 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:04.971+0000 7f77f55e1640 1 -- 172.21.15.43:0/1621775052 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f77f00aa610 con 0x7f77d80039e0 2023-08-09T13:48:04.977 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:04.972+0000 7f77f55e1640 1 -- 172.21.15.43:0/1621775052 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f77f00739f0 con 0x7f77d80039e0 2023-08-09T13:48:04.977 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:04.973+0000 7f77d77fe640 1 -- 172.21.15.43:0/1621775052 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 18) v1 ==== 88443+0+0 (secure 0 0 0) 0x7f77e0003cd0 con 0x7f77d80039e0 2023-08-09T13:48:04.977 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:04.974+0000 7f77d77fe640 1 --2- 172.21.15.43:0/1621775052 >> [v2:172.21.15.203:6832/272917756,v1:172.21.15.203:6833/272917756] conn(0x7f77dc06a170 0x7f77dc06c630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:48:04.977 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:04.974+0000 7f77d77fe640 1 -- 172.21.15.43:0/1621775052 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(65..65 src has 1..65) v4 ==== 6777+0+0 (secure 0 0 0) 0x7f77e00aa7f0 con 0x7f77d80039e0 2023-08-09T13:48:04.977 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:04.974+0000 7f77ee7fc640 1 --2- 172.21.15.43:0/1621775052 >> [v2:172.21.15.203:6832/272917756,v1:172.21.15.203:6833/272917756] conn(0x7f77dc06a170 0x7f77dc06c630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:48:04.978 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:04.976+0000 7f77ee7fc640 1 --2- 172.21.15.43:0/1621775052 >> [v2:172.21.15.203:6832/272917756,v1:172.21.15.203:6833/272917756] conn(0x7f77dc06a170 0x7f77dc06c630 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7f77e40093d0 tx=0x7f77e4009300 comp rx=0 tx=0).ready entity=mgr.5533 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-09T13:48:04.981 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:04.979+0000 7f77d77fe640 1 -- 172.21.15.43:0/1621775052 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178066 (secure 0 0 0) 0x7f77e0075b90 con 0x7f77d80039e0 2023-08-09T13:48:05.180 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:05.177+0000 7f77f55e1640 1 -- 172.21.15.43:0/1621775052 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7f77f00cc9f0 con 0x7f77d80039e0 2023-08-09T13:48:05.719 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:05.715+0000 7f77d77fe640 1 -- 172.21.15.43:0/1621775052 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data' removed v66) v1 ==== 169+0+0 (secure 0 0 0) 0x7f77e00791b0 con 0x7f77d80039e0 2023-08-09T13:48:05.719 INFO:teuthology.orchestra.run.smithi043.stderr:pool 'cephfs_data' removed 2023-08-09T13:48:05.724 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:05.722+0000 7f77f55e1640 1 -- 172.21.15.43:0/1621775052 >> [v2:172.21.15.203:6832/272917756,v1:172.21.15.203:6833/272917756] conn(0x7f77dc06a170 msgr2=0x7f77dc06c630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:48:05.725 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:05.722+0000 7f77f55e1640 1 --2- 172.21.15.43:0/1621775052 >> [v2:172.21.15.203:6832/272917756,v1:172.21.15.203:6833/272917756] conn(0x7f77dc06a170 0x7f77dc06c630 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7f77e40093d0 tx=0x7f77e4009300 comp rx=0 tx=0).stop 2023-08-09T13:48:05.725 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:05.722+0000 7f77f55e1640 1 -- 172.21.15.43:0/1621775052 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f77d80039e0 msgr2=0x7f77f014a180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:48:05.725 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:05.722+0000 7f77f55e1640 1 --2- 172.21.15.43:0/1621775052 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f77d80039e0 0x7f77f014a180 secure :-1 s=READY pgs=793 cs=0 l=1 rev1=1 crypto rx=0x7f77e002c2b0 tx=0x7f77e0004730 comp rx=0 tx=0).stop 2023-08-09T13:48:05.726 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:05.722+0000 7f77f55e1640 1 -- 172.21.15.43:0/1621775052 shutdown_connections 2023-08-09T13:48:05.726 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:05.722+0000 7f77f55e1640 1 --2- 172.21.15.43:0/1621775052 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f77f00a9000 0x7f77f00a9490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:48:05.726 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:05.722+0000 7f77f55e1640 1 --2- 172.21.15.43:0/1621775052 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f77f01f1ad0 0x7f77f01f1f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:48:05.727 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:05.722+0000 7f77f55e1640 1 --2- 172.21.15.43:0/1621775052 >> [v2:172.21.15.203:6832/272917756,v1:172.21.15.203:6833/272917756] conn(0x7f77dc06a170 0x7f77dc06c630 unknown :-1 s=CLOSED pgs=35 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:48:05.727 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:05.722+0000 7f77f55e1640 1 --2- 172.21.15.43:0/1621775052 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f77d80039e0 0x7f77f014a180 unknown :-1 s=CLOSED pgs=793 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:48:05.727 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:05.722+0000 7f77f55e1640 1 -- 172.21.15.43:0/1621775052 >> 172.21.15.43:0/1621775052 conn(0x7f77f00712e0 msgr2=0x7f77f006a420 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-09T13:48:05.727 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:05.723+0000 7f77f55e1640 1 -- 172.21.15.43:0/1621775052 shutdown_connections 2023-08-09T13:48:05.728 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:05.723+0000 7f77f55e1640 1 -- 172.21.15.43:0/1621775052 wait complete. 2023-08-09T13:48:05.751 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config rm mgr mgr/snap_schedule/dump_on_update 2023-08-09T13:48:05.970 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:05.967+0000 7f56fc857640 1 Processor -- start 2023-08-09T13:48:05.970 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:05.968+0000 7f56fc857640 1 -- start start 2023-08-09T13:48:05.972 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:05.969+0000 7f56fc857640 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7f56f815c550 0x7f56f80cd6d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:48:05.972 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:05.969+0000 7f56fc857640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f56f80d04b0 0x7f56f80b3ee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:48:05.972 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:05.969+0000 7f56fc857640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f56f80b4420 0x7f56f806a760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:48:05.972 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:05.969+0000 7f56fc857640 1 -- --> v2:172.21.15.43:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f56f80b4870 con 0x7f56f80d04b0 2023-08-09T13:48:05.972 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:05.969+0000 7f56fc857640 1 -- --> v2:172.21.15.203:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f56f80b7bd0 con 0x7f56f815c550 2023-08-09T13:48:05.973 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:05.969+0000 7f56fc857640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f56f80d1c00 con 0x7f56f80b4420 2023-08-09T13:48:05.973 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:05.970+0000 7f56effff640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f56f80d04b0 0x7f56f80b3ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:48:05.974 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:05.970+0000 7f56effff640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f56f80d04b0 0x7f56f80b3ee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:35024/0 (socket says 172.21.15.43:35024) 2023-08-09T13:48:05.974 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:05.970+0000 7f56effff640 1 -- 172.21.15.43:0/133743549 learned_addr learned my addr 172.21.15.43:0/133743549 (peer_addr_for_me v2:172.21.15.43:0/0) 2023-08-09T13:48:05.974 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:05.970+0000 7f56f659c640 1 --2- 172.21.15.43:0/133743549 >> v2:172.21.15.203:3300/0 conn(0x7f56f815c550 0x7f56f80cd6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:48:05.974 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:05.970+0000 7f56f6d9d640 1 --2- 172.21.15.43:0/133743549 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f56f80b4420 0x7f56f806a760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:48:05.974 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:05.970+0000 7f56effff640 1 -- 172.21.15.43:0/133743549 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f56f80b4420 msgr2=0x7f56f806a760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-09T13:48:05.974 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:05.970+0000 7f56effff640 1 --2- 172.21.15.43:0/133743549 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f56f80b4420 0x7f56f806a760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:48:05.974 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:05.970+0000 7f56effff640 1 -- 172.21.15.43:0/133743549 >> v2:172.21.15.203:3300/0 conn(0x7f56f815c550 msgr2=0x7f56f80cd6d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-09T13:48:05.975 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:05.970+0000 7f56effff640 1 --2- 172.21.15.43:0/133743549 >> v2:172.21.15.203:3300/0 conn(0x7f56f815c550 0x7f56f80cd6d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:48:05.975 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:05.970+0000 7f56effff640 1 -- 172.21.15.43:0/133743549 --> v2:172.21.15.43:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f56f815c930 con 0x7f56f80d04b0 2023-08-09T13:48:05.976 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:05.972+0000 7f56effff640 1 --2- 172.21.15.43:0/133743549 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f56f80d04b0 0x7f56f80b3ee0 secure :-1 s=READY pgs=794 cs=0 l=1 rev1=1 crypto rx=0x7f56f80d01c0 tx=0x7f56e002b880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c37f7d003fd47b05 server_cookie=0 in_seq=0 out_seq=0 2023-08-09T13:48:05.976 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:05.972+0000 7f56f5d9b640 1 -- 172.21.15.43:0/133743549 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f56e00317e0 con 0x7f56f80d04b0 2023-08-09T13:48:05.976 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:05.972+0000 7f56f5d9b640 1 -- 172.21.15.43:0/133743549 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f56e0031940 con 0x7f56f80d04b0 2023-08-09T13:48:05.976 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:05.972+0000 7f56f5d9b640 1 -- 172.21.15.43:0/133743549 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f56e0031cd0 con 0x7f56f80d04b0 2023-08-09T13:48:05.976 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:05.973+0000 7f56fc857640 1 -- 172.21.15.43:0/133743549 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f56f80d04b0 msgr2=0x7f56f80b3ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:48:05.976 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:05.973+0000 7f56fc857640 1 --2- 172.21.15.43:0/133743549 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f56f80d04b0 0x7f56f80b3ee0 secure :-1 s=READY pgs=794 cs=0 l=1 rev1=1 crypto rx=0x7f56f80d01c0 tx=0x7f56e002b880 comp rx=0 tx=0).stop 2023-08-09T13:48:05.977 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:05.973+0000 7f56fc857640 1 -- 172.21.15.43:0/133743549 shutdown_connections 2023-08-09T13:48:05.977 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:05.973+0000 7f56fc857640 1 --2- 172.21.15.43:0/133743549 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f56f80d04b0 0x7f56f80b3ee0 unknown :-1 s=CLOSED pgs=794 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:48:05.977 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:05.973+0000 7f56fc857640 1 --2- 172.21.15.43:0/133743549 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f56f80b4420 0x7f56f806a760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:48:05.977 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:05.973+0000 7f56fc857640 1 --2- 172.21.15.43:0/133743549 >> v2:172.21.15.203:3300/0 conn(0x7f56f815c550 0x7f56f80cd6d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:48:05.977 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:05.973+0000 7f56fc857640 1 -- 172.21.15.43:0/133743549 >> 172.21.15.43:0/133743549 conn(0x7f56f8071520 msgr2=0x7f56f80ceee0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-09T13:48:05.977 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:05.973+0000 7f56fc857640 1 -- 172.21.15.43:0/133743549 shutdown_connections 2023-08-09T13:48:05.977 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:05.973+0000 7f56fc857640 1 -- 172.21.15.43:0/133743549 wait complete. 2023-08-09T13:48:05.978 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:05.975+0000 7f56fc857640 1 Processor -- start 2023-08-09T13:48:05.978 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:05.976+0000 7f56fc857640 1 -- start start 2023-08-09T13:48:05.980 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:05.977+0000 7f56fc857640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f56f80b4420 0x7f56f80aa740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:48:05.980 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:05.977+0000 7f56fc857640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f56f80d04b0 0x7f56f80a8dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:48:05.980 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:05.977+0000 7f56fc857640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f56f815c550 0x7f56f80c9530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:48:05.980 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:05.977+0000 7f56fc857640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f56f80aa320 con 0x7f56f815c550 2023-08-09T13:48:05.980 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:05.977+0000 7f56fc857640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f56f80a96a0 con 0x7f56f80d04b0 2023-08-09T13:48:05.980 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:05.977+0000 7f56fc857640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f56f80a9980 con 0x7f56f80b4420 2023-08-09T13:48:05.981 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:05.977+0000 7f56f6d9d640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f56f815c550 0x7f56f80c9530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:48:05.981 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:05.977+0000 7f56f6d9d640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f56f815c550 0x7f56f80c9530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:35036/0 (socket says 172.21.15.43:35036) 2023-08-09T13:48:05.981 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:05.977+0000 7f56f6d9d640 1 -- 172.21.15.43:0/1932778849 learned_addr learned my addr 172.21.15.43:0/1932778849 (peer_addr_for_me v2:172.21.15.43:0/0) 2023-08-09T13:48:05.982 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:05.978+0000 7f56effff640 1 --2- 172.21.15.43:0/1932778849 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f56f80d04b0 0x7f56f80a8dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:48:05.982 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:05.978+0000 7f56f659c640 1 --2- 172.21.15.43:0/1932778849 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f56f80b4420 0x7f56f80aa740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:48:05.982 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:05.978+0000 7f56f6d9d640 1 -- 172.21.15.43:0/1932778849 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f56f80b4420 msgr2=0x7f56f80aa740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:48:05.982 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:05.978+0000 7f56f6d9d640 1 --2- 172.21.15.43:0/1932778849 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f56f80b4420 0x7f56f80aa740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:48:05.983 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:05.978+0000 7f56f6d9d640 1 -- 172.21.15.43:0/1932778849 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f56f80d04b0 msgr2=0x7f56f80a8dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:48:05.983 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:05.978+0000 7f56f6d9d640 1 --2- 172.21.15.43:0/1932778849 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f56f80d04b0 0x7f56f80a8dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:48:05.983 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:05.978+0000 7f56f6d9d640 1 -- 172.21.15.43:0/1932778849 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f56e00094d0 con 0x7f56f815c550 2023-08-09T13:48:05.983 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:05.978+0000 7f56f6d9d640 1 --2- 172.21.15.43:0/1932778849 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f56f815c550 0x7f56f80c9530 secure :-1 s=READY pgs=795 cs=0 l=1 rev1=1 crypto rx=0x7f56f0007e40 tx=0x7f56f000e4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-09T13:48:05.983 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:05.979+0000 7f56eeffd640 1 -- 172.21.15.43:0/1932778849 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f56f0014620 con 0x7f56f815c550 2023-08-09T13:48:05.983 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:05.979+0000 7f56eeffd640 1 -- 172.21.15.43:0/1932778849 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f56f000ee90 con 0x7f56f815c550 2023-08-09T13:48:05.983 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:05.979+0000 7f56eeffd640 1 -- 172.21.15.43:0/1932778849 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f56f00149e0 con 0x7f56f815c550 2023-08-09T13:48:05.984 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:05.979+0000 7f56fc857640 1 -- 172.21.15.43:0/1932778849 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f56f80a9c60 con 0x7f56f815c550 2023-08-09T13:48:05.984 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:05.979+0000 7f56fc857640 1 -- 172.21.15.43:0/1932778849 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f56f80aac80 con 0x7f56f815c550 2023-08-09T13:48:05.985 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:05.980+0000 7f56fc857640 1 -- 172.21.15.43:0/1932778849 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f56f80aa460 con 0x7f56f815c550 2023-08-09T13:48:05.985 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:05.981+0000 7f56eeffd640 1 -- 172.21.15.43:0/1932778849 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 18) v1 ==== 88443+0+0 (secure 0 0 0) 0x7f56f0014b40 con 0x7f56f815c550 2023-08-09T13:48:05.985 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:05.982+0000 7f56eeffd640 1 --2- 172.21.15.43:0/1932778849 >> [v2:172.21.15.203:6832/272917756,v1:172.21.15.203:6833/272917756] conn(0x7f56d806a170 0x7f56d806c630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:48:05.985 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:05.983+0000 7f56eeffd640 1 -- 172.21.15.43:0/1932778849 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(66..66 src has 1..66) v4 ==== 6305+0+0 (secure 0 0 0) 0x7f56f0089210 con 0x7f56f815c550 2023-08-09T13:48:05.986 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:05.983+0000 7f56f659c640 1 --2- 172.21.15.43:0/1932778849 >> [v2:172.21.15.203:6832/272917756,v1:172.21.15.203:6833/272917756] conn(0x7f56d806a170 0x7f56d806c630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:48:05.987 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:05.984+0000 7f56f659c640 1 --2- 172.21.15.43:0/1932778849 >> [v2:172.21.15.203:6832/272917756,v1:172.21.15.203:6833/272917756] conn(0x7f56d806a170 0x7f56d806c630 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7f56e8006fd0 tx=0x7f56e8008040 comp rx=0 tx=0).ready entity=mgr.5533 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-09T13:48:05.990 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:05.987+0000 7f56eeffd640 1 -- 172.21.15.43:0/1932778849 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178066 (secure 0 0 0) 0x7f56f80aa460 con 0x7f56f815c550 2023-08-09T13:48:06.188 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:06.186+0000 7f56fc857640 1 -- 172.21.15.43:0/1932778849 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "config rm", "who": "mgr", "name": "mgr/snap_schedule/dump_on_update"} v 0) v1 -- 0x7f56f80aa460 con 0x7f56f815c550 2023-08-09T13:48:06.195 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:06.192+0000 7f56eeffd640 1 -- 172.21.15.43:0/1932778849 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "config rm", "who": "mgr", "name": "mgr/snap_schedule/dump_on_update"}]=0 v13) v1 ==== 115+0+0 (secure 0 0 0) 0x7f56f80aa460 con 0x7f56f815c550 2023-08-09T13:48:06.198 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:06.195+0000 7f56fc857640 1 -- 172.21.15.43:0/1932778849 >> [v2:172.21.15.203:6832/272917756,v1:172.21.15.203:6833/272917756] conn(0x7f56d806a170 msgr2=0x7f56d806c630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:48:06.198 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:06.195+0000 7f56fc857640 1 --2- 172.21.15.43:0/1932778849 >> [v2:172.21.15.203:6832/272917756,v1:172.21.15.203:6833/272917756] conn(0x7f56d806a170 0x7f56d806c630 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7f56e8006fd0 tx=0x7f56e8008040 comp rx=0 tx=0).stop 2023-08-09T13:48:06.198 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:06.195+0000 7f56fc857640 1 -- 172.21.15.43:0/1932778849 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f56f815c550 msgr2=0x7f56f80c9530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:48:06.198 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:06.195+0000 7f56fc857640 1 --2- 172.21.15.43:0/1932778849 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f56f815c550 0x7f56f80c9530 secure :-1 s=READY pgs=795 cs=0 l=1 rev1=1 crypto rx=0x7f56f0007e40 tx=0x7f56f000e4b0 comp rx=0 tx=0).stop 2023-08-09T13:48:06.199 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:06.195+0000 7f56fc857640 1 -- 172.21.15.43:0/1932778849 shutdown_connections 2023-08-09T13:48:06.199 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:06.195+0000 7f56fc857640 1 --2- 172.21.15.43:0/1932778849 >> [v2:172.21.15.203:6832/272917756,v1:172.21.15.203:6833/272917756] conn(0x7f56d806a170 0x7f56d806c630 unknown :-1 s=CLOSED pgs=36 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:48:06.199 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:06.195+0000 7f56fc857640 1 --2- 172.21.15.43:0/1932778849 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f56f815c550 0x7f56f80c9530 unknown :-1 s=CLOSED pgs=795 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:48:06.199 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:06.195+0000 7f56fc857640 1 --2- 172.21.15.43:0/1932778849 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f56f80d04b0 0x7f56f80a8dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:48:06.199 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:06.195+0000 7f56fc857640 1 --2- 172.21.15.43:0/1932778849 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f56f80b4420 0x7f56f80aa740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:48:06.199 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:06.195+0000 7f56fc857640 1 -- 172.21.15.43:0/1932778849 >> 172.21.15.43:0/1932778849 conn(0x7f56f8071520 msgr2=0x7f56f815ee90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-09T13:48:06.199 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:06.196+0000 7f56fc857640 1 -- 172.21.15.43:0/1932778849 shutdown_connections 2023-08-09T13:48:06.200 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:06.196+0000 7f56fc857640 1 -- 172.21.15.43:0/1932778849 wait complete. 2023-08-09T13:48:06.216 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config rm mgr mgr/snap_schedule/allow_m_granularity 2023-08-09T13:48:06.434 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:06.431+0000 7fa367153640 1 Processor -- start 2023-08-09T13:48:06.434 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:06.431+0000 7fa367153640 1 -- start start 2023-08-09T13:48:06.436 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:06.433+0000 7fa367153640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fa360070ef0 0x7fa3600ab8d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:48:06.436 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:06.433+0000 7fa367153640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7fa3600cc0b0 0x7fa36006f310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:48:06.436 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:06.433+0000 7fa367153640 1 -- --> v1:172.21.15.43:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa3600acfa0 con 0x7fa3600abe10 2023-08-09T13:48:06.436 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:06.433+0000 7fa367153640 1 -- --> v2:172.21.15.43:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa36015c940 con 0x7fa3600cc0b0 2023-08-09T13:48:06.436 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:06.433+0000 7fa367153640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa3600ac390 con 0x7fa360070ef0 2023-08-09T13:48:06.437 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:06.434+0000 7fa366952640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7fa3600cc0b0 0x7fa36006f310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:48:06.437 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:06.434+0000 7fa366952640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7fa3600cc0b0 0x7fa36006f310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:35050/0 (socket says 172.21.15.43:35050) 2023-08-09T13:48:06.437 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:06.434+0000 7fa366952640 1 -- 172.21.15.43:0/2161102614 learned_addr learned my addr 172.21.15.43:0/2161102614 (peer_addr_for_me v2:172.21.15.43:0/0) 2023-08-09T13:48:06.438 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:06.434+0000 7fa366151640 1 --2- 172.21.15.43:0/2161102614 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fa360070ef0 0x7fa3600ab8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:48:06.438 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:06.434+0000 7fa366952640 1 -- 172.21.15.43:0/2161102614 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fa360070ef0 msgr2=0x7fa3600ab8d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-09T13:48:06.438 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:06.434+0000 7fa366952640 1 --2- 172.21.15.43:0/2161102614 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fa360070ef0 0x7fa3600ab8d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:48:06.438 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:06.434+0000 7fa366952640 1 -- 172.21.15.43:0/2161102614 >> v1:172.21.15.43:6789/0 conn(0x7fa3600abe10 legacy=0x7fa3600cb9a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:48:06.438 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:06.434+0000 7fa366952640 1 -- 172.21.15.43:0/2161102614 --> v2:172.21.15.43:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa3600ac670 con 0x7fa3600cc0b0 2023-08-09T13:48:06.438 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:06.434+0000 7fa366151640 1 --2- 172.21.15.43:0/2161102614 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fa360070ef0 0x7fa3600ab8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-08-09T13:48:06.439 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:06.435+0000 7fa366952640 1 --2- 172.21.15.43:0/2161102614 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa3600cc0b0 0x7fa36006f310 secure :-1 s=READY pgs=797 cs=0 l=1 rev1=1 crypto rx=0x7fa358009e50 tx=0x7fa35802bff0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9eae41c10a0774bf server_cookie=0 in_seq=0 out_seq=0 2023-08-09T13:48:06.439 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:06.436+0000 7fa36514f640 1 -- 172.21.15.43:0/2161102614 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa35802cb90 con 0x7fa3600cc0b0 2023-08-09T13:48:06.439 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:06.436+0000 7fa36514f640 1 -- 172.21.15.43:0/2161102614 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa35802ccf0 con 0x7fa3600cc0b0 2023-08-09T13:48:06.440 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:06.436+0000 7fa36514f640 1 -- 172.21.15.43:0/2161102614 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa358034610 con 0x7fa3600cc0b0 2023-08-09T13:48:06.440 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:06.436+0000 7fa367153640 1 -- 172.21.15.43:0/2161102614 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa3600cc0b0 msgr2=0x7fa36006f310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:48:06.440 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:06.436+0000 7fa367153640 1 --2- 172.21.15.43:0/2161102614 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa3600cc0b0 0x7fa36006f310 secure :-1 s=READY pgs=797 cs=0 l=1 rev1=1 crypto rx=0x7fa358009e50 tx=0x7fa35802bff0 comp rx=0 tx=0).stop 2023-08-09T13:48:06.440 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:06.436+0000 7fa367153640 1 -- 172.21.15.43:0/2161102614 shutdown_connections 2023-08-09T13:48:06.440 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:06.436+0000 7fa367153640 1 --2- 172.21.15.43:0/2161102614 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa3600cc0b0 0x7fa36006f310 unknown :-1 s=CLOSED pgs=797 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:48:06.440 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:06.436+0000 7fa367153640 1 --2- 172.21.15.43:0/2161102614 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fa360070ef0 0x7fa3600ab8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:48:06.441 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:06.436+0000 7fa367153640 1 -- 172.21.15.43:0/2161102614 >> 172.21.15.43:0/2161102614 conn(0x7fa3600cddd0 msgr2=0x7fa3600ce1e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-09T13:48:06.441 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:06.436+0000 7fa367153640 1 -- 172.21.15.43:0/2161102614 shutdown_connections 2023-08-09T13:48:06.441 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:06.436+0000 7fa367153640 1 -- 172.21.15.43:0/2161102614 wait complete. 2023-08-09T13:48:06.442 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:06.438+0000 7fa367153640 1 Processor -- start 2023-08-09T13:48:06.442 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:06.438+0000 7fa367153640 1 -- start start 2023-08-09T13:48:06.442 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:06.439+0000 7fa367153640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa360070ef0 0x7fa360143cb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:48:06.442 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:06.439+0000 7fa367153640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fa3600abe10 0x7fa36015cb10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:48:06.442 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:06.439+0000 7fa367153640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fa3600cc0b0 0x7fa360158950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:48:06.442 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:06.439+0000 7fa367153640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa36015dfe0 con 0x7fa360070ef0 2023-08-09T13:48:06.443 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:06.439+0000 7fa367153640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa36015d360 con 0x7fa3600cc0b0 2023-08-09T13:48:06.443 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:06.439+0000 7fa367153640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa36015d640 con 0x7fa3600abe10 2023-08-09T13:48:06.444 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:06.440+0000 7fa366151640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa360070ef0 0x7fa360143cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:48:06.444 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:06.440+0000 7fa366151640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa360070ef0 0x7fa360143cb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:35056/0 (socket says 172.21.15.43:35056) 2023-08-09T13:48:06.445 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:06.440+0000 7fa366151640 1 -- 172.21.15.43:0/4222502984 learned_addr learned my addr 172.21.15.43:0/4222502984 (peer_addr_for_me v2:172.21.15.43:0/0) 2023-08-09T13:48:06.445 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:06.440+0000 7fa366151640 1 -- 172.21.15.43:0/4222502984 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fa3600abe10 msgr2=0x7fa36015cb10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:48:06.445 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:06.440+0000 7fa366151640 1 --2- 172.21.15.43:0/4222502984 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fa3600abe10 0x7fa36015cb10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:48:06.445 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:06.440+0000 7fa366151640 1 -- 172.21.15.43:0/4222502984 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fa3600cc0b0 msgr2=0x7fa360158950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:48:06.445 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:06.440+0000 7fa366151640 1 --2- 172.21.15.43:0/4222502984 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fa3600cc0b0 0x7fa360158950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:48:06.445 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:06.440+0000 7fa366151640 1 -- 172.21.15.43:0/4222502984 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa358009b30 con 0x7fa360070ef0 2023-08-09T13:48:06.445 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:06.441+0000 7fa366151640 1 --2- 172.21.15.43:0/4222502984 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa360070ef0 0x7fa360143cb0 secure :-1 s=READY pgs=798 cs=0 l=1 rev1=1 crypto rx=0x7fa35c00c7f0 tx=0x7fa35c00ccb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-09T13:48:06.446 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:06.441+0000 7fa356ffd640 1 -- 172.21.15.43:0/4222502984 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa35c007820 con 0x7fa360070ef0 2023-08-09T13:48:06.446 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:06.441+0000 7fa356ffd640 1 -- 172.21.15.43:0/4222502984 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa35c007980 con 0x7fa360070ef0 2023-08-09T13:48:06.446 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:06.441+0000 7fa356ffd640 1 -- 172.21.15.43:0/4222502984 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa35c007cf0 con 0x7fa360070ef0 2023-08-09T13:48:06.446 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:06.441+0000 7fa367153640 1 -- 172.21.15.43:0/4222502984 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa36015d920 con 0x7fa360070ef0 2023-08-09T13:48:06.446 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:06.441+0000 7fa367153640 1 -- 172.21.15.43:0/4222502984 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa36015d050 con 0x7fa360070ef0 2023-08-09T13:48:06.447 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:06.442+0000 7fa367153640 1 -- 172.21.15.43:0/4222502984 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa36004e390 con 0x7fa360070ef0 2023-08-09T13:48:06.447 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:06.443+0000 7fa356ffd640 1 -- 172.21.15.43:0/4222502984 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 18) v1 ==== 88443+0+0 (secure 0 0 0) 0x7fa35c0113f0 con 0x7fa360070ef0 2023-08-09T13:48:06.447 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:06.444+0000 7fa356ffd640 1 --2- 172.21.15.43:0/4222502984 >> [v2:172.21.15.203:6832/272917756,v1:172.21.15.203:6833/272917756] conn(0x7fa34406a170 0x7fa34406c630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:48:06.447 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:06.444+0000 7fa356ffd640 1 -- 172.21.15.43:0/4222502984 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(66..66 src has 1..66) v4 ==== 6305+0+0 (secure 0 0 0) 0x7fa35c088cc0 con 0x7fa360070ef0 2023-08-09T13:48:06.448 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:06.445+0000 7fa365950640 1 --2- 172.21.15.43:0/4222502984 >> [v2:172.21.15.203:6832/272917756,v1:172.21.15.203:6833/272917756] conn(0x7fa34406a170 0x7fa34406c630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:48:06.448 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:06.446+0000 7fa365950640 1 --2- 172.21.15.43:0/4222502984 >> [v2:172.21.15.203:6832/272917756,v1:172.21.15.203:6833/272917756] conn(0x7fa34406a170 0x7fa34406c630 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7fa350000f60 tx=0x7fa3500079d0 comp rx=0 tx=0).ready entity=mgr.5533 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-09T13:48:06.451 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:06.448+0000 7fa356ffd640 1 -- 172.21.15.43:0/4222502984 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178066 (secure 0 0 0) 0x7fa35c08f080 con 0x7fa360070ef0 2023-08-09T13:48:06.648 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:06.645+0000 7fa367153640 1 -- 172.21.15.43:0/4222502984 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "config rm", "who": "mgr", "name": "mgr/snap_schedule/allow_m_granularity"} v 0) v1 -- 0x7fa3600acc00 con 0x7fa360070ef0 2023-08-09T13:48:06.656 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:06.654+0000 7fa356ffd640 1 -- 172.21.15.43:0/4222502984 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "config rm", "who": "mgr", "name": "mgr/snap_schedule/allow_m_granularity"}]=0 v14) v1 ==== 120+0+0 (secure 0 0 0) 0x7fa35c054080 con 0x7fa360070ef0 2023-08-09T13:48:06.660 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:06.657+0000 7fa367153640 1 -- 172.21.15.43:0/4222502984 >> [v2:172.21.15.203:6832/272917756,v1:172.21.15.203:6833/272917756] conn(0x7fa34406a170 msgr2=0x7fa34406c630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:48:06.660 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:06.657+0000 7fa367153640 1 --2- 172.21.15.43:0/4222502984 >> [v2:172.21.15.203:6832/272917756,v1:172.21.15.203:6833/272917756] conn(0x7fa34406a170 0x7fa34406c630 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7fa350000f60 tx=0x7fa3500079d0 comp rx=0 tx=0).stop 2023-08-09T13:48:06.660 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:06.657+0000 7fa367153640 1 -- 172.21.15.43:0/4222502984 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa360070ef0 msgr2=0x7fa360143cb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:48:06.660 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:06.657+0000 7fa367153640 1 --2- 172.21.15.43:0/4222502984 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa360070ef0 0x7fa360143cb0 secure :-1 s=READY pgs=798 cs=0 l=1 rev1=1 crypto rx=0x7fa35c00c7f0 tx=0x7fa35c00ccb0 comp rx=0 tx=0).stop 2023-08-09T13:48:06.660 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:06.657+0000 7fa367153640 1 -- 172.21.15.43:0/4222502984 shutdown_connections 2023-08-09T13:48:06.660 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:06.657+0000 7fa367153640 1 --2- 172.21.15.43:0/4222502984 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fa3600cc0b0 0x7fa360158950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:48:06.661 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:06.657+0000 7fa367153640 1 --2- 172.21.15.43:0/4222502984 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fa3600abe10 0x7fa36015cb10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:48:06.661 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:06.657+0000 7fa367153640 1 --2- 172.21.15.43:0/4222502984 >> [v2:172.21.15.203:6832/272917756,v1:172.21.15.203:6833/272917756] conn(0x7fa34406a170 0x7fa34406c630 unknown :-1 s=CLOSED pgs=37 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:48:06.661 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:06.657+0000 7fa367153640 1 --2- 172.21.15.43:0/4222502984 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa360070ef0 0x7fa360143cb0 unknown :-1 s=CLOSED pgs=798 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:48:06.661 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:06.657+0000 7fa367153640 1 -- 172.21.15.43:0/4222502984 >> 172.21.15.43:0/4222502984 conn(0x7fa3600cddd0 msgr2=0x7fa36015bdd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-09T13:48:06.661 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:06.657+0000 7fa367153640 1 -- 172.21.15.43:0/4222502984 shutdown_connections 2023-08-09T13:48:06.661 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:06.657+0000 7fa367153640 1 -- 172.21.15.43:0/4222502984 wait complete. 2023-08-09T13:48:06.678 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config rm mon mon_allow_pool_delete 2023-08-09T13:48:06.892 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:06.889+0000 7fbacc0cc640 1 Processor -- start 2023-08-09T13:48:06.892 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:06.889+0000 7fbacc0cc640 1 -- start start 2023-08-09T13:48:06.893 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:06.891+0000 7fbacc0cc640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fbac415c7e0 0x7fbac40ccd10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:48:06.894 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:06.891+0000 7fbacc0cc640 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7fbac40b45d0 0x7fbac40d0990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:48:06.894 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:06.891+0000 7fbacc0cc640 1 -- --> v1:172.21.15.43:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbac40cd3a0 con 0x7fbac4160dd0 2023-08-09T13:48:06.894 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:06.891+0000 7fbacc0cc640 1 -- --> v2:172.21.15.203:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbac40cdf20 con 0x7fbac40b45d0 2023-08-09T13:48:06.894 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:06.891+0000 7fbacc0cc640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbac40d0540 con 0x7fbac415c7e0 2023-08-09T13:48:06.896 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:06.891+0000 7fbac9667640 1 --1- >> v1:172.21.15.43:6789/0 conn(0x7fbac4160dd0 0x7fbac40b3ec0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.43:6789/0 says I am v1:172.21.15.43:41652/0 (socket says 172.21.15.43:41652) 2023-08-09T13:48:06.896 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:06.891+0000 7fbac9667640 1 -- 172.21.15.43:0/235439754 learned_addr learned my addr 172.21.15.43:0/235439754 (peer_addr_for_me v1:172.21.15.43:0/0) 2023-08-09T13:48:06.896 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:06.892+0000 7fbaca669640 1 --2- 172.21.15.43:0/235439754 >> v2:172.21.15.203:3300/0 conn(0x7fbac40b45d0 0x7fbac40d0990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:48:06.896 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:06.892+0000 7fbac8e66640 1 -- 172.21.15.43:0/235439754 <== mon.0 v1:172.21.15.43:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2069012059 0 0) 0x7fbac40cd3a0 con 0x7fbac4160dd0 2023-08-09T13:48:06.896 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:06.892+0000 7fbac9e68640 1 --2- 172.21.15.43:0/235439754 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fbac415c7e0 0x7fbac40ccd10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:48:06.896 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:06.892+0000 7fbac8e66640 1 -- 172.21.15.43:0/235439754 --> v1:172.21.15.43:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fbab0003520 con 0x7fbac4160dd0 2023-08-09T13:48:06.897 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:06.892+0000 7fbac8e66640 1 -- 172.21.15.43:0/235439754 <== mon.0 v1:172.21.15.43:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3652356675 0 0) 0x7fbab0003520 con 0x7fbac4160dd0 2023-08-09T13:48:06.897 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:06.892+0000 7fbac8e66640 1 -- 172.21.15.43:0/235439754 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fbac415c7e0 msgr2=0x7fbac40ccd10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-09T13:48:06.897 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:06.892+0000 7fbac8e66640 1 --2- 172.21.15.43:0/235439754 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fbac415c7e0 0x7fbac40ccd10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:48:06.897 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:06.892+0000 7fbac8e66640 1 -- 172.21.15.43:0/235439754 >> v2:172.21.15.203:3300/0 conn(0x7fbac40b45d0 msgr2=0x7fbac40d0990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-09T13:48:06.897 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:06.892+0000 7fbac8e66640 1 --2- 172.21.15.43:0/235439754 >> v2:172.21.15.203:3300/0 conn(0x7fbac40b45d0 0x7fbac40d0990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:48:06.897 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:06.892+0000 7fbac8e66640 1 -- 172.21.15.43:0/235439754 --> v1:172.21.15.43:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbac40cd680 con 0x7fbac4160dd0 2023-08-09T13:48:06.897 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:06.892+0000 7fbac8e66640 1 -- 172.21.15.43:0/235439754 <== mon.0 v1:172.21.15.43:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1285150740 0 0) 0x7fbab4001670 con 0x7fbac4160dd0 2023-08-09T13:48:06.897 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:06.892+0000 7fbaca669640 1 --2- 172.21.15.43:0/235439754 >> v2:172.21.15.203:3300/0 conn(0x7fbac40b45d0 0x7fbac40d0990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-08-09T13:48:06.898 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:06.892+0000 7fbac8e66640 1 -- 172.21.15.43:0/235439754 >> v1:172.21.15.43:6789/0 conn(0x7fbac4160dd0 legacy=0x7fbac40b3ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:48:06.898 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:06.892+0000 7fbac8e66640 1 --2- 172.21.15.43:0/235439754 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fbac415c7e0 0x7fbab0003970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:48:06.898 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:06.893+0000 7fbac8e66640 1 --2- 172.21.15.43:0/235439754 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fbab0003fa0 0x7fbab00047c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:48:06.898 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:06.893+0000 7fbac8e66640 1 --2- 172.21.15.43:0/235439754 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fbab0004d00 0x7fbab0005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:48:06.898 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:06.893+0000 7fbac8e66640 1 -- 172.21.15.43:0/235439754 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbac40cdf20 con 0x7fbab0003fa0 2023-08-09T13:48:06.898 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:06.893+0000 7fbac8e66640 1 -- 172.21.15.43:0/235439754 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbac40d0540 con 0x7fbab0004d00 2023-08-09T13:48:06.898 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:06.893+0000 7fbac8e66640 1 -- 172.21.15.43:0/235439754 --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbab000ba80 con 0x7fbac415c7e0 2023-08-09T13:48:06.899 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:06.893+0000 7fbac9667640 1 --2- 172.21.15.43:0/235439754 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fbab0003fa0 0x7fbab00047c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:48:06.899 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:06.893+0000 7fbac9e68640 1 --2- 172.21.15.43:0/235439754 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fbac415c7e0 0x7fbab0003970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:48:06.899 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:06.893+0000 7fbaca669640 1 --2- 172.21.15.43:0/235439754 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fbab0004d00 0x7fbab0005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:48:06.899 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:06.893+0000 7fbac9667640 1 -- 172.21.15.43:0/235439754 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fbac415c7e0 msgr2=0x7fbab0003970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-09T13:48:06.899 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:06.893+0000 7fbac9667640 1 --2- 172.21.15.43:0/235439754 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fbac415c7e0 0x7fbab0003970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:48:06.899 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:06.893+0000 7fbac9667640 1 -- 172.21.15.43:0/235439754 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fbab0004d00 msgr2=0x7fbab0005180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-09T13:48:06.899 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:06.893+0000 7fbac9667640 1 --2- 172.21.15.43:0/235439754 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fbab0004d00 0x7fbab0005180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:48:06.899 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:06.893+0000 7fbac9667640 1 -- 172.21.15.43:0/235439754 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbab0003520 con 0x7fbab0003fa0 2023-08-09T13:48:06.901 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:06.895+0000 7fbac9667640 1 --2- 172.21.15.43:0/235439754 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fbab0003fa0 0x7fbab00047c0 secure :-1 s=READY pgs=800 cs=0 l=1 rev1=1 crypto rx=0x7fbab400c240 tx=0x7fbab00057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6b745b1f9a452c12 server_cookie=0 in_seq=0 out_seq=0 2023-08-09T13:48:06.901 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:06.895+0000 7fbac8e66640 1 -- 172.21.15.43:0/235439754 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbab402cb00 con 0x7fbab0003fa0 2023-08-09T13:48:06.901 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:06.895+0000 7fbac8e66640 1 -- 172.21.15.43:0/235439754 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbab402cc60 con 0x7fbab0003fa0 2023-08-09T13:48:06.901 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:06.895+0000 7fbac8e66640 1 -- 172.21.15.43:0/235439754 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbab4005b10 con 0x7fbab0003fa0 2023-08-09T13:48:06.901 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:06.895+0000 7fbacc0cc640 1 -- 172.21.15.43:0/235439754 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fbab0003fa0 msgr2=0x7fbab00047c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:48:06.901 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:06.895+0000 7fbacc0cc640 1 --2- 172.21.15.43:0/235439754 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fbab0003fa0 0x7fbab00047c0 secure :-1 s=READY pgs=800 cs=0 l=1 rev1=1 crypto rx=0x7fbab400c240 tx=0x7fbab00057f0 comp rx=0 tx=0).stop 2023-08-09T13:48:06.901 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:06.895+0000 7fbaca669640 1 -- 172.21.15.43:0/235439754 reap_dead start 2023-08-09T13:48:06.902 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:06.896+0000 7fbacc0cc640 1 -- 172.21.15.43:0/235439754 shutdown_connections 2023-08-09T13:48:06.902 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:06.896+0000 7fbacc0cc640 1 -- 172.21.15.43:0/235439754 >> 172.21.15.43:0/235439754 conn(0x7fbac40ceec0 msgr2=0x7fbac40cf2d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-09T13:48:06.902 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:06.896+0000 7fbacc0cc640 1 -- 172.21.15.43:0/235439754 shutdown_connections 2023-08-09T13:48:06.902 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:06.896+0000 7fbacc0cc640 1 -- 172.21.15.43:0/235439754 wait complete. 2023-08-09T13:48:06.902 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:06.897+0000 7fbacc0cc640 1 Processor -- start 2023-08-09T13:48:06.902 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:06.898+0000 7fbacc0cc640 1 -- start start 2023-08-09T13:48:06.903 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:06.899+0000 7fbacc0cc640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fbab0003fa0 0x7fbac40c5f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:48:06.904 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:06.899+0000 7fbacc0cc640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fbac40b34f0 0x7fbac41488b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:48:06.904 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:06.899+0000 7fbacc0cc640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fbac415a2e0 0x7fbac415a760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:48:06.904 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:06.899+0000 7fbacc0cc640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbac415aca0 con 0x7fbac40b34f0 2023-08-09T13:48:06.904 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:06.899+0000 7fbacc0cc640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbac40c67c0 con 0x7fbac415a2e0 2023-08-09T13:48:06.904 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:06.899+0000 7fbacc0cc640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbac40c6aa0 con 0x7fbab0003fa0 2023-08-09T13:48:06.904 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:06.899+0000 7fbac9667640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fbac40b34f0 0x7fbac41488b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:48:06.905 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:06.899+0000 7fbac9667640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fbac40b34f0 0x7fbac41488b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:35076/0 (socket says 172.21.15.43:35076) 2023-08-09T13:48:06.905 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:06.899+0000 7fbac9667640 1 -- 172.21.15.43:0/1910083871 learned_addr learned my addr 172.21.15.43:0/1910083871 (peer_addr_for_me v2:172.21.15.43:0/0) 2023-08-09T13:48:06.905 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:06.899+0000 7fbac9667640 1 -- 172.21.15.43:0/1910083871 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fbab0003fa0 msgr2=0x7fbac40c5f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:48:06.905 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:06.899+0000 7fbac9667640 1 --2- 172.21.15.43:0/1910083871 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fbab0003fa0 0x7fbac40c5f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:48:06.905 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:06.899+0000 7fbac9667640 1 -- 172.21.15.43:0/1910083871 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fbac415a2e0 msgr2=0x7fbac415a760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:48:06.905 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:06.899+0000 7fbaca669640 1 --2- 172.21.15.43:0/1910083871 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fbac415a2e0 0x7fbac415a760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:48:06.905 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:06.900+0000 7fbac9667640 1 --2- 172.21.15.43:0/1910083871 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fbac415a2e0 0x7fbac415a760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:48:06.905 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:06.900+0000 7fbac9667640 1 -- 172.21.15.43:0/1910083871 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbac40c6d80 con 0x7fbac40b34f0 2023-08-09T13:48:06.906 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:06.900+0000 7fbaca669640 1 --2- 172.21.15.43:0/1910083871 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fbac415a2e0 0x7fbac415a760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-08-09T13:48:06.906 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:06.900+0000 7fbac9667640 1 --2- 172.21.15.43:0/1910083871 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fbac40b34f0 0x7fbac41488b0 secure :-1 s=READY pgs=801 cs=0 l=1 rev1=1 crypto rx=0x7fbab400c240 tx=0x7fbab4001a00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-09T13:48:06.906 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:06.900+0000 7fbaba7fc640 1 -- 172.21.15.43:0/1910083871 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbab4004bf0 con 0x7fbac40b34f0 2023-08-09T13:48:06.906 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:06.900+0000 7fbaba7fc640 1 -- 172.21.15.43:0/1910083871 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbab4004d50 con 0x7fbac40b34f0 2023-08-09T13:48:06.906 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:06.900+0000 7fbaba7fc640 1 -- 172.21.15.43:0/1910083871 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbab4007270 con 0x7fbac40b34f0 2023-08-09T13:48:06.906 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:06.900+0000 7fbacc0cc640 1 -- 172.21.15.43:0/1910083871 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbac40c7060 con 0x7fbac40b34f0 2023-08-09T13:48:06.906 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:06.901+0000 7fbacc0cc640 1 -- 172.21.15.43:0/1910083871 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbac40c64b0 con 0x7fbac40b34f0 2023-08-09T13:48:06.907 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:06.902+0000 7fbacc0cc640 1 -- 172.21.15.43:0/1910083871 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbac40739f0 con 0x7fbac40b34f0 2023-08-09T13:48:06.907 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:06.903+0000 7fbaba7fc640 1 -- 172.21.15.43:0/1910083871 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 18) v1 ==== 88443+0+0 (secure 0 0 0) 0x7fbab4032070 con 0x7fbac40b34f0 2023-08-09T13:48:06.907 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:06.904+0000 7fbaba7fc640 1 --2- 172.21.15.43:0/1910083871 >> [v2:172.21.15.203:6832/272917756,v1:172.21.15.203:6833/272917756] conn(0x7fbaa806a1a0 0x7fbaa806c660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:48:06.908 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:06.904+0000 7fbaba7fc640 1 -- 172.21.15.43:0/1910083871 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(66..66 src has 1..66) v4 ==== 6305+0+0 (secure 0 0 0) 0x7fbab40a9d30 con 0x7fbac40b34f0 2023-08-09T13:48:06.908 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:06.904+0000 7fbac9e68640 1 --2- 172.21.15.43:0/1910083871 >> [v2:172.21.15.203:6832/272917756,v1:172.21.15.203:6833/272917756] conn(0x7fbaa806a1a0 0x7fbaa806c660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:48:06.908 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:06.905+0000 7fbac9e68640 1 --2- 172.21.15.43:0/1910083871 >> [v2:172.21.15.203:6832/272917756,v1:172.21.15.203:6833/272917756] conn(0x7fbaa806a1a0 0x7fbaa806c660 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7fbac0006a90 tx=0x7fbac0006a20 comp rx=0 tx=0).ready entity=mgr.5533 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-09T13:48:06.910 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:06.908+0000 7fbaba7fc640 1 -- 172.21.15.43:0/1910083871 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178066 (secure 0 0 0) 0x7fbab40751f0 con 0x7fbac40b34f0 2023-08-09T13:48:07.122 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:07.120+0000 7fbacc0cc640 1 -- 172.21.15.43:0/1910083871 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"} v 0) v1 -- 0x7fbac40d0320 con 0x7fbac40b34f0 2023-08-09T13:48:07.129 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:07.126+0000 7fbaba7fc640 1 -- 172.21.15.43:0/1910083871 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"}]=0 v15) v1 ==== 104+0+0 (secure 0 0 0) 0x7fbab4078810 con 0x7fbac40b34f0 2023-08-09T13:48:07.132 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:07.130+0000 7fbacc0cc640 1 -- 172.21.15.43:0/1910083871 >> [v2:172.21.15.203:6832/272917756,v1:172.21.15.203:6833/272917756] conn(0x7fbaa806a1a0 msgr2=0x7fbaa806c660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:48:07.133 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:07.130+0000 7fbacc0cc640 1 --2- 172.21.15.43:0/1910083871 >> [v2:172.21.15.203:6832/272917756,v1:172.21.15.203:6833/272917756] conn(0x7fbaa806a1a0 0x7fbaa806c660 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7fbac0006a90 tx=0x7fbac0006a20 comp rx=0 tx=0).stop 2023-08-09T13:48:07.133 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:07.130+0000 7fbacc0cc640 1 -- 172.21.15.43:0/1910083871 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fbac40b34f0 msgr2=0x7fbac41488b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:48:07.133 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:07.130+0000 7fbacc0cc640 1 --2- 172.21.15.43:0/1910083871 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fbac40b34f0 0x7fbac41488b0 secure :-1 s=READY pgs=801 cs=0 l=1 rev1=1 crypto rx=0x7fbab400c240 tx=0x7fbab4001a00 comp rx=0 tx=0).stop 2023-08-09T13:48:07.133 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:07.130+0000 7fbacc0cc640 1 -- 172.21.15.43:0/1910083871 shutdown_connections 2023-08-09T13:48:07.133 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:07.130+0000 7fbacc0cc640 1 --2- 172.21.15.43:0/1910083871 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fbac415a2e0 0x7fbac415a760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:48:07.133 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:07.130+0000 7fbacc0cc640 1 --2- 172.21.15.43:0/1910083871 >> [v2:172.21.15.203:6832/272917756,v1:172.21.15.203:6833/272917756] conn(0x7fbaa806a1a0 0x7fbaa806c660 unknown :-1 s=CLOSED pgs=38 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:48:07.133 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:07.130+0000 7fbacc0cc640 1 --2- 172.21.15.43:0/1910083871 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fbac40b34f0 0x7fbac41488b0 unknown :-1 s=CLOSED pgs=801 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:48:07.134 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:07.130+0000 7fbacc0cc640 1 --2- 172.21.15.43:0/1910083871 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fbab0003fa0 0x7fbac40c5f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:48:07.134 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:07.130+0000 7fbacc0cc640 1 -- 172.21.15.43:0/1910083871 >> 172.21.15.43:0/1910083871 conn(0x7fbac40ceec0 msgr2=0x7fbac40b3e90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-09T13:48:07.134 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:07.130+0000 7fbacc0cc640 1 -- 172.21.15.43:0/1910083871 shutdown_connections 2023-08-09T13:48:07.134 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:07.130+0000 7fbacc0cc640 1 -- 172.21.15.43:0/1910083871 wait complete. 2023-08-09T13:48:07.151 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Ended test tasks.cephfs.test_snap_schedules.TestSnapSchedules.test_concurrent_snap_creates' 2023-08-09T13:48:07.383 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:07.380+0000 7f3c752d9640 1 Processor -- start 2023-08-09T13:48:07.383 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:07.381+0000 7f3c752d9640 1 -- start start 2023-08-09T13:48:07.385 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:07.382+0000 7f3c752d9640 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7f3c70142ca0 0x7f3c700c96e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:48:07.385 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:07.382+0000 7f3c752d9640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f3c7015ce40 0x7f3c700cce00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:48:07.385 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:07.382+0000 7f3c752d9640 1 -- --> v1:172.21.15.203:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3c70143080 con 0x7f3c700cd340 2023-08-09T13:48:07.386 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:07.382+0000 7f3c752d9640 1 -- --> v2:172.21.15.203:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3c700cd7b0 con 0x7f3c70142ca0 2023-08-09T13:48:07.386 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:07.382+0000 7f3c752d9640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3c7006d400 con 0x7f3c7015ce40 2023-08-09T13:48:07.386 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:07.383+0000 7f3c6e7fc640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f3c7015ce40 0x7f3c700cce00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:48:07.387 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:07.383+0000 7f3c6effd640 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7f3c70142ca0 0x7f3c700c96e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:48:07.387 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:07.383+0000 7f3c6e7fc640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f3c7015ce40 0x7f3c700cce00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.203:3301/0 says I am v2:172.21.15.43:55116/0 (socket says 172.21.15.43:55116) 2023-08-09T13:48:07.387 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:07.383+0000 7f3c6e7fc640 1 -- 172.21.15.43:0/3738958358 learned_addr learned my addr 172.21.15.43:0/3738958358 (peer_addr_for_me v2:172.21.15.43:0/0) 2023-08-09T13:48:07.387 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:07.384+0000 7f3c6e7fc640 1 -- 172.21.15.43:0/3738958358 >> v2:172.21.15.203:3300/0 conn(0x7f3c70142ca0 msgr2=0x7f3c700c96e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-09T13:48:07.387 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:07.384+0000 7f3c6e7fc640 1 --2- 172.21.15.43:0/3738958358 >> v2:172.21.15.203:3300/0 conn(0x7f3c70142ca0 0x7f3c700c96e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:48:07.387 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:07.384+0000 7f3c6e7fc640 1 -- 172.21.15.43:0/3738958358 >> v1:172.21.15.203:6789/0 conn(0x7f3c700cd340 legacy=0x7f3c700cbc70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:48:07.388 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:07.384+0000 7f3c6e7fc640 1 -- 172.21.15.43:0/3738958358 --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3c700c9d90 con 0x7f3c7015ce40 2023-08-09T13:48:07.388 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:07.384+0000 7f3c6effd640 1 --2- 172.21.15.43:0/3738958358 >> v2:172.21.15.203:3300/0 conn(0x7f3c70142ca0 0x7f3c700c96e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-08-09T13:48:07.388 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:07.386+0000 7f3c6e7fc640 1 --2- 172.21.15.43:0/3738958358 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f3c7015ce40 0x7f3c700cce00 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7f3c6400bb10 tx=0x7f3c6402e150 comp rx=0 tx=0).ready entity=mon.2 client_cookie=7ceb3b0f7b39c95c server_cookie=0 in_seq=0 out_seq=0 2023-08-09T13:48:07.390 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:07.386+0000 7f3c6dffb640 1 -- 172.21.15.43:0/3738958358 <== mon.2 v2:172.21.15.203:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3c64036440 con 0x7f3c7015ce40 2023-08-09T13:48:07.390 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:07.387+0000 7f3c6dffb640 1 -- 172.21.15.43:0/3738958358 <== mon.2 v2:172.21.15.203:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3c640365a0 con 0x7f3c7015ce40 2023-08-09T13:48:07.390 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:07.387+0000 7f3c6dffb640 1 -- 172.21.15.43:0/3738958358 <== mon.2 v2:172.21.15.203:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3c64036930 con 0x7f3c7015ce40 2023-08-09T13:48:07.390 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:07.387+0000 7f3c752d9640 1 -- 172.21.15.43:0/3738958358 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f3c7015ce40 msgr2=0x7f3c700cce00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:48:07.390 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:07.387+0000 7f3c752d9640 1 --2- 172.21.15.43:0/3738958358 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f3c7015ce40 0x7f3c700cce00 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7f3c6400bb10 tx=0x7f3c6402e150 comp rx=0 tx=0).stop 2023-08-09T13:48:07.390 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:07.387+0000 7f3c752d9640 1 -- 172.21.15.43:0/3738958358 shutdown_connections 2023-08-09T13:48:07.391 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:07.387+0000 7f3c752d9640 1 --2- 172.21.15.43:0/3738958358 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f3c7015ce40 0x7f3c700cce00 unknown :-1 s=CLOSED pgs=102 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:48:07.391 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:07.387+0000 7f3c752d9640 1 --2- 172.21.15.43:0/3738958358 >> v2:172.21.15.203:3300/0 conn(0x7f3c70142ca0 0x7f3c700c96e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:48:07.391 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:07.387+0000 7f3c752d9640 1 -- 172.21.15.43:0/3738958358 >> 172.21.15.43:0/3738958358 conn(0x7f3c700ce0a0 msgr2=0x7f3c700ce480 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-09T13:48:07.391 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:07.387+0000 7f3c752d9640 1 -- 172.21.15.43:0/3738958358 shutdown_connections 2023-08-09T13:48:07.391 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:07.387+0000 7f3c752d9640 1 -- 172.21.15.43:0/3738958358 wait complete. 2023-08-09T13:48:07.392 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:07.389+0000 7f3c752d9640 1 Processor -- start 2023-08-09T13:48:07.392 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:07.389+0000 7f3c752d9640 1 -- start start 2023-08-09T13:48:07.393 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:07.390+0000 7f3c752d9640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f3c700cd340 0x7f3c7013ad00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:48:07.394 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:07.390+0000 7f3c752d9640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f3c70142ca0 0x7f3c7015d320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:48:07.394 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:07.390+0000 7f3c752d9640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f3c7015ce40 0x7f3c7015d860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:48:07.394 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:07.391+0000 7f3c752d9640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3c700ad6f0 con 0x7f3c7015ce40 2023-08-09T13:48:07.394 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:07.391+0000 7f3c752d9640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3c7013b240 con 0x7f3c700cd340 2023-08-09T13:48:07.394 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:07.391+0000 7f3c752d9640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3c700acae0 con 0x7f3c70142ca0 2023-08-09T13:48:07.394 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:07.391+0000 7f3c6f7fe640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f3c7015ce40 0x7f3c7015d860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:48:07.395 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:07.391+0000 7f3c6f7fe640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f3c7015ce40 0x7f3c7015d860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:35084/0 (socket says 172.21.15.43:35084) 2023-08-09T13:48:07.395 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:07.391+0000 7f3c6f7fe640 1 -- 172.21.15.43:0/3609312790 learned_addr learned my addr 172.21.15.43:0/3609312790 (peer_addr_for_me v2:172.21.15.43:0/0) 2023-08-09T13:48:07.395 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:07.391+0000 7f3c6effd640 1 --2- 172.21.15.43:0/3609312790 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f3c700cd340 0x7f3c7013ad00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:48:07.395 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:07.391+0000 7f3c6e7fc640 1 --2- 172.21.15.43:0/3609312790 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f3c70142ca0 0x7f3c7015d320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:48:07.396 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:07.391+0000 7f3c6f7fe640 1 -- 172.21.15.43:0/3609312790 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f3c70142ca0 msgr2=0x7f3c7015d320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:48:07.396 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:07.391+0000 7f3c6f7fe640 1 --2- 172.21.15.43:0/3609312790 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f3c70142ca0 0x7f3c7015d320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:48:07.396 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:07.391+0000 7f3c6f7fe640 1 -- 172.21.15.43:0/3609312790 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f3c700cd340 msgr2=0x7f3c7013ad00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:48:07.396 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:07.391+0000 7f3c6f7fe640 1 --2- 172.21.15.43:0/3609312790 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f3c700cd340 0x7f3c7013ad00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:48:07.397 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:07.391+0000 7f3c6f7fe640 1 -- 172.21.15.43:0/3609312790 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3c60009400 con 0x7f3c7015ce40 2023-08-09T13:48:07.397 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:07.392+0000 7f3c6f7fe640 1 --2- 172.21.15.43:0/3609312790 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f3c7015ce40 0x7f3c7015d860 secure :-1 s=READY pgs=802 cs=0 l=1 rev1=1 crypto rx=0x7f3c6800c760 tx=0x7f3c6800cc20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-09T13:48:07.397 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:07.392+0000 7f3c577fe640 1 -- 172.21.15.43:0/3609312790 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3c68003b70 con 0x7f3c7015ce40 2023-08-09T13:48:07.397 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:07.392+0000 7f3c577fe640 1 -- 172.21.15.43:0/3609312790 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3c68003cf0 con 0x7f3c7015ce40 2023-08-09T13:48:07.397 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:07.392+0000 7f3c577fe640 1 -- 172.21.15.43:0/3609312790 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3c68005310 con 0x7f3c7015ce40 2023-08-09T13:48:07.397 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:07.392+0000 7f3c752d9640 1 -- 172.21.15.43:0/3609312790 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3c6400b770 con 0x7f3c7015ce40 2023-08-09T13:48:07.397 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:07.392+0000 7f3c752d9640 1 -- 172.21.15.43:0/3609312790 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3c700ad0a0 con 0x7f3c7015ce40 2023-08-09T13:48:07.398 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:07.393+0000 7f3c752d9640 1 -- 172.21.15.43:0/3609312790 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3c7015cbd0 con 0x7f3c7015ce40 2023-08-09T13:48:07.398 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:07.394+0000 7f3c577fe640 1 -- 172.21.15.43:0/3609312790 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 18) v1 ==== 88443+0+0 (secure 0 0 0) 0x7f3c68005470 con 0x7f3c7015ce40 2023-08-09T13:48:07.398 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:07.395+0000 7f3c577fe640 1 --2- 172.21.15.43:0/3609312790 >> [v2:172.21.15.203:6832/272917756,v1:172.21.15.203:6833/272917756] conn(0x7f3c5c06a170 0x7f3c5c06c630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:48:07.398 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:07.396+0000 7f3c577fe640 1 -- 172.21.15.43:0/3609312790 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(66..66 src has 1..66) v4 ==== 6305+0+0 (secure 0 0 0) 0x7f3c68088720 con 0x7f3c7015ce40 2023-08-09T13:48:07.399 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:07.396+0000 7f3c6effd640 1 --2- 172.21.15.43:0/3609312790 >> [v2:172.21.15.203:6832/272917756,v1:172.21.15.203:6833/272917756] conn(0x7f3c5c06a170 0x7f3c5c06c630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:48:07.400 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:07.397+0000 7f3c6effd640 1 --2- 172.21.15.43:0/3609312790 >> [v2:172.21.15.203:6832/272917756,v1:172.21.15.203:6833/272917756] conn(0x7f3c5c06a170 0x7f3c5c06c630 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7f3c60009c60 tx=0x7f3c60008040 comp rx=0 tx=0).ready entity=mgr.5533 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-09T13:48:07.403 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:07.400+0000 7f3c577fe640 1 -- 172.21.15.43:0/3609312790 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178066 (secure 0 0 0) 0x7f3c68053ad0 con 0x7f3c7015ce40 2023-08-09T13:48:07.666 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:07.664+0000 7f3c752d9640 1 -- 172.21.15.43:0/3609312790 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_snap_schedules.TestSnapSchedules.test_concurrent_snap_creates"]} v 0) v1 -- 0x7f3c700739f0 con 0x7f3c7015ce40 2023-08-09T13:48:08.035 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:08.032+0000 7f3c577fe640 1 -- 172.21.15.43:0/3609312790 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_snap_schedules.TestSnapSchedules.test_concurrent_snap_creates"]}]=0 v527) v1 ==== 158+0+0 (secure 0 0 0) 0x7f3c68003e60 con 0x7f3c7015ce40 2023-08-09T13:48:08.038 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:08.035+0000 7f3c752d9640 1 -- 172.21.15.43:0/3609312790 >> [v2:172.21.15.203:6832/272917756,v1:172.21.15.203:6833/272917756] conn(0x7f3c5c06a170 msgr2=0x7f3c5c06c630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:48:08.038 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:08.035+0000 7f3c752d9640 1 --2- 172.21.15.43:0/3609312790 >> [v2:172.21.15.203:6832/272917756,v1:172.21.15.203:6833/272917756] conn(0x7f3c5c06a170 0x7f3c5c06c630 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7f3c60009c60 tx=0x7f3c60008040 comp rx=0 tx=0).stop 2023-08-09T13:48:08.039 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:08.036+0000 7f3c752d9640 1 -- 172.21.15.43:0/3609312790 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f3c7015ce40 msgr2=0x7f3c7015d860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:48:08.039 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:08.036+0000 7f3c752d9640 1 --2- 172.21.15.43:0/3609312790 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f3c7015ce40 0x7f3c7015d860 secure :-1 s=READY pgs=802 cs=0 l=1 rev1=1 crypto rx=0x7f3c6800c760 tx=0x7f3c6800cc20 comp rx=0 tx=0).stop 2023-08-09T13:48:08.039 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:08.036+0000 7f3c752d9640 1 -- 172.21.15.43:0/3609312790 shutdown_connections 2023-08-09T13:48:08.039 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:08.036+0000 7f3c752d9640 1 --2- 172.21.15.43:0/3609312790 >> [v2:172.21.15.203:6832/272917756,v1:172.21.15.203:6833/272917756] conn(0x7f3c5c06a170 0x7f3c5c06c630 unknown :-1 s=CLOSED pgs=39 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:48:08.039 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:08.036+0000 7f3c752d9640 1 --2- 172.21.15.43:0/3609312790 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f3c7015ce40 0x7f3c7015d860 unknown :-1 s=CLOSED pgs=802 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:48:08.039 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:08.036+0000 7f3c752d9640 1 --2- 172.21.15.43:0/3609312790 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f3c70142ca0 0x7f3c7015d320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:48:08.040 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:08.036+0000 7f3c752d9640 1 --2- 172.21.15.43:0/3609312790 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f3c700cd340 0x7f3c7013ad00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:48:08.040 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:08.036+0000 7f3c752d9640 1 -- 172.21.15.43:0/3609312790 >> 172.21.15.43:0/3609312790 conn(0x7f3c700ce0a0 msgr2=0x7f3c700af490 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-09T13:48:08.040 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:08.036+0000 7f3c752d9640 1 -- 172.21.15.43:0/3609312790 shutdown_connections 2023-08-09T13:48:08.040 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:08.036+0000 7f3c752d9640 1 -- 172.21.15.43:0/3609312790 wait complete. 2023-08-09T13:48:08.060 INFO:tasks.cephfs_test_runner:Test concurrent snap creates in same file-system without db issues ... ok 2023-08-09T13:48:08.060 INFO:tasks.cephfs_test_runner:Starting test: test_multi_snap_schedule (tasks.cephfs.test_snap_schedules.TestSnapSchedules) Test exisitence of multiple scheduled snapshots 2023-08-09T13:48:08.061 INFO:tasks.cephfs_test_runner:test_multi_snap_schedule (tasks.cephfs.test_snap_schedules.TestSnapSchedules) 2023-08-09T13:48:08.061 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Starting test tasks.cephfs.test_snap_schedules.TestSnapSchedules.test_multi_snap_schedule' 2023-08-09T13:48:08.295 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:08.292+0000 7fb43b498640 1 Processor -- start 2023-08-09T13:48:08.295 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:08.292+0000 7fb43b498640 1 -- start start 2023-08-09T13:48:08.297 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:08.294+0000 7fb43b498640 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7fb4340c8750 0x7fb4340ad340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:48:08.297 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:08.294+0000 7fb43b498640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fb4340acdf0 0x7fb43415b2c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:48:08.297 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:08.294+0000 7fb43b498640 1 -- --> v1:172.21.15.43:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb4340cbd90 con 0x7fb4340c8b30 2023-08-09T13:48:08.297 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:08.294+0000 7fb43b498640 1 -- --> v2:172.21.15.203:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb43415e810 con 0x7fb4340c8750 2023-08-09T13:48:08.297 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:08.294+0000 7fb43b498640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb4340c9f70 con 0x7fb4340acdf0 2023-08-09T13:48:08.298 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:08.294+0000 7fb439c95640 1 --1- >> v1:172.21.15.43:6789/0 conn(0x7fb4340c8b30 0x7fb43406f410 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.43:6789/0 says I am v1:172.21.15.43:41660/0 (socket says 172.21.15.43:41660) 2023-08-09T13:48:08.298 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:08.294+0000 7fb439c95640 1 -- 172.21.15.43:0/3688399023 learned_addr learned my addr 172.21.15.43:0/3688399023 (peer_addr_for_me v1:172.21.15.43:0/0) 2023-08-09T13:48:08.299 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:08.295+0000 7fb43ac97640 1 --2- 172.21.15.43:0/3688399023 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fb4340acdf0 0x7fb43415b2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:48:08.299 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:08.295+0000 7fb43a496640 1 --2- 172.21.15.43:0/3688399023 >> v2:172.21.15.203:3300/0 conn(0x7fb4340c8750 0x7fb4340ad340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:48:08.300 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:08.295+0000 7fb439494640 1 -- 172.21.15.43:0/3688399023 <== mon.0 v1:172.21.15.43:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2784078273 0 0) 0x7fb4340cbd90 con 0x7fb4340c8b30 2023-08-09T13:48:08.300 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:08.295+0000 7fb439494640 1 -- 172.21.15.43:0/3688399023 --> v1:172.21.15.43:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fb420003520 con 0x7fb4340c8b30 2023-08-09T13:48:08.300 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:08.295+0000 7fb439494640 1 -- 172.21.15.43:0/3688399023 <== mon.0 v1:172.21.15.43:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2182034581 0 0) 0x7fb420003520 con 0x7fb4340c8b30 2023-08-09T13:48:08.300 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:08.296+0000 7fb439494640 1 -- 172.21.15.43:0/3688399023 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fb4340acdf0 msgr2=0x7fb43415b2c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-09T13:48:08.300 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:08.296+0000 7fb439494640 1 --2- 172.21.15.43:0/3688399023 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fb4340acdf0 0x7fb43415b2c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:48:08.300 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:08.296+0000 7fb439494640 1 -- 172.21.15.43:0/3688399023 >> v2:172.21.15.203:3300/0 conn(0x7fb4340c8750 msgr2=0x7fb4340ad340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-09T13:48:08.300 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:08.296+0000 7fb439494640 1 --2- 172.21.15.43:0/3688399023 >> v2:172.21.15.203:3300/0 conn(0x7fb4340c8750 0x7fb4340ad340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:48:08.301 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:08.296+0000 7fb439494640 1 -- 172.21.15.43:0/3688399023 --> v1:172.21.15.43:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb4340ca2b0 con 0x7fb4340c8b30 2023-08-09T13:48:08.301 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:08.296+0000 7fb439494640 1 -- 172.21.15.43:0/3688399023 <== mon.0 v1:172.21.15.43:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1285150740 0 0) 0x7fb424001670 con 0x7fb4340c8b30 2023-08-09T13:48:08.301 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:08.296+0000 7fb439494640 1 -- 172.21.15.43:0/3688399023 >> v1:172.21.15.43:6789/0 conn(0x7fb4340c8b30 legacy=0x7fb43406f410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:48:08.301 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:08.296+0000 7fb439494640 1 --2- 172.21.15.43:0/3688399023 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fb420003970 0x7fb420003d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:48:08.301 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:08.296+0000 7fb439494640 1 --2- 172.21.15.43:0/3688399023 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb4200047c0 0x7fb420004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:48:08.301 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:08.296+0000 7fb439494640 1 --2- 172.21.15.43:0/3688399023 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fb4340acdf0 0x7fb420005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:48:08.301 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:08.296+0000 7fb439494640 1 -- 172.21.15.43:0/3688399023 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb43415e810 con 0x7fb4200047c0 2023-08-09T13:48:08.302 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:08.296+0000 7fb439494640 1 -- 172.21.15.43:0/3688399023 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb4340c9f70 con 0x7fb420003970 2023-08-09T13:48:08.302 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:08.296+0000 7fb439494640 1 -- 172.21.15.43:0/3688399023 --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb42000b980 con 0x7fb4340acdf0 2023-08-09T13:48:08.302 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:08.296+0000 7fb439c95640 1 --2- 172.21.15.43:0/3688399023 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb4200047c0 0x7fb420004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:48:08.302 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:08.296+0000 7fb43a496640 1 --2- 172.21.15.43:0/3688399023 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fb420003970 0x7fb420003d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:48:08.302 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:08.296+0000 7fb439c95640 1 -- 172.21.15.43:0/3688399023 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fb4340acdf0 msgr2=0x7fb420005180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-09T13:48:08.303 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:08.296+0000 7fb439c95640 1 --2- 172.21.15.43:0/3688399023 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fb4340acdf0 0x7fb420005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:48:08.303 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:08.296+0000 7fb439c95640 1 -- 172.21.15.43:0/3688399023 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fb420003970 msgr2=0x7fb420003d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-09T13:48:08.303 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:08.296+0000 7fb439c95640 1 --2- 172.21.15.43:0/3688399023 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fb420003970 0x7fb420003d90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:48:08.303 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:08.296+0000 7fb439c95640 1 -- 172.21.15.43:0/3688399023 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb420003520 con 0x7fb4200047c0 2023-08-09T13:48:08.304 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:08.298+0000 7fb439c95640 1 --2- 172.21.15.43:0/3688399023 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb4200047c0 0x7fb420004c40 secure :-1 s=READY pgs=804 cs=0 l=1 rev1=1 crypto rx=0x7fb42400c270 tx=0x7fb4200057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b965e43abb40f56b server_cookie=0 in_seq=0 out_seq=0 2023-08-09T13:48:08.305 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:08.298+0000 7fb439494640 1 -- 172.21.15.43:0/3688399023 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb42402cae0 con 0x7fb4200047c0 2023-08-09T13:48:08.305 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:08.298+0000 7fb439494640 1 -- 172.21.15.43:0/3688399023 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb42402cc40 con 0x7fb4200047c0 2023-08-09T13:48:08.305 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:08.298+0000 7fb439494640 1 -- 172.21.15.43:0/3688399023 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb4240058d0 con 0x7fb4200047c0 2023-08-09T13:48:08.305 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:08.298+0000 7fb43b498640 1 -- 172.21.15.43:0/3688399023 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb4200047c0 msgr2=0x7fb420004c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:48:08.305 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:08.298+0000 7fb43b498640 1 --2- 172.21.15.43:0/3688399023 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb4200047c0 0x7fb420004c40 secure :-1 s=READY pgs=804 cs=0 l=1 rev1=1 crypto rx=0x7fb42400c270 tx=0x7fb4200057f0 comp rx=0 tx=0).stop 2023-08-09T13:48:08.305 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:08.298+0000 7fb43ac97640 1 -- 172.21.15.43:0/3688399023 reap_dead start 2023-08-09T13:48:08.305 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:08.299+0000 7fb43b498640 1 -- 172.21.15.43:0/3688399023 shutdown_connections 2023-08-09T13:48:08.306 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:08.299+0000 7fb43b498640 1 -- 172.21.15.43:0/3688399023 >> 172.21.15.43:0/3688399023 conn(0x7fb4340cde90 msgr2=0x7fb4340ce2a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-09T13:48:08.306 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:08.299+0000 7fb43b498640 1 -- 172.21.15.43:0/3688399023 shutdown_connections 2023-08-09T13:48:08.306 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:08.299+0000 7fb43b498640 1 -- 172.21.15.43:0/3688399023 wait complete. 2023-08-09T13:48:08.306 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:08.300+0000 7fb43b498640 1 Processor -- start 2023-08-09T13:48:08.306 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:08.301+0000 7fb43b498640 1 -- start start 2023-08-09T13:48:08.306 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:08.302+0000 7fb43b498640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fb4200047c0 0x7fb4340c4f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:48:08.307 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:08.302+0000 7fb43b498640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fb43414c310 0x7fb4340c7240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:48:08.307 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:08.302+0000 7fb43b498640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb4340c7780 0x7fb43415b9d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:48:08.307 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:08.302+0000 7fb43b498640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb4340c7c00 con 0x7fb4340c7780 2023-08-09T13:48:08.307 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:08.302+0000 7fb43b498640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb4340c6aa0 con 0x7fb4200047c0 2023-08-09T13:48:08.307 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:08.302+0000 7fb43b498640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb43409ac30 con 0x7fb43414c310 2023-08-09T13:48:08.307 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:08.302+0000 7fb43ac97640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb4340c7780 0x7fb43415b9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:48:08.307 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:08.302+0000 7fb43ac97640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb4340c7780 0x7fb43415b9d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:35108/0 (socket says 172.21.15.43:35108) 2023-08-09T13:48:08.307 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:08.302+0000 7fb43ac97640 1 -- 172.21.15.43:0/1388455829 learned_addr learned my addr 172.21.15.43:0/1388455829 (peer_addr_for_me v2:172.21.15.43:0/0) 2023-08-09T13:48:08.309 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:08.302+0000 7fb43ac97640 1 -- 172.21.15.43:0/1388455829 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fb43414c310 msgr2=0x7fb4340c7240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:48:08.309 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:08.302+0000 7fb439c95640 1 --2- 172.21.15.43:0/1388455829 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fb43414c310 0x7fb4340c7240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:48:08.309 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:08.302+0000 7fb43a496640 1 --2- 172.21.15.43:0/1388455829 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fb4200047c0 0x7fb4340c4f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:48:08.309 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:08.302+0000 7fb43ac97640 1 --2- 172.21.15.43:0/1388455829 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fb43414c310 0x7fb4340c7240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:48:08.310 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:08.302+0000 7fb43ac97640 1 -- 172.21.15.43:0/1388455829 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fb4200047c0 msgr2=0x7fb4340c4f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:48:08.310 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:08.302+0000 7fb43ac97640 1 --2- 172.21.15.43:0/1388455829 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fb4200047c0 0x7fb4340c4f60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:48:08.310 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:08.302+0000 7fb43ac97640 1 -- 172.21.15.43:0/1388455829 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb43409af10 con 0x7fb4340c7780 2023-08-09T13:48:08.310 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:08.303+0000 7fb43ac97640 1 --2- 172.21.15.43:0/1388455829 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb4340c7780 0x7fb43415b9d0 secure :-1 s=READY pgs=805 cs=0 l=1 rev1=1 crypto rx=0x7fb4340c8f10 tx=0x7fb42c00b480 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-09T13:48:08.310 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:08.303+0000 7fb42affd640 1 -- 172.21.15.43:0/1388455829 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb42c010040 con 0x7fb4340c7780 2023-08-09T13:48:08.310 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:08.303+0000 7fb42affd640 1 -- 172.21.15.43:0/1388455829 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb42c00bce0 con 0x7fb4340c7780 2023-08-09T13:48:08.310 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:08.303+0000 7fb42affd640 1 -- 172.21.15.43:0/1388455829 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb42c014640 con 0x7fb4340c7780 2023-08-09T13:48:08.311 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:08.303+0000 7fb43b498640 1 -- 172.21.15.43:0/1388455829 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb43409b1f0 con 0x7fb4340c7780 2023-08-09T13:48:08.311 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:08.303+0000 7fb43b498640 1 -- 172.21.15.43:0/1388455829 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb43414c770 con 0x7fb4340c7780 2023-08-09T13:48:08.311 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:08.304+0000 7fb43b498640 1 -- 172.21.15.43:0/1388455829 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb4340ad5b0 con 0x7fb4340c7780 2023-08-09T13:48:08.311 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:08.305+0000 7fb42affd640 1 -- 172.21.15.43:0/1388455829 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 18) v1 ==== 88443+0+0 (secure 0 0 0) 0x7fb42c0147a0 con 0x7fb4340c7780 2023-08-09T13:48:08.311 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:08.306+0000 7fb42affd640 1 --2- 172.21.15.43:0/1388455829 >> [v2:172.21.15.203:6832/272917756,v1:172.21.15.203:6833/272917756] conn(0x7fb41806a170 0x7fb41806c630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:48:08.311 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:08.306+0000 7fb42affd640 1 -- 172.21.15.43:0/1388455829 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(66..66 src has 1..66) v4 ==== 6305+0+0 (secure 0 0 0) 0x7fb42c088d30 con 0x7fb4340c7780 2023-08-09T13:48:08.312 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:08.307+0000 7fb43a496640 1 --2- 172.21.15.43:0/1388455829 >> [v2:172.21.15.203:6832/272917756,v1:172.21.15.203:6833/272917756] conn(0x7fb41806a170 0x7fb41806c630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:48:08.312 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:08.308+0000 7fb43a496640 1 --2- 172.21.15.43:0/1388455829 >> [v2:172.21.15.203:6832/272917756,v1:172.21.15.203:6833/272917756] conn(0x7fb41806a170 0x7fb41806c630 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7fb430009910 tx=0x7fb430008040 comp rx=0 tx=0).ready entity=mgr.5533 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-09T13:48:08.313 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:08.310+0000 7fb42affd640 1 -- 172.21.15.43:0/1388455829 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178066 (secure 0 0 0) 0x7fb42c054060 con 0x7fb4340c7780 2023-08-09T13:48:08.576 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:08.573+0000 7fb43b498640 1 -- 172.21.15.43:0/1388455829 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_snap_schedules.TestSnapSchedules.test_multi_snap_schedule"]} v 0) v1 -- 0x7fb4340c9230 con 0x7fb4340c7780 2023-08-09T13:48:09.038 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:09.035+0000 7fb42affd640 1 -- 172.21.15.43:0/1388455829 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_snap_schedules.TestSnapSchedules.test_multi_snap_schedule"]}]=0 v528) v1 ==== 157+0+0 (secure 0 0 0) 0x7fb42c057680 con 0x7fb4340c7780 2023-08-09T13:48:09.041 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:09.038+0000 7fb43b498640 1 -- 172.21.15.43:0/1388455829 >> [v2:172.21.15.203:6832/272917756,v1:172.21.15.203:6833/272917756] conn(0x7fb41806a170 msgr2=0x7fb41806c630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:48:09.042 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:09.038+0000 7fb43b498640 1 --2- 172.21.15.43:0/1388455829 >> [v2:172.21.15.203:6832/272917756,v1:172.21.15.203:6833/272917756] conn(0x7fb41806a170 0x7fb41806c630 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7fb430009910 tx=0x7fb430008040 comp rx=0 tx=0).stop 2023-08-09T13:48:09.042 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:09.039+0000 7fb43b498640 1 -- 172.21.15.43:0/1388455829 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb4340c7780 msgr2=0x7fb43415b9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:48:09.042 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:09.039+0000 7fb43b498640 1 --2- 172.21.15.43:0/1388455829 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb4340c7780 0x7fb43415b9d0 secure :-1 s=READY pgs=805 cs=0 l=1 rev1=1 crypto rx=0x7fb4340c8f10 tx=0x7fb42c00b480 comp rx=0 tx=0).stop 2023-08-09T13:48:09.043 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:09.039+0000 7fb43b498640 1 -- 172.21.15.43:0/1388455829 shutdown_connections 2023-08-09T13:48:09.043 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:09.039+0000 7fb43b498640 1 --2- 172.21.15.43:0/1388455829 >> [v2:172.21.15.203:6832/272917756,v1:172.21.15.203:6833/272917756] conn(0x7fb41806a170 0x7fb41806c630 unknown :-1 s=CLOSED pgs=40 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:48:09.043 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:09.039+0000 7fb43b498640 1 --2- 172.21.15.43:0/1388455829 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb4340c7780 0x7fb43415b9d0 unknown :-1 s=CLOSED pgs=805 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:48:09.044 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:09.039+0000 7fb43b498640 1 --2- 172.21.15.43:0/1388455829 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fb43414c310 0x7fb4340c7240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:48:09.044 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:09.039+0000 7fb43b498640 1 --2- 172.21.15.43:0/1388455829 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fb4200047c0 0x7fb4340c4f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:48:09.044 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:09.039+0000 7fb43b498640 1 -- 172.21.15.43:0/1388455829 >> 172.21.15.43:0/1388455829 conn(0x7fb4340cde90 msgr2=0x7fb4340adda0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-09T13:48:09.044 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:09.039+0000 7fb43b498640 1 -- 172.21.15.43:0/1388455829 shutdown_connections 2023-08-09T13:48:09.044 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:09.039+0000 7fb43b498640 1 -- 172.21.15.43:0/1388455829 wait complete. 2023-08-09T13:48:09.064 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-08-09T13:48:09.296 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:09.294+0000 7f4ff0ce7640 1 Processor -- start 2023-08-09T13:48:09.296 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:09.294+0000 7f4ff0ce7640 1 -- start start 2023-08-09T13:48:09.298 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:09.296+0000 7f4ff0ce7640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f4fec0c86f0 0x7f4fec0c8b50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:48:09.298 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:09.296+0000 7f4ff0ce7640 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7f4fec06c8d0 0x7f4fec06cd60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:48:09.298 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:09.296+0000 7f4ff0ce7640 1 -- --> v1:172.21.15.43:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4fec0c9090 con 0x7f4fec143c20 2023-08-09T13:48:09.298 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:09.296+0000 7f4ff0ce7640 1 -- --> v2:172.21.15.203:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4fec144000 con 0x7f4fec06c8d0 2023-08-09T13:48:09.299 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:09.296+0000 7f4ff0ce7640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4fec06c440 con 0x7f4fec0c86f0 2023-08-09T13:48:09.300 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:09.296+0000 7f4fea59c640 1 --1- >> v1:172.21.15.43:6789/0 conn(0x7f4fec143c20 0x7f4fec06f540 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.43:6789/0 says I am v1:172.21.15.43:41674/0 (socket says 172.21.15.43:41674) 2023-08-09T13:48:09.300 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:09.296+0000 7f4fea59c640 1 -- 172.21.15.43:0/1574976522 learned_addr learned my addr 172.21.15.43:0/1574976522 (peer_addr_for_me v1:172.21.15.43:0/0) 2023-08-09T13:48:09.300 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:09.296+0000 7f4fe3fff640 1 --2- 172.21.15.43:0/1574976522 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f4fec0c86f0 0x7f4fec0c8b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:48:09.300 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:09.296+0000 7f4fead9d640 1 --2- 172.21.15.43:0/1574976522 >> v2:172.21.15.203:3300/0 conn(0x7f4fec06c8d0 0x7f4fec06cd60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:48:09.301 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:09.296+0000 7f4fe9d9b640 1 -- 172.21.15.43:0/1574976522 <== mon.0 v1:172.21.15.43:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 491894560 0 0) 0x7f4fec0c9090 con 0x7f4fec143c20 2023-08-09T13:48:09.301 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:09.297+0000 7f4fe9d9b640 1 -- 172.21.15.43:0/1574976522 --> v1:172.21.15.43:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f4fd8003520 con 0x7f4fec143c20 2023-08-09T13:48:09.301 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:09.297+0000 7f4fe9d9b640 1 -- 172.21.15.43:0/1574976522 <== mon.0 v1:172.21.15.43:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3711463450 0 0) 0x7f4fd8003520 con 0x7f4fec143c20 2023-08-09T13:48:09.301 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:09.297+0000 7f4fe9d9b640 1 -- 172.21.15.43:0/1574976522 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f4fec0c86f0 msgr2=0x7f4fec0c8b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-09T13:48:09.301 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:09.297+0000 7f4fe9d9b640 1 --2- 172.21.15.43:0/1574976522 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f4fec0c86f0 0x7f4fec0c8b50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:48:09.301 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:09.297+0000 7f4fe9d9b640 1 -- 172.21.15.43:0/1574976522 >> v2:172.21.15.203:3300/0 conn(0x7f4fec06c8d0 msgr2=0x7f4fec06cd60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-09T13:48:09.301 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:09.297+0000 7f4fe9d9b640 1 --2- 172.21.15.43:0/1574976522 >> v2:172.21.15.203:3300/0 conn(0x7f4fec06c8d0 0x7f4fec06cd60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:48:09.301 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:09.297+0000 7f4fe9d9b640 1 -- 172.21.15.43:0/1574976522 --> v1:172.21.15.43:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4fec06fc50 con 0x7f4fec143c20 2023-08-09T13:48:09.302 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:09.297+0000 7f4fe9d9b640 1 -- 172.21.15.43:0/1574976522 <== mon.0 v1:172.21.15.43:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1285150740 0 0) 0x7f4fdc001670 con 0x7f4fec143c20 2023-08-09T13:48:09.302 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:09.297+0000 7f4fe9d9b640 1 -- 172.21.15.43:0/1574976522 >> v1:172.21.15.43:6789/0 conn(0x7f4fec143c20 legacy=0x7f4fec06f540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:48:09.302 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:09.297+0000 7f4fe9d9b640 1 --2- 172.21.15.43:0/1574976522 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f4fd80039e0 0x7f4fd8003e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:48:09.302 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:09.297+0000 7f4fe9d9b640 1 --2- 172.21.15.43:0/1574976522 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f4fd80047c0 0x7f4fd8004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:48:09.302 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:09.297+0000 7f4fe9d9b640 1 --2- 172.21.15.43:0/1574976522 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f4fd8005180 0x7f4fd800b980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:48:09.302 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:09.297+0000 7f4fe9d9b640 1 -- 172.21.15.43:0/1574976522 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4fec144000 con 0x7f4fd8005180 2023-08-09T13:48:09.302 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:09.297+0000 7f4fe9d9b640 1 -- 172.21.15.43:0/1574976522 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4fec06c440 con 0x7f4fd80047c0 2023-08-09T13:48:09.302 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:09.297+0000 7f4fe9d9b640 1 -- 172.21.15.43:0/1574976522 --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4fd800bf70 con 0x7f4fd80039e0 2023-08-09T13:48:09.303 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:09.298+0000 7f4fead9d640 1 --2- 172.21.15.43:0/1574976522 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f4fd8005180 0x7f4fd800b980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:48:09.303 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:09.298+0000 7f4fea59c640 1 --2- 172.21.15.43:0/1574976522 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f4fd80039e0 0x7f4fd8003e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:48:09.304 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:09.298+0000 7f4fead9d640 1 -- 172.21.15.43:0/1574976522 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f4fd80039e0 msgr2=0x7f4fd8003e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-09T13:48:09.304 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:09.298+0000 7f4fead9d640 1 --2- 172.21.15.43:0/1574976522 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f4fd80039e0 0x7f4fd8003e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:48:09.304 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:09.298+0000 7f4fead9d640 1 -- 172.21.15.43:0/1574976522 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f4fd80047c0 msgr2=0x7f4fd8004c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-09T13:48:09.304 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:09.298+0000 7f4fead9d640 1 --2- 172.21.15.43:0/1574976522 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f4fd80047c0 0x7f4fd8004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:48:09.304 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:09.298+0000 7f4fead9d640 1 -- 172.21.15.43:0/1574976522 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4fd8003520 con 0x7f4fd8005180 2023-08-09T13:48:09.305 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:09.298+0000 7f4fea59c640 1 --2- 172.21.15.43:0/1574976522 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f4fd80039e0 0x7f4fd8003e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-08-09T13:48:09.305 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:09.299+0000 7f4fead9d640 1 --2- 172.21.15.43:0/1574976522 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f4fd8005180 0x7f4fd800b980 secure :-1 s=READY pgs=807 cs=0 l=1 rev1=1 crypto rx=0x7f4fe4009fc0 tx=0x7f4fd80057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2959fbcfffdee7d7 server_cookie=0 in_seq=0 out_seq=0 2023-08-09T13:48:09.305 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:09.300+0000 7f4fe9d9b640 1 -- 172.21.15.43:0/1574976522 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4fe402cc60 con 0x7f4fd8005180 2023-08-09T13:48:09.305 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:09.300+0000 7f4fe9d9b640 1 -- 172.21.15.43:0/1574976522 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4fe400f040 con 0x7f4fd8005180 2023-08-09T13:48:09.305 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:09.300+0000 7f4fe9d9b640 1 -- 172.21.15.43:0/1574976522 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4fe4034440 con 0x7f4fd8005180 2023-08-09T13:48:09.305 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:09.300+0000 7f4ff0ce7640 1 -- 172.21.15.43:0/1574976522 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f4fd8005180 msgr2=0x7f4fd800b980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:48:09.305 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:09.300+0000 7f4ff0ce7640 1 --2- 172.21.15.43:0/1574976522 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f4fd8005180 0x7f4fd800b980 secure :-1 s=READY pgs=807 cs=0 l=1 rev1=1 crypto rx=0x7f4fe4009fc0 tx=0x7f4fd80057f0 comp rx=0 tx=0).stop 2023-08-09T13:48:09.305 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:09.300+0000 7f4fead9d640 1 -- 172.21.15.43:0/1574976522 reap_dead start 2023-08-09T13:48:09.306 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:09.300+0000 7f4ff0ce7640 1 -- 172.21.15.43:0/1574976522 shutdown_connections 2023-08-09T13:48:09.306 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:09.300+0000 7f4ff0ce7640 1 -- 172.21.15.43:0/1574976522 >> 172.21.15.43:0/1574976522 conn(0x7f4fec0ce0a0 msgr2=0x7f4fec0ce480 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-09T13:48:09.306 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:09.300+0000 7f4ff0ce7640 1 -- 172.21.15.43:0/1574976522 shutdown_connections 2023-08-09T13:48:09.306 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:09.300+0000 7f4ff0ce7640 1 -- 172.21.15.43:0/1574976522 wait complete. 2023-08-09T13:48:09.307 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:09.302+0000 7f4ff0ce7640 1 Processor -- start 2023-08-09T13:48:09.308 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:09.302+0000 7f4ff0ce7640 1 -- start start 2023-08-09T13:48:09.308 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:09.303+0000 7f4ff0ce7640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f4fd8005180 0x7f4fec0c7c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:48:09.308 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:09.303+0000 7f4ff0ce7640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f4fec0cad10 0x7f4fec0b0d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:48:09.308 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:09.303+0000 7f4ff0ce7640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f4fec14d930 0x7f4fec0b1710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:48:09.308 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:09.303+0000 7f4ff0ce7640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4fec0b3590 con 0x7f4fec0cad10 2023-08-09T13:48:09.308 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:09.303+0000 7f4ff0ce7640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4fec0b3830 con 0x7f4fec14d930 2023-08-09T13:48:09.309 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:09.303+0000 7f4ff0ce7640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4fec0b3b10 con 0x7f4fd8005180 2023-08-09T13:48:09.309 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:09.304+0000 7f4fe3fff640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f4fec0cad10 0x7f4fec0b0d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:48:09.309 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:09.304+0000 7f4fe3fff640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f4fec0cad10 0x7f4fec0b0d80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:35122/0 (socket says 172.21.15.43:35122) 2023-08-09T13:48:09.309 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:09.304+0000 7f4fe3fff640 1 -- 172.21.15.43:0/142132462 learned_addr learned my addr 172.21.15.43:0/142132462 (peer_addr_for_me v2:172.21.15.43:0/0) 2023-08-09T13:48:09.309 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:09.304+0000 7f4fea59c640 1 --2- 172.21.15.43:0/142132462 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f4fd8005180 0x7f4fec0c7c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:48:09.309 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:09.304+0000 7f4fead9d640 1 --2- 172.21.15.43:0/142132462 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f4fec14d930 0x7f4fec0b1710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:48:09.309 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:09.304+0000 7f4fe3fff640 1 -- 172.21.15.43:0/142132462 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f4fd8005180 msgr2=0x7f4fec0c7c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:48:09.309 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:09.304+0000 7f4fe3fff640 1 --2- 172.21.15.43:0/142132462 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f4fd8005180 0x7f4fec0c7c20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:48:09.310 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:09.304+0000 7f4fe3fff640 1 -- 172.21.15.43:0/142132462 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f4fec14d930 msgr2=0x7f4fec0b1710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:48:09.310 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:09.304+0000 7f4fe3fff640 1 --2- 172.21.15.43:0/142132462 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f4fec14d930 0x7f4fec0b1710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:48:09.310 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:09.304+0000 7f4fe3fff640 1 -- 172.21.15.43:0/142132462 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4fec0b3df0 con 0x7f4fec0cad10 2023-08-09T13:48:09.310 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:09.304+0000 7f4fe3fff640 1 --2- 172.21.15.43:0/142132462 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f4fec0cad10 0x7f4fec0b0d80 secure :-1 s=READY pgs=808 cs=0 l=1 rev1=1 crypto rx=0x7f4fd4004650 tx=0x7f4fd400ec70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-09T13:48:09.310 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:09.305+0000 7f4fe2ffd640 1 -- 172.21.15.43:0/142132462 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4fd400eec0 con 0x7f4fec0cad10 2023-08-09T13:48:09.310 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:09.305+0000 7f4fe2ffd640 1 -- 172.21.15.43:0/142132462 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4fd400c7e0 con 0x7f4fec0cad10 2023-08-09T13:48:09.310 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:09.305+0000 7f4fe2ffd640 1 -- 172.21.15.43:0/142132462 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4fd400cb90 con 0x7f4fec0cad10 2023-08-09T13:48:09.311 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:09.305+0000 7f4ff0ce7640 1 -- 172.21.15.43:0/142132462 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4fec0b40d0 con 0x7f4fec0cad10 2023-08-09T13:48:09.311 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:09.305+0000 7f4ff0ce7640 1 -- 172.21.15.43:0/142132462 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4fec0b1c50 con 0x7f4fec0cad10 2023-08-09T13:48:09.311 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:09.306+0000 7f4ff0ce7640 1 -- 172.21.15.43:0/142132462 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4fec0b42c0 con 0x7f4fec0cad10 2023-08-09T13:48:09.311 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:09.307+0000 7f4fe2ffd640 1 -- 172.21.15.43:0/142132462 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 18) v1 ==== 88443+0+0 (secure 0 0 0) 0x7f4fd400ccf0 con 0x7f4fec0cad10 2023-08-09T13:48:09.312 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:09.308+0000 7f4fe2ffd640 1 --2- 172.21.15.43:0/142132462 >> [v2:172.21.15.203:6832/272917756,v1:172.21.15.203:6833/272917756] conn(0x7f4fcc06a1a0 0x7f4fcc06c660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:48:09.312 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:09.308+0000 7f4fe2ffd640 1 -- 172.21.15.43:0/142132462 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(66..66 src has 1..66) v4 ==== 6305+0+0 (secure 0 0 0) 0x7f4fd4088850 con 0x7f4fec0cad10 2023-08-09T13:48:09.312 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:09.309+0000 7f4fea59c640 1 --2- 172.21.15.43:0/142132462 >> [v2:172.21.15.203:6832/272917756,v1:172.21.15.203:6833/272917756] conn(0x7f4fcc06a1a0 0x7f4fcc06c660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:48:09.312 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:09.310+0000 7f4fea59c640 1 --2- 172.21.15.43:0/142132462 >> [v2:172.21.15.203:6832/272917756,v1:172.21.15.203:6833/272917756] conn(0x7f4fcc06a1a0 0x7f4fcc06c660 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7f4fdc005b10 tx=0x7f4fdc009040 comp rx=0 tx=0).ready entity=mgr.5533 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-09T13:48:09.315 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:09.312+0000 7f4fe2ffd640 1 -- 172.21.15.43:0/142132462 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178066 (secure 0 0 0) 0x7f4fd4053c90 con 0x7f4fec0cad10 2023-08-09T13:48:09.509 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:09.507+0000 7f4ff0ce7640 1 -- 172.21.15.43:0/142132462 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f4fec0739f0 con 0x7f4fec0cad10 2023-08-09T13:48:09.511 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:09.509+0000 7f4fe2ffd640 1 -- 172.21.15.43:0/142132462 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v66) v1 ==== 74+0+15922 (secure 0 0 0) 0x7f4fd40572b0 con 0x7f4fec0cad10 2023-08-09T13:48:09.512 INFO:teuthology.orchestra.run.smithi043.stdout: 2023-08-09T13:48:09.512 INFO:teuthology.orchestra.run.smithi043.stdout:{"epoch":66,"fsid":"daac87ee-1337-4250-a955-aea4e408a39b","created":"2023-08-09T13:33:41.701198+0000","modified":"2023-08-09T13:48:05.693869+0000","last_up_change":"2023-08-09T13:33:53.783453+0000","last_in_change":"2023-08-09T13:33:45.148904+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":7,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":"rbd","create_time":"2023-08-09T13:33:54.365137+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":".mgr","create_time":"2023-08-09T13:33:54.831803+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"cbd49ecd-e896-4fbd-864e-5c37a829b165","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":37,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6800","nonce":502667335},{"type":"v1","addr":"172.21.15.43:6801","nonce":502667335}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6802","nonce":502667335},{"type":"v1","addr":"172.21.15.43:6803","nonce":502667335}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6806","nonce":502667335},{"type":"v1","addr":"172.21.15.43:6807","nonce":502667335}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6804","nonce":502667335},{"type":"v1","addr":"172.21.15.43:6805","nonce":502667335}]},"public_addr":"172.21.15.43:6801/502667335","cluster_addr":"172.21.15.43:6803/502667335","heartbeat_back_addr":"172.21.15.43:6807/502667335","heartbeat_front_addr":"172.21.15.43:6805/502667335","state":["exists","up"]},{"osd":1,"uuid":"75c7e0fe-1ec7-4fe8-a2f2-3661a65b8500","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":37,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6808","nonce":3289134610},{"type":"v1","addr":"172.21.15.43:6809","nonce":3289134610}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6810","nonce":3289134610},{"type":"v1","addr":"172.21.15.43:6812","nonce":3289134610}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6821","nonce":3289134610},{"type":"v1","addr":"172.21.15.43:6824","nonce":3289134610}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6815","nonce":3289134610},{"type":"v1","addr":"172.21.15.43:6818","nonce":3289134610}]},"public_addr":"172.21.15.43:6809/3289134610","cluster_addr":"172.21.15.43:6812/3289134610","heartbeat_back_addr":"172.21.15.43:6824/3289134610","heartbeat_front_addr":"172.21.15.43:6818/3289134610","state":["exists","up"]},{"osd":2,"uuid":"bb5c41e9-7f5f-4125-9b01-3bb8bfc2ea83","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":37,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6811","nonce":4079709736},{"type":"v1","addr":"172.21.15.43:6814","nonce":4079709736}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6817","nonce":4079709736},{"type":"v1","addr":"172.21.15.43:6820","nonce":4079709736}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6829","nonce":4079709736},{"type":"v1","addr":"172.21.15.43:6831","nonce":4079709736}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6823","nonce":4079709736},{"type":"v1","addr":"172.21.15.43:6826","nonce":4079709736}]},"public_addr":"172.21.15.43:6814/4079709736","cluster_addr":"172.21.15.43:6820/4079709736","heartbeat_back_addr":"172.21.15.43:6831/4079709736","heartbeat_front_addr":"172.21.15.43:6826/4079709736","state":["exists","up"]},{"osd":3,"uuid":"bfe4c8b4-694a-465c-87e6-2b2aab852613","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":37,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6813","nonce":1709538759},{"type":"v1","addr":"172.21.15.43:6816","nonce":1709538759}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6819","nonce":1709538759},{"type":"v1","addr":"172.21.15.43:6822","nonce":1709538759}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6828","nonce":1709538759},{"type":"v1","addr":"172.21.15.43:6830","nonce":1709538759}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6825","nonce":1709538759},{"type":"v1","addr":"172.21.15.43:6827","nonce":1709538759}]},"public_addr":"172.21.15.43:6816/1709538759","cluster_addr":"172.21.15.43:6822/1709538759","heartbeat_back_addr":"172.21.15.43:6830/1709538759","heartbeat_front_addr":"172.21.15.43:6827/1709538759","state":["exists","up"]},{"osd":4,"uuid":"fa2e8a77-aba1-4756-853e-0d174a5bdeba","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":41,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6800","nonce":4196866481},{"type":"v1","addr":"172.21.15.203:6801","nonce":4196866481}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6802","nonce":4196866481},{"type":"v1","addr":"172.21.15.203:6803","nonce":4196866481}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6806","nonce":4196866481},{"type":"v1","addr":"172.21.15.203:6807","nonce":4196866481}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6804","nonce":4196866481},{"type":"v1","addr":"172.21.15.203:6805","nonce":4196866481}]},"public_addr":"172.21.15.203:6801/4196866481","cluster_addr":"172.21.15.203:6803/4196866481","heartbeat_back_addr":"172.21.15.203:6807/4196866481","heartbeat_front_addr":"172.21.15.203:6805/4196866481","state":["exists","up"]},{"osd":5,"uuid":"81811e4b-4d24-4f51-a82c-f1d700a2a00d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":41,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6816","nonce":3059290081},{"type":"v1","addr":"172.21.15.203:6817","nonce":3059290081}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6818","nonce":3059290081},{"type":"v1","addr":"172.21.15.203:6819","nonce":3059290081}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6822","nonce":3059290081},{"type":"v1","addr":"172.21.15.203:6823","nonce":3059290081}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6820","nonce":3059290081},{"type":"v1","addr":"172.21.15.203:6821","nonce":3059290081}]},"public_addr":"172.21.15.203:6817/3059290081","cluster_addr":"172.21.15.203:6819/3059290081","heartbeat_back_addr":"172.21.15.203:6823/3059290081","heartbeat_front_addr":"172.21.15.203:6821/3059290081","state":["exists","up"]},{"osd":6,"uuid":"307bac4d-6a5b-443d-af75-56d695d7674f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":37,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6824","nonce":512155569},{"type":"v1","addr":"172.21.15.203:6825","nonce":512155569}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6826","nonce":512155569},{"type":"v1","addr":"172.21.15.203:6827","nonce":512155569}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6830","nonce":512155569},{"type":"v1","addr":"172.21.15.203:6831","nonce":512155569}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6828","nonce":512155569},{"type":"v1","addr":"172.21.15.203:6829","nonce":512155569}]},"public_addr":"172.21.15.203:6825/512155569","cluster_addr":"172.21.15.203:6827/512155569","heartbeat_back_addr":"172.21.15.203:6831/512155569","heartbeat_front_addr":"172.21.15.203:6829/512155569","state":["exists","up"]},{"osd":7,"uuid":"a95dc7ca-9f1b-4295-af70-68a22a2d7d5e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":37,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6808","nonce":3331450438},{"type":"v1","addr":"172.21.15.203:6809","nonce":3331450438}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6810","nonce":3331450438},{"type":"v1","addr":"172.21.15.203:6811","nonce":3331450438}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6814","nonce":3331450438},{"type":"v1","addr":"172.21.15.203:6815","nonce":3331450438}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6812","nonce":3331450438},{"type":"v1","addr":"172.21.15.203:6813","nonce":3331450438}]},"public_addr":"172.21.15.203:6809/3331450438","cluster_addr":"172.21.15.203:6811/3331450438","heartbeat_back_addr":"172.21.15.203:6815/3331450438","heartbeat_front_addr":"172.21.15.203:6813/3331450438","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-09T13:33:50.547652+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-09T13:33:50.589469+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-09T13:33:50.561724+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-09T13:33:50.599650+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-09T13:33:50.411855+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-09T13:33:50.219681+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-09T13:33:50.504664+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-09T13:33:50.441248+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.43:6835/2019295969":"2023-08-10T13:48:02.673860+0000","172.21.15.203:0/830217892":"2023-08-10T13:47:57.215872+0000","172.21.15.203:0/1493361813":"2023-08-10T13:47:57.215872+0000","172.21.15.203:0/3309257000":"2023-08-10T13:36:27.970754+0000","172.21.15.203:0/2444066336":"2023-08-10T13:47:57.215872+0000","172.21.15.203:6832/830097474":"2023-08-10T13:36:27.970754+0000","172.21.15.203:6833/830097474":"2023-08-10T13:36:27.970754+0000","172.21.15.203:0/2439593041":"2023-08-10T13:36:27.970754+0000","172.21.15.203:0/2454266731":"2023-08-10T13:47:57.215872+0000","172.21.15.43:6833/2019295969":"2023-08-10T13:48:02.673860+0000","172.21.15.203:0/1761968483":"2023-08-10T13:36:27.970754+0000","172.21.15.203:0/890377927":"2023-08-10T13:36:27.970754+0000","172.21.15.203:0/560220444":"2023-08-10T13:47:57.215872+0000","172.21.15.203:6832/998005540":"2023-08-10T13:47:57.215872+0000","172.21.15.203:6833/998005540":"2023-08-10T13:47:57.215872+0000","172.21.15.203:0/315780925":"2023-08-10T13:47:57.215872+0000","172.21.15.203:0/1813604691":"2023-08-10T13:47:57.215872+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":7,"snaps":[{"begin":104,"length":1},{"begin":221,"length":1}]}],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-08-09T13:48:09.514 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:09.511+0000 7f4ff0ce7640 1 -- 172.21.15.43:0/142132462 >> [v2:172.21.15.203:6832/272917756,v1:172.21.15.203:6833/272917756] conn(0x7f4fcc06a1a0 msgr2=0x7f4fcc06c660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:48:09.514 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:09.511+0000 7f4ff0ce7640 1 --2- 172.21.15.43:0/142132462 >> [v2:172.21.15.203:6832/272917756,v1:172.21.15.203:6833/272917756] conn(0x7f4fcc06a1a0 0x7f4fcc06c660 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7f4fdc005b10 tx=0x7f4fdc009040 comp rx=0 tx=0).stop 2023-08-09T13:48:09.514 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:09.511+0000 7f4ff0ce7640 1 -- 172.21.15.43:0/142132462 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f4fec0cad10 msgr2=0x7f4fec0b0d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:48:09.515 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:09.511+0000 7f4ff0ce7640 1 --2- 172.21.15.43:0/142132462 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f4fec0cad10 0x7f4fec0b0d80 secure :-1 s=READY pgs=808 cs=0 l=1 rev1=1 crypto rx=0x7f4fd4004650 tx=0x7f4fd400ec70 comp rx=0 tx=0).stop 2023-08-09T13:48:09.515 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:09.512+0000 7f4ff0ce7640 1 -- 172.21.15.43:0/142132462 shutdown_connections 2023-08-09T13:48:09.515 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:09.512+0000 7f4ff0ce7640 1 --2- 172.21.15.43:0/142132462 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f4fec14d930 0x7f4fec0b1710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:48:09.515 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:09.512+0000 7f4ff0ce7640 1 --2- 172.21.15.43:0/142132462 >> [v2:172.21.15.203:6832/272917756,v1:172.21.15.203:6833/272917756] conn(0x7f4fcc06a1a0 0x7f4fcc06c660 unknown :-1 s=CLOSED pgs=41 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:48:09.515 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:09.512+0000 7f4ff0ce7640 1 --2- 172.21.15.43:0/142132462 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f4fec0cad10 0x7f4fec0b0d80 unknown :-1 s=CLOSED pgs=808 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:48:09.515 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:09.512+0000 7f4ff0ce7640 1 --2- 172.21.15.43:0/142132462 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f4fd8005180 0x7f4fec0c7c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:48:09.515 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:09.512+0000 7f4ff0ce7640 1 -- 172.21.15.43:0/142132462 >> 172.21.15.43:0/142132462 conn(0x7f4fec0ce0a0 msgr2=0x7f4fec15a9d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-09T13:48:09.516 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:09.512+0000 7f4ff0ce7640 1 -- 172.21.15.43:0/142132462 shutdown_connections 2023-08-09T13:48:09.516 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:09.512+0000 7f4ff0ce7640 1 -- 172.21.15.43:0/142132462 wait complete. 2023-08-09T13:48:09.534 INFO:tasks.cephfs.cephfs_test_case.ceph_manager:[{'pool': 1, 'pool_name': 'rbd', 'create_time': '2023-08-09T13:33:54.365137+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}, 'read_balance': {'score_acting': 2, 'score_stable': 2, 'optimal_score': 1, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': '.mgr', 'create_time': '2023-08-09T13:33:54.831803+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-08-09T13:48:09.534 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get rbd pg_num 2023-08-09T13:48:09.767 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:09.764+0000 7f3723c3f640 1 Processor -- start 2023-08-09T13:48:09.768 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:09.764+0000 7f3723c3f640 1 -- start start 2023-08-09T13:48:09.769 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:09.766+0000 7f3723c3f640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f371c159470 0x7f371c0c9420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:48:09.769 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:09.766+0000 7f3723c3f640 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7f371c0cc8b0 0x7f371c0ccd40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:48:09.769 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:09.766+0000 7f3723c3f640 1 -- --> v1:172.21.15.203:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f371c0c9d40 con 0x7f371c15da80 2023-08-09T13:48:09.769 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:09.766+0000 7f3723c3f640 1 -- --> v2:172.21.15.43:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f371c070990 con 0x7f371c159470 2023-08-09T13:48:09.769 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:09.766+0000 7f3723c3f640 1 -- --> v2:172.21.15.203:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f371c0ca020 con 0x7f371c0cc8b0 2023-08-09T13:48:09.770 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:09.767+0000 7f37219db640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f371c159470 0x7f371c0c9420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:48:09.770 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:09.767+0000 7f37219db640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f371c159470 0x7f371c0c9420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:35138/0 (socket says 172.21.15.43:35138) 2023-08-09T13:48:09.770 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:09.767+0000 7f37219db640 1 -- 172.21.15.43:0/2591331512 learned_addr learned my addr 172.21.15.43:0/2591331512 (peer_addr_for_me v2:172.21.15.43:0/0) 2023-08-09T13:48:09.770 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:09.767+0000 7f37219db640 1 -- 172.21.15.43:0/2591331512 >> v2:172.21.15.203:3300/0 conn(0x7f371c0cc8b0 msgr2=0x7f371c0ccd40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-09T13:48:09.771 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:09.767+0000 7f37219db640 1 --2- 172.21.15.43:0/2591331512 >> v2:172.21.15.203:3300/0 conn(0x7f371c0cc8b0 0x7f371c0ccd40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:48:09.771 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:09.767+0000 7f37219db640 1 -- 172.21.15.43:0/2591331512 >> v1:172.21.15.203:6789/0 conn(0x7f371c15da80 legacy=0x7f371c070db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-09T13:48:09.771 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:09.767+0000 7f37219db640 1 -- 172.21.15.43:0/2591331512 --> v2:172.21.15.43:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f371c0705b0 con 0x7f371c159470 2023-08-09T13:48:09.772 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:09.768+0000 7f37219db640 1 --2- 172.21.15.43:0/2591331512 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f371c159470 0x7f371c0c9420 secure :-1 s=READY pgs=809 cs=0 l=1 rev1=1 crypto rx=0x7f371c0c5010 tx=0x7f371802b910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a667f6884d379331 server_cookie=0 in_seq=0 out_seq=0 2023-08-09T13:48:09.772 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:09.769+0000 7f37209d9640 1 -- 172.21.15.43:0/2591331512 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3718031870 con 0x7f371c159470 2023-08-09T13:48:09.772 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:09.769+0000 7f37209d9640 1 -- 172.21.15.43:0/2591331512 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f37180319d0 con 0x7f371c159470 2023-08-09T13:48:09.772 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:09.769+0000 7f37209d9640 1 -- 172.21.15.43:0/2591331512 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3718031d60 con 0x7f371c159470 2023-08-09T13:48:09.772 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:09.769+0000 7f3723c3f640 1 -- 172.21.15.43:0/2591331512 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f371c159470 msgr2=0x7f371c0c9420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:48:09.773 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:09.769+0000 7f3723c3f640 1 --2- 172.21.15.43:0/2591331512 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f371c159470 0x7f371c0c9420 secure :-1 s=READY pgs=809 cs=0 l=1 rev1=1 crypto rx=0x7f371c0c5010 tx=0x7f371802b910 comp rx=0 tx=0).stop 2023-08-09T13:48:09.773 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:09.769+0000 7f3723c3f640 1 -- 172.21.15.43:0/2591331512 shutdown_connections 2023-08-09T13:48:09.773 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:09.769+0000 7f3723c3f640 1 --2- 172.21.15.43:0/2591331512 >> v2:172.21.15.203:3300/0 conn(0x7f371c0cc8b0 0x7f371c0ccd40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:48:09.773 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:09.769+0000 7f3723c3f640 1 --2- 172.21.15.43:0/2591331512 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f371c159470 0x7f371c0c9420 unknown :-1 s=CLOSED pgs=809 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:48:09.773 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:09.769+0000 7f3723c3f640 1 -- 172.21.15.43:0/2591331512 >> 172.21.15.43:0/2591331512 conn(0x7f371c0aff50 msgr2=0x7f371c0b0360 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-09T13:48:09.773 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:09.769+0000 7f3723c3f640 1 -- 172.21.15.43:0/2591331512 shutdown_connections 2023-08-09T13:48:09.773 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:09.769+0000 7f3723c3f640 1 -- 172.21.15.43:0/2591331512 wait complete. 2023-08-09T13:48:09.774 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:09.771+0000 7f3723c3f640 1 Processor -- start 2023-08-09T13:48:09.774 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:09.771+0000 7f3723c3f640 1 -- start start 2023-08-09T13:48:09.775 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:09.772+0000 7f3723c3f640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f371c0cc8b0 0x7f371c1497c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:48:09.775 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:09.772+0000 7f3723c3f640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f371c159470 0x7f371c0ae530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:48:09.775 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:09.772+0000 7f3723c3f640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f371c15da80 0x7f371c0aea70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:48:09.775 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:09.772+0000 7f3723c3f640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f371c0c75e0 con 0x7f371c159470 2023-08-09T13:48:09.775 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:09.772+0000 7f3723c3f640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f371c0c78c0 con 0x7f371c15da80 2023-08-09T13:48:09.775 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:09.772+0000 7f3723c3f640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f371c0c7ba0 con 0x7f371c0cc8b0 2023-08-09T13:48:09.777 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:09.773+0000 7f37211da640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f371c159470 0x7f371c0ae530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:48:09.777 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:09.773+0000 7f37211da640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f371c159470 0x7f371c0ae530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:35140/0 (socket says 172.21.15.43:35140) 2023-08-09T13:48:09.777 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:09.773+0000 7f37211da640 1 -- 172.21.15.43:0/1658043450 learned_addr learned my addr 172.21.15.43:0/1658043450 (peer_addr_for_me v2:172.21.15.43:0/0) 2023-08-09T13:48:09.777 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:09.773+0000 7f37219db640 1 --2- 172.21.15.43:0/1658043450 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f371c0cc8b0 0x7f371c1497c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:48:09.777 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:09.773+0000 7f37221dc640 1 --2- 172.21.15.43:0/1658043450 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f371c15da80 0x7f371c0aea70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:48:09.777 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:09.773+0000 7f37211da640 1 -- 172.21.15.43:0/1658043450 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f371c0cc8b0 msgr2=0x7f371c1497c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:48:09.777 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:09.773+0000 7f37211da640 1 --2- 172.21.15.43:0/1658043450 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f371c0cc8b0 0x7f371c1497c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:48:09.778 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:09.773+0000 7f37211da640 1 -- 172.21.15.43:0/1658043450 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f371c15da80 msgr2=0x7f371c0aea70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:48:09.778 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:09.773+0000 7f37211da640 1 --2- 172.21.15.43:0/1658043450 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f371c15da80 0x7f371c0aea70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:48:09.778 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:09.773+0000 7f37211da640 1 -- 172.21.15.43:0/1658043450 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f37180094d0 con 0x7f371c159470 2023-08-09T13:48:09.778 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:09.774+0000 7f37211da640 1 --2- 172.21.15.43:0/1658043450 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f371c159470 0x7f371c0ae530 secure :-1 s=READY pgs=810 cs=0 l=1 rev1=1 crypto rx=0x7f370c00b3e0 tx=0x7f370c00b8a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-09T13:48:09.778 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:09.774+0000 7f37127fc640 1 -- 172.21.15.43:0/1658043450 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f370c011830 con 0x7f371c159470 2023-08-09T13:48:09.778 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:09.774+0000 7f37127fc640 1 -- 172.21.15.43:0/1658043450 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f370c0119b0 con 0x7f371c159470 2023-08-09T13:48:09.778 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:09.774+0000 7f37127fc640 1 -- 172.21.15.43:0/1658043450 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f370c011d60 con 0x7f371c159470 2023-08-09T13:48:09.779 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:09.774+0000 7f3723c3f640 1 -- 172.21.15.43:0/1658043450 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f371c0c7e80 con 0x7f371c159470 2023-08-09T13:48:09.779 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:09.774+0000 7f3723c3f640 1 -- 172.21.15.43:0/1658043450 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f371c0c7200 con 0x7f371c159470 2023-08-09T13:48:09.779 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:09.775+0000 7f3723c3f640 1 -- 172.21.15.43:0/1658043450 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f371c0c83a0 con 0x7f371c159470 2023-08-09T13:48:09.779 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:09.776+0000 7f37127fc640 1 -- 172.21.15.43:0/1658043450 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 18) v1 ==== 88443+0+0 (secure 0 0 0) 0x7f370c01a400 con 0x7f371c159470 2023-08-09T13:48:09.780 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:09.777+0000 7f37127fc640 1 --2- 172.21.15.43:0/1658043450 >> [v2:172.21.15.203:6832/272917756,v1:172.21.15.203:6833/272917756] conn(0x7f370006a170 0x7f370006c630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:48:09.780 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:09.777+0000 7f37127fc640 1 -- 172.21.15.43:0/1658043450 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(66..66 src has 1..66) v4 ==== 6305+0+0 (secure 0 0 0) 0x7f370c088a30 con 0x7f371c159470 2023-08-09T13:48:09.780 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:09.778+0000 7f37219db640 1 --2- 172.21.15.43:0/1658043450 >> [v2:172.21.15.203:6832/272917756,v1:172.21.15.203:6833/272917756] conn(0x7f370006a170 0x7f370006c630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:48:09.781 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:09.779+0000 7f37219db640 1 --2- 172.21.15.43:0/1658043450 >> [v2:172.21.15.203:6832/272917756,v1:172.21.15.203:6833/272917756] conn(0x7f370006a170 0x7f370006c630 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7f3718036000 tx=0x7f371803c040 comp rx=0 tx=0).ready entity=mgr.5533 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-09T13:48:09.784 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:09.781+0000 7f37127fc640 1 -- 172.21.15.43:0/1658043450 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178066 (secure 0 0 0) 0x7f371c0c83a0 con 0x7f371c159470 2023-08-09T13:48:09.976 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:09.973+0000 7f3723c3f640 1 -- 172.21.15.43:0/1658043450 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"} v 0) v1 -- 0x7f371c0c83a0 con 0x7f371c159470 2023-08-09T13:48:09.977 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:09.975+0000 7f37127fc640 1 -- 172.21.15.43:0/1658043450 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"}]=0 v66) v1 ==== 92+0+10 (secure 0 0 0) 0x7f371c0c83a0 con 0x7f371c159470 2023-08-09T13:48:09.977 INFO:teuthology.orchestra.run.smithi043.stdout:pg_num: 8 2023-08-09T13:48:09.980 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:09.977+0000 7f3723c3f640 1 -- 172.21.15.43:0/1658043450 >> [v2:172.21.15.203:6832/272917756,v1:172.21.15.203:6833/272917756] conn(0x7f370006a170 msgr2=0x7f370006c630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:48:09.980 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:09.977+0000 7f3723c3f640 1 --2- 172.21.15.43:0/1658043450 >> [v2:172.21.15.203:6832/272917756,v1:172.21.15.203:6833/272917756] conn(0x7f370006a170 0x7f370006c630 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7f3718036000 tx=0x7f371803c040 comp rx=0 tx=0).stop 2023-08-09T13:48:09.981 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:09.977+0000 7f3723c3f640 1 -- 172.21.15.43:0/1658043450 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f371c159470 msgr2=0x7f371c0ae530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:48:09.981 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:09.977+0000 7f3723c3f640 1 --2- 172.21.15.43:0/1658043450 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f371c159470 0x7f371c0ae530 secure :-1 s=READY pgs=810 cs=0 l=1 rev1=1 crypto rx=0x7f370c00b3e0 tx=0x7f370c00b8a0 comp rx=0 tx=0).stop 2023-08-09T13:48:09.981 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:09.978+0000 7f3723c3f640 1 -- 172.21.15.43:0/1658043450 shutdown_connections 2023-08-09T13:48:09.981 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:09.978+0000 7f3723c3f640 1 --2- 172.21.15.43:0/1658043450 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f371c15da80 0x7f371c0aea70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:48:09.981 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:09.978+0000 7f3723c3f640 1 --2- 172.21.15.43:0/1658043450 >> [v2:172.21.15.203:6832/272917756,v1:172.21.15.203:6833/272917756] conn(0x7f370006a170 0x7f370006c630 unknown :-1 s=CLOSED pgs=42 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:48:09.981 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:09.978+0000 7f3723c3f640 1 --2- 172.21.15.43:0/1658043450 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f371c159470 0x7f371c0ae530 unknown :-1 s=CLOSED pgs=810 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:48:09.981 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:09.978+0000 7f3723c3f640 1 --2- 172.21.15.43:0/1658043450 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f371c0cc8b0 0x7f371c1497c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:48:09.982 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:09.978+0000 7f3723c3f640 1 -- 172.21.15.43:0/1658043450 >> 172.21.15.43:0/1658043450 conn(0x7f371c0aff50 msgr2=0x7f371c0ad9b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-09T13:48:09.982 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:09.978+0000 7f3723c3f640 1 -- 172.21.15.43:0/1658043450 shutdown_connections 2023-08-09T13:48:09.982 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:09.978+0000 7f3723c3f640 1 -- 172.21.15.43:0/1658043450 wait complete. 2023-08-09T13:48:10.000 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-08-09T13:48:10.232 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:10.230+0000 7fcf3b5b5640 1 Processor -- start 2023-08-09T13:48:10.275 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:10.230+0000 7fcf3b5b5640 1 -- start start 2023-08-09T13:48:10.279 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:10.232+0000 7fcf3b5b5640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fcf3415c550 0x7fcf340cd6d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:48:10.279 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:10.232+0000 7fcf3b5b5640 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7fcf340b45f0 0x7fcf340b59d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:48:10.280 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:10.232+0000 7fcf3b5b5640 1 -- --> v1:172.21.15.203:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fcf340b7bd0 con 0x7fcf340d04b0 2023-08-09T13:48:10.280 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:10.232+0000 7fcf3b5b5640 1 -- --> v2:172.21.15.203:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fcf340d1c00 con 0x7fcf340b45f0 2023-08-09T13:48:10.280 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:10.232+0000 7fcf3b5b5640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcf3415c930 con 0x7fcf3415c550 2023-08-09T13:48:10.281 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:10.233+0000 7fcf39351640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fcf3415c550 0x7fcf340cd6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:48:10.281 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:10.233+0000 7fcf39351640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fcf3415c550 0x7fcf340cd6d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.203:3301/0 says I am v2:172.21.15.43:55188/0 (socket says 172.21.15.43:55188) 2023-08-09T13:48:10.281 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:10.233+0000 7fcf39351640 1 -- 172.21.15.43:0/2116780054 learned_addr learned my addr 172.21.15.43:0/2116780054 (peer_addr_for_me v2:172.21.15.43:0/0) 2023-08-09T13:48:10.281 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:10.233+0000 7fcf39b52640 1 --2- 172.21.15.43:0/2116780054 >> v2:172.21.15.203:3300/0 conn(0x7fcf340b45f0 0x7fcf340b59d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:48:10.282 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:10.233+0000 7fcf39351640 1 -- 172.21.15.43:0/2116780054 >> v2:172.21.15.203:3300/0 conn(0x7fcf340b45f0 msgr2=0x7fcf340b59d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-09T13:48:10.282 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:10.233+0000 7fcf39351640 1 --2- 172.21.15.43:0/2116780054 >> v2:172.21.15.203:3300/0 conn(0x7fcf340b45f0 0x7fcf340b59d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:48:10.282 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:10.233+0000 7fcf39351640 1 -- 172.21.15.43:0/2116780054 >> v1:172.21.15.203:6789/0 conn(0x7fcf340d04b0 legacy=0x7fcf340b3ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:48:10.283 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:10.233+0000 7fcf39351640 1 -- 172.21.15.43:0/2116780054 --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcf340b54f0 con 0x7fcf3415c550 2023-08-09T13:48:10.283 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:10.235+0000 7fcf39351640 1 --2- 172.21.15.43:0/2116780054 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fcf3415c550 0x7fcf340cd6d0 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7fcf30009870 tx=0x7fcf3002b980 comp rx=0 tx=0).ready entity=mon.2 client_cookie=dceabc0a7a172da1 server_cookie=0 in_seq=0 out_seq=0 2023-08-09T13:48:10.283 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:10.235+0000 7fcf2bfff640 1 -- 172.21.15.43:0/2116780054 <== mon.2 v2:172.21.15.203:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcf30030840 con 0x7fcf3415c550 2023-08-09T13:48:10.283 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:10.235+0000 7fcf2bfff640 1 -- 172.21.15.43:0/2116780054 <== mon.2 v2:172.21.15.203:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcf300309a0 con 0x7fcf3415c550 2023-08-09T13:48:10.284 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:10.235+0000 7fcf2bfff640 1 -- 172.21.15.43:0/2116780054 <== mon.2 v2:172.21.15.203:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcf30030d30 con 0x7fcf3415c550 2023-08-09T13:48:10.284 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:10.236+0000 7fcf3b5b5640 1 -- 172.21.15.43:0/2116780054 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fcf3415c550 msgr2=0x7fcf340cd6d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:48:10.284 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:10.236+0000 7fcf3b5b5640 1 --2- 172.21.15.43:0/2116780054 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fcf3415c550 0x7fcf340cd6d0 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7fcf30009870 tx=0x7fcf3002b980 comp rx=0 tx=0).stop 2023-08-09T13:48:10.285 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:10.236+0000 7fcf3b5b5640 1 -- 172.21.15.43:0/2116780054 shutdown_connections 2023-08-09T13:48:10.285 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:10.236+0000 7fcf3b5b5640 1 --2- 172.21.15.43:0/2116780054 >> v2:172.21.15.203:3300/0 conn(0x7fcf340b45f0 0x7fcf340b59d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:48:10.285 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:10.236+0000 7fcf3b5b5640 1 --2- 172.21.15.43:0/2116780054 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fcf3415c550 0x7fcf340cd6d0 unknown :-1 s=CLOSED pgs=103 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:48:10.286 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:10.236+0000 7fcf3b5b5640 1 -- 172.21.15.43:0/2116780054 >> 172.21.15.43:0/2116780054 conn(0x7fcf34071520 msgr2=0x7fcf340ceee0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-09T13:48:10.286 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:10.236+0000 7fcf3b5b5640 1 -- 172.21.15.43:0/2116780054 shutdown_connections 2023-08-09T13:48:10.286 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:10.236+0000 7fcf3b5b5640 1 -- 172.21.15.43:0/2116780054 wait complete. 2023-08-09T13:48:10.286 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:10.239+0000 7fcf3b5b5640 1 Processor -- start 2023-08-09T13:48:10.286 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:10.239+0000 7fcf3b5b5640 1 -- start start 2023-08-09T13:48:10.287 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:10.240+0000 7fcf3b5b5640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fcf340b45f0 0x7fcf340cc040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:48:10.287 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:10.240+0000 7fcf3b5b5640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fcf340d04b0 0x7fcf341f12b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:48:10.287 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:10.240+0000 7fcf3b5b5640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fcf3415c550 0x7fcf340ab140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:48:10.287 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:10.241+0000 7fcf3b5b5640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcf340cc580 con 0x7fcf3415c550 2023-08-09T13:48:10.287 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:10.241+0000 7fcf3b5b5640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcf340a8dc0 con 0x7fcf340d04b0 2023-08-09T13:48:10.288 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:10.241+0000 7fcf3b5b5640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcf34151e90 con 0x7fcf340b45f0 2023-08-09T13:48:10.288 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:10.241+0000 7fcf39b52640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fcf3415c550 0x7fcf340ab140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:48:10.288 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:10.241+0000 7fcf39b52640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fcf3415c550 0x7fcf340ab140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:35150/0 (socket says 172.21.15.43:35150) 2023-08-09T13:48:10.289 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:10.241+0000 7fcf39b52640 1 -- 172.21.15.43:0/3021576699 learned_addr learned my addr 172.21.15.43:0/3021576699 (peer_addr_for_me v2:172.21.15.43:0/0) 2023-08-09T13:48:10.289 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:10.241+0000 7fcf39351640 1 --2- 172.21.15.43:0/3021576699 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fcf340b45f0 0x7fcf340cc040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:48:10.289 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:10.241+0000 7fcf38b50640 1 --2- 172.21.15.43:0/3021576699 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fcf340d04b0 0x7fcf341f12b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:48:10.289 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:10.241+0000 7fcf39b52640 1 -- 172.21.15.43:0/3021576699 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fcf340b45f0 msgr2=0x7fcf340cc040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:48:10.289 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:10.241+0000 7fcf39b52640 1 --2- 172.21.15.43:0/3021576699 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fcf340b45f0 0x7fcf340cc040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:48:10.290 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:10.241+0000 7fcf39b52640 1 -- 172.21.15.43:0/3021576699 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fcf340d04b0 msgr2=0x7fcf341f12b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:48:10.290 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:10.241+0000 7fcf39b52640 1 --2- 172.21.15.43:0/3021576699 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fcf340d04b0 0x7fcf341f12b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:48:10.290 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:10.242+0000 7fcf39b52640 1 -- 172.21.15.43:0/3021576699 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcf300094d0 con 0x7fcf3415c550 2023-08-09T13:48:10.290 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:10.242+0000 7fcf39b52640 1 --2- 172.21.15.43:0/3021576699 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fcf3415c550 0x7fcf340ab140 secure :-1 s=READY pgs=811 cs=0 l=1 rev1=1 crypto rx=0x7fcf2c007e30 tx=0x7fcf2c00e4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-09T13:48:10.290 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:10.242+0000 7fcf29ffb640 1 -- 172.21.15.43:0/3021576699 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcf2c014620 con 0x7fcf3415c550 2023-08-09T13:48:10.290 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:10.242+0000 7fcf29ffb640 1 -- 172.21.15.43:0/3021576699 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcf2c014780 con 0x7fcf3415c550 2023-08-09T13:48:10.290 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:10.242+0000 7fcf3b5b5640 1 -- 172.21.15.43:0/3021576699 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcf341f17f0 con 0x7fcf3415c550 2023-08-09T13:48:10.291 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:10.242+0000 7fcf29ffb640 1 -- 172.21.15.43:0/3021576699 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcf2c014b30 con 0x7fcf3415c550 2023-08-09T13:48:10.291 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:10.243+0000 7fcf3b5b5640 1 -- 172.21.15.43:0/3021576699 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcf340ca750 con 0x7fcf3415c550 2023-08-09T13:48:10.291 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:10.244+0000 7fcf3b5b5640 1 -- 172.21.15.43:0/3021576699 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcf340739f0 con 0x7fcf3415c550 2023-08-09T13:48:10.291 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:10.245+0000 7fcf29ffb640 1 -- 172.21.15.43:0/3021576699 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 18) v1 ==== 88443+0+0 (secure 0 0 0) 0x7fcf2c014c90 con 0x7fcf3415c550 2023-08-09T13:48:10.291 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:10.246+0000 7fcf29ffb640 1 --2- 172.21.15.43:0/3021576699 >> [v2:172.21.15.203:6832/272917756,v1:172.21.15.203:6833/272917756] conn(0x7fcf1806a170 0x7fcf1806c630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:48:10.291 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:10.246+0000 7fcf29ffb640 1 -- 172.21.15.43:0/3021576699 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(66..66 src has 1..66) v4 ==== 6305+0+0 (secure 0 0 0) 0x7fcf2c089200 con 0x7fcf3415c550 2023-08-09T13:48:10.291 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:10.247+0000 7fcf39351640 1 --2- 172.21.15.43:0/3021576699 >> [v2:172.21.15.203:6832/272917756,v1:172.21.15.203:6833/272917756] conn(0x7fcf1806a170 0x7fcf1806c630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:48:10.291 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:10.248+0000 7fcf39351640 1 --2- 172.21.15.43:0/3021576699 >> [v2:172.21.15.203:6832/272917756,v1:172.21.15.203:6833/272917756] conn(0x7fcf1806a170 0x7fcf1806c630 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7fcf30036000 tx=0x7fcf3003b040 comp rx=0 tx=0).ready entity=mgr.5533 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-09T13:48:10.292 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:10.251+0000 7fcf29ffb640 1 -- 172.21.15.43:0/3021576699 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178066 (secure 0 0 0) 0x7fcf2c054650 con 0x7fcf3415c550 2023-08-09T13:48:10.447 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:10.445+0000 7fcf3b5b5640 1 -- 172.21.15.43:0/3021576699 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7fcf340a8f00 con 0x7fcf3415c550 2023-08-09T13:48:10.448 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:10.446+0000 7fcf29ffb640 1 -- 172.21.15.43:0/3021576699 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v66) v1 ==== 93+0+10 (secure 0 0 0) 0x7fcf2c057c70 con 0x7fcf3415c550 2023-08-09T13:48:10.448 INFO:teuthology.orchestra.run.smithi043.stdout:pg_num: 1 2023-08-09T13:48:10.451 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:10.448+0000 7fcf3b5b5640 1 -- 172.21.15.43:0/3021576699 >> [v2:172.21.15.203:6832/272917756,v1:172.21.15.203:6833/272917756] conn(0x7fcf1806a170 msgr2=0x7fcf1806c630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:48:10.451 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:10.448+0000 7fcf3b5b5640 1 --2- 172.21.15.43:0/3021576699 >> [v2:172.21.15.203:6832/272917756,v1:172.21.15.203:6833/272917756] conn(0x7fcf1806a170 0x7fcf1806c630 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7fcf30036000 tx=0x7fcf3003b040 comp rx=0 tx=0).stop 2023-08-09T13:48:10.451 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:10.448+0000 7fcf3b5b5640 1 -- 172.21.15.43:0/3021576699 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fcf3415c550 msgr2=0x7fcf340ab140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:48:10.451 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:10.448+0000 7fcf3b5b5640 1 --2- 172.21.15.43:0/3021576699 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fcf3415c550 0x7fcf340ab140 secure :-1 s=READY pgs=811 cs=0 l=1 rev1=1 crypto rx=0x7fcf2c007e30 tx=0x7fcf2c00e4b0 comp rx=0 tx=0).stop 2023-08-09T13:48:10.452 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:10.449+0000 7fcf3b5b5640 1 -- 172.21.15.43:0/3021576699 shutdown_connections 2023-08-09T13:48:10.452 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:10.449+0000 7fcf3b5b5640 1 --2- 172.21.15.43:0/3021576699 >> [v2:172.21.15.203:6832/272917756,v1:172.21.15.203:6833/272917756] conn(0x7fcf1806a170 0x7fcf1806c630 unknown :-1 s=CLOSED pgs=43 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:48:10.452 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:10.449+0000 7fcf3b5b5640 1 --2- 172.21.15.43:0/3021576699 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fcf3415c550 0x7fcf340ab140 unknown :-1 s=CLOSED pgs=811 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:48:10.452 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:10.449+0000 7fcf3b5b5640 1 --2- 172.21.15.43:0/3021576699 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fcf340d04b0 0x7fcf341f12b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:48:10.452 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:10.449+0000 7fcf3b5b5640 1 --2- 172.21.15.43:0/3021576699 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fcf340b45f0 0x7fcf340cc040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:48:10.453 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:10.449+0000 7fcf3b5b5640 1 -- 172.21.15.43:0/3021576699 >> 172.21.15.43:0/3021576699 conn(0x7fcf34071520 msgr2=0x7fcf3415e240 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-09T13:48:10.453 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:10.449+0000 7fcf3b5b5640 1 -- 172.21.15.43:0/3021576699 shutdown_connections 2023-08-09T13:48:10.453 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:10.449+0000 7fcf3b5b5640 1 -- 172.21.15.43:0/3021576699 wait complete. 2023-08-09T13:48:10.469 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mon mon_allow_pool_delete True 2023-08-09T13:48:10.688 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:10.686+0000 7f81bf779640 1 Processor -- start 2023-08-09T13:48:10.688 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:10.686+0000 7f81bf779640 1 -- start start 2023-08-09T13:48:10.690 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:10.687+0000 7f81bf779640 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7f81b815c680 0x7f81b80ce760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:48:10.690 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:10.688+0000 7f81bf779640 1 -- --> v1:172.21.15.43:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f81b80cdf00 con 0x7f81b80d0a70 2023-08-09T13:48:10.690 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:10.688+0000 7f81bf779640 1 -- --> v1:172.21.15.203:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f81b80d0f70 con 0x7f81b80b3fa0 2023-08-09T13:48:10.690 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:10.688+0000 7f81bf779640 1 -- --> v2:172.21.15.203:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f81b80ccd10 con 0x7f81b815c680 2023-08-09T13:48:10.691 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:10.688+0000 7f81bcd14640 1 --1- >> v1:172.21.15.43:6789/0 conn(0x7f81b80d0a70 0x7f81b80cd7f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.43:6789/0 says I am v1:172.21.15.43:41690/0 (socket says 172.21.15.43:41690) 2023-08-09T13:48:10.691 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:10.688+0000 7f81bcd14640 1 -- 172.21.15.43:0/3531410949 learned_addr learned my addr 172.21.15.43:0/3531410949 (peer_addr_for_me v1:172.21.15.43:0/0) 2023-08-09T13:48:10.691 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:10.689+0000 7f81bd515640 1 --2- 172.21.15.43:0/3531410949 >> v2:172.21.15.203:3300/0 conn(0x7f81b815c680 0x7f81b80ce760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:48:10.691 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:10.689+0000 7f81affff640 1 -- 172.21.15.43:0/3531410949 <== mon.0 v1:172.21.15.43:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 536233569 0 0) 0x7f81b80cdf00 con 0x7f81b80d0a70 2023-08-09T13:48:10.691 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:10.689+0000 7f81affff640 1 -- 172.21.15.43:0/3531410949 --> v1:172.21.15.43:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f81a4003520 con 0x7f81b80d0a70 2023-08-09T13:48:10.693 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:10.689+0000 7f81affff640 1 -- 172.21.15.43:0/3531410949 <== mon.0 v1:172.21.15.43:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 844316199 0 0) 0x7f81a4003520 con 0x7f81b80d0a70 2023-08-09T13:48:10.693 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:10.689+0000 7f81affff640 1 -- 172.21.15.43:0/3531410949 >> v2:172.21.15.203:3300/0 conn(0x7f81b815c680 msgr2=0x7f81b80ce760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-09T13:48:10.693 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:10.689+0000 7f81affff640 1 --2- 172.21.15.43:0/3531410949 >> v2:172.21.15.203:3300/0 conn(0x7f81b815c680 0x7f81b80ce760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:48:10.693 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:10.689+0000 7f81affff640 1 -- 172.21.15.43:0/3531410949 >> v1:172.21.15.203:6789/0 conn(0x7f81b80b3fa0 legacy=0x7f81b80b43b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:48:10.694 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:10.689+0000 7f81affff640 1 -- 172.21.15.43:0/3531410949 --> v1:172.21.15.43:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f81b80ccff0 con 0x7f81b80d0a70 2023-08-09T13:48:10.694 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:10.689+0000 7f81affff640 1 -- 172.21.15.43:0/3531410949 <== mon.0 v1:172.21.15.43:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1285150740 0 0) 0x7f81a8001670 con 0x7f81b80d0a70 2023-08-09T13:48:10.694 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:10.689+0000 7f81affff640 1 -- 172.21.15.43:0/3531410949 >> v1:172.21.15.43:6789/0 conn(0x7f81b80d0a70 legacy=0x7f81b80cd7f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:48:10.694 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:10.689+0000 7f81affff640 1 --2- 172.21.15.43:0/3531410949 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f81a4003a30 0x7f81a4003e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:48:10.694 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:10.689+0000 7f81affff640 1 --2- 172.21.15.43:0/3531410949 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f81a40047c0 0x7f81a4004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:48:10.694 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:10.689+0000 7f81affff640 1 --2- 172.21.15.43:0/3531410949 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f81a40051d0 0x7f81a400b980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:48:10.694 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:10.689+0000 7f81affff640 1 -- 172.21.15.43:0/3531410949 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f81b80ccd10 con 0x7f81a40051d0 2023-08-09T13:48:10.695 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:10.689+0000 7f81affff640 1 -- 172.21.15.43:0/3531410949 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f81a400c100 con 0x7f81a40047c0 2023-08-09T13:48:10.695 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:10.689+0000 7f81affff640 1 -- 172.21.15.43:0/3531410949 --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f81a400c2c0 con 0x7f81a4003a30 2023-08-09T13:48:10.695 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:10.689+0000 7f81bdd16640 1 --2- 172.21.15.43:0/3531410949 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f81a40051d0 0x7f81a400b980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:48:10.695 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:10.690+0000 7f81bd515640 1 --2- 172.21.15.43:0/3531410949 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f81a4003a30 0x7f81a4003e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:48:10.695 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:10.690+0000 7f81bdd16640 1 -- 172.21.15.43:0/3531410949 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f81a4003a30 msgr2=0x7f81a4003e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-09T13:48:10.695 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:10.690+0000 7f81bcd14640 1 --2- 172.21.15.43:0/3531410949 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f81a40047c0 0x7f81a4004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:48:10.695 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:10.690+0000 7f81bdd16640 1 --2- 172.21.15.43:0/3531410949 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f81a4003a30 0x7f81a4003e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:48:10.695 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:10.690+0000 7f81bdd16640 1 -- 172.21.15.43:0/3531410949 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f81a40047c0 msgr2=0x7f81a4004c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-09T13:48:10.696 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:10.690+0000 7f81bdd16640 1 --2- 172.21.15.43:0/3531410949 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f81a40047c0 0x7f81a4004c40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:48:10.696 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:10.690+0000 7f81bdd16640 1 -- 172.21.15.43:0/3531410949 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f81a4003520 con 0x7f81a40051d0 2023-08-09T13:48:10.697 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:10.691+0000 7f81bdd16640 1 -- 172.21.15.43:0/3531410949 reap_dead start 2023-08-09T13:48:10.697 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:10.691+0000 7f81bdd16640 1 --2- 172.21.15.43:0/3531410949 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f81a40051d0 0x7f81a400b980 secure :-1 s=READY pgs=813 cs=0 l=1 rev1=1 crypto rx=0x7f81b0001750 tx=0x7f81a40057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=999247e9bf17f0b6 server_cookie=0 in_seq=0 out_seq=0 2023-08-09T13:48:10.697 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:10.691+0000 7f81affff640 1 -- 172.21.15.43:0/3531410949 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f81b00052d0 con 0x7f81a40051d0 2023-08-09T13:48:10.697 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:10.691+0000 7f81affff640 1 -- 172.21.15.43:0/3531410949 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f81b002e050 con 0x7f81a40051d0 2023-08-09T13:48:10.697 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:10.691+0000 7f81affff640 1 -- 172.21.15.43:0/3531410949 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f81b0003830 con 0x7f81a40051d0 2023-08-09T13:48:10.698 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:10.692+0000 7f81bf779640 1 -- 172.21.15.43:0/3531410949 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f81a40051d0 msgr2=0x7f81a400b980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:48:10.698 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:10.692+0000 7f81bf779640 1 --2- 172.21.15.43:0/3531410949 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f81a40051d0 0x7f81a400b980 secure :-1 s=READY pgs=813 cs=0 l=1 rev1=1 crypto rx=0x7f81b0001750 tx=0x7f81a40057f0 comp rx=0 tx=0).stop 2023-08-09T13:48:10.698 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:10.692+0000 7f81bf779640 1 -- 172.21.15.43:0/3531410949 shutdown_connections 2023-08-09T13:48:10.698 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:10.692+0000 7f81bf779640 1 --2- 172.21.15.43:0/3531410949 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f81a40051d0 0x7f81a400b980 unknown :-1 s=CLOSED pgs=813 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:48:10.698 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:10.692+0000 7f81bf779640 1 -- 172.21.15.43:0/3531410949 >> 172.21.15.43:0/3531410949 conn(0x7f81b8071520 msgr2=0x7f81b80d2520 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-09T13:48:10.698 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:10.692+0000 7f81bf779640 1 -- 172.21.15.43:0/3531410949 shutdown_connections 2023-08-09T13:48:10.698 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:10.692+0000 7f81bf779640 1 -- 172.21.15.43:0/3531410949 wait complete. 2023-08-09T13:48:10.698 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:10.694+0000 7f81bf779640 1 Processor -- start 2023-08-09T13:48:10.699 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:10.694+0000 7f81bf779640 1 -- start start 2023-08-09T13:48:10.700 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:10.695+0000 7f81bf779640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f81a40051d0 0x7f81b813f720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:48:10.700 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:10.695+0000 7f81bf779640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f81b81527f0 0x7f81b815d150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:48:10.700 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:10.695+0000 7f81bf779640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f81b8150040 0x7f81b80cb230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:48:10.700 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:10.695+0000 7f81bf779640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f81b8152c70 con 0x7f81a40051d0 2023-08-09T13:48:10.701 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:10.695+0000 7f81bf779640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f81b80cada0 con 0x7f81b8150040 2023-08-09T13:48:10.701 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:10.695+0000 7f81bf779640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f81b815d690 con 0x7f81b81527f0 2023-08-09T13:48:10.701 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:10.696+0000 7f81bd515640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f81a40051d0 0x7f81b813f720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:48:10.701 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:10.696+0000 7f81bd515640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f81a40051d0 0x7f81b813f720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:35162/0 (socket says 172.21.15.43:35162) 2023-08-09T13:48:10.701 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:10.696+0000 7f81bd515640 1 -- 172.21.15.43:0/3506788827 learned_addr learned my addr 172.21.15.43:0/3506788827 (peer_addr_for_me v2:172.21.15.43:0/0) 2023-08-09T13:48:10.701 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:10.696+0000 7f81bdd16640 1 --2- 172.21.15.43:0/3506788827 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f81b8150040 0x7f81b80cb230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:48:10.701 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:10.696+0000 7f81bcd14640 1 --2- 172.21.15.43:0/3506788827 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f81b81527f0 0x7f81b815d150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:48:10.701 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:10.696+0000 7f81bd515640 1 -- 172.21.15.43:0/3506788827 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f81b81527f0 msgr2=0x7f81b815d150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:48:10.702 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:10.696+0000 7f81bd515640 1 --2- 172.21.15.43:0/3506788827 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f81b81527f0 0x7f81b815d150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:48:10.702 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:10.696+0000 7f81bd515640 1 -- 172.21.15.43:0/3506788827 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f81b8150040 msgr2=0x7f81b80cb230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:48:10.702 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:10.696+0000 7f81bd515640 1 --2- 172.21.15.43:0/3506788827 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f81b8150040 0x7f81b80cb230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:48:10.702 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:10.696+0000 7f81bd515640 1 -- 172.21.15.43:0/3506788827 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f81b8153160 con 0x7f81a40051d0 2023-08-09T13:48:10.702 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:10.696+0000 7f81bcd14640 1 --2- 172.21.15.43:0/3506788827 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f81b81527f0 0x7f81b815d150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-08-09T13:48:10.702 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:10.696+0000 7f81bdd16640 1 --2- 172.21.15.43:0/3506788827 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f81b8150040 0x7f81b80cb230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-08-09T13:48:10.702 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:10.696+0000 7f81bd515640 1 --2- 172.21.15.43:0/3506788827 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f81a40051d0 0x7f81b813f720 secure :-1 s=READY pgs=814 cs=0 l=1 rev1=1 crypto rx=0x7f81b4007f00 tx=0x7f81b40115a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-09T13:48:10.703 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:10.697+0000 7f81adffb640 1 -- 172.21.15.43:0/3506788827 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f81b400e030 con 0x7f81a40051d0 2023-08-09T13:48:10.703 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:10.697+0000 7f81adffb640 1 -- 172.21.15.43:0/3506788827 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f81b4016470 con 0x7f81a40051d0 2023-08-09T13:48:10.703 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:10.697+0000 7f81adffb640 1 -- 172.21.15.43:0/3506788827 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f81b4016820 con 0x7f81a40051d0 2023-08-09T13:48:10.703 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:10.697+0000 7f81bf779640 1 -- 172.21.15.43:0/3506788827 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f81b8152e80 con 0x7f81a40051d0 2023-08-09T13:48:10.703 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:10.697+0000 7f81bf779640 1 -- 172.21.15.43:0/3506788827 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f81b80caa30 con 0x7f81a40051d0 2023-08-09T13:48:10.704 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:10.698+0000 7f81bf779640 1 -- 172.21.15.43:0/3506788827 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f81b80739f0 con 0x7f81a40051d0 2023-08-09T13:48:10.704 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:10.699+0000 7f81adffb640 1 -- 172.21.15.43:0/3506788827 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 18) v1 ==== 88443+0+0 (secure 0 0 0) 0x7f81b4016980 con 0x7f81a40051d0 2023-08-09T13:48:10.704 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:10.700+0000 7f81adffb640 1 --2- 172.21.15.43:0/3506788827 >> [v2:172.21.15.203:6832/272917756,v1:172.21.15.203:6833/272917756] conn(0x7f819c06a170 0x7f819c06c630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:48:10.704 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:10.700+0000 7f81adffb640 1 -- 172.21.15.43:0/3506788827 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(66..66 src has 1..66) v4 ==== 6305+0+0 (secure 0 0 0) 0x7f81b400f070 con 0x7f81a40051d0 2023-08-09T13:48:10.704 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:10.701+0000 7f81bcd14640 1 --2- 172.21.15.43:0/3506788827 >> [v2:172.21.15.203:6832/272917756,v1:172.21.15.203:6833/272917756] conn(0x7f819c06a170 0x7f819c06c630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:48:10.705 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:10.702+0000 7f81bcd14640 1 --2- 172.21.15.43:0/3506788827 >> [v2:172.21.15.203:6832/272917756,v1:172.21.15.203:6833/272917756] conn(0x7f819c06a170 0x7f819c06c630 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7f81b815d860 tx=0x7f81a8009040 comp rx=0 tx=0).ready entity=mgr.5533 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-09T13:48:10.707 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:10.704+0000 7f81adffb640 1 -- 172.21.15.43:0/3506788827 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178066 (secure 0 0 0) 0x7f81b4054c60 con 0x7f81a40051d0 2023-08-09T13:48:10.902 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:10.899+0000 7f81bf779640 1 -- 172.21.15.43:0/3506788827 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command([{prefix=config set, name=mon_allow_pool_delete}] v 0) v1 -- 0x7f81b80d14e0 con 0x7f81a40051d0 2023-08-09T13:48:10.909 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:10.906+0000 7f81adffb640 1 -- 172.21.15.43:0/3506788827 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_allow_pool_delete}]=0 v16)=0 v16) v1 ==== 122+0+0 (secure 0 0 0) 0x7f81b4058280 con 0x7f81a40051d0 2023-08-09T13:48:10.911 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:10.909+0000 7f81bf779640 1 -- 172.21.15.43:0/3506788827 >> [v2:172.21.15.203:6832/272917756,v1:172.21.15.203:6833/272917756] conn(0x7f819c06a170 msgr2=0x7f819c06c630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:48:10.912 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:10.909+0000 7f81bf779640 1 --2- 172.21.15.43:0/3506788827 >> [v2:172.21.15.203:6832/272917756,v1:172.21.15.203:6833/272917756] conn(0x7f819c06a170 0x7f819c06c630 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7f81b815d860 tx=0x7f81a8009040 comp rx=0 tx=0).stop 2023-08-09T13:48:10.912 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:10.909+0000 7f81bf779640 1 -- 172.21.15.43:0/3506788827 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f81a40051d0 msgr2=0x7f81b813f720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:48:10.912 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:10.909+0000 7f81bf779640 1 --2- 172.21.15.43:0/3506788827 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f81a40051d0 0x7f81b813f720 secure :-1 s=READY pgs=814 cs=0 l=1 rev1=1 crypto rx=0x7f81b4007f00 tx=0x7f81b40115a0 comp rx=0 tx=0).stop 2023-08-09T13:48:10.912 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:10.909+0000 7f81bf779640 1 -- 172.21.15.43:0/3506788827 shutdown_connections 2023-08-09T13:48:10.912 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:10.909+0000 7f81bf779640 1 --2- 172.21.15.43:0/3506788827 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f81b8150040 0x7f81b80cb230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:48:10.912 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:10.909+0000 7f81bf779640 1 --2- 172.21.15.43:0/3506788827 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f81b81527f0 0x7f81b815d150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:48:10.912 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:10.909+0000 7f81bf779640 1 --2- 172.21.15.43:0/3506788827 >> [v2:172.21.15.203:6832/272917756,v1:172.21.15.203:6833/272917756] conn(0x7f819c06a170 0x7f819c06c630 unknown :-1 s=CLOSED pgs=44 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:48:10.913 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:10.909+0000 7f81bf779640 1 --2- 172.21.15.43:0/3506788827 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f81a40051d0 0x7f81b813f720 unknown :-1 s=CLOSED pgs=814 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:48:10.913 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:10.909+0000 7f81bf779640 1 -- 172.21.15.43:0/3506788827 >> 172.21.15.43:0/3506788827 conn(0x7f81b8071520 msgr2=0x7f81b80d0060 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-09T13:48:10.913 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:10.909+0000 7f81bf779640 1 -- 172.21.15.43:0/3506788827 shutdown_connections 2023-08-09T13:48:10.913 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:10.909+0000 7f81bf779640 1 -- 172.21.15.43:0/3506788827 wait complete. 2023-08-09T13:48:10.928 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2023-08-09T13:48:10.929 DEBUG:teuthology.orchestra.run.smithi043:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2023-08-09T13:48:11.053 DEBUG:teuthology.orchestra.run.smithi203:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2023-08-09T13:48:11.164 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2023-08-09T13:48:11.165 DEBUG:teuthology.orchestra.run.smithi043:> set -ex 2023-08-09T13:48:11.165 DEBUG:teuthology.orchestra.run.smithi043:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-09T13:48:11.230 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4054441,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6498844k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=27,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3249420k,nr_inodes=812355,mode=700,uid=1000,gid=1275 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6498844k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2023-08-09T13:48:11.231 DEBUG:teuthology.orchestra.run.smithi203:> set -ex 2023-08-09T13:48:11.231 DEBUG:teuthology.orchestra.run.smithi203:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-09T13:48:11.289 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4053648,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497576k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=27,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248784k,nr_inodes=812196,mode=700,uid=1000,gid=1275 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497576k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2023-08-09T13:48:11.290 DEBUG:teuthology.orchestra.run.smithi043:> set -ex 2023-08-09T13:48:11.290 DEBUG:teuthology.orchestra.run.smithi043:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-09T13:48:11.354 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4054441,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6498844k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=27,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3249420k,nr_inodes=812355,mode=700,uid=1000,gid=1275 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6498844k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2023-08-09T13:48:11.355 DEBUG:teuthology.orchestra.run.smithi203:> set -ex 2023-08-09T13:48:11.355 DEBUG:teuthology.orchestra.run.smithi203:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-09T13:48:11.414 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4053648,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497576k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=27,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248784k,nr_inodes=812196,mode=700,uid=1000,gid=1275 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497576k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2023-08-09T13:48:11.414 INFO:tasks.cephfs.cephfs_test_case:[, , , ] 2023-08-09T13:48:11.415 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-08-09T13:48:11.657 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:11.654+0000 7fcc66438640 1 Processor -- start 2023-08-09T13:48:11.657 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:11.655+0000 7fcc66438640 1 -- start start 2023-08-09T13:48:11.658 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:11.656+0000 7fcc66438640 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7fcc6015c670 0x7fcc600c9a40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:48:11.659 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:11.656+0000 7fcc66438640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fcc6006c1d0 0x7fcc600ab8c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:48:11.659 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:11.656+0000 7fcc66438640 1 -- --> v1:172.21.15.203:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fcc600ca390 con 0x7fcc600abe00 2023-08-09T13:48:11.659 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:11.656+0000 7fcc66438640 1 -- --> v2:172.21.15.203:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fcc6015ca50 con 0x7fcc6015c670 2023-08-09T13:48:11.659 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:11.656+0000 7fcc66438640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcc600ca670 con 0x7fcc6006c1d0 2023-08-09T13:48:11.660 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:11.657+0000 7fcc577fe640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fcc6006c1d0 0x7fcc600ab8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:48:11.660 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:11.657+0000 7fcc577fe640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fcc6006c1d0 0x7fcc600ab8c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.203:3301/0 says I am v2:172.21.15.43:55228/0 (socket says 172.21.15.43:55228) 2023-08-09T13:48:11.660 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:11.657+0000 7fcc577fe640 1 -- 172.21.15.43:0/2206502892 learned_addr learned my addr 172.21.15.43:0/2206502892 (peer_addr_for_me v2:172.21.15.43:0/0) 2023-08-09T13:48:11.660 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:11.657+0000 7fcc5ffff640 1 --2- 172.21.15.43:0/2206502892 >> v2:172.21.15.203:3300/0 conn(0x7fcc6015c670 0x7fcc600c9a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:48:11.661 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:11.658+0000 7fcc577fe640 1 -- 172.21.15.43:0/2206502892 >> v2:172.21.15.203:3300/0 conn(0x7fcc6015c670 msgr2=0x7fcc600c9a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-09T13:48:11.661 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:11.658+0000 7fcc577fe640 1 --2- 172.21.15.43:0/2206502892 >> v2:172.21.15.203:3300/0 conn(0x7fcc6015c670 0x7fcc600c9a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:48:11.661 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:11.658+0000 7fcc577fe640 1 -- 172.21.15.43:0/2206502892 >> v1:172.21.15.203:6789/0 conn(0x7fcc600abe00 legacy=0x7fcc600cd220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:48:11.661 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:11.658+0000 7fcc577fe640 1 -- 172.21.15.43:0/2206502892 --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcc600ca950 con 0x7fcc6006c1d0 2023-08-09T13:48:11.662 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:11.660+0000 7fcc577fe640 1 --2- 172.21.15.43:0/2206502892 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fcc6006c1d0 0x7fcc600ab8c0 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7fcc48009870 tx=0x7fcc4802b980 comp rx=0 tx=0).ready entity=mon.2 client_cookie=7a0de2dde31cf39d server_cookie=0 in_seq=0 out_seq=0 2023-08-09T13:48:11.662 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:11.660+0000 7fcc5f7fe640 1 -- 172.21.15.43:0/2206502892 <== mon.2 v2:172.21.15.203:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcc48030840 con 0x7fcc6006c1d0 2023-08-09T13:48:11.663 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:11.660+0000 7fcc5f7fe640 1 -- 172.21.15.43:0/2206502892 <== mon.2 v2:172.21.15.203:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcc480309a0 con 0x7fcc6006c1d0 2023-08-09T13:48:11.663 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:11.660+0000 7fcc5f7fe640 1 -- 172.21.15.43:0/2206502892 <== mon.2 v2:172.21.15.203:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcc48030d30 con 0x7fcc6006c1d0 2023-08-09T13:48:11.663 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:11.661+0000 7fcc66438640 1 -- 172.21.15.43:0/2206502892 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fcc6006c1d0 msgr2=0x7fcc600ab8c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:48:11.664 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:11.661+0000 7fcc66438640 1 --2- 172.21.15.43:0/2206502892 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fcc6006c1d0 0x7fcc600ab8c0 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7fcc48009870 tx=0x7fcc4802b980 comp rx=0 tx=0).stop 2023-08-09T13:48:11.664 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:11.661+0000 7fcc66438640 1 -- 172.21.15.43:0/2206502892 shutdown_connections 2023-08-09T13:48:11.664 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:11.661+0000 7fcc66438640 1 --2- 172.21.15.43:0/2206502892 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fcc6006c1d0 0x7fcc600ab8c0 unknown :-1 s=CLOSED pgs=104 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:48:11.664 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:11.661+0000 7fcc66438640 1 --2- 172.21.15.43:0/2206502892 >> v2:172.21.15.203:3300/0 conn(0x7fcc6015c670 0x7fcc600c9a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:48:11.664 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:11.661+0000 7fcc66438640 1 -- 172.21.15.43:0/2206502892 >> 172.21.15.43:0/2206502892 conn(0x7fcc600697c0 msgr2=0x7fcc60069ba0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-09T13:48:11.664 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:11.661+0000 7fcc66438640 1 -- 172.21.15.43:0/2206502892 shutdown_connections 2023-08-09T13:48:11.665 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:11.661+0000 7fcc66438640 1 -- 172.21.15.43:0/2206502892 wait complete. 2023-08-09T13:48:11.665 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:11.663+0000 7fcc66438640 1 Processor -- start 2023-08-09T13:48:11.665 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:11.663+0000 7fcc66438640 1 -- start start 2023-08-09T13:48:11.666 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:11.664+0000 7fcc66438640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fcc6006c1d0 0x7fcc6015e5b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:48:11.667 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:11.664+0000 7fcc66438640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fcc600abe00 0x7fcc60144cb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:48:11.667 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:11.664+0000 7fcc66438640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fcc6015c670 0x7fcc60142200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:48:11.667 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:11.664+0000 7fcc66438640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcc60145c80 con 0x7fcc6006c1d0 2023-08-09T13:48:11.667 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:11.664+0000 7fcc66438640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcc601456c0 con 0x7fcc600abe00 2023-08-09T13:48:11.667 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:11.664+0000 7fcc66438640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcc601459a0 con 0x7fcc6015c670 2023-08-09T13:48:11.668 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:11.664+0000 7fcc5ffff640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fcc6006c1d0 0x7fcc6015e5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:48:11.669 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:11.665+0000 7fcc5ffff640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fcc6006c1d0 0x7fcc6015e5b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:35176/0 (socket says 172.21.15.43:35176) 2023-08-09T13:48:11.669 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:11.665+0000 7fcc5ffff640 1 -- 172.21.15.43:0/2362877704 learned_addr learned my addr 172.21.15.43:0/2362877704 (peer_addr_for_me v2:172.21.15.43:0/0) 2023-08-09T13:48:11.669 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:11.665+0000 7fcc649d5640 1 --2- 172.21.15.43:0/2362877704 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fcc6015c670 0x7fcc60142200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:48:11.669 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:11.665+0000 7fcc577fe640 1 --2- 172.21.15.43:0/2362877704 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fcc600abe00 0x7fcc60144cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:48:11.669 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:11.665+0000 7fcc5ffff640 1 -- 172.21.15.43:0/2362877704 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fcc6015c670 msgr2=0x7fcc60142200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:48:11.669 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:11.665+0000 7fcc5ffff640 1 --2- 172.21.15.43:0/2362877704 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fcc6015c670 0x7fcc60142200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:48:11.669 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:11.665+0000 7fcc5ffff640 1 -- 172.21.15.43:0/2362877704 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fcc600abe00 msgr2=0x7fcc60144cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:48:11.670 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:11.665+0000 7fcc5ffff640 1 --2- 172.21.15.43:0/2362877704 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fcc600abe00 0x7fcc60144cb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:48:11.670 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:11.665+0000 7fcc5ffff640 1 -- 172.21.15.43:0/2362877704 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcc480094d0 con 0x7fcc6006c1d0 2023-08-09T13:48:11.670 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:11.665+0000 7fcc577fe640 1 --2- 172.21.15.43:0/2362877704 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fcc600abe00 0x7fcc60144cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-08-09T13:48:11.670 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:11.665+0000 7fcc5ffff640 1 --2- 172.21.15.43:0/2362877704 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fcc6006c1d0 0x7fcc6015e5b0 secure :-1 s=READY pgs=815 cs=0 l=1 rev1=1 crypto rx=0x7fcc5000e870 tx=0x7fcc5000ed30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-09T13:48:11.670 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:11.666+0000 7fcc5d7fa640 1 -- 172.21.15.43:0/2362877704 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcc500094f0 con 0x7fcc6006c1d0 2023-08-09T13:48:11.670 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:11.666+0000 7fcc5d7fa640 1 -- 172.21.15.43:0/2362877704 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcc5000c940 con 0x7fcc6006c1d0 2023-08-09T13:48:11.670 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:11.666+0000 7fcc5d7fa640 1 -- 172.21.15.43:0/2362877704 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcc5000ccf0 con 0x7fcc6006c1d0 2023-08-09T13:48:11.670 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:11.666+0000 7fcc66438640 1 -- 172.21.15.43:0/2362877704 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcc6015ddb0 con 0x7fcc6006c1d0 2023-08-09T13:48:11.671 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:11.666+0000 7fcc66438640 1 -- 172.21.15.43:0/2362877704 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcc6015eaf0 con 0x7fcc6006c1d0 2023-08-09T13:48:11.671 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:11.667+0000 7fcc66438640 1 -- 172.21.15.43:0/2362877704 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcc6015dc10 con 0x7fcc6006c1d0 2023-08-09T13:48:11.671 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:11.668+0000 7fcc5d7fa640 1 -- 172.21.15.43:0/2362877704 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 18) v1 ==== 88443+0+0 (secure 0 0 0) 0x7fcc50013470 con 0x7fcc6006c1d0 2023-08-09T13:48:11.672 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:11.669+0000 7fcc5d7fa640 1 --2- 172.21.15.43:0/2362877704 >> [v2:172.21.15.203:6832/272917756,v1:172.21.15.203:6833/272917756] conn(0x7fcc4006a1a0 0x7fcc4006c660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:48:11.672 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:11.669+0000 7fcc5d7fa640 1 -- 172.21.15.43:0/2362877704 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(66..66 src has 1..66) v4 ==== 6305+0+0 (secure 0 0 0) 0x7fcc500889a0 con 0x7fcc6006c1d0 2023-08-09T13:48:11.672 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:11.670+0000 7fcc577fe640 1 --2- 172.21.15.43:0/2362877704 >> [v2:172.21.15.203:6832/272917756,v1:172.21.15.203:6833/272917756] conn(0x7fcc4006a1a0 0x7fcc4006c660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:48:11.673 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:11.671+0000 7fcc577fe640 1 --2- 172.21.15.43:0/2362877704 >> [v2:172.21.15.203:6832/272917756,v1:172.21.15.203:6833/272917756] conn(0x7fcc4006a1a0 0x7fcc4006c660 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7fcc48036000 tx=0x7fcc4803b040 comp rx=0 tx=0).ready entity=mgr.5533 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-09T13:48:11.676 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:11.674+0000 7fcc5d7fa640 1 -- 172.21.15.43:0/2362877704 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178066 (secure 0 0 0) 0x7fcc6015dc10 con 0x7fcc6006c1d0 2023-08-09T13:48:11.917 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:11.914+0000 7fcc66438640 1 -- 172.21.15.43:0/2362877704 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fcc6015dc10 con 0x7fcc6006c1d0 2023-08-09T13:48:11.918 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:11.915+0000 7fcc5d7fa640 1 -- 172.21.15.43:0/2362877704 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 19 v19) v1 ==== 94+0+3702 (secure 0 0 0) 0x7fcc50021e90 con 0x7fcc6006c1d0 2023-08-09T13:48:11.918 INFO:teuthology.orchestra.run.smithi043.stdout: 2023-08-09T13:48:11.919 INFO:teuthology.orchestra.run.smithi043.stdout:{"epoch":19,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4919,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.203:6836/302214806","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6834","nonce":302214806},{"type":"v1","addr":"172.21.15.203:6836","nonce":302214806}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":12},{"gid":4922,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.203:6837/1677639100","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6835","nonce":1677639100},{"type":"v1","addr":"172.21.15.203:6837","nonce":1677639100}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":12},{"gid":5619,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.43:6834/2339480142","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6832","nonce":2339480142},{"type":"v1","addr":"172.21.15.43:6834","nonce":2339480142}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":12},{"gid":6384,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.43:6835/2872254667","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6833","nonce":2872254667},{"type":"v1","addr":"172.21.15.43:6835","nonce":2872254667}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":19}],"filesystems":[]} 2023-08-09T13:48:11.919 INFO:teuthology.orchestra.run.smithi043.stderr:dumped fsmap epoch 19 2023-08-09T13:48:11.921 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:11.918+0000 7fcc66438640 1 -- 172.21.15.43:0/2362877704 >> [v2:172.21.15.203:6832/272917756,v1:172.21.15.203:6833/272917756] conn(0x7fcc4006a1a0 msgr2=0x7fcc4006c660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:48:11.921 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:11.918+0000 7fcc66438640 1 --2- 172.21.15.43:0/2362877704 >> [v2:172.21.15.203:6832/272917756,v1:172.21.15.203:6833/272917756] conn(0x7fcc4006a1a0 0x7fcc4006c660 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7fcc48036000 tx=0x7fcc4803b040 comp rx=0 tx=0).stop 2023-08-09T13:48:11.922 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:11.918+0000 7fcc66438640 1 -- 172.21.15.43:0/2362877704 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fcc6006c1d0 msgr2=0x7fcc6015e5b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:48:11.922 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:11.918+0000 7fcc66438640 1 --2- 172.21.15.43:0/2362877704 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fcc6006c1d0 0x7fcc6015e5b0 secure :-1 s=READY pgs=815 cs=0 l=1 rev1=1 crypto rx=0x7fcc5000e870 tx=0x7fcc5000ed30 comp rx=0 tx=0).stop 2023-08-09T13:48:11.922 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:11.919+0000 7fcc66438640 1 -- 172.21.15.43:0/2362877704 shutdown_connections 2023-08-09T13:48:11.922 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:11.919+0000 7fcc66438640 1 --2- 172.21.15.43:0/2362877704 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fcc6015c670 0x7fcc60142200 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:48:11.922 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:11.919+0000 7fcc66438640 1 --2- 172.21.15.43:0/2362877704 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fcc600abe00 0x7fcc60144cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:48:11.922 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:11.919+0000 7fcc66438640 1 --2- 172.21.15.43:0/2362877704 >> [v2:172.21.15.203:6832/272917756,v1:172.21.15.203:6833/272917756] conn(0x7fcc4006a1a0 0x7fcc4006c660 unknown :-1 s=CLOSED pgs=45 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:48:11.923 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:11.919+0000 7fcc66438640 1 --2- 172.21.15.43:0/2362877704 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fcc6006c1d0 0x7fcc6015e5b0 unknown :-1 s=CLOSED pgs=815 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:48:11.923 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:11.919+0000 7fcc66438640 1 -- 172.21.15.43:0/2362877704 >> 172.21.15.43:0/2362877704 conn(0x7fcc600697c0 msgr2=0x7fcc601503a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-09T13:48:11.923 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:11.919+0000 7fcc66438640 1 -- 172.21.15.43:0/2362877704 shutdown_connections 2023-08-09T13:48:11.923 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:11.919+0000 7fcc66438640 1 -- 172.21.15.43:0/2362877704 wait complete. 2023-08-09T13:48:11.942 INFO:tasks.ceph.mds.a:Restarting daemon 2023-08-09T13:48:11.943 INFO:tasks.ceph.mds.a:Stopping old one... 2023-08-09T13:48:11.943 DEBUG:tasks.ceph.mds.a:waiting for process to exit 2023-08-09T13:48:11.943 INFO:teuthology.orchestra.run:waiting for 300 2023-08-09T13:48:11.943 INFO:tasks.ceph.mds.c:Restarting daemon 2023-08-09T13:48:11.944 INFO:tasks.ceph.mds.c:Stopping old one... 2023-08-09T13:48:11.944 DEBUG:tasks.ceph.mds.c:waiting for process to exit 2023-08-09T13:48:11.944 INFO:teuthology.orchestra.run:waiting for 300 2023-08-09T13:48:11.944 INFO:tasks.ceph.mds.b:Restarting daemon 2023-08-09T13:48:11.945 INFO:tasks.ceph.mds.b:Stopping old one... 2023-08-09T13:48:11.945 DEBUG:tasks.ceph.mds.b:waiting for process to exit 2023-08-09T13:48:11.945 INFO:teuthology.orchestra.run:waiting for 300 2023-08-09T13:48:11.945 INFO:tasks.ceph.mds.d:Restarting daemon 2023-08-09T13:48:11.945 INFO:tasks.ceph.mds.d:Stopping old one... 2023-08-09T13:48:11.946 DEBUG:tasks.ceph.mds.d:waiting for process to exit 2023-08-09T13:48:11.946 INFO:teuthology.orchestra.run:waiting for 300 2023-08-09T13:48:11.961 INFO:tasks.ceph.mds.a:Stopped 2023-08-09T13:48:11.962 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i a 2023-08-09T13:48:11.966 INFO:tasks.ceph.mds.b:Stopped 2023-08-09T13:48:11.966 DEBUG:teuthology.orchestra.run.smithi203:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i b 2023-08-09T13:48:11.967 INFO:tasks.ceph.mds.c:Stopped 2023-08-09T13:48:11.968 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i c 2023-08-09T13:48:11.974 INFO:tasks.ceph.mds.d:Stopped 2023-08-09T13:48:11.974 DEBUG:teuthology.orchestra.run.smithi203:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i d 2023-08-09T13:48:12.015 INFO:tasks.ceph.mds.a:Started 2023-08-09T13:48:12.016 DEBUG:teuthology.parallel:result is None 2023-08-09T13:48:12.022 INFO:tasks.ceph.mds.b:Started 2023-08-09T13:48:12.022 DEBUG:teuthology.parallel:result is None 2023-08-09T13:48:12.023 INFO:tasks.ceph.mds.c:Started 2023-08-09T13:48:12.023 DEBUG:teuthology.parallel:result is None 2023-08-09T13:48:12.031 INFO:tasks.ceph.mds.d:Started 2023-08-09T13:48:12.032 DEBUG:teuthology.parallel:result is None 2023-08-09T13:48:12.033 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist clear 2023-08-09T13:48:12.183 INFO:tasks.ceph.mds.a.smithi043.stdout:starting mds.a at 2023-08-09T13:48:12.188 INFO:tasks.ceph.mds.c.smithi043.stdout:starting mds.c at 2023-08-09T13:48:12.189 INFO:tasks.ceph.mds.b.smithi203.stdout:starting mds.b at 2023-08-09T13:48:12.202 INFO:tasks.ceph.mds.d.smithi203.stdout:starting mds.d at 2023-08-09T13:48:12.233 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:12.231+0000 7f0a132c5640 1 Processor -- start 2023-08-09T13:48:12.233 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:12.231+0000 7f0a132c5640 1 -- start start 2023-08-09T13:48:12.234 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:12.232+0000 7f0a132c5640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f0a0c142ca0 0x7f0a0c0c96e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:48:12.235 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:12.232+0000 7f0a132c5640 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7f0a0c0cd510 0x7f0a0c0c3290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:48:12.235 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:12.232+0000 7f0a132c5640 1 -- --> v1:172.21.15.203:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0a0c143080 con 0x7f0a0c15ce40 2023-08-09T13:48:12.235 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:12.232+0000 7f0a132c5640 1 -- --> v2:172.21.15.203:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0a0c06d400 con 0x7f0a0c0cd510 2023-08-09T13:48:12.235 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:12.232+0000 7f0a132c5640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0a0c0c9d90 con 0x7f0a0c142ca0 2023-08-09T13:48:12.236 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:12.233+0000 7f0a11862640 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7f0a0c0cd510 0x7f0a0c0c3290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:48:12.236 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:12.233+0000 7f0a10860640 1 --1- >> v1:172.21.15.203:6789/0 conn(0x7f0a0c15ce40 0x7f0a0c0cce00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.203:6789/0 says I am v1:172.21.15.43:56626/0 (socket says 172.21.15.43:56626) 2023-08-09T13:48:12.236 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:12.233+0000 7f0a10860640 1 -- 172.21.15.43:0/671038524 learned_addr learned my addr 172.21.15.43:0/671038524 (peer_addr_for_me v1:172.21.15.43:0/0) 2023-08-09T13:48:12.236 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:12.233+0000 7f0a11061640 1 --2- 172.21.15.43:0/671038524 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f0a0c142ca0 0x7f0a0c0c96e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:48:12.236 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:12.233+0000 7f0a03fff640 1 -- 172.21.15.43:0/671038524 <== mon.1 v1:172.21.15.203:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 344188790 0 0) 0x7f0a0c143080 con 0x7f0a0c15ce40 2023-08-09T13:48:12.237 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:12.233+0000 7f0a11061640 1 -- 172.21.15.43:0/671038524 >> v2:172.21.15.203:3300/0 conn(0x7f0a0c0cd510 msgr2=0x7f0a0c0c3290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-09T13:48:12.237 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:12.233+0000 7f0a11061640 1 --2- 172.21.15.43:0/671038524 >> v2:172.21.15.203:3300/0 conn(0x7f0a0c0cd510 0x7f0a0c0c3290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:48:12.237 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:12.233+0000 7f0a11061640 1 -- 172.21.15.43:0/671038524 >> v1:172.21.15.203:6789/0 conn(0x7f0a0c15ce40 legacy=0x7f0a0c0cce00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:48:12.237 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:12.233+0000 7f0a11061640 1 -- 172.21.15.43:0/671038524 --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0a0c0ca070 con 0x7f0a0c142ca0 2023-08-09T13:48:12.237 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:12.233+0000 7f0a11862640 1 --2- 172.21.15.43:0/671038524 >> v2:172.21.15.203:3300/0 conn(0x7f0a0c0cd510 0x7f0a0c0c3290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-08-09T13:48:12.238 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:12.234+0000 7f0a11061640 1 --2- 172.21.15.43:0/671038524 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f0a0c142ca0 0x7f0a0c0c96e0 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7f0a080097a0 tx=0x7f0a0802b8b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=8c3a753272e1137b server_cookie=0 in_seq=0 out_seq=0 2023-08-09T13:48:12.238 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:12.235+0000 7f0a03fff640 1 -- 172.21.15.43:0/671038524 <== mon.2 v2:172.21.15.203:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0a0802d040 con 0x7f0a0c142ca0 2023-08-09T13:48:12.238 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:12.235+0000 7f0a03fff640 1 -- 172.21.15.43:0/671038524 <== mon.2 v2:172.21.15.203:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0a08031810 con 0x7f0a0c142ca0 2023-08-09T13:48:12.239 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:12.235+0000 7f0a03fff640 1 -- 172.21.15.43:0/671038524 <== mon.2 v2:172.21.15.203:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0a08031ba0 con 0x7f0a0c142ca0 2023-08-09T13:48:12.239 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:12.235+0000 7f0a132c5640 1 -- 172.21.15.43:0/671038524 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f0a0c142ca0 msgr2=0x7f0a0c0c96e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:48:12.239 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:12.235+0000 7f0a132c5640 1 --2- 172.21.15.43:0/671038524 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f0a0c142ca0 0x7f0a0c0c96e0 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7f0a080097a0 tx=0x7f0a0802b8b0 comp rx=0 tx=0).stop 2023-08-09T13:48:12.239 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:12.235+0000 7f0a132c5640 1 -- 172.21.15.43:0/671038524 shutdown_connections 2023-08-09T13:48:12.239 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:12.235+0000 7f0a132c5640 1 --2- 172.21.15.43:0/671038524 >> v2:172.21.15.203:3300/0 conn(0x7f0a0c0cd510 0x7f0a0c0c3290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:48:12.239 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:12.235+0000 7f0a132c5640 1 --2- 172.21.15.43:0/671038524 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f0a0c142ca0 0x7f0a0c0c96e0 unknown :-1 s=CLOSED pgs=109 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:48:12.239 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:12.235+0000 7f0a132c5640 1 -- 172.21.15.43:0/671038524 >> 172.21.15.43:0/671038524 conn(0x7f0a0c0ce0a0 msgr2=0x7f0a0c0ce480 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-09T13:48:12.239 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:12.235+0000 7f0a132c5640 1 -- 172.21.15.43:0/671038524 shutdown_connections 2023-08-09T13:48:12.240 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:12.236+0000 7f0a132c5640 1 -- 172.21.15.43:0/671038524 wait complete. 2023-08-09T13:48:12.240 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:12.237+0000 7f0a132c5640 1 Processor -- start 2023-08-09T13:48:12.240 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:12.237+0000 7f0a132c5640 1 -- start start 2023-08-09T13:48:12.241 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:12.238+0000 7f0a132c5640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f0a0c0cd510 0x7f0a0c1598e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:48:12.241 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:12.238+0000 7f0a132c5640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f0a0c142ca0 0x7f0a0c136ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:48:12.242 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:12.238+0000 7f0a132c5640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f0a0c15ce40 0x7f0a0c1388e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:48:12.242 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:12.238+0000 7f0a132c5640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0a0c137630 con 0x7f0a0c0cd510 2023-08-09T13:48:12.242 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:12.238+0000 7f0a132c5640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0a0c159e20 con 0x7f0a0c15ce40 2023-08-09T13:48:12.242 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:12.238+0000 7f0a132c5640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0a0c158de0 con 0x7f0a0c142ca0 2023-08-09T13:48:12.242 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:12.239+0000 7f0a11061640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f0a0c0cd510 0x7f0a0c1598e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:48:12.242 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:12.239+0000 7f0a11061640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f0a0c0cd510 0x7f0a0c1598e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:35202/0 (socket says 172.21.15.43:35202) 2023-08-09T13:48:12.242 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:12.239+0000 7f0a11061640 1 -- 172.21.15.43:0/808944672 learned_addr learned my addr 172.21.15.43:0/808944672 (peer_addr_for_me v2:172.21.15.43:0/0) 2023-08-09T13:48:12.243 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:12.239+0000 7f0a10860640 1 --2- 172.21.15.43:0/808944672 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f0a0c142ca0 0x7f0a0c136ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:48:12.243 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:12.239+0000 7f0a11862640 1 --2- 172.21.15.43:0/808944672 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f0a0c15ce40 0x7f0a0c1388e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:48:12.244 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:12.239+0000 7f0a11061640 1 -- 172.21.15.43:0/808944672 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f0a0c142ca0 msgr2=0x7f0a0c136ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:48:12.244 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:12.239+0000 7f0a11061640 1 --2- 172.21.15.43:0/808944672 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f0a0c142ca0 0x7f0a0c136ac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:48:12.244 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:12.239+0000 7f0a11061640 1 -- 172.21.15.43:0/808944672 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f0a0c15ce40 msgr2=0x7f0a0c1388e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:48:12.244 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:12.239+0000 7f0a11061640 1 --2- 172.21.15.43:0/808944672 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f0a0c15ce40 0x7f0a0c1388e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:48:12.244 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:12.239+0000 7f0a11061640 1 -- 172.21.15.43:0/808944672 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0a04009a70 con 0x7f0a0c0cd510 2023-08-09T13:48:12.244 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:12.239+0000 7f0a10860640 1 --2- 172.21.15.43:0/808944672 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f0a0c142ca0 0x7f0a0c136ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-08-09T13:48:12.245 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:12.240+0000 7f0a11061640 1 --2- 172.21.15.43:0/808944672 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f0a0c0cd510 0x7f0a0c1598e0 secure :-1 s=READY pgs=820 cs=0 l=1 rev1=1 crypto rx=0x7f0a080098c0 tx=0x7f0a080046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-09T13:48:12.245 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:12.240+0000 7f0a01ffb640 1 -- 172.21.15.43:0/808944672 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0a08006e10 con 0x7f0a0c0cd510 2023-08-09T13:48:12.245 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:12.240+0000 7f0a01ffb640 1 -- 172.21.15.43:0/808944672 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0a0803ad60 con 0x7f0a0c0cd510 2023-08-09T13:48:12.245 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:12.240+0000 7f0a01ffb640 1 -- 172.21.15.43:0/808944672 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0a08044620 con 0x7f0a0c0cd510 2023-08-09T13:48:12.245 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:12.240+0000 7f0a132c5640 1 -- 172.21.15.43:0/808944672 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0a08009400 con 0x7f0a0c0cd510 2023-08-09T13:48:12.245 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:12.240+0000 7f0a132c5640 1 -- 172.21.15.43:0/808944672 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0a0c137b20 con 0x7f0a0c0cd510 2023-08-09T13:48:12.245 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:12.241+0000 7f0a132c5640 1 -- 172.21.15.43:0/808944672 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0a0c0739f0 con 0x7f0a0c0cd510 2023-08-09T13:48:12.246 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:12.242+0000 7f0a01ffb640 1 -- 172.21.15.43:0/808944672 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 18) v1 ==== 88443+0+0 (secure 0 0 0) 0x7f0a0804d440 con 0x7f0a0c0cd510 2023-08-09T13:48:12.246 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:12.243+0000 7f0a01ffb640 1 --2- 172.21.15.43:0/808944672 >> [v2:172.21.15.203:6832/272917756,v1:172.21.15.203:6833/272917756] conn(0x7f09f006a170 0x7f09f006c630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:48:12.246 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:12.243+0000 7f0a01ffb640 1 -- 172.21.15.43:0/808944672 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(66..66 src has 1..66) v4 ==== 6305+0+0 (secure 0 0 0) 0x7f0a080aada0 con 0x7f0a0c0cd510 2023-08-09T13:48:12.247 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:12.244+0000 7f0a10860640 1 --2- 172.21.15.43:0/808944672 >> [v2:172.21.15.203:6832/272917756,v1:172.21.15.203:6833/272917756] conn(0x7f09f006a170 0x7f09f006c630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:48:12.247 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:12.245+0000 7f0a10860640 1 --2- 172.21.15.43:0/808944672 >> [v2:172.21.15.203:6832/272917756,v1:172.21.15.203:6833/272917756] conn(0x7f09f006a170 0x7f09f006c630 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7f0a0c0cdc60 tx=0x7f09fc007cc0 comp rx=0 tx=0).ready entity=mgr.5533 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-09T13:48:12.250 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:12.248+0000 7f0a01ffb640 1 -- 172.21.15.43:0/808944672 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178066 (secure 0 0 0) 0x7f0a08076040 con 0x7f0a0c0cd510 2023-08-09T13:48:12.460 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:12.458+0000 7f0a132c5640 1 -- 172.21.15.43:0/808944672 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "osd blocklist clear"} v 0) v1 -- 0x7f0a0c0739f0 con 0x7f0a0c0cd510 2023-08-09T13:48:13.080 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:13.077+0000 7f0a01ffb640 1 -- 172.21.15.43:0/808944672 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist clear"}]=0 removed all blocklist entries v68) v1 ==== 97+0+0 (secure 0 0 0) 0x7f0a08035020 con 0x7f0a0c0cd510 2023-08-09T13:48:13.080 INFO:teuthology.orchestra.run.smithi043.stderr: removed all blocklist entries 2023-08-09T13:48:13.084 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:13.081+0000 7f0a132c5640 1 -- 172.21.15.43:0/808944672 >> [v2:172.21.15.203:6832/272917756,v1:172.21.15.203:6833/272917756] conn(0x7f09f006a170 msgr2=0x7f09f006c630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:48:13.084 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:13.081+0000 7f0a132c5640 1 --2- 172.21.15.43:0/808944672 >> [v2:172.21.15.203:6832/272917756,v1:172.21.15.203:6833/272917756] conn(0x7f09f006a170 0x7f09f006c630 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7f0a0c0cdc60 tx=0x7f09fc007cc0 comp rx=0 tx=0).stop 2023-08-09T13:48:13.084 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:13.081+0000 7f0a132c5640 1 -- 172.21.15.43:0/808944672 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f0a0c0cd510 msgr2=0x7f0a0c1598e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:48:13.084 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:13.081+0000 7f0a132c5640 1 --2- 172.21.15.43:0/808944672 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f0a0c0cd510 0x7f0a0c1598e0 secure :-1 s=READY pgs=820 cs=0 l=1 rev1=1 crypto rx=0x7f0a080098c0 tx=0x7f0a080046c0 comp rx=0 tx=0).stop 2023-08-09T13:48:13.085 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:13.082+0000 7f0a132c5640 1 -- 172.21.15.43:0/808944672 shutdown_connections 2023-08-09T13:48:13.085 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:13.082+0000 7f0a132c5640 1 --2- 172.21.15.43:0/808944672 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f0a0c15ce40 0x7f0a0c1388e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:48:13.085 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:13.082+0000 7f0a132c5640 1 --2- 172.21.15.43:0/808944672 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f0a0c142ca0 0x7f0a0c136ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:48:13.085 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:13.082+0000 7f0a132c5640 1 --2- 172.21.15.43:0/808944672 >> [v2:172.21.15.203:6832/272917756,v1:172.21.15.203:6833/272917756] conn(0x7f09f006a170 0x7f09f006c630 unknown :-1 s=CLOSED pgs=46 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:48:13.085 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:13.082+0000 7f0a132c5640 1 --2- 172.21.15.43:0/808944672 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f0a0c0cd510 0x7f0a0c1598e0 unknown :-1 s=CLOSED pgs=820 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:48:13.086 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:13.082+0000 7f0a132c5640 1 -- 172.21.15.43:0/808944672 >> 172.21.15.43:0/808944672 conn(0x7f0a0c0ce0a0 msgr2=0x7f0a0c070720 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-09T13:48:13.086 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:13.082+0000 7f0a132c5640 1 -- 172.21.15.43:0/808944672 shutdown_connections 2023-08-09T13:48:13.086 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:13.082+0000 7f0a132c5640 1 -- 172.21.15.43:0/808944672 wait complete. 2023-08-09T13:48:13.107 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth ls --format=json-pretty 2023-08-09T13:48:13.338 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:13.335+0000 7f517359e640 1 Processor -- start 2023-08-09T13:48:13.338 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:13.336+0000 7f517359e640 1 -- start start 2023-08-09T13:48:13.340 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:13.337+0000 7f517359e640 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7f51740acdf0 0x7f51740ad340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:48:13.340 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:13.337+0000 7f517359e640 1 -- --> v1:172.21.15.43:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f51740cbd90 con 0x7f51740c8b30 2023-08-09T13:48:13.340 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:13.337+0000 7f517359e640 1 -- --> v1:172.21.15.203:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f51740ca2b0 con 0x7f51740c8750 2023-08-09T13:48:13.340 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:13.337+0000 7f517359e640 1 -- --> v2:172.21.15.203:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f517415e810 con 0x7f51740acdf0 2023-08-09T13:48:13.342 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:13.338+0000 7f5169d9b640 1 --1- >> v1:172.21.15.43:6789/0 conn(0x7f51740c8b30 0x7f5174070f00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.43:6789/0 says I am v1:172.21.15.43:41706/0 (socket says 172.21.15.43:41706) 2023-08-09T13:48:13.342 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:13.338+0000 7f5169d9b640 1 -- 172.21.15.43:0/2035764691 learned_addr learned my addr 172.21.15.43:0/2035764691 (peer_addr_for_me v1:172.21.15.43:0/0) 2023-08-09T13:48:13.343 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:13.338+0000 7f5172d9d640 1 --2- 172.21.15.43:0/2035764691 >> v2:172.21.15.203:3300/0 conn(0x7f51740acdf0 0x7f51740ad340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:48:13.343 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:13.338+0000 7f5171d9b640 1 -- 172.21.15.43:0/2035764691 <== mon.0 v1:172.21.15.43:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3953115151 0 0) 0x7f51740cbd90 con 0x7f51740c8b30 2023-08-09T13:48:13.343 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:13.338+0000 7f5171d9b640 1 -- 172.21.15.43:0/2035764691 --> v1:172.21.15.43:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f5160003520 con 0x7f51740c8b30 2023-08-09T13:48:13.343 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:13.339+0000 7f5171d9b640 1 -- 172.21.15.43:0/2035764691 <== mon.0 v1:172.21.15.43:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1702077148 0 0) 0x7f5160003520 con 0x7f51740c8b30 2023-08-09T13:48:13.343 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:13.339+0000 7f5171d9b640 1 -- 172.21.15.43:0/2035764691 >> v2:172.21.15.203:3300/0 conn(0x7f51740acdf0 msgr2=0x7f51740ad340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-09T13:48:13.343 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:13.339+0000 7f5171d9b640 1 --2- 172.21.15.43:0/2035764691 >> v2:172.21.15.203:3300/0 conn(0x7f51740acdf0 0x7f51740ad340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:48:13.343 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:13.339+0000 7f5171d9b640 1 -- 172.21.15.43:0/2035764691 >> v1:172.21.15.203:6789/0 conn(0x7f51740c8750 legacy=0x7f517406f410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:48:13.344 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:13.339+0000 7f5171d9b640 1 -- 172.21.15.43:0/2035764691 --> v1:172.21.15.43:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f51740c9a20 con 0x7f51740c8b30 2023-08-09T13:48:13.344 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:13.339+0000 7f5171d9b640 1 -- 172.21.15.43:0/2035764691 <== mon.0 v1:172.21.15.43:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1285150740 0 0) 0x7f515c001670 con 0x7f51740c8b30 2023-08-09T13:48:13.344 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:13.339+0000 7f5171d9b640 1 -- 172.21.15.43:0/2035764691 >> v1:172.21.15.43:6789/0 conn(0x7f51740c8b30 legacy=0x7f5174070f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:48:13.344 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:13.339+0000 7f5171d9b640 1 --2- 172.21.15.43:0/2035764691 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f5160003970 0x7f5160003d50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:48:13.344 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:13.339+0000 7f5171d9b640 1 --2- 172.21.15.43:0/2035764691 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f5160004370 0x7f5160004840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:48:13.344 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:13.339+0000 7f5171d9b640 1 --2- 172.21.15.43:0/2035764691 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f51600050e0 0x7f516000b980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:48:13.345 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:13.339+0000 7f5171d9b640 1 -- 172.21.15.43:0/2035764691 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f517415e810 con 0x7f51600050e0 2023-08-09T13:48:13.345 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:13.339+0000 7f5171d9b640 1 -- 172.21.15.43:0/2035764691 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f516000c020 con 0x7f5160003970 2023-08-09T13:48:13.345 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:13.339+0000 7f5171d9b640 1 -- 172.21.15.43:0/2035764691 --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f516000c1e0 con 0x7f5160004370 2023-08-09T13:48:13.345 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:13.339+0000 7f5172d9d640 1 --2- 172.21.15.43:0/2035764691 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f51600050e0 0x7f516000b980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:48:13.345 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:13.340+0000 7f517259c640 1 --2- 172.21.15.43:0/2035764691 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f5160003970 0x7f5160003d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:48:13.345 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:13.340+0000 7f5169d9b640 1 --2- 172.21.15.43:0/2035764691 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f5160004370 0x7f5160004840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:48:13.345 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:13.340+0000 7f5172d9d640 1 -- 172.21.15.43:0/2035764691 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f5160004370 msgr2=0x7f5160004840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-09T13:48:13.346 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:13.340+0000 7f5172d9d640 1 --2- 172.21.15.43:0/2035764691 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f5160004370 0x7f5160004840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:48:13.346 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:13.340+0000 7f5172d9d640 1 -- 172.21.15.43:0/2035764691 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f5160003970 msgr2=0x7f5160003d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-09T13:48:13.346 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:13.340+0000 7f5172d9d640 1 --2- 172.21.15.43:0/2035764691 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f5160003970 0x7f5160003d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:48:13.346 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:13.340+0000 7f5172d9d640 1 -- 172.21.15.43:0/2035764691 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5160003520 con 0x7f51600050e0 2023-08-09T13:48:13.348 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:13.341+0000 7f5172d9d640 1 -- 172.21.15.43:0/2035764691 reap_dead start 2023-08-09T13:48:13.348 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:13.341+0000 7f5172d9d640 1 --2- 172.21.15.43:0/2035764691 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f51600050e0 0x7f516000b980 secure :-1 s=READY pgs=822 cs=0 l=1 rev1=1 crypto rx=0x7f516c009fc0 tx=0x7f51600057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a222446966ec34d9 server_cookie=0 in_seq=0 out_seq=0 2023-08-09T13:48:13.348 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:13.341+0000 7f5171d9b640 1 -- 172.21.15.43:0/2035764691 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f516c02cc60 con 0x7f51600050e0 2023-08-09T13:48:13.348 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:13.341+0000 7f5171d9b640 1 -- 172.21.15.43:0/2035764691 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f516c00f040 con 0x7f51600050e0 2023-08-09T13:48:13.348 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:13.341+0000 7f5171d9b640 1 -- 172.21.15.43:0/2035764691 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f516c003990 con 0x7f51600050e0 2023-08-09T13:48:13.349 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:13.342+0000 7f517359e640 1 -- 172.21.15.43:0/2035764691 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f51600050e0 msgr2=0x7f516000b980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:48:13.349 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:13.342+0000 7f517359e640 1 --2- 172.21.15.43:0/2035764691 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f51600050e0 0x7f516000b980 secure :-1 s=READY pgs=822 cs=0 l=1 rev1=1 crypto rx=0x7f516c009fc0 tx=0x7f51600057f0 comp rx=0 tx=0).stop 2023-08-09T13:48:13.349 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:13.342+0000 7f517359e640 1 -- 172.21.15.43:0/2035764691 shutdown_connections 2023-08-09T13:48:13.349 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:13.342+0000 7f517359e640 1 --2- 172.21.15.43:0/2035764691 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f51600050e0 0x7f516000b980 unknown :-1 s=CLOSED pgs=822 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:48:13.349 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:13.342+0000 7f517359e640 1 -- 172.21.15.43:0/2035764691 >> 172.21.15.43:0/2035764691 conn(0x7f51740cde90 msgr2=0x7f51740ce2a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-09T13:48:13.349 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:13.342+0000 7f517359e640 1 -- 172.21.15.43:0/2035764691 shutdown_connections 2023-08-09T13:48:13.349 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:13.342+0000 7f517359e640 1 -- 172.21.15.43:0/2035764691 wait complete. 2023-08-09T13:48:13.350 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:13.344+0000 7f517359e640 1 Processor -- start 2023-08-09T13:48:13.350 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:13.344+0000 7f517359e640 1 -- start start 2023-08-09T13:48:13.350 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:13.345+0000 7f517359e640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f51600050e0 0x7f51741f1ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:48:13.350 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:13.345+0000 7f517359e640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f51741f26e0 0x7f5174099aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:48:13.350 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:13.345+0000 7f517359e640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f51740abf30 0x7f51740ac3b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:48:13.350 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:13.345+0000 7f517359e640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f51740c9f70 con 0x7f51600050e0 2023-08-09T13:48:13.350 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:13.345+0000 7f517359e640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f51741f2b60 con 0x7f51741f26e0 2023-08-09T13:48:13.350 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:13.345+0000 7f517359e640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f51740ac8f0 con 0x7f51740abf30 2023-08-09T13:48:13.351 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:13.345+0000 7f517259c640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f51600050e0 0x7f51741f1ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:48:13.351 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:13.345+0000 7f517259c640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f51600050e0 0x7f51741f1ae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:35210/0 (socket says 172.21.15.43:35210) 2023-08-09T13:48:13.351 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:13.345+0000 7f517259c640 1 -- 172.21.15.43:0/4224293632 learned_addr learned my addr 172.21.15.43:0/4224293632 (peer_addr_for_me v2:172.21.15.43:0/0) 2023-08-09T13:48:13.352 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:13.346+0000 7f517259c640 1 -- 172.21.15.43:0/4224293632 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f51740abf30 msgr2=0x7f51740ac3b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:48:13.352 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:13.346+0000 7f517259c640 1 --2- 172.21.15.43:0/4224293632 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f51740abf30 0x7f51740ac3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:48:13.353 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:13.346+0000 7f517259c640 1 -- 172.21.15.43:0/4224293632 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f51741f26e0 msgr2=0x7f5174099aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:48:13.353 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:13.346+0000 7f517259c640 1 --2- 172.21.15.43:0/4224293632 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f51741f26e0 0x7f5174099aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:48:13.353 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:13.346+0000 7f517259c640 1 -- 172.21.15.43:0/4224293632 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f517409ae00 con 0x7f51600050e0 2023-08-09T13:48:13.353 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:13.346+0000 7f517259c640 1 --2- 172.21.15.43:0/4224293632 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f51600050e0 0x7f51741f1ae0 secure :-1 s=READY pgs=823 cs=0 l=1 rev1=1 crypto rx=0x7f516400e5f0 tx=0x7f516400eab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-09T13:48:13.353 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:13.346+0000 7f516b7fe640 1 -- 172.21.15.43:0/4224293632 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f51640052e0 con 0x7f51600050e0 2023-08-09T13:48:13.353 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:13.346+0000 7f516b7fe640 1 -- 172.21.15.43:0/4224293632 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5164005440 con 0x7f51600050e0 2023-08-09T13:48:13.353 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:13.346+0000 7f516b7fe640 1 -- 172.21.15.43:0/4224293632 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5164003830 con 0x7f51600050e0 2023-08-09T13:48:13.353 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:13.346+0000 7f517359e640 1 -- 172.21.15.43:0/4224293632 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5174099fe0 con 0x7f51600050e0 2023-08-09T13:48:13.354 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:13.347+0000 7f517359e640 1 -- 172.21.15.43:0/4224293632 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f517409a560 con 0x7f51600050e0 2023-08-09T13:48:13.354 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:13.348+0000 7f517359e640 1 -- 172.21.15.43:0/4224293632 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f51740b0b40 con 0x7f51600050e0 2023-08-09T13:48:13.354 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:13.349+0000 7f516b7fe640 1 -- 172.21.15.43:0/4224293632 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 18) v1 ==== 88443+0+0 (secure 0 0 0) 0x7f5164003990 con 0x7f51600050e0 2023-08-09T13:48:13.354 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:13.350+0000 7f516b7fe640 1 --2- 172.21.15.43:0/4224293632 >> [v2:172.21.15.203:6832/272917756,v1:172.21.15.203:6833/272917756] conn(0x7f515406a1a0 0x7f515406c660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:48:13.354 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:13.350+0000 7f516b7fe640 1 -- 172.21.15.43:0/4224293632 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(70..70 src has 1..70) v4 ==== 5574+0+0 (secure 0 0 0) 0x7f5164088310 con 0x7f51600050e0 2023-08-09T13:48:13.355 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:13.351+0000 7f5169d9b640 1 --2- 172.21.15.43:0/4224293632 >> [v2:172.21.15.203:6832/272917756,v1:172.21.15.203:6833/272917756] conn(0x7f515406a1a0 0x7f515406c660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:48:13.355 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:13.352+0000 7f5169d9b640 1 --2- 172.21.15.43:0/4224293632 >> [v2:172.21.15.203:6832/272917756,v1:172.21.15.203:6833/272917756] conn(0x7f515406a1a0 0x7f515406c660 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7f515c001200 tx=0x7f515c009040 comp rx=0 tx=0).ready entity=mgr.5533 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-09T13:48:13.356 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:13.354+0000 7f516b7fe640 1 -- 172.21.15.43:0/4224293632 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178066 (secure 0 0 0) 0x7f51740b0b40 con 0x7f51600050e0 2023-08-09T13:48:13.625 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:13.622+0000 7f517359e640 1 -- 172.21.15.43:0/4224293632 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "auth ls", "format": "json-pretty"} v 0) v1 -- 0x7f51740b0b40 con 0x7f51600050e0 2023-08-09T13:48:13.626 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:13.623+0000 7f516b7fe640 1 -- 172.21.15.43:0/4224293632 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "auth ls", "format": "json-pretty"}]=0 v12) v1 ==== 80+0+5304 (secure 0 0 0) 0x7f51740b0b40 con 0x7f51600050e0 2023-08-09T13:48:13.626 INFO:teuthology.orchestra.run.smithi043.stdout: 2023-08-09T13:48:13.626 INFO:teuthology.orchestra.run.smithi043.stdout:{ 2023-08-09T13:48:13.626 INFO:teuthology.orchestra.run.smithi043.stdout: "auth_dump": [ 2023-08-09T13:48:13.626 INFO:teuthology.orchestra.run.smithi043.stdout: { 2023-08-09T13:48:13.627 INFO:teuthology.orchestra.run.smithi043.stdout: "entity": "mds.a", 2023-08-09T13:48:13.627 INFO:teuthology.orchestra.run.smithi043.stdout: "key": "AQCOldNkPu6GCRAAZecwctvMji0Zubb60KMlyQ==", 2023-08-09T13:48:13.627 INFO:teuthology.orchestra.run.smithi043.stdout: "caps": { 2023-08-09T13:48:13.627 INFO:teuthology.orchestra.run.smithi043.stdout: "mds": "allow", 2023-08-09T13:48:13.627 INFO:teuthology.orchestra.run.smithi043.stdout: "mgr": "allow *", 2023-08-09T13:48:13.627 INFO:teuthology.orchestra.run.smithi043.stdout: "mon": "allow *", 2023-08-09T13:48:13.627 INFO:teuthology.orchestra.run.smithi043.stdout: "osd": "allow *" 2023-08-09T13:48:13.627 INFO:teuthology.orchestra.run.smithi043.stdout: } 2023-08-09T13:48:13.628 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2023-08-09T13:48:13.628 INFO:teuthology.orchestra.run.smithi043.stdout: { 2023-08-09T13:48:13.628 INFO:teuthology.orchestra.run.smithi043.stdout: "entity": "mds.b", 2023-08-09T13:48:13.628 INFO:teuthology.orchestra.run.smithi043.stdout: "key": "AQCOldNkt2+5KxAAHcckB+S5NZHsF5z22StXmw==", 2023-08-09T13:48:13.628 INFO:teuthology.orchestra.run.smithi043.stdout: "caps": { 2023-08-09T13:48:13.628 INFO:teuthology.orchestra.run.smithi043.stdout: "mds": "allow", 2023-08-09T13:48:13.628 INFO:teuthology.orchestra.run.smithi043.stdout: "mgr": "allow *", 2023-08-09T13:48:13.629 INFO:teuthology.orchestra.run.smithi043.stdout: "mon": "allow *", 2023-08-09T13:48:13.629 INFO:teuthology.orchestra.run.smithi043.stdout: "osd": "allow *" 2023-08-09T13:48:13.629 INFO:teuthology.orchestra.run.smithi043.stdout: } 2023-08-09T13:48:13.629 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2023-08-09T13:48:13.629 INFO:teuthology.orchestra.run.smithi043.stdout: { 2023-08-09T13:48:13.629 INFO:teuthology.orchestra.run.smithi043.stdout: "entity": "mds.c", 2023-08-09T13:48:13.629 INFO:teuthology.orchestra.run.smithi043.stdout: "key": "AQCOldNk5V46GhAAQBhkvfjVSJCSHLS/YkhgCA==", 2023-08-09T13:48:13.629 INFO:teuthology.orchestra.run.smithi043.stdout: "caps": { 2023-08-09T13:48:13.630 INFO:teuthology.orchestra.run.smithi043.stdout: "mds": "allow", 2023-08-09T13:48:13.630 INFO:teuthology.orchestra.run.smithi043.stdout: "mgr": "allow *", 2023-08-09T13:48:13.630 INFO:teuthology.orchestra.run.smithi043.stdout: "mon": "allow *", 2023-08-09T13:48:13.630 INFO:teuthology.orchestra.run.smithi043.stdout: "osd": "allow *" 2023-08-09T13:48:13.630 INFO:teuthology.orchestra.run.smithi043.stdout: } 2023-08-09T13:48:13.630 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2023-08-09T13:48:13.630 INFO:teuthology.orchestra.run.smithi043.stdout: { 2023-08-09T13:48:13.630 INFO:teuthology.orchestra.run.smithi043.stdout: "entity": "mds.d", 2023-08-09T13:48:13.631 INFO:teuthology.orchestra.run.smithi043.stdout: "key": "AQCPldNkNBoTARAAMVW+rSyXQKaau6I90fUX9Q==", 2023-08-09T13:48:13.631 INFO:teuthology.orchestra.run.smithi043.stdout: "caps": { 2023-08-09T13:48:13.631 INFO:teuthology.orchestra.run.smithi043.stdout: "mds": "allow", 2023-08-09T13:48:13.631 INFO:teuthology.orchestra.run.smithi043.stdout: "mgr": "allow *", 2023-08-09T13:48:13.631 INFO:teuthology.orchestra.run.smithi043.stdout: "mon": "allow *", 2023-08-09T13:48:13.631 INFO:teuthology.orchestra.run.smithi043.stdout: "osd": "allow *" 2023-08-09T13:48:13.631 INFO:teuthology.orchestra.run.smithi043.stdout: } 2023-08-09T13:48:13.631 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2023-08-09T13:48:13.632 INFO:teuthology.orchestra.run.smithi043.stdout: { 2023-08-09T13:48:13.632 INFO:teuthology.orchestra.run.smithi043.stdout: "entity": "osd.0", 2023-08-09T13:48:13.632 INFO:teuthology.orchestra.run.smithi043.stdout: "key": "AQCRldNkltPlMhAAAt3rFvraCtI0aSqGvt6UxA==", 2023-08-09T13:48:13.632 INFO:teuthology.orchestra.run.smithi043.stdout: "caps": { 2023-08-09T13:48:13.632 INFO:teuthology.orchestra.run.smithi043.stdout: "mgr": "allow profile osd", 2023-08-09T13:48:13.632 INFO:teuthology.orchestra.run.smithi043.stdout: "mon": "allow profile osd", 2023-08-09T13:48:13.632 INFO:teuthology.orchestra.run.smithi043.stdout: "osd": "allow *" 2023-08-09T13:48:13.633 INFO:teuthology.orchestra.run.smithi043.stdout: } 2023-08-09T13:48:13.633 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2023-08-09T13:48:13.633 INFO:teuthology.orchestra.run.smithi043.stdout: { 2023-08-09T13:48:13.633 INFO:teuthology.orchestra.run.smithi043.stdout: "entity": "osd.1", 2023-08-09T13:48:13.633 INFO:teuthology.orchestra.run.smithi043.stdout: "key": "AQCVldNkzpKYEhAA+wen0kV/vUDLQztq3vhfOg==", 2023-08-09T13:48:13.633 INFO:teuthology.orchestra.run.smithi043.stdout: "caps": { 2023-08-09T13:48:13.633 INFO:teuthology.orchestra.run.smithi043.stdout: "mgr": "allow profile osd", 2023-08-09T13:48:13.634 INFO:teuthology.orchestra.run.smithi043.stdout: "mon": "allow profile osd", 2023-08-09T13:48:13.634 INFO:teuthology.orchestra.run.smithi043.stdout: "osd": "allow *" 2023-08-09T13:48:13.634 INFO:teuthology.orchestra.run.smithi043.stdout: } 2023-08-09T13:48:13.634 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2023-08-09T13:48:13.634 INFO:teuthology.orchestra.run.smithi043.stdout: { 2023-08-09T13:48:13.634 INFO:teuthology.orchestra.run.smithi043.stdout: "entity": "osd.2", 2023-08-09T13:48:13.635 INFO:teuthology.orchestra.run.smithi043.stdout: "key": "AQCYldNkYbwNHhAAQY8g9dldb1V9M2MM2SHKDg==", 2023-08-09T13:48:13.635 INFO:teuthology.orchestra.run.smithi043.stdout: "caps": { 2023-08-09T13:48:13.635 INFO:teuthology.orchestra.run.smithi043.stdout: "mgr": "allow profile osd", 2023-08-09T13:48:13.635 INFO:teuthology.orchestra.run.smithi043.stdout: "mon": "allow profile osd", 2023-08-09T13:48:13.635 INFO:teuthology.orchestra.run.smithi043.stdout: "osd": "allow *" 2023-08-09T13:48:13.635 INFO:teuthology.orchestra.run.smithi043.stdout: } 2023-08-09T13:48:13.635 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2023-08-09T13:48:13.636 INFO:teuthology.orchestra.run.smithi043.stdout: { 2023-08-09T13:48:13.636 INFO:teuthology.orchestra.run.smithi043.stdout: "entity": "osd.3", 2023-08-09T13:48:13.636 INFO:teuthology.orchestra.run.smithi043.stdout: "key": "AQCbldNkeyW6KRAAsd+kgqik2aD4U7PXwccWtA==", 2023-08-09T13:48:13.636 INFO:teuthology.orchestra.run.smithi043.stdout: "caps": { 2023-08-09T13:48:13.636 INFO:teuthology.orchestra.run.smithi043.stdout: "mgr": "allow profile osd", 2023-08-09T13:48:13.637 INFO:teuthology.orchestra.run.smithi043.stdout: "mon": "allow profile osd", 2023-08-09T13:48:13.637 INFO:teuthology.orchestra.run.smithi043.stdout: "osd": "allow *" 2023-08-09T13:48:13.637 INFO:teuthology.orchestra.run.smithi043.stdout: } 2023-08-09T13:48:13.637 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2023-08-09T13:48:13.637 INFO:teuthology.orchestra.run.smithi043.stdout: { 2023-08-09T13:48:13.637 INFO:teuthology.orchestra.run.smithi043.stdout: "entity": "osd.4", 2023-08-09T13:48:13.637 INFO:teuthology.orchestra.run.smithi043.stdout: "key": "AQCildNk80UkOBAAtX+4JGpOVcfZEHetJoeZKw==", 2023-08-09T13:48:13.637 INFO:teuthology.orchestra.run.smithi043.stdout: "caps": { 2023-08-09T13:48:13.638 INFO:teuthology.orchestra.run.smithi043.stdout: "mgr": "allow profile osd", 2023-08-09T13:48:13.638 INFO:teuthology.orchestra.run.smithi043.stdout: "mon": "allow profile osd", 2023-08-09T13:48:13.638 INFO:teuthology.orchestra.run.smithi043.stdout: "osd": "allow *" 2023-08-09T13:48:13.638 INFO:teuthology.orchestra.run.smithi043.stdout: } 2023-08-09T13:48:13.638 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2023-08-09T13:48:13.638 INFO:teuthology.orchestra.run.smithi043.stdout: { 2023-08-09T13:48:13.638 INFO:teuthology.orchestra.run.smithi043.stdout: "entity": "osd.5", 2023-08-09T13:48:13.638 INFO:teuthology.orchestra.run.smithi043.stdout: "key": "AQCmldNkCA9SBxAAZj4So6MRj2Eq1I9H9JzLDA==", 2023-08-09T13:48:13.639 INFO:teuthology.orchestra.run.smithi043.stdout: "caps": { 2023-08-09T13:48:13.639 INFO:teuthology.orchestra.run.smithi043.stdout: "mgr": "allow profile osd", 2023-08-09T13:48:13.639 INFO:teuthology.orchestra.run.smithi043.stdout: "mon": "allow profile osd", 2023-08-09T13:48:13.639 INFO:teuthology.orchestra.run.smithi043.stdout: "osd": "allow *" 2023-08-09T13:48:13.639 INFO:teuthology.orchestra.run.smithi043.stdout: } 2023-08-09T13:48:13.639 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2023-08-09T13:48:13.639 INFO:teuthology.orchestra.run.smithi043.stdout: { 2023-08-09T13:48:13.639 INFO:teuthology.orchestra.run.smithi043.stdout: "entity": "osd.6", 2023-08-09T13:48:13.640 INFO:teuthology.orchestra.run.smithi043.stdout: "key": "AQCpldNkaOwAExAAraV1tUwr9uU6GheZ+hvdqQ==", 2023-08-09T13:48:13.640 INFO:teuthology.orchestra.run.smithi043.stdout: "caps": { 2023-08-09T13:48:13.640 INFO:teuthology.orchestra.run.smithi043.stdout: "mgr": "allow profile osd", 2023-08-09T13:48:13.640 INFO:teuthology.orchestra.run.smithi043.stdout: "mon": "allow profile osd", 2023-08-09T13:48:13.640 INFO:teuthology.orchestra.run.smithi043.stdout: "osd": "allow *" 2023-08-09T13:48:13.640 INFO:teuthology.orchestra.run.smithi043.stdout: } 2023-08-09T13:48:13.641 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2023-08-09T13:48:13.641 INFO:teuthology.orchestra.run.smithi043.stdout: { 2023-08-09T13:48:13.641 INFO:teuthology.orchestra.run.smithi043.stdout: "entity": "osd.7", 2023-08-09T13:48:13.641 INFO:teuthology.orchestra.run.smithi043.stdout: "key": "AQCsldNk2/GwLBAAP42EXPivzLk44X8aMEVVCg==", 2023-08-09T13:48:13.641 INFO:teuthology.orchestra.run.smithi043.stdout: "caps": { 2023-08-09T13:48:13.641 INFO:teuthology.orchestra.run.smithi043.stdout: "mgr": "allow profile osd", 2023-08-09T13:48:13.641 INFO:teuthology.orchestra.run.smithi043.stdout: "mon": "allow profile osd", 2023-08-09T13:48:13.641 INFO:teuthology.orchestra.run.smithi043.stdout: "osd": "allow *" 2023-08-09T13:48:13.642 INFO:teuthology.orchestra.run.smithi043.stdout: } 2023-08-09T13:48:13.642 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2023-08-09T13:48:13.642 INFO:teuthology.orchestra.run.smithi043.stdout: { 2023-08-09T13:48:13.642 INFO:teuthology.orchestra.run.smithi043.stdout: "entity": "client.0", 2023-08-09T13:48:13.642 INFO:teuthology.orchestra.run.smithi043.stdout: "key": "AQCPldNkg8zrDxAAM/hnJaE6S9HU/fRwQeBV5w==", 2023-08-09T13:48:13.642 INFO:teuthology.orchestra.run.smithi043.stdout: "caps": { 2023-08-09T13:48:13.642 INFO:teuthology.orchestra.run.smithi043.stdout: "mds": "allow", 2023-08-09T13:48:13.642 INFO:teuthology.orchestra.run.smithi043.stdout: "mon": "allow r", 2023-08-09T13:48:13.643 INFO:teuthology.orchestra.run.smithi043.stdout: "osd": "allow rw tag cephfs data=cephfs" 2023-08-09T13:48:13.643 INFO:teuthology.orchestra.run.smithi043.stdout: } 2023-08-09T13:48:13.643 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2023-08-09T13:48:13.643 INFO:teuthology.orchestra.run.smithi043.stdout: { 2023-08-09T13:48:13.643 INFO:teuthology.orchestra.run.smithi043.stdout: "entity": "client.1", 2023-08-09T13:48:13.643 INFO:teuthology.orchestra.run.smithi043.stdout: "key": "AQCPldNklfFxJxAA19+L78rW6jU1uOyjey6WNA==", 2023-08-09T13:48:13.643 INFO:teuthology.orchestra.run.smithi043.stdout: "caps": { 2023-08-09T13:48:13.644 INFO:teuthology.orchestra.run.smithi043.stdout: "mds": "allow", 2023-08-09T13:48:13.644 INFO:teuthology.orchestra.run.smithi043.stdout: "mgr": "allow r", 2023-08-09T13:48:13.644 INFO:teuthology.orchestra.run.smithi043.stdout: "mon": "allow rw", 2023-08-09T13:48:13.644 INFO:teuthology.orchestra.run.smithi043.stdout: "osd": "allow rwx" 2023-08-09T13:48:13.644 INFO:teuthology.orchestra.run.smithi043.stdout: } 2023-08-09T13:48:13.644 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2023-08-09T13:48:13.644 INFO:teuthology.orchestra.run.smithi043.stdout: { 2023-08-09T13:48:13.645 INFO:teuthology.orchestra.run.smithi043.stdout: "entity": "client.2", 2023-08-09T13:48:13.645 INFO:teuthology.orchestra.run.smithi043.stdout: "key": "AQCPldNkC2lgGxAAZ8qZboOCUaQlcpkPL1ozHA==", 2023-08-09T13:48:13.645 INFO:teuthology.orchestra.run.smithi043.stdout: "caps": { 2023-08-09T13:48:13.645 INFO:teuthology.orchestra.run.smithi043.stdout: "mds": "allow", 2023-08-09T13:48:13.645 INFO:teuthology.orchestra.run.smithi043.stdout: "mgr": "allow r", 2023-08-09T13:48:13.645 INFO:teuthology.orchestra.run.smithi043.stdout: "mon": "allow rw", 2023-08-09T13:48:13.645 INFO:teuthology.orchestra.run.smithi043.stdout: "osd": "allow rwx" 2023-08-09T13:48:13.646 INFO:teuthology.orchestra.run.smithi043.stdout: } 2023-08-09T13:48:13.646 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2023-08-09T13:48:13.646 INFO:teuthology.orchestra.run.smithi043.stdout: { 2023-08-09T13:48:13.646 INFO:teuthology.orchestra.run.smithi043.stdout: "entity": "client.3", 2023-08-09T13:48:13.646 INFO:teuthology.orchestra.run.smithi043.stdout: "key": "AQCPldNkl+rXMhAA9rf3KdnSYbvngqmLYnIGGA==", 2023-08-09T13:48:13.646 INFO:teuthology.orchestra.run.smithi043.stdout: "caps": { 2023-08-09T13:48:13.646 INFO:teuthology.orchestra.run.smithi043.stdout: "mds": "allow", 2023-08-09T13:48:13.647 INFO:teuthology.orchestra.run.smithi043.stdout: "mgr": "allow r", 2023-08-09T13:48:13.647 INFO:teuthology.orchestra.run.smithi043.stdout: "mon": "allow rw", 2023-08-09T13:48:13.647 INFO:teuthology.orchestra.run.smithi043.stdout: "osd": "allow rwx" 2023-08-09T13:48:13.647 INFO:teuthology.orchestra.run.smithi043.stdout: } 2023-08-09T13:48:13.647 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2023-08-09T13:48:13.647 INFO:teuthology.orchestra.run.smithi043.stdout: { 2023-08-09T13:48:13.647 INFO:teuthology.orchestra.run.smithi043.stdout: "entity": "client.admin", 2023-08-09T13:48:13.647 INFO:teuthology.orchestra.run.smithi043.stdout: "key": "AQCMldNkUae6NRAA2FBTr0QPBdKNDoe1WK5zyg==", 2023-08-09T13:48:13.648 INFO:teuthology.orchestra.run.smithi043.stdout: "caps": { 2023-08-09T13:48:13.648 INFO:teuthology.orchestra.run.smithi043.stdout: "mds": "allow *", 2023-08-09T13:48:13.648 INFO:teuthology.orchestra.run.smithi043.stdout: "mgr": "allow *", 2023-08-09T13:48:13.648 INFO:teuthology.orchestra.run.smithi043.stdout: "mon": "allow *", 2023-08-09T13:48:13.648 INFO:teuthology.orchestra.run.smithi043.stdout: "osd": "allow *" 2023-08-09T13:48:13.648 INFO:teuthology.orchestra.run.smithi043.stdout: } 2023-08-09T13:48:13.649 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2023-08-09T13:48:13.649 INFO:teuthology.orchestra.run.smithi043.stdout: { 2023-08-09T13:48:13.649 INFO:teuthology.orchestra.run.smithi043.stdout: "entity": "mgr.x", 2023-08-09T13:48:13.649 INFO:teuthology.orchestra.run.smithi043.stdout: "key": "AQCNldNk2ufSNxAAU7msRkwLaoLZHTIsk0ck/g==", 2023-08-09T13:48:13.649 INFO:teuthology.orchestra.run.smithi043.stdout: "caps": { 2023-08-09T13:48:13.649 INFO:teuthology.orchestra.run.smithi043.stdout: "mds": "allow *", 2023-08-09T13:48:13.649 INFO:teuthology.orchestra.run.smithi043.stdout: "mon": "allow profile mgr", 2023-08-09T13:48:13.649 INFO:teuthology.orchestra.run.smithi043.stdout: "osd": "allow *" 2023-08-09T13:48:13.650 INFO:teuthology.orchestra.run.smithi043.stdout: } 2023-08-09T13:48:13.650 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2023-08-09T13:48:13.650 INFO:teuthology.orchestra.run.smithi043.stdout: { 2023-08-09T13:48:13.650 INFO:teuthology.orchestra.run.smithi043.stdout: "entity": "mgr.y", 2023-08-09T13:48:13.650 INFO:teuthology.orchestra.run.smithi043.stdout: "key": "AQCNldNk2YgzLBAAA11Q8uAMEgE/bUni+tXIKA==", 2023-08-09T13:48:13.651 INFO:teuthology.orchestra.run.smithi043.stdout: "caps": { 2023-08-09T13:48:13.651 INFO:teuthology.orchestra.run.smithi043.stdout: "mds": "allow *", 2023-08-09T13:48:13.651 INFO:teuthology.orchestra.run.smithi043.stdout: "mon": "allow profile mgr", 2023-08-09T13:48:13.651 INFO:teuthology.orchestra.run.smithi043.stdout: "osd": "allow *" 2023-08-09T13:48:13.651 INFO:teuthology.orchestra.run.smithi043.stdout: } 2023-08-09T13:48:13.651 INFO:teuthology.orchestra.run.smithi043.stdout: } 2023-08-09T13:48:13.652 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2023-08-09T13:48:13.652 INFO:teuthology.orchestra.run.smithi043.stdout:} 2023-08-09T13:48:13.653 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:13.626+0000 7f517359e640 1 -- 172.21.15.43:0/4224293632 >> [v2:172.21.15.203:6832/272917756,v1:172.21.15.203:6833/272917756] conn(0x7f515406a1a0 msgr2=0x7f515406c660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:48:13.653 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:13.626+0000 7f517359e640 1 --2- 172.21.15.43:0/4224293632 >> [v2:172.21.15.203:6832/272917756,v1:172.21.15.203:6833/272917756] conn(0x7f515406a1a0 0x7f515406c660 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7f515c001200 tx=0x7f515c009040 comp rx=0 tx=0).stop 2023-08-09T13:48:13.653 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:13.626+0000 7f517359e640 1 -- 172.21.15.43:0/4224293632 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f51600050e0 msgr2=0x7f51741f1ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:48:13.654 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:13.626+0000 7f517359e640 1 --2- 172.21.15.43:0/4224293632 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f51600050e0 0x7f51741f1ae0 secure :-1 s=READY pgs=823 cs=0 l=1 rev1=1 crypto rx=0x7f516400e5f0 tx=0x7f516400eab0 comp rx=0 tx=0).stop 2023-08-09T13:48:13.654 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:13.626+0000 7f517359e640 1 -- 172.21.15.43:0/4224293632 shutdown_connections 2023-08-09T13:48:13.654 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:13.626+0000 7f517359e640 1 --2- 172.21.15.43:0/4224293632 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f51740abf30 0x7f51740ac3b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:48:13.654 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:13.626+0000 7f517359e640 1 --2- 172.21.15.43:0/4224293632 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f51741f26e0 0x7f5174099aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:48:13.655 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:13.626+0000 7f517359e640 1 --2- 172.21.15.43:0/4224293632 >> [v2:172.21.15.203:6832/272917756,v1:172.21.15.203:6833/272917756] conn(0x7f515406a1a0 0x7f515406c660 unknown :-1 s=CLOSED pgs=48 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:48:13.655 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:13.626+0000 7f517359e640 1 --2- 172.21.15.43:0/4224293632 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f51600050e0 0x7f51741f1ae0 unknown :-1 s=CLOSED pgs=823 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:48:13.655 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:13.626+0000 7f517359e640 1 -- 172.21.15.43:0/4224293632 >> 172.21.15.43:0/4224293632 conn(0x7f51740cde90 msgr2=0x7f51740aad80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-09T13:48:13.655 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:13.626+0000 7f517359e640 1 -- 172.21.15.43:0/4224293632 shutdown_connections 2023-08-09T13:48:13.655 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:13.626+0000 7f517359e640 1 -- 172.21.15.43:0/4224293632 wait complete. 2023-08-09T13:48:13.656 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-08-09T13:48:13.884 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:13.881+0000 7f3561501640 1 Processor -- start 2023-08-09T13:48:13.884 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:13.881+0000 7f3561501640 1 -- start start 2023-08-09T13:48:13.886 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:13.883+0000 7f3561501640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f355c0c86f0 0x7f355c0c8b50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:48:13.886 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:13.883+0000 7f3561501640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f355c06c8d0 0x7f355c06cd40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:48:13.886 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:13.883+0000 7f3561501640 1 -- --> v1:172.21.15.43:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f355c0c9090 con 0x7f355c143c20 2023-08-09T13:48:13.886 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:13.883+0000 7f3561501640 1 -- --> v2:172.21.15.43:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f355c144000 con 0x7f355c0c86f0 2023-08-09T13:48:13.886 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:13.883+0000 7f3561501640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f355c06c440 con 0x7f355c06c8d0 2023-08-09T13:48:13.887 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:13.884+0000 7f3553fff640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f355c0c86f0 0x7f355c0c8b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:48:13.887 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:13.884+0000 7f355affd640 1 --1- >> v1:172.21.15.43:6789/0 conn(0x7f355c143c20 0x7f355c06f540 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.43:6789/0 says I am v1:172.21.15.43:41720/0 (socket says 172.21.15.43:41720) 2023-08-09T13:48:13.887 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:13.884+0000 7f355affd640 1 -- 172.21.15.43:0/3108189065 learned_addr learned my addr 172.21.15.43:0/3108189065 (peer_addr_for_me v1:172.21.15.43:0/0) 2023-08-09T13:48:13.888 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:13.884+0000 7f355b7fe640 1 --2- 172.21.15.43:0/3108189065 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f355c06c8d0 0x7f355c06cd40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:48:13.888 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:13.884+0000 7f3553fff640 1 -- 172.21.15.43:0/3108189065 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f355c06c8d0 msgr2=0x7f355c06cd40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-09T13:48:13.888 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:13.884+0000 7f3553fff640 1 --2- 172.21.15.43:0/3108189065 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f355c06c8d0 0x7f355c06cd40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:48:13.888 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:13.884+0000 7f3553fff640 1 -- 172.21.15.43:0/3108189065 >> v1:172.21.15.43:6789/0 conn(0x7f355c143c20 legacy=0x7f355c06f540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:48:13.888 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:13.884+0000 7f3553fff640 1 -- 172.21.15.43:0/3108189065 --> v2:172.21.15.43:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f355c06fc50 con 0x7f355c0c86f0 2023-08-09T13:48:13.889 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:13.886+0000 7f3553fff640 1 --2- 172.21.15.43:0/3108189065 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f355c0c86f0 0x7f355c0c8b50 secure :-1 s=READY pgs=825 cs=0 l=1 rev1=1 crypto rx=0x7f35440097a0 tx=0x7f354402b820 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c8d747ef380d7696 server_cookie=0 in_seq=0 out_seq=0 2023-08-09T13:48:13.889 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:13.886+0000 7f355a7fc640 1 -- 172.21.15.43:0/3108189065 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3544031780 con 0x7f355c0c86f0 2023-08-09T13:48:13.890 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:13.886+0000 7f355a7fc640 1 -- 172.21.15.43:0/3108189065 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f35440318e0 con 0x7f355c0c86f0 2023-08-09T13:48:13.890 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:13.886+0000 7f355a7fc640 1 -- 172.21.15.43:0/3108189065 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3544031c70 con 0x7f355c0c86f0 2023-08-09T13:48:13.890 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:13.886+0000 7f3561501640 1 -- 172.21.15.43:0/3108189065 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f355c0c86f0 msgr2=0x7f355c0c8b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:48:13.890 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:13.886+0000 7f3561501640 1 --2- 172.21.15.43:0/3108189065 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f355c0c86f0 0x7f355c0c8b50 secure :-1 s=READY pgs=825 cs=0 l=1 rev1=1 crypto rx=0x7f35440097a0 tx=0x7f354402b820 comp rx=0 tx=0).stop 2023-08-09T13:48:13.890 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:13.887+0000 7f3561501640 1 -- 172.21.15.43:0/3108189065 shutdown_connections 2023-08-09T13:48:13.890 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:13.887+0000 7f3561501640 1 --2- 172.21.15.43:0/3108189065 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f355c06c8d0 0x7f355c06cd40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:48:13.890 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:13.887+0000 7f3561501640 1 --2- 172.21.15.43:0/3108189065 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f355c0c86f0 0x7f355c0c8b50 unknown :-1 s=CLOSED pgs=825 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:48:13.891 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:13.887+0000 7f3561501640 1 -- 172.21.15.43:0/3108189065 >> 172.21.15.43:0/3108189065 conn(0x7f355c0ce0a0 msgr2=0x7f355c0ce480 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-09T13:48:13.891 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:13.887+0000 7f3561501640 1 -- 172.21.15.43:0/3108189065 shutdown_connections 2023-08-09T13:48:13.891 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:13.887+0000 7f3561501640 1 -- 172.21.15.43:0/3108189065 wait complete. 2023-08-09T13:48:13.891 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:13.889+0000 7f3561501640 1 Processor -- start 2023-08-09T13:48:13.891 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:13.889+0000 7f3561501640 1 -- start start 2023-08-09T13:48:13.892 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:13.890+0000 7f3561501640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f355c06c8d0 0x7f355c1e2a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:48:13.893 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:13.890+0000 7f3561501640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f355c0c86f0 0x7f355c1f2e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:48:13.893 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:13.890+0000 7f3561501640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f355c143c20 0x7f355c0b31c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:48:13.893 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:13.890+0000 7f3561501640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f355c0c66f0 con 0x7f355c06c8d0 2023-08-09T13:48:13.893 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:13.890+0000 7f3561501640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f355c1f3380 con 0x7f355c0c86f0 2023-08-09T13:48:13.893 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:13.890+0000 7f3561501640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f355c1f35b0 con 0x7f355c143c20 2023-08-09T13:48:13.893 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:13.890+0000 7f355affd640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f355c06c8d0 0x7f355c1e2a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:48:13.893 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:13.890+0000 7f355affd640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f355c06c8d0 0x7f355c1e2a10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:35222/0 (socket says 172.21.15.43:35222) 2023-08-09T13:48:13.894 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:13.890+0000 7f355affd640 1 -- 172.21.15.43:0/4115569870 learned_addr learned my addr 172.21.15.43:0/4115569870 (peer_addr_for_me v2:172.21.15.43:0/0) 2023-08-09T13:48:13.895 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:13.891+0000 7f3553fff640 1 --2- 172.21.15.43:0/4115569870 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f355c0c86f0 0x7f355c1f2e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:48:13.895 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:13.891+0000 7f355b7fe640 1 --2- 172.21.15.43:0/4115569870 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f355c143c20 0x7f355c0b31c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:48:13.895 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:13.891+0000 7f355affd640 1 -- 172.21.15.43:0/4115569870 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f355c143c20 msgr2=0x7f355c0b31c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:48:13.895 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:13.891+0000 7f355affd640 1 --2- 172.21.15.43:0/4115569870 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f355c143c20 0x7f355c0b31c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:48:13.895 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:13.891+0000 7f355affd640 1 -- 172.21.15.43:0/4115569870 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f355c0c86f0 msgr2=0x7f355c1f2e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:48:13.895 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:13.891+0000 7f355affd640 1 --2- 172.21.15.43:0/4115569870 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f355c0c86f0 0x7f355c1f2e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:48:13.895 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:13.891+0000 7f355affd640 1 -- 172.21.15.43:0/4115569870 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3544009400 con 0x7f355c06c8d0 2023-08-09T13:48:13.896 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:13.891+0000 7f3553fff640 1 --2- 172.21.15.43:0/4115569870 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f355c0c86f0 0x7f355c1f2e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-08-09T13:48:13.896 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:13.891+0000 7f355affd640 1 --2- 172.21.15.43:0/4115569870 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f355c06c8d0 0x7f355c1e2a10 secure :-1 s=READY pgs=826 cs=0 l=1 rev1=1 crypto rx=0x7f354c00c4a0 tx=0x7f354c00c960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-09T13:48:13.896 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:13.891+0000 7f35537fe640 1 -- 172.21.15.43:0/4115569870 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f354c005ce0 con 0x7f355c06c8d0 2023-08-09T13:48:13.896 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:13.891+0000 7f35537fe640 1 -- 172.21.15.43:0/4115569870 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f354c005e40 con 0x7f355c06c8d0 2023-08-09T13:48:13.896 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:13.891+0000 7f3561501640 1 -- 172.21.15.43:0/4115569870 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f355c1f3890 con 0x7f355c06c8d0 2023-08-09T13:48:13.896 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:13.891+0000 7f35537fe640 1 -- 172.21.15.43:0/4115569870 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f354c007270 con 0x7f355c06c8d0 2023-08-09T13:48:13.896 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:13.892+0000 7f3561501640 1 -- 172.21.15.43:0/4115569870 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f355c0b3700 con 0x7f355c06c8d0 2023-08-09T13:48:13.897 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:13.893+0000 7f3561501640 1 -- 172.21.15.43:0/4115569870 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f355c0b38f0 con 0x7f355c06c8d0 2023-08-09T13:48:13.897 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:13.894+0000 7f35537fe640 1 -- 172.21.15.43:0/4115569870 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 18) v1 ==== 88443+0+0 (secure 0 0 0) 0x7f354c007490 con 0x7f355c06c8d0 2023-08-09T13:48:13.897 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:13.895+0000 7f35537fe640 1 --2- 172.21.15.43:0/4115569870 >> [v2:172.21.15.203:6832/272917756,v1:172.21.15.203:6833/272917756] conn(0x7f353c06a170 0x7f353c06c630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:48:13.897 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:13.895+0000 7f35537fe640 1 -- 172.21.15.43:0/4115569870 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(70..70 src has 1..70) v4 ==== 5574+0+0 (secure 0 0 0) 0x7f354c0879b0 con 0x7f355c06c8d0 2023-08-09T13:48:13.898 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:13.895+0000 7f3553fff640 1 --2- 172.21.15.43:0/4115569870 >> [v2:172.21.15.203:6832/272917756,v1:172.21.15.203:6833/272917756] conn(0x7f353c06a170 0x7f353c06c630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:48:13.899 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:13.897+0000 7f3553fff640 1 --2- 172.21.15.43:0/4115569870 >> [v2:172.21.15.203:6832/272917756,v1:172.21.15.203:6833/272917756] conn(0x7f353c06a170 0x7f353c06c630 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7f3544036000 tx=0x7f354403d040 comp rx=0 tx=0).ready entity=mgr.5533 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-09T13:48:13.901 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:13.899+0000 7f35537fe640 1 -- 172.21.15.43:0/4115569870 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178066 (secure 0 0 0) 0x7f354c053210 con 0x7f355c06c8d0 2023-08-09T13:48:14.098 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:14.095+0000 7f3561501640 1 -- 172.21.15.43:0/4115569870 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f355c0c6830 con 0x7f355c06c8d0 2023-08-09T13:48:14.099 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:14.096+0000 7f35537fe640 1 -- 172.21.15.43:0/4115569870 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v70) v1 ==== 74+0+14844 (secure 0 0 0) 0x7f355c0c6830 con 0x7f355c06c8d0 2023-08-09T13:48:14.099 INFO:teuthology.orchestra.run.smithi043.stdout: 2023-08-09T13:48:14.099 INFO:teuthology.orchestra.run.smithi043.stdout:{"epoch":70,"fsid":"daac87ee-1337-4250-a955-aea4e408a39b","created":"2023-08-09T13:33:41.701198+0000","modified":"2023-08-09T13:48:13.087909+0000","last_up_change":"2023-08-09T13:33:53.783453+0000","last_in_change":"2023-08-09T13:33:45.148904+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":7,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":"rbd","create_time":"2023-08-09T13:33:54.365137+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":".mgr","create_time":"2023-08-09T13:33:54.831803+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"cbd49ecd-e896-4fbd-864e-5c37a829b165","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":37,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6800","nonce":502667335},{"type":"v1","addr":"172.21.15.43:6801","nonce":502667335}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6802","nonce":502667335},{"type":"v1","addr":"172.21.15.43:6803","nonce":502667335}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6806","nonce":502667335},{"type":"v1","addr":"172.21.15.43:6807","nonce":502667335}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6804","nonce":502667335},{"type":"v1","addr":"172.21.15.43:6805","nonce":502667335}]},"public_addr":"172.21.15.43:6801/502667335","cluster_addr":"172.21.15.43:6803/502667335","heartbeat_back_addr":"172.21.15.43:6807/502667335","heartbeat_front_addr":"172.21.15.43:6805/502667335","state":["exists","up"]},{"osd":1,"uuid":"75c7e0fe-1ec7-4fe8-a2f2-3661a65b8500","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":37,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6808","nonce":3289134610},{"type":"v1","addr":"172.21.15.43:6809","nonce":3289134610}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6810","nonce":3289134610},{"type":"v1","addr":"172.21.15.43:6812","nonce":3289134610}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6821","nonce":3289134610},{"type":"v1","addr":"172.21.15.43:6824","nonce":3289134610}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6815","nonce":3289134610},{"type":"v1","addr":"172.21.15.43:6818","nonce":3289134610}]},"public_addr":"172.21.15.43:6809/3289134610","cluster_addr":"172.21.15.43:6812/3289134610","heartbeat_back_addr":"172.21.15.43:6824/3289134610","heartbeat_front_addr":"172.21.15.43:6818/3289134610","state":["exists","up"]},{"osd":2,"uuid":"bb5c41e9-7f5f-4125-9b01-3bb8bfc2ea83","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":37,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6811","nonce":4079709736},{"type":"v1","addr":"172.21.15.43:6814","nonce":4079709736}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6817","nonce":4079709736},{"type":"v1","addr":"172.21.15.43:6820","nonce":4079709736}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6829","nonce":4079709736},{"type":"v1","addr":"172.21.15.43:6831","nonce":4079709736}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6823","nonce":4079709736},{"type":"v1","addr":"172.21.15.43:6826","nonce":4079709736}]},"public_addr":"172.21.15.43:6814/4079709736","cluster_addr":"172.21.15.43:6820/4079709736","heartbeat_back_addr":"172.21.15.43:6831/4079709736","heartbeat_front_addr":"172.21.15.43:6826/4079709736","state":["exists","up"]},{"osd":3,"uuid":"bfe4c8b4-694a-465c-87e6-2b2aab852613","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":37,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6813","nonce":1709538759},{"type":"v1","addr":"172.21.15.43:6816","nonce":1709538759}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6819","nonce":1709538759},{"type":"v1","addr":"172.21.15.43:6822","nonce":1709538759}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6828","nonce":1709538759},{"type":"v1","addr":"172.21.15.43:6830","nonce":1709538759}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6825","nonce":1709538759},{"type":"v1","addr":"172.21.15.43:6827","nonce":1709538759}]},"public_addr":"172.21.15.43:6816/1709538759","cluster_addr":"172.21.15.43:6822/1709538759","heartbeat_back_addr":"172.21.15.43:6830/1709538759","heartbeat_front_addr":"172.21.15.43:6827/1709538759","state":["exists","up"]},{"osd":4,"uuid":"fa2e8a77-aba1-4756-853e-0d174a5bdeba","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":41,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6800","nonce":4196866481},{"type":"v1","addr":"172.21.15.203:6801","nonce":4196866481}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6802","nonce":4196866481},{"type":"v1","addr":"172.21.15.203:6803","nonce":4196866481}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6806","nonce":4196866481},{"type":"v1","addr":"172.21.15.203:6807","nonce":4196866481}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6804","nonce":4196866481},{"type":"v1","addr":"172.21.15.203:6805","nonce":4196866481}]},"public_addr":"172.21.15.203:6801/4196866481","cluster_addr":"172.21.15.203:6803/4196866481","heartbeat_back_addr":"172.21.15.203:6807/4196866481","heartbeat_front_addr":"172.21.15.203:6805/4196866481","state":["exists","up"]},{"osd":5,"uuid":"81811e4b-4d24-4f51-a82c-f1d700a2a00d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":41,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6816","nonce":3059290081},{"type":"v1","addr":"172.21.15.203:6817","nonce":3059290081}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6818","nonce":3059290081},{"type":"v1","addr":"172.21.15.203:6819","nonce":3059290081}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6822","nonce":3059290081},{"type":"v1","addr":"172.21.15.203:6823","nonce":3059290081}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6820","nonce":3059290081},{"type":"v1","addr":"172.21.15.203:6821","nonce":3059290081}]},"public_addr":"172.21.15.203:6817/3059290081","cluster_addr":"172.21.15.203:6819/3059290081","heartbeat_back_addr":"172.21.15.203:6823/3059290081","heartbeat_front_addr":"172.21.15.203:6821/3059290081","state":["exists","up"]},{"osd":6,"uuid":"307bac4d-6a5b-443d-af75-56d695d7674f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":37,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6824","nonce":512155569},{"type":"v1","addr":"172.21.15.203:6825","nonce":512155569}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6826","nonce":512155569},{"type":"v1","addr":"172.21.15.203:6827","nonce":512155569}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6830","nonce":512155569},{"type":"v1","addr":"172.21.15.203:6831","nonce":512155569}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6828","nonce":512155569},{"type":"v1","addr":"172.21.15.203:6829","nonce":512155569}]},"public_addr":"172.21.15.203:6825/512155569","cluster_addr":"172.21.15.203:6827/512155569","heartbeat_back_addr":"172.21.15.203:6831/512155569","heartbeat_front_addr":"172.21.15.203:6829/512155569","state":["exists","up"]},{"osd":7,"uuid":"a95dc7ca-9f1b-4295-af70-68a22a2d7d5e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":37,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6808","nonce":3331450438},{"type":"v1","addr":"172.21.15.203:6809","nonce":3331450438}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6810","nonce":3331450438},{"type":"v1","addr":"172.21.15.203:6811","nonce":3331450438}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6814","nonce":3331450438},{"type":"v1","addr":"172.21.15.203:6815","nonce":3331450438}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6812","nonce":3331450438},{"type":"v1","addr":"172.21.15.203:6813","nonce":3331450438}]},"public_addr":"172.21.15.203:6809/3331450438","cluster_addr":"172.21.15.203:6811/3331450438","heartbeat_back_addr":"172.21.15.203:6815/3331450438","heartbeat_front_addr":"172.21.15.203:6813/3331450438","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-09T13:33:50.547652+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-09T13:33:50.589469+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-09T13:33:50.561724+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-09T13:33:50.599650+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-09T13:33:50.411855+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-09T13:33:50.219681+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-09T13:33:50.504664+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-09T13:33:50.441248+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":7,"snaps":[{"begin":104,"length":1},{"begin":221,"length":1}]}],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-08-09T13:48:14.101 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:14.099+0000 7f3561501640 1 -- 172.21.15.43:0/4115569870 >> [v2:172.21.15.203:6832/272917756,v1:172.21.15.203:6833/272917756] conn(0x7f353c06a170 msgr2=0x7f353c06c630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:48:14.102 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:14.099+0000 7f3561501640 1 --2- 172.21.15.43:0/4115569870 >> [v2:172.21.15.203:6832/272917756,v1:172.21.15.203:6833/272917756] conn(0x7f353c06a170 0x7f353c06c630 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7f3544036000 tx=0x7f354403d040 comp rx=0 tx=0).stop 2023-08-09T13:48:14.102 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:14.099+0000 7f3561501640 1 -- 172.21.15.43:0/4115569870 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f355c06c8d0 msgr2=0x7f355c1e2a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:48:14.102 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:14.099+0000 7f3561501640 1 --2- 172.21.15.43:0/4115569870 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f355c06c8d0 0x7f355c1e2a10 secure :-1 s=READY pgs=826 cs=0 l=1 rev1=1 crypto rx=0x7f354c00c4a0 tx=0x7f354c00c960 comp rx=0 tx=0).stop 2023-08-09T13:48:14.102 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:14.099+0000 7f3561501640 1 -- 172.21.15.43:0/4115569870 shutdown_connections 2023-08-09T13:48:14.102 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:14.099+0000 7f3561501640 1 --2- 172.21.15.43:0/4115569870 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f355c143c20 0x7f355c0b31c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:48:14.102 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:14.099+0000 7f3561501640 1 --2- 172.21.15.43:0/4115569870 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f355c0c86f0 0x7f355c1f2e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:48:14.103 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:14.099+0000 7f3561501640 1 --2- 172.21.15.43:0/4115569870 >> [v2:172.21.15.203:6832/272917756,v1:172.21.15.203:6833/272917756] conn(0x7f353c06a170 0x7f353c06c630 unknown :-1 s=CLOSED pgs=49 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:48:14.103 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:14.099+0000 7f3561501640 1 --2- 172.21.15.43:0/4115569870 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f355c06c8d0 0x7f355c1e2a10 unknown :-1 s=CLOSED pgs=826 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:48:14.103 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:14.099+0000 7f3561501640 1 -- 172.21.15.43:0/4115569870 >> 172.21.15.43:0/4115569870 conn(0x7f355c0ce0a0 msgr2=0x7f355c06fe40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-09T13:48:14.103 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:14.099+0000 7f3561501640 1 -- 172.21.15.43:0/4115569870 shutdown_connections 2023-08-09T13:48:14.103 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:14.099+0000 7f3561501640 1 -- 172.21.15.43:0/4115569870 wait complete. 2023-08-09T13:48:14.121 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': 'rbd', 'create_time': '2023-08-09T13:33:54.365137+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}, 'read_balance': {'score_acting': 2, 'score_stable': 2, 'optimal_score': 1, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': '.mgr', 'create_time': '2023-08-09T13:33:54.831803+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-08-09T13:48:14.122 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get rbd pg_num 2023-08-09T13:48:14.355 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:14.353+0000 7f4611ecc640 1 Processor -- start 2023-08-09T13:48:14.356 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:14.353+0000 7f4611ecc640 1 -- start start 2023-08-09T13:48:14.358 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:14.355+0000 7f4611ecc640 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7f460c158880 0x7f460c06fa40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:48:14.358 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:14.355+0000 7f4611ecc640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f460c06ff80 0x7f460c0c9180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:48:14.359 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:14.355+0000 7f4611ecc640 1 -- --> v1:172.21.15.203:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f460c0c8cf0 con 0x7f460c0c96c0 2023-08-09T13:48:14.359 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:14.355+0000 7f4611ecc640 1 -- --> v2:172.21.15.43:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f460c0c9b30 con 0x7f460c06ff80 2023-08-09T13:48:14.359 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:14.355+0000 7f4611ecc640 1 -- --> v2:172.21.15.203:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f460c159ab0 con 0x7f460c158880 2023-08-09T13:48:14.359 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:14.355+0000 7f460affd640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f460c06ff80 0x7f460c0c9180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:48:14.360 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:14.355+0000 7f460affd640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f460c06ff80 0x7f460c0c9180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:35232/0 (socket says 172.21.15.43:35232) 2023-08-09T13:48:14.360 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:14.355+0000 7f460affd640 1 -- 172.21.15.43:0/711776044 learned_addr learned my addr 172.21.15.43:0/711776044 (peer_addr_for_me v2:172.21.15.43:0/0) 2023-08-09T13:48:14.360 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:14.355+0000 7f460b7fe640 1 --2- 172.21.15.43:0/711776044 >> v2:172.21.15.203:3300/0 conn(0x7f460c158880 0x7f460c06fa40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:48:14.361 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:14.356+0000 7f460affd640 1 -- 172.21.15.43:0/711776044 >> v2:172.21.15.203:3300/0 conn(0x7f460c158880 msgr2=0x7f460c06fa40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-09T13:48:14.361 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:14.356+0000 7f460affd640 1 --2- 172.21.15.43:0/711776044 >> v2:172.21.15.203:3300/0 conn(0x7f460c158880 0x7f460c06fa40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:48:14.361 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:14.356+0000 7f460affd640 1 -- 172.21.15.43:0/711776044 >> v1:172.21.15.203:6789/0 conn(0x7f460c0c96c0 legacy=0x7f460c0ca960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-09T13:48:14.361 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:14.356+0000 7f460affd640 1 -- 172.21.15.43:0/711776044 --> v2:172.21.15.43:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f460c159d90 con 0x7f460c06ff80 2023-08-09T13:48:14.362 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:14.356+0000 7f460b7fe640 1 --2- 172.21.15.43:0/711776044 >> v2:172.21.15.203:3300/0 conn(0x7f460c158880 0x7f460c06fa40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-08-09T13:48:14.366 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:14.357+0000 7f460affd640 1 --2- 172.21.15.43:0/711776044 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f460c06ff80 0x7f460c0c9180 secure :-1 s=READY pgs=827 cs=0 l=1 rev1=1 crypto rx=0x7f460c06d7e0 tx=0x7f460002b910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1ea3d7c832c85a8a server_cookie=0 in_seq=0 out_seq=0 2023-08-09T13:48:14.366 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:14.357+0000 7f460a7fc640 1 -- 172.21.15.43:0/711776044 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4600031870 con 0x7f460c06ff80 2023-08-09T13:48:14.366 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:14.357+0000 7f460a7fc640 1 -- 172.21.15.43:0/711776044 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f46000319d0 con 0x7f460c06ff80 2023-08-09T13:48:14.366 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:14.357+0000 7f460a7fc640 1 -- 172.21.15.43:0/711776044 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4600031d60 con 0x7f460c06ff80 2023-08-09T13:48:14.367 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:14.358+0000 7f4611ecc640 1 -- 172.21.15.43:0/711776044 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f460c06ff80 msgr2=0x7f460c0c9180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:48:14.367 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:14.358+0000 7f4611ecc640 1 --2- 172.21.15.43:0/711776044 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f460c06ff80 0x7f460c0c9180 secure :-1 s=READY pgs=827 cs=0 l=1 rev1=1 crypto rx=0x7f460c06d7e0 tx=0x7f460002b910 comp rx=0 tx=0).stop 2023-08-09T13:48:14.367 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:14.358+0000 7f4611ecc640 1 -- 172.21.15.43:0/711776044 shutdown_connections 2023-08-09T13:48:14.367 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:14.358+0000 7f4611ecc640 1 --2- 172.21.15.43:0/711776044 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f460c06ff80 0x7f460c0c9180 unknown :-1 s=CLOSED pgs=827 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:48:14.367 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:14.358+0000 7f4611ecc640 1 --2- 172.21.15.43:0/711776044 >> v2:172.21.15.203:3300/0 conn(0x7f460c158880 0x7f460c06fa40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:48:14.367 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:14.358+0000 7f4611ecc640 1 -- 172.21.15.43:0/711776044 >> 172.21.15.43:0/711776044 conn(0x7f460c0aff50 msgr2=0x7f460c0b0360 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-09T13:48:14.367 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:14.358+0000 7f4611ecc640 1 -- 172.21.15.43:0/711776044 shutdown_connections 2023-08-09T13:48:14.367 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:14.358+0000 7f4611ecc640 1 -- 172.21.15.43:0/711776044 wait complete. 2023-08-09T13:48:14.368 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:14.360+0000 7f4611ecc640 1 Processor -- start 2023-08-09T13:48:14.368 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:14.360+0000 7f4611ecc640 1 -- start start 2023-08-09T13:48:14.368 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:14.361+0000 7f4611ecc640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f460c06ff80 0x7f460c151370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:48:14.368 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:14.361+0000 7f4611ecc640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f460c0c96c0 0x7f460c14ee80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:48:14.368 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:14.361+0000 7f4611ecc640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f460c158880 0x7f460c14f3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:48:14.368 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:14.361+0000 7f4611ecc640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f460c150020 con 0x7f460c158880 2023-08-09T13:48:14.368 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:14.361+0000 7f4611ecc640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f460c150300 con 0x7f460c0c96c0 2023-08-09T13:48:14.369 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:14.361+0000 7f4611ecc640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f460c1505e0 con 0x7f460c06ff80 2023-08-09T13:48:14.369 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:14.361+0000 7f460bfff640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f460c158880 0x7f460c14f3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:48:14.369 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:14.361+0000 7f460bfff640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f460c158880 0x7f460c14f3c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:35244/0 (socket says 172.21.15.43:35244) 2023-08-09T13:48:14.369 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:14.361+0000 7f460bfff640 1 -- 172.21.15.43:0/1172774116 learned_addr learned my addr 172.21.15.43:0/1172774116 (peer_addr_for_me v2:172.21.15.43:0/0) 2023-08-09T13:48:14.369 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:14.362+0000 7f460affd640 1 --2- 172.21.15.43:0/1172774116 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f460c0c96c0 0x7f460c14ee80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:48:14.369 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:14.362+0000 7f460b7fe640 1 --2- 172.21.15.43:0/1172774116 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f460c06ff80 0x7f460c151370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:48:14.369 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:14.362+0000 7f460bfff640 1 -- 172.21.15.43:0/1172774116 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f460c06ff80 msgr2=0x7f460c151370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:48:14.370 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:14.362+0000 7f460bfff640 1 --2- 172.21.15.43:0/1172774116 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f460c06ff80 0x7f460c151370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:48:14.370 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:14.362+0000 7f460bfff640 1 -- 172.21.15.43:0/1172774116 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f460c0c96c0 msgr2=0x7f460c14ee80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:48:14.370 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:14.362+0000 7f460bfff640 1 --2- 172.21.15.43:0/1172774116 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f460c0c96c0 0x7f460c14ee80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:48:14.370 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:14.362+0000 7f460bfff640 1 -- 172.21.15.43:0/1172774116 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f46000094d0 con 0x7f460c158880 2023-08-09T13:48:14.370 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:14.362+0000 7f460bfff640 1 --2- 172.21.15.43:0/1172774116 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f460c158880 0x7f460c14f3c0 secure :-1 s=READY pgs=828 cs=0 l=1 rev1=1 crypto rx=0x7f460400b970 tx=0x7f460400be30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-09T13:48:14.370 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:14.362+0000 7f45f3fff640 1 -- 172.21.15.43:0/1172774116 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f460400cb40 con 0x7f460c158880 2023-08-09T13:48:14.370 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:14.362+0000 7f45f3fff640 1 -- 172.21.15.43:0/1172774116 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f460400cca0 con 0x7f460c158880 2023-08-09T13:48:14.370 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:14.362+0000 7f45f3fff640 1 -- 172.21.15.43:0/1172774116 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4604014680 con 0x7f460c158880 2023-08-09T13:48:14.371 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:14.362+0000 7f4611ecc640 1 -- 172.21.15.43:0/1172774116 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f460c1508c0 con 0x7f460c158880 2023-08-09T13:48:14.371 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:14.363+0000 7f4611ecc640 1 -- 172.21.15.43:0/1172774116 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f460c150e80 con 0x7f460c158880 2023-08-09T13:48:14.371 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:14.364+0000 7f4611ecc640 1 -- 172.21.15.43:0/1172774116 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f460c0c65c0 con 0x7f460c158880 2023-08-09T13:48:14.372 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:14.365+0000 7f45f3fff640 1 -- 172.21.15.43:0/1172774116 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 18) v1 ==== 88443+0+0 (secure 0 0 0) 0x7f460401d3b0 con 0x7f460c158880 2023-08-09T13:48:14.372 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:14.366+0000 7f45f3fff640 1 --2- 172.21.15.43:0/1172774116 >> [v2:172.21.15.203:6832/272917756,v1:172.21.15.203:6833/272917756] conn(0x7f45f806a170 0x7f45f806c630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:48:14.372 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:14.366+0000 7f45f3fff640 1 -- 172.21.15.43:0/1172774116 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(70..70 src has 1..70) v4 ==== 5574+0+0 (secure 0 0 0) 0x7f4604088430 con 0x7f460c158880 2023-08-09T13:48:14.372 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:14.366+0000 7f460b7fe640 1 --2- 172.21.15.43:0/1172774116 >> [v2:172.21.15.203:6832/272917756,v1:172.21.15.203:6833/272917756] conn(0x7f45f806a170 0x7f45f806c630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:48:14.372 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:14.368+0000 7f460b7fe640 1 --2- 172.21.15.43:0/1172774116 >> [v2:172.21.15.203:6832/272917756,v1:172.21.15.203:6833/272917756] conn(0x7f45f806a170 0x7f45f806c630 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f460c070950 tx=0x7f45fc008040 comp rx=0 tx=0).ready entity=mgr.5533 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-09T13:48:14.373 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:14.370+0000 7f45f3fff640 1 -- 172.21.15.43:0/1172774116 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178066 (secure 0 0 0) 0x7f4604053b50 con 0x7f460c158880 2023-08-09T13:48:14.568 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:14.565+0000 7f4611ecc640 1 -- 172.21.15.43:0/1172774116 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"} v 0) v1 -- 0x7f460c0739f0 con 0x7f460c158880 2023-08-09T13:48:14.568 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:14.566+0000 7f45f3fff640 1 -- 172.21.15.43:0/1172774116 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"}]=0 v70) v1 ==== 92+0+10 (secure 0 0 0) 0x7f4604057170 con 0x7f460c158880 2023-08-09T13:48:14.569 INFO:teuthology.orchestra.run.smithi043.stdout:pg_num: 8 2023-08-09T13:48:14.572 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:14.569+0000 7f4611ecc640 1 -- 172.21.15.43:0/1172774116 >> [v2:172.21.15.203:6832/272917756,v1:172.21.15.203:6833/272917756] conn(0x7f45f806a170 msgr2=0x7f45f806c630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:48:14.572 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:14.569+0000 7f4611ecc640 1 --2- 172.21.15.43:0/1172774116 >> [v2:172.21.15.203:6832/272917756,v1:172.21.15.203:6833/272917756] conn(0x7f45f806a170 0x7f45f806c630 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f460c070950 tx=0x7f45fc008040 comp rx=0 tx=0).stop 2023-08-09T13:48:14.572 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:14.569+0000 7f4611ecc640 1 -- 172.21.15.43:0/1172774116 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f460c158880 msgr2=0x7f460c14f3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:48:14.572 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:14.569+0000 7f4611ecc640 1 --2- 172.21.15.43:0/1172774116 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f460c158880 0x7f460c14f3c0 secure :-1 s=READY pgs=828 cs=0 l=1 rev1=1 crypto rx=0x7f460400b970 tx=0x7f460400be30 comp rx=0 tx=0).stop 2023-08-09T13:48:14.572 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:14.569+0000 7f4611ecc640 1 -- 172.21.15.43:0/1172774116 shutdown_connections 2023-08-09T13:48:14.573 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:14.569+0000 7f4611ecc640 1 --2- 172.21.15.43:0/1172774116 >> [v2:172.21.15.203:6832/272917756,v1:172.21.15.203:6833/272917756] conn(0x7f45f806a170 0x7f45f806c630 unknown :-1 s=CLOSED pgs=53 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:48:14.573 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:14.569+0000 7f4611ecc640 1 --2- 172.21.15.43:0/1172774116 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f460c158880 0x7f460c14f3c0 unknown :-1 s=CLOSED pgs=828 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:48:14.573 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:14.569+0000 7f4611ecc640 1 --2- 172.21.15.43:0/1172774116 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f460c0c96c0 0x7f460c14ee80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:48:14.573 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:14.569+0000 7f4611ecc640 1 --2- 172.21.15.43:0/1172774116 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f460c06ff80 0x7f460c151370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:48:14.573 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:14.569+0000 7f4611ecc640 1 -- 172.21.15.43:0/1172774116 >> 172.21.15.43:0/1172774116 conn(0x7f460c0aff50 msgr2=0x7f460c0cc0b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-09T13:48:14.573 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:14.569+0000 7f4611ecc640 1 -- 172.21.15.43:0/1172774116 shutdown_connections 2023-08-09T13:48:14.573 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:14.570+0000 7f4611ecc640 1 -- 172.21.15.43:0/1172774116 wait complete. 2023-08-09T13:48:14.591 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-08-09T13:48:14.830 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:14.828+0000 7fb0921e8640 1 Processor -- start 2023-08-09T13:48:14.831 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:14.828+0000 7fb0921e8640 1 -- start start 2023-08-09T13:48:14.833 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:14.830+0000 7fb0921e8640 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7fb08c0b45f0 0x7fb08c0cd6d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:48:14.833 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:14.830+0000 7fb0921e8640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7fb08c0d04b0 0x7fb08c0b59d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:48:14.834 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:14.830+0000 7fb0921e8640 1 -- --> v1:172.21.15.203:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb08c0b7bd0 con 0x7fb08c15c550 2023-08-09T13:48:14.834 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:14.830+0000 7fb0921e8640 1 -- --> v2:172.21.15.43:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb08c0d1c00 con 0x7fb08c0d04b0 2023-08-09T13:48:14.834 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:14.830+0000 7fb0921e8640 1 -- --> v2:172.21.15.203:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb08c15c930 con 0x7fb08c0b45f0 2023-08-09T13:48:14.834 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:14.830+0000 7fb08bfff640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7fb08c0d04b0 0x7fb08c0b59d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:48:14.834 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:14.830+0000 7fb08bfff640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7fb08c0d04b0 0x7fb08c0b59d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:35246/0 (socket says 172.21.15.43:35246) 2023-08-09T13:48:14.834 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:14.830+0000 7fb08bfff640 1 -- 172.21.15.43:0/2883231529 learned_addr learned my addr 172.21.15.43:0/2883231529 (peer_addr_for_me v2:172.21.15.43:0/0) 2023-08-09T13:48:14.834 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:14.831+0000 7fb08bfff640 1 -- 172.21.15.43:0/2883231529 >> v2:172.21.15.203:3300/0 conn(0x7fb08c0b45f0 msgr2=0x7fb08c0cd6d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-09T13:48:14.835 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:14.831+0000 7fb08bfff640 1 --2- 172.21.15.43:0/2883231529 >> v2:172.21.15.203:3300/0 conn(0x7fb08c0b45f0 0x7fb08c0cd6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:48:14.835 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:14.831+0000 7fb08bfff640 1 -- 172.21.15.43:0/2883231529 >> v1:172.21.15.203:6789/0 conn(0x7fb08c15c550 legacy=0x7fb08c0b3ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-09T13:48:14.835 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:14.831+0000 7fb08bfff640 1 -- 172.21.15.43:0/2883231529 --> v2:172.21.15.43:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb08c0b54f0 con 0x7fb08c0d04b0 2023-08-09T13:48:14.836 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:14.832+0000 7fb08bfff640 1 --2- 172.21.15.43:0/2883231529 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb08c0d04b0 0x7fb08c0b59d0 secure :-1 s=READY pgs=829 cs=0 l=1 rev1=1 crypto rx=0x7fb08c0b5050 tx=0x7fb08402b910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fe259b6d356bccfb server_cookie=0 in_seq=0 out_seq=0 2023-08-09T13:48:14.836 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:14.832+0000 7fb08affd640 1 -- 172.21.15.43:0/2883231529 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb084031870 con 0x7fb08c0d04b0 2023-08-09T13:48:14.836 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:14.832+0000 7fb08affd640 1 -- 172.21.15.43:0/2883231529 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb0840319d0 con 0x7fb08c0d04b0 2023-08-09T13:48:14.836 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:14.832+0000 7fb08affd640 1 -- 172.21.15.43:0/2883231529 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb084031d60 con 0x7fb08c0d04b0 2023-08-09T13:48:14.836 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:14.833+0000 7fb0921e8640 1 -- 172.21.15.43:0/2883231529 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb08c0d04b0 msgr2=0x7fb08c0b59d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:48:14.837 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:14.833+0000 7fb0921e8640 1 --2- 172.21.15.43:0/2883231529 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb08c0d04b0 0x7fb08c0b59d0 secure :-1 s=READY pgs=829 cs=0 l=1 rev1=1 crypto rx=0x7fb08c0b5050 tx=0x7fb08402b910 comp rx=0 tx=0).stop 2023-08-09T13:48:14.837 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:14.833+0000 7fb0921e8640 1 -- 172.21.15.43:0/2883231529 shutdown_connections 2023-08-09T13:48:14.837 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:14.833+0000 7fb0921e8640 1 --2- 172.21.15.43:0/2883231529 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb08c0d04b0 0x7fb08c0b59d0 unknown :-1 s=CLOSED pgs=829 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:48:14.837 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:14.833+0000 7fb0921e8640 1 --2- 172.21.15.43:0/2883231529 >> v2:172.21.15.203:3300/0 conn(0x7fb08c0b45f0 0x7fb08c0cd6d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:48:14.837 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:14.833+0000 7fb0921e8640 1 -- 172.21.15.43:0/2883231529 >> 172.21.15.43:0/2883231529 conn(0x7fb08c071520 msgr2=0x7fb08c0ceee0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-09T13:48:14.837 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:14.833+0000 7fb0921e8640 1 -- 172.21.15.43:0/2883231529 shutdown_connections 2023-08-09T13:48:14.837 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:14.833+0000 7fb0921e8640 1 -- 172.21.15.43:0/2883231529 wait complete. 2023-08-09T13:48:14.838 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:14.835+0000 7fb0921e8640 1 Processor -- start 2023-08-09T13:48:14.838 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:14.835+0000 7fb0921e8640 1 -- start start 2023-08-09T13:48:14.839 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:14.836+0000 7fb0921e8640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fb08c0b45f0 0x7fb08c14f0e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:48:14.839 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:14.836+0000 7fb0921e8640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb08c0d04b0 0x7fb08c1f1290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:48:14.839 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:14.836+0000 7fb0921e8640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fb08c15c550 0x7fb08c0ab2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:48:14.839 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:14.836+0000 7fb0921e8640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb08c0a9a30 con 0x7fb08c0d04b0 2023-08-09T13:48:14.839 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:14.836+0000 7fb0921e8640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb08c1f17d0 con 0x7fb08c15c550 2023-08-09T13:48:14.840 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:14.836+0000 7fb0921e8640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb08c0a8e60 con 0x7fb08c0b45f0 2023-08-09T13:48:14.840 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:14.836+0000 7fb082ffd640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb08c0d04b0 0x7fb08c1f1290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:48:14.840 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:14.837+0000 7fb082ffd640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb08c0d04b0 0x7fb08c1f1290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:35252/0 (socket says 172.21.15.43:35252) 2023-08-09T13:48:14.840 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:14.837+0000 7fb082ffd640 1 -- 172.21.15.43:0/1834914786 learned_addr learned my addr 172.21.15.43:0/1834914786 (peer_addr_for_me v2:172.21.15.43:0/0) 2023-08-09T13:48:14.841 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:14.837+0000 7fb08b7fe640 1 --2- 172.21.15.43:0/1834914786 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fb08c0b45f0 0x7fb08c14f0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:48:14.841 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:14.837+0000 7fb08bfff640 1 --2- 172.21.15.43:0/1834914786 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fb08c15c550 0x7fb08c0ab2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:48:14.841 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:14.837+0000 7fb082ffd640 1 -- 172.21.15.43:0/1834914786 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fb08c0b45f0 msgr2=0x7fb08c14f0e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:48:14.841 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:14.837+0000 7fb082ffd640 1 --2- 172.21.15.43:0/1834914786 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fb08c0b45f0 0x7fb08c14f0e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:48:14.842 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:14.837+0000 7fb082ffd640 1 -- 172.21.15.43:0/1834914786 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fb08c15c550 msgr2=0x7fb08c0ab2f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:48:14.842 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:14.837+0000 7fb082ffd640 1 --2- 172.21.15.43:0/1834914786 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fb08c15c550 0x7fb08c0ab2f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:48:14.842 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:14.837+0000 7fb082ffd640 1 -- 172.21.15.43:0/1834914786 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb0840094d0 con 0x7fb08c0d04b0 2023-08-09T13:48:14.842 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:14.837+0000 7fb082ffd640 1 --2- 172.21.15.43:0/1834914786 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb08c0d04b0 0x7fb08c1f1290 secure :-1 s=READY pgs=830 cs=0 l=1 rev1=1 crypto rx=0x7fb07400b4a0 tx=0x7fb07400b960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-09T13:48:14.842 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:14.838+0000 7fb088ff9640 1 -- 172.21.15.43:0/1834914786 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb074010610 con 0x7fb08c0d04b0 2023-08-09T13:48:14.842 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:14.838+0000 7fb088ff9640 1 -- 172.21.15.43:0/1834914786 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb074010770 con 0x7fb08c0d04b0 2023-08-09T13:48:14.842 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:14.838+0000 7fb088ff9640 1 -- 172.21.15.43:0/1834914786 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb074010b20 con 0x7fb08c0d04b0 2023-08-09T13:48:14.843 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:14.838+0000 7fb0921e8640 1 -- 172.21.15.43:0/1834914786 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb08c0a9100 con 0x7fb08c0d04b0 2023-08-09T13:48:14.843 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:14.838+0000 7fb0921e8640 1 -- 172.21.15.43:0/1834914786 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb08c0a96c0 con 0x7fb08c0d04b0 2023-08-09T13:48:14.843 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:14.839+0000 7fb0921e8640 1 -- 172.21.15.43:0/1834914786 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb08c0a9ba0 con 0x7fb08c0d04b0 2023-08-09T13:48:14.843 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:14.840+0000 7fb088ff9640 1 -- 172.21.15.43:0/1834914786 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 18) v1 ==== 88443+0+0 (secure 0 0 0) 0x7fb074010c80 con 0x7fb08c0d04b0 2023-08-09T13:48:14.844 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:14.841+0000 7fb088ff9640 1 --2- 172.21.15.43:0/1834914786 >> [v2:172.21.15.203:6832/272917756,v1:172.21.15.203:6833/272917756] conn(0x7fb06c06a1a0 0x7fb06c06c660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:48:14.844 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:14.841+0000 7fb088ff9640 1 -- 172.21.15.43:0/1834914786 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(70..70 src has 1..70) v4 ==== 5574+0+0 (secure 0 0 0) 0x7fb074088170 con 0x7fb08c0d04b0 2023-08-09T13:48:14.844 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:14.842+0000 7fb08b7fe640 1 --2- 172.21.15.43:0/1834914786 >> [v2:172.21.15.203:6832/272917756,v1:172.21.15.203:6833/272917756] conn(0x7fb06c06a1a0 0x7fb06c06c660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:48:14.845 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:14.843+0000 7fb08b7fe640 1 --2- 172.21.15.43:0/1834914786 >> [v2:172.21.15.203:6832/272917756,v1:172.21.15.203:6833/272917756] conn(0x7fb06c06a1a0 0x7fb06c06c660 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7fb07c009f60 tx=0x7fb07c0094f0 comp rx=0 tx=0).ready entity=mgr.5533 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-09T13:48:14.848 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:14.845+0000 7fb088ff9640 1 -- 172.21.15.43:0/1834914786 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178066 (secure 0 0 0) 0x7fb074053920 con 0x7fb08c0d04b0 2023-08-09T13:48:15.049 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:15.046+0000 7fb0921e8640 1 -- 172.21.15.43:0/1834914786 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7fb08c0739f0 con 0x7fb08c0d04b0 2023-08-09T13:48:15.049 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:15.047+0000 7fb088ff9640 1 -- 172.21.15.43:0/1834914786 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v70) v1 ==== 93+0+10 (secure 0 0 0) 0x7fb074056f40 con 0x7fb08c0d04b0 2023-08-09T13:48:15.050 INFO:teuthology.orchestra.run.smithi043.stdout:pg_num: 1 2023-08-09T13:48:15.052 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:15.050+0000 7fb0921e8640 1 -- 172.21.15.43:0/1834914786 >> [v2:172.21.15.203:6832/272917756,v1:172.21.15.203:6833/272917756] conn(0x7fb06c06a1a0 msgr2=0x7fb06c06c660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:48:15.053 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:15.050+0000 7fb0921e8640 1 --2- 172.21.15.43:0/1834914786 >> [v2:172.21.15.203:6832/272917756,v1:172.21.15.203:6833/272917756] conn(0x7fb06c06a1a0 0x7fb06c06c660 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7fb07c009f60 tx=0x7fb07c0094f0 comp rx=0 tx=0).stop 2023-08-09T13:48:15.053 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:15.050+0000 7fb0921e8640 1 -- 172.21.15.43:0/1834914786 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb08c0d04b0 msgr2=0x7fb08c1f1290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:48:15.053 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:15.050+0000 7fb0921e8640 1 --2- 172.21.15.43:0/1834914786 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb08c0d04b0 0x7fb08c1f1290 secure :-1 s=READY pgs=830 cs=0 l=1 rev1=1 crypto rx=0x7fb07400b4a0 tx=0x7fb07400b960 comp rx=0 tx=0).stop 2023-08-09T13:48:15.053 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:15.050+0000 7fb0921e8640 1 -- 172.21.15.43:0/1834914786 shutdown_connections 2023-08-09T13:48:15.053 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:15.050+0000 7fb0921e8640 1 --2- 172.21.15.43:0/1834914786 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fb08c15c550 0x7fb08c0ab2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:48:15.053 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:15.050+0000 7fb0921e8640 1 --2- 172.21.15.43:0/1834914786 >> [v2:172.21.15.203:6832/272917756,v1:172.21.15.203:6833/272917756] conn(0x7fb06c06a1a0 0x7fb06c06c660 unknown :-1 s=CLOSED pgs=54 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:48:15.053 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:15.050+0000 7fb0921e8640 1 --2- 172.21.15.43:0/1834914786 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb08c0d04b0 0x7fb08c1f1290 unknown :-1 s=CLOSED pgs=830 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:48:15.054 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:15.050+0000 7fb0921e8640 1 --2- 172.21.15.43:0/1834914786 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fb08c0b45f0 0x7fb08c14f0e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:48:15.054 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:15.050+0000 7fb0921e8640 1 -- 172.21.15.43:0/1834914786 >> 172.21.15.43:0/1834914786 conn(0x7fb08c071520 msgr2=0x7fb08c0c7f80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-09T13:48:15.054 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:15.050+0000 7fb0921e8640 1 -- 172.21.15.43:0/1834914786 shutdown_connections 2023-08-09T13:48:15.054 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:15.050+0000 7fb0921e8640 1 -- 172.21.15.43:0/1834914786 wait complete. 2023-08-09T13:48:15.074 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --format=json-pretty osd lspools 2023-08-09T13:48:15.291 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:15.289+0000 7fdb0bfff640 1 Processor -- start 2023-08-09T13:48:15.291 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:15.289+0000 7fdb0bfff640 1 -- start start 2023-08-09T13:48:15.293 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:15.291+0000 7fdb0bfff640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7fdb0c15f7b0 0x7fdb0c0b4760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:48:15.293 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:15.291+0000 7fdb0bfff640 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7fdb0c0b1050 0x7fdb0c0ccfa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:48:15.293 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:15.291+0000 7fdb0bfff640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fdb0c0cd4e0 0x7fdb0c0ceae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:48:15.294 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:15.291+0000 7fdb0bfff640 1 -- --> v2:172.21.15.43:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fdb0c0cd970 con 0x7fdb0c15f7b0 2023-08-09T13:48:15.294 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:15.291+0000 7fdb0bfff640 1 -- --> v2:172.21.15.203:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fdb0c0b7bf0 con 0x7fdb0c0b1050 2023-08-09T13:48:15.294 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:15.291+0000 7fdb0bfff640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdb0c0b7ed0 con 0x7fdb0c0cd4e0 2023-08-09T13:48:15.294 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:15.291+0000 7fdb0affd640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7fdb0c15f7b0 0x7fdb0c0b4760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:48:15.295 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:15.291+0000 7fdb0affd640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7fdb0c15f7b0 0x7fdb0c0b4760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:36666/0 (socket says 172.21.15.43:36666) 2023-08-09T13:48:15.295 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:15.291+0000 7fdb0affd640 1 -- 172.21.15.43:0/572237791 learned_addr learned my addr 172.21.15.43:0/572237791 (peer_addr_for_me v2:172.21.15.43:0/0) 2023-08-09T13:48:15.295 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:15.291+0000 7fdb0a7fc640 1 --2- 172.21.15.43:0/572237791 >> v2:172.21.15.203:3300/0 conn(0x7fdb0c0b1050 0x7fdb0c0ccfa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:48:15.295 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:15.291+0000 7fdb0affd640 1 -- 172.21.15.43:0/572237791 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fdb0c0cd4e0 msgr2=0x7fdb0c0ceae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-09T13:48:15.295 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:15.292+0000 7fdb0affd640 1 --2- 172.21.15.43:0/572237791 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fdb0c0cd4e0 0x7fdb0c0ceae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:48:15.295 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:15.292+0000 7fdb0affd640 1 -- 172.21.15.43:0/572237791 >> v2:172.21.15.203:3300/0 conn(0x7fdb0c0b1050 msgr2=0x7fdb0c0ccfa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-09T13:48:15.295 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:15.292+0000 7fdb0affd640 1 --2- 172.21.15.43:0/572237791 >> v2:172.21.15.203:3300/0 conn(0x7fdb0c0b1050 0x7fdb0c0ccfa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:48:15.296 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:15.292+0000 7fdb0affd640 1 -- 172.21.15.43:0/572237791 --> v2:172.21.15.43:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdb0c0d0370 con 0x7fdb0c15f7b0 2023-08-09T13:48:15.296 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:15.292+0000 7fdb0a7fc640 1 --2- 172.21.15.43:0/572237791 >> v2:172.21.15.203:3300/0 conn(0x7fdb0c0b1050 0x7fdb0c0ccfa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-08-09T13:48:15.297 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:15.293+0000 7fdb0affd640 1 --2- 172.21.15.43:0/572237791 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fdb0c15f7b0 0x7fdb0c0b4760 secure :-1 s=READY pgs=831 cs=0 l=1 rev1=1 crypto rx=0x7fdb0c0ce9d0 tx=0x7fdafc02bee0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c46d01dc8f78f45 server_cookie=0 in_seq=0 out_seq=0 2023-08-09T13:48:15.297 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:15.293+0000 7fdb09ffb640 1 -- 172.21.15.43:0/572237791 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdafc02ca80 con 0x7fdb0c15f7b0 2023-08-09T13:48:15.297 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:15.293+0000 7fdb09ffb640 1 -- 172.21.15.43:0/572237791 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdafc02cbe0 con 0x7fdb0c15f7b0 2023-08-09T13:48:15.297 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:15.293+0000 7fdb09ffb640 1 -- 172.21.15.43:0/572237791 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdafc0343f0 con 0x7fdb0c15f7b0 2023-08-09T13:48:15.297 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:15.294+0000 7fdb0bfff640 1 -- 172.21.15.43:0/572237791 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fdb0c15f7b0 msgr2=0x7fdb0c0b4760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:48:15.298 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:15.294+0000 7fdb0bfff640 1 --2- 172.21.15.43:0/572237791 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fdb0c15f7b0 0x7fdb0c0b4760 secure :-1 s=READY pgs=831 cs=0 l=1 rev1=1 crypto rx=0x7fdb0c0ce9d0 tx=0x7fdafc02bee0 comp rx=0 tx=0).stop 2023-08-09T13:48:15.298 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:15.294+0000 7fdb0bfff640 1 -- 172.21.15.43:0/572237791 shutdown_connections 2023-08-09T13:48:15.298 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:15.294+0000 7fdb0bfff640 1 --2- 172.21.15.43:0/572237791 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fdb0c0cd4e0 0x7fdb0c0ceae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:48:15.298 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:15.294+0000 7fdb0bfff640 1 --2- 172.21.15.43:0/572237791 >> v2:172.21.15.203:3300/0 conn(0x7fdb0c0b1050 0x7fdb0c0ccfa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:48:15.298 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:15.294+0000 7fdb0bfff640 1 --2- 172.21.15.43:0/572237791 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fdb0c15f7b0 0x7fdb0c0b4760 unknown :-1 s=CLOSED pgs=831 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:48:15.298 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:15.294+0000 7fdb0bfff640 1 -- 172.21.15.43:0/572237791 >> 172.21.15.43:0/572237791 conn(0x7fdb0c06a2b0 msgr2=0x7fdb0c06a690 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-09T13:48:15.298 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:15.294+0000 7fdb0bfff640 1 -- 172.21.15.43:0/572237791 shutdown_connections 2023-08-09T13:48:15.298 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:15.294+0000 7fdb0bfff640 1 -- 172.21.15.43:0/572237791 wait complete. 2023-08-09T13:48:15.299 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:15.296+0000 7fdb0bfff640 1 Processor -- start 2023-08-09T13:48:15.299 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:15.296+0000 7fdb0bfff640 1 -- start start 2023-08-09T13:48:15.300 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:15.297+0000 7fdb0bfff640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fdb0c0b1050 0x7fdb0c0c6db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:48:15.300 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:15.297+0000 7fdb0bfff640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fdb0c0cd4e0 0x7fdb0c1529e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:48:15.300 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:15.297+0000 7fdb0bfff640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fdb0c15f7b0 0x7fdb0c0b6580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:48:15.300 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:15.297+0000 7fdb0bfff640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdb0c0cb0e0 con 0x7fdb0c0b1050 2023-08-09T13:48:15.300 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:15.297+0000 7fdb0bfff640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdb0c0cb3c0 con 0x7fdb0c0cd4e0 2023-08-09T13:48:15.300 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:15.297+0000 7fdb0bfff640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdb0c0b1b60 con 0x7fdb0c15f7b0 2023-08-09T13:48:15.301 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:15.298+0000 7fdb0affd640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fdb0c0b1050 0x7fdb0c0c6db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:48:15.301 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:15.298+0000 7fdb0affd640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fdb0c0b1050 0x7fdb0c0c6db0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:36668/0 (socket says 172.21.15.43:36668) 2023-08-09T13:48:15.301 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:15.298+0000 7fdb0affd640 1 -- 172.21.15.43:0/4143609313 learned_addr learned my addr 172.21.15.43:0/4143609313 (peer_addr_for_me v2:172.21.15.43:0/0) 2023-08-09T13:48:15.302 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:15.298+0000 7fdb0b7fe640 1 --2- 172.21.15.43:0/4143609313 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fdb0c15f7b0 0x7fdb0c0b6580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:48:15.302 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:15.298+0000 7fdb0a7fc640 1 --2- 172.21.15.43:0/4143609313 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fdb0c0cd4e0 0x7fdb0c1529e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:48:15.302 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:15.298+0000 7fdb0affd640 1 -- 172.21.15.43:0/4143609313 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fdb0c15f7b0 msgr2=0x7fdb0c0b6580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:48:15.302 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:15.298+0000 7fdb0affd640 1 --2- 172.21.15.43:0/4143609313 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fdb0c15f7b0 0x7fdb0c0b6580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:48:15.302 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:15.298+0000 7fdb0affd640 1 -- 172.21.15.43:0/4143609313 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fdb0c0cd4e0 msgr2=0x7fdb0c1529e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:48:15.303 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:15.298+0000 7fdb0affd640 1 --2- 172.21.15.43:0/4143609313 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fdb0c0cd4e0 0x7fdb0c1529e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:48:15.303 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:15.298+0000 7fdb0affd640 1 -- 172.21.15.43:0/4143609313 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdafc009b30 con 0x7fdb0c0b1050 2023-08-09T13:48:15.303 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:15.298+0000 7fdb0b7fe640 1 --2- 172.21.15.43:0/4143609313 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fdb0c15f7b0 0x7fdb0c0b6580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-08-09T13:48:15.303 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:15.298+0000 7fdb0a7fc640 1 --2- 172.21.15.43:0/4143609313 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fdb0c0cd4e0 0x7fdb0c1529e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-08-09T13:48:15.303 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:15.298+0000 7fdb0affd640 1 --2- 172.21.15.43:0/4143609313 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fdb0c0b1050 0x7fdb0c0c6db0 secure :-1 s=READY pgs=832 cs=0 l=1 rev1=1 crypto rx=0x7fdafc000f80 tx=0x7fdafc004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-09T13:48:15.303 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:15.299+0000 7fdaf37fe640 1 -- 172.21.15.43:0/4143609313 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdafc03e450 con 0x7fdb0c0b1050 2023-08-09T13:48:15.303 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:15.299+0000 7fdaf37fe640 1 -- 172.21.15.43:0/4143609313 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdafc03e5f0 con 0x7fdb0c0b1050 2023-08-09T13:48:15.303 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:15.299+0000 7fdaf37fe640 1 -- 172.21.15.43:0/4143609313 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdafc03e9a0 con 0x7fdb0c0b1050 2023-08-09T13:48:15.304 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:15.299+0000 7fdb0bfff640 1 -- 172.21.15.43:0/4143609313 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdb0c0b1e40 con 0x7fdb0c0b1050 2023-08-09T13:48:15.304 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:15.299+0000 7fdb0bfff640 1 -- 172.21.15.43:0/4143609313 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdb0c152f20 con 0x7fdb0c0b1050 2023-08-09T13:48:15.304 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:15.300+0000 7fdb0bfff640 1 -- 172.21.15.43:0/4143609313 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdb0c0b2360 con 0x7fdb0c0b1050 2023-08-09T13:48:15.304 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:15.301+0000 7fdaf37fe640 1 -- 172.21.15.43:0/4143609313 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 18) v1 ==== 88443+0+0 (secure 0 0 0) 0x7fdafc02cd50 con 0x7fdb0c0b1050 2023-08-09T13:48:15.305 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:15.302+0000 7fdaf37fe640 1 --2- 172.21.15.43:0/4143609313 >> [v2:172.21.15.203:6832/272917756,v1:172.21.15.203:6833/272917756] conn(0x7fdaf806a170 0x7fdaf806c630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:48:15.305 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:15.302+0000 7fdaf37fe640 1 -- 172.21.15.43:0/4143609313 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(70..70 src has 1..70) v4 ==== 5574+0+0 (secure 0 0 0) 0x7fdafc0ab210 con 0x7fdb0c0b1050 2023-08-09T13:48:15.305 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:15.303+0000 7fdb0a7fc640 1 --2- 172.21.15.43:0/4143609313 >> [v2:172.21.15.203:6832/272917756,v1:172.21.15.203:6833/272917756] conn(0x7fdaf806a170 0x7fdaf806c630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:48:15.306 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:15.304+0000 7fdb0a7fc640 1 --2- 172.21.15.43:0/4143609313 >> [v2:172.21.15.203:6832/272917756,v1:172.21.15.203:6833/272917756] conn(0x7fdaf806a170 0x7fdaf806c630 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7fdb0c153ca0 tx=0x7fdb00008040 comp rx=0 tx=0).ready entity=mgr.5533 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-09T13:48:15.309 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:15.307+0000 7fdaf37fe640 1 -- 172.21.15.43:0/4143609313 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178066 (secure 0 0 0) 0x7fdb0c0b2360 con 0x7fdb0c0b1050 2023-08-09T13:48:15.507 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:15.504+0000 7fdb0bfff640 1 -- 172.21.15.43:0/4143609313 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) v1 -- 0x7fdb0c0cb530 con 0x7fdb0c0b1050 2023-08-09T13:48:15.507 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:15.505+0000 7fdaf37fe640 1 -- 172.21.15.43:0/4143609313 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v70) v1 ==== 84+0+126 (secure 0 0 0) 0x7fdafc076a50 con 0x7fdb0c0b1050 2023-08-09T13:48:15.508 INFO:teuthology.orchestra.run.smithi043.stdout: 2023-08-09T13:48:15.508 INFO:teuthology.orchestra.run.smithi043.stdout:[ 2023-08-09T13:48:15.508 INFO:teuthology.orchestra.run.smithi043.stdout: { 2023-08-09T13:48:15.508 INFO:teuthology.orchestra.run.smithi043.stdout: "poolnum": 1, 2023-08-09T13:48:15.508 INFO:teuthology.orchestra.run.smithi043.stdout: "poolname": "rbd" 2023-08-09T13:48:15.509 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2023-08-09T13:48:15.509 INFO:teuthology.orchestra.run.smithi043.stdout: { 2023-08-09T13:48:15.509 INFO:teuthology.orchestra.run.smithi043.stdout: "poolnum": 2, 2023-08-09T13:48:15.509 INFO:teuthology.orchestra.run.smithi043.stdout: "poolname": ".mgr" 2023-08-09T13:48:15.509 INFO:teuthology.orchestra.run.smithi043.stdout: } 2023-08-09T13:48:15.509 INFO:teuthology.orchestra.run.smithi043.stdout:] 2023-08-09T13:48:15.511 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:15.508+0000 7fdb0bfff640 1 -- 172.21.15.43:0/4143609313 >> [v2:172.21.15.203:6832/272917756,v1:172.21.15.203:6833/272917756] conn(0x7fdaf806a170 msgr2=0x7fdaf806c630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:48:15.511 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:15.508+0000 7fdb0bfff640 1 --2- 172.21.15.43:0/4143609313 >> [v2:172.21.15.203:6832/272917756,v1:172.21.15.203:6833/272917756] conn(0x7fdaf806a170 0x7fdaf806c630 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7fdb0c153ca0 tx=0x7fdb00008040 comp rx=0 tx=0).stop 2023-08-09T13:48:15.511 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:15.508+0000 7fdb0bfff640 1 -- 172.21.15.43:0/4143609313 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fdb0c0b1050 msgr2=0x7fdb0c0c6db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:48:15.511 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:15.508+0000 7fdb0bfff640 1 --2- 172.21.15.43:0/4143609313 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fdb0c0b1050 0x7fdb0c0c6db0 secure :-1 s=READY pgs=832 cs=0 l=1 rev1=1 crypto rx=0x7fdafc000f80 tx=0x7fdafc004a40 comp rx=0 tx=0).stop 2023-08-09T13:48:15.511 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:15.508+0000 7fdb0bfff640 1 -- 172.21.15.43:0/4143609313 shutdown_connections 2023-08-09T13:48:15.511 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:15.508+0000 7fdb0bfff640 1 --2- 172.21.15.43:0/4143609313 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fdb0c15f7b0 0x7fdb0c0b6580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:48:15.511 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:15.508+0000 7fdb0bfff640 1 --2- 172.21.15.43:0/4143609313 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fdb0c0cd4e0 0x7fdb0c1529e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:48:15.512 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:15.508+0000 7fdb0bfff640 1 --2- 172.21.15.43:0/4143609313 >> [v2:172.21.15.203:6832/272917756,v1:172.21.15.203:6833/272917756] conn(0x7fdaf806a170 0x7fdaf806c630 unknown :-1 s=CLOSED pgs=55 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:48:15.512 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:15.508+0000 7fdb0bfff640 1 --2- 172.21.15.43:0/4143609313 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fdb0c0b1050 0x7fdb0c0c6db0 unknown :-1 s=CLOSED pgs=832 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:48:15.512 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:15.508+0000 7fdb0bfff640 1 -- 172.21.15.43:0/4143609313 >> 172.21.15.43:0/4143609313 conn(0x7fdb0c06a2b0 msgr2=0x7fdb0c0d24a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-09T13:48:15.512 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:15.508+0000 7fdb0bfff640 1 -- 172.21.15.43:0/4143609313 shutdown_connections 2023-08-09T13:48:15.512 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:15.508+0000 7fdb0bfff640 1 -- 172.21.15.43:0/4143609313 wait complete. 2023-08-09T13:48:15.530 DEBUG:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2023-08-09T13:48:15.530 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_metadata --pg_num_min 64 2023-08-09T13:48:15.762 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:15.760+0000 7f1bcbf0f640 1 Processor -- start 2023-08-09T13:48:15.763 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:15.760+0000 7f1bcbf0f640 1 -- start start 2023-08-09T13:48:15.764 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:15.761+0000 7f1bcbf0f640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f1bc406fb70 0x7f1bc40caab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:48:15.764 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:15.761+0000 7f1bcbf0f640 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7f1bc40ca370 0x7f1bc415b2a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:48:15.764 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:15.761+0000 7f1bcbf0f640 1 -- --> v1:172.21.15.203:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1bc415ae10 con 0x7f1bc40ce4c0 2023-08-09T13:48:15.765 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:15.762+0000 7f1bcbf0f640 1 -- --> v2:172.21.15.43:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1bc4154e70 con 0x7f1bc406fb70 2023-08-09T13:48:15.765 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:15.762+0000 7f1bcbf0f640 1 -- --> v2:172.21.15.203:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1bc406ff50 con 0x7f1bc40ca370 2023-08-09T13:48:15.765 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:15.762+0000 7f1bc9cab640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f1bc406fb70 0x7f1bc40caab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:48:15.765 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:15.762+0000 7f1bc9cab640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f1bc406fb70 0x7f1bc40caab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:36676/0 (socket says 172.21.15.43:36676) 2023-08-09T13:48:15.765 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:15.762+0000 7f1bc9cab640 1 -- 172.21.15.43:0/684739089 learned_addr learned my addr 172.21.15.43:0/684739089 (peer_addr_for_me v2:172.21.15.43:0/0) 2023-08-09T13:48:15.766 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:15.762+0000 7f1bc9cab640 1 -- 172.21.15.43:0/684739089 >> v2:172.21.15.203:3300/0 conn(0x7f1bc40ca370 msgr2=0x7f1bc415b2a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-09T13:48:15.766 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:15.762+0000 7f1bc9cab640 1 --2- 172.21.15.43:0/684739089 >> v2:172.21.15.203:3300/0 conn(0x7f1bc40ca370 0x7f1bc415b2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:48:15.766 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:15.762+0000 7f1bc9cab640 1 -- 172.21.15.43:0/684739089 >> v1:172.21.15.203:6789/0 conn(0x7f1bc40ce4c0 legacy=0x7f1bc40c86f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-09T13:48:15.766 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:15.762+0000 7f1bc9cab640 1 -- 172.21.15.43:0/684739089 --> v2:172.21.15.43:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1bc40cb320 con 0x7f1bc406fb70 2023-08-09T13:48:15.767 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:15.764+0000 7f1bc9cab640 1 --2- 172.21.15.43:0/684739089 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f1bc406fb70 0x7f1bc40caab0 secure :-1 s=READY pgs=833 cs=0 l=1 rev1=1 crypto rx=0x7f1bc40711a0 tx=0x7f1bc002b910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c4143e1909a417d8 server_cookie=0 in_seq=0 out_seq=0 2023-08-09T13:48:15.767 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:15.764+0000 7f1bc8ca9640 1 -- 172.21.15.43:0/684739089 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1bc0031870 con 0x7f1bc406fb70 2023-08-09T13:48:15.767 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:15.764+0000 7f1bc8ca9640 1 -- 172.21.15.43:0/684739089 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1bc00319d0 con 0x7f1bc406fb70 2023-08-09T13:48:15.767 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:15.764+0000 7f1bc8ca9640 1 -- 172.21.15.43:0/684739089 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1bc0031d60 con 0x7f1bc406fb70 2023-08-09T13:48:15.768 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:15.765+0000 7f1bcbf0f640 1 -- 172.21.15.43:0/684739089 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f1bc406fb70 msgr2=0x7f1bc40caab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:48:15.768 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:15.765+0000 7f1bcbf0f640 1 --2- 172.21.15.43:0/684739089 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f1bc406fb70 0x7f1bc40caab0 secure :-1 s=READY pgs=833 cs=0 l=1 rev1=1 crypto rx=0x7f1bc40711a0 tx=0x7f1bc002b910 comp rx=0 tx=0).stop 2023-08-09T13:48:15.768 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:15.765+0000 7f1bcbf0f640 1 -- 172.21.15.43:0/684739089 shutdown_connections 2023-08-09T13:48:15.768 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:15.765+0000 7f1bcbf0f640 1 --2- 172.21.15.43:0/684739089 >> v2:172.21.15.203:3300/0 conn(0x7f1bc40ca370 0x7f1bc415b2a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:48:15.769 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:15.765+0000 7f1bcbf0f640 1 --2- 172.21.15.43:0/684739089 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f1bc406fb70 0x7f1bc40caab0 unknown :-1 s=CLOSED pgs=833 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:48:15.769 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:15.765+0000 7f1bcbf0f640 1 -- 172.21.15.43:0/684739089 >> 172.21.15.43:0/684739089 conn(0x7f1bc406b990 msgr2=0x7f1bc406bda0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-09T13:48:15.769 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:15.765+0000 7f1bcbf0f640 1 -- 172.21.15.43:0/684739089 shutdown_connections 2023-08-09T13:48:15.769 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:15.765+0000 7f1bcbf0f640 1 -- 172.21.15.43:0/684739089 wait complete. 2023-08-09T13:48:15.769 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:15.767+0000 7f1bcbf0f640 1 Processor -- start 2023-08-09T13:48:15.770 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:15.767+0000 7f1bcbf0f640 1 -- start start 2023-08-09T13:48:15.771 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:15.768+0000 7f1bcbf0f640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f1bc406fb70 0x7f1bc41f0d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:48:15.771 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:15.768+0000 7f1bcbf0f640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f1bc40ca370 0x7f1bc4146e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:48:15.771 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:15.768+0000 7f1bcbf0f640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f1bc40ce4c0 0x7f1bc41595e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:48:15.771 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:15.768+0000 7f1bcbf0f640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1bc41ed780 con 0x7f1bc40ce4c0 2023-08-09T13:48:15.771 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:15.768+0000 7f1bcbf0f640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1bc40c3140 con 0x7f1bc40ca370 2023-08-09T13:48:15.771 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:15.768+0000 7f1bcbf0f640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1bc40c3420 con 0x7f1bc406fb70 2023-08-09T13:48:15.771 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:15.768+0000 7f1bca4ac640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f1bc40ce4c0 0x7f1bc41595e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:48:15.772 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:15.769+0000 7f1bca4ac640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f1bc40ce4c0 0x7f1bc41595e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:36680/0 (socket says 172.21.15.43:36680) 2023-08-09T13:48:15.772 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:15.769+0000 7f1bca4ac640 1 -- 172.21.15.43:0/971023147 learned_addr learned my addr 172.21.15.43:0/971023147 (peer_addr_for_me v2:172.21.15.43:0/0) 2023-08-09T13:48:15.773 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:15.769+0000 7f1bc9cab640 1 --2- 172.21.15.43:0/971023147 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f1bc406fb70 0x7f1bc41f0d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:48:15.773 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:15.769+0000 7f1bca4ac640 1 -- 172.21.15.43:0/971023147 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f1bc406fb70 msgr2=0x7f1bc41f0d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:48:15.773 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:15.769+0000 7f1bca4ac640 1 --2- 172.21.15.43:0/971023147 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f1bc406fb70 0x7f1bc41f0d80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:48:15.773 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:15.769+0000 7f1bc94aa640 1 --2- 172.21.15.43:0/971023147 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f1bc40ca370 0x7f1bc4146e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:48:15.774 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:15.769+0000 7f1bca4ac640 1 -- 172.21.15.43:0/971023147 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f1bc40ca370 msgr2=0x7f1bc4146e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:48:15.774 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:15.769+0000 7f1bca4ac640 1 --2- 172.21.15.43:0/971023147 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f1bc40ca370 0x7f1bc4146e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:48:15.774 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:15.769+0000 7f1bca4ac640 1 -- 172.21.15.43:0/971023147 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1bc00094d0 con 0x7f1bc40ce4c0 2023-08-09T13:48:15.774 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:15.769+0000 7f1bc94aa640 1 --2- 172.21.15.43:0/971023147 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f1bc40ca370 0x7f1bc4146e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-08-09T13:48:15.774 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:15.769+0000 7f1bca4ac640 1 --2- 172.21.15.43:0/971023147 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f1bc40ce4c0 0x7f1bc41595e0 secure :-1 s=READY pgs=834 cs=0 l=1 rev1=1 crypto rx=0x7f1bbc00c780 tx=0x7f1bbc00cc40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-09T13:48:15.774 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:15.769+0000 7f1bba7fc640 1 -- 172.21.15.43:0/971023147 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1bbc003d80 con 0x7f1bc40ce4c0 2023-08-09T13:48:15.774 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:15.770+0000 7f1bba7fc640 1 -- 172.21.15.43:0/971023147 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1bbc00ce90 con 0x7f1bc40ce4c0 2023-08-09T13:48:15.775 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:15.770+0000 7f1bba7fc640 1 -- 172.21.15.43:0/971023147 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1bbc0052b0 con 0x7f1bc40ce4c0 2023-08-09T13:48:15.775 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:15.770+0000 7f1bcbf0f640 1 -- 172.21.15.43:0/971023147 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1bc41f12c0 con 0x7f1bc40ce4c0 2023-08-09T13:48:15.775 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:15.770+0000 7f1bcbf0f640 1 -- 172.21.15.43:0/971023147 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1bc41f1560 con 0x7f1bc40ce4c0 2023-08-09T13:48:15.775 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:15.771+0000 7f1bcbf0f640 1 -- 172.21.15.43:0/971023147 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1bc40c35b0 con 0x7f1bc40ce4c0 2023-08-09T13:48:15.776 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:15.772+0000 7f1bba7fc640 1 -- 172.21.15.43:0/971023147 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 18) v1 ==== 88443+0+0 (secure 0 0 0) 0x7f1bbc005410 con 0x7f1bc40ce4c0 2023-08-09T13:48:15.776 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:15.773+0000 7f1bba7fc640 1 --2- 172.21.15.43:0/971023147 >> [v2:172.21.15.203:6832/272917756,v1:172.21.15.203:6833/272917756] conn(0x7f1ba806a170 0x7f1ba806c630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:48:15.776 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:15.773+0000 7f1bba7fc640 1 -- 172.21.15.43:0/971023147 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(70..70 src has 1..70) v4 ==== 5574+0+0 (secure 0 0 0) 0x7f1bbc087a40 con 0x7f1bc40ce4c0 2023-08-09T13:48:15.776 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:15.773+0000 7f1bc9cab640 1 --2- 172.21.15.43:0/971023147 >> [v2:172.21.15.203:6832/272917756,v1:172.21.15.203:6833/272917756] conn(0x7f1ba806a170 0x7f1ba806c630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:48:15.777 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:15.775+0000 7f1bc9cab640 1 --2- 172.21.15.43:0/971023147 >> [v2:172.21.15.203:6832/272917756,v1:172.21.15.203:6833/272917756] conn(0x7f1ba806a170 0x7f1ba806c630 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f1bc0036000 tx=0x7f1bc003c040 comp rx=0 tx=0).ready entity=mgr.5533 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-09T13:48:15.779 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:15.777+0000 7f1bba7fc640 1 -- 172.21.15.43:0/971023147 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178066 (secure 0 0 0) 0x7f1bbc0531f0 con 0x7f1bc40ce4c0 2023-08-09T13:48:15.982 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:15.980+0000 7f1bcbf0f640 1 -- 172.21.15.43:0/971023147 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64} v 0) v1 -- 0x7f1bc40739f0 con 0x7f1bc40ce4c0 2023-08-09T13:48:16.097 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:16.094+0000 7f1bba7fc640 1 -- 172.21.15.43:0/971023147 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]=0 pool 'cephfs_metadata' created v71) v1 ==== 138+0+0 (secure 0 0 0) 0x7f1bbc0170b0 con 0x7f1bc40ce4c0 2023-08-09T13:48:16.099 INFO:teuthology.orchestra.run.smithi043.stderr:pool 'cephfs_metadata' created 2023-08-09T13:48:16.104 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:16.101+0000 7f1bcbf0f640 1 -- 172.21.15.43:0/971023147 >> [v2:172.21.15.203:6832/272917756,v1:172.21.15.203:6833/272917756] conn(0x7f1ba806a170 msgr2=0x7f1ba806c630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:48:16.104 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:16.101+0000 7f1bcbf0f640 1 --2- 172.21.15.43:0/971023147 >> [v2:172.21.15.203:6832/272917756,v1:172.21.15.203:6833/272917756] conn(0x7f1ba806a170 0x7f1ba806c630 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f1bc0036000 tx=0x7f1bc003c040 comp rx=0 tx=0).stop 2023-08-09T13:48:16.104 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:16.101+0000 7f1bcbf0f640 1 -- 172.21.15.43:0/971023147 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f1bc40ce4c0 msgr2=0x7f1bc41595e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:48:16.105 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:16.101+0000 7f1bcbf0f640 1 --2- 172.21.15.43:0/971023147 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f1bc40ce4c0 0x7f1bc41595e0 secure :-1 s=READY pgs=834 cs=0 l=1 rev1=1 crypto rx=0x7f1bbc00c780 tx=0x7f1bbc00cc40 comp rx=0 tx=0).stop 2023-08-09T13:48:16.105 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:16.102+0000 7f1bcbf0f640 1 -- 172.21.15.43:0/971023147 shutdown_connections 2023-08-09T13:48:16.105 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:16.102+0000 7f1bcbf0f640 1 --2- 172.21.15.43:0/971023147 >> [v2:172.21.15.203:6832/272917756,v1:172.21.15.203:6833/272917756] conn(0x7f1ba806a170 0x7f1ba806c630 secure :-1 s=CLOSED pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f1bc0036000 tx=0x7f1bc003c040 comp rx=0 tx=0).stop 2023-08-09T13:48:16.105 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:16.102+0000 7f1bcbf0f640 1 --2- 172.21.15.43:0/971023147 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f1bc40ce4c0 0x7f1bc41595e0 unknown :-1 s=CLOSED pgs=834 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:48:16.105 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:16.102+0000 7f1bcbf0f640 1 --2- 172.21.15.43:0/971023147 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f1bc40ca370 0x7f1bc4146e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:48:16.105 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:16.102+0000 7f1bcbf0f640 1 --2- 172.21.15.43:0/971023147 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f1bc406fb70 0x7f1bc41f0d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:48:16.105 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:16.102+0000 7f1bcbf0f640 1 -- 172.21.15.43:0/971023147 >> 172.21.15.43:0/971023147 conn(0x7f1bc406b990 msgr2=0x7f1bc40b2cd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-09T13:48:16.106 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:16.102+0000 7f1bcbf0f640 1 -- 172.21.15.43:0/971023147 shutdown_connections 2023-08-09T13:48:16.106 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:16.103+0000 7f1bcbf0f640 1 -- 172.21.15.43:0/971023147 wait complete. 2023-08-09T13:48:16.126 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data 64 --pg_num_min 64 --target_size_ratio 0.9 2023-08-09T13:48:16.353 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:16.350+0000 7f3d5886a640 1 Processor -- start 2023-08-09T13:48:16.353 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:16.351+0000 7f3d5886a640 1 -- start start 2023-08-09T13:48:16.355 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:16.352+0000 7f3d5886a640 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7f3d540c9560 0x7f3d540cb9a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:48:16.355 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:16.352+0000 7f3d5886a640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f3d540c8900 0x7f3d5415a270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:48:16.355 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:16.352+0000 7f3d5886a640 1 -- --> v1:172.21.15.203:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3d5415a7b0 con 0x7f3d54159400 2023-08-09T13:48:16.355 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:16.352+0000 7f3d5886a640 1 -- --> v2:172.21.15.43:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3d54070250 con 0x7f3d540c8900 2023-08-09T13:48:16.355 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:16.352+0000 7f3d5886a640 1 -- --> v2:172.21.15.203:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3d540c86c0 con 0x7f3d540c9560 2023-08-09T13:48:16.356 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:16.353+0000 7f3d53fff640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f3d540c8900 0x7f3d5415a270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:48:16.356 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:16.353+0000 7f3d53fff640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f3d540c8900 0x7f3d5415a270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:36686/0 (socket says 172.21.15.43:36686) 2023-08-09T13:48:16.356 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:16.353+0000 7f3d53fff640 1 -- 172.21.15.43:0/647920049 learned_addr learned my addr 172.21.15.43:0/647920049 (peer_addr_for_me v2:172.21.15.43:0/0) 2023-08-09T13:48:16.356 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:16.353+0000 7f3d4bfff640 1 --2- 172.21.15.43:0/647920049 >> v2:172.21.15.203:3300/0 conn(0x7f3d540c9560 0x7f3d540cb9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:48:16.357 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:16.353+0000 7f3d53fff640 1 -- 172.21.15.43:0/647920049 >> v2:172.21.15.203:3300/0 conn(0x7f3d540c9560 msgr2=0x7f3d540cb9a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-09T13:48:16.357 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:16.353+0000 7f3d53fff640 1 --2- 172.21.15.43:0/647920049 >> v2:172.21.15.203:3300/0 conn(0x7f3d540c9560 0x7f3d540cb9a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:48:16.357 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:16.353+0000 7f3d53fff640 1 -- 172.21.15.43:0/647920049 >> v1:172.21.15.203:6789/0 conn(0x7f3d54159400 legacy=0x7f3d540c8e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-09T13:48:16.357 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:16.353+0000 7f3d53fff640 1 -- 172.21.15.43:0/647920049 --> v2:172.21.15.43:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3d5406fc60 con 0x7f3d540c8900 2023-08-09T13:48:16.358 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:16.354+0000 7f3d53fff640 1 --2- 172.21.15.43:0/647920049 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f3d540c8900 0x7f3d5415a270 secure :-1 s=READY pgs=835 cs=0 l=1 rev1=1 crypto rx=0x7f3d5415ccc0 tx=0x7f3d4c02be50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b5023ca4f7b0bd8e server_cookie=0 in_seq=0 out_seq=0 2023-08-09T13:48:16.358 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:16.355+0000 7f3d52ffd640 1 -- 172.21.15.43:0/647920049 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3d4c02c9f0 con 0x7f3d540c8900 2023-08-09T13:48:16.358 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:16.355+0000 7f3d52ffd640 1 -- 172.21.15.43:0/647920049 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3d4c02cb50 con 0x7f3d540c8900 2023-08-09T13:48:16.359 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:16.355+0000 7f3d52ffd640 1 -- 172.21.15.43:0/647920049 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3d4c0343f0 con 0x7f3d540c8900 2023-08-09T13:48:16.359 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:16.355+0000 7f3d5886a640 1 -- 172.21.15.43:0/647920049 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f3d540c8900 msgr2=0x7f3d5415a270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:48:16.359 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:16.355+0000 7f3d5886a640 1 --2- 172.21.15.43:0/647920049 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f3d540c8900 0x7f3d5415a270 secure :-1 s=READY pgs=835 cs=0 l=1 rev1=1 crypto rx=0x7f3d5415ccc0 tx=0x7f3d4c02be50 comp rx=0 tx=0).stop 2023-08-09T13:48:16.359 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:16.356+0000 7f3d5886a640 1 -- 172.21.15.43:0/647920049 shutdown_connections 2023-08-09T13:48:16.359 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:16.356+0000 7f3d5886a640 1 --2- 172.21.15.43:0/647920049 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f3d540c8900 0x7f3d5415a270 unknown :-1 s=CLOSED pgs=835 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:48:16.359 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:16.356+0000 7f3d5886a640 1 --2- 172.21.15.43:0/647920049 >> v2:172.21.15.203:3300/0 conn(0x7f3d540c9560 0x7f3d540cb9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:48:16.359 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:16.356+0000 7f3d5886a640 1 -- 172.21.15.43:0/647920049 >> 172.21.15.43:0/647920049 conn(0x7f3d540cddd0 msgr2=0x7f3d540ce1e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-09T13:48:16.359 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:16.356+0000 7f3d5886a640 1 -- 172.21.15.43:0/647920049 shutdown_connections 2023-08-09T13:48:16.360 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:16.356+0000 7f3d5886a640 1 -- 172.21.15.43:0/647920049 wait complete. 2023-08-09T13:48:16.360 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:16.358+0000 7f3d5886a640 1 Processor -- start 2023-08-09T13:48:16.360 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:16.358+0000 7f3d5886a640 1 -- start start 2023-08-09T13:48:16.361 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:16.359+0000 7f3d5886a640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f3d540c8900 0x7f3d54156eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:48:16.361 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:16.359+0000 7f3d5886a640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f3d540c9560 0x7f3d54156720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:48:16.361 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:16.359+0000 7f3d5886a640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f3d54157470 0x7f3d5406b3e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:48:16.362 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:16.359+0000 7f3d5886a640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3d540c5560 con 0x7f3d540c9560 2023-08-09T13:48:16.362 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:16.359+0000 7f3d5886a640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3d540c6010 con 0x7f3d54157470 2023-08-09T13:48:16.362 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:16.359+0000 7f3d5886a640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3d54156c60 con 0x7f3d540c8900 2023-08-09T13:48:16.363 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:16.359+0000 7f3d4bfff640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f3d540c9560 0x7f3d54156720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:48:16.363 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:16.359+0000 7f3d4bfff640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f3d540c9560 0x7f3d54156720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:36702/0 (socket says 172.21.15.43:36702) 2023-08-09T13:48:16.363 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:16.359+0000 7f3d4bfff640 1 -- 172.21.15.43:0/3491081274 learned_addr learned my addr 172.21.15.43:0/3491081274 (peer_addr_for_me v2:172.21.15.43:0/0) 2023-08-09T13:48:16.363 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:16.360+0000 7f3d53fff640 1 --2- 172.21.15.43:0/3491081274 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f3d54157470 0x7f3d5406b3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:48:16.364 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:16.360+0000 7f3d4bfff640 1 -- 172.21.15.43:0/3491081274 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f3d540c8900 msgr2=0x7f3d54156eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:48:16.364 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:16.360+0000 7f3d4bfff640 1 --2- 172.21.15.43:0/3491081274 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f3d540c8900 0x7f3d54156eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:48:16.364 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:16.360+0000 7f3d4bfff640 1 -- 172.21.15.43:0/3491081274 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f3d54157470 msgr2=0x7f3d5406b3e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:48:16.364 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:16.360+0000 7f3d4bfff640 1 --2- 172.21.15.43:0/3491081274 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f3d54157470 0x7f3d5406b3e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:48:16.364 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:16.360+0000 7f3d4bfff640 1 -- 172.21.15.43:0/3491081274 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3d4c009b30 con 0x7f3d540c9560 2023-08-09T13:48:16.364 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:16.360+0000 7f3d53fff640 1 --2- 172.21.15.43:0/3491081274 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f3d54157470 0x7f3d5406b3e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-08-09T13:48:16.365 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:16.360+0000 7f3d4bfff640 1 --2- 172.21.15.43:0/3491081274 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f3d540c9560 0x7f3d54156720 secure :-1 s=READY pgs=836 cs=0 l=1 rev1=1 crypto rx=0x7f3d3c00d800 tx=0x7f3d3c00dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-09T13:48:16.365 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:16.360+0000 7f3d50ff9640 1 -- 172.21.15.43:0/3491081274 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3d3c00b6c0 con 0x7f3d540c9560 2023-08-09T13:48:16.365 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:16.360+0000 7f3d50ff9640 1 -- 172.21.15.43:0/3491081274 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3d3c00b820 con 0x7f3d540c9560 2023-08-09T13:48:16.365 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:16.360+0000 7f3d50ff9640 1 -- 172.21.15.43:0/3491081274 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3d3c00bbb0 con 0x7f3d540c9560 2023-08-09T13:48:16.366 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:16.360+0000 7f3d5886a640 1 -- 172.21.15.43:0/3491081274 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3d540c5280 con 0x7f3d540c9560 2023-08-09T13:48:16.366 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:16.361+0000 7f3d5886a640 1 -- 172.21.15.43:0/3491081274 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3d540c5ca0 con 0x7f3d540c9560 2023-08-09T13:48:16.366 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:16.362+0000 7f3d5886a640 1 -- 172.21.15.43:0/3491081274 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3d5404e390 con 0x7f3d540c9560 2023-08-09T13:48:16.366 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:16.363+0000 7f3d50ff9640 1 -- 172.21.15.43:0/3491081274 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 18) v1 ==== 88443+0+0 (secure 0 0 0) 0x7f3d3c00bd10 con 0x7f3d540c9560 2023-08-09T13:48:16.367 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:16.364+0000 7f3d50ff9640 1 --2- 172.21.15.43:0/3491081274 >> [v2:172.21.15.203:6832/272917756,v1:172.21.15.203:6833/272917756] conn(0x7f3d3406a170 0x7f3d3406c630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:48:16.367 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:16.364+0000 7f3d50ff9640 1 -- 172.21.15.43:0/3491081274 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(71..71 src has 1..71) v4 ==== 5944+0+0 (secure 0 0 0) 0x7f3d3c089720 con 0x7f3d540c9560 2023-08-09T13:48:16.367 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:16.365+0000 7f3d537fe640 1 --2- 172.21.15.43:0/3491081274 >> [v2:172.21.15.203:6832/272917756,v1:172.21.15.203:6833/272917756] conn(0x7f3d3406a170 0x7f3d3406c630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:48:16.368 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:16.366+0000 7f3d537fe640 1 --2- 172.21.15.43:0/3491081274 >> [v2:172.21.15.203:6832/272917756,v1:172.21.15.203:6833/272917756] conn(0x7f3d3406a170 0x7f3d3406c630 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f3d44006fd0 tx=0x7f3d44006bc0 comp rx=0 tx=0).ready entity=mgr.5533 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-09T13:48:16.370 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:16.368+0000 7f3d50ff9640 1 -- 172.21.15.43:0/3491081274 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178066 (secure 0 0 0) 0x7f3d3c054de0 con 0x7f3d540c9560 2023-08-09T13:48:16.564 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:16.561+0000 7f3d5886a640 1 -- 172.21.15.43:0/3491081274 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9} v 0) v1 -- 0x7f3d540cea30 con 0x7f3d540c9560 2023-08-09T13:48:17.112 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:17.109+0000 7f3d50ff9640 1 -- 172.21.15.43:0/3491081274 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]=0 pool 'cephfs_data' created v72) v1 ==== 170+0+0 (secure 0 0 0) 0x7f3d3c058400 con 0x7f3d540c9560 2023-08-09T13:48:17.113 INFO:teuthology.orchestra.run.smithi043.stderr:pool 'cephfs_data' created 2023-08-09T13:48:17.120 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:17.117+0000 7f3d5886a640 1 -- 172.21.15.43:0/3491081274 >> [v2:172.21.15.203:6832/272917756,v1:172.21.15.203:6833/272917756] conn(0x7f3d3406a170 msgr2=0x7f3d3406c630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:48:17.120 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:17.117+0000 7f3d5886a640 1 --2- 172.21.15.43:0/3491081274 >> [v2:172.21.15.203:6832/272917756,v1:172.21.15.203:6833/272917756] conn(0x7f3d3406a170 0x7f3d3406c630 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f3d44006fd0 tx=0x7f3d44006bc0 comp rx=0 tx=0).stop 2023-08-09T13:48:17.120 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:17.117+0000 7f3d5886a640 1 -- 172.21.15.43:0/3491081274 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f3d540c9560 msgr2=0x7f3d54156720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:48:17.121 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:17.117+0000 7f3d5886a640 1 --2- 172.21.15.43:0/3491081274 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f3d540c9560 0x7f3d54156720 secure :-1 s=READY pgs=836 cs=0 l=1 rev1=1 crypto rx=0x7f3d3c00d800 tx=0x7f3d3c00dcc0 comp rx=0 tx=0).stop 2023-08-09T13:48:17.128 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:17.126+0000 7f3d5886a640 1 -- 172.21.15.43:0/3491081274 shutdown_connections 2023-08-09T13:48:17.129 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:17.126+0000 7f3d5886a640 1 --2- 172.21.15.43:0/3491081274 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f3d54157470 0x7f3d5406b3e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:48:17.129 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:17.127+0000 7f3d5886a640 1 --2- 172.21.15.43:0/3491081274 >> [v2:172.21.15.203:6832/272917756,v1:172.21.15.203:6833/272917756] conn(0x7f3d3406a170 0x7f3d3406c630 unknown :-1 s=CLOSED pgs=57 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:48:17.129 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:17.127+0000 7f3d5886a640 1 --2- 172.21.15.43:0/3491081274 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f3d540c9560 0x7f3d54156720 unknown :-1 s=CLOSED pgs=836 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:48:17.130 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:17.127+0000 7f3d5886a640 1 --2- 172.21.15.43:0/3491081274 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f3d540c8900 0x7f3d54156eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:48:17.130 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:17.127+0000 7f3d5886a640 1 -- 172.21.15.43:0/3491081274 >> 172.21.15.43:0/3491081274 conn(0x7f3d540cddd0 msgr2=0x7f3d540ad910 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-09T13:48:17.130 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:17.128+0000 7f3d5886a640 1 -- 172.21.15.43:0/3491081274 shutdown_connections 2023-08-09T13:48:17.131 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:17.128+0000 7f3d5886a640 1 -- 172.21.15.43:0/3491081274 wait complete. 2023-08-09T13:48:17.162 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs new cephfs cephfs_metadata cephfs_data 2023-08-09T13:48:17.360 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:17.358+0000 7f95c7e01640 1 Processor -- start 2023-08-09T13:48:17.361 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:17.358+0000 7f95c7e01640 1 -- start start 2023-08-09T13:48:17.362 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:17.359+0000 7f95c7e01640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f95c015da80 0x7f95c0070db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:48:17.362 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:17.359+0000 7f95c7e01640 1 -- --> v1:172.21.15.43:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f95c00c9d40 con 0x7f95c0159470 2023-08-09T13:48:17.362 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:17.359+0000 7f95c7e01640 1 -- --> v1:172.21.15.203:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f95c00ca020 con 0x7f95c00712f0 2023-08-09T13:48:17.362 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:17.359+0000 7f95c7e01640 1 -- --> v2:172.21.15.43:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f95c0070990 con 0x7f95c015da80 2023-08-09T13:48:17.363 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:17.360+0000 7f95c539c640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f95c015da80 0x7f95c0070db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:48:17.363 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:17.360+0000 7f95c5b9d640 1 --1- >> v1:172.21.15.43:6789/0 conn(0x7f95c0159470 0x7f95c00c9420 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.43:6789/0 says I am v1:172.21.15.43:46474/0 (socket says 172.21.15.43:46474) 2023-08-09T13:48:17.363 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:17.360+0000 7f95c5b9d640 1 -- 172.21.15.43:0/3322317096 learned_addr learned my addr 172.21.15.43:0/3322317096 (peer_addr_for_me v1:172.21.15.43:0/0) 2023-08-09T13:48:17.363 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:17.360+0000 7f95c4b9b640 1 -- 172.21.15.43:0/3322317096 <== mon.0 v1:172.21.15.43:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 767874242 0 0) 0x7f95c00c9d40 con 0x7f95c0159470 2023-08-09T13:48:17.364 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:17.360+0000 7f95c4b9b640 1 -- 172.21.15.43:0/3322317096 --> v1:172.21.15.43:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f95ac005730 con 0x7f95c0159470 2023-08-09T13:48:17.364 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:17.361+0000 7f95c539c640 1 -- 172.21.15.43:0/3322317096 >> v1:172.21.15.203:6789/0 conn(0x7f95c00712f0 legacy=0x7f95c00cc8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:48:17.364 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:17.361+0000 7f95c539c640 1 -- 172.21.15.43:0/3322317096 >> v1:172.21.15.43:6789/0 conn(0x7f95c0159470 legacy=0x7f95c00c9420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:48:17.364 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:17.361+0000 7f95c539c640 1 -- 172.21.15.43:0/3322317096 --> v2:172.21.15.43:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f95c00705b0 con 0x7f95c015da80 2023-08-09T13:48:17.365 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:17.362+0000 7f95c539c640 1 --2- 172.21.15.43:0/3322317096 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f95c015da80 0x7f95c0070db0 secure :-1 s=READY pgs=838 cs=0 l=1 rev1=1 crypto rx=0x7f95b00097a0 tx=0x7f95b002b960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=14d62f89203d2fa0 server_cookie=0 in_seq=0 out_seq=0 2023-08-09T13:48:17.365 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:17.362+0000 7f95c4b9b640 1 -- 172.21.15.43:0/3322317096 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f95b0030610 con 0x7f95c015da80 2023-08-09T13:48:17.365 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:17.362+0000 7f95c4b9b640 1 -- 172.21.15.43:0/3322317096 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f95b0030770 con 0x7f95c015da80 2023-08-09T13:48:17.365 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:17.362+0000 7f95c4b9b640 1 -- 172.21.15.43:0/3322317096 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f95b0030ae0 con 0x7f95c015da80 2023-08-09T13:48:17.366 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:17.363+0000 7f95c7e01640 1 -- 172.21.15.43:0/3322317096 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f95c015da80 msgr2=0x7f95c0070db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:48:17.366 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:17.363+0000 7f95c7e01640 1 --2- 172.21.15.43:0/3322317096 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f95c015da80 0x7f95c0070db0 secure :-1 s=READY pgs=838 cs=0 l=1 rev1=1 crypto rx=0x7f95b00097a0 tx=0x7f95b002b960 comp rx=0 tx=0).stop 2023-08-09T13:48:17.366 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:17.363+0000 7f95c7e01640 1 -- 172.21.15.43:0/3322317096 shutdown_connections 2023-08-09T13:48:17.366 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:17.363+0000 7f95c7e01640 1 --2- 172.21.15.43:0/3322317096 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f95c015da80 0x7f95c0070db0 unknown :-1 s=CLOSED pgs=838 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:48:17.366 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:17.363+0000 7f95c7e01640 1 -- 172.21.15.43:0/3322317096 >> 172.21.15.43:0/3322317096 conn(0x7f95c00aff50 msgr2=0x7f95c00b0360 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-09T13:48:17.366 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:17.363+0000 7f95c7e01640 1 -- 172.21.15.43:0/3322317096 shutdown_connections 2023-08-09T13:48:17.367 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:17.363+0000 7f95c7e01640 1 -- 172.21.15.43:0/3322317096 wait complete. 2023-08-09T13:48:17.367 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:17.365+0000 7f95c7e01640 1 Processor -- start 2023-08-09T13:48:17.367 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:17.365+0000 7f95c7e01640 1 -- start start 2023-08-09T13:48:17.368 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:17.366+0000 7f95c7e01640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f95c00712f0 0x7f95c015a8c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:48:17.369 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:17.366+0000 7f95c7e01640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f95c0159470 0x7f95c0151680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:48:17.369 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:17.366+0000 7f95c7e01640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f95c015da80 0x7f95c00cab20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:48:17.369 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:17.366+0000 7f95c7e01640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f95c00abd40 con 0x7f95c015da80 2023-08-09T13:48:17.369 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:17.366+0000 7f95c7e01640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f95c00ac020 con 0x7f95c0159470 2023-08-09T13:48:17.369 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:17.366+0000 7f95c7e01640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f95c00ac300 con 0x7f95c00712f0 2023-08-09T13:48:17.369 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:17.366+0000 7f95c639e640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f95c015da80 0x7f95c00cab20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:48:17.369 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:17.366+0000 7f95c639e640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f95c015da80 0x7f95c00cab20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:36720/0 (socket says 172.21.15.43:36720) 2023-08-09T13:48:17.369 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:17.366+0000 7f95c639e640 1 -- 172.21.15.43:0/1004828821 learned_addr learned my addr 172.21.15.43:0/1004828821 (peer_addr_for_me v2:172.21.15.43:0/0) 2023-08-09T13:48:17.370 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:17.367+0000 7f95c639e640 1 -- 172.21.15.43:0/1004828821 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f95c00712f0 msgr2=0x7f95c015a8c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:48:17.370 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:17.367+0000 7f95c639e640 1 --2- 172.21.15.43:0/1004828821 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f95c00712f0 0x7f95c015a8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:48:17.371 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:17.367+0000 7f95c639e640 1 -- 172.21.15.43:0/1004828821 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f95c0159470 msgr2=0x7f95c0151680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:48:17.371 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:17.367+0000 7f95c639e640 1 --2- 172.21.15.43:0/1004828821 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f95c0159470 0x7f95c0151680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:48:17.371 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:17.367+0000 7f95c639e640 1 -- 172.21.15.43:0/1004828821 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f95b0009400 con 0x7f95c015da80 2023-08-09T13:48:17.371 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:17.367+0000 7f95c639e640 1 --2- 172.21.15.43:0/1004828821 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f95c015da80 0x7f95c00cab20 secure :-1 s=READY pgs=839 cs=0 l=1 rev1=1 crypto rx=0x7f95b800c6b0 tx=0x7f95b800cb70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-09T13:48:17.371 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:17.367+0000 7f95b67fc640 1 -- 172.21.15.43:0/1004828821 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f95b8003b70 con 0x7f95c015da80 2023-08-09T13:48:17.371 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:17.367+0000 7f95b67fc640 1 -- 172.21.15.43:0/1004828821 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f95b8003cf0 con 0x7f95c015da80 2023-08-09T13:48:17.371 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:17.367+0000 7f95b67fc640 1 -- 172.21.15.43:0/1004828821 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f95b8003ea0 con 0x7f95c015da80 2023-08-09T13:48:17.372 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:17.367+0000 7f95c7e01640 1 -- 172.21.15.43:0/1004828821 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f95c00ac5e0 con 0x7f95c015da80 2023-08-09T13:48:17.372 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:17.367+0000 7f95c7e01640 1 -- 172.21.15.43:0/1004828821 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f95c0151bc0 con 0x7f95c015da80 2023-08-09T13:48:17.372 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:17.369+0000 7f95c7e01640 1 -- 172.21.15.43:0/1004828821 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f95c00acb00 con 0x7f95c015da80 2023-08-09T13:48:17.372 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:17.370+0000 7f95b67fc640 1 -- 172.21.15.43:0/1004828821 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 18) v1 ==== 88443+0+0 (secure 0 0 0) 0x7f95b8005310 con 0x7f95c015da80 2023-08-09T13:48:17.373 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:17.371+0000 7f95b67fc640 1 --2- 172.21.15.43:0/1004828821 >> [v2:172.21.15.203:6832/272917756,v1:172.21.15.203:6833/272917756] conn(0x7f95a406a170 0x7f95a406c630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:48:17.373 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:17.371+0000 7f95b67fc640 1 -- 172.21.15.43:0/1004828821 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(72..72 src has 1..72) v4 ==== 6326+0+0 (secure 0 0 0) 0x7f95b8088f90 con 0x7f95c015da80 2023-08-09T13:48:17.374 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:17.372+0000 7f95c5b9d640 1 --2- 172.21.15.43:0/1004828821 >> [v2:172.21.15.203:6832/272917756,v1:172.21.15.203:6833/272917756] conn(0x7f95a406a170 0x7f95a406c630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:48:17.375 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:17.373+0000 7f95c5b9d640 1 --2- 172.21.15.43:0/1004828821 >> [v2:172.21.15.203:6832/272917756,v1:172.21.15.203:6833/272917756] conn(0x7f95a406a170 0x7f95a406c630 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f95bc001610 tx=0x7f95bc007930 comp rx=0 tx=0).ready entity=mgr.5533 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-09T13:48:17.376 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:17.374+0000 7f95b67fc640 1 -- 172.21.15.43:0/1004828821 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178066 (secure 0 0 0) 0x7f95b8054440 con 0x7f95c015da80 2023-08-09T13:48:17.606 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:17.604+0000 7f95c7e01640 1 -- 172.21.15.43:0/1004828821 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"} v 0) v1 -- 0x7f95c00acb00 con 0x7f95c015da80 2023-08-09T13:48:18.162 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:18.160+0000 7f95b67fc640 1 -- 172.21.15.43:0/1004828821 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]=0 new fs with metadata pool 8 and data pool 9 v22) v1 ==== 172+0+0 (secure 0 0 0) 0x7f95c00acb00 con 0x7f95c015da80 2023-08-09T13:48:18.163 INFO:teuthology.orchestra.run.smithi043.stderr:new fs with metadata pool 8 and data pool 9 2023-08-09T13:48:18.167 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:18.164+0000 7f95c7e01640 1 -- 172.21.15.43:0/1004828821 >> [v2:172.21.15.203:6832/272917756,v1:172.21.15.203:6833/272917756] conn(0x7f95a406a170 msgr2=0x7f95a406c630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:48:18.167 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:18.164+0000 7f95c7e01640 1 --2- 172.21.15.43:0/1004828821 >> [v2:172.21.15.203:6832/272917756,v1:172.21.15.203:6833/272917756] conn(0x7f95a406a170 0x7f95a406c630 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f95bc001610 tx=0x7f95bc007930 comp rx=0 tx=0).stop 2023-08-09T13:48:18.167 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:18.164+0000 7f95c7e01640 1 -- 172.21.15.43:0/1004828821 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f95c015da80 msgr2=0x7f95c00cab20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:48:18.167 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:18.164+0000 7f95c7e01640 1 --2- 172.21.15.43:0/1004828821 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f95c015da80 0x7f95c00cab20 secure :-1 s=READY pgs=839 cs=0 l=1 rev1=1 crypto rx=0x7f95b800c6b0 tx=0x7f95b800cb70 comp rx=0 tx=0).stop 2023-08-09T13:48:18.167 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:18.164+0000 7f95c7e01640 1 -- 172.21.15.43:0/1004828821 shutdown_connections 2023-08-09T13:48:18.167 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:18.164+0000 7f95c7e01640 1 --2- 172.21.15.43:0/1004828821 >> [v2:172.21.15.203:6832/272917756,v1:172.21.15.203:6833/272917756] conn(0x7f95a406a170 0x7f95a406c630 unknown :-1 s=CLOSED pgs=58 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:48:18.168 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:18.164+0000 7f95c7e01640 1 --2- 172.21.15.43:0/1004828821 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f95c015da80 0x7f95c00cab20 unknown :-1 s=CLOSED pgs=839 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:48:18.168 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:18.164+0000 7f95c7e01640 1 --2- 172.21.15.43:0/1004828821 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f95c0159470 0x7f95c0151680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:48:18.168 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:18.164+0000 7f95c7e01640 1 --2- 172.21.15.43:0/1004828821 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f95c00712f0 0x7f95c015a8c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:48:18.168 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:18.164+0000 7f95c7e01640 1 -- 172.21.15.43:0/1004828821 >> 172.21.15.43:0/1004828821 conn(0x7f95c00aff50 msgr2=0x7f95c00cd6d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-09T13:48:18.168 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:18.164+0000 7f95c7e01640 1 -- 172.21.15.43:0/1004828821 shutdown_connections 2023-08-09T13:48:18.168 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:18.164+0000 7f95c7e01640 1 -- 172.21.15.43:0/1004828821 wait complete. 2023-08-09T13:48:18.191 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-08-09T13:48:18.376 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:18.374+0000 7f2d16586640 1 Processor -- start 2023-08-09T13:48:18.377 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:18.374+0000 7f2d16586640 1 -- start start 2023-08-09T13:48:18.378 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:18.375+0000 7f2d16586640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f2d10160cb0 0x7f2d1015c070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:48:18.378 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:18.375+0000 7f2d16586640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f2d10160760 0x7f2d1015c5b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:48:18.378 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:18.375+0000 7f2d16586640 1 -- --> v1:172.21.15.203:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2d100c8680 con 0x7f2d10160380 2023-08-09T13:48:18.378 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:18.375+0000 7f2d16586640 1 -- --> v2:172.21.15.43:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2d100ca7d0 con 0x7f2d10160760 2023-08-09T13:48:18.378 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:18.375+0000 7f2d16586640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2d100c8960 con 0x7f2d10160cb0 2023-08-09T13:48:18.379 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:18.376+0000 7f2d15d85640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f2d10160760 0x7f2d1015c5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:48:18.379 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:18.376+0000 7f2d15d85640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f2d10160760 0x7f2d1015c5b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:36734/0 (socket says 172.21.15.43:36734) 2023-08-09T13:48:18.380 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:18.376+0000 7f2d15d85640 1 -- 172.21.15.43:0/1973540991 learned_addr learned my addr 172.21.15.43:0/1973540991 (peer_addr_for_me v2:172.21.15.43:0/0) 2023-08-09T13:48:18.380 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:18.376+0000 7f2d14d83640 1 --2- 172.21.15.43:0/1973540991 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f2d10160cb0 0x7f2d1015c070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:48:18.380 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:18.376+0000 7f2d15d85640 1 -- 172.21.15.43:0/1973540991 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f2d10160cb0 msgr2=0x7f2d1015c070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-09T13:48:18.380 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:18.376+0000 7f2d15d85640 1 --2- 172.21.15.43:0/1973540991 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f2d10160cb0 0x7f2d1015c070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:48:18.380 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:18.376+0000 7f2d15d85640 1 -- 172.21.15.43:0/1973540991 >> v1:172.21.15.203:6789/0 conn(0x7f2d10160380 legacy=0x7f2d100ca0c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-09T13:48:18.380 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:18.376+0000 7f2d15d85640 1 -- 172.21.15.43:0/1973540991 --> v2:172.21.15.43:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2d100aaf00 con 0x7f2d10160760 2023-08-09T13:48:18.381 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:18.378+0000 7f2d15d85640 1 --2- 172.21.15.43:0/1973540991 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f2d10160760 0x7f2d1015c5b0 secure :-1 s=READY pgs=840 cs=0 l=1 rev1=1 crypto rx=0x7f2d08009e50 tx=0x7f2d0802bf60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=73a9ff44ecc5d7d5 server_cookie=0 in_seq=0 out_seq=0 2023-08-09T13:48:18.381 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:18.378+0000 7f2d07fff640 1 -- 172.21.15.43:0/1973540991 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2d0802cb00 con 0x7f2d10160760 2023-08-09T13:48:18.382 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:18.378+0000 7f2d07fff640 1 -- 172.21.15.43:0/1973540991 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2d0802cc60 con 0x7f2d10160760 2023-08-09T13:48:18.382 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:18.378+0000 7f2d07fff640 1 -- 172.21.15.43:0/1973540991 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2d080343f0 con 0x7f2d10160760 2023-08-09T13:48:18.382 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:18.379+0000 7f2d16586640 1 -- 172.21.15.43:0/1973540991 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f2d10160760 msgr2=0x7f2d1015c5b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:48:18.382 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:18.379+0000 7f2d16586640 1 --2- 172.21.15.43:0/1973540991 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f2d10160760 0x7f2d1015c5b0 secure :-1 s=READY pgs=840 cs=0 l=1 rev1=1 crypto rx=0x7f2d08009e50 tx=0x7f2d0802bf60 comp rx=0 tx=0).stop 2023-08-09T13:48:18.382 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:18.379+0000 7f2d16586640 1 -- 172.21.15.43:0/1973540991 shutdown_connections 2023-08-09T13:48:18.382 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:18.379+0000 7f2d16586640 1 --2- 172.21.15.43:0/1973540991 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f2d10160760 0x7f2d1015c5b0 unknown :-1 s=CLOSED pgs=840 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:48:18.382 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:18.379+0000 7f2d16586640 1 --2- 172.21.15.43:0/1973540991 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f2d10160cb0 0x7f2d1015c070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:48:18.383 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:18.379+0000 7f2d16586640 1 -- 172.21.15.43:0/1973540991 >> 172.21.15.43:0/1973540991 conn(0x7f2d100cde90 msgr2=0x7f2d100ce2a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-09T13:48:18.383 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:18.379+0000 7f2d16586640 1 -- 172.21.15.43:0/1973540991 shutdown_connections 2023-08-09T13:48:18.383 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:18.379+0000 7f2d16586640 1 -- 172.21.15.43:0/1973540991 wait complete. 2023-08-09T13:48:18.383 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:18.381+0000 7f2d16586640 1 Processor -- start 2023-08-09T13:48:18.384 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:18.381+0000 7f2d16586640 1 -- start start 2023-08-09T13:48:18.385 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:18.383+0000 7f2d16586640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f2d10160380 0x7f2d10145ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:48:18.385 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:18.383+0000 7f2d16586640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f2d10160760 0x7f2d1009a8c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:48:18.386 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:18.383+0000 7f2d16586640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f2d10160cb0 0x7f2d10098cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:48:18.386 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:18.383+0000 7f2d16586640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2d101575a0 con 0x7f2d10160cb0 2023-08-09T13:48:18.386 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:18.383+0000 7f2d16586640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2d1009ae00 con 0x7f2d10160760 2023-08-09T13:48:18.386 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:18.383+0000 7f2d16586640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2d1009b0d0 con 0x7f2d10160380 2023-08-09T13:48:18.386 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:18.383+0000 7f2d15d85640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f2d10160cb0 0x7f2d10098cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:48:18.386 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:18.383+0000 7f2d15d85640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f2d10160cb0 0x7f2d10098cc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:36746/0 (socket says 172.21.15.43:36746) 2023-08-09T13:48:18.386 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:18.383+0000 7f2d15d85640 1 -- 172.21.15.43:0/3846541924 learned_addr learned my addr 172.21.15.43:0/3846541924 (peer_addr_for_me v2:172.21.15.43:0/0) 2023-08-09T13:48:18.387 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:18.383+0000 7f2d14d83640 1 --2- 172.21.15.43:0/3846541924 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f2d10160760 0x7f2d1009a8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:48:18.387 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:18.383+0000 7f2d15d85640 1 -- 172.21.15.43:0/3846541924 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f2d10160380 msgr2=0x7f2d10145ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:48:18.388 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:18.383+0000 7f2d15d85640 1 --2- 172.21.15.43:0/3846541924 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f2d10160380 0x7f2d10145ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:48:18.388 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:18.383+0000 7f2d15d85640 1 -- 172.21.15.43:0/3846541924 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f2d10160760 msgr2=0x7f2d1009a8c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:48:18.388 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:18.384+0000 7f2d15d85640 1 --2- 172.21.15.43:0/3846541924 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f2d10160760 0x7f2d1009a8c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:48:18.388 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:18.384+0000 7f2d15d85640 1 -- 172.21.15.43:0/3846541924 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2d08009b30 con 0x7f2d10160cb0 2023-08-09T13:48:18.388 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:18.384+0000 7f2d14d83640 1 --2- 172.21.15.43:0/3846541924 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f2d10160760 0x7f2d1009a8c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-08-09T13:48:18.388 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:18.384+0000 7f2d15d85640 1 --2- 172.21.15.43:0/3846541924 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f2d10160cb0 0x7f2d10098cc0 secure :-1 s=READY pgs=841 cs=0 l=1 rev1=1 crypto rx=0x7f2d08036040 tx=0x7f2d08004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-09T13:48:18.388 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:18.384+0000 7f2d05ffb640 1 -- 172.21.15.43:0/3846541924 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2d08009510 con 0x7f2d10160cb0 2023-08-09T13:48:18.389 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:18.384+0000 7f2d05ffb640 1 -- 172.21.15.43:0/3846541924 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2d0803e450 con 0x7f2d10160cb0 2023-08-09T13:48:18.389 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:18.384+0000 7f2d05ffb640 1 -- 172.21.15.43:0/3846541924 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2d0803e7e0 con 0x7f2d10160cb0 2023-08-09T13:48:18.389 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:18.384+0000 7f2d16586640 1 -- 172.21.15.43:0/3846541924 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2d1009b3b0 con 0x7f2d10160cb0 2023-08-09T13:48:18.389 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:18.384+0000 7f2d16586640 1 -- 172.21.15.43:0/3846541924 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2d1009b970 con 0x7f2d10160cb0 2023-08-09T13:48:18.389 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:18.386+0000 7f2d16586640 1 -- 172.21.15.43:0/3846541924 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2d100739f0 con 0x7f2d10160cb0 2023-08-09T13:48:18.390 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:18.386+0000 7f2d05ffb640 1 -- 172.21.15.43:0/3846541924 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 18) v1 ==== 88443+0+0 (secure 0 0 0) 0x7f2d0803e940 con 0x7f2d10160cb0 2023-08-09T13:48:18.390 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:18.387+0000 7f2d05ffb640 1 --2- 172.21.15.43:0/3846541924 >> [v2:172.21.15.203:6832/272917756,v1:172.21.15.203:6833/272917756] conn(0x7f2cf406a170 0x7f2cf406c630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:48:18.390 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:18.388+0000 7f2d05ffb640 1 -- 172.21.15.43:0/3846541924 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(73..73 src has 1..73) v4 ==== 6426+0+0 (secure 0 0 0) 0x7f2d080ab6f0 con 0x7f2d10160cb0 2023-08-09T13:48:18.390 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:18.388+0000 7f2d15584640 1 --2- 172.21.15.43:0/3846541924 >> [v2:172.21.15.203:6832/272917756,v1:172.21.15.203:6833/272917756] conn(0x7f2cf406a170 0x7f2cf406c630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:48:18.392 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:18.389+0000 7f2d15584640 1 --2- 172.21.15.43:0/3846541924 >> [v2:172.21.15.203:6832/272917756,v1:172.21.15.203:6833/272917756] conn(0x7f2cf406a170 0x7f2cf406c630 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f2d0c003cc0 tx=0x7f2d0c0079b0 comp rx=0 tx=0).ready entity=mgr.5533 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-09T13:48:18.394 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:18.392+0000 7f2d05ffb640 1 -- 172.21.15.43:0/3846541924 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178066 (secure 0 0 0) 0x7f2d08076ac0 con 0x7f2d10160cb0 2023-08-09T13:48:18.588 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:18.585+0000 7f2d16586640 1 -- 172.21.15.43:0/3846541924 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f2d100cdb30 con 0x7f2d10160cb0 2023-08-09T13:48:18.589 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:18.587+0000 7f2d05ffb640 1 -- 172.21.15.43:0/3846541924 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v73) v1 ==== 74+0+18565 (secure 0 0 0) 0x7f2d0807a0e0 con 0x7f2d10160cb0 2023-08-09T13:48:18.590 INFO:teuthology.orchestra.run.smithi043.stdout: 2023-08-09T13:48:18.590 INFO:teuthology.orchestra.run.smithi043.stdout:{"epoch":73,"fsid":"daac87ee-1337-4250-a955-aea4e408a39b","created":"2023-08-09T13:33:41.701198+0000","modified":"2023-08-09T13:48:17.606423+0000","last_up_change":"2023-08-09T13:33:53.783453+0000","last_in_change":"2023-08-09T13:33:45.148904+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":9,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":"rbd","create_time":"2023-08-09T13:33:54.365137+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":".mgr","create_time":"2023-08-09T13:33:54.831803+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":8,"pool_name":"cephfs_metadata","create_time":"2023-08-09T13:48:16.039756+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"73","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":9,"pool_name":"cephfs_data","create_time":"2023-08-09T13:48:16.615400+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"73","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.6299999952316284,"score_stable":1.6299999952316284,"optimal_score":1,"raw_score_acting":1.6299999952316284,"raw_score_stable":1.6299999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"cbd49ecd-e896-4fbd-864e-5c37a829b165","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":72,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6800","nonce":502667335},{"type":"v1","addr":"172.21.15.43:6801","nonce":502667335}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6802","nonce":502667335},{"type":"v1","addr":"172.21.15.43:6803","nonce":502667335}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6806","nonce":502667335},{"type":"v1","addr":"172.21.15.43:6807","nonce":502667335}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6804","nonce":502667335},{"type":"v1","addr":"172.21.15.43:6805","nonce":502667335}]},"public_addr":"172.21.15.43:6801/502667335","cluster_addr":"172.21.15.43:6803/502667335","heartbeat_back_addr":"172.21.15.43:6807/502667335","heartbeat_front_addr":"172.21.15.43:6805/502667335","state":["exists","up"]},{"osd":1,"uuid":"75c7e0fe-1ec7-4fe8-a2f2-3661a65b8500","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":72,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6808","nonce":3289134610},{"type":"v1","addr":"172.21.15.43:6809","nonce":3289134610}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6810","nonce":3289134610},{"type":"v1","addr":"172.21.15.43:6812","nonce":3289134610}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6821","nonce":3289134610},{"type":"v1","addr":"172.21.15.43:6824","nonce":3289134610}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6815","nonce":3289134610},{"type":"v1","addr":"172.21.15.43:6818","nonce":3289134610}]},"public_addr":"172.21.15.43:6809/3289134610","cluster_addr":"172.21.15.43:6812/3289134610","heartbeat_back_addr":"172.21.15.43:6824/3289134610","heartbeat_front_addr":"172.21.15.43:6818/3289134610","state":["exists","up"]},{"osd":2,"uuid":"bb5c41e9-7f5f-4125-9b01-3bb8bfc2ea83","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":72,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6811","nonce":4079709736},{"type":"v1","addr":"172.21.15.43:6814","nonce":4079709736}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6817","nonce":4079709736},{"type":"v1","addr":"172.21.15.43:6820","nonce":4079709736}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6829","nonce":4079709736},{"type":"v1","addr":"172.21.15.43:6831","nonce":4079709736}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6823","nonce":4079709736},{"type":"v1","addr":"172.21.15.43:6826","nonce":4079709736}]},"public_addr":"172.21.15.43:6814/4079709736","cluster_addr":"172.21.15.43:6820/4079709736","heartbeat_back_addr":"172.21.15.43:6831/4079709736","heartbeat_front_addr":"172.21.15.43:6826/4079709736","state":["exists","up"]},{"osd":3,"uuid":"bfe4c8b4-694a-465c-87e6-2b2aab852613","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":72,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6813","nonce":1709538759},{"type":"v1","addr":"172.21.15.43:6816","nonce":1709538759}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6819","nonce":1709538759},{"type":"v1","addr":"172.21.15.43:6822","nonce":1709538759}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6828","nonce":1709538759},{"type":"v1","addr":"172.21.15.43:6830","nonce":1709538759}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6825","nonce":1709538759},{"type":"v1","addr":"172.21.15.43:6827","nonce":1709538759}]},"public_addr":"172.21.15.43:6816/1709538759","cluster_addr":"172.21.15.43:6822/1709538759","heartbeat_back_addr":"172.21.15.43:6830/1709538759","heartbeat_front_addr":"172.21.15.43:6827/1709538759","state":["exists","up"]},{"osd":4,"uuid":"fa2e8a77-aba1-4756-853e-0d174a5bdeba","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":72,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6800","nonce":4196866481},{"type":"v1","addr":"172.21.15.203:6801","nonce":4196866481}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6802","nonce":4196866481},{"type":"v1","addr":"172.21.15.203:6803","nonce":4196866481}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6806","nonce":4196866481},{"type":"v1","addr":"172.21.15.203:6807","nonce":4196866481}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6804","nonce":4196866481},{"type":"v1","addr":"172.21.15.203:6805","nonce":4196866481}]},"public_addr":"172.21.15.203:6801/4196866481","cluster_addr":"172.21.15.203:6803/4196866481","heartbeat_back_addr":"172.21.15.203:6807/4196866481","heartbeat_front_addr":"172.21.15.203:6805/4196866481","state":["exists","up"]},{"osd":5,"uuid":"81811e4b-4d24-4f51-a82c-f1d700a2a00d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":72,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6816","nonce":3059290081},{"type":"v1","addr":"172.21.15.203:6817","nonce":3059290081}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6818","nonce":3059290081},{"type":"v1","addr":"172.21.15.203:6819","nonce":3059290081}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6822","nonce":3059290081},{"type":"v1","addr":"172.21.15.203:6823","nonce":3059290081}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6820","nonce":3059290081},{"type":"v1","addr":"172.21.15.203:6821","nonce":3059290081}]},"public_addr":"172.21.15.203:6817/3059290081","cluster_addr":"172.21.15.203:6819/3059290081","heartbeat_back_addr":"172.21.15.203:6823/3059290081","heartbeat_front_addr":"172.21.15.203:6821/3059290081","state":["exists","up"]},{"osd":6,"uuid":"307bac4d-6a5b-443d-af75-56d695d7674f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":72,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6824","nonce":512155569},{"type":"v1","addr":"172.21.15.203:6825","nonce":512155569}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6826","nonce":512155569},{"type":"v1","addr":"172.21.15.203:6827","nonce":512155569}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6830","nonce":512155569},{"type":"v1","addr":"172.21.15.203:6831","nonce":512155569}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6828","nonce":512155569},{"type":"v1","addr":"172.21.15.203:6829","nonce":512155569}]},"public_addr":"172.21.15.203:6825/512155569","cluster_addr":"172.21.15.203:6827/512155569","heartbeat_back_addr":"172.21.15.203:6831/512155569","heartbeat_front_addr":"172.21.15.203:6829/512155569","state":["exists","up"]},{"osd":7,"uuid":"a95dc7ca-9f1b-4295-af70-68a22a2d7d5e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":72,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6808","nonce":3331450438},{"type":"v1","addr":"172.21.15.203:6809","nonce":3331450438}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6810","nonce":3331450438},{"type":"v1","addr":"172.21.15.203:6811","nonce":3331450438}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6814","nonce":3331450438},{"type":"v1","addr":"172.21.15.203:6815","nonce":3331450438}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6812","nonce":3331450438},{"type":"v1","addr":"172.21.15.203:6813","nonce":3331450438}]},"public_addr":"172.21.15.203:6809/3331450438","cluster_addr":"172.21.15.203:6811/3331450438","heartbeat_back_addr":"172.21.15.203:6815/3331450438","heartbeat_front_addr":"172.21.15.203:6813/3331450438","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-09T13:33:50.547652+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-09T13:33:50.589469+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-09T13:33:50.561724+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-09T13:33:50.599650+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-09T13:33:50.411855+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-09T13:33:50.219681+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-09T13:33:50.504664+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-09T13:33:50.441248+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":7,"snaps":[{"begin":104,"length":1},{"begin":221,"length":1}]}],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-08-09T13:48:18.593 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:18.590+0000 7f2d16586640 1 -- 172.21.15.43:0/3846541924 >> [v2:172.21.15.203:6832/272917756,v1:172.21.15.203:6833/272917756] conn(0x7f2cf406a170 msgr2=0x7f2cf406c630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:48:18.593 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:18.590+0000 7f2d16586640 1 --2- 172.21.15.43:0/3846541924 >> [v2:172.21.15.203:6832/272917756,v1:172.21.15.203:6833/272917756] conn(0x7f2cf406a170 0x7f2cf406c630 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f2d0c003cc0 tx=0x7f2d0c0079b0 comp rx=0 tx=0).stop 2023-08-09T13:48:18.593 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:18.590+0000 7f2d16586640 1 -- 172.21.15.43:0/3846541924 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f2d10160cb0 msgr2=0x7f2d10098cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:48:18.593 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:18.590+0000 7f2d16586640 1 --2- 172.21.15.43:0/3846541924 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f2d10160cb0 0x7f2d10098cc0 secure :-1 s=READY pgs=841 cs=0 l=1 rev1=1 crypto rx=0x7f2d08036040 tx=0x7f2d08004a40 comp rx=0 tx=0).stop 2023-08-09T13:48:18.593 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:18.590+0000 7f2d16586640 1 -- 172.21.15.43:0/3846541924 shutdown_connections 2023-08-09T13:48:18.593 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:18.590+0000 7f2d16586640 1 --2- 172.21.15.43:0/3846541924 >> [v2:172.21.15.203:6832/272917756,v1:172.21.15.203:6833/272917756] conn(0x7f2cf406a170 0x7f2cf406c630 unknown :-1 s=CLOSED pgs=59 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:48:18.594 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:18.590+0000 7f2d16586640 1 --2- 172.21.15.43:0/3846541924 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f2d10160cb0 0x7f2d10098cc0 unknown :-1 s=CLOSED pgs=841 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:48:18.594 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:18.590+0000 7f2d16586640 1 --2- 172.21.15.43:0/3846541924 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f2d10160760 0x7f2d1009a8c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:48:18.594 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:18.590+0000 7f2d16586640 1 --2- 172.21.15.43:0/3846541924 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f2d10160380 0x7f2d10145ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:48:18.594 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:18.590+0000 7f2d16586640 1 -- 172.21.15.43:0/3846541924 >> 172.21.15.43:0/3846541924 conn(0x7f2d100cde90 msgr2=0x7f2d1015b590 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-09T13:48:18.594 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:18.590+0000 7f2d16586640 1 -- 172.21.15.43:0/3846541924 shutdown_connections 2023-08-09T13:48:18.595 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:18.591+0000 7f2d16586640 1 -- 172.21.15.43:0/3846541924 wait complete. 2023-08-09T13:48:18.613 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-08-09T13:48:18.827 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:18.824+0000 7f83f249f640 1 Processor -- start 2023-08-09T13:48:18.827 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:18.824+0000 7f83f249f640 1 -- start start 2023-08-09T13:48:18.828 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:18.826+0000 7f83f249f640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f83ec0acdf0 0x7f83ec0ad340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:48:18.829 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:18.826+0000 7f83f249f640 1 -- --> v1:172.21.15.43:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f83ec0cbd90 con 0x7f83ec0c8b30 2023-08-09T13:48:18.829 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:18.826+0000 7f83f249f640 1 -- --> v1:172.21.15.203:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f83ec0ca2b0 con 0x7f83ec0c8750 2023-08-09T13:48:18.829 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:18.826+0000 7f83f249f640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f83ec15e810 con 0x7f83ec0acdf0 2023-08-09T13:48:18.830 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:18.827+0000 7f83f0c9c640 1 --1- >> v1:172.21.15.43:6789/0 conn(0x7f83ec0c8b30 0x7f83ec070f00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.43:6789/0 says I am v1:172.21.15.43:46482/0 (socket says 172.21.15.43:46482) 2023-08-09T13:48:18.831 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:18.827+0000 7f83f0c9c640 1 -- 172.21.15.43:0/3790366169 learned_addr learned my addr 172.21.15.43:0/3790366169 (peer_addr_for_me v1:172.21.15.43:0/0) 2023-08-09T13:48:18.831 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:18.827+0000 7f83f1c9e640 1 --2- 172.21.15.43:0/3790366169 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f83ec0acdf0 0x7f83ec0ad340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:48:18.831 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:18.827+0000 7f83e3fff640 1 -- 172.21.15.43:0/3790366169 <== mon.0 v1:172.21.15.43:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4119697523 0 0) 0x7f83ec0cbd90 con 0x7f83ec0c8b30 2023-08-09T13:48:18.831 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:18.827+0000 7f83e3fff640 1 -- 172.21.15.43:0/3790366169 --> v1:172.21.15.43:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f83d8003520 con 0x7f83ec0c8b30 2023-08-09T13:48:18.831 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:18.827+0000 7f83e3fff640 1 -- 172.21.15.43:0/3790366169 <== mon.0 v1:172.21.15.43:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2638900485 0 0) 0x7f83d8003520 con 0x7f83ec0c8b30 2023-08-09T13:48:18.831 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:18.827+0000 7f83e3fff640 1 -- 172.21.15.43:0/3790366169 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f83ec0acdf0 msgr2=0x7f83ec0ad340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-09T13:48:18.831 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:18.827+0000 7f83e3fff640 1 --2- 172.21.15.43:0/3790366169 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f83ec0acdf0 0x7f83ec0ad340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:48:18.832 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:18.827+0000 7f83e3fff640 1 -- 172.21.15.43:0/3790366169 >> v1:172.21.15.203:6789/0 conn(0x7f83ec0c8750 legacy=0x7f83ec06f410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:48:18.832 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:18.828+0000 7f83e3fff640 1 -- 172.21.15.43:0/3790366169 --> v1:172.21.15.43:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f83ec0c9a20 con 0x7f83ec0c8b30 2023-08-09T13:48:18.832 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:18.828+0000 7f83e3fff640 1 -- 172.21.15.43:0/3790366169 <== mon.0 v1:172.21.15.43:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1285150740 0 0) 0x7f83dc001670 con 0x7f83ec0c8b30 2023-08-09T13:48:18.832 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:18.828+0000 7f83e3fff640 1 -- 172.21.15.43:0/3790366169 >> v1:172.21.15.43:6789/0 conn(0x7f83ec0c8b30 legacy=0x7f83ec070f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:48:18.832 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:18.828+0000 7f83e3fff640 1 --2- 172.21.15.43:0/3790366169 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f83d80039a0 0x7f83d8003dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:48:18.832 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:18.828+0000 7f83e3fff640 1 --2- 172.21.15.43:0/3790366169 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f83d80047c0 0x7f83d8004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:48:18.832 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:18.828+0000 7f83e3fff640 1 --2- 172.21.15.43:0/3790366169 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f83ec0acdf0 0x7f83d80051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:48:18.833 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:18.828+0000 7f83e3fff640 1 -- 172.21.15.43:0/3790366169 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f83ec15e810 con 0x7f83d80047c0 2023-08-09T13:48:18.833 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:18.828+0000 7f83e3fff640 1 -- 172.21.15.43:0/3790366169 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f83d800fec0 con 0x7f83d80039a0 2023-08-09T13:48:18.833 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:18.828+0000 7f83e3fff640 1 -- 172.21.15.43:0/3790366169 --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f83d8010080 con 0x7f83ec0acdf0 2023-08-09T13:48:18.833 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:18.828+0000 7f83f0c9c640 1 --2- 172.21.15.43:0/3790366169 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f83d80047c0 0x7f83d8004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:48:18.833 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:18.828+0000 7f83f149d640 1 --2- 172.21.15.43:0/3790366169 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f83d80039a0 0x7f83d8003dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:48:18.833 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:18.828+0000 7f83f1c9e640 1 --2- 172.21.15.43:0/3790366169 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f83ec0acdf0 0x7f83d80051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:48:18.834 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:18.828+0000 7f83f0c9c640 1 -- 172.21.15.43:0/3790366169 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f83ec0acdf0 msgr2=0x7f83d80051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-09T13:48:18.834 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:18.828+0000 7f83f0c9c640 1 --2- 172.21.15.43:0/3790366169 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f83ec0acdf0 0x7f83d80051c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:48:18.835 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:18.828+0000 7f83f0c9c640 1 -- 172.21.15.43:0/3790366169 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f83d80039a0 msgr2=0x7f83d8003dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-09T13:48:18.835 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:18.828+0000 7f83f0c9c640 1 --2- 172.21.15.43:0/3790366169 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f83d80039a0 0x7f83d8003dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:48:18.835 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:18.828+0000 7f83f0c9c640 1 -- 172.21.15.43:0/3790366169 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f83d8003520 con 0x7f83d80047c0 2023-08-09T13:48:18.835 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:18.830+0000 7f83f0c9c640 1 --2- 172.21.15.43:0/3790366169 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f83d80047c0 0x7f83d8004c40 secure :-1 s=READY pgs=843 cs=0 l=1 rev1=1 crypto rx=0x7f83dc00c190 tx=0x7f83d80057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=55f398b8ff922f3d server_cookie=0 in_seq=0 out_seq=0 2023-08-09T13:48:18.835 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:18.830+0000 7f83e3fff640 1 -- 172.21.15.43:0/3790366169 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f83dc005ad0 con 0x7f83d80047c0 2023-08-09T13:48:18.835 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:18.830+0000 7f83e3fff640 1 -- 172.21.15.43:0/3790366169 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f83dc005c30 con 0x7f83d80047c0 2023-08-09T13:48:18.835 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:18.830+0000 7f83e3fff640 1 -- 172.21.15.43:0/3790366169 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f83dc02c8a0 con 0x7f83d80047c0 2023-08-09T13:48:18.836 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:18.830+0000 7f83f249f640 1 -- 172.21.15.43:0/3790366169 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f83d80047c0 msgr2=0x7f83d8004c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:48:18.836 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:18.830+0000 7f83f249f640 1 --2- 172.21.15.43:0/3790366169 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f83d80047c0 0x7f83d8004c40 secure :-1 s=READY pgs=843 cs=0 l=1 rev1=1 crypto rx=0x7f83dc00c190 tx=0x7f83d80057f0 comp rx=0 tx=0).stop 2023-08-09T13:48:18.836 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:18.830+0000 7f83f1c9e640 1 -- 172.21.15.43:0/3790366169 reap_dead start 2023-08-09T13:48:18.836 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:18.831+0000 7f83f249f640 1 -- 172.21.15.43:0/3790366169 shutdown_connections 2023-08-09T13:48:18.836 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:18.831+0000 7f83f249f640 1 -- 172.21.15.43:0/3790366169 >> 172.21.15.43:0/3790366169 conn(0x7f83ec0cde90 msgr2=0x7f83ec0ce2a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-09T13:48:18.836 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:18.831+0000 7f83f249f640 1 -- 172.21.15.43:0/3790366169 shutdown_connections 2023-08-09T13:48:18.836 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:18.831+0000 7f83f249f640 1 -- 172.21.15.43:0/3790366169 wait complete. 2023-08-09T13:48:18.838 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:18.833+0000 7f83f249f640 1 Processor -- start 2023-08-09T13:48:18.838 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:18.833+0000 7f83f249f640 1 -- start start 2023-08-09T13:48:18.838 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:18.834+0000 7f83f249f640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f83ec156bb0 0x7f83ec158f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:48:18.838 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:18.834+0000 7f83f249f640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f83ec159460 0x7f83ec0c6890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:48:18.839 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:18.834+0000 7f83f249f640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f83ec0c6dd0 0x7f83ec0c4f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:48:18.839 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:18.834+0000 7f83f249f640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f83ec0c9f70 con 0x7f83ec156bb0 2023-08-09T13:48:18.839 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:18.834+0000 7f83f249f640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f83ec1598e0 con 0x7f83ec159460 2023-08-09T13:48:18.839 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:18.834+0000 7f83f249f640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f83ec0c7250 con 0x7f83ec0c6dd0 2023-08-09T13:48:18.839 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:18.834+0000 7f83f149d640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f83ec156bb0 0x7f83ec158f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:48:18.839 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:18.834+0000 7f83f149d640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f83ec156bb0 0x7f83ec158f20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:36754/0 (socket says 172.21.15.43:36754) 2023-08-09T13:48:18.839 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:18.834+0000 7f83f149d640 1 -- 172.21.15.43:0/161408112 learned_addr learned my addr 172.21.15.43:0/161408112 (peer_addr_for_me v2:172.21.15.43:0/0) 2023-08-09T13:48:18.839 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:18.834+0000 7f83f0c9c640 1 --2- 172.21.15.43:0/161408112 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f83ec159460 0x7f83ec0c6890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:48:18.840 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:18.834+0000 7f83f149d640 1 -- 172.21.15.43:0/161408112 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f83ec0c6dd0 msgr2=0x7f83ec0c4f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:48:18.840 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:18.834+0000 7f83f149d640 1 --2- 172.21.15.43:0/161408112 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f83ec0c6dd0 0x7f83ec0c4f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:48:18.840 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:18.834+0000 7f83f149d640 1 -- 172.21.15.43:0/161408112 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f83ec159460 msgr2=0x7f83ec0c6890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:48:18.840 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:18.835+0000 7f83f149d640 1 --2- 172.21.15.43:0/161408112 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f83ec159460 0x7f83ec0c6890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:48:18.840 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:18.835+0000 7f83f149d640 1 -- 172.21.15.43:0/161408112 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f83ec15c580 con 0x7f83ec156bb0 2023-08-09T13:48:18.840 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:18.835+0000 7f83f0c9c640 1 --2- 172.21.15.43:0/161408112 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f83ec159460 0x7f83ec0c6890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-08-09T13:48:18.840 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:18.835+0000 7f83f149d640 1 --2- 172.21.15.43:0/161408112 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f83ec156bb0 0x7f83ec158f20 secure :-1 s=READY pgs=844 cs=0 l=1 rev1=1 crypto rx=0x7f83e800e480 tx=0x7f83e800e940 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-09T13:48:18.841 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:18.835+0000 7f83e1ffb640 1 -- 172.21.15.43:0/161408112 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f83e80052e0 con 0x7f83ec156bb0 2023-08-09T13:48:18.841 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:18.835+0000 7f83e1ffb640 1 -- 172.21.15.43:0/161408112 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f83e8005440 con 0x7f83ec156bb0 2023-08-09T13:48:18.841 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:18.835+0000 7f83e1ffb640 1 -- 172.21.15.43:0/161408112 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f83e8003810 con 0x7f83ec156bb0 2023-08-09T13:48:18.841 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:18.835+0000 7f83f249f640 1 -- 172.21.15.43:0/161408112 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f83ec15c860 con 0x7f83ec156bb0 2023-08-09T13:48:18.841 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:18.836+0000 7f83f249f640 1 -- 172.21.15.43:0/161408112 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f83ec15ce20 con 0x7f83ec156bb0 2023-08-09T13:48:18.842 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:18.837+0000 7f83f249f640 1 -- 172.21.15.43:0/161408112 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f83ec0c9230 con 0x7f83ec156bb0 2023-08-09T13:48:18.842 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:18.838+0000 7f83e1ffb640 1 -- 172.21.15.43:0/161408112 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 18) v1 ==== 88443+0+0 (secure 0 0 0) 0x7f83e8003970 con 0x7f83ec156bb0 2023-08-09T13:48:18.842 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:18.838+0000 7f83e1ffb640 1 --2- 172.21.15.43:0/161408112 >> [v2:172.21.15.203:6832/272917756,v1:172.21.15.203:6833/272917756] conn(0x7f83d006a170 0x7f83d006c630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:48:18.842 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:18.839+0000 7f83e1ffb640 1 -- 172.21.15.43:0/161408112 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(73..73 src has 1..73) v4 ==== 6426+0+0 (secure 0 0 0) 0x7f83e8088590 con 0x7f83ec156bb0 2023-08-09T13:48:18.842 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:18.839+0000 7f83f0c9c640 1 --2- 172.21.15.43:0/161408112 >> [v2:172.21.15.203:6832/272917756,v1:172.21.15.203:6833/272917756] conn(0x7f83d006a170 0x7f83d006c630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:48:18.843 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:18.841+0000 7f83f0c9c640 1 --2- 172.21.15.43:0/161408112 >> [v2:172.21.15.203:6832/272917756,v1:172.21.15.203:6833/272917756] conn(0x7f83d006a170 0x7f83d006c630 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f83dc034000 tx=0x7f83dc009030 comp rx=0 tx=0).ready entity=mgr.5533 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-09T13:48:18.845 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:18.843+0000 7f83e1ffb640 1 -- 172.21.15.43:0/161408112 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178066 (secure 0 0 0) 0x7f83e80539f0 con 0x7f83ec156bb0 2023-08-09T13:48:19.047 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:19.044+0000 7f83f249f640 1 -- 172.21.15.43:0/161408112 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f83ec0739f0 con 0x7f83ec156bb0 2023-08-09T13:48:19.048 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:19.046+0000 7f83e1ffb640 1 -- 172.21.15.43:0/161408112 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v73) v1 ==== 74+0+18565 (secure 0 0 0) 0x7f83e8057010 con 0x7f83ec156bb0 2023-08-09T13:48:19.049 INFO:teuthology.orchestra.run.smithi043.stdout: 2023-08-09T13:48:19.049 INFO:teuthology.orchestra.run.smithi043.stdout:{"epoch":73,"fsid":"daac87ee-1337-4250-a955-aea4e408a39b","created":"2023-08-09T13:33:41.701198+0000","modified":"2023-08-09T13:48:17.606423+0000","last_up_change":"2023-08-09T13:33:53.783453+0000","last_in_change":"2023-08-09T13:33:45.148904+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":9,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":"rbd","create_time":"2023-08-09T13:33:54.365137+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":".mgr","create_time":"2023-08-09T13:33:54.831803+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":8,"pool_name":"cephfs_metadata","create_time":"2023-08-09T13:48:16.039756+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"73","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":9,"pool_name":"cephfs_data","create_time":"2023-08-09T13:48:16.615400+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"73","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.6299999952316284,"score_stable":1.6299999952316284,"optimal_score":1,"raw_score_acting":1.6299999952316284,"raw_score_stable":1.6299999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"cbd49ecd-e896-4fbd-864e-5c37a829b165","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":72,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6800","nonce":502667335},{"type":"v1","addr":"172.21.15.43:6801","nonce":502667335}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6802","nonce":502667335},{"type":"v1","addr":"172.21.15.43:6803","nonce":502667335}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6806","nonce":502667335},{"type":"v1","addr":"172.21.15.43:6807","nonce":502667335}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6804","nonce":502667335},{"type":"v1","addr":"172.21.15.43:6805","nonce":502667335}]},"public_addr":"172.21.15.43:6801/502667335","cluster_addr":"172.21.15.43:6803/502667335","heartbeat_back_addr":"172.21.15.43:6807/502667335","heartbeat_front_addr":"172.21.15.43:6805/502667335","state":["exists","up"]},{"osd":1,"uuid":"75c7e0fe-1ec7-4fe8-a2f2-3661a65b8500","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":72,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6808","nonce":3289134610},{"type":"v1","addr":"172.21.15.43:6809","nonce":3289134610}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6810","nonce":3289134610},{"type":"v1","addr":"172.21.15.43:6812","nonce":3289134610}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6821","nonce":3289134610},{"type":"v1","addr":"172.21.15.43:6824","nonce":3289134610}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6815","nonce":3289134610},{"type":"v1","addr":"172.21.15.43:6818","nonce":3289134610}]},"public_addr":"172.21.15.43:6809/3289134610","cluster_addr":"172.21.15.43:6812/3289134610","heartbeat_back_addr":"172.21.15.43:6824/3289134610","heartbeat_front_addr":"172.21.15.43:6818/3289134610","state":["exists","up"]},{"osd":2,"uuid":"bb5c41e9-7f5f-4125-9b01-3bb8bfc2ea83","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":72,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6811","nonce":4079709736},{"type":"v1","addr":"172.21.15.43:6814","nonce":4079709736}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6817","nonce":4079709736},{"type":"v1","addr":"172.21.15.43:6820","nonce":4079709736}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6829","nonce":4079709736},{"type":"v1","addr":"172.21.15.43:6831","nonce":4079709736}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6823","nonce":4079709736},{"type":"v1","addr":"172.21.15.43:6826","nonce":4079709736}]},"public_addr":"172.21.15.43:6814/4079709736","cluster_addr":"172.21.15.43:6820/4079709736","heartbeat_back_addr":"172.21.15.43:6831/4079709736","heartbeat_front_addr":"172.21.15.43:6826/4079709736","state":["exists","up"]},{"osd":3,"uuid":"bfe4c8b4-694a-465c-87e6-2b2aab852613","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":72,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6813","nonce":1709538759},{"type":"v1","addr":"172.21.15.43:6816","nonce":1709538759}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6819","nonce":1709538759},{"type":"v1","addr":"172.21.15.43:6822","nonce":1709538759}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6828","nonce":1709538759},{"type":"v1","addr":"172.21.15.43:6830","nonce":1709538759}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6825","nonce":1709538759},{"type":"v1","addr":"172.21.15.43:6827","nonce":1709538759}]},"public_addr":"172.21.15.43:6816/1709538759","cluster_addr":"172.21.15.43:6822/1709538759","heartbeat_back_addr":"172.21.15.43:6830/1709538759","heartbeat_front_addr":"172.21.15.43:6827/1709538759","state":["exists","up"]},{"osd":4,"uuid":"fa2e8a77-aba1-4756-853e-0d174a5bdeba","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":72,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6800","nonce":4196866481},{"type":"v1","addr":"172.21.15.203:6801","nonce":4196866481}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6802","nonce":4196866481},{"type":"v1","addr":"172.21.15.203:6803","nonce":4196866481}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6806","nonce":4196866481},{"type":"v1","addr":"172.21.15.203:6807","nonce":4196866481}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6804","nonce":4196866481},{"type":"v1","addr":"172.21.15.203:6805","nonce":4196866481}]},"public_addr":"172.21.15.203:6801/4196866481","cluster_addr":"172.21.15.203:6803/4196866481","heartbeat_back_addr":"172.21.15.203:6807/4196866481","heartbeat_front_addr":"172.21.15.203:6805/4196866481","state":["exists","up"]},{"osd":5,"uuid":"81811e4b-4d24-4f51-a82c-f1d700a2a00d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":72,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6816","nonce":3059290081},{"type":"v1","addr":"172.21.15.203:6817","nonce":3059290081}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6818","nonce":3059290081},{"type":"v1","addr":"172.21.15.203:6819","nonce":3059290081}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6822","nonce":3059290081},{"type":"v1","addr":"172.21.15.203:6823","nonce":3059290081}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6820","nonce":3059290081},{"type":"v1","addr":"172.21.15.203:6821","nonce":3059290081}]},"public_addr":"172.21.15.203:6817/3059290081","cluster_addr":"172.21.15.203:6819/3059290081","heartbeat_back_addr":"172.21.15.203:6823/3059290081","heartbeat_front_addr":"172.21.15.203:6821/3059290081","state":["exists","up"]},{"osd":6,"uuid":"307bac4d-6a5b-443d-af75-56d695d7674f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":72,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6824","nonce":512155569},{"type":"v1","addr":"172.21.15.203:6825","nonce":512155569}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6826","nonce":512155569},{"type":"v1","addr":"172.21.15.203:6827","nonce":512155569}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6830","nonce":512155569},{"type":"v1","addr":"172.21.15.203:6831","nonce":512155569}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6828","nonce":512155569},{"type":"v1","addr":"172.21.15.203:6829","nonce":512155569}]},"public_addr":"172.21.15.203:6825/512155569","cluster_addr":"172.21.15.203:6827/512155569","heartbeat_back_addr":"172.21.15.203:6831/512155569","heartbeat_front_addr":"172.21.15.203:6829/512155569","state":["exists","up"]},{"osd":7,"uuid":"a95dc7ca-9f1b-4295-af70-68a22a2d7d5e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":72,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6808","nonce":3331450438},{"type":"v1","addr":"172.21.15.203:6809","nonce":3331450438}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6810","nonce":3331450438},{"type":"v1","addr":"172.21.15.203:6811","nonce":3331450438}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6814","nonce":3331450438},{"type":"v1","addr":"172.21.15.203:6815","nonce":3331450438}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6812","nonce":3331450438},{"type":"v1","addr":"172.21.15.203:6813","nonce":3331450438}]},"public_addr":"172.21.15.203:6809/3331450438","cluster_addr":"172.21.15.203:6811/3331450438","heartbeat_back_addr":"172.21.15.203:6815/3331450438","heartbeat_front_addr":"172.21.15.203:6813/3331450438","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-09T13:33:50.547652+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-09T13:33:50.589469+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-09T13:33:50.561724+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-09T13:33:50.599650+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-09T13:33:50.411855+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-09T13:33:50.219681+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-09T13:33:50.504664+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-09T13:33:50.441248+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":7,"snaps":[{"begin":104,"length":1},{"begin":221,"length":1}]}],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-08-09T13:48:19.051 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:19.049+0000 7f83f249f640 1 -- 172.21.15.43:0/161408112 >> [v2:172.21.15.203:6832/272917756,v1:172.21.15.203:6833/272917756] conn(0x7f83d006a170 msgr2=0x7f83d006c630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:48:19.052 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:19.049+0000 7f83f249f640 1 --2- 172.21.15.43:0/161408112 >> [v2:172.21.15.203:6832/272917756,v1:172.21.15.203:6833/272917756] conn(0x7f83d006a170 0x7f83d006c630 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f83dc034000 tx=0x7f83dc009030 comp rx=0 tx=0).stop 2023-08-09T13:48:19.052 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:19.049+0000 7f83f249f640 1 -- 172.21.15.43:0/161408112 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f83ec156bb0 msgr2=0x7f83ec158f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:48:19.052 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:19.049+0000 7f83f249f640 1 --2- 172.21.15.43:0/161408112 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f83ec156bb0 0x7f83ec158f20 secure :-1 s=READY pgs=844 cs=0 l=1 rev1=1 crypto rx=0x7f83e800e480 tx=0x7f83e800e940 comp rx=0 tx=0).stop 2023-08-09T13:48:19.052 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:19.049+0000 7f83f249f640 1 -- 172.21.15.43:0/161408112 shutdown_connections 2023-08-09T13:48:19.052 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:19.049+0000 7f83f249f640 1 --2- 172.21.15.43:0/161408112 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f83ec0c6dd0 0x7f83ec0c4f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:48:19.053 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:19.049+0000 7f83f249f640 1 --2- 172.21.15.43:0/161408112 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f83ec159460 0x7f83ec0c6890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:48:19.053 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:19.049+0000 7f83f249f640 1 --2- 172.21.15.43:0/161408112 >> [v2:172.21.15.203:6832/272917756,v1:172.21.15.203:6833/272917756] conn(0x7f83d006a170 0x7f83d006c630 unknown :-1 s=CLOSED pgs=60 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:48:19.053 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:19.049+0000 7f83f249f640 1 --2- 172.21.15.43:0/161408112 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f83ec156bb0 0x7f83ec158f20 unknown :-1 s=CLOSED pgs=844 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:48:19.053 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:19.049+0000 7f83f249f640 1 -- 172.21.15.43:0/161408112 >> 172.21.15.43:0/161408112 conn(0x7f83ec0cde90 msgr2=0x7f83ec06b190 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-09T13:48:19.053 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:19.049+0000 7f83f249f640 1 -- 172.21.15.43:0/161408112 shutdown_connections 2023-08-09T13:48:19.053 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:19.049+0000 7f83f249f640 1 -- 172.21.15.43:0/161408112 wait complete. 2023-08-09T13:48:19.070 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs standby_count_wanted 0 2023-08-09T13:48:19.290 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:19.287+0000 7f79c8b74640 1 Processor -- start 2023-08-09T13:48:19.290 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:19.288+0000 7f79c8b74640 1 -- start start 2023-08-09T13:48:19.291 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:19.289+0000 7f79c8b74640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f79c41590b0 0x7f79c40cafa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:48:19.291 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:19.289+0000 7f79c8b74640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f79c40c96a0 0x7f79c40cb9f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:48:19.291 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:19.289+0000 7f79c8b74640 1 -- --> v1:172.21.15.203:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f79c40b07e0 con 0x7f79c406fd40 2023-08-09T13:48:19.292 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:19.289+0000 7f79c8b74640 1 -- --> v2:172.21.15.43:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f79c40701a0 con 0x7f79c40c96a0 2023-08-09T13:48:19.292 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:19.289+0000 7f79c8b74640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f79c4070420 con 0x7f79c41590b0 2023-08-09T13:48:19.292 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:19.289+0000 7f79c3fff640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f79c40c96a0 0x7f79c40cb9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:48:19.293 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:19.290+0000 7f79c3fff640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f79c40c96a0 0x7f79c40cb9f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:36762/0 (socket says 172.21.15.43:36762) 2023-08-09T13:48:19.293 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:19.290+0000 7f79c3fff640 1 -- 172.21.15.43:0/314938774 learned_addr learned my addr 172.21.15.43:0/314938774 (peer_addr_for_me v2:172.21.15.43:0/0) 2023-08-09T13:48:19.293 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:19.290+0000 7f79c37fe640 1 --2- 172.21.15.43:0/314938774 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f79c41590b0 0x7f79c40cafa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:48:19.293 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:19.290+0000 7f79c3fff640 1 -- 172.21.15.43:0/314938774 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f79c41590b0 msgr2=0x7f79c40cafa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-09T13:48:19.293 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:19.290+0000 7f79c3fff640 1 --2- 172.21.15.43:0/314938774 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f79c41590b0 0x7f79c40cafa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:48:19.293 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:19.290+0000 7f79c3fff640 1 -- 172.21.15.43:0/314938774 >> v1:172.21.15.203:6789/0 conn(0x7f79c406fd40 legacy=0x7f79c40c8f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:48:19.294 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:19.290+0000 7f79c3fff640 1 -- 172.21.15.43:0/314938774 --> v2:172.21.15.43:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f79c4070700 con 0x7f79c40c96a0 2023-08-09T13:48:19.294 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:19.290+0000 7f79c37fe640 1 --2- 172.21.15.43:0/314938774 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f79c41590b0 0x7f79c40cafa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-08-09T13:48:19.294 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:19.292+0000 7f79c3fff640 1 --2- 172.21.15.43:0/314938774 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f79c40c96a0 0x7f79c40cb9f0 secure :-1 s=READY pgs=845 cs=0 l=1 rev1=1 crypto rx=0x7f79bc009e50 tx=0x7f79bc02bf60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a71034941599a756 server_cookie=0 in_seq=0 out_seq=0 2023-08-09T13:48:19.294 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:19.292+0000 7f79c27fc640 1 -- 172.21.15.43:0/314938774 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f79bc02cb00 con 0x7f79c40c96a0 2023-08-09T13:48:19.295 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:19.292+0000 7f79c27fc640 1 -- 172.21.15.43:0/314938774 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f79bc02cc60 con 0x7f79c40c96a0 2023-08-09T13:48:19.295 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:19.292+0000 7f79c27fc640 1 -- 172.21.15.43:0/314938774 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f79bc0343f0 con 0x7f79c40c96a0 2023-08-09T13:48:19.295 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:19.293+0000 7f79c8b74640 1 -- 172.21.15.43:0/314938774 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f79c40c96a0 msgr2=0x7f79c40cb9f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:48:19.296 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:19.293+0000 7f79c8b74640 1 --2- 172.21.15.43:0/314938774 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f79c40c96a0 0x7f79c40cb9f0 secure :-1 s=READY pgs=845 cs=0 l=1 rev1=1 crypto rx=0x7f79bc009e50 tx=0x7f79bc02bf60 comp rx=0 tx=0).stop 2023-08-09T13:48:19.296 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:19.293+0000 7f79c8b74640 1 -- 172.21.15.43:0/314938774 shutdown_connections 2023-08-09T13:48:19.296 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:19.293+0000 7f79c8b74640 1 --2- 172.21.15.43:0/314938774 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f79c40c96a0 0x7f79c40cb9f0 unknown :-1 s=CLOSED pgs=845 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:48:19.296 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:19.293+0000 7f79c8b74640 1 --2- 172.21.15.43:0/314938774 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f79c41590b0 0x7f79c40cafa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:48:19.296 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:19.293+0000 7f79c8b74640 1 -- 172.21.15.43:0/314938774 >> 172.21.15.43:0/314938774 conn(0x7f79c40af9f0 msgr2=0x7f79c40afdd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-09T13:48:19.296 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:19.293+0000 7f79c8b74640 1 -- 172.21.15.43:0/314938774 shutdown_connections 2023-08-09T13:48:19.297 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:19.293+0000 7f79c8b74640 1 -- 172.21.15.43:0/314938774 wait complete. 2023-08-09T13:48:19.297 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:19.294+0000 7f79c8b74640 1 Processor -- start 2023-08-09T13:48:19.297 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:19.295+0000 7f79c8b74640 1 -- start start 2023-08-09T13:48:19.298 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:19.296+0000 7f79c8b74640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f79c406fd40 0x7f79c41f0190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:48:19.298 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:19.296+0000 7f79c8b74640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f79c40c96a0 0x7f79c414f310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:48:19.298 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:19.296+0000 7f79c8b74640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f79c41590b0 0x7f79c41f34d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:48:19.298 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:19.296+0000 7f79c8b74640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f79c4153690 con 0x7f79c41590b0 2023-08-09T13:48:19.298 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:19.296+0000 7f79c8b74640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f79c41ef780 con 0x7f79c40c96a0 2023-08-09T13:48:19.299 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:19.296+0000 7f79c8b74640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f79c41f06d0 con 0x7f79c406fd40 2023-08-09T13:48:19.300 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:19.296+0000 7f79c3fff640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f79c41590b0 0x7f79c41f34d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:48:19.300 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:19.296+0000 7f79c3fff640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f79c41590b0 0x7f79c41f34d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:36772/0 (socket says 172.21.15.43:36772) 2023-08-09T13:48:19.300 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:19.296+0000 7f79c3fff640 1 -- 172.21.15.43:0/818947371 learned_addr learned my addr 172.21.15.43:0/818947371 (peer_addr_for_me v2:172.21.15.43:0/0) 2023-08-09T13:48:19.300 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:19.296+0000 7f79c37fe640 1 --2- 172.21.15.43:0/818947371 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f79c406fd40 0x7f79c41f0190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:48:19.300 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:19.296+0000 7f79c2ffd640 1 --2- 172.21.15.43:0/818947371 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f79c40c96a0 0x7f79c414f310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:48:19.301 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:19.297+0000 7f79c3fff640 1 -- 172.21.15.43:0/818947371 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f79c406fd40 msgr2=0x7f79c41f0190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:48:19.301 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:19.297+0000 7f79c3fff640 1 --2- 172.21.15.43:0/818947371 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f79c406fd40 0x7f79c41f0190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:48:19.301 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:19.297+0000 7f79c3fff640 1 -- 172.21.15.43:0/818947371 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f79c40c96a0 msgr2=0x7f79c414f310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:48:19.301 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:19.297+0000 7f79c3fff640 1 --2- 172.21.15.43:0/818947371 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f79c40c96a0 0x7f79c414f310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:48:19.301 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:19.297+0000 7f79c3fff640 1 -- 172.21.15.43:0/818947371 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f79bc009b30 con 0x7f79c41590b0 2023-08-09T13:48:19.301 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:19.297+0000 7f79c3fff640 1 --2- 172.21.15.43:0/818947371 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f79c41590b0 0x7f79c41f34d0 secure :-1 s=READY pgs=846 cs=0 l=1 rev1=1 crypto rx=0x7f79bc009e50 tx=0x7f79bc004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-09T13:48:19.301 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:19.297+0000 7f79bbfff640 1 -- 172.21.15.43:0/818947371 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f79bc009510 con 0x7f79c41590b0 2023-08-09T13:48:19.301 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:19.297+0000 7f79bbfff640 1 -- 172.21.15.43:0/818947371 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f79bc03e450 con 0x7f79c41590b0 2023-08-09T13:48:19.302 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:19.297+0000 7f79bbfff640 1 -- 172.21.15.43:0/818947371 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f79bc03e800 con 0x7f79c41590b0 2023-08-09T13:48:19.302 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:19.297+0000 7f79c8b74640 1 -- 172.21.15.43:0/818947371 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f79c41530d0 con 0x7f79c41590b0 2023-08-09T13:48:19.302 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:19.298+0000 7f79c8b74640 1 -- 172.21.15.43:0/818947371 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f79c41ef990 con 0x7f79c41590b0 2023-08-09T13:48:19.302 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:19.299+0000 7f79c8b74640 1 -- 172.21.15.43:0/818947371 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f79c40c9410 con 0x7f79c41590b0 2023-08-09T13:48:19.302 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:19.300+0000 7f79bbfff640 1 -- 172.21.15.43:0/818947371 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 18) v1 ==== 88443+0+0 (secure 0 0 0) 0x7f79bc03e960 con 0x7f79c41590b0 2023-08-09T13:48:19.303 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:19.301+0000 7f79bbfff640 1 --2- 172.21.15.43:0/818947371 >> [v2:172.21.15.203:6832/272917756,v1:172.21.15.203:6833/272917756] conn(0x7f79a406a170 0x7f79a406c630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:48:19.303 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:19.301+0000 7f79bbfff640 1 -- 172.21.15.43:0/818947371 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(74..74 src has 1..74) v4 ==== 6426+0+0 (secure 0 0 0) 0x7f79bc0ab570 con 0x7f79c41590b0 2023-08-09T13:48:19.303 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:19.301+0000 7f79c37fe640 1 --2- 172.21.15.43:0/818947371 >> [v2:172.21.15.203:6832/272917756,v1:172.21.15.203:6833/272917756] conn(0x7f79a406a170 0x7f79a406c630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:48:19.305 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:19.303+0000 7f79c37fe640 1 --2- 172.21.15.43:0/818947371 >> [v2:172.21.15.203:6832/272917756,v1:172.21.15.203:6833/272917756] conn(0x7f79a406a170 0x7f79a406c630 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f79c4155d10 tx=0x7f79b4008040 comp rx=0 tx=0).ready entity=mgr.5533 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-09T13:48:19.307 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:19.305+0000 7f79bbfff640 1 -- 172.21.15.43:0/818947371 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178066 (secure 0 0 0) 0x7f79bc076a70 con 0x7f79c41590b0 2023-08-09T13:48:19.542 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:19.539+0000 7f79c8b74640 1 -- 172.21.15.43:0/818947371 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} v 0) v1 -- 0x7f79c404e390 con 0x7f79c41590b0 2023-08-09T13:48:20.183 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:20.181+0000 7f79bbfff640 1 -- 172.21.15.43:0/818947371 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]=0 v25) v1 ==== 118+0+0 (secure 0 0 0) 0x7f79bc07a090 con 0x7f79c41590b0 2023-08-09T13:48:20.188 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:20.185+0000 7f79c8b74640 1 -- 172.21.15.43:0/818947371 >> [v2:172.21.15.203:6832/272917756,v1:172.21.15.203:6833/272917756] conn(0x7f79a406a170 msgr2=0x7f79a406c630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:48:20.188 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:20.185+0000 7f79c8b74640 1 --2- 172.21.15.43:0/818947371 >> [v2:172.21.15.203:6832/272917756,v1:172.21.15.203:6833/272917756] conn(0x7f79a406a170 0x7f79a406c630 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f79c4155d10 tx=0x7f79b4008040 comp rx=0 tx=0).stop 2023-08-09T13:48:20.188 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:20.185+0000 7f79c8b74640 1 -- 172.21.15.43:0/818947371 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f79c41590b0 msgr2=0x7f79c41f34d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:48:20.188 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:20.185+0000 7f79c8b74640 1 --2- 172.21.15.43:0/818947371 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f79c41590b0 0x7f79c41f34d0 secure :-1 s=READY pgs=846 cs=0 l=1 rev1=1 crypto rx=0x7f79bc009e50 tx=0x7f79bc004a40 comp rx=0 tx=0).stop 2023-08-09T13:48:20.189 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:20.185+0000 7f79c8b74640 1 -- 172.21.15.43:0/818947371 shutdown_connections 2023-08-09T13:48:20.189 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:20.185+0000 7f79c8b74640 1 --2- 172.21.15.43:0/818947371 >> [v2:172.21.15.203:6832/272917756,v1:172.21.15.203:6833/272917756] conn(0x7f79a406a170 0x7f79a406c630 unknown :-1 s=CLOSED pgs=61 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:48:20.189 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:20.185+0000 7f79c8b74640 1 --2- 172.21.15.43:0/818947371 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f79c41590b0 0x7f79c41f34d0 unknown :-1 s=CLOSED pgs=846 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:48:20.189 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:20.185+0000 7f79c8b74640 1 --2- 172.21.15.43:0/818947371 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f79c40c96a0 0x7f79c414f310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:48:20.189 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:20.185+0000 7f79c8b74640 1 --2- 172.21.15.43:0/818947371 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f79c406fd40 0x7f79c41f0190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:48:20.189 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:20.185+0000 7f79c8b74640 1 -- 172.21.15.43:0/818947371 >> 172.21.15.43:0/818947371 conn(0x7f79c40af9f0 msgr2=0x7f79c4157e30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-09T13:48:20.189 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:20.185+0000 7f79c8b74640 1 -- 172.21.15.43:0/818947371 shutdown_connections 2023-08-09T13:48:20.189 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:20.186+0000 7f79c8b74640 1 -- 172.21.15.43:0/818947371 wait complete. 2023-08-09T13:48:20.213 DEBUG:tasks.cephfs.filesystem:fs_config: {} 2023-08-09T13:48:20.213 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs allow_standby_replay false 2023-08-09T13:48:20.449 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:20.447+0000 7fd92a61a640 1 Processor -- start 2023-08-09T13:48:20.450 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:20.447+0000 7fd92a61a640 1 -- start start 2023-08-09T13:48:20.451 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:20.449+0000 7fd92a61a640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fd9240b0f00 0x7fd9240ccef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:48:20.451 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:20.449+0000 7fd92a61a640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7fd9240b1450 0x7fd9240cd430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:48:20.452 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:20.449+0000 7fd92a61a640 1 -- --> v1:172.21.15.203:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd9240b7c00 con 0x7fd92415c3a0 2023-08-09T13:48:20.452 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:20.449+0000 7fd92a61a640 1 -- --> v2:172.21.15.43:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd9240b81c0 con 0x7fd9240b1450 2023-08-09T13:48:20.452 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:20.449+0000 7fd92a61a640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd9240b7ee0 con 0x7fd9240b0f00 2023-08-09T13:48:20.453 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:20.449+0000 7fd929e19640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7fd9240b1450 0x7fd9240cd430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:48:20.453 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:20.449+0000 7fd929e19640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7fd9240b1450 0x7fd9240cd430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:36786/0 (socket says 172.21.15.43:36786) 2023-08-09T13:48:20.453 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:20.449+0000 7fd929e19640 1 -- 172.21.15.43:0/2934621650 learned_addr learned my addr 172.21.15.43:0/2934621650 (peer_addr_for_me v2:172.21.15.43:0/0) 2023-08-09T13:48:20.453 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:20.449+0000 7fd928e17640 1 --2- 172.21.15.43:0/2934621650 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fd9240b0f00 0x7fd9240ccef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:48:20.453 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:20.450+0000 7fd929e19640 1 -- 172.21.15.43:0/2934621650 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fd9240b0f00 msgr2=0x7fd9240ccef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-09T13:48:20.454 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:20.450+0000 7fd929e19640 1 --2- 172.21.15.43:0/2934621650 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fd9240b0f00 0x7fd9240ccef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:48:20.454 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:20.450+0000 7fd929e19640 1 -- 172.21.15.43:0/2934621650 >> v1:172.21.15.203:6789/0 conn(0x7fd92415c3a0 legacy=0x7fd9241608a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:48:20.454 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:20.450+0000 7fd929e19640 1 -- 172.21.15.43:0/2934621650 --> v2:172.21.15.43:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd9241600c0 con 0x7fd9240b1450 2023-08-09T13:48:20.454 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:20.451+0000 7fd929e19640 1 --2- 172.21.15.43:0/2934621650 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fd9240b1450 0x7fd9240cd430 secure :-1 s=READY pgs=847 cs=0 l=1 rev1=1 crypto rx=0x7fd91c009e50 tx=0x7fd91c02bed0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e038c18cdb76903e server_cookie=0 in_seq=0 out_seq=0 2023-08-09T13:48:20.455 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:20.452+0000 7fd91bfff640 1 -- 172.21.15.43:0/2934621650 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd91c02ca70 con 0x7fd9240b1450 2023-08-09T13:48:20.455 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:20.452+0000 7fd91bfff640 1 -- 172.21.15.43:0/2934621650 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd91c02cbd0 con 0x7fd9240b1450 2023-08-09T13:48:20.455 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:20.452+0000 7fd91bfff640 1 -- 172.21.15.43:0/2934621650 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd91c0343f0 con 0x7fd9240b1450 2023-08-09T13:48:20.456 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:20.452+0000 7fd92a61a640 1 -- 172.21.15.43:0/2934621650 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fd9240b1450 msgr2=0x7fd9240cd430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:48:20.456 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:20.452+0000 7fd92a61a640 1 --2- 172.21.15.43:0/2934621650 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fd9240b1450 0x7fd9240cd430 secure :-1 s=READY pgs=847 cs=0 l=1 rev1=1 crypto rx=0x7fd91c009e50 tx=0x7fd91c02bed0 comp rx=0 tx=0).stop 2023-08-09T13:48:20.456 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:20.453+0000 7fd92a61a640 1 -- 172.21.15.43:0/2934621650 shutdown_connections 2023-08-09T13:48:20.456 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:20.453+0000 7fd92a61a640 1 --2- 172.21.15.43:0/2934621650 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fd9240b1450 0x7fd9240cd430 unknown :-1 s=CLOSED pgs=847 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:48:20.456 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:20.453+0000 7fd92a61a640 1 --2- 172.21.15.43:0/2934621650 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fd9240b0f00 0x7fd9240ccef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:48:20.456 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:20.453+0000 7fd92a61a640 1 -- 172.21.15.43:0/2934621650 >> 172.21.15.43:0/2934621650 conn(0x7fd92406a210 msgr2=0x7fd92406a620 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-09T13:48:20.456 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:20.453+0000 7fd92a61a640 1 -- 172.21.15.43:0/2934621650 shutdown_connections 2023-08-09T13:48:20.457 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:20.453+0000 7fd92a61a640 1 -- 172.21.15.43:0/2934621650 wait complete. 2023-08-09T13:48:20.457 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:20.455+0000 7fd92a61a640 1 Processor -- start 2023-08-09T13:48:20.457 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:20.455+0000 7fd92a61a640 1 -- start start 2023-08-09T13:48:20.458 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:20.456+0000 7fd92a61a640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fd9240b0f00 0x7fd9240c68f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:48:20.458 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:20.456+0000 7fd92a61a640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fd9240b1450 0x7fd9241537e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:48:20.458 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:20.456+0000 7fd92a61a640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fd92415c3a0 0x7fd92415db70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:48:20.459 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:20.456+0000 7fd92a61a640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd9240c6e30 con 0x7fd9240b1450 2023-08-09T13:48:20.459 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:20.456+0000 7fd92a61a640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd9241f6510 con 0x7fd92415c3a0 2023-08-09T13:48:20.459 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:20.456+0000 7fd92a61a640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd9240c6460 con 0x7fd9240b0f00 2023-08-09T13:48:20.460 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:20.456+0000 7fd928e17640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fd9240b1450 0x7fd9241537e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:48:20.460 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:20.456+0000 7fd928e17640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fd9240b1450 0x7fd9241537e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:36798/0 (socket says 172.21.15.43:36798) 2023-08-09T13:48:20.460 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:20.456+0000 7fd928e17640 1 -- 172.21.15.43:0/4279439694 learned_addr learned my addr 172.21.15.43:0/4279439694 (peer_addr_for_me v2:172.21.15.43:0/0) 2023-08-09T13:48:20.461 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:20.456+0000 7fd929618640 1 --2- 172.21.15.43:0/4279439694 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fd9240b0f00 0x7fd9240c68f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:48:20.461 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:20.457+0000 7fd928e17640 1 -- 172.21.15.43:0/4279439694 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fd9240b0f00 msgr2=0x7fd9240c68f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:48:20.461 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:20.457+0000 7fd929e19640 1 --2- 172.21.15.43:0/4279439694 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fd92415c3a0 0x7fd92415db70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:48:20.461 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:20.457+0000 7fd928e17640 1 --2- 172.21.15.43:0/4279439694 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fd9240b0f00 0x7fd9240c68f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:48:20.461 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:20.457+0000 7fd928e17640 1 -- 172.21.15.43:0/4279439694 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fd92415c3a0 msgr2=0x7fd92415db70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:48:20.461 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:20.457+0000 7fd928e17640 1 --2- 172.21.15.43:0/4279439694 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fd92415c3a0 0x7fd92415db70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:48:20.461 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:20.457+0000 7fd928e17640 1 -- 172.21.15.43:0/4279439694 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd91c009b30 con 0x7fd9240b1450 2023-08-09T13:48:20.462 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:20.457+0000 7fd929618640 1 --2- 172.21.15.43:0/4279439694 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fd9240b0f00 0x7fd9240c68f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-08-09T13:48:20.462 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:20.457+0000 7fd929e19640 1 --2- 172.21.15.43:0/4279439694 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fd92415c3a0 0x7fd92415db70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-08-09T13:48:20.462 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:20.457+0000 7fd928e17640 1 --2- 172.21.15.43:0/4279439694 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fd9240b1450 0x7fd9241537e0 secure :-1 s=READY pgs=848 cs=0 l=1 rev1=1 crypto rx=0x7fd91400e8a0 tx=0x7fd91400ed60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-09T13:48:20.462 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:20.457+0000 7fd919ffb640 1 -- 172.21.15.43:0/4279439694 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd9140095c0 con 0x7fd9240b1450 2023-08-09T13:48:20.462 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:20.457+0000 7fd919ffb640 1 -- 172.21.15.43:0/4279439694 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd91400c940 con 0x7fd9240b1450 2023-08-09T13:48:20.462 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:20.457+0000 7fd919ffb640 1 -- 172.21.15.43:0/4279439694 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd91400ccb0 con 0x7fd9240b1450 2023-08-09T13:48:20.462 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:20.457+0000 7fd92a61a640 1 -- 172.21.15.43:0/4279439694 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd9241f6230 con 0x7fd9240b1450 2023-08-09T13:48:20.462 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:20.458+0000 7fd92a61a640 1 -- 172.21.15.43:0/4279439694 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd9240c60f0 con 0x7fd9240b1450 2023-08-09T13:48:20.463 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:20.459+0000 7fd92a61a640 1 -- 172.21.15.43:0/4279439694 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd9240b72c0 con 0x7fd9240b1450 2023-08-09T13:48:20.463 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:20.460+0000 7fd919ffb640 1 -- 172.21.15.43:0/4279439694 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 18) v1 ==== 88443+0+0 (secure 0 0 0) 0x7fd914013470 con 0x7fd9240b1450 2023-08-09T13:48:20.463 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:20.461+0000 7fd919ffb640 1 --2- 172.21.15.43:0/4279439694 >> [v2:172.21.15.203:6832/272917756,v1:172.21.15.203:6833/272917756] conn(0x7fd90806a170 0x7fd90806c630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:48:20.464 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:20.462+0000 7fd919ffb640 1 -- 172.21.15.43:0/4279439694 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(74..74 src has 1..74) v4 ==== 6426+0+0 (secure 0 0 0) 0x7fd914088df0 con 0x7fd9240b1450 2023-08-09T13:48:20.464 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:20.462+0000 7fd929618640 1 --2- 172.21.15.43:0/4279439694 >> [v2:172.21.15.203:6832/272917756,v1:172.21.15.203:6833/272917756] conn(0x7fd90806a170 0x7fd90806c630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:48:20.465 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:20.463+0000 7fd929618640 1 --2- 172.21.15.43:0/4279439694 >> [v2:172.21.15.203:6832/272917756,v1:172.21.15.203:6833/272917756] conn(0x7fd90806a170 0x7fd90806c630 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7fd920000f60 tx=0x7fd920009040 comp rx=0 tx=0).ready entity=mgr.5533 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-09T13:48:20.468 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:20.466+0000 7fd919ffb640 1 -- 172.21.15.43:0/4279439694 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178066 (secure 0 0 0) 0x7fd914057ce0 con 0x7fd9240b1450 2023-08-09T13:48:20.698 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:20.696+0000 7fd92a61a640 1 -- 172.21.15.43:0/4279439694 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"} v 0) v1 -- 0x7fd92404e390 con 0x7fd9240b1450 2023-08-09T13:48:21.196 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:21.194+0000 7fd919ffb640 1 -- 172.21.15.43:0/4279439694 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"}]=0 v26) v1 ==== 122+0+0 (secure 0 0 0) 0x7fd9140170c0 con 0x7fd9240b1450 2023-08-09T13:48:21.200 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:21.197+0000 7fd92a61a640 1 -- 172.21.15.43:0/4279439694 >> [v2:172.21.15.203:6832/272917756,v1:172.21.15.203:6833/272917756] conn(0x7fd90806a170 msgr2=0x7fd90806c630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:48:21.200 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:21.197+0000 7fd92a61a640 1 --2- 172.21.15.43:0/4279439694 >> [v2:172.21.15.203:6832/272917756,v1:172.21.15.203:6833/272917756] conn(0x7fd90806a170 0x7fd90806c630 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7fd920000f60 tx=0x7fd920009040 comp rx=0 tx=0).stop 2023-08-09T13:48:21.200 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:21.197+0000 7fd92a61a640 1 -- 172.21.15.43:0/4279439694 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fd9240b1450 msgr2=0x7fd9241537e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:48:21.200 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:21.197+0000 7fd92a61a640 1 --2- 172.21.15.43:0/4279439694 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fd9240b1450 0x7fd9241537e0 secure :-1 s=READY pgs=848 cs=0 l=1 rev1=1 crypto rx=0x7fd91400e8a0 tx=0x7fd91400ed60 comp rx=0 tx=0).stop 2023-08-09T13:48:21.200 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:21.197+0000 7fd92a61a640 1 -- 172.21.15.43:0/4279439694 shutdown_connections 2023-08-09T13:48:21.200 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:21.197+0000 7fd92a61a640 1 --2- 172.21.15.43:0/4279439694 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fd92415c3a0 0x7fd92415db70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:48:21.201 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:21.197+0000 7fd92a61a640 1 --2- 172.21.15.43:0/4279439694 >> [v2:172.21.15.203:6832/272917756,v1:172.21.15.203:6833/272917756] conn(0x7fd90806a170 0x7fd90806c630 unknown :-1 s=CLOSED pgs=62 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:48:21.201 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:21.197+0000 7fd92a61a640 1 --2- 172.21.15.43:0/4279439694 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fd9240b1450 0x7fd9241537e0 unknown :-1 s=CLOSED pgs=848 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:48:21.201 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:21.197+0000 7fd92a61a640 1 --2- 172.21.15.43:0/4279439694 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fd9240b0f00 0x7fd9240c68f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:48:21.201 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:21.197+0000 7fd92a61a640 1 -- 172.21.15.43:0/4279439694 >> 172.21.15.43:0/4279439694 conn(0x7fd92406a210 msgr2=0x7fd924160880 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-09T13:48:21.201 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:21.198+0000 7fd92a61a640 1 -- 172.21.15.43:0/4279439694 shutdown_connections 2023-08-09T13:48:21.202 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:21.198+0000 7fd92a61a640 1 -- 172.21.15.43:0/4279439694 wait complete. 2023-08-09T13:48:21.220 DEBUG:tasks.cephfs.filesystem:Not Creating any subvols for filesystem 'cephfs' 2023-08-09T13:48:21.221 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-08-09T13:48:21.443 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:21.441+0000 7ff163a21640 1 Processor -- start 2023-08-09T13:48:21.444 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:21.441+0000 7ff163a21640 1 -- start start 2023-08-09T13:48:21.445 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:21.443+0000 7ff163a21640 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7ff15c15c630 0x7ff15c0abef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:48:21.445 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:21.443+0000 7ff163a21640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7ff15c0cdc60 0x7ff15c15ca10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:48:21.446 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:21.443+0000 7ff163a21640 1 -- --> v1:172.21.15.43:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff15c0cca90 con 0x7ff15c0ccf20 2023-08-09T13:48:21.446 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:21.443+0000 7ff163a21640 1 -- --> v2:172.21.15.43:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff15c0cbc70 con 0x7ff15c0cdc60 2023-08-09T13:48:21.446 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:21.443+0000 7ff163a21640 1 -- --> v2:172.21.15.203:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff15c0cbe80 con 0x7ff15c15c630 2023-08-09T13:48:21.447 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:21.443+0000 7ff160fbc640 1 --1- >> v1:172.21.15.43:6789/0 conn(0x7ff15c0ccf20 0x7ff15c0cd550 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.43:6789/0 says I am v1:172.21.15.43:46496/0 (socket says 172.21.15.43:46496) 2023-08-09T13:48:21.447 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:21.443+0000 7ff160fbc640 1 -- 172.21.15.43:0/1710714516 learned_addr learned my addr 172.21.15.43:0/1710714516 (peer_addr_for_me v1:172.21.15.43:0/0) 2023-08-09T13:48:21.447 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:21.443+0000 7ff161fbe640 1 --2- 172.21.15.43:0/1710714516 >> v2:172.21.15.43:3300/0 conn(0x7ff15c0cdc60 0x7ff15c15ca10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:48:21.447 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:21.444+0000 7ff1617bd640 1 --2- 172.21.15.43:0/1710714516 >> v2:172.21.15.203:3300/0 conn(0x7ff15c15c630 0x7ff15c0abef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:48:21.447 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:21.444+0000 7ff153fff640 1 -- 172.21.15.43:0/1710714516 <== mon.0 v1:172.21.15.43:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 853837237 0 0) 0x7ff15c0cca90 con 0x7ff15c0ccf20 2023-08-09T13:48:21.448 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:21.444+0000 7ff153fff640 1 -- 172.21.15.43:0/1710714516 --> v1:172.21.15.43:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ff148003520 con 0x7ff15c0ccf20 2023-08-09T13:48:21.448 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:21.444+0000 7ff161fbe640 1 -- 172.21.15.43:0/1710714516 >> v2:172.21.15.203:3300/0 conn(0x7ff15c15c630 msgr2=0x7ff15c0abef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-09T13:48:21.448 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:21.444+0000 7ff161fbe640 1 --2- 172.21.15.43:0/1710714516 >> v2:172.21.15.203:3300/0 conn(0x7ff15c15c630 0x7ff15c0abef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:48:21.448 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:21.444+0000 7ff161fbe640 1 -- 172.21.15.43:0/1710714516 >> v1:172.21.15.43:6789/0 conn(0x7ff15c0ccf20 legacy=0x7ff15c0cd550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:48:21.448 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:21.444+0000 7ff153fff640 1 -- 172.21.15.43:0/1710714516 <== mon.0 v1:172.21.15.43:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 124153966 0 0) 0x7ff148003520 con 0x7ff15c0ccf20 2023-08-09T13:48:21.448 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:21.444+0000 7ff161fbe640 1 -- 172.21.15.43:0/1710714516 --> v2:172.21.15.43:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff15c0cc160 con 0x7ff15c0cdc60 2023-08-09T13:48:21.449 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:21.445+0000 7ff161fbe640 1 --2- 172.21.15.43:0/1710714516 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff15c0cdc60 0x7ff15c15ca10 secure :-1 s=READY pgs=850 cs=0 l=1 rev1=1 crypto rx=0x7ff15c0cbbb0 tx=0x7ff15402bd90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=24fba9c2f89e849a server_cookie=0 in_seq=0 out_seq=0 2023-08-09T13:48:21.450 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:21.446+0000 7ff153fff640 1 -- 172.21.15.43:0/1710714516 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff14c001670 con 0x7ff15c0cdc60 2023-08-09T13:48:21.450 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:21.446+0000 7ff153fff640 1 -- 172.21.15.43:0/1710714516 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff15402c8c0 con 0x7ff15c0cdc60 2023-08-09T13:48:21.450 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:21.446+0000 7ff153fff640 1 -- 172.21.15.43:0/1710714516 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff15402cc30 con 0x7ff15c0cdc60 2023-08-09T13:48:21.450 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:21.446+0000 7ff163a21640 1 -- 172.21.15.43:0/1710714516 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff15c0cdc60 msgr2=0x7ff15c15ca10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:48:21.450 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:21.446+0000 7ff163a21640 1 --2- 172.21.15.43:0/1710714516 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff15c0cdc60 0x7ff15c15ca10 secure :-1 s=READY pgs=850 cs=0 l=1 rev1=1 crypto rx=0x7ff15c0cbbb0 tx=0x7ff15402bd90 comp rx=0 tx=0).stop 2023-08-09T13:48:21.450 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:21.447+0000 7ff163a21640 1 -- 172.21.15.43:0/1710714516 shutdown_connections 2023-08-09T13:48:21.450 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:21.447+0000 7ff163a21640 1 --2- 172.21.15.43:0/1710714516 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff15c0cdc60 0x7ff15c15ca10 unknown :-1 s=CLOSED pgs=850 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:48:21.451 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:21.447+0000 7ff163a21640 1 --2- 172.21.15.43:0/1710714516 >> v2:172.21.15.203:3300/0 conn(0x7ff15c15c630 0x7ff15c0abef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:48:21.451 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:21.447+0000 7ff163a21640 1 -- 172.21.15.43:0/1710714516 >> 172.21.15.43:0/1710714516 conn(0x7ff15c0ce0a0 msgr2=0x7ff15c0ce480 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-09T13:48:21.451 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:21.447+0000 7ff163a21640 1 -- 172.21.15.43:0/1710714516 shutdown_connections 2023-08-09T13:48:21.451 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:21.447+0000 7ff163a21640 1 -- 172.21.15.43:0/1710714516 wait complete. 2023-08-09T13:48:21.451 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:21.448+0000 7ff163a21640 1 Processor -- start 2023-08-09T13:48:21.452 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:21.449+0000 7ff163a21640 1 -- start start 2023-08-09T13:48:21.454 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:21.449+0000 7ff163a21640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7ff15c0ccf20 0x7ff15c159a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:48:21.454 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:21.450+0000 7ff163a21640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff15c0cdc60 0x7ff15c0aedc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:48:21.454 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:21.450+0000 7ff163a21640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7ff15c15c630 0x7ff15c0c5170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:48:21.454 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:21.450+0000 7ff163a21640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff15c159710 con 0x7ff15c0cdc60 2023-08-09T13:48:21.454 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:21.450+0000 7ff163a21640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff15c159f80 con 0x7ff15c0ccf20 2023-08-09T13:48:21.455 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:21.450+0000 7ff163a21640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff15c158d50 con 0x7ff15c15c630 2023-08-09T13:48:21.455 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:21.450+0000 7ff160fbc640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff15c0cdc60 0x7ff15c0aedc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:48:21.455 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:21.450+0000 7ff160fbc640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff15c0cdc60 0x7ff15c0aedc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:36806/0 (socket says 172.21.15.43:36806) 2023-08-09T13:48:21.455 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:21.450+0000 7ff160fbc640 1 -- 172.21.15.43:0/1148237359 learned_addr learned my addr 172.21.15.43:0/1148237359 (peer_addr_for_me v2:172.21.15.43:0/0) 2023-08-09T13:48:21.455 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:21.450+0000 7ff1617bd640 1 --2- 172.21.15.43:0/1148237359 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7ff15c0ccf20 0x7ff15c159a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:48:21.455 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:21.450+0000 7ff160fbc640 1 -- 172.21.15.43:0/1148237359 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7ff15c15c630 msgr2=0x7ff15c0c5170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:48:21.455 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:21.450+0000 7ff160fbc640 1 --2- 172.21.15.43:0/1148237359 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7ff15c15c630 0x7ff15c0c5170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:48:21.456 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:21.450+0000 7ff160fbc640 1 -- 172.21.15.43:0/1148237359 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7ff15c0ccf20 msgr2=0x7ff15c159a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:48:21.456 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:21.450+0000 7ff160fbc640 1 --2- 172.21.15.43:0/1148237359 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7ff15c0ccf20 0x7ff15c159a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:48:21.456 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:21.450+0000 7ff160fbc640 1 -- 172.21.15.43:0/1148237359 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff154009a70 con 0x7ff15c0cdc60 2023-08-09T13:48:21.456 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:21.451+0000 7ff160fbc640 1 --2- 172.21.15.43:0/1148237359 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff15c0cdc60 0x7ff15c0aedc0 secure :-1 s=READY pgs=851 cs=0 l=1 rev1=1 crypto rx=0x7ff14c00c710 tx=0x7ff14c00cbd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-09T13:48:21.456 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:21.451+0000 7ff151ffb640 1 -- 172.21.15.43:0/1148237359 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff14c005ad0 con 0x7ff15c0cdc60 2023-08-09T13:48:21.456 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:21.451+0000 7ff151ffb640 1 -- 172.21.15.43:0/1148237359 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff14c005c30 con 0x7ff15c0cdc60 2023-08-09T13:48:21.456 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:21.451+0000 7ff151ffb640 1 -- 172.21.15.43:0/1148237359 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff14c007070 con 0x7ff15c0cdc60 2023-08-09T13:48:21.457 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:21.451+0000 7ff163a21640 1 -- 172.21.15.43:0/1148237359 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff15c1581d0 con 0x7ff15c0cdc60 2023-08-09T13:48:21.457 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:21.451+0000 7ff163a21640 1 -- 172.21.15.43:0/1148237359 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff15c158790 con 0x7ff15c0cdc60 2023-08-09T13:48:21.458 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:21.452+0000 7ff163a21640 1 -- 172.21.15.43:0/1148237359 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff15c0739f0 con 0x7ff15c0cdc60 2023-08-09T13:48:21.458 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:21.453+0000 7ff151ffb640 1 -- 172.21.15.43:0/1148237359 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 18) v1 ==== 88443+0+0 (secure 0 0 0) 0x7ff14c0071f0 con 0x7ff15c0cdc60 2023-08-09T13:48:21.458 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:21.454+0000 7ff151ffb640 1 --2- 172.21.15.43:0/1148237359 >> [v2:172.21.15.203:6832/272917756,v1:172.21.15.203:6833/272917756] conn(0x7ff14006a170 0x7ff14006c630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:48:21.458 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:21.454+0000 7ff151ffb640 1 -- 172.21.15.43:0/1148237359 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(74..74 src has 1..74) v4 ==== 6426+0+0 (secure 0 0 0) 0x7ff14c088760 con 0x7ff15c0cdc60 2023-08-09T13:48:21.458 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:21.455+0000 7ff1617bd640 1 --2- 172.21.15.43:0/1148237359 >> [v2:172.21.15.203:6832/272917756,v1:172.21.15.203:6833/272917756] conn(0x7ff14006a170 0x7ff14006c630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:48:21.459 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:21.456+0000 7ff1617bd640 1 --2- 172.21.15.43:0/1148237359 >> [v2:172.21.15.203:6832/272917756,v1:172.21.15.203:6833/272917756] conn(0x7ff14006a170 0x7ff14006c630 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7ff158009a30 tx=0x7ff158008040 comp rx=0 tx=0).ready entity=mgr.5533 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-09T13:48:21.461 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:21.459+0000 7ff151ffb640 1 -- 172.21.15.43:0/1148237359 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178066 (secure 0 0 0) 0x7ff14c053b30 con 0x7ff15c0cdc60 2023-08-09T13:48:21.694 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:21.691+0000 7ff163a21640 1 -- 172.21.15.43:0/1148237359 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7ff15c0b4050 con 0x7ff15c0cdc60 2023-08-09T13:48:21.695 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:21.693+0000 7ff151ffb640 1 -- 172.21.15.43:0/1148237359 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 26 v26) v1 ==== 94+0+4881 (secure 0 0 0) 0x7ff14c057150 con 0x7ff15c0cdc60 2023-08-09T13:48:21.696 INFO:teuthology.orchestra.run.smithi043.stdout: 2023-08-09T13:48:21.696 INFO:teuthology.orchestra.run.smithi043.stdout:{"epoch":26,"default_fscid":3,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5369,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.203:6837/1426673901","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6836","nonce":1426673901},{"type":"v1","addr":"172.21.15.203:6837","nonce":1426673901}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":21},{"gid":5728,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.43:6833/1096124235","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6832","nonce":1096124235},{"type":"v1","addr":"172.21.15.43:6833","nonce":1096124235}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":20},{"gid":5731,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.43:6835/1688804357","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6834","nonce":1688804357},{"type":"v1","addr":"172.21.15.43:6835","nonce":1688804357}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":21}],"filesystems":[{"mdsmap":{"epoch":26,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-08-09T13:48:17.607195+0000","modified":"2023-08-09T13:48:21.179296+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":5734},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5734":{"gid":5734,"name":"b","rank":0,"incarnation":22,"state":"up:active","state_seq":3,"addr":"172.21.15.203:6835/3324252767","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6834","nonce":3324252767},{"type":"v1","addr":"172.21.15.203:6835","nonce":3324252767}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[9],"metadata_pool":8,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":3}]} 2023-08-09T13:48:21.696 INFO:teuthology.orchestra.run.smithi043.stderr:dumped fsmap epoch 26 2023-08-09T13:48:21.698 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:21.695+0000 7ff163a21640 1 -- 172.21.15.43:0/1148237359 >> [v2:172.21.15.203:6832/272917756,v1:172.21.15.203:6833/272917756] conn(0x7ff14006a170 msgr2=0x7ff14006c630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:48:21.698 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:21.695+0000 7ff163a21640 1 --2- 172.21.15.43:0/1148237359 >> [v2:172.21.15.203:6832/272917756,v1:172.21.15.203:6833/272917756] conn(0x7ff14006a170 0x7ff14006c630 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7ff158009a30 tx=0x7ff158008040 comp rx=0 tx=0).stop 2023-08-09T13:48:21.698 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:21.695+0000 7ff163a21640 1 -- 172.21.15.43:0/1148237359 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff15c0cdc60 msgr2=0x7ff15c0aedc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:48:21.698 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:21.695+0000 7ff163a21640 1 --2- 172.21.15.43:0/1148237359 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff15c0cdc60 0x7ff15c0aedc0 secure :-1 s=READY pgs=851 cs=0 l=1 rev1=1 crypto rx=0x7ff14c00c710 tx=0x7ff14c00cbd0 comp rx=0 tx=0).stop 2023-08-09T13:48:21.698 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:21.695+0000 7ff163a21640 1 -- 172.21.15.43:0/1148237359 shutdown_connections 2023-08-09T13:48:21.698 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:21.695+0000 7ff163a21640 1 --2- 172.21.15.43:0/1148237359 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7ff15c15c630 0x7ff15c0c5170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:48:21.698 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:21.695+0000 7ff163a21640 1 --2- 172.21.15.43:0/1148237359 >> [v2:172.21.15.203:6832/272917756,v1:172.21.15.203:6833/272917756] conn(0x7ff14006a170 0x7ff14006c630 unknown :-1 s=CLOSED pgs=63 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:48:21.699 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:21.695+0000 7ff163a21640 1 --2- 172.21.15.43:0/1148237359 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff15c0cdc60 0x7ff15c0aedc0 unknown :-1 s=CLOSED pgs=851 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:48:21.699 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:21.695+0000 7ff163a21640 1 --2- 172.21.15.43:0/1148237359 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7ff15c0ccf20 0x7ff15c159a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:48:21.699 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:21.695+0000 7ff163a21640 1 -- 172.21.15.43:0/1148237359 >> 172.21.15.43:0/1148237359 conn(0x7ff15c0ce0a0 msgr2=0x7ff15c06b790 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-09T13:48:21.699 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:21.695+0000 7ff163a21640 1 -- 172.21.15.43:0/1148237359 shutdown_connections 2023-08-09T13:48:21.700 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:21.695+0000 7ff163a21640 1 -- 172.21.15.43:0/1148237359 wait complete. 2023-08-09T13:48:21.717 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-08-09T13:48:21.945 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:21.942+0000 7fa23a4ec640 1 Processor -- start 2023-08-09T13:48:21.945 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:21.942+0000 7fa23a4ec640 1 -- start start 2023-08-09T13:48:21.946 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:21.944+0000 7fa23a4ec640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fa234142ca0 0x7fa2340c96e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:48:21.947 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:21.944+0000 7fa23a4ec640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7fa23415ce40 0x7fa2340cce00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:48:21.947 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:21.944+0000 7fa23a4ec640 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7fa2340cd340 0x7fa2340cbc70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:48:21.947 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:21.944+0000 7fa23a4ec640 1 -- --> v2:172.21.15.43:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa2340cd7b0 con 0x7fa23415ce40 2023-08-09T13:48:21.947 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:21.944+0000 7fa23a4ec640 1 -- --> v2:172.21.15.203:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa234143080 con 0x7fa2340cd340 2023-08-09T13:48:21.947 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:21.944+0000 7fa23a4ec640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa23406d400 con 0x7fa234142ca0 2023-08-09T13:48:21.948 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:21.945+0000 7fa22bfff640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7fa23415ce40 0x7fa2340cce00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:48:21.949 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:21.945+0000 7fa22bfff640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7fa23415ce40 0x7fa2340cce00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:36814/0 (socket says 172.21.15.43:36814) 2023-08-09T13:48:21.949 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:21.945+0000 7fa22bfff640 1 -- 172.21.15.43:0/1694027407 learned_addr learned my addr 172.21.15.43:0/1694027407 (peer_addr_for_me v2:172.21.15.43:0/0) 2023-08-09T13:48:21.949 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:21.945+0000 7fa233fff640 1 --2- 172.21.15.43:0/1694027407 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fa234142ca0 0x7fa2340c96e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:48:21.949 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:21.945+0000 7fa238a89640 1 --2- 172.21.15.43:0/1694027407 >> v2:172.21.15.203:3300/0 conn(0x7fa2340cd340 0x7fa2340cbc70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:48:21.949 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:21.945+0000 7fa22bfff640 1 -- 172.21.15.43:0/1694027407 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fa234142ca0 msgr2=0x7fa2340c96e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-09T13:48:21.949 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:21.945+0000 7fa22bfff640 1 --2- 172.21.15.43:0/1694027407 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fa234142ca0 0x7fa2340c96e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:48:21.949 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:21.945+0000 7fa22bfff640 1 -- 172.21.15.43:0/1694027407 >> v2:172.21.15.203:3300/0 conn(0x7fa2340cd340 msgr2=0x7fa2340cbc70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-09T13:48:21.950 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:21.945+0000 7fa22bfff640 1 --2- 172.21.15.43:0/1694027407 >> v2:172.21.15.203:3300/0 conn(0x7fa2340cd340 0x7fa2340cbc70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:48:21.950 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:21.945+0000 7fa22bfff640 1 -- 172.21.15.43:0/1694027407 --> v2:172.21.15.43:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa2340c9d90 con 0x7fa23415ce40 2023-08-09T13:48:21.951 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:21.946+0000 7fa22bfff640 1 --2- 172.21.15.43:0/1694027407 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa23415ce40 0x7fa2340cce00 secure :-1 s=READY pgs=852 cs=0 l=1 rev1=1 crypto rx=0x7fa2340ccd10 tx=0x7fa21c02db20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b21dab98ab656779 server_cookie=0 in_seq=0 out_seq=0 2023-08-09T13:48:21.951 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:21.947+0000 7fa2337fe640 1 -- 172.21.15.43:0/1694027407 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa21c02f8e0 con 0x7fa23415ce40 2023-08-09T13:48:21.951 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:21.947+0000 7fa2337fe640 1 -- 172.21.15.43:0/1694027407 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa21c02fa40 con 0x7fa23415ce40 2023-08-09T13:48:21.951 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:21.947+0000 7fa2337fe640 1 -- 172.21.15.43:0/1694027407 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa21c02fdd0 con 0x7fa23415ce40 2023-08-09T13:48:21.951 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:21.948+0000 7fa23a4ec640 1 -- 172.21.15.43:0/1694027407 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa23415ce40 msgr2=0x7fa2340cce00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:48:21.951 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:21.948+0000 7fa23a4ec640 1 --2- 172.21.15.43:0/1694027407 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa23415ce40 0x7fa2340cce00 secure :-1 s=READY pgs=852 cs=0 l=1 rev1=1 crypto rx=0x7fa2340ccd10 tx=0x7fa21c02db20 comp rx=0 tx=0).stop 2023-08-09T13:48:21.951 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:21.948+0000 7fa23a4ec640 1 -- 172.21.15.43:0/1694027407 shutdown_connections 2023-08-09T13:48:21.952 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:21.948+0000 7fa23a4ec640 1 --2- 172.21.15.43:0/1694027407 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa23415ce40 0x7fa2340cce00 unknown :-1 s=CLOSED pgs=852 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:48:21.952 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:21.948+0000 7fa23a4ec640 1 --2- 172.21.15.43:0/1694027407 >> v2:172.21.15.203:3300/0 conn(0x7fa2340cd340 0x7fa2340cbc70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:48:21.952 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:21.948+0000 7fa23a4ec640 1 --2- 172.21.15.43:0/1694027407 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fa234142ca0 0x7fa2340c96e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:48:21.952 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:21.948+0000 7fa23a4ec640 1 -- 172.21.15.43:0/1694027407 >> 172.21.15.43:0/1694027407 conn(0x7fa2340ce0a0 msgr2=0x7fa2340ce480 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-09T13:48:21.952 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:21.948+0000 7fa23a4ec640 1 -- 172.21.15.43:0/1694027407 shutdown_connections 2023-08-09T13:48:21.952 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:21.948+0000 7fa23a4ec640 1 -- 172.21.15.43:0/1694027407 wait complete. 2023-08-09T13:48:21.953 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:21.950+0000 7fa23a4ec640 1 Processor -- start 2023-08-09T13:48:21.953 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:21.950+0000 7fa23a4ec640 1 -- start start 2023-08-09T13:48:21.954 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:21.951+0000 7fa23a4ec640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fa2340cd340 0x7fa23415aa00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:48:21.955 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:21.951+0000 7fa23a4ec640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fa234142ca0 0x7fa23415be90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:48:21.955 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:21.951+0000 7fa23a4ec640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa23415ce40 0x7fa23415c3d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:48:21.955 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:21.951+0000 7fa23a4ec640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa2340c6590 con 0x7fa23415ce40 2023-08-09T13:48:21.955 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:21.951+0000 7fa23a4ec640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa23415a570 con 0x7fa234142ca0 2023-08-09T13:48:21.955 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:21.951+0000 7fa23a4ec640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa2341599a0 con 0x7fa2340cd340 2023-08-09T13:48:21.955 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:21.951+0000 7fa238a89640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa23415ce40 0x7fa23415c3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:48:21.956 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:21.951+0000 7fa238a89640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa23415ce40 0x7fa23415c3d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:36828/0 (socket says 172.21.15.43:36828) 2023-08-09T13:48:21.956 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:21.951+0000 7fa238a89640 1 -- 172.21.15.43:0/2351445588 learned_addr learned my addr 172.21.15.43:0/2351445588 (peer_addr_for_me v2:172.21.15.43:0/0) 2023-08-09T13:48:21.956 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:21.952+0000 7fa233fff640 1 --2- 172.21.15.43:0/2351445588 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fa2340cd340 0x7fa23415aa00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:48:21.956 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:21.952+0000 7fa22bfff640 1 --2- 172.21.15.43:0/2351445588 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fa234142ca0 0x7fa23415be90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:48:21.956 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:21.952+0000 7fa238a89640 1 -- 172.21.15.43:0/2351445588 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fa2340cd340 msgr2=0x7fa23415aa00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:48:21.956 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:21.952+0000 7fa238a89640 1 --2- 172.21.15.43:0/2351445588 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fa2340cd340 0x7fa23415aa00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:48:21.957 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:21.952+0000 7fa238a89640 1 -- 172.21.15.43:0/2351445588 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fa234142ca0 msgr2=0x7fa23415be90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:48:21.957 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:21.952+0000 7fa238a89640 1 --2- 172.21.15.43:0/2351445588 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fa234142ca0 0x7fa23415be90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:48:21.957 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:21.952+0000 7fa238a89640 1 -- 172.21.15.43:0/2351445588 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa21c0094d0 con 0x7fa23415ce40 2023-08-09T13:48:21.957 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:21.952+0000 7fa238a89640 1 --2- 172.21.15.43:0/2351445588 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa23415ce40 0x7fa23415c3d0 secure :-1 s=READY pgs=853 cs=0 l=1 rev1=1 crypto rx=0x7fa22c007d40 tx=0x7fa22c00e4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-09T13:48:21.958 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:21.953+0000 7fa2317fa640 1 -- 172.21.15.43:0/2351445588 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa22c014420 con 0x7fa23415ce40 2023-08-09T13:48:21.958 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:21.953+0000 7fa2317fa640 1 -- 172.21.15.43:0/2351445588 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa22c0145a0 con 0x7fa23415ce40 2023-08-09T13:48:21.958 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:21.953+0000 7fa2317fa640 1 -- 172.21.15.43:0/2351445588 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa22c014950 con 0x7fa23415ce40 2023-08-09T13:48:21.959 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:21.953+0000 7fa23a4ec640 1 -- 172.21.15.43:0/2351445588 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa234159c40 con 0x7fa23415ce40 2023-08-09T13:48:21.959 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:21.953+0000 7fa23a4ec640 1 -- 172.21.15.43:0/2351445588 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa23415a200 con 0x7fa23415ce40 2023-08-09T13:48:21.959 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:21.954+0000 7fa23a4ec640 1 -- 172.21.15.43:0/2351445588 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa23415a6e0 con 0x7fa23415ce40 2023-08-09T13:48:21.959 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:21.955+0000 7fa2317fa640 1 -- 172.21.15.43:0/2351445588 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 18) v1 ==== 88443+0+0 (secure 0 0 0) 0x7fa22c014ab0 con 0x7fa23415ce40 2023-08-09T13:48:21.959 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:21.956+0000 7fa2317fa640 1 --2- 172.21.15.43:0/2351445588 >> [v2:172.21.15.203:6832/272917756,v1:172.21.15.203:6833/272917756] conn(0x7fa21406a170 0x7fa21406c630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:48:21.960 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:21.956+0000 7fa2317fa640 1 -- 172.21.15.43:0/2351445588 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(74..74 src has 1..74) v4 ==== 6426+0+0 (secure 0 0 0) 0x7fa22c089150 con 0x7fa23415ce40 2023-08-09T13:48:21.960 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:21.957+0000 7fa233fff640 1 --2- 172.21.15.43:0/2351445588 >> [v2:172.21.15.203:6832/272917756,v1:172.21.15.203:6833/272917756] conn(0x7fa21406a170 0x7fa21406c630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:48:21.960 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:21.958+0000 7fa233fff640 1 --2- 172.21.15.43:0/2351445588 >> [v2:172.21.15.203:6832/272917756,v1:172.21.15.203:6833/272917756] conn(0x7fa21406a170 0x7fa21406c630 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7fa224006fd0 tx=0x7fa224008040 comp rx=0 tx=0).ready entity=mgr.5533 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-09T13:48:21.962 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:21.960+0000 7fa2317fa640 1 -- 172.21.15.43:0/2351445588 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178066 (secure 0 0 0) 0x7fa22c054520 con 0x7fa23415ce40 2023-08-09T13:48:22.162 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:22.160+0000 7fa23a4ec640 1 -- 172.21.15.43:0/2351445588 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fa2340739f0 con 0x7fa23415ce40 2023-08-09T13:48:22.165 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:22.163+0000 7fa2317fa640 1 -- 172.21.15.43:0/2351445588 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v74) v1 ==== 74+0+18552 (secure 0 0 0) 0x7fa22c057b40 con 0x7fa23415ce40 2023-08-09T13:48:22.166 INFO:teuthology.orchestra.run.smithi043.stdout: 2023-08-09T13:48:22.166 INFO:teuthology.orchestra.run.smithi043.stdout:{"epoch":74,"fsid":"daac87ee-1337-4250-a955-aea4e408a39b","created":"2023-08-09T13:33:41.701198+0000","modified":"2023-08-09T13:48:19.106167+0000","last_up_change":"2023-08-09T13:33:53.783453+0000","last_in_change":"2023-08-09T13:33:45.148904+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":9,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":"rbd","create_time":"2023-08-09T13:33:54.365137+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":".mgr","create_time":"2023-08-09T13:33:54.831803+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":8,"pool_name":"cephfs_metadata","create_time":"2023-08-09T13:48:16.039756+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"73","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":9,"pool_name":"cephfs_data","create_time":"2023-08-09T13:48:16.615400+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"74","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.6299999952316284,"score_stable":1.6299999952316284,"optimal_score":1,"raw_score_acting":1.6299999952316284,"raw_score_stable":1.6299999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"cbd49ecd-e896-4fbd-864e-5c37a829b165","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":72,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6800","nonce":502667335},{"type":"v1","addr":"172.21.15.43:6801","nonce":502667335}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6802","nonce":502667335},{"type":"v1","addr":"172.21.15.43:6803","nonce":502667335}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6806","nonce":502667335},{"type":"v1","addr":"172.21.15.43:6807","nonce":502667335}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6804","nonce":502667335},{"type":"v1","addr":"172.21.15.43:6805","nonce":502667335}]},"public_addr":"172.21.15.43:6801/502667335","cluster_addr":"172.21.15.43:6803/502667335","heartbeat_back_addr":"172.21.15.43:6807/502667335","heartbeat_front_addr":"172.21.15.43:6805/502667335","state":["exists","up"]},{"osd":1,"uuid":"75c7e0fe-1ec7-4fe8-a2f2-3661a65b8500","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":72,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6808","nonce":3289134610},{"type":"v1","addr":"172.21.15.43:6809","nonce":3289134610}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6810","nonce":3289134610},{"type":"v1","addr":"172.21.15.43:6812","nonce":3289134610}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6821","nonce":3289134610},{"type":"v1","addr":"172.21.15.43:6824","nonce":3289134610}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6815","nonce":3289134610},{"type":"v1","addr":"172.21.15.43:6818","nonce":3289134610}]},"public_addr":"172.21.15.43:6809/3289134610","cluster_addr":"172.21.15.43:6812/3289134610","heartbeat_back_addr":"172.21.15.43:6824/3289134610","heartbeat_front_addr":"172.21.15.43:6818/3289134610","state":["exists","up"]},{"osd":2,"uuid":"bb5c41e9-7f5f-4125-9b01-3bb8bfc2ea83","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":72,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6811","nonce":4079709736},{"type":"v1","addr":"172.21.15.43:6814","nonce":4079709736}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6817","nonce":4079709736},{"type":"v1","addr":"172.21.15.43:6820","nonce":4079709736}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6829","nonce":4079709736},{"type":"v1","addr":"172.21.15.43:6831","nonce":4079709736}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6823","nonce":4079709736},{"type":"v1","addr":"172.21.15.43:6826","nonce":4079709736}]},"public_addr":"172.21.15.43:6814/4079709736","cluster_addr":"172.21.15.43:6820/4079709736","heartbeat_back_addr":"172.21.15.43:6831/4079709736","heartbeat_front_addr":"172.21.15.43:6826/4079709736","state":["exists","up"]},{"osd":3,"uuid":"bfe4c8b4-694a-465c-87e6-2b2aab852613","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":72,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6813","nonce":1709538759},{"type":"v1","addr":"172.21.15.43:6816","nonce":1709538759}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6819","nonce":1709538759},{"type":"v1","addr":"172.21.15.43:6822","nonce":1709538759}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6828","nonce":1709538759},{"type":"v1","addr":"172.21.15.43:6830","nonce":1709538759}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6825","nonce":1709538759},{"type":"v1","addr":"172.21.15.43:6827","nonce":1709538759}]},"public_addr":"172.21.15.43:6816/1709538759","cluster_addr":"172.21.15.43:6822/1709538759","heartbeat_back_addr":"172.21.15.43:6830/1709538759","heartbeat_front_addr":"172.21.15.43:6827/1709538759","state":["exists","up"]},{"osd":4,"uuid":"fa2e8a77-aba1-4756-853e-0d174a5bdeba","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":72,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6800","nonce":4196866481},{"type":"v1","addr":"172.21.15.203:6801","nonce":4196866481}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6802","nonce":4196866481},{"type":"v1","addr":"172.21.15.203:6803","nonce":4196866481}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6806","nonce":4196866481},{"type":"v1","addr":"172.21.15.203:6807","nonce":4196866481}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6804","nonce":4196866481},{"type":"v1","addr":"172.21.15.203:6805","nonce":4196866481}]},"public_addr":"172.21.15.203:6801/4196866481","cluster_addr":"172.21.15.203:6803/4196866481","heartbeat_back_addr":"172.21.15.203:6807/4196866481","heartbeat_front_addr":"172.21.15.203:6805/4196866481","state":["exists","up"]},{"osd":5,"uuid":"81811e4b-4d24-4f51-a82c-f1d700a2a00d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":72,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6816","nonce":3059290081},{"type":"v1","addr":"172.21.15.203:6817","nonce":3059290081}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6818","nonce":3059290081},{"type":"v1","addr":"172.21.15.203:6819","nonce":3059290081}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6822","nonce":3059290081},{"type":"v1","addr":"172.21.15.203:6823","nonce":3059290081}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6820","nonce":3059290081},{"type":"v1","addr":"172.21.15.203:6821","nonce":3059290081}]},"public_addr":"172.21.15.203:6817/3059290081","cluster_addr":"172.21.15.203:6819/3059290081","heartbeat_back_addr":"172.21.15.203:6823/3059290081","heartbeat_front_addr":"172.21.15.203:6821/3059290081","state":["exists","up"]},{"osd":6,"uuid":"307bac4d-6a5b-443d-af75-56d695d7674f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":72,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6824","nonce":512155569},{"type":"v1","addr":"172.21.15.203:6825","nonce":512155569}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6826","nonce":512155569},{"type":"v1","addr":"172.21.15.203:6827","nonce":512155569}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6830","nonce":512155569},{"type":"v1","addr":"172.21.15.203:6831","nonce":512155569}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6828","nonce":512155569},{"type":"v1","addr":"172.21.15.203:6829","nonce":512155569}]},"public_addr":"172.21.15.203:6825/512155569","cluster_addr":"172.21.15.203:6827/512155569","heartbeat_back_addr":"172.21.15.203:6831/512155569","heartbeat_front_addr":"172.21.15.203:6829/512155569","state":["exists","up"]},{"osd":7,"uuid":"a95dc7ca-9f1b-4295-af70-68a22a2d7d5e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":72,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6808","nonce":3331450438},{"type":"v1","addr":"172.21.15.203:6809","nonce":3331450438}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6810","nonce":3331450438},{"type":"v1","addr":"172.21.15.203:6811","nonce":3331450438}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6814","nonce":3331450438},{"type":"v1","addr":"172.21.15.203:6815","nonce":3331450438}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6812","nonce":3331450438},{"type":"v1","addr":"172.21.15.203:6813","nonce":3331450438}]},"public_addr":"172.21.15.203:6809/3331450438","cluster_addr":"172.21.15.203:6811/3331450438","heartbeat_back_addr":"172.21.15.203:6815/3331450438","heartbeat_front_addr":"172.21.15.203:6813/3331450438","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-09T13:33:50.547652+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-09T13:33:50.589469+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-09T13:33:50.561724+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-09T13:33:50.599650+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-09T13:33:50.411855+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-09T13:33:50.219681+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-09T13:33:50.504664+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-09T13:33:50.441248+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":7,"snaps":[{"begin":104,"length":1},{"begin":221,"length":1}]}],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-08-09T13:48:22.168 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:22.166+0000 7fa23a4ec640 1 -- 172.21.15.43:0/2351445588 >> [v2:172.21.15.203:6832/272917756,v1:172.21.15.203:6833/272917756] conn(0x7fa21406a170 msgr2=0x7fa21406c630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:48:22.169 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:22.166+0000 7fa23a4ec640 1 --2- 172.21.15.43:0/2351445588 >> [v2:172.21.15.203:6832/272917756,v1:172.21.15.203:6833/272917756] conn(0x7fa21406a170 0x7fa21406c630 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7fa224006fd0 tx=0x7fa224008040 comp rx=0 tx=0).stop 2023-08-09T13:48:22.169 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:22.166+0000 7fa23a4ec640 1 -- 172.21.15.43:0/2351445588 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa23415ce40 msgr2=0x7fa23415c3d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:48:22.169 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:22.166+0000 7fa23a4ec640 1 --2- 172.21.15.43:0/2351445588 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa23415ce40 0x7fa23415c3d0 secure :-1 s=READY pgs=853 cs=0 l=1 rev1=1 crypto rx=0x7fa22c007d40 tx=0x7fa22c00e4b0 comp rx=0 tx=0).stop 2023-08-09T13:48:22.169 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:22.166+0000 7fa23a4ec640 1 -- 172.21.15.43:0/2351445588 shutdown_connections 2023-08-09T13:48:22.169 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:22.166+0000 7fa23a4ec640 1 --2- 172.21.15.43:0/2351445588 >> [v2:172.21.15.203:6832/272917756,v1:172.21.15.203:6833/272917756] conn(0x7fa21406a170 0x7fa21406c630 unknown :-1 s=CLOSED pgs=64 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:48:22.169 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:22.166+0000 7fa23a4ec640 1 --2- 172.21.15.43:0/2351445588 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa23415ce40 0x7fa23415c3d0 unknown :-1 s=CLOSED pgs=853 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:48:22.170 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:22.166+0000 7fa23a4ec640 1 --2- 172.21.15.43:0/2351445588 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fa234142ca0 0x7fa23415be90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:48:22.170 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:22.166+0000 7fa23a4ec640 1 --2- 172.21.15.43:0/2351445588 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fa2340cd340 0x7fa23415aa00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:48:22.170 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:22.166+0000 7fa23a4ec640 1 -- 172.21.15.43:0/2351445588 >> 172.21.15.43:0/2351445588 conn(0x7fa2340ce0a0 msgr2=0x7fa23415d2e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-09T13:48:22.170 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:22.166+0000 7fa23a4ec640 1 -- 172.21.15.43:0/2351445588 shutdown_connections 2023-08-09T13:48:22.170 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:22.166+0000 7fa23a4ec640 1 -- 172.21.15.43:0/2351445588 wait complete. 2023-08-09T13:48:22.189 INFO:tasks.cephfs.filesystem.ceph_manager:waiting for clean 2023-08-09T13:48:22.189 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-08-09T13:48:22.419 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:22.416+0000 7fb1554c0640 1 Processor -- start 2023-08-09T13:48:22.419 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:22.416+0000 7fb1554c0640 1 -- start start 2023-08-09T13:48:22.420 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:22.418+0000 7fb1554c0640 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7fb1500b1050 0x7fb1500b4760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:48:22.420 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:22.418+0000 7fb1554c0640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7fb1500cd6b0 0x7fb1500d0e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:48:22.421 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:22.418+0000 7fb1554c0640 1 -- --> v1:172.21.15.203:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb1500b7bf0 con 0x7fb15015f7b0 2023-08-09T13:48:22.421 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:22.418+0000 7fb1554c0640 1 -- --> v2:172.21.15.43:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb1500b7ed0 con 0x7fb1500cd6b0 2023-08-09T13:48:22.421 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:22.418+0000 7fb1554c0640 1 -- --> v2:172.21.15.203:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb1500d0370 con 0x7fb1500b1050 2023-08-09T13:48:22.422 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:22.418+0000 7fb154cbf640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7fb1500cd6b0 0x7fb1500d0e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:48:22.423 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:22.418+0000 7fb154cbf640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7fb1500cd6b0 0x7fb1500d0e50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:36836/0 (socket says 172.21.15.43:36836) 2023-08-09T13:48:22.423 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:22.418+0000 7fb154cbf640 1 -- 172.21.15.43:0/2423950355 learned_addr learned my addr 172.21.15.43:0/2423950355 (peer_addr_for_me v2:172.21.15.43:0/0) 2023-08-09T13:48:22.423 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:22.419+0000 7fb14f7fe640 1 --2- 172.21.15.43:0/2423950355 >> v2:172.21.15.203:3300/0 conn(0x7fb1500b1050 0x7fb1500b4760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:48:22.423 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:22.419+0000 7fb154cbf640 1 -- 172.21.15.43:0/2423950355 >> v2:172.21.15.203:3300/0 conn(0x7fb1500b1050 msgr2=0x7fb1500b4760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-09T13:48:22.423 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:22.419+0000 7fb154cbf640 1 --2- 172.21.15.43:0/2423950355 >> v2:172.21.15.203:3300/0 conn(0x7fb1500b1050 0x7fb1500b4760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:48:22.423 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:22.419+0000 7fb154cbf640 1 -- 172.21.15.43:0/2423950355 >> v1:172.21.15.203:6789/0 conn(0x7fb15015f7b0 legacy=0x7fb1500ccfa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-09T13:48:22.423 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:22.419+0000 7fb154cbf640 1 -- 172.21.15.43:0/2423950355 --> v2:172.21.15.43:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb1500d0650 con 0x7fb1500cd6b0 2023-08-09T13:48:22.423 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:22.419+0000 7fb14f7fe640 1 --2- 172.21.15.43:0/2423950355 >> v2:172.21.15.203:3300/0 conn(0x7fb1500b1050 0x7fb1500b4760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-08-09T13:48:22.424 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:22.420+0000 7fb154cbf640 1 --2- 172.21.15.43:0/2423950355 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb1500cd6b0 0x7fb1500d0e50 secure :-1 s=READY pgs=854 cs=0 l=1 rev1=1 crypto rx=0x7fb1500cfaf0 tx=0x7fb14802bee0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=71116a3b9866e3ec server_cookie=0 in_seq=0 out_seq=0 2023-08-09T13:48:22.425 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:22.420+0000 7fb14effd640 1 -- 172.21.15.43:0/2423950355 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb14802ca80 con 0x7fb1500cd6b0 2023-08-09T13:48:22.425 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:22.421+0000 7fb14effd640 1 -- 172.21.15.43:0/2423950355 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb14802cbe0 con 0x7fb1500cd6b0 2023-08-09T13:48:22.425 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:22.421+0000 7fb14effd640 1 -- 172.21.15.43:0/2423950355 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb1480343f0 con 0x7fb1500cd6b0 2023-08-09T13:48:22.425 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:22.421+0000 7fb1554c0640 1 -- 172.21.15.43:0/2423950355 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb1500cd6b0 msgr2=0x7fb1500d0e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:48:22.425 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:22.421+0000 7fb1554c0640 1 --2- 172.21.15.43:0/2423950355 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb1500cd6b0 0x7fb1500d0e50 secure :-1 s=READY pgs=854 cs=0 l=1 rev1=1 crypto rx=0x7fb1500cfaf0 tx=0x7fb14802bee0 comp rx=0 tx=0).stop 2023-08-09T13:48:22.426 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:22.421+0000 7fb1554c0640 1 -- 172.21.15.43:0/2423950355 shutdown_connections 2023-08-09T13:48:22.426 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:22.421+0000 7fb1554c0640 1 --2- 172.21.15.43:0/2423950355 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb1500cd6b0 0x7fb1500d0e50 unknown :-1 s=CLOSED pgs=854 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:48:22.426 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:22.421+0000 7fb1554c0640 1 --2- 172.21.15.43:0/2423950355 >> v2:172.21.15.203:3300/0 conn(0x7fb1500b1050 0x7fb1500b4760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:48:22.426 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:22.421+0000 7fb1554c0640 1 -- 172.21.15.43:0/2423950355 >> 172.21.15.43:0/2423950355 conn(0x7fb15006a2b0 msgr2=0x7fb15006a690 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-09T13:48:22.426 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:22.421+0000 7fb1554c0640 1 -- 172.21.15.43:0/2423950355 shutdown_connections 2023-08-09T13:48:22.426 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:22.421+0000 7fb1554c0640 1 -- 172.21.15.43:0/2423950355 wait complete. 2023-08-09T13:48:22.427 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:22.424+0000 7fb1554c0640 1 Processor -- start 2023-08-09T13:48:22.427 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:22.424+0000 7fb1554c0640 1 -- start start 2023-08-09T13:48:22.428 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:22.425+0000 7fb1554c0640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb1500b1050 0x7fb150154a90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:48:22.428 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:22.425+0000 7fb1554c0640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fb1500cd6b0 0x7fb150157c90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:48:22.428 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:22.425+0000 7fb1554c0640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fb15015f7b0 0x7fb150152bc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:48:22.428 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:22.425+0000 7fb1554c0640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb150154fd0 con 0x7fb1500b1050 2023-08-09T13:48:22.429 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:22.425+0000 7fb1554c0640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb150158450 con 0x7fb1500cd6b0 2023-08-09T13:48:22.429 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:22.425+0000 7fb1554c0640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb150158730 con 0x7fb15015f7b0 2023-08-09T13:48:22.429 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:22.426+0000 7fb14ffff640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb1500b1050 0x7fb150154a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:48:22.429 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:22.426+0000 7fb14ffff640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb1500b1050 0x7fb150154a90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:36840/0 (socket says 172.21.15.43:36840) 2023-08-09T13:48:22.429 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:22.426+0000 7fb14ffff640 1 -- 172.21.15.43:0/3522861759 learned_addr learned my addr 172.21.15.43:0/3522861759 (peer_addr_for_me v2:172.21.15.43:0/0) 2023-08-09T13:48:22.430 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:22.426+0000 7fb154cbf640 1 --2- 172.21.15.43:0/3522861759 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fb15015f7b0 0x7fb150152bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:48:22.431 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:22.426+0000 7fb14f7fe640 1 --2- 172.21.15.43:0/3522861759 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fb1500cd6b0 0x7fb150157c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:48:22.431 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:22.426+0000 7fb14ffff640 1 -- 172.21.15.43:0/3522861759 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fb15015f7b0 msgr2=0x7fb150152bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:48:22.431 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:22.426+0000 7fb14ffff640 1 --2- 172.21.15.43:0/3522861759 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fb15015f7b0 0x7fb150152bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:48:22.431 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:22.426+0000 7fb14ffff640 1 -- 172.21.15.43:0/3522861759 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fb1500cd6b0 msgr2=0x7fb150157c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:48:22.431 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:22.426+0000 7fb14ffff640 1 --2- 172.21.15.43:0/3522861759 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fb1500cd6b0 0x7fb150157c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:48:22.431 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:22.426+0000 7fb14ffff640 1 -- 172.21.15.43:0/3522861759 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb148009b30 con 0x7fb1500b1050 2023-08-09T13:48:22.432 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:22.426+0000 7fb14f7fe640 1 --2- 172.21.15.43:0/3522861759 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fb1500cd6b0 0x7fb150157c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-08-09T13:48:22.432 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:22.427+0000 7fb14ffff640 1 --2- 172.21.15.43:0/3522861759 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb1500b1050 0x7fb150154a90 secure :-1 s=READY pgs=855 cs=0 l=1 rev1=1 crypto rx=0x7fb14000c430 tx=0x7fb14000c8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-09T13:48:22.432 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:22.427+0000 7fb14cff9640 1 -- 172.21.15.43:0/3522861759 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb140011790 con 0x7fb1500b1050 2023-08-09T13:48:22.432 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:22.427+0000 7fb14cff9640 1 -- 172.21.15.43:0/3522861759 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb1400118f0 con 0x7fb1500b1050 2023-08-09T13:48:22.432 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:22.427+0000 7fb1554c0640 1 -- 172.21.15.43:0/3522861759 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb150158a10 con 0x7fb1500b1050 2023-08-09T13:48:22.432 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:22.427+0000 7fb14cff9640 1 -- 172.21.15.43:0/3522861759 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb140011ce0 con 0x7fb1500b1050 2023-08-09T13:48:22.433 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:22.427+0000 7fb1554c0640 1 -- 172.21.15.43:0/3522861759 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb150158fd0 con 0x7fb1500b1050 2023-08-09T13:48:22.433 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:22.428+0000 7fb1554c0640 1 -- 172.21.15.43:0/3522861759 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb1500739f0 con 0x7fb1500b1050 2023-08-09T13:48:22.433 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:22.429+0000 7fb14cff9640 1 -- 172.21.15.43:0/3522861759 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 18) v1 ==== 88443+0+0 (secure 0 0 0) 0x7fb14001c460 con 0x7fb1500b1050 2023-08-09T13:48:22.434 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:22.430+0000 7fb14cff9640 1 --2- 172.21.15.43:0/3522861759 >> [v2:172.21.15.203:6832/272917756,v1:172.21.15.203:6833/272917756] conn(0x7fb13c06a1a0 0x7fb13c06c660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:48:22.434 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:22.430+0000 7fb14cff9640 1 -- 172.21.15.43:0/3522861759 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(74..74 src has 1..74) v4 ==== 6426+0+0 (secure 0 0 0) 0x7fb140088820 con 0x7fb1500b1050 2023-08-09T13:48:22.434 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:22.431+0000 7fb14f7fe640 1 --2- 172.21.15.43:0/3522861759 >> [v2:172.21.15.203:6832/272917756,v1:172.21.15.203:6833/272917756] conn(0x7fb13c06a1a0 0x7fb13c06c660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:48:22.434 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:22.432+0000 7fb14f7fe640 1 --2- 172.21.15.43:0/3522861759 >> [v2:172.21.15.203:6832/272917756,v1:172.21.15.203:6833/272917756] conn(0x7fb13c06a1a0 0x7fb13c06c660 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7fb144006fd0 tx=0x7fb144008040 comp rx=0 tx=0).ready entity=mgr.5533 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-09T13:48:22.437 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:22.435+0000 7fb14cff9640 1 -- 172.21.15.43:0/3522861759 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178066 (secure 0 0 0) 0x7fb140053c80 con 0x7fb1500b1050 2023-08-09T13:48:22.627 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:22.624+0000 7fb1554c0640 1 -- 172.21.15.43:0/3522861759 --> [v2:172.21.15.203:6832/272917756,v1:172.21.15.203:6833/272917756] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7fb1500b81b0 con 0x7fb13c06a1a0 2023-08-09T13:48:22.657 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:22.655+0000 7fb14cff9640 1 -- 172.21.15.43:0/3522861759 <== mgr.5533 v2:172.21.15.203:6832/272917756 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+308467 (secure 0 0 0) 0x7fb14400c040 con 0x7fb13c06a1a0 2023-08-09T13:48:22.661 INFO:teuthology.orchestra.run.smithi043.stdout: 2023-08-09T13:48:22.662 INFO:teuthology.orchestra.run.smithi043.stderr:dumped all 2023-08-09T13:48:22.664 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:22.660+0000 7fb1554c0640 1 -- 172.21.15.43:0/3522861759 >> [v2:172.21.15.203:6832/272917756,v1:172.21.15.203:6833/272917756] conn(0x7fb13c06a1a0 msgr2=0x7fb13c06c660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:48:22.664 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:22.660+0000 7fb1554c0640 1 --2- 172.21.15.43:0/3522861759 >> [v2:172.21.15.203:6832/272917756,v1:172.21.15.203:6833/272917756] conn(0x7fb13c06a1a0 0x7fb13c06c660 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7fb144006fd0 tx=0x7fb144008040 comp rx=0 tx=0).stop 2023-08-09T13:48:22.665 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:22.660+0000 7fb1554c0640 1 -- 172.21.15.43:0/3522861759 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb1500b1050 msgr2=0x7fb150154a90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:48:22.665 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:22.660+0000 7fb1554c0640 1 --2- 172.21.15.43:0/3522861759 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb1500b1050 0x7fb150154a90 secure :-1 s=READY pgs=855 cs=0 l=1 rev1=1 crypto rx=0x7fb14000c430 tx=0x7fb14000c8f0 comp rx=0 tx=0).stop 2023-08-09T13:48:22.665 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:22.661+0000 7fb1554c0640 1 -- 172.21.15.43:0/3522861759 shutdown_connections 2023-08-09T13:48:22.665 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:22.661+0000 7fb1554c0640 1 --2- 172.21.15.43:0/3522861759 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fb15015f7b0 0x7fb150152bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:48:22.665 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:22.661+0000 7fb1554c0640 1 --2- 172.21.15.43:0/3522861759 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fb1500cd6b0 0x7fb150157c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:48:22.665 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:22.661+0000 7fb1554c0640 1 --2- 172.21.15.43:0/3522861759 >> [v2:172.21.15.203:6832/272917756,v1:172.21.15.203:6833/272917756] conn(0x7fb13c06a1a0 0x7fb13c06c660 unknown :-1 s=CLOSED pgs=65 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:48:22.666 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:22.661+0000 7fb1554c0640 1 --2- 172.21.15.43:0/3522861759 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb1500b1050 0x7fb150154a90 unknown :-1 s=CLOSED pgs=855 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:48:22.666 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:22.661+0000 7fb1554c0640 1 -- 172.21.15.43:0/3522861759 >> 172.21.15.43:0/3522861759 conn(0x7fb15006a2b0 msgr2=0x7fb15015ddb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-09T13:48:22.666 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:22.661+0000 7fb1554c0640 1 -- 172.21.15.43:0/3522861759 shutdown_connections 2023-08-09T13:48:22.666 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:22.661+0000 7fb1554c0640 1 -- 172.21.15.43:0/3522861759 wait complete. 2023-08-09T13:48:22.682 INFO:teuthology.orchestra.run.smithi043.stdout:{"pg_ready":true,"pg_map":{"version":28,"stamp":"2023-08-09T13:48:21.256543+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":23,"num_object_clones":0,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":0,"num_read":308,"num_read_kb":512,"num_write":243,"num_write_kb":2430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":177,"ondisk_log_size":177,"up":210,"acting":210,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":61,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":269144,"kb_used_data":7904,"kb_used_omap":13,"kb_used_meta":261106,"kb_avail":754705576,"statfs":{"total":773094113280,"available":772818509824,"internally_reserved":0,"allocated":8093696,"data_stored":4822022,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":14248,"internal_metadata":267372632},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":37,"apply_latency_ms":37,"commit_latency_ns":37000000,"apply_latency_ns":37000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"5.167888"},"pg_stats":[{"pgid":"9.3f","version":"0'0","reported_seq":9,"reported_epoch":73,"state":"active+clean","last_fresh":"2023-08-09T13:48:18.158396+0000","last_change":"2023-08-09T13:48:18.152837+0000","last_active":"2023-08-09T13:48:18.158396+0000","last_peered":"2023-08-09T13:48:18.158396+0000","last_clean":"2023-08-09T13:48:18.158396+0000","last_became_active":"2023-08-09T13:48:18.152573+0000","last_became_peered":"2023-08-09T13:48:18.152573+0000","last_unstale":"2023-08-09T13:48:18.158396+0000","last_undegraded":"2023-08-09T13:48:18.158396+0000","last_fullsized":"2023-08-09T13:48:18.158396+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:48:17.087857+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:48:17.087857+0000","last_clean_scrub_stamp":"2023-08-09T13:48:17.087857+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T14:47:18.483275+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.3e","version":"0'0","reported_seq":9,"reported_epoch":73,"state":"active+clean","last_fresh":"2023-08-09T13:48:18.157639+0000","last_change":"2023-08-09T13:48:18.155184+0000","last_active":"2023-08-09T13:48:18.157639+0000","last_peered":"2023-08-09T13:48:18.157639+0000","last_clean":"2023-08-09T13:48:18.157639+0000","last_became_active":"2023-08-09T13:48:18.154907+0000","last_became_peered":"2023-08-09T13:48:18.154907+0000","last_unstale":"2023-08-09T13:48:18.157639+0000","last_undegraded":"2023-08-09T13:48:18.157639+0000","last_fullsized":"2023-08-09T13:48:18.157639+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:48:17.087857+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:48:17.087857+0000","last_clean_scrub_stamp":"2023-08-09T13:48:17.087857+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T15:37:16.143199+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.3d","version":"0'0","reported_seq":9,"reported_epoch":73,"state":"active+clean","last_fresh":"2023-08-09T13:48:18.151515+0000","last_change":"2023-08-09T13:48:18.146062+0000","last_active":"2023-08-09T13:48:18.151515+0000","last_peered":"2023-08-09T13:48:18.151515+0000","last_clean":"2023-08-09T13:48:18.151515+0000","last_became_active":"2023-08-09T13:48:18.145368+0000","last_became_peered":"2023-08-09T13:48:18.145368+0000","last_unstale":"2023-08-09T13:48:18.151515+0000","last_undegraded":"2023-08-09T13:48:18.151515+0000","last_fullsized":"2023-08-09T13:48:18.151515+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:48:17.087857+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:48:17.087857+0000","last_clean_scrub_stamp":"2023-08-09T13:48:17.087857+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-11T00:38:33.875100+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.3c","version":"0'0","reported_seq":9,"reported_epoch":73,"state":"active+clean","last_fresh":"2023-08-09T13:48:18.151811+0000","last_change":"2023-08-09T13:48:18.146500+0000","last_active":"2023-08-09T13:48:18.151811+0000","last_peered":"2023-08-09T13:48:18.151811+0000","last_clean":"2023-08-09T13:48:18.151811+0000","last_became_active":"2023-08-09T13:48:18.146233+0000","last_became_peered":"2023-08-09T13:48:18.146233+0000","last_unstale":"2023-08-09T13:48:18.151811+0000","last_undegraded":"2023-08-09T13:48:18.151811+0000","last_fullsized":"2023-08-09T13:48:18.151811+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:48:17.087857+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:48:17.087857+0000","last_clean_scrub_stamp":"2023-08-09T13:48:17.087857+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-11T00:25:21.491809+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.3b","version":"0'0","reported_seq":9,"reported_epoch":73,"state":"active+clean","last_fresh":"2023-08-09T13:48:18.155506+0000","last_change":"2023-08-09T13:48:18.143602+0000","last_active":"2023-08-09T13:48:18.155506+0000","last_peered":"2023-08-09T13:48:18.155506+0000","last_clean":"2023-08-09T13:48:18.155506+0000","last_became_active":"2023-08-09T13:48:18.143312+0000","last_became_peered":"2023-08-09T13:48:18.143312+0000","last_unstale":"2023-08-09T13:48:18.155506+0000","last_undegraded":"2023-08-09T13:48:18.155506+0000","last_fullsized":"2023-08-09T13:48:18.155506+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:48:17.087857+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:48:17.087857+0000","last_clean_scrub_stamp":"2023-08-09T13:48:17.087857+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-11T01:42:07.858900+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.3a","version":"0'0","reported_seq":9,"reported_epoch":73,"state":"active+clean","last_fresh":"2023-08-09T13:48:18.158645+0000","last_change":"2023-08-09T13:48:18.144574+0000","last_active":"2023-08-09T13:48:18.158645+0000","last_peered":"2023-08-09T13:48:18.158645+0000","last_clean":"2023-08-09T13:48:18.158645+0000","last_became_active":"2023-08-09T13:48:18.144221+0000","last_became_peered":"2023-08-09T13:48:18.144221+0000","last_unstale":"2023-08-09T13:48:18.158645+0000","last_undegraded":"2023-08-09T13:48:18.158645+0000","last_fullsized":"2023-08-09T13:48:18.158645+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:48:17.087857+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:48:17.087857+0000","last_clean_scrub_stamp":"2023-08-09T13:48:17.087857+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T22:24:54.272226+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.39","version":"0'0","reported_seq":9,"reported_epoch":73,"state":"active+clean","last_fresh":"2023-08-09T13:48:18.144564+0000","last_change":"2023-08-09T13:48:18.142714+0000","last_active":"2023-08-09T13:48:18.144564+0000","last_peered":"2023-08-09T13:48:18.144564+0000","last_clean":"2023-08-09T13:48:18.144564+0000","last_became_active":"2023-08-09T13:48:18.141836+0000","last_became_peered":"2023-08-09T13:48:18.141836+0000","last_unstale":"2023-08-09T13:48:18.144564+0000","last_undegraded":"2023-08-09T13:48:18.144564+0000","last_fullsized":"2023-08-09T13:48:18.144564+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:48:17.087857+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:48:17.087857+0000","last_clean_scrub_stamp":"2023-08-09T13:48:17.087857+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T22:54:27.458384+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.38","version":"0'0","reported_seq":9,"reported_epoch":73,"state":"active+clean","last_fresh":"2023-08-09T13:48:18.156563+0000","last_change":"2023-08-09T13:48:18.151342+0000","last_active":"2023-08-09T13:48:18.156563+0000","last_peered":"2023-08-09T13:48:18.156563+0000","last_clean":"2023-08-09T13:48:18.156563+0000","last_became_active":"2023-08-09T13:48:18.151000+0000","last_became_peered":"2023-08-09T13:48:18.151000+0000","last_unstale":"2023-08-09T13:48:18.156563+0000","last_undegraded":"2023-08-09T13:48:18.156563+0000","last_fullsized":"2023-08-09T13:48:18.156563+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:48:17.087857+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:48:17.087857+0000","last_clean_scrub_stamp":"2023-08-09T13:48:17.087857+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T22:58:06.877210+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.37","version":"0'0","reported_seq":9,"reported_epoch":73,"state":"active+clean","last_fresh":"2023-08-09T13:48:18.148105+0000","last_change":"2023-08-09T13:48:18.146306+0000","last_active":"2023-08-09T13:48:18.148105+0000","last_peered":"2023-08-09T13:48:18.148105+0000","last_clean":"2023-08-09T13:48:18.148105+0000","last_became_active":"2023-08-09T13:48:18.145883+0000","last_became_peered":"2023-08-09T13:48:18.145883+0000","last_unstale":"2023-08-09T13:48:18.148105+0000","last_undegraded":"2023-08-09T13:48:18.148105+0000","last_fullsized":"2023-08-09T13:48:18.148105+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:48:17.087857+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:48:17.087857+0000","last_clean_scrub_stamp":"2023-08-09T13:48:17.087857+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T17:36:56.420913+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.36","version":"0'0","reported_seq":9,"reported_epoch":73,"state":"active+clean","last_fresh":"2023-08-09T13:48:18.143817+0000","last_change":"2023-08-09T13:48:18.127654+0000","last_active":"2023-08-09T13:48:18.143817+0000","last_peered":"2023-08-09T13:48:18.143817+0000","last_clean":"2023-08-09T13:48:18.143817+0000","last_became_active":"2023-08-09T13:48:18.126751+0000","last_became_peered":"2023-08-09T13:48:18.126751+0000","last_unstale":"2023-08-09T13:48:18.143817+0000","last_undegraded":"2023-08-09T13:48:18.143817+0000","last_fullsized":"2023-08-09T13:48:18.143817+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:48:17.087857+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:48:17.087857+0000","last_clean_scrub_stamp":"2023-08-09T13:48:17.087857+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T22:03:13.433435+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.35","version":"0'0","reported_seq":9,"reported_epoch":73,"state":"active+clean","last_fresh":"2023-08-09T13:48:18.159348+0000","last_change":"2023-08-09T13:48:18.154916+0000","last_active":"2023-08-09T13:48:18.159348+0000","last_peered":"2023-08-09T13:48:18.159348+0000","last_clean":"2023-08-09T13:48:18.159348+0000","last_became_active":"2023-08-09T13:48:18.154656+0000","last_became_peered":"2023-08-09T13:48:18.154656+0000","last_unstale":"2023-08-09T13:48:18.159348+0000","last_undegraded":"2023-08-09T13:48:18.159348+0000","last_fullsized":"2023-08-09T13:48:18.159348+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:48:17.087857+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:48:17.087857+0000","last_clean_scrub_stamp":"2023-08-09T13:48:17.087857+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T18:52:15.342531+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.34","version":"0'0","reported_seq":9,"reported_epoch":73,"state":"active+clean","last_fresh":"2023-08-09T13:48:18.141946+0000","last_change":"2023-08-09T13:48:18.139766+0000","last_active":"2023-08-09T13:48:18.141946+0000","last_peered":"2023-08-09T13:48:18.141946+0000","last_clean":"2023-08-09T13:48:18.141946+0000","last_became_active":"2023-08-09T13:48:18.139484+0000","last_became_peered":"2023-08-09T13:48:18.139484+0000","last_unstale":"2023-08-09T13:48:18.141946+0000","last_undegraded":"2023-08-09T13:48:18.141946+0000","last_fullsized":"2023-08-09T13:48:18.141946+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:48:17.087857+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:48:17.087857+0000","last_clean_scrub_stamp":"2023-08-09T13:48:17.087857+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T23:01:48.230063+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.33","version":"0'0","reported_seq":9,"reported_epoch":73,"state":"active+clean","last_fresh":"2023-08-09T13:48:18.149292+0000","last_change":"2023-08-09T13:48:18.147141+0000","last_active":"2023-08-09T13:48:18.149292+0000","last_peered":"2023-08-09T13:48:18.149292+0000","last_clean":"2023-08-09T13:48:18.149292+0000","last_became_active":"2023-08-09T13:48:18.146762+0000","last_became_peered":"2023-08-09T13:48:18.146762+0000","last_unstale":"2023-08-09T13:48:18.149292+0000","last_undegraded":"2023-08-09T13:48:18.149292+0000","last_fullsized":"2023-08-09T13:48:18.149292+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:48:17.087857+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:48:17.087857+0000","last_clean_scrub_stamp":"2023-08-09T13:48:17.087857+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T20:22:59.125660+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.32","version":"0'0","reported_seq":9,"reported_epoch":73,"state":"active+clean","last_fresh":"2023-08-09T13:48:18.147828+0000","last_change":"2023-08-09T13:48:18.146755+0000","last_active":"2023-08-09T13:48:18.147828+0000","last_peered":"2023-08-09T13:48:18.147828+0000","last_clean":"2023-08-09T13:48:18.147828+0000","last_became_active":"2023-08-09T13:48:18.146476+0000","last_became_peered":"2023-08-09T13:48:18.146476+0000","last_unstale":"2023-08-09T13:48:18.147828+0000","last_undegraded":"2023-08-09T13:48:18.147828+0000","last_fullsized":"2023-08-09T13:48:18.147828+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:48:17.087857+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:48:17.087857+0000","last_clean_scrub_stamp":"2023-08-09T13:48:17.087857+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T19:48:03.345033+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.31","version":"0'0","reported_seq":9,"reported_epoch":73,"state":"active+clean","last_fresh":"2023-08-09T13:48:18.152536+0000","last_change":"2023-08-09T13:48:18.151585+0000","last_active":"2023-08-09T13:48:18.152536+0000","last_peered":"2023-08-09T13:48:18.152536+0000","last_clean":"2023-08-09T13:48:18.152536+0000","last_became_active":"2023-08-09T13:48:18.151285+0000","last_became_peered":"2023-08-09T13:48:18.151285+0000","last_unstale":"2023-08-09T13:48:18.152536+0000","last_undegraded":"2023-08-09T13:48:18.152536+0000","last_fullsized":"2023-08-09T13:48:18.152536+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:48:17.087857+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:48:17.087857+0000","last_clean_scrub_stamp":"2023-08-09T13:48:17.087857+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T21:51:35.876003+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.30","version":"0'0","reported_seq":9,"reported_epoch":73,"state":"active+clean","last_fresh":"2023-08-09T13:48:18.156188+0000","last_change":"2023-08-09T13:48:18.140225+0000","last_active":"2023-08-09T13:48:18.156188+0000","last_peered":"2023-08-09T13:48:18.156188+0000","last_clean":"2023-08-09T13:48:18.156188+0000","last_became_active":"2023-08-09T13:48:18.139432+0000","last_became_peered":"2023-08-09T13:48:18.139432+0000","last_unstale":"2023-08-09T13:48:18.156188+0000","last_undegraded":"2023-08-09T13:48:18.156188+0000","last_fullsized":"2023-08-09T13:48:18.156188+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:48:17.087857+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:48:17.087857+0000","last_clean_scrub_stamp":"2023-08-09T13:48:17.087857+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T21:49:05.091868+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.2f","version":"0'0","reported_seq":9,"reported_epoch":73,"state":"active+clean","last_fresh":"2023-08-09T13:48:18.158014+0000","last_change":"2023-08-09T13:48:18.153848+0000","last_active":"2023-08-09T13:48:18.158014+0000","last_peered":"2023-08-09T13:48:18.158014+0000","last_clean":"2023-08-09T13:48:18.158014+0000","last_became_active":"2023-08-09T13:48:18.153548+0000","last_became_peered":"2023-08-09T13:48:18.153548+0000","last_unstale":"2023-08-09T13:48:18.158014+0000","last_undegraded":"2023-08-09T13:48:18.158014+0000","last_fullsized":"2023-08-09T13:48:18.158014+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:48:17.087857+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:48:17.087857+0000","last_clean_scrub_stamp":"2023-08-09T13:48:17.087857+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T18:38:04.068552+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.2e","version":"0'0","reported_seq":9,"reported_epoch":73,"state":"active+clean","last_fresh":"2023-08-09T13:48:18.151978+0000","last_change":"2023-08-09T13:48:18.150438+0000","last_active":"2023-08-09T13:48:18.151978+0000","last_peered":"2023-08-09T13:48:18.151978+0000","last_clean":"2023-08-09T13:48:18.151978+0000","last_became_active":"2023-08-09T13:48:18.150136+0000","last_became_peered":"2023-08-09T13:48:18.150136+0000","last_unstale":"2023-08-09T13:48:18.151978+0000","last_undegraded":"2023-08-09T13:48:18.151978+0000","last_fullsized":"2023-08-09T13:48:18.151978+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:48:17.087857+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:48:17.087857+0000","last_clean_scrub_stamp":"2023-08-09T13:48:17.087857+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T17:55:28.354834+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.2d","version":"0'0","reported_seq":9,"reported_epoch":73,"state":"active+clean","last_fresh":"2023-08-09T13:48:18.158447+0000","last_change":"2023-08-09T13:48:18.151532+0000","last_active":"2023-08-09T13:48:18.158447+0000","last_peered":"2023-08-09T13:48:18.158447+0000","last_clean":"2023-08-09T13:48:18.158447+0000","last_became_active":"2023-08-09T13:48:18.151242+0000","last_became_peered":"2023-08-09T13:48:18.151242+0000","last_unstale":"2023-08-09T13:48:18.158447+0000","last_undegraded":"2023-08-09T13:48:18.158447+0000","last_fullsized":"2023-08-09T13:48:18.158447+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:48:17.087857+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:48:17.087857+0000","last_clean_scrub_stamp":"2023-08-09T13:48:17.087857+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-11T01:28:23.557859+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.2c","version":"0'0","reported_seq":9,"reported_epoch":73,"state":"active+clean","last_fresh":"2023-08-09T13:48:18.159547+0000","last_change":"2023-08-09T13:48:18.156432+0000","last_active":"2023-08-09T13:48:18.159547+0000","last_peered":"2023-08-09T13:48:18.159547+0000","last_clean":"2023-08-09T13:48:18.159547+0000","last_became_active":"2023-08-09T13:48:18.156107+0000","last_became_peered":"2023-08-09T13:48:18.156107+0000","last_unstale":"2023-08-09T13:48:18.159547+0000","last_undegraded":"2023-08-09T13:48:18.159547+0000","last_fullsized":"2023-08-09T13:48:18.159547+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:48:17.087857+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:48:17.087857+0000","last_clean_scrub_stamp":"2023-08-09T13:48:17.087857+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T14:49:40.094879+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.2b","version":"0'0","reported_seq":9,"reported_epoch":73,"state":"active+clean","last_fresh":"2023-08-09T13:48:18.145479+0000","last_change":"2023-08-09T13:48:18.139749+0000","last_active":"2023-08-09T13:48:18.145479+0000","last_peered":"2023-08-09T13:48:18.145479+0000","last_clean":"2023-08-09T13:48:18.145479+0000","last_became_active":"2023-08-09T13:48:18.139434+0000","last_became_peered":"2023-08-09T13:48:18.139434+0000","last_unstale":"2023-08-09T13:48:18.145479+0000","last_undegraded":"2023-08-09T13:48:18.145479+0000","last_fullsized":"2023-08-09T13:48:18.145479+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:48:17.087857+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:48:17.087857+0000","last_clean_scrub_stamp":"2023-08-09T13:48:17.087857+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T15:10:50.771773+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.2a","version":"0'0","reported_seq":9,"reported_epoch":73,"state":"active+clean","last_fresh":"2023-08-09T13:48:18.157368+0000","last_change":"2023-08-09T13:48:18.152437+0000","last_active":"2023-08-09T13:48:18.157368+0000","last_peered":"2023-08-09T13:48:18.157368+0000","last_clean":"2023-08-09T13:48:18.157368+0000","last_became_active":"2023-08-09T13:48:18.151855+0000","last_became_peered":"2023-08-09T13:48:18.151855+0000","last_unstale":"2023-08-09T13:48:18.157368+0000","last_undegraded":"2023-08-09T13:48:18.157368+0000","last_fullsized":"2023-08-09T13:48:18.157368+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:48:17.087857+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:48:17.087857+0000","last_clean_scrub_stamp":"2023-08-09T13:48:17.087857+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T16:46:30.811573+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.29","version":"0'0","reported_seq":9,"reported_epoch":73,"state":"active+clean","last_fresh":"2023-08-09T13:48:18.155696+0000","last_change":"2023-08-09T13:48:18.154494+0000","last_active":"2023-08-09T13:48:18.155696+0000","last_peered":"2023-08-09T13:48:18.155696+0000","last_clean":"2023-08-09T13:48:18.155696+0000","last_became_active":"2023-08-09T13:48:18.154151+0000","last_became_peered":"2023-08-09T13:48:18.154151+0000","last_unstale":"2023-08-09T13:48:18.155696+0000","last_undegraded":"2023-08-09T13:48:18.155696+0000","last_fullsized":"2023-08-09T13:48:18.155696+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:48:17.087857+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:48:17.087857+0000","last_clean_scrub_stamp":"2023-08-09T13:48:17.087857+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T16:12:49.856088+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.28","version":"0'0","reported_seq":9,"reported_epoch":73,"state":"active+clean","last_fresh":"2023-08-09T13:48:18.143861+0000","last_change":"2023-08-09T13:48:18.140604+0000","last_active":"2023-08-09T13:48:18.143861+0000","last_peered":"2023-08-09T13:48:18.143861+0000","last_clean":"2023-08-09T13:48:18.143861+0000","last_became_active":"2023-08-09T13:48:18.140183+0000","last_became_peered":"2023-08-09T13:48:18.140183+0000","last_unstale":"2023-08-09T13:48:18.143861+0000","last_undegraded":"2023-08-09T13:48:18.143861+0000","last_fullsized":"2023-08-09T13:48:18.143861+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:48:17.087857+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:48:17.087857+0000","last_clean_scrub_stamp":"2023-08-09T13:48:17.087857+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T23:23:02.255139+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.27","version":"0'0","reported_seq":9,"reported_epoch":73,"state":"active+clean","last_fresh":"2023-08-09T13:48:18.158367+0000","last_change":"2023-08-09T13:48:18.149483+0000","last_active":"2023-08-09T13:48:18.158367+0000","last_peered":"2023-08-09T13:48:18.158367+0000","last_clean":"2023-08-09T13:48:18.158367+0000","last_became_active":"2023-08-09T13:48:18.149185+0000","last_became_peered":"2023-08-09T13:48:18.149185+0000","last_unstale":"2023-08-09T13:48:18.158367+0000","last_undegraded":"2023-08-09T13:48:18.158367+0000","last_fullsized":"2023-08-09T13:48:18.158367+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:48:17.087857+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:48:17.087857+0000","last_clean_scrub_stamp":"2023-08-09T13:48:17.087857+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T20:38:37.445234+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.26","version":"0'0","reported_seq":9,"reported_epoch":73,"state":"active+clean","last_fresh":"2023-08-09T13:48:18.145007+0000","last_change":"2023-08-09T13:48:18.143866+0000","last_active":"2023-08-09T13:48:18.145007+0000","last_peered":"2023-08-09T13:48:18.145007+0000","last_clean":"2023-08-09T13:48:18.145007+0000","last_became_active":"2023-08-09T13:48:18.143552+0000","last_became_peered":"2023-08-09T13:48:18.143552+0000","last_unstale":"2023-08-09T13:48:18.145007+0000","last_undegraded":"2023-08-09T13:48:18.145007+0000","last_fullsized":"2023-08-09T13:48:18.145007+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:48:17.087857+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:48:17.087857+0000","last_clean_scrub_stamp":"2023-08-09T13:48:17.087857+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T14:47:35.887828+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.25","version":"0'0","reported_seq":9,"reported_epoch":73,"state":"active+clean","last_fresh":"2023-08-09T13:48:18.158812+0000","last_change":"2023-08-09T13:48:18.147805+0000","last_active":"2023-08-09T13:48:18.158812+0000","last_peered":"2023-08-09T13:48:18.158812+0000","last_clean":"2023-08-09T13:48:18.158812+0000","last_became_active":"2023-08-09T13:48:18.147104+0000","last_became_peered":"2023-08-09T13:48:18.147104+0000","last_unstale":"2023-08-09T13:48:18.158812+0000","last_undegraded":"2023-08-09T13:48:18.158812+0000","last_fullsized":"2023-08-09T13:48:18.158812+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:48:17.087857+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:48:17.087857+0000","last_clean_scrub_stamp":"2023-08-09T13:48:17.087857+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T21:51:35.876003+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.24","version":"0'0","reported_seq":9,"reported_epoch":73,"state":"active+clean","last_fresh":"2023-08-09T13:48:18.145059+0000","last_change":"2023-08-09T13:48:18.139602+0000","last_active":"2023-08-09T13:48:18.145059+0000","last_peered":"2023-08-09T13:48:18.145059+0000","last_clean":"2023-08-09T13:48:18.145059+0000","last_became_active":"2023-08-09T13:48:18.139295+0000","last_became_peered":"2023-08-09T13:48:18.139295+0000","last_unstale":"2023-08-09T13:48:18.145059+0000","last_undegraded":"2023-08-09T13:48:18.145059+0000","last_fullsized":"2023-08-09T13:48:18.145059+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:48:17.087857+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:48:17.087857+0000","last_clean_scrub_stamp":"2023-08-09T13:48:17.087857+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-11T01:06:40.133788+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.23","version":"0'0","reported_seq":9,"reported_epoch":73,"state":"active+clean","last_fresh":"2023-08-09T13:48:18.157221+0000","last_change":"2023-08-09T13:48:18.152809+0000","last_active":"2023-08-09T13:48:18.157221+0000","last_peered":"2023-08-09T13:48:18.157221+0000","last_clean":"2023-08-09T13:48:18.157221+0000","last_became_active":"2023-08-09T13:48:18.152487+0000","last_became_peered":"2023-08-09T13:48:18.152487+0000","last_unstale":"2023-08-09T13:48:18.157221+0000","last_undegraded":"2023-08-09T13:48:18.157221+0000","last_fullsized":"2023-08-09T13:48:18.157221+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:48:17.087857+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:48:17.087857+0000","last_clean_scrub_stamp":"2023-08-09T13:48:17.087857+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T17:48:41.690755+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.22","version":"0'0","reported_seq":9,"reported_epoch":73,"state":"active+clean","last_fresh":"2023-08-09T13:48:18.149485+0000","last_change":"2023-08-09T13:48:18.144300+0000","last_active":"2023-08-09T13:48:18.149485+0000","last_peered":"2023-08-09T13:48:18.149485+0000","last_clean":"2023-08-09T13:48:18.149485+0000","last_became_active":"2023-08-09T13:48:18.143917+0000","last_became_peered":"2023-08-09T13:48:18.143917+0000","last_unstale":"2023-08-09T13:48:18.149485+0000","last_undegraded":"2023-08-09T13:48:18.149485+0000","last_fullsized":"2023-08-09T13:48:18.149485+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:48:17.087857+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:48:17.087857+0000","last_clean_scrub_stamp":"2023-08-09T13:48:17.087857+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T18:20:16.882105+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.21","version":"0'0","reported_seq":9,"reported_epoch":73,"state":"active+clean","last_fresh":"2023-08-09T13:48:18.158292+0000","last_change":"2023-08-09T13:48:18.150149+0000","last_active":"2023-08-09T13:48:18.158292+0000","last_peered":"2023-08-09T13:48:18.158292+0000","last_clean":"2023-08-09T13:48:18.158292+0000","last_became_active":"2023-08-09T13:48:18.149859+0000","last_became_peered":"2023-08-09T13:48:18.149859+0000","last_unstale":"2023-08-09T13:48:18.158292+0000","last_undegraded":"2023-08-09T13:48:18.158292+0000","last_fullsized":"2023-08-09T13:48:18.158292+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:48:17.087857+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:48:17.087857+0000","last_clean_scrub_stamp":"2023-08-09T13:48:17.087857+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T16:44:17.531789+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.20","version":"0'0","reported_seq":9,"reported_epoch":73,"state":"active+clean","last_fresh":"2023-08-09T13:48:18.158987+0000","last_change":"2023-08-09T13:48:18.152970+0000","last_active":"2023-08-09T13:48:18.158987+0000","last_peered":"2023-08-09T13:48:18.158987+0000","last_clean":"2023-08-09T13:48:18.158987+0000","last_became_active":"2023-08-09T13:48:18.152669+0000","last_became_peered":"2023-08-09T13:48:18.152669+0000","last_unstale":"2023-08-09T13:48:18.158987+0000","last_undegraded":"2023-08-09T13:48:18.158987+0000","last_fullsized":"2023-08-09T13:48:18.158987+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:48:17.087857+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:48:17.087857+0000","last_clean_scrub_stamp":"2023-08-09T13:48:17.087857+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T18:57:02.550710+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.1e","version":"0'0","reported_seq":9,"reported_epoch":73,"state":"active+clean","last_fresh":"2023-08-09T13:48:18.154692+0000","last_change":"2023-08-09T13:48:18.141045+0000","last_active":"2023-08-09T13:48:18.154692+0000","last_peered":"2023-08-09T13:48:18.154692+0000","last_clean":"2023-08-09T13:48:18.154692+0000","last_became_active":"2023-08-09T13:48:18.139631+0000","last_became_peered":"2023-08-09T13:48:18.139631+0000","last_unstale":"2023-08-09T13:48:18.154692+0000","last_undegraded":"2023-08-09T13:48:18.154692+0000","last_fullsized":"2023-08-09T13:48:18.154692+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:48:17.087857+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:48:17.087857+0000","last_clean_scrub_stamp":"2023-08-09T13:48:17.087857+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T17:07:09.487635+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.1f","version":"73'4","reported_seq":15,"reported_epoch":73,"state":"active+clean","last_fresh":"2023-08-09T13:48:18.173947+0000","last_change":"2023-08-09T13:48:17.136238+0000","last_active":"2023-08-09T13:48:18.173947+0000","last_peered":"2023-08-09T13:48:18.173947+0000","last_clean":"2023-08-09T13:48:18.173947+0000","last_became_active":"2023-08-09T13:48:17.135961+0000","last_became_peered":"2023-08-09T13:48:17.135961+0000","last_unstale":"2023-08-09T13:48:18.173947+0000","last_undegraded":"2023-08-09T13:48:18.173947+0000","last_fullsized":"2023-08-09T13:48:18.173947+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:48:16.072766+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:48:16.072766+0000","last_clean_scrub_stamp":"2023-08-09T13:48:16.072766+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T18:40:14.841360+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.1f","version":"0'0","reported_seq":9,"reported_epoch":73,"state":"active+clean","last_fresh":"2023-08-09T13:48:18.140829+0000","last_change":"2023-08-09T13:48:18.131283+0000","last_active":"2023-08-09T13:48:18.140829+0000","last_peered":"2023-08-09T13:48:18.140829+0000","last_clean":"2023-08-09T13:48:18.140829+0000","last_became_active":"2023-08-09T13:48:18.130497+0000","last_became_peered":"2023-08-09T13:48:18.130497+0000","last_unstale":"2023-08-09T13:48:18.140829+0000","last_undegraded":"2023-08-09T13:48:18.140829+0000","last_fullsized":"2023-08-09T13:48:18.140829+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:48:17.087857+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:48:17.087857+0000","last_clean_scrub_stamp":"2023-08-09T13:48:17.087857+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T15:49:55.658613+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.1e","version":"73'1","reported_seq":12,"reported_epoch":73,"state":"active+clean","last_fresh":"2023-08-09T13:48:18.170845+0000","last_change":"2023-08-09T13:48:17.133120+0000","last_active":"2023-08-09T13:48:18.170845+0000","last_peered":"2023-08-09T13:48:18.170845+0000","last_clean":"2023-08-09T13:48:18.170845+0000","last_became_active":"2023-08-09T13:48:17.132760+0000","last_became_peered":"2023-08-09T13:48:17.132760+0000","last_unstale":"2023-08-09T13:48:18.170845+0000","last_undegraded":"2023-08-09T13:48:18.170845+0000","last_fullsized":"2023-08-09T13:48:18.170845+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:48:16.072766+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:48:16.072766+0000","last_clean_scrub_stamp":"2023-08-09T13:48:16.072766+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T23:40:53.809406+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.1c","version":"0'0","reported_seq":9,"reported_epoch":73,"state":"active+clean","last_fresh":"2023-08-09T13:48:18.145253+0000","last_change":"2023-08-09T13:48:18.133502+0000","last_active":"2023-08-09T13:48:18.145253+0000","last_peered":"2023-08-09T13:48:18.145253+0000","last_clean":"2023-08-09T13:48:18.145253+0000","last_became_active":"2023-08-09T13:48:18.132934+0000","last_became_peered":"2023-08-09T13:48:18.132934+0000","last_unstale":"2023-08-09T13:48:18.145253+0000","last_undegraded":"2023-08-09T13:48:18.145253+0000","last_fullsized":"2023-08-09T13:48:18.145253+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:48:17.087857+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:48:17.087857+0000","last_clean_scrub_stamp":"2023-08-09T13:48:17.087857+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T16:14:25.369270+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.1d","version":"73'2","reported_seq":13,"reported_epoch":73,"state":"active+clean","last_fresh":"2023-08-09T13:48:18.174000+0000","last_change":"2023-08-09T13:48:17.128733+0000","last_active":"2023-08-09T13:48:18.174000+0000","last_peered":"2023-08-09T13:48:18.174000+0000","last_clean":"2023-08-09T13:48:18.174000+0000","last_became_active":"2023-08-09T13:48:17.128462+0000","last_became_peered":"2023-08-09T13:48:17.128462+0000","last_unstale":"2023-08-09T13:48:18.174000+0000","last_undegraded":"2023-08-09T13:48:18.174000+0000","last_fullsized":"2023-08-09T13:48:18.174000+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:48:16.072766+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:48:16.072766+0000","last_clean_scrub_stamp":"2023-08-09T13:48:16.072766+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T15:33:28.704733+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.1d","version":"0'0","reported_seq":9,"reported_epoch":73,"state":"active+clean","last_fresh":"2023-08-09T13:48:18.158789+0000","last_change":"2023-08-09T13:48:18.147544+0000","last_active":"2023-08-09T13:48:18.158789+0000","last_peered":"2023-08-09T13:48:18.158789+0000","last_clean":"2023-08-09T13:48:18.158789+0000","last_became_active":"2023-08-09T13:48:18.146694+0000","last_became_peered":"2023-08-09T13:48:18.146694+0000","last_unstale":"2023-08-09T13:48:18.158789+0000","last_undegraded":"2023-08-09T13:48:18.158789+0000","last_fullsized":"2023-08-09T13:48:18.158789+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:48:17.087857+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:48:17.087857+0000","last_clean_scrub_stamp":"2023-08-09T13:48:17.087857+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-11T01:20:29.944656+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.1c","version":"73'2","reported_seq":13,"reported_epoch":73,"state":"active+clean","last_fresh":"2023-08-09T13:48:18.173560+0000","last_change":"2023-08-09T13:48:17.134613+0000","last_active":"2023-08-09T13:48:18.173560+0000","last_peered":"2023-08-09T13:48:18.173560+0000","last_clean":"2023-08-09T13:48:18.173560+0000","last_became_active":"2023-08-09T13:48:17.134264+0000","last_became_peered":"2023-08-09T13:48:17.134264+0000","last_unstale":"2023-08-09T13:48:18.173560+0000","last_undegraded":"2023-08-09T13:48:18.173560+0000","last_fullsized":"2023-08-09T13:48:18.173560+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:48:16.072766+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:48:16.072766+0000","last_clean_scrub_stamp":"2023-08-09T13:48:16.072766+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T19:50:07.010215+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.1a","version":"0'0","reported_seq":9,"reported_epoch":73,"state":"active+clean","last_fresh":"2023-08-09T13:48:18.157345+0000","last_change":"2023-08-09T13:48:18.152365+0000","last_active":"2023-08-09T13:48:18.157345+0000","last_peered":"2023-08-09T13:48:18.157345+0000","last_clean":"2023-08-09T13:48:18.157345+0000","last_became_active":"2023-08-09T13:48:18.151513+0000","last_became_peered":"2023-08-09T13:48:18.151513+0000","last_unstale":"2023-08-09T13:48:18.157345+0000","last_undegraded":"2023-08-09T13:48:18.157345+0000","last_fullsized":"2023-08-09T13:48:18.157345+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:48:17.087857+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:48:17.087857+0000","last_clean_scrub_stamp":"2023-08-09T13:48:17.087857+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T16:58:20.620035+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.1b","version":"73'1","reported_seq":12,"reported_epoch":73,"state":"active+clean","last_fresh":"2023-08-09T13:48:18.171629+0000","last_change":"2023-08-09T13:48:17.137906+0000","last_active":"2023-08-09T13:48:18.171629+0000","last_peered":"2023-08-09T13:48:18.171629+0000","last_clean":"2023-08-09T13:48:18.171629+0000","last_became_active":"2023-08-09T13:48:17.137624+0000","last_became_peered":"2023-08-09T13:48:17.137624+0000","last_unstale":"2023-08-09T13:48:18.171629+0000","last_undegraded":"2023-08-09T13:48:18.171629+0000","last_fullsized":"2023-08-09T13:48:18.171629+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:48:16.072766+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:48:16.072766+0000","last_clean_scrub_stamp":"2023-08-09T13:48:16.072766+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T19:30:54.824510+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.1b","version":"0'0","reported_seq":9,"reported_epoch":73,"state":"active+clean","last_fresh":"2023-08-09T13:48:18.160444+0000","last_change":"2023-08-09T13:48:18.157276+0000","last_active":"2023-08-09T13:48:18.160444+0000","last_peered":"2023-08-09T13:48:18.160444+0000","last_clean":"2023-08-09T13:48:18.160444+0000","last_became_active":"2023-08-09T13:48:18.157001+0000","last_became_peered":"2023-08-09T13:48:18.157001+0000","last_unstale":"2023-08-09T13:48:18.160444+0000","last_undegraded":"2023-08-09T13:48:18.160444+0000","last_fullsized":"2023-08-09T13:48:18.160444+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:48:17.087857+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:48:17.087857+0000","last_clean_scrub_stamp":"2023-08-09T13:48:17.087857+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T22:25:03.373484+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.1a","version":"73'1","reported_seq":12,"reported_epoch":73,"state":"active+clean","last_fresh":"2023-08-09T13:48:18.173513+0000","last_change":"2023-08-09T13:48:17.136309+0000","last_active":"2023-08-09T13:48:18.173513+0000","last_peered":"2023-08-09T13:48:18.173513+0000","last_clean":"2023-08-09T13:48:18.173513+0000","last_became_active":"2023-08-09T13:48:17.136046+0000","last_became_peered":"2023-08-09T13:48:17.136046+0000","last_unstale":"2023-08-09T13:48:18.173513+0000","last_undegraded":"2023-08-09T13:48:18.173513+0000","last_fullsized":"2023-08-09T13:48:18.173513+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:48:16.072766+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:48:16.072766+0000","last_clean_scrub_stamp":"2023-08-09T13:48:16.072766+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T18:49:45.672784+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.18","version":"0'0","reported_seq":9,"reported_epoch":73,"state":"active+clean","last_fresh":"2023-08-09T13:48:18.137208+0000","last_change":"2023-08-09T13:48:18.136226+0000","last_active":"2023-08-09T13:48:18.137208+0000","last_peered":"2023-08-09T13:48:18.137208+0000","last_clean":"2023-08-09T13:48:18.137208+0000","last_became_active":"2023-08-09T13:48:18.135880+0000","last_became_peered":"2023-08-09T13:48:18.135880+0000","last_unstale":"2023-08-09T13:48:18.137208+0000","last_undegraded":"2023-08-09T13:48:18.137208+0000","last_fullsized":"2023-08-09T13:48:18.137208+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:48:17.087857+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:48:17.087857+0000","last_clean_scrub_stamp":"2023-08-09T13:48:17.087857+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T16:42:47.998599+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.19","version":"0'0","reported_seq":11,"reported_epoch":73,"state":"active+clean","last_fresh":"2023-08-09T13:48:18.133469+0000","last_change":"2023-08-09T13:48:17.138354+0000","last_active":"2023-08-09T13:48:18.133469+0000","last_peered":"2023-08-09T13:48:18.133469+0000","last_clean":"2023-08-09T13:48:18.133469+0000","last_became_active":"2023-08-09T13:48:17.137887+0000","last_became_peered":"2023-08-09T13:48:17.137887+0000","last_unstale":"2023-08-09T13:48:18.133469+0000","last_undegraded":"2023-08-09T13:48:18.133469+0000","last_fullsized":"2023-08-09T13:48:18.133469+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:48:16.072766+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:48:16.072766+0000","last_clean_scrub_stamp":"2023-08-09T13:48:16.072766+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T15:00:43.613037+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.19","version":"0'0","reported_seq":9,"reported_epoch":73,"state":"active+clean","last_fresh":"2023-08-09T13:48:18.158353+0000","last_change":"2023-08-09T13:48:18.152221+0000","last_active":"2023-08-09T13:48:18.158353+0000","last_peered":"2023-08-09T13:48:18.158353+0000","last_clean":"2023-08-09T13:48:18.158353+0000","last_became_active":"2023-08-09T13:48:18.151934+0000","last_became_peered":"2023-08-09T13:48:18.151934+0000","last_unstale":"2023-08-09T13:48:18.158353+0000","last_undegraded":"2023-08-09T13:48:18.158353+0000","last_fullsized":"2023-08-09T13:48:18.158353+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:48:17.087857+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:48:17.087857+0000","last_clean_scrub_stamp":"2023-08-09T13:48:17.087857+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T23:00:27.746151+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.18","version":"0'0","reported_seq":11,"reported_epoch":73,"state":"active+clean","last_fresh":"2023-08-09T13:48:18.128990+0000","last_change":"2023-08-09T13:48:17.140981+0000","last_active":"2023-08-09T13:48:18.128990+0000","last_peered":"2023-08-09T13:48:18.128990+0000","last_clean":"2023-08-09T13:48:18.128990+0000","last_became_active":"2023-08-09T13:48:17.140712+0000","last_became_peered":"2023-08-09T13:48:17.140712+0000","last_unstale":"2023-08-09T13:48:18.128990+0000","last_undegraded":"2023-08-09T13:48:18.128990+0000","last_fullsized":"2023-08-09T13:48:18.128990+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:48:16.072766+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:48:16.072766+0000","last_clean_scrub_stamp":"2023-08-09T13:48:16.072766+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T18:57:01.535619+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.16","version":"0'0","reported_seq":9,"reported_epoch":73,"state":"active+clean","last_fresh":"2023-08-09T13:48:18.152546+0000","last_change":"2023-08-09T13:48:18.151658+0000","last_active":"2023-08-09T13:48:18.152546+0000","last_peered":"2023-08-09T13:48:18.152546+0000","last_clean":"2023-08-09T13:48:18.152546+0000","last_became_active":"2023-08-09T13:48:18.151238+0000","last_became_peered":"2023-08-09T13:48:18.151238+0000","last_unstale":"2023-08-09T13:48:18.152546+0000","last_undegraded":"2023-08-09T13:48:18.152546+0000","last_fullsized":"2023-08-09T13:48:18.152546+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:48:17.087857+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:48:17.087857+0000","last_clean_scrub_stamp":"2023-08-09T13:48:17.087857+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T16:56:02.522493+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"8.17","version":"0'0","reported_seq":11,"reported_epoch":73,"state":"active+clean","last_fresh":"2023-08-09T13:48:18.121930+0000","last_change":"2023-08-09T13:48:17.137380+0000","last_active":"2023-08-09T13:48:18.121930+0000","last_peered":"2023-08-09T13:48:18.121930+0000","last_clean":"2023-08-09T13:48:18.121930+0000","last_became_active":"2023-08-09T13:48:17.137062+0000","last_became_peered":"2023-08-09T13:48:17.137062+0000","last_unstale":"2023-08-09T13:48:18.121930+0000","last_undegraded":"2023-08-09T13:48:18.121930+0000","last_fullsized":"2023-08-09T13:48:18.121930+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:48:16.072766+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:48:16.072766+0000","last_clean_scrub_stamp":"2023-08-09T13:48:16.072766+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-11T01:26:58.358070+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.17","version":"0'0","reported_seq":9,"reported_epoch":73,"state":"active+clean","last_fresh":"2023-08-09T13:48:18.141090+0000","last_change":"2023-08-09T13:48:18.137992+0000","last_active":"2023-08-09T13:48:18.141090+0000","last_peered":"2023-08-09T13:48:18.141090+0000","last_clean":"2023-08-09T13:48:18.141090+0000","last_became_active":"2023-08-09T13:48:18.137621+0000","last_became_peered":"2023-08-09T13:48:18.137621+0000","last_unstale":"2023-08-09T13:48:18.141090+0000","last_undegraded":"2023-08-09T13:48:18.141090+0000","last_fullsized":"2023-08-09T13:48:18.141090+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:48:17.087857+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:48:17.087857+0000","last_clean_scrub_stamp":"2023-08-09T13:48:17.087857+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T18:36:12.306897+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.16","version":"0'0","reported_seq":11,"reported_epoch":73,"state":"active+clean","last_fresh":"2023-08-09T13:48:18.147086+0000","last_change":"2023-08-09T13:48:17.129879+0000","last_active":"2023-08-09T13:48:18.147086+0000","last_peered":"2023-08-09T13:48:18.147086+0000","last_clean":"2023-08-09T13:48:18.147086+0000","last_became_active":"2023-08-09T13:48:17.129599+0000","last_became_peered":"2023-08-09T13:48:17.129599+0000","last_unstale":"2023-08-09T13:48:18.147086+0000","last_undegraded":"2023-08-09T13:48:18.147086+0000","last_fullsized":"2023-08-09T13:48:18.147086+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:48:16.072766+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:48:16.072766+0000","last_clean_scrub_stamp":"2023-08-09T13:48:16.072766+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T18:30:53.375445+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.14","version":"0'0","reported_seq":9,"reported_epoch":73,"state":"active+clean","last_fresh":"2023-08-09T13:48:18.150408+0000","last_change":"2023-08-09T13:48:18.145545+0000","last_active":"2023-08-09T13:48:18.150408+0000","last_peered":"2023-08-09T13:48:18.150408+0000","last_clean":"2023-08-09T13:48:18.150408+0000","last_became_active":"2023-08-09T13:48:18.144442+0000","last_became_peered":"2023-08-09T13:48:18.144442+0000","last_unstale":"2023-08-09T13:48:18.150408+0000","last_undegraded":"2023-08-09T13:48:18.150408+0000","last_fullsized":"2023-08-09T13:48:18.150408+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:48:17.087857+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:48:17.087857+0000","last_clean_scrub_stamp":"2023-08-09T13:48:17.087857+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-11T00:04:12.888732+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.15","version":"73'3","reported_seq":14,"reported_epoch":73,"state":"active+clean","last_fresh":"2023-08-09T13:48:18.173971+0000","last_change":"2023-08-09T13:48:17.133125+0000","last_active":"2023-08-09T13:48:18.173971+0000","last_peered":"2023-08-09T13:48:18.173971+0000","last_clean":"2023-08-09T13:48:18.173971+0000","last_became_active":"2023-08-09T13:48:17.132824+0000","last_became_peered":"2023-08-09T13:48:17.132824+0000","last_unstale":"2023-08-09T13:48:18.173971+0000","last_undegraded":"2023-08-09T13:48:18.173971+0000","last_fullsized":"2023-08-09T13:48:18.173971+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:48:16.072766+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:48:16.072766+0000","last_clean_scrub_stamp":"2023-08-09T13:48:16.072766+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T18:21:07.867154+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.15","version":"0'0","reported_seq":9,"reported_epoch":73,"state":"active+clean","last_fresh":"2023-08-09T13:48:18.137274+0000","last_change":"2023-08-09T13:48:18.128249+0000","last_active":"2023-08-09T13:48:18.137274+0000","last_peered":"2023-08-09T13:48:18.137274+0000","last_clean":"2023-08-09T13:48:18.137274+0000","last_became_active":"2023-08-09T13:48:18.127649+0000","last_became_peered":"2023-08-09T13:48:18.127649+0000","last_unstale":"2023-08-09T13:48:18.137274+0000","last_undegraded":"2023-08-09T13:48:18.137274+0000","last_fullsized":"2023-08-09T13:48:18.137274+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:48:17.087857+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:48:17.087857+0000","last_clean_scrub_stamp":"2023-08-09T13:48:17.087857+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-11T00:04:40.558145+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.14","version":"73'6","reported_seq":17,"reported_epoch":73,"state":"active+clean","last_fresh":"2023-08-09T13:48:18.176391+0000","last_change":"2023-08-09T13:48:17.138705+0000","last_active":"2023-08-09T13:48:18.176391+0000","last_peered":"2023-08-09T13:48:18.176391+0000","last_clean":"2023-08-09T13:48:18.176391+0000","last_became_active":"2023-08-09T13:48:17.138297+0000","last_became_peered":"2023-08-09T13:48:17.138297+0000","last_unstale":"2023-08-09T13:48:18.176391+0000","last_undegraded":"2023-08-09T13:48:18.176391+0000","last_fullsized":"2023-08-09T13:48:18.176391+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:48:16.072766+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:48:16.072766+0000","last_clean_scrub_stamp":"2023-08-09T13:48:16.072766+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T15:33:28.704733+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.12","version":"0'0","reported_seq":9,"reported_epoch":73,"state":"active+clean","last_fresh":"2023-08-09T13:48:18.151949+0000","last_change":"2023-08-09T13:48:18.149462+0000","last_active":"2023-08-09T13:48:18.151949+0000","last_peered":"2023-08-09T13:48:18.151949+0000","last_clean":"2023-08-09T13:48:18.151949+0000","last_became_active":"2023-08-09T13:48:18.149210+0000","last_became_peered":"2023-08-09T13:48:18.149210+0000","last_unstale":"2023-08-09T13:48:18.151949+0000","last_undegraded":"2023-08-09T13:48:18.151949+0000","last_fullsized":"2023-08-09T13:48:18.151949+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:48:17.087857+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:48:17.087857+0000","last_clean_scrub_stamp":"2023-08-09T13:48:17.087857+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T20:59:54.537402+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.13","version":"73'2","reported_seq":13,"reported_epoch":73,"state":"active+clean","last_fresh":"2023-08-09T13:48:18.173271+0000","last_change":"2023-08-09T13:48:17.129248+0000","last_active":"2023-08-09T13:48:18.173271+0000","last_peered":"2023-08-09T13:48:18.173271+0000","last_clean":"2023-08-09T13:48:18.173271+0000","last_became_active":"2023-08-09T13:48:17.128954+0000","last_became_peered":"2023-08-09T13:48:17.128954+0000","last_unstale":"2023-08-09T13:48:18.173271+0000","last_undegraded":"2023-08-09T13:48:18.173271+0000","last_fullsized":"2023-08-09T13:48:18.173271+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:48:16.072766+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:48:16.072766+0000","last_clean_scrub_stamp":"2023-08-09T13:48:16.072766+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T17:25:19.626774+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.13","version":"0'0","reported_seq":9,"reported_epoch":73,"state":"active+clean","last_fresh":"2023-08-09T13:48:18.136870+0000","last_change":"2023-08-09T13:48:18.133565+0000","last_active":"2023-08-09T13:48:18.136870+0000","last_peered":"2023-08-09T13:48:18.136870+0000","last_clean":"2023-08-09T13:48:18.136870+0000","last_became_active":"2023-08-09T13:48:18.133122+0000","last_became_peered":"2023-08-09T13:48:18.133122+0000","last_unstale":"2023-08-09T13:48:18.136870+0000","last_undegraded":"2023-08-09T13:48:18.136870+0000","last_fullsized":"2023-08-09T13:48:18.136870+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:48:17.087857+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:48:17.087857+0000","last_clean_scrub_stamp":"2023-08-09T13:48:17.087857+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-11T01:19:52.281855+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.12","version":"0'0","reported_seq":11,"reported_epoch":73,"state":"active+clean","last_fresh":"2023-08-09T13:48:18.119303+0000","last_change":"2023-08-09T13:48:17.130854+0000","last_active":"2023-08-09T13:48:18.119303+0000","last_peered":"2023-08-09T13:48:18.119303+0000","last_clean":"2023-08-09T13:48:18.119303+0000","last_became_active":"2023-08-09T13:48:17.130566+0000","last_became_peered":"2023-08-09T13:48:17.130566+0000","last_unstale":"2023-08-09T13:48:18.119303+0000","last_undegraded":"2023-08-09T13:48:18.119303+0000","last_fullsized":"2023-08-09T13:48:18.119303+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:48:16.072766+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:48:16.072766+0000","last_clean_scrub_stamp":"2023-08-09T13:48:16.072766+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T22:03:03.015035+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.10","version":"0'0","reported_seq":9,"reported_epoch":73,"state":"active+clean","last_fresh":"2023-08-09T13:48:18.144369+0000","last_change":"2023-08-09T13:48:18.142388+0000","last_active":"2023-08-09T13:48:18.144369+0000","last_peered":"2023-08-09T13:48:18.144369+0000","last_clean":"2023-08-09T13:48:18.144369+0000","last_became_active":"2023-08-09T13:48:18.141862+0000","last_became_peered":"2023-08-09T13:48:18.141862+0000","last_unstale":"2023-08-09T13:48:18.144369+0000","last_undegraded":"2023-08-09T13:48:18.144369+0000","last_fullsized":"2023-08-09T13:48:18.144369+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:48:17.087857+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:48:17.087857+0000","last_clean_scrub_stamp":"2023-08-09T13:48:17.087857+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-11T01:20:29.944656+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"8.11","version":"0'0","reported_seq":11,"reported_epoch":73,"state":"active+clean","last_fresh":"2023-08-09T13:48:18.133797+0000","last_change":"2023-08-09T13:48:17.133973+0000","last_active":"2023-08-09T13:48:18.133797+0000","last_peered":"2023-08-09T13:48:18.133797+0000","last_clean":"2023-08-09T13:48:18.133797+0000","last_became_active":"2023-08-09T13:48:17.133679+0000","last_became_peered":"2023-08-09T13:48:17.133679+0000","last_unstale":"2023-08-09T13:48:18.133797+0000","last_undegraded":"2023-08-09T13:48:18.133797+0000","last_fullsized":"2023-08-09T13:48:18.133797+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:48:16.072766+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:48:16.072766+0000","last_clean_scrub_stamp":"2023-08-09T13:48:16.072766+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T19:37:33.525527+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.11","version":"0'0","reported_seq":9,"reported_epoch":73,"state":"active+clean","last_fresh":"2023-08-09T13:48:18.155678+0000","last_change":"2023-08-09T13:48:18.154453+0000","last_active":"2023-08-09T13:48:18.155678+0000","last_peered":"2023-08-09T13:48:18.155678+0000","last_clean":"2023-08-09T13:48:18.155678+0000","last_became_active":"2023-08-09T13:48:18.154112+0000","last_became_peered":"2023-08-09T13:48:18.154112+0000","last_unstale":"2023-08-09T13:48:18.155678+0000","last_undegraded":"2023-08-09T13:48:18.155678+0000","last_fullsized":"2023-08-09T13:48:18.155678+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:48:17.087857+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:48:17.087857+0000","last_clean_scrub_stamp":"2023-08-09T13:48:17.087857+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T21:14:02.421900+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.10","version":"73'8","reported_seq":19,"reported_epoch":73,"state":"active+clean","last_fresh":"2023-08-09T13:48:18.173716+0000","last_change":"2023-08-09T13:48:17.134689+0000","last_active":"2023-08-09T13:48:18.173716+0000","last_peered":"2023-08-09T13:48:18.173716+0000","last_clean":"2023-08-09T13:48:18.173716+0000","last_became_active":"2023-08-09T13:48:17.134378+0000","last_became_peered":"2023-08-09T13:48:17.134378+0000","last_unstale":"2023-08-09T13:48:18.173716+0000","last_undegraded":"2023-08-09T13:48:18.173716+0000","last_fullsized":"2023-08-09T13:48:18.173716+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:48:16.072766+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:48:16.072766+0000","last_clean_scrub_stamp":"2023-08-09T13:48:16.072766+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T21:29:33.710061+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.6","version":"0'0","reported_seq":9,"reported_epoch":73,"state":"active+clean","last_fresh":"2023-08-09T13:48:18.151771+0000","last_change":"2023-08-09T13:48:18.146139+0000","last_active":"2023-08-09T13:48:18.151771+0000","last_peered":"2023-08-09T13:48:18.151771+0000","last_clean":"2023-08-09T13:48:18.151771+0000","last_became_active":"2023-08-09T13:48:18.145842+0000","last_became_peered":"2023-08-09T13:48:18.145842+0000","last_unstale":"2023-08-09T13:48:18.151771+0000","last_undegraded":"2023-08-09T13:48:18.151771+0000","last_fullsized":"2023-08-09T13:48:18.151771+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:48:17.087857+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:48:17.087857+0000","last_clean_scrub_stamp":"2023-08-09T13:48:17.087857+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T22:21:27.281375+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.7","version":"73'2","reported_seq":13,"reported_epoch":73,"state":"active+clean","last_fresh":"2023-08-09T13:48:18.175555+0000","last_change":"2023-08-09T13:48:17.135531+0000","last_active":"2023-08-09T13:48:18.175555+0000","last_peered":"2023-08-09T13:48:18.175555+0000","last_clean":"2023-08-09T13:48:18.175555+0000","last_became_active":"2023-08-09T13:48:17.135112+0000","last_became_peered":"2023-08-09T13:48:17.135112+0000","last_unstale":"2023-08-09T13:48:18.175555+0000","last_undegraded":"2023-08-09T13:48:18.175555+0000","last_fullsized":"2023-08-09T13:48:18.175555+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:48:16.072766+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:48:16.072766+0000","last_clean_scrub_stamp":"2023-08-09T13:48:16.072766+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T14:17:18.474299+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.7","version":"0'0","reported_seq":9,"reported_epoch":73,"state":"active+clean","last_fresh":"2023-08-09T13:48:18.141348+0000","last_change":"2023-08-09T13:48:18.139622+0000","last_active":"2023-08-09T13:48:18.141348+0000","last_peered":"2023-08-09T13:48:18.141348+0000","last_clean":"2023-08-09T13:48:18.141348+0000","last_became_active":"2023-08-09T13:48:18.139332+0000","last_became_peered":"2023-08-09T13:48:18.139332+0000","last_unstale":"2023-08-09T13:48:18.141348+0000","last_undegraded":"2023-08-09T13:48:18.141348+0000","last_fullsized":"2023-08-09T13:48:18.141348+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:48:17.087857+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:48:17.087857+0000","last_clean_scrub_stamp":"2023-08-09T13:48:17.087857+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T20:42:45.679132+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.6","version":"0'0","reported_seq":11,"reported_epoch":73,"state":"active+clean","last_fresh":"2023-08-09T13:48:18.124688+0000","last_change":"2023-08-09T13:48:17.135522+0000","last_active":"2023-08-09T13:48:18.124688+0000","last_peered":"2023-08-09T13:48:18.124688+0000","last_clean":"2023-08-09T13:48:18.124688+0000","last_became_active":"2023-08-09T13:48:17.135229+0000","last_became_peered":"2023-08-09T13:48:17.135229+0000","last_unstale":"2023-08-09T13:48:18.124688+0000","last_undegraded":"2023-08-09T13:48:18.124688+0000","last_fullsized":"2023-08-09T13:48:18.124688+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:48:16.072766+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:48:16.072766+0000","last_clean_scrub_stamp":"2023-08-09T13:48:16.072766+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T16:56:01.507402+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.4","version":"0'0","reported_seq":9,"reported_epoch":73,"state":"active+clean","last_fresh":"2023-08-09T13:48:18.150431+0000","last_change":"2023-08-09T13:48:18.146073+0000","last_active":"2023-08-09T13:48:18.150431+0000","last_peered":"2023-08-09T13:48:18.150431+0000","last_clean":"2023-08-09T13:48:18.150431+0000","last_became_active":"2023-08-09T13:48:18.145037+0000","last_became_peered":"2023-08-09T13:48:18.145037+0000","last_unstale":"2023-08-09T13:48:18.150431+0000","last_undegraded":"2023-08-09T13:48:18.150431+0000","last_fullsized":"2023-08-09T13:48:18.150431+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:48:17.087857+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:48:17.087857+0000","last_clean_scrub_stamp":"2023-08-09T13:48:17.087857+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T16:40:28.280266+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.5","version":"0'0","reported_seq":11,"reported_epoch":73,"state":"active+clean","last_fresh":"2023-08-09T13:48:18.119655+0000","last_change":"2023-08-09T13:48:17.136747+0000","last_active":"2023-08-09T13:48:18.119655+0000","last_peered":"2023-08-09T13:48:18.119655+0000","last_clean":"2023-08-09T13:48:18.119655+0000","last_became_active":"2023-08-09T13:48:17.136353+0000","last_became_peered":"2023-08-09T13:48:17.136353+0000","last_unstale":"2023-08-09T13:48:18.119655+0000","last_undegraded":"2023-08-09T13:48:18.119655+0000","last_fullsized":"2023-08-09T13:48:18.119655+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:48:16.072766+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:48:16.072766+0000","last_clean_scrub_stamp":"2023-08-09T13:48:16.072766+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T22:16:07.036190+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.5","version":"0'0","reported_seq":9,"reported_epoch":73,"state":"active+clean","last_fresh":"2023-08-09T13:48:18.155580+0000","last_change":"2023-08-09T13:48:18.142737+0000","last_active":"2023-08-09T13:48:18.155580+0000","last_peered":"2023-08-09T13:48:18.155580+0000","last_clean":"2023-08-09T13:48:18.155580+0000","last_became_active":"2023-08-09T13:48:18.141654+0000","last_became_peered":"2023-08-09T13:48:18.141654+0000","last_unstale":"2023-08-09T13:48:18.155580+0000","last_undegraded":"2023-08-09T13:48:18.155580+0000","last_fullsized":"2023-08-09T13:48:18.155580+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:48:17.087857+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:48:17.087857+0000","last_clean_scrub_stamp":"2023-08-09T13:48:17.087857+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T20:59:41.322587+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.4","version":"0'0","reported_seq":11,"reported_epoch":73,"state":"active+clean","last_fresh":"2023-08-09T13:48:18.136486+0000","last_change":"2023-08-09T13:48:17.139103+0000","last_active":"2023-08-09T13:48:18.136486+0000","last_peered":"2023-08-09T13:48:18.136486+0000","last_clean":"2023-08-09T13:48:18.136486+0000","last_became_active":"2023-08-09T13:48:17.138848+0000","last_became_peered":"2023-08-09T13:48:17.138848+0000","last_unstale":"2023-08-09T13:48:18.136486+0000","last_undegraded":"2023-08-09T13:48:18.136486+0000","last_fullsized":"2023-08-09T13:48:18.136486+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:48:16.072766+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:48:16.072766+0000","last_clean_scrub_stamp":"2023-08-09T13:48:16.072766+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T18:40:08.613673+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.2","version":"0'0","reported_seq":9,"reported_epoch":73,"state":"active+clean","last_fresh":"2023-08-09T13:48:18.160553+0000","last_change":"2023-08-09T13:48:18.158030+0000","last_active":"2023-08-09T13:48:18.160553+0000","last_peered":"2023-08-09T13:48:18.160553+0000","last_clean":"2023-08-09T13:48:18.160553+0000","last_became_active":"2023-08-09T13:48:18.157119+0000","last_became_peered":"2023-08-09T13:48:18.157119+0000","last_unstale":"2023-08-09T13:48:18.160553+0000","last_undegraded":"2023-08-09T13:48:18.160553+0000","last_fullsized":"2023-08-09T13:48:18.160553+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:48:17.087857+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:48:17.087857+0000","last_clean_scrub_stamp":"2023-08-09T13:48:17.087857+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T16:11:34.105483+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.3","version":"0'0","reported_seq":11,"reported_epoch":73,"state":"active+clean","last_fresh":"2023-08-09T13:48:18.111274+0000","last_change":"2023-08-09T13:48:17.122100+0000","last_active":"2023-08-09T13:48:18.111274+0000","last_peered":"2023-08-09T13:48:18.111274+0000","last_clean":"2023-08-09T13:48:18.111274+0000","last_became_active":"2023-08-09T13:48:17.121794+0000","last_became_peered":"2023-08-09T13:48:17.121794+0000","last_unstale":"2023-08-09T13:48:18.111274+0000","last_undegraded":"2023-08-09T13:48:18.111274+0000","last_fullsized":"2023-08-09T13:48:18.111274+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:48:16.072766+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:48:16.072766+0000","last_clean_scrub_stamp":"2023-08-09T13:48:16.072766+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T23:45:17.019687+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.3","version":"0'0","reported_seq":9,"reported_epoch":73,"state":"active+clean","last_fresh":"2023-08-09T13:48:18.155656+0000","last_change":"2023-08-09T13:48:18.154534+0000","last_active":"2023-08-09T13:48:18.155656+0000","last_peered":"2023-08-09T13:48:18.155656+0000","last_clean":"2023-08-09T13:48:18.155656+0000","last_became_active":"2023-08-09T13:48:18.154225+0000","last_became_peered":"2023-08-09T13:48:18.154225+0000","last_unstale":"2023-08-09T13:48:18.155656+0000","last_undegraded":"2023-08-09T13:48:18.155656+0000","last_fullsized":"2023-08-09T13:48:18.155656+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:48:17.087857+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:48:17.087857+0000","last_clean_scrub_stamp":"2023-08-09T13:48:17.087857+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T22:39:27.852461+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.2","version":"0'0","reported_seq":11,"reported_epoch":73,"state":"active+clean","last_fresh":"2023-08-09T13:48:18.141034+0000","last_change":"2023-08-09T13:48:17.139591+0000","last_active":"2023-08-09T13:48:18.141034+0000","last_peered":"2023-08-09T13:48:18.141034+0000","last_clean":"2023-08-09T13:48:18.141034+0000","last_became_active":"2023-08-09T13:48:17.139320+0000","last_became_peered":"2023-08-09T13:48:17.139320+0000","last_unstale":"2023-08-09T13:48:18.141034+0000","last_undegraded":"2023-08-09T13:48:18.141034+0000","last_fullsized":"2023-08-09T13:48:18.141034+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:48:16.072766+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:48:16.072766+0000","last_clean_scrub_stamp":"2023-08-09T13:48:16.072766+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-11T01:15:12.569060+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.0","version":"0'0","reported_seq":9,"reported_epoch":73,"state":"active+clean","last_fresh":"2023-08-09T13:48:18.137937+0000","last_change":"2023-08-09T13:48:18.127709+0000","last_active":"2023-08-09T13:48:18.137937+0000","last_peered":"2023-08-09T13:48:18.137937+0000","last_clean":"2023-08-09T13:48:18.137937+0000","last_became_active":"2023-08-09T13:48:18.126930+0000","last_became_peered":"2023-08-09T13:48:18.126930+0000","last_unstale":"2023-08-09T13:48:18.137937+0000","last_undegraded":"2023-08-09T13:48:18.137937+0000","last_fullsized":"2023-08-09T13:48:18.137937+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:48:17.087857+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:48:17.087857+0000","last_clean_scrub_stamp":"2023-08-09T13:48:17.087857+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-11T00:13:56.323548+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.1","version":"0'0","reported_seq":11,"reported_epoch":73,"state":"active+clean","last_fresh":"2023-08-09T13:48:18.110950+0000","last_change":"2023-08-09T13:48:17.125898+0000","last_active":"2023-08-09T13:48:18.110950+0000","last_peered":"2023-08-09T13:48:18.110950+0000","last_clean":"2023-08-09T13:48:18.110950+0000","last_became_active":"2023-08-09T13:48:17.125609+0000","last_became_peered":"2023-08-09T13:48:17.125609+0000","last_unstale":"2023-08-09T13:48:18.110950+0000","last_undegraded":"2023-08-09T13:48:18.110950+0000","last_fullsized":"2023-08-09T13:48:18.110950+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:48:16.072766+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:48:16.072766+0000","last_clean_scrub_stamp":"2023-08-09T13:48:16.072766+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T19:04:18.439807+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.1","version":"0'0","reported_seq":9,"reported_epoch":73,"state":"active+clean","last_fresh":"2023-08-09T13:48:18.159634+0000","last_change":"2023-08-09T13:48:18.156163+0000","last_active":"2023-08-09T13:48:18.159634+0000","last_peered":"2023-08-09T13:48:18.159634+0000","last_clean":"2023-08-09T13:48:18.159634+0000","last_became_active":"2023-08-09T13:48:18.155181+0000","last_became_peered":"2023-08-09T13:48:18.155181+0000","last_unstale":"2023-08-09T13:48:18.159634+0000","last_undegraded":"2023-08-09T13:48:18.159634+0000","last_fullsized":"2023-08-09T13:48:18.159634+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:48:17.087857+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:48:17.087857+0000","last_clean_scrub_stamp":"2023-08-09T13:48:17.087857+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-11T01:14:04.304684+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.0","version":"0'0","reported_seq":11,"reported_epoch":73,"state":"active+clean","last_fresh":"2023-08-09T13:48:18.146883+0000","last_change":"2023-08-09T13:48:17.134067+0000","last_active":"2023-08-09T13:48:18.146883+0000","last_peered":"2023-08-09T13:48:18.146883+0000","last_clean":"2023-08-09T13:48:18.146883+0000","last_became_active":"2023-08-09T13:48:17.133792+0000","last_became_peered":"2023-08-09T13:48:17.133792+0000","last_unstale":"2023-08-09T13:48:18.146883+0000","last_undegraded":"2023-08-09T13:48:18.146883+0000","last_fullsized":"2023-08-09T13:48:18.146883+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:48:16.072766+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:48:16.072766+0000","last_clean_scrub_stamp":"2023-08-09T13:48:16.072766+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T20:10:04.314162+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.8","version":"0'0","reported_seq":9,"reported_epoch":73,"state":"active+clean","last_fresh":"2023-08-09T13:48:18.160521+0000","last_change":"2023-08-09T13:48:18.157902+0000","last_active":"2023-08-09T13:48:18.160521+0000","last_peered":"2023-08-09T13:48:18.160521+0000","last_clean":"2023-08-09T13:48:18.160521+0000","last_became_active":"2023-08-09T13:48:18.157606+0000","last_became_peered":"2023-08-09T13:48:18.157606+0000","last_unstale":"2023-08-09T13:48:18.160521+0000","last_undegraded":"2023-08-09T13:48:18.160521+0000","last_fullsized":"2023-08-09T13:48:18.160521+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:48:17.087857+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:48:17.087857+0000","last_clean_scrub_stamp":"2023-08-09T13:48:17.087857+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T17:18:22.010079+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.9","version":"0'0","reported_seq":11,"reported_epoch":73,"state":"active+clean","last_fresh":"2023-08-09T13:48:18.116903+0000","last_change":"2023-08-09T13:48:17.136780+0000","last_active":"2023-08-09T13:48:18.116903+0000","last_peered":"2023-08-09T13:48:18.116903+0000","last_clean":"2023-08-09T13:48:18.116903+0000","last_became_active":"2023-08-09T13:48:17.136480+0000","last_became_peered":"2023-08-09T13:48:17.136480+0000","last_unstale":"2023-08-09T13:48:18.116903+0000","last_undegraded":"2023-08-09T13:48:18.116903+0000","last_fullsized":"2023-08-09T13:48:18.116903+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:48:16.072766+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:48:16.072766+0000","last_clean_scrub_stamp":"2023-08-09T13:48:16.072766+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T14:33:01.425713+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":120,"reported_epoch":73,"state":"active+clean","last_fresh":"2023-08-09T13:48:18.115289+0000","last_change":"2023-08-09T13:33:56.401406+0000","last_active":"2023-08-09T13:48:18.115289+0000","last_peered":"2023-08-09T13:48:18.115289+0000","last_clean":"2023-08-09T13:48:18.115289+0000","last_became_active":"2023-08-09T13:33:56.400487+0000","last_became_peered":"2023-08-09T13:33:56.400487+0000","last_unstale":"2023-08-09T13:48:18.115289+0000","last_undegraded":"2023-08-09T13:48:18.115289+0000","last_fullsized":"2023-08-09T13:48:18.115289+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:33:54.845869+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:33:54.845869+0000","last_clean_scrub_stamp":"2023-08-09T13:33:54.845869+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T23:29:59.861309+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.9","version":"0'0","reported_seq":9,"reported_epoch":73,"state":"active+clean","last_fresh":"2023-08-09T13:48:18.155107+0000","last_change":"2023-08-09T13:48:18.140747+0000","last_active":"2023-08-09T13:48:18.155107+0000","last_peered":"2023-08-09T13:48:18.155107+0000","last_clean":"2023-08-09T13:48:18.155107+0000","last_became_active":"2023-08-09T13:48:18.140237+0000","last_became_peered":"2023-08-09T13:48:18.140237+0000","last_unstale":"2023-08-09T13:48:18.155107+0000","last_undegraded":"2023-08-09T13:48:18.155107+0000","last_fullsized":"2023-08-09T13:48:18.155107+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:48:17.087857+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:48:17.087857+0000","last_clean_scrub_stamp":"2023-08-09T13:48:17.087857+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T23:12:15.650700+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.8","version":"0'0","reported_seq":11,"reported_epoch":73,"state":"active+clean","last_fresh":"2023-08-09T13:48:18.134039+0000","last_change":"2023-08-09T13:48:17.139548+0000","last_active":"2023-08-09T13:48:18.134039+0000","last_peered":"2023-08-09T13:48:18.134039+0000","last_clean":"2023-08-09T13:48:18.134039+0000","last_became_active":"2023-08-09T13:48:17.139270+0000","last_became_peered":"2023-08-09T13:48:17.139270+0000","last_unstale":"2023-08-09T13:48:18.134039+0000","last_undegraded":"2023-08-09T13:48:18.134039+0000","last_fullsized":"2023-08-09T13:48:18.134039+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:48:16.072766+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:48:16.072766+0000","last_clean_scrub_stamp":"2023-08-09T13:48:16.072766+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T21:41:50.040105+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":119,"reported_epoch":73,"state":"active+clean","last_fresh":"2023-08-09T13:48:18.133585+0000","last_change":"2023-08-09T13:33:56.401123+0000","last_active":"2023-08-09T13:48:18.133585+0000","last_peered":"2023-08-09T13:48:18.133585+0000","last_clean":"2023-08-09T13:48:18.133585+0000","last_became_active":"2023-08-09T13:33:56.399933+0000","last_became_peered":"2023-08-09T13:33:56.399933+0000","last_unstale":"2023-08-09T13:48:18.133585+0000","last_undegraded":"2023-08-09T13:48:18.133585+0000","last_fullsized":"2023-08-09T13:48:18.133585+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:33:54.845869+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:33:54.845869+0000","last_clean_scrub_stamp":"2023-08-09T13:33:54.845869+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-11T00:44:56.386308+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.a","version":"0'0","reported_seq":9,"reported_epoch":73,"state":"active+clean","last_fresh":"2023-08-09T13:48:18.151929+0000","last_change":"2023-08-09T13:48:18.148933+0000","last_active":"2023-08-09T13:48:18.151929+0000","last_peered":"2023-08-09T13:48:18.151929+0000","last_clean":"2023-08-09T13:48:18.151929+0000","last_became_active":"2023-08-09T13:48:18.148669+0000","last_became_peered":"2023-08-09T13:48:18.148669+0000","last_unstale":"2023-08-09T13:48:18.151929+0000","last_undegraded":"2023-08-09T13:48:18.151929+0000","last_fullsized":"2023-08-09T13:48:18.151929+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:48:17.087857+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:48:17.087857+0000","last_clean_scrub_stamp":"2023-08-09T13:48:17.087857+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T19:22:19.856884+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.b","version":"73'2","reported_seq":13,"reported_epoch":73,"state":"active+clean","last_fresh":"2023-08-09T13:48:18.172999+0000","last_change":"2023-08-09T13:48:17.136357+0000","last_active":"2023-08-09T13:48:18.172999+0000","last_peered":"2023-08-09T13:48:18.172999+0000","last_clean":"2023-08-09T13:48:18.172999+0000","last_became_active":"2023-08-09T13:48:17.136065+0000","last_became_peered":"2023-08-09T13:48:17.136065+0000","last_unstale":"2023-08-09T13:48:18.172999+0000","last_undegraded":"2023-08-09T13:48:18.172999+0000","last_fullsized":"2023-08-09T13:48:18.172999+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:48:16.072766+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:48:16.072766+0000","last_clean_scrub_stamp":"2023-08-09T13:48:16.072766+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T18:36:41.123187+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.2","version":"0'0","reported_seq":119,"reported_epoch":73,"state":"active+clean","last_fresh":"2023-08-09T13:48:18.115983+0000","last_change":"2023-08-09T13:33:56.402794+0000","last_active":"2023-08-09T13:48:18.115983+0000","last_peered":"2023-08-09T13:48:18.115983+0000","last_clean":"2023-08-09T13:48:18.115983+0000","last_became_active":"2023-08-09T13:33:56.402038+0000","last_became_peered":"2023-08-09T13:33:56.402038+0000","last_unstale":"2023-08-09T13:48:18.115983+0000","last_undegraded":"2023-08-09T13:48:18.115983+0000","last_fullsized":"2023-08-09T13:48:18.115983+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:33:54.845869+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:33:54.845869+0000","last_clean_scrub_stamp":"2023-08-09T13:33:54.845869+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T18:58:03.028401+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.b","version":"0'0","reported_seq":9,"reported_epoch":73,"state":"active+clean","last_fresh":"2023-08-09T13:48:18.158680+0000","last_change":"2023-08-09T13:48:18.144582+0000","last_active":"2023-08-09T13:48:18.158680+0000","last_peered":"2023-08-09T13:48:18.158680+0000","last_clean":"2023-08-09T13:48:18.158680+0000","last_became_active":"2023-08-09T13:48:18.144257+0000","last_became_peered":"2023-08-09T13:48:18.144257+0000","last_unstale":"2023-08-09T13:48:18.158680+0000","last_undegraded":"2023-08-09T13:48:18.158680+0000","last_fullsized":"2023-08-09T13:48:18.158680+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:48:17.087857+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:48:17.087857+0000","last_clean_scrub_stamp":"2023-08-09T13:48:17.087857+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T20:38:23.607892+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.a","version":"0'0","reported_seq":11,"reported_epoch":73,"state":"active+clean","last_fresh":"2023-08-09T13:48:18.111024+0000","last_change":"2023-08-09T13:48:17.128719+0000","last_active":"2023-08-09T13:48:18.111024+0000","last_peered":"2023-08-09T13:48:18.111024+0000","last_clean":"2023-08-09T13:48:18.111024+0000","last_became_active":"2023-08-09T13:48:17.128447+0000","last_became_peered":"2023-08-09T13:48:17.128447+0000","last_unstale":"2023-08-09T13:48:18.111024+0000","last_undegraded":"2023-08-09T13:48:18.111024+0000","last_fullsized":"2023-08-09T13:48:18.111024+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:48:16.072766+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:48:16.072766+0000","last_clean_scrub_stamp":"2023-08-09T13:48:16.072766+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T19:42:11.460732+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.3","version":"0'0","reported_seq":121,"reported_epoch":73,"state":"active+clean","last_fresh":"2023-08-09T13:48:18.111186+0000","last_change":"2023-08-09T13:33:55.886749+0000","last_active":"2023-08-09T13:48:18.111186+0000","last_peered":"2023-08-09T13:48:18.111186+0000","last_clean":"2023-08-09T13:48:18.111186+0000","last_became_active":"2023-08-09T13:33:55.886116+0000","last_became_peered":"2023-08-09T13:33:55.886116+0000","last_unstale":"2023-08-09T13:48:18.111186+0000","last_undegraded":"2023-08-09T13:48:18.111186+0000","last_fullsized":"2023-08-09T13:48:18.111186+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:33:54.845869+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:33:54.845869+0000","last_clean_scrub_stamp":"2023-08-09T13:33:54.845869+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T18:20:02.086899+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"62'139","reported_seq":348,"reported_epoch":73,"state":"active+clean","last_fresh":"2023-08-09T13:48:18.114880+0000","last_change":"2023-08-09T13:33:55.886345+0000","last_active":"2023-08-09T13:48:18.114880+0000","last_peered":"2023-08-09T13:48:18.114880+0000","last_clean":"2023-08-09T13:48:18.114880+0000","last_became_active":"2023-08-09T13:33:55.885781+0000","last_became_peered":"2023-08-09T13:33:55.885781+0000","last_unstale":"2023-08-09T13:48:18.114880+0000","last_undegraded":"2023-08-09T13:48:18.114880+0000","last_fullsized":"2023-08-09T13:48:18.114880+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:33:54.845869+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:33:54.845869+0000","last_clean_scrub_stamp":"2023-08-09T13:33:54.845869+0000","objects_scrubbed":0,"log_size":139,"log_dups_size":0,"ondisk_log_size":139,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T19:19:10.686060+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":308,"num_read_kb":512,"num_write":199,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.c","version":"0'0","reported_seq":9,"reported_epoch":73,"state":"active+clean","last_fresh":"2023-08-09T13:48:18.156594+0000","last_change":"2023-08-09T13:48:18.150584+0000","last_active":"2023-08-09T13:48:18.156594+0000","last_peered":"2023-08-09T13:48:18.156594+0000","last_clean":"2023-08-09T13:48:18.156594+0000","last_became_active":"2023-08-09T13:48:18.150254+0000","last_became_peered":"2023-08-09T13:48:18.150254+0000","last_unstale":"2023-08-09T13:48:18.156594+0000","last_undegraded":"2023-08-09T13:48:18.156594+0000","last_fullsized":"2023-08-09T13:48:18.156594+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:48:17.087857+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:48:17.087857+0000","last_clean_scrub_stamp":"2023-08-09T13:48:17.087857+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T21:07:01.343134+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.d","version":"73'1","reported_seq":12,"reported_epoch":73,"state":"active+clean","last_fresh":"2023-08-09T13:48:18.173679+0000","last_change":"2023-08-09T13:48:17.134706+0000","last_active":"2023-08-09T13:48:18.173679+0000","last_peered":"2023-08-09T13:48:18.173679+0000","last_clean":"2023-08-09T13:48:18.173679+0000","last_became_active":"2023-08-09T13:48:17.134427+0000","last_became_peered":"2023-08-09T13:48:17.134427+0000","last_unstale":"2023-08-09T13:48:18.173679+0000","last_undegraded":"2023-08-09T13:48:18.173679+0000","last_fullsized":"2023-08-09T13:48:18.173679+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:48:16.072766+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:48:16.072766+0000","last_clean_scrub_stamp":"2023-08-09T13:48:16.072766+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T23:23:54.631430+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":127,"reported_epoch":73,"state":"active+clean","last_fresh":"2023-08-09T13:48:18.146475+0000","last_change":"2023-08-09T13:33:55.889565+0000","last_active":"2023-08-09T13:48:18.146475+0000","last_peered":"2023-08-09T13:48:18.146475+0000","last_clean":"2023-08-09T13:48:18.146475+0000","last_became_active":"2023-08-09T13:33:55.889164+0000","last_became_peered":"2023-08-09T13:33:55.889164+0000","last_unstale":"2023-08-09T13:48:18.146475+0000","last_undegraded":"2023-08-09T13:48:18.146475+0000","last_fullsized":"2023-08-09T13:48:18.146475+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:33:54.845869+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:33:54.845869+0000","last_clean_scrub_stamp":"2023-08-09T13:33:54.845869+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T21:05:04.235125+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.d","version":"0'0","reported_seq":9,"reported_epoch":73,"state":"active+clean","last_fresh":"2023-08-09T13:48:18.140977+0000","last_change":"2023-08-09T13:48:18.137115+0000","last_active":"2023-08-09T13:48:18.140977+0000","last_peered":"2023-08-09T13:48:18.140977+0000","last_clean":"2023-08-09T13:48:18.140977+0000","last_became_active":"2023-08-09T13:48:18.136785+0000","last_became_peered":"2023-08-09T13:48:18.136785+0000","last_unstale":"2023-08-09T13:48:18.140977+0000","last_undegraded":"2023-08-09T13:48:18.140977+0000","last_fullsized":"2023-08-09T13:48:18.140977+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:48:17.087857+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:48:17.087857+0000","last_clean_scrub_stamp":"2023-08-09T13:48:17.087857+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T21:23:48.688566+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.c","version":"0'0","reported_seq":11,"reported_epoch":73,"state":"active+clean","last_fresh":"2023-08-09T13:48:18.120812+0000","last_change":"2023-08-09T13:48:17.136927+0000","last_active":"2023-08-09T13:48:18.120812+0000","last_peered":"2023-08-09T13:48:18.120812+0000","last_clean":"2023-08-09T13:48:18.120812+0000","last_became_active":"2023-08-09T13:48:17.136614+0000","last_became_peered":"2023-08-09T13:48:17.136614+0000","last_unstale":"2023-08-09T13:48:18.120812+0000","last_undegraded":"2023-08-09T13:48:18.120812+0000","last_fullsized":"2023-08-09T13:48:18.120812+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:48:16.072766+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:48:16.072766+0000","last_clean_scrub_stamp":"2023-08-09T13:48:16.072766+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T17:36:58.038094+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":119,"reported_epoch":73,"state":"active+clean","last_fresh":"2023-08-09T13:48:18.132851+0000","last_change":"2023-08-09T13:33:56.404236+0000","last_active":"2023-08-09T13:48:18.132851+0000","last_peered":"2023-08-09T13:48:18.132851+0000","last_clean":"2023-08-09T13:48:18.132851+0000","last_became_active":"2023-08-09T13:33:56.402485+0000","last_became_peered":"2023-08-09T13:33:56.402485+0000","last_unstale":"2023-08-09T13:48:18.132851+0000","last_undegraded":"2023-08-09T13:48:18.132851+0000","last_fullsized":"2023-08-09T13:48:18.132851+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:33:54.845869+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:33:54.845869+0000","last_clean_scrub_stamp":"2023-08-09T13:33:54.845869+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-11T00:31:43.966707+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.e","version":"0'0","reported_seq":9,"reported_epoch":73,"state":"active+clean","last_fresh":"2023-08-09T13:48:18.149440+0000","last_change":"2023-08-09T13:48:18.143367+0000","last_active":"2023-08-09T13:48:18.149440+0000","last_peered":"2023-08-09T13:48:18.149440+0000","last_clean":"2023-08-09T13:48:18.149440+0000","last_became_active":"2023-08-09T13:48:18.143024+0000","last_became_peered":"2023-08-09T13:48:18.143024+0000","last_unstale":"2023-08-09T13:48:18.149440+0000","last_undegraded":"2023-08-09T13:48:18.149440+0000","last_fullsized":"2023-08-09T13:48:18.149440+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:48:17.087857+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:48:17.087857+0000","last_clean_scrub_stamp":"2023-08-09T13:48:17.087857+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T17:40:48.922810+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.f","version":"73'1","reported_seq":12,"reported_epoch":73,"state":"active+clean","last_fresh":"2023-08-09T13:48:18.173195+0000","last_change":"2023-08-09T13:48:17.133336+0000","last_active":"2023-08-09T13:48:18.173195+0000","last_peered":"2023-08-09T13:48:18.173195+0000","last_clean":"2023-08-09T13:48:18.173195+0000","last_became_active":"2023-08-09T13:48:17.133055+0000","last_became_peered":"2023-08-09T13:48:17.133055+0000","last_unstale":"2023-08-09T13:48:18.173195+0000","last_undegraded":"2023-08-09T13:48:18.173195+0000","last_fullsized":"2023-08-09T13:48:18.173195+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:48:16.072766+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:48:16.072766+0000","last_clean_scrub_stamp":"2023-08-09T13:48:16.072766+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-11T00:10:18.577968+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":117,"reported_epoch":73,"state":"active+clean","last_fresh":"2023-08-09T13:48:18.120260+0000","last_change":"2023-08-09T13:33:56.401781+0000","last_active":"2023-08-09T13:48:18.120260+0000","last_peered":"2023-08-09T13:48:18.120260+0000","last_clean":"2023-08-09T13:48:18.120260+0000","last_became_active":"2023-08-09T13:33:56.400336+0000","last_became_peered":"2023-08-09T13:33:56.400336+0000","last_unstale":"2023-08-09T13:48:18.120260+0000","last_undegraded":"2023-08-09T13:48:18.120260+0000","last_fullsized":"2023-08-09T13:48:18.120260+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:33:54.845869+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:33:54.845869+0000","last_clean_scrub_stamp":"2023-08-09T13:33:54.845869+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T16:41:47.077169+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.f","version":"0'0","reported_seq":9,"reported_epoch":73,"state":"active+clean","last_fresh":"2023-08-09T13:48:18.156161+0000","last_change":"2023-08-09T13:48:18.155218+0000","last_active":"2023-08-09T13:48:18.156161+0000","last_peered":"2023-08-09T13:48:18.156161+0000","last_clean":"2023-08-09T13:48:18.156161+0000","last_became_active":"2023-08-09T13:48:18.154894+0000","last_became_peered":"2023-08-09T13:48:18.154894+0000","last_unstale":"2023-08-09T13:48:18.156161+0000","last_undegraded":"2023-08-09T13:48:18.156161+0000","last_fullsized":"2023-08-09T13:48:18.156161+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:48:17.087857+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:48:17.087857+0000","last_clean_scrub_stamp":"2023-08-09T13:48:17.087857+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-11T00:56:32.885321+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.e","version":"73'2","reported_seq":13,"reported_epoch":73,"state":"active+clean","last_fresh":"2023-08-09T13:48:18.173818+0000","last_change":"2023-08-09T13:48:17.129292+0000","last_active":"2023-08-09T13:48:18.173818+0000","last_peered":"2023-08-09T13:48:18.173818+0000","last_clean":"2023-08-09T13:48:18.173818+0000","last_became_active":"2023-08-09T13:48:17.128999+0000","last_became_peered":"2023-08-09T13:48:17.128999+0000","last_unstale":"2023-08-09T13:48:18.173818+0000","last_undegraded":"2023-08-09T13:48:18.173818+0000","last_fullsized":"2023-08-09T13:48:18.173818+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:48:16.072766+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:48:16.072766+0000","last_clean_scrub_stamp":"2023-08-09T13:48:16.072766+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T22:54:26.443293+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":117,"reported_epoch":73,"state":"active+clean","last_fresh":"2023-08-09T13:48:18.117136+0000","last_change":"2023-08-09T13:33:55.886389+0000","last_active":"2023-08-09T13:48:18.117136+0000","last_peered":"2023-08-09T13:48:18.117136+0000","last_clean":"2023-08-09T13:48:18.117136+0000","last_became_active":"2023-08-09T13:33:55.885638+0000","last_became_peered":"2023-08-09T13:33:55.885638+0000","last_unstale":"2023-08-09T13:48:18.117136+0000","last_undegraded":"2023-08-09T13:48:18.117136+0000","last_fullsized":"2023-08-09T13:48:18.117136+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:33:54.845869+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:33:54.845869+0000","last_clean_scrub_stamp":"2023-08-09T13:33:54.845869+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-11T00:48:14.667108+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]}],"pool_stats":[{"poolid":9,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":8,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":38,"ondisk_log_size":38,"up":64,"acting":64,"num_store_stats":8},{"poolid":2,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":308,"num_read_kb":512,"num_write":199,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":139,"ondisk_log_size":139,"up":2,"acting":2,"num_store_stats":2},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":8}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834575022,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":36640,"kb_used_data":1424,"kb_used_omap":1,"kb_used_meta":35198,"kb_avail":94335200,"statfs":{"total":96636764160,"available":96599244800,"internally_reserved":0,"allocated":1458176,"data_stored":1045869,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1781,"internal_metadata":36043019},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Aug 9 13:48:11 2023","interfaces":[{"interface":"back","average":{"1min":1.242,"5min":1.22,"15min":1.23},"min":{"1min":0.71999999999999997,"5min":0.71999999999999997,"15min":0.61099999999999999},"max":{"1min":1.7190000000000001,"5min":1.7190000000000001,"15min":2.1709999999999998},"last":1.8919999999999999},{"interface":"front","average":{"1min":1.212,"5min":1.238,"15min":1.2569999999999999},"min":{"1min":0.85499999999999998,"5min":0.78100000000000003,"15min":0.65000000000000002},"max":{"1min":1.536,"5min":1.8520000000000001,"15min":2.6240000000000001},"last":1.6080000000000001}]},{"osd":1,"last update":"Wed Aug 9 13:48:11 2023","interfaces":[{"interface":"back","average":{"1min":1.1559999999999999,"5min":1.244,"15min":1.3360000000000001},"min":{"1min":0.73099999999999998,"5min":0.73099999999999998,"15min":0.59799999999999998},"max":{"1min":1.6339999999999999,"5min":1.923,"15min":10.823},"last":1.3240000000000001},{"interface":"front","average":{"1min":1.1499999999999999,"5min":1.2609999999999999,"15min":1.347},"min":{"1min":0.72899999999999998,"5min":0.72899999999999998,"15min":0.72899999999999998},"max":{"1min":1.8899999999999999,"5min":1.8899999999999999,"15min":10.576000000000001},"last":1.9410000000000001}]},{"osd":2,"last update":"Wed Aug 9 13:48:11 2023","interfaces":[{"interface":"back","average":{"1min":1.212,"5min":1.2509999999999999,"15min":1.2569999999999999},"min":{"1min":0.65700000000000003,"5min":0.65700000000000003,"15min":0.65700000000000003},"max":{"1min":1.5920000000000001,"5min":1.736,"15min":3.5270000000000001},"last":1.016},{"interface":"front","average":{"1min":1.2170000000000001,"5min":1.3759999999999999,"15min":1.395},"min":{"1min":0.86899999999999999,"5min":0.80900000000000005,"15min":0.67800000000000005},"max":{"1min":1.8360000000000001,"5min":1.8360000000000001,"15min":3.1190000000000002},"last":1.0620000000000001}]},{"osd":3,"last update":"Wed Aug 9 13:48:11 2023","interfaces":[{"interface":"back","average":{"1min":1.2490000000000001,"5min":1.3300000000000001,"15min":1.333},"min":{"1min":0.86799999999999999,"5min":0.86799999999999999,"15min":0.77500000000000002},"max":{"1min":1.9359999999999999,"5min":1.9359999999999999,"15min":2.3010000000000002},"last":1.532},{"interface":"front","average":{"1min":1.4399999999999999,"5min":1.444,"15min":1.4419999999999999},"min":{"1min":0.88500000000000001,"5min":0.88500000000000001,"15min":0.82099999999999995},"max":{"1min":2.0790000000000002,"5min":2.0790000000000002,"15min":3.2770000000000001},"last":1.1839999999999999}]},{"osd":4,"last update":"Wed Aug 9 13:48:11 2023","interfaces":[{"interface":"back","average":{"1min":1.2210000000000001,"5min":1.2629999999999999,"15min":1.3049999999999999},"min":{"1min":0.59499999999999997,"5min":0.59499999999999997,"15min":0.59499999999999997},"max":{"1min":1.738,"5min":1.8120000000000001,"15min":3.0209999999999999},"last":2.032},{"interface":"front","average":{"1min":1.4139999999999999,"5min":1.3420000000000001,"15min":1.375},"min":{"1min":1.052,"5min":0.874,"15min":0.56799999999999995},"max":{"1min":2.044,"5min":2.044,"15min":3.169},"last":2.1419999999999999}]},{"osd":5,"last update":"Wed Aug 9 13:48:11 2023","interfaces":[{"interface":"back","average":{"1min":1.452,"5min":1.4399999999999999,"15min":1.452},"min":{"1min":0.98399999999999999,"5min":0.82599999999999996,"15min":0.71499999999999997},"max":{"1min":2.1480000000000001,"5min":2.1480000000000001,"15min":3.5830000000000002},"last":1.988},{"interface":"front","average":{"1min":1.395,"5min":1.458,"15min":1.4770000000000001},"min":{"1min":0.95699999999999996,"5min":0.85699999999999998,"15min":0.69199999999999995},"max":{"1min":2.0880000000000001,"5min":2.1520000000000001,"15min":3.3439999999999999},"last":2.113}]},{"osd":6,"last update":"Wed Aug 9 13:48:11 2023","interfaces":[{"interface":"back","average":{"1min":1.484,"5min":1.528,"15min":1.5760000000000001},"min":{"1min":0.92700000000000005,"5min":0.92700000000000005,"15min":0.92700000000000005},"max":{"1min":2.2069999999999999,"5min":2.3490000000000002,"15min":3.2189999999999999},"last":2.2010000000000001},{"interface":"front","average":{"1min":1.506,"5min":1.4970000000000001,"15min":1.508},"min":{"1min":1.0209999999999999,"5min":1,"15min":0.60799999999999998},"max":{"1min":2.1320000000000001,"5min":2.2530000000000001,"15min":2.9590000000000001},"last":1.6679999999999999}]}]},{"osd":6,"up_from":13,"seq":55834575022,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31960,"kb_used_data":840,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":94339880,"statfs":{"total":96636764160,"available":96604037120,"internally_reserved":0,"allocated":860160,"data_stored":454140,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1781,"internal_metadata":31848715},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Aug 9 13:47:19 2023","interfaces":[{"interface":"back","average":{"1min":1.1879999999999999,"5min":1.171,"15min":1.1659999999999999},"min":{"1min":0.72499999999999998,"5min":0.72499999999999998,"15min":0.56599999999999995},"max":{"1min":1.633,"5min":1.9870000000000001,"15min":1.9870000000000001},"last":1.591},{"interface":"front","average":{"1min":1.165,"5min":1.212,"15min":1.1779999999999999},"min":{"1min":0.88200000000000001,"5min":0.86699999999999999,"15min":0.72999999999999998},"max":{"1min":1.526,"5min":1.7090000000000001,"15min":1.833},"last":1.6779999999999999}]},{"osd":1,"last update":"Wed Aug 9 13:47:19 2023","interfaces":[{"interface":"back","average":{"1min":1.1679999999999999,"5min":1.238,"15min":1.2050000000000001},"min":{"1min":0.71399999999999997,"5min":0.71399999999999997,"15min":0.65500000000000003},"max":{"1min":1.587,"5min":1.823,"15min":1.823},"last":1.2989999999999999},{"interface":"front","average":{"1min":1.266,"5min":1.2549999999999999,"15min":1.25},"min":{"1min":0.97499999999999998,"5min":0.93000000000000005,"15min":0.68100000000000005},"max":{"1min":1.8660000000000001,"5min":1.8660000000000001,"15min":1.8660000000000001},"last":1.2230000000000001}]},{"osd":2,"last update":"Wed Aug 9 13:47:19 2023","interfaces":[{"interface":"back","average":{"1min":1.3839999999999999,"5min":1.347,"15min":1.296},"min":{"1min":1.006,"5min":0.92000000000000004,"15min":0.68700000000000006},"max":{"1min":1.677,"5min":1.7509999999999999,"15min":1.774},"last":1.341},{"interface":"front","average":{"1min":1.329,"5min":1.3180000000000001,"15min":1.2789999999999999},"min":{"1min":0.90300000000000002,"5min":0.90300000000000002,"15min":0.59499999999999997},"max":{"1min":1.6479999999999999,"5min":1.883,"15min":1.883},"last":1.532}]},{"osd":3,"last update":"Wed Aug 9 13:47:19 2023","interfaces":[{"interface":"back","average":{"1min":1.3620000000000001,"5min":1.4370000000000001,"15min":1.349},"min":{"1min":1.0569999999999999,"5min":1.0509999999999999,"15min":0.73799999999999999},"max":{"1min":1.8149999999999999,"5min":1.921,"15min":1.921},"last":1.8140000000000001},{"interface":"front","average":{"1min":1.3540000000000001,"5min":1.3360000000000001,"15min":1.3080000000000001},"min":{"1min":0.78200000000000003,"5min":0.78200000000000003,"15min":0.74299999999999999},"max":{"1min":1.7410000000000001,"5min":1.7669999999999999,"15min":1.915},"last":1.177}]},{"osd":4,"last update":"Wed Aug 9 13:47:19 2023","interfaces":[{"interface":"back","average":{"1min":1.286,"5min":1.224,"15min":1.286},"min":{"1min":0.90200000000000002,"5min":0.73499999999999999,"15min":0.64300000000000002},"max":{"1min":1.917,"5min":2.3479999999999999,"15min":2.3479999999999999},"last":1.4339999999999999},{"interface":"front","average":{"1min":1.3859999999999999,"5min":1.3160000000000001,"15min":1.3440000000000001},"min":{"1min":0.98099999999999998,"5min":0.82599999999999996,"15min":0.69099999999999995},"max":{"1min":1.71,"5min":1.97,"15min":1.998},"last":1.3779999999999999}]},{"osd":5,"last update":"Wed Aug 9 13:47:19 2023","interfaces":[{"interface":"back","average":{"1min":1.27,"5min":1.327,"15min":1.335},"min":{"1min":0.83799999999999997,"5min":0.83499999999999996,"15min":0.65400000000000003},"max":{"1min":1.887,"5min":1.887,"15min":2.0910000000000002},"last":1.484},{"interface":"front","average":{"1min":1.5609999999999999,"5min":1.5580000000000001,"15min":1.5640000000000001},"min":{"1min":1.3360000000000001,"5min":0.97999999999999998,"15min":0.83799999999999997},"max":{"1min":1.97,"5min":3.2370000000000001,"15min":3.2370000000000001},"last":1.756}]},{"osd":7,"last update":"Wed Aug 9 13:47:19 2023","interfaces":[{"interface":"back","average":{"1min":1.5680000000000001,"5min":1.542,"15min":1.55},"min":{"1min":1.2609999999999999,"5min":0.62,"15min":0.62},"max":{"1min":1.9419999999999999,"5min":2.0579999999999998,"15min":2.3570000000000002},"last":2.161},{"interface":"front","average":{"1min":1.456,"5min":1.5189999999999999,"15min":1.534},"min":{"1min":0.94399999999999995,"5min":0.88500000000000001,"15min":0.88500000000000001},"max":{"1min":1.9930000000000001,"5min":2.427,"15min":2.427},"last":1.6399999999999999}]}]},{"osd":5,"up_from":13,"seq":55834575022,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":36056,"kb_used_data":844,"kb_used_omap":1,"kb_used_meta":35198,"kb_avail":94335784,"statfs":{"total":96636764160,"available":96599842816,"internally_reserved":0,"allocated":864256,"data_stored":455501,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1781,"internal_metadata":36043019},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Aug 9 13:48:16 2023","interfaces":[{"interface":"back","average":{"1min":1.119,"5min":1.1759999999999999,"15min":1.1659999999999999},"min":{"1min":0.65600000000000003,"5min":0.65600000000000003,"15min":0.65600000000000003},"max":{"1min":2.1869999999999998,"5min":2.1869999999999998,"15min":2.1869999999999998},"last":1.621},{"interface":"front","average":{"1min":1.1859999999999999,"5min":1.2250000000000001,"15min":1.2},"min":{"1min":0.73799999999999999,"5min":0.73799999999999999,"15min":0.64200000000000002},"max":{"1min":1.6619999999999999,"5min":1.6619999999999999,"15min":1.7989999999999999},"last":1.175}]},{"osd":1,"last update":"Wed Aug 9 13:48:16 2023","interfaces":[{"interface":"back","average":{"1min":1.093,"5min":1.2529999999999999,"15min":1.2170000000000001},"min":{"1min":0.73199999999999998,"5min":0.73199999999999998,"15min":0.64100000000000001},"max":{"1min":1.6379999999999999,"5min":1.6379999999999999,"15min":1.736},"last":0.90000000000000002},{"interface":"front","average":{"1min":1.169,"5min":1.3160000000000001,"15min":1.264},"min":{"1min":0.66900000000000004,"5min":0.66900000000000004,"15min":0.66900000000000004},"max":{"1min":1.607,"5min":1.825,"15min":1.825},"last":1.4299999999999999}]},{"osd":2,"last update":"Wed Aug 9 13:48:16 2023","interfaces":[{"interface":"back","average":{"1min":1.2509999999999999,"5min":1.29,"15min":1.272},"min":{"1min":0.68999999999999995,"5min":0.68999999999999995,"15min":0.61599999999999999},"max":{"1min":1.756,"5min":1.756,"15min":1.899},"last":1.0489999999999999},{"interface":"front","average":{"1min":1.2989999999999999,"5min":1.3200000000000001,"15min":1.3009999999999999},"min":{"1min":0.82399999999999995,"5min":0.82399999999999995,"15min":0.66800000000000004},"max":{"1min":1.589,"5min":1.7390000000000001,"15min":1.7969999999999999},"last":1.482}]},{"osd":3,"last update":"Wed Aug 9 13:48:16 2023","interfaces":[{"interface":"back","average":{"1min":1.4119999999999999,"5min":1.3819999999999999,"15min":1.3420000000000001},"min":{"1min":1.115,"5min":1.079,"15min":0.59799999999999998},"max":{"1min":2.1840000000000002,"5min":2.1840000000000002,"15min":2.2690000000000001},"last":1.5589999999999999},{"interface":"front","average":{"1min":1.4650000000000001,"5min":1.4430000000000001,"15min":1.415},"min":{"1min":0.94899999999999995,"5min":0.94899999999999995,"15min":0.70999999999999996},"max":{"1min":2.3780000000000001,"5min":2.3780000000000001,"15min":2.3780000000000001},"last":1.5269999999999999}]},{"osd":4,"last update":"Wed Aug 9 13:48:16 2023","interfaces":[{"interface":"back","average":{"1min":1.2669999999999999,"5min":1.2769999999999999,"15min":1.2929999999999999},"min":{"1min":0.89900000000000002,"5min":0.84699999999999998,"15min":0.56599999999999995},"max":{"1min":2.0800000000000001,"5min":2.0800000000000001,"15min":2.121},"last":1.3759999999999999},{"interface":"front","average":{"1min":1.3720000000000001,"5min":1.3600000000000001,"15min":1.391},"min":{"1min":0.97099999999999997,"5min":0.85099999999999998,"15min":0.58599999999999997},"max":{"1min":2.1230000000000002,"5min":2.1230000000000002,"15min":2.3679999999999999},"last":1.7350000000000001}]},{"osd":6,"last update":"Wed Aug 9 13:48:16 2023","interfaces":[{"interface":"back","average":{"1min":1.462,"5min":1.3819999999999999,"15min":1.407},"min":{"1min":1.052,"5min":0.91800000000000004,"15min":0.66700000000000004},"max":{"1min":2.306,"5min":2.306,"15min":2.306},"last":1.327},{"interface":"front","average":{"1min":1.4370000000000001,"5min":1.446,"15min":1.4770000000000001},"min":{"1min":1.0900000000000001,"5min":1.0149999999999999,"15min":0.71199999999999997},"max":{"1min":2.2320000000000002,"5min":2.2320000000000002,"15min":2.2320000000000002},"last":1.25}]},{"osd":7,"last update":"Wed Aug 9 13:48:16 2023","interfaces":[{"interface":"back","average":{"1min":1.5940000000000001,"5min":1.5549999999999999,"15min":1.5609999999999999},"min":{"1min":1.1419999999999999,"5min":1.05,"15min":0.70899999999999996},"max":{"1min":2.452,"5min":2.452,"15min":2.4729999999999999},"last":1.6699999999999999},{"interface":"front","average":{"1min":1.5920000000000001,"5min":1.601,"15min":1.613},"min":{"1min":1.163,"5min":1.163,"15min":0.77600000000000002},"max":{"1min":2.1070000000000002,"5min":2.1070000000000002,"15min":2.3820000000000001},"last":1.597}]}]},{"osd":4,"up_from":13,"seq":55834575022,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27868,"kb_used_data":844,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343972,"statfs":{"total":96636764160,"available":96608227328,"internally_reserved":0,"allocated":864256,"data_stored":455501,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1781,"internal_metadata":27654411},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Aug 9 13:47:18 2023","interfaces":[{"interface":"back","average":{"1min":1.29,"5min":1.226,"15min":1.177},"min":{"1min":0.96799999999999997,"5min":0.83199999999999996,"15min":0.64200000000000002},"max":{"1min":1.7589999999999999,"5min":1.7589999999999999,"15min":1.7589999999999999},"last":1.012},{"interface":"front","average":{"1min":1.2629999999999999,"5min":1.304,"15min":1.2589999999999999},"min":{"1min":0.97199999999999998,"5min":0.85299999999999998,"15min":0.69999999999999996},"max":{"1min":1.5760000000000001,"5min":1.863,"15min":1.863},"last":1.157}]},{"osd":1,"last update":"Wed Aug 9 13:47:18 2023","interfaces":[{"interface":"back","average":{"1min":1.329,"5min":1.369,"15min":1.306},"min":{"1min":0.92500000000000004,"5min":0.91400000000000003,"15min":0.77100000000000002},"max":{"1min":1.8480000000000001,"5min":1.8480000000000001,"15min":1.8480000000000001},"last":1.0720000000000001},{"interface":"front","average":{"1min":1.262,"5min":1.2789999999999999,"15min":1.2310000000000001},"min":{"1min":1.022,"5min":0.81499999999999995,"15min":0.64900000000000002},"max":{"1min":1.6279999999999999,"5min":1.722,"15min":1.7290000000000001},"last":1.4350000000000001}]},{"osd":2,"last update":"Wed Aug 9 13:47:18 2023","interfaces":[{"interface":"back","average":{"1min":1.321,"5min":1.274,"15min":1.2689999999999999},"min":{"1min":0.91000000000000003,"5min":0.79900000000000004,"15min":0.78000000000000003},"max":{"1min":1.726,"5min":1.726,"15min":1.8049999999999999},"last":1.379},{"interface":"front","average":{"1min":1.3500000000000001,"5min":1.405,"15min":1.3859999999999999},"min":{"1min":1.111,"5min":0.88700000000000001,"15min":0.74299999999999999},"max":{"1min":1.649,"5min":1.7190000000000001,"15min":1.948},"last":1.268}]},{"osd":3,"last update":"Wed Aug 9 13:47:18 2023","interfaces":[{"interface":"back","average":{"1min":1.2390000000000001,"5min":1.3089999999999999,"15min":1.2969999999999999},"min":{"1min":0.97599999999999998,"5min":0.92600000000000005,"15min":0.84699999999999998},"max":{"1min":1.4670000000000001,"5min":1.7050000000000001,"15min":2.3780000000000001},"last":1.224},{"interface":"front","average":{"1min":1.4510000000000001,"5min":1.3700000000000001,"15min":1.379},"min":{"1min":1.1479999999999999,"5min":0.96999999999999997,"15min":0.63100000000000001},"max":{"1min":1.679,"5min":1.833,"15min":2.113},"last":1.4670000000000001}]},{"osd":5,"last update":"Wed Aug 9 13:47:18 2023","interfaces":[{"interface":"back","average":{"1min":1.294,"5min":1.331,"15min":1.3500000000000001},"min":{"1min":0.83099999999999996,"5min":0.63200000000000001,"15min":0.63200000000000001},"max":{"1min":1.774,"5min":1.774,"15min":2.2959999999999998},"last":0.91200000000000003},{"interface":"front","average":{"1min":1.296,"5min":1.2909999999999999,"15min":1.3160000000000001},"min":{"1min":0.93000000000000005,"5min":0.70499999999999996,"15min":0.626},"max":{"1min":1.6799999999999999,"5min":1.786,"15min":2.2240000000000002},"last":1.111}]},{"osd":6,"last update":"Wed Aug 9 13:47:18 2023","interfaces":[{"interface":"back","average":{"1min":1.2370000000000001,"5min":1.266,"15min":1.345},"min":{"1min":0.84699999999999998,"5min":0.59099999999999997,"15min":0.59099999999999997},"max":{"1min":1.6719999999999999,"5min":1.786,"15min":1.952},"last":1.5229999999999999},{"interface":"front","average":{"1min":1.472,"5min":1.5,"15min":1.508},"min":{"1min":1.022,"5min":0.76400000000000001,"15min":0.76400000000000001},"max":{"1min":1.8979999999999999,"5min":1.8979999999999999,"15min":2.1899999999999999},"last":1.351}]},{"osd":7,"last update":"Wed Aug 9 13:47:18 2023","interfaces":[{"interface":"back","average":{"1min":1.417,"5min":1.466,"15min":1.476},"min":{"1min":1.0820000000000001,"5min":1.03,"15min":0.73899999999999999},"max":{"1min":1.708,"5min":1.9450000000000001,"15min":2.2210000000000001},"last":1.3260000000000001},{"interface":"front","average":{"1min":1.468,"5min":1.484,"15min":1.518},"min":{"1min":0.97199999999999998,"5min":0.74099999999999999,"15min":0.72899999999999998},"max":{"1min":1.696,"5min":1.9750000000000001,"15min":2.0489999999999999},"last":1.577}]}]},{"osd":3,"up_from":13,"seq":55834575023,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":36056,"kb_used_data":844,"kb_used_omap":1,"kb_used_meta":35198,"kb_avail":94335784,"statfs":{"total":96636764160,"available":96599842816,"internally_reserved":0,"allocated":864256,"data_stored":455501,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1781,"internal_metadata":36043019},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Aug 9 13:47:19 2023","interfaces":[{"interface":"back","average":{"1min":0.93100000000000005,"5min":0.96399999999999997,"15min":0.93200000000000005},"min":{"1min":0.67900000000000005,"5min":0.67700000000000005,"15min":0.38200000000000001},"max":{"1min":1.234,"5min":1.6240000000000001,"15min":1.968},"last":0.95899999999999996},{"interface":"front","average":{"1min":1.157,"5min":1.1719999999999999,"15min":1.075},"min":{"1min":0.80700000000000005,"5min":0.68600000000000005,"15min":0.40799999999999997},"max":{"1min":1.528,"5min":2.0590000000000002,"15min":2.2360000000000002},"last":1.1120000000000001}]},{"osd":1,"last update":"Wed Aug 9 13:47:19 2023","interfaces":[{"interface":"back","average":{"1min":1.032,"5min":1.0800000000000001,"15min":0.98999999999999999},"min":{"1min":0.78200000000000003,"5min":0.754,"15min":0.438},"max":{"1min":1.6519999999999999,"5min":2.5649999999999999,"15min":2.5649999999999999},"last":1.367},{"interface":"front","average":{"1min":1.224,"5min":1.2470000000000001,"15min":1.171},"min":{"1min":0.92600000000000005,"5min":0.872,"15min":0.42099999999999999},"max":{"1min":1.462,"5min":2.3500000000000001,"15min":2.6760000000000002},"last":1.4730000000000001}]},{"osd":2,"last update":"Wed Aug 9 13:47:19 2023","interfaces":[{"interface":"back","average":{"1min":1.3680000000000001,"5min":1.2749999999999999,"15min":1.1579999999999999},"min":{"1min":0.96199999999999997,"5min":0.86699999999999999,"15min":0.53500000000000003},"max":{"1min":1.905,"5min":1.905,"15min":1.923},"last":1.913},{"interface":"front","average":{"1min":1.387,"5min":1.389,"15min":1.3109999999999999},"min":{"1min":1.054,"5min":0.91300000000000003,"15min":0.52000000000000002},"max":{"1min":1.635,"5min":2.0259999999999998,"15min":2.1429999999999998},"last":1.3080000000000001}]},{"osd":4,"last update":"Wed Aug 9 13:47:19 2023","interfaces":[{"interface":"back","average":{"1min":1.3220000000000001,"5min":1.319,"15min":1.292},"min":{"1min":1.03,"5min":0.89500000000000002,"15min":0.627},"max":{"1min":1.7969999999999999,"5min":1.7969999999999999,"15min":1.992},"last":1.8160000000000001},{"interface":"front","average":{"1min":1.4990000000000001,"5min":1.464,"15min":1.405},"min":{"1min":1.214,"5min":1.034,"15min":0.72999999999999998},"max":{"1min":1.964,"5min":1.964,"15min":3.0649999999999999},"last":1.5600000000000001}]},{"osd":5,"last update":"Wed Aug 9 13:47:19 2023","interfaces":[{"interface":"back","average":{"1min":1.571,"5min":1.544,"15min":1.472},"min":{"1min":1.1719999999999999,"5min":1.1719999999999999,"15min":0.67200000000000004},"max":{"1min":1.99,"5min":1.992,"15min":2.3610000000000002},"last":1.778},{"interface":"front","average":{"1min":1.5860000000000001,"5min":1.573,"15min":1.4870000000000001},"min":{"1min":1.236,"5min":1.0960000000000001,"15min":0.79100000000000004},"max":{"1min":2.177,"5min":2.2970000000000002,"15min":2.2970000000000002},"last":1.621}]},{"osd":6,"last update":"Wed Aug 9 13:47:19 2023","interfaces":[{"interface":"back","average":{"1min":1.494,"5min":1.5129999999999999,"15min":1.454},"min":{"1min":1.083,"5min":1.083,"15min":0.72599999999999998},"max":{"1min":1.7769999999999999,"5min":2.242,"15min":2.242},"last":1.972},{"interface":"front","average":{"1min":1.71,"5min":1.625,"15min":1.5529999999999999},"min":{"1min":1.3420000000000001,"5min":1.1819999999999999,"15min":0.68600000000000005},"max":{"1min":2.0790000000000002,"5min":2.0790000000000002,"15min":3.0049999999999999},"last":2.0179999999999998}]},{"osd":7,"last update":"Wed Aug 9 13:47:19 2023","interfaces":[{"interface":"back","average":{"1min":1.681,"5min":1.706,"15min":1.617},"min":{"1min":1.389,"5min":1.304,"15min":0.71599999999999997},"max":{"1min":2.323,"5min":2.419,"15min":2.6120000000000001},"last":1.863},{"interface":"front","average":{"1min":1.593,"5min":1.6200000000000001,"15min":1.5600000000000001},"min":{"1min":1.264,"5min":1.149,"15min":0.61599999999999999},"max":{"1min":1.9350000000000001,"5min":2.484,"15min":2.484},"last":2.0899999999999999}]}]},{"osd":2,"up_from":13,"seq":55834575022,"num_pgs":13,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":36060,"kb_used_data":844,"kb_used_omap":1,"kb_used_meta":35198,"kb_avail":94335780,"statfs":{"total":96636764160,"available":96599838720,"internally_reserved":0,"allocated":864256,"data_stored":455501,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1781,"internal_metadata":36043019},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Aug 9 13:48:16 2023","interfaces":[{"interface":"back","average":{"1min":0.84899999999999998,"5min":0.98699999999999999,"15min":0.95499999999999996},"min":{"1min":0.36499999999999999,"5min":0.36499999999999999,"15min":0.36499999999999999},"max":{"1min":1.306,"5min":1.752,"15min":1.8380000000000001},"last":0.63100000000000001},{"interface":"front","average":{"1min":0.85099999999999998,"5min":0.99199999999999999,"15min":1.018},"min":{"1min":0.374,"5min":0.374,"15min":0.374},"max":{"1min":2.3929999999999998,"5min":2.3929999999999998,"15min":2.3929999999999998},"last":0.59299999999999997}]},{"osd":1,"last update":"Wed Aug 9 13:48:16 2023","interfaces":[{"interface":"back","average":{"1min":1,"5min":1.1950000000000001,"15min":1.1830000000000001},"min":{"1min":0.55200000000000005,"5min":0.55200000000000005,"15min":0.54400000000000004},"max":{"1min":1.744,"5min":1.744,"15min":2.0249999999999999},"last":0.745},{"interface":"front","average":{"1min":0.97999999999999998,"5min":1.335,"15min":1.335},"min":{"1min":0.52900000000000003,"5min":0.52900000000000003,"15min":0.52900000000000003},"max":{"1min":2.4870000000000001,"5min":2.4870000000000001,"15min":2.4870000000000001},"last":0.77000000000000002}]},{"osd":3,"last update":"Wed Aug 9 13:48:16 2023","interfaces":[{"interface":"back","average":{"1min":0.75900000000000001,"5min":1.04,"15min":1.048},"min":{"1min":0.38900000000000001,"5min":0.38900000000000001,"15min":0.38900000000000001},"max":{"1min":1.55,"5min":1.7609999999999999,"15min":2.0409999999999999},"last":0.68700000000000006},{"interface":"front","average":{"1min":1.3680000000000001,"5min":1.3500000000000001,"15min":1.331},"min":{"1min":0.46200000000000002,"5min":0.46200000000000002,"15min":0.46200000000000002},"max":{"1min":8.8130000000000006,"5min":8.8130000000000006,"15min":8.8130000000000006},"last":0.80300000000000005}]},{"osd":4,"last update":"Wed Aug 9 13:48:16 2023","interfaces":[{"interface":"back","average":{"1min":1.25,"5min":1.452,"15min":1.4650000000000001},"min":{"1min":0.628,"5min":0.628,"15min":0.628},"max":{"1min":1.869,"5min":1.869,"15min":3.137},"last":1.01},{"interface":"front","average":{"1min":1.208,"5min":1.371,"15min":1.4039999999999999},"min":{"1min":0.68600000000000005,"5min":0.68600000000000005,"15min":0.68600000000000005},"max":{"1min":2.4460000000000002,"5min":2.4460000000000002,"15min":2.5150000000000001},"last":1.212}]},{"osd":5,"last update":"Wed Aug 9 13:48:16 2023","interfaces":[{"interface":"back","average":{"1min":1.208,"5min":1.4350000000000001,"15min":1.4570000000000001},"min":{"1min":0.59999999999999998,"5min":0.59999999999999998,"15min":0.59999999999999998},"max":{"1min":2.5840000000000001,"5min":2.5840000000000001,"15min":2.5840000000000001},"last":1.633},{"interface":"front","average":{"1min":1.2529999999999999,"5min":1.3580000000000001,"15min":1.371},"min":{"1min":0.56799999999999995,"5min":0.56799999999999995,"15min":0.56799999999999995},"max":{"1min":3.5659999999999998,"5min":3.5659999999999998,"15min":3.8599999999999999},"last":1.339}]},{"osd":6,"last update":"Wed Aug 9 13:48:16 2023","interfaces":[{"interface":"back","average":{"1min":1.28,"5min":1.4399999999999999,"15min":1.427},"min":{"1min":0.73799999999999999,"5min":0.73799999999999999,"15min":0.73799999999999999},"max":{"1min":3.5099999999999998,"5min":3.5099999999999998,"15min":3.5099999999999998},"last":1.4379999999999999},{"interface":"front","average":{"1min":1.3100000000000001,"5min":1.5329999999999999,"15min":1.5620000000000001},"min":{"1min":0.66300000000000003,"5min":0.66300000000000003,"15min":0.66300000000000003},"max":{"1min":2.0430000000000001,"5min":2.1400000000000001,"15min":3.5350000000000001},"last":1.659}]},{"osd":7,"last update":"Wed Aug 9 13:48:16 2023","interfaces":[{"interface":"back","average":{"1min":1.3180000000000001,"5min":1.5249999999999999,"15min":1.583},"min":{"1min":0.64600000000000002,"5min":0.64600000000000002,"15min":0.64600000000000002},"max":{"1min":2.5249999999999999,"5min":2.5249999999999999,"15min":5.2969999999999997},"last":1.2949999999999999},{"interface":"front","average":{"1min":1.385,"5min":1.5249999999999999,"15min":1.534},"min":{"1min":0.70499999999999996,"5min":0.70499999999999996,"15min":0.70499999999999996},"max":{"1min":3.609,"5min":3.609,"15min":5.4539999999999997},"last":1.601}]}]},{"osd":1,"up_from":13,"seq":55834575022,"num_pgs":14,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28448,"kb_used_data":1424,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343392,"statfs":{"total":96636764160,"available":96607633408,"internally_reserved":0,"allocated":1458176,"data_stored":1045869,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1781,"internal_metadata":27654411},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Aug 9 13:47:22 2023","interfaces":[{"interface":"back","average":{"1min":0.88900000000000001,"5min":0.95199999999999996,"15min":0.94799999999999995},"min":{"1min":0.68500000000000005,"5min":0.52100000000000002,"15min":0.32400000000000001},"max":{"1min":1.401,"5min":2.0369999999999999,"15min":2.7080000000000002},"last":0.71199999999999997},{"interface":"front","average":{"1min":1.143,"5min":1.141,"15min":1.1020000000000001},"min":{"1min":0.65700000000000003,"5min":0.65700000000000003,"15min":0.42499999999999999},"max":{"1min":1.506,"5min":2.2490000000000001,"15min":2.6349999999999998},"last":1.4550000000000001}]},{"osd":2,"last update":"Wed Aug 9 13:47:22 2023","interfaces":[{"interface":"back","average":{"1min":1.0169999999999999,"5min":1.0149999999999999,"15min":1},"min":{"1min":0.73599999999999999,"5min":0.71399999999999997,"15min":0.54200000000000004},"max":{"1min":1.3440000000000001,"5min":1.718,"15min":1.762},"last":0.75},{"interface":"front","average":{"1min":1.139,"5min":1.228,"15min":1.198},"min":{"1min":0.749,"5min":0.749,"15min":0.46500000000000002},"max":{"1min":1.639,"5min":2.145,"15min":2.2330000000000001},"last":1.0680000000000001}]},{"osd":3,"last update":"Wed Aug 9 13:47:22 2023","interfaces":[{"interface":"back","average":{"1min":1.3129999999999999,"5min":1.3819999999999999,"15min":1.3240000000000001},"min":{"1min":0.83599999999999997,"5min":0.83599999999999997,"15min":0.432},"max":{"1min":1.71,"5min":2.3210000000000002,"15min":2.3210000000000002},"last":1.2809999999999999},{"interface":"front","average":{"1min":1.3140000000000001,"5min":1.3360000000000001,"15min":1.27},"min":{"1min":0.876,"5min":0.45300000000000001,"15min":0.44400000000000001},"max":{"1min":1.7709999999999999,"5min":2.7669999999999999,"15min":3.3069999999999999},"last":1.3260000000000001}]},{"osd":4,"last update":"Wed Aug 9 13:47:22 2023","interfaces":[{"interface":"back","average":{"1min":1.397,"5min":1.4490000000000001,"15min":1.4079999999999999},"min":{"1min":0.98799999999999999,"5min":0.58099999999999996,"15min":0.58099999999999996},"max":{"1min":1.8109999999999999,"5min":2.4089999999999998,"15min":2.504},"last":1.617},{"interface":"front","average":{"1min":1.395,"5min":1.387,"15min":1.355},"min":{"1min":1.0600000000000001,"5min":1.034,"15min":0.76400000000000001},"max":{"1min":1.6659999999999999,"5min":1.901,"15min":2.5099999999999998},"last":1.236}]},{"osd":5,"last update":"Wed Aug 9 13:47:22 2023","interfaces":[{"interface":"back","average":{"1min":1.3120000000000001,"5min":1.415,"15min":1.391},"min":{"1min":0.95899999999999996,"5min":0.95899999999999996,"15min":0.81599999999999995},"max":{"1min":1.6160000000000001,"5min":2.0750000000000002,"15min":2.2570000000000001},"last":1.5169999999999999},{"interface":"front","average":{"1min":1.446,"5min":1.5920000000000001,"15min":1.5820000000000001},"min":{"1min":1.1160000000000001,"5min":1.1160000000000001,"15min":0.77700000000000002},"max":{"1min":1.774,"5min":2.4649999999999999,"15min":4.9560000000000004},"last":1.3819999999999999}]},{"osd":6,"last update":"Wed Aug 9 13:47:22 2023","interfaces":[{"interface":"back","average":{"1min":1.492,"5min":1.5449999999999999,"15min":1.5289999999999999},"min":{"1min":0.77500000000000002,"5min":0.67400000000000004,"15min":0.67400000000000004},"max":{"1min":1.8939999999999999,"5min":2.5270000000000001,"15min":3.0529999999999999},"last":1.6879999999999999},{"interface":"front","average":{"1min":1.478,"5min":1.5369999999999999,"15min":1.5289999999999999},"min":{"1min":0.88200000000000001,"5min":0.88200000000000001,"15min":0.79800000000000004},"max":{"1min":1.7669999999999999,"5min":2.1040000000000001,"15min":2.7330000000000001},"last":1.419}]},{"osd":7,"last update":"Wed Aug 9 13:47:22 2023","interfaces":[{"interface":"back","average":{"1min":1.556,"5min":1.6870000000000001,"15min":1.677},"min":{"1min":0.90300000000000002,"5min":0.90300000000000002,"15min":0.77100000000000002},"max":{"1min":2.0289999999999999,"5min":2.5790000000000002,"15min":5.1360000000000001},"last":1.486},{"interface":"front","average":{"1min":1.548,"5min":1.653,"15min":1.6259999999999999},"min":{"1min":1.1459999999999999,"5min":0.89500000000000002,"15min":0.75900000000000001},"max":{"1min":1.9670000000000001,"5min":2.6509999999999998,"15min":3.194},"last":1.5569999999999999}]}]},{"osd":0,"up_from":13,"seq":55834575022,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":36056,"kb_used_data":840,"kb_used_omap":1,"kb_used_meta":35198,"kb_avail":94335784,"statfs":{"total":96636764160,"available":96599842816,"internally_reserved":0,"allocated":860160,"data_stored":454140,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1781,"internal_metadata":36043019},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Wed Aug 9 13:47:23 2023","interfaces":[{"interface":"back","average":{"1min":0.99299999999999999,"5min":0.95099999999999996,"15min":0.91800000000000004},"min":{"1min":0.70999999999999996,"5min":0.66200000000000003,"15min":0.39600000000000002},"max":{"1min":1.756,"5min":1.756,"15min":1.756},"last":0.46300000000000002},{"interface":"front","average":{"1min":1.1899999999999999,"5min":1.131,"15min":1.0920000000000001},"min":{"1min":0.83299999999999996,"5min":0.70699999999999996,"15min":0.53200000000000003},"max":{"1min":1.6359999999999999,"5min":1.6839999999999999,"15min":1.7809999999999999},"last":0.66600000000000004}]},{"osd":2,"last update":"Wed Aug 9 13:47:23 2023","interfaces":[{"interface":"back","average":{"1min":1.2230000000000001,"5min":1.194,"15min":1.1439999999999999},"min":{"1min":0.90800000000000003,"5min":0.78900000000000003,"15min":0.44800000000000001},"max":{"1min":1.776,"5min":1.776,"15min":1.889},"last":0.77500000000000002},{"interface":"front","average":{"1min":1.1850000000000001,"5min":1.226,"15min":1.181},"min":{"1min":0.85999999999999999,"5min":0.82799999999999996,"15min":0.52700000000000002},"max":{"1min":1.7669999999999999,"5min":1.9490000000000001,"15min":2.2519999999999998},"last":0.72699999999999998}]},{"osd":3,"last update":"Wed Aug 9 13:47:23 2023","interfaces":[{"interface":"back","average":{"1min":1.175,"5min":1.1659999999999999,"15min":1.1160000000000001},"min":{"1min":0.80400000000000005,"5min":0.80400000000000005,"15min":0.54800000000000004},"max":{"1min":1.823,"5min":1.881,"15min":2.5649999999999999},"last":0.60599999999999998},{"interface":"front","average":{"1min":1.325,"5min":1.341,"15min":1.3260000000000001},"min":{"1min":0.99299999999999999,"5min":0.94199999999999995,"15min":0.55400000000000005},"max":{"1min":1.738,"5min":2.4649999999999999,"15min":2.6819999999999999},"last":0.63700000000000001}]},{"osd":4,"last update":"Wed Aug 9 13:47:23 2023","interfaces":[{"interface":"back","average":{"1min":1.357,"5min":1.363,"15min":1.351},"min":{"1min":1.0700000000000001,"5min":1.014,"15min":0.74299999999999999},"max":{"1min":2.0569999999999999,"5min":2.0569999999999999,"15min":2.0569999999999999},"last":1.196},{"interface":"front","average":{"1min":1.4790000000000001,"5min":1.4890000000000001,"15min":1.464},"min":{"1min":1.125,"5min":1.123,"15min":0.74099999999999999},"max":{"1min":1.9259999999999999,"5min":2.5390000000000001,"15min":2.5390000000000001},"last":0.86799999999999999}]},{"osd":5,"last update":"Wed Aug 9 13:47:23 2023","interfaces":[{"interface":"back","average":{"1min":1.4299999999999999,"5min":1.5,"15min":1.4690000000000001},"min":{"1min":1.115,"5min":1.0680000000000001,"15min":0.80100000000000005},"max":{"1min":2.1880000000000002,"5min":2.1880000000000002,"15min":2.1880000000000002},"last":1.012},{"interface":"front","average":{"1min":1.5389999999999999,"5min":1.4990000000000001,"15min":1.45},"min":{"1min":1.2150000000000001,"5min":1.036,"15min":0.79600000000000004},"max":{"1min":1.8999999999999999,"5min":2.0910000000000002,"15min":2.0910000000000002},"last":0.83899999999999997}]},{"osd":6,"last update":"Wed Aug 9 13:47:23 2023","interfaces":[{"interface":"back","average":{"1min":1.627,"5min":1.5780000000000001,"15min":1.5649999999999999},"min":{"1min":1.264,"5min":1.2190000000000001,"15min":0.69799999999999995},"max":{"1min":2.0569999999999999,"5min":2.7250000000000001,"15min":2.7250000000000001},"last":0.92700000000000005},{"interface":"front","average":{"1min":1.627,"5min":1.581,"15min":1.5569999999999999},"min":{"1min":1.3140000000000001,"5min":1.131,"15min":0.96499999999999997},"max":{"1min":2.097,"5min":2.1400000000000001,"15min":2.2959999999999998},"last":0.97899999999999998}]},{"osd":7,"last update":"Wed Aug 9 13:47:23 2023","interfaces":[{"interface":"back","average":{"1min":1.6870000000000001,"5min":1.6739999999999999,"15min":1.631},"min":{"1min":1.371,"5min":1.194,"15min":0.97899999999999998},"max":{"1min":2.262,"5min":2.3170000000000002,"15min":2.464},"last":0.93999999999999995},{"interface":"front","average":{"1min":1.653,"5min":1.6060000000000001,"15min":1.579},"min":{"1min":1.181,"5min":1.111,"15min":0.83199999999999996},"max":{"1min":2.04,"5min":2.1640000000000001,"15min":2.1640000000000001},"last":0.89200000000000002}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":2194,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":56,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":56,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":652,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":160,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-08-09T13:48:22.691 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-08-09T13:48:22.959 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:22.957+0000 7f97adb44640 1 Processor -- start 2023-08-09T13:48:22.960 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:22.957+0000 7f97adb44640 1 -- start start 2023-08-09T13:48:22.961 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:22.959+0000 7f97adb44640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f97a815f7b0 0x7f97a80b4760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:48:22.961 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:22.959+0000 7f97adb44640 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7f97a80b1050 0x7f97a80ccfa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:48:22.962 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:22.959+0000 7f97adb44640 1 -- --> v1:172.21.15.43:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f97a80cd930 con 0x7f97a80cd4e0 2023-08-09T13:48:22.962 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:22.959+0000 7f97adb44640 1 -- --> v2:172.21.15.203:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f97a80b7bf0 con 0x7f97a80b1050 2023-08-09T13:48:22.962 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:22.959+0000 7f97adb44640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f97a80b7ed0 con 0x7f97a815f7b0 2023-08-09T13:48:22.962 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:22.959+0000 7f97ad343640 1 --1- >> v1:172.21.15.43:6789/0 conn(0x7f97a80cd4e0 0x7f97a80ceae0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.43:6789/0 says I am v1:172.21.15.43:46498/0 (socket says 172.21.15.43:46498) 2023-08-09T13:48:22.962 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:22.959+0000 7f97ad343640 1 -- 172.21.15.43:0/1254121122 learned_addr learned my addr 172.21.15.43:0/1254121122 (peer_addr_for_me v1:172.21.15.43:0/0) 2023-08-09T13:48:22.964 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:22.959+0000 7f97acb42640 1 --2- 172.21.15.43:0/1254121122 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f97a815f7b0 0x7f97a80b4760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:48:22.964 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:22.959+0000 7f97a77fe640 1 -- 172.21.15.43:0/1254121122 <== mon.0 v1:172.21.15.43:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3673230702 0 0) 0x7f97a0001110 con 0x7f97a80cd4e0 2023-08-09T13:48:22.964 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:22.960+0000 7f97a77fe640 1 -- 172.21.15.43:0/1254121122 --> v1:172.21.15.43:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f9790003520 con 0x7f97a80cd4e0 2023-08-09T13:48:22.964 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:22.960+0000 7f97a7fff640 1 --2- 172.21.15.43:0/1254121122 >> v2:172.21.15.203:3300/0 conn(0x7f97a80b1050 0x7f97a80ccfa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:48:22.965 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:22.960+0000 7f97a77fe640 1 -- 172.21.15.43:0/1254121122 <== mon.0 v1:172.21.15.43:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2218240470 0 0) 0x7f9790003520 con 0x7f97a80cd4e0 2023-08-09T13:48:22.965 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:22.960+0000 7f97a77fe640 1 -- 172.21.15.43:0/1254121122 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f97a815f7b0 msgr2=0x7f97a80b4760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-09T13:48:22.965 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:22.960+0000 7f97a77fe640 1 --2- 172.21.15.43:0/1254121122 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f97a815f7b0 0x7f97a80b4760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:48:22.965 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:22.960+0000 7f97a77fe640 1 -- 172.21.15.43:0/1254121122 >> v2:172.21.15.203:3300/0 conn(0x7f97a80b1050 msgr2=0x7f97a80ccfa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-09T13:48:22.965 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:22.960+0000 7f97a77fe640 1 --2- 172.21.15.43:0/1254121122 >> v2:172.21.15.203:3300/0 conn(0x7f97a80b1050 0x7f97a80ccfa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:48:22.965 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:22.960+0000 7f97a77fe640 1 -- 172.21.15.43:0/1254121122 --> v1:172.21.15.43:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f97a80d0370 con 0x7f97a80cd4e0 2023-08-09T13:48:22.966 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:22.960+0000 7f97a77fe640 1 -- 172.21.15.43:0/1254121122 <== mon.0 v1:172.21.15.43:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1285150740 0 0) 0x7f97a0001800 con 0x7f97a80cd4e0 2023-08-09T13:48:22.966 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:22.960+0000 7f97a77fe640 1 -- 172.21.15.43:0/1254121122 >> v1:172.21.15.43:6789/0 conn(0x7f97a80cd4e0 legacy=0x7f97a80ceae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:48:22.966 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:22.960+0000 7f97a77fe640 1 --2- 172.21.15.43:0/1254121122 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f9790003970 0x7f9790003d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:48:22.966 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:22.960+0000 7f97a77fe640 1 --2- 172.21.15.43:0/1254121122 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f97900047c0 0x7f9790004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:48:22.966 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:22.960+0000 7f97a77fe640 1 --2- 172.21.15.43:0/1254121122 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f9790005180 0x7f979000b980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:48:22.966 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:22.960+0000 7f97a77fe640 1 -- 172.21.15.43:0/1254121122 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f97a80b7bf0 con 0x7f9790003970 2023-08-09T13:48:22.967 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:22.960+0000 7f97a77fe640 1 -- 172.21.15.43:0/1254121122 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f97a80b7ed0 con 0x7f9790005180 2023-08-09T13:48:22.967 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:22.960+0000 7f97a77fe640 1 -- 172.21.15.43:0/1254121122 --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f979000bfc0 con 0x7f97900047c0 2023-08-09T13:48:22.967 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:22.960+0000 7f97acb42640 1 --2- 172.21.15.43:0/1254121122 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f97a815f7b0 0x7f97a80b4760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-08-09T13:48:22.967 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:22.960+0000 7f97acb42640 1 --2- 172.21.15.43:0/1254121122 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f9790003970 0x7f9790003d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:48:22.967 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:22.961+0000 7f97ad343640 1 --2- 172.21.15.43:0/1254121122 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f9790005180 0x7f979000b980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:48:22.967 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:22.961+0000 7f97a7fff640 1 --2- 172.21.15.43:0/1254121122 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f97900047c0 0x7f9790004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:48:22.968 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:22.961+0000 7f97acb42640 1 -- 172.21.15.43:0/1254121122 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f97900047c0 msgr2=0x7f9790004c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-09T13:48:22.968 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:22.961+0000 7f97acb42640 1 --2- 172.21.15.43:0/1254121122 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f97900047c0 0x7f9790004c40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:48:22.968 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:22.961+0000 7f97acb42640 1 -- 172.21.15.43:0/1254121122 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f9790005180 msgr2=0x7f979000b980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-09T13:48:22.968 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:22.961+0000 7f97acb42640 1 --2- 172.21.15.43:0/1254121122 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f9790005180 0x7f979000b980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:48:22.968 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:22.961+0000 7f97acb42640 1 -- 172.21.15.43:0/1254121122 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9790003520 con 0x7f9790003970 2023-08-09T13:48:22.970 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:22.962+0000 7f97acb42640 1 --2- 172.21.15.43:0/1254121122 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f9790003970 0x7f9790003d90 secure :-1 s=READY pgs=857 cs=0 l=1 rev1=1 crypto rx=0x7f9798009f00 tx=0x7f97900057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e4e32db96a149f64 server_cookie=0 in_seq=0 out_seq=0 2023-08-09T13:48:22.971 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:22.962+0000 7f97a77fe640 1 -- 172.21.15.43:0/1254121122 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f979802cc70 con 0x7f9790003970 2023-08-09T13:48:22.971 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:22.962+0000 7f97a77fe640 1 -- 172.21.15.43:0/1254121122 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f979802cdd0 con 0x7f9790003970 2023-08-09T13:48:22.971 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:22.962+0000 7f97a77fe640 1 -- 172.21.15.43:0/1254121122 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9798003bb0 con 0x7f9790003970 2023-08-09T13:48:22.971 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:22.963+0000 7f97adb44640 1 -- 172.21.15.43:0/1254121122 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f9790003970 msgr2=0x7f9790003d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:48:22.971 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:22.963+0000 7f97adb44640 1 --2- 172.21.15.43:0/1254121122 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f9790003970 0x7f9790003d90 secure :-1 s=READY pgs=857 cs=0 l=1 rev1=1 crypto rx=0x7f9798009f00 tx=0x7f97900057f0 comp rx=0 tx=0).stop 2023-08-09T13:48:22.972 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:22.963+0000 7f97ad343640 1 -- 172.21.15.43:0/1254121122 reap_dead start 2023-08-09T13:48:22.972 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:22.963+0000 7f97adb44640 1 -- 172.21.15.43:0/1254121122 shutdown_connections 2023-08-09T13:48:22.972 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:22.963+0000 7f97adb44640 1 -- 172.21.15.43:0/1254121122 >> 172.21.15.43:0/1254121122 conn(0x7f97a806a2b0 msgr2=0x7f97a806a690 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-09T13:48:22.972 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:22.963+0000 7f97adb44640 1 -- 172.21.15.43:0/1254121122 shutdown_connections 2023-08-09T13:48:22.972 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:22.963+0000 7f97adb44640 1 -- 172.21.15.43:0/1254121122 wait complete. 2023-08-09T13:48:22.972 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:22.965+0000 7f97adb44640 1 Processor -- start 2023-08-09T13:48:22.973 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:22.965+0000 7f97adb44640 1 -- start start 2023-08-09T13:48:22.973 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:22.966+0000 7f97adb44640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f9790003970 0x7f97a81539a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:48:22.973 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:22.966+0000 7f97adb44640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f97a81f2820 0x7f97a80c7ca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:48:22.973 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:22.966+0000 7f97adb44640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f97a80b1b60 0x7f97a80d2790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:48:22.973 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:22.966+0000 7f97adb44640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f97a81f2d90 con 0x7f9790003970 2023-08-09T13:48:22.973 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:22.966+0000 7f97adb44640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f97a8152530 con 0x7f97a81f2820 2023-08-09T13:48:22.974 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:22.966+0000 7f97adb44640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f97a80d19d0 con 0x7f97a80b1b60 2023-08-09T13:48:22.974 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:22.967+0000 7f97acb42640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f9790003970 0x7f97a81539a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:48:22.974 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:22.967+0000 7f97acb42640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f9790003970 0x7f97a81539a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:36854/0 (socket says 172.21.15.43:36854) 2023-08-09T13:48:22.974 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:22.967+0000 7f97acb42640 1 -- 172.21.15.43:0/415732664 learned_addr learned my addr 172.21.15.43:0/415732664 (peer_addr_for_me v2:172.21.15.43:0/0) 2023-08-09T13:48:22.974 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:22.967+0000 7f97a7fff640 1 --2- 172.21.15.43:0/415732664 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f97a81f2820 0x7f97a80c7ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:48:22.974 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:22.967+0000 7f97ad343640 1 --2- 172.21.15.43:0/415732664 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f97a80b1b60 0x7f97a80d2790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:48:22.975 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:22.967+0000 7f97acb42640 1 -- 172.21.15.43:0/415732664 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f97a80b1b60 msgr2=0x7f97a80d2790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:48:22.975 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:22.967+0000 7f97acb42640 1 --2- 172.21.15.43:0/415732664 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f97a80b1b60 0x7f97a80d2790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:48:22.975 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:22.967+0000 7f97acb42640 1 -- 172.21.15.43:0/415732664 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f97a81f2820 msgr2=0x7f97a80c7ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:48:22.975 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:22.967+0000 7f97acb42640 1 --2- 172.21.15.43:0/415732664 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f97a81f2820 0x7f97a80c7ca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:48:22.975 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:22.967+0000 7f97acb42640 1 -- 172.21.15.43:0/415732664 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f97a80d1cb0 con 0x7f9790003970 2023-08-09T13:48:22.975 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:22.967+0000 7f97acb42640 1 --2- 172.21.15.43:0/415732664 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f9790003970 0x7f97a81539a0 secure :-1 s=READY pgs=858 cs=0 l=1 rev1=1 crypto rx=0x7f9798007360 tx=0x7f9798004ab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-09T13:48:22.976 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:22.968+0000 7f97a57fa640 1 -- 172.21.15.43:0/415732664 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f979803c070 con 0x7f9790003970 2023-08-09T13:48:22.976 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:22.968+0000 7f97a57fa640 1 -- 172.21.15.43:0/415732664 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f97980045f0 con 0x7f9790003970 2023-08-09T13:48:22.976 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:22.968+0000 7f97a57fa640 1 -- 172.21.15.43:0/415732664 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9798037460 con 0x7f9790003970 2023-08-09T13:48:22.976 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:22.968+0000 7f97adb44640 1 -- 172.21.15.43:0/415732664 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f97a80d1f90 con 0x7f9790003970 2023-08-09T13:48:22.976 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:22.968+0000 7f97adb44640 1 -- 172.21.15.43:0/415732664 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f97a8153ee0 con 0x7f9790003970 2023-08-09T13:48:22.977 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:22.969+0000 7f97adb44640 1 -- 172.21.15.43:0/415732664 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f97a80739f0 con 0x7f9790003970 2023-08-09T13:48:22.977 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:22.970+0000 7f97a57fa640 1 -- 172.21.15.43:0/415732664 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 18) v1 ==== 88443+0+0 (secure 0 0 0) 0x7f97980375c0 con 0x7f9790003970 2023-08-09T13:48:22.977 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:22.971+0000 7f97a57fa640 1 --2- 172.21.15.43:0/415732664 >> [v2:172.21.15.203:6832/272917756,v1:172.21.15.203:6833/272917756] conn(0x7f979406a1a0 0x7f979406c660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:48:22.978 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:22.971+0000 7f97a57fa640 1 -- 172.21.15.43:0/415732664 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(74..74 src has 1..74) v4 ==== 6426+0+0 (secure 0 0 0) 0x7f97980a9e90 con 0x7f9790003970 2023-08-09T13:48:22.978 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:22.972+0000 7f97a7fff640 1 --2- 172.21.15.43:0/415732664 >> [v2:172.21.15.203:6832/272917756,v1:172.21.15.203:6833/272917756] conn(0x7f979406a1a0 0x7f979406c660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:48:22.978 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:22.973+0000 7f97a7fff640 1 --2- 172.21.15.43:0/415732664 >> [v2:172.21.15.203:6832/272917756,v1:172.21.15.203:6833/272917756] conn(0x7f979406a1a0 0x7f979406c660 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f97a815c920 tx=0x7f979c008040 comp rx=0 tx=0).ready entity=mgr.5533 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-09T13:48:22.978 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:22.975+0000 7f97a57fa640 1 -- 172.21.15.43:0/415732664 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178066 (secure 0 0 0) 0x7f97980752f0 con 0x7f9790003970 2023-08-09T13:48:23.167 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:23.165+0000 7f97adb44640 1 -- 172.21.15.43:0/415732664 --> [v2:172.21.15.203:6832/272917756,v1:172.21.15.203:6833/272917756] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f97a80d0930 con 0x7f979406a1a0 2023-08-09T13:48:23.191 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:23.188+0000 7f97a57fa640 1 -- 172.21.15.43:0/415732664 <== mgr.5533 v2:172.21.15.203:6832/272917756 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+308467 (secure 0 0 0) 0x7f979c00cd70 con 0x7f979406a1a0 2023-08-09T13:48:23.193 INFO:teuthology.orchestra.run.smithi043.stdout: 2023-08-09T13:48:23.196 INFO:teuthology.orchestra.run.smithi043.stderr:dumped all 2023-08-09T13:48:23.198 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:23.195+0000 7f97adb44640 1 -- 172.21.15.43:0/415732664 >> [v2:172.21.15.203:6832/272917756,v1:172.21.15.203:6833/272917756] conn(0x7f979406a1a0 msgr2=0x7f979406c660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:48:23.198 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:23.195+0000 7f97adb44640 1 --2- 172.21.15.43:0/415732664 >> [v2:172.21.15.203:6832/272917756,v1:172.21.15.203:6833/272917756] conn(0x7f979406a1a0 0x7f979406c660 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f97a815c920 tx=0x7f979c008040 comp rx=0 tx=0).stop 2023-08-09T13:48:23.198 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:23.195+0000 7f97adb44640 1 -- 172.21.15.43:0/415732664 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f9790003970 msgr2=0x7f97a81539a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:48:23.198 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:23.195+0000 7f97adb44640 1 --2- 172.21.15.43:0/415732664 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f9790003970 0x7f97a81539a0 secure :-1 s=READY pgs=858 cs=0 l=1 rev1=1 crypto rx=0x7f9798007360 tx=0x7f9798004ab0 comp rx=0 tx=0).stop 2023-08-09T13:48:23.199 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:23.196+0000 7f97adb44640 1 -- 172.21.15.43:0/415732664 shutdown_connections 2023-08-09T13:48:23.199 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:23.196+0000 7f97adb44640 1 --2- 172.21.15.43:0/415732664 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f97a80b1b60 0x7f97a80d2790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:48:23.199 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:23.196+0000 7f97adb44640 1 --2- 172.21.15.43:0/415732664 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f97a81f2820 0x7f97a80c7ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:48:23.199 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:23.196+0000 7f97adb44640 1 --2- 172.21.15.43:0/415732664 >> [v2:172.21.15.203:6832/272917756,v1:172.21.15.203:6833/272917756] conn(0x7f979406a1a0 0x7f979406c660 unknown :-1 s=CLOSED pgs=66 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:48:23.199 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:23.196+0000 7f97adb44640 1 --2- 172.21.15.43:0/415732664 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f9790003970 0x7f97a81539a0 unknown :-1 s=CLOSED pgs=858 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:48:23.199 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:23.196+0000 7f97adb44640 1 -- 172.21.15.43:0/415732664 >> 172.21.15.43:0/415732664 conn(0x7f97a806a2b0 msgr2=0x7f97a80c8700 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-09T13:48:23.200 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:23.196+0000 7f97adb44640 1 -- 172.21.15.43:0/415732664 shutdown_connections 2023-08-09T13:48:23.200 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:23.196+0000 7f97adb44640 1 -- 172.21.15.43:0/415732664 wait complete. 2023-08-09T13:48:23.218 INFO:teuthology.orchestra.run.smithi043.stdout:{"pg_ready":true,"pg_map":{"version":28,"stamp":"2023-08-09T13:48:21.256543+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":23,"num_object_clones":0,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":0,"num_read":308,"num_read_kb":512,"num_write":243,"num_write_kb":2430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":177,"ondisk_log_size":177,"up":210,"acting":210,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":61,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":269144,"kb_used_data":7904,"kb_used_omap":13,"kb_used_meta":261106,"kb_avail":754705576,"statfs":{"total":773094113280,"available":772818509824,"internally_reserved":0,"allocated":8093696,"data_stored":4822022,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":14248,"internal_metadata":267372632},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":37,"apply_latency_ms":37,"commit_latency_ns":37000000,"apply_latency_ns":37000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"5.167888"},"pg_stats":[{"pgid":"9.3f","version":"0'0","reported_seq":9,"reported_epoch":73,"state":"active+clean","last_fresh":"2023-08-09T13:48:18.158396+0000","last_change":"2023-08-09T13:48:18.152837+0000","last_active":"2023-08-09T13:48:18.158396+0000","last_peered":"2023-08-09T13:48:18.158396+0000","last_clean":"2023-08-09T13:48:18.158396+0000","last_became_active":"2023-08-09T13:48:18.152573+0000","last_became_peered":"2023-08-09T13:48:18.152573+0000","last_unstale":"2023-08-09T13:48:18.158396+0000","last_undegraded":"2023-08-09T13:48:18.158396+0000","last_fullsized":"2023-08-09T13:48:18.158396+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:48:17.087857+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:48:17.087857+0000","last_clean_scrub_stamp":"2023-08-09T13:48:17.087857+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T14:47:18.483275+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.3e","version":"0'0","reported_seq":9,"reported_epoch":73,"state":"active+clean","last_fresh":"2023-08-09T13:48:18.157639+0000","last_change":"2023-08-09T13:48:18.155184+0000","last_active":"2023-08-09T13:48:18.157639+0000","last_peered":"2023-08-09T13:48:18.157639+0000","last_clean":"2023-08-09T13:48:18.157639+0000","last_became_active":"2023-08-09T13:48:18.154907+0000","last_became_peered":"2023-08-09T13:48:18.154907+0000","last_unstale":"2023-08-09T13:48:18.157639+0000","last_undegraded":"2023-08-09T13:48:18.157639+0000","last_fullsized":"2023-08-09T13:48:18.157639+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:48:17.087857+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:48:17.087857+0000","last_clean_scrub_stamp":"2023-08-09T13:48:17.087857+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T15:37:16.143199+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.3d","version":"0'0","reported_seq":9,"reported_epoch":73,"state":"active+clean","last_fresh":"2023-08-09T13:48:18.151515+0000","last_change":"2023-08-09T13:48:18.146062+0000","last_active":"2023-08-09T13:48:18.151515+0000","last_peered":"2023-08-09T13:48:18.151515+0000","last_clean":"2023-08-09T13:48:18.151515+0000","last_became_active":"2023-08-09T13:48:18.145368+0000","last_became_peered":"2023-08-09T13:48:18.145368+0000","last_unstale":"2023-08-09T13:48:18.151515+0000","last_undegraded":"2023-08-09T13:48:18.151515+0000","last_fullsized":"2023-08-09T13:48:18.151515+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:48:17.087857+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:48:17.087857+0000","last_clean_scrub_stamp":"2023-08-09T13:48:17.087857+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-11T00:38:33.875100+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.3c","version":"0'0","reported_seq":9,"reported_epoch":73,"state":"active+clean","last_fresh":"2023-08-09T13:48:18.151811+0000","last_change":"2023-08-09T13:48:18.146500+0000","last_active":"2023-08-09T13:48:18.151811+0000","last_peered":"2023-08-09T13:48:18.151811+0000","last_clean":"2023-08-09T13:48:18.151811+0000","last_became_active":"2023-08-09T13:48:18.146233+0000","last_became_peered":"2023-08-09T13:48:18.146233+0000","last_unstale":"2023-08-09T13:48:18.151811+0000","last_undegraded":"2023-08-09T13:48:18.151811+0000","last_fullsized":"2023-08-09T13:48:18.151811+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:48:17.087857+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:48:17.087857+0000","last_clean_scrub_stamp":"2023-08-09T13:48:17.087857+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-11T00:25:21.491809+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.3b","version":"0'0","reported_seq":9,"reported_epoch":73,"state":"active+clean","last_fresh":"2023-08-09T13:48:18.155506+0000","last_change":"2023-08-09T13:48:18.143602+0000","last_active":"2023-08-09T13:48:18.155506+0000","last_peered":"2023-08-09T13:48:18.155506+0000","last_clean":"2023-08-09T13:48:18.155506+0000","last_became_active":"2023-08-09T13:48:18.143312+0000","last_became_peered":"2023-08-09T13:48:18.143312+0000","last_unstale":"2023-08-09T13:48:18.155506+0000","last_undegraded":"2023-08-09T13:48:18.155506+0000","last_fullsized":"2023-08-09T13:48:18.155506+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:48:17.087857+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:48:17.087857+0000","last_clean_scrub_stamp":"2023-08-09T13:48:17.087857+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-11T01:42:07.858900+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.3a","version":"0'0","reported_seq":9,"reported_epoch":73,"state":"active+clean","last_fresh":"2023-08-09T13:48:18.158645+0000","last_change":"2023-08-09T13:48:18.144574+0000","last_active":"2023-08-09T13:48:18.158645+0000","last_peered":"2023-08-09T13:48:18.158645+0000","last_clean":"2023-08-09T13:48:18.158645+0000","last_became_active":"2023-08-09T13:48:18.144221+0000","last_became_peered":"2023-08-09T13:48:18.144221+0000","last_unstale":"2023-08-09T13:48:18.158645+0000","last_undegraded":"2023-08-09T13:48:18.158645+0000","last_fullsized":"2023-08-09T13:48:18.158645+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:48:17.087857+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:48:17.087857+0000","last_clean_scrub_stamp":"2023-08-09T13:48:17.087857+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T22:24:54.272226+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.39","version":"0'0","reported_seq":9,"reported_epoch":73,"state":"active+clean","last_fresh":"2023-08-09T13:48:18.144564+0000","last_change":"2023-08-09T13:48:18.142714+0000","last_active":"2023-08-09T13:48:18.144564+0000","last_peered":"2023-08-09T13:48:18.144564+0000","last_clean":"2023-08-09T13:48:18.144564+0000","last_became_active":"2023-08-09T13:48:18.141836+0000","last_became_peered":"2023-08-09T13:48:18.141836+0000","last_unstale":"2023-08-09T13:48:18.144564+0000","last_undegraded":"2023-08-09T13:48:18.144564+0000","last_fullsized":"2023-08-09T13:48:18.144564+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:48:17.087857+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:48:17.087857+0000","last_clean_scrub_stamp":"2023-08-09T13:48:17.087857+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T22:54:27.458384+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.38","version":"0'0","reported_seq":9,"reported_epoch":73,"state":"active+clean","last_fresh":"2023-08-09T13:48:18.156563+0000","last_change":"2023-08-09T13:48:18.151342+0000","last_active":"2023-08-09T13:48:18.156563+0000","last_peered":"2023-08-09T13:48:18.156563+0000","last_clean":"2023-08-09T13:48:18.156563+0000","last_became_active":"2023-08-09T13:48:18.151000+0000","last_became_peered":"2023-08-09T13:48:18.151000+0000","last_unstale":"2023-08-09T13:48:18.156563+0000","last_undegraded":"2023-08-09T13:48:18.156563+0000","last_fullsized":"2023-08-09T13:48:18.156563+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:48:17.087857+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:48:17.087857+0000","last_clean_scrub_stamp":"2023-08-09T13:48:17.087857+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T22:58:06.877210+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.37","version":"0'0","reported_seq":9,"reported_epoch":73,"state":"active+clean","last_fresh":"2023-08-09T13:48:18.148105+0000","last_change":"2023-08-09T13:48:18.146306+0000","last_active":"2023-08-09T13:48:18.148105+0000","last_peered":"2023-08-09T13:48:18.148105+0000","last_clean":"2023-08-09T13:48:18.148105+0000","last_became_active":"2023-08-09T13:48:18.145883+0000","last_became_peered":"2023-08-09T13:48:18.145883+0000","last_unstale":"2023-08-09T13:48:18.148105+0000","last_undegraded":"2023-08-09T13:48:18.148105+0000","last_fullsized":"2023-08-09T13:48:18.148105+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:48:17.087857+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:48:17.087857+0000","last_clean_scrub_stamp":"2023-08-09T13:48:17.087857+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T17:36:56.420913+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.36","version":"0'0","reported_seq":9,"reported_epoch":73,"state":"active+clean","last_fresh":"2023-08-09T13:48:18.143817+0000","last_change":"2023-08-09T13:48:18.127654+0000","last_active":"2023-08-09T13:48:18.143817+0000","last_peered":"2023-08-09T13:48:18.143817+0000","last_clean":"2023-08-09T13:48:18.143817+0000","last_became_active":"2023-08-09T13:48:18.126751+0000","last_became_peered":"2023-08-09T13:48:18.126751+0000","last_unstale":"2023-08-09T13:48:18.143817+0000","last_undegraded":"2023-08-09T13:48:18.143817+0000","last_fullsized":"2023-08-09T13:48:18.143817+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:48:17.087857+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:48:17.087857+0000","last_clean_scrub_stamp":"2023-08-09T13:48:17.087857+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T22:03:13.433435+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.35","version":"0'0","reported_seq":9,"reported_epoch":73,"state":"active+clean","last_fresh":"2023-08-09T13:48:18.159348+0000","last_change":"2023-08-09T13:48:18.154916+0000","last_active":"2023-08-09T13:48:18.159348+0000","last_peered":"2023-08-09T13:48:18.159348+0000","last_clean":"2023-08-09T13:48:18.159348+0000","last_became_active":"2023-08-09T13:48:18.154656+0000","last_became_peered":"2023-08-09T13:48:18.154656+0000","last_unstale":"2023-08-09T13:48:18.159348+0000","last_undegraded":"2023-08-09T13:48:18.159348+0000","last_fullsized":"2023-08-09T13:48:18.159348+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:48:17.087857+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:48:17.087857+0000","last_clean_scrub_stamp":"2023-08-09T13:48:17.087857+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T18:52:15.342531+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.34","version":"0'0","reported_seq":9,"reported_epoch":73,"state":"active+clean","last_fresh":"2023-08-09T13:48:18.141946+0000","last_change":"2023-08-09T13:48:18.139766+0000","last_active":"2023-08-09T13:48:18.141946+0000","last_peered":"2023-08-09T13:48:18.141946+0000","last_clean":"2023-08-09T13:48:18.141946+0000","last_became_active":"2023-08-09T13:48:18.139484+0000","last_became_peered":"2023-08-09T13:48:18.139484+0000","last_unstale":"2023-08-09T13:48:18.141946+0000","last_undegraded":"2023-08-09T13:48:18.141946+0000","last_fullsized":"2023-08-09T13:48:18.141946+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:48:17.087857+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:48:17.087857+0000","last_clean_scrub_stamp":"2023-08-09T13:48:17.087857+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T23:01:48.230063+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.33","version":"0'0","reported_seq":9,"reported_epoch":73,"state":"active+clean","last_fresh":"2023-08-09T13:48:18.149292+0000","last_change":"2023-08-09T13:48:18.147141+0000","last_active":"2023-08-09T13:48:18.149292+0000","last_peered":"2023-08-09T13:48:18.149292+0000","last_clean":"2023-08-09T13:48:18.149292+0000","last_became_active":"2023-08-09T13:48:18.146762+0000","last_became_peered":"2023-08-09T13:48:18.146762+0000","last_unstale":"2023-08-09T13:48:18.149292+0000","last_undegraded":"2023-08-09T13:48:18.149292+0000","last_fullsized":"2023-08-09T13:48:18.149292+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:48:17.087857+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:48:17.087857+0000","last_clean_scrub_stamp":"2023-08-09T13:48:17.087857+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T20:22:59.125660+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.32","version":"0'0","reported_seq":9,"reported_epoch":73,"state":"active+clean","last_fresh":"2023-08-09T13:48:18.147828+0000","last_change":"2023-08-09T13:48:18.146755+0000","last_active":"2023-08-09T13:48:18.147828+0000","last_peered":"2023-08-09T13:48:18.147828+0000","last_clean":"2023-08-09T13:48:18.147828+0000","last_became_active":"2023-08-09T13:48:18.146476+0000","last_became_peered":"2023-08-09T13:48:18.146476+0000","last_unstale":"2023-08-09T13:48:18.147828+0000","last_undegraded":"2023-08-09T13:48:18.147828+0000","last_fullsized":"2023-08-09T13:48:18.147828+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:48:17.087857+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:48:17.087857+0000","last_clean_scrub_stamp":"2023-08-09T13:48:17.087857+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T19:48:03.345033+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.31","version":"0'0","reported_seq":9,"reported_epoch":73,"state":"active+clean","last_fresh":"2023-08-09T13:48:18.152536+0000","last_change":"2023-08-09T13:48:18.151585+0000","last_active":"2023-08-09T13:48:18.152536+0000","last_peered":"2023-08-09T13:48:18.152536+0000","last_clean":"2023-08-09T13:48:18.152536+0000","last_became_active":"2023-08-09T13:48:18.151285+0000","last_became_peered":"2023-08-09T13:48:18.151285+0000","last_unstale":"2023-08-09T13:48:18.152536+0000","last_undegraded":"2023-08-09T13:48:18.152536+0000","last_fullsized":"2023-08-09T13:48:18.152536+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:48:17.087857+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:48:17.087857+0000","last_clean_scrub_stamp":"2023-08-09T13:48:17.087857+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T21:51:35.876003+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.30","version":"0'0","reported_seq":9,"reported_epoch":73,"state":"active+clean","last_fresh":"2023-08-09T13:48:18.156188+0000","last_change":"2023-08-09T13:48:18.140225+0000","last_active":"2023-08-09T13:48:18.156188+0000","last_peered":"2023-08-09T13:48:18.156188+0000","last_clean":"2023-08-09T13:48:18.156188+0000","last_became_active":"2023-08-09T13:48:18.139432+0000","last_became_peered":"2023-08-09T13:48:18.139432+0000","last_unstale":"2023-08-09T13:48:18.156188+0000","last_undegraded":"2023-08-09T13:48:18.156188+0000","last_fullsized":"2023-08-09T13:48:18.156188+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:48:17.087857+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:48:17.087857+0000","last_clean_scrub_stamp":"2023-08-09T13:48:17.087857+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T21:49:05.091868+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.2f","version":"0'0","reported_seq":9,"reported_epoch":73,"state":"active+clean","last_fresh":"2023-08-09T13:48:18.158014+0000","last_change":"2023-08-09T13:48:18.153848+0000","last_active":"2023-08-09T13:48:18.158014+0000","last_peered":"2023-08-09T13:48:18.158014+0000","last_clean":"2023-08-09T13:48:18.158014+0000","last_became_active":"2023-08-09T13:48:18.153548+0000","last_became_peered":"2023-08-09T13:48:18.153548+0000","last_unstale":"2023-08-09T13:48:18.158014+0000","last_undegraded":"2023-08-09T13:48:18.158014+0000","last_fullsized":"2023-08-09T13:48:18.158014+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:48:17.087857+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:48:17.087857+0000","last_clean_scrub_stamp":"2023-08-09T13:48:17.087857+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T18:38:04.068552+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.2e","version":"0'0","reported_seq":9,"reported_epoch":73,"state":"active+clean","last_fresh":"2023-08-09T13:48:18.151978+0000","last_change":"2023-08-09T13:48:18.150438+0000","last_active":"2023-08-09T13:48:18.151978+0000","last_peered":"2023-08-09T13:48:18.151978+0000","last_clean":"2023-08-09T13:48:18.151978+0000","last_became_active":"2023-08-09T13:48:18.150136+0000","last_became_peered":"2023-08-09T13:48:18.150136+0000","last_unstale":"2023-08-09T13:48:18.151978+0000","last_undegraded":"2023-08-09T13:48:18.151978+0000","last_fullsized":"2023-08-09T13:48:18.151978+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:48:17.087857+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:48:17.087857+0000","last_clean_scrub_stamp":"2023-08-09T13:48:17.087857+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T17:55:28.354834+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.2d","version":"0'0","reported_seq":9,"reported_epoch":73,"state":"active+clean","last_fresh":"2023-08-09T13:48:18.158447+0000","last_change":"2023-08-09T13:48:18.151532+0000","last_active":"2023-08-09T13:48:18.158447+0000","last_peered":"2023-08-09T13:48:18.158447+0000","last_clean":"2023-08-09T13:48:18.158447+0000","last_became_active":"2023-08-09T13:48:18.151242+0000","last_became_peered":"2023-08-09T13:48:18.151242+0000","last_unstale":"2023-08-09T13:48:18.158447+0000","last_undegraded":"2023-08-09T13:48:18.158447+0000","last_fullsized":"2023-08-09T13:48:18.158447+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:48:17.087857+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:48:17.087857+0000","last_clean_scrub_stamp":"2023-08-09T13:48:17.087857+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-11T01:28:23.557859+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.2c","version":"0'0","reported_seq":9,"reported_epoch":73,"state":"active+clean","last_fresh":"2023-08-09T13:48:18.159547+0000","last_change":"2023-08-09T13:48:18.156432+0000","last_active":"2023-08-09T13:48:18.159547+0000","last_peered":"2023-08-09T13:48:18.159547+0000","last_clean":"2023-08-09T13:48:18.159547+0000","last_became_active":"2023-08-09T13:48:18.156107+0000","last_became_peered":"2023-08-09T13:48:18.156107+0000","last_unstale":"2023-08-09T13:48:18.159547+0000","last_undegraded":"2023-08-09T13:48:18.159547+0000","last_fullsized":"2023-08-09T13:48:18.159547+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:48:17.087857+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:48:17.087857+0000","last_clean_scrub_stamp":"2023-08-09T13:48:17.087857+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T14:49:40.094879+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.2b","version":"0'0","reported_seq":9,"reported_epoch":73,"state":"active+clean","last_fresh":"2023-08-09T13:48:18.145479+0000","last_change":"2023-08-09T13:48:18.139749+0000","last_active":"2023-08-09T13:48:18.145479+0000","last_peered":"2023-08-09T13:48:18.145479+0000","last_clean":"2023-08-09T13:48:18.145479+0000","last_became_active":"2023-08-09T13:48:18.139434+0000","last_became_peered":"2023-08-09T13:48:18.139434+0000","last_unstale":"2023-08-09T13:48:18.145479+0000","last_undegraded":"2023-08-09T13:48:18.145479+0000","last_fullsized":"2023-08-09T13:48:18.145479+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:48:17.087857+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:48:17.087857+0000","last_clean_scrub_stamp":"2023-08-09T13:48:17.087857+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T15:10:50.771773+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.2a","version":"0'0","reported_seq":9,"reported_epoch":73,"state":"active+clean","last_fresh":"2023-08-09T13:48:18.157368+0000","last_change":"2023-08-09T13:48:18.152437+0000","last_active":"2023-08-09T13:48:18.157368+0000","last_peered":"2023-08-09T13:48:18.157368+0000","last_clean":"2023-08-09T13:48:18.157368+0000","last_became_active":"2023-08-09T13:48:18.151855+0000","last_became_peered":"2023-08-09T13:48:18.151855+0000","last_unstale":"2023-08-09T13:48:18.157368+0000","last_undegraded":"2023-08-09T13:48:18.157368+0000","last_fullsized":"2023-08-09T13:48:18.157368+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:48:17.087857+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:48:17.087857+0000","last_clean_scrub_stamp":"2023-08-09T13:48:17.087857+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T16:46:30.811573+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.29","version":"0'0","reported_seq":9,"reported_epoch":73,"state":"active+clean","last_fresh":"2023-08-09T13:48:18.155696+0000","last_change":"2023-08-09T13:48:18.154494+0000","last_active":"2023-08-09T13:48:18.155696+0000","last_peered":"2023-08-09T13:48:18.155696+0000","last_clean":"2023-08-09T13:48:18.155696+0000","last_became_active":"2023-08-09T13:48:18.154151+0000","last_became_peered":"2023-08-09T13:48:18.154151+0000","last_unstale":"2023-08-09T13:48:18.155696+0000","last_undegraded":"2023-08-09T13:48:18.155696+0000","last_fullsized":"2023-08-09T13:48:18.155696+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:48:17.087857+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:48:17.087857+0000","last_clean_scrub_stamp":"2023-08-09T13:48:17.087857+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T16:12:49.856088+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.28","version":"0'0","reported_seq":9,"reported_epoch":73,"state":"active+clean","last_fresh":"2023-08-09T13:48:18.143861+0000","last_change":"2023-08-09T13:48:18.140604+0000","last_active":"2023-08-09T13:48:18.143861+0000","last_peered":"2023-08-09T13:48:18.143861+0000","last_clean":"2023-08-09T13:48:18.143861+0000","last_became_active":"2023-08-09T13:48:18.140183+0000","last_became_peered":"2023-08-09T13:48:18.140183+0000","last_unstale":"2023-08-09T13:48:18.143861+0000","last_undegraded":"2023-08-09T13:48:18.143861+0000","last_fullsized":"2023-08-09T13:48:18.143861+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:48:17.087857+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:48:17.087857+0000","last_clean_scrub_stamp":"2023-08-09T13:48:17.087857+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T23:23:02.255139+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.27","version":"0'0","reported_seq":9,"reported_epoch":73,"state":"active+clean","last_fresh":"2023-08-09T13:48:18.158367+0000","last_change":"2023-08-09T13:48:18.149483+0000","last_active":"2023-08-09T13:48:18.158367+0000","last_peered":"2023-08-09T13:48:18.158367+0000","last_clean":"2023-08-09T13:48:18.158367+0000","last_became_active":"2023-08-09T13:48:18.149185+0000","last_became_peered":"2023-08-09T13:48:18.149185+0000","last_unstale":"2023-08-09T13:48:18.158367+0000","last_undegraded":"2023-08-09T13:48:18.158367+0000","last_fullsized":"2023-08-09T13:48:18.158367+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:48:17.087857+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:48:17.087857+0000","last_clean_scrub_stamp":"2023-08-09T13:48:17.087857+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T20:38:37.445234+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.26","version":"0'0","reported_seq":9,"reported_epoch":73,"state":"active+clean","last_fresh":"2023-08-09T13:48:18.145007+0000","last_change":"2023-08-09T13:48:18.143866+0000","last_active":"2023-08-09T13:48:18.145007+0000","last_peered":"2023-08-09T13:48:18.145007+0000","last_clean":"2023-08-09T13:48:18.145007+0000","last_became_active":"2023-08-09T13:48:18.143552+0000","last_became_peered":"2023-08-09T13:48:18.143552+0000","last_unstale":"2023-08-09T13:48:18.145007+0000","last_undegraded":"2023-08-09T13:48:18.145007+0000","last_fullsized":"2023-08-09T13:48:18.145007+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:48:17.087857+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:48:17.087857+0000","last_clean_scrub_stamp":"2023-08-09T13:48:17.087857+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T14:47:35.887828+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.25","version":"0'0","reported_seq":9,"reported_epoch":73,"state":"active+clean","last_fresh":"2023-08-09T13:48:18.158812+0000","last_change":"2023-08-09T13:48:18.147805+0000","last_active":"2023-08-09T13:48:18.158812+0000","last_peered":"2023-08-09T13:48:18.158812+0000","last_clean":"2023-08-09T13:48:18.158812+0000","last_became_active":"2023-08-09T13:48:18.147104+0000","last_became_peered":"2023-08-09T13:48:18.147104+0000","last_unstale":"2023-08-09T13:48:18.158812+0000","last_undegraded":"2023-08-09T13:48:18.158812+0000","last_fullsized":"2023-08-09T13:48:18.158812+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:48:17.087857+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:48:17.087857+0000","last_clean_scrub_stamp":"2023-08-09T13:48:17.087857+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T21:51:35.876003+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.24","version":"0'0","reported_seq":9,"reported_epoch":73,"state":"active+clean","last_fresh":"2023-08-09T13:48:18.145059+0000","last_change":"2023-08-09T13:48:18.139602+0000","last_active":"2023-08-09T13:48:18.145059+0000","last_peered":"2023-08-09T13:48:18.145059+0000","last_clean":"2023-08-09T13:48:18.145059+0000","last_became_active":"2023-08-09T13:48:18.139295+0000","last_became_peered":"2023-08-09T13:48:18.139295+0000","last_unstale":"2023-08-09T13:48:18.145059+0000","last_undegraded":"2023-08-09T13:48:18.145059+0000","last_fullsized":"2023-08-09T13:48:18.145059+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:48:17.087857+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:48:17.087857+0000","last_clean_scrub_stamp":"2023-08-09T13:48:17.087857+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-11T01:06:40.133788+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.23","version":"0'0","reported_seq":9,"reported_epoch":73,"state":"active+clean","last_fresh":"2023-08-09T13:48:18.157221+0000","last_change":"2023-08-09T13:48:18.152809+0000","last_active":"2023-08-09T13:48:18.157221+0000","last_peered":"2023-08-09T13:48:18.157221+0000","last_clean":"2023-08-09T13:48:18.157221+0000","last_became_active":"2023-08-09T13:48:18.152487+0000","last_became_peered":"2023-08-09T13:48:18.152487+0000","last_unstale":"2023-08-09T13:48:18.157221+0000","last_undegraded":"2023-08-09T13:48:18.157221+0000","last_fullsized":"2023-08-09T13:48:18.157221+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:48:17.087857+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:48:17.087857+0000","last_clean_scrub_stamp":"2023-08-09T13:48:17.087857+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T17:48:41.690755+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.22","version":"0'0","reported_seq":9,"reported_epoch":73,"state":"active+clean","last_fresh":"2023-08-09T13:48:18.149485+0000","last_change":"2023-08-09T13:48:18.144300+0000","last_active":"2023-08-09T13:48:18.149485+0000","last_peered":"2023-08-09T13:48:18.149485+0000","last_clean":"2023-08-09T13:48:18.149485+0000","last_became_active":"2023-08-09T13:48:18.143917+0000","last_became_peered":"2023-08-09T13:48:18.143917+0000","last_unstale":"2023-08-09T13:48:18.149485+0000","last_undegraded":"2023-08-09T13:48:18.149485+0000","last_fullsized":"2023-08-09T13:48:18.149485+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:48:17.087857+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:48:17.087857+0000","last_clean_scrub_stamp":"2023-08-09T13:48:17.087857+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T18:20:16.882105+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.21","version":"0'0","reported_seq":9,"reported_epoch":73,"state":"active+clean","last_fresh":"2023-08-09T13:48:18.158292+0000","last_change":"2023-08-09T13:48:18.150149+0000","last_active":"2023-08-09T13:48:18.158292+0000","last_peered":"2023-08-09T13:48:18.158292+0000","last_clean":"2023-08-09T13:48:18.158292+0000","last_became_active":"2023-08-09T13:48:18.149859+0000","last_became_peered":"2023-08-09T13:48:18.149859+0000","last_unstale":"2023-08-09T13:48:18.158292+0000","last_undegraded":"2023-08-09T13:48:18.158292+0000","last_fullsized":"2023-08-09T13:48:18.158292+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:48:17.087857+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:48:17.087857+0000","last_clean_scrub_stamp":"2023-08-09T13:48:17.087857+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T16:44:17.531789+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.20","version":"0'0","reported_seq":9,"reported_epoch":73,"state":"active+clean","last_fresh":"2023-08-09T13:48:18.158987+0000","last_change":"2023-08-09T13:48:18.152970+0000","last_active":"2023-08-09T13:48:18.158987+0000","last_peered":"2023-08-09T13:48:18.158987+0000","last_clean":"2023-08-09T13:48:18.158987+0000","last_became_active":"2023-08-09T13:48:18.152669+0000","last_became_peered":"2023-08-09T13:48:18.152669+0000","last_unstale":"2023-08-09T13:48:18.158987+0000","last_undegraded":"2023-08-09T13:48:18.158987+0000","last_fullsized":"2023-08-09T13:48:18.158987+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:48:17.087857+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:48:17.087857+0000","last_clean_scrub_stamp":"2023-08-09T13:48:17.087857+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T18:57:02.550710+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.1e","version":"0'0","reported_seq":9,"reported_epoch":73,"state":"active+clean","last_fresh":"2023-08-09T13:48:18.154692+0000","last_change":"2023-08-09T13:48:18.141045+0000","last_active":"2023-08-09T13:48:18.154692+0000","last_peered":"2023-08-09T13:48:18.154692+0000","last_clean":"2023-08-09T13:48:18.154692+0000","last_became_active":"2023-08-09T13:48:18.139631+0000","last_became_peered":"2023-08-09T13:48:18.139631+0000","last_unstale":"2023-08-09T13:48:18.154692+0000","last_undegraded":"2023-08-09T13:48:18.154692+0000","last_fullsized":"2023-08-09T13:48:18.154692+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:48:17.087857+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:48:17.087857+0000","last_clean_scrub_stamp":"2023-08-09T13:48:17.087857+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T17:07:09.487635+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.1f","version":"73'4","reported_seq":15,"reported_epoch":73,"state":"active+clean","last_fresh":"2023-08-09T13:48:18.173947+0000","last_change":"2023-08-09T13:48:17.136238+0000","last_active":"2023-08-09T13:48:18.173947+0000","last_peered":"2023-08-09T13:48:18.173947+0000","last_clean":"2023-08-09T13:48:18.173947+0000","last_became_active":"2023-08-09T13:48:17.135961+0000","last_became_peered":"2023-08-09T13:48:17.135961+0000","last_unstale":"2023-08-09T13:48:18.173947+0000","last_undegraded":"2023-08-09T13:48:18.173947+0000","last_fullsized":"2023-08-09T13:48:18.173947+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:48:16.072766+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:48:16.072766+0000","last_clean_scrub_stamp":"2023-08-09T13:48:16.072766+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T18:40:14.841360+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.1f","version":"0'0","reported_seq":9,"reported_epoch":73,"state":"active+clean","last_fresh":"2023-08-09T13:48:18.140829+0000","last_change":"2023-08-09T13:48:18.131283+0000","last_active":"2023-08-09T13:48:18.140829+0000","last_peered":"2023-08-09T13:48:18.140829+0000","last_clean":"2023-08-09T13:48:18.140829+0000","last_became_active":"2023-08-09T13:48:18.130497+0000","last_became_peered":"2023-08-09T13:48:18.130497+0000","last_unstale":"2023-08-09T13:48:18.140829+0000","last_undegraded":"2023-08-09T13:48:18.140829+0000","last_fullsized":"2023-08-09T13:48:18.140829+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:48:17.087857+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:48:17.087857+0000","last_clean_scrub_stamp":"2023-08-09T13:48:17.087857+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T15:49:55.658613+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.1e","version":"73'1","reported_seq":12,"reported_epoch":73,"state":"active+clean","last_fresh":"2023-08-09T13:48:18.170845+0000","last_change":"2023-08-09T13:48:17.133120+0000","last_active":"2023-08-09T13:48:18.170845+0000","last_peered":"2023-08-09T13:48:18.170845+0000","last_clean":"2023-08-09T13:48:18.170845+0000","last_became_active":"2023-08-09T13:48:17.132760+0000","last_became_peered":"2023-08-09T13:48:17.132760+0000","last_unstale":"2023-08-09T13:48:18.170845+0000","last_undegraded":"2023-08-09T13:48:18.170845+0000","last_fullsized":"2023-08-09T13:48:18.170845+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:48:16.072766+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:48:16.072766+0000","last_clean_scrub_stamp":"2023-08-09T13:48:16.072766+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T23:40:53.809406+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.1c","version":"0'0","reported_seq":9,"reported_epoch":73,"state":"active+clean","last_fresh":"2023-08-09T13:48:18.145253+0000","last_change":"2023-08-09T13:48:18.133502+0000","last_active":"2023-08-09T13:48:18.145253+0000","last_peered":"2023-08-09T13:48:18.145253+0000","last_clean":"2023-08-09T13:48:18.145253+0000","last_became_active":"2023-08-09T13:48:18.132934+0000","last_became_peered":"2023-08-09T13:48:18.132934+0000","last_unstale":"2023-08-09T13:48:18.145253+0000","last_undegraded":"2023-08-09T13:48:18.145253+0000","last_fullsized":"2023-08-09T13:48:18.145253+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:48:17.087857+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:48:17.087857+0000","last_clean_scrub_stamp":"2023-08-09T13:48:17.087857+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T16:14:25.369270+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.1d","version":"73'2","reported_seq":13,"reported_epoch":73,"state":"active+clean","last_fresh":"2023-08-09T13:48:18.174000+0000","last_change":"2023-08-09T13:48:17.128733+0000","last_active":"2023-08-09T13:48:18.174000+0000","last_peered":"2023-08-09T13:48:18.174000+0000","last_clean":"2023-08-09T13:48:18.174000+0000","last_became_active":"2023-08-09T13:48:17.128462+0000","last_became_peered":"2023-08-09T13:48:17.128462+0000","last_unstale":"2023-08-09T13:48:18.174000+0000","last_undegraded":"2023-08-09T13:48:18.174000+0000","last_fullsized":"2023-08-09T13:48:18.174000+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:48:16.072766+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:48:16.072766+0000","last_clean_scrub_stamp":"2023-08-09T13:48:16.072766+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T15:33:28.704733+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.1d","version":"0'0","reported_seq":9,"reported_epoch":73,"state":"active+clean","last_fresh":"2023-08-09T13:48:18.158789+0000","last_change":"2023-08-09T13:48:18.147544+0000","last_active":"2023-08-09T13:48:18.158789+0000","last_peered":"2023-08-09T13:48:18.158789+0000","last_clean":"2023-08-09T13:48:18.158789+0000","last_became_active":"2023-08-09T13:48:18.146694+0000","last_became_peered":"2023-08-09T13:48:18.146694+0000","last_unstale":"2023-08-09T13:48:18.158789+0000","last_undegraded":"2023-08-09T13:48:18.158789+0000","last_fullsized":"2023-08-09T13:48:18.158789+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:48:17.087857+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:48:17.087857+0000","last_clean_scrub_stamp":"2023-08-09T13:48:17.087857+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-11T01:20:29.944656+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.1c","version":"73'2","reported_seq":13,"reported_epoch":73,"state":"active+clean","last_fresh":"2023-08-09T13:48:18.173560+0000","last_change":"2023-08-09T13:48:17.134613+0000","last_active":"2023-08-09T13:48:18.173560+0000","last_peered":"2023-08-09T13:48:18.173560+0000","last_clean":"2023-08-09T13:48:18.173560+0000","last_became_active":"2023-08-09T13:48:17.134264+0000","last_became_peered":"2023-08-09T13:48:17.134264+0000","last_unstale":"2023-08-09T13:48:18.173560+0000","last_undegraded":"2023-08-09T13:48:18.173560+0000","last_fullsized":"2023-08-09T13:48:18.173560+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:48:16.072766+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:48:16.072766+0000","last_clean_scrub_stamp":"2023-08-09T13:48:16.072766+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T19:50:07.010215+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.1a","version":"0'0","reported_seq":9,"reported_epoch":73,"state":"active+clean","last_fresh":"2023-08-09T13:48:18.157345+0000","last_change":"2023-08-09T13:48:18.152365+0000","last_active":"2023-08-09T13:48:18.157345+0000","last_peered":"2023-08-09T13:48:18.157345+0000","last_clean":"2023-08-09T13:48:18.157345+0000","last_became_active":"2023-08-09T13:48:18.151513+0000","last_became_peered":"2023-08-09T13:48:18.151513+0000","last_unstale":"2023-08-09T13:48:18.157345+0000","last_undegraded":"2023-08-09T13:48:18.157345+0000","last_fullsized":"2023-08-09T13:48:18.157345+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:48:17.087857+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:48:17.087857+0000","last_clean_scrub_stamp":"2023-08-09T13:48:17.087857+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T16:58:20.620035+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.1b","version":"73'1","reported_seq":12,"reported_epoch":73,"state":"active+clean","last_fresh":"2023-08-09T13:48:18.171629+0000","last_change":"2023-08-09T13:48:17.137906+0000","last_active":"2023-08-09T13:48:18.171629+0000","last_peered":"2023-08-09T13:48:18.171629+0000","last_clean":"2023-08-09T13:48:18.171629+0000","last_became_active":"2023-08-09T13:48:17.137624+0000","last_became_peered":"2023-08-09T13:48:17.137624+0000","last_unstale":"2023-08-09T13:48:18.171629+0000","last_undegraded":"2023-08-09T13:48:18.171629+0000","last_fullsized":"2023-08-09T13:48:18.171629+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:48:16.072766+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:48:16.072766+0000","last_clean_scrub_stamp":"2023-08-09T13:48:16.072766+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T19:30:54.824510+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.1b","version":"0'0","reported_seq":9,"reported_epoch":73,"state":"active+clean","last_fresh":"2023-08-09T13:48:18.160444+0000","last_change":"2023-08-09T13:48:18.157276+0000","last_active":"2023-08-09T13:48:18.160444+0000","last_peered":"2023-08-09T13:48:18.160444+0000","last_clean":"2023-08-09T13:48:18.160444+0000","last_became_active":"2023-08-09T13:48:18.157001+0000","last_became_peered":"2023-08-09T13:48:18.157001+0000","last_unstale":"2023-08-09T13:48:18.160444+0000","last_undegraded":"2023-08-09T13:48:18.160444+0000","last_fullsized":"2023-08-09T13:48:18.160444+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:48:17.087857+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:48:17.087857+0000","last_clean_scrub_stamp":"2023-08-09T13:48:17.087857+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T22:25:03.373484+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.1a","version":"73'1","reported_seq":12,"reported_epoch":73,"state":"active+clean","last_fresh":"2023-08-09T13:48:18.173513+0000","last_change":"2023-08-09T13:48:17.136309+0000","last_active":"2023-08-09T13:48:18.173513+0000","last_peered":"2023-08-09T13:48:18.173513+0000","last_clean":"2023-08-09T13:48:18.173513+0000","last_became_active":"2023-08-09T13:48:17.136046+0000","last_became_peered":"2023-08-09T13:48:17.136046+0000","last_unstale":"2023-08-09T13:48:18.173513+0000","last_undegraded":"2023-08-09T13:48:18.173513+0000","last_fullsized":"2023-08-09T13:48:18.173513+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:48:16.072766+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:48:16.072766+0000","last_clean_scrub_stamp":"2023-08-09T13:48:16.072766+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T18:49:45.672784+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.18","version":"0'0","reported_seq":9,"reported_epoch":73,"state":"active+clean","last_fresh":"2023-08-09T13:48:18.137208+0000","last_change":"2023-08-09T13:48:18.136226+0000","last_active":"2023-08-09T13:48:18.137208+0000","last_peered":"2023-08-09T13:48:18.137208+0000","last_clean":"2023-08-09T13:48:18.137208+0000","last_became_active":"2023-08-09T13:48:18.135880+0000","last_became_peered":"2023-08-09T13:48:18.135880+0000","last_unstale":"2023-08-09T13:48:18.137208+0000","last_undegraded":"2023-08-09T13:48:18.137208+0000","last_fullsized":"2023-08-09T13:48:18.137208+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:48:17.087857+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:48:17.087857+0000","last_clean_scrub_stamp":"2023-08-09T13:48:17.087857+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T16:42:47.998599+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.19","version":"0'0","reported_seq":11,"reported_epoch":73,"state":"active+clean","last_fresh":"2023-08-09T13:48:18.133469+0000","last_change":"2023-08-09T13:48:17.138354+0000","last_active":"2023-08-09T13:48:18.133469+0000","last_peered":"2023-08-09T13:48:18.133469+0000","last_clean":"2023-08-09T13:48:18.133469+0000","last_became_active":"2023-08-09T13:48:17.137887+0000","last_became_peered":"2023-08-09T13:48:17.137887+0000","last_unstale":"2023-08-09T13:48:18.133469+0000","last_undegraded":"2023-08-09T13:48:18.133469+0000","last_fullsized":"2023-08-09T13:48:18.133469+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:48:16.072766+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:48:16.072766+0000","last_clean_scrub_stamp":"2023-08-09T13:48:16.072766+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T15:00:43.613037+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.19","version":"0'0","reported_seq":9,"reported_epoch":73,"state":"active+clean","last_fresh":"2023-08-09T13:48:18.158353+0000","last_change":"2023-08-09T13:48:18.152221+0000","last_active":"2023-08-09T13:48:18.158353+0000","last_peered":"2023-08-09T13:48:18.158353+0000","last_clean":"2023-08-09T13:48:18.158353+0000","last_became_active":"2023-08-09T13:48:18.151934+0000","last_became_peered":"2023-08-09T13:48:18.151934+0000","last_unstale":"2023-08-09T13:48:18.158353+0000","last_undegraded":"2023-08-09T13:48:18.158353+0000","last_fullsized":"2023-08-09T13:48:18.158353+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:48:17.087857+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:48:17.087857+0000","last_clean_scrub_stamp":"2023-08-09T13:48:17.087857+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T23:00:27.746151+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.18","version":"0'0","reported_seq":11,"reported_epoch":73,"state":"active+clean","last_fresh":"2023-08-09T13:48:18.128990+0000","last_change":"2023-08-09T13:48:17.140981+0000","last_active":"2023-08-09T13:48:18.128990+0000","last_peered":"2023-08-09T13:48:18.128990+0000","last_clean":"2023-08-09T13:48:18.128990+0000","last_became_active":"2023-08-09T13:48:17.140712+0000","last_became_peered":"2023-08-09T13:48:17.140712+0000","last_unstale":"2023-08-09T13:48:18.128990+0000","last_undegraded":"2023-08-09T13:48:18.128990+0000","last_fullsized":"2023-08-09T13:48:18.128990+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:48:16.072766+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:48:16.072766+0000","last_clean_scrub_stamp":"2023-08-09T13:48:16.072766+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T18:57:01.535619+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.16","version":"0'0","reported_seq":9,"reported_epoch":73,"state":"active+clean","last_fresh":"2023-08-09T13:48:18.152546+0000","last_change":"2023-08-09T13:48:18.151658+0000","last_active":"2023-08-09T13:48:18.152546+0000","last_peered":"2023-08-09T13:48:18.152546+0000","last_clean":"2023-08-09T13:48:18.152546+0000","last_became_active":"2023-08-09T13:48:18.151238+0000","last_became_peered":"2023-08-09T13:48:18.151238+0000","last_unstale":"2023-08-09T13:48:18.152546+0000","last_undegraded":"2023-08-09T13:48:18.152546+0000","last_fullsized":"2023-08-09T13:48:18.152546+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:48:17.087857+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:48:17.087857+0000","last_clean_scrub_stamp":"2023-08-09T13:48:17.087857+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T16:56:02.522493+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"8.17","version":"0'0","reported_seq":11,"reported_epoch":73,"state":"active+clean","last_fresh":"2023-08-09T13:48:18.121930+0000","last_change":"2023-08-09T13:48:17.137380+0000","last_active":"2023-08-09T13:48:18.121930+0000","last_peered":"2023-08-09T13:48:18.121930+0000","last_clean":"2023-08-09T13:48:18.121930+0000","last_became_active":"2023-08-09T13:48:17.137062+0000","last_became_peered":"2023-08-09T13:48:17.137062+0000","last_unstale":"2023-08-09T13:48:18.121930+0000","last_undegraded":"2023-08-09T13:48:18.121930+0000","last_fullsized":"2023-08-09T13:48:18.121930+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:48:16.072766+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:48:16.072766+0000","last_clean_scrub_stamp":"2023-08-09T13:48:16.072766+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-11T01:26:58.358070+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.17","version":"0'0","reported_seq":9,"reported_epoch":73,"state":"active+clean","last_fresh":"2023-08-09T13:48:18.141090+0000","last_change":"2023-08-09T13:48:18.137992+0000","last_active":"2023-08-09T13:48:18.141090+0000","last_peered":"2023-08-09T13:48:18.141090+0000","last_clean":"2023-08-09T13:48:18.141090+0000","last_became_active":"2023-08-09T13:48:18.137621+0000","last_became_peered":"2023-08-09T13:48:18.137621+0000","last_unstale":"2023-08-09T13:48:18.141090+0000","last_undegraded":"2023-08-09T13:48:18.141090+0000","last_fullsized":"2023-08-09T13:48:18.141090+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:48:17.087857+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:48:17.087857+0000","last_clean_scrub_stamp":"2023-08-09T13:48:17.087857+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T18:36:12.306897+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.16","version":"0'0","reported_seq":11,"reported_epoch":73,"state":"active+clean","last_fresh":"2023-08-09T13:48:18.147086+0000","last_change":"2023-08-09T13:48:17.129879+0000","last_active":"2023-08-09T13:48:18.147086+0000","last_peered":"2023-08-09T13:48:18.147086+0000","last_clean":"2023-08-09T13:48:18.147086+0000","last_became_active":"2023-08-09T13:48:17.129599+0000","last_became_peered":"2023-08-09T13:48:17.129599+0000","last_unstale":"2023-08-09T13:48:18.147086+0000","last_undegraded":"2023-08-09T13:48:18.147086+0000","last_fullsized":"2023-08-09T13:48:18.147086+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:48:16.072766+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:48:16.072766+0000","last_clean_scrub_stamp":"2023-08-09T13:48:16.072766+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T18:30:53.375445+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.14","version":"0'0","reported_seq":9,"reported_epoch":73,"state":"active+clean","last_fresh":"2023-08-09T13:48:18.150408+0000","last_change":"2023-08-09T13:48:18.145545+0000","last_active":"2023-08-09T13:48:18.150408+0000","last_peered":"2023-08-09T13:48:18.150408+0000","last_clean":"2023-08-09T13:48:18.150408+0000","last_became_active":"2023-08-09T13:48:18.144442+0000","last_became_peered":"2023-08-09T13:48:18.144442+0000","last_unstale":"2023-08-09T13:48:18.150408+0000","last_undegraded":"2023-08-09T13:48:18.150408+0000","last_fullsized":"2023-08-09T13:48:18.150408+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:48:17.087857+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:48:17.087857+0000","last_clean_scrub_stamp":"2023-08-09T13:48:17.087857+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-11T00:04:12.888732+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.15","version":"73'3","reported_seq":14,"reported_epoch":73,"state":"active+clean","last_fresh":"2023-08-09T13:48:18.173971+0000","last_change":"2023-08-09T13:48:17.133125+0000","last_active":"2023-08-09T13:48:18.173971+0000","last_peered":"2023-08-09T13:48:18.173971+0000","last_clean":"2023-08-09T13:48:18.173971+0000","last_became_active":"2023-08-09T13:48:17.132824+0000","last_became_peered":"2023-08-09T13:48:17.132824+0000","last_unstale":"2023-08-09T13:48:18.173971+0000","last_undegraded":"2023-08-09T13:48:18.173971+0000","last_fullsized":"2023-08-09T13:48:18.173971+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:48:16.072766+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:48:16.072766+0000","last_clean_scrub_stamp":"2023-08-09T13:48:16.072766+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T18:21:07.867154+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.15","version":"0'0","reported_seq":9,"reported_epoch":73,"state":"active+clean","last_fresh":"2023-08-09T13:48:18.137274+0000","last_change":"2023-08-09T13:48:18.128249+0000","last_active":"2023-08-09T13:48:18.137274+0000","last_peered":"2023-08-09T13:48:18.137274+0000","last_clean":"2023-08-09T13:48:18.137274+0000","last_became_active":"2023-08-09T13:48:18.127649+0000","last_became_peered":"2023-08-09T13:48:18.127649+0000","last_unstale":"2023-08-09T13:48:18.137274+0000","last_undegraded":"2023-08-09T13:48:18.137274+0000","last_fullsized":"2023-08-09T13:48:18.137274+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:48:17.087857+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:48:17.087857+0000","last_clean_scrub_stamp":"2023-08-09T13:48:17.087857+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-11T00:04:40.558145+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.14","version":"73'6","reported_seq":17,"reported_epoch":73,"state":"active+clean","last_fresh":"2023-08-09T13:48:18.176391+0000","last_change":"2023-08-09T13:48:17.138705+0000","last_active":"2023-08-09T13:48:18.176391+0000","last_peered":"2023-08-09T13:48:18.176391+0000","last_clean":"2023-08-09T13:48:18.176391+0000","last_became_active":"2023-08-09T13:48:17.138297+0000","last_became_peered":"2023-08-09T13:48:17.138297+0000","last_unstale":"2023-08-09T13:48:18.176391+0000","last_undegraded":"2023-08-09T13:48:18.176391+0000","last_fullsized":"2023-08-09T13:48:18.176391+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:48:16.072766+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:48:16.072766+0000","last_clean_scrub_stamp":"2023-08-09T13:48:16.072766+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T15:33:28.704733+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.12","version":"0'0","reported_seq":9,"reported_epoch":73,"state":"active+clean","last_fresh":"2023-08-09T13:48:18.151949+0000","last_change":"2023-08-09T13:48:18.149462+0000","last_active":"2023-08-09T13:48:18.151949+0000","last_peered":"2023-08-09T13:48:18.151949+0000","last_clean":"2023-08-09T13:48:18.151949+0000","last_became_active":"2023-08-09T13:48:18.149210+0000","last_became_peered":"2023-08-09T13:48:18.149210+0000","last_unstale":"2023-08-09T13:48:18.151949+0000","last_undegraded":"2023-08-09T13:48:18.151949+0000","last_fullsized":"2023-08-09T13:48:18.151949+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:48:17.087857+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:48:17.087857+0000","last_clean_scrub_stamp":"2023-08-09T13:48:17.087857+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T20:59:54.537402+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.13","version":"73'2","reported_seq":13,"reported_epoch":73,"state":"active+clean","last_fresh":"2023-08-09T13:48:18.173271+0000","last_change":"2023-08-09T13:48:17.129248+0000","last_active":"2023-08-09T13:48:18.173271+0000","last_peered":"2023-08-09T13:48:18.173271+0000","last_clean":"2023-08-09T13:48:18.173271+0000","last_became_active":"2023-08-09T13:48:17.128954+0000","last_became_peered":"2023-08-09T13:48:17.128954+0000","last_unstale":"2023-08-09T13:48:18.173271+0000","last_undegraded":"2023-08-09T13:48:18.173271+0000","last_fullsized":"2023-08-09T13:48:18.173271+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:48:16.072766+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:48:16.072766+0000","last_clean_scrub_stamp":"2023-08-09T13:48:16.072766+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T17:25:19.626774+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.13","version":"0'0","reported_seq":9,"reported_epoch":73,"state":"active+clean","last_fresh":"2023-08-09T13:48:18.136870+0000","last_change":"2023-08-09T13:48:18.133565+0000","last_active":"2023-08-09T13:48:18.136870+0000","last_peered":"2023-08-09T13:48:18.136870+0000","last_clean":"2023-08-09T13:48:18.136870+0000","last_became_active":"2023-08-09T13:48:18.133122+0000","last_became_peered":"2023-08-09T13:48:18.133122+0000","last_unstale":"2023-08-09T13:48:18.136870+0000","last_undegraded":"2023-08-09T13:48:18.136870+0000","last_fullsized":"2023-08-09T13:48:18.136870+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:48:17.087857+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:48:17.087857+0000","last_clean_scrub_stamp":"2023-08-09T13:48:17.087857+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-11T01:19:52.281855+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.12","version":"0'0","reported_seq":11,"reported_epoch":73,"state":"active+clean","last_fresh":"2023-08-09T13:48:18.119303+0000","last_change":"2023-08-09T13:48:17.130854+0000","last_active":"2023-08-09T13:48:18.119303+0000","last_peered":"2023-08-09T13:48:18.119303+0000","last_clean":"2023-08-09T13:48:18.119303+0000","last_became_active":"2023-08-09T13:48:17.130566+0000","last_became_peered":"2023-08-09T13:48:17.130566+0000","last_unstale":"2023-08-09T13:48:18.119303+0000","last_undegraded":"2023-08-09T13:48:18.119303+0000","last_fullsized":"2023-08-09T13:48:18.119303+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:48:16.072766+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:48:16.072766+0000","last_clean_scrub_stamp":"2023-08-09T13:48:16.072766+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T22:03:03.015035+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.10","version":"0'0","reported_seq":9,"reported_epoch":73,"state":"active+clean","last_fresh":"2023-08-09T13:48:18.144369+0000","last_change":"2023-08-09T13:48:18.142388+0000","last_active":"2023-08-09T13:48:18.144369+0000","last_peered":"2023-08-09T13:48:18.144369+0000","last_clean":"2023-08-09T13:48:18.144369+0000","last_became_active":"2023-08-09T13:48:18.141862+0000","last_became_peered":"2023-08-09T13:48:18.141862+0000","last_unstale":"2023-08-09T13:48:18.144369+0000","last_undegraded":"2023-08-09T13:48:18.144369+0000","last_fullsized":"2023-08-09T13:48:18.144369+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:48:17.087857+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:48:17.087857+0000","last_clean_scrub_stamp":"2023-08-09T13:48:17.087857+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-11T01:20:29.944656+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"8.11","version":"0'0","reported_seq":11,"reported_epoch":73,"state":"active+clean","last_fresh":"2023-08-09T13:48:18.133797+0000","last_change":"2023-08-09T13:48:17.133973+0000","last_active":"2023-08-09T13:48:18.133797+0000","last_peered":"2023-08-09T13:48:18.133797+0000","last_clean":"2023-08-09T13:48:18.133797+0000","last_became_active":"2023-08-09T13:48:17.133679+0000","last_became_peered":"2023-08-09T13:48:17.133679+0000","last_unstale":"2023-08-09T13:48:18.133797+0000","last_undegraded":"2023-08-09T13:48:18.133797+0000","last_fullsized":"2023-08-09T13:48:18.133797+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:48:16.072766+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:48:16.072766+0000","last_clean_scrub_stamp":"2023-08-09T13:48:16.072766+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T19:37:33.525527+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.11","version":"0'0","reported_seq":9,"reported_epoch":73,"state":"active+clean","last_fresh":"2023-08-09T13:48:18.155678+0000","last_change":"2023-08-09T13:48:18.154453+0000","last_active":"2023-08-09T13:48:18.155678+0000","last_peered":"2023-08-09T13:48:18.155678+0000","last_clean":"2023-08-09T13:48:18.155678+0000","last_became_active":"2023-08-09T13:48:18.154112+0000","last_became_peered":"2023-08-09T13:48:18.154112+0000","last_unstale":"2023-08-09T13:48:18.155678+0000","last_undegraded":"2023-08-09T13:48:18.155678+0000","last_fullsized":"2023-08-09T13:48:18.155678+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:48:17.087857+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:48:17.087857+0000","last_clean_scrub_stamp":"2023-08-09T13:48:17.087857+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T21:14:02.421900+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.10","version":"73'8","reported_seq":19,"reported_epoch":73,"state":"active+clean","last_fresh":"2023-08-09T13:48:18.173716+0000","last_change":"2023-08-09T13:48:17.134689+0000","last_active":"2023-08-09T13:48:18.173716+0000","last_peered":"2023-08-09T13:48:18.173716+0000","last_clean":"2023-08-09T13:48:18.173716+0000","last_became_active":"2023-08-09T13:48:17.134378+0000","last_became_peered":"2023-08-09T13:48:17.134378+0000","last_unstale":"2023-08-09T13:48:18.173716+0000","last_undegraded":"2023-08-09T13:48:18.173716+0000","last_fullsized":"2023-08-09T13:48:18.173716+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:48:16.072766+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:48:16.072766+0000","last_clean_scrub_stamp":"2023-08-09T13:48:16.072766+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T21:29:33.710061+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.6","version":"0'0","reported_seq":9,"reported_epoch":73,"state":"active+clean","last_fresh":"2023-08-09T13:48:18.151771+0000","last_change":"2023-08-09T13:48:18.146139+0000","last_active":"2023-08-09T13:48:18.151771+0000","last_peered":"2023-08-09T13:48:18.151771+0000","last_clean":"2023-08-09T13:48:18.151771+0000","last_became_active":"2023-08-09T13:48:18.145842+0000","last_became_peered":"2023-08-09T13:48:18.145842+0000","last_unstale":"2023-08-09T13:48:18.151771+0000","last_undegraded":"2023-08-09T13:48:18.151771+0000","last_fullsized":"2023-08-09T13:48:18.151771+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:48:17.087857+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:48:17.087857+0000","last_clean_scrub_stamp":"2023-08-09T13:48:17.087857+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T22:21:27.281375+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.7","version":"73'2","reported_seq":13,"reported_epoch":73,"state":"active+clean","last_fresh":"2023-08-09T13:48:18.175555+0000","last_change":"2023-08-09T13:48:17.135531+0000","last_active":"2023-08-09T13:48:18.175555+0000","last_peered":"2023-08-09T13:48:18.175555+0000","last_clean":"2023-08-09T13:48:18.175555+0000","last_became_active":"2023-08-09T13:48:17.135112+0000","last_became_peered":"2023-08-09T13:48:17.135112+0000","last_unstale":"2023-08-09T13:48:18.175555+0000","last_undegraded":"2023-08-09T13:48:18.175555+0000","last_fullsized":"2023-08-09T13:48:18.175555+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:48:16.072766+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:48:16.072766+0000","last_clean_scrub_stamp":"2023-08-09T13:48:16.072766+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T14:17:18.474299+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.7","version":"0'0","reported_seq":9,"reported_epoch":73,"state":"active+clean","last_fresh":"2023-08-09T13:48:18.141348+0000","last_change":"2023-08-09T13:48:18.139622+0000","last_active":"2023-08-09T13:48:18.141348+0000","last_peered":"2023-08-09T13:48:18.141348+0000","last_clean":"2023-08-09T13:48:18.141348+0000","last_became_active":"2023-08-09T13:48:18.139332+0000","last_became_peered":"2023-08-09T13:48:18.139332+0000","last_unstale":"2023-08-09T13:48:18.141348+0000","last_undegraded":"2023-08-09T13:48:18.141348+0000","last_fullsized":"2023-08-09T13:48:18.141348+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:48:17.087857+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:48:17.087857+0000","last_clean_scrub_stamp":"2023-08-09T13:48:17.087857+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T20:42:45.679132+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.6","version":"0'0","reported_seq":11,"reported_epoch":73,"state":"active+clean","last_fresh":"2023-08-09T13:48:18.124688+0000","last_change":"2023-08-09T13:48:17.135522+0000","last_active":"2023-08-09T13:48:18.124688+0000","last_peered":"2023-08-09T13:48:18.124688+0000","last_clean":"2023-08-09T13:48:18.124688+0000","last_became_active":"2023-08-09T13:48:17.135229+0000","last_became_peered":"2023-08-09T13:48:17.135229+0000","last_unstale":"2023-08-09T13:48:18.124688+0000","last_undegraded":"2023-08-09T13:48:18.124688+0000","last_fullsized":"2023-08-09T13:48:18.124688+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:48:16.072766+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:48:16.072766+0000","last_clean_scrub_stamp":"2023-08-09T13:48:16.072766+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T16:56:01.507402+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.4","version":"0'0","reported_seq":9,"reported_epoch":73,"state":"active+clean","last_fresh":"2023-08-09T13:48:18.150431+0000","last_change":"2023-08-09T13:48:18.146073+0000","last_active":"2023-08-09T13:48:18.150431+0000","last_peered":"2023-08-09T13:48:18.150431+0000","last_clean":"2023-08-09T13:48:18.150431+0000","last_became_active":"2023-08-09T13:48:18.145037+0000","last_became_peered":"2023-08-09T13:48:18.145037+0000","last_unstale":"2023-08-09T13:48:18.150431+0000","last_undegraded":"2023-08-09T13:48:18.150431+0000","last_fullsized":"2023-08-09T13:48:18.150431+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:48:17.087857+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:48:17.087857+0000","last_clean_scrub_stamp":"2023-08-09T13:48:17.087857+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T16:40:28.280266+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.5","version":"0'0","reported_seq":11,"reported_epoch":73,"state":"active+clean","last_fresh":"2023-08-09T13:48:18.119655+0000","last_change":"2023-08-09T13:48:17.136747+0000","last_active":"2023-08-09T13:48:18.119655+0000","last_peered":"2023-08-09T13:48:18.119655+0000","last_clean":"2023-08-09T13:48:18.119655+0000","last_became_active":"2023-08-09T13:48:17.136353+0000","last_became_peered":"2023-08-09T13:48:17.136353+0000","last_unstale":"2023-08-09T13:48:18.119655+0000","last_undegraded":"2023-08-09T13:48:18.119655+0000","last_fullsized":"2023-08-09T13:48:18.119655+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:48:16.072766+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:48:16.072766+0000","last_clean_scrub_stamp":"2023-08-09T13:48:16.072766+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T22:16:07.036190+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.5","version":"0'0","reported_seq":9,"reported_epoch":73,"state":"active+clean","last_fresh":"2023-08-09T13:48:18.155580+0000","last_change":"2023-08-09T13:48:18.142737+0000","last_active":"2023-08-09T13:48:18.155580+0000","last_peered":"2023-08-09T13:48:18.155580+0000","last_clean":"2023-08-09T13:48:18.155580+0000","last_became_active":"2023-08-09T13:48:18.141654+0000","last_became_peered":"2023-08-09T13:48:18.141654+0000","last_unstale":"2023-08-09T13:48:18.155580+0000","last_undegraded":"2023-08-09T13:48:18.155580+0000","last_fullsized":"2023-08-09T13:48:18.155580+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:48:17.087857+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:48:17.087857+0000","last_clean_scrub_stamp":"2023-08-09T13:48:17.087857+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T20:59:41.322587+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.4","version":"0'0","reported_seq":11,"reported_epoch":73,"state":"active+clean","last_fresh":"2023-08-09T13:48:18.136486+0000","last_change":"2023-08-09T13:48:17.139103+0000","last_active":"2023-08-09T13:48:18.136486+0000","last_peered":"2023-08-09T13:48:18.136486+0000","last_clean":"2023-08-09T13:48:18.136486+0000","last_became_active":"2023-08-09T13:48:17.138848+0000","last_became_peered":"2023-08-09T13:48:17.138848+0000","last_unstale":"2023-08-09T13:48:18.136486+0000","last_undegraded":"2023-08-09T13:48:18.136486+0000","last_fullsized":"2023-08-09T13:48:18.136486+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:48:16.072766+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:48:16.072766+0000","last_clean_scrub_stamp":"2023-08-09T13:48:16.072766+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T18:40:08.613673+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.2","version":"0'0","reported_seq":9,"reported_epoch":73,"state":"active+clean","last_fresh":"2023-08-09T13:48:18.160553+0000","last_change":"2023-08-09T13:48:18.158030+0000","last_active":"2023-08-09T13:48:18.160553+0000","last_peered":"2023-08-09T13:48:18.160553+0000","last_clean":"2023-08-09T13:48:18.160553+0000","last_became_active":"2023-08-09T13:48:18.157119+0000","last_became_peered":"2023-08-09T13:48:18.157119+0000","last_unstale":"2023-08-09T13:48:18.160553+0000","last_undegraded":"2023-08-09T13:48:18.160553+0000","last_fullsized":"2023-08-09T13:48:18.160553+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:48:17.087857+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:48:17.087857+0000","last_clean_scrub_stamp":"2023-08-09T13:48:17.087857+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T16:11:34.105483+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.3","version":"0'0","reported_seq":11,"reported_epoch":73,"state":"active+clean","last_fresh":"2023-08-09T13:48:18.111274+0000","last_change":"2023-08-09T13:48:17.122100+0000","last_active":"2023-08-09T13:48:18.111274+0000","last_peered":"2023-08-09T13:48:18.111274+0000","last_clean":"2023-08-09T13:48:18.111274+0000","last_became_active":"2023-08-09T13:48:17.121794+0000","last_became_peered":"2023-08-09T13:48:17.121794+0000","last_unstale":"2023-08-09T13:48:18.111274+0000","last_undegraded":"2023-08-09T13:48:18.111274+0000","last_fullsized":"2023-08-09T13:48:18.111274+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:48:16.072766+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:48:16.072766+0000","last_clean_scrub_stamp":"2023-08-09T13:48:16.072766+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T23:45:17.019687+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.3","version":"0'0","reported_seq":9,"reported_epoch":73,"state":"active+clean","last_fresh":"2023-08-09T13:48:18.155656+0000","last_change":"2023-08-09T13:48:18.154534+0000","last_active":"2023-08-09T13:48:18.155656+0000","last_peered":"2023-08-09T13:48:18.155656+0000","last_clean":"2023-08-09T13:48:18.155656+0000","last_became_active":"2023-08-09T13:48:18.154225+0000","last_became_peered":"2023-08-09T13:48:18.154225+0000","last_unstale":"2023-08-09T13:48:18.155656+0000","last_undegraded":"2023-08-09T13:48:18.155656+0000","last_fullsized":"2023-08-09T13:48:18.155656+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:48:17.087857+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:48:17.087857+0000","last_clean_scrub_stamp":"2023-08-09T13:48:17.087857+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T22:39:27.852461+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.2","version":"0'0","reported_seq":11,"reported_epoch":73,"state":"active+clean","last_fresh":"2023-08-09T13:48:18.141034+0000","last_change":"2023-08-09T13:48:17.139591+0000","last_active":"2023-08-09T13:48:18.141034+0000","last_peered":"2023-08-09T13:48:18.141034+0000","last_clean":"2023-08-09T13:48:18.141034+0000","last_became_active":"2023-08-09T13:48:17.139320+0000","last_became_peered":"2023-08-09T13:48:17.139320+0000","last_unstale":"2023-08-09T13:48:18.141034+0000","last_undegraded":"2023-08-09T13:48:18.141034+0000","last_fullsized":"2023-08-09T13:48:18.141034+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:48:16.072766+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:48:16.072766+0000","last_clean_scrub_stamp":"2023-08-09T13:48:16.072766+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-11T01:15:12.569060+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.0","version":"0'0","reported_seq":9,"reported_epoch":73,"state":"active+clean","last_fresh":"2023-08-09T13:48:18.137937+0000","last_change":"2023-08-09T13:48:18.127709+0000","last_active":"2023-08-09T13:48:18.137937+0000","last_peered":"2023-08-09T13:48:18.137937+0000","last_clean":"2023-08-09T13:48:18.137937+0000","last_became_active":"2023-08-09T13:48:18.126930+0000","last_became_peered":"2023-08-09T13:48:18.126930+0000","last_unstale":"2023-08-09T13:48:18.137937+0000","last_undegraded":"2023-08-09T13:48:18.137937+0000","last_fullsized":"2023-08-09T13:48:18.137937+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:48:17.087857+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:48:17.087857+0000","last_clean_scrub_stamp":"2023-08-09T13:48:17.087857+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-11T00:13:56.323548+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.1","version":"0'0","reported_seq":11,"reported_epoch":73,"state":"active+clean","last_fresh":"2023-08-09T13:48:18.110950+0000","last_change":"2023-08-09T13:48:17.125898+0000","last_active":"2023-08-09T13:48:18.110950+0000","last_peered":"2023-08-09T13:48:18.110950+0000","last_clean":"2023-08-09T13:48:18.110950+0000","last_became_active":"2023-08-09T13:48:17.125609+0000","last_became_peered":"2023-08-09T13:48:17.125609+0000","last_unstale":"2023-08-09T13:48:18.110950+0000","last_undegraded":"2023-08-09T13:48:18.110950+0000","last_fullsized":"2023-08-09T13:48:18.110950+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:48:16.072766+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:48:16.072766+0000","last_clean_scrub_stamp":"2023-08-09T13:48:16.072766+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T19:04:18.439807+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.1","version":"0'0","reported_seq":9,"reported_epoch":73,"state":"active+clean","last_fresh":"2023-08-09T13:48:18.159634+0000","last_change":"2023-08-09T13:48:18.156163+0000","last_active":"2023-08-09T13:48:18.159634+0000","last_peered":"2023-08-09T13:48:18.159634+0000","last_clean":"2023-08-09T13:48:18.159634+0000","last_became_active":"2023-08-09T13:48:18.155181+0000","last_became_peered":"2023-08-09T13:48:18.155181+0000","last_unstale":"2023-08-09T13:48:18.159634+0000","last_undegraded":"2023-08-09T13:48:18.159634+0000","last_fullsized":"2023-08-09T13:48:18.159634+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:48:17.087857+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:48:17.087857+0000","last_clean_scrub_stamp":"2023-08-09T13:48:17.087857+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-11T01:14:04.304684+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.0","version":"0'0","reported_seq":11,"reported_epoch":73,"state":"active+clean","last_fresh":"2023-08-09T13:48:18.146883+0000","last_change":"2023-08-09T13:48:17.134067+0000","last_active":"2023-08-09T13:48:18.146883+0000","last_peered":"2023-08-09T13:48:18.146883+0000","last_clean":"2023-08-09T13:48:18.146883+0000","last_became_active":"2023-08-09T13:48:17.133792+0000","last_became_peered":"2023-08-09T13:48:17.133792+0000","last_unstale":"2023-08-09T13:48:18.146883+0000","last_undegraded":"2023-08-09T13:48:18.146883+0000","last_fullsized":"2023-08-09T13:48:18.146883+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:48:16.072766+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:48:16.072766+0000","last_clean_scrub_stamp":"2023-08-09T13:48:16.072766+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T20:10:04.314162+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.8","version":"0'0","reported_seq":9,"reported_epoch":73,"state":"active+clean","last_fresh":"2023-08-09T13:48:18.160521+0000","last_change":"2023-08-09T13:48:18.157902+0000","last_active":"2023-08-09T13:48:18.160521+0000","last_peered":"2023-08-09T13:48:18.160521+0000","last_clean":"2023-08-09T13:48:18.160521+0000","last_became_active":"2023-08-09T13:48:18.157606+0000","last_became_peered":"2023-08-09T13:48:18.157606+0000","last_unstale":"2023-08-09T13:48:18.160521+0000","last_undegraded":"2023-08-09T13:48:18.160521+0000","last_fullsized":"2023-08-09T13:48:18.160521+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:48:17.087857+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:48:17.087857+0000","last_clean_scrub_stamp":"2023-08-09T13:48:17.087857+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T17:18:22.010079+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.9","version":"0'0","reported_seq":11,"reported_epoch":73,"state":"active+clean","last_fresh":"2023-08-09T13:48:18.116903+0000","last_change":"2023-08-09T13:48:17.136780+0000","last_active":"2023-08-09T13:48:18.116903+0000","last_peered":"2023-08-09T13:48:18.116903+0000","last_clean":"2023-08-09T13:48:18.116903+0000","last_became_active":"2023-08-09T13:48:17.136480+0000","last_became_peered":"2023-08-09T13:48:17.136480+0000","last_unstale":"2023-08-09T13:48:18.116903+0000","last_undegraded":"2023-08-09T13:48:18.116903+0000","last_fullsized":"2023-08-09T13:48:18.116903+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:48:16.072766+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:48:16.072766+0000","last_clean_scrub_stamp":"2023-08-09T13:48:16.072766+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T14:33:01.425713+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":120,"reported_epoch":73,"state":"active+clean","last_fresh":"2023-08-09T13:48:18.115289+0000","last_change":"2023-08-09T13:33:56.401406+0000","last_active":"2023-08-09T13:48:18.115289+0000","last_peered":"2023-08-09T13:48:18.115289+0000","last_clean":"2023-08-09T13:48:18.115289+0000","last_became_active":"2023-08-09T13:33:56.400487+0000","last_became_peered":"2023-08-09T13:33:56.400487+0000","last_unstale":"2023-08-09T13:48:18.115289+0000","last_undegraded":"2023-08-09T13:48:18.115289+0000","last_fullsized":"2023-08-09T13:48:18.115289+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:33:54.845869+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:33:54.845869+0000","last_clean_scrub_stamp":"2023-08-09T13:33:54.845869+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T23:29:59.861309+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.9","version":"0'0","reported_seq":9,"reported_epoch":73,"state":"active+clean","last_fresh":"2023-08-09T13:48:18.155107+0000","last_change":"2023-08-09T13:48:18.140747+0000","last_active":"2023-08-09T13:48:18.155107+0000","last_peered":"2023-08-09T13:48:18.155107+0000","last_clean":"2023-08-09T13:48:18.155107+0000","last_became_active":"2023-08-09T13:48:18.140237+0000","last_became_peered":"2023-08-09T13:48:18.140237+0000","last_unstale":"2023-08-09T13:48:18.155107+0000","last_undegraded":"2023-08-09T13:48:18.155107+0000","last_fullsized":"2023-08-09T13:48:18.155107+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:48:17.087857+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:48:17.087857+0000","last_clean_scrub_stamp":"2023-08-09T13:48:17.087857+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T23:12:15.650700+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.8","version":"0'0","reported_seq":11,"reported_epoch":73,"state":"active+clean","last_fresh":"2023-08-09T13:48:18.134039+0000","last_change":"2023-08-09T13:48:17.139548+0000","last_active":"2023-08-09T13:48:18.134039+0000","last_peered":"2023-08-09T13:48:18.134039+0000","last_clean":"2023-08-09T13:48:18.134039+0000","last_became_active":"2023-08-09T13:48:17.139270+0000","last_became_peered":"2023-08-09T13:48:17.139270+0000","last_unstale":"2023-08-09T13:48:18.134039+0000","last_undegraded":"2023-08-09T13:48:18.134039+0000","last_fullsized":"2023-08-09T13:48:18.134039+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:48:16.072766+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:48:16.072766+0000","last_clean_scrub_stamp":"2023-08-09T13:48:16.072766+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T21:41:50.040105+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":119,"reported_epoch":73,"state":"active+clean","last_fresh":"2023-08-09T13:48:18.133585+0000","last_change":"2023-08-09T13:33:56.401123+0000","last_active":"2023-08-09T13:48:18.133585+0000","last_peered":"2023-08-09T13:48:18.133585+0000","last_clean":"2023-08-09T13:48:18.133585+0000","last_became_active":"2023-08-09T13:33:56.399933+0000","last_became_peered":"2023-08-09T13:33:56.399933+0000","last_unstale":"2023-08-09T13:48:18.133585+0000","last_undegraded":"2023-08-09T13:48:18.133585+0000","last_fullsized":"2023-08-09T13:48:18.133585+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:33:54.845869+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:33:54.845869+0000","last_clean_scrub_stamp":"2023-08-09T13:33:54.845869+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-11T00:44:56.386308+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.a","version":"0'0","reported_seq":9,"reported_epoch":73,"state":"active+clean","last_fresh":"2023-08-09T13:48:18.151929+0000","last_change":"2023-08-09T13:48:18.148933+0000","last_active":"2023-08-09T13:48:18.151929+0000","last_peered":"2023-08-09T13:48:18.151929+0000","last_clean":"2023-08-09T13:48:18.151929+0000","last_became_active":"2023-08-09T13:48:18.148669+0000","last_became_peered":"2023-08-09T13:48:18.148669+0000","last_unstale":"2023-08-09T13:48:18.151929+0000","last_undegraded":"2023-08-09T13:48:18.151929+0000","last_fullsized":"2023-08-09T13:48:18.151929+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:48:17.087857+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:48:17.087857+0000","last_clean_scrub_stamp":"2023-08-09T13:48:17.087857+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T19:22:19.856884+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.b","version":"73'2","reported_seq":13,"reported_epoch":73,"state":"active+clean","last_fresh":"2023-08-09T13:48:18.172999+0000","last_change":"2023-08-09T13:48:17.136357+0000","last_active":"2023-08-09T13:48:18.172999+0000","last_peered":"2023-08-09T13:48:18.172999+0000","last_clean":"2023-08-09T13:48:18.172999+0000","last_became_active":"2023-08-09T13:48:17.136065+0000","last_became_peered":"2023-08-09T13:48:17.136065+0000","last_unstale":"2023-08-09T13:48:18.172999+0000","last_undegraded":"2023-08-09T13:48:18.172999+0000","last_fullsized":"2023-08-09T13:48:18.172999+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:48:16.072766+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:48:16.072766+0000","last_clean_scrub_stamp":"2023-08-09T13:48:16.072766+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T18:36:41.123187+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.2","version":"0'0","reported_seq":119,"reported_epoch":73,"state":"active+clean","last_fresh":"2023-08-09T13:48:18.115983+0000","last_change":"2023-08-09T13:33:56.402794+0000","last_active":"2023-08-09T13:48:18.115983+0000","last_peered":"2023-08-09T13:48:18.115983+0000","last_clean":"2023-08-09T13:48:18.115983+0000","last_became_active":"2023-08-09T13:33:56.402038+0000","last_became_peered":"2023-08-09T13:33:56.402038+0000","last_unstale":"2023-08-09T13:48:18.115983+0000","last_undegraded":"2023-08-09T13:48:18.115983+0000","last_fullsized":"2023-08-09T13:48:18.115983+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:33:54.845869+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:33:54.845869+0000","last_clean_scrub_stamp":"2023-08-09T13:33:54.845869+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T18:58:03.028401+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.b","version":"0'0","reported_seq":9,"reported_epoch":73,"state":"active+clean","last_fresh":"2023-08-09T13:48:18.158680+0000","last_change":"2023-08-09T13:48:18.144582+0000","last_active":"2023-08-09T13:48:18.158680+0000","last_peered":"2023-08-09T13:48:18.158680+0000","last_clean":"2023-08-09T13:48:18.158680+0000","last_became_active":"2023-08-09T13:48:18.144257+0000","last_became_peered":"2023-08-09T13:48:18.144257+0000","last_unstale":"2023-08-09T13:48:18.158680+0000","last_undegraded":"2023-08-09T13:48:18.158680+0000","last_fullsized":"2023-08-09T13:48:18.158680+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:48:17.087857+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:48:17.087857+0000","last_clean_scrub_stamp":"2023-08-09T13:48:17.087857+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T20:38:23.607892+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.a","version":"0'0","reported_seq":11,"reported_epoch":73,"state":"active+clean","last_fresh":"2023-08-09T13:48:18.111024+0000","last_change":"2023-08-09T13:48:17.128719+0000","last_active":"2023-08-09T13:48:18.111024+0000","last_peered":"2023-08-09T13:48:18.111024+0000","last_clean":"2023-08-09T13:48:18.111024+0000","last_became_active":"2023-08-09T13:48:17.128447+0000","last_became_peered":"2023-08-09T13:48:17.128447+0000","last_unstale":"2023-08-09T13:48:18.111024+0000","last_undegraded":"2023-08-09T13:48:18.111024+0000","last_fullsized":"2023-08-09T13:48:18.111024+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:48:16.072766+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:48:16.072766+0000","last_clean_scrub_stamp":"2023-08-09T13:48:16.072766+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T19:42:11.460732+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.3","version":"0'0","reported_seq":121,"reported_epoch":73,"state":"active+clean","last_fresh":"2023-08-09T13:48:18.111186+0000","last_change":"2023-08-09T13:33:55.886749+0000","last_active":"2023-08-09T13:48:18.111186+0000","last_peered":"2023-08-09T13:48:18.111186+0000","last_clean":"2023-08-09T13:48:18.111186+0000","last_became_active":"2023-08-09T13:33:55.886116+0000","last_became_peered":"2023-08-09T13:33:55.886116+0000","last_unstale":"2023-08-09T13:48:18.111186+0000","last_undegraded":"2023-08-09T13:48:18.111186+0000","last_fullsized":"2023-08-09T13:48:18.111186+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:33:54.845869+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:33:54.845869+0000","last_clean_scrub_stamp":"2023-08-09T13:33:54.845869+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T18:20:02.086899+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"62'139","reported_seq":348,"reported_epoch":73,"state":"active+clean","last_fresh":"2023-08-09T13:48:18.114880+0000","last_change":"2023-08-09T13:33:55.886345+0000","last_active":"2023-08-09T13:48:18.114880+0000","last_peered":"2023-08-09T13:48:18.114880+0000","last_clean":"2023-08-09T13:48:18.114880+0000","last_became_active":"2023-08-09T13:33:55.885781+0000","last_became_peered":"2023-08-09T13:33:55.885781+0000","last_unstale":"2023-08-09T13:48:18.114880+0000","last_undegraded":"2023-08-09T13:48:18.114880+0000","last_fullsized":"2023-08-09T13:48:18.114880+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:33:54.845869+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:33:54.845869+0000","last_clean_scrub_stamp":"2023-08-09T13:33:54.845869+0000","objects_scrubbed":0,"log_size":139,"log_dups_size":0,"ondisk_log_size":139,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T19:19:10.686060+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":308,"num_read_kb":512,"num_write":199,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.c","version":"0'0","reported_seq":9,"reported_epoch":73,"state":"active+clean","last_fresh":"2023-08-09T13:48:18.156594+0000","last_change":"2023-08-09T13:48:18.150584+0000","last_active":"2023-08-09T13:48:18.156594+0000","last_peered":"2023-08-09T13:48:18.156594+0000","last_clean":"2023-08-09T13:48:18.156594+0000","last_became_active":"2023-08-09T13:48:18.150254+0000","last_became_peered":"2023-08-09T13:48:18.150254+0000","last_unstale":"2023-08-09T13:48:18.156594+0000","last_undegraded":"2023-08-09T13:48:18.156594+0000","last_fullsized":"2023-08-09T13:48:18.156594+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:48:17.087857+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:48:17.087857+0000","last_clean_scrub_stamp":"2023-08-09T13:48:17.087857+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T21:07:01.343134+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.d","version":"73'1","reported_seq":12,"reported_epoch":73,"state":"active+clean","last_fresh":"2023-08-09T13:48:18.173679+0000","last_change":"2023-08-09T13:48:17.134706+0000","last_active":"2023-08-09T13:48:18.173679+0000","last_peered":"2023-08-09T13:48:18.173679+0000","last_clean":"2023-08-09T13:48:18.173679+0000","last_became_active":"2023-08-09T13:48:17.134427+0000","last_became_peered":"2023-08-09T13:48:17.134427+0000","last_unstale":"2023-08-09T13:48:18.173679+0000","last_undegraded":"2023-08-09T13:48:18.173679+0000","last_fullsized":"2023-08-09T13:48:18.173679+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:48:16.072766+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:48:16.072766+0000","last_clean_scrub_stamp":"2023-08-09T13:48:16.072766+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T23:23:54.631430+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":127,"reported_epoch":73,"state":"active+clean","last_fresh":"2023-08-09T13:48:18.146475+0000","last_change":"2023-08-09T13:33:55.889565+0000","last_active":"2023-08-09T13:48:18.146475+0000","last_peered":"2023-08-09T13:48:18.146475+0000","last_clean":"2023-08-09T13:48:18.146475+0000","last_became_active":"2023-08-09T13:33:55.889164+0000","last_became_peered":"2023-08-09T13:33:55.889164+0000","last_unstale":"2023-08-09T13:48:18.146475+0000","last_undegraded":"2023-08-09T13:48:18.146475+0000","last_fullsized":"2023-08-09T13:48:18.146475+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:33:54.845869+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:33:54.845869+0000","last_clean_scrub_stamp":"2023-08-09T13:33:54.845869+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T21:05:04.235125+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.d","version":"0'0","reported_seq":9,"reported_epoch":73,"state":"active+clean","last_fresh":"2023-08-09T13:48:18.140977+0000","last_change":"2023-08-09T13:48:18.137115+0000","last_active":"2023-08-09T13:48:18.140977+0000","last_peered":"2023-08-09T13:48:18.140977+0000","last_clean":"2023-08-09T13:48:18.140977+0000","last_became_active":"2023-08-09T13:48:18.136785+0000","last_became_peered":"2023-08-09T13:48:18.136785+0000","last_unstale":"2023-08-09T13:48:18.140977+0000","last_undegraded":"2023-08-09T13:48:18.140977+0000","last_fullsized":"2023-08-09T13:48:18.140977+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:48:17.087857+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:48:17.087857+0000","last_clean_scrub_stamp":"2023-08-09T13:48:17.087857+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T21:23:48.688566+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.c","version":"0'0","reported_seq":11,"reported_epoch":73,"state":"active+clean","last_fresh":"2023-08-09T13:48:18.120812+0000","last_change":"2023-08-09T13:48:17.136927+0000","last_active":"2023-08-09T13:48:18.120812+0000","last_peered":"2023-08-09T13:48:18.120812+0000","last_clean":"2023-08-09T13:48:18.120812+0000","last_became_active":"2023-08-09T13:48:17.136614+0000","last_became_peered":"2023-08-09T13:48:17.136614+0000","last_unstale":"2023-08-09T13:48:18.120812+0000","last_undegraded":"2023-08-09T13:48:18.120812+0000","last_fullsized":"2023-08-09T13:48:18.120812+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:48:16.072766+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:48:16.072766+0000","last_clean_scrub_stamp":"2023-08-09T13:48:16.072766+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T17:36:58.038094+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":119,"reported_epoch":73,"state":"active+clean","last_fresh":"2023-08-09T13:48:18.132851+0000","last_change":"2023-08-09T13:33:56.404236+0000","last_active":"2023-08-09T13:48:18.132851+0000","last_peered":"2023-08-09T13:48:18.132851+0000","last_clean":"2023-08-09T13:48:18.132851+0000","last_became_active":"2023-08-09T13:33:56.402485+0000","last_became_peered":"2023-08-09T13:33:56.402485+0000","last_unstale":"2023-08-09T13:48:18.132851+0000","last_undegraded":"2023-08-09T13:48:18.132851+0000","last_fullsized":"2023-08-09T13:48:18.132851+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:33:54.845869+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:33:54.845869+0000","last_clean_scrub_stamp":"2023-08-09T13:33:54.845869+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-11T00:31:43.966707+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.e","version":"0'0","reported_seq":9,"reported_epoch":73,"state":"active+clean","last_fresh":"2023-08-09T13:48:18.149440+0000","last_change":"2023-08-09T13:48:18.143367+0000","last_active":"2023-08-09T13:48:18.149440+0000","last_peered":"2023-08-09T13:48:18.149440+0000","last_clean":"2023-08-09T13:48:18.149440+0000","last_became_active":"2023-08-09T13:48:18.143024+0000","last_became_peered":"2023-08-09T13:48:18.143024+0000","last_unstale":"2023-08-09T13:48:18.149440+0000","last_undegraded":"2023-08-09T13:48:18.149440+0000","last_fullsized":"2023-08-09T13:48:18.149440+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:48:17.087857+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:48:17.087857+0000","last_clean_scrub_stamp":"2023-08-09T13:48:17.087857+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T17:40:48.922810+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.f","version":"73'1","reported_seq":12,"reported_epoch":73,"state":"active+clean","last_fresh":"2023-08-09T13:48:18.173195+0000","last_change":"2023-08-09T13:48:17.133336+0000","last_active":"2023-08-09T13:48:18.173195+0000","last_peered":"2023-08-09T13:48:18.173195+0000","last_clean":"2023-08-09T13:48:18.173195+0000","last_became_active":"2023-08-09T13:48:17.133055+0000","last_became_peered":"2023-08-09T13:48:17.133055+0000","last_unstale":"2023-08-09T13:48:18.173195+0000","last_undegraded":"2023-08-09T13:48:18.173195+0000","last_fullsized":"2023-08-09T13:48:18.173195+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:48:16.072766+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:48:16.072766+0000","last_clean_scrub_stamp":"2023-08-09T13:48:16.072766+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-11T00:10:18.577968+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":117,"reported_epoch":73,"state":"active+clean","last_fresh":"2023-08-09T13:48:18.120260+0000","last_change":"2023-08-09T13:33:56.401781+0000","last_active":"2023-08-09T13:48:18.120260+0000","last_peered":"2023-08-09T13:48:18.120260+0000","last_clean":"2023-08-09T13:48:18.120260+0000","last_became_active":"2023-08-09T13:33:56.400336+0000","last_became_peered":"2023-08-09T13:33:56.400336+0000","last_unstale":"2023-08-09T13:48:18.120260+0000","last_undegraded":"2023-08-09T13:48:18.120260+0000","last_fullsized":"2023-08-09T13:48:18.120260+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:33:54.845869+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:33:54.845869+0000","last_clean_scrub_stamp":"2023-08-09T13:33:54.845869+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T16:41:47.077169+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.f","version":"0'0","reported_seq":9,"reported_epoch":73,"state":"active+clean","last_fresh":"2023-08-09T13:48:18.156161+0000","last_change":"2023-08-09T13:48:18.155218+0000","last_active":"2023-08-09T13:48:18.156161+0000","last_peered":"2023-08-09T13:48:18.156161+0000","last_clean":"2023-08-09T13:48:18.156161+0000","last_became_active":"2023-08-09T13:48:18.154894+0000","last_became_peered":"2023-08-09T13:48:18.154894+0000","last_unstale":"2023-08-09T13:48:18.156161+0000","last_undegraded":"2023-08-09T13:48:18.156161+0000","last_fullsized":"2023-08-09T13:48:18.156161+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:48:17.087857+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:48:17.087857+0000","last_clean_scrub_stamp":"2023-08-09T13:48:17.087857+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-11T00:56:32.885321+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.e","version":"73'2","reported_seq":13,"reported_epoch":73,"state":"active+clean","last_fresh":"2023-08-09T13:48:18.173818+0000","last_change":"2023-08-09T13:48:17.129292+0000","last_active":"2023-08-09T13:48:18.173818+0000","last_peered":"2023-08-09T13:48:18.173818+0000","last_clean":"2023-08-09T13:48:18.173818+0000","last_became_active":"2023-08-09T13:48:17.128999+0000","last_became_peered":"2023-08-09T13:48:17.128999+0000","last_unstale":"2023-08-09T13:48:18.173818+0000","last_undegraded":"2023-08-09T13:48:18.173818+0000","last_fullsized":"2023-08-09T13:48:18.173818+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:48:16.072766+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:48:16.072766+0000","last_clean_scrub_stamp":"2023-08-09T13:48:16.072766+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T22:54:26.443293+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":117,"reported_epoch":73,"state":"active+clean","last_fresh":"2023-08-09T13:48:18.117136+0000","last_change":"2023-08-09T13:33:55.886389+0000","last_active":"2023-08-09T13:48:18.117136+0000","last_peered":"2023-08-09T13:48:18.117136+0000","last_clean":"2023-08-09T13:48:18.117136+0000","last_became_active":"2023-08-09T13:33:55.885638+0000","last_became_peered":"2023-08-09T13:33:55.885638+0000","last_unstale":"2023-08-09T13:48:18.117136+0000","last_undegraded":"2023-08-09T13:48:18.117136+0000","last_fullsized":"2023-08-09T13:48:18.117136+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:33:54.845869+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:33:54.845869+0000","last_clean_scrub_stamp":"2023-08-09T13:33:54.845869+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-11T00:48:14.667108+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]}],"pool_stats":[{"poolid":9,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":8,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":38,"ondisk_log_size":38,"up":64,"acting":64,"num_store_stats":8},{"poolid":2,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":308,"num_read_kb":512,"num_write":199,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":139,"ondisk_log_size":139,"up":2,"acting":2,"num_store_stats":2},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":8}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834575022,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":36640,"kb_used_data":1424,"kb_used_omap":1,"kb_used_meta":35198,"kb_avail":94335200,"statfs":{"total":96636764160,"available":96599244800,"internally_reserved":0,"allocated":1458176,"data_stored":1045869,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1781,"internal_metadata":36043019},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Aug 9 13:48:11 2023","interfaces":[{"interface":"back","average":{"1min":1.242,"5min":1.22,"15min":1.23},"min":{"1min":0.71999999999999997,"5min":0.71999999999999997,"15min":0.61099999999999999},"max":{"1min":1.7190000000000001,"5min":1.7190000000000001,"15min":2.1709999999999998},"last":1.8919999999999999},{"interface":"front","average":{"1min":1.212,"5min":1.238,"15min":1.2569999999999999},"min":{"1min":0.85499999999999998,"5min":0.78100000000000003,"15min":0.65000000000000002},"max":{"1min":1.536,"5min":1.8520000000000001,"15min":2.6240000000000001},"last":1.6080000000000001}]},{"osd":1,"last update":"Wed Aug 9 13:48:11 2023","interfaces":[{"interface":"back","average":{"1min":1.1559999999999999,"5min":1.244,"15min":1.3360000000000001},"min":{"1min":0.73099999999999998,"5min":0.73099999999999998,"15min":0.59799999999999998},"max":{"1min":1.6339999999999999,"5min":1.923,"15min":10.823},"last":1.3240000000000001},{"interface":"front","average":{"1min":1.1499999999999999,"5min":1.2609999999999999,"15min":1.347},"min":{"1min":0.72899999999999998,"5min":0.72899999999999998,"15min":0.72899999999999998},"max":{"1min":1.8899999999999999,"5min":1.8899999999999999,"15min":10.576000000000001},"last":1.9410000000000001}]},{"osd":2,"last update":"Wed Aug 9 13:48:11 2023","interfaces":[{"interface":"back","average":{"1min":1.212,"5min":1.2509999999999999,"15min":1.2569999999999999},"min":{"1min":0.65700000000000003,"5min":0.65700000000000003,"15min":0.65700000000000003},"max":{"1min":1.5920000000000001,"5min":1.736,"15min":3.5270000000000001},"last":1.016},{"interface":"front","average":{"1min":1.2170000000000001,"5min":1.3759999999999999,"15min":1.395},"min":{"1min":0.86899999999999999,"5min":0.80900000000000005,"15min":0.67800000000000005},"max":{"1min":1.8360000000000001,"5min":1.8360000000000001,"15min":3.1190000000000002},"last":1.0620000000000001}]},{"osd":3,"last update":"Wed Aug 9 13:48:11 2023","interfaces":[{"interface":"back","average":{"1min":1.2490000000000001,"5min":1.3300000000000001,"15min":1.333},"min":{"1min":0.86799999999999999,"5min":0.86799999999999999,"15min":0.77500000000000002},"max":{"1min":1.9359999999999999,"5min":1.9359999999999999,"15min":2.3010000000000002},"last":1.532},{"interface":"front","average":{"1min":1.4399999999999999,"5min":1.444,"15min":1.4419999999999999},"min":{"1min":0.88500000000000001,"5min":0.88500000000000001,"15min":0.82099999999999995},"max":{"1min":2.0790000000000002,"5min":2.0790000000000002,"15min":3.2770000000000001},"last":1.1839999999999999}]},{"osd":4,"last update":"Wed Aug 9 13:48:11 2023","interfaces":[{"interface":"back","average":{"1min":1.2210000000000001,"5min":1.2629999999999999,"15min":1.3049999999999999},"min":{"1min":0.59499999999999997,"5min":0.59499999999999997,"15min":0.59499999999999997},"max":{"1min":1.738,"5min":1.8120000000000001,"15min":3.0209999999999999},"last":2.032},{"interface":"front","average":{"1min":1.4139999999999999,"5min":1.3420000000000001,"15min":1.375},"min":{"1min":1.052,"5min":0.874,"15min":0.56799999999999995},"max":{"1min":2.044,"5min":2.044,"15min":3.169},"last":2.1419999999999999}]},{"osd":5,"last update":"Wed Aug 9 13:48:11 2023","interfaces":[{"interface":"back","average":{"1min":1.452,"5min":1.4399999999999999,"15min":1.452},"min":{"1min":0.98399999999999999,"5min":0.82599999999999996,"15min":0.71499999999999997},"max":{"1min":2.1480000000000001,"5min":2.1480000000000001,"15min":3.5830000000000002},"last":1.988},{"interface":"front","average":{"1min":1.395,"5min":1.458,"15min":1.4770000000000001},"min":{"1min":0.95699999999999996,"5min":0.85699999999999998,"15min":0.69199999999999995},"max":{"1min":2.0880000000000001,"5min":2.1520000000000001,"15min":3.3439999999999999},"last":2.113}]},{"osd":6,"last update":"Wed Aug 9 13:48:11 2023","interfaces":[{"interface":"back","average":{"1min":1.484,"5min":1.528,"15min":1.5760000000000001},"min":{"1min":0.92700000000000005,"5min":0.92700000000000005,"15min":0.92700000000000005},"max":{"1min":2.2069999999999999,"5min":2.3490000000000002,"15min":3.2189999999999999},"last":2.2010000000000001},{"interface":"front","average":{"1min":1.506,"5min":1.4970000000000001,"15min":1.508},"min":{"1min":1.0209999999999999,"5min":1,"15min":0.60799999999999998},"max":{"1min":2.1320000000000001,"5min":2.2530000000000001,"15min":2.9590000000000001},"last":1.6679999999999999}]}]},{"osd":6,"up_from":13,"seq":55834575022,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31960,"kb_used_data":840,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":94339880,"statfs":{"total":96636764160,"available":96604037120,"internally_reserved":0,"allocated":860160,"data_stored":454140,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1781,"internal_metadata":31848715},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Aug 9 13:47:19 2023","interfaces":[{"interface":"back","average":{"1min":1.1879999999999999,"5min":1.171,"15min":1.1659999999999999},"min":{"1min":0.72499999999999998,"5min":0.72499999999999998,"15min":0.56599999999999995},"max":{"1min":1.633,"5min":1.9870000000000001,"15min":1.9870000000000001},"last":1.591},{"interface":"front","average":{"1min":1.165,"5min":1.212,"15min":1.1779999999999999},"min":{"1min":0.88200000000000001,"5min":0.86699999999999999,"15min":0.72999999999999998},"max":{"1min":1.526,"5min":1.7090000000000001,"15min":1.833},"last":1.6779999999999999}]},{"osd":1,"last update":"Wed Aug 9 13:47:19 2023","interfaces":[{"interface":"back","average":{"1min":1.1679999999999999,"5min":1.238,"15min":1.2050000000000001},"min":{"1min":0.71399999999999997,"5min":0.71399999999999997,"15min":0.65500000000000003},"max":{"1min":1.587,"5min":1.823,"15min":1.823},"last":1.2989999999999999},{"interface":"front","average":{"1min":1.266,"5min":1.2549999999999999,"15min":1.25},"min":{"1min":0.97499999999999998,"5min":0.93000000000000005,"15min":0.68100000000000005},"max":{"1min":1.8660000000000001,"5min":1.8660000000000001,"15min":1.8660000000000001},"last":1.2230000000000001}]},{"osd":2,"last update":"Wed Aug 9 13:47:19 2023","interfaces":[{"interface":"back","average":{"1min":1.3839999999999999,"5min":1.347,"15min":1.296},"min":{"1min":1.006,"5min":0.92000000000000004,"15min":0.68700000000000006},"max":{"1min":1.677,"5min":1.7509999999999999,"15min":1.774},"last":1.341},{"interface":"front","average":{"1min":1.329,"5min":1.3180000000000001,"15min":1.2789999999999999},"min":{"1min":0.90300000000000002,"5min":0.90300000000000002,"15min":0.59499999999999997},"max":{"1min":1.6479999999999999,"5min":1.883,"15min":1.883},"last":1.532}]},{"osd":3,"last update":"Wed Aug 9 13:47:19 2023","interfaces":[{"interface":"back","average":{"1min":1.3620000000000001,"5min":1.4370000000000001,"15min":1.349},"min":{"1min":1.0569999999999999,"5min":1.0509999999999999,"15min":0.73799999999999999},"max":{"1min":1.8149999999999999,"5min":1.921,"15min":1.921},"last":1.8140000000000001},{"interface":"front","average":{"1min":1.3540000000000001,"5min":1.3360000000000001,"15min":1.3080000000000001},"min":{"1min":0.78200000000000003,"5min":0.78200000000000003,"15min":0.74299999999999999},"max":{"1min":1.7410000000000001,"5min":1.7669999999999999,"15min":1.915},"last":1.177}]},{"osd":4,"last update":"Wed Aug 9 13:47:19 2023","interfaces":[{"interface":"back","average":{"1min":1.286,"5min":1.224,"15min":1.286},"min":{"1min":0.90200000000000002,"5min":0.73499999999999999,"15min":0.64300000000000002},"max":{"1min":1.917,"5min":2.3479999999999999,"15min":2.3479999999999999},"last":1.4339999999999999},{"interface":"front","average":{"1min":1.3859999999999999,"5min":1.3160000000000001,"15min":1.3440000000000001},"min":{"1min":0.98099999999999998,"5min":0.82599999999999996,"15min":0.69099999999999995},"max":{"1min":1.71,"5min":1.97,"15min":1.998},"last":1.3779999999999999}]},{"osd":5,"last update":"Wed Aug 9 13:47:19 2023","interfaces":[{"interface":"back","average":{"1min":1.27,"5min":1.327,"15min":1.335},"min":{"1min":0.83799999999999997,"5min":0.83499999999999996,"15min":0.65400000000000003},"max":{"1min":1.887,"5min":1.887,"15min":2.0910000000000002},"last":1.484},{"interface":"front","average":{"1min":1.5609999999999999,"5min":1.5580000000000001,"15min":1.5640000000000001},"min":{"1min":1.3360000000000001,"5min":0.97999999999999998,"15min":0.83799999999999997},"max":{"1min":1.97,"5min":3.2370000000000001,"15min":3.2370000000000001},"last":1.756}]},{"osd":7,"last update":"Wed Aug 9 13:47:19 2023","interfaces":[{"interface":"back","average":{"1min":1.5680000000000001,"5min":1.542,"15min":1.55},"min":{"1min":1.2609999999999999,"5min":0.62,"15min":0.62},"max":{"1min":1.9419999999999999,"5min":2.0579999999999998,"15min":2.3570000000000002},"last":2.161},{"interface":"front","average":{"1min":1.456,"5min":1.5189999999999999,"15min":1.534},"min":{"1min":0.94399999999999995,"5min":0.88500000000000001,"15min":0.88500000000000001},"max":{"1min":1.9930000000000001,"5min":2.427,"15min":2.427},"last":1.6399999999999999}]}]},{"osd":5,"up_from":13,"seq":55834575022,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":36056,"kb_used_data":844,"kb_used_omap":1,"kb_used_meta":35198,"kb_avail":94335784,"statfs":{"total":96636764160,"available":96599842816,"internally_reserved":0,"allocated":864256,"data_stored":455501,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1781,"internal_metadata":36043019},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Aug 9 13:48:16 2023","interfaces":[{"interface":"back","average":{"1min":1.119,"5min":1.1759999999999999,"15min":1.1659999999999999},"min":{"1min":0.65600000000000003,"5min":0.65600000000000003,"15min":0.65600000000000003},"max":{"1min":2.1869999999999998,"5min":2.1869999999999998,"15min":2.1869999999999998},"last":1.621},{"interface":"front","average":{"1min":1.1859999999999999,"5min":1.2250000000000001,"15min":1.2},"min":{"1min":0.73799999999999999,"5min":0.73799999999999999,"15min":0.64200000000000002},"max":{"1min":1.6619999999999999,"5min":1.6619999999999999,"15min":1.7989999999999999},"last":1.175}]},{"osd":1,"last update":"Wed Aug 9 13:48:16 2023","interfaces":[{"interface":"back","average":{"1min":1.093,"5min":1.2529999999999999,"15min":1.2170000000000001},"min":{"1min":0.73199999999999998,"5min":0.73199999999999998,"15min":0.64100000000000001},"max":{"1min":1.6379999999999999,"5min":1.6379999999999999,"15min":1.736},"last":0.90000000000000002},{"interface":"front","average":{"1min":1.169,"5min":1.3160000000000001,"15min":1.264},"min":{"1min":0.66900000000000004,"5min":0.66900000000000004,"15min":0.66900000000000004},"max":{"1min":1.607,"5min":1.825,"15min":1.825},"last":1.4299999999999999}]},{"osd":2,"last update":"Wed Aug 9 13:48:16 2023","interfaces":[{"interface":"back","average":{"1min":1.2509999999999999,"5min":1.29,"15min":1.272},"min":{"1min":0.68999999999999995,"5min":0.68999999999999995,"15min":0.61599999999999999},"max":{"1min":1.756,"5min":1.756,"15min":1.899},"last":1.0489999999999999},{"interface":"front","average":{"1min":1.2989999999999999,"5min":1.3200000000000001,"15min":1.3009999999999999},"min":{"1min":0.82399999999999995,"5min":0.82399999999999995,"15min":0.66800000000000004},"max":{"1min":1.589,"5min":1.7390000000000001,"15min":1.7969999999999999},"last":1.482}]},{"osd":3,"last update":"Wed Aug 9 13:48:16 2023","interfaces":[{"interface":"back","average":{"1min":1.4119999999999999,"5min":1.3819999999999999,"15min":1.3420000000000001},"min":{"1min":1.115,"5min":1.079,"15min":0.59799999999999998},"max":{"1min":2.1840000000000002,"5min":2.1840000000000002,"15min":2.2690000000000001},"last":1.5589999999999999},{"interface":"front","average":{"1min":1.4650000000000001,"5min":1.4430000000000001,"15min":1.415},"min":{"1min":0.94899999999999995,"5min":0.94899999999999995,"15min":0.70999999999999996},"max":{"1min":2.3780000000000001,"5min":2.3780000000000001,"15min":2.3780000000000001},"last":1.5269999999999999}]},{"osd":4,"last update":"Wed Aug 9 13:48:16 2023","interfaces":[{"interface":"back","average":{"1min":1.2669999999999999,"5min":1.2769999999999999,"15min":1.2929999999999999},"min":{"1min":0.89900000000000002,"5min":0.84699999999999998,"15min":0.56599999999999995},"max":{"1min":2.0800000000000001,"5min":2.0800000000000001,"15min":2.121},"last":1.3759999999999999},{"interface":"front","average":{"1min":1.3720000000000001,"5min":1.3600000000000001,"15min":1.391},"min":{"1min":0.97099999999999997,"5min":0.85099999999999998,"15min":0.58599999999999997},"max":{"1min":2.1230000000000002,"5min":2.1230000000000002,"15min":2.3679999999999999},"last":1.7350000000000001}]},{"osd":6,"last update":"Wed Aug 9 13:48:16 2023","interfaces":[{"interface":"back","average":{"1min":1.462,"5min":1.3819999999999999,"15min":1.407},"min":{"1min":1.052,"5min":0.91800000000000004,"15min":0.66700000000000004},"max":{"1min":2.306,"5min":2.306,"15min":2.306},"last":1.327},{"interface":"front","average":{"1min":1.4370000000000001,"5min":1.446,"15min":1.4770000000000001},"min":{"1min":1.0900000000000001,"5min":1.0149999999999999,"15min":0.71199999999999997},"max":{"1min":2.2320000000000002,"5min":2.2320000000000002,"15min":2.2320000000000002},"last":1.25}]},{"osd":7,"last update":"Wed Aug 9 13:48:16 2023","interfaces":[{"interface":"back","average":{"1min":1.5940000000000001,"5min":1.5549999999999999,"15min":1.5609999999999999},"min":{"1min":1.1419999999999999,"5min":1.05,"15min":0.70899999999999996},"max":{"1min":2.452,"5min":2.452,"15min":2.4729999999999999},"last":1.6699999999999999},{"interface":"front","average":{"1min":1.5920000000000001,"5min":1.601,"15min":1.613},"min":{"1min":1.163,"5min":1.163,"15min":0.77600000000000002},"max":{"1min":2.1070000000000002,"5min":2.1070000000000002,"15min":2.3820000000000001},"last":1.597}]}]},{"osd":4,"up_from":13,"seq":55834575022,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27868,"kb_used_data":844,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343972,"statfs":{"total":96636764160,"available":96608227328,"internally_reserved":0,"allocated":864256,"data_stored":455501,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1781,"internal_metadata":27654411},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Aug 9 13:47:18 2023","interfaces":[{"interface":"back","average":{"1min":1.29,"5min":1.226,"15min":1.177},"min":{"1min":0.96799999999999997,"5min":0.83199999999999996,"15min":0.64200000000000002},"max":{"1min":1.7589999999999999,"5min":1.7589999999999999,"15min":1.7589999999999999},"last":1.012},{"interface":"front","average":{"1min":1.2629999999999999,"5min":1.304,"15min":1.2589999999999999},"min":{"1min":0.97199999999999998,"5min":0.85299999999999998,"15min":0.69999999999999996},"max":{"1min":1.5760000000000001,"5min":1.863,"15min":1.863},"last":1.157}]},{"osd":1,"last update":"Wed Aug 9 13:47:18 2023","interfaces":[{"interface":"back","average":{"1min":1.329,"5min":1.369,"15min":1.306},"min":{"1min":0.92500000000000004,"5min":0.91400000000000003,"15min":0.77100000000000002},"max":{"1min":1.8480000000000001,"5min":1.8480000000000001,"15min":1.8480000000000001},"last":1.0720000000000001},{"interface":"front","average":{"1min":1.262,"5min":1.2789999999999999,"15min":1.2310000000000001},"min":{"1min":1.022,"5min":0.81499999999999995,"15min":0.64900000000000002},"max":{"1min":1.6279999999999999,"5min":1.722,"15min":1.7290000000000001},"last":1.4350000000000001}]},{"osd":2,"last update":"Wed Aug 9 13:47:18 2023","interfaces":[{"interface":"back","average":{"1min":1.321,"5min":1.274,"15min":1.2689999999999999},"min":{"1min":0.91000000000000003,"5min":0.79900000000000004,"15min":0.78000000000000003},"max":{"1min":1.726,"5min":1.726,"15min":1.8049999999999999},"last":1.379},{"interface":"front","average":{"1min":1.3500000000000001,"5min":1.405,"15min":1.3859999999999999},"min":{"1min":1.111,"5min":0.88700000000000001,"15min":0.74299999999999999},"max":{"1min":1.649,"5min":1.7190000000000001,"15min":1.948},"last":1.268}]},{"osd":3,"last update":"Wed Aug 9 13:47:18 2023","interfaces":[{"interface":"back","average":{"1min":1.2390000000000001,"5min":1.3089999999999999,"15min":1.2969999999999999},"min":{"1min":0.97599999999999998,"5min":0.92600000000000005,"15min":0.84699999999999998},"max":{"1min":1.4670000000000001,"5min":1.7050000000000001,"15min":2.3780000000000001},"last":1.224},{"interface":"front","average":{"1min":1.4510000000000001,"5min":1.3700000000000001,"15min":1.379},"min":{"1min":1.1479999999999999,"5min":0.96999999999999997,"15min":0.63100000000000001},"max":{"1min":1.679,"5min":1.833,"15min":2.113},"last":1.4670000000000001}]},{"osd":5,"last update":"Wed Aug 9 13:47:18 2023","interfaces":[{"interface":"back","average":{"1min":1.294,"5min":1.331,"15min":1.3500000000000001},"min":{"1min":0.83099999999999996,"5min":0.63200000000000001,"15min":0.63200000000000001},"max":{"1min":1.774,"5min":1.774,"15min":2.2959999999999998},"last":0.91200000000000003},{"interface":"front","average":{"1min":1.296,"5min":1.2909999999999999,"15min":1.3160000000000001},"min":{"1min":0.93000000000000005,"5min":0.70499999999999996,"15min":0.626},"max":{"1min":1.6799999999999999,"5min":1.786,"15min":2.2240000000000002},"last":1.111}]},{"osd":6,"last update":"Wed Aug 9 13:47:18 2023","interfaces":[{"interface":"back","average":{"1min":1.2370000000000001,"5min":1.266,"15min":1.345},"min":{"1min":0.84699999999999998,"5min":0.59099999999999997,"15min":0.59099999999999997},"max":{"1min":1.6719999999999999,"5min":1.786,"15min":1.952},"last":1.5229999999999999},{"interface":"front","average":{"1min":1.472,"5min":1.5,"15min":1.508},"min":{"1min":1.022,"5min":0.76400000000000001,"15min":0.76400000000000001},"max":{"1min":1.8979999999999999,"5min":1.8979999999999999,"15min":2.1899999999999999},"last":1.351}]},{"osd":7,"last update":"Wed Aug 9 13:47:18 2023","interfaces":[{"interface":"back","average":{"1min":1.417,"5min":1.466,"15min":1.476},"min":{"1min":1.0820000000000001,"5min":1.03,"15min":0.73899999999999999},"max":{"1min":1.708,"5min":1.9450000000000001,"15min":2.2210000000000001},"last":1.3260000000000001},{"interface":"front","average":{"1min":1.468,"5min":1.484,"15min":1.518},"min":{"1min":0.97199999999999998,"5min":0.74099999999999999,"15min":0.72899999999999998},"max":{"1min":1.696,"5min":1.9750000000000001,"15min":2.0489999999999999},"last":1.577}]}]},{"osd":3,"up_from":13,"seq":55834575023,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":36056,"kb_used_data":844,"kb_used_omap":1,"kb_used_meta":35198,"kb_avail":94335784,"statfs":{"total":96636764160,"available":96599842816,"internally_reserved":0,"allocated":864256,"data_stored":455501,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1781,"internal_metadata":36043019},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Aug 9 13:47:19 2023","interfaces":[{"interface":"back","average":{"1min":0.93100000000000005,"5min":0.96399999999999997,"15min":0.93200000000000005},"min":{"1min":0.67900000000000005,"5min":0.67700000000000005,"15min":0.38200000000000001},"max":{"1min":1.234,"5min":1.6240000000000001,"15min":1.968},"last":0.95899999999999996},{"interface":"front","average":{"1min":1.157,"5min":1.1719999999999999,"15min":1.075},"min":{"1min":0.80700000000000005,"5min":0.68600000000000005,"15min":0.40799999999999997},"max":{"1min":1.528,"5min":2.0590000000000002,"15min":2.2360000000000002},"last":1.1120000000000001}]},{"osd":1,"last update":"Wed Aug 9 13:47:19 2023","interfaces":[{"interface":"back","average":{"1min":1.032,"5min":1.0800000000000001,"15min":0.98999999999999999},"min":{"1min":0.78200000000000003,"5min":0.754,"15min":0.438},"max":{"1min":1.6519999999999999,"5min":2.5649999999999999,"15min":2.5649999999999999},"last":1.367},{"interface":"front","average":{"1min":1.224,"5min":1.2470000000000001,"15min":1.171},"min":{"1min":0.92600000000000005,"5min":0.872,"15min":0.42099999999999999},"max":{"1min":1.462,"5min":2.3500000000000001,"15min":2.6760000000000002},"last":1.4730000000000001}]},{"osd":2,"last update":"Wed Aug 9 13:47:19 2023","interfaces":[{"interface":"back","average":{"1min":1.3680000000000001,"5min":1.2749999999999999,"15min":1.1579999999999999},"min":{"1min":0.96199999999999997,"5min":0.86699999999999999,"15min":0.53500000000000003},"max":{"1min":1.905,"5min":1.905,"15min":1.923},"last":1.913},{"interface":"front","average":{"1min":1.387,"5min":1.389,"15min":1.3109999999999999},"min":{"1min":1.054,"5min":0.91300000000000003,"15min":0.52000000000000002},"max":{"1min":1.635,"5min":2.0259999999999998,"15min":2.1429999999999998},"last":1.3080000000000001}]},{"osd":4,"last update":"Wed Aug 9 13:47:19 2023","interfaces":[{"interface":"back","average":{"1min":1.3220000000000001,"5min":1.319,"15min":1.292},"min":{"1min":1.03,"5min":0.89500000000000002,"15min":0.627},"max":{"1min":1.7969999999999999,"5min":1.7969999999999999,"15min":1.992},"last":1.8160000000000001},{"interface":"front","average":{"1min":1.4990000000000001,"5min":1.464,"15min":1.405},"min":{"1min":1.214,"5min":1.034,"15min":0.72999999999999998},"max":{"1min":1.964,"5min":1.964,"15min":3.0649999999999999},"last":1.5600000000000001}]},{"osd":5,"last update":"Wed Aug 9 13:47:19 2023","interfaces":[{"interface":"back","average":{"1min":1.571,"5min":1.544,"15min":1.472},"min":{"1min":1.1719999999999999,"5min":1.1719999999999999,"15min":0.67200000000000004},"max":{"1min":1.99,"5min":1.992,"15min":2.3610000000000002},"last":1.778},{"interface":"front","average":{"1min":1.5860000000000001,"5min":1.573,"15min":1.4870000000000001},"min":{"1min":1.236,"5min":1.0960000000000001,"15min":0.79100000000000004},"max":{"1min":2.177,"5min":2.2970000000000002,"15min":2.2970000000000002},"last":1.621}]},{"osd":6,"last update":"Wed Aug 9 13:47:19 2023","interfaces":[{"interface":"back","average":{"1min":1.494,"5min":1.5129999999999999,"15min":1.454},"min":{"1min":1.083,"5min":1.083,"15min":0.72599999999999998},"max":{"1min":1.7769999999999999,"5min":2.242,"15min":2.242},"last":1.972},{"interface":"front","average":{"1min":1.71,"5min":1.625,"15min":1.5529999999999999},"min":{"1min":1.3420000000000001,"5min":1.1819999999999999,"15min":0.68600000000000005},"max":{"1min":2.0790000000000002,"5min":2.0790000000000002,"15min":3.0049999999999999},"last":2.0179999999999998}]},{"osd":7,"last update":"Wed Aug 9 13:47:19 2023","interfaces":[{"interface":"back","average":{"1min":1.681,"5min":1.706,"15min":1.617},"min":{"1min":1.389,"5min":1.304,"15min":0.71599999999999997},"max":{"1min":2.323,"5min":2.419,"15min":2.6120000000000001},"last":1.863},{"interface":"front","average":{"1min":1.593,"5min":1.6200000000000001,"15min":1.5600000000000001},"min":{"1min":1.264,"5min":1.149,"15min":0.61599999999999999},"max":{"1min":1.9350000000000001,"5min":2.484,"15min":2.484},"last":2.0899999999999999}]}]},{"osd":2,"up_from":13,"seq":55834575022,"num_pgs":13,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":36060,"kb_used_data":844,"kb_used_omap":1,"kb_used_meta":35198,"kb_avail":94335780,"statfs":{"total":96636764160,"available":96599838720,"internally_reserved":0,"allocated":864256,"data_stored":455501,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1781,"internal_metadata":36043019},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Aug 9 13:48:16 2023","interfaces":[{"interface":"back","average":{"1min":0.84899999999999998,"5min":0.98699999999999999,"15min":0.95499999999999996},"min":{"1min":0.36499999999999999,"5min":0.36499999999999999,"15min":0.36499999999999999},"max":{"1min":1.306,"5min":1.752,"15min":1.8380000000000001},"last":0.63100000000000001},{"interface":"front","average":{"1min":0.85099999999999998,"5min":0.99199999999999999,"15min":1.018},"min":{"1min":0.374,"5min":0.374,"15min":0.374},"max":{"1min":2.3929999999999998,"5min":2.3929999999999998,"15min":2.3929999999999998},"last":0.59299999999999997}]},{"osd":1,"last update":"Wed Aug 9 13:48:16 2023","interfaces":[{"interface":"back","average":{"1min":1,"5min":1.1950000000000001,"15min":1.1830000000000001},"min":{"1min":0.55200000000000005,"5min":0.55200000000000005,"15min":0.54400000000000004},"max":{"1min":1.744,"5min":1.744,"15min":2.0249999999999999},"last":0.745},{"interface":"front","average":{"1min":0.97999999999999998,"5min":1.335,"15min":1.335},"min":{"1min":0.52900000000000003,"5min":0.52900000000000003,"15min":0.52900000000000003},"max":{"1min":2.4870000000000001,"5min":2.4870000000000001,"15min":2.4870000000000001},"last":0.77000000000000002}]},{"osd":3,"last update":"Wed Aug 9 13:48:16 2023","interfaces":[{"interface":"back","average":{"1min":0.75900000000000001,"5min":1.04,"15min":1.048},"min":{"1min":0.38900000000000001,"5min":0.38900000000000001,"15min":0.38900000000000001},"max":{"1min":1.55,"5min":1.7609999999999999,"15min":2.0409999999999999},"last":0.68700000000000006},{"interface":"front","average":{"1min":1.3680000000000001,"5min":1.3500000000000001,"15min":1.331},"min":{"1min":0.46200000000000002,"5min":0.46200000000000002,"15min":0.46200000000000002},"max":{"1min":8.8130000000000006,"5min":8.8130000000000006,"15min":8.8130000000000006},"last":0.80300000000000005}]},{"osd":4,"last update":"Wed Aug 9 13:48:16 2023","interfaces":[{"interface":"back","average":{"1min":1.25,"5min":1.452,"15min":1.4650000000000001},"min":{"1min":0.628,"5min":0.628,"15min":0.628},"max":{"1min":1.869,"5min":1.869,"15min":3.137},"last":1.01},{"interface":"front","average":{"1min":1.208,"5min":1.371,"15min":1.4039999999999999},"min":{"1min":0.68600000000000005,"5min":0.68600000000000005,"15min":0.68600000000000005},"max":{"1min":2.4460000000000002,"5min":2.4460000000000002,"15min":2.5150000000000001},"last":1.212}]},{"osd":5,"last update":"Wed Aug 9 13:48:16 2023","interfaces":[{"interface":"back","average":{"1min":1.208,"5min":1.4350000000000001,"15min":1.4570000000000001},"min":{"1min":0.59999999999999998,"5min":0.59999999999999998,"15min":0.59999999999999998},"max":{"1min":2.5840000000000001,"5min":2.5840000000000001,"15min":2.5840000000000001},"last":1.633},{"interface":"front","average":{"1min":1.2529999999999999,"5min":1.3580000000000001,"15min":1.371},"min":{"1min":0.56799999999999995,"5min":0.56799999999999995,"15min":0.56799999999999995},"max":{"1min":3.5659999999999998,"5min":3.5659999999999998,"15min":3.8599999999999999},"last":1.339}]},{"osd":6,"last update":"Wed Aug 9 13:48:16 2023","interfaces":[{"interface":"back","average":{"1min":1.28,"5min":1.4399999999999999,"15min":1.427},"min":{"1min":0.73799999999999999,"5min":0.73799999999999999,"15min":0.73799999999999999},"max":{"1min":3.5099999999999998,"5min":3.5099999999999998,"15min":3.5099999999999998},"last":1.4379999999999999},{"interface":"front","average":{"1min":1.3100000000000001,"5min":1.5329999999999999,"15min":1.5620000000000001},"min":{"1min":0.66300000000000003,"5min":0.66300000000000003,"15min":0.66300000000000003},"max":{"1min":2.0430000000000001,"5min":2.1400000000000001,"15min":3.5350000000000001},"last":1.659}]},{"osd":7,"last update":"Wed Aug 9 13:48:16 2023","interfaces":[{"interface":"back","average":{"1min":1.3180000000000001,"5min":1.5249999999999999,"15min":1.583},"min":{"1min":0.64600000000000002,"5min":0.64600000000000002,"15min":0.64600000000000002},"max":{"1min":2.5249999999999999,"5min":2.5249999999999999,"15min":5.2969999999999997},"last":1.2949999999999999},{"interface":"front","average":{"1min":1.385,"5min":1.5249999999999999,"15min":1.534},"min":{"1min":0.70499999999999996,"5min":0.70499999999999996,"15min":0.70499999999999996},"max":{"1min":3.609,"5min":3.609,"15min":5.4539999999999997},"last":1.601}]}]},{"osd":1,"up_from":13,"seq":55834575022,"num_pgs":14,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28448,"kb_used_data":1424,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343392,"statfs":{"total":96636764160,"available":96607633408,"internally_reserved":0,"allocated":1458176,"data_stored":1045869,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1781,"internal_metadata":27654411},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Aug 9 13:47:22 2023","interfaces":[{"interface":"back","average":{"1min":0.88900000000000001,"5min":0.95199999999999996,"15min":0.94799999999999995},"min":{"1min":0.68500000000000005,"5min":0.52100000000000002,"15min":0.32400000000000001},"max":{"1min":1.401,"5min":2.0369999999999999,"15min":2.7080000000000002},"last":0.71199999999999997},{"interface":"front","average":{"1min":1.143,"5min":1.141,"15min":1.1020000000000001},"min":{"1min":0.65700000000000003,"5min":0.65700000000000003,"15min":0.42499999999999999},"max":{"1min":1.506,"5min":2.2490000000000001,"15min":2.6349999999999998},"last":1.4550000000000001}]},{"osd":2,"last update":"Wed Aug 9 13:47:22 2023","interfaces":[{"interface":"back","average":{"1min":1.0169999999999999,"5min":1.0149999999999999,"15min":1},"min":{"1min":0.73599999999999999,"5min":0.71399999999999997,"15min":0.54200000000000004},"max":{"1min":1.3440000000000001,"5min":1.718,"15min":1.762},"last":0.75},{"interface":"front","average":{"1min":1.139,"5min":1.228,"15min":1.198},"min":{"1min":0.749,"5min":0.749,"15min":0.46500000000000002},"max":{"1min":1.639,"5min":2.145,"15min":2.2330000000000001},"last":1.0680000000000001}]},{"osd":3,"last update":"Wed Aug 9 13:47:22 2023","interfaces":[{"interface":"back","average":{"1min":1.3129999999999999,"5min":1.3819999999999999,"15min":1.3240000000000001},"min":{"1min":0.83599999999999997,"5min":0.83599999999999997,"15min":0.432},"max":{"1min":1.71,"5min":2.3210000000000002,"15min":2.3210000000000002},"last":1.2809999999999999},{"interface":"front","average":{"1min":1.3140000000000001,"5min":1.3360000000000001,"15min":1.27},"min":{"1min":0.876,"5min":0.45300000000000001,"15min":0.44400000000000001},"max":{"1min":1.7709999999999999,"5min":2.7669999999999999,"15min":3.3069999999999999},"last":1.3260000000000001}]},{"osd":4,"last update":"Wed Aug 9 13:47:22 2023","interfaces":[{"interface":"back","average":{"1min":1.397,"5min":1.4490000000000001,"15min":1.4079999999999999},"min":{"1min":0.98799999999999999,"5min":0.58099999999999996,"15min":0.58099999999999996},"max":{"1min":1.8109999999999999,"5min":2.4089999999999998,"15min":2.504},"last":1.617},{"interface":"front","average":{"1min":1.395,"5min":1.387,"15min":1.355},"min":{"1min":1.0600000000000001,"5min":1.034,"15min":0.76400000000000001},"max":{"1min":1.6659999999999999,"5min":1.901,"15min":2.5099999999999998},"last":1.236}]},{"osd":5,"last update":"Wed Aug 9 13:47:22 2023","interfaces":[{"interface":"back","average":{"1min":1.3120000000000001,"5min":1.415,"15min":1.391},"min":{"1min":0.95899999999999996,"5min":0.95899999999999996,"15min":0.81599999999999995},"max":{"1min":1.6160000000000001,"5min":2.0750000000000002,"15min":2.2570000000000001},"last":1.5169999999999999},{"interface":"front","average":{"1min":1.446,"5min":1.5920000000000001,"15min":1.5820000000000001},"min":{"1min":1.1160000000000001,"5min":1.1160000000000001,"15min":0.77700000000000002},"max":{"1min":1.774,"5min":2.4649999999999999,"15min":4.9560000000000004},"last":1.3819999999999999}]},{"osd":6,"last update":"Wed Aug 9 13:47:22 2023","interfaces":[{"interface":"back","average":{"1min":1.492,"5min":1.5449999999999999,"15min":1.5289999999999999},"min":{"1min":0.77500000000000002,"5min":0.67400000000000004,"15min":0.67400000000000004},"max":{"1min":1.8939999999999999,"5min":2.5270000000000001,"15min":3.0529999999999999},"last":1.6879999999999999},{"interface":"front","average":{"1min":1.478,"5min":1.5369999999999999,"15min":1.5289999999999999},"min":{"1min":0.88200000000000001,"5min":0.88200000000000001,"15min":0.79800000000000004},"max":{"1min":1.7669999999999999,"5min":2.1040000000000001,"15min":2.7330000000000001},"last":1.419}]},{"osd":7,"last update":"Wed Aug 9 13:47:22 2023","interfaces":[{"interface":"back","average":{"1min":1.556,"5min":1.6870000000000001,"15min":1.677},"min":{"1min":0.90300000000000002,"5min":0.90300000000000002,"15min":0.77100000000000002},"max":{"1min":2.0289999999999999,"5min":2.5790000000000002,"15min":5.1360000000000001},"last":1.486},{"interface":"front","average":{"1min":1.548,"5min":1.653,"15min":1.6259999999999999},"min":{"1min":1.1459999999999999,"5min":0.89500000000000002,"15min":0.75900000000000001},"max":{"1min":1.9670000000000001,"5min":2.6509999999999998,"15min":3.194},"last":1.5569999999999999}]}]},{"osd":0,"up_from":13,"seq":55834575022,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":36056,"kb_used_data":840,"kb_used_omap":1,"kb_used_meta":35198,"kb_avail":94335784,"statfs":{"total":96636764160,"available":96599842816,"internally_reserved":0,"allocated":860160,"data_stored":454140,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1781,"internal_metadata":36043019},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Wed Aug 9 13:47:23 2023","interfaces":[{"interface":"back","average":{"1min":0.99299999999999999,"5min":0.95099999999999996,"15min":0.91800000000000004},"min":{"1min":0.70999999999999996,"5min":0.66200000000000003,"15min":0.39600000000000002},"max":{"1min":1.756,"5min":1.756,"15min":1.756},"last":0.46300000000000002},{"interface":"front","average":{"1min":1.1899999999999999,"5min":1.131,"15min":1.0920000000000001},"min":{"1min":0.83299999999999996,"5min":0.70699999999999996,"15min":0.53200000000000003},"max":{"1min":1.6359999999999999,"5min":1.6839999999999999,"15min":1.7809999999999999},"last":0.66600000000000004}]},{"osd":2,"last update":"Wed Aug 9 13:47:23 2023","interfaces":[{"interface":"back","average":{"1min":1.2230000000000001,"5min":1.194,"15min":1.1439999999999999},"min":{"1min":0.90800000000000003,"5min":0.78900000000000003,"15min":0.44800000000000001},"max":{"1min":1.776,"5min":1.776,"15min":1.889},"last":0.77500000000000002},{"interface":"front","average":{"1min":1.1850000000000001,"5min":1.226,"15min":1.181},"min":{"1min":0.85999999999999999,"5min":0.82799999999999996,"15min":0.52700000000000002},"max":{"1min":1.7669999999999999,"5min":1.9490000000000001,"15min":2.2519999999999998},"last":0.72699999999999998}]},{"osd":3,"last update":"Wed Aug 9 13:47:23 2023","interfaces":[{"interface":"back","average":{"1min":1.175,"5min":1.1659999999999999,"15min":1.1160000000000001},"min":{"1min":0.80400000000000005,"5min":0.80400000000000005,"15min":0.54800000000000004},"max":{"1min":1.823,"5min":1.881,"15min":2.5649999999999999},"last":0.60599999999999998},{"interface":"front","average":{"1min":1.325,"5min":1.341,"15min":1.3260000000000001},"min":{"1min":0.99299999999999999,"5min":0.94199999999999995,"15min":0.55400000000000005},"max":{"1min":1.738,"5min":2.4649999999999999,"15min":2.6819999999999999},"last":0.63700000000000001}]},{"osd":4,"last update":"Wed Aug 9 13:47:23 2023","interfaces":[{"interface":"back","average":{"1min":1.357,"5min":1.363,"15min":1.351},"min":{"1min":1.0700000000000001,"5min":1.014,"15min":0.74299999999999999},"max":{"1min":2.0569999999999999,"5min":2.0569999999999999,"15min":2.0569999999999999},"last":1.196},{"interface":"front","average":{"1min":1.4790000000000001,"5min":1.4890000000000001,"15min":1.464},"min":{"1min":1.125,"5min":1.123,"15min":0.74099999999999999},"max":{"1min":1.9259999999999999,"5min":2.5390000000000001,"15min":2.5390000000000001},"last":0.86799999999999999}]},{"osd":5,"last update":"Wed Aug 9 13:47:23 2023","interfaces":[{"interface":"back","average":{"1min":1.4299999999999999,"5min":1.5,"15min":1.4690000000000001},"min":{"1min":1.115,"5min":1.0680000000000001,"15min":0.80100000000000005},"max":{"1min":2.1880000000000002,"5min":2.1880000000000002,"15min":2.1880000000000002},"last":1.012},{"interface":"front","average":{"1min":1.5389999999999999,"5min":1.4990000000000001,"15min":1.45},"min":{"1min":1.2150000000000001,"5min":1.036,"15min":0.79600000000000004},"max":{"1min":1.8999999999999999,"5min":2.0910000000000002,"15min":2.0910000000000002},"last":0.83899999999999997}]},{"osd":6,"last update":"Wed Aug 9 13:47:23 2023","interfaces":[{"interface":"back","average":{"1min":1.627,"5min":1.5780000000000001,"15min":1.5649999999999999},"min":{"1min":1.264,"5min":1.2190000000000001,"15min":0.69799999999999995},"max":{"1min":2.0569999999999999,"5min":2.7250000000000001,"15min":2.7250000000000001},"last":0.92700000000000005},{"interface":"front","average":{"1min":1.627,"5min":1.581,"15min":1.5569999999999999},"min":{"1min":1.3140000000000001,"5min":1.131,"15min":0.96499999999999997},"max":{"1min":2.097,"5min":2.1400000000000001,"15min":2.2959999999999998},"last":0.97899999999999998}]},{"osd":7,"last update":"Wed Aug 9 13:47:23 2023","interfaces":[{"interface":"back","average":{"1min":1.6870000000000001,"5min":1.6739999999999999,"15min":1.631},"min":{"1min":1.371,"5min":1.194,"15min":0.97899999999999998},"max":{"1min":2.262,"5min":2.3170000000000002,"15min":2.464},"last":0.93999999999999995},{"interface":"front","average":{"1min":1.653,"5min":1.6060000000000001,"15min":1.579},"min":{"1min":1.181,"5min":1.111,"15min":0.83199999999999996},"max":{"1min":2.04,"5min":2.1640000000000001,"15min":2.1640000000000001},"last":0.89200000000000002}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":2194,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":56,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":56,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":652,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":160,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-08-09T13:48:23.225 INFO:tasks.cephfs.filesystem.ceph_manager:clean! 2023-08-09T13:48:23.225 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth caps client.0 mon 'allow r' osd 'allow rw tag cephfs data=cephfs' mds allow 2023-08-09T13:48:23.505 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:23.502+0000 7f32b4fe8640 1 Processor -- start 2023-08-09T13:48:23.505 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:23.503+0000 7f32b4fe8640 1 -- start start 2023-08-09T13:48:23.506 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:23.504+0000 7f32b4fe8640 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7f32b015df60 0x7f32b00ce1c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:48:23.507 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:23.504+0000 7f32b4fe8640 1 -- --> v1:172.21.15.43:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f32b015aef0 con 0x7f32b015e3d0 2023-08-09T13:48:23.507 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:23.504+0000 7f32b4fe8640 1 -- --> v1:172.21.15.203:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f32b015e940 con 0x7f32b00cb2d0 2023-08-09T13:48:23.507 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:23.504+0000 7f32b4fe8640 1 -- --> v2:172.21.15.203:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f32b015a970 con 0x7f32b015df60 2023-08-09T13:48:23.508 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:23.505+0000 7f32aed9d640 1 --1- >> v1:172.21.15.43:6789/0 conn(0x7f32b015e3d0 0x7f32b00c9f90 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.43:6789/0 says I am v1:172.21.15.43:46512/0 (socket says 172.21.15.43:46512) 2023-08-09T13:48:23.508 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:23.505+0000 7f32aed9d640 1 -- 172.21.15.43:0/1745146779 learned_addr learned my addr 172.21.15.43:0/1745146779 (peer_addr_for_me v1:172.21.15.43:0/0) 2023-08-09T13:48:23.508 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:23.505+0000 7f32add9b640 1 --2- 172.21.15.43:0/1745146779 >> v2:172.21.15.203:3300/0 conn(0x7f32b015df60 0x7f32b00ce1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:48:23.508 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:23.505+0000 7f32ad59a640 1 -- 172.21.15.43:0/1745146779 <== mon.0 v1:172.21.15.43:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 556610918 0 0) 0x7f32b015aef0 con 0x7f32b015e3d0 2023-08-09T13:48:23.508 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:23.505+0000 7f32ad59a640 1 -- 172.21.15.43:0/1745146779 --> v1:172.21.15.43:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f3298003520 con 0x7f32b015e3d0 2023-08-09T13:48:23.509 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:23.506+0000 7f32ad59a640 1 -- 172.21.15.43:0/1745146779 <== mon.0 v1:172.21.15.43:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2249649959 0 0) 0x7f3298003520 con 0x7f32b015e3d0 2023-08-09T13:48:23.510 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:23.506+0000 7f32ad59a640 1 -- 172.21.15.43:0/1745146779 >> v2:172.21.15.203:3300/0 conn(0x7f32b015df60 msgr2=0x7f32b00ce1c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-09T13:48:23.510 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:23.506+0000 7f32ad59a640 1 --2- 172.21.15.43:0/1745146779 >> v2:172.21.15.203:3300/0 conn(0x7f32b015df60 0x7f32b00ce1c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:48:23.510 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:23.506+0000 7f32ad59a640 1 -- 172.21.15.43:0/1745146779 >> v1:172.21.15.203:6789/0 conn(0x7f32b00cb2d0 legacy=0x7f32b015b380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:48:23.510 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:23.506+0000 7f32ad59a640 1 -- 172.21.15.43:0/1745146779 --> v1:172.21.15.43:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f32b015ec20 con 0x7f32b015e3d0 2023-08-09T13:48:23.510 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:23.506+0000 7f32ad59a640 1 -- 172.21.15.43:0/1745146779 <== mon.0 v1:172.21.15.43:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1285150740 0 0) 0x7f32a80019f0 con 0x7f32b015e3d0 2023-08-09T13:48:23.510 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:23.506+0000 7f32ad59a640 1 -- 172.21.15.43:0/1745146779 >> v1:172.21.15.43:6789/0 conn(0x7f32b015e3d0 legacy=0x7f32b00c9f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:48:23.510 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:23.506+0000 7f32ad59a640 1 --2- 172.21.15.43:0/1745146779 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f3298003970 0x7f3298003d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:48:23.511 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:23.506+0000 7f32ad59a640 1 --2- 172.21.15.43:0/1745146779 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f32980047c0 0x7f3298004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:48:23.511 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:23.506+0000 7f32ad59a640 1 --2- 172.21.15.43:0/1745146779 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f3298005180 0x7f329800b980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:48:23.511 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:23.506+0000 7f32ad59a640 1 -- 172.21.15.43:0/1745146779 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f32b015a970 con 0x7f3298005180 2023-08-09T13:48:23.511 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:23.506+0000 7f32ad59a640 1 -- 172.21.15.43:0/1745146779 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f329800c120 con 0x7f32980047c0 2023-08-09T13:48:23.511 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:23.506+0000 7f32ad59a640 1 -- 172.21.15.43:0/1745146779 --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f329800c2e0 con 0x7f3298003970 2023-08-09T13:48:23.511 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:23.506+0000 7f32aed9d640 1 --2- 172.21.15.43:0/1745146779 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f3298005180 0x7f329800b980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:48:23.511 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:23.507+0000 7f32add9b640 1 --2- 172.21.15.43:0/1745146779 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f32980047c0 0x7f3298004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:48:23.511 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:23.507+0000 7f32ae59c640 1 --2- 172.21.15.43:0/1745146779 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f3298003970 0x7f3298003d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:48:23.512 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:23.507+0000 7f32aed9d640 1 -- 172.21.15.43:0/1745146779 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f3298003970 msgr2=0x7f3298003d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-09T13:48:23.512 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:23.507+0000 7f32aed9d640 1 --2- 172.21.15.43:0/1745146779 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f3298003970 0x7f3298003d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:48:23.512 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:23.507+0000 7f32aed9d640 1 -- 172.21.15.43:0/1745146779 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f32980047c0 msgr2=0x7f3298004c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-09T13:48:23.512 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:23.507+0000 7f32aed9d640 1 --2- 172.21.15.43:0/1745146779 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f32980047c0 0x7f3298004c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:48:23.512 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:23.507+0000 7f32aed9d640 1 -- 172.21.15.43:0/1745146779 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3298003520 con 0x7f3298005180 2023-08-09T13:48:23.512 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:23.507+0000 7f32add9b640 1 --2- 172.21.15.43:0/1745146779 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f32980047c0 0x7f3298004c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-08-09T13:48:23.513 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:23.508+0000 7f32aed9d640 1 -- 172.21.15.43:0/1745146779 reap_dead start 2023-08-09T13:48:23.514 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:23.508+0000 7f32aed9d640 1 --2- 172.21.15.43:0/1745146779 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f3298005180 0x7f329800b980 secure :-1 s=READY pgs=860 cs=0 l=1 rev1=1 crypto rx=0x7f32a800a830 tx=0x7f32980057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bb73c17a6b7a6a61 server_cookie=0 in_seq=0 out_seq=0 2023-08-09T13:48:23.514 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:23.508+0000 7f32ad59a640 1 -- 172.21.15.43:0/1745146779 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f32a80072d0 con 0x7f3298005180 2023-08-09T13:48:23.514 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:23.508+0000 7f32ad59a640 1 -- 172.21.15.43:0/1745146779 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f32a802f070 con 0x7f3298005180 2023-08-09T13:48:23.514 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:23.508+0000 7f32ad59a640 1 -- 172.21.15.43:0/1745146779 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f32a80076b0 con 0x7f3298005180 2023-08-09T13:48:23.514 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:23.509+0000 7f32b4fe8640 1 -- 172.21.15.43:0/1745146779 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f3298005180 msgr2=0x7f329800b980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:48:23.514 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:23.509+0000 7f32b4fe8640 1 --2- 172.21.15.43:0/1745146779 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f3298005180 0x7f329800b980 secure :-1 s=READY pgs=860 cs=0 l=1 rev1=1 crypto rx=0x7f32a800a830 tx=0x7f32980057f0 comp rx=0 tx=0).stop 2023-08-09T13:48:23.514 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:23.509+0000 7f32b4fe8640 1 -- 172.21.15.43:0/1745146779 shutdown_connections 2023-08-09T13:48:23.515 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:23.509+0000 7f32b4fe8640 1 --2- 172.21.15.43:0/1745146779 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f3298005180 0x7f329800b980 unknown :-1 s=CLOSED pgs=860 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:48:23.515 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:23.509+0000 7f32b4fe8640 1 -- 172.21.15.43:0/1745146779 >> 172.21.15.43:0/1745146779 conn(0x7f32b00aff50 msgr2=0x7f32b00b0360 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-09T13:48:23.515 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:23.509+0000 7f32b4fe8640 1 -- 172.21.15.43:0/1745146779 shutdown_connections 2023-08-09T13:48:23.515 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:23.509+0000 7f32b4fe8640 1 -- 172.21.15.43:0/1745146779 wait complete. 2023-08-09T13:48:23.515 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:23.511+0000 7f32b4fe8640 1 Processor -- start 2023-08-09T13:48:23.515 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:23.511+0000 7f32b4fe8640 1 -- start start 2023-08-09T13:48:23.517 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:23.512+0000 7f32b4fe8640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f3298005180 0x7f32b00cbb80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:48:23.517 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:23.512+0000 7f32b4fe8640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f32b00aca00 0x7f32b00cd3a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:48:23.517 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:23.512+0000 7f32b4fe8640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f32b00cd8e0 0x7f32b00ce190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:48:23.517 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:23.512+0000 7f32b4fe8640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f32b00ace80 con 0x7f32b00cd8e0 2023-08-09T13:48:23.517 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:23.512+0000 7f32b4fe8640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f32b01f27b0 con 0x7f32b00aca00 2023-08-09T13:48:23.517 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:23.512+0000 7f32b4fe8640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f32b01519d0 con 0x7f3298005180 2023-08-09T13:48:23.517 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:23.513+0000 7f32aed9d640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f32b00cd8e0 0x7f32b00ce190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:48:23.518 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:23.513+0000 7f32aed9d640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f32b00cd8e0 0x7f32b00ce190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:36874/0 (socket says 172.21.15.43:36874) 2023-08-09T13:48:23.518 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:23.513+0000 7f32aed9d640 1 -- 172.21.15.43:0/279112725 learned_addr learned my addr 172.21.15.43:0/279112725 (peer_addr_for_me v2:172.21.15.43:0/0) 2023-08-09T13:48:23.518 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:23.513+0000 7f32ae59c640 1 --2- 172.21.15.43:0/279112725 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f3298005180 0x7f32b00cbb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:48:23.518 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:23.513+0000 7f32add9b640 1 --2- 172.21.15.43:0/279112725 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f32b00aca00 0x7f32b00cd3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:48:23.518 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:23.513+0000 7f32aed9d640 1 -- 172.21.15.43:0/279112725 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f3298005180 msgr2=0x7f32b00cbb80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:48:23.518 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:23.513+0000 7f32aed9d640 1 --2- 172.21.15.43:0/279112725 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f3298005180 0x7f32b00cbb80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:48:23.518 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:23.513+0000 7f32aed9d640 1 -- 172.21.15.43:0/279112725 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f32b00aca00 msgr2=0x7f32b00cd3a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:48:23.519 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:23.513+0000 7f32aed9d640 1 --2- 172.21.15.43:0/279112725 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f32b00aca00 0x7f32b00cd3a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:48:23.519 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:23.513+0000 7f32aed9d640 1 -- 172.21.15.43:0/279112725 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f32b01f1ba0 con 0x7f32b00cd8e0 2023-08-09T13:48:23.519 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:23.513+0000 7f32aed9d640 1 --2- 172.21.15.43:0/279112725 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f32b00cd8e0 0x7f32b00ce190 secure :-1 s=READY pgs=861 cs=0 l=1 rev1=1 crypto rx=0x7f32a802cc40 tx=0x7f32a8001d80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-09T13:48:23.519 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:23.514+0000 7f329effd640 1 -- 172.21.15.43:0/279112725 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f32a80019b0 con 0x7f32b00cd8e0 2023-08-09T13:48:23.519 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:23.514+0000 7f329effd640 1 -- 172.21.15.43:0/279112725 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f32a8004b60 con 0x7f32b00cd8e0 2023-08-09T13:48:23.519 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:23.514+0000 7f329effd640 1 -- 172.21.15.43:0/279112725 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f32a8007450 con 0x7f32b00cd8e0 2023-08-09T13:48:23.519 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:23.514+0000 7f32b4fe8640 1 -- 172.21.15.43:0/279112725 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f32b01f1e80 con 0x7f32b00cd8e0 2023-08-09T13:48:23.519 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:23.514+0000 7f32b4fe8640 1 -- 172.21.15.43:0/279112725 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f32b01f2440 con 0x7f32b00cd8e0 2023-08-09T13:48:23.520 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:23.515+0000 7f32b4fe8640 1 -- 172.21.15.43:0/279112725 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f32b01f4e00 con 0x7f32b00cd8e0 2023-08-09T13:48:23.520 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:23.516+0000 7f329effd640 1 -- 172.21.15.43:0/279112725 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 18) v1 ==== 88443+0+0 (secure 0 0 0) 0x7f32a8031030 con 0x7f32b00cd8e0 2023-08-09T13:48:23.520 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:23.517+0000 7f329effd640 1 --2- 172.21.15.43:0/279112725 >> [v2:172.21.15.203:6832/272917756,v1:172.21.15.203:6833/272917756] conn(0x7f329006a1a0 0x7f329006c660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:48:23.521 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:23.517+0000 7f329effd640 1 -- 172.21.15.43:0/279112725 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(74..74 src has 1..74) v4 ==== 6426+0+0 (secure 0 0 0) 0x7f32a80aa700 con 0x7f32b00cd8e0 2023-08-09T13:48:23.521 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:23.518+0000 7f32ae59c640 1 --2- 172.21.15.43:0/279112725 >> [v2:172.21.15.203:6832/272917756,v1:172.21.15.203:6833/272917756] conn(0x7f329006a1a0 0x7f329006c660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:48:23.521 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:23.519+0000 7f32ae59c640 1 --2- 172.21.15.43:0/279112725 >> [v2:172.21.15.203:6832/272917756,v1:172.21.15.203:6833/272917756] conn(0x7f329006a1a0 0x7f329006c660 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f32b00abc30 tx=0x7f32a0009040 comp rx=0 tx=0).ready entity=mgr.5533 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-09T13:48:23.524 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:23.522+0000 7f329effd640 1 -- 172.21.15.43:0/279112725 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178066 (secure 0 0 0) 0x7f32b01f4e00 con 0x7f32b00cd8e0 2023-08-09T13:48:23.788 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:23.785+0000 7f32b4fe8640 1 -- 172.21.15.43:0/279112725 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]} v 0) v1 -- 0x7f32b01f4e00 con 0x7f32b00cd8e0 2023-08-09T13:48:23.794 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:23.791+0000 7f329effd640 1 -- 172.21.15.43:0/279112725 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]}]=0 updated caps for client.0 v13) v1 ==== 190+0+147 (secure 0 0 0) 0x7f32b01f4e00 con 0x7f32b00cd8e0 2023-08-09T13:48:23.794 INFO:teuthology.orchestra.run.smithi043.stdout:[client.0] 2023-08-09T13:48:23.794 INFO:teuthology.orchestra.run.smithi043.stdout: key = AQCPldNkg8zrDxAAM/hnJaE6S9HU/fRwQeBV5w== 2023-08-09T13:48:23.794 INFO:teuthology.orchestra.run.smithi043.stdout: caps mds = "allow" 2023-08-09T13:48:23.795 INFO:teuthology.orchestra.run.smithi043.stdout: caps mon = "allow r" 2023-08-09T13:48:23.795 INFO:teuthology.orchestra.run.smithi043.stdout: caps osd = "allow rw tag cephfs data=cephfs" 2023-08-09T13:48:23.795 INFO:teuthology.orchestra.run.smithi043.stderr:updated caps for client.0 2023-08-09T13:48:23.797 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:23.794+0000 7f32b4fe8640 1 -- 172.21.15.43:0/279112725 >> [v2:172.21.15.203:6832/272917756,v1:172.21.15.203:6833/272917756] conn(0x7f329006a1a0 msgr2=0x7f329006c660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:48:23.797 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:23.794+0000 7f32b4fe8640 1 --2- 172.21.15.43:0/279112725 >> [v2:172.21.15.203:6832/272917756,v1:172.21.15.203:6833/272917756] conn(0x7f329006a1a0 0x7f329006c660 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f32b00abc30 tx=0x7f32a0009040 comp rx=0 tx=0).stop 2023-08-09T13:48:23.797 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:23.794+0000 7f32b4fe8640 1 -- 172.21.15.43:0/279112725 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f32b00cd8e0 msgr2=0x7f32b00ce190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:48:23.797 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:23.794+0000 7f32b4fe8640 1 --2- 172.21.15.43:0/279112725 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f32b00cd8e0 0x7f32b00ce190 secure :-1 s=READY pgs=861 cs=0 l=1 rev1=1 crypto rx=0x7f32a802cc40 tx=0x7f32a8001d80 comp rx=0 tx=0).stop 2023-08-09T13:48:23.797 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:23.794+0000 7f32b4fe8640 1 -- 172.21.15.43:0/279112725 shutdown_connections 2023-08-09T13:48:23.797 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:23.794+0000 7f32b4fe8640 1 --2- 172.21.15.43:0/279112725 >> [v2:172.21.15.203:6832/272917756,v1:172.21.15.203:6833/272917756] conn(0x7f329006a1a0 0x7f329006c660 unknown :-1 s=CLOSED pgs=67 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:48:23.798 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:23.794+0000 7f32b4fe8640 1 --2- 172.21.15.43:0/279112725 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f32b00cd8e0 0x7f32b00ce190 unknown :-1 s=CLOSED pgs=861 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:48:23.798 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:23.794+0000 7f32b4fe8640 1 --2- 172.21.15.43:0/279112725 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f32b00aca00 0x7f32b00cd3a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:48:23.798 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:23.794+0000 7f32b4fe8640 1 --2- 172.21.15.43:0/279112725 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f3298005180 0x7f32b00cbb80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:48:23.798 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:23.794+0000 7f32b4fe8640 1 -- 172.21.15.43:0/279112725 >> 172.21.15.43:0/279112725 conn(0x7f32b00aff50 msgr2=0x7f32b0151dc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-09T13:48:23.798 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:23.795+0000 7f32b4fe8640 1 -- 172.21.15.43:0/279112725 shutdown_connections 2023-08-09T13:48:23.799 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:23.795+0000 7f32b4fe8640 1 -- 172.21.15.43:0/279112725 wait complete. 2023-08-09T13:48:23.815 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-08-09T13:48:24.042 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:24.039+0000 7fe443f8b640 1 Processor -- start 2023-08-09T13:48:24.042 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:24.039+0000 7fe443f8b640 1 -- start start 2023-08-09T13:48:24.043 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:24.041+0000 7fe443f8b640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7fe43c0b2790 0x7fe43c0d2ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:48:24.044 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:24.041+0000 7fe443f8b640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fe43c0b3050 0x7fe43c0b3f60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:48:24.044 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:24.041+0000 7fe443f8b640 1 -- --> v1:172.21.15.203:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe43c0ce090 con 0x7fe43c0b2b70 2023-08-09T13:48:24.044 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:24.041+0000 7fe443f8b640 1 -- --> v2:172.21.15.43:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe43c0b3950 con 0x7fe43c0b2790 2023-08-09T13:48:24.044 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:24.041+0000 7fe443f8b640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe43c0cd7f0 con 0x7fe43c0b3050 2023-08-09T13:48:24.044 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:24.041+0000 7fe441d27640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7fe43c0b2790 0x7fe43c0d2ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:48:24.044 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:24.041+0000 7fe441d27640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7fe43c0b2790 0x7fe43c0d2ed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:36876/0 (socket says 172.21.15.43:36876) 2023-08-09T13:48:24.044 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:24.041+0000 7fe441d27640 1 -- 172.21.15.43:0/268971350 learned_addr learned my addr 172.21.15.43:0/268971350 (peer_addr_for_me v2:172.21.15.43:0/0) 2023-08-09T13:48:24.045 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:24.042+0000 7fe442528640 1 --2- 172.21.15.43:0/268971350 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fe43c0b3050 0x7fe43c0b3f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:48:24.045 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:24.042+0000 7fe441d27640 1 -- 172.21.15.43:0/268971350 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fe43c0b3050 msgr2=0x7fe43c0b3f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-09T13:48:24.045 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:24.042+0000 7fe441d27640 1 --2- 172.21.15.43:0/268971350 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fe43c0b3050 0x7fe43c0b3f60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:48:24.046 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:24.042+0000 7fe441d27640 1 -- 172.21.15.43:0/268971350 >> v1:172.21.15.203:6789/0 conn(0x7fe43c0b2b70 legacy=0x7fe43c0b5330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-09T13:48:24.046 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:24.042+0000 7fe441d27640 1 -- 172.21.15.43:0/268971350 --> v2:172.21.15.43:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe43c0cdad0 con 0x7fe43c0b2790 2023-08-09T13:48:24.046 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:24.043+0000 7fe441d27640 1 --2- 172.21.15.43:0/268971350 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fe43c0b2790 0x7fe43c0d2ed0 secure :-1 s=READY pgs=862 cs=0 l=1 rev1=1 crypto rx=0x7fe438009870 tx=0x7fe43802b8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5723214b2a32a24e server_cookie=0 in_seq=0 out_seq=0 2023-08-09T13:48:24.047 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:24.043+0000 7fe440d25640 1 -- 172.21.15.43:0/268971350 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe438031640 con 0x7fe43c0b2790 2023-08-09T13:48:24.047 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:24.043+0000 7fe440d25640 1 -- 172.21.15.43:0/268971350 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe4380317a0 con 0x7fe43c0b2790 2023-08-09T13:48:24.047 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:24.043+0000 7fe440d25640 1 -- 172.21.15.43:0/268971350 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe438031b30 con 0x7fe43c0b2790 2023-08-09T13:48:24.047 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:24.044+0000 7fe443f8b640 1 -- 172.21.15.43:0/268971350 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fe43c0b2790 msgr2=0x7fe43c0d2ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:48:24.047 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:24.044+0000 7fe443f8b640 1 --2- 172.21.15.43:0/268971350 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fe43c0b2790 0x7fe43c0d2ed0 secure :-1 s=READY pgs=862 cs=0 l=1 rev1=1 crypto rx=0x7fe438009870 tx=0x7fe43802b8f0 comp rx=0 tx=0).stop 2023-08-09T13:48:24.047 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:24.044+0000 7fe443f8b640 1 -- 172.21.15.43:0/268971350 shutdown_connections 2023-08-09T13:48:24.048 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:24.044+0000 7fe443f8b640 1 --2- 172.21.15.43:0/268971350 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fe43c0b3050 0x7fe43c0b3f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:48:24.048 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:24.044+0000 7fe443f8b640 1 --2- 172.21.15.43:0/268971350 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fe43c0b2790 0x7fe43c0d2ed0 unknown :-1 s=CLOSED pgs=862 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:48:24.048 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:24.044+0000 7fe443f8b640 1 -- 172.21.15.43:0/268971350 >> 172.21.15.43:0/268971350 conn(0x7fe43c09b7b0 msgr2=0x7fe43c0d24e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-09T13:48:24.048 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:24.044+0000 7fe443f8b640 1 -- 172.21.15.43:0/268971350 shutdown_connections 2023-08-09T13:48:24.048 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:24.044+0000 7fe443f8b640 1 -- 172.21.15.43:0/268971350 wait complete. 2023-08-09T13:48:24.052 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:24.046+0000 7fe443f8b640 1 Processor -- start 2023-08-09T13:48:24.052 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:24.046+0000 7fe443f8b640 1 -- start start 2023-08-09T13:48:24.053 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:24.047+0000 7fe443f8b640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fe43c0b2790 0x7fe43c154930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:48:24.053 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:24.047+0000 7fe443f8b640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fe43c0b2b70 0x7fe43c0c95b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:48:24.053 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:24.047+0000 7fe443f8b640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fe43c0b3050 0x7fe43c0b7840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:48:24.053 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:24.047+0000 7fe443f8b640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe43c156900 con 0x7fe43c0b3050 2023-08-09T13:48:24.054 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:24.047+0000 7fe443f8b640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe43c0c9e00 con 0x7fe43c0b2b70 2023-08-09T13:48:24.054 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:24.047+0000 7fe443f8b640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe43c0ca0e0 con 0x7fe43c0b2790 2023-08-09T13:48:24.054 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:24.047+0000 7fe442528640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fe43c0b3050 0x7fe43c0b7840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:48:24.054 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:24.048+0000 7fe442528640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fe43c0b3050 0x7fe43c0b7840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:36882/0 (socket says 172.21.15.43:36882) 2023-08-09T13:48:24.054 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:24.048+0000 7fe442528640 1 -- 172.21.15.43:0/2242726949 learned_addr learned my addr 172.21.15.43:0/2242726949 (peer_addr_for_me v2:172.21.15.43:0/0) 2023-08-09T13:48:24.054 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:24.048+0000 7fe441d27640 1 --2- 172.21.15.43:0/2242726949 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fe43c0b2790 0x7fe43c154930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:48:24.055 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:24.048+0000 7fe441526640 1 --2- 172.21.15.43:0/2242726949 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fe43c0b2b70 0x7fe43c0c95b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:48:24.055 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:24.048+0000 7fe442528640 1 -- 172.21.15.43:0/2242726949 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fe43c0b2790 msgr2=0x7fe43c154930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:48:24.055 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:24.048+0000 7fe442528640 1 --2- 172.21.15.43:0/2242726949 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fe43c0b2790 0x7fe43c154930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:48:24.055 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:24.048+0000 7fe442528640 1 -- 172.21.15.43:0/2242726949 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fe43c0b2b70 msgr2=0x7fe43c0c95b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:48:24.055 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:24.048+0000 7fe442528640 1 --2- 172.21.15.43:0/2242726949 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fe43c0b2b70 0x7fe43c0c95b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:48:24.055 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:24.048+0000 7fe442528640 1 -- 172.21.15.43:0/2242726949 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe4380094d0 con 0x7fe43c0b3050 2023-08-09T13:48:24.056 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:24.049+0000 7fe442528640 1 --2- 172.21.15.43:0/2242726949 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fe43c0b3050 0x7fe43c0b7840 secure :-1 s=READY pgs=863 cs=0 l=1 rev1=1 crypto rx=0x7fe43400cd70 tx=0x7fe43400a660 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-09T13:48:24.056 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:24.049+0000 7fe4327fc640 1 -- 172.21.15.43:0/2242726949 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe434003b90 con 0x7fe43c0b3050 2023-08-09T13:48:24.056 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:24.049+0000 7fe4327fc640 1 -- 172.21.15.43:0/2242726949 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe434003cf0 con 0x7fe43c0b3050 2023-08-09T13:48:24.056 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:24.049+0000 7fe443f8b640 1 -- 172.21.15.43:0/2242726949 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe43c0ca3c0 con 0x7fe43c0b3050 2023-08-09T13:48:24.056 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:24.049+0000 7fe4327fc640 1 -- 172.21.15.43:0/2242726949 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe434004240 con 0x7fe43c0b3050 2023-08-09T13:48:24.056 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:24.049+0000 7fe443f8b640 1 -- 172.21.15.43:0/2242726949 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe43c0c9af0 con 0x7fe43c0b3050 2023-08-09T13:48:24.057 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:24.050+0000 7fe443f8b640 1 -- 172.21.15.43:0/2242726949 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe43c0739f0 con 0x7fe43c0b3050 2023-08-09T13:48:24.057 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:24.051+0000 7fe4327fc640 1 -- 172.21.15.43:0/2242726949 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 18) v1 ==== 88443+0+0 (secure 0 0 0) 0x7fe4340043a0 con 0x7fe43c0b3050 2023-08-09T13:48:24.058 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:24.053+0000 7fe4327fc640 1 --2- 172.21.15.43:0/2242726949 >> [v2:172.21.15.203:6832/272917756,v1:172.21.15.203:6833/272917756] conn(0x7fe42006a170 0x7fe42006c630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:48:24.058 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:24.053+0000 7fe4327fc640 1 -- 172.21.15.43:0/2242726949 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(74..74 src has 1..74) v4 ==== 6426+0+0 (secure 0 0 0) 0x7fe434012070 con 0x7fe43c0b3050 2023-08-09T13:48:24.058 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:24.053+0000 7fe441d27640 1 --2- 172.21.15.43:0/2242726949 >> [v2:172.21.15.203:6832/272917756,v1:172.21.15.203:6833/272917756] conn(0x7fe42006a170 0x7fe42006c630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:48:24.058 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:24.055+0000 7fe441d27640 1 --2- 172.21.15.43:0/2242726949 >> [v2:172.21.15.203:6832/272917756,v1:172.21.15.203:6833/272917756] conn(0x7fe42006a170 0x7fe42006c630 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7fe438036000 tx=0x7fe43803c040 comp rx=0 tx=0).ready entity=mgr.5533 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-09T13:48:24.059 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:24.057+0000 7fe4327fc640 1 -- 172.21.15.43:0/2242726949 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178066 (secure 0 0 0) 0x7fe434054640 con 0x7fe43c0b3050 2023-08-09T13:48:24.086 INFO:tasks.ceph.mds.b.smithi203.stderr:2023-08-09T13:48:24.084+0000 7f7ef76fc640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2023-08-09T13:48:24.293 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:24.290+0000 7fe443f8b640 1 -- 172.21.15.43:0/2242726949 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fe43c0b4b00 con 0x7fe43c0b3050 2023-08-09T13:48:24.294 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:24.292+0000 7fe4327fc640 1 -- 172.21.15.43:0/2242726949 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 26 v26) v1 ==== 94+0+4881 (secure 0 0 0) 0x7fe434057c60 con 0x7fe43c0b3050 2023-08-09T13:48:24.295 INFO:teuthology.orchestra.run.smithi043.stdout: 2023-08-09T13:48:24.295 INFO:teuthology.orchestra.run.smithi043.stdout:{"epoch":26,"default_fscid":3,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5369,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.203:6837/1426673901","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6836","nonce":1426673901},{"type":"v1","addr":"172.21.15.203:6837","nonce":1426673901}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":21},{"gid":5728,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.43:6833/1096124235","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6832","nonce":1096124235},{"type":"v1","addr":"172.21.15.43:6833","nonce":1096124235}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":20},{"gid":5731,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.43:6835/1688804357","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6834","nonce":1688804357},{"type":"v1","addr":"172.21.15.43:6835","nonce":1688804357}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":21}],"filesystems":[{"mdsmap":{"epoch":26,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-08-09T13:48:17.607195+0000","modified":"2023-08-09T13:48:21.179296+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":5734},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5734":{"gid":5734,"name":"b","rank":0,"incarnation":22,"state":"up:active","state_seq":3,"addr":"172.21.15.203:6835/3324252767","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6834","nonce":3324252767},{"type":"v1","addr":"172.21.15.203:6835","nonce":3324252767}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[9],"metadata_pool":8,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":3}]} 2023-08-09T13:48:24.295 INFO:teuthology.orchestra.run.smithi043.stderr:dumped fsmap epoch 26 2023-08-09T13:48:24.298 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:24.295+0000 7fe443f8b640 1 -- 172.21.15.43:0/2242726949 >> [v2:172.21.15.203:6832/272917756,v1:172.21.15.203:6833/272917756] conn(0x7fe42006a170 msgr2=0x7fe42006c630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:48:24.298 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:24.295+0000 7fe443f8b640 1 --2- 172.21.15.43:0/2242726949 >> [v2:172.21.15.203:6832/272917756,v1:172.21.15.203:6833/272917756] conn(0x7fe42006a170 0x7fe42006c630 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7fe438036000 tx=0x7fe43803c040 comp rx=0 tx=0).stop 2023-08-09T13:48:24.298 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:24.295+0000 7fe443f8b640 1 -- 172.21.15.43:0/2242726949 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fe43c0b3050 msgr2=0x7fe43c0b7840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:48:24.299 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:24.295+0000 7fe443f8b640 1 --2- 172.21.15.43:0/2242726949 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fe43c0b3050 0x7fe43c0b7840 secure :-1 s=READY pgs=863 cs=0 l=1 rev1=1 crypto rx=0x7fe43400cd70 tx=0x7fe43400a660 comp rx=0 tx=0).stop 2023-08-09T13:48:24.299 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:24.295+0000 7fe443f8b640 1 -- 172.21.15.43:0/2242726949 shutdown_connections 2023-08-09T13:48:24.299 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:24.295+0000 7fe443f8b640 1 --2- 172.21.15.43:0/2242726949 >> [v2:172.21.15.203:6832/272917756,v1:172.21.15.203:6833/272917756] conn(0x7fe42006a170 0x7fe42006c630 unknown :-1 s=CLOSED pgs=68 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:48:24.299 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:24.295+0000 7fe443f8b640 1 --2- 172.21.15.43:0/2242726949 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fe43c0b3050 0x7fe43c0b7840 unknown :-1 s=CLOSED pgs=863 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:48:24.300 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:24.295+0000 7fe443f8b640 1 --2- 172.21.15.43:0/2242726949 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fe43c0b2b70 0x7fe43c0c95b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:48:24.300 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:24.295+0000 7fe443f8b640 1 --2- 172.21.15.43:0/2242726949 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fe43c0b2790 0x7fe43c154930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:48:24.300 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:24.295+0000 7fe443f8b640 1 -- 172.21.15.43:0/2242726949 >> 172.21.15.43:0/2242726949 conn(0x7fe43c09b7b0 msgr2=0x7fe43c15ae70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-09T13:48:24.300 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:24.295+0000 7fe443f8b640 1 -- 172.21.15.43:0/2242726949 shutdown_connections 2023-08-09T13:48:24.300 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:24.295+0000 7fe443f8b640 1 -- 172.21.15.43:0/2242726949 wait complete. 2023-08-09T13:48:24.718 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 26, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2023-08-09T13:48:17.607195+0000', 'modified': '2023-08-09T13:48:21.179296+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 5734}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_5734': {'gid': 5734, 'name': 'b', 'rank': 0, 'incarnation': 22, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.203:6835/3324252767', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.203:6834', 'nonce': 3324252767}, {'type': 'v1', 'addr': '172.21.15.203:6835', 'nonce': 3324252767}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138322906710015, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}}, 'data_pools': [9], 'metadata_pool': 8, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0} 2023-08-09T13:48:24.719 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2023-08-09T13:48:24.719 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.b status 2023-08-09T13:48:24.953 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:24.951+0000 7f896edbd640 1 Processor -- start 2023-08-09T13:48:24.953 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:24.951+0000 7f896edbd640 1 -- start start 2023-08-09T13:48:24.955 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:24.952+0000 7f896edbd640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f89680c8670 0x7f896815c8e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:48:24.955 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:24.952+0000 7f896edbd640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f89680abbb0 0x7f89680ac040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:48:24.955 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:24.952+0000 7f896edbd640 1 -- --> v1:172.21.15.203:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8968070730 con 0x7f896806f3d0 2023-08-09T13:48:24.955 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:24.953+0000 7f896edbd640 1 -- --> v2:172.21.15.43:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8968156e60 con 0x7f89680abbb0 2023-08-09T13:48:24.956 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:24.953+0000 7f896edbd640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8968071620 con 0x7f89680c8670 2023-08-09T13:48:24.956 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:24.953+0000 7f896e5bc640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f89680abbb0 0x7f89680ac040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:48:24.957 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:24.953+0000 7f896e5bc640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f89680abbb0 0x7f89680ac040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:36888/0 (socket says 172.21.15.43:36888) 2023-08-09T13:48:24.957 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:24.953+0000 7f896e5bc640 1 -- 172.21.15.43:0/2555846263 learned_addr learned my addr 172.21.15.43:0/2555846263 (peer_addr_for_me v2:172.21.15.43:0/0) 2023-08-09T13:48:24.957 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:24.953+0000 7f896ddbb640 1 --2- 172.21.15.43:0/2555846263 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f89680c8670 0x7f896815c8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:48:24.957 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:24.953+0000 7f896e5bc640 1 -- 172.21.15.43:0/2555846263 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f89680c8670 msgr2=0x7f896815c8e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-09T13:48:24.957 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:24.953+0000 7f896e5bc640 1 --2- 172.21.15.43:0/2555846263 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f89680c8670 0x7f896815c8e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:48:24.957 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:24.954+0000 7f896e5bc640 1 -- 172.21.15.43:0/2555846263 >> v1:172.21.15.203:6789/0 conn(0x7f896806f3d0 legacy=0x7f89680c8a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-09T13:48:24.958 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:24.954+0000 7f896e5bc640 1 -- 172.21.15.43:0/2555846263 --> v2:172.21.15.43:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f896806fb20 con 0x7f89680abbb0 2023-08-09T13:48:24.959 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:24.955+0000 7f896e5bc640 1 --2- 172.21.15.43:0/2555846263 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f89680abbb0 0x7f89680ac040 secure :-1 s=READY pgs=864 cs=0 l=1 rev1=1 crypto rx=0x7f8960009e50 tx=0x7f896002bdb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=68dd65789aedb224 server_cookie=0 in_seq=0 out_seq=0 2023-08-09T13:48:24.959 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:24.955+0000 7f896cdb9640 1 -- 172.21.15.43:0/2555846263 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f896002c950 con 0x7f89680abbb0 2023-08-09T13:48:24.959 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:24.955+0000 7f896cdb9640 1 -- 172.21.15.43:0/2555846263 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f896002cab0 con 0x7f89680abbb0 2023-08-09T13:48:24.959 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:24.955+0000 7f896cdb9640 1 -- 172.21.15.43:0/2555846263 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f896002ce40 con 0x7f89680abbb0 2023-08-09T13:48:24.959 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:24.956+0000 7f896edbd640 1 -- 172.21.15.43:0/2555846263 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f89680abbb0 msgr2=0x7f89680ac040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:48:24.959 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:24.956+0000 7f896edbd640 1 --2- 172.21.15.43:0/2555846263 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f89680abbb0 0x7f89680ac040 secure :-1 s=READY pgs=864 cs=0 l=1 rev1=1 crypto rx=0x7f8960009e50 tx=0x7f896002bdb0 comp rx=0 tx=0).stop 2023-08-09T13:48:24.959 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:24.956+0000 7f896edbd640 1 -- 172.21.15.43:0/2555846263 shutdown_connections 2023-08-09T13:48:24.960 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:24.956+0000 7f896edbd640 1 --2- 172.21.15.43:0/2555846263 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f89680abbb0 0x7f89680ac040 unknown :-1 s=CLOSED pgs=864 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:48:24.960 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:24.956+0000 7f896edbd640 1 --2- 172.21.15.43:0/2555846263 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f89680c8670 0x7f896815c8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:48:24.960 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:24.956+0000 7f896edbd640 1 -- 172.21.15.43:0/2555846263 >> 172.21.15.43:0/2555846263 conn(0x7f89680af950 msgr2=0x7f89680afd60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-09T13:48:24.960 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:24.956+0000 7f896edbd640 1 -- 172.21.15.43:0/2555846263 shutdown_connections 2023-08-09T13:48:24.960 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:24.956+0000 7f896edbd640 1 -- 172.21.15.43:0/2555846263 wait complete. 2023-08-09T13:48:24.961 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:24.958+0000 7f896edbd640 1 Processor -- start 2023-08-09T13:48:24.961 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:24.958+0000 7f896edbd640 1 -- start start 2023-08-09T13:48:24.962 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:24.959+0000 7f896edbd640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f896806f3d0 0x7f89680c4020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:48:24.962 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:24.959+0000 7f896edbd640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f89680abbb0 0x7f89681540c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:48:24.962 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:24.959+0000 7f896edbd640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f89680c8670 0x7f896806b3d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:48:24.963 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:24.959+0000 7f896edbd640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f89680c4560 con 0x7f896806f3d0 2023-08-09T13:48:24.963 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:24.959+0000 7f896edbd640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f89680b1e50 con 0x7f89680c8670 2023-08-09T13:48:24.963 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:24.959+0000 7f896edbd640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f89680b2130 con 0x7f89680abbb0 2023-08-09T13:48:24.963 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:24.959+0000 7f896ddbb640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f896806f3d0 0x7f89680c4020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:48:24.963 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:24.959+0000 7f896ddbb640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f896806f3d0 0x7f89680c4020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:33882/0 (socket says 172.21.15.43:33882) 2023-08-09T13:48:24.963 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:24.959+0000 7f896ddbb640 1 -- 172.21.15.43:0/1993480912 learned_addr learned my addr 172.21.15.43:0/1993480912 (peer_addr_for_me v2:172.21.15.43:0/0) 2023-08-09T13:48:24.963 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:24.959+0000 7f896e5bc640 1 --2- 172.21.15.43:0/1993480912 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f89680c8670 0x7f896806b3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:48:24.963 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:24.960+0000 7f896ddbb640 1 -- 172.21.15.43:0/1993480912 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f89680abbb0 msgr2=0x7f89681540c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:48:24.964 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:24.960+0000 7f896d5ba640 1 --2- 172.21.15.43:0/1993480912 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f89680abbb0 0x7f89681540c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:48:24.964 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:24.960+0000 7f896ddbb640 1 --2- 172.21.15.43:0/1993480912 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f89680abbb0 0x7f89681540c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:48:24.964 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:24.960+0000 7f896ddbb640 1 -- 172.21.15.43:0/1993480912 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f89680c8670 msgr2=0x7f896806b3d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:48:24.964 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:24.960+0000 7f896ddbb640 1 --2- 172.21.15.43:0/1993480912 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f89680c8670 0x7f896806b3d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:48:24.964 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:24.960+0000 7f896ddbb640 1 -- 172.21.15.43:0/1993480912 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8960009b30 con 0x7f896806f3d0 2023-08-09T13:48:24.964 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:24.960+0000 7f896e5bc640 1 --2- 172.21.15.43:0/1993480912 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f89680c8670 0x7f896806b3d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-08-09T13:48:24.965 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:24.960+0000 7f896d5ba640 1 --2- 172.21.15.43:0/1993480912 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f89680abbb0 0x7f89681540c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-08-09T13:48:24.965 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:24.960+0000 7f896ddbb640 1 --2- 172.21.15.43:0/1993480912 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f896806f3d0 0x7f89680c4020 secure :-1 s=READY pgs=865 cs=0 l=1 rev1=1 crypto rx=0x7f896400c7f0 tx=0x7f896400ccb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-09T13:48:24.965 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:24.960+0000 7f895e7fc640 1 -- 172.21.15.43:0/1993480912 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8964004220 con 0x7f896806f3d0 2023-08-09T13:48:24.965 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:24.960+0000 7f895e7fc640 1 -- 172.21.15.43:0/1993480912 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f89640043a0 con 0x7f896806f3d0 2023-08-09T13:48:24.966 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:24.960+0000 7f895e7fc640 1 -- 172.21.15.43:0/1993480912 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8964007860 con 0x7f896806f3d0 2023-08-09T13:48:24.966 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:24.960+0000 7f896edbd640 1 -- 172.21.15.43:0/1993480912 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f89680b2410 con 0x7f896806f3d0 2023-08-09T13:48:24.966 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:24.961+0000 7f896edbd640 1 -- 172.21.15.43:0/1993480912 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f89680b29d0 con 0x7f896806f3d0 2023-08-09T13:48:24.966 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:24.964+0000 7f895e7fc640 1 -- 172.21.15.43:0/1993480912 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 18) v1 ==== 88443+0+0 (secure 0 0 0) 0x7f8964007bd0 con 0x7f896806f3d0 2023-08-09T13:48:24.967 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:24.965+0000 7f895e7fc640 1 --2- 172.21.15.43:0/1993480912 >> [v2:172.21.15.203:6832/272917756,v1:172.21.15.203:6833/272917756] conn(0x7f894c06a170 0x7f894c06c630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:48:24.967 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:24.965+0000 7f895e7fc640 1 -- 172.21.15.43:0/1993480912 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(74..74 src has 1..74) v4 ==== 6426+0+0 (secure 0 0 0) 0x7f8964088a70 con 0x7f896806f3d0 2023-08-09T13:48:24.968 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:24.965+0000 7f8972b8e740 1 Processor -- start 2023-08-09T13:48:24.968 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:24.966+0000 7f896d5ba640 1 --2- 172.21.15.43:0/1993480912 >> [v2:172.21.15.203:6832/272917756,v1:172.21.15.203:6833/272917756] conn(0x7f894c06a170 0x7f894c06c630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:48:24.968 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:24.966+0000 7f8972b8e740 1 -- start start 2023-08-09T13:48:24.969 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:24.966+0000 7f8972b8e740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x564823dfae30 0x564823dfb210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:48:24.969 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:24.966+0000 7f8972b8e740 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x564823dcadb0 0x564823df80a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:48:24.969 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:24.967+0000 7f8972b8e740 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x564823dce9b0 0x564823df85e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:48:24.970 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:24.967+0000 7f8972b8e740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x564823d3c8f0 con 0x564823dfae30 2023-08-09T13:48:24.970 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:24.967+0000 7f8972b8e740 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x564823d39fe0 con 0x564823dcadb0 2023-08-09T13:48:24.970 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:24.967+0000 7f8972b8e740 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x564823d358d0 con 0x564823dce9b0 2023-08-09T13:48:24.970 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:24.967+0000 7f896d5ba640 1 --2- 172.21.15.43:0/1993480912 >> [v2:172.21.15.203:6832/272917756,v1:172.21.15.203:6833/272917756] conn(0x7f894c06a170 0x7f894c06c630 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f89680b3d20 tx=0x7f8958008040 comp rx=0 tx=0).ready entity=mgr.5533 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-09T13:48:24.970 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:24.967+0000 7f896ddbb640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x564823dfae30 0x564823dfb210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:48:24.970 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:24.967+0000 7f896ddbb640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x564823dfae30 0x564823dfb210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:33884/0 (socket says 172.21.15.43:33884) 2023-08-09T13:48:24.970 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:24.967+0000 7f896ddbb640 1 -- 172.21.15.43:0/2118466847 learned_addr learned my addr 172.21.15.43:0/2118466847 (peer_addr_for_me v2:172.21.15.43:0/0) 2023-08-09T13:48:24.972 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:24.967+0000 7f896d5ba640 1 --2- 172.21.15.43:0/2118466847 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x564823dcadb0 0x564823df80a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:48:24.972 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:24.967+0000 7f896e5bc640 1 --2- 172.21.15.43:0/2118466847 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x564823dce9b0 0x564823df85e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:48:24.972 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:24.967+0000 7f896ddbb640 1 -- 172.21.15.43:0/2118466847 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x564823dce9b0 msgr2=0x564823df85e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-09T13:48:24.972 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:24.967+0000 7f896ddbb640 1 --2- 172.21.15.43:0/2118466847 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x564823dce9b0 0x564823df85e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:48:24.972 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:24.967+0000 7f896ddbb640 1 -- 172.21.15.43:0/2118466847 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x564823dcadb0 msgr2=0x564823df80a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-09T13:48:24.972 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:24.967+0000 7f896ddbb640 1 --2- 172.21.15.43:0/2118466847 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x564823dcadb0 0x564823df80a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:48:24.973 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:24.967+0000 7f896ddbb640 1 -- 172.21.15.43:0/2118466847 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x564823ceeac0 con 0x564823dfae30 2023-08-09T13:48:24.973 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:24.968+0000 7f896ddbb640 1 --2- 172.21.15.43:0/2118466847 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x564823dfae30 0x564823dfb210 secure :-1 s=READY pgs=866 cs=0 l=1 rev1=1 crypto rx=0x7f896405f1d0 tx=0x7f8964057660 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9e3a60471c26d904 server_cookie=0 in_seq=0 out_seq=0 2023-08-09T13:48:24.973 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:24.968+0000 7f894b7fe640 1 -- 172.21.15.43:0/2118466847 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8964086c20 con 0x564823dfae30 2023-08-09T13:48:24.973 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:24.968+0000 7f894b7fe640 1 -- 172.21.15.43:0/2118466847 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8964086d80 con 0x564823dfae30 2023-08-09T13:48:24.973 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:24.968+0000 7f894b7fe640 1 -- 172.21.15.43:0/2118466847 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8964080b50 con 0x564823dfae30 2023-08-09T13:48:24.974 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:24.969+0000 7f8972b8e740 1 -- 172.21.15.43:0/2118466847 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x564823dfae30 msgr2=0x564823dfb210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:48:24.974 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:24.969+0000 7f8972b8e740 1 --2- 172.21.15.43:0/2118466847 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x564823dfae30 0x564823dfb210 secure :-1 s=READY pgs=866 cs=0 l=1 rev1=1 crypto rx=0x7f896405f1d0 tx=0x7f8964057660 comp rx=0 tx=0).stop 2023-08-09T13:48:24.974 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:24.969+0000 7f8972b8e740 1 -- 172.21.15.43:0/2118466847 shutdown_connections 2023-08-09T13:48:24.974 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:24.969+0000 7f8972b8e740 1 --2- 172.21.15.43:0/2118466847 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x564823dce9b0 0x564823df85e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:48:24.974 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:24.969+0000 7f8972b8e740 1 --2- 172.21.15.43:0/2118466847 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x564823dcadb0 0x564823df80a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:48:24.974 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:24.969+0000 7f8972b8e740 1 --2- 172.21.15.43:0/2118466847 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x564823dfae30 0x564823dfb210 unknown :-1 s=CLOSED pgs=866 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:48:24.975 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:24.969+0000 7f8972b8e740 1 -- 172.21.15.43:0/2118466847 >> 172.21.15.43:0/2118466847 conn(0x564823dcd0c0 msgr2=0x564823e15c40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-09T13:48:24.975 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:24.969+0000 7f8972b8e740 1 -- 172.21.15.43:0/2118466847 shutdown_connections 2023-08-09T13:48:24.975 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:24.969+0000 7f8972b8e740 1 -- 172.21.15.43:0/2118466847 wait complete. 2023-08-09T13:48:24.975 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:24.970+0000 7f8972b8e740 1 -- start start 2023-08-09T13:48:24.977 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:24.970+0000 7f8972b8e740 1 Processor -- start 2023-08-09T13:48:24.978 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:24.971+0000 7f8972b8e740 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x564823dce9b0 0x564823df2b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:48:24.978 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:24.971+0000 7f8972b8e740 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x564823dfae30 0x564823df0a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:48:24.978 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:24.971+0000 7f8972b8e740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x564823df0f40 0x564823df3f80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:48:24.978 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:24.971+0000 7f8972b8e740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x564823cd5d40 con 0x564823df0f40 2023-08-09T13:48:24.978 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:24.971+0000 7f8972b8e740 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x564823cd59e0 con 0x564823dce9b0 2023-08-09T13:48:24.979 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:24.971+0000 7f8972b8e740 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x564823a9bf20 con 0x564823dfae30 2023-08-09T13:48:24.979 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:24.971+0000 7f896e5bc640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x564823df0f40 0x564823df3f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:48:24.979 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:24.971+0000 7f8972b8e740 10 client.0 _ll_register_callbacks cb 0x564823cee160 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-08-09T13:48:24.979 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:24.971+0000 7f896e5bc640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x564823df0f40 0x564823df3f80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:33900/0 (socket says 172.21.15.43:33900) 2023-08-09T13:48:24.979 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:24.971+0000 7f896e5bc640 1 -- 172.21.15.43:0/2523878735 learned_addr learned my addr 172.21.15.43:0/2523878735 (peer_addr_for_me v2:172.21.15.43:0/0) 2023-08-09T13:48:24.979 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:24.972+0000 7f896ddbb640 1 --2- 172.21.15.43:0/2523878735 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x564823dce9b0 0x564823df2b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:48:24.979 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:24.972+0000 7f896d5ba640 1 --2- 172.21.15.43:0/2523878735 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x564823dfae30 0x564823df0a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:48:24.980 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:24.972+0000 7f896e5bc640 1 -- 172.21.15.43:0/2523878735 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x564823dfae30 msgr2=0x564823df0a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-09T13:48:24.980 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:24.972+0000 7f896e5bc640 1 --2- 172.21.15.43:0/2523878735 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x564823dfae30 0x564823df0a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:48:24.980 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:24.972+0000 7f896e5bc640 1 -- 172.21.15.43:0/2523878735 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x564823dce9b0 msgr2=0x564823df2b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-09T13:48:24.980 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:24.972+0000 7f896e5bc640 1 --2- 172.21.15.43:0/2523878735 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x564823dce9b0 0x564823df2b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:48:24.980 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:24.972+0000 7f896e5bc640 1 -- 172.21.15.43:0/2523878735 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f89680b29d0 con 0x564823df0f40 2023-08-09T13:48:24.980 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:24.973+0000 7f896e5bc640 1 --2- 172.21.15.43:0/2523878735 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x564823df0f40 0x564823df3f80 secure :-1 s=READY pgs=867 cs=0 l=1 rev1=1 crypto rx=0x7f8960036040 tx=0x7f8960004ab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f4736778b752a0dc server_cookie=0 in_seq=0 out_seq=0 2023-08-09T13:48:24.981 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:24.973+0000 7f8948ff9640 10 client.0 ms_handle_connect on v2:172.21.15.43:3300/0 2023-08-09T13:48:24.981 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:24.973+0000 7f8948ff9640 1 -- 172.21.15.43:0/2523878735 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8960034be0 con 0x564823df0f40 2023-08-09T13:48:24.981 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:24.973+0000 7f8972b8e740 1 -- 172.21.15.43:0/2523878735 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x564823d0d500 con 0x564823df0f40 2023-08-09T13:48:24.981 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:24.973+0000 7f8948ff9640 1 -- 172.21.15.43:0/2523878735 <== mon.0 v2:172.21.15.43:3300/0 2 ==== osd_map(74..74 src has 1..74) v4 ==== 6426+0+0 (secure 0 0 0) 0x7f8960034d40 con 0x564823df0f40 2023-08-09T13:48:24.981 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:24.973+0000 7f8948ff9640 1 -- 172.21.15.43:0/2523878735 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) v2 ==== 24+0+0 (secure 0 0 0) 0x7f896003e430 con 0x564823df0f40 2023-08-09T13:48:24.981 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:24.973+0000 7f8972b8e740 10 client.6615 fetch_fsmap learned FSMap version 26 2023-08-09T13:48:24.981 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:24.973+0000 7f8972b8e740 1 -- 172.21.15.43:0/2523878735 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=2+}) v3 -- 0x564823d33f50 con 0x564823df0f40 2023-08-09T13:48:24.982 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:24.973+0000 7f8948ff9640 1 -- 172.21.15.43:0/2523878735 <== mon.0 v2:172.21.15.43:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x564823d0d500 con 0x564823df0f40 2023-08-09T13:48:24.982 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:24.974+0000 7f8948ff9640 1 -- 172.21.15.43:0/2523878735 <== mon.0 v2:172.21.15.43:3300/0 5 ==== fsmap(e 26) v1 ==== 3294+0+0 (secure 0 0 0) 0x7f896003c040 con 0x564823df0f40 2023-08-09T13:48:24.982 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:24.974+0000 7f8972b8e740 10 client.6615 fetch_fsmap finished waiting for FSMap version 26 2023-08-09T13:48:24.982 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:24.974+0000 7f8972b8e740 10 client.6615 resolve_mds: resolved name 'b' to daemon mds.b 2023-08-09T13:48:24.982 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:24.974+0000 7f8972b8e740 20 client.6615 populate_metadata read hostname 'smithi043' 2023-08-09T13:48:24.982 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:24.974+0000 7f8972b8e740 1 --2- 172.21.15.43:0/2523878735 >> [v2:172.21.15.203:6834/3324252767,v1:172.21.15.203:6835/3324252767] conn(0x564823e29e50 0x564823dfc1a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:48:24.982 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:24.974+0000 7f8972b8e740 4 client.6615 mds_command: new command op to 5734 tid=0[{"prefix": "get_command_descriptions"}] 2023-08-09T13:48:24.983 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:24.974+0000 7f8972b8e740 1 -- 172.21.15.43:0/2523878735 --> [v2:172.21.15.203:6834/3324252767,v1:172.21.15.203:6835/3324252767] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x564823d0abc0 con 0x564823e29e50 2023-08-09T13:48:24.983 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:24.974+0000 7f896ddbb640 1 --2- 172.21.15.43:0/2523878735 >> [v2:172.21.15.203:6834/3324252767,v1:172.21.15.203:6835/3324252767] conn(0x564823e29e50 0x564823dfc1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:48:24.983 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:24.975+0000 7f896ddbb640 1 --2- 172.21.15.43:0/2523878735 >> [v2:172.21.15.203:6834/3324252767,v1:172.21.15.203:6835/3324252767] conn(0x564823e29e50 0x564823dfc1a0 crc :-1 s=READY pgs=11 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=2c0256d66b1d8df2 server_cookie=f450ef4e09dde34a in_seq=0 out_seq=0 2023-08-09T13:48:24.983 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:24.975+0000 7f8948ff9640 10 client.6615 ms_handle_connect on v2:172.21.15.203:6834/3324252767 2023-08-09T13:48:24.984 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:24.977+0000 7f8948ff9640 1 -- 172.21.15.43:0/2523878735 <== mds.0 v2:172.21.15.203:6834/3324252767 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18017 (crc 0 0 0) 0x564823d0abc0 con 0x564823e29e50 2023-08-09T13:48:24.984 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:24.977+0000 7f8948ff9640 10 client.6615 handle_command_reply: tid=0 2023-08-09T13:48:24.985 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:24.977+0000 7f8972b8e740 1 client.6615 shutdown 2023-08-09T13:48:24.985 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:24.978+0000 7f8972b8e740 1 -- 172.21.15.43:0/2523878735 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x564823df0f40 msgr2=0x564823df3f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:48:24.985 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:24.978+0000 7f8972b8e740 1 --2- 172.21.15.43:0/2523878735 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x564823df0f40 0x564823df3f80 secure :-1 s=READY pgs=867 cs=0 l=1 rev1=1 crypto rx=0x7f8960036040 tx=0x7f8960004ab0 comp rx=0 tx=0).stop 2023-08-09T13:48:24.985 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:24.978+0000 7f8972b8e740 1 -- 172.21.15.43:0/2523878735 shutdown_connections 2023-08-09T13:48:24.985 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:24.978+0000 7f8972b8e740 1 --2- 172.21.15.43:0/2523878735 >> [v2:172.21.15.203:6834/3324252767,v1:172.21.15.203:6835/3324252767] conn(0x564823e29e50 0x564823dfc1a0 crc :-1 s=READY pgs=11 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:48:24.985 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:24.978+0000 7f8972b8e740 1 --2- 172.21.15.43:0/2523878735 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x564823df0f40 0x564823df3f80 unknown :-1 s=CLOSED pgs=867 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:48:24.985 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:24.978+0000 7f8972b8e740 1 --2- 172.21.15.43:0/2523878735 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x564823dfae30 0x564823df0a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:48:24.986 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:24.978+0000 7f8972b8e740 1 --2- 172.21.15.43:0/2523878735 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x564823dce9b0 0x564823df2b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:48:24.986 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:24.978+0000 7f8972b8e740 1 -- 172.21.15.43:0/2523878735 >> 172.21.15.43:0/2523878735 conn(0x564823dcadb0 msgr2=0x564823dcf370 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-09T13:48:24.986 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:24.978+0000 7f8948ff9640 0 client.6615 ms_handle_reset on v2:172.21.15.203:6834/3324252767 2023-08-09T13:48:24.986 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:24.978+0000 7f8972b8e740 1 -- 172.21.15.43:0/2523878735 shutdown_connections 2023-08-09T13:48:24.986 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:24.978+0000 7f8972b8e740 1 -- 172.21.15.43:0/2523878735 wait complete. 2023-08-09T13:48:24.986 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:24.978+0000 7f8972b8e740 20 client.6615 trim_cache size 0 max 16384 2023-08-09T13:48:24.998 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:24.995+0000 7f8972b8e740 1 Processor -- start 2023-08-09T13:48:24.998 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:24.995+0000 7f8972b8e740 1 -- start start 2023-08-09T13:48:24.999 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:24.996+0000 7f8972b8e740 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x564823dcadb0 0x564823dcf1d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:48:24.999 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:24.996+0000 7f8972b8e740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x564823dce9b0 0x564823dfea80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:48:24.999 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:24.996+0000 7f8972b8e740 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x564823df0f40 0x564823e19150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:48:24.999 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:24.996+0000 7f8972b8e740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x564823dd2490 con 0x564823dce9b0 2023-08-09T13:48:24.999 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:24.996+0000 7f8972b8e740 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x564823df89b0 con 0x564823df0f40 2023-08-09T13:48:25.000 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:24.996+0000 7f8972b8e740 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x564823dcc960 con 0x564823dcadb0 2023-08-09T13:48:25.001 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:24.997+0000 7f896d5ba640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x564823dce9b0 0x564823dfea80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:48:25.001 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:24.997+0000 7f896d5ba640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x564823dce9b0 0x564823dfea80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:33908/0 (socket says 172.21.15.43:33908) 2023-08-09T13:48:25.002 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:24.997+0000 7f896d5ba640 1 -- 172.21.15.43:0/1446553617 learned_addr learned my addr 172.21.15.43:0/1446553617 (peer_addr_for_me v2:172.21.15.43:0/0) 2023-08-09T13:48:25.002 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:24.997+0000 7f896ddbb640 1 --2- 172.21.15.43:0/1446553617 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x564823dcadb0 0x564823dcf1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:48:25.002 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:24.997+0000 7f896e5bc640 1 --2- 172.21.15.43:0/1446553617 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x564823df0f40 0x564823e19150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:48:25.002 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:24.997+0000 7f896d5ba640 1 -- 172.21.15.43:0/1446553617 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x564823dcadb0 msgr2=0x564823dcf1d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-09T13:48:25.002 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:24.997+0000 7f896d5ba640 1 --2- 172.21.15.43:0/1446553617 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x564823dcadb0 0x564823dcf1d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:48:25.002 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:24.997+0000 7f896d5ba640 1 -- 172.21.15.43:0/1446553617 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x564823df0f40 msgr2=0x564823e19150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-09T13:48:25.002 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:24.997+0000 7f896d5ba640 1 --2- 172.21.15.43:0/1446553617 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x564823df0f40 0x564823e19150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:48:25.003 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:24.997+0000 7f896d5ba640 1 -- 172.21.15.43:0/1446553617 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x564823d67300 con 0x564823dce9b0 2023-08-09T13:48:25.003 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:24.997+0000 7f896ddbb640 1 --2- 172.21.15.43:0/1446553617 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x564823dcadb0 0x564823dcf1d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-08-09T13:48:25.003 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:24.997+0000 7f896e5bc640 1 --2- 172.21.15.43:0/1446553617 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x564823df0f40 0x564823e19150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-08-09T13:48:25.003 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:24.997+0000 7f896d5ba640 1 --2- 172.21.15.43:0/1446553617 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x564823dce9b0 0x564823dfea80 secure :-1 s=READY pgs=868 cs=0 l=1 rev1=1 crypto rx=0x7f895800c990 tx=0x7f8958012000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a95cf83f87eafbc5 server_cookie=0 in_seq=0 out_seq=0 2023-08-09T13:48:25.003 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:24.998+0000 7f894bfff640 1 -- 172.21.15.43:0/1446553617 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8958013bb0 con 0x564823dce9b0 2023-08-09T13:48:25.003 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:24.998+0000 7f894bfff640 1 -- 172.21.15.43:0/1446553617 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8958013d10 con 0x564823dce9b0 2023-08-09T13:48:25.003 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:24.998+0000 7f894bfff640 1 -- 172.21.15.43:0/1446553617 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8958021670 con 0x564823dce9b0 2023-08-09T13:48:25.003 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:24.998+0000 7f8972b8e740 1 -- 172.21.15.43:0/1446553617 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x564823dce9b0 msgr2=0x564823dfea80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:48:25.004 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:24.998+0000 7f8972b8e740 1 --2- 172.21.15.43:0/1446553617 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x564823dce9b0 0x564823dfea80 secure :-1 s=READY pgs=868 cs=0 l=1 rev1=1 crypto rx=0x7f895800c990 tx=0x7f8958012000 comp rx=0 tx=0).stop 2023-08-09T13:48:25.004 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:24.999+0000 7f8972b8e740 1 -- 172.21.15.43:0/1446553617 shutdown_connections 2023-08-09T13:48:25.004 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:24.999+0000 7f8972b8e740 1 --2- 172.21.15.43:0/1446553617 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x564823df0f40 0x564823e19150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:48:25.004 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:24.999+0000 7f8972b8e740 1 --2- 172.21.15.43:0/1446553617 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x564823dce9b0 0x564823dfea80 unknown :-1 s=CLOSED pgs=868 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:48:25.004 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:24.999+0000 7f8972b8e740 1 --2- 172.21.15.43:0/1446553617 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x564823dcadb0 0x564823dcf1d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:48:25.004 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:24.999+0000 7f8972b8e740 1 -- 172.21.15.43:0/1446553617 >> 172.21.15.43:0/1446553617 conn(0x564823dc69c0 msgr2=0x564823df7a50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-09T13:48:25.004 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:24.999+0000 7f8972b8e740 1 -- 172.21.15.43:0/1446553617 shutdown_connections 2023-08-09T13:48:25.005 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:24.999+0000 7f8972b8e740 1 -- 172.21.15.43:0/1446553617 wait complete. 2023-08-09T13:48:25.006 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:24.999+0000 7f8972b8e740 1 -- start start 2023-08-09T13:48:25.007 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:25.000+0000 7f8972b8e740 1 Processor -- start 2023-08-09T13:48:25.007 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:25.001+0000 7f8972b8e740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x564823dcadb0 0x564823e19150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:48:25.007 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:25.001+0000 7f8972b8e740 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x564823df0f40 0x564823e19690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:48:25.007 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:25.001+0000 7f8972b8e740 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x564823dfae30 0x564823e7ea20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:48:25.007 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:25.001+0000 7f8972b8e740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x564823e82290 con 0x564823dcadb0 2023-08-09T13:48:25.007 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:25.001+0000 7f8972b8e740 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x564823e28570 con 0x564823dfae30 2023-08-09T13:48:25.007 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:25.001+0000 7f8972b8e740 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x564823e24c60 con 0x564823df0f40 2023-08-09T13:48:25.008 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:25.001+0000 7f8972b8e740 10 client.0 _ll_register_callbacks cb 0x7f8960009d50 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-08-09T13:48:25.008 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:25.001+0000 7f896ddbb640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x564823dcadb0 0x564823e19150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:48:25.008 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:25.001+0000 7f896ddbb640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x564823dcadb0 0x564823e19150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:33910/0 (socket says 172.21.15.43:33910) 2023-08-09T13:48:25.008 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:25.001+0000 7f896ddbb640 1 -- 172.21.15.43:0/3649367995 learned_addr learned my addr 172.21.15.43:0/3649367995 (peer_addr_for_me v2:172.21.15.43:0/0) 2023-08-09T13:48:25.008 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:25.001+0000 7f896ddbb640 1 -- 172.21.15.43:0/3649367995 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x564823df0f40 msgr2=0x564823e19690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-09T13:48:25.008 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:25.001+0000 7f896ddbb640 1 --2- 172.21.15.43:0/3649367995 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x564823df0f40 0x564823e19690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:48:25.008 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:25.001+0000 7f896ddbb640 1 -- 172.21.15.43:0/3649367995 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x564823dfae30 msgr2=0x564823e7ea20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-09T13:48:25.008 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:25.001+0000 7f896ddbb640 1 --2- 172.21.15.43:0/3649367995 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x564823dfae30 0x564823e7ea20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:48:25.009 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:25.001+0000 7f896ddbb640 1 -- 172.21.15.43:0/3649367995 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f895801b040 con 0x564823dcadb0 2023-08-09T13:48:25.009 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:25.002+0000 7f896ddbb640 1 --2- 172.21.15.43:0/3649367995 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x564823dcadb0 0x564823e19150 secure :-1 s=READY pgs=869 cs=0 l=1 rev1=1 crypto rx=0x7f8964055d40 tx=0x7f896401b070 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f1058f4183edb6cc server_cookie=0 in_seq=0 out_seq=0 2023-08-09T13:48:25.009 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:25.002+0000 7f8948ff9640 10 client.0 ms_handle_connect on v2:172.21.15.43:3300/0 2023-08-09T13:48:25.009 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:25.002+0000 7f8948ff9640 1 -- 172.21.15.43:0/3649367995 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8964055e50 con 0x564823dcadb0 2023-08-09T13:48:25.009 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:25.002+0000 7f8948ff9640 1 -- 172.21.15.43:0/3649367995 <== mon.0 v2:172.21.15.43:3300/0 2 ==== osd_map(74..74 src has 1..74) v4 ==== 6426+0+0 (secure 0 0 0) 0x7f8964080b50 con 0x564823dcadb0 2023-08-09T13:48:25.009 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:25.002+0000 7f8972b8e740 1 -- 172.21.15.43:0/3649367995 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x564823d5d800 con 0x564823dcadb0 2023-08-09T13:48:25.009 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:25.002+0000 7f8948ff9640 1 -- 172.21.15.43:0/3649367995 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) v2 ==== 24+0+0 (secure 0 0 0) 0x7f8964080d10 con 0x564823dcadb0 2023-08-09T13:48:25.010 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:25.002+0000 7f8972b8e740 10 client.6621 fetch_fsmap learned FSMap version 26 2023-08-09T13:48:25.010 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:25.002+0000 7f8972b8e740 1 -- 172.21.15.43:0/3649367995 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=2+}) v3 -- 0x564823cd4a10 con 0x564823dcadb0 2023-08-09T13:48:25.010 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:25.002+0000 7f8948ff9640 1 -- 172.21.15.43:0/3649367995 <== mon.0 v2:172.21.15.43:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x564823d5d800 con 0x564823dcadb0 2023-08-09T13:48:25.010 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:25.002+0000 7f8948ff9640 1 -- 172.21.15.43:0/3649367995 <== mon.0 v2:172.21.15.43:3300/0 5 ==== fsmap(e 26) v1 ==== 3294+0+0 (secure 0 0 0) 0x7f89640537e0 con 0x564823dcadb0 2023-08-09T13:48:25.010 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:25.002+0000 7f8972b8e740 10 client.6621 fetch_fsmap finished waiting for FSMap version 26 2023-08-09T13:48:25.010 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:25.002+0000 7f8972b8e740 10 client.6621 resolve_mds: resolved name 'b' to daemon mds.b 2023-08-09T13:48:25.011 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:25.002+0000 7f8972b8e740 20 client.6621 populate_metadata read hostname 'smithi043' 2023-08-09T13:48:25.011 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:25.002+0000 7f8972b8e740 1 --2- 172.21.15.43:0/3649367995 >> [v2:172.21.15.203:6834/3324252767,v1:172.21.15.203:6835/3324252767] conn(0x564823df7f20 0x564823e7ef60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:48:25.011 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:25.002+0000 7f8972b8e740 4 client.6621 mds_command: new command op to 5734 tid=0[{"prefix": "status"}] 2023-08-09T13:48:25.011 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:25.002+0000 7f8972b8e740 1 -- 172.21.15.43:0/3649367995 --> [v2:172.21.15.203:6834/3324252767,v1:172.21.15.203:6835/3324252767] -- command(tid 0: {"prefix": "status"}) v1 -- 0x564823d2e8b0 con 0x564823df7f20 2023-08-09T13:48:25.011 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:25.003+0000 7f896d5ba640 1 --2- 172.21.15.43:0/3649367995 >> [v2:172.21.15.203:6834/3324252767,v1:172.21.15.203:6835/3324252767] conn(0x564823df7f20 0x564823e7ef60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:48:25.012 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:25.004+0000 7f896d5ba640 1 --2- 172.21.15.43:0/3649367995 >> [v2:172.21.15.203:6834/3324252767,v1:172.21.15.203:6835/3324252767] conn(0x564823df7f20 0x564823e7ef60 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=4ed50d074f41f221 server_cookie=eb539d44cfa946d2 in_seq=0 out_seq=0 2023-08-09T13:48:25.013 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:25.004+0000 7f8948ff9640 10 client.6621 ms_handle_connect on v2:172.21.15.203:6834/3324252767 2023-08-09T13:48:25.013 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:25.005+0000 7f8948ff9640 1 -- 172.21.15.43:0/3649367995 <== mds.0 v2:172.21.15.203:6834/3324252767 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+324 (crc 0 0 0) 0x564823d2e8b0 con 0x564823df7f20 2023-08-09T13:48:25.013 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:25.005+0000 7f8948ff9640 10 client.6621 handle_command_reply: tid=0 2023-08-09T13:48:25.013 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:25.005+0000 7f8972b8e740 1 client.6621 shutdown 2023-08-09T13:48:25.013 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:25.005+0000 7f8972b8e740 1 -- 172.21.15.43:0/3649367995 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x564823dcadb0 msgr2=0x564823e19150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:48:25.013 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:25.005+0000 7f8972b8e740 1 --2- 172.21.15.43:0/3649367995 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x564823dcadb0 0x564823e19150 secure :-1 s=READY pgs=869 cs=0 l=1 rev1=1 crypto rx=0x7f8964055d40 tx=0x7f896401b070 comp rx=0 tx=0).stop 2023-08-09T13:48:25.013 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:25.005+0000 7f8972b8e740 1 -- 172.21.15.43:0/3649367995 shutdown_connections 2023-08-09T13:48:25.014 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:25.005+0000 7f8972b8e740 1 --2- 172.21.15.43:0/3649367995 >> [v2:172.21.15.203:6834/3324252767,v1:172.21.15.203:6835/3324252767] conn(0x564823df7f20 0x564823e7ef60 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:48:25.014 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:25.005+0000 7f8972b8e740 1 --2- 172.21.15.43:0/3649367995 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x564823dfae30 0x564823e7ea20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:48:25.014 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:25.005+0000 7f8972b8e740 1 --2- 172.21.15.43:0/3649367995 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x564823df0f40 0x564823e19690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:48:25.014 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:25.005+0000 7f8972b8e740 1 --2- 172.21.15.43:0/3649367995 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x564823dcadb0 0x564823e19150 unknown :-1 s=CLOSED pgs=869 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:48:25.014 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:25.005+0000 7f8948ff9640 0 client.6621 ms_handle_reset on v2:172.21.15.203:6834/3324252767 2023-08-09T13:48:25.014 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:25.005+0000 7f8972b8e740 1 -- 172.21.15.43:0/3649367995 >> 172.21.15.43:0/3649367995 conn(0x564823dc69c0 msgr2=0x564823defc20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-09T13:48:25.014 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:25.006+0000 7f8972b8e740 1 -- 172.21.15.43:0/3649367995 shutdown_connections 2023-08-09T13:48:25.014 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:25.006+0000 7f8972b8e740 1 -- 172.21.15.43:0/3649367995 wait complete. 2023-08-09T13:48:25.015 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:25.006+0000 7f8972b8e740 20 client.6621 trim_cache size 0 max 16384 2023-08-09T13:48:25.015 INFO:teuthology.orchestra.run.smithi043.stdout:{ 2023-08-09T13:48:25.015 INFO:teuthology.orchestra.run.smithi043.stdout: "cluster_fsid": "daac87ee-1337-4250-a955-aea4e408a39b", 2023-08-09T13:48:25.015 INFO:teuthology.orchestra.run.smithi043.stdout: "whoami": 0, 2023-08-09T13:48:25.015 INFO:teuthology.orchestra.run.smithi043.stdout: "id": 5734, 2023-08-09T13:48:25.016 INFO:teuthology.orchestra.run.smithi043.stdout: "want_state": "up:active", 2023-08-09T13:48:25.016 INFO:teuthology.orchestra.run.smithi043.stdout: "state": "up:active", 2023-08-09T13:48:25.016 INFO:teuthology.orchestra.run.smithi043.stdout: "fs_name": "cephfs", 2023-08-09T13:48:25.016 INFO:teuthology.orchestra.run.smithi043.stdout: "rank_uptime": 6.8468369600000001, 2023-08-09T13:48:25.017 INFO:teuthology.orchestra.run.smithi043.stdout: "mdsmap_epoch": 26, 2023-08-09T13:48:25.017 INFO:teuthology.orchestra.run.smithi043.stdout: "osdmap_epoch": 74, 2023-08-09T13:48:25.017 INFO:teuthology.orchestra.run.smithi043.stdout: "osdmap_epoch_barrier": 74, 2023-08-09T13:48:25.017 INFO:teuthology.orchestra.run.smithi043.stdout: "uptime": 12.81376962 2023-08-09T13:48:25.017 INFO:teuthology.orchestra.run.smithi043.stdout:} 2023-08-09T13:48:25.018 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:25.006+0000 7f896edbd640 1 -- 172.21.15.43:0/1993480912 >> [v2:172.21.15.203:6832/272917756,v1:172.21.15.203:6833/272917756] conn(0x7f894c06a170 msgr2=0x7f894c06c630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:48:25.018 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:25.006+0000 7f896edbd640 1 --2- 172.21.15.43:0/1993480912 >> [v2:172.21.15.203:6832/272917756,v1:172.21.15.203:6833/272917756] conn(0x7f894c06a170 0x7f894c06c630 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f89680b3d20 tx=0x7f8958008040 comp rx=0 tx=0).stop 2023-08-09T13:48:25.018 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:25.006+0000 7f896edbd640 1 -- 172.21.15.43:0/1993480912 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f896806f3d0 msgr2=0x7f89680c4020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:48:25.018 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:25.006+0000 7f896edbd640 1 --2- 172.21.15.43:0/1993480912 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f896806f3d0 0x7f89680c4020 secure :-1 s=READY pgs=865 cs=0 l=1 rev1=1 crypto rx=0x7f896400c7f0 tx=0x7f896400ccb0 comp rx=0 tx=0).stop 2023-08-09T13:48:25.018 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:25.006+0000 7f896edbd640 1 -- 172.21.15.43:0/1993480912 shutdown_connections 2023-08-09T13:48:25.019 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:25.006+0000 7f896edbd640 1 --2- 172.21.15.43:0/1993480912 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f89680c8670 0x7f896806b3d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:48:25.019 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:25.006+0000 7f896edbd640 1 --2- 172.21.15.43:0/1993480912 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f89680abbb0 0x7f89681540c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:48:25.019 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:25.006+0000 7f896edbd640 1 --2- 172.21.15.43:0/1993480912 >> [v2:172.21.15.203:6832/272917756,v1:172.21.15.203:6833/272917756] conn(0x7f894c06a170 0x7f894c06c630 unknown :-1 s=CLOSED pgs=69 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:48:25.019 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:25.006+0000 7f896edbd640 1 --2- 172.21.15.43:0/1993480912 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f896806f3d0 0x7f89680c4020 unknown :-1 s=CLOSED pgs=865 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:48:25.019 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:25.006+0000 7f896edbd640 1 -- 172.21.15.43:0/1993480912 >> 172.21.15.43:0/1993480912 conn(0x7f89680af950 msgr2=0x7f89680708e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-09T13:48:25.019 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:25.007+0000 7f896edbd640 1 -- 172.21.15.43:0/1993480912 shutdown_connections 2023-08-09T13:48:25.019 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:25.007+0000 7f896edbd640 1 -- 172.21.15.43:0/1993480912 wait complete. 2023-08-09T13:48:25.026 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2023-08-09T13:48:25.249 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:25.246+0000 7feac29d2640 1 Processor -- start 2023-08-09T13:48:25.249 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:25.246+0000 7feac29d2640 1 -- start start 2023-08-09T13:48:25.251 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:25.248+0000 7feac29d2640 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7feabc15a5e0 0x7feabc0cb140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:48:25.251 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:25.248+0000 7feac29d2640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7feabc0ade00 0x7feabc0cb680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:48:25.251 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:25.248+0000 7feac29d2640 1 -- --> v1:172.21.15.203:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7feabc0c9750 con 0x7feabc15a200 2023-08-09T13:48:25.251 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:25.248+0000 7feac29d2640 1 -- --> v2:172.21.15.203:3300/0 -- mon_getmap magic: 0 v1 -- 0x7feabc0cbbc0 con 0x7feabc15a5e0 2023-08-09T13:48:25.252 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:25.248+0000 7feac29d2640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7feabc15ee40 con 0x7feabc0ade00 2023-08-09T13:48:25.253 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:25.249+0000 7feac0f6f640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7feabc0ade00 0x7feabc0cb680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:48:25.254 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:25.249+0000 7feac0f6f640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7feabc0ade00 0x7feabc0cb680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.203:3301/0 says I am v2:172.21.15.43:35002/0 (socket says 172.21.15.43:35002) 2023-08-09T13:48:25.254 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:25.249+0000 7feac0f6f640 1 -- 172.21.15.43:0/2090014718 learned_addr learned my addr 172.21.15.43:0/2090014718 (peer_addr_for_me v2:172.21.15.43:0/0) 2023-08-09T13:48:25.254 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:25.249+0000 7feabb7fe640 1 --2- 172.21.15.43:0/2090014718 >> v2:172.21.15.203:3300/0 conn(0x7feabc15a5e0 0x7feabc0cb140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:48:25.255 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:25.249+0000 7feabbfff640 1 --1- 172.21.15.43:0/2090014718 >> v1:172.21.15.203:6789/0 conn(0x7feabc15a200 0x7feabc0c8ec0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.203:6789/0 says I am v1:172.21.15.43:48522/0 (socket says 172.21.15.43:48522) 2023-08-09T13:48:25.255 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:25.250+0000 7feac0f6f640 1 -- 172.21.15.43:0/2090014718 >> v2:172.21.15.203:3300/0 conn(0x7feabc15a5e0 msgr2=0x7feabc0cb140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-09T13:48:25.255 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:25.250+0000 7feac0f6f640 1 --2- 172.21.15.43:0/2090014718 >> v2:172.21.15.203:3300/0 conn(0x7feabc15a5e0 0x7feabc0cb140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:48:25.256 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:25.250+0000 7feac0f6f640 1 -- 172.21.15.43:0/2090014718 >> v1:172.21.15.203:6789/0 conn(0x7feabc15a200 legacy=0x7feabc0c8ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:48:25.256 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:25.250+0000 7feac0f6f640 1 -- 172.21.15.43:0/2090014718 --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7feabc15e080 con 0x7feabc0ade00 2023-08-09T13:48:25.256 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:25.250+0000 7feabb7fe640 1 --2- 172.21.15.43:0/2090014718 >> v2:172.21.15.203:3300/0 conn(0x7feabc15a5e0 0x7feabc0cb140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-08-09T13:48:25.257 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:25.251+0000 7feac0f6f640 1 --2- 172.21.15.43:0/2090014718 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7feabc0ade00 0x7feabc0cb680 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7feab4009e50 tx=0x7feab402bee0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=81aa6dffb1926b7 server_cookie=0 in_seq=0 out_seq=0 2023-08-09T13:48:25.260 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:25.251+0000 7feabaffd640 1 -- 172.21.15.43:0/2090014718 <== mon.2 v2:172.21.15.203:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feab402ca80 con 0x7feabc0ade00 2023-08-09T13:48:25.260 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:25.251+0000 7feabaffd640 1 -- 172.21.15.43:0/2090014718 <== mon.2 v2:172.21.15.203:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7feab402cbe0 con 0x7feabc0ade00 2023-08-09T13:48:25.261 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:25.251+0000 7feabaffd640 1 -- 172.21.15.43:0/2090014718 <== mon.2 v2:172.21.15.203:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feab40343f0 con 0x7feabc0ade00 2023-08-09T13:48:25.261 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:25.252+0000 7feac29d2640 1 -- 172.21.15.43:0/2090014718 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7feabc0ade00 msgr2=0x7feabc0cb680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:48:25.261 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:25.252+0000 7feac29d2640 1 --2- 172.21.15.43:0/2090014718 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7feabc0ade00 0x7feabc0cb680 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7feab4009e50 tx=0x7feab402bee0 comp rx=0 tx=0).stop 2023-08-09T13:48:25.261 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:25.252+0000 7feac29d2640 1 -- 172.21.15.43:0/2090014718 shutdown_connections 2023-08-09T13:48:25.261 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:25.252+0000 7feac29d2640 1 --2- 172.21.15.43:0/2090014718 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7feabc0ade00 0x7feabc0cb680 unknown :-1 s=CLOSED pgs=110 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:48:25.261 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:25.252+0000 7feac29d2640 1 --2- 172.21.15.43:0/2090014718 >> v2:172.21.15.203:3300/0 conn(0x7feabc15a5e0 0x7feabc0cb140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:48:25.261 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:25.252+0000 7feac29d2640 1 -- 172.21.15.43:0/2090014718 >> 172.21.15.43:0/2090014718 conn(0x7feabc0697c0 msgr2=0x7feabc069ba0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-09T13:48:25.262 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:25.252+0000 7feac29d2640 1 -- 172.21.15.43:0/2090014718 shutdown_connections 2023-08-09T13:48:25.262 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:25.252+0000 7feac29d2640 1 -- 172.21.15.43:0/2090014718 wait complete. 2023-08-09T13:48:25.262 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:25.254+0000 7feac29d2640 1 Processor -- start 2023-08-09T13:48:25.262 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:25.254+0000 7feac29d2640 1 -- start start 2023-08-09T13:48:25.262 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:25.255+0000 7feac29d2640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7feabc0ade00 0x7feabc14d5a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:48:25.262 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:25.255+0000 7feac29d2640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7feabc15a200 0x7feabc1f1220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:48:25.262 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:25.255+0000 7feac29d2640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7feabc15d350 0x7feabc14f520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:48:25.262 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:25.255+0000 7feac29d2640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feabc1f21c0 con 0x7feabc0ade00 2023-08-09T13:48:25.263 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:25.255+0000 7feac29d2640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feabc14dae0 con 0x7feabc15a200 2023-08-09T13:48:25.263 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:25.255+0000 7feac29d2640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7feabc1f1c00 con 0x7feabc15d350 2023-08-09T13:48:25.263 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:25.255+0000 7feabbfff640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7feabc0ade00 0x7feabc14d5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:48:25.263 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:25.255+0000 7feabbfff640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7feabc0ade00 0x7feabc14d5a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:33920/0 (socket says 172.21.15.43:33920) 2023-08-09T13:48:25.263 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:25.255+0000 7feabbfff640 1 -- 172.21.15.43:0/15552332 learned_addr learned my addr 172.21.15.43:0/15552332 (peer_addr_for_me v2:172.21.15.43:0/0) 2023-08-09T13:48:25.263 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:25.256+0000 7feac0f6f640 1 --2- 172.21.15.43:0/15552332 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7feabc15d350 0x7feabc14f520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:48:25.263 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:25.256+0000 7feabb7fe640 1 --2- 172.21.15.43:0/15552332 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7feabc15a200 0x7feabc1f1220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:48:25.263 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:25.256+0000 7feabbfff640 1 -- 172.21.15.43:0/15552332 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7feabc15d350 msgr2=0x7feabc14f520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:48:25.264 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:25.256+0000 7feabbfff640 1 --2- 172.21.15.43:0/15552332 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7feabc15d350 0x7feabc14f520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:48:25.264 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:25.256+0000 7feabbfff640 1 -- 172.21.15.43:0/15552332 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7feabc15a200 msgr2=0x7feabc1f1220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:48:25.264 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:25.256+0000 7feabbfff640 1 --2- 172.21.15.43:0/15552332 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7feabc15a200 0x7feabc1f1220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:48:25.264 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:25.256+0000 7feabbfff640 1 -- 172.21.15.43:0/15552332 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7feab0009400 con 0x7feabc0ade00 2023-08-09T13:48:25.264 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:25.256+0000 7feabbfff640 1 --2- 172.21.15.43:0/15552332 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7feabc0ade00 0x7feabc14d5a0 secure :-1 s=READY pgs=870 cs=0 l=1 rev1=1 crypto rx=0x7feaac010c50 tx=0x7feaac00e430 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-09T13:48:25.264 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:25.256+0000 7feab8ff9640 1 -- 172.21.15.43:0/15552332 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feaac010e60 con 0x7feabc0ade00 2023-08-09T13:48:25.264 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:25.256+0000 7feab8ff9640 1 -- 172.21.15.43:0/15552332 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7feaac00b070 con 0x7feabc0ade00 2023-08-09T13:48:25.265 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:25.256+0000 7feab8ff9640 1 -- 172.21.15.43:0/15552332 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feaac00b3e0 con 0x7feabc0ade00 2023-08-09T13:48:25.265 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:25.256+0000 7feac29d2640 1 -- 172.21.15.43:0/15552332 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7feab4009b30 con 0x7feabc0ade00 2023-08-09T13:48:25.265 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:25.257+0000 7feac29d2640 1 -- 172.21.15.43:0/15552332 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7feabc1ef2c0 con 0x7feabc0ade00 2023-08-09T13:48:25.265 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:25.258+0000 7feac29d2640 1 -- 172.21.15.43:0/15552332 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7feabc1ef120 con 0x7feabc0ade00 2023-08-09T13:48:25.266 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:25.259+0000 7feab8ff9640 1 -- 172.21.15.43:0/15552332 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 18) v1 ==== 88443+0+0 (secure 0 0 0) 0x7feaac00b540 con 0x7feabc0ade00 2023-08-09T13:48:25.266 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:25.260+0000 7feab8ff9640 1 --2- 172.21.15.43:0/15552332 >> [v2:172.21.15.203:6832/272917756,v1:172.21.15.203:6833/272917756] conn(0x7feaa806a170 0x7feaa806c630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:48:25.266 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:25.260+0000 7feab8ff9640 1 -- 172.21.15.43:0/15552332 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(74..74 src has 1..74) v4 ==== 6426+0+0 (secure 0 0 0) 0x7feaac08c900 con 0x7feabc0ade00 2023-08-09T13:48:25.266 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:25.261+0000 7feabb7fe640 1 --2- 172.21.15.43:0/15552332 >> [v2:172.21.15.203:6832/272917756,v1:172.21.15.203:6833/272917756] conn(0x7feaa806a170 0x7feaa806c630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:48:25.266 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:25.262+0000 7feabb7fe640 1 --2- 172.21.15.43:0/15552332 >> [v2:172.21.15.203:6832/272917756,v1:172.21.15.203:6833/272917756] conn(0x7feaa806a170 0x7feaa806c630 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7feabc15d200 tx=0x7feab0008040 comp rx=0 tx=0).ready entity=mgr.5533 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-09T13:48:25.267 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:25.264+0000 7feab8ff9640 1 -- 172.21.15.43:0/15552332 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178066 (secure 0 0 0) 0x7feabc1ef120 con 0x7feabc0ade00 2023-08-09T13:48:25.501 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:25.498+0000 7feac29d2640 1 -- 172.21.15.43:0/15552332 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7feabc1ef120 con 0x7feabc0ade00 2023-08-09T13:48:25.501 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:25.499+0000 7feab8ff9640 1 -- 172.21.15.43:0/15552332 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v26) v1 ==== 53+0+73 (secure 0 0 0) 0x7feabc1ef120 con 0x7feabc0ade00 2023-08-09T13:48:25.502 INFO:teuthology.orchestra.run.smithi043.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2023-08-09T13:48:25.504 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:25.502+0000 7feac29d2640 1 -- 172.21.15.43:0/15552332 >> [v2:172.21.15.203:6832/272917756,v1:172.21.15.203:6833/272917756] conn(0x7feaa806a170 msgr2=0x7feaa806c630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:48:25.505 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:25.502+0000 7feac29d2640 1 --2- 172.21.15.43:0/15552332 >> [v2:172.21.15.203:6832/272917756,v1:172.21.15.203:6833/272917756] conn(0x7feaa806a170 0x7feaa806c630 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7feabc15d200 tx=0x7feab0008040 comp rx=0 tx=0).stop 2023-08-09T13:48:25.505 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:25.502+0000 7feac29d2640 1 -- 172.21.15.43:0/15552332 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7feabc0ade00 msgr2=0x7feabc14d5a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:48:25.505 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:25.502+0000 7feac29d2640 1 --2- 172.21.15.43:0/15552332 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7feabc0ade00 0x7feabc14d5a0 secure :-1 s=READY pgs=870 cs=0 l=1 rev1=1 crypto rx=0x7feaac010c50 tx=0x7feaac00e430 comp rx=0 tx=0).stop 2023-08-09T13:48:25.505 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:25.502+0000 7feac29d2640 1 -- 172.21.15.43:0/15552332 shutdown_connections 2023-08-09T13:48:25.505 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:25.502+0000 7feac29d2640 1 --2- 172.21.15.43:0/15552332 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7feabc15d350 0x7feabc14f520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:48:25.505 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:25.502+0000 7feac29d2640 1 --2- 172.21.15.43:0/15552332 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7feabc15a200 0x7feabc1f1220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:48:25.505 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:25.502+0000 7feac29d2640 1 --2- 172.21.15.43:0/15552332 >> [v2:172.21.15.203:6832/272917756,v1:172.21.15.203:6833/272917756] conn(0x7feaa806a170 0x7feaa806c630 unknown :-1 s=CLOSED pgs=70 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:48:25.506 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:25.502+0000 7feac29d2640 1 --2- 172.21.15.43:0/15552332 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7feabc0ade00 0x7feabc14d5a0 unknown :-1 s=CLOSED pgs=870 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:48:25.506 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:25.502+0000 7feac29d2640 1 -- 172.21.15.43:0/15552332 >> 172.21.15.43:0/15552332 conn(0x7feabc0697c0 msgr2=0x7feabc150580 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-09T13:48:25.506 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:25.502+0000 7feac29d2640 1 -- 172.21.15.43:0/15552332 shutdown_connections 2023-08-09T13:48:25.506 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:25.502+0000 7feac29d2640 1 -- 172.21.15.43:0/15552332 wait complete. 2023-08-09T13:48:25.525 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2023-08-09T13:48:25.526 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2023-08-09T13:48:25.526 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi043.front.sepia.ceph.com 2023-08-09T13:48:25.526 INFO:tasks.cephfs.mount:self.client.name = client.0 2023-08-09T13:48:25.526 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2023-08-09T13:48:25.527 INFO:tasks.cephfs.mount:self.cephfs_name = None 2023-08-09T13:48:25.527 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2023-08-09T13:48:25.527 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2023-08-09T13:48:25.528 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2023-08-09T13:48:25.528 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-09T13:48:25.529 DEBUG:teuthology.orchestra.run.smithi043:> ip addr 2023-08-09T13:48:25.580 INFO:teuthology.orchestra.run.smithi043.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2023-08-09T13:48:25.580 INFO:teuthology.orchestra.run.smithi043.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2023-08-09T13:48:25.580 INFO:teuthology.orchestra.run.smithi043.stdout: inet 127.0.0.1/8 scope host lo 2023-08-09T13:48:25.581 INFO:teuthology.orchestra.run.smithi043.stdout: valid_lft forever preferred_lft forever 2023-08-09T13:48:25.581 INFO:teuthology.orchestra.run.smithi043.stdout: inet6 ::1/128 scope host 2023-08-09T13:48:25.581 INFO:teuthology.orchestra.run.smithi043.stdout: valid_lft forever preferred_lft forever 2023-08-09T13:48:25.581 INFO:teuthology.orchestra.run.smithi043.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-08-09T13:48:25.581 INFO:teuthology.orchestra.run.smithi043.stdout: link/ether 0c:c4:7a:6b:fd:9c brd ff:ff:ff:ff:ff:ff 2023-08-09T13:48:25.581 INFO:teuthology.orchestra.run.smithi043.stdout: altname enp7s0f0 2023-08-09T13:48:25.581 INFO:teuthology.orchestra.run.smithi043.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-08-09T13:48:25.582 INFO:teuthology.orchestra.run.smithi043.stdout: link/ether 0c:c4:7a:6b:fd:9d brd ff:ff:ff:ff:ff:ff 2023-08-09T13:48:25.582 INFO:teuthology.orchestra.run.smithi043.stdout: altname enp7s0f1 2023-08-09T13:48:25.582 INFO:teuthology.orchestra.run.smithi043.stdout:4: ens1f0: mtu 1500 qdisc mq state UP group default qlen 1000 2023-08-09T13:48:25.582 INFO:teuthology.orchestra.run.smithi043.stdout: link/ether 0c:c4:7a:bd:13:58 brd ff:ff:ff:ff:ff:ff 2023-08-09T13:48:25.582 INFO:teuthology.orchestra.run.smithi043.stdout: altname enp1s0f0 2023-08-09T13:48:25.582 INFO:teuthology.orchestra.run.smithi043.stdout: inet 172.21.15.43/20 brd 172.21.15.255 scope global dynamic noprefixroute ens1f0 2023-08-09T13:48:25.582 INFO:teuthology.orchestra.run.smithi043.stdout: valid_lft 41398sec preferred_lft 41398sec 2023-08-09T13:48:25.583 INFO:teuthology.orchestra.run.smithi043.stdout: inet6 fe80::ec4:7aff:febd:1358/64 scope link 2023-08-09T13:48:25.583 INFO:teuthology.orchestra.run.smithi043.stdout: valid_lft forever preferred_lft forever 2023-08-09T13:48:25.583 INFO:teuthology.orchestra.run.smithi043.stdout:5: ens1f1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-08-09T13:48:25.583 INFO:teuthology.orchestra.run.smithi043.stdout: link/ether 0c:c4:7a:bd:13:59 brd ff:ff:ff:ff:ff:ff 2023-08-09T13:48:25.583 INFO:teuthology.orchestra.run.smithi043.stdout: altname enp1s0f1 2023-08-09T13:48:25.583 INFO:teuthology.orchestra.run.smithi043.stdout:8: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2023-08-09T13:48:25.583 INFO:teuthology.orchestra.run.smithi043.stdout: link/ether 56:43:50:19:f4:0c brd ff:ff:ff:ff:ff:ff 2023-08-09T13:48:25.584 INFO:teuthology.orchestra.run.smithi043.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2023-08-09T13:48:25.584 INFO:teuthology.orchestra.run.smithi043.stdout: valid_lft forever preferred_lft forever 2023-08-09T13:48:25.584 INFO:teuthology.orchestra.run.smithi043.stdout: inet6 fe80::e494:65ff:fe92:a182/64 scope link 2023-08-09T13:48:25.584 INFO:teuthology.orchestra.run.smithi043.stdout: valid_lft forever preferred_lft forever 2023-08-09T13:48:25.584 INFO:teuthology.orchestra.run.smithi043.stdout:9: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2023-08-09T13:48:25.584 INFO:teuthology.orchestra.run.smithi043.stdout: link/ether 7e:93:26:fa:07:8f brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.0 2023-08-09T13:48:25.584 INFO:teuthology.orchestra.run.smithi043.stdout: inet6 fe80::7c93:26ff:fefa:78f/64 scope link 2023-08-09T13:48:25.584 INFO:teuthology.orchestra.run.smithi043.stdout: valid_lft forever preferred_lft forever 2023-08-09T13:48:25.585 INFO:teuthology.orchestra.run.smithi043.stdout:10: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2023-08-09T13:48:25.585 INFO:teuthology.orchestra.run.smithi043.stdout: link/ether 56:43:50:19:f4:0c brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.2 2023-08-09T13:48:25.585 INFO:teuthology.orchestra.run.smithi043.stdout: inet6 fe80::5443:50ff:fe19:f40c/64 scope link 2023-08-09T13:48:25.585 INFO:teuthology.orchestra.run.smithi043.stdout: valid_lft forever preferred_lft forever 2023-08-09T13:48:25.586 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-09T13:48:25.586 DEBUG:teuthology.orchestra.run.smithi043:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2023-08-09T13:48:25.673 INFO:teuthology.orchestra.run.smithi043.stdout:1 2023-08-09T13:48:25.678 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-09T13:48:25.678 DEBUG:teuthology.orchestra.run.smithi043:> route 2023-08-09T13:48:25.737 INFO:teuthology.orchestra.run.smithi043.stdout:Kernel IP routing table 2023-08-09T13:48:25.737 INFO:teuthology.orchestra.run.smithi043.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2023-08-09T13:48:25.737 INFO:teuthology.orchestra.run.smithi043.stdout:default _gateway 0.0.0.0 UG 100 0 0 ens1f0 2023-08-09T13:48:25.738 INFO:teuthology.orchestra.run.smithi043.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 ens1f0 2023-08-09T13:48:25.738 INFO:teuthology.orchestra.run.smithi043.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2023-08-09T13:48:25.739 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-09T13:48:25.739 DEBUG:teuthology.orchestra.run.smithi043:> (cd / && exec bash -c ' 2023-08-09T13:48:25.739 DEBUG:teuthology.orchestra.run.smithi043:> set -e 2023-08-09T13:48:25.740 DEBUG:teuthology.orchestra.run.smithi043:> sudo iptables -A FORWARD -o ens1f0 -i ceph-brx -j ACCEPT 2023-08-09T13:48:25.740 DEBUG:teuthology.orchestra.run.smithi043:> sudo iptables -A FORWARD -i ens1f0 -o ceph-brx -j ACCEPT 2023-08-09T13:48:25.740 DEBUG:teuthology.orchestra.run.smithi043:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o ens1f0 -j MASQUERADE 2023-08-09T13:48:25.741 DEBUG:teuthology.orchestra.run.smithi043:> ') 2023-08-09T13:48:25.940 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-09T13:48:25.940 DEBUG:teuthology.orchestra.run.smithi043:> ip netns list 2023-08-09T13:48:25.997 INFO:teuthology.orchestra.run.smithi043.stdout:ceph-ns--home-ubuntu-cephtest-mnt.2 (id: 1) 2023-08-09T13:48:25.997 INFO:teuthology.orchestra.run.smithi043.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2023-08-09T13:48:25.999 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2023-08-09T13:48:26.000 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-09T13:48:26.000 DEBUG:teuthology.orchestra.run.smithi043:> sudo ip link set brx.0 up 2023-08-09T13:48:26.104 INFO:tasks.cephfs.fuse_mount:Client client.0 config is {} 2023-08-09T13:48:26.105 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-08-09T13:48:26.105 DEBUG:teuthology.orchestra.run.smithi043:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2023-08-09T13:48:26.167 INFO:teuthology.orchestra.run.smithi043.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2023-08-09T13:48:26.167 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-08-09T13:48:26.168 DEBUG:teuthology.orchestra.run.smithi043:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2023-08-09T13:48:26.228 DEBUG:teuthology.orchestra.run.smithi043:> sudo modprobe fuse 2023-08-09T13:48:26.332 INFO:teuthology.orchestra.run:Running command with timeout 30 2023-08-09T13:48:26.333 DEBUG:teuthology.orchestra.run.smithi043:> sudo mount -t fusectl /sys/fs/fuse/connections /sys/fs/fuse/connections 2023-08-09T13:48:26.432 INFO:teuthology.orchestra.run.smithi043.stderr:mount: /sys/fs/fuse/connections: /sys/fs/fuse/connections already mounted or mount point busy. 2023-08-09T13:48:26.436 DEBUG:teuthology.orchestra.run:got remote process result: 32 2023-08-09T13:48:26.437 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-09T13:48:26.437 DEBUG:teuthology.orchestra.run.smithi043:> ls /sys/fs/fuse/connections 2023-08-09T13:48:26.497 INFO:tasks.cephfs.fuse_mount:Pre-mount connections: [] 2023-08-09T13:48:26.497 DEBUG:teuthology.orchestra.run.smithi043:> (cd /home/ubuntu/cephtest && exec sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-fuse -f --admin-socket '/var/run/ceph/$cluster-$name.$pid.asok' /home/ubuntu/cephtest/mnt.0 --id 0) 2023-08-09T13:48:26.504 DEBUG:teuthology.orchestra.run.smithi043:> sudo modprobe fuse 2023-08-09T13:48:26.588 INFO:teuthology.orchestra.run:Running command with timeout 30 2023-08-09T13:48:26.588 DEBUG:teuthology.orchestra.run.smithi043:> sudo mount -t fusectl /sys/fs/fuse/connections /sys/fs/fuse/connections 2023-08-09T13:48:26.664 INFO:teuthology.orchestra.run.smithi043.stderr:mount: /sys/fs/fuse/connections: /sys/fs/fuse/connections already mounted or mount point busy. 2023-08-09T13:48:26.667 DEBUG:teuthology.orchestra.run:got remote process result: 32 2023-08-09T13:48:26.668 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-09T13:48:26.668 DEBUG:teuthology.orchestra.run.smithi043:> ls /sys/fs/fuse/connections 2023-08-09T13:48:26.706 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi043.stderr:2023-08-09T13:48:26.703+0000 7fb57d43a480 -1 init, newargv = 0x55815e1ac390 newargc=15 2023-08-09T13:48:26.706 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi043.stderr:ceph-fuse[113478]: starting ceph client 2023-08-09T13:48:26.716 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi043.stderr:ceph-fuse[113478]: starting fuse 2023-08-09T13:48:27.815 DEBUG:teuthology.orchestra.run.smithi043:> sudo modprobe fuse 2023-08-09T13:48:27.933 INFO:teuthology.orchestra.run:Running command with timeout 30 2023-08-09T13:48:27.934 DEBUG:teuthology.orchestra.run.smithi043:> sudo mount -t fusectl /sys/fs/fuse/connections /sys/fs/fuse/connections 2023-08-09T13:48:28.039 INFO:teuthology.orchestra.run.smithi043.stderr:mount: /sys/fs/fuse/connections: /sys/fs/fuse/connections already mounted or mount point busy. 2023-08-09T13:48:28.044 DEBUG:teuthology.orchestra.run:got remote process result: 32 2023-08-09T13:48:28.045 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-09T13:48:28.046 DEBUG:teuthology.orchestra.run.smithi043:> ls /sys/fs/fuse/connections 2023-08-09T13:48:28.103 INFO:teuthology.orchestra.run.smithi043.stdout:40 2023-08-09T13:48:28.104 INFO:tasks.cephfs.fuse_mount:Post-mount connections: [40] 2023-08-09T13:48:28.104 DEBUG:teuthology.orchestra.run.smithi043:> sudo stdin-killer -- python3 -c ' 2023-08-09T13:48:28.105 DEBUG:teuthology.orchestra.run.smithi043:> import glob 2023-08-09T13:48:28.105 DEBUG:teuthology.orchestra.run.smithi043:> import re 2023-08-09T13:48:28.105 DEBUG:teuthology.orchestra.run.smithi043:> import os 2023-08-09T13:48:28.106 DEBUG:teuthology.orchestra.run.smithi043:> import subprocess 2023-08-09T13:48:28.106 DEBUG:teuthology.orchestra.run.smithi043:> 2023-08-09T13:48:28.106 DEBUG:teuthology.orchestra.run.smithi043:> def _find_admin_socket(client_name): 2023-08-09T13:48:28.107 DEBUG:teuthology.orchestra.run.smithi043:> asok_path = "/var/run/ceph/ceph-client.0.*.asok" 2023-08-09T13:48:28.107 DEBUG:teuthology.orchestra.run.smithi043:> files = glob.glob(asok_path) 2023-08-09T13:48:28.107 DEBUG:teuthology.orchestra.run.smithi043:> mountpoint = "/home/ubuntu/cephtest/mnt.0" 2023-08-09T13:48:28.108 DEBUG:teuthology.orchestra.run.smithi043:> 2023-08-09T13:48:28.108 DEBUG:teuthology.orchestra.run.smithi043:> # Given a non-glob path, it better be there 2023-08-09T13:48:28.108 DEBUG:teuthology.orchestra.run.smithi043:> if "*" not in asok_path: 2023-08-09T13:48:28.109 DEBUG:teuthology.orchestra.run.smithi043:> assert(len(files) == 1) 2023-08-09T13:48:28.109 DEBUG:teuthology.orchestra.run.smithi043:> return files[0] 2023-08-09T13:48:28.109 DEBUG:teuthology.orchestra.run.smithi043:> 2023-08-09T13:48:28.110 DEBUG:teuthology.orchestra.run.smithi043:> for f in files: 2023-08-09T13:48:28.110 DEBUG:teuthology.orchestra.run.smithi043:> pid = re.match(".*\.(\d+)\.asok$", f).group(1) 2023-08-09T13:48:28.110 DEBUG:teuthology.orchestra.run.smithi043:> if os.path.exists("/proc/{0}".format(pid)): 2023-08-09T13:48:28.110 DEBUG:teuthology.orchestra.run.smithi043:> with open("/proc/{0}/cmdline".format(pid), '"'"'r'"'"') as proc_f: 2023-08-09T13:48:28.111 DEBUG:teuthology.orchestra.run.smithi043:> contents = proc_f.read() 2023-08-09T13:48:28.111 DEBUG:teuthology.orchestra.run.smithi043:> if mountpoint in contents: 2023-08-09T13:48:28.111 DEBUG:teuthology.orchestra.run.smithi043:> return f 2023-08-09T13:48:28.112 DEBUG:teuthology.orchestra.run.smithi043:> raise RuntimeError("Client socket {0} not found".format(client_name)) 2023-08-09T13:48:28.112 DEBUG:teuthology.orchestra.run.smithi043:> 2023-08-09T13:48:28.112 DEBUG:teuthology.orchestra.run.smithi043:> print(_find_admin_socket("client.0")) 2023-08-09T13:48:28.113 DEBUG:teuthology.orchestra.run.smithi043:> ' 2023-08-09T13:48:28.304 INFO:teuthology.orchestra.run.smithi043.stdout:/var/run/ceph/ceph-client.0.113478.asok 2023-08-09T13:48:28.309 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:28 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2023-08-09T13:48:28.323 INFO:tasks.cephfs.fuse_mount:Found client admin socket at /var/run/ceph/ceph-client.0.113478.asok 2023-08-09T13:48:28.323 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-09T13:48:28.324 DEBUG:teuthology.orchestra.run.smithi043:> sudo ceph --admin-daemon /var/run/ceph/ceph-client.0.113478.asok status 2023-08-09T13:48:28.578 INFO:teuthology.orchestra.run.smithi043.stdout:{ 2023-08-09T13:48:28.579 INFO:teuthology.orchestra.run.smithi043.stdout: "metadata": { 2023-08-09T13:48:28.579 INFO:teuthology.orchestra.run.smithi043.stdout: "ceph_sha1": "668a86f232c9454b2ce5928d1432a29a72964d85", 2023-08-09T13:48:28.579 INFO:teuthology.orchestra.run.smithi043.stdout: "ceph_version": "ceph version 18.0.0-5389-g668a86f2 (668a86f232c9454b2ce5928d1432a29a72964d85) reef (dev)", 2023-08-09T13:48:28.579 INFO:teuthology.orchestra.run.smithi043.stdout: "entity_id": "0", 2023-08-09T13:48:28.579 INFO:teuthology.orchestra.run.smithi043.stdout: "hostname": "smithi043", 2023-08-09T13:48:28.579 INFO:teuthology.orchestra.run.smithi043.stdout: "mount_point": "/home/ubuntu/cephtest/mnt.0", 2023-08-09T13:48:28.580 INFO:teuthology.orchestra.run.smithi043.stdout: "pid": "113478", 2023-08-09T13:48:28.580 INFO:teuthology.orchestra.run.smithi043.stdout: "root": "/" 2023-08-09T13:48:28.580 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2023-08-09T13:48:28.580 INFO:teuthology.orchestra.run.smithi043.stdout: "dentry_count": 0, 2023-08-09T13:48:28.580 INFO:teuthology.orchestra.run.smithi043.stdout: "dentry_pinned_count": 0, 2023-08-09T13:48:28.580 INFO:teuthology.orchestra.run.smithi043.stdout: "id": 6633, 2023-08-09T13:48:28.580 INFO:teuthology.orchestra.run.smithi043.stdout: "inst": { 2023-08-09T13:48:28.581 INFO:teuthology.orchestra.run.smithi043.stdout: "name": { 2023-08-09T13:48:28.581 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "client", 2023-08-09T13:48:28.581 INFO:teuthology.orchestra.run.smithi043.stdout: "num": 6633 2023-08-09T13:48:28.581 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2023-08-09T13:48:28.581 INFO:teuthology.orchestra.run.smithi043.stdout: "addr": { 2023-08-09T13:48:28.581 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "v1", 2023-08-09T13:48:28.581 INFO:teuthology.orchestra.run.smithi043.stdout: "addr": "192.168.0.1:0", 2023-08-09T13:48:28.581 INFO:teuthology.orchestra.run.smithi043.stdout: "nonce": 3131784456 2023-08-09T13:48:28.582 INFO:teuthology.orchestra.run.smithi043.stdout: } 2023-08-09T13:48:28.582 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2023-08-09T13:48:28.582 INFO:teuthology.orchestra.run.smithi043.stdout: "addr": { 2023-08-09T13:48:28.582 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "v1", 2023-08-09T13:48:28.582 INFO:teuthology.orchestra.run.smithi043.stdout: "addr": "192.168.0.1:0", 2023-08-09T13:48:28.583 INFO:teuthology.orchestra.run.smithi043.stdout: "nonce": 3131784456 2023-08-09T13:48:28.583 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2023-08-09T13:48:28.583 INFO:teuthology.orchestra.run.smithi043.stdout: "inst_str": "client.6633 192.168.0.1:0/3131784456", 2023-08-09T13:48:28.583 INFO:teuthology.orchestra.run.smithi043.stdout: "addr_str": "192.168.0.1:0/3131784456", 2023-08-09T13:48:28.583 INFO:teuthology.orchestra.run.smithi043.stdout: "inode_count": 1, 2023-08-09T13:48:28.583 INFO:teuthology.orchestra.run.smithi043.stdout: "mds_epoch": 26, 2023-08-09T13:48:28.583 INFO:teuthology.orchestra.run.smithi043.stdout: "osd_epoch": 74, 2023-08-09T13:48:28.584 INFO:teuthology.orchestra.run.smithi043.stdout: "osd_epoch_barrier": 0, 2023-08-09T13:48:28.584 INFO:teuthology.orchestra.run.smithi043.stdout: "blocklisted": false, 2023-08-09T13:48:28.584 INFO:teuthology.orchestra.run.smithi043.stdout: "fs_name": "cephfs" 2023-08-09T13:48:28.584 INFO:teuthology.orchestra.run.smithi043.stdout:} 2023-08-09T13:48:28.594 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-09T13:48:28.594 DEBUG:teuthology.orchestra.run.smithi043:> stat --file-system '--printf=%T 2023-08-09T13:48:28.595 DEBUG:teuthology.orchestra.run.smithi043:> ' -- /home/ubuntu/cephtest/mnt.0 2023-08-09T13:48:28.654 INFO:teuthology.orchestra.run.smithi043.stdout:fuseblk 2023-08-09T13:48:28.655 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.0 2023-08-09T13:48:28.655 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-09T13:48:28.656 DEBUG:teuthology.orchestra.run.smithi043:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2023-08-09T13:48:28.766 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs volume ls 2023-08-09T13:48:29.000 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:28.997+0000 7f8668938640 1 Processor -- start 2023-08-09T13:48:29.000 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:28.998+0000 7f8668938640 1 -- start start 2023-08-09T13:48:29.002 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:28.999+0000 7f8668938640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f866415f7b0 0x7f86640b4760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:48:29.002 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:28.999+0000 7f8668938640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f86640b1050 0x7f86640ccfa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:48:29.002 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:28.999+0000 7f8668938640 1 -- --> v1:172.21.15.203:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f86640cd930 con 0x7f86640cd4e0 2023-08-09T13:48:29.002 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:28.999+0000 7f8668938640 1 -- --> v2:172.21.15.43:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f86640b7bf0 con 0x7f866415f7b0 2023-08-09T13:48:29.003 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:28.999+0000 7f8668938640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f86640b7ed0 con 0x7f86640b1050 2023-08-09T13:48:29.006 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:29.000+0000 7f8662d9d640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f866415f7b0 0x7f86640b4760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:48:29.006 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:29.000+0000 7f8662d9d640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f866415f7b0 0x7f86640b4760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:33924/0 (socket says 172.21.15.43:33924) 2023-08-09T13:48:29.006 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:29.000+0000 7f8662d9d640 1 -- 172.21.15.43:0/3542178261 learned_addr learned my addr 172.21.15.43:0/3542178261 (peer_addr_for_me v2:172.21.15.43:0/0) 2023-08-09T13:48:29.006 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:29.000+0000 7f866259c640 1 --2- 172.21.15.43:0/3542178261 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f86640b1050 0x7f86640ccfa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:48:29.006 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:29.000+0000 7f8662d9d640 1 -- 172.21.15.43:0/3542178261 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f86640b1050 msgr2=0x7f86640ccfa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-09T13:48:29.007 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:29.000+0000 7f8662d9d640 1 --2- 172.21.15.43:0/3542178261 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f86640b1050 0x7f86640ccfa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:48:29.007 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:29.000+0000 7f8662d9d640 1 -- 172.21.15.43:0/3542178261 >> v1:172.21.15.203:6789/0 conn(0x7f86640cd4e0 legacy=0x7f86640ceae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-09T13:48:29.007 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:29.000+0000 7f8662d9d640 1 -- 172.21.15.43:0/3542178261 --> v2:172.21.15.43:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f86640d0370 con 0x7f866415f7b0 2023-08-09T13:48:29.007 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:29.002+0000 7f8662d9d640 1 --2- 172.21.15.43:0/3542178261 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f866415f7b0 0x7f86640b4760 secure :-1 s=READY pgs=874 cs=0 l=1 rev1=1 crypto rx=0x7f8654009870 tx=0x7f865402b8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fe7bc1e2e4010bca server_cookie=0 in_seq=0 out_seq=0 2023-08-09T13:48:29.007 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:29.002+0000 7f8661d9b640 1 -- 172.21.15.43:0/3542178261 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8654031640 con 0x7f866415f7b0 2023-08-09T13:48:29.007 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:29.002+0000 7f8661d9b640 1 -- 172.21.15.43:0/3542178261 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f86540317a0 con 0x7f866415f7b0 2023-08-09T13:48:29.007 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:29.002+0000 7f8661d9b640 1 -- 172.21.15.43:0/3542178261 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8654031b30 con 0x7f866415f7b0 2023-08-09T13:48:29.008 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:29.002+0000 7f8668938640 1 -- 172.21.15.43:0/3542178261 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f866415f7b0 msgr2=0x7f86640b4760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:48:29.008 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:29.002+0000 7f8668938640 1 --2- 172.21.15.43:0/3542178261 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f866415f7b0 0x7f86640b4760 secure :-1 s=READY pgs=874 cs=0 l=1 rev1=1 crypto rx=0x7f8654009870 tx=0x7f865402b8f0 comp rx=0 tx=0).stop 2023-08-09T13:48:29.008 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:29.003+0000 7f8668938640 1 -- 172.21.15.43:0/3542178261 shutdown_connections 2023-08-09T13:48:29.008 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:29.003+0000 7f8668938640 1 --2- 172.21.15.43:0/3542178261 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f86640b1050 0x7f86640ccfa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:48:29.008 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:29.003+0000 7f8668938640 1 --2- 172.21.15.43:0/3542178261 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f866415f7b0 0x7f86640b4760 unknown :-1 s=CLOSED pgs=874 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:48:29.008 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:29.003+0000 7f8668938640 1 -- 172.21.15.43:0/3542178261 >> 172.21.15.43:0/3542178261 conn(0x7f866406a2b0 msgr2=0x7f866406a690 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-09T13:48:29.008 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:29.003+0000 7f8668938640 1 -- 172.21.15.43:0/3542178261 shutdown_connections 2023-08-09T13:48:29.009 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:29.003+0000 7f8668938640 1 -- 172.21.15.43:0/3542178261 wait complete. 2023-08-09T13:48:29.010 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:29.005+0000 7f8668938640 1 Processor -- start 2023-08-09T13:48:29.010 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:29.005+0000 7f8668938640 1 -- start start 2023-08-09T13:48:29.011 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:29.006+0000 7f8668938640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f86640b1050 0x7f8664149750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:48:29.011 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:29.006+0000 7f8668938640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f86640cd4e0 0x7f86640b46d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:48:29.011 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:29.006+0000 7f8668938640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f866415f7b0 0x7f86640cb970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:48:29.011 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:29.006+0000 7f8668938640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f86640cced0 con 0x7f866415f7b0 2023-08-09T13:48:29.011 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:29.006+0000 7f8668938640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f86640cd2f0 con 0x7f86640cd4e0 2023-08-09T13:48:29.011 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:29.006+0000 7f8668938640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f86640cc250 con 0x7f86640b1050 2023-08-09T13:48:29.011 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:29.006+0000 7f866359e640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f866415f7b0 0x7f86640cb970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:48:29.011 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:29.006+0000 7f866359e640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f866415f7b0 0x7f86640cb970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:33928/0 (socket says 172.21.15.43:33928) 2023-08-09T13:48:29.012 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:29.006+0000 7f866359e640 1 -- 172.21.15.43:0/3953662709 learned_addr learned my addr 172.21.15.43:0/3953662709 (peer_addr_for_me v2:172.21.15.43:0/0) 2023-08-09T13:48:29.012 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:29.006+0000 7f8662d9d640 1 --2- 172.21.15.43:0/3953662709 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f86640b1050 0x7f8664149750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:48:29.012 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:29.006+0000 7f866259c640 1 --2- 172.21.15.43:0/3953662709 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f86640cd4e0 0x7f86640b46d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:48:29.012 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:29.006+0000 7f866359e640 1 -- 172.21.15.43:0/3953662709 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f86640b1050 msgr2=0x7f8664149750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:48:29.012 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:29.006+0000 7f866359e640 1 --2- 172.21.15.43:0/3953662709 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f86640b1050 0x7f8664149750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:48:29.012 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:29.006+0000 7f866359e640 1 -- 172.21.15.43:0/3953662709 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f86640cd4e0 msgr2=0x7f86640b46d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:48:29.012 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:29.006+0000 7f866359e640 1 --2- 172.21.15.43:0/3953662709 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f86640cd4e0 0x7f86640b46d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:48:29.013 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:29.006+0000 7f866359e640 1 -- 172.21.15.43:0/3953662709 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f86540094d0 con 0x7f866415f7b0 2023-08-09T13:48:29.013 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:29.007+0000 7f866359e640 1 --2- 172.21.15.43:0/3953662709 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f866415f7b0 0x7f86640cb970 secure :-1 s=READY pgs=875 cs=0 l=1 rev1=1 crypto rx=0x7f865c00c780 tx=0x7f865c00cc40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-09T13:48:29.013 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:29.007+0000 7f864b7fe640 1 -- 172.21.15.43:0/3953662709 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f865c004b60 con 0x7f866415f7b0 2023-08-09T13:48:29.013 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:29.007+0000 7f864b7fe640 1 -- 172.21.15.43:0/3953662709 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f865c00ce90 con 0x7f866415f7b0 2023-08-09T13:48:29.013 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:29.007+0000 7f864b7fe640 1 -- 172.21.15.43:0/3953662709 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f865c00a850 con 0x7f866415f7b0 2023-08-09T13:48:29.013 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:29.007+0000 7f8668938640 1 -- 172.21.15.43:0/3953662709 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f86640cc530 con 0x7f866415f7b0 2023-08-09T13:48:29.013 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:29.007+0000 7f8668938640 1 -- 172.21.15.43:0/3953662709 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f86640ccaf0 con 0x7f866415f7b0 2023-08-09T13:48:29.014 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:29.008+0000 7f8668938640 1 -- 172.21.15.43:0/3953662709 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f86640cd010 con 0x7f866415f7b0 2023-08-09T13:48:29.014 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:29.010+0000 7f864b7fe640 1 -- 172.21.15.43:0/3953662709 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 18) v1 ==== 88443+0+0 (secure 0 0 0) 0x7f865c00a9b0 con 0x7f866415f7b0 2023-08-09T13:48:29.015 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:29.011+0000 7f864b7fe640 1 --2- 172.21.15.43:0/3953662709 >> [v2:172.21.15.203:6832/272917756,v1:172.21.15.203:6833/272917756] conn(0x7f865006a170 0x7f865006c630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:48:29.015 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:29.011+0000 7f864b7fe640 1 -- 172.21.15.43:0/3953662709 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(74..74 src has 1..74) v4 ==== 6426+0+0 (secure 0 0 0) 0x7f865c088a40 con 0x7f866415f7b0 2023-08-09T13:48:29.015 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:29.011+0000 7f8662d9d640 1 --2- 172.21.15.43:0/3953662709 >> [v2:172.21.15.203:6832/272917756,v1:172.21.15.203:6833/272917756] conn(0x7f865006a170 0x7f865006c630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:48:29.015 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:29.013+0000 7f8662d9d640 1 --2- 172.21.15.43:0/3953662709 >> [v2:172.21.15.203:6832/272917756,v1:172.21.15.203:6833/272917756] conn(0x7f865006a170 0x7f865006c630 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f8654036000 tx=0x7f865403c040 comp rx=0 tx=0).ready entity=mgr.5533 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-09T13:48:29.017 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:29.015+0000 7f864b7fe640 1 -- 172.21.15.43:0/3953662709 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178066 (secure 0 0 0) 0x7f86640cd010 con 0x7f866415f7b0 2023-08-09T13:48:29.238 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:29.236+0000 7f8668938640 1 -- 172.21.15.43:0/3953662709 --> [v2:172.21.15.203:6832/272917756,v1:172.21.15.203:6833/272917756] -- mgr_command(tid 0: {"prefix": "fs volume ls", "target": ["mon-mgr", ""]}) v1 -- 0x7f86640d0930 con 0x7f865006a170 2023-08-09T13:48:29.242 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:29.240+0000 7f864b7fe640 1 -- 172.21.15.43:0/3953662709 <== mgr.5533 v2:172.21.15.203:6832/272917756 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+41 (secure 0 0 0) 0x7f8654031990 con 0x7f865006a170 2023-08-09T13:48:29.243 INFO:teuthology.orchestra.run.smithi043.stdout:[ 2023-08-09T13:48:29.243 INFO:teuthology.orchestra.run.smithi043.stdout: { 2023-08-09T13:48:29.243 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "cephfs" 2023-08-09T13:48:29.243 INFO:teuthology.orchestra.run.smithi043.stdout: } 2023-08-09T13:48:29.243 INFO:teuthology.orchestra.run.smithi043.stdout:] 2023-08-09T13:48:29.245 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:29.242+0000 7f8668938640 1 -- 172.21.15.43:0/3953662709 >> [v2:172.21.15.203:6832/272917756,v1:172.21.15.203:6833/272917756] conn(0x7f865006a170 msgr2=0x7f865006c630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:48:29.245 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:29.243+0000 7f8668938640 1 --2- 172.21.15.43:0/3953662709 >> [v2:172.21.15.203:6832/272917756,v1:172.21.15.203:6833/272917756] conn(0x7f865006a170 0x7f865006c630 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f8654036000 tx=0x7f865403c040 comp rx=0 tx=0).stop 2023-08-09T13:48:29.246 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:29.243+0000 7f8668938640 1 -- 172.21.15.43:0/3953662709 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f866415f7b0 msgr2=0x7f86640cb970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:48:29.246 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:29.243+0000 7f8668938640 1 --2- 172.21.15.43:0/3953662709 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f866415f7b0 0x7f86640cb970 secure :-1 s=READY pgs=875 cs=0 l=1 rev1=1 crypto rx=0x7f865c00c780 tx=0x7f865c00cc40 comp rx=0 tx=0).stop 2023-08-09T13:48:29.246 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:29.243+0000 7f8668938640 1 -- 172.21.15.43:0/3953662709 shutdown_connections 2023-08-09T13:48:29.246 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:29.243+0000 7f8668938640 1 --2- 172.21.15.43:0/3953662709 >> [v2:172.21.15.203:6832/272917756,v1:172.21.15.203:6833/272917756] conn(0x7f865006a170 0x7f865006c630 unknown :-1 s=CLOSED pgs=71 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:48:29.246 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:29.243+0000 7f8668938640 1 --2- 172.21.15.43:0/3953662709 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f866415f7b0 0x7f86640cb970 unknown :-1 s=CLOSED pgs=875 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:48:29.246 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:29.243+0000 7f8668938640 1 --2- 172.21.15.43:0/3953662709 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f86640cd4e0 0x7f86640b46d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:48:29.247 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:29.243+0000 7f8668938640 1 --2- 172.21.15.43:0/3953662709 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f86640b1050 0x7f8664149750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:48:29.247 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:29.243+0000 7f8668938640 1 -- 172.21.15.43:0/3953662709 >> 172.21.15.43:0/3953662709 conn(0x7f866406a2b0 msgr2=0x7f86640d1a90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-09T13:48:29.247 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:29.243+0000 7f8668938640 1 -- 172.21.15.43:0/3953662709 shutdown_connections 2023-08-09T13:48:29.247 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:29.243+0000 7f8668938640 1 -- 172.21.15.43:0/3953662709 wait complete. 2023-08-09T13:48:29.266 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph mgr module enable snap_schedule 2023-08-09T13:48:29.495 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:29.492+0000 7f0355608640 1 Processor -- start 2023-08-09T13:48:29.495 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:29.492+0000 7f0355608640 1 -- start start 2023-08-09T13:48:29.497 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:29.494+0000 7f0355608640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f03500b0f00 0x7f03500ccef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:48:29.497 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:29.494+0000 7f0355608640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f03500b1450 0x7f03500cd430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:48:29.497 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:29.494+0000 7f0355608640 1 -- --> v1:172.21.15.203:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f03500b7c00 con 0x7f035015c3a0 2023-08-09T13:48:29.497 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:29.494+0000 7f0355608640 1 -- --> v2:172.21.15.43:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f03500b81c0 con 0x7f03500b0f00 2023-08-09T13:48:29.497 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:29.494+0000 7f0355608640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f03500b7ee0 con 0x7f03500b1450 2023-08-09T13:48:29.497 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:29.494+0000 7f034f7fe640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f03500b0f00 0x7f03500ccef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:48:29.498 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:29.495+0000 7f034f7fe640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f03500b0f00 0x7f03500ccef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:33932/0 (socket says 172.21.15.43:33932) 2023-08-09T13:48:29.498 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:29.495+0000 7f034f7fe640 1 -- 172.21.15.43:0/4176289726 learned_addr learned my addr 172.21.15.43:0/4176289726 (peer_addr_for_me v2:172.21.15.43:0/0) 2023-08-09T13:48:29.498 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:29.495+0000 7f0354e07640 1 --2- 172.21.15.43:0/4176289726 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f03500b1450 0x7f03500cd430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:48:29.499 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:29.495+0000 7f034f7fe640 1 -- 172.21.15.43:0/4176289726 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f03500b1450 msgr2=0x7f03500cd430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-09T13:48:29.499 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:29.495+0000 7f034f7fe640 1 --2- 172.21.15.43:0/4176289726 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f03500b1450 0x7f03500cd430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:48:29.499 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:29.495+0000 7f034f7fe640 1 -- 172.21.15.43:0/4176289726 >> v1:172.21.15.203:6789/0 conn(0x7f035015c3a0 legacy=0x7f03501608a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-09T13:48:29.499 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:29.495+0000 7f034f7fe640 1 -- 172.21.15.43:0/4176289726 --> v2:172.21.15.43:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f03501600c0 con 0x7f03500b0f00 2023-08-09T13:48:29.499 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:29.496+0000 7f034f7fe640 1 --2- 172.21.15.43:0/4176289726 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f03500b0f00 0x7f03500ccef0 secure :-1 s=READY pgs=876 cs=0 l=1 rev1=1 crypto rx=0x7f035008b610 tx=0x7f034402b820 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7d58c0b72647645d server_cookie=0 in_seq=0 out_seq=0 2023-08-09T13:48:29.500 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:29.497+0000 7f034effd640 1 -- 172.21.15.43:0/4176289726 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0344004230 con 0x7f03500b0f00 2023-08-09T13:48:29.500 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:29.497+0000 7f034effd640 1 -- 172.21.15.43:0/4176289726 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0344004390 con 0x7f03500b0f00 2023-08-09T13:48:29.501 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:29.497+0000 7f034effd640 1 -- 172.21.15.43:0/4176289726 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0344003810 con 0x7f03500b0f00 2023-08-09T13:48:29.501 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:29.497+0000 7f0355608640 1 -- 172.21.15.43:0/4176289726 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f03500b0f00 msgr2=0x7f03500ccef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:48:29.501 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:29.497+0000 7f0355608640 1 --2- 172.21.15.43:0/4176289726 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f03500b0f00 0x7f03500ccef0 secure :-1 s=READY pgs=876 cs=0 l=1 rev1=1 crypto rx=0x7f035008b610 tx=0x7f034402b820 comp rx=0 tx=0).stop 2023-08-09T13:48:29.501 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:29.497+0000 7f0355608640 1 -- 172.21.15.43:0/4176289726 shutdown_connections 2023-08-09T13:48:29.501 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:29.497+0000 7f0355608640 1 --2- 172.21.15.43:0/4176289726 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f03500b1450 0x7f03500cd430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:48:29.501 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:29.497+0000 7f0355608640 1 --2- 172.21.15.43:0/4176289726 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f03500b0f00 0x7f03500ccef0 unknown :-1 s=CLOSED pgs=876 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:48:29.501 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:29.497+0000 7f0355608640 1 -- 172.21.15.43:0/4176289726 >> 172.21.15.43:0/4176289726 conn(0x7f035006a210 msgr2=0x7f035006a620 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-09T13:48:29.502 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:29.497+0000 7f0355608640 1 -- 172.21.15.43:0/4176289726 shutdown_connections 2023-08-09T13:48:29.502 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:29.497+0000 7f0355608640 1 -- 172.21.15.43:0/4176289726 wait complete. 2023-08-09T13:48:29.503 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:29.499+0000 7f0355608640 1 Processor -- start 2023-08-09T13:48:29.503 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:29.499+0000 7f0355608640 1 -- start start 2023-08-09T13:48:29.503 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:29.500+0000 7f0355608640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f03500b0f00 0x7f03500cbbb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:48:29.503 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:29.500+0000 7f0355608640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f03500b1450 0x7f0350150630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:48:29.503 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:29.500+0000 7f0355608640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f035015c3a0 0x7f03500c7190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:48:29.503 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:29.500+0000 7f0355608640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f03501f2170 con 0x7f035015c3a0 2023-08-09T13:48:29.503 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:29.500+0000 7f0355608640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f03501f5b10 con 0x7f03500b1450 2023-08-09T13:48:29.504 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:29.500+0000 7f0355608640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f03501f5df0 con 0x7f03500b0f00 2023-08-09T13:48:29.505 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:29.501+0000 7f0354e07640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f035015c3a0 0x7f03500c7190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:48:29.505 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:29.501+0000 7f0354e07640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f035015c3a0 0x7f03500c7190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:33948/0 (socket says 172.21.15.43:33948) 2023-08-09T13:48:29.505 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:29.501+0000 7f0354e07640 1 -- 172.21.15.43:0/470379616 learned_addr learned my addr 172.21.15.43:0/470379616 (peer_addr_for_me v2:172.21.15.43:0/0) 2023-08-09T13:48:29.505 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:29.501+0000 7f034ffff640 1 --2- 172.21.15.43:0/470379616 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f03500b0f00 0x7f03500cbbb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:48:29.505 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:29.501+0000 7f0354e07640 1 -- 172.21.15.43:0/470379616 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f03500b0f00 msgr2=0x7f03500cbbb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:48:29.505 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:29.501+0000 7f0354e07640 1 --2- 172.21.15.43:0/470379616 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f03500b0f00 0x7f03500cbbb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:48:29.505 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:29.501+0000 7f0354e07640 1 -- 172.21.15.43:0/470379616 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f03500b1450 msgr2=0x7f0350150630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:48:29.506 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:29.501+0000 7f0354e07640 1 --2- 172.21.15.43:0/470379616 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f03500b1450 0x7f0350150630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:48:29.506 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:29.501+0000 7f0354e07640 1 -- 172.21.15.43:0/470379616 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0344009460 con 0x7f035015c3a0 2023-08-09T13:48:29.506 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:29.501+0000 7f0354e07640 1 --2- 172.21.15.43:0/470379616 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f035015c3a0 0x7f03500c7190 secure :-1 s=READY pgs=877 cs=0 l=1 rev1=1 crypto rx=0x7f0348007e60 tx=0x7f034800e4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-09T13:48:29.506 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:29.502+0000 7f034cff9640 1 -- 172.21.15.43:0/470379616 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0348014420 con 0x7f035015c3a0 2023-08-09T13:48:29.506 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:29.502+0000 7f034cff9640 1 -- 172.21.15.43:0/470379616 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f03480145a0 con 0x7f035015c3a0 2023-08-09T13:48:29.506 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:29.502+0000 7f034cff9640 1 -- 172.21.15.43:0/470379616 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0348014950 con 0x7f035015c3a0 2023-08-09T13:48:29.506 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:29.502+0000 7f0355608640 1 -- 172.21.15.43:0/470379616 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f03501f60d0 con 0x7f035015c3a0 2023-08-09T13:48:29.507 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:29.502+0000 7f0355608640 1 -- 172.21.15.43:0/470379616 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f03501f63b0 con 0x7f035015c3a0 2023-08-09T13:48:29.507 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:29.503+0000 7f0355608640 1 -- 172.21.15.43:0/470379616 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f03501f22b0 con 0x7f035015c3a0 2023-08-09T13:48:29.507 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:29.504+0000 7f034cff9640 1 -- 172.21.15.43:0/470379616 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 18) v1 ==== 88443+0+0 (secure 0 0 0) 0x7f0348014ab0 con 0x7f035015c3a0 2023-08-09T13:48:29.508 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:29.505+0000 7f034cff9640 1 --2- 172.21.15.43:0/470379616 >> [v2:172.21.15.203:6832/272917756,v1:172.21.15.203:6833/272917756] conn(0x7f033c06a170 0x7f033c06c630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:48:29.508 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:29.505+0000 7f034cff9640 1 -- 172.21.15.43:0/470379616 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(74..74 src has 1..74) v4 ==== 6426+0+0 (secure 0 0 0) 0x7f03480891d0 con 0x7f035015c3a0 2023-08-09T13:48:29.508 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:29.506+0000 7f034ffff640 1 --2- 172.21.15.43:0/470379616 >> [v2:172.21.15.203:6832/272917756,v1:172.21.15.203:6833/272917756] conn(0x7f033c06a170 0x7f033c06c630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:48:29.509 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:29.507+0000 7f034ffff640 1 --2- 172.21.15.43:0/470379616 >> [v2:172.21.15.203:6832/272917756,v1:172.21.15.203:6833/272917756] conn(0x7f033c06a170 0x7f033c06c630 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f0340006fd0 tx=0x7f0340008040 comp rx=0 tx=0).ready entity=mgr.5533 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-09T13:48:29.512 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:29.510+0000 7f034cff9640 1 -- 172.21.15.43:0/470379616 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178066 (secure 0 0 0) 0x7f03501f22b0 con 0x7f035015c3a0 2023-08-09T13:48:29.755 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:29.750+0000 7f0355608640 1 -- 172.21.15.43:0/470379616 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "mgr module enable", "module": "snap_schedule"} v 0) v1 -- 0x7f03501f22b0 con 0x7f035015c3a0 2023-08-09T13:48:30.068 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:30.065+0000 7f034cff9640 1 -- 172.21.15.43:0/470379616 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mgrmap(e 19) v1 ==== 88460+0+0 (secure 0 0 0) 0x7f0348087420 con 0x7f035015c3a0 2023-08-09T13:48:30.070 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:30.067+0000 7f034cff9640 1 -- 172.21.15.43:0/470379616 <== mon.0 v2:172.21.15.43:3300/0 8 ==== mon_command_ack([{"prefix": "mgr module enable", "module": "snap_schedule"}]=0 v19) v1 ==== 92+0+0 (secure 0 0 0) 0x7f0348057e40 con 0x7f035015c3a0 2023-08-09T13:48:30.073 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:30.070+0000 7f0355608640 1 -- 172.21.15.43:0/470379616 >> [v2:172.21.15.203:6832/272917756,v1:172.21.15.203:6833/272917756] conn(0x7f033c06a170 msgr2=0x7f033c06c630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:48:30.073 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:30.070+0000 7f0355608640 1 --2- 172.21.15.43:0/470379616 >> [v2:172.21.15.203:6832/272917756,v1:172.21.15.203:6833/272917756] conn(0x7f033c06a170 0x7f033c06c630 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f0340006fd0 tx=0x7f0340008040 comp rx=0 tx=0).stop 2023-08-09T13:48:30.073 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:30.070+0000 7f0355608640 1 -- 172.21.15.43:0/470379616 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f035015c3a0 msgr2=0x7f03500c7190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:48:30.073 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:30.070+0000 7f0355608640 1 --2- 172.21.15.43:0/470379616 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f035015c3a0 0x7f03500c7190 secure :-1 s=READY pgs=877 cs=0 l=1 rev1=1 crypto rx=0x7f0348007e60 tx=0x7f034800e4b0 comp rx=0 tx=0).stop 2023-08-09T13:48:30.074 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:30.071+0000 7f0355608640 1 -- 172.21.15.43:0/470379616 shutdown_connections 2023-08-09T13:48:30.074 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:30.071+0000 7f0355608640 1 --2- 172.21.15.43:0/470379616 >> [v2:172.21.15.203:6832/272917756,v1:172.21.15.203:6833/272917756] conn(0x7f033c06a170 0x7f033c06c630 unknown :-1 s=CLOSED pgs=72 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:48:30.074 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:30.071+0000 7f0355608640 1 --2- 172.21.15.43:0/470379616 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f035015c3a0 0x7f03500c7190 unknown :-1 s=CLOSED pgs=877 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:48:30.074 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:30.071+0000 7f0355608640 1 --2- 172.21.15.43:0/470379616 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f03500b1450 0x7f0350150630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:48:30.075 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:30.071+0000 7f0355608640 1 --2- 172.21.15.43:0/470379616 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f03500b0f00 0x7f03500cbbb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:48:30.075 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:30.071+0000 7f0355608640 1 -- 172.21.15.43:0/470379616 >> 172.21.15.43:0/470379616 conn(0x7f035006a210 msgr2=0x7f03500d2140 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-09T13:48:30.075 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:30.071+0000 7f0355608640 1 -- 172.21.15.43:0/470379616 shutdown_connections 2023-08-09T13:48:30.075 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:30.071+0000 7f0355608640 1 -- 172.21.15.43:0/470379616 wait complete. 2023-08-09T13:48:30.094 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mgr mgr/snap_schedule/allow_m_granularity True 2023-08-09T13:48:30.281 INFO:tasks.ceph.mgr.x.smithi203.stderr:2023-08-09T13:48:30.278+0000 7f4d27e32040 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2023-08-09T13:48:30.310 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:30.308+0000 7fcdfde1f640 1 Processor -- start 2023-08-09T13:48:30.310 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:30.308+0000 7fcdfde1f640 1 -- start start 2023-08-09T13:48:30.312 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:30.309+0000 7fcdfde1f640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7fcdf815c550 0x7fcdf80cd6d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:48:30.312 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:30.309+0000 7fcdfde1f640 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7fcdf80b45f0 0x7fcdf80b59d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:48:30.312 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:30.309+0000 7fcdfde1f640 1 -- --> v1:172.21.15.203:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fcdf80b7bd0 con 0x7fcdf80d04b0 2023-08-09T13:48:30.312 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:30.309+0000 7fcdfde1f640 1 -- --> v2:172.21.15.43:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fcdf80d1c00 con 0x7fcdf815c550 2023-08-09T13:48:30.313 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:30.309+0000 7fcdfde1f640 1 -- --> v2:172.21.15.203:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fcdf815c930 con 0x7fcdf80b45f0 2023-08-09T13:48:30.313 INFO:tasks.ceph.mgr.y.smithi043.stderr:2023-08-09T13:48:30.309+0000 7f4a4ae0d040 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2023-08-09T13:48:30.314 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:30.310+0000 7fcdf77fe640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7fcdf815c550 0x7fcdf80cd6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:48:30.314 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:30.310+0000 7fcdf77fe640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7fcdf815c550 0x7fcdf80cd6d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:33962/0 (socket says 172.21.15.43:33962) 2023-08-09T13:48:30.314 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:30.310+0000 7fcdf77fe640 1 -- 172.21.15.43:0/3440497415 learned_addr learned my addr 172.21.15.43:0/3440497415 (peer_addr_for_me v2:172.21.15.43:0/0) 2023-08-09T13:48:30.314 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:30.310+0000 7fcdf7fff640 1 --2- 172.21.15.43:0/3440497415 >> v2:172.21.15.203:3300/0 conn(0x7fcdf80b45f0 0x7fcdf80b59d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:48:30.315 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:30.310+0000 7fcdf77fe640 1 -- 172.21.15.43:0/3440497415 >> v2:172.21.15.203:3300/0 conn(0x7fcdf80b45f0 msgr2=0x7fcdf80b59d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-09T13:48:30.315 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:30.310+0000 7fcdf77fe640 1 --2- 172.21.15.43:0/3440497415 >> v2:172.21.15.203:3300/0 conn(0x7fcdf80b45f0 0x7fcdf80b59d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:48:30.315 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:30.310+0000 7fcdf77fe640 1 -- 172.21.15.43:0/3440497415 >> v1:172.21.15.203:6789/0 conn(0x7fcdf80d04b0 legacy=0x7fcdf80b3ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-09T13:48:30.315 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:30.310+0000 7fcdf77fe640 1 -- 172.21.15.43:0/3440497415 --> v2:172.21.15.43:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcdf80b54f0 con 0x7fcdf815c550 2023-08-09T13:48:30.315 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:30.311+0000 7fcdf77fe640 1 --2- 172.21.15.43:0/3440497415 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fcdf815c550 0x7fcdf80cd6d0 secure :-1 s=READY pgs=878 cs=0 l=1 rev1=1 crypto rx=0x7fcdf80d01c0 tx=0x7fcde802b860 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2b3b56fcdcab4bc7 server_cookie=0 in_seq=0 out_seq=0 2023-08-09T13:48:30.315 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:30.312+0000 7fcdf67fc640 1 -- 172.21.15.43:0/3440497415 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcde80317c0 con 0x7fcdf815c550 2023-08-09T13:48:30.315 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:30.312+0000 7fcdf67fc640 1 -- 172.21.15.43:0/3440497415 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcde8031920 con 0x7fcdf815c550 2023-08-09T13:48:30.316 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:30.312+0000 7fcdf67fc640 1 -- 172.21.15.43:0/3440497415 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcde8031cb0 con 0x7fcdf815c550 2023-08-09T13:48:30.316 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:30.312+0000 7fcdfde1f640 1 -- 172.21.15.43:0/3440497415 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fcdf815c550 msgr2=0x7fcdf80cd6d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:48:30.316 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:30.312+0000 7fcdfde1f640 1 --2- 172.21.15.43:0/3440497415 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fcdf815c550 0x7fcdf80cd6d0 secure :-1 s=READY pgs=878 cs=0 l=1 rev1=1 crypto rx=0x7fcdf80d01c0 tx=0x7fcde802b860 comp rx=0 tx=0).stop 2023-08-09T13:48:30.317 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:30.313+0000 7fcdfde1f640 1 -- 172.21.15.43:0/3440497415 shutdown_connections 2023-08-09T13:48:30.317 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:30.313+0000 7fcdfde1f640 1 --2- 172.21.15.43:0/3440497415 >> v2:172.21.15.203:3300/0 conn(0x7fcdf80b45f0 0x7fcdf80b59d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:48:30.317 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:30.313+0000 7fcdfde1f640 1 --2- 172.21.15.43:0/3440497415 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fcdf815c550 0x7fcdf80cd6d0 unknown :-1 s=CLOSED pgs=878 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:48:30.317 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:30.313+0000 7fcdfde1f640 1 -- 172.21.15.43:0/3440497415 >> 172.21.15.43:0/3440497415 conn(0x7fcdf8071520 msgr2=0x7fcdf80ceee0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-09T13:48:30.317 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:30.313+0000 7fcdfde1f640 1 -- 172.21.15.43:0/3440497415 shutdown_connections 2023-08-09T13:48:30.317 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:30.313+0000 7fcdfde1f640 1 -- 172.21.15.43:0/3440497415 wait complete. 2023-08-09T13:48:30.318 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:30.314+0000 7fcdfde1f640 1 Processor -- start 2023-08-09T13:48:30.318 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:30.315+0000 7fcdfde1f640 1 -- start start 2023-08-09T13:48:30.318 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:30.316+0000 7fcdfde1f640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fcdf80b45f0 0x7fcdf80b3d70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:48:30.319 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:30.316+0000 7fcdfde1f640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fcdf80d04b0 0x7fcdf80ab9b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:48:30.319 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:30.316+0000 7fcdfde1f640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fcdf815c550 0x7fcdf80abef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:48:30.319 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:30.316+0000 7fcdfde1f640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcdf814b720 con 0x7fcdf815c550 2023-08-09T13:48:30.319 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:30.316+0000 7fcdfde1f640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcdf814c0c0 con 0x7fcdf80d04b0 2023-08-09T13:48:30.319 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:30.316+0000 7fcdfde1f640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcdf80c7e10 con 0x7fcdf80b45f0 2023-08-09T13:48:30.319 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:30.316+0000 7fcdf7fff640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fcdf815c550 0x7fcdf80abef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:48:30.319 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:30.316+0000 7fcdf7fff640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fcdf815c550 0x7fcdf80abef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:33964/0 (socket says 172.21.15.43:33964) 2023-08-09T13:48:30.320 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:30.316+0000 7fcdf7fff640 1 -- 172.21.15.43:0/3088455397 learned_addr learned my addr 172.21.15.43:0/3088455397 (peer_addr_for_me v2:172.21.15.43:0/0) 2023-08-09T13:48:30.321 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:30.316+0000 7fcdf6ffd640 1 --2- 172.21.15.43:0/3088455397 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fcdf80d04b0 0x7fcdf80ab9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:48:30.321 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:30.316+0000 7fcdf77fe640 1 --2- 172.21.15.43:0/3088455397 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fcdf80b45f0 0x7fcdf80b3d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:48:30.321 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:30.317+0000 7fcdf7fff640 1 -- 172.21.15.43:0/3088455397 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fcdf80b45f0 msgr2=0x7fcdf80b3d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:48:30.321 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:30.317+0000 7fcdf7fff640 1 --2- 172.21.15.43:0/3088455397 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fcdf80b45f0 0x7fcdf80b3d70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:48:30.321 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:30.317+0000 7fcdf7fff640 1 -- 172.21.15.43:0/3088455397 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fcdf80d04b0 msgr2=0x7fcdf80ab9b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:48:30.322 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:30.317+0000 7fcdf7fff640 1 --2- 172.21.15.43:0/3088455397 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fcdf80d04b0 0x7fcdf80ab9b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:48:30.322 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:30.317+0000 7fcdf7fff640 1 -- 172.21.15.43:0/3088455397 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcde80094d0 con 0x7fcdf815c550 2023-08-09T13:48:30.322 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:30.317+0000 7fcdf7fff640 1 --2- 172.21.15.43:0/3088455397 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fcdf815c550 0x7fcdf80abef0 secure :-1 s=READY pgs=879 cs=0 l=1 rev1=1 crypto rx=0x7fcdf0007e70 tx=0x7fcdf000e4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-09T13:48:30.322 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:30.317+0000 7fcddffff640 1 -- 172.21.15.43:0/3088455397 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcdf0014620 con 0x7fcdf815c550 2023-08-09T13:48:30.322 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:30.317+0000 7fcddffff640 1 -- 172.21.15.43:0/3088455397 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcdf00147a0 con 0x7fcdf815c550 2023-08-09T13:48:30.323 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:30.317+0000 7fcddffff640 1 -- 172.21.15.43:0/3088455397 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcdf0014b50 con 0x7fcdf815c550 2023-08-09T13:48:30.323 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:30.317+0000 7fcdfde1f640 1 -- 172.21.15.43:0/3088455397 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcdf80c7b30 con 0x7fcdf815c550 2023-08-09T13:48:30.324 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:30.317+0000 7fcdfde1f640 1 -- 172.21.15.43:0/3088455397 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcdf814bd50 con 0x7fcdf815c550 2023-08-09T13:48:30.324 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:30.319+0000 7fcdfde1f640 1 -- 172.21.15.43:0/3088455397 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcdf814c250 con 0x7fcdf815c550 2023-08-09T13:48:30.324 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:30.321+0000 7fcddffff640 1 -- 172.21.15.43:0/3088455397 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 19) v1 ==== 88460+0+0 (secure 0 0 0) 0x7fcdf0014cb0 con 0x7fcdf815c550 2023-08-09T13:48:30.325 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:30.322+0000 7fcddffff640 1 --2- 172.21.15.43:0/3088455397 >> [v2:172.21.15.203:6832/272917756,v1:172.21.15.203:6833/272917756] conn(0x7fcde406a1c0 0x7fcde406c680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:48:30.325 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:30.322+0000 7fcddffff640 1 -- 172.21.15.43:0/3088455397 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(74..74 src has 1..74) v4 ==== 6426+0+0 (secure 0 0 0) 0x7fcdf00893e0 con 0x7fcdf815c550 2023-08-09T13:48:30.325 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:30.322+0000 7fcdf77fe640 1 -- 172.21.15.43:0/3088455397 >> [v2:172.21.15.203:6832/272917756,v1:172.21.15.203:6833/272917756] conn(0x7fcde406a1c0 msgr2=0x7fcde406c680 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.203:6832/272917756 2023-08-09T13:48:30.325 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:30.322+0000 7fcdf77fe640 1 --2- 172.21.15.43:0/3088455397 >> [v2:172.21.15.203:6832/272917756,v1:172.21.15.203:6833/272917756] conn(0x7fcde406a1c0 0x7fcde406c680 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-08-09T13:48:30.329 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:30.327+0000 7fcddffff640 1 -- 172.21.15.43:0/3088455397 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178066 (secure 0 0 0) 0x7fcdf00581b0 con 0x7fcdf815c550 2023-08-09T13:48:30.523 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:30.521+0000 7fcdfde1f640 1 -- 172.21.15.43:0/3088455397 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command([{prefix=config set, name=mgr/snap_schedule/allow_m_granularity}] v 0) v1 -- 0x7fcdf80739f0 con 0x7fcdf815c550 2023-08-09T13:48:30.525 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:30.523+0000 7fcdf77fe640 1 -- 172.21.15.43:0/3088455397 >> [v2:172.21.15.203:6832/272917756,v1:172.21.15.203:6833/272917756] conn(0x7fcde406a1c0 msgr2=0x7fcde406c680 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.203:6832/272917756 2023-08-09T13:48:30.525 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:30.523+0000 7fcdf77fe640 1 --2- 172.21.15.43:0/3088455397 >> [v2:172.21.15.203:6832/272917756,v1:172.21.15.203:6833/272917756] conn(0x7fcde406a1c0 0x7fcde406c680 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2023-08-09T13:48:30.529 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:30.527+0000 7fcddffff640 1 -- 172.21.15.43:0/3088455397 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mgr/snap_schedule/allow_m_granularity}]=0 v17)=0 v17) v1 ==== 138+0+0 (secure 0 0 0) 0x7fcdf0057d40 con 0x7fcdf815c550 2023-08-09T13:48:30.532 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:30.529+0000 7fcdfde1f640 1 -- 172.21.15.43:0/3088455397 >> [v2:172.21.15.203:6832/272917756,v1:172.21.15.203:6833/272917756] conn(0x7fcde406a1c0 msgr2=0x7fcde406c680 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-08-09T13:48:30.533 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:30.530+0000 7fcdfde1f640 1 --2- 172.21.15.43:0/3088455397 >> [v2:172.21.15.203:6832/272917756,v1:172.21.15.203:6833/272917756] conn(0x7fcde406a1c0 0x7fcde406c680 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:48:30.533 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:30.530+0000 7fcdfde1f640 1 -- 172.21.15.43:0/3088455397 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fcdf815c550 msgr2=0x7fcdf80abef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:48:30.533 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:30.530+0000 7fcdfde1f640 1 --2- 172.21.15.43:0/3088455397 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fcdf815c550 0x7fcdf80abef0 secure :-1 s=READY pgs=879 cs=0 l=1 rev1=1 crypto rx=0x7fcdf0007e70 tx=0x7fcdf000e4b0 comp rx=0 tx=0).stop 2023-08-09T13:48:30.533 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:30.530+0000 7fcdfde1f640 1 -- 172.21.15.43:0/3088455397 shutdown_connections 2023-08-09T13:48:30.533 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:30.530+0000 7fcdfde1f640 1 --2- 172.21.15.43:0/3088455397 >> [v2:172.21.15.203:6832/272917756,v1:172.21.15.203:6833/272917756] conn(0x7fcde406a1c0 0x7fcde406c680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:48:30.533 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:30.530+0000 7fcdfde1f640 1 --2- 172.21.15.43:0/3088455397 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fcdf815c550 0x7fcdf80abef0 unknown :-1 s=CLOSED pgs=879 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:48:30.533 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:30.530+0000 7fcdfde1f640 1 --2- 172.21.15.43:0/3088455397 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fcdf80d04b0 0x7fcdf80ab9b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:48:30.534 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:30.530+0000 7fcdfde1f640 1 --2- 172.21.15.43:0/3088455397 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fcdf80b45f0 0x7fcdf80b3d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:48:30.534 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:30.530+0000 7fcdfde1f640 1 -- 172.21.15.43:0/3088455397 >> 172.21.15.43:0/3088455397 conn(0x7fcdf8071520 msgr2=0x7fcdf80d2440 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-09T13:48:30.534 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:30.530+0000 7fcdfde1f640 1 -- 172.21.15.43:0/3088455397 shutdown_connections 2023-08-09T13:48:30.534 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:30.530+0000 7fcdfde1f640 1 -- 172.21.15.43:0/3088455397 wait complete. 2023-08-09T13:48:30.552 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mgr mgr/snap_schedule/dump_on_update True 2023-08-09T13:48:30.778 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:30.776+0000 7fe6ac342640 1 Processor -- start 2023-08-09T13:48:30.778 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:30.776+0000 7fe6ac342640 1 -- start start 2023-08-09T13:48:30.780 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:30.777+0000 7fe6ac342640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7fe6a415c620 0x7fe6a40cceb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:48:30.780 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:30.777+0000 7fe6ac342640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fe6a40cf510 0x7fe6a40d0350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:48:30.780 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:30.778+0000 7fe6ac342640 1 -- --> v1:172.21.15.43:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe6a40cd700 con 0x7fe6a40b1260 2023-08-09T13:48:30.780 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:30.778+0000 7fe6ac342640 1 -- --> v2:172.21.15.43:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe6a40b1b70 con 0x7fe6a415c620 2023-08-09T13:48:30.780 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:30.778+0000 7fe6ac342640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe6a40cd9e0 con 0x7fe6a40cf510 2023-08-09T13:48:30.781 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:30.778+0000 7fe6a98dd640 1 --1- >> v1:172.21.15.43:6789/0 conn(0x7fe6a40b1260 0x7fe6a40cee00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.43:6789/0 says I am v1:172.21.15.43:45588/0 (socket says 172.21.15.43:45588) 2023-08-09T13:48:30.781 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:30.778+0000 7fe6a98dd640 1 -- 172.21.15.43:0/2505531489 learned_addr learned my addr 172.21.15.43:0/2505531489 (peer_addr_for_me v1:172.21.15.43:0/0) 2023-08-09T13:48:30.782 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:30.778+0000 7fe6aa0de640 1 --2- 172.21.15.43:0/2505531489 >> v2:172.21.15.43:3300/0 conn(0x7fe6a415c620 0x7fe6a40cceb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:48:30.782 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:30.778+0000 7fe6aa8df640 1 --2- 172.21.15.43:0/2505531489 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fe6a40cf510 0x7fe6a40d0350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:48:30.782 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:30.778+0000 7fe6aa0de640 1 -- 172.21.15.43:0/2505531489 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fe6a40cf510 msgr2=0x7fe6a40d0350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-09T13:48:30.782 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:30.778+0000 7fe6aa0de640 1 --2- 172.21.15.43:0/2505531489 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fe6a40cf510 0x7fe6a40d0350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:48:30.782 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:30.778+0000 7fe6aa0de640 1 -- 172.21.15.43:0/2505531489 >> v1:172.21.15.43:6789/0 conn(0x7fe6a40b1260 legacy=0x7fe6a40cee00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:48:30.782 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:30.778+0000 7fe6aa0de640 1 -- 172.21.15.43:0/2505531489 --> v2:172.21.15.43:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe6a40cdcc0 con 0x7fe6a415c620 2023-08-09T13:48:30.782 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:30.778+0000 7fe6aa8df640 1 --2- 172.21.15.43:0/2505531489 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fe6a40cf510 0x7fe6a40d0350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-08-09T13:48:30.783 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:30.779+0000 7fe6aa0de640 1 --2- 172.21.15.43:0/2505531489 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fe6a415c620 0x7fe6a40cceb0 secure :-1 s=READY pgs=881 cs=0 l=1 rev1=1 crypto rx=0x7fe6a0009790 tx=0x7fe6a002b780 comp rx=0 tx=0).ready entity=mon.0 client_cookie=aede5a28d46bcb6a server_cookie=0 in_seq=0 out_seq=0 2023-08-09T13:48:30.783 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:30.780+0000 7fe6a90dc640 1 -- 172.21.15.43:0/2505531489 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe6a0030710 con 0x7fe6a415c620 2023-08-09T13:48:30.784 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:30.780+0000 7fe6a90dc640 1 -- 172.21.15.43:0/2505531489 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe6a002e050 con 0x7fe6a415c620 2023-08-09T13:48:30.784 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:30.780+0000 7fe6a90dc640 1 -- 172.21.15.43:0/2505531489 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe6a0030a90 con 0x7fe6a415c620 2023-08-09T13:48:30.784 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:30.780+0000 7fe6ac342640 1 -- 172.21.15.43:0/2505531489 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fe6a415c620 msgr2=0x7fe6a40cceb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:48:30.784 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:30.780+0000 7fe6ac342640 1 --2- 172.21.15.43:0/2505531489 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fe6a415c620 0x7fe6a40cceb0 secure :-1 s=READY pgs=881 cs=0 l=1 rev1=1 crypto rx=0x7fe6a0009790 tx=0x7fe6a002b780 comp rx=0 tx=0).stop 2023-08-09T13:48:30.784 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:30.781+0000 7fe6ac342640 1 -- 172.21.15.43:0/2505531489 shutdown_connections 2023-08-09T13:48:30.784 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:30.781+0000 7fe6ac342640 1 --2- 172.21.15.43:0/2505531489 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fe6a40cf510 0x7fe6a40d0350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:48:30.784 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:30.781+0000 7fe6ac342640 1 --2- 172.21.15.43:0/2505531489 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fe6a415c620 0x7fe6a40cceb0 unknown :-1 s=CLOSED pgs=881 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:48:30.785 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:30.781+0000 7fe6ac342640 1 -- 172.21.15.43:0/2505531489 >> 172.21.15.43:0/2505531489 conn(0x7fe6a409b750 msgr2=0x7fe6a40d2410 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-09T13:48:30.785 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:30.781+0000 7fe6ac342640 1 -- 172.21.15.43:0/2505531489 shutdown_connections 2023-08-09T13:48:30.785 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:30.781+0000 7fe6ac342640 1 -- 172.21.15.43:0/2505531489 wait complete. 2023-08-09T13:48:30.786 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:30.782+0000 7fe6ac342640 1 Processor -- start 2023-08-09T13:48:30.787 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:30.783+0000 7fe6ac342640 1 -- start start 2023-08-09T13:48:30.787 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:30.783+0000 7fe6ac342640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fe6a40b1260 0x7fe6a4155970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:48:30.787 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:30.783+0000 7fe6ac342640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fe6a40cf510 0x7fe6a41f9840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:48:30.787 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:30.783+0000 7fe6ac342640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fe6a415c620 0x7fe6a4158bb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:48:30.787 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:30.784+0000 7fe6ac342640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe6a4156d20 con 0x7fe6a40b1260 2023-08-09T13:48:30.787 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:30.784+0000 7fe6ac342640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe6a4156730 con 0x7fe6a40cf510 2023-08-09T13:48:30.788 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:30.784+0000 7fe6ac342640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe6a41fa300 con 0x7fe6a415c620 2023-08-09T13:48:30.788 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:30.784+0000 7fe6aa0de640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fe6a40b1260 0x7fe6a4155970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:48:30.788 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:30.784+0000 7fe6aa0de640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fe6a40b1260 0x7fe6a4155970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:33984/0 (socket says 172.21.15.43:33984) 2023-08-09T13:48:30.788 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:30.784+0000 7fe6aa0de640 1 -- 172.21.15.43:0/3755584665 learned_addr learned my addr 172.21.15.43:0/3755584665 (peer_addr_for_me v2:172.21.15.43:0/0) 2023-08-09T13:48:30.788 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:30.784+0000 7fe6aa8df640 1 --2- 172.21.15.43:0/3755584665 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fe6a415c620 0x7fe6a4158bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:48:30.788 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:30.784+0000 7fe6a98dd640 1 --2- 172.21.15.43:0/3755584665 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fe6a40cf510 0x7fe6a41f9840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:48:30.789 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:30.784+0000 7fe6aa0de640 1 -- 172.21.15.43:0/3755584665 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fe6a415c620 msgr2=0x7fe6a4158bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:48:30.789 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:30.784+0000 7fe6aa0de640 1 --2- 172.21.15.43:0/3755584665 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fe6a415c620 0x7fe6a4158bb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:48:30.789 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:30.784+0000 7fe6aa0de640 1 -- 172.21.15.43:0/3755584665 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fe6a40cf510 msgr2=0x7fe6a41f9840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:48:30.789 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:30.784+0000 7fe6aa0de640 1 --2- 172.21.15.43:0/3755584665 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fe6a40cf510 0x7fe6a41f9840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:48:30.789 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:30.784+0000 7fe6aa0de640 1 -- 172.21.15.43:0/3755584665 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe6a00093a0 con 0x7fe6a40b1260 2023-08-09T13:48:30.791 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:30.785+0000 7fe6aa0de640 1 --2- 172.21.15.43:0/3755584665 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fe6a40b1260 0x7fe6a4155970 secure :-1 s=READY pgs=882 cs=0 l=1 rev1=1 crypto rx=0x7fe6a00098b0 tx=0x7fe6a00046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-09T13:48:30.791 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:30.785+0000 7fe69affd640 1 -- 172.21.15.43:0/3755584665 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe6a0032a50 con 0x7fe6a40b1260 2023-08-09T13:48:30.792 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:30.785+0000 7fe69affd640 1 -- 172.21.15.43:0/3755584665 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe6a0032bb0 con 0x7fe6a40b1260 2023-08-09T13:48:30.792 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:30.785+0000 7fe69affd640 1 -- 172.21.15.43:0/3755584665 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe6a003b400 con 0x7fe6a40b1260 2023-08-09T13:48:30.792 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:30.785+0000 7fe6ac342640 1 -- 172.21.15.43:0/3755584665 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe6a41fa5e0 con 0x7fe6a40b1260 2023-08-09T13:48:30.793 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:30.785+0000 7fe6ac342640 1 -- 172.21.15.43:0/3755584665 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe6a4155eb0 con 0x7fe6a40b1260 2023-08-09T13:48:30.793 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:30.786+0000 7fe6ac342640 1 -- 172.21.15.43:0/3755584665 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe6a40739f0 con 0x7fe6a40b1260 2023-08-09T13:48:30.793 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:30.787+0000 7fe69affd640 1 -- 172.21.15.43:0/3755584665 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 19) v1 ==== 88460+0+0 (secure 0 0 0) 0x7fe6a0039040 con 0x7fe6a40b1260 2023-08-09T13:48:30.793 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:30.788+0000 7fe69affd640 1 --2- 172.21.15.43:0/3755584665 >> [v2:172.21.15.203:6832/272917756,v1:172.21.15.203:6833/272917756] conn(0x7fe68806a1f0 0x7fe68806c6b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:48:30.794 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:30.788+0000 7fe69affd640 1 -- 172.21.15.43:0/3755584665 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(74..74 src has 1..74) v4 ==== 6426+0+0 (secure 0 0 0) 0x7fe6a00aa3f0 con 0x7fe6a40b1260 2023-08-09T13:48:30.794 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:30.788+0000 7fe6a98dd640 1 -- 172.21.15.43:0/3755584665 >> [v2:172.21.15.203:6832/272917756,v1:172.21.15.203:6833/272917756] conn(0x7fe68806a1f0 msgr2=0x7fe68806c6b0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.203:6832/272917756 2023-08-09T13:48:30.794 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:30.789+0000 7fe6a98dd640 1 --2- 172.21.15.43:0/3755584665 >> [v2:172.21.15.203:6832/272917756,v1:172.21.15.203:6833/272917756] conn(0x7fe68806a1f0 0x7fe68806c6b0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-08-09T13:48:30.795 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:30.792+0000 7fe69affd640 1 -- 172.21.15.43:0/3755584665 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178066 (secure 0 0 0) 0x7fe6a0075740 con 0x7fe6a40b1260 2023-08-09T13:48:30.971 INFO:tasks.ceph.mgr.x.smithi203.stderr:2023-08-09T13:48:30.969+0000 7f4d27e32040 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2023-08-09T13:48:30.988 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:30.985+0000 7fe6ac342640 1 -- 172.21.15.43:0/3755584665 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command([{prefix=config set, name=mgr/snap_schedule/dump_on_update}] v 0) v1 -- 0x7fe6a41fa770 con 0x7fe6a40b1260 2023-08-09T13:48:30.992 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:30.990+0000 7fe6a98dd640 1 -- 172.21.15.43:0/3755584665 >> [v2:172.21.15.203:6832/272917756,v1:172.21.15.203:6833/272917756] conn(0x7fe68806a1f0 msgr2=0x7fe68806c6b0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.203:6832/272917756 2023-08-09T13:48:30.993 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:30.990+0000 7fe6a98dd640 1 --2- 172.21.15.43:0/3755584665 >> [v2:172.21.15.203:6832/272917756,v1:172.21.15.203:6833/272917756] conn(0x7fe68806a1f0 0x7fe68806c6b0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2023-08-09T13:48:30.995 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:30.993+0000 7fe69affd640 1 -- 172.21.15.43:0/3755584665 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mgr/snap_schedule/dump_on_update}]=0 v18)=0 v18) v1 ==== 133+0+0 (secure 0 0 0) 0x7fe6a0078d60 con 0x7fe6a40b1260 2023-08-09T13:48:30.999 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:30.995+0000 7fe6ac342640 1 -- 172.21.15.43:0/3755584665 >> [v2:172.21.15.203:6832/272917756,v1:172.21.15.203:6833/272917756] conn(0x7fe68806a1f0 msgr2=0x7fe68806c6b0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-08-09T13:48:31.000 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:30.995+0000 7fe6ac342640 1 --2- 172.21.15.43:0/3755584665 >> [v2:172.21.15.203:6832/272917756,v1:172.21.15.203:6833/272917756] conn(0x7fe68806a1f0 0x7fe68806c6b0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:48:31.000 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:30.995+0000 7fe6ac342640 1 -- 172.21.15.43:0/3755584665 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fe6a40b1260 msgr2=0x7fe6a4155970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:48:31.001 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:30.995+0000 7fe6ac342640 1 --2- 172.21.15.43:0/3755584665 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fe6a40b1260 0x7fe6a4155970 secure :-1 s=READY pgs=882 cs=0 l=1 rev1=1 crypto rx=0x7fe6a00098b0 tx=0x7fe6a00046c0 comp rx=0 tx=0).stop 2023-08-09T13:48:31.001 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:30.996+0000 7fe6ac342640 1 -- 172.21.15.43:0/3755584665 shutdown_connections 2023-08-09T13:48:31.001 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:30.996+0000 7fe6ac342640 1 --2- 172.21.15.43:0/3755584665 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fe6a415c620 0x7fe6a4158bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:48:31.001 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:30.996+0000 7fe6ac342640 1 --2- 172.21.15.43:0/3755584665 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fe6a40cf510 0x7fe6a41f9840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:48:31.002 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:30.996+0000 7fe6ac342640 1 --2- 172.21.15.43:0/3755584665 >> [v2:172.21.15.203:6832/272917756,v1:172.21.15.203:6833/272917756] conn(0x7fe68806a1f0 0x7fe68806c6b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:48:31.002 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:30.996+0000 7fe6ac342640 1 --2- 172.21.15.43:0/3755584665 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fe6a40b1260 0x7fe6a4155970 unknown :-1 s=CLOSED pgs=882 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:48:31.002 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:30.996+0000 7fe6ac342640 1 -- 172.21.15.43:0/3755584665 >> 172.21.15.43:0/3755584665 conn(0x7fe6a409b750 msgr2=0x7fe6a40b03f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-09T13:48:31.003 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:30.996+0000 7fe6ac342640 1 -- 172.21.15.43:0/3755584665 shutdown_connections 2023-08-09T13:48:31.003 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:30.996+0000 7fe6ac342640 1 -- 172.21.15.43:0/3755584665 wait complete. 2023-08-09T13:48:31.018 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-09T13:48:31.019 DEBUG:teuthology.orchestra.run.smithi043:> (cd /home/ubuntu/cephtest/mnt.0 && exec mkdir -p snap_test_dir1) 2023-08-09T13:48:31.088 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs snap-schedule add --fs cephfs snap_test_dir1 1M 2023-08-09T13:48:31.094 INFO:tasks.ceph.mgr.x.smithi203.stderr:2023-08-09T13:48:31.091+0000 7f4d27e32040 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2023-08-09T13:48:31.096 INFO:tasks.ceph.mgr.y.smithi043.stderr:2023-08-09T13:48:31.090+0000 7f4a4ae0d040 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2023-08-09T13:48:31.149 INFO:tasks.ceph.mgr.x.smithi203.stderr:2023-08-09T13:48:31.147+0000 7f4d27e32040 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2023-08-09T13:48:31.199 INFO:tasks.ceph.mgr.x.smithi203.stderr:2023-08-09T13:48:31.196+0000 7f4d27e32040 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2023-08-09T13:48:31.220 INFO:tasks.ceph.mgr.y.smithi043.stderr:2023-08-09T13:48:31.218+0000 7f4a4ae0d040 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2023-08-09T13:48:31.281 INFO:tasks.ceph.mgr.y.smithi043.stderr:2023-08-09T13:48:31.278+0000 7f4a4ae0d040 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2023-08-09T13:48:31.307 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:31.304+0000 7f374f838640 1 Processor -- start 2023-08-09T13:48:31.307 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:31.305+0000 7f374f838640 1 -- start start 2023-08-09T13:48:31.309 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:31.306+0000 7f374f838640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f374815a510 0x7f3748160fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:48:31.309 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:31.306+0000 7f374f838640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f374815d5e0 0x7f37480b3ec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:48:31.309 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:31.306+0000 7f374f838640 1 -- --> v1:172.21.15.203:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f37480ccd30 con 0x7f374815da60 2023-08-09T13:48:31.309 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:31.306+0000 7f374f838640 1 -- --> v2:172.21.15.43:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f374815a8f0 con 0x7f374815a510 2023-08-09T13:48:31.310 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:31.306+0000 7f374f838640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f37480cdd40 con 0x7f374815d5e0 2023-08-09T13:48:31.310 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:31.307+0000 7f374d5d4640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f374815a510 0x7f3748160fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:48:31.310 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:31.307+0000 7f374d5d4640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f374815a510 0x7f3748160fe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:33998/0 (socket says 172.21.15.43:33998) 2023-08-09T13:48:31.310 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:31.307+0000 7f374d5d4640 1 -- 172.21.15.43:0/2152873639 learned_addr learned my addr 172.21.15.43:0/2152873639 (peer_addr_for_me v2:172.21.15.43:0/0) 2023-08-09T13:48:31.311 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:31.307+0000 7f374cdd3640 1 --2- 172.21.15.43:0/2152873639 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f374815d5e0 0x7f37480b3ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:48:31.311 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:31.307+0000 7f374d5d4640 1 -- 172.21.15.43:0/2152873639 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f374815d5e0 msgr2=0x7f37480b3ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-09T13:48:31.311 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:31.307+0000 7f374d5d4640 1 --2- 172.21.15.43:0/2152873639 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f374815d5e0 0x7f37480b3ec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:48:31.311 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:31.307+0000 7f374d5d4640 1 -- 172.21.15.43:0/2152873639 >> v1:172.21.15.203:6789/0 conn(0x7f374815da60 legacy=0x7f374806a1f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-09T13:48:31.311 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:31.307+0000 7f374d5d4640 1 -- 172.21.15.43:0/2152873639 --> v2:172.21.15.43:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f37480ccf80 con 0x7f374815a510 2023-08-09T13:48:31.311 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:31.308+0000 7f374d5d4640 1 --2- 172.21.15.43:0/2152873639 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f374815a510 0x7f3748160fe0 secure :-1 s=READY pgs=883 cs=0 l=1 rev1=1 crypto rx=0x7f3744009870 tx=0x7f374402b8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fc2e2c54060ba117 server_cookie=0 in_seq=0 out_seq=0 2023-08-09T13:48:31.312 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:31.309+0000 7f373ffff640 1 -- 172.21.15.43:0/2152873639 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3744031640 con 0x7f374815a510 2023-08-09T13:48:31.312 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:31.309+0000 7f373ffff640 1 -- 172.21.15.43:0/2152873639 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f37440317a0 con 0x7f374815a510 2023-08-09T13:48:31.313 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:31.309+0000 7f373ffff640 1 -- 172.21.15.43:0/2152873639 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3744031b30 con 0x7f374815a510 2023-08-09T13:48:31.313 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:31.309+0000 7f374f838640 1 -- 172.21.15.43:0/2152873639 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f374815a510 msgr2=0x7f3748160fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:48:31.313 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:31.309+0000 7f374f838640 1 --2- 172.21.15.43:0/2152873639 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f374815a510 0x7f3748160fe0 secure :-1 s=READY pgs=883 cs=0 l=1 rev1=1 crypto rx=0x7f3744009870 tx=0x7f374402b8f0 comp rx=0 tx=0).stop 2023-08-09T13:48:31.313 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:31.310+0000 7f374f838640 1 -- 172.21.15.43:0/2152873639 shutdown_connections 2023-08-09T13:48:31.313 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:31.310+0000 7f374f838640 1 --2- 172.21.15.43:0/2152873639 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f374815d5e0 0x7f37480b3ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:48:31.313 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:31.310+0000 7f374f838640 1 --2- 172.21.15.43:0/2152873639 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f374815a510 0x7f3748160fe0 unknown :-1 s=CLOSED pgs=883 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:48:31.313 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:31.310+0000 7f374f838640 1 -- 172.21.15.43:0/2152873639 >> 172.21.15.43:0/2152873639 conn(0x7f37480ceec0 msgr2=0x7f37480cf2d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-09T13:48:31.314 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:31.310+0000 7f374f838640 1 -- 172.21.15.43:0/2152873639 shutdown_connections 2023-08-09T13:48:31.314 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:31.310+0000 7f374f838640 1 -- 172.21.15.43:0/2152873639 wait complete. 2023-08-09T13:48:31.314 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:31.311+0000 7f374f838640 1 Processor -- start 2023-08-09T13:48:31.314 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:31.311+0000 7f374f838640 1 -- start start 2023-08-09T13:48:31.315 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:31.312+0000 7f374f838640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f374815a510 0x7f37481f5c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:48:31.316 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:31.312+0000 7f374f838640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f374815d5e0 0x7f37481463d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:48:31.316 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:31.312+0000 7f374f838640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f374815da60 0x7f37480d0520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:48:31.316 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:31.312+0000 7f374f838640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f37480c6f70 con 0x7f374815a510 2023-08-09T13:48:31.316 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:31.312+0000 7f374f838640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f374814d050 con 0x7f374815d5e0 2023-08-09T13:48:31.316 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:31.312+0000 7f374f838640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f37481f61c0 con 0x7f374815da60 2023-08-09T13:48:31.316 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:31.313+0000 7f374d5d4640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f374815a510 0x7f37481f5c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:48:31.316 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:31.313+0000 7f374d5d4640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f374815a510 0x7f37481f5c80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:34002/0 (socket says 172.21.15.43:34002) 2023-08-09T13:48:31.317 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:31.313+0000 7f374d5d4640 1 -- 172.21.15.43:0/757098364 learned_addr learned my addr 172.21.15.43:0/757098364 (peer_addr_for_me v2:172.21.15.43:0/0) 2023-08-09T13:48:31.317 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:31.313+0000 7f374cdd3640 1 --2- 172.21.15.43:0/757098364 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f374815d5e0 0x7f37481463d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:48:31.317 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:31.313+0000 7f374ddd5640 1 --2- 172.21.15.43:0/757098364 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f374815da60 0x7f37480d0520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:48:31.317 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:31.313+0000 7f374d5d4640 1 -- 172.21.15.43:0/757098364 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f374815da60 msgr2=0x7f37480d0520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:48:31.317 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:31.313+0000 7f374d5d4640 1 --2- 172.21.15.43:0/757098364 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f374815da60 0x7f37480d0520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:48:31.317 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:31.313+0000 7f374d5d4640 1 -- 172.21.15.43:0/757098364 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f374815d5e0 msgr2=0x7f37481463d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:48:31.317 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:31.313+0000 7f374d5d4640 1 --2- 172.21.15.43:0/757098364 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f374815d5e0 0x7f37481463d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:48:31.317 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:31.313+0000 7f374d5d4640 1 -- 172.21.15.43:0/757098364 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f37440094d0 con 0x7f374815a510 2023-08-09T13:48:31.318 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:31.313+0000 7f374cdd3640 1 --2- 172.21.15.43:0/757098364 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f374815d5e0 0x7f37481463d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-08-09T13:48:31.318 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:31.313+0000 7f374d5d4640 1 --2- 172.21.15.43:0/757098364 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f374815a510 0x7f37481f5c80 secure :-1 s=READY pgs=884 cs=0 l=1 rev1=1 crypto rx=0x7f3744009870 tx=0x7f37440046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-09T13:48:31.319 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:31.314+0000 7f373dffb640 1 -- 172.21.15.43:0/757098364 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f374403ac60 con 0x7f374815a510 2023-08-09T13:48:31.319 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:31.314+0000 7f373dffb640 1 -- 172.21.15.43:0/757098364 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f374403adc0 con 0x7f374815a510 2023-08-09T13:48:31.319 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:31.314+0000 7f373dffb640 1 -- 172.21.15.43:0/757098364 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3744044630 con 0x7f374815a510 2023-08-09T13:48:31.319 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:31.314+0000 7f374f838640 1 -- 172.21.15.43:0/757098364 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3748146c90 con 0x7f374815a510 2023-08-09T13:48:31.319 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:31.314+0000 7f374f838640 1 -- 172.21.15.43:0/757098364 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f37480c66d0 con 0x7f374815a510 2023-08-09T13:48:31.319 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:31.315+0000 7f374f838640 1 -- 172.21.15.43:0/757098364 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f37480739f0 con 0x7f374815a510 2023-08-09T13:48:31.319 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:31.316+0000 7f373dffb640 1 -- 172.21.15.43:0/757098364 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 19) v1 ==== 88460+0+0 (secure 0 0 0) 0x7f374402f050 con 0x7f374815a510 2023-08-09T13:48:31.320 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:31.317+0000 7f373dffb640 1 --2- 172.21.15.43:0/757098364 >> [v2:172.21.15.203:6832/272917756,v1:172.21.15.203:6833/272917756] conn(0x7f372c06a1c0 0x7f372c06c680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:48:31.320 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:31.317+0000 7f373dffb640 1 -- 172.21.15.43:0/757098364 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(74..74 src has 1..74) v4 ==== 6426+0+0 (secure 0 0 0) 0x7f37440aa7c0 con 0x7f374815a510 2023-08-09T13:48:31.320 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:31.317+0000 7f374cdd3640 1 -- 172.21.15.43:0/757098364 >> [v2:172.21.15.203:6832/272917756,v1:172.21.15.203:6833/272917756] conn(0x7f372c06a1c0 msgr2=0x7f372c06c680 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.203:6832/272917756 2023-08-09T13:48:31.320 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:31.317+0000 7f374cdd3640 1 --2- 172.21.15.43:0/757098364 >> [v2:172.21.15.203:6832/272917756,v1:172.21.15.203:6833/272917756] conn(0x7f372c06a1c0 0x7f372c06c680 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-08-09T13:48:31.327 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:31.325+0000 7f373dffb640 1 -- 172.21.15.43:0/757098364 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178066 (secure 0 0 0) 0x7f3744075bb0 con 0x7f374815a510 2023-08-09T13:48:31.340 INFO:tasks.ceph.mgr.y.smithi043.stderr:2023-08-09T13:48:31.337+0000 7f4a4ae0d040 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2023-08-09T13:48:31.520 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:31.518+0000 7f374cdd3640 1 -- 172.21.15.43:0/757098364 >> [v2:172.21.15.203:6832/272917756,v1:172.21.15.203:6833/272917756] conn(0x7f372c06a1c0 msgr2=0x7f372c06c680 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.203:6832/272917756 2023-08-09T13:48:31.521 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:31.518+0000 7f374cdd3640 1 --2- 172.21.15.43:0/757098364 >> [v2:172.21.15.203:6832/272917756,v1:172.21.15.203:6833/272917756] conn(0x7f372c06a1c0 0x7f372c06c680 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2023-08-09T13:48:31.544 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:31.541+0000 7f374f838640 1 -- 172.21.15.43:0/757098364 --> [v2:172.21.15.203:6832/272917756,v1:172.21.15.203:6833/272917756] -- mgr_command(tid 0: {"prefix": "fs snap-schedule add", "fs": "cephfs", "path": "snap_test_dir1", "snap_schedule": "1M", "target": ["mon-mgr", ""]}) v1 -- 0x7f37480cd540 con 0x7f372c06a1c0 2023-08-09T13:48:31.892 INFO:tasks.ceph.mgr.x.smithi203.stderr:2023-08-09T13:48:31.890+0000 7f4d27e32040 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2023-08-09T13:48:31.922 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:31.919+0000 7f374cdd3640 1 -- 172.21.15.43:0/757098364 >> [v2:172.21.15.203:6832/272917756,v1:172.21.15.203:6833/272917756] conn(0x7f372c06a1c0 msgr2=0x7f372c06c680 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.203:6832/272917756 2023-08-09T13:48:31.922 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:31.919+0000 7f374cdd3640 1 --2- 172.21.15.43:0/757098364 >> [v2:172.21.15.203:6832/272917756,v1:172.21.15.203:6833/272917756] conn(0x7f372c06a1c0 0x7f372c06c680 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2023-08-09T13:48:31.953 INFO:tasks.ceph.mgr.x.smithi203.stderr:2023-08-09T13:48:31.950+0000 7f4d27e32040 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2023-08-09T13:48:32.002 INFO:tasks.ceph.mgr.x.smithi203.stderr:2023-08-09T13:48:31.999+0000 7f4d27e32040 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2023-08-09T13:48:32.089 INFO:tasks.ceph.mgr.y.smithi043.stderr:2023-08-09T13:48:32.086+0000 7f4a4ae0d040 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2023-08-09T13:48:32.118 INFO:tasks.ceph.mgr.x.smithi203.stderr:2023-08-09T13:48:32.116+0000 7f4d27e32040 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2023-08-09T13:48:32.153 INFO:tasks.ceph.mgr.y.smithi043.stderr:2023-08-09T13:48:32.150+0000 7f4a4ae0d040 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2023-08-09T13:48:32.206 INFO:tasks.ceph.mgr.y.smithi043.stderr:2023-08-09T13:48:32.203+0000 7f4a4ae0d040 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2023-08-09T13:48:32.276 INFO:tasks.ceph.mgr.x.smithi203.stderr:2023-08-09T13:48:32.273+0000 7f4d27e32040 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2023-08-09T13:48:32.331 INFO:tasks.ceph.mgr.y.smithi043.stderr:2023-08-09T13:48:32.328+0000 7f4a4ae0d040 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2023-08-09T13:48:32.382 INFO:tasks.ceph.mgr.x.smithi203.stderr:2023-08-09T13:48:32.379+0000 7f4d27e32040 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2023-08-09T13:48:32.430 INFO:tasks.ceph.mgr.x.smithi203.stderr:2023-08-09T13:48:32.428+0000 7f4d27e32040 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2023-08-09T13:48:32.496 INFO:tasks.ceph.mgr.y.smithi043.stderr:2023-08-09T13:48:32.494+0000 7f4a4ae0d040 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2023-08-09T13:48:32.590 INFO:tasks.ceph.mgr.x.smithi203.stderr:2023-08-09T13:48:32.587+0000 7f4d27e32040 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2023-08-09T13:48:32.612 INFO:tasks.ceph.mgr.y.smithi043.stderr:2023-08-09T13:48:32.609+0000 7f4a4ae0d040 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2023-08-09T13:48:32.664 INFO:tasks.ceph.mgr.y.smithi043.stderr:2023-08-09T13:48:32.661+0000 7f4a4ae0d040 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2023-08-09T13:48:32.723 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:32.720+0000 7f374cdd3640 1 -- 172.21.15.43:0/757098364 >> [v2:172.21.15.203:6832/272917756,v1:172.21.15.203:6833/272917756] conn(0x7f372c06a1c0 msgr2=0x7f372c06c680 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.203:6832/272917756 2023-08-09T13:48:32.723 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:32.720+0000 7f374cdd3640 1 --2- 172.21.15.43:0/757098364 >> [v2:172.21.15.203:6832/272917756,v1:172.21.15.203:6833/272917756] conn(0x7f372c06a1c0 0x7f372c06c680 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2023-08-09T13:48:32.834 INFO:tasks.ceph.mgr.y.smithi043.stderr:2023-08-09T13:48:32.831+0000 7f4a4ae0d040 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2023-08-09T13:48:32.969 INFO:tasks.ceph.mgr.x.smithi203.stderr:2023-08-09T13:48:32.967+0000 7f4d27e32040 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2023-08-09T13:48:33.074 DEBUG:teuthology.orchestra.run.smithi043:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-08-09T13:48:33.081 DEBUG:teuthology.orchestra.run.smithi203:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-08-09T13:48:33.194 INFO:tasks.ceph.mgr.x.smithi203.stderr:2023-08-09T13:48:33.192+0000 7f4d27e32040 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2023-08-09T13:48:33.252 INFO:tasks.ceph.mgr.x.smithi203.stderr:2023-08-09T13:48:33.251+0000 7f4d27e32040 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2023-08-09T13:48:33.258 INFO:tasks.ceph.mgr.y.smithi043.stderr:2023-08-09T13:48:33.256+0000 7f4a4ae0d040 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2023-08-09T13:48:33.303 INFO:tasks.ceph.mgr.x.smithi203.stderr:2023-08-09T13:48:33.302+0000 7f4d27e32040 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2023-08-09T13:48:33.355 INFO:tasks.ceph.mgr.x.smithi203.stderr:2023-08-09T13:48:33.353+0000 7f4d27e32040 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2023-08-09T13:48:33.490 INFO:tasks.ceph.mgr.y.smithi043.stderr:2023-08-09T13:48:33.487+0000 7f4a4ae0d040 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2023-08-09T13:48:33.550 INFO:tasks.ceph.mgr.y.smithi043.stderr:2023-08-09T13:48:33.547+0000 7f4a4ae0d040 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2023-08-09T13:48:33.603 INFO:tasks.ceph.mgr.y.smithi043.stderr:2023-08-09T13:48:33.601+0000 7f4a4ae0d040 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2023-08-09T13:48:33.647 INFO:tasks.ceph.mgr.x.smithi203.stderr:2023-08-09T13:48:33.646+0000 7f4d27e32040 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2023-08-09T13:48:33.658 INFO:tasks.ceph.mgr.y.smithi043.stderr:2023-08-09T13:48:33.655+0000 7f4a4ae0d040 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2023-08-09T13:48:33.704 INFO:tasks.ceph.mgr.x.smithi203.stderr:2023-08-09T13:48:33.702+0000 7f4d27e32040 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2023-08-09T13:48:33.755 INFO:tasks.ceph.mgr.x.smithi203.stderr:2023-08-09T13:48:33.753+0000 7f4d27e32040 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2023-08-09T13:48:33.913 INFO:tasks.ceph.mgr.x.smithi203.stderr:2023-08-09T13:48:33.911+0000 7f4d27e32040 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2023-08-09T13:48:33.961 INFO:tasks.ceph.mgr.x.smithi203.stderr:2023-08-09T13:48:33.960+0000 7f4d27e32040 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2023-08-09T13:48:33.968 INFO:tasks.ceph.mgr.y.smithi043.stderr:2023-08-09T13:48:33.966+0000 7f4a4ae0d040 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2023-08-09T13:48:34.013 INFO:tasks.ceph.mgr.x.smithi203.stderr:2023-08-09T13:48:34.011+0000 7f4d27e32040 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2023-08-09T13:48:34.029 INFO:tasks.ceph.mgr.y.smithi043.stderr:2023-08-09T13:48:34.026+0000 7f4a4ae0d040 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2023-08-09T13:48:34.062 INFO:tasks.ceph.mgr.x.smithi203.stderr:2023-08-09T13:48:34.060+0000 7f4d27e32040 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2023-08-09T13:48:34.084 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:34.080+0000 7f373dffb640 1 -- 172.21.15.43:0/757098364 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mgrmap(e 20) v1 ==== 88069+0+0 (secure 0 0 0) 0x7f37440a8a10 con 0x7f374815a510 2023-08-09T13:48:34.085 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:34.081+0000 7f373dffb640 1 -- 172.21.15.43:0/757098364 >> [v2:172.21.15.203:6832/272917756,v1:172.21.15.203:6833/272917756] conn(0x7f372c06a1c0 msgr2=0x7f372c06c680 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-08-09T13:48:34.085 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:34.081+0000 7f373dffb640 1 --2- 172.21.15.43:0/757098364 >> [v2:172.21.15.203:6832/272917756,v1:172.21.15.203:6833/272917756] conn(0x7f372c06a1c0 0x7f372c06c680 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:48:34.088 INFO:tasks.ceph.mgr.y.smithi043.stderr:2023-08-09T13:48:34.086+0000 7f4a4ae0d040 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2023-08-09T13:48:34.245 INFO:tasks.ceph.mgr.x.smithi203.stderr:2023-08-09T13:48:34.244+0000 7f4ca4226640 -1 client.0 error registering admin socket command: (17) File exists 2023-08-09T13:48:34.246 INFO:tasks.ceph.mgr.x.smithi203.stderr:2023-08-09T13:48:34.244+0000 7f4ca4226640 -1 client.0 error registering admin socket command: (17) File exists 2023-08-09T13:48:34.246 INFO:tasks.ceph.mgr.x.smithi203.stderr:2023-08-09T13:48:34.244+0000 7f4ca4226640 -1 client.0 error registering admin socket command: (17) File exists 2023-08-09T13:48:34.246 INFO:tasks.ceph.mgr.x.smithi203.stderr:2023-08-09T13:48:34.244+0000 7f4ca4226640 -1 client.0 error registering admin socket command: (17) File exists 2023-08-09T13:48:34.246 INFO:tasks.ceph.mgr.x.smithi203.stderr:2023-08-09T13:48:34.244+0000 7f4ca4226640 -1 client.0 error registering admin socket command: (17) File exists 2023-08-09T13:48:34.246 INFO:tasks.ceph.mgr.x.smithi203.stderr:2023-08-09T13:48:34.244+0000 7f4ca722c640 -1 client.0 error registering admin socket command: (17) File exists 2023-08-09T13:48:34.246 INFO:tasks.ceph.mgr.x.smithi203.stderr:2023-08-09T13:48:34.244+0000 7f4ca722c640 -1 client.0 error registering admin socket command: (17) File exists 2023-08-09T13:48:34.247 INFO:tasks.ceph.mgr.x.smithi203.stderr:2023-08-09T13:48:34.244+0000 7f4ca722c640 -1 client.0 error registering admin socket command: (17) File exists 2023-08-09T13:48:34.247 INFO:tasks.ceph.mgr.x.smithi203.stderr:2023-08-09T13:48:34.244+0000 7f4ca722c640 -1 client.0 error registering admin socket command: (17) File exists 2023-08-09T13:48:34.247 INFO:tasks.ceph.mgr.x.smithi203.stderr:2023-08-09T13:48:34.244+0000 7f4ca722c640 -1 client.0 error registering admin socket command: (17) File exists 2023-08-09T13:48:34.281 INFO:tasks.ceph.mgr.y.smithi043.stderr:2023-08-09T13:48:34.278+0000 7f4a4ae0d040 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2023-08-09T13:48:34.333 INFO:tasks.ceph.mgr.y.smithi043.stderr:2023-08-09T13:48:34.330+0000 7f4a4ae0d040 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2023-08-09T13:48:34.388 INFO:tasks.ceph.mgr.y.smithi043.stderr:2023-08-09T13:48:34.386+0000 7f4a4ae0d040 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2023-08-09T13:48:34.441 INFO:tasks.ceph.mgr.y.smithi043.stderr:2023-08-09T13:48:34.439+0000 7f4a4ae0d040 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2023-08-09T13:48:35.097 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:35.094+0000 7f373dffb640 1 -- 172.21.15.43:0/757098364 <== mon.0 v2:172.21.15.43:3300/0 8 ==== mgrmap(e 21) v1 ==== 88196+0+0 (secure 0 0 0) 0x7f3744077690 con 0x7f374815a510 2023-08-09T13:48:35.098 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:35.095+0000 7f373dffb640 1 --2- 172.21.15.43:0/757098364 >> [v2:172.21.15.203:6832/2515481191,v1:172.21.15.203:6833/2515481191] conn(0x7f372c072cb0 0x7f372c0750a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:48:35.098 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:35.095+0000 7f373dffb640 1 -- 172.21.15.43:0/757098364 --> [v2:172.21.15.203:6832/2515481191,v1:172.21.15.203:6833/2515481191] -- mgr_command(tid 0: {"prefix": "fs snap-schedule add", "fs": "cephfs", "path": "snap_test_dir1", "snap_schedule": "1M", "target": ["mon-mgr", ""]}) v1 -- 0x7f372c036270 con 0x7f372c072cb0 2023-08-09T13:48:35.099 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:35.096+0000 7f374cdd3640 1 --2- 172.21.15.43:0/757098364 >> [v2:172.21.15.203:6832/2515481191,v1:172.21.15.203:6833/2515481191] conn(0x7f372c072cb0 0x7f372c0750a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:48:35.102 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:35.099+0000 7f374cdd3640 1 --2- 172.21.15.43:0/757098364 >> [v2:172.21.15.203:6832/2515481191,v1:172.21.15.203:6833/2515481191] conn(0x7f372c072cb0 0x7f372c0750a0 secure :-1 s=READY pgs=4 cs=0 l=1 rev1=1 crypto rx=0x7f3748160490 tx=0x7f3738008040 comp rx=0 tx=0).ready entity=mgr.5444 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-09T13:48:35.270 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:35.268+0000 7f373dffb640 1 -- 172.21.15.43:0/757098364 <== mgr.5444 v2:172.21.15.203:6832/2515481191 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+37 (secure 0 0 0) 0x7f373800d820 con 0x7f372c072cb0 2023-08-09T13:48:35.271 INFO:teuthology.orchestra.run.smithi043.stdout:Schedule set for path snap_test_dir1 2023-08-09T13:48:35.274 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:35.271+0000 7f374f838640 1 -- 172.21.15.43:0/757098364 >> [v2:172.21.15.203:6832/2515481191,v1:172.21.15.203:6833/2515481191] conn(0x7f372c072cb0 msgr2=0x7f372c0750a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:48:35.274 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:35.271+0000 7f374f838640 1 --2- 172.21.15.43:0/757098364 >> [v2:172.21.15.203:6832/2515481191,v1:172.21.15.203:6833/2515481191] conn(0x7f372c072cb0 0x7f372c0750a0 secure :-1 s=READY pgs=4 cs=0 l=1 rev1=1 crypto rx=0x7f3748160490 tx=0x7f3738008040 comp rx=0 tx=0).stop 2023-08-09T13:48:35.274 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:35.271+0000 7f374f838640 1 -- 172.21.15.43:0/757098364 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f374815a510 msgr2=0x7f37481f5c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:48:35.274 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:35.271+0000 7f374f838640 1 --2- 172.21.15.43:0/757098364 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f374815a510 0x7f37481f5c80 secure :-1 s=READY pgs=884 cs=0 l=1 rev1=1 crypto rx=0x7f3744009870 tx=0x7f37440046c0 comp rx=0 tx=0).stop 2023-08-09T13:48:35.274 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:35.271+0000 7f374ddd5640 1 -- 172.21.15.43:0/757098364 reap_dead start 2023-08-09T13:48:35.274 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:35.271+0000 7f374f838640 1 -- 172.21.15.43:0/757098364 shutdown_connections 2023-08-09T13:48:35.275 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:35.271+0000 7f374f838640 1 -- 172.21.15.43:0/757098364 >> 172.21.15.43:0/757098364 conn(0x7f37480ceec0 msgr2=0x7f3748160570 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-09T13:48:35.275 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:35.271+0000 7f374f838640 1 -- 172.21.15.43:0/757098364 shutdown_connections 2023-08-09T13:48:35.275 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:35.271+0000 7f374f838640 1 -- 172.21.15.43:0/757098364 wait complete. 2023-08-09T13:48:35.296 DEBUG:tasks.cephfs.test_snap_schedules:res=Schedule set for path snap_test_dir1 2023-08-09T13:48:35.297 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs snap-schedule add --fs cephfs snap_test_dir1 2M 2023-08-09T13:48:35.527 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:35.524+0000 7f20da20f640 1 Processor -- start 2023-08-09T13:48:35.527 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:35.525+0000 7f20da20f640 1 -- start start 2023-08-09T13:48:35.529 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:35.527+0000 7f20da20f640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f20d40cc4a0 0x7f20d415a790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:48:35.529 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:35.527+0000 7f20da20f640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f20d40b1ad0 0x7f20d40b1f60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:48:35.529 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:35.527+0000 7f20da20f640 1 -- --> v1:172.21.15.43:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f20d40cfe60 con 0x7f20d40cfa80 2023-08-09T13:48:35.530 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:35.527+0000 7f20da20f640 1 -- --> v2:172.21.15.43:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f20d40b24a0 con 0x7f20d40cc4a0 2023-08-09T13:48:35.530 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:35.527+0000 7f20da20f640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f20d415de20 con 0x7f20d40b1ad0 2023-08-09T13:48:35.531 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:35.527+0000 7f20d8a0c640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f20d40cc4a0 0x7f20d415a790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:48:35.531 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:35.527+0000 7f20d8a0c640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f20d40cc4a0 0x7f20d415a790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:40970/0 (socket says 172.21.15.43:40970) 2023-08-09T13:48:35.531 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:35.527+0000 7f20d8a0c640 1 -- 172.21.15.43:0/867108611 learned_addr learned my addr 172.21.15.43:0/867108611 (peer_addr_for_me v2:172.21.15.43:0/0) 2023-08-09T13:48:35.531 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:35.527+0000 7f20d9a0e640 1 --2- 172.21.15.43:0/867108611 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f20d40b1ad0 0x7f20d40b1f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:48:35.531 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:35.527+0000 7f20d8a0c640 1 -- 172.21.15.43:0/867108611 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f20d40b1ad0 msgr2=0x7f20d40b1f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-09T13:48:35.532 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:35.527+0000 7f20d8a0c640 1 --2- 172.21.15.43:0/867108611 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f20d40b1ad0 0x7f20d40b1f60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:48:35.532 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:35.528+0000 7f20d8a0c640 1 -- 172.21.15.43:0/867108611 >> v1:172.21.15.43:6789/0 conn(0x7f20d40cfa80 legacy=0x7f20d40ca6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:48:35.533 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:35.528+0000 7f20d8a0c640 1 -- 172.21.15.43:0/867108611 --> v2:172.21.15.43:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f20d415e100 con 0x7f20d40cc4a0 2023-08-09T13:48:35.534 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:35.529+0000 7f20d8a0c640 1 --2- 172.21.15.43:0/867108611 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f20d40cc4a0 0x7f20d415a790 secure :-1 s=READY pgs=886 cs=0 l=1 rev1=1 crypto rx=0x7f20c4009870 tx=0x7f20c402b980 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9fd5af408c7726e server_cookie=0 in_seq=0 out_seq=0 2023-08-09T13:48:35.534 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:35.529+0000 7f20cbfff640 1 -- 172.21.15.43:0/867108611 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f20c4030840 con 0x7f20d40cc4a0 2023-08-09T13:48:35.534 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:35.529+0000 7f20cbfff640 1 -- 172.21.15.43:0/867108611 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f20c40309a0 con 0x7f20d40cc4a0 2023-08-09T13:48:35.534 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:35.529+0000 7f20cbfff640 1 -- 172.21.15.43:0/867108611 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f20c4030d30 con 0x7f20d40cc4a0 2023-08-09T13:48:35.534 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:35.530+0000 7f20da20f640 1 -- 172.21.15.43:0/867108611 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f20d40cc4a0 msgr2=0x7f20d415a790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:48:35.534 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:35.530+0000 7f20da20f640 1 --2- 172.21.15.43:0/867108611 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f20d40cc4a0 0x7f20d415a790 secure :-1 s=READY pgs=886 cs=0 l=1 rev1=1 crypto rx=0x7f20c4009870 tx=0x7f20c402b980 comp rx=0 tx=0).stop 2023-08-09T13:48:35.535 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:35.530+0000 7f20da20f640 1 -- 172.21.15.43:0/867108611 shutdown_connections 2023-08-09T13:48:35.535 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:35.530+0000 7f20da20f640 1 --2- 172.21.15.43:0/867108611 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f20d40b1ad0 0x7f20d40b1f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:48:35.535 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:35.530+0000 7f20da20f640 1 --2- 172.21.15.43:0/867108611 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f20d40cc4a0 0x7f20d415a790 unknown :-1 s=CLOSED pgs=886 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:48:35.535 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:35.530+0000 7f20da20f640 1 -- 172.21.15.43:0/867108611 >> 172.21.15.43:0/867108611 conn(0x7f20d40ccad0 msgr2=0x7f20d40ccee0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-09T13:48:35.535 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:35.530+0000 7f20da20f640 1 -- 172.21.15.43:0/867108611 shutdown_connections 2023-08-09T13:48:35.535 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:35.530+0000 7f20da20f640 1 -- 172.21.15.43:0/867108611 wait complete. 2023-08-09T13:48:35.536 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:35.533+0000 7f20da20f640 1 Processor -- start 2023-08-09T13:48:35.536 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:35.533+0000 7f20da20f640 1 -- start start 2023-08-09T13:48:35.537 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:35.534+0000 7f20da20f640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f20d40b1ad0 0x7f20d40c4090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:48:35.537 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:35.534+0000 7f20da20f640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f20d40cc4a0 0x7f20d40b4990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:48:35.537 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:35.534+0000 7f20da20f640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f20d40cfa80 0x7f20d41f5050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:48:35.537 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:35.534+0000 7f20da20f640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f20d40c80b0 con 0x7f20d40cc4a0 2023-08-09T13:48:35.537 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:35.534+0000 7f20da20f640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f20d40c74a0 con 0x7f20d40cfa80 2023-08-09T13:48:35.538 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:35.534+0000 7f20da20f640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f20d40c7780 con 0x7f20d40b1ad0 2023-08-09T13:48:35.538 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:35.535+0000 7f20d8a0c640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f20d40cc4a0 0x7f20d40b4990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:48:35.538 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:35.535+0000 7f20d8a0c640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f20d40cc4a0 0x7f20d40b4990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:40978/0 (socket says 172.21.15.43:40978) 2023-08-09T13:48:35.538 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:35.535+0000 7f20d8a0c640 1 -- 172.21.15.43:0/222276333 learned_addr learned my addr 172.21.15.43:0/222276333 (peer_addr_for_me v2:172.21.15.43:0/0) 2023-08-09T13:48:35.539 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:35.535+0000 7f20d920d640 1 --2- 172.21.15.43:0/222276333 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f20d40b1ad0 0x7f20d40c4090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:48:35.539 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:35.535+0000 7f20d9a0e640 1 --2- 172.21.15.43:0/222276333 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f20d40cfa80 0x7f20d41f5050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:48:35.539 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:35.535+0000 7f20d8a0c640 1 -- 172.21.15.43:0/222276333 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f20d40b1ad0 msgr2=0x7f20d40c4090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:48:35.540 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:35.535+0000 7f20d8a0c640 1 --2- 172.21.15.43:0/222276333 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f20d40b1ad0 0x7f20d40c4090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:48:35.540 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:35.535+0000 7f20d8a0c640 1 -- 172.21.15.43:0/222276333 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f20d40cfa80 msgr2=0x7f20d41f5050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:48:35.540 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:35.535+0000 7f20d8a0c640 1 --2- 172.21.15.43:0/222276333 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f20d40cfa80 0x7f20d41f5050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:48:35.540 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:35.535+0000 7f20d8a0c640 1 -- 172.21.15.43:0/222276333 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f20c40094d0 con 0x7f20d40cc4a0 2023-08-09T13:48:35.540 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:35.535+0000 7f20d920d640 1 --2- 172.21.15.43:0/222276333 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f20d40b1ad0 0x7f20d40c4090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-08-09T13:48:35.540 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:35.536+0000 7f20d8a0c640 1 --2- 172.21.15.43:0/222276333 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f20d40cc4a0 0x7f20d40b4990 secure :-1 s=READY pgs=887 cs=0 l=1 rev1=1 crypto rx=0x7f20c4000c00 tx=0x7f20c4004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-09T13:48:35.541 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:35.536+0000 7f20c9ffb640 1 -- 172.21.15.43:0/222276333 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f20c4030810 con 0x7f20d40cc4a0 2023-08-09T13:48:35.541 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:35.536+0000 7f20c9ffb640 1 -- 172.21.15.43:0/222276333 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f20c4043400 con 0x7f20d40cc4a0 2023-08-09T13:48:35.541 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:35.536+0000 7f20da20f640 1 -- 172.21.15.43:0/222276333 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f20d40c7a60 con 0x7f20d40cc4a0 2023-08-09T13:48:35.541 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:35.536+0000 7f20c9ffb640 1 -- 172.21.15.43:0/222276333 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f20c40437d0 con 0x7f20d40cc4a0 2023-08-09T13:48:35.541 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:35.536+0000 7f20da20f640 1 -- 172.21.15.43:0/222276333 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f20d40c7120 con 0x7f20d40cc4a0 2023-08-09T13:48:35.542 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:35.537+0000 7f20da20f640 1 -- 172.21.15.43:0/222276333 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f20d40739f0 con 0x7f20d40cc4a0 2023-08-09T13:48:35.542 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:35.538+0000 7f20c9ffb640 1 -- 172.21.15.43:0/222276333 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 21) v1 ==== 88196+0+0 (secure 0 0 0) 0x7f20c4043930 con 0x7f20d40cc4a0 2023-08-09T13:48:35.542 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:35.540+0000 7f20c9ffb640 1 --2- 172.21.15.43:0/222276333 >> [v2:172.21.15.203:6832/2515481191,v1:172.21.15.203:6833/2515481191] conn(0x7f20b8069ef0 0x7f20b806c3b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:48:35.543 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:35.540+0000 7f20c9ffb640 1 -- 172.21.15.43:0/222276333 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(75..75 src has 1..75) v4 ==== 6770+0+0 (secure 0 0 0) 0x7f20c403e070 con 0x7f20d40cc4a0 2023-08-09T13:48:35.543 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:35.540+0000 7f20d920d640 1 --2- 172.21.15.43:0/222276333 >> [v2:172.21.15.203:6832/2515481191,v1:172.21.15.203:6833/2515481191] conn(0x7f20b8069ef0 0x7f20b806c3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:48:35.544 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:35.542+0000 7f20d920d640 1 --2- 172.21.15.43:0/222276333 >> [v2:172.21.15.203:6832/2515481191,v1:172.21.15.203:6833/2515481191] conn(0x7f20b8069ef0 0x7f20b806c3b0 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7f20d00012e0 tx=0x7f20d0009040 comp rx=0 tx=0).ready entity=mgr.5444 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-09T13:48:35.547 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:35.545+0000 7f20c9ffb640 1 -- 172.21.15.43:0/222276333 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178066 (secure 0 0 0) 0x7f20c4076ed0 con 0x7f20d40cc4a0 2023-08-09T13:48:35.767 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:35.764+0000 7f20da20f640 1 -- 172.21.15.43:0/222276333 --> [v2:172.21.15.203:6832/2515481191,v1:172.21.15.203:6833/2515481191] -- mgr_command(tid 0: {"prefix": "fs snap-schedule add", "fs": "cephfs", "path": "snap_test_dir1", "snap_schedule": "2M", "target": ["mon-mgr", ""]}) v1 -- 0x7f20d415e6c0 con 0x7f20b8069ef0 2023-08-09T13:48:35.911 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:35.908+0000 7f20c9ffb640 1 -- 172.21.15.43:0/222276333 <== mgr.5444 v2:172.21.15.203:6832/2515481191 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+37 (secure 0 0 0) 0x7f20d0001a50 con 0x7f20b8069ef0 2023-08-09T13:48:35.912 INFO:teuthology.orchestra.run.smithi043.stdout:Schedule set for path snap_test_dir1 2023-08-09T13:48:35.914 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:35.912+0000 7f20da20f640 1 -- 172.21.15.43:0/222276333 >> [v2:172.21.15.203:6832/2515481191,v1:172.21.15.203:6833/2515481191] conn(0x7f20b8069ef0 msgr2=0x7f20b806c3b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:48:35.915 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:35.912+0000 7f20da20f640 1 --2- 172.21.15.43:0/222276333 >> [v2:172.21.15.203:6832/2515481191,v1:172.21.15.203:6833/2515481191] conn(0x7f20b8069ef0 0x7f20b806c3b0 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7f20d00012e0 tx=0x7f20d0009040 comp rx=0 tx=0).stop 2023-08-09T13:48:35.915 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:35.912+0000 7f20da20f640 1 -- 172.21.15.43:0/222276333 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f20d40cc4a0 msgr2=0x7f20d40b4990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:48:35.915 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:35.912+0000 7f20da20f640 1 --2- 172.21.15.43:0/222276333 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f20d40cc4a0 0x7f20d40b4990 secure :-1 s=READY pgs=887 cs=0 l=1 rev1=1 crypto rx=0x7f20c4000c00 tx=0x7f20c4004730 comp rx=0 tx=0).stop 2023-08-09T13:48:35.916 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:35.912+0000 7f20da20f640 1 -- 172.21.15.43:0/222276333 shutdown_connections 2023-08-09T13:48:35.916 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:35.912+0000 7f20da20f640 1 --2- 172.21.15.43:0/222276333 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f20d40cfa80 0x7f20d41f5050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:48:35.916 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:35.912+0000 7f20da20f640 1 --2- 172.21.15.43:0/222276333 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f20d40cc4a0 0x7f20d40b4990 unknown :-1 s=CLOSED pgs=887 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:48:35.916 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:35.912+0000 7f20da20f640 1 --2- 172.21.15.43:0/222276333 >> [v2:172.21.15.203:6832/2515481191,v1:172.21.15.203:6833/2515481191] conn(0x7f20b8069ef0 0x7f20b806c3b0 unknown :-1 s=CLOSED pgs=23 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:48:35.916 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:35.912+0000 7f20da20f640 1 --2- 172.21.15.43:0/222276333 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f20d40b1ad0 0x7f20d40c4090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:48:35.916 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:35.912+0000 7f20da20f640 1 -- 172.21.15.43:0/222276333 >> 172.21.15.43:0/222276333 conn(0x7f20d40ccad0 msgr2=0x7f20d415c960 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-09T13:48:35.916 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:35.912+0000 7f20da20f640 1 -- 172.21.15.43:0/222276333 shutdown_connections 2023-08-09T13:48:35.917 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:35.913+0000 7f20da20f640 1 -- 172.21.15.43:0/222276333 wait complete. 2023-08-09T13:48:35.938 DEBUG:tasks.cephfs.test_snap_schedules:res=Schedule set for path snap_test_dir1 2023-08-09T13:48:35.939 DEBUG:tasks.cephfs.test_snap_schedules:expecting snap snap_test_dir1/.snap/scheduled-2023-08-09-13_49_00 in ~25s... 2023-08-09T13:48:35.939 DEBUG:tasks.cephfs.test_snap_schedules:expecting snap snap_test_dir1/.snap/scheduled-2023-08-09-13_50_00 in ~85s... 2023-08-09T13:48:35.939 DEBUG:tasks.cephfs.test_snap_schedules:expected_schedule: ['1M', '2M'], expected_retention: [] 2023-08-09T13:48:35.940 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs snap-schedule list --fs cephfs --format json snap_test_dir1 2023-08-09T13:48:36.181 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:36.179+0000 7fa6f060d640 1 Processor -- start 2023-08-09T13:48:36.182 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:36.179+0000 7fa6f060d640 1 -- start start 2023-08-09T13:48:36.183 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:36.181+0000 7fa6f060d640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fa6e80ceac0 0x7fa6e80712f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:48:36.183 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:36.181+0000 7fa6f060d640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7fa6e80ca880 0x7fa6e80c8a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:48:36.184 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:36.181+0000 7fa6f060d640 1 -- --> v1:172.21.15.43:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa6e80cadf0 con 0x7fa6e806f990 2023-08-09T13:48:36.184 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:36.181+0000 7fa6f060d640 1 -- --> v2:172.21.15.43:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa6e815a880 con 0x7fa6e80ca880 2023-08-09T13:48:36.184 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:36.181+0000 7fa6f060d640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa6e80c85e0 con 0x7fa6e80ceac0 2023-08-09T13:48:36.184 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:36.181+0000 7fa6eebaa640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7fa6e80ca880 0x7fa6e80c8a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:48:36.184 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:36.181+0000 7fa6eebaa640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7fa6e80ca880 0x7fa6e80c8a30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:40984/0 (socket says 172.21.15.43:40984) 2023-08-09T13:48:36.184 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:36.181+0000 7fa6eebaa640 1 -- 172.21.15.43:0/3819929590 learned_addr learned my addr 172.21.15.43:0/3819929590 (peer_addr_for_me v2:172.21.15.43:0/0) 2023-08-09T13:48:36.185 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:36.181+0000 7fa6edba8640 1 --2- 172.21.15.43:0/3819929590 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fa6e80ceac0 0x7fa6e80712f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:48:36.185 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:36.181+0000 7fa6eebaa640 1 -- 172.21.15.43:0/3819929590 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fa6e80ceac0 msgr2=0x7fa6e80712f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-09T13:48:36.186 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:36.182+0000 7fa6eebaa640 1 --2- 172.21.15.43:0/3819929590 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fa6e80ceac0 0x7fa6e80712f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:48:36.186 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:36.182+0000 7fa6eebaa640 1 -- 172.21.15.43:0/3819929590 >> v1:172.21.15.43:6789/0 conn(0x7fa6e806f990 legacy=0x7fa6e80ce3b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:48:36.186 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:36.182+0000 7fa6eebaa640 1 -- 172.21.15.43:0/3819929590 --> v2:172.21.15.43:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa6e80cb0d0 con 0x7fa6e80ca880 2023-08-09T13:48:36.186 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:36.182+0000 7fa6edba8640 1 --2- 172.21.15.43:0/3819929590 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fa6e80ceac0 0x7fa6e80712f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-08-09T13:48:36.186 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:36.183+0000 7fa6eebaa640 1 --2- 172.21.15.43:0/3819929590 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa6e80ca880 0x7fa6e80c8a30 secure :-1 s=READY pgs=889 cs=0 l=1 rev1=1 crypto rx=0x7fa6e0009e50 tx=0x7fa6e002bee0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7a57820124d41eac server_cookie=0 in_seq=0 out_seq=0 2023-08-09T13:48:36.186 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:36.183+0000 7fa6ed3a7640 1 -- 172.21.15.43:0/3819929590 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa6e002ca80 con 0x7fa6e80ca880 2023-08-09T13:48:36.186 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:36.183+0000 7fa6ed3a7640 1 -- 172.21.15.43:0/3819929590 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa6e002cbe0 con 0x7fa6e80ca880 2023-08-09T13:48:36.187 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:36.183+0000 7fa6ed3a7640 1 -- 172.21.15.43:0/3819929590 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa6e00343f0 con 0x7fa6e80ca880 2023-08-09T13:48:36.187 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:36.184+0000 7fa6f060d640 1 -- 172.21.15.43:0/3819929590 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa6e80ca880 msgr2=0x7fa6e80c8a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:48:36.187 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:36.184+0000 7fa6f060d640 1 --2- 172.21.15.43:0/3819929590 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa6e80ca880 0x7fa6e80c8a30 secure :-1 s=READY pgs=889 cs=0 l=1 rev1=1 crypto rx=0x7fa6e0009e50 tx=0x7fa6e002bee0 comp rx=0 tx=0).stop 2023-08-09T13:48:36.188 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:36.184+0000 7fa6f060d640 1 -- 172.21.15.43:0/3819929590 shutdown_connections 2023-08-09T13:48:36.188 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:36.184+0000 7fa6f060d640 1 --2- 172.21.15.43:0/3819929590 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa6e80ca880 0x7fa6e80c8a30 unknown :-1 s=CLOSED pgs=889 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:48:36.188 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:36.184+0000 7fa6f060d640 1 --2- 172.21.15.43:0/3819929590 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fa6e80ceac0 0x7fa6e80712f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:48:36.188 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:36.184+0000 7fa6f060d640 1 -- 172.21.15.43:0/3819929590 >> 172.21.15.43:0/3819929590 conn(0x7fa6e806b9a0 msgr2=0x7fa6e806bdb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-09T13:48:36.188 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:36.184+0000 7fa6f060d640 1 -- 172.21.15.43:0/3819929590 shutdown_connections 2023-08-09T13:48:36.188 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:36.184+0000 7fa6f060d640 1 -- 172.21.15.43:0/3819929590 wait complete. 2023-08-09T13:48:36.189 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:36.186+0000 7fa6f060d640 1 Processor -- start 2023-08-09T13:48:36.189 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:36.186+0000 7fa6f060d640 1 -- start start 2023-08-09T13:48:36.190 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:36.187+0000 7fa6f060d640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fa6e806f990 0x7fa6e80c3320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:48:36.190 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:36.187+0000 7fa6f060d640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa6e80ca880 0x7fa6e81f25e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:48:36.191 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:36.187+0000 7fa6f060d640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fa6e80ceac0 0x7fa6e81f2b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:48:36.191 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:36.187+0000 7fa6f060d640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa6e80b25f0 con 0x7fa6e80ca880 2023-08-09T13:48:36.191 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:36.187+0000 7fa6f060d640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa6e80b0350 con 0x7fa6e806f990 2023-08-09T13:48:36.191 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:36.187+0000 7fa6f060d640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa6e80b0630 con 0x7fa6e80ceac0 2023-08-09T13:48:36.191 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:36.187+0000 7fa6edba8640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa6e80ca880 0x7fa6e81f25e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:48:36.191 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:36.187+0000 7fa6edba8640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa6e80ca880 0x7fa6e81f25e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:40986/0 (socket says 172.21.15.43:40986) 2023-08-09T13:48:36.191 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:36.187+0000 7fa6edba8640 1 -- 172.21.15.43:0/3826368487 learned_addr learned my addr 172.21.15.43:0/3826368487 (peer_addr_for_me v2:172.21.15.43:0/0) 2023-08-09T13:48:36.192 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:36.187+0000 7fa6ee3a9640 1 --2- 172.21.15.43:0/3826368487 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fa6e806f990 0x7fa6e80c3320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:48:36.192 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:36.188+0000 7fa6eebaa640 1 --2- 172.21.15.43:0/3826368487 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fa6e80ceac0 0x7fa6e81f2b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:48:36.192 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:36.188+0000 7fa6edba8640 1 -- 172.21.15.43:0/3826368487 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fa6e80ceac0 msgr2=0x7fa6e81f2b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:48:36.192 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:36.188+0000 7fa6edba8640 1 --2- 172.21.15.43:0/3826368487 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fa6e80ceac0 0x7fa6e81f2b20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:48:36.192 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:36.188+0000 7fa6edba8640 1 -- 172.21.15.43:0/3826368487 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fa6e806f990 msgr2=0x7fa6e80c3320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:48:36.192 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:36.188+0000 7fa6edba8640 1 --2- 172.21.15.43:0/3826368487 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fa6e806f990 0x7fa6e80c3320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:48:36.192 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:36.188+0000 7fa6edba8640 1 -- 172.21.15.43:0/3826368487 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa6e0009b30 con 0x7fa6e80ca880 2023-08-09T13:48:36.192 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:36.188+0000 7fa6ee3a9640 1 --2- 172.21.15.43:0/3826368487 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fa6e806f990 0x7fa6e80c3320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-08-09T13:48:36.193 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:36.188+0000 7fa6edba8640 1 --2- 172.21.15.43:0/3826368487 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa6e80ca880 0x7fa6e81f25e0 secure :-1 s=READY pgs=891 cs=0 l=1 rev1=1 crypto rx=0x7fa6dc00c7f0 tx=0x7fa6dc00ccb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-09T13:48:36.193 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:36.188+0000 7fa6e6ffd640 1 -- 172.21.15.43:0/3826368487 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa6dc007820 con 0x7fa6e80ca880 2023-08-09T13:48:36.194 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:36.188+0000 7fa6e6ffd640 1 -- 172.21.15.43:0/3826368487 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa6dc007980 con 0x7fa6e80ca880 2023-08-09T13:48:36.194 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:36.188+0000 7fa6e6ffd640 1 -- 172.21.15.43:0/3826368487 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa6dc007cf0 con 0x7fa6e80ca880 2023-08-09T13:48:36.194 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:36.189+0000 7fa6f060d640 1 -- 172.21.15.43:0/3826368487 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa6e80b2890 con 0x7fa6e80ca880 2023-08-09T13:48:36.194 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:36.189+0000 7fa6f060d640 1 -- 172.21.15.43:0/3826368487 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa6e80b0910 con 0x7fa6e80ca880 2023-08-09T13:48:36.194 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:36.190+0000 7fa6f060d640 1 -- 172.21.15.43:0/3826368487 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa6e80739f0 con 0x7fa6e80ca880 2023-08-09T13:48:36.194 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:36.191+0000 7fa6e6ffd640 1 -- 172.21.15.43:0/3826368487 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 22) v1 ==== 88461+0+0 (secure 0 0 0) 0x7fa6dc0113f0 con 0x7fa6e80ca880 2023-08-09T13:48:36.195 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:36.192+0000 7fa6e6ffd640 1 --2- 172.21.15.43:0/3826368487 >> [v2:172.21.15.203:6832/2515481191,v1:172.21.15.203:6833/2515481191] conn(0x7fa6d406a1c0 0x7fa6d406c680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:48:36.195 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:36.192+0000 7fa6e6ffd640 1 -- 172.21.15.43:0/3826368487 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(75..75 src has 1..75) v4 ==== 6770+0+0 (secure 0 0 0) 0x7fa6dc088f40 con 0x7fa6e80ca880 2023-08-09T13:48:36.195 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:36.192+0000 7fa6ee3a9640 1 --2- 172.21.15.43:0/3826368487 >> [v2:172.21.15.203:6832/2515481191,v1:172.21.15.203:6833/2515481191] conn(0x7fa6d406a1c0 0x7fa6d406c680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:48:36.196 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:36.193+0000 7fa6ee3a9640 1 --2- 172.21.15.43:0/3826368487 >> [v2:172.21.15.203:6832/2515481191,v1:172.21.15.203:6833/2515481191] conn(0x7fa6d406a1c0 0x7fa6d406c680 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7fa6d8007fd0 tx=0x7fa6d8009040 comp rx=0 tx=0).ready entity=mgr.5444 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-09T13:48:36.199 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:36.197+0000 7fa6e6ffd640 1 -- 172.21.15.43:0/3826368487 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178066 (secure 0 0 0) 0x7fa6dc0540a0 con 0x7fa6e80ca880 2023-08-09T13:48:36.416 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:36.413+0000 7fa6f060d640 1 -- 172.21.15.43:0/3826368487 --> [v2:172.21.15.203:6832/2515481191,v1:172.21.15.203:6833/2515481191] -- mgr_command(tid 0: {"prefix": "fs snap-schedule list", "fs": "cephfs", "path": "snap_test_dir1", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7fa6e80b2470 con 0x7fa6d406a1c0 2023-08-09T13:48:36.433 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:36.430+0000 7fa6e6ffd640 1 -- 172.21.15.43:0/3826368487 <== mgr.5444 v2:172.21.15.203:6832/2515481191 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+76 (secure 0 0 0) 0x7fa6d80013e0 con 0x7fa6d406a1c0 2023-08-09T13:48:36.433 INFO:teuthology.orchestra.run.smithi043.stdout: 2023-08-09T13:48:36.434 INFO:teuthology.orchestra.run.smithi043.stdout:{"path": "snap_test_dir1", "schedule": ["1M", "2M"], "retention": [{}, {}]} 2023-08-09T13:48:36.436 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:36.433+0000 7fa6f060d640 1 -- 172.21.15.43:0/3826368487 >> [v2:172.21.15.203:6832/2515481191,v1:172.21.15.203:6833/2515481191] conn(0x7fa6d406a1c0 msgr2=0x7fa6d406c680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:48:36.436 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:36.433+0000 7fa6f060d640 1 --2- 172.21.15.43:0/3826368487 >> [v2:172.21.15.203:6832/2515481191,v1:172.21.15.203:6833/2515481191] conn(0x7fa6d406a1c0 0x7fa6d406c680 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7fa6d8007fd0 tx=0x7fa6d8009040 comp rx=0 tx=0).stop 2023-08-09T13:48:36.437 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:36.433+0000 7fa6f060d640 1 -- 172.21.15.43:0/3826368487 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa6e80ca880 msgr2=0x7fa6e81f25e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:48:36.437 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:36.433+0000 7fa6f060d640 1 --2- 172.21.15.43:0/3826368487 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa6e80ca880 0x7fa6e81f25e0 secure :-1 s=READY pgs=891 cs=0 l=1 rev1=1 crypto rx=0x7fa6dc00c7f0 tx=0x7fa6dc00ccb0 comp rx=0 tx=0).stop 2023-08-09T13:48:36.437 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:36.433+0000 7fa6f060d640 1 -- 172.21.15.43:0/3826368487 shutdown_connections 2023-08-09T13:48:36.438 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:36.433+0000 7fa6f060d640 1 --2- 172.21.15.43:0/3826368487 >> [v2:172.21.15.203:6832/2515481191,v1:172.21.15.203:6833/2515481191] conn(0x7fa6d406a1c0 0x7fa6d406c680 unknown :-1 s=CLOSED pgs=24 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:48:36.438 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:36.433+0000 7fa6f060d640 1 --2- 172.21.15.43:0/3826368487 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fa6e80ceac0 0x7fa6e81f2b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:48:36.438 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:36.433+0000 7fa6f060d640 1 --2- 172.21.15.43:0/3826368487 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa6e80ca880 0x7fa6e81f25e0 unknown :-1 s=CLOSED pgs=891 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:48:36.438 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:36.433+0000 7fa6f060d640 1 --2- 172.21.15.43:0/3826368487 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fa6e806f990 0x7fa6e80c3320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:48:36.439 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:36.433+0000 7fa6f060d640 1 -- 172.21.15.43:0/3826368487 >> 172.21.15.43:0/3826368487 conn(0x7fa6e806b9a0 msgr2=0x7fa6e80cc180 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-09T13:48:36.439 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:36.433+0000 7fa6f060d640 1 -- 172.21.15.43:0/3826368487 shutdown_connections 2023-08-09T13:48:36.439 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:48:36.433+0000 7fa6f060d640 1 -- 172.21.15.43:0/3826368487 wait complete. 2023-08-09T13:48:36.454 DEBUG:tasks.cephfs.test_snap_schedules:res= {"path": "snap_test_dir1", "schedule": ["1M", "2M"], "retention": [{}, {}]} 2023-08-09T13:48:36.454 DEBUG:tasks.cephfs.test_snap_schedules:json_res: {'path': 'snap_test_dir1', 'schedule': ['1M', '2M'], 'retention': [{}, {}]} 2023-08-09T13:48:36.455 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-09T13:48:36.455 DEBUG:teuthology.orchestra.run.smithi043:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-08-09T13:48:36.528 INFO:tasks.cephfs.test_snap_schedules:snapshots: set() 2023-08-09T13:48:36.528 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-08-09T13:48:36.528 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-08-09T13:48:37.530 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-09T13:48:37.531 DEBUG:teuthology.orchestra.run.smithi043:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-08-09T13:48:37.610 INFO:tasks.cephfs.test_snap_schedules:snapshots: set() 2023-08-09T13:48:37.611 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-08-09T13:48:37.611 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-08-09T13:48:38.612 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-09T13:48:38.613 DEBUG:teuthology.orchestra.run.smithi043:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-08-09T13:48:38.693 INFO:tasks.cephfs.test_snap_schedules:snapshots: set() 2023-08-09T13:48:38.693 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-08-09T13:48:38.694 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-08-09T13:48:39.695 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-09T13:48:39.696 DEBUG:teuthology.orchestra.run.smithi043:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-08-09T13:48:39.777 INFO:tasks.cephfs.test_snap_schedules:snapshots: set() 2023-08-09T13:48:39.778 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-08-09T13:48:39.779 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-08-09T13:48:40.781 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-09T13:48:40.782 DEBUG:teuthology.orchestra.run.smithi043:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-08-09T13:48:40.862 INFO:tasks.cephfs.test_snap_schedules:snapshots: set() 2023-08-09T13:48:40.863 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-08-09T13:48:40.863 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-08-09T13:48:41.864 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-09T13:48:41.865 DEBUG:teuthology.orchestra.run.smithi043:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-08-09T13:48:41.945 INFO:tasks.cephfs.test_snap_schedules:snapshots: set() 2023-08-09T13:48:41.945 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-08-09T13:48:41.945 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-08-09T13:48:42.946 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-09T13:48:42.947 DEBUG:teuthology.orchestra.run.smithi043:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-08-09T13:48:43.024 INFO:tasks.cephfs.test_snap_schedules:snapshots: set() 2023-08-09T13:48:43.024 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-08-09T13:48:43.025 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-08-09T13:48:44.026 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-09T13:48:44.027 DEBUG:teuthology.orchestra.run.smithi043:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-08-09T13:48:44.104 INFO:tasks.cephfs.test_snap_schedules:snapshots: set() 2023-08-09T13:48:44.105 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-08-09T13:48:44.105 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-08-09T13:48:45.106 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-09T13:48:45.106 DEBUG:teuthology.orchestra.run.smithi043:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-08-09T13:48:45.184 INFO:tasks.cephfs.test_snap_schedules:snapshots: set() 2023-08-09T13:48:45.184 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-08-09T13:48:45.184 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-08-09T13:48:46.186 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-09T13:48:46.186 DEBUG:teuthology.orchestra.run.smithi043:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-08-09T13:48:46.268 INFO:tasks.cephfs.test_snap_schedules:snapshots: set() 2023-08-09T13:48:46.268 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-08-09T13:48:46.269 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-08-09T13:48:47.269 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-09T13:48:47.270 DEBUG:teuthology.orchestra.run.smithi043:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-08-09T13:48:47.347 INFO:tasks.cephfs.test_snap_schedules:snapshots: set() 2023-08-09T13:48:47.347 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-08-09T13:48:47.347 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-08-09T13:48:48.349 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-09T13:48:48.349 DEBUG:teuthology.orchestra.run.smithi043:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-08-09T13:48:48.430 INFO:tasks.cephfs.test_snap_schedules:snapshots: set() 2023-08-09T13:48:48.431 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-08-09T13:48:48.431 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-08-09T13:48:49.432 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-09T13:48:49.433 DEBUG:teuthology.orchestra.run.smithi043:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-08-09T13:48:49.512 INFO:tasks.cephfs.test_snap_schedules:snapshots: set() 2023-08-09T13:48:49.513 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-08-09T13:48:49.513 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-08-09T13:48:50.514 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-09T13:48:50.515 DEBUG:teuthology.orchestra.run.smithi043:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-08-09T13:48:50.594 INFO:tasks.cephfs.test_snap_schedules:snapshots: set() 2023-08-09T13:48:50.595 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-08-09T13:48:50.595 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-08-09T13:48:51.595 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-09T13:48:51.596 DEBUG:teuthology.orchestra.run.smithi043:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-08-09T13:48:51.677 INFO:tasks.cephfs.test_snap_schedules:snapshots: set() 2023-08-09T13:48:51.677 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-08-09T13:48:51.678 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-08-09T13:48:52.679 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-09T13:48:52.679 DEBUG:teuthology.orchestra.run.smithi043:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-08-09T13:48:52.761 INFO:tasks.cephfs.test_snap_schedules:snapshots: set() 2023-08-09T13:48:52.761 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-08-09T13:48:52.762 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-08-09T13:48:53.762 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-09T13:48:53.763 DEBUG:teuthology.orchestra.run.smithi043:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-08-09T13:48:53.845 INFO:tasks.cephfs.test_snap_schedules:snapshots: set() 2023-08-09T13:48:53.846 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-08-09T13:48:53.847 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-08-09T13:48:54.848 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-09T13:48:54.849 DEBUG:teuthology.orchestra.run.smithi043:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-08-09T13:48:54.924 INFO:tasks.cephfs.test_snap_schedules:snapshots: set() 2023-08-09T13:48:54.925 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-08-09T13:48:54.925 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-08-09T13:48:55.926 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-09T13:48:55.926 DEBUG:teuthology.orchestra.run.smithi043:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-08-09T13:48:56.004 INFO:tasks.cephfs.test_snap_schedules:snapshots: set() 2023-08-09T13:48:56.005 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-08-09T13:48:56.005 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-08-09T13:48:57.006 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-09T13:48:57.006 DEBUG:teuthology.orchestra.run.smithi043:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-08-09T13:48:57.085 INFO:tasks.cephfs.test_snap_schedules:snapshots: set() 2023-08-09T13:48:57.086 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-08-09T13:48:57.086 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-08-09T13:48:58.086 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-09T13:48:58.087 DEBUG:teuthology.orchestra.run.smithi043:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-08-09T13:48:58.167 INFO:tasks.cephfs.test_snap_schedules:snapshots: set() 2023-08-09T13:48:58.167 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-08-09T13:48:58.167 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-08-09T13:48:59.168 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-09T13:48:59.169 DEBUG:teuthology.orchestra.run.smithi043:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-08-09T13:48:59.249 INFO:tasks.cephfs.test_snap_schedules:snapshots: set() 2023-08-09T13:48:59.249 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-08-09T13:48:59.250 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-08-09T13:49:00.251 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-09T13:49:00.252 DEBUG:teuthology.orchestra.run.smithi043:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-08-09T13:49:00.329 INFO:tasks.cephfs.test_snap_schedules:snapshots: set() 2023-08-09T13:49:00.329 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-08-09T13:49:00.330 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-08-09T13:49:00.935 INFO:tasks.ceph.mgr.x.smithi203.stderr:2023-08-09T13:49:00.933+0000 7f4c9620a640 -1 client.0 error registering admin socket command: (17) File exists 2023-08-09T13:49:00.936 INFO:tasks.ceph.mgr.x.smithi203.stderr:2023-08-09T13:49:00.933+0000 7f4c9620a640 -1 client.0 error registering admin socket command: (17) File exists 2023-08-09T13:49:00.936 INFO:tasks.ceph.mgr.x.smithi203.stderr:2023-08-09T13:49:00.933+0000 7f4c9620a640 -1 client.0 error registering admin socket command: (17) File exists 2023-08-09T13:49:00.936 INFO:tasks.ceph.mgr.x.smithi203.stderr:2023-08-09T13:49:00.933+0000 7f4c9620a640 -1 client.0 error registering admin socket command: (17) File exists 2023-08-09T13:49:00.936 INFO:tasks.ceph.mgr.x.smithi203.stderr:2023-08-09T13:49:00.933+0000 7f4c9620a640 -1 client.0 error registering admin socket command: (17) File exists 2023-08-09T13:49:01.331 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-09T13:49:01.331 DEBUG:teuthology.orchestra.run.smithi043:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-08-09T13:49:01.413 INFO:teuthology.orchestra.run.smithi043.stdout:scheduled-2023-08-09-13_49_00_UTC 2023-08-09T13:49:01.414 INFO:tasks.cephfs.test_snap_schedules:snapshots: {'scheduled-2023-08-09-13_49_00_UTC'} 2023-08-09T13:49:01.415 INFO:tasks.cephfs.test_snap_schedules:added: {'scheduled-2023-08-09-13_49_00_UTC'} 2023-08-09T13:49:01.415 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-08-09T13:49:01.415 DEBUG:tasks.cephfs.test_snap_schedules:snapshots added=['scheduled-2023-08-09-13_49_00_UTC'] 2023-08-09T13:49:01.416 DEBUG:tasks.cephfs.test_snap_schedules:exec=1691588915.9392138, now = 1691588941.4161718, timo = 25 2023-08-09T13:49:02.417 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-09T13:49:02.417 DEBUG:teuthology.orchestra.run.smithi043:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-08-09T13:49:02.495 INFO:teuthology.orchestra.run.smithi043.stdout:scheduled-2023-08-09-13_49_00_UTC 2023-08-09T13:49:02.496 INFO:tasks.cephfs.test_snap_schedules:snapshots: {'scheduled-2023-08-09-13_49_00_UTC'} 2023-08-09T13:49:02.496 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-08-09T13:49:02.496 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-08-09T13:49:03.221 DEBUG:teuthology.orchestra.run.smithi043:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-08-09T13:49:03.231 DEBUG:teuthology.orchestra.run.smithi203:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-08-09T13:49:03.497 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-09T13:49:03.498 DEBUG:teuthology.orchestra.run.smithi043:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-08-09T13:49:03.578 INFO:teuthology.orchestra.run.smithi043.stdout:scheduled-2023-08-09-13_49_00_UTC 2023-08-09T13:49:03.580 INFO:tasks.cephfs.test_snap_schedules:snapshots: {'scheduled-2023-08-09-13_49_00_UTC'} 2023-08-09T13:49:03.580 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-08-09T13:49:03.580 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-08-09T13:49:04.581 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-09T13:49:04.582 DEBUG:teuthology.orchestra.run.smithi043:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-08-09T13:49:04.659 INFO:teuthology.orchestra.run.smithi043.stdout:scheduled-2023-08-09-13_49_00_UTC 2023-08-09T13:49:04.660 INFO:tasks.cephfs.test_snap_schedules:snapshots: {'scheduled-2023-08-09-13_49_00_UTC'} 2023-08-09T13:49:04.661 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-08-09T13:49:04.661 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-08-09T13:49:05.662 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-09T13:49:05.663 DEBUG:teuthology.orchestra.run.smithi043:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-08-09T13:49:05.741 INFO:teuthology.orchestra.run.smithi043.stdout:scheduled-2023-08-09-13_49_00_UTC 2023-08-09T13:49:05.742 INFO:tasks.cephfs.test_snap_schedules:snapshots: {'scheduled-2023-08-09-13_49_00_UTC'} 2023-08-09T13:49:05.743 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-08-09T13:49:05.743 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-08-09T13:49:06.744 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-09T13:49:06.745 DEBUG:teuthology.orchestra.run.smithi043:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-08-09T13:49:06.824 INFO:teuthology.orchestra.run.smithi043.stdout:scheduled-2023-08-09-13_49_00_UTC 2023-08-09T13:49:06.825 INFO:tasks.cephfs.test_snap_schedules:snapshots: {'scheduled-2023-08-09-13_49_00_UTC'} 2023-08-09T13:49:06.825 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-08-09T13:49:06.825 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-08-09T13:49:07.826 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-09T13:49:07.827 DEBUG:teuthology.orchestra.run.smithi043:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-08-09T13:49:07.906 INFO:teuthology.orchestra.run.smithi043.stdout:scheduled-2023-08-09-13_49_00_UTC 2023-08-09T13:49:07.907 INFO:tasks.cephfs.test_snap_schedules:snapshots: {'scheduled-2023-08-09-13_49_00_UTC'} 2023-08-09T13:49:07.908 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-08-09T13:49:07.908 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-08-09T13:49:08.909 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-09T13:49:08.910 DEBUG:teuthology.orchestra.run.smithi043:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-08-09T13:49:08.988 INFO:teuthology.orchestra.run.smithi043.stdout:scheduled-2023-08-09-13_49_00_UTC 2023-08-09T13:49:08.989 INFO:tasks.cephfs.test_snap_schedules:snapshots: {'scheduled-2023-08-09-13_49_00_UTC'} 2023-08-09T13:49:08.990 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-08-09T13:49:08.990 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-08-09T13:49:09.991 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-09T13:49:09.992 DEBUG:teuthology.orchestra.run.smithi043:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-08-09T13:49:10.071 INFO:teuthology.orchestra.run.smithi043.stdout:scheduled-2023-08-09-13_49_00_UTC 2023-08-09T13:49:10.072 INFO:tasks.cephfs.test_snap_schedules:snapshots: {'scheduled-2023-08-09-13_49_00_UTC'} 2023-08-09T13:49:10.072 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-08-09T13:49:10.073 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-08-09T13:49:11.073 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-09T13:49:11.074 DEBUG:teuthology.orchestra.run.smithi043:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-08-09T13:49:11.151 INFO:teuthology.orchestra.run.smithi043.stdout:scheduled-2023-08-09-13_49_00_UTC 2023-08-09T13:49:11.152 INFO:tasks.cephfs.test_snap_schedules:snapshots: {'scheduled-2023-08-09-13_49_00_UTC'} 2023-08-09T13:49:11.153 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-08-09T13:49:11.153 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-08-09T13:49:12.154 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-09T13:49:12.155 DEBUG:teuthology.orchestra.run.smithi043:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-08-09T13:49:12.235 INFO:teuthology.orchestra.run.smithi043.stdout:scheduled-2023-08-09-13_49_00_UTC 2023-08-09T13:49:12.237 INFO:tasks.cephfs.test_snap_schedules:snapshots: {'scheduled-2023-08-09-13_49_00_UTC'} 2023-08-09T13:49:12.237 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-08-09T13:49:12.238 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-08-09T13:49:13.239 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-09T13:49:13.239 DEBUG:teuthology.orchestra.run.smithi043:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-08-09T13:49:13.318 INFO:teuthology.orchestra.run.smithi043.stdout:scheduled-2023-08-09-13_49_00_UTC 2023-08-09T13:49:13.319 INFO:tasks.cephfs.test_snap_schedules:snapshots: {'scheduled-2023-08-09-13_49_00_UTC'} 2023-08-09T13:49:13.319 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-08-09T13:49:13.319 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-08-09T13:49:14.321 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-09T13:49:14.321 DEBUG:teuthology.orchestra.run.smithi043:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-08-09T13:49:14.398 INFO:teuthology.orchestra.run.smithi043.stdout:scheduled-2023-08-09-13_49_00_UTC 2023-08-09T13:49:14.399 INFO:tasks.cephfs.test_snap_schedules:snapshots: {'scheduled-2023-08-09-13_49_00_UTC'} 2023-08-09T13:49:14.400 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-08-09T13:49:14.400 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-08-09T13:49:15.401 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-09T13:49:15.401 DEBUG:teuthology.orchestra.run.smithi043:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-08-09T13:49:15.479 INFO:teuthology.orchestra.run.smithi043.stdout:scheduled-2023-08-09-13_49_00_UTC 2023-08-09T13:49:15.481 INFO:tasks.cephfs.test_snap_schedules:snapshots: {'scheduled-2023-08-09-13_49_00_UTC'} 2023-08-09T13:49:15.481 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-08-09T13:49:15.481 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-08-09T13:49:16.482 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-09T13:49:16.483 DEBUG:teuthology.orchestra.run.smithi043:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-08-09T13:49:16.559 INFO:teuthology.orchestra.run.smithi043.stdout:scheduled-2023-08-09-13_49_00_UTC 2023-08-09T13:49:16.560 INFO:tasks.cephfs.test_snap_schedules:snapshots: {'scheduled-2023-08-09-13_49_00_UTC'} 2023-08-09T13:49:16.561 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-08-09T13:49:16.561 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-08-09T13:49:17.562 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-09T13:49:17.562 DEBUG:teuthology.orchestra.run.smithi043:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-08-09T13:49:17.640 INFO:teuthology.orchestra.run.smithi043.stdout:scheduled-2023-08-09-13_49_00_UTC 2023-08-09T13:49:17.642 INFO:tasks.cephfs.test_snap_schedules:snapshots: {'scheduled-2023-08-09-13_49_00_UTC'} 2023-08-09T13:49:17.642 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-08-09T13:49:17.642 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-08-09T13:49:18.644 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-09T13:49:18.644 DEBUG:teuthology.orchestra.run.smithi043:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-08-09T13:49:18.724 INFO:teuthology.orchestra.run.smithi043.stdout:scheduled-2023-08-09-13_49_00_UTC 2023-08-09T13:49:18.726 INFO:tasks.cephfs.test_snap_schedules:snapshots: {'scheduled-2023-08-09-13_49_00_UTC'} 2023-08-09T13:49:18.726 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-08-09T13:49:18.726 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-08-09T13:49:19.727 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-09T13:49:19.728 DEBUG:teuthology.orchestra.run.smithi043:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-08-09T13:49:19.807 INFO:teuthology.orchestra.run.smithi043.stdout:scheduled-2023-08-09-13_49_00_UTC 2023-08-09T13:49:19.809 INFO:tasks.cephfs.test_snap_schedules:snapshots: {'scheduled-2023-08-09-13_49_00_UTC'} 2023-08-09T13:49:19.810 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-08-09T13:49:19.810 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-08-09T13:49:20.811 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-09T13:49:20.811 DEBUG:teuthology.orchestra.run.smithi043:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-08-09T13:49:20.890 INFO:teuthology.orchestra.run.smithi043.stdout:scheduled-2023-08-09-13_49_00_UTC 2023-08-09T13:49:20.891 INFO:tasks.cephfs.test_snap_schedules:snapshots: {'scheduled-2023-08-09-13_49_00_UTC'} 2023-08-09T13:49:20.892 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-08-09T13:49:20.892 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-08-09T13:49:21.893 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-09T13:49:21.894 DEBUG:teuthology.orchestra.run.smithi043:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-08-09T13:49:21.969 INFO:teuthology.orchestra.run.smithi043.stdout:scheduled-2023-08-09-13_49_00_UTC 2023-08-09T13:49:21.970 INFO:tasks.cephfs.test_snap_schedules:snapshots: {'scheduled-2023-08-09-13_49_00_UTC'} 2023-08-09T13:49:21.971 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-08-09T13:49:21.971 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-08-09T13:49:22.972 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-09T13:49:22.973 DEBUG:teuthology.orchestra.run.smithi043:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-08-09T13:49:23.049 INFO:teuthology.orchestra.run.smithi043.stdout:scheduled-2023-08-09-13_49_00_UTC 2023-08-09T13:49:23.051 INFO:tasks.cephfs.test_snap_schedules:snapshots: {'scheduled-2023-08-09-13_49_00_UTC'} 2023-08-09T13:49:23.051 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-08-09T13:49:23.052 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-08-09T13:49:24.053 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-09T13:49:24.054 DEBUG:teuthology.orchestra.run.smithi043:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-08-09T13:49:24.129 INFO:teuthology.orchestra.run.smithi043.stdout:scheduled-2023-08-09-13_49_00_UTC 2023-08-09T13:49:24.130 INFO:tasks.cephfs.test_snap_schedules:snapshots: {'scheduled-2023-08-09-13_49_00_UTC'} 2023-08-09T13:49:24.130 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-08-09T13:49:24.131 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-08-09T13:49:25.131 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-09T13:49:25.132 DEBUG:teuthology.orchestra.run.smithi043:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-08-09T13:49:25.207 INFO:teuthology.orchestra.run.smithi043.stdout:scheduled-2023-08-09-13_49_00_UTC 2023-08-09T13:49:25.208 INFO:tasks.cephfs.test_snap_schedules:snapshots: {'scheduled-2023-08-09-13_49_00_UTC'} 2023-08-09T13:49:25.208 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-08-09T13:49:25.209 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-08-09T13:49:26.209 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-09T13:49:26.210 DEBUG:teuthology.orchestra.run.smithi043:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-08-09T13:49:26.285 INFO:teuthology.orchestra.run.smithi043.stdout:scheduled-2023-08-09-13_49_00_UTC 2023-08-09T13:49:26.287 INFO:tasks.cephfs.test_snap_schedules:snapshots: {'scheduled-2023-08-09-13_49_00_UTC'} 2023-08-09T13:49:26.287 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-08-09T13:49:26.287 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-08-09T13:49:27.288 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-09T13:49:27.289 DEBUG:teuthology.orchestra.run.smithi043:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-08-09T13:49:27.366 INFO:teuthology.orchestra.run.smithi043.stdout:scheduled-2023-08-09-13_49_00_UTC 2023-08-09T13:49:27.368 INFO:tasks.cephfs.test_snap_schedules:snapshots: {'scheduled-2023-08-09-13_49_00_UTC'} 2023-08-09T13:49:27.368 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-08-09T13:49:27.369 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-08-09T13:49:28.370 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-09T13:49:28.371 DEBUG:teuthology.orchestra.run.smithi043:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-08-09T13:49:28.447 INFO:teuthology.orchestra.run.smithi043.stdout:scheduled-2023-08-09-13_49_00_UTC 2023-08-09T13:49:28.448 INFO:tasks.cephfs.test_snap_schedules:snapshots: {'scheduled-2023-08-09-13_49_00_UTC'} 2023-08-09T13:49:28.449 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-08-09T13:49:28.449 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-08-09T13:49:29.449 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-09T13:49:29.450 DEBUG:teuthology.orchestra.run.smithi043:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-08-09T13:49:29.527 INFO:teuthology.orchestra.run.smithi043.stdout:scheduled-2023-08-09-13_49_00_UTC 2023-08-09T13:49:29.529 INFO:tasks.cephfs.test_snap_schedules:snapshots: {'scheduled-2023-08-09-13_49_00_UTC'} 2023-08-09T13:49:29.530 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-08-09T13:49:29.530 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-08-09T13:49:30.531 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-09T13:49:30.532 DEBUG:teuthology.orchestra.run.smithi043:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-08-09T13:49:30.610 INFO:teuthology.orchestra.run.smithi043.stdout:scheduled-2023-08-09-13_49_00_UTC 2023-08-09T13:49:30.611 INFO:tasks.cephfs.test_snap_schedules:snapshots: {'scheduled-2023-08-09-13_49_00_UTC'} 2023-08-09T13:49:30.612 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-08-09T13:49:30.612 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-08-09T13:49:31.613 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-09T13:49:31.613 DEBUG:teuthology.orchestra.run.smithi043:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-08-09T13:49:31.690 INFO:teuthology.orchestra.run.smithi043.stdout:scheduled-2023-08-09-13_49_00_UTC 2023-08-09T13:49:31.691 INFO:tasks.cephfs.test_snap_schedules:snapshots: {'scheduled-2023-08-09-13_49_00_UTC'} 2023-08-09T13:49:31.692 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-08-09T13:49:31.692 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-08-09T13:49:32.693 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-09T13:49:32.694 DEBUG:teuthology.orchestra.run.smithi043:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-08-09T13:49:32.771 INFO:teuthology.orchestra.run.smithi043.stdout:scheduled-2023-08-09-13_49_00_UTC 2023-08-09T13:49:32.773 INFO:tasks.cephfs.test_snap_schedules:snapshots: {'scheduled-2023-08-09-13_49_00_UTC'} 2023-08-09T13:49:32.773 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-08-09T13:49:32.774 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-08-09T13:49:33.384 DEBUG:teuthology.orchestra.run.smithi043:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-08-09T13:49:33.394 DEBUG:teuthology.orchestra.run.smithi203:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-08-09T13:49:33.775 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-09T13:49:33.776 DEBUG:teuthology.orchestra.run.smithi043:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-08-09T13:49:33.852 INFO:teuthology.orchestra.run.smithi043.stdout:scheduled-2023-08-09-13_49_00_UTC 2023-08-09T13:49:33.853 INFO:tasks.cephfs.test_snap_schedules:snapshots: {'scheduled-2023-08-09-13_49_00_UTC'} 2023-08-09T13:49:33.854 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-08-09T13:49:33.854 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-08-09T13:49:34.855 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-09T13:49:34.856 DEBUG:teuthology.orchestra.run.smithi043:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-08-09T13:49:34.931 INFO:teuthology.orchestra.run.smithi043.stdout:scheduled-2023-08-09-13_49_00_UTC 2023-08-09T13:49:34.932 INFO:tasks.cephfs.test_snap_schedules:snapshots: {'scheduled-2023-08-09-13_49_00_UTC'} 2023-08-09T13:49:34.933 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-08-09T13:49:34.933 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-08-09T13:49:35.934 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-09T13:49:35.935 DEBUG:teuthology.orchestra.run.smithi043:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-08-09T13:49:36.010 INFO:teuthology.orchestra.run.smithi043.stdout:scheduled-2023-08-09-13_49_00_UTC 2023-08-09T13:49:36.012 INFO:tasks.cephfs.test_snap_schedules:snapshots: {'scheduled-2023-08-09-13_49_00_UTC'} 2023-08-09T13:49:36.013 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-08-09T13:49:36.013 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-08-09T13:49:37.015 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-09T13:49:37.015 DEBUG:teuthology.orchestra.run.smithi043:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-08-09T13:49:37.085 INFO:teuthology.orchestra.run.smithi043.stdout:scheduled-2023-08-09-13_49_00_UTC 2023-08-09T13:49:37.086 INFO:tasks.cephfs.test_snap_schedules:snapshots: {'scheduled-2023-08-09-13_49_00_UTC'} 2023-08-09T13:49:37.086 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-08-09T13:49:37.087 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-08-09T13:49:38.088 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-09T13:49:38.088 DEBUG:teuthology.orchestra.run.smithi043:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-08-09T13:49:38.166 INFO:teuthology.orchestra.run.smithi043.stdout:scheduled-2023-08-09-13_49_00_UTC 2023-08-09T13:49:38.168 INFO:tasks.cephfs.test_snap_schedules:snapshots: {'scheduled-2023-08-09-13_49_00_UTC'} 2023-08-09T13:49:38.168 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-08-09T13:49:38.169 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-08-09T13:49:39.169 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-09T13:49:39.170 DEBUG:teuthology.orchestra.run.smithi043:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-08-09T13:49:39.246 INFO:teuthology.orchestra.run.smithi043.stdout:scheduled-2023-08-09-13_49_00_UTC 2023-08-09T13:49:39.247 INFO:tasks.cephfs.test_snap_schedules:snapshots: {'scheduled-2023-08-09-13_49_00_UTC'} 2023-08-09T13:49:39.247 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-08-09T13:49:39.248 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-08-09T13:49:40.249 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-09T13:49:40.249 DEBUG:teuthology.orchestra.run.smithi043:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-08-09T13:49:40.326 INFO:teuthology.orchestra.run.smithi043.stdout:scheduled-2023-08-09-13_49_00_UTC 2023-08-09T13:49:40.327 INFO:tasks.cephfs.test_snap_schedules:snapshots: {'scheduled-2023-08-09-13_49_00_UTC'} 2023-08-09T13:49:40.327 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-08-09T13:49:40.328 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-08-09T13:49:41.328 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-09T13:49:41.329 DEBUG:teuthology.orchestra.run.smithi043:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-08-09T13:49:41.403 INFO:teuthology.orchestra.run.smithi043.stdout:scheduled-2023-08-09-13_49_00_UTC 2023-08-09T13:49:41.405 INFO:tasks.cephfs.test_snap_schedules:snapshots: {'scheduled-2023-08-09-13_49_00_UTC'} 2023-08-09T13:49:41.405 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-08-09T13:49:41.405 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-08-09T13:49:42.406 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-09T13:49:42.407 DEBUG:teuthology.orchestra.run.smithi043:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-08-09T13:49:42.484 INFO:teuthology.orchestra.run.smithi043.stdout:scheduled-2023-08-09-13_49_00_UTC 2023-08-09T13:49:42.486 INFO:tasks.cephfs.test_snap_schedules:snapshots: {'scheduled-2023-08-09-13_49_00_UTC'} 2023-08-09T13:49:42.487 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-08-09T13:49:42.487 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-08-09T13:49:43.488 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-09T13:49:43.488 DEBUG:teuthology.orchestra.run.smithi043:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-08-09T13:49:43.565 INFO:teuthology.orchestra.run.smithi043.stdout:scheduled-2023-08-09-13_49_00_UTC 2023-08-09T13:49:43.566 INFO:tasks.cephfs.test_snap_schedules:snapshots: {'scheduled-2023-08-09-13_49_00_UTC'} 2023-08-09T13:49:43.566 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-08-09T13:49:43.567 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-08-09T13:49:44.568 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-09T13:49:44.568 DEBUG:teuthology.orchestra.run.smithi043:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-08-09T13:49:44.644 INFO:teuthology.orchestra.run.smithi043.stdout:scheduled-2023-08-09-13_49_00_UTC 2023-08-09T13:49:44.645 INFO:tasks.cephfs.test_snap_schedules:snapshots: {'scheduled-2023-08-09-13_49_00_UTC'} 2023-08-09T13:49:44.646 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-08-09T13:49:44.646 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-08-09T13:49:45.646 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-09T13:49:45.647 DEBUG:teuthology.orchestra.run.smithi043:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-08-09T13:49:45.722 INFO:teuthology.orchestra.run.smithi043.stdout:scheduled-2023-08-09-13_49_00_UTC 2023-08-09T13:49:45.724 INFO:tasks.cephfs.test_snap_schedules:snapshots: {'scheduled-2023-08-09-13_49_00_UTC'} 2023-08-09T13:49:45.724 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-08-09T13:49:45.724 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-08-09T13:49:46.725 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-09T13:49:46.726 DEBUG:teuthology.orchestra.run.smithi043:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-08-09T13:49:46.802 INFO:teuthology.orchestra.run.smithi043.stdout:scheduled-2023-08-09-13_49_00_UTC 2023-08-09T13:49:46.803 INFO:tasks.cephfs.test_snap_schedules:snapshots: {'scheduled-2023-08-09-13_49_00_UTC'} 2023-08-09T13:49:46.803 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-08-09T13:49:46.803 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-08-09T13:49:47.804 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-09T13:49:47.805 DEBUG:teuthology.orchestra.run.smithi043:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-08-09T13:49:47.882 INFO:teuthology.orchestra.run.smithi043.stdout:scheduled-2023-08-09-13_49_00_UTC 2023-08-09T13:49:47.883 INFO:tasks.cephfs.test_snap_schedules:snapshots: {'scheduled-2023-08-09-13_49_00_UTC'} 2023-08-09T13:49:47.883 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-08-09T13:49:47.884 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-08-09T13:49:48.885 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-09T13:49:48.885 DEBUG:teuthology.orchestra.run.smithi043:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-08-09T13:49:48.964 INFO:teuthology.orchestra.run.smithi043.stdout:scheduled-2023-08-09-13_49_00_UTC 2023-08-09T13:49:48.965 INFO:tasks.cephfs.test_snap_schedules:snapshots: {'scheduled-2023-08-09-13_49_00_UTC'} 2023-08-09T13:49:48.966 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-08-09T13:49:48.967 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-08-09T13:49:49.967 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-09T13:49:49.968 DEBUG:teuthology.orchestra.run.smithi043:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-08-09T13:49:50.047 INFO:teuthology.orchestra.run.smithi043.stdout:scheduled-2023-08-09-13_49_00_UTC 2023-08-09T13:49:50.049 INFO:tasks.cephfs.test_snap_schedules:snapshots: {'scheduled-2023-08-09-13_49_00_UTC'} 2023-08-09T13:49:50.049 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-08-09T13:49:50.049 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-08-09T13:49:51.051 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-09T13:49:51.051 DEBUG:teuthology.orchestra.run.smithi043:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-08-09T13:49:51.130 INFO:teuthology.orchestra.run.smithi043.stdout:scheduled-2023-08-09-13_49_00_UTC 2023-08-09T13:49:51.132 INFO:tasks.cephfs.test_snap_schedules:snapshots: {'scheduled-2023-08-09-13_49_00_UTC'} 2023-08-09T13:49:51.133 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-08-09T13:49:51.133 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-08-09T13:49:52.134 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-09T13:49:52.135 DEBUG:teuthology.orchestra.run.smithi043:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-08-09T13:49:52.211 INFO:teuthology.orchestra.run.smithi043.stdout:scheduled-2023-08-09-13_49_00_UTC 2023-08-09T13:49:52.212 INFO:tasks.cephfs.test_snap_schedules:snapshots: {'scheduled-2023-08-09-13_49_00_UTC'} 2023-08-09T13:49:52.213 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-08-09T13:49:52.213 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-08-09T13:49:53.214 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-09T13:49:53.215 DEBUG:teuthology.orchestra.run.smithi043:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-08-09T13:49:53.293 INFO:teuthology.orchestra.run.smithi043.stdout:scheduled-2023-08-09-13_49_00_UTC 2023-08-09T13:49:53.294 INFO:tasks.cephfs.test_snap_schedules:snapshots: {'scheduled-2023-08-09-13_49_00_UTC'} 2023-08-09T13:49:53.295 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-08-09T13:49:53.295 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-08-09T13:49:54.296 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-09T13:49:54.297 DEBUG:teuthology.orchestra.run.smithi043:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-08-09T13:49:54.373 INFO:teuthology.orchestra.run.smithi043.stdout:scheduled-2023-08-09-13_49_00_UTC 2023-08-09T13:49:54.375 INFO:tasks.cephfs.test_snap_schedules:snapshots: {'scheduled-2023-08-09-13_49_00_UTC'} 2023-08-09T13:49:54.375 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-08-09T13:49:54.375 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-08-09T13:49:55.376 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-09T13:49:55.377 DEBUG:teuthology.orchestra.run.smithi043:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-08-09T13:49:55.458 INFO:teuthology.orchestra.run.smithi043.stdout:scheduled-2023-08-09-13_49_00_UTC 2023-08-09T13:49:55.459 INFO:tasks.cephfs.test_snap_schedules:snapshots: {'scheduled-2023-08-09-13_49_00_UTC'} 2023-08-09T13:49:55.460 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-08-09T13:49:55.460 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-08-09T13:49:56.461 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-09T13:49:56.462 DEBUG:teuthology.orchestra.run.smithi043:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-08-09T13:49:56.540 INFO:teuthology.orchestra.run.smithi043.stdout:scheduled-2023-08-09-13_49_00_UTC 2023-08-09T13:49:56.541 INFO:tasks.cephfs.test_snap_schedules:snapshots: {'scheduled-2023-08-09-13_49_00_UTC'} 2023-08-09T13:49:56.541 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-08-09T13:49:56.542 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-08-09T13:49:57.543 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-09T13:49:57.543 DEBUG:teuthology.orchestra.run.smithi043:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-08-09T13:49:57.621 INFO:teuthology.orchestra.run.smithi043.stdout:scheduled-2023-08-09-13_49_00_UTC 2023-08-09T13:49:57.622 INFO:tasks.cephfs.test_snap_schedules:snapshots: {'scheduled-2023-08-09-13_49_00_UTC'} 2023-08-09T13:49:57.622 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-08-09T13:49:57.623 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-08-09T13:49:58.624 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-09T13:49:58.624 DEBUG:teuthology.orchestra.run.smithi043:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-08-09T13:49:58.699 INFO:teuthology.orchestra.run.smithi043.stdout:scheduled-2023-08-09-13_49_00_UTC 2023-08-09T13:49:58.700 INFO:tasks.cephfs.test_snap_schedules:snapshots: {'scheduled-2023-08-09-13_49_00_UTC'} 2023-08-09T13:49:58.700 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-08-09T13:49:58.701 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-08-09T13:49:59.702 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-09T13:49:59.703 DEBUG:teuthology.orchestra.run.smithi043:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-08-09T13:49:59.779 INFO:teuthology.orchestra.run.smithi043.stdout:scheduled-2023-08-09-13_49_00_UTC 2023-08-09T13:49:59.780 INFO:tasks.cephfs.test_snap_schedules:snapshots: {'scheduled-2023-08-09-13_49_00_UTC'} 2023-08-09T13:49:59.781 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-08-09T13:49:59.781 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-08-09T13:50:00.782 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-09T13:50:00.782 DEBUG:teuthology.orchestra.run.smithi043:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-08-09T13:50:00.861 INFO:teuthology.orchestra.run.smithi043.stdout:scheduled-2023-08-09-13_49_00_UTC 2023-08-09T13:50:00.862 INFO:teuthology.orchestra.run.smithi043.stdout:scheduled-2023-08-09-13_50_00_UTC 2023-08-09T13:50:00.863 INFO:tasks.cephfs.test_snap_schedules:snapshots: {'scheduled-2023-08-09-13_49_00_UTC', 'scheduled-2023-08-09-13_50_00_UTC'} 2023-08-09T13:50:00.863 INFO:tasks.cephfs.test_snap_schedules:added: {'scheduled-2023-08-09-13_50_00_UTC'} 2023-08-09T13:50:00.863 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-08-09T13:50:00.864 DEBUG:tasks.cephfs.test_snap_schedules:snapshots added=['scheduled-2023-08-09-13_50_00_UTC'] 2023-08-09T13:50:00.864 DEBUG:tasks.cephfs.test_snap_schedules:exec=1691588915.9392138, now = 1691589000.864297, timo = 85 2023-08-09T13:50:01.865 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs snap-schedule remove --fs cephfs snap_test_dir1 2023-08-09T13:50:02.128 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:02.126+0000 7f45d5b4f640 1 Processor -- start 2023-08-09T13:50:02.129 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:02.126+0000 7f45d5b4f640 1 -- start start 2023-08-09T13:50:02.130 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:02.127+0000 7f45d5b4f640 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7f45d00c99b0 0x7f45d015d3c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:50:02.130 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:02.127+0000 7f45d5b4f640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f45d006c510 0x7f45d006c960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:50:02.130 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:02.127+0000 7f45d5b4f640 1 -- --> v1:172.21.15.43:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f45d006cea0 con 0x7f45d00c86c0 2023-08-09T13:50:02.130 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:02.127+0000 7f45d5b4f640 1 -- --> v2:172.21.15.203:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f45d0157000 con 0x7f45d00c99b0 2023-08-09T13:50:02.131 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:02.128+0000 7f45d5b4f640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f45d0159c40 con 0x7f45d006c510 2023-08-09T13:50:02.131 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:02.128+0000 7f45cffff640 1 --1- >> v1:172.21.15.43:6789/0 conn(0x7f45d00c86c0 0x7f45d0159530 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.43:6789/0 says I am v1:172.21.15.43:48472/0 (socket says 172.21.15.43:48472) 2023-08-09T13:50:02.131 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:02.128+0000 7f45cffff640 1 -- 172.21.15.43:0/2503686424 learned_addr learned my addr 172.21.15.43:0/2503686424 (peer_addr_for_me v1:172.21.15.43:0/0) 2023-08-09T13:50:02.133 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:02.128+0000 7f45d4b4d640 1 --2- 172.21.15.43:0/2503686424 >> v2:172.21.15.203:3300/0 conn(0x7f45d00c99b0 0x7f45d015d3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:50:02.133 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:02.128+0000 7f45d534e640 1 --2- 172.21.15.43:0/2503686424 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f45d006c510 0x7f45d006c960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:50:02.133 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:02.128+0000 7f45cf7fe640 1 -- 172.21.15.43:0/2503686424 <== mon.0 v1:172.21.15.43:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1344630275 0 0) 0x7f45c0001110 con 0x7f45d00c86c0 2023-08-09T13:50:02.133 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:02.128+0000 7f45cf7fe640 1 -- 172.21.15.43:0/2503686424 --> v1:172.21.15.43:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f45b8005730 con 0x7f45d00c86c0 2023-08-09T13:50:02.133 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:02.129+0000 7f45cf7fe640 1 -- 172.21.15.43:0/2503686424 <== mon.0 v1:172.21.15.43:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1192889109 0 0) 0x7f45b8005730 con 0x7f45d00c86c0 2023-08-09T13:50:02.133 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:02.129+0000 7f45cf7fe640 1 -- 172.21.15.43:0/2503686424 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f45d006c510 msgr2=0x7f45d006c960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-09T13:50:02.134 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:02.129+0000 7f45cf7fe640 1 --2- 172.21.15.43:0/2503686424 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f45d006c510 0x7f45d006c960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:50:02.134 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:02.129+0000 7f45cf7fe640 1 -- 172.21.15.43:0/2503686424 >> v2:172.21.15.203:3300/0 conn(0x7f45d00c99b0 msgr2=0x7f45d015d3c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-09T13:50:02.134 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:02.129+0000 7f45cf7fe640 1 --2- 172.21.15.43:0/2503686424 >> v2:172.21.15.203:3300/0 conn(0x7f45d00c99b0 0x7f45d015d3c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:50:02.134 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:02.129+0000 7f45cf7fe640 1 -- 172.21.15.43:0/2503686424 --> v1:172.21.15.43:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f45d0159ea0 con 0x7f45d00c86c0 2023-08-09T13:50:02.134 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:02.129+0000 7f45cf7fe640 1 -- 172.21.15.43:0/2503686424 <== mon.0 v1:172.21.15.43:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1285150740 0 0) 0x7f45c0001800 con 0x7f45d00c86c0 2023-08-09T13:50:02.134 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:02.129+0000 7f45cf7fe640 1 -- 172.21.15.43:0/2503686424 >> v1:172.21.15.43:6789/0 conn(0x7f45d00c86c0 legacy=0x7f45d0159530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:50:02.134 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:02.129+0000 7f45cf7fe640 1 --2- 172.21.15.43:0/2503686424 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f45b8005b80 0x7f45b8005fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:50:02.135 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:02.129+0000 7f45cf7fe640 1 --2- 172.21.15.43:0/2503686424 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f45b8006570 0x7f45b8006a50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:50:02.135 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:02.129+0000 7f45cf7fe640 1 --2- 172.21.15.43:0/2503686424 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f45b8007170 0x7f45b800db90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:50:02.135 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:02.129+0000 7f45cf7fe640 1 -- 172.21.15.43:0/2503686424 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f45d0159c40 con 0x7f45b8006570 2023-08-09T13:50:02.135 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:02.129+0000 7f45cf7fe640 1 -- 172.21.15.43:0/2503686424 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f45b800e0d0 con 0x7f45b8007170 2023-08-09T13:50:02.135 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:02.129+0000 7f45cf7fe640 1 -- 172.21.15.43:0/2503686424 --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f45b800e240 con 0x7f45b8005b80 2023-08-09T13:50:02.135 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:02.129+0000 7f45d534e640 1 --2- 172.21.15.43:0/2503686424 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f45d006c510 0x7f45d006c960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-08-09T13:50:02.136 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:02.129+0000 7f45cffff640 1 --2- 172.21.15.43:0/2503686424 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f45b8006570 0x7f45b8006a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:50:02.136 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:02.130+0000 7f45d4b4d640 1 --2- 172.21.15.43:0/2503686424 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f45b8005b80 0x7f45b8005fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:50:02.136 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:02.130+0000 7f45d534e640 1 --2- 172.21.15.43:0/2503686424 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f45b8007170 0x7f45b800db90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:50:02.136 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:02.130+0000 7f45cffff640 1 -- 172.21.15.43:0/2503686424 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f45b8005b80 msgr2=0x7f45b8005fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-09T13:50:02.136 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:02.130+0000 7f45cffff640 1 --2- 172.21.15.43:0/2503686424 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f45b8005b80 0x7f45b8005fa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:50:02.136 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:02.130+0000 7f45cffff640 1 -- 172.21.15.43:0/2503686424 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f45b8007170 msgr2=0x7f45b800db90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-09T13:50:02.136 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:02.130+0000 7f45cffff640 1 --2- 172.21.15.43:0/2503686424 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f45b8007170 0x7f45b800db90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:50:02.137 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:02.130+0000 7f45cffff640 1 -- 172.21.15.43:0/2503686424 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f45b8005730 con 0x7f45b8006570 2023-08-09T13:50:02.138 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:02.132+0000 7f45cffff640 1 --2- 172.21.15.43:0/2503686424 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f45b8006570 0x7f45b8006a50 secure :-1 s=READY pgs=893 cs=0 l=1 rev1=1 crypto rx=0x7f45c000c210 tx=0x7f45b8007a00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cab27b13ec963030 server_cookie=0 in_seq=0 out_seq=0 2023-08-09T13:50:02.138 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:02.132+0000 7f45cf7fe640 1 -- 172.21.15.43:0/2503686424 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f45c002cae0 con 0x7f45b8006570 2023-08-09T13:50:02.138 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:02.132+0000 7f45cf7fe640 1 -- 172.21.15.43:0/2503686424 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f45c002cc40 con 0x7f45b8006570 2023-08-09T13:50:02.138 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:02.132+0000 7f45cf7fe640 1 -- 172.21.15.43:0/2503686424 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f45c0005b10 con 0x7f45b8006570 2023-08-09T13:50:02.138 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:02.133+0000 7f45d5b4f640 1 -- 172.21.15.43:0/2503686424 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f45b8006570 msgr2=0x7f45b8006a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:50:02.138 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:02.133+0000 7f45d5b4f640 1 --2- 172.21.15.43:0/2503686424 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f45b8006570 0x7f45b8006a50 secure :-1 s=READY pgs=893 cs=0 l=1 rev1=1 crypto rx=0x7f45c000c210 tx=0x7f45b8007a00 comp rx=0 tx=0).stop 2023-08-09T13:50:02.138 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:02.133+0000 7f45d534e640 1 -- 172.21.15.43:0/2503686424 reap_dead start 2023-08-09T13:50:02.139 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:02.133+0000 7f45d5b4f640 1 -- 172.21.15.43:0/2503686424 shutdown_connections 2023-08-09T13:50:02.139 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:02.133+0000 7f45d5b4f640 1 -- 172.21.15.43:0/2503686424 >> 172.21.15.43:0/2503686424 conn(0x7f45d00ca870 msgr2=0x7f45d00cac50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-09T13:50:02.139 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:02.133+0000 7f45d5b4f640 1 -- 172.21.15.43:0/2503686424 shutdown_connections 2023-08-09T13:50:02.139 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:02.133+0000 7f45d5b4f640 1 -- 172.21.15.43:0/2503686424 wait complete. 2023-08-09T13:50:02.139 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:02.135+0000 7f45d5b4f640 1 Processor -- start 2023-08-09T13:50:02.139 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:02.135+0000 7f45d5b4f640 1 -- start start 2023-08-09T13:50:02.141 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:02.136+0000 7f45d5b4f640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f45b8006570 0x7f45d01493e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:50:02.141 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:02.136+0000 7f45d5b4f640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f45d00c2580 0x7f45d00ac250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:50:02.141 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:02.136+0000 7f45d5b4f640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f45d00c2a00 0x7f45d00ac790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:50:02.141 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:02.136+0000 7f45d5b4f640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f45d00c20f0 con 0x7f45b8006570 2023-08-09T13:50:02.141 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:02.136+0000 7f45d5b4f640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f45d00c6e70 con 0x7f45d00c2580 2023-08-09T13:50:02.142 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:02.136+0000 7f45d5b4f640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f45d00c1aa0 con 0x7f45d00c2a00 2023-08-09T13:50:02.142 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:02.137+0000 7f45d4b4d640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f45b8006570 0x7f45d01493e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:50:02.142 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:02.137+0000 7f45d4b4d640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f45b8006570 0x7f45d01493e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:38258/0 (socket says 172.21.15.43:38258) 2023-08-09T13:50:02.142 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:02.137+0000 7f45d4b4d640 1 -- 172.21.15.43:0/1605825435 learned_addr learned my addr 172.21.15.43:0/1605825435 (peer_addr_for_me v2:172.21.15.43:0/0) 2023-08-09T13:50:02.142 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:02.137+0000 7f45cffff640 1 --2- 172.21.15.43:0/1605825435 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f45d00c2580 0x7f45d00ac250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:50:02.142 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:02.137+0000 7f45d534e640 1 --2- 172.21.15.43:0/1605825435 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f45d00c2a00 0x7f45d00ac790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:50:02.142 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:02.137+0000 7f45d4b4d640 1 -- 172.21.15.43:0/1605825435 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f45d00c2a00 msgr2=0x7f45d00ac790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:50:02.143 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:02.137+0000 7f45d4b4d640 1 --2- 172.21.15.43:0/1605825435 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f45d00c2a00 0x7f45d00ac790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:50:02.143 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:02.137+0000 7f45d4b4d640 1 -- 172.21.15.43:0/1605825435 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f45d00c2580 msgr2=0x7f45d00ac250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:50:02.143 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:02.137+0000 7f45d4b4d640 1 --2- 172.21.15.43:0/1605825435 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f45d00c2580 0x7f45d00ac250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:50:02.143 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:02.137+0000 7f45d4b4d640 1 -- 172.21.15.43:0/1605825435 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f45d00c1d80 con 0x7f45b8006570 2023-08-09T13:50:02.143 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:02.137+0000 7f45d4b4d640 1 --2- 172.21.15.43:0/1605825435 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f45b8006570 0x7f45d01493e0 secure :-1 s=READY pgs=894 cs=0 l=1 rev1=1 crypto rx=0x7f45c4011600 tx=0x7f45c4011ac0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-09T13:50:02.143 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:02.138+0000 7f45cd7fa640 1 -- 172.21.15.43:0/1605825435 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f45c400f920 con 0x7f45b8006570 2023-08-09T13:50:02.143 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:02.138+0000 7f45cd7fa640 1 -- 172.21.15.43:0/1605825435 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f45c400fa80 con 0x7f45b8006570 2023-08-09T13:50:02.144 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:02.138+0000 7f45cd7fa640 1 -- 172.21.15.43:0/1605825435 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f45c400fdf0 con 0x7f45b8006570 2023-08-09T13:50:02.144 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:02.138+0000 7f45d5b4f640 1 -- 172.21.15.43:0/1605825435 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f45d01f1740 con 0x7f45b8006570 2023-08-09T13:50:02.144 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:02.138+0000 7f45d5b4f640 1 -- 172.21.15.43:0/1605825435 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f45d01f1c70 con 0x7f45b8006570 2023-08-09T13:50:02.145 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:02.139+0000 7f45d5b4f640 1 -- 172.21.15.43:0/1605825435 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f45d00c1f10 con 0x7f45b8006570 2023-08-09T13:50:02.145 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:02.140+0000 7f45cd7fa640 1 -- 172.21.15.43:0/1605825435 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 24) v1 ==== 88574+0+0 (secure 0 0 0) 0x7f45c4015460 con 0x7f45b8006570 2023-08-09T13:50:02.145 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:02.141+0000 7f45cd7fa640 1 --2- 172.21.15.43:0/1605825435 >> [v2:172.21.15.203:6832/2515481191,v1:172.21.15.203:6833/2515481191] conn(0x7f45bc06a2e0 0x7f45bc06c7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:50:02.145 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:02.141+0000 7f45cd7fa640 1 -- 172.21.15.43:0/1605825435 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(75..75 src has 1..75) v4 ==== 6770+0+0 (secure 0 0 0) 0x7f45c40892c0 con 0x7f45b8006570 2023-08-09T13:50:02.145 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:02.142+0000 7f45cffff640 1 --2- 172.21.15.43:0/1605825435 >> [v2:172.21.15.203:6832/2515481191,v1:172.21.15.203:6833/2515481191] conn(0x7f45bc06a2e0 0x7f45bc06c7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:50:02.146 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:02.144+0000 7f45cffff640 1 --2- 172.21.15.43:0/1605825435 >> [v2:172.21.15.203:6832/2515481191,v1:172.21.15.203:6833/2515481191] conn(0x7f45bc06a2e0 0x7f45bc06c7a0 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7f45c0034000 tx=0x7f45c0007000 comp rx=0 tx=0).ready entity=mgr.5444 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-09T13:50:02.149 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:02.146+0000 7f45cd7fa640 1 -- 172.21.15.43:0/1605825435 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178066 (secure 0 0 0) 0x7f45c40542e0 con 0x7f45b8006570 2023-08-09T13:50:02.367 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:02.364+0000 7f45d5b4f640 1 -- 172.21.15.43:0/1605825435 --> [v2:172.21.15.203:6832/2515481191,v1:172.21.15.203:6833/2515481191] -- mgr_command(tid 0: {"prefix": "fs snap-schedule remove", "fs": "cephfs", "path": "snap_test_dir1", "target": ["mon-mgr", ""]}) v1 -- 0x7f45d00c1c10 con 0x7f45bc06a2e0 2023-08-09T13:50:02.543 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:02.540+0000 7f45cd7fa640 1 -- 172.21.15.43:0/1605825435 <== mgr.5444 v2:172.21.15.203:6832/2515481191 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+41 (secure 0 0 0) 0x7f45d00c1c10 con 0x7f45bc06a2e0 2023-08-09T13:50:02.543 INFO:teuthology.orchestra.run.smithi043.stdout:Schedule removed for path snap_test_dir1 2023-08-09T13:50:02.546 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:02.544+0000 7f45d5b4f640 1 -- 172.21.15.43:0/1605825435 >> [v2:172.21.15.203:6832/2515481191,v1:172.21.15.203:6833/2515481191] conn(0x7f45bc06a2e0 msgr2=0x7f45bc06c7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:50:02.546 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:02.544+0000 7f45d5b4f640 1 --2- 172.21.15.43:0/1605825435 >> [v2:172.21.15.203:6832/2515481191,v1:172.21.15.203:6833/2515481191] conn(0x7f45bc06a2e0 0x7f45bc06c7a0 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7f45c0034000 tx=0x7f45c0007000 comp rx=0 tx=0).stop 2023-08-09T13:50:02.547 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:02.544+0000 7f45d5b4f640 1 -- 172.21.15.43:0/1605825435 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f45b8006570 msgr2=0x7f45d01493e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:50:02.547 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:02.544+0000 7f45d5b4f640 1 --2- 172.21.15.43:0/1605825435 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f45b8006570 0x7f45d01493e0 secure :-1 s=READY pgs=894 cs=0 l=1 rev1=1 crypto rx=0x7f45c4011600 tx=0x7f45c4011ac0 comp rx=0 tx=0).stop 2023-08-09T13:50:02.547 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:02.544+0000 7f45d5b4f640 1 -- 172.21.15.43:0/1605825435 shutdown_connections 2023-08-09T13:50:02.547 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:02.544+0000 7f45d5b4f640 1 --2- 172.21.15.43:0/1605825435 >> [v2:172.21.15.203:6832/2515481191,v1:172.21.15.203:6833/2515481191] conn(0x7f45bc06a2e0 0x7f45bc06c7a0 unknown :-1 s=CLOSED pgs=26 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:50:02.547 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:02.544+0000 7f45d5b4f640 1 --2- 172.21.15.43:0/1605825435 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f45d00c2a00 0x7f45d00ac790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:50:02.547 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:02.544+0000 7f45d5b4f640 1 --2- 172.21.15.43:0/1605825435 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f45d00c2580 0x7f45d00ac250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:50:02.547 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:02.544+0000 7f45d5b4f640 1 --2- 172.21.15.43:0/1605825435 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f45b8006570 0x7f45d01493e0 unknown :-1 s=CLOSED pgs=894 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:50:02.548 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:02.544+0000 7f45d5b4f640 1 -- 172.21.15.43:0/1605825435 >> 172.21.15.43:0/1605825435 conn(0x7f45d00ca870 msgr2=0x7f45d006fb50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-09T13:50:02.548 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:02.544+0000 7f45d5b4f640 1 -- 172.21.15.43:0/1605825435 shutdown_connections 2023-08-09T13:50:02.548 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:02.544+0000 7f45d5b4f640 1 -- 172.21.15.43:0/1605825435 wait complete. 2023-08-09T13:50:02.566 DEBUG:tasks.cephfs.test_snap_schedules:res=Schedule removed for path snap_test_dir1 2023-08-09T13:50:02.567 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-09T13:50:02.567 DEBUG:teuthology.orchestra.run.smithi043:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-08-09T13:50:02.630 INFO:teuthology.orchestra.run.smithi043.stdout:scheduled-2023-08-09-13_49_00_UTC 2023-08-09T13:50:02.630 INFO:teuthology.orchestra.run.smithi043.stdout:scheduled-2023-08-09-13_50_00_UTC 2023-08-09T13:50:02.631 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/.snap/scheduled-2023-08-09-13_49_00_UTC 2023-08-09T13:50:02.632 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-09T13:50:02.632 DEBUG:teuthology.orchestra.run.smithi043:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/.snap/scheduled-2023-08-09-13_49_00_UTC) 2023-08-09T13:50:02.703 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/.snap/scheduled-2023-08-09-13_50_00_UTC 2023-08-09T13:50:02.704 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-09T13:50:02.704 DEBUG:teuthology.orchestra.run.smithi043:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/.snap/scheduled-2023-08-09-13_50_00_UTC) 2023-08-09T13:50:02.781 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-09T13:50:02.781 DEBUG:teuthology.orchestra.run.smithi043:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1) 2023-08-09T13:50:02.850 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph mgr module disable snap_schedule 2023-08-09T13:50:03.084 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:03.081+0000 7fa1bf06a640 1 Processor -- start 2023-08-09T13:50:03.084 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:03.081+0000 7fa1bf06a640 1 -- start start 2023-08-09T13:50:03.085 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:03.083+0000 7fa1bf06a640 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7fa1b815c3a0 0x7fa1b81608a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:50:03.086 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:03.083+0000 7fa1bf06a640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7fa1b80cd600 0x7fa1b80d0580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:50:03.086 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:03.083+0000 7fa1bf06a640 1 -- --> v1:172.21.15.43:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa1b8160de0 con 0x7fa1b80b0f00 2023-08-09T13:50:03.086 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:03.083+0000 7fa1bf06a640 1 -- --> v2:172.21.15.43:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa1b80b81c0 con 0x7fa1b80cd600 2023-08-09T13:50:03.086 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:03.083+0000 7fa1bf06a640 1 -- --> v2:172.21.15.203:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa1b80b7c00 con 0x7fa1b815c3a0 2023-08-09T13:50:03.087 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:03.083+0000 7fa1bd867640 1 --1- >> v1:172.21.15.43:6789/0 conn(0x7fa1b80b0f00 0x7fa1b80ccef0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.43:6789/0 says I am v1:172.21.15.43:48478/0 (socket says 172.21.15.43:48478) 2023-08-09T13:50:03.087 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:03.084+0000 7fa1bd867640 1 -- 172.21.15.43:0/2030547583 learned_addr learned my addr 172.21.15.43:0/2030547583 (peer_addr_for_me v1:172.21.15.43:0/0) 2023-08-09T13:50:03.087 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:03.084+0000 7fa1be869640 1 --2- 172.21.15.43:0/2030547583 >> v2:172.21.15.43:3300/0 conn(0x7fa1b80cd600 0x7fa1b80d0580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:50:03.087 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:03.084+0000 7fa1be068640 1 --2- 172.21.15.43:0/2030547583 >> v2:172.21.15.203:3300/0 conn(0x7fa1b815c3a0 0x7fa1b81608a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:50:03.087 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:03.084+0000 7fa1be869640 1 -- 172.21.15.43:0/2030547583 >> v2:172.21.15.203:3300/0 conn(0x7fa1b815c3a0 msgr2=0x7fa1b81608a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-09T13:50:03.088 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:03.084+0000 7fa1be869640 1 --2- 172.21.15.43:0/2030547583 >> v2:172.21.15.203:3300/0 conn(0x7fa1b815c3a0 0x7fa1b81608a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:50:03.088 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:03.084+0000 7fa1be869640 1 -- 172.21.15.43:0/2030547583 >> v1:172.21.15.43:6789/0 conn(0x7fa1b80b0f00 legacy=0x7fa1b80ccef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:50:03.088 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:03.084+0000 7fa1be869640 1 -- 172.21.15.43:0/2030547583 --> v2:172.21.15.43:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa1b80b7ee0 con 0x7fa1b80cd600 2023-08-09T13:50:03.089 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:03.085+0000 7fa1be869640 1 --2- 172.21.15.43:0/2030547583 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa1b80cd600 0x7fa1b80d0580 secure :-1 s=READY pgs=896 cs=0 l=1 rev1=1 crypto rx=0x7fa1b80cfd30 tx=0x7fa1b002be40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b893f2678fd1d2f0 server_cookie=0 in_seq=0 out_seq=0 2023-08-09T13:50:03.089 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:03.086+0000 7fa1bd066640 1 -- 172.21.15.43:0/2030547583 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa1b002cbf0 con 0x7fa1b80cd600 2023-08-09T13:50:03.089 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:03.086+0000 7fa1bd066640 1 -- 172.21.15.43:0/2030547583 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa1b002cd50 con 0x7fa1b80cd600 2023-08-09T13:50:03.089 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:03.086+0000 7fa1bd066640 1 -- 172.21.15.43:0/2030547583 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa1b0034660 con 0x7fa1b80cd600 2023-08-09T13:50:03.089 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:03.086+0000 7fa1bf06a640 1 -- 172.21.15.43:0/2030547583 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa1b80cd600 msgr2=0x7fa1b80d0580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:50:03.090 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:03.086+0000 7fa1bf06a640 1 --2- 172.21.15.43:0/2030547583 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa1b80cd600 0x7fa1b80d0580 secure :-1 s=READY pgs=896 cs=0 l=1 rev1=1 crypto rx=0x7fa1b80cfd30 tx=0x7fa1b002be40 comp rx=0 tx=0).stop 2023-08-09T13:50:03.090 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:03.087+0000 7fa1bf06a640 1 -- 172.21.15.43:0/2030547583 shutdown_connections 2023-08-09T13:50:03.090 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:03.087+0000 7fa1bf06a640 1 --2- 172.21.15.43:0/2030547583 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa1b80cd600 0x7fa1b80d0580 unknown :-1 s=CLOSED pgs=896 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:50:03.090 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:03.087+0000 7fa1bf06a640 1 --2- 172.21.15.43:0/2030547583 >> v2:172.21.15.203:3300/0 conn(0x7fa1b815c3a0 0x7fa1b81608a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:50:03.090 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:03.087+0000 7fa1bf06a640 1 -- 172.21.15.43:0/2030547583 >> 172.21.15.43:0/2030547583 conn(0x7fa1b806a210 msgr2=0x7fa1b806a620 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-09T13:50:03.090 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:03.087+0000 7fa1bf06a640 1 -- 172.21.15.43:0/2030547583 shutdown_connections 2023-08-09T13:50:03.090 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:03.087+0000 7fa1bf06a640 1 -- 172.21.15.43:0/2030547583 wait complete. 2023-08-09T13:50:03.091 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:03.088+0000 7fa1bf06a640 1 Processor -- start 2023-08-09T13:50:03.091 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:03.089+0000 7fa1bf06a640 1 -- start start 2023-08-09T13:50:03.092 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:03.090+0000 7fa1bf06a640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fa1b80b0f00 0x7fa1b81f46b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:50:03.092 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:03.090+0000 7fa1bf06a640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa1b80cd600 0x7fa1b8156c90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:50:03.092 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:03.090+0000 7fa1bf06a640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fa1b815c3a0 0x7fa1b80b65e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:50:03.093 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:03.090+0000 7fa1bf06a640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa1b80b47e0 con 0x7fa1b80cd600 2023-08-09T13:50:03.093 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:03.090+0000 7fa1bf06a640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa1b80b4350 con 0x7fa1b80b0f00 2023-08-09T13:50:03.093 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:03.090+0000 7fa1bf06a640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa1b80b3740 con 0x7fa1b815c3a0 2023-08-09T13:50:03.093 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:03.090+0000 7fa1bd867640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa1b80cd600 0x7fa1b8156c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:50:03.093 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:03.090+0000 7fa1bd867640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa1b80cd600 0x7fa1b8156c90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:38268/0 (socket says 172.21.15.43:38268) 2023-08-09T13:50:03.093 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:03.090+0000 7fa1bd867640 1 -- 172.21.15.43:0/2424525688 learned_addr learned my addr 172.21.15.43:0/2424525688 (peer_addr_for_me v2:172.21.15.43:0/0) 2023-08-09T13:50:03.094 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:03.090+0000 7fa1be869640 1 --2- 172.21.15.43:0/2424525688 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fa1b815c3a0 0x7fa1b80b65e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:50:03.095 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:03.090+0000 7fa1be068640 1 --2- 172.21.15.43:0/2424525688 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fa1b80b0f00 0x7fa1b81f46b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:50:03.095 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:03.090+0000 7fa1bd867640 1 -- 172.21.15.43:0/2424525688 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fa1b815c3a0 msgr2=0x7fa1b80b65e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:50:03.095 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:03.090+0000 7fa1bd867640 1 --2- 172.21.15.43:0/2424525688 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fa1b815c3a0 0x7fa1b80b65e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:50:03.095 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:03.090+0000 7fa1bd867640 1 -- 172.21.15.43:0/2424525688 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fa1b80b0f00 msgr2=0x7fa1b81f46b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:50:03.095 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:03.090+0000 7fa1bd867640 1 --2- 172.21.15.43:0/2424525688 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fa1b80b0f00 0x7fa1b81f46b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:50:03.095 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:03.090+0000 7fa1bd867640 1 -- 172.21.15.43:0/2424525688 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa1b0009a70 con 0x7fa1b80cd600 2023-08-09T13:50:03.095 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:03.090+0000 7fa1be068640 1 --2- 172.21.15.43:0/2424525688 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fa1b80b0f00 0x7fa1b81f46b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-08-09T13:50:03.095 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:03.090+0000 7fa1be869640 1 --2- 172.21.15.43:0/2424525688 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fa1b815c3a0 0x7fa1b80b65e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-08-09T13:50:03.096 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:03.091+0000 7fa1bd867640 1 --2- 172.21.15.43:0/2424525688 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa1b80cd600 0x7fa1b8156c90 secure :-1 s=READY pgs=897 cs=0 l=1 rev1=1 crypto rx=0x7fa1a800c3e0 tx=0x7fa1a800c8a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-09T13:50:03.096 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:03.091+0000 7fa1aeffd640 1 -- 172.21.15.43:0/2424525688 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa1a8005ce0 con 0x7fa1b80cd600 2023-08-09T13:50:03.096 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:03.091+0000 7fa1aeffd640 1 -- 172.21.15.43:0/2424525688 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa1a8005e40 con 0x7fa1b80cd600 2023-08-09T13:50:03.096 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:03.091+0000 7fa1aeffd640 1 -- 172.21.15.43:0/2424525688 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa1a80072d0 con 0x7fa1b80cd600 2023-08-09T13:50:03.096 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:03.091+0000 7fa1bf06a640 1 -- 172.21.15.43:0/2424525688 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa1b80b3a20 con 0x7fa1b80cd600 2023-08-09T13:50:03.096 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:03.091+0000 7fa1bf06a640 1 -- 172.21.15.43:0/2424525688 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa1b80b3fe0 con 0x7fa1b80cd600 2023-08-09T13:50:03.097 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:03.092+0000 7fa1bf06a640 1 -- 172.21.15.43:0/2424525688 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa1b80b41a0 con 0x7fa1b80cd600 2023-08-09T13:50:03.097 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:03.093+0000 7fa1aeffd640 1 -- 172.21.15.43:0/2424525688 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 24) v1 ==== 88574+0+0 (secure 0 0 0) 0x7fa1a80074f0 con 0x7fa1b80cd600 2023-08-09T13:50:03.097 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:03.094+0000 7fa1aeffd640 1 --2- 172.21.15.43:0/2424525688 >> [v2:172.21.15.203:6832/2515481191,v1:172.21.15.203:6833/2515481191] conn(0x7fa19c06a2e0 0x7fa19c06c7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:50:03.098 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:03.094+0000 7fa1aeffd640 1 -- 172.21.15.43:0/2424525688 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(75..75 src has 1..75) v4 ==== 6770+0+0 (secure 0 0 0) 0x7fa1a80881c0 con 0x7fa1b80cd600 2023-08-09T13:50:03.098 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:03.095+0000 7fa1be068640 1 --2- 172.21.15.43:0/2424525688 >> [v2:172.21.15.203:6832/2515481191,v1:172.21.15.203:6833/2515481191] conn(0x7fa19c06a2e0 0x7fa19c06c7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:50:03.098 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:03.096+0000 7fa1be068640 1 --2- 172.21.15.43:0/2424525688 >> [v2:172.21.15.203:6832/2515481191,v1:172.21.15.203:6833/2515481191] conn(0x7fa19c06a2e0 0x7fa19c06c7a0 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7fa1b4006fd0 tx=0x7fa1b4008040 comp rx=0 tx=0).ready entity=mgr.5444 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-09T13:50:03.101 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:03.098+0000 7fa1aeffd640 1 -- 172.21.15.43:0/2424525688 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178066 (secure 0 0 0) 0x7fa1a8053290 con 0x7fa1b80cd600 2023-08-09T13:50:03.344 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:03.341+0000 7fa1bf06a640 1 -- 172.21.15.43:0/2424525688 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "mgr module disable", "module": "snap_schedule"} v 0) v1 -- 0x7fa1b80b44c0 con 0x7fa1b80cd600 2023-08-09T13:50:03.536 DEBUG:teuthology.orchestra.run.smithi043:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-08-09T13:50:03.545 DEBUG:teuthology.orchestra.run.smithi203:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-08-09T13:50:04.098 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:04.096+0000 7fa1aeffd640 1 -- 172.21.15.43:0/2424525688 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mgrmap(e 25) v1 ==== 88557+0+0 (secure 0 0 0) 0x7fa1a8053660 con 0x7fa1b80cd600 2023-08-09T13:50:04.102 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:04.099+0000 7fa1aeffd640 1 -- 172.21.15.43:0/2424525688 <== mon.0 v2:172.21.15.43:3300/0 8 ==== mon_command_ack([{"prefix": "mgr module disable", "module": "snap_schedule"}]=0 v25) v1 ==== 93+0+0 (secure 0 0 0) 0x7fa1a8053290 con 0x7fa1b80cd600 2023-08-09T13:50:04.106 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:04.103+0000 7fa1bf06a640 1 -- 172.21.15.43:0/2424525688 >> [v2:172.21.15.203:6832/2515481191,v1:172.21.15.203:6833/2515481191] conn(0x7fa19c06a2e0 msgr2=0x7fa19c06c7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:50:04.106 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:04.103+0000 7fa1bf06a640 1 --2- 172.21.15.43:0/2424525688 >> [v2:172.21.15.203:6832/2515481191,v1:172.21.15.203:6833/2515481191] conn(0x7fa19c06a2e0 0x7fa19c06c7a0 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7fa1b4006fd0 tx=0x7fa1b4008040 comp rx=0 tx=0).stop 2023-08-09T13:50:04.107 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:04.103+0000 7fa1bf06a640 1 -- 172.21.15.43:0/2424525688 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa1b80cd600 msgr2=0x7fa1b8156c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:50:04.107 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:04.103+0000 7fa1bf06a640 1 --2- 172.21.15.43:0/2424525688 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa1b80cd600 0x7fa1b8156c90 secure :-1 s=READY pgs=897 cs=0 l=1 rev1=1 crypto rx=0x7fa1a800c3e0 tx=0x7fa1a800c8a0 comp rx=0 tx=0).stop 2023-08-09T13:50:04.107 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:04.104+0000 7fa1bf06a640 1 -- 172.21.15.43:0/2424525688 shutdown_connections 2023-08-09T13:50:04.107 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:04.104+0000 7fa1bf06a640 1 --2- 172.21.15.43:0/2424525688 >> [v2:172.21.15.203:6832/2515481191,v1:172.21.15.203:6833/2515481191] conn(0x7fa19c06a2e0 0x7fa19c06c7a0 unknown :-1 s=CLOSED pgs=27 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:50:04.107 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:04.104+0000 7fa1bf06a640 1 --2- 172.21.15.43:0/2424525688 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fa1b815c3a0 0x7fa1b80b65e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:50:04.107 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:04.104+0000 7fa1bf06a640 1 --2- 172.21.15.43:0/2424525688 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa1b80cd600 0x7fa1b8156c90 unknown :-1 s=CLOSED pgs=897 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:50:04.108 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:04.104+0000 7fa1bf06a640 1 --2- 172.21.15.43:0/2424525688 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fa1b80b0f00 0x7fa1b81f46b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:50:04.108 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:04.104+0000 7fa1bf06a640 1 -- 172.21.15.43:0/2424525688 >> 172.21.15.43:0/2424525688 conn(0x7fa1b806a210 msgr2=0x7fa1b80c6210 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-09T13:50:04.108 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:04.104+0000 7fa1bf06a640 1 -- 172.21.15.43:0/2424525688 shutdown_connections 2023-08-09T13:50:04.108 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:04.104+0000 7fa1bf06a640 1 -- 172.21.15.43:0/2424525688 wait complete. 2023-08-09T13:50:04.130 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2023-08-09T13:50:04.131 DEBUG:teuthology.orchestra.run.smithi043:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2023-08-09T13:50:04.247 DEBUG:teuthology.orchestra.run.smithi203:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2023-08-09T13:50:04.330 INFO:tasks.ceph.mgr.x.smithi203.stderr:2023-08-09T13:50:04.327+0000 7fbb9f40e040 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2023-08-09T13:50:04.340 INFO:tasks.ceph.mgr.y.smithi043.stderr:2023-08-09T13:50:04.337+0000 7f76db0f8040 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2023-08-09T13:50:04.346 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2023-08-09T13:50:04.347 DEBUG:teuthology.orchestra.run.smithi043:> set -ex 2023-08-09T13:50:04.347 DEBUG:teuthology.orchestra.run.smithi043:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-09T13:50:04.406 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-08-09T13:50:04.618 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:04.616+0000 7fb5ee2cc640 1 Processor -- start 2023-08-09T13:50:04.619 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:04.616+0000 7fb5ee2cc640 1 -- start start 2023-08-09T13:50:04.620 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:04.617+0000 7fb5ee2cc640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7fb5e80b2790 0x7fb5e80d2ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:50:04.620 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:04.618+0000 7fb5ee2cc640 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7fb5e80b2b70 0x7fb5e80b5330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:50:04.620 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:04.618+0000 7fb5ee2cc640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fb5e80b3050 0x7fb5e80b3f60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:50:04.621 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:04.618+0000 7fb5ee2cc640 1 -- --> v2:172.21.15.43:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb5e80b3950 con 0x7fb5e80b2790 2023-08-09T13:50:04.621 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:04.618+0000 7fb5ee2cc640 1 -- --> v2:172.21.15.203:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb5e80ce090 con 0x7fb5e80b2b70 2023-08-09T13:50:04.621 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:04.618+0000 7fb5ee2cc640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb5e80cd7f0 con 0x7fb5e80b3050 2023-08-09T13:50:04.621 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:04.618+0000 7fb5e7fff640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7fb5e80b2790 0x7fb5e80d2ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:50:04.621 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:04.618+0000 7fb5e7fff640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7fb5e80b2790 0x7fb5e80d2ed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:38286/0 (socket says 172.21.15.43:38286) 2023-08-09T13:50:04.621 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:04.618+0000 7fb5e7fff640 1 -- 172.21.15.43:0/4086173198 learned_addr learned my addr 172.21.15.43:0/4086173198 (peer_addr_for_me v2:172.21.15.43:0/0) 2023-08-09T13:50:04.622 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:04.618+0000 7fb5e77fe640 1 --2- 172.21.15.43:0/4086173198 >> v2:172.21.15.203:3300/0 conn(0x7fb5e80b2b70 0x7fb5e80b5330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:50:04.622 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:04.618+0000 7fb5ec869640 1 --2- 172.21.15.43:0/4086173198 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fb5e80b3050 0x7fb5e80b3f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:50:04.622 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:04.618+0000 7fb5e7fff640 1 -- 172.21.15.43:0/4086173198 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fb5e80b3050 msgr2=0x7fb5e80b3f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-09T13:50:04.623 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:04.618+0000 7fb5e7fff640 1 --2- 172.21.15.43:0/4086173198 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fb5e80b3050 0x7fb5e80b3f60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:50:04.623 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:04.618+0000 7fb5e7fff640 1 -- 172.21.15.43:0/4086173198 >> v2:172.21.15.203:3300/0 conn(0x7fb5e80b2b70 msgr2=0x7fb5e80b5330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-09T13:50:04.623 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:04.618+0000 7fb5e7fff640 1 --2- 172.21.15.43:0/4086173198 >> v2:172.21.15.203:3300/0 conn(0x7fb5e80b2b70 0x7fb5e80b5330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:50:04.623 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:04.618+0000 7fb5e7fff640 1 -- 172.21.15.43:0/4086173198 --> v2:172.21.15.43:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb5e80cdad0 con 0x7fb5e80b2790 2023-08-09T13:50:04.623 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:04.619+0000 7fb5e7fff640 1 --2- 172.21.15.43:0/4086173198 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb5e80b2790 0x7fb5e80d2ed0 secure :-1 s=READY pgs=902 cs=0 l=1 rev1=1 crypto rx=0x7fb5e8066af0 tx=0x7fb5d802b910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a2c2e3ce46eaf7b0 server_cookie=0 in_seq=0 out_seq=0 2023-08-09T13:50:04.624 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:04.620+0000 7fb5e6ffd640 1 -- 172.21.15.43:0/4086173198 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb5d8031870 con 0x7fb5e80b2790 2023-08-09T13:50:04.624 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:04.620+0000 7fb5e6ffd640 1 -- 172.21.15.43:0/4086173198 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb5d80319d0 con 0x7fb5e80b2790 2023-08-09T13:50:04.624 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:04.620+0000 7fb5e6ffd640 1 -- 172.21.15.43:0/4086173198 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb5d8031d60 con 0x7fb5e80b2790 2023-08-09T13:50:04.625 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:04.621+0000 7fb5ee2cc640 1 -- 172.21.15.43:0/4086173198 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb5e80b2790 msgr2=0x7fb5e80d2ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:50:04.625 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:04.621+0000 7fb5ee2cc640 1 --2- 172.21.15.43:0/4086173198 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb5e80b2790 0x7fb5e80d2ed0 secure :-1 s=READY pgs=902 cs=0 l=1 rev1=1 crypto rx=0x7fb5e8066af0 tx=0x7fb5d802b910 comp rx=0 tx=0).stop 2023-08-09T13:50:04.625 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:04.621+0000 7fb5ee2cc640 1 -- 172.21.15.43:0/4086173198 shutdown_connections 2023-08-09T13:50:04.625 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:04.621+0000 7fb5ee2cc640 1 --2- 172.21.15.43:0/4086173198 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fb5e80b3050 0x7fb5e80b3f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:50:04.625 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:04.621+0000 7fb5ee2cc640 1 --2- 172.21.15.43:0/4086173198 >> v2:172.21.15.203:3300/0 conn(0x7fb5e80b2b70 0x7fb5e80b5330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:50:04.626 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:04.621+0000 7fb5ee2cc640 1 --2- 172.21.15.43:0/4086173198 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb5e80b2790 0x7fb5e80d2ed0 unknown :-1 s=CLOSED pgs=902 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:50:04.626 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:04.621+0000 7fb5ee2cc640 1 -- 172.21.15.43:0/4086173198 >> 172.21.15.43:0/4086173198 conn(0x7fb5e809b7b0 msgr2=0x7fb5e80d24e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-09T13:50:04.626 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:04.621+0000 7fb5ee2cc640 1 -- 172.21.15.43:0/4086173198 shutdown_connections 2023-08-09T13:50:04.626 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:04.621+0000 7fb5ee2cc640 1 -- 172.21.15.43:0/4086173198 wait complete. 2023-08-09T13:50:04.626 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:04.622+0000 7fb5ee2cc640 1 Processor -- start 2023-08-09T13:50:04.626 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:04.623+0000 7fb5ee2cc640 1 -- start start 2023-08-09T13:50:04.628 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:04.624+0000 7fb5ee2cc640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fb5e80b2790 0x7fb5e809d080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:50:04.628 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:04.624+0000 7fb5ee2cc640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb5e80b2b70 0x7fb5e80cbe10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:50:04.628 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:04.624+0000 7fb5ee2cc640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fb5e80b3050 0x7fb5e815a5b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:50:04.629 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:04.624+0000 7fb5ee2cc640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb5e80a02b0 con 0x7fb5e80b2b70 2023-08-09T13:50:04.629 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:04.624+0000 7fb5ee2cc640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb5e80c70a0 con 0x7fb5e80b2790 2023-08-09T13:50:04.629 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:04.624+0000 7fb5ee2cc640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb5e80c6520 con 0x7fb5e80b3050 2023-08-09T13:50:04.629 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:04.624+0000 7fb5e77fe640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb5e80b2b70 0x7fb5e80cbe10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:50:04.629 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:04.624+0000 7fb5e77fe640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb5e80b2b70 0x7fb5e80cbe10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:38302/0 (socket says 172.21.15.43:38302) 2023-08-09T13:50:04.629 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:04.624+0000 7fb5e77fe640 1 -- 172.21.15.43:0/3343069590 learned_addr learned my addr 172.21.15.43:0/3343069590 (peer_addr_for_me v2:172.21.15.43:0/0) 2023-08-09T13:50:04.630 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:04.624+0000 7fb5e7fff640 1 --2- 172.21.15.43:0/3343069590 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fb5e80b2790 0x7fb5e809d080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:50:04.630 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:04.624+0000 7fb5e77fe640 1 -- 172.21.15.43:0/3343069590 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fb5e80b3050 msgr2=0x7fb5e815a5b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:50:04.630 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:04.624+0000 7fb5e77fe640 1 --2- 172.21.15.43:0/3343069590 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fb5e80b3050 0x7fb5e815a5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:50:04.630 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:04.624+0000 7fb5e77fe640 1 -- 172.21.15.43:0/3343069590 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fb5e80b2790 msgr2=0x7fb5e809d080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:50:04.630 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:04.624+0000 7fb5e77fe640 1 --2- 172.21.15.43:0/3343069590 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fb5e80b2790 0x7fb5e809d080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:50:04.630 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:04.624+0000 7fb5e77fe640 1 -- 172.21.15.43:0/3343069590 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb5d80094d0 con 0x7fb5e80b2b70 2023-08-09T13:50:04.630 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:04.624+0000 7fb5e7fff640 1 --2- 172.21.15.43:0/3343069590 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fb5e80b2790 0x7fb5e809d080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-08-09T13:50:04.631 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:04.625+0000 7fb5e77fe640 1 --2- 172.21.15.43:0/3343069590 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb5e80b2b70 0x7fb5e80cbe10 secure :-1 s=READY pgs=903 cs=0 l=1 rev1=1 crypto rx=0x7fb5dc00d800 tx=0x7fb5dc00dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-09T13:50:04.631 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:04.625+0000 7fb5e4ff9640 1 -- 172.21.15.43:0/3343069590 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb5dc0096a0 con 0x7fb5e80b2b70 2023-08-09T13:50:04.631 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:04.625+0000 7fb5e4ff9640 1 -- 172.21.15.43:0/3343069590 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb5dc009800 con 0x7fb5e80b2b70 2023-08-09T13:50:04.631 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:04.625+0000 7fb5e4ff9640 1 -- 172.21.15.43:0/3343069590 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb5dc009990 con 0x7fb5e80b2b70 2023-08-09T13:50:04.631 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:04.625+0000 7fb5ee2cc640 1 -- 172.21.15.43:0/3343069590 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb5e80c6800 con 0x7fb5e80b2b70 2023-08-09T13:50:04.631 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:04.625+0000 7fb5ee2cc640 1 -- 172.21.15.43:0/3343069590 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb5e80c6dc0 con 0x7fb5e80b2b70 2023-08-09T13:50:04.632 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:04.626+0000 7fb5ee2cc640 1 -- 172.21.15.43:0/3343069590 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb5e80739f0 con 0x7fb5e80b2b70 2023-08-09T13:50:04.632 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:04.627+0000 7fb5e4ff9640 1 -- 172.21.15.43:0/3343069590 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 25) v1 ==== 88557+0+0 (secure 0 0 0) 0x7fb5dc009af0 con 0x7fb5e80b2b70 2023-08-09T13:50:04.633 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:04.628+0000 7fb5e4ff9640 1 --2- 172.21.15.43:0/3343069590 >> [v2:172.21.15.203:6832/2515481191,v1:172.21.15.203:6833/2515481191] conn(0x7fb5d406a290 0x7fb5d406c750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:50:04.633 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:04.629+0000 7fb5e4ff9640 1 -- 172.21.15.43:0/3343069590 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(75..75 src has 1..75) v4 ==== 6770+0+0 (secure 0 0 0) 0x7fb5dc089060 con 0x7fb5e80b2b70 2023-08-09T13:50:04.633 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:04.629+0000 7fb5e7fff640 1 -- 172.21.15.43:0/3343069590 >> [v2:172.21.15.203:6832/2515481191,v1:172.21.15.203:6833/2515481191] conn(0x7fb5d406a290 msgr2=0x7fb5d406c750 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.203:6832/2515481191 2023-08-09T13:50:04.633 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:04.629+0000 7fb5e7fff640 1 --2- 172.21.15.43:0/3343069590 >> [v2:172.21.15.203:6832/2515481191,v1:172.21.15.203:6833/2515481191] conn(0x7fb5d406a290 0x7fb5d406c750 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-08-09T13:50:04.639 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:04.637+0000 7fb5e4ff9640 1 -- 172.21.15.43:0/3343069590 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178066 (secure 0 0 0) 0x7fb5dc0bb7b0 con 0x7fb5e80b2b70 2023-08-09T13:50:04.832 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:04.830+0000 7fb5e7fff640 1 -- 172.21.15.43:0/3343069590 >> [v2:172.21.15.203:6832/2515481191,v1:172.21.15.203:6833/2515481191] conn(0x7fb5d406a290 msgr2=0x7fb5d406c750 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.203:6832/2515481191 2023-08-09T13:50:04.833 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:04.830+0000 7fb5e7fff640 1 --2- 172.21.15.43:0/3343069590 >> [v2:172.21.15.203:6832/2515481191,v1:172.21.15.203:6833/2515481191] conn(0x7fb5d406a290 0x7fb5d406c750 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2023-08-09T13:50:04.838 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:04.835+0000 7fb5ee2cc640 1 -- 172.21.15.43:0/3343069590 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fb5e80c6990 con 0x7fb5e80b2b70 2023-08-09T13:50:04.840 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:04.838+0000 7fb5e4ff9640 1 -- 172.21.15.43:0/3343069590 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v75) v1 ==== 74+0+19057 (secure 0 0 0) 0x7fb5dc0541e0 con 0x7fb5e80b2b70 2023-08-09T13:50:04.841 INFO:teuthology.orchestra.run.smithi043.stdout: 2023-08-09T13:50:04.841 INFO:teuthology.orchestra.run.smithi043.stdout:{"epoch":75,"fsid":"daac87ee-1337-4250-a955-aea4e408a39b","created":"2023-08-09T13:33:41.701198+0000","modified":"2023-08-09T13:48:34.065606+0000","last_up_change":"2023-08-09T13:33:53.783453+0000","last_in_change":"2023-08-09T13:33:45.148904+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":9,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":"rbd","create_time":"2023-08-09T13:33:54.365137+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":".mgr","create_time":"2023-08-09T13:33:54.831803+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":8,"pool_name":"cephfs_metadata","create_time":"2023-08-09T13:48:16.039756+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"73","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":9,"pool_name":"cephfs_data","create_time":"2023-08-09T13:48:16.615400+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"74","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.6299999952316284,"score_stable":1.6299999952316284,"optimal_score":1,"raw_score_acting":1.6299999952316284,"raw_score_stable":1.6299999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"cbd49ecd-e896-4fbd-864e-5c37a829b165","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":72,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6800","nonce":502667335},{"type":"v1","addr":"172.21.15.43:6801","nonce":502667335}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6802","nonce":502667335},{"type":"v1","addr":"172.21.15.43:6803","nonce":502667335}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6806","nonce":502667335},{"type":"v1","addr":"172.21.15.43:6807","nonce":502667335}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6804","nonce":502667335},{"type":"v1","addr":"172.21.15.43:6805","nonce":502667335}]},"public_addr":"172.21.15.43:6801/502667335","cluster_addr":"172.21.15.43:6803/502667335","heartbeat_back_addr":"172.21.15.43:6807/502667335","heartbeat_front_addr":"172.21.15.43:6805/502667335","state":["exists","up"]},{"osd":1,"uuid":"75c7e0fe-1ec7-4fe8-a2f2-3661a65b8500","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":72,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6808","nonce":3289134610},{"type":"v1","addr":"172.21.15.43:6809","nonce":3289134610}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6810","nonce":3289134610},{"type":"v1","addr":"172.21.15.43:6812","nonce":3289134610}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6821","nonce":3289134610},{"type":"v1","addr":"172.21.15.43:6824","nonce":3289134610}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6815","nonce":3289134610},{"type":"v1","addr":"172.21.15.43:6818","nonce":3289134610}]},"public_addr":"172.21.15.43:6809/3289134610","cluster_addr":"172.21.15.43:6812/3289134610","heartbeat_back_addr":"172.21.15.43:6824/3289134610","heartbeat_front_addr":"172.21.15.43:6818/3289134610","state":["exists","up"]},{"osd":2,"uuid":"bb5c41e9-7f5f-4125-9b01-3bb8bfc2ea83","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":72,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6811","nonce":4079709736},{"type":"v1","addr":"172.21.15.43:6814","nonce":4079709736}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6817","nonce":4079709736},{"type":"v1","addr":"172.21.15.43:6820","nonce":4079709736}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6829","nonce":4079709736},{"type":"v1","addr":"172.21.15.43:6831","nonce":4079709736}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6823","nonce":4079709736},{"type":"v1","addr":"172.21.15.43:6826","nonce":4079709736}]},"public_addr":"172.21.15.43:6814/4079709736","cluster_addr":"172.21.15.43:6820/4079709736","heartbeat_back_addr":"172.21.15.43:6831/4079709736","heartbeat_front_addr":"172.21.15.43:6826/4079709736","state":["exists","up"]},{"osd":3,"uuid":"bfe4c8b4-694a-465c-87e6-2b2aab852613","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":72,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6813","nonce":1709538759},{"type":"v1","addr":"172.21.15.43:6816","nonce":1709538759}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6819","nonce":1709538759},{"type":"v1","addr":"172.21.15.43:6822","nonce":1709538759}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6828","nonce":1709538759},{"type":"v1","addr":"172.21.15.43:6830","nonce":1709538759}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6825","nonce":1709538759},{"type":"v1","addr":"172.21.15.43:6827","nonce":1709538759}]},"public_addr":"172.21.15.43:6816/1709538759","cluster_addr":"172.21.15.43:6822/1709538759","heartbeat_back_addr":"172.21.15.43:6830/1709538759","heartbeat_front_addr":"172.21.15.43:6827/1709538759","state":["exists","up"]},{"osd":4,"uuid":"fa2e8a77-aba1-4756-853e-0d174a5bdeba","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":72,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6800","nonce":4196866481},{"type":"v1","addr":"172.21.15.203:6801","nonce":4196866481}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6802","nonce":4196866481},{"type":"v1","addr":"172.21.15.203:6803","nonce":4196866481}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6806","nonce":4196866481},{"type":"v1","addr":"172.21.15.203:6807","nonce":4196866481}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6804","nonce":4196866481},{"type":"v1","addr":"172.21.15.203:6805","nonce":4196866481}]},"public_addr":"172.21.15.203:6801/4196866481","cluster_addr":"172.21.15.203:6803/4196866481","heartbeat_back_addr":"172.21.15.203:6807/4196866481","heartbeat_front_addr":"172.21.15.203:6805/4196866481","state":["exists","up"]},{"osd":5,"uuid":"81811e4b-4d24-4f51-a82c-f1d700a2a00d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":72,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6816","nonce":3059290081},{"type":"v1","addr":"172.21.15.203:6817","nonce":3059290081}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6818","nonce":3059290081},{"type":"v1","addr":"172.21.15.203:6819","nonce":3059290081}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6822","nonce":3059290081},{"type":"v1","addr":"172.21.15.203:6823","nonce":3059290081}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6820","nonce":3059290081},{"type":"v1","addr":"172.21.15.203:6821","nonce":3059290081}]},"public_addr":"172.21.15.203:6817/3059290081","cluster_addr":"172.21.15.203:6819/3059290081","heartbeat_back_addr":"172.21.15.203:6823/3059290081","heartbeat_front_addr":"172.21.15.203:6821/3059290081","state":["exists","up"]},{"osd":6,"uuid":"307bac4d-6a5b-443d-af75-56d695d7674f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":72,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6824","nonce":512155569},{"type":"v1","addr":"172.21.15.203:6825","nonce":512155569}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6826","nonce":512155569},{"type":"v1","addr":"172.21.15.203:6827","nonce":512155569}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6830","nonce":512155569},{"type":"v1","addr":"172.21.15.203:6831","nonce":512155569}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6828","nonce":512155569},{"type":"v1","addr":"172.21.15.203:6829","nonce":512155569}]},"public_addr":"172.21.15.203:6825/512155569","cluster_addr":"172.21.15.203:6827/512155569","heartbeat_back_addr":"172.21.15.203:6831/512155569","heartbeat_front_addr":"172.21.15.203:6829/512155569","state":["exists","up"]},{"osd":7,"uuid":"a95dc7ca-9f1b-4295-af70-68a22a2d7d5e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":72,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6808","nonce":3331450438},{"type":"v1","addr":"172.21.15.203:6809","nonce":3331450438}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6810","nonce":3331450438},{"type":"v1","addr":"172.21.15.203:6811","nonce":3331450438}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6814","nonce":3331450438},{"type":"v1","addr":"172.21.15.203:6815","nonce":3331450438}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6812","nonce":3331450438},{"type":"v1","addr":"172.21.15.203:6813","nonce":3331450438}]},"public_addr":"172.21.15.203:6809/3331450438","cluster_addr":"172.21.15.203:6811/3331450438","heartbeat_back_addr":"172.21.15.203:6815/3331450438","heartbeat_front_addr":"172.21.15.203:6813/3331450438","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-09T13:33:50.547652+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-09T13:33:50.589469+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-09T13:33:50.561724+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-09T13:33:50.599650+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-09T13:33:50.411855+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-09T13:33:50.219681+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-09T13:33:50.504664+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-09T13:33:50.441248+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.203:6833/272917756":"2023-08-10T13:48:34.065493+0000","172.21.15.203:6832/272917756":"2023-08-10T13:48:34.065493+0000","172.21.15.203:0/665971116":"2023-08-10T13:48:34.065493+0000","172.21.15.203:0/2868823959":"2023-08-10T13:48:34.065493+0000","172.21.15.203:0/3168500363":"2023-08-10T13:48:34.065493+0000","172.21.15.203:0/752327795":"2023-08-10T13:48:34.065493+0000","172.21.15.203:0/1970263661":"2023-08-10T13:48:34.065493+0000","172.21.15.203:0/1407921417":"2023-08-10T13:48:34.065493+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":7,"snaps":[{"begin":104,"length":1},{"begin":221,"length":1}]}],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-08-09T13:50:04.843 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:04.840+0000 7fb5ee2cc640 1 -- 172.21.15.43:0/3343069590 >> [v2:172.21.15.203:6832/2515481191,v1:172.21.15.203:6833/2515481191] conn(0x7fb5d406a290 msgr2=0x7fb5d406c750 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-08-09T13:50:04.843 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:04.840+0000 7fb5ee2cc640 1 --2- 172.21.15.43:0/3343069590 >> [v2:172.21.15.203:6832/2515481191,v1:172.21.15.203:6833/2515481191] conn(0x7fb5d406a290 0x7fb5d406c750 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:50:04.844 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:04.840+0000 7fb5ee2cc640 1 -- 172.21.15.43:0/3343069590 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb5e80b2b70 msgr2=0x7fb5e80cbe10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:50:04.844 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:04.840+0000 7fb5ee2cc640 1 --2- 172.21.15.43:0/3343069590 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb5e80b2b70 0x7fb5e80cbe10 secure :-1 s=READY pgs=903 cs=0 l=1 rev1=1 crypto rx=0x7fb5dc00d800 tx=0x7fb5dc00dcc0 comp rx=0 tx=0).stop 2023-08-09T13:50:04.844 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:04.841+0000 7fb5ee2cc640 1 -- 172.21.15.43:0/3343069590 shutdown_connections 2023-08-09T13:50:04.844 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:04.841+0000 7fb5ee2cc640 1 --2- 172.21.15.43:0/3343069590 >> [v2:172.21.15.203:6832/2515481191,v1:172.21.15.203:6833/2515481191] conn(0x7fb5d406a290 0x7fb5d406c750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:50:04.844 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:04.841+0000 7fb5ee2cc640 1 --2- 172.21.15.43:0/3343069590 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fb5e80b3050 0x7fb5e815a5b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:50:04.844 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:04.841+0000 7fb5ee2cc640 1 --2- 172.21.15.43:0/3343069590 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb5e80b2b70 0x7fb5e80cbe10 unknown :-1 s=CLOSED pgs=903 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:50:04.844 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:04.841+0000 7fb5ee2cc640 1 --2- 172.21.15.43:0/3343069590 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fb5e80b2790 0x7fb5e809d080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:50:04.845 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:04.841+0000 7fb5ee2cc640 1 -- 172.21.15.43:0/3343069590 >> 172.21.15.43:0/3343069590 conn(0x7fb5e809b7b0 msgr2=0x7fb5e806adf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-09T13:50:04.845 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:04.841+0000 7fb5ee2cc640 1 -- 172.21.15.43:0/3343069590 shutdown_connections 2023-08-09T13:50:04.845 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:04.841+0000 7fb5ee2cc640 1 -- 172.21.15.43:0/3343069590 wait complete. 2023-08-09T13:50:04.862 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': 'rbd', 'create_time': '2023-08-09T13:33:54.365137+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}, 'read_balance': {'score_acting': 2, 'score_stable': 2, 'optimal_score': 1, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': '.mgr', 'create_time': '2023-08-09T13:33:54.831803+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 8, 'pool_name': 'cephfs_metadata', 'create_time': '2023-08-09T13:48:16.039756+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '73', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 9, 'pool_name': 'cephfs_data', 'create_time': '2023-08-09T13:48:16.615400+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '74', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.6299999952316284, 'score_stable': 1.6299999952316284, 'optimal_score': 1, 'raw_score_acting': 1.6299999952316284, 'raw_score_stable': 1.6299999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-08-09T13:50:04.863 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get rbd pg_num 2023-08-09T13:50:05.049 INFO:tasks.ceph.mgr.x.smithi203.stderr:2023-08-09T13:50:05.046+0000 7fbb9f40e040 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2023-08-09T13:50:05.078 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:05.075+0000 7fdc58ea3640 1 Processor -- start 2023-08-09T13:50:05.078 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:05.076+0000 7fdc58ea3640 1 -- start start 2023-08-09T13:50:05.079 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:05.077+0000 7fdc58ea3640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7fdc5415c620 0x7fdc540ce980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:50:05.079 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:05.077+0000 7fdc58ea3640 1 -- --> v1:172.21.15.43:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fdc5415ca00 con 0x7fdc5406c960 2023-08-09T13:50:05.079 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:05.077+0000 7fdc58ea3640 1 -- --> v1:172.21.15.203:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fdc5406cdc0 con 0x7fdc54069ed0 2023-08-09T13:50:05.080 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:05.077+0000 7fdc58ea3640 1 -- --> v2:172.21.15.43:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fdc540cce60 con 0x7fdc5415c620 2023-08-09T13:50:05.080 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:05.077+0000 7fdc5259c640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7fdc5415c620 0x7fdc540ce980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:50:05.081 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:05.077+0000 7fdc4bfff640 1 --1- >> v1:172.21.15.43:6789/0 conn(0x7fdc5406c960 0x7fdc540697c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.43:6789/0 says I am v1:172.21.15.43:58738/0 (socket says 172.21.15.43:58738) 2023-08-09T13:50:05.081 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:05.077+0000 7fdc4bfff640 1 -- 172.21.15.43:0/3834493775 learned_addr learned my addr 172.21.15.43:0/3834493775 (peer_addr_for_me v1:172.21.15.43:0/0) 2023-08-09T13:50:05.081 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:05.078+0000 7fdc5259c640 1 -- 172.21.15.43:0/3834493775 >> v1:172.21.15.203:6789/0 conn(0x7fdc54069ed0 legacy=0x7fdc540c9720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-09T13:50:05.081 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:05.078+0000 7fdc5259c640 1 -- 172.21.15.43:0/3834493775 >> v1:172.21.15.43:6789/0 conn(0x7fdc5406c960 legacy=0x7fdc540697c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:50:05.081 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:05.078+0000 7fdc5259c640 1 -- 172.21.15.43:0/3834493775 --> v2:172.21.15.43:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdc540cd140 con 0x7fdc5415c620 2023-08-09T13:50:05.081 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:05.078+0000 7fdc51d9b640 1 -- 172.21.15.43:0/3834493775 <== mon.0 v1:172.21.15.43:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1785167506 0 0) 0x7fdc5415ca00 con 0x7fdc5406c960 2023-08-09T13:50:05.082 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:05.079+0000 7fdc5259c640 1 --2- 172.21.15.43:0/3834493775 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fdc5415c620 0x7fdc540ce980 secure :-1 s=READY pgs=905 cs=0 l=1 rev1=1 crypto rx=0x7fdc440097a0 tx=0x7fdc4402b960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d5aafd44430aaef6 server_cookie=0 in_seq=0 out_seq=0 2023-08-09T13:50:05.082 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:05.079+0000 7fdc51d9b640 1 -- 172.21.15.43:0/3834493775 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdc44030610 con 0x7fdc5415c620 2023-08-09T13:50:05.082 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:05.079+0000 7fdc51d9b640 1 -- 172.21.15.43:0/3834493775 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdc44030770 con 0x7fdc5415c620 2023-08-09T13:50:05.082 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:05.079+0000 7fdc51d9b640 1 -- 172.21.15.43:0/3834493775 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdc44030b00 con 0x7fdc5415c620 2023-08-09T13:50:05.083 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:05.080+0000 7fdc58ea3640 1 -- 172.21.15.43:0/3834493775 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fdc5415c620 msgr2=0x7fdc540ce980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:50:05.083 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:05.080+0000 7fdc58ea3640 1 --2- 172.21.15.43:0/3834493775 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fdc5415c620 0x7fdc540ce980 secure :-1 s=READY pgs=905 cs=0 l=1 rev1=1 crypto rx=0x7fdc440097a0 tx=0x7fdc4402b960 comp rx=0 tx=0).stop 2023-08-09T13:50:05.083 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:05.080+0000 7fdc58ea3640 1 -- 172.21.15.43:0/3834493775 shutdown_connections 2023-08-09T13:50:05.084 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:05.080+0000 7fdc58ea3640 1 --2- 172.21.15.43:0/3834493775 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fdc5415c620 0x7fdc540ce980 unknown :-1 s=CLOSED pgs=905 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:50:05.084 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:05.080+0000 7fdc58ea3640 1 -- 172.21.15.43:0/3834493775 >> 172.21.15.43:0/3834493775 conn(0x7fdc540b0cc0 msgr2=0x7fdc540b10d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-09T13:50:05.084 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:05.080+0000 7fdc58ea3640 1 -- 172.21.15.43:0/3834493775 shutdown_connections 2023-08-09T13:50:05.084 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:05.080+0000 7fdc58ea3640 1 -- 172.21.15.43:0/3834493775 wait complete. 2023-08-09T13:50:05.085 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:05.082+0000 7fdc58ea3640 1 Processor -- start 2023-08-09T13:50:05.085 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:05.082+0000 7fdc58ea3640 1 -- start start 2023-08-09T13:50:05.086 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:05.083+0000 7fdc58ea3640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fdc54069ed0 0x7fdc5414c620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:50:05.086 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:05.083+0000 7fdc58ea3640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fdc5406c960 0x7fdc540c4460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:50:05.086 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:05.083+0000 7fdc58ea3640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fdc5415c620 0x7fdc540c49a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:50:05.086 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:05.083+0000 7fdc58ea3640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdc541f2fb0 con 0x7fdc5415c620 2023-08-09T13:50:05.087 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:05.083+0000 7fdc58ea3640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdc540c2ba0 con 0x7fdc54069ed0 2023-08-09T13:50:05.087 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:05.083+0000 7fdc58ea3640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdc540c2e80 con 0x7fdc5406c960 2023-08-09T13:50:05.087 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:05.083+0000 7fdc52d9d640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fdc5415c620 0x7fdc540c49a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:50:05.087 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:05.083+0000 7fdc52d9d640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fdc5415c620 0x7fdc540c49a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:45266/0 (socket says 172.21.15.43:45266) 2023-08-09T13:50:05.087 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:05.083+0000 7fdc52d9d640 1 -- 172.21.15.43:0/3822719133 learned_addr learned my addr 172.21.15.43:0/3822719133 (peer_addr_for_me v2:172.21.15.43:0/0) 2023-08-09T13:50:05.087 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:05.084+0000 7fdc5259c640 1 --2- 172.21.15.43:0/3822719133 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fdc54069ed0 0x7fdc5414c620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:50:05.087 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:05.084+0000 7fdc4bfff640 1 --2- 172.21.15.43:0/3822719133 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fdc5406c960 0x7fdc540c4460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:50:05.089 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:05.084+0000 7fdc52d9d640 1 -- 172.21.15.43:0/3822719133 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fdc5406c960 msgr2=0x7fdc540c4460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:50:05.090 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:05.084+0000 7fdc52d9d640 1 --2- 172.21.15.43:0/3822719133 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fdc5406c960 0x7fdc540c4460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:50:05.090 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:05.084+0000 7fdc52d9d640 1 -- 172.21.15.43:0/3822719133 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fdc54069ed0 msgr2=0x7fdc5414c620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:50:05.091 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:05.084+0000 7fdc52d9d640 1 --2- 172.21.15.43:0/3822719133 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fdc54069ed0 0x7fdc5414c620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:50:05.091 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:05.084+0000 7fdc52d9d640 1 -- 172.21.15.43:0/3822719133 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdc44009400 con 0x7fdc5415c620 2023-08-09T13:50:05.091 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:05.084+0000 7fdc52d9d640 1 --2- 172.21.15.43:0/3822719133 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fdc5415c620 0x7fdc540c49a0 secure :-1 s=READY pgs=906 cs=0 l=1 rev1=1 crypto rx=0x7fdc4c00c680 tx=0x7fdc4c00cb40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-09T13:50:05.091 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:05.085+0000 7fdc4affd640 1 -- 172.21.15.43:0/3822719133 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdc4c003b70 con 0x7fdc5415c620 2023-08-09T13:50:05.091 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:05.085+0000 7fdc4affd640 1 -- 172.21.15.43:0/3822719133 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdc4c003cd0 con 0x7fdc5415c620 2023-08-09T13:50:05.091 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:05.085+0000 7fdc4affd640 1 -- 172.21.15.43:0/3822719133 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdc4c005310 con 0x7fdc5415c620 2023-08-09T13:50:05.091 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:05.085+0000 7fdc58ea3640 1 -- 172.21.15.43:0/3822719133 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdc540c3160 con 0x7fdc5415c620 2023-08-09T13:50:05.091 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:05.085+0000 7fdc58ea3640 1 -- 172.21.15.43:0/3822719133 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdc5414cb60 con 0x7fdc5415c620 2023-08-09T13:50:05.092 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:05.086+0000 7fdc58ea3640 1 -- 172.21.15.43:0/3822719133 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdc540739f0 con 0x7fdc5415c620 2023-08-09T13:50:05.092 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:05.087+0000 7fdc4affd640 1 -- 172.21.15.43:0/3822719133 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 25) v1 ==== 88557+0+0 (secure 0 0 0) 0x7fdc4c005470 con 0x7fdc5415c620 2023-08-09T13:50:05.093 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:05.088+0000 7fdc4affd640 1 --2- 172.21.15.43:0/3822719133 >> [v2:172.21.15.203:6832/2515481191,v1:172.21.15.203:6833/2515481191] conn(0x7fdc3406a290 0x7fdc3406c750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:50:05.093 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:05.088+0000 7fdc4affd640 1 -- 172.21.15.43:0/3822719133 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(75..75 src has 1..75) v4 ==== 6770+0+0 (secure 0 0 0) 0x7fdc4c088a40 con 0x7fdc5415c620 2023-08-09T13:50:05.093 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:05.088+0000 7fdc5259c640 1 -- 172.21.15.43:0/3822719133 >> [v2:172.21.15.203:6832/2515481191,v1:172.21.15.203:6833/2515481191] conn(0x7fdc3406a290 msgr2=0x7fdc3406c750 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.203:6832/2515481191 2023-08-09T13:50:05.093 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:05.088+0000 7fdc5259c640 1 --2- 172.21.15.43:0/3822719133 >> [v2:172.21.15.203:6832/2515481191,v1:172.21.15.203:6833/2515481191] conn(0x7fdc3406a290 0x7fdc3406c750 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-08-09T13:50:05.093 INFO:tasks.ceph.mgr.y.smithi043.stderr:2023-08-09T13:50:05.089+0000 7f76db0f8040 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2023-08-09T13:50:05.095 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:05.093+0000 7fdc4affd640 1 -- 172.21.15.43:0/3822719133 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178066 (secure 0 0 0) 0x7fdc4c053a60 con 0x7fdc5415c620 2023-08-09T13:50:05.170 INFO:tasks.ceph.mgr.x.smithi203.stderr:2023-08-09T13:50:05.167+0000 7fbb9f40e040 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2023-08-09T13:50:05.219 INFO:tasks.ceph.mgr.y.smithi043.stderr:2023-08-09T13:50:05.216+0000 7f76db0f8040 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2023-08-09T13:50:05.226 INFO:tasks.ceph.mgr.x.smithi203.stderr:2023-08-09T13:50:05.224+0000 7fbb9f40e040 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2023-08-09T13:50:05.276 INFO:tasks.ceph.mgr.x.smithi203.stderr:2023-08-09T13:50:05.274+0000 7fbb9f40e040 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2023-08-09T13:50:05.278 INFO:tasks.ceph.mgr.y.smithi043.stderr:2023-08-09T13:50:05.276+0000 7f76db0f8040 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2023-08-09T13:50:05.287 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:05.285+0000 7fdc58ea3640 1 -- 172.21.15.43:0/3822719133 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"} v 0) v1 -- 0x7fdc5415fbf0 con 0x7fdc5415c620 2023-08-09T13:50:05.288 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:05.285+0000 7fdc4affd640 1 -- 172.21.15.43:0/3822719133 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"}]=0 v75) v1 ==== 92+0+10 (secure 0 0 0) 0x7fdc4c057080 con 0x7fdc5415c620 2023-08-09T13:50:05.288 INFO:teuthology.orchestra.run.smithi043.stdout:pg_num: 8 2023-08-09T13:50:05.290 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:05.288+0000 7fdc58ea3640 1 -- 172.21.15.43:0/3822719133 >> [v2:172.21.15.203:6832/2515481191,v1:172.21.15.203:6833/2515481191] conn(0x7fdc3406a290 msgr2=0x7fdc3406c750 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-08-09T13:50:05.290 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:05.288+0000 7fdc58ea3640 1 --2- 172.21.15.43:0/3822719133 >> [v2:172.21.15.203:6832/2515481191,v1:172.21.15.203:6833/2515481191] conn(0x7fdc3406a290 0x7fdc3406c750 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:50:05.291 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:05.288+0000 7fdc58ea3640 1 -- 172.21.15.43:0/3822719133 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fdc5415c620 msgr2=0x7fdc540c49a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:50:05.291 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:05.288+0000 7fdc58ea3640 1 --2- 172.21.15.43:0/3822719133 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fdc5415c620 0x7fdc540c49a0 secure :-1 s=READY pgs=906 cs=0 l=1 rev1=1 crypto rx=0x7fdc4c00c680 tx=0x7fdc4c00cb40 comp rx=0 tx=0).stop 2023-08-09T13:50:05.291 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:05.288+0000 7fdc58ea3640 1 -- 172.21.15.43:0/3822719133 shutdown_connections 2023-08-09T13:50:05.291 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:05.288+0000 7fdc58ea3640 1 --2- 172.21.15.43:0/3822719133 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fdc5415c620 0x7fdc540c49a0 unknown :-1 s=CLOSED pgs=906 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:50:05.291 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:05.288+0000 7fdc58ea3640 1 --2- 172.21.15.43:0/3822719133 >> [v2:172.21.15.203:6832/2515481191,v1:172.21.15.203:6833/2515481191] conn(0x7fdc3406a290 0x7fdc3406c750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:50:05.292 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:05.288+0000 7fdc58ea3640 1 --2- 172.21.15.43:0/3822719133 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fdc5406c960 0x7fdc540c4460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:50:05.292 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:05.288+0000 7fdc58ea3640 1 --2- 172.21.15.43:0/3822719133 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fdc54069ed0 0x7fdc5414c620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:50:05.292 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:05.288+0000 7fdc58ea3640 1 -- 172.21.15.43:0/3822719133 >> 172.21.15.43:0/3822719133 conn(0x7fdc540b0cc0 msgr2=0x7fdc540aec30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-09T13:50:05.292 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:05.288+0000 7fdc58ea3640 1 -- 172.21.15.43:0/3822719133 shutdown_connections 2023-08-09T13:50:05.292 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:05.288+0000 7fdc58ea3640 1 -- 172.21.15.43:0/3822719133 wait complete. 2023-08-09T13:50:05.310 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-08-09T13:50:05.332 INFO:tasks.ceph.mgr.y.smithi043.stderr:2023-08-09T13:50:05.330+0000 7f76db0f8040 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2023-08-09T13:50:05.524 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:05.521+0000 7f5a183d6640 1 Processor -- start 2023-08-09T13:50:05.524 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:05.521+0000 7f5a183d6640 1 -- start start 2023-08-09T13:50:05.526 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:05.523+0000 7f5a183d6640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f5a100cb930 0x7f5a100cd2e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:50:05.526 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:05.523+0000 7f5a183d6640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f5a100add90 0x7f5a100b3bd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:50:05.527 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:05.523+0000 7f5a183d6640 1 -- --> v1:172.21.15.203:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5a100cd820 con 0x7f5a100b4110 2023-08-09T13:50:05.527 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:05.523+0000 7f5a183d6640 1 -- --> v2:172.21.15.43:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5a100b45a0 con 0x7f5a100cb930 2023-08-09T13:50:05.527 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:05.523+0000 7f5a183d6640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5a1015abc0 con 0x7f5a100add90 2023-08-09T13:50:05.527 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:05.524+0000 7f5a16172640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f5a100cb930 0x7f5a100cd2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:50:05.527 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:05.524+0000 7f5a16172640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f5a100cb930 0x7f5a100cd2e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:45282/0 (socket says 172.21.15.43:45282) 2023-08-09T13:50:05.527 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:05.524+0000 7f5a16172640 1 -- 172.21.15.43:0/3351102219 learned_addr learned my addr 172.21.15.43:0/3351102219 (peer_addr_for_me v2:172.21.15.43:0/0) 2023-08-09T13:50:05.528 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:05.524+0000 7f5a15971640 1 --2- 172.21.15.43:0/3351102219 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f5a100add90 0x7f5a100b3bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:50:05.528 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:05.524+0000 7f5a16172640 1 -- 172.21.15.43:0/3351102219 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f5a100add90 msgr2=0x7f5a100b3bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-09T13:50:05.528 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:05.524+0000 7f5a16172640 1 --2- 172.21.15.43:0/3351102219 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f5a100add90 0x7f5a100b3bd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:50:05.529 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:05.524+0000 7f5a16172640 1 -- 172.21.15.43:0/3351102219 >> v1:172.21.15.203:6789/0 conn(0x7f5a100b4110 legacy=0x7f5a1015e140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-09T13:50:05.529 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:05.524+0000 7f5a16172640 1 -- 172.21.15.43:0/3351102219 --> v2:172.21.15.43:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5a1015aea0 con 0x7f5a100cb930 2023-08-09T13:50:05.529 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:05.525+0000 7f5a16172640 1 --2- 172.21.15.43:0/3351102219 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f5a100cb930 0x7f5a100cd2e0 secure :-1 s=READY pgs=907 cs=0 l=1 rev1=1 crypto rx=0x7f5a0c009870 tx=0x7f5a0c02b860 comp rx=0 tx=0).ready entity=mon.0 client_cookie=eea3f7b5632b41e3 server_cookie=0 in_seq=0 out_seq=0 2023-08-09T13:50:05.529 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:05.526+0000 7f5a15170640 1 -- 172.21.15.43:0/3351102219 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5a0c0317c0 con 0x7f5a100cb930 2023-08-09T13:50:05.529 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:05.526+0000 7f5a15170640 1 -- 172.21.15.43:0/3351102219 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5a0c031920 con 0x7f5a100cb930 2023-08-09T13:50:05.529 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:05.526+0000 7f5a15170640 1 -- 172.21.15.43:0/3351102219 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5a0c031cb0 con 0x7f5a100cb930 2023-08-09T13:50:05.530 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:05.526+0000 7f5a183d6640 1 -- 172.21.15.43:0/3351102219 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f5a100cb930 msgr2=0x7f5a100cd2e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:50:05.530 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:05.526+0000 7f5a183d6640 1 --2- 172.21.15.43:0/3351102219 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f5a100cb930 0x7f5a100cd2e0 secure :-1 s=READY pgs=907 cs=0 l=1 rev1=1 crypto rx=0x7f5a0c009870 tx=0x7f5a0c02b860 comp rx=0 tx=0).stop 2023-08-09T13:50:05.530 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:05.527+0000 7f5a183d6640 1 -- 172.21.15.43:0/3351102219 shutdown_connections 2023-08-09T13:50:05.530 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:05.527+0000 7f5a183d6640 1 --2- 172.21.15.43:0/3351102219 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f5a100add90 0x7f5a100b3bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:50:05.531 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:05.527+0000 7f5a183d6640 1 --2- 172.21.15.43:0/3351102219 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f5a100cb930 0x7f5a100cd2e0 unknown :-1 s=CLOSED pgs=907 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:50:05.531 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:05.527+0000 7f5a183d6640 1 -- 172.21.15.43:0/3351102219 >> 172.21.15.43:0/3351102219 conn(0x7f5a100697c0 msgr2=0x7f5a10069ba0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-09T13:50:05.531 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:05.527+0000 7f5a183d6640 1 -- 172.21.15.43:0/3351102219 shutdown_connections 2023-08-09T13:50:05.531 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:05.527+0000 7f5a183d6640 1 -- 172.21.15.43:0/3351102219 wait complete. 2023-08-09T13:50:05.531 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:05.529+0000 7f5a183d6640 1 Processor -- start 2023-08-09T13:50:05.532 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:05.529+0000 7f5a183d6640 1 -- start start 2023-08-09T13:50:05.532 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:05.530+0000 7f5a183d6640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f5a100add90 0x7f5a101f1800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:50:05.533 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:05.530+0000 7f5a183d6640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f5a100b4110 0x7f5a100a74b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:50:05.533 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:05.530+0000 7f5a183d6640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f5a100cb930 0x7f5a101528e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:50:05.533 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:05.530+0000 7f5a183d6640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5a100a79f0 con 0x7f5a100b4110 2023-08-09T13:50:05.533 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:05.530+0000 7f5a183d6640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5a100a8810 con 0x7f5a100cb930 2023-08-09T13:50:05.533 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:05.530+0000 7f5a183d6640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5a100a7c00 con 0x7f5a100add90 2023-08-09T13:50:05.533 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:05.530+0000 7f5a15971640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f5a100b4110 0x7f5a100a74b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:50:05.534 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:05.530+0000 7f5a15971640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f5a100b4110 0x7f5a100a74b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:45292/0 (socket says 172.21.15.43:45292) 2023-08-09T13:50:05.534 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:05.530+0000 7f5a15971640 1 -- 172.21.15.43:0/797470521 learned_addr learned my addr 172.21.15.43:0/797470521 (peer_addr_for_me v2:172.21.15.43:0/0) 2023-08-09T13:50:05.534 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:05.530+0000 7f5a16172640 1 --2- 172.21.15.43:0/797470521 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f5a100add90 0x7f5a101f1800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:50:05.534 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:05.530+0000 7f5a16973640 1 --2- 172.21.15.43:0/797470521 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f5a100cb930 0x7f5a101528e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:50:05.535 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:05.531+0000 7f5a15971640 1 -- 172.21.15.43:0/797470521 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f5a100add90 msgr2=0x7f5a101f1800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:50:05.535 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:05.531+0000 7f5a15971640 1 --2- 172.21.15.43:0/797470521 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f5a100add90 0x7f5a101f1800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:50:05.535 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:05.531+0000 7f5a15971640 1 -- 172.21.15.43:0/797470521 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f5a100cb930 msgr2=0x7f5a101528e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:50:05.535 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:05.531+0000 7f5a15971640 1 --2- 172.21.15.43:0/797470521 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f5a100cb930 0x7f5a101528e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:50:05.535 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:05.531+0000 7f5a15971640 1 -- 172.21.15.43:0/797470521 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5a0c0094d0 con 0x7f5a100b4110 2023-08-09T13:50:05.536 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:05.531+0000 7f5a15971640 1 --2- 172.21.15.43:0/797470521 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f5a100b4110 0x7f5a100a74b0 secure :-1 s=READY pgs=908 cs=0 l=1 rev1=1 crypto rx=0x7f5a0000d800 tx=0x7f5a0000dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-09T13:50:05.536 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:05.531+0000 7f5a06ffd640 1 -- 172.21.15.43:0/797470521 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5a0000b6c0 con 0x7f5a100b4110 2023-08-09T13:50:05.536 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:05.531+0000 7f5a06ffd640 1 -- 172.21.15.43:0/797470521 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5a0000b820 con 0x7f5a100b4110 2023-08-09T13:50:05.536 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:05.531+0000 7f5a06ffd640 1 -- 172.21.15.43:0/797470521 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5a0000bbb0 con 0x7f5a100b4110 2023-08-09T13:50:05.536 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:05.531+0000 7f5a183d6640 1 -- 172.21.15.43:0/797470521 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5a100a7ee0 con 0x7f5a100b4110 2023-08-09T13:50:05.536 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:05.531+0000 7f5a183d6640 1 -- 172.21.15.43:0/797470521 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5a100a84a0 con 0x7f5a100b4110 2023-08-09T13:50:05.536 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:05.533+0000 7f5a183d6640 1 -- 172.21.15.43:0/797470521 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5a100a89a0 con 0x7f5a100b4110 2023-08-09T13:50:05.537 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:05.534+0000 7f5a06ffd640 1 -- 172.21.15.43:0/797470521 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 25) v1 ==== 88557+0+0 (secure 0 0 0) 0x7f5a0000bd10 con 0x7f5a100b4110 2023-08-09T13:50:05.537 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:05.535+0000 7f5a06ffd640 1 --2- 172.21.15.43:0/797470521 >> [v2:172.21.15.203:6832/2515481191,v1:172.21.15.203:6833/2515481191] conn(0x7f59f406a290 0x7f59f406c750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:50:05.538 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:05.535+0000 7f5a06ffd640 1 -- 172.21.15.43:0/797470521 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(75..75 src has 1..75) v4 ==== 6770+0+0 (secure 0 0 0) 0x7f5a00089220 con 0x7f5a100b4110 2023-08-09T13:50:05.538 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:05.535+0000 7f5a16172640 1 -- 172.21.15.43:0/797470521 >> [v2:172.21.15.203:6832/2515481191,v1:172.21.15.203:6833/2515481191] conn(0x7f59f406a290 msgr2=0x7f59f406c750 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.203:6832/2515481191 2023-08-09T13:50:05.538 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:05.536+0000 7f5a16172640 1 --2- 172.21.15.43:0/797470521 >> [v2:172.21.15.203:6832/2515481191,v1:172.21.15.203:6833/2515481191] conn(0x7f59f406a290 0x7f59f406c750 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-08-09T13:50:05.542 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:05.540+0000 7f5a06ffd640 1 -- 172.21.15.43:0/797470521 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178066 (secure 0 0 0) 0x7f5a000542e0 con 0x7f5a100b4110 2023-08-09T13:50:05.737 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:05.734+0000 7f5a183d6640 1 -- 172.21.15.43:0/797470521 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f5a1006a7d0 con 0x7f5a100b4110 2023-08-09T13:50:05.737 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:05.735+0000 7f5a06ffd640 1 -- 172.21.15.43:0/797470521 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v75) v1 ==== 93+0+10 (secure 0 0 0) 0x7f5a00057900 con 0x7f5a100b4110 2023-08-09T13:50:05.737 INFO:teuthology.orchestra.run.smithi043.stdout:pg_num: 1 2023-08-09T13:50:05.739 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:05.737+0000 7f5a16172640 1 -- 172.21.15.43:0/797470521 >> [v2:172.21.15.203:6832/2515481191,v1:172.21.15.203:6833/2515481191] conn(0x7f59f406a290 msgr2=0x7f59f406c750 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.203:6832/2515481191 2023-08-09T13:50:05.739 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:05.737+0000 7f5a16172640 1 --2- 172.21.15.43:0/797470521 >> [v2:172.21.15.203:6832/2515481191,v1:172.21.15.203:6833/2515481191] conn(0x7f59f406a290 0x7f59f406c750 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2023-08-09T13:50:05.740 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:05.737+0000 7f5a183d6640 1 -- 172.21.15.43:0/797470521 >> [v2:172.21.15.203:6832/2515481191,v1:172.21.15.203:6833/2515481191] conn(0x7f59f406a290 msgr2=0x7f59f406c750 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-08-09T13:50:05.740 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:05.737+0000 7f5a183d6640 1 --2- 172.21.15.43:0/797470521 >> [v2:172.21.15.203:6832/2515481191,v1:172.21.15.203:6833/2515481191] conn(0x7f59f406a290 0x7f59f406c750 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:50:05.740 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:05.737+0000 7f5a183d6640 1 -- 172.21.15.43:0/797470521 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f5a100b4110 msgr2=0x7f5a100a74b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:50:05.740 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:05.737+0000 7f5a183d6640 1 --2- 172.21.15.43:0/797470521 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f5a100b4110 0x7f5a100a74b0 secure :-1 s=READY pgs=908 cs=0 l=1 rev1=1 crypto rx=0x7f5a0000d800 tx=0x7f5a0000dcc0 comp rx=0 tx=0).stop 2023-08-09T13:50:05.740 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:05.737+0000 7f5a183d6640 1 -- 172.21.15.43:0/797470521 shutdown_connections 2023-08-09T13:50:05.741 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:05.737+0000 7f5a183d6640 1 --2- 172.21.15.43:0/797470521 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f5a100cb930 0x7f5a101528e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:50:05.741 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:05.737+0000 7f5a183d6640 1 --2- 172.21.15.43:0/797470521 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f5a100b4110 0x7f5a100a74b0 unknown :-1 s=CLOSED pgs=908 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:50:05.741 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:05.737+0000 7f5a183d6640 1 --2- 172.21.15.43:0/797470521 >> [v2:172.21.15.203:6832/2515481191,v1:172.21.15.203:6833/2515481191] conn(0x7f59f406a290 0x7f59f406c750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:50:05.741 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:05.737+0000 7f5a183d6640 1 --2- 172.21.15.43:0/797470521 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f5a100add90 0x7f5a101f1800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:50:05.741 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:05.737+0000 7f5a183d6640 1 -- 172.21.15.43:0/797470521 >> 172.21.15.43:0/797470521 conn(0x7f5a100697c0 msgr2=0x7f5a1015c820 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-09T13:50:05.741 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:05.737+0000 7f5a183d6640 1 -- 172.21.15.43:0/797470521 shutdown_connections 2023-08-09T13:50:05.741 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:05.737+0000 7f5a183d6640 1 -- 172.21.15.43:0/797470521 wait complete. 2023-08-09T13:50:05.757 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-08-09T13:50:05.960 INFO:tasks.ceph.mgr.x.smithi203.stderr:2023-08-09T13:50:05.958+0000 7fbb9f40e040 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2023-08-09T13:50:06.014 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:06.011+0000 7f79202a9640 1 Processor -- start 2023-08-09T13:50:06.014 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:06.012+0000 7f79202a9640 1 -- start start 2023-08-09T13:50:06.015 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:06.013+0000 7f79202a9640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f79180ad3e0 0x7f79180b03f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:50:06.016 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:06.013+0000 7f79202a9640 1 -- --> v1:172.21.15.43:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f79180b0e20 con 0x7f79180ccc50 2023-08-09T13:50:06.016 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:06.013+0000 7f79202a9640 1 -- --> v1:172.21.15.203:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f791806bf90 con 0x7f79180b0930 2023-08-09T13:50:06.016 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:06.013+0000 7f79202a9640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f791806c270 con 0x7f79180ad3e0 2023-08-09T13:50:06.018 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:06.014+0000 7f791e846640 1 --1- >> v1:172.21.15.43:6789/0 conn(0x7f79180ccc50 0x7f791806cd50 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.43:6789/0 says I am v1:172.21.15.43:58748/0 (socket says 172.21.15.43:58748) 2023-08-09T13:50:06.019 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:06.014+0000 7f791e846640 1 -- 172.21.15.43:0/3428650904 learned_addr learned my addr 172.21.15.43:0/3428650904 (peer_addr_for_me v1:172.21.15.43:0/0) 2023-08-09T13:50:06.019 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:06.014+0000 7f791e045640 1 --2- 172.21.15.43:0/3428650904 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f79180ad3e0 0x7f79180b03f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:50:06.019 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:06.014+0000 7f791d043640 1 -- 172.21.15.43:0/3428650904 <== mon.0 v1:172.21.15.43:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1822636631 0 0) 0x7f79180b0e20 con 0x7f79180ccc50 2023-08-09T13:50:06.019 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:06.015+0000 7f791d043640 1 -- 172.21.15.43:0/3428650904 --> v1:172.21.15.43:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f7908003520 con 0x7f79180ccc50 2023-08-09T13:50:06.019 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:06.015+0000 7f791d043640 1 -- 172.21.15.43:0/3428650904 <== mon.1 v1:172.21.15.203:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 624257297 0 0) 0x7f791806bf90 con 0x7f79180b0930 2023-08-09T13:50:06.019 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:06.015+0000 7f791d043640 1 -- 172.21.15.43:0/3428650904 --> v1:172.21.15.203:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f79180b0e20 con 0x7f79180b0930 2023-08-09T13:50:06.020 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:06.015+0000 7f791d043640 1 -- 172.21.15.43:0/3428650904 <== mon.0 v1:172.21.15.43:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 14476379 0 0) 0x7f7908003520 con 0x7f79180ccc50 2023-08-09T13:50:06.020 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:06.015+0000 7f791d043640 1 -- 172.21.15.43:0/3428650904 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f79180ad3e0 msgr2=0x7f79180b03f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-09T13:50:06.020 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:06.015+0000 7f791d043640 1 --2- 172.21.15.43:0/3428650904 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f79180ad3e0 0x7f79180b03f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:50:06.020 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:06.015+0000 7f791d043640 1 -- 172.21.15.43:0/3428650904 >> v1:172.21.15.203:6789/0 conn(0x7f79180b0930 legacy=0x7f79180cc540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:50:06.020 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:06.015+0000 7f791d043640 1 -- 172.21.15.43:0/3428650904 --> v1:172.21.15.43:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f791806c550 con 0x7f79180ccc50 2023-08-09T13:50:06.020 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:06.015+0000 7f791d043640 1 -- 172.21.15.43:0/3428650904 <== mon.0 v1:172.21.15.43:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1285150740 0 0) 0x7f7910001670 con 0x7f79180ccc50 2023-08-09T13:50:06.020 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:06.015+0000 7f791d043640 1 -- 172.21.15.43:0/3428650904 >> v1:172.21.15.43:6789/0 conn(0x7f79180ccc50 legacy=0x7f791806cd50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:50:06.020 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:06.015+0000 7f791d043640 1 --2- 172.21.15.43:0/3428650904 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f79180ad3e0 0x7f7908003ba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:50:06.021 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:06.015+0000 7f791d043640 1 --2- 172.21.15.43:0/3428650904 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f7908004100 0x7f79080047c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:50:06.021 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:06.015+0000 7f791d043640 1 --2- 172.21.15.43:0/3428650904 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f7908004d00 0x7f7908005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:50:06.021 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:06.015+0000 7f791d043640 1 -- 172.21.15.43:0/3428650904 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f791806c270 con 0x7f7908004100 2023-08-09T13:50:06.021 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:06.015+0000 7f791d043640 1 -- 172.21.15.43:0/3428650904 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f790800bac0 con 0x7f7908004d00 2023-08-09T13:50:06.021 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:06.015+0000 7f791d043640 1 -- 172.21.15.43:0/3428650904 --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f790800bc80 con 0x7f79180ad3e0 2023-08-09T13:50:06.021 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:06.016+0000 7f791d844640 1 --2- 172.21.15.43:0/3428650904 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f7908004100 0x7f79080047c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:50:06.021 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:06.016+0000 7f791e846640 1 --2- 172.21.15.43:0/3428650904 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f7908004d00 0x7f7908005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:50:06.021 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:06.016+0000 7f791e045640 1 --2- 172.21.15.43:0/3428650904 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f79180ad3e0 0x7f7908003ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:50:06.022 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:06.016+0000 7f791d844640 1 -- 172.21.15.43:0/3428650904 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f79180ad3e0 msgr2=0x7f7908003ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-09T13:50:06.022 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:06.016+0000 7f791d844640 1 --2- 172.21.15.43:0/3428650904 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f79180ad3e0 0x7f7908003ba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:50:06.022 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:06.016+0000 7f791d844640 1 -- 172.21.15.43:0/3428650904 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f7908004d00 msgr2=0x7f7908005180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-09T13:50:06.022 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:06.016+0000 7f791d844640 1 --2- 172.21.15.43:0/3428650904 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f7908004d00 0x7f7908005180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:50:06.022 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:06.016+0000 7f791d844640 1 -- 172.21.15.43:0/3428650904 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7908003520 con 0x7f7908004100 2023-08-09T13:50:06.023 INFO:tasks.ceph.mgr.x.smithi203.stderr:2023-08-09T13:50:06.019+0000 7fbb9f40e040 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2023-08-09T13:50:06.025 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:06.017+0000 7f791d844640 1 --2- 172.21.15.43:0/3428650904 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f7908004100 0x7f79080047c0 secure :-1 s=READY pgs=910 cs=0 l=1 rev1=1 crypto rx=0x7f79040018a0 tx=0x7f79080057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8df9cc71ddd5cc21 server_cookie=0 in_seq=0 out_seq=0 2023-08-09T13:50:06.025 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:06.017+0000 7f791d043640 1 -- 172.21.15.43:0/3428650904 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f79040052d0 con 0x7f7908004100 2023-08-09T13:50:06.025 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:06.018+0000 7f791d043640 1 -- 172.21.15.43:0/3428650904 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f790402e050 con 0x7f7908004100 2023-08-09T13:50:06.025 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:06.018+0000 7f791d043640 1 -- 172.21.15.43:0/3428650904 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7904003810 con 0x7f7908004100 2023-08-09T13:50:06.025 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:06.018+0000 7f79202a9640 1 -- 172.21.15.43:0/3428650904 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f7908004100 msgr2=0x7f79080047c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:50:06.026 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:06.018+0000 7f79202a9640 1 --2- 172.21.15.43:0/3428650904 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f7908004100 0x7f79080047c0 secure :-1 s=READY pgs=910 cs=0 l=1 rev1=1 crypto rx=0x7f79040018a0 tx=0x7f79080057f0 comp rx=0 tx=0).stop 2023-08-09T13:50:06.026 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:06.018+0000 7f791e846640 1 -- 172.21.15.43:0/3428650904 reap_dead start 2023-08-09T13:50:06.026 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:06.018+0000 7f79202a9640 1 -- 172.21.15.43:0/3428650904 shutdown_connections 2023-08-09T13:50:06.026 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:06.018+0000 7f79202a9640 1 -- 172.21.15.43:0/3428650904 >> 172.21.15.43:0/3428650904 conn(0x7f79180cb3f0 msgr2=0x7f79180cb800 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-09T13:50:06.026 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:06.018+0000 7f79202a9640 1 -- 172.21.15.43:0/3428650904 shutdown_connections 2023-08-09T13:50:06.026 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:06.019+0000 7f79202a9640 1 -- 172.21.15.43:0/3428650904 wait complete. 2023-08-09T13:50:06.026 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:06.020+0000 7f79202a9640 1 Processor -- start 2023-08-09T13:50:06.027 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:06.021+0000 7f79202a9640 1 -- start start 2023-08-09T13:50:06.027 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:06.021+0000 7f79202a9640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f7908004100 0x7f791814bcb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:50:06.027 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:06.021+0000 7f79202a9640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f791806d150 0x7f791806d5d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:50:06.027 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:06.022+0000 7f79202a9640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f79180afb80 0x7f79180b0000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:50:06.027 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:06.022+0000 7f79202a9640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f791806db10 con 0x7f791806d150 2023-08-09T13:50:06.027 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:06.022+0000 7f79202a9640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f79180c5b70 con 0x7f79180afb80 2023-08-09T13:50:06.027 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:06.022+0000 7f79202a9640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f79180c5e50 con 0x7f7908004100 2023-08-09T13:50:06.028 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:06.022+0000 7f791d844640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f791806d150 0x7f791806d5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:50:06.028 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:06.022+0000 7f791d844640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f791806d150 0x7f791806d5d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:45300/0 (socket says 172.21.15.43:45300) 2023-08-09T13:50:06.028 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:06.022+0000 7f791d844640 1 -- 172.21.15.43:0/4174596710 learned_addr learned my addr 172.21.15.43:0/4174596710 (peer_addr_for_me v2:172.21.15.43:0/0) 2023-08-09T13:50:06.028 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:06.022+0000 7f791e045640 1 --2- 172.21.15.43:0/4174596710 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f7908004100 0x7f791814bcb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:50:06.028 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:06.022+0000 7f791e846640 1 --2- 172.21.15.43:0/4174596710 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f79180afb80 0x7f79180b0000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:50:06.028 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:06.022+0000 7f791d844640 1 -- 172.21.15.43:0/4174596710 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f7908004100 msgr2=0x7f791814bcb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:50:06.028 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:06.022+0000 7f791d844640 1 --2- 172.21.15.43:0/4174596710 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f7908004100 0x7f791814bcb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:50:06.028 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:06.022+0000 7f791d844640 1 -- 172.21.15.43:0/4174596710 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f79180afb80 msgr2=0x7f79180b0000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:50:06.029 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:06.022+0000 7f791d844640 1 --2- 172.21.15.43:0/4174596710 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f79180afb80 0x7f79180b0000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:50:06.029 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:06.023+0000 7f791d844640 1 -- 172.21.15.43:0/4174596710 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f79180aba50 con 0x7f791806d150 2023-08-09T13:50:06.030 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:06.023+0000 7f791d844640 1 --2- 172.21.15.43:0/4174596710 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f791806d150 0x7f791806d5d0 secure :-1 s=READY pgs=911 cs=0 l=1 rev1=1 crypto rx=0x7f7904034040 tx=0x7f79040012d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-09T13:50:06.030 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:06.023+0000 7f7916ffd640 1 -- 172.21.15.43:0/4174596710 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7904007250 con 0x7f791806d150 2023-08-09T13:50:06.030 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:06.023+0000 7f7916ffd640 1 -- 172.21.15.43:0/4174596710 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f79040073b0 con 0x7f791806d150 2023-08-09T13:50:06.030 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:06.023+0000 7f7916ffd640 1 -- 172.21.15.43:0/4174596710 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f79040035f0 con 0x7f791806d150 2023-08-09T13:50:06.030 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:06.023+0000 7f79202a9640 1 -- 172.21.15.43:0/4174596710 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f79180c6130 con 0x7f791806d150 2023-08-09T13:50:06.031 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:06.023+0000 7f79202a9640 1 -- 172.21.15.43:0/4174596710 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f79180c66f0 con 0x7f791806d150 2023-08-09T13:50:06.031 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:06.025+0000 7f79202a9640 1 -- 172.21.15.43:0/4174596710 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f79180672d0 con 0x7f791806d150 2023-08-09T13:50:06.031 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:06.026+0000 7f7916ffd640 1 -- 172.21.15.43:0/4174596710 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 25) v1 ==== 88557+0+0 (secure 0 0 0) 0x7f7904003750 con 0x7f791806d150 2023-08-09T13:50:06.031 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:06.027+0000 7f7916ffd640 1 --2- 172.21.15.43:0/4174596710 >> [v2:172.21.15.203:6832/2515481191,v1:172.21.15.203:6833/2515481191] conn(0x7f78fc06a2c0 0x7f78fc06c780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:50:06.031 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:06.027+0000 7f7916ffd640 1 -- 172.21.15.43:0/4174596710 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(75..75 src has 1..75) v4 ==== 6770+0+0 (secure 0 0 0) 0x7f79040aaaa0 con 0x7f791806d150 2023-08-09T13:50:06.031 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:06.028+0000 7f791e045640 1 -- 172.21.15.43:0/4174596710 >> [v2:172.21.15.203:6832/2515481191,v1:172.21.15.203:6833/2515481191] conn(0x7f78fc06a2c0 msgr2=0x7f78fc06c780 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.203:6832/2515481191 2023-08-09T13:50:06.031 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:06.028+0000 7f791e045640 1 --2- 172.21.15.43:0/4174596710 >> [v2:172.21.15.203:6832/2515481191,v1:172.21.15.203:6833/2515481191] conn(0x7f78fc06a2c0 0x7f78fc06c780 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-08-09T13:50:06.034 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:06.032+0000 7f7916ffd640 1 -- 172.21.15.43:0/4174596710 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178066 (secure 0 0 0) 0x7f7904075bb0 con 0x7f791806d150 2023-08-09T13:50:06.072 INFO:tasks.ceph.mgr.x.smithi203.stderr:2023-08-09T13:50:06.069+0000 7fbb9f40e040 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2023-08-09T13:50:06.073 INFO:tasks.ceph.mgr.y.smithi043.stderr:2023-08-09T13:50:06.070+0000 7f76db0f8040 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2023-08-09T13:50:06.138 INFO:tasks.ceph.mgr.y.smithi043.stderr:2023-08-09T13:50:06.135+0000 7f76db0f8040 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2023-08-09T13:50:06.190 INFO:tasks.ceph.mgr.y.smithi043.stderr:2023-08-09T13:50:06.187+0000 7f76db0f8040 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2023-08-09T13:50:06.192 INFO:tasks.ceph.mgr.x.smithi203.stderr:2023-08-09T13:50:06.190+0000 7fbb9f40e040 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2023-08-09T13:50:06.224 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:06.221+0000 7f79202a9640 1 -- 172.21.15.43:0/4174596710 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f79180739f0 con 0x7f791806d150 2023-08-09T13:50:06.224 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:06.222+0000 7f7916ffd640 1 -- 172.21.15.43:0/4174596710 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v75) v1 ==== 104+0+11 (secure 0 0 0) 0x7f79040791d0 con 0x7f791806d150 2023-08-09T13:50:06.225 INFO:teuthology.orchestra.run.smithi043.stdout:pg_num: 32 2023-08-09T13:50:06.228 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:06.225+0000 7f79202a9640 1 -- 172.21.15.43:0/4174596710 >> [v2:172.21.15.203:6832/2515481191,v1:172.21.15.203:6833/2515481191] conn(0x7f78fc06a2c0 msgr2=0x7f78fc06c780 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-08-09T13:50:06.228 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:06.225+0000 7f79202a9640 1 --2- 172.21.15.43:0/4174596710 >> [v2:172.21.15.203:6832/2515481191,v1:172.21.15.203:6833/2515481191] conn(0x7f78fc06a2c0 0x7f78fc06c780 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:50:06.228 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:06.225+0000 7f79202a9640 1 -- 172.21.15.43:0/4174596710 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f791806d150 msgr2=0x7f791806d5d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:50:06.228 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:06.225+0000 7f79202a9640 1 --2- 172.21.15.43:0/4174596710 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f791806d150 0x7f791806d5d0 secure :-1 s=READY pgs=911 cs=0 l=1 rev1=1 crypto rx=0x7f7904034040 tx=0x7f79040012d0 comp rx=0 tx=0).stop 2023-08-09T13:50:06.228 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:06.225+0000 7f79202a9640 1 -- 172.21.15.43:0/4174596710 shutdown_connections 2023-08-09T13:50:06.228 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:06.225+0000 7f79202a9640 1 --2- 172.21.15.43:0/4174596710 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f79180afb80 0x7f79180b0000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:50:06.229 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:06.225+0000 7f79202a9640 1 --2- 172.21.15.43:0/4174596710 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f791806d150 0x7f791806d5d0 unknown :-1 s=CLOSED pgs=911 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:50:06.229 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:06.225+0000 7f79202a9640 1 --2- 172.21.15.43:0/4174596710 >> [v2:172.21.15.203:6832/2515481191,v1:172.21.15.203:6833/2515481191] conn(0x7f78fc06a2c0 0x7f78fc06c780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:50:06.229 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:06.225+0000 7f79202a9640 1 --2- 172.21.15.43:0/4174596710 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f7908004100 0x7f791814bcb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:50:06.229 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:06.225+0000 7f79202a9640 1 -- 172.21.15.43:0/4174596710 >> 172.21.15.43:0/4174596710 conn(0x7f79180cb3f0 msgr2=0x7f79180acc60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-09T13:50:06.229 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:06.225+0000 7f79202a9640 1 -- 172.21.15.43:0/4174596710 shutdown_connections 2023-08-09T13:50:06.229 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:06.225+0000 7f79202a9640 1 -- 172.21.15.43:0/4174596710 wait complete. 2023-08-09T13:50:06.246 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-08-09T13:50:06.316 INFO:tasks.ceph.mgr.y.smithi043.stderr:2023-08-09T13:50:06.314+0000 7f76db0f8040 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2023-08-09T13:50:06.351 INFO:tasks.ceph.mgr.x.smithi203.stderr:2023-08-09T13:50:06.348+0000 7fbb9f40e040 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2023-08-09T13:50:06.457 INFO:tasks.ceph.mgr.x.smithi203.stderr:2023-08-09T13:50:06.455+0000 7fbb9f40e040 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2023-08-09T13:50:06.460 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:06.457+0000 7f6bf2ccc640 1 Processor -- start 2023-08-09T13:50:06.460 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:06.458+0000 7f6bf2ccc640 1 -- start start 2023-08-09T13:50:06.461 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:06.459+0000 7f6bf2ccc640 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7f6bec15ca70 0x7f6bec0705e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:50:06.462 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:06.459+0000 7f6bf2ccc640 1 -- --> v1:172.21.15.43:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6bec15d990 con 0x7f6bec070b20 2023-08-09T13:50:06.462 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:06.459+0000 7f6bf2ccc640 1 -- --> v1:172.21.15.203:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6bec06ea70 con 0x7f6bec15d560 2023-08-09T13:50:06.462 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:06.459+0000 7f6bf2ccc640 1 -- --> v2:172.21.15.203:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6bec070fb0 con 0x7f6bec15ca70 2023-08-09T13:50:06.463 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:06.459+0000 7f6be3fff640 1 --1- >> v1:172.21.15.43:6789/0 conn(0x7f6bec070b20 0x7f6bec15ce50 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.43:6789/0 says I am v1:172.21.15.43:58752/0 (socket says 172.21.15.43:58752) 2023-08-09T13:50:06.463 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:06.460+0000 7f6be3fff640 1 -- 172.21.15.43:0/2083604338 learned_addr learned my addr 172.21.15.43:0/2083604338 (peer_addr_for_me v1:172.21.15.43:0/0) 2023-08-09T13:50:06.463 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:06.460+0000 7f6bf0a68640 1 --2- 172.21.15.43:0/2083604338 >> v2:172.21.15.203:3300/0 conn(0x7f6bec15ca70 0x7f6bec0705e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:50:06.463 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:06.460+0000 7f6be37fe640 1 -- 172.21.15.43:0/2083604338 <== mon.0 v1:172.21.15.43:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2822008209 0 0) 0x7f6bec15d990 con 0x7f6bec070b20 2023-08-09T13:50:06.463 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:06.460+0000 7f6be37fe640 1 -- 172.21.15.43:0/2083604338 --> v1:172.21.15.43:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f6bd4003520 con 0x7f6bec070b20 2023-08-09T13:50:06.463 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:06.460+0000 7f6bf0a68640 1 -- 172.21.15.43:0/2083604338 >> v1:172.21.15.203:6789/0 conn(0x7f6bec15d560 legacy=0x7f6bec0697c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:50:06.464 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:06.460+0000 7f6be37fe640 1 -- 172.21.15.43:0/2083604338 <== mon.0 v1:172.21.15.43:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 501693247 0 0) 0x7f6bd4003520 con 0x7f6bec070b20 2023-08-09T13:50:06.464 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:06.460+0000 7f6bf0a68640 1 -- 172.21.15.43:0/2083604338 >> v1:172.21.15.43:6789/0 conn(0x7f6bec070b20 legacy=0x7f6bec15ce50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:50:06.464 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:06.460+0000 7f6bf0a68640 1 -- 172.21.15.43:0/2083604338 --> v2:172.21.15.203:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6bec06dcb0 con 0x7f6bec15ca70 2023-08-09T13:50:06.464 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:06.462+0000 7f6bf0a68640 1 --2- 172.21.15.43:0/2083604338 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f6bec15ca70 0x7f6bec0705e0 secure :-1 s=READY pgs=326 cs=0 l=1 rev1=1 crypto rx=0x7f6be80097a0 tx=0x7f6be802b790 comp rx=0 tx=0).ready entity=mon.1 client_cookie=12e08009d66a75e8 server_cookie=0 in_seq=0 out_seq=0 2023-08-09T13:50:06.465 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:06.462+0000 7f6be37fe640 1 -- 172.21.15.43:0/2083604338 <== mon.1 v2:172.21.15.203:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6bdc001670 con 0x7f6bec15ca70 2023-08-09T13:50:06.465 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:06.462+0000 7f6be37fe640 1 -- 172.21.15.43:0/2083604338 <== mon.1 v2:172.21.15.203:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6be802e050 con 0x7f6bec15ca70 2023-08-09T13:50:06.465 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:06.462+0000 7f6be37fe640 1 -- 172.21.15.43:0/2083604338 <== mon.1 v2:172.21.15.203:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6be8030730 con 0x7f6bec15ca70 2023-08-09T13:50:06.466 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:06.463+0000 7f6bf2ccc640 1 -- 172.21.15.43:0/2083604338 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f6bec15ca70 msgr2=0x7f6bec0705e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:50:06.466 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:06.463+0000 7f6bf2ccc640 1 --2- 172.21.15.43:0/2083604338 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f6bec15ca70 0x7f6bec0705e0 secure :-1 s=READY pgs=326 cs=0 l=1 rev1=1 crypto rx=0x7f6be80097a0 tx=0x7f6be802b790 comp rx=0 tx=0).stop 2023-08-09T13:50:06.466 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:06.463+0000 7f6bf2ccc640 1 -- 172.21.15.43:0/2083604338 shutdown_connections 2023-08-09T13:50:06.466 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:06.463+0000 7f6bf2ccc640 1 --2- 172.21.15.43:0/2083604338 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f6bec15ca70 0x7f6bec0705e0 unknown :-1 s=CLOSED pgs=326 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:50:06.466 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:06.463+0000 7f6bf2ccc640 1 -- 172.21.15.43:0/2083604338 >> 172.21.15.43:0/2083604338 conn(0x7f6bec0cf0b0 msgr2=0x7f6bec0cf4c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-09T13:50:06.466 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:06.463+0000 7f6bf2ccc640 1 -- 172.21.15.43:0/2083604338 shutdown_connections 2023-08-09T13:50:06.466 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:06.463+0000 7f6bf2ccc640 1 -- 172.21.15.43:0/2083604338 wait complete. 2023-08-09T13:50:06.467 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:06.465+0000 7f6bf2ccc640 1 Processor -- start 2023-08-09T13:50:06.467 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:06.465+0000 7f6bf2ccc640 1 -- start start 2023-08-09T13:50:06.468 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:06.466+0000 7f6bf2ccc640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f6bec070b20 0x7f6bec0b4400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:50:06.468 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:06.466+0000 7f6bf2ccc640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f6bec15ca70 0x7f6bec15c060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:50:06.469 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:06.466+0000 7f6bf2ccc640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f6bec15d560 0x7f6bec0b2cd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:50:06.469 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:06.466+0000 7f6bf2ccc640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6bec0b3fe0 con 0x7f6bec15d560 2023-08-09T13:50:06.469 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:06.466+0000 7f6bf2ccc640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6bec0b3360 con 0x7f6bec15ca70 2023-08-09T13:50:06.469 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:06.466+0000 7f6bf2ccc640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6bec0b3640 con 0x7f6bec070b20 2023-08-09T13:50:06.469 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:06.466+0000 7f6bf1269640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f6bec15d560 0x7f6bec0b2cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:50:06.469 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:06.466+0000 7f6bf1269640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f6bec15d560 0x7f6bec0b2cd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:45306/0 (socket says 172.21.15.43:45306) 2023-08-09T13:50:06.469 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:06.466+0000 7f6bf1269640 1 -- 172.21.15.43:0/1353447856 learned_addr learned my addr 172.21.15.43:0/1353447856 (peer_addr_for_me v2:172.21.15.43:0/0) 2023-08-09T13:50:06.470 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:06.466+0000 7f6be3fff640 1 --2- 172.21.15.43:0/1353447856 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f6bec15ca70 0x7f6bec15c060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:50:06.471 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:06.466+0000 7f6bf0a68640 1 --2- 172.21.15.43:0/1353447856 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f6bec070b20 0x7f6bec0b4400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:50:06.471 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:06.466+0000 7f6bf1269640 1 -- 172.21.15.43:0/1353447856 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f6bec070b20 msgr2=0x7f6bec0b4400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:50:06.471 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:06.466+0000 7f6bf1269640 1 --2- 172.21.15.43:0/1353447856 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f6bec070b20 0x7f6bec0b4400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:50:06.471 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:06.466+0000 7f6bf1269640 1 -- 172.21.15.43:0/1353447856 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f6bec15ca70 msgr2=0x7f6bec15c060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:50:06.471 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:06.466+0000 7f6bf1269640 1 --2- 172.21.15.43:0/1353447856 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f6bec15ca70 0x7f6bec15c060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:50:06.471 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:06.466+0000 7f6bf1269640 1 -- 172.21.15.43:0/1353447856 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6be8009400 con 0x7f6bec15d560 2023-08-09T13:50:06.471 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:06.467+0000 7f6bf1269640 1 --2- 172.21.15.43:0/1353447856 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f6bec15d560 0x7f6bec0b2cd0 secure :-1 s=READY pgs=913 cs=0 l=1 rev1=1 crypto rx=0x7f6be400c6b0 tx=0x7f6be400cb70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-09T13:50:06.472 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:06.467+0000 7f6be17fa640 1 -- 172.21.15.43:0/1353447856 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6be4003b70 con 0x7f6bec15d560 2023-08-09T13:50:06.472 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:06.467+0000 7f6be17fa640 1 -- 172.21.15.43:0/1353447856 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6be4003cf0 con 0x7f6bec15d560 2023-08-09T13:50:06.472 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:06.467+0000 7f6be17fa640 1 -- 172.21.15.43:0/1353447856 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6be4003ea0 con 0x7f6bec15d560 2023-08-09T13:50:06.472 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:06.467+0000 7f6bf2ccc640 1 -- 172.21.15.43:0/1353447856 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6bec0b3920 con 0x7f6bec15d560 2023-08-09T13:50:06.472 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:06.467+0000 7f6bf2ccc640 1 -- 172.21.15.43:0/1353447856 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6bec1f2570 con 0x7f6bec15d560 2023-08-09T13:50:06.472 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:06.468+0000 7f6bf2ccc640 1 -- 172.21.15.43:0/1353447856 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6bec0b4120 con 0x7f6bec15d560 2023-08-09T13:50:06.473 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:06.469+0000 7f6be17fa640 1 -- 172.21.15.43:0/1353447856 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 25) v1 ==== 88557+0+0 (secure 0 0 0) 0x7f6be4005310 con 0x7f6bec15d560 2023-08-09T13:50:06.473 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:06.470+0000 7f6be17fa640 1 --2- 172.21.15.43:0/1353447856 >> [v2:172.21.15.203:6832/2515481191,v1:172.21.15.203:6833/2515481191] conn(0x7f6bd806a290 0x7f6bd806c750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:50:06.473 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:06.470+0000 7f6be17fa640 1 -- 172.21.15.43:0/1353447856 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(75..75 src has 1..75) v4 ==== 6770+0+0 (secure 0 0 0) 0x7f6be4088a30 con 0x7f6bec15d560 2023-08-09T13:50:06.474 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:06.470+0000 7f6bf0a68640 1 -- 172.21.15.43:0/1353447856 >> [v2:172.21.15.203:6832/2515481191,v1:172.21.15.203:6833/2515481191] conn(0x7f6bd806a290 msgr2=0x7f6bd806c750 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.203:6832/2515481191 2023-08-09T13:50:06.474 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:06.470+0000 7f6bf0a68640 1 --2- 172.21.15.43:0/1353447856 >> [v2:172.21.15.203:6832/2515481191,v1:172.21.15.203:6833/2515481191] conn(0x7f6bd806a290 0x7f6bd806c750 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-08-09T13:50:06.477 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:06.474+0000 7f6be17fa640 1 -- 172.21.15.43:0/1353447856 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178066 (secure 0 0 0) 0x7f6bec0b4120 con 0x7f6bec15d560 2023-08-09T13:50:06.489 INFO:tasks.ceph.mgr.y.smithi043.stderr:2023-08-09T13:50:06.487+0000 7f76db0f8040 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2023-08-09T13:50:06.508 INFO:tasks.ceph.mgr.x.smithi203.stderr:2023-08-09T13:50:06.506+0000 7fbb9f40e040 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2023-08-09T13:50:06.604 INFO:tasks.ceph.mgr.y.smithi043.stderr:2023-08-09T13:50:06.602+0000 7f76db0f8040 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2023-08-09T13:50:06.656 INFO:tasks.ceph.mgr.y.smithi043.stderr:2023-08-09T13:50:06.654+0000 7f76db0f8040 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2023-08-09T13:50:06.667 INFO:tasks.ceph.mgr.x.smithi203.stderr:2023-08-09T13:50:06.665+0000 7fbb9f40e040 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2023-08-09T13:50:06.670 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:06.667+0000 7f6bf2ccc640 1 -- 172.21.15.43:0/1353447856 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f6bec0b4120 con 0x7f6bec15d560 2023-08-09T13:50:06.670 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:06.668+0000 7f6be17fa640 1 -- 172.21.15.43:0/1353447856 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v75) v1 ==== 100+0+11 (secure 0 0 0) 0x7f6bec0b4120 con 0x7f6bec15d560 2023-08-09T13:50:06.671 INFO:teuthology.orchestra.run.smithi043.stdout:pg_num: 64 2023-08-09T13:50:06.673 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:06.671+0000 7f6bf2ccc640 1 -- 172.21.15.43:0/1353447856 >> [v2:172.21.15.203:6832/2515481191,v1:172.21.15.203:6833/2515481191] conn(0x7f6bd806a290 msgr2=0x7f6bd806c750 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2023-08-09T13:50:06.673 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:06.671+0000 7f6bf0a68640 1 -- 172.21.15.43:0/1353447856 >> [v2:172.21.15.203:6832/2515481191,v1:172.21.15.203:6833/2515481191] conn(0x7f6bd806a290 msgr2=0x7f6bd806c750 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.203:6832/2515481191 2023-08-09T13:50:06.674 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:06.671+0000 7f6bf0a68640 1 --2- 172.21.15.43:0/1353447856 >> [v2:172.21.15.203:6832/2515481191,v1:172.21.15.203:6833/2515481191] conn(0x7f6bd806a290 0x7f6bd806c750 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2023-08-09T13:50:06.674 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:06.671+0000 7f6bf2ccc640 1 --2- 172.21.15.43:0/1353447856 >> [v2:172.21.15.203:6832/2515481191,v1:172.21.15.203:6833/2515481191] conn(0x7f6bd806a290 0x7f6bd806c750 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:50:06.674 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:06.671+0000 7f6bf2ccc640 1 -- 172.21.15.43:0/1353447856 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f6bec15d560 msgr2=0x7f6bec0b2cd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:50:06.674 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:06.671+0000 7f6bf2ccc640 1 --2- 172.21.15.43:0/1353447856 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f6bec15d560 0x7f6bec0b2cd0 secure :-1 s=READY pgs=913 cs=0 l=1 rev1=1 crypto rx=0x7f6be400c6b0 tx=0x7f6be400cb70 comp rx=0 tx=0).stop 2023-08-09T13:50:06.674 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:06.671+0000 7f6bf2ccc640 1 -- 172.21.15.43:0/1353447856 shutdown_connections 2023-08-09T13:50:06.675 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:06.671+0000 7f6bf2ccc640 1 --2- 172.21.15.43:0/1353447856 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f6bec15d560 0x7f6bec0b2cd0 unknown :-1 s=CLOSED pgs=913 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:50:06.675 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:06.671+0000 7f6bf2ccc640 1 --2- 172.21.15.43:0/1353447856 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f6bec15ca70 0x7f6bec15c060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:50:06.675 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:06.671+0000 7f6bf2ccc640 1 --2- 172.21.15.43:0/1353447856 >> [v2:172.21.15.203:6832/2515481191,v1:172.21.15.203:6833/2515481191] conn(0x7f6bd806a290 0x7f6bd806c750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:50:06.675 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:06.671+0000 7f6bf2ccc640 1 --2- 172.21.15.43:0/1353447856 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f6bec070b20 0x7f6bec0b4400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:50:06.675 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:06.671+0000 7f6bf2ccc640 1 -- 172.21.15.43:0/1353447856 >> 172.21.15.43:0/1353447856 conn(0x7f6bec0cf0b0 msgr2=0x7f6bec0adf10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-09T13:50:06.675 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:06.671+0000 7f6bf2ccc640 1 -- 172.21.15.43:0/1353447856 shutdown_connections 2023-08-09T13:50:06.676 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:06.671+0000 7f6bf2ccc640 1 -- 172.21.15.43:0/1353447856 wait complete. 2023-08-09T13:50:06.690 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2023-08-09T13:50:06.828 INFO:tasks.ceph.mgr.y.smithi043.stderr:2023-08-09T13:50:06.826+0000 7f76db0f8040 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2023-08-09T13:50:06.899 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:06.896+0000 7ff230a97640 1 Processor -- start 2023-08-09T13:50:06.899 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:06.896+0000 7ff230a97640 1 -- start start 2023-08-09T13:50:06.900 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:06.898+0000 7ff230a97640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7ff22c0cbfc0 0x7ff22c070710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:50:06.901 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:06.898+0000 7ff230a97640 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7ff22c0cc440 0x7ff22c070c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:50:06.901 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:06.898+0000 7ff230a97640 1 -- --> v1:172.21.15.43:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff22c0c9710 con 0x7ff22c158980 2023-08-09T13:50:06.901 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:06.898+0000 7ff230a97640 1 -- --> v2:172.21.15.43:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff22c158d60 con 0x7ff22c0cbfc0 2023-08-09T13:50:06.901 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:06.898+0000 7ff230a97640 1 -- --> v2:172.21.15.203:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff22c070280 con 0x7ff22c0cc440 2023-08-09T13:50:06.901 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:06.898+0000 7ff22b7fe640 1 --1- >> v1:172.21.15.43:6789/0 conn(0x7ff22c158980 0x7ff22c0c8ec0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.43:6789/0 says I am v1:172.21.15.43:58760/0 (socket says 172.21.15.43:58760) 2023-08-09T13:50:06.901 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:06.898+0000 7ff22b7fe640 1 -- 172.21.15.43:0/1236757025 learned_addr learned my addr 172.21.15.43:0/1236757025 (peer_addr_for_me v1:172.21.15.43:0/0) 2023-08-09T13:50:06.902 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:06.898+0000 7ff22affd640 1 --2- 172.21.15.43:0/1236757025 >> v2:172.21.15.43:3300/0 conn(0x7ff22c0cbfc0 0x7ff22c070710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:50:06.902 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:06.898+0000 7ff22bfff640 1 --2- 172.21.15.43:0/1236757025 >> v2:172.21.15.203:3300/0 conn(0x7ff22c0cc440 0x7ff22c070c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:50:06.902 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:06.899+0000 7ff22a7fc640 1 -- 172.21.15.43:0/1236757025 <== mon.0 v1:172.21.15.43:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1465017352 0 0) 0x7ff21c001110 con 0x7ff22c158980 2023-08-09T13:50:06.903 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:06.899+0000 7ff22a7fc640 1 -- 172.21.15.43:0/1236757025 --> v1:172.21.15.43:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ff214005730 con 0x7ff22c158980 2023-08-09T13:50:06.903 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:06.899+0000 7ff22affd640 1 -- 172.21.15.43:0/1236757025 >> v2:172.21.15.203:3300/0 conn(0x7ff22c0cc440 msgr2=0x7ff22c070c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-09T13:50:06.903 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:06.899+0000 7ff22affd640 1 --2- 172.21.15.43:0/1236757025 >> v2:172.21.15.203:3300/0 conn(0x7ff22c0cc440 0x7ff22c070c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:50:06.903 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:06.899+0000 7ff22affd640 1 -- 172.21.15.43:0/1236757025 >> v1:172.21.15.43:6789/0 conn(0x7ff22c158980 legacy=0x7ff22c0c8ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:50:06.903 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:06.899+0000 7ff22affd640 1 -- 172.21.15.43:0/1236757025 --> v2:172.21.15.43:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff22c071190 con 0x7ff22c0cbfc0 2023-08-09T13:50:06.903 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:06.899+0000 7ff22bfff640 1 --2- 172.21.15.43:0/1236757025 >> v2:172.21.15.203:3300/0 conn(0x7ff22c0cc440 0x7ff22c070c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-08-09T13:50:06.904 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:06.900+0000 7ff22affd640 1 --2- 172.21.15.43:0/1236757025 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff22c0cbfc0 0x7ff22c070710 secure :-1 s=READY pgs=915 cs=0 l=1 rev1=1 crypto rx=0x7ff22c06ac50 tx=0x7ff22002b840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=373b24d83af42c09 server_cookie=0 in_seq=0 out_seq=0 2023-08-09T13:50:06.905 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:06.901+0000 7ff22a7fc640 1 -- 172.21.15.43:0/1236757025 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff22002e070 con 0x7ff22c0cbfc0 2023-08-09T13:50:06.905 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:06.901+0000 7ff22a7fc640 1 -- 172.21.15.43:0/1236757025 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff2200315a0 con 0x7ff22c0cbfc0 2023-08-09T13:50:06.905 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:06.901+0000 7ff22a7fc640 1 -- 172.21.15.43:0/1236757025 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff220031910 con 0x7ff22c0cbfc0 2023-08-09T13:50:06.905 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:06.901+0000 7ff230a97640 1 -- 172.21.15.43:0/1236757025 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff22c0cbfc0 msgr2=0x7ff22c070710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:50:06.905 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:06.901+0000 7ff230a97640 1 --2- 172.21.15.43:0/1236757025 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff22c0cbfc0 0x7ff22c070710 secure :-1 s=READY pgs=915 cs=0 l=1 rev1=1 crypto rx=0x7ff22c06ac50 tx=0x7ff22002b840 comp rx=0 tx=0).stop 2023-08-09T13:50:06.905 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:06.901+0000 7ff230a97640 1 -- 172.21.15.43:0/1236757025 shutdown_connections 2023-08-09T13:50:06.905 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:06.901+0000 7ff230a97640 1 --2- 172.21.15.43:0/1236757025 >> v2:172.21.15.203:3300/0 conn(0x7ff22c0cc440 0x7ff22c070c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:50:06.906 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:06.901+0000 7ff230a97640 1 --2- 172.21.15.43:0/1236757025 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff22c0cbfc0 0x7ff22c070710 unknown :-1 s=CLOSED pgs=915 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:50:06.906 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:06.901+0000 7ff230a97640 1 -- 172.21.15.43:0/1236757025 >> 172.21.15.43:0/1236757025 conn(0x7ff22c0af950 msgr2=0x7ff22c0afd60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-09T13:50:06.906 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:06.902+0000 7ff230a97640 1 -- 172.21.15.43:0/1236757025 shutdown_connections 2023-08-09T13:50:06.906 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:06.902+0000 7ff230a97640 1 -- 172.21.15.43:0/1236757025 wait complete. 2023-08-09T13:50:06.906 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:06.904+0000 7ff230a97640 1 Processor -- start 2023-08-09T13:50:06.907 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:06.904+0000 7ff230a97640 1 -- start start 2023-08-09T13:50:06.907 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:06.905+0000 7ff230a97640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7ff22c0cbfc0 0x7ff22c0c7880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:50:06.908 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:06.905+0000 7ff230a97640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7ff22c0cc440 0x7ff22c152760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:50:06.908 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:06.905+0000 7ff230a97640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff22c158980 0x7ff22c0b0a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:50:06.908 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:06.905+0000 7ff230a97640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff22c0c7dc0 con 0x7ff22c158980 2023-08-09T13:50:06.908 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:06.905+0000 7ff230a97640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff22c152ca0 con 0x7ff22c0cbfc0 2023-08-09T13:50:06.908 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:06.905+0000 7ff230a97640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff22c0b0fc0 con 0x7ff22c0cc440 2023-08-09T13:50:06.910 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:06.906+0000 7ff22bfff640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff22c158980 0x7ff22c0b0a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:50:06.910 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:06.906+0000 7ff22bfff640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff22c158980 0x7ff22c0b0a80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:45314/0 (socket says 172.21.15.43:45314) 2023-08-09T13:50:06.910 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:06.906+0000 7ff22bfff640 1 -- 172.21.15.43:0/3283989075 learned_addr learned my addr 172.21.15.43:0/3283989075 (peer_addr_for_me v2:172.21.15.43:0/0) 2023-08-09T13:50:06.910 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:06.906+0000 7ff22b7fe640 1 --2- 172.21.15.43:0/3283989075 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7ff22c0cbfc0 0x7ff22c0c7880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:50:06.911 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:06.906+0000 7ff22affd640 1 --2- 172.21.15.43:0/3283989075 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7ff22c0cc440 0x7ff22c152760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:50:06.911 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:06.906+0000 7ff22bfff640 1 -- 172.21.15.43:0/3283989075 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7ff22c0cc440 msgr2=0x7ff22c152760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:50:06.911 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:06.906+0000 7ff22bfff640 1 --2- 172.21.15.43:0/3283989075 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7ff22c0cc440 0x7ff22c152760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:50:06.911 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:06.906+0000 7ff22bfff640 1 -- 172.21.15.43:0/3283989075 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7ff22c0cbfc0 msgr2=0x7ff22c0c7880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:50:06.912 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:06.906+0000 7ff22bfff640 1 --2- 172.21.15.43:0/3283989075 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7ff22c0cbfc0 0x7ff22c0c7880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:50:06.912 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:06.906+0000 7ff22bfff640 1 -- 172.21.15.43:0/3283989075 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff220009400 con 0x7ff22c158980 2023-08-09T13:50:06.912 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:06.907+0000 7ff22bfff640 1 --2- 172.21.15.43:0/3283989075 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff22c158980 0x7ff22c0b0a80 secure :-1 s=READY pgs=916 cs=0 l=1 rev1=1 crypto rx=0x7ff224007f50 tx=0x7ff22400d560 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-09T13:50:06.912 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:06.907+0000 7ff213fff640 1 -- 172.21.15.43:0/3283989075 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff224003b90 con 0x7ff22c158980 2023-08-09T13:50:06.913 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:06.907+0000 7ff213fff640 1 -- 172.21.15.43:0/3283989075 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff22400f040 con 0x7ff22c158980 2023-08-09T13:50:06.913 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:06.907+0000 7ff213fff640 1 -- 172.21.15.43:0/3283989075 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff22400dec0 con 0x7ff22c158980 2023-08-09T13:50:06.913 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:06.907+0000 7ff230a97640 1 -- 172.21.15.43:0/3283989075 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff22c0b11f0 con 0x7ff22c158980 2023-08-09T13:50:06.914 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:06.907+0000 7ff230a97640 1 -- 172.21.15.43:0/3283989075 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff22c0b17b0 con 0x7ff22c158980 2023-08-09T13:50:06.915 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:06.908+0000 7ff230a97640 1 -- 172.21.15.43:0/3283989075 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff22c0739f0 con 0x7ff22c158980 2023-08-09T13:50:06.915 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:06.909+0000 7ff213fff640 1 -- 172.21.15.43:0/3283989075 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 25) v1 ==== 88557+0+0 (secure 0 0 0) 0x7ff224004030 con 0x7ff22c158980 2023-08-09T13:50:06.916 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:06.910+0000 7ff213fff640 1 --2- 172.21.15.43:0/3283989075 >> [v2:172.21.15.203:6832/2515481191,v1:172.21.15.203:6833/2515481191] conn(0x7ff21806a2c0 0x7ff21806c780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:50:06.916 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:06.910+0000 7ff213fff640 1 -- 172.21.15.43:0/3283989075 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(75..75 src has 1..75) v4 ==== 6770+0+0 (secure 0 0 0) 0x7ff224089170 con 0x7ff22c158980 2023-08-09T13:50:06.916 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:06.910+0000 7ff22b7fe640 1 -- 172.21.15.43:0/3283989075 >> [v2:172.21.15.203:6832/2515481191,v1:172.21.15.203:6833/2515481191] conn(0x7ff21806a2c0 msgr2=0x7ff21806c780 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.203:6832/2515481191 2023-08-09T13:50:06.917 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:06.910+0000 7ff22b7fe640 1 --2- 172.21.15.43:0/3283989075 >> [v2:172.21.15.203:6832/2515481191,v1:172.21.15.203:6833/2515481191] conn(0x7ff21806a2c0 0x7ff21806c780 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-08-09T13:50:06.917 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:06.914+0000 7ff213fff640 1 -- 172.21.15.43:0/3283989075 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178066 (secure 0 0 0) 0x7ff2240540a0 con 0x7ff22c158980 2023-08-09T13:50:07.047 INFO:tasks.ceph.mgr.x.smithi203.stderr:2023-08-09T13:50:07.044+0000 7fbb9f40e040 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2023-08-09T13:50:07.114 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:07.111+0000 7ff22b7fe640 1 -- 172.21.15.43:0/3283989075 >> [v2:172.21.15.203:6832/2515481191,v1:172.21.15.203:6833/2515481191] conn(0x7ff21806a2c0 msgr2=0x7ff21806c780 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.203:6832/2515481191 2023-08-09T13:50:07.114 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:07.111+0000 7ff22b7fe640 1 --2- 172.21.15.43:0/3283989075 >> [v2:172.21.15.203:6832/2515481191,v1:172.21.15.203:6833/2515481191] conn(0x7ff21806a2c0 0x7ff21806c780 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2023-08-09T13:50:07.124 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:07.122+0000 7ff230a97640 1 -- 172.21.15.43:0/3283989075 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7ff22c0cbb70 con 0x7ff22c158980 2023-08-09T13:50:07.125 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:07.122+0000 7ff213fff640 1 -- 172.21.15.43:0/3283989075 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 8 entries v75) v1 ==== 80+0+474 (secure 0 0 0) 0x7ff22401a1f0 con 0x7ff22c158980 2023-08-09T13:50:07.125 INFO:teuthology.orchestra.run.smithi043.stdout:172.21.15.203:6833/272917756 2023-08-10T13:48:34.065493+0000 2023-08-09T13:50:07.125 INFO:teuthology.orchestra.run.smithi043.stdout:172.21.15.203:6832/272917756 2023-08-10T13:48:34.065493+0000 2023-08-09T13:50:07.125 INFO:teuthology.orchestra.run.smithi043.stdout:172.21.15.203:0/665971116 2023-08-10T13:48:34.065493+0000 2023-08-09T13:50:07.126 INFO:teuthology.orchestra.run.smithi043.stdout:172.21.15.203:0/2868823959 2023-08-10T13:48:34.065493+0000 2023-08-09T13:50:07.126 INFO:teuthology.orchestra.run.smithi043.stdout:172.21.15.203:0/3168500363 2023-08-10T13:48:34.065493+0000 2023-08-09T13:50:07.126 INFO:teuthology.orchestra.run.smithi043.stdout:172.21.15.203:0/752327795 2023-08-10T13:48:34.065493+0000 2023-08-09T13:50:07.126 INFO:teuthology.orchestra.run.smithi043.stdout:172.21.15.203:0/1970263661 2023-08-10T13:48:34.065493+0000 2023-08-09T13:50:07.126 INFO:teuthology.orchestra.run.smithi043.stdout:172.21.15.203:0/1407921417 2023-08-10T13:48:34.065493+0000 2023-08-09T13:50:07.126 INFO:teuthology.orchestra.run.smithi043.stderr:listed 8 entries 2023-08-09T13:50:07.127 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:07.125+0000 7ff230a97640 1 -- 172.21.15.43:0/3283989075 >> [v2:172.21.15.203:6832/2515481191,v1:172.21.15.203:6833/2515481191] conn(0x7ff21806a2c0 msgr2=0x7ff21806c780 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-08-09T13:50:07.127 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:07.125+0000 7ff230a97640 1 --2- 172.21.15.43:0/3283989075 >> [v2:172.21.15.203:6832/2515481191,v1:172.21.15.203:6833/2515481191] conn(0x7ff21806a2c0 0x7ff21806c780 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:50:07.128 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:07.125+0000 7ff230a97640 1 -- 172.21.15.43:0/3283989075 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff22c158980 msgr2=0x7ff22c0b0a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:50:07.128 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:07.125+0000 7ff230a97640 1 --2- 172.21.15.43:0/3283989075 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff22c158980 0x7ff22c0b0a80 secure :-1 s=READY pgs=916 cs=0 l=1 rev1=1 crypto rx=0x7ff224007f50 tx=0x7ff22400d560 comp rx=0 tx=0).stop 2023-08-09T13:50:07.128 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:07.125+0000 7ff230a97640 1 -- 172.21.15.43:0/3283989075 shutdown_connections 2023-08-09T13:50:07.129 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:07.125+0000 7ff230a97640 1 --2- 172.21.15.43:0/3283989075 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff22c158980 0x7ff22c0b0a80 unknown :-1 s=CLOSED pgs=916 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:50:07.129 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:07.125+0000 7ff230a97640 1 --2- 172.21.15.43:0/3283989075 >> [v2:172.21.15.203:6832/2515481191,v1:172.21.15.203:6833/2515481191] conn(0x7ff21806a2c0 0x7ff21806c780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:50:07.129 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:07.125+0000 7ff230a97640 1 --2- 172.21.15.43:0/3283989075 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7ff22c0cc440 0x7ff22c152760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:50:07.129 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:07.125+0000 7ff230a97640 1 --2- 172.21.15.43:0/3283989075 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7ff22c0cbfc0 0x7ff22c0c7880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:50:07.129 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:07.125+0000 7ff230a97640 1 -- 172.21.15.43:0/3283989075 >> 172.21.15.43:0/3283989075 conn(0x7ff22c0af950 msgr2=0x7ff22c0cc8c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-09T13:50:07.129 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:07.125+0000 7ff230a97640 1 -- 172.21.15.43:0/3283989075 shutdown_connections 2023-08-09T13:50:07.129 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:07.125+0000 7ff230a97640 1 -- 172.21.15.43:0/3283989075 wait complete. 2023-08-09T13:50:07.145 INFO:tasks.cephfs.fuse_mount:Running fusermount -u on ubuntu@smithi043.front.sepia.ceph.com... 2023-08-09T13:50:07.146 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-09T13:50:07.146 DEBUG:teuthology.orchestra.run.smithi043:> sudo fusermount -u /home/ubuntu/cephtest/mnt.0 2023-08-09T13:50:07.233 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi043.stderr:ceph-fuse[113478]: fuse finished with error 0 and tester_r 0 2023-08-09T13:50:07.237 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi043.front.sepia.ceph.com 2023-08-09T13:50:07.237 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-09T13:50:07.238 DEBUG:teuthology.orchestra.run.smithi043:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2023-08-09T13:50:07.244 INFO:tasks.ceph.mgr.y.smithi043.stderr:2023-08-09T13:50:07.237+0000 7f76db0f8040 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2023-08-09T13:50:07.274 INFO:tasks.ceph.mgr.x.smithi203.stderr:2023-08-09T13:50:07.272+0000 7fbb9f40e040 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2023-08-09T13:50:07.331 INFO:tasks.ceph.mgr.x.smithi203.stderr:2023-08-09T13:50:07.329+0000 7fbb9f40e040 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2023-08-09T13:50:07.381 INFO:tasks.ceph.mgr.x.smithi203.stderr:2023-08-09T13:50:07.379+0000 7fbb9f40e040 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2023-08-09T13:50:07.416 DEBUG:teuthology.orchestra.run.smithi203:> set -ex 2023-08-09T13:50:07.417 DEBUG:teuthology.orchestra.run.smithi203:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-09T13:50:07.433 INFO:tasks.ceph.mgr.x.smithi203.stderr:2023-08-09T13:50:07.431+0000 7fbb9f40e040 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2023-08-09T13:50:07.473 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4053648,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497576k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=27,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248784k,nr_inodes=812196,mode=700,uid=1000,gid=1275 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497576k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2023-08-09T13:50:07.474 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi203.front.sepia.ceph.com 2023-08-09T13:50:07.474 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-09T13:50:07.475 DEBUG:teuthology.orchestra.run.smithi203:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.1) 2023-08-09T13:50:07.483 INFO:tasks.ceph.mgr.y.smithi043.stderr:2023-08-09T13:50:07.480+0000 7f76db0f8040 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2023-08-09T13:50:07.542 INFO:tasks.ceph.mgr.y.smithi043.stderr:2023-08-09T13:50:07.540+0000 7f76db0f8040 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2023-08-09T13:50:07.569 INFO:teuthology.orchestra.run.smithi203.stderr:rmdir: failed to remove '/home/ubuntu/cephtest/mnt.1': No such file or directory 2023-08-09T13:50:07.570 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-08-09T13:50:07.571 DEBUG:teuthology.orchestra.run.smithi043:> set -ex 2023-08-09T13:50:07.571 DEBUG:teuthology.orchestra.run.smithi043:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-09T13:50:07.596 INFO:tasks.ceph.mgr.y.smithi043.stderr:2023-08-09T13:50:07.594+0000 7f76db0f8040 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2023-08-09T13:50:07.627 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4054441,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6498844k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=27,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3249420k,nr_inodes=812355,mode=700,uid=1000,gid=1275 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6498844k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2023-08-09T13:50:07.628 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi043.front.sepia.ceph.com 2023-08-09T13:50:07.628 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-09T13:50:07.629 DEBUG:teuthology.orchestra.run.smithi043:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.2) 2023-08-09T13:50:07.654 INFO:tasks.ceph.mgr.y.smithi043.stderr:2023-08-09T13:50:07.651+0000 7f76db0f8040 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2023-08-09T13:50:07.710 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-08-09T13:50:07.711 INFO:teuthology.orchestra.run.smithi043.stderr:rmdir: failed to remove '/home/ubuntu/cephtest/mnt.2': No such file or directory 2023-08-09T13:50:07.711 DEBUG:teuthology.orchestra.run.smithi203:> set -ex 2023-08-09T13:50:07.712 DEBUG:teuthology.orchestra.run.smithi203:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-09T13:50:07.735 INFO:tasks.ceph.mgr.x.smithi203.stderr:2023-08-09T13:50:07.733+0000 7fbb9f40e040 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2023-08-09T13:50:07.769 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4053648,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497576k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=27,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248784k,nr_inodes=812196,mode=700,uid=1000,gid=1275 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497576k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2023-08-09T13:50:07.769 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi203.front.sepia.ceph.com 2023-08-09T13:50:07.770 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-09T13:50:07.770 DEBUG:teuthology.orchestra.run.smithi203:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.3) 2023-08-09T13:50:07.796 INFO:tasks.ceph.mgr.x.smithi203.stderr:2023-08-09T13:50:07.794+0000 7fbb9f40e040 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2023-08-09T13:50:07.847 INFO:teuthology.orchestra.run.smithi203.stderr:rmdir: failed to remove '/home/ubuntu/cephtest/mnt.3': No such file or directory 2023-08-09T13:50:07.848 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-08-09T13:50:07.849 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-08-09T13:50:07.851 INFO:tasks.ceph.mgr.x.smithi203.stderr:2023-08-09T13:50:07.849+0000 7fbb9f40e040 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2023-08-09T13:50:07.971 INFO:tasks.ceph.mgr.y.smithi043.stderr:2023-08-09T13:50:07.969+0000 7f76db0f8040 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2023-08-09T13:50:08.012 INFO:tasks.ceph.mgr.x.smithi203.stderr:2023-08-09T13:50:08.010+0000 7fbb9f40e040 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2023-08-09T13:50:08.033 INFO:tasks.ceph.mgr.y.smithi043.stderr:2023-08-09T13:50:08.030+0000 7f76db0f8040 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2023-08-09T13:50:08.061 INFO:tasks.ceph.mgr.x.smithi203.stderr:2023-08-09T13:50:08.059+0000 7fbb9f40e040 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2023-08-09T13:50:08.070 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:08.068+0000 7facac739640 1 Processor -- start 2023-08-09T13:50:08.071 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:08.068+0000 7facac739640 1 -- start start 2023-08-09T13:50:08.072 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:08.070+0000 7facac739640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7faca40ce720 0x7faca40ceb00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:50:08.072 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:08.070+0000 7facac739640 1 -- --> v1:172.21.15.43:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7faca40cd8a0 con 0x7faca415ecc0 2023-08-09T13:50:08.073 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:08.070+0000 7facac739640 1 -- --> v1:172.21.15.203:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7faca40cf350 con 0x7faca40ce080 2023-08-09T13:50:08.073 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:08.070+0000 7facac739640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7faca40cf040 con 0x7faca40ce720 2023-08-09T13:50:08.074 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:08.070+0000 7facaacd6640 1 --1- >> v1:172.21.15.43:6789/0 conn(0x7faca415ecc0 0x7faca406d2a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.43:6789/0 says I am v1:172.21.15.43:58764/0 (socket says 172.21.15.43:58764) 2023-08-09T13:50:08.074 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:08.070+0000 7facaacd6640 1 -- 172.21.15.43:0/2160983868 learned_addr learned my addr 172.21.15.43:0/2160983868 (peer_addr_for_me v1:172.21.15.43:0/0) 2023-08-09T13:50:08.074 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:08.071+0000 7facaa4d5640 1 --2- 172.21.15.43:0/2160983868 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7faca40ce720 0x7faca40ceb00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:50:08.074 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:08.071+0000 7faca94d3640 1 -- 172.21.15.43:0/2160983868 <== mon.0 v1:172.21.15.43:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1447113596 0 0) 0x7faca40cd8a0 con 0x7faca415ecc0 2023-08-09T13:50:08.074 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:08.071+0000 7faca94d3640 1 -- 172.21.15.43:0/2160983868 --> v1:172.21.15.43:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fac90003520 con 0x7faca415ecc0 2023-08-09T13:50:08.074 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:08.071+0000 7faca94d3640 1 -- 172.21.15.43:0/2160983868 <== mon.0 v1:172.21.15.43:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3443513103 0 0) 0x7fac90003520 con 0x7faca415ecc0 2023-08-09T13:50:08.075 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:08.071+0000 7faca94d3640 1 -- 172.21.15.43:0/2160983868 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7faca40ce720 msgr2=0x7faca40ceb00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-09T13:50:08.075 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:08.071+0000 7faca94d3640 1 --2- 172.21.15.43:0/2160983868 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7faca40ce720 0x7faca40ceb00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:50:08.075 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:08.071+0000 7faca94d3640 1 -- 172.21.15.43:0/2160983868 >> v1:172.21.15.203:6789/0 conn(0x7faca40ce080 legacy=0x7faca415e5b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:50:08.075 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:08.071+0000 7faca94d3640 1 -- 172.21.15.43:0/2160983868 --> v1:172.21.15.43:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faca4071a70 con 0x7faca415ecc0 2023-08-09T13:50:08.075 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:08.071+0000 7faca94d3640 1 -- 172.21.15.43:0/2160983868 <== mon.0 v1:172.21.15.43:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1285150740 0 0) 0x7fac9c0019f0 con 0x7faca415ecc0 2023-08-09T13:50:08.075 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:08.071+0000 7faca94d3640 1 -- 172.21.15.43:0/2160983868 >> v1:172.21.15.43:6789/0 conn(0x7faca415ecc0 legacy=0x7faca406d2a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:50:08.075 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:08.071+0000 7faca94d3640 1 --2- 172.21.15.43:0/2160983868 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fac90003a10 0x7fac90003e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:50:08.076 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:08.071+0000 7faca94d3640 1 --2- 172.21.15.43:0/2160983868 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fac900047c0 0x7fac90004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:50:08.076 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:08.071+0000 7faca94d3640 1 --2- 172.21.15.43:0/2160983868 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7faca40ce720 0x7fac9000b980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:50:08.076 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:08.072+0000 7faca94d3640 1 -- 172.21.15.43:0/2160983868 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faca40cf040 con 0x7fac900047c0 2023-08-09T13:50:08.076 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:08.072+0000 7faca94d3640 1 -- 172.21.15.43:0/2160983868 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fac900054c0 con 0x7fac90003a10 2023-08-09T13:50:08.076 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:08.072+0000 7faca94d3640 1 -- 172.21.15.43:0/2160983868 --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fac90005680 con 0x7faca40ce720 2023-08-09T13:50:08.077 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:08.072+0000 7faca9cd4640 1 --2- 172.21.15.43:0/2160983868 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fac900047c0 0x7fac90004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:50:08.077 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:08.072+0000 7facaa4d5640 1 --2- 172.21.15.43:0/2160983868 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fac90003a10 0x7fac90003e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:50:08.078 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:08.072+0000 7facaacd6640 1 --2- 172.21.15.43:0/2160983868 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7faca40ce720 0x7fac9000b980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:50:08.078 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:08.072+0000 7faca9cd4640 1 -- 172.21.15.43:0/2160983868 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7faca40ce720 msgr2=0x7fac9000b980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-09T13:50:08.078 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:08.072+0000 7faca9cd4640 1 --2- 172.21.15.43:0/2160983868 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7faca40ce720 0x7fac9000b980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:50:08.078 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:08.072+0000 7faca9cd4640 1 -- 172.21.15.43:0/2160983868 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fac90003a10 msgr2=0x7fac90003e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-09T13:50:08.078 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:08.072+0000 7faca9cd4640 1 --2- 172.21.15.43:0/2160983868 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fac90003a10 0x7fac90003e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:50:08.078 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:08.072+0000 7faca9cd4640 1 -- 172.21.15.43:0/2160983868 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fac90003520 con 0x7fac900047c0 2023-08-09T13:50:08.078 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:08.073+0000 7faca9cd4640 1 --2- 172.21.15.43:0/2160983868 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fac900047c0 0x7fac90004c40 secure :-1 s=READY pgs=918 cs=0 l=1 rev1=1 crypto rx=0x7faca4070800 tx=0x7fac900057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bf450f1b20b441d6 server_cookie=0 in_seq=0 out_seq=0 2023-08-09T13:50:08.079 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:08.073+0000 7faca94d3640 1 -- 172.21.15.43:0/2160983868 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fac980052d0 con 0x7fac900047c0 2023-08-09T13:50:08.079 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:08.073+0000 7faca94d3640 1 -- 172.21.15.43:0/2160983868 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fac9802e050 con 0x7fac900047c0 2023-08-09T13:50:08.079 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:08.073+0000 7faca94d3640 1 -- 172.21.15.43:0/2160983868 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fac980035f0 con 0x7fac900047c0 2023-08-09T13:50:08.079 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:08.074+0000 7facac739640 1 -- 172.21.15.43:0/2160983868 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fac900047c0 msgr2=0x7fac90004c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:50:08.079 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:08.074+0000 7facac739640 1 --2- 172.21.15.43:0/2160983868 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fac900047c0 0x7fac90004c40 secure :-1 s=READY pgs=918 cs=0 l=1 rev1=1 crypto rx=0x7faca4070800 tx=0x7fac900057f0 comp rx=0 tx=0).stop 2023-08-09T13:50:08.079 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:08.074+0000 7facaacd6640 1 -- 172.21.15.43:0/2160983868 reap_dead start 2023-08-09T13:50:08.079 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:08.074+0000 7facac739640 1 -- 172.21.15.43:0/2160983868 shutdown_connections 2023-08-09T13:50:08.079 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:08.074+0000 7facac739640 1 -- 172.21.15.43:0/2160983868 >> 172.21.15.43:0/2160983868 conn(0x7faca40ccd80 msgr2=0x7faca40cd190 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-09T13:50:08.080 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:08.074+0000 7facac739640 1 -- 172.21.15.43:0/2160983868 shutdown_connections 2023-08-09T13:50:08.080 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:08.074+0000 7facac739640 1 -- 172.21.15.43:0/2160983868 wait complete. 2023-08-09T13:50:08.081 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:08.076+0000 7facac739640 1 Processor -- start 2023-08-09T13:50:08.081 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:08.076+0000 7facac739640 1 -- start start 2023-08-09T13:50:08.082 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:08.077+0000 7facac739640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fac900047c0 0x7faca40c7340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:50:08.082 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:08.077+0000 7facac739640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7faca40b1da0 0x7faca40c7ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:50:08.082 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:08.077+0000 7facac739640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7faca40c8010 0x7faca40c9280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:50:08.082 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:08.077+0000 7facac739640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faca40c8490 con 0x7faca40b1da0 2023-08-09T13:50:08.082 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:08.077+0000 7facac739640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faca40b2340 con 0x7faca40c8010 2023-08-09T13:50:08.082 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:08.077+0000 7facac739640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7faca40b2620 con 0x7fac900047c0 2023-08-09T13:50:08.082 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:08.077+0000 7faca9cd4640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7faca40b1da0 0x7faca40c7ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:50:08.082 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:08.077+0000 7faca9cd4640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7faca40b1da0 0x7faca40c7ad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:45328/0 (socket says 172.21.15.43:45328) 2023-08-09T13:50:08.083 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:08.077+0000 7faca9cd4640 1 -- 172.21.15.43:0/977352438 learned_addr learned my addr 172.21.15.43:0/977352438 (peer_addr_for_me v2:172.21.15.43:0/0) 2023-08-09T13:50:08.083 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:08.078+0000 7facaacd6640 1 --2- 172.21.15.43:0/977352438 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7faca40c8010 0x7faca40c9280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:50:08.083 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:08.078+0000 7facaa4d5640 1 --2- 172.21.15.43:0/977352438 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fac900047c0 0x7faca40c7340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:50:08.083 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:08.078+0000 7faca9cd4640 1 -- 172.21.15.43:0/977352438 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fac900047c0 msgr2=0x7faca40c7340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:50:08.083 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:08.078+0000 7faca9cd4640 1 --2- 172.21.15.43:0/977352438 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fac900047c0 0x7faca40c7340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:50:08.083 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:08.078+0000 7faca9cd4640 1 -- 172.21.15.43:0/977352438 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7faca40c8010 msgr2=0x7faca40c9280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:50:08.083 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:08.078+0000 7faca9cd4640 1 --2- 172.21.15.43:0/977352438 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7faca40c8010 0x7faca40c9280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:50:08.084 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:08.078+0000 7faca9cd4640 1 -- 172.21.15.43:0/977352438 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faca40b2900 con 0x7faca40b1da0 2023-08-09T13:50:08.084 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:08.078+0000 7faca9cd4640 1 --2- 172.21.15.43:0/977352438 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7faca40b1da0 0x7faca40c7ad0 secure :-1 s=READY pgs=919 cs=0 l=1 rev1=1 crypto rx=0x7fac9802bbb0 tx=0x7fac98001120 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-09T13:50:08.084 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:08.079+0000 7fac96ffd640 1 -- 172.21.15.43:0/977352438 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fac980076c0 con 0x7faca40b1da0 2023-08-09T13:50:08.084 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:08.079+0000 7fac96ffd640 1 -- 172.21.15.43:0/977352438 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fac98007a30 con 0x7faca40b1da0 2023-08-09T13:50:08.084 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:08.079+0000 7fac96ffd640 1 -- 172.21.15.43:0/977352438 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fac98006be0 con 0x7faca40b1da0 2023-08-09T13:50:08.084 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:08.079+0000 7facac739640 1 -- 172.21.15.43:0/977352438 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7faca40c4e40 con 0x7faca40b1da0 2023-08-09T13:50:08.084 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:08.079+0000 7facac739640 1 -- 172.21.15.43:0/977352438 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7faca40c88b0 con 0x7faca40b1da0 2023-08-09T13:50:08.085 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:08.080+0000 7facac739640 1 -- 172.21.15.43:0/977352438 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7faca40739f0 con 0x7faca40b1da0 2023-08-09T13:50:08.085 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:08.081+0000 7fac96ffd640 1 -- 172.21.15.43:0/977352438 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 25) v1 ==== 88557+0+0 (secure 0 0 0) 0x7fac98006d40 con 0x7faca40b1da0 2023-08-09T13:50:08.085 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:08.082+0000 7fac96ffd640 1 --2- 172.21.15.43:0/977352438 >> [v2:172.21.15.203:6832/2515481191,v1:172.21.15.203:6833/2515481191] conn(0x7fac8806a290 0x7fac8806c750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:50:08.086 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:08.082+0000 7fac96ffd640 1 -- 172.21.15.43:0/977352438 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(75..75 src has 1..75) v4 ==== 6770+0+0 (secure 0 0 0) 0x7fac980aaa00 con 0x7faca40b1da0 2023-08-09T13:50:08.086 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:08.082+0000 7facaa4d5640 1 -- 172.21.15.43:0/977352438 >> [v2:172.21.15.203:6832/2515481191,v1:172.21.15.203:6833/2515481191] conn(0x7fac8806a290 msgr2=0x7fac8806c750 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.203:6832/2515481191 2023-08-09T13:50:08.086 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:08.082+0000 7facaa4d5640 1 --2- 172.21.15.43:0/977352438 >> [v2:172.21.15.203:6832/2515481191,v1:172.21.15.203:6833/2515481191] conn(0x7fac8806a290 0x7fac8806c750 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-08-09T13:50:08.089 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:08.086+0000 7fac96ffd640 1 -- 172.21.15.43:0/977352438 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178066 (secure 0 0 0) 0x7fac98075af0 con 0x7faca40b1da0 2023-08-09T13:50:08.091 INFO:tasks.ceph.mgr.y.smithi043.stderr:2023-08-09T13:50:08.089+0000 7f76db0f8040 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2023-08-09T13:50:08.116 INFO:tasks.ceph.mgr.x.smithi203.stderr:2023-08-09T13:50:08.113+0000 7fbb9f40e040 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2023-08-09T13:50:08.171 INFO:tasks.ceph.mgr.x.smithi203.stderr:2023-08-09T13:50:08.168+0000 7fbb9f40e040 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2023-08-09T13:50:08.185 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:08.182+0000 7fac96ffd640 1 -- 172.21.15.43:0/977352438 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mgrmap(e 26) v1 ==== 88052+0+0 (secure 0 0 0) 0x7fac98075ec0 con 0x7faca40b1da0 2023-08-09T13:50:08.186 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:08.182+0000 7fac96ffd640 1 -- 172.21.15.43:0/977352438 >> [v2:172.21.15.203:6832/2515481191,v1:172.21.15.203:6833/2515481191] conn(0x7fac8806a290 msgr2=0x7fac8806c750 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-08-09T13:50:08.186 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:08.182+0000 7fac96ffd640 1 --2- 172.21.15.43:0/977352438 >> [v2:172.21.15.203:6832/2515481191,v1:172.21.15.203:6833/2515481191] conn(0x7fac8806a290 0x7fac8806c750 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:50:08.282 INFO:tasks.ceph.mgr.x.smithi203.stderr:2023-08-09T13:50:08.279+0000 7fbb24c9e640 -1 client.0 error registering admin socket command: (17) File exists 2023-08-09T13:50:08.282 INFO:tasks.ceph.mgr.x.smithi203.stderr:2023-08-09T13:50:08.279+0000 7fbb24c9e640 -1 client.0 error registering admin socket command: (17) File exists 2023-08-09T13:50:08.282 INFO:tasks.ceph.mgr.x.smithi203.stderr:2023-08-09T13:50:08.279+0000 7fbb24c9e640 -1 client.0 error registering admin socket command: (17) File exists 2023-08-09T13:50:08.282 INFO:tasks.ceph.mgr.x.smithi203.stderr:2023-08-09T13:50:08.279+0000 7fbb24c9e640 -1 client.0 error registering admin socket command: (17) File exists 2023-08-09T13:50:08.282 INFO:tasks.ceph.mgr.x.smithi203.stderr:2023-08-09T13:50:08.279+0000 7fbb24c9e640 -1 client.0 error registering admin socket command: (17) File exists 2023-08-09T13:50:08.282 INFO:tasks.ceph.mgr.x.smithi203.stderr:2023-08-09T13:50:08.279+0000 7fbb21c98640 -1 client.0 error registering admin socket command: (17) File exists 2023-08-09T13:50:08.283 INFO:tasks.ceph.mgr.x.smithi203.stderr:2023-08-09T13:50:08.279+0000 7fbb21c98640 -1 client.0 error registering admin socket command: (17) File exists 2023-08-09T13:50:08.283 INFO:tasks.ceph.mgr.x.smithi203.stderr:2023-08-09T13:50:08.279+0000 7fbb21c98640 -1 client.0 error registering admin socket command: (17) File exists 2023-08-09T13:50:08.283 INFO:tasks.ceph.mgr.x.smithi203.stderr:2023-08-09T13:50:08.279+0000 7fbb21c98640 -1 client.0 error registering admin socket command: (17) File exists 2023-08-09T13:50:08.283 INFO:tasks.ceph.mgr.x.smithi203.stderr:2023-08-09T13:50:08.279+0000 7fbb21c98640 -1 client.0 error registering admin socket command: (17) File exists 2023-08-09T13:50:08.295 INFO:tasks.ceph.mgr.y.smithi043.stderr:2023-08-09T13:50:08.292+0000 7f76db0f8040 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2023-08-09T13:50:08.348 INFO:tasks.ceph.mgr.y.smithi043.stderr:2023-08-09T13:50:08.344+0000 7f76db0f8040 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2023-08-09T13:50:08.354 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:08.352+0000 7facac739640 1 -- 172.21.15.43:0/977352438 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7faca40c78b0 con 0x7faca40b1da0 2023-08-09T13:50:08.356 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:08.353+0000 7fac96ffd640 1 -- 172.21.15.43:0/977352438 <== mon.0 v2:172.21.15.43:3300/0 8 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 26 v26) v1 ==== 94+0+4881 (secure 0 0 0) 0x7fac980a49d0 con 0x7faca40b1da0 2023-08-09T13:50:08.356 INFO:teuthology.orchestra.run.smithi043.stdout: 2023-08-09T13:50:08.356 INFO:teuthology.orchestra.run.smithi043.stdout:{"epoch":26,"default_fscid":3,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5369,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.203:6837/1426673901","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6836","nonce":1426673901},{"type":"v1","addr":"172.21.15.203:6837","nonce":1426673901}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":21},{"gid":5728,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.43:6833/1096124235","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6832","nonce":1096124235},{"type":"v1","addr":"172.21.15.43:6833","nonce":1096124235}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":20},{"gid":5731,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.43:6835/1688804357","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6834","nonce":1688804357},{"type":"v1","addr":"172.21.15.43:6835","nonce":1688804357}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":21}],"filesystems":[{"mdsmap":{"epoch":26,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-08-09T13:48:17.607195+0000","modified":"2023-08-09T13:48:21.179296+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":5734},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5734":{"gid":5734,"name":"b","rank":0,"incarnation":22,"state":"up:active","state_seq":3,"addr":"172.21.15.203:6835/3324252767","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6834","nonce":3324252767},{"type":"v1","addr":"172.21.15.203:6835","nonce":3324252767}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[9],"metadata_pool":8,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":3}]} 2023-08-09T13:50:08.357 INFO:teuthology.orchestra.run.smithi043.stderr:dumped fsmap epoch 26 2023-08-09T13:50:08.359 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:08.356+0000 7facac739640 1 -- 172.21.15.43:0/977352438 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7faca40b1da0 msgr2=0x7faca40c7ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:50:08.359 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:08.356+0000 7facac739640 1 --2- 172.21.15.43:0/977352438 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7faca40b1da0 0x7faca40c7ad0 secure :-1 s=READY pgs=919 cs=0 l=1 rev1=1 crypto rx=0x7fac9802bbb0 tx=0x7fac98001120 comp rx=0 tx=0).stop 2023-08-09T13:50:08.359 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:08.356+0000 7facac739640 1 -- 172.21.15.43:0/977352438 shutdown_connections 2023-08-09T13:50:08.359 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:08.356+0000 7facac739640 1 --2- 172.21.15.43:0/977352438 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7faca40c8010 0x7faca40c9280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:50:08.359 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:08.356+0000 7facac739640 1 --2- 172.21.15.43:0/977352438 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7faca40b1da0 0x7faca40c7ad0 unknown :-1 s=CLOSED pgs=919 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:50:08.359 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:08.356+0000 7facac739640 1 --2- 172.21.15.43:0/977352438 >> [v2:172.21.15.203:6832/2515481191,v1:172.21.15.203:6833/2515481191] conn(0x7fac8806a290 0x7fac8806c750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:50:08.360 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:08.356+0000 7facac739640 1 --2- 172.21.15.43:0/977352438 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fac900047c0 0x7faca40c7340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:50:08.360 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:08.356+0000 7facac739640 1 -- 172.21.15.43:0/977352438 >> 172.21.15.43:0/977352438 conn(0x7faca40ccd80 msgr2=0x7faca4159a00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-09T13:50:08.360 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:08.356+0000 7facac739640 1 -- 172.21.15.43:0/977352438 shutdown_connections 2023-08-09T13:50:08.360 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:08.356+0000 7facac739640 1 -- 172.21.15.43:0/977352438 wait complete. 2023-08-09T13:50:08.376 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-08-09T13:50:08.406 INFO:tasks.ceph.mgr.y.smithi043.stderr:2023-08-09T13:50:08.403+0000 7f76db0f8040 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2023-08-09T13:50:08.463 INFO:tasks.ceph.mgr.y.smithi043.stderr:2023-08-09T13:50:08.459+0000 7f76db0f8040 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2023-08-09T13:50:08.594 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:08.591+0000 7f46b538b640 1 Processor -- start 2023-08-09T13:50:08.594 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:08.591+0000 7f46b538b640 1 -- start start 2023-08-09T13:50:08.595 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:08.592+0000 7f46b538b640 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7f46b00ce720 0x7f46b00ceb00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:50:08.596 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:08.592+0000 7f46b538b640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f46b015ecc0 0x7f46b006d2a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:50:08.596 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:08.592+0000 7f46b538b640 1 -- --> v1:172.21.15.203:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f46b00cd8a0 con 0x7f46b00ce080 2023-08-09T13:50:08.596 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:08.592+0000 7f46b538b640 1 -- --> v2:172.21.15.43:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f46b00cf350 con 0x7f46b015ecc0 2023-08-09T13:50:08.596 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:08.592+0000 7f46b538b640 1 -- --> v2:172.21.15.203:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f46b00cf040 con 0x7f46b00ce720 2023-08-09T13:50:08.596 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:08.593+0000 7f46af7fe640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f46b015ecc0 0x7f46b006d2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:50:08.597 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:08.593+0000 7f46af7fe640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f46b015ecc0 0x7f46b006d2a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:45336/0 (socket says 172.21.15.43:45336) 2023-08-09T13:50:08.597 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:08.593+0000 7f46af7fe640 1 -- 172.21.15.43:0/2186738629 learned_addr learned my addr 172.21.15.43:0/2186738629 (peer_addr_for_me v2:172.21.15.43:0/0) 2023-08-09T13:50:08.597 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:08.593+0000 7f46aeffd640 1 --2- 172.21.15.43:0/2186738629 >> v2:172.21.15.203:3300/0 conn(0x7f46b00ce720 0x7f46b00ceb00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:50:08.598 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:08.593+0000 7f46af7fe640 1 -- 172.21.15.43:0/2186738629 >> v2:172.21.15.203:3300/0 conn(0x7f46b00ce720 msgr2=0x7f46b00ceb00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-09T13:50:08.598 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:08.593+0000 7f46af7fe640 1 --2- 172.21.15.43:0/2186738629 >> v2:172.21.15.203:3300/0 conn(0x7f46b00ce720 0x7f46b00ceb00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:50:08.598 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:08.593+0000 7f46af7fe640 1 -- 172.21.15.43:0/2186738629 >> v1:172.21.15.203:6789/0 conn(0x7f46b00ce080 legacy=0x7f46b015e5b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-09T13:50:08.598 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:08.593+0000 7f46af7fe640 1 -- 172.21.15.43:0/2186738629 --> v2:172.21.15.43:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f46b0071a70 con 0x7f46b015ecc0 2023-08-09T13:50:08.598 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:08.595+0000 7f46af7fe640 1 --2- 172.21.15.43:0/2186738629 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f46b015ecc0 0x7f46b006d2a0 secure :-1 s=READY pgs=922 cs=0 l=1 rev1=1 crypto rx=0x7f46b00b2ec0 tx=0x7f46a802be50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d9f8176775b42a7d server_cookie=0 in_seq=0 out_seq=0 2023-08-09T13:50:08.598 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:08.595+0000 7f46adffb640 1 -- 172.21.15.43:0/2186738629 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f46a802c9f0 con 0x7f46b015ecc0 2023-08-09T13:50:08.599 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:08.595+0000 7f46adffb640 1 -- 172.21.15.43:0/2186738629 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f46a802cb50 con 0x7f46b015ecc0 2023-08-09T13:50:08.599 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:08.595+0000 7f46adffb640 1 -- 172.21.15.43:0/2186738629 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f46a80343f0 con 0x7f46b015ecc0 2023-08-09T13:50:08.599 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:08.595+0000 7f46b538b640 1 -- 172.21.15.43:0/2186738629 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f46b015ecc0 msgr2=0x7f46b006d2a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:50:08.600 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:08.595+0000 7f46b538b640 1 --2- 172.21.15.43:0/2186738629 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f46b015ecc0 0x7f46b006d2a0 secure :-1 s=READY pgs=922 cs=0 l=1 rev1=1 crypto rx=0x7f46b00b2ec0 tx=0x7f46a802be50 comp rx=0 tx=0).stop 2023-08-09T13:50:08.600 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:08.596+0000 7f46b538b640 1 -- 172.21.15.43:0/2186738629 shutdown_connections 2023-08-09T13:50:08.600 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:08.596+0000 7f46b538b640 1 --2- 172.21.15.43:0/2186738629 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f46b015ecc0 0x7f46b006d2a0 unknown :-1 s=CLOSED pgs=922 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:50:08.600 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:08.596+0000 7f46b538b640 1 --2- 172.21.15.43:0/2186738629 >> v2:172.21.15.203:3300/0 conn(0x7f46b00ce720 0x7f46b00ceb00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:50:08.600 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:08.596+0000 7f46b538b640 1 -- 172.21.15.43:0/2186738629 >> 172.21.15.43:0/2186738629 conn(0x7f46b00ccd80 msgr2=0x7f46b00cd190 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-09T13:50:08.600 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:08.596+0000 7f46b538b640 1 -- 172.21.15.43:0/2186738629 shutdown_connections 2023-08-09T13:50:08.600 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:08.596+0000 7f46b538b640 1 -- 172.21.15.43:0/2186738629 wait complete. 2023-08-09T13:50:08.601 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:08.597+0000 7f46b538b640 1 Processor -- start 2023-08-09T13:50:08.601 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:08.598+0000 7f46b538b640 1 -- start start 2023-08-09T13:50:08.602 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:08.599+0000 7f46b538b640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f46b00ce080 0x7f46b01ed310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:50:08.602 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:08.599+0000 7f46b538b640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f46b00ce720 0x7f46b00c3e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:50:08.602 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:08.599+0000 7f46b538b640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f46b015ecc0 0x7f46b00c43b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:50:08.602 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:08.599+0000 7f46b538b640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f46b00b2190 con 0x7f46b00ce720 2023-08-09T13:50:08.602 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:08.599+0000 7f46b538b640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f46b015e400 con 0x7f46b015ecc0 2023-08-09T13:50:08.602 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:08.599+0000 7f46b538b640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f46b00c9770 con 0x7f46b00ce080 2023-08-09T13:50:08.602 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:08.599+0000 7f46ae7fc640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f46b00ce720 0x7f46b00c3e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:50:08.603 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:08.599+0000 7f46ae7fc640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f46b00ce720 0x7f46b00c3e70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:45352/0 (socket says 172.21.15.43:45352) 2023-08-09T13:50:08.603 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:08.599+0000 7f46ae7fc640 1 -- 172.21.15.43:0/2217562432 learned_addr learned my addr 172.21.15.43:0/2217562432 (peer_addr_for_me v2:172.21.15.43:0/0) 2023-08-09T13:50:08.604 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:08.599+0000 7f46aeffd640 1 --2- 172.21.15.43:0/2217562432 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f46b00ce080 0x7f46b01ed310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:50:08.604 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:08.599+0000 7f46af7fe640 1 --2- 172.21.15.43:0/2217562432 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f46b015ecc0 0x7f46b00c43b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:50:08.604 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:08.599+0000 7f46ae7fc640 1 -- 172.21.15.43:0/2217562432 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f46b00ce080 msgr2=0x7f46b01ed310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:50:08.604 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:08.599+0000 7f46ae7fc640 1 --2- 172.21.15.43:0/2217562432 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f46b00ce080 0x7f46b01ed310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:50:08.604 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:08.599+0000 7f46ae7fc640 1 -- 172.21.15.43:0/2217562432 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f46b015ecc0 msgr2=0x7f46b00c43b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:50:08.604 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:08.599+0000 7f46ae7fc640 1 --2- 172.21.15.43:0/2217562432 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f46b015ecc0 0x7f46b00c43b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:50:08.605 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:08.599+0000 7f46ae7fc640 1 -- 172.21.15.43:0/2217562432 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f46a8009b30 con 0x7f46b00ce720 2023-08-09T13:50:08.605 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:08.600+0000 7f46ae7fc640 1 --2- 172.21.15.43:0/2217562432 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f46b00ce720 0x7f46b00c3e70 secure :-1 s=READY pgs=923 cs=0 l=1 rev1=1 crypto rx=0x7f469800b3c0 tx=0x7f469800b880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-09T13:50:08.605 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:08.600+0000 7f46a77fe640 1 -- 172.21.15.43:0/2217562432 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4698011830 con 0x7f46b00ce720 2023-08-09T13:50:08.605 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:08.600+0000 7f46a77fe640 1 -- 172.21.15.43:0/2217562432 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4698011990 con 0x7f46b00ce720 2023-08-09T13:50:08.605 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:08.600+0000 7f46a77fe640 1 -- 172.21.15.43:0/2217562432 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4698011d80 con 0x7f46b00ce720 2023-08-09T13:50:08.605 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:08.600+0000 7f46b538b640 1 -- 172.21.15.43:0/2217562432 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f46b00c6bc0 con 0x7f46b00ce720 2023-08-09T13:50:08.605 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:08.600+0000 7f46b538b640 1 -- 172.21.15.43:0/2217562432 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f46b00c9120 con 0x7f46b00ce720 2023-08-09T13:50:08.606 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:08.601+0000 7f46b538b640 1 -- 172.21.15.43:0/2217562432 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f46b00739f0 con 0x7f46b00ce720 2023-08-09T13:50:08.606 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:08.602+0000 7f46a77fe640 1 -- 172.21.15.43:0/2217562432 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 26) v1 ==== 88052+0+0 (secure 0 0 0) 0x7f469801a400 con 0x7f46b00ce720 2023-08-09T13:50:08.606 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:08.603+0000 7f46a77fe640 1 -- 172.21.15.43:0/2217562432 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(77..77 src has 1..77) v4 ==== 7272+0+0 (secure 0 0 0) 0x7f46980883f0 con 0x7f46b00ce720 2023-08-09T13:50:08.610 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:08.608+0000 7f46a77fe640 1 -- 172.21.15.43:0/2217562432 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178066 (secure 0 0 0) 0x7f4698053a30 con 0x7f46b00ce720 2023-08-09T13:50:08.819 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:08.816+0000 7f46b538b640 1 -- 172.21.15.43:0/2217562432 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f46b00cd440 con 0x7f46b00ce720 2023-08-09T13:50:08.823 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:08.821+0000 7f46a77fe640 1 -- 172.21.15.43:0/2217562432 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v77) v1 ==== 74+0+19824 (secure 0 0 0) 0x7f4698057050 con 0x7f46b00ce720 2023-08-09T13:50:08.824 INFO:teuthology.orchestra.run.smithi043.stdout: 2023-08-09T13:50:08.824 INFO:teuthology.orchestra.run.smithi043.stdout:{"epoch":77,"fsid":"daac87ee-1337-4250-a955-aea4e408a39b","created":"2023-08-09T13:33:41.701198+0000","modified":"2023-08-09T13:50:08.172832+0000","last_up_change":"2023-08-09T13:33:53.783453+0000","last_in_change":"2023-08-09T13:33:45.148904+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":9,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":"rbd","create_time":"2023-08-09T13:33:54.365137+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":".mgr","create_time":"2023-08-09T13:33:54.831803+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":8,"pool_name":"cephfs_metadata","create_time":"2023-08-09T13:48:16.039756+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"73","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":9,"pool_name":"cephfs_data","create_time":"2023-08-09T13:48:16.615400+0000","flags":8193,"flags_names":"hashpspool,selfmanaged_snaps","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"77","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":5,"snap_epoch":77,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.6299999952316284,"score_stable":1.6299999952316284,"optimal_score":1,"raw_score_acting":1.6299999952316284,"raw_score_stable":1.6299999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"cbd49ecd-e896-4fbd-864e-5c37a829b165","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":72,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6800","nonce":502667335},{"type":"v1","addr":"172.21.15.43:6801","nonce":502667335}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6802","nonce":502667335},{"type":"v1","addr":"172.21.15.43:6803","nonce":502667335}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6806","nonce":502667335},{"type":"v1","addr":"172.21.15.43:6807","nonce":502667335}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6804","nonce":502667335},{"type":"v1","addr":"172.21.15.43:6805","nonce":502667335}]},"public_addr":"172.21.15.43:6801/502667335","cluster_addr":"172.21.15.43:6803/502667335","heartbeat_back_addr":"172.21.15.43:6807/502667335","heartbeat_front_addr":"172.21.15.43:6805/502667335","state":["exists","up"]},{"osd":1,"uuid":"75c7e0fe-1ec7-4fe8-a2f2-3661a65b8500","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":72,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6808","nonce":3289134610},{"type":"v1","addr":"172.21.15.43:6809","nonce":3289134610}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6810","nonce":3289134610},{"type":"v1","addr":"172.21.15.43:6812","nonce":3289134610}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6821","nonce":3289134610},{"type":"v1","addr":"172.21.15.43:6824","nonce":3289134610}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6815","nonce":3289134610},{"type":"v1","addr":"172.21.15.43:6818","nonce":3289134610}]},"public_addr":"172.21.15.43:6809/3289134610","cluster_addr":"172.21.15.43:6812/3289134610","heartbeat_back_addr":"172.21.15.43:6824/3289134610","heartbeat_front_addr":"172.21.15.43:6818/3289134610","state":["exists","up"]},{"osd":2,"uuid":"bb5c41e9-7f5f-4125-9b01-3bb8bfc2ea83","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":72,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6811","nonce":4079709736},{"type":"v1","addr":"172.21.15.43:6814","nonce":4079709736}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6817","nonce":4079709736},{"type":"v1","addr":"172.21.15.43:6820","nonce":4079709736}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6829","nonce":4079709736},{"type":"v1","addr":"172.21.15.43:6831","nonce":4079709736}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6823","nonce":4079709736},{"type":"v1","addr":"172.21.15.43:6826","nonce":4079709736}]},"public_addr":"172.21.15.43:6814/4079709736","cluster_addr":"172.21.15.43:6820/4079709736","heartbeat_back_addr":"172.21.15.43:6831/4079709736","heartbeat_front_addr":"172.21.15.43:6826/4079709736","state":["exists","up"]},{"osd":3,"uuid":"bfe4c8b4-694a-465c-87e6-2b2aab852613","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":72,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6813","nonce":1709538759},{"type":"v1","addr":"172.21.15.43:6816","nonce":1709538759}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6819","nonce":1709538759},{"type":"v1","addr":"172.21.15.43:6822","nonce":1709538759}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6828","nonce":1709538759},{"type":"v1","addr":"172.21.15.43:6830","nonce":1709538759}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6825","nonce":1709538759},{"type":"v1","addr":"172.21.15.43:6827","nonce":1709538759}]},"public_addr":"172.21.15.43:6816/1709538759","cluster_addr":"172.21.15.43:6822/1709538759","heartbeat_back_addr":"172.21.15.43:6830/1709538759","heartbeat_front_addr":"172.21.15.43:6827/1709538759","state":["exists","up"]},{"osd":4,"uuid":"fa2e8a77-aba1-4756-853e-0d174a5bdeba","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":72,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6800","nonce":4196866481},{"type":"v1","addr":"172.21.15.203:6801","nonce":4196866481}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6802","nonce":4196866481},{"type":"v1","addr":"172.21.15.203:6803","nonce":4196866481}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6806","nonce":4196866481},{"type":"v1","addr":"172.21.15.203:6807","nonce":4196866481}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6804","nonce":4196866481},{"type":"v1","addr":"172.21.15.203:6805","nonce":4196866481}]},"public_addr":"172.21.15.203:6801/4196866481","cluster_addr":"172.21.15.203:6803/4196866481","heartbeat_back_addr":"172.21.15.203:6807/4196866481","heartbeat_front_addr":"172.21.15.203:6805/4196866481","state":["exists","up"]},{"osd":5,"uuid":"81811e4b-4d24-4f51-a82c-f1d700a2a00d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":72,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6816","nonce":3059290081},{"type":"v1","addr":"172.21.15.203:6817","nonce":3059290081}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6818","nonce":3059290081},{"type":"v1","addr":"172.21.15.203:6819","nonce":3059290081}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6822","nonce":3059290081},{"type":"v1","addr":"172.21.15.203:6823","nonce":3059290081}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6820","nonce":3059290081},{"type":"v1","addr":"172.21.15.203:6821","nonce":3059290081}]},"public_addr":"172.21.15.203:6817/3059290081","cluster_addr":"172.21.15.203:6819/3059290081","heartbeat_back_addr":"172.21.15.203:6823/3059290081","heartbeat_front_addr":"172.21.15.203:6821/3059290081","state":["exists","up"]},{"osd":6,"uuid":"307bac4d-6a5b-443d-af75-56d695d7674f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":72,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6824","nonce":512155569},{"type":"v1","addr":"172.21.15.203:6825","nonce":512155569}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6826","nonce":512155569},{"type":"v1","addr":"172.21.15.203:6827","nonce":512155569}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6830","nonce":512155569},{"type":"v1","addr":"172.21.15.203:6831","nonce":512155569}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6828","nonce":512155569},{"type":"v1","addr":"172.21.15.203:6829","nonce":512155569}]},"public_addr":"172.21.15.203:6825/512155569","cluster_addr":"172.21.15.203:6827/512155569","heartbeat_back_addr":"172.21.15.203:6831/512155569","heartbeat_front_addr":"172.21.15.203:6829/512155569","state":["exists","up"]},{"osd":7,"uuid":"a95dc7ca-9f1b-4295-af70-68a22a2d7d5e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":72,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6808","nonce":3331450438},{"type":"v1","addr":"172.21.15.203:6809","nonce":3331450438}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6810","nonce":3331450438},{"type":"v1","addr":"172.21.15.203:6811","nonce":3331450438}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6814","nonce":3331450438},{"type":"v1","addr":"172.21.15.203:6815","nonce":3331450438}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6812","nonce":3331450438},{"type":"v1","addr":"172.21.15.203:6813","nonce":3331450438}]},"public_addr":"172.21.15.203:6809/3331450438","cluster_addr":"172.21.15.203:6811/3331450438","heartbeat_back_addr":"172.21.15.203:6815/3331450438","heartbeat_front_addr":"172.21.15.203:6813/3331450438","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-09T13:33:50.547652+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-09T13:33:50.589469+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-09T13:33:50.561724+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-09T13:33:50.599650+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-09T13:33:50.411855+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-09T13:33:50.219681+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-09T13:33:50.504664+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-09T13:33:50.441248+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.203:0/2251174608":"2023-08-10T13:50:08.172778+0000","172.21.15.203:0/4008902326":"2023-08-10T13:50:08.172778+0000","172.21.15.203:0/1825829254":"2023-08-10T13:50:08.172778+0000","172.21.15.203:0/4097180925":"2023-08-10T13:50:08.172778+0000","172.21.15.203:6832/2515481191":"2023-08-10T13:50:08.172778+0000","172.21.15.203:0/4038087778":"2023-08-10T13:50:08.172778+0000","172.21.15.203:0/826156596":"2023-08-10T13:50:08.172778+0000","172.21.15.203:0/1084459051":"2023-08-10T13:50:08.172778+0000","172.21.15.203:6833/272917756":"2023-08-10T13:48:34.065493+0000","172.21.15.203:6832/272917756":"2023-08-10T13:48:34.065493+0000","172.21.15.203:0/665971116":"2023-08-10T13:48:34.065493+0000","172.21.15.203:0/2868823959":"2023-08-10T13:48:34.065493+0000","172.21.15.203:6833/2515481191":"2023-08-10T13:50:08.172778+0000","172.21.15.203:0/3168500363":"2023-08-10T13:48:34.065493+0000","172.21.15.203:0/752327795":"2023-08-10T13:48:34.065493+0000","172.21.15.203:0/1970263661":"2023-08-10T13:48:34.065493+0000","172.21.15.203:0/2726495528":"2023-08-10T13:50:08.172778+0000","172.21.15.203:0/1407921417":"2023-08-10T13:48:34.065493+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":7,"snaps":[{"begin":104,"length":1},{"begin":221,"length":1}]},{"pool":9,"snaps":[{"begin":2,"length":4}]}],"new_removed_snaps":[{"pool":9,"snaps":[{"begin":3,"length":1},{"begin":5,"length":1}]}],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-08-09T13:50:08.826 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:08.824+0000 7f46b538b640 1 -- 172.21.15.43:0/2217562432 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f46b00ce720 msgr2=0x7f46b00c3e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:50:08.827 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:08.824+0000 7f46b538b640 1 --2- 172.21.15.43:0/2217562432 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f46b00ce720 0x7f46b00c3e70 secure :-1 s=READY pgs=923 cs=0 l=1 rev1=1 crypto rx=0x7f469800b3c0 tx=0x7f469800b880 comp rx=0 tx=0).stop 2023-08-09T13:50:08.827 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:08.824+0000 7f46b538b640 1 -- 172.21.15.43:0/2217562432 shutdown_connections 2023-08-09T13:50:08.827 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:08.824+0000 7f46b538b640 1 --2- 172.21.15.43:0/2217562432 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f46b015ecc0 0x7f46b00c43b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:50:08.828 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:08.824+0000 7f46b538b640 1 --2- 172.21.15.43:0/2217562432 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f46b00ce720 0x7f46b00c3e70 unknown :-1 s=CLOSED pgs=923 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:50:08.828 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:08.824+0000 7f46b538b640 1 --2- 172.21.15.43:0/2217562432 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f46b00ce080 0x7f46b01ed310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:50:08.828 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:08.824+0000 7f46b538b640 1 -- 172.21.15.43:0/2217562432 >> 172.21.15.43:0/2217562432 conn(0x7f46b00ccd80 msgr2=0x7f46b0157850 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-09T13:50:08.828 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:08.824+0000 7f46b538b640 1 -- 172.21.15.43:0/2217562432 shutdown_connections 2023-08-09T13:50:08.828 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:08.824+0000 7f46b538b640 1 -- 172.21.15.43:0/2217562432 wait complete. 2023-08-09T13:50:08.849 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': 'rbd', 'create_time': '2023-08-09T13:33:54.365137+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}, 'read_balance': {'score_acting': 2, 'score_stable': 2, 'optimal_score': 1, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': '.mgr', 'create_time': '2023-08-09T13:33:54.831803+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 8, 'pool_name': 'cephfs_metadata', 'create_time': '2023-08-09T13:48:16.039756+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '73', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 9, 'pool_name': 'cephfs_data', 'create_time': '2023-08-09T13:48:16.615400+0000', 'flags': 8193, 'flags_names': 'hashpspool,selfmanaged_snaps', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '77', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 5, 'snap_epoch': 77, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.6299999952316284, 'score_stable': 1.6299999952316284, 'optimal_score': 1, 'raw_score_acting': 1.6299999952316284, 'raw_score_stable': 1.6299999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-08-09T13:50:08.850 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get rbd pg_num 2023-08-09T13:50:09.110 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:09.107+0000 7f059f59e640 1 Processor -- start 2023-08-09T13:50:09.111 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:09.108+0000 7f059f59e640 1 -- start start 2023-08-09T13:50:09.112 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:09.109+0000 7f059f59e640 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7f05a015c2d0 0x7f05a00b7df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:50:09.113 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:09.109+0000 7f059f59e640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f05a00cebf0 0x7f05a00cf080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:50:09.113 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:09.109+0000 7f059f59e640 1 -- --> v1:172.21.15.203:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f05a00d16a0 con 0x7f05a00d1b30 2023-08-09T13:50:09.113 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:09.109+0000 7f059f59e640 1 -- --> v2:172.21.15.203:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f05a00cf5c0 con 0x7f05a015c2d0 2023-08-09T13:50:09.114 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:09.109+0000 7f059f59e640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f05a00d1fa0 con 0x7f05a00cebf0 2023-08-09T13:50:09.115 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:09.110+0000 7f059ed9d640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f05a00cebf0 0x7f05a00cf080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:50:09.115 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:09.110+0000 7f059ed9d640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f05a00cebf0 0x7f05a00cf080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.203:3301/0 says I am v2:172.21.15.43:34034/0 (socket says 172.21.15.43:34034) 2023-08-09T13:50:09.115 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:09.110+0000 7f059ed9d640 1 -- 172.21.15.43:0/1700977217 learned_addr learned my addr 172.21.15.43:0/1700977217 (peer_addr_for_me v2:172.21.15.43:0/0) 2023-08-09T13:50:09.115 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:09.110+0000 7f059e59c640 1 --2- 172.21.15.43:0/1700977217 >> v2:172.21.15.203:3300/0 conn(0x7f05a015c2d0 0x7f05a00b7df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:50:09.115 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:09.111+0000 7f059ed9d640 1 -- 172.21.15.43:0/1700977217 >> v2:172.21.15.203:3300/0 conn(0x7f05a015c2d0 msgr2=0x7f05a00b7df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-09T13:50:09.115 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:09.111+0000 7f059ed9d640 1 --2- 172.21.15.43:0/1700977217 >> v2:172.21.15.203:3300/0 conn(0x7f05a015c2d0 0x7f05a00b7df0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:50:09.115 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:09.111+0000 7f059ed9d640 1 -- 172.21.15.43:0/1700977217 >> v1:172.21.15.203:6789/0 conn(0x7f05a00d1b30 legacy=0x7f05a00cd020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:50:09.116 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:09.111+0000 7f059ed9d640 1 -- 172.21.15.43:0/1700977217 --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f05a00d0d70 con 0x7f05a00cebf0 2023-08-09T13:50:09.116 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:09.112+0000 7f059ed9d640 1 --2- 172.21.15.43:0/1700977217 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f05a00cebf0 0x7f05a00cf080 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7f0598009870 tx=0x7f059802b980 comp rx=0 tx=0).ready entity=mon.2 client_cookie=638561dd85a450db server_cookie=0 in_seq=0 out_seq=0 2023-08-09T13:50:09.117 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:09.113+0000 7f059d59a640 1 -- 172.21.15.43:0/1700977217 <== mon.2 v2:172.21.15.203:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0598030840 con 0x7f05a00cebf0 2023-08-09T13:50:09.117 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:09.113+0000 7f059d59a640 1 -- 172.21.15.43:0/1700977217 <== mon.2 v2:172.21.15.203:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f05980309a0 con 0x7f05a00cebf0 2023-08-09T13:50:09.117 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:09.113+0000 7f059d59a640 1 -- 172.21.15.43:0/1700977217 <== mon.2 v2:172.21.15.203:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0598030d30 con 0x7f05a00cebf0 2023-08-09T13:50:09.117 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:09.113+0000 7f059f59e640 1 -- 172.21.15.43:0/1700977217 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f05a00cebf0 msgr2=0x7f05a00cf080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:50:09.117 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:09.113+0000 7f059f59e640 1 --2- 172.21.15.43:0/1700977217 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f05a00cebf0 0x7f05a00cf080 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7f0598009870 tx=0x7f059802b980 comp rx=0 tx=0).stop 2023-08-09T13:50:09.117 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:09.114+0000 7f059f59e640 1 -- 172.21.15.43:0/1700977217 shutdown_connections 2023-08-09T13:50:09.118 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:09.114+0000 7f059f59e640 1 --2- 172.21.15.43:0/1700977217 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f05a00cebf0 0x7f05a00cf080 unknown :-1 s=CLOSED pgs=130 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:50:09.118 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:09.114+0000 7f059f59e640 1 --2- 172.21.15.43:0/1700977217 >> v2:172.21.15.203:3300/0 conn(0x7f05a015c2d0 0x7f05a00b7df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:50:09.118 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:09.114+0000 7f059f59e640 1 -- 172.21.15.43:0/1700977217 >> 172.21.15.43:0/1700977217 conn(0x7f05a006a230 msgr2=0x7f05a006a640 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-09T13:50:09.118 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:09.114+0000 7f059f59e640 1 -- 172.21.15.43:0/1700977217 shutdown_connections 2023-08-09T13:50:09.118 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:09.114+0000 7f059f59e640 1 -- 172.21.15.43:0/1700977217 wait complete. 2023-08-09T13:50:09.118 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:09.116+0000 7f059f59e640 1 Processor -- start 2023-08-09T13:50:09.119 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:09.116+0000 7f059f59e640 1 -- start start 2023-08-09T13:50:09.120 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:09.118+0000 7f059f59e640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f05a00cebf0 0x7f05a015e880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:50:09.120 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:09.118+0000 7f059f59e640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f05a00d1b30 0x7f05a01f11c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:50:09.121 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:09.118+0000 7f059f59e640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f05a015c2d0 0x7f05a0154da0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:50:09.121 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:09.118+0000 7f059f59e640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f05a00c92b0 con 0x7f05a015c2d0 2023-08-09T13:50:09.121 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:09.118+0000 7f059f59e640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f05a00c9500 con 0x7f05a00cebf0 2023-08-09T13:50:09.121 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:09.118+0000 7f059f59e640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f05a00c97e0 con 0x7f05a00d1b30 2023-08-09T13:50:09.121 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:09.118+0000 7f059ed9d640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f05a015c2d0 0x7f05a0154da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:50:09.121 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:09.118+0000 7f059ed9d640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f05a015c2d0 0x7f05a0154da0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:45368/0 (socket says 172.21.15.43:45368) 2023-08-09T13:50:09.121 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:09.118+0000 7f059ed9d640 1 -- 172.21.15.43:0/3931784835 learned_addr learned my addr 172.21.15.43:0/3931784835 (peer_addr_for_me v2:172.21.15.43:0/0) 2023-08-09T13:50:09.122 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:09.118+0000 7f059e59c640 1 --2- 172.21.15.43:0/3931784835 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f05a00cebf0 0x7f05a015e880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:50:09.123 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:09.118+0000 7f059dd9b640 1 --2- 172.21.15.43:0/3931784835 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f05a00d1b30 0x7f05a01f11c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:50:09.123 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:09.119+0000 7f059ed9d640 1 -- 172.21.15.43:0/3931784835 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f05a00d1b30 msgr2=0x7f05a01f11c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:50:09.123 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:09.119+0000 7f059ed9d640 1 --2- 172.21.15.43:0/3931784835 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f05a00d1b30 0x7f05a01f11c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:50:09.123 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:09.119+0000 7f059ed9d640 1 -- 172.21.15.43:0/3931784835 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f05a00cebf0 msgr2=0x7f05a015e880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:50:09.123 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:09.119+0000 7f059ed9d640 1 --2- 172.21.15.43:0/3931784835 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f05a00cebf0 0x7f05a015e880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:50:09.123 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:09.119+0000 7f059ed9d640 1 -- 172.21.15.43:0/3931784835 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f05980094d0 con 0x7f05a015c2d0 2023-08-09T13:50:09.124 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:09.119+0000 7f059ed9d640 1 --2- 172.21.15.43:0/3931784835 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f05a015c2d0 0x7f05a0154da0 secure :-1 s=READY pgs=924 cs=0 l=1 rev1=1 crypto rx=0x7f0598009870 tx=0x7f0598004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-09T13:50:09.124 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:09.119+0000 7f0596ffd640 1 -- 172.21.15.43:0/3931784835 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0598039a90 con 0x7f05a015c2d0 2023-08-09T13:50:09.124 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:09.119+0000 7f0596ffd640 1 -- 172.21.15.43:0/3931784835 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0598039c10 con 0x7f05a015c2d0 2023-08-09T13:50:09.124 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:09.119+0000 7f059f59e640 1 -- 172.21.15.43:0/3931784835 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f05a00c9ac0 con 0x7f05a015c2d0 2023-08-09T13:50:09.124 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:09.119+0000 7f0596ffd640 1 -- 172.21.15.43:0/3931784835 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0598043410 con 0x7f05a015c2d0 2023-08-09T13:50:09.124 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:09.120+0000 7f059f59e640 1 -- 172.21.15.43:0/3931784835 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f05a015edc0 con 0x7f05a015c2d0 2023-08-09T13:50:09.125 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:09.121+0000 7f059f59e640 1 -- 172.21.15.43:0/3931784835 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f05a00c9fe0 con 0x7f05a015c2d0 2023-08-09T13:50:09.125 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:09.122+0000 7f0596ffd640 1 -- 172.21.15.43:0/3931784835 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 26) v1 ==== 88052+0+0 (secure 0 0 0) 0x7f059802f390 con 0x7f05a015c2d0 2023-08-09T13:50:09.125 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:09.123+0000 7f0596ffd640 1 -- 172.21.15.43:0/3931784835 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(77..77 src has 1..77) v4 ==== 7272+0+0 (secure 0 0 0) 0x7f05980aa660 con 0x7f05a015c2d0 2023-08-09T13:50:09.129 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:09.127+0000 7f0596ffd640 1 -- 172.21.15.43:0/3931784835 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178066 (secure 0 0 0) 0x7f0598075bf0 con 0x7f05a015c2d0 2023-08-09T13:50:09.198 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:09.196+0000 7f0596ffd640 1 -- 172.21.15.43:0/3931784835 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mgrmap(e 27) v1 ==== 88179+0+0 (secure 0 0 0) 0x7f05980756a0 con 0x7f05a015c2d0 2023-08-09T13:50:09.199 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:09.197+0000 7f0596ffd640 1 --2- 172.21.15.43:0/3931784835 >> [v2:172.21.15.203:6832/3325141227,v1:172.21.15.203:6833/3325141227] conn(0x7f05800705c0 0x7f05800729b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:50:09.200 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:09.198+0000 7f059e59c640 1 --2- 172.21.15.43:0/3931784835 >> [v2:172.21.15.203:6832/3325141227,v1:172.21.15.203:6833/3325141227] conn(0x7f05800705c0 0x7f05800729b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:50:09.202 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:09.200+0000 7f059e59c640 1 --2- 172.21.15.43:0/3931784835 >> [v2:172.21.15.203:6832/3325141227,v1:172.21.15.203:6833/3325141227] conn(0x7f05800705c0 0x7f05800729b0 secure :-1 s=READY pgs=4 cs=0 l=1 rev1=1 crypto rx=0x7f05900074a0 tx=0x7f059000b040 comp rx=0 tx=0).ready entity=mgr.5519 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-09T13:50:09.333 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:09.331+0000 7f059f59e640 1 -- 172.21.15.43:0/3931784835 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"} v 0) v1 -- 0x7f05a004e390 con 0x7f05a015c2d0 2023-08-09T13:50:09.334 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:09.331+0000 7f0596ffd640 1 -- 172.21.15.43:0/3931784835 <== mon.0 v2:172.21.15.43:3300/0 8 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"}]=0 v77) v1 ==== 92+0+10 (secure 0 0 0) 0x7f0598034070 con 0x7f05a015c2d0 2023-08-09T13:50:09.334 INFO:teuthology.orchestra.run.smithi043.stdout:pg_num: 8 2023-08-09T13:50:09.336 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:09.334+0000 7f059f59e640 1 -- 172.21.15.43:0/3931784835 >> [v2:172.21.15.203:6832/3325141227,v1:172.21.15.203:6833/3325141227] conn(0x7f05800705c0 msgr2=0x7f05800729b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:50:09.337 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:09.334+0000 7f059f59e640 1 --2- 172.21.15.43:0/3931784835 >> [v2:172.21.15.203:6832/3325141227,v1:172.21.15.203:6833/3325141227] conn(0x7f05800705c0 0x7f05800729b0 secure :-1 s=READY pgs=4 cs=0 l=1 rev1=1 crypto rx=0x7f05900074a0 tx=0x7f059000b040 comp rx=0 tx=0).stop 2023-08-09T13:50:09.337 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:09.334+0000 7f059f59e640 1 -- 172.21.15.43:0/3931784835 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f05a015c2d0 msgr2=0x7f05a0154da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:50:09.337 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:09.334+0000 7f059f59e640 1 --2- 172.21.15.43:0/3931784835 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f05a015c2d0 0x7f05a0154da0 secure :-1 s=READY pgs=924 cs=0 l=1 rev1=1 crypto rx=0x7f0598009870 tx=0x7f0598004730 comp rx=0 tx=0).stop 2023-08-09T13:50:09.337 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:09.334+0000 7f059f59e640 1 -- 172.21.15.43:0/3931784835 shutdown_connections 2023-08-09T13:50:09.338 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:09.334+0000 7f059f59e640 1 --2- 172.21.15.43:0/3931784835 >> [v2:172.21.15.203:6832/3325141227,v1:172.21.15.203:6833/3325141227] conn(0x7f05800705c0 0x7f05800729b0 unknown :-1 s=CLOSED pgs=4 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:50:09.338 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:09.334+0000 7f059f59e640 1 --2- 172.21.15.43:0/3931784835 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f05a015c2d0 0x7f05a0154da0 unknown :-1 s=CLOSED pgs=924 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:50:09.338 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:09.334+0000 7f059f59e640 1 --2- 172.21.15.43:0/3931784835 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f05a00d1b30 0x7f05a01f11c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:50:09.338 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:09.334+0000 7f059f59e640 1 --2- 172.21.15.43:0/3931784835 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f05a00cebf0 0x7f05a015e880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:50:09.338 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:09.334+0000 7f059f59e640 1 -- 172.21.15.43:0/3931784835 >> 172.21.15.43:0/3931784835 conn(0x7f05a006a230 msgr2=0x7f05a00b1e20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-09T13:50:09.338 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:09.335+0000 7f059f59e640 1 -- 172.21.15.43:0/3931784835 shutdown_connections 2023-08-09T13:50:09.338 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:09.335+0000 7f059f59e640 1 -- 172.21.15.43:0/3931784835 wait complete. 2023-08-09T13:50:09.356 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-08-09T13:50:09.584 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:09.581+0000 7f239e60a640 1 Processor -- start 2023-08-09T13:50:09.585 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:09.582+0000 7f239e60a640 1 -- start start 2023-08-09T13:50:09.586 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:09.583+0000 7f239e60a640 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7f23980cf300 0x7f23980d1b30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:50:09.586 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:09.583+0000 7f239e60a640 1 -- --> v1:172.21.15.43:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f23980d16a0 con 0x7f23980b7df0 2023-08-09T13:50:09.586 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:09.583+0000 7f239e60a640 1 -- --> v1:172.21.15.203:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f23980d0d70 con 0x7f239815c2d0 2023-08-09T13:50:09.586 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:09.583+0000 7f239e60a640 1 -- --> v2:172.21.15.203:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f23980b8230 con 0x7f23980cf300 2023-08-09T13:50:09.587 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:09.584+0000 7f239ce07640 1 --1- >> v1:172.21.15.43:6789/0 conn(0x7f23980b7df0 0x7f23980cebf0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.43:6789/0 says I am v1:172.21.15.43:58772/0 (socket says 172.21.15.43:58772) 2023-08-09T13:50:09.588 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:09.584+0000 7f239ce07640 1 -- 172.21.15.43:0/3580708002 learned_addr learned my addr 172.21.15.43:0/3580708002 (peer_addr_for_me v1:172.21.15.43:0/0) 2023-08-09T13:50:09.588 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:09.584+0000 7f239de09640 1 --2- 172.21.15.43:0/3580708002 >> v2:172.21.15.203:3300/0 conn(0x7f23980cf300 0x7f23980d1b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:50:09.588 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:09.584+0000 7f2397fff640 1 -- 172.21.15.43:0/3580708002 <== mon.0 v1:172.21.15.43:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3452088176 0 0) 0x7f23980d16a0 con 0x7f23980b7df0 2023-08-09T13:50:09.588 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:09.584+0000 7f2397fff640 1 -- 172.21.15.43:0/3580708002 --> v1:172.21.15.43:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f2380003520 con 0x7f23980b7df0 2023-08-09T13:50:09.588 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:09.585+0000 7f2397fff640 1 -- 172.21.15.43:0/3580708002 <== mon.0 v1:172.21.15.43:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 877511557 0 0) 0x7f2380003520 con 0x7f23980b7df0 2023-08-09T13:50:09.588 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:09.585+0000 7f2397fff640 1 -- 172.21.15.43:0/3580708002 >> v2:172.21.15.203:3300/0 conn(0x7f23980cf300 msgr2=0x7f23980d1b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-09T13:50:09.589 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:09.585+0000 7f2397fff640 1 --2- 172.21.15.43:0/3580708002 >> v2:172.21.15.203:3300/0 conn(0x7f23980cf300 0x7f23980d1b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:50:09.589 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:09.585+0000 7f2397fff640 1 -- 172.21.15.43:0/3580708002 >> v1:172.21.15.203:6789/0 conn(0x7f239815c2d0 legacy=0x7f23980cd020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:50:09.589 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:09.585+0000 7f2397fff640 1 -- 172.21.15.43:0/3580708002 --> v1:172.21.15.43:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f23980d1050 con 0x7f23980b7df0 2023-08-09T13:50:09.589 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:09.585+0000 7f2397fff640 1 -- 172.21.15.43:0/3580708002 <== mon.0 v1:172.21.15.43:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1285150740 0 0) 0x7f238c001670 con 0x7f23980b7df0 2023-08-09T13:50:09.589 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:09.585+0000 7f2397fff640 1 -- 172.21.15.43:0/3580708002 >> v1:172.21.15.43:6789/0 conn(0x7f23980b7df0 legacy=0x7f23980cebf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:50:09.589 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:09.585+0000 7f2397fff640 1 --2- 172.21.15.43:0/3580708002 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f2380003970 0x7f2380003d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:50:09.589 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:09.585+0000 7f2397fff640 1 --2- 172.21.15.43:0/3580708002 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f23800047c0 0x7f2380004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:50:09.589 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:09.585+0000 7f2397fff640 1 --2- 172.21.15.43:0/3580708002 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f2380005180 0x7f238000b980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:50:09.590 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:09.585+0000 7f2397fff640 1 -- 172.21.15.43:0/3580708002 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f238000bff0 con 0x7f23800047c0 2023-08-09T13:50:09.590 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:09.585+0000 7f2397fff640 1 -- 172.21.15.43:0/3580708002 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f238000c160 con 0x7f2380005180 2023-08-09T13:50:09.590 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:09.585+0000 7f2397fff640 1 -- 172.21.15.43:0/3580708002 --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f238000c320 con 0x7f2380003970 2023-08-09T13:50:09.590 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:09.585+0000 7f239ce07640 1 --2- 172.21.15.43:0/3580708002 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f23800047c0 0x7f2380004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:50:09.591 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:09.586+0000 7f239d608640 1 --2- 172.21.15.43:0/3580708002 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f2380003970 0x7f2380003d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:50:09.591 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:09.586+0000 7f239ce07640 1 -- 172.21.15.43:0/3580708002 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f2380003970 msgr2=0x7f2380003d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-09T13:50:09.591 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:09.586+0000 7f239ce07640 1 --2- 172.21.15.43:0/3580708002 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f2380003970 0x7f2380003d90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:50:09.592 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:09.586+0000 7f239ce07640 1 -- 172.21.15.43:0/3580708002 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f2380005180 msgr2=0x7f238000b980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-09T13:50:09.592 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:09.586+0000 7f239ce07640 1 --2- 172.21.15.43:0/3580708002 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f2380005180 0x7f238000b980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:50:09.592 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:09.586+0000 7f239ce07640 1 -- 172.21.15.43:0/3580708002 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2380003520 con 0x7f23800047c0 2023-08-09T13:50:09.592 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:09.586+0000 7f239de09640 1 -- 172.21.15.43:0/3580708002 reap_dead start 2023-08-09T13:50:09.592 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:09.587+0000 7f239ce07640 1 --2- 172.21.15.43:0/3580708002 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f23800047c0 0x7f2380004c40 secure :-1 s=READY pgs=926 cs=0 l=1 rev1=1 crypto rx=0x7f238c00c240 tx=0x7f23800057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b4667230920e99c1 server_cookie=0 in_seq=0 out_seq=0 2023-08-09T13:50:09.592 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:09.587+0000 7f2397fff640 1 -- 172.21.15.43:0/3580708002 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f238c005ad0 con 0x7f23800047c0 2023-08-09T13:50:09.592 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:09.587+0000 7f2397fff640 1 -- 172.21.15.43:0/3580708002 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f238c005c30 con 0x7f23800047c0 2023-08-09T13:50:09.592 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:09.587+0000 7f2397fff640 1 -- 172.21.15.43:0/3580708002 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f238c02caf0 con 0x7f23800047c0 2023-08-09T13:50:09.593 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:09.588+0000 7f239e60a640 1 -- 172.21.15.43:0/3580708002 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f23800047c0 msgr2=0x7f2380004c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:50:09.593 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:09.588+0000 7f239e60a640 1 --2- 172.21.15.43:0/3580708002 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f23800047c0 0x7f2380004c40 secure :-1 s=READY pgs=926 cs=0 l=1 rev1=1 crypto rx=0x7f238c00c240 tx=0x7f23800057f0 comp rx=0 tx=0).stop 2023-08-09T13:50:09.593 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:09.588+0000 7f239e60a640 1 -- 172.21.15.43:0/3580708002 shutdown_connections 2023-08-09T13:50:09.593 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:09.588+0000 7f239e60a640 1 --2- 172.21.15.43:0/3580708002 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f23800047c0 0x7f2380004c40 unknown :-1 s=CLOSED pgs=926 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:50:09.593 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:09.588+0000 7f239e60a640 1 -- 172.21.15.43:0/3580708002 >> 172.21.15.43:0/3580708002 conn(0x7f239806a230 msgr2=0x7f239806a640 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-09T13:50:09.593 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:09.588+0000 7f239e60a640 1 -- 172.21.15.43:0/3580708002 shutdown_connections 2023-08-09T13:50:09.593 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:09.588+0000 7f239e60a640 1 -- 172.21.15.43:0/3580708002 wait complete. 2023-08-09T13:50:09.595 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:09.590+0000 7f239e60a640 1 Processor -- start 2023-08-09T13:50:09.595 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:09.590+0000 7f239e60a640 1 -- start start 2023-08-09T13:50:09.595 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:09.591+0000 7f239e60a640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f23800047c0 0x7f239814bdd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:50:09.596 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:09.591+0000 7f239e60a640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f2398149250 0x7f239814aa00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:50:09.596 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:09.591+0000 7f239e60a640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f23980b04c0 0x7f23980b3ba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:50:09.596 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:09.591+0000 7f239e60a640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f23981496d0 con 0x7f23980b04c0 2023-08-09T13:50:09.596 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:09.591+0000 7f239e60a640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2398149960 con 0x7f23800047c0 2023-08-09T13:50:09.596 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:09.591+0000 7f239e60a640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2398149c40 con 0x7f2398149250 2023-08-09T13:50:09.596 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:09.591+0000 7f239de09640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f23980b04c0 0x7f23980b3ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:50:09.596 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:09.591+0000 7f239de09640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f23980b04c0 0x7f23980b3ba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:45378/0 (socket says 172.21.15.43:45378) 2023-08-09T13:50:09.597 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:09.591+0000 7f239de09640 1 -- 172.21.15.43:0/4172668547 learned_addr learned my addr 172.21.15.43:0/4172668547 (peer_addr_for_me v2:172.21.15.43:0/0) 2023-08-09T13:50:09.597 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:09.591+0000 7f239ce07640 1 --2- 172.21.15.43:0/4172668547 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f2398149250 0x7f239814aa00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:50:09.597 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:09.591+0000 7f239d608640 1 --2- 172.21.15.43:0/4172668547 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f23800047c0 0x7f239814bdd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:50:09.597 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:09.592+0000 7f239de09640 1 -- 172.21.15.43:0/4172668547 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f2398149250 msgr2=0x7f239814aa00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:50:09.597 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:09.592+0000 7f239de09640 1 --2- 172.21.15.43:0/4172668547 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f2398149250 0x7f239814aa00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:50:09.597 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:09.592+0000 7f239de09640 1 -- 172.21.15.43:0/4172668547 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f23800047c0 msgr2=0x7f239814bdd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:50:09.597 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:09.592+0000 7f239de09640 1 --2- 172.21.15.43:0/4172668547 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f23800047c0 0x7f239814bdd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:50:09.597 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:09.592+0000 7f239de09640 1 -- 172.21.15.43:0/4172668547 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2398149f20 con 0x7f23980b04c0 2023-08-09T13:50:09.598 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:09.592+0000 7f239d608640 1 --2- 172.21.15.43:0/4172668547 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f23800047c0 0x7f239814bdd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-08-09T13:50:09.598 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:09.592+0000 7f239de09640 1 --2- 172.21.15.43:0/4172668547 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f23980b04c0 0x7f23980b3ba0 secure :-1 s=READY pgs=927 cs=0 l=1 rev1=1 crypto rx=0x7f23980d1610 tx=0x7f239000eea0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-09T13:50:09.598 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:09.592+0000 7f2395ffb640 1 -- 172.21.15.43:0/4172668547 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2390010040 con 0x7f23980b04c0 2023-08-09T13:50:09.598 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:09.592+0000 7f2395ffb640 1 -- 172.21.15.43:0/4172668547 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f239000bab0 con 0x7f23980b04c0 2023-08-09T13:50:09.598 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:09.592+0000 7f2395ffb640 1 -- 172.21.15.43:0/4172668547 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f239000be60 con 0x7f23980b04c0 2023-08-09T13:50:09.598 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:09.592+0000 7f239e60a640 1 -- 172.21.15.43:0/4172668547 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f239814a200 con 0x7f23980b04c0 2023-08-09T13:50:09.598 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:09.593+0000 7f239e60a640 1 -- 172.21.15.43:0/4172668547 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f239814af40 con 0x7f23980b04c0 2023-08-09T13:50:09.599 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:09.594+0000 7f239e60a640 1 -- 172.21.15.43:0/4172668547 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f239804e390 con 0x7f23980b04c0 2023-08-09T13:50:09.599 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:09.595+0000 7f2395ffb640 1 -- 172.21.15.43:0/4172668547 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 27) v1 ==== 88179+0+0 (secure 0 0 0) 0x7f239001d400 con 0x7f23980b04c0 2023-08-09T13:50:09.599 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:09.596+0000 7f2395ffb640 1 --2- 172.21.15.43:0/4172668547 >> [v2:172.21.15.203:6832/3325141227,v1:172.21.15.203:6833/3325141227] conn(0x7f2384069e70 0x7f238406c330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:50:09.600 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:09.596+0000 7f2395ffb640 1 -- 172.21.15.43:0/4172668547 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(77..77 src has 1..77) v4 ==== 7272+0+0 (secure 0 0 0) 0x7f2390088e40 con 0x7f23980b04c0 2023-08-09T13:50:09.600 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:09.596+0000 7f239d608640 1 --2- 172.21.15.43:0/4172668547 >> [v2:172.21.15.203:6832/3325141227,v1:172.21.15.203:6833/3325141227] conn(0x7f2384069e70 0x7f238406c330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:50:09.600 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:09.598+0000 7f239d608640 1 --2- 172.21.15.43:0/4172668547 >> [v2:172.21.15.203:6832/3325141227,v1:172.21.15.203:6833/3325141227] conn(0x7f2384069e70 0x7f238406c330 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7f2398063390 tx=0x7f2388009040 comp rx=0 tx=0).ready entity=mgr.5519 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-09T13:50:09.602 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:09.600+0000 7f2395ffb640 1 -- 172.21.15.43:0/4172668547 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178066 (secure 0 0 0) 0x7f2390057d70 con 0x7f23980b04c0 2023-08-09T13:50:09.794 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:09.791+0000 7f239e60a640 1 -- 172.21.15.43:0/4172668547 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f239815f170 con 0x7f23980b04c0 2023-08-09T13:50:09.794 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:09.792+0000 7f2395ffb640 1 -- 172.21.15.43:0/4172668547 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v77) v1 ==== 93+0+10 (secure 0 0 0) 0x7f2390057900 con 0x7f23980b04c0 2023-08-09T13:50:09.795 INFO:teuthology.orchestra.run.smithi043.stdout:pg_num: 1 2023-08-09T13:50:09.796 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:09.794+0000 7f239e60a640 1 -- 172.21.15.43:0/4172668547 >> [v2:172.21.15.203:6832/3325141227,v1:172.21.15.203:6833/3325141227] conn(0x7f2384069e70 msgr2=0x7f238406c330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:50:09.797 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:09.794+0000 7f239e60a640 1 --2- 172.21.15.43:0/4172668547 >> [v2:172.21.15.203:6832/3325141227,v1:172.21.15.203:6833/3325141227] conn(0x7f2384069e70 0x7f238406c330 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7f2398063390 tx=0x7f2388009040 comp rx=0 tx=0).stop 2023-08-09T13:50:09.797 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:09.794+0000 7f239e60a640 1 -- 172.21.15.43:0/4172668547 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f23980b04c0 msgr2=0x7f23980b3ba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:50:09.798 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:09.794+0000 7f239e60a640 1 --2- 172.21.15.43:0/4172668547 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f23980b04c0 0x7f23980b3ba0 secure :-1 s=READY pgs=927 cs=0 l=1 rev1=1 crypto rx=0x7f23980d1610 tx=0x7f239000eea0 comp rx=0 tx=0).stop 2023-08-09T13:50:09.798 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:09.794+0000 7f239e60a640 1 -- 172.21.15.43:0/4172668547 shutdown_connections 2023-08-09T13:50:09.798 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:09.794+0000 7f239e60a640 1 --2- 172.21.15.43:0/4172668547 >> [v2:172.21.15.203:6832/3325141227,v1:172.21.15.203:6833/3325141227] conn(0x7f2384069e70 0x7f238406c330 unknown :-1 s=CLOSED pgs=22 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:50:09.798 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:09.794+0000 7f239e60a640 1 --2- 172.21.15.43:0/4172668547 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f23980b04c0 0x7f23980b3ba0 unknown :-1 s=CLOSED pgs=927 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:50:09.798 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:09.794+0000 7f239e60a640 1 --2- 172.21.15.43:0/4172668547 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f2398149250 0x7f239814aa00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:50:09.798 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:09.794+0000 7f239e60a640 1 --2- 172.21.15.43:0/4172668547 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f23800047c0 0x7f239814bdd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:50:09.798 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:09.794+0000 7f239e60a640 1 -- 172.21.15.43:0/4172668547 >> 172.21.15.43:0/4172668547 conn(0x7f239806a230 msgr2=0x7f23980cccd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-09T13:50:09.799 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:09.795+0000 7f239e60a640 1 -- 172.21.15.43:0/4172668547 shutdown_connections 2023-08-09T13:50:09.799 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:09.795+0000 7f239e60a640 1 -- 172.21.15.43:0/4172668547 wait complete. 2023-08-09T13:50:09.816 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-08-09T13:50:10.045 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:10.043+0000 7f2465b64640 1 Processor -- start 2023-08-09T13:50:10.046 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:10.043+0000 7f2465b64640 1 -- start start 2023-08-09T13:50:10.047 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:10.045+0000 7f2465b64640 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7f24600b3fa0 0x7f24600b43b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:50:10.047 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:10.045+0000 7f2465b64640 1 -- --> v1:172.21.15.43:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f24600b48f0 con 0x7f24600d0a70 2023-08-09T13:50:10.048 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:10.045+0000 7f2465b64640 1 -- --> v1:172.21.15.203:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f24600cdf00 con 0x7f246015c680 2023-08-09T13:50:10.048 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:10.045+0000 7f2465b64640 1 -- --> v2:172.21.15.203:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f24600d0f70 con 0x7f24600b3fa0 2023-08-09T13:50:10.048 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:10.045+0000 7f2457fff640 1 --1- >> v1:172.21.15.43:6789/0 conn(0x7f24600d0a70 0x7f24600cd7f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.43:6789/0 says I am v1:172.21.15.43:58780/0 (socket says 172.21.15.43:58780) 2023-08-09T13:50:10.049 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:10.045+0000 7f2457fff640 1 -- 172.21.15.43:0/2614554329 learned_addr learned my addr 172.21.15.43:0/2614554329 (peer_addr_for_me v1:172.21.15.43:0/0) 2023-08-09T13:50:10.049 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:10.046+0000 7f245ffff640 1 --2- 172.21.15.43:0/2614554329 >> v2:172.21.15.203:3300/0 conn(0x7f24600b3fa0 0x7f24600b43b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:50:10.049 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:10.046+0000 7f245effd640 1 -- 172.21.15.43:0/2614554329 <== mon.0 v1:172.21.15.43:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1709633193 0 0) 0x7f24600b48f0 con 0x7f24600d0a70 2023-08-09T13:50:10.049 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:10.046+0000 7f245effd640 1 -- 172.21.15.43:0/2614554329 --> v1:172.21.15.43:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f244c003520 con 0x7f24600d0a70 2023-08-09T13:50:10.050 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:10.046+0000 7f245effd640 1 -- 172.21.15.43:0/2614554329 <== mon.0 v1:172.21.15.43:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 138470078 0 0) 0x7f244c003520 con 0x7f24600d0a70 2023-08-09T13:50:10.050 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:10.046+0000 7f245effd640 1 -- 172.21.15.43:0/2614554329 >> v2:172.21.15.203:3300/0 conn(0x7f24600b3fa0 msgr2=0x7f24600b43b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-09T13:50:10.051 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:10.046+0000 7f245effd640 1 --2- 172.21.15.43:0/2614554329 >> v2:172.21.15.203:3300/0 conn(0x7f24600b3fa0 0x7f24600b43b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:50:10.051 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:10.046+0000 7f245effd640 1 -- 172.21.15.43:0/2614554329 >> v1:172.21.15.203:6789/0 conn(0x7f246015c680 legacy=0x7f24600ce760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:50:10.051 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:10.046+0000 7f245effd640 1 -- 172.21.15.43:0/2614554329 --> v1:172.21.15.43:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f24600ccd10 con 0x7f24600d0a70 2023-08-09T13:50:10.051 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:10.047+0000 7f245effd640 1 -- 172.21.15.43:0/2614554329 <== mon.0 v1:172.21.15.43:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1285150740 0 0) 0x7f2448001670 con 0x7f24600d0a70 2023-08-09T13:50:10.051 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:10.047+0000 7f245effd640 1 -- 172.21.15.43:0/2614554329 >> v1:172.21.15.43:6789/0 conn(0x7f24600d0a70 legacy=0x7f24600cd7f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:50:10.051 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:10.047+0000 7f245effd640 1 --2- 172.21.15.43:0/2614554329 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f244c003a30 0x7f244c003e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:50:10.051 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:10.047+0000 7f245effd640 1 --2- 172.21.15.43:0/2614554329 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f244c0047c0 0x7f244c004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:50:10.051 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:10.047+0000 7f245effd640 1 --2- 172.21.15.43:0/2614554329 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f244c0051d0 0x7f244c00b980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:50:10.052 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:10.047+0000 7f245effd640 1 -- 172.21.15.43:0/2614554329 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f24600d0f70 con 0x7f244c0047c0 2023-08-09T13:50:10.052 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:10.047+0000 7f245effd640 1 -- 172.21.15.43:0/2614554329 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f244c00c100 con 0x7f244c003a30 2023-08-09T13:50:10.052 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:10.047+0000 7f245effd640 1 -- 172.21.15.43:0/2614554329 --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f244c00c2c0 con 0x7f244c0051d0 2023-08-09T13:50:10.052 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:10.047+0000 7f2457fff640 1 --2- 172.21.15.43:0/2614554329 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f244c0047c0 0x7f244c004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:50:10.052 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:10.047+0000 7f245ffff640 1 --2- 172.21.15.43:0/2614554329 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f244c0051d0 0x7f244c00b980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:50:10.052 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:10.047+0000 7f245f7fe640 1 --2- 172.21.15.43:0/2614554329 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f244c003a30 0x7f244c003e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:50:10.052 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:10.047+0000 7f2457fff640 1 -- 172.21.15.43:0/2614554329 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f244c0051d0 msgr2=0x7f244c00b980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-09T13:50:10.053 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:10.047+0000 7f2457fff640 1 --2- 172.21.15.43:0/2614554329 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f244c0051d0 0x7f244c00b980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:50:10.053 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:10.047+0000 7f2457fff640 1 -- 172.21.15.43:0/2614554329 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f244c003a30 msgr2=0x7f244c003e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-09T13:50:10.053 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:10.047+0000 7f2457fff640 1 --2- 172.21.15.43:0/2614554329 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f244c003a30 0x7f244c003e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:50:10.053 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:10.047+0000 7f245ffff640 1 -- 172.21.15.43:0/2614554329 reap_dead start 2023-08-09T13:50:10.053 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:10.047+0000 7f2457fff640 1 -- 172.21.15.43:0/2614554329 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f244c003520 con 0x7f244c0047c0 2023-08-09T13:50:10.053 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:10.048+0000 7f245f7fe640 1 --2- 172.21.15.43:0/2614554329 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f244c003a30 0x7f244c003e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-08-09T13:50:10.054 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:10.049+0000 7f2457fff640 1 --2- 172.21.15.43:0/2614554329 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f244c0047c0 0x7f244c004c40 secure :-1 s=READY pgs=929 cs=0 l=1 rev1=1 crypto rx=0x7f24600d0ea0 tx=0x7f244c0057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=68072ca23deb7fc7 server_cookie=0 in_seq=0 out_seq=0 2023-08-09T13:50:10.054 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:10.049+0000 7f245effd640 1 -- 172.21.15.43:0/2614554329 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2448005ad0 con 0x7f244c0047c0 2023-08-09T13:50:10.055 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:10.049+0000 7f245effd640 1 -- 172.21.15.43:0/2614554329 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2448005c30 con 0x7f244c0047c0 2023-08-09T13:50:10.055 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:10.049+0000 7f245effd640 1 -- 172.21.15.43:0/2614554329 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f24480072b0 con 0x7f244c0047c0 2023-08-09T13:50:10.055 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:10.050+0000 7f2465b64640 1 -- 172.21.15.43:0/2614554329 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f244c0047c0 msgr2=0x7f244c004c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:50:10.055 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:10.050+0000 7f2465b64640 1 --2- 172.21.15.43:0/2614554329 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f244c0047c0 0x7f244c004c40 secure :-1 s=READY pgs=929 cs=0 l=1 rev1=1 crypto rx=0x7f24600d0ea0 tx=0x7f244c0057f0 comp rx=0 tx=0).stop 2023-08-09T13:50:10.055 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:10.050+0000 7f2465b64640 1 -- 172.21.15.43:0/2614554329 shutdown_connections 2023-08-09T13:50:10.055 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:10.050+0000 7f2465b64640 1 --2- 172.21.15.43:0/2614554329 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f244c0047c0 0x7f244c004c40 unknown :-1 s=CLOSED pgs=929 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:50:10.055 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:10.050+0000 7f2465b64640 1 -- 172.21.15.43:0/2614554329 >> 172.21.15.43:0/2614554329 conn(0x7f2460071520 msgr2=0x7f24600d2520 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-09T13:50:10.055 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:10.050+0000 7f2465b64640 1 -- 172.21.15.43:0/2614554329 shutdown_connections 2023-08-09T13:50:10.056 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:10.050+0000 7f2465b64640 1 -- 172.21.15.43:0/2614554329 wait complete. 2023-08-09T13:50:10.056 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:10.052+0000 7f2465b64640 1 Processor -- start 2023-08-09T13:50:10.056 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:10.052+0000 7f2465b64640 1 -- start start 2023-08-09T13:50:10.057 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:10.053+0000 7f2465b64640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f244c0047c0 0x7f24601f1730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:50:10.057 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:10.053+0000 7f2465b64640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f24600bed60 0x7f246015d8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:50:10.058 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:10.053+0000 7f2465b64640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f24600bf2b0 0x7f246014a4a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:50:10.058 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:10.053+0000 7f2465b64640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f24600ca640 con 0x7f244c0047c0 2023-08-09T13:50:10.058 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:10.053+0000 7f2465b64640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f246015de30 con 0x7f24600bf2b0 2023-08-09T13:50:10.058 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:10.053+0000 7f2465b64640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f24600caea0 con 0x7f24600bed60 2023-08-09T13:50:10.058 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:10.053+0000 7f245f7fe640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f244c0047c0 0x7f24601f1730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:50:10.058 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:10.053+0000 7f245f7fe640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f244c0047c0 0x7f24601f1730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:45390/0 (socket says 172.21.15.43:45390) 2023-08-09T13:50:10.058 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:10.053+0000 7f245f7fe640 1 -- 172.21.15.43:0/1346111422 learned_addr learned my addr 172.21.15.43:0/1346111422 (peer_addr_for_me v2:172.21.15.43:0/0) 2023-08-09T13:50:10.059 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:10.053+0000 7f2457fff640 1 --2- 172.21.15.43:0/1346111422 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f24600bed60 0x7f246015d8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:50:10.059 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:10.054+0000 7f245f7fe640 1 -- 172.21.15.43:0/1346111422 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f24600bed60 msgr2=0x7f246015d8f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:50:10.059 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:10.054+0000 7f245f7fe640 1 --2- 172.21.15.43:0/1346111422 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f24600bed60 0x7f246015d8f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:50:10.059 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:10.054+0000 7f245f7fe640 1 -- 172.21.15.43:0/1346111422 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f24600bf2b0 msgr2=0x7f246014a4a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:50:10.059 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:10.054+0000 7f245ffff640 1 --2- 172.21.15.43:0/1346111422 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f24600bf2b0 0x7f246014a4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:50:10.059 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:10.054+0000 7f245f7fe640 1 --2- 172.21.15.43:0/1346111422 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f24600bf2b0 0x7f246014a4a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:50:10.059 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:10.054+0000 7f245f7fe640 1 -- 172.21.15.43:0/1346111422 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f24601f6070 con 0x7f244c0047c0 2023-08-09T13:50:10.059 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:10.054+0000 7f2457fff640 1 --2- 172.21.15.43:0/1346111422 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f24600bed60 0x7f246015d8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-08-09T13:50:10.060 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:10.054+0000 7f245f7fe640 1 --2- 172.21.15.43:0/1346111422 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f244c0047c0 0x7f24601f1730 secure :-1 s=READY pgs=930 cs=0 l=1 rev1=1 crypto rx=0x7f245000d400 tx=0x7f245000d8c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-09T13:50:10.060 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:10.054+0000 7f245cff9640 1 -- 172.21.15.43:0/1346111422 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f245000b070 con 0x7f244c0047c0 2023-08-09T13:50:10.060 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:10.054+0000 7f245cff9640 1 -- 172.21.15.43:0/1346111422 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f24500054e0 con 0x7f244c0047c0 2023-08-09T13:50:10.060 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:10.054+0000 7f245cff9640 1 -- 172.21.15.43:0/1346111422 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2450003830 con 0x7f244c0047c0 2023-08-09T13:50:10.060 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:10.054+0000 7f2465b64640 1 -- 172.21.15.43:0/1346111422 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f24600ca850 con 0x7f244c0047c0 2023-08-09T13:50:10.060 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:10.054+0000 7f2465b64640 1 -- 172.21.15.43:0/1346111422 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f24601f5d00 con 0x7f244c0047c0 2023-08-09T13:50:10.061 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:10.055+0000 7f2465b64640 1 -- 172.21.15.43:0/1346111422 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f24600739f0 con 0x7f244c0047c0 2023-08-09T13:50:10.061 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:10.057+0000 7f245cff9640 1 -- 172.21.15.43:0/1346111422 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 27) v1 ==== 88179+0+0 (secure 0 0 0) 0x7f2450003990 con 0x7f244c0047c0 2023-08-09T13:50:10.061 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:10.058+0000 7f245cff9640 1 --2- 172.21.15.43:0/1346111422 >> [v2:172.21.15.203:6832/3325141227,v1:172.21.15.203:6833/3325141227] conn(0x7f2440069ea0 0x7f244006c360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:50:10.061 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:10.058+0000 7f245cff9640 1 -- 172.21.15.43:0/1346111422 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(77..77 src has 1..77) v4 ==== 7272+0+0 (secure 0 0 0) 0x7f2450087e40 con 0x7f244c0047c0 2023-08-09T13:50:10.062 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:10.059+0000 7f2457fff640 1 --2- 172.21.15.43:0/1346111422 >> [v2:172.21.15.203:6832/3325141227,v1:172.21.15.203:6833/3325141227] conn(0x7f2440069ea0 0x7f244006c360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:50:10.062 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:10.060+0000 7f2457fff640 1 --2- 172.21.15.43:0/1346111422 >> [v2:172.21.15.203:6832/3325141227,v1:172.21.15.203:6833/3325141227] conn(0x7f2440069ea0 0x7f244006c360 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7f2448034000 tx=0x7f2448009030 comp rx=0 tx=0).ready entity=mgr.5519 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-09T13:50:10.065 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:10.063+0000 7f245cff9640 1 -- 172.21.15.43:0/1346111422 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178066 (secure 0 0 0) 0x7f2450056e20 con 0x7f244c0047c0 2023-08-09T13:50:10.211 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:10.208+0000 7f245cff9640 1 -- 172.21.15.43:0/1346111422 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mgrmap(e 28) v1 ==== 88387+0+0 (secure 0 0 0) 0x7f2450053760 con 0x7f244c0047c0 2023-08-09T13:50:10.275 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:10.272+0000 7f2465b64640 1 -- 172.21.15.43:0/1346111422 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f24600cd2a0 con 0x7f244c0047c0 2023-08-09T13:50:10.276 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:10.274+0000 7f245cff9640 1 -- 172.21.15.43:0/1346111422 <== mon.0 v2:172.21.15.43:3300/0 8 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v77) v1 ==== 104+0+11 (secure 0 0 0) 0x7f2450056b90 con 0x7f244c0047c0 2023-08-09T13:50:10.276 INFO:teuthology.orchestra.run.smithi043.stdout:pg_num: 32 2023-08-09T13:50:10.278 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:10.276+0000 7f2465b64640 1 -- 172.21.15.43:0/1346111422 >> [v2:172.21.15.203:6832/3325141227,v1:172.21.15.203:6833/3325141227] conn(0x7f2440069ea0 msgr2=0x7f244006c360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:50:10.278 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:10.276+0000 7f2465b64640 1 --2- 172.21.15.43:0/1346111422 >> [v2:172.21.15.203:6832/3325141227,v1:172.21.15.203:6833/3325141227] conn(0x7f2440069ea0 0x7f244006c360 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7f2448034000 tx=0x7f2448009030 comp rx=0 tx=0).stop 2023-08-09T13:50:10.279 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:10.276+0000 7f2465b64640 1 -- 172.21.15.43:0/1346111422 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f244c0047c0 msgr2=0x7f24601f1730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:50:10.279 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:10.276+0000 7f2465b64640 1 --2- 172.21.15.43:0/1346111422 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f244c0047c0 0x7f24601f1730 secure :-1 s=READY pgs=930 cs=0 l=1 rev1=1 crypto rx=0x7f245000d400 tx=0x7f245000d8c0 comp rx=0 tx=0).stop 2023-08-09T13:50:10.279 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:10.276+0000 7f2465b64640 1 -- 172.21.15.43:0/1346111422 shutdown_connections 2023-08-09T13:50:10.279 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:10.276+0000 7f2465b64640 1 --2- 172.21.15.43:0/1346111422 >> [v2:172.21.15.203:6832/3325141227,v1:172.21.15.203:6833/3325141227] conn(0x7f2440069ea0 0x7f244006c360 unknown :-1 s=CLOSED pgs=23 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:50:10.280 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:10.276+0000 7f2465b64640 1 --2- 172.21.15.43:0/1346111422 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f24600bf2b0 0x7f246014a4a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:50:10.280 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:10.276+0000 7f2465b64640 1 --2- 172.21.15.43:0/1346111422 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f24600bed60 0x7f246015d8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:50:10.280 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:10.276+0000 7f2465b64640 1 --2- 172.21.15.43:0/1346111422 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f244c0047c0 0x7f24601f1730 unknown :-1 s=CLOSED pgs=930 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:50:10.280 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:10.276+0000 7f2465b64640 1 -- 172.21.15.43:0/1346111422 >> 172.21.15.43:0/1346111422 conn(0x7f2460071520 msgr2=0x7f246015b2e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-09T13:50:10.280 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:10.276+0000 7f2465b64640 1 -- 172.21.15.43:0/1346111422 shutdown_connections 2023-08-09T13:50:10.280 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:10.276+0000 7f2465b64640 1 -- 172.21.15.43:0/1346111422 wait complete. 2023-08-09T13:50:10.300 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-08-09T13:50:10.532 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:10.530+0000 7f1c33fff640 1 Processor -- start 2023-08-09T13:50:10.532 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:10.530+0000 7f1c33fff640 1 -- start start 2023-08-09T13:50:10.534 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:10.531+0000 7f1c33fff640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f1c34159400 0x7f1c340cb9a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:50:10.534 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:10.531+0000 7f1c33fff640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f1c340c8900 0x7f1c3415a270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:50:10.534 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:10.532+0000 7f1c33fff640 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7f1c340c8e50 0x7f1c340c92e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:50:10.534 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:10.532+0000 7f1c33fff640 1 -- --> v2:172.21.15.43:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1c340c9820 con 0x7f1c340c8900 2023-08-09T13:50:10.535 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:10.532+0000 7f1c33fff640 1 -- --> v2:172.21.15.203:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1c3415a7b0 con 0x7f1c340c8e50 2023-08-09T13:50:10.535 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:10.532+0000 7f1c33fff640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1c34070250 con 0x7f1c34159400 2023-08-09T13:50:10.535 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:10.532+0000 7f1c2bfff640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f1c340c8900 0x7f1c3415a270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:50:10.536 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:10.532+0000 7f1c2bfff640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f1c340c8900 0x7f1c3415a270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:45392/0 (socket says 172.21.15.43:45392) 2023-08-09T13:50:10.536 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:10.532+0000 7f1c2bfff640 1 -- 172.21.15.43:0/3375466788 learned_addr learned my addr 172.21.15.43:0/3375466788 (peer_addr_for_me v2:172.21.15.43:0/0) 2023-08-09T13:50:10.536 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:10.532+0000 7f1c337fe640 1 --2- 172.21.15.43:0/3375466788 >> v2:172.21.15.203:3300/0 conn(0x7f1c340c8e50 0x7f1c340c92e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:50:10.536 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:10.532+0000 7f1c32ffd640 1 --2- 172.21.15.43:0/3375466788 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f1c34159400 0x7f1c340cb9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:50:10.536 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:10.533+0000 7f1c2bfff640 1 -- 172.21.15.43:0/3375466788 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f1c34159400 msgr2=0x7f1c340cb9a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-09T13:50:10.536 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:10.533+0000 7f1c2bfff640 1 --2- 172.21.15.43:0/3375466788 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f1c34159400 0x7f1c340cb9a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:50:10.536 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:10.533+0000 7f1c2bfff640 1 -- 172.21.15.43:0/3375466788 >> v2:172.21.15.203:3300/0 conn(0x7f1c340c8e50 msgr2=0x7f1c340c92e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-09T13:50:10.537 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:10.533+0000 7f1c2bfff640 1 --2- 172.21.15.43:0/3375466788 >> v2:172.21.15.203:3300/0 conn(0x7f1c340c8e50 0x7f1c340c92e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:50:10.537 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:10.533+0000 7f1c2bfff640 1 -- 172.21.15.43:0/3375466788 --> v2:172.21.15.43:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1c340c86c0 con 0x7f1c340c8900 2023-08-09T13:50:10.538 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:10.534+0000 7f1c2bfff640 1 --2- 172.21.15.43:0/3375466788 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f1c340c8900 0x7f1c3415a270 secure :-1 s=READY pgs=931 cs=0 l=1 rev1=1 crypto rx=0x7f1c3406e730 tx=0x7f1c1c02b880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ace8e0eba66c4520 server_cookie=0 in_seq=0 out_seq=0 2023-08-09T13:50:10.538 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:10.534+0000 7f1c327fc640 1 -- 172.21.15.43:0/3375466788 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1c1c0317e0 con 0x7f1c340c8900 2023-08-09T13:50:10.538 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:10.534+0000 7f1c327fc640 1 -- 172.21.15.43:0/3375466788 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1c1c031940 con 0x7f1c340c8900 2023-08-09T13:50:10.538 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:10.534+0000 7f1c327fc640 1 -- 172.21.15.43:0/3375466788 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1c1c031cd0 con 0x7f1c340c8900 2023-08-09T13:50:10.538 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:10.535+0000 7f1c33fff640 1 -- 172.21.15.43:0/3375466788 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f1c340c8900 msgr2=0x7f1c3415a270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:50:10.538 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:10.535+0000 7f1c33fff640 1 --2- 172.21.15.43:0/3375466788 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f1c340c8900 0x7f1c3415a270 secure :-1 s=READY pgs=931 cs=0 l=1 rev1=1 crypto rx=0x7f1c3406e730 tx=0x7f1c1c02b880 comp rx=0 tx=0).stop 2023-08-09T13:50:10.539 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:10.535+0000 7f1c33fff640 1 -- 172.21.15.43:0/3375466788 shutdown_connections 2023-08-09T13:50:10.539 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:10.535+0000 7f1c33fff640 1 --2- 172.21.15.43:0/3375466788 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f1c340c8900 0x7f1c3415a270 unknown :-1 s=CLOSED pgs=931 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:50:10.539 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:10.535+0000 7f1c33fff640 1 --2- 172.21.15.43:0/3375466788 >> v2:172.21.15.203:3300/0 conn(0x7f1c340c8e50 0x7f1c340c92e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:50:10.539 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:10.535+0000 7f1c33fff640 1 --2- 172.21.15.43:0/3375466788 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f1c34159400 0x7f1c340cb9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:50:10.539 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:10.535+0000 7f1c33fff640 1 -- 172.21.15.43:0/3375466788 >> 172.21.15.43:0/3375466788 conn(0x7f1c340cddd0 msgr2=0x7f1c340ce1e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-09T13:50:10.539 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:10.535+0000 7f1c33fff640 1 -- 172.21.15.43:0/3375466788 shutdown_connections 2023-08-09T13:50:10.539 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:10.535+0000 7f1c33fff640 1 -- 172.21.15.43:0/3375466788 wait complete. 2023-08-09T13:50:10.540 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:10.537+0000 7f1c33fff640 1 Processor -- start 2023-08-09T13:50:10.540 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:10.537+0000 7f1c33fff640 1 -- start start 2023-08-09T13:50:10.541 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:10.538+0000 7f1c33fff640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f1c340c8900 0x7f1c341f6210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:50:10.541 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:10.538+0000 7f1c33fff640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f1c340c8e50 0x7f1c340b3380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:50:10.541 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:10.538+0000 7f1c33fff640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f1c3414c590 0x7f1c3406b3e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:50:10.541 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:10.538+0000 7f1c33fff640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1c340c3f90 con 0x7f1c340c8900 2023-08-09T13:50:10.541 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:10.538+0000 7f1c33fff640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1c341f6750 con 0x7f1c340c8e50 2023-08-09T13:50:10.542 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:10.538+0000 7f1c33fff640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1c340c43b0 con 0x7f1c3414c590 2023-08-09T13:50:10.542 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:10.538+0000 7f1c32ffd640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f1c340c8900 0x7f1c341f6210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:50:10.542 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:10.538+0000 7f1c32ffd640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f1c340c8900 0x7f1c341f6210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:45408/0 (socket says 172.21.15.43:45408) 2023-08-09T13:50:10.542 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:10.538+0000 7f1c32ffd640 1 -- 172.21.15.43:0/3470627055 learned_addr learned my addr 172.21.15.43:0/3470627055 (peer_addr_for_me v2:172.21.15.43:0/0) 2023-08-09T13:50:10.543 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:10.539+0000 7f1c32ffd640 1 -- 172.21.15.43:0/3470627055 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f1c3414c590 msgr2=0x7f1c3406b3e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:50:10.543 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:10.539+0000 7f1c32ffd640 1 --2- 172.21.15.43:0/3470627055 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f1c3414c590 0x7f1c3406b3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:50:10.543 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:10.539+0000 7f1c32ffd640 1 -- 172.21.15.43:0/3470627055 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f1c340c8e50 msgr2=0x7f1c340b3380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:50:10.543 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:10.539+0000 7f1c32ffd640 1 --2- 172.21.15.43:0/3470627055 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f1c340c8e50 0x7f1c340b3380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:50:10.543 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:10.539+0000 7f1c32ffd640 1 -- 172.21.15.43:0/3470627055 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1c1c0094d0 con 0x7f1c340c8900 2023-08-09T13:50:10.544 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:10.539+0000 7f1c32ffd640 1 --2- 172.21.15.43:0/3470627055 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f1c340c8900 0x7f1c341f6210 secure :-1 s=READY pgs=932 cs=0 l=1 rev1=1 crypto rx=0x7f1c2400d5f0 tx=0x7f1c2400dab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-09T13:50:10.544 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:10.539+0000 7f1c2b7fe640 1 -- 172.21.15.43:0/3470627055 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1c240125f0 con 0x7f1c340c8900 2023-08-09T13:50:10.544 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:10.540+0000 7f1c2b7fe640 1 -- 172.21.15.43:0/3470627055 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1c24012750 con 0x7f1c340c8900 2023-08-09T13:50:10.544 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:10.540+0000 7f1c2b7fe640 1 -- 172.21.15.43:0/3470627055 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1c24012b00 con 0x7f1c340c8900 2023-08-09T13:50:10.544 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:10.540+0000 7f1c33fff640 1 -- 172.21.15.43:0/3470627055 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1c340c3310 con 0x7f1c340c8900 2023-08-09T13:50:10.544 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:10.540+0000 7f1c33fff640 1 -- 172.21.15.43:0/3470627055 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1c340c38d0 con 0x7f1c340c8900 2023-08-09T13:50:10.545 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:10.541+0000 7f1c33fff640 1 -- 172.21.15.43:0/3470627055 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1c340c4200 con 0x7f1c340c8900 2023-08-09T13:50:10.545 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:10.542+0000 7f1c2b7fe640 1 -- 172.21.15.43:0/3470627055 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 28) v1 ==== 88387+0+0 (secure 0 0 0) 0x7f1c24012c60 con 0x7f1c340c8900 2023-08-09T13:50:10.545 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:10.543+0000 7f1c2b7fe640 1 --2- 172.21.15.43:0/3470627055 >> [v2:172.21.15.203:6832/3325141227,v1:172.21.15.203:6833/3325141227] conn(0x7f1c1406a0e0 0x7f1c1406c5a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:50:10.546 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:10.543+0000 7f1c2b7fe640 1 -- 172.21.15.43:0/3470627055 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(77..77 src has 1..77) v4 ==== 7272+0+0 (secure 0 0 0) 0x7f1c24088ff0 con 0x7f1c340c8900 2023-08-09T13:50:10.546 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:10.544+0000 7f1c2bfff640 1 --2- 172.21.15.43:0/3470627055 >> [v2:172.21.15.203:6832/3325141227,v1:172.21.15.203:6833/3325141227] conn(0x7f1c1406a0e0 0x7f1c1406c5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:50:10.547 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:10.545+0000 7f1c2bfff640 1 --2- 172.21.15.43:0/3470627055 >> [v2:172.21.15.203:6832/3325141227,v1:172.21.15.203:6833/3325141227] conn(0x7f1c1406a0e0 0x7f1c1406c5a0 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7f1c1c036000 tx=0x7f1c1c030000 comp rx=0 tx=0).ready entity=mgr.5519 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-09T13:50:10.550 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:10.548+0000 7f1c2b7fe640 1 -- 172.21.15.43:0/3470627055 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178066 (secure 0 0 0) 0x7f1c240bb7b0 con 0x7f1c340c8900 2023-08-09T13:50:10.742 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:10.739+0000 7f1c33fff640 1 -- 172.21.15.43:0/3470627055 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f1c3404e390 con 0x7f1c340c8900 2023-08-09T13:50:10.743 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:10.740+0000 7f1c2b7fe640 1 -- 172.21.15.43:0/3470627055 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v77) v1 ==== 100+0+11 (secure 0 0 0) 0x7f1c24054170 con 0x7f1c340c8900 2023-08-09T13:50:10.743 INFO:teuthology.orchestra.run.smithi043.stdout:pg_num: 64 2023-08-09T13:50:10.745 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:10.743+0000 7f1c33fff640 1 -- 172.21.15.43:0/3470627055 >> [v2:172.21.15.203:6832/3325141227,v1:172.21.15.203:6833/3325141227] conn(0x7f1c1406a0e0 msgr2=0x7f1c1406c5a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:50:10.746 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:10.743+0000 7f1c33fff640 1 --2- 172.21.15.43:0/3470627055 >> [v2:172.21.15.203:6832/3325141227,v1:172.21.15.203:6833/3325141227] conn(0x7f1c1406a0e0 0x7f1c1406c5a0 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7f1c1c036000 tx=0x7f1c1c030000 comp rx=0 tx=0).stop 2023-08-09T13:50:10.746 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:10.743+0000 7f1c33fff640 1 -- 172.21.15.43:0/3470627055 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f1c340c8900 msgr2=0x7f1c341f6210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:50:10.746 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:10.743+0000 7f1c33fff640 1 --2- 172.21.15.43:0/3470627055 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f1c340c8900 0x7f1c341f6210 secure :-1 s=READY pgs=932 cs=0 l=1 rev1=1 crypto rx=0x7f1c2400d5f0 tx=0x7f1c2400dab0 comp rx=0 tx=0).stop 2023-08-09T13:50:10.746 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:10.743+0000 7f1c33fff640 1 -- 172.21.15.43:0/3470627055 shutdown_connections 2023-08-09T13:50:10.746 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:10.743+0000 7f1c33fff640 1 --2- 172.21.15.43:0/3470627055 >> [v2:172.21.15.203:6832/3325141227,v1:172.21.15.203:6833/3325141227] conn(0x7f1c1406a0e0 0x7f1c1406c5a0 unknown :-1 s=CLOSED pgs=25 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:50:10.746 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:10.743+0000 7f1c33fff640 1 --2- 172.21.15.43:0/3470627055 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f1c3414c590 0x7f1c3406b3e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:50:10.746 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:10.743+0000 7f1c33fff640 1 --2- 172.21.15.43:0/3470627055 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f1c340c8e50 0x7f1c340b3380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:50:10.747 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:10.743+0000 7f1c33fff640 1 --2- 172.21.15.43:0/3470627055 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f1c340c8900 0x7f1c341f6210 unknown :-1 s=CLOSED pgs=932 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:50:10.747 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:10.743+0000 7f1c33fff640 1 -- 172.21.15.43:0/3470627055 >> 172.21.15.43:0/3470627055 conn(0x7f1c340cddd0 msgr2=0x7f1c340adff0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-09T13:50:10.747 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:10.743+0000 7f1c33fff640 1 -- 172.21.15.43:0/3470627055 shutdown_connections 2023-08-09T13:50:10.747 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:10.744+0000 7f1c33fff640 1 -- 172.21.15.43:0/3470627055 wait complete. 2023-08-09T13:50:10.765 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-08-09T13:50:10.992 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:10.989+0000 7f4369955640 1 Processor -- start 2023-08-09T13:50:10.992 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:10.989+0000 7f4369955640 1 -- start start 2023-08-09T13:50:10.994 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:10.991+0000 7f4369955640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f43640713b0 0x7f43640c8fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:50:10.994 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:10.991+0000 7f4369955640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f43640c94f0 0x7f436415ad20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:50:10.994 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:10.991+0000 7f4369955640 1 -- --> v1:172.21.15.203:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f43640c9940 con 0x7f436415a6a0 2023-08-09T13:50:10.994 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:10.991+0000 7f4369955640 1 -- --> v2:172.21.15.43:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f43640cdc70 con 0x7f43640713b0 2023-08-09T13:50:10.995 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:10.991+0000 7f4369955640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4364070910 con 0x7f43640c94f0 2023-08-09T13:50:10.995 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:10.992+0000 7f4363fff640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f43640713b0 0x7f43640c8fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:50:10.995 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:10.992+0000 7f4363fff640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f43640713b0 0x7f43640c8fb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:45412/0 (socket says 172.21.15.43:45412) 2023-08-09T13:50:10.995 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:10.992+0000 7f4363fff640 1 -- 172.21.15.43:0/2615926959 learned_addr learned my addr 172.21.15.43:0/2615926959 (peer_addr_for_me v2:172.21.15.43:0/0) 2023-08-09T13:50:10.996 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:10.992+0000 7f4369154640 1 --2- 172.21.15.43:0/2615926959 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f43640c94f0 0x7f436415ad20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:50:10.996 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:10.992+0000 7f4363fff640 1 -- 172.21.15.43:0/2615926959 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f43640c94f0 msgr2=0x7f436415ad20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-09T13:50:10.996 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:10.992+0000 7f4363fff640 1 --2- 172.21.15.43:0/2615926959 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f43640c94f0 0x7f436415ad20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:50:10.996 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:10.992+0000 7f4363fff640 1 -- 172.21.15.43:0/2615926959 >> v1:172.21.15.203:6789/0 conn(0x7f436415a6a0 legacy=0x7f43640cbd90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:50:10.996 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:10.992+0000 7f4363fff640 1 -- 172.21.15.43:0/2615926959 --> v2:172.21.15.43:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f43640c88c0 con 0x7f43640713b0 2023-08-09T13:50:10.997 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:10.993+0000 7f4363fff640 1 --2- 172.21.15.43:0/2615926959 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f43640713b0 0x7f43640c8fb0 secure :-1 s=READY pgs=933 cs=0 l=1 rev1=1 crypto rx=0x7f4350009870 tx=0x7f435002b8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7e89d34c1af903a8 server_cookie=0 in_seq=0 out_seq=0 2023-08-09T13:50:10.997 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:10.994+0000 7f43637fe640 1 -- 172.21.15.43:0/2615926959 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4350031640 con 0x7f43640713b0 2023-08-09T13:50:10.997 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:10.994+0000 7f43637fe640 1 -- 172.21.15.43:0/2615926959 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f43500317a0 con 0x7f43640713b0 2023-08-09T13:50:10.997 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:10.994+0000 7f43637fe640 1 -- 172.21.15.43:0/2615926959 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4350031b30 con 0x7f43640713b0 2023-08-09T13:50:10.998 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:10.994+0000 7f4369955640 1 -- 172.21.15.43:0/2615926959 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f43640713b0 msgr2=0x7f43640c8fb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:50:10.998 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:10.994+0000 7f4369955640 1 --2- 172.21.15.43:0/2615926959 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f43640713b0 0x7f43640c8fb0 secure :-1 s=READY pgs=933 cs=0 l=1 rev1=1 crypto rx=0x7f4350009870 tx=0x7f435002b8f0 comp rx=0 tx=0).stop 2023-08-09T13:50:10.998 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:10.994+0000 7f4369955640 1 -- 172.21.15.43:0/2615926959 shutdown_connections 2023-08-09T13:50:10.998 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:10.994+0000 7f4369955640 1 --2- 172.21.15.43:0/2615926959 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f43640c94f0 0x7f436415ad20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:50:10.998 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:10.994+0000 7f4369955640 1 --2- 172.21.15.43:0/2615926959 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f43640713b0 0x7f43640c8fb0 unknown :-1 s=CLOSED pgs=933 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:50:10.998 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:10.995+0000 7f4369955640 1 -- 172.21.15.43:0/2615926959 >> 172.21.15.43:0/2615926959 conn(0x7f43640cde90 msgr2=0x7f43640ce2a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-09T13:50:10.998 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:10.995+0000 7f4369955640 1 -- 172.21.15.43:0/2615926959 shutdown_connections 2023-08-09T13:50:10.999 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:10.995+0000 7f4369955640 1 -- 172.21.15.43:0/2615926959 wait complete. 2023-08-09T13:50:10.999 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:10.996+0000 7f4369955640 1 Processor -- start 2023-08-09T13:50:10.999 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:10.996+0000 7f4369955640 1 -- start start 2023-08-09T13:50:11.000 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:10.997+0000 7f4369955640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f43640713b0 0x7f43640c4710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:50:11.000 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:10.997+0000 7f4369955640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f43640c94f0 0x7f43640abfb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:50:11.000 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:10.997+0000 7f4369955640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f436415a6a0 0x7f436406c1e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:50:11.001 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:10.997+0000 7f4369955640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f43640c5700 con 0x7f43640c94f0 2023-08-09T13:50:11.001 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:10.997+0000 7f4369955640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4364098ec0 con 0x7f436415a6a0 2023-08-09T13:50:11.001 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:10.997+0000 7f4369955640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f43640991a0 con 0x7f43640713b0 2023-08-09T13:50:11.001 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:10.998+0000 7f4363fff640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f43640c94f0 0x7f43640abfb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:50:11.001 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:10.998+0000 7f4363fff640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f43640c94f0 0x7f43640abfb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:45422/0 (socket says 172.21.15.43:45422) 2023-08-09T13:50:11.001 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:10.998+0000 7f4363fff640 1 -- 172.21.15.43:0/868938928 learned_addr learned my addr 172.21.15.43:0/868938928 (peer_addr_for_me v2:172.21.15.43:0/0) 2023-08-09T13:50:11.002 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:10.998+0000 7f4368953640 1 --2- 172.21.15.43:0/868938928 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f43640713b0 0x7f43640c4710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:50:11.002 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:10.998+0000 7f4369154640 1 --2- 172.21.15.43:0/868938928 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f436415a6a0 0x7f436406c1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:50:11.002 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:10.998+0000 7f4363fff640 1 -- 172.21.15.43:0/868938928 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f43640713b0 msgr2=0x7f43640c4710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:50:11.003 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:10.998+0000 7f4363fff640 1 --2- 172.21.15.43:0/868938928 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f43640713b0 0x7f43640c4710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:50:11.003 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:10.998+0000 7f4363fff640 1 -- 172.21.15.43:0/868938928 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f436415a6a0 msgr2=0x7f436406c1e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:50:11.003 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:10.998+0000 7f4363fff640 1 --2- 172.21.15.43:0/868938928 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f436415a6a0 0x7f436406c1e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:50:11.003 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:10.998+0000 7f4363fff640 1 -- 172.21.15.43:0/868938928 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f43500094d0 con 0x7f43640c94f0 2023-08-09T13:50:11.003 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:10.998+0000 7f4369154640 1 --2- 172.21.15.43:0/868938928 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f436415a6a0 0x7f436406c1e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-08-09T13:50:11.003 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:10.999+0000 7f4363fff640 1 --2- 172.21.15.43:0/868938928 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f43640c94f0 0x7f43640abfb0 secure :-1 s=READY pgs=934 cs=0 l=1 rev1=1 crypto rx=0x7f4350000c00 tx=0x7f43500046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-09T13:50:11.003 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:10.999+0000 7f43617fa640 1 -- 172.21.15.43:0/868938928 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f435003ac60 con 0x7f43640c94f0 2023-08-09T13:50:11.004 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:10.999+0000 7f43617fa640 1 -- 172.21.15.43:0/868938928 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f435003adc0 con 0x7f43640c94f0 2023-08-09T13:50:11.004 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:10.999+0000 7f43617fa640 1 -- 172.21.15.43:0/868938928 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4350044630 con 0x7f43640c94f0 2023-08-09T13:50:11.004 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:10.999+0000 7f4369955640 1 -- 172.21.15.43:0/868938928 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4364099480 con 0x7f43640c94f0 2023-08-09T13:50:11.004 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:10.999+0000 7f4369955640 1 -- 172.21.15.43:0/868938928 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4364099a40 con 0x7f43640c94f0 2023-08-09T13:50:11.005 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:11.000+0000 7f4369955640 1 -- 172.21.15.43:0/868938928 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4364099c30 con 0x7f43640c94f0 2023-08-09T13:50:11.005 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:11.001+0000 7f43617fa640 1 -- 172.21.15.43:0/868938928 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 28) v1 ==== 88387+0+0 (secure 0 0 0) 0x7f435002f050 con 0x7f43640c94f0 2023-08-09T13:50:11.005 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:11.002+0000 7f43617fa640 1 --2- 172.21.15.43:0/868938928 >> [v2:172.21.15.203:6832/3325141227,v1:172.21.15.203:6833/3325141227] conn(0x7f434806a0e0 0x7f434806c5a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:50:11.005 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:11.002+0000 7f43617fa640 1 -- 172.21.15.43:0/868938928 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(77..77 src has 1..77) v4 ==== 7272+0+0 (secure 0 0 0) 0x7f43500aaaf0 con 0x7f43640c94f0 2023-08-09T13:50:11.005 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:11.003+0000 7f4368953640 1 --2- 172.21.15.43:0/868938928 >> [v2:172.21.15.203:6832/3325141227,v1:172.21.15.203:6833/3325141227] conn(0x7f434806a0e0 0x7f434806c5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:50:11.006 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:11.004+0000 7f4368953640 1 --2- 172.21.15.43:0/868938928 >> [v2:172.21.15.203:6832/3325141227,v1:172.21.15.203:6833/3325141227] conn(0x7f434806a0e0 0x7f434806c5a0 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7f4358000f60 tx=0x7f4358009040 comp rx=0 tx=0).ready entity=mgr.5519 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-09T13:50:11.009 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:11.006+0000 7f43617fa640 1 -- 172.21.15.43:0/868938928 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178066 (secure 0 0 0) 0x7f4350075b20 con 0x7f43640c94f0 2023-08-09T13:50:11.244 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:11.242+0000 7f4369955640 1 -- 172.21.15.43:0/868938928 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f43640739f0 con 0x7f43640c94f0 2023-08-09T13:50:11.245 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:11.243+0000 7f43617fa640 1 -- 172.21.15.43:0/868938928 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 26 v26) v1 ==== 94+0+4881 (secure 0 0 0) 0x7f4350031e60 con 0x7f43640c94f0 2023-08-09T13:50:11.245 INFO:teuthology.orchestra.run.smithi043.stdout: 2023-08-09T13:50:11.246 INFO:teuthology.orchestra.run.smithi043.stdout:{"epoch":26,"default_fscid":3,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5369,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.203:6837/1426673901","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6836","nonce":1426673901},{"type":"v1","addr":"172.21.15.203:6837","nonce":1426673901}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":21},{"gid":5728,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.43:6833/1096124235","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6832","nonce":1096124235},{"type":"v1","addr":"172.21.15.43:6833","nonce":1096124235}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":20},{"gid":5731,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.43:6835/1688804357","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6834","nonce":1688804357},{"type":"v1","addr":"172.21.15.43:6835","nonce":1688804357}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":21}],"filesystems":[{"mdsmap":{"epoch":26,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-08-09T13:48:17.607195+0000","modified":"2023-08-09T13:48:21.179296+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":5734},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5734":{"gid":5734,"name":"b","rank":0,"incarnation":22,"state":"up:active","state_seq":3,"addr":"172.21.15.203:6835/3324252767","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6834","nonce":3324252767},{"type":"v1","addr":"172.21.15.203:6835","nonce":3324252767}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[9],"metadata_pool":8,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":3}]} 2023-08-09T13:50:11.246 INFO:teuthology.orchestra.run.smithi043.stderr:dumped fsmap epoch 26 2023-08-09T13:50:11.248 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:11.245+0000 7f4369955640 1 -- 172.21.15.43:0/868938928 >> [v2:172.21.15.203:6832/3325141227,v1:172.21.15.203:6833/3325141227] conn(0x7f434806a0e0 msgr2=0x7f434806c5a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:50:11.248 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:11.245+0000 7f4369955640 1 --2- 172.21.15.43:0/868938928 >> [v2:172.21.15.203:6832/3325141227,v1:172.21.15.203:6833/3325141227] conn(0x7f434806a0e0 0x7f434806c5a0 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7f4358000f60 tx=0x7f4358009040 comp rx=0 tx=0).stop 2023-08-09T13:50:11.248 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:11.245+0000 7f4369955640 1 -- 172.21.15.43:0/868938928 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f43640c94f0 msgr2=0x7f43640abfb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:50:11.249 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:11.245+0000 7f4369955640 1 --2- 172.21.15.43:0/868938928 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f43640c94f0 0x7f43640abfb0 secure :-1 s=READY pgs=934 cs=0 l=1 rev1=1 crypto rx=0x7f4350000c00 tx=0x7f43500046c0 comp rx=0 tx=0).stop 2023-08-09T13:50:11.249 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:11.246+0000 7f4369955640 1 -- 172.21.15.43:0/868938928 shutdown_connections 2023-08-09T13:50:11.249 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:11.246+0000 7f4369955640 1 --2- 172.21.15.43:0/868938928 >> [v2:172.21.15.203:6832/3325141227,v1:172.21.15.203:6833/3325141227] conn(0x7f434806a0e0 0x7f434806c5a0 unknown :-1 s=CLOSED pgs=26 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:50:11.249 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:11.246+0000 7f4369955640 1 --2- 172.21.15.43:0/868938928 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f436415a6a0 0x7f436406c1e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:50:11.249 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:11.246+0000 7f4369955640 1 --2- 172.21.15.43:0/868938928 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f43640c94f0 0x7f43640abfb0 unknown :-1 s=CLOSED pgs=934 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:50:11.249 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:11.246+0000 7f4369955640 1 --2- 172.21.15.43:0/868938928 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f43640713b0 0x7f43640c4710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:50:11.249 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:11.246+0000 7f4369955640 1 -- 172.21.15.43:0/868938928 >> 172.21.15.43:0/868938928 conn(0x7f43640cde90 msgr2=0x7f43640ae370 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-09T13:50:11.249 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:11.246+0000 7f4369955640 1 -- 172.21.15.43:0/868938928 shutdown_connections 2023-08-09T13:50:11.250 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:11.246+0000 7f4369955640 1 -- 172.21.15.43:0/868938928 wait complete. 2023-08-09T13:50:11.267 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-08-09T13:50:11.502 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:11.500+0000 7f8cfed42640 1 Processor -- start 2023-08-09T13:50:11.503 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:11.500+0000 7f8cfed42640 1 -- start start 2023-08-09T13:50:11.504 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:11.502+0000 7f8cfed42640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f8cf814a5c0 0x7f8cf8160f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:50:11.504 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:11.502+0000 7f8cfed42640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f8cf80c3130 0x7f8cf80cbc70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:50:11.504 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:11.502+0000 7f8cfed42640 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7f8cf80cc1b0 0x7f8cf8164510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:50:11.505 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:11.502+0000 7f8cfed42640 1 -- --> v2:172.21.15.43:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8cf80c4420 con 0x7f8cf80c3130 2023-08-09T13:50:11.505 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:11.502+0000 7f8cfed42640 1 -- --> v2:172.21.15.203:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8cf80cc640 con 0x7f8cf80cc1b0 2023-08-09T13:50:11.505 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:11.502+0000 7f8cfed42640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8cf80c38a0 con 0x7f8cf814a5c0 2023-08-09T13:50:11.506 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:11.502+0000 7f8ceffff640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f8cf80c3130 0x7f8cf80cbc70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:50:11.506 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:11.502+0000 7f8ceffff640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f8cf80c3130 0x7f8cf80cbc70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:45434/0 (socket says 172.21.15.43:45434) 2023-08-09T13:50:11.506 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:11.502+0000 7f8ceffff640 1 -- 172.21.15.43:0/4164964576 learned_addr learned my addr 172.21.15.43:0/4164964576 (peer_addr_for_me v2:172.21.15.43:0/0) 2023-08-09T13:50:11.506 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:11.502+0000 7f8cfcade640 1 --2- 172.21.15.43:0/4164964576 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f8cf814a5c0 0x7f8cf8160f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:50:11.506 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:11.503+0000 7f8cfd2df640 1 --2- 172.21.15.43:0/4164964576 >> v2:172.21.15.203:3300/0 conn(0x7f8cf80cc1b0 0x7f8cf8164510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:50:11.506 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:11.503+0000 7f8ceffff640 1 -- 172.21.15.43:0/4164964576 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f8cf814a5c0 msgr2=0x7f8cf8160f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-09T13:50:11.507 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:11.503+0000 7f8ceffff640 1 --2- 172.21.15.43:0/4164964576 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f8cf814a5c0 0x7f8cf8160f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:50:11.507 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:11.503+0000 7f8ceffff640 1 -- 172.21.15.43:0/4164964576 >> v2:172.21.15.203:3300/0 conn(0x7f8cf80cc1b0 msgr2=0x7f8cf8164510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-09T13:50:11.507 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:11.503+0000 7f8ceffff640 1 --2- 172.21.15.43:0/4164964576 >> v2:172.21.15.203:3300/0 conn(0x7f8cf80cc1b0 0x7f8cf8164510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:50:11.507 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:11.503+0000 7f8ceffff640 1 -- 172.21.15.43:0/4164964576 --> v2:172.21.15.43:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8cf80c3b80 con 0x7f8cf80c3130 2023-08-09T13:50:11.508 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:11.504+0000 7f8ceffff640 1 --2- 172.21.15.43:0/4164964576 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f8cf80c3130 0x7f8cf80cbc70 secure :-1 s=READY pgs=935 cs=0 l=1 rev1=1 crypto rx=0x7f8cf806f4d0 tx=0x7f8ce802b870 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a14aae0bd6a8e475 server_cookie=0 in_seq=0 out_seq=0 2023-08-09T13:50:11.508 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:11.504+0000 7f8cef7fe640 1 -- 172.21.15.43:0/4164964576 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8ce80317d0 con 0x7f8cf80c3130 2023-08-09T13:50:11.508 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:11.504+0000 7f8cef7fe640 1 -- 172.21.15.43:0/4164964576 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8ce8031930 con 0x7f8cf80c3130 2023-08-09T13:50:11.508 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:11.504+0000 7f8cef7fe640 1 -- 172.21.15.43:0/4164964576 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8ce8031cc0 con 0x7f8cf80c3130 2023-08-09T13:50:11.509 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:11.505+0000 7f8cfed42640 1 -- 172.21.15.43:0/4164964576 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f8cf80c3130 msgr2=0x7f8cf80cbc70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:50:11.509 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:11.505+0000 7f8cfed42640 1 --2- 172.21.15.43:0/4164964576 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f8cf80c3130 0x7f8cf80cbc70 secure :-1 s=READY pgs=935 cs=0 l=1 rev1=1 crypto rx=0x7f8cf806f4d0 tx=0x7f8ce802b870 comp rx=0 tx=0).stop 2023-08-09T13:50:11.509 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:11.505+0000 7f8cfed42640 1 -- 172.21.15.43:0/4164964576 shutdown_connections 2023-08-09T13:50:11.509 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:11.505+0000 7f8cfed42640 1 --2- 172.21.15.43:0/4164964576 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f8cf80c3130 0x7f8cf80cbc70 unknown :-1 s=CLOSED pgs=935 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:50:11.509 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:11.505+0000 7f8cfed42640 1 --2- 172.21.15.43:0/4164964576 >> v2:172.21.15.203:3300/0 conn(0x7f8cf80cc1b0 0x7f8cf8164510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:50:11.509 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:11.505+0000 7f8cfed42640 1 --2- 172.21.15.43:0/4164964576 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f8cf814a5c0 0x7f8cf8160f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:50:11.509 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:11.505+0000 7f8cfed42640 1 -- 172.21.15.43:0/4164964576 >> 172.21.15.43:0/4164964576 conn(0x7f8cf80ce0a0 msgr2=0x7f8cf80ce480 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-09T13:50:11.509 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:11.505+0000 7f8cfed42640 1 -- 172.21.15.43:0/4164964576 shutdown_connections 2023-08-09T13:50:11.510 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:11.505+0000 7f8cfed42640 1 -- 172.21.15.43:0/4164964576 wait complete. 2023-08-09T13:50:11.510 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:11.507+0000 7f8cfed42640 1 Processor -- start 2023-08-09T13:50:11.510 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:11.507+0000 7f8cfed42640 1 -- start start 2023-08-09T13:50:11.511 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:11.508+0000 7f8cfed42640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f8cf80c3130 0x7f8cf8161640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:50:11.511 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:11.508+0000 7f8cfed42640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f8cf80cc1b0 0x7f8cf80c5a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:50:11.511 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:11.508+0000 7f8cfed42640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f8cf814a5c0 0x7f8cf80c78a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:50:11.511 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:11.508+0000 7f8cfed42640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8cf80c5f80 con 0x7f8cf80c3130 2023-08-09T13:50:11.511 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:11.508+0000 7f8cfed42640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8cf80b2ac0 con 0x7f8cf80cc1b0 2023-08-09T13:50:11.511 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:11.508+0000 7f8cfed42640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8cf80b2da0 con 0x7f8cf814a5c0 2023-08-09T13:50:11.512 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:11.508+0000 7f8cfcade640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f8cf80c3130 0x7f8cf8161640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:50:11.513 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:11.508+0000 7f8cfcade640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f8cf80c3130 0x7f8cf8161640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:45446/0 (socket says 172.21.15.43:45446) 2023-08-09T13:50:11.513 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:11.508+0000 7f8cfcade640 1 -- 172.21.15.43:0/193579981 learned_addr learned my addr 172.21.15.43:0/193579981 (peer_addr_for_me v2:172.21.15.43:0/0) 2023-08-09T13:50:11.513 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:11.509+0000 7f8ceffff640 1 --2- 172.21.15.43:0/193579981 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f8cf80cc1b0 0x7f8cf80c5a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:50:11.513 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:11.509+0000 7f8cfd2df640 1 --2- 172.21.15.43:0/193579981 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f8cf814a5c0 0x7f8cf80c78a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:50:11.513 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:11.509+0000 7f8cfcade640 1 -- 172.21.15.43:0/193579981 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f8cf814a5c0 msgr2=0x7f8cf80c78a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:50:11.513 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:11.509+0000 7f8cfcade640 1 --2- 172.21.15.43:0/193579981 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f8cf814a5c0 0x7f8cf80c78a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:50:11.513 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:11.509+0000 7f8cfcade640 1 -- 172.21.15.43:0/193579981 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f8cf80cc1b0 msgr2=0x7f8cf80c5a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:50:11.514 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:11.509+0000 7f8cfcade640 1 --2- 172.21.15.43:0/193579981 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f8cf80cc1b0 0x7f8cf80c5a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:50:11.514 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:11.509+0000 7f8cfcade640 1 -- 172.21.15.43:0/193579981 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8ce80094d0 con 0x7f8cf80c3130 2023-08-09T13:50:11.514 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:11.509+0000 7f8cfcade640 1 --2- 172.21.15.43:0/193579981 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f8cf80c3130 0x7f8cf8161640 secure :-1 s=READY pgs=936 cs=0 l=1 rev1=1 crypto rx=0x7f8cf400d5f0 tx=0x7f8cf400dab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-09T13:50:11.514 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:11.509+0000 7f8ced7fa640 1 -- 172.21.15.43:0/193579981 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8cf40125f0 con 0x7f8cf80c3130 2023-08-09T13:50:11.514 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:11.510+0000 7f8ced7fa640 1 -- 172.21.15.43:0/193579981 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8cf4012750 con 0x7f8cf80c3130 2023-08-09T13:50:11.514 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:11.510+0000 7f8ced7fa640 1 -- 172.21.15.43:0/193579981 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8cf4012b00 con 0x7f8cf80c3130 2023-08-09T13:50:11.514 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:11.510+0000 7f8cfed42640 1 -- 172.21.15.43:0/193579981 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8cf80b3080 con 0x7f8cf80c3130 2023-08-09T13:50:11.514 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:11.510+0000 7f8cfed42640 1 -- 172.21.15.43:0/193579981 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8cf80b2790 con 0x7f8cf80c3130 2023-08-09T13:50:11.515 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:11.511+0000 7f8cfed42640 1 -- 172.21.15.43:0/193579981 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8cf80739f0 con 0x7f8cf80c3130 2023-08-09T13:50:11.515 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:11.512+0000 7f8ced7fa640 1 -- 172.21.15.43:0/193579981 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 28) v1 ==== 88387+0+0 (secure 0 0 0) 0x7f8cf4012c60 con 0x7f8cf80c3130 2023-08-09T13:50:11.515 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:11.513+0000 7f8ced7fa640 1 --2- 172.21.15.43:0/193579981 >> [v2:172.21.15.203:6832/3325141227,v1:172.21.15.203:6833/3325141227] conn(0x7f8ce406a0e0 0x7f8ce406c5a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:50:11.516 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:11.513+0000 7f8ced7fa640 1 -- 172.21.15.43:0/193579981 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(77..77 src has 1..77) v4 ==== 7272+0+0 (secure 0 0 0) 0x7f8cf4088f30 con 0x7f8cf80c3130 2023-08-09T13:50:11.516 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:11.513+0000 7f8ceffff640 1 --2- 172.21.15.43:0/193579981 >> [v2:172.21.15.203:6832/3325141227,v1:172.21.15.203:6833/3325141227] conn(0x7f8ce406a0e0 0x7f8ce406c5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:50:11.517 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:11.515+0000 7f8ceffff640 1 --2- 172.21.15.43:0/193579981 >> [v2:172.21.15.203:6832/3325141227,v1:172.21.15.203:6833/3325141227] conn(0x7f8ce406a0e0 0x7f8ce406c5a0 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7f8ce8036000 tx=0x7f8ce803d040 comp rx=0 tx=0).ready entity=mgr.5519 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-09T13:50:11.520 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:11.518+0000 7f8ced7fa640 1 -- 172.21.15.43:0/193579981 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178066 (secure 0 0 0) 0x7f8cf40bb7b0 con 0x7f8cf80c3130 2023-08-09T13:50:11.714 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:11.711+0000 7f8cfed42640 1 -- 172.21.15.43:0/193579981 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f8cf80cc830 con 0x7f8cf80c3130 2023-08-09T13:50:11.717 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:11.715+0000 7f8ced7fa640 1 -- 172.21.15.43:0/193579981 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v77) v1 ==== 74+0+19824 (secure 0 0 0) 0x7f8cf4054170 con 0x7f8cf80c3130 2023-08-09T13:50:11.718 INFO:teuthology.orchestra.run.smithi043.stdout: 2023-08-09T13:50:11.718 INFO:teuthology.orchestra.run.smithi043.stdout:{"epoch":77,"fsid":"daac87ee-1337-4250-a955-aea4e408a39b","created":"2023-08-09T13:33:41.701198+0000","modified":"2023-08-09T13:50:08.172832+0000","last_up_change":"2023-08-09T13:33:53.783453+0000","last_in_change":"2023-08-09T13:33:45.148904+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":9,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":"rbd","create_time":"2023-08-09T13:33:54.365137+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":".mgr","create_time":"2023-08-09T13:33:54.831803+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":8,"pool_name":"cephfs_metadata","create_time":"2023-08-09T13:48:16.039756+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"73","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":9,"pool_name":"cephfs_data","create_time":"2023-08-09T13:48:16.615400+0000","flags":8193,"flags_names":"hashpspool,selfmanaged_snaps","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"77","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":5,"snap_epoch":77,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.6299999952316284,"score_stable":1.6299999952316284,"optimal_score":1,"raw_score_acting":1.6299999952316284,"raw_score_stable":1.6299999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"cbd49ecd-e896-4fbd-864e-5c37a829b165","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":72,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6800","nonce":502667335},{"type":"v1","addr":"172.21.15.43:6801","nonce":502667335}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6802","nonce":502667335},{"type":"v1","addr":"172.21.15.43:6803","nonce":502667335}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6806","nonce":502667335},{"type":"v1","addr":"172.21.15.43:6807","nonce":502667335}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6804","nonce":502667335},{"type":"v1","addr":"172.21.15.43:6805","nonce":502667335}]},"public_addr":"172.21.15.43:6801/502667335","cluster_addr":"172.21.15.43:6803/502667335","heartbeat_back_addr":"172.21.15.43:6807/502667335","heartbeat_front_addr":"172.21.15.43:6805/502667335","state":["exists","up"]},{"osd":1,"uuid":"75c7e0fe-1ec7-4fe8-a2f2-3661a65b8500","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":72,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6808","nonce":3289134610},{"type":"v1","addr":"172.21.15.43:6809","nonce":3289134610}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6810","nonce":3289134610},{"type":"v1","addr":"172.21.15.43:6812","nonce":3289134610}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6821","nonce":3289134610},{"type":"v1","addr":"172.21.15.43:6824","nonce":3289134610}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6815","nonce":3289134610},{"type":"v1","addr":"172.21.15.43:6818","nonce":3289134610}]},"public_addr":"172.21.15.43:6809/3289134610","cluster_addr":"172.21.15.43:6812/3289134610","heartbeat_back_addr":"172.21.15.43:6824/3289134610","heartbeat_front_addr":"172.21.15.43:6818/3289134610","state":["exists","up"]},{"osd":2,"uuid":"bb5c41e9-7f5f-4125-9b01-3bb8bfc2ea83","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":72,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6811","nonce":4079709736},{"type":"v1","addr":"172.21.15.43:6814","nonce":4079709736}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6817","nonce":4079709736},{"type":"v1","addr":"172.21.15.43:6820","nonce":4079709736}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6829","nonce":4079709736},{"type":"v1","addr":"172.21.15.43:6831","nonce":4079709736}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6823","nonce":4079709736},{"type":"v1","addr":"172.21.15.43:6826","nonce":4079709736}]},"public_addr":"172.21.15.43:6814/4079709736","cluster_addr":"172.21.15.43:6820/4079709736","heartbeat_back_addr":"172.21.15.43:6831/4079709736","heartbeat_front_addr":"172.21.15.43:6826/4079709736","state":["exists","up"]},{"osd":3,"uuid":"bfe4c8b4-694a-465c-87e6-2b2aab852613","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":72,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6813","nonce":1709538759},{"type":"v1","addr":"172.21.15.43:6816","nonce":1709538759}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6819","nonce":1709538759},{"type":"v1","addr":"172.21.15.43:6822","nonce":1709538759}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6828","nonce":1709538759},{"type":"v1","addr":"172.21.15.43:6830","nonce":1709538759}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6825","nonce":1709538759},{"type":"v1","addr":"172.21.15.43:6827","nonce":1709538759}]},"public_addr":"172.21.15.43:6816/1709538759","cluster_addr":"172.21.15.43:6822/1709538759","heartbeat_back_addr":"172.21.15.43:6830/1709538759","heartbeat_front_addr":"172.21.15.43:6827/1709538759","state":["exists","up"]},{"osd":4,"uuid":"fa2e8a77-aba1-4756-853e-0d174a5bdeba","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":72,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6800","nonce":4196866481},{"type":"v1","addr":"172.21.15.203:6801","nonce":4196866481}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6802","nonce":4196866481},{"type":"v1","addr":"172.21.15.203:6803","nonce":4196866481}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6806","nonce":4196866481},{"type":"v1","addr":"172.21.15.203:6807","nonce":4196866481}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6804","nonce":4196866481},{"type":"v1","addr":"172.21.15.203:6805","nonce":4196866481}]},"public_addr":"172.21.15.203:6801/4196866481","cluster_addr":"172.21.15.203:6803/4196866481","heartbeat_back_addr":"172.21.15.203:6807/4196866481","heartbeat_front_addr":"172.21.15.203:6805/4196866481","state":["exists","up"]},{"osd":5,"uuid":"81811e4b-4d24-4f51-a82c-f1d700a2a00d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":72,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6816","nonce":3059290081},{"type":"v1","addr":"172.21.15.203:6817","nonce":3059290081}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6818","nonce":3059290081},{"type":"v1","addr":"172.21.15.203:6819","nonce":3059290081}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6822","nonce":3059290081},{"type":"v1","addr":"172.21.15.203:6823","nonce":3059290081}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6820","nonce":3059290081},{"type":"v1","addr":"172.21.15.203:6821","nonce":3059290081}]},"public_addr":"172.21.15.203:6817/3059290081","cluster_addr":"172.21.15.203:6819/3059290081","heartbeat_back_addr":"172.21.15.203:6823/3059290081","heartbeat_front_addr":"172.21.15.203:6821/3059290081","state":["exists","up"]},{"osd":6,"uuid":"307bac4d-6a5b-443d-af75-56d695d7674f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":72,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6824","nonce":512155569},{"type":"v1","addr":"172.21.15.203:6825","nonce":512155569}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6826","nonce":512155569},{"type":"v1","addr":"172.21.15.203:6827","nonce":512155569}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6830","nonce":512155569},{"type":"v1","addr":"172.21.15.203:6831","nonce":512155569}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6828","nonce":512155569},{"type":"v1","addr":"172.21.15.203:6829","nonce":512155569}]},"public_addr":"172.21.15.203:6825/512155569","cluster_addr":"172.21.15.203:6827/512155569","heartbeat_back_addr":"172.21.15.203:6831/512155569","heartbeat_front_addr":"172.21.15.203:6829/512155569","state":["exists","up"]},{"osd":7,"uuid":"a95dc7ca-9f1b-4295-af70-68a22a2d7d5e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":72,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6808","nonce":3331450438},{"type":"v1","addr":"172.21.15.203:6809","nonce":3331450438}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6810","nonce":3331450438},{"type":"v1","addr":"172.21.15.203:6811","nonce":3331450438}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6814","nonce":3331450438},{"type":"v1","addr":"172.21.15.203:6815","nonce":3331450438}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6812","nonce":3331450438},{"type":"v1","addr":"172.21.15.203:6813","nonce":3331450438}]},"public_addr":"172.21.15.203:6809/3331450438","cluster_addr":"172.21.15.203:6811/3331450438","heartbeat_back_addr":"172.21.15.203:6815/3331450438","heartbeat_front_addr":"172.21.15.203:6813/3331450438","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-09T13:33:50.547652+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-09T13:33:50.589469+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-09T13:33:50.561724+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-09T13:33:50.599650+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-09T13:33:50.411855+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-09T13:33:50.219681+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-09T13:33:50.504664+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-09T13:33:50.441248+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.203:0/2251174608":"2023-08-10T13:50:08.172778+0000","172.21.15.203:0/4008902326":"2023-08-10T13:50:08.172778+0000","172.21.15.203:0/1825829254":"2023-08-10T13:50:08.172778+0000","172.21.15.203:0/4097180925":"2023-08-10T13:50:08.172778+0000","172.21.15.203:6832/2515481191":"2023-08-10T13:50:08.172778+0000","172.21.15.203:0/4038087778":"2023-08-10T13:50:08.172778+0000","172.21.15.203:0/826156596":"2023-08-10T13:50:08.172778+0000","172.21.15.203:0/1084459051":"2023-08-10T13:50:08.172778+0000","172.21.15.203:6833/272917756":"2023-08-10T13:48:34.065493+0000","172.21.15.203:6832/272917756":"2023-08-10T13:48:34.065493+0000","172.21.15.203:0/665971116":"2023-08-10T13:48:34.065493+0000","172.21.15.203:0/2868823959":"2023-08-10T13:48:34.065493+0000","172.21.15.203:6833/2515481191":"2023-08-10T13:50:08.172778+0000","172.21.15.203:0/3168500363":"2023-08-10T13:48:34.065493+0000","172.21.15.203:0/752327795":"2023-08-10T13:48:34.065493+0000","172.21.15.203:0/1970263661":"2023-08-10T13:48:34.065493+0000","172.21.15.203:0/2726495528":"2023-08-10T13:50:08.172778+0000","172.21.15.203:0/1407921417":"2023-08-10T13:48:34.065493+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":7,"snaps":[{"begin":104,"length":1},{"begin":221,"length":1}]},{"pool":9,"snaps":[{"begin":2,"length":4}]}],"new_removed_snaps":[{"pool":9,"snaps":[{"begin":3,"length":1},{"begin":5,"length":1}]}],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-08-09T13:50:11.720 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:11.718+0000 7f8cfed42640 1 -- 172.21.15.43:0/193579981 >> [v2:172.21.15.203:6832/3325141227,v1:172.21.15.203:6833/3325141227] conn(0x7f8ce406a0e0 msgr2=0x7f8ce406c5a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:50:11.721 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:11.718+0000 7f8cfed42640 1 --2- 172.21.15.43:0/193579981 >> [v2:172.21.15.203:6832/3325141227,v1:172.21.15.203:6833/3325141227] conn(0x7f8ce406a0e0 0x7f8ce406c5a0 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7f8ce8036000 tx=0x7f8ce803d040 comp rx=0 tx=0).stop 2023-08-09T13:50:11.721 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:11.718+0000 7f8cfed42640 1 -- 172.21.15.43:0/193579981 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f8cf80c3130 msgr2=0x7f8cf8161640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:50:11.721 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:11.718+0000 7f8cfed42640 1 --2- 172.21.15.43:0/193579981 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f8cf80c3130 0x7f8cf8161640 secure :-1 s=READY pgs=936 cs=0 l=1 rev1=1 crypto rx=0x7f8cf400d5f0 tx=0x7f8cf400dab0 comp rx=0 tx=0).stop 2023-08-09T13:50:11.721 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:11.718+0000 7f8cfed42640 1 -- 172.21.15.43:0/193579981 shutdown_connections 2023-08-09T13:50:11.721 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:11.718+0000 7f8cfed42640 1 --2- 172.21.15.43:0/193579981 >> [v2:172.21.15.203:6832/3325141227,v1:172.21.15.203:6833/3325141227] conn(0x7f8ce406a0e0 0x7f8ce406c5a0 unknown :-1 s=CLOSED pgs=27 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:50:11.721 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:11.718+0000 7f8cfed42640 1 --2- 172.21.15.43:0/193579981 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f8cf814a5c0 0x7f8cf80c78a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:50:11.721 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:11.718+0000 7f8cfed42640 1 --2- 172.21.15.43:0/193579981 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f8cf80cc1b0 0x7f8cf80c5a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:50:11.722 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:11.718+0000 7f8cfed42640 1 --2- 172.21.15.43:0/193579981 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f8cf80c3130 0x7f8cf8161640 unknown :-1 s=CLOSED pgs=936 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:50:11.722 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:11.718+0000 7f8cfed42640 1 -- 172.21.15.43:0/193579981 >> 172.21.15.43:0/193579981 conn(0x7f8cf80ce0a0 msgr2=0x7f8cf80c9a50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-09T13:50:11.722 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:11.718+0000 7f8cfed42640 1 -- 172.21.15.43:0/193579981 shutdown_connections 2023-08-09T13:50:11.722 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:11.718+0000 7f8cfed42640 1 -- 172.21.15.43:0/193579981 wait complete. 2023-08-09T13:50:11.738 INFO:tasks.cephfs.filesystem:Destroying file system cephfs and related pools 2023-08-09T13:50:11.739 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-08-09T13:50:11.970 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:11.968+0000 7fdbc5af9640 1 Processor -- start 2023-08-09T13:50:11.971 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:11.968+0000 7fdbc5af9640 1 -- start start 2023-08-09T13:50:11.972 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:11.969+0000 7fdbc5af9640 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7fdbc0143c20 0x7fdbc006f540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:50:11.972 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:11.969+0000 7fdbc5af9640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7fdbc00c86f0 0x7fdbc00c8b50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:50:11.972 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:11.969+0000 7fdbc5af9640 1 -- --> v1:172.21.15.203:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fdbc00c9090 con 0x7fdbc006c8d0 2023-08-09T13:50:11.972 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:11.969+0000 7fdbc5af9640 1 -- --> v2:172.21.15.43:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fdbc0144000 con 0x7fdbc00c86f0 2023-08-09T13:50:11.972 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:11.969+0000 7fdbc5af9640 1 -- --> v2:172.21.15.203:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fdbc006c440 con 0x7fdbc0143c20 2023-08-09T13:50:11.973 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:11.970+0000 7fdbbeffd640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7fdbc00c86f0 0x7fdbc00c8b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:50:11.973 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:11.970+0000 7fdbbeffd640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7fdbc00c86f0 0x7fdbc00c8b50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:45454/0 (socket says 172.21.15.43:45454) 2023-08-09T13:50:11.973 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:11.970+0000 7fdbbeffd640 1 -- 172.21.15.43:0/2985640279 learned_addr learned my addr 172.21.15.43:0/2985640279 (peer_addr_for_me v2:172.21.15.43:0/0) 2023-08-09T13:50:11.974 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:11.970+0000 7fdbbf7fe640 1 --2- 172.21.15.43:0/2985640279 >> v2:172.21.15.203:3300/0 conn(0x7fdbc0143c20 0x7fdbc006f540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:50:11.974 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:11.970+0000 7fdbbeffd640 1 -- 172.21.15.43:0/2985640279 >> v2:172.21.15.203:3300/0 conn(0x7fdbc0143c20 msgr2=0x7fdbc006f540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-09T13:50:11.974 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:11.970+0000 7fdbbeffd640 1 --2- 172.21.15.43:0/2985640279 >> v2:172.21.15.203:3300/0 conn(0x7fdbc0143c20 0x7fdbc006f540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:50:11.974 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:11.970+0000 7fdbbeffd640 1 -- 172.21.15.43:0/2985640279 >> v1:172.21.15.203:6789/0 conn(0x7fdbc006c8d0 legacy=0x7fdbc006cd00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-09T13:50:11.974 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:11.970+0000 7fdbbeffd640 1 -- 172.21.15.43:0/2985640279 --> v2:172.21.15.43:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdbc006b830 con 0x7fdbc00c86f0 2023-08-09T13:50:11.975 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:11.971+0000 7fdbbeffd640 1 --2- 172.21.15.43:0/2985640279 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fdbc00c86f0 0x7fdbc00c8b50 secure :-1 s=READY pgs=937 cs=0 l=1 rev1=1 crypto rx=0x7fdbc006e0f0 tx=0x7fdbb402b7f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2a8c0baef6769c87 server_cookie=0 in_seq=0 out_seq=0 2023-08-09T13:50:11.975 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:11.972+0000 7fdbbe7fc640 1 -- 172.21.15.43:0/2985640279 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdbb4031750 con 0x7fdbc00c86f0 2023-08-09T13:50:11.976 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:11.972+0000 7fdbbe7fc640 1 -- 172.21.15.43:0/2985640279 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdbb40318b0 con 0x7fdbc00c86f0 2023-08-09T13:50:11.976 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:11.972+0000 7fdbbe7fc640 1 -- 172.21.15.43:0/2985640279 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdbb4031c40 con 0x7fdbc00c86f0 2023-08-09T13:50:11.976 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:11.972+0000 7fdbc5af9640 1 -- 172.21.15.43:0/2985640279 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fdbc00c86f0 msgr2=0x7fdbc00c8b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:50:11.976 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:11.972+0000 7fdbc5af9640 1 --2- 172.21.15.43:0/2985640279 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fdbc00c86f0 0x7fdbc00c8b50 secure :-1 s=READY pgs=937 cs=0 l=1 rev1=1 crypto rx=0x7fdbc006e0f0 tx=0x7fdbb402b7f0 comp rx=0 tx=0).stop 2023-08-09T13:50:11.976 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:11.973+0000 7fdbc5af9640 1 -- 172.21.15.43:0/2985640279 shutdown_connections 2023-08-09T13:50:11.976 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:11.973+0000 7fdbc5af9640 1 --2- 172.21.15.43:0/2985640279 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fdbc00c86f0 0x7fdbc00c8b50 unknown :-1 s=CLOSED pgs=937 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:50:11.976 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:11.973+0000 7fdbc5af9640 1 --2- 172.21.15.43:0/2985640279 >> v2:172.21.15.203:3300/0 conn(0x7fdbc0143c20 0x7fdbc006f540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:50:11.977 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:11.973+0000 7fdbc5af9640 1 -- 172.21.15.43:0/2985640279 >> 172.21.15.43:0/2985640279 conn(0x7fdbc00ce0a0 msgr2=0x7fdbc00ce480 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-09T13:50:11.977 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:11.973+0000 7fdbc5af9640 1 -- 172.21.15.43:0/2985640279 shutdown_connections 2023-08-09T13:50:11.977 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:11.973+0000 7fdbc5af9640 1 -- 172.21.15.43:0/2985640279 wait complete. 2023-08-09T13:50:11.977 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:11.974+0000 7fdbc5af9640 1 Processor -- start 2023-08-09T13:50:11.977 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:11.975+0000 7fdbc5af9640 1 -- start start 2023-08-09T13:50:11.979 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:11.975+0000 7fdbc5af9640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fdbc006c8d0 0x7fdbc0159df0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:50:11.979 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:11.975+0000 7fdbc5af9640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fdbc00c86f0 0x7fdbc015c040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:50:11.979 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:11.976+0000 7fdbc5af9640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fdbc0143c20 0x7fdbc00adf00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:50:11.979 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:11.976+0000 7fdbc5af9640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdbc00c6100 con 0x7fdbc006c8d0 2023-08-09T13:50:11.979 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:11.976+0000 7fdbc5af9640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdbc015c940 con 0x7fdbc0143c20 2023-08-09T13:50:11.979 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:11.976+0000 7fdbc5af9640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdbc015cc20 con 0x7fdbc00c86f0 2023-08-09T13:50:11.979 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:11.976+0000 7fdbbf7fe640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fdbc006c8d0 0x7fdbc0159df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:50:11.980 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:11.976+0000 7fdbbf7fe640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fdbc006c8d0 0x7fdbc0159df0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:45466/0 (socket says 172.21.15.43:45466) 2023-08-09T13:50:11.980 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:11.976+0000 7fdbbf7fe640 1 -- 172.21.15.43:0/3566122849 learned_addr learned my addr 172.21.15.43:0/3566122849 (peer_addr_for_me v2:172.21.15.43:0/0) 2023-08-09T13:50:11.980 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:11.976+0000 7fdbbeffd640 1 --2- 172.21.15.43:0/3566122849 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fdbc00c86f0 0x7fdbc015c040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:50:11.980 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:11.976+0000 7fdbbffff640 1 --2- 172.21.15.43:0/3566122849 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fdbc0143c20 0x7fdbc00adf00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:50:11.980 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:11.976+0000 7fdbbf7fe640 1 -- 172.21.15.43:0/3566122849 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fdbc00c86f0 msgr2=0x7fdbc015c040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:50:11.980 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:11.976+0000 7fdbbf7fe640 1 --2- 172.21.15.43:0/3566122849 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fdbc00c86f0 0x7fdbc015c040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:50:11.980 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:11.976+0000 7fdbbf7fe640 1 -- 172.21.15.43:0/3566122849 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fdbc0143c20 msgr2=0x7fdbc00adf00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:50:11.981 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:11.976+0000 7fdbbf7fe640 1 --2- 172.21.15.43:0/3566122849 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fdbc0143c20 0x7fdbc00adf00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:50:11.981 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:11.976+0000 7fdbbf7fe640 1 -- 172.21.15.43:0/3566122849 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdbb40094d0 con 0x7fdbc006c8d0 2023-08-09T13:50:11.981 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:11.976+0000 7fdbbffff640 1 --2- 172.21.15.43:0/3566122849 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fdbc0143c20 0x7fdbc00adf00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-08-09T13:50:11.982 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:11.977+0000 7fdbbf7fe640 1 --2- 172.21.15.43:0/3566122849 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fdbc006c8d0 0x7fdbc0159df0 secure :-1 s=READY pgs=938 cs=0 l=1 rev1=1 crypto rx=0x7fdbb000d800 tx=0x7fdbb000dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-09T13:50:11.982 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:11.977+0000 7fdba7fff640 1 -- 172.21.15.43:0/3566122849 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdbb000b6c0 con 0x7fdbc006c8d0 2023-08-09T13:50:11.982 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:11.977+0000 7fdba7fff640 1 -- 172.21.15.43:0/3566122849 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdbb000b820 con 0x7fdbc006c8d0 2023-08-09T13:50:11.982 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:11.977+0000 7fdba7fff640 1 -- 172.21.15.43:0/3566122849 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdbb000bbb0 con 0x7fdbc006c8d0 2023-08-09T13:50:11.982 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:11.977+0000 7fdbc5af9640 1 -- 172.21.15.43:0/3566122849 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdbc00c5d90 con 0x7fdbc006c8d0 2023-08-09T13:50:11.982 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:11.977+0000 7fdbc5af9640 1 -- 172.21.15.43:0/3566122849 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdbc015cf00 con 0x7fdbc006c8d0 2023-08-09T13:50:11.982 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:11.978+0000 7fdbc5af9640 1 -- 172.21.15.43:0/3566122849 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdbc00739f0 con 0x7fdbc006c8d0 2023-08-09T13:50:11.983 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:11.979+0000 7fdba7fff640 1 -- 172.21.15.43:0/3566122849 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 28) v1 ==== 88387+0+0 (secure 0 0 0) 0x7fdbb000bd10 con 0x7fdbc006c8d0 2023-08-09T13:50:11.983 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:11.980+0000 7fdba7fff640 1 --2- 172.21.15.43:0/3566122849 >> [v2:172.21.15.203:6832/3325141227,v1:172.21.15.203:6833/3325141227] conn(0x7fdbac06a0e0 0x7fdbac06c5a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:50:11.983 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:11.980+0000 7fdba7fff640 1 -- 172.21.15.43:0/3566122849 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(77..77 src has 1..77) v4 ==== 7272+0+0 (secure 0 0 0) 0x7fdbb0088f70 con 0x7fdbc006c8d0 2023-08-09T13:50:11.983 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:11.981+0000 7fdbbeffd640 1 --2- 172.21.15.43:0/3566122849 >> [v2:172.21.15.203:6832/3325141227,v1:172.21.15.203:6833/3325141227] conn(0x7fdbac06a0e0 0x7fdbac06c5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:50:11.984 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:11.982+0000 7fdbbeffd640 1 --2- 172.21.15.43:0/3566122849 >> [v2:172.21.15.203:6832/3325141227,v1:172.21.15.203:6833/3325141227] conn(0x7fdbac06a0e0 0x7fdbac06c5a0 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7fdbb4035000 tx=0x7fdbb4008030 comp rx=0 tx=0).ready entity=mgr.5519 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-09T13:50:11.987 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:11.985+0000 7fdba7fff640 1 -- 172.21.15.43:0/3566122849 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178066 (secure 0 0 0) 0x7fdbb00bb7b0 con 0x7fdbc006c8d0 2023-08-09T13:50:12.225 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:12.222+0000 7fdbc5af9640 1 -- 172.21.15.43:0/3566122849 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fdbc00cec90 con 0x7fdbc006c8d0 2023-08-09T13:50:12.226 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:12.224+0000 7fdba7fff640 1 -- 172.21.15.43:0/3566122849 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 26 v26) v1 ==== 94+0+4881 (secure 0 0 0) 0x7fdbb0057cd0 con 0x7fdbc006c8d0 2023-08-09T13:50:12.226 INFO:teuthology.orchestra.run.smithi043.stdout: 2023-08-09T13:50:12.227 INFO:teuthology.orchestra.run.smithi043.stdout:{"epoch":26,"default_fscid":3,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5369,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.203:6837/1426673901","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6836","nonce":1426673901},{"type":"v1","addr":"172.21.15.203:6837","nonce":1426673901}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":21},{"gid":5728,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.43:6833/1096124235","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6832","nonce":1096124235},{"type":"v1","addr":"172.21.15.43:6833","nonce":1096124235}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":20},{"gid":5731,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.43:6835/1688804357","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6834","nonce":1688804357},{"type":"v1","addr":"172.21.15.43:6835","nonce":1688804357}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":21}],"filesystems":[{"mdsmap":{"epoch":26,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-08-09T13:48:17.607195+0000","modified":"2023-08-09T13:48:21.179296+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":5734},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5734":{"gid":5734,"name":"b","rank":0,"incarnation":22,"state":"up:active","state_seq":3,"addr":"172.21.15.203:6835/3324252767","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6834","nonce":3324252767},{"type":"v1","addr":"172.21.15.203:6835","nonce":3324252767}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[9],"metadata_pool":8,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":3}]} 2023-08-09T13:50:12.227 INFO:teuthology.orchestra.run.smithi043.stderr:dumped fsmap epoch 26 2023-08-09T13:50:12.229 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:12.226+0000 7fdbc5af9640 1 -- 172.21.15.43:0/3566122849 >> [v2:172.21.15.203:6832/3325141227,v1:172.21.15.203:6833/3325141227] conn(0x7fdbac06a0e0 msgr2=0x7fdbac06c5a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:50:12.229 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:12.226+0000 7fdbc5af9640 1 --2- 172.21.15.43:0/3566122849 >> [v2:172.21.15.203:6832/3325141227,v1:172.21.15.203:6833/3325141227] conn(0x7fdbac06a0e0 0x7fdbac06c5a0 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7fdbb4035000 tx=0x7fdbb4008030 comp rx=0 tx=0).stop 2023-08-09T13:50:12.229 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:12.226+0000 7fdbc5af9640 1 -- 172.21.15.43:0/3566122849 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fdbc006c8d0 msgr2=0x7fdbc0159df0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:50:12.229 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:12.226+0000 7fdbc5af9640 1 --2- 172.21.15.43:0/3566122849 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fdbc006c8d0 0x7fdbc0159df0 secure :-1 s=READY pgs=938 cs=0 l=1 rev1=1 crypto rx=0x7fdbb000d800 tx=0x7fdbb000dcc0 comp rx=0 tx=0).stop 2023-08-09T13:50:12.230 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:12.226+0000 7fdbc5af9640 1 -- 172.21.15.43:0/3566122849 shutdown_connections 2023-08-09T13:50:12.230 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:12.226+0000 7fdbc5af9640 1 --2- 172.21.15.43:0/3566122849 >> [v2:172.21.15.203:6832/3325141227,v1:172.21.15.203:6833/3325141227] conn(0x7fdbac06a0e0 0x7fdbac06c5a0 unknown :-1 s=CLOSED pgs=28 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:50:12.230 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:12.226+0000 7fdbc5af9640 1 --2- 172.21.15.43:0/3566122849 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fdbc0143c20 0x7fdbc00adf00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:50:12.230 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:12.226+0000 7fdbc5af9640 1 --2- 172.21.15.43:0/3566122849 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fdbc00c86f0 0x7fdbc015c040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:50:12.230 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:12.226+0000 7fdbc5af9640 1 --2- 172.21.15.43:0/3566122849 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fdbc006c8d0 0x7fdbc0159df0 unknown :-1 s=CLOSED pgs=938 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:50:12.230 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:12.226+0000 7fdbc5af9640 1 -- 172.21.15.43:0/3566122849 >> 172.21.15.43:0/3566122849 conn(0x7fdbc00ce0a0 msgr2=0x7fdbc00c9220 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-09T13:50:12.230 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:12.227+0000 7fdbc5af9640 1 -- 172.21.15.43:0/3566122849 shutdown_connections 2023-08-09T13:50:12.231 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:12.227+0000 7fdbc5af9640 1 -- 172.21.15.43:0/3566122849 wait complete. 2023-08-09T13:50:12.249 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-08-09T13:50:12.464 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:12.461+0000 7ff4628ad640 1 Processor -- start 2023-08-09T13:50:12.464 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:12.462+0000 7ff4628ad640 1 -- start start 2023-08-09T13:50:12.465 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:12.463+0000 7ff4628ad640 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7ff45c0b4670 0x7ff45c0d2ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:50:12.465 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:12.463+0000 7ff4628ad640 1 -- --> v1:172.21.15.43:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff45c0ce090 con 0x7ff45c0b2b70 2023-08-09T13:50:12.466 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:12.463+0000 7ff4628ad640 1 -- --> v1:172.21.15.203:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff45c0cd7f0 con 0x7ff45c0b2790 2023-08-09T13:50:12.466 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:12.463+0000 7ff4628ad640 1 -- --> v2:172.21.15.203:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff45c0b3950 con 0x7ff45c0b4670 2023-08-09T13:50:12.466 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:12.463+0000 7ff453fff640 1 --1- >> v1:172.21.15.43:6789/0 conn(0x7ff45c0b2b70 0x7ff45c0b3f60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.43:6789/0 says I am v1:172.21.15.43:58792/0 (socket says 172.21.15.43:58792) 2023-08-09T13:50:12.467 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:12.463+0000 7ff453fff640 1 -- 172.21.15.43:0/3041753483 learned_addr learned my addr 172.21.15.43:0/3041753483 (peer_addr_for_me v1:172.21.15.43:0/0) 2023-08-09T13:50:12.467 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:12.464+0000 7ff460e4a640 1 --2- 172.21.15.43:0/3041753483 >> v2:172.21.15.203:3300/0 conn(0x7ff45c0b4670 0x7ff45c0d2ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:50:12.467 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:12.464+0000 7ff45b7fe640 1 -- 172.21.15.43:0/3041753483 <== mon.0 v1:172.21.15.43:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2260353863 0 0) 0x7ff45c0ce090 con 0x7ff45c0b2b70 2023-08-09T13:50:12.467 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:12.464+0000 7ff45b7fe640 1 -- 172.21.15.43:0/3041753483 --> v1:172.21.15.43:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ff448003520 con 0x7ff45c0b2b70 2023-08-09T13:50:12.467 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:12.464+0000 7ff45b7fe640 1 -- 172.21.15.43:0/3041753483 <== mon.0 v1:172.21.15.43:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3653896157 0 0) 0x7ff448003520 con 0x7ff45c0b2b70 2023-08-09T13:50:12.467 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:12.464+0000 7ff45b7fe640 1 -- 172.21.15.43:0/3041753483 >> v2:172.21.15.203:3300/0 conn(0x7ff45c0b4670 msgr2=0x7ff45c0d2ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-09T13:50:12.467 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:12.464+0000 7ff45b7fe640 1 --2- 172.21.15.43:0/3041753483 >> v2:172.21.15.203:3300/0 conn(0x7ff45c0b4670 0x7ff45c0d2ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:50:12.468 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:12.464+0000 7ff45b7fe640 1 -- 172.21.15.43:0/3041753483 >> v1:172.21.15.203:6789/0 conn(0x7ff45c0b2790 legacy=0x7ff45c0b5330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:50:12.468 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:12.464+0000 7ff45b7fe640 1 -- 172.21.15.43:0/3041753483 --> v1:172.21.15.43:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff45c0cdad0 con 0x7ff45c0b2b70 2023-08-09T13:50:12.469 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:12.464+0000 7ff45b7fe640 1 -- 172.21.15.43:0/3041753483 <== mon.0 v1:172.21.15.43:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1285150740 0 0) 0x7ff444001670 con 0x7ff45c0b2b70 2023-08-09T13:50:12.469 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:12.464+0000 7ff45b7fe640 1 -- 172.21.15.43:0/3041753483 >> v1:172.21.15.43:6789/0 conn(0x7ff45c0b2b70 legacy=0x7ff45c0b3f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:50:12.469 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:12.464+0000 7ff45b7fe640 1 --2- 172.21.15.43:0/3041753483 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff4480039e0 0x7ff448003dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:50:12.469 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:12.464+0000 7ff45b7fe640 1 --2- 172.21.15.43:0/3041753483 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7ff4480047c0 0x7ff448004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:50:12.469 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:12.464+0000 7ff45b7fe640 1 --2- 172.21.15.43:0/3041753483 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7ff448005180 0x7ff44800b980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:50:12.469 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:12.464+0000 7ff45b7fe640 1 -- 172.21.15.43:0/3041753483 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff45c0b3950 con 0x7ff4480039e0 2023-08-09T13:50:12.469 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:12.464+0000 7ff45b7fe640 1 -- 172.21.15.43:0/3041753483 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff44800c0b0 con 0x7ff448005180 2023-08-09T13:50:12.470 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:12.464+0000 7ff45b7fe640 1 -- 172.21.15.43:0/3041753483 --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff44800c270 con 0x7ff4480047c0 2023-08-09T13:50:12.470 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:12.465+0000 7ff45bfff640 1 --2- 172.21.15.43:0/3041753483 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff4480039e0 0x7ff448003dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:50:12.470 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:12.465+0000 7ff453fff640 1 --2- 172.21.15.43:0/3041753483 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7ff4480047c0 0x7ff448004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:50:12.470 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:12.465+0000 7ff45bfff640 1 -- 172.21.15.43:0/3041753483 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7ff4480047c0 msgr2=0x7ff448004c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-09T13:50:12.470 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:12.465+0000 7ff45bfff640 1 --2- 172.21.15.43:0/3041753483 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7ff4480047c0 0x7ff448004c40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:50:12.470 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:12.465+0000 7ff45bfff640 1 -- 172.21.15.43:0/3041753483 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7ff448005180 msgr2=0x7ff44800b980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-09T13:50:12.470 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:12.465+0000 7ff460e4a640 1 --2- 172.21.15.43:0/3041753483 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7ff448005180 0x7ff44800b980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:50:12.470 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:12.465+0000 7ff45bfff640 1 --2- 172.21.15.43:0/3041753483 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7ff448005180 0x7ff44800b980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:50:12.471 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:12.465+0000 7ff45bfff640 1 -- 172.21.15.43:0/3041753483 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff448003520 con 0x7ff4480039e0 2023-08-09T13:50:12.471 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:12.465+0000 7ff460e4a640 1 --2- 172.21.15.43:0/3041753483 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7ff448005180 0x7ff44800b980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-08-09T13:50:12.471 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:12.465+0000 7ff460e4a640 1 -- 172.21.15.43:0/3041753483 reap_dead start 2023-08-09T13:50:12.472 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:12.467+0000 7ff45bfff640 1 --2- 172.21.15.43:0/3041753483 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff4480039e0 0x7ff448003dc0 secure :-1 s=READY pgs=940 cs=0 l=1 rev1=1 crypto rx=0x7ff44c001830 tx=0x7ff4480057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=227446d8f5ddeab8 server_cookie=0 in_seq=0 out_seq=0 2023-08-09T13:50:12.472 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:12.467+0000 7ff45b7fe640 1 -- 172.21.15.43:0/3041753483 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff44c034680 con 0x7ff4480039e0 2023-08-09T13:50:12.472 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:12.467+0000 7ff45b7fe640 1 -- 172.21.15.43:0/3041753483 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff44c034800 con 0x7ff4480039e0 2023-08-09T13:50:12.472 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:12.467+0000 7ff45b7fe640 1 -- 172.21.15.43:0/3041753483 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff44c034bb0 con 0x7ff4480039e0 2023-08-09T13:50:12.472 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:12.467+0000 7ff4628ad640 1 -- 172.21.15.43:0/3041753483 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff4480039e0 msgr2=0x7ff448003dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:50:12.472 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:12.467+0000 7ff4628ad640 1 --2- 172.21.15.43:0/3041753483 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff4480039e0 0x7ff448003dc0 secure :-1 s=READY pgs=940 cs=0 l=1 rev1=1 crypto rx=0x7ff44c001830 tx=0x7ff4480057f0 comp rx=0 tx=0).stop 2023-08-09T13:50:12.473 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:12.468+0000 7ff4628ad640 1 -- 172.21.15.43:0/3041753483 shutdown_connections 2023-08-09T13:50:12.473 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:12.468+0000 7ff4628ad640 1 --2- 172.21.15.43:0/3041753483 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff4480039e0 0x7ff448003dc0 unknown :-1 s=CLOSED pgs=940 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:50:12.473 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:12.468+0000 7ff4628ad640 1 -- 172.21.15.43:0/3041753483 >> 172.21.15.43:0/3041753483 conn(0x7ff45c09b7b0 msgr2=0x7ff45c0d24e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-09T13:50:12.473 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:12.468+0000 7ff4628ad640 1 -- 172.21.15.43:0/3041753483 shutdown_connections 2023-08-09T13:50:12.473 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:12.468+0000 7ff4628ad640 1 -- 172.21.15.43:0/3041753483 wait complete. 2023-08-09T13:50:12.474 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:12.470+0000 7ff4628ad640 1 Processor -- start 2023-08-09T13:50:12.474 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:12.470+0000 7ff4628ad640 1 -- start start 2023-08-09T13:50:12.474 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:12.471+0000 7ff4628ad640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff4480039e0 0x7ff45c1f5f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:50:12.474 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:12.471+0000 7ff4628ad640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7ff45c0b17b0 0x7ff45c0c8290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:50:12.475 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:12.471+0000 7ff4628ad640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7ff45c154320 0x7ff45c0b5100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:50:12.475 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:12.472+0000 7ff4628ad640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff45c1f5b10 con 0x7ff4480039e0 2023-08-09T13:50:12.475 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:12.472+0000 7ff4628ad640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff45c1f5490 con 0x7ff45c0b17b0 2023-08-09T13:50:12.475 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:12.472+0000 7ff4628ad640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff45c0c8ac0 con 0x7ff45c154320 2023-08-09T13:50:12.475 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:12.472+0000 7ff45bfff640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff4480039e0 0x7ff45c1f5f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:50:12.475 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:12.472+0000 7ff45bfff640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff4480039e0 0x7ff45c1f5f30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:45482/0 (socket says 172.21.15.43:45482) 2023-08-09T13:50:12.475 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:12.472+0000 7ff45bfff640 1 -- 172.21.15.43:0/3067577148 learned_addr learned my addr 172.21.15.43:0/3067577148 (peer_addr_for_me v2:172.21.15.43:0/0) 2023-08-09T13:50:12.476 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:12.472+0000 7ff453fff640 1 --2- 172.21.15.43:0/3067577148 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7ff45c0b17b0 0x7ff45c0c8290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:50:12.477 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:12.472+0000 7ff460e4a640 1 --2- 172.21.15.43:0/3067577148 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7ff45c154320 0x7ff45c0b5100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:50:12.477 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:12.472+0000 7ff45bfff640 1 -- 172.21.15.43:0/3067577148 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7ff45c154320 msgr2=0x7ff45c0b5100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:50:12.477 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:12.472+0000 7ff45bfff640 1 --2- 172.21.15.43:0/3067577148 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7ff45c154320 0x7ff45c0b5100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:50:12.477 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:12.472+0000 7ff45bfff640 1 -- 172.21.15.43:0/3067577148 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7ff45c0b17b0 msgr2=0x7ff45c0c8290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:50:12.477 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:12.472+0000 7ff45bfff640 1 --2- 172.21.15.43:0/3067577148 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7ff45c0b17b0 0x7ff45c0c8290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:50:12.477 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:12.472+0000 7ff45bfff640 1 -- 172.21.15.43:0/3067577148 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff45c1f5730 con 0x7ff4480039e0 2023-08-09T13:50:12.477 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:12.472+0000 7ff460e4a640 1 --2- 172.21.15.43:0/3067577148 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7ff45c154320 0x7ff45c0b5100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-08-09T13:50:12.477 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:12.473+0000 7ff45bfff640 1 --2- 172.21.15.43:0/3067577148 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff4480039e0 0x7ff45c1f5f30 secure :-1 s=READY pgs=941 cs=0 l=1 rev1=1 crypto rx=0x7ff44c037040 tx=0x7ff44c001230 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-09T13:50:12.478 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:12.473+0000 7ff4597fa640 1 -- 172.21.15.43:0/3067577148 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff44c03e610 con 0x7ff4480039e0 2023-08-09T13:50:12.478 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:12.473+0000 7ff4597fa640 1 -- 172.21.15.43:0/3067577148 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff44c03e770 con 0x7ff4480039e0 2023-08-09T13:50:12.478 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:12.473+0000 7ff4597fa640 1 -- 172.21.15.43:0/3067577148 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff44c03eb00 con 0x7ff4480039e0 2023-08-09T13:50:12.478 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:12.473+0000 7ff4628ad640 1 -- 172.21.15.43:0/3067577148 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff45c0c87d0 con 0x7ff4480039e0 2023-08-09T13:50:12.478 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:12.473+0000 7ff4628ad640 1 -- 172.21.15.43:0/3067577148 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff45c1547a0 con 0x7ff4480039e0 2023-08-09T13:50:12.479 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:12.475+0000 7ff4628ad640 1 -- 172.21.15.43:0/3067577148 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff45c1f5d80 con 0x7ff4480039e0 2023-08-09T13:50:12.479 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:12.475+0000 7ff4597fa640 1 -- 172.21.15.43:0/3067577148 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 28) v1 ==== 88387+0+0 (secure 0 0 0) 0x7ff44c03b050 con 0x7ff4480039e0 2023-08-09T13:50:12.479 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:12.476+0000 7ff4597fa640 1 --2- 172.21.15.43:0/3067577148 >> [v2:172.21.15.203:6832/3325141227,v1:172.21.15.203:6833/3325141227] conn(0x7ff43c06a0e0 0x7ff43c06c5a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:50:12.479 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:12.477+0000 7ff4597fa640 1 -- 172.21.15.43:0/3067577148 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(78..78 src has 1..78) v4 ==== 7228+0+0 (secure 0 0 0) 0x7ff44c0aa500 con 0x7ff4480039e0 2023-08-09T13:50:12.480 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:12.477+0000 7ff453fff640 1 --2- 172.21.15.43:0/3067577148 >> [v2:172.21.15.203:6832/3325141227,v1:172.21.15.203:6833/3325141227] conn(0x7ff43c06a0e0 0x7ff43c06c5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:50:12.480 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:12.478+0000 7ff453fff640 1 --2- 172.21.15.43:0/3067577148 >> [v2:172.21.15.203:6832/3325141227,v1:172.21.15.203:6833/3325141227] conn(0x7ff43c06a0e0 0x7ff43c06c5a0 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7ff45c0c7ca0 tx=0x7ff444009040 comp rx=0 tx=0).ready entity=mgr.5519 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-09T13:50:12.484 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:12.482+0000 7ff4597fa640 1 -- 172.21.15.43:0/3067577148 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178066 (secure 0 0 0) 0x7ff44c075730 con 0x7ff4480039e0 2023-08-09T13:50:12.720 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:12.717+0000 7ff4628ad640 1 -- 172.21.15.43:0/3067577148 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7ff45c1f58c0 con 0x7ff4480039e0 2023-08-09T13:50:12.720 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:12.718+0000 7ff4597fa640 1 -- 172.21.15.43:0/3067577148 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 26 v26) v1 ==== 94+0+4881 (secure 0 0 0) 0x7ff44c039090 con 0x7ff4480039e0 2023-08-09T13:50:12.721 INFO:teuthology.orchestra.run.smithi043.stdout: 2023-08-09T13:50:12.721 INFO:teuthology.orchestra.run.smithi043.stdout:{"epoch":26,"default_fscid":3,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5369,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.203:6837/1426673901","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6836","nonce":1426673901},{"type":"v1","addr":"172.21.15.203:6837","nonce":1426673901}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":21},{"gid":5728,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.43:6833/1096124235","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6832","nonce":1096124235},{"type":"v1","addr":"172.21.15.43:6833","nonce":1096124235}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":20},{"gid":5731,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.43:6835/1688804357","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6834","nonce":1688804357},{"type":"v1","addr":"172.21.15.43:6835","nonce":1688804357}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":21}],"filesystems":[{"mdsmap":{"epoch":26,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-08-09T13:48:17.607195+0000","modified":"2023-08-09T13:48:21.179296+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":5734},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5734":{"gid":5734,"name":"b","rank":0,"incarnation":22,"state":"up:active","state_seq":3,"addr":"172.21.15.203:6835/3324252767","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6834","nonce":3324252767},{"type":"v1","addr":"172.21.15.203:6835","nonce":3324252767}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[9],"metadata_pool":8,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":3}]} 2023-08-09T13:50:12.721 INFO:teuthology.orchestra.run.smithi043.stderr:dumped fsmap epoch 26 2023-08-09T13:50:12.723 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:12.720+0000 7ff4628ad640 1 -- 172.21.15.43:0/3067577148 >> [v2:172.21.15.203:6832/3325141227,v1:172.21.15.203:6833/3325141227] conn(0x7ff43c06a0e0 msgr2=0x7ff43c06c5a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:50:12.723 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:12.720+0000 7ff4628ad640 1 --2- 172.21.15.43:0/3067577148 >> [v2:172.21.15.203:6832/3325141227,v1:172.21.15.203:6833/3325141227] conn(0x7ff43c06a0e0 0x7ff43c06c5a0 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7ff45c0c7ca0 tx=0x7ff444009040 comp rx=0 tx=0).stop 2023-08-09T13:50:12.724 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:12.721+0000 7ff4628ad640 1 -- 172.21.15.43:0/3067577148 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff4480039e0 msgr2=0x7ff45c1f5f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:50:12.724 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:12.721+0000 7ff4628ad640 1 --2- 172.21.15.43:0/3067577148 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff4480039e0 0x7ff45c1f5f30 secure :-1 s=READY pgs=941 cs=0 l=1 rev1=1 crypto rx=0x7ff44c037040 tx=0x7ff44c001230 comp rx=0 tx=0).stop 2023-08-09T13:50:12.724 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:12.721+0000 7ff4628ad640 1 -- 172.21.15.43:0/3067577148 shutdown_connections 2023-08-09T13:50:12.724 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:12.721+0000 7ff4628ad640 1 --2- 172.21.15.43:0/3067577148 >> [v2:172.21.15.203:6832/3325141227,v1:172.21.15.203:6833/3325141227] conn(0x7ff43c06a0e0 0x7ff43c06c5a0 unknown :-1 s=CLOSED pgs=29 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:50:12.724 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:12.721+0000 7ff4628ad640 1 --2- 172.21.15.43:0/3067577148 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7ff45c154320 0x7ff45c0b5100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:50:12.724 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:12.721+0000 7ff4628ad640 1 --2- 172.21.15.43:0/3067577148 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7ff45c0b17b0 0x7ff45c0c8290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:50:12.724 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:12.721+0000 7ff4628ad640 1 --2- 172.21.15.43:0/3067577148 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff4480039e0 0x7ff45c1f5f30 unknown :-1 s=CLOSED pgs=941 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:50:12.725 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:12.721+0000 7ff4628ad640 1 -- 172.21.15.43:0/3067577148 >> 172.21.15.43:0/3067577148 conn(0x7ff45c09b7b0 msgr2=0x7ff45c0cdff0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-09T13:50:12.725 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:12.721+0000 7ff4628ad640 1 -- 172.21.15.43:0/3067577148 shutdown_connections 2023-08-09T13:50:12.725 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:12.721+0000 7ff4628ad640 1 -- 172.21.15.43:0/3067577148 wait complete. 2023-08-09T13:50:12.741 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-08-09T13:50:12.961 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:12.959+0000 7feef39ea640 1 Processor -- start 2023-08-09T13:50:12.961 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:12.959+0000 7feef39ea640 1 -- start start 2023-08-09T13:50:12.963 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:12.961+0000 7feef39ea640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7feeec142ca0 0x7feeec0c96e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:50:12.963 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:12.961+0000 7feef39ea640 1 -- --> v1:172.21.15.43:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7feeec143080 con 0x7feeec15ce40 2023-08-09T13:50:12.963 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:12.961+0000 7feef39ea640 1 -- --> v1:172.21.15.203:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7feeec06d400 con 0x7feeec0cd510 2023-08-09T13:50:12.963 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:12.961+0000 7feef39ea640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7feeec0c9d90 con 0x7feeec142ca0 2023-08-09T13:50:12.965 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:12.961+0000 7feef0f85640 1 --1- >> v1:172.21.15.43:6789/0 conn(0x7feeec15ce40 0x7feeec0cce00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.43:6789/0 says I am v1:172.21.15.43:58804/0 (socket says 172.21.15.43:58804) 2023-08-09T13:50:12.965 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:12.961+0000 7feef0f85640 1 -- 172.21.15.43:0/1754652759 learned_addr learned my addr 172.21.15.43:0/1754652759 (peer_addr_for_me v1:172.21.15.43:0/0) 2023-08-09T13:50:12.965 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:12.962+0000 7feef1786640 1 --2- 172.21.15.43:0/1754652759 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7feeec142ca0 0x7feeec0c96e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:50:12.965 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:12.962+0000 7feee3fff640 1 -- 172.21.15.43:0/1754652759 <== mon.0 v1:172.21.15.43:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3231819978 0 0) 0x7feeec143080 con 0x7feeec15ce40 2023-08-09T13:50:12.965 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:12.962+0000 7feee3fff640 1 -- 172.21.15.43:0/1754652759 --> v1:172.21.15.43:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7feed8005730 con 0x7feeec15ce40 2023-08-09T13:50:12.965 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:12.962+0000 7feee3fff640 1 -- 172.21.15.43:0/1754652759 <== mon.0 v1:172.21.15.43:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 443486354 0 0) 0x7feed8005730 con 0x7feeec15ce40 2023-08-09T13:50:12.965 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:12.962+0000 7feee3fff640 1 -- 172.21.15.43:0/1754652759 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7feeec142ca0 msgr2=0x7feeec0c96e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-09T13:50:12.966 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:12.962+0000 7feee3fff640 1 --2- 172.21.15.43:0/1754652759 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7feeec142ca0 0x7feeec0c96e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:50:12.966 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:12.962+0000 7feee3fff640 1 -- 172.21.15.43:0/1754652759 >> v1:172.21.15.203:6789/0 conn(0x7feeec0cd510 legacy=0x7feeec0c3290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:50:12.966 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:12.962+0000 7feee3fff640 1 -- 172.21.15.43:0/1754652759 --> v1:172.21.15.43:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7feeec0ca070 con 0x7feeec15ce40 2023-08-09T13:50:12.966 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:12.962+0000 7feee3fff640 1 -- 172.21.15.43:0/1754652759 <== mon.0 v1:172.21.15.43:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1285150740 0 0) 0x7feedc001670 con 0x7feeec15ce40 2023-08-09T13:50:12.966 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:12.962+0000 7feee3fff640 1 -- 172.21.15.43:0/1754652759 >> v1:172.21.15.43:6789/0 conn(0x7feeec15ce40 legacy=0x7feeec0cce00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:50:12.966 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:12.962+0000 7feee3fff640 1 --2- 172.21.15.43:0/1754652759 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7feed8005c20 0x7feed8006040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:50:12.966 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:12.962+0000 7feee3fff640 1 --2- 172.21.15.43:0/1754652759 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7feed80069d0 0x7feed8006e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:50:12.967 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:12.962+0000 7feee3fff640 1 --2- 172.21.15.43:0/1754652759 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7feed80073e0 0x7feed800db90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:50:12.967 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:12.962+0000 7feee3fff640 1 -- 172.21.15.43:0/1754652759 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feeec0c9d90 con 0x7feed80069d0 2023-08-09T13:50:12.967 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:12.962+0000 7feee3fff640 1 -- 172.21.15.43:0/1754652759 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feed800e310 con 0x7feed80073e0 2023-08-09T13:50:12.967 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:12.962+0000 7feee3fff640 1 -- 172.21.15.43:0/1754652759 --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7feed800e4d0 con 0x7feed8005c20 2023-08-09T13:50:12.967 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:12.962+0000 7feef0f85640 1 --2- 172.21.15.43:0/1754652759 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7feed80069d0 0x7feed8006e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:50:12.968 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:12.963+0000 7feef1786640 1 --2- 172.21.15.43:0/1754652759 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7feed8005c20 0x7feed8006040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:50:12.968 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:12.963+0000 7feef0f85640 1 -- 172.21.15.43:0/1754652759 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7feed8005c20 msgr2=0x7feed8006040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-09T13:50:12.969 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:12.963+0000 7feef1f87640 1 --2- 172.21.15.43:0/1754652759 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7feed80073e0 0x7feed800db90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:50:12.969 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:12.963+0000 7feef0f85640 1 --2- 172.21.15.43:0/1754652759 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7feed8005c20 0x7feed8006040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:50:12.969 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:12.963+0000 7feef0f85640 1 -- 172.21.15.43:0/1754652759 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7feed80073e0 msgr2=0x7feed800db90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-09T13:50:12.969 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:12.963+0000 7feef0f85640 1 --2- 172.21.15.43:0/1754652759 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7feed80073e0 0x7feed800db90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:50:12.969 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:12.963+0000 7feef0f85640 1 -- 172.21.15.43:0/1754652759 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7feed8005730 con 0x7feed80069d0 2023-08-09T13:50:12.969 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:12.963+0000 7feef1f87640 1 --2- 172.21.15.43:0/1754652759 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7feed80073e0 0x7feed800db90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-08-09T13:50:12.969 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:12.964+0000 7feef0f85640 1 --2- 172.21.15.43:0/1754652759 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7feed80069d0 0x7feed8006e50 secure :-1 s=READY pgs=943 cs=0 l=1 rev1=1 crypto rx=0x7feedc00c0e0 tx=0x7feed8007a00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7ba678b9ede9a67 server_cookie=0 in_seq=0 out_seq=0 2023-08-09T13:50:12.970 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:12.964+0000 7feee3fff640 1 -- 172.21.15.43:0/1754652759 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feedc005ad0 con 0x7feed80069d0 2023-08-09T13:50:12.970 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:12.965+0000 7feee3fff640 1 -- 172.21.15.43:0/1754652759 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7feedc005c30 con 0x7feed80069d0 2023-08-09T13:50:12.970 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:12.965+0000 7feee3fff640 1 -- 172.21.15.43:0/1754652759 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feedc02ca50 con 0x7feed80069d0 2023-08-09T13:50:12.970 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:12.965+0000 7feef39ea640 1 -- 172.21.15.43:0/1754652759 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7feed80069d0 msgr2=0x7feed8006e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:50:12.970 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:12.965+0000 7feef39ea640 1 --2- 172.21.15.43:0/1754652759 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7feed80069d0 0x7feed8006e50 secure :-1 s=READY pgs=943 cs=0 l=1 rev1=1 crypto rx=0x7feedc00c0e0 tx=0x7feed8007a00 comp rx=0 tx=0).stop 2023-08-09T13:50:12.970 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:12.965+0000 7feef1f87640 1 -- 172.21.15.43:0/1754652759 reap_dead start 2023-08-09T13:50:12.970 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:12.965+0000 7feef39ea640 1 -- 172.21.15.43:0/1754652759 shutdown_connections 2023-08-09T13:50:12.970 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:12.965+0000 7feef39ea640 1 -- 172.21.15.43:0/1754652759 >> 172.21.15.43:0/1754652759 conn(0x7feeec0ce0a0 msgr2=0x7feeec0ce480 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-09T13:50:12.971 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:12.965+0000 7feef39ea640 1 -- 172.21.15.43:0/1754652759 shutdown_connections 2023-08-09T13:50:12.971 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:12.965+0000 7feef39ea640 1 -- 172.21.15.43:0/1754652759 wait complete. 2023-08-09T13:50:12.972 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:12.967+0000 7feef39ea640 1 Processor -- start 2023-08-09T13:50:12.972 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:12.967+0000 7feef39ea640 1 -- start start 2023-08-09T13:50:12.972 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:12.968+0000 7feef39ea640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7feeec158b50 0x7feeec0b3830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:50:12.972 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:12.968+0000 7feef39ea640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7feeec0c7750 0x7feeec0c7bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:50:12.973 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:12.968+0000 7feef39ea640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7feeec13db60 0x7feeec13dfe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:50:12.973 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:12.968+0000 7feef39ea640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feeec13e520 con 0x7feeec0c7750 2023-08-09T13:50:12.973 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:12.968+0000 7feef39ea640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feeec0b24f0 con 0x7feeec13db60 2023-08-09T13:50:12.973 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:12.968+0000 7feef39ea640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7feeec1f1d80 con 0x7feeec158b50 2023-08-09T13:50:12.973 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:12.969+0000 7feef0f85640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7feeec0c7750 0x7feeec0c7bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:50:12.973 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:12.969+0000 7feef0f85640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7feeec0c7750 0x7feeec0c7bd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:45500/0 (socket says 172.21.15.43:45500) 2023-08-09T13:50:12.973 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:12.969+0000 7feef0f85640 1 -- 172.21.15.43:0/3706824103 learned_addr learned my addr 172.21.15.43:0/3706824103 (peer_addr_for_me v2:172.21.15.43:0/0) 2023-08-09T13:50:12.974 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:12.969+0000 7feef1786640 1 --2- 172.21.15.43:0/3706824103 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7feeec158b50 0x7feeec0b3830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:50:12.974 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:12.969+0000 7feef1f87640 1 --2- 172.21.15.43:0/3706824103 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7feeec13db60 0x7feeec13dfe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:50:12.974 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:12.969+0000 7feef0f85640 1 -- 172.21.15.43:0/3706824103 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7feeec158b50 msgr2=0x7feeec0b3830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:50:12.974 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:12.969+0000 7feef0f85640 1 --2- 172.21.15.43:0/3706824103 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7feeec158b50 0x7feeec0b3830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:50:12.974 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:12.969+0000 7feef0f85640 1 -- 172.21.15.43:0/3706824103 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7feeec13db60 msgr2=0x7feeec13dfe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:50:12.974 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:12.969+0000 7feef0f85640 1 --2- 172.21.15.43:0/3706824103 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7feeec13db60 0x7feeec13dfe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:50:12.974 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:12.969+0000 7feef0f85640 1 -- 172.21.15.43:0/3706824103 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7feeec0b2790 con 0x7feeec0c7750 2023-08-09T13:50:12.974 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:12.970+0000 7feef0f85640 1 --2- 172.21.15.43:0/3706824103 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7feeec0c7750 0x7feeec0c7bd0 secure :-1 s=READY pgs=944 cs=0 l=1 rev1=1 crypto rx=0x7feedc00c200 tx=0x7feedc02e070 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-09T13:50:12.975 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:12.970+0000 7feee1ffb640 1 -- 172.21.15.43:0/3706824103 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feedc03c040 con 0x7feeec0c7750 2023-08-09T13:50:12.975 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:12.970+0000 7feee1ffb640 1 -- 172.21.15.43:0/3706824103 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7feedc004aa0 con 0x7feeec0c7750 2023-08-09T13:50:12.975 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:12.970+0000 7feee1ffb640 1 -- 172.21.15.43:0/3706824103 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feedc004e30 con 0x7feeec0c7750 2023-08-09T13:50:12.976 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:12.970+0000 7feef39ea640 1 -- 172.21.15.43:0/3706824103 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7feeec0b2a70 con 0x7feeec0c7750 2023-08-09T13:50:12.976 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:12.970+0000 7feef39ea640 1 -- 172.21.15.43:0/3706824103 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7feeec0b3030 con 0x7feeec0c7750 2023-08-09T13:50:12.976 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:12.972+0000 7feef39ea640 1 -- 172.21.15.43:0/3706824103 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7feeec0c96a0 con 0x7feeec0c7750 2023-08-09T13:50:12.976 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:12.973+0000 7feee1ffb640 1 -- 172.21.15.43:0/3706824103 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 28) v1 ==== 88387+0+0 (secure 0 0 0) 0x7feedc032070 con 0x7feeec0c7750 2023-08-09T13:50:12.977 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:12.973+0000 7feee1ffb640 1 --2- 172.21.15.43:0/3706824103 >> [v2:172.21.15.203:6832/3325141227,v1:172.21.15.203:6833/3325141227] conn(0x7feed006a110 0x7feed006c5d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:50:12.977 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:12.974+0000 7feee1ffb640 1 -- 172.21.15.43:0/3706824103 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(78..78 src has 1..78) v4 ==== 7228+0+0 (secure 0 0 0) 0x7feedc0a9da0 con 0x7feeec0c7750 2023-08-09T13:50:12.977 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:12.974+0000 7feef1786640 1 --2- 172.21.15.43:0/3706824103 >> [v2:172.21.15.203:6832/3325141227,v1:172.21.15.203:6833/3325141227] conn(0x7feed006a110 0x7feed006c5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:50:12.977 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:12.975+0000 7feef1786640 1 --2- 172.21.15.43:0/3706824103 >> [v2:172.21.15.203:6832/3325141227,v1:172.21.15.203:6833/3325141227] conn(0x7feed006a110 0x7feed006c5d0 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0x7feeec0c5120 tx=0x7feee800b040 comp rx=0 tx=0).ready entity=mgr.5519 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-09T13:50:12.980 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:12.978+0000 7feee1ffb640 1 -- 172.21.15.43:0/3706824103 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178066 (secure 0 0 0) 0x7feedc074f10 con 0x7feeec0c7750 2023-08-09T13:50:13.110 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:13.105+0000 7feee1ffb640 1 -- 172.21.15.43:0/3706824103 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mgrmap(e 29) v1 ==== 88443+0+0 (secure 0 0 0) 0x7feedc0752e0 con 0x7feeec0c7750 2023-08-09T13:50:13.179 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:13.176+0000 7feef39ea640 1 -- 172.21.15.43:0/3706824103 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7feeec0c96a0 con 0x7feeec0c7750 2023-08-09T13:50:13.182 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:13.179+0000 7feee1ffb640 1 -- 172.21.15.43:0/3706824103 <== mon.0 v2:172.21.15.43:3300/0 8 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v78) v1 ==== 74+0+19757 (secure 0 0 0) 0x7feedc0789a0 con 0x7feeec0c7750 2023-08-09T13:50:13.183 INFO:teuthology.orchestra.run.smithi043.stdout: 2023-08-09T13:50:13.183 INFO:teuthology.orchestra.run.smithi043.stdout:{"epoch":78,"fsid":"daac87ee-1337-4250-a955-aea4e408a39b","created":"2023-08-09T13:33:41.701198+0000","modified":"2023-08-09T13:50:12.073550+0000","last_up_change":"2023-08-09T13:33:53.783453+0000","last_in_change":"2023-08-09T13:33:45.148904+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":9,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":"rbd","create_time":"2023-08-09T13:33:54.365137+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":".mgr","create_time":"2023-08-09T13:33:54.831803+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":8,"pool_name":"cephfs_metadata","create_time":"2023-08-09T13:48:16.039756+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"73","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":9,"pool_name":"cephfs_data","create_time":"2023-08-09T13:48:16.615400+0000","flags":8193,"flags_names":"hashpspool,selfmanaged_snaps","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"77","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":5,"snap_epoch":77,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.6299999952316284,"score_stable":1.6299999952316284,"optimal_score":1,"raw_score_acting":1.6299999952316284,"raw_score_stable":1.6299999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"cbd49ecd-e896-4fbd-864e-5c37a829b165","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":72,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6800","nonce":502667335},{"type":"v1","addr":"172.21.15.43:6801","nonce":502667335}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6802","nonce":502667335},{"type":"v1","addr":"172.21.15.43:6803","nonce":502667335}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6806","nonce":502667335},{"type":"v1","addr":"172.21.15.43:6807","nonce":502667335}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6804","nonce":502667335},{"type":"v1","addr":"172.21.15.43:6805","nonce":502667335}]},"public_addr":"172.21.15.43:6801/502667335","cluster_addr":"172.21.15.43:6803/502667335","heartbeat_back_addr":"172.21.15.43:6807/502667335","heartbeat_front_addr":"172.21.15.43:6805/502667335","state":["exists","up"]},{"osd":1,"uuid":"75c7e0fe-1ec7-4fe8-a2f2-3661a65b8500","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":72,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6808","nonce":3289134610},{"type":"v1","addr":"172.21.15.43:6809","nonce":3289134610}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6810","nonce":3289134610},{"type":"v1","addr":"172.21.15.43:6812","nonce":3289134610}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6821","nonce":3289134610},{"type":"v1","addr":"172.21.15.43:6824","nonce":3289134610}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6815","nonce":3289134610},{"type":"v1","addr":"172.21.15.43:6818","nonce":3289134610}]},"public_addr":"172.21.15.43:6809/3289134610","cluster_addr":"172.21.15.43:6812/3289134610","heartbeat_back_addr":"172.21.15.43:6824/3289134610","heartbeat_front_addr":"172.21.15.43:6818/3289134610","state":["exists","up"]},{"osd":2,"uuid":"bb5c41e9-7f5f-4125-9b01-3bb8bfc2ea83","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":72,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6811","nonce":4079709736},{"type":"v1","addr":"172.21.15.43:6814","nonce":4079709736}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6817","nonce":4079709736},{"type":"v1","addr":"172.21.15.43:6820","nonce":4079709736}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6829","nonce":4079709736},{"type":"v1","addr":"172.21.15.43:6831","nonce":4079709736}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6823","nonce":4079709736},{"type":"v1","addr":"172.21.15.43:6826","nonce":4079709736}]},"public_addr":"172.21.15.43:6814/4079709736","cluster_addr":"172.21.15.43:6820/4079709736","heartbeat_back_addr":"172.21.15.43:6831/4079709736","heartbeat_front_addr":"172.21.15.43:6826/4079709736","state":["exists","up"]},{"osd":3,"uuid":"bfe4c8b4-694a-465c-87e6-2b2aab852613","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":72,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6813","nonce":1709538759},{"type":"v1","addr":"172.21.15.43:6816","nonce":1709538759}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6819","nonce":1709538759},{"type":"v1","addr":"172.21.15.43:6822","nonce":1709538759}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6828","nonce":1709538759},{"type":"v1","addr":"172.21.15.43:6830","nonce":1709538759}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6825","nonce":1709538759},{"type":"v1","addr":"172.21.15.43:6827","nonce":1709538759}]},"public_addr":"172.21.15.43:6816/1709538759","cluster_addr":"172.21.15.43:6822/1709538759","heartbeat_back_addr":"172.21.15.43:6830/1709538759","heartbeat_front_addr":"172.21.15.43:6827/1709538759","state":["exists","up"]},{"osd":4,"uuid":"fa2e8a77-aba1-4756-853e-0d174a5bdeba","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":72,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6800","nonce":4196866481},{"type":"v1","addr":"172.21.15.203:6801","nonce":4196866481}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6802","nonce":4196866481},{"type":"v1","addr":"172.21.15.203:6803","nonce":4196866481}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6806","nonce":4196866481},{"type":"v1","addr":"172.21.15.203:6807","nonce":4196866481}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6804","nonce":4196866481},{"type":"v1","addr":"172.21.15.203:6805","nonce":4196866481}]},"public_addr":"172.21.15.203:6801/4196866481","cluster_addr":"172.21.15.203:6803/4196866481","heartbeat_back_addr":"172.21.15.203:6807/4196866481","heartbeat_front_addr":"172.21.15.203:6805/4196866481","state":["exists","up"]},{"osd":5,"uuid":"81811e4b-4d24-4f51-a82c-f1d700a2a00d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":72,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6816","nonce":3059290081},{"type":"v1","addr":"172.21.15.203:6817","nonce":3059290081}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6818","nonce":3059290081},{"type":"v1","addr":"172.21.15.203:6819","nonce":3059290081}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6822","nonce":3059290081},{"type":"v1","addr":"172.21.15.203:6823","nonce":3059290081}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6820","nonce":3059290081},{"type":"v1","addr":"172.21.15.203:6821","nonce":3059290081}]},"public_addr":"172.21.15.203:6817/3059290081","cluster_addr":"172.21.15.203:6819/3059290081","heartbeat_back_addr":"172.21.15.203:6823/3059290081","heartbeat_front_addr":"172.21.15.203:6821/3059290081","state":["exists","up"]},{"osd":6,"uuid":"307bac4d-6a5b-443d-af75-56d695d7674f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":72,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6824","nonce":512155569},{"type":"v1","addr":"172.21.15.203:6825","nonce":512155569}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6826","nonce":512155569},{"type":"v1","addr":"172.21.15.203:6827","nonce":512155569}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6830","nonce":512155569},{"type":"v1","addr":"172.21.15.203:6831","nonce":512155569}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6828","nonce":512155569},{"type":"v1","addr":"172.21.15.203:6829","nonce":512155569}]},"public_addr":"172.21.15.203:6825/512155569","cluster_addr":"172.21.15.203:6827/512155569","heartbeat_back_addr":"172.21.15.203:6831/512155569","heartbeat_front_addr":"172.21.15.203:6829/512155569","state":["exists","up"]},{"osd":7,"uuid":"a95dc7ca-9f1b-4295-af70-68a22a2d7d5e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":72,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6808","nonce":3331450438},{"type":"v1","addr":"172.21.15.203:6809","nonce":3331450438}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6810","nonce":3331450438},{"type":"v1","addr":"172.21.15.203:6811","nonce":3331450438}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6814","nonce":3331450438},{"type":"v1","addr":"172.21.15.203:6815","nonce":3331450438}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6812","nonce":3331450438},{"type":"v1","addr":"172.21.15.203:6813","nonce":3331450438}]},"public_addr":"172.21.15.203:6809/3331450438","cluster_addr":"172.21.15.203:6811/3331450438","heartbeat_back_addr":"172.21.15.203:6815/3331450438","heartbeat_front_addr":"172.21.15.203:6813/3331450438","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-09T13:33:50.547652+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-09T13:33:50.589469+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-09T13:33:50.561724+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-09T13:33:50.599650+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-09T13:33:50.411855+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-09T13:33:50.219681+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-09T13:33:50.504664+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-09T13:33:50.441248+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.203:0/2251174608":"2023-08-10T13:50:08.172778+0000","172.21.15.203:0/4008902326":"2023-08-10T13:50:08.172778+0000","172.21.15.203:0/1825829254":"2023-08-10T13:50:08.172778+0000","172.21.15.203:0/4097180925":"2023-08-10T13:50:08.172778+0000","172.21.15.203:6832/2515481191":"2023-08-10T13:50:08.172778+0000","172.21.15.203:0/4038087778":"2023-08-10T13:50:08.172778+0000","172.21.15.203:0/826156596":"2023-08-10T13:50:08.172778+0000","172.21.15.203:0/1084459051":"2023-08-10T13:50:08.172778+0000","172.21.15.203:6833/272917756":"2023-08-10T13:48:34.065493+0000","172.21.15.203:6832/272917756":"2023-08-10T13:48:34.065493+0000","172.21.15.203:0/665971116":"2023-08-10T13:48:34.065493+0000","172.21.15.203:0/2868823959":"2023-08-10T13:48:34.065493+0000","172.21.15.203:6833/2515481191":"2023-08-10T13:50:08.172778+0000","172.21.15.203:0/3168500363":"2023-08-10T13:48:34.065493+0000","172.21.15.203:0/752327795":"2023-08-10T13:48:34.065493+0000","172.21.15.203:0/1970263661":"2023-08-10T13:48:34.065493+0000","172.21.15.203:0/2726495528":"2023-08-10T13:50:08.172778+0000","172.21.15.203:0/1407921417":"2023-08-10T13:48:34.065493+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":7,"snaps":[{"begin":104,"length":1},{"begin":221,"length":1}]}],"new_removed_snaps":[],"new_purged_snaps":[{"pool":9,"snaps":[{"begin":2,"length":4}]}],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-08-09T13:50:13.185 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:13.182+0000 7feef39ea640 1 -- 172.21.15.43:0/3706824103 >> [v2:172.21.15.203:6832/3325141227,v1:172.21.15.203:6833/3325141227] conn(0x7feed006a110 msgr2=0x7feed006c5d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:50:13.185 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:13.182+0000 7feef39ea640 1 --2- 172.21.15.43:0/3706824103 >> [v2:172.21.15.203:6832/3325141227,v1:172.21.15.203:6833/3325141227] conn(0x7feed006a110 0x7feed006c5d0 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0x7feeec0c5120 tx=0x7feee800b040 comp rx=0 tx=0).stop 2023-08-09T13:50:13.185 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:13.182+0000 7feef39ea640 1 -- 172.21.15.43:0/3706824103 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7feeec0c7750 msgr2=0x7feeec0c7bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:50:13.186 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:13.182+0000 7feef39ea640 1 --2- 172.21.15.43:0/3706824103 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7feeec0c7750 0x7feeec0c7bd0 secure :-1 s=READY pgs=944 cs=0 l=1 rev1=1 crypto rx=0x7feedc00c200 tx=0x7feedc02e070 comp rx=0 tx=0).stop 2023-08-09T13:50:13.186 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:13.182+0000 7feef39ea640 1 -- 172.21.15.43:0/3706824103 shutdown_connections 2023-08-09T13:50:13.186 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:13.182+0000 7feef39ea640 1 --2- 172.21.15.43:0/3706824103 >> [v2:172.21.15.203:6832/3325141227,v1:172.21.15.203:6833/3325141227] conn(0x7feed006a110 0x7feed006c5d0 unknown :-1 s=CLOSED pgs=30 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:50:13.186 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:13.182+0000 7feef39ea640 1 --2- 172.21.15.43:0/3706824103 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7feeec13db60 0x7feeec13dfe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:50:13.186 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:13.182+0000 7feef39ea640 1 --2- 172.21.15.43:0/3706824103 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7feeec0c7750 0x7feeec0c7bd0 unknown :-1 s=CLOSED pgs=944 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:50:13.186 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:13.182+0000 7feef39ea640 1 --2- 172.21.15.43:0/3706824103 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7feeec158b50 0x7feeec0b3830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:50:13.186 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:13.182+0000 7feef39ea640 1 -- 172.21.15.43:0/3706824103 >> 172.21.15.43:0/3706824103 conn(0x7feeec0ce0a0 msgr2=0x7feeec0c4240 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-09T13:50:13.187 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:13.183+0000 7feef39ea640 1 -- 172.21.15.43:0/3706824103 shutdown_connections 2023-08-09T13:50:13.187 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:13.183+0000 7feef39ea640 1 -- 172.21.15.43:0/3706824103 wait complete. 2023-08-09T13:50:13.205 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2023-08-09T13:50:13.424 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:13.421+0000 7f8f5a142640 1 Processor -- start 2023-08-09T13:50:13.425 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:13.421+0000 7f8f5a142640 1 -- start start 2023-08-09T13:50:13.427 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:13.422+0000 7f8f5a142640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f8f5415c680 0x7f8f540ce760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:50:13.427 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:13.422+0000 7f8f5a142640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f8f540b3fa0 0x7f8f540b43d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:50:13.428 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:13.422+0000 7f8f5a142640 1 -- --> v1:172.21.15.203:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8f540b4910 con 0x7f8f540d0a70 2023-08-09T13:50:13.428 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:13.422+0000 7f8f5a142640 1 -- --> v2:172.21.15.43:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8f540cdf00 con 0x7f8f540b3fa0 2023-08-09T13:50:13.428 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:13.422+0000 7f8f5a142640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8f540d0f70 con 0x7f8f5415c680 2023-08-09T13:50:13.429 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:13.423+0000 7f8f53fff640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f8f540b3fa0 0x7f8f540b43d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:50:13.429 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:13.423+0000 7f8f53fff640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f8f540b3fa0 0x7f8f540b43d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:45514/0 (socket says 172.21.15.43:45514) 2023-08-09T13:50:13.429 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:13.423+0000 7f8f53fff640 1 -- 172.21.15.43:0/904307862 learned_addr learned my addr 172.21.15.43:0/904307862 (peer_addr_for_me v2:172.21.15.43:0/0) 2023-08-09T13:50:13.429 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:13.423+0000 7f8f537fe640 1 --2- 172.21.15.43:0/904307862 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f8f5415c680 0x7f8f540ce760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:50:13.430 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:13.423+0000 7f8f53fff640 1 -- 172.21.15.43:0/904307862 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f8f5415c680 msgr2=0x7f8f540ce760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-09T13:50:13.430 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:13.423+0000 7f8f53fff640 1 --2- 172.21.15.43:0/904307862 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f8f5415c680 0x7f8f540ce760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:50:13.430 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:13.423+0000 7f8f53fff640 1 -- 172.21.15.43:0/904307862 >> v1:172.21.15.203:6789/0 conn(0x7f8f540d0a70 legacy=0x7f8f540cd7f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-09T13:50:13.430 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:13.423+0000 7f8f53fff640 1 -- 172.21.15.43:0/904307862 --> v2:172.21.15.43:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8f540ccd10 con 0x7f8f540b3fa0 2023-08-09T13:50:13.431 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:13.424+0000 7f8f53fff640 1 --2- 172.21.15.43:0/904307862 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f8f540b3fa0 0x7f8f540b43d0 secure :-1 s=READY pgs=945 cs=0 l=1 rev1=1 crypto rx=0x7f8f4c009870 tx=0x7f8f4c02b8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f95564e04551dc49 server_cookie=0 in_seq=0 out_seq=0 2023-08-09T13:50:13.431 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:13.425+0000 7f8f527fc640 1 -- 172.21.15.43:0/904307862 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8f4c031640 con 0x7f8f540b3fa0 2023-08-09T13:50:13.431 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:13.425+0000 7f8f527fc640 1 -- 172.21.15.43:0/904307862 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8f4c0317a0 con 0x7f8f540b3fa0 2023-08-09T13:50:13.431 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:13.425+0000 7f8f527fc640 1 -- 172.21.15.43:0/904307862 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8f4c031b30 con 0x7f8f540b3fa0 2023-08-09T13:50:13.435 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:13.425+0000 7f8f5a142640 1 -- 172.21.15.43:0/904307862 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f8f540b3fa0 msgr2=0x7f8f540b43d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:50:13.435 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:13.425+0000 7f8f5a142640 1 --2- 172.21.15.43:0/904307862 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f8f540b3fa0 0x7f8f540b43d0 secure :-1 s=READY pgs=945 cs=0 l=1 rev1=1 crypto rx=0x7f8f4c009870 tx=0x7f8f4c02b8f0 comp rx=0 tx=0).stop 2023-08-09T13:50:13.435 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:13.425+0000 7f8f5a142640 1 -- 172.21.15.43:0/904307862 shutdown_connections 2023-08-09T13:50:13.436 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:13.425+0000 7f8f5a142640 1 --2- 172.21.15.43:0/904307862 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f8f540b3fa0 0x7f8f540b43d0 unknown :-1 s=CLOSED pgs=945 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:50:13.436 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:13.425+0000 7f8f5a142640 1 --2- 172.21.15.43:0/904307862 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f8f5415c680 0x7f8f540ce760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:50:13.436 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:13.425+0000 7f8f5a142640 1 -- 172.21.15.43:0/904307862 >> 172.21.15.43:0/904307862 conn(0x7f8f54071520 msgr2=0x7f8f540d2520 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-09T13:50:13.436 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:13.426+0000 7f8f5a142640 1 -- 172.21.15.43:0/904307862 shutdown_connections 2023-08-09T13:50:13.437 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:13.426+0000 7f8f5a142640 1 -- 172.21.15.43:0/904307862 wait complete. 2023-08-09T13:50:13.437 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:13.427+0000 7f8f5a142640 1 Processor -- start 2023-08-09T13:50:13.437 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:13.427+0000 7f8f5a142640 1 -- start start 2023-08-09T13:50:13.438 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:13.428+0000 7f8f5a142640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f8f540b3fa0 0x7f8f54154fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:50:13.438 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:13.428+0000 7f8f5a142640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f8f540d0a70 0x7f8f540c8af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:50:13.438 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:13.428+0000 7f8f5a142640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f8f5415c680 0x7f8f54152220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:50:13.438 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:13.428+0000 7f8f5a142640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8f541f28c0 con 0x7f8f540b3fa0 2023-08-09T13:50:13.439 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:13.428+0000 7f8f5a142640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8f5415db70 con 0x7f8f5415c680 2023-08-09T13:50:13.439 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:13.428+0000 7f8f5a142640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8f540c9030 con 0x7f8f540d0a70 2023-08-09T13:50:13.439 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:13.429+0000 7f8f537fe640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f8f540b3fa0 0x7f8f54154fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:50:13.439 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:13.429+0000 7f8f537fe640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f8f540b3fa0 0x7f8f54154fb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:45528/0 (socket says 172.21.15.43:45528) 2023-08-09T13:50:13.440 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:13.429+0000 7f8f537fe640 1 -- 172.21.15.43:0/3182293907 learned_addr learned my addr 172.21.15.43:0/3182293907 (peer_addr_for_me v2:172.21.15.43:0/0) 2023-08-09T13:50:13.440 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:13.429+0000 7f8f53fff640 1 --2- 172.21.15.43:0/3182293907 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f8f5415c680 0x7f8f54152220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:50:13.440 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:13.429+0000 7f8f52ffd640 1 --2- 172.21.15.43:0/3182293907 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f8f540d0a70 0x7f8f540c8af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:50:13.440 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:13.429+0000 7f8f537fe640 1 -- 172.21.15.43:0/3182293907 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f8f540d0a70 msgr2=0x7f8f540c8af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:50:13.441 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:13.429+0000 7f8f537fe640 1 --2- 172.21.15.43:0/3182293907 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f8f540d0a70 0x7f8f540c8af0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:50:13.441 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:13.429+0000 7f8f537fe640 1 -- 172.21.15.43:0/3182293907 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f8f5415c680 msgr2=0x7f8f54152220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:50:13.441 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:13.429+0000 7f8f537fe640 1 --2- 172.21.15.43:0/3182293907 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f8f5415c680 0x7f8f54152220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:50:13.442 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:13.429+0000 7f8f537fe640 1 -- 172.21.15.43:0/3182293907 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8f4c0094d0 con 0x7f8f540b3fa0 2023-08-09T13:50:13.442 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:13.429+0000 7f8f537fe640 1 --2- 172.21.15.43:0/3182293907 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f8f540b3fa0 0x7f8f54154fb0 secure :-1 s=READY pgs=946 cs=0 l=1 rev1=1 crypto rx=0x7f8f4400ccd0 tx=0x7f8f4400a5c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-09T13:50:13.442 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:13.430+0000 7f8f3bfff640 1 -- 172.21.15.43:0/3182293907 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8f44003990 con 0x7f8f540b3fa0 2023-08-09T13:50:13.442 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:13.430+0000 7f8f3bfff640 1 -- 172.21.15.43:0/3182293907 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8f44003af0 con 0x7f8f540b3fa0 2023-08-09T13:50:13.443 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:13.430+0000 7f8f3bfff640 1 -- 172.21.15.43:0/3182293907 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8f44004020 con 0x7f8f540b3fa0 2023-08-09T13:50:13.443 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:13.430+0000 7f8f5a142640 1 -- 172.21.15.43:0/3182293907 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8f540c92d0 con 0x7f8f540b3fa0 2023-08-09T13:50:13.443 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:13.430+0000 7f8f5a142640 1 -- 172.21.15.43:0/3182293907 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8f541f24e0 con 0x7f8f540b3fa0 2023-08-09T13:50:13.443 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:13.431+0000 7f8f5a142640 1 -- 172.21.15.43:0/3182293907 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8f540739f0 con 0x7f8f540b3fa0 2023-08-09T13:50:13.444 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:13.432+0000 7f8f3bfff640 1 -- 172.21.15.43:0/3182293907 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 29) v1 ==== 88443+0+0 (secure 0 0 0) 0x7f8f44004180 con 0x7f8f540b3fa0 2023-08-09T13:50:13.446 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:13.433+0000 7f8f3bfff640 1 --2- 172.21.15.43:0/3182293907 >> [v2:172.21.15.203:6832/3325141227,v1:172.21.15.203:6833/3325141227] conn(0x7f8f4006a170 0x7f8f4006c630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:50:13.446 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:13.433+0000 7f8f3bfff640 1 -- 172.21.15.43:0/3182293907 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(78..78 src has 1..78) v4 ==== 7228+0+0 (secure 0 0 0) 0x7f8f44089110 con 0x7f8f540b3fa0 2023-08-09T13:50:13.446 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:13.434+0000 7f8f52ffd640 1 --2- 172.21.15.43:0/3182293907 >> [v2:172.21.15.203:6832/3325141227,v1:172.21.15.203:6833/3325141227] conn(0x7f8f4006a170 0x7f8f4006c630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:50:13.447 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:13.435+0000 7f8f52ffd640 1 --2- 172.21.15.43:0/3182293907 >> [v2:172.21.15.203:6832/3325141227,v1:172.21.15.203:6833/3325141227] conn(0x7f8f4006a170 0x7f8f4006c630 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0x7f8f5415db20 tx=0x7f8f48009040 comp rx=0 tx=0).ready entity=mgr.5519 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-09T13:50:13.447 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:13.437+0000 7f8f3bfff640 1 -- 172.21.15.43:0/3182293907 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178066 (secure 0 0 0) 0x7f8f440540b0 con 0x7f8f540b3fa0 2023-08-09T13:50:13.672 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:13.670+0000 7f8f5a142640 1 -- 172.21.15.43:0/3182293907 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) v1 -- 0x7f8f5415fe60 con 0x7f8f540b3fa0 2023-08-09T13:50:13.673 INFO:tasks.ceph.mon.a.smithi043.stderr:2023-08-09T13:50:13.671+0000 7f32867f3640 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2023-08-09T13:50:13.683 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:13.680+0000 7f8f3bfff640 1 -- 172.21.15.43:0/3182293907 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v27) v1 ==== 161+0+0 (secure 0 0 0) 0x7f8f440576d0 con 0x7f8f540b3fa0 2023-08-09T13:50:13.683 INFO:teuthology.orchestra.run.smithi043.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2023-08-09T13:50:13.686 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:13.683+0000 7f8f5a142640 1 -- 172.21.15.43:0/3182293907 >> [v2:172.21.15.203:6832/3325141227,v1:172.21.15.203:6833/3325141227] conn(0x7f8f4006a170 msgr2=0x7f8f4006c630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:50:13.686 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:13.683+0000 7f8f5a142640 1 --2- 172.21.15.43:0/3182293907 >> [v2:172.21.15.203:6832/3325141227,v1:172.21.15.203:6833/3325141227] conn(0x7f8f4006a170 0x7f8f4006c630 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0x7f8f5415db20 tx=0x7f8f48009040 comp rx=0 tx=0).stop 2023-08-09T13:50:13.687 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:13.683+0000 7f8f5a142640 1 -- 172.21.15.43:0/3182293907 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f8f540b3fa0 msgr2=0x7f8f54154fb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:50:13.687 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:13.683+0000 7f8f5a142640 1 --2- 172.21.15.43:0/3182293907 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f8f540b3fa0 0x7f8f54154fb0 secure :-1 s=READY pgs=946 cs=0 l=1 rev1=1 crypto rx=0x7f8f4400ccd0 tx=0x7f8f4400a5c0 comp rx=0 tx=0).stop 2023-08-09T13:50:13.687 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:13.684+0000 7f8f5a142640 1 -- 172.21.15.43:0/3182293907 shutdown_connections 2023-08-09T13:50:13.687 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:13.684+0000 7f8f5a142640 1 --2- 172.21.15.43:0/3182293907 >> [v2:172.21.15.203:6832/3325141227,v1:172.21.15.203:6833/3325141227] conn(0x7f8f4006a170 0x7f8f4006c630 unknown :-1 s=CLOSED pgs=31 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:50:13.687 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:13.684+0000 7f8f5a142640 1 --2- 172.21.15.43:0/3182293907 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f8f5415c680 0x7f8f54152220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:50:13.687 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:13.684+0000 7f8f5a142640 1 --2- 172.21.15.43:0/3182293907 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f8f540d0a70 0x7f8f540c8af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:50:13.687 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:13.684+0000 7f8f5a142640 1 --2- 172.21.15.43:0/3182293907 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f8f540b3fa0 0x7f8f54154fb0 unknown :-1 s=CLOSED pgs=946 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:50:13.688 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:13.684+0000 7f8f5a142640 1 -- 172.21.15.43:0/3182293907 >> 172.21.15.43:0/3182293907 conn(0x7f8f54071520 msgr2=0x7f8f540ce040 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-09T13:50:13.688 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:13.684+0000 7f8f5a142640 1 -- 172.21.15.43:0/3182293907 shutdown_connections 2023-08-09T13:50:13.688 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:13.684+0000 7f8f5a142640 1 -- 172.21.15.43:0/3182293907 wait complete. 2023-08-09T13:50:13.705 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs rm cephfs --yes-i-really-mean-it 2023-08-09T13:50:13.815 INFO:tasks.ceph.mds.b.smithi203.stdout:starting mds.b at 2023-08-09T13:50:13.918 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:13.916+0000 7f8983869640 1 Processor -- start 2023-08-09T13:50:13.919 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:13.916+0000 7f8983869640 1 -- start start 2023-08-09T13:50:13.920 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:13.917+0000 7f8983869640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f897c15c550 0x7f897c0cd6d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:50:13.921 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:13.917+0000 7f8983869640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f897c0b45f0 0x7f897c0b59d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:50:13.921 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:13.917+0000 7f8983869640 1 -- --> v1:172.21.15.203:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f897c0b7bd0 con 0x7f897c0d04b0 2023-08-09T13:50:13.921 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:13.918+0000 7f8983869640 1 -- --> v2:172.21.15.43:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f897c0d1c00 con 0x7f897c15c550 2023-08-09T13:50:13.921 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:13.918+0000 7f8983869640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f897c15c930 con 0x7f897c0b45f0 2023-08-09T13:50:13.922 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:13.918+0000 7f8981605640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f897c15c550 0x7f897c0cd6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:50:13.922 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:13.918+0000 7f8981605640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f897c15c550 0x7f897c0cd6d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:45544/0 (socket says 172.21.15.43:45544) 2023-08-09T13:50:13.922 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:13.918+0000 7f8981605640 1 -- 172.21.15.43:0/4015284336 learned_addr learned my addr 172.21.15.43:0/4015284336 (peer_addr_for_me v2:172.21.15.43:0/0) 2023-08-09T13:50:13.922 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:13.918+0000 7f8981e06640 1 --2- 172.21.15.43:0/4015284336 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f897c0b45f0 0x7f897c0b59d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:50:13.923 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:13.918+0000 7f8981605640 1 -- 172.21.15.43:0/4015284336 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f897c0b45f0 msgr2=0x7f897c0b59d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-09T13:50:13.924 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:13.918+0000 7f8981605640 1 --2- 172.21.15.43:0/4015284336 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f897c0b45f0 0x7f897c0b59d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:50:13.924 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:13.918+0000 7f8981605640 1 -- 172.21.15.43:0/4015284336 >> v1:172.21.15.203:6789/0 conn(0x7f897c0d04b0 legacy=0x7f897c0b3ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-09T13:50:13.924 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:13.919+0000 7f8981605640 1 -- 172.21.15.43:0/4015284336 --> v2:172.21.15.43:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f897c0b54f0 con 0x7f897c15c550 2023-08-09T13:50:13.924 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:13.920+0000 7f8981605640 1 --2- 172.21.15.43:0/4015284336 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f897c15c550 0x7f897c0cd6d0 secure :-1 s=READY pgs=947 cs=0 l=1 rev1=1 crypto rx=0x7f8978009870 tx=0x7f897802b860 comp rx=0 tx=0).ready entity=mon.0 client_cookie=df472469f2363a28 server_cookie=0 in_seq=0 out_seq=0 2023-08-09T13:50:13.924 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:13.920+0000 7f8973fff640 1 -- 172.21.15.43:0/4015284336 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f89780317c0 con 0x7f897c15c550 2023-08-09T13:50:13.924 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:13.920+0000 7f8973fff640 1 -- 172.21.15.43:0/4015284336 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8978031920 con 0x7f897c15c550 2023-08-09T13:50:13.925 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:13.920+0000 7f8973fff640 1 -- 172.21.15.43:0/4015284336 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8978031cb0 con 0x7f897c15c550 2023-08-09T13:50:13.925 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:13.921+0000 7f8983869640 1 -- 172.21.15.43:0/4015284336 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f897c15c550 msgr2=0x7f897c0cd6d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:50:13.925 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:13.921+0000 7f8983869640 1 --2- 172.21.15.43:0/4015284336 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f897c15c550 0x7f897c0cd6d0 secure :-1 s=READY pgs=947 cs=0 l=1 rev1=1 crypto rx=0x7f8978009870 tx=0x7f897802b860 comp rx=0 tx=0).stop 2023-08-09T13:50:13.925 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:13.921+0000 7f8983869640 1 -- 172.21.15.43:0/4015284336 shutdown_connections 2023-08-09T13:50:13.925 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:13.921+0000 7f8983869640 1 --2- 172.21.15.43:0/4015284336 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f897c0b45f0 0x7f897c0b59d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:50:13.925 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:13.921+0000 7f8983869640 1 --2- 172.21.15.43:0/4015284336 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f897c15c550 0x7f897c0cd6d0 unknown :-1 s=CLOSED pgs=947 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:50:13.925 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:13.921+0000 7f8983869640 1 -- 172.21.15.43:0/4015284336 >> 172.21.15.43:0/4015284336 conn(0x7f897c071520 msgr2=0x7f897c0ceee0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-09T13:50:13.925 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:13.921+0000 7f8983869640 1 -- 172.21.15.43:0/4015284336 shutdown_connections 2023-08-09T13:50:13.926 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:13.921+0000 7f8983869640 1 -- 172.21.15.43:0/4015284336 wait complete. 2023-08-09T13:50:13.926 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:13.923+0000 7f8983869640 1 Processor -- start 2023-08-09T13:50:13.926 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:13.923+0000 7f8983869640 1 -- start start 2023-08-09T13:50:13.927 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:13.925+0000 7f8983869640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f897c0b45f0 0x7f897c0c79d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:50:13.927 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:13.925+0000 7f8983869640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f897c0d04b0 0x7f897c14b560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:50:13.928 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:13.925+0000 7f8983869640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f897c15c550 0x7f897c14deb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:50:13.928 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:13.925+0000 7f8983869640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f897c15b7f0 con 0x7f897c0d04b0 2023-08-09T13:50:13.928 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:13.925+0000 7f8983869640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f897c14bfc0 con 0x7f897c0b45f0 2023-08-09T13:50:13.928 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:13.925+0000 7f8983869640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f897c0c8930 con 0x7f897c15c550 2023-08-09T13:50:13.929 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:13.925+0000 7f8980e04640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f897c0d04b0 0x7f897c14b560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:50:13.929 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:13.926+0000 7f8980e04640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f897c0d04b0 0x7f897c14b560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:45554/0 (socket says 172.21.15.43:45554) 2023-08-09T13:50:13.929 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:13.926+0000 7f8980e04640 1 -- 172.21.15.43:0/2354268092 learned_addr learned my addr 172.21.15.43:0/2354268092 (peer_addr_for_me v2:172.21.15.43:0/0) 2023-08-09T13:50:13.929 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:13.926+0000 7f8981e06640 1 --2- 172.21.15.43:0/2354268092 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f897c15c550 0x7f897c14deb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:50:13.930 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:13.926+0000 7f8981605640 1 --2- 172.21.15.43:0/2354268092 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f897c0b45f0 0x7f897c0c79d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:50:13.930 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:13.926+0000 7f8980e04640 1 -- 172.21.15.43:0/2354268092 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f897c15c550 msgr2=0x7f897c14deb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:50:13.930 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:13.926+0000 7f8980e04640 1 --2- 172.21.15.43:0/2354268092 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f897c15c550 0x7f897c14deb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:50:13.930 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:13.926+0000 7f8980e04640 1 -- 172.21.15.43:0/2354268092 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f897c0b45f0 msgr2=0x7f897c0c79d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:50:13.930 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:13.926+0000 7f8980e04640 1 --2- 172.21.15.43:0/2354268092 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f897c0b45f0 0x7f897c0c79d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:50:13.931 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:13.926+0000 7f8980e04640 1 -- 172.21.15.43:0/2354268092 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f89780094d0 con 0x7f897c0d04b0 2023-08-09T13:50:13.931 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:13.927+0000 7f8981605640 1 --2- 172.21.15.43:0/2354268092 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f897c0b45f0 0x7f897c0c79d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-08-09T13:50:13.931 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:13.927+0000 7f8980e04640 1 --2- 172.21.15.43:0/2354268092 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f897c0d04b0 0x7f897c14b560 secure :-1 s=READY pgs=948 cs=0 l=1 rev1=1 crypto rx=0x7f896c00c460 tx=0x7f896c00c920 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-09T13:50:13.931 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:13.927+0000 7f8971ffb640 1 -- 172.21.15.43:0/2354268092 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f896c012830 con 0x7f897c0d04b0 2023-08-09T13:50:13.931 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:13.927+0000 7f8971ffb640 1 -- 172.21.15.43:0/2354268092 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f896c0129d0 con 0x7f897c0d04b0 2023-08-09T13:50:13.931 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:13.927+0000 7f8971ffb640 1 -- 172.21.15.43:0/2354268092 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f896c012d80 con 0x7f897c0d04b0 2023-08-09T13:50:13.931 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:13.927+0000 7f8983869640 1 -- 172.21.15.43:0/2354268092 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f897c0c8090 con 0x7f897c0d04b0 2023-08-09T13:50:13.932 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:13.928+0000 7f8983869640 1 -- 172.21.15.43:0/2354268092 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f897c0c8650 con 0x7f897c0d04b0 2023-08-09T13:50:13.932 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:13.929+0000 7f8983869640 1 -- 172.21.15.43:0/2354268092 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f897c14c170 con 0x7f897c0d04b0 2023-08-09T13:50:13.932 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:13.930+0000 7f8971ffb640 1 -- 172.21.15.43:0/2354268092 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 29) v1 ==== 88443+0+0 (secure 0 0 0) 0x7f896c01b400 con 0x7f897c0d04b0 2023-08-09T13:50:13.933 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:13.931+0000 7f8971ffb640 1 --2- 172.21.15.43:0/2354268092 >> [v2:172.21.15.203:6832/3325141227,v1:172.21.15.203:6833/3325141227] conn(0x7f896006a170 0x7f896006c630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:50:13.933 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:13.931+0000 7f8971ffb640 1 -- 172.21.15.43:0/2354268092 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(79..79 src has 1..79) v4 ==== 7286+0+0 (secure 0 0 0) 0x7f896c088f80 con 0x7f897c0d04b0 2023-08-09T13:50:13.934 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:13.932+0000 7f8981605640 1 --2- 172.21.15.43:0/2354268092 >> [v2:172.21.15.203:6832/3325141227,v1:172.21.15.203:6833/3325141227] conn(0x7f896006a170 0x7f896006c630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:50:13.935 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:13.933+0000 7f8981605640 1 --2- 172.21.15.43:0/2354268092 >> [v2:172.21.15.203:6832/3325141227,v1:172.21.15.203:6833/3325141227] conn(0x7f896006a170 0x7f896006c630 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0x7f8978036000 tx=0x7f897803d040 comp rx=0 tx=0).ready entity=mgr.5519 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-09T13:50:13.937 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:13.934+0000 7f8971ffb640 1 -- 172.21.15.43:0/2354268092 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178066 (secure 0 0 0) 0x7f896c054070 con 0x7f897c0d04b0 2023-08-09T13:50:14.169 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:14.166+0000 7f8983869640 1 -- 172.21.15.43:0/2354268092 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} v 0) v1 -- 0x7f897c14c170 con 0x7f897c0d04b0 2023-08-09T13:50:14.685 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:14.682+0000 7f8971ffb640 1 -- 172.21.15.43:0/2354268092 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]=0 v28) v1 ==== 104+0+0 (secure 0 0 0) 0x7f897c14c170 con 0x7f897c0d04b0 2023-08-09T13:50:14.689 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:14.686+0000 7f8983869640 1 -- 172.21.15.43:0/2354268092 >> [v2:172.21.15.203:6832/3325141227,v1:172.21.15.203:6833/3325141227] conn(0x7f896006a170 msgr2=0x7f896006c630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:50:14.689 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:14.686+0000 7f8983869640 1 --2- 172.21.15.43:0/2354268092 >> [v2:172.21.15.203:6832/3325141227,v1:172.21.15.203:6833/3325141227] conn(0x7f896006a170 0x7f896006c630 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0x7f8978036000 tx=0x7f897803d040 comp rx=0 tx=0).stop 2023-08-09T13:50:14.689 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:14.686+0000 7f8983869640 1 -- 172.21.15.43:0/2354268092 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f897c0d04b0 msgr2=0x7f897c14b560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:50:14.689 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:14.686+0000 7f8983869640 1 --2- 172.21.15.43:0/2354268092 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f897c0d04b0 0x7f897c14b560 secure :-1 s=READY pgs=948 cs=0 l=1 rev1=1 crypto rx=0x7f896c00c460 tx=0x7f896c00c920 comp rx=0 tx=0).stop 2023-08-09T13:50:14.690 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:14.686+0000 7f8983869640 1 -- 172.21.15.43:0/2354268092 shutdown_connections 2023-08-09T13:50:14.690 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:14.686+0000 7f8983869640 1 --2- 172.21.15.43:0/2354268092 >> [v2:172.21.15.203:6832/3325141227,v1:172.21.15.203:6833/3325141227] conn(0x7f896006a170 0x7f896006c630 unknown :-1 s=CLOSED pgs=32 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:50:14.690 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:14.686+0000 7f8983869640 1 --2- 172.21.15.43:0/2354268092 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f897c15c550 0x7f897c14deb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:50:14.690 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:14.686+0000 7f8983869640 1 --2- 172.21.15.43:0/2354268092 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f897c0d04b0 0x7f897c14b560 unknown :-1 s=CLOSED pgs=948 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:50:14.690 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:14.686+0000 7f8983869640 1 --2- 172.21.15.43:0/2354268092 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f897c0b45f0 0x7f897c0c79d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:50:14.691 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:14.686+0000 7f8983869640 1 -- 172.21.15.43:0/2354268092 >> 172.21.15.43:0/2354268092 conn(0x7f897c071520 msgr2=0x7f897c15cad0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-09T13:50:14.691 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:14.686+0000 7f8983869640 1 -- 172.21.15.43:0/2354268092 shutdown_connections 2023-08-09T13:50:14.691 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:14.687+0000 7f8983869640 1 -- 172.21.15.43:0/2354268092 wait complete. 2023-08-09T13:50:14.710 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_metadata cephfs_metadata --yes-i-really-really-mean-it 2023-08-09T13:50:14.953 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:14.950+0000 7ff19a05d640 1 Processor -- start 2023-08-09T13:50:14.953 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:14.951+0000 7ff19a05d640 1 -- start start 2023-08-09T13:50:14.955 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:14.952+0000 7ff19a05d640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7ff19415a330 0x7ff1940c9780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:50:14.955 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:14.952+0000 7ff19a05d640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7ff1940aff80 0x7ff194070d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:50:14.955 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:14.952+0000 7ff19a05d640 1 -- --> v1:172.21.15.43:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff19415dd30 con 0x7ff19415ea60 2023-08-09T13:50:14.956 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:14.952+0000 7ff19a05d640 1 -- --> v2:172.21.15.43:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff1940c9cc0 con 0x7ff19415a330 2023-08-09T13:50:14.956 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:14.952+0000 7ff19a05d640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff194071240 con 0x7ff1940aff80 2023-08-09T13:50:14.957 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:14.953+0000 7ff19885a640 1 --1- >> v1:172.21.15.43:6789/0 conn(0x7ff19415ea60 0x7ff1940af870 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.43:6789/0 says I am v1:172.21.15.43:58816/0 (socket says 172.21.15.43:58816) 2023-08-09T13:50:14.957 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:14.953+0000 7ff19905b640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7ff19415a330 0x7ff1940c9780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:50:14.957 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:14.953+0000 7ff19885a640 1 -- 172.21.15.43:0/819436789 learned_addr learned my addr 172.21.15.43:0/819436789 (peer_addr_for_me v1:172.21.15.43:0/0) 2023-08-09T13:50:14.957 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:14.953+0000 7ff19985c640 1 --2- 172.21.15.43:0/819436789 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7ff1940aff80 0x7ff194070d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:50:14.957 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:14.953+0000 7ff18bfff640 1 -- 172.21.15.43:0/819436789 <== mon.0 v1:172.21.15.43:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 684744571 0 0) 0x7ff19415dd30 con 0x7ff19415ea60 2023-08-09T13:50:14.957 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:14.954+0000 7ff18bfff640 1 -- 172.21.15.43:0/819436789 --> v1:172.21.15.43:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ff180003520 con 0x7ff19415ea60 2023-08-09T13:50:14.958 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:14.954+0000 7ff19905b640 1 -- 172.21.15.43:0/819436789 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7ff1940aff80 msgr2=0x7ff194070d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-09T13:50:14.958 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:14.954+0000 7ff19905b640 1 --2- 172.21.15.43:0/819436789 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7ff1940aff80 0x7ff194070d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:50:14.958 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:14.954+0000 7ff19905b640 1 -- 172.21.15.43:0/819436789 >> v1:172.21.15.43:6789/0 conn(0x7ff19415ea60 legacy=0x7ff1940af870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:50:14.958 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:14.954+0000 7ff19905b640 1 -- 172.21.15.43:0/819436789 --> v2:172.21.15.43:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff194071490 con 0x7ff19415a330 2023-08-09T13:50:14.959 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:14.955+0000 7ff19905b640 1 --2- 172.21.15.43:0/819436789 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff19415a330 0x7ff1940c9780 secure :-1 s=READY pgs=950 cs=0 l=1 rev1=1 crypto rx=0x7ff1900097a0 tx=0x7ff19002b8b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=25aa53910c0004d9 server_cookie=0 in_seq=0 out_seq=0 2023-08-09T13:50:14.959 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:14.956+0000 7ff18bfff640 1 -- 172.21.15.43:0/819436789 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff19002d040 con 0x7ff19415a330 2023-08-09T13:50:14.959 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:14.956+0000 7ff18bfff640 1 -- 172.21.15.43:0/819436789 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff190031810 con 0x7ff19415a330 2023-08-09T13:50:14.960 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:14.956+0000 7ff18bfff640 1 -- 172.21.15.43:0/819436789 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff190031b80 con 0x7ff19415a330 2023-08-09T13:50:14.960 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:14.956+0000 7ff19a05d640 1 -- 172.21.15.43:0/819436789 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff19415a330 msgr2=0x7ff1940c9780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:50:14.960 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:14.956+0000 7ff19a05d640 1 --2- 172.21.15.43:0/819436789 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff19415a330 0x7ff1940c9780 secure :-1 s=READY pgs=950 cs=0 l=1 rev1=1 crypto rx=0x7ff1900097a0 tx=0x7ff19002b8b0 comp rx=0 tx=0).stop 2023-08-09T13:50:14.960 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:14.957+0000 7ff19a05d640 1 -- 172.21.15.43:0/819436789 shutdown_connections 2023-08-09T13:50:14.960 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:14.957+0000 7ff19a05d640 1 --2- 172.21.15.43:0/819436789 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7ff1940aff80 0x7ff194070d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:50:14.960 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:14.957+0000 7ff19a05d640 1 --2- 172.21.15.43:0/819436789 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff19415a330 0x7ff1940c9780 unknown :-1 s=CLOSED pgs=950 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:50:14.960 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:14.957+0000 7ff19a05d640 1 -- 172.21.15.43:0/819436789 >> 172.21.15.43:0/819436789 conn(0x7ff1940ca870 msgr2=0x7ff1940cac50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-09T13:50:14.961 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:14.957+0000 7ff19a05d640 1 -- 172.21.15.43:0/819436789 shutdown_connections 2023-08-09T13:50:14.961 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:14.957+0000 7ff19a05d640 1 -- 172.21.15.43:0/819436789 wait complete. 2023-08-09T13:50:14.962 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:14.959+0000 7ff19a05d640 1 Processor -- start 2023-08-09T13:50:14.962 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:14.959+0000 7ff19a05d640 1 -- start start 2023-08-09T13:50:14.966 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:14.961+0000 7ff19a05d640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7ff1940aff80 0x7ff1940c5670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:50:14.966 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:14.961+0000 7ff19a05d640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7ff19415a330 0x7ff1940cbe90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:50:14.966 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:14.961+0000 7ff19a05d640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff19415ea60 0x7ff1940c7400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:50:14.966 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:14.961+0000 7ff19a05d640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff1940cd210 con 0x7ff19415ea60 2023-08-09T13:50:14.966 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:14.961+0000 7ff19a05d640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff1940c5bb0 con 0x7ff1940aff80 2023-08-09T13:50:14.967 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:14.961+0000 7ff19a05d640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff1940c6f90 con 0x7ff19415a330 2023-08-09T13:50:14.967 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:14.961+0000 7ff19985c640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff19415ea60 0x7ff1940c7400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:50:14.967 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:14.961+0000 7ff19985c640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff19415ea60 0x7ff1940c7400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:43996/0 (socket says 172.21.15.43:43996) 2023-08-09T13:50:14.967 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:14.961+0000 7ff19985c640 1 -- 172.21.15.43:0/3038163745 learned_addr learned my addr 172.21.15.43:0/3038163745 (peer_addr_for_me v2:172.21.15.43:0/0) 2023-08-09T13:50:14.967 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:14.961+0000 7ff19905b640 1 --2- 172.21.15.43:0/3038163745 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7ff1940aff80 0x7ff1940c5670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:50:14.968 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:14.962+0000 7ff19985c640 1 -- 172.21.15.43:0/3038163745 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7ff19415a330 msgr2=0x7ff1940cbe90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:50:14.968 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:14.962+0000 7ff19885a640 1 --2- 172.21.15.43:0/3038163745 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7ff19415a330 0x7ff1940cbe90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:50:14.968 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:14.962+0000 7ff19985c640 1 --2- 172.21.15.43:0/3038163745 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7ff19415a330 0x7ff1940cbe90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:50:14.968 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:14.962+0000 7ff19985c640 1 -- 172.21.15.43:0/3038163745 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7ff1940aff80 msgr2=0x7ff1940c5670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:50:14.968 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:14.962+0000 7ff19985c640 1 --2- 172.21.15.43:0/3038163745 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7ff1940aff80 0x7ff1940c5670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:50:14.969 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:14.962+0000 7ff19985c640 1 -- 172.21.15.43:0/3038163745 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff190009400 con 0x7ff19415ea60 2023-08-09T13:50:14.969 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:14.962+0000 7ff19905b640 1 --2- 172.21.15.43:0/3038163745 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7ff1940aff80 0x7ff1940c5670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-08-09T13:50:14.969 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:14.962+0000 7ff19885a640 1 --2- 172.21.15.43:0/3038163745 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7ff19415a330 0x7ff1940cbe90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-08-09T13:50:14.969 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:14.962+0000 7ff19985c640 1 --2- 172.21.15.43:0/3038163745 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff19415ea60 0x7ff1940c7400 secure :-1 s=READY pgs=951 cs=0 l=1 rev1=1 crypto rx=0x7ff18c007e60 tx=0x7ff18c00e4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-09T13:50:14.969 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:14.962+0000 7ff189ffb640 1 -- 172.21.15.43:0/3038163745 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff18c014420 con 0x7ff19415ea60 2023-08-09T13:50:14.970 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:14.962+0000 7ff189ffb640 1 -- 172.21.15.43:0/3038163745 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff18c0145a0 con 0x7ff19415ea60 2023-08-09T13:50:14.970 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:14.962+0000 7ff189ffb640 1 -- 172.21.15.43:0/3038163745 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff18c014950 con 0x7ff19415ea60 2023-08-09T13:50:14.970 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:14.962+0000 7ff19a05d640 1 -- 172.21.15.43:0/3038163745 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff1940cd9b0 con 0x7ff19415ea60 2023-08-09T13:50:14.971 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:14.963+0000 7ff19a05d640 1 -- 172.21.15.43:0/3038163745 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff1940cd3f0 con 0x7ff19415ea60 2023-08-09T13:50:14.972 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:14.964+0000 7ff19a05d640 1 -- 172.21.15.43:0/3038163745 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff1940c70d0 con 0x7ff19415ea60 2023-08-09T13:50:14.973 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:14.965+0000 7ff189ffb640 1 -- 172.21.15.43:0/3038163745 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 29) v1 ==== 88443+0+0 (secure 0 0 0) 0x7ff18c014ab0 con 0x7ff19415ea60 2023-08-09T13:50:14.973 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:14.967+0000 7ff189ffb640 1 --2- 172.21.15.43:0/3038163745 >> [v2:172.21.15.203:6832/3325141227,v1:172.21.15.203:6833/3325141227] conn(0x7ff17806a170 0x7ff17806c630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:50:14.973 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:14.967+0000 7ff189ffb640 1 -- 172.21.15.43:0/3038163745 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(79..79 src has 1..79) v4 ==== 7286+0+0 (secure 0 0 0) 0x7ff18c0895d0 con 0x7ff19415ea60 2023-08-09T13:50:14.973 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:14.967+0000 7ff19905b640 1 --2- 172.21.15.43:0/3038163745 >> [v2:172.21.15.203:6832/3325141227,v1:172.21.15.203:6833/3325141227] conn(0x7ff17806a170 0x7ff17806c630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:50:14.974 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:14.968+0000 7ff19905b640 1 --2- 172.21.15.43:0/3038163745 >> [v2:172.21.15.203:6832/3325141227,v1:172.21.15.203:6833/3325141227] conn(0x7ff17806a170 0x7ff17806c630 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0x7ff190036000 tx=0x7ff190008030 comp rx=0 tx=0).ready entity=mgr.5519 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-09T13:50:14.974 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:14.970+0000 7ff189ffb640 1 -- 172.21.15.43:0/3038163745 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178066 (secure 0 0 0) 0x7ff18c0580d0 con 0x7ff19415ea60 2023-08-09T13:50:15.166 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:15.163+0000 7ff19a05d640 1 -- 172.21.15.43:0/3038163745 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7ff1940739f0 con 0x7ff19415ea60 2023-08-09T13:50:15.695 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:15.693+0000 7ff189ffb640 1 -- 172.21.15.43:0/3038163745 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_metadata' removed v80) v1 ==== 181+0+0 (secure 0 0 0) 0x7ff18c057c60 con 0x7ff19415ea60 2023-08-09T13:50:15.697 INFO:teuthology.orchestra.run.smithi043.stderr:pool 'cephfs_metadata' removed 2023-08-09T13:50:15.703 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:15.701+0000 7ff19a05d640 1 -- 172.21.15.43:0/3038163745 >> [v2:172.21.15.203:6832/3325141227,v1:172.21.15.203:6833/3325141227] conn(0x7ff17806a170 msgr2=0x7ff17806c630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:50:15.703 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:15.701+0000 7ff19a05d640 1 --2- 172.21.15.43:0/3038163745 >> [v2:172.21.15.203:6832/3325141227,v1:172.21.15.203:6833/3325141227] conn(0x7ff17806a170 0x7ff17806c630 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0x7ff190036000 tx=0x7ff190008030 comp rx=0 tx=0).stop 2023-08-09T13:50:15.703 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:15.701+0000 7ff19a05d640 1 -- 172.21.15.43:0/3038163745 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff19415ea60 msgr2=0x7ff1940c7400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:50:15.703 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:15.701+0000 7ff19a05d640 1 --2- 172.21.15.43:0/3038163745 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff19415ea60 0x7ff1940c7400 secure :-1 s=READY pgs=951 cs=0 l=1 rev1=1 crypto rx=0x7ff18c007e60 tx=0x7ff18c00e4b0 comp rx=0 tx=0).stop 2023-08-09T13:50:15.704 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:15.702+0000 7ff19a05d640 1 -- 172.21.15.43:0/3038163745 shutdown_connections 2023-08-09T13:50:15.704 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:15.702+0000 7ff19a05d640 1 --2- 172.21.15.43:0/3038163745 >> [v2:172.21.15.203:6832/3325141227,v1:172.21.15.203:6833/3325141227] conn(0x7ff17806a170 0x7ff17806c630 unknown :-1 s=CLOSED pgs=34 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:50:15.704 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:15.702+0000 7ff19a05d640 1 --2- 172.21.15.43:0/3038163745 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff19415ea60 0x7ff1940c7400 unknown :-1 s=CLOSED pgs=951 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:50:15.704 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:15.702+0000 7ff19a05d640 1 --2- 172.21.15.43:0/3038163745 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7ff19415a330 0x7ff1940cbe90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:50:15.705 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:15.702+0000 7ff19a05d640 1 --2- 172.21.15.43:0/3038163745 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7ff1940aff80 0x7ff1940c5670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:50:15.705 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:15.702+0000 7ff19a05d640 1 -- 172.21.15.43:0/3038163745 >> 172.21.15.43:0/3038163745 conn(0x7ff1940ca870 msgr2=0x7ff1940acf30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-09T13:50:15.705 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:15.702+0000 7ff19a05d640 1 -- 172.21.15.43:0/3038163745 shutdown_connections 2023-08-09T13:50:15.705 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:15.703+0000 7ff19a05d640 1 -- 172.21.15.43:0/3038163745 wait complete. 2023-08-09T13:50:15.725 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_data cephfs_data --yes-i-really-really-mean-it 2023-08-09T13:50:15.947 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:15.945+0000 7feaf56f3640 1 Processor -- start 2023-08-09T13:50:15.947 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:15.946+0000 7feaf56f3640 1 -- start start 2023-08-09T13:50:15.949 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:15.947+0000 7feaf56f3640 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7feaf00cbd70 0x7feaf00ce4c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:50:15.949 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:15.947+0000 7feaf56f3640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7feaf00cea00 0x7feaf00ca540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:50:15.949 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:15.947+0000 7feaf56f3640 1 -- --> v1:172.21.15.43:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7feaf00cee50 con 0x7feaf0153590 2023-08-09T13:50:15.949 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:15.947+0000 7feaf56f3640 1 -- --> v2:172.21.15.43:3300/0 -- mon_getmap magic: 0 v1 -- 0x7feaf00ccb00 con 0x7feaf00cea00 2023-08-09T13:50:15.949 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:15.947+0000 7feaf56f3640 1 -- --> v2:172.21.15.203:3300/0 -- mon_getmap magic: 0 v1 -- 0x7feaf00c86f0 con 0x7feaf00cbd70 2023-08-09T13:50:15.950 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:15.948+0000 7feaeeffd640 1 --1- >> v1:172.21.15.43:6789/0 conn(0x7feaf0153590 0x7feaf00cc3f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.43:6789/0 says I am v1:172.21.15.43:44584/0 (socket says 172.21.15.43:44584) 2023-08-09T13:50:15.950 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:15.948+0000 7feaeeffd640 1 -- 172.21.15.43:0/2448089426 learned_addr learned my addr 172.21.15.43:0/2448089426 (peer_addr_for_me v1:172.21.15.43:0/0) 2023-08-09T13:50:15.950 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:15.948+0000 7feaef7fe640 1 --2- 172.21.15.43:0/2448089426 >> v2:172.21.15.43:3300/0 conn(0x7feaf00cea00 0x7feaf00ca540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:50:15.951 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:15.948+0000 7feae7fff640 1 --2- 172.21.15.43:0/2448089426 >> v2:172.21.15.203:3300/0 conn(0x7feaf00cbd70 0x7feaf00ce4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:50:15.951 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:15.948+0000 7feaee7fc640 1 -- 172.21.15.43:0/2448089426 <== mon.0 v1:172.21.15.43:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1393058835 0 0) 0x7feae0001110 con 0x7feaf0153590 2023-08-09T13:50:15.951 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:15.948+0000 7feaef7fe640 1 -- 172.21.15.43:0/2448089426 >> v2:172.21.15.203:3300/0 conn(0x7feaf00cbd70 msgr2=0x7feaf00ce4c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-09T13:50:15.951 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:15.948+0000 7feaef7fe640 1 --2- 172.21.15.43:0/2448089426 >> v2:172.21.15.203:3300/0 conn(0x7feaf00cbd70 0x7feaf00ce4c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:50:15.951 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:15.948+0000 7feaef7fe640 1 -- 172.21.15.43:0/2448089426 >> v1:172.21.15.43:6789/0 conn(0x7feaf0153590 legacy=0x7feaf00cc3f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:50:15.951 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:15.948+0000 7feaef7fe640 1 -- 172.21.15.43:0/2448089426 --> v2:172.21.15.43:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7feaf00c9510 con 0x7feaf00cea00 2023-08-09T13:50:15.952 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:15.950+0000 7feaef7fe640 1 --2- 172.21.15.43:0/2448089426 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7feaf00cea00 0x7feaf00ca540 secure :-1 s=READY pgs=953 cs=0 l=1 rev1=1 crypto rx=0x7feaf00afb80 tx=0x7feae802bec0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=be3c535adc1a79a1 server_cookie=0 in_seq=0 out_seq=0 2023-08-09T13:50:15.953 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:15.950+0000 7feaee7fc640 1 -- 172.21.15.43:0/2448089426 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feae802cc70 con 0x7feaf00cea00 2023-08-09T13:50:15.953 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:15.950+0000 7feaee7fc640 1 -- 172.21.15.43:0/2448089426 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7feae802cdd0 con 0x7feaf00cea00 2023-08-09T13:50:15.953 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:15.950+0000 7feaee7fc640 1 -- 172.21.15.43:0/2448089426 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feae8034660 con 0x7feaf00cea00 2023-08-09T13:50:15.953 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:15.951+0000 7feaf56f3640 1 -- 172.21.15.43:0/2448089426 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7feaf00cea00 msgr2=0x7feaf00ca540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:50:15.953 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:15.951+0000 7feaf56f3640 1 --2- 172.21.15.43:0/2448089426 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7feaf00cea00 0x7feaf00ca540 secure :-1 s=READY pgs=953 cs=0 l=1 rev1=1 crypto rx=0x7feaf00afb80 tx=0x7feae802bec0 comp rx=0 tx=0).stop 2023-08-09T13:50:15.953 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:15.951+0000 7feaf56f3640 1 -- 172.21.15.43:0/2448089426 shutdown_connections 2023-08-09T13:50:15.954 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:15.951+0000 7feaf56f3640 1 --2- 172.21.15.43:0/2448089426 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7feaf00cea00 0x7feaf00ca540 unknown :-1 s=CLOSED pgs=953 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:50:15.954 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:15.951+0000 7feaf56f3640 1 --2- 172.21.15.43:0/2448089426 >> v2:172.21.15.203:3300/0 conn(0x7feaf00cbd70 0x7feaf00ce4c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:50:15.954 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:15.951+0000 7feaf56f3640 1 -- 172.21.15.43:0/2448089426 >> 172.21.15.43:0/2448089426 conn(0x7feaf006b990 msgr2=0x7feaf006bda0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-09T13:50:15.954 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:15.951+0000 7feaf56f3640 1 -- 172.21.15.43:0/2448089426 shutdown_connections 2023-08-09T13:50:15.954 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:15.951+0000 7feaf56f3640 1 -- 172.21.15.43:0/2448089426 wait complete. 2023-08-09T13:50:15.955 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:15.953+0000 7feaf56f3640 1 Processor -- start 2023-08-09T13:50:15.955 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:15.953+0000 7feaf56f3640 1 -- start start 2023-08-09T13:50:15.956 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:15.954+0000 7feaf56f3640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7feaf00cbd70 0x7feaf01524d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:50:15.956 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:15.954+0000 7feaf56f3640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7feaf00cea00 0x7feaf0159010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:50:15.956 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:15.954+0000 7feaf56f3640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7feaf0153590 0x7feaf0159550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:50:15.956 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:15.954+0000 7feaf56f3640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feaf0158110 con 0x7feaf00cbd70 2023-08-09T13:50:15.956 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:15.954+0000 7feaf56f3640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feaf0157240 con 0x7feaf0153590 2023-08-09T13:50:15.956 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:15.954+0000 7feaf56f3640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7feaf0157490 con 0x7feaf00cea00 2023-08-09T13:50:15.956 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:15.954+0000 7feaeeffd640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7feaf00cbd70 0x7feaf01524d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:50:15.957 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:15.954+0000 7feaeeffd640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7feaf00cbd70 0x7feaf01524d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:44022/0 (socket says 172.21.15.43:44022) 2023-08-09T13:50:15.957 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:15.954+0000 7feaeeffd640 1 -- 172.21.15.43:0/215372645 learned_addr learned my addr 172.21.15.43:0/215372645 (peer_addr_for_me v2:172.21.15.43:0/0) 2023-08-09T13:50:15.957 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:15.955+0000 7feaeeffd640 1 -- 172.21.15.43:0/215372645 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7feaf00cea00 msgr2=0x7feaf0159010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:50:15.958 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:15.955+0000 7feaeeffd640 1 --2- 172.21.15.43:0/215372645 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7feaf00cea00 0x7feaf0159010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:50:15.958 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:15.955+0000 7feaeeffd640 1 -- 172.21.15.43:0/215372645 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7feaf0153590 msgr2=0x7feaf0159550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:50:15.958 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:15.955+0000 7feaeeffd640 1 --2- 172.21.15.43:0/215372645 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7feaf0153590 0x7feaf0159550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:50:15.958 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:15.955+0000 7feaeeffd640 1 -- 172.21.15.43:0/215372645 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7feae8009a70 con 0x7feaf00cbd70 2023-08-09T13:50:15.958 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:15.955+0000 7feaeeffd640 1 --2- 172.21.15.43:0/215372645 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7feaf00cbd70 0x7feaf01524d0 secure :-1 s=READY pgs=954 cs=0 l=1 rev1=1 crypto rx=0x7feae000c430 tx=0x7feae000c8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-09T13:50:15.958 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:15.955+0000 7feae77fe640 1 -- 172.21.15.43:0/215372645 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feae00074e0 con 0x7feaf00cbd70 2023-08-09T13:50:15.959 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:15.955+0000 7feae77fe640 1 -- 172.21.15.43:0/215372645 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7feae0007660 con 0x7feaf00cbd70 2023-08-09T13:50:15.959 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:15.955+0000 7feae77fe640 1 -- 172.21.15.43:0/215372645 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feae0004a20 con 0x7feaf00cbd70 2023-08-09T13:50:15.959 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:15.955+0000 7feaf56f3640 1 -- 172.21.15.43:0/215372645 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7feaf0157770 con 0x7feaf00cbd70 2023-08-09T13:50:15.959 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:15.956+0000 7feaf56f3640 1 -- 172.21.15.43:0/215372645 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7feaf0157d30 con 0x7feaf00cbd70 2023-08-09T13:50:15.959 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:15.957+0000 7feaf56f3640 1 -- 172.21.15.43:0/215372645 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7feaf00c9980 con 0x7feaf00cbd70 2023-08-09T13:50:15.960 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:15.958+0000 7feae77fe640 1 -- 172.21.15.43:0/215372645 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 29) v1 ==== 88443+0+0 (secure 0 0 0) 0x7feae0004b80 con 0x7feaf00cbd70 2023-08-09T13:50:15.960 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:15.959+0000 7feae77fe640 1 --2- 172.21.15.43:0/215372645 >> [v2:172.21.15.203:6832/3325141227,v1:172.21.15.203:6833/3325141227] conn(0x7fead006a170 0x7fead006c630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:50:15.961 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:15.959+0000 7feae77fe640 1 -- 172.21.15.43:0/215372645 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(80..80 src has 1..80) v4 ==== 6848+0+0 (secure 0 0 0) 0x7feae0088710 con 0x7feaf00cbd70 2023-08-09T13:50:15.961 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:15.960+0000 7feae7fff640 1 --2- 172.21.15.43:0/215372645 >> [v2:172.21.15.203:6832/3325141227,v1:172.21.15.203:6833/3325141227] conn(0x7fead006a170 0x7fead006c630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:50:15.962 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:15.961+0000 7feae7fff640 1 --2- 172.21.15.43:0/215372645 >> [v2:172.21.15.203:6832/3325141227,v1:172.21.15.203:6833/3325141227] conn(0x7fead006a170 0x7fead006c630 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7feaf0157050 tx=0x7fead8008040 comp rx=0 tx=0).ready entity=mgr.5519 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-09T13:50:15.965 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:15.964+0000 7feae77fe640 1 -- 172.21.15.43:0/215372645 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178066 (secure 0 0 0) 0x7feaf00c9980 con 0x7feaf00cbd70 2023-08-09T13:50:16.164 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:16.163+0000 7feaf56f3640 1 -- 172.21.15.43:0/215372645 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7feaf00cee50 con 0x7feaf00cbd70 2023-08-09T13:50:16.713 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:16.710+0000 7feae77fe640 1 -- 172.21.15.43:0/215372645 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data' removed v81) v1 ==== 169+0+0 (secure 0 0 0) 0x7feaf00cee50 con 0x7feaf00cbd70 2023-08-09T13:50:16.716 INFO:teuthology.orchestra.run.smithi043.stderr:pool 'cephfs_data' removed 2023-08-09T13:50:16.723 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:16.722+0000 7feaf56f3640 1 -- 172.21.15.43:0/215372645 >> [v2:172.21.15.203:6832/3325141227,v1:172.21.15.203:6833/3325141227] conn(0x7fead006a170 msgr2=0x7fead006c630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:50:16.724 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:16.722+0000 7feaf56f3640 1 --2- 172.21.15.43:0/215372645 >> [v2:172.21.15.203:6832/3325141227,v1:172.21.15.203:6833/3325141227] conn(0x7fead006a170 0x7fead006c630 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7feaf0157050 tx=0x7fead8008040 comp rx=0 tx=0).stop 2023-08-09T13:50:16.724 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:16.722+0000 7feaf56f3640 1 -- 172.21.15.43:0/215372645 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7feaf00cbd70 msgr2=0x7feaf01524d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:50:16.724 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:16.722+0000 7feaf56f3640 1 --2- 172.21.15.43:0/215372645 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7feaf00cbd70 0x7feaf01524d0 secure :-1 s=READY pgs=954 cs=0 l=1 rev1=1 crypto rx=0x7feae000c430 tx=0x7feae000c8f0 comp rx=0 tx=0).stop 2023-08-09T13:50:16.725 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:16.722+0000 7feaf56f3640 1 -- 172.21.15.43:0/215372645 shutdown_connections 2023-08-09T13:50:16.725 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:16.722+0000 7feaf56f3640 1 --2- 172.21.15.43:0/215372645 >> [v2:172.21.15.203:6832/3325141227,v1:172.21.15.203:6833/3325141227] conn(0x7fead006a170 0x7fead006c630 unknown :-1 s=CLOSED pgs=35 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:50:16.725 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:16.723+0000 7feaf56f3640 1 --2- 172.21.15.43:0/215372645 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7feaf0153590 0x7feaf0159550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:50:16.726 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:16.723+0000 7feaf56f3640 1 --2- 172.21.15.43:0/215372645 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7feaf00cea00 0x7feaf0159010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:50:16.726 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:16.723+0000 7feaf56f3640 1 --2- 172.21.15.43:0/215372645 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7feaf00cbd70 0x7feaf01524d0 unknown :-1 s=CLOSED pgs=954 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:50:16.726 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:16.723+0000 7feaf56f3640 1 -- 172.21.15.43:0/215372645 >> 172.21.15.43:0/215372645 conn(0x7feaf006b990 msgr2=0x7feaf00c7b90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-09T13:50:16.726 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:16.723+0000 7feaf56f3640 1 -- 172.21.15.43:0/215372645 shutdown_connections 2023-08-09T13:50:16.726 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:16.723+0000 7feaf56f3640 1 -- 172.21.15.43:0/215372645 wait complete. 2023-08-09T13:50:16.747 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config rm mgr mgr/snap_schedule/dump_on_update 2023-08-09T13:50:16.947 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:16.946+0000 7ff340e37640 1 Processor -- start 2023-08-09T13:50:16.948 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:16.946+0000 7ff340e37640 1 -- start start 2023-08-09T13:50:16.949 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:16.948+0000 7ff340e37640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7ff33c070ef0 0x7ff33c0ab8d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:50:16.950 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:16.948+0000 7ff340e37640 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7ff33c0cc0b0 0x7ff33c06f310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:50:16.950 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:16.948+0000 7ff340e37640 1 -- --> v1:172.21.15.203:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff33c0acfa0 con 0x7ff33c0abe10 2023-08-09T13:50:16.950 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:16.948+0000 7ff340e37640 1 -- --> v2:172.21.15.43:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff33c15c940 con 0x7ff33c070ef0 2023-08-09T13:50:16.950 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:16.948+0000 7ff340e37640 1 -- --> v2:172.21.15.203:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff33c0ac390 con 0x7ff33c0cc0b0 2023-08-09T13:50:16.950 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:16.948+0000 7ff33b7fe640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7ff33c070ef0 0x7ff33c0ab8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:50:16.950 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:16.948+0000 7ff33b7fe640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7ff33c070ef0 0x7ff33c0ab8d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:44026/0 (socket says 172.21.15.43:44026) 2023-08-09T13:50:16.951 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:16.948+0000 7ff33b7fe640 1 -- 172.21.15.43:0/3708604150 learned_addr learned my addr 172.21.15.43:0/3708604150 (peer_addr_for_me v2:172.21.15.43:0/0) 2023-08-09T13:50:16.951 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:16.949+0000 7ff33bfff640 1 --2- 172.21.15.43:0/3708604150 >> v2:172.21.15.203:3300/0 conn(0x7ff33c0cc0b0 0x7ff33c06f310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:50:16.952 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:16.949+0000 7ff33b7fe640 1 -- 172.21.15.43:0/3708604150 >> v2:172.21.15.203:3300/0 conn(0x7ff33c0cc0b0 msgr2=0x7ff33c06f310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-09T13:50:16.953 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:16.949+0000 7ff33b7fe640 1 --2- 172.21.15.43:0/3708604150 >> v2:172.21.15.203:3300/0 conn(0x7ff33c0cc0b0 0x7ff33c06f310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:50:16.953 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:16.949+0000 7ff33b7fe640 1 -- 172.21.15.43:0/3708604150 >> v1:172.21.15.203:6789/0 conn(0x7ff33c0abe10 legacy=0x7ff33c0cb9a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-09T13:50:16.953 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:16.949+0000 7ff33b7fe640 1 -- 172.21.15.43:0/3708604150 --> v2:172.21.15.43:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff33c0ac670 con 0x7ff33c070ef0 2023-08-09T13:50:16.953 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:16.949+0000 7ff33bfff640 1 --2- 172.21.15.43:0/3708604150 >> v2:172.21.15.203:3300/0 conn(0x7ff33c0cc0b0 0x7ff33c06f310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-08-09T13:50:16.953 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:16.950+0000 7ff33b7fe640 1 --2- 172.21.15.43:0/3708604150 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff33c070ef0 0x7ff33c0ab8d0 secure :-1 s=READY pgs=955 cs=0 l=1 rev1=1 crypto rx=0x7ff33c146380 tx=0x7ff32c02b9a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3bd27b3b7fe7d25c server_cookie=0 in_seq=0 out_seq=0 2023-08-09T13:50:16.953 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:16.950+0000 7ff33a7fc640 1 -- 172.21.15.43:0/3708604150 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff32c030860 con 0x7ff33c070ef0 2023-08-09T13:50:16.953 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:16.950+0000 7ff33a7fc640 1 -- 172.21.15.43:0/3708604150 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff32c0309c0 con 0x7ff33c070ef0 2023-08-09T13:50:16.954 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:16.950+0000 7ff33a7fc640 1 -- 172.21.15.43:0/3708604150 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff32c030d50 con 0x7ff33c070ef0 2023-08-09T13:50:16.955 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:16.951+0000 7ff340e37640 1 -- 172.21.15.43:0/3708604150 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff33c070ef0 msgr2=0x7ff33c0ab8d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:50:16.955 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:16.951+0000 7ff340e37640 1 --2- 172.21.15.43:0/3708604150 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff33c070ef0 0x7ff33c0ab8d0 secure :-1 s=READY pgs=955 cs=0 l=1 rev1=1 crypto rx=0x7ff33c146380 tx=0x7ff32c02b9a0 comp rx=0 tx=0).stop 2023-08-09T13:50:16.955 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:16.951+0000 7ff340e37640 1 -- 172.21.15.43:0/3708604150 shutdown_connections 2023-08-09T13:50:16.955 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:16.951+0000 7ff340e37640 1 --2- 172.21.15.43:0/3708604150 >> v2:172.21.15.203:3300/0 conn(0x7ff33c0cc0b0 0x7ff33c06f310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:50:16.955 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:16.951+0000 7ff340e37640 1 --2- 172.21.15.43:0/3708604150 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff33c070ef0 0x7ff33c0ab8d0 unknown :-1 s=CLOSED pgs=955 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:50:16.955 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:16.951+0000 7ff340e37640 1 -- 172.21.15.43:0/3708604150 >> 172.21.15.43:0/3708604150 conn(0x7ff33c0cddd0 msgr2=0x7ff33c0ce1e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-09T13:50:16.955 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:16.951+0000 7ff340e37640 1 -- 172.21.15.43:0/3708604150 shutdown_connections 2023-08-09T13:50:16.956 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:16.951+0000 7ff340e37640 1 -- 172.21.15.43:0/3708604150 wait complete. 2023-08-09T13:50:16.956 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:16.953+0000 7ff340e37640 1 Processor -- start 2023-08-09T13:50:16.956 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:16.953+0000 7ff340e37640 1 -- start start 2023-08-09T13:50:16.957 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:16.954+0000 7ff340e37640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7ff33c070ef0 0x7ff33c157190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:50:16.958 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:16.955+0000 7ff340e37640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff33c0abe10 0x7ff33c1f2010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:50:16.958 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:16.955+0000 7ff340e37640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7ff33c0cc0b0 0x7ff33c15cb10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:50:16.958 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:16.955+0000 7ff340e37640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff33c1576d0 con 0x7ff33c0abe10 2023-08-09T13:50:16.958 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:16.955+0000 7ff340e37640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff33c1443b0 con 0x7ff33c0cc0b0 2023-08-09T13:50:16.958 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:16.955+0000 7ff340e37640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff33c1f2550 con 0x7ff33c070ef0 2023-08-09T13:50:16.958 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:16.955+0000 7ff33affd640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff33c0abe10 0x7ff33c1f2010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:50:16.958 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:16.955+0000 7ff33affd640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff33c0abe10 0x7ff33c1f2010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:44040/0 (socket says 172.21.15.43:44040) 2023-08-09T13:50:16.959 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:16.955+0000 7ff33affd640 1 -- 172.21.15.43:0/688284634 learned_addr learned my addr 172.21.15.43:0/688284634 (peer_addr_for_me v2:172.21.15.43:0/0) 2023-08-09T13:50:16.959 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:16.955+0000 7ff33b7fe640 1 --2- 172.21.15.43:0/688284634 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7ff33c070ef0 0x7ff33c157190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:50:16.959 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:16.955+0000 7ff33bfff640 1 --2- 172.21.15.43:0/688284634 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7ff33c0cc0b0 0x7ff33c15cb10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:50:16.959 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:16.955+0000 7ff33affd640 1 -- 172.21.15.43:0/688284634 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7ff33c070ef0 msgr2=0x7ff33c157190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:50:16.959 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:16.955+0000 7ff33affd640 1 --2- 172.21.15.43:0/688284634 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7ff33c070ef0 0x7ff33c157190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:50:16.959 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:16.955+0000 7ff33affd640 1 -- 172.21.15.43:0/688284634 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7ff33c0cc0b0 msgr2=0x7ff33c15cb10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:50:16.959 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:16.955+0000 7ff33affd640 1 --2- 172.21.15.43:0/688284634 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7ff33c0cc0b0 0x7ff33c15cb10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:50:16.960 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:16.955+0000 7ff33affd640 1 -- 172.21.15.43:0/688284634 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff32c0094d0 con 0x7ff33c0abe10 2023-08-09T13:50:16.960 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:16.956+0000 7ff33affd640 1 --2- 172.21.15.43:0/688284634 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff33c0abe10 0x7ff33c1f2010 secure :-1 s=READY pgs=956 cs=0 l=1 rev1=1 crypto rx=0x7ff33000c410 tx=0x7ff33000c8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-09T13:50:16.960 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:16.956+0000 7ff323fff640 1 -- 172.21.15.43:0/688284634 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff3300035f0 con 0x7ff33c0abe10 2023-08-09T13:50:16.960 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:16.956+0000 7ff323fff640 1 -- 172.21.15.43:0/688284634 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff330003750 con 0x7ff33c0abe10 2023-08-09T13:50:16.960 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:16.956+0000 7ff323fff640 1 -- 172.21.15.43:0/688284634 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff330003b00 con 0x7ff33c0abe10 2023-08-09T13:50:16.960 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:16.956+0000 7ff340e37640 1 -- 172.21.15.43:0/688284634 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff33c15d050 con 0x7ff33c0abe10 2023-08-09T13:50:16.960 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:16.956+0000 7ff340e37640 1 -- 172.21.15.43:0/688284634 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff33c143e30 con 0x7ff33c0abe10 2023-08-09T13:50:16.961 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:16.958+0000 7ff340e37640 1 -- 172.21.15.43:0/688284634 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff33c04e390 con 0x7ff33c0abe10 2023-08-09T13:50:16.961 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:16.958+0000 7ff323fff640 1 -- 172.21.15.43:0/688284634 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 29) v1 ==== 88443+0+0 (secure 0 0 0) 0x7ff330003c60 con 0x7ff33c0abe10 2023-08-09T13:50:16.961 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:16.960+0000 7ff323fff640 1 --2- 172.21.15.43:0/688284634 >> [v2:172.21.15.203:6832/3325141227,v1:172.21.15.203:6833/3325141227] conn(0x7ff32806a170 0x7ff32806c630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:50:16.962 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:16.960+0000 7ff323fff640 1 -- 172.21.15.43:0/688284634 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(81..81 src has 1..81) v4 ==== 6434+0+0 (secure 0 0 0) 0x7ff330087aa0 con 0x7ff33c0abe10 2023-08-09T13:50:16.962 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:16.961+0000 7ff33b7fe640 1 --2- 172.21.15.43:0/688284634 >> [v2:172.21.15.203:6832/3325141227,v1:172.21.15.203:6833/3325141227] conn(0x7ff32806a170 0x7ff32806c630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:50:16.963 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:16.962+0000 7ff33b7fe640 1 --2- 172.21.15.43:0/688284634 >> [v2:172.21.15.203:6832/3325141227,v1:172.21.15.203:6833/3325141227] conn(0x7ff32806a170 0x7ff32806c630 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7ff32c036000 tx=0x7ff32c03b040 comp rx=0 tx=0).ready entity=mgr.5519 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-09T13:50:16.966 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:16.964+0000 7ff323fff640 1 -- 172.21.15.43:0/688284634 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178066 (secure 0 0 0) 0x7ff330052ef0 con 0x7ff33c0abe10 2023-08-09T13:50:17.165 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:17.164+0000 7ff340e37640 1 -- 172.21.15.43:0/688284634 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "config rm", "who": "mgr", "name": "mgr/snap_schedule/dump_on_update"} v 0) v1 -- 0x7ff33c0acc00 con 0x7ff33c0abe10 2023-08-09T13:50:17.174 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:17.172+0000 7ff323fff640 1 -- 172.21.15.43:0/688284634 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "config rm", "who": "mgr", "name": "mgr/snap_schedule/dump_on_update"}]=0 v19) v1 ==== 115+0+0 (secure 0 0 0) 0x7ff330056510 con 0x7ff33c0abe10 2023-08-09T13:50:17.177 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:17.175+0000 7ff340e37640 1 -- 172.21.15.43:0/688284634 >> [v2:172.21.15.203:6832/3325141227,v1:172.21.15.203:6833/3325141227] conn(0x7ff32806a170 msgr2=0x7ff32806c630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:50:17.178 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:17.175+0000 7ff340e37640 1 --2- 172.21.15.43:0/688284634 >> [v2:172.21.15.203:6832/3325141227,v1:172.21.15.203:6833/3325141227] conn(0x7ff32806a170 0x7ff32806c630 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7ff32c036000 tx=0x7ff32c03b040 comp rx=0 tx=0).stop 2023-08-09T13:50:17.178 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:17.175+0000 7ff340e37640 1 -- 172.21.15.43:0/688284634 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff33c0abe10 msgr2=0x7ff33c1f2010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:50:17.178 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:17.175+0000 7ff340e37640 1 --2- 172.21.15.43:0/688284634 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff33c0abe10 0x7ff33c1f2010 secure :-1 s=READY pgs=956 cs=0 l=1 rev1=1 crypto rx=0x7ff33000c410 tx=0x7ff33000c8d0 comp rx=0 tx=0).stop 2023-08-09T13:50:17.178 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:17.176+0000 7ff340e37640 1 -- 172.21.15.43:0/688284634 shutdown_connections 2023-08-09T13:50:17.178 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:17.176+0000 7ff340e37640 1 --2- 172.21.15.43:0/688284634 >> [v2:172.21.15.203:6832/3325141227,v1:172.21.15.203:6833/3325141227] conn(0x7ff32806a170 0x7ff32806c630 unknown :-1 s=CLOSED pgs=36 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:50:17.178 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:17.176+0000 7ff340e37640 1 --2- 172.21.15.43:0/688284634 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7ff33c0cc0b0 0x7ff33c15cb10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:50:17.178 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:17.176+0000 7ff340e37640 1 --2- 172.21.15.43:0/688284634 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff33c0abe10 0x7ff33c1f2010 unknown :-1 s=CLOSED pgs=956 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:50:17.179 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:17.176+0000 7ff340e37640 1 --2- 172.21.15.43:0/688284634 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7ff33c070ef0 0x7ff33c157190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:50:17.179 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:17.176+0000 7ff340e37640 1 -- 172.21.15.43:0/688284634 >> 172.21.15.43:0/688284634 conn(0x7ff33c0cddd0 msgr2=0x7ff33c15be90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-09T13:50:17.179 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:17.176+0000 7ff340e37640 1 -- 172.21.15.43:0/688284634 shutdown_connections 2023-08-09T13:50:17.179 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:17.176+0000 7ff340e37640 1 -- 172.21.15.43:0/688284634 wait complete. 2023-08-09T13:50:17.195 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config rm mgr mgr/snap_schedule/allow_m_granularity 2023-08-09T13:50:17.419 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:17.418+0000 7f28a6dfc640 1 Processor -- start 2023-08-09T13:50:17.420 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:17.418+0000 7f28a6dfc640 1 -- start start 2023-08-09T13:50:17.421 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:17.419+0000 7f28a6dfc640 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7f28a0070e50 0x7f28a00c9040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:50:17.421 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:17.420+0000 7f28a6dfc640 1 -- --> v1:172.21.15.43:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f28a0071630 con 0x7f28a0070720 2023-08-09T13:50:17.422 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:17.420+0000 7f28a6dfc640 1 -- --> v1:172.21.15.203:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f28a006f680 con 0x7f28a0158940 2023-08-09T13:50:17.422 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:17.420+0000 7f28a6dfc640 1 -- --> v2:172.21.15.203:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f28a006f4a0 con 0x7f28a0070e50 2023-08-09T13:50:17.423 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:17.420+0000 7f28a65fb640 1 --1- >> v1:172.21.15.43:6789/0 conn(0x7f28a0070720 0x7f28a00ad850 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.43:6789/0 says I am v1:172.21.15.43:44594/0 (socket says 172.21.15.43:44594) 2023-08-09T13:50:17.423 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:17.420+0000 7f28a65fb640 1 -- 172.21.15.43:0/154140544 learned_addr learned my addr 172.21.15.43:0/154140544 (peer_addr_for_me v1:172.21.15.43:0/0) 2023-08-09T13:50:17.424 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:17.421+0000 7f28a5dfa640 1 --2- 172.21.15.43:0/154140544 >> v2:172.21.15.203:3300/0 conn(0x7f28a0070e50 0x7f28a00c9040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:50:17.424 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:17.422+0000 7f28a4df8640 1 -- 172.21.15.43:0/154140544 <== mon.0 v1:172.21.15.43:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 17286168 0 0) 0x7f28a0071630 con 0x7f28a0070720 2023-08-09T13:50:17.424 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:17.422+0000 7f28a4df8640 1 -- 172.21.15.43:0/154140544 --> v1:172.21.15.43:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f288c005730 con 0x7f28a0070720 2023-08-09T13:50:17.425 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:17.422+0000 7f28a5dfa640 1 -- 172.21.15.43:0/154140544 >> v1:172.21.15.203:6789/0 conn(0x7f28a0158940 legacy=0x7f28a00cb9a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:50:17.426 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:17.422+0000 7f28a5dfa640 1 -- 172.21.15.43:0/154140544 >> v1:172.21.15.43:6789/0 conn(0x7f28a0070720 legacy=0x7f28a00ad850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:50:17.426 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:17.422+0000 7f28a5dfa640 1 -- 172.21.15.43:0/154140544 --> v2:172.21.15.203:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f28a006f960 con 0x7f28a0070e50 2023-08-09T13:50:17.426 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:17.425+0000 7f28a5dfa640 1 --2- 172.21.15.43:0/154140544 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f28a0070e50 0x7f28a00c9040 secure :-1 s=READY pgs=344 cs=0 l=1 rev1=1 crypto rx=0x7f289c0097a0 tx=0x7f289c02b8d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=72a5f695112943ea server_cookie=0 in_seq=0 out_seq=0 2023-08-09T13:50:17.427 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:17.425+0000 7f28a4df8640 1 -- 172.21.15.43:0/154140544 <== mon.1 v2:172.21.15.203:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f289c031830 con 0x7f28a0070e50 2023-08-09T13:50:17.427 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:17.425+0000 7f28a4df8640 1 -- 172.21.15.43:0/154140544 <== mon.1 v2:172.21.15.203:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f289c031990 con 0x7f28a0070e50 2023-08-09T13:50:17.427 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:17.425+0000 7f28a4df8640 1 -- 172.21.15.43:0/154140544 <== mon.1 v2:172.21.15.203:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f289c031d20 con 0x7f28a0070e50 2023-08-09T13:50:17.428 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:17.426+0000 7f28a6dfc640 1 -- 172.21.15.43:0/154140544 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f28a0070e50 msgr2=0x7f28a00c9040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:50:17.428 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:17.426+0000 7f28a6dfc640 1 --2- 172.21.15.43:0/154140544 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f28a0070e50 0x7f28a00c9040 secure :-1 s=READY pgs=344 cs=0 l=1 rev1=1 crypto rx=0x7f289c0097a0 tx=0x7f289c02b8d0 comp rx=0 tx=0).stop 2023-08-09T13:50:17.428 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:17.426+0000 7f28a6dfc640 1 -- 172.21.15.43:0/154140544 shutdown_connections 2023-08-09T13:50:17.428 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:17.426+0000 7f28a6dfc640 1 --2- 172.21.15.43:0/154140544 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f28a0070e50 0x7f28a00c9040 unknown :-1 s=CLOSED pgs=344 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:50:17.429 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:17.426+0000 7f28a6dfc640 1 -- 172.21.15.43:0/154140544 >> 172.21.15.43:0/154140544 conn(0x7f28a00cddd0 msgr2=0x7f28a00ce1e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-09T13:50:17.429 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:17.427+0000 7f28a6dfc640 1 -- 172.21.15.43:0/154140544 shutdown_connections 2023-08-09T13:50:17.429 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:17.427+0000 7f28a6dfc640 1 -- 172.21.15.43:0/154140544 wait complete. 2023-08-09T13:50:17.430 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:17.429+0000 7f28a6dfc640 1 Processor -- start 2023-08-09T13:50:17.430 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:17.429+0000 7f28a6dfc640 1 -- start start 2023-08-09T13:50:17.431 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:17.430+0000 7f28a6dfc640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f28a0070720 0x7f28a01545c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:50:17.432 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:17.430+0000 7f28a6dfc640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f28a0070e50 0x7f28a01e3140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:50:17.432 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:17.430+0000 7f28a6dfc640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f28a0158940 0x7f28a015a200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:50:17.432 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:17.430+0000 7f28a6dfc640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f28a0154b00 con 0x7f28a0070e50 2023-08-09T13:50:17.432 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:17.430+0000 7f28a6dfc640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f28a00cada0 con 0x7f28a0158940 2023-08-09T13:50:17.432 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:17.430+0000 7f28a6dfc640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f28a00cb080 con 0x7f28a0070720 2023-08-09T13:50:17.432 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:17.430+0000 7f28a55f9640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f28a0070e50 0x7f28a01e3140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:50:17.433 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:17.430+0000 7f28a55f9640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f28a0070e50 0x7f28a01e3140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:44052/0 (socket says 172.21.15.43:44052) 2023-08-09T13:50:17.433 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:17.430+0000 7f28a55f9640 1 -- 172.21.15.43:0/3120947479 learned_addr learned my addr 172.21.15.43:0/3120947479 (peer_addr_for_me v2:172.21.15.43:0/0) 2023-08-09T13:50:17.434 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:17.431+0000 7f28a65fb640 1 --2- 172.21.15.43:0/3120947479 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f28a0158940 0x7f28a015a200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:50:17.434 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:17.431+0000 7f28a5dfa640 1 --2- 172.21.15.43:0/3120947479 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f28a0070720 0x7f28a01545c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:50:17.434 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:17.431+0000 7f28a55f9640 1 -- 172.21.15.43:0/3120947479 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f28a0070720 msgr2=0x7f28a01545c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:50:17.434 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:17.431+0000 7f28a55f9640 1 --2- 172.21.15.43:0/3120947479 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f28a0070720 0x7f28a01545c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:50:17.434 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:17.431+0000 7f28a55f9640 1 -- 172.21.15.43:0/3120947479 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f28a0158940 msgr2=0x7f28a015a200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:50:17.435 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:17.431+0000 7f28a55f9640 1 --2- 172.21.15.43:0/3120947479 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f28a0158940 0x7f28a015a200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:50:17.435 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:17.431+0000 7f28a55f9640 1 -- 172.21.15.43:0/3120947479 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f289c009400 con 0x7f28a0070e50 2023-08-09T13:50:17.435 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:17.431+0000 7f28a55f9640 1 --2- 172.21.15.43:0/3120947479 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f28a0070e50 0x7f28a01e3140 secure :-1 s=READY pgs=958 cs=0 l=1 rev1=1 crypto rx=0x7f289000c3c0 tx=0x7f289000c880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-09T13:50:17.435 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:17.431+0000 7f28967fc640 1 -- 172.21.15.43:0/3120947479 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f28900052d0 con 0x7f28a0070e50 2023-08-09T13:50:17.435 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:17.431+0000 7f28967fc640 1 -- 172.21.15.43:0/3120947479 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2890005450 con 0x7f28a0070e50 2023-08-09T13:50:17.435 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:17.431+0000 7f28967fc640 1 -- 172.21.15.43:0/3120947479 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2890007260 con 0x7f28a0070e50 2023-08-09T13:50:17.435 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:17.431+0000 7f28a6dfc640 1 -- 172.21.15.43:0/3120947479 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f28a00cb360 con 0x7f28a0070e50 2023-08-09T13:50:17.436 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:17.432+0000 7f28a6dfc640 1 -- 172.21.15.43:0/3120947479 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f28a00cb920 con 0x7f28a0070e50 2023-08-09T13:50:17.436 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:17.433+0000 7f28a6dfc640 1 -- 172.21.15.43:0/3120947479 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f28a004e390 con 0x7f28a0070e50 2023-08-09T13:50:17.437 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:17.434+0000 7f28967fc640 1 -- 172.21.15.43:0/3120947479 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 29) v1 ==== 88443+0+0 (secure 0 0 0) 0x7f28900073c0 con 0x7f28a0070e50 2023-08-09T13:50:17.437 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:17.434+0000 7f28967fc640 1 --2- 172.21.15.43:0/3120947479 >> [v2:172.21.15.203:6832/3325141227,v1:172.21.15.203:6833/3325141227] conn(0x7f288406a170 0x7f288406c630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:50:17.437 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:17.435+0000 7f28967fc640 1 -- 172.21.15.43:0/3120947479 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(81..81 src has 1..81) v4 ==== 6434+0+0 (secure 0 0 0) 0x7f2890087c30 con 0x7f28a0070e50 2023-08-09T13:50:17.437 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:17.435+0000 7f28a5dfa640 1 --2- 172.21.15.43:0/3120947479 >> [v2:172.21.15.203:6832/3325141227,v1:172.21.15.203:6833/3325141227] conn(0x7f288406a170 0x7f288406c630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:50:17.438 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:17.436+0000 7f28a5dfa640 1 --2- 172.21.15.43:0/3120947479 >> [v2:172.21.15.203:6832/3325141227,v1:172.21.15.203:6833/3325141227] conn(0x7f288406a170 0x7f288406c630 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7f289c036000 tx=0x7f289c008030 comp rx=0 tx=0).ready entity=mgr.5519 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-09T13:50:17.441 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:17.439+0000 7f28967fc640 1 -- 172.21.15.43:0/3120947479 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178066 (secure 0 0 0) 0x7f2890053080 con 0x7f28a0070e50 2023-08-09T13:50:17.636 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:17.635+0000 7f28a6dfc640 1 -- 172.21.15.43:0/3120947479 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "config rm", "who": "mgr", "name": "mgr/snap_schedule/allow_m_granularity"} v 0) v1 -- 0x7f28a00cea30 con 0x7f28a0070e50 2023-08-09T13:50:17.643 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:17.641+0000 7f28967fc640 1 -- 172.21.15.43:0/3120947479 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "config rm", "who": "mgr", "name": "mgr/snap_schedule/allow_m_granularity"}]=0 v20) v1 ==== 120+0+0 (secure 0 0 0) 0x7f28900566a0 con 0x7f28a0070e50 2023-08-09T13:50:17.646 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:17.644+0000 7f28a6dfc640 1 -- 172.21.15.43:0/3120947479 >> [v2:172.21.15.203:6832/3325141227,v1:172.21.15.203:6833/3325141227] conn(0x7f288406a170 msgr2=0x7f288406c630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:50:17.646 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:17.644+0000 7f28a6dfc640 1 --2- 172.21.15.43:0/3120947479 >> [v2:172.21.15.203:6832/3325141227,v1:172.21.15.203:6833/3325141227] conn(0x7f288406a170 0x7f288406c630 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7f289c036000 tx=0x7f289c008030 comp rx=0 tx=0).stop 2023-08-09T13:50:17.646 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:17.644+0000 7f28a6dfc640 1 -- 172.21.15.43:0/3120947479 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f28a0070e50 msgr2=0x7f28a01e3140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:50:17.646 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:17.644+0000 7f28a6dfc640 1 --2- 172.21.15.43:0/3120947479 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f28a0070e50 0x7f28a01e3140 secure :-1 s=READY pgs=958 cs=0 l=1 rev1=1 crypto rx=0x7f289000c3c0 tx=0x7f289000c880 comp rx=0 tx=0).stop 2023-08-09T13:50:17.647 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:17.644+0000 7f28a6dfc640 1 -- 172.21.15.43:0/3120947479 shutdown_connections 2023-08-09T13:50:17.647 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:17.644+0000 7f28a6dfc640 1 --2- 172.21.15.43:0/3120947479 >> [v2:172.21.15.203:6832/3325141227,v1:172.21.15.203:6833/3325141227] conn(0x7f288406a170 0x7f288406c630 unknown :-1 s=CLOSED pgs=37 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:50:17.647 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:17.644+0000 7f28a6dfc640 1 --2- 172.21.15.43:0/3120947479 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f28a0158940 0x7f28a015a200 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:50:17.647 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:17.644+0000 7f28a6dfc640 1 --2- 172.21.15.43:0/3120947479 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f28a0070e50 0x7f28a01e3140 unknown :-1 s=CLOSED pgs=958 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:50:17.648 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:17.644+0000 7f28a6dfc640 1 --2- 172.21.15.43:0/3120947479 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f28a0070720 0x7f28a01545c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:50:17.648 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:17.644+0000 7f28a6dfc640 1 -- 172.21.15.43:0/3120947479 >> 172.21.15.43:0/3120947479 conn(0x7f28a00cddd0 msgr2=0x7f28a00b09f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-09T13:50:17.648 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:17.645+0000 7f28a6dfc640 1 -- 172.21.15.43:0/3120947479 shutdown_connections 2023-08-09T13:50:17.648 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:17.645+0000 7f28a6dfc640 1 -- 172.21.15.43:0/3120947479 wait complete. 2023-08-09T13:50:17.664 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config rm mon mon_allow_pool_delete 2023-08-09T13:50:17.900 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:17.898+0000 7f183359e640 1 Processor -- start 2023-08-09T13:50:17.900 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:17.898+0000 7f183359e640 1 -- start start 2023-08-09T13:50:17.902 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:17.900+0000 7f183359e640 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7f18340b1160 0x7f1834159740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:50:17.902 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:17.900+0000 7f183359e640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f18340706e0 0x7f183415a900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:50:17.902 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:17.900+0000 7f183359e640 1 -- --> v1:172.21.15.203:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f183415a730 con 0x7f183415ae40 2023-08-09T13:50:17.902 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:17.900+0000 7f183359e640 1 -- --> v2:172.21.15.43:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1834159c80 con 0x7f18340706e0 2023-08-09T13:50:17.902 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:17.900+0000 7f183359e640 1 -- --> v2:172.21.15.203:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f18340ae950 con 0x7f18340b1160 2023-08-09T13:50:17.903 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:17.901+0000 7f1829d9b640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f18340706e0 0x7f183415a900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:50:17.903 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:17.901+0000 7f1829d9b640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f18340706e0 0x7f183415a900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:44066/0 (socket says 172.21.15.43:44066) 2023-08-09T13:50:17.903 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:17.901+0000 7f1829d9b640 1 -- 172.21.15.43:0/39032170 learned_addr learned my addr 172.21.15.43:0/39032170 (peer_addr_for_me v2:172.21.15.43:0/0) 2023-08-09T13:50:17.903 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:17.901+0000 7f183259c640 1 --2- 172.21.15.43:0/39032170 >> v2:172.21.15.203:3300/0 conn(0x7f18340b1160 0x7f1834159740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:50:17.904 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:17.901+0000 7f1829d9b640 1 -- 172.21.15.43:0/39032170 >> v2:172.21.15.203:3300/0 conn(0x7f18340b1160 msgr2=0x7f1834159740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-09T13:50:17.904 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:17.901+0000 7f1829d9b640 1 --2- 172.21.15.43:0/39032170 >> v2:172.21.15.203:3300/0 conn(0x7f18340b1160 0x7f1834159740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:50:17.904 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:17.901+0000 7f1829d9b640 1 -- 172.21.15.43:0/39032170 >> v1:172.21.15.203:6789/0 conn(0x7f183415ae40 legacy=0x7f18340b2530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-09T13:50:17.904 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:17.901+0000 7f1829d9b640 1 -- 172.21.15.43:0/39032170 --> v2:172.21.15.43:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1834159e90 con 0x7f18340706e0 2023-08-09T13:50:17.905 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:17.903+0000 7f1829d9b640 1 --2- 172.21.15.43:0/39032170 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f18340706e0 0x7f183415a900 secure :-1 s=READY pgs=959 cs=0 l=1 rev1=1 crypto rx=0x7f18340b3060 tx=0x7f181c02b880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b86b4a6a361acca5 server_cookie=0 in_seq=0 out_seq=0 2023-08-09T13:50:17.905 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:17.903+0000 7f1831d9b640 1 -- 172.21.15.43:0/39032170 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f181c0317e0 con 0x7f18340706e0 2023-08-09T13:50:17.905 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:17.903+0000 7f1831d9b640 1 -- 172.21.15.43:0/39032170 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f181c031940 con 0x7f18340706e0 2023-08-09T13:50:17.905 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:17.903+0000 7f1831d9b640 1 -- 172.21.15.43:0/39032170 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f181c031cd0 con 0x7f18340706e0 2023-08-09T13:50:17.906 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:17.904+0000 7f183359e640 1 -- 172.21.15.43:0/39032170 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f18340706e0 msgr2=0x7f183415a900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:50:17.906 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:17.904+0000 7f183359e640 1 --2- 172.21.15.43:0/39032170 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f18340706e0 0x7f183415a900 secure :-1 s=READY pgs=959 cs=0 l=1 rev1=1 crypto rx=0x7f18340b3060 tx=0x7f181c02b880 comp rx=0 tx=0).stop 2023-08-09T13:50:17.906 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:17.904+0000 7f183359e640 1 -- 172.21.15.43:0/39032170 shutdown_connections 2023-08-09T13:50:17.906 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:17.904+0000 7f183359e640 1 --2- 172.21.15.43:0/39032170 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f18340706e0 0x7f183415a900 unknown :-1 s=CLOSED pgs=959 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:50:17.906 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:17.904+0000 7f183359e640 1 --2- 172.21.15.43:0/39032170 >> v2:172.21.15.203:3300/0 conn(0x7f18340b1160 0x7f1834159740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:50:17.907 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:17.904+0000 7f183359e640 1 -- 172.21.15.43:0/39032170 >> 172.21.15.43:0/39032170 conn(0x7f18340b4400 msgr2=0x7f18340b47e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-09T13:50:17.907 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:17.904+0000 7f183359e640 1 -- 172.21.15.43:0/39032170 shutdown_connections 2023-08-09T13:50:17.907 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:17.904+0000 7f183359e640 1 -- 172.21.15.43:0/39032170 wait complete. 2023-08-09T13:50:17.907 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:17.906+0000 7f183359e640 1 Processor -- start 2023-08-09T13:50:17.908 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:17.906+0000 7f183359e640 1 -- start start 2023-08-09T13:50:17.909 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:17.907+0000 7f183359e640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f18340706e0 0x7f18341478b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:50:17.909 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:17.907+0000 7f183359e640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f18340b1160 0x7f18340b3120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:50:17.909 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:17.907+0000 7f183359e640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f183415ae40 0x7f18340b3660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:50:17.909 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:17.907+0000 7f183359e640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f18340cd170 con 0x7f183415ae40 2023-08-09T13:50:17.909 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:17.907+0000 7f183359e640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f18340ce760 con 0x7f18340706e0 2023-08-09T13:50:17.910 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:17.907+0000 7f183359e640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f18340cc860 con 0x7f18340b1160 2023-08-09T13:50:17.910 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:17.907+0000 7f1832d9d640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f183415ae40 0x7f18340b3660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:50:17.910 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:17.908+0000 7f1832d9d640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f183415ae40 0x7f18340b3660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:44080/0 (socket says 172.21.15.43:44080) 2023-08-09T13:50:17.910 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:17.908+0000 7f1832d9d640 1 -- 172.21.15.43:0/3935839513 learned_addr learned my addr 172.21.15.43:0/3935839513 (peer_addr_for_me v2:172.21.15.43:0/0) 2023-08-09T13:50:17.911 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:17.908+0000 7f183259c640 1 --2- 172.21.15.43:0/3935839513 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f18340706e0 0x7f18341478b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:50:17.912 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:17.908+0000 7f1832d9d640 1 -- 172.21.15.43:0/3935839513 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f18340b1160 msgr2=0x7f18340b3120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:50:17.912 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:17.908+0000 7f1832d9d640 1 --2- 172.21.15.43:0/3935839513 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f18340b1160 0x7f18340b3120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:50:17.912 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:17.908+0000 7f1832d9d640 1 -- 172.21.15.43:0/3935839513 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f18340706e0 msgr2=0x7f18341478b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:50:17.912 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:17.908+0000 7f1832d9d640 1 --2- 172.21.15.43:0/3935839513 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f18340706e0 0x7f18341478b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:50:17.912 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:17.908+0000 7f1832d9d640 1 -- 172.21.15.43:0/3935839513 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f181c0094d0 con 0x7f183415ae40 2023-08-09T13:50:17.912 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:17.908+0000 7f183259c640 1 --2- 172.21.15.43:0/3935839513 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f18340706e0 0x7f18341478b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-08-09T13:50:17.913 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:17.908+0000 7f1832d9d640 1 --2- 172.21.15.43:0/3935839513 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f183415ae40 0x7f18340b3660 secure :-1 s=READY pgs=960 cs=0 l=1 rev1=1 crypto rx=0x7f182c00b970 tx=0x7f182c00be30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-09T13:50:17.913 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:17.909+0000 7f182b7fe640 1 -- 172.21.15.43:0/3935839513 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f182c00cb40 con 0x7f183415ae40 2023-08-09T13:50:17.913 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:17.909+0000 7f182b7fe640 1 -- 172.21.15.43:0/3935839513 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f182c00cca0 con 0x7f183415ae40 2023-08-09T13:50:17.913 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:17.909+0000 7f182b7fe640 1 -- 172.21.15.43:0/3935839513 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f182c014680 con 0x7f183415ae40 2023-08-09T13:50:17.913 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:17.909+0000 7f183359e640 1 -- 172.21.15.43:0/3935839513 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f18340ce480 con 0x7f183415ae40 2023-08-09T13:50:17.913 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:17.909+0000 7f183359e640 1 -- 172.21.15.43:0/3935839513 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f18340ccd90 con 0x7f183415ae40 2023-08-09T13:50:17.913 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:17.910+0000 7f183359e640 1 -- 172.21.15.43:0/3935839513 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f18340cd2b0 con 0x7f183415ae40 2023-08-09T13:50:17.914 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:17.911+0000 7f182b7fe640 1 -- 172.21.15.43:0/3935839513 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 29) v1 ==== 88443+0+0 (secure 0 0 0) 0x7f182c01d3b0 con 0x7f183415ae40 2023-08-09T13:50:17.915 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:17.912+0000 7f182b7fe640 1 --2- 172.21.15.43:0/3935839513 >> [v2:172.21.15.203:6832/3325141227,v1:172.21.15.203:6833/3325141227] conn(0x7f181406a170 0x7f181406c630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:50:17.915 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:17.912+0000 7f182b7fe640 1 -- 172.21.15.43:0/3935839513 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(81..81 src has 1..81) v4 ==== 6434+0+0 (secure 0 0 0) 0x7f182c0886a0 con 0x7f183415ae40 2023-08-09T13:50:17.915 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:17.913+0000 7f183259c640 1 --2- 172.21.15.43:0/3935839513 >> [v2:172.21.15.203:6832/3325141227,v1:172.21.15.203:6833/3325141227] conn(0x7f181406a170 0x7f181406c630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:50:17.915 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:17.914+0000 7f183259c640 1 --2- 172.21.15.43:0/3935839513 >> [v2:172.21.15.203:6832/3325141227,v1:172.21.15.203:6833/3325141227] conn(0x7f181406a170 0x7f181406c630 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7f1824006fd0 tx=0x7f1824008040 comp rx=0 tx=0).ready entity=mgr.5519 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-09T13:50:17.918 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:17.916+0000 7f182b7fe640 1 -- 172.21.15.43:0/3935839513 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178066 (secure 0 0 0) 0x7f182c057580 con 0x7f183415ae40 2023-08-09T13:50:18.112 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:18.111+0000 7f183359e640 1 -- 172.21.15.43:0/3935839513 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"} v 0) v1 -- 0x7f1834070c00 con 0x7f183415ae40 2023-08-09T13:50:18.119 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:18.117+0000 7f182b7fe640 1 -- 172.21.15.43:0/3935839513 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"}]=0 v21) v1 ==== 104+0+0 (secure 0 0 0) 0x7f182c057110 con 0x7f183415ae40 2023-08-09T13:50:18.121 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:18.120+0000 7f183359e640 1 -- 172.21.15.43:0/3935839513 >> [v2:172.21.15.203:6832/3325141227,v1:172.21.15.203:6833/3325141227] conn(0x7f181406a170 msgr2=0x7f181406c630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:50:18.122 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:18.120+0000 7f183359e640 1 --2- 172.21.15.43:0/3935839513 >> [v2:172.21.15.203:6832/3325141227,v1:172.21.15.203:6833/3325141227] conn(0x7f181406a170 0x7f181406c630 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7f1824006fd0 tx=0x7f1824008040 comp rx=0 tx=0).stop 2023-08-09T13:50:18.122 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:18.120+0000 7f183359e640 1 -- 172.21.15.43:0/3935839513 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f183415ae40 msgr2=0x7f18340b3660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:50:18.122 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:18.120+0000 7f183359e640 1 --2- 172.21.15.43:0/3935839513 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f183415ae40 0x7f18340b3660 secure :-1 s=READY pgs=960 cs=0 l=1 rev1=1 crypto rx=0x7f182c00b970 tx=0x7f182c00be30 comp rx=0 tx=0).stop 2023-08-09T13:50:18.123 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:18.120+0000 7f183359e640 1 -- 172.21.15.43:0/3935839513 shutdown_connections 2023-08-09T13:50:18.123 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:18.120+0000 7f183359e640 1 --2- 172.21.15.43:0/3935839513 >> [v2:172.21.15.203:6832/3325141227,v1:172.21.15.203:6833/3325141227] conn(0x7f181406a170 0x7f181406c630 unknown :-1 s=CLOSED pgs=38 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:50:18.123 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:18.120+0000 7f183359e640 1 --2- 172.21.15.43:0/3935839513 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f183415ae40 0x7f18340b3660 unknown :-1 s=CLOSED pgs=960 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:50:18.123 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:18.120+0000 7f183359e640 1 --2- 172.21.15.43:0/3935839513 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f18340b1160 0x7f18340b3120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:50:18.123 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:18.120+0000 7f183359e640 1 --2- 172.21.15.43:0/3935839513 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f18340706e0 0x7f18341478b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:50:18.123 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:18.120+0000 7f183359e640 1 -- 172.21.15.43:0/3935839513 >> 172.21.15.43:0/3935839513 conn(0x7f18340b4400 msgr2=0x7f183415bca0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-09T13:50:18.124 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:18.121+0000 7f183359e640 1 -- 172.21.15.43:0/3935839513 shutdown_connections 2023-08-09T13:50:18.124 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:18.121+0000 7f183359e640 1 -- 172.21.15.43:0/3935839513 wait complete. 2023-08-09T13:50:18.141 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Ended test tasks.cephfs.test_snap_schedules.TestSnapSchedules.test_multi_snap_schedule' 2023-08-09T13:50:18.373 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:18.372+0000 7fedc53a4640 1 Processor -- start 2023-08-09T13:50:18.373 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:18.372+0000 7fedc53a4640 1 -- start start 2023-08-09T13:50:18.375 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:18.373+0000 7fedc53a4640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7fedc00cd6b0 0x7fedc00b4760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:50:18.375 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:18.373+0000 7fedc53a4640 1 -- --> v1:172.21.15.43:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fedc00b7bf0 con 0x7fedc00b1050 2023-08-09T13:50:18.375 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:18.373+0000 7fedc53a4640 1 -- --> v1:172.21.15.203:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fedc00b7ed0 con 0x7fedc015f7b0 2023-08-09T13:50:18.375 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:18.373+0000 7fedc53a4640 1 -- --> v2:172.21.15.43:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fedc00d0370 con 0x7fedc00cd6b0 2023-08-09T13:50:18.375 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:18.373+0000 7fedbf7fe640 1 --1- >> v1:172.21.15.43:6789/0 conn(0x7fedc00b1050 0x7fedc00d0e50 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.43:6789/0 says I am v1:172.21.15.43:44604/0 (socket says 172.21.15.43:44604) 2023-08-09T13:50:18.375 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:18.374+0000 7fedbf7fe640 1 -- 172.21.15.43:0/3961884749 learned_addr learned my addr 172.21.15.43:0/3961884749 (peer_addr_for_me v1:172.21.15.43:0/0) 2023-08-09T13:50:18.376 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:18.374+0000 7fedc4ba3640 1 --2- 172.21.15.43:0/3961884749 >> v2:172.21.15.43:3300/0 conn(0x7fedc00cd6b0 0x7fedc00b4760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:50:18.376 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:18.374+0000 7fedbeffd640 1 -- 172.21.15.43:0/3961884749 <== mon.0 v1:172.21.15.43:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3125160518 0 0) 0x7fedc00b7bf0 con 0x7fedc00b1050 2023-08-09T13:50:18.376 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:18.374+0000 7fedbeffd640 1 -- 172.21.15.43:0/3961884749 --> v1:172.21.15.43:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7feda8003520 con 0x7fedc00b1050 2023-08-09T13:50:18.377 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:18.374+0000 7fedc4ba3640 1 -- 172.21.15.43:0/3961884749 >> v1:172.21.15.203:6789/0 conn(0x7fedc015f7b0 legacy=0x7fedc00ccfa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:50:18.377 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:18.374+0000 7fedc4ba3640 1 -- 172.21.15.43:0/3961884749 >> v1:172.21.15.43:6789/0 conn(0x7fedc00b1050 legacy=0x7fedc00d0e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:50:18.377 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:18.374+0000 7fedc4ba3640 1 -- 172.21.15.43:0/3961884749 --> v2:172.21.15.43:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fedc00d0650 con 0x7fedc00cd6b0 2023-08-09T13:50:18.378 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:18.375+0000 7fedc4ba3640 1 --2- 172.21.15.43:0/3961884749 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fedc00cd6b0 0x7fedc00b4760 secure :-1 s=READY pgs=962 cs=0 l=1 rev1=1 crypto rx=0x7fedb80097a0 tx=0x7fedb802b960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f57a11bf6c61edca server_cookie=0 in_seq=0 out_seq=0 2023-08-09T13:50:18.378 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:18.376+0000 7fedbeffd640 1 -- 172.21.15.43:0/3961884749 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fedb8030610 con 0x7fedc00cd6b0 2023-08-09T13:50:18.378 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:18.376+0000 7fedbeffd640 1 -- 172.21.15.43:0/3961884749 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fedb8030770 con 0x7fedc00cd6b0 2023-08-09T13:50:18.378 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:18.376+0000 7fedbeffd640 1 -- 172.21.15.43:0/3961884749 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fedb8030ae0 con 0x7fedc00cd6b0 2023-08-09T13:50:18.379 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:18.376+0000 7fedc53a4640 1 -- 172.21.15.43:0/3961884749 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fedc00cd6b0 msgr2=0x7fedc00b4760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:50:18.379 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:18.376+0000 7fedc53a4640 1 --2- 172.21.15.43:0/3961884749 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fedc00cd6b0 0x7fedc00b4760 secure :-1 s=READY pgs=962 cs=0 l=1 rev1=1 crypto rx=0x7fedb80097a0 tx=0x7fedb802b960 comp rx=0 tx=0).stop 2023-08-09T13:50:18.379 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:18.376+0000 7fedc53a4640 1 -- 172.21.15.43:0/3961884749 shutdown_connections 2023-08-09T13:50:18.379 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:18.376+0000 7fedc53a4640 1 --2- 172.21.15.43:0/3961884749 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fedc00cd6b0 0x7fedc00b4760 unknown :-1 s=CLOSED pgs=962 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:50:18.379 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:18.376+0000 7fedc53a4640 1 -- 172.21.15.43:0/3961884749 >> 172.21.15.43:0/3961884749 conn(0x7fedc006a2b0 msgr2=0x7fedc006a690 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-09T13:50:18.379 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:18.376+0000 7fedc53a4640 1 -- 172.21.15.43:0/3961884749 shutdown_connections 2023-08-09T13:50:18.379 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:18.376+0000 7fedc53a4640 1 -- 172.21.15.43:0/3961884749 wait complete. 2023-08-09T13:50:18.380 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:18.378+0000 7fedc53a4640 1 Processor -- start 2023-08-09T13:50:18.380 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:18.378+0000 7fedc53a4640 1 -- start start 2023-08-09T13:50:18.381 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:18.379+0000 7fedc53a4640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fedc00b1050 0x7fedc014c7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:50:18.381 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:18.379+0000 7fedc53a4640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fedc00cd6b0 0x7fedc0154be0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:50:18.381 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:18.379+0000 7fedc53a4640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fedc015f7b0 0x7fedc015c9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:50:18.381 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:18.379+0000 7fedc53a4640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fedc01f60b0 con 0x7fedc00b1050 2023-08-09T13:50:18.381 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:18.379+0000 7fedc53a4640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fedc014cce0 con 0x7fedc015f7b0 2023-08-09T13:50:18.382 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:18.379+0000 7fedc53a4640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fedc01f54a0 con 0x7fedc00cd6b0 2023-08-09T13:50:18.382 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:18.380+0000 7fedbffff640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fedc00b1050 0x7fedc014c7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:50:18.382 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:18.380+0000 7fedbffff640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fedc00b1050 0x7fedc014c7a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:44088/0 (socket says 172.21.15.43:44088) 2023-08-09T13:50:18.382 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:18.380+0000 7fedbffff640 1 -- 172.21.15.43:0/430504939 learned_addr learned my addr 172.21.15.43:0/430504939 (peer_addr_for_me v2:172.21.15.43:0/0) 2023-08-09T13:50:18.383 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:18.380+0000 7fedc4ba3640 1 --2- 172.21.15.43:0/430504939 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fedc015f7b0 0x7fedc015c9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:50:18.383 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:18.380+0000 7fedbf7fe640 1 --2- 172.21.15.43:0/430504939 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fedc00cd6b0 0x7fedc0154be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:50:18.383 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:18.380+0000 7fedbffff640 1 -- 172.21.15.43:0/430504939 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fedc00cd6b0 msgr2=0x7fedc0154be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:50:18.383 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:18.380+0000 7fedbffff640 1 --2- 172.21.15.43:0/430504939 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fedc00cd6b0 0x7fedc0154be0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:50:18.384 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:18.380+0000 7fedbffff640 1 -- 172.21.15.43:0/430504939 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fedc015f7b0 msgr2=0x7fedc015c9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:50:18.384 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:18.380+0000 7fedbffff640 1 --2- 172.21.15.43:0/430504939 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fedc015f7b0 0x7fedc015c9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:50:18.384 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:18.380+0000 7fedbffff640 1 -- 172.21.15.43:0/430504939 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fedb8009400 con 0x7fedc00b1050 2023-08-09T13:50:18.384 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:18.380+0000 7fedc4ba3640 1 --2- 172.21.15.43:0/430504939 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fedc015f7b0 0x7fedc015c9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-08-09T13:50:18.384 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:18.380+0000 7fedbffff640 1 --2- 172.21.15.43:0/430504939 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fedc00b1050 0x7fedc014c7a0 secure :-1 s=READY pgs=963 cs=0 l=1 rev1=1 crypto rx=0x7fedb000c730 tx=0x7fedb000cbf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-09T13:50:18.384 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:18.381+0000 7fedbcff9640 1 -- 172.21.15.43:0/430504939 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fedb0003b70 con 0x7fedc00b1050 2023-08-09T13:50:18.384 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:18.381+0000 7fedbcff9640 1 -- 172.21.15.43:0/430504939 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fedb0003cf0 con 0x7fedc00b1050 2023-08-09T13:50:18.385 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:18.381+0000 7fedbcff9640 1 -- 172.21.15.43:0/430504939 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fedb0003ea0 con 0x7fedc00b1050 2023-08-09T13:50:18.385 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:18.381+0000 7fedc53a4640 1 -- 172.21.15.43:0/430504939 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fedc01f5780 con 0x7fedc00b1050 2023-08-09T13:50:18.385 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:18.381+0000 7fedc53a4640 1 -- 172.21.15.43:0/430504939 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fedc01f5d40 con 0x7fedc00b1050 2023-08-09T13:50:18.385 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:18.382+0000 7fedc53a4640 1 -- 172.21.15.43:0/430504939 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fedc00739f0 con 0x7fedc00b1050 2023-08-09T13:50:18.386 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:18.383+0000 7fedbcff9640 1 -- 172.21.15.43:0/430504939 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 29) v1 ==== 88443+0+0 (secure 0 0 0) 0x7fedb000aa90 con 0x7fedc00b1050 2023-08-09T13:50:18.386 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:18.384+0000 7fedbcff9640 1 --2- 172.21.15.43:0/430504939 >> [v2:172.21.15.203:6832/3325141227,v1:172.21.15.203:6833/3325141227] conn(0x7fedac06a170 0x7fedac06c630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:50:18.386 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:18.384+0000 7fedbcff9640 1 -- 172.21.15.43:0/430504939 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(81..81 src has 1..81) v4 ==== 6434+0+0 (secure 0 0 0) 0x7fedb00886d0 con 0x7fedc00b1050 2023-08-09T13:50:18.386 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:18.385+0000 7fedbf7fe640 1 --2- 172.21.15.43:0/430504939 >> [v2:172.21.15.203:6832/3325141227,v1:172.21.15.203:6833/3325141227] conn(0x7fedac06a170 0x7fedac06c630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:50:18.387 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:18.386+0000 7fedbf7fe640 1 --2- 172.21.15.43:0/430504939 >> [v2:172.21.15.203:6832/3325141227,v1:172.21.15.203:6833/3325141227] conn(0x7fedac06a170 0x7fedac06c630 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7fedc01539e0 tx=0x7fedb4009040 comp rx=0 tx=0).ready entity=mgr.5519 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-09T13:50:18.389 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:18.388+0000 7fedbcff9640 1 -- 172.21.15.43:0/430504939 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178066 (secure 0 0 0) 0x7fedb0053b20 con 0x7fedc00b1050 2023-08-09T13:50:18.654 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:18.653+0000 7fedc53a4640 1 -- 172.21.15.43:0/430504939 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_snap_schedules.TestSnapSchedules.test_multi_snap_schedule"]} v 0) v1 -- 0x7fedc00b4940 con 0x7fedc00b1050 2023-08-09T13:50:18.708 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:18.707+0000 7fedbcff9640 1 -- 172.21.15.43:0/430504939 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_snap_schedules.TestSnapSchedules.test_multi_snap_schedule"]}]=0 v612) v1 ==== 154+0+0 (secure 0 0 0) 0x7fedb0057140 con 0x7fedc00b1050 2023-08-09T13:50:18.711 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:18.709+0000 7fedc53a4640 1 -- 172.21.15.43:0/430504939 >> [v2:172.21.15.203:6832/3325141227,v1:172.21.15.203:6833/3325141227] conn(0x7fedac06a170 msgr2=0x7fedac06c630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:50:18.711 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:18.709+0000 7fedc53a4640 1 --2- 172.21.15.43:0/430504939 >> [v2:172.21.15.203:6832/3325141227,v1:172.21.15.203:6833/3325141227] conn(0x7fedac06a170 0x7fedac06c630 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7fedc01539e0 tx=0x7fedb4009040 comp rx=0 tx=0).stop 2023-08-09T13:50:18.712 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:18.709+0000 7fedc53a4640 1 -- 172.21.15.43:0/430504939 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fedc00b1050 msgr2=0x7fedc014c7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:50:18.712 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:18.709+0000 7fedc53a4640 1 --2- 172.21.15.43:0/430504939 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fedc00b1050 0x7fedc014c7a0 secure :-1 s=READY pgs=963 cs=0 l=1 rev1=1 crypto rx=0x7fedb000c730 tx=0x7fedb000cbf0 comp rx=0 tx=0).stop 2023-08-09T13:50:18.712 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:18.710+0000 7fedc53a4640 1 -- 172.21.15.43:0/430504939 shutdown_connections 2023-08-09T13:50:18.712 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:18.710+0000 7fedc53a4640 1 --2- 172.21.15.43:0/430504939 >> [v2:172.21.15.203:6832/3325141227,v1:172.21.15.203:6833/3325141227] conn(0x7fedac06a170 0x7fedac06c630 unknown :-1 s=CLOSED pgs=39 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:50:18.712 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:18.710+0000 7fedc53a4640 1 --2- 172.21.15.43:0/430504939 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fedc015f7b0 0x7fedc015c9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:50:18.712 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:18.710+0000 7fedc53a4640 1 --2- 172.21.15.43:0/430504939 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fedc00cd6b0 0x7fedc0154be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:50:18.713 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:18.710+0000 7fedc53a4640 1 --2- 172.21.15.43:0/430504939 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fedc00b1050 0x7fedc014c7a0 unknown :-1 s=CLOSED pgs=963 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:50:18.713 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:18.710+0000 7fedc53a4640 1 -- 172.21.15.43:0/430504939 >> 172.21.15.43:0/430504939 conn(0x7fedc006a2b0 msgr2=0x7fedc0156840 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-09T13:50:18.713 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:18.710+0000 7fedc53a4640 1 -- 172.21.15.43:0/430504939 shutdown_connections 2023-08-09T13:50:18.713 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:18.710+0000 7fedc53a4640 1 -- 172.21.15.43:0/430504939 wait complete. 2023-08-09T13:50:18.730 INFO:tasks.cephfs_test_runner:Test exisitence of multiple scheduled snapshots ... ok 2023-08-09T13:50:18.731 INFO:tasks.cephfs_test_runner:Starting test: test_non_existent_schedule (tasks.cephfs.test_snap_schedules.TestSnapSchedules) Test listing non-existing snap schedules failure 2023-08-09T13:50:18.731 INFO:tasks.cephfs_test_runner:test_non_existent_schedule (tasks.cephfs.test_snap_schedules.TestSnapSchedules) 2023-08-09T13:50:18.731 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Starting test tasks.cephfs.test_snap_schedules.TestSnapSchedules.test_non_existent_schedule' 2023-08-09T13:50:18.960 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:18.959+0000 7f3514a06640 1 Processor -- start 2023-08-09T13:50:18.961 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:18.959+0000 7f3514a06640 1 -- start start 2023-08-09T13:50:18.962 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:18.961+0000 7f3514a06640 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7f3510143c20 0x7f351006f540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:50:18.963 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:18.961+0000 7f3514a06640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f351006cfe0 0x7f35100c8b50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:50:18.963 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:18.961+0000 7f3514a06640 1 -- --> v1:172.21.15.203:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f35100c9090 con 0x7f35100c86f0 2023-08-09T13:50:18.963 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:18.961+0000 7f3514a06640 1 -- --> v2:172.21.15.43:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3510144000 con 0x7f351006cfe0 2023-08-09T13:50:18.963 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:18.961+0000 7f3514a06640 1 -- --> v2:172.21.15.203:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f351006c440 con 0x7f3510143c20 2023-08-09T13:50:18.964 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:18.961+0000 7f350ed9d640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f351006cfe0 0x7f35100c8b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:50:18.964 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:18.961+0000 7f350ed9d640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f351006cfe0 0x7f35100c8b50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:44104/0 (socket says 172.21.15.43:44104) 2023-08-09T13:50:18.965 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:18.962+0000 7f350ed9d640 1 -- 172.21.15.43:0/3171235216 learned_addr learned my addr 172.21.15.43:0/3171235216 (peer_addr_for_me v2:172.21.15.43:0/0) 2023-08-09T13:50:18.965 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:18.962+0000 7f350e59c640 1 --2- 172.21.15.43:0/3171235216 >> v2:172.21.15.203:3300/0 conn(0x7f3510143c20 0x7f351006f540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:50:18.966 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:18.962+0000 7f350ed9d640 1 -- 172.21.15.43:0/3171235216 >> v2:172.21.15.203:3300/0 conn(0x7f3510143c20 msgr2=0x7f351006f540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-09T13:50:18.966 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:18.962+0000 7f350ed9d640 1 --2- 172.21.15.43:0/3171235216 >> v2:172.21.15.203:3300/0 conn(0x7f3510143c20 0x7f351006f540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:50:18.966 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:18.962+0000 7f350ed9d640 1 -- 172.21.15.43:0/3171235216 >> v1:172.21.15.203:6789/0 conn(0x7f35100c86f0 legacy=0x7f351006c8d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:50:18.966 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:18.962+0000 7f350ed9d640 1 -- 172.21.15.43:0/3171235216 --> v2:172.21.15.43:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f351006b830 con 0x7f351006cfe0 2023-08-09T13:50:18.967 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:18.963+0000 7f350ed9d640 1 --2- 172.21.15.43:0/3171235216 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f351006cfe0 0x7f35100c8b50 secure :-1 s=READY pgs=964 cs=0 l=1 rev1=1 crypto rx=0x7f351006e0f0 tx=0x7f350802be50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a0d6818bc24e86fe server_cookie=0 in_seq=0 out_seq=0 2023-08-09T13:50:18.967 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:18.964+0000 7f350d59a640 1 -- 172.21.15.43:0/3171235216 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f350802c9f0 con 0x7f351006cfe0 2023-08-09T13:50:18.967 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:18.964+0000 7f350d59a640 1 -- 172.21.15.43:0/3171235216 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f350802cb50 con 0x7f351006cfe0 2023-08-09T13:50:18.968 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:18.964+0000 7f350d59a640 1 -- 172.21.15.43:0/3171235216 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f35080343f0 con 0x7f351006cfe0 2023-08-09T13:50:18.968 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:18.964+0000 7f3514a06640 1 -- 172.21.15.43:0/3171235216 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f351006cfe0 msgr2=0x7f35100c8b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:50:18.968 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:18.964+0000 7f3514a06640 1 --2- 172.21.15.43:0/3171235216 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f351006cfe0 0x7f35100c8b50 secure :-1 s=READY pgs=964 cs=0 l=1 rev1=1 crypto rx=0x7f351006e0f0 tx=0x7f350802be50 comp rx=0 tx=0).stop 2023-08-09T13:50:18.968 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:18.965+0000 7f3514a06640 1 -- 172.21.15.43:0/3171235216 shutdown_connections 2023-08-09T13:50:18.968 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:18.965+0000 7f3514a06640 1 --2- 172.21.15.43:0/3171235216 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f351006cfe0 0x7f35100c8b50 unknown :-1 s=CLOSED pgs=964 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:50:18.968 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:18.965+0000 7f3514a06640 1 --2- 172.21.15.43:0/3171235216 >> v2:172.21.15.203:3300/0 conn(0x7f3510143c20 0x7f351006f540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:50:18.968 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:18.965+0000 7f3514a06640 1 -- 172.21.15.43:0/3171235216 >> 172.21.15.43:0/3171235216 conn(0x7f35100ce0a0 msgr2=0x7f35100ce480 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-09T13:50:18.969 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:18.965+0000 7f3514a06640 1 -- 172.21.15.43:0/3171235216 shutdown_connections 2023-08-09T13:50:18.969 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:18.965+0000 7f3514a06640 1 -- 172.21.15.43:0/3171235216 wait complete. 2023-08-09T13:50:18.970 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:18.966+0000 7f3514a06640 1 Processor -- start 2023-08-09T13:50:18.970 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:18.967+0000 7f3514a06640 1 -- start start 2023-08-09T13:50:18.970 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:18.968+0000 7f3514a06640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f351006cfe0 0x7f3510159c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:50:18.970 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:18.968+0000 7f3514a06640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f35100c86f0 0x7f351015cbf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:50:18.971 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:18.968+0000 7f3514a06640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f3510143c20 0x7f35100c7c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:50:18.971 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:18.968+0000 7f3514a06640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f351015bb50 con 0x7f3510143c20 2023-08-09T13:50:18.971 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:18.968+0000 7f3514a06640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f351015be30 con 0x7f351006cfe0 2023-08-09T13:50:18.971 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:18.968+0000 7f3514a06640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f351015c110 con 0x7f35100c86f0 2023-08-09T13:50:18.971 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:18.968+0000 7f350ed9d640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f3510143c20 0x7f35100c7c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:50:18.971 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:18.968+0000 7f350ed9d640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f3510143c20 0x7f35100c7c60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:44114/0 (socket says 172.21.15.43:44114) 2023-08-09T13:50:18.971 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:18.968+0000 7f350ed9d640 1 -- 172.21.15.43:0/1734814515 learned_addr learned my addr 172.21.15.43:0/1734814515 (peer_addr_for_me v2:172.21.15.43:0/0) 2023-08-09T13:50:18.972 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:18.968+0000 7f350e59c640 1 --2- 172.21.15.43:0/1734814515 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f351006cfe0 0x7f3510159c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:50:18.972 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:18.968+0000 7f350dd9b640 1 --2- 172.21.15.43:0/1734814515 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f35100c86f0 0x7f351015cbf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:50:18.972 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:18.968+0000 7f350ed9d640 1 -- 172.21.15.43:0/1734814515 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f35100c86f0 msgr2=0x7f351015cbf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:50:18.972 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:18.968+0000 7f350ed9d640 1 --2- 172.21.15.43:0/1734814515 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f35100c86f0 0x7f351015cbf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:50:18.972 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:18.968+0000 7f350ed9d640 1 -- 172.21.15.43:0/1734814515 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f351006cfe0 msgr2=0x7f3510159c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:50:18.972 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:18.968+0000 7f350ed9d640 1 --2- 172.21.15.43:0/1734814515 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f351006cfe0 0x7f3510159c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:50:18.973 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:18.969+0000 7f350ed9d640 1 -- 172.21.15.43:0/1734814515 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3508009b30 con 0x7f3510143c20 2023-08-09T13:50:18.973 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:18.969+0000 7f350dd9b640 1 --2- 172.21.15.43:0/1734814515 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f35100c86f0 0x7f351015cbf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-08-09T13:50:18.973 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:18.969+0000 7f350ed9d640 1 --2- 172.21.15.43:0/1734814515 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f3510143c20 0x7f35100c7c60 secure :-1 s=READY pgs=965 cs=0 l=1 rev1=1 crypto rx=0x7f3508036040 tx=0x7f3508004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-09T13:50:18.974 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:18.969+0000 7f34feffd640 1 -- 172.21.15.43:0/1734814515 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3508009510 con 0x7f3510143c20 2023-08-09T13:50:18.974 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:18.969+0000 7f34feffd640 1 -- 172.21.15.43:0/1734814515 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f350803e450 con 0x7f3510143c20 2023-08-09T13:50:18.975 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:18.969+0000 7f34feffd640 1 -- 172.21.15.43:0/1734814515 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f350803e800 con 0x7f3510143c20 2023-08-09T13:50:18.975 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:18.969+0000 7f3514a06640 1 -- 172.21.15.43:0/1734814515 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f351015c3f0 con 0x7f3510143c20 2023-08-09T13:50:18.975 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:18.969+0000 7f3514a06640 1 -- 172.21.15.43:0/1734814515 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f351015a1c0 con 0x7f3510143c20 2023-08-09T13:50:18.975 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:18.971+0000 7f3514a06640 1 -- 172.21.15.43:0/1734814515 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f351006c0b0 con 0x7f3510143c20 2023-08-09T13:50:18.975 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:18.972+0000 7f34feffd640 1 -- 172.21.15.43:0/1734814515 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 29) v1 ==== 88443+0+0 (secure 0 0 0) 0x7f350803e960 con 0x7f3510143c20 2023-08-09T13:50:18.975 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:18.972+0000 7f34feffd640 1 --2- 172.21.15.43:0/1734814515 >> [v2:172.21.15.203:6832/3325141227,v1:172.21.15.203:6833/3325141227] conn(0x7f34f006a170 0x7f34f006c630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:50:18.975 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:18.973+0000 7f34feffd640 1 -- 172.21.15.43:0/1734814515 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(81..81 src has 1..81) v4 ==== 6434+0+0 (secure 0 0 0) 0x7f35080ab570 con 0x7f3510143c20 2023-08-09T13:50:18.976 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:18.973+0000 7f350e59c640 1 --2- 172.21.15.43:0/1734814515 >> [v2:172.21.15.203:6832/3325141227,v1:172.21.15.203:6833/3325141227] conn(0x7f34f006a170 0x7f34f006c630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:50:18.976 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:18.974+0000 7f350e59c640 1 --2- 172.21.15.43:0/1734814515 >> [v2:172.21.15.203:6832/3325141227,v1:172.21.15.203:6833/3325141227] conn(0x7f34f006a170 0x7f34f006c630 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7f3500009a30 tx=0x7f3500008040 comp rx=0 tx=0).ready entity=mgr.5519 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-09T13:50:18.978 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:18.977+0000 7f34feffd640 1 -- 172.21.15.43:0/1734814515 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178066 (secure 0 0 0) 0x7f351006c0b0 con 0x7f3510143c20 2023-08-09T13:50:19.247 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:19.246+0000 7f3514a06640 1 -- 172.21.15.43:0/1734814515 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_snap_schedules.TestSnapSchedules.test_non_existent_schedule"]} v 0) v1 -- 0x7f351015c910 con 0x7f3510143c20 2023-08-09T13:50:19.712 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:19.710+0000 7f34feffd640 1 -- 172.21.15.43:0/1734814515 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_snap_schedules.TestSnapSchedules.test_non_existent_schedule"]}]=0 v613) v1 ==== 159+0+0 (secure 0 0 0) 0x7f351015c910 con 0x7f3510143c20 2023-08-09T13:50:19.715 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:19.714+0000 7f3514a06640 1 -- 172.21.15.43:0/1734814515 >> [v2:172.21.15.203:6832/3325141227,v1:172.21.15.203:6833/3325141227] conn(0x7f34f006a170 msgr2=0x7f34f006c630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:50:19.716 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:19.714+0000 7f3514a06640 1 --2- 172.21.15.43:0/1734814515 >> [v2:172.21.15.203:6832/3325141227,v1:172.21.15.203:6833/3325141227] conn(0x7f34f006a170 0x7f34f006c630 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7f3500009a30 tx=0x7f3500008040 comp rx=0 tx=0).stop 2023-08-09T13:50:19.716 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:19.714+0000 7f3514a06640 1 -- 172.21.15.43:0/1734814515 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f3510143c20 msgr2=0x7f35100c7c60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:50:19.716 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:19.714+0000 7f3514a06640 1 --2- 172.21.15.43:0/1734814515 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f3510143c20 0x7f35100c7c60 secure :-1 s=READY pgs=965 cs=0 l=1 rev1=1 crypto rx=0x7f3508036040 tx=0x7f3508004a40 comp rx=0 tx=0).stop 2023-08-09T13:50:19.716 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:19.714+0000 7f3514a06640 1 -- 172.21.15.43:0/1734814515 shutdown_connections 2023-08-09T13:50:19.716 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:19.714+0000 7f3514a06640 1 --2- 172.21.15.43:0/1734814515 >> [v2:172.21.15.203:6832/3325141227,v1:172.21.15.203:6833/3325141227] conn(0x7f34f006a170 0x7f34f006c630 unknown :-1 s=CLOSED pgs=40 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:50:19.716 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:19.714+0000 7f3514a06640 1 --2- 172.21.15.43:0/1734814515 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f3510143c20 0x7f35100c7c60 unknown :-1 s=CLOSED pgs=965 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:50:19.716 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:19.714+0000 7f3514a06640 1 --2- 172.21.15.43:0/1734814515 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f35100c86f0 0x7f351015cbf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:50:19.717 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:19.714+0000 7f3514a06640 1 --2- 172.21.15.43:0/1734814515 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f351006cfe0 0x7f3510159c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:50:19.717 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:19.714+0000 7f3514a06640 1 -- 172.21.15.43:0/1734814515 >> 172.21.15.43:0/1734814515 conn(0x7f35100ce0a0 msgr2=0x7f35100ca9d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-09T13:50:19.717 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:19.714+0000 7f3514a06640 1 -- 172.21.15.43:0/1734814515 shutdown_connections 2023-08-09T13:50:19.717 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:19.714+0000 7f3514a06640 1 -- 172.21.15.43:0/1734814515 wait complete. 2023-08-09T13:50:19.737 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-08-09T13:50:19.968 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:19.967+0000 7f9bd64ba640 1 Processor -- start 2023-08-09T13:50:19.968 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:19.967+0000 7f9bd64ba640 1 -- start start 2023-08-09T13:50:19.970 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:19.969+0000 7f9bd64ba640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f9bd014a5c0 0x7f9bd0160f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:50:19.970 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:19.969+0000 7f9bd64ba640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f9bd00cc380 0x7f9bd006bb50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:50:19.970 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:19.969+0000 7f9bd64ba640 1 -- --> v1:172.21.15.203:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9bd00c35e0 con 0x7f9bd00c3130 2023-08-09T13:50:19.971 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:19.969+0000 7f9bd64ba640 1 -- --> v2:172.21.15.43:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9bd00c4420 con 0x7f9bd014a5c0 2023-08-09T13:50:19.971 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:19.969+0000 7f9bd64ba640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9bd00c38a0 con 0x7f9bd00cc380 2023-08-09T13:50:19.971 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:19.969+0000 7f9bcffff640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f9bd014a5c0 0x7f9bd0160f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:50:19.971 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:19.969+0000 7f9bcffff640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f9bd014a5c0 0x7f9bd0160f20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:44118/0 (socket says 172.21.15.43:44118) 2023-08-09T13:50:19.971 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:19.969+0000 7f9bcffff640 1 -- 172.21.15.43:0/2369873441 learned_addr learned my addr 172.21.15.43:0/2369873441 (peer_addr_for_me v2:172.21.15.43:0/0) 2023-08-09T13:50:19.972 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:19.969+0000 7f9bcffff640 1 -- 172.21.15.43:0/2369873441 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f9bd00cc380 msgr2=0x7f9bd006bb50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-09T13:50:19.972 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:19.969+0000 7f9bcffff640 1 --2- 172.21.15.43:0/2369873441 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f9bd00cc380 0x7f9bd006bb50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:50:19.972 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:19.969+0000 7f9bcffff640 1 -- 172.21.15.43:0/2369873441 >> v1:172.21.15.203:6789/0 conn(0x7f9bd00c3130 legacy=0x7f9bd00cbc70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-09T13:50:19.972 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:19.969+0000 7f9bcffff640 1 -- 172.21.15.43:0/2369873441 --> v2:172.21.15.43:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9bd00c3b80 con 0x7f9bd014a5c0 2023-08-09T13:50:19.972 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:19.971+0000 7f9bcffff640 1 --2- 172.21.15.43:0/2369873441 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f9bd014a5c0 0x7f9bd0160f20 secure :-1 s=READY pgs=966 cs=0 l=1 rev1=1 crypto rx=0x7f9bc00098a0 tx=0x7f9bc002b860 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6ecfbb3254d8ae02 server_cookie=0 in_seq=0 out_seq=0 2023-08-09T13:50:19.973 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:19.971+0000 7f9bcf7fe640 1 -- 172.21.15.43:0/2369873441 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9bc00317c0 con 0x7f9bd014a5c0 2023-08-09T13:50:19.973 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:19.971+0000 7f9bcf7fe640 1 -- 172.21.15.43:0/2369873441 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9bc0031920 con 0x7f9bd014a5c0 2023-08-09T13:50:19.974 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:19.971+0000 7f9bcf7fe640 1 -- 172.21.15.43:0/2369873441 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9bc0031cb0 con 0x7f9bd014a5c0 2023-08-09T13:50:19.974 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:19.972+0000 7f9bd64ba640 1 -- 172.21.15.43:0/2369873441 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f9bd014a5c0 msgr2=0x7f9bd0160f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:50:19.974 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:19.972+0000 7f9bd64ba640 1 --2- 172.21.15.43:0/2369873441 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f9bd014a5c0 0x7f9bd0160f20 secure :-1 s=READY pgs=966 cs=0 l=1 rev1=1 crypto rx=0x7f9bc00098a0 tx=0x7f9bc002b860 comp rx=0 tx=0).stop 2023-08-09T13:50:19.974 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:19.972+0000 7f9bd64ba640 1 -- 172.21.15.43:0/2369873441 shutdown_connections 2023-08-09T13:50:19.974 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:19.972+0000 7f9bd64ba640 1 --2- 172.21.15.43:0/2369873441 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f9bd00cc380 0x7f9bd006bb50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:50:19.974 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:19.972+0000 7f9bd64ba640 1 --2- 172.21.15.43:0/2369873441 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f9bd014a5c0 0x7f9bd0160f20 unknown :-1 s=CLOSED pgs=966 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:50:19.974 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:19.972+0000 7f9bd64ba640 1 -- 172.21.15.43:0/2369873441 >> 172.21.15.43:0/2369873441 conn(0x7f9bd00ce0a0 msgr2=0x7f9bd00ce480 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-09T13:50:19.975 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:19.972+0000 7f9bd64ba640 1 -- 172.21.15.43:0/2369873441 shutdown_connections 2023-08-09T13:50:19.975 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:19.972+0000 7f9bd64ba640 1 -- 172.21.15.43:0/2369873441 wait complete. 2023-08-09T13:50:19.975 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:19.974+0000 7f9bd64ba640 1 Processor -- start 2023-08-09T13:50:19.976 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:19.974+0000 7f9bd64ba640 1 -- start start 2023-08-09T13:50:19.977 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:19.976+0000 7f9bd64ba640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f9bd00c3130 0x7f9bd0155af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:50:19.978 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:19.976+0000 7f9bd64ba640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f9bd00cc380 0x7f9bd00caa80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:50:19.978 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:19.976+0000 7f9bd64ba640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f9bd014a5c0 0x7f9bd00cafc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:50:19.978 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:19.976+0000 7f9bd64ba640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9bd00c5e50 con 0x7f9bd00c3130 2023-08-09T13:50:19.978 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:19.976+0000 7f9bd64ba640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9bd0156030 con 0x7f9bd00cc380 2023-08-09T13:50:19.978 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:19.976+0000 7f9bd64ba640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9bd00c5210 con 0x7f9bd014a5c0 2023-08-09T13:50:19.978 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:19.976+0000 7f9bcffff640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f9bd00c3130 0x7f9bd0155af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:50:19.978 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:19.976+0000 7f9bcffff640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f9bd00c3130 0x7f9bd0155af0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:44120/0 (socket says 172.21.15.43:44120) 2023-08-09T13:50:19.979 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:19.976+0000 7f9bcffff640 1 -- 172.21.15.43:0/1544685170 learned_addr learned my addr 172.21.15.43:0/1544685170 (peer_addr_for_me v2:172.21.15.43:0/0) 2023-08-09T13:50:19.979 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:19.976+0000 7f9bcffff640 1 -- 172.21.15.43:0/1544685170 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f9bd014a5c0 msgr2=0x7f9bd00cafc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:50:19.980 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:19.977+0000 7f9bcffff640 1 --2- 172.21.15.43:0/1544685170 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f9bd014a5c0 0x7f9bd00cafc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:50:19.980 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:19.977+0000 7f9bcffff640 1 -- 172.21.15.43:0/1544685170 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f9bd00cc380 msgr2=0x7f9bd00caa80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:50:19.980 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:19.977+0000 7f9bc7fff640 1 --2- 172.21.15.43:0/1544685170 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f9bd00cc380 0x7f9bd00caa80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:50:19.980 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:19.977+0000 7f9bcffff640 1 --2- 172.21.15.43:0/1544685170 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f9bd00cc380 0x7f9bd00caa80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:50:19.980 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:19.977+0000 7f9bcffff640 1 -- 172.21.15.43:0/1544685170 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9bc00094d0 con 0x7f9bd00c3130 2023-08-09T13:50:19.980 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:19.977+0000 7f9bcffff640 1 --2- 172.21.15.43:0/1544685170 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f9bd00c3130 0x7f9bd0155af0 secure :-1 s=READY pgs=967 cs=0 l=1 rev1=1 crypto rx=0x7f9bc0000c00 tx=0x7f9bc00046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-09T13:50:19.980 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:19.977+0000 7f9bcd7fa640 1 -- 172.21.15.43:0/1544685170 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9bc00329a0 con 0x7f9bd00c3130 2023-08-09T13:50:19.981 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:19.977+0000 7f9bcd7fa640 1 -- 172.21.15.43:0/1544685170 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9bc0032b20 con 0x7f9bd00c3130 2023-08-09T13:50:19.981 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:19.977+0000 7f9bcd7fa640 1 -- 172.21.15.43:0/1544685170 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9bc00453f0 con 0x7f9bd00c3130 2023-08-09T13:50:19.981 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:19.977+0000 7f9bd64ba640 1 -- 172.21.15.43:0/1544685170 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9bd00c54b0 con 0x7f9bd00c3130 2023-08-09T13:50:19.981 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:19.978+0000 7f9bd64ba640 1 -- 172.21.15.43:0/1544685170 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9bd00c5a70 con 0x7f9bd00c3130 2023-08-09T13:50:19.981 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:19.979+0000 7f9bd64ba640 1 -- 172.21.15.43:0/1544685170 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9bd00739f0 con 0x7f9bd00c3130 2023-08-09T13:50:19.982 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:19.980+0000 7f9bcd7fa640 1 -- 172.21.15.43:0/1544685170 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 29) v1 ==== 88443+0+0 (secure 0 0 0) 0x7f9bc004e450 con 0x7f9bd00c3130 2023-08-09T13:50:19.982 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:19.981+0000 7f9bcd7fa640 1 --2- 172.21.15.43:0/1544685170 >> [v2:172.21.15.203:6832/3325141227,v1:172.21.15.203:6833/3325141227] conn(0x7f9bb006a170 0x7f9bb006c630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:50:19.982 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:19.981+0000 7f9bcd7fa640 1 -- 172.21.15.43:0/1544685170 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(81..81 src has 1..81) v4 ==== 6434+0+0 (secure 0 0 0) 0x7f9bc00aad00 con 0x7f9bd00c3130 2023-08-09T13:50:19.983 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:19.981+0000 7f9bc7fff640 1 --2- 172.21.15.43:0/1544685170 >> [v2:172.21.15.203:6832/3325141227,v1:172.21.15.203:6833/3325141227] conn(0x7f9bb006a170 0x7f9bb006c630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:50:19.984 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:19.982+0000 7f9bc7fff640 1 --2- 172.21.15.43:0/1544685170 >> [v2:172.21.15.203:6832/3325141227,v1:172.21.15.203:6833/3325141227] conn(0x7f9bb006a170 0x7f9bb006c630 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7f9bd01f95f0 tx=0x7f9bb8009040 comp rx=0 tx=0).ready entity=mgr.5519 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-09T13:50:19.986 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:19.985+0000 7f9bcd7fa640 1 -- 172.21.15.43:0/1544685170 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178066 (secure 0 0 0) 0x7f9bc00760c0 con 0x7f9bd00c3130 2023-08-09T13:50:20.181 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:20.180+0000 7f9bd64ba640 1 -- 172.21.15.43:0/1544685170 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f9bd00cc830 con 0x7f9bd00c3130 2023-08-09T13:50:20.183 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:20.181+0000 7f9bcd7fa640 1 -- 172.21.15.43:0/1544685170 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v81) v1 ==== 74+0+16116 (secure 0 0 0) 0x7f9bc00796e0 con 0x7f9bd00c3130 2023-08-09T13:50:20.183 INFO:teuthology.orchestra.run.smithi043.stdout: 2023-08-09T13:50:20.184 INFO:teuthology.orchestra.run.smithi043.stdout:{"epoch":81,"fsid":"daac87ee-1337-4250-a955-aea4e408a39b","created":"2023-08-09T13:33:41.701198+0000","modified":"2023-08-09T13:50:16.683675+0000","last_up_change":"2023-08-09T13:33:53.783453+0000","last_in_change":"2023-08-09T13:33:45.148904+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":9,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":"rbd","create_time":"2023-08-09T13:33:54.365137+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":".mgr","create_time":"2023-08-09T13:33:54.831803+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"cbd49ecd-e896-4fbd-864e-5c37a829b165","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":72,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6800","nonce":502667335},{"type":"v1","addr":"172.21.15.43:6801","nonce":502667335}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6802","nonce":502667335},{"type":"v1","addr":"172.21.15.43:6803","nonce":502667335}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6806","nonce":502667335},{"type":"v1","addr":"172.21.15.43:6807","nonce":502667335}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6804","nonce":502667335},{"type":"v1","addr":"172.21.15.43:6805","nonce":502667335}]},"public_addr":"172.21.15.43:6801/502667335","cluster_addr":"172.21.15.43:6803/502667335","heartbeat_back_addr":"172.21.15.43:6807/502667335","heartbeat_front_addr":"172.21.15.43:6805/502667335","state":["exists","up"]},{"osd":1,"uuid":"75c7e0fe-1ec7-4fe8-a2f2-3661a65b8500","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":72,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6808","nonce":3289134610},{"type":"v1","addr":"172.21.15.43:6809","nonce":3289134610}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6810","nonce":3289134610},{"type":"v1","addr":"172.21.15.43:6812","nonce":3289134610}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6821","nonce":3289134610},{"type":"v1","addr":"172.21.15.43:6824","nonce":3289134610}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6815","nonce":3289134610},{"type":"v1","addr":"172.21.15.43:6818","nonce":3289134610}]},"public_addr":"172.21.15.43:6809/3289134610","cluster_addr":"172.21.15.43:6812/3289134610","heartbeat_back_addr":"172.21.15.43:6824/3289134610","heartbeat_front_addr":"172.21.15.43:6818/3289134610","state":["exists","up"]},{"osd":2,"uuid":"bb5c41e9-7f5f-4125-9b01-3bb8bfc2ea83","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":72,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6811","nonce":4079709736},{"type":"v1","addr":"172.21.15.43:6814","nonce":4079709736}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6817","nonce":4079709736},{"type":"v1","addr":"172.21.15.43:6820","nonce":4079709736}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6829","nonce":4079709736},{"type":"v1","addr":"172.21.15.43:6831","nonce":4079709736}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6823","nonce":4079709736},{"type":"v1","addr":"172.21.15.43:6826","nonce":4079709736}]},"public_addr":"172.21.15.43:6814/4079709736","cluster_addr":"172.21.15.43:6820/4079709736","heartbeat_back_addr":"172.21.15.43:6831/4079709736","heartbeat_front_addr":"172.21.15.43:6826/4079709736","state":["exists","up"]},{"osd":3,"uuid":"bfe4c8b4-694a-465c-87e6-2b2aab852613","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":72,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6813","nonce":1709538759},{"type":"v1","addr":"172.21.15.43:6816","nonce":1709538759}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6819","nonce":1709538759},{"type":"v1","addr":"172.21.15.43:6822","nonce":1709538759}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6828","nonce":1709538759},{"type":"v1","addr":"172.21.15.43:6830","nonce":1709538759}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6825","nonce":1709538759},{"type":"v1","addr":"172.21.15.43:6827","nonce":1709538759}]},"public_addr":"172.21.15.43:6816/1709538759","cluster_addr":"172.21.15.43:6822/1709538759","heartbeat_back_addr":"172.21.15.43:6830/1709538759","heartbeat_front_addr":"172.21.15.43:6827/1709538759","state":["exists","up"]},{"osd":4,"uuid":"fa2e8a77-aba1-4756-853e-0d174a5bdeba","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":72,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6800","nonce":4196866481},{"type":"v1","addr":"172.21.15.203:6801","nonce":4196866481}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6802","nonce":4196866481},{"type":"v1","addr":"172.21.15.203:6803","nonce":4196866481}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6806","nonce":4196866481},{"type":"v1","addr":"172.21.15.203:6807","nonce":4196866481}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6804","nonce":4196866481},{"type":"v1","addr":"172.21.15.203:6805","nonce":4196866481}]},"public_addr":"172.21.15.203:6801/4196866481","cluster_addr":"172.21.15.203:6803/4196866481","heartbeat_back_addr":"172.21.15.203:6807/4196866481","heartbeat_front_addr":"172.21.15.203:6805/4196866481","state":["exists","up"]},{"osd":5,"uuid":"81811e4b-4d24-4f51-a82c-f1d700a2a00d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":72,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6816","nonce":3059290081},{"type":"v1","addr":"172.21.15.203:6817","nonce":3059290081}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6818","nonce":3059290081},{"type":"v1","addr":"172.21.15.203:6819","nonce":3059290081}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6822","nonce":3059290081},{"type":"v1","addr":"172.21.15.203:6823","nonce":3059290081}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6820","nonce":3059290081},{"type":"v1","addr":"172.21.15.203:6821","nonce":3059290081}]},"public_addr":"172.21.15.203:6817/3059290081","cluster_addr":"172.21.15.203:6819/3059290081","heartbeat_back_addr":"172.21.15.203:6823/3059290081","heartbeat_front_addr":"172.21.15.203:6821/3059290081","state":["exists","up"]},{"osd":6,"uuid":"307bac4d-6a5b-443d-af75-56d695d7674f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":72,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6824","nonce":512155569},{"type":"v1","addr":"172.21.15.203:6825","nonce":512155569}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6826","nonce":512155569},{"type":"v1","addr":"172.21.15.203:6827","nonce":512155569}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6830","nonce":512155569},{"type":"v1","addr":"172.21.15.203:6831","nonce":512155569}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6828","nonce":512155569},{"type":"v1","addr":"172.21.15.203:6829","nonce":512155569}]},"public_addr":"172.21.15.203:6825/512155569","cluster_addr":"172.21.15.203:6827/512155569","heartbeat_back_addr":"172.21.15.203:6831/512155569","heartbeat_front_addr":"172.21.15.203:6829/512155569","state":["exists","up"]},{"osd":7,"uuid":"a95dc7ca-9f1b-4295-af70-68a22a2d7d5e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":72,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6808","nonce":3331450438},{"type":"v1","addr":"172.21.15.203:6809","nonce":3331450438}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6810","nonce":3331450438},{"type":"v1","addr":"172.21.15.203:6811","nonce":3331450438}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6814","nonce":3331450438},{"type":"v1","addr":"172.21.15.203:6815","nonce":3331450438}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6812","nonce":3331450438},{"type":"v1","addr":"172.21.15.203:6813","nonce":3331450438}]},"public_addr":"172.21.15.203:6809/3331450438","cluster_addr":"172.21.15.203:6811/3331450438","heartbeat_back_addr":"172.21.15.203:6815/3331450438","heartbeat_front_addr":"172.21.15.203:6813/3331450438","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-09T13:33:50.547652+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-09T13:33:50.589469+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-09T13:33:50.561724+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-09T13:33:50.599650+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-09T13:33:50.411855+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-09T13:33:50.219681+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-09T13:33:50.504664+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-09T13:33:50.441248+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.203:6834/3324252767":"2023-08-10T13:50:13.672121+0000","172.21.15.203:0/2251174608":"2023-08-10T13:50:08.172778+0000","172.21.15.203:6835/3324252767":"2023-08-10T13:50:13.672121+0000","172.21.15.203:0/4008902326":"2023-08-10T13:50:08.172778+0000","172.21.15.203:0/1825829254":"2023-08-10T13:50:08.172778+0000","172.21.15.203:0/4097180925":"2023-08-10T13:50:08.172778+0000","172.21.15.203:6832/2515481191":"2023-08-10T13:50:08.172778+0000","172.21.15.203:0/4038087778":"2023-08-10T13:50:08.172778+0000","172.21.15.203:0/826156596":"2023-08-10T13:50:08.172778+0000","172.21.15.203:0/1084459051":"2023-08-10T13:50:08.172778+0000","172.21.15.203:6833/272917756":"2023-08-10T13:48:34.065493+0000","172.21.15.203:6832/272917756":"2023-08-10T13:48:34.065493+0000","172.21.15.203:0/665971116":"2023-08-10T13:48:34.065493+0000","172.21.15.203:0/2868823959":"2023-08-10T13:48:34.065493+0000","172.21.15.203:6833/2515481191":"2023-08-10T13:50:08.172778+0000","172.21.15.203:0/3168500363":"2023-08-10T13:48:34.065493+0000","172.21.15.203:0/752327795":"2023-08-10T13:48:34.065493+0000","172.21.15.203:0/1970263661":"2023-08-10T13:48:34.065493+0000","172.21.15.203:0/2726495528":"2023-08-10T13:50:08.172778+0000","172.21.15.203:0/1407921417":"2023-08-10T13:48:34.065493+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":7,"snaps":[{"begin":104,"length":1},{"begin":221,"length":1}]}],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-08-09T13:50:20.186 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:20.185+0000 7f9bd64ba640 1 -- 172.21.15.43:0/1544685170 >> [v2:172.21.15.203:6832/3325141227,v1:172.21.15.203:6833/3325141227] conn(0x7f9bb006a170 msgr2=0x7f9bb006c630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:50:20.187 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:20.185+0000 7f9bd64ba640 1 --2- 172.21.15.43:0/1544685170 >> [v2:172.21.15.203:6832/3325141227,v1:172.21.15.203:6833/3325141227] conn(0x7f9bb006a170 0x7f9bb006c630 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7f9bd01f95f0 tx=0x7f9bb8009040 comp rx=0 tx=0).stop 2023-08-09T13:50:20.187 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:20.185+0000 7f9bd64ba640 1 -- 172.21.15.43:0/1544685170 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f9bd00c3130 msgr2=0x7f9bd0155af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:50:20.187 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:20.185+0000 7f9bd64ba640 1 --2- 172.21.15.43:0/1544685170 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f9bd00c3130 0x7f9bd0155af0 secure :-1 s=READY pgs=967 cs=0 l=1 rev1=1 crypto rx=0x7f9bc0000c00 tx=0x7f9bc00046c0 comp rx=0 tx=0).stop 2023-08-09T13:50:20.187 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:20.185+0000 7f9bd64ba640 1 -- 172.21.15.43:0/1544685170 shutdown_connections 2023-08-09T13:50:20.187 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:20.185+0000 7f9bd64ba640 1 --2- 172.21.15.43:0/1544685170 >> [v2:172.21.15.203:6832/3325141227,v1:172.21.15.203:6833/3325141227] conn(0x7f9bb006a170 0x7f9bb006c630 unknown :-1 s=CLOSED pgs=41 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:50:20.187 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:20.185+0000 7f9bd64ba640 1 --2- 172.21.15.43:0/1544685170 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f9bd014a5c0 0x7f9bd00cafc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:50:20.187 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:20.185+0000 7f9bd64ba640 1 --2- 172.21.15.43:0/1544685170 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f9bd00cc380 0x7f9bd00caa80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:50:20.188 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:20.185+0000 7f9bd64ba640 1 --2- 172.21.15.43:0/1544685170 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f9bd00c3130 0x7f9bd0155af0 unknown :-1 s=CLOSED pgs=967 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:50:20.188 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:20.185+0000 7f9bd64ba640 1 -- 172.21.15.43:0/1544685170 >> 172.21.15.43:0/1544685170 conn(0x7f9bd00ce0a0 msgr2=0x7f9bd00710c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-09T13:50:20.188 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:20.185+0000 7f9bd64ba640 1 -- 172.21.15.43:0/1544685170 shutdown_connections 2023-08-09T13:50:20.188 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:20.185+0000 7f9bd64ba640 1 -- 172.21.15.43:0/1544685170 wait complete. 2023-08-09T13:50:20.207 INFO:tasks.cephfs.cephfs_test_case.ceph_manager:[{'pool': 1, 'pool_name': 'rbd', 'create_time': '2023-08-09T13:33:54.365137+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}, 'read_balance': {'score_acting': 2, 'score_stable': 2, 'optimal_score': 1, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': '.mgr', 'create_time': '2023-08-09T13:33:54.831803+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-08-09T13:50:20.208 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get rbd pg_num 2023-08-09T13:50:20.442 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:20.441+0000 7f75ea4b8640 1 Processor -- start 2023-08-09T13:50:20.443 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:20.441+0000 7f75ea4b8640 1 -- start start 2023-08-09T13:50:20.444 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:20.443+0000 7f75ea4b8640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f75e40cb2d0 0x7f75e415b380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:50:20.444 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:20.443+0000 7f75ea4b8640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f75e415df60 0x7f75e40ce1c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:50:20.445 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:20.443+0000 7f75ea4b8640 1 -- --> v1:172.21.15.203:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f75e415aef0 con 0x7f75e415e3c0 2023-08-09T13:50:20.445 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:20.443+0000 7f75ea4b8640 1 -- --> v2:172.21.15.43:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f75e415a970 con 0x7f75e40cb2d0 2023-08-09T13:50:20.445 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:20.443+0000 7f75ea4b8640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f75e415b8c0 con 0x7f75e415df60 2023-08-09T13:50:20.445 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:20.443+0000 7f75e3fff640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f75e40cb2d0 0x7f75e415b380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:50:20.446 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:20.443+0000 7f75e3fff640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f75e40cb2d0 0x7f75e415b380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:44132/0 (socket says 172.21.15.43:44132) 2023-08-09T13:50:20.446 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:20.443+0000 7f75e3fff640 1 -- 172.21.15.43:0/418767758 learned_addr learned my addr 172.21.15.43:0/418767758 (peer_addr_for_me v2:172.21.15.43:0/0) 2023-08-09T13:50:20.446 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:20.444+0000 7f75e37fe640 1 --2- 172.21.15.43:0/418767758 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f75e415df60 0x7f75e40ce1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:50:20.446 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:20.444+0000 7f75e3fff640 1 -- 172.21.15.43:0/418767758 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f75e415df60 msgr2=0x7f75e40ce1c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-09T13:50:20.446 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:20.444+0000 7f75e3fff640 1 --2- 172.21.15.43:0/418767758 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f75e415df60 0x7f75e40ce1c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:50:20.446 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:20.444+0000 7f75e3fff640 1 -- 172.21.15.43:0/418767758 >> v1:172.21.15.203:6789/0 conn(0x7f75e415e3c0 legacy=0x7f75e40c9f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-09T13:50:20.447 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:20.444+0000 7f75e3fff640 1 -- 172.21.15.43:0/418767758 --> v2:172.21.15.43:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f75e415e940 con 0x7f75e40cb2d0 2023-08-09T13:50:20.447 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:20.445+0000 7f75e3fff640 1 --2- 172.21.15.43:0/418767758 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f75e40cb2d0 0x7f75e415b380 secure :-1 s=READY pgs=968 cs=0 l=1 rev1=1 crypto rx=0x7f75d4009870 tx=0x7f75d402b8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c650ecd4bc97c9f server_cookie=0 in_seq=0 out_seq=0 2023-08-09T13:50:20.447 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:20.445+0000 7f75e2ffd640 1 -- 172.21.15.43:0/418767758 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f75d4031640 con 0x7f75e40cb2d0 2023-08-09T13:50:20.447 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:20.445+0000 7f75e2ffd640 1 -- 172.21.15.43:0/418767758 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f75d40317a0 con 0x7f75e40cb2d0 2023-08-09T13:50:20.448 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:20.445+0000 7f75e2ffd640 1 -- 172.21.15.43:0/418767758 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f75d4031b30 con 0x7f75e40cb2d0 2023-08-09T13:50:20.448 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:20.446+0000 7f75ea4b8640 1 -- 172.21.15.43:0/418767758 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f75e40cb2d0 msgr2=0x7f75e415b380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:50:20.448 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:20.446+0000 7f75ea4b8640 1 --2- 172.21.15.43:0/418767758 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f75e40cb2d0 0x7f75e415b380 secure :-1 s=READY pgs=968 cs=0 l=1 rev1=1 crypto rx=0x7f75d4009870 tx=0x7f75d402b8f0 comp rx=0 tx=0).stop 2023-08-09T13:50:20.448 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:20.446+0000 7f75ea4b8640 1 -- 172.21.15.43:0/418767758 shutdown_connections 2023-08-09T13:50:20.449 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:20.446+0000 7f75ea4b8640 1 --2- 172.21.15.43:0/418767758 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f75e415df60 0x7f75e40ce1c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:50:20.449 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:20.446+0000 7f75ea4b8640 1 --2- 172.21.15.43:0/418767758 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f75e40cb2d0 0x7f75e415b380 unknown :-1 s=CLOSED pgs=968 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:50:20.449 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:20.446+0000 7f75ea4b8640 1 -- 172.21.15.43:0/418767758 >> 172.21.15.43:0/418767758 conn(0x7f75e40aff50 msgr2=0x7f75e40b0360 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-09T13:50:20.449 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:20.446+0000 7f75ea4b8640 1 -- 172.21.15.43:0/418767758 shutdown_connections 2023-08-09T13:50:20.449 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:20.446+0000 7f75ea4b8640 1 -- 172.21.15.43:0/418767758 wait complete. 2023-08-09T13:50:20.450 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:20.448+0000 7f75ea4b8640 1 Processor -- start 2023-08-09T13:50:20.450 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:20.448+0000 7f75ea4b8640 1 -- start start 2023-08-09T13:50:20.451 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:20.449+0000 7f75ea4b8640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f75e40cb2d0 0x7f75e40c5080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:50:20.451 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:20.449+0000 7f75ea4b8640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f75e415df60 0x7f75e415d430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:50:20.451 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:20.449+0000 7f75ea4b8640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f75e415e3c0 0x7f75e40c94e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:50:20.451 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:20.449+0000 7f75ea4b8640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f75e40c5660 con 0x7f75e40cb2d0 2023-08-09T13:50:20.451 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:20.449+0000 7f75ea4b8640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f75e40c5940 con 0x7f75e415df60 2023-08-09T13:50:20.452 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:20.449+0000 7f75ea4b8640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f75e40c5c20 con 0x7f75e415e3c0 2023-08-09T13:50:20.452 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:20.450+0000 7f75e3fff640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f75e40cb2d0 0x7f75e40c5080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:50:20.452 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:20.450+0000 7f75e3fff640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f75e40cb2d0 0x7f75e40c5080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:44140/0 (socket says 172.21.15.43:44140) 2023-08-09T13:50:20.452 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:20.450+0000 7f75e3fff640 1 -- 172.21.15.43:0/327337685 learned_addr learned my addr 172.21.15.43:0/327337685 (peer_addr_for_me v2:172.21.15.43:0/0) 2023-08-09T13:50:20.453 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:20.450+0000 7f75e37fe640 1 --2- 172.21.15.43:0/327337685 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f75e415df60 0x7f75e415d430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:50:20.453 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:20.450+0000 7f75e8a55640 1 --2- 172.21.15.43:0/327337685 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f75e415e3c0 0x7f75e40c94e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:50:20.453 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:20.450+0000 7f75e3fff640 1 -- 172.21.15.43:0/327337685 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f75e415e3c0 msgr2=0x7f75e40c94e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:50:20.453 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:20.450+0000 7f75e3fff640 1 --2- 172.21.15.43:0/327337685 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f75e415e3c0 0x7f75e40c94e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:50:20.454 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:20.450+0000 7f75e3fff640 1 -- 172.21.15.43:0/327337685 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f75e415df60 msgr2=0x7f75e415d430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:50:20.454 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:20.450+0000 7f75e3fff640 1 --2- 172.21.15.43:0/327337685 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f75e415df60 0x7f75e415d430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:50:20.454 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:20.450+0000 7f75e3fff640 1 -- 172.21.15.43:0/327337685 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f75d40094d0 con 0x7f75e40cb2d0 2023-08-09T13:50:20.454 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:20.450+0000 7f75e37fe640 1 --2- 172.21.15.43:0/327337685 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f75e415df60 0x7f75e415d430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-08-09T13:50:20.454 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:20.451+0000 7f75e3fff640 1 --2- 172.21.15.43:0/327337685 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f75e40cb2d0 0x7f75e40c5080 secure :-1 s=READY pgs=969 cs=0 l=1 rev1=1 crypto rx=0x7f75d4009870 tx=0x7f75d40046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-09T13:50:20.454 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:20.451+0000 7f75e0ff9640 1 -- 172.21.15.43:0/327337685 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f75d403ac60 con 0x7f75e40cb2d0 2023-08-09T13:50:20.454 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:20.451+0000 7f75e0ff9640 1 -- 172.21.15.43:0/327337685 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f75d403ade0 con 0x7f75e40cb2d0 2023-08-09T13:50:20.455 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:20.451+0000 7f75e0ff9640 1 -- 172.21.15.43:0/327337685 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f75d4044630 con 0x7f75e40cb2d0 2023-08-09T13:50:20.455 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:20.451+0000 7f75ea4b8640 1 -- 172.21.15.43:0/327337685 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f75e40c5f00 con 0x7f75e40cb2d0 2023-08-09T13:50:20.455 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:20.451+0000 7f75ea4b8640 1 -- 172.21.15.43:0/327337685 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f75e415d970 con 0x7f75e40cb2d0 2023-08-09T13:50:20.455 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:20.452+0000 7f75ea4b8640 1 -- 172.21.15.43:0/327337685 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f75e40739f0 con 0x7f75e40cb2d0 2023-08-09T13:50:20.455 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:20.453+0000 7f75e0ff9640 1 -- 172.21.15.43:0/327337685 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 29) v1 ==== 88443+0+0 (secure 0 0 0) 0x7f75d402f050 con 0x7f75e40cb2d0 2023-08-09T13:50:20.456 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:20.454+0000 7f75e0ff9640 1 --2- 172.21.15.43:0/327337685 >> [v2:172.21.15.203:6832/3325141227,v1:172.21.15.203:6833/3325141227] conn(0x7f75d006a1a0 0x7f75d006c660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:50:20.456 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:20.454+0000 7f75e0ff9640 1 -- 172.21.15.43:0/327337685 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(81..81 src has 1..81) v4 ==== 6434+0+0 (secure 0 0 0) 0x7f75d40aaaa0 con 0x7f75e40cb2d0 2023-08-09T13:50:20.457 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:20.455+0000 7f75e37fe640 1 --2- 172.21.15.43:0/327337685 >> [v2:172.21.15.203:6832/3325141227,v1:172.21.15.203:6833/3325141227] conn(0x7f75d006a1a0 0x7f75d006c660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:50:20.457 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:20.456+0000 7f75e37fe640 1 --2- 172.21.15.43:0/327337685 >> [v2:172.21.15.203:6832/3325141227,v1:172.21.15.203:6833/3325141227] conn(0x7f75d006a1a0 0x7f75d006c660 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7f75e40afb00 tx=0x7f75d80074a0 comp rx=0 tx=0).ready entity=mgr.5519 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-09T13:50:20.460 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:20.459+0000 7f75e0ff9640 1 -- 172.21.15.43:0/327337685 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178066 (secure 0 0 0) 0x7f75d4075e60 con 0x7f75e40cb2d0 2023-08-09T13:50:20.656 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:20.654+0000 7f75ea4b8640 1 -- 172.21.15.43:0/327337685 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"} v 0) v1 -- 0x7f75e4070690 con 0x7f75e40cb2d0 2023-08-09T13:50:20.657 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:20.655+0000 7f75e0ff9640 1 -- 172.21.15.43:0/327337685 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"}]=0 v81) v1 ==== 92+0+10 (secure 0 0 0) 0x7f75d4079480 con 0x7f75e40cb2d0 2023-08-09T13:50:20.657 INFO:teuthology.orchestra.run.smithi043.stdout:pg_num: 8 2023-08-09T13:50:20.660 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:20.658+0000 7f75ea4b8640 1 -- 172.21.15.43:0/327337685 >> [v2:172.21.15.203:6832/3325141227,v1:172.21.15.203:6833/3325141227] conn(0x7f75d006a1a0 msgr2=0x7f75d006c660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:50:20.660 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:20.658+0000 7f75ea4b8640 1 --2- 172.21.15.43:0/327337685 >> [v2:172.21.15.203:6832/3325141227,v1:172.21.15.203:6833/3325141227] conn(0x7f75d006a1a0 0x7f75d006c660 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7f75e40afb00 tx=0x7f75d80074a0 comp rx=0 tx=0).stop 2023-08-09T13:50:20.660 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:20.658+0000 7f75ea4b8640 1 -- 172.21.15.43:0/327337685 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f75e40cb2d0 msgr2=0x7f75e40c5080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:50:20.660 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:20.658+0000 7f75ea4b8640 1 --2- 172.21.15.43:0/327337685 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f75e40cb2d0 0x7f75e40c5080 secure :-1 s=READY pgs=969 cs=0 l=1 rev1=1 crypto rx=0x7f75d4009870 tx=0x7f75d40046c0 comp rx=0 tx=0).stop 2023-08-09T13:50:20.660 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:20.658+0000 7f75ea4b8640 1 -- 172.21.15.43:0/327337685 shutdown_connections 2023-08-09T13:50:20.661 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:20.658+0000 7f75ea4b8640 1 --2- 172.21.15.43:0/327337685 >> [v2:172.21.15.203:6832/3325141227,v1:172.21.15.203:6833/3325141227] conn(0x7f75d006a1a0 0x7f75d006c660 unknown :-1 s=CLOSED pgs=42 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:50:20.661 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:20.658+0000 7f75ea4b8640 1 --2- 172.21.15.43:0/327337685 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f75e415e3c0 0x7f75e40c94e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:50:20.661 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:20.658+0000 7f75ea4b8640 1 --2- 172.21.15.43:0/327337685 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f75e415df60 0x7f75e415d430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:50:20.661 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:20.658+0000 7f75ea4b8640 1 --2- 172.21.15.43:0/327337685 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f75e40cb2d0 0x7f75e40c5080 unknown :-1 s=CLOSED pgs=969 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:50:20.661 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:20.658+0000 7f75ea4b8640 1 -- 172.21.15.43:0/327337685 >> 172.21.15.43:0/327337685 conn(0x7f75e40aff50 msgr2=0x7f75e40ce190 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-09T13:50:20.661 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:20.658+0000 7f75ea4b8640 1 -- 172.21.15.43:0/327337685 shutdown_connections 2023-08-09T13:50:20.662 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:20.658+0000 7f75ea4b8640 1 -- 172.21.15.43:0/327337685 wait complete. 2023-08-09T13:50:20.678 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-08-09T13:50:20.902 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:20.901+0000 7f56ed555640 1 Processor -- start 2023-08-09T13:50:20.902 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:20.901+0000 7f56ed555640 1 -- start start 2023-08-09T13:50:20.904 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:20.903+0000 7f56ed555640 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7f56e80b45f0 0x7f56e80cd6d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:50:20.904 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:20.903+0000 7f56ed555640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f56e80d04b0 0x7f56e80b59d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:50:20.905 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:20.903+0000 7f56ed555640 1 -- --> v1:172.21.15.203:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f56e80b7bd0 con 0x7f56e815c550 2023-08-09T13:50:20.905 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:20.903+0000 7f56ed555640 1 -- --> v2:172.21.15.43:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f56e80d1c00 con 0x7f56e80d04b0 2023-08-09T13:50:20.905 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:20.903+0000 7f56ed555640 1 -- --> v2:172.21.15.203:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f56e815c930 con 0x7f56e80b45f0 2023-08-09T13:50:20.905 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:20.903+0000 7f56e77fe640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f56e80d04b0 0x7f56e80b59d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:50:20.905 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:20.903+0000 7f56e77fe640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f56e80d04b0 0x7f56e80b59d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:44148/0 (socket says 172.21.15.43:44148) 2023-08-09T13:50:20.905 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:20.903+0000 7f56e77fe640 1 -- 172.21.15.43:0/3885468953 learned_addr learned my addr 172.21.15.43:0/3885468953 (peer_addr_for_me v2:172.21.15.43:0/0) 2023-08-09T13:50:20.906 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:20.903+0000 7f56e67fc640 1 --2- 172.21.15.43:0/3885468953 >> v2:172.21.15.203:3300/0 conn(0x7f56e80b45f0 0x7f56e80cd6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:50:20.906 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:20.904+0000 7f56e77fe640 1 -- 172.21.15.43:0/3885468953 >> v2:172.21.15.203:3300/0 conn(0x7f56e80b45f0 msgr2=0x7f56e80cd6d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-09T13:50:20.906 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:20.904+0000 7f56e77fe640 1 --2- 172.21.15.43:0/3885468953 >> v2:172.21.15.203:3300/0 conn(0x7f56e80b45f0 0x7f56e80cd6d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:50:20.907 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:20.904+0000 7f56e77fe640 1 -- 172.21.15.43:0/3885468953 >> v1:172.21.15.203:6789/0 conn(0x7f56e815c550 legacy=0x7f56e80b3ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-09T13:50:20.907 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:20.904+0000 7f56e77fe640 1 -- 172.21.15.43:0/3885468953 --> v2:172.21.15.43:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f56e80b54f0 con 0x7f56e80d04b0 2023-08-09T13:50:20.908 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:20.905+0000 7f56e77fe640 1 --2- 172.21.15.43:0/3885468953 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f56e80d04b0 0x7f56e80b59d0 secure :-1 s=READY pgs=970 cs=0 l=1 rev1=1 crypto rx=0x7f56e80b5050 tx=0x7f56e002b910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f59f75b519577367 server_cookie=0 in_seq=0 out_seq=0 2023-08-09T13:50:20.909 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:20.905+0000 7f56e5ffb640 1 -- 172.21.15.43:0/3885468953 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f56e0031870 con 0x7f56e80d04b0 2023-08-09T13:50:20.909 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:20.905+0000 7f56e5ffb640 1 -- 172.21.15.43:0/3885468953 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f56e00319d0 con 0x7f56e80d04b0 2023-08-09T13:50:20.909 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:20.905+0000 7f56e5ffb640 1 -- 172.21.15.43:0/3885468953 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f56e0031d60 con 0x7f56e80d04b0 2023-08-09T13:50:20.909 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:20.906+0000 7f56ed555640 1 -- 172.21.15.43:0/3885468953 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f56e80d04b0 msgr2=0x7f56e80b59d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:50:20.909 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:20.906+0000 7f56ed555640 1 --2- 172.21.15.43:0/3885468953 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f56e80d04b0 0x7f56e80b59d0 secure :-1 s=READY pgs=970 cs=0 l=1 rev1=1 crypto rx=0x7f56e80b5050 tx=0x7f56e002b910 comp rx=0 tx=0).stop 2023-08-09T13:50:20.910 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:20.906+0000 7f56ed555640 1 -- 172.21.15.43:0/3885468953 shutdown_connections 2023-08-09T13:50:20.910 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:20.906+0000 7f56ed555640 1 --2- 172.21.15.43:0/3885468953 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f56e80d04b0 0x7f56e80b59d0 unknown :-1 s=CLOSED pgs=970 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:50:20.910 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:20.906+0000 7f56ed555640 1 --2- 172.21.15.43:0/3885468953 >> v2:172.21.15.203:3300/0 conn(0x7f56e80b45f0 0x7f56e80cd6d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:50:20.910 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:20.906+0000 7f56ed555640 1 -- 172.21.15.43:0/3885468953 >> 172.21.15.43:0/3885468953 conn(0x7f56e8071520 msgr2=0x7f56e80ceee0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-09T13:50:20.910 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:20.906+0000 7f56ed555640 1 -- 172.21.15.43:0/3885468953 shutdown_connections 2023-08-09T13:50:20.910 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:20.906+0000 7f56ed555640 1 -- 172.21.15.43:0/3885468953 wait complete. 2023-08-09T13:50:20.911 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:20.908+0000 7f56ed555640 1 Processor -- start 2023-08-09T13:50:20.912 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:20.908+0000 7f56ed555640 1 -- start start 2023-08-09T13:50:20.912 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:20.909+0000 7f56ed555640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f56e80b45f0 0x7f56e815ac10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:50:20.912 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:20.909+0000 7f56ed555640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f56e80d04b0 0x7f56e80aaac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:50:20.912 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:20.909+0000 7f56ed555640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f56e815c550 0x7f56e80cb540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:50:20.912 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:20.909+0000 7f56ed555640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f56e815b150 con 0x7f56e80b45f0 2023-08-09T13:50:20.912 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:20.909+0000 7f56ed555640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f56e81e6be0 con 0x7f56e815c550 2023-08-09T13:50:20.912 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:20.909+0000 7f56ed555640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f56e80a9d00 con 0x7f56e80d04b0 2023-08-09T13:50:20.913 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:20.910+0000 7f56e6ffd640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f56e80b45f0 0x7f56e815ac10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:50:20.913 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:20.910+0000 7f56e6ffd640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f56e80b45f0 0x7f56e815ac10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:44164/0 (socket says 172.21.15.43:44164) 2023-08-09T13:50:20.913 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:20.910+0000 7f56e6ffd640 1 -- 172.21.15.43:0/2336337933 learned_addr learned my addr 172.21.15.43:0/2336337933 (peer_addr_for_me v2:172.21.15.43:0/0) 2023-08-09T13:50:20.913 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:20.910+0000 7f56e67fc640 1 --2- 172.21.15.43:0/2336337933 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f56e80d04b0 0x7f56e80aaac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:50:20.913 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:20.910+0000 7f56e77fe640 1 --2- 172.21.15.43:0/2336337933 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f56e815c550 0x7f56e80cb540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:50:20.913 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:20.910+0000 7f56e6ffd640 1 -- 172.21.15.43:0/2336337933 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f56e80d04b0 msgr2=0x7f56e80aaac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:50:20.914 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:20.910+0000 7f56e6ffd640 1 --2- 172.21.15.43:0/2336337933 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f56e80d04b0 0x7f56e80aaac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:50:20.914 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:20.910+0000 7f56e6ffd640 1 -- 172.21.15.43:0/2336337933 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f56e815c550 msgr2=0x7f56e80cb540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:50:20.914 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:20.910+0000 7f56e6ffd640 1 --2- 172.21.15.43:0/2336337933 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f56e815c550 0x7f56e80cb540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:50:20.914 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:20.910+0000 7f56e6ffd640 1 -- 172.21.15.43:0/2336337933 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f56e00094d0 con 0x7f56e80b45f0 2023-08-09T13:50:20.915 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:20.911+0000 7f56e6ffd640 1 --2- 172.21.15.43:0/2336337933 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f56e80b45f0 0x7f56e815ac10 secure :-1 s=READY pgs=971 cs=0 l=1 rev1=1 crypto rx=0x7f56d800ba80 tx=0x7f56d800bf40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-09T13:50:20.915 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:20.911+0000 7f56df7fe640 1 -- 172.21.15.43:0/2336337933 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f56d800cc00 con 0x7f56e80b45f0 2023-08-09T13:50:20.915 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:20.911+0000 7f56df7fe640 1 -- 172.21.15.43:0/2336337933 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f56d800cd60 con 0x7f56e80b45f0 2023-08-09T13:50:20.915 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:20.911+0000 7f56ed555640 1 -- 172.21.15.43:0/2336337933 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f56e80a9fe0 con 0x7f56e80b45f0 2023-08-09T13:50:20.915 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:20.911+0000 7f56df7fe640 1 -- 172.21.15.43:0/2336337933 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f56d8003bb0 con 0x7f56e80b45f0 2023-08-09T13:50:20.915 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:20.911+0000 7f56ed555640 1 -- 172.21.15.43:0/2336337933 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f56e80aa2c0 con 0x7f56e80b45f0 2023-08-09T13:50:20.916 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:20.912+0000 7f56ed555640 1 -- 172.21.15.43:0/2336337933 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f56e80739f0 con 0x7f56e80b45f0 2023-08-09T13:50:20.916 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:20.913+0000 7f56df7fe640 1 -- 172.21.15.43:0/2336337933 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 29) v1 ==== 88443+0+0 (secure 0 0 0) 0x7f56d8014440 con 0x7f56e80b45f0 2023-08-09T13:50:20.916 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:20.914+0000 7f56df7fe640 1 --2- 172.21.15.43:0/2336337933 >> [v2:172.21.15.203:6832/3325141227,v1:172.21.15.203:6833/3325141227] conn(0x7f56c806a170 0x7f56c806c630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:50:20.916 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:20.914+0000 7f56df7fe640 1 -- 172.21.15.43:0/2336337933 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(81..81 src has 1..81) v4 ==== 6434+0+0 (secure 0 0 0) 0x7f56d8088750 con 0x7f56e80b45f0 2023-08-09T13:50:20.917 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:20.915+0000 7f56e67fc640 1 --2- 172.21.15.43:0/2336337933 >> [v2:172.21.15.203:6832/3325141227,v1:172.21.15.203:6833/3325141227] conn(0x7f56c806a170 0x7f56c806c630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:50:20.917 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:20.916+0000 7f56e67fc640 1 --2- 172.21.15.43:0/2336337933 >> [v2:172.21.15.203:6832/3325141227,v1:172.21.15.203:6833/3325141227] conn(0x7f56c806a170 0x7f56c806c630 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7f56e80cc560 tx=0x7f56d0008040 comp rx=0 tx=0).ready entity=mgr.5519 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-09T13:50:20.920 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:20.919+0000 7f56df7fe640 1 -- 172.21.15.43:0/2336337933 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178066 (secure 0 0 0) 0x7f56d8053b10 con 0x7f56e80b45f0 2023-08-09T13:50:21.117 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:21.115+0000 7f56ed555640 1 -- 172.21.15.43:0/2336337933 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f56e80ccff0 con 0x7f56e80b45f0 2023-08-09T13:50:21.118 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:21.117+0000 7f56df7fe640 1 -- 172.21.15.43:0/2336337933 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v81) v1 ==== 93+0+10 (secure 0 0 0) 0x7f56d8057130 con 0x7f56e80b45f0 2023-08-09T13:50:21.118 INFO:teuthology.orchestra.run.smithi043.stdout:pg_num: 1 2023-08-09T13:50:21.121 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:21.119+0000 7f56ed555640 1 -- 172.21.15.43:0/2336337933 >> [v2:172.21.15.203:6832/3325141227,v1:172.21.15.203:6833/3325141227] conn(0x7f56c806a170 msgr2=0x7f56c806c630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:50:21.121 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:21.119+0000 7f56ed555640 1 --2- 172.21.15.43:0/2336337933 >> [v2:172.21.15.203:6832/3325141227,v1:172.21.15.203:6833/3325141227] conn(0x7f56c806a170 0x7f56c806c630 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7f56e80cc560 tx=0x7f56d0008040 comp rx=0 tx=0).stop 2023-08-09T13:50:21.121 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:21.119+0000 7f56ed555640 1 -- 172.21.15.43:0/2336337933 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f56e80b45f0 msgr2=0x7f56e815ac10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:50:21.121 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:21.119+0000 7f56ed555640 1 --2- 172.21.15.43:0/2336337933 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f56e80b45f0 0x7f56e815ac10 secure :-1 s=READY pgs=971 cs=0 l=1 rev1=1 crypto rx=0x7f56d800ba80 tx=0x7f56d800bf40 comp rx=0 tx=0).stop 2023-08-09T13:50:21.121 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:21.119+0000 7f56ed555640 1 -- 172.21.15.43:0/2336337933 shutdown_connections 2023-08-09T13:50:21.122 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:21.119+0000 7f56ed555640 1 --2- 172.21.15.43:0/2336337933 >> [v2:172.21.15.203:6832/3325141227,v1:172.21.15.203:6833/3325141227] conn(0x7f56c806a170 0x7f56c806c630 unknown :-1 s=CLOSED pgs=43 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:50:21.122 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:21.119+0000 7f56ed555640 1 --2- 172.21.15.43:0/2336337933 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f56e815c550 0x7f56e80cb540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:50:21.122 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:21.119+0000 7f56ed555640 1 --2- 172.21.15.43:0/2336337933 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f56e80d04b0 0x7f56e80aaac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:50:21.122 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:21.119+0000 7f56ed555640 1 --2- 172.21.15.43:0/2336337933 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f56e80b45f0 0x7f56e815ac10 unknown :-1 s=CLOSED pgs=971 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:50:21.122 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:21.119+0000 7f56ed555640 1 -- 172.21.15.43:0/2336337933 >> 172.21.15.43:0/2336337933 conn(0x7f56e8071520 msgr2=0x7f56e815f270 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-09T13:50:21.122 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:21.119+0000 7f56ed555640 1 -- 172.21.15.43:0/2336337933 shutdown_connections 2023-08-09T13:50:21.123 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:21.119+0000 7f56ed555640 1 -- 172.21.15.43:0/2336337933 wait complete. 2023-08-09T13:50:21.140 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mon mon_allow_pool_delete True 2023-08-09T13:50:21.364 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:21.363+0000 7f33dfdcb640 1 Processor -- start 2023-08-09T13:50:21.364 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:21.363+0000 7f33dfdcb640 1 -- start start 2023-08-09T13:50:21.366 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:21.365+0000 7f33dfdcb640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f33d815c630 0x7f33d80abef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:50:21.366 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:21.365+0000 7f33dfdcb640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f33d80ccf20 0x7f33d80cd550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:50:21.367 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:21.365+0000 7f33dfdcb640 1 -- --> v1:172.21.15.203:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f33d80cca90 con 0x7f33d80cda90 2023-08-09T13:50:21.367 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:21.365+0000 7f33dfdcb640 1 -- --> v2:172.21.15.43:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f33d80cbc70 con 0x7f33d80ccf20 2023-08-09T13:50:21.367 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:21.365+0000 7f33dfdcb640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f33d80cdf00 con 0x7f33d815c630 2023-08-09T13:50:21.368 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:21.365+0000 7f33dd366640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f33d80ccf20 0x7f33d80cd550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:50:21.368 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:21.365+0000 7f33dd366640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f33d80ccf20 0x7f33d80cd550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:44170/0 (socket says 172.21.15.43:44170) 2023-08-09T13:50:21.368 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:21.365+0000 7f33dd366640 1 -- 172.21.15.43:0/2122109454 learned_addr learned my addr 172.21.15.43:0/2122109454 (peer_addr_for_me v2:172.21.15.43:0/0) 2023-08-09T13:50:21.368 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:21.366+0000 7f33ddb67640 1 --2- 172.21.15.43:0/2122109454 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f33d815c630 0x7f33d80abef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:50:21.368 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:21.366+0000 7f33dd366640 1 -- 172.21.15.43:0/2122109454 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f33d815c630 msgr2=0x7f33d80abef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-09T13:50:21.369 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:21.366+0000 7f33dd366640 1 --2- 172.21.15.43:0/2122109454 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f33d815c630 0x7f33d80abef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:50:21.369 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:21.366+0000 7f33dd366640 1 -- 172.21.15.43:0/2122109454 >> v1:172.21.15.203:6789/0 conn(0x7f33d80cda90 legacy=0x7f33d80c2f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-09T13:50:21.369 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:21.366+0000 7f33dd366640 1 -- 172.21.15.43:0/2122109454 --> v2:172.21.15.43:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f33d815da20 con 0x7f33d80ccf20 2023-08-09T13:50:21.370 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:21.367+0000 7f33dd366640 1 --2- 172.21.15.43:0/2122109454 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f33d80ccf20 0x7f33d80cd550 secure :-1 s=READY pgs=972 cs=0 l=1 rev1=1 crypto rx=0x7f33c8009870 tx=0x7f33c802b8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f69fdd0898550d11 server_cookie=0 in_seq=0 out_seq=0 2023-08-09T13:50:21.370 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:21.367+0000 7f33dcb65640 1 -- 172.21.15.43:0/2122109454 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f33c8031640 con 0x7f33d80ccf20 2023-08-09T13:50:21.370 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:21.367+0000 7f33dcb65640 1 -- 172.21.15.43:0/2122109454 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f33c80317a0 con 0x7f33d80ccf20 2023-08-09T13:50:21.370 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:21.367+0000 7f33dcb65640 1 -- 172.21.15.43:0/2122109454 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f33c8031b30 con 0x7f33d80ccf20 2023-08-09T13:50:21.370 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:21.368+0000 7f33dfdcb640 1 -- 172.21.15.43:0/2122109454 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f33d80ccf20 msgr2=0x7f33d80cd550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:50:21.371 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:21.368+0000 7f33dfdcb640 1 --2- 172.21.15.43:0/2122109454 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f33d80ccf20 0x7f33d80cd550 secure :-1 s=READY pgs=972 cs=0 l=1 rev1=1 crypto rx=0x7f33c8009870 tx=0x7f33c802b8f0 comp rx=0 tx=0).stop 2023-08-09T13:50:21.371 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:21.368+0000 7f33dfdcb640 1 -- 172.21.15.43:0/2122109454 shutdown_connections 2023-08-09T13:50:21.371 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:21.368+0000 7f33dfdcb640 1 --2- 172.21.15.43:0/2122109454 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f33d80ccf20 0x7f33d80cd550 unknown :-1 s=CLOSED pgs=972 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:50:21.371 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:21.368+0000 7f33dfdcb640 1 --2- 172.21.15.43:0/2122109454 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f33d815c630 0x7f33d80abef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:50:21.371 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:21.368+0000 7f33dfdcb640 1 -- 172.21.15.43:0/2122109454 >> 172.21.15.43:0/2122109454 conn(0x7f33d80ce0a0 msgr2=0x7f33d80ce480 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-09T13:50:21.371 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:21.368+0000 7f33dfdcb640 1 -- 172.21.15.43:0/2122109454 shutdown_connections 2023-08-09T13:50:21.371 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:21.368+0000 7f33dfdcb640 1 -- 172.21.15.43:0/2122109454 wait complete. 2023-08-09T13:50:21.372 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:21.370+0000 7f33dfdcb640 1 Processor -- start 2023-08-09T13:50:21.372 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:21.370+0000 7f33dfdcb640 1 -- start start 2023-08-09T13:50:21.373 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:21.371+0000 7f33dfdcb640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f33d80ccf20 0x7f33d8155dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:50:21.373 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:21.371+0000 7f33dfdcb640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f33d80cda90 0x7f33d80c5b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:50:21.374 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:21.371+0000 7f33dfdcb640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f33d815c630 0x7f33d80c9770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:50:21.374 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:21.371+0000 7f33dfdcb640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f33d8156310 con 0x7f33d80cda90 2023-08-09T13:50:21.374 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:21.371+0000 7f33dfdcb640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f33d80c64d0 con 0x7f33d815c630 2023-08-09T13:50:21.374 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:21.371+0000 7f33dfdcb640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f33d80c67b0 con 0x7f33d80ccf20 2023-08-09T13:50:21.374 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:21.371+0000 7f33dd366640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f33d80cda90 0x7f33d80c5b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:50:21.374 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:21.371+0000 7f33dd366640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f33d80cda90 0x7f33d80c5b80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:44176/0 (socket says 172.21.15.43:44176) 2023-08-09T13:50:21.374 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:21.371+0000 7f33dd366640 1 -- 172.21.15.43:0/3126693416 learned_addr learned my addr 172.21.15.43:0/3126693416 (peer_addr_for_me v2:172.21.15.43:0/0) 2023-08-09T13:50:21.374 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:21.372+0000 7f33ddb67640 1 --2- 172.21.15.43:0/3126693416 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f33d80ccf20 0x7f33d8155dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:50:21.375 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:21.372+0000 7f33de368640 1 --2- 172.21.15.43:0/3126693416 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f33d815c630 0x7f33d80c9770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:50:21.375 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:21.372+0000 7f33dd366640 1 -- 172.21.15.43:0/3126693416 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f33d80ccf20 msgr2=0x7f33d8155dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:50:21.375 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:21.372+0000 7f33dd366640 1 --2- 172.21.15.43:0/3126693416 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f33d80ccf20 0x7f33d8155dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:50:21.375 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:21.372+0000 7f33dd366640 1 -- 172.21.15.43:0/3126693416 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f33d815c630 msgr2=0x7f33d80c9770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:50:21.375 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:21.372+0000 7f33dd366640 1 --2- 172.21.15.43:0/3126693416 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f33d815c630 0x7f33d80c9770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:50:21.375 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:21.372+0000 7f33dd366640 1 -- 172.21.15.43:0/3126693416 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f33c80094d0 con 0x7f33d80cda90 2023-08-09T13:50:21.376 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:21.372+0000 7f33dd366640 1 --2- 172.21.15.43:0/3126693416 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f33d80cda90 0x7f33d80c5b80 secure :-1 s=READY pgs=973 cs=0 l=1 rev1=1 crypto rx=0x7f33c8000c00 tx=0x7f33c80046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-09T13:50:21.376 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:21.373+0000 7f33ce7fc640 1 -- 172.21.15.43:0/3126693416 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f33c803ac60 con 0x7f33d80cda90 2023-08-09T13:50:21.376 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:21.373+0000 7f33ce7fc640 1 -- 172.21.15.43:0/3126693416 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f33c803adc0 con 0x7f33d80cda90 2023-08-09T13:50:21.377 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:21.373+0000 7f33dfdcb640 1 -- 172.21.15.43:0/3126693416 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f33d80c6a90 con 0x7f33d80cda90 2023-08-09T13:50:21.377 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:21.373+0000 7f33ce7fc640 1 -- 172.21.15.43:0/3126693416 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f33c8044630 con 0x7f33d80cda90 2023-08-09T13:50:21.377 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:21.373+0000 7f33dfdcb640 1 -- 172.21.15.43:0/3126693416 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f33d80b0ff0 con 0x7f33d80cda90 2023-08-09T13:50:21.377 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:21.374+0000 7f33dfdcb640 1 -- 172.21.15.43:0/3126693416 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f33d80739f0 con 0x7f33d80cda90 2023-08-09T13:50:21.377 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:21.375+0000 7f33ce7fc640 1 -- 172.21.15.43:0/3126693416 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 29) v1 ==== 88443+0+0 (secure 0 0 0) 0x7f33c802f050 con 0x7f33d80cda90 2023-08-09T13:50:21.378 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:21.376+0000 7f33ce7fc640 1 --2- 172.21.15.43:0/3126693416 >> [v2:172.21.15.203:6832/3325141227,v1:172.21.15.203:6833/3325141227] conn(0x7f33bc06a170 0x7f33bc06c630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:50:21.378 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:21.376+0000 7f33ce7fc640 1 -- 172.21.15.43:0/3126693416 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(81..81 src has 1..81) v4 ==== 6434+0+0 (secure 0 0 0) 0x7f33c80aa810 con 0x7f33d80cda90 2023-08-09T13:50:21.378 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:21.377+0000 7f33ddb67640 1 --2- 172.21.15.43:0/3126693416 >> [v2:172.21.15.203:6832/3325141227,v1:172.21.15.203:6833/3325141227] conn(0x7f33bc06a170 0x7f33bc06c630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:50:21.379 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:21.378+0000 7f33ddb67640 1 --2- 172.21.15.43:0/3126693416 >> [v2:172.21.15.203:6832/3325141227,v1:172.21.15.203:6833/3325141227] conn(0x7f33bc06a170 0x7f33bc06c630 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7f33d4006fd0 tx=0x7f33d4008040 comp rx=0 tx=0).ready entity=mgr.5519 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-09T13:50:21.381 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:21.380+0000 7f33ce7fc640 1 -- 172.21.15.43:0/3126693416 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178066 (secure 0 0 0) 0x7f33c8075b50 con 0x7f33d80cda90 2023-08-09T13:50:21.576 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:21.574+0000 7f33dfdcb640 1 -- 172.21.15.43:0/3126693416 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command([{prefix=config set, name=mon_allow_pool_delete}] v 0) v1 -- 0x7f33d8072690 con 0x7f33d80cda90 2023-08-09T13:50:21.582 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:21.581+0000 7f33ce7fc640 1 -- 172.21.15.43:0/3126693416 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_allow_pool_delete}]=0 v22)=0 v22) v1 ==== 122+0+0 (secure 0 0 0) 0x7f33c8031e60 con 0x7f33d80cda90 2023-08-09T13:50:21.585 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:21.584+0000 7f33dfdcb640 1 -- 172.21.15.43:0/3126693416 >> [v2:172.21.15.203:6832/3325141227,v1:172.21.15.203:6833/3325141227] conn(0x7f33bc06a170 msgr2=0x7f33bc06c630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:50:21.586 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:21.584+0000 7f33dfdcb640 1 --2- 172.21.15.43:0/3126693416 >> [v2:172.21.15.203:6832/3325141227,v1:172.21.15.203:6833/3325141227] conn(0x7f33bc06a170 0x7f33bc06c630 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7f33d4006fd0 tx=0x7f33d4008040 comp rx=0 tx=0).stop 2023-08-09T13:50:21.586 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:21.584+0000 7f33dfdcb640 1 -- 172.21.15.43:0/3126693416 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f33d80cda90 msgr2=0x7f33d80c5b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:50:21.586 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:21.584+0000 7f33dfdcb640 1 --2- 172.21.15.43:0/3126693416 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f33d80cda90 0x7f33d80c5b80 secure :-1 s=READY pgs=973 cs=0 l=1 rev1=1 crypto rx=0x7f33c8000c00 tx=0x7f33c80046c0 comp rx=0 tx=0).stop 2023-08-09T13:50:21.586 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:21.584+0000 7f33dfdcb640 1 -- 172.21.15.43:0/3126693416 shutdown_connections 2023-08-09T13:50:21.586 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:21.584+0000 7f33dfdcb640 1 --2- 172.21.15.43:0/3126693416 >> [v2:172.21.15.203:6832/3325141227,v1:172.21.15.203:6833/3325141227] conn(0x7f33bc06a170 0x7f33bc06c630 unknown :-1 s=CLOSED pgs=44 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:50:21.586 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:21.584+0000 7f33dfdcb640 1 --2- 172.21.15.43:0/3126693416 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f33d815c630 0x7f33d80c9770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:50:21.587 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:21.584+0000 7f33dfdcb640 1 --2- 172.21.15.43:0/3126693416 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f33d80cda90 0x7f33d80c5b80 unknown :-1 s=CLOSED pgs=973 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:50:21.587 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:21.584+0000 7f33dfdcb640 1 --2- 172.21.15.43:0/3126693416 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f33d80ccf20 0x7f33d8155dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:50:21.587 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:21.584+0000 7f33dfdcb640 1 -- 172.21.15.43:0/3126693416 >> 172.21.15.43:0/3126693416 conn(0x7f33d80ce0a0 msgr2=0x7f33d80c8860 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-09T13:50:21.587 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:21.584+0000 7f33dfdcb640 1 -- 172.21.15.43:0/3126693416 shutdown_connections 2023-08-09T13:50:21.587 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:21.584+0000 7f33dfdcb640 1 -- 172.21.15.43:0/3126693416 wait complete. 2023-08-09T13:50:21.603 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2023-08-09T13:50:21.604 DEBUG:teuthology.orchestra.run.smithi043:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2023-08-09T13:50:21.718 DEBUG:teuthology.orchestra.run.smithi203:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2023-08-09T13:50:21.837 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2023-08-09T13:50:21.838 DEBUG:teuthology.orchestra.run.smithi043:> set -ex 2023-08-09T13:50:21.839 DEBUG:teuthology.orchestra.run.smithi043:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-09T13:50:21.902 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4054441,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6498844k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=27,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3249420k,nr_inodes=812355,mode=700,uid=1000,gid=1275 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6498844k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2023-08-09T13:50:21.902 DEBUG:teuthology.orchestra.run.smithi203:> set -ex 2023-08-09T13:50:21.903 DEBUG:teuthology.orchestra.run.smithi203:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-09T13:50:21.959 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4053648,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497576k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=27,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248784k,nr_inodes=812196,mode=700,uid=1000,gid=1275 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497576k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2023-08-09T13:50:21.960 DEBUG:teuthology.orchestra.run.smithi043:> set -ex 2023-08-09T13:50:21.960 DEBUG:teuthology.orchestra.run.smithi043:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-09T13:50:22.024 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4054441,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6498844k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=27,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3249420k,nr_inodes=812355,mode=700,uid=1000,gid=1275 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6498844k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2023-08-09T13:50:22.024 DEBUG:teuthology.orchestra.run.smithi203:> set -ex 2023-08-09T13:50:22.025 DEBUG:teuthology.orchestra.run.smithi203:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-09T13:50:22.086 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4053648,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497576k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=27,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248784k,nr_inodes=812196,mode=700,uid=1000,gid=1275 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497576k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2023-08-09T13:50:22.086 INFO:tasks.cephfs.cephfs_test_case:[, , , ] 2023-08-09T13:50:22.087 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-08-09T13:50:22.326 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:22.325+0000 7f1b3cbd6640 1 Processor -- start 2023-08-09T13:50:22.327 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:22.325+0000 7f1b3cbd6640 1 -- start start 2023-08-09T13:50:22.328 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:22.327+0000 7f1b3cbd6640 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7f1b3815c630 0x7f1b380abef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:50:22.328 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:22.327+0000 7f1b3cbd6640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f1b380ccf20 0x7f1b380cd550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:50:22.329 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:22.327+0000 7f1b3cbd6640 1 -- --> v1:172.21.15.203:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1b380cca90 con 0x7f1b380cda90 2023-08-09T13:50:22.329 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:22.327+0000 7f1b3cbd6640 1 -- --> v2:172.21.15.43:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1b380cbc70 con 0x7f1b380ccf20 2023-08-09T13:50:22.329 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:22.327+0000 7f1b3cbd6640 1 -- --> v2:172.21.15.203:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1b380cdea0 con 0x7f1b3815c630 2023-08-09T13:50:22.330 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:22.327+0000 7f1b2ffff640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f1b380ccf20 0x7f1b380cd550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:50:22.330 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:22.327+0000 7f1b2ffff640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f1b380ccf20 0x7f1b380cd550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:44188/0 (socket says 172.21.15.43:44188) 2023-08-09T13:50:22.330 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:22.327+0000 7f1b2ffff640 1 -- 172.21.15.43:0/3010500516 learned_addr learned my addr 172.21.15.43:0/3010500516 (peer_addr_for_me v2:172.21.15.43:0/0) 2023-08-09T13:50:22.330 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:22.328+0000 7f1b3659c640 1 --2- 172.21.15.43:0/3010500516 >> v2:172.21.15.203:3300/0 conn(0x7f1b3815c630 0x7f1b380abef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:50:22.330 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:22.328+0000 7f1b2ffff640 1 -- 172.21.15.43:0/3010500516 >> v2:172.21.15.203:3300/0 conn(0x7f1b3815c630 msgr2=0x7f1b380abef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-09T13:50:22.330 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:22.328+0000 7f1b2ffff640 1 --2- 172.21.15.43:0/3010500516 >> v2:172.21.15.203:3300/0 conn(0x7f1b3815c630 0x7f1b380abef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:50:22.331 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:22.328+0000 7f1b2ffff640 1 -- 172.21.15.43:0/3010500516 >> v1:172.21.15.203:6789/0 conn(0x7f1b380cda90 legacy=0x7f1b380c2f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-09T13:50:22.331 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:22.328+0000 7f1b2ffff640 1 -- 172.21.15.43:0/3010500516 --> v2:172.21.15.43:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1b3815da20 con 0x7f1b380ccf20 2023-08-09T13:50:22.332 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:22.329+0000 7f1b2ffff640 1 --2- 172.21.15.43:0/3010500516 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f1b380ccf20 0x7f1b380cd550 secure :-1 s=READY pgs=974 cs=0 l=1 rev1=1 crypto rx=0x7f1b380cbbb0 tx=0x7f1b2002b910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=26dc2628632a2ab8 server_cookie=0 in_seq=0 out_seq=0 2023-08-09T13:50:22.332 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:22.329+0000 7f1b35d9b640 1 -- 172.21.15.43:0/3010500516 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1b20031870 con 0x7f1b380ccf20 2023-08-09T13:50:22.332 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:22.329+0000 7f1b35d9b640 1 -- 172.21.15.43:0/3010500516 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1b200319d0 con 0x7f1b380ccf20 2023-08-09T13:50:22.332 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:22.329+0000 7f1b35d9b640 1 -- 172.21.15.43:0/3010500516 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1b20031d60 con 0x7f1b380ccf20 2023-08-09T13:50:22.332 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:22.330+0000 7f1b3cbd6640 1 -- 172.21.15.43:0/3010500516 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f1b380ccf20 msgr2=0x7f1b380cd550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:50:22.333 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:22.330+0000 7f1b3cbd6640 1 --2- 172.21.15.43:0/3010500516 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f1b380ccf20 0x7f1b380cd550 secure :-1 s=READY pgs=974 cs=0 l=1 rev1=1 crypto rx=0x7f1b380cbbb0 tx=0x7f1b2002b910 comp rx=0 tx=0).stop 2023-08-09T13:50:22.333 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:22.330+0000 7f1b3cbd6640 1 -- 172.21.15.43:0/3010500516 shutdown_connections 2023-08-09T13:50:22.333 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:22.330+0000 7f1b3cbd6640 1 --2- 172.21.15.43:0/3010500516 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f1b380ccf20 0x7f1b380cd550 unknown :-1 s=CLOSED pgs=974 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:50:22.333 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:22.330+0000 7f1b3cbd6640 1 --2- 172.21.15.43:0/3010500516 >> v2:172.21.15.203:3300/0 conn(0x7f1b3815c630 0x7f1b380abef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:50:22.333 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:22.330+0000 7f1b3cbd6640 1 -- 172.21.15.43:0/3010500516 >> 172.21.15.43:0/3010500516 conn(0x7f1b380ce0a0 msgr2=0x7f1b380ce480 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-09T13:50:22.333 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:22.330+0000 7f1b3cbd6640 1 -- 172.21.15.43:0/3010500516 shutdown_connections 2023-08-09T13:50:22.333 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:22.330+0000 7f1b3cbd6640 1 -- 172.21.15.43:0/3010500516 wait complete. 2023-08-09T13:50:22.334 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:22.332+0000 7f1b3cbd6640 1 Processor -- start 2023-08-09T13:50:22.334 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:22.333+0000 7f1b3cbd6640 1 -- start start 2023-08-09T13:50:22.336 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:22.334+0000 7f1b3cbd6640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f1b380ccf20 0x7f1b380adb00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:50:22.336 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:22.334+0000 7f1b3cbd6640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f1b380cda90 0x7f1b3814bf30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:50:22.337 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:22.334+0000 7f1b3cbd6640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f1b3815c630 0x7f1b380c6140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:50:22.337 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:22.334+0000 7f1b3cbd6640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1b3814c470 con 0x7f1b380ccf20 2023-08-09T13:50:22.337 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:22.334+0000 7f1b3cbd6640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1b380c7420 con 0x7f1b380cda90 2023-08-09T13:50:22.337 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:22.334+0000 7f1b3cbd6640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1b380c6e10 con 0x7f1b3815c630 2023-08-09T13:50:22.337 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:22.334+0000 7f1b3659c640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f1b380ccf20 0x7f1b380adb00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:50:22.337 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:22.334+0000 7f1b3659c640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f1b380ccf20 0x7f1b380adb00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:44196/0 (socket says 172.21.15.43:44196) 2023-08-09T13:50:22.337 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:22.334+0000 7f1b3659c640 1 -- 172.21.15.43:0/2729040761 learned_addr learned my addr 172.21.15.43:0/2729040761 (peer_addr_for_me v2:172.21.15.43:0/0) 2023-08-09T13:50:22.338 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:22.335+0000 7f1b3659c640 1 -- 172.21.15.43:0/2729040761 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f1b3815c630 msgr2=0x7f1b380c6140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:50:22.338 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:22.335+0000 7f1b3659c640 1 --2- 172.21.15.43:0/2729040761 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f1b3815c630 0x7f1b380c6140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:50:22.338 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:22.335+0000 7f1b3659c640 1 -- 172.21.15.43:0/2729040761 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f1b380cda90 msgr2=0x7f1b3814bf30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:50:22.338 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:22.335+0000 7f1b2ffff640 1 --2- 172.21.15.43:0/2729040761 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f1b380cda90 0x7f1b3814bf30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:50:22.338 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:22.335+0000 7f1b3659c640 1 --2- 172.21.15.43:0/2729040761 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f1b380cda90 0x7f1b3814bf30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:50:22.338 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:22.335+0000 7f1b3659c640 1 -- 172.21.15.43:0/2729040761 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1b200094d0 con 0x7f1b380ccf20 2023-08-09T13:50:22.339 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:22.335+0000 7f1b3659c640 1 --2- 172.21.15.43:0/2729040761 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f1b380ccf20 0x7f1b380adb00 secure :-1 s=READY pgs=975 cs=0 l=1 rev1=1 crypto rx=0x7f1b2800d620 tx=0x7f1b2800dae0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-09T13:50:22.339 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:22.335+0000 7f1b2effd640 1 -- 172.21.15.43:0/2729040761 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1b2800b620 con 0x7f1b380ccf20 2023-08-09T13:50:22.339 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:22.335+0000 7f1b2effd640 1 -- 172.21.15.43:0/2729040761 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1b2800b780 con 0x7f1b380ccf20 2023-08-09T13:50:22.340 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:22.335+0000 7f1b2effd640 1 -- 172.21.15.43:0/2729040761 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1b2800bb30 con 0x7f1b380ccf20 2023-08-09T13:50:22.340 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:22.336+0000 7f1b3cbd6640 1 -- 172.21.15.43:0/2729040761 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1b3815f8e0 con 0x7f1b380ccf20 2023-08-09T13:50:22.340 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:22.336+0000 7f1b3cbd6640 1 -- 172.21.15.43:0/2729040761 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1b3815fbc0 con 0x7f1b380ccf20 2023-08-09T13:50:22.340 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:22.337+0000 7f1b3cbd6640 1 -- 172.21.15.43:0/2729040761 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1b380739f0 con 0x7f1b380ccf20 2023-08-09T13:50:22.340 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:22.338+0000 7f1b2effd640 1 -- 172.21.15.43:0/2729040761 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 29) v1 ==== 88443+0+0 (secure 0 0 0) 0x7f1b2800bc90 con 0x7f1b380ccf20 2023-08-09T13:50:22.341 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:22.339+0000 7f1b2effd640 1 --2- 172.21.15.43:0/2729040761 >> [v2:172.21.15.203:6832/3325141227,v1:172.21.15.203:6833/3325141227] conn(0x7f1b1806a1a0 0x7f1b1806c660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:50:22.341 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:22.339+0000 7f1b2effd640 1 -- 172.21.15.43:0/2729040761 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(81..81 src has 1..81) v4 ==== 6434+0+0 (secure 0 0 0) 0x7f1b28088cf0 con 0x7f1b380ccf20 2023-08-09T13:50:22.341 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:22.340+0000 7f1b2ffff640 1 --2- 172.21.15.43:0/2729040761 >> [v2:172.21.15.203:6832/3325141227,v1:172.21.15.203:6833/3325141227] conn(0x7f1b1806a1a0 0x7f1b1806c660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:50:22.342 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:22.341+0000 7f1b2ffff640 1 --2- 172.21.15.43:0/2729040761 >> [v2:172.21.15.203:6832/3325141227,v1:172.21.15.203:6833/3325141227] conn(0x7f1b1806a1a0 0x7f1b1806c660 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7f1b20036000 tx=0x7f1b2003c040 comp rx=0 tx=0).ready entity=mgr.5519 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-09T13:50:22.345 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:22.344+0000 7f1b2effd640 1 -- 172.21.15.43:0/2729040761 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178066 (secure 0 0 0) 0x7f1b2808d070 con 0x7f1b380ccf20 2023-08-09T13:50:22.580 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:22.579+0000 7f1b3cbd6640 1 -- 172.21.15.43:0/2729040761 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f1b380cc710 con 0x7f1b380ccf20 2023-08-09T13:50:22.582 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:22.581+0000 7f1b2effd640 1 -- 172.21.15.43:0/2729040761 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 28 v28) v1 ==== 94+0+3705 (secure 0 0 0) 0x7f1b28054150 con 0x7f1b380ccf20 2023-08-09T13:50:22.583 INFO:teuthology.orchestra.run.smithi043.stdout: 2023-08-09T13:50:22.583 INFO:teuthology.orchestra.run.smithi043.stdout:{"epoch":28,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5369,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.203:6837/1426673901","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6836","nonce":1426673901},{"type":"v1","addr":"172.21.15.203:6837","nonce":1426673901}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":21},{"gid":5728,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.43:6833/1096124235","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6832","nonce":1096124235},{"type":"v1","addr":"172.21.15.43:6833","nonce":1096124235}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":20},{"gid":5731,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.43:6835/1688804357","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6834","nonce":1688804357},{"type":"v1","addr":"172.21.15.43:6835","nonce":1688804357}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":21},{"gid":6040,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.203:6835/3640054235","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6834","nonce":3640054235},{"type":"v1","addr":"172.21.15.203:6835","nonce":3640054235}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":28}],"filesystems":[]} 2023-08-09T13:50:22.583 INFO:teuthology.orchestra.run.smithi043.stderr:dumped fsmap epoch 28 2023-08-09T13:50:22.586 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:22.584+0000 7f1b3cbd6640 1 -- 172.21.15.43:0/2729040761 >> [v2:172.21.15.203:6832/3325141227,v1:172.21.15.203:6833/3325141227] conn(0x7f1b1806a1a0 msgr2=0x7f1b1806c660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:50:22.586 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:22.584+0000 7f1b3cbd6640 1 --2- 172.21.15.43:0/2729040761 >> [v2:172.21.15.203:6832/3325141227,v1:172.21.15.203:6833/3325141227] conn(0x7f1b1806a1a0 0x7f1b1806c660 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7f1b20036000 tx=0x7f1b2003c040 comp rx=0 tx=0).stop 2023-08-09T13:50:22.587 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:22.584+0000 7f1b3cbd6640 1 -- 172.21.15.43:0/2729040761 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f1b380ccf20 msgr2=0x7f1b380adb00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:50:22.587 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:22.584+0000 7f1b3cbd6640 1 --2- 172.21.15.43:0/2729040761 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f1b380ccf20 0x7f1b380adb00 secure :-1 s=READY pgs=975 cs=0 l=1 rev1=1 crypto rx=0x7f1b2800d620 tx=0x7f1b2800dae0 comp rx=0 tx=0).stop 2023-08-09T13:50:22.587 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:22.584+0000 7f1b3cbd6640 1 -- 172.21.15.43:0/2729040761 shutdown_connections 2023-08-09T13:50:22.587 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:22.584+0000 7f1b3cbd6640 1 --2- 172.21.15.43:0/2729040761 >> [v2:172.21.15.203:6832/3325141227,v1:172.21.15.203:6833/3325141227] conn(0x7f1b1806a1a0 0x7f1b1806c660 unknown :-1 s=CLOSED pgs=45 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:50:22.587 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:22.584+0000 7f1b3cbd6640 1 --2- 172.21.15.43:0/2729040761 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f1b3815c630 0x7f1b380c6140 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:50:22.587 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:22.584+0000 7f1b3cbd6640 1 --2- 172.21.15.43:0/2729040761 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f1b380cda90 0x7f1b3814bf30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:50:22.588 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:22.584+0000 7f1b3cbd6640 1 --2- 172.21.15.43:0/2729040761 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f1b380ccf20 0x7f1b380adb00 unknown :-1 s=CLOSED pgs=975 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:50:22.588 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:22.584+0000 7f1b3cbd6640 1 -- 172.21.15.43:0/2729040761 >> 172.21.15.43:0/2729040761 conn(0x7f1b380ce0a0 msgr2=0x7f1b380abdb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-09T13:50:22.588 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:22.584+0000 7f1b3cbd6640 1 -- 172.21.15.43:0/2729040761 shutdown_connections 2023-08-09T13:50:22.588 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:22.584+0000 7f1b3cbd6640 1 -- 172.21.15.43:0/2729040761 wait complete. 2023-08-09T13:50:22.605 INFO:tasks.ceph.mds.a:Restarting daemon 2023-08-09T13:50:22.605 INFO:tasks.ceph.mds.a:Stopping old one... 2023-08-09T13:50:22.606 DEBUG:tasks.ceph.mds.a:waiting for process to exit 2023-08-09T13:50:22.606 INFO:teuthology.orchestra.run:waiting for 300 2023-08-09T13:50:22.606 INFO:tasks.ceph.mds.c:Restarting daemon 2023-08-09T13:50:22.606 INFO:tasks.ceph.mds.c:Stopping old one... 2023-08-09T13:50:22.606 DEBUG:tasks.ceph.mds.c:waiting for process to exit 2023-08-09T13:50:22.607 INFO:teuthology.orchestra.run:waiting for 300 2023-08-09T13:50:22.607 INFO:tasks.ceph.mds.b:Restarting daemon 2023-08-09T13:50:22.608 INFO:tasks.ceph.mds.b:Stopping old one... 2023-08-09T13:50:22.608 DEBUG:tasks.ceph.mds.b:waiting for process to exit 2023-08-09T13:50:22.608 INFO:teuthology.orchestra.run:waiting for 300 2023-08-09T13:50:22.608 INFO:tasks.ceph.mds.d:Restarting daemon 2023-08-09T13:50:22.609 INFO:tasks.ceph.mds.d:Stopping old one... 2023-08-09T13:50:22.609 DEBUG:tasks.ceph.mds.d:waiting for process to exit 2023-08-09T13:50:22.609 INFO:teuthology.orchestra.run:waiting for 300 2023-08-09T13:50:22.623 INFO:tasks.ceph.mds.a:Stopped 2023-08-09T13:50:22.624 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i a 2023-08-09T13:50:22.625 INFO:tasks.ceph.mds.c:Stopped 2023-08-09T13:50:22.625 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i c 2023-08-09T13:50:22.635 INFO:tasks.ceph.mds.b:Stopped 2023-08-09T13:50:22.636 DEBUG:teuthology.orchestra.run.smithi203:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i b 2023-08-09T13:50:22.637 INFO:tasks.ceph.mds.d:Stopped 2023-08-09T13:50:22.638 DEBUG:teuthology.orchestra.run.smithi203:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i d 2023-08-09T13:50:22.673 INFO:tasks.ceph.mds.a:Started 2023-08-09T13:50:22.674 DEBUG:teuthology.parallel:result is None 2023-08-09T13:50:22.683 INFO:tasks.ceph.mds.c:Started 2023-08-09T13:50:22.683 DEBUG:teuthology.parallel:result is None 2023-08-09T13:50:22.685 INFO:tasks.ceph.mds.b:Started 2023-08-09T13:50:22.685 DEBUG:teuthology.parallel:result is None 2023-08-09T13:50:22.692 INFO:tasks.ceph.mds.d:Started 2023-08-09T13:50:22.693 DEBUG:teuthology.parallel:result is None 2023-08-09T13:50:22.693 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist clear 2023-08-09T13:50:22.847 INFO:tasks.ceph.mds.c.smithi043.stdout:starting mds.c at 2023-08-09T13:50:22.848 INFO:tasks.ceph.mds.a.smithi043.stdout:starting mds.a at 2023-08-09T13:50:22.853 INFO:tasks.ceph.mds.b.smithi203.stdout:starting mds.b at 2023-08-09T13:50:22.853 INFO:tasks.ceph.mds.d.smithi203.stdout:starting mds.d at 2023-08-09T13:50:22.898 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:22.896+0000 7f790656a640 1 Processor -- start 2023-08-09T13:50:22.899 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:22.897+0000 7f790656a640 1 -- start start 2023-08-09T13:50:22.899 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:22.898+0000 7f790656a640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f7900142ca0 0x7f79000c96e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:50:22.900 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:22.898+0000 7f790656a640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f79000cd510 0x7f79000c3290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:50:22.900 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:22.898+0000 7f790656a640 1 -- --> v1:172.21.15.43:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7900143080 con 0x7f790015ce40 2023-08-09T13:50:22.900 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:22.898+0000 7f790656a640 1 -- --> v2:172.21.15.43:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f790006d400 con 0x7f7900142ca0 2023-08-09T13:50:22.900 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:22.898+0000 7f790656a640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f79000c9d90 con 0x7f79000cd510 2023-08-09T13:50:22.900 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:22.898+0000 7f78f7fff640 1 --1- >> v1:172.21.15.43:6789/0 conn(0x7f790015ce40 0x7f79000cce00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.43:6789/0 says I am v1:172.21.15.43:44628/0 (socket says 172.21.15.43:44628) 2023-08-09T13:50:22.900 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:22.898+0000 7f78f7fff640 1 -- 172.21.15.43:0/3805327027 learned_addr learned my addr 172.21.15.43:0/3805327027 (peer_addr_for_me v1:172.21.15.43:0/0) 2023-08-09T13:50:22.901 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:22.898+0000 7f78fffff640 1 --2- 172.21.15.43:0/3805327027 >> v2:172.21.15.43:3300/0 conn(0x7f7900142ca0 0x7f79000c96e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:50:22.901 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:22.899+0000 7f7904b07640 1 --2- 172.21.15.43:0/3805327027 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f79000cd510 0x7f79000c3290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:50:22.902 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:22.899+0000 7f78ff7fe640 1 -- 172.21.15.43:0/3805327027 <== mon.0 v1:172.21.15.43:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 182147559 0 0) 0x7f7900143080 con 0x7f790015ce40 2023-08-09T13:50:22.902 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:22.899+0000 7f78ff7fe640 1 -- 172.21.15.43:0/3805327027 --> v1:172.21.15.43:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f78ec003520 con 0x7f790015ce40 2023-08-09T13:50:22.902 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:22.899+0000 7f78fffff640 1 -- 172.21.15.43:0/3805327027 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f79000cd510 msgr2=0x7f79000c3290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-09T13:50:22.902 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:22.899+0000 7f78fffff640 1 --2- 172.21.15.43:0/3805327027 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f79000cd510 0x7f79000c3290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:50:22.902 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:22.899+0000 7f78fffff640 1 -- 172.21.15.43:0/3805327027 >> v1:172.21.15.43:6789/0 conn(0x7f790015ce40 legacy=0x7f79000cce00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:50:22.902 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:22.899+0000 7f78fffff640 1 -- 172.21.15.43:0/3805327027 --> v2:172.21.15.43:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f79000ca070 con 0x7f7900142ca0 2023-08-09T13:50:22.902 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:22.899+0000 7f7904b07640 1 --2- 172.21.15.43:0/3805327027 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f79000cd510 0x7f79000c3290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-08-09T13:50:22.903 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:22.900+0000 7f78fffff640 1 --2- 172.21.15.43:0/3805327027 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f7900142ca0 0x7f79000c96e0 secure :-1 s=READY pgs=982 cs=0 l=1 rev1=1 crypto rx=0x7f78f00097a0 tx=0x7f78f002b8b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6a2d4518db737962 server_cookie=0 in_seq=0 out_seq=0 2023-08-09T13:50:22.904 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:22.901+0000 7f78ff7fe640 1 -- 172.21.15.43:0/3805327027 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f78f002d040 con 0x7f7900142ca0 2023-08-09T13:50:22.904 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:22.901+0000 7f78ff7fe640 1 -- 172.21.15.43:0/3805327027 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f78f0031810 con 0x7f7900142ca0 2023-08-09T13:50:22.904 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:22.901+0000 7f78ff7fe640 1 -- 172.21.15.43:0/3805327027 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f78f0031b80 con 0x7f7900142ca0 2023-08-09T13:50:22.904 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:22.901+0000 7f790656a640 1 -- 172.21.15.43:0/3805327027 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f7900142ca0 msgr2=0x7f79000c96e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:50:22.904 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:22.901+0000 7f790656a640 1 --2- 172.21.15.43:0/3805327027 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f7900142ca0 0x7f79000c96e0 secure :-1 s=READY pgs=982 cs=0 l=1 rev1=1 crypto rx=0x7f78f00097a0 tx=0x7f78f002b8b0 comp rx=0 tx=0).stop 2023-08-09T13:50:22.904 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:22.901+0000 7f790656a640 1 -- 172.21.15.43:0/3805327027 shutdown_connections 2023-08-09T13:50:22.904 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:22.901+0000 7f790656a640 1 --2- 172.21.15.43:0/3805327027 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f79000cd510 0x7f79000c3290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:50:22.905 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:22.901+0000 7f790656a640 1 --2- 172.21.15.43:0/3805327027 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f7900142ca0 0x7f79000c96e0 unknown :-1 s=CLOSED pgs=982 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:50:22.905 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:22.901+0000 7f790656a640 1 -- 172.21.15.43:0/3805327027 >> 172.21.15.43:0/3805327027 conn(0x7f79000ce0a0 msgr2=0x7f79000ce480 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-09T13:50:22.905 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:22.901+0000 7f790656a640 1 -- 172.21.15.43:0/3805327027 shutdown_connections 2023-08-09T13:50:22.905 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:22.902+0000 7f790656a640 1 -- 172.21.15.43:0/3805327027 wait complete. 2023-08-09T13:50:22.906 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:22.903+0000 7f790656a640 1 Processor -- start 2023-08-09T13:50:22.906 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:22.903+0000 7f790656a640 1 -- start start 2023-08-09T13:50:22.906 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:22.904+0000 7f790656a640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f79000cd510 0x7f7900159a90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:50:22.906 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:22.904+0000 7f790656a640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f7900142ca0 0x7f79001575a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:50:22.906 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:22.905+0000 7f790656a640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f790015ce40 0x7f79000c5170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:50:22.906 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:22.905+0000 7f790656a640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f79000c7090 con 0x7f79000cd510 2023-08-09T13:50:22.907 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:22.905+0000 7f790656a640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f79001588d0 con 0x7f790015ce40 2023-08-09T13:50:22.907 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:22.905+0000 7f790656a640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7900158bb0 con 0x7f7900142ca0 2023-08-09T13:50:22.908 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:22.905+0000 7f78fffff640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f79000cd510 0x7f7900159a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:50:22.908 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:22.905+0000 7f78fffff640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f79000cd510 0x7f7900159a90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:44238/0 (socket says 172.21.15.43:44238) 2023-08-09T13:50:22.908 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:22.905+0000 7f78fffff640 1 -- 172.21.15.43:0/4208473253 learned_addr learned my addr 172.21.15.43:0/4208473253 (peer_addr_for_me v2:172.21.15.43:0/0) 2023-08-09T13:50:22.909 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:22.905+0000 7f78f7fff640 1 --2- 172.21.15.43:0/4208473253 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f7900142ca0 0x7f79001575a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:50:22.909 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:22.905+0000 7f7904b07640 1 --2- 172.21.15.43:0/4208473253 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f790015ce40 0x7f79000c5170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:50:22.909 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:22.905+0000 7f78fffff640 1 -- 172.21.15.43:0/4208473253 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f7900142ca0 msgr2=0x7f79001575a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:50:22.909 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:22.905+0000 7f78fffff640 1 --2- 172.21.15.43:0/4208473253 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f7900142ca0 0x7f79001575a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:50:22.909 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:22.905+0000 7f78fffff640 1 -- 172.21.15.43:0/4208473253 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f790015ce40 msgr2=0x7f79000c5170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:50:22.909 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:22.905+0000 7f78fffff640 1 --2- 172.21.15.43:0/4208473253 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f790015ce40 0x7f79000c5170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:50:22.909 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:22.905+0000 7f78fffff640 1 -- 172.21.15.43:0/4208473253 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f78f8009a70 con 0x7f79000cd510 2023-08-09T13:50:22.910 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:22.905+0000 7f7904b07640 1 --2- 172.21.15.43:0/4208473253 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f790015ce40 0x7f79000c5170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-08-09T13:50:22.910 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:22.906+0000 7f78fffff640 1 --2- 172.21.15.43:0/4208473253 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f79000cd510 0x7f7900159a90 secure :-1 s=READY pgs=983 cs=0 l=1 rev1=1 crypto rx=0x7f78f002b8f0 tx=0x7f78f00046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-09T13:50:22.910 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:22.906+0000 7f78fd7fa640 1 -- 172.21.15.43:0/4208473253 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f78f003ad60 con 0x7f79000cd510 2023-08-09T13:50:22.910 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:22.906+0000 7f78fd7fa640 1 -- 172.21.15.43:0/4208473253 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f78f00443f0 con 0x7f79000cd510 2023-08-09T13:50:22.910 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:22.906+0000 7f790656a640 1 -- 172.21.15.43:0/4208473253 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f78f0009400 con 0x7f79000cd510 2023-08-09T13:50:22.910 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:22.906+0000 7f78fd7fa640 1 -- 172.21.15.43:0/4208473253 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f78f0044580 con 0x7f79000cd510 2023-08-09T13:50:22.910 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:22.906+0000 7f790656a640 1 -- 172.21.15.43:0/4208473253 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f79000c56b0 con 0x7f79000cd510 2023-08-09T13:50:22.911 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:22.908+0000 7f790656a640 1 -- 172.21.15.43:0/4208473253 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f79000c6b80 con 0x7f79000cd510 2023-08-09T13:50:22.911 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:22.908+0000 7f78fd7fa640 1 -- 172.21.15.43:0/4208473253 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 29) v1 ==== 88443+0+0 (secure 0 0 0) 0x7f78f004d450 con 0x7f79000cd510 2023-08-09T13:50:22.911 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:22.909+0000 7f78fd7fa640 1 --2- 172.21.15.43:0/4208473253 >> [v2:172.21.15.203:6832/3325141227,v1:172.21.15.203:6833/3325141227] conn(0x7f78e006a1a0 0x7f78e006c660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:50:22.912 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:22.910+0000 7f78fd7fa640 1 -- 172.21.15.43:0/4208473253 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(81..81 src has 1..81) v4 ==== 6434+0+0 (secure 0 0 0) 0x7f78f00aab50 con 0x7f79000cd510 2023-08-09T13:50:22.912 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:22.910+0000 7f78f7fff640 1 --2- 172.21.15.43:0/4208473253 >> [v2:172.21.15.203:6832/3325141227,v1:172.21.15.203:6833/3325141227] conn(0x7f78e006a1a0 0x7f78e006c660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:50:22.912 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:22.911+0000 7f78f7fff640 1 --2- 172.21.15.43:0/4208473253 >> [v2:172.21.15.203:6832/3325141227,v1:172.21.15.203:6833/3325141227] conn(0x7f78e006a1a0 0x7f78e006c660 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7f7900159570 tx=0x7f78e8009040 comp rx=0 tx=0).ready entity=mgr.5519 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-09T13:50:22.915 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:22.914+0000 7f78fd7fa640 1 -- 172.21.15.43:0/4208473253 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178066 (secure 0 0 0) 0x7f79000c6b80 con 0x7f79000cd510 2023-08-09T13:50:23.133 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:23.132+0000 7f790656a640 1 -- 172.21.15.43:0/4208473253 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "osd blocklist clear"} v 0) v1 -- 0x7f79000c6b80 con 0x7f79000cd510 2023-08-09T13:50:24.151 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:24.140+0000 7f78fd7fa640 1 -- 172.21.15.43:0/4208473253 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist clear"}]=0 removed all blocklist entries v86) v1 ==== 97+0+0 (secure 0 0 0) 0x7f79000c6b80 con 0x7f79000cd510 2023-08-09T13:50:24.152 INFO:teuthology.orchestra.run.smithi043.stderr: removed all blocklist entries 2023-08-09T13:50:24.152 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:24.143+0000 7f790656a640 1 -- 172.21.15.43:0/4208473253 >> [v2:172.21.15.203:6832/3325141227,v1:172.21.15.203:6833/3325141227] conn(0x7f78e006a1a0 msgr2=0x7f78e006c660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:50:24.152 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:24.143+0000 7f790656a640 1 --2- 172.21.15.43:0/4208473253 >> [v2:172.21.15.203:6832/3325141227,v1:172.21.15.203:6833/3325141227] conn(0x7f78e006a1a0 0x7f78e006c660 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7f7900159570 tx=0x7f78e8009040 comp rx=0 tx=0).stop 2023-08-09T13:50:24.153 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:24.143+0000 7f790656a640 1 -- 172.21.15.43:0/4208473253 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f79000cd510 msgr2=0x7f7900159a90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:50:24.153 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:24.143+0000 7f790656a640 1 --2- 172.21.15.43:0/4208473253 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f79000cd510 0x7f7900159a90 secure :-1 s=READY pgs=983 cs=0 l=1 rev1=1 crypto rx=0x7f78f002b8f0 tx=0x7f78f00046c0 comp rx=0 tx=0).stop 2023-08-09T13:50:24.153 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:24.144+0000 7f790656a640 1 -- 172.21.15.43:0/4208473253 shutdown_connections 2023-08-09T13:50:24.153 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:24.144+0000 7f790656a640 1 --2- 172.21.15.43:0/4208473253 >> [v2:172.21.15.203:6832/3325141227,v1:172.21.15.203:6833/3325141227] conn(0x7f78e006a1a0 0x7f78e006c660 unknown :-1 s=CLOSED pgs=46 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:50:24.153 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:24.144+0000 7f790656a640 1 --2- 172.21.15.43:0/4208473253 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f790015ce40 0x7f79000c5170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:50:24.153 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:24.144+0000 7f790656a640 1 --2- 172.21.15.43:0/4208473253 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f7900142ca0 0x7f79001575a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:50:24.153 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:24.144+0000 7f790656a640 1 --2- 172.21.15.43:0/4208473253 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f79000cd510 0x7f7900159a90 unknown :-1 s=CLOSED pgs=983 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:50:24.154 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:24.144+0000 7f790656a640 1 -- 172.21.15.43:0/4208473253 >> 172.21.15.43:0/4208473253 conn(0x7f79000ce0a0 msgr2=0x7f79000c85f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-09T13:50:24.154 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:24.144+0000 7f790656a640 1 -- 172.21.15.43:0/4208473253 shutdown_connections 2023-08-09T13:50:24.154 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:24.144+0000 7f790656a640 1 -- 172.21.15.43:0/4208473253 wait complete. 2023-08-09T13:50:24.169 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth ls --format=json-pretty 2023-08-09T13:50:24.406 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:24.404+0000 7ff641bb5640 1 Processor -- start 2023-08-09T13:50:24.407 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:24.405+0000 7ff641bb5640 1 -- start start 2023-08-09T13:50:24.408 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:24.406+0000 7ff641bb5640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7ff63c0b2790 0x7ff63c0d2ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:50:24.408 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:24.406+0000 7ff641bb5640 1 -- --> v1:172.21.15.43:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff63c0ce090 con 0x7ff63c0b2b70 2023-08-09T13:50:24.408 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:24.406+0000 7ff641bb5640 1 -- --> v1:172.21.15.203:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff63c0cd7f0 con 0x7ff63c0b3050 2023-08-09T13:50:24.409 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:24.406+0000 7ff641bb5640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff63c0b3950 con 0x7ff63c0b2790 2023-08-09T13:50:24.409 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:24.406+0000 7ff632ffd640 1 --1- >> v1:172.21.15.43:6789/0 conn(0x7ff63c0b2b70 0x7ff63c0b5330 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.43:6789/0 says I am v1:172.21.15.43:44638/0 (socket says 172.21.15.43:44638) 2023-08-09T13:50:24.409 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:24.406+0000 7ff632ffd640 1 -- 172.21.15.43:0/3115994926 learned_addr learned my addr 172.21.15.43:0/3115994926 (peer_addr_for_me v1:172.21.15.43:0/0) 2023-08-09T13:50:24.411 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:24.407+0000 7ff63affd640 1 -- 172.21.15.43:0/3115994926 <== mon.0 v1:172.21.15.43:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2018484522 0 0) 0x7ff63c0ce090 con 0x7ff63c0b2b70 2023-08-09T13:50:24.412 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:24.407+0000 7ff63affd640 1 -- 172.21.15.43:0/3115994926 --> v1:172.21.15.43:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ff628003520 con 0x7ff63c0b2b70 2023-08-09T13:50:24.412 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:24.407+0000 7ff63b7fe640 1 --2- 172.21.15.43:0/3115994926 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7ff63c0b2790 0x7ff63c0d2ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:50:24.412 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:24.407+0000 7ff63affd640 1 -- 172.21.15.43:0/3115994926 <== mon.0 v1:172.21.15.43:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1946847344 0 0) 0x7ff628003520 con 0x7ff63c0b2b70 2023-08-09T13:50:24.412 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:24.407+0000 7ff63affd640 1 -- 172.21.15.43:0/3115994926 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7ff63c0b2790 msgr2=0x7ff63c0d2ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-09T13:50:24.412 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:24.407+0000 7ff63affd640 1 --2- 172.21.15.43:0/3115994926 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7ff63c0b2790 0x7ff63c0d2ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:50:24.412 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:24.407+0000 7ff63affd640 1 -- 172.21.15.43:0/3115994926 >> v1:172.21.15.203:6789/0 conn(0x7ff63c0b3050 legacy=0x7ff63c0b3f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:50:24.413 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:24.407+0000 7ff63affd640 1 -- 172.21.15.43:0/3115994926 --> v1:172.21.15.43:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff63c0cdad0 con 0x7ff63c0b2b70 2023-08-09T13:50:24.413 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:24.407+0000 7ff63affd640 1 -- 172.21.15.43:0/3115994926 <== mon.0 v1:172.21.15.43:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1285150740 0 0) 0x7ff6240019f0 con 0x7ff63c0b2b70 2023-08-09T13:50:24.413 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:24.407+0000 7ff63affd640 1 -- 172.21.15.43:0/3115994926 >> v1:172.21.15.43:6789/0 conn(0x7ff63c0b2b70 legacy=0x7ff63c0b5330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:50:24.413 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:24.407+0000 7ff63affd640 1 --2- 172.21.15.43:0/3115994926 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff628003a10 0x7ff628003e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:50:24.413 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:24.407+0000 7ff63affd640 1 --2- 172.21.15.43:0/3115994926 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7ff63c0b2790 0x7ff6280047c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:50:24.413 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:24.407+0000 7ff63affd640 1 --2- 172.21.15.43:0/3115994926 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7ff628004d00 0x7ff628005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:50:24.413 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:24.407+0000 7ff63affd640 1 -- 172.21.15.43:0/3115994926 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff63c0b3950 con 0x7ff628003a10 2023-08-09T13:50:24.414 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:24.407+0000 7ff63affd640 1 -- 172.21.15.43:0/3115994926 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff62800bb10 con 0x7ff628004d00 2023-08-09T13:50:24.414 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:24.407+0000 7ff63affd640 1 -- 172.21.15.43:0/3115994926 --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff62800bca0 con 0x7ff63c0b2790 2023-08-09T13:50:24.414 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:24.408+0000 7ff63b7fe640 1 --2- 172.21.15.43:0/3115994926 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff628003a10 0x7ff628003e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:50:24.414 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:24.408+0000 7ff632ffd640 1 --2- 172.21.15.43:0/3115994926 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7ff63c0b2790 0x7ff6280047c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:50:24.414 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:24.408+0000 7ff63bfff640 1 --2- 172.21.15.43:0/3115994926 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7ff628004d00 0x7ff628005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:50:24.414 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:24.408+0000 7ff63b7fe640 1 -- 172.21.15.43:0/3115994926 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7ff63c0b2790 msgr2=0x7ff6280047c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-09T13:50:24.415 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:24.408+0000 7ff63b7fe640 1 --2- 172.21.15.43:0/3115994926 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7ff63c0b2790 0x7ff6280047c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:50:24.415 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:24.408+0000 7ff63b7fe640 1 -- 172.21.15.43:0/3115994926 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7ff628004d00 msgr2=0x7ff628005180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-09T13:50:24.415 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:24.408+0000 7ff63b7fe640 1 --2- 172.21.15.43:0/3115994926 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7ff628004d00 0x7ff628005180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:50:24.415 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:24.408+0000 7ff63b7fe640 1 -- 172.21.15.43:0/3115994926 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff628003520 con 0x7ff628003a10 2023-08-09T13:50:24.415 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:24.408+0000 7ff632ffd640 1 --2- 172.21.15.43:0/3115994926 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7ff63c0b2790 0x7ff6280047c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-08-09T13:50:24.415 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:24.409+0000 7ff63b7fe640 1 --2- 172.21.15.43:0/3115994926 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff628003a10 0x7ff628003e30 secure :-1 s=READY pgs=985 cs=0 l=1 rev1=1 crypto rx=0x7ff62c009ac0 tx=0x7ff6280057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=85068e464eb5463a server_cookie=0 in_seq=0 out_seq=0 2023-08-09T13:50:24.418 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:24.410+0000 7ff63affd640 1 -- 172.21.15.43:0/3115994926 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff62c02c660 con 0x7ff628003a10 2023-08-09T13:50:24.418 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:24.410+0000 7ff63affd640 1 -- 172.21.15.43:0/3115994926 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff62c02c7c0 con 0x7ff628003a10 2023-08-09T13:50:24.418 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:24.410+0000 7ff63affd640 1 -- 172.21.15.43:0/3115994926 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff62c02cb50 con 0x7ff628003a10 2023-08-09T13:50:24.418 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:24.410+0000 7ff641bb5640 1 -- 172.21.15.43:0/3115994926 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff628003a10 msgr2=0x7ff628003e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:50:24.419 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:24.410+0000 7ff641bb5640 1 --2- 172.21.15.43:0/3115994926 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff628003a10 0x7ff628003e30 secure :-1 s=READY pgs=985 cs=0 l=1 rev1=1 crypto rx=0x7ff62c009ac0 tx=0x7ff6280057f0 comp rx=0 tx=0).stop 2023-08-09T13:50:24.419 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:24.410+0000 7ff63bfff640 1 -- 172.21.15.43:0/3115994926 reap_dead start 2023-08-09T13:50:24.419 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:24.410+0000 7ff641bb5640 1 -- 172.21.15.43:0/3115994926 shutdown_connections 2023-08-09T13:50:24.419 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:24.410+0000 7ff641bb5640 1 -- 172.21.15.43:0/3115994926 >> 172.21.15.43:0/3115994926 conn(0x7ff63c09b7b0 msgr2=0x7ff63c0d24e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-09T13:50:24.419 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:24.410+0000 7ff641bb5640 1 -- 172.21.15.43:0/3115994926 shutdown_connections 2023-08-09T13:50:24.420 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:24.410+0000 7ff641bb5640 1 -- 172.21.15.43:0/3115994926 wait complete. 2023-08-09T13:50:24.420 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:24.412+0000 7ff641bb5640 1 Processor -- start 2023-08-09T13:50:24.420 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:24.412+0000 7ff641bb5640 1 -- start start 2023-08-09T13:50:24.420 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:24.413+0000 7ff641bb5640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7ff628003a10 0x7ff63c0b0d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:50:24.420 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:24.413+0000 7ff641bb5640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff63c0af710 0x7ff63c154450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:50:24.420 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:24.413+0000 7ff641bb5640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7ff63c154be0 0x7ff63c1f5570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:50:24.421 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:24.413+0000 7ff641bb5640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff63c154990 con 0x7ff63c0af710 2023-08-09T13:50:24.421 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:24.413+0000 7ff641bb5640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff63c0afcb0 con 0x7ff63c154be0 2023-08-09T13:50:24.421 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:24.413+0000 7ff641bb5640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff63c0aff90 con 0x7ff628003a10 2023-08-09T13:50:24.421 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:24.413+0000 7ff632ffd640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff63c0af710 0x7ff63c154450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:50:24.421 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:24.414+0000 7ff632ffd640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff63c0af710 0x7ff63c154450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:44250/0 (socket says 172.21.15.43:44250) 2023-08-09T13:50:24.421 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:24.414+0000 7ff632ffd640 1 -- 172.21.15.43:0/944558248 learned_addr learned my addr 172.21.15.43:0/944558248 (peer_addr_for_me v2:172.21.15.43:0/0) 2023-08-09T13:50:24.422 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:24.414+0000 7ff63bfff640 1 --2- 172.21.15.43:0/944558248 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7ff63c154be0 0x7ff63c1f5570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:50:24.422 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:24.414+0000 7ff63b7fe640 1 --2- 172.21.15.43:0/944558248 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7ff628003a10 0x7ff63c0b0d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:50:24.422 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:24.414+0000 7ff632ffd640 1 -- 172.21.15.43:0/944558248 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7ff628003a10 msgr2=0x7ff63c0b0d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:50:24.422 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:24.414+0000 7ff632ffd640 1 --2- 172.21.15.43:0/944558248 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7ff628003a10 0x7ff63c0b0d50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:50:24.422 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:24.414+0000 7ff632ffd640 1 -- 172.21.15.43:0/944558248 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7ff63c154be0 msgr2=0x7ff63c1f5570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:50:24.422 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:24.414+0000 7ff632ffd640 1 --2- 172.21.15.43:0/944558248 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7ff63c154be0 0x7ff63c1f5570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:50:24.422 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:24.414+0000 7ff632ffd640 1 -- 172.21.15.43:0/944558248 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff63c0b0270 con 0x7ff63c0af710 2023-08-09T13:50:24.423 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:24.414+0000 7ff63bfff640 1 --2- 172.21.15.43:0/944558248 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7ff63c154be0 0x7ff63c1f5570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-08-09T13:50:24.423 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:24.414+0000 7ff632ffd640 1 --2- 172.21.15.43:0/944558248 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff63c0af710 0x7ff63c154450 secure :-1 s=READY pgs=986 cs=0 l=1 rev1=1 crypto rx=0x7ff624002120 tx=0x7ff62400b4a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-09T13:50:24.423 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:24.415+0000 7ff638ff9640 1 -- 172.21.15.43:0/944558248 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff624004d60 con 0x7ff63c0af710 2023-08-09T13:50:24.423 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:24.415+0000 7ff638ff9640 1 -- 172.21.15.43:0/944558248 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff6240070d0 con 0x7ff63c0af710 2023-08-09T13:50:24.423 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:24.415+0000 7ff638ff9640 1 -- 172.21.15.43:0/944558248 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff624007440 con 0x7ff63c0af710 2023-08-09T13:50:24.423 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:24.415+0000 7ff641bb5640 1 -- 172.21.15.43:0/944558248 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff63c0b0550 con 0x7ff63c0af710 2023-08-09T13:50:24.423 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:24.415+0000 7ff641bb5640 1 -- 172.21.15.43:0/944558248 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff63c155060 con 0x7ff63c0af710 2023-08-09T13:50:24.424 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:24.416+0000 7ff641bb5640 1 -- 172.21.15.43:0/944558248 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff63c0739f0 con 0x7ff63c0af710 2023-08-09T13:50:24.425 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:24.417+0000 7ff638ff9640 1 -- 172.21.15.43:0/944558248 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 29) v1 ==== 88443+0+0 (secure 0 0 0) 0x7ff6240075a0 con 0x7ff63c0af710 2023-08-09T13:50:24.425 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:24.418+0000 7ff638ff9640 1 --2- 172.21.15.43:0/944558248 >> [v2:172.21.15.203:6832/3325141227,v1:172.21.15.203:6833/3325141227] conn(0x7ff61c06a1a0 0x7ff61c06c660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:50:24.425 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:24.418+0000 7ff638ff9640 1 -- 172.21.15.43:0/944558248 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 5574+0+0 (secure 0 0 0) 0x7ff624088740 con 0x7ff63c0af710 2023-08-09T13:50:24.425 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:24.419+0000 7ff63b7fe640 1 --2- 172.21.15.43:0/944558248 >> [v2:172.21.15.203:6832/3325141227,v1:172.21.15.203:6833/3325141227] conn(0x7ff61c06a1a0 0x7ff61c06c660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:50:24.425 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:24.420+0000 7ff63b7fe640 1 --2- 172.21.15.43:0/944558248 >> [v2:172.21.15.203:6832/3325141227,v1:172.21.15.203:6833/3325141227] conn(0x7ff61c06a1a0 0x7ff61c06c660 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7ff62c036000 tx=0x7ff62c030000 comp rx=0 tx=0).ready entity=mgr.5519 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-09T13:50:24.426 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:24.422+0000 7ff638ff9640 1 -- 172.21.15.43:0/944558248 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178066 (secure 0 0 0) 0x7ff624053d50 con 0x7ff63c0af710 2023-08-09T13:50:24.688 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:24.687+0000 7ff641bb5640 1 -- 172.21.15.43:0/944558248 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "auth ls", "format": "json-pretty"} v 0) v1 -- 0x7ff63c0b4b00 con 0x7ff63c0af710 2023-08-09T13:50:24.690 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:24.688+0000 7ff638ff9640 1 -- 172.21.15.43:0/944558248 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "auth ls", "format": "json-pretty"}]=0 v14) v1 ==== 80+0+5304 (secure 0 0 0) 0x7ff624057370 con 0x7ff63c0af710 2023-08-09T13:50:24.690 INFO:teuthology.orchestra.run.smithi043.stdout: 2023-08-09T13:50:24.690 INFO:teuthology.orchestra.run.smithi043.stdout:{ 2023-08-09T13:50:24.690 INFO:teuthology.orchestra.run.smithi043.stdout: "auth_dump": [ 2023-08-09T13:50:24.690 INFO:teuthology.orchestra.run.smithi043.stdout: { 2023-08-09T13:50:24.691 INFO:teuthology.orchestra.run.smithi043.stdout: "entity": "mds.a", 2023-08-09T13:50:24.691 INFO:teuthology.orchestra.run.smithi043.stdout: "key": "AQCOldNkPu6GCRAAZecwctvMji0Zubb60KMlyQ==", 2023-08-09T13:50:24.691 INFO:teuthology.orchestra.run.smithi043.stdout: "caps": { 2023-08-09T13:50:24.691 INFO:teuthology.orchestra.run.smithi043.stdout: "mds": "allow", 2023-08-09T13:50:24.691 INFO:teuthology.orchestra.run.smithi043.stdout: "mgr": "allow *", 2023-08-09T13:50:24.691 INFO:teuthology.orchestra.run.smithi043.stdout: "mon": "allow *", 2023-08-09T13:50:24.691 INFO:teuthology.orchestra.run.smithi043.stdout: "osd": "allow *" 2023-08-09T13:50:24.692 INFO:teuthology.orchestra.run.smithi043.stdout: } 2023-08-09T13:50:24.692 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2023-08-09T13:50:24.692 INFO:teuthology.orchestra.run.smithi043.stdout: { 2023-08-09T13:50:24.692 INFO:teuthology.orchestra.run.smithi043.stdout: "entity": "mds.b", 2023-08-09T13:50:24.692 INFO:teuthology.orchestra.run.smithi043.stdout: "key": "AQCOldNkt2+5KxAAHcckB+S5NZHsF5z22StXmw==", 2023-08-09T13:50:24.692 INFO:teuthology.orchestra.run.smithi043.stdout: "caps": { 2023-08-09T13:50:24.692 INFO:teuthology.orchestra.run.smithi043.stdout: "mds": "allow", 2023-08-09T13:50:24.693 INFO:teuthology.orchestra.run.smithi043.stdout: "mgr": "allow *", 2023-08-09T13:50:24.693 INFO:teuthology.orchestra.run.smithi043.stdout: "mon": "allow *", 2023-08-09T13:50:24.693 INFO:teuthology.orchestra.run.smithi043.stdout: "osd": "allow *" 2023-08-09T13:50:24.693 INFO:teuthology.orchestra.run.smithi043.stdout: } 2023-08-09T13:50:24.693 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2023-08-09T13:50:24.693 INFO:teuthology.orchestra.run.smithi043.stdout: { 2023-08-09T13:50:24.693 INFO:teuthology.orchestra.run.smithi043.stdout: "entity": "mds.c", 2023-08-09T13:50:24.693 INFO:teuthology.orchestra.run.smithi043.stdout: "key": "AQCOldNk5V46GhAAQBhkvfjVSJCSHLS/YkhgCA==", 2023-08-09T13:50:24.694 INFO:teuthology.orchestra.run.smithi043.stdout: "caps": { 2023-08-09T13:50:24.694 INFO:teuthology.orchestra.run.smithi043.stdout: "mds": "allow", 2023-08-09T13:50:24.694 INFO:teuthology.orchestra.run.smithi043.stdout: "mgr": "allow *", 2023-08-09T13:50:24.694 INFO:teuthology.orchestra.run.smithi043.stdout: "mon": "allow *", 2023-08-09T13:50:24.694 INFO:teuthology.orchestra.run.smithi043.stdout: "osd": "allow *" 2023-08-09T13:50:24.694 INFO:teuthology.orchestra.run.smithi043.stdout: } 2023-08-09T13:50:24.694 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2023-08-09T13:50:24.694 INFO:teuthology.orchestra.run.smithi043.stdout: { 2023-08-09T13:50:24.695 INFO:teuthology.orchestra.run.smithi043.stdout: "entity": "mds.d", 2023-08-09T13:50:24.695 INFO:teuthology.orchestra.run.smithi043.stdout: "key": "AQCPldNkNBoTARAAMVW+rSyXQKaau6I90fUX9Q==", 2023-08-09T13:50:24.695 INFO:teuthology.orchestra.run.smithi043.stdout: "caps": { 2023-08-09T13:50:24.695 INFO:teuthology.orchestra.run.smithi043.stdout: "mds": "allow", 2023-08-09T13:50:24.695 INFO:teuthology.orchestra.run.smithi043.stdout: "mgr": "allow *", 2023-08-09T13:50:24.695 INFO:teuthology.orchestra.run.smithi043.stdout: "mon": "allow *", 2023-08-09T13:50:24.695 INFO:teuthology.orchestra.run.smithi043.stdout: "osd": "allow *" 2023-08-09T13:50:24.695 INFO:teuthology.orchestra.run.smithi043.stdout: } 2023-08-09T13:50:24.696 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2023-08-09T13:50:24.696 INFO:teuthology.orchestra.run.smithi043.stdout: { 2023-08-09T13:50:24.696 INFO:teuthology.orchestra.run.smithi043.stdout: "entity": "osd.0", 2023-08-09T13:50:24.696 INFO:teuthology.orchestra.run.smithi043.stdout: "key": "AQCRldNkltPlMhAAAt3rFvraCtI0aSqGvt6UxA==", 2023-08-09T13:50:24.696 INFO:teuthology.orchestra.run.smithi043.stdout: "caps": { 2023-08-09T13:50:24.696 INFO:teuthology.orchestra.run.smithi043.stdout: "mgr": "allow profile osd", 2023-08-09T13:50:24.696 INFO:teuthology.orchestra.run.smithi043.stdout: "mon": "allow profile osd", 2023-08-09T13:50:24.697 INFO:teuthology.orchestra.run.smithi043.stdout: "osd": "allow *" 2023-08-09T13:50:24.697 INFO:teuthology.orchestra.run.smithi043.stdout: } 2023-08-09T13:50:24.697 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2023-08-09T13:50:24.697 INFO:teuthology.orchestra.run.smithi043.stdout: { 2023-08-09T13:50:24.697 INFO:teuthology.orchestra.run.smithi043.stdout: "entity": "osd.1", 2023-08-09T13:50:24.697 INFO:teuthology.orchestra.run.smithi043.stdout: "key": "AQCVldNkzpKYEhAA+wen0kV/vUDLQztq3vhfOg==", 2023-08-09T13:50:24.697 INFO:teuthology.orchestra.run.smithi043.stdout: "caps": { 2023-08-09T13:50:24.697 INFO:teuthology.orchestra.run.smithi043.stdout: "mgr": "allow profile osd", 2023-08-09T13:50:24.698 INFO:teuthology.orchestra.run.smithi043.stdout: "mon": "allow profile osd", 2023-08-09T13:50:24.698 INFO:teuthology.orchestra.run.smithi043.stdout: "osd": "allow *" 2023-08-09T13:50:24.698 INFO:teuthology.orchestra.run.smithi043.stdout: } 2023-08-09T13:50:24.698 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2023-08-09T13:50:24.698 INFO:teuthology.orchestra.run.smithi043.stdout: { 2023-08-09T13:50:24.698 INFO:teuthology.orchestra.run.smithi043.stdout: "entity": "osd.2", 2023-08-09T13:50:24.698 INFO:teuthology.orchestra.run.smithi043.stdout: "key": "AQCYldNkYbwNHhAAQY8g9dldb1V9M2MM2SHKDg==", 2023-08-09T13:50:24.698 INFO:teuthology.orchestra.run.smithi043.stdout: "caps": { 2023-08-09T13:50:24.699 INFO:teuthology.orchestra.run.smithi043.stdout: "mgr": "allow profile osd", 2023-08-09T13:50:24.699 INFO:teuthology.orchestra.run.smithi043.stdout: "mon": "allow profile osd", 2023-08-09T13:50:24.699 INFO:teuthology.orchestra.run.smithi043.stdout: "osd": "allow *" 2023-08-09T13:50:24.699 INFO:teuthology.orchestra.run.smithi043.stdout: } 2023-08-09T13:50:24.699 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2023-08-09T13:50:24.699 INFO:teuthology.orchestra.run.smithi043.stdout: { 2023-08-09T13:50:24.699 INFO:teuthology.orchestra.run.smithi043.stdout: "entity": "osd.3", 2023-08-09T13:50:24.700 INFO:teuthology.orchestra.run.smithi043.stdout: "key": "AQCbldNkeyW6KRAAsd+kgqik2aD4U7PXwccWtA==", 2023-08-09T13:50:24.700 INFO:teuthology.orchestra.run.smithi043.stdout: "caps": { 2023-08-09T13:50:24.700 INFO:teuthology.orchestra.run.smithi043.stdout: "mgr": "allow profile osd", 2023-08-09T13:50:24.700 INFO:teuthology.orchestra.run.smithi043.stdout: "mon": "allow profile osd", 2023-08-09T13:50:24.700 INFO:teuthology.orchestra.run.smithi043.stdout: "osd": "allow *" 2023-08-09T13:50:24.700 INFO:teuthology.orchestra.run.smithi043.stdout: } 2023-08-09T13:50:24.700 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2023-08-09T13:50:24.700 INFO:teuthology.orchestra.run.smithi043.stdout: { 2023-08-09T13:50:24.701 INFO:teuthology.orchestra.run.smithi043.stdout: "entity": "osd.4", 2023-08-09T13:50:24.701 INFO:teuthology.orchestra.run.smithi043.stdout: "key": "AQCildNk80UkOBAAtX+4JGpOVcfZEHetJoeZKw==", 2023-08-09T13:50:24.701 INFO:teuthology.orchestra.run.smithi043.stdout: "caps": { 2023-08-09T13:50:24.701 INFO:teuthology.orchestra.run.smithi043.stdout: "mgr": "allow profile osd", 2023-08-09T13:50:24.701 INFO:teuthology.orchestra.run.smithi043.stdout: "mon": "allow profile osd", 2023-08-09T13:50:24.701 INFO:teuthology.orchestra.run.smithi043.stdout: "osd": "allow *" 2023-08-09T13:50:24.701 INFO:teuthology.orchestra.run.smithi043.stdout: } 2023-08-09T13:50:24.701 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2023-08-09T13:50:24.702 INFO:teuthology.orchestra.run.smithi043.stdout: { 2023-08-09T13:50:24.702 INFO:teuthology.orchestra.run.smithi043.stdout: "entity": "osd.5", 2023-08-09T13:50:24.702 INFO:teuthology.orchestra.run.smithi043.stdout: "key": "AQCmldNkCA9SBxAAZj4So6MRj2Eq1I9H9JzLDA==", 2023-08-09T13:50:24.702 INFO:teuthology.orchestra.run.smithi043.stdout: "caps": { 2023-08-09T13:50:24.702 INFO:teuthology.orchestra.run.smithi043.stdout: "mgr": "allow profile osd", 2023-08-09T13:50:24.702 INFO:teuthology.orchestra.run.smithi043.stdout: "mon": "allow profile osd", 2023-08-09T13:50:24.702 INFO:teuthology.orchestra.run.smithi043.stdout: "osd": "allow *" 2023-08-09T13:50:24.702 INFO:teuthology.orchestra.run.smithi043.stdout: } 2023-08-09T13:50:24.703 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2023-08-09T13:50:24.703 INFO:teuthology.orchestra.run.smithi043.stdout: { 2023-08-09T13:50:24.703 INFO:teuthology.orchestra.run.smithi043.stdout: "entity": "osd.6", 2023-08-09T13:50:24.703 INFO:teuthology.orchestra.run.smithi043.stdout: "key": "AQCpldNkaOwAExAAraV1tUwr9uU6GheZ+hvdqQ==", 2023-08-09T13:50:24.703 INFO:teuthology.orchestra.run.smithi043.stdout: "caps": { 2023-08-09T13:50:24.703 INFO:teuthology.orchestra.run.smithi043.stdout: "mgr": "allow profile osd", 2023-08-09T13:50:24.703 INFO:teuthology.orchestra.run.smithi043.stdout: "mon": "allow profile osd", 2023-08-09T13:50:24.703 INFO:teuthology.orchestra.run.smithi043.stdout: "osd": "allow *" 2023-08-09T13:50:24.704 INFO:teuthology.orchestra.run.smithi043.stdout: } 2023-08-09T13:50:24.704 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2023-08-09T13:50:24.704 INFO:teuthology.orchestra.run.smithi043.stdout: { 2023-08-09T13:50:24.704 INFO:teuthology.orchestra.run.smithi043.stdout: "entity": "osd.7", 2023-08-09T13:50:24.704 INFO:teuthology.orchestra.run.smithi043.stdout: "key": "AQCsldNk2/GwLBAAP42EXPivzLk44X8aMEVVCg==", 2023-08-09T13:50:24.704 INFO:teuthology.orchestra.run.smithi043.stdout: "caps": { 2023-08-09T13:50:24.704 INFO:teuthology.orchestra.run.smithi043.stdout: "mgr": "allow profile osd", 2023-08-09T13:50:24.705 INFO:teuthology.orchestra.run.smithi043.stdout: "mon": "allow profile osd", 2023-08-09T13:50:24.705 INFO:teuthology.orchestra.run.smithi043.stdout: "osd": "allow *" 2023-08-09T13:50:24.705 INFO:teuthology.orchestra.run.smithi043.stdout: } 2023-08-09T13:50:24.705 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2023-08-09T13:50:24.705 INFO:teuthology.orchestra.run.smithi043.stdout: { 2023-08-09T13:50:24.705 INFO:teuthology.orchestra.run.smithi043.stdout: "entity": "client.0", 2023-08-09T13:50:24.705 INFO:teuthology.orchestra.run.smithi043.stdout: "key": "AQCPldNkg8zrDxAAM/hnJaE6S9HU/fRwQeBV5w==", 2023-08-09T13:50:24.705 INFO:teuthology.orchestra.run.smithi043.stdout: "caps": { 2023-08-09T13:50:24.706 INFO:teuthology.orchestra.run.smithi043.stdout: "mds": "allow", 2023-08-09T13:50:24.706 INFO:teuthology.orchestra.run.smithi043.stdout: "mon": "allow r", 2023-08-09T13:50:24.706 INFO:teuthology.orchestra.run.smithi043.stdout: "osd": "allow rw tag cephfs data=cephfs" 2023-08-09T13:50:24.706 INFO:teuthology.orchestra.run.smithi043.stdout: } 2023-08-09T13:50:24.706 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2023-08-09T13:50:24.706 INFO:teuthology.orchestra.run.smithi043.stdout: { 2023-08-09T13:50:24.706 INFO:teuthology.orchestra.run.smithi043.stdout: "entity": "client.1", 2023-08-09T13:50:24.706 INFO:teuthology.orchestra.run.smithi043.stdout: "key": "AQCPldNklfFxJxAA19+L78rW6jU1uOyjey6WNA==", 2023-08-09T13:50:24.707 INFO:teuthology.orchestra.run.smithi043.stdout: "caps": { 2023-08-09T13:50:24.707 INFO:teuthology.orchestra.run.smithi043.stdout: "mds": "allow", 2023-08-09T13:50:24.707 INFO:teuthology.orchestra.run.smithi043.stdout: "mgr": "allow r", 2023-08-09T13:50:24.707 INFO:teuthology.orchestra.run.smithi043.stdout: "mon": "allow rw", 2023-08-09T13:50:24.707 INFO:teuthology.orchestra.run.smithi043.stdout: "osd": "allow rwx" 2023-08-09T13:50:24.707 INFO:teuthology.orchestra.run.smithi043.stdout: } 2023-08-09T13:50:24.707 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2023-08-09T13:50:24.707 INFO:teuthology.orchestra.run.smithi043.stdout: { 2023-08-09T13:50:24.708 INFO:teuthology.orchestra.run.smithi043.stdout: "entity": "client.2", 2023-08-09T13:50:24.708 INFO:teuthology.orchestra.run.smithi043.stdout: "key": "AQCPldNkC2lgGxAAZ8qZboOCUaQlcpkPL1ozHA==", 2023-08-09T13:50:24.708 INFO:teuthology.orchestra.run.smithi043.stdout: "caps": { 2023-08-09T13:50:24.708 INFO:teuthology.orchestra.run.smithi043.stdout: "mds": "allow", 2023-08-09T13:50:24.708 INFO:teuthology.orchestra.run.smithi043.stdout: "mgr": "allow r", 2023-08-09T13:50:24.708 INFO:teuthology.orchestra.run.smithi043.stdout: "mon": "allow rw", 2023-08-09T13:50:24.708 INFO:teuthology.orchestra.run.smithi043.stdout: "osd": "allow rwx" 2023-08-09T13:50:24.708 INFO:teuthology.orchestra.run.smithi043.stdout: } 2023-08-09T13:50:24.709 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2023-08-09T13:50:24.709 INFO:teuthology.orchestra.run.smithi043.stdout: { 2023-08-09T13:50:24.709 INFO:teuthology.orchestra.run.smithi043.stdout: "entity": "client.3", 2023-08-09T13:50:24.709 INFO:teuthology.orchestra.run.smithi043.stdout: "key": "AQCPldNkl+rXMhAA9rf3KdnSYbvngqmLYnIGGA==", 2023-08-09T13:50:24.709 INFO:teuthology.orchestra.run.smithi043.stdout: "caps": { 2023-08-09T13:50:24.709 INFO:teuthology.orchestra.run.smithi043.stdout: "mds": "allow", 2023-08-09T13:50:24.709 INFO:teuthology.orchestra.run.smithi043.stdout: "mgr": "allow r", 2023-08-09T13:50:24.710 INFO:teuthology.orchestra.run.smithi043.stdout: "mon": "allow rw", 2023-08-09T13:50:24.710 INFO:teuthology.orchestra.run.smithi043.stdout: "osd": "allow rwx" 2023-08-09T13:50:24.710 INFO:teuthology.orchestra.run.smithi043.stdout: } 2023-08-09T13:50:24.710 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2023-08-09T13:50:24.710 INFO:teuthology.orchestra.run.smithi043.stdout: { 2023-08-09T13:50:24.710 INFO:teuthology.orchestra.run.smithi043.stdout: "entity": "client.admin", 2023-08-09T13:50:24.710 INFO:teuthology.orchestra.run.smithi043.stdout: "key": "AQCMldNkUae6NRAA2FBTr0QPBdKNDoe1WK5zyg==", 2023-08-09T13:50:24.710 INFO:teuthology.orchestra.run.smithi043.stdout: "caps": { 2023-08-09T13:50:24.711 INFO:teuthology.orchestra.run.smithi043.stdout: "mds": "allow *", 2023-08-09T13:50:24.711 INFO:teuthology.orchestra.run.smithi043.stdout: "mgr": "allow *", 2023-08-09T13:50:24.711 INFO:teuthology.orchestra.run.smithi043.stdout: "mon": "allow *", 2023-08-09T13:50:24.711 INFO:teuthology.orchestra.run.smithi043.stdout: "osd": "allow *" 2023-08-09T13:50:24.711 INFO:teuthology.orchestra.run.smithi043.stdout: } 2023-08-09T13:50:24.711 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2023-08-09T13:50:24.711 INFO:teuthology.orchestra.run.smithi043.stdout: { 2023-08-09T13:50:24.711 INFO:teuthology.orchestra.run.smithi043.stdout: "entity": "mgr.x", 2023-08-09T13:50:24.712 INFO:teuthology.orchestra.run.smithi043.stdout: "key": "AQCNldNk2ufSNxAAU7msRkwLaoLZHTIsk0ck/g==", 2023-08-09T13:50:24.712 INFO:teuthology.orchestra.run.smithi043.stdout: "caps": { 2023-08-09T13:50:24.712 INFO:teuthology.orchestra.run.smithi043.stdout: "mds": "allow *", 2023-08-09T13:50:24.712 INFO:teuthology.orchestra.run.smithi043.stdout: "mon": "allow profile mgr", 2023-08-09T13:50:24.712 INFO:teuthology.orchestra.run.smithi043.stdout: "osd": "allow *" 2023-08-09T13:50:24.712 INFO:teuthology.orchestra.run.smithi043.stdout: } 2023-08-09T13:50:24.712 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2023-08-09T13:50:24.712 INFO:teuthology.orchestra.run.smithi043.stdout: { 2023-08-09T13:50:24.713 INFO:teuthology.orchestra.run.smithi043.stdout: "entity": "mgr.y", 2023-08-09T13:50:24.713 INFO:teuthology.orchestra.run.smithi043.stdout: "key": "AQCNldNk2YgzLBAAA11Q8uAMEgE/bUni+tXIKA==", 2023-08-09T13:50:24.713 INFO:teuthology.orchestra.run.smithi043.stdout: "caps": { 2023-08-09T13:50:24.713 INFO:teuthology.orchestra.run.smithi043.stdout: "mds": "allow *", 2023-08-09T13:50:24.713 INFO:teuthology.orchestra.run.smithi043.stdout: "mon": "allow profile mgr", 2023-08-09T13:50:24.713 INFO:teuthology.orchestra.run.smithi043.stdout: "osd": "allow *" 2023-08-09T13:50:24.713 INFO:teuthology.orchestra.run.smithi043.stdout: } 2023-08-09T13:50:24.713 INFO:teuthology.orchestra.run.smithi043.stdout: } 2023-08-09T13:50:24.714 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2023-08-09T13:50:24.714 INFO:teuthology.orchestra.run.smithi043.stdout:} 2023-08-09T13:50:24.715 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:24.691+0000 7ff641bb5640 1 -- 172.21.15.43:0/944558248 >> [v2:172.21.15.203:6832/3325141227,v1:172.21.15.203:6833/3325141227] conn(0x7ff61c06a1a0 msgr2=0x7ff61c06c660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:50:24.715 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:24.691+0000 7ff641bb5640 1 --2- 172.21.15.43:0/944558248 >> [v2:172.21.15.203:6832/3325141227,v1:172.21.15.203:6833/3325141227] conn(0x7ff61c06a1a0 0x7ff61c06c660 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7ff62c036000 tx=0x7ff62c030000 comp rx=0 tx=0).stop 2023-08-09T13:50:24.715 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:24.691+0000 7ff641bb5640 1 -- 172.21.15.43:0/944558248 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff63c0af710 msgr2=0x7ff63c154450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:50:24.715 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:24.691+0000 7ff641bb5640 1 --2- 172.21.15.43:0/944558248 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff63c0af710 0x7ff63c154450 secure :-1 s=READY pgs=986 cs=0 l=1 rev1=1 crypto rx=0x7ff624002120 tx=0x7ff62400b4a0 comp rx=0 tx=0).stop 2023-08-09T13:50:24.716 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:24.691+0000 7ff641bb5640 1 -- 172.21.15.43:0/944558248 shutdown_connections 2023-08-09T13:50:24.716 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:24.691+0000 7ff641bb5640 1 --2- 172.21.15.43:0/944558248 >> [v2:172.21.15.203:6832/3325141227,v1:172.21.15.203:6833/3325141227] conn(0x7ff61c06a1a0 0x7ff61c06c660 unknown :-1 s=CLOSED pgs=51 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:50:24.716 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:24.691+0000 7ff641bb5640 1 --2- 172.21.15.43:0/944558248 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7ff63c154be0 0x7ff63c1f5570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:50:24.716 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:24.691+0000 7ff641bb5640 1 --2- 172.21.15.43:0/944558248 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff63c0af710 0x7ff63c154450 unknown :-1 s=CLOSED pgs=986 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:50:24.716 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:24.691+0000 7ff641bb5640 1 --2- 172.21.15.43:0/944558248 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7ff628003a10 0x7ff63c0b0d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:50:24.716 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:24.691+0000 7ff641bb5640 1 -- 172.21.15.43:0/944558248 >> 172.21.15.43:0/944558248 conn(0x7ff63c09b7b0 msgr2=0x7ff63c1515d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-09T13:50:24.716 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:24.692+0000 7ff641bb5640 1 -- 172.21.15.43:0/944558248 shutdown_connections 2023-08-09T13:50:24.717 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:24.692+0000 7ff641bb5640 1 -- 172.21.15.43:0/944558248 wait complete. 2023-08-09T13:50:24.717 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-08-09T13:50:24.938 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:24.936+0000 7f9a79c21640 1 Processor -- start 2023-08-09T13:50:24.938 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:24.936+0000 7f9a79c21640 1 -- start start 2023-08-09T13:50:24.939 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:24.938+0000 7f9a79c21640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f9a740c8750 0x7f9a740ad340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:50:24.940 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:24.938+0000 7f9a79c21640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f9a740c8b30 0x7f9a7406f410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:50:24.940 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:24.938+0000 7f9a79c21640 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7f9a740acdf0 0x7f9a74070f00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:50:24.940 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:24.938+0000 7f9a79c21640 1 -- --> v2:172.21.15.43:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9a7415e810 con 0x7f9a740c8b30 2023-08-09T13:50:24.940 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:24.938+0000 7f9a79c21640 1 -- --> v2:172.21.15.203:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9a740c9f70 con 0x7f9a740acdf0 2023-08-09T13:50:24.940 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:24.938+0000 7f9a79c21640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9a740cbd90 con 0x7f9a740c8750 2023-08-09T13:50:24.941 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:24.939+0000 7f9a73fff640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f9a740c8b30 0x7f9a7406f410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:50:24.941 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:24.939+0000 7f9a73fff640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f9a740c8b30 0x7f9a7406f410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:44254/0 (socket says 172.21.15.43:44254) 2023-08-09T13:50:24.941 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:24.939+0000 7f9a73fff640 1 -- 172.21.15.43:0/1323748101 learned_addr learned my addr 172.21.15.43:0/1323748101 (peer_addr_for_me v2:172.21.15.43:0/0) 2023-08-09T13:50:24.942 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:24.939+0000 7f9a78c1f640 1 --2- 172.21.15.43:0/1323748101 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f9a740c8750 0x7f9a740ad340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:50:24.942 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:24.939+0000 7f9a79420640 1 --2- 172.21.15.43:0/1323748101 >> v2:172.21.15.203:3300/0 conn(0x7f9a740acdf0 0x7f9a74070f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:50:24.942 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:24.939+0000 7f9a73fff640 1 -- 172.21.15.43:0/1323748101 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f9a740c8750 msgr2=0x7f9a740ad340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-09T13:50:24.942 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:24.939+0000 7f9a73fff640 1 --2- 172.21.15.43:0/1323748101 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f9a740c8750 0x7f9a740ad340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:50:24.942 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:24.939+0000 7f9a73fff640 1 -- 172.21.15.43:0/1323748101 >> v2:172.21.15.203:3300/0 conn(0x7f9a740acdf0 msgr2=0x7f9a74070f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-09T13:50:24.942 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:24.939+0000 7f9a73fff640 1 --2- 172.21.15.43:0/1323748101 >> v2:172.21.15.203:3300/0 conn(0x7f9a740acdf0 0x7f9a74070f00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:50:24.942 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:24.939+0000 7f9a73fff640 1 -- 172.21.15.43:0/1323748101 --> v2:172.21.15.43:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9a740ca2b0 con 0x7f9a740c8b30 2023-08-09T13:50:24.943 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:24.941+0000 7f9a73fff640 1 --2- 172.21.15.43:0/1323748101 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f9a740c8b30 0x7f9a7406f410 secure :-1 s=READY pgs=987 cs=0 l=1 rev1=1 crypto rx=0x7f9a7415b9a0 tx=0x7f9a6002b8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c20b8b0eff8236de server_cookie=0 in_seq=0 out_seq=0 2023-08-09T13:50:24.943 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:24.941+0000 7f9a737fe640 1 -- 172.21.15.43:0/1323748101 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9a60031640 con 0x7f9a740c8b30 2023-08-09T13:50:24.943 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:24.941+0000 7f9a737fe640 1 -- 172.21.15.43:0/1323748101 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9a600317a0 con 0x7f9a740c8b30 2023-08-09T13:50:24.944 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:24.941+0000 7f9a737fe640 1 -- 172.21.15.43:0/1323748101 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9a60031b30 con 0x7f9a740c8b30 2023-08-09T13:50:24.944 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:24.942+0000 7f9a79c21640 1 -- 172.21.15.43:0/1323748101 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f9a740c8b30 msgr2=0x7f9a7406f410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:50:24.944 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:24.942+0000 7f9a79c21640 1 --2- 172.21.15.43:0/1323748101 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f9a740c8b30 0x7f9a7406f410 secure :-1 s=READY pgs=987 cs=0 l=1 rev1=1 crypto rx=0x7f9a7415b9a0 tx=0x7f9a6002b8f0 comp rx=0 tx=0).stop 2023-08-09T13:50:24.944 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:24.942+0000 7f9a79c21640 1 -- 172.21.15.43:0/1323748101 shutdown_connections 2023-08-09T13:50:24.944 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:24.942+0000 7f9a79c21640 1 --2- 172.21.15.43:0/1323748101 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f9a740c8b30 0x7f9a7406f410 unknown :-1 s=CLOSED pgs=987 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:50:24.944 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:24.942+0000 7f9a79c21640 1 --2- 172.21.15.43:0/1323748101 >> v2:172.21.15.203:3300/0 conn(0x7f9a740acdf0 0x7f9a74070f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:50:24.944 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:24.942+0000 7f9a79c21640 1 --2- 172.21.15.43:0/1323748101 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f9a740c8750 0x7f9a740ad340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:50:24.945 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:24.942+0000 7f9a79c21640 1 -- 172.21.15.43:0/1323748101 >> 172.21.15.43:0/1323748101 conn(0x7f9a740cde90 msgr2=0x7f9a740ce2a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-09T13:50:24.945 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:24.942+0000 7f9a79c21640 1 -- 172.21.15.43:0/1323748101 shutdown_connections 2023-08-09T13:50:24.945 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:24.942+0000 7f9a79c21640 1 -- 172.21.15.43:0/1323748101 wait complete. 2023-08-09T13:50:24.945 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:24.944+0000 7f9a79c21640 1 Processor -- start 2023-08-09T13:50:24.945 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:24.944+0000 7f9a79c21640 1 -- start start 2023-08-09T13:50:24.948 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:24.945+0000 7f9a79c21640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f9a740acdf0 0x7f9a74157df0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:50:24.949 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:24.945+0000 7f9a79c21640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f9a740c8750 0x7f9a740c5870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:50:24.949 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:24.945+0000 7f9a79c21640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f9a740c8b30 0x7f9a740c4370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:50:24.949 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:24.945+0000 7f9a79c21640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9a740c8260 con 0x7f9a740acdf0 2023-08-09T13:50:24.949 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:24.945+0000 7f9a79c21640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9a740c7930 con 0x7f9a740c8b30 2023-08-09T13:50:24.950 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:24.945+0000 7f9a79c21640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9a740c7c10 con 0x7f9a740c8750 2023-08-09T13:50:24.950 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:24.945+0000 7f9a78c1f640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f9a740acdf0 0x7f9a74157df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:50:24.950 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:24.945+0000 7f9a78c1f640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f9a740acdf0 0x7f9a74157df0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:44260/0 (socket says 172.21.15.43:44260) 2023-08-09T13:50:24.950 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:24.945+0000 7f9a78c1f640 1 -- 172.21.15.43:0/3351621737 learned_addr learned my addr 172.21.15.43:0/3351621737 (peer_addr_for_me v2:172.21.15.43:0/0) 2023-08-09T13:50:24.951 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:24.946+0000 7f9a78c1f640 1 -- 172.21.15.43:0/3351621737 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f9a740c8750 msgr2=0x7f9a740c5870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:50:24.951 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:24.946+0000 7f9a78c1f640 1 --2- 172.21.15.43:0/3351621737 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f9a740c8750 0x7f9a740c5870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:50:24.951 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:24.946+0000 7f9a78c1f640 1 -- 172.21.15.43:0/3351621737 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f9a740c8b30 msgr2=0x7f9a740c4370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:50:24.951 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:24.946+0000 7f9a78c1f640 1 --2- 172.21.15.43:0/3351621737 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f9a740c8b30 0x7f9a740c4370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:50:24.951 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:24.946+0000 7f9a78c1f640 1 -- 172.21.15.43:0/3351621737 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9a600094d0 con 0x7f9a740acdf0 2023-08-09T13:50:24.952 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:24.946+0000 7f9a78c1f640 1 --2- 172.21.15.43:0/3351621737 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f9a740acdf0 0x7f9a74157df0 secure :-1 s=READY pgs=988 cs=0 l=1 rev1=1 crypto rx=0x7f9a6800d800 tx=0x7f9a6800dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-09T13:50:24.952 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:24.946+0000 7f9a717fa640 1 -- 172.21.15.43:0/3351621737 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9a6800b6c0 con 0x7f9a740acdf0 2023-08-09T13:50:24.952 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:24.946+0000 7f9a717fa640 1 -- 172.21.15.43:0/3351621737 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9a6800b820 con 0x7f9a740acdf0 2023-08-09T13:50:24.952 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:24.946+0000 7f9a717fa640 1 -- 172.21.15.43:0/3351621737 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9a6800bbb0 con 0x7f9a740acdf0 2023-08-09T13:50:24.953 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:24.946+0000 7f9a79c21640 1 -- 172.21.15.43:0/3351621737 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9a740c7ef0 con 0x7f9a740acdf0 2023-08-09T13:50:24.953 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:24.946+0000 7f9a79c21640 1 -- 172.21.15.43:0/3351621737 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9a740c7570 con 0x7f9a740acdf0 2023-08-09T13:50:24.954 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:24.948+0000 7f9a79c21640 1 -- 172.21.15.43:0/3351621737 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9a740739f0 con 0x7f9a740acdf0 2023-08-09T13:50:24.954 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:24.949+0000 7f9a717fa640 1 -- 172.21.15.43:0/3351621737 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 29) v1 ==== 88443+0+0 (secure 0 0 0) 0x7f9a6800bd10 con 0x7f9a740acdf0 2023-08-09T13:50:24.954 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:24.949+0000 7f9a717fa640 1 --2- 172.21.15.43:0/3351621737 >> [v2:172.21.15.203:6832/3325141227,v1:172.21.15.203:6833/3325141227] conn(0x7f9a5806a170 0x7f9a5806c630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:50:24.955 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:24.950+0000 7f9a717fa640 1 -- 172.21.15.43:0/3351621737 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 5574+0+0 (secure 0 0 0) 0x7f9a68088ad0 con 0x7f9a740acdf0 2023-08-09T13:50:24.955 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:24.950+0000 7f9a73fff640 1 --2- 172.21.15.43:0/3351621737 >> [v2:172.21.15.203:6832/3325141227,v1:172.21.15.203:6833/3325141227] conn(0x7f9a5806a170 0x7f9a5806c630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:50:24.955 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:24.952+0000 7f9a73fff640 1 --2- 172.21.15.43:0/3351621737 >> [v2:172.21.15.203:6832/3325141227,v1:172.21.15.203:6833/3325141227] conn(0x7f9a5806a170 0x7f9a5806c630 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7f9a60036000 tx=0x7f9a60030000 comp rx=0 tx=0).ready entity=mgr.5519 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-09T13:50:24.956 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:24.954+0000 7f9a717fa640 1 -- 172.21.15.43:0/3351621737 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178066 (secure 0 0 0) 0x7f9a680541f0 con 0x7f9a740acdf0 2023-08-09T13:50:25.151 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:25.150+0000 7f9a79c21640 1 -- 172.21.15.43:0/3351621737 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f9a740c9c80 con 0x7f9a740acdf0 2023-08-09T13:50:25.152 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:25.151+0000 7f9a717fa640 1 -- 172.21.15.43:0/3351621737 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v86) v1 ==== 74+0+14844 (secure 0 0 0) 0x7f9a68057810 con 0x7f9a740acdf0 2023-08-09T13:50:25.153 INFO:teuthology.orchestra.run.smithi043.stdout: 2023-08-09T13:50:25.154 INFO:teuthology.orchestra.run.smithi043.stdout:{"epoch":86,"fsid":"daac87ee-1337-4250-a955-aea4e408a39b","created":"2023-08-09T13:33:41.701198+0000","modified":"2023-08-09T13:50:24.131118+0000","last_up_change":"2023-08-09T13:33:53.783453+0000","last_in_change":"2023-08-09T13:33:45.148904+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":9,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":"rbd","create_time":"2023-08-09T13:33:54.365137+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":".mgr","create_time":"2023-08-09T13:33:54.831803+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"cbd49ecd-e896-4fbd-864e-5c37a829b165","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":72,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6800","nonce":502667335},{"type":"v1","addr":"172.21.15.43:6801","nonce":502667335}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6802","nonce":502667335},{"type":"v1","addr":"172.21.15.43:6803","nonce":502667335}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6806","nonce":502667335},{"type":"v1","addr":"172.21.15.43:6807","nonce":502667335}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6804","nonce":502667335},{"type":"v1","addr":"172.21.15.43:6805","nonce":502667335}]},"public_addr":"172.21.15.43:6801/502667335","cluster_addr":"172.21.15.43:6803/502667335","heartbeat_back_addr":"172.21.15.43:6807/502667335","heartbeat_front_addr":"172.21.15.43:6805/502667335","state":["exists","up"]},{"osd":1,"uuid":"75c7e0fe-1ec7-4fe8-a2f2-3661a65b8500","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":72,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6808","nonce":3289134610},{"type":"v1","addr":"172.21.15.43:6809","nonce":3289134610}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6810","nonce":3289134610},{"type":"v1","addr":"172.21.15.43:6812","nonce":3289134610}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6821","nonce":3289134610},{"type":"v1","addr":"172.21.15.43:6824","nonce":3289134610}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6815","nonce":3289134610},{"type":"v1","addr":"172.21.15.43:6818","nonce":3289134610}]},"public_addr":"172.21.15.43:6809/3289134610","cluster_addr":"172.21.15.43:6812/3289134610","heartbeat_back_addr":"172.21.15.43:6824/3289134610","heartbeat_front_addr":"172.21.15.43:6818/3289134610","state":["exists","up"]},{"osd":2,"uuid":"bb5c41e9-7f5f-4125-9b01-3bb8bfc2ea83","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":72,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6811","nonce":4079709736},{"type":"v1","addr":"172.21.15.43:6814","nonce":4079709736}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6817","nonce":4079709736},{"type":"v1","addr":"172.21.15.43:6820","nonce":4079709736}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6829","nonce":4079709736},{"type":"v1","addr":"172.21.15.43:6831","nonce":4079709736}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6823","nonce":4079709736},{"type":"v1","addr":"172.21.15.43:6826","nonce":4079709736}]},"public_addr":"172.21.15.43:6814/4079709736","cluster_addr":"172.21.15.43:6820/4079709736","heartbeat_back_addr":"172.21.15.43:6831/4079709736","heartbeat_front_addr":"172.21.15.43:6826/4079709736","state":["exists","up"]},{"osd":3,"uuid":"bfe4c8b4-694a-465c-87e6-2b2aab852613","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":72,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6813","nonce":1709538759},{"type":"v1","addr":"172.21.15.43:6816","nonce":1709538759}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6819","nonce":1709538759},{"type":"v1","addr":"172.21.15.43:6822","nonce":1709538759}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6828","nonce":1709538759},{"type":"v1","addr":"172.21.15.43:6830","nonce":1709538759}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6825","nonce":1709538759},{"type":"v1","addr":"172.21.15.43:6827","nonce":1709538759}]},"public_addr":"172.21.15.43:6816/1709538759","cluster_addr":"172.21.15.43:6822/1709538759","heartbeat_back_addr":"172.21.15.43:6830/1709538759","heartbeat_front_addr":"172.21.15.43:6827/1709538759","state":["exists","up"]},{"osd":4,"uuid":"fa2e8a77-aba1-4756-853e-0d174a5bdeba","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":72,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6800","nonce":4196866481},{"type":"v1","addr":"172.21.15.203:6801","nonce":4196866481}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6802","nonce":4196866481},{"type":"v1","addr":"172.21.15.203:6803","nonce":4196866481}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6806","nonce":4196866481},{"type":"v1","addr":"172.21.15.203:6807","nonce":4196866481}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6804","nonce":4196866481},{"type":"v1","addr":"172.21.15.203:6805","nonce":4196866481}]},"public_addr":"172.21.15.203:6801/4196866481","cluster_addr":"172.21.15.203:6803/4196866481","heartbeat_back_addr":"172.21.15.203:6807/4196866481","heartbeat_front_addr":"172.21.15.203:6805/4196866481","state":["exists","up"]},{"osd":5,"uuid":"81811e4b-4d24-4f51-a82c-f1d700a2a00d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":72,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6816","nonce":3059290081},{"type":"v1","addr":"172.21.15.203:6817","nonce":3059290081}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6818","nonce":3059290081},{"type":"v1","addr":"172.21.15.203:6819","nonce":3059290081}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6822","nonce":3059290081},{"type":"v1","addr":"172.21.15.203:6823","nonce":3059290081}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6820","nonce":3059290081},{"type":"v1","addr":"172.21.15.203:6821","nonce":3059290081}]},"public_addr":"172.21.15.203:6817/3059290081","cluster_addr":"172.21.15.203:6819/3059290081","heartbeat_back_addr":"172.21.15.203:6823/3059290081","heartbeat_front_addr":"172.21.15.203:6821/3059290081","state":["exists","up"]},{"osd":6,"uuid":"307bac4d-6a5b-443d-af75-56d695d7674f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":72,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6824","nonce":512155569},{"type":"v1","addr":"172.21.15.203:6825","nonce":512155569}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6826","nonce":512155569},{"type":"v1","addr":"172.21.15.203:6827","nonce":512155569}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6830","nonce":512155569},{"type":"v1","addr":"172.21.15.203:6831","nonce":512155569}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6828","nonce":512155569},{"type":"v1","addr":"172.21.15.203:6829","nonce":512155569}]},"public_addr":"172.21.15.203:6825/512155569","cluster_addr":"172.21.15.203:6827/512155569","heartbeat_back_addr":"172.21.15.203:6831/512155569","heartbeat_front_addr":"172.21.15.203:6829/512155569","state":["exists","up"]},{"osd":7,"uuid":"a95dc7ca-9f1b-4295-af70-68a22a2d7d5e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":72,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6808","nonce":3331450438},{"type":"v1","addr":"172.21.15.203:6809","nonce":3331450438}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6810","nonce":3331450438},{"type":"v1","addr":"172.21.15.203:6811","nonce":3331450438}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6814","nonce":3331450438},{"type":"v1","addr":"172.21.15.203:6815","nonce":3331450438}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6812","nonce":3331450438},{"type":"v1","addr":"172.21.15.203:6813","nonce":3331450438}]},"public_addr":"172.21.15.203:6809/3331450438","cluster_addr":"172.21.15.203:6811/3331450438","heartbeat_back_addr":"172.21.15.203:6815/3331450438","heartbeat_front_addr":"172.21.15.203:6813/3331450438","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-09T13:33:50.547652+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-09T13:33:50.589469+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-09T13:33:50.561724+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-09T13:33:50.599650+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-09T13:33:50.411855+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-09T13:33:50.219681+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-09T13:33:50.504664+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-09T13:33:50.441248+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":7,"snaps":[{"begin":104,"length":1},{"begin":221,"length":1}]}],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-08-09T13:50:25.155 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:25.154+0000 7f9a79c21640 1 -- 172.21.15.43:0/3351621737 >> [v2:172.21.15.203:6832/3325141227,v1:172.21.15.203:6833/3325141227] conn(0x7f9a5806a170 msgr2=0x7f9a5806c630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:50:25.156 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:25.154+0000 7f9a79c21640 1 --2- 172.21.15.43:0/3351621737 >> [v2:172.21.15.203:6832/3325141227,v1:172.21.15.203:6833/3325141227] conn(0x7f9a5806a170 0x7f9a5806c630 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7f9a60036000 tx=0x7f9a60030000 comp rx=0 tx=0).stop 2023-08-09T13:50:25.156 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:25.154+0000 7f9a79c21640 1 -- 172.21.15.43:0/3351621737 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f9a740acdf0 msgr2=0x7f9a74157df0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:50:25.156 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:25.154+0000 7f9a79c21640 1 --2- 172.21.15.43:0/3351621737 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f9a740acdf0 0x7f9a74157df0 secure :-1 s=READY pgs=988 cs=0 l=1 rev1=1 crypto rx=0x7f9a6800d800 tx=0x7f9a6800dcc0 comp rx=0 tx=0).stop 2023-08-09T13:50:25.156 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:25.154+0000 7f9a79c21640 1 -- 172.21.15.43:0/3351621737 shutdown_connections 2023-08-09T13:50:25.156 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:25.154+0000 7f9a79c21640 1 --2- 172.21.15.43:0/3351621737 >> [v2:172.21.15.203:6832/3325141227,v1:172.21.15.203:6833/3325141227] conn(0x7f9a5806a170 0x7f9a5806c630 unknown :-1 s=CLOSED pgs=52 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:50:25.156 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:25.154+0000 7f9a79c21640 1 --2- 172.21.15.43:0/3351621737 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f9a740c8b30 0x7f9a740c4370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:50:25.157 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:25.154+0000 7f9a79c21640 1 --2- 172.21.15.43:0/3351621737 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f9a740c8750 0x7f9a740c5870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:50:25.157 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:25.154+0000 7f9a79c21640 1 --2- 172.21.15.43:0/3351621737 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f9a740acdf0 0x7f9a74157df0 unknown :-1 s=CLOSED pgs=988 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:50:25.157 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:25.154+0000 7f9a79c21640 1 -- 172.21.15.43:0/3351621737 >> 172.21.15.43:0/3351621737 conn(0x7f9a740cde90 msgr2=0x7f9a740aad80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-09T13:50:25.157 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:25.154+0000 7f9a79c21640 1 -- 172.21.15.43:0/3351621737 shutdown_connections 2023-08-09T13:50:25.157 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:25.155+0000 7f9a79c21640 1 -- 172.21.15.43:0/3351621737 wait complete. 2023-08-09T13:50:25.175 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': 'rbd', 'create_time': '2023-08-09T13:33:54.365137+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}, 'read_balance': {'score_acting': 2, 'score_stable': 2, 'optimal_score': 1, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': '.mgr', 'create_time': '2023-08-09T13:33:54.831803+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-08-09T13:50:25.176 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get rbd pg_num 2023-08-09T13:50:25.409 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:25.408+0000 7fd2df59e640 1 Processor -- start 2023-08-09T13:50:25.410 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:25.408+0000 7fd2df59e640 1 -- start start 2023-08-09T13:50:25.411 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:25.410+0000 7fd2df59e640 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7fd2e00b7df0 0x7fd2e00d1b30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:50:25.411 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:25.410+0000 7fd2df59e640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7fd2e00cebf0 0x7fd2e00cf040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:50:25.411 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:25.410+0000 7fd2df59e640 1 -- --> v1:172.21.15.43:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd2e00cf580 con 0x7fd2e015c2d0 2023-08-09T13:50:25.412 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:25.410+0000 7fd2df59e640 1 -- --> v2:172.21.15.43:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd2e00b8230 con 0x7fd2e00cebf0 2023-08-09T13:50:25.412 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:25.410+0000 7fd2df59e640 1 -- --> v2:172.21.15.203:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd2e00d16a0 con 0x7fd2e00b7df0 2023-08-09T13:50:25.412 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:25.410+0000 7fd2de59c640 1 --1- >> v1:172.21.15.43:6789/0 conn(0x7fd2e015c2d0 0x7fd2e00cd020 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.43:6789/0 says I am v1:172.21.15.43:32936/0 (socket says 172.21.15.43:32936) 2023-08-09T13:50:25.412 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:25.410+0000 7fd2de59c640 1 -- 172.21.15.43:0/2439791313 learned_addr learned my addr 172.21.15.43:0/2439791313 (peer_addr_for_me v1:172.21.15.43:0/0) 2023-08-09T13:50:25.413 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:25.410+0000 7fd2ded9d640 1 --2- 172.21.15.43:0/2439791313 >> v2:172.21.15.43:3300/0 conn(0x7fd2e00cebf0 0x7fd2e00cf040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:50:25.413 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:25.410+0000 7fd2dd59a640 1 -- 172.21.15.43:0/2439791313 <== mon.0 v1:172.21.15.43:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1737701013 0 0) 0x7fd2d0001490 con 0x7fd2e015c2d0 2023-08-09T13:50:25.413 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:25.410+0000 7fd2ddd9b640 1 --2- 172.21.15.43:0/2439791313 >> v2:172.21.15.203:3300/0 conn(0x7fd2e00b7df0 0x7fd2e00d1b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:50:25.413 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:25.411+0000 7fd2dd59a640 1 -- 172.21.15.43:0/2439791313 --> v1:172.21.15.43:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fd2cc003520 con 0x7fd2e015c2d0 2023-08-09T13:50:25.413 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:25.411+0000 7fd2ded9d640 1 -- 172.21.15.43:0/2439791313 >> v2:172.21.15.203:3300/0 conn(0x7fd2e00b7df0 msgr2=0x7fd2e00d1b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-09T13:50:25.414 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:25.411+0000 7fd2ded9d640 1 --2- 172.21.15.43:0/2439791313 >> v2:172.21.15.203:3300/0 conn(0x7fd2e00b7df0 0x7fd2e00d1b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:50:25.414 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:25.411+0000 7fd2ded9d640 1 -- 172.21.15.43:0/2439791313 >> v1:172.21.15.43:6789/0 conn(0x7fd2e015c2d0 legacy=0x7fd2e00cd020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:50:25.414 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:25.411+0000 7fd2ded9d640 1 -- 172.21.15.43:0/2439791313 --> v2:172.21.15.43:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd2e00d0d70 con 0x7fd2e00cebf0 2023-08-09T13:50:25.414 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:25.411+0000 7fd2ddd9b640 1 --2- 172.21.15.43:0/2439791313 >> v2:172.21.15.203:3300/0 conn(0x7fd2e00b7df0 0x7fd2e00d1b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-08-09T13:50:25.415 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:25.412+0000 7fd2ded9d640 1 --2- 172.21.15.43:0/2439791313 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fd2e00cebf0 0x7fd2e00cf040 secure :-1 s=READY pgs=990 cs=0 l=1 rev1=1 crypto rx=0x7fd2e00c5ce0 tx=0x7fd2d802b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=66a060d3bbe9e926 server_cookie=0 in_seq=0 out_seq=0 2023-08-09T13:50:25.415 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:25.412+0000 7fd2dd59a640 1 -- 172.21.15.43:0/2439791313 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd2d8031830 con 0x7fd2e00cebf0 2023-08-09T13:50:25.415 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:25.412+0000 7fd2dd59a640 1 -- 172.21.15.43:0/2439791313 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd2d8031990 con 0x7fd2e00cebf0 2023-08-09T13:50:25.416 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:25.412+0000 7fd2dd59a640 1 -- 172.21.15.43:0/2439791313 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd2d8031d00 con 0x7fd2e00cebf0 2023-08-09T13:50:25.416 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:25.413+0000 7fd2df59e640 1 -- 172.21.15.43:0/2439791313 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fd2e00cebf0 msgr2=0x7fd2e00cf040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:50:25.416 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:25.413+0000 7fd2df59e640 1 --2- 172.21.15.43:0/2439791313 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fd2e00cebf0 0x7fd2e00cf040 secure :-1 s=READY pgs=990 cs=0 l=1 rev1=1 crypto rx=0x7fd2e00c5ce0 tx=0x7fd2d802b8d0 comp rx=0 tx=0).stop 2023-08-09T13:50:25.416 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:25.413+0000 7fd2df59e640 1 -- 172.21.15.43:0/2439791313 shutdown_connections 2023-08-09T13:50:25.416 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:25.413+0000 7fd2df59e640 1 --2- 172.21.15.43:0/2439791313 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fd2e00cebf0 0x7fd2e00cf040 unknown :-1 s=CLOSED pgs=990 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:50:25.416 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:25.413+0000 7fd2df59e640 1 --2- 172.21.15.43:0/2439791313 >> v2:172.21.15.203:3300/0 conn(0x7fd2e00b7df0 0x7fd2e00d1b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:50:25.417 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:25.413+0000 7fd2df59e640 1 -- 172.21.15.43:0/2439791313 >> 172.21.15.43:0/2439791313 conn(0x7fd2e006a230 msgr2=0x7fd2e006a640 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-09T13:50:25.417 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:25.413+0000 7fd2df59e640 1 -- 172.21.15.43:0/2439791313 shutdown_connections 2023-08-09T13:50:25.417 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:25.413+0000 7fd2df59e640 1 -- 172.21.15.43:0/2439791313 wait complete. 2023-08-09T13:50:25.418 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:25.415+0000 7fd2df59e640 1 Processor -- start 2023-08-09T13:50:25.418 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:25.415+0000 7fd2df59e640 1 -- start start 2023-08-09T13:50:25.418 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:25.416+0000 7fd2df59e640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fd2e00b7df0 0x7fd2e0158060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:50:25.419 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:25.416+0000 7fd2df59e640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fd2e00cebf0 0x7fd2e015d230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:50:25.419 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:25.416+0000 7fd2df59e640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fd2e015c2d0 0x7fd2e00ca8a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:50:25.419 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:25.416+0000 7fd2df59e640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd2e01585a0 con 0x7fd2e00b7df0 2023-08-09T13:50:25.419 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:25.416+0000 7fd2df59e640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd2e00a9700 con 0x7fd2e015c2d0 2023-08-09T13:50:25.419 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:25.416+0000 7fd2df59e640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd2e015d770 con 0x7fd2e00cebf0 2023-08-09T13:50:25.419 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:25.417+0000 7fd2de59c640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fd2e00b7df0 0x7fd2e0158060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:50:25.419 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:25.417+0000 7fd2de59c640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fd2e00b7df0 0x7fd2e0158060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:35590/0 (socket says 172.21.15.43:35590) 2023-08-09T13:50:25.420 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:25.417+0000 7fd2de59c640 1 -- 172.21.15.43:0/3514995903 learned_addr learned my addr 172.21.15.43:0/3514995903 (peer_addr_for_me v2:172.21.15.43:0/0) 2023-08-09T13:50:25.421 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:25.417+0000 7fd2ddd9b640 1 --2- 172.21.15.43:0/3514995903 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fd2e00cebf0 0x7fd2e015d230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:50:25.421 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:25.417+0000 7fd2ded9d640 1 --2- 172.21.15.43:0/3514995903 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fd2e015c2d0 0x7fd2e00ca8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:50:25.421 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:25.417+0000 7fd2de59c640 1 -- 172.21.15.43:0/3514995903 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fd2e00cebf0 msgr2=0x7fd2e015d230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:50:25.422 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:25.417+0000 7fd2de59c640 1 --2- 172.21.15.43:0/3514995903 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fd2e00cebf0 0x7fd2e015d230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:50:25.422 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:25.417+0000 7fd2de59c640 1 -- 172.21.15.43:0/3514995903 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fd2e015c2d0 msgr2=0x7fd2e00ca8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:50:25.422 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:25.417+0000 7fd2de59c640 1 --2- 172.21.15.43:0/3514995903 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fd2e015c2d0 0x7fd2e00ca8a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:50:25.422 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:25.417+0000 7fd2de59c640 1 -- 172.21.15.43:0/3514995903 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd2d8009400 con 0x7fd2e00b7df0 2023-08-09T13:50:25.422 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:25.417+0000 7fd2ded9d640 1 --2- 172.21.15.43:0/3514995903 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fd2e015c2d0 0x7fd2e00ca8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-08-09T13:50:25.422 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:25.417+0000 7fd2ddd9b640 1 --2- 172.21.15.43:0/3514995903 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fd2e00cebf0 0x7fd2e015d230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-08-09T13:50:25.423 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:25.417+0000 7fd2de59c640 1 --2- 172.21.15.43:0/3514995903 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fd2e00b7df0 0x7fd2e0158060 secure :-1 s=READY pgs=991 cs=0 l=1 rev1=1 crypto rx=0x7fd2d000cae0 tx=0x7fd2d000a060 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-09T13:50:25.423 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:25.418+0000 7fd2d6ffd640 1 -- 172.21.15.43:0/3514995903 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd2d000ac40 con 0x7fd2e00b7df0 2023-08-09T13:50:25.423 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:25.418+0000 7fd2d6ffd640 1 -- 172.21.15.43:0/3514995903 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd2d000adc0 con 0x7fd2e00b7df0 2023-08-09T13:50:25.423 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:25.418+0000 7fd2d6ffd640 1 -- 172.21.15.43:0/3514995903 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd2d0012640 con 0x7fd2e00b7df0 2023-08-09T13:50:25.423 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:25.418+0000 7fd2df59e640 1 -- 172.21.15.43:0/3514995903 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd2e01f2770 con 0x7fd2e00b7df0 2023-08-09T13:50:25.423 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:25.418+0000 7fd2df59e640 1 -- 172.21.15.43:0/3514995903 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd2e015c750 con 0x7fd2e00b7df0 2023-08-09T13:50:25.424 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:25.419+0000 7fd2df59e640 1 -- 172.21.15.43:0/3514995903 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd2e004e390 con 0x7fd2e00b7df0 2023-08-09T13:50:25.425 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:25.420+0000 7fd2d6ffd640 1 -- 172.21.15.43:0/3514995903 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 29) v1 ==== 88443+0+0 (secure 0 0 0) 0x7fd2d0007070 con 0x7fd2e00b7df0 2023-08-09T13:50:25.425 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:25.421+0000 7fd2d6ffd640 1 --2- 172.21.15.43:0/3514995903 >> [v2:172.21.15.203:6832/3325141227,v1:172.21.15.203:6833/3325141227] conn(0x7fd2c006a170 0x7fd2c006c630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:50:25.425 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:25.421+0000 7fd2d6ffd640 1 -- 172.21.15.43:0/3514995903 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 5574+0+0 (secure 0 0 0) 0x7fd2d0088360 con 0x7fd2e00b7df0 2023-08-09T13:50:25.425 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:25.422+0000 7fd2ddd9b640 1 --2- 172.21.15.43:0/3514995903 >> [v2:172.21.15.203:6832/3325141227,v1:172.21.15.203:6833/3325141227] conn(0x7fd2c006a170 0x7fd2c006c630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:50:25.425 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:25.423+0000 7fd2ddd9b640 1 --2- 172.21.15.43:0/3514995903 >> [v2:172.21.15.203:6832/3325141227,v1:172.21.15.203:6833/3325141227] conn(0x7fd2c006a170 0x7fd2c006c630 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7fd2c8006fd0 tx=0x7fd2c8008040 comp rx=0 tx=0).ready entity=mgr.5519 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-09T13:50:25.427 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:25.425+0000 7fd2d6ffd640 1 -- 172.21.15.43:0/3514995903 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178066 (secure 0 0 0) 0x7fd2d0053b20 con 0x7fd2e00b7df0 2023-08-09T13:50:25.621 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:25.620+0000 7fd2df59e640 1 -- 172.21.15.43:0/3514995903 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"} v 0) v1 -- 0x7fd2e00cf580 con 0x7fd2e00b7df0 2023-08-09T13:50:25.622 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:25.621+0000 7fd2d6ffd640 1 -- 172.21.15.43:0/3514995903 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"}]=0 v86) v1 ==== 92+0+10 (secure 0 0 0) 0x7fd2d0017020 con 0x7fd2e00b7df0 2023-08-09T13:50:25.622 INFO:teuthology.orchestra.run.smithi043.stdout:pg_num: 8 2023-08-09T13:50:25.625 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:25.623+0000 7fd2df59e640 1 -- 172.21.15.43:0/3514995903 >> [v2:172.21.15.203:6832/3325141227,v1:172.21.15.203:6833/3325141227] conn(0x7fd2c006a170 msgr2=0x7fd2c006c630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:50:25.625 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:25.623+0000 7fd2df59e640 1 --2- 172.21.15.43:0/3514995903 >> [v2:172.21.15.203:6832/3325141227,v1:172.21.15.203:6833/3325141227] conn(0x7fd2c006a170 0x7fd2c006c630 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7fd2c8006fd0 tx=0x7fd2c8008040 comp rx=0 tx=0).stop 2023-08-09T13:50:25.625 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:25.623+0000 7fd2df59e640 1 -- 172.21.15.43:0/3514995903 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fd2e00b7df0 msgr2=0x7fd2e0158060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:50:25.625 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:25.623+0000 7fd2df59e640 1 --2- 172.21.15.43:0/3514995903 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fd2e00b7df0 0x7fd2e0158060 secure :-1 s=READY pgs=991 cs=0 l=1 rev1=1 crypto rx=0x7fd2d000cae0 tx=0x7fd2d000a060 comp rx=0 tx=0).stop 2023-08-09T13:50:25.625 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:25.623+0000 7fd2df59e640 1 -- 172.21.15.43:0/3514995903 shutdown_connections 2023-08-09T13:50:25.626 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:25.623+0000 7fd2df59e640 1 --2- 172.21.15.43:0/3514995903 >> [v2:172.21.15.203:6832/3325141227,v1:172.21.15.203:6833/3325141227] conn(0x7fd2c006a170 0x7fd2c006c630 unknown :-1 s=CLOSED pgs=53 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:50:25.626 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:25.623+0000 7fd2df59e640 1 --2- 172.21.15.43:0/3514995903 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fd2e015c2d0 0x7fd2e00ca8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:50:25.626 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:25.623+0000 7fd2df59e640 1 --2- 172.21.15.43:0/3514995903 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fd2e00cebf0 0x7fd2e015d230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:50:25.626 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:25.623+0000 7fd2df59e640 1 --2- 172.21.15.43:0/3514995903 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fd2e00b7df0 0x7fd2e0158060 unknown :-1 s=CLOSED pgs=991 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:50:25.626 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:25.623+0000 7fd2df59e640 1 -- 172.21.15.43:0/3514995903 >> 172.21.15.43:0/3514995903 conn(0x7fd2e006a230 msgr2=0x7fd2e00b2860 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-09T13:50:25.626 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:25.623+0000 7fd2df59e640 1 -- 172.21.15.43:0/3514995903 shutdown_connections 2023-08-09T13:50:25.626 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:25.624+0000 7fd2df59e640 1 -- 172.21.15.43:0/3514995903 wait complete. 2023-08-09T13:50:25.644 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-08-09T13:50:25.873 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:25.872+0000 7ffb1ac2f640 1 Processor -- start 2023-08-09T13:50:25.873 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:25.872+0000 7ffb1ac2f640 1 -- start start 2023-08-09T13:50:25.875 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:25.874+0000 7ffb1ac2f640 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7ffb1415df60 0x7ffb140ce1c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:50:25.876 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:25.874+0000 7ffb1ac2f640 1 -- --> v1:172.21.15.43:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ffb1415aef0 con 0x7ffb140cb2d0 2023-08-09T13:50:25.876 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:25.874+0000 7ffb1ac2f640 1 -- --> v1:172.21.15.203:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ffb1415e940 con 0x7ffb1415e3d0 2023-08-09T13:50:25.876 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:25.874+0000 7ffb1ac2f640 1 -- --> v2:172.21.15.203:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ffb1415a970 con 0x7ffb1415df60 2023-08-09T13:50:25.876 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:25.874+0000 7ffb189cb640 1 --1- >> v1:172.21.15.43:6789/0 conn(0x7ffb140cb2d0 0x7ffb1415b380 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.43:6789/0 says I am v1:172.21.15.43:32938/0 (socket says 172.21.15.43:32938) 2023-08-09T13:50:25.876 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:25.874+0000 7ffb189cb640 1 -- 172.21.15.43:0/385802417 learned_addr learned my addr 172.21.15.43:0/385802417 (peer_addr_for_me v1:172.21.15.43:0/0) 2023-08-09T13:50:25.878 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:25.874+0000 7ffb0bfff640 1 --2- 172.21.15.43:0/385802417 >> v2:172.21.15.203:3300/0 conn(0x7ffb1415df60 0x7ffb140ce1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:50:25.879 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:25.875+0000 7ffb0b7fe640 1 -- 172.21.15.43:0/385802417 <== mon.0 v1:172.21.15.43:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1785826969 0 0) 0x7ffb1415aef0 con 0x7ffb140cb2d0 2023-08-09T13:50:25.879 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:25.875+0000 7ffb0b7fe640 1 -- 172.21.15.43:0/385802417 --> v1:172.21.15.43:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ffafc003520 con 0x7ffb140cb2d0 2023-08-09T13:50:25.879 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:25.875+0000 7ffb0b7fe640 1 -- 172.21.15.43:0/385802417 <== mon.0 v1:172.21.15.43:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2407735951 0 0) 0x7ffafc003520 con 0x7ffb140cb2d0 2023-08-09T13:50:25.879 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:25.875+0000 7ffb0b7fe640 1 -- 172.21.15.43:0/385802417 >> v2:172.21.15.203:3300/0 conn(0x7ffb1415df60 msgr2=0x7ffb140ce1c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-09T13:50:25.879 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:25.875+0000 7ffb0b7fe640 1 --2- 172.21.15.43:0/385802417 >> v2:172.21.15.203:3300/0 conn(0x7ffb1415df60 0x7ffb140ce1c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:50:25.879 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:25.875+0000 7ffb0b7fe640 1 -- 172.21.15.43:0/385802417 >> v1:172.21.15.203:6789/0 conn(0x7ffb1415e3d0 legacy=0x7ffb140c9f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:50:25.880 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:25.875+0000 7ffb0b7fe640 1 -- 172.21.15.43:0/385802417 --> v1:172.21.15.43:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ffb1415ec20 con 0x7ffb140cb2d0 2023-08-09T13:50:25.880 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:25.875+0000 7ffb0b7fe640 1 -- 172.21.15.43:0/385802417 <== mon.0 v1:172.21.15.43:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1285150740 0 0) 0x7ffb100019f0 con 0x7ffb140cb2d0 2023-08-09T13:50:25.880 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:25.875+0000 7ffb0b7fe640 1 -- 172.21.15.43:0/385802417 >> v1:172.21.15.43:6789/0 conn(0x7ffb140cb2d0 legacy=0x7ffb1415b380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:50:25.880 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:25.875+0000 7ffb0b7fe640 1 --2- 172.21.15.43:0/385802417 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7ffafc003970 0x7ffafc003d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:50:25.881 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:25.875+0000 7ffb0b7fe640 1 --2- 172.21.15.43:0/385802417 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7ffafc0047c0 0x7ffafc004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:50:25.881 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:25.875+0000 7ffb0b7fe640 1 --2- 172.21.15.43:0/385802417 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ffafc005180 0x7ffafc00b980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:50:25.881 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:25.875+0000 7ffb0b7fe640 1 -- 172.21.15.43:0/385802417 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffb1415a970 con 0x7ffafc005180 2023-08-09T13:50:25.881 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:25.875+0000 7ffb0b7fe640 1 -- 172.21.15.43:0/385802417 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffafc00c120 con 0x7ffafc0047c0 2023-08-09T13:50:25.881 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:25.875+0000 7ffb0b7fe640 1 -- 172.21.15.43:0/385802417 --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ffafc00c2e0 con 0x7ffafc003970 2023-08-09T13:50:25.881 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:25.876+0000 7ffb191cc640 1 --2- 172.21.15.43:0/385802417 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ffafc005180 0x7ffafc00b980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:50:25.881 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:25.876+0000 7ffb189cb640 1 --2- 172.21.15.43:0/385802417 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7ffafc003970 0x7ffafc003d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:50:25.881 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:25.876+0000 7ffb0bfff640 1 --2- 172.21.15.43:0/385802417 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7ffafc0047c0 0x7ffafc004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:50:25.882 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:25.876+0000 7ffb191cc640 1 -- 172.21.15.43:0/385802417 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7ffafc003970 msgr2=0x7ffafc003d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-09T13:50:25.882 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:25.876+0000 7ffb191cc640 1 --2- 172.21.15.43:0/385802417 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7ffafc003970 0x7ffafc003d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:50:25.882 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:25.876+0000 7ffb191cc640 1 -- 172.21.15.43:0/385802417 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7ffafc0047c0 msgr2=0x7ffafc004c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-09T13:50:25.883 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:25.876+0000 7ffb191cc640 1 --2- 172.21.15.43:0/385802417 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7ffafc0047c0 0x7ffafc004c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:50:25.883 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:25.876+0000 7ffb191cc640 1 -- 172.21.15.43:0/385802417 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ffafc003520 con 0x7ffafc005180 2023-08-09T13:50:25.883 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:25.877+0000 7ffb191cc640 1 -- 172.21.15.43:0/385802417 reap_dead start 2023-08-09T13:50:25.883 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:25.877+0000 7ffb191cc640 1 --2- 172.21.15.43:0/385802417 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ffafc005180 0x7ffafc00b980 secure :-1 s=READY pgs=993 cs=0 l=1 rev1=1 crypto rx=0x7ffb0c001750 tx=0x7ffafc0057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5d2a540d9740dc36 server_cookie=0 in_seq=0 out_seq=0 2023-08-09T13:50:25.886 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:25.877+0000 7ffb0b7fe640 1 -- 172.21.15.43:0/385802417 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffb0c0052d0 con 0x7ffafc005180 2023-08-09T13:50:25.886 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:25.878+0000 7ffb0b7fe640 1 -- 172.21.15.43:0/385802417 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ffb0c02e050 con 0x7ffafc005180 2023-08-09T13:50:25.887 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:25.878+0000 7ffb0b7fe640 1 -- 172.21.15.43:0/385802417 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffb0c0035f0 con 0x7ffafc005180 2023-08-09T13:50:25.887 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:25.878+0000 7ffb1ac2f640 1 -- 172.21.15.43:0/385802417 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ffafc005180 msgr2=0x7ffafc00b980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:50:25.887 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:25.878+0000 7ffb1ac2f640 1 --2- 172.21.15.43:0/385802417 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ffafc005180 0x7ffafc00b980 secure :-1 s=READY pgs=993 cs=0 l=1 rev1=1 crypto rx=0x7ffb0c001750 tx=0x7ffafc0057f0 comp rx=0 tx=0).stop 2023-08-09T13:50:25.888 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:25.878+0000 7ffb1ac2f640 1 -- 172.21.15.43:0/385802417 shutdown_connections 2023-08-09T13:50:25.888 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:25.878+0000 7ffb1ac2f640 1 --2- 172.21.15.43:0/385802417 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ffafc005180 0x7ffafc00b980 unknown :-1 s=CLOSED pgs=993 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:50:25.889 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:25.878+0000 7ffb1ac2f640 1 -- 172.21.15.43:0/385802417 >> 172.21.15.43:0/385802417 conn(0x7ffb140aff50 msgr2=0x7ffb140b0360 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-09T13:50:25.889 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:25.878+0000 7ffb1ac2f640 1 -- 172.21.15.43:0/385802417 shutdown_connections 2023-08-09T13:50:25.889 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:25.878+0000 7ffb1ac2f640 1 -- 172.21.15.43:0/385802417 wait complete. 2023-08-09T13:50:25.889 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:25.880+0000 7ffb1ac2f640 1 Processor -- start 2023-08-09T13:50:25.889 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:25.880+0000 7ffb1ac2f640 1 -- start start 2023-08-09T13:50:25.889 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:25.881+0000 7ffb1ac2f640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7ffafc005180 0x7ffb141527b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:50:25.890 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:25.881+0000 7ffb1ac2f640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ffb140b0f20 0x7ffb141e4900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:50:25.890 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:25.881+0000 7ffb1ac2f640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7ffb141e58f0 0x7ffb140b2f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:50:25.890 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:25.881+0000 7ffb1ac2f640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffb140b1450 con 0x7ffb140b0f20 2023-08-09T13:50:25.890 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:25.881+0000 7ffb1ac2f640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffb140b1730 con 0x7ffb141e58f0 2023-08-09T13:50:25.890 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:25.881+0000 7ffb1ac2f640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ffb141e50f0 con 0x7ffafc005180 2023-08-09T13:50:25.890 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:25.882+0000 7ffb0bfff640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ffb140b0f20 0x7ffb141e4900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:50:25.890 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:25.882+0000 7ffb0bfff640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ffb140b0f20 0x7ffb141e4900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:35608/0 (socket says 172.21.15.43:35608) 2023-08-09T13:50:25.891 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:25.882+0000 7ffb0bfff640 1 -- 172.21.15.43:0/2074584370 learned_addr learned my addr 172.21.15.43:0/2074584370 (peer_addr_for_me v2:172.21.15.43:0/0) 2023-08-09T13:50:25.891 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:25.882+0000 7ffb189cb640 1 --2- 172.21.15.43:0/2074584370 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7ffafc005180 0x7ffb141527b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:50:25.891 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:25.882+0000 7ffb191cc640 1 --2- 172.21.15.43:0/2074584370 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7ffb141e58f0 0x7ffb140b2f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:50:25.891 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:25.882+0000 7ffb0bfff640 1 -- 172.21.15.43:0/2074584370 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7ffafc005180 msgr2=0x7ffb141527b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:50:25.891 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:25.882+0000 7ffb0bfff640 1 --2- 172.21.15.43:0/2074584370 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7ffafc005180 0x7ffb141527b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:50:25.891 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:25.882+0000 7ffb0bfff640 1 -- 172.21.15.43:0/2074584370 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7ffb141e58f0 msgr2=0x7ffb140b2f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:50:25.891 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:25.882+0000 7ffb0bfff640 1 --2- 172.21.15.43:0/2074584370 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7ffb141e58f0 0x7ffb140b2f50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:50:25.892 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:25.882+0000 7ffb0bfff640 1 -- 172.21.15.43:0/2074584370 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ffb141e53d0 con 0x7ffb140b0f20 2023-08-09T13:50:25.892 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:25.883+0000 7ffb0bfff640 1 --2- 172.21.15.43:0/2074584370 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ffb140b0f20 0x7ffb141e4900 secure :-1 s=READY pgs=994 cs=0 l=1 rev1=1 crypto rx=0x7ffb04011540 tx=0x7ffb04011a00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-09T13:50:25.892 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:25.883+0000 7ffb097fa640 1 -- 172.21.15.43:0/2074584370 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffb04015930 con 0x7ffb140b0f20 2023-08-09T13:50:25.892 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:25.883+0000 7ffb097fa640 1 -- 172.21.15.43:0/2074584370 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ffb04015a90 con 0x7ffb140b0f20 2023-08-09T13:50:25.892 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:25.883+0000 7ffb097fa640 1 -- 172.21.15.43:0/2074584370 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffb04015e60 con 0x7ffb140b0f20 2023-08-09T13:50:25.892 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:25.883+0000 7ffb1ac2f640 1 -- 172.21.15.43:0/2074584370 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ffb141e6ad0 con 0x7ffb140b0f20 2023-08-09T13:50:25.892 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:25.883+0000 7ffb1ac2f640 1 -- 172.21.15.43:0/2074584370 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ffb141e5d70 con 0x7ffb140b0f20 2023-08-09T13:50:25.893 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:25.884+0000 7ffb1ac2f640 1 -- 172.21.15.43:0/2074584370 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ffb141e5230 con 0x7ffb140b0f20 2023-08-09T13:50:25.893 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:25.885+0000 7ffb097fa640 1 -- 172.21.15.43:0/2074584370 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 29) v1 ==== 88443+0+0 (secure 0 0 0) 0x7ffb0400f460 con 0x7ffb140b0f20 2023-08-09T13:50:25.894 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:25.886+0000 7ffb097fa640 1 --2- 172.21.15.43:0/2074584370 >> [v2:172.21.15.203:6832/3325141227,v1:172.21.15.203:6833/3325141227] conn(0x7ffb0006a170 0x7ffb0006c630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:50:25.894 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:25.886+0000 7ffb097fa640 1 -- 172.21.15.43:0/2074584370 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 5574+0+0 (secure 0 0 0) 0x7ffb04088e60 con 0x7ffb140b0f20 2023-08-09T13:50:25.894 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:25.887+0000 7ffb189cb640 1 --2- 172.21.15.43:0/2074584370 >> [v2:172.21.15.203:6832/3325141227,v1:172.21.15.203:6833/3325141227] conn(0x7ffb0006a170 0x7ffb0006c630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:50:25.894 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:25.888+0000 7ffb189cb640 1 --2- 172.21.15.43:0/2074584370 >> [v2:172.21.15.203:6832/3325141227,v1:172.21.15.203:6833/3325141227] conn(0x7ffb0006a170 0x7ffb0006c630 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7ffb100013d0 tx=0x7ffb10009040 comp rx=0 tx=0).ready entity=mgr.5519 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-09T13:50:25.894 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:25.890+0000 7ffb097fa640 1 -- 172.21.15.43:0/2074584370 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178066 (secure 0 0 0) 0x7ffb141e5230 con 0x7ffb140b0f20 2023-08-09T13:50:26.085 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:26.083+0000 7ffb1ac2f640 1 -- 172.21.15.43:0/2074584370 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7ffb141e5230 con 0x7ffb140b0f20 2023-08-09T13:50:26.085 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:26.084+0000 7ffb097fa640 1 -- 172.21.15.43:0/2074584370 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v86) v1 ==== 93+0+10 (secure 0 0 0) 0x7ffb141e5230 con 0x7ffb140b0f20 2023-08-09T13:50:26.086 INFO:teuthology.orchestra.run.smithi043.stdout:pg_num: 1 2023-08-09T13:50:26.088 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:26.087+0000 7ffb1ac2f640 1 -- 172.21.15.43:0/2074584370 >> [v2:172.21.15.203:6832/3325141227,v1:172.21.15.203:6833/3325141227] conn(0x7ffb0006a170 msgr2=0x7ffb0006c630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:50:26.089 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:26.087+0000 7ffb1ac2f640 1 --2- 172.21.15.43:0/2074584370 >> [v2:172.21.15.203:6832/3325141227,v1:172.21.15.203:6833/3325141227] conn(0x7ffb0006a170 0x7ffb0006c630 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7ffb100013d0 tx=0x7ffb10009040 comp rx=0 tx=0).stop 2023-08-09T13:50:26.089 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:26.087+0000 7ffb1ac2f640 1 -- 172.21.15.43:0/2074584370 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ffb140b0f20 msgr2=0x7ffb141e4900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:50:26.089 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:26.087+0000 7ffb1ac2f640 1 --2- 172.21.15.43:0/2074584370 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ffb140b0f20 0x7ffb141e4900 secure :-1 s=READY pgs=994 cs=0 l=1 rev1=1 crypto rx=0x7ffb04011540 tx=0x7ffb04011a00 comp rx=0 tx=0).stop 2023-08-09T13:50:26.089 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:26.087+0000 7ffb1ac2f640 1 -- 172.21.15.43:0/2074584370 shutdown_connections 2023-08-09T13:50:26.089 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:26.087+0000 7ffb1ac2f640 1 --2- 172.21.15.43:0/2074584370 >> [v2:172.21.15.203:6832/3325141227,v1:172.21.15.203:6833/3325141227] conn(0x7ffb0006a170 0x7ffb0006c630 unknown :-1 s=CLOSED pgs=54 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:50:26.089 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:26.087+0000 7ffb1ac2f640 1 --2- 172.21.15.43:0/2074584370 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7ffb141e58f0 0x7ffb140b2f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:50:26.089 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:26.087+0000 7ffb1ac2f640 1 --2- 172.21.15.43:0/2074584370 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ffb140b0f20 0x7ffb141e4900 unknown :-1 s=CLOSED pgs=994 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:50:26.090 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:26.087+0000 7ffb1ac2f640 1 --2- 172.21.15.43:0/2074584370 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7ffafc005180 0x7ffb141527b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:50:26.090 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:26.087+0000 7ffb1ac2f640 1 -- 172.21.15.43:0/2074584370 >> 172.21.15.43:0/2074584370 conn(0x7ffb140aff50 msgr2=0x7ffb140c85e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-09T13:50:26.090 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:26.087+0000 7ffb1ac2f640 1 -- 172.21.15.43:0/2074584370 shutdown_connections 2023-08-09T13:50:26.090 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:26.087+0000 7ffb1ac2f640 1 -- 172.21.15.43:0/2074584370 wait complete. 2023-08-09T13:50:26.109 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --format=json-pretty osd lspools 2023-08-09T13:50:26.322 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:26.320+0000 7fb28b59e640 1 Processor -- start 2023-08-09T13:50:26.323 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:26.321+0000 7fb28b59e640 1 -- start start 2023-08-09T13:50:26.324 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:26.322+0000 7fb28b59e640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fb28c0b1050 0x7fb28c0b4760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:50:26.324 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:26.322+0000 7fb28b59e640 1 -- --> v1:172.21.15.43:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb28c0b7bf0 con 0x7fb28c15f7b0 2023-08-09T13:50:26.324 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:26.322+0000 7fb28b59e640 1 -- --> v1:172.21.15.203:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb28c0b7ed0 con 0x7fb28c0cd6b0 2023-08-09T13:50:26.325 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:26.322+0000 7fb28b59e640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb28c0d0370 con 0x7fb28c0b1050 2023-08-09T13:50:26.325 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:26.323+0000 7fb28a59c640 1 --1- >> v1:172.21.15.43:6789/0 conn(0x7fb28c15f7b0 0x7fb28c0ccfa0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.43:6789/0 says I am v1:172.21.15.43:32946/0 (socket says 172.21.15.43:32946) 2023-08-09T13:50:26.325 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:26.323+0000 7fb28a59c640 1 -- 172.21.15.43:0/582580146 learned_addr learned my addr 172.21.15.43:0/582580146 (peer_addr_for_me v1:172.21.15.43:0/0) 2023-08-09T13:50:26.326 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:26.323+0000 7fb283fff640 1 --2- 172.21.15.43:0/582580146 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fb28c0b1050 0x7fb28c0b4760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:50:26.326 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:26.324+0000 7fb289d9b640 1 -- 172.21.15.43:0/582580146 <== mon.0 v1:172.21.15.43:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1542090890 0 0) 0x7fb28c0b7bf0 con 0x7fb28c15f7b0 2023-08-09T13:50:26.326 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:26.324+0000 7fb289d9b640 1 -- 172.21.15.43:0/582580146 --> v1:172.21.15.43:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fb278003520 con 0x7fb28c15f7b0 2023-08-09T13:50:26.326 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:26.324+0000 7fb289d9b640 1 -- 172.21.15.43:0/582580146 <== mon.1 v1:172.21.15.203:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3274402695 0 0) 0x7fb28c0b7ed0 con 0x7fb28c0cd6b0 2023-08-09T13:50:26.327 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:26.324+0000 7fb289d9b640 1 -- 172.21.15.43:0/582580146 --> v1:172.21.15.203:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fb28c0b7bf0 con 0x7fb28c0cd6b0 2023-08-09T13:50:26.327 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:26.324+0000 7fb289d9b640 1 -- 172.21.15.43:0/582580146 <== mon.0 v1:172.21.15.43:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1807876725 0 0) 0x7fb278003520 con 0x7fb28c15f7b0 2023-08-09T13:50:26.327 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:26.324+0000 7fb283fff640 1 -- 172.21.15.43:0/582580146 >> v1:172.21.15.203:6789/0 conn(0x7fb28c0cd6b0 legacy=0x7fb28c0d0e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:50:26.327 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:26.324+0000 7fb283fff640 1 -- 172.21.15.43:0/582580146 >> v1:172.21.15.43:6789/0 conn(0x7fb28c15f7b0 legacy=0x7fb28c0ccfa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:50:26.327 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:26.324+0000 7fb283fff640 1 -- 172.21.15.43:0/582580146 --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb28c0d0650 con 0x7fb28c0b1050 2023-08-09T13:50:26.328 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:26.325+0000 7fb283fff640 1 --2- 172.21.15.43:0/582580146 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fb28c0b1050 0x7fb28c0b4760 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7fb2740097d0 tx=0x7fb27402b800 comp rx=0 tx=0).ready entity=mon.2 client_cookie=90f27db67c6f1a7a server_cookie=0 in_seq=0 out_seq=0 2023-08-09T13:50:26.329 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:26.326+0000 7fb289d9b640 1 -- 172.21.15.43:0/582580146 <== mon.2 v2:172.21.15.203:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb27c001670 con 0x7fb28c0b1050 2023-08-09T13:50:26.329 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:26.326+0000 7fb289d9b640 1 -- 172.21.15.43:0/582580146 <== mon.2 v2:172.21.15.203:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb27402e050 con 0x7fb28c0b1050 2023-08-09T13:50:26.329 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:26.326+0000 7fb289d9b640 1 -- 172.21.15.43:0/582580146 <== mon.2 v2:172.21.15.203:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb274030710 con 0x7fb28c0b1050 2023-08-09T13:50:26.329 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:26.327+0000 7fb28b59e640 1 -- 172.21.15.43:0/582580146 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fb28c0b1050 msgr2=0x7fb28c0b4760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:50:26.329 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:26.327+0000 7fb28b59e640 1 --2- 172.21.15.43:0/582580146 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fb28c0b1050 0x7fb28c0b4760 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7fb2740097d0 tx=0x7fb27402b800 comp rx=0 tx=0).stop 2023-08-09T13:50:26.329 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:26.327+0000 7fb28b59e640 1 -- 172.21.15.43:0/582580146 shutdown_connections 2023-08-09T13:50:26.330 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:26.327+0000 7fb28b59e640 1 --2- 172.21.15.43:0/582580146 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fb28c0b1050 0x7fb28c0b4760 unknown :-1 s=CLOSED pgs=133 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:50:26.330 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:26.327+0000 7fb28b59e640 1 -- 172.21.15.43:0/582580146 >> 172.21.15.43:0/582580146 conn(0x7fb28c06a2b0 msgr2=0x7fb28c06a690 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-09T13:50:26.330 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:26.327+0000 7fb28b59e640 1 -- 172.21.15.43:0/582580146 shutdown_connections 2023-08-09T13:50:26.330 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:26.327+0000 7fb28b59e640 1 -- 172.21.15.43:0/582580146 wait complete. 2023-08-09T13:50:26.331 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:26.329+0000 7fb28b59e640 1 Processor -- start 2023-08-09T13:50:26.331 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:26.329+0000 7fb28b59e640 1 -- start start 2023-08-09T13:50:26.333 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:26.330+0000 7fb28b59e640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fb28c0b1050 0x7fb28c153710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:50:26.333 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:26.330+0000 7fb28b59e640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fb28c0cd6b0 0x7fb28c0c8480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:50:26.334 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:26.330+0000 7fb28b59e640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb28c15f7b0 0x7fb28c1f4fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:50:26.334 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:26.330+0000 7fb28b59e640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb28c0d1810 con 0x7fb28c15f7b0 2023-08-09T13:50:26.334 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:26.330+0000 7fb28b59e640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb28c153c50 con 0x7fb28c0cd6b0 2023-08-09T13:50:26.334 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:26.330+0000 7fb28b59e640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb28c0d2610 con 0x7fb28c0b1050 2023-08-09T13:50:26.334 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:26.330+0000 7fb28ad9d640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb28c15f7b0 0x7fb28c1f4fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:50:26.335 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:26.330+0000 7fb28ad9d640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb28c15f7b0 0x7fb28c1f4fe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:35618/0 (socket says 172.21.15.43:35618) 2023-08-09T13:50:26.335 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:26.330+0000 7fb28ad9d640 1 -- 172.21.15.43:0/214512807 learned_addr learned my addr 172.21.15.43:0/214512807 (peer_addr_for_me v2:172.21.15.43:0/0) 2023-08-09T13:50:26.335 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:26.331+0000 7fb28a59c640 1 --2- 172.21.15.43:0/214512807 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fb28c0b1050 0x7fb28c153710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:50:26.335 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:26.331+0000 7fb283fff640 1 --2- 172.21.15.43:0/214512807 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fb28c0cd6b0 0x7fb28c0c8480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:50:26.335 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:26.331+0000 7fb28ad9d640 1 -- 172.21.15.43:0/214512807 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fb28c0b1050 msgr2=0x7fb28c153710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:50:26.335 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:26.331+0000 7fb28ad9d640 1 --2- 172.21.15.43:0/214512807 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fb28c0b1050 0x7fb28c153710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:50:26.335 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:26.331+0000 7fb28ad9d640 1 -- 172.21.15.43:0/214512807 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fb28c0cd6b0 msgr2=0x7fb28c0c8480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:50:26.336 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:26.331+0000 7fb28ad9d640 1 --2- 172.21.15.43:0/214512807 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fb28c0cd6b0 0x7fb28c0c8480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:50:26.336 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:26.331+0000 7fb28ad9d640 1 -- 172.21.15.43:0/214512807 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb274009400 con 0x7fb28c15f7b0 2023-08-09T13:50:26.336 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:26.331+0000 7fb28a59c640 1 --2- 172.21.15.43:0/214512807 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fb28c0b1050 0x7fb28c153710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-08-09T13:50:26.336 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:26.331+0000 7fb283fff640 1 --2- 172.21.15.43:0/214512807 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fb28c0cd6b0 0x7fb28c0c8480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-08-09T13:50:26.336 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:26.331+0000 7fb28ad9d640 1 --2- 172.21.15.43:0/214512807 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb28c15f7b0 0x7fb28c1f4fe0 secure :-1 s=READY pgs=996 cs=0 l=1 rev1=1 crypto rx=0x7fb28400c7c0 tx=0x7fb28400cc80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-09T13:50:26.336 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:26.331+0000 7fb282ffd640 1 -- 172.21.15.43:0/214512807 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb284003b70 con 0x7fb28c15f7b0 2023-08-09T13:50:26.336 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:26.331+0000 7fb282ffd640 1 -- 172.21.15.43:0/214512807 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb284003cd0 con 0x7fb28c15f7b0 2023-08-09T13:50:26.336 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:26.331+0000 7fb282ffd640 1 -- 172.21.15.43:0/214512807 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb28400aa50 con 0x7fb28c15f7b0 2023-08-09T13:50:26.337 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:26.331+0000 7fb28b59e640 1 -- 172.21.15.43:0/214512807 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb28c0d1a00 con 0x7fb28c15f7b0 2023-08-09T13:50:26.337 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:26.332+0000 7fb28b59e640 1 -- 172.21.15.43:0/214512807 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb28c0d1fc0 con 0x7fb28c15f7b0 2023-08-09T13:50:26.337 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:26.333+0000 7fb28b59e640 1 -- 172.21.15.43:0/214512807 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb28c0cfe90 con 0x7fb28c15f7b0 2023-08-09T13:50:26.338 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:26.334+0000 7fb282ffd640 1 -- 172.21.15.43:0/214512807 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 29) v1 ==== 88443+0+0 (secure 0 0 0) 0x7fb28400abb0 con 0x7fb28c15f7b0 2023-08-09T13:50:26.338 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:26.335+0000 7fb282ffd640 1 --2- 172.21.15.43:0/214512807 >> [v2:172.21.15.203:6832/3325141227,v1:172.21.15.203:6833/3325141227] conn(0x7fb26c06a170 0x7fb26c06c630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:50:26.338 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:26.335+0000 7fb282ffd640 1 -- 172.21.15.43:0/214512807 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 5574+0+0 (secure 0 0 0) 0x7fb2840884a0 con 0x7fb28c15f7b0 2023-08-09T13:50:26.338 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:26.336+0000 7fb28a59c640 1 --2- 172.21.15.43:0/214512807 >> [v2:172.21.15.203:6832/3325141227,v1:172.21.15.203:6833/3325141227] conn(0x7fb26c06a170 0x7fb26c06c630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:50:26.338 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:26.337+0000 7fb28a59c640 1 --2- 172.21.15.43:0/214512807 >> [v2:172.21.15.203:6832/3325141227,v1:172.21.15.203:6833/3325141227] conn(0x7fb26c06a170 0x7fb26c06c630 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7fb27c0011e0 tx=0x7fb27c009040 comp rx=0 tx=0).ready entity=mgr.5519 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-09T13:50:26.340 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:26.339+0000 7fb282ffd640 1 -- 172.21.15.43:0/214512807 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178066 (secure 0 0 0) 0x7fb284053bc0 con 0x7fb28c15f7b0 2023-08-09T13:50:26.544 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:26.542+0000 7fb28b59e640 1 -- 172.21.15.43:0/214512807 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) v1 -- 0x7fb28c0d2780 con 0x7fb28c15f7b0 2023-08-09T13:50:26.545 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:26.543+0000 7fb282ffd640 1 -- 172.21.15.43:0/214512807 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v86) v1 ==== 84+0+126 (secure 0 0 0) 0x7fb28400ae60 con 0x7fb28c15f7b0 2023-08-09T13:50:26.545 INFO:teuthology.orchestra.run.smithi043.stdout: 2023-08-09T13:50:26.546 INFO:teuthology.orchestra.run.smithi043.stdout:[ 2023-08-09T13:50:26.546 INFO:teuthology.orchestra.run.smithi043.stdout: { 2023-08-09T13:50:26.546 INFO:teuthology.orchestra.run.smithi043.stdout: "poolnum": 1, 2023-08-09T13:50:26.546 INFO:teuthology.orchestra.run.smithi043.stdout: "poolname": "rbd" 2023-08-09T13:50:26.546 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2023-08-09T13:50:26.546 INFO:teuthology.orchestra.run.smithi043.stdout: { 2023-08-09T13:50:26.547 INFO:teuthology.orchestra.run.smithi043.stdout: "poolnum": 2, 2023-08-09T13:50:26.547 INFO:teuthology.orchestra.run.smithi043.stdout: "poolname": ".mgr" 2023-08-09T13:50:26.547 INFO:teuthology.orchestra.run.smithi043.stdout: } 2023-08-09T13:50:26.547 INFO:teuthology.orchestra.run.smithi043.stdout:] 2023-08-09T13:50:26.548 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:26.546+0000 7fb28b59e640 1 -- 172.21.15.43:0/214512807 >> [v2:172.21.15.203:6832/3325141227,v1:172.21.15.203:6833/3325141227] conn(0x7fb26c06a170 msgr2=0x7fb26c06c630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:50:26.548 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:26.546+0000 7fb28b59e640 1 --2- 172.21.15.43:0/214512807 >> [v2:172.21.15.203:6832/3325141227,v1:172.21.15.203:6833/3325141227] conn(0x7fb26c06a170 0x7fb26c06c630 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7fb27c0011e0 tx=0x7fb27c009040 comp rx=0 tx=0).stop 2023-08-09T13:50:26.548 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:26.546+0000 7fb28b59e640 1 -- 172.21.15.43:0/214512807 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb28c15f7b0 msgr2=0x7fb28c1f4fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:50:26.548 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:26.546+0000 7fb28b59e640 1 --2- 172.21.15.43:0/214512807 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb28c15f7b0 0x7fb28c1f4fe0 secure :-1 s=READY pgs=996 cs=0 l=1 rev1=1 crypto rx=0x7fb28400c7c0 tx=0x7fb28400cc80 comp rx=0 tx=0).stop 2023-08-09T13:50:26.549 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:26.546+0000 7fb28b59e640 1 -- 172.21.15.43:0/214512807 shutdown_connections 2023-08-09T13:50:26.549 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:26.546+0000 7fb28b59e640 1 --2- 172.21.15.43:0/214512807 >> [v2:172.21.15.203:6832/3325141227,v1:172.21.15.203:6833/3325141227] conn(0x7fb26c06a170 0x7fb26c06c630 unknown :-1 s=CLOSED pgs=55 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:50:26.549 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:26.546+0000 7fb28b59e640 1 --2- 172.21.15.43:0/214512807 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb28c15f7b0 0x7fb28c1f4fe0 unknown :-1 s=CLOSED pgs=996 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:50:26.549 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:26.546+0000 7fb28b59e640 1 --2- 172.21.15.43:0/214512807 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fb28c0cd6b0 0x7fb28c0c8480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:50:26.549 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:26.546+0000 7fb28b59e640 1 --2- 172.21.15.43:0/214512807 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fb28c0b1050 0x7fb28c153710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:50:26.549 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:26.546+0000 7fb28b59e640 1 -- 172.21.15.43:0/214512807 >> 172.21.15.43:0/214512807 conn(0x7fb28c06a2b0 msgr2=0x7fb28c15ea00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-09T13:50:26.549 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:26.546+0000 7fb28b59e640 1 -- 172.21.15.43:0/214512807 shutdown_connections 2023-08-09T13:50:26.550 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:26.546+0000 7fb28b59e640 1 -- 172.21.15.43:0/214512807 wait complete. 2023-08-09T13:50:26.566 DEBUG:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2023-08-09T13:50:26.566 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_metadata --pg_num_min 64 2023-08-09T13:50:26.778 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:26.777+0000 7fa7c4586640 1 Processor -- start 2023-08-09T13:50:26.779 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:26.777+0000 7fa7c4586640 1 -- start start 2023-08-09T13:50:26.781 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:26.779+0000 7fa7c4586640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fa7bc0cc240 0x7fa7bc06a8d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:50:26.781 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:26.779+0000 7fa7c4586640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7fa7bc160830 0x7fa7bc0ca910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:50:26.781 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:26.779+0000 7fa7c4586640 1 -- --> v1:172.21.15.203:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa7bc0cf0d0 con 0x7fa7bc160d30 2023-08-09T13:50:26.781 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:26.779+0000 7fa7c4586640 1 -- --> v2:172.21.15.43:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa7bc0ce7a0 con 0x7fa7bc160830 2023-08-09T13:50:26.782 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:26.779+0000 7fa7c4586640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa7bc0cea80 con 0x7fa7bc0cc240 2023-08-09T13:50:26.782 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:26.779+0000 7fa7c2b23640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7fa7bc160830 0x7fa7bc0ca910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:50:26.782 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:26.779+0000 7fa7c2b23640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7fa7bc160830 0x7fa7bc0ca910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:35624/0 (socket says 172.21.15.43:35624) 2023-08-09T13:50:26.782 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:26.779+0000 7fa7c2b23640 1 -- 172.21.15.43:0/3116839126 learned_addr learned my addr 172.21.15.43:0/3116839126 (peer_addr_for_me v2:172.21.15.43:0/0) 2023-08-09T13:50:26.782 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:26.779+0000 7fa7c2322640 1 --2- 172.21.15.43:0/3116839126 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fa7bc0cc240 0x7fa7bc06a8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:50:26.783 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:26.780+0000 7fa7c2b23640 1 -- 172.21.15.43:0/3116839126 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fa7bc0cc240 msgr2=0x7fa7bc06a8d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-09T13:50:26.783 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:26.780+0000 7fa7c2b23640 1 --2- 172.21.15.43:0/3116839126 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fa7bc0cc240 0x7fa7bc06a8d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:50:26.784 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:26.780+0000 7fa7c2b23640 1 -- 172.21.15.43:0/3116839126 >> v1:172.21.15.203:6789/0 conn(0x7fa7bc160d30 legacy=0x7fa7bc160120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-09T13:50:26.784 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:26.780+0000 7fa7c2b23640 1 -- 172.21.15.43:0/3116839126 --> v2:172.21.15.43:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa7bc0ced60 con 0x7fa7bc160830 2023-08-09T13:50:26.784 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:26.781+0000 7fa7c2b23640 1 --2- 172.21.15.43:0/3116839126 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa7bc160830 0x7fa7bc0ca910 secure :-1 s=READY pgs=997 cs=0 l=1 rev1=1 crypto rx=0x7fa7b4009e50 tx=0x7fa7b402bed0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=416f49e05cfa4477 server_cookie=0 in_seq=0 out_seq=0 2023-08-09T13:50:26.784 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:26.781+0000 7fa7c1320640 1 -- 172.21.15.43:0/3116839126 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa7b402ca70 con 0x7fa7bc160830 2023-08-09T13:50:26.784 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:26.781+0000 7fa7c1320640 1 -- 172.21.15.43:0/3116839126 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa7b402cbd0 con 0x7fa7bc160830 2023-08-09T13:50:26.785 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:26.781+0000 7fa7c1320640 1 -- 172.21.15.43:0/3116839126 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa7b40343f0 con 0x7fa7bc160830 2023-08-09T13:50:26.785 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:26.782+0000 7fa7c4586640 1 -- 172.21.15.43:0/3116839126 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa7bc160830 msgr2=0x7fa7bc0ca910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:50:26.785 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:26.782+0000 7fa7c4586640 1 --2- 172.21.15.43:0/3116839126 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa7bc160830 0x7fa7bc0ca910 secure :-1 s=READY pgs=997 cs=0 l=1 rev1=1 crypto rx=0x7fa7b4009e50 tx=0x7fa7b402bed0 comp rx=0 tx=0).stop 2023-08-09T13:50:26.786 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:26.782+0000 7fa7c4586640 1 -- 172.21.15.43:0/3116839126 shutdown_connections 2023-08-09T13:50:26.786 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:26.782+0000 7fa7c4586640 1 --2- 172.21.15.43:0/3116839126 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa7bc160830 0x7fa7bc0ca910 unknown :-1 s=CLOSED pgs=997 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:50:26.786 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:26.782+0000 7fa7c4586640 1 --2- 172.21.15.43:0/3116839126 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fa7bc0cc240 0x7fa7bc06a8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:50:26.787 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:26.782+0000 7fa7c4586640 1 -- 172.21.15.43:0/3116839126 >> 172.21.15.43:0/3116839126 conn(0x7fa7bc099600 msgr2=0x7fa7bc0d02c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-09T13:50:26.787 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:26.782+0000 7fa7c4586640 1 -- 172.21.15.43:0/3116839126 shutdown_connections 2023-08-09T13:50:26.787 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:26.782+0000 7fa7c4586640 1 -- 172.21.15.43:0/3116839126 wait complete. 2023-08-09T13:50:26.787 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:26.784+0000 7fa7c4586640 1 Processor -- start 2023-08-09T13:50:26.787 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:26.784+0000 7fa7c4586640 1 -- start start 2023-08-09T13:50:26.789 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:26.785+0000 7fa7c4586640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fa7bc0cc240 0x7fa7bc1f3280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:50:26.789 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:26.785+0000 7fa7c4586640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fa7bc160830 0x7fa7bc0c5860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:50:26.789 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:26.785+0000 7fa7c4586640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa7bc160d30 0x7fa7bc1f1800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:50:26.789 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:26.785+0000 7fa7c4586640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa7bc0c4d30 con 0x7fa7bc160d30 2023-08-09T13:50:26.789 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:26.785+0000 7fa7c4586640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa7bc15b0d0 con 0x7fa7bc0cc240 2023-08-09T13:50:26.789 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:26.785+0000 7fa7c4586640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa7bc1f1d40 con 0x7fa7bc160830 2023-08-09T13:50:26.790 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:26.786+0000 7fa7c2b23640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa7bc160d30 0x7fa7bc1f1800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:50:26.790 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:26.786+0000 7fa7c2b23640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa7bc160d30 0x7fa7bc1f1800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:35632/0 (socket says 172.21.15.43:35632) 2023-08-09T13:50:26.790 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:26.786+0000 7fa7c2b23640 1 -- 172.21.15.43:0/3772571295 learned_addr learned my addr 172.21.15.43:0/3772571295 (peer_addr_for_me v2:172.21.15.43:0/0) 2023-08-09T13:50:26.791 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:26.786+0000 7fa7c2322640 1 --2- 172.21.15.43:0/3772571295 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fa7bc0cc240 0x7fa7bc1f3280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:50:26.791 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:26.786+0000 7fa7c1b21640 1 --2- 172.21.15.43:0/3772571295 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fa7bc160830 0x7fa7bc0c5860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:50:26.791 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:26.786+0000 7fa7c2b23640 1 -- 172.21.15.43:0/3772571295 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fa7bc160830 msgr2=0x7fa7bc0c5860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:50:26.791 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:26.786+0000 7fa7c2b23640 1 --2- 172.21.15.43:0/3772571295 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fa7bc160830 0x7fa7bc0c5860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:50:26.792 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:26.786+0000 7fa7c2b23640 1 -- 172.21.15.43:0/3772571295 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fa7bc0cc240 msgr2=0x7fa7bc1f3280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:50:26.792 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:26.786+0000 7fa7c2b23640 1 --2- 172.21.15.43:0/3772571295 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fa7bc0cc240 0x7fa7bc1f3280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:50:26.792 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:26.786+0000 7fa7c2b23640 1 -- 172.21.15.43:0/3772571295 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa7b4009b30 con 0x7fa7bc160d30 2023-08-09T13:50:26.792 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:26.786+0000 7fa7c1b21640 1 --2- 172.21.15.43:0/3772571295 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fa7bc160830 0x7fa7bc0c5860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-08-09T13:50:26.792 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:26.786+0000 7fa7c2322640 1 --2- 172.21.15.43:0/3772571295 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fa7bc0cc240 0x7fa7bc1f3280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-08-09T13:50:26.792 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:26.787+0000 7fa7c2b23640 1 --2- 172.21.15.43:0/3772571295 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa7bc160d30 0x7fa7bc1f1800 secure :-1 s=READY pgs=998 cs=0 l=1 rev1=1 crypto rx=0x7fa7b4009e50 tx=0x7fa7b4004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-09T13:50:26.792 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:26.787+0000 7fa7baffd640 1 -- 172.21.15.43:0/3772571295 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa7b4009510 con 0x7fa7bc160d30 2023-08-09T13:50:26.793 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:26.787+0000 7fa7baffd640 1 -- 172.21.15.43:0/3772571295 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa7b403e450 con 0x7fa7bc160d30 2023-08-09T13:50:26.793 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:26.787+0000 7fa7baffd640 1 -- 172.21.15.43:0/3772571295 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa7b403e800 con 0x7fa7bc160d30 2023-08-09T13:50:26.793 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:26.787+0000 7fa7c4586640 1 -- 172.21.15.43:0/3772571295 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa7bc0cc7a0 con 0x7fa7bc160d30 2023-08-09T13:50:26.793 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:26.787+0000 7fa7c4586640 1 -- 172.21.15.43:0/3772571295 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa7bc0c4490 con 0x7fa7bc160d30 2023-08-09T13:50:26.794 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:26.788+0000 7fa7c4586640 1 -- 172.21.15.43:0/3772571295 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa7bc1f1f00 con 0x7fa7bc160d30 2023-08-09T13:50:26.794 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:26.789+0000 7fa7baffd640 1 -- 172.21.15.43:0/3772571295 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 29) v1 ==== 88443+0+0 (secure 0 0 0) 0x7fa7b403e960 con 0x7fa7bc160d30 2023-08-09T13:50:26.794 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:26.790+0000 7fa7baffd640 1 --2- 172.21.15.43:0/3772571295 >> [v2:172.21.15.203:6832/3325141227,v1:172.21.15.203:6833/3325141227] conn(0x7fa7a806a170 0x7fa7a806c630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:50:26.794 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:26.790+0000 7fa7baffd640 1 -- 172.21.15.43:0/3772571295 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 5574+0+0 (secure 0 0 0) 0x7fa7b40ab2c0 con 0x7fa7bc160d30 2023-08-09T13:50:26.795 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:26.791+0000 7fa7c2322640 1 --2- 172.21.15.43:0/3772571295 >> [v2:172.21.15.203:6832/3325141227,v1:172.21.15.203:6833/3325141227] conn(0x7fa7a806a170 0x7fa7a806c630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:50:26.795 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:26.792+0000 7fa7c2322640 1 --2- 172.21.15.43:0/3772571295 >> [v2:172.21.15.203:6832/3325141227,v1:172.21.15.203:6833/3325141227] conn(0x7fa7a806a170 0x7fa7a806c630 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7fa7ac009820 tx=0x7fa7ac008040 comp rx=0 tx=0).ready entity=mgr.5519 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-09T13:50:26.796 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:26.794+0000 7fa7baffd640 1 -- 172.21.15.43:0/3772571295 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178066 (secure 0 0 0) 0x7fa7bc1f1f00 con 0x7fa7bc160d30 2023-08-09T13:50:26.993 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:26.992+0000 7fa7c4586640 1 -- 172.21.15.43:0/3772571295 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64} v 0) v1 -- 0x7fa7bc1f1f00 con 0x7fa7bc160d30 2023-08-09T13:50:27.158 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:27.156+0000 7fa7baffd640 1 -- 172.21.15.43:0/3772571295 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]=0 pool 'cephfs_metadata' created v87) v1 ==== 138+0+0 (secure 0 0 0) 0x7fa7bc1f1f00 con 0x7fa7bc160d30 2023-08-09T13:50:27.159 INFO:teuthology.orchestra.run.smithi043.stderr:pool 'cephfs_metadata' created 2023-08-09T13:50:27.164 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:27.163+0000 7fa7c4586640 1 -- 172.21.15.43:0/3772571295 >> [v2:172.21.15.203:6832/3325141227,v1:172.21.15.203:6833/3325141227] conn(0x7fa7a806a170 msgr2=0x7fa7a806c630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:50:27.165 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:27.163+0000 7fa7c4586640 1 --2- 172.21.15.43:0/3772571295 >> [v2:172.21.15.203:6832/3325141227,v1:172.21.15.203:6833/3325141227] conn(0x7fa7a806a170 0x7fa7a806c630 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7fa7ac009820 tx=0x7fa7ac008040 comp rx=0 tx=0).stop 2023-08-09T13:50:27.165 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:27.163+0000 7fa7c4586640 1 -- 172.21.15.43:0/3772571295 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa7bc160d30 msgr2=0x7fa7bc1f1800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:50:27.165 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:27.163+0000 7fa7c4586640 1 --2- 172.21.15.43:0/3772571295 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa7bc160d30 0x7fa7bc1f1800 secure :-1 s=READY pgs=998 cs=0 l=1 rev1=1 crypto rx=0x7fa7b4009e50 tx=0x7fa7b4004a40 comp rx=0 tx=0).stop 2023-08-09T13:50:27.165 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:27.163+0000 7fa7c4586640 1 -- 172.21.15.43:0/3772571295 shutdown_connections 2023-08-09T13:50:27.166 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:27.163+0000 7fa7c4586640 1 --2- 172.21.15.43:0/3772571295 >> [v2:172.21.15.203:6832/3325141227,v1:172.21.15.203:6833/3325141227] conn(0x7fa7a806a170 0x7fa7a806c630 unknown :-1 s=CLOSED pgs=56 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:50:27.166 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:27.163+0000 7fa7c4586640 1 --2- 172.21.15.43:0/3772571295 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa7bc160d30 0x7fa7bc1f1800 unknown :-1 s=CLOSED pgs=998 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:50:27.166 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:27.163+0000 7fa7c4586640 1 --2- 172.21.15.43:0/3772571295 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fa7bc160830 0x7fa7bc0c5860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:50:27.166 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:27.163+0000 7fa7c4586640 1 --2- 172.21.15.43:0/3772571295 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fa7bc0cc240 0x7fa7bc1f3280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:50:27.167 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:27.163+0000 7fa7c4586640 1 -- 172.21.15.43:0/3772571295 >> 172.21.15.43:0/3772571295 conn(0x7fa7bc099600 msgr2=0x7fa7bc0cb4e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-09T13:50:27.167 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:27.165+0000 7fa7c4586640 1 -- 172.21.15.43:0/3772571295 shutdown_connections 2023-08-09T13:50:27.168 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:27.165+0000 7fa7c4586640 1 -- 172.21.15.43:0/3772571295 wait complete. 2023-08-09T13:50:27.190 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data 64 --pg_num_min 64 --target_size_ratio 0.9 2023-08-09T13:50:27.425 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:27.423+0000 7fdf76947640 1 Processor -- start 2023-08-09T13:50:27.425 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:27.423+0000 7fdf76947640 1 -- start start 2023-08-09T13:50:27.426 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:27.425+0000 7fdf76947640 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7fdf7015dcb0 0x7fdf700d0970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:50:27.426 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:27.425+0000 7fdf76947640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7fdf700cef00 0x7fdf700cbc60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:50:27.427 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:27.425+0000 7fdf76947640 1 -- --> v1:172.21.15.203:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fdf700d0fc0 con 0x7fdf700cc1a0 2023-08-09T13:50:27.427 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:27.425+0000 7fdf76947640 1 -- --> v2:172.21.15.43:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fdf7015e090 con 0x7fdf700cef00 2023-08-09T13:50:27.427 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:27.425+0000 7fdf76947640 1 -- --> v2:172.21.15.203:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fdf700cf380 con 0x7fdf7015dcb0 2023-08-09T13:50:27.428 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:27.425+0000 7fdf677fe640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7fdf700cef00 0x7fdf700cbc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:50:27.428 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:27.425+0000 7fdf677fe640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7fdf700cef00 0x7fdf700cbc60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:35634/0 (socket says 172.21.15.43:35634) 2023-08-09T13:50:27.428 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:27.425+0000 7fdf677fe640 1 -- 172.21.15.43:0/3418285045 learned_addr learned my addr 172.21.15.43:0/3418285045 (peer_addr_for_me v2:172.21.15.43:0/0) 2023-08-09T13:50:27.428 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:27.426+0000 7fdf6ffff640 1 --2- 172.21.15.43:0/3418285045 >> v2:172.21.15.203:3300/0 conn(0x7fdf7015dcb0 0x7fdf700d0970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:50:27.428 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:27.426+0000 7fdf677fe640 1 -- 172.21.15.43:0/3418285045 >> v2:172.21.15.203:3300/0 conn(0x7fdf7015dcb0 msgr2=0x7fdf700d0970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-09T13:50:27.428 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:27.426+0000 7fdf677fe640 1 --2- 172.21.15.43:0/3418285045 >> v2:172.21.15.203:3300/0 conn(0x7fdf7015dcb0 0x7fdf700d0970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:50:27.429 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:27.426+0000 7fdf677fe640 1 -- 172.21.15.43:0/3418285045 >> v1:172.21.15.203:6789/0 conn(0x7fdf700cc1a0 legacy=0x7fdf700afb40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-09T13:50:27.429 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:27.426+0000 7fdf677fe640 1 -- 172.21.15.43:0/3418285045 --> v2:172.21.15.43:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdf700cb180 con 0x7fdf700cef00 2023-08-09T13:50:27.429 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:27.426+0000 7fdf6ffff640 1 --2- 172.21.15.43:0/3418285045 >> v2:172.21.15.203:3300/0 conn(0x7fdf7015dcb0 0x7fdf700d0970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-08-09T13:50:27.430 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:27.427+0000 7fdf677fe640 1 --2- 172.21.15.43:0/3418285045 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fdf700cef00 0x7fdf700cbc60 secure :-1 s=READY pgs=999 cs=0 l=1 rev1=1 crypto rx=0x7fdf70070f90 tx=0x7fdf5802b880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5a16a1d39f5cacf server_cookie=0 in_seq=0 out_seq=0 2023-08-09T13:50:27.430 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:27.428+0000 7fdf6f7fe640 1 -- 172.21.15.43:0/3418285045 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdf580317e0 con 0x7fdf700cef00 2023-08-09T13:50:27.430 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:27.428+0000 7fdf6f7fe640 1 -- 172.21.15.43:0/3418285045 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdf58031940 con 0x7fdf700cef00 2023-08-09T13:50:27.430 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:27.428+0000 7fdf6f7fe640 1 -- 172.21.15.43:0/3418285045 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdf58031cd0 con 0x7fdf700cef00 2023-08-09T13:50:27.431 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:27.428+0000 7fdf76947640 1 -- 172.21.15.43:0/3418285045 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fdf700cef00 msgr2=0x7fdf700cbc60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:50:27.431 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:27.428+0000 7fdf76947640 1 --2- 172.21.15.43:0/3418285045 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fdf700cef00 0x7fdf700cbc60 secure :-1 s=READY pgs=999 cs=0 l=1 rev1=1 crypto rx=0x7fdf70070f90 tx=0x7fdf5802b880 comp rx=0 tx=0).stop 2023-08-09T13:50:27.431 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:27.429+0000 7fdf76947640 1 -- 172.21.15.43:0/3418285045 shutdown_connections 2023-08-09T13:50:27.431 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:27.429+0000 7fdf76947640 1 --2- 172.21.15.43:0/3418285045 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fdf700cef00 0x7fdf700cbc60 unknown :-1 s=CLOSED pgs=999 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:50:27.431 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:27.429+0000 7fdf76947640 1 --2- 172.21.15.43:0/3418285045 >> v2:172.21.15.203:3300/0 conn(0x7fdf7015dcb0 0x7fdf700d0970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:50:27.431 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:27.429+0000 7fdf76947640 1 -- 172.21.15.43:0/3418285045 >> 172.21.15.43:0/3418285045 conn(0x7fdf70089fe0 msgr2=0x7fdf700d0260 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-09T13:50:27.431 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:27.429+0000 7fdf76947640 1 -- 172.21.15.43:0/3418285045 shutdown_connections 2023-08-09T13:50:27.432 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:27.429+0000 7fdf76947640 1 -- 172.21.15.43:0/3418285045 wait complete. 2023-08-09T13:50:27.432 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:27.431+0000 7fdf76947640 1 Processor -- start 2023-08-09T13:50:27.432 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:27.431+0000 7fdf76947640 1 -- start start 2023-08-09T13:50:27.433 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:27.432+0000 7fdf76947640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fdf700cc1a0 0x7fdf700b0da0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:50:27.433 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:27.432+0000 7fdf76947640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fdf700cef00 0x7fdf70158050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:50:27.434 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:27.432+0000 7fdf76947640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fdf7015dcb0 0x7fdf700b4ec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:50:27.434 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:27.432+0000 7fdf76947640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdf70158590 con 0x7fdf700cef00 2023-08-09T13:50:27.434 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:27.432+0000 7fdf76947640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdf700b4410 con 0x7fdf7015dcb0 2023-08-09T13:50:27.434 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:27.432+0000 7fdf76947640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdf700b3b70 con 0x7fdf700cc1a0 2023-08-09T13:50:27.434 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:27.432+0000 7fdf677fe640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fdf700cef00 0x7fdf70158050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:50:27.434 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:27.432+0000 7fdf677fe640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fdf700cef00 0x7fdf70158050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:35650/0 (socket says 172.21.15.43:35650) 2023-08-09T13:50:27.434 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:27.432+0000 7fdf677fe640 1 -- 172.21.15.43:0/2583882699 learned_addr learned my addr 172.21.15.43:0/2583882699 (peer_addr_for_me v2:172.21.15.43:0/0) 2023-08-09T13:50:27.436 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:27.433+0000 7fdf74ee4640 1 --2- 172.21.15.43:0/2583882699 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fdf7015dcb0 0x7fdf700b4ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:50:27.436 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:27.433+0000 7fdf6ffff640 1 --2- 172.21.15.43:0/2583882699 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fdf700cc1a0 0x7fdf700b0da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:50:27.436 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:27.433+0000 7fdf677fe640 1 -- 172.21.15.43:0/2583882699 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fdf700cc1a0 msgr2=0x7fdf700b0da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:50:27.436 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:27.433+0000 7fdf677fe640 1 --2- 172.21.15.43:0/2583882699 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fdf700cc1a0 0x7fdf700b0da0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:50:27.436 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:27.433+0000 7fdf677fe640 1 -- 172.21.15.43:0/2583882699 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fdf7015dcb0 msgr2=0x7fdf700b4ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:50:27.436 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:27.433+0000 7fdf677fe640 1 --2- 172.21.15.43:0/2583882699 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fdf7015dcb0 0x7fdf700b4ec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:50:27.437 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:27.433+0000 7fdf677fe640 1 -- 172.21.15.43:0/2583882699 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdf580094d0 con 0x7fdf700cef00 2023-08-09T13:50:27.437 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:27.433+0000 7fdf74ee4640 1 --2- 172.21.15.43:0/2583882699 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fdf7015dcb0 0x7fdf700b4ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-08-09T13:50:27.437 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:27.433+0000 7fdf677fe640 1 --2- 172.21.15.43:0/2583882699 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fdf700cef00 0x7fdf70158050 secure :-1 s=READY pgs=1000 cs=0 l=1 rev1=1 crypto rx=0x7fdf58000c00 tx=0x7fdf580046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-09T13:50:27.437 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:27.433+0000 7fdf6d7fa640 1 -- 172.21.15.43:0/2583882699 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdf58032900 con 0x7fdf700cef00 2023-08-09T13:50:27.437 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:27.433+0000 7fdf6d7fa640 1 -- 172.21.15.43:0/2583882699 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdf58032a60 con 0x7fdf700cef00 2023-08-09T13:50:27.437 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:27.434+0000 7fdf6d7fa640 1 -- 172.21.15.43:0/2583882699 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdf58032e10 con 0x7fdf700cef00 2023-08-09T13:50:27.437 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:27.434+0000 7fdf76947640 1 -- 172.21.15.43:0/2583882699 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdf700b3e50 con 0x7fdf700cef00 2023-08-09T13:50:27.437 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:27.434+0000 7fdf76947640 1 -- 172.21.15.43:0/2583882699 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdf7015b3d0 con 0x7fdf700cef00 2023-08-09T13:50:27.438 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:27.435+0000 7fdf76947640 1 -- 172.21.15.43:0/2583882699 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdf7015b5c0 con 0x7fdf700cef00 2023-08-09T13:50:27.438 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:27.436+0000 7fdf6d7fa640 1 -- 172.21.15.43:0/2583882699 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 29) v1 ==== 88443+0+0 (secure 0 0 0) 0x7fdf580453f0 con 0x7fdf700cef00 2023-08-09T13:50:27.439 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:27.437+0000 7fdf6d7fa640 1 --2- 172.21.15.43:0/2583882699 >> [v2:172.21.15.203:6832/3325141227,v1:172.21.15.203:6833/3325141227] conn(0x7fdf5006a170 0x7fdf5006c630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:50:27.439 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:27.437+0000 7fdf6d7fa640 1 -- 172.21.15.43:0/2583882699 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 5944+0+0 (secure 0 0 0) 0x7fdf580ab350 con 0x7fdf700cef00 2023-08-09T13:50:27.439 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:27.438+0000 7fdf6ffff640 1 --2- 172.21.15.43:0/2583882699 >> [v2:172.21.15.203:6832/3325141227,v1:172.21.15.203:6833/3325141227] conn(0x7fdf5006a170 0x7fdf5006c630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:50:27.440 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:27.439+0000 7fdf6ffff640 1 --2- 172.21.15.43:0/2583882699 >> [v2:172.21.15.203:6832/3325141227,v1:172.21.15.203:6833/3325141227] conn(0x7fdf5006a170 0x7fdf5006c630 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7fdf6000b840 tx=0x7fdf60008040 comp rx=0 tx=0).ready entity=mgr.5519 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-09T13:50:27.442 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:27.441+0000 7fdf6d7fa640 1 -- 172.21.15.43:0/2583882699 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178066 (secure 0 0 0) 0x7fdf58076900 con 0x7fdf700cef00 2023-08-09T13:50:27.636 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:27.635+0000 7fdf76947640 1 -- 172.21.15.43:0/2583882699 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9} v 0) v1 -- 0x7fdf700739f0 con 0x7fdf700cef00 2023-08-09T13:50:28.181 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:28.178+0000 7fdf6d7fa640 1 -- 172.21.15.43:0/2583882699 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]=0 pool 'cephfs_data' created v88) v1 ==== 170+0+0 (secure 0 0 0) 0x7fdf580530d0 con 0x7fdf700cef00 2023-08-09T13:50:28.187 INFO:teuthology.orchestra.run.smithi043.stderr:pool 'cephfs_data' created 2023-08-09T13:50:28.194 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:28.192+0000 7fdf76947640 1 -- 172.21.15.43:0/2583882699 >> [v2:172.21.15.203:6832/3325141227,v1:172.21.15.203:6833/3325141227] conn(0x7fdf5006a170 msgr2=0x7fdf5006c630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:50:28.194 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:28.192+0000 7fdf76947640 1 --2- 172.21.15.43:0/2583882699 >> [v2:172.21.15.203:6832/3325141227,v1:172.21.15.203:6833/3325141227] conn(0x7fdf5006a170 0x7fdf5006c630 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7fdf6000b840 tx=0x7fdf60008040 comp rx=0 tx=0).stop 2023-08-09T13:50:28.194 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:28.193+0000 7fdf76947640 1 -- 172.21.15.43:0/2583882699 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fdf700cef00 msgr2=0x7fdf70158050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:50:28.195 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:28.193+0000 7fdf76947640 1 --2- 172.21.15.43:0/2583882699 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fdf700cef00 0x7fdf70158050 secure :-1 s=READY pgs=1000 cs=0 l=1 rev1=1 crypto rx=0x7fdf58000c00 tx=0x7fdf580046c0 comp rx=0 tx=0).stop 2023-08-09T13:50:28.198 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:28.196+0000 7fdf76947640 1 -- 172.21.15.43:0/2583882699 shutdown_connections 2023-08-09T13:50:28.198 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:28.196+0000 7fdf76947640 1 --2- 172.21.15.43:0/2583882699 >> [v2:172.21.15.203:6832/3325141227,v1:172.21.15.203:6833/3325141227] conn(0x7fdf5006a170 0x7fdf5006c630 unknown :-1 s=CLOSED pgs=57 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:50:28.198 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:28.196+0000 7fdf76947640 1 --2- 172.21.15.43:0/2583882699 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fdf7015dcb0 0x7fdf700b4ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:50:28.198 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:28.196+0000 7fdf76947640 1 --2- 172.21.15.43:0/2583882699 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fdf700cef00 0x7fdf70158050 unknown :-1 s=CLOSED pgs=1000 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:50:28.199 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:28.196+0000 7fdf76947640 1 --2- 172.21.15.43:0/2583882699 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fdf700cc1a0 0x7fdf700b0da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:50:28.199 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:28.196+0000 7fdf76947640 1 -- 172.21.15.43:0/2583882699 >> 172.21.15.43:0/2583882699 conn(0x7fdf70089fe0 msgr2=0x7fdf700b2570 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-09T13:50:28.199 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:28.197+0000 7fdf76947640 1 -- 172.21.15.43:0/2583882699 shutdown_connections 2023-08-09T13:50:28.199 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:28.197+0000 7fdf76947640 1 -- 172.21.15.43:0/2583882699 wait complete. 2023-08-09T13:50:28.222 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs new cephfs cephfs_metadata cephfs_data 2023-08-09T13:50:28.421 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:28.420+0000 7f921de8a640 1 Processor -- start 2023-08-09T13:50:28.422 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:28.420+0000 7f921de8a640 1 -- start start 2023-08-09T13:50:28.424 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:28.421+0000 7f921de8a640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f92180c1020 0x7f9218162360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:50:28.424 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:28.421+0000 7f921de8a640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f9218164940 0x7f9218164df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:50:28.424 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:28.421+0000 7f921de8a640 1 -- --> v1:172.21.15.203:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f92180cc3b0 con 0x7f9218164560 2023-08-09T13:50:28.425 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:28.421+0000 7f921de8a640 1 -- --> v2:172.21.15.43:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f92180ca6d0 con 0x7f9218164940 2023-08-09T13:50:28.425 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:28.421+0000 7f921de8a640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f92180af2e0 con 0x7f92180c1020 2023-08-09T13:50:28.425 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:28.422+0000 7f921d689640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f9218164940 0x7f9218164df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:50:28.425 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:28.422+0000 7f921d689640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f9218164940 0x7f9218164df0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:35660/0 (socket says 172.21.15.43:35660) 2023-08-09T13:50:28.425 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:28.422+0000 7f921d689640 1 -- 172.21.15.43:0/721528278 learned_addr learned my addr 172.21.15.43:0/721528278 (peer_addr_for_me v2:172.21.15.43:0/0) 2023-08-09T13:50:28.425 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:28.422+0000 7f920ffff640 1 --2- 172.21.15.43:0/721528278 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f92180c1020 0x7f9218162360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:50:28.425 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:28.422+0000 7f921d689640 1 -- 172.21.15.43:0/721528278 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f92180c1020 msgr2=0x7f9218162360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-09T13:50:28.426 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:28.422+0000 7f921d689640 1 --2- 172.21.15.43:0/721528278 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f92180c1020 0x7f9218162360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:50:28.426 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:28.422+0000 7f921d689640 1 -- 172.21.15.43:0/721528278 >> v1:172.21.15.203:6789/0 conn(0x7f9218164560 legacy=0x7f92180c8420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-09T13:50:28.426 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:28.423+0000 7f921d689640 1 -- 172.21.15.43:0/721528278 --> v2:172.21.15.43:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f92180cd790 con 0x7f9218164940 2023-08-09T13:50:28.427 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:28.424+0000 7f921d689640 1 --2- 172.21.15.43:0/721528278 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f9218164940 0x7f9218164df0 secure :-1 s=READY pgs=1001 cs=0 l=1 rev1=1 crypto rx=0x7f9210009870 tx=0x7f921002b7d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=179229f91bbae6ce server_cookie=0 in_seq=0 out_seq=0 2023-08-09T13:50:28.427 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:28.424+0000 7f920f7fe640 1 -- 172.21.15.43:0/721528278 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9210033620 con 0x7f9218164940 2023-08-09T13:50:28.427 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:28.424+0000 7f920f7fe640 1 -- 172.21.15.43:0/721528278 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9210033780 con 0x7f9218164940 2023-08-09T13:50:28.428 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:28.424+0000 7f920f7fe640 1 -- 172.21.15.43:0/721528278 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9210033b10 con 0x7f9218164940 2023-08-09T13:50:28.428 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:28.425+0000 7f921de8a640 1 -- 172.21.15.43:0/721528278 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f9218164940 msgr2=0x7f9218164df0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:50:28.428 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:28.425+0000 7f921de8a640 1 --2- 172.21.15.43:0/721528278 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f9218164940 0x7f9218164df0 secure :-1 s=READY pgs=1001 cs=0 l=1 rev1=1 crypto rx=0x7f9210009870 tx=0x7f921002b7d0 comp rx=0 tx=0).stop 2023-08-09T13:50:28.428 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:28.425+0000 7f921de8a640 1 -- 172.21.15.43:0/721528278 shutdown_connections 2023-08-09T13:50:28.429 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:28.425+0000 7f921de8a640 1 --2- 172.21.15.43:0/721528278 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f9218164940 0x7f9218164df0 unknown :-1 s=CLOSED pgs=1001 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:50:28.429 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:28.425+0000 7f921de8a640 1 --2- 172.21.15.43:0/721528278 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f92180c1020 0x7f9218162360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:50:28.429 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:28.425+0000 7f921de8a640 1 -- 172.21.15.43:0/721528278 >> 172.21.15.43:0/721528278 conn(0x7f92180cddd0 msgr2=0x7f92180ce1e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-09T13:50:28.429 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:28.425+0000 7f921de8a640 1 -- 172.21.15.43:0/721528278 shutdown_connections 2023-08-09T13:50:28.429 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:28.425+0000 7f921de8a640 1 -- 172.21.15.43:0/721528278 wait complete. 2023-08-09T13:50:28.431 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:28.427+0000 7f921de8a640 1 Processor -- start 2023-08-09T13:50:28.431 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:28.427+0000 7f921de8a640 1 -- start start 2023-08-09T13:50:28.431 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:28.428+0000 7f921de8a640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f92180c1020 0x7f921815d120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:50:28.431 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:28.428+0000 7f921de8a640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f9218164560 0x7f921806c530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:50:28.432 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:28.428+0000 7f921de8a640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f9218164940 0x7f921806ca70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:50:28.432 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:28.428+0000 7f921de8a640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f92180b1760 con 0x7f9218164560 2023-08-09T13:50:28.432 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:28.428+0000 7f921de8a640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f92180b0b90 con 0x7f92180c1020 2023-08-09T13:50:28.432 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:28.428+0000 7f921de8a640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f92180b0e30 con 0x7f9218164940 2023-08-09T13:50:28.432 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:28.428+0000 7f920ffff640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f9218164560 0x7f921806c530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:50:28.432 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:28.428+0000 7f920ffff640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f9218164560 0x7f921806c530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:35668/0 (socket says 172.21.15.43:35668) 2023-08-09T13:50:28.432 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:28.428+0000 7f920ffff640 1 -- 172.21.15.43:0/3038612669 learned_addr learned my addr 172.21.15.43:0/3038612669 (peer_addr_for_me v2:172.21.15.43:0/0) 2023-08-09T13:50:28.433 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:28.429+0000 7f921d689640 1 --2- 172.21.15.43:0/3038612669 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f9218164940 0x7f921806ca70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:50:28.433 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:28.429+0000 7f920ffff640 1 -- 172.21.15.43:0/3038612669 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f9218164940 msgr2=0x7f921806ca70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:50:28.433 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:28.429+0000 7f921ce88640 1 --2- 172.21.15.43:0/3038612669 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f92180c1020 0x7f921815d120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:50:28.433 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:28.429+0000 7f920ffff640 1 --2- 172.21.15.43:0/3038612669 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f9218164940 0x7f921806ca70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:50:28.433 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:28.429+0000 7f920ffff640 1 -- 172.21.15.43:0/3038612669 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f92180c1020 msgr2=0x7f921815d120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:50:28.433 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:28.429+0000 7f920ffff640 1 --2- 172.21.15.43:0/3038612669 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f92180c1020 0x7f921815d120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:50:28.434 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:28.429+0000 7f920ffff640 1 -- 172.21.15.43:0/3038612669 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f92100094d0 con 0x7f9218164560 2023-08-09T13:50:28.434 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:28.429+0000 7f921ce88640 1 --2- 172.21.15.43:0/3038612669 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f92180c1020 0x7f921815d120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-08-09T13:50:28.434 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:28.429+0000 7f920ffff640 1 --2- 172.21.15.43:0/3038612669 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f9218164560 0x7f921806c530 secure :-1 s=READY pgs=1002 cs=0 l=1 rev1=1 crypto rx=0x7f920800d5f0 tx=0x7f920800dab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-09T13:50:28.434 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:28.429+0000 7f920d7fa640 1 -- 172.21.15.43:0/3038612669 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f92080125f0 con 0x7f9218164560 2023-08-09T13:50:28.434 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:28.429+0000 7f920d7fa640 1 -- 172.21.15.43:0/3038612669 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9208012750 con 0x7f9218164560 2023-08-09T13:50:28.434 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:28.430+0000 7f920d7fa640 1 -- 172.21.15.43:0/3038612669 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9208012b00 con 0x7f9218164560 2023-08-09T13:50:28.434 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:28.430+0000 7f921de8a640 1 -- 172.21.15.43:0/3038612669 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f92180b1110 con 0x7f9218164560 2023-08-09T13:50:28.435 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:28.430+0000 7f921de8a640 1 -- 172.21.15.43:0/3038612669 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f92180b2850 con 0x7f9218164560 2023-08-09T13:50:28.435 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:28.431+0000 7f921de8a640 1 -- 172.21.15.43:0/3038612669 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f92180b1910 con 0x7f9218164560 2023-08-09T13:50:28.436 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:28.432+0000 7f920d7fa640 1 -- 172.21.15.43:0/3038612669 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 29) v1 ==== 88443+0+0 (secure 0 0 0) 0x7f9208012c60 con 0x7f9218164560 2023-08-09T13:50:28.436 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:28.433+0000 7f920d7fa640 1 --2- 172.21.15.43:0/3038612669 >> [v2:172.21.15.203:6832/3325141227,v1:172.21.15.203:6833/3325141227] conn(0x7f920406a170 0x7f920406c630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:50:28.436 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:28.433+0000 7f920d7fa640 1 -- 172.21.15.43:0/3038612669 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 6326+0+0 (secure 0 0 0) 0x7f9208088d70 con 0x7f9218164560 2023-08-09T13:50:28.436 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:28.434+0000 7f921ce88640 1 --2- 172.21.15.43:0/3038612669 >> [v2:172.21.15.203:6832/3325141227,v1:172.21.15.203:6833/3325141227] conn(0x7f920406a170 0x7f920406c630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:50:28.436 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:28.435+0000 7f921ce88640 1 --2- 172.21.15.43:0/3038612669 >> [v2:172.21.15.203:6832/3325141227,v1:172.21.15.203:6833/3325141227] conn(0x7f920406a170 0x7f920406c630 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f92180b2550 tx=0x7f92140073c0 comp rx=0 tx=0).ready entity=mgr.5519 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-09T13:50:28.439 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:28.438+0000 7f920d7fa640 1 -- 172.21.15.43:0/3038612669 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178066 (secure 0 0 0) 0x7f92180b1910 con 0x7f9218164560 2023-08-09T13:50:28.671 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:28.670+0000 7f921de8a640 1 -- 172.21.15.43:0/3038612669 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"} v 0) v1 -- 0x7f92180b1910 con 0x7f9218164560 2023-08-09T13:50:29.246 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:29.243+0000 7f920d7fa640 1 -- 172.21.15.43:0/3038612669 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]=0 new fs with metadata pool 10 and data pool 11 v31) v1 ==== 174+0+0 (secure 0 0 0) 0x7f92180b1910 con 0x7f9218164560 2023-08-09T13:50:29.246 INFO:teuthology.orchestra.run.smithi043.stderr:new fs with metadata pool 10 and data pool 11 2023-08-09T13:50:29.251 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:29.249+0000 7f921de8a640 1 -- 172.21.15.43:0/3038612669 >> [v2:172.21.15.203:6832/3325141227,v1:172.21.15.203:6833/3325141227] conn(0x7f920406a170 msgr2=0x7f920406c630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:50:29.251 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:29.249+0000 7f921de8a640 1 --2- 172.21.15.43:0/3038612669 >> [v2:172.21.15.203:6832/3325141227,v1:172.21.15.203:6833/3325141227] conn(0x7f920406a170 0x7f920406c630 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f92180b2550 tx=0x7f92140073c0 comp rx=0 tx=0).stop 2023-08-09T13:50:29.251 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:29.249+0000 7f921de8a640 1 -- 172.21.15.43:0/3038612669 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f9218164560 msgr2=0x7f921806c530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:50:29.252 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:29.249+0000 7f921de8a640 1 --2- 172.21.15.43:0/3038612669 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f9218164560 0x7f921806c530 secure :-1 s=READY pgs=1002 cs=0 l=1 rev1=1 crypto rx=0x7f920800d5f0 tx=0x7f920800dab0 comp rx=0 tx=0).stop 2023-08-09T13:50:29.252 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:29.249+0000 7f921de8a640 1 -- 172.21.15.43:0/3038612669 shutdown_connections 2023-08-09T13:50:29.252 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:29.249+0000 7f921de8a640 1 --2- 172.21.15.43:0/3038612669 >> [v2:172.21.15.203:6832/3325141227,v1:172.21.15.203:6833/3325141227] conn(0x7f920406a170 0x7f920406c630 unknown :-1 s=CLOSED pgs=58 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:50:29.252 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:29.249+0000 7f921de8a640 1 --2- 172.21.15.43:0/3038612669 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f9218164940 0x7f921806ca70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:50:29.252 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:29.249+0000 7f921de8a640 1 --2- 172.21.15.43:0/3038612669 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f9218164560 0x7f921806c530 unknown :-1 s=CLOSED pgs=1002 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:50:29.253 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:29.249+0000 7f921de8a640 1 --2- 172.21.15.43:0/3038612669 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f92180c1020 0x7f921815d120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:50:29.253 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:29.249+0000 7f921de8a640 1 -- 172.21.15.43:0/3038612669 >> 172.21.15.43:0/3038612669 conn(0x7f92180cddd0 msgr2=0x7f9218160520 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-09T13:50:29.253 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:29.250+0000 7f921de8a640 1 -- 172.21.15.43:0/3038612669 shutdown_connections 2023-08-09T13:50:29.254 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:29.250+0000 7f921de8a640 1 -- 172.21.15.43:0/3038612669 wait complete. 2023-08-09T13:50:29.276 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-08-09T13:50:29.474 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:29.472+0000 7f05a3c5e640 1 Processor -- start 2023-08-09T13:50:29.474 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:29.473+0000 7f05a3c5e640 1 -- start start 2023-08-09T13:50:29.476 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:29.474+0000 7f05a3c5e640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f059c0b2790 0x7f059c0d2ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:50:29.476 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:29.474+0000 7f05a3c5e640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f059c0b2b70 0x7f059c0b5330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:50:29.476 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:29.474+0000 7f05a3c5e640 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7f059c0b3050 0x7f059c0b3f60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:50:29.476 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:29.474+0000 7f05a3c5e640 1 -- --> v2:172.21.15.43:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f059c0b3950 con 0x7f059c0b2b70 2023-08-09T13:50:29.477 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:29.474+0000 7f05a3c5e640 1 -- --> v2:172.21.15.203:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f059c0ce090 con 0x7f059c0b3050 2023-08-09T13:50:29.477 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:29.474+0000 7f05a3c5e640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f059c0cd7f0 con 0x7f059c0b2790 2023-08-09T13:50:29.477 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:29.475+0000 7f05a11f9640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f059c0b2b70 0x7f059c0b5330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:50:29.478 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:29.475+0000 7f05a11f9640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f059c0b2b70 0x7f059c0b5330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:35682/0 (socket says 172.21.15.43:35682) 2023-08-09T13:50:29.478 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:29.475+0000 7f05a11f9640 1 -- 172.21.15.43:0/4147319279 learned_addr learned my addr 172.21.15.43:0/4147319279 (peer_addr_for_me v2:172.21.15.43:0/0) 2023-08-09T13:50:29.478 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:29.475+0000 7f05a19fa640 1 --2- 172.21.15.43:0/4147319279 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f059c0b2790 0x7f059c0d2ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:50:29.478 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:29.475+0000 7f05a21fb640 1 --2- 172.21.15.43:0/4147319279 >> v2:172.21.15.203:3300/0 conn(0x7f059c0b3050 0x7f059c0b3f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:50:29.478 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:29.475+0000 7f05a11f9640 1 -- 172.21.15.43:0/4147319279 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f059c0b2790 msgr2=0x7f059c0d2ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-09T13:50:29.478 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:29.475+0000 7f05a11f9640 1 --2- 172.21.15.43:0/4147319279 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f059c0b2790 0x7f059c0d2ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:50:29.479 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:29.475+0000 7f05a11f9640 1 -- 172.21.15.43:0/4147319279 >> v2:172.21.15.203:3300/0 conn(0x7f059c0b3050 msgr2=0x7f059c0b3f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-09T13:50:29.479 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:29.475+0000 7f05a11f9640 1 --2- 172.21.15.43:0/4147319279 >> v2:172.21.15.203:3300/0 conn(0x7f059c0b3050 0x7f059c0b3f60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:50:29.479 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:29.475+0000 7f05a11f9640 1 -- 172.21.15.43:0/4147319279 --> v2:172.21.15.43:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f059c0cdad0 con 0x7f059c0b2b70 2023-08-09T13:50:29.479 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:29.476+0000 7f05a19fa640 1 --2- 172.21.15.43:0/4147319279 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f059c0b2790 0x7f059c0d2ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-08-09T13:50:29.480 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:29.477+0000 7f05a11f9640 1 --2- 172.21.15.43:0/4147319279 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f059c0b2b70 0x7f059c0b5330 secure :-1 s=READY pgs=1003 cs=0 l=1 rev1=1 crypto rx=0x7f059c066ba0 tx=0x7f058c02b900 comp rx=0 tx=0).ready entity=mon.0 client_cookie=35f9f4308236222f server_cookie=0 in_seq=0 out_seq=0 2023-08-09T13:50:29.480 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:29.477+0000 7f05a09f8640 1 -- 172.21.15.43:0/4147319279 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f058c031860 con 0x7f059c0b2b70 2023-08-09T13:50:29.480 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:29.477+0000 7f05a09f8640 1 -- 172.21.15.43:0/4147319279 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f058c0319c0 con 0x7f059c0b2b70 2023-08-09T13:50:29.480 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:29.477+0000 7f05a09f8640 1 -- 172.21.15.43:0/4147319279 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f058c031d50 con 0x7f059c0b2b70 2023-08-09T13:50:29.480 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:29.478+0000 7f05a3c5e640 1 -- 172.21.15.43:0/4147319279 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f059c0b2b70 msgr2=0x7f059c0b5330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:50:29.481 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:29.478+0000 7f05a3c5e640 1 --2- 172.21.15.43:0/4147319279 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f059c0b2b70 0x7f059c0b5330 secure :-1 s=READY pgs=1003 cs=0 l=1 rev1=1 crypto rx=0x7f059c066ba0 tx=0x7f058c02b900 comp rx=0 tx=0).stop 2023-08-09T13:50:29.481 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:29.478+0000 7f05a3c5e640 1 -- 172.21.15.43:0/4147319279 shutdown_connections 2023-08-09T13:50:29.481 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:29.478+0000 7f05a3c5e640 1 --2- 172.21.15.43:0/4147319279 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f059c0b2b70 0x7f059c0b5330 unknown :-1 s=CLOSED pgs=1003 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:50:29.481 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:29.478+0000 7f05a3c5e640 1 --2- 172.21.15.43:0/4147319279 >> v2:172.21.15.203:3300/0 conn(0x7f059c0b3050 0x7f059c0b3f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:50:29.481 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:29.478+0000 7f05a3c5e640 1 --2- 172.21.15.43:0/4147319279 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f059c0b2790 0x7f059c0d2ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:50:29.481 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:29.478+0000 7f05a3c5e640 1 -- 172.21.15.43:0/4147319279 >> 172.21.15.43:0/4147319279 conn(0x7f059c09b7b0 msgr2=0x7f059c0d24e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-09T13:50:29.481 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:29.478+0000 7f05a3c5e640 1 -- 172.21.15.43:0/4147319279 shutdown_connections 2023-08-09T13:50:29.482 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:29.478+0000 7f05a3c5e640 1 -- 172.21.15.43:0/4147319279 wait complete. 2023-08-09T13:50:29.482 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:29.480+0000 7f05a3c5e640 1 Processor -- start 2023-08-09T13:50:29.482 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:29.480+0000 7f05a3c5e640 1 -- start start 2023-08-09T13:50:29.483 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:29.481+0000 7f05a3c5e640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f059c0b2790 0x7f059c0c7d20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:50:29.483 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:29.481+0000 7f05a3c5e640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f059c0b2b70 0x7f059c0b0c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:50:29.483 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:29.481+0000 7f05a3c5e640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f059c0b3050 0x7f059c15a740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:50:29.483 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:29.481+0000 7f05a3c5e640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f059c0c8ff0 con 0x7f059c0b3050 2023-08-09T13:50:29.483 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:29.481+0000 7f05a3c5e640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f059c0b1160 con 0x7f059c0b2b70 2023-08-09T13:50:29.483 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:29.481+0000 7f05a3c5e640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f059c0c8370 con 0x7f059c0b2790 2023-08-09T13:50:29.484 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:29.481+0000 7f05a21fb640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f059c0b3050 0x7f059c15a740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:50:29.485 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:29.481+0000 7f05a21fb640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f059c0b3050 0x7f059c15a740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:35692/0 (socket says 172.21.15.43:35692) 2023-08-09T13:50:29.485 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:29.481+0000 7f05a21fb640 1 -- 172.21.15.43:0/68819590 learned_addr learned my addr 172.21.15.43:0/68819590 (peer_addr_for_me v2:172.21.15.43:0/0) 2023-08-09T13:50:29.485 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:29.482+0000 7f05a11f9640 1 --2- 172.21.15.43:0/68819590 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f059c0b2b70 0x7f059c0b0c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:50:29.485 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:29.482+0000 7f05a19fa640 1 --2- 172.21.15.43:0/68819590 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f059c0b2790 0x7f059c0c7d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:50:29.485 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:29.482+0000 7f05a21fb640 1 -- 172.21.15.43:0/68819590 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f059c0b2790 msgr2=0x7f059c0c7d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:50:29.485 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:29.482+0000 7f05a21fb640 1 --2- 172.21.15.43:0/68819590 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f059c0b2790 0x7f059c0c7d20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:50:29.485 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:29.482+0000 7f05a21fb640 1 -- 172.21.15.43:0/68819590 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f059c0b2b70 msgr2=0x7f059c0b0c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:50:29.486 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:29.482+0000 7f05a21fb640 1 --2- 172.21.15.43:0/68819590 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f059c0b2b70 0x7f059c0b0c20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:50:29.486 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:29.482+0000 7f05a21fb640 1 -- 172.21.15.43:0/68819590 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f058c0094d0 con 0x7f059c0b3050 2023-08-09T13:50:29.486 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:29.482+0000 7f05a11f9640 1 --2- 172.21.15.43:0/68819590 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f059c0b2b70 0x7f059c0b0c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-08-09T13:50:29.486 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:29.482+0000 7f05a21fb640 1 --2- 172.21.15.43:0/68819590 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f059c0b3050 0x7f059c15a740 secure :-1 s=READY pgs=1004 cs=0 l=1 rev1=1 crypto rx=0x7f059400d600 tx=0x7f059400dac0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-09T13:50:29.486 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:29.482+0000 7f05927fc640 1 -- 172.21.15.43:0/68819590 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f05940125f0 con 0x7f059c0b3050 2023-08-09T13:50:29.486 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:29.482+0000 7f05927fc640 1 -- 172.21.15.43:0/68819590 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0594012770 con 0x7f059c0b3050 2023-08-09T13:50:29.486 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:29.482+0000 7f05927fc640 1 -- 172.21.15.43:0/68819590 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0594012b20 con 0x7f059c0b3050 2023-08-09T13:50:29.487 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:29.483+0000 7f05a3c5e640 1 -- 172.21.15.43:0/68819590 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f059c0c8650 con 0x7f059c0b3050 2023-08-09T13:50:29.487 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:29.483+0000 7f05a3c5e640 1 -- 172.21.15.43:0/68819590 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f059c0c8c10 con 0x7f059c0b3050 2023-08-09T13:50:29.487 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:29.484+0000 7f05a3c5e640 1 -- 172.21.15.43:0/68819590 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f059c0739f0 con 0x7f059c0b3050 2023-08-09T13:50:29.487 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:29.485+0000 7f05927fc640 1 -- 172.21.15.43:0/68819590 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 29) v1 ==== 88443+0+0 (secure 0 0 0) 0x7f0594012c80 con 0x7f059c0b3050 2023-08-09T13:50:29.488 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:29.486+0000 7f05927fc640 1 --2- 172.21.15.43:0/68819590 >> [v2:172.21.15.203:6832/3325141227,v1:172.21.15.203:6833/3325141227] conn(0x7f058006a170 0x7f058006c630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:50:29.488 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:29.486+0000 7f05927fc640 1 -- 172.21.15.43:0/68819590 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(89..89 src has 1..89) v4 ==== 6426+0+0 (secure 0 0 0) 0x7f0594088dd0 con 0x7f059c0b3050 2023-08-09T13:50:29.488 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:29.487+0000 7f05a19fa640 1 --2- 172.21.15.43:0/68819590 >> [v2:172.21.15.203:6832/3325141227,v1:172.21.15.203:6833/3325141227] conn(0x7f058006a170 0x7f058006c630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:50:29.489 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:29.488+0000 7f05a19fa640 1 --2- 172.21.15.43:0/68819590 >> [v2:172.21.15.203:6832/3325141227,v1:172.21.15.203:6833/3325141227] conn(0x7f058006a170 0x7f058006c630 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f059c0b63d0 tx=0x7f059800b040 comp rx=0 tx=0).ready entity=mgr.5519 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-09T13:50:29.492 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:29.491+0000 7f05927fc640 1 -- 172.21.15.43:0/68819590 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178066 (secure 0 0 0) 0x7f05940541b0 con 0x7f059c0b3050 2023-08-09T13:50:29.686 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:29.684+0000 7f05a3c5e640 1 -- 172.21.15.43:0/68819590 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f059c0b4b00 con 0x7f059c0b3050 2023-08-09T13:50:29.690 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:29.688+0000 7f05927fc640 1 -- 172.21.15.43:0/68819590 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v89) v1 ==== 74+0+18568 (secure 0 0 0) 0x7f05940577d0 con 0x7f059c0b3050 2023-08-09T13:50:29.691 INFO:teuthology.orchestra.run.smithi043.stdout: 2023-08-09T13:50:29.691 INFO:teuthology.orchestra.run.smithi043.stdout:{"epoch":89,"fsid":"daac87ee-1337-4250-a955-aea4e408a39b","created":"2023-08-09T13:33:41.701198+0000","modified":"2023-08-09T13:50:28.671721+0000","last_up_change":"2023-08-09T13:33:53.783453+0000","last_in_change":"2023-08-09T13:33:45.148904+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":11,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":"rbd","create_time":"2023-08-09T13:33:54.365137+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":".mgr","create_time":"2023-08-09T13:33:54.831803+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":10,"pool_name":"cephfs_metadata","create_time":"2023-08-09T13:50:27.051781+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"89","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":2.5,"score_stable":2.5,"optimal_score":1,"raw_score_acting":2.5,"raw_score_stable":2.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":11,"pool_name":"cephfs_data","create_time":"2023-08-09T13:50:27.692933+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"89","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"cbd49ecd-e896-4fbd-864e-5c37a829b165","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6800","nonce":502667335},{"type":"v1","addr":"172.21.15.43:6801","nonce":502667335}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6802","nonce":502667335},{"type":"v1","addr":"172.21.15.43:6803","nonce":502667335}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6806","nonce":502667335},{"type":"v1","addr":"172.21.15.43:6807","nonce":502667335}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6804","nonce":502667335},{"type":"v1","addr":"172.21.15.43:6805","nonce":502667335}]},"public_addr":"172.21.15.43:6801/502667335","cluster_addr":"172.21.15.43:6803/502667335","heartbeat_back_addr":"172.21.15.43:6807/502667335","heartbeat_front_addr":"172.21.15.43:6805/502667335","state":["exists","up"]},{"osd":1,"uuid":"75c7e0fe-1ec7-4fe8-a2f2-3661a65b8500","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6808","nonce":3289134610},{"type":"v1","addr":"172.21.15.43:6809","nonce":3289134610}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6810","nonce":3289134610},{"type":"v1","addr":"172.21.15.43:6812","nonce":3289134610}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6821","nonce":3289134610},{"type":"v1","addr":"172.21.15.43:6824","nonce":3289134610}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6815","nonce":3289134610},{"type":"v1","addr":"172.21.15.43:6818","nonce":3289134610}]},"public_addr":"172.21.15.43:6809/3289134610","cluster_addr":"172.21.15.43:6812/3289134610","heartbeat_back_addr":"172.21.15.43:6824/3289134610","heartbeat_front_addr":"172.21.15.43:6818/3289134610","state":["exists","up"]},{"osd":2,"uuid":"bb5c41e9-7f5f-4125-9b01-3bb8bfc2ea83","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6811","nonce":4079709736},{"type":"v1","addr":"172.21.15.43:6814","nonce":4079709736}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6817","nonce":4079709736},{"type":"v1","addr":"172.21.15.43:6820","nonce":4079709736}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6829","nonce":4079709736},{"type":"v1","addr":"172.21.15.43:6831","nonce":4079709736}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6823","nonce":4079709736},{"type":"v1","addr":"172.21.15.43:6826","nonce":4079709736}]},"public_addr":"172.21.15.43:6814/4079709736","cluster_addr":"172.21.15.43:6820/4079709736","heartbeat_back_addr":"172.21.15.43:6831/4079709736","heartbeat_front_addr":"172.21.15.43:6826/4079709736","state":["exists","up"]},{"osd":3,"uuid":"bfe4c8b4-694a-465c-87e6-2b2aab852613","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6813","nonce":1709538759},{"type":"v1","addr":"172.21.15.43:6816","nonce":1709538759}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6819","nonce":1709538759},{"type":"v1","addr":"172.21.15.43:6822","nonce":1709538759}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6828","nonce":1709538759},{"type":"v1","addr":"172.21.15.43:6830","nonce":1709538759}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6825","nonce":1709538759},{"type":"v1","addr":"172.21.15.43:6827","nonce":1709538759}]},"public_addr":"172.21.15.43:6816/1709538759","cluster_addr":"172.21.15.43:6822/1709538759","heartbeat_back_addr":"172.21.15.43:6830/1709538759","heartbeat_front_addr":"172.21.15.43:6827/1709538759","state":["exists","up"]},{"osd":4,"uuid":"fa2e8a77-aba1-4756-853e-0d174a5bdeba","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6800","nonce":4196866481},{"type":"v1","addr":"172.21.15.203:6801","nonce":4196866481}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6802","nonce":4196866481},{"type":"v1","addr":"172.21.15.203:6803","nonce":4196866481}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6806","nonce":4196866481},{"type":"v1","addr":"172.21.15.203:6807","nonce":4196866481}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6804","nonce":4196866481},{"type":"v1","addr":"172.21.15.203:6805","nonce":4196866481}]},"public_addr":"172.21.15.203:6801/4196866481","cluster_addr":"172.21.15.203:6803/4196866481","heartbeat_back_addr":"172.21.15.203:6807/4196866481","heartbeat_front_addr":"172.21.15.203:6805/4196866481","state":["exists","up"]},{"osd":5,"uuid":"81811e4b-4d24-4f51-a82c-f1d700a2a00d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6816","nonce":3059290081},{"type":"v1","addr":"172.21.15.203:6817","nonce":3059290081}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6818","nonce":3059290081},{"type":"v1","addr":"172.21.15.203:6819","nonce":3059290081}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6822","nonce":3059290081},{"type":"v1","addr":"172.21.15.203:6823","nonce":3059290081}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6820","nonce":3059290081},{"type":"v1","addr":"172.21.15.203:6821","nonce":3059290081}]},"public_addr":"172.21.15.203:6817/3059290081","cluster_addr":"172.21.15.203:6819/3059290081","heartbeat_back_addr":"172.21.15.203:6823/3059290081","heartbeat_front_addr":"172.21.15.203:6821/3059290081","state":["exists","up"]},{"osd":6,"uuid":"307bac4d-6a5b-443d-af75-56d695d7674f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6824","nonce":512155569},{"type":"v1","addr":"172.21.15.203:6825","nonce":512155569}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6826","nonce":512155569},{"type":"v1","addr":"172.21.15.203:6827","nonce":512155569}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6830","nonce":512155569},{"type":"v1","addr":"172.21.15.203:6831","nonce":512155569}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6828","nonce":512155569},{"type":"v1","addr":"172.21.15.203:6829","nonce":512155569}]},"public_addr":"172.21.15.203:6825/512155569","cluster_addr":"172.21.15.203:6827/512155569","heartbeat_back_addr":"172.21.15.203:6831/512155569","heartbeat_front_addr":"172.21.15.203:6829/512155569","state":["exists","up"]},{"osd":7,"uuid":"a95dc7ca-9f1b-4295-af70-68a22a2d7d5e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6808","nonce":3331450438},{"type":"v1","addr":"172.21.15.203:6809","nonce":3331450438}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6810","nonce":3331450438},{"type":"v1","addr":"172.21.15.203:6811","nonce":3331450438}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6814","nonce":3331450438},{"type":"v1","addr":"172.21.15.203:6815","nonce":3331450438}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6812","nonce":3331450438},{"type":"v1","addr":"172.21.15.203:6813","nonce":3331450438}]},"public_addr":"172.21.15.203:6809/3331450438","cluster_addr":"172.21.15.203:6811/3331450438","heartbeat_back_addr":"172.21.15.203:6815/3331450438","heartbeat_front_addr":"172.21.15.203:6813/3331450438","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-09T13:33:50.547652+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-09T13:33:50.589469+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-09T13:33:50.561724+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-09T13:33:50.599650+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-09T13:33:50.411855+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-09T13:33:50.219681+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-09T13:33:50.504664+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-09T13:33:50.441248+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":7,"snaps":[{"begin":104,"length":1},{"begin":221,"length":1}]}],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-08-09T13:50:29.695 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:29.690+0000 7f05a3c5e640 1 -- 172.21.15.43:0/68819590 >> [v2:172.21.15.203:6832/3325141227,v1:172.21.15.203:6833/3325141227] conn(0x7f058006a170 msgr2=0x7f058006c630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:50:29.695 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:29.690+0000 7f05a3c5e640 1 --2- 172.21.15.43:0/68819590 >> [v2:172.21.15.203:6832/3325141227,v1:172.21.15.203:6833/3325141227] conn(0x7f058006a170 0x7f058006c630 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f059c0b63d0 tx=0x7f059800b040 comp rx=0 tx=0).stop 2023-08-09T13:50:29.695 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:29.690+0000 7f05a3c5e640 1 -- 172.21.15.43:0/68819590 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f059c0b3050 msgr2=0x7f059c15a740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:50:29.695 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:29.690+0000 7f05a3c5e640 1 --2- 172.21.15.43:0/68819590 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f059c0b3050 0x7f059c15a740 secure :-1 s=READY pgs=1004 cs=0 l=1 rev1=1 crypto rx=0x7f059400d600 tx=0x7f059400dac0 comp rx=0 tx=0).stop 2023-08-09T13:50:29.696 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:29.690+0000 7f05a3c5e640 1 -- 172.21.15.43:0/68819590 shutdown_connections 2023-08-09T13:50:29.696 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:29.691+0000 7f05a3c5e640 1 --2- 172.21.15.43:0/68819590 >> [v2:172.21.15.203:6832/3325141227,v1:172.21.15.203:6833/3325141227] conn(0x7f058006a170 0x7f058006c630 unknown :-1 s=CLOSED pgs=59 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:50:29.696 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:29.691+0000 7f05a3c5e640 1 --2- 172.21.15.43:0/68819590 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f059c0b3050 0x7f059c15a740 unknown :-1 s=CLOSED pgs=1004 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:50:29.697 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:29.691+0000 7f05a3c5e640 1 --2- 172.21.15.43:0/68819590 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f059c0b2b70 0x7f059c0b0c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:50:29.697 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:29.691+0000 7f05a3c5e640 1 --2- 172.21.15.43:0/68819590 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f059c0b2790 0x7f059c0c7d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:50:29.697 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:29.691+0000 7f05a3c5e640 1 -- 172.21.15.43:0/68819590 >> 172.21.15.43:0/68819590 conn(0x7f059c09b7b0 msgr2=0x7f059c15bbc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-09T13:50:29.697 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:29.691+0000 7f05a3c5e640 1 -- 172.21.15.43:0/68819590 shutdown_connections 2023-08-09T13:50:29.697 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:29.691+0000 7f05a3c5e640 1 -- 172.21.15.43:0/68819590 wait complete. 2023-08-09T13:50:29.711 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-08-09T13:50:29.943 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:29.942+0000 7f6b8152f640 1 Processor -- start 2023-08-09T13:50:29.944 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:29.942+0000 7f6b8152f640 1 -- start start 2023-08-09T13:50:29.945 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:29.944+0000 7f6b8152f640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f6b7c142ca0 0x7f6b7c0c96e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:50:29.945 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:29.944+0000 7f6b8152f640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f6b7c0cd510 0x7f6b7c0c3290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:50:29.945 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:29.944+0000 7f6b8152f640 1 -- --> v1:172.21.15.43:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6b7c143080 con 0x7f6b7c15ce40 2023-08-09T13:50:29.945 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:29.944+0000 7f6b8152f640 1 -- --> v2:172.21.15.43:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6b7c06d400 con 0x7f6b7c142ca0 2023-08-09T13:50:29.946 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:29.944+0000 7f6b8152f640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6b7c0c9d90 con 0x7f6b7c0cd510 2023-08-09T13:50:29.946 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:29.944+0000 7f6b7affd640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f6b7c142ca0 0x7f6b7c0c96e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:50:29.946 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:29.944+0000 7f6b73fff640 1 --1- >> v1:172.21.15.43:6789/0 conn(0x7f6b7c15ce40 0x7f6b7c0cce00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.43:6789/0 says I am v1:172.21.15.43:32956/0 (socket says 172.21.15.43:32956) 2023-08-09T13:50:29.947 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:29.944+0000 7f6b73fff640 1 -- 172.21.15.43:0/3345698222 learned_addr learned my addr 172.21.15.43:0/3345698222 (peer_addr_for_me v1:172.21.15.43:0/0) 2023-08-09T13:50:29.947 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:29.944+0000 7f6b7b7fe640 1 --2- 172.21.15.43:0/3345698222 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f6b7c0cd510 0x7f6b7c0c3290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:50:29.947 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:29.945+0000 7f6b7affd640 1 -- 172.21.15.43:0/3345698222 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f6b7c0cd510 msgr2=0x7f6b7c0c3290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-09T13:50:29.947 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:29.945+0000 7f6b7affd640 1 --2- 172.21.15.43:0/3345698222 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f6b7c0cd510 0x7f6b7c0c3290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:50:29.947 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:29.945+0000 7f6b7affd640 1 -- 172.21.15.43:0/3345698222 >> v1:172.21.15.43:6789/0 conn(0x7f6b7c15ce40 legacy=0x7f6b7c0cce00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:50:29.947 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:29.945+0000 7f6b7affd640 1 -- 172.21.15.43:0/3345698222 --> v2:172.21.15.43:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6b7c0ca070 con 0x7f6b7c142ca0 2023-08-09T13:50:29.947 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:29.945+0000 7f6b7a7fc640 1 -- 172.21.15.43:0/3345698222 <== mon.0 v1:172.21.15.43:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2513133221 0 0) 0x7f6b7c143080 con 0x7f6b7c15ce40 2023-08-09T13:50:29.948 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:29.946+0000 7f6b7affd640 1 --2- 172.21.15.43:0/3345698222 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f6b7c142ca0 0x7f6b7c0c96e0 secure :-1 s=READY pgs=1006 cs=0 l=1 rev1=1 crypto rx=0x7f6b6c0097a0 tx=0x7f6b6c02b8b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6c1626de944cceee server_cookie=0 in_seq=0 out_seq=0 2023-08-09T13:50:29.949 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:29.946+0000 7f6b7a7fc640 1 -- 172.21.15.43:0/3345698222 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6b6c02d040 con 0x7f6b7c142ca0 2023-08-09T13:50:29.949 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:29.946+0000 7f6b7a7fc640 1 -- 172.21.15.43:0/3345698222 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6b6c031810 con 0x7f6b7c142ca0 2023-08-09T13:50:29.949 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:29.946+0000 7f6b7a7fc640 1 -- 172.21.15.43:0/3345698222 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6b6c031ba0 con 0x7f6b7c142ca0 2023-08-09T13:50:29.949 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:29.947+0000 7f6b8152f640 1 -- 172.21.15.43:0/3345698222 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f6b7c142ca0 msgr2=0x7f6b7c0c96e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:50:29.949 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:29.947+0000 7f6b8152f640 1 --2- 172.21.15.43:0/3345698222 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f6b7c142ca0 0x7f6b7c0c96e0 secure :-1 s=READY pgs=1006 cs=0 l=1 rev1=1 crypto rx=0x7f6b6c0097a0 tx=0x7f6b6c02b8b0 comp rx=0 tx=0).stop 2023-08-09T13:50:29.949 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:29.947+0000 7f6b8152f640 1 -- 172.21.15.43:0/3345698222 shutdown_connections 2023-08-09T13:50:29.950 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:29.947+0000 7f6b8152f640 1 --2- 172.21.15.43:0/3345698222 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f6b7c0cd510 0x7f6b7c0c3290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:50:29.950 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:29.947+0000 7f6b8152f640 1 --2- 172.21.15.43:0/3345698222 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f6b7c142ca0 0x7f6b7c0c96e0 unknown :-1 s=CLOSED pgs=1006 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:50:29.950 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:29.947+0000 7f6b8152f640 1 -- 172.21.15.43:0/3345698222 >> 172.21.15.43:0/3345698222 conn(0x7f6b7c0ce0a0 msgr2=0x7f6b7c0ce480 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-09T13:50:29.950 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:29.947+0000 7f6b8152f640 1 -- 172.21.15.43:0/3345698222 shutdown_connections 2023-08-09T13:50:29.950 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:29.947+0000 7f6b8152f640 1 -- 172.21.15.43:0/3345698222 wait complete. 2023-08-09T13:50:29.951 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:29.949+0000 7f6b8152f640 1 Processor -- start 2023-08-09T13:50:29.951 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:29.949+0000 7f6b8152f640 1 -- start start 2023-08-09T13:50:29.952 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:29.950+0000 7f6b8152f640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f6b7c0cd510 0x7f6b7c0b29d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:50:29.952 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:29.950+0000 7f6b8152f640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f6b7c142ca0 0x7f6b7c1f1f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:50:29.952 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:29.950+0000 7f6b8152f640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f6b7c15ce40 0x7f6b7c158380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:50:29.952 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:29.950+0000 7f6b8152f640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6b7c0b31e0 con 0x7f6b7c15ce40 2023-08-09T13:50:29.952 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:29.950+0000 7f6b8152f640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6b7c0b2f10 con 0x7f6b7c142ca0 2023-08-09T13:50:29.952 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:29.950+0000 7f6b8152f640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6b7c1e3bc0 con 0x7f6b7c0cd510 2023-08-09T13:50:29.954 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:29.951+0000 7f6b7b7fe640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f6b7c15ce40 0x7f6b7c158380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:50:29.954 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:29.951+0000 7f6b7b7fe640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f6b7c15ce40 0x7f6b7c158380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:35714/0 (socket says 172.21.15.43:35714) 2023-08-09T13:50:29.954 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:29.951+0000 7f6b7b7fe640 1 -- 172.21.15.43:0/2054749361 learned_addr learned my addr 172.21.15.43:0/2054749361 (peer_addr_for_me v2:172.21.15.43:0/0) 2023-08-09T13:50:29.954 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:29.951+0000 7f6b7affd640 1 --2- 172.21.15.43:0/2054749361 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f6b7c0cd510 0x7f6b7c0b29d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:50:29.954 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:29.951+0000 7f6b73fff640 1 --2- 172.21.15.43:0/2054749361 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f6b7c142ca0 0x7f6b7c1f1f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:50:29.954 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:29.951+0000 7f6b7b7fe640 1 -- 172.21.15.43:0/2054749361 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f6b7c0cd510 msgr2=0x7f6b7c0b29d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:50:29.955 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:29.951+0000 7f6b7b7fe640 1 --2- 172.21.15.43:0/2054749361 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f6b7c0cd510 0x7f6b7c0b29d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:50:29.955 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:29.951+0000 7f6b7b7fe640 1 -- 172.21.15.43:0/2054749361 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f6b7c142ca0 msgr2=0x7f6b7c1f1f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:50:29.956 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:29.951+0000 7f6b7b7fe640 1 --2- 172.21.15.43:0/2054749361 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f6b7c142ca0 0x7f6b7c1f1f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:50:29.956 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:29.951+0000 7f6b7b7fe640 1 -- 172.21.15.43:0/2054749361 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6b6c009400 con 0x7f6b7c15ce40 2023-08-09T13:50:29.957 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:29.952+0000 7f6b7b7fe640 1 --2- 172.21.15.43:0/2054749361 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f6b7c15ce40 0x7f6b7c158380 secure :-1 s=READY pgs=1007 cs=0 l=1 rev1=1 crypto rx=0x7f6b74007d80 tx=0x7f6b7400e4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-09T13:50:29.957 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:29.952+0000 7f6b737fe640 1 -- 172.21.15.43:0/2054749361 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6b74014420 con 0x7f6b7c15ce40 2023-08-09T13:50:29.957 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:29.952+0000 7f6b737fe640 1 -- 172.21.15.43:0/2054749361 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6b74014580 con 0x7f6b7c15ce40 2023-08-09T13:50:29.957 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:29.952+0000 7f6b737fe640 1 -- 172.21.15.43:0/2054749361 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6b74014930 con 0x7f6b7c15ce40 2023-08-09T13:50:29.958 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:29.952+0000 7f6b8152f640 1 -- 172.21.15.43:0/2054749361 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6b7c1e3ea0 con 0x7f6b7c15ce40 2023-08-09T13:50:29.958 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:29.952+0000 7f6b8152f640 1 -- 172.21.15.43:0/2054749361 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6b7c1e4460 con 0x7f6b7c15ce40 2023-08-09T13:50:29.959 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:29.953+0000 7f6b8152f640 1 -- 172.21.15.43:0/2054749361 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6b7c0739f0 con 0x7f6b7c15ce40 2023-08-09T13:50:29.959 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:29.954+0000 7f6b737fe640 1 -- 172.21.15.43:0/2054749361 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 29) v1 ==== 88443+0+0 (secure 0 0 0) 0x7f6b74014a90 con 0x7f6b7c15ce40 2023-08-09T13:50:29.960 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:29.955+0000 7f6b737fe640 1 --2- 172.21.15.43:0/2054749361 >> [v2:172.21.15.203:6832/3325141227,v1:172.21.15.203:6833/3325141227] conn(0x7f6b5c06a170 0x7f6b5c06c630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:50:29.960 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:29.955+0000 7f6b737fe640 1 -- 172.21.15.43:0/2054749361 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(89..89 src has 1..89) v4 ==== 6426+0+0 (secure 0 0 0) 0x7f6b740891a0 con 0x7f6b7c15ce40 2023-08-09T13:50:29.960 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:29.956+0000 7f6b7affd640 1 --2- 172.21.15.43:0/2054749361 >> [v2:172.21.15.203:6832/3325141227,v1:172.21.15.203:6833/3325141227] conn(0x7f6b5c06a170 0x7f6b5c06c630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:50:29.961 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:29.957+0000 7f6b7affd640 1 --2- 172.21.15.43:0/2054749361 >> [v2:172.21.15.203:6832/3325141227,v1:172.21.15.203:6833/3325141227] conn(0x7f6b5c06a170 0x7f6b5c06c630 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f6b6c036000 tx=0x7f6b6c03c040 comp rx=0 tx=0).ready entity=mgr.5519 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-09T13:50:29.961 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:29.960+0000 7f6b737fe640 1 -- 172.21.15.43:0/2054749361 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178066 (secure 0 0 0) 0x7f6b74058090 con 0x7f6b7c15ce40 2023-08-09T13:50:30.158 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:30.156+0000 7f6b8152f640 1 -- 172.21.15.43:0/2054749361 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f6b7c0c4410 con 0x7f6b7c15ce40 2023-08-09T13:50:30.161 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:30.159+0000 7f6b737fe640 1 -- 172.21.15.43:0/2054749361 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v89) v1 ==== 74+0+18568 (secure 0 0 0) 0x7f6b74057c20 con 0x7f6b7c15ce40 2023-08-09T13:50:30.161 INFO:teuthology.orchestra.run.smithi043.stdout: 2023-08-09T13:50:30.162 INFO:teuthology.orchestra.run.smithi043.stdout:{"epoch":89,"fsid":"daac87ee-1337-4250-a955-aea4e408a39b","created":"2023-08-09T13:33:41.701198+0000","modified":"2023-08-09T13:50:28.671721+0000","last_up_change":"2023-08-09T13:33:53.783453+0000","last_in_change":"2023-08-09T13:33:45.148904+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":11,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":"rbd","create_time":"2023-08-09T13:33:54.365137+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":".mgr","create_time":"2023-08-09T13:33:54.831803+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":10,"pool_name":"cephfs_metadata","create_time":"2023-08-09T13:50:27.051781+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"89","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":2.5,"score_stable":2.5,"optimal_score":1,"raw_score_acting":2.5,"raw_score_stable":2.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":11,"pool_name":"cephfs_data","create_time":"2023-08-09T13:50:27.692933+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"89","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"cbd49ecd-e896-4fbd-864e-5c37a829b165","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6800","nonce":502667335},{"type":"v1","addr":"172.21.15.43:6801","nonce":502667335}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6802","nonce":502667335},{"type":"v1","addr":"172.21.15.43:6803","nonce":502667335}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6806","nonce":502667335},{"type":"v1","addr":"172.21.15.43:6807","nonce":502667335}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6804","nonce":502667335},{"type":"v1","addr":"172.21.15.43:6805","nonce":502667335}]},"public_addr":"172.21.15.43:6801/502667335","cluster_addr":"172.21.15.43:6803/502667335","heartbeat_back_addr":"172.21.15.43:6807/502667335","heartbeat_front_addr":"172.21.15.43:6805/502667335","state":["exists","up"]},{"osd":1,"uuid":"75c7e0fe-1ec7-4fe8-a2f2-3661a65b8500","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6808","nonce":3289134610},{"type":"v1","addr":"172.21.15.43:6809","nonce":3289134610}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6810","nonce":3289134610},{"type":"v1","addr":"172.21.15.43:6812","nonce":3289134610}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6821","nonce":3289134610},{"type":"v1","addr":"172.21.15.43:6824","nonce":3289134610}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6815","nonce":3289134610},{"type":"v1","addr":"172.21.15.43:6818","nonce":3289134610}]},"public_addr":"172.21.15.43:6809/3289134610","cluster_addr":"172.21.15.43:6812/3289134610","heartbeat_back_addr":"172.21.15.43:6824/3289134610","heartbeat_front_addr":"172.21.15.43:6818/3289134610","state":["exists","up"]},{"osd":2,"uuid":"bb5c41e9-7f5f-4125-9b01-3bb8bfc2ea83","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6811","nonce":4079709736},{"type":"v1","addr":"172.21.15.43:6814","nonce":4079709736}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6817","nonce":4079709736},{"type":"v1","addr":"172.21.15.43:6820","nonce":4079709736}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6829","nonce":4079709736},{"type":"v1","addr":"172.21.15.43:6831","nonce":4079709736}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6823","nonce":4079709736},{"type":"v1","addr":"172.21.15.43:6826","nonce":4079709736}]},"public_addr":"172.21.15.43:6814/4079709736","cluster_addr":"172.21.15.43:6820/4079709736","heartbeat_back_addr":"172.21.15.43:6831/4079709736","heartbeat_front_addr":"172.21.15.43:6826/4079709736","state":["exists","up"]},{"osd":3,"uuid":"bfe4c8b4-694a-465c-87e6-2b2aab852613","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6813","nonce":1709538759},{"type":"v1","addr":"172.21.15.43:6816","nonce":1709538759}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6819","nonce":1709538759},{"type":"v1","addr":"172.21.15.43:6822","nonce":1709538759}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6828","nonce":1709538759},{"type":"v1","addr":"172.21.15.43:6830","nonce":1709538759}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6825","nonce":1709538759},{"type":"v1","addr":"172.21.15.43:6827","nonce":1709538759}]},"public_addr":"172.21.15.43:6816/1709538759","cluster_addr":"172.21.15.43:6822/1709538759","heartbeat_back_addr":"172.21.15.43:6830/1709538759","heartbeat_front_addr":"172.21.15.43:6827/1709538759","state":["exists","up"]},{"osd":4,"uuid":"fa2e8a77-aba1-4756-853e-0d174a5bdeba","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6800","nonce":4196866481},{"type":"v1","addr":"172.21.15.203:6801","nonce":4196866481}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6802","nonce":4196866481},{"type":"v1","addr":"172.21.15.203:6803","nonce":4196866481}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6806","nonce":4196866481},{"type":"v1","addr":"172.21.15.203:6807","nonce":4196866481}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6804","nonce":4196866481},{"type":"v1","addr":"172.21.15.203:6805","nonce":4196866481}]},"public_addr":"172.21.15.203:6801/4196866481","cluster_addr":"172.21.15.203:6803/4196866481","heartbeat_back_addr":"172.21.15.203:6807/4196866481","heartbeat_front_addr":"172.21.15.203:6805/4196866481","state":["exists","up"]},{"osd":5,"uuid":"81811e4b-4d24-4f51-a82c-f1d700a2a00d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6816","nonce":3059290081},{"type":"v1","addr":"172.21.15.203:6817","nonce":3059290081}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6818","nonce":3059290081},{"type":"v1","addr":"172.21.15.203:6819","nonce":3059290081}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6822","nonce":3059290081},{"type":"v1","addr":"172.21.15.203:6823","nonce":3059290081}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6820","nonce":3059290081},{"type":"v1","addr":"172.21.15.203:6821","nonce":3059290081}]},"public_addr":"172.21.15.203:6817/3059290081","cluster_addr":"172.21.15.203:6819/3059290081","heartbeat_back_addr":"172.21.15.203:6823/3059290081","heartbeat_front_addr":"172.21.15.203:6821/3059290081","state":["exists","up"]},{"osd":6,"uuid":"307bac4d-6a5b-443d-af75-56d695d7674f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6824","nonce":512155569},{"type":"v1","addr":"172.21.15.203:6825","nonce":512155569}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6826","nonce":512155569},{"type":"v1","addr":"172.21.15.203:6827","nonce":512155569}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6830","nonce":512155569},{"type":"v1","addr":"172.21.15.203:6831","nonce":512155569}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6828","nonce":512155569},{"type":"v1","addr":"172.21.15.203:6829","nonce":512155569}]},"public_addr":"172.21.15.203:6825/512155569","cluster_addr":"172.21.15.203:6827/512155569","heartbeat_back_addr":"172.21.15.203:6831/512155569","heartbeat_front_addr":"172.21.15.203:6829/512155569","state":["exists","up"]},{"osd":7,"uuid":"a95dc7ca-9f1b-4295-af70-68a22a2d7d5e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6808","nonce":3331450438},{"type":"v1","addr":"172.21.15.203:6809","nonce":3331450438}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6810","nonce":3331450438},{"type":"v1","addr":"172.21.15.203:6811","nonce":3331450438}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6814","nonce":3331450438},{"type":"v1","addr":"172.21.15.203:6815","nonce":3331450438}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6812","nonce":3331450438},{"type":"v1","addr":"172.21.15.203:6813","nonce":3331450438}]},"public_addr":"172.21.15.203:6809/3331450438","cluster_addr":"172.21.15.203:6811/3331450438","heartbeat_back_addr":"172.21.15.203:6815/3331450438","heartbeat_front_addr":"172.21.15.203:6813/3331450438","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-09T13:33:50.547652+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-09T13:33:50.589469+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-09T13:33:50.561724+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-09T13:33:50.599650+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-09T13:33:50.411855+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-09T13:33:50.219681+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-09T13:33:50.504664+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-09T13:33:50.441248+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":7,"snaps":[{"begin":104,"length":1},{"begin":221,"length":1}]}],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-08-09T13:50:30.163 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:30.162+0000 7f6b8152f640 1 -- 172.21.15.43:0/2054749361 >> [v2:172.21.15.203:6832/3325141227,v1:172.21.15.203:6833/3325141227] conn(0x7f6b5c06a170 msgr2=0x7f6b5c06c630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:50:30.164 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:30.162+0000 7f6b8152f640 1 --2- 172.21.15.43:0/2054749361 >> [v2:172.21.15.203:6832/3325141227,v1:172.21.15.203:6833/3325141227] conn(0x7f6b5c06a170 0x7f6b5c06c630 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f6b6c036000 tx=0x7f6b6c03c040 comp rx=0 tx=0).stop 2023-08-09T13:50:30.164 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:30.162+0000 7f6b8152f640 1 -- 172.21.15.43:0/2054749361 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f6b7c15ce40 msgr2=0x7f6b7c158380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:50:30.164 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:30.162+0000 7f6b8152f640 1 --2- 172.21.15.43:0/2054749361 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f6b7c15ce40 0x7f6b7c158380 secure :-1 s=READY pgs=1007 cs=0 l=1 rev1=1 crypto rx=0x7f6b74007d80 tx=0x7f6b7400e4b0 comp rx=0 tx=0).stop 2023-08-09T13:50:30.164 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:30.162+0000 7f6b8152f640 1 -- 172.21.15.43:0/2054749361 shutdown_connections 2023-08-09T13:50:30.164 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:30.162+0000 7f6b8152f640 1 --2- 172.21.15.43:0/2054749361 >> [v2:172.21.15.203:6832/3325141227,v1:172.21.15.203:6833/3325141227] conn(0x7f6b5c06a170 0x7f6b5c06c630 unknown :-1 s=CLOSED pgs=60 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:50:30.164 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:30.162+0000 7f6b8152f640 1 --2- 172.21.15.43:0/2054749361 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f6b7c15ce40 0x7f6b7c158380 unknown :-1 s=CLOSED pgs=1007 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:50:30.165 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:30.162+0000 7f6b8152f640 1 --2- 172.21.15.43:0/2054749361 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f6b7c142ca0 0x7f6b7c1f1f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:50:30.165 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:30.162+0000 7f6b8152f640 1 --2- 172.21.15.43:0/2054749361 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f6b7c0cd510 0x7f6b7c0b29d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:50:30.165 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:30.162+0000 7f6b8152f640 1 -- 172.21.15.43:0/2054749361 >> 172.21.15.43:0/2054749361 conn(0x7f6b7c0ce0a0 msgr2=0x7f6b7c0ca870 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-09T13:50:30.165 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:30.162+0000 7f6b8152f640 1 -- 172.21.15.43:0/2054749361 shutdown_connections 2023-08-09T13:50:30.165 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:30.162+0000 7f6b8152f640 1 -- 172.21.15.43:0/2054749361 wait complete. 2023-08-09T13:50:30.184 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs standby_count_wanted 0 2023-08-09T13:50:30.421 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:30.420+0000 7f761886c640 1 Processor -- start 2023-08-09T13:50:30.422 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:30.420+0000 7f761886c640 1 -- start start 2023-08-09T13:50:30.423 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:30.422+0000 7f761886c640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f76140cc380 0x7f7614160f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:50:30.423 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:30.422+0000 7f761886c640 1 -- --> v1:172.21.15.43:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f76140c35e0 con 0x7f76140c3130 2023-08-09T13:50:30.423 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:30.422+0000 7f761886c640 1 -- --> v1:172.21.15.203:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f76140c38a0 con 0x7f761414a5c0 2023-08-09T13:50:30.424 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:30.422+0000 7f761886c640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f76140c4420 con 0x7f76140cc380 2023-08-09T13:50:30.425 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:30.422+0000 7f7612d9d640 1 --1- >> v1:172.21.15.43:6789/0 conn(0x7f76140c3130 0x7f761406bb50 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.43:6789/0 says I am v1:172.21.15.43:32962/0 (socket says 172.21.15.43:32962) 2023-08-09T13:50:30.426 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:30.422+0000 7f7612d9d640 1 -- 172.21.15.43:0/333497410 learned_addr learned my addr 172.21.15.43:0/333497410 (peer_addr_for_me v1:172.21.15.43:0/0) 2023-08-09T13:50:30.426 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:30.423+0000 7f7609d9b640 1 --2- 172.21.15.43:0/333497410 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f76140cc380 0x7f7614160f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:50:30.426 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:30.423+0000 7f7611d9b640 1 -- 172.21.15.43:0/333497410 <== mon.0 v1:172.21.15.43:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 950989303 0 0) 0x7f76140c35e0 con 0x7f76140c3130 2023-08-09T13:50:30.426 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:30.423+0000 7f7611d9b640 1 -- 172.21.15.43:0/333497410 --> v1:172.21.15.43:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f7600003520 con 0x7f76140c3130 2023-08-09T13:50:30.426 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:30.423+0000 7f7611d9b640 1 -- 172.21.15.43:0/333497410 <== mon.0 v1:172.21.15.43:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2757014574 0 0) 0x7f7600003520 con 0x7f76140c3130 2023-08-09T13:50:30.426 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:30.423+0000 7f7611d9b640 1 -- 172.21.15.43:0/333497410 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f76140cc380 msgr2=0x7f7614160f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-09T13:50:30.426 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:30.423+0000 7f7611d9b640 1 --2- 172.21.15.43:0/333497410 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f76140cc380 0x7f7614160f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:50:30.426 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:30.423+0000 7f7611d9b640 1 -- 172.21.15.43:0/333497410 >> v1:172.21.15.203:6789/0 conn(0x7f761414a5c0 legacy=0x7f76140cbc70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:50:30.427 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:30.423+0000 7f7611d9b640 1 -- 172.21.15.43:0/333497410 --> v1:172.21.15.43:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f76140c3b80 con 0x7f76140c3130 2023-08-09T13:50:30.427 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:30.423+0000 7f7611d9b640 1 -- 172.21.15.43:0/333497410 <== mon.0 v1:172.21.15.43:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1285150740 0 0) 0x7f760c0019f0 con 0x7f76140c3130 2023-08-09T13:50:30.427 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:30.423+0000 7f7611d9b640 1 -- 172.21.15.43:0/333497410 >> v1:172.21.15.43:6789/0 conn(0x7f76140c3130 legacy=0x7f761406bb50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:50:30.427 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:30.423+0000 7f7611d9b640 1 --2- 172.21.15.43:0/333497410 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f76140cc380 0x7f7600003a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:50:30.427 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:30.423+0000 7f7611d9b640 1 --2- 172.21.15.43:0/333497410 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f7600003f90 0x7f76000047c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:50:30.427 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:30.423+0000 7f7611d9b640 1 --2- 172.21.15.43:0/333497410 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f7600004d00 0x7f7600005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:50:30.427 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:30.423+0000 7f7611d9b640 1 -- 172.21.15.43:0/333497410 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f76140c4420 con 0x7f7600004d00 2023-08-09T13:50:30.428 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:30.423+0000 7f7611d9b640 1 -- 172.21.15.43:0/333497410 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f760000bb70 con 0x7f7600003f90 2023-08-09T13:50:30.428 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:30.423+0000 7f7611d9b640 1 -- 172.21.15.43:0/333497410 --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f760000bd30 con 0x7f76140cc380 2023-08-09T13:50:30.428 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:30.424+0000 7f7612d9d640 1 --2- 172.21.15.43:0/333497410 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f7600004d00 0x7f7600005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:50:30.428 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:30.424+0000 7f761259c640 1 --2- 172.21.15.43:0/333497410 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f76140cc380 0x7f7600003a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:50:30.428 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:30.424+0000 7f7609d9b640 1 --2- 172.21.15.43:0/333497410 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f7600003f90 0x7f76000047c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:50:30.428 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:30.424+0000 7f7612d9d640 1 -- 172.21.15.43:0/333497410 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f76140cc380 msgr2=0x7f7600003a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-09T13:50:30.428 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:30.424+0000 7f7612d9d640 1 --2- 172.21.15.43:0/333497410 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f76140cc380 0x7f7600003a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:50:30.429 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:30.424+0000 7f7612d9d640 1 -- 172.21.15.43:0/333497410 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f7600003f90 msgr2=0x7f76000047c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-09T13:50:30.429 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:30.424+0000 7f7612d9d640 1 --2- 172.21.15.43:0/333497410 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f7600003f90 0x7f76000047c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:50:30.429 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:30.424+0000 7f7612d9d640 1 -- 172.21.15.43:0/333497410 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7600003520 con 0x7f7600004d00 2023-08-09T13:50:30.430 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:30.425+0000 7f7612d9d640 1 --2- 172.21.15.43:0/333497410 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f7600004d00 0x7f7600005180 secure :-1 s=READY pgs=1009 cs=0 l=1 rev1=1 crypto rx=0x7f760c00a830 tx=0x7f76000057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=14b029129ff0adb8 server_cookie=0 in_seq=0 out_seq=0 2023-08-09T13:50:30.430 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:30.426+0000 7f7611d9b640 1 -- 172.21.15.43:0/333497410 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f760c0072d0 con 0x7f7600004d00 2023-08-09T13:50:30.430 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:30.426+0000 7f7611d9b640 1 -- 172.21.15.43:0/333497410 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f760c02f070 con 0x7f7600004d00 2023-08-09T13:50:30.430 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:30.426+0000 7f7611d9b640 1 -- 172.21.15.43:0/333497410 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f760c0076f0 con 0x7f7600004d00 2023-08-09T13:50:30.430 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:30.426+0000 7f761886c640 1 -- 172.21.15.43:0/333497410 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f7600004d00 msgr2=0x7f7600005180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:50:30.431 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:30.426+0000 7f761886c640 1 --2- 172.21.15.43:0/333497410 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f7600004d00 0x7f7600005180 secure :-1 s=READY pgs=1009 cs=0 l=1 rev1=1 crypto rx=0x7f760c00a830 tx=0x7f76000057f0 comp rx=0 tx=0).stop 2023-08-09T13:50:30.431 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:30.426+0000 7f7612d9d640 1 -- 172.21.15.43:0/333497410 reap_dead start 2023-08-09T13:50:30.431 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:30.426+0000 7f761886c640 1 -- 172.21.15.43:0/333497410 shutdown_connections 2023-08-09T13:50:30.431 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:30.426+0000 7f761886c640 1 -- 172.21.15.43:0/333497410 >> 172.21.15.43:0/333497410 conn(0x7f76140ce0a0 msgr2=0x7f76140ce480 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-09T13:50:30.431 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:30.426+0000 7f761886c640 1 -- 172.21.15.43:0/333497410 shutdown_connections 2023-08-09T13:50:30.431 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:30.426+0000 7f761886c640 1 -- 172.21.15.43:0/333497410 wait complete. 2023-08-09T13:50:30.431 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:30.428+0000 7f761886c640 1 Processor -- start 2023-08-09T13:50:30.432 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:30.428+0000 7f761886c640 1 -- start start 2023-08-09T13:50:30.433 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:30.429+0000 7f761886c640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f7600004d00 0x7f76140c7f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:50:30.433 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:30.429+0000 7f761886c640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f7614160700 0x7f761415fdf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:50:30.433 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:30.429+0000 7f761886c640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f761406f620 0x7f761406fab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:50:30.433 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:30.429+0000 7f761886c640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f76140c8490 con 0x7f7614160700 2023-08-09T13:50:30.433 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:30.429+0000 7f761886c640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f761406fff0 con 0x7f7600004d00 2023-08-09T13:50:30.434 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:30.429+0000 7f761886c640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7614162620 con 0x7f761406f620 2023-08-09T13:50:30.434 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:30.430+0000 7f7609d9b640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f7614160700 0x7f761415fdf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:50:30.434 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:30.430+0000 7f7609d9b640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f7614160700 0x7f761415fdf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:35730/0 (socket says 172.21.15.43:35730) 2023-08-09T13:50:30.434 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:30.430+0000 7f7609d9b640 1 -- 172.21.15.43:0/2911525241 learned_addr learned my addr 172.21.15.43:0/2911525241 (peer_addr_for_me v2:172.21.15.43:0/0) 2023-08-09T13:50:30.434 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:30.430+0000 7f761259c640 1 --2- 172.21.15.43:0/2911525241 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f7600004d00 0x7f76140c7f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:50:30.434 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:30.430+0000 7f7609d9b640 1 -- 172.21.15.43:0/2911525241 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f761406f620 msgr2=0x7f761406fab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:50:30.434 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:30.430+0000 7f7609d9b640 1 --2- 172.21.15.43:0/2911525241 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f761406f620 0x7f761406fab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:50:30.434 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:30.430+0000 7f7609d9b640 1 -- 172.21.15.43:0/2911525241 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f7600004d00 msgr2=0x7f76140c7f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:50:30.435 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:30.430+0000 7f7609d9b640 1 --2- 172.21.15.43:0/2911525241 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f7600004d00 0x7f76140c7f50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:50:30.435 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:30.430+0000 7f7609d9b640 1 -- 172.21.15.43:0/2911525241 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7614162060 con 0x7f7614160700 2023-08-09T13:50:30.435 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:30.430+0000 7f7609d9b640 1 --2- 172.21.15.43:0/2911525241 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f7614160700 0x7f761415fdf0 secure :-1 s=READY pgs=1010 cs=0 l=1 rev1=1 crypto rx=0x7f75fc011630 tx=0x7f75fc011af0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-09T13:50:30.435 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:30.431+0000 7f760b7fe640 1 -- 172.21.15.43:0/2911525241 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f75fc015870 con 0x7f7614160700 2023-08-09T13:50:30.435 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:30.431+0000 7f760b7fe640 1 -- 172.21.15.43:0/2911525241 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f75fc0159d0 con 0x7f7614160700 2023-08-09T13:50:30.435 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:30.431+0000 7f760b7fe640 1 -- 172.21.15.43:0/2911525241 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f75fc015d40 con 0x7f7614160700 2023-08-09T13:50:30.435 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:30.431+0000 7f761886c640 1 -- 172.21.15.43:0/2911525241 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7614162340 con 0x7f7614160700 2023-08-09T13:50:30.436 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:30.431+0000 7f761886c640 1 -- 172.21.15.43:0/2911525241 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7614162d80 con 0x7f7614160700 2023-08-09T13:50:30.436 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:30.432+0000 7f761886c640 1 -- 172.21.15.43:0/2911525241 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f76140739f0 con 0x7f7614160700 2023-08-09T13:50:30.436 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:30.433+0000 7f760b7fe640 1 -- 172.21.15.43:0/2911525241 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 29) v1 ==== 88443+0+0 (secure 0 0 0) 0x7f75fc00f490 con 0x7f7614160700 2023-08-09T13:50:30.437 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:30.434+0000 7f760b7fe640 1 --2- 172.21.15.43:0/2911525241 >> [v2:172.21.15.203:6832/3325141227,v1:172.21.15.203:6833/3325141227] conn(0x7f75f406a170 0x7f75f406c630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:50:30.437 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:30.434+0000 7f760b7fe640 1 -- 172.21.15.43:0/2911525241 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(90..90 src has 1..90) v4 ==== 6426+0+0 (secure 0 0 0) 0x7f75fc088cd0 con 0x7f7614160700 2023-08-09T13:50:30.437 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:30.435+0000 7f761259c640 1 --2- 172.21.15.43:0/2911525241 >> [v2:172.21.15.203:6832/3325141227,v1:172.21.15.203:6833/3325141227] conn(0x7f75f406a170 0x7f75f406c630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:50:30.437 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:30.436+0000 7f761259c640 1 --2- 172.21.15.43:0/2911525241 >> [v2:172.21.15.203:6832/3325141227,v1:172.21.15.203:6833/3325141227] conn(0x7f75f406a170 0x7f75f406c630 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f7604007fd0 tx=0x7f7604009040 comp rx=0 tx=0).ready entity=mgr.5519 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-09T13:50:30.439 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:30.438+0000 7f760b7fe640 1 -- 172.21.15.43:0/2911525241 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178066 (secure 0 0 0) 0x7f75fc057bc0 con 0x7f7614160700 2023-08-09T13:50:30.669 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:30.668+0000 7f761886c640 1 -- 172.21.15.43:0/2911525241 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} v 0) v1 -- 0x7f76140cc830 con 0x7f7614160700 2023-08-09T13:50:31.258 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:31.257+0000 7f760b7fe640 1 -- 172.21.15.43:0/2911525241 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]=0 v34) v1 ==== 118+0+0 (secure 0 0 0) 0x7f75fc057750 con 0x7f7614160700 2023-08-09T13:50:31.263 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:31.260+0000 7f761886c640 1 -- 172.21.15.43:0/2911525241 >> [v2:172.21.15.203:6832/3325141227,v1:172.21.15.203:6833/3325141227] conn(0x7f75f406a170 msgr2=0x7f75f406c630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:50:31.263 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:31.260+0000 7f761886c640 1 --2- 172.21.15.43:0/2911525241 >> [v2:172.21.15.203:6832/3325141227,v1:172.21.15.203:6833/3325141227] conn(0x7f75f406a170 0x7f75f406c630 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f7604007fd0 tx=0x7f7604009040 comp rx=0 tx=0).stop 2023-08-09T13:50:31.263 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:31.261+0000 7f761886c640 1 -- 172.21.15.43:0/2911525241 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f7614160700 msgr2=0x7f761415fdf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:50:31.263 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:31.261+0000 7f761886c640 1 --2- 172.21.15.43:0/2911525241 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f7614160700 0x7f761415fdf0 secure :-1 s=READY pgs=1010 cs=0 l=1 rev1=1 crypto rx=0x7f75fc011630 tx=0x7f75fc011af0 comp rx=0 tx=0).stop 2023-08-09T13:50:31.263 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:31.261+0000 7f761886c640 1 -- 172.21.15.43:0/2911525241 shutdown_connections 2023-08-09T13:50:31.263 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:31.261+0000 7f761886c640 1 --2- 172.21.15.43:0/2911525241 >> [v2:172.21.15.203:6832/3325141227,v1:172.21.15.203:6833/3325141227] conn(0x7f75f406a170 0x7f75f406c630 unknown :-1 s=CLOSED pgs=61 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:50:31.264 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:31.261+0000 7f761886c640 1 --2- 172.21.15.43:0/2911525241 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f761406f620 0x7f761406fab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:50:31.264 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:31.261+0000 7f761886c640 1 --2- 172.21.15.43:0/2911525241 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f7614160700 0x7f761415fdf0 unknown :-1 s=CLOSED pgs=1010 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:50:31.264 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:31.261+0000 7f761886c640 1 --2- 172.21.15.43:0/2911525241 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f7600004d00 0x7f76140c7f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:50:31.264 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:31.261+0000 7f761886c640 1 -- 172.21.15.43:0/2911525241 >> 172.21.15.43:0/2911525241 conn(0x7f76140ce0a0 msgr2=0x7f7614144bd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-09T13:50:31.265 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:31.261+0000 7f761886c640 1 -- 172.21.15.43:0/2911525241 shutdown_connections 2023-08-09T13:50:31.265 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:31.261+0000 7f761886c640 1 -- 172.21.15.43:0/2911525241 wait complete. 2023-08-09T13:50:31.284 DEBUG:tasks.cephfs.filesystem:fs_config: {} 2023-08-09T13:50:31.285 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs allow_standby_replay false 2023-08-09T13:50:31.511 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:31.510+0000 7f2f9e80a640 1 Processor -- start 2023-08-09T13:50:31.512 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:31.510+0000 7f2f9e80a640 1 -- start start 2023-08-09T13:50:31.513 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:31.511+0000 7f2f9e80a640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f2f9806f2c0 0x7f2f9815b0f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:50:31.514 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:31.511+0000 7f2f9e80a640 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7f2f980c8470 0x7f2f980c8880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:50:31.514 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:31.511+0000 7f2f9e80a640 1 -- --> v1:172.21.15.203:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2f980c8dc0 con 0x7f2f98157ce0 2023-08-09T13:50:31.514 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:31.512+0000 7f2f9e80a640 1 -- --> v2:172.21.15.43:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2f981580c0 con 0x7f2f9806f2c0 2023-08-09T13:50:31.514 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:31.512+0000 7f2f9e80a640 1 -- --> v2:172.21.15.203:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2f980c92c0 con 0x7f2f980c8470 2023-08-09T13:50:31.514 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:31.512+0000 7f2f9d007640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f2f9806f2c0 0x7f2f9815b0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:50:31.514 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:31.512+0000 7f2f9d007640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f2f9806f2c0 0x7f2f9815b0f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:35744/0 (socket says 172.21.15.43:35744) 2023-08-09T13:50:31.515 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:31.512+0000 7f2f9d007640 1 -- 172.21.15.43:0/470320549 learned_addr learned my addr 172.21.15.43:0/470320549 (peer_addr_for_me v2:172.21.15.43:0/0) 2023-08-09T13:50:31.516 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:31.512+0000 7f2f9e009640 1 --2- 172.21.15.43:0/470320549 >> v2:172.21.15.203:3300/0 conn(0x7f2f980c8470 0x7f2f980c8880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:50:31.516 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:31.512+0000 7f2f9d007640 1 -- 172.21.15.43:0/470320549 >> v2:172.21.15.203:3300/0 conn(0x7f2f980c8470 msgr2=0x7f2f980c8880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-09T13:50:31.517 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:31.512+0000 7f2f9d007640 1 --2- 172.21.15.43:0/470320549 >> v2:172.21.15.203:3300/0 conn(0x7f2f980c8470 0x7f2f980c8880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:50:31.517 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:31.512+0000 7f2f9d007640 1 -- 172.21.15.43:0/470320549 >> v1:172.21.15.203:6789/0 conn(0x7f2f98157ce0 legacy=0x7f2f9806bc40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-09T13:50:31.517 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:31.513+0000 7f2f9d007640 1 -- 172.21.15.43:0/470320549 --> v2:172.21.15.43:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2f9806b480 con 0x7f2f9806f2c0 2023-08-09T13:50:31.517 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:31.513+0000 7f2f9e009640 1 --2- 172.21.15.43:0/470320549 >> v2:172.21.15.203:3300/0 conn(0x7f2f980c8470 0x7f2f980c8880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-08-09T13:50:31.517 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:31.514+0000 7f2f9d007640 1 --2- 172.21.15.43:0/470320549 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f2f9806f2c0 0x7f2f9815b0f0 secure :-1 s=READY pgs=1011 cs=0 l=1 rev1=1 crypto rx=0x7f2f980c9400 tx=0x7f2f8802b870 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c22cca248c6673b7 server_cookie=0 in_seq=0 out_seq=0 2023-08-09T13:50:31.517 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:31.514+0000 7f2f9c806640 1 -- 172.21.15.43:0/470320549 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2f880317d0 con 0x7f2f9806f2c0 2023-08-09T13:50:31.517 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:31.514+0000 7f2f9c806640 1 -- 172.21.15.43:0/470320549 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2f88031930 con 0x7f2f9806f2c0 2023-08-09T13:50:31.518 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:31.514+0000 7f2f9c806640 1 -- 172.21.15.43:0/470320549 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2f88031cc0 con 0x7f2f9806f2c0 2023-08-09T13:50:31.518 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:31.515+0000 7f2f9e80a640 1 -- 172.21.15.43:0/470320549 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f2f9806f2c0 msgr2=0x7f2f9815b0f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:50:31.518 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:31.515+0000 7f2f9e80a640 1 --2- 172.21.15.43:0/470320549 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f2f9806f2c0 0x7f2f9815b0f0 secure :-1 s=READY pgs=1011 cs=0 l=1 rev1=1 crypto rx=0x7f2f980c9400 tx=0x7f2f8802b870 comp rx=0 tx=0).stop 2023-08-09T13:50:31.518 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:31.515+0000 7f2f9e80a640 1 -- 172.21.15.43:0/470320549 shutdown_connections 2023-08-09T13:50:31.518 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:31.515+0000 7f2f9e80a640 1 --2- 172.21.15.43:0/470320549 >> v2:172.21.15.203:3300/0 conn(0x7f2f980c8470 0x7f2f980c8880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:50:31.518 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:31.515+0000 7f2f9e80a640 1 --2- 172.21.15.43:0/470320549 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f2f9806f2c0 0x7f2f9815b0f0 unknown :-1 s=CLOSED pgs=1011 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:50:31.518 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:31.515+0000 7f2f9e80a640 1 -- 172.21.15.43:0/470320549 >> 172.21.15.43:0/470320549 conn(0x7f2f980af9f0 msgr2=0x7f2f980afdd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-09T13:50:31.518 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:31.515+0000 7f2f9e80a640 1 -- 172.21.15.43:0/470320549 shutdown_connections 2023-08-09T13:50:31.519 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:31.515+0000 7f2f9e80a640 1 -- 172.21.15.43:0/470320549 wait complete. 2023-08-09T13:50:31.519 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:31.517+0000 7f2f9e80a640 1 Processor -- start 2023-08-09T13:50:31.520 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:31.517+0000 7f2f9e80a640 1 -- start start 2023-08-09T13:50:31.521 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:31.518+0000 7f2f9e80a640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f2f9806f2c0 0x7f2f98151a90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:50:31.521 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:31.518+0000 7f2f9e80a640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f2f980c8470 0x7f2f981594c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:50:31.521 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:31.518+0000 7f2f9e80a640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f2f98157ce0 0x7f2f980c6420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:50:31.521 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:31.518+0000 7f2f9e80a640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2f980c5f90 con 0x7f2f980c8470 2023-08-09T13:50:31.521 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:31.518+0000 7f2f9e80a640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2f980c5380 con 0x7f2f9806f2c0 2023-08-09T13:50:31.522 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:31.518+0000 7f2f9e80a640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2f980c5660 con 0x7f2f98157ce0 2023-08-09T13:50:31.522 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:31.519+0000 7f2f9d007640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f2f980c8470 0x7f2f981594c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:50:31.522 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:31.519+0000 7f2f9d007640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f2f980c8470 0x7f2f981594c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:35748/0 (socket says 172.21.15.43:35748) 2023-08-09T13:50:31.522 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:31.519+0000 7f2f9d007640 1 -- 172.21.15.43:0/968255865 learned_addr learned my addr 172.21.15.43:0/968255865 (peer_addr_for_me v2:172.21.15.43:0/0) 2023-08-09T13:50:31.522 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:31.519+0000 7f2f9d808640 1 --2- 172.21.15.43:0/968255865 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f2f9806f2c0 0x7f2f98151a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:50:31.522 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:31.519+0000 7f2f9e009640 1 --2- 172.21.15.43:0/968255865 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f2f98157ce0 0x7f2f980c6420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:50:31.522 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:31.519+0000 7f2f9d007640 1 -- 172.21.15.43:0/968255865 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f2f98157ce0 msgr2=0x7f2f980c6420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:50:31.522 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:31.519+0000 7f2f9d007640 1 --2- 172.21.15.43:0/968255865 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f2f98157ce0 0x7f2f980c6420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:50:31.523 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:31.519+0000 7f2f9d007640 1 -- 172.21.15.43:0/968255865 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f2f9806f2c0 msgr2=0x7f2f98151a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:50:31.523 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:31.519+0000 7f2f9d007640 1 --2- 172.21.15.43:0/968255865 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f2f9806f2c0 0x7f2f98151a90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:50:31.523 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:31.519+0000 7f2f9d007640 1 -- 172.21.15.43:0/968255865 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2f880094d0 con 0x7f2f980c8470 2023-08-09T13:50:31.523 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:31.520+0000 7f2f9d007640 1 --2- 172.21.15.43:0/968255865 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f2f980c8470 0x7f2f981594c0 secure :-1 s=READY pgs=1012 cs=0 l=1 rev1=1 crypto rx=0x7f2f88000c00 tx=0x7f2f880046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-09T13:50:31.524 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:31.520+0000 7f2f8e7fc640 1 -- 172.21.15.43:0/968255865 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2f88032970 con 0x7f2f980c8470 2023-08-09T13:50:31.524 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:31.520+0000 7f2f8e7fc640 1 -- 172.21.15.43:0/968255865 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2f88032ad0 con 0x7f2f980c8470 2023-08-09T13:50:31.524 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:31.520+0000 7f2f8e7fc640 1 -- 172.21.15.43:0/968255865 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2f88032e60 con 0x7f2f980c8470 2023-08-09T13:50:31.524 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:31.520+0000 7f2f9e80a640 1 -- 172.21.15.43:0/968255865 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2f980c5940 con 0x7f2f980c8470 2023-08-09T13:50:31.524 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:31.520+0000 7f2f9e80a640 1 -- 172.21.15.43:0/968255865 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2f98152200 con 0x7f2f980c8470 2023-08-09T13:50:31.525 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:31.521+0000 7f2f9e80a640 1 -- 172.21.15.43:0/968255865 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2f980c6100 con 0x7f2f980c8470 2023-08-09T13:50:31.525 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:31.522+0000 7f2f8e7fc640 1 -- 172.21.15.43:0/968255865 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 29) v1 ==== 88443+0+0 (secure 0 0 0) 0x7f2f88030070 con 0x7f2f980c8470 2023-08-09T13:50:31.525 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:31.524+0000 7f2f8e7fc640 1 --2- 172.21.15.43:0/968255865 >> [v2:172.21.15.203:6832/3325141227,v1:172.21.15.203:6833/3325141227] conn(0x7f2f7c06a170 0x7f2f7c06c630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:50:31.526 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:31.524+0000 7f2f8e7fc640 1 -- 172.21.15.43:0/968255865 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(90..90 src has 1..90) v4 ==== 6426+0+0 (secure 0 0 0) 0x7f2f880aa690 con 0x7f2f980c8470 2023-08-09T13:50:31.526 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:31.524+0000 7f2f9d808640 1 --2- 172.21.15.43:0/968255865 >> [v2:172.21.15.203:6832/3325141227,v1:172.21.15.203:6833/3325141227] conn(0x7f2f7c06a170 0x7f2f7c06c630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:50:31.527 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:31.526+0000 7f2f9d808640 1 --2- 172.21.15.43:0/968255865 >> [v2:172.21.15.203:6832/3325141227,v1:172.21.15.203:6833/3325141227] conn(0x7f2f7c06a170 0x7f2f7c06c630 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f2f94004ea0 tx=0x7f2f94009040 comp rx=0 tx=0).ready entity=mgr.5519 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-09T13:50:31.530 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:31.528+0000 7f2f8e7fc640 1 -- 172.21.15.43:0/968255865 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178066 (secure 0 0 0) 0x7f2f88075a90 con 0x7f2f980c8470 2023-08-09T13:50:31.768 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:31.766+0000 7f2f9e80a640 1 -- 172.21.15.43:0/968255865 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"} v 0) v1 -- 0x7f2f98071550 con 0x7f2f980c8470 2023-08-09T13:50:32.262 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:32.260+0000 7f2f8e7fc640 1 -- 172.21.15.43:0/968255865 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"}]=0 v35) v1 ==== 122+0+0 (secure 0 0 0) 0x7f2f880790b0 con 0x7f2f980c8470 2023-08-09T13:50:32.265 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:32.263+0000 7f2f9e80a640 1 -- 172.21.15.43:0/968255865 >> [v2:172.21.15.203:6832/3325141227,v1:172.21.15.203:6833/3325141227] conn(0x7f2f7c06a170 msgr2=0x7f2f7c06c630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:50:32.265 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:32.263+0000 7f2f9e80a640 1 --2- 172.21.15.43:0/968255865 >> [v2:172.21.15.203:6832/3325141227,v1:172.21.15.203:6833/3325141227] conn(0x7f2f7c06a170 0x7f2f7c06c630 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f2f94004ea0 tx=0x7f2f94009040 comp rx=0 tx=0).stop 2023-08-09T13:50:32.265 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:32.264+0000 7f2f9e80a640 1 -- 172.21.15.43:0/968255865 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f2f980c8470 msgr2=0x7f2f981594c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:50:32.266 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:32.264+0000 7f2f9e80a640 1 --2- 172.21.15.43:0/968255865 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f2f980c8470 0x7f2f981594c0 secure :-1 s=READY pgs=1012 cs=0 l=1 rev1=1 crypto rx=0x7f2f88000c00 tx=0x7f2f880046c0 comp rx=0 tx=0).stop 2023-08-09T13:50:32.266 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:32.264+0000 7f2f9e80a640 1 -- 172.21.15.43:0/968255865 shutdown_connections 2023-08-09T13:50:32.266 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:32.264+0000 7f2f9e80a640 1 --2- 172.21.15.43:0/968255865 >> [v2:172.21.15.203:6832/3325141227,v1:172.21.15.203:6833/3325141227] conn(0x7f2f7c06a170 0x7f2f7c06c630 unknown :-1 s=CLOSED pgs=62 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:50:32.266 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:32.264+0000 7f2f9e80a640 1 --2- 172.21.15.43:0/968255865 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f2f98157ce0 0x7f2f980c6420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:50:32.266 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:32.264+0000 7f2f9e80a640 1 --2- 172.21.15.43:0/968255865 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f2f980c8470 0x7f2f981594c0 unknown :-1 s=CLOSED pgs=1012 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:50:32.267 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:32.264+0000 7f2f9e80a640 1 --2- 172.21.15.43:0/968255865 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f2f9806f2c0 0x7f2f98151a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:50:32.267 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:32.264+0000 7f2f9e80a640 1 -- 172.21.15.43:0/968255865 >> 172.21.15.43:0/968255865 conn(0x7f2f980af9f0 msgr2=0x7f2f980cbb10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-09T13:50:32.267 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:32.264+0000 7f2f9e80a640 1 -- 172.21.15.43:0/968255865 shutdown_connections 2023-08-09T13:50:32.267 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:32.264+0000 7f2f9e80a640 1 -- 172.21.15.43:0/968255865 wait complete. 2023-08-09T13:50:32.285 DEBUG:tasks.cephfs.filesystem:Not Creating any subvols for filesystem 'cephfs' 2023-08-09T13:50:32.286 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-08-09T13:50:32.515 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:32.513+0000 7fd5e621f640 1 Processor -- start 2023-08-09T13:50:32.515 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:32.513+0000 7fd5e621f640 1 -- start start 2023-08-09T13:50:32.517 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:32.515+0000 7fd5e621f640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fd5e014a5c0 0x7fd5e0160f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:50:32.517 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:32.515+0000 7fd5e621f640 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7fd5e00c3130 0x7fd5e00cbc70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:50:32.517 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:32.515+0000 7fd5e621f640 1 -- --> v1:172.21.15.203:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd5e00c38a0 con 0x7fd5e00cc1b0 2023-08-09T13:50:32.517 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:32.515+0000 7fd5e621f640 1 -- --> v2:172.21.15.203:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd5e00c4420 con 0x7fd5e00c3130 2023-08-09T13:50:32.517 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:32.515+0000 7fd5e621f640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd5e00cc640 con 0x7fd5e014a5c0 2023-08-09T13:50:32.518 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:32.516+0000 7fd5df7fe640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fd5e014a5c0 0x7fd5e0160f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:50:32.518 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:32.516+0000 7fd5df7fe640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fd5e014a5c0 0x7fd5e0160f20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.203:3301/0 says I am v2:172.21.15.43:33504/0 (socket says 172.21.15.43:33504) 2023-08-09T13:50:32.518 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:32.516+0000 7fd5df7fe640 1 -- 172.21.15.43:0/2256226977 learned_addr learned my addr 172.21.15.43:0/2256226977 (peer_addr_for_me v2:172.21.15.43:0/0) 2023-08-09T13:50:32.518 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:32.516+0000 7fd5deffd640 1 --2- 172.21.15.43:0/2256226977 >> v2:172.21.15.203:3300/0 conn(0x7fd5e00c3130 0x7fd5e00cbc70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:50:32.519 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:32.517+0000 7fd5df7fe640 1 -- 172.21.15.43:0/2256226977 >> v2:172.21.15.203:3300/0 conn(0x7fd5e00c3130 msgr2=0x7fd5e00cbc70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-09T13:50:32.519 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:32.517+0000 7fd5df7fe640 1 --2- 172.21.15.43:0/2256226977 >> v2:172.21.15.203:3300/0 conn(0x7fd5e00c3130 0x7fd5e00cbc70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:50:32.519 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:32.517+0000 7fd5df7fe640 1 -- 172.21.15.43:0/2256226977 >> v1:172.21.15.203:6789/0 conn(0x7fd5e00cc1b0 legacy=0x7fd5e0164510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:50:32.519 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:32.517+0000 7fd5df7fe640 1 -- 172.21.15.43:0/2256226977 --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd5e00c3b80 con 0x7fd5e014a5c0 2023-08-09T13:50:32.521 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:32.518+0000 7fd5df7fe640 1 --2- 172.21.15.43:0/2256226977 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fd5e014a5c0 0x7fd5e0160f20 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7fd5d0009870 tx=0x7fd5d002b8f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=16aa26639ed45c29 server_cookie=0 in_seq=0 out_seq=0 2023-08-09T13:50:32.521 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:32.518+0000 7fd5de7fc640 1 -- 172.21.15.43:0/2256226977 <== mon.2 v2:172.21.15.203:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd5d0031640 con 0x7fd5e014a5c0 2023-08-09T13:50:32.521 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:32.519+0000 7fd5de7fc640 1 -- 172.21.15.43:0/2256226977 <== mon.2 v2:172.21.15.203:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd5d00317a0 con 0x7fd5e014a5c0 2023-08-09T13:50:32.521 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:32.519+0000 7fd5de7fc640 1 -- 172.21.15.43:0/2256226977 <== mon.2 v2:172.21.15.203:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd5d0031b30 con 0x7fd5e014a5c0 2023-08-09T13:50:32.521 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:32.519+0000 7fd5e621f640 1 -- 172.21.15.43:0/2256226977 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fd5e014a5c0 msgr2=0x7fd5e0160f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:50:32.522 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:32.519+0000 7fd5e621f640 1 --2- 172.21.15.43:0/2256226977 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fd5e014a5c0 0x7fd5e0160f20 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7fd5d0009870 tx=0x7fd5d002b8f0 comp rx=0 tx=0).stop 2023-08-09T13:50:32.522 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:32.519+0000 7fd5e621f640 1 -- 172.21.15.43:0/2256226977 shutdown_connections 2023-08-09T13:50:32.522 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:32.519+0000 7fd5e621f640 1 --2- 172.21.15.43:0/2256226977 >> v2:172.21.15.203:3300/0 conn(0x7fd5e00c3130 0x7fd5e00cbc70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:50:32.522 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:32.519+0000 7fd5e621f640 1 --2- 172.21.15.43:0/2256226977 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fd5e014a5c0 0x7fd5e0160f20 unknown :-1 s=CLOSED pgs=134 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:50:32.522 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:32.519+0000 7fd5e621f640 1 -- 172.21.15.43:0/2256226977 >> 172.21.15.43:0/2256226977 conn(0x7fd5e00ce0a0 msgr2=0x7fd5e00ce480 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-09T13:50:32.522 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:32.519+0000 7fd5e621f640 1 -- 172.21.15.43:0/2256226977 shutdown_connections 2023-08-09T13:50:32.522 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:32.519+0000 7fd5e621f640 1 -- 172.21.15.43:0/2256226977 wait complete. 2023-08-09T13:50:32.523 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:32.521+0000 7fd5e621f640 1 Processor -- start 2023-08-09T13:50:32.523 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:32.521+0000 7fd5e621f640 1 -- start start 2023-08-09T13:50:32.524 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:32.522+0000 7fd5e621f640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fd5e00c3130 0x7fd5e0161030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:50:32.525 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:32.522+0000 7fd5e621f640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fd5e00cc1b0 0x7fd5e0163750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:50:32.525 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:32.522+0000 7fd5e621f640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fd5e014a5c0 0x7fd5e01f7230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:50:32.525 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:32.522+0000 7fd5e621f640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd5e00c7fd0 con 0x7fd5e00c3130 2023-08-09T13:50:32.525 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:32.522+0000 7fd5e621f640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd5e0163420 con 0x7fd5e014a5c0 2023-08-09T13:50:32.525 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:32.522+0000 7fd5e621f640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd5e01f6060 con 0x7fd5e00cc1b0 2023-08-09T13:50:32.525 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:32.523+0000 7fd5df7fe640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fd5e00c3130 0x7fd5e0161030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:50:32.525 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:32.523+0000 7fd5df7fe640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fd5e00c3130 0x7fd5e0161030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:35762/0 (socket says 172.21.15.43:35762) 2023-08-09T13:50:32.526 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:32.523+0000 7fd5df7fe640 1 -- 172.21.15.43:0/4034669804 learned_addr learned my addr 172.21.15.43:0/4034669804 (peer_addr_for_me v2:172.21.15.43:0/0) 2023-08-09T13:50:32.526 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:32.523+0000 7fd5deffd640 1 --2- 172.21.15.43:0/4034669804 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fd5e00cc1b0 0x7fd5e0163750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:50:32.526 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:32.523+0000 7fd5dffff640 1 --2- 172.21.15.43:0/4034669804 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fd5e014a5c0 0x7fd5e01f7230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:50:32.527 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:32.523+0000 7fd5df7fe640 1 -- 172.21.15.43:0/4034669804 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fd5e00cc1b0 msgr2=0x7fd5e0163750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:50:32.527 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:32.523+0000 7fd5df7fe640 1 --2- 172.21.15.43:0/4034669804 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fd5e00cc1b0 0x7fd5e0163750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:50:32.528 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:32.523+0000 7fd5df7fe640 1 -- 172.21.15.43:0/4034669804 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fd5e014a5c0 msgr2=0x7fd5e01f7230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:50:32.528 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:32.523+0000 7fd5df7fe640 1 --2- 172.21.15.43:0/4034669804 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fd5e014a5c0 0x7fd5e01f7230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:50:32.528 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:32.523+0000 7fd5df7fe640 1 -- 172.21.15.43:0/4034669804 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd5d00094d0 con 0x7fd5e00c3130 2023-08-09T13:50:32.528 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:32.524+0000 7fd5df7fe640 1 --2- 172.21.15.43:0/4034669804 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fd5e00c3130 0x7fd5e0161030 secure :-1 s=READY pgs=1013 cs=0 l=1 rev1=1 crypto rx=0x7fd5d00090b0 tx=0x7fd5d00046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-09T13:50:32.528 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:32.524+0000 7fd5c7fff640 1 -- 172.21.15.43:0/4034669804 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd5d003a930 con 0x7fd5e00c3130 2023-08-09T13:50:32.528 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:32.524+0000 7fd5c7fff640 1 -- 172.21.15.43:0/4034669804 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd5d003aad0 con 0x7fd5e00c3130 2023-08-09T13:50:32.529 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:32.524+0000 7fd5c7fff640 1 -- 172.21.15.43:0/4034669804 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd5d003ae80 con 0x7fd5e00c3130 2023-08-09T13:50:32.529 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:32.524+0000 7fd5e621f640 1 -- 172.21.15.43:0/4034669804 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd5e00c65a0 con 0x7fd5e00c3130 2023-08-09T13:50:32.529 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:32.524+0000 7fd5e621f640 1 -- 172.21.15.43:0/4034669804 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd5e00c6b60 con 0x7fd5e00c3130 2023-08-09T13:50:32.529 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:32.525+0000 7fd5e621f640 1 -- 172.21.15.43:0/4034669804 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd5e00739f0 con 0x7fd5e00c3130 2023-08-09T13:50:32.530 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:32.526+0000 7fd5c7fff640 1 -- 172.21.15.43:0/4034669804 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 29) v1 ==== 88443+0+0 (secure 0 0 0) 0x7fd5d002f050 con 0x7fd5e00c3130 2023-08-09T13:50:32.530 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:32.527+0000 7fd5c7fff640 1 --2- 172.21.15.43:0/4034669804 >> [v2:172.21.15.203:6832/3325141227,v1:172.21.15.203:6833/3325141227] conn(0x7fd5cc06a170 0x7fd5cc06c630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:50:32.530 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:32.527+0000 7fd5c7fff640 1 -- 172.21.15.43:0/4034669804 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(90..90 src has 1..90) v4 ==== 6426+0+0 (secure 0 0 0) 0x7fd5d00aa8b0 con 0x7fd5e00c3130 2023-08-09T13:50:32.530 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:32.528+0000 7fd5deffd640 1 --2- 172.21.15.43:0/4034669804 >> [v2:172.21.15.203:6832/3325141227,v1:172.21.15.203:6833/3325141227] conn(0x7fd5cc06a170 0x7fd5cc06c630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:50:32.531 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:32.529+0000 7fd5deffd640 1 --2- 172.21.15.43:0/4034669804 >> [v2:172.21.15.203:6832/3325141227,v1:172.21.15.203:6833/3325141227] conn(0x7fd5cc06a170 0x7fd5cc06c630 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7fd5e00cbd20 tx=0x7fd5d4008040 comp rx=0 tx=0).ready entity=mgr.5519 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-09T13:50:32.534 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:32.532+0000 7fd5c7fff640 1 -- 172.21.15.43:0/4034669804 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178066 (secure 0 0 0) 0x7fd5d0075b80 con 0x7fd5e00c3130 2023-08-09T13:50:32.766 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:32.764+0000 7fd5e621f640 1 -- 172.21.15.43:0/4034669804 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fd5e00cc830 con 0x7fd5e00c3130 2023-08-09T13:50:32.767 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:32.766+0000 7fd5c7fff640 1 -- 172.21.15.43:0/4034669804 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 35 v35) v1 ==== 94+0+4880 (secure 0 0 0) 0x7fd5d00791a0 con 0x7fd5e00c3130 2023-08-09T13:50:32.768 INFO:teuthology.orchestra.run.smithi043.stdout: 2023-08-09T13:50:32.768 INFO:teuthology.orchestra.run.smithi043.stdout:{"epoch":35,"default_fscid":4,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5654,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.203:6837/1227342478","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6835","nonce":1227342478},{"type":"v1","addr":"172.21.15.203:6837","nonce":1227342478}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":30},{"gid":7005,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.43:6834/1431309163","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6832","nonce":1431309163},{"type":"v1","addr":"172.21.15.43:6834","nonce":1431309163}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":29},{"gid":7008,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.43:6835/611328204","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6833","nonce":611328204},{"type":"v1","addr":"172.21.15.43:6835","nonce":611328204}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":30}],"filesystems":[{"mdsmap":{"epoch":35,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-08-09T13:50:28.672581+0000","modified":"2023-08-09T13:50:32.253466+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":7011},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7011":{"gid":7011,"name":"d","rank":0,"incarnation":31,"state":"up:active","state_seq":3,"addr":"172.21.15.203:6836/3061388209","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6834","nonce":3061388209},{"type":"v1","addr":"172.21.15.203:6836","nonce":3061388209}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[11],"metadata_pool":10,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":4}]} 2023-08-09T13:50:32.769 INFO:teuthology.orchestra.run.smithi043.stderr:dumped fsmap epoch 35 2023-08-09T13:50:32.770 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:32.768+0000 7fd5e621f640 1 -- 172.21.15.43:0/4034669804 >> [v2:172.21.15.203:6832/3325141227,v1:172.21.15.203:6833/3325141227] conn(0x7fd5cc06a170 msgr2=0x7fd5cc06c630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:50:32.771 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:32.768+0000 7fd5e621f640 1 --2- 172.21.15.43:0/4034669804 >> [v2:172.21.15.203:6832/3325141227,v1:172.21.15.203:6833/3325141227] conn(0x7fd5cc06a170 0x7fd5cc06c630 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7fd5e00cbd20 tx=0x7fd5d4008040 comp rx=0 tx=0).stop 2023-08-09T13:50:32.771 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:32.768+0000 7fd5e621f640 1 -- 172.21.15.43:0/4034669804 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fd5e00c3130 msgr2=0x7fd5e0161030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:50:32.771 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:32.768+0000 7fd5e621f640 1 --2- 172.21.15.43:0/4034669804 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fd5e00c3130 0x7fd5e0161030 secure :-1 s=READY pgs=1013 cs=0 l=1 rev1=1 crypto rx=0x7fd5d00090b0 tx=0x7fd5d00046c0 comp rx=0 tx=0).stop 2023-08-09T13:50:32.771 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:32.768+0000 7fd5e621f640 1 -- 172.21.15.43:0/4034669804 shutdown_connections 2023-08-09T13:50:32.772 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:32.768+0000 7fd5e621f640 1 --2- 172.21.15.43:0/4034669804 >> [v2:172.21.15.203:6832/3325141227,v1:172.21.15.203:6833/3325141227] conn(0x7fd5cc06a170 0x7fd5cc06c630 unknown :-1 s=CLOSED pgs=63 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:50:32.772 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:32.768+0000 7fd5e621f640 1 --2- 172.21.15.43:0/4034669804 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fd5e014a5c0 0x7fd5e01f7230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:50:32.772 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:32.768+0000 7fd5e621f640 1 --2- 172.21.15.43:0/4034669804 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fd5e00cc1b0 0x7fd5e0163750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:50:32.773 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:32.768+0000 7fd5e621f640 1 --2- 172.21.15.43:0/4034669804 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fd5e00c3130 0x7fd5e0161030 unknown :-1 s=CLOSED pgs=1013 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:50:32.773 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:32.768+0000 7fd5e621f640 1 -- 172.21.15.43:0/4034669804 >> 172.21.15.43:0/4034669804 conn(0x7fd5e00ce0a0 msgr2=0x7fd5e006c3e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-09T13:50:32.773 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:32.768+0000 7fd5e621f640 1 -- 172.21.15.43:0/4034669804 shutdown_connections 2023-08-09T13:50:32.773 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:32.768+0000 7fd5e621f640 1 -- 172.21.15.43:0/4034669804 wait complete. 2023-08-09T13:50:32.791 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-08-09T13:50:33.019 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:33.017+0000 7f2b64a4b640 1 Processor -- start 2023-08-09T13:50:33.019 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:33.017+0000 7f2b64a4b640 1 -- start start 2023-08-09T13:50:33.021 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:33.019+0000 7f2b64a4b640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f2b600c96c0 0x7f2b6015ad20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:50:33.021 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:33.019+0000 7f2b64a4b640 1 -- --> v1:172.21.15.43:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2b600cdc70 con 0x7f2b600713b0 2023-08-09T13:50:33.021 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:33.019+0000 7f2b64a4b640 1 -- --> v1:172.21.15.203:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2b60070910 con 0x7f2b6015a6a0 2023-08-09T13:50:33.021 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:33.019+0000 7f2b64a4b640 1 -- --> v2:172.21.15.43:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2b600c88c0 con 0x7f2b600c96c0 2023-08-09T13:50:33.022 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:33.019+0000 7f2b5ffff640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f2b600c96c0 0x7f2b6015ad20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:50:33.022 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:33.019+0000 7f2b56ffd640 1 --1- >> v1:172.21.15.43:6789/0 conn(0x7f2b600713b0 0x7f2b600c8fb0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.43:6789/0 says I am v1:172.21.15.43:32974/0 (socket says 172.21.15.43:32974) 2023-08-09T13:50:33.022 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:33.019+0000 7f2b56ffd640 1 -- 172.21.15.43:0/107843659 learned_addr learned my addr 172.21.15.43:0/107843659 (peer_addr_for_me v1:172.21.15.43:0/0) 2023-08-09T13:50:33.025 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:33.020+0000 7f2b5ffff640 1 -- 172.21.15.43:0/107843659 >> v1:172.21.15.203:6789/0 conn(0x7f2b6015a6a0 legacy=0x7f2b600cbd90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-09T13:50:33.026 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:33.020+0000 7f2b5ffff640 1 -- 172.21.15.43:0/107843659 >> v1:172.21.15.43:6789/0 conn(0x7f2b600713b0 legacy=0x7f2b600c8fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:50:33.027 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:33.020+0000 7f2b5ffff640 1 -- 172.21.15.43:0/107843659 --> v2:172.21.15.43:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2b60070bb0 con 0x7f2b600c96c0 2023-08-09T13:50:33.027 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:33.020+0000 7f2b5effd640 1 -- 172.21.15.43:0/107843659 <== mon.0 v1:172.21.15.43:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3778571221 0 0) 0x7f2b600cdc70 con 0x7f2b600713b0 2023-08-09T13:50:33.027 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:33.021+0000 7f2b5ffff640 1 --2- 172.21.15.43:0/107843659 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f2b600c96c0 0x7f2b6015ad20 secure :-1 s=READY pgs=1015 cs=0 l=1 rev1=1 crypto rx=0x7f2b580097a0 tx=0x7f2b5802b840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=145a6ca0a7b9de6c server_cookie=0 in_seq=0 out_seq=0 2023-08-09T13:50:33.028 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:33.021+0000 7f2b5effd640 1 -- 172.21.15.43:0/107843659 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2b5802e070 con 0x7f2b600c96c0 2023-08-09T13:50:33.028 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:33.021+0000 7f2b5effd640 1 -- 172.21.15.43:0/107843659 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2b580315a0 con 0x7f2b600c96c0 2023-08-09T13:50:33.028 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:33.021+0000 7f2b5effd640 1 -- 172.21.15.43:0/107843659 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2b58031930 con 0x7f2b600c96c0 2023-08-09T13:50:33.028 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:33.022+0000 7f2b64a4b640 1 -- 172.21.15.43:0/107843659 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f2b600c96c0 msgr2=0x7f2b6015ad20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:50:33.029 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:33.022+0000 7f2b64a4b640 1 --2- 172.21.15.43:0/107843659 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f2b600c96c0 0x7f2b6015ad20 secure :-1 s=READY pgs=1015 cs=0 l=1 rev1=1 crypto rx=0x7f2b580097a0 tx=0x7f2b5802b840 comp rx=0 tx=0).stop 2023-08-09T13:50:33.029 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:33.022+0000 7f2b64a4b640 1 -- 172.21.15.43:0/107843659 shutdown_connections 2023-08-09T13:50:33.029 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:33.022+0000 7f2b64a4b640 1 --2- 172.21.15.43:0/107843659 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f2b600c96c0 0x7f2b6015ad20 unknown :-1 s=CLOSED pgs=1015 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:50:33.030 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:33.022+0000 7f2b64a4b640 1 -- 172.21.15.43:0/107843659 >> 172.21.15.43:0/107843659 conn(0x7f2b600cde90 msgr2=0x7f2b600ce2a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-09T13:50:33.030 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:33.022+0000 7f2b64a4b640 1 -- 172.21.15.43:0/107843659 shutdown_connections 2023-08-09T13:50:33.030 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:33.022+0000 7f2b64a4b640 1 -- 172.21.15.43:0/107843659 wait complete. 2023-08-09T13:50:33.033 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:33.024+0000 7f2b64a4b640 1 Processor -- start 2023-08-09T13:50:33.033 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:33.025+0000 7f2b64a4b640 1 -- start start 2023-08-09T13:50:33.034 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:33.026+0000 7f2b64a4b640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f2b600713b0 0x7f2b6006c2d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:50:33.034 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:33.026+0000 7f2b64a4b640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f2b600c96c0 0x7f2b601f2440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:50:33.034 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:33.026+0000 7f2b64a4b640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f2b6015a6a0 0x7f2b600aaff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:50:33.034 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:33.026+0000 7f2b64a4b640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2b6006beb0 con 0x7f2b600c96c0 2023-08-09T13:50:33.034 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:33.026+0000 7f2b64a4b640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2b600abc40 con 0x7f2b600713b0 2023-08-09T13:50:33.034 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:33.026+0000 7f2b64a4b640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2b601f2980 con 0x7f2b6015a6a0 2023-08-09T13:50:33.034 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:33.026+0000 7f2b56ffd640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f2b600c96c0 0x7f2b601f2440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:50:33.035 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:33.026+0000 7f2b56ffd640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f2b600c96c0 0x7f2b601f2440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:35778/0 (socket says 172.21.15.43:35778) 2023-08-09T13:50:33.035 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:33.026+0000 7f2b56ffd640 1 -- 172.21.15.43:0/2516323341 learned_addr learned my addr 172.21.15.43:0/2516323341 (peer_addr_for_me v2:172.21.15.43:0/0) 2023-08-09T13:50:33.035 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:33.026+0000 7f2b5f7fe640 1 --2- 172.21.15.43:0/2516323341 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f2b600713b0 0x7f2b6006c2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:50:33.035 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:33.026+0000 7f2b5ffff640 1 --2- 172.21.15.43:0/2516323341 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f2b6015a6a0 0x7f2b600aaff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:50:33.035 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:33.027+0000 7f2b56ffd640 1 -- 172.21.15.43:0/2516323341 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f2b6015a6a0 msgr2=0x7f2b600aaff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:50:33.035 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:33.027+0000 7f2b56ffd640 1 --2- 172.21.15.43:0/2516323341 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f2b6015a6a0 0x7f2b600aaff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:50:33.036 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:33.027+0000 7f2b56ffd640 1 -- 172.21.15.43:0/2516323341 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f2b600713b0 msgr2=0x7f2b6006c2d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:50:33.036 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:33.027+0000 7f2b56ffd640 1 --2- 172.21.15.43:0/2516323341 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f2b600713b0 0x7f2b6006c2d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:50:33.036 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:33.027+0000 7f2b56ffd640 1 -- 172.21.15.43:0/2516323341 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2b58009400 con 0x7f2b600c96c0 2023-08-09T13:50:33.036 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:33.027+0000 7f2b56ffd640 1 --2- 172.21.15.43:0/2516323341 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f2b600c96c0 0x7f2b601f2440 secure :-1 s=READY pgs=1016 cs=0 l=1 rev1=1 crypto rx=0x7f2b4800c430 tx=0x7f2b4800c8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-09T13:50:33.036 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:33.027+0000 7f2b5cff9640 1 -- 172.21.15.43:0/2516323341 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2b480074e0 con 0x7f2b600c96c0 2023-08-09T13:50:33.037 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:33.027+0000 7f2b5cff9640 1 -- 172.21.15.43:0/2516323341 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2b48007660 con 0x7f2b600c96c0 2023-08-09T13:50:33.037 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:33.027+0000 7f2b5cff9640 1 -- 172.21.15.43:0/2516323341 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2b48004a20 con 0x7f2b600c96c0 2023-08-09T13:50:33.037 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:33.027+0000 7f2b64a4b640 1 -- 172.21.15.43:0/2516323341 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2b6006b230 con 0x7f2b600c96c0 2023-08-09T13:50:33.037 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:33.027+0000 7f2b64a4b640 1 -- 172.21.15.43:0/2516323341 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2b6006b7f0 con 0x7f2b600c96c0 2023-08-09T13:50:33.037 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:33.028+0000 7f2b64a4b640 1 -- 172.21.15.43:0/2516323341 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2b6006bff0 con 0x7f2b600c96c0 2023-08-09T13:50:33.037 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:33.029+0000 7f2b5cff9640 1 -- 172.21.15.43:0/2516323341 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 29) v1 ==== 88443+0+0 (secure 0 0 0) 0x7f2b48004b80 con 0x7f2b600c96c0 2023-08-09T13:50:33.037 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:33.031+0000 7f2b5cff9640 1 --2- 172.21.15.43:0/2516323341 >> [v2:172.21.15.203:6832/3325141227,v1:172.21.15.203:6833/3325141227] conn(0x7f2b4006a170 0x7f2b4006c630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:50:33.038 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:33.031+0000 7f2b5cff9640 1 -- 172.21.15.43:0/2516323341 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(90..90 src has 1..90) v4 ==== 6426+0+0 (secure 0 0 0) 0x7f2b48088660 con 0x7f2b600c96c0 2023-08-09T13:50:33.038 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:33.032+0000 7f2b5f7fe640 1 --2- 172.21.15.43:0/2516323341 >> [v2:172.21.15.203:6832/3325141227,v1:172.21.15.203:6833/3325141227] conn(0x7f2b4006a170 0x7f2b4006c630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:50:33.038 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:33.033+0000 7f2b5f7fe640 1 --2- 172.21.15.43:0/2516323341 >> [v2:172.21.15.203:6832/3325141227,v1:172.21.15.203:6833/3325141227] conn(0x7f2b4006a170 0x7f2b4006c630 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f2b50005520 tx=0x7f2b50009040 comp rx=0 tx=0).ready entity=mgr.5519 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-09T13:50:33.039 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:33.035+0000 7f2b5cff9640 1 -- 172.21.15.43:0/2516323341 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178066 (secure 0 0 0) 0x7f2b6006bff0 con 0x7f2b600c96c0 2023-08-09T13:50:33.233 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:33.232+0000 7f2b64a4b640 1 -- 172.21.15.43:0/2516323341 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f2b6006bff0 con 0x7f2b600c96c0 2023-08-09T13:50:33.236 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:33.235+0000 7f2b5cff9640 1 -- 172.21.15.43:0/2516323341 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v90) v1 ==== 74+0+18555 (secure 0 0 0) 0x7f2b6006bff0 con 0x7f2b600c96c0 2023-08-09T13:50:33.237 INFO:teuthology.orchestra.run.smithi043.stdout: 2023-08-09T13:50:33.238 INFO:teuthology.orchestra.run.smithi043.stdout:{"epoch":90,"fsid":"daac87ee-1337-4250-a955-aea4e408a39b","created":"2023-08-09T13:33:41.701198+0000","modified":"2023-08-09T13:50:30.199714+0000","last_up_change":"2023-08-09T13:33:53.783453+0000","last_in_change":"2023-08-09T13:33:45.148904+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":11,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":"rbd","create_time":"2023-08-09T13:33:54.365137+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":".mgr","create_time":"2023-08-09T13:33:54.831803+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":10,"pool_name":"cephfs_metadata","create_time":"2023-08-09T13:50:27.051781+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"89","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":2.5,"score_stable":2.5,"optimal_score":1,"raw_score_acting":2.5,"raw_score_stable":2.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":11,"pool_name":"cephfs_data","create_time":"2023-08-09T13:50:27.692933+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"90","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"cbd49ecd-e896-4fbd-864e-5c37a829b165","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6800","nonce":502667335},{"type":"v1","addr":"172.21.15.43:6801","nonce":502667335}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6802","nonce":502667335},{"type":"v1","addr":"172.21.15.43:6803","nonce":502667335}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6806","nonce":502667335},{"type":"v1","addr":"172.21.15.43:6807","nonce":502667335}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6804","nonce":502667335},{"type":"v1","addr":"172.21.15.43:6805","nonce":502667335}]},"public_addr":"172.21.15.43:6801/502667335","cluster_addr":"172.21.15.43:6803/502667335","heartbeat_back_addr":"172.21.15.43:6807/502667335","heartbeat_front_addr":"172.21.15.43:6805/502667335","state":["exists","up"]},{"osd":1,"uuid":"75c7e0fe-1ec7-4fe8-a2f2-3661a65b8500","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6808","nonce":3289134610},{"type":"v1","addr":"172.21.15.43:6809","nonce":3289134610}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6810","nonce":3289134610},{"type":"v1","addr":"172.21.15.43:6812","nonce":3289134610}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6821","nonce":3289134610},{"type":"v1","addr":"172.21.15.43:6824","nonce":3289134610}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6815","nonce":3289134610},{"type":"v1","addr":"172.21.15.43:6818","nonce":3289134610}]},"public_addr":"172.21.15.43:6809/3289134610","cluster_addr":"172.21.15.43:6812/3289134610","heartbeat_back_addr":"172.21.15.43:6824/3289134610","heartbeat_front_addr":"172.21.15.43:6818/3289134610","state":["exists","up"]},{"osd":2,"uuid":"bb5c41e9-7f5f-4125-9b01-3bb8bfc2ea83","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6811","nonce":4079709736},{"type":"v1","addr":"172.21.15.43:6814","nonce":4079709736}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6817","nonce":4079709736},{"type":"v1","addr":"172.21.15.43:6820","nonce":4079709736}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6829","nonce":4079709736},{"type":"v1","addr":"172.21.15.43:6831","nonce":4079709736}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6823","nonce":4079709736},{"type":"v1","addr":"172.21.15.43:6826","nonce":4079709736}]},"public_addr":"172.21.15.43:6814/4079709736","cluster_addr":"172.21.15.43:6820/4079709736","heartbeat_back_addr":"172.21.15.43:6831/4079709736","heartbeat_front_addr":"172.21.15.43:6826/4079709736","state":["exists","up"]},{"osd":3,"uuid":"bfe4c8b4-694a-465c-87e6-2b2aab852613","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6813","nonce":1709538759},{"type":"v1","addr":"172.21.15.43:6816","nonce":1709538759}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6819","nonce":1709538759},{"type":"v1","addr":"172.21.15.43:6822","nonce":1709538759}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6828","nonce":1709538759},{"type":"v1","addr":"172.21.15.43:6830","nonce":1709538759}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6825","nonce":1709538759},{"type":"v1","addr":"172.21.15.43:6827","nonce":1709538759}]},"public_addr":"172.21.15.43:6816/1709538759","cluster_addr":"172.21.15.43:6822/1709538759","heartbeat_back_addr":"172.21.15.43:6830/1709538759","heartbeat_front_addr":"172.21.15.43:6827/1709538759","state":["exists","up"]},{"osd":4,"uuid":"fa2e8a77-aba1-4756-853e-0d174a5bdeba","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6800","nonce":4196866481},{"type":"v1","addr":"172.21.15.203:6801","nonce":4196866481}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6802","nonce":4196866481},{"type":"v1","addr":"172.21.15.203:6803","nonce":4196866481}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6806","nonce":4196866481},{"type":"v1","addr":"172.21.15.203:6807","nonce":4196866481}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6804","nonce":4196866481},{"type":"v1","addr":"172.21.15.203:6805","nonce":4196866481}]},"public_addr":"172.21.15.203:6801/4196866481","cluster_addr":"172.21.15.203:6803/4196866481","heartbeat_back_addr":"172.21.15.203:6807/4196866481","heartbeat_front_addr":"172.21.15.203:6805/4196866481","state":["exists","up"]},{"osd":5,"uuid":"81811e4b-4d24-4f51-a82c-f1d700a2a00d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6816","nonce":3059290081},{"type":"v1","addr":"172.21.15.203:6817","nonce":3059290081}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6818","nonce":3059290081},{"type":"v1","addr":"172.21.15.203:6819","nonce":3059290081}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6822","nonce":3059290081},{"type":"v1","addr":"172.21.15.203:6823","nonce":3059290081}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6820","nonce":3059290081},{"type":"v1","addr":"172.21.15.203:6821","nonce":3059290081}]},"public_addr":"172.21.15.203:6817/3059290081","cluster_addr":"172.21.15.203:6819/3059290081","heartbeat_back_addr":"172.21.15.203:6823/3059290081","heartbeat_front_addr":"172.21.15.203:6821/3059290081","state":["exists","up"]},{"osd":6,"uuid":"307bac4d-6a5b-443d-af75-56d695d7674f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6824","nonce":512155569},{"type":"v1","addr":"172.21.15.203:6825","nonce":512155569}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6826","nonce":512155569},{"type":"v1","addr":"172.21.15.203:6827","nonce":512155569}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6830","nonce":512155569},{"type":"v1","addr":"172.21.15.203:6831","nonce":512155569}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6828","nonce":512155569},{"type":"v1","addr":"172.21.15.203:6829","nonce":512155569}]},"public_addr":"172.21.15.203:6825/512155569","cluster_addr":"172.21.15.203:6827/512155569","heartbeat_back_addr":"172.21.15.203:6831/512155569","heartbeat_front_addr":"172.21.15.203:6829/512155569","state":["exists","up"]},{"osd":7,"uuid":"a95dc7ca-9f1b-4295-af70-68a22a2d7d5e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6808","nonce":3331450438},{"type":"v1","addr":"172.21.15.203:6809","nonce":3331450438}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6810","nonce":3331450438},{"type":"v1","addr":"172.21.15.203:6811","nonce":3331450438}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6814","nonce":3331450438},{"type":"v1","addr":"172.21.15.203:6815","nonce":3331450438}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6812","nonce":3331450438},{"type":"v1","addr":"172.21.15.203:6813","nonce":3331450438}]},"public_addr":"172.21.15.203:6809/3331450438","cluster_addr":"172.21.15.203:6811/3331450438","heartbeat_back_addr":"172.21.15.203:6815/3331450438","heartbeat_front_addr":"172.21.15.203:6813/3331450438","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-09T13:33:50.547652+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-09T13:33:50.589469+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-09T13:33:50.561724+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-09T13:33:50.599650+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-09T13:33:50.411855+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-09T13:33:50.219681+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-09T13:33:50.504664+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-09T13:33:50.441248+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":7,"snaps":[{"begin":104,"length":1},{"begin":221,"length":1}]}],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-08-09T13:50:33.240 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:33.238+0000 7f2b64a4b640 1 -- 172.21.15.43:0/2516323341 >> [v2:172.21.15.203:6832/3325141227,v1:172.21.15.203:6833/3325141227] conn(0x7f2b4006a170 msgr2=0x7f2b4006c630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:50:33.240 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:33.238+0000 7f2b64a4b640 1 --2- 172.21.15.43:0/2516323341 >> [v2:172.21.15.203:6832/3325141227,v1:172.21.15.203:6833/3325141227] conn(0x7f2b4006a170 0x7f2b4006c630 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f2b50005520 tx=0x7f2b50009040 comp rx=0 tx=0).stop 2023-08-09T13:50:33.240 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:33.238+0000 7f2b64a4b640 1 -- 172.21.15.43:0/2516323341 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f2b600c96c0 msgr2=0x7f2b601f2440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:50:33.240 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:33.238+0000 7f2b64a4b640 1 --2- 172.21.15.43:0/2516323341 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f2b600c96c0 0x7f2b601f2440 secure :-1 s=READY pgs=1016 cs=0 l=1 rev1=1 crypto rx=0x7f2b4800c430 tx=0x7f2b4800c8f0 comp rx=0 tx=0).stop 2023-08-09T13:50:33.240 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:33.238+0000 7f2b64a4b640 1 -- 172.21.15.43:0/2516323341 shutdown_connections 2023-08-09T13:50:33.241 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:33.238+0000 7f2b64a4b640 1 --2- 172.21.15.43:0/2516323341 >> [v2:172.21.15.203:6832/3325141227,v1:172.21.15.203:6833/3325141227] conn(0x7f2b4006a170 0x7f2b4006c630 unknown :-1 s=CLOSED pgs=64 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:50:33.241 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:33.238+0000 7f2b64a4b640 1 --2- 172.21.15.43:0/2516323341 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f2b6015a6a0 0x7f2b600aaff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:50:33.241 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:33.238+0000 7f2b64a4b640 1 --2- 172.21.15.43:0/2516323341 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f2b600c96c0 0x7f2b601f2440 unknown :-1 s=CLOSED pgs=1016 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:50:33.241 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:33.238+0000 7f2b64a4b640 1 --2- 172.21.15.43:0/2516323341 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f2b600713b0 0x7f2b6006c2d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:50:33.242 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:33.238+0000 7f2b64a4b640 1 -- 172.21.15.43:0/2516323341 >> 172.21.15.43:0/2516323341 conn(0x7f2b600cde90 msgr2=0x7f2b600cc950 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-09T13:50:33.242 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:33.238+0000 7f2b64a4b640 1 -- 172.21.15.43:0/2516323341 shutdown_connections 2023-08-09T13:50:33.242 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:33.238+0000 7f2b64a4b640 1 -- 172.21.15.43:0/2516323341 wait complete. 2023-08-09T13:50:33.258 INFO:tasks.cephfs.filesystem.ceph_manager:waiting for clean 2023-08-09T13:50:33.258 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-08-09T13:50:33.480 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:33.479+0000 7f702ffff640 1 Processor -- start 2023-08-09T13:50:33.481 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:33.479+0000 7f702ffff640 1 -- start start 2023-08-09T13:50:33.482 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:33.481+0000 7f702ffff640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f70300713b0 0x7f70300c8fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:50:33.482 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:33.481+0000 7f702ffff640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f70300c94f0 0x7f703015ad20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:50:33.482 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:33.481+0000 7f702ffff640 1 -- --> v1:172.21.15.43:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f70300c9940 con 0x7f703015a6a0 2023-08-09T13:50:33.483 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:33.481+0000 7f702ffff640 1 -- --> v2:172.21.15.43:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f70300cdc70 con 0x7f70300713b0 2023-08-09T13:50:33.483 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:33.481+0000 7f702ffff640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7030070910 con 0x7f70300c94f0 2023-08-09T13:50:33.483 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:33.481+0000 7f702e7fc640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f70300713b0 0x7f70300c8fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:50:33.484 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:33.481+0000 7f702effd640 1 --1- >> v1:172.21.15.43:6789/0 conn(0x7f703015a6a0 0x7f70300cbd90 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.43:6789/0 says I am v1:172.21.15.43:32984/0 (socket says 172.21.15.43:32984) 2023-08-09T13:50:33.484 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:33.481+0000 7f702effd640 1 -- 172.21.15.43:0/4089098031 learned_addr learned my addr 172.21.15.43:0/4089098031 (peer_addr_for_me v1:172.21.15.43:0/0) 2023-08-09T13:50:33.484 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:33.481+0000 7f702f7fe640 1 --2- 172.21.15.43:0/4089098031 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f70300c94f0 0x7f703015ad20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:50:33.484 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:33.482+0000 7f702dffb640 1 -- 172.21.15.43:0/4089098031 <== mon.0 v1:172.21.15.43:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2777558771 0 0) 0x7f7020001110 con 0x7f703015a6a0 2023-08-09T13:50:33.484 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:33.482+0000 7f702dffb640 1 -- 172.21.15.43:0/4089098031 --> v1:172.21.15.43:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f7018003520 con 0x7f703015a6a0 2023-08-09T13:50:33.484 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:33.482+0000 7f702e7fc640 1 -- 172.21.15.43:0/4089098031 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f70300c94f0 msgr2=0x7f703015ad20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-09T13:50:33.485 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:33.482+0000 7f702e7fc640 1 --2- 172.21.15.43:0/4089098031 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f70300c94f0 0x7f703015ad20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:50:33.485 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:33.482+0000 7f702e7fc640 1 -- 172.21.15.43:0/4089098031 >> v1:172.21.15.43:6789/0 conn(0x7f703015a6a0 legacy=0x7f70300cbd90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:50:33.485 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:33.482+0000 7f702e7fc640 1 -- 172.21.15.43:0/4089098031 --> v2:172.21.15.43:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f70300c88c0 con 0x7f70300713b0 2023-08-09T13:50:33.486 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:33.483+0000 7f702e7fc640 1 --2- 172.21.15.43:0/4089098031 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f70300713b0 0x7f70300c8fb0 secure :-1 s=READY pgs=1018 cs=0 l=1 rev1=1 crypto rx=0x7f70240097a0 tx=0x7f702402b820 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f54249aed0e1a74c server_cookie=0 in_seq=0 out_seq=0 2023-08-09T13:50:33.486 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:33.483+0000 7f702dffb640 1 -- 172.21.15.43:0/4089098031 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7024031780 con 0x7f70300713b0 2023-08-09T13:50:33.486 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:33.483+0000 7f702dffb640 1 -- 172.21.15.43:0/4089098031 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f70240318e0 con 0x7f70300713b0 2023-08-09T13:50:33.486 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:33.483+0000 7f702dffb640 1 -- 172.21.15.43:0/4089098031 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7024031c70 con 0x7f70300713b0 2023-08-09T13:50:33.486 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:33.484+0000 7f702ffff640 1 -- 172.21.15.43:0/4089098031 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f70300713b0 msgr2=0x7f70300c8fb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:50:33.486 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:33.484+0000 7f702ffff640 1 --2- 172.21.15.43:0/4089098031 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f70300713b0 0x7f70300c8fb0 secure :-1 s=READY pgs=1018 cs=0 l=1 rev1=1 crypto rx=0x7f70240097a0 tx=0x7f702402b820 comp rx=0 tx=0).stop 2023-08-09T13:50:33.487 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:33.484+0000 7f702ffff640 1 -- 172.21.15.43:0/4089098031 shutdown_connections 2023-08-09T13:50:33.487 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:33.484+0000 7f702ffff640 1 --2- 172.21.15.43:0/4089098031 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f70300c94f0 0x7f703015ad20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:50:33.487 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:33.484+0000 7f702ffff640 1 --2- 172.21.15.43:0/4089098031 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f70300713b0 0x7f70300c8fb0 unknown :-1 s=CLOSED pgs=1018 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:50:33.487 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:33.484+0000 7f702ffff640 1 -- 172.21.15.43:0/4089098031 >> 172.21.15.43:0/4089098031 conn(0x7f70300cde90 msgr2=0x7f70300ce2a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-09T13:50:33.487 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:33.484+0000 7f702ffff640 1 -- 172.21.15.43:0/4089098031 shutdown_connections 2023-08-09T13:50:33.487 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:33.484+0000 7f702ffff640 1 -- 172.21.15.43:0/4089098031 wait complete. 2023-08-09T13:50:33.488 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:33.486+0000 7f702ffff640 1 Processor -- start 2023-08-09T13:50:33.488 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:33.486+0000 7f702ffff640 1 -- start start 2023-08-09T13:50:33.488 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:33.487+0000 7f702ffff640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f70300713b0 0x7f7030151a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:50:33.489 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:33.487+0000 7f702ffff640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f70300c94f0 0x7f703015d080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:50:33.489 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:33.487+0000 7f702ffff640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f703015a6a0 0x7f70300ac6f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:50:33.489 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:33.487+0000 7f702ffff640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f703006c290 con 0x7f703015a6a0 2023-08-09T13:50:33.489 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:33.487+0000 7f702ffff640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f703006bcd0 con 0x7f70300c94f0 2023-08-09T13:50:33.489 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:33.487+0000 7f702ffff640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f703006bfb0 con 0x7f70300713b0 2023-08-09T13:50:33.489 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:33.487+0000 7f702f7fe640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f703015a6a0 0x7f70300ac6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:50:33.490 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:33.488+0000 7f702f7fe640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f703015a6a0 0x7f70300ac6f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:35800/0 (socket says 172.21.15.43:35800) 2023-08-09T13:50:33.491 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:33.488+0000 7f702f7fe640 1 -- 172.21.15.43:0/3766772299 learned_addr learned my addr 172.21.15.43:0/3766772299 (peer_addr_for_me v2:172.21.15.43:0/0) 2023-08-09T13:50:33.491 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:33.488+0000 7f702effd640 1 --2- 172.21.15.43:0/3766772299 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f70300713b0 0x7f7030151a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:50:33.491 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:33.488+0000 7f702e7fc640 1 --2- 172.21.15.43:0/3766772299 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f70300c94f0 0x7f703015d080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:50:33.491 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:33.488+0000 7f702f7fe640 1 -- 172.21.15.43:0/3766772299 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f70300713b0 msgr2=0x7f7030151a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:50:33.491 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:33.488+0000 7f702f7fe640 1 --2- 172.21.15.43:0/3766772299 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f70300713b0 0x7f7030151a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:50:33.491 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:33.488+0000 7f702f7fe640 1 -- 172.21.15.43:0/3766772299 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f70300c94f0 msgr2=0x7f703015d080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:50:33.491 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:33.488+0000 7f702f7fe640 1 --2- 172.21.15.43:0/3766772299 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f70300c94f0 0x7f703015d080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:50:33.492 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:33.488+0000 7f702f7fe640 1 -- 172.21.15.43:0/3766772299 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7024009400 con 0x7f703015a6a0 2023-08-09T13:50:33.492 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:33.488+0000 7f702f7fe640 1 --2- 172.21.15.43:0/3766772299 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f703015a6a0 0x7f70300ac6f0 secure :-1 s=READY pgs=1019 cs=0 l=1 rev1=1 crypto rx=0x7f7028007e40 tx=0x7f702800e4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-09T13:50:33.492 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:33.489+0000 7f70177fe640 1 -- 172.21.15.43:0/3766772299 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7028014620 con 0x7f703015a6a0 2023-08-09T13:50:33.492 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:33.489+0000 7f70177fe640 1 -- 172.21.15.43:0/3766772299 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f702800ee90 con 0x7f703015a6a0 2023-08-09T13:50:33.492 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:33.489+0000 7f70177fe640 1 -- 172.21.15.43:0/3766772299 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f70280149e0 con 0x7f703015a6a0 2023-08-09T13:50:33.492 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:33.489+0000 7f702ffff640 1 -- 172.21.15.43:0/3766772299 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7030150050 con 0x7f703015a6a0 2023-08-09T13:50:33.492 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:33.489+0000 7f702ffff640 1 -- 172.21.15.43:0/3766772299 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7030151fb0 con 0x7f703015a6a0 2023-08-09T13:50:33.493 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:33.490+0000 7f702ffff640 1 -- 172.21.15.43:0/3766772299 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f703014feb0 con 0x7f703015a6a0 2023-08-09T13:50:33.493 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:33.491+0000 7f70177fe640 1 -- 172.21.15.43:0/3766772299 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 29) v1 ==== 88443+0+0 (secure 0 0 0) 0x7f7028014b40 con 0x7f703015a6a0 2023-08-09T13:50:33.494 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:33.492+0000 7f70177fe640 1 --2- 172.21.15.43:0/3766772299 >> [v2:172.21.15.203:6832/3325141227,v1:172.21.15.203:6833/3325141227] conn(0x7f701c06a170 0x7f701c06c630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:50:33.494 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:33.492+0000 7f70177fe640 1 -- 172.21.15.43:0/3766772299 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(90..90 src has 1..90) v4 ==== 6426+0+0 (secure 0 0 0) 0x7f7028089310 con 0x7f703015a6a0 2023-08-09T13:50:33.494 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:33.493+0000 7f702effd640 1 --2- 172.21.15.43:0/3766772299 >> [v2:172.21.15.203:6832/3325141227,v1:172.21.15.203:6833/3325141227] conn(0x7f701c06a170 0x7f701c06c630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:50:33.495 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:33.494+0000 7f702effd640 1 --2- 172.21.15.43:0/3766772299 >> [v2:172.21.15.203:6832/3325141227,v1:172.21.15.203:6833/3325141227] conn(0x7f701c06a170 0x7f701c06c630 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f7020007fd0 tx=0x7f7020009040 comp rx=0 tx=0).ready entity=mgr.5519 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-09T13:50:33.497 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:33.496+0000 7f70177fe640 1 -- 172.21.15.43:0/3766772299 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178066 (secure 0 0 0) 0x7f703014feb0 con 0x7f703015a6a0 2023-08-09T13:50:33.681 DEBUG:teuthology.orchestra.run.smithi043:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-08-09T13:50:33.691 DEBUG:teuthology.orchestra.run.smithi203:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-08-09T13:50:33.691 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:33.684+0000 7f702ffff640 1 -- 172.21.15.43:0/3766772299 --> [v2:172.21.15.203:6832/3325141227,v1:172.21.15.203:6833/3325141227] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f7030070e90 con 0x7f701c06a170 2023-08-09T13:50:33.715 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:33.714+0000 7f70177fe640 1 -- 172.21.15.43:0/3766772299 <== mgr.5519 v2:172.21.15.203:6832/3325141227 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+307350 (secure 0 0 0) 0x7f7020001d10 con 0x7f701c06a170 2023-08-09T13:50:33.718 INFO:teuthology.orchestra.run.smithi043.stdout: 2023-08-09T13:50:33.720 INFO:teuthology.orchestra.run.smithi043.stderr:dumped all 2023-08-09T13:50:33.721 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:33.719+0000 7f702ffff640 1 -- 172.21.15.43:0/3766772299 >> [v2:172.21.15.203:6832/3325141227,v1:172.21.15.203:6833/3325141227] conn(0x7f701c06a170 msgr2=0x7f701c06c630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:50:33.721 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:33.719+0000 7f702ffff640 1 --2- 172.21.15.43:0/3766772299 >> [v2:172.21.15.203:6832/3325141227,v1:172.21.15.203:6833/3325141227] conn(0x7f701c06a170 0x7f701c06c630 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f7020007fd0 tx=0x7f7020009040 comp rx=0 tx=0).stop 2023-08-09T13:50:33.722 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:33.719+0000 7f702ffff640 1 -- 172.21.15.43:0/3766772299 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f703015a6a0 msgr2=0x7f70300ac6f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:50:33.722 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:33.719+0000 7f702ffff640 1 --2- 172.21.15.43:0/3766772299 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f703015a6a0 0x7f70300ac6f0 secure :-1 s=READY pgs=1019 cs=0 l=1 rev1=1 crypto rx=0x7f7028007e40 tx=0x7f702800e4b0 comp rx=0 tx=0).stop 2023-08-09T13:50:33.722 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:33.719+0000 7f702ffff640 1 -- 172.21.15.43:0/3766772299 shutdown_connections 2023-08-09T13:50:33.722 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:33.719+0000 7f702ffff640 1 --2- 172.21.15.43:0/3766772299 >> [v2:172.21.15.203:6832/3325141227,v1:172.21.15.203:6833/3325141227] conn(0x7f701c06a170 0x7f701c06c630 unknown :-1 s=CLOSED pgs=65 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:50:33.722 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:33.719+0000 7f702ffff640 1 --2- 172.21.15.43:0/3766772299 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f703015a6a0 0x7f70300ac6f0 unknown :-1 s=CLOSED pgs=1019 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:50:33.722 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:33.719+0000 7f702ffff640 1 --2- 172.21.15.43:0/3766772299 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f70300c94f0 0x7f703015d080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:50:33.722 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:33.719+0000 7f702ffff640 1 --2- 172.21.15.43:0/3766772299 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f70300713b0 0x7f7030151a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:50:33.723 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:33.719+0000 7f702ffff640 1 -- 172.21.15.43:0/3766772299 >> 172.21.15.43:0/3766772299 conn(0x7f70300cde90 msgr2=0x7f7030152b90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-09T13:50:33.723 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:33.719+0000 7f702ffff640 1 -- 172.21.15.43:0/3766772299 shutdown_connections 2023-08-09T13:50:33.723 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:33.719+0000 7f702ffff640 1 -- 172.21.15.43:0/3766772299 wait complete. 2023-08-09T13:50:33.739 INFO:teuthology.orchestra.run.smithi043.stdout:{"pg_ready":true,"pg_map":{"version":28,"stamp":"2023-08-09T13:50:32.231945+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":500,"num_read_kb":924,"num_write":239,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":161,"ondisk_log_size":161,"up":210,"acting":210,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":61,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":278924,"kb_used_data":9504,"kb_used_omap":13,"kb_used_meta":269298,"kb_avail":754695796,"statfs":{"total":773094113280,"available":772808495104,"internally_reserved":0,"allocated":9732096,"data_stored":5732158,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":14248,"internal_metadata":275761240},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":32,"apply_latency_ms":32,"commit_latency_ns":32000000,"apply_latency_ns":32000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"5.076138"},"pg_stats":[{"pgid":"11.3f","version":"0'0","reported_seq":8,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-08-09T13:50:29.233104+0000","last_change":"2023-08-09T13:50:29.233104+0000","last_active":"2023-08-09T13:50:29.233104+0000","last_peered":"2023-08-09T13:50:29.233104+0000","last_clean":"2023-08-09T13:50:29.233104+0000","last_became_active":"2023-08-09T13:50:29.232795+0000","last_became_peered":"2023-08-09T13:50:29.232795+0000","last_unstale":"2023-08-09T13:50:29.233104+0000","last_undegraded":"2023-08-09T13:50:29.233104+0000","last_fullsized":"2023-08-09T13:50:29.233104+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:50:28.150437+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:50:28.150437+0000","last_clean_scrub_stamp":"2023-08-09T13:50:28.150437+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T17:05:45.752848+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"11.3e","version":"0'0","reported_seq":3,"reported_epoch":89,"state":"creating+activating","last_fresh":"2023-08-09T13:50:29.231349+0000","last_change":"2023-08-09T13:50:29.231349+0000","last_active":"2023-08-09T13:50:28.150437+0000","last_peered":"2023-08-09T13:50:28.150437+0000","last_clean":"2023-08-09T13:50:28.150437+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-09T13:50:29.231349+0000","last_undegraded":"2023-08-09T13:50:29.231349+0000","last_fullsized":"2023-08-09T13:50:29.231349+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:50:28.150437+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:50:28.150437+0000","last_clean_scrub_stamp":"2023-08-09T13:50:28.150437+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T18:13:21.364990+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":["6","4"],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.3d","version":"0'0","reported_seq":9,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-08-09T13:50:29.233399+0000","last_change":"2023-08-09T13:50:29.225958+0000","last_active":"2023-08-09T13:50:29.233399+0000","last_peered":"2023-08-09T13:50:29.233399+0000","last_clean":"2023-08-09T13:50:29.233399+0000","last_became_active":"2023-08-09T13:50:29.225536+0000","last_became_peered":"2023-08-09T13:50:29.225536+0000","last_unstale":"2023-08-09T13:50:29.233399+0000","last_undegraded":"2023-08-09T13:50:29.233399+0000","last_fullsized":"2023-08-09T13:50:29.233399+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:50:28.150437+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:50:28.150437+0000","last_clean_scrub_stamp":"2023-08-09T13:50:28.150437+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T22:25:44.631240+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.3c","version":"0'0","reported_seq":1,"reported_epoch":88,"state":"creating+peering","last_fresh":"2023-08-09T13:50:28.188648+0000","last_change":"2023-08-09T13:50:28.188648+0000","last_active":"2023-08-09T13:50:28.150437+0000","last_peered":"2023-08-09T13:50:28.150437+0000","last_clean":"2023-08-09T13:50:28.150437+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-09T13:50:28.188648+0000","last_undegraded":"2023-08-09T13:50:28.188648+0000","last_fullsized":"2023-08-09T13:50:28.188648+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:50:28.150437+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:50:28.150437+0000","last_clean_scrub_stamp":"2023-08-09T13:50:28.150437+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.3b","version":"0'0","reported_seq":3,"reported_epoch":89,"state":"creating+activating","last_fresh":"2023-08-09T13:50:29.221854+0000","last_change":"2023-08-09T13:50:29.221854+0000","last_active":"2023-08-09T13:50:28.150437+0000","last_peered":"2023-08-09T13:50:28.150437+0000","last_clean":"2023-08-09T13:50:28.150437+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-09T13:50:29.221854+0000","last_undegraded":"2023-08-09T13:50:29.221854+0000","last_fullsized":"2023-08-09T13:50:29.221854+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:50:28.150437+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:50:28.150437+0000","last_clean_scrub_stamp":"2023-08-09T13:50:28.150437+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T22:54:27.616846+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":["2","7"],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.3a","version":"0'0","reported_seq":8,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-08-09T13:50:29.250572+0000","last_change":"2023-08-09T13:50:29.250572+0000","last_active":"2023-08-09T13:50:29.250572+0000","last_peered":"2023-08-09T13:50:29.250572+0000","last_clean":"2023-08-09T13:50:29.250572+0000","last_became_active":"2023-08-09T13:50:29.250267+0000","last_became_peered":"2023-08-09T13:50:29.250267+0000","last_unstale":"2023-08-09T13:50:29.250572+0000","last_undegraded":"2023-08-09T13:50:29.250572+0000","last_fullsized":"2023-08-09T13:50:29.250572+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:50:28.150437+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:50:28.150437+0000","last_clean_scrub_stamp":"2023-08-09T13:50:28.150437+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T17:21:38.689120+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.39","version":"0'0","reported_seq":3,"reported_epoch":89,"state":"creating+activating","last_fresh":"2023-08-09T13:50:29.220028+0000","last_change":"2023-08-09T13:50:29.220028+0000","last_active":"2023-08-09T13:50:28.150437+0000","last_peered":"2023-08-09T13:50:28.150437+0000","last_clean":"2023-08-09T13:50:28.150437+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-09T13:50:29.220028+0000","last_undegraded":"2023-08-09T13:50:29.220028+0000","last_fullsized":"2023-08-09T13:50:29.220028+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:50:28.150437+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:50:28.150437+0000","last_clean_scrub_stamp":"2023-08-09T13:50:28.150437+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T23:19:57.122069+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":["5","1"],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.38","version":"0'0","reported_seq":8,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-08-09T13:50:29.246308+0000","last_change":"2023-08-09T13:50:29.246308+0000","last_active":"2023-08-09T13:50:29.246308+0000","last_peered":"2023-08-09T13:50:29.246308+0000","last_clean":"2023-08-09T13:50:29.246308+0000","last_became_active":"2023-08-09T13:50:29.245268+0000","last_became_peered":"2023-08-09T13:50:29.245268+0000","last_unstale":"2023-08-09T13:50:29.246308+0000","last_undegraded":"2023-08-09T13:50:29.246308+0000","last_fullsized":"2023-08-09T13:50:29.246308+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:50:28.150437+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:50:28.150437+0000","last_clean_scrub_stamp":"2023-08-09T13:50:28.150437+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T18:15:31.389069+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.37","version":"0'0","reported_seq":3,"reported_epoch":89,"state":"creating+activating","last_fresh":"2023-08-09T13:50:29.223778+0000","last_change":"2023-08-09T13:50:29.223778+0000","last_active":"2023-08-09T13:50:28.150437+0000","last_peered":"2023-08-09T13:50:28.150437+0000","last_clean":"2023-08-09T13:50:28.150437+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-09T13:50:29.223778+0000","last_undegraded":"2023-08-09T13:50:29.223778+0000","last_fullsized":"2023-08-09T13:50:29.223778+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:50:28.150437+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:50:28.150437+0000","last_clean_scrub_stamp":"2023-08-09T13:50:28.150437+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T15:38:50.620059+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":["2","3"],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.36","version":"0'0","reported_seq":3,"reported_epoch":89,"state":"creating+activating","last_fresh":"2023-08-09T13:50:29.207900+0000","last_change":"2023-08-09T13:50:29.207900+0000","last_active":"2023-08-09T13:50:28.150437+0000","last_peered":"2023-08-09T13:50:28.150437+0000","last_clean":"2023-08-09T13:50:28.150437+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-09T13:50:29.207900+0000","last_undegraded":"2023-08-09T13:50:29.207900+0000","last_fullsized":"2023-08-09T13:50:29.207900+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:50:28.150437+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:50:28.150437+0000","last_clean_scrub_stamp":"2023-08-09T13:50:28.150437+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T21:32:28.121459+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":["4","2"],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.35","version":"0'0","reported_seq":8,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-08-09T13:50:29.237075+0000","last_change":"2023-08-09T13:50:29.237075+0000","last_active":"2023-08-09T13:50:29.237075+0000","last_peered":"2023-08-09T13:50:29.237075+0000","last_clean":"2023-08-09T13:50:29.237075+0000","last_became_active":"2023-08-09T13:50:29.236351+0000","last_became_peered":"2023-08-09T13:50:29.236351+0000","last_unstale":"2023-08-09T13:50:29.237075+0000","last_undegraded":"2023-08-09T13:50:29.237075+0000","last_fullsized":"2023-08-09T13:50:29.237075+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:50:28.150437+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:50:28.150437+0000","last_clean_scrub_stamp":"2023-08-09T13:50:28.150437+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T18:17:12.675765+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"11.34","version":"0'0","reported_seq":1,"reported_epoch":88,"state":"creating+peering","last_fresh":"2023-08-09T13:50:28.188207+0000","last_change":"2023-08-09T13:50:28.188207+0000","last_active":"2023-08-09T13:50:28.150437+0000","last_peered":"2023-08-09T13:50:28.150437+0000","last_clean":"2023-08-09T13:50:28.150437+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-09T13:50:28.188207+0000","last_undegraded":"2023-08-09T13:50:28.188207+0000","last_fullsized":"2023-08-09T13:50:28.188207+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:50:28.150437+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:50:28.150437+0000","last_clean_scrub_stamp":"2023-08-09T13:50:28.150437+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.33","version":"0'0","reported_seq":3,"reported_epoch":89,"state":"creating+activating","last_fresh":"2023-08-09T13:50:29.207581+0000","last_change":"2023-08-09T13:50:29.207581+0000","last_active":"2023-08-09T13:50:28.150437+0000","last_peered":"2023-08-09T13:50:28.150437+0000","last_clean":"2023-08-09T13:50:28.150437+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-09T13:50:29.207581+0000","last_undegraded":"2023-08-09T13:50:29.207581+0000","last_fullsized":"2023-08-09T13:50:29.207581+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:50:28.150437+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:50:28.150437+0000","last_clean_scrub_stamp":"2023-08-09T13:50:28.150437+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T18:17:12.675765+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":["0","6"],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.32","version":"0'0","reported_seq":3,"reported_epoch":89,"state":"creating+activating","last_fresh":"2023-08-09T13:50:29.224087+0000","last_change":"2023-08-09T13:50:29.224087+0000","last_active":"2023-08-09T13:50:28.150437+0000","last_peered":"2023-08-09T13:50:28.150437+0000","last_clean":"2023-08-09T13:50:28.150437+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-09T13:50:29.224087+0000","last_undegraded":"2023-08-09T13:50:29.224087+0000","last_fullsized":"2023-08-09T13:50:29.224087+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:50:28.150437+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:50:28.150437+0000","last_clean_scrub_stamp":"2023-08-09T13:50:28.150437+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T23:57:06.314706+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":["2","5"],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.31","version":"0'0","reported_seq":3,"reported_epoch":89,"state":"creating+activating","last_fresh":"2023-08-09T13:50:29.206432+0000","last_change":"2023-08-09T13:50:29.206432+0000","last_active":"2023-08-09T13:50:28.150437+0000","last_peered":"2023-08-09T13:50:28.150437+0000","last_clean":"2023-08-09T13:50:28.150437+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-09T13:50:29.206432+0000","last_undegraded":"2023-08-09T13:50:29.206432+0000","last_fullsized":"2023-08-09T13:50:29.206432+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:50:28.150437+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:50:28.150437+0000","last_clean_scrub_stamp":"2023-08-09T13:50:28.150437+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T22:39:17.713654+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":["0","2"],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.30","version":"0'0","reported_seq":3,"reported_epoch":89,"state":"creating+activating","last_fresh":"2023-08-09T13:50:29.208580+0000","last_change":"2023-08-09T13:50:29.208580+0000","last_active":"2023-08-09T13:50:28.150437+0000","last_peered":"2023-08-09T13:50:28.150437+0000","last_clean":"2023-08-09T13:50:28.150437+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-09T13:50:29.208580+0000","last_undegraded":"2023-08-09T13:50:29.208580+0000","last_fullsized":"2023-08-09T13:50:29.208580+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:50:28.150437+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:50:28.150437+0000","last_clean_scrub_stamp":"2023-08-09T13:50:28.150437+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T15:14:37.215348+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":["4","6"],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.2f","version":"0'0","reported_seq":3,"reported_epoch":89,"state":"creating+activating","last_fresh":"2023-08-09T13:50:29.223293+0000","last_change":"2023-08-09T13:50:29.223293+0000","last_active":"2023-08-09T13:50:28.150437+0000","last_peered":"2023-08-09T13:50:28.150437+0000","last_clean":"2023-08-09T13:50:28.150437+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-09T13:50:29.223293+0000","last_undegraded":"2023-08-09T13:50:29.223293+0000","last_fullsized":"2023-08-09T13:50:29.223293+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:50:28.150437+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:50:28.150437+0000","last_clean_scrub_stamp":"2023-08-09T13:50:28.150437+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T22:44:32.845197+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":["2","3"],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.2e","version":"0'0","reported_seq":1,"reported_epoch":88,"state":"creating+peering","last_fresh":"2023-08-09T13:50:28.187070+0000","last_change":"2023-08-09T13:50:28.187070+0000","last_active":"2023-08-09T13:50:28.150437+0000","last_peered":"2023-08-09T13:50:28.150437+0000","last_clean":"2023-08-09T13:50:28.150437+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-09T13:50:28.187070+0000","last_undegraded":"2023-08-09T13:50:28.187070+0000","last_fullsized":"2023-08-09T13:50:28.187070+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:50:28.150437+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:50:28.150437+0000","last_clean_scrub_stamp":"2023-08-09T13:50:28.150437+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.2d","version":"0'0","reported_seq":9,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-08-09T13:50:29.248797+0000","last_change":"2023-08-09T13:50:29.241875+0000","last_active":"2023-08-09T13:50:29.248797+0000","last_peered":"2023-08-09T13:50:29.248797+0000","last_clean":"2023-08-09T13:50:29.248797+0000","last_became_active":"2023-08-09T13:50:29.241532+0000","last_became_peered":"2023-08-09T13:50:29.241532+0000","last_unstale":"2023-08-09T13:50:29.248797+0000","last_undegraded":"2023-08-09T13:50:29.248797+0000","last_fullsized":"2023-08-09T13:50:29.248797+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:50:28.150437+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:50:28.150437+0000","last_clean_scrub_stamp":"2023-08-09T13:50:28.150437+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-11T00:33:32.657666+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.2c","version":"0'0","reported_seq":3,"reported_epoch":89,"state":"creating+activating","last_fresh":"2023-08-09T13:50:29.205840+0000","last_change":"2023-08-09T13:50:29.205840+0000","last_active":"2023-08-09T13:50:28.150437+0000","last_peered":"2023-08-09T13:50:28.150437+0000","last_clean":"2023-08-09T13:50:28.150437+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-09T13:50:29.205840+0000","last_undegraded":"2023-08-09T13:50:29.205840+0000","last_fullsized":"2023-08-09T13:50:29.205840+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:50:28.150437+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:50:28.150437+0000","last_clean_scrub_stamp":"2023-08-09T13:50:28.150437+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T16:53:03.610834+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":["0","5"],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.2b","version":"0'0","reported_seq":9,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-08-09T13:50:29.233438+0000","last_change":"2023-08-09T13:50:29.227706+0000","last_active":"2023-08-09T13:50:29.233438+0000","last_peered":"2023-08-09T13:50:29.233438+0000","last_clean":"2023-08-09T13:50:29.233438+0000","last_became_active":"2023-08-09T13:50:29.226933+0000","last_became_peered":"2023-08-09T13:50:29.226933+0000","last_unstale":"2023-08-09T13:50:29.233438+0000","last_undegraded":"2023-08-09T13:50:29.233438+0000","last_fullsized":"2023-08-09T13:50:29.233438+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:50:28.150437+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:50:28.150437+0000","last_clean_scrub_stamp":"2023-08-09T13:50:28.150437+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T17:57:56.069305+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.2a","version":"0'0","reported_seq":3,"reported_epoch":89,"state":"creating+activating","last_fresh":"2023-08-09T13:50:29.207312+0000","last_change":"2023-08-09T13:50:29.207312+0000","last_active":"2023-08-09T13:50:28.150437+0000","last_peered":"2023-08-09T13:50:28.150437+0000","last_clean":"2023-08-09T13:50:28.150437+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-09T13:50:29.207312+0000","last_undegraded":"2023-08-09T13:50:29.207312+0000","last_fullsized":"2023-08-09T13:50:29.207312+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:50:28.150437+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:50:28.150437+0000","last_clean_scrub_stamp":"2023-08-09T13:50:28.150437+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T19:03:45.295280+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":["4","2"],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.29","version":"0'0","reported_seq":8,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-08-09T13:50:29.232548+0000","last_change":"2023-08-09T13:50:29.232548+0000","last_active":"2023-08-09T13:50:29.232548+0000","last_peered":"2023-08-09T13:50:29.232548+0000","last_clean":"2023-08-09T13:50:29.232548+0000","last_became_active":"2023-08-09T13:50:29.232205+0000","last_became_peered":"2023-08-09T13:50:29.232205+0000","last_unstale":"2023-08-09T13:50:29.232548+0000","last_undegraded":"2023-08-09T13:50:29.232548+0000","last_fullsized":"2023-08-09T13:50:29.232548+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:50:28.150437+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:50:28.150437+0000","last_clean_scrub_stamp":"2023-08-09T13:50:28.150437+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T21:11:03.639832+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"11.28","version":"0'0","reported_seq":3,"reported_epoch":89,"state":"creating+activating","last_fresh":"2023-08-09T13:50:29.217553+0000","last_change":"2023-08-09T13:50:29.217553+0000","last_active":"2023-08-09T13:50:28.150437+0000","last_peered":"2023-08-09T13:50:28.150437+0000","last_clean":"2023-08-09T13:50:28.150437+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-09T13:50:29.217553+0000","last_undegraded":"2023-08-09T13:50:29.217553+0000","last_fullsized":"2023-08-09T13:50:29.217553+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:50:28.150437+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:50:28.150437+0000","last_clean_scrub_stamp":"2023-08-09T13:50:28.150437+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T22:39:17.713654+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":["7","6"],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"11.27","version":"0'0","reported_seq":3,"reported_epoch":89,"state":"creating+activating","last_fresh":"2023-08-09T13:50:29.208200+0000","last_change":"2023-08-09T13:50:29.208200+0000","last_active":"2023-08-09T13:50:28.150437+0000","last_peered":"2023-08-09T13:50:28.150437+0000","last_clean":"2023-08-09T13:50:28.150437+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-09T13:50:29.208200+0000","last_undegraded":"2023-08-09T13:50:29.208200+0000","last_fullsized":"2023-08-09T13:50:29.208200+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:50:28.150437+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:50:28.150437+0000","last_clean_scrub_stamp":"2023-08-09T13:50:28.150437+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T14:52:59.954591+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":["4","7"],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.26","version":"0'0","reported_seq":3,"reported_epoch":89,"state":"creating+activating","last_fresh":"2023-08-09T13:50:29.223117+0000","last_change":"2023-08-09T13:50:29.223117+0000","last_active":"2023-08-09T13:50:28.150437+0000","last_peered":"2023-08-09T13:50:28.150437+0000","last_clean":"2023-08-09T13:50:28.150437+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-09T13:50:29.223117+0000","last_undegraded":"2023-08-09T13:50:29.223117+0000","last_fullsized":"2023-08-09T13:50:29.223117+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:50:28.150437+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:50:28.150437+0000","last_clean_scrub_stamp":"2023-08-09T13:50:28.150437+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T21:18:31.512056+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":["5","6"],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.25","version":"0'0","reported_seq":8,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-08-09T13:50:29.233554+0000","last_change":"2023-08-09T13:50:29.233554+0000","last_active":"2023-08-09T13:50:29.233554+0000","last_peered":"2023-08-09T13:50:29.233554+0000","last_clean":"2023-08-09T13:50:29.233554+0000","last_became_active":"2023-08-09T13:50:29.233153+0000","last_became_peered":"2023-08-09T13:50:29.233153+0000","last_unstale":"2023-08-09T13:50:29.233554+0000","last_undegraded":"2023-08-09T13:50:29.233554+0000","last_fullsized":"2023-08-09T13:50:29.233554+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:50:28.150437+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:50:28.150437+0000","last_clean_scrub_stamp":"2023-08-09T13:50:28.150437+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T21:36:11.302948+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.24","version":"0'0","reported_seq":3,"reported_epoch":89,"state":"creating+activating","last_fresh":"2023-08-09T13:50:29.207497+0000","last_change":"2023-08-09T13:50:29.207497+0000","last_active":"2023-08-09T13:50:28.150437+0000","last_peered":"2023-08-09T13:50:28.150437+0000","last_clean":"2023-08-09T13:50:28.150437+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-09T13:50:29.207497+0000","last_undegraded":"2023-08-09T13:50:29.207497+0000","last_fullsized":"2023-08-09T13:50:29.207497+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:50:28.150437+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:50:28.150437+0000","last_clean_scrub_stamp":"2023-08-09T13:50:28.150437+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T18:47:18.558930+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":["4","2"],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.23","version":"0'0","reported_seq":8,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-08-09T13:50:29.233513+0000","last_change":"2023-08-09T13:50:29.233513+0000","last_active":"2023-08-09T13:50:29.233513+0000","last_peered":"2023-08-09T13:50:29.233513+0000","last_clean":"2023-08-09T13:50:29.233513+0000","last_became_active":"2023-08-09T13:50:29.233199+0000","last_became_peered":"2023-08-09T13:50:29.233199+0000","last_unstale":"2023-08-09T13:50:29.233513+0000","last_undegraded":"2023-08-09T13:50:29.233513+0000","last_fullsized":"2023-08-09T13:50:29.233513+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:50:28.150437+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:50:28.150437+0000","last_clean_scrub_stamp":"2023-08-09T13:50:28.150437+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T21:47:07.987840+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.22","version":"0'0","reported_seq":3,"reported_epoch":89,"state":"creating+activating","last_fresh":"2023-08-09T13:50:29.219724+0000","last_change":"2023-08-09T13:50:29.219724+0000","last_active":"2023-08-09T13:50:28.150437+0000","last_peered":"2023-08-09T13:50:28.150437+0000","last_clean":"2023-08-09T13:50:28.150437+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-09T13:50:29.219724+0000","last_undegraded":"2023-08-09T13:50:29.219724+0000","last_fullsized":"2023-08-09T13:50:29.219724+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:50:28.150437+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:50:28.150437+0000","last_clean_scrub_stamp":"2023-08-09T13:50:28.150437+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T21:52:25.852199+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":["5","6"],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.21","version":"0'0","reported_seq":8,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-08-09T13:50:29.233755+0000","last_change":"2023-08-09T13:50:29.233755+0000","last_active":"2023-08-09T13:50:29.233755+0000","last_peered":"2023-08-09T13:50:29.233755+0000","last_clean":"2023-08-09T13:50:29.233755+0000","last_became_active":"2023-08-09T13:50:29.233150+0000","last_became_peered":"2023-08-09T13:50:29.233150+0000","last_unstale":"2023-08-09T13:50:29.233755+0000","last_undegraded":"2023-08-09T13:50:29.233755+0000","last_fullsized":"2023-08-09T13:50:29.233755+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:50:28.150437+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:50:28.150437+0000","last_clean_scrub_stamp":"2023-08-09T13:50:28.150437+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T16:18:45.331895+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.20","version":"0'0","reported_seq":8,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-08-09T13:50:29.245742+0000","last_change":"2023-08-09T13:50:29.245742+0000","last_active":"2023-08-09T13:50:29.245742+0000","last_peered":"2023-08-09T13:50:29.245742+0000","last_clean":"2023-08-09T13:50:29.245742+0000","last_became_active":"2023-08-09T13:50:29.242577+0000","last_became_peered":"2023-08-09T13:50:29.242577+0000","last_unstale":"2023-08-09T13:50:29.245742+0000","last_undegraded":"2023-08-09T13:50:29.245742+0000","last_fullsized":"2023-08-09T13:50:29.245742+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:50:28.150437+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:50:28.150437+0000","last_clean_scrub_stamp":"2023-08-09T13:50:28.150437+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T16:18:45.331895+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.1e","version":"0'0","reported_seq":3,"reported_epoch":89,"state":"creating+activating","last_fresh":"2023-08-09T13:50:29.220103+0000","last_change":"2023-08-09T13:50:29.220103+0000","last_active":"2023-08-09T13:50:28.150437+0000","last_peered":"2023-08-09T13:50:28.150437+0000","last_clean":"2023-08-09T13:50:28.150437+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-09T13:50:29.220103+0000","last_undegraded":"2023-08-09T13:50:29.220103+0000","last_fullsized":"2023-08-09T13:50:29.220103+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:50:28.150437+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:50:28.150437+0000","last_clean_scrub_stamp":"2023-08-09T13:50:28.150437+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T19:44:56.828003+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":["7","6"],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.1f","version":"0'0","reported_seq":11,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-08-09T13:50:29.221681+0000","last_change":"2023-08-09T13:50:28.198750+0000","last_active":"2023-08-09T13:50:29.221681+0000","last_peered":"2023-08-09T13:50:29.221681+0000","last_clean":"2023-08-09T13:50:29.221681+0000","last_became_active":"2023-08-09T13:50:28.198375+0000","last_became_peered":"2023-08-09T13:50:28.198375+0000","last_unstale":"2023-08-09T13:50:29.221681+0000","last_undegraded":"2023-08-09T13:50:29.221681+0000","last_fullsized":"2023-08-09T13:50:29.221681+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:50:27.140775+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:50:27.140775+0000","last_clean_scrub_stamp":"2023-08-09T13:50:27.140775+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T23:54:42.158506+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"11.1f","version":"0'0","reported_seq":3,"reported_epoch":89,"state":"creating+activating","last_fresh":"2023-08-09T13:50:29.223179+0000","last_change":"2023-08-09T13:50:29.223179+0000","last_active":"2023-08-09T13:50:28.150437+0000","last_peered":"2023-08-09T13:50:28.150437+0000","last_clean":"2023-08-09T13:50:28.150437+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-09T13:50:29.223179+0000","last_undegraded":"2023-08-09T13:50:29.223179+0000","last_fullsized":"2023-08-09T13:50:29.223179+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:50:28.150437+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:50:28.150437+0000","last_clean_scrub_stamp":"2023-08-09T13:50:28.150437+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T20:43:24.475264+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":["2","3"],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.1e","version":"0'0","reported_seq":11,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-08-09T13:50:29.209208+0000","last_change":"2023-08-09T13:50:28.189590+0000","last_active":"2023-08-09T13:50:29.209208+0000","last_peered":"2023-08-09T13:50:29.209208+0000","last_clean":"2023-08-09T13:50:29.209208+0000","last_became_active":"2023-08-09T13:50:28.189227+0000","last_became_peered":"2023-08-09T13:50:28.189227+0000","last_unstale":"2023-08-09T13:50:29.209208+0000","last_undegraded":"2023-08-09T13:50:29.209208+0000","last_fullsized":"2023-08-09T13:50:29.209208+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:50:27.140775+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:50:27.140775+0000","last_clean_scrub_stamp":"2023-08-09T13:50:27.140775+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T19:44:55.818341+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.1c","version":"0'0","reported_seq":4,"reported_epoch":89,"state":"creating+activating","last_fresh":"2023-08-09T13:50:29.250266+0000","last_change":"2023-08-09T13:50:29.225957+0000","last_active":"2023-08-09T13:50:28.150437+0000","last_peered":"2023-08-09T13:50:28.150437+0000","last_clean":"2023-08-09T13:50:28.150437+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-09T13:50:29.250266+0000","last_undegraded":"2023-08-09T13:50:29.250266+0000","last_fullsized":"2023-08-09T13:50:29.250266+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:50:28.150437+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:50:28.150437+0000","last_clean_scrub_stamp":"2023-08-09T13:50:28.150437+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T21:57:09.347886+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":["6","7"],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"10.1d","version":"0'0","reported_seq":11,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-08-09T13:50:29.210839+0000","last_change":"2023-08-09T13:50:28.206216+0000","last_active":"2023-08-09T13:50:29.210839+0000","last_peered":"2023-08-09T13:50:29.210839+0000","last_clean":"2023-08-09T13:50:29.210839+0000","last_became_active":"2023-08-09T13:50:28.205914+0000","last_became_peered":"2023-08-09T13:50:28.205914+0000","last_unstale":"2023-08-09T13:50:29.210839+0000","last_undegraded":"2023-08-09T13:50:29.210839+0000","last_fullsized":"2023-08-09T13:50:29.210839+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:50:27.140775+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:50:27.140775+0000","last_clean_scrub_stamp":"2023-08-09T13:50:27.140775+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T16:26:26.712563+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.1d","version":"0'0","reported_seq":8,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-08-09T13:50:29.231490+0000","last_change":"2023-08-09T13:50:29.231490+0000","last_active":"2023-08-09T13:50:29.231490+0000","last_peered":"2023-08-09T13:50:29.231490+0000","last_clean":"2023-08-09T13:50:29.231490+0000","last_became_active":"2023-08-09T13:50:29.231190+0000","last_became_peered":"2023-08-09T13:50:29.231190+0000","last_unstale":"2023-08-09T13:50:29.231490+0000","last_undegraded":"2023-08-09T13:50:29.231490+0000","last_fullsized":"2023-08-09T13:50:29.231490+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:50:28.150437+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:50:28.150437+0000","last_clean_scrub_stamp":"2023-08-09T13:50:28.150437+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T14:46:45.632501+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.1c","version":"0'0","reported_seq":11,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-08-09T13:50:29.235770+0000","last_change":"2023-08-09T13:50:28.195729+0000","last_active":"2023-08-09T13:50:29.235770+0000","last_peered":"2023-08-09T13:50:29.235770+0000","last_clean":"2023-08-09T13:50:29.235770+0000","last_became_active":"2023-08-09T13:50:28.195433+0000","last_became_peered":"2023-08-09T13:50:28.195433+0000","last_unstale":"2023-08-09T13:50:29.235770+0000","last_undegraded":"2023-08-09T13:50:29.235770+0000","last_fullsized":"2023-08-09T13:50:29.235770+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:50:27.140775+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:50:27.140775+0000","last_clean_scrub_stamp":"2023-08-09T13:50:27.140775+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T16:52:42.838615+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.1a","version":"0'0","reported_seq":8,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-08-09T13:50:29.234000+0000","last_change":"2023-08-09T13:50:29.234000+0000","last_active":"2023-08-09T13:50:29.234000+0000","last_peered":"2023-08-09T13:50:29.234000+0000","last_clean":"2023-08-09T13:50:29.234000+0000","last_became_active":"2023-08-09T13:50:29.233727+0000","last_became_peered":"2023-08-09T13:50:29.233727+0000","last_unstale":"2023-08-09T13:50:29.234000+0000","last_undegraded":"2023-08-09T13:50:29.234000+0000","last_fullsized":"2023-08-09T13:50:29.234000+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:50:28.150437+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:50:28.150437+0000","last_clean_scrub_stamp":"2023-08-09T13:50:28.150437+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T20:21:45.115966+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.1b","version":"0'0","reported_seq":11,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-08-09T13:50:29.214221+0000","last_change":"2023-08-09T13:50:28.196698+0000","last_active":"2023-08-09T13:50:29.214221+0000","last_peered":"2023-08-09T13:50:29.214221+0000","last_clean":"2023-08-09T13:50:29.214221+0000","last_became_active":"2023-08-09T13:50:28.196362+0000","last_became_peered":"2023-08-09T13:50:28.196362+0000","last_unstale":"2023-08-09T13:50:29.214221+0000","last_undegraded":"2023-08-09T13:50:29.214221+0000","last_fullsized":"2023-08-09T13:50:29.214221+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:50:27.140775+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:50:27.140775+0000","last_clean_scrub_stamp":"2023-08-09T13:50:27.140775+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T16:26:00.621455+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.1b","version":"0'0","reported_seq":3,"reported_epoch":89,"state":"creating+activating","last_fresh":"2023-08-09T13:50:29.208495+0000","last_change":"2023-08-09T13:50:29.208495+0000","last_active":"2023-08-09T13:50:28.150437+0000","last_peered":"2023-08-09T13:50:28.150437+0000","last_clean":"2023-08-09T13:50:28.150437+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-09T13:50:29.208495+0000","last_undegraded":"2023-08-09T13:50:29.208495+0000","last_fullsized":"2023-08-09T13:50:29.208495+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:50:28.150437+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:50:28.150437+0000","last_clean_scrub_stamp":"2023-08-09T13:50:28.150437+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T16:20:35.941253+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":["4","7"],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"10.1a","version":"0'0","reported_seq":11,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-08-09T13:50:29.228557+0000","last_change":"2023-08-09T13:50:28.205333+0000","last_active":"2023-08-09T13:50:29.228557+0000","last_peered":"2023-08-09T13:50:29.228557+0000","last_clean":"2023-08-09T13:50:29.228557+0000","last_became_active":"2023-08-09T13:50:28.204881+0000","last_became_peered":"2023-08-09T13:50:28.204881+0000","last_unstale":"2023-08-09T13:50:29.228557+0000","last_undegraded":"2023-08-09T13:50:29.228557+0000","last_fullsized":"2023-08-09T13:50:29.228557+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:50:27.140775+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:50:27.140775+0000","last_clean_scrub_stamp":"2023-08-09T13:50:27.140775+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T18:06:20.762713+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.18","version":"0'0","reported_seq":3,"reported_epoch":89,"state":"creating+activating","last_fresh":"2023-08-09T13:50:29.218450+0000","last_change":"2023-08-09T13:50:29.218450+0000","last_active":"2023-08-09T13:50:28.150437+0000","last_peered":"2023-08-09T13:50:28.150437+0000","last_clean":"2023-08-09T13:50:28.150437+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-09T13:50:29.218450+0000","last_undegraded":"2023-08-09T13:50:29.218450+0000","last_fullsized":"2023-08-09T13:50:29.218450+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:50:28.150437+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:50:28.150437+0000","last_clean_scrub_stamp":"2023-08-09T13:50:28.150437+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T15:00:53.649507+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":["2","4"],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.19","version":"0'0","reported_seq":9,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-08-09T13:50:28.205519+0000","last_change":"2023-08-09T13:50:28.204197+0000","last_active":"2023-08-09T13:50:28.205519+0000","last_peered":"2023-08-09T13:50:28.205519+0000","last_clean":"2023-08-09T13:50:28.205519+0000","last_became_active":"2023-08-09T13:50:28.203911+0000","last_became_peered":"2023-08-09T13:50:28.203911+0000","last_unstale":"2023-08-09T13:50:28.205519+0000","last_undegraded":"2023-08-09T13:50:28.205519+0000","last_fullsized":"2023-08-09T13:50:28.205519+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:50:27.140775+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:50:27.140775+0000","last_clean_scrub_stamp":"2023-08-09T13:50:27.140775+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T18:08:46.110989+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.19","version":"0'0","reported_seq":3,"reported_epoch":89,"state":"creating+activating","last_fresh":"2023-08-09T13:50:29.206649+0000","last_change":"2023-08-09T13:50:29.206649+0000","last_active":"2023-08-09T13:50:28.150437+0000","last_peered":"2023-08-09T13:50:28.150437+0000","last_clean":"2023-08-09T13:50:28.150437+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-09T13:50:29.206649+0000","last_undegraded":"2023-08-09T13:50:29.206649+0000","last_fullsized":"2023-08-09T13:50:29.206649+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:50:28.150437+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:50:28.150437+0000","last_clean_scrub_stamp":"2023-08-09T13:50:28.150437+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T15:32:41.261189+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":["0","7"],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.18","version":"0'0","reported_seq":11,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-08-09T13:50:29.224563+0000","last_change":"2023-08-09T13:50:28.189872+0000","last_active":"2023-08-09T13:50:29.224563+0000","last_peered":"2023-08-09T13:50:29.224563+0000","last_clean":"2023-08-09T13:50:29.224563+0000","last_became_active":"2023-08-09T13:50:28.189560+0000","last_became_peered":"2023-08-09T13:50:28.189560+0000","last_unstale":"2023-08-09T13:50:29.224563+0000","last_undegraded":"2023-08-09T13:50:29.224563+0000","last_fullsized":"2023-08-09T13:50:29.224563+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:50:27.140775+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:50:27.140775+0000","last_clean_scrub_stamp":"2023-08-09T13:50:27.140775+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T18:02:42.591074+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.16","version":"0'0","reported_seq":3,"reported_epoch":89,"state":"creating+activating","last_fresh":"2023-08-09T13:50:29.223029+0000","last_change":"2023-08-09T13:50:29.223029+0000","last_active":"2023-08-09T13:50:28.150437+0000","last_peered":"2023-08-09T13:50:28.150437+0000","last_clean":"2023-08-09T13:50:28.150437+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-09T13:50:29.223029+0000","last_undegraded":"2023-08-09T13:50:29.223029+0000","last_fullsized":"2023-08-09T13:50:29.223029+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:50:28.150437+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:50:28.150437+0000","last_clean_scrub_stamp":"2023-08-09T13:50:28.150437+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T18:35:45.000310+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":["5","7"],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.17","version":"0'0","reported_seq":11,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-08-09T13:50:29.210879+0000","last_change":"2023-08-09T13:50:28.198990+0000","last_active":"2023-08-09T13:50:29.210879+0000","last_peered":"2023-08-09T13:50:29.210879+0000","last_clean":"2023-08-09T13:50:29.210879+0000","last_became_active":"2023-08-09T13:50:28.197376+0000","last_became_peered":"2023-08-09T13:50:28.197376+0000","last_unstale":"2023-08-09T13:50:29.210879+0000","last_undegraded":"2023-08-09T13:50:29.210879+0000","last_fullsized":"2023-08-09T13:50:29.210879+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:50:27.140775+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:50:27.140775+0000","last_clean_scrub_stamp":"2023-08-09T13:50:27.140775+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T18:00:13.480919+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.17","version":"0'0","reported_seq":3,"reported_epoch":89,"state":"creating+activating","last_fresh":"2023-08-09T13:50:29.221024+0000","last_change":"2023-08-09T13:50:29.221024+0000","last_active":"2023-08-09T13:50:28.150437+0000","last_peered":"2023-08-09T13:50:28.150437+0000","last_clean":"2023-08-09T13:50:28.150437+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-09T13:50:29.221024+0000","last_undegraded":"2023-08-09T13:50:29.221024+0000","last_fullsized":"2023-08-09T13:50:29.221024+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:50:28.150437+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:50:28.150437+0000","last_clean_scrub_stamp":"2023-08-09T13:50:28.150437+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T19:41:45.324636+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":["7","6"],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.16","version":"0'0","reported_seq":11,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-08-09T13:50:29.236992+0000","last_change":"2023-08-09T13:50:28.203893+0000","last_active":"2023-08-09T13:50:29.236992+0000","last_peered":"2023-08-09T13:50:29.236992+0000","last_clean":"2023-08-09T13:50:29.236992+0000","last_became_active":"2023-08-09T13:50:28.203596+0000","last_became_peered":"2023-08-09T13:50:28.203596+0000","last_unstale":"2023-08-09T13:50:29.236992+0000","last_undegraded":"2023-08-09T13:50:29.236992+0000","last_fullsized":"2023-08-09T13:50:29.236992+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:50:27.140775+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:50:27.140775+0000","last_clean_scrub_stamp":"2023-08-09T13:50:27.140775+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-11T01:07:24.154763+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.14","version":"0'0","reported_seq":8,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-08-09T13:50:29.236992+0000","last_change":"2023-08-09T13:50:29.236992+0000","last_active":"2023-08-09T13:50:29.236992+0000","last_peered":"2023-08-09T13:50:29.236992+0000","last_clean":"2023-08-09T13:50:29.236992+0000","last_became_active":"2023-08-09T13:50:29.236708+0000","last_became_peered":"2023-08-09T13:50:29.236708+0000","last_unstale":"2023-08-09T13:50:29.236992+0000","last_undegraded":"2023-08-09T13:50:29.236992+0000","last_fullsized":"2023-08-09T13:50:29.236992+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:50:28.150437+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:50:28.150437+0000","last_clean_scrub_stamp":"2023-08-09T13:50:28.150437+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T23:19:57.122069+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"10.15","version":"0'0","reported_seq":11,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-08-09T13:50:29.215134+0000","last_change":"2023-08-09T13:50:28.198818+0000","last_active":"2023-08-09T13:50:29.215134+0000","last_peered":"2023-08-09T13:50:29.215134+0000","last_clean":"2023-08-09T13:50:29.215134+0000","last_became_active":"2023-08-09T13:50:28.198282+0000","last_became_peered":"2023-08-09T13:50:28.198282+0000","last_unstale":"2023-08-09T13:50:29.215134+0000","last_undegraded":"2023-08-09T13:50:29.215134+0000","last_fullsized":"2023-08-09T13:50:29.215134+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:50:27.140775+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:50:27.140775+0000","last_clean_scrub_stamp":"2023-08-09T13:50:27.140775+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T13:56:48.165182+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.15","version":"0'0","reported_seq":8,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-08-09T13:50:29.232045+0000","last_change":"2023-08-09T13:50:29.232045+0000","last_active":"2023-08-09T13:50:29.232045+0000","last_peered":"2023-08-09T13:50:29.232045+0000","last_clean":"2023-08-09T13:50:29.232045+0000","last_became_active":"2023-08-09T13:50:29.231691+0000","last_became_peered":"2023-08-09T13:50:29.231691+0000","last_unstale":"2023-08-09T13:50:29.232045+0000","last_undegraded":"2023-08-09T13:50:29.232045+0000","last_fullsized":"2023-08-09T13:50:29.232045+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:50:28.150437+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:50:28.150437+0000","last_clean_scrub_stamp":"2023-08-09T13:50:28.150437+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T14:41:33.537815+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.14","version":"0'0","reported_seq":9,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-08-09T13:50:28.198290+0000","last_change":"2023-08-09T13:50:28.194890+0000","last_active":"2023-08-09T13:50:28.198290+0000","last_peered":"2023-08-09T13:50:28.198290+0000","last_clean":"2023-08-09T13:50:28.198290+0000","last_became_active":"2023-08-09T13:50:28.194607+0000","last_became_peered":"2023-08-09T13:50:28.194607+0000","last_unstale":"2023-08-09T13:50:28.198290+0000","last_undegraded":"2023-08-09T13:50:28.198290+0000","last_fullsized":"2023-08-09T13:50:28.198290+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:50:27.140775+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:50:27.140775+0000","last_clean_scrub_stamp":"2023-08-09T13:50:27.140775+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-11T01:35:47.708771+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.12","version":"0'0","reported_seq":8,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-08-09T13:50:29.233976+0000","last_change":"2023-08-09T13:50:29.233976+0000","last_active":"2023-08-09T13:50:29.233976+0000","last_peered":"2023-08-09T13:50:29.233976+0000","last_clean":"2023-08-09T13:50:29.233976+0000","last_became_active":"2023-08-09T13:50:29.233696+0000","last_became_peered":"2023-08-09T13:50:29.233696+0000","last_unstale":"2023-08-09T13:50:29.233976+0000","last_undegraded":"2023-08-09T13:50:29.233976+0000","last_fullsized":"2023-08-09T13:50:29.233976+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:50:28.150437+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:50:28.150437+0000","last_clean_scrub_stamp":"2023-08-09T13:50:28.150437+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T15:32:41.261189+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.13","version":"0'0","reported_seq":9,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-08-09T13:50:28.200538+0000","last_change":"2023-08-09T13:50:28.198586+0000","last_active":"2023-08-09T13:50:28.200538+0000","last_peered":"2023-08-09T13:50:28.200538+0000","last_clean":"2023-08-09T13:50:28.200538+0000","last_became_active":"2023-08-09T13:50:28.198247+0000","last_became_peered":"2023-08-09T13:50:28.198247+0000","last_unstale":"2023-08-09T13:50:28.200538+0000","last_undegraded":"2023-08-09T13:50:28.200538+0000","last_fullsized":"2023-08-09T13:50:28.200538+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:50:27.140775+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:50:27.140775+0000","last_clean_scrub_stamp":"2023-08-09T13:50:27.140775+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T21:31:16.170439+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.13","version":"0'0","reported_seq":3,"reported_epoch":89,"state":"creating+activating","last_fresh":"2023-08-09T13:50:29.220908+0000","last_change":"2023-08-09T13:50:29.220908+0000","last_active":"2023-08-09T13:50:28.150437+0000","last_peered":"2023-08-09T13:50:28.150437+0000","last_clean":"2023-08-09T13:50:28.150437+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-09T13:50:29.220908+0000","last_undegraded":"2023-08-09T13:50:29.220908+0000","last_fullsized":"2023-08-09T13:50:29.220908+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:50:28.150437+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:50:28.150437+0000","last_clean_scrub_stamp":"2023-08-09T13:50:28.150437+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T18:15:31.389069+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":["5","4"],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.12","version":"0'0","reported_seq":11,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-08-09T13:50:29.235230+0000","last_change":"2023-08-09T13:50:28.201822+0000","last_active":"2023-08-09T13:50:29.235230+0000","last_peered":"2023-08-09T13:50:29.235230+0000","last_clean":"2023-08-09T13:50:29.235230+0000","last_became_active":"2023-08-09T13:50:28.201187+0000","last_became_peered":"2023-08-09T13:50:28.201187+0000","last_unstale":"2023-08-09T13:50:29.235230+0000","last_undegraded":"2023-08-09T13:50:29.235230+0000","last_fullsized":"2023-08-09T13:50:29.235230+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:50:27.140775+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:50:27.140775+0000","last_clean_scrub_stamp":"2023-08-09T13:50:27.140775+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T23:52:03.500625+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.10","version":"0'0","reported_seq":3,"reported_epoch":89,"state":"creating+activating","last_fresh":"2023-08-09T13:50:29.205817+0000","last_change":"2023-08-09T13:50:29.205817+0000","last_active":"2023-08-09T13:50:28.150437+0000","last_peered":"2023-08-09T13:50:28.150437+0000","last_clean":"2023-08-09T13:50:28.150437+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-09T13:50:29.205817+0000","last_undegraded":"2023-08-09T13:50:29.205817+0000","last_fullsized":"2023-08-09T13:50:29.205817+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:50:28.150437+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:50:28.150437+0000","last_clean_scrub_stamp":"2023-08-09T13:50:28.150437+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-11T01:02:33.470393+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":["0","6"],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.11","version":"0'0","reported_seq":11,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-08-09T13:50:29.221870+0000","last_change":"2023-08-09T13:50:28.193484+0000","last_active":"2023-08-09T13:50:29.221870+0000","last_peered":"2023-08-09T13:50:29.221870+0000","last_clean":"2023-08-09T13:50:29.221870+0000","last_became_active":"2023-08-09T13:50:28.193098+0000","last_became_peered":"2023-08-09T13:50:28.193098+0000","last_unstale":"2023-08-09T13:50:29.221870+0000","last_undegraded":"2023-08-09T13:50:29.221870+0000","last_fullsized":"2023-08-09T13:50:29.221870+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:50:27.140775+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:50:27.140775+0000","last_clean_scrub_stamp":"2023-08-09T13:50:27.140775+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T21:57:08.338223+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.11","version":"0'0","reported_seq":3,"reported_epoch":89,"state":"creating+activating","last_fresh":"2023-08-09T13:50:29.224448+0000","last_change":"2023-08-09T13:50:29.224448+0000","last_active":"2023-08-09T13:50:28.150437+0000","last_peered":"2023-08-09T13:50:28.150437+0000","last_clean":"2023-08-09T13:50:28.150437+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-09T13:50:29.224448+0000","last_undegraded":"2023-08-09T13:50:29.224448+0000","last_fullsized":"2023-08-09T13:50:29.224448+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:50:28.150437+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:50:28.150437+0000","last_clean_scrub_stamp":"2023-08-09T13:50:28.150437+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T19:49:56.370072+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":["2","3"],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.10","version":"0'0","reported_seq":11,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-08-09T13:50:29.209343+0000","last_change":"2023-08-09T13:50:28.200670+0000","last_active":"2023-08-09T13:50:29.209343+0000","last_peered":"2023-08-09T13:50:29.209343+0000","last_clean":"2023-08-09T13:50:29.209343+0000","last_became_active":"2023-08-09T13:50:28.200419+0000","last_became_peered":"2023-08-09T13:50:28.200419+0000","last_unstale":"2023-08-09T13:50:29.209343+0000","last_undegraded":"2023-08-09T13:50:29.209343+0000","last_fullsized":"2023-08-09T13:50:29.209343+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:50:27.140775+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:50:27.140775+0000","last_clean_scrub_stamp":"2023-08-09T13:50:27.140775+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T15:23:31.632434+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.6","version":"0'0","reported_seq":3,"reported_epoch":89,"state":"creating+activating","last_fresh":"2023-08-09T13:50:29.208662+0000","last_change":"2023-08-09T13:50:29.208662+0000","last_active":"2023-08-09T13:50:28.150437+0000","last_peered":"2023-08-09T13:50:28.150437+0000","last_clean":"2023-08-09T13:50:28.150437+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-09T13:50:29.208662+0000","last_undegraded":"2023-08-09T13:50:29.208662+0000","last_fullsized":"2023-08-09T13:50:29.208662+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:50:28.150437+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:50:28.150437+0000","last_clean_scrub_stamp":"2023-08-09T13:50:28.150437+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T17:05:45.752848+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":["0","2"],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.7","version":"0'0","reported_seq":11,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-08-09T13:50:29.230633+0000","last_change":"2023-08-09T13:50:28.204446+0000","last_active":"2023-08-09T13:50:29.230633+0000","last_peered":"2023-08-09T13:50:29.230633+0000","last_clean":"2023-08-09T13:50:29.230633+0000","last_became_active":"2023-08-09T13:50:28.204093+0000","last_became_peered":"2023-08-09T13:50:28.204093+0000","last_unstale":"2023-08-09T13:50:29.230633+0000","last_undegraded":"2023-08-09T13:50:29.230633+0000","last_fullsized":"2023-08-09T13:50:29.230633+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:50:27.140775+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:50:27.140775+0000","last_clean_scrub_stamp":"2023-08-09T13:50:27.140775+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T22:56:15.821115+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.7","version":"0'0","reported_seq":3,"reported_epoch":89,"state":"creating+activating","last_fresh":"2023-08-09T13:50:29.223626+0000","last_change":"2023-08-09T13:50:29.223626+0000","last_active":"2023-08-09T13:50:28.150437+0000","last_peered":"2023-08-09T13:50:28.150437+0000","last_clean":"2023-08-09T13:50:28.150437+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-09T13:50:29.223626+0000","last_undegraded":"2023-08-09T13:50:29.223626+0000","last_fullsized":"2023-08-09T13:50:29.223626+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:50:28.150437+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:50:28.150437+0000","last_clean_scrub_stamp":"2023-08-09T13:50:28.150437+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T18:06:21.772376+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":["5","1"],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.6","version":"0'0","reported_seq":11,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-08-09T13:50:29.223926+0000","last_change":"2023-08-09T13:50:28.191955+0000","last_active":"2023-08-09T13:50:29.223926+0000","last_peered":"2023-08-09T13:50:29.223926+0000","last_clean":"2023-08-09T13:50:29.223926+0000","last_became_active":"2023-08-09T13:50:28.191264+0000","last_became_peered":"2023-08-09T13:50:28.191264+0000","last_unstale":"2023-08-09T13:50:29.223926+0000","last_undegraded":"2023-08-09T13:50:29.223926+0000","last_fullsized":"2023-08-09T13:50:29.223926+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:50:27.140775+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:50:27.140775+0000","last_clean_scrub_stamp":"2023-08-09T13:50:27.140775+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T18:00:04.982575+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.4","version":"0'0","reported_seq":3,"reported_epoch":89,"state":"creating+activating","last_fresh":"2023-08-09T13:50:29.216528+0000","last_change":"2023-08-09T13:50:29.216528+0000","last_active":"2023-08-09T13:50:28.150437+0000","last_peered":"2023-08-09T13:50:28.150437+0000","last_clean":"2023-08-09T13:50:28.150437+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-09T13:50:29.216528+0000","last_undegraded":"2023-08-09T13:50:29.216528+0000","last_fullsized":"2023-08-09T13:50:29.216528+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:50:28.150437+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:50:28.150437+0000","last_clean_scrub_stamp":"2023-08-09T13:50:28.150437+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T21:36:45.909480+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":["7","6"],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.5","version":"0'0","reported_seq":11,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-08-09T13:50:29.224819+0000","last_change":"2023-08-09T13:50:28.193912+0000","last_active":"2023-08-09T13:50:29.224819+0000","last_peered":"2023-08-09T13:50:29.224819+0000","last_clean":"2023-08-09T13:50:29.224819+0000","last_became_active":"2023-08-09T13:50:28.193602+0000","last_became_peered":"2023-08-09T13:50:28.193602+0000","last_unstale":"2023-08-09T13:50:29.224819+0000","last_undegraded":"2023-08-09T13:50:29.224819+0000","last_fullsized":"2023-08-09T13:50:29.224819+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:50:27.140775+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:50:27.140775+0000","last_clean_scrub_stamp":"2023-08-09T13:50:27.140775+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T19:25:18.625389+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.5","version":"0'0","reported_seq":8,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-08-09T13:50:29.237250+0000","last_change":"2023-08-09T13:50:29.237250+0000","last_active":"2023-08-09T13:50:29.237250+0000","last_peered":"2023-08-09T13:50:29.237250+0000","last_clean":"2023-08-09T13:50:29.237250+0000","last_became_active":"2023-08-09T13:50:29.236821+0000","last_became_peered":"2023-08-09T13:50:29.236821+0000","last_unstale":"2023-08-09T13:50:29.237250+0000","last_undegraded":"2023-08-09T13:50:29.237250+0000","last_fullsized":"2023-08-09T13:50:29.237250+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:50:28.150437+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:50:28.150437+0000","last_clean_scrub_stamp":"2023-08-09T13:50:28.150437+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T15:02:09.473638+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.4","version":"0'0","reported_seq":11,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-08-09T13:50:29.208326+0000","last_change":"2023-08-09T13:50:28.197934+0000","last_active":"2023-08-09T13:50:29.208326+0000","last_peered":"2023-08-09T13:50:29.208326+0000","last_clean":"2023-08-09T13:50:29.208326+0000","last_became_active":"2023-08-09T13:50:28.197639+0000","last_became_peered":"2023-08-09T13:50:28.197639+0000","last_unstale":"2023-08-09T13:50:29.208326+0000","last_undegraded":"2023-08-09T13:50:29.208326+0000","last_fullsized":"2023-08-09T13:50:29.208326+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:50:27.140775+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:50:27.140775+0000","last_clean_scrub_stamp":"2023-08-09T13:50:27.140775+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T14:46:44.622839+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.2","version":"0'0","reported_seq":3,"reported_epoch":89,"state":"creating+activating","last_fresh":"2023-08-09T13:50:29.217549+0000","last_change":"2023-08-09T13:50:29.217549+0000","last_active":"2023-08-09T13:50:28.150437+0000","last_peered":"2023-08-09T13:50:28.150437+0000","last_clean":"2023-08-09T13:50:28.150437+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-09T13:50:29.217549+0000","last_undegraded":"2023-08-09T13:50:29.217549+0000","last_fullsized":"2023-08-09T13:50:29.217549+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:50:28.150437+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:50:28.150437+0000","last_clean_scrub_stamp":"2023-08-09T13:50:28.150437+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-11T00:03:22.851529+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":["2","3"],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.3","version":"0'0","reported_seq":11,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-08-09T13:50:29.224120+0000","last_change":"2023-08-09T13:50:28.189872+0000","last_active":"2023-08-09T13:50:29.224120+0000","last_peered":"2023-08-09T13:50:29.224120+0000","last_clean":"2023-08-09T13:50:29.224120+0000","last_became_active":"2023-08-09T13:50:28.189588+0000","last_became_peered":"2023-08-09T13:50:28.189588+0000","last_unstale":"2023-08-09T13:50:29.224120+0000","last_undegraded":"2023-08-09T13:50:29.224120+0000","last_fullsized":"2023-08-09T13:50:29.224120+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:50:27.140775+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:50:27.140775+0000","last_clean_scrub_stamp":"2023-08-09T13:50:27.140775+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T16:47:26.737473+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.3","version":"0'0","reported_seq":8,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-08-09T13:50:29.250246+0000","last_change":"2023-08-09T13:50:29.250246+0000","last_active":"2023-08-09T13:50:29.250246+0000","last_peered":"2023-08-09T13:50:29.250246+0000","last_clean":"2023-08-09T13:50:29.250246+0000","last_became_active":"2023-08-09T13:50:29.249885+0000","last_became_peered":"2023-08-09T13:50:29.249885+0000","last_unstale":"2023-08-09T13:50:29.250246+0000","last_undegraded":"2023-08-09T13:50:29.250246+0000","last_fullsized":"2023-08-09T13:50:29.250246+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:50:28.150437+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:50:28.150437+0000","last_clean_scrub_stamp":"2023-08-09T13:50:28.150437+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T21:52:25.852199+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"10.2","version":"0'0","reported_seq":9,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-08-09T13:50:28.203714+0000","last_change":"2023-08-09T13:50:28.202029+0000","last_active":"2023-08-09T13:50:28.203714+0000","last_peered":"2023-08-09T13:50:28.203714+0000","last_clean":"2023-08-09T13:50:28.203714+0000","last_became_active":"2023-08-09T13:50:28.201720+0000","last_became_peered":"2023-08-09T13:50:28.201720+0000","last_unstale":"2023-08-09T13:50:28.203714+0000","last_undegraded":"2023-08-09T13:50:28.203714+0000","last_fullsized":"2023-08-09T13:50:28.203714+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:50:27.140775+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:50:27.140775+0000","last_clean_scrub_stamp":"2023-08-09T13:50:27.140775+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T19:37:29.224002+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.0","version":"0'0","reported_seq":1,"reported_epoch":88,"state":"creating+peering","last_fresh":"2023-08-09T13:50:28.189500+0000","last_change":"2023-08-09T13:50:28.189500+0000","last_active":"2023-08-09T13:50:28.150437+0000","last_peered":"2023-08-09T13:50:28.150437+0000","last_clean":"2023-08-09T13:50:28.150437+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-09T13:50:28.189500+0000","last_undegraded":"2023-08-09T13:50:28.189500+0000","last_fullsized":"2023-08-09T13:50:28.189500+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:50:28.150437+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:50:28.150437+0000","last_clean_scrub_stamp":"2023-08-09T13:50:28.150437+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.1","version":"0'0","reported_seq":11,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-08-09T13:50:29.209627+0000","last_change":"2023-08-09T13:50:28.200080+0000","last_active":"2023-08-09T13:50:29.209627+0000","last_peered":"2023-08-09T13:50:29.209627+0000","last_clean":"2023-08-09T13:50:29.209627+0000","last_became_active":"2023-08-09T13:50:28.199831+0000","last_became_peered":"2023-08-09T13:50:28.199831+0000","last_unstale":"2023-08-09T13:50:29.209627+0000","last_undegraded":"2023-08-09T13:50:29.209627+0000","last_fullsized":"2023-08-09T13:50:29.209627+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:50:27.140775+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:50:27.140775+0000","last_clean_scrub_stamp":"2023-08-09T13:50:27.140775+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T15:18:11.293013+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.1","version":"0'0","reported_seq":8,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-08-09T13:50:29.249892+0000","last_change":"2023-08-09T13:50:29.249892+0000","last_active":"2023-08-09T13:50:29.249892+0000","last_peered":"2023-08-09T13:50:29.249892+0000","last_clean":"2023-08-09T13:50:29.249892+0000","last_became_active":"2023-08-09T13:50:29.242554+0000","last_became_peered":"2023-08-09T13:50:29.242554+0000","last_unstale":"2023-08-09T13:50:29.249892+0000","last_undegraded":"2023-08-09T13:50:29.249892+0000","last_fullsized":"2023-08-09T13:50:29.249892+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:50:28.150437+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:50:28.150437+0000","last_clean_scrub_stamp":"2023-08-09T13:50:28.150437+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T21:18:31.512056+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"10.0","version":"0'0","reported_seq":11,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-08-09T13:50:29.242250+0000","last_change":"2023-08-09T13:50:28.197319+0000","last_active":"2023-08-09T13:50:29.242250+0000","last_peered":"2023-08-09T13:50:29.242250+0000","last_clean":"2023-08-09T13:50:29.242250+0000","last_became_active":"2023-08-09T13:50:28.197031+0000","last_became_peered":"2023-08-09T13:50:28.197031+0000","last_unstale":"2023-08-09T13:50:29.242250+0000","last_undegraded":"2023-08-09T13:50:29.242250+0000","last_fullsized":"2023-08-09T13:50:29.242250+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:50:27.140775+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:50:27.140775+0000","last_clean_scrub_stamp":"2023-08-09T13:50:27.140775+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T16:15:05.176106+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.a","version":"0'0","reported_seq":3,"reported_epoch":89,"state":"creating+activating","last_fresh":"2023-08-09T13:50:29.212247+0000","last_change":"2023-08-09T13:50:29.212247+0000","last_active":"2023-08-09T13:50:28.150437+0000","last_peered":"2023-08-09T13:50:28.150437+0000","last_clean":"2023-08-09T13:50:28.150437+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-09T13:50:29.212247+0000","last_undegraded":"2023-08-09T13:50:29.212247+0000","last_fullsized":"2023-08-09T13:50:29.212247+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:50:28.150437+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:50:28.150437+0000","last_clean_scrub_stamp":"2023-08-09T13:50:28.150437+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T19:55:58.171731+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":["3","6"],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.b","version":"0'0","reported_seq":11,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-08-09T13:50:29.230951+0000","last_change":"2023-08-09T13:50:28.203783+0000","last_active":"2023-08-09T13:50:29.230951+0000","last_peered":"2023-08-09T13:50:29.230951+0000","last_clean":"2023-08-09T13:50:29.230951+0000","last_became_active":"2023-08-09T13:50:28.202391+0000","last_became_peered":"2023-08-09T13:50:28.202391+0000","last_unstale":"2023-08-09T13:50:29.230951+0000","last_undegraded":"2023-08-09T13:50:29.230951+0000","last_fullsized":"2023-08-09T13:50:29.230951+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:50:27.140775+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:50:27.140775+0000","last_clean_scrub_stamp":"2023-08-09T13:50:27.140775+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T21:39:38.493243+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":149,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-08-09T13:50:29.216433+0000","last_change":"2023-08-09T13:33:56.401406+0000","last_active":"2023-08-09T13:50:29.216433+0000","last_peered":"2023-08-09T13:50:29.216433+0000","last_clean":"2023-08-09T13:50:29.216433+0000","last_became_active":"2023-08-09T13:33:56.400487+0000","last_became_peered":"2023-08-09T13:33:56.400487+0000","last_unstale":"2023-08-09T13:50:29.216433+0000","last_undegraded":"2023-08-09T13:50:29.216433+0000","last_fullsized":"2023-08-09T13:50:29.216433+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:33:54.845869+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:33:54.845869+0000","last_clean_scrub_stamp":"2023-08-09T13:33:54.845869+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-11T00:46:00.165825+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"11.b","version":"0'0","reported_seq":3,"reported_epoch":89,"state":"creating+activating","last_fresh":"2023-08-09T13:50:29.211663+0000","last_change":"2023-08-09T13:50:29.211663+0000","last_active":"2023-08-09T13:50:28.150437+0000","last_peered":"2023-08-09T13:50:28.150437+0000","last_clean":"2023-08-09T13:50:28.150437+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-09T13:50:29.211663+0000","last_undegraded":"2023-08-09T13:50:29.211663+0000","last_fullsized":"2023-08-09T13:50:29.211663+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:50:28.150437+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:50:28.150437+0000","last_clean_scrub_stamp":"2023-08-09T13:50:28.150437+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-11T01:04:03.965638+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":["3","1"],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.a","version":"0'0","reported_seq":11,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-08-09T13:50:29.230486+0000","last_change":"2023-08-09T13:50:28.205148+0000","last_active":"2023-08-09T13:50:29.230486+0000","last_peered":"2023-08-09T13:50:29.230486+0000","last_clean":"2023-08-09T13:50:29.230486+0000","last_became_active":"2023-08-09T13:50:28.204528+0000","last_became_peered":"2023-08-09T13:50:28.204528+0000","last_unstale":"2023-08-09T13:50:29.230486+0000","last_undegraded":"2023-08-09T13:50:29.230486+0000","last_fullsized":"2023-08-09T13:50:29.230486+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:50:27.140775+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:50:27.140775+0000","last_clean_scrub_stamp":"2023-08-09T13:50:27.140775+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T18:00:04.982575+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":148,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-08-09T13:50:29.222206+0000","last_change":"2023-08-09T13:33:56.401123+0000","last_active":"2023-08-09T13:50:29.222206+0000","last_peered":"2023-08-09T13:50:29.222206+0000","last_clean":"2023-08-09T13:50:29.222206+0000","last_became_active":"2023-08-09T13:33:56.399933+0000","last_became_peered":"2023-08-09T13:33:56.399933+0000","last_unstale":"2023-08-09T13:50:29.222206+0000","last_undegraded":"2023-08-09T13:50:29.222206+0000","last_fullsized":"2023-08-09T13:50:29.222206+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:33:54.845869+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:33:54.845869+0000","last_clean_scrub_stamp":"2023-08-09T13:33:54.845869+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T17:05:05.384552+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.8","version":"0'0","reported_seq":3,"reported_epoch":89,"state":"creating+activating","last_fresh":"2023-08-09T13:50:29.213241+0000","last_change":"2023-08-09T13:50:29.213241+0000","last_active":"2023-08-09T13:50:28.150437+0000","last_peered":"2023-08-09T13:50:28.150437+0000","last_clean":"2023-08-09T13:50:28.150437+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-09T13:50:29.213241+0000","last_undegraded":"2023-08-09T13:50:29.213241+0000","last_fullsized":"2023-08-09T13:50:29.213241+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:50:28.150437+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:50:28.150437+0000","last_clean_scrub_stamp":"2023-08-09T13:50:28.150437+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T22:09:32.464173+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":["3","6"],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.9","version":"0'0","reported_seq":11,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-08-09T13:50:29.222528+0000","last_change":"2023-08-09T13:50:28.192232+0000","last_active":"2023-08-09T13:50:29.222528+0000","last_peered":"2023-08-09T13:50:29.222528+0000","last_clean":"2023-08-09T13:50:29.222528+0000","last_became_active":"2023-08-09T13:50:28.191350+0000","last_became_peered":"2023-08-09T13:50:28.191350+0000","last_unstale":"2023-08-09T13:50:29.222528+0000","last_undegraded":"2023-08-09T13:50:29.222528+0000","last_fullsized":"2023-08-09T13:50:29.222528+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:50:27.140775+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:50:27.140775+0000","last_clean_scrub_stamp":"2023-08-09T13:50:27.140775+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-11T00:33:31.648004+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.2","version":"0'0","reported_seq":148,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-08-09T13:50:29.215918+0000","last_change":"2023-08-09T13:33:56.402794+0000","last_active":"2023-08-09T13:50:29.215918+0000","last_peered":"2023-08-09T13:50:29.215918+0000","last_clean":"2023-08-09T13:50:29.215918+0000","last_became_active":"2023-08-09T13:33:56.402038+0000","last_became_peered":"2023-08-09T13:33:56.402038+0000","last_unstale":"2023-08-09T13:50:29.215918+0000","last_undegraded":"2023-08-09T13:50:29.215918+0000","last_fullsized":"2023-08-09T13:50:29.215918+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:33:54.845869+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:33:54.845869+0000","last_clean_scrub_stamp":"2023-08-09T13:33:54.845869+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T16:35:28.133841+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"11.9","version":"0'0","reported_seq":3,"reported_epoch":89,"state":"creating+activating","last_fresh":"2023-08-09T13:50:29.211902+0000","last_change":"2023-08-09T13:50:29.211902+0000","last_active":"2023-08-09T13:50:28.150437+0000","last_peered":"2023-08-09T13:50:28.150437+0000","last_clean":"2023-08-09T13:50:28.150437+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-09T13:50:29.211902+0000","last_undegraded":"2023-08-09T13:50:29.211902+0000","last_fullsized":"2023-08-09T13:50:29.211902+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:50:28.150437+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:50:28.150437+0000","last_clean_scrub_stamp":"2023-08-09T13:50:28.150437+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T15:32:31.232979+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":["3","6"],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.8","version":"0'0","reported_seq":11,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-08-09T13:50:29.205186+0000","last_change":"2023-08-09T13:50:28.190965+0000","last_active":"2023-08-09T13:50:29.205186+0000","last_peered":"2023-08-09T13:50:29.205186+0000","last_clean":"2023-08-09T13:50:29.205186+0000","last_became_active":"2023-08-09T13:50:28.190696+0000","last_became_peered":"2023-08-09T13:50:28.190696+0000","last_unstale":"2023-08-09T13:50:29.205186+0000","last_undegraded":"2023-08-09T13:50:29.205186+0000","last_fullsized":"2023-08-09T13:50:29.205186+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:50:27.140775+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:50:27.140775+0000","last_clean_scrub_stamp":"2023-08-09T13:50:27.140775+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T21:36:44.899817+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.3","version":"0'0","reported_seq":146,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-08-09T13:50:28.184637+0000","last_change":"2023-08-09T13:33:55.886749+0000","last_active":"2023-08-09T13:50:28.184637+0000","last_peered":"2023-08-09T13:50:28.184637+0000","last_clean":"2023-08-09T13:50:28.184637+0000","last_became_active":"2023-08-09T13:33:55.886116+0000","last_became_peered":"2023-08-09T13:33:55.886116+0000","last_unstale":"2023-08-09T13:50:28.184637+0000","last_undegraded":"2023-08-09T13:50:28.184637+0000","last_fullsized":"2023-08-09T13:50:28.184637+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:33:54.845869+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:33:54.845869+0000","last_clean_scrub_stamp":"2023-08-09T13:33:54.845869+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T16:59:16.380973+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"77'161","reported_seq":455,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-08-09T13:50:29.216076+0000","last_change":"2023-08-09T13:33:55.886345+0000","last_active":"2023-08-09T13:50:29.216076+0000","last_peered":"2023-08-09T13:50:29.216076+0000","last_clean":"2023-08-09T13:50:29.216076+0000","last_became_active":"2023-08-09T13:33:55.885781+0000","last_became_peered":"2023-08-09T13:33:55.885781+0000","last_unstale":"2023-08-09T13:50:29.216076+0000","last_undegraded":"2023-08-09T13:50:29.216076+0000","last_fullsized":"2023-08-09T13:50:29.216076+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:33:54.845869+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:33:54.845869+0000","last_clean_scrub_stamp":"2023-08-09T13:33:54.845869+0000","objects_scrubbed":0,"log_size":161,"log_dups_size":0,"ondisk_log_size":161,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T19:49:09.757332+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":500,"num_read_kb":924,"num_write":239,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"11.e","version":"0'0","reported_seq":9,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-08-09T13:50:29.232907+0000","last_change":"2023-08-09T13:50:29.227243+0000","last_active":"2023-08-09T13:50:29.232907+0000","last_peered":"2023-08-09T13:50:29.232907+0000","last_clean":"2023-08-09T13:50:29.232907+0000","last_became_active":"2023-08-09T13:50:29.226756+0000","last_became_peered":"2023-08-09T13:50:29.226756+0000","last_unstale":"2023-08-09T13:50:29.232907+0000","last_undegraded":"2023-08-09T13:50:29.232907+0000","last_fullsized":"2023-08-09T13:50:29.232907+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:50:28.150437+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:50:28.150437+0000","last_clean_scrub_stamp":"2023-08-09T13:50:28.150437+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T18:35:12.621508+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"10.f","version":"0'0","reported_seq":11,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-08-09T13:50:29.230485+0000","last_change":"2023-08-09T13:50:28.205120+0000","last_active":"2023-08-09T13:50:29.230485+0000","last_peered":"2023-08-09T13:50:29.230485+0000","last_clean":"2023-08-09T13:50:29.230485+0000","last_became_active":"2023-08-09T13:50:28.204771+0000","last_became_peered":"2023-08-09T13:50:28.204771+0000","last_unstale":"2023-08-09T13:50:29.230485+0000","last_undegraded":"2023-08-09T13:50:29.230485+0000","last_fullsized":"2023-08-09T13:50:29.230485+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:50:27.140775+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:50:27.140775+0000","last_clean_scrub_stamp":"2023-08-09T13:50:27.140775+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T17:34:29.132908+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":158,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-08-09T13:50:29.211123+0000","last_change":"2023-08-09T13:33:55.889565+0000","last_active":"2023-08-09T13:50:29.211123+0000","last_peered":"2023-08-09T13:50:29.211123+0000","last_clean":"2023-08-09T13:50:29.211123+0000","last_became_active":"2023-08-09T13:33:55.889164+0000","last_became_peered":"2023-08-09T13:33:55.889164+0000","last_unstale":"2023-08-09T13:50:29.211123+0000","last_undegraded":"2023-08-09T13:50:29.211123+0000","last_fullsized":"2023-08-09T13:50:29.211123+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:33:54.845869+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:33:54.845869+0000","last_clean_scrub_stamp":"2023-08-09T13:33:54.845869+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T17:48:30.355598+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.f","version":"0'0","reported_seq":8,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-08-09T13:50:29.238559+0000","last_change":"2023-08-09T13:50:29.238559+0000","last_active":"2023-08-09T13:50:29.238559+0000","last_peered":"2023-08-09T13:50:29.238559+0000","last_clean":"2023-08-09T13:50:29.238559+0000","last_became_active":"2023-08-09T13:50:29.238287+0000","last_became_peered":"2023-08-09T13:50:29.238287+0000","last_unstale":"2023-08-09T13:50:29.238559+0000","last_undegraded":"2023-08-09T13:50:29.238559+0000","last_fullsized":"2023-08-09T13:50:29.238559+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:50:28.150437+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:50:28.150437+0000","last_clean_scrub_stamp":"2023-08-09T13:50:28.150437+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T16:41:09.645304+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.e","version":"0'0","reported_seq":11,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-08-09T13:50:29.211102+0000","last_change":"2023-08-09T13:50:28.206561+0000","last_active":"2023-08-09T13:50:29.211102+0000","last_peered":"2023-08-09T13:50:29.211102+0000","last_clean":"2023-08-09T13:50:29.211102+0000","last_became_active":"2023-08-09T13:50:28.206264+0000","last_became_peered":"2023-08-09T13:50:28.206264+0000","last_unstale":"2023-08-09T13:50:29.211102+0000","last_undegraded":"2023-08-09T13:50:29.211102+0000","last_fullsized":"2023-08-09T13:50:29.211102+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:50:27.140775+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:50:27.140775+0000","last_clean_scrub_stamp":"2023-08-09T13:50:27.140775+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T15:29:27.312440+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":148,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-08-09T13:50:29.210681+0000","last_change":"2023-08-09T13:33:56.404236+0000","last_active":"2023-08-09T13:50:29.210681+0000","last_peered":"2023-08-09T13:50:29.210681+0000","last_clean":"2023-08-09T13:50:29.210681+0000","last_became_active":"2023-08-09T13:33:56.402485+0000","last_became_peered":"2023-08-09T13:33:56.402485+0000","last_unstale":"2023-08-09T13:50:29.210681+0000","last_undegraded":"2023-08-09T13:50:29.210681+0000","last_fullsized":"2023-08-09T13:50:29.210681+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:33:54.845869+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:33:54.845869+0000","last_clean_scrub_stamp":"2023-08-09T13:33:54.845869+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T13:49:18.686986+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.c","version":"0'0","reported_seq":1,"reported_epoch":88,"state":"creating+peering","last_fresh":"2023-08-09T13:50:28.187674+0000","last_change":"2023-08-09T13:50:28.187674+0000","last_active":"2023-08-09T13:50:28.150437+0000","last_peered":"2023-08-09T13:50:28.150437+0000","last_clean":"2023-08-09T13:50:28.150437+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-09T13:50:28.187674+0000","last_undegraded":"2023-08-09T13:50:28.187674+0000","last_fullsized":"2023-08-09T13:50:28.187674+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:50:28.150437+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:50:28.150437+0000","last_clean_scrub_stamp":"2023-08-09T13:50:28.150437+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.d","version":"0'0","reported_seq":11,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-08-09T13:50:29.227255+0000","last_change":"2023-08-09T13:50:28.199480+0000","last_active":"2023-08-09T13:50:29.227255+0000","last_peered":"2023-08-09T13:50:29.227255+0000","last_clean":"2023-08-09T13:50:29.227255+0000","last_became_active":"2023-08-09T13:50:28.199211+0000","last_became_peered":"2023-08-09T13:50:28.199211+0000","last_unstale":"2023-08-09T13:50:29.227255+0000","last_undegraded":"2023-08-09T13:50:29.227255+0000","last_fullsized":"2023-08-09T13:50:29.227255+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:50:27.140775+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:50:27.140775+0000","last_clean_scrub_stamp":"2023-08-09T13:50:27.140775+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T18:35:43.990648+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":146,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-08-09T13:50:29.237073+0000","last_change":"2023-08-09T13:33:56.401781+0000","last_active":"2023-08-09T13:50:29.237073+0000","last_peered":"2023-08-09T13:50:29.237073+0000","last_clean":"2023-08-09T13:50:29.237073+0000","last_became_active":"2023-08-09T13:33:56.400336+0000","last_became_peered":"2023-08-09T13:33:56.400336+0000","last_unstale":"2023-08-09T13:50:29.237073+0000","last_undegraded":"2023-08-09T13:50:29.237073+0000","last_fullsized":"2023-08-09T13:50:29.237073+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:33:54.845869+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:33:54.845869+0000","last_clean_scrub_stamp":"2023-08-09T13:33:54.845869+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-11T01:01:47.058457+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.d","version":"0'0","reported_seq":3,"reported_epoch":89,"state":"creating+activating","last_fresh":"2023-08-09T13:50:29.220533+0000","last_change":"2023-08-09T13:50:29.220533+0000","last_active":"2023-08-09T13:50:28.150437+0000","last_peered":"2023-08-09T13:50:28.150437+0000","last_clean":"2023-08-09T13:50:28.150437+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-09T13:50:29.220533+0000","last_undegraded":"2023-08-09T13:50:29.220533+0000","last_fullsized":"2023-08-09T13:50:29.220533+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:50:28.150437+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:50:28.150437+0000","last_clean_scrub_stamp":"2023-08-09T13:50:28.150437+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T17:52:40.304270+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":["2","4"],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.c","version":"0'0","reported_seq":11,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-08-09T13:50:29.216926+0000","last_change":"2023-08-09T13:50:28.203868+0000","last_active":"2023-08-09T13:50:29.216926+0000","last_peered":"2023-08-09T13:50:29.216926+0000","last_clean":"2023-08-09T13:50:29.216926+0000","last_became_active":"2023-08-09T13:50:28.203323+0000","last_became_peered":"2023-08-09T13:50:28.203323+0000","last_unstale":"2023-08-09T13:50:29.216926+0000","last_undegraded":"2023-08-09T13:50:29.216926+0000","last_fullsized":"2023-08-09T13:50:29.216926+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:50:27.140775+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:50:27.140775+0000","last_clean_scrub_stamp":"2023-08-09T13:50:27.140775+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T16:53:02.601172+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":146,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-08-09T13:50:29.230723+0000","last_change":"2023-08-09T13:33:55.886389+0000","last_active":"2023-08-09T13:50:29.230723+0000","last_peered":"2023-08-09T13:50:29.230723+0000","last_clean":"2023-08-09T13:50:29.230723+0000","last_became_active":"2023-08-09T13:33:55.885638+0000","last_became_peered":"2023-08-09T13:33:55.885638+0000","last_unstale":"2023-08-09T13:50:29.230723+0000","last_undegraded":"2023-08-09T13:50:29.230723+0000","last_fullsized":"2023-08-09T13:50:29.230723+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:33:54.845869+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:33:54.845869+0000","last_clean_scrub_stamp":"2023-08-09T13:33:54.845869+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T22:08:50.990029+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]}],"pool_stats":[{"poolid":11,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":10,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":64,"acting":64,"num_store_stats":8},{"poolid":2,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":500,"num_read_kb":924,"num_write":239,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":161,"ondisk_log_size":161,"up":2,"acting":2,"num_store_stats":2},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":8}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834575047,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":36840,"kb_used_data":1624,"kb_used_omap":1,"kb_used_meta":35198,"kb_avail":94335000,"statfs":{"total":96636764160,"available":96599040000,"internally_reserved":0,"allocated":1662976,"data_stored":1159636,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1781,"internal_metadata":36043019},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Aug 9 13:50:16 2023","interfaces":[{"interface":"back","average":{"1min":1.181,"5min":1.222,"15min":1.232},"min":{"1min":0.69499999999999995,"5min":0.63400000000000001,"15min":0.61099999999999999},"max":{"1min":1.633,"5min":1.8919999999999999,"15min":2.1280000000000001},"last":1.2569999999999999},{"interface":"front","average":{"1min":1.1819999999999999,"5min":1.232,"15min":1.24},"min":{"1min":0.72399999999999998,"5min":0.72399999999999998,"15min":0.65000000000000002},"max":{"1min":1.6100000000000001,"5min":1.839,"15min":2.0099999999999998},"last":1.1259999999999999}]},{"osd":1,"last update":"Wed Aug 9 13:50:16 2023","interfaces":[{"interface":"back","average":{"1min":1.2010000000000001,"5min":1.2450000000000001,"15min":1.256},"min":{"1min":0.72799999999999998,"5min":0.72799999999999998,"15min":0.59799999999999998},"max":{"1min":1.8,"5min":1.8,"15min":1.9470000000000001},"last":1.403},{"interface":"front","average":{"1min":1.2569999999999999,"5min":1.248,"15min":1.2869999999999999},"min":{"1min":0.83099999999999996,"5min":0.59299999999999997,"15min":0.59299999999999997},"max":{"1min":1.726,"5min":1.9410000000000001,"15min":1.9410000000000001},"last":1.327}]},{"osd":2,"last update":"Wed Aug 9 13:50:16 2023","interfaces":[{"interface":"back","average":{"1min":1.278,"5min":1.2470000000000001,"15min":1.2430000000000001},"min":{"1min":0.81000000000000005,"5min":0.65700000000000003,"15min":0.65700000000000003},"max":{"1min":1.742,"5min":2.0129999999999999,"15min":2.0129999999999999},"last":0.99399999999999999},{"interface":"front","average":{"1min":1.2889999999999999,"5min":1.323,"15min":1.3759999999999999},"min":{"1min":0.92600000000000005,"5min":0.80900000000000005,"15min":0.67800000000000005},"max":{"1min":1.7130000000000001,"5min":1.8740000000000001,"15min":2.4140000000000001},"last":1.623}]},{"osd":3,"last update":"Wed Aug 9 13:50:16 2023","interfaces":[{"interface":"back","average":{"1min":1.2529999999999999,"5min":1.3180000000000001,"15min":1.331},"min":{"1min":0.98799999999999999,"5min":0.86799999999999999,"15min":0.77500000000000002},"max":{"1min":1.7809999999999999,"5min":1.97,"15min":1.97},"last":1.3680000000000001},{"interface":"front","average":{"1min":1.3740000000000001,"5min":1.419,"15min":1.4159999999999999},"min":{"1min":0.93899999999999995,"5min":0.88300000000000001,"15min":0.82099999999999995},"max":{"1min":1.8160000000000001,"5min":2.0790000000000002,"15min":2.2480000000000002},"last":1.526}]},{"osd":4,"last update":"Wed Aug 9 13:50:16 2023","interfaces":[{"interface":"back","average":{"1min":1.343,"5min":1.2649999999999999,"15min":1.2769999999999999},"min":{"1min":0.86399999999999999,"5min":0.59499999999999997,"15min":0.59499999999999997},"max":{"1min":1.772,"5min":2.032,"15min":2.343},"last":0.82499999999999996},{"interface":"front","average":{"1min":1.296,"5min":1.3160000000000001,"15min":1.339},"min":{"1min":0.93600000000000005,"5min":0.77100000000000002,"15min":0.56799999999999995},"max":{"1min":1.6739999999999999,"5min":2.1419999999999999,"15min":2.1419999999999999},"last":1.2010000000000001}]},{"osd":5,"last update":"Wed Aug 9 13:50:16 2023","interfaces":[{"interface":"back","average":{"1min":1.4430000000000001,"5min":1.4350000000000001,"15min":1.419},"min":{"1min":1.048,"5min":0.88600000000000001,"15min":0.71499999999999997},"max":{"1min":1.9039999999999999,"5min":2.1480000000000001,"15min":2.6749999999999998},"last":1.6879999999999999},{"interface":"front","average":{"1min":1.4810000000000001,"5min":1.4319999999999999,"15min":1.4510000000000001},"min":{"1min":0.95999999999999996,"5min":0.85699999999999998,"15min":0.69199999999999995},"max":{"1min":1.877,"5min":2.113,"15min":2.9569999999999999},"last":1.4510000000000001}]},{"osd":6,"last update":"Wed Aug 9 13:50:16 2023","interfaces":[{"interface":"back","average":{"1min":1.6020000000000001,"5min":1.518,"15min":1.5489999999999999},"min":{"1min":1.2,"5min":0.85499999999999998,"15min":0.85499999999999998},"max":{"1min":1.8879999999999999,"5min":2.2069999999999999,"15min":2.8149999999999999},"last":1.575},{"interface":"front","average":{"1min":1.4139999999999999,"5min":1.4570000000000001,"15min":1.474},"min":{"1min":1.111,"5min":0.90600000000000003,"15min":0.60799999999999998},"max":{"1min":1.9419999999999999,"5min":2.1320000000000001,"15min":2.2530000000000001},"last":1.4910000000000001}]}]},{"osd":6,"up_from":13,"seq":55834575047,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32156,"kb_used_data":1040,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":94339684,"statfs":{"total":96636764160,"available":96603836416,"internally_reserved":0,"allocated":1064960,"data_stored":567907,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1781,"internal_metadata":31848715},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Aug 9 13:50:20 2023","interfaces":[{"interface":"back","average":{"1min":1.1160000000000001,"5min":1.143,"15min":1.161},"min":{"1min":0.75,"5min":0.72499999999999998,"15min":0.56599999999999995},"max":{"1min":1.5489999999999999,"5min":1.633,"15min":1.9870000000000001},"last":1.359},{"interface":"front","average":{"1min":1.21,"5min":1.532,"15min":1.3080000000000001},"min":{"1min":0.79300000000000004,"5min":0.79300000000000004,"15min":0.76000000000000001},"max":{"1min":1.6519999999999999,"5min":28.036000000000001,"15min":28.036000000000001},"last":1.4550000000000001}]},{"osd":1,"last update":"Wed Aug 9 13:50:20 2023","interfaces":[{"interface":"back","average":{"1min":1.147,"5min":1.272,"15min":1.226},"min":{"1min":0.70399999999999996,"5min":0.70399999999999996,"15min":0.65500000000000003},"max":{"1min":1.659,"5min":7.742,"15min":7.742},"last":1.256},{"interface":"front","average":{"1min":1.252,"5min":1.2849999999999999,"15min":1.276},"min":{"1min":0.84199999999999997,"5min":0.84199999999999997,"15min":0.68100000000000005},"max":{"1min":1.8899999999999999,"5min":1.8899999999999999,"15min":1.8899999999999999},"last":0.89100000000000001}]},{"osd":2,"last update":"Wed Aug 9 13:50:20 2023","interfaces":[{"interface":"back","average":{"1min":1.2989999999999999,"5min":1.3480000000000001,"15min":1.3220000000000001},"min":{"1min":0.877,"5min":0.81999999999999995,"15min":0.68700000000000006},"max":{"1min":1.73,"5min":4.0640000000000001,"15min":4.0640000000000001},"last":1.151},{"interface":"front","average":{"1min":1.244,"5min":1.4430000000000001,"15min":1.3460000000000001},"min":{"1min":0.76800000000000002,"5min":0.70999999999999996,"15min":0.70999999999999996},"max":{"1min":2.0390000000000001,"5min":13.154999999999999,"15min":13.154999999999999},"last":0.95399999999999996}]},{"osd":3,"last update":"Wed Aug 9 13:50:20 2023","interfaces":[{"interface":"back","average":{"1min":1.3939999999999999,"5min":1.52,"15min":1.419},"min":{"1min":0.73099999999999998,"5min":0.73099999999999998,"15min":0.73099999999999998},"max":{"1min":2.2519999999999998,"5min":13.349,"15min":13.349},"last":1.298},{"interface":"front","average":{"1min":1.258,"5min":1.3620000000000001,"15min":1.325},"min":{"1min":0.88400000000000001,"5min":0.76400000000000001,"15min":0.74299999999999999},"max":{"1min":1.8560000000000001,"5min":7.9029999999999996,"15min":7.9029999999999996},"last":1.397}]},{"osd":4,"last update":"Wed Aug 9 13:50:20 2023","interfaces":[{"interface":"back","average":{"1min":1.302,"5min":1.335,"15min":1.28},"min":{"1min":0.90700000000000003,"5min":0.78500000000000003,"15min":0.64300000000000002},"max":{"1min":1.992,"5min":5.2290000000000001,"15min":5.2290000000000001},"last":1.486},{"interface":"front","average":{"1min":1.4019999999999999,"5min":1.498,"15min":1.389},"min":{"1min":0.88900000000000001,"5min":0.78700000000000003,"15min":0.69099999999999995},"max":{"1min":1.734,"5min":12.1,"15min":12.1},"last":1.0860000000000001}]},{"osd":5,"last update":"Wed Aug 9 13:50:20 2023","interfaces":[{"interface":"back","average":{"1min":1.3169999999999999,"5min":1.4399999999999999,"15min":1.357},"min":{"1min":0.93999999999999995,"5min":0.83799999999999997,"15min":0.65400000000000003},"max":{"1min":2.1139999999999999,"5min":8.1899999999999995,"15min":8.1899999999999995},"last":1.0289999999999999},{"interface":"front","average":{"1min":1.5649999999999999,"5min":1.6639999999999999,"15min":1.589},"min":{"1min":1.2010000000000001,"5min":1.2010000000000001,"15min":0.83799999999999997},"max":{"1min":2.2000000000000002,"5min":8.8889999999999993,"15min":8.8889999999999993},"last":1.4299999999999999}]},{"osd":7,"last update":"Wed Aug 9 13:50:20 2023","interfaces":[{"interface":"back","average":{"1min":1.577,"5min":1.6970000000000001,"15min":1.5880000000000001},"min":{"1min":1.1850000000000001,"5min":1.0029999999999999,"15min":0.62},"max":{"1min":1.8240000000000001,"5min":9.3320000000000007,"15min":9.3320000000000007},"last":1.5289999999999999},{"interface":"front","average":{"1min":1.4410000000000001,"5min":1.6060000000000001,"15min":1.548},"min":{"1min":0.93899999999999995,"5min":0.88500000000000001,"15min":0.88500000000000001},"max":{"1min":1.9470000000000001,"5min":8.0399999999999991,"15min":8.0399999999999991},"last":1.5029999999999999}]}]},{"osd":5,"up_from":13,"seq":55834575047,"num_pgs":16,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":36260,"kb_used_data":1044,"kb_used_omap":1,"kb_used_meta":35198,"kb_avail":94335580,"statfs":{"total":96636764160,"available":96599633920,"internally_reserved":0,"allocated":1069056,"data_stored":569268,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1781,"internal_metadata":36043019},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Aug 9 13:50:19 2023","interfaces":[{"interface":"back","average":{"1min":1.2509999999999999,"5min":1.1759999999999999,"15min":1.179},"min":{"1min":0.63500000000000001,"5min":0.63500000000000001,"15min":0.63500000000000001},"max":{"1min":1.518,"5min":2.1869999999999998,"15min":2.1869999999999998},"last":1.1020000000000001},{"interface":"front","average":{"1min":1.1719999999999999,"5min":1.2,"15min":1.198},"min":{"1min":0.68600000000000005,"5min":0.68600000000000005,"15min":0.64200000000000002},"max":{"1min":1.579,"5min":1.6619999999999999,"15min":1.7989999999999999},"last":1.1950000000000001}]},{"osd":1,"last update":"Wed Aug 9 13:50:19 2023","interfaces":[{"interface":"back","average":{"1min":1.165,"5min":1.1990000000000001,"15min":1.2190000000000001},"min":{"1min":0.81599999999999995,"5min":0.73099999999999998,"15min":0.64100000000000001},"max":{"1min":1.631,"5min":1.6739999999999999,"15min":1.736},"last":1.0049999999999999},{"interface":"front","average":{"1min":1.264,"5min":1.296,"15min":1.2769999999999999},"min":{"1min":0.93300000000000005,"5min":0.66900000000000004,"15min":0.66900000000000004},"max":{"1min":1.6699999999999999,"5min":1.887,"15min":1.887},"last":0.84999999999999998}]},{"osd":2,"last update":"Wed Aug 9 13:50:19 2023","interfaces":[{"interface":"back","average":{"1min":1.2410000000000001,"5min":1.2569999999999999,"15min":1.2849999999999999},"min":{"1min":0.98199999999999998,"5min":0.68999999999999995,"15min":0.61599999999999999},"max":{"1min":1.5760000000000001,"5min":1.756,"15min":1.899},"last":1.4670000000000001},{"interface":"front","average":{"1min":1.3049999999999999,"5min":1.302,"15min":1.3149999999999999},"min":{"1min":0.78400000000000003,"5min":0.78400000000000003,"15min":0.66800000000000004},"max":{"1min":1.6599999999999999,"5min":1.7190000000000001,"15min":1.7969999999999999},"last":1.1419999999999999}]},{"osd":3,"last update":"Wed Aug 9 13:50:19 2023","interfaces":[{"interface":"back","average":{"1min":1.365,"5min":1.3720000000000001,"15min":1.3600000000000001},"min":{"1min":0.95399999999999996,"5min":0.95399999999999996,"15min":0.59799999999999998},"max":{"1min":1.6890000000000001,"5min":2.1840000000000002,"15min":2.2690000000000001},"last":1.3160000000000001},{"interface":"front","average":{"1min":1.4259999999999999,"5min":1.4379999999999999,"15min":1.4139999999999999},"min":{"1min":1.087,"5min":0.94899999999999995,"15min":0.70999999999999996},"max":{"1min":1.73,"5min":2.3780000000000001,"15min":2.3780000000000001},"last":1.3600000000000001}]},{"osd":4,"last update":"Wed Aug 9 13:50:19 2023","interfaces":[{"interface":"back","average":{"1min":1.262,"5min":1.2729999999999999,"15min":1.2889999999999999},"min":{"1min":0.92700000000000005,"5min":0.84699999999999998,"15min":0.56599999999999995},"max":{"1min":1.673,"5min":2.0800000000000001,"15min":2.121},"last":1.2789999999999999},{"interface":"front","average":{"1min":1.3919999999999999,"5min":1.3400000000000001,"15min":1.3779999999999999},"min":{"1min":0.81999999999999995,"5min":0.76100000000000001,"15min":0.58599999999999997},"max":{"1min":1.944,"5min":2.1230000000000002,"15min":2.3679999999999999},"last":1.403}]},{"osd":6,"last update":"Wed Aug 9 13:50:19 2023","interfaces":[{"interface":"back","average":{"1min":1.4299999999999999,"5min":1.3520000000000001,"15min":1.391},"min":{"1min":0.99099999999999999,"5min":0.90300000000000002,"15min":0.66700000000000004},"max":{"1min":1.9810000000000001,"5min":2.306,"15min":2.306},"last":1.5609999999999999},{"interface":"front","average":{"1min":1.49,"5min":1.4399999999999999,"15min":1.4710000000000001},"min":{"1min":1.1220000000000001,"5min":1.034,"15min":0.71199999999999997},"max":{"1min":1.827,"5min":2.2320000000000002,"15min":2.2320000000000002},"last":1.343}]},{"osd":7,"last update":"Wed Aug 9 13:50:19 2023","interfaces":[{"interface":"back","average":{"1min":1.577,"5min":1.5580000000000001,"15min":1.5529999999999999},"min":{"1min":1.1160000000000001,"5min":1.0589999999999999,"15min":0.70899999999999996},"max":{"1min":1.9610000000000001,"5min":2.452,"15min":2.4729999999999999},"last":1.5980000000000001},{"interface":"front","average":{"1min":1.611,"5min":1.585,"15min":1.6080000000000001},"min":{"1min":1.2110000000000001,"5min":1.163,"15min":0.77600000000000002},"max":{"1min":2.097,"5min":2.1070000000000002,"15min":2.3820000000000001},"last":1.6499999999999999}]}]},{"osd":4,"up_from":13,"seq":55834575047,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32160,"kb_used_data":1044,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":94339680,"statfs":{"total":96636764160,"available":96603832320,"internally_reserved":0,"allocated":1069056,"data_stored":569268,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1781,"internal_metadata":31848715},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Aug 9 13:50:21 2023","interfaces":[{"interface":"back","average":{"1min":1.2330000000000001,"5min":1.2390000000000001,"15min":1.2030000000000001},"min":{"1min":0.875,"5min":0.70699999999999996,"15min":0.67700000000000005},"max":{"1min":1.5940000000000001,"5min":2.524,"15min":2.524},"last":1.179},{"interface":"front","average":{"1min":1.206,"5min":1.236,"15min":1.2569999999999999},"min":{"1min":0.62,"5min":0.62,"15min":0.62},"max":{"1min":1.5980000000000001,"5min":1.863,"15min":1.863},"last":1.468}]},{"osd":1,"last update":"Wed Aug 9 13:50:21 2023","interfaces":[{"interface":"back","average":{"1min":1.244,"5min":1.3169999999999999,"15min":1.2929999999999999},"min":{"1min":0.70999999999999996,"5min":0.67800000000000005,"15min":0.67800000000000005},"max":{"1min":1.48,"5min":1.8480000000000001,"15min":1.8480000000000001},"last":1.026},{"interface":"front","average":{"1min":1.242,"5min":1.2649999999999999,"15min":1.248},"min":{"1min":0.81799999999999995,"5min":0.81799999999999995,"15min":0.64900000000000002},"max":{"1min":1.9330000000000001,"5min":1.9330000000000001,"15min":1.9330000000000001},"last":0.98499999999999999}]},{"osd":2,"last update":"Wed Aug 9 13:50:21 2023","interfaces":[{"interface":"back","average":{"1min":1.306,"5min":1.2869999999999999,"15min":1.2809999999999999},"min":{"1min":1.022,"5min":0.621,"15min":0.621},"max":{"1min":1.619,"5min":2.6429999999999998,"15min":2.6429999999999998},"last":0.69999999999999996},{"interface":"front","average":{"1min":1.343,"5min":1.3720000000000001,"15min":1.3660000000000001},"min":{"1min":0.88300000000000001,"5min":0.88300000000000001,"15min":0.74299999999999999},"max":{"1min":1.7569999999999999,"5min":1.7569999999999999,"15min":1.8149999999999999},"last":0.89000000000000001}]},{"osd":3,"last update":"Wed Aug 9 13:50:21 2023","interfaces":[{"interface":"back","average":{"1min":1.377,"5min":1.3300000000000001,"15min":1.298},"min":{"1min":1.083,"5min":0.86399999999999999,"15min":0.84699999999999998},"max":{"1min":1.6739999999999999,"5min":2.6099999999999999,"15min":2.6099999999999999},"last":1.212},{"interface":"front","average":{"1min":1.4099999999999999,"5min":1.3979999999999999,"15min":1.3799999999999999},"min":{"1min":0.94799999999999995,"5min":0.88800000000000001,"15min":0.63100000000000001},"max":{"1min":1.6890000000000001,"5min":2.7629999999999999,"15min":2.7629999999999999},"last":0.95599999999999996}]},{"osd":5,"last update":"Wed Aug 9 13:50:21 2023","interfaces":[{"interface":"back","average":{"1min":1.288,"5min":1.3400000000000001,"15min":1.3300000000000001},"min":{"1min":0.876,"5min":0.73199999999999998,"15min":0.63200000000000001},"max":{"1min":1.734,"5min":1.8,"15min":2.0099999999999998},"last":1.343},{"interface":"front","average":{"1min":1.2509999999999999,"5min":1.323,"15min":1.3180000000000001},"min":{"1min":0.92000000000000004,"5min":0.72399999999999998,"15min":0.626},"max":{"1min":1.71,"5min":2.7090000000000001,"15min":2.7090000000000001},"last":1.421}]},{"osd":6,"last update":"Wed Aug 9 13:50:21 2023","interfaces":[{"interface":"back","average":{"1min":1.379,"5min":1.3160000000000001,"15min":1.3360000000000001},"min":{"1min":0.94699999999999995,"5min":0.74399999999999999,"15min":0.59099999999999997},"max":{"1min":1.7509999999999999,"5min":2.8559999999999999,"15min":2.8559999999999999},"last":1.083},{"interface":"front","average":{"1min":1.4430000000000001,"5min":1.518,"15min":1.504},"min":{"1min":1.0760000000000001,"5min":0.97799999999999998,"15min":0.76400000000000001},"max":{"1min":1.669,"5min":2.2719999999999998,"15min":2.2719999999999998},"last":1.5409999999999999}]},{"osd":7,"last update":"Wed Aug 9 13:50:21 2023","interfaces":[{"interface":"back","average":{"1min":1.46,"5min":1.4830000000000001,"15min":1.474},"min":{"1min":1.1499999999999999,"5min":0.85999999999999999,"15min":0.73899999999999999},"max":{"1min":1.8460000000000001,"5min":2.8050000000000002,"15min":2.8050000000000002},"last":1.1100000000000001},{"interface":"front","average":{"1min":1.474,"5min":1.522,"15min":1.514},"min":{"1min":1.236,"5min":0.97199999999999998,"15min":0.72899999999999998},"max":{"1min":1.754,"5min":3.415,"15min":3.415},"last":1.3959999999999999}]}]},{"osd":3,"up_from":13,"seq":55834575048,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":36260,"kb_used_data":1044,"kb_used_omap":1,"kb_used_meta":35198,"kb_avail":94335580,"statfs":{"total":96636764160,"available":96599633920,"internally_reserved":0,"allocated":1069056,"data_stored":569268,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1781,"internal_metadata":36043019},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Aug 9 13:50:25 2023","interfaces":[{"interface":"back","average":{"1min":0.84599999999999997,"5min":0.88300000000000001,"15min":0.91600000000000004},"min":{"1min":0.43099999999999999,"5min":0.27200000000000002,"15min":0.27200000000000002},"max":{"1min":1.3939999999999999,"5min":1.8500000000000001,"15min":1.968},"last":0.65900000000000003},{"interface":"front","average":{"1min":1.071,"5min":1.0920000000000001,"15min":1.1060000000000001},"min":{"1min":0.48199999999999998,"5min":0.36399999999999999,"15min":0.36399999999999999},"max":{"1min":1.6910000000000001,"5min":1.6910000000000001,"15min":2.2360000000000002},"last":0.75600000000000001}]},{"osd":1,"last update":"Wed Aug 9 13:50:25 2023","interfaces":[{"interface":"back","average":{"1min":1.042,"5min":1.0089999999999999,"15min":1.012},"min":{"1min":0.59899999999999998,"5min":0.496,"15min":0.438},"max":{"1min":1.518,"5min":1.7110000000000001,"15min":2.5649999999999999},"last":0.83999999999999997},{"interface":"front","average":{"1min":1.091,"5min":1.1279999999999999,"15min":1.1639999999999999},"min":{"1min":0.51800000000000002,"5min":0.51800000000000002,"15min":0.42099999999999999},"max":{"1min":1.6839999999999999,"5min":1.6839999999999999,"15min":2.6760000000000002},"last":1.202}]},{"osd":2,"last update":"Wed Aug 9 13:50:25 2023","interfaces":[{"interface":"back","average":{"1min":1.121,"5min":1.177,"15min":1.1930000000000001},"min":{"1min":0.53600000000000003,"5min":0.53600000000000003,"15min":0.53500000000000003},"max":{"1min":1.6240000000000001,"5min":1.913,"15min":1.923},"last":1.1399999999999999},{"interface":"front","average":{"1min":1.1899999999999999,"5min":1.2929999999999999,"15min":1.3080000000000001},"min":{"1min":0.625,"5min":0.52300000000000002,"15min":0.52000000000000002},"max":{"1min":1.9399999999999999,"5min":2.0259999999999998,"15min":2.0259999999999998},"last":1.0960000000000001}]},{"osd":4,"last update":"Wed Aug 9 13:50:25 2023","interfaces":[{"interface":"back","average":{"1min":1.232,"5min":1.258,"15min":1.282},"min":{"1min":0.91500000000000004,"5min":0.57899999999999996,"15min":0.57899999999999996},"max":{"1min":1.742,"5min":1.8160000000000001,"15min":1.8160000000000001},"last":1.6819999999999999},{"interface":"front","average":{"1min":1.349,"5min":1.397,"15min":1.3919999999999999},"min":{"1min":0.67600000000000005,"5min":0.628,"15min":0.628},"max":{"1min":1.8149999999999999,"5min":2.2040000000000002,"15min":2.2370000000000001},"last":1.3080000000000001}]},{"osd":5,"last update":"Wed Aug 9 13:50:25 2023","interfaces":[{"interface":"back","average":{"1min":1.456,"5min":1.468,"15min":1.474},"min":{"1min":0.67600000000000005,"5min":0.67600000000000005,"15min":0.67200000000000004},"max":{"1min":1.855,"5min":2.254,"15min":2.254},"last":1.3959999999999999},{"interface":"front","average":{"1min":1.4850000000000001,"5min":1.4910000000000001,"15min":1.49},"min":{"1min":0.77600000000000002,"5min":0.65900000000000003,"15min":0.65900000000000003},"max":{"1min":2.3069999999999999,"5min":2.3069999999999999,"15min":2.3069999999999999},"last":1.3260000000000001}]},{"osd":6,"last update":"Wed Aug 9 13:50:25 2023","interfaces":[{"interface":"back","average":{"1min":1.3759999999999999,"5min":1.4370000000000001,"15min":1.4590000000000001},"min":{"1min":0.878,"5min":0.68200000000000005,"15min":0.68200000000000005},"max":{"1min":1.8759999999999999,"5min":2.1829999999999998,"15min":2.242},"last":1.4670000000000001},{"interface":"front","average":{"1min":1.466,"5min":1.5580000000000001,"15min":1.5529999999999999},"min":{"1min":1.0600000000000001,"5min":0.71099999999999997,"15min":0.68600000000000005},"max":{"1min":1.825,"5min":2.3420000000000001,"15min":2.3420000000000001},"last":1.252}]},{"osd":7,"last update":"Wed Aug 9 13:50:25 2023","interfaces":[{"interface":"back","average":{"1min":1.601,"5min":1.627,"15min":1.6160000000000001},"min":{"1min":1.115,"5min":0.69399999999999995,"15min":0.69399999999999995},"max":{"1min":2.46,"5min":2.46,"15min":2.46},"last":1.4279999999999999},{"interface":"front","average":{"1min":1.4910000000000001,"5min":1.546,"15min":1.5600000000000001},"min":{"1min":0.78200000000000003,"5min":0.64200000000000002,"15min":0.61599999999999999},"max":{"1min":2.0579999999999998,"5min":2.359,"15min":2.484},"last":1.571}]}]},{"osd":2,"up_from":13,"seq":55834575047,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":36256,"kb_used_data":1044,"kb_used_omap":1,"kb_used_meta":35198,"kb_avail":94335584,"statfs":{"total":96636764160,"available":96599638016,"internally_reserved":0,"allocated":1069056,"data_stored":569268,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1781,"internal_metadata":36043019},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Aug 9 13:50:17 2023","interfaces":[{"interface":"back","average":{"1min":0.96999999999999997,"5min":0.95299999999999996,"15min":0.95499999999999996},"min":{"1min":0.47799999999999998,"5min":0.36499999999999999,"15min":0.36499999999999999},"max":{"1min":1.5760000000000001,"5min":1.6180000000000001,"15min":1.8380000000000001},"last":1.484},{"interface":"front","average":{"1min":0.97099999999999997,"5min":0.998,"15min":1.0049999999999999},"min":{"1min":0.45300000000000001,"5min":0.374,"15min":0.374},"max":{"1min":1.4199999999999999,"5min":2.3929999999999998,"15min":2.3929999999999998},"last":0.80200000000000005}]},{"osd":1,"last update":"Wed Aug 9 13:50:17 2023","interfaces":[{"interface":"back","average":{"1min":1.2529999999999999,"5min":1.1739999999999999,"15min":1.1870000000000001},"min":{"1min":0.61299999999999999,"5min":0.55200000000000005,"15min":0.55200000000000005},"max":{"1min":1.6870000000000001,"5min":1.77,"15min":1.8420000000000001},"last":1.615},{"interface":"front","average":{"1min":1.216,"5min":1.2749999999999999,"15min":1.335},"min":{"1min":0.52500000000000002,"5min":0.52500000000000002,"15min":0.52500000000000002},"max":{"1min":1.7390000000000001,"5min":2.4870000000000001,"15min":2.4870000000000001},"last":1.03}]},{"osd":3,"last update":"Wed Aug 9 13:50:17 2023","interfaces":[{"interface":"back","average":{"1min":0.98099999999999998,"5min":1.006,"15min":1.0469999999999999},"min":{"1min":0.44800000000000001,"5min":0.38900000000000001,"15min":0.38900000000000001},"max":{"1min":1.3109999999999999,"5min":1.708,"15min":2.0409999999999999},"last":0.63600000000000001},{"interface":"front","average":{"1min":1.2110000000000001,"5min":1.304,"15min":1.3089999999999999},"min":{"1min":0.55500000000000005,"5min":0.46200000000000002,"15min":0.46200000000000002},"max":{"1min":1.8009999999999999,"5min":8.8130000000000006,"15min":8.8130000000000006},"last":1.2529999999999999}]},{"osd":4,"last update":"Wed Aug 9 13:50:17 2023","interfaces":[{"interface":"back","average":{"1min":1.415,"5min":1.4179999999999999,"15min":1.4510000000000001},"min":{"1min":0.83999999999999997,"5min":0.628,"15min":0.628},"max":{"1min":1.958,"5min":2.089,"15min":2.089},"last":1.556},{"interface":"front","average":{"1min":1.462,"5min":1.359,"15min":1.3979999999999999},"min":{"1min":0.73799999999999999,"5min":0.68600000000000005,"15min":0.68600000000000005},"max":{"1min":2.472,"5min":2.472,"15min":2.472},"last":1.151}]},{"osd":5,"last update":"Wed Aug 9 13:50:17 2023","interfaces":[{"interface":"back","average":{"1min":1.5189999999999999,"5min":1.4379999999999999,"15min":1.458},"min":{"1min":0.76600000000000001,"5min":0.59999999999999998,"15min":0.59999999999999998},"max":{"1min":2.27,"5min":2.5840000000000001,"15min":2.5840000000000001},"last":1.3240000000000001},{"interface":"front","average":{"1min":1.304,"5min":1.3540000000000001,"15min":1.349},"min":{"1min":0.77100000000000002,"5min":0.56799999999999995,"15min":0.56799999999999995},"max":{"1min":1.8859999999999999,"5min":3.5659999999999998,"15min":3.5659999999999998},"last":1.379}]},{"osd":6,"last update":"Wed Aug 9 13:50:17 2023","interfaces":[{"interface":"back","average":{"1min":1.4019999999999999,"5min":1.425,"15min":1.4259999999999999},"min":{"1min":0.79800000000000004,"5min":0.73799999999999999,"15min":0.73799999999999999},"max":{"1min":2.222,"5min":3.5099999999999998,"15min":3.5099999999999998},"last":1.079},{"interface":"front","average":{"1min":1.4790000000000001,"5min":1.4990000000000001,"15min":1.5409999999999999},"min":{"1min":0.82199999999999995,"5min":0.66300000000000003,"15min":0.66300000000000003},"max":{"1min":2.3769999999999998,"5min":2.3769999999999998,"15min":2.3769999999999998},"last":1.6930000000000001}]},{"osd":7,"last update":"Wed Aug 9 13:50:17 2023","interfaces":[{"interface":"back","average":{"1min":1.54,"5min":1.51,"15min":1.548},"min":{"1min":0.78800000000000003,"5min":0.64600000000000002,"15min":0.64600000000000002},"max":{"1min":2.1970000000000001,"5min":2.5249999999999999,"15min":2.5249999999999999},"last":1.4379999999999999},{"interface":"front","average":{"1min":1.4970000000000001,"5min":1.52,"15min":1.51},"min":{"1min":0.99199999999999999,"5min":0.70499999999999996,"15min":0.70499999999999996},"max":{"1min":2.089,"5min":3.609,"15min":3.609},"last":1.214}]}]},{"osd":1,"up_from":13,"seq":55834575047,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28644,"kb_used_data":1624,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343196,"statfs":{"total":96636764160,"available":96607432704,"internally_reserved":0,"allocated":1662976,"data_stored":1159636,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1781,"internal_metadata":27654411},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Aug 9 13:49:27 2023","interfaces":[{"interface":"back","average":{"1min":0.98399999999999999,"5min":0.90500000000000003,"15min":0.93899999999999995},"min":{"1min":0.64500000000000002,"5min":0.50900000000000001,"15min":0.32400000000000001},"max":{"1min":1.575,"5min":1.575,"15min":2.7080000000000002},"last":1.272},{"interface":"front","average":{"1min":1.1719999999999999,"5min":1.093,"15min":1.1000000000000001},"min":{"1min":0.74299999999999999,"5min":0.432,"15min":0.42499999999999999},"max":{"1min":2.468,"5min":2.468,"15min":2.6349999999999998},"last":1.7190000000000001}]},{"osd":2,"last update":"Wed Aug 9 13:49:27 2023","interfaces":[{"interface":"back","average":{"1min":1.0249999999999999,"5min":0.97699999999999998,"15min":0.99299999999999999},"min":{"1min":0.623,"5min":0.41099999999999998,"15min":0.41099999999999998},"max":{"1min":1.3160000000000001,"5min":1.907,"15min":1.907},"last":1.087},{"interface":"front","average":{"1min":1.212,"5min":1.1870000000000001,"15min":1.1910000000000001},"min":{"1min":0.74299999999999999,"5min":0.54400000000000004,"15min":0.46500000000000002},"max":{"1min":2.1280000000000001,"5min":2.1280000000000001,"15min":2.2330000000000001},"last":1.4970000000000001}]},{"osd":3,"last update":"Wed Aug 9 13:49:27 2023","interfaces":[{"interface":"back","average":{"1min":1.3320000000000001,"5min":1.2949999999999999,"15min":1.325},"min":{"1min":0.90500000000000003,"5min":0.54400000000000004,"15min":0.432},"max":{"1min":2.0270000000000001,"5min":2.0270000000000001,"15min":2.3210000000000002},"last":1.6759999999999999},{"interface":"front","average":{"1min":1.27,"5min":1.258,"15min":1.252},"min":{"1min":0.96699999999999997,"5min":0.49299999999999999,"15min":0.44400000000000001},"max":{"1min":2.3540000000000001,"5min":2.3540000000000001,"15min":3.3069999999999999},"last":1.2030000000000001}]},{"osd":4,"last update":"Wed Aug 9 13:49:27 2023","interfaces":[{"interface":"back","average":{"1min":1.4379999999999999,"5min":1.4330000000000001,"15min":1.4119999999999999},"min":{"1min":1.1899999999999999,"5min":0.79800000000000004,"15min":0.58099999999999996},"max":{"1min":1.966,"5min":1.97,"15min":2.504},"last":1.5940000000000001},{"interface":"front","average":{"1min":1.3640000000000001,"5min":1.3360000000000001,"15min":1.3400000000000001},"min":{"1min":0.871,"5min":0.70099999999999996,"15min":0.70099999999999996},"max":{"1min":1.6950000000000001,"5min":1.901,"15min":2.5099999999999998},"last":1.425}]},{"osd":5,"last update":"Wed Aug 9 13:49:27 2023","interfaces":[{"interface":"back","average":{"1min":1.3149999999999999,"5min":1.3620000000000001,"15min":1.377},"min":{"1min":1.0389999999999999,"5min":0.76900000000000002,"15min":0.76900000000000002},"max":{"1min":1.55,"5min":2.294,"15min":2.294},"last":1.55},{"interface":"front","average":{"1min":1.456,"5min":1.502,"15min":1.538},"min":{"1min":0.98599999999999999,"5min":0.91000000000000003,"15min":0.77700000000000002},"max":{"1min":1.738,"5min":2.1389999999999998,"15min":4.9560000000000004},"last":1.837}]},{"osd":6,"last update":"Wed Aug 9 13:49:27 2023","interfaces":[{"interface":"back","average":{"1min":1.5329999999999999,"5min":1.5109999999999999,"15min":1.5209999999999999},"min":{"1min":1.0740000000000001,"5min":0.77500000000000002,"15min":0.67400000000000004},"max":{"1min":2.2080000000000002,"5min":2.2080000000000002,"15min":3.0529999999999999},"last":1.925},{"interface":"front","average":{"1min":1.4510000000000001,"5min":1.478,"15min":1.508},"min":{"1min":0.94199999999999995,"5min":0.82499999999999996,"15min":0.79800000000000004},"max":{"1min":1.8160000000000001,"5min":2.1040000000000001,"15min":2.7330000000000001},"last":1.774}]},{"osd":7,"last update":"Wed Aug 9 13:49:27 2023","interfaces":[{"interface":"back","average":{"1min":1.5629999999999999,"5min":1.601,"15min":1.637},"min":{"1min":0.92000000000000004,"5min":0.86899999999999999,"15min":0.77100000000000002},"max":{"1min":1.9339999999999999,"5min":2.25,"15min":5.1360000000000001},"last":2.4169999999999998},{"interface":"front","average":{"1min":1.5600000000000001,"5min":1.5760000000000001,"15min":1.605},"min":{"1min":1.008,"5min":0.91300000000000003,"15min":0.75900000000000001},"max":{"1min":2.0680000000000001,"5min":2.1040000000000001,"15min":3.194},"last":2.036}]}]},{"osd":0,"up_from":13,"seq":55834575047,"num_pgs":7,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":40348,"kb_used_data":1040,"kb_used_omap":1,"kb_used_meta":39294,"kb_avail":94331492,"statfs":{"total":96636764160,"available":96595447808,"internally_reserved":0,"allocated":1064960,"data_stored":567907,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1781,"internal_metadata":40237323},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Wed Aug 9 13:50:27 2023","interfaces":[{"interface":"back","average":{"1min":0.91800000000000004,"5min":0.92200000000000004,"15min":0.91600000000000004},"min":{"1min":0.32500000000000001,"5min":0.32500000000000001,"15min":0.32500000000000001},"max":{"1min":1.5820000000000001,"5min":1.756,"15min":1.756},"last":0.78500000000000003},{"interface":"front","average":{"1min":1.0680000000000001,"5min":1.1020000000000001,"15min":1.113},"min":{"1min":0.47099999999999997,"5min":0.45300000000000001,"15min":0.45300000000000001},"max":{"1min":1.9019999999999999,"5min":1.9019999999999999,"15min":1.9019999999999999},"last":0.85499999999999998}]},{"osd":2,"last update":"Wed Aug 9 13:50:27 2023","interfaces":[{"interface":"back","average":{"1min":1.048,"5min":1.1000000000000001,"15min":1.135},"min":{"1min":0.53800000000000003,"5min":0.51400000000000001,"15min":0.44800000000000001},"max":{"1min":1.702,"5min":1.776,"15min":1.889},"last":1.28},{"interface":"front","average":{"1min":1.1819999999999999,"5min":1.149,"15min":1.179},"min":{"1min":0.58499999999999996,"5min":0.56399999999999995,"15min":0.52700000000000002},"max":{"1min":2.0129999999999999,"5min":2.0129999999999999,"15min":2.2519999999999998},"last":1.4370000000000001}]},{"osd":3,"last update":"Wed Aug 9 13:50:27 2023","interfaces":[{"interface":"back","average":{"1min":0.98499999999999999,"5min":1.0509999999999999,"15min":1.105},"min":{"1min":0.54900000000000004,"5min":0.46300000000000002,"15min":0.46300000000000002},"max":{"1min":1.345,"5min":1.823,"15min":2.5190000000000001},"last":1.024},{"interface":"front","average":{"1min":1.272,"5min":1.2629999999999999,"15min":1.3140000000000001},"min":{"1min":0.56399999999999995,"5min":0.54700000000000004,"15min":0.54700000000000004},"max":{"1min":2.347,"5min":2.347,"15min":2.4649999999999999},"last":1.1459999999999999}]},{"osd":4,"last update":"Wed Aug 9 13:50:27 2023","interfaces":[{"interface":"back","average":{"1min":1.302,"5min":1.3129999999999999,"15min":1.347},"min":{"1min":0.84699999999999998,"5min":0.84699999999999998,"15min":0.74299999999999999},"max":{"1min":1.9630000000000001,"5min":2.2309999999999999,"15min":2.2309999999999999},"last":1.345},{"interface":"front","average":{"1min":1.3999999999999999,"5min":1.4390000000000001,"15min":1.452},"min":{"1min":0.82699999999999996,"5min":0.77600000000000002,"15min":0.74099999999999999},"max":{"1min":1.8440000000000001,"5min":2.5390000000000001,"15min":2.5390000000000001},"last":1.554}]},{"osd":5,"last update":"Wed Aug 9 13:50:27 2023","interfaces":[{"interface":"back","average":{"1min":1.387,"5min":1.4219999999999999,"15min":1.4650000000000001},"min":{"1min":0.66300000000000003,"5min":0.66300000000000003,"15min":0.66300000000000003},"max":{"1min":2.6669999999999998,"5min":2.6669999999999998,"15min":2.6669999999999998},"last":1.1890000000000001},{"interface":"front","average":{"1min":1.3400000000000001,"5min":1.415,"15min":1.4490000000000001},"min":{"1min":0.70099999999999996,"5min":0.629,"15min":0.629},"max":{"1min":1.6160000000000001,"5min":2.1499999999999999,"15min":2.1499999999999999},"last":1.462}]},{"osd":6,"last update":"Wed Aug 9 13:50:27 2023","interfaces":[{"interface":"back","average":{"1min":1.51,"5min":1.532,"15min":1.5469999999999999},"min":{"1min":0.73599999999999999,"5min":0.73599999999999999,"15min":0.69799999999999995},"max":{"1min":2.3100000000000001,"5min":2.7250000000000001,"15min":2.7250000000000001},"last":1.3740000000000001},{"interface":"front","average":{"1min":1.5169999999999999,"5min":1.5389999999999999,"15min":1.5600000000000001},"min":{"1min":0.86699999999999999,"5min":0.86699999999999999,"15min":0.86699999999999999},"max":{"1min":2.3199999999999998,"5min":2.399,"15min":2.399},"last":1.306}]},{"osd":7,"last update":"Wed Aug 9 13:50:27 2023","interfaces":[{"interface":"back","average":{"1min":1.532,"5min":1.595,"15min":1.625},"min":{"1min":0.72299999999999998,"5min":0.72299999999999998,"15min":0.72299999999999998},"max":{"1min":2.3900000000000001,"5min":2.3900000000000001,"15min":2.464},"last":1.5149999999999999},{"interface":"front","average":{"1min":1.4350000000000001,"5min":1.512,"15min":1.55},"min":{"1min":0.84499999999999997,"5min":0.76700000000000002,"15min":0.76700000000000002},"max":{"1min":2.274,"5min":2.274,"15min":2.274},"last":1.603}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-08-09T13:50:33.748 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-08-09T13:50:34.005 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:34.002+0000 7f6234c57640 1 Processor -- start 2023-08-09T13:50:34.005 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:34.002+0000 7f6234c57640 1 -- start start 2023-08-09T13:50:34.008 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:34.004+0000 7f6234c57640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f623006cfe0 0x7f62300c8b50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:50:34.009 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:34.004+0000 7f6234c57640 1 -- --> v1:172.21.15.43:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f62300c9090 con 0x7f62300c86f0 2023-08-09T13:50:34.009 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:34.004+0000 7f6234c57640 1 -- --> v1:172.21.15.203:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f623006c440 con 0x7f6230143c20 2023-08-09T13:50:34.009 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:34.004+0000 7f6234c57640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6230144000 con 0x7f623006cfe0 2023-08-09T13:50:34.009 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:34.004+0000 7f6227fff640 1 --1- >> v1:172.21.15.43:6789/0 conn(0x7f62300c86f0 0x7f623006c8d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.43:6789/0 says I am v1:172.21.15.43:32988/0 (socket says 172.21.15.43:32988) 2023-08-09T13:50:34.010 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:34.004+0000 7f6227fff640 1 -- 172.21.15.43:0/3400482686 learned_addr learned my addr 172.21.15.43:0/3400482686 (peer_addr_for_me v1:172.21.15.43:0/0) 2023-08-09T13:50:34.010 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:34.005+0000 7f622dd9b640 1 -- 172.21.15.43:0/3400482686 <== mon.0 v1:172.21.15.43:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1894285063 0 0) 0x7f62300c9090 con 0x7f62300c86f0 2023-08-09T13:50:34.010 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:34.005+0000 7f622dd9b640 1 -- 172.21.15.43:0/3400482686 --> v1:172.21.15.43:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f621c003520 con 0x7f62300c86f0 2023-08-09T13:50:34.011 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:34.005+0000 7f622ed9d640 1 --2- 172.21.15.43:0/3400482686 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f623006cfe0 0x7f62300c8b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:50:34.011 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:34.005+0000 7f622dd9b640 1 -- 172.21.15.43:0/3400482686 <== mon.0 v1:172.21.15.43:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2043123994 0 0) 0x7f621c003520 con 0x7f62300c86f0 2023-08-09T13:50:34.011 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:34.005+0000 7f622dd9b640 1 -- 172.21.15.43:0/3400482686 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f623006cfe0 msgr2=0x7f62300c8b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-09T13:50:34.012 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:34.005+0000 7f622dd9b640 1 --2- 172.21.15.43:0/3400482686 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f623006cfe0 0x7f62300c8b50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:50:34.012 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:34.005+0000 7f622dd9b640 1 -- 172.21.15.43:0/3400482686 >> v1:172.21.15.203:6789/0 conn(0x7f6230143c20 legacy=0x7f623006f540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:50:34.012 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:34.005+0000 7f622dd9b640 1 -- 172.21.15.43:0/3400482686 --> v1:172.21.15.43:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f623006fc50 con 0x7f62300c86f0 2023-08-09T13:50:34.012 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:34.005+0000 7f622dd9b640 1 -- 172.21.15.43:0/3400482686 <== mon.0 v1:172.21.15.43:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1285150740 0 0) 0x7f6218001670 con 0x7f62300c86f0 2023-08-09T13:50:34.013 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:34.005+0000 7f622dd9b640 1 -- 172.21.15.43:0/3400482686 >> v1:172.21.15.43:6789/0 conn(0x7f62300c86f0 legacy=0x7f623006c8d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:50:34.013 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:34.005+0000 7f622dd9b640 1 --2- 172.21.15.43:0/3400482686 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f621c003a10 0x7f621c003e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:50:34.014 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:34.005+0000 7f622dd9b640 1 --2- 172.21.15.43:0/3400482686 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f623006cfe0 0x7f621c0047c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:50:34.014 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:34.005+0000 7f622dd9b640 1 --2- 172.21.15.43:0/3400482686 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f621c004d70 0x7f621c0051f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:50:34.014 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:34.006+0000 7f622dd9b640 1 -- 172.21.15.43:0/3400482686 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6230144000 con 0x7f621c003a10 2023-08-09T13:50:34.015 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:34.006+0000 7f622dd9b640 1 -- 172.21.15.43:0/3400482686 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f621c00bc20 con 0x7f621c004d70 2023-08-09T13:50:34.015 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:34.006+0000 7f622dd9b640 1 -- 172.21.15.43:0/3400482686 --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f621c00bde0 con 0x7f623006cfe0 2023-08-09T13:50:34.015 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:34.006+0000 7f622e59c640 1 --2- 172.21.15.43:0/3400482686 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f621c003a10 0x7f621c003e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:50:34.015 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:34.006+0000 7f6227fff640 1 --2- 172.21.15.43:0/3400482686 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f623006cfe0 0x7f621c0047c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:50:34.016 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:34.006+0000 7f622ed9d640 1 --2- 172.21.15.43:0/3400482686 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f621c004d70 0x7f621c0051f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:50:34.016 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:34.006+0000 7f622e59c640 1 -- 172.21.15.43:0/3400482686 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f623006cfe0 msgr2=0x7f621c0047c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-09T13:50:34.016 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:34.006+0000 7f622e59c640 1 --2- 172.21.15.43:0/3400482686 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f623006cfe0 0x7f621c0047c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:50:34.017 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:34.006+0000 7f622e59c640 1 -- 172.21.15.43:0/3400482686 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f621c004d70 msgr2=0x7f621c0051f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-09T13:50:34.017 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:34.006+0000 7f622e59c640 1 --2- 172.21.15.43:0/3400482686 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f621c004d70 0x7f621c0051f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:50:34.017 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:34.006+0000 7f622e59c640 1 -- 172.21.15.43:0/3400482686 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f621c003520 con 0x7f621c003a10 2023-08-09T13:50:34.022 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:34.007+0000 7f622e59c640 1 --2- 172.21.15.43:0/3400482686 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f621c003a10 0x7f621c003e30 secure :-1 s=READY pgs=1021 cs=0 l=1 rev1=1 crypto rx=0x7f6220001750 tx=0x7f621c0057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=690cdac0f1a9b2ca server_cookie=0 in_seq=0 out_seq=0 2023-08-09T13:50:34.023 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:34.008+0000 7f622dd9b640 1 -- 172.21.15.43:0/3400482686 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f62200052d0 con 0x7f621c003a10 2023-08-09T13:50:34.023 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:34.008+0000 7f622dd9b640 1 -- 172.21.15.43:0/3400482686 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f622002e050 con 0x7f621c003a10 2023-08-09T13:50:34.023 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:34.008+0000 7f622dd9b640 1 -- 172.21.15.43:0/3400482686 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f62200035f0 con 0x7f621c003a10 2023-08-09T13:50:34.024 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:34.008+0000 7f6234c57640 1 -- 172.21.15.43:0/3400482686 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f621c003a10 msgr2=0x7f621c003e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:50:34.024 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:34.008+0000 7f6234c57640 1 --2- 172.21.15.43:0/3400482686 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f621c003a10 0x7f621c003e30 secure :-1 s=READY pgs=1021 cs=0 l=1 rev1=1 crypto rx=0x7f6220001750 tx=0x7f621c0057f0 comp rx=0 tx=0).stop 2023-08-09T13:50:34.024 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:34.008+0000 7f622ed9d640 1 -- 172.21.15.43:0/3400482686 reap_dead start 2023-08-09T13:50:34.024 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:34.008+0000 7f6234c57640 1 -- 172.21.15.43:0/3400482686 shutdown_connections 2023-08-09T13:50:34.024 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:34.008+0000 7f6234c57640 1 -- 172.21.15.43:0/3400482686 >> 172.21.15.43:0/3400482686 conn(0x7f62300ce0a0 msgr2=0x7f62300ce480 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-09T13:50:34.025 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:34.009+0000 7f6234c57640 1 -- 172.21.15.43:0/3400482686 shutdown_connections 2023-08-09T13:50:34.025 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:34.009+0000 7f6234c57640 1 -- 172.21.15.43:0/3400482686 wait complete. 2023-08-09T13:50:34.025 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:34.010+0000 7f6234c57640 1 Processor -- start 2023-08-09T13:50:34.025 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:34.010+0000 7f6234c57640 1 -- start start 2023-08-09T13:50:34.025 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:34.011+0000 7f6234c57640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f621c003a10 0x7f623015a010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:50:34.026 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:34.011+0000 7f6234c57640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f623013e630 0x7f623015c830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:50:34.026 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:34.011+0000 7f6234c57640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f623015cd70 0x7f623013eec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:50:34.026 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:34.011+0000 7f6234c57640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f623015d1f0 con 0x7f623013e630 2023-08-09T13:50:34.026 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:34.011+0000 7f6234c57640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f623013eab0 con 0x7f621c003a10 2023-08-09T13:50:34.026 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:34.011+0000 7f6234c57640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f623013e1a0 con 0x7f623015cd70 2023-08-09T13:50:34.026 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:34.012+0000 7f6227fff640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f623013e630 0x7f623015c830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:50:34.027 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:34.012+0000 7f6227fff640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f623013e630 0x7f623015c830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:35808/0 (socket says 172.21.15.43:35808) 2023-08-09T13:50:34.027 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:34.012+0000 7f6227fff640 1 -- 172.21.15.43:0/626054742 learned_addr learned my addr 172.21.15.43:0/626054742 (peer_addr_for_me v2:172.21.15.43:0/0) 2023-08-09T13:50:34.027 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:34.012+0000 7f622e59c640 1 --2- 172.21.15.43:0/626054742 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f621c003a10 0x7f623015a010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:50:34.027 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:34.012+0000 7f622ed9d640 1 --2- 172.21.15.43:0/626054742 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f623015cd70 0x7f623013eec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:50:34.027 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:34.012+0000 7f6227fff640 1 -- 172.21.15.43:0/626054742 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f623015cd70 msgr2=0x7f623013eec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:50:34.027 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:34.012+0000 7f6227fff640 1 --2- 172.21.15.43:0/626054742 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f623015cd70 0x7f623013eec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:50:34.028 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:34.012+0000 7f6227fff640 1 -- 172.21.15.43:0/626054742 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f621c003a10 msgr2=0x7f623015a010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:50:34.028 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:34.012+0000 7f6227fff640 1 --2- 172.21.15.43:0/626054742 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f621c003a10 0x7f623015a010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:50:34.028 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:34.012+0000 7f6227fff640 1 -- 172.21.15.43:0/626054742 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f623013d590 con 0x7f623013e630 2023-08-09T13:50:34.028 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:34.012+0000 7f622e59c640 1 --2- 172.21.15.43:0/626054742 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f621c003a10 0x7f623015a010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-08-09T13:50:34.028 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:34.012+0000 7f6227fff640 1 --2- 172.21.15.43:0/626054742 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f623013e630 0x7f623015c830 secure :-1 s=READY pgs=1022 cs=0 l=1 rev1=1 crypto rx=0x7f621800e730 tx=0x7f621800ebf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-09T13:50:34.028 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:34.013+0000 7f6226ffd640 1 -- 172.21.15.43:0/626054742 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f621800c6c0 con 0x7f623013e630 2023-08-09T13:50:34.028 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:34.013+0000 7f6226ffd640 1 -- 172.21.15.43:0/626054742 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f621800c820 con 0x7f623013e630 2023-08-09T13:50:34.029 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:34.013+0000 7f6226ffd640 1 -- 172.21.15.43:0/626054742 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f621800cbb0 con 0x7f623013e630 2023-08-09T13:50:34.029 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:34.013+0000 7f6234c57640 1 -- 172.21.15.43:0/626054742 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f623013d870 con 0x7f623013e630 2023-08-09T13:50:34.029 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:34.013+0000 7f6234c57640 1 -- 172.21.15.43:0/626054742 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f623013de30 con 0x7f623013e630 2023-08-09T13:50:34.029 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:34.014+0000 7f6234c57640 1 -- 172.21.15.43:0/626054742 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f623013dff0 con 0x7f623013e630 2023-08-09T13:50:34.029 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:34.015+0000 7f6226ffd640 1 -- 172.21.15.43:0/626054742 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 29) v1 ==== 88443+0+0 (secure 0 0 0) 0x7f621801d460 con 0x7f623013e630 2023-08-09T13:50:34.029 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:34.016+0000 7f6226ffd640 1 --2- 172.21.15.43:0/626054742 >> [v2:172.21.15.203:6832/3325141227,v1:172.21.15.203:6833/3325141227] conn(0x7f621006a170 0x7f621006c630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:50:34.029 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:34.016+0000 7f6226ffd640 1 -- 172.21.15.43:0/626054742 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(90..90 src has 1..90) v4 ==== 6426+0+0 (secure 0 0 0) 0x7f6218088a60 con 0x7f623013e630 2023-08-09T13:50:34.029 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:34.017+0000 7f622e59c640 1 --2- 172.21.15.43:0/626054742 >> [v2:172.21.15.203:6832/3325141227,v1:172.21.15.203:6833/3325141227] conn(0x7f621006a170 0x7f621006c630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:50:34.030 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:34.018+0000 7f622e59c640 1 --2- 172.21.15.43:0/626054742 >> [v2:172.21.15.203:6832/3325141227,v1:172.21.15.203:6833/3325141227] conn(0x7f621006a170 0x7f621006c630 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f6220033000 tx=0x7f6220009030 comp rx=0 tx=0).ready entity=mgr.5519 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-09T13:50:34.030 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:34.021+0000 7f6226ffd640 1 -- 172.21.15.43:0/626054742 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178066 (secure 0 0 0) 0x7f6218053e30 con 0x7f623013e630 2023-08-09T13:50:34.124 INFO:tasks.ceph.mds.d.smithi203.stderr:2023-08-09T13:50:34.123+0000 7f4eed864640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2023-08-09T13:50:34.211 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:34.210+0000 7f6234c57640 1 -- 172.21.15.43:0/626054742 --> [v2:172.21.15.203:6832/3325141227,v1:172.21.15.203:6833/3325141227] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f623006bb10 con 0x7f621006a170 2023-08-09T13:50:34.236 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:34.235+0000 7f6226ffd640 1 -- 172.21.15.43:0/626054742 <== mgr.5519 v2:172.21.15.203:6832/3325141227 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+307350 (secure 0 0 0) 0x7f62200051f0 con 0x7f621006a170 2023-08-09T13:50:34.238 INFO:teuthology.orchestra.run.smithi043.stdout: 2023-08-09T13:50:34.241 INFO:teuthology.orchestra.run.smithi043.stderr:dumped all 2023-08-09T13:50:34.243 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:34.241+0000 7f6234c57640 1 -- 172.21.15.43:0/626054742 >> [v2:172.21.15.203:6832/3325141227,v1:172.21.15.203:6833/3325141227] conn(0x7f621006a170 msgr2=0x7f621006c630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:50:34.243 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:34.241+0000 7f6234c57640 1 --2- 172.21.15.43:0/626054742 >> [v2:172.21.15.203:6832/3325141227,v1:172.21.15.203:6833/3325141227] conn(0x7f621006a170 0x7f621006c630 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f6220033000 tx=0x7f6220009030 comp rx=0 tx=0).stop 2023-08-09T13:50:34.243 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:34.241+0000 7f6234c57640 1 -- 172.21.15.43:0/626054742 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f623013e630 msgr2=0x7f623015c830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:50:34.243 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:34.241+0000 7f6234c57640 1 --2- 172.21.15.43:0/626054742 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f623013e630 0x7f623015c830 secure :-1 s=READY pgs=1022 cs=0 l=1 rev1=1 crypto rx=0x7f621800e730 tx=0x7f621800ebf0 comp rx=0 tx=0).stop 2023-08-09T13:50:34.243 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:34.241+0000 7f6234c57640 1 -- 172.21.15.43:0/626054742 shutdown_connections 2023-08-09T13:50:34.244 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:34.241+0000 7f6234c57640 1 --2- 172.21.15.43:0/626054742 >> [v2:172.21.15.203:6832/3325141227,v1:172.21.15.203:6833/3325141227] conn(0x7f621006a170 0x7f621006c630 unknown :-1 s=CLOSED pgs=66 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:50:34.244 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:34.241+0000 7f6234c57640 1 --2- 172.21.15.43:0/626054742 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f623015cd70 0x7f623013eec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:50:34.244 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:34.241+0000 7f6234c57640 1 --2- 172.21.15.43:0/626054742 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f623013e630 0x7f623015c830 unknown :-1 s=CLOSED pgs=1022 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:50:34.244 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:34.241+0000 7f6234c57640 1 --2- 172.21.15.43:0/626054742 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f621c003a10 0x7f623015a010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:50:34.244 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:34.241+0000 7f6234c57640 1 -- 172.21.15.43:0/626054742 >> 172.21.15.43:0/626054742 conn(0x7f62300ce0a0 msgr2=0x7f62300af360 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-09T13:50:34.244 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:34.241+0000 7f6234c57640 1 -- 172.21.15.43:0/626054742 shutdown_connections 2023-08-09T13:50:34.244 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:34.242+0000 7f6234c57640 1 -- 172.21.15.43:0/626054742 wait complete. 2023-08-09T13:50:34.262 INFO:teuthology.orchestra.run.smithi043.stdout:{"pg_ready":true,"pg_map":{"version":28,"stamp":"2023-08-09T13:50:32.231945+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":500,"num_read_kb":924,"num_write":239,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":161,"ondisk_log_size":161,"up":210,"acting":210,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":61,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":278924,"kb_used_data":9504,"kb_used_omap":13,"kb_used_meta":269298,"kb_avail":754695796,"statfs":{"total":773094113280,"available":772808495104,"internally_reserved":0,"allocated":9732096,"data_stored":5732158,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":14248,"internal_metadata":275761240},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":32,"apply_latency_ms":32,"commit_latency_ns":32000000,"apply_latency_ns":32000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"5.076138"},"pg_stats":[{"pgid":"11.3f","version":"0'0","reported_seq":8,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-08-09T13:50:29.233104+0000","last_change":"2023-08-09T13:50:29.233104+0000","last_active":"2023-08-09T13:50:29.233104+0000","last_peered":"2023-08-09T13:50:29.233104+0000","last_clean":"2023-08-09T13:50:29.233104+0000","last_became_active":"2023-08-09T13:50:29.232795+0000","last_became_peered":"2023-08-09T13:50:29.232795+0000","last_unstale":"2023-08-09T13:50:29.233104+0000","last_undegraded":"2023-08-09T13:50:29.233104+0000","last_fullsized":"2023-08-09T13:50:29.233104+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:50:28.150437+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:50:28.150437+0000","last_clean_scrub_stamp":"2023-08-09T13:50:28.150437+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T17:05:45.752848+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"11.3e","version":"0'0","reported_seq":3,"reported_epoch":89,"state":"creating+activating","last_fresh":"2023-08-09T13:50:29.231349+0000","last_change":"2023-08-09T13:50:29.231349+0000","last_active":"2023-08-09T13:50:28.150437+0000","last_peered":"2023-08-09T13:50:28.150437+0000","last_clean":"2023-08-09T13:50:28.150437+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-09T13:50:29.231349+0000","last_undegraded":"2023-08-09T13:50:29.231349+0000","last_fullsized":"2023-08-09T13:50:29.231349+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:50:28.150437+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:50:28.150437+0000","last_clean_scrub_stamp":"2023-08-09T13:50:28.150437+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T18:13:21.364990+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":["6","4"],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.3d","version":"0'0","reported_seq":9,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-08-09T13:50:29.233399+0000","last_change":"2023-08-09T13:50:29.225958+0000","last_active":"2023-08-09T13:50:29.233399+0000","last_peered":"2023-08-09T13:50:29.233399+0000","last_clean":"2023-08-09T13:50:29.233399+0000","last_became_active":"2023-08-09T13:50:29.225536+0000","last_became_peered":"2023-08-09T13:50:29.225536+0000","last_unstale":"2023-08-09T13:50:29.233399+0000","last_undegraded":"2023-08-09T13:50:29.233399+0000","last_fullsized":"2023-08-09T13:50:29.233399+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:50:28.150437+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:50:28.150437+0000","last_clean_scrub_stamp":"2023-08-09T13:50:28.150437+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T22:25:44.631240+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.3c","version":"0'0","reported_seq":1,"reported_epoch":88,"state":"creating+peering","last_fresh":"2023-08-09T13:50:28.188648+0000","last_change":"2023-08-09T13:50:28.188648+0000","last_active":"2023-08-09T13:50:28.150437+0000","last_peered":"2023-08-09T13:50:28.150437+0000","last_clean":"2023-08-09T13:50:28.150437+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-09T13:50:28.188648+0000","last_undegraded":"2023-08-09T13:50:28.188648+0000","last_fullsized":"2023-08-09T13:50:28.188648+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:50:28.150437+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:50:28.150437+0000","last_clean_scrub_stamp":"2023-08-09T13:50:28.150437+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.3b","version":"0'0","reported_seq":3,"reported_epoch":89,"state":"creating+activating","last_fresh":"2023-08-09T13:50:29.221854+0000","last_change":"2023-08-09T13:50:29.221854+0000","last_active":"2023-08-09T13:50:28.150437+0000","last_peered":"2023-08-09T13:50:28.150437+0000","last_clean":"2023-08-09T13:50:28.150437+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-09T13:50:29.221854+0000","last_undegraded":"2023-08-09T13:50:29.221854+0000","last_fullsized":"2023-08-09T13:50:29.221854+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:50:28.150437+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:50:28.150437+0000","last_clean_scrub_stamp":"2023-08-09T13:50:28.150437+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T22:54:27.616846+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":["2","7"],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.3a","version":"0'0","reported_seq":8,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-08-09T13:50:29.250572+0000","last_change":"2023-08-09T13:50:29.250572+0000","last_active":"2023-08-09T13:50:29.250572+0000","last_peered":"2023-08-09T13:50:29.250572+0000","last_clean":"2023-08-09T13:50:29.250572+0000","last_became_active":"2023-08-09T13:50:29.250267+0000","last_became_peered":"2023-08-09T13:50:29.250267+0000","last_unstale":"2023-08-09T13:50:29.250572+0000","last_undegraded":"2023-08-09T13:50:29.250572+0000","last_fullsized":"2023-08-09T13:50:29.250572+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:50:28.150437+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:50:28.150437+0000","last_clean_scrub_stamp":"2023-08-09T13:50:28.150437+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T17:21:38.689120+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.39","version":"0'0","reported_seq":3,"reported_epoch":89,"state":"creating+activating","last_fresh":"2023-08-09T13:50:29.220028+0000","last_change":"2023-08-09T13:50:29.220028+0000","last_active":"2023-08-09T13:50:28.150437+0000","last_peered":"2023-08-09T13:50:28.150437+0000","last_clean":"2023-08-09T13:50:28.150437+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-09T13:50:29.220028+0000","last_undegraded":"2023-08-09T13:50:29.220028+0000","last_fullsized":"2023-08-09T13:50:29.220028+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:50:28.150437+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:50:28.150437+0000","last_clean_scrub_stamp":"2023-08-09T13:50:28.150437+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T23:19:57.122069+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":["5","1"],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.38","version":"0'0","reported_seq":8,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-08-09T13:50:29.246308+0000","last_change":"2023-08-09T13:50:29.246308+0000","last_active":"2023-08-09T13:50:29.246308+0000","last_peered":"2023-08-09T13:50:29.246308+0000","last_clean":"2023-08-09T13:50:29.246308+0000","last_became_active":"2023-08-09T13:50:29.245268+0000","last_became_peered":"2023-08-09T13:50:29.245268+0000","last_unstale":"2023-08-09T13:50:29.246308+0000","last_undegraded":"2023-08-09T13:50:29.246308+0000","last_fullsized":"2023-08-09T13:50:29.246308+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:50:28.150437+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:50:28.150437+0000","last_clean_scrub_stamp":"2023-08-09T13:50:28.150437+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T18:15:31.389069+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.37","version":"0'0","reported_seq":3,"reported_epoch":89,"state":"creating+activating","last_fresh":"2023-08-09T13:50:29.223778+0000","last_change":"2023-08-09T13:50:29.223778+0000","last_active":"2023-08-09T13:50:28.150437+0000","last_peered":"2023-08-09T13:50:28.150437+0000","last_clean":"2023-08-09T13:50:28.150437+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-09T13:50:29.223778+0000","last_undegraded":"2023-08-09T13:50:29.223778+0000","last_fullsized":"2023-08-09T13:50:29.223778+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:50:28.150437+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:50:28.150437+0000","last_clean_scrub_stamp":"2023-08-09T13:50:28.150437+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T15:38:50.620059+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":["2","3"],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.36","version":"0'0","reported_seq":3,"reported_epoch":89,"state":"creating+activating","last_fresh":"2023-08-09T13:50:29.207900+0000","last_change":"2023-08-09T13:50:29.207900+0000","last_active":"2023-08-09T13:50:28.150437+0000","last_peered":"2023-08-09T13:50:28.150437+0000","last_clean":"2023-08-09T13:50:28.150437+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-09T13:50:29.207900+0000","last_undegraded":"2023-08-09T13:50:29.207900+0000","last_fullsized":"2023-08-09T13:50:29.207900+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:50:28.150437+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:50:28.150437+0000","last_clean_scrub_stamp":"2023-08-09T13:50:28.150437+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T21:32:28.121459+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":["4","2"],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.35","version":"0'0","reported_seq":8,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-08-09T13:50:29.237075+0000","last_change":"2023-08-09T13:50:29.237075+0000","last_active":"2023-08-09T13:50:29.237075+0000","last_peered":"2023-08-09T13:50:29.237075+0000","last_clean":"2023-08-09T13:50:29.237075+0000","last_became_active":"2023-08-09T13:50:29.236351+0000","last_became_peered":"2023-08-09T13:50:29.236351+0000","last_unstale":"2023-08-09T13:50:29.237075+0000","last_undegraded":"2023-08-09T13:50:29.237075+0000","last_fullsized":"2023-08-09T13:50:29.237075+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:50:28.150437+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:50:28.150437+0000","last_clean_scrub_stamp":"2023-08-09T13:50:28.150437+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T18:17:12.675765+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"11.34","version":"0'0","reported_seq":1,"reported_epoch":88,"state":"creating+peering","last_fresh":"2023-08-09T13:50:28.188207+0000","last_change":"2023-08-09T13:50:28.188207+0000","last_active":"2023-08-09T13:50:28.150437+0000","last_peered":"2023-08-09T13:50:28.150437+0000","last_clean":"2023-08-09T13:50:28.150437+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-09T13:50:28.188207+0000","last_undegraded":"2023-08-09T13:50:28.188207+0000","last_fullsized":"2023-08-09T13:50:28.188207+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:50:28.150437+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:50:28.150437+0000","last_clean_scrub_stamp":"2023-08-09T13:50:28.150437+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.33","version":"0'0","reported_seq":3,"reported_epoch":89,"state":"creating+activating","last_fresh":"2023-08-09T13:50:29.207581+0000","last_change":"2023-08-09T13:50:29.207581+0000","last_active":"2023-08-09T13:50:28.150437+0000","last_peered":"2023-08-09T13:50:28.150437+0000","last_clean":"2023-08-09T13:50:28.150437+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-09T13:50:29.207581+0000","last_undegraded":"2023-08-09T13:50:29.207581+0000","last_fullsized":"2023-08-09T13:50:29.207581+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:50:28.150437+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:50:28.150437+0000","last_clean_scrub_stamp":"2023-08-09T13:50:28.150437+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T18:17:12.675765+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":["0","6"],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.32","version":"0'0","reported_seq":3,"reported_epoch":89,"state":"creating+activating","last_fresh":"2023-08-09T13:50:29.224087+0000","last_change":"2023-08-09T13:50:29.224087+0000","last_active":"2023-08-09T13:50:28.150437+0000","last_peered":"2023-08-09T13:50:28.150437+0000","last_clean":"2023-08-09T13:50:28.150437+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-09T13:50:29.224087+0000","last_undegraded":"2023-08-09T13:50:29.224087+0000","last_fullsized":"2023-08-09T13:50:29.224087+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:50:28.150437+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:50:28.150437+0000","last_clean_scrub_stamp":"2023-08-09T13:50:28.150437+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T23:57:06.314706+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":["2","5"],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.31","version":"0'0","reported_seq":3,"reported_epoch":89,"state":"creating+activating","last_fresh":"2023-08-09T13:50:29.206432+0000","last_change":"2023-08-09T13:50:29.206432+0000","last_active":"2023-08-09T13:50:28.150437+0000","last_peered":"2023-08-09T13:50:28.150437+0000","last_clean":"2023-08-09T13:50:28.150437+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-09T13:50:29.206432+0000","last_undegraded":"2023-08-09T13:50:29.206432+0000","last_fullsized":"2023-08-09T13:50:29.206432+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:50:28.150437+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:50:28.150437+0000","last_clean_scrub_stamp":"2023-08-09T13:50:28.150437+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T22:39:17.713654+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":["0","2"],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.30","version":"0'0","reported_seq":3,"reported_epoch":89,"state":"creating+activating","last_fresh":"2023-08-09T13:50:29.208580+0000","last_change":"2023-08-09T13:50:29.208580+0000","last_active":"2023-08-09T13:50:28.150437+0000","last_peered":"2023-08-09T13:50:28.150437+0000","last_clean":"2023-08-09T13:50:28.150437+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-09T13:50:29.208580+0000","last_undegraded":"2023-08-09T13:50:29.208580+0000","last_fullsized":"2023-08-09T13:50:29.208580+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:50:28.150437+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:50:28.150437+0000","last_clean_scrub_stamp":"2023-08-09T13:50:28.150437+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T15:14:37.215348+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":["4","6"],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.2f","version":"0'0","reported_seq":3,"reported_epoch":89,"state":"creating+activating","last_fresh":"2023-08-09T13:50:29.223293+0000","last_change":"2023-08-09T13:50:29.223293+0000","last_active":"2023-08-09T13:50:28.150437+0000","last_peered":"2023-08-09T13:50:28.150437+0000","last_clean":"2023-08-09T13:50:28.150437+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-09T13:50:29.223293+0000","last_undegraded":"2023-08-09T13:50:29.223293+0000","last_fullsized":"2023-08-09T13:50:29.223293+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:50:28.150437+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:50:28.150437+0000","last_clean_scrub_stamp":"2023-08-09T13:50:28.150437+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T22:44:32.845197+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":["2","3"],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.2e","version":"0'0","reported_seq":1,"reported_epoch":88,"state":"creating+peering","last_fresh":"2023-08-09T13:50:28.187070+0000","last_change":"2023-08-09T13:50:28.187070+0000","last_active":"2023-08-09T13:50:28.150437+0000","last_peered":"2023-08-09T13:50:28.150437+0000","last_clean":"2023-08-09T13:50:28.150437+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-09T13:50:28.187070+0000","last_undegraded":"2023-08-09T13:50:28.187070+0000","last_fullsized":"2023-08-09T13:50:28.187070+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:50:28.150437+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:50:28.150437+0000","last_clean_scrub_stamp":"2023-08-09T13:50:28.150437+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.2d","version":"0'0","reported_seq":9,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-08-09T13:50:29.248797+0000","last_change":"2023-08-09T13:50:29.241875+0000","last_active":"2023-08-09T13:50:29.248797+0000","last_peered":"2023-08-09T13:50:29.248797+0000","last_clean":"2023-08-09T13:50:29.248797+0000","last_became_active":"2023-08-09T13:50:29.241532+0000","last_became_peered":"2023-08-09T13:50:29.241532+0000","last_unstale":"2023-08-09T13:50:29.248797+0000","last_undegraded":"2023-08-09T13:50:29.248797+0000","last_fullsized":"2023-08-09T13:50:29.248797+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:50:28.150437+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:50:28.150437+0000","last_clean_scrub_stamp":"2023-08-09T13:50:28.150437+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-11T00:33:32.657666+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.2c","version":"0'0","reported_seq":3,"reported_epoch":89,"state":"creating+activating","last_fresh":"2023-08-09T13:50:29.205840+0000","last_change":"2023-08-09T13:50:29.205840+0000","last_active":"2023-08-09T13:50:28.150437+0000","last_peered":"2023-08-09T13:50:28.150437+0000","last_clean":"2023-08-09T13:50:28.150437+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-09T13:50:29.205840+0000","last_undegraded":"2023-08-09T13:50:29.205840+0000","last_fullsized":"2023-08-09T13:50:29.205840+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:50:28.150437+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:50:28.150437+0000","last_clean_scrub_stamp":"2023-08-09T13:50:28.150437+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T16:53:03.610834+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":["0","5"],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.2b","version":"0'0","reported_seq":9,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-08-09T13:50:29.233438+0000","last_change":"2023-08-09T13:50:29.227706+0000","last_active":"2023-08-09T13:50:29.233438+0000","last_peered":"2023-08-09T13:50:29.233438+0000","last_clean":"2023-08-09T13:50:29.233438+0000","last_became_active":"2023-08-09T13:50:29.226933+0000","last_became_peered":"2023-08-09T13:50:29.226933+0000","last_unstale":"2023-08-09T13:50:29.233438+0000","last_undegraded":"2023-08-09T13:50:29.233438+0000","last_fullsized":"2023-08-09T13:50:29.233438+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:50:28.150437+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:50:28.150437+0000","last_clean_scrub_stamp":"2023-08-09T13:50:28.150437+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T17:57:56.069305+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.2a","version":"0'0","reported_seq":3,"reported_epoch":89,"state":"creating+activating","last_fresh":"2023-08-09T13:50:29.207312+0000","last_change":"2023-08-09T13:50:29.207312+0000","last_active":"2023-08-09T13:50:28.150437+0000","last_peered":"2023-08-09T13:50:28.150437+0000","last_clean":"2023-08-09T13:50:28.150437+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-09T13:50:29.207312+0000","last_undegraded":"2023-08-09T13:50:29.207312+0000","last_fullsized":"2023-08-09T13:50:29.207312+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:50:28.150437+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:50:28.150437+0000","last_clean_scrub_stamp":"2023-08-09T13:50:28.150437+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T19:03:45.295280+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":["4","2"],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.29","version":"0'0","reported_seq":8,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-08-09T13:50:29.232548+0000","last_change":"2023-08-09T13:50:29.232548+0000","last_active":"2023-08-09T13:50:29.232548+0000","last_peered":"2023-08-09T13:50:29.232548+0000","last_clean":"2023-08-09T13:50:29.232548+0000","last_became_active":"2023-08-09T13:50:29.232205+0000","last_became_peered":"2023-08-09T13:50:29.232205+0000","last_unstale":"2023-08-09T13:50:29.232548+0000","last_undegraded":"2023-08-09T13:50:29.232548+0000","last_fullsized":"2023-08-09T13:50:29.232548+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:50:28.150437+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:50:28.150437+0000","last_clean_scrub_stamp":"2023-08-09T13:50:28.150437+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T21:11:03.639832+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"11.28","version":"0'0","reported_seq":3,"reported_epoch":89,"state":"creating+activating","last_fresh":"2023-08-09T13:50:29.217553+0000","last_change":"2023-08-09T13:50:29.217553+0000","last_active":"2023-08-09T13:50:28.150437+0000","last_peered":"2023-08-09T13:50:28.150437+0000","last_clean":"2023-08-09T13:50:28.150437+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-09T13:50:29.217553+0000","last_undegraded":"2023-08-09T13:50:29.217553+0000","last_fullsized":"2023-08-09T13:50:29.217553+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:50:28.150437+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:50:28.150437+0000","last_clean_scrub_stamp":"2023-08-09T13:50:28.150437+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T22:39:17.713654+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":["7","6"],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"11.27","version":"0'0","reported_seq":3,"reported_epoch":89,"state":"creating+activating","last_fresh":"2023-08-09T13:50:29.208200+0000","last_change":"2023-08-09T13:50:29.208200+0000","last_active":"2023-08-09T13:50:28.150437+0000","last_peered":"2023-08-09T13:50:28.150437+0000","last_clean":"2023-08-09T13:50:28.150437+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-09T13:50:29.208200+0000","last_undegraded":"2023-08-09T13:50:29.208200+0000","last_fullsized":"2023-08-09T13:50:29.208200+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:50:28.150437+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:50:28.150437+0000","last_clean_scrub_stamp":"2023-08-09T13:50:28.150437+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T14:52:59.954591+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":["4","7"],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.26","version":"0'0","reported_seq":3,"reported_epoch":89,"state":"creating+activating","last_fresh":"2023-08-09T13:50:29.223117+0000","last_change":"2023-08-09T13:50:29.223117+0000","last_active":"2023-08-09T13:50:28.150437+0000","last_peered":"2023-08-09T13:50:28.150437+0000","last_clean":"2023-08-09T13:50:28.150437+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-09T13:50:29.223117+0000","last_undegraded":"2023-08-09T13:50:29.223117+0000","last_fullsized":"2023-08-09T13:50:29.223117+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:50:28.150437+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:50:28.150437+0000","last_clean_scrub_stamp":"2023-08-09T13:50:28.150437+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T21:18:31.512056+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":["5","6"],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.25","version":"0'0","reported_seq":8,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-08-09T13:50:29.233554+0000","last_change":"2023-08-09T13:50:29.233554+0000","last_active":"2023-08-09T13:50:29.233554+0000","last_peered":"2023-08-09T13:50:29.233554+0000","last_clean":"2023-08-09T13:50:29.233554+0000","last_became_active":"2023-08-09T13:50:29.233153+0000","last_became_peered":"2023-08-09T13:50:29.233153+0000","last_unstale":"2023-08-09T13:50:29.233554+0000","last_undegraded":"2023-08-09T13:50:29.233554+0000","last_fullsized":"2023-08-09T13:50:29.233554+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:50:28.150437+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:50:28.150437+0000","last_clean_scrub_stamp":"2023-08-09T13:50:28.150437+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T21:36:11.302948+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.24","version":"0'0","reported_seq":3,"reported_epoch":89,"state":"creating+activating","last_fresh":"2023-08-09T13:50:29.207497+0000","last_change":"2023-08-09T13:50:29.207497+0000","last_active":"2023-08-09T13:50:28.150437+0000","last_peered":"2023-08-09T13:50:28.150437+0000","last_clean":"2023-08-09T13:50:28.150437+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-09T13:50:29.207497+0000","last_undegraded":"2023-08-09T13:50:29.207497+0000","last_fullsized":"2023-08-09T13:50:29.207497+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:50:28.150437+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:50:28.150437+0000","last_clean_scrub_stamp":"2023-08-09T13:50:28.150437+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T18:47:18.558930+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":["4","2"],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.23","version":"0'0","reported_seq":8,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-08-09T13:50:29.233513+0000","last_change":"2023-08-09T13:50:29.233513+0000","last_active":"2023-08-09T13:50:29.233513+0000","last_peered":"2023-08-09T13:50:29.233513+0000","last_clean":"2023-08-09T13:50:29.233513+0000","last_became_active":"2023-08-09T13:50:29.233199+0000","last_became_peered":"2023-08-09T13:50:29.233199+0000","last_unstale":"2023-08-09T13:50:29.233513+0000","last_undegraded":"2023-08-09T13:50:29.233513+0000","last_fullsized":"2023-08-09T13:50:29.233513+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:50:28.150437+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:50:28.150437+0000","last_clean_scrub_stamp":"2023-08-09T13:50:28.150437+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T21:47:07.987840+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.22","version":"0'0","reported_seq":3,"reported_epoch":89,"state":"creating+activating","last_fresh":"2023-08-09T13:50:29.219724+0000","last_change":"2023-08-09T13:50:29.219724+0000","last_active":"2023-08-09T13:50:28.150437+0000","last_peered":"2023-08-09T13:50:28.150437+0000","last_clean":"2023-08-09T13:50:28.150437+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-09T13:50:29.219724+0000","last_undegraded":"2023-08-09T13:50:29.219724+0000","last_fullsized":"2023-08-09T13:50:29.219724+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:50:28.150437+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:50:28.150437+0000","last_clean_scrub_stamp":"2023-08-09T13:50:28.150437+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T21:52:25.852199+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":["5","6"],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.21","version":"0'0","reported_seq":8,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-08-09T13:50:29.233755+0000","last_change":"2023-08-09T13:50:29.233755+0000","last_active":"2023-08-09T13:50:29.233755+0000","last_peered":"2023-08-09T13:50:29.233755+0000","last_clean":"2023-08-09T13:50:29.233755+0000","last_became_active":"2023-08-09T13:50:29.233150+0000","last_became_peered":"2023-08-09T13:50:29.233150+0000","last_unstale":"2023-08-09T13:50:29.233755+0000","last_undegraded":"2023-08-09T13:50:29.233755+0000","last_fullsized":"2023-08-09T13:50:29.233755+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:50:28.150437+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:50:28.150437+0000","last_clean_scrub_stamp":"2023-08-09T13:50:28.150437+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T16:18:45.331895+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.20","version":"0'0","reported_seq":8,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-08-09T13:50:29.245742+0000","last_change":"2023-08-09T13:50:29.245742+0000","last_active":"2023-08-09T13:50:29.245742+0000","last_peered":"2023-08-09T13:50:29.245742+0000","last_clean":"2023-08-09T13:50:29.245742+0000","last_became_active":"2023-08-09T13:50:29.242577+0000","last_became_peered":"2023-08-09T13:50:29.242577+0000","last_unstale":"2023-08-09T13:50:29.245742+0000","last_undegraded":"2023-08-09T13:50:29.245742+0000","last_fullsized":"2023-08-09T13:50:29.245742+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:50:28.150437+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:50:28.150437+0000","last_clean_scrub_stamp":"2023-08-09T13:50:28.150437+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T16:18:45.331895+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.1e","version":"0'0","reported_seq":3,"reported_epoch":89,"state":"creating+activating","last_fresh":"2023-08-09T13:50:29.220103+0000","last_change":"2023-08-09T13:50:29.220103+0000","last_active":"2023-08-09T13:50:28.150437+0000","last_peered":"2023-08-09T13:50:28.150437+0000","last_clean":"2023-08-09T13:50:28.150437+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-09T13:50:29.220103+0000","last_undegraded":"2023-08-09T13:50:29.220103+0000","last_fullsized":"2023-08-09T13:50:29.220103+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:50:28.150437+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:50:28.150437+0000","last_clean_scrub_stamp":"2023-08-09T13:50:28.150437+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T19:44:56.828003+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":["7","6"],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.1f","version":"0'0","reported_seq":11,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-08-09T13:50:29.221681+0000","last_change":"2023-08-09T13:50:28.198750+0000","last_active":"2023-08-09T13:50:29.221681+0000","last_peered":"2023-08-09T13:50:29.221681+0000","last_clean":"2023-08-09T13:50:29.221681+0000","last_became_active":"2023-08-09T13:50:28.198375+0000","last_became_peered":"2023-08-09T13:50:28.198375+0000","last_unstale":"2023-08-09T13:50:29.221681+0000","last_undegraded":"2023-08-09T13:50:29.221681+0000","last_fullsized":"2023-08-09T13:50:29.221681+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:50:27.140775+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:50:27.140775+0000","last_clean_scrub_stamp":"2023-08-09T13:50:27.140775+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T23:54:42.158506+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"11.1f","version":"0'0","reported_seq":3,"reported_epoch":89,"state":"creating+activating","last_fresh":"2023-08-09T13:50:29.223179+0000","last_change":"2023-08-09T13:50:29.223179+0000","last_active":"2023-08-09T13:50:28.150437+0000","last_peered":"2023-08-09T13:50:28.150437+0000","last_clean":"2023-08-09T13:50:28.150437+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-09T13:50:29.223179+0000","last_undegraded":"2023-08-09T13:50:29.223179+0000","last_fullsized":"2023-08-09T13:50:29.223179+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:50:28.150437+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:50:28.150437+0000","last_clean_scrub_stamp":"2023-08-09T13:50:28.150437+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T20:43:24.475264+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":["2","3"],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.1e","version":"0'0","reported_seq":11,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-08-09T13:50:29.209208+0000","last_change":"2023-08-09T13:50:28.189590+0000","last_active":"2023-08-09T13:50:29.209208+0000","last_peered":"2023-08-09T13:50:29.209208+0000","last_clean":"2023-08-09T13:50:29.209208+0000","last_became_active":"2023-08-09T13:50:28.189227+0000","last_became_peered":"2023-08-09T13:50:28.189227+0000","last_unstale":"2023-08-09T13:50:29.209208+0000","last_undegraded":"2023-08-09T13:50:29.209208+0000","last_fullsized":"2023-08-09T13:50:29.209208+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:50:27.140775+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:50:27.140775+0000","last_clean_scrub_stamp":"2023-08-09T13:50:27.140775+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T19:44:55.818341+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.1c","version":"0'0","reported_seq":4,"reported_epoch":89,"state":"creating+activating","last_fresh":"2023-08-09T13:50:29.250266+0000","last_change":"2023-08-09T13:50:29.225957+0000","last_active":"2023-08-09T13:50:28.150437+0000","last_peered":"2023-08-09T13:50:28.150437+0000","last_clean":"2023-08-09T13:50:28.150437+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-09T13:50:29.250266+0000","last_undegraded":"2023-08-09T13:50:29.250266+0000","last_fullsized":"2023-08-09T13:50:29.250266+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:50:28.150437+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:50:28.150437+0000","last_clean_scrub_stamp":"2023-08-09T13:50:28.150437+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T21:57:09.347886+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":["6","7"],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"10.1d","version":"0'0","reported_seq":11,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-08-09T13:50:29.210839+0000","last_change":"2023-08-09T13:50:28.206216+0000","last_active":"2023-08-09T13:50:29.210839+0000","last_peered":"2023-08-09T13:50:29.210839+0000","last_clean":"2023-08-09T13:50:29.210839+0000","last_became_active":"2023-08-09T13:50:28.205914+0000","last_became_peered":"2023-08-09T13:50:28.205914+0000","last_unstale":"2023-08-09T13:50:29.210839+0000","last_undegraded":"2023-08-09T13:50:29.210839+0000","last_fullsized":"2023-08-09T13:50:29.210839+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:50:27.140775+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:50:27.140775+0000","last_clean_scrub_stamp":"2023-08-09T13:50:27.140775+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T16:26:26.712563+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.1d","version":"0'0","reported_seq":8,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-08-09T13:50:29.231490+0000","last_change":"2023-08-09T13:50:29.231490+0000","last_active":"2023-08-09T13:50:29.231490+0000","last_peered":"2023-08-09T13:50:29.231490+0000","last_clean":"2023-08-09T13:50:29.231490+0000","last_became_active":"2023-08-09T13:50:29.231190+0000","last_became_peered":"2023-08-09T13:50:29.231190+0000","last_unstale":"2023-08-09T13:50:29.231490+0000","last_undegraded":"2023-08-09T13:50:29.231490+0000","last_fullsized":"2023-08-09T13:50:29.231490+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:50:28.150437+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:50:28.150437+0000","last_clean_scrub_stamp":"2023-08-09T13:50:28.150437+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T14:46:45.632501+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.1c","version":"0'0","reported_seq":11,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-08-09T13:50:29.235770+0000","last_change":"2023-08-09T13:50:28.195729+0000","last_active":"2023-08-09T13:50:29.235770+0000","last_peered":"2023-08-09T13:50:29.235770+0000","last_clean":"2023-08-09T13:50:29.235770+0000","last_became_active":"2023-08-09T13:50:28.195433+0000","last_became_peered":"2023-08-09T13:50:28.195433+0000","last_unstale":"2023-08-09T13:50:29.235770+0000","last_undegraded":"2023-08-09T13:50:29.235770+0000","last_fullsized":"2023-08-09T13:50:29.235770+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:50:27.140775+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:50:27.140775+0000","last_clean_scrub_stamp":"2023-08-09T13:50:27.140775+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T16:52:42.838615+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.1a","version":"0'0","reported_seq":8,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-08-09T13:50:29.234000+0000","last_change":"2023-08-09T13:50:29.234000+0000","last_active":"2023-08-09T13:50:29.234000+0000","last_peered":"2023-08-09T13:50:29.234000+0000","last_clean":"2023-08-09T13:50:29.234000+0000","last_became_active":"2023-08-09T13:50:29.233727+0000","last_became_peered":"2023-08-09T13:50:29.233727+0000","last_unstale":"2023-08-09T13:50:29.234000+0000","last_undegraded":"2023-08-09T13:50:29.234000+0000","last_fullsized":"2023-08-09T13:50:29.234000+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:50:28.150437+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:50:28.150437+0000","last_clean_scrub_stamp":"2023-08-09T13:50:28.150437+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T20:21:45.115966+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.1b","version":"0'0","reported_seq":11,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-08-09T13:50:29.214221+0000","last_change":"2023-08-09T13:50:28.196698+0000","last_active":"2023-08-09T13:50:29.214221+0000","last_peered":"2023-08-09T13:50:29.214221+0000","last_clean":"2023-08-09T13:50:29.214221+0000","last_became_active":"2023-08-09T13:50:28.196362+0000","last_became_peered":"2023-08-09T13:50:28.196362+0000","last_unstale":"2023-08-09T13:50:29.214221+0000","last_undegraded":"2023-08-09T13:50:29.214221+0000","last_fullsized":"2023-08-09T13:50:29.214221+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:50:27.140775+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:50:27.140775+0000","last_clean_scrub_stamp":"2023-08-09T13:50:27.140775+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T16:26:00.621455+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.1b","version":"0'0","reported_seq":3,"reported_epoch":89,"state":"creating+activating","last_fresh":"2023-08-09T13:50:29.208495+0000","last_change":"2023-08-09T13:50:29.208495+0000","last_active":"2023-08-09T13:50:28.150437+0000","last_peered":"2023-08-09T13:50:28.150437+0000","last_clean":"2023-08-09T13:50:28.150437+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-09T13:50:29.208495+0000","last_undegraded":"2023-08-09T13:50:29.208495+0000","last_fullsized":"2023-08-09T13:50:29.208495+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:50:28.150437+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:50:28.150437+0000","last_clean_scrub_stamp":"2023-08-09T13:50:28.150437+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T16:20:35.941253+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":["4","7"],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"10.1a","version":"0'0","reported_seq":11,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-08-09T13:50:29.228557+0000","last_change":"2023-08-09T13:50:28.205333+0000","last_active":"2023-08-09T13:50:29.228557+0000","last_peered":"2023-08-09T13:50:29.228557+0000","last_clean":"2023-08-09T13:50:29.228557+0000","last_became_active":"2023-08-09T13:50:28.204881+0000","last_became_peered":"2023-08-09T13:50:28.204881+0000","last_unstale":"2023-08-09T13:50:29.228557+0000","last_undegraded":"2023-08-09T13:50:29.228557+0000","last_fullsized":"2023-08-09T13:50:29.228557+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:50:27.140775+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:50:27.140775+0000","last_clean_scrub_stamp":"2023-08-09T13:50:27.140775+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T18:06:20.762713+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.18","version":"0'0","reported_seq":3,"reported_epoch":89,"state":"creating+activating","last_fresh":"2023-08-09T13:50:29.218450+0000","last_change":"2023-08-09T13:50:29.218450+0000","last_active":"2023-08-09T13:50:28.150437+0000","last_peered":"2023-08-09T13:50:28.150437+0000","last_clean":"2023-08-09T13:50:28.150437+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-09T13:50:29.218450+0000","last_undegraded":"2023-08-09T13:50:29.218450+0000","last_fullsized":"2023-08-09T13:50:29.218450+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:50:28.150437+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:50:28.150437+0000","last_clean_scrub_stamp":"2023-08-09T13:50:28.150437+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T15:00:53.649507+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":["2","4"],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.19","version":"0'0","reported_seq":9,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-08-09T13:50:28.205519+0000","last_change":"2023-08-09T13:50:28.204197+0000","last_active":"2023-08-09T13:50:28.205519+0000","last_peered":"2023-08-09T13:50:28.205519+0000","last_clean":"2023-08-09T13:50:28.205519+0000","last_became_active":"2023-08-09T13:50:28.203911+0000","last_became_peered":"2023-08-09T13:50:28.203911+0000","last_unstale":"2023-08-09T13:50:28.205519+0000","last_undegraded":"2023-08-09T13:50:28.205519+0000","last_fullsized":"2023-08-09T13:50:28.205519+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:50:27.140775+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:50:27.140775+0000","last_clean_scrub_stamp":"2023-08-09T13:50:27.140775+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T18:08:46.110989+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.19","version":"0'0","reported_seq":3,"reported_epoch":89,"state":"creating+activating","last_fresh":"2023-08-09T13:50:29.206649+0000","last_change":"2023-08-09T13:50:29.206649+0000","last_active":"2023-08-09T13:50:28.150437+0000","last_peered":"2023-08-09T13:50:28.150437+0000","last_clean":"2023-08-09T13:50:28.150437+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-09T13:50:29.206649+0000","last_undegraded":"2023-08-09T13:50:29.206649+0000","last_fullsized":"2023-08-09T13:50:29.206649+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:50:28.150437+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:50:28.150437+0000","last_clean_scrub_stamp":"2023-08-09T13:50:28.150437+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T15:32:41.261189+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":["0","7"],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.18","version":"0'0","reported_seq":11,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-08-09T13:50:29.224563+0000","last_change":"2023-08-09T13:50:28.189872+0000","last_active":"2023-08-09T13:50:29.224563+0000","last_peered":"2023-08-09T13:50:29.224563+0000","last_clean":"2023-08-09T13:50:29.224563+0000","last_became_active":"2023-08-09T13:50:28.189560+0000","last_became_peered":"2023-08-09T13:50:28.189560+0000","last_unstale":"2023-08-09T13:50:29.224563+0000","last_undegraded":"2023-08-09T13:50:29.224563+0000","last_fullsized":"2023-08-09T13:50:29.224563+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:50:27.140775+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:50:27.140775+0000","last_clean_scrub_stamp":"2023-08-09T13:50:27.140775+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T18:02:42.591074+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.16","version":"0'0","reported_seq":3,"reported_epoch":89,"state":"creating+activating","last_fresh":"2023-08-09T13:50:29.223029+0000","last_change":"2023-08-09T13:50:29.223029+0000","last_active":"2023-08-09T13:50:28.150437+0000","last_peered":"2023-08-09T13:50:28.150437+0000","last_clean":"2023-08-09T13:50:28.150437+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-09T13:50:29.223029+0000","last_undegraded":"2023-08-09T13:50:29.223029+0000","last_fullsized":"2023-08-09T13:50:29.223029+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:50:28.150437+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:50:28.150437+0000","last_clean_scrub_stamp":"2023-08-09T13:50:28.150437+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T18:35:45.000310+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":["5","7"],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.17","version":"0'0","reported_seq":11,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-08-09T13:50:29.210879+0000","last_change":"2023-08-09T13:50:28.198990+0000","last_active":"2023-08-09T13:50:29.210879+0000","last_peered":"2023-08-09T13:50:29.210879+0000","last_clean":"2023-08-09T13:50:29.210879+0000","last_became_active":"2023-08-09T13:50:28.197376+0000","last_became_peered":"2023-08-09T13:50:28.197376+0000","last_unstale":"2023-08-09T13:50:29.210879+0000","last_undegraded":"2023-08-09T13:50:29.210879+0000","last_fullsized":"2023-08-09T13:50:29.210879+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:50:27.140775+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:50:27.140775+0000","last_clean_scrub_stamp":"2023-08-09T13:50:27.140775+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T18:00:13.480919+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.17","version":"0'0","reported_seq":3,"reported_epoch":89,"state":"creating+activating","last_fresh":"2023-08-09T13:50:29.221024+0000","last_change":"2023-08-09T13:50:29.221024+0000","last_active":"2023-08-09T13:50:28.150437+0000","last_peered":"2023-08-09T13:50:28.150437+0000","last_clean":"2023-08-09T13:50:28.150437+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-09T13:50:29.221024+0000","last_undegraded":"2023-08-09T13:50:29.221024+0000","last_fullsized":"2023-08-09T13:50:29.221024+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:50:28.150437+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:50:28.150437+0000","last_clean_scrub_stamp":"2023-08-09T13:50:28.150437+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T19:41:45.324636+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":["7","6"],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.16","version":"0'0","reported_seq":11,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-08-09T13:50:29.236992+0000","last_change":"2023-08-09T13:50:28.203893+0000","last_active":"2023-08-09T13:50:29.236992+0000","last_peered":"2023-08-09T13:50:29.236992+0000","last_clean":"2023-08-09T13:50:29.236992+0000","last_became_active":"2023-08-09T13:50:28.203596+0000","last_became_peered":"2023-08-09T13:50:28.203596+0000","last_unstale":"2023-08-09T13:50:29.236992+0000","last_undegraded":"2023-08-09T13:50:29.236992+0000","last_fullsized":"2023-08-09T13:50:29.236992+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:50:27.140775+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:50:27.140775+0000","last_clean_scrub_stamp":"2023-08-09T13:50:27.140775+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-11T01:07:24.154763+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.14","version":"0'0","reported_seq":8,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-08-09T13:50:29.236992+0000","last_change":"2023-08-09T13:50:29.236992+0000","last_active":"2023-08-09T13:50:29.236992+0000","last_peered":"2023-08-09T13:50:29.236992+0000","last_clean":"2023-08-09T13:50:29.236992+0000","last_became_active":"2023-08-09T13:50:29.236708+0000","last_became_peered":"2023-08-09T13:50:29.236708+0000","last_unstale":"2023-08-09T13:50:29.236992+0000","last_undegraded":"2023-08-09T13:50:29.236992+0000","last_fullsized":"2023-08-09T13:50:29.236992+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:50:28.150437+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:50:28.150437+0000","last_clean_scrub_stamp":"2023-08-09T13:50:28.150437+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T23:19:57.122069+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"10.15","version":"0'0","reported_seq":11,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-08-09T13:50:29.215134+0000","last_change":"2023-08-09T13:50:28.198818+0000","last_active":"2023-08-09T13:50:29.215134+0000","last_peered":"2023-08-09T13:50:29.215134+0000","last_clean":"2023-08-09T13:50:29.215134+0000","last_became_active":"2023-08-09T13:50:28.198282+0000","last_became_peered":"2023-08-09T13:50:28.198282+0000","last_unstale":"2023-08-09T13:50:29.215134+0000","last_undegraded":"2023-08-09T13:50:29.215134+0000","last_fullsized":"2023-08-09T13:50:29.215134+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:50:27.140775+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:50:27.140775+0000","last_clean_scrub_stamp":"2023-08-09T13:50:27.140775+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T13:56:48.165182+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.15","version":"0'0","reported_seq":8,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-08-09T13:50:29.232045+0000","last_change":"2023-08-09T13:50:29.232045+0000","last_active":"2023-08-09T13:50:29.232045+0000","last_peered":"2023-08-09T13:50:29.232045+0000","last_clean":"2023-08-09T13:50:29.232045+0000","last_became_active":"2023-08-09T13:50:29.231691+0000","last_became_peered":"2023-08-09T13:50:29.231691+0000","last_unstale":"2023-08-09T13:50:29.232045+0000","last_undegraded":"2023-08-09T13:50:29.232045+0000","last_fullsized":"2023-08-09T13:50:29.232045+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:50:28.150437+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:50:28.150437+0000","last_clean_scrub_stamp":"2023-08-09T13:50:28.150437+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T14:41:33.537815+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.14","version":"0'0","reported_seq":9,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-08-09T13:50:28.198290+0000","last_change":"2023-08-09T13:50:28.194890+0000","last_active":"2023-08-09T13:50:28.198290+0000","last_peered":"2023-08-09T13:50:28.198290+0000","last_clean":"2023-08-09T13:50:28.198290+0000","last_became_active":"2023-08-09T13:50:28.194607+0000","last_became_peered":"2023-08-09T13:50:28.194607+0000","last_unstale":"2023-08-09T13:50:28.198290+0000","last_undegraded":"2023-08-09T13:50:28.198290+0000","last_fullsized":"2023-08-09T13:50:28.198290+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:50:27.140775+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:50:27.140775+0000","last_clean_scrub_stamp":"2023-08-09T13:50:27.140775+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-11T01:35:47.708771+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.12","version":"0'0","reported_seq":8,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-08-09T13:50:29.233976+0000","last_change":"2023-08-09T13:50:29.233976+0000","last_active":"2023-08-09T13:50:29.233976+0000","last_peered":"2023-08-09T13:50:29.233976+0000","last_clean":"2023-08-09T13:50:29.233976+0000","last_became_active":"2023-08-09T13:50:29.233696+0000","last_became_peered":"2023-08-09T13:50:29.233696+0000","last_unstale":"2023-08-09T13:50:29.233976+0000","last_undegraded":"2023-08-09T13:50:29.233976+0000","last_fullsized":"2023-08-09T13:50:29.233976+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:50:28.150437+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:50:28.150437+0000","last_clean_scrub_stamp":"2023-08-09T13:50:28.150437+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T15:32:41.261189+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.13","version":"0'0","reported_seq":9,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-08-09T13:50:28.200538+0000","last_change":"2023-08-09T13:50:28.198586+0000","last_active":"2023-08-09T13:50:28.200538+0000","last_peered":"2023-08-09T13:50:28.200538+0000","last_clean":"2023-08-09T13:50:28.200538+0000","last_became_active":"2023-08-09T13:50:28.198247+0000","last_became_peered":"2023-08-09T13:50:28.198247+0000","last_unstale":"2023-08-09T13:50:28.200538+0000","last_undegraded":"2023-08-09T13:50:28.200538+0000","last_fullsized":"2023-08-09T13:50:28.200538+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:50:27.140775+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:50:27.140775+0000","last_clean_scrub_stamp":"2023-08-09T13:50:27.140775+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T21:31:16.170439+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.13","version":"0'0","reported_seq":3,"reported_epoch":89,"state":"creating+activating","last_fresh":"2023-08-09T13:50:29.220908+0000","last_change":"2023-08-09T13:50:29.220908+0000","last_active":"2023-08-09T13:50:28.150437+0000","last_peered":"2023-08-09T13:50:28.150437+0000","last_clean":"2023-08-09T13:50:28.150437+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-09T13:50:29.220908+0000","last_undegraded":"2023-08-09T13:50:29.220908+0000","last_fullsized":"2023-08-09T13:50:29.220908+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:50:28.150437+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:50:28.150437+0000","last_clean_scrub_stamp":"2023-08-09T13:50:28.150437+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T18:15:31.389069+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":["5","4"],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.12","version":"0'0","reported_seq":11,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-08-09T13:50:29.235230+0000","last_change":"2023-08-09T13:50:28.201822+0000","last_active":"2023-08-09T13:50:29.235230+0000","last_peered":"2023-08-09T13:50:29.235230+0000","last_clean":"2023-08-09T13:50:29.235230+0000","last_became_active":"2023-08-09T13:50:28.201187+0000","last_became_peered":"2023-08-09T13:50:28.201187+0000","last_unstale":"2023-08-09T13:50:29.235230+0000","last_undegraded":"2023-08-09T13:50:29.235230+0000","last_fullsized":"2023-08-09T13:50:29.235230+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:50:27.140775+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:50:27.140775+0000","last_clean_scrub_stamp":"2023-08-09T13:50:27.140775+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T23:52:03.500625+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.10","version":"0'0","reported_seq":3,"reported_epoch":89,"state":"creating+activating","last_fresh":"2023-08-09T13:50:29.205817+0000","last_change":"2023-08-09T13:50:29.205817+0000","last_active":"2023-08-09T13:50:28.150437+0000","last_peered":"2023-08-09T13:50:28.150437+0000","last_clean":"2023-08-09T13:50:28.150437+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-09T13:50:29.205817+0000","last_undegraded":"2023-08-09T13:50:29.205817+0000","last_fullsized":"2023-08-09T13:50:29.205817+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:50:28.150437+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:50:28.150437+0000","last_clean_scrub_stamp":"2023-08-09T13:50:28.150437+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-11T01:02:33.470393+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":["0","6"],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.11","version":"0'0","reported_seq":11,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-08-09T13:50:29.221870+0000","last_change":"2023-08-09T13:50:28.193484+0000","last_active":"2023-08-09T13:50:29.221870+0000","last_peered":"2023-08-09T13:50:29.221870+0000","last_clean":"2023-08-09T13:50:29.221870+0000","last_became_active":"2023-08-09T13:50:28.193098+0000","last_became_peered":"2023-08-09T13:50:28.193098+0000","last_unstale":"2023-08-09T13:50:29.221870+0000","last_undegraded":"2023-08-09T13:50:29.221870+0000","last_fullsized":"2023-08-09T13:50:29.221870+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:50:27.140775+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:50:27.140775+0000","last_clean_scrub_stamp":"2023-08-09T13:50:27.140775+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T21:57:08.338223+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.11","version":"0'0","reported_seq":3,"reported_epoch":89,"state":"creating+activating","last_fresh":"2023-08-09T13:50:29.224448+0000","last_change":"2023-08-09T13:50:29.224448+0000","last_active":"2023-08-09T13:50:28.150437+0000","last_peered":"2023-08-09T13:50:28.150437+0000","last_clean":"2023-08-09T13:50:28.150437+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-09T13:50:29.224448+0000","last_undegraded":"2023-08-09T13:50:29.224448+0000","last_fullsized":"2023-08-09T13:50:29.224448+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:50:28.150437+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:50:28.150437+0000","last_clean_scrub_stamp":"2023-08-09T13:50:28.150437+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T19:49:56.370072+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":["2","3"],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.10","version":"0'0","reported_seq":11,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-08-09T13:50:29.209343+0000","last_change":"2023-08-09T13:50:28.200670+0000","last_active":"2023-08-09T13:50:29.209343+0000","last_peered":"2023-08-09T13:50:29.209343+0000","last_clean":"2023-08-09T13:50:29.209343+0000","last_became_active":"2023-08-09T13:50:28.200419+0000","last_became_peered":"2023-08-09T13:50:28.200419+0000","last_unstale":"2023-08-09T13:50:29.209343+0000","last_undegraded":"2023-08-09T13:50:29.209343+0000","last_fullsized":"2023-08-09T13:50:29.209343+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:50:27.140775+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:50:27.140775+0000","last_clean_scrub_stamp":"2023-08-09T13:50:27.140775+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T15:23:31.632434+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.6","version":"0'0","reported_seq":3,"reported_epoch":89,"state":"creating+activating","last_fresh":"2023-08-09T13:50:29.208662+0000","last_change":"2023-08-09T13:50:29.208662+0000","last_active":"2023-08-09T13:50:28.150437+0000","last_peered":"2023-08-09T13:50:28.150437+0000","last_clean":"2023-08-09T13:50:28.150437+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-09T13:50:29.208662+0000","last_undegraded":"2023-08-09T13:50:29.208662+0000","last_fullsized":"2023-08-09T13:50:29.208662+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:50:28.150437+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:50:28.150437+0000","last_clean_scrub_stamp":"2023-08-09T13:50:28.150437+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T17:05:45.752848+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":["0","2"],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.7","version":"0'0","reported_seq":11,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-08-09T13:50:29.230633+0000","last_change":"2023-08-09T13:50:28.204446+0000","last_active":"2023-08-09T13:50:29.230633+0000","last_peered":"2023-08-09T13:50:29.230633+0000","last_clean":"2023-08-09T13:50:29.230633+0000","last_became_active":"2023-08-09T13:50:28.204093+0000","last_became_peered":"2023-08-09T13:50:28.204093+0000","last_unstale":"2023-08-09T13:50:29.230633+0000","last_undegraded":"2023-08-09T13:50:29.230633+0000","last_fullsized":"2023-08-09T13:50:29.230633+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:50:27.140775+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:50:27.140775+0000","last_clean_scrub_stamp":"2023-08-09T13:50:27.140775+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T22:56:15.821115+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.7","version":"0'0","reported_seq":3,"reported_epoch":89,"state":"creating+activating","last_fresh":"2023-08-09T13:50:29.223626+0000","last_change":"2023-08-09T13:50:29.223626+0000","last_active":"2023-08-09T13:50:28.150437+0000","last_peered":"2023-08-09T13:50:28.150437+0000","last_clean":"2023-08-09T13:50:28.150437+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-09T13:50:29.223626+0000","last_undegraded":"2023-08-09T13:50:29.223626+0000","last_fullsized":"2023-08-09T13:50:29.223626+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:50:28.150437+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:50:28.150437+0000","last_clean_scrub_stamp":"2023-08-09T13:50:28.150437+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T18:06:21.772376+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":["5","1"],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.6","version":"0'0","reported_seq":11,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-08-09T13:50:29.223926+0000","last_change":"2023-08-09T13:50:28.191955+0000","last_active":"2023-08-09T13:50:29.223926+0000","last_peered":"2023-08-09T13:50:29.223926+0000","last_clean":"2023-08-09T13:50:29.223926+0000","last_became_active":"2023-08-09T13:50:28.191264+0000","last_became_peered":"2023-08-09T13:50:28.191264+0000","last_unstale":"2023-08-09T13:50:29.223926+0000","last_undegraded":"2023-08-09T13:50:29.223926+0000","last_fullsized":"2023-08-09T13:50:29.223926+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:50:27.140775+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:50:27.140775+0000","last_clean_scrub_stamp":"2023-08-09T13:50:27.140775+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T18:00:04.982575+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.4","version":"0'0","reported_seq":3,"reported_epoch":89,"state":"creating+activating","last_fresh":"2023-08-09T13:50:29.216528+0000","last_change":"2023-08-09T13:50:29.216528+0000","last_active":"2023-08-09T13:50:28.150437+0000","last_peered":"2023-08-09T13:50:28.150437+0000","last_clean":"2023-08-09T13:50:28.150437+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-09T13:50:29.216528+0000","last_undegraded":"2023-08-09T13:50:29.216528+0000","last_fullsized":"2023-08-09T13:50:29.216528+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:50:28.150437+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:50:28.150437+0000","last_clean_scrub_stamp":"2023-08-09T13:50:28.150437+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T21:36:45.909480+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":["7","6"],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.5","version":"0'0","reported_seq":11,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-08-09T13:50:29.224819+0000","last_change":"2023-08-09T13:50:28.193912+0000","last_active":"2023-08-09T13:50:29.224819+0000","last_peered":"2023-08-09T13:50:29.224819+0000","last_clean":"2023-08-09T13:50:29.224819+0000","last_became_active":"2023-08-09T13:50:28.193602+0000","last_became_peered":"2023-08-09T13:50:28.193602+0000","last_unstale":"2023-08-09T13:50:29.224819+0000","last_undegraded":"2023-08-09T13:50:29.224819+0000","last_fullsized":"2023-08-09T13:50:29.224819+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:50:27.140775+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:50:27.140775+0000","last_clean_scrub_stamp":"2023-08-09T13:50:27.140775+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T19:25:18.625389+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.5","version":"0'0","reported_seq":8,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-08-09T13:50:29.237250+0000","last_change":"2023-08-09T13:50:29.237250+0000","last_active":"2023-08-09T13:50:29.237250+0000","last_peered":"2023-08-09T13:50:29.237250+0000","last_clean":"2023-08-09T13:50:29.237250+0000","last_became_active":"2023-08-09T13:50:29.236821+0000","last_became_peered":"2023-08-09T13:50:29.236821+0000","last_unstale":"2023-08-09T13:50:29.237250+0000","last_undegraded":"2023-08-09T13:50:29.237250+0000","last_fullsized":"2023-08-09T13:50:29.237250+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:50:28.150437+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:50:28.150437+0000","last_clean_scrub_stamp":"2023-08-09T13:50:28.150437+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T15:02:09.473638+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.4","version":"0'0","reported_seq":11,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-08-09T13:50:29.208326+0000","last_change":"2023-08-09T13:50:28.197934+0000","last_active":"2023-08-09T13:50:29.208326+0000","last_peered":"2023-08-09T13:50:29.208326+0000","last_clean":"2023-08-09T13:50:29.208326+0000","last_became_active":"2023-08-09T13:50:28.197639+0000","last_became_peered":"2023-08-09T13:50:28.197639+0000","last_unstale":"2023-08-09T13:50:29.208326+0000","last_undegraded":"2023-08-09T13:50:29.208326+0000","last_fullsized":"2023-08-09T13:50:29.208326+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:50:27.140775+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:50:27.140775+0000","last_clean_scrub_stamp":"2023-08-09T13:50:27.140775+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T14:46:44.622839+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.2","version":"0'0","reported_seq":3,"reported_epoch":89,"state":"creating+activating","last_fresh":"2023-08-09T13:50:29.217549+0000","last_change":"2023-08-09T13:50:29.217549+0000","last_active":"2023-08-09T13:50:28.150437+0000","last_peered":"2023-08-09T13:50:28.150437+0000","last_clean":"2023-08-09T13:50:28.150437+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-09T13:50:29.217549+0000","last_undegraded":"2023-08-09T13:50:29.217549+0000","last_fullsized":"2023-08-09T13:50:29.217549+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:50:28.150437+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:50:28.150437+0000","last_clean_scrub_stamp":"2023-08-09T13:50:28.150437+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-11T00:03:22.851529+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":["2","3"],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.3","version":"0'0","reported_seq":11,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-08-09T13:50:29.224120+0000","last_change":"2023-08-09T13:50:28.189872+0000","last_active":"2023-08-09T13:50:29.224120+0000","last_peered":"2023-08-09T13:50:29.224120+0000","last_clean":"2023-08-09T13:50:29.224120+0000","last_became_active":"2023-08-09T13:50:28.189588+0000","last_became_peered":"2023-08-09T13:50:28.189588+0000","last_unstale":"2023-08-09T13:50:29.224120+0000","last_undegraded":"2023-08-09T13:50:29.224120+0000","last_fullsized":"2023-08-09T13:50:29.224120+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:50:27.140775+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:50:27.140775+0000","last_clean_scrub_stamp":"2023-08-09T13:50:27.140775+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T16:47:26.737473+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.3","version":"0'0","reported_seq":8,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-08-09T13:50:29.250246+0000","last_change":"2023-08-09T13:50:29.250246+0000","last_active":"2023-08-09T13:50:29.250246+0000","last_peered":"2023-08-09T13:50:29.250246+0000","last_clean":"2023-08-09T13:50:29.250246+0000","last_became_active":"2023-08-09T13:50:29.249885+0000","last_became_peered":"2023-08-09T13:50:29.249885+0000","last_unstale":"2023-08-09T13:50:29.250246+0000","last_undegraded":"2023-08-09T13:50:29.250246+0000","last_fullsized":"2023-08-09T13:50:29.250246+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:50:28.150437+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:50:28.150437+0000","last_clean_scrub_stamp":"2023-08-09T13:50:28.150437+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T21:52:25.852199+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"10.2","version":"0'0","reported_seq":9,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-08-09T13:50:28.203714+0000","last_change":"2023-08-09T13:50:28.202029+0000","last_active":"2023-08-09T13:50:28.203714+0000","last_peered":"2023-08-09T13:50:28.203714+0000","last_clean":"2023-08-09T13:50:28.203714+0000","last_became_active":"2023-08-09T13:50:28.201720+0000","last_became_peered":"2023-08-09T13:50:28.201720+0000","last_unstale":"2023-08-09T13:50:28.203714+0000","last_undegraded":"2023-08-09T13:50:28.203714+0000","last_fullsized":"2023-08-09T13:50:28.203714+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:50:27.140775+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:50:27.140775+0000","last_clean_scrub_stamp":"2023-08-09T13:50:27.140775+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T19:37:29.224002+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.0","version":"0'0","reported_seq":1,"reported_epoch":88,"state":"creating+peering","last_fresh":"2023-08-09T13:50:28.189500+0000","last_change":"2023-08-09T13:50:28.189500+0000","last_active":"2023-08-09T13:50:28.150437+0000","last_peered":"2023-08-09T13:50:28.150437+0000","last_clean":"2023-08-09T13:50:28.150437+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-09T13:50:28.189500+0000","last_undegraded":"2023-08-09T13:50:28.189500+0000","last_fullsized":"2023-08-09T13:50:28.189500+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:50:28.150437+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:50:28.150437+0000","last_clean_scrub_stamp":"2023-08-09T13:50:28.150437+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.1","version":"0'0","reported_seq":11,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-08-09T13:50:29.209627+0000","last_change":"2023-08-09T13:50:28.200080+0000","last_active":"2023-08-09T13:50:29.209627+0000","last_peered":"2023-08-09T13:50:29.209627+0000","last_clean":"2023-08-09T13:50:29.209627+0000","last_became_active":"2023-08-09T13:50:28.199831+0000","last_became_peered":"2023-08-09T13:50:28.199831+0000","last_unstale":"2023-08-09T13:50:29.209627+0000","last_undegraded":"2023-08-09T13:50:29.209627+0000","last_fullsized":"2023-08-09T13:50:29.209627+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:50:27.140775+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:50:27.140775+0000","last_clean_scrub_stamp":"2023-08-09T13:50:27.140775+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T15:18:11.293013+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.1","version":"0'0","reported_seq":8,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-08-09T13:50:29.249892+0000","last_change":"2023-08-09T13:50:29.249892+0000","last_active":"2023-08-09T13:50:29.249892+0000","last_peered":"2023-08-09T13:50:29.249892+0000","last_clean":"2023-08-09T13:50:29.249892+0000","last_became_active":"2023-08-09T13:50:29.242554+0000","last_became_peered":"2023-08-09T13:50:29.242554+0000","last_unstale":"2023-08-09T13:50:29.249892+0000","last_undegraded":"2023-08-09T13:50:29.249892+0000","last_fullsized":"2023-08-09T13:50:29.249892+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:50:28.150437+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:50:28.150437+0000","last_clean_scrub_stamp":"2023-08-09T13:50:28.150437+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T21:18:31.512056+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"10.0","version":"0'0","reported_seq":11,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-08-09T13:50:29.242250+0000","last_change":"2023-08-09T13:50:28.197319+0000","last_active":"2023-08-09T13:50:29.242250+0000","last_peered":"2023-08-09T13:50:29.242250+0000","last_clean":"2023-08-09T13:50:29.242250+0000","last_became_active":"2023-08-09T13:50:28.197031+0000","last_became_peered":"2023-08-09T13:50:28.197031+0000","last_unstale":"2023-08-09T13:50:29.242250+0000","last_undegraded":"2023-08-09T13:50:29.242250+0000","last_fullsized":"2023-08-09T13:50:29.242250+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:50:27.140775+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:50:27.140775+0000","last_clean_scrub_stamp":"2023-08-09T13:50:27.140775+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T16:15:05.176106+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.a","version":"0'0","reported_seq":3,"reported_epoch":89,"state":"creating+activating","last_fresh":"2023-08-09T13:50:29.212247+0000","last_change":"2023-08-09T13:50:29.212247+0000","last_active":"2023-08-09T13:50:28.150437+0000","last_peered":"2023-08-09T13:50:28.150437+0000","last_clean":"2023-08-09T13:50:28.150437+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-09T13:50:29.212247+0000","last_undegraded":"2023-08-09T13:50:29.212247+0000","last_fullsized":"2023-08-09T13:50:29.212247+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:50:28.150437+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:50:28.150437+0000","last_clean_scrub_stamp":"2023-08-09T13:50:28.150437+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T19:55:58.171731+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":["3","6"],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.b","version":"0'0","reported_seq":11,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-08-09T13:50:29.230951+0000","last_change":"2023-08-09T13:50:28.203783+0000","last_active":"2023-08-09T13:50:29.230951+0000","last_peered":"2023-08-09T13:50:29.230951+0000","last_clean":"2023-08-09T13:50:29.230951+0000","last_became_active":"2023-08-09T13:50:28.202391+0000","last_became_peered":"2023-08-09T13:50:28.202391+0000","last_unstale":"2023-08-09T13:50:29.230951+0000","last_undegraded":"2023-08-09T13:50:29.230951+0000","last_fullsized":"2023-08-09T13:50:29.230951+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:50:27.140775+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:50:27.140775+0000","last_clean_scrub_stamp":"2023-08-09T13:50:27.140775+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T21:39:38.493243+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":149,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-08-09T13:50:29.216433+0000","last_change":"2023-08-09T13:33:56.401406+0000","last_active":"2023-08-09T13:50:29.216433+0000","last_peered":"2023-08-09T13:50:29.216433+0000","last_clean":"2023-08-09T13:50:29.216433+0000","last_became_active":"2023-08-09T13:33:56.400487+0000","last_became_peered":"2023-08-09T13:33:56.400487+0000","last_unstale":"2023-08-09T13:50:29.216433+0000","last_undegraded":"2023-08-09T13:50:29.216433+0000","last_fullsized":"2023-08-09T13:50:29.216433+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:33:54.845869+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:33:54.845869+0000","last_clean_scrub_stamp":"2023-08-09T13:33:54.845869+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-11T00:46:00.165825+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"11.b","version":"0'0","reported_seq":3,"reported_epoch":89,"state":"creating+activating","last_fresh":"2023-08-09T13:50:29.211663+0000","last_change":"2023-08-09T13:50:29.211663+0000","last_active":"2023-08-09T13:50:28.150437+0000","last_peered":"2023-08-09T13:50:28.150437+0000","last_clean":"2023-08-09T13:50:28.150437+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-09T13:50:29.211663+0000","last_undegraded":"2023-08-09T13:50:29.211663+0000","last_fullsized":"2023-08-09T13:50:29.211663+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:50:28.150437+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:50:28.150437+0000","last_clean_scrub_stamp":"2023-08-09T13:50:28.150437+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-11T01:04:03.965638+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":["3","1"],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.a","version":"0'0","reported_seq":11,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-08-09T13:50:29.230486+0000","last_change":"2023-08-09T13:50:28.205148+0000","last_active":"2023-08-09T13:50:29.230486+0000","last_peered":"2023-08-09T13:50:29.230486+0000","last_clean":"2023-08-09T13:50:29.230486+0000","last_became_active":"2023-08-09T13:50:28.204528+0000","last_became_peered":"2023-08-09T13:50:28.204528+0000","last_unstale":"2023-08-09T13:50:29.230486+0000","last_undegraded":"2023-08-09T13:50:29.230486+0000","last_fullsized":"2023-08-09T13:50:29.230486+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:50:27.140775+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:50:27.140775+0000","last_clean_scrub_stamp":"2023-08-09T13:50:27.140775+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T18:00:04.982575+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":148,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-08-09T13:50:29.222206+0000","last_change":"2023-08-09T13:33:56.401123+0000","last_active":"2023-08-09T13:50:29.222206+0000","last_peered":"2023-08-09T13:50:29.222206+0000","last_clean":"2023-08-09T13:50:29.222206+0000","last_became_active":"2023-08-09T13:33:56.399933+0000","last_became_peered":"2023-08-09T13:33:56.399933+0000","last_unstale":"2023-08-09T13:50:29.222206+0000","last_undegraded":"2023-08-09T13:50:29.222206+0000","last_fullsized":"2023-08-09T13:50:29.222206+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:33:54.845869+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:33:54.845869+0000","last_clean_scrub_stamp":"2023-08-09T13:33:54.845869+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T17:05:05.384552+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.8","version":"0'0","reported_seq":3,"reported_epoch":89,"state":"creating+activating","last_fresh":"2023-08-09T13:50:29.213241+0000","last_change":"2023-08-09T13:50:29.213241+0000","last_active":"2023-08-09T13:50:28.150437+0000","last_peered":"2023-08-09T13:50:28.150437+0000","last_clean":"2023-08-09T13:50:28.150437+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-09T13:50:29.213241+0000","last_undegraded":"2023-08-09T13:50:29.213241+0000","last_fullsized":"2023-08-09T13:50:29.213241+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:50:28.150437+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:50:28.150437+0000","last_clean_scrub_stamp":"2023-08-09T13:50:28.150437+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T22:09:32.464173+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":["3","6"],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.9","version":"0'0","reported_seq":11,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-08-09T13:50:29.222528+0000","last_change":"2023-08-09T13:50:28.192232+0000","last_active":"2023-08-09T13:50:29.222528+0000","last_peered":"2023-08-09T13:50:29.222528+0000","last_clean":"2023-08-09T13:50:29.222528+0000","last_became_active":"2023-08-09T13:50:28.191350+0000","last_became_peered":"2023-08-09T13:50:28.191350+0000","last_unstale":"2023-08-09T13:50:29.222528+0000","last_undegraded":"2023-08-09T13:50:29.222528+0000","last_fullsized":"2023-08-09T13:50:29.222528+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:50:27.140775+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:50:27.140775+0000","last_clean_scrub_stamp":"2023-08-09T13:50:27.140775+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-11T00:33:31.648004+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.2","version":"0'0","reported_seq":148,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-08-09T13:50:29.215918+0000","last_change":"2023-08-09T13:33:56.402794+0000","last_active":"2023-08-09T13:50:29.215918+0000","last_peered":"2023-08-09T13:50:29.215918+0000","last_clean":"2023-08-09T13:50:29.215918+0000","last_became_active":"2023-08-09T13:33:56.402038+0000","last_became_peered":"2023-08-09T13:33:56.402038+0000","last_unstale":"2023-08-09T13:50:29.215918+0000","last_undegraded":"2023-08-09T13:50:29.215918+0000","last_fullsized":"2023-08-09T13:50:29.215918+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:33:54.845869+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:33:54.845869+0000","last_clean_scrub_stamp":"2023-08-09T13:33:54.845869+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T16:35:28.133841+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"11.9","version":"0'0","reported_seq":3,"reported_epoch":89,"state":"creating+activating","last_fresh":"2023-08-09T13:50:29.211902+0000","last_change":"2023-08-09T13:50:29.211902+0000","last_active":"2023-08-09T13:50:28.150437+0000","last_peered":"2023-08-09T13:50:28.150437+0000","last_clean":"2023-08-09T13:50:28.150437+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-09T13:50:29.211902+0000","last_undegraded":"2023-08-09T13:50:29.211902+0000","last_fullsized":"2023-08-09T13:50:29.211902+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:50:28.150437+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:50:28.150437+0000","last_clean_scrub_stamp":"2023-08-09T13:50:28.150437+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T15:32:31.232979+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":["3","6"],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.8","version":"0'0","reported_seq":11,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-08-09T13:50:29.205186+0000","last_change":"2023-08-09T13:50:28.190965+0000","last_active":"2023-08-09T13:50:29.205186+0000","last_peered":"2023-08-09T13:50:29.205186+0000","last_clean":"2023-08-09T13:50:29.205186+0000","last_became_active":"2023-08-09T13:50:28.190696+0000","last_became_peered":"2023-08-09T13:50:28.190696+0000","last_unstale":"2023-08-09T13:50:29.205186+0000","last_undegraded":"2023-08-09T13:50:29.205186+0000","last_fullsized":"2023-08-09T13:50:29.205186+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:50:27.140775+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:50:27.140775+0000","last_clean_scrub_stamp":"2023-08-09T13:50:27.140775+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T21:36:44.899817+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.3","version":"0'0","reported_seq":146,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-08-09T13:50:28.184637+0000","last_change":"2023-08-09T13:33:55.886749+0000","last_active":"2023-08-09T13:50:28.184637+0000","last_peered":"2023-08-09T13:50:28.184637+0000","last_clean":"2023-08-09T13:50:28.184637+0000","last_became_active":"2023-08-09T13:33:55.886116+0000","last_became_peered":"2023-08-09T13:33:55.886116+0000","last_unstale":"2023-08-09T13:50:28.184637+0000","last_undegraded":"2023-08-09T13:50:28.184637+0000","last_fullsized":"2023-08-09T13:50:28.184637+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:33:54.845869+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:33:54.845869+0000","last_clean_scrub_stamp":"2023-08-09T13:33:54.845869+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T16:59:16.380973+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"77'161","reported_seq":455,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-08-09T13:50:29.216076+0000","last_change":"2023-08-09T13:33:55.886345+0000","last_active":"2023-08-09T13:50:29.216076+0000","last_peered":"2023-08-09T13:50:29.216076+0000","last_clean":"2023-08-09T13:50:29.216076+0000","last_became_active":"2023-08-09T13:33:55.885781+0000","last_became_peered":"2023-08-09T13:33:55.885781+0000","last_unstale":"2023-08-09T13:50:29.216076+0000","last_undegraded":"2023-08-09T13:50:29.216076+0000","last_fullsized":"2023-08-09T13:50:29.216076+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:33:54.845869+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:33:54.845869+0000","last_clean_scrub_stamp":"2023-08-09T13:33:54.845869+0000","objects_scrubbed":0,"log_size":161,"log_dups_size":0,"ondisk_log_size":161,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T19:49:09.757332+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":500,"num_read_kb":924,"num_write":239,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"11.e","version":"0'0","reported_seq":9,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-08-09T13:50:29.232907+0000","last_change":"2023-08-09T13:50:29.227243+0000","last_active":"2023-08-09T13:50:29.232907+0000","last_peered":"2023-08-09T13:50:29.232907+0000","last_clean":"2023-08-09T13:50:29.232907+0000","last_became_active":"2023-08-09T13:50:29.226756+0000","last_became_peered":"2023-08-09T13:50:29.226756+0000","last_unstale":"2023-08-09T13:50:29.232907+0000","last_undegraded":"2023-08-09T13:50:29.232907+0000","last_fullsized":"2023-08-09T13:50:29.232907+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:50:28.150437+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:50:28.150437+0000","last_clean_scrub_stamp":"2023-08-09T13:50:28.150437+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T18:35:12.621508+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"10.f","version":"0'0","reported_seq":11,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-08-09T13:50:29.230485+0000","last_change":"2023-08-09T13:50:28.205120+0000","last_active":"2023-08-09T13:50:29.230485+0000","last_peered":"2023-08-09T13:50:29.230485+0000","last_clean":"2023-08-09T13:50:29.230485+0000","last_became_active":"2023-08-09T13:50:28.204771+0000","last_became_peered":"2023-08-09T13:50:28.204771+0000","last_unstale":"2023-08-09T13:50:29.230485+0000","last_undegraded":"2023-08-09T13:50:29.230485+0000","last_fullsized":"2023-08-09T13:50:29.230485+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:50:27.140775+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:50:27.140775+0000","last_clean_scrub_stamp":"2023-08-09T13:50:27.140775+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T17:34:29.132908+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":158,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-08-09T13:50:29.211123+0000","last_change":"2023-08-09T13:33:55.889565+0000","last_active":"2023-08-09T13:50:29.211123+0000","last_peered":"2023-08-09T13:50:29.211123+0000","last_clean":"2023-08-09T13:50:29.211123+0000","last_became_active":"2023-08-09T13:33:55.889164+0000","last_became_peered":"2023-08-09T13:33:55.889164+0000","last_unstale":"2023-08-09T13:50:29.211123+0000","last_undegraded":"2023-08-09T13:50:29.211123+0000","last_fullsized":"2023-08-09T13:50:29.211123+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:33:54.845869+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:33:54.845869+0000","last_clean_scrub_stamp":"2023-08-09T13:33:54.845869+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T17:48:30.355598+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.f","version":"0'0","reported_seq":8,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-08-09T13:50:29.238559+0000","last_change":"2023-08-09T13:50:29.238559+0000","last_active":"2023-08-09T13:50:29.238559+0000","last_peered":"2023-08-09T13:50:29.238559+0000","last_clean":"2023-08-09T13:50:29.238559+0000","last_became_active":"2023-08-09T13:50:29.238287+0000","last_became_peered":"2023-08-09T13:50:29.238287+0000","last_unstale":"2023-08-09T13:50:29.238559+0000","last_undegraded":"2023-08-09T13:50:29.238559+0000","last_fullsized":"2023-08-09T13:50:29.238559+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:50:28.150437+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:50:28.150437+0000","last_clean_scrub_stamp":"2023-08-09T13:50:28.150437+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T16:41:09.645304+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.e","version":"0'0","reported_seq":11,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-08-09T13:50:29.211102+0000","last_change":"2023-08-09T13:50:28.206561+0000","last_active":"2023-08-09T13:50:29.211102+0000","last_peered":"2023-08-09T13:50:29.211102+0000","last_clean":"2023-08-09T13:50:29.211102+0000","last_became_active":"2023-08-09T13:50:28.206264+0000","last_became_peered":"2023-08-09T13:50:28.206264+0000","last_unstale":"2023-08-09T13:50:29.211102+0000","last_undegraded":"2023-08-09T13:50:29.211102+0000","last_fullsized":"2023-08-09T13:50:29.211102+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:50:27.140775+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:50:27.140775+0000","last_clean_scrub_stamp":"2023-08-09T13:50:27.140775+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T15:29:27.312440+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":148,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-08-09T13:50:29.210681+0000","last_change":"2023-08-09T13:33:56.404236+0000","last_active":"2023-08-09T13:50:29.210681+0000","last_peered":"2023-08-09T13:50:29.210681+0000","last_clean":"2023-08-09T13:50:29.210681+0000","last_became_active":"2023-08-09T13:33:56.402485+0000","last_became_peered":"2023-08-09T13:33:56.402485+0000","last_unstale":"2023-08-09T13:50:29.210681+0000","last_undegraded":"2023-08-09T13:50:29.210681+0000","last_fullsized":"2023-08-09T13:50:29.210681+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:33:54.845869+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:33:54.845869+0000","last_clean_scrub_stamp":"2023-08-09T13:33:54.845869+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T13:49:18.686986+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.c","version":"0'0","reported_seq":1,"reported_epoch":88,"state":"creating+peering","last_fresh":"2023-08-09T13:50:28.187674+0000","last_change":"2023-08-09T13:50:28.187674+0000","last_active":"2023-08-09T13:50:28.150437+0000","last_peered":"2023-08-09T13:50:28.150437+0000","last_clean":"2023-08-09T13:50:28.150437+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-09T13:50:28.187674+0000","last_undegraded":"2023-08-09T13:50:28.187674+0000","last_fullsized":"2023-08-09T13:50:28.187674+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:50:28.150437+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:50:28.150437+0000","last_clean_scrub_stamp":"2023-08-09T13:50:28.150437+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.d","version":"0'0","reported_seq":11,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-08-09T13:50:29.227255+0000","last_change":"2023-08-09T13:50:28.199480+0000","last_active":"2023-08-09T13:50:29.227255+0000","last_peered":"2023-08-09T13:50:29.227255+0000","last_clean":"2023-08-09T13:50:29.227255+0000","last_became_active":"2023-08-09T13:50:28.199211+0000","last_became_peered":"2023-08-09T13:50:28.199211+0000","last_unstale":"2023-08-09T13:50:29.227255+0000","last_undegraded":"2023-08-09T13:50:29.227255+0000","last_fullsized":"2023-08-09T13:50:29.227255+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:50:27.140775+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:50:27.140775+0000","last_clean_scrub_stamp":"2023-08-09T13:50:27.140775+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T18:35:43.990648+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":146,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-08-09T13:50:29.237073+0000","last_change":"2023-08-09T13:33:56.401781+0000","last_active":"2023-08-09T13:50:29.237073+0000","last_peered":"2023-08-09T13:50:29.237073+0000","last_clean":"2023-08-09T13:50:29.237073+0000","last_became_active":"2023-08-09T13:33:56.400336+0000","last_became_peered":"2023-08-09T13:33:56.400336+0000","last_unstale":"2023-08-09T13:50:29.237073+0000","last_undegraded":"2023-08-09T13:50:29.237073+0000","last_fullsized":"2023-08-09T13:50:29.237073+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:33:54.845869+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:33:54.845869+0000","last_clean_scrub_stamp":"2023-08-09T13:33:54.845869+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-11T01:01:47.058457+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.d","version":"0'0","reported_seq":3,"reported_epoch":89,"state":"creating+activating","last_fresh":"2023-08-09T13:50:29.220533+0000","last_change":"2023-08-09T13:50:29.220533+0000","last_active":"2023-08-09T13:50:28.150437+0000","last_peered":"2023-08-09T13:50:28.150437+0000","last_clean":"2023-08-09T13:50:28.150437+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-09T13:50:29.220533+0000","last_undegraded":"2023-08-09T13:50:29.220533+0000","last_fullsized":"2023-08-09T13:50:29.220533+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:50:28.150437+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:50:28.150437+0000","last_clean_scrub_stamp":"2023-08-09T13:50:28.150437+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T17:52:40.304270+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":["2","4"],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.c","version":"0'0","reported_seq":11,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-08-09T13:50:29.216926+0000","last_change":"2023-08-09T13:50:28.203868+0000","last_active":"2023-08-09T13:50:29.216926+0000","last_peered":"2023-08-09T13:50:29.216926+0000","last_clean":"2023-08-09T13:50:29.216926+0000","last_became_active":"2023-08-09T13:50:28.203323+0000","last_became_peered":"2023-08-09T13:50:28.203323+0000","last_unstale":"2023-08-09T13:50:29.216926+0000","last_undegraded":"2023-08-09T13:50:29.216926+0000","last_fullsized":"2023-08-09T13:50:29.216926+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:50:27.140775+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:50:27.140775+0000","last_clean_scrub_stamp":"2023-08-09T13:50:27.140775+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T16:53:02.601172+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":146,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-08-09T13:50:29.230723+0000","last_change":"2023-08-09T13:33:55.886389+0000","last_active":"2023-08-09T13:50:29.230723+0000","last_peered":"2023-08-09T13:50:29.230723+0000","last_clean":"2023-08-09T13:50:29.230723+0000","last_became_active":"2023-08-09T13:33:55.885638+0000","last_became_peered":"2023-08-09T13:33:55.885638+0000","last_unstale":"2023-08-09T13:50:29.230723+0000","last_undegraded":"2023-08-09T13:50:29.230723+0000","last_fullsized":"2023-08-09T13:50:29.230723+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:33:54.845869+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:33:54.845869+0000","last_clean_scrub_stamp":"2023-08-09T13:33:54.845869+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T22:08:50.990029+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]}],"pool_stats":[{"poolid":11,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":10,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":64,"acting":64,"num_store_stats":8},{"poolid":2,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":500,"num_read_kb":924,"num_write":239,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":161,"ondisk_log_size":161,"up":2,"acting":2,"num_store_stats":2},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":8}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834575047,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":36840,"kb_used_data":1624,"kb_used_omap":1,"kb_used_meta":35198,"kb_avail":94335000,"statfs":{"total":96636764160,"available":96599040000,"internally_reserved":0,"allocated":1662976,"data_stored":1159636,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1781,"internal_metadata":36043019},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Aug 9 13:50:16 2023","interfaces":[{"interface":"back","average":{"1min":1.181,"5min":1.222,"15min":1.232},"min":{"1min":0.69499999999999995,"5min":0.63400000000000001,"15min":0.61099999999999999},"max":{"1min":1.633,"5min":1.8919999999999999,"15min":2.1280000000000001},"last":1.2569999999999999},{"interface":"front","average":{"1min":1.1819999999999999,"5min":1.232,"15min":1.24},"min":{"1min":0.72399999999999998,"5min":0.72399999999999998,"15min":0.65000000000000002},"max":{"1min":1.6100000000000001,"5min":1.839,"15min":2.0099999999999998},"last":1.1259999999999999}]},{"osd":1,"last update":"Wed Aug 9 13:50:16 2023","interfaces":[{"interface":"back","average":{"1min":1.2010000000000001,"5min":1.2450000000000001,"15min":1.256},"min":{"1min":0.72799999999999998,"5min":0.72799999999999998,"15min":0.59799999999999998},"max":{"1min":1.8,"5min":1.8,"15min":1.9470000000000001},"last":1.403},{"interface":"front","average":{"1min":1.2569999999999999,"5min":1.248,"15min":1.2869999999999999},"min":{"1min":0.83099999999999996,"5min":0.59299999999999997,"15min":0.59299999999999997},"max":{"1min":1.726,"5min":1.9410000000000001,"15min":1.9410000000000001},"last":1.327}]},{"osd":2,"last update":"Wed Aug 9 13:50:16 2023","interfaces":[{"interface":"back","average":{"1min":1.278,"5min":1.2470000000000001,"15min":1.2430000000000001},"min":{"1min":0.81000000000000005,"5min":0.65700000000000003,"15min":0.65700000000000003},"max":{"1min":1.742,"5min":2.0129999999999999,"15min":2.0129999999999999},"last":0.99399999999999999},{"interface":"front","average":{"1min":1.2889999999999999,"5min":1.323,"15min":1.3759999999999999},"min":{"1min":0.92600000000000005,"5min":0.80900000000000005,"15min":0.67800000000000005},"max":{"1min":1.7130000000000001,"5min":1.8740000000000001,"15min":2.4140000000000001},"last":1.623}]},{"osd":3,"last update":"Wed Aug 9 13:50:16 2023","interfaces":[{"interface":"back","average":{"1min":1.2529999999999999,"5min":1.3180000000000001,"15min":1.331},"min":{"1min":0.98799999999999999,"5min":0.86799999999999999,"15min":0.77500000000000002},"max":{"1min":1.7809999999999999,"5min":1.97,"15min":1.97},"last":1.3680000000000001},{"interface":"front","average":{"1min":1.3740000000000001,"5min":1.419,"15min":1.4159999999999999},"min":{"1min":0.93899999999999995,"5min":0.88300000000000001,"15min":0.82099999999999995},"max":{"1min":1.8160000000000001,"5min":2.0790000000000002,"15min":2.2480000000000002},"last":1.526}]},{"osd":4,"last update":"Wed Aug 9 13:50:16 2023","interfaces":[{"interface":"back","average":{"1min":1.343,"5min":1.2649999999999999,"15min":1.2769999999999999},"min":{"1min":0.86399999999999999,"5min":0.59499999999999997,"15min":0.59499999999999997},"max":{"1min":1.772,"5min":2.032,"15min":2.343},"last":0.82499999999999996},{"interface":"front","average":{"1min":1.296,"5min":1.3160000000000001,"15min":1.339},"min":{"1min":0.93600000000000005,"5min":0.77100000000000002,"15min":0.56799999999999995},"max":{"1min":1.6739999999999999,"5min":2.1419999999999999,"15min":2.1419999999999999},"last":1.2010000000000001}]},{"osd":5,"last update":"Wed Aug 9 13:50:16 2023","interfaces":[{"interface":"back","average":{"1min":1.4430000000000001,"5min":1.4350000000000001,"15min":1.419},"min":{"1min":1.048,"5min":0.88600000000000001,"15min":0.71499999999999997},"max":{"1min":1.9039999999999999,"5min":2.1480000000000001,"15min":2.6749999999999998},"last":1.6879999999999999},{"interface":"front","average":{"1min":1.4810000000000001,"5min":1.4319999999999999,"15min":1.4510000000000001},"min":{"1min":0.95999999999999996,"5min":0.85699999999999998,"15min":0.69199999999999995},"max":{"1min":1.877,"5min":2.113,"15min":2.9569999999999999},"last":1.4510000000000001}]},{"osd":6,"last update":"Wed Aug 9 13:50:16 2023","interfaces":[{"interface":"back","average":{"1min":1.6020000000000001,"5min":1.518,"15min":1.5489999999999999},"min":{"1min":1.2,"5min":0.85499999999999998,"15min":0.85499999999999998},"max":{"1min":1.8879999999999999,"5min":2.2069999999999999,"15min":2.8149999999999999},"last":1.575},{"interface":"front","average":{"1min":1.4139999999999999,"5min":1.4570000000000001,"15min":1.474},"min":{"1min":1.111,"5min":0.90600000000000003,"15min":0.60799999999999998},"max":{"1min":1.9419999999999999,"5min":2.1320000000000001,"15min":2.2530000000000001},"last":1.4910000000000001}]}]},{"osd":6,"up_from":13,"seq":55834575047,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32156,"kb_used_data":1040,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":94339684,"statfs":{"total":96636764160,"available":96603836416,"internally_reserved":0,"allocated":1064960,"data_stored":567907,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1781,"internal_metadata":31848715},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Aug 9 13:50:20 2023","interfaces":[{"interface":"back","average":{"1min":1.1160000000000001,"5min":1.143,"15min":1.161},"min":{"1min":0.75,"5min":0.72499999999999998,"15min":0.56599999999999995},"max":{"1min":1.5489999999999999,"5min":1.633,"15min":1.9870000000000001},"last":1.359},{"interface":"front","average":{"1min":1.21,"5min":1.532,"15min":1.3080000000000001},"min":{"1min":0.79300000000000004,"5min":0.79300000000000004,"15min":0.76000000000000001},"max":{"1min":1.6519999999999999,"5min":28.036000000000001,"15min":28.036000000000001},"last":1.4550000000000001}]},{"osd":1,"last update":"Wed Aug 9 13:50:20 2023","interfaces":[{"interface":"back","average":{"1min":1.147,"5min":1.272,"15min":1.226},"min":{"1min":0.70399999999999996,"5min":0.70399999999999996,"15min":0.65500000000000003},"max":{"1min":1.659,"5min":7.742,"15min":7.742},"last":1.256},{"interface":"front","average":{"1min":1.252,"5min":1.2849999999999999,"15min":1.276},"min":{"1min":0.84199999999999997,"5min":0.84199999999999997,"15min":0.68100000000000005},"max":{"1min":1.8899999999999999,"5min":1.8899999999999999,"15min":1.8899999999999999},"last":0.89100000000000001}]},{"osd":2,"last update":"Wed Aug 9 13:50:20 2023","interfaces":[{"interface":"back","average":{"1min":1.2989999999999999,"5min":1.3480000000000001,"15min":1.3220000000000001},"min":{"1min":0.877,"5min":0.81999999999999995,"15min":0.68700000000000006},"max":{"1min":1.73,"5min":4.0640000000000001,"15min":4.0640000000000001},"last":1.151},{"interface":"front","average":{"1min":1.244,"5min":1.4430000000000001,"15min":1.3460000000000001},"min":{"1min":0.76800000000000002,"5min":0.70999999999999996,"15min":0.70999999999999996},"max":{"1min":2.0390000000000001,"5min":13.154999999999999,"15min":13.154999999999999},"last":0.95399999999999996}]},{"osd":3,"last update":"Wed Aug 9 13:50:20 2023","interfaces":[{"interface":"back","average":{"1min":1.3939999999999999,"5min":1.52,"15min":1.419},"min":{"1min":0.73099999999999998,"5min":0.73099999999999998,"15min":0.73099999999999998},"max":{"1min":2.2519999999999998,"5min":13.349,"15min":13.349},"last":1.298},{"interface":"front","average":{"1min":1.258,"5min":1.3620000000000001,"15min":1.325},"min":{"1min":0.88400000000000001,"5min":0.76400000000000001,"15min":0.74299999999999999},"max":{"1min":1.8560000000000001,"5min":7.9029999999999996,"15min":7.9029999999999996},"last":1.397}]},{"osd":4,"last update":"Wed Aug 9 13:50:20 2023","interfaces":[{"interface":"back","average":{"1min":1.302,"5min":1.335,"15min":1.28},"min":{"1min":0.90700000000000003,"5min":0.78500000000000003,"15min":0.64300000000000002},"max":{"1min":1.992,"5min":5.2290000000000001,"15min":5.2290000000000001},"last":1.486},{"interface":"front","average":{"1min":1.4019999999999999,"5min":1.498,"15min":1.389},"min":{"1min":0.88900000000000001,"5min":0.78700000000000003,"15min":0.69099999999999995},"max":{"1min":1.734,"5min":12.1,"15min":12.1},"last":1.0860000000000001}]},{"osd":5,"last update":"Wed Aug 9 13:50:20 2023","interfaces":[{"interface":"back","average":{"1min":1.3169999999999999,"5min":1.4399999999999999,"15min":1.357},"min":{"1min":0.93999999999999995,"5min":0.83799999999999997,"15min":0.65400000000000003},"max":{"1min":2.1139999999999999,"5min":8.1899999999999995,"15min":8.1899999999999995},"last":1.0289999999999999},{"interface":"front","average":{"1min":1.5649999999999999,"5min":1.6639999999999999,"15min":1.589},"min":{"1min":1.2010000000000001,"5min":1.2010000000000001,"15min":0.83799999999999997},"max":{"1min":2.2000000000000002,"5min":8.8889999999999993,"15min":8.8889999999999993},"last":1.4299999999999999}]},{"osd":7,"last update":"Wed Aug 9 13:50:20 2023","interfaces":[{"interface":"back","average":{"1min":1.577,"5min":1.6970000000000001,"15min":1.5880000000000001},"min":{"1min":1.1850000000000001,"5min":1.0029999999999999,"15min":0.62},"max":{"1min":1.8240000000000001,"5min":9.3320000000000007,"15min":9.3320000000000007},"last":1.5289999999999999},{"interface":"front","average":{"1min":1.4410000000000001,"5min":1.6060000000000001,"15min":1.548},"min":{"1min":0.93899999999999995,"5min":0.88500000000000001,"15min":0.88500000000000001},"max":{"1min":1.9470000000000001,"5min":8.0399999999999991,"15min":8.0399999999999991},"last":1.5029999999999999}]}]},{"osd":5,"up_from":13,"seq":55834575047,"num_pgs":16,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":36260,"kb_used_data":1044,"kb_used_omap":1,"kb_used_meta":35198,"kb_avail":94335580,"statfs":{"total":96636764160,"available":96599633920,"internally_reserved":0,"allocated":1069056,"data_stored":569268,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1781,"internal_metadata":36043019},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Aug 9 13:50:19 2023","interfaces":[{"interface":"back","average":{"1min":1.2509999999999999,"5min":1.1759999999999999,"15min":1.179},"min":{"1min":0.63500000000000001,"5min":0.63500000000000001,"15min":0.63500000000000001},"max":{"1min":1.518,"5min":2.1869999999999998,"15min":2.1869999999999998},"last":1.1020000000000001},{"interface":"front","average":{"1min":1.1719999999999999,"5min":1.2,"15min":1.198},"min":{"1min":0.68600000000000005,"5min":0.68600000000000005,"15min":0.64200000000000002},"max":{"1min":1.579,"5min":1.6619999999999999,"15min":1.7989999999999999},"last":1.1950000000000001}]},{"osd":1,"last update":"Wed Aug 9 13:50:19 2023","interfaces":[{"interface":"back","average":{"1min":1.165,"5min":1.1990000000000001,"15min":1.2190000000000001},"min":{"1min":0.81599999999999995,"5min":0.73099999999999998,"15min":0.64100000000000001},"max":{"1min":1.631,"5min":1.6739999999999999,"15min":1.736},"last":1.0049999999999999},{"interface":"front","average":{"1min":1.264,"5min":1.296,"15min":1.2769999999999999},"min":{"1min":0.93300000000000005,"5min":0.66900000000000004,"15min":0.66900000000000004},"max":{"1min":1.6699999999999999,"5min":1.887,"15min":1.887},"last":0.84999999999999998}]},{"osd":2,"last update":"Wed Aug 9 13:50:19 2023","interfaces":[{"interface":"back","average":{"1min":1.2410000000000001,"5min":1.2569999999999999,"15min":1.2849999999999999},"min":{"1min":0.98199999999999998,"5min":0.68999999999999995,"15min":0.61599999999999999},"max":{"1min":1.5760000000000001,"5min":1.756,"15min":1.899},"last":1.4670000000000001},{"interface":"front","average":{"1min":1.3049999999999999,"5min":1.302,"15min":1.3149999999999999},"min":{"1min":0.78400000000000003,"5min":0.78400000000000003,"15min":0.66800000000000004},"max":{"1min":1.6599999999999999,"5min":1.7190000000000001,"15min":1.7969999999999999},"last":1.1419999999999999}]},{"osd":3,"last update":"Wed Aug 9 13:50:19 2023","interfaces":[{"interface":"back","average":{"1min":1.365,"5min":1.3720000000000001,"15min":1.3600000000000001},"min":{"1min":0.95399999999999996,"5min":0.95399999999999996,"15min":0.59799999999999998},"max":{"1min":1.6890000000000001,"5min":2.1840000000000002,"15min":2.2690000000000001},"last":1.3160000000000001},{"interface":"front","average":{"1min":1.4259999999999999,"5min":1.4379999999999999,"15min":1.4139999999999999},"min":{"1min":1.087,"5min":0.94899999999999995,"15min":0.70999999999999996},"max":{"1min":1.73,"5min":2.3780000000000001,"15min":2.3780000000000001},"last":1.3600000000000001}]},{"osd":4,"last update":"Wed Aug 9 13:50:19 2023","interfaces":[{"interface":"back","average":{"1min":1.262,"5min":1.2729999999999999,"15min":1.2889999999999999},"min":{"1min":0.92700000000000005,"5min":0.84699999999999998,"15min":0.56599999999999995},"max":{"1min":1.673,"5min":2.0800000000000001,"15min":2.121},"last":1.2789999999999999},{"interface":"front","average":{"1min":1.3919999999999999,"5min":1.3400000000000001,"15min":1.3779999999999999},"min":{"1min":0.81999999999999995,"5min":0.76100000000000001,"15min":0.58599999999999997},"max":{"1min":1.944,"5min":2.1230000000000002,"15min":2.3679999999999999},"last":1.403}]},{"osd":6,"last update":"Wed Aug 9 13:50:19 2023","interfaces":[{"interface":"back","average":{"1min":1.4299999999999999,"5min":1.3520000000000001,"15min":1.391},"min":{"1min":0.99099999999999999,"5min":0.90300000000000002,"15min":0.66700000000000004},"max":{"1min":1.9810000000000001,"5min":2.306,"15min":2.306},"last":1.5609999999999999},{"interface":"front","average":{"1min":1.49,"5min":1.4399999999999999,"15min":1.4710000000000001},"min":{"1min":1.1220000000000001,"5min":1.034,"15min":0.71199999999999997},"max":{"1min":1.827,"5min":2.2320000000000002,"15min":2.2320000000000002},"last":1.343}]},{"osd":7,"last update":"Wed Aug 9 13:50:19 2023","interfaces":[{"interface":"back","average":{"1min":1.577,"5min":1.5580000000000001,"15min":1.5529999999999999},"min":{"1min":1.1160000000000001,"5min":1.0589999999999999,"15min":0.70899999999999996},"max":{"1min":1.9610000000000001,"5min":2.452,"15min":2.4729999999999999},"last":1.5980000000000001},{"interface":"front","average":{"1min":1.611,"5min":1.585,"15min":1.6080000000000001},"min":{"1min":1.2110000000000001,"5min":1.163,"15min":0.77600000000000002},"max":{"1min":2.097,"5min":2.1070000000000002,"15min":2.3820000000000001},"last":1.6499999999999999}]}]},{"osd":4,"up_from":13,"seq":55834575047,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32160,"kb_used_data":1044,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":94339680,"statfs":{"total":96636764160,"available":96603832320,"internally_reserved":0,"allocated":1069056,"data_stored":569268,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1781,"internal_metadata":31848715},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Aug 9 13:50:21 2023","interfaces":[{"interface":"back","average":{"1min":1.2330000000000001,"5min":1.2390000000000001,"15min":1.2030000000000001},"min":{"1min":0.875,"5min":0.70699999999999996,"15min":0.67700000000000005},"max":{"1min":1.5940000000000001,"5min":2.524,"15min":2.524},"last":1.179},{"interface":"front","average":{"1min":1.206,"5min":1.236,"15min":1.2569999999999999},"min":{"1min":0.62,"5min":0.62,"15min":0.62},"max":{"1min":1.5980000000000001,"5min":1.863,"15min":1.863},"last":1.468}]},{"osd":1,"last update":"Wed Aug 9 13:50:21 2023","interfaces":[{"interface":"back","average":{"1min":1.244,"5min":1.3169999999999999,"15min":1.2929999999999999},"min":{"1min":0.70999999999999996,"5min":0.67800000000000005,"15min":0.67800000000000005},"max":{"1min":1.48,"5min":1.8480000000000001,"15min":1.8480000000000001},"last":1.026},{"interface":"front","average":{"1min":1.242,"5min":1.2649999999999999,"15min":1.248},"min":{"1min":0.81799999999999995,"5min":0.81799999999999995,"15min":0.64900000000000002},"max":{"1min":1.9330000000000001,"5min":1.9330000000000001,"15min":1.9330000000000001},"last":0.98499999999999999}]},{"osd":2,"last update":"Wed Aug 9 13:50:21 2023","interfaces":[{"interface":"back","average":{"1min":1.306,"5min":1.2869999999999999,"15min":1.2809999999999999},"min":{"1min":1.022,"5min":0.621,"15min":0.621},"max":{"1min":1.619,"5min":2.6429999999999998,"15min":2.6429999999999998},"last":0.69999999999999996},{"interface":"front","average":{"1min":1.343,"5min":1.3720000000000001,"15min":1.3660000000000001},"min":{"1min":0.88300000000000001,"5min":0.88300000000000001,"15min":0.74299999999999999},"max":{"1min":1.7569999999999999,"5min":1.7569999999999999,"15min":1.8149999999999999},"last":0.89000000000000001}]},{"osd":3,"last update":"Wed Aug 9 13:50:21 2023","interfaces":[{"interface":"back","average":{"1min":1.377,"5min":1.3300000000000001,"15min":1.298},"min":{"1min":1.083,"5min":0.86399999999999999,"15min":0.84699999999999998},"max":{"1min":1.6739999999999999,"5min":2.6099999999999999,"15min":2.6099999999999999},"last":1.212},{"interface":"front","average":{"1min":1.4099999999999999,"5min":1.3979999999999999,"15min":1.3799999999999999},"min":{"1min":0.94799999999999995,"5min":0.88800000000000001,"15min":0.63100000000000001},"max":{"1min":1.6890000000000001,"5min":2.7629999999999999,"15min":2.7629999999999999},"last":0.95599999999999996}]},{"osd":5,"last update":"Wed Aug 9 13:50:21 2023","interfaces":[{"interface":"back","average":{"1min":1.288,"5min":1.3400000000000001,"15min":1.3300000000000001},"min":{"1min":0.876,"5min":0.73199999999999998,"15min":0.63200000000000001},"max":{"1min":1.734,"5min":1.8,"15min":2.0099999999999998},"last":1.343},{"interface":"front","average":{"1min":1.2509999999999999,"5min":1.323,"15min":1.3180000000000001},"min":{"1min":0.92000000000000004,"5min":0.72399999999999998,"15min":0.626},"max":{"1min":1.71,"5min":2.7090000000000001,"15min":2.7090000000000001},"last":1.421}]},{"osd":6,"last update":"Wed Aug 9 13:50:21 2023","interfaces":[{"interface":"back","average":{"1min":1.379,"5min":1.3160000000000001,"15min":1.3360000000000001},"min":{"1min":0.94699999999999995,"5min":0.74399999999999999,"15min":0.59099999999999997},"max":{"1min":1.7509999999999999,"5min":2.8559999999999999,"15min":2.8559999999999999},"last":1.083},{"interface":"front","average":{"1min":1.4430000000000001,"5min":1.518,"15min":1.504},"min":{"1min":1.0760000000000001,"5min":0.97799999999999998,"15min":0.76400000000000001},"max":{"1min":1.669,"5min":2.2719999999999998,"15min":2.2719999999999998},"last":1.5409999999999999}]},{"osd":7,"last update":"Wed Aug 9 13:50:21 2023","interfaces":[{"interface":"back","average":{"1min":1.46,"5min":1.4830000000000001,"15min":1.474},"min":{"1min":1.1499999999999999,"5min":0.85999999999999999,"15min":0.73899999999999999},"max":{"1min":1.8460000000000001,"5min":2.8050000000000002,"15min":2.8050000000000002},"last":1.1100000000000001},{"interface":"front","average":{"1min":1.474,"5min":1.522,"15min":1.514},"min":{"1min":1.236,"5min":0.97199999999999998,"15min":0.72899999999999998},"max":{"1min":1.754,"5min":3.415,"15min":3.415},"last":1.3959999999999999}]}]},{"osd":3,"up_from":13,"seq":55834575048,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":36260,"kb_used_data":1044,"kb_used_omap":1,"kb_used_meta":35198,"kb_avail":94335580,"statfs":{"total":96636764160,"available":96599633920,"internally_reserved":0,"allocated":1069056,"data_stored":569268,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1781,"internal_metadata":36043019},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Aug 9 13:50:25 2023","interfaces":[{"interface":"back","average":{"1min":0.84599999999999997,"5min":0.88300000000000001,"15min":0.91600000000000004},"min":{"1min":0.43099999999999999,"5min":0.27200000000000002,"15min":0.27200000000000002},"max":{"1min":1.3939999999999999,"5min":1.8500000000000001,"15min":1.968},"last":0.65900000000000003},{"interface":"front","average":{"1min":1.071,"5min":1.0920000000000001,"15min":1.1060000000000001},"min":{"1min":0.48199999999999998,"5min":0.36399999999999999,"15min":0.36399999999999999},"max":{"1min":1.6910000000000001,"5min":1.6910000000000001,"15min":2.2360000000000002},"last":0.75600000000000001}]},{"osd":1,"last update":"Wed Aug 9 13:50:25 2023","interfaces":[{"interface":"back","average":{"1min":1.042,"5min":1.0089999999999999,"15min":1.012},"min":{"1min":0.59899999999999998,"5min":0.496,"15min":0.438},"max":{"1min":1.518,"5min":1.7110000000000001,"15min":2.5649999999999999},"last":0.83999999999999997},{"interface":"front","average":{"1min":1.091,"5min":1.1279999999999999,"15min":1.1639999999999999},"min":{"1min":0.51800000000000002,"5min":0.51800000000000002,"15min":0.42099999999999999},"max":{"1min":1.6839999999999999,"5min":1.6839999999999999,"15min":2.6760000000000002},"last":1.202}]},{"osd":2,"last update":"Wed Aug 9 13:50:25 2023","interfaces":[{"interface":"back","average":{"1min":1.121,"5min":1.177,"15min":1.1930000000000001},"min":{"1min":0.53600000000000003,"5min":0.53600000000000003,"15min":0.53500000000000003},"max":{"1min":1.6240000000000001,"5min":1.913,"15min":1.923},"last":1.1399999999999999},{"interface":"front","average":{"1min":1.1899999999999999,"5min":1.2929999999999999,"15min":1.3080000000000001},"min":{"1min":0.625,"5min":0.52300000000000002,"15min":0.52000000000000002},"max":{"1min":1.9399999999999999,"5min":2.0259999999999998,"15min":2.0259999999999998},"last":1.0960000000000001}]},{"osd":4,"last update":"Wed Aug 9 13:50:25 2023","interfaces":[{"interface":"back","average":{"1min":1.232,"5min":1.258,"15min":1.282},"min":{"1min":0.91500000000000004,"5min":0.57899999999999996,"15min":0.57899999999999996},"max":{"1min":1.742,"5min":1.8160000000000001,"15min":1.8160000000000001},"last":1.6819999999999999},{"interface":"front","average":{"1min":1.349,"5min":1.397,"15min":1.3919999999999999},"min":{"1min":0.67600000000000005,"5min":0.628,"15min":0.628},"max":{"1min":1.8149999999999999,"5min":2.2040000000000002,"15min":2.2370000000000001},"last":1.3080000000000001}]},{"osd":5,"last update":"Wed Aug 9 13:50:25 2023","interfaces":[{"interface":"back","average":{"1min":1.456,"5min":1.468,"15min":1.474},"min":{"1min":0.67600000000000005,"5min":0.67600000000000005,"15min":0.67200000000000004},"max":{"1min":1.855,"5min":2.254,"15min":2.254},"last":1.3959999999999999},{"interface":"front","average":{"1min":1.4850000000000001,"5min":1.4910000000000001,"15min":1.49},"min":{"1min":0.77600000000000002,"5min":0.65900000000000003,"15min":0.65900000000000003},"max":{"1min":2.3069999999999999,"5min":2.3069999999999999,"15min":2.3069999999999999},"last":1.3260000000000001}]},{"osd":6,"last update":"Wed Aug 9 13:50:25 2023","interfaces":[{"interface":"back","average":{"1min":1.3759999999999999,"5min":1.4370000000000001,"15min":1.4590000000000001},"min":{"1min":0.878,"5min":0.68200000000000005,"15min":0.68200000000000005},"max":{"1min":1.8759999999999999,"5min":2.1829999999999998,"15min":2.242},"last":1.4670000000000001},{"interface":"front","average":{"1min":1.466,"5min":1.5580000000000001,"15min":1.5529999999999999},"min":{"1min":1.0600000000000001,"5min":0.71099999999999997,"15min":0.68600000000000005},"max":{"1min":1.825,"5min":2.3420000000000001,"15min":2.3420000000000001},"last":1.252}]},{"osd":7,"last update":"Wed Aug 9 13:50:25 2023","interfaces":[{"interface":"back","average":{"1min":1.601,"5min":1.627,"15min":1.6160000000000001},"min":{"1min":1.115,"5min":0.69399999999999995,"15min":0.69399999999999995},"max":{"1min":2.46,"5min":2.46,"15min":2.46},"last":1.4279999999999999},{"interface":"front","average":{"1min":1.4910000000000001,"5min":1.546,"15min":1.5600000000000001},"min":{"1min":0.78200000000000003,"5min":0.64200000000000002,"15min":0.61599999999999999},"max":{"1min":2.0579999999999998,"5min":2.359,"15min":2.484},"last":1.571}]}]},{"osd":2,"up_from":13,"seq":55834575047,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":36256,"kb_used_data":1044,"kb_used_omap":1,"kb_used_meta":35198,"kb_avail":94335584,"statfs":{"total":96636764160,"available":96599638016,"internally_reserved":0,"allocated":1069056,"data_stored":569268,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1781,"internal_metadata":36043019},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Aug 9 13:50:17 2023","interfaces":[{"interface":"back","average":{"1min":0.96999999999999997,"5min":0.95299999999999996,"15min":0.95499999999999996},"min":{"1min":0.47799999999999998,"5min":0.36499999999999999,"15min":0.36499999999999999},"max":{"1min":1.5760000000000001,"5min":1.6180000000000001,"15min":1.8380000000000001},"last":1.484},{"interface":"front","average":{"1min":0.97099999999999997,"5min":0.998,"15min":1.0049999999999999},"min":{"1min":0.45300000000000001,"5min":0.374,"15min":0.374},"max":{"1min":1.4199999999999999,"5min":2.3929999999999998,"15min":2.3929999999999998},"last":0.80200000000000005}]},{"osd":1,"last update":"Wed Aug 9 13:50:17 2023","interfaces":[{"interface":"back","average":{"1min":1.2529999999999999,"5min":1.1739999999999999,"15min":1.1870000000000001},"min":{"1min":0.61299999999999999,"5min":0.55200000000000005,"15min":0.55200000000000005},"max":{"1min":1.6870000000000001,"5min":1.77,"15min":1.8420000000000001},"last":1.615},{"interface":"front","average":{"1min":1.216,"5min":1.2749999999999999,"15min":1.335},"min":{"1min":0.52500000000000002,"5min":0.52500000000000002,"15min":0.52500000000000002},"max":{"1min":1.7390000000000001,"5min":2.4870000000000001,"15min":2.4870000000000001},"last":1.03}]},{"osd":3,"last update":"Wed Aug 9 13:50:17 2023","interfaces":[{"interface":"back","average":{"1min":0.98099999999999998,"5min":1.006,"15min":1.0469999999999999},"min":{"1min":0.44800000000000001,"5min":0.38900000000000001,"15min":0.38900000000000001},"max":{"1min":1.3109999999999999,"5min":1.708,"15min":2.0409999999999999},"last":0.63600000000000001},{"interface":"front","average":{"1min":1.2110000000000001,"5min":1.304,"15min":1.3089999999999999},"min":{"1min":0.55500000000000005,"5min":0.46200000000000002,"15min":0.46200000000000002},"max":{"1min":1.8009999999999999,"5min":8.8130000000000006,"15min":8.8130000000000006},"last":1.2529999999999999}]},{"osd":4,"last update":"Wed Aug 9 13:50:17 2023","interfaces":[{"interface":"back","average":{"1min":1.415,"5min":1.4179999999999999,"15min":1.4510000000000001},"min":{"1min":0.83999999999999997,"5min":0.628,"15min":0.628},"max":{"1min":1.958,"5min":2.089,"15min":2.089},"last":1.556},{"interface":"front","average":{"1min":1.462,"5min":1.359,"15min":1.3979999999999999},"min":{"1min":0.73799999999999999,"5min":0.68600000000000005,"15min":0.68600000000000005},"max":{"1min":2.472,"5min":2.472,"15min":2.472},"last":1.151}]},{"osd":5,"last update":"Wed Aug 9 13:50:17 2023","interfaces":[{"interface":"back","average":{"1min":1.5189999999999999,"5min":1.4379999999999999,"15min":1.458},"min":{"1min":0.76600000000000001,"5min":0.59999999999999998,"15min":0.59999999999999998},"max":{"1min":2.27,"5min":2.5840000000000001,"15min":2.5840000000000001},"last":1.3240000000000001},{"interface":"front","average":{"1min":1.304,"5min":1.3540000000000001,"15min":1.349},"min":{"1min":0.77100000000000002,"5min":0.56799999999999995,"15min":0.56799999999999995},"max":{"1min":1.8859999999999999,"5min":3.5659999999999998,"15min":3.5659999999999998},"last":1.379}]},{"osd":6,"last update":"Wed Aug 9 13:50:17 2023","interfaces":[{"interface":"back","average":{"1min":1.4019999999999999,"5min":1.425,"15min":1.4259999999999999},"min":{"1min":0.79800000000000004,"5min":0.73799999999999999,"15min":0.73799999999999999},"max":{"1min":2.222,"5min":3.5099999999999998,"15min":3.5099999999999998},"last":1.079},{"interface":"front","average":{"1min":1.4790000000000001,"5min":1.4990000000000001,"15min":1.5409999999999999},"min":{"1min":0.82199999999999995,"5min":0.66300000000000003,"15min":0.66300000000000003},"max":{"1min":2.3769999999999998,"5min":2.3769999999999998,"15min":2.3769999999999998},"last":1.6930000000000001}]},{"osd":7,"last update":"Wed Aug 9 13:50:17 2023","interfaces":[{"interface":"back","average":{"1min":1.54,"5min":1.51,"15min":1.548},"min":{"1min":0.78800000000000003,"5min":0.64600000000000002,"15min":0.64600000000000002},"max":{"1min":2.1970000000000001,"5min":2.5249999999999999,"15min":2.5249999999999999},"last":1.4379999999999999},{"interface":"front","average":{"1min":1.4970000000000001,"5min":1.52,"15min":1.51},"min":{"1min":0.99199999999999999,"5min":0.70499999999999996,"15min":0.70499999999999996},"max":{"1min":2.089,"5min":3.609,"15min":3.609},"last":1.214}]}]},{"osd":1,"up_from":13,"seq":55834575047,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28644,"kb_used_data":1624,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343196,"statfs":{"total":96636764160,"available":96607432704,"internally_reserved":0,"allocated":1662976,"data_stored":1159636,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1781,"internal_metadata":27654411},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Aug 9 13:49:27 2023","interfaces":[{"interface":"back","average":{"1min":0.98399999999999999,"5min":0.90500000000000003,"15min":0.93899999999999995},"min":{"1min":0.64500000000000002,"5min":0.50900000000000001,"15min":0.32400000000000001},"max":{"1min":1.575,"5min":1.575,"15min":2.7080000000000002},"last":1.272},{"interface":"front","average":{"1min":1.1719999999999999,"5min":1.093,"15min":1.1000000000000001},"min":{"1min":0.74299999999999999,"5min":0.432,"15min":0.42499999999999999},"max":{"1min":2.468,"5min":2.468,"15min":2.6349999999999998},"last":1.7190000000000001}]},{"osd":2,"last update":"Wed Aug 9 13:49:27 2023","interfaces":[{"interface":"back","average":{"1min":1.0249999999999999,"5min":0.97699999999999998,"15min":0.99299999999999999},"min":{"1min":0.623,"5min":0.41099999999999998,"15min":0.41099999999999998},"max":{"1min":1.3160000000000001,"5min":1.907,"15min":1.907},"last":1.087},{"interface":"front","average":{"1min":1.212,"5min":1.1870000000000001,"15min":1.1910000000000001},"min":{"1min":0.74299999999999999,"5min":0.54400000000000004,"15min":0.46500000000000002},"max":{"1min":2.1280000000000001,"5min":2.1280000000000001,"15min":2.2330000000000001},"last":1.4970000000000001}]},{"osd":3,"last update":"Wed Aug 9 13:49:27 2023","interfaces":[{"interface":"back","average":{"1min":1.3320000000000001,"5min":1.2949999999999999,"15min":1.325},"min":{"1min":0.90500000000000003,"5min":0.54400000000000004,"15min":0.432},"max":{"1min":2.0270000000000001,"5min":2.0270000000000001,"15min":2.3210000000000002},"last":1.6759999999999999},{"interface":"front","average":{"1min":1.27,"5min":1.258,"15min":1.252},"min":{"1min":0.96699999999999997,"5min":0.49299999999999999,"15min":0.44400000000000001},"max":{"1min":2.3540000000000001,"5min":2.3540000000000001,"15min":3.3069999999999999},"last":1.2030000000000001}]},{"osd":4,"last update":"Wed Aug 9 13:49:27 2023","interfaces":[{"interface":"back","average":{"1min":1.4379999999999999,"5min":1.4330000000000001,"15min":1.4119999999999999},"min":{"1min":1.1899999999999999,"5min":0.79800000000000004,"15min":0.58099999999999996},"max":{"1min":1.966,"5min":1.97,"15min":2.504},"last":1.5940000000000001},{"interface":"front","average":{"1min":1.3640000000000001,"5min":1.3360000000000001,"15min":1.3400000000000001},"min":{"1min":0.871,"5min":0.70099999999999996,"15min":0.70099999999999996},"max":{"1min":1.6950000000000001,"5min":1.901,"15min":2.5099999999999998},"last":1.425}]},{"osd":5,"last update":"Wed Aug 9 13:49:27 2023","interfaces":[{"interface":"back","average":{"1min":1.3149999999999999,"5min":1.3620000000000001,"15min":1.377},"min":{"1min":1.0389999999999999,"5min":0.76900000000000002,"15min":0.76900000000000002},"max":{"1min":1.55,"5min":2.294,"15min":2.294},"last":1.55},{"interface":"front","average":{"1min":1.456,"5min":1.502,"15min":1.538},"min":{"1min":0.98599999999999999,"5min":0.91000000000000003,"15min":0.77700000000000002},"max":{"1min":1.738,"5min":2.1389999999999998,"15min":4.9560000000000004},"last":1.837}]},{"osd":6,"last update":"Wed Aug 9 13:49:27 2023","interfaces":[{"interface":"back","average":{"1min":1.5329999999999999,"5min":1.5109999999999999,"15min":1.5209999999999999},"min":{"1min":1.0740000000000001,"5min":0.77500000000000002,"15min":0.67400000000000004},"max":{"1min":2.2080000000000002,"5min":2.2080000000000002,"15min":3.0529999999999999},"last":1.925},{"interface":"front","average":{"1min":1.4510000000000001,"5min":1.478,"15min":1.508},"min":{"1min":0.94199999999999995,"5min":0.82499999999999996,"15min":0.79800000000000004},"max":{"1min":1.8160000000000001,"5min":2.1040000000000001,"15min":2.7330000000000001},"last":1.774}]},{"osd":7,"last update":"Wed Aug 9 13:49:27 2023","interfaces":[{"interface":"back","average":{"1min":1.5629999999999999,"5min":1.601,"15min":1.637},"min":{"1min":0.92000000000000004,"5min":0.86899999999999999,"15min":0.77100000000000002},"max":{"1min":1.9339999999999999,"5min":2.25,"15min":5.1360000000000001},"last":2.4169999999999998},{"interface":"front","average":{"1min":1.5600000000000001,"5min":1.5760000000000001,"15min":1.605},"min":{"1min":1.008,"5min":0.91300000000000003,"15min":0.75900000000000001},"max":{"1min":2.0680000000000001,"5min":2.1040000000000001,"15min":3.194},"last":2.036}]}]},{"osd":0,"up_from":13,"seq":55834575047,"num_pgs":7,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":40348,"kb_used_data":1040,"kb_used_omap":1,"kb_used_meta":39294,"kb_avail":94331492,"statfs":{"total":96636764160,"available":96595447808,"internally_reserved":0,"allocated":1064960,"data_stored":567907,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1781,"internal_metadata":40237323},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Wed Aug 9 13:50:27 2023","interfaces":[{"interface":"back","average":{"1min":0.91800000000000004,"5min":0.92200000000000004,"15min":0.91600000000000004},"min":{"1min":0.32500000000000001,"5min":0.32500000000000001,"15min":0.32500000000000001},"max":{"1min":1.5820000000000001,"5min":1.756,"15min":1.756},"last":0.78500000000000003},{"interface":"front","average":{"1min":1.0680000000000001,"5min":1.1020000000000001,"15min":1.113},"min":{"1min":0.47099999999999997,"5min":0.45300000000000001,"15min":0.45300000000000001},"max":{"1min":1.9019999999999999,"5min":1.9019999999999999,"15min":1.9019999999999999},"last":0.85499999999999998}]},{"osd":2,"last update":"Wed Aug 9 13:50:27 2023","interfaces":[{"interface":"back","average":{"1min":1.048,"5min":1.1000000000000001,"15min":1.135},"min":{"1min":0.53800000000000003,"5min":0.51400000000000001,"15min":0.44800000000000001},"max":{"1min":1.702,"5min":1.776,"15min":1.889},"last":1.28},{"interface":"front","average":{"1min":1.1819999999999999,"5min":1.149,"15min":1.179},"min":{"1min":0.58499999999999996,"5min":0.56399999999999995,"15min":0.52700000000000002},"max":{"1min":2.0129999999999999,"5min":2.0129999999999999,"15min":2.2519999999999998},"last":1.4370000000000001}]},{"osd":3,"last update":"Wed Aug 9 13:50:27 2023","interfaces":[{"interface":"back","average":{"1min":0.98499999999999999,"5min":1.0509999999999999,"15min":1.105},"min":{"1min":0.54900000000000004,"5min":0.46300000000000002,"15min":0.46300000000000002},"max":{"1min":1.345,"5min":1.823,"15min":2.5190000000000001},"last":1.024},{"interface":"front","average":{"1min":1.272,"5min":1.2629999999999999,"15min":1.3140000000000001},"min":{"1min":0.56399999999999995,"5min":0.54700000000000004,"15min":0.54700000000000004},"max":{"1min":2.347,"5min":2.347,"15min":2.4649999999999999},"last":1.1459999999999999}]},{"osd":4,"last update":"Wed Aug 9 13:50:27 2023","interfaces":[{"interface":"back","average":{"1min":1.302,"5min":1.3129999999999999,"15min":1.347},"min":{"1min":0.84699999999999998,"5min":0.84699999999999998,"15min":0.74299999999999999},"max":{"1min":1.9630000000000001,"5min":2.2309999999999999,"15min":2.2309999999999999},"last":1.345},{"interface":"front","average":{"1min":1.3999999999999999,"5min":1.4390000000000001,"15min":1.452},"min":{"1min":0.82699999999999996,"5min":0.77600000000000002,"15min":0.74099999999999999},"max":{"1min":1.8440000000000001,"5min":2.5390000000000001,"15min":2.5390000000000001},"last":1.554}]},{"osd":5,"last update":"Wed Aug 9 13:50:27 2023","interfaces":[{"interface":"back","average":{"1min":1.387,"5min":1.4219999999999999,"15min":1.4650000000000001},"min":{"1min":0.66300000000000003,"5min":0.66300000000000003,"15min":0.66300000000000003},"max":{"1min":2.6669999999999998,"5min":2.6669999999999998,"15min":2.6669999999999998},"last":1.1890000000000001},{"interface":"front","average":{"1min":1.3400000000000001,"5min":1.415,"15min":1.4490000000000001},"min":{"1min":0.70099999999999996,"5min":0.629,"15min":0.629},"max":{"1min":1.6160000000000001,"5min":2.1499999999999999,"15min":2.1499999999999999},"last":1.462}]},{"osd":6,"last update":"Wed Aug 9 13:50:27 2023","interfaces":[{"interface":"back","average":{"1min":1.51,"5min":1.532,"15min":1.5469999999999999},"min":{"1min":0.73599999999999999,"5min":0.73599999999999999,"15min":0.69799999999999995},"max":{"1min":2.3100000000000001,"5min":2.7250000000000001,"15min":2.7250000000000001},"last":1.3740000000000001},{"interface":"front","average":{"1min":1.5169999999999999,"5min":1.5389999999999999,"15min":1.5600000000000001},"min":{"1min":0.86699999999999999,"5min":0.86699999999999999,"15min":0.86699999999999999},"max":{"1min":2.3199999999999998,"5min":2.399,"15min":2.399},"last":1.306}]},{"osd":7,"last update":"Wed Aug 9 13:50:27 2023","interfaces":[{"interface":"back","average":{"1min":1.532,"5min":1.595,"15min":1.625},"min":{"1min":0.72299999999999998,"5min":0.72299999999999998,"15min":0.72299999999999998},"max":{"1min":2.3900000000000001,"5min":2.3900000000000001,"15min":2.464},"last":1.5149999999999999},{"interface":"front","average":{"1min":1.4350000000000001,"5min":1.512,"15min":1.55},"min":{"1min":0.84499999999999997,"5min":0.76700000000000002,"15min":0.76700000000000002},"max":{"1min":2.274,"5min":2.274,"15min":2.274},"last":1.603}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-08-09T13:50:34.270 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-08-09T13:50:34.533 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:34.531+0000 7f4ef2ebf640 1 Processor -- start 2023-08-09T13:50:34.533 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:34.531+0000 7f4ef2ebf640 1 -- start start 2023-08-09T13:50:34.534 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:34.533+0000 7f4ef2ebf640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f4eec143c20 0x7f4eec06f540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:50:34.535 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:34.533+0000 7f4ef2ebf640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f4eec0c86f0 0x7f4eec0c8b50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:50:34.535 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:34.533+0000 7f4ef2ebf640 1 -- --> v1:172.21.15.43:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4eec0c9090 con 0x7f4eec06c8d0 2023-08-09T13:50:34.535 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:34.533+0000 7f4ef2ebf640 1 -- --> v2:172.21.15.43:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4eec144000 con 0x7f4eec0c86f0 2023-08-09T13:50:34.535 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:34.533+0000 7f4ef2ebf640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4eec06c440 con 0x7f4eec143c20 2023-08-09T13:50:34.536 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:34.533+0000 7f4eebfff640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f4eec0c86f0 0x7f4eec0c8b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:50:34.536 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:34.533+0000 7f4ef145c640 1 --1- >> v1:172.21.15.43:6789/0 conn(0x7f4eec06c8d0 0x7f4eec06cd60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.43:6789/0 says I am v1:172.21.15.43:32990/0 (socket says 172.21.15.43:32990) 2023-08-09T13:50:34.536 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:34.533+0000 7f4ef145c640 1 -- 172.21.15.43:0/2482778946 learned_addr learned my addr 172.21.15.43:0/2482778946 (peer_addr_for_me v1:172.21.15.43:0/0) 2023-08-09T13:50:34.536 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:34.534+0000 7f4ef0c5b640 1 --2- 172.21.15.43:0/2482778946 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f4eec143c20 0x7f4eec06f540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:50:34.537 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:34.534+0000 7f4eebfff640 1 -- 172.21.15.43:0/2482778946 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f4eec143c20 msgr2=0x7f4eec06f540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-09T13:50:34.537 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:34.534+0000 7f4eebfff640 1 --2- 172.21.15.43:0/2482778946 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f4eec143c20 0x7f4eec06f540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:50:34.537 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:34.534+0000 7f4eebfff640 1 -- 172.21.15.43:0/2482778946 >> v1:172.21.15.43:6789/0 conn(0x7f4eec06c8d0 legacy=0x7f4eec06cd60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:50:34.537 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:34.534+0000 7f4eebfff640 1 -- 172.21.15.43:0/2482778946 --> v2:172.21.15.43:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4eec06b830 con 0x7f4eec0c86f0 2023-08-09T13:50:34.538 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:34.535+0000 7f4eebfff640 1 --2- 172.21.15.43:0/2482778946 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f4eec0c86f0 0x7f4eec0c8b50 secure :-1 s=READY pgs=1024 cs=0 l=1 rev1=1 crypto rx=0x7f4ed80097a0 tx=0x7f4ed802b820 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a32f65006c64e693 server_cookie=0 in_seq=0 out_seq=0 2023-08-09T13:50:34.538 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:34.535+0000 7f4eeb7fe640 1 -- 172.21.15.43:0/2482778946 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4ed8031780 con 0x7f4eec0c86f0 2023-08-09T13:50:34.538 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:34.535+0000 7f4eeb7fe640 1 -- 172.21.15.43:0/2482778946 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4ed80318e0 con 0x7f4eec0c86f0 2023-08-09T13:50:34.539 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:34.535+0000 7f4eeb7fe640 1 -- 172.21.15.43:0/2482778946 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4ed8031c70 con 0x7f4eec0c86f0 2023-08-09T13:50:34.539 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:34.536+0000 7f4ef2ebf640 1 -- 172.21.15.43:0/2482778946 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f4eec0c86f0 msgr2=0x7f4eec0c8b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:50:34.539 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:34.536+0000 7f4ef2ebf640 1 --2- 172.21.15.43:0/2482778946 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f4eec0c86f0 0x7f4eec0c8b50 secure :-1 s=READY pgs=1024 cs=0 l=1 rev1=1 crypto rx=0x7f4ed80097a0 tx=0x7f4ed802b820 comp rx=0 tx=0).stop 2023-08-09T13:50:34.539 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:34.536+0000 7f4ef2ebf640 1 -- 172.21.15.43:0/2482778946 shutdown_connections 2023-08-09T13:50:34.539 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:34.536+0000 7f4ef2ebf640 1 --2- 172.21.15.43:0/2482778946 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f4eec0c86f0 0x7f4eec0c8b50 unknown :-1 s=CLOSED pgs=1024 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:50:34.539 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:34.536+0000 7f4ef2ebf640 1 --2- 172.21.15.43:0/2482778946 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f4eec143c20 0x7f4eec06f540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:50:34.539 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:34.536+0000 7f4ef2ebf640 1 -- 172.21.15.43:0/2482778946 >> 172.21.15.43:0/2482778946 conn(0x7f4eec0ce0a0 msgr2=0x7f4eec0ce480 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-09T13:50:34.540 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:34.536+0000 7f4ef2ebf640 1 -- 172.21.15.43:0/2482778946 shutdown_connections 2023-08-09T13:50:34.540 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:34.536+0000 7f4ef2ebf640 1 -- 172.21.15.43:0/2482778946 wait complete. 2023-08-09T13:50:34.542 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:34.538+0000 7f4ef2ebf640 1 Processor -- start 2023-08-09T13:50:34.542 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:34.538+0000 7f4ef2ebf640 1 -- start start 2023-08-09T13:50:34.542 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:34.539+0000 7f4ef2ebf640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f4eec06c8d0 0x7f4eec13f440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:50:34.542 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:34.539+0000 7f4ef2ebf640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f4eec0c86f0 0x7f4eec0c7da0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:50:34.542 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:34.539+0000 7f4ef2ebf640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f4eec143c20 0x7f4eec0b0d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:50:34.542 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:34.539+0000 7f4ef2ebf640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4eec15ca80 con 0x7f4eec143c20 2023-08-09T13:50:34.543 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:34.539+0000 7f4ef2ebf640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4eec15be00 con 0x7f4eec06c8d0 2023-08-09T13:50:34.543 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:34.539+0000 7f4ef2ebf640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4eec15c0e0 con 0x7f4eec0c86f0 2023-08-09T13:50:34.543 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:34.539+0000 7f4ef145c640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f4eec143c20 0x7f4eec0b0d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:50:34.543 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:34.539+0000 7f4ef145c640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f4eec143c20 0x7f4eec0b0d80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:35824/0 (socket says 172.21.15.43:35824) 2023-08-09T13:50:34.543 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:34.539+0000 7f4ef145c640 1 -- 172.21.15.43:0/2929406363 learned_addr learned my addr 172.21.15.43:0/2929406363 (peer_addr_for_me v2:172.21.15.43:0/0) 2023-08-09T13:50:34.543 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:34.540+0000 7f4ef145c640 1 -- 172.21.15.43:0/2929406363 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f4eec0c86f0 msgr2=0x7f4eec0c7da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:50:34.543 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:34.540+0000 7f4ef145c640 1 --2- 172.21.15.43:0/2929406363 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f4eec0c86f0 0x7f4eec0c7da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:50:34.543 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:34.540+0000 7f4ef145c640 1 -- 172.21.15.43:0/2929406363 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f4eec06c8d0 msgr2=0x7f4eec13f440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:50:34.544 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:34.540+0000 7f4ef145c640 1 --2- 172.21.15.43:0/2929406363 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f4eec06c8d0 0x7f4eec13f440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:50:34.544 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:34.540+0000 7f4ef145c640 1 -- 172.21.15.43:0/2929406363 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4ed8009400 con 0x7f4eec143c20 2023-08-09T13:50:34.544 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:34.540+0000 7f4ef145c640 1 --2- 172.21.15.43:0/2929406363 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f4eec143c20 0x7f4eec0b0d80 secure :-1 s=READY pgs=1025 cs=0 l=1 rev1=1 crypto rx=0x7f4ee400caa0 tx=0x7f4ee400cf60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-09T13:50:34.544 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:34.540+0000 7f4ee97fa640 1 -- 172.21.15.43:0/2929406363 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4ee400ab40 con 0x7f4eec143c20 2023-08-09T13:50:34.544 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:34.540+0000 7f4ee97fa640 1 -- 172.21.15.43:0/2929406363 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4ee400aca0 con 0x7f4eec143c20 2023-08-09T13:50:34.545 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:34.540+0000 7f4ee97fa640 1 -- 172.21.15.43:0/2929406363 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4ee4012600 con 0x7f4eec143c20 2023-08-09T13:50:34.545 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:34.540+0000 7f4ef2ebf640 1 -- 172.21.15.43:0/2929406363 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4eec15c3c0 con 0x7f4eec143c20 2023-08-09T13:50:34.545 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:34.540+0000 7f4ef2ebf640 1 -- 172.21.15.43:0/2929406363 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4eec13f980 con 0x7f4eec143c20 2023-08-09T13:50:34.546 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:34.541+0000 7f4ef2ebf640 1 -- 172.21.15.43:0/2929406363 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4eec15cbc0 con 0x7f4eec143c20 2023-08-09T13:50:34.546 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:34.542+0000 7f4ee97fa640 1 -- 172.21.15.43:0/2929406363 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 29) v1 ==== 88443+0+0 (secure 0 0 0) 0x7f4ee4007070 con 0x7f4eec143c20 2023-08-09T13:50:34.546 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:34.543+0000 7f4ee97fa640 1 --2- 172.21.15.43:0/2929406363 >> [v2:172.21.15.203:6832/3325141227,v1:172.21.15.203:6833/3325141227] conn(0x7f4ed006a170 0x7f4ed006c630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:50:34.546 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:34.543+0000 7f4ee97fa640 1 -- 172.21.15.43:0/2929406363 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(90..90 src has 1..90) v4 ==== 6426+0+0 (secure 0 0 0) 0x7f4ee4088590 con 0x7f4eec143c20 2023-08-09T13:50:34.546 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:34.544+0000 7f4ef0c5b640 1 --2- 172.21.15.43:0/2929406363 >> [v2:172.21.15.203:6832/3325141227,v1:172.21.15.203:6833/3325141227] conn(0x7f4ed006a170 0x7f4ed006c630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:50:34.547 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:34.545+0000 7f4ef0c5b640 1 --2- 172.21.15.43:0/2929406363 >> [v2:172.21.15.203:6832/3325141227,v1:172.21.15.203:6833/3325141227] conn(0x7f4ed006a170 0x7f4ed006c630 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f4eec070bc0 tx=0x7f4ee0008040 comp rx=0 tx=0).ready entity=mgr.5519 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-09T13:50:34.549 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:34.548+0000 7f4ee97fa640 1 -- 172.21.15.43:0/2929406363 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178066 (secure 0 0 0) 0x7f4eec15cbc0 con 0x7f4eec143c20 2023-08-09T13:50:34.736 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:34.734+0000 7f4ef2ebf640 1 -- 172.21.15.43:0/2929406363 --> [v2:172.21.15.203:6832/3325141227,v1:172.21.15.203:6833/3325141227] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f4eec06bdf0 con 0x7f4ed006a170 2023-08-09T13:50:34.761 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:34.759+0000 7f4ee97fa640 1 -- 172.21.15.43:0/2929406363 <== mgr.5519 v2:172.21.15.203:6832/3325141227 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+308019 (secure 0 0 0) 0x7f4ee0009e40 con 0x7f4ed006a170 2023-08-09T13:50:34.763 INFO:teuthology.orchestra.run.smithi043.stdout: 2023-08-09T13:50:34.766 INFO:teuthology.orchestra.run.smithi043.stderr:dumped all 2023-08-09T13:50:34.767 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:34.766+0000 7f4ef2ebf640 1 -- 172.21.15.43:0/2929406363 >> [v2:172.21.15.203:6832/3325141227,v1:172.21.15.203:6833/3325141227] conn(0x7f4ed006a170 msgr2=0x7f4ed006c630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:50:34.768 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:34.766+0000 7f4ef2ebf640 1 --2- 172.21.15.43:0/2929406363 >> [v2:172.21.15.203:6832/3325141227,v1:172.21.15.203:6833/3325141227] conn(0x7f4ed006a170 0x7f4ed006c630 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f4eec070bc0 tx=0x7f4ee0008040 comp rx=0 tx=0).stop 2023-08-09T13:50:34.768 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:34.766+0000 7f4ef2ebf640 1 -- 172.21.15.43:0/2929406363 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f4eec143c20 msgr2=0x7f4eec0b0d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:50:34.768 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:34.766+0000 7f4ef2ebf640 1 --2- 172.21.15.43:0/2929406363 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f4eec143c20 0x7f4eec0b0d80 secure :-1 s=READY pgs=1025 cs=0 l=1 rev1=1 crypto rx=0x7f4ee400caa0 tx=0x7f4ee400cf60 comp rx=0 tx=0).stop 2023-08-09T13:50:34.768 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:34.766+0000 7f4ef2ebf640 1 -- 172.21.15.43:0/2929406363 shutdown_connections 2023-08-09T13:50:34.768 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:34.766+0000 7f4ef2ebf640 1 --2- 172.21.15.43:0/2929406363 >> [v2:172.21.15.203:6832/3325141227,v1:172.21.15.203:6833/3325141227] conn(0x7f4ed006a170 0x7f4ed006c630 unknown :-1 s=CLOSED pgs=67 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:50:34.769 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:34.766+0000 7f4ef2ebf640 1 --2- 172.21.15.43:0/2929406363 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f4eec143c20 0x7f4eec0b0d80 unknown :-1 s=CLOSED pgs=1025 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:50:34.769 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:34.766+0000 7f4ef2ebf640 1 --2- 172.21.15.43:0/2929406363 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f4eec0c86f0 0x7f4eec0c7da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:50:34.769 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:34.766+0000 7f4ef2ebf640 1 --2- 172.21.15.43:0/2929406363 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f4eec06c8d0 0x7f4eec13f440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:50:34.769 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:34.766+0000 7f4ef2ebf640 1 -- 172.21.15.43:0/2929406363 >> 172.21.15.43:0/2929406363 conn(0x7f4eec0ce0a0 msgr2=0x7f4eec0c30d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-09T13:50:34.770 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:34.767+0000 7f4ef2ebf640 1 -- 172.21.15.43:0/2929406363 shutdown_connections 2023-08-09T13:50:34.770 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:34.767+0000 7f4ef2ebf640 1 -- 172.21.15.43:0/2929406363 wait complete. 2023-08-09T13:50:34.790 INFO:teuthology.orchestra.run.smithi043.stdout:{"pg_ready":true,"pg_map":{"version":29,"stamp":"2023-08-09T13:50:34.232744+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592014,"num_objects":8,"num_object_clones":0,"num_object_copies":16,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":500,"num_read_kb":924,"num_write":248,"num_write_kb":2426,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":170,"ondisk_log_size":170,"up":210,"acting":210,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":97,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":279008,"kb_used_data":9572,"kb_used_omap":13,"kb_used_meta":269298,"kb_avail":754695712,"statfs":{"total":773094113280,"available":772808409088,"internally_reserved":0,"allocated":9801728,"data_stored":5763404,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":14248,"internal_metadata":275761240},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":35,"apply_latency_ms":35,"commit_latency_ns":35000000,"apply_latency_ns":35000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":1646,"num_objects":6,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9,"num_write_kb":10,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.066696"},"pg_stats":[{"pgid":"11.3f","version":"0'0","reported_seq":8,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-08-09T13:50:29.233104+0000","last_change":"2023-08-09T13:50:29.233104+0000","last_active":"2023-08-09T13:50:29.233104+0000","last_peered":"2023-08-09T13:50:29.233104+0000","last_clean":"2023-08-09T13:50:29.233104+0000","last_became_active":"2023-08-09T13:50:29.232795+0000","last_became_peered":"2023-08-09T13:50:29.232795+0000","last_unstale":"2023-08-09T13:50:29.233104+0000","last_undegraded":"2023-08-09T13:50:29.233104+0000","last_fullsized":"2023-08-09T13:50:29.233104+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:50:28.150437+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:50:28.150437+0000","last_clean_scrub_stamp":"2023-08-09T13:50:28.150437+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T17:05:45.752848+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"11.3e","version":"0'0","reported_seq":3,"reported_epoch":89,"state":"creating+activating","last_fresh":"2023-08-09T13:50:29.231349+0000","last_change":"2023-08-09T13:50:29.231349+0000","last_active":"2023-08-09T13:50:28.150437+0000","last_peered":"2023-08-09T13:50:28.150437+0000","last_clean":"2023-08-09T13:50:28.150437+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-09T13:50:29.231349+0000","last_undegraded":"2023-08-09T13:50:29.231349+0000","last_fullsized":"2023-08-09T13:50:29.231349+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:50:28.150437+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:50:28.150437+0000","last_clean_scrub_stamp":"2023-08-09T13:50:28.150437+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T18:13:21.364990+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":["6","4"],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.3d","version":"0'0","reported_seq":9,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-08-09T13:50:29.233399+0000","last_change":"2023-08-09T13:50:29.225958+0000","last_active":"2023-08-09T13:50:29.233399+0000","last_peered":"2023-08-09T13:50:29.233399+0000","last_clean":"2023-08-09T13:50:29.233399+0000","last_became_active":"2023-08-09T13:50:29.225536+0000","last_became_peered":"2023-08-09T13:50:29.225536+0000","last_unstale":"2023-08-09T13:50:29.233399+0000","last_undegraded":"2023-08-09T13:50:29.233399+0000","last_fullsized":"2023-08-09T13:50:29.233399+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:50:28.150437+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:50:28.150437+0000","last_clean_scrub_stamp":"2023-08-09T13:50:28.150437+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T22:25:44.631240+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.3c","version":"0'0","reported_seq":11,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-08-09T13:50:30.207338+0000","last_change":"2023-08-09T13:50:29.257870+0000","last_active":"2023-08-09T13:50:30.207338+0000","last_peered":"2023-08-09T13:50:30.207338+0000","last_clean":"2023-08-09T13:50:30.207338+0000","last_became_active":"2023-08-09T13:50:29.257284+0000","last_became_peered":"2023-08-09T13:50:29.257284+0000","last_unstale":"2023-08-09T13:50:30.207338+0000","last_undegraded":"2023-08-09T13:50:30.207338+0000","last_fullsized":"2023-08-09T13:50:30.207338+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:50:28.150437+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:50:28.150437+0000","last_clean_scrub_stamp":"2023-08-09T13:50:28.150437+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T19:35:43.990628+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.3b","version":"0'0","reported_seq":3,"reported_epoch":89,"state":"creating+activating","last_fresh":"2023-08-09T13:50:29.221854+0000","last_change":"2023-08-09T13:50:29.221854+0000","last_active":"2023-08-09T13:50:28.150437+0000","last_peered":"2023-08-09T13:50:28.150437+0000","last_clean":"2023-08-09T13:50:28.150437+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-09T13:50:29.221854+0000","last_undegraded":"2023-08-09T13:50:29.221854+0000","last_fullsized":"2023-08-09T13:50:29.221854+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:50:28.150437+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:50:28.150437+0000","last_clean_scrub_stamp":"2023-08-09T13:50:28.150437+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T22:54:27.616846+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":["2","7"],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.3a","version":"0'0","reported_seq":8,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-08-09T13:50:29.250572+0000","last_change":"2023-08-09T13:50:29.250572+0000","last_active":"2023-08-09T13:50:29.250572+0000","last_peered":"2023-08-09T13:50:29.250572+0000","last_clean":"2023-08-09T13:50:29.250572+0000","last_became_active":"2023-08-09T13:50:29.250267+0000","last_became_peered":"2023-08-09T13:50:29.250267+0000","last_unstale":"2023-08-09T13:50:29.250572+0000","last_undegraded":"2023-08-09T13:50:29.250572+0000","last_fullsized":"2023-08-09T13:50:29.250572+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:50:28.150437+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:50:28.150437+0000","last_clean_scrub_stamp":"2023-08-09T13:50:28.150437+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T17:21:38.689120+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.39","version":"0'0","reported_seq":3,"reported_epoch":89,"state":"creating+activating","last_fresh":"2023-08-09T13:50:29.220028+0000","last_change":"2023-08-09T13:50:29.220028+0000","last_active":"2023-08-09T13:50:28.150437+0000","last_peered":"2023-08-09T13:50:28.150437+0000","last_clean":"2023-08-09T13:50:28.150437+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-09T13:50:29.220028+0000","last_undegraded":"2023-08-09T13:50:29.220028+0000","last_fullsized":"2023-08-09T13:50:29.220028+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:50:28.150437+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:50:28.150437+0000","last_clean_scrub_stamp":"2023-08-09T13:50:28.150437+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T23:19:57.122069+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":["5","1"],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.38","version":"0'0","reported_seq":8,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-08-09T13:50:29.246308+0000","last_change":"2023-08-09T13:50:29.246308+0000","last_active":"2023-08-09T13:50:29.246308+0000","last_peered":"2023-08-09T13:50:29.246308+0000","last_clean":"2023-08-09T13:50:29.246308+0000","last_became_active":"2023-08-09T13:50:29.245268+0000","last_became_peered":"2023-08-09T13:50:29.245268+0000","last_unstale":"2023-08-09T13:50:29.246308+0000","last_undegraded":"2023-08-09T13:50:29.246308+0000","last_fullsized":"2023-08-09T13:50:29.246308+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:50:28.150437+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:50:28.150437+0000","last_clean_scrub_stamp":"2023-08-09T13:50:28.150437+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T18:15:31.389069+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.37","version":"0'0","reported_seq":3,"reported_epoch":89,"state":"creating+activating","last_fresh":"2023-08-09T13:50:29.223778+0000","last_change":"2023-08-09T13:50:29.223778+0000","last_active":"2023-08-09T13:50:28.150437+0000","last_peered":"2023-08-09T13:50:28.150437+0000","last_clean":"2023-08-09T13:50:28.150437+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-09T13:50:29.223778+0000","last_undegraded":"2023-08-09T13:50:29.223778+0000","last_fullsized":"2023-08-09T13:50:29.223778+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:50:28.150437+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:50:28.150437+0000","last_clean_scrub_stamp":"2023-08-09T13:50:28.150437+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T15:38:50.620059+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":["2","3"],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.36","version":"0'0","reported_seq":3,"reported_epoch":89,"state":"creating+activating","last_fresh":"2023-08-09T13:50:29.207900+0000","last_change":"2023-08-09T13:50:29.207900+0000","last_active":"2023-08-09T13:50:28.150437+0000","last_peered":"2023-08-09T13:50:28.150437+0000","last_clean":"2023-08-09T13:50:28.150437+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-09T13:50:29.207900+0000","last_undegraded":"2023-08-09T13:50:29.207900+0000","last_fullsized":"2023-08-09T13:50:29.207900+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:50:28.150437+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:50:28.150437+0000","last_clean_scrub_stamp":"2023-08-09T13:50:28.150437+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T21:32:28.121459+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":["4","2"],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.35","version":"0'0","reported_seq":8,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-08-09T13:50:29.237075+0000","last_change":"2023-08-09T13:50:29.237075+0000","last_active":"2023-08-09T13:50:29.237075+0000","last_peered":"2023-08-09T13:50:29.237075+0000","last_clean":"2023-08-09T13:50:29.237075+0000","last_became_active":"2023-08-09T13:50:29.236351+0000","last_became_peered":"2023-08-09T13:50:29.236351+0000","last_unstale":"2023-08-09T13:50:29.237075+0000","last_undegraded":"2023-08-09T13:50:29.237075+0000","last_fullsized":"2023-08-09T13:50:29.237075+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:50:28.150437+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:50:28.150437+0000","last_clean_scrub_stamp":"2023-08-09T13:50:28.150437+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T18:17:12.675765+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"11.34","version":"0'0","reported_seq":11,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-08-09T13:50:30.207548+0000","last_change":"2023-08-09T13:50:29.234197+0000","last_active":"2023-08-09T13:50:30.207548+0000","last_peered":"2023-08-09T13:50:30.207548+0000","last_clean":"2023-08-09T13:50:30.207548+0000","last_became_active":"2023-08-09T13:50:29.233877+0000","last_became_peered":"2023-08-09T13:50:29.233877+0000","last_unstale":"2023-08-09T13:50:30.207548+0000","last_undegraded":"2023-08-09T13:50:30.207548+0000","last_fullsized":"2023-08-09T13:50:30.207548+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:50:28.150437+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:50:28.150437+0000","last_clean_scrub_stamp":"2023-08-09T13:50:28.150437+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T21:16:13.484480+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.33","version":"0'0","reported_seq":11,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-08-09T13:50:30.405888+0000","last_change":"2023-08-09T13:50:29.243093+0000","last_active":"2023-08-09T13:50:30.405888+0000","last_peered":"2023-08-09T13:50:30.405888+0000","last_clean":"2023-08-09T13:50:30.405888+0000","last_became_active":"2023-08-09T13:50:29.242740+0000","last_became_peered":"2023-08-09T13:50:29.242740+0000","last_unstale":"2023-08-09T13:50:30.405888+0000","last_undegraded":"2023-08-09T13:50:30.405888+0000","last_fullsized":"2023-08-09T13:50:30.405888+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:50:28.150437+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:50:28.150437+0000","last_clean_scrub_stamp":"2023-08-09T13:50:28.150437+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T15:42:16.315503+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.32","version":"0'0","reported_seq":3,"reported_epoch":89,"state":"creating+activating","last_fresh":"2023-08-09T13:50:29.224087+0000","last_change":"2023-08-09T13:50:29.224087+0000","last_active":"2023-08-09T13:50:28.150437+0000","last_peered":"2023-08-09T13:50:28.150437+0000","last_clean":"2023-08-09T13:50:28.150437+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-09T13:50:29.224087+0000","last_undegraded":"2023-08-09T13:50:29.224087+0000","last_fullsized":"2023-08-09T13:50:29.224087+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:50:28.150437+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:50:28.150437+0000","last_clean_scrub_stamp":"2023-08-09T13:50:28.150437+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T23:57:06.314706+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":["2","5"],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.31","version":"0'0","reported_seq":11,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-08-09T13:50:30.405358+0000","last_change":"2023-08-09T13:50:29.231343+0000","last_active":"2023-08-09T13:50:30.405358+0000","last_peered":"2023-08-09T13:50:30.405358+0000","last_clean":"2023-08-09T13:50:30.405358+0000","last_became_active":"2023-08-09T13:50:29.230980+0000","last_became_peered":"2023-08-09T13:50:29.230980+0000","last_unstale":"2023-08-09T13:50:30.405358+0000","last_undegraded":"2023-08-09T13:50:30.405358+0000","last_fullsized":"2023-08-09T13:50:30.405358+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:50:28.150437+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:50:28.150437+0000","last_clean_scrub_stamp":"2023-08-09T13:50:28.150437+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T23:54:43.168168+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.30","version":"0'0","reported_seq":3,"reported_epoch":89,"state":"creating+activating","last_fresh":"2023-08-09T13:50:29.208580+0000","last_change":"2023-08-09T13:50:29.208580+0000","last_active":"2023-08-09T13:50:28.150437+0000","last_peered":"2023-08-09T13:50:28.150437+0000","last_clean":"2023-08-09T13:50:28.150437+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-09T13:50:29.208580+0000","last_undegraded":"2023-08-09T13:50:29.208580+0000","last_fullsized":"2023-08-09T13:50:29.208580+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:50:28.150437+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:50:28.150437+0000","last_clean_scrub_stamp":"2023-08-09T13:50:28.150437+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T15:14:37.215348+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":["4","6"],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.2f","version":"0'0","reported_seq":3,"reported_epoch":89,"state":"creating+activating","last_fresh":"2023-08-09T13:50:29.223293+0000","last_change":"2023-08-09T13:50:29.223293+0000","last_active":"2023-08-09T13:50:28.150437+0000","last_peered":"2023-08-09T13:50:28.150437+0000","last_clean":"2023-08-09T13:50:28.150437+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-09T13:50:29.223293+0000","last_undegraded":"2023-08-09T13:50:29.223293+0000","last_fullsized":"2023-08-09T13:50:29.223293+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:50:28.150437+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:50:28.150437+0000","last_clean_scrub_stamp":"2023-08-09T13:50:28.150437+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T22:44:32.845197+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":["2","3"],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.2e","version":"0'0","reported_seq":11,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-08-09T13:50:30.208026+0000","last_change":"2023-08-09T13:50:29.244582+0000","last_active":"2023-08-09T13:50:30.208026+0000","last_peered":"2023-08-09T13:50:30.208026+0000","last_clean":"2023-08-09T13:50:30.208026+0000","last_became_active":"2023-08-09T13:50:29.244222+0000","last_became_peered":"2023-08-09T13:50:29.244222+0000","last_unstale":"2023-08-09T13:50:30.208026+0000","last_undegraded":"2023-08-09T13:50:30.208026+0000","last_fullsized":"2023-08-09T13:50:30.208026+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:50:28.150437+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:50:28.150437+0000","last_clean_scrub_stamp":"2023-08-09T13:50:28.150437+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-11T00:06:51.620725+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.2d","version":"0'0","reported_seq":9,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-08-09T13:50:29.248797+0000","last_change":"2023-08-09T13:50:29.241875+0000","last_active":"2023-08-09T13:50:29.248797+0000","last_peered":"2023-08-09T13:50:29.248797+0000","last_clean":"2023-08-09T13:50:29.248797+0000","last_became_active":"2023-08-09T13:50:29.241532+0000","last_became_peered":"2023-08-09T13:50:29.241532+0000","last_unstale":"2023-08-09T13:50:29.248797+0000","last_undegraded":"2023-08-09T13:50:29.248797+0000","last_fullsized":"2023-08-09T13:50:29.248797+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:50:28.150437+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:50:28.150437+0000","last_clean_scrub_stamp":"2023-08-09T13:50:28.150437+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-11T00:33:32.657666+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.2c","version":"0'0","reported_seq":11,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-08-09T13:50:30.406101+0000","last_change":"2023-08-09T13:50:29.231091+0000","last_active":"2023-08-09T13:50:30.406101+0000","last_peered":"2023-08-09T13:50:30.406101+0000","last_clean":"2023-08-09T13:50:30.406101+0000","last_became_active":"2023-08-09T13:50:29.230501+0000","last_became_peered":"2023-08-09T13:50:29.230501+0000","last_unstale":"2023-08-09T13:50:30.406101+0000","last_undegraded":"2023-08-09T13:50:30.406101+0000","last_fullsized":"2023-08-09T13:50:30.406101+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:50:28.150437+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:50:28.150437+0000","last_clean_scrub_stamp":"2023-08-09T13:50:28.150437+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-11T01:19:18.466874+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.2b","version":"0'0","reported_seq":9,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-08-09T13:50:29.233438+0000","last_change":"2023-08-09T13:50:29.227706+0000","last_active":"2023-08-09T13:50:29.233438+0000","last_peered":"2023-08-09T13:50:29.233438+0000","last_clean":"2023-08-09T13:50:29.233438+0000","last_became_active":"2023-08-09T13:50:29.226933+0000","last_became_peered":"2023-08-09T13:50:29.226933+0000","last_unstale":"2023-08-09T13:50:29.233438+0000","last_undegraded":"2023-08-09T13:50:29.233438+0000","last_fullsized":"2023-08-09T13:50:29.233438+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:50:28.150437+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:50:28.150437+0000","last_clean_scrub_stamp":"2023-08-09T13:50:28.150437+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T17:57:56.069305+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.2a","version":"0'0","reported_seq":3,"reported_epoch":89,"state":"creating+activating","last_fresh":"2023-08-09T13:50:29.207312+0000","last_change":"2023-08-09T13:50:29.207312+0000","last_active":"2023-08-09T13:50:28.150437+0000","last_peered":"2023-08-09T13:50:28.150437+0000","last_clean":"2023-08-09T13:50:28.150437+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-09T13:50:29.207312+0000","last_undegraded":"2023-08-09T13:50:29.207312+0000","last_fullsized":"2023-08-09T13:50:29.207312+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:50:28.150437+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:50:28.150437+0000","last_clean_scrub_stamp":"2023-08-09T13:50:28.150437+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T19:03:45.295280+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":["4","2"],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.29","version":"0'0","reported_seq":8,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-08-09T13:50:29.232548+0000","last_change":"2023-08-09T13:50:29.232548+0000","last_active":"2023-08-09T13:50:29.232548+0000","last_peered":"2023-08-09T13:50:29.232548+0000","last_clean":"2023-08-09T13:50:29.232548+0000","last_became_active":"2023-08-09T13:50:29.232205+0000","last_became_peered":"2023-08-09T13:50:29.232205+0000","last_unstale":"2023-08-09T13:50:29.232548+0000","last_undegraded":"2023-08-09T13:50:29.232548+0000","last_fullsized":"2023-08-09T13:50:29.232548+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:50:28.150437+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:50:28.150437+0000","last_clean_scrub_stamp":"2023-08-09T13:50:28.150437+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T21:11:03.639832+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"11.28","version":"0'0","reported_seq":3,"reported_epoch":89,"state":"creating+activating","last_fresh":"2023-08-09T13:50:29.217553+0000","last_change":"2023-08-09T13:50:29.217553+0000","last_active":"2023-08-09T13:50:28.150437+0000","last_peered":"2023-08-09T13:50:28.150437+0000","last_clean":"2023-08-09T13:50:28.150437+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-09T13:50:29.217553+0000","last_undegraded":"2023-08-09T13:50:29.217553+0000","last_fullsized":"2023-08-09T13:50:29.217553+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:50:28.150437+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:50:28.150437+0000","last_clean_scrub_stamp":"2023-08-09T13:50:28.150437+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T22:39:17.713654+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":["7","6"],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"11.27","version":"0'0","reported_seq":3,"reported_epoch":89,"state":"creating+activating","last_fresh":"2023-08-09T13:50:29.208200+0000","last_change":"2023-08-09T13:50:29.208200+0000","last_active":"2023-08-09T13:50:28.150437+0000","last_peered":"2023-08-09T13:50:28.150437+0000","last_clean":"2023-08-09T13:50:28.150437+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-09T13:50:29.208200+0000","last_undegraded":"2023-08-09T13:50:29.208200+0000","last_fullsized":"2023-08-09T13:50:29.208200+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:50:28.150437+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:50:28.150437+0000","last_clean_scrub_stamp":"2023-08-09T13:50:28.150437+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T14:52:59.954591+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":["4","7"],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.26","version":"0'0","reported_seq":3,"reported_epoch":89,"state":"creating+activating","last_fresh":"2023-08-09T13:50:29.223117+0000","last_change":"2023-08-09T13:50:29.223117+0000","last_active":"2023-08-09T13:50:28.150437+0000","last_peered":"2023-08-09T13:50:28.150437+0000","last_clean":"2023-08-09T13:50:28.150437+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-09T13:50:29.223117+0000","last_undegraded":"2023-08-09T13:50:29.223117+0000","last_fullsized":"2023-08-09T13:50:29.223117+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:50:28.150437+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:50:28.150437+0000","last_clean_scrub_stamp":"2023-08-09T13:50:28.150437+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T21:18:31.512056+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":["5","6"],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.25","version":"0'0","reported_seq":8,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-08-09T13:50:29.233554+0000","last_change":"2023-08-09T13:50:29.233554+0000","last_active":"2023-08-09T13:50:29.233554+0000","last_peered":"2023-08-09T13:50:29.233554+0000","last_clean":"2023-08-09T13:50:29.233554+0000","last_became_active":"2023-08-09T13:50:29.233153+0000","last_became_peered":"2023-08-09T13:50:29.233153+0000","last_unstale":"2023-08-09T13:50:29.233554+0000","last_undegraded":"2023-08-09T13:50:29.233554+0000","last_fullsized":"2023-08-09T13:50:29.233554+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:50:28.150437+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:50:28.150437+0000","last_clean_scrub_stamp":"2023-08-09T13:50:28.150437+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T21:36:11.302948+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.24","version":"0'0","reported_seq":3,"reported_epoch":89,"state":"creating+activating","last_fresh":"2023-08-09T13:50:29.207497+0000","last_change":"2023-08-09T13:50:29.207497+0000","last_active":"2023-08-09T13:50:28.150437+0000","last_peered":"2023-08-09T13:50:28.150437+0000","last_clean":"2023-08-09T13:50:28.150437+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-09T13:50:29.207497+0000","last_undegraded":"2023-08-09T13:50:29.207497+0000","last_fullsized":"2023-08-09T13:50:29.207497+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:50:28.150437+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:50:28.150437+0000","last_clean_scrub_stamp":"2023-08-09T13:50:28.150437+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T18:47:18.558930+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":["4","2"],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.23","version":"0'0","reported_seq":8,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-08-09T13:50:29.233513+0000","last_change":"2023-08-09T13:50:29.233513+0000","last_active":"2023-08-09T13:50:29.233513+0000","last_peered":"2023-08-09T13:50:29.233513+0000","last_clean":"2023-08-09T13:50:29.233513+0000","last_became_active":"2023-08-09T13:50:29.233199+0000","last_became_peered":"2023-08-09T13:50:29.233199+0000","last_unstale":"2023-08-09T13:50:29.233513+0000","last_undegraded":"2023-08-09T13:50:29.233513+0000","last_fullsized":"2023-08-09T13:50:29.233513+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:50:28.150437+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:50:28.150437+0000","last_clean_scrub_stamp":"2023-08-09T13:50:28.150437+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T21:47:07.987840+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.22","version":"0'0","reported_seq":3,"reported_epoch":89,"state":"creating+activating","last_fresh":"2023-08-09T13:50:29.219724+0000","last_change":"2023-08-09T13:50:29.219724+0000","last_active":"2023-08-09T13:50:28.150437+0000","last_peered":"2023-08-09T13:50:28.150437+0000","last_clean":"2023-08-09T13:50:28.150437+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-09T13:50:29.219724+0000","last_undegraded":"2023-08-09T13:50:29.219724+0000","last_fullsized":"2023-08-09T13:50:29.219724+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:50:28.150437+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:50:28.150437+0000","last_clean_scrub_stamp":"2023-08-09T13:50:28.150437+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T21:52:25.852199+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":["5","6"],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.21","version":"0'0","reported_seq":8,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-08-09T13:50:29.233755+0000","last_change":"2023-08-09T13:50:29.233755+0000","last_active":"2023-08-09T13:50:29.233755+0000","last_peered":"2023-08-09T13:50:29.233755+0000","last_clean":"2023-08-09T13:50:29.233755+0000","last_became_active":"2023-08-09T13:50:29.233150+0000","last_became_peered":"2023-08-09T13:50:29.233150+0000","last_unstale":"2023-08-09T13:50:29.233755+0000","last_undegraded":"2023-08-09T13:50:29.233755+0000","last_fullsized":"2023-08-09T13:50:29.233755+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:50:28.150437+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:50:28.150437+0000","last_clean_scrub_stamp":"2023-08-09T13:50:28.150437+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T16:18:45.331895+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.20","version":"0'0","reported_seq":8,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-08-09T13:50:29.245742+0000","last_change":"2023-08-09T13:50:29.245742+0000","last_active":"2023-08-09T13:50:29.245742+0000","last_peered":"2023-08-09T13:50:29.245742+0000","last_clean":"2023-08-09T13:50:29.245742+0000","last_became_active":"2023-08-09T13:50:29.242577+0000","last_became_peered":"2023-08-09T13:50:29.242577+0000","last_unstale":"2023-08-09T13:50:29.245742+0000","last_undegraded":"2023-08-09T13:50:29.245742+0000","last_fullsized":"2023-08-09T13:50:29.245742+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:50:28.150437+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:50:28.150437+0000","last_clean_scrub_stamp":"2023-08-09T13:50:28.150437+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T16:18:45.331895+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.1e","version":"0'0","reported_seq":3,"reported_epoch":89,"state":"creating+activating","last_fresh":"2023-08-09T13:50:29.220103+0000","last_change":"2023-08-09T13:50:29.220103+0000","last_active":"2023-08-09T13:50:28.150437+0000","last_peered":"2023-08-09T13:50:28.150437+0000","last_clean":"2023-08-09T13:50:28.150437+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-09T13:50:29.220103+0000","last_undegraded":"2023-08-09T13:50:29.220103+0000","last_fullsized":"2023-08-09T13:50:29.220103+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:50:28.150437+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:50:28.150437+0000","last_clean_scrub_stamp":"2023-08-09T13:50:28.150437+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T19:44:56.828003+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":["7","6"],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.1f","version":"0'0","reported_seq":11,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-08-09T13:50:29.221681+0000","last_change":"2023-08-09T13:50:28.198750+0000","last_active":"2023-08-09T13:50:29.221681+0000","last_peered":"2023-08-09T13:50:29.221681+0000","last_clean":"2023-08-09T13:50:29.221681+0000","last_became_active":"2023-08-09T13:50:28.198375+0000","last_became_peered":"2023-08-09T13:50:28.198375+0000","last_unstale":"2023-08-09T13:50:29.221681+0000","last_undegraded":"2023-08-09T13:50:29.221681+0000","last_fullsized":"2023-08-09T13:50:29.221681+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:50:27.140775+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:50:27.140775+0000","last_clean_scrub_stamp":"2023-08-09T13:50:27.140775+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T23:54:42.158506+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"11.1f","version":"0'0","reported_seq":3,"reported_epoch":89,"state":"creating+activating","last_fresh":"2023-08-09T13:50:29.223179+0000","last_change":"2023-08-09T13:50:29.223179+0000","last_active":"2023-08-09T13:50:28.150437+0000","last_peered":"2023-08-09T13:50:28.150437+0000","last_clean":"2023-08-09T13:50:28.150437+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-09T13:50:29.223179+0000","last_undegraded":"2023-08-09T13:50:29.223179+0000","last_fullsized":"2023-08-09T13:50:29.223179+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:50:28.150437+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:50:28.150437+0000","last_clean_scrub_stamp":"2023-08-09T13:50:28.150437+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T20:43:24.475264+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":["2","3"],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.1e","version":"89'1","reported_seq":14,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-08-09T13:50:30.407030+0000","last_change":"2023-08-09T13:50:28.189590+0000","last_active":"2023-08-09T13:50:30.407030+0000","last_peered":"2023-08-09T13:50:30.407030+0000","last_clean":"2023-08-09T13:50:30.407030+0000","last_became_active":"2023-08-09T13:50:28.189227+0000","last_became_peered":"2023-08-09T13:50:28.189227+0000","last_unstale":"2023-08-09T13:50:30.407030+0000","last_undegraded":"2023-08-09T13:50:30.407030+0000","last_fullsized":"2023-08-09T13:50:30.407030+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:50:27.140775+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:50:27.140775+0000","last_clean_scrub_stamp":"2023-08-09T13:50:27.140775+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T18:15:27.464113+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.1c","version":"0'0","reported_seq":4,"reported_epoch":89,"state":"creating+activating","last_fresh":"2023-08-09T13:50:29.250266+0000","last_change":"2023-08-09T13:50:29.225957+0000","last_active":"2023-08-09T13:50:28.150437+0000","last_peered":"2023-08-09T13:50:28.150437+0000","last_clean":"2023-08-09T13:50:28.150437+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-09T13:50:29.250266+0000","last_undegraded":"2023-08-09T13:50:29.250266+0000","last_fullsized":"2023-08-09T13:50:29.250266+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:50:28.150437+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:50:28.150437+0000","last_clean_scrub_stamp":"2023-08-09T13:50:28.150437+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T21:57:09.347886+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":["6","7"],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"10.1d","version":"0'0","reported_seq":11,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-08-09T13:50:29.210839+0000","last_change":"2023-08-09T13:50:28.206216+0000","last_active":"2023-08-09T13:50:29.210839+0000","last_peered":"2023-08-09T13:50:29.210839+0000","last_clean":"2023-08-09T13:50:29.210839+0000","last_became_active":"2023-08-09T13:50:28.205914+0000","last_became_peered":"2023-08-09T13:50:28.205914+0000","last_unstale":"2023-08-09T13:50:29.210839+0000","last_undegraded":"2023-08-09T13:50:29.210839+0000","last_fullsized":"2023-08-09T13:50:29.210839+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:50:27.140775+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:50:27.140775+0000","last_clean_scrub_stamp":"2023-08-09T13:50:27.140775+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T16:26:26.712563+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.1d","version":"0'0","reported_seq":8,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-08-09T13:50:29.231490+0000","last_change":"2023-08-09T13:50:29.231490+0000","last_active":"2023-08-09T13:50:29.231490+0000","last_peered":"2023-08-09T13:50:29.231490+0000","last_clean":"2023-08-09T13:50:29.231490+0000","last_became_active":"2023-08-09T13:50:29.231190+0000","last_became_peered":"2023-08-09T13:50:29.231190+0000","last_unstale":"2023-08-09T13:50:29.231490+0000","last_undegraded":"2023-08-09T13:50:29.231490+0000","last_fullsized":"2023-08-09T13:50:29.231490+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:50:28.150437+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:50:28.150437+0000","last_clean_scrub_stamp":"2023-08-09T13:50:28.150437+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T14:46:45.632501+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.1c","version":"0'0","reported_seq":11,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-08-09T13:50:29.235770+0000","last_change":"2023-08-09T13:50:28.195729+0000","last_active":"2023-08-09T13:50:29.235770+0000","last_peered":"2023-08-09T13:50:29.235770+0000","last_clean":"2023-08-09T13:50:29.235770+0000","last_became_active":"2023-08-09T13:50:28.195433+0000","last_became_peered":"2023-08-09T13:50:28.195433+0000","last_unstale":"2023-08-09T13:50:29.235770+0000","last_undegraded":"2023-08-09T13:50:29.235770+0000","last_fullsized":"2023-08-09T13:50:29.235770+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:50:27.140775+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:50:27.140775+0000","last_clean_scrub_stamp":"2023-08-09T13:50:27.140775+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T16:52:42.838615+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.1a","version":"0'0","reported_seq":8,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-08-09T13:50:29.234000+0000","last_change":"2023-08-09T13:50:29.234000+0000","last_active":"2023-08-09T13:50:29.234000+0000","last_peered":"2023-08-09T13:50:29.234000+0000","last_clean":"2023-08-09T13:50:29.234000+0000","last_became_active":"2023-08-09T13:50:29.233727+0000","last_became_peered":"2023-08-09T13:50:29.233727+0000","last_unstale":"2023-08-09T13:50:29.234000+0000","last_undegraded":"2023-08-09T13:50:29.234000+0000","last_fullsized":"2023-08-09T13:50:29.234000+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:50:28.150437+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:50:28.150437+0000","last_clean_scrub_stamp":"2023-08-09T13:50:28.150437+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T20:21:45.115966+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.1b","version":"0'0","reported_seq":11,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-08-09T13:50:29.214221+0000","last_change":"2023-08-09T13:50:28.196698+0000","last_active":"2023-08-09T13:50:29.214221+0000","last_peered":"2023-08-09T13:50:29.214221+0000","last_clean":"2023-08-09T13:50:29.214221+0000","last_became_active":"2023-08-09T13:50:28.196362+0000","last_became_peered":"2023-08-09T13:50:28.196362+0000","last_unstale":"2023-08-09T13:50:29.214221+0000","last_undegraded":"2023-08-09T13:50:29.214221+0000","last_fullsized":"2023-08-09T13:50:29.214221+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:50:27.140775+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:50:27.140775+0000","last_clean_scrub_stamp":"2023-08-09T13:50:27.140775+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T16:26:00.621455+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.1b","version":"0'0","reported_seq":3,"reported_epoch":89,"state":"creating+activating","last_fresh":"2023-08-09T13:50:29.208495+0000","last_change":"2023-08-09T13:50:29.208495+0000","last_active":"2023-08-09T13:50:28.150437+0000","last_peered":"2023-08-09T13:50:28.150437+0000","last_clean":"2023-08-09T13:50:28.150437+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-09T13:50:29.208495+0000","last_undegraded":"2023-08-09T13:50:29.208495+0000","last_fullsized":"2023-08-09T13:50:29.208495+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:50:28.150437+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:50:28.150437+0000","last_clean_scrub_stamp":"2023-08-09T13:50:28.150437+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T16:20:35.941253+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":["4","7"],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"10.1a","version":"0'0","reported_seq":11,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-08-09T13:50:29.228557+0000","last_change":"2023-08-09T13:50:28.205333+0000","last_active":"2023-08-09T13:50:29.228557+0000","last_peered":"2023-08-09T13:50:29.228557+0000","last_clean":"2023-08-09T13:50:29.228557+0000","last_became_active":"2023-08-09T13:50:28.204881+0000","last_became_peered":"2023-08-09T13:50:28.204881+0000","last_unstale":"2023-08-09T13:50:29.228557+0000","last_undegraded":"2023-08-09T13:50:29.228557+0000","last_fullsized":"2023-08-09T13:50:29.228557+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:50:27.140775+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:50:27.140775+0000","last_clean_scrub_stamp":"2023-08-09T13:50:27.140775+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T18:06:20.762713+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.18","version":"0'0","reported_seq":3,"reported_epoch":89,"state":"creating+activating","last_fresh":"2023-08-09T13:50:29.218450+0000","last_change":"2023-08-09T13:50:29.218450+0000","last_active":"2023-08-09T13:50:28.150437+0000","last_peered":"2023-08-09T13:50:28.150437+0000","last_clean":"2023-08-09T13:50:28.150437+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-09T13:50:29.218450+0000","last_undegraded":"2023-08-09T13:50:29.218450+0000","last_fullsized":"2023-08-09T13:50:29.218450+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:50:28.150437+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:50:28.150437+0000","last_clean_scrub_stamp":"2023-08-09T13:50:28.150437+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T15:00:53.649507+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":["2","4"],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.19","version":"0'0","reported_seq":13,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-08-09T13:50:30.207222+0000","last_change":"2023-08-09T13:50:28.204197+0000","last_active":"2023-08-09T13:50:30.207222+0000","last_peered":"2023-08-09T13:50:30.207222+0000","last_clean":"2023-08-09T13:50:30.207222+0000","last_became_active":"2023-08-09T13:50:28.203911+0000","last_became_peered":"2023-08-09T13:50:28.203911+0000","last_unstale":"2023-08-09T13:50:30.207222+0000","last_undegraded":"2023-08-09T13:50:30.207222+0000","last_fullsized":"2023-08-09T13:50:30.207222+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:50:27.140775+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:50:27.140775+0000","last_clean_scrub_stamp":"2023-08-09T13:50:27.140775+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T19:31:34.788085+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.19","version":"0'0","reported_seq":11,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-08-09T13:50:30.405358+0000","last_change":"2023-08-09T13:50:29.223620+0000","last_active":"2023-08-09T13:50:30.405358+0000","last_peered":"2023-08-09T13:50:30.405358+0000","last_clean":"2023-08-09T13:50:30.405358+0000","last_became_active":"2023-08-09T13:50:29.223063+0000","last_became_peered":"2023-08-09T13:50:29.223063+0000","last_unstale":"2023-08-09T13:50:30.405358+0000","last_undegraded":"2023-08-09T13:50:30.405358+0000","last_fullsized":"2023-08-09T13:50:30.405358+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:50:28.150437+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:50:28.150437+0000","last_clean_scrub_stamp":"2023-08-09T13:50:28.150437+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T16:41:09.645304+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.18","version":"0'0","reported_seq":11,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-08-09T13:50:29.224563+0000","last_change":"2023-08-09T13:50:28.189872+0000","last_active":"2023-08-09T13:50:29.224563+0000","last_peered":"2023-08-09T13:50:29.224563+0000","last_clean":"2023-08-09T13:50:29.224563+0000","last_became_active":"2023-08-09T13:50:28.189560+0000","last_became_peered":"2023-08-09T13:50:28.189560+0000","last_unstale":"2023-08-09T13:50:29.224563+0000","last_undegraded":"2023-08-09T13:50:29.224563+0000","last_fullsized":"2023-08-09T13:50:29.224563+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:50:27.140775+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:50:27.140775+0000","last_clean_scrub_stamp":"2023-08-09T13:50:27.140775+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T18:02:42.591074+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.16","version":"0'0","reported_seq":3,"reported_epoch":89,"state":"creating+activating","last_fresh":"2023-08-09T13:50:29.223029+0000","last_change":"2023-08-09T13:50:29.223029+0000","last_active":"2023-08-09T13:50:28.150437+0000","last_peered":"2023-08-09T13:50:28.150437+0000","last_clean":"2023-08-09T13:50:28.150437+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-09T13:50:29.223029+0000","last_undegraded":"2023-08-09T13:50:29.223029+0000","last_fullsized":"2023-08-09T13:50:29.223029+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:50:28.150437+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:50:28.150437+0000","last_clean_scrub_stamp":"2023-08-09T13:50:28.150437+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T18:35:45.000310+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":["5","7"],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.17","version":"0'0","reported_seq":11,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-08-09T13:50:29.210879+0000","last_change":"2023-08-09T13:50:28.198990+0000","last_active":"2023-08-09T13:50:29.210879+0000","last_peered":"2023-08-09T13:50:29.210879+0000","last_clean":"2023-08-09T13:50:29.210879+0000","last_became_active":"2023-08-09T13:50:28.197376+0000","last_became_peered":"2023-08-09T13:50:28.197376+0000","last_unstale":"2023-08-09T13:50:29.210879+0000","last_undegraded":"2023-08-09T13:50:29.210879+0000","last_fullsized":"2023-08-09T13:50:29.210879+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:50:27.140775+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:50:27.140775+0000","last_clean_scrub_stamp":"2023-08-09T13:50:27.140775+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T18:00:13.480919+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.17","version":"0'0","reported_seq":3,"reported_epoch":89,"state":"creating+activating","last_fresh":"2023-08-09T13:50:29.221024+0000","last_change":"2023-08-09T13:50:29.221024+0000","last_active":"2023-08-09T13:50:28.150437+0000","last_peered":"2023-08-09T13:50:28.150437+0000","last_clean":"2023-08-09T13:50:28.150437+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-09T13:50:29.221024+0000","last_undegraded":"2023-08-09T13:50:29.221024+0000","last_fullsized":"2023-08-09T13:50:29.221024+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:50:28.150437+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:50:28.150437+0000","last_clean_scrub_stamp":"2023-08-09T13:50:28.150437+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T19:41:45.324636+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":["7","6"],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.16","version":"0'0","reported_seq":11,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-08-09T13:50:29.236992+0000","last_change":"2023-08-09T13:50:28.203893+0000","last_active":"2023-08-09T13:50:29.236992+0000","last_peered":"2023-08-09T13:50:29.236992+0000","last_clean":"2023-08-09T13:50:29.236992+0000","last_became_active":"2023-08-09T13:50:28.203596+0000","last_became_peered":"2023-08-09T13:50:28.203596+0000","last_unstale":"2023-08-09T13:50:29.236992+0000","last_undegraded":"2023-08-09T13:50:29.236992+0000","last_fullsized":"2023-08-09T13:50:29.236992+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:50:27.140775+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:50:27.140775+0000","last_clean_scrub_stamp":"2023-08-09T13:50:27.140775+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-11T01:07:24.154763+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.14","version":"0'0","reported_seq":8,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-08-09T13:50:29.236992+0000","last_change":"2023-08-09T13:50:29.236992+0000","last_active":"2023-08-09T13:50:29.236992+0000","last_peered":"2023-08-09T13:50:29.236992+0000","last_clean":"2023-08-09T13:50:29.236992+0000","last_became_active":"2023-08-09T13:50:29.236708+0000","last_became_peered":"2023-08-09T13:50:29.236708+0000","last_unstale":"2023-08-09T13:50:29.236992+0000","last_undegraded":"2023-08-09T13:50:29.236992+0000","last_fullsized":"2023-08-09T13:50:29.236992+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:50:28.150437+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:50:28.150437+0000","last_clean_scrub_stamp":"2023-08-09T13:50:28.150437+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T23:19:57.122069+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"10.15","version":"0'0","reported_seq":11,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-08-09T13:50:29.215134+0000","last_change":"2023-08-09T13:50:28.198818+0000","last_active":"2023-08-09T13:50:29.215134+0000","last_peered":"2023-08-09T13:50:29.215134+0000","last_clean":"2023-08-09T13:50:29.215134+0000","last_became_active":"2023-08-09T13:50:28.198282+0000","last_became_peered":"2023-08-09T13:50:28.198282+0000","last_unstale":"2023-08-09T13:50:29.215134+0000","last_undegraded":"2023-08-09T13:50:29.215134+0000","last_fullsized":"2023-08-09T13:50:29.215134+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:50:27.140775+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:50:27.140775+0000","last_clean_scrub_stamp":"2023-08-09T13:50:27.140775+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T13:56:48.165182+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.15","version":"0'0","reported_seq":8,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-08-09T13:50:29.232045+0000","last_change":"2023-08-09T13:50:29.232045+0000","last_active":"2023-08-09T13:50:29.232045+0000","last_peered":"2023-08-09T13:50:29.232045+0000","last_clean":"2023-08-09T13:50:29.232045+0000","last_became_active":"2023-08-09T13:50:29.231691+0000","last_became_peered":"2023-08-09T13:50:29.231691+0000","last_unstale":"2023-08-09T13:50:29.232045+0000","last_undegraded":"2023-08-09T13:50:29.232045+0000","last_fullsized":"2023-08-09T13:50:29.232045+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:50:28.150437+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:50:28.150437+0000","last_clean_scrub_stamp":"2023-08-09T13:50:28.150437+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T14:41:33.537815+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.14","version":"89'6","reported_seq":19,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-08-09T13:50:30.207724+0000","last_change":"2023-08-09T13:50:28.194890+0000","last_active":"2023-08-09T13:50:30.207724+0000","last_peered":"2023-08-09T13:50:30.207724+0000","last_clean":"2023-08-09T13:50:30.207724+0000","last_became_active":"2023-08-09T13:50:28.194607+0000","last_became_peered":"2023-08-09T13:50:28.194607+0000","last_unstale":"2023-08-09T13:50:30.207724+0000","last_undegraded":"2023-08-09T13:50:30.207724+0000","last_fullsized":"2023-08-09T13:50:30.207724+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:50:27.140775+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:50:27.140775+0000","last_clean_scrub_stamp":"2023-08-09T13:50:27.140775+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T19:14:35.323306+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.12","version":"0'0","reported_seq":8,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-08-09T13:50:29.233976+0000","last_change":"2023-08-09T13:50:29.233976+0000","last_active":"2023-08-09T13:50:29.233976+0000","last_peered":"2023-08-09T13:50:29.233976+0000","last_clean":"2023-08-09T13:50:29.233976+0000","last_became_active":"2023-08-09T13:50:29.233696+0000","last_became_peered":"2023-08-09T13:50:29.233696+0000","last_unstale":"2023-08-09T13:50:29.233976+0000","last_undegraded":"2023-08-09T13:50:29.233976+0000","last_fullsized":"2023-08-09T13:50:29.233976+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:50:28.150437+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:50:28.150437+0000","last_clean_scrub_stamp":"2023-08-09T13:50:28.150437+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T15:32:41.261189+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.13","version":"89'2","reported_seq":15,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-08-09T13:50:30.207465+0000","last_change":"2023-08-09T13:50:28.198586+0000","last_active":"2023-08-09T13:50:30.207465+0000","last_peered":"2023-08-09T13:50:30.207465+0000","last_clean":"2023-08-09T13:50:30.207465+0000","last_became_active":"2023-08-09T13:50:28.198247+0000","last_became_peered":"2023-08-09T13:50:28.198247+0000","last_unstale":"2023-08-09T13:50:30.207465+0000","last_undegraded":"2023-08-09T13:50:30.207465+0000","last_fullsized":"2023-08-09T13:50:30.207465+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:50:27.140775+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:50:27.140775+0000","last_clean_scrub_stamp":"2023-08-09T13:50:27.140775+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T21:31:44.778069+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.13","version":"0'0","reported_seq":3,"reported_epoch":89,"state":"creating+activating","last_fresh":"2023-08-09T13:50:29.220908+0000","last_change":"2023-08-09T13:50:29.220908+0000","last_active":"2023-08-09T13:50:28.150437+0000","last_peered":"2023-08-09T13:50:28.150437+0000","last_clean":"2023-08-09T13:50:28.150437+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-09T13:50:29.220908+0000","last_undegraded":"2023-08-09T13:50:29.220908+0000","last_fullsized":"2023-08-09T13:50:29.220908+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:50:28.150437+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:50:28.150437+0000","last_clean_scrub_stamp":"2023-08-09T13:50:28.150437+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T18:15:31.389069+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":["5","4"],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.12","version":"0'0","reported_seq":11,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-08-09T13:50:29.235230+0000","last_change":"2023-08-09T13:50:28.201822+0000","last_active":"2023-08-09T13:50:29.235230+0000","last_peered":"2023-08-09T13:50:29.235230+0000","last_clean":"2023-08-09T13:50:29.235230+0000","last_became_active":"2023-08-09T13:50:28.201187+0000","last_became_peered":"2023-08-09T13:50:28.201187+0000","last_unstale":"2023-08-09T13:50:29.235230+0000","last_undegraded":"2023-08-09T13:50:29.235230+0000","last_fullsized":"2023-08-09T13:50:29.235230+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:50:27.140775+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:50:27.140775+0000","last_clean_scrub_stamp":"2023-08-09T13:50:27.140775+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T23:52:03.500625+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.10","version":"0'0","reported_seq":11,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-08-09T13:50:30.405345+0000","last_change":"2023-08-09T13:50:29.246513+0000","last_active":"2023-08-09T13:50:30.405345+0000","last_peered":"2023-08-09T13:50:30.405345+0000","last_clean":"2023-08-09T13:50:30.405345+0000","last_became_active":"2023-08-09T13:50:29.246176+0000","last_became_peered":"2023-08-09T13:50:29.246176+0000","last_unstale":"2023-08-09T13:50:30.405345+0000","last_undegraded":"2023-08-09T13:50:30.405345+0000","last_fullsized":"2023-08-09T13:50:30.405345+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:50:28.150437+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:50:28.150437+0000","last_clean_scrub_stamp":"2023-08-09T13:50:28.150437+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T19:41:45.324636+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.11","version":"0'0","reported_seq":11,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-08-09T13:50:29.221870+0000","last_change":"2023-08-09T13:50:28.193484+0000","last_active":"2023-08-09T13:50:29.221870+0000","last_peered":"2023-08-09T13:50:29.221870+0000","last_clean":"2023-08-09T13:50:29.221870+0000","last_became_active":"2023-08-09T13:50:28.193098+0000","last_became_peered":"2023-08-09T13:50:28.193098+0000","last_unstale":"2023-08-09T13:50:29.221870+0000","last_undegraded":"2023-08-09T13:50:29.221870+0000","last_fullsized":"2023-08-09T13:50:29.221870+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:50:27.140775+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:50:27.140775+0000","last_clean_scrub_stamp":"2023-08-09T13:50:27.140775+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T21:57:08.338223+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.11","version":"0'0","reported_seq":3,"reported_epoch":89,"state":"creating+activating","last_fresh":"2023-08-09T13:50:29.224448+0000","last_change":"2023-08-09T13:50:29.224448+0000","last_active":"2023-08-09T13:50:28.150437+0000","last_peered":"2023-08-09T13:50:28.150437+0000","last_clean":"2023-08-09T13:50:28.150437+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-09T13:50:29.224448+0000","last_undegraded":"2023-08-09T13:50:29.224448+0000","last_fullsized":"2023-08-09T13:50:29.224448+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:50:28.150437+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:50:28.150437+0000","last_clean_scrub_stamp":"2023-08-09T13:50:28.150437+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T19:49:56.370072+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":["2","3"],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.10","version":"0'0","reported_seq":11,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-08-09T13:50:29.209343+0000","last_change":"2023-08-09T13:50:28.200670+0000","last_active":"2023-08-09T13:50:29.209343+0000","last_peered":"2023-08-09T13:50:29.209343+0000","last_clean":"2023-08-09T13:50:29.209343+0000","last_became_active":"2023-08-09T13:50:28.200419+0000","last_became_peered":"2023-08-09T13:50:28.200419+0000","last_unstale":"2023-08-09T13:50:29.209343+0000","last_undegraded":"2023-08-09T13:50:29.209343+0000","last_fullsized":"2023-08-09T13:50:29.209343+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:50:27.140775+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:50:27.140775+0000","last_clean_scrub_stamp":"2023-08-09T13:50:27.140775+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T15:23:31.632434+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.6","version":"0'0","reported_seq":11,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-08-09T13:50:30.406773+0000","last_change":"2023-08-09T13:50:29.229974+0000","last_active":"2023-08-09T13:50:30.406773+0000","last_peered":"2023-08-09T13:50:30.406773+0000","last_clean":"2023-08-09T13:50:30.406773+0000","last_became_active":"2023-08-09T13:50:29.229589+0000","last_became_peered":"2023-08-09T13:50:29.229589+0000","last_unstale":"2023-08-09T13:50:30.406773+0000","last_undegraded":"2023-08-09T13:50:30.406773+0000","last_fullsized":"2023-08-09T13:50:30.406773+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:50:28.150437+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:50:28.150437+0000","last_clean_scrub_stamp":"2023-08-09T13:50:28.150437+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T23:11:20.894653+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.7","version":"0'0","reported_seq":11,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-08-09T13:50:29.230633+0000","last_change":"2023-08-09T13:50:28.204446+0000","last_active":"2023-08-09T13:50:29.230633+0000","last_peered":"2023-08-09T13:50:29.230633+0000","last_clean":"2023-08-09T13:50:29.230633+0000","last_became_active":"2023-08-09T13:50:28.204093+0000","last_became_peered":"2023-08-09T13:50:28.204093+0000","last_unstale":"2023-08-09T13:50:29.230633+0000","last_undegraded":"2023-08-09T13:50:29.230633+0000","last_fullsized":"2023-08-09T13:50:29.230633+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:50:27.140775+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:50:27.140775+0000","last_clean_scrub_stamp":"2023-08-09T13:50:27.140775+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T22:56:15.821115+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.7","version":"0'0","reported_seq":3,"reported_epoch":89,"state":"creating+activating","last_fresh":"2023-08-09T13:50:29.223626+0000","last_change":"2023-08-09T13:50:29.223626+0000","last_active":"2023-08-09T13:50:28.150437+0000","last_peered":"2023-08-09T13:50:28.150437+0000","last_clean":"2023-08-09T13:50:28.150437+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-09T13:50:29.223626+0000","last_undegraded":"2023-08-09T13:50:29.223626+0000","last_fullsized":"2023-08-09T13:50:29.223626+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:50:28.150437+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:50:28.150437+0000","last_clean_scrub_stamp":"2023-08-09T13:50:28.150437+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T18:06:21.772376+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":["5","1"],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.6","version":"0'0","reported_seq":11,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-08-09T13:50:29.223926+0000","last_change":"2023-08-09T13:50:28.191955+0000","last_active":"2023-08-09T13:50:29.223926+0000","last_peered":"2023-08-09T13:50:29.223926+0000","last_clean":"2023-08-09T13:50:29.223926+0000","last_became_active":"2023-08-09T13:50:28.191264+0000","last_became_peered":"2023-08-09T13:50:28.191264+0000","last_unstale":"2023-08-09T13:50:29.223926+0000","last_undegraded":"2023-08-09T13:50:29.223926+0000","last_fullsized":"2023-08-09T13:50:29.223926+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:50:27.140775+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:50:27.140775+0000","last_clean_scrub_stamp":"2023-08-09T13:50:27.140775+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T18:00:04.982575+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.4","version":"0'0","reported_seq":3,"reported_epoch":89,"state":"creating+activating","last_fresh":"2023-08-09T13:50:29.216528+0000","last_change":"2023-08-09T13:50:29.216528+0000","last_active":"2023-08-09T13:50:28.150437+0000","last_peered":"2023-08-09T13:50:28.150437+0000","last_clean":"2023-08-09T13:50:28.150437+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-09T13:50:29.216528+0000","last_undegraded":"2023-08-09T13:50:29.216528+0000","last_fullsized":"2023-08-09T13:50:29.216528+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:50:28.150437+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:50:28.150437+0000","last_clean_scrub_stamp":"2023-08-09T13:50:28.150437+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T21:36:45.909480+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":["7","6"],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.5","version":"0'0","reported_seq":11,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-08-09T13:50:29.224819+0000","last_change":"2023-08-09T13:50:28.193912+0000","last_active":"2023-08-09T13:50:29.224819+0000","last_peered":"2023-08-09T13:50:29.224819+0000","last_clean":"2023-08-09T13:50:29.224819+0000","last_became_active":"2023-08-09T13:50:28.193602+0000","last_became_peered":"2023-08-09T13:50:28.193602+0000","last_unstale":"2023-08-09T13:50:29.224819+0000","last_undegraded":"2023-08-09T13:50:29.224819+0000","last_fullsized":"2023-08-09T13:50:29.224819+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:50:27.140775+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:50:27.140775+0000","last_clean_scrub_stamp":"2023-08-09T13:50:27.140775+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T19:25:18.625389+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.5","version":"0'0","reported_seq":8,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-08-09T13:50:29.237250+0000","last_change":"2023-08-09T13:50:29.237250+0000","last_active":"2023-08-09T13:50:29.237250+0000","last_peered":"2023-08-09T13:50:29.237250+0000","last_clean":"2023-08-09T13:50:29.237250+0000","last_became_active":"2023-08-09T13:50:29.236821+0000","last_became_peered":"2023-08-09T13:50:29.236821+0000","last_unstale":"2023-08-09T13:50:29.237250+0000","last_undegraded":"2023-08-09T13:50:29.237250+0000","last_fullsized":"2023-08-09T13:50:29.237250+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:50:28.150437+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:50:28.150437+0000","last_clean_scrub_stamp":"2023-08-09T13:50:28.150437+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T15:02:09.473638+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.4","version":"0'0","reported_seq":13,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-08-09T13:50:30.406255+0000","last_change":"2023-08-09T13:50:28.197934+0000","last_active":"2023-08-09T13:50:30.406255+0000","last_peered":"2023-08-09T13:50:30.406255+0000","last_clean":"2023-08-09T13:50:30.406255+0000","last_became_active":"2023-08-09T13:50:28.197639+0000","last_became_peered":"2023-08-09T13:50:28.197639+0000","last_unstale":"2023-08-09T13:50:30.406255+0000","last_undegraded":"2023-08-09T13:50:30.406255+0000","last_fullsized":"2023-08-09T13:50:30.406255+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:50:27.140775+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:50:27.140775+0000","last_clean_scrub_stamp":"2023-08-09T13:50:27.140775+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T23:10:53.423417+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.2","version":"0'0","reported_seq":3,"reported_epoch":89,"state":"creating+activating","last_fresh":"2023-08-09T13:50:29.217549+0000","last_change":"2023-08-09T13:50:29.217549+0000","last_active":"2023-08-09T13:50:28.150437+0000","last_peered":"2023-08-09T13:50:28.150437+0000","last_clean":"2023-08-09T13:50:28.150437+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-09T13:50:29.217549+0000","last_undegraded":"2023-08-09T13:50:29.217549+0000","last_fullsized":"2023-08-09T13:50:29.217549+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:50:28.150437+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:50:28.150437+0000","last_clean_scrub_stamp":"2023-08-09T13:50:28.150437+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-11T00:03:22.851529+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":["2","3"],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.3","version":"0'0","reported_seq":11,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-08-09T13:50:29.224120+0000","last_change":"2023-08-09T13:50:28.189872+0000","last_active":"2023-08-09T13:50:29.224120+0000","last_peered":"2023-08-09T13:50:29.224120+0000","last_clean":"2023-08-09T13:50:29.224120+0000","last_became_active":"2023-08-09T13:50:28.189588+0000","last_became_peered":"2023-08-09T13:50:28.189588+0000","last_unstale":"2023-08-09T13:50:29.224120+0000","last_undegraded":"2023-08-09T13:50:29.224120+0000","last_fullsized":"2023-08-09T13:50:29.224120+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:50:27.140775+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:50:27.140775+0000","last_clean_scrub_stamp":"2023-08-09T13:50:27.140775+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T16:47:26.737473+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.3","version":"0'0","reported_seq":8,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-08-09T13:50:29.250246+0000","last_change":"2023-08-09T13:50:29.250246+0000","last_active":"2023-08-09T13:50:29.250246+0000","last_peered":"2023-08-09T13:50:29.250246+0000","last_clean":"2023-08-09T13:50:29.250246+0000","last_became_active":"2023-08-09T13:50:29.249885+0000","last_became_peered":"2023-08-09T13:50:29.249885+0000","last_unstale":"2023-08-09T13:50:29.250246+0000","last_undegraded":"2023-08-09T13:50:29.250246+0000","last_fullsized":"2023-08-09T13:50:29.250246+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:50:28.150437+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:50:28.150437+0000","last_clean_scrub_stamp":"2023-08-09T13:50:28.150437+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T21:52:25.852199+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"10.2","version":"0'0","reported_seq":13,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-08-09T13:50:30.207205+0000","last_change":"2023-08-09T13:50:28.202029+0000","last_active":"2023-08-09T13:50:30.207205+0000","last_peered":"2023-08-09T13:50:30.207205+0000","last_clean":"2023-08-09T13:50:30.207205+0000","last_became_active":"2023-08-09T13:50:28.201720+0000","last_became_peered":"2023-08-09T13:50:28.201720+0000","last_unstale":"2023-08-09T13:50:30.207205+0000","last_undegraded":"2023-08-09T13:50:30.207205+0000","last_fullsized":"2023-08-09T13:50:30.207205+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:50:27.140775+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:50:27.140775+0000","last_clean_scrub_stamp":"2023-08-09T13:50:27.140775+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T16:25:36.557353+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.0","version":"0'0","reported_seq":11,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-08-09T13:50:30.207491+0000","last_change":"2023-08-09T13:50:29.233938+0000","last_active":"2023-08-09T13:50:30.207491+0000","last_peered":"2023-08-09T13:50:30.207491+0000","last_clean":"2023-08-09T13:50:30.207491+0000","last_became_active":"2023-08-09T13:50:29.233536+0000","last_became_peered":"2023-08-09T13:50:29.233536+0000","last_unstale":"2023-08-09T13:50:30.207491+0000","last_undegraded":"2023-08-09T13:50:30.207491+0000","last_fullsized":"2023-08-09T13:50:30.207491+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:50:28.150437+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:50:28.150437+0000","last_clean_scrub_stamp":"2023-08-09T13:50:28.150437+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T16:15:00.918668+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.1","version":"0'0","reported_seq":11,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-08-09T13:50:29.209627+0000","last_change":"2023-08-09T13:50:28.200080+0000","last_active":"2023-08-09T13:50:29.209627+0000","last_peered":"2023-08-09T13:50:29.209627+0000","last_clean":"2023-08-09T13:50:29.209627+0000","last_became_active":"2023-08-09T13:50:28.199831+0000","last_became_peered":"2023-08-09T13:50:28.199831+0000","last_unstale":"2023-08-09T13:50:29.209627+0000","last_undegraded":"2023-08-09T13:50:29.209627+0000","last_fullsized":"2023-08-09T13:50:29.209627+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:50:27.140775+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:50:27.140775+0000","last_clean_scrub_stamp":"2023-08-09T13:50:27.140775+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T15:18:11.293013+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.1","version":"0'0","reported_seq":8,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-08-09T13:50:29.249892+0000","last_change":"2023-08-09T13:50:29.249892+0000","last_active":"2023-08-09T13:50:29.249892+0000","last_peered":"2023-08-09T13:50:29.249892+0000","last_clean":"2023-08-09T13:50:29.249892+0000","last_became_active":"2023-08-09T13:50:29.242554+0000","last_became_peered":"2023-08-09T13:50:29.242554+0000","last_unstale":"2023-08-09T13:50:29.249892+0000","last_undegraded":"2023-08-09T13:50:29.249892+0000","last_fullsized":"2023-08-09T13:50:29.249892+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:50:28.150437+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:50:28.150437+0000","last_clean_scrub_stamp":"2023-08-09T13:50:28.150437+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T21:18:31.512056+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"10.0","version":"0'0","reported_seq":11,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-08-09T13:50:29.242250+0000","last_change":"2023-08-09T13:50:28.197319+0000","last_active":"2023-08-09T13:50:29.242250+0000","last_peered":"2023-08-09T13:50:29.242250+0000","last_clean":"2023-08-09T13:50:29.242250+0000","last_became_active":"2023-08-09T13:50:28.197031+0000","last_became_peered":"2023-08-09T13:50:28.197031+0000","last_unstale":"2023-08-09T13:50:29.242250+0000","last_undegraded":"2023-08-09T13:50:29.242250+0000","last_fullsized":"2023-08-09T13:50:29.242250+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:50:27.140775+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:50:27.140775+0000","last_clean_scrub_stamp":"2023-08-09T13:50:27.140775+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T16:15:05.176106+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.a","version":"0'0","reported_seq":3,"reported_epoch":89,"state":"creating+activating","last_fresh":"2023-08-09T13:50:29.212247+0000","last_change":"2023-08-09T13:50:29.212247+0000","last_active":"2023-08-09T13:50:28.150437+0000","last_peered":"2023-08-09T13:50:28.150437+0000","last_clean":"2023-08-09T13:50:28.150437+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-09T13:50:29.212247+0000","last_undegraded":"2023-08-09T13:50:29.212247+0000","last_fullsized":"2023-08-09T13:50:29.212247+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:50:28.150437+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:50:28.150437+0000","last_clean_scrub_stamp":"2023-08-09T13:50:28.150437+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T19:55:58.171731+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":["3","6"],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.b","version":"0'0","reported_seq":11,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-08-09T13:50:29.230951+0000","last_change":"2023-08-09T13:50:28.203783+0000","last_active":"2023-08-09T13:50:29.230951+0000","last_peered":"2023-08-09T13:50:29.230951+0000","last_clean":"2023-08-09T13:50:29.230951+0000","last_became_active":"2023-08-09T13:50:28.202391+0000","last_became_peered":"2023-08-09T13:50:28.202391+0000","last_unstale":"2023-08-09T13:50:29.230951+0000","last_undegraded":"2023-08-09T13:50:29.230951+0000","last_fullsized":"2023-08-09T13:50:29.230951+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:50:27.140775+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:50:27.140775+0000","last_clean_scrub_stamp":"2023-08-09T13:50:27.140775+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T21:39:38.493243+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":149,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-08-09T13:50:29.216433+0000","last_change":"2023-08-09T13:33:56.401406+0000","last_active":"2023-08-09T13:50:29.216433+0000","last_peered":"2023-08-09T13:50:29.216433+0000","last_clean":"2023-08-09T13:50:29.216433+0000","last_became_active":"2023-08-09T13:33:56.400487+0000","last_became_peered":"2023-08-09T13:33:56.400487+0000","last_unstale":"2023-08-09T13:50:29.216433+0000","last_undegraded":"2023-08-09T13:50:29.216433+0000","last_fullsized":"2023-08-09T13:50:29.216433+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:33:54.845869+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:33:54.845869+0000","last_clean_scrub_stamp":"2023-08-09T13:33:54.845869+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-11T00:46:00.165825+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"11.b","version":"0'0","reported_seq":3,"reported_epoch":89,"state":"creating+activating","last_fresh":"2023-08-09T13:50:29.211663+0000","last_change":"2023-08-09T13:50:29.211663+0000","last_active":"2023-08-09T13:50:28.150437+0000","last_peered":"2023-08-09T13:50:28.150437+0000","last_clean":"2023-08-09T13:50:28.150437+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-09T13:50:29.211663+0000","last_undegraded":"2023-08-09T13:50:29.211663+0000","last_fullsized":"2023-08-09T13:50:29.211663+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:50:28.150437+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:50:28.150437+0000","last_clean_scrub_stamp":"2023-08-09T13:50:28.150437+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-11T01:04:03.965638+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":["3","1"],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.a","version":"0'0","reported_seq":11,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-08-09T13:50:29.230486+0000","last_change":"2023-08-09T13:50:28.205148+0000","last_active":"2023-08-09T13:50:29.230486+0000","last_peered":"2023-08-09T13:50:29.230486+0000","last_clean":"2023-08-09T13:50:29.230486+0000","last_became_active":"2023-08-09T13:50:28.204528+0000","last_became_peered":"2023-08-09T13:50:28.204528+0000","last_unstale":"2023-08-09T13:50:29.230486+0000","last_undegraded":"2023-08-09T13:50:29.230486+0000","last_fullsized":"2023-08-09T13:50:29.230486+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:50:27.140775+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:50:27.140775+0000","last_clean_scrub_stamp":"2023-08-09T13:50:27.140775+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T18:00:04.982575+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":148,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-08-09T13:50:29.222206+0000","last_change":"2023-08-09T13:33:56.401123+0000","last_active":"2023-08-09T13:50:29.222206+0000","last_peered":"2023-08-09T13:50:29.222206+0000","last_clean":"2023-08-09T13:50:29.222206+0000","last_became_active":"2023-08-09T13:33:56.399933+0000","last_became_peered":"2023-08-09T13:33:56.399933+0000","last_unstale":"2023-08-09T13:50:29.222206+0000","last_undegraded":"2023-08-09T13:50:29.222206+0000","last_fullsized":"2023-08-09T13:50:29.222206+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:33:54.845869+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:33:54.845869+0000","last_clean_scrub_stamp":"2023-08-09T13:33:54.845869+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T17:05:05.384552+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.8","version":"0'0","reported_seq":3,"reported_epoch":89,"state":"creating+activating","last_fresh":"2023-08-09T13:50:29.213241+0000","last_change":"2023-08-09T13:50:29.213241+0000","last_active":"2023-08-09T13:50:28.150437+0000","last_peered":"2023-08-09T13:50:28.150437+0000","last_clean":"2023-08-09T13:50:28.150437+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-09T13:50:29.213241+0000","last_undegraded":"2023-08-09T13:50:29.213241+0000","last_fullsized":"2023-08-09T13:50:29.213241+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:50:28.150437+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:50:28.150437+0000","last_clean_scrub_stamp":"2023-08-09T13:50:28.150437+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T22:09:32.464173+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":["3","6"],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.9","version":"0'0","reported_seq":11,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-08-09T13:50:29.222528+0000","last_change":"2023-08-09T13:50:28.192232+0000","last_active":"2023-08-09T13:50:29.222528+0000","last_peered":"2023-08-09T13:50:29.222528+0000","last_clean":"2023-08-09T13:50:29.222528+0000","last_became_active":"2023-08-09T13:50:28.191350+0000","last_became_peered":"2023-08-09T13:50:28.191350+0000","last_unstale":"2023-08-09T13:50:29.222528+0000","last_undegraded":"2023-08-09T13:50:29.222528+0000","last_fullsized":"2023-08-09T13:50:29.222528+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:50:27.140775+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:50:27.140775+0000","last_clean_scrub_stamp":"2023-08-09T13:50:27.140775+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-11T00:33:31.648004+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.2","version":"0'0","reported_seq":148,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-08-09T13:50:29.215918+0000","last_change":"2023-08-09T13:33:56.402794+0000","last_active":"2023-08-09T13:50:29.215918+0000","last_peered":"2023-08-09T13:50:29.215918+0000","last_clean":"2023-08-09T13:50:29.215918+0000","last_became_active":"2023-08-09T13:33:56.402038+0000","last_became_peered":"2023-08-09T13:33:56.402038+0000","last_unstale":"2023-08-09T13:50:29.215918+0000","last_undegraded":"2023-08-09T13:50:29.215918+0000","last_fullsized":"2023-08-09T13:50:29.215918+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:33:54.845869+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:33:54.845869+0000","last_clean_scrub_stamp":"2023-08-09T13:33:54.845869+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T16:35:28.133841+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"11.9","version":"0'0","reported_seq":3,"reported_epoch":89,"state":"creating+activating","last_fresh":"2023-08-09T13:50:29.211902+0000","last_change":"2023-08-09T13:50:29.211902+0000","last_active":"2023-08-09T13:50:28.150437+0000","last_peered":"2023-08-09T13:50:28.150437+0000","last_clean":"2023-08-09T13:50:28.150437+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-09T13:50:29.211902+0000","last_undegraded":"2023-08-09T13:50:29.211902+0000","last_fullsized":"2023-08-09T13:50:29.211902+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:50:28.150437+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:50:28.150437+0000","last_clean_scrub_stamp":"2023-08-09T13:50:28.150437+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T15:32:31.232979+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":["3","6"],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.8","version":"0'0","reported_seq":13,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-08-09T13:50:30.406875+0000","last_change":"2023-08-09T13:50:28.190965+0000","last_active":"2023-08-09T13:50:30.406875+0000","last_peered":"2023-08-09T13:50:30.406875+0000","last_clean":"2023-08-09T13:50:30.406875+0000","last_became_active":"2023-08-09T13:50:28.190696+0000","last_became_peered":"2023-08-09T13:50:28.190696+0000","last_unstale":"2023-08-09T13:50:30.406875+0000","last_undegraded":"2023-08-09T13:50:30.406875+0000","last_fullsized":"2023-08-09T13:50:30.406875+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:50:27.140775+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:50:27.140775+0000","last_clean_scrub_stamp":"2023-08-09T13:50:27.140775+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T22:28:09.956793+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.3","version":"0'0","reported_seq":150,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-08-09T13:50:30.207472+0000","last_change":"2023-08-09T13:33:55.886749+0000","last_active":"2023-08-09T13:50:30.207472+0000","last_peered":"2023-08-09T13:50:30.207472+0000","last_clean":"2023-08-09T13:50:30.207472+0000","last_became_active":"2023-08-09T13:33:55.886116+0000","last_became_peered":"2023-08-09T13:33:55.886116+0000","last_unstale":"2023-08-09T13:50:30.207472+0000","last_undegraded":"2023-08-09T13:50:30.207472+0000","last_fullsized":"2023-08-09T13:50:30.207472+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:33:54.845869+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:33:54.845869+0000","last_clean_scrub_stamp":"2023-08-09T13:33:54.845869+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T23:29:59.861309+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"77'161","reported_seq":455,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-08-09T13:50:29.216076+0000","last_change":"2023-08-09T13:33:55.886345+0000","last_active":"2023-08-09T13:50:29.216076+0000","last_peered":"2023-08-09T13:50:29.216076+0000","last_clean":"2023-08-09T13:50:29.216076+0000","last_became_active":"2023-08-09T13:33:55.885781+0000","last_became_peered":"2023-08-09T13:33:55.885781+0000","last_unstale":"2023-08-09T13:50:29.216076+0000","last_undegraded":"2023-08-09T13:50:29.216076+0000","last_fullsized":"2023-08-09T13:50:29.216076+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:33:54.845869+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:33:54.845869+0000","last_clean_scrub_stamp":"2023-08-09T13:33:54.845869+0000","objects_scrubbed":0,"log_size":161,"log_dups_size":0,"ondisk_log_size":161,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T19:49:09.757332+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":500,"num_read_kb":924,"num_write":239,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"11.e","version":"0'0","reported_seq":9,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-08-09T13:50:29.232907+0000","last_change":"2023-08-09T13:50:29.227243+0000","last_active":"2023-08-09T13:50:29.232907+0000","last_peered":"2023-08-09T13:50:29.232907+0000","last_clean":"2023-08-09T13:50:29.232907+0000","last_became_active":"2023-08-09T13:50:29.226756+0000","last_became_peered":"2023-08-09T13:50:29.226756+0000","last_unstale":"2023-08-09T13:50:29.232907+0000","last_undegraded":"2023-08-09T13:50:29.232907+0000","last_fullsized":"2023-08-09T13:50:29.232907+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:50:28.150437+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:50:28.150437+0000","last_clean_scrub_stamp":"2023-08-09T13:50:28.150437+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T18:35:12.621508+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"10.f","version":"0'0","reported_seq":11,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-08-09T13:50:29.230485+0000","last_change":"2023-08-09T13:50:28.205120+0000","last_active":"2023-08-09T13:50:29.230485+0000","last_peered":"2023-08-09T13:50:29.230485+0000","last_clean":"2023-08-09T13:50:29.230485+0000","last_became_active":"2023-08-09T13:50:28.204771+0000","last_became_peered":"2023-08-09T13:50:28.204771+0000","last_unstale":"2023-08-09T13:50:29.230485+0000","last_undegraded":"2023-08-09T13:50:29.230485+0000","last_fullsized":"2023-08-09T13:50:29.230485+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:50:27.140775+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:50:27.140775+0000","last_clean_scrub_stamp":"2023-08-09T13:50:27.140775+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T17:34:29.132908+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":158,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-08-09T13:50:29.211123+0000","last_change":"2023-08-09T13:33:55.889565+0000","last_active":"2023-08-09T13:50:29.211123+0000","last_peered":"2023-08-09T13:50:29.211123+0000","last_clean":"2023-08-09T13:50:29.211123+0000","last_became_active":"2023-08-09T13:33:55.889164+0000","last_became_peered":"2023-08-09T13:33:55.889164+0000","last_unstale":"2023-08-09T13:50:29.211123+0000","last_undegraded":"2023-08-09T13:50:29.211123+0000","last_fullsized":"2023-08-09T13:50:29.211123+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:33:54.845869+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:33:54.845869+0000","last_clean_scrub_stamp":"2023-08-09T13:33:54.845869+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T17:48:30.355598+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.f","version":"0'0","reported_seq":8,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-08-09T13:50:29.238559+0000","last_change":"2023-08-09T13:50:29.238559+0000","last_active":"2023-08-09T13:50:29.238559+0000","last_peered":"2023-08-09T13:50:29.238559+0000","last_clean":"2023-08-09T13:50:29.238559+0000","last_became_active":"2023-08-09T13:50:29.238287+0000","last_became_peered":"2023-08-09T13:50:29.238287+0000","last_unstale":"2023-08-09T13:50:29.238559+0000","last_undegraded":"2023-08-09T13:50:29.238559+0000","last_fullsized":"2023-08-09T13:50:29.238559+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:50:28.150437+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:50:28.150437+0000","last_clean_scrub_stamp":"2023-08-09T13:50:28.150437+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T16:41:09.645304+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.e","version":"0'0","reported_seq":11,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-08-09T13:50:29.211102+0000","last_change":"2023-08-09T13:50:28.206561+0000","last_active":"2023-08-09T13:50:29.211102+0000","last_peered":"2023-08-09T13:50:29.211102+0000","last_clean":"2023-08-09T13:50:29.211102+0000","last_became_active":"2023-08-09T13:50:28.206264+0000","last_became_peered":"2023-08-09T13:50:28.206264+0000","last_unstale":"2023-08-09T13:50:29.211102+0000","last_undegraded":"2023-08-09T13:50:29.211102+0000","last_fullsized":"2023-08-09T13:50:29.211102+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:50:27.140775+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:50:27.140775+0000","last_clean_scrub_stamp":"2023-08-09T13:50:27.140775+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T15:29:27.312440+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":148,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-08-09T13:50:29.210681+0000","last_change":"2023-08-09T13:33:56.404236+0000","last_active":"2023-08-09T13:50:29.210681+0000","last_peered":"2023-08-09T13:50:29.210681+0000","last_clean":"2023-08-09T13:50:29.210681+0000","last_became_active":"2023-08-09T13:33:56.402485+0000","last_became_peered":"2023-08-09T13:33:56.402485+0000","last_unstale":"2023-08-09T13:50:29.210681+0000","last_undegraded":"2023-08-09T13:50:29.210681+0000","last_fullsized":"2023-08-09T13:50:29.210681+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:33:54.845869+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:33:54.845869+0000","last_clean_scrub_stamp":"2023-08-09T13:33:54.845869+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T13:49:18.686986+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.c","version":"0'0","reported_seq":11,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-08-09T13:50:30.207708+0000","last_change":"2023-08-09T13:50:29.230700+0000","last_active":"2023-08-09T13:50:30.207708+0000","last_peered":"2023-08-09T13:50:30.207708+0000","last_clean":"2023-08-09T13:50:30.207708+0000","last_became_active":"2023-08-09T13:50:29.230389+0000","last_became_peered":"2023-08-09T13:50:29.230389+0000","last_unstale":"2023-08-09T13:50:30.207708+0000","last_undegraded":"2023-08-09T13:50:30.207708+0000","last_fullsized":"2023-08-09T13:50:30.207708+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:50:28.150437+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:50:28.150437+0000","last_clean_scrub_stamp":"2023-08-09T13:50:28.150437+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-11T00:40:44.937679+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.d","version":"0'0","reported_seq":11,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-08-09T13:50:29.227255+0000","last_change":"2023-08-09T13:50:28.199480+0000","last_active":"2023-08-09T13:50:29.227255+0000","last_peered":"2023-08-09T13:50:29.227255+0000","last_clean":"2023-08-09T13:50:29.227255+0000","last_became_active":"2023-08-09T13:50:28.199211+0000","last_became_peered":"2023-08-09T13:50:28.199211+0000","last_unstale":"2023-08-09T13:50:29.227255+0000","last_undegraded":"2023-08-09T13:50:29.227255+0000","last_fullsized":"2023-08-09T13:50:29.227255+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:50:27.140775+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:50:27.140775+0000","last_clean_scrub_stamp":"2023-08-09T13:50:27.140775+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T18:35:43.990648+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":146,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-08-09T13:50:29.237073+0000","last_change":"2023-08-09T13:33:56.401781+0000","last_active":"2023-08-09T13:50:29.237073+0000","last_peered":"2023-08-09T13:50:29.237073+0000","last_clean":"2023-08-09T13:50:29.237073+0000","last_became_active":"2023-08-09T13:33:56.400336+0000","last_became_peered":"2023-08-09T13:33:56.400336+0000","last_unstale":"2023-08-09T13:50:29.237073+0000","last_undegraded":"2023-08-09T13:50:29.237073+0000","last_fullsized":"2023-08-09T13:50:29.237073+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:33:54.845869+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:33:54.845869+0000","last_clean_scrub_stamp":"2023-08-09T13:33:54.845869+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-11T01:01:47.058457+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.d","version":"0'0","reported_seq":3,"reported_epoch":89,"state":"creating+activating","last_fresh":"2023-08-09T13:50:29.220533+0000","last_change":"2023-08-09T13:50:29.220533+0000","last_active":"2023-08-09T13:50:28.150437+0000","last_peered":"2023-08-09T13:50:28.150437+0000","last_clean":"2023-08-09T13:50:28.150437+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-09T13:50:29.220533+0000","last_undegraded":"2023-08-09T13:50:29.220533+0000","last_fullsized":"2023-08-09T13:50:29.220533+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:50:28.150437+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:50:28.150437+0000","last_clean_scrub_stamp":"2023-08-09T13:50:28.150437+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T17:52:40.304270+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":["2","4"],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.c","version":"0'0","reported_seq":11,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-08-09T13:50:29.216926+0000","last_change":"2023-08-09T13:50:28.203868+0000","last_active":"2023-08-09T13:50:29.216926+0000","last_peered":"2023-08-09T13:50:29.216926+0000","last_clean":"2023-08-09T13:50:29.216926+0000","last_became_active":"2023-08-09T13:50:28.203323+0000","last_became_peered":"2023-08-09T13:50:28.203323+0000","last_unstale":"2023-08-09T13:50:29.216926+0000","last_undegraded":"2023-08-09T13:50:29.216926+0000","last_fullsized":"2023-08-09T13:50:29.216926+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:50:27.140775+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:50:27.140775+0000","last_clean_scrub_stamp":"2023-08-09T13:50:27.140775+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T16:53:02.601172+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":146,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-08-09T13:50:29.230723+0000","last_change":"2023-08-09T13:33:55.886389+0000","last_active":"2023-08-09T13:50:29.230723+0000","last_peered":"2023-08-09T13:50:29.230723+0000","last_clean":"2023-08-09T13:50:29.230723+0000","last_became_active":"2023-08-09T13:33:55.885638+0000","last_became_peered":"2023-08-09T13:33:55.885638+0000","last_unstale":"2023-08-09T13:50:29.230723+0000","last_undegraded":"2023-08-09T13:50:29.230723+0000","last_fullsized":"2023-08-09T13:50:29.230723+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:33:54.845869+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:33:54.845869+0000","last_clean_scrub_stamp":"2023-08-09T13:33:54.845869+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T22:08:50.990029+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]}],"pool_stats":[{"poolid":11,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":10,"num_pg":32,"stat_sum":{"num_bytes":1646,"num_objects":6,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9,"num_write_kb":10,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":2228,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":9,"ondisk_log_size":9,"up":64,"acting":64,"num_store_stats":8},{"poolid":2,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":500,"num_read_kb":924,"num_write":239,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":161,"ondisk_log_size":161,"up":2,"acting":2,"num_store_stats":2},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":8}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834575047,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":36840,"kb_used_data":1624,"kb_used_omap":1,"kb_used_meta":35198,"kb_avail":94335000,"statfs":{"total":96636764160,"available":96599040000,"internally_reserved":0,"allocated":1662976,"data_stored":1159636,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1781,"internal_metadata":36043019},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Aug 9 13:50:16 2023","interfaces":[{"interface":"back","average":{"1min":1.181,"5min":1.222,"15min":1.232},"min":{"1min":0.69499999999999995,"5min":0.63400000000000001,"15min":0.61099999999999999},"max":{"1min":1.633,"5min":1.8919999999999999,"15min":2.1280000000000001},"last":1.2569999999999999},{"interface":"front","average":{"1min":1.1819999999999999,"5min":1.232,"15min":1.24},"min":{"1min":0.72399999999999998,"5min":0.72399999999999998,"15min":0.65000000000000002},"max":{"1min":1.6100000000000001,"5min":1.839,"15min":2.0099999999999998},"last":1.1259999999999999}]},{"osd":1,"last update":"Wed Aug 9 13:50:16 2023","interfaces":[{"interface":"back","average":{"1min":1.2010000000000001,"5min":1.2450000000000001,"15min":1.256},"min":{"1min":0.72799999999999998,"5min":0.72799999999999998,"15min":0.59799999999999998},"max":{"1min":1.8,"5min":1.8,"15min":1.9470000000000001},"last":1.403},{"interface":"front","average":{"1min":1.2569999999999999,"5min":1.248,"15min":1.2869999999999999},"min":{"1min":0.83099999999999996,"5min":0.59299999999999997,"15min":0.59299999999999997},"max":{"1min":1.726,"5min":1.9410000000000001,"15min":1.9410000000000001},"last":1.327}]},{"osd":2,"last update":"Wed Aug 9 13:50:16 2023","interfaces":[{"interface":"back","average":{"1min":1.278,"5min":1.2470000000000001,"15min":1.2430000000000001},"min":{"1min":0.81000000000000005,"5min":0.65700000000000003,"15min":0.65700000000000003},"max":{"1min":1.742,"5min":2.0129999999999999,"15min":2.0129999999999999},"last":0.99399999999999999},{"interface":"front","average":{"1min":1.2889999999999999,"5min":1.323,"15min":1.3759999999999999},"min":{"1min":0.92600000000000005,"5min":0.80900000000000005,"15min":0.67800000000000005},"max":{"1min":1.7130000000000001,"5min":1.8740000000000001,"15min":2.4140000000000001},"last":1.623}]},{"osd":3,"last update":"Wed Aug 9 13:50:16 2023","interfaces":[{"interface":"back","average":{"1min":1.2529999999999999,"5min":1.3180000000000001,"15min":1.331},"min":{"1min":0.98799999999999999,"5min":0.86799999999999999,"15min":0.77500000000000002},"max":{"1min":1.7809999999999999,"5min":1.97,"15min":1.97},"last":1.3680000000000001},{"interface":"front","average":{"1min":1.3740000000000001,"5min":1.419,"15min":1.4159999999999999},"min":{"1min":0.93899999999999995,"5min":0.88300000000000001,"15min":0.82099999999999995},"max":{"1min":1.8160000000000001,"5min":2.0790000000000002,"15min":2.2480000000000002},"last":1.526}]},{"osd":4,"last update":"Wed Aug 9 13:50:16 2023","interfaces":[{"interface":"back","average":{"1min":1.343,"5min":1.2649999999999999,"15min":1.2769999999999999},"min":{"1min":0.86399999999999999,"5min":0.59499999999999997,"15min":0.59499999999999997},"max":{"1min":1.772,"5min":2.032,"15min":2.343},"last":0.82499999999999996},{"interface":"front","average":{"1min":1.296,"5min":1.3160000000000001,"15min":1.339},"min":{"1min":0.93600000000000005,"5min":0.77100000000000002,"15min":0.56799999999999995},"max":{"1min":1.6739999999999999,"5min":2.1419999999999999,"15min":2.1419999999999999},"last":1.2010000000000001}]},{"osd":5,"last update":"Wed Aug 9 13:50:16 2023","interfaces":[{"interface":"back","average":{"1min":1.4430000000000001,"5min":1.4350000000000001,"15min":1.419},"min":{"1min":1.048,"5min":0.88600000000000001,"15min":0.71499999999999997},"max":{"1min":1.9039999999999999,"5min":2.1480000000000001,"15min":2.6749999999999998},"last":1.6879999999999999},{"interface":"front","average":{"1min":1.4810000000000001,"5min":1.4319999999999999,"15min":1.4510000000000001},"min":{"1min":0.95999999999999996,"5min":0.85699999999999998,"15min":0.69199999999999995},"max":{"1min":1.877,"5min":2.113,"15min":2.9569999999999999},"last":1.4510000000000001}]},{"osd":6,"last update":"Wed Aug 9 13:50:16 2023","interfaces":[{"interface":"back","average":{"1min":1.6020000000000001,"5min":1.518,"15min":1.5489999999999999},"min":{"1min":1.2,"5min":0.85499999999999998,"15min":0.85499999999999998},"max":{"1min":1.8879999999999999,"5min":2.2069999999999999,"15min":2.8149999999999999},"last":1.575},{"interface":"front","average":{"1min":1.4139999999999999,"5min":1.4570000000000001,"15min":1.474},"min":{"1min":1.111,"5min":0.90600000000000003,"15min":0.60799999999999998},"max":{"1min":1.9419999999999999,"5min":2.1320000000000001,"15min":2.2530000000000001},"last":1.4910000000000001}]}]},{"osd":6,"up_from":13,"seq":55834575047,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32156,"kb_used_data":1040,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":94339684,"statfs":{"total":96636764160,"available":96603836416,"internally_reserved":0,"allocated":1064960,"data_stored":567907,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1781,"internal_metadata":31848715},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Aug 9 13:50:20 2023","interfaces":[{"interface":"back","average":{"1min":1.1160000000000001,"5min":1.143,"15min":1.161},"min":{"1min":0.75,"5min":0.72499999999999998,"15min":0.56599999999999995},"max":{"1min":1.5489999999999999,"5min":1.633,"15min":1.9870000000000001},"last":1.359},{"interface":"front","average":{"1min":1.21,"5min":1.532,"15min":1.3080000000000001},"min":{"1min":0.79300000000000004,"5min":0.79300000000000004,"15min":0.76000000000000001},"max":{"1min":1.6519999999999999,"5min":28.036000000000001,"15min":28.036000000000001},"last":1.4550000000000001}]},{"osd":1,"last update":"Wed Aug 9 13:50:20 2023","interfaces":[{"interface":"back","average":{"1min":1.147,"5min":1.272,"15min":1.226},"min":{"1min":0.70399999999999996,"5min":0.70399999999999996,"15min":0.65500000000000003},"max":{"1min":1.659,"5min":7.742,"15min":7.742},"last":1.256},{"interface":"front","average":{"1min":1.252,"5min":1.2849999999999999,"15min":1.276},"min":{"1min":0.84199999999999997,"5min":0.84199999999999997,"15min":0.68100000000000005},"max":{"1min":1.8899999999999999,"5min":1.8899999999999999,"15min":1.8899999999999999},"last":0.89100000000000001}]},{"osd":2,"last update":"Wed Aug 9 13:50:20 2023","interfaces":[{"interface":"back","average":{"1min":1.2989999999999999,"5min":1.3480000000000001,"15min":1.3220000000000001},"min":{"1min":0.877,"5min":0.81999999999999995,"15min":0.68700000000000006},"max":{"1min":1.73,"5min":4.0640000000000001,"15min":4.0640000000000001},"last":1.151},{"interface":"front","average":{"1min":1.244,"5min":1.4430000000000001,"15min":1.3460000000000001},"min":{"1min":0.76800000000000002,"5min":0.70999999999999996,"15min":0.70999999999999996},"max":{"1min":2.0390000000000001,"5min":13.154999999999999,"15min":13.154999999999999},"last":0.95399999999999996}]},{"osd":3,"last update":"Wed Aug 9 13:50:20 2023","interfaces":[{"interface":"back","average":{"1min":1.3939999999999999,"5min":1.52,"15min":1.419},"min":{"1min":0.73099999999999998,"5min":0.73099999999999998,"15min":0.73099999999999998},"max":{"1min":2.2519999999999998,"5min":13.349,"15min":13.349},"last":1.298},{"interface":"front","average":{"1min":1.258,"5min":1.3620000000000001,"15min":1.325},"min":{"1min":0.88400000000000001,"5min":0.76400000000000001,"15min":0.74299999999999999},"max":{"1min":1.8560000000000001,"5min":7.9029999999999996,"15min":7.9029999999999996},"last":1.397}]},{"osd":4,"last update":"Wed Aug 9 13:50:20 2023","interfaces":[{"interface":"back","average":{"1min":1.302,"5min":1.335,"15min":1.28},"min":{"1min":0.90700000000000003,"5min":0.78500000000000003,"15min":0.64300000000000002},"max":{"1min":1.992,"5min":5.2290000000000001,"15min":5.2290000000000001},"last":1.486},{"interface":"front","average":{"1min":1.4019999999999999,"5min":1.498,"15min":1.389},"min":{"1min":0.88900000000000001,"5min":0.78700000000000003,"15min":0.69099999999999995},"max":{"1min":1.734,"5min":12.1,"15min":12.1},"last":1.0860000000000001}]},{"osd":5,"last update":"Wed Aug 9 13:50:20 2023","interfaces":[{"interface":"back","average":{"1min":1.3169999999999999,"5min":1.4399999999999999,"15min":1.357},"min":{"1min":0.93999999999999995,"5min":0.83799999999999997,"15min":0.65400000000000003},"max":{"1min":2.1139999999999999,"5min":8.1899999999999995,"15min":8.1899999999999995},"last":1.0289999999999999},{"interface":"front","average":{"1min":1.5649999999999999,"5min":1.6639999999999999,"15min":1.589},"min":{"1min":1.2010000000000001,"5min":1.2010000000000001,"15min":0.83799999999999997},"max":{"1min":2.2000000000000002,"5min":8.8889999999999993,"15min":8.8889999999999993},"last":1.4299999999999999}]},{"osd":7,"last update":"Wed Aug 9 13:50:20 2023","interfaces":[{"interface":"back","average":{"1min":1.577,"5min":1.6970000000000001,"15min":1.5880000000000001},"min":{"1min":1.1850000000000001,"5min":1.0029999999999999,"15min":0.62},"max":{"1min":1.8240000000000001,"5min":9.3320000000000007,"15min":9.3320000000000007},"last":1.5289999999999999},{"interface":"front","average":{"1min":1.4410000000000001,"5min":1.6060000000000001,"15min":1.548},"min":{"1min":0.93899999999999995,"5min":0.88500000000000001,"15min":0.88500000000000001},"max":{"1min":1.9470000000000001,"5min":8.0399999999999991,"15min":8.0399999999999991},"last":1.5029999999999999}]}]},{"osd":5,"up_from":13,"seq":55834575047,"num_pgs":16,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":36260,"kb_used_data":1044,"kb_used_omap":1,"kb_used_meta":35198,"kb_avail":94335580,"statfs":{"total":96636764160,"available":96599633920,"internally_reserved":0,"allocated":1069056,"data_stored":569268,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1781,"internal_metadata":36043019},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Aug 9 13:50:19 2023","interfaces":[{"interface":"back","average":{"1min":1.2509999999999999,"5min":1.1759999999999999,"15min":1.179},"min":{"1min":0.63500000000000001,"5min":0.63500000000000001,"15min":0.63500000000000001},"max":{"1min":1.518,"5min":2.1869999999999998,"15min":2.1869999999999998},"last":1.1020000000000001},{"interface":"front","average":{"1min":1.1719999999999999,"5min":1.2,"15min":1.198},"min":{"1min":0.68600000000000005,"5min":0.68600000000000005,"15min":0.64200000000000002},"max":{"1min":1.579,"5min":1.6619999999999999,"15min":1.7989999999999999},"last":1.1950000000000001}]},{"osd":1,"last update":"Wed Aug 9 13:50:19 2023","interfaces":[{"interface":"back","average":{"1min":1.165,"5min":1.1990000000000001,"15min":1.2190000000000001},"min":{"1min":0.81599999999999995,"5min":0.73099999999999998,"15min":0.64100000000000001},"max":{"1min":1.631,"5min":1.6739999999999999,"15min":1.736},"last":1.0049999999999999},{"interface":"front","average":{"1min":1.264,"5min":1.296,"15min":1.2769999999999999},"min":{"1min":0.93300000000000005,"5min":0.66900000000000004,"15min":0.66900000000000004},"max":{"1min":1.6699999999999999,"5min":1.887,"15min":1.887},"last":0.84999999999999998}]},{"osd":2,"last update":"Wed Aug 9 13:50:19 2023","interfaces":[{"interface":"back","average":{"1min":1.2410000000000001,"5min":1.2569999999999999,"15min":1.2849999999999999},"min":{"1min":0.98199999999999998,"5min":0.68999999999999995,"15min":0.61599999999999999},"max":{"1min":1.5760000000000001,"5min":1.756,"15min":1.899},"last":1.4670000000000001},{"interface":"front","average":{"1min":1.3049999999999999,"5min":1.302,"15min":1.3149999999999999},"min":{"1min":0.78400000000000003,"5min":0.78400000000000003,"15min":0.66800000000000004},"max":{"1min":1.6599999999999999,"5min":1.7190000000000001,"15min":1.7969999999999999},"last":1.1419999999999999}]},{"osd":3,"last update":"Wed Aug 9 13:50:19 2023","interfaces":[{"interface":"back","average":{"1min":1.365,"5min":1.3720000000000001,"15min":1.3600000000000001},"min":{"1min":0.95399999999999996,"5min":0.95399999999999996,"15min":0.59799999999999998},"max":{"1min":1.6890000000000001,"5min":2.1840000000000002,"15min":2.2690000000000001},"last":1.3160000000000001},{"interface":"front","average":{"1min":1.4259999999999999,"5min":1.4379999999999999,"15min":1.4139999999999999},"min":{"1min":1.087,"5min":0.94899999999999995,"15min":0.70999999999999996},"max":{"1min":1.73,"5min":2.3780000000000001,"15min":2.3780000000000001},"last":1.3600000000000001}]},{"osd":4,"last update":"Wed Aug 9 13:50:19 2023","interfaces":[{"interface":"back","average":{"1min":1.262,"5min":1.2729999999999999,"15min":1.2889999999999999},"min":{"1min":0.92700000000000005,"5min":0.84699999999999998,"15min":0.56599999999999995},"max":{"1min":1.673,"5min":2.0800000000000001,"15min":2.121},"last":1.2789999999999999},{"interface":"front","average":{"1min":1.3919999999999999,"5min":1.3400000000000001,"15min":1.3779999999999999},"min":{"1min":0.81999999999999995,"5min":0.76100000000000001,"15min":0.58599999999999997},"max":{"1min":1.944,"5min":2.1230000000000002,"15min":2.3679999999999999},"last":1.403}]},{"osd":6,"last update":"Wed Aug 9 13:50:19 2023","interfaces":[{"interface":"back","average":{"1min":1.4299999999999999,"5min":1.3520000000000001,"15min":1.391},"min":{"1min":0.99099999999999999,"5min":0.90300000000000002,"15min":0.66700000000000004},"max":{"1min":1.9810000000000001,"5min":2.306,"15min":2.306},"last":1.5609999999999999},{"interface":"front","average":{"1min":1.49,"5min":1.4399999999999999,"15min":1.4710000000000001},"min":{"1min":1.1220000000000001,"5min":1.034,"15min":0.71199999999999997},"max":{"1min":1.827,"5min":2.2320000000000002,"15min":2.2320000000000002},"last":1.343}]},{"osd":7,"last update":"Wed Aug 9 13:50:19 2023","interfaces":[{"interface":"back","average":{"1min":1.577,"5min":1.5580000000000001,"15min":1.5529999999999999},"min":{"1min":1.1160000000000001,"5min":1.0589999999999999,"15min":0.70899999999999996},"max":{"1min":1.9610000000000001,"5min":2.452,"15min":2.4729999999999999},"last":1.5980000000000001},{"interface":"front","average":{"1min":1.611,"5min":1.585,"15min":1.6080000000000001},"min":{"1min":1.2110000000000001,"5min":1.163,"15min":0.77600000000000002},"max":{"1min":2.097,"5min":2.1070000000000002,"15min":2.3820000000000001},"last":1.6499999999999999}]}]},{"osd":4,"up_from":13,"seq":55834575047,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32160,"kb_used_data":1044,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":94339680,"statfs":{"total":96636764160,"available":96603832320,"internally_reserved":0,"allocated":1069056,"data_stored":569268,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1781,"internal_metadata":31848715},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Aug 9 13:50:21 2023","interfaces":[{"interface":"back","average":{"1min":1.2330000000000001,"5min":1.2390000000000001,"15min":1.2030000000000001},"min":{"1min":0.875,"5min":0.70699999999999996,"15min":0.67700000000000005},"max":{"1min":1.5940000000000001,"5min":2.524,"15min":2.524},"last":1.179},{"interface":"front","average":{"1min":1.206,"5min":1.236,"15min":1.2569999999999999},"min":{"1min":0.62,"5min":0.62,"15min":0.62},"max":{"1min":1.5980000000000001,"5min":1.863,"15min":1.863},"last":1.468}]},{"osd":1,"last update":"Wed Aug 9 13:50:21 2023","interfaces":[{"interface":"back","average":{"1min":1.244,"5min":1.3169999999999999,"15min":1.2929999999999999},"min":{"1min":0.70999999999999996,"5min":0.67800000000000005,"15min":0.67800000000000005},"max":{"1min":1.48,"5min":1.8480000000000001,"15min":1.8480000000000001},"last":1.026},{"interface":"front","average":{"1min":1.242,"5min":1.2649999999999999,"15min":1.248},"min":{"1min":0.81799999999999995,"5min":0.81799999999999995,"15min":0.64900000000000002},"max":{"1min":1.9330000000000001,"5min":1.9330000000000001,"15min":1.9330000000000001},"last":0.98499999999999999}]},{"osd":2,"last update":"Wed Aug 9 13:50:21 2023","interfaces":[{"interface":"back","average":{"1min":1.306,"5min":1.2869999999999999,"15min":1.2809999999999999},"min":{"1min":1.022,"5min":0.621,"15min":0.621},"max":{"1min":1.619,"5min":2.6429999999999998,"15min":2.6429999999999998},"last":0.69999999999999996},{"interface":"front","average":{"1min":1.343,"5min":1.3720000000000001,"15min":1.3660000000000001},"min":{"1min":0.88300000000000001,"5min":0.88300000000000001,"15min":0.74299999999999999},"max":{"1min":1.7569999999999999,"5min":1.7569999999999999,"15min":1.8149999999999999},"last":0.89000000000000001}]},{"osd":3,"last update":"Wed Aug 9 13:50:21 2023","interfaces":[{"interface":"back","average":{"1min":1.377,"5min":1.3300000000000001,"15min":1.298},"min":{"1min":1.083,"5min":0.86399999999999999,"15min":0.84699999999999998},"max":{"1min":1.6739999999999999,"5min":2.6099999999999999,"15min":2.6099999999999999},"last":1.212},{"interface":"front","average":{"1min":1.4099999999999999,"5min":1.3979999999999999,"15min":1.3799999999999999},"min":{"1min":0.94799999999999995,"5min":0.88800000000000001,"15min":0.63100000000000001},"max":{"1min":1.6890000000000001,"5min":2.7629999999999999,"15min":2.7629999999999999},"last":0.95599999999999996}]},{"osd":5,"last update":"Wed Aug 9 13:50:21 2023","interfaces":[{"interface":"back","average":{"1min":1.288,"5min":1.3400000000000001,"15min":1.3300000000000001},"min":{"1min":0.876,"5min":0.73199999999999998,"15min":0.63200000000000001},"max":{"1min":1.734,"5min":1.8,"15min":2.0099999999999998},"last":1.343},{"interface":"front","average":{"1min":1.2509999999999999,"5min":1.323,"15min":1.3180000000000001},"min":{"1min":0.92000000000000004,"5min":0.72399999999999998,"15min":0.626},"max":{"1min":1.71,"5min":2.7090000000000001,"15min":2.7090000000000001},"last":1.421}]},{"osd":6,"last update":"Wed Aug 9 13:50:21 2023","interfaces":[{"interface":"back","average":{"1min":1.379,"5min":1.3160000000000001,"15min":1.3360000000000001},"min":{"1min":0.94699999999999995,"5min":0.74399999999999999,"15min":0.59099999999999997},"max":{"1min":1.7509999999999999,"5min":2.8559999999999999,"15min":2.8559999999999999},"last":1.083},{"interface":"front","average":{"1min":1.4430000000000001,"5min":1.518,"15min":1.504},"min":{"1min":1.0760000000000001,"5min":0.97799999999999998,"15min":0.76400000000000001},"max":{"1min":1.669,"5min":2.2719999999999998,"15min":2.2719999999999998},"last":1.5409999999999999}]},{"osd":7,"last update":"Wed Aug 9 13:50:21 2023","interfaces":[{"interface":"back","average":{"1min":1.46,"5min":1.4830000000000001,"15min":1.474},"min":{"1min":1.1499999999999999,"5min":0.85999999999999999,"15min":0.73899999999999999},"max":{"1min":1.8460000000000001,"5min":2.8050000000000002,"15min":2.8050000000000002},"last":1.1100000000000001},{"interface":"front","average":{"1min":1.474,"5min":1.522,"15min":1.514},"min":{"1min":1.236,"5min":0.97199999999999998,"15min":0.72899999999999998},"max":{"1min":1.754,"5min":3.415,"15min":3.415},"last":1.3959999999999999}]}]},{"osd":3,"up_from":13,"seq":55834575048,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":36260,"kb_used_data":1044,"kb_used_omap":1,"kb_used_meta":35198,"kb_avail":94335580,"statfs":{"total":96636764160,"available":96599633920,"internally_reserved":0,"allocated":1069056,"data_stored":569268,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1781,"internal_metadata":36043019},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Aug 9 13:50:25 2023","interfaces":[{"interface":"back","average":{"1min":0.84599999999999997,"5min":0.88300000000000001,"15min":0.91600000000000004},"min":{"1min":0.43099999999999999,"5min":0.27200000000000002,"15min":0.27200000000000002},"max":{"1min":1.3939999999999999,"5min":1.8500000000000001,"15min":1.968},"last":0.65900000000000003},{"interface":"front","average":{"1min":1.071,"5min":1.0920000000000001,"15min":1.1060000000000001},"min":{"1min":0.48199999999999998,"5min":0.36399999999999999,"15min":0.36399999999999999},"max":{"1min":1.6910000000000001,"5min":1.6910000000000001,"15min":2.2360000000000002},"last":0.75600000000000001}]},{"osd":1,"last update":"Wed Aug 9 13:50:25 2023","interfaces":[{"interface":"back","average":{"1min":1.042,"5min":1.0089999999999999,"15min":1.012},"min":{"1min":0.59899999999999998,"5min":0.496,"15min":0.438},"max":{"1min":1.518,"5min":1.7110000000000001,"15min":2.5649999999999999},"last":0.83999999999999997},{"interface":"front","average":{"1min":1.091,"5min":1.1279999999999999,"15min":1.1639999999999999},"min":{"1min":0.51800000000000002,"5min":0.51800000000000002,"15min":0.42099999999999999},"max":{"1min":1.6839999999999999,"5min":1.6839999999999999,"15min":2.6760000000000002},"last":1.202}]},{"osd":2,"last update":"Wed Aug 9 13:50:25 2023","interfaces":[{"interface":"back","average":{"1min":1.121,"5min":1.177,"15min":1.1930000000000001},"min":{"1min":0.53600000000000003,"5min":0.53600000000000003,"15min":0.53500000000000003},"max":{"1min":1.6240000000000001,"5min":1.913,"15min":1.923},"last":1.1399999999999999},{"interface":"front","average":{"1min":1.1899999999999999,"5min":1.2929999999999999,"15min":1.3080000000000001},"min":{"1min":0.625,"5min":0.52300000000000002,"15min":0.52000000000000002},"max":{"1min":1.9399999999999999,"5min":2.0259999999999998,"15min":2.0259999999999998},"last":1.0960000000000001}]},{"osd":4,"last update":"Wed Aug 9 13:50:25 2023","interfaces":[{"interface":"back","average":{"1min":1.232,"5min":1.258,"15min":1.282},"min":{"1min":0.91500000000000004,"5min":0.57899999999999996,"15min":0.57899999999999996},"max":{"1min":1.742,"5min":1.8160000000000001,"15min":1.8160000000000001},"last":1.6819999999999999},{"interface":"front","average":{"1min":1.349,"5min":1.397,"15min":1.3919999999999999},"min":{"1min":0.67600000000000005,"5min":0.628,"15min":0.628},"max":{"1min":1.8149999999999999,"5min":2.2040000000000002,"15min":2.2370000000000001},"last":1.3080000000000001}]},{"osd":5,"last update":"Wed Aug 9 13:50:25 2023","interfaces":[{"interface":"back","average":{"1min":1.456,"5min":1.468,"15min":1.474},"min":{"1min":0.67600000000000005,"5min":0.67600000000000005,"15min":0.67200000000000004},"max":{"1min":1.855,"5min":2.254,"15min":2.254},"last":1.3959999999999999},{"interface":"front","average":{"1min":1.4850000000000001,"5min":1.4910000000000001,"15min":1.49},"min":{"1min":0.77600000000000002,"5min":0.65900000000000003,"15min":0.65900000000000003},"max":{"1min":2.3069999999999999,"5min":2.3069999999999999,"15min":2.3069999999999999},"last":1.3260000000000001}]},{"osd":6,"last update":"Wed Aug 9 13:50:25 2023","interfaces":[{"interface":"back","average":{"1min":1.3759999999999999,"5min":1.4370000000000001,"15min":1.4590000000000001},"min":{"1min":0.878,"5min":0.68200000000000005,"15min":0.68200000000000005},"max":{"1min":1.8759999999999999,"5min":2.1829999999999998,"15min":2.242},"last":1.4670000000000001},{"interface":"front","average":{"1min":1.466,"5min":1.5580000000000001,"15min":1.5529999999999999},"min":{"1min":1.0600000000000001,"5min":0.71099999999999997,"15min":0.68600000000000005},"max":{"1min":1.825,"5min":2.3420000000000001,"15min":2.3420000000000001},"last":1.252}]},{"osd":7,"last update":"Wed Aug 9 13:50:25 2023","interfaces":[{"interface":"back","average":{"1min":1.601,"5min":1.627,"15min":1.6160000000000001},"min":{"1min":1.115,"5min":0.69399999999999995,"15min":0.69399999999999995},"max":{"1min":2.46,"5min":2.46,"15min":2.46},"last":1.4279999999999999},{"interface":"front","average":{"1min":1.4910000000000001,"5min":1.546,"15min":1.5600000000000001},"min":{"1min":0.78200000000000003,"5min":0.64200000000000002,"15min":0.61599999999999999},"max":{"1min":2.0579999999999998,"5min":2.359,"15min":2.484},"last":1.571}]}]},{"osd":2,"up_from":13,"seq":55834575047,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":36256,"kb_used_data":1044,"kb_used_omap":1,"kb_used_meta":35198,"kb_avail":94335584,"statfs":{"total":96636764160,"available":96599638016,"internally_reserved":0,"allocated":1069056,"data_stored":569268,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1781,"internal_metadata":36043019},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Aug 9 13:50:17 2023","interfaces":[{"interface":"back","average":{"1min":0.96999999999999997,"5min":0.95299999999999996,"15min":0.95499999999999996},"min":{"1min":0.47799999999999998,"5min":0.36499999999999999,"15min":0.36499999999999999},"max":{"1min":1.5760000000000001,"5min":1.6180000000000001,"15min":1.8380000000000001},"last":1.484},{"interface":"front","average":{"1min":0.97099999999999997,"5min":0.998,"15min":1.0049999999999999},"min":{"1min":0.45300000000000001,"5min":0.374,"15min":0.374},"max":{"1min":1.4199999999999999,"5min":2.3929999999999998,"15min":2.3929999999999998},"last":0.80200000000000005}]},{"osd":1,"last update":"Wed Aug 9 13:50:17 2023","interfaces":[{"interface":"back","average":{"1min":1.2529999999999999,"5min":1.1739999999999999,"15min":1.1870000000000001},"min":{"1min":0.61299999999999999,"5min":0.55200000000000005,"15min":0.55200000000000005},"max":{"1min":1.6870000000000001,"5min":1.77,"15min":1.8420000000000001},"last":1.615},{"interface":"front","average":{"1min":1.216,"5min":1.2749999999999999,"15min":1.335},"min":{"1min":0.52500000000000002,"5min":0.52500000000000002,"15min":0.52500000000000002},"max":{"1min":1.7390000000000001,"5min":2.4870000000000001,"15min":2.4870000000000001},"last":1.03}]},{"osd":3,"last update":"Wed Aug 9 13:50:17 2023","interfaces":[{"interface":"back","average":{"1min":0.98099999999999998,"5min":1.006,"15min":1.0469999999999999},"min":{"1min":0.44800000000000001,"5min":0.38900000000000001,"15min":0.38900000000000001},"max":{"1min":1.3109999999999999,"5min":1.708,"15min":2.0409999999999999},"last":0.63600000000000001},{"interface":"front","average":{"1min":1.2110000000000001,"5min":1.304,"15min":1.3089999999999999},"min":{"1min":0.55500000000000005,"5min":0.46200000000000002,"15min":0.46200000000000002},"max":{"1min":1.8009999999999999,"5min":8.8130000000000006,"15min":8.8130000000000006},"last":1.2529999999999999}]},{"osd":4,"last update":"Wed Aug 9 13:50:17 2023","interfaces":[{"interface":"back","average":{"1min":1.415,"5min":1.4179999999999999,"15min":1.4510000000000001},"min":{"1min":0.83999999999999997,"5min":0.628,"15min":0.628},"max":{"1min":1.958,"5min":2.089,"15min":2.089},"last":1.556},{"interface":"front","average":{"1min":1.462,"5min":1.359,"15min":1.3979999999999999},"min":{"1min":0.73799999999999999,"5min":0.68600000000000005,"15min":0.68600000000000005},"max":{"1min":2.472,"5min":2.472,"15min":2.472},"last":1.151}]},{"osd":5,"last update":"Wed Aug 9 13:50:17 2023","interfaces":[{"interface":"back","average":{"1min":1.5189999999999999,"5min":1.4379999999999999,"15min":1.458},"min":{"1min":0.76600000000000001,"5min":0.59999999999999998,"15min":0.59999999999999998},"max":{"1min":2.27,"5min":2.5840000000000001,"15min":2.5840000000000001},"last":1.3240000000000001},{"interface":"front","average":{"1min":1.304,"5min":1.3540000000000001,"15min":1.349},"min":{"1min":0.77100000000000002,"5min":0.56799999999999995,"15min":0.56799999999999995},"max":{"1min":1.8859999999999999,"5min":3.5659999999999998,"15min":3.5659999999999998},"last":1.379}]},{"osd":6,"last update":"Wed Aug 9 13:50:17 2023","interfaces":[{"interface":"back","average":{"1min":1.4019999999999999,"5min":1.425,"15min":1.4259999999999999},"min":{"1min":0.79800000000000004,"5min":0.73799999999999999,"15min":0.73799999999999999},"max":{"1min":2.222,"5min":3.5099999999999998,"15min":3.5099999999999998},"last":1.079},{"interface":"front","average":{"1min":1.4790000000000001,"5min":1.4990000000000001,"15min":1.5409999999999999},"min":{"1min":0.82199999999999995,"5min":0.66300000000000003,"15min":0.66300000000000003},"max":{"1min":2.3769999999999998,"5min":2.3769999999999998,"15min":2.3769999999999998},"last":1.6930000000000001}]},{"osd":7,"last update":"Wed Aug 9 13:50:17 2023","interfaces":[{"interface":"back","average":{"1min":1.54,"5min":1.51,"15min":1.548},"min":{"1min":0.78800000000000003,"5min":0.64600000000000002,"15min":0.64600000000000002},"max":{"1min":2.1970000000000001,"5min":2.5249999999999999,"15min":2.5249999999999999},"last":1.4379999999999999},{"interface":"front","average":{"1min":1.4970000000000001,"5min":1.52,"15min":1.51},"min":{"1min":0.99199999999999999,"5min":0.70499999999999996,"15min":0.70499999999999996},"max":{"1min":2.089,"5min":3.609,"15min":3.609},"last":1.214}]}]},{"osd":1,"up_from":13,"seq":55834575048,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28696,"kb_used_data":1668,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343144,"statfs":{"total":96636764160,"available":96607379456,"internally_reserved":0,"allocated":1708032,"data_stored":1176373,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1781,"internal_metadata":27654411},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Aug 9 13:50:31 2023","interfaces":[{"interface":"back","average":{"1min":0.99099999999999999,"5min":0.92000000000000004,"15min":0.93999999999999995},"min":{"1min":0.50600000000000001,"5min":0.50600000000000001,"15min":0.32400000000000001},"max":{"1min":1.7110000000000001,"5min":1.7110000000000001,"15min":2.1080000000000001},"last":0.66600000000000004},{"interface":"front","average":{"1min":0.98999999999999999,"5min":1.0740000000000001,"15min":1.095},"min":{"1min":0.59799999999999998,"5min":0.432,"15min":0.42499999999999999},"max":{"1min":1.7190000000000001,"5min":2.468,"15min":2.468},"last":0.63400000000000001}]},{"osd":2,"last update":"Wed Aug 9 13:50:31 2023","interfaces":[{"interface":"back","average":{"1min":1.008,"5min":0.97799999999999998,"15min":0.995},"min":{"1min":0.56699999999999995,"5min":0.41099999999999998,"15min":0.41099999999999998},"max":{"1min":1.6679999999999999,"5min":1.907,"15min":1.907},"last":0.60799999999999998},{"interface":"front","average":{"1min":1.1459999999999999,"5min":1.159,"15min":1.1879999999999999},"min":{"1min":0.80600000000000005,"5min":0.54400000000000004,"15min":0.46500000000000002},"max":{"1min":1.5189999999999999,"5min":2.1280000000000001,"15min":2.145},"last":0.81999999999999995}]},{"osd":3,"last update":"Wed Aug 9 13:50:31 2023","interfaces":[{"interface":"back","average":{"1min":1.218,"5min":1.2629999999999999,"15min":1.3260000000000001},"min":{"1min":0.63200000000000001,"5min":0.54400000000000004,"15min":0.432},"max":{"1min":1.7430000000000001,"5min":2.0270000000000001,"15min":2.3210000000000002},"last":0.86199999999999999},{"interface":"front","average":{"1min":1.1779999999999999,"5min":1.234,"15min":1.2450000000000001},"min":{"1min":0.70399999999999996,"5min":0.49299999999999999,"15min":0.44400000000000001},"max":{"1min":1.786,"5min":2.3540000000000001,"15min":2.7669999999999999},"last":0.72899999999999998}]},{"osd":4,"last update":"Wed Aug 9 13:50:31 2023","interfaces":[{"interface":"back","average":{"1min":1.3149999999999999,"5min":1.399,"15min":1.4079999999999999},"min":{"1min":0.94399999999999995,"5min":0.79800000000000004,"15min":0.58099999999999996},"max":{"1min":1.696,"5min":1.97,"15min":2.5},"last":1.173},{"interface":"front","average":{"1min":1.2689999999999999,"5min":1.3180000000000001,"15min":1.333},"min":{"1min":0.88200000000000001,"5min":0.70099999999999996,"15min":0.70099999999999996},"max":{"1min":1.6950000000000001,"5min":1.901,"15min":1.901},"last":0.80600000000000005}]},{"osd":5,"last update":"Wed Aug 9 13:50:31 2023","interfaces":[{"interface":"back","average":{"1min":1.3200000000000001,"5min":1.323,"15min":1.371},"min":{"1min":0.90700000000000003,"5min":0.76900000000000002,"15min":0.76900000000000002},"max":{"1min":1.837,"5min":2.294,"15min":2.294},"last":0.76800000000000002},{"interface":"front","average":{"1min":1.427,"5min":1.464,"15min":1.516},"min":{"1min":0.97899999999999998,"5min":0.91000000000000003,"15min":0.77700000000000002},"max":{"1min":1.837,"5min":2.1389999999999998,"15min":2.4649999999999999},"last":1.2010000000000001}]},{"osd":6,"last update":"Wed Aug 9 13:50:31 2023","interfaces":[{"interface":"back","average":{"1min":1.4219999999999999,"5min":1.48,"15min":1.514},"min":{"1min":1.026,"5min":0.77500000000000002,"15min":0.67400000000000004},"max":{"1min":1.925,"5min":2.2080000000000002,"15min":2.5270000000000001},"last":1.004},{"interface":"front","average":{"1min":1.4370000000000001,"5min":1.45,"15min":1.5},"min":{"1min":1.1379999999999999,"5min":0.82499999999999996,"15min":0.79800000000000004},"max":{"1min":2.02,"5min":2.02,"15min":2.1040000000000001},"last":0.92400000000000004}]},{"osd":7,"last update":"Wed Aug 9 13:50:31 2023","interfaces":[{"interface":"back","average":{"1min":1.552,"5min":1.5649999999999999,"15min":1.619},"min":{"1min":1.0389999999999999,"5min":0.86899999999999999,"15min":0.77100000000000002},"max":{"1min":2.4169999999999998,"5min":2.4169999999999998,"15min":2.5790000000000002},"last":1.252},{"interface":"front","average":{"1min":1.5740000000000001,"5min":1.5549999999999999,"15min":1.6000000000000001},"min":{"1min":1.2589999999999999,"5min":0.91300000000000003,"15min":0.75900000000000001},"max":{"1min":2.036,"5min":2.1040000000000001,"15min":2.6509999999999998},"last":0.96899999999999997}]}]},{"osd":0,"up_from":13,"seq":55834575048,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":40380,"kb_used_data":1064,"kb_used_omap":1,"kb_used_meta":39294,"kb_avail":94331460,"statfs":{"total":96636764160,"available":96595415040,"internally_reserved":0,"allocated":1089536,"data_stored":582416,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1781,"internal_metadata":40237323},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Wed Aug 9 13:50:27 2023","interfaces":[{"interface":"back","average":{"1min":0.91800000000000004,"5min":0.92200000000000004,"15min":0.91600000000000004},"min":{"1min":0.32500000000000001,"5min":0.32500000000000001,"15min":0.32500000000000001},"max":{"1min":1.5820000000000001,"5min":1.756,"15min":1.756},"last":0.76400000000000001},{"interface":"front","average":{"1min":1.0680000000000001,"5min":1.1020000000000001,"15min":1.113},"min":{"1min":0.47099999999999997,"5min":0.45300000000000001,"15min":0.45300000000000001},"max":{"1min":1.9019999999999999,"5min":1.9019999999999999,"15min":1.9019999999999999},"last":1.171}]},{"osd":2,"last update":"Wed Aug 9 13:50:27 2023","interfaces":[{"interface":"back","average":{"1min":1.048,"5min":1.1000000000000001,"15min":1.135},"min":{"1min":0.53800000000000003,"5min":0.51400000000000001,"15min":0.44800000000000001},"max":{"1min":1.702,"5min":1.776,"15min":1.889},"last":0.95199999999999996},{"interface":"front","average":{"1min":1.1819999999999999,"5min":1.149,"15min":1.179},"min":{"1min":0.58499999999999996,"5min":0.56399999999999995,"15min":0.52700000000000002},"max":{"1min":2.0129999999999999,"5min":2.0129999999999999,"15min":2.2519999999999998},"last":1.3620000000000001}]},{"osd":3,"last update":"Wed Aug 9 13:50:27 2023","interfaces":[{"interface":"back","average":{"1min":0.98499999999999999,"5min":1.0509999999999999,"15min":1.105},"min":{"1min":0.54900000000000004,"5min":0.46300000000000002,"15min":0.46300000000000002},"max":{"1min":1.345,"5min":1.823,"15min":2.5190000000000001},"last":1.1040000000000001},{"interface":"front","average":{"1min":1.272,"5min":1.2629999999999999,"15min":1.3140000000000001},"min":{"1min":0.56399999999999995,"5min":0.54700000000000004,"15min":0.54700000000000004},"max":{"1min":2.347,"5min":2.347,"15min":2.4649999999999999},"last":1.0740000000000001}]},{"osd":4,"last update":"Wed Aug 9 13:50:27 2023","interfaces":[{"interface":"back","average":{"1min":1.302,"5min":1.3129999999999999,"15min":1.347},"min":{"1min":0.84699999999999998,"5min":0.84699999999999998,"15min":0.74299999999999999},"max":{"1min":1.9630000000000001,"5min":2.2309999999999999,"15min":2.2309999999999999},"last":1.323},{"interface":"front","average":{"1min":1.3999999999999999,"5min":1.4390000000000001,"15min":1.452},"min":{"1min":0.82699999999999996,"5min":0.77600000000000002,"15min":0.74099999999999999},"max":{"1min":1.8440000000000001,"5min":2.5390000000000001,"15min":2.5390000000000001},"last":1.248}]},{"osd":5,"last update":"Wed Aug 9 13:50:27 2023","interfaces":[{"interface":"back","average":{"1min":1.387,"5min":1.4219999999999999,"15min":1.4650000000000001},"min":{"1min":0.66300000000000003,"5min":0.66300000000000003,"15min":0.66300000000000003},"max":{"1min":2.6669999999999998,"5min":2.6669999999999998,"15min":2.6669999999999998},"last":1.649},{"interface":"front","average":{"1min":1.3400000000000001,"5min":1.415,"15min":1.4490000000000001},"min":{"1min":0.70099999999999996,"5min":0.629,"15min":0.629},"max":{"1min":1.6160000000000001,"5min":2.1499999999999999,"15min":2.1499999999999999},"last":1.444}]},{"osd":6,"last update":"Wed Aug 9 13:50:27 2023","interfaces":[{"interface":"back","average":{"1min":1.51,"5min":1.532,"15min":1.5469999999999999},"min":{"1min":0.73599999999999999,"5min":0.73599999999999999,"15min":0.69799999999999995},"max":{"1min":2.3100000000000001,"5min":2.7250000000000001,"15min":2.7250000000000001},"last":1.397},{"interface":"front","average":{"1min":1.5169999999999999,"5min":1.5389999999999999,"15min":1.5600000000000001},"min":{"1min":0.86699999999999999,"5min":0.86699999999999999,"15min":0.86699999999999999},"max":{"1min":2.3199999999999998,"5min":2.399,"15min":2.399},"last":1.4670000000000001}]},{"osd":7,"last update":"Wed Aug 9 13:50:27 2023","interfaces":[{"interface":"back","average":{"1min":1.532,"5min":1.595,"15min":1.625},"min":{"1min":0.72299999999999998,"5min":0.72299999999999998,"15min":0.72299999999999998},"max":{"1min":2.3900000000000001,"5min":2.3900000000000001,"15min":2.464},"last":1.5600000000000001},{"interface":"front","average":{"1min":1.4350000000000001,"5min":1.512,"15min":1.55},"min":{"1min":0.84499999999999997,"5min":0.76700000000000002,"15min":0.76700000000000002},"max":{"1min":2.274,"5min":2.274,"15min":2.274},"last":1.5049999999999999}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":2228,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-08-09T13:50:34.797 INFO:tasks.cephfs.filesystem.ceph_manager:PG 11.3e is not active+clean 2023-08-09T13:50:34.798 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '11.3e', 'version': "0'0", 'reported_seq': 3, 'reported_epoch': 89, 'state': 'creating+activating', 'last_fresh': '2023-08-09T13:50:29.231349+0000', 'last_change': '2023-08-09T13:50:29.231349+0000', 'last_active': '2023-08-09T13:50:28.150437+0000', 'last_peered': '2023-08-09T13:50:28.150437+0000', 'last_clean': '2023-08-09T13:50:28.150437+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-08-09T13:50:29.231349+0000', 'last_undegraded': '2023-08-09T13:50:29.231349+0000', 'last_fullsized': '2023-08-09T13:50:29.231349+0000', 'mapping_epoch': 88, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 88, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-08-09T13:50:28.150437+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-08-09T13:50:28.150437+0000', 'last_clean_scrub_stamp': '2023-08-09T13:50:28.150437+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-08-10T18:13:21.364990+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [6, 4], 'acting': [6, 4], 'avail_no_missing': ['6', '4'], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 6, 'acting_primary': 6, 'purged_snaps': []} 2023-08-09T13:50:34.798 INFO:tasks.cephfs.filesystem.ceph_manager:PG 11.3b is not active+clean 2023-08-09T13:50:34.799 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '11.3b', 'version': "0'0", 'reported_seq': 3, 'reported_epoch': 89, 'state': 'creating+activating', 'last_fresh': '2023-08-09T13:50:29.221854+0000', 'last_change': '2023-08-09T13:50:29.221854+0000', 'last_active': '2023-08-09T13:50:28.150437+0000', 'last_peered': '2023-08-09T13:50:28.150437+0000', 'last_clean': '2023-08-09T13:50:28.150437+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-08-09T13:50:29.221854+0000', 'last_undegraded': '2023-08-09T13:50:29.221854+0000', 'last_fullsized': '2023-08-09T13:50:29.221854+0000', 'mapping_epoch': 88, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 88, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-08-09T13:50:28.150437+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-08-09T13:50:28.150437+0000', 'last_clean_scrub_stamp': '2023-08-09T13:50:28.150437+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-08-10T22:54:27.616846+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 7], 'acting': [2, 7], 'avail_no_missing': ['2', '7'], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2023-08-09T13:50:34.799 INFO:tasks.cephfs.filesystem.ceph_manager:PG 11.39 is not active+clean 2023-08-09T13:50:34.799 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '11.39', 'version': "0'0", 'reported_seq': 3, 'reported_epoch': 89, 'state': 'creating+activating', 'last_fresh': '2023-08-09T13:50:29.220028+0000', 'last_change': '2023-08-09T13:50:29.220028+0000', 'last_active': '2023-08-09T13:50:28.150437+0000', 'last_peered': '2023-08-09T13:50:28.150437+0000', 'last_clean': '2023-08-09T13:50:28.150437+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-08-09T13:50:29.220028+0000', 'last_undegraded': '2023-08-09T13:50:29.220028+0000', 'last_fullsized': '2023-08-09T13:50:29.220028+0000', 'mapping_epoch': 88, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 88, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-08-09T13:50:28.150437+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-08-09T13:50:28.150437+0000', 'last_clean_scrub_stamp': '2023-08-09T13:50:28.150437+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-08-10T23:19:57.122069+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [5, 1], 'acting': [5, 1], 'avail_no_missing': ['5', '1'], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 5, 'acting_primary': 5, 'purged_snaps': []} 2023-08-09T13:50:34.800 INFO:tasks.cephfs.filesystem.ceph_manager:PG 11.37 is not active+clean 2023-08-09T13:50:34.800 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '11.37', 'version': "0'0", 'reported_seq': 3, 'reported_epoch': 89, 'state': 'creating+activating', 'last_fresh': '2023-08-09T13:50:29.223778+0000', 'last_change': '2023-08-09T13:50:29.223778+0000', 'last_active': '2023-08-09T13:50:28.150437+0000', 'last_peered': '2023-08-09T13:50:28.150437+0000', 'last_clean': '2023-08-09T13:50:28.150437+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-08-09T13:50:29.223778+0000', 'last_undegraded': '2023-08-09T13:50:29.223778+0000', 'last_fullsized': '2023-08-09T13:50:29.223778+0000', 'mapping_epoch': 88, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 88, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-08-09T13:50:28.150437+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-08-09T13:50:28.150437+0000', 'last_clean_scrub_stamp': '2023-08-09T13:50:28.150437+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-08-10T15:38:50.620059+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 3], 'acting': [2, 3], 'avail_no_missing': ['2', '3'], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2023-08-09T13:50:34.800 INFO:tasks.cephfs.filesystem.ceph_manager:PG 11.36 is not active+clean 2023-08-09T13:50:34.801 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '11.36', 'version': "0'0", 'reported_seq': 3, 'reported_epoch': 89, 'state': 'creating+activating', 'last_fresh': '2023-08-09T13:50:29.207900+0000', 'last_change': '2023-08-09T13:50:29.207900+0000', 'last_active': '2023-08-09T13:50:28.150437+0000', 'last_peered': '2023-08-09T13:50:28.150437+0000', 'last_clean': '2023-08-09T13:50:28.150437+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-08-09T13:50:29.207900+0000', 'last_undegraded': '2023-08-09T13:50:29.207900+0000', 'last_fullsized': '2023-08-09T13:50:29.207900+0000', 'mapping_epoch': 88, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 88, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-08-09T13:50:28.150437+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-08-09T13:50:28.150437+0000', 'last_clean_scrub_stamp': '2023-08-09T13:50:28.150437+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-08-10T21:32:28.121459+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [4, 2], 'acting': [4, 2], 'avail_no_missing': ['4', '2'], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 4, 'acting_primary': 4, 'purged_snaps': []} 2023-08-09T13:50:34.801 INFO:tasks.cephfs.filesystem.ceph_manager:PG 11.32 is not active+clean 2023-08-09T13:50:34.801 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '11.32', 'version': "0'0", 'reported_seq': 3, 'reported_epoch': 89, 'state': 'creating+activating', 'last_fresh': '2023-08-09T13:50:29.224087+0000', 'last_change': '2023-08-09T13:50:29.224087+0000', 'last_active': '2023-08-09T13:50:28.150437+0000', 'last_peered': '2023-08-09T13:50:28.150437+0000', 'last_clean': '2023-08-09T13:50:28.150437+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-08-09T13:50:29.224087+0000', 'last_undegraded': '2023-08-09T13:50:29.224087+0000', 'last_fullsized': '2023-08-09T13:50:29.224087+0000', 'mapping_epoch': 88, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 88, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-08-09T13:50:28.150437+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-08-09T13:50:28.150437+0000', 'last_clean_scrub_stamp': '2023-08-09T13:50:28.150437+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-08-10T23:57:06.314706+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 5], 'acting': [2, 5], 'avail_no_missing': ['2', '5'], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2023-08-09T13:50:34.802 INFO:tasks.cephfs.filesystem.ceph_manager:PG 11.30 is not active+clean 2023-08-09T13:50:34.802 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '11.30', 'version': "0'0", 'reported_seq': 3, 'reported_epoch': 89, 'state': 'creating+activating', 'last_fresh': '2023-08-09T13:50:29.208580+0000', 'last_change': '2023-08-09T13:50:29.208580+0000', 'last_active': '2023-08-09T13:50:28.150437+0000', 'last_peered': '2023-08-09T13:50:28.150437+0000', 'last_clean': '2023-08-09T13:50:28.150437+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-08-09T13:50:29.208580+0000', 'last_undegraded': '2023-08-09T13:50:29.208580+0000', 'last_fullsized': '2023-08-09T13:50:29.208580+0000', 'mapping_epoch': 88, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 88, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-08-09T13:50:28.150437+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-08-09T13:50:28.150437+0000', 'last_clean_scrub_stamp': '2023-08-09T13:50:28.150437+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-08-10T15:14:37.215348+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [4, 6], 'acting': [4, 6], 'avail_no_missing': ['4', '6'], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 4, 'acting_primary': 4, 'purged_snaps': []} 2023-08-09T13:50:34.803 INFO:tasks.cephfs.filesystem.ceph_manager:PG 11.2f is not active+clean 2023-08-09T13:50:34.803 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '11.2f', 'version': "0'0", 'reported_seq': 3, 'reported_epoch': 89, 'state': 'creating+activating', 'last_fresh': '2023-08-09T13:50:29.223293+0000', 'last_change': '2023-08-09T13:50:29.223293+0000', 'last_active': '2023-08-09T13:50:28.150437+0000', 'last_peered': '2023-08-09T13:50:28.150437+0000', 'last_clean': '2023-08-09T13:50:28.150437+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-08-09T13:50:29.223293+0000', 'last_undegraded': '2023-08-09T13:50:29.223293+0000', 'last_fullsized': '2023-08-09T13:50:29.223293+0000', 'mapping_epoch': 88, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 88, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-08-09T13:50:28.150437+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-08-09T13:50:28.150437+0000', 'last_clean_scrub_stamp': '2023-08-09T13:50:28.150437+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-08-10T22:44:32.845197+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 3], 'acting': [2, 3], 'avail_no_missing': ['2', '3'], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2023-08-09T13:50:34.803 INFO:tasks.cephfs.filesystem.ceph_manager:PG 11.2a is not active+clean 2023-08-09T13:50:34.804 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '11.2a', 'version': "0'0", 'reported_seq': 3, 'reported_epoch': 89, 'state': 'creating+activating', 'last_fresh': '2023-08-09T13:50:29.207312+0000', 'last_change': '2023-08-09T13:50:29.207312+0000', 'last_active': '2023-08-09T13:50:28.150437+0000', 'last_peered': '2023-08-09T13:50:28.150437+0000', 'last_clean': '2023-08-09T13:50:28.150437+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-08-09T13:50:29.207312+0000', 'last_undegraded': '2023-08-09T13:50:29.207312+0000', 'last_fullsized': '2023-08-09T13:50:29.207312+0000', 'mapping_epoch': 88, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 88, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-08-09T13:50:28.150437+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-08-09T13:50:28.150437+0000', 'last_clean_scrub_stamp': '2023-08-09T13:50:28.150437+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-08-10T19:03:45.295280+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [4, 2], 'acting': [4, 2], 'avail_no_missing': ['4', '2'], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 4, 'acting_primary': 4, 'purged_snaps': []} 2023-08-09T13:50:34.804 INFO:tasks.cephfs.filesystem.ceph_manager:PG 11.28 is not active+clean 2023-08-09T13:50:34.804 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '11.28', 'version': "0'0", 'reported_seq': 3, 'reported_epoch': 89, 'state': 'creating+activating', 'last_fresh': '2023-08-09T13:50:29.217553+0000', 'last_change': '2023-08-09T13:50:29.217553+0000', 'last_active': '2023-08-09T13:50:28.150437+0000', 'last_peered': '2023-08-09T13:50:28.150437+0000', 'last_clean': '2023-08-09T13:50:28.150437+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-08-09T13:50:29.217553+0000', 'last_undegraded': '2023-08-09T13:50:29.217553+0000', 'last_fullsized': '2023-08-09T13:50:29.217553+0000', 'mapping_epoch': 88, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 88, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-08-09T13:50:28.150437+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-08-09T13:50:28.150437+0000', 'last_clean_scrub_stamp': '2023-08-09T13:50:28.150437+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-08-10T22:39:17.713654+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [7, 6], 'acting': [7, 6], 'avail_no_missing': ['7', '6'], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 7, 'acting_primary': 7, 'purged_snaps': []} 2023-08-09T13:50:34.805 INFO:tasks.cephfs.filesystem.ceph_manager:PG 11.27 is not active+clean 2023-08-09T13:50:34.805 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '11.27', 'version': "0'0", 'reported_seq': 3, 'reported_epoch': 89, 'state': 'creating+activating', 'last_fresh': '2023-08-09T13:50:29.208200+0000', 'last_change': '2023-08-09T13:50:29.208200+0000', 'last_active': '2023-08-09T13:50:28.150437+0000', 'last_peered': '2023-08-09T13:50:28.150437+0000', 'last_clean': '2023-08-09T13:50:28.150437+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-08-09T13:50:29.208200+0000', 'last_undegraded': '2023-08-09T13:50:29.208200+0000', 'last_fullsized': '2023-08-09T13:50:29.208200+0000', 'mapping_epoch': 88, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 88, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-08-09T13:50:28.150437+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-08-09T13:50:28.150437+0000', 'last_clean_scrub_stamp': '2023-08-09T13:50:28.150437+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-08-10T14:52:59.954591+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [4, 7], 'acting': [4, 7], 'avail_no_missing': ['4', '7'], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 4, 'acting_primary': 4, 'purged_snaps': []} 2023-08-09T13:50:34.805 INFO:tasks.cephfs.filesystem.ceph_manager:PG 11.26 is not active+clean 2023-08-09T13:50:34.806 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '11.26', 'version': "0'0", 'reported_seq': 3, 'reported_epoch': 89, 'state': 'creating+activating', 'last_fresh': '2023-08-09T13:50:29.223117+0000', 'last_change': '2023-08-09T13:50:29.223117+0000', 'last_active': '2023-08-09T13:50:28.150437+0000', 'last_peered': '2023-08-09T13:50:28.150437+0000', 'last_clean': '2023-08-09T13:50:28.150437+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-08-09T13:50:29.223117+0000', 'last_undegraded': '2023-08-09T13:50:29.223117+0000', 'last_fullsized': '2023-08-09T13:50:29.223117+0000', 'mapping_epoch': 88, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 88, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-08-09T13:50:28.150437+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-08-09T13:50:28.150437+0000', 'last_clean_scrub_stamp': '2023-08-09T13:50:28.150437+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-08-10T21:18:31.512056+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [5, 6], 'acting': [5, 6], 'avail_no_missing': ['5', '6'], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 5, 'acting_primary': 5, 'purged_snaps': []} 2023-08-09T13:50:34.806 INFO:tasks.cephfs.filesystem.ceph_manager:PG 11.24 is not active+clean 2023-08-09T13:50:34.806 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '11.24', 'version': "0'0", 'reported_seq': 3, 'reported_epoch': 89, 'state': 'creating+activating', 'last_fresh': '2023-08-09T13:50:29.207497+0000', 'last_change': '2023-08-09T13:50:29.207497+0000', 'last_active': '2023-08-09T13:50:28.150437+0000', 'last_peered': '2023-08-09T13:50:28.150437+0000', 'last_clean': '2023-08-09T13:50:28.150437+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-08-09T13:50:29.207497+0000', 'last_undegraded': '2023-08-09T13:50:29.207497+0000', 'last_fullsized': '2023-08-09T13:50:29.207497+0000', 'mapping_epoch': 88, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 88, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-08-09T13:50:28.150437+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-08-09T13:50:28.150437+0000', 'last_clean_scrub_stamp': '2023-08-09T13:50:28.150437+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-08-10T18:47:18.558930+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [4, 2], 'acting': [4, 2], 'avail_no_missing': ['4', '2'], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 4, 'acting_primary': 4, 'purged_snaps': []} 2023-08-09T13:50:34.807 INFO:tasks.cephfs.filesystem.ceph_manager:PG 11.22 is not active+clean 2023-08-09T13:50:34.807 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '11.22', 'version': "0'0", 'reported_seq': 3, 'reported_epoch': 89, 'state': 'creating+activating', 'last_fresh': '2023-08-09T13:50:29.219724+0000', 'last_change': '2023-08-09T13:50:29.219724+0000', 'last_active': '2023-08-09T13:50:28.150437+0000', 'last_peered': '2023-08-09T13:50:28.150437+0000', 'last_clean': '2023-08-09T13:50:28.150437+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-08-09T13:50:29.219724+0000', 'last_undegraded': '2023-08-09T13:50:29.219724+0000', 'last_fullsized': '2023-08-09T13:50:29.219724+0000', 'mapping_epoch': 88, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 88, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-08-09T13:50:28.150437+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-08-09T13:50:28.150437+0000', 'last_clean_scrub_stamp': '2023-08-09T13:50:28.150437+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-08-10T21:52:25.852199+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [5, 6], 'acting': [5, 6], 'avail_no_missing': ['5', '6'], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 5, 'acting_primary': 5, 'purged_snaps': []} 2023-08-09T13:50:34.807 INFO:tasks.cephfs.filesystem.ceph_manager:PG 11.1e is not active+clean 2023-08-09T13:50:34.807 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '11.1e', 'version': "0'0", 'reported_seq': 3, 'reported_epoch': 89, 'state': 'creating+activating', 'last_fresh': '2023-08-09T13:50:29.220103+0000', 'last_change': '2023-08-09T13:50:29.220103+0000', 'last_active': '2023-08-09T13:50:28.150437+0000', 'last_peered': '2023-08-09T13:50:28.150437+0000', 'last_clean': '2023-08-09T13:50:28.150437+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-08-09T13:50:29.220103+0000', 'last_undegraded': '2023-08-09T13:50:29.220103+0000', 'last_fullsized': '2023-08-09T13:50:29.220103+0000', 'mapping_epoch': 88, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 88, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-08-09T13:50:28.150437+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-08-09T13:50:28.150437+0000', 'last_clean_scrub_stamp': '2023-08-09T13:50:28.150437+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-08-10T19:44:56.828003+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [7, 6], 'acting': [7, 6], 'avail_no_missing': ['7', '6'], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 7, 'acting_primary': 7, 'purged_snaps': []} 2023-08-09T13:50:34.808 INFO:tasks.cephfs.filesystem.ceph_manager:PG 11.1f is not active+clean 2023-08-09T13:50:34.808 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '11.1f', 'version': "0'0", 'reported_seq': 3, 'reported_epoch': 89, 'state': 'creating+activating', 'last_fresh': '2023-08-09T13:50:29.223179+0000', 'last_change': '2023-08-09T13:50:29.223179+0000', 'last_active': '2023-08-09T13:50:28.150437+0000', 'last_peered': '2023-08-09T13:50:28.150437+0000', 'last_clean': '2023-08-09T13:50:28.150437+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-08-09T13:50:29.223179+0000', 'last_undegraded': '2023-08-09T13:50:29.223179+0000', 'last_fullsized': '2023-08-09T13:50:29.223179+0000', 'mapping_epoch': 88, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 88, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-08-09T13:50:28.150437+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-08-09T13:50:28.150437+0000', 'last_clean_scrub_stamp': '2023-08-09T13:50:28.150437+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-08-10T20:43:24.475264+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 3], 'acting': [2, 3], 'avail_no_missing': ['2', '3'], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2023-08-09T13:50:34.808 INFO:tasks.cephfs.filesystem.ceph_manager:PG 11.1c is not active+clean 2023-08-09T13:50:34.809 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '11.1c', 'version': "0'0", 'reported_seq': 4, 'reported_epoch': 89, 'state': 'creating+activating', 'last_fresh': '2023-08-09T13:50:29.250266+0000', 'last_change': '2023-08-09T13:50:29.225957+0000', 'last_active': '2023-08-09T13:50:28.150437+0000', 'last_peered': '2023-08-09T13:50:28.150437+0000', 'last_clean': '2023-08-09T13:50:28.150437+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-08-09T13:50:29.250266+0000', 'last_undegraded': '2023-08-09T13:50:29.250266+0000', 'last_fullsized': '2023-08-09T13:50:29.250266+0000', 'mapping_epoch': 88, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 88, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-08-09T13:50:28.150437+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-08-09T13:50:28.150437+0000', 'last_clean_scrub_stamp': '2023-08-09T13:50:28.150437+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-08-10T21:57:09.347886+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [6, 7], 'acting': [6, 7], 'avail_no_missing': ['6', '7'], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 6, 'acting_primary': 6, 'purged_snaps': []} 2023-08-09T13:50:34.809 INFO:tasks.cephfs.filesystem.ceph_manager:PG 11.1b is not active+clean 2023-08-09T13:50:34.809 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '11.1b', 'version': "0'0", 'reported_seq': 3, 'reported_epoch': 89, 'state': 'creating+activating', 'last_fresh': '2023-08-09T13:50:29.208495+0000', 'last_change': '2023-08-09T13:50:29.208495+0000', 'last_active': '2023-08-09T13:50:28.150437+0000', 'last_peered': '2023-08-09T13:50:28.150437+0000', 'last_clean': '2023-08-09T13:50:28.150437+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-08-09T13:50:29.208495+0000', 'last_undegraded': '2023-08-09T13:50:29.208495+0000', 'last_fullsized': '2023-08-09T13:50:29.208495+0000', 'mapping_epoch': 88, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 88, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-08-09T13:50:28.150437+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-08-09T13:50:28.150437+0000', 'last_clean_scrub_stamp': '2023-08-09T13:50:28.150437+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-08-10T16:20:35.941253+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [4, 7], 'acting': [4, 7], 'avail_no_missing': ['4', '7'], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 4, 'acting_primary': 4, 'purged_snaps': []} 2023-08-09T13:50:34.810 INFO:tasks.cephfs.filesystem.ceph_manager:PG 11.18 is not active+clean 2023-08-09T13:50:34.810 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '11.18', 'version': "0'0", 'reported_seq': 3, 'reported_epoch': 89, 'state': 'creating+activating', 'last_fresh': '2023-08-09T13:50:29.218450+0000', 'last_change': '2023-08-09T13:50:29.218450+0000', 'last_active': '2023-08-09T13:50:28.150437+0000', 'last_peered': '2023-08-09T13:50:28.150437+0000', 'last_clean': '2023-08-09T13:50:28.150437+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-08-09T13:50:29.218450+0000', 'last_undegraded': '2023-08-09T13:50:29.218450+0000', 'last_fullsized': '2023-08-09T13:50:29.218450+0000', 'mapping_epoch': 88, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 88, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-08-09T13:50:28.150437+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-08-09T13:50:28.150437+0000', 'last_clean_scrub_stamp': '2023-08-09T13:50:28.150437+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-08-10T15:00:53.649507+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 4], 'acting': [2, 4], 'avail_no_missing': ['2', '4'], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2023-08-09T13:50:34.810 INFO:tasks.cephfs.filesystem.ceph_manager:PG 11.16 is not active+clean 2023-08-09T13:50:34.811 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '11.16', 'version': "0'0", 'reported_seq': 3, 'reported_epoch': 89, 'state': 'creating+activating', 'last_fresh': '2023-08-09T13:50:29.223029+0000', 'last_change': '2023-08-09T13:50:29.223029+0000', 'last_active': '2023-08-09T13:50:28.150437+0000', 'last_peered': '2023-08-09T13:50:28.150437+0000', 'last_clean': '2023-08-09T13:50:28.150437+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-08-09T13:50:29.223029+0000', 'last_undegraded': '2023-08-09T13:50:29.223029+0000', 'last_fullsized': '2023-08-09T13:50:29.223029+0000', 'mapping_epoch': 88, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 88, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-08-09T13:50:28.150437+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-08-09T13:50:28.150437+0000', 'last_clean_scrub_stamp': '2023-08-09T13:50:28.150437+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-08-10T18:35:45.000310+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [5, 7], 'acting': [5, 7], 'avail_no_missing': ['5', '7'], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 5, 'acting_primary': 5, 'purged_snaps': []} 2023-08-09T13:50:34.811 INFO:tasks.cephfs.filesystem.ceph_manager:PG 11.17 is not active+clean 2023-08-09T13:50:34.811 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '11.17', 'version': "0'0", 'reported_seq': 3, 'reported_epoch': 89, 'state': 'creating+activating', 'last_fresh': '2023-08-09T13:50:29.221024+0000', 'last_change': '2023-08-09T13:50:29.221024+0000', 'last_active': '2023-08-09T13:50:28.150437+0000', 'last_peered': '2023-08-09T13:50:28.150437+0000', 'last_clean': '2023-08-09T13:50:28.150437+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-08-09T13:50:29.221024+0000', 'last_undegraded': '2023-08-09T13:50:29.221024+0000', 'last_fullsized': '2023-08-09T13:50:29.221024+0000', 'mapping_epoch': 88, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 88, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-08-09T13:50:28.150437+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-08-09T13:50:28.150437+0000', 'last_clean_scrub_stamp': '2023-08-09T13:50:28.150437+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-08-10T19:41:45.324636+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [7, 6], 'acting': [7, 6], 'avail_no_missing': ['7', '6'], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 7, 'acting_primary': 7, 'purged_snaps': []} 2023-08-09T13:50:34.812 INFO:tasks.cephfs.filesystem.ceph_manager:PG 11.13 is not active+clean 2023-08-09T13:50:34.812 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '11.13', 'version': "0'0", 'reported_seq': 3, 'reported_epoch': 89, 'state': 'creating+activating', 'last_fresh': '2023-08-09T13:50:29.220908+0000', 'last_change': '2023-08-09T13:50:29.220908+0000', 'last_active': '2023-08-09T13:50:28.150437+0000', 'last_peered': '2023-08-09T13:50:28.150437+0000', 'last_clean': '2023-08-09T13:50:28.150437+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-08-09T13:50:29.220908+0000', 'last_undegraded': '2023-08-09T13:50:29.220908+0000', 'last_fullsized': '2023-08-09T13:50:29.220908+0000', 'mapping_epoch': 88, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 88, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-08-09T13:50:28.150437+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-08-09T13:50:28.150437+0000', 'last_clean_scrub_stamp': '2023-08-09T13:50:28.150437+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-08-10T18:15:31.389069+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [5, 4], 'acting': [5, 4], 'avail_no_missing': ['5', '4'], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 5, 'acting_primary': 5, 'purged_snaps': []} 2023-08-09T13:50:34.812 INFO:tasks.cephfs.filesystem.ceph_manager:PG 11.11 is not active+clean 2023-08-09T13:50:34.813 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '11.11', 'version': "0'0", 'reported_seq': 3, 'reported_epoch': 89, 'state': 'creating+activating', 'last_fresh': '2023-08-09T13:50:29.224448+0000', 'last_change': '2023-08-09T13:50:29.224448+0000', 'last_active': '2023-08-09T13:50:28.150437+0000', 'last_peered': '2023-08-09T13:50:28.150437+0000', 'last_clean': '2023-08-09T13:50:28.150437+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-08-09T13:50:29.224448+0000', 'last_undegraded': '2023-08-09T13:50:29.224448+0000', 'last_fullsized': '2023-08-09T13:50:29.224448+0000', 'mapping_epoch': 88, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 88, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-08-09T13:50:28.150437+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-08-09T13:50:28.150437+0000', 'last_clean_scrub_stamp': '2023-08-09T13:50:28.150437+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-08-10T19:49:56.370072+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 3], 'acting': [2, 3], 'avail_no_missing': ['2', '3'], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2023-08-09T13:50:34.813 INFO:tasks.cephfs.filesystem.ceph_manager:PG 11.7 is not active+clean 2023-08-09T13:50:34.813 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '11.7', 'version': "0'0", 'reported_seq': 3, 'reported_epoch': 89, 'state': 'creating+activating', 'last_fresh': '2023-08-09T13:50:29.223626+0000', 'last_change': '2023-08-09T13:50:29.223626+0000', 'last_active': '2023-08-09T13:50:28.150437+0000', 'last_peered': '2023-08-09T13:50:28.150437+0000', 'last_clean': '2023-08-09T13:50:28.150437+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-08-09T13:50:29.223626+0000', 'last_undegraded': '2023-08-09T13:50:29.223626+0000', 'last_fullsized': '2023-08-09T13:50:29.223626+0000', 'mapping_epoch': 88, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 88, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-08-09T13:50:28.150437+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-08-09T13:50:28.150437+0000', 'last_clean_scrub_stamp': '2023-08-09T13:50:28.150437+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-08-10T18:06:21.772376+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [5, 1], 'acting': [5, 1], 'avail_no_missing': ['5', '1'], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 5, 'acting_primary': 5, 'purged_snaps': []} 2023-08-09T13:50:34.814 INFO:tasks.cephfs.filesystem.ceph_manager:PG 11.4 is not active+clean 2023-08-09T13:50:34.814 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '11.4', 'version': "0'0", 'reported_seq': 3, 'reported_epoch': 89, 'state': 'creating+activating', 'last_fresh': '2023-08-09T13:50:29.216528+0000', 'last_change': '2023-08-09T13:50:29.216528+0000', 'last_active': '2023-08-09T13:50:28.150437+0000', 'last_peered': '2023-08-09T13:50:28.150437+0000', 'last_clean': '2023-08-09T13:50:28.150437+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-08-09T13:50:29.216528+0000', 'last_undegraded': '2023-08-09T13:50:29.216528+0000', 'last_fullsized': '2023-08-09T13:50:29.216528+0000', 'mapping_epoch': 88, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 88, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-08-09T13:50:28.150437+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-08-09T13:50:28.150437+0000', 'last_clean_scrub_stamp': '2023-08-09T13:50:28.150437+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-08-10T21:36:45.909480+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [7, 6], 'acting': [7, 6], 'avail_no_missing': ['7', '6'], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 7, 'acting_primary': 7, 'purged_snaps': []} 2023-08-09T13:50:34.814 INFO:tasks.cephfs.filesystem.ceph_manager:PG 11.2 is not active+clean 2023-08-09T13:50:34.815 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '11.2', 'version': "0'0", 'reported_seq': 3, 'reported_epoch': 89, 'state': 'creating+activating', 'last_fresh': '2023-08-09T13:50:29.217549+0000', 'last_change': '2023-08-09T13:50:29.217549+0000', 'last_active': '2023-08-09T13:50:28.150437+0000', 'last_peered': '2023-08-09T13:50:28.150437+0000', 'last_clean': '2023-08-09T13:50:28.150437+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-08-09T13:50:29.217549+0000', 'last_undegraded': '2023-08-09T13:50:29.217549+0000', 'last_fullsized': '2023-08-09T13:50:29.217549+0000', 'mapping_epoch': 88, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 88, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-08-09T13:50:28.150437+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-08-09T13:50:28.150437+0000', 'last_clean_scrub_stamp': '2023-08-09T13:50:28.150437+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-08-11T00:03:22.851529+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 3], 'acting': [2, 3], 'avail_no_missing': ['2', '3'], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2023-08-09T13:50:34.815 INFO:tasks.cephfs.filesystem.ceph_manager:PG 11.a is not active+clean 2023-08-09T13:50:34.815 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '11.a', 'version': "0'0", 'reported_seq': 3, 'reported_epoch': 89, 'state': 'creating+activating', 'last_fresh': '2023-08-09T13:50:29.212247+0000', 'last_change': '2023-08-09T13:50:29.212247+0000', 'last_active': '2023-08-09T13:50:28.150437+0000', 'last_peered': '2023-08-09T13:50:28.150437+0000', 'last_clean': '2023-08-09T13:50:28.150437+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-08-09T13:50:29.212247+0000', 'last_undegraded': '2023-08-09T13:50:29.212247+0000', 'last_fullsized': '2023-08-09T13:50:29.212247+0000', 'mapping_epoch': 88, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 88, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-08-09T13:50:28.150437+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-08-09T13:50:28.150437+0000', 'last_clean_scrub_stamp': '2023-08-09T13:50:28.150437+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-08-10T19:55:58.171731+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 6], 'acting': [3, 6], 'avail_no_missing': ['3', '6'], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2023-08-09T13:50:34.816 INFO:tasks.cephfs.filesystem.ceph_manager:PG 11.b is not active+clean 2023-08-09T13:50:34.816 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '11.b', 'version': "0'0", 'reported_seq': 3, 'reported_epoch': 89, 'state': 'creating+activating', 'last_fresh': '2023-08-09T13:50:29.211663+0000', 'last_change': '2023-08-09T13:50:29.211663+0000', 'last_active': '2023-08-09T13:50:28.150437+0000', 'last_peered': '2023-08-09T13:50:28.150437+0000', 'last_clean': '2023-08-09T13:50:28.150437+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-08-09T13:50:29.211663+0000', 'last_undegraded': '2023-08-09T13:50:29.211663+0000', 'last_fullsized': '2023-08-09T13:50:29.211663+0000', 'mapping_epoch': 88, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 88, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-08-09T13:50:28.150437+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-08-09T13:50:28.150437+0000', 'last_clean_scrub_stamp': '2023-08-09T13:50:28.150437+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-08-11T01:04:03.965638+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 1], 'acting': [3, 1], 'avail_no_missing': ['3', '1'], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2023-08-09T13:50:34.816 INFO:tasks.cephfs.filesystem.ceph_manager:PG 11.8 is not active+clean 2023-08-09T13:50:34.817 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '11.8', 'version': "0'0", 'reported_seq': 3, 'reported_epoch': 89, 'state': 'creating+activating', 'last_fresh': '2023-08-09T13:50:29.213241+0000', 'last_change': '2023-08-09T13:50:29.213241+0000', 'last_active': '2023-08-09T13:50:28.150437+0000', 'last_peered': '2023-08-09T13:50:28.150437+0000', 'last_clean': '2023-08-09T13:50:28.150437+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-08-09T13:50:29.213241+0000', 'last_undegraded': '2023-08-09T13:50:29.213241+0000', 'last_fullsized': '2023-08-09T13:50:29.213241+0000', 'mapping_epoch': 88, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 88, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-08-09T13:50:28.150437+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-08-09T13:50:28.150437+0000', 'last_clean_scrub_stamp': '2023-08-09T13:50:28.150437+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-08-10T22:09:32.464173+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 6], 'acting': [3, 6], 'avail_no_missing': ['3', '6'], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2023-08-09T13:50:34.817 INFO:tasks.cephfs.filesystem.ceph_manager:PG 11.9 is not active+clean 2023-08-09T13:50:34.817 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '11.9', 'version': "0'0", 'reported_seq': 3, 'reported_epoch': 89, 'state': 'creating+activating', 'last_fresh': '2023-08-09T13:50:29.211902+0000', 'last_change': '2023-08-09T13:50:29.211902+0000', 'last_active': '2023-08-09T13:50:28.150437+0000', 'last_peered': '2023-08-09T13:50:28.150437+0000', 'last_clean': '2023-08-09T13:50:28.150437+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-08-09T13:50:29.211902+0000', 'last_undegraded': '2023-08-09T13:50:29.211902+0000', 'last_fullsized': '2023-08-09T13:50:29.211902+0000', 'mapping_epoch': 88, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 88, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-08-09T13:50:28.150437+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-08-09T13:50:28.150437+0000', 'last_clean_scrub_stamp': '2023-08-09T13:50:28.150437+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-08-10T15:32:31.232979+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 6], 'acting': [3, 6], 'avail_no_missing': ['3', '6'], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2023-08-09T13:50:34.818 INFO:tasks.cephfs.filesystem.ceph_manager:PG 11.d is not active+clean 2023-08-09T13:50:34.818 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '11.d', 'version': "0'0", 'reported_seq': 3, 'reported_epoch': 89, 'state': 'creating+activating', 'last_fresh': '2023-08-09T13:50:29.220533+0000', 'last_change': '2023-08-09T13:50:29.220533+0000', 'last_active': '2023-08-09T13:50:28.150437+0000', 'last_peered': '2023-08-09T13:50:28.150437+0000', 'last_clean': '2023-08-09T13:50:28.150437+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-08-09T13:50:29.220533+0000', 'last_undegraded': '2023-08-09T13:50:29.220533+0000', 'last_fullsized': '2023-08-09T13:50:29.220533+0000', 'mapping_epoch': 88, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 88, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-08-09T13:50:28.150437+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-08-09T13:50:28.150437+0000', 'last_clean_scrub_stamp': '2023-08-09T13:50:28.150437+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-08-10T17:52:40.304270+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 4], 'acting': [2, 4], 'avail_no_missing': ['2', '4'], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2023-08-09T13:50:34.819 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2023-08-09T13:50:35.061 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:35.060+0000 7f1cfb555640 1 Processor -- start 2023-08-09T13:50:35.062 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:35.060+0000 7f1cfb555640 1 -- start start 2023-08-09T13:50:35.063 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:35.062+0000 7f1cfb555640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f1cf40c8750 0x7f1cf40ad340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:50:35.064 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:35.062+0000 7f1cfb555640 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7f1cf40acdf0 0x7f1cf415b2c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:50:35.064 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:35.062+0000 7f1cfb555640 1 -- --> v1:172.21.15.203:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1cf40cbd90 con 0x7f1cf40c8b30 2023-08-09T13:50:35.064 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:35.062+0000 7f1cfb555640 1 -- --> v2:172.21.15.43:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1cf415e810 con 0x7f1cf40c8750 2023-08-09T13:50:35.064 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:35.062+0000 7f1cfb555640 1 -- --> v2:172.21.15.203:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1cf40c9f70 con 0x7f1cf40acdf0 2023-08-09T13:50:35.065 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:35.062+0000 7f1cfa553640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f1cf40c8750 0x7f1cf40ad340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:50:35.065 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:35.062+0000 7f1cfa553640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f1cf40c8750 0x7f1cf40ad340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:44976/0 (socket says 172.21.15.43:44976) 2023-08-09T13:50:35.065 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:35.062+0000 7f1cfa553640 1 -- 172.21.15.43:0/2255626830 learned_addr learned my addr 172.21.15.43:0/2255626830 (peer_addr_for_me v2:172.21.15.43:0/0) 2023-08-09T13:50:35.066 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:35.062+0000 7f1cfad54640 1 --2- 172.21.15.43:0/2255626830 >> v2:172.21.15.203:3300/0 conn(0x7f1cf40acdf0 0x7f1cf415b2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:50:35.066 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:35.063+0000 7f1cfa553640 1 -- 172.21.15.43:0/2255626830 >> v2:172.21.15.203:3300/0 conn(0x7f1cf40acdf0 msgr2=0x7f1cf415b2c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-09T13:50:35.066 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:35.063+0000 7f1cfa553640 1 --2- 172.21.15.43:0/2255626830 >> v2:172.21.15.203:3300/0 conn(0x7f1cf40acdf0 0x7f1cf415b2c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:50:35.066 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:35.063+0000 7f1cfa553640 1 -- 172.21.15.43:0/2255626830 >> v1:172.21.15.203:6789/0 conn(0x7f1cf40c8b30 legacy=0x7f1cf406f410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-09T13:50:35.067 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:35.063+0000 7f1cfa553640 1 -- 172.21.15.43:0/2255626830 --> v2:172.21.15.43:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1cf40ca2b0 con 0x7f1cf40c8750 2023-08-09T13:50:35.067 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:35.063+0000 7f1cfad54640 1 --2- 172.21.15.43:0/2255626830 >> v2:172.21.15.203:3300/0 conn(0x7f1cf40acdf0 0x7f1cf415b2c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-08-09T13:50:35.067 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:35.064+0000 7f1cfa553640 1 --2- 172.21.15.43:0/2255626830 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f1cf40c8750 0x7f1cf40ad340 secure :-1 s=READY pgs=1026 cs=0 l=1 rev1=1 crypto rx=0x7f1cf4070560 tx=0x7f1cf002b880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5d7162cc0857601 server_cookie=0 in_seq=0 out_seq=0 2023-08-09T13:50:35.067 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:35.064+0000 7f1cf9551640 1 -- 172.21.15.43:0/2255626830 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1cf00317e0 con 0x7f1cf40c8750 2023-08-09T13:50:35.067 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:35.064+0000 7f1cf9551640 1 -- 172.21.15.43:0/2255626830 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1cf0031940 con 0x7f1cf40c8750 2023-08-09T13:50:35.067 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:35.064+0000 7f1cf9551640 1 -- 172.21.15.43:0/2255626830 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1cf0031cd0 con 0x7f1cf40c8750 2023-08-09T13:50:35.069 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:35.065+0000 7f1cfb555640 1 -- 172.21.15.43:0/2255626830 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f1cf40c8750 msgr2=0x7f1cf40ad340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:50:35.069 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:35.065+0000 7f1cfb555640 1 --2- 172.21.15.43:0/2255626830 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f1cf40c8750 0x7f1cf40ad340 secure :-1 s=READY pgs=1026 cs=0 l=1 rev1=1 crypto rx=0x7f1cf4070560 tx=0x7f1cf002b880 comp rx=0 tx=0).stop 2023-08-09T13:50:35.069 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:35.065+0000 7f1cfb555640 1 -- 172.21.15.43:0/2255626830 shutdown_connections 2023-08-09T13:50:35.069 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:35.065+0000 7f1cfb555640 1 --2- 172.21.15.43:0/2255626830 >> v2:172.21.15.203:3300/0 conn(0x7f1cf40acdf0 0x7f1cf415b2c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:50:35.069 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:35.065+0000 7f1cfb555640 1 --2- 172.21.15.43:0/2255626830 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f1cf40c8750 0x7f1cf40ad340 unknown :-1 s=CLOSED pgs=1026 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:50:35.070 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:35.065+0000 7f1cfb555640 1 -- 172.21.15.43:0/2255626830 >> 172.21.15.43:0/2255626830 conn(0x7f1cf40cde90 msgr2=0x7f1cf40ce2a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-09T13:50:35.070 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:35.065+0000 7f1cfb555640 1 -- 172.21.15.43:0/2255626830 shutdown_connections 2023-08-09T13:50:35.070 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:35.065+0000 7f1cfb555640 1 -- 172.21.15.43:0/2255626830 wait complete. 2023-08-09T13:50:35.070 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:35.067+0000 7f1cfb555640 1 Processor -- start 2023-08-09T13:50:35.070 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:35.067+0000 7f1cfb555640 1 -- start start 2023-08-09T13:50:35.071 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:35.068+0000 7f1cfb555640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f1cf40acdf0 0x7f1cf4099c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:50:35.072 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:35.068+0000 7f1cfb555640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f1cf40c8750 0x7f1cf4157910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:50:35.072 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:35.068+0000 7f1cfb555640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f1cf40c8b30 0x7f1cf409a8b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:50:35.072 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:35.068+0000 7f1cfb555640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1cf409a160 con 0x7f1cf40acdf0 2023-08-09T13:50:35.072 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:35.068+0000 7f1cfb555640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1cf4098b80 con 0x7f1cf40c8b30 2023-08-09T13:50:35.072 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:35.068+0000 7f1cfb555640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1cf4098e60 con 0x7f1cf40c8750 2023-08-09T13:50:35.072 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:35.068+0000 7f1cfa553640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f1cf40acdf0 0x7f1cf4099c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:50:35.073 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:35.069+0000 7f1cfa553640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f1cf40acdf0 0x7f1cf4099c20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:44984/0 (socket says 172.21.15.43:44984) 2023-08-09T13:50:35.073 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:35.069+0000 7f1cfa553640 1 -- 172.21.15.43:0/2791367127 learned_addr learned my addr 172.21.15.43:0/2791367127 (peer_addr_for_me v2:172.21.15.43:0/0) 2023-08-09T13:50:35.073 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:35.069+0000 7f1cf9d52640 1 --2- 172.21.15.43:0/2791367127 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f1cf40c8750 0x7f1cf4157910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:50:35.073 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:35.069+0000 7f1cfad54640 1 --2- 172.21.15.43:0/2791367127 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f1cf40c8b30 0x7f1cf409a8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:50:35.073 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:35.069+0000 7f1cfa553640 1 -- 172.21.15.43:0/2791367127 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f1cf40c8750 msgr2=0x7f1cf4157910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:50:35.073 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:35.069+0000 7f1cfa553640 1 --2- 172.21.15.43:0/2791367127 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f1cf40c8750 0x7f1cf4157910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:50:35.073 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:35.069+0000 7f1cfa553640 1 -- 172.21.15.43:0/2791367127 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f1cf40c8b30 msgr2=0x7f1cf409a8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:50:35.074 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:35.069+0000 7f1cfa553640 1 --2- 172.21.15.43:0/2791367127 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f1cf40c8b30 0x7f1cf409a8b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:50:35.074 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:35.069+0000 7f1cfa553640 1 -- 172.21.15.43:0/2791367127 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1cf00094d0 con 0x7f1cf40acdf0 2023-08-09T13:50:35.074 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:35.069+0000 7f1cfad54640 1 --2- 172.21.15.43:0/2791367127 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f1cf40c8b30 0x7f1cf409a8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-08-09T13:50:35.074 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:35.069+0000 7f1cfa553640 1 --2- 172.21.15.43:0/2791367127 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f1cf40acdf0 0x7f1cf4099c20 secure :-1 s=READY pgs=1027 cs=0 l=1 rev1=1 crypto rx=0x7f1cf002b8c0 tx=0x7f1cf00046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-09T13:50:35.074 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:35.070+0000 7f1ceaffd640 1 -- 172.21.15.43:0/2791367127 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1cf00329c0 con 0x7f1cf40acdf0 2023-08-09T13:50:35.074 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:35.070+0000 7f1ceaffd640 1 -- 172.21.15.43:0/2791367127 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1cf0032b40 con 0x7f1cf40acdf0 2023-08-09T13:50:35.074 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:35.070+0000 7f1ceaffd640 1 -- 172.21.15.43:0/2791367127 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1cf00453f0 con 0x7f1cf40acdf0 2023-08-09T13:50:35.075 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:35.070+0000 7f1cfb555640 1 -- 172.21.15.43:0/2791367127 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1cf4099140 con 0x7f1cf40acdf0 2023-08-09T13:50:35.075 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:35.070+0000 7f1cfb555640 1 -- 172.21.15.43:0/2791367127 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1cf4099700 con 0x7f1cf40acdf0 2023-08-09T13:50:35.075 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:35.071+0000 7f1cfb555640 1 -- 172.21.15.43:0/2791367127 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1cf40c9230 con 0x7f1cf40acdf0 2023-08-09T13:50:35.076 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:35.072+0000 7f1ceaffd640 1 -- 172.21.15.43:0/2791367127 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 29) v1 ==== 88443+0+0 (secure 0 0 0) 0x7f1cf004d3f0 con 0x7f1cf40acdf0 2023-08-09T13:50:35.076 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:35.073+0000 7f1ceaffd640 1 --2- 172.21.15.43:0/2791367127 >> [v2:172.21.15.203:6832/3325141227,v1:172.21.15.203:6833/3325141227] conn(0x7f1cd806a170 0x7f1cd806c630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:50:35.076 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:35.073+0000 7f1ceaffd640 1 -- 172.21.15.43:0/2791367127 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(90..90 src has 1..90) v4 ==== 6426+0+0 (secure 0 0 0) 0x7f1cf00aa7f0 con 0x7f1cf40acdf0 2023-08-09T13:50:35.076 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:35.074+0000 7f1cf9d52640 1 --2- 172.21.15.43:0/2791367127 >> [v2:172.21.15.203:6832/3325141227,v1:172.21.15.203:6833/3325141227] conn(0x7f1cd806a170 0x7f1cd806c630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:50:35.076 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:35.075+0000 7f1cf9d52640 1 --2- 172.21.15.43:0/2791367127 >> [v2:172.21.15.203:6832/3325141227,v1:172.21.15.203:6833/3325141227] conn(0x7f1cd806a170 0x7f1cd806c630 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f1ce4006fd0 tx=0x7f1ce4008040 comp rx=0 tx=0).ready entity=mgr.5519 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-09T13:50:35.079 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:35.077+0000 7f1ceaffd640 1 -- 172.21.15.43:0/2791367127 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178066 (secure 0 0 0) 0x7f1cf0075c90 con 0x7f1cf40acdf0 2023-08-09T13:50:35.349 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:35.347+0000 7f1cfb555640 1 -- 172.21.15.43:0/2791367127 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "status", "format": "json"} v 0) v1 -- 0x7f1cf40739f0 con 0x7f1cf40acdf0 2023-08-09T13:50:35.350 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:35.348+0000 7f1ceaffd640 1 -- 172.21.15.43:0/2791367127 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json"}]=0 v0) v1 ==== 72+0+3298 (secure 0 0 0) 0x7f1cf00792b0 con 0x7f1cf40acdf0 2023-08-09T13:50:35.350 INFO:teuthology.orchestra.run.smithi043.stdout: 2023-08-09T13:50:35.350 INFO:teuthology.orchestra.run.smithi043.stdout:{"fsid":"daac87ee-1337-4250-a955-aea4e408a39b","health":{"status":"HEALTH_OK","checks":{},"mutes":[]},"election_epoch":4,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":1013,"monmap":{"epoch":1,"min_mon_release_name":"reef","num_mons":3},"osdmap":{"epoch":90,"num_osds":8,"num_up_osds":8,"osd_up_since":1691588033,"num_in_osds":8,"osd_in_since":1691588025,"num_remapped_pgs":0},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":74},{"state_name":"creating+activating","count":31}],"num_pgs":105,"num_pools":4,"num_objects":8,"data_bytes":592014,"bytes_used":285704192,"bytes_avail":772808409088,"bytes_total":773094113280,"inactive_pgs_ratio":0.29523810744285583,"write_bytes_sec":1687,"read_op_per_sec":0,"write_op_per_sec":1},"fsmap":{"epoch":35,"id":4,"up":1,"in":1,"max":1,"by_rank":[{"filesystem_id":4,"rank":0,"name":"d","status":"up:active","gid":7011}],"up:standby":3},"mgrmap":{"available":true,"num_standbys":1,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":10,"modified":"2023-08-09T13:50:14.226045+0000","services":{"mds":{"daemons":{"summary":"","a":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}},"c":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}},"d":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}}}},"mgr":{"daemons":{"summary":"","x":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}},"y":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}}}},"mon":{"daemons":{"summary":"","a":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}},"b":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}},"c":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}}}},"osd":{"daemons":{"summary":"","0":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}},"1":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}},"2":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}},"3":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}},"4":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}},"5":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}},"6":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}},"7":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}}}}}},"progress_events":{}} 2023-08-09T13:50:35.354 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:35.351+0000 7f1cfb555640 1 -- 172.21.15.43:0/2791367127 >> [v2:172.21.15.203:6832/3325141227,v1:172.21.15.203:6833/3325141227] conn(0x7f1cd806a170 msgr2=0x7f1cd806c630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:50:35.354 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:35.351+0000 7f1cfb555640 1 --2- 172.21.15.43:0/2791367127 >> [v2:172.21.15.203:6832/3325141227,v1:172.21.15.203:6833/3325141227] conn(0x7f1cd806a170 0x7f1cd806c630 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f1ce4006fd0 tx=0x7f1ce4008040 comp rx=0 tx=0).stop 2023-08-09T13:50:35.355 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:35.351+0000 7f1cfb555640 1 -- 172.21.15.43:0/2791367127 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f1cf40acdf0 msgr2=0x7f1cf4099c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:50:35.355 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:35.351+0000 7f1cfb555640 1 --2- 172.21.15.43:0/2791367127 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f1cf40acdf0 0x7f1cf4099c20 secure :-1 s=READY pgs=1027 cs=0 l=1 rev1=1 crypto rx=0x7f1cf002b8c0 tx=0x7f1cf00046c0 comp rx=0 tx=0).stop 2023-08-09T13:50:35.355 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:35.351+0000 7f1cfb555640 1 -- 172.21.15.43:0/2791367127 shutdown_connections 2023-08-09T13:50:35.355 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:35.351+0000 7f1cfb555640 1 --2- 172.21.15.43:0/2791367127 >> [v2:172.21.15.203:6832/3325141227,v1:172.21.15.203:6833/3325141227] conn(0x7f1cd806a170 0x7f1cd806c630 unknown :-1 s=CLOSED pgs=68 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:50:35.355 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:35.351+0000 7f1cfb555640 1 --2- 172.21.15.43:0/2791367127 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f1cf40c8b30 0x7f1cf409a8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:50:35.355 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:35.351+0000 7f1cfb555640 1 --2- 172.21.15.43:0/2791367127 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f1cf40c8750 0x7f1cf4157910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:50:35.355 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:35.351+0000 7f1cfb555640 1 --2- 172.21.15.43:0/2791367127 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f1cf40acdf0 0x7f1cf4099c20 unknown :-1 s=CLOSED pgs=1027 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:50:35.356 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:35.351+0000 7f1cfb555640 1 -- 172.21.15.43:0/2791367127 >> 172.21.15.43:0/2791367127 conn(0x7f1cf40cde90 msgr2=0x7f1cf4159500 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-09T13:50:35.356 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:35.351+0000 7f1cfb555640 1 -- 172.21.15.43:0/2791367127 shutdown_connections 2023-08-09T13:50:35.356 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:35.351+0000 7f1cfb555640 1 -- 172.21.15.43:0/2791367127 wait complete. 2023-08-09T13:50:35.371 INFO:tasks.cephfs.filesystem.ceph_manager:no progress seen, keeping timeout for now 2023-08-09T13:50:35.372 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-08-09T13:50:35.602 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:35.598+0000 7f5cd8d5b640 1 Processor -- start 2023-08-09T13:50:35.602 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:35.598+0000 7f5cd8d5b640 1 -- start start 2023-08-09T13:50:35.602 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:35.600+0000 7f5cd8d5b640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f5cd40cf760 0x7f5cd406dcb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:50:35.602 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:35.600+0000 7f5cd8d5b640 1 -- --> v1:172.21.15.43:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5cd40af1a0 con 0x7f5cd406b140 2023-08-09T13:50:35.603 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:35.600+0000 7f5cd8d5b640 1 -- --> v1:172.21.15.203:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5cd40cdc20 con 0x7f5cd40af6f0 2023-08-09T13:50:35.603 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:35.600+0000 7f5cd8d5b640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5cd406be10 con 0x7f5cd40cf760 2023-08-09T13:50:35.604 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:35.600+0000 7f5cd259c640 1 --1- >> v1:172.21.15.43:6789/0 conn(0x7f5cd406b140 0x7f5cd406b520 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.43:6789/0 says I am v1:172.21.15.43:55208/0 (socket says 172.21.15.43:55208) 2023-08-09T13:50:35.604 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:35.600+0000 7f5cd259c640 1 -- 172.21.15.43:0/2551971871 learned_addr learned my addr 172.21.15.43:0/2551971871 (peer_addr_for_me v1:172.21.15.43:0/0) 2023-08-09T13:50:35.604 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:35.600+0000 7f5cd159a640 1 -- 172.21.15.43:0/2551971871 <== mon.0 v1:172.21.15.43:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3256622566 0 0) 0x7f5cd40af1a0 con 0x7f5cd406b140 2023-08-09T13:50:35.605 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:35.601+0000 7f5cd159a640 1 -- 172.21.15.43:0/2551971871 --> v1:172.21.15.43:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f5cc0003520 con 0x7f5cd406b140 2023-08-09T13:50:35.605 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:35.601+0000 7f5cd2d9d640 1 --2- 172.21.15.43:0/2551971871 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f5cd40cf760 0x7f5cd406dcb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:50:35.607 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:35.601+0000 7f5cd159a640 1 -- 172.21.15.43:0/2551971871 <== mon.0 v1:172.21.15.43:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1837121781 0 0) 0x7f5cc0003520 con 0x7f5cd406b140 2023-08-09T13:50:35.607 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:35.601+0000 7f5cd159a640 1 -- 172.21.15.43:0/2551971871 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f5cd40cf760 msgr2=0x7f5cd406dcb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-09T13:50:35.607 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:35.601+0000 7f5cd159a640 1 --2- 172.21.15.43:0/2551971871 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f5cd40cf760 0x7f5cd406dcb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:50:35.607 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:35.601+0000 7f5cd159a640 1 -- 172.21.15.43:0/2551971871 >> v1:172.21.15.203:6789/0 conn(0x7f5cd40af6f0 legacy=0x7f5cd406a8d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:50:35.607 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:35.601+0000 7f5cd159a640 1 -- 172.21.15.43:0/2551971871 --> v1:172.21.15.43:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5cd4160100 con 0x7f5cd406b140 2023-08-09T13:50:35.607 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:35.601+0000 7f5cd159a640 1 -- 172.21.15.43:0/2551971871 <== mon.0 v1:172.21.15.43:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1285150740 0 0) 0x7f5cc4001b70 con 0x7f5cd406b140 2023-08-09T13:50:35.607 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:35.601+0000 7f5cd159a640 1 -- 172.21.15.43:0/2551971871 >> v1:172.21.15.43:6789/0 conn(0x7f5cd406b140 legacy=0x7f5cd406b520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:50:35.608 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:35.601+0000 7f5cd159a640 1 --2- 172.21.15.43:0/2551971871 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f5cc0003a10 0x7f5cc0003e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:50:35.608 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:35.601+0000 7f5cd159a640 1 --2- 172.21.15.43:0/2551971871 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f5cd40cf760 0x7f5cc00047c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:50:35.608 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:35.601+0000 7f5cd159a640 1 --2- 172.21.15.43:0/2551971871 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f5cc0004d00 0x7f5cc0005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:50:35.608 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:35.601+0000 7f5cd159a640 1 -- 172.21.15.43:0/2551971871 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5cd406be10 con 0x7f5cc0003a10 2023-08-09T13:50:35.608 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:35.601+0000 7f5cd159a640 1 -- 172.21.15.43:0/2551971871 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5cc000bac0 con 0x7f5cc0004d00 2023-08-09T13:50:35.608 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:35.601+0000 7f5cd159a640 1 -- 172.21.15.43:0/2551971871 --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5cc000bc80 con 0x7f5cd40cf760 2023-08-09T13:50:35.608 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:35.601+0000 7f5cd259c640 1 --2- 172.21.15.43:0/2551971871 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f5cc0003a10 0x7f5cc0003e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:50:35.609 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:35.602+0000 7f5cd1d9b640 1 --2- 172.21.15.43:0/2551971871 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f5cd40cf760 0x7f5cc00047c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:50:35.609 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:35.602+0000 7f5cd259c640 1 -- 172.21.15.43:0/2551971871 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f5cd40cf760 msgr2=0x7f5cc00047c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-09T13:50:35.609 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:35.602+0000 7f5cd259c640 1 --2- 172.21.15.43:0/2551971871 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f5cd40cf760 0x7f5cc00047c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:50:35.609 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:35.602+0000 7f5cd259c640 1 -- 172.21.15.43:0/2551971871 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f5cc0004d00 msgr2=0x7f5cc0005180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-09T13:50:35.609 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:35.602+0000 7f5cd259c640 1 --2- 172.21.15.43:0/2551971871 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f5cc0004d00 0x7f5cc0005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:50:35.609 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:35.602+0000 7f5cd259c640 1 -- 172.21.15.43:0/2551971871 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5cc0003520 con 0x7f5cc0003a10 2023-08-09T13:50:35.609 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:35.603+0000 7f5cd259c640 1 --2- 172.21.15.43:0/2551971871 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f5cc0003a10 0x7f5cc0003e30 secure :-1 s=READY pgs=1029 cs=0 l=1 rev1=1 crypto rx=0x7f5cc400c4f0 tx=0x7f5cc00057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2362540f8b70e293 server_cookie=0 in_seq=0 out_seq=0 2023-08-09T13:50:35.610 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:35.603+0000 7f5cd159a640 1 -- 172.21.15.43:0/2551971871 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5cc402cd50 con 0x7f5cc0003a10 2023-08-09T13:50:35.610 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:35.603+0000 7f5cd159a640 1 -- 172.21.15.43:0/2551971871 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5cc4005b20 con 0x7f5cc0003a10 2023-08-09T13:50:35.610 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:35.603+0000 7f5cd159a640 1 -- 172.21.15.43:0/2551971871 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5cc4007070 con 0x7f5cc0003a10 2023-08-09T13:50:35.610 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:35.604+0000 7f5cd8d5b640 1 -- 172.21.15.43:0/2551971871 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f5cc0003a10 msgr2=0x7f5cc0003e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:50:35.610 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:35.604+0000 7f5cd8d5b640 1 --2- 172.21.15.43:0/2551971871 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f5cc0003a10 0x7f5cc0003e30 secure :-1 s=READY pgs=1029 cs=0 l=1 rev1=1 crypto rx=0x7f5cc400c4f0 tx=0x7f5cc00057f0 comp rx=0 tx=0).stop 2023-08-09T13:50:35.610 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:35.604+0000 7f5cd2d9d640 1 -- 172.21.15.43:0/2551971871 reap_dead start 2023-08-09T13:50:35.610 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:35.604+0000 7f5cd8d5b640 1 -- 172.21.15.43:0/2551971871 shutdown_connections 2023-08-09T13:50:35.610 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:35.604+0000 7f5cd8d5b640 1 -- 172.21.15.43:0/2551971871 >> 172.21.15.43:0/2551971871 conn(0x7f5cd40ccd80 msgr2=0x7f5cd40cd190 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-09T13:50:35.611 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:35.604+0000 7f5cd8d5b640 1 -- 172.21.15.43:0/2551971871 shutdown_connections 2023-08-09T13:50:35.611 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:35.604+0000 7f5cd8d5b640 1 -- 172.21.15.43:0/2551971871 wait complete. 2023-08-09T13:50:35.612 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:35.606+0000 7f5cd8d5b640 1 Processor -- start 2023-08-09T13:50:35.612 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:35.606+0000 7f5cd8d5b640 1 -- start start 2023-08-09T13:50:35.613 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:35.607+0000 7f5cd8d5b640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f5cc0003a10 0x7f5cd41f4aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:50:35.613 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:35.607+0000 7f5cd8d5b640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f5cd41f2f80 0x7f5cd41f1d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:50:35.613 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:35.607+0000 7f5cd8d5b640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f5cd41f2280 0x7f5cd40ada30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:50:35.613 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:35.607+0000 7f5cd8d5b640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5cd406afe0 con 0x7f5cc0003a10 2023-08-09T13:50:35.613 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:35.607+0000 7f5cd8d5b640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5cd40b4e60 con 0x7f5cd41f2f80 2023-08-09T13:50:35.613 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:35.607+0000 7f5cd8d5b640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5cd41f2700 con 0x7f5cd41f2280 2023-08-09T13:50:35.613 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:35.607+0000 7f5cd259c640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f5cc0003a10 0x7f5cd41f4aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:50:35.614 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:35.607+0000 7f5cd259c640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f5cc0003a10 0x7f5cd41f4aa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:45000/0 (socket says 172.21.15.43:45000) 2023-08-09T13:50:35.614 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:35.607+0000 7f5cd259c640 1 -- 172.21.15.43:0/1138981353 learned_addr learned my addr 172.21.15.43:0/1138981353 (peer_addr_for_me v2:172.21.15.43:0/0) 2023-08-09T13:50:35.614 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:35.608+0000 7f5cd259c640 1 -- 172.21.15.43:0/1138981353 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f5cd41f2280 msgr2=0x7f5cd40ada30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:50:35.614 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:35.608+0000 7f5cd259c640 1 --2- 172.21.15.43:0/1138981353 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f5cd41f2280 0x7f5cd40ada30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:50:35.614 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:35.608+0000 7f5cd259c640 1 -- 172.21.15.43:0/1138981353 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f5cd41f2f80 msgr2=0x7f5cd41f1d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:50:35.614 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:35.608+0000 7f5cd259c640 1 --2- 172.21.15.43:0/1138981353 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f5cd41f2f80 0x7f5cd41f1d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:50:35.614 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:35.608+0000 7f5cd259c640 1 -- 172.21.15.43:0/1138981353 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5cd41f28f0 con 0x7f5cc0003a10 2023-08-09T13:50:35.614 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:35.608+0000 7f5cd259c640 1 --2- 172.21.15.43:0/1138981353 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f5cc0003a10 0x7f5cd41f4aa0 secure :-1 s=READY pgs=1030 cs=0 l=1 rev1=1 crypto rx=0x7f5cc4001e30 tx=0x7f5cc4001f00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-09T13:50:35.615 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:35.608+0000 7f5ccaffd640 1 -- 172.21.15.43:0/1138981353 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5cc4007480 con 0x7f5cc0003a10 2023-08-09T13:50:35.615 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:35.608+0000 7f5ccaffd640 1 -- 172.21.15.43:0/1138981353 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5cc40075e0 con 0x7f5cc0003a10 2023-08-09T13:50:35.615 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:35.608+0000 7f5ccaffd640 1 -- 172.21.15.43:0/1138981353 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5cc4007970 con 0x7f5cc0003a10 2023-08-09T13:50:35.615 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:35.608+0000 7f5cd8d5b640 1 -- 172.21.15.43:0/1138981353 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5cd41f2bd0 con 0x7f5cc0003a10 2023-08-09T13:50:35.615 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:35.609+0000 7f5cd8d5b640 1 -- 172.21.15.43:0/1138981353 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5cd415b350 con 0x7f5cc0003a10 2023-08-09T13:50:35.615 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:35.610+0000 7f5cd8d5b640 1 -- 172.21.15.43:0/1138981353 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5cd40b4870 con 0x7f5cc0003a10 2023-08-09T13:50:35.615 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:35.611+0000 7f5ccaffd640 1 -- 172.21.15.43:0/1138981353 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 29) v1 ==== 88443+0+0 (secure 0 0 0) 0x7f5cc4032070 con 0x7f5cc0003a10 2023-08-09T13:50:35.616 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:35.612+0000 7f5ccaffd640 1 --2- 172.21.15.43:0/1138981353 >> [v2:172.21.15.203:6832/3325141227,v1:172.21.15.203:6833/3325141227] conn(0x7f5cb406a170 0x7f5cb406c630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:50:35.616 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:35.612+0000 7f5ccaffd640 1 -- 172.21.15.43:0/1138981353 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(90..90 src has 1..90) v4 ==== 6426+0+0 (secure 0 0 0) 0x7f5cc40a9a60 con 0x7f5cc0003a10 2023-08-09T13:50:35.617 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:35.613+0000 7f5cd1d9b640 1 --2- 172.21.15.43:0/1138981353 >> [v2:172.21.15.203:6832/3325141227,v1:172.21.15.203:6833/3325141227] conn(0x7f5cb406a170 0x7f5cb406c630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:50:35.617 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:35.614+0000 7f5cd1d9b640 1 --2- 172.21.15.43:0/1138981353 >> [v2:172.21.15.203:6832/3325141227,v1:172.21.15.203:6833/3325141227] conn(0x7f5cb406a170 0x7f5cb406c630 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f5cbc007fd0 tx=0x7f5cbc009040 comp rx=0 tx=0).ready entity=mgr.5519 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-09T13:50:35.618 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:35.617+0000 7f5ccaffd640 1 -- 172.21.15.43:0/1138981353 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178066 (secure 0 0 0) 0x7f5cc4074f60 con 0x7f5cc0003a10 2023-08-09T13:50:35.808 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:35.807+0000 7f5cd8d5b640 1 -- 172.21.15.43:0/1138981353 --> [v2:172.21.15.203:6832/3325141227,v1:172.21.15.203:6833/3325141227] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f5cd40cd940 con 0x7f5cb406a170 2023-08-09T13:50:35.832 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:35.830+0000 7f5ccaffd640 1 -- 172.21.15.43:0/1138981353 <== mgr.5519 v2:172.21.15.203:6832/3325141227 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+308019 (secure 0 0 0) 0x7f5cbc0050d0 con 0x7f5cb406a170 2023-08-09T13:50:35.834 INFO:teuthology.orchestra.run.smithi043.stdout: 2023-08-09T13:50:35.837 INFO:teuthology.orchestra.run.smithi043.stderr:dumped all 2023-08-09T13:50:35.839 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:35.837+0000 7f5cd8d5b640 1 -- 172.21.15.43:0/1138981353 >> [v2:172.21.15.203:6832/3325141227,v1:172.21.15.203:6833/3325141227] conn(0x7f5cb406a170 msgr2=0x7f5cb406c630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:50:35.839 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:35.837+0000 7f5cd8d5b640 1 --2- 172.21.15.43:0/1138981353 >> [v2:172.21.15.203:6832/3325141227,v1:172.21.15.203:6833/3325141227] conn(0x7f5cb406a170 0x7f5cb406c630 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f5cbc007fd0 tx=0x7f5cbc009040 comp rx=0 tx=0).stop 2023-08-09T13:50:35.839 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:35.837+0000 7f5cd8d5b640 1 -- 172.21.15.43:0/1138981353 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f5cc0003a10 msgr2=0x7f5cd41f4aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:50:35.840 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:35.837+0000 7f5cd8d5b640 1 --2- 172.21.15.43:0/1138981353 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f5cc0003a10 0x7f5cd41f4aa0 secure :-1 s=READY pgs=1030 cs=0 l=1 rev1=1 crypto rx=0x7f5cc4001e30 tx=0x7f5cc4001f00 comp rx=0 tx=0).stop 2023-08-09T13:50:35.840 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:35.838+0000 7f5cd8d5b640 1 -- 172.21.15.43:0/1138981353 shutdown_connections 2023-08-09T13:50:35.840 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:35.838+0000 7f5cd8d5b640 1 --2- 172.21.15.43:0/1138981353 >> [v2:172.21.15.203:6832/3325141227,v1:172.21.15.203:6833/3325141227] conn(0x7f5cb406a170 0x7f5cb406c630 unknown :-1 s=CLOSED pgs=69 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:50:35.840 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:35.838+0000 7f5cd8d5b640 1 --2- 172.21.15.43:0/1138981353 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f5cd41f2280 0x7f5cd40ada30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:50:35.840 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:35.838+0000 7f5cd8d5b640 1 --2- 172.21.15.43:0/1138981353 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f5cd41f2f80 0x7f5cd41f1d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:50:35.840 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:35.838+0000 7f5cd8d5b640 1 --2- 172.21.15.43:0/1138981353 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f5cc0003a10 0x7f5cd41f4aa0 unknown :-1 s=CLOSED pgs=1030 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:50:35.840 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:35.838+0000 7f5cd8d5b640 1 -- 172.21.15.43:0/1138981353 >> 172.21.15.43:0/1138981353 conn(0x7f5cd40ccd80 msgr2=0x7f5cd40c5db0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-09T13:50:35.841 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:35.838+0000 7f5cd8d5b640 1 -- 172.21.15.43:0/1138981353 shutdown_connections 2023-08-09T13:50:35.841 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:35.838+0000 7f5cd8d5b640 1 -- 172.21.15.43:0/1138981353 wait complete. 2023-08-09T13:50:35.862 INFO:teuthology.orchestra.run.smithi043.stdout:{"pg_ready":true,"pg_map":{"version":29,"stamp":"2023-08-09T13:50:34.232744+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592014,"num_objects":8,"num_object_clones":0,"num_object_copies":16,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":500,"num_read_kb":924,"num_write":248,"num_write_kb":2426,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":170,"ondisk_log_size":170,"up":210,"acting":210,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":97,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":279008,"kb_used_data":9572,"kb_used_omap":13,"kb_used_meta":269298,"kb_avail":754695712,"statfs":{"total":773094113280,"available":772808409088,"internally_reserved":0,"allocated":9801728,"data_stored":5763404,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":14248,"internal_metadata":275761240},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":35,"apply_latency_ms":35,"commit_latency_ns":35000000,"apply_latency_ns":35000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":1646,"num_objects":6,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9,"num_write_kb":10,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.066696"},"pg_stats":[{"pgid":"11.3f","version":"0'0","reported_seq":8,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-08-09T13:50:29.233104+0000","last_change":"2023-08-09T13:50:29.233104+0000","last_active":"2023-08-09T13:50:29.233104+0000","last_peered":"2023-08-09T13:50:29.233104+0000","last_clean":"2023-08-09T13:50:29.233104+0000","last_became_active":"2023-08-09T13:50:29.232795+0000","last_became_peered":"2023-08-09T13:50:29.232795+0000","last_unstale":"2023-08-09T13:50:29.233104+0000","last_undegraded":"2023-08-09T13:50:29.233104+0000","last_fullsized":"2023-08-09T13:50:29.233104+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:50:28.150437+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:50:28.150437+0000","last_clean_scrub_stamp":"2023-08-09T13:50:28.150437+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T17:05:45.752848+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"11.3e","version":"0'0","reported_seq":3,"reported_epoch":89,"state":"creating+activating","last_fresh":"2023-08-09T13:50:29.231349+0000","last_change":"2023-08-09T13:50:29.231349+0000","last_active":"2023-08-09T13:50:28.150437+0000","last_peered":"2023-08-09T13:50:28.150437+0000","last_clean":"2023-08-09T13:50:28.150437+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-09T13:50:29.231349+0000","last_undegraded":"2023-08-09T13:50:29.231349+0000","last_fullsized":"2023-08-09T13:50:29.231349+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:50:28.150437+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:50:28.150437+0000","last_clean_scrub_stamp":"2023-08-09T13:50:28.150437+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T18:13:21.364990+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":["6","4"],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.3d","version":"0'0","reported_seq":9,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-08-09T13:50:29.233399+0000","last_change":"2023-08-09T13:50:29.225958+0000","last_active":"2023-08-09T13:50:29.233399+0000","last_peered":"2023-08-09T13:50:29.233399+0000","last_clean":"2023-08-09T13:50:29.233399+0000","last_became_active":"2023-08-09T13:50:29.225536+0000","last_became_peered":"2023-08-09T13:50:29.225536+0000","last_unstale":"2023-08-09T13:50:29.233399+0000","last_undegraded":"2023-08-09T13:50:29.233399+0000","last_fullsized":"2023-08-09T13:50:29.233399+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:50:28.150437+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:50:28.150437+0000","last_clean_scrub_stamp":"2023-08-09T13:50:28.150437+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T22:25:44.631240+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.3c","version":"0'0","reported_seq":11,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-08-09T13:50:30.207338+0000","last_change":"2023-08-09T13:50:29.257870+0000","last_active":"2023-08-09T13:50:30.207338+0000","last_peered":"2023-08-09T13:50:30.207338+0000","last_clean":"2023-08-09T13:50:30.207338+0000","last_became_active":"2023-08-09T13:50:29.257284+0000","last_became_peered":"2023-08-09T13:50:29.257284+0000","last_unstale":"2023-08-09T13:50:30.207338+0000","last_undegraded":"2023-08-09T13:50:30.207338+0000","last_fullsized":"2023-08-09T13:50:30.207338+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:50:28.150437+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:50:28.150437+0000","last_clean_scrub_stamp":"2023-08-09T13:50:28.150437+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T19:35:43.990628+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.3b","version":"0'0","reported_seq":3,"reported_epoch":89,"state":"creating+activating","last_fresh":"2023-08-09T13:50:29.221854+0000","last_change":"2023-08-09T13:50:29.221854+0000","last_active":"2023-08-09T13:50:28.150437+0000","last_peered":"2023-08-09T13:50:28.150437+0000","last_clean":"2023-08-09T13:50:28.150437+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-09T13:50:29.221854+0000","last_undegraded":"2023-08-09T13:50:29.221854+0000","last_fullsized":"2023-08-09T13:50:29.221854+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:50:28.150437+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:50:28.150437+0000","last_clean_scrub_stamp":"2023-08-09T13:50:28.150437+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T22:54:27.616846+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":["2","7"],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.3a","version":"0'0","reported_seq":8,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-08-09T13:50:29.250572+0000","last_change":"2023-08-09T13:50:29.250572+0000","last_active":"2023-08-09T13:50:29.250572+0000","last_peered":"2023-08-09T13:50:29.250572+0000","last_clean":"2023-08-09T13:50:29.250572+0000","last_became_active":"2023-08-09T13:50:29.250267+0000","last_became_peered":"2023-08-09T13:50:29.250267+0000","last_unstale":"2023-08-09T13:50:29.250572+0000","last_undegraded":"2023-08-09T13:50:29.250572+0000","last_fullsized":"2023-08-09T13:50:29.250572+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:50:28.150437+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:50:28.150437+0000","last_clean_scrub_stamp":"2023-08-09T13:50:28.150437+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T17:21:38.689120+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.39","version":"0'0","reported_seq":3,"reported_epoch":89,"state":"creating+activating","last_fresh":"2023-08-09T13:50:29.220028+0000","last_change":"2023-08-09T13:50:29.220028+0000","last_active":"2023-08-09T13:50:28.150437+0000","last_peered":"2023-08-09T13:50:28.150437+0000","last_clean":"2023-08-09T13:50:28.150437+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-09T13:50:29.220028+0000","last_undegraded":"2023-08-09T13:50:29.220028+0000","last_fullsized":"2023-08-09T13:50:29.220028+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:50:28.150437+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:50:28.150437+0000","last_clean_scrub_stamp":"2023-08-09T13:50:28.150437+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T23:19:57.122069+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":["5","1"],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.38","version":"0'0","reported_seq":8,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-08-09T13:50:29.246308+0000","last_change":"2023-08-09T13:50:29.246308+0000","last_active":"2023-08-09T13:50:29.246308+0000","last_peered":"2023-08-09T13:50:29.246308+0000","last_clean":"2023-08-09T13:50:29.246308+0000","last_became_active":"2023-08-09T13:50:29.245268+0000","last_became_peered":"2023-08-09T13:50:29.245268+0000","last_unstale":"2023-08-09T13:50:29.246308+0000","last_undegraded":"2023-08-09T13:50:29.246308+0000","last_fullsized":"2023-08-09T13:50:29.246308+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:50:28.150437+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:50:28.150437+0000","last_clean_scrub_stamp":"2023-08-09T13:50:28.150437+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T18:15:31.389069+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.37","version":"0'0","reported_seq":3,"reported_epoch":89,"state":"creating+activating","last_fresh":"2023-08-09T13:50:29.223778+0000","last_change":"2023-08-09T13:50:29.223778+0000","last_active":"2023-08-09T13:50:28.150437+0000","last_peered":"2023-08-09T13:50:28.150437+0000","last_clean":"2023-08-09T13:50:28.150437+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-09T13:50:29.223778+0000","last_undegraded":"2023-08-09T13:50:29.223778+0000","last_fullsized":"2023-08-09T13:50:29.223778+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:50:28.150437+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:50:28.150437+0000","last_clean_scrub_stamp":"2023-08-09T13:50:28.150437+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T15:38:50.620059+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":["2","3"],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.36","version":"0'0","reported_seq":3,"reported_epoch":89,"state":"creating+activating","last_fresh":"2023-08-09T13:50:29.207900+0000","last_change":"2023-08-09T13:50:29.207900+0000","last_active":"2023-08-09T13:50:28.150437+0000","last_peered":"2023-08-09T13:50:28.150437+0000","last_clean":"2023-08-09T13:50:28.150437+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-09T13:50:29.207900+0000","last_undegraded":"2023-08-09T13:50:29.207900+0000","last_fullsized":"2023-08-09T13:50:29.207900+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:50:28.150437+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:50:28.150437+0000","last_clean_scrub_stamp":"2023-08-09T13:50:28.150437+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T21:32:28.121459+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":["4","2"],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.35","version":"0'0","reported_seq":8,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-08-09T13:50:29.237075+0000","last_change":"2023-08-09T13:50:29.237075+0000","last_active":"2023-08-09T13:50:29.237075+0000","last_peered":"2023-08-09T13:50:29.237075+0000","last_clean":"2023-08-09T13:50:29.237075+0000","last_became_active":"2023-08-09T13:50:29.236351+0000","last_became_peered":"2023-08-09T13:50:29.236351+0000","last_unstale":"2023-08-09T13:50:29.237075+0000","last_undegraded":"2023-08-09T13:50:29.237075+0000","last_fullsized":"2023-08-09T13:50:29.237075+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:50:28.150437+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:50:28.150437+0000","last_clean_scrub_stamp":"2023-08-09T13:50:28.150437+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T18:17:12.675765+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"11.34","version":"0'0","reported_seq":11,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-08-09T13:50:30.207548+0000","last_change":"2023-08-09T13:50:29.234197+0000","last_active":"2023-08-09T13:50:30.207548+0000","last_peered":"2023-08-09T13:50:30.207548+0000","last_clean":"2023-08-09T13:50:30.207548+0000","last_became_active":"2023-08-09T13:50:29.233877+0000","last_became_peered":"2023-08-09T13:50:29.233877+0000","last_unstale":"2023-08-09T13:50:30.207548+0000","last_undegraded":"2023-08-09T13:50:30.207548+0000","last_fullsized":"2023-08-09T13:50:30.207548+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:50:28.150437+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:50:28.150437+0000","last_clean_scrub_stamp":"2023-08-09T13:50:28.150437+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T21:16:13.484480+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.33","version":"0'0","reported_seq":11,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-08-09T13:50:30.405888+0000","last_change":"2023-08-09T13:50:29.243093+0000","last_active":"2023-08-09T13:50:30.405888+0000","last_peered":"2023-08-09T13:50:30.405888+0000","last_clean":"2023-08-09T13:50:30.405888+0000","last_became_active":"2023-08-09T13:50:29.242740+0000","last_became_peered":"2023-08-09T13:50:29.242740+0000","last_unstale":"2023-08-09T13:50:30.405888+0000","last_undegraded":"2023-08-09T13:50:30.405888+0000","last_fullsized":"2023-08-09T13:50:30.405888+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:50:28.150437+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:50:28.150437+0000","last_clean_scrub_stamp":"2023-08-09T13:50:28.150437+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T15:42:16.315503+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.32","version":"0'0","reported_seq":3,"reported_epoch":89,"state":"creating+activating","last_fresh":"2023-08-09T13:50:29.224087+0000","last_change":"2023-08-09T13:50:29.224087+0000","last_active":"2023-08-09T13:50:28.150437+0000","last_peered":"2023-08-09T13:50:28.150437+0000","last_clean":"2023-08-09T13:50:28.150437+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-09T13:50:29.224087+0000","last_undegraded":"2023-08-09T13:50:29.224087+0000","last_fullsized":"2023-08-09T13:50:29.224087+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:50:28.150437+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:50:28.150437+0000","last_clean_scrub_stamp":"2023-08-09T13:50:28.150437+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T23:57:06.314706+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":["2","5"],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.31","version":"0'0","reported_seq":11,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-08-09T13:50:30.405358+0000","last_change":"2023-08-09T13:50:29.231343+0000","last_active":"2023-08-09T13:50:30.405358+0000","last_peered":"2023-08-09T13:50:30.405358+0000","last_clean":"2023-08-09T13:50:30.405358+0000","last_became_active":"2023-08-09T13:50:29.230980+0000","last_became_peered":"2023-08-09T13:50:29.230980+0000","last_unstale":"2023-08-09T13:50:30.405358+0000","last_undegraded":"2023-08-09T13:50:30.405358+0000","last_fullsized":"2023-08-09T13:50:30.405358+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:50:28.150437+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:50:28.150437+0000","last_clean_scrub_stamp":"2023-08-09T13:50:28.150437+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T23:54:43.168168+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.30","version":"0'0","reported_seq":3,"reported_epoch":89,"state":"creating+activating","last_fresh":"2023-08-09T13:50:29.208580+0000","last_change":"2023-08-09T13:50:29.208580+0000","last_active":"2023-08-09T13:50:28.150437+0000","last_peered":"2023-08-09T13:50:28.150437+0000","last_clean":"2023-08-09T13:50:28.150437+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-09T13:50:29.208580+0000","last_undegraded":"2023-08-09T13:50:29.208580+0000","last_fullsized":"2023-08-09T13:50:29.208580+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:50:28.150437+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:50:28.150437+0000","last_clean_scrub_stamp":"2023-08-09T13:50:28.150437+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T15:14:37.215348+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":["4","6"],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.2f","version":"0'0","reported_seq":3,"reported_epoch":89,"state":"creating+activating","last_fresh":"2023-08-09T13:50:29.223293+0000","last_change":"2023-08-09T13:50:29.223293+0000","last_active":"2023-08-09T13:50:28.150437+0000","last_peered":"2023-08-09T13:50:28.150437+0000","last_clean":"2023-08-09T13:50:28.150437+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-09T13:50:29.223293+0000","last_undegraded":"2023-08-09T13:50:29.223293+0000","last_fullsized":"2023-08-09T13:50:29.223293+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:50:28.150437+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:50:28.150437+0000","last_clean_scrub_stamp":"2023-08-09T13:50:28.150437+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T22:44:32.845197+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":["2","3"],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.2e","version":"0'0","reported_seq":11,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-08-09T13:50:30.208026+0000","last_change":"2023-08-09T13:50:29.244582+0000","last_active":"2023-08-09T13:50:30.208026+0000","last_peered":"2023-08-09T13:50:30.208026+0000","last_clean":"2023-08-09T13:50:30.208026+0000","last_became_active":"2023-08-09T13:50:29.244222+0000","last_became_peered":"2023-08-09T13:50:29.244222+0000","last_unstale":"2023-08-09T13:50:30.208026+0000","last_undegraded":"2023-08-09T13:50:30.208026+0000","last_fullsized":"2023-08-09T13:50:30.208026+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:50:28.150437+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:50:28.150437+0000","last_clean_scrub_stamp":"2023-08-09T13:50:28.150437+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-11T00:06:51.620725+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.2d","version":"0'0","reported_seq":9,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-08-09T13:50:29.248797+0000","last_change":"2023-08-09T13:50:29.241875+0000","last_active":"2023-08-09T13:50:29.248797+0000","last_peered":"2023-08-09T13:50:29.248797+0000","last_clean":"2023-08-09T13:50:29.248797+0000","last_became_active":"2023-08-09T13:50:29.241532+0000","last_became_peered":"2023-08-09T13:50:29.241532+0000","last_unstale":"2023-08-09T13:50:29.248797+0000","last_undegraded":"2023-08-09T13:50:29.248797+0000","last_fullsized":"2023-08-09T13:50:29.248797+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:50:28.150437+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:50:28.150437+0000","last_clean_scrub_stamp":"2023-08-09T13:50:28.150437+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-11T00:33:32.657666+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.2c","version":"0'0","reported_seq":11,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-08-09T13:50:30.406101+0000","last_change":"2023-08-09T13:50:29.231091+0000","last_active":"2023-08-09T13:50:30.406101+0000","last_peered":"2023-08-09T13:50:30.406101+0000","last_clean":"2023-08-09T13:50:30.406101+0000","last_became_active":"2023-08-09T13:50:29.230501+0000","last_became_peered":"2023-08-09T13:50:29.230501+0000","last_unstale":"2023-08-09T13:50:30.406101+0000","last_undegraded":"2023-08-09T13:50:30.406101+0000","last_fullsized":"2023-08-09T13:50:30.406101+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:50:28.150437+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:50:28.150437+0000","last_clean_scrub_stamp":"2023-08-09T13:50:28.150437+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-11T01:19:18.466874+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.2b","version":"0'0","reported_seq":9,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-08-09T13:50:29.233438+0000","last_change":"2023-08-09T13:50:29.227706+0000","last_active":"2023-08-09T13:50:29.233438+0000","last_peered":"2023-08-09T13:50:29.233438+0000","last_clean":"2023-08-09T13:50:29.233438+0000","last_became_active":"2023-08-09T13:50:29.226933+0000","last_became_peered":"2023-08-09T13:50:29.226933+0000","last_unstale":"2023-08-09T13:50:29.233438+0000","last_undegraded":"2023-08-09T13:50:29.233438+0000","last_fullsized":"2023-08-09T13:50:29.233438+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:50:28.150437+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:50:28.150437+0000","last_clean_scrub_stamp":"2023-08-09T13:50:28.150437+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T17:57:56.069305+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.2a","version":"0'0","reported_seq":3,"reported_epoch":89,"state":"creating+activating","last_fresh":"2023-08-09T13:50:29.207312+0000","last_change":"2023-08-09T13:50:29.207312+0000","last_active":"2023-08-09T13:50:28.150437+0000","last_peered":"2023-08-09T13:50:28.150437+0000","last_clean":"2023-08-09T13:50:28.150437+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-09T13:50:29.207312+0000","last_undegraded":"2023-08-09T13:50:29.207312+0000","last_fullsized":"2023-08-09T13:50:29.207312+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:50:28.150437+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:50:28.150437+0000","last_clean_scrub_stamp":"2023-08-09T13:50:28.150437+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T19:03:45.295280+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":["4","2"],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.29","version":"0'0","reported_seq":8,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-08-09T13:50:29.232548+0000","last_change":"2023-08-09T13:50:29.232548+0000","last_active":"2023-08-09T13:50:29.232548+0000","last_peered":"2023-08-09T13:50:29.232548+0000","last_clean":"2023-08-09T13:50:29.232548+0000","last_became_active":"2023-08-09T13:50:29.232205+0000","last_became_peered":"2023-08-09T13:50:29.232205+0000","last_unstale":"2023-08-09T13:50:29.232548+0000","last_undegraded":"2023-08-09T13:50:29.232548+0000","last_fullsized":"2023-08-09T13:50:29.232548+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:50:28.150437+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:50:28.150437+0000","last_clean_scrub_stamp":"2023-08-09T13:50:28.150437+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T21:11:03.639832+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"11.28","version":"0'0","reported_seq":3,"reported_epoch":89,"state":"creating+activating","last_fresh":"2023-08-09T13:50:29.217553+0000","last_change":"2023-08-09T13:50:29.217553+0000","last_active":"2023-08-09T13:50:28.150437+0000","last_peered":"2023-08-09T13:50:28.150437+0000","last_clean":"2023-08-09T13:50:28.150437+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-09T13:50:29.217553+0000","last_undegraded":"2023-08-09T13:50:29.217553+0000","last_fullsized":"2023-08-09T13:50:29.217553+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:50:28.150437+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:50:28.150437+0000","last_clean_scrub_stamp":"2023-08-09T13:50:28.150437+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T22:39:17.713654+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":["7","6"],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"11.27","version":"0'0","reported_seq":3,"reported_epoch":89,"state":"creating+activating","last_fresh":"2023-08-09T13:50:29.208200+0000","last_change":"2023-08-09T13:50:29.208200+0000","last_active":"2023-08-09T13:50:28.150437+0000","last_peered":"2023-08-09T13:50:28.150437+0000","last_clean":"2023-08-09T13:50:28.150437+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-09T13:50:29.208200+0000","last_undegraded":"2023-08-09T13:50:29.208200+0000","last_fullsized":"2023-08-09T13:50:29.208200+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:50:28.150437+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:50:28.150437+0000","last_clean_scrub_stamp":"2023-08-09T13:50:28.150437+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T14:52:59.954591+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":["4","7"],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.26","version":"0'0","reported_seq":3,"reported_epoch":89,"state":"creating+activating","last_fresh":"2023-08-09T13:50:29.223117+0000","last_change":"2023-08-09T13:50:29.223117+0000","last_active":"2023-08-09T13:50:28.150437+0000","last_peered":"2023-08-09T13:50:28.150437+0000","last_clean":"2023-08-09T13:50:28.150437+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-09T13:50:29.223117+0000","last_undegraded":"2023-08-09T13:50:29.223117+0000","last_fullsized":"2023-08-09T13:50:29.223117+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:50:28.150437+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:50:28.150437+0000","last_clean_scrub_stamp":"2023-08-09T13:50:28.150437+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T21:18:31.512056+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":["5","6"],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.25","version":"0'0","reported_seq":8,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-08-09T13:50:29.233554+0000","last_change":"2023-08-09T13:50:29.233554+0000","last_active":"2023-08-09T13:50:29.233554+0000","last_peered":"2023-08-09T13:50:29.233554+0000","last_clean":"2023-08-09T13:50:29.233554+0000","last_became_active":"2023-08-09T13:50:29.233153+0000","last_became_peered":"2023-08-09T13:50:29.233153+0000","last_unstale":"2023-08-09T13:50:29.233554+0000","last_undegraded":"2023-08-09T13:50:29.233554+0000","last_fullsized":"2023-08-09T13:50:29.233554+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:50:28.150437+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:50:28.150437+0000","last_clean_scrub_stamp":"2023-08-09T13:50:28.150437+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T21:36:11.302948+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.24","version":"0'0","reported_seq":3,"reported_epoch":89,"state":"creating+activating","last_fresh":"2023-08-09T13:50:29.207497+0000","last_change":"2023-08-09T13:50:29.207497+0000","last_active":"2023-08-09T13:50:28.150437+0000","last_peered":"2023-08-09T13:50:28.150437+0000","last_clean":"2023-08-09T13:50:28.150437+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-09T13:50:29.207497+0000","last_undegraded":"2023-08-09T13:50:29.207497+0000","last_fullsized":"2023-08-09T13:50:29.207497+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:50:28.150437+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:50:28.150437+0000","last_clean_scrub_stamp":"2023-08-09T13:50:28.150437+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T18:47:18.558930+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":["4","2"],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.23","version":"0'0","reported_seq":8,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-08-09T13:50:29.233513+0000","last_change":"2023-08-09T13:50:29.233513+0000","last_active":"2023-08-09T13:50:29.233513+0000","last_peered":"2023-08-09T13:50:29.233513+0000","last_clean":"2023-08-09T13:50:29.233513+0000","last_became_active":"2023-08-09T13:50:29.233199+0000","last_became_peered":"2023-08-09T13:50:29.233199+0000","last_unstale":"2023-08-09T13:50:29.233513+0000","last_undegraded":"2023-08-09T13:50:29.233513+0000","last_fullsized":"2023-08-09T13:50:29.233513+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:50:28.150437+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:50:28.150437+0000","last_clean_scrub_stamp":"2023-08-09T13:50:28.150437+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T21:47:07.987840+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.22","version":"0'0","reported_seq":3,"reported_epoch":89,"state":"creating+activating","last_fresh":"2023-08-09T13:50:29.219724+0000","last_change":"2023-08-09T13:50:29.219724+0000","last_active":"2023-08-09T13:50:28.150437+0000","last_peered":"2023-08-09T13:50:28.150437+0000","last_clean":"2023-08-09T13:50:28.150437+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-09T13:50:29.219724+0000","last_undegraded":"2023-08-09T13:50:29.219724+0000","last_fullsized":"2023-08-09T13:50:29.219724+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:50:28.150437+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:50:28.150437+0000","last_clean_scrub_stamp":"2023-08-09T13:50:28.150437+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T21:52:25.852199+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":["5","6"],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.21","version":"0'0","reported_seq":8,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-08-09T13:50:29.233755+0000","last_change":"2023-08-09T13:50:29.233755+0000","last_active":"2023-08-09T13:50:29.233755+0000","last_peered":"2023-08-09T13:50:29.233755+0000","last_clean":"2023-08-09T13:50:29.233755+0000","last_became_active":"2023-08-09T13:50:29.233150+0000","last_became_peered":"2023-08-09T13:50:29.233150+0000","last_unstale":"2023-08-09T13:50:29.233755+0000","last_undegraded":"2023-08-09T13:50:29.233755+0000","last_fullsized":"2023-08-09T13:50:29.233755+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:50:28.150437+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:50:28.150437+0000","last_clean_scrub_stamp":"2023-08-09T13:50:28.150437+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T16:18:45.331895+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.20","version":"0'0","reported_seq":8,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-08-09T13:50:29.245742+0000","last_change":"2023-08-09T13:50:29.245742+0000","last_active":"2023-08-09T13:50:29.245742+0000","last_peered":"2023-08-09T13:50:29.245742+0000","last_clean":"2023-08-09T13:50:29.245742+0000","last_became_active":"2023-08-09T13:50:29.242577+0000","last_became_peered":"2023-08-09T13:50:29.242577+0000","last_unstale":"2023-08-09T13:50:29.245742+0000","last_undegraded":"2023-08-09T13:50:29.245742+0000","last_fullsized":"2023-08-09T13:50:29.245742+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:50:28.150437+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:50:28.150437+0000","last_clean_scrub_stamp":"2023-08-09T13:50:28.150437+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T16:18:45.331895+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.1e","version":"0'0","reported_seq":3,"reported_epoch":89,"state":"creating+activating","last_fresh":"2023-08-09T13:50:29.220103+0000","last_change":"2023-08-09T13:50:29.220103+0000","last_active":"2023-08-09T13:50:28.150437+0000","last_peered":"2023-08-09T13:50:28.150437+0000","last_clean":"2023-08-09T13:50:28.150437+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-09T13:50:29.220103+0000","last_undegraded":"2023-08-09T13:50:29.220103+0000","last_fullsized":"2023-08-09T13:50:29.220103+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:50:28.150437+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:50:28.150437+0000","last_clean_scrub_stamp":"2023-08-09T13:50:28.150437+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T19:44:56.828003+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":["7","6"],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.1f","version":"0'0","reported_seq":11,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-08-09T13:50:29.221681+0000","last_change":"2023-08-09T13:50:28.198750+0000","last_active":"2023-08-09T13:50:29.221681+0000","last_peered":"2023-08-09T13:50:29.221681+0000","last_clean":"2023-08-09T13:50:29.221681+0000","last_became_active":"2023-08-09T13:50:28.198375+0000","last_became_peered":"2023-08-09T13:50:28.198375+0000","last_unstale":"2023-08-09T13:50:29.221681+0000","last_undegraded":"2023-08-09T13:50:29.221681+0000","last_fullsized":"2023-08-09T13:50:29.221681+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:50:27.140775+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:50:27.140775+0000","last_clean_scrub_stamp":"2023-08-09T13:50:27.140775+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T23:54:42.158506+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"11.1f","version":"0'0","reported_seq":3,"reported_epoch":89,"state":"creating+activating","last_fresh":"2023-08-09T13:50:29.223179+0000","last_change":"2023-08-09T13:50:29.223179+0000","last_active":"2023-08-09T13:50:28.150437+0000","last_peered":"2023-08-09T13:50:28.150437+0000","last_clean":"2023-08-09T13:50:28.150437+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-09T13:50:29.223179+0000","last_undegraded":"2023-08-09T13:50:29.223179+0000","last_fullsized":"2023-08-09T13:50:29.223179+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:50:28.150437+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:50:28.150437+0000","last_clean_scrub_stamp":"2023-08-09T13:50:28.150437+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T20:43:24.475264+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":["2","3"],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.1e","version":"89'1","reported_seq":14,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-08-09T13:50:30.407030+0000","last_change":"2023-08-09T13:50:28.189590+0000","last_active":"2023-08-09T13:50:30.407030+0000","last_peered":"2023-08-09T13:50:30.407030+0000","last_clean":"2023-08-09T13:50:30.407030+0000","last_became_active":"2023-08-09T13:50:28.189227+0000","last_became_peered":"2023-08-09T13:50:28.189227+0000","last_unstale":"2023-08-09T13:50:30.407030+0000","last_undegraded":"2023-08-09T13:50:30.407030+0000","last_fullsized":"2023-08-09T13:50:30.407030+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:50:27.140775+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:50:27.140775+0000","last_clean_scrub_stamp":"2023-08-09T13:50:27.140775+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T18:15:27.464113+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.1c","version":"0'0","reported_seq":4,"reported_epoch":89,"state":"creating+activating","last_fresh":"2023-08-09T13:50:29.250266+0000","last_change":"2023-08-09T13:50:29.225957+0000","last_active":"2023-08-09T13:50:28.150437+0000","last_peered":"2023-08-09T13:50:28.150437+0000","last_clean":"2023-08-09T13:50:28.150437+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-09T13:50:29.250266+0000","last_undegraded":"2023-08-09T13:50:29.250266+0000","last_fullsized":"2023-08-09T13:50:29.250266+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:50:28.150437+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:50:28.150437+0000","last_clean_scrub_stamp":"2023-08-09T13:50:28.150437+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T21:57:09.347886+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":["6","7"],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"10.1d","version":"0'0","reported_seq":11,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-08-09T13:50:29.210839+0000","last_change":"2023-08-09T13:50:28.206216+0000","last_active":"2023-08-09T13:50:29.210839+0000","last_peered":"2023-08-09T13:50:29.210839+0000","last_clean":"2023-08-09T13:50:29.210839+0000","last_became_active":"2023-08-09T13:50:28.205914+0000","last_became_peered":"2023-08-09T13:50:28.205914+0000","last_unstale":"2023-08-09T13:50:29.210839+0000","last_undegraded":"2023-08-09T13:50:29.210839+0000","last_fullsized":"2023-08-09T13:50:29.210839+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:50:27.140775+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:50:27.140775+0000","last_clean_scrub_stamp":"2023-08-09T13:50:27.140775+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T16:26:26.712563+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.1d","version":"0'0","reported_seq":8,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-08-09T13:50:29.231490+0000","last_change":"2023-08-09T13:50:29.231490+0000","last_active":"2023-08-09T13:50:29.231490+0000","last_peered":"2023-08-09T13:50:29.231490+0000","last_clean":"2023-08-09T13:50:29.231490+0000","last_became_active":"2023-08-09T13:50:29.231190+0000","last_became_peered":"2023-08-09T13:50:29.231190+0000","last_unstale":"2023-08-09T13:50:29.231490+0000","last_undegraded":"2023-08-09T13:50:29.231490+0000","last_fullsized":"2023-08-09T13:50:29.231490+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:50:28.150437+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:50:28.150437+0000","last_clean_scrub_stamp":"2023-08-09T13:50:28.150437+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T14:46:45.632501+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.1c","version":"0'0","reported_seq":11,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-08-09T13:50:29.235770+0000","last_change":"2023-08-09T13:50:28.195729+0000","last_active":"2023-08-09T13:50:29.235770+0000","last_peered":"2023-08-09T13:50:29.235770+0000","last_clean":"2023-08-09T13:50:29.235770+0000","last_became_active":"2023-08-09T13:50:28.195433+0000","last_became_peered":"2023-08-09T13:50:28.195433+0000","last_unstale":"2023-08-09T13:50:29.235770+0000","last_undegraded":"2023-08-09T13:50:29.235770+0000","last_fullsized":"2023-08-09T13:50:29.235770+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:50:27.140775+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:50:27.140775+0000","last_clean_scrub_stamp":"2023-08-09T13:50:27.140775+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T16:52:42.838615+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.1a","version":"0'0","reported_seq":8,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-08-09T13:50:29.234000+0000","last_change":"2023-08-09T13:50:29.234000+0000","last_active":"2023-08-09T13:50:29.234000+0000","last_peered":"2023-08-09T13:50:29.234000+0000","last_clean":"2023-08-09T13:50:29.234000+0000","last_became_active":"2023-08-09T13:50:29.233727+0000","last_became_peered":"2023-08-09T13:50:29.233727+0000","last_unstale":"2023-08-09T13:50:29.234000+0000","last_undegraded":"2023-08-09T13:50:29.234000+0000","last_fullsized":"2023-08-09T13:50:29.234000+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:50:28.150437+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:50:28.150437+0000","last_clean_scrub_stamp":"2023-08-09T13:50:28.150437+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T20:21:45.115966+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.1b","version":"0'0","reported_seq":11,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-08-09T13:50:29.214221+0000","last_change":"2023-08-09T13:50:28.196698+0000","last_active":"2023-08-09T13:50:29.214221+0000","last_peered":"2023-08-09T13:50:29.214221+0000","last_clean":"2023-08-09T13:50:29.214221+0000","last_became_active":"2023-08-09T13:50:28.196362+0000","last_became_peered":"2023-08-09T13:50:28.196362+0000","last_unstale":"2023-08-09T13:50:29.214221+0000","last_undegraded":"2023-08-09T13:50:29.214221+0000","last_fullsized":"2023-08-09T13:50:29.214221+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:50:27.140775+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:50:27.140775+0000","last_clean_scrub_stamp":"2023-08-09T13:50:27.140775+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T16:26:00.621455+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.1b","version":"0'0","reported_seq":3,"reported_epoch":89,"state":"creating+activating","last_fresh":"2023-08-09T13:50:29.208495+0000","last_change":"2023-08-09T13:50:29.208495+0000","last_active":"2023-08-09T13:50:28.150437+0000","last_peered":"2023-08-09T13:50:28.150437+0000","last_clean":"2023-08-09T13:50:28.150437+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-09T13:50:29.208495+0000","last_undegraded":"2023-08-09T13:50:29.208495+0000","last_fullsized":"2023-08-09T13:50:29.208495+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:50:28.150437+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:50:28.150437+0000","last_clean_scrub_stamp":"2023-08-09T13:50:28.150437+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T16:20:35.941253+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":["4","7"],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"10.1a","version":"0'0","reported_seq":11,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-08-09T13:50:29.228557+0000","last_change":"2023-08-09T13:50:28.205333+0000","last_active":"2023-08-09T13:50:29.228557+0000","last_peered":"2023-08-09T13:50:29.228557+0000","last_clean":"2023-08-09T13:50:29.228557+0000","last_became_active":"2023-08-09T13:50:28.204881+0000","last_became_peered":"2023-08-09T13:50:28.204881+0000","last_unstale":"2023-08-09T13:50:29.228557+0000","last_undegraded":"2023-08-09T13:50:29.228557+0000","last_fullsized":"2023-08-09T13:50:29.228557+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:50:27.140775+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:50:27.140775+0000","last_clean_scrub_stamp":"2023-08-09T13:50:27.140775+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T18:06:20.762713+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.18","version":"0'0","reported_seq":3,"reported_epoch":89,"state":"creating+activating","last_fresh":"2023-08-09T13:50:29.218450+0000","last_change":"2023-08-09T13:50:29.218450+0000","last_active":"2023-08-09T13:50:28.150437+0000","last_peered":"2023-08-09T13:50:28.150437+0000","last_clean":"2023-08-09T13:50:28.150437+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-09T13:50:29.218450+0000","last_undegraded":"2023-08-09T13:50:29.218450+0000","last_fullsized":"2023-08-09T13:50:29.218450+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:50:28.150437+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:50:28.150437+0000","last_clean_scrub_stamp":"2023-08-09T13:50:28.150437+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T15:00:53.649507+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":["2","4"],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.19","version":"0'0","reported_seq":13,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-08-09T13:50:30.207222+0000","last_change":"2023-08-09T13:50:28.204197+0000","last_active":"2023-08-09T13:50:30.207222+0000","last_peered":"2023-08-09T13:50:30.207222+0000","last_clean":"2023-08-09T13:50:30.207222+0000","last_became_active":"2023-08-09T13:50:28.203911+0000","last_became_peered":"2023-08-09T13:50:28.203911+0000","last_unstale":"2023-08-09T13:50:30.207222+0000","last_undegraded":"2023-08-09T13:50:30.207222+0000","last_fullsized":"2023-08-09T13:50:30.207222+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:50:27.140775+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:50:27.140775+0000","last_clean_scrub_stamp":"2023-08-09T13:50:27.140775+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T19:31:34.788085+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.19","version":"0'0","reported_seq":11,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-08-09T13:50:30.405358+0000","last_change":"2023-08-09T13:50:29.223620+0000","last_active":"2023-08-09T13:50:30.405358+0000","last_peered":"2023-08-09T13:50:30.405358+0000","last_clean":"2023-08-09T13:50:30.405358+0000","last_became_active":"2023-08-09T13:50:29.223063+0000","last_became_peered":"2023-08-09T13:50:29.223063+0000","last_unstale":"2023-08-09T13:50:30.405358+0000","last_undegraded":"2023-08-09T13:50:30.405358+0000","last_fullsized":"2023-08-09T13:50:30.405358+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:50:28.150437+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:50:28.150437+0000","last_clean_scrub_stamp":"2023-08-09T13:50:28.150437+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T16:41:09.645304+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.18","version":"0'0","reported_seq":11,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-08-09T13:50:29.224563+0000","last_change":"2023-08-09T13:50:28.189872+0000","last_active":"2023-08-09T13:50:29.224563+0000","last_peered":"2023-08-09T13:50:29.224563+0000","last_clean":"2023-08-09T13:50:29.224563+0000","last_became_active":"2023-08-09T13:50:28.189560+0000","last_became_peered":"2023-08-09T13:50:28.189560+0000","last_unstale":"2023-08-09T13:50:29.224563+0000","last_undegraded":"2023-08-09T13:50:29.224563+0000","last_fullsized":"2023-08-09T13:50:29.224563+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:50:27.140775+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:50:27.140775+0000","last_clean_scrub_stamp":"2023-08-09T13:50:27.140775+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T18:02:42.591074+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.16","version":"0'0","reported_seq":3,"reported_epoch":89,"state":"creating+activating","last_fresh":"2023-08-09T13:50:29.223029+0000","last_change":"2023-08-09T13:50:29.223029+0000","last_active":"2023-08-09T13:50:28.150437+0000","last_peered":"2023-08-09T13:50:28.150437+0000","last_clean":"2023-08-09T13:50:28.150437+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-09T13:50:29.223029+0000","last_undegraded":"2023-08-09T13:50:29.223029+0000","last_fullsized":"2023-08-09T13:50:29.223029+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:50:28.150437+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:50:28.150437+0000","last_clean_scrub_stamp":"2023-08-09T13:50:28.150437+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T18:35:45.000310+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":["5","7"],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.17","version":"0'0","reported_seq":11,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-08-09T13:50:29.210879+0000","last_change":"2023-08-09T13:50:28.198990+0000","last_active":"2023-08-09T13:50:29.210879+0000","last_peered":"2023-08-09T13:50:29.210879+0000","last_clean":"2023-08-09T13:50:29.210879+0000","last_became_active":"2023-08-09T13:50:28.197376+0000","last_became_peered":"2023-08-09T13:50:28.197376+0000","last_unstale":"2023-08-09T13:50:29.210879+0000","last_undegraded":"2023-08-09T13:50:29.210879+0000","last_fullsized":"2023-08-09T13:50:29.210879+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:50:27.140775+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:50:27.140775+0000","last_clean_scrub_stamp":"2023-08-09T13:50:27.140775+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T18:00:13.480919+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.17","version":"0'0","reported_seq":3,"reported_epoch":89,"state":"creating+activating","last_fresh":"2023-08-09T13:50:29.221024+0000","last_change":"2023-08-09T13:50:29.221024+0000","last_active":"2023-08-09T13:50:28.150437+0000","last_peered":"2023-08-09T13:50:28.150437+0000","last_clean":"2023-08-09T13:50:28.150437+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-09T13:50:29.221024+0000","last_undegraded":"2023-08-09T13:50:29.221024+0000","last_fullsized":"2023-08-09T13:50:29.221024+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:50:28.150437+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:50:28.150437+0000","last_clean_scrub_stamp":"2023-08-09T13:50:28.150437+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T19:41:45.324636+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":["7","6"],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.16","version":"0'0","reported_seq":11,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-08-09T13:50:29.236992+0000","last_change":"2023-08-09T13:50:28.203893+0000","last_active":"2023-08-09T13:50:29.236992+0000","last_peered":"2023-08-09T13:50:29.236992+0000","last_clean":"2023-08-09T13:50:29.236992+0000","last_became_active":"2023-08-09T13:50:28.203596+0000","last_became_peered":"2023-08-09T13:50:28.203596+0000","last_unstale":"2023-08-09T13:50:29.236992+0000","last_undegraded":"2023-08-09T13:50:29.236992+0000","last_fullsized":"2023-08-09T13:50:29.236992+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:50:27.140775+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:50:27.140775+0000","last_clean_scrub_stamp":"2023-08-09T13:50:27.140775+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-11T01:07:24.154763+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.14","version":"0'0","reported_seq":8,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-08-09T13:50:29.236992+0000","last_change":"2023-08-09T13:50:29.236992+0000","last_active":"2023-08-09T13:50:29.236992+0000","last_peered":"2023-08-09T13:50:29.236992+0000","last_clean":"2023-08-09T13:50:29.236992+0000","last_became_active":"2023-08-09T13:50:29.236708+0000","last_became_peered":"2023-08-09T13:50:29.236708+0000","last_unstale":"2023-08-09T13:50:29.236992+0000","last_undegraded":"2023-08-09T13:50:29.236992+0000","last_fullsized":"2023-08-09T13:50:29.236992+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:50:28.150437+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:50:28.150437+0000","last_clean_scrub_stamp":"2023-08-09T13:50:28.150437+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T23:19:57.122069+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"10.15","version":"0'0","reported_seq":11,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-08-09T13:50:29.215134+0000","last_change":"2023-08-09T13:50:28.198818+0000","last_active":"2023-08-09T13:50:29.215134+0000","last_peered":"2023-08-09T13:50:29.215134+0000","last_clean":"2023-08-09T13:50:29.215134+0000","last_became_active":"2023-08-09T13:50:28.198282+0000","last_became_peered":"2023-08-09T13:50:28.198282+0000","last_unstale":"2023-08-09T13:50:29.215134+0000","last_undegraded":"2023-08-09T13:50:29.215134+0000","last_fullsized":"2023-08-09T13:50:29.215134+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:50:27.140775+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:50:27.140775+0000","last_clean_scrub_stamp":"2023-08-09T13:50:27.140775+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T13:56:48.165182+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.15","version":"0'0","reported_seq":8,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-08-09T13:50:29.232045+0000","last_change":"2023-08-09T13:50:29.232045+0000","last_active":"2023-08-09T13:50:29.232045+0000","last_peered":"2023-08-09T13:50:29.232045+0000","last_clean":"2023-08-09T13:50:29.232045+0000","last_became_active":"2023-08-09T13:50:29.231691+0000","last_became_peered":"2023-08-09T13:50:29.231691+0000","last_unstale":"2023-08-09T13:50:29.232045+0000","last_undegraded":"2023-08-09T13:50:29.232045+0000","last_fullsized":"2023-08-09T13:50:29.232045+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:50:28.150437+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:50:28.150437+0000","last_clean_scrub_stamp":"2023-08-09T13:50:28.150437+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T14:41:33.537815+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.14","version":"89'6","reported_seq":19,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-08-09T13:50:30.207724+0000","last_change":"2023-08-09T13:50:28.194890+0000","last_active":"2023-08-09T13:50:30.207724+0000","last_peered":"2023-08-09T13:50:30.207724+0000","last_clean":"2023-08-09T13:50:30.207724+0000","last_became_active":"2023-08-09T13:50:28.194607+0000","last_became_peered":"2023-08-09T13:50:28.194607+0000","last_unstale":"2023-08-09T13:50:30.207724+0000","last_undegraded":"2023-08-09T13:50:30.207724+0000","last_fullsized":"2023-08-09T13:50:30.207724+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:50:27.140775+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:50:27.140775+0000","last_clean_scrub_stamp":"2023-08-09T13:50:27.140775+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T19:14:35.323306+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.12","version":"0'0","reported_seq":8,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-08-09T13:50:29.233976+0000","last_change":"2023-08-09T13:50:29.233976+0000","last_active":"2023-08-09T13:50:29.233976+0000","last_peered":"2023-08-09T13:50:29.233976+0000","last_clean":"2023-08-09T13:50:29.233976+0000","last_became_active":"2023-08-09T13:50:29.233696+0000","last_became_peered":"2023-08-09T13:50:29.233696+0000","last_unstale":"2023-08-09T13:50:29.233976+0000","last_undegraded":"2023-08-09T13:50:29.233976+0000","last_fullsized":"2023-08-09T13:50:29.233976+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:50:28.150437+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:50:28.150437+0000","last_clean_scrub_stamp":"2023-08-09T13:50:28.150437+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T15:32:41.261189+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.13","version":"89'2","reported_seq":15,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-08-09T13:50:30.207465+0000","last_change":"2023-08-09T13:50:28.198586+0000","last_active":"2023-08-09T13:50:30.207465+0000","last_peered":"2023-08-09T13:50:30.207465+0000","last_clean":"2023-08-09T13:50:30.207465+0000","last_became_active":"2023-08-09T13:50:28.198247+0000","last_became_peered":"2023-08-09T13:50:28.198247+0000","last_unstale":"2023-08-09T13:50:30.207465+0000","last_undegraded":"2023-08-09T13:50:30.207465+0000","last_fullsized":"2023-08-09T13:50:30.207465+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:50:27.140775+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:50:27.140775+0000","last_clean_scrub_stamp":"2023-08-09T13:50:27.140775+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T21:31:44.778069+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.13","version":"0'0","reported_seq":3,"reported_epoch":89,"state":"creating+activating","last_fresh":"2023-08-09T13:50:29.220908+0000","last_change":"2023-08-09T13:50:29.220908+0000","last_active":"2023-08-09T13:50:28.150437+0000","last_peered":"2023-08-09T13:50:28.150437+0000","last_clean":"2023-08-09T13:50:28.150437+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-09T13:50:29.220908+0000","last_undegraded":"2023-08-09T13:50:29.220908+0000","last_fullsized":"2023-08-09T13:50:29.220908+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:50:28.150437+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:50:28.150437+0000","last_clean_scrub_stamp":"2023-08-09T13:50:28.150437+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T18:15:31.389069+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":["5","4"],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.12","version":"0'0","reported_seq":11,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-08-09T13:50:29.235230+0000","last_change":"2023-08-09T13:50:28.201822+0000","last_active":"2023-08-09T13:50:29.235230+0000","last_peered":"2023-08-09T13:50:29.235230+0000","last_clean":"2023-08-09T13:50:29.235230+0000","last_became_active":"2023-08-09T13:50:28.201187+0000","last_became_peered":"2023-08-09T13:50:28.201187+0000","last_unstale":"2023-08-09T13:50:29.235230+0000","last_undegraded":"2023-08-09T13:50:29.235230+0000","last_fullsized":"2023-08-09T13:50:29.235230+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:50:27.140775+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:50:27.140775+0000","last_clean_scrub_stamp":"2023-08-09T13:50:27.140775+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T23:52:03.500625+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.10","version":"0'0","reported_seq":11,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-08-09T13:50:30.405345+0000","last_change":"2023-08-09T13:50:29.246513+0000","last_active":"2023-08-09T13:50:30.405345+0000","last_peered":"2023-08-09T13:50:30.405345+0000","last_clean":"2023-08-09T13:50:30.405345+0000","last_became_active":"2023-08-09T13:50:29.246176+0000","last_became_peered":"2023-08-09T13:50:29.246176+0000","last_unstale":"2023-08-09T13:50:30.405345+0000","last_undegraded":"2023-08-09T13:50:30.405345+0000","last_fullsized":"2023-08-09T13:50:30.405345+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:50:28.150437+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:50:28.150437+0000","last_clean_scrub_stamp":"2023-08-09T13:50:28.150437+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T19:41:45.324636+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.11","version":"0'0","reported_seq":11,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-08-09T13:50:29.221870+0000","last_change":"2023-08-09T13:50:28.193484+0000","last_active":"2023-08-09T13:50:29.221870+0000","last_peered":"2023-08-09T13:50:29.221870+0000","last_clean":"2023-08-09T13:50:29.221870+0000","last_became_active":"2023-08-09T13:50:28.193098+0000","last_became_peered":"2023-08-09T13:50:28.193098+0000","last_unstale":"2023-08-09T13:50:29.221870+0000","last_undegraded":"2023-08-09T13:50:29.221870+0000","last_fullsized":"2023-08-09T13:50:29.221870+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:50:27.140775+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:50:27.140775+0000","last_clean_scrub_stamp":"2023-08-09T13:50:27.140775+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T21:57:08.338223+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.11","version":"0'0","reported_seq":3,"reported_epoch":89,"state":"creating+activating","last_fresh":"2023-08-09T13:50:29.224448+0000","last_change":"2023-08-09T13:50:29.224448+0000","last_active":"2023-08-09T13:50:28.150437+0000","last_peered":"2023-08-09T13:50:28.150437+0000","last_clean":"2023-08-09T13:50:28.150437+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-09T13:50:29.224448+0000","last_undegraded":"2023-08-09T13:50:29.224448+0000","last_fullsized":"2023-08-09T13:50:29.224448+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:50:28.150437+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:50:28.150437+0000","last_clean_scrub_stamp":"2023-08-09T13:50:28.150437+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T19:49:56.370072+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":["2","3"],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.10","version":"0'0","reported_seq":11,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-08-09T13:50:29.209343+0000","last_change":"2023-08-09T13:50:28.200670+0000","last_active":"2023-08-09T13:50:29.209343+0000","last_peered":"2023-08-09T13:50:29.209343+0000","last_clean":"2023-08-09T13:50:29.209343+0000","last_became_active":"2023-08-09T13:50:28.200419+0000","last_became_peered":"2023-08-09T13:50:28.200419+0000","last_unstale":"2023-08-09T13:50:29.209343+0000","last_undegraded":"2023-08-09T13:50:29.209343+0000","last_fullsized":"2023-08-09T13:50:29.209343+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:50:27.140775+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:50:27.140775+0000","last_clean_scrub_stamp":"2023-08-09T13:50:27.140775+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T15:23:31.632434+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.6","version":"0'0","reported_seq":11,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-08-09T13:50:30.406773+0000","last_change":"2023-08-09T13:50:29.229974+0000","last_active":"2023-08-09T13:50:30.406773+0000","last_peered":"2023-08-09T13:50:30.406773+0000","last_clean":"2023-08-09T13:50:30.406773+0000","last_became_active":"2023-08-09T13:50:29.229589+0000","last_became_peered":"2023-08-09T13:50:29.229589+0000","last_unstale":"2023-08-09T13:50:30.406773+0000","last_undegraded":"2023-08-09T13:50:30.406773+0000","last_fullsized":"2023-08-09T13:50:30.406773+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:50:28.150437+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:50:28.150437+0000","last_clean_scrub_stamp":"2023-08-09T13:50:28.150437+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T23:11:20.894653+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.7","version":"0'0","reported_seq":11,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-08-09T13:50:29.230633+0000","last_change":"2023-08-09T13:50:28.204446+0000","last_active":"2023-08-09T13:50:29.230633+0000","last_peered":"2023-08-09T13:50:29.230633+0000","last_clean":"2023-08-09T13:50:29.230633+0000","last_became_active":"2023-08-09T13:50:28.204093+0000","last_became_peered":"2023-08-09T13:50:28.204093+0000","last_unstale":"2023-08-09T13:50:29.230633+0000","last_undegraded":"2023-08-09T13:50:29.230633+0000","last_fullsized":"2023-08-09T13:50:29.230633+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:50:27.140775+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:50:27.140775+0000","last_clean_scrub_stamp":"2023-08-09T13:50:27.140775+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T22:56:15.821115+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.7","version":"0'0","reported_seq":3,"reported_epoch":89,"state":"creating+activating","last_fresh":"2023-08-09T13:50:29.223626+0000","last_change":"2023-08-09T13:50:29.223626+0000","last_active":"2023-08-09T13:50:28.150437+0000","last_peered":"2023-08-09T13:50:28.150437+0000","last_clean":"2023-08-09T13:50:28.150437+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-09T13:50:29.223626+0000","last_undegraded":"2023-08-09T13:50:29.223626+0000","last_fullsized":"2023-08-09T13:50:29.223626+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:50:28.150437+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:50:28.150437+0000","last_clean_scrub_stamp":"2023-08-09T13:50:28.150437+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T18:06:21.772376+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":["5","1"],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.6","version":"0'0","reported_seq":11,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-08-09T13:50:29.223926+0000","last_change":"2023-08-09T13:50:28.191955+0000","last_active":"2023-08-09T13:50:29.223926+0000","last_peered":"2023-08-09T13:50:29.223926+0000","last_clean":"2023-08-09T13:50:29.223926+0000","last_became_active":"2023-08-09T13:50:28.191264+0000","last_became_peered":"2023-08-09T13:50:28.191264+0000","last_unstale":"2023-08-09T13:50:29.223926+0000","last_undegraded":"2023-08-09T13:50:29.223926+0000","last_fullsized":"2023-08-09T13:50:29.223926+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:50:27.140775+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:50:27.140775+0000","last_clean_scrub_stamp":"2023-08-09T13:50:27.140775+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T18:00:04.982575+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.4","version":"0'0","reported_seq":3,"reported_epoch":89,"state":"creating+activating","last_fresh":"2023-08-09T13:50:29.216528+0000","last_change":"2023-08-09T13:50:29.216528+0000","last_active":"2023-08-09T13:50:28.150437+0000","last_peered":"2023-08-09T13:50:28.150437+0000","last_clean":"2023-08-09T13:50:28.150437+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-09T13:50:29.216528+0000","last_undegraded":"2023-08-09T13:50:29.216528+0000","last_fullsized":"2023-08-09T13:50:29.216528+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:50:28.150437+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:50:28.150437+0000","last_clean_scrub_stamp":"2023-08-09T13:50:28.150437+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T21:36:45.909480+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":["7","6"],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.5","version":"0'0","reported_seq":11,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-08-09T13:50:29.224819+0000","last_change":"2023-08-09T13:50:28.193912+0000","last_active":"2023-08-09T13:50:29.224819+0000","last_peered":"2023-08-09T13:50:29.224819+0000","last_clean":"2023-08-09T13:50:29.224819+0000","last_became_active":"2023-08-09T13:50:28.193602+0000","last_became_peered":"2023-08-09T13:50:28.193602+0000","last_unstale":"2023-08-09T13:50:29.224819+0000","last_undegraded":"2023-08-09T13:50:29.224819+0000","last_fullsized":"2023-08-09T13:50:29.224819+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:50:27.140775+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:50:27.140775+0000","last_clean_scrub_stamp":"2023-08-09T13:50:27.140775+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T19:25:18.625389+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.5","version":"0'0","reported_seq":8,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-08-09T13:50:29.237250+0000","last_change":"2023-08-09T13:50:29.237250+0000","last_active":"2023-08-09T13:50:29.237250+0000","last_peered":"2023-08-09T13:50:29.237250+0000","last_clean":"2023-08-09T13:50:29.237250+0000","last_became_active":"2023-08-09T13:50:29.236821+0000","last_became_peered":"2023-08-09T13:50:29.236821+0000","last_unstale":"2023-08-09T13:50:29.237250+0000","last_undegraded":"2023-08-09T13:50:29.237250+0000","last_fullsized":"2023-08-09T13:50:29.237250+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:50:28.150437+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:50:28.150437+0000","last_clean_scrub_stamp":"2023-08-09T13:50:28.150437+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T15:02:09.473638+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.4","version":"0'0","reported_seq":13,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-08-09T13:50:30.406255+0000","last_change":"2023-08-09T13:50:28.197934+0000","last_active":"2023-08-09T13:50:30.406255+0000","last_peered":"2023-08-09T13:50:30.406255+0000","last_clean":"2023-08-09T13:50:30.406255+0000","last_became_active":"2023-08-09T13:50:28.197639+0000","last_became_peered":"2023-08-09T13:50:28.197639+0000","last_unstale":"2023-08-09T13:50:30.406255+0000","last_undegraded":"2023-08-09T13:50:30.406255+0000","last_fullsized":"2023-08-09T13:50:30.406255+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:50:27.140775+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:50:27.140775+0000","last_clean_scrub_stamp":"2023-08-09T13:50:27.140775+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T23:10:53.423417+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.2","version":"0'0","reported_seq":3,"reported_epoch":89,"state":"creating+activating","last_fresh":"2023-08-09T13:50:29.217549+0000","last_change":"2023-08-09T13:50:29.217549+0000","last_active":"2023-08-09T13:50:28.150437+0000","last_peered":"2023-08-09T13:50:28.150437+0000","last_clean":"2023-08-09T13:50:28.150437+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-09T13:50:29.217549+0000","last_undegraded":"2023-08-09T13:50:29.217549+0000","last_fullsized":"2023-08-09T13:50:29.217549+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:50:28.150437+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:50:28.150437+0000","last_clean_scrub_stamp":"2023-08-09T13:50:28.150437+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-11T00:03:22.851529+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":["2","3"],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.3","version":"0'0","reported_seq":11,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-08-09T13:50:29.224120+0000","last_change":"2023-08-09T13:50:28.189872+0000","last_active":"2023-08-09T13:50:29.224120+0000","last_peered":"2023-08-09T13:50:29.224120+0000","last_clean":"2023-08-09T13:50:29.224120+0000","last_became_active":"2023-08-09T13:50:28.189588+0000","last_became_peered":"2023-08-09T13:50:28.189588+0000","last_unstale":"2023-08-09T13:50:29.224120+0000","last_undegraded":"2023-08-09T13:50:29.224120+0000","last_fullsized":"2023-08-09T13:50:29.224120+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:50:27.140775+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:50:27.140775+0000","last_clean_scrub_stamp":"2023-08-09T13:50:27.140775+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T16:47:26.737473+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.3","version":"0'0","reported_seq":8,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-08-09T13:50:29.250246+0000","last_change":"2023-08-09T13:50:29.250246+0000","last_active":"2023-08-09T13:50:29.250246+0000","last_peered":"2023-08-09T13:50:29.250246+0000","last_clean":"2023-08-09T13:50:29.250246+0000","last_became_active":"2023-08-09T13:50:29.249885+0000","last_became_peered":"2023-08-09T13:50:29.249885+0000","last_unstale":"2023-08-09T13:50:29.250246+0000","last_undegraded":"2023-08-09T13:50:29.250246+0000","last_fullsized":"2023-08-09T13:50:29.250246+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:50:28.150437+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:50:28.150437+0000","last_clean_scrub_stamp":"2023-08-09T13:50:28.150437+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T21:52:25.852199+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"10.2","version":"0'0","reported_seq":13,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-08-09T13:50:30.207205+0000","last_change":"2023-08-09T13:50:28.202029+0000","last_active":"2023-08-09T13:50:30.207205+0000","last_peered":"2023-08-09T13:50:30.207205+0000","last_clean":"2023-08-09T13:50:30.207205+0000","last_became_active":"2023-08-09T13:50:28.201720+0000","last_became_peered":"2023-08-09T13:50:28.201720+0000","last_unstale":"2023-08-09T13:50:30.207205+0000","last_undegraded":"2023-08-09T13:50:30.207205+0000","last_fullsized":"2023-08-09T13:50:30.207205+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:50:27.140775+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:50:27.140775+0000","last_clean_scrub_stamp":"2023-08-09T13:50:27.140775+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T16:25:36.557353+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.0","version":"0'0","reported_seq":11,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-08-09T13:50:30.207491+0000","last_change":"2023-08-09T13:50:29.233938+0000","last_active":"2023-08-09T13:50:30.207491+0000","last_peered":"2023-08-09T13:50:30.207491+0000","last_clean":"2023-08-09T13:50:30.207491+0000","last_became_active":"2023-08-09T13:50:29.233536+0000","last_became_peered":"2023-08-09T13:50:29.233536+0000","last_unstale":"2023-08-09T13:50:30.207491+0000","last_undegraded":"2023-08-09T13:50:30.207491+0000","last_fullsized":"2023-08-09T13:50:30.207491+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:50:28.150437+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:50:28.150437+0000","last_clean_scrub_stamp":"2023-08-09T13:50:28.150437+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T16:15:00.918668+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.1","version":"0'0","reported_seq":11,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-08-09T13:50:29.209627+0000","last_change":"2023-08-09T13:50:28.200080+0000","last_active":"2023-08-09T13:50:29.209627+0000","last_peered":"2023-08-09T13:50:29.209627+0000","last_clean":"2023-08-09T13:50:29.209627+0000","last_became_active":"2023-08-09T13:50:28.199831+0000","last_became_peered":"2023-08-09T13:50:28.199831+0000","last_unstale":"2023-08-09T13:50:29.209627+0000","last_undegraded":"2023-08-09T13:50:29.209627+0000","last_fullsized":"2023-08-09T13:50:29.209627+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:50:27.140775+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:50:27.140775+0000","last_clean_scrub_stamp":"2023-08-09T13:50:27.140775+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T15:18:11.293013+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.1","version":"0'0","reported_seq":8,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-08-09T13:50:29.249892+0000","last_change":"2023-08-09T13:50:29.249892+0000","last_active":"2023-08-09T13:50:29.249892+0000","last_peered":"2023-08-09T13:50:29.249892+0000","last_clean":"2023-08-09T13:50:29.249892+0000","last_became_active":"2023-08-09T13:50:29.242554+0000","last_became_peered":"2023-08-09T13:50:29.242554+0000","last_unstale":"2023-08-09T13:50:29.249892+0000","last_undegraded":"2023-08-09T13:50:29.249892+0000","last_fullsized":"2023-08-09T13:50:29.249892+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:50:28.150437+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:50:28.150437+0000","last_clean_scrub_stamp":"2023-08-09T13:50:28.150437+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T21:18:31.512056+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"10.0","version":"0'0","reported_seq":11,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-08-09T13:50:29.242250+0000","last_change":"2023-08-09T13:50:28.197319+0000","last_active":"2023-08-09T13:50:29.242250+0000","last_peered":"2023-08-09T13:50:29.242250+0000","last_clean":"2023-08-09T13:50:29.242250+0000","last_became_active":"2023-08-09T13:50:28.197031+0000","last_became_peered":"2023-08-09T13:50:28.197031+0000","last_unstale":"2023-08-09T13:50:29.242250+0000","last_undegraded":"2023-08-09T13:50:29.242250+0000","last_fullsized":"2023-08-09T13:50:29.242250+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:50:27.140775+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:50:27.140775+0000","last_clean_scrub_stamp":"2023-08-09T13:50:27.140775+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T16:15:05.176106+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.a","version":"0'0","reported_seq":3,"reported_epoch":89,"state":"creating+activating","last_fresh":"2023-08-09T13:50:29.212247+0000","last_change":"2023-08-09T13:50:29.212247+0000","last_active":"2023-08-09T13:50:28.150437+0000","last_peered":"2023-08-09T13:50:28.150437+0000","last_clean":"2023-08-09T13:50:28.150437+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-09T13:50:29.212247+0000","last_undegraded":"2023-08-09T13:50:29.212247+0000","last_fullsized":"2023-08-09T13:50:29.212247+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:50:28.150437+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:50:28.150437+0000","last_clean_scrub_stamp":"2023-08-09T13:50:28.150437+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T19:55:58.171731+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":["3","6"],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.b","version":"0'0","reported_seq":11,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-08-09T13:50:29.230951+0000","last_change":"2023-08-09T13:50:28.203783+0000","last_active":"2023-08-09T13:50:29.230951+0000","last_peered":"2023-08-09T13:50:29.230951+0000","last_clean":"2023-08-09T13:50:29.230951+0000","last_became_active":"2023-08-09T13:50:28.202391+0000","last_became_peered":"2023-08-09T13:50:28.202391+0000","last_unstale":"2023-08-09T13:50:29.230951+0000","last_undegraded":"2023-08-09T13:50:29.230951+0000","last_fullsized":"2023-08-09T13:50:29.230951+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:50:27.140775+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:50:27.140775+0000","last_clean_scrub_stamp":"2023-08-09T13:50:27.140775+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T21:39:38.493243+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":149,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-08-09T13:50:29.216433+0000","last_change":"2023-08-09T13:33:56.401406+0000","last_active":"2023-08-09T13:50:29.216433+0000","last_peered":"2023-08-09T13:50:29.216433+0000","last_clean":"2023-08-09T13:50:29.216433+0000","last_became_active":"2023-08-09T13:33:56.400487+0000","last_became_peered":"2023-08-09T13:33:56.400487+0000","last_unstale":"2023-08-09T13:50:29.216433+0000","last_undegraded":"2023-08-09T13:50:29.216433+0000","last_fullsized":"2023-08-09T13:50:29.216433+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:33:54.845869+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:33:54.845869+0000","last_clean_scrub_stamp":"2023-08-09T13:33:54.845869+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-11T00:46:00.165825+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"11.b","version":"0'0","reported_seq":3,"reported_epoch":89,"state":"creating+activating","last_fresh":"2023-08-09T13:50:29.211663+0000","last_change":"2023-08-09T13:50:29.211663+0000","last_active":"2023-08-09T13:50:28.150437+0000","last_peered":"2023-08-09T13:50:28.150437+0000","last_clean":"2023-08-09T13:50:28.150437+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-09T13:50:29.211663+0000","last_undegraded":"2023-08-09T13:50:29.211663+0000","last_fullsized":"2023-08-09T13:50:29.211663+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:50:28.150437+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:50:28.150437+0000","last_clean_scrub_stamp":"2023-08-09T13:50:28.150437+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-11T01:04:03.965638+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":["3","1"],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.a","version":"0'0","reported_seq":11,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-08-09T13:50:29.230486+0000","last_change":"2023-08-09T13:50:28.205148+0000","last_active":"2023-08-09T13:50:29.230486+0000","last_peered":"2023-08-09T13:50:29.230486+0000","last_clean":"2023-08-09T13:50:29.230486+0000","last_became_active":"2023-08-09T13:50:28.204528+0000","last_became_peered":"2023-08-09T13:50:28.204528+0000","last_unstale":"2023-08-09T13:50:29.230486+0000","last_undegraded":"2023-08-09T13:50:29.230486+0000","last_fullsized":"2023-08-09T13:50:29.230486+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:50:27.140775+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:50:27.140775+0000","last_clean_scrub_stamp":"2023-08-09T13:50:27.140775+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T18:00:04.982575+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":148,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-08-09T13:50:29.222206+0000","last_change":"2023-08-09T13:33:56.401123+0000","last_active":"2023-08-09T13:50:29.222206+0000","last_peered":"2023-08-09T13:50:29.222206+0000","last_clean":"2023-08-09T13:50:29.222206+0000","last_became_active":"2023-08-09T13:33:56.399933+0000","last_became_peered":"2023-08-09T13:33:56.399933+0000","last_unstale":"2023-08-09T13:50:29.222206+0000","last_undegraded":"2023-08-09T13:50:29.222206+0000","last_fullsized":"2023-08-09T13:50:29.222206+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:33:54.845869+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:33:54.845869+0000","last_clean_scrub_stamp":"2023-08-09T13:33:54.845869+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T17:05:05.384552+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.8","version":"0'0","reported_seq":3,"reported_epoch":89,"state":"creating+activating","last_fresh":"2023-08-09T13:50:29.213241+0000","last_change":"2023-08-09T13:50:29.213241+0000","last_active":"2023-08-09T13:50:28.150437+0000","last_peered":"2023-08-09T13:50:28.150437+0000","last_clean":"2023-08-09T13:50:28.150437+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-09T13:50:29.213241+0000","last_undegraded":"2023-08-09T13:50:29.213241+0000","last_fullsized":"2023-08-09T13:50:29.213241+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:50:28.150437+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:50:28.150437+0000","last_clean_scrub_stamp":"2023-08-09T13:50:28.150437+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T22:09:32.464173+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":["3","6"],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.9","version":"0'0","reported_seq":11,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-08-09T13:50:29.222528+0000","last_change":"2023-08-09T13:50:28.192232+0000","last_active":"2023-08-09T13:50:29.222528+0000","last_peered":"2023-08-09T13:50:29.222528+0000","last_clean":"2023-08-09T13:50:29.222528+0000","last_became_active":"2023-08-09T13:50:28.191350+0000","last_became_peered":"2023-08-09T13:50:28.191350+0000","last_unstale":"2023-08-09T13:50:29.222528+0000","last_undegraded":"2023-08-09T13:50:29.222528+0000","last_fullsized":"2023-08-09T13:50:29.222528+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:50:27.140775+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:50:27.140775+0000","last_clean_scrub_stamp":"2023-08-09T13:50:27.140775+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-11T00:33:31.648004+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.2","version":"0'0","reported_seq":148,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-08-09T13:50:29.215918+0000","last_change":"2023-08-09T13:33:56.402794+0000","last_active":"2023-08-09T13:50:29.215918+0000","last_peered":"2023-08-09T13:50:29.215918+0000","last_clean":"2023-08-09T13:50:29.215918+0000","last_became_active":"2023-08-09T13:33:56.402038+0000","last_became_peered":"2023-08-09T13:33:56.402038+0000","last_unstale":"2023-08-09T13:50:29.215918+0000","last_undegraded":"2023-08-09T13:50:29.215918+0000","last_fullsized":"2023-08-09T13:50:29.215918+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:33:54.845869+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:33:54.845869+0000","last_clean_scrub_stamp":"2023-08-09T13:33:54.845869+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T16:35:28.133841+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"11.9","version":"0'0","reported_seq":3,"reported_epoch":89,"state":"creating+activating","last_fresh":"2023-08-09T13:50:29.211902+0000","last_change":"2023-08-09T13:50:29.211902+0000","last_active":"2023-08-09T13:50:28.150437+0000","last_peered":"2023-08-09T13:50:28.150437+0000","last_clean":"2023-08-09T13:50:28.150437+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-09T13:50:29.211902+0000","last_undegraded":"2023-08-09T13:50:29.211902+0000","last_fullsized":"2023-08-09T13:50:29.211902+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:50:28.150437+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:50:28.150437+0000","last_clean_scrub_stamp":"2023-08-09T13:50:28.150437+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T15:32:31.232979+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":["3","6"],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.8","version":"0'0","reported_seq":13,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-08-09T13:50:30.406875+0000","last_change":"2023-08-09T13:50:28.190965+0000","last_active":"2023-08-09T13:50:30.406875+0000","last_peered":"2023-08-09T13:50:30.406875+0000","last_clean":"2023-08-09T13:50:30.406875+0000","last_became_active":"2023-08-09T13:50:28.190696+0000","last_became_peered":"2023-08-09T13:50:28.190696+0000","last_unstale":"2023-08-09T13:50:30.406875+0000","last_undegraded":"2023-08-09T13:50:30.406875+0000","last_fullsized":"2023-08-09T13:50:30.406875+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:50:27.140775+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:50:27.140775+0000","last_clean_scrub_stamp":"2023-08-09T13:50:27.140775+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T22:28:09.956793+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.3","version":"0'0","reported_seq":150,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-08-09T13:50:30.207472+0000","last_change":"2023-08-09T13:33:55.886749+0000","last_active":"2023-08-09T13:50:30.207472+0000","last_peered":"2023-08-09T13:50:30.207472+0000","last_clean":"2023-08-09T13:50:30.207472+0000","last_became_active":"2023-08-09T13:33:55.886116+0000","last_became_peered":"2023-08-09T13:33:55.886116+0000","last_unstale":"2023-08-09T13:50:30.207472+0000","last_undegraded":"2023-08-09T13:50:30.207472+0000","last_fullsized":"2023-08-09T13:50:30.207472+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:33:54.845869+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:33:54.845869+0000","last_clean_scrub_stamp":"2023-08-09T13:33:54.845869+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T23:29:59.861309+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"77'161","reported_seq":455,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-08-09T13:50:29.216076+0000","last_change":"2023-08-09T13:33:55.886345+0000","last_active":"2023-08-09T13:50:29.216076+0000","last_peered":"2023-08-09T13:50:29.216076+0000","last_clean":"2023-08-09T13:50:29.216076+0000","last_became_active":"2023-08-09T13:33:55.885781+0000","last_became_peered":"2023-08-09T13:33:55.885781+0000","last_unstale":"2023-08-09T13:50:29.216076+0000","last_undegraded":"2023-08-09T13:50:29.216076+0000","last_fullsized":"2023-08-09T13:50:29.216076+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:33:54.845869+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:33:54.845869+0000","last_clean_scrub_stamp":"2023-08-09T13:33:54.845869+0000","objects_scrubbed":0,"log_size":161,"log_dups_size":0,"ondisk_log_size":161,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T19:49:09.757332+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":500,"num_read_kb":924,"num_write":239,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"11.e","version":"0'0","reported_seq":9,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-08-09T13:50:29.232907+0000","last_change":"2023-08-09T13:50:29.227243+0000","last_active":"2023-08-09T13:50:29.232907+0000","last_peered":"2023-08-09T13:50:29.232907+0000","last_clean":"2023-08-09T13:50:29.232907+0000","last_became_active":"2023-08-09T13:50:29.226756+0000","last_became_peered":"2023-08-09T13:50:29.226756+0000","last_unstale":"2023-08-09T13:50:29.232907+0000","last_undegraded":"2023-08-09T13:50:29.232907+0000","last_fullsized":"2023-08-09T13:50:29.232907+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:50:28.150437+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:50:28.150437+0000","last_clean_scrub_stamp":"2023-08-09T13:50:28.150437+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T18:35:12.621508+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"10.f","version":"0'0","reported_seq":11,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-08-09T13:50:29.230485+0000","last_change":"2023-08-09T13:50:28.205120+0000","last_active":"2023-08-09T13:50:29.230485+0000","last_peered":"2023-08-09T13:50:29.230485+0000","last_clean":"2023-08-09T13:50:29.230485+0000","last_became_active":"2023-08-09T13:50:28.204771+0000","last_became_peered":"2023-08-09T13:50:28.204771+0000","last_unstale":"2023-08-09T13:50:29.230485+0000","last_undegraded":"2023-08-09T13:50:29.230485+0000","last_fullsized":"2023-08-09T13:50:29.230485+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:50:27.140775+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:50:27.140775+0000","last_clean_scrub_stamp":"2023-08-09T13:50:27.140775+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T17:34:29.132908+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":158,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-08-09T13:50:29.211123+0000","last_change":"2023-08-09T13:33:55.889565+0000","last_active":"2023-08-09T13:50:29.211123+0000","last_peered":"2023-08-09T13:50:29.211123+0000","last_clean":"2023-08-09T13:50:29.211123+0000","last_became_active":"2023-08-09T13:33:55.889164+0000","last_became_peered":"2023-08-09T13:33:55.889164+0000","last_unstale":"2023-08-09T13:50:29.211123+0000","last_undegraded":"2023-08-09T13:50:29.211123+0000","last_fullsized":"2023-08-09T13:50:29.211123+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:33:54.845869+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:33:54.845869+0000","last_clean_scrub_stamp":"2023-08-09T13:33:54.845869+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T17:48:30.355598+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.f","version":"0'0","reported_seq":8,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-08-09T13:50:29.238559+0000","last_change":"2023-08-09T13:50:29.238559+0000","last_active":"2023-08-09T13:50:29.238559+0000","last_peered":"2023-08-09T13:50:29.238559+0000","last_clean":"2023-08-09T13:50:29.238559+0000","last_became_active":"2023-08-09T13:50:29.238287+0000","last_became_peered":"2023-08-09T13:50:29.238287+0000","last_unstale":"2023-08-09T13:50:29.238559+0000","last_undegraded":"2023-08-09T13:50:29.238559+0000","last_fullsized":"2023-08-09T13:50:29.238559+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:50:28.150437+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:50:28.150437+0000","last_clean_scrub_stamp":"2023-08-09T13:50:28.150437+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T16:41:09.645304+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.e","version":"0'0","reported_seq":11,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-08-09T13:50:29.211102+0000","last_change":"2023-08-09T13:50:28.206561+0000","last_active":"2023-08-09T13:50:29.211102+0000","last_peered":"2023-08-09T13:50:29.211102+0000","last_clean":"2023-08-09T13:50:29.211102+0000","last_became_active":"2023-08-09T13:50:28.206264+0000","last_became_peered":"2023-08-09T13:50:28.206264+0000","last_unstale":"2023-08-09T13:50:29.211102+0000","last_undegraded":"2023-08-09T13:50:29.211102+0000","last_fullsized":"2023-08-09T13:50:29.211102+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:50:27.140775+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:50:27.140775+0000","last_clean_scrub_stamp":"2023-08-09T13:50:27.140775+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T15:29:27.312440+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":148,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-08-09T13:50:29.210681+0000","last_change":"2023-08-09T13:33:56.404236+0000","last_active":"2023-08-09T13:50:29.210681+0000","last_peered":"2023-08-09T13:50:29.210681+0000","last_clean":"2023-08-09T13:50:29.210681+0000","last_became_active":"2023-08-09T13:33:56.402485+0000","last_became_peered":"2023-08-09T13:33:56.402485+0000","last_unstale":"2023-08-09T13:50:29.210681+0000","last_undegraded":"2023-08-09T13:50:29.210681+0000","last_fullsized":"2023-08-09T13:50:29.210681+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:33:54.845869+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:33:54.845869+0000","last_clean_scrub_stamp":"2023-08-09T13:33:54.845869+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T13:49:18.686986+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.c","version":"0'0","reported_seq":11,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-08-09T13:50:30.207708+0000","last_change":"2023-08-09T13:50:29.230700+0000","last_active":"2023-08-09T13:50:30.207708+0000","last_peered":"2023-08-09T13:50:30.207708+0000","last_clean":"2023-08-09T13:50:30.207708+0000","last_became_active":"2023-08-09T13:50:29.230389+0000","last_became_peered":"2023-08-09T13:50:29.230389+0000","last_unstale":"2023-08-09T13:50:30.207708+0000","last_undegraded":"2023-08-09T13:50:30.207708+0000","last_fullsized":"2023-08-09T13:50:30.207708+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:50:28.150437+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:50:28.150437+0000","last_clean_scrub_stamp":"2023-08-09T13:50:28.150437+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-11T00:40:44.937679+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.d","version":"0'0","reported_seq":11,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-08-09T13:50:29.227255+0000","last_change":"2023-08-09T13:50:28.199480+0000","last_active":"2023-08-09T13:50:29.227255+0000","last_peered":"2023-08-09T13:50:29.227255+0000","last_clean":"2023-08-09T13:50:29.227255+0000","last_became_active":"2023-08-09T13:50:28.199211+0000","last_became_peered":"2023-08-09T13:50:28.199211+0000","last_unstale":"2023-08-09T13:50:29.227255+0000","last_undegraded":"2023-08-09T13:50:29.227255+0000","last_fullsized":"2023-08-09T13:50:29.227255+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:50:27.140775+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:50:27.140775+0000","last_clean_scrub_stamp":"2023-08-09T13:50:27.140775+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T18:35:43.990648+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":146,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-08-09T13:50:29.237073+0000","last_change":"2023-08-09T13:33:56.401781+0000","last_active":"2023-08-09T13:50:29.237073+0000","last_peered":"2023-08-09T13:50:29.237073+0000","last_clean":"2023-08-09T13:50:29.237073+0000","last_became_active":"2023-08-09T13:33:56.400336+0000","last_became_peered":"2023-08-09T13:33:56.400336+0000","last_unstale":"2023-08-09T13:50:29.237073+0000","last_undegraded":"2023-08-09T13:50:29.237073+0000","last_fullsized":"2023-08-09T13:50:29.237073+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:33:54.845869+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:33:54.845869+0000","last_clean_scrub_stamp":"2023-08-09T13:33:54.845869+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-11T01:01:47.058457+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.d","version":"0'0","reported_seq":3,"reported_epoch":89,"state":"creating+activating","last_fresh":"2023-08-09T13:50:29.220533+0000","last_change":"2023-08-09T13:50:29.220533+0000","last_active":"2023-08-09T13:50:28.150437+0000","last_peered":"2023-08-09T13:50:28.150437+0000","last_clean":"2023-08-09T13:50:28.150437+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-09T13:50:29.220533+0000","last_undegraded":"2023-08-09T13:50:29.220533+0000","last_fullsized":"2023-08-09T13:50:29.220533+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:50:28.150437+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:50:28.150437+0000","last_clean_scrub_stamp":"2023-08-09T13:50:28.150437+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T17:52:40.304270+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":["2","4"],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.c","version":"0'0","reported_seq":11,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-08-09T13:50:29.216926+0000","last_change":"2023-08-09T13:50:28.203868+0000","last_active":"2023-08-09T13:50:29.216926+0000","last_peered":"2023-08-09T13:50:29.216926+0000","last_clean":"2023-08-09T13:50:29.216926+0000","last_became_active":"2023-08-09T13:50:28.203323+0000","last_became_peered":"2023-08-09T13:50:28.203323+0000","last_unstale":"2023-08-09T13:50:29.216926+0000","last_undegraded":"2023-08-09T13:50:29.216926+0000","last_fullsized":"2023-08-09T13:50:29.216926+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:50:27.140775+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:50:27.140775+0000","last_clean_scrub_stamp":"2023-08-09T13:50:27.140775+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T16:53:02.601172+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":146,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-08-09T13:50:29.230723+0000","last_change":"2023-08-09T13:33:55.886389+0000","last_active":"2023-08-09T13:50:29.230723+0000","last_peered":"2023-08-09T13:50:29.230723+0000","last_clean":"2023-08-09T13:50:29.230723+0000","last_became_active":"2023-08-09T13:33:55.885638+0000","last_became_peered":"2023-08-09T13:33:55.885638+0000","last_unstale":"2023-08-09T13:50:29.230723+0000","last_undegraded":"2023-08-09T13:50:29.230723+0000","last_fullsized":"2023-08-09T13:50:29.230723+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:33:54.845869+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:33:54.845869+0000","last_clean_scrub_stamp":"2023-08-09T13:33:54.845869+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T22:08:50.990029+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]}],"pool_stats":[{"poolid":11,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":10,"num_pg":32,"stat_sum":{"num_bytes":1646,"num_objects":6,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9,"num_write_kb":10,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":2228,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":9,"ondisk_log_size":9,"up":64,"acting":64,"num_store_stats":8},{"poolid":2,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":500,"num_read_kb":924,"num_write":239,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":161,"ondisk_log_size":161,"up":2,"acting":2,"num_store_stats":2},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":8}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834575047,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":36840,"kb_used_data":1624,"kb_used_omap":1,"kb_used_meta":35198,"kb_avail":94335000,"statfs":{"total":96636764160,"available":96599040000,"internally_reserved":0,"allocated":1662976,"data_stored":1159636,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1781,"internal_metadata":36043019},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Aug 9 13:50:16 2023","interfaces":[{"interface":"back","average":{"1min":1.181,"5min":1.222,"15min":1.232},"min":{"1min":0.69499999999999995,"5min":0.63400000000000001,"15min":0.61099999999999999},"max":{"1min":1.633,"5min":1.8919999999999999,"15min":2.1280000000000001},"last":1.2569999999999999},{"interface":"front","average":{"1min":1.1819999999999999,"5min":1.232,"15min":1.24},"min":{"1min":0.72399999999999998,"5min":0.72399999999999998,"15min":0.65000000000000002},"max":{"1min":1.6100000000000001,"5min":1.839,"15min":2.0099999999999998},"last":1.1259999999999999}]},{"osd":1,"last update":"Wed Aug 9 13:50:16 2023","interfaces":[{"interface":"back","average":{"1min":1.2010000000000001,"5min":1.2450000000000001,"15min":1.256},"min":{"1min":0.72799999999999998,"5min":0.72799999999999998,"15min":0.59799999999999998},"max":{"1min":1.8,"5min":1.8,"15min":1.9470000000000001},"last":1.403},{"interface":"front","average":{"1min":1.2569999999999999,"5min":1.248,"15min":1.2869999999999999},"min":{"1min":0.83099999999999996,"5min":0.59299999999999997,"15min":0.59299999999999997},"max":{"1min":1.726,"5min":1.9410000000000001,"15min":1.9410000000000001},"last":1.327}]},{"osd":2,"last update":"Wed Aug 9 13:50:16 2023","interfaces":[{"interface":"back","average":{"1min":1.278,"5min":1.2470000000000001,"15min":1.2430000000000001},"min":{"1min":0.81000000000000005,"5min":0.65700000000000003,"15min":0.65700000000000003},"max":{"1min":1.742,"5min":2.0129999999999999,"15min":2.0129999999999999},"last":0.99399999999999999},{"interface":"front","average":{"1min":1.2889999999999999,"5min":1.323,"15min":1.3759999999999999},"min":{"1min":0.92600000000000005,"5min":0.80900000000000005,"15min":0.67800000000000005},"max":{"1min":1.7130000000000001,"5min":1.8740000000000001,"15min":2.4140000000000001},"last":1.623}]},{"osd":3,"last update":"Wed Aug 9 13:50:16 2023","interfaces":[{"interface":"back","average":{"1min":1.2529999999999999,"5min":1.3180000000000001,"15min":1.331},"min":{"1min":0.98799999999999999,"5min":0.86799999999999999,"15min":0.77500000000000002},"max":{"1min":1.7809999999999999,"5min":1.97,"15min":1.97},"last":1.3680000000000001},{"interface":"front","average":{"1min":1.3740000000000001,"5min":1.419,"15min":1.4159999999999999},"min":{"1min":0.93899999999999995,"5min":0.88300000000000001,"15min":0.82099999999999995},"max":{"1min":1.8160000000000001,"5min":2.0790000000000002,"15min":2.2480000000000002},"last":1.526}]},{"osd":4,"last update":"Wed Aug 9 13:50:16 2023","interfaces":[{"interface":"back","average":{"1min":1.343,"5min":1.2649999999999999,"15min":1.2769999999999999},"min":{"1min":0.86399999999999999,"5min":0.59499999999999997,"15min":0.59499999999999997},"max":{"1min":1.772,"5min":2.032,"15min":2.343},"last":0.82499999999999996},{"interface":"front","average":{"1min":1.296,"5min":1.3160000000000001,"15min":1.339},"min":{"1min":0.93600000000000005,"5min":0.77100000000000002,"15min":0.56799999999999995},"max":{"1min":1.6739999999999999,"5min":2.1419999999999999,"15min":2.1419999999999999},"last":1.2010000000000001}]},{"osd":5,"last update":"Wed Aug 9 13:50:16 2023","interfaces":[{"interface":"back","average":{"1min":1.4430000000000001,"5min":1.4350000000000001,"15min":1.419},"min":{"1min":1.048,"5min":0.88600000000000001,"15min":0.71499999999999997},"max":{"1min":1.9039999999999999,"5min":2.1480000000000001,"15min":2.6749999999999998},"last":1.6879999999999999},{"interface":"front","average":{"1min":1.4810000000000001,"5min":1.4319999999999999,"15min":1.4510000000000001},"min":{"1min":0.95999999999999996,"5min":0.85699999999999998,"15min":0.69199999999999995},"max":{"1min":1.877,"5min":2.113,"15min":2.9569999999999999},"last":1.4510000000000001}]},{"osd":6,"last update":"Wed Aug 9 13:50:16 2023","interfaces":[{"interface":"back","average":{"1min":1.6020000000000001,"5min":1.518,"15min":1.5489999999999999},"min":{"1min":1.2,"5min":0.85499999999999998,"15min":0.85499999999999998},"max":{"1min":1.8879999999999999,"5min":2.2069999999999999,"15min":2.8149999999999999},"last":1.575},{"interface":"front","average":{"1min":1.4139999999999999,"5min":1.4570000000000001,"15min":1.474},"min":{"1min":1.111,"5min":0.90600000000000003,"15min":0.60799999999999998},"max":{"1min":1.9419999999999999,"5min":2.1320000000000001,"15min":2.2530000000000001},"last":1.4910000000000001}]}]},{"osd":6,"up_from":13,"seq":55834575047,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32156,"kb_used_data":1040,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":94339684,"statfs":{"total":96636764160,"available":96603836416,"internally_reserved":0,"allocated":1064960,"data_stored":567907,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1781,"internal_metadata":31848715},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Aug 9 13:50:20 2023","interfaces":[{"interface":"back","average":{"1min":1.1160000000000001,"5min":1.143,"15min":1.161},"min":{"1min":0.75,"5min":0.72499999999999998,"15min":0.56599999999999995},"max":{"1min":1.5489999999999999,"5min":1.633,"15min":1.9870000000000001},"last":1.359},{"interface":"front","average":{"1min":1.21,"5min":1.532,"15min":1.3080000000000001},"min":{"1min":0.79300000000000004,"5min":0.79300000000000004,"15min":0.76000000000000001},"max":{"1min":1.6519999999999999,"5min":28.036000000000001,"15min":28.036000000000001},"last":1.4550000000000001}]},{"osd":1,"last update":"Wed Aug 9 13:50:20 2023","interfaces":[{"interface":"back","average":{"1min":1.147,"5min":1.272,"15min":1.226},"min":{"1min":0.70399999999999996,"5min":0.70399999999999996,"15min":0.65500000000000003},"max":{"1min":1.659,"5min":7.742,"15min":7.742},"last":1.256},{"interface":"front","average":{"1min":1.252,"5min":1.2849999999999999,"15min":1.276},"min":{"1min":0.84199999999999997,"5min":0.84199999999999997,"15min":0.68100000000000005},"max":{"1min":1.8899999999999999,"5min":1.8899999999999999,"15min":1.8899999999999999},"last":0.89100000000000001}]},{"osd":2,"last update":"Wed Aug 9 13:50:20 2023","interfaces":[{"interface":"back","average":{"1min":1.2989999999999999,"5min":1.3480000000000001,"15min":1.3220000000000001},"min":{"1min":0.877,"5min":0.81999999999999995,"15min":0.68700000000000006},"max":{"1min":1.73,"5min":4.0640000000000001,"15min":4.0640000000000001},"last":1.151},{"interface":"front","average":{"1min":1.244,"5min":1.4430000000000001,"15min":1.3460000000000001},"min":{"1min":0.76800000000000002,"5min":0.70999999999999996,"15min":0.70999999999999996},"max":{"1min":2.0390000000000001,"5min":13.154999999999999,"15min":13.154999999999999},"last":0.95399999999999996}]},{"osd":3,"last update":"Wed Aug 9 13:50:20 2023","interfaces":[{"interface":"back","average":{"1min":1.3939999999999999,"5min":1.52,"15min":1.419},"min":{"1min":0.73099999999999998,"5min":0.73099999999999998,"15min":0.73099999999999998},"max":{"1min":2.2519999999999998,"5min":13.349,"15min":13.349},"last":1.298},{"interface":"front","average":{"1min":1.258,"5min":1.3620000000000001,"15min":1.325},"min":{"1min":0.88400000000000001,"5min":0.76400000000000001,"15min":0.74299999999999999},"max":{"1min":1.8560000000000001,"5min":7.9029999999999996,"15min":7.9029999999999996},"last":1.397}]},{"osd":4,"last update":"Wed Aug 9 13:50:20 2023","interfaces":[{"interface":"back","average":{"1min":1.302,"5min":1.335,"15min":1.28},"min":{"1min":0.90700000000000003,"5min":0.78500000000000003,"15min":0.64300000000000002},"max":{"1min":1.992,"5min":5.2290000000000001,"15min":5.2290000000000001},"last":1.486},{"interface":"front","average":{"1min":1.4019999999999999,"5min":1.498,"15min":1.389},"min":{"1min":0.88900000000000001,"5min":0.78700000000000003,"15min":0.69099999999999995},"max":{"1min":1.734,"5min":12.1,"15min":12.1},"last":1.0860000000000001}]},{"osd":5,"last update":"Wed Aug 9 13:50:20 2023","interfaces":[{"interface":"back","average":{"1min":1.3169999999999999,"5min":1.4399999999999999,"15min":1.357},"min":{"1min":0.93999999999999995,"5min":0.83799999999999997,"15min":0.65400000000000003},"max":{"1min":2.1139999999999999,"5min":8.1899999999999995,"15min":8.1899999999999995},"last":1.0289999999999999},{"interface":"front","average":{"1min":1.5649999999999999,"5min":1.6639999999999999,"15min":1.589},"min":{"1min":1.2010000000000001,"5min":1.2010000000000001,"15min":0.83799999999999997},"max":{"1min":2.2000000000000002,"5min":8.8889999999999993,"15min":8.8889999999999993},"last":1.4299999999999999}]},{"osd":7,"last update":"Wed Aug 9 13:50:20 2023","interfaces":[{"interface":"back","average":{"1min":1.577,"5min":1.6970000000000001,"15min":1.5880000000000001},"min":{"1min":1.1850000000000001,"5min":1.0029999999999999,"15min":0.62},"max":{"1min":1.8240000000000001,"5min":9.3320000000000007,"15min":9.3320000000000007},"last":1.5289999999999999},{"interface":"front","average":{"1min":1.4410000000000001,"5min":1.6060000000000001,"15min":1.548},"min":{"1min":0.93899999999999995,"5min":0.88500000000000001,"15min":0.88500000000000001},"max":{"1min":1.9470000000000001,"5min":8.0399999999999991,"15min":8.0399999999999991},"last":1.5029999999999999}]}]},{"osd":5,"up_from":13,"seq":55834575047,"num_pgs":16,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":36260,"kb_used_data":1044,"kb_used_omap":1,"kb_used_meta":35198,"kb_avail":94335580,"statfs":{"total":96636764160,"available":96599633920,"internally_reserved":0,"allocated":1069056,"data_stored":569268,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1781,"internal_metadata":36043019},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Aug 9 13:50:19 2023","interfaces":[{"interface":"back","average":{"1min":1.2509999999999999,"5min":1.1759999999999999,"15min":1.179},"min":{"1min":0.63500000000000001,"5min":0.63500000000000001,"15min":0.63500000000000001},"max":{"1min":1.518,"5min":2.1869999999999998,"15min":2.1869999999999998},"last":1.1020000000000001},{"interface":"front","average":{"1min":1.1719999999999999,"5min":1.2,"15min":1.198},"min":{"1min":0.68600000000000005,"5min":0.68600000000000005,"15min":0.64200000000000002},"max":{"1min":1.579,"5min":1.6619999999999999,"15min":1.7989999999999999},"last":1.1950000000000001}]},{"osd":1,"last update":"Wed Aug 9 13:50:19 2023","interfaces":[{"interface":"back","average":{"1min":1.165,"5min":1.1990000000000001,"15min":1.2190000000000001},"min":{"1min":0.81599999999999995,"5min":0.73099999999999998,"15min":0.64100000000000001},"max":{"1min":1.631,"5min":1.6739999999999999,"15min":1.736},"last":1.0049999999999999},{"interface":"front","average":{"1min":1.264,"5min":1.296,"15min":1.2769999999999999},"min":{"1min":0.93300000000000005,"5min":0.66900000000000004,"15min":0.66900000000000004},"max":{"1min":1.6699999999999999,"5min":1.887,"15min":1.887},"last":0.84999999999999998}]},{"osd":2,"last update":"Wed Aug 9 13:50:19 2023","interfaces":[{"interface":"back","average":{"1min":1.2410000000000001,"5min":1.2569999999999999,"15min":1.2849999999999999},"min":{"1min":0.98199999999999998,"5min":0.68999999999999995,"15min":0.61599999999999999},"max":{"1min":1.5760000000000001,"5min":1.756,"15min":1.899},"last":1.4670000000000001},{"interface":"front","average":{"1min":1.3049999999999999,"5min":1.302,"15min":1.3149999999999999},"min":{"1min":0.78400000000000003,"5min":0.78400000000000003,"15min":0.66800000000000004},"max":{"1min":1.6599999999999999,"5min":1.7190000000000001,"15min":1.7969999999999999},"last":1.1419999999999999}]},{"osd":3,"last update":"Wed Aug 9 13:50:19 2023","interfaces":[{"interface":"back","average":{"1min":1.365,"5min":1.3720000000000001,"15min":1.3600000000000001},"min":{"1min":0.95399999999999996,"5min":0.95399999999999996,"15min":0.59799999999999998},"max":{"1min":1.6890000000000001,"5min":2.1840000000000002,"15min":2.2690000000000001},"last":1.3160000000000001},{"interface":"front","average":{"1min":1.4259999999999999,"5min":1.4379999999999999,"15min":1.4139999999999999},"min":{"1min":1.087,"5min":0.94899999999999995,"15min":0.70999999999999996},"max":{"1min":1.73,"5min":2.3780000000000001,"15min":2.3780000000000001},"last":1.3600000000000001}]},{"osd":4,"last update":"Wed Aug 9 13:50:19 2023","interfaces":[{"interface":"back","average":{"1min":1.262,"5min":1.2729999999999999,"15min":1.2889999999999999},"min":{"1min":0.92700000000000005,"5min":0.84699999999999998,"15min":0.56599999999999995},"max":{"1min":1.673,"5min":2.0800000000000001,"15min":2.121},"last":1.2789999999999999},{"interface":"front","average":{"1min":1.3919999999999999,"5min":1.3400000000000001,"15min":1.3779999999999999},"min":{"1min":0.81999999999999995,"5min":0.76100000000000001,"15min":0.58599999999999997},"max":{"1min":1.944,"5min":2.1230000000000002,"15min":2.3679999999999999},"last":1.403}]},{"osd":6,"last update":"Wed Aug 9 13:50:19 2023","interfaces":[{"interface":"back","average":{"1min":1.4299999999999999,"5min":1.3520000000000001,"15min":1.391},"min":{"1min":0.99099999999999999,"5min":0.90300000000000002,"15min":0.66700000000000004},"max":{"1min":1.9810000000000001,"5min":2.306,"15min":2.306},"last":1.5609999999999999},{"interface":"front","average":{"1min":1.49,"5min":1.4399999999999999,"15min":1.4710000000000001},"min":{"1min":1.1220000000000001,"5min":1.034,"15min":0.71199999999999997},"max":{"1min":1.827,"5min":2.2320000000000002,"15min":2.2320000000000002},"last":1.343}]},{"osd":7,"last update":"Wed Aug 9 13:50:19 2023","interfaces":[{"interface":"back","average":{"1min":1.577,"5min":1.5580000000000001,"15min":1.5529999999999999},"min":{"1min":1.1160000000000001,"5min":1.0589999999999999,"15min":0.70899999999999996},"max":{"1min":1.9610000000000001,"5min":2.452,"15min":2.4729999999999999},"last":1.5980000000000001},{"interface":"front","average":{"1min":1.611,"5min":1.585,"15min":1.6080000000000001},"min":{"1min":1.2110000000000001,"5min":1.163,"15min":0.77600000000000002},"max":{"1min":2.097,"5min":2.1070000000000002,"15min":2.3820000000000001},"last":1.6499999999999999}]}]},{"osd":4,"up_from":13,"seq":55834575047,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32160,"kb_used_data":1044,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":94339680,"statfs":{"total":96636764160,"available":96603832320,"internally_reserved":0,"allocated":1069056,"data_stored":569268,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1781,"internal_metadata":31848715},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Aug 9 13:50:21 2023","interfaces":[{"interface":"back","average":{"1min":1.2330000000000001,"5min":1.2390000000000001,"15min":1.2030000000000001},"min":{"1min":0.875,"5min":0.70699999999999996,"15min":0.67700000000000005},"max":{"1min":1.5940000000000001,"5min":2.524,"15min":2.524},"last":1.179},{"interface":"front","average":{"1min":1.206,"5min":1.236,"15min":1.2569999999999999},"min":{"1min":0.62,"5min":0.62,"15min":0.62},"max":{"1min":1.5980000000000001,"5min":1.863,"15min":1.863},"last":1.468}]},{"osd":1,"last update":"Wed Aug 9 13:50:21 2023","interfaces":[{"interface":"back","average":{"1min":1.244,"5min":1.3169999999999999,"15min":1.2929999999999999},"min":{"1min":0.70999999999999996,"5min":0.67800000000000005,"15min":0.67800000000000005},"max":{"1min":1.48,"5min":1.8480000000000001,"15min":1.8480000000000001},"last":1.026},{"interface":"front","average":{"1min":1.242,"5min":1.2649999999999999,"15min":1.248},"min":{"1min":0.81799999999999995,"5min":0.81799999999999995,"15min":0.64900000000000002},"max":{"1min":1.9330000000000001,"5min":1.9330000000000001,"15min":1.9330000000000001},"last":0.98499999999999999}]},{"osd":2,"last update":"Wed Aug 9 13:50:21 2023","interfaces":[{"interface":"back","average":{"1min":1.306,"5min":1.2869999999999999,"15min":1.2809999999999999},"min":{"1min":1.022,"5min":0.621,"15min":0.621},"max":{"1min":1.619,"5min":2.6429999999999998,"15min":2.6429999999999998},"last":0.69999999999999996},{"interface":"front","average":{"1min":1.343,"5min":1.3720000000000001,"15min":1.3660000000000001},"min":{"1min":0.88300000000000001,"5min":0.88300000000000001,"15min":0.74299999999999999},"max":{"1min":1.7569999999999999,"5min":1.7569999999999999,"15min":1.8149999999999999},"last":0.89000000000000001}]},{"osd":3,"last update":"Wed Aug 9 13:50:21 2023","interfaces":[{"interface":"back","average":{"1min":1.377,"5min":1.3300000000000001,"15min":1.298},"min":{"1min":1.083,"5min":0.86399999999999999,"15min":0.84699999999999998},"max":{"1min":1.6739999999999999,"5min":2.6099999999999999,"15min":2.6099999999999999},"last":1.212},{"interface":"front","average":{"1min":1.4099999999999999,"5min":1.3979999999999999,"15min":1.3799999999999999},"min":{"1min":0.94799999999999995,"5min":0.88800000000000001,"15min":0.63100000000000001},"max":{"1min":1.6890000000000001,"5min":2.7629999999999999,"15min":2.7629999999999999},"last":0.95599999999999996}]},{"osd":5,"last update":"Wed Aug 9 13:50:21 2023","interfaces":[{"interface":"back","average":{"1min":1.288,"5min":1.3400000000000001,"15min":1.3300000000000001},"min":{"1min":0.876,"5min":0.73199999999999998,"15min":0.63200000000000001},"max":{"1min":1.734,"5min":1.8,"15min":2.0099999999999998},"last":1.343},{"interface":"front","average":{"1min":1.2509999999999999,"5min":1.323,"15min":1.3180000000000001},"min":{"1min":0.92000000000000004,"5min":0.72399999999999998,"15min":0.626},"max":{"1min":1.71,"5min":2.7090000000000001,"15min":2.7090000000000001},"last":1.421}]},{"osd":6,"last update":"Wed Aug 9 13:50:21 2023","interfaces":[{"interface":"back","average":{"1min":1.379,"5min":1.3160000000000001,"15min":1.3360000000000001},"min":{"1min":0.94699999999999995,"5min":0.74399999999999999,"15min":0.59099999999999997},"max":{"1min":1.7509999999999999,"5min":2.8559999999999999,"15min":2.8559999999999999},"last":1.083},{"interface":"front","average":{"1min":1.4430000000000001,"5min":1.518,"15min":1.504},"min":{"1min":1.0760000000000001,"5min":0.97799999999999998,"15min":0.76400000000000001},"max":{"1min":1.669,"5min":2.2719999999999998,"15min":2.2719999999999998},"last":1.5409999999999999}]},{"osd":7,"last update":"Wed Aug 9 13:50:21 2023","interfaces":[{"interface":"back","average":{"1min":1.46,"5min":1.4830000000000001,"15min":1.474},"min":{"1min":1.1499999999999999,"5min":0.85999999999999999,"15min":0.73899999999999999},"max":{"1min":1.8460000000000001,"5min":2.8050000000000002,"15min":2.8050000000000002},"last":1.1100000000000001},{"interface":"front","average":{"1min":1.474,"5min":1.522,"15min":1.514},"min":{"1min":1.236,"5min":0.97199999999999998,"15min":0.72899999999999998},"max":{"1min":1.754,"5min":3.415,"15min":3.415},"last":1.3959999999999999}]}]},{"osd":3,"up_from":13,"seq":55834575048,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":36260,"kb_used_data":1044,"kb_used_omap":1,"kb_used_meta":35198,"kb_avail":94335580,"statfs":{"total":96636764160,"available":96599633920,"internally_reserved":0,"allocated":1069056,"data_stored":569268,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1781,"internal_metadata":36043019},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Aug 9 13:50:25 2023","interfaces":[{"interface":"back","average":{"1min":0.84599999999999997,"5min":0.88300000000000001,"15min":0.91600000000000004},"min":{"1min":0.43099999999999999,"5min":0.27200000000000002,"15min":0.27200000000000002},"max":{"1min":1.3939999999999999,"5min":1.8500000000000001,"15min":1.968},"last":0.65900000000000003},{"interface":"front","average":{"1min":1.071,"5min":1.0920000000000001,"15min":1.1060000000000001},"min":{"1min":0.48199999999999998,"5min":0.36399999999999999,"15min":0.36399999999999999},"max":{"1min":1.6910000000000001,"5min":1.6910000000000001,"15min":2.2360000000000002},"last":0.75600000000000001}]},{"osd":1,"last update":"Wed Aug 9 13:50:25 2023","interfaces":[{"interface":"back","average":{"1min":1.042,"5min":1.0089999999999999,"15min":1.012},"min":{"1min":0.59899999999999998,"5min":0.496,"15min":0.438},"max":{"1min":1.518,"5min":1.7110000000000001,"15min":2.5649999999999999},"last":0.83999999999999997},{"interface":"front","average":{"1min":1.091,"5min":1.1279999999999999,"15min":1.1639999999999999},"min":{"1min":0.51800000000000002,"5min":0.51800000000000002,"15min":0.42099999999999999},"max":{"1min":1.6839999999999999,"5min":1.6839999999999999,"15min":2.6760000000000002},"last":1.202}]},{"osd":2,"last update":"Wed Aug 9 13:50:25 2023","interfaces":[{"interface":"back","average":{"1min":1.121,"5min":1.177,"15min":1.1930000000000001},"min":{"1min":0.53600000000000003,"5min":0.53600000000000003,"15min":0.53500000000000003},"max":{"1min":1.6240000000000001,"5min":1.913,"15min":1.923},"last":1.1399999999999999},{"interface":"front","average":{"1min":1.1899999999999999,"5min":1.2929999999999999,"15min":1.3080000000000001},"min":{"1min":0.625,"5min":0.52300000000000002,"15min":0.52000000000000002},"max":{"1min":1.9399999999999999,"5min":2.0259999999999998,"15min":2.0259999999999998},"last":1.0960000000000001}]},{"osd":4,"last update":"Wed Aug 9 13:50:25 2023","interfaces":[{"interface":"back","average":{"1min":1.232,"5min":1.258,"15min":1.282},"min":{"1min":0.91500000000000004,"5min":0.57899999999999996,"15min":0.57899999999999996},"max":{"1min":1.742,"5min":1.8160000000000001,"15min":1.8160000000000001},"last":1.6819999999999999},{"interface":"front","average":{"1min":1.349,"5min":1.397,"15min":1.3919999999999999},"min":{"1min":0.67600000000000005,"5min":0.628,"15min":0.628},"max":{"1min":1.8149999999999999,"5min":2.2040000000000002,"15min":2.2370000000000001},"last":1.3080000000000001}]},{"osd":5,"last update":"Wed Aug 9 13:50:25 2023","interfaces":[{"interface":"back","average":{"1min":1.456,"5min":1.468,"15min":1.474},"min":{"1min":0.67600000000000005,"5min":0.67600000000000005,"15min":0.67200000000000004},"max":{"1min":1.855,"5min":2.254,"15min":2.254},"last":1.3959999999999999},{"interface":"front","average":{"1min":1.4850000000000001,"5min":1.4910000000000001,"15min":1.49},"min":{"1min":0.77600000000000002,"5min":0.65900000000000003,"15min":0.65900000000000003},"max":{"1min":2.3069999999999999,"5min":2.3069999999999999,"15min":2.3069999999999999},"last":1.3260000000000001}]},{"osd":6,"last update":"Wed Aug 9 13:50:25 2023","interfaces":[{"interface":"back","average":{"1min":1.3759999999999999,"5min":1.4370000000000001,"15min":1.4590000000000001},"min":{"1min":0.878,"5min":0.68200000000000005,"15min":0.68200000000000005},"max":{"1min":1.8759999999999999,"5min":2.1829999999999998,"15min":2.242},"last":1.4670000000000001},{"interface":"front","average":{"1min":1.466,"5min":1.5580000000000001,"15min":1.5529999999999999},"min":{"1min":1.0600000000000001,"5min":0.71099999999999997,"15min":0.68600000000000005},"max":{"1min":1.825,"5min":2.3420000000000001,"15min":2.3420000000000001},"last":1.252}]},{"osd":7,"last update":"Wed Aug 9 13:50:25 2023","interfaces":[{"interface":"back","average":{"1min":1.601,"5min":1.627,"15min":1.6160000000000001},"min":{"1min":1.115,"5min":0.69399999999999995,"15min":0.69399999999999995},"max":{"1min":2.46,"5min":2.46,"15min":2.46},"last":1.4279999999999999},{"interface":"front","average":{"1min":1.4910000000000001,"5min":1.546,"15min":1.5600000000000001},"min":{"1min":0.78200000000000003,"5min":0.64200000000000002,"15min":0.61599999999999999},"max":{"1min":2.0579999999999998,"5min":2.359,"15min":2.484},"last":1.571}]}]},{"osd":2,"up_from":13,"seq":55834575047,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":36256,"kb_used_data":1044,"kb_used_omap":1,"kb_used_meta":35198,"kb_avail":94335584,"statfs":{"total":96636764160,"available":96599638016,"internally_reserved":0,"allocated":1069056,"data_stored":569268,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1781,"internal_metadata":36043019},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Aug 9 13:50:17 2023","interfaces":[{"interface":"back","average":{"1min":0.96999999999999997,"5min":0.95299999999999996,"15min":0.95499999999999996},"min":{"1min":0.47799999999999998,"5min":0.36499999999999999,"15min":0.36499999999999999},"max":{"1min":1.5760000000000001,"5min":1.6180000000000001,"15min":1.8380000000000001},"last":1.484},{"interface":"front","average":{"1min":0.97099999999999997,"5min":0.998,"15min":1.0049999999999999},"min":{"1min":0.45300000000000001,"5min":0.374,"15min":0.374},"max":{"1min":1.4199999999999999,"5min":2.3929999999999998,"15min":2.3929999999999998},"last":0.80200000000000005}]},{"osd":1,"last update":"Wed Aug 9 13:50:17 2023","interfaces":[{"interface":"back","average":{"1min":1.2529999999999999,"5min":1.1739999999999999,"15min":1.1870000000000001},"min":{"1min":0.61299999999999999,"5min":0.55200000000000005,"15min":0.55200000000000005},"max":{"1min":1.6870000000000001,"5min":1.77,"15min":1.8420000000000001},"last":1.615},{"interface":"front","average":{"1min":1.216,"5min":1.2749999999999999,"15min":1.335},"min":{"1min":0.52500000000000002,"5min":0.52500000000000002,"15min":0.52500000000000002},"max":{"1min":1.7390000000000001,"5min":2.4870000000000001,"15min":2.4870000000000001},"last":1.03}]},{"osd":3,"last update":"Wed Aug 9 13:50:17 2023","interfaces":[{"interface":"back","average":{"1min":0.98099999999999998,"5min":1.006,"15min":1.0469999999999999},"min":{"1min":0.44800000000000001,"5min":0.38900000000000001,"15min":0.38900000000000001},"max":{"1min":1.3109999999999999,"5min":1.708,"15min":2.0409999999999999},"last":0.63600000000000001},{"interface":"front","average":{"1min":1.2110000000000001,"5min":1.304,"15min":1.3089999999999999},"min":{"1min":0.55500000000000005,"5min":0.46200000000000002,"15min":0.46200000000000002},"max":{"1min":1.8009999999999999,"5min":8.8130000000000006,"15min":8.8130000000000006},"last":1.2529999999999999}]},{"osd":4,"last update":"Wed Aug 9 13:50:17 2023","interfaces":[{"interface":"back","average":{"1min":1.415,"5min":1.4179999999999999,"15min":1.4510000000000001},"min":{"1min":0.83999999999999997,"5min":0.628,"15min":0.628},"max":{"1min":1.958,"5min":2.089,"15min":2.089},"last":1.556},{"interface":"front","average":{"1min":1.462,"5min":1.359,"15min":1.3979999999999999},"min":{"1min":0.73799999999999999,"5min":0.68600000000000005,"15min":0.68600000000000005},"max":{"1min":2.472,"5min":2.472,"15min":2.472},"last":1.151}]},{"osd":5,"last update":"Wed Aug 9 13:50:17 2023","interfaces":[{"interface":"back","average":{"1min":1.5189999999999999,"5min":1.4379999999999999,"15min":1.458},"min":{"1min":0.76600000000000001,"5min":0.59999999999999998,"15min":0.59999999999999998},"max":{"1min":2.27,"5min":2.5840000000000001,"15min":2.5840000000000001},"last":1.3240000000000001},{"interface":"front","average":{"1min":1.304,"5min":1.3540000000000001,"15min":1.349},"min":{"1min":0.77100000000000002,"5min":0.56799999999999995,"15min":0.56799999999999995},"max":{"1min":1.8859999999999999,"5min":3.5659999999999998,"15min":3.5659999999999998},"last":1.379}]},{"osd":6,"last update":"Wed Aug 9 13:50:17 2023","interfaces":[{"interface":"back","average":{"1min":1.4019999999999999,"5min":1.425,"15min":1.4259999999999999},"min":{"1min":0.79800000000000004,"5min":0.73799999999999999,"15min":0.73799999999999999},"max":{"1min":2.222,"5min":3.5099999999999998,"15min":3.5099999999999998},"last":1.079},{"interface":"front","average":{"1min":1.4790000000000001,"5min":1.4990000000000001,"15min":1.5409999999999999},"min":{"1min":0.82199999999999995,"5min":0.66300000000000003,"15min":0.66300000000000003},"max":{"1min":2.3769999999999998,"5min":2.3769999999999998,"15min":2.3769999999999998},"last":1.6930000000000001}]},{"osd":7,"last update":"Wed Aug 9 13:50:17 2023","interfaces":[{"interface":"back","average":{"1min":1.54,"5min":1.51,"15min":1.548},"min":{"1min":0.78800000000000003,"5min":0.64600000000000002,"15min":0.64600000000000002},"max":{"1min":2.1970000000000001,"5min":2.5249999999999999,"15min":2.5249999999999999},"last":1.4379999999999999},{"interface":"front","average":{"1min":1.4970000000000001,"5min":1.52,"15min":1.51},"min":{"1min":0.99199999999999999,"5min":0.70499999999999996,"15min":0.70499999999999996},"max":{"1min":2.089,"5min":3.609,"15min":3.609},"last":1.214}]}]},{"osd":1,"up_from":13,"seq":55834575048,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28696,"kb_used_data":1668,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343144,"statfs":{"total":96636764160,"available":96607379456,"internally_reserved":0,"allocated":1708032,"data_stored":1176373,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1781,"internal_metadata":27654411},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Aug 9 13:50:31 2023","interfaces":[{"interface":"back","average":{"1min":0.99099999999999999,"5min":0.92000000000000004,"15min":0.93999999999999995},"min":{"1min":0.50600000000000001,"5min":0.50600000000000001,"15min":0.32400000000000001},"max":{"1min":1.7110000000000001,"5min":1.7110000000000001,"15min":2.1080000000000001},"last":0.66600000000000004},{"interface":"front","average":{"1min":0.98999999999999999,"5min":1.0740000000000001,"15min":1.095},"min":{"1min":0.59799999999999998,"5min":0.432,"15min":0.42499999999999999},"max":{"1min":1.7190000000000001,"5min":2.468,"15min":2.468},"last":0.63400000000000001}]},{"osd":2,"last update":"Wed Aug 9 13:50:31 2023","interfaces":[{"interface":"back","average":{"1min":1.008,"5min":0.97799999999999998,"15min":0.995},"min":{"1min":0.56699999999999995,"5min":0.41099999999999998,"15min":0.41099999999999998},"max":{"1min":1.6679999999999999,"5min":1.907,"15min":1.907},"last":0.60799999999999998},{"interface":"front","average":{"1min":1.1459999999999999,"5min":1.159,"15min":1.1879999999999999},"min":{"1min":0.80600000000000005,"5min":0.54400000000000004,"15min":0.46500000000000002},"max":{"1min":1.5189999999999999,"5min":2.1280000000000001,"15min":2.145},"last":0.81999999999999995}]},{"osd":3,"last update":"Wed Aug 9 13:50:31 2023","interfaces":[{"interface":"back","average":{"1min":1.218,"5min":1.2629999999999999,"15min":1.3260000000000001},"min":{"1min":0.63200000000000001,"5min":0.54400000000000004,"15min":0.432},"max":{"1min":1.7430000000000001,"5min":2.0270000000000001,"15min":2.3210000000000002},"last":0.86199999999999999},{"interface":"front","average":{"1min":1.1779999999999999,"5min":1.234,"15min":1.2450000000000001},"min":{"1min":0.70399999999999996,"5min":0.49299999999999999,"15min":0.44400000000000001},"max":{"1min":1.786,"5min":2.3540000000000001,"15min":2.7669999999999999},"last":0.72899999999999998}]},{"osd":4,"last update":"Wed Aug 9 13:50:31 2023","interfaces":[{"interface":"back","average":{"1min":1.3149999999999999,"5min":1.399,"15min":1.4079999999999999},"min":{"1min":0.94399999999999995,"5min":0.79800000000000004,"15min":0.58099999999999996},"max":{"1min":1.696,"5min":1.97,"15min":2.5},"last":1.173},{"interface":"front","average":{"1min":1.2689999999999999,"5min":1.3180000000000001,"15min":1.333},"min":{"1min":0.88200000000000001,"5min":0.70099999999999996,"15min":0.70099999999999996},"max":{"1min":1.6950000000000001,"5min":1.901,"15min":1.901},"last":0.80600000000000005}]},{"osd":5,"last update":"Wed Aug 9 13:50:31 2023","interfaces":[{"interface":"back","average":{"1min":1.3200000000000001,"5min":1.323,"15min":1.371},"min":{"1min":0.90700000000000003,"5min":0.76900000000000002,"15min":0.76900000000000002},"max":{"1min":1.837,"5min":2.294,"15min":2.294},"last":0.76800000000000002},{"interface":"front","average":{"1min":1.427,"5min":1.464,"15min":1.516},"min":{"1min":0.97899999999999998,"5min":0.91000000000000003,"15min":0.77700000000000002},"max":{"1min":1.837,"5min":2.1389999999999998,"15min":2.4649999999999999},"last":1.2010000000000001}]},{"osd":6,"last update":"Wed Aug 9 13:50:31 2023","interfaces":[{"interface":"back","average":{"1min":1.4219999999999999,"5min":1.48,"15min":1.514},"min":{"1min":1.026,"5min":0.77500000000000002,"15min":0.67400000000000004},"max":{"1min":1.925,"5min":2.2080000000000002,"15min":2.5270000000000001},"last":1.004},{"interface":"front","average":{"1min":1.4370000000000001,"5min":1.45,"15min":1.5},"min":{"1min":1.1379999999999999,"5min":0.82499999999999996,"15min":0.79800000000000004},"max":{"1min":2.02,"5min":2.02,"15min":2.1040000000000001},"last":0.92400000000000004}]},{"osd":7,"last update":"Wed Aug 9 13:50:31 2023","interfaces":[{"interface":"back","average":{"1min":1.552,"5min":1.5649999999999999,"15min":1.619},"min":{"1min":1.0389999999999999,"5min":0.86899999999999999,"15min":0.77100000000000002},"max":{"1min":2.4169999999999998,"5min":2.4169999999999998,"15min":2.5790000000000002},"last":1.252},{"interface":"front","average":{"1min":1.5740000000000001,"5min":1.5549999999999999,"15min":1.6000000000000001},"min":{"1min":1.2589999999999999,"5min":0.91300000000000003,"15min":0.75900000000000001},"max":{"1min":2.036,"5min":2.1040000000000001,"15min":2.6509999999999998},"last":0.96899999999999997}]}]},{"osd":0,"up_from":13,"seq":55834575048,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":40380,"kb_used_data":1064,"kb_used_omap":1,"kb_used_meta":39294,"kb_avail":94331460,"statfs":{"total":96636764160,"available":96595415040,"internally_reserved":0,"allocated":1089536,"data_stored":582416,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1781,"internal_metadata":40237323},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Wed Aug 9 13:50:27 2023","interfaces":[{"interface":"back","average":{"1min":0.91800000000000004,"5min":0.92200000000000004,"15min":0.91600000000000004},"min":{"1min":0.32500000000000001,"5min":0.32500000000000001,"15min":0.32500000000000001},"max":{"1min":1.5820000000000001,"5min":1.756,"15min":1.756},"last":0.76400000000000001},{"interface":"front","average":{"1min":1.0680000000000001,"5min":1.1020000000000001,"15min":1.113},"min":{"1min":0.47099999999999997,"5min":0.45300000000000001,"15min":0.45300000000000001},"max":{"1min":1.9019999999999999,"5min":1.9019999999999999,"15min":1.9019999999999999},"last":1.171}]},{"osd":2,"last update":"Wed Aug 9 13:50:27 2023","interfaces":[{"interface":"back","average":{"1min":1.048,"5min":1.1000000000000001,"15min":1.135},"min":{"1min":0.53800000000000003,"5min":0.51400000000000001,"15min":0.44800000000000001},"max":{"1min":1.702,"5min":1.776,"15min":1.889},"last":0.95199999999999996},{"interface":"front","average":{"1min":1.1819999999999999,"5min":1.149,"15min":1.179},"min":{"1min":0.58499999999999996,"5min":0.56399999999999995,"15min":0.52700000000000002},"max":{"1min":2.0129999999999999,"5min":2.0129999999999999,"15min":2.2519999999999998},"last":1.3620000000000001}]},{"osd":3,"last update":"Wed Aug 9 13:50:27 2023","interfaces":[{"interface":"back","average":{"1min":0.98499999999999999,"5min":1.0509999999999999,"15min":1.105},"min":{"1min":0.54900000000000004,"5min":0.46300000000000002,"15min":0.46300000000000002},"max":{"1min":1.345,"5min":1.823,"15min":2.5190000000000001},"last":1.1040000000000001},{"interface":"front","average":{"1min":1.272,"5min":1.2629999999999999,"15min":1.3140000000000001},"min":{"1min":0.56399999999999995,"5min":0.54700000000000004,"15min":0.54700000000000004},"max":{"1min":2.347,"5min":2.347,"15min":2.4649999999999999},"last":1.0740000000000001}]},{"osd":4,"last update":"Wed Aug 9 13:50:27 2023","interfaces":[{"interface":"back","average":{"1min":1.302,"5min":1.3129999999999999,"15min":1.347},"min":{"1min":0.84699999999999998,"5min":0.84699999999999998,"15min":0.74299999999999999},"max":{"1min":1.9630000000000001,"5min":2.2309999999999999,"15min":2.2309999999999999},"last":1.323},{"interface":"front","average":{"1min":1.3999999999999999,"5min":1.4390000000000001,"15min":1.452},"min":{"1min":0.82699999999999996,"5min":0.77600000000000002,"15min":0.74099999999999999},"max":{"1min":1.8440000000000001,"5min":2.5390000000000001,"15min":2.5390000000000001},"last":1.248}]},{"osd":5,"last update":"Wed Aug 9 13:50:27 2023","interfaces":[{"interface":"back","average":{"1min":1.387,"5min":1.4219999999999999,"15min":1.4650000000000001},"min":{"1min":0.66300000000000003,"5min":0.66300000000000003,"15min":0.66300000000000003},"max":{"1min":2.6669999999999998,"5min":2.6669999999999998,"15min":2.6669999999999998},"last":1.649},{"interface":"front","average":{"1min":1.3400000000000001,"5min":1.415,"15min":1.4490000000000001},"min":{"1min":0.70099999999999996,"5min":0.629,"15min":0.629},"max":{"1min":1.6160000000000001,"5min":2.1499999999999999,"15min":2.1499999999999999},"last":1.444}]},{"osd":6,"last update":"Wed Aug 9 13:50:27 2023","interfaces":[{"interface":"back","average":{"1min":1.51,"5min":1.532,"15min":1.5469999999999999},"min":{"1min":0.73599999999999999,"5min":0.73599999999999999,"15min":0.69799999999999995},"max":{"1min":2.3100000000000001,"5min":2.7250000000000001,"15min":2.7250000000000001},"last":1.397},{"interface":"front","average":{"1min":1.5169999999999999,"5min":1.5389999999999999,"15min":1.5600000000000001},"min":{"1min":0.86699999999999999,"5min":0.86699999999999999,"15min":0.86699999999999999},"max":{"1min":2.3199999999999998,"5min":2.399,"15min":2.399},"last":1.4670000000000001}]},{"osd":7,"last update":"Wed Aug 9 13:50:27 2023","interfaces":[{"interface":"back","average":{"1min":1.532,"5min":1.595,"15min":1.625},"min":{"1min":0.72299999999999998,"5min":0.72299999999999998,"15min":0.72299999999999998},"max":{"1min":2.3900000000000001,"5min":2.3900000000000001,"15min":2.464},"last":1.5600000000000001},{"interface":"front","average":{"1min":1.4350000000000001,"5min":1.512,"15min":1.55},"min":{"1min":0.84499999999999997,"5min":0.76700000000000002,"15min":0.76700000000000002},"max":{"1min":2.274,"5min":2.274,"15min":2.274},"last":1.5049999999999999}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":2228,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-08-09T13:50:38.870 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-08-09T13:50:39.110 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:39.108+0000 7f6f15999640 1 Processor -- start 2023-08-09T13:50:39.110 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:39.108+0000 7f6f15999640 1 -- start start 2023-08-09T13:50:39.111 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:39.110+0000 7f6f15999640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f6f100ccf20 0x7f6f100cd550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:50:39.111 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:39.110+0000 7f6f15999640 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7f6f100cda90 0x7f6f1015ca10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:50:39.112 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:39.110+0000 7f6f15999640 1 -- --> v1:172.21.15.203:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6f100cca90 con 0x7f6f1015c630 2023-08-09T13:50:39.112 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:39.110+0000 7f6f15999640 1 -- --> v2:172.21.15.203:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6f100ac600 con 0x7f6f100cda90 2023-08-09T13:50:39.112 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:39.110+0000 7f6f15999640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6f100cbc70 con 0x7f6f100ccf20 2023-08-09T13:50:39.112 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:39.111+0000 7f6f0f7fe640 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7f6f100cda90 0x7f6f1015ca10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:50:39.113 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:39.111+0000 7f6f07fff640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f6f100ccf20 0x7f6f100cd550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:50:39.113 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:39.111+0000 7f6f0f7fe640 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7f6f100cda90 0x7f6f1015ca10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.203:3300/0 says I am v2:172.21.15.43:40024/0 (socket says 172.21.15.43:40024) 2023-08-09T13:50:39.113 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:39.111+0000 7f6f0f7fe640 1 -- 172.21.15.43:0/2687102906 learned_addr learned my addr 172.21.15.43:0/2687102906 (peer_addr_for_me v2:172.21.15.43:0/0) 2023-08-09T13:50:39.113 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:39.112+0000 7f6f0f7fe640 1 -- 172.21.15.43:0/2687102906 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f6f100ccf20 msgr2=0x7f6f100cd550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-09T13:50:39.114 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:39.112+0000 7f6f0f7fe640 1 --2- 172.21.15.43:0/2687102906 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f6f100ccf20 0x7f6f100cd550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:50:39.114 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:39.112+0000 7f6f0f7fe640 1 -- 172.21.15.43:0/2687102906 >> v1:172.21.15.203:6789/0 conn(0x7f6f1015c630 legacy=0x7f6f100abef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:50:39.114 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:39.112+0000 7f6f0f7fe640 1 -- 172.21.15.43:0/2687102906 --> v2:172.21.15.203:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6f100cbe80 con 0x7f6f100cda90 2023-08-09T13:50:39.115 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:39.113+0000 7f6f0f7fe640 1 --2- 172.21.15.43:0/2687102906 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f6f100cda90 0x7f6f1015ca10 secure :-1 s=READY pgs=371 cs=0 l=1 rev1=1 crypto rx=0x7f6f08009e50 tx=0x7f6f0802bf50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=361de9a273346b48 server_cookie=0 in_seq=0 out_seq=0 2023-08-09T13:50:39.115 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:39.113+0000 7f6f0e7fc640 1 -- 172.21.15.43:0/2687102906 <== mon.1 v2:172.21.15.203:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6f0802caf0 con 0x7f6f100cda90 2023-08-09T13:50:39.115 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:39.113+0000 7f6f0e7fc640 1 -- 172.21.15.43:0/2687102906 <== mon.1 v2:172.21.15.203:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6f0802cc50 con 0x7f6f100cda90 2023-08-09T13:50:39.115 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:39.113+0000 7f6f0e7fc640 1 -- 172.21.15.43:0/2687102906 <== mon.1 v2:172.21.15.203:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6f080343f0 con 0x7f6f100cda90 2023-08-09T13:50:39.116 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:39.114+0000 7f6f15999640 1 -- 172.21.15.43:0/2687102906 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f6f100cda90 msgr2=0x7f6f1015ca10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:50:39.116 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:39.114+0000 7f6f15999640 1 --2- 172.21.15.43:0/2687102906 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f6f100cda90 0x7f6f1015ca10 secure :-1 s=READY pgs=371 cs=0 l=1 rev1=1 crypto rx=0x7f6f08009e50 tx=0x7f6f0802bf50 comp rx=0 tx=0).stop 2023-08-09T13:50:39.116 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:39.114+0000 7f6f15999640 1 -- 172.21.15.43:0/2687102906 shutdown_connections 2023-08-09T13:50:39.116 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:39.114+0000 7f6f15999640 1 --2- 172.21.15.43:0/2687102906 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f6f100cda90 0x7f6f1015ca10 unknown :-1 s=CLOSED pgs=371 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:50:39.117 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:39.114+0000 7f6f15999640 1 --2- 172.21.15.43:0/2687102906 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f6f100ccf20 0x7f6f100cd550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:50:39.117 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:39.114+0000 7f6f15999640 1 -- 172.21.15.43:0/2687102906 >> 172.21.15.43:0/2687102906 conn(0x7f6f100ce0a0 msgr2=0x7f6f100ce480 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-09T13:50:39.117 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:39.114+0000 7f6f15999640 1 -- 172.21.15.43:0/2687102906 shutdown_connections 2023-08-09T13:50:39.117 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:39.114+0000 7f6f15999640 1 -- 172.21.15.43:0/2687102906 wait complete. 2023-08-09T13:50:39.117 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:39.116+0000 7f6f15999640 1 Processor -- start 2023-08-09T13:50:39.118 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:39.116+0000 7f6f15999640 1 -- start start 2023-08-09T13:50:39.118 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:39.117+0000 7f6f15999640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f6f100ccf20 0x7f6f100b0ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:50:39.119 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:39.117+0000 7f6f15999640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f6f100cda90 0x7f6f100adb00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:50:39.119 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:39.117+0000 7f6f15999640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f6f1015c630 0x7f6f100ae040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:50:39.119 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:39.117+0000 7f6f15999640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6f100ae580 con 0x7f6f100ccf20 2023-08-09T13:50:39.119 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:39.117+0000 7f6f15999640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6f100ae7c0 con 0x7f6f100cda90 2023-08-09T13:50:39.120 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:39.117+0000 7f6f15999640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6f100aeaa0 con 0x7f6f1015c630 2023-08-09T13:50:39.120 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:39.117+0000 7f6f0effd640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f6f100ccf20 0x7f6f100b0ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:50:39.121 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:39.117+0000 7f6f0effd640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f6f100ccf20 0x7f6f100b0ff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:45008/0 (socket says 172.21.15.43:45008) 2023-08-09T13:50:39.121 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:39.117+0000 7f6f0effd640 1 -- 172.21.15.43:0/1701932591 learned_addr learned my addr 172.21.15.43:0/1701932591 (peer_addr_for_me v2:172.21.15.43:0/0) 2023-08-09T13:50:39.121 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:39.118+0000 7f6f07fff640 1 --2- 172.21.15.43:0/1701932591 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f6f100cda90 0x7f6f100adb00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:50:39.121 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:39.118+0000 7f6f0f7fe640 1 --2- 172.21.15.43:0/1701932591 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f6f1015c630 0x7f6f100ae040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:50:39.121 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:39.118+0000 7f6f0effd640 1 -- 172.21.15.43:0/1701932591 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f6f1015c630 msgr2=0x7f6f100ae040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:50:39.122 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:39.118+0000 7f6f0effd640 1 --2- 172.21.15.43:0/1701932591 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f6f1015c630 0x7f6f100ae040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:50:39.122 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:39.118+0000 7f6f0effd640 1 -- 172.21.15.43:0/1701932591 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f6f100cda90 msgr2=0x7f6f100adb00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:50:39.122 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:39.118+0000 7f6f0effd640 1 --2- 172.21.15.43:0/1701932591 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f6f100cda90 0x7f6f100adb00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:50:39.122 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:39.118+0000 7f6f0effd640 1 -- 172.21.15.43:0/1701932591 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6f08009b30 con 0x7f6f100ccf20 2023-08-09T13:50:39.122 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:39.118+0000 7f6f0f7fe640 1 --2- 172.21.15.43:0/1701932591 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f6f1015c630 0x7f6f100ae040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-08-09T13:50:39.122 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:39.118+0000 7f6f0effd640 1 --2- 172.21.15.43:0/1701932591 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f6f100ccf20 0x7f6f100b0ff0 secure :-1 s=READY pgs=1031 cs=0 l=1 rev1=1 crypto rx=0x7f6f0000c400 tx=0x7f6f0000c8c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-09T13:50:39.122 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:39.118+0000 7f6f0cff9640 1 -- 172.21.15.43:0/1701932591 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6f000052d0 con 0x7f6f100ccf20 2023-08-09T13:50:39.123 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:39.118+0000 7f6f0cff9640 1 -- 172.21.15.43:0/1701932591 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6f00005450 con 0x7f6f100ccf20 2023-08-09T13:50:39.123 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:39.118+0000 7f6f15999640 1 -- 172.21.15.43:0/1701932591 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6f100aed80 con 0x7f6f100ccf20 2023-08-09T13:50:39.123 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:39.118+0000 7f6f0cff9640 1 -- 172.21.15.43:0/1701932591 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6f00007260 con 0x7f6f100ccf20 2023-08-09T13:50:39.123 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:39.119+0000 7f6f15999640 1 -- 172.21.15.43:0/1701932591 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6f100af340 con 0x7f6f100ccf20 2023-08-09T13:50:39.123 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:39.120+0000 7f6f15999640 1 -- 172.21.15.43:0/1701932591 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6f100739f0 con 0x7f6f100ccf20 2023-08-09T13:50:39.124 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:39.121+0000 7f6f0cff9640 1 -- 172.21.15.43:0/1701932591 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 29) v1 ==== 88443+0+0 (secure 0 0 0) 0x7f6f000073c0 con 0x7f6f100ccf20 2023-08-09T13:50:39.125 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:39.122+0000 7f6f0cff9640 1 --2- 172.21.15.43:0/1701932591 >> [v2:172.21.15.203:6832/3325141227,v1:172.21.15.203:6833/3325141227] conn(0x7f6ef006a170 0x7f6ef006c630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:50:39.125 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:39.122+0000 7f6f0cff9640 1 -- 172.21.15.43:0/1701932591 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(90..90 src has 1..90) v4 ==== 6426+0+0 (secure 0 0 0) 0x7f6f00087d80 con 0x7f6f100ccf20 2023-08-09T13:50:39.125 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:39.122+0000 7f6f07fff640 1 --2- 172.21.15.43:0/1701932591 >> [v2:172.21.15.203:6832/3325141227,v1:172.21.15.203:6833/3325141227] conn(0x7f6ef006a170 0x7f6ef006c630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:50:39.125 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:39.123+0000 7f6f07fff640 1 --2- 172.21.15.43:0/1701932591 >> [v2:172.21.15.203:6832/3325141227,v1:172.21.15.203:6833/3325141227] conn(0x7f6ef006a170 0x7f6ef006c630 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f6f10144010 tx=0x7f6ef8008040 comp rx=0 tx=0).ready entity=mgr.5519 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-09T13:50:39.128 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:39.126+0000 7f6f0cff9640 1 -- 172.21.15.43:0/1701932591 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178066 (secure 0 0 0) 0x7f6f00053150 con 0x7f6f100ccf20 2023-08-09T13:50:39.320 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:39.319+0000 7f6f15999640 1 -- 172.21.15.43:0/1701932591 --> [v2:172.21.15.203:6832/3325141227,v1:172.21.15.203:6833/3325141227] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f6f100cc160 con 0x7f6ef006a170 2023-08-09T13:50:39.346 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:39.344+0000 7f6f0cff9640 1 -- 172.21.15.43:0/1701932591 <== mgr.5519 v2:172.21.15.203:6832/3325141227 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+309060 (secure 0 0 0) 0x7f6ef80095c0 con 0x7f6ef006a170 2023-08-09T13:50:39.347 INFO:teuthology.orchestra.run.smithi043.stdout: 2023-08-09T13:50:39.351 INFO:teuthology.orchestra.run.smithi043.stderr:dumped all 2023-08-09T13:50:39.352 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:39.351+0000 7f6f15999640 1 -- 172.21.15.43:0/1701932591 >> [v2:172.21.15.203:6832/3325141227,v1:172.21.15.203:6833/3325141227] conn(0x7f6ef006a170 msgr2=0x7f6ef006c630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:50:39.352 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:39.351+0000 7f6f15999640 1 --2- 172.21.15.43:0/1701932591 >> [v2:172.21.15.203:6832/3325141227,v1:172.21.15.203:6833/3325141227] conn(0x7f6ef006a170 0x7f6ef006c630 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f6f10144010 tx=0x7f6ef8008040 comp rx=0 tx=0).stop 2023-08-09T13:50:39.353 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:39.351+0000 7f6f15999640 1 -- 172.21.15.43:0/1701932591 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f6f100ccf20 msgr2=0x7f6f100b0ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:50:39.353 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:39.351+0000 7f6f15999640 1 --2- 172.21.15.43:0/1701932591 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f6f100ccf20 0x7f6f100b0ff0 secure :-1 s=READY pgs=1031 cs=0 l=1 rev1=1 crypto rx=0x7f6f0000c400 tx=0x7f6f0000c8c0 comp rx=0 tx=0).stop 2023-08-09T13:50:39.353 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:39.351+0000 7f6f15999640 1 -- 172.21.15.43:0/1701932591 shutdown_connections 2023-08-09T13:50:39.353 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:39.351+0000 7f6f15999640 1 --2- 172.21.15.43:0/1701932591 >> [v2:172.21.15.203:6832/3325141227,v1:172.21.15.203:6833/3325141227] conn(0x7f6ef006a170 0x7f6ef006c630 unknown :-1 s=CLOSED pgs=70 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:50:39.354 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:39.351+0000 7f6f15999640 1 --2- 172.21.15.43:0/1701932591 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f6f1015c630 0x7f6f100ae040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:50:39.354 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:39.351+0000 7f6f15999640 1 --2- 172.21.15.43:0/1701932591 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f6f100cda90 0x7f6f100adb00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:50:39.354 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:39.351+0000 7f6f15999640 1 --2- 172.21.15.43:0/1701932591 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f6f100ccf20 0x7f6f100b0ff0 unknown :-1 s=CLOSED pgs=1031 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:50:39.354 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:39.351+0000 7f6f15999640 1 -- 172.21.15.43:0/1701932591 >> 172.21.15.43:0/1701932591 conn(0x7f6f100ce0a0 msgr2=0x7f6f100c93c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-09T13:50:39.354 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:39.351+0000 7f6f15999640 1 -- 172.21.15.43:0/1701932591 shutdown_connections 2023-08-09T13:50:39.354 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:39.351+0000 7f6f15999640 1 -- 172.21.15.43:0/1701932591 wait complete. 2023-08-09T13:50:39.375 INFO:teuthology.orchestra.run.smithi043.stdout:{"pg_ready":true,"pg_map":{"version":31,"stamp":"2023-08-09T13:50:38.234712+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":500,"num_read_kb":924,"num_write":284,"num_write_kb":2430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":200,"ondisk_log_size":200,"up":210,"acting":210,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":210,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":279212,"kb_used_data":9760,"kb_used_omap":13,"kb_used_meta":269298,"kb_avail":754695508,"statfs":{"total":773094113280,"available":772808200192,"internally_reserved":0,"allocated":9994240,"data_stored":5853050,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":14248,"internal_metadata":275761240},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":33,"apply_latency_ms":33,"commit_latency_ns":33000000,"apply_latency_ns":33000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"9.031732"},"pg_stats":[{"pgid":"11.3f","version":"0'0","reported_seq":11,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-08-09T13:50:30.411278+0000","last_change":"2023-08-09T13:50:29.233104+0000","last_active":"2023-08-09T13:50:30.411278+0000","last_peered":"2023-08-09T13:50:30.411278+0000","last_clean":"2023-08-09T13:50:30.411278+0000","last_became_active":"2023-08-09T13:50:29.232795+0000","last_became_peered":"2023-08-09T13:50:29.232795+0000","last_unstale":"2023-08-09T13:50:30.411278+0000","last_undegraded":"2023-08-09T13:50:30.411278+0000","last_fullsized":"2023-08-09T13:50:30.411278+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:50:28.150437+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:50:28.150437+0000","last_clean_scrub_stamp":"2023-08-09T13:50:28.150437+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T13:51:32.747391+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"11.3e","version":"0'0","reported_seq":11,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-08-09T13:50:30.409928+0000","last_change":"2023-08-09T13:50:29.253232+0000","last_active":"2023-08-09T13:50:30.409928+0000","last_peered":"2023-08-09T13:50:30.409928+0000","last_clean":"2023-08-09T13:50:30.409928+0000","last_became_active":"2023-08-09T13:50:29.252404+0000","last_became_peered":"2023-08-09T13:50:29.252404+0000","last_unstale":"2023-08-09T13:50:30.409928+0000","last_undegraded":"2023-08-09T13:50:30.409928+0000","last_fullsized":"2023-08-09T13:50:30.409928+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:50:28.150437+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:50:28.150437+0000","last_clean_scrub_stamp":"2023-08-09T13:50:28.150437+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-11T00:10:29.921271+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.3d","version":"0'0","reported_seq":11,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-08-09T13:50:30.209034+0000","last_change":"2023-08-09T13:50:29.225958+0000","last_active":"2023-08-09T13:50:30.209034+0000","last_peered":"2023-08-09T13:50:30.209034+0000","last_clean":"2023-08-09T13:50:30.209034+0000","last_became_active":"2023-08-09T13:50:29.225536+0000","last_became_peered":"2023-08-09T13:50:29.225536+0000","last_unstale":"2023-08-09T13:50:30.209034+0000","last_undegraded":"2023-08-09T13:50:30.209034+0000","last_fullsized":"2023-08-09T13:50:30.209034+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:50:28.150437+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:50:28.150437+0000","last_clean_scrub_stamp":"2023-08-09T13:50:28.150437+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T21:34:33.303902+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.3c","version":"0'0","reported_seq":11,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-08-09T13:50:30.207338+0000","last_change":"2023-08-09T13:50:29.257870+0000","last_active":"2023-08-09T13:50:30.207338+0000","last_peered":"2023-08-09T13:50:30.207338+0000","last_clean":"2023-08-09T13:50:30.207338+0000","last_became_active":"2023-08-09T13:50:29.257284+0000","last_became_peered":"2023-08-09T13:50:29.257284+0000","last_unstale":"2023-08-09T13:50:30.207338+0000","last_undegraded":"2023-08-09T13:50:30.207338+0000","last_fullsized":"2023-08-09T13:50:30.207338+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:50:28.150437+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:50:28.150437+0000","last_clean_scrub_stamp":"2023-08-09T13:50:28.150437+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T19:35:43.990628+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.3b","version":"0'0","reported_seq":11,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-08-09T13:50:30.406934+0000","last_change":"2023-08-09T13:50:29.241464+0000","last_active":"2023-08-09T13:50:30.406934+0000","last_peered":"2023-08-09T13:50:30.406934+0000","last_clean":"2023-08-09T13:50:30.406934+0000","last_became_active":"2023-08-09T13:50:29.241153+0000","last_became_peered":"2023-08-09T13:50:29.241153+0000","last_unstale":"2023-08-09T13:50:30.406934+0000","last_undegraded":"2023-08-09T13:50:30.406934+0000","last_fullsized":"2023-08-09T13:50:30.406934+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:50:28.150437+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:50:28.150437+0000","last_clean_scrub_stamp":"2023-08-09T13:50:28.150437+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-11T01:37:34.785410+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.3a","version":"0'0","reported_seq":11,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-08-09T13:50:30.407807+0000","last_change":"2023-08-09T13:50:29.250572+0000","last_active":"2023-08-09T13:50:30.407807+0000","last_peered":"2023-08-09T13:50:30.407807+0000","last_clean":"2023-08-09T13:50:30.407807+0000","last_became_active":"2023-08-09T13:50:29.250267+0000","last_became_peered":"2023-08-09T13:50:29.250267+0000","last_unstale":"2023-08-09T13:50:30.407807+0000","last_undegraded":"2023-08-09T13:50:30.407807+0000","last_fullsized":"2023-08-09T13:50:30.407807+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:50:28.150437+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:50:28.150437+0000","last_clean_scrub_stamp":"2023-08-09T13:50:28.150437+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-11T00:49:00.074020+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.39","version":"0'0","reported_seq":11,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-08-09T13:50:30.406242+0000","last_change":"2023-08-09T13:50:29.239702+0000","last_active":"2023-08-09T13:50:30.406242+0000","last_peered":"2023-08-09T13:50:30.406242+0000","last_clean":"2023-08-09T13:50:30.406242+0000","last_became_active":"2023-08-09T13:50:29.239438+0000","last_became_peered":"2023-08-09T13:50:29.239438+0000","last_unstale":"2023-08-09T13:50:30.406242+0000","last_undegraded":"2023-08-09T13:50:30.406242+0000","last_fullsized":"2023-08-09T13:50:30.406242+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:50:28.150437+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:50:28.150437+0000","last_clean_scrub_stamp":"2023-08-09T13:50:28.150437+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T22:56:16.830778+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.38","version":"0'0","reported_seq":11,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-08-09T13:50:30.408161+0000","last_change":"2023-08-09T13:50:29.246308+0000","last_active":"2023-08-09T13:50:30.408161+0000","last_peered":"2023-08-09T13:50:30.408161+0000","last_clean":"2023-08-09T13:50:30.408161+0000","last_became_active":"2023-08-09T13:50:29.245268+0000","last_became_peered":"2023-08-09T13:50:29.245268+0000","last_unstale":"2023-08-09T13:50:30.408161+0000","last_undegraded":"2023-08-09T13:50:30.408161+0000","last_fullsized":"2023-08-09T13:50:30.408161+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:50:28.150437+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:50:28.150437+0000","last_clean_scrub_stamp":"2023-08-09T13:50:28.150437+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T15:41:47.049357+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.37","version":"0'0","reported_seq":11,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-08-09T13:50:30.406492+0000","last_change":"2023-08-09T13:50:29.238754+0000","last_active":"2023-08-09T13:50:30.406492+0000","last_peered":"2023-08-09T13:50:30.406492+0000","last_clean":"2023-08-09T13:50:30.406492+0000","last_became_active":"2023-08-09T13:50:29.238398+0000","last_became_peered":"2023-08-09T13:50:29.238398+0000","last_unstale":"2023-08-09T13:50:30.406492+0000","last_undegraded":"2023-08-09T13:50:30.406492+0000","last_fullsized":"2023-08-09T13:50:30.406492+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:50:28.150437+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:50:28.150437+0000","last_clean_scrub_stamp":"2023-08-09T13:50:28.150437+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T23:55:59.694448+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.36","version":"0'0","reported_seq":11,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-08-09T13:50:30.209480+0000","last_change":"2023-08-09T13:50:29.242771+0000","last_active":"2023-08-09T13:50:30.209480+0000","last_peered":"2023-08-09T13:50:30.209480+0000","last_clean":"2023-08-09T13:50:30.209480+0000","last_became_active":"2023-08-09T13:50:29.242499+0000","last_became_peered":"2023-08-09T13:50:29.242499+0000","last_unstale":"2023-08-09T13:50:30.209480+0000","last_undegraded":"2023-08-09T13:50:30.209480+0000","last_fullsized":"2023-08-09T13:50:30.209480+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:50:28.150437+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:50:28.150437+0000","last_clean_scrub_stamp":"2023-08-09T13:50:28.150437+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T20:55:02.191154+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.35","version":"0'0","reported_seq":11,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-08-09T13:50:30.410412+0000","last_change":"2023-08-09T13:50:29.237075+0000","last_active":"2023-08-09T13:50:30.410412+0000","last_peered":"2023-08-09T13:50:30.410412+0000","last_clean":"2023-08-09T13:50:30.410412+0000","last_became_active":"2023-08-09T13:50:29.236351+0000","last_became_peered":"2023-08-09T13:50:29.236351+0000","last_unstale":"2023-08-09T13:50:30.410412+0000","last_undegraded":"2023-08-09T13:50:30.410412+0000","last_fullsized":"2023-08-09T13:50:30.410412+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:50:28.150437+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:50:28.150437+0000","last_clean_scrub_stamp":"2023-08-09T13:50:28.150437+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T14:28:39.370278+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"11.34","version":"0'0","reported_seq":11,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-08-09T13:50:30.207548+0000","last_change":"2023-08-09T13:50:29.234197+0000","last_active":"2023-08-09T13:50:30.207548+0000","last_peered":"2023-08-09T13:50:30.207548+0000","last_clean":"2023-08-09T13:50:30.207548+0000","last_became_active":"2023-08-09T13:50:29.233877+0000","last_became_peered":"2023-08-09T13:50:29.233877+0000","last_unstale":"2023-08-09T13:50:30.207548+0000","last_undegraded":"2023-08-09T13:50:30.207548+0000","last_fullsized":"2023-08-09T13:50:30.207548+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:50:28.150437+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:50:28.150437+0000","last_clean_scrub_stamp":"2023-08-09T13:50:28.150437+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T21:16:13.484480+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.33","version":"0'0","reported_seq":11,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-08-09T13:50:30.405888+0000","last_change":"2023-08-09T13:50:29.243093+0000","last_active":"2023-08-09T13:50:30.405888+0000","last_peered":"2023-08-09T13:50:30.405888+0000","last_clean":"2023-08-09T13:50:30.405888+0000","last_became_active":"2023-08-09T13:50:29.242740+0000","last_became_peered":"2023-08-09T13:50:29.242740+0000","last_unstale":"2023-08-09T13:50:30.405888+0000","last_undegraded":"2023-08-09T13:50:30.405888+0000","last_fullsized":"2023-08-09T13:50:30.405888+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:50:28.150437+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:50:28.150437+0000","last_clean_scrub_stamp":"2023-08-09T13:50:28.150437+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T15:42:16.315503+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.32","version":"0'0","reported_seq":11,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-08-09T13:50:30.406451+0000","last_change":"2023-08-09T13:50:29.240731+0000","last_active":"2023-08-09T13:50:30.406451+0000","last_peered":"2023-08-09T13:50:30.406451+0000","last_clean":"2023-08-09T13:50:30.406451+0000","last_became_active":"2023-08-09T13:50:29.240397+0000","last_became_peered":"2023-08-09T13:50:29.240397+0000","last_unstale":"2023-08-09T13:50:30.406451+0000","last_undegraded":"2023-08-09T13:50:30.406451+0000","last_fullsized":"2023-08-09T13:50:30.406451+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:50:28.150437+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:50:28.150437+0000","last_clean_scrub_stamp":"2023-08-09T13:50:28.150437+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T20:28:47.626509+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.31","version":"0'0","reported_seq":11,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-08-09T13:50:30.405358+0000","last_change":"2023-08-09T13:50:29.231343+0000","last_active":"2023-08-09T13:50:30.405358+0000","last_peered":"2023-08-09T13:50:30.405358+0000","last_clean":"2023-08-09T13:50:30.405358+0000","last_became_active":"2023-08-09T13:50:29.230980+0000","last_became_peered":"2023-08-09T13:50:29.230980+0000","last_unstale":"2023-08-09T13:50:30.405358+0000","last_undegraded":"2023-08-09T13:50:30.405358+0000","last_fullsized":"2023-08-09T13:50:30.405358+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:50:28.150437+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:50:28.150437+0000","last_clean_scrub_stamp":"2023-08-09T13:50:28.150437+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T23:54:43.168168+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.30","version":"0'0","reported_seq":11,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-08-09T13:50:30.208641+0000","last_change":"2023-08-09T13:50:29.251193+0000","last_active":"2023-08-09T13:50:30.208641+0000","last_peered":"2023-08-09T13:50:30.208641+0000","last_clean":"2023-08-09T13:50:30.208641+0000","last_became_active":"2023-08-09T13:50:29.250899+0000","last_became_peered":"2023-08-09T13:50:29.250899+0000","last_unstale":"2023-08-09T13:50:30.208641+0000","last_undegraded":"2023-08-09T13:50:30.208641+0000","last_fullsized":"2023-08-09T13:50:30.208641+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:50:28.150437+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:50:28.150437+0000","last_clean_scrub_stamp":"2023-08-09T13:50:28.150437+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T21:19:32.729587+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.2f","version":"0'0","reported_seq":11,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-08-09T13:50:30.406621+0000","last_change":"2023-08-09T13:50:29.238750+0000","last_active":"2023-08-09T13:50:30.406621+0000","last_peered":"2023-08-09T13:50:30.406621+0000","last_clean":"2023-08-09T13:50:30.406621+0000","last_became_active":"2023-08-09T13:50:29.238157+0000","last_became_peered":"2023-08-09T13:50:29.238157+0000","last_unstale":"2023-08-09T13:50:30.406621+0000","last_undegraded":"2023-08-09T13:50:30.406621+0000","last_fullsized":"2023-08-09T13:50:30.406621+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:50:28.150437+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:50:28.150437+0000","last_clean_scrub_stamp":"2023-08-09T13:50:28.150437+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T22:29:42.554110+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.2e","version":"0'0","reported_seq":11,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-08-09T13:50:30.208026+0000","last_change":"2023-08-09T13:50:29.244582+0000","last_active":"2023-08-09T13:50:30.208026+0000","last_peered":"2023-08-09T13:50:30.208026+0000","last_clean":"2023-08-09T13:50:30.208026+0000","last_became_active":"2023-08-09T13:50:29.244222+0000","last_became_peered":"2023-08-09T13:50:29.244222+0000","last_unstale":"2023-08-09T13:50:30.208026+0000","last_undegraded":"2023-08-09T13:50:30.208026+0000","last_fullsized":"2023-08-09T13:50:30.208026+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:50:28.150437+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:50:28.150437+0000","last_clean_scrub_stamp":"2023-08-09T13:50:28.150437+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-11T00:06:51.620725+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.2d","version":"0'0","reported_seq":11,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-08-09T13:50:30.407993+0000","last_change":"2023-08-09T13:50:29.241875+0000","last_active":"2023-08-09T13:50:30.407993+0000","last_peered":"2023-08-09T13:50:30.407993+0000","last_clean":"2023-08-09T13:50:30.407993+0000","last_became_active":"2023-08-09T13:50:29.241532+0000","last_became_peered":"2023-08-09T13:50:29.241532+0000","last_unstale":"2023-08-09T13:50:30.407993+0000","last_undegraded":"2023-08-09T13:50:30.407993+0000","last_fullsized":"2023-08-09T13:50:30.407993+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:50:28.150437+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:50:28.150437+0000","last_clean_scrub_stamp":"2023-08-09T13:50:28.150437+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T21:10:30.083848+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.2c","version":"0'0","reported_seq":11,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-08-09T13:50:30.406101+0000","last_change":"2023-08-09T13:50:29.231091+0000","last_active":"2023-08-09T13:50:30.406101+0000","last_peered":"2023-08-09T13:50:30.406101+0000","last_clean":"2023-08-09T13:50:30.406101+0000","last_became_active":"2023-08-09T13:50:29.230501+0000","last_became_peered":"2023-08-09T13:50:29.230501+0000","last_unstale":"2023-08-09T13:50:30.406101+0000","last_undegraded":"2023-08-09T13:50:30.406101+0000","last_fullsized":"2023-08-09T13:50:30.406101+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:50:28.150437+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:50:28.150437+0000","last_clean_scrub_stamp":"2023-08-09T13:50:28.150437+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-11T01:19:18.466874+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.2b","version":"0'0","reported_seq":11,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-08-09T13:50:30.209481+0000","last_change":"2023-08-09T13:50:29.227706+0000","last_active":"2023-08-09T13:50:30.209481+0000","last_peered":"2023-08-09T13:50:30.209481+0000","last_clean":"2023-08-09T13:50:30.209481+0000","last_became_active":"2023-08-09T13:50:29.226933+0000","last_became_peered":"2023-08-09T13:50:29.226933+0000","last_unstale":"2023-08-09T13:50:30.209481+0000","last_undegraded":"2023-08-09T13:50:30.209481+0000","last_fullsized":"2023-08-09T13:50:30.209481+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:50:28.150437+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:50:28.150437+0000","last_clean_scrub_stamp":"2023-08-09T13:50:28.150437+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T19:26:32.726448+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.2a","version":"0'0","reported_seq":11,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-08-09T13:50:30.209098+0000","last_change":"2023-08-09T13:50:29.243402+0000","last_active":"2023-08-09T13:50:30.209098+0000","last_peered":"2023-08-09T13:50:30.209098+0000","last_clean":"2023-08-09T13:50:30.209098+0000","last_became_active":"2023-08-09T13:50:29.243059+0000","last_became_peered":"2023-08-09T13:50:29.243059+0000","last_unstale":"2023-08-09T13:50:30.209098+0000","last_undegraded":"2023-08-09T13:50:30.209098+0000","last_fullsized":"2023-08-09T13:50:30.209098+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:50:28.150437+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:50:28.150437+0000","last_clean_scrub_stamp":"2023-08-09T13:50:28.150437+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T22:59:20.650038+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.29","version":"0'0","reported_seq":11,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-08-09T13:50:30.409359+0000","last_change":"2023-08-09T13:50:29.232548+0000","last_active":"2023-08-09T13:50:30.409359+0000","last_peered":"2023-08-09T13:50:30.409359+0000","last_clean":"2023-08-09T13:50:30.409359+0000","last_became_active":"2023-08-09T13:50:29.232205+0000","last_became_peered":"2023-08-09T13:50:29.232205+0000","last_unstale":"2023-08-09T13:50:30.409359+0000","last_undegraded":"2023-08-09T13:50:30.409359+0000","last_fullsized":"2023-08-09T13:50:30.409359+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:50:28.150437+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:50:28.150437+0000","last_clean_scrub_stamp":"2023-08-09T13:50:28.150437+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T18:15:28.473776+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"11.28","version":"0'0","reported_seq":11,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-08-09T13:50:30.406777+0000","last_change":"2023-08-09T13:50:29.258070+0000","last_active":"2023-08-09T13:50:30.406777+0000","last_peered":"2023-08-09T13:50:30.406777+0000","last_clean":"2023-08-09T13:50:30.406777+0000","last_became_active":"2023-08-09T13:50:29.257575+0000","last_became_peered":"2023-08-09T13:50:29.257575+0000","last_unstale":"2023-08-09T13:50:30.406777+0000","last_undegraded":"2023-08-09T13:50:30.406777+0000","last_fullsized":"2023-08-09T13:50:30.406777+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:50:28.150437+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:50:28.150437+0000","last_clean_scrub_stamp":"2023-08-09T13:50:28.150437+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T23:10:54.433079+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"11.27","version":"0'0","reported_seq":11,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-08-09T13:50:30.210329+0000","last_change":"2023-08-09T13:50:29.243408+0000","last_active":"2023-08-09T13:50:30.210329+0000","last_peered":"2023-08-09T13:50:30.210329+0000","last_clean":"2023-08-09T13:50:30.210329+0000","last_became_active":"2023-08-09T13:50:29.243094+0000","last_became_peered":"2023-08-09T13:50:29.243094+0000","last_unstale":"2023-08-09T13:50:30.210329+0000","last_undegraded":"2023-08-09T13:50:30.210329+0000","last_fullsized":"2023-08-09T13:50:30.210329+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:50:28.150437+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:50:28.150437+0000","last_clean_scrub_stamp":"2023-08-09T13:50:28.150437+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T15:57:46.494988+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.26","version":"0'0","reported_seq":11,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-08-09T13:50:30.407285+0000","last_change":"2023-08-09T13:50:29.256544+0000","last_active":"2023-08-09T13:50:30.407285+0000","last_peered":"2023-08-09T13:50:30.407285+0000","last_clean":"2023-08-09T13:50:30.407285+0000","last_became_active":"2023-08-09T13:50:29.254529+0000","last_became_peered":"2023-08-09T13:50:29.254529+0000","last_unstale":"2023-08-09T13:50:30.407285+0000","last_undegraded":"2023-08-09T13:50:30.407285+0000","last_fullsized":"2023-08-09T13:50:30.407285+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:50:28.150437+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:50:28.150437+0000","last_clean_scrub_stamp":"2023-08-09T13:50:28.150437+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-11T01:18:20.363025+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.25","version":"0'0","reported_seq":11,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-08-09T13:50:30.209087+0000","last_change":"2023-08-09T13:50:29.233554+0000","last_active":"2023-08-09T13:50:30.209087+0000","last_peered":"2023-08-09T13:50:30.209087+0000","last_clean":"2023-08-09T13:50:30.209087+0000","last_became_active":"2023-08-09T13:50:29.233153+0000","last_became_peered":"2023-08-09T13:50:29.233153+0000","last_unstale":"2023-08-09T13:50:30.209087+0000","last_undegraded":"2023-08-09T13:50:30.209087+0000","last_fullsized":"2023-08-09T13:50:30.209087+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:50:28.150437+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:50:28.150437+0000","last_clean_scrub_stamp":"2023-08-09T13:50:28.150437+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T15:52:51.777375+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.24","version":"0'0","reported_seq":11,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-08-09T13:50:30.209564+0000","last_change":"2023-08-09T13:50:29.235136+0000","last_active":"2023-08-09T13:50:30.209564+0000","last_peered":"2023-08-09T13:50:30.209564+0000","last_clean":"2023-08-09T13:50:30.209564+0000","last_became_active":"2023-08-09T13:50:29.234431+0000","last_became_peered":"2023-08-09T13:50:29.234431+0000","last_unstale":"2023-08-09T13:50:30.209564+0000","last_undegraded":"2023-08-09T13:50:30.209564+0000","last_fullsized":"2023-08-09T13:50:30.209564+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:50:28.150437+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:50:28.150437+0000","last_clean_scrub_stamp":"2023-08-09T13:50:28.150437+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T20:21:37.756099+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.23","version":"0'0","reported_seq":11,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-08-09T13:50:30.208586+0000","last_change":"2023-08-09T13:50:29.233513+0000","last_active":"2023-08-09T13:50:30.208586+0000","last_peered":"2023-08-09T13:50:30.208586+0000","last_clean":"2023-08-09T13:50:30.208586+0000","last_became_active":"2023-08-09T13:50:29.233199+0000","last_became_peered":"2023-08-09T13:50:29.233199+0000","last_unstale":"2023-08-09T13:50:30.208586+0000","last_undegraded":"2023-08-09T13:50:30.208586+0000","last_fullsized":"2023-08-09T13:50:30.208586+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:50:28.150437+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:50:28.150437+0000","last_clean_scrub_stamp":"2023-08-09T13:50:28.150437+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T14:16:12.214664+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.22","version":"0'0","reported_seq":11,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-08-09T13:50:30.406734+0000","last_change":"2023-08-09T13:50:29.254874+0000","last_active":"2023-08-09T13:50:30.406734+0000","last_peered":"2023-08-09T13:50:30.406734+0000","last_clean":"2023-08-09T13:50:30.406734+0000","last_became_active":"2023-08-09T13:50:29.254558+0000","last_became_peered":"2023-08-09T13:50:29.254558+0000","last_unstale":"2023-08-09T13:50:30.406734+0000","last_undegraded":"2023-08-09T13:50:30.406734+0000","last_fullsized":"2023-08-09T13:50:30.406734+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:50:28.150437+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:50:28.150437+0000","last_clean_scrub_stamp":"2023-08-09T13:50:28.150437+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T18:13:21.364990+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.21","version":"0'0","reported_seq":11,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-08-09T13:50:30.406701+0000","last_change":"2023-08-09T13:50:29.233755+0000","last_active":"2023-08-09T13:50:30.406701+0000","last_peered":"2023-08-09T13:50:30.406701+0000","last_clean":"2023-08-09T13:50:30.406701+0000","last_became_active":"2023-08-09T13:50:29.233150+0000","last_became_peered":"2023-08-09T13:50:29.233150+0000","last_unstale":"2023-08-09T13:50:30.406701+0000","last_undegraded":"2023-08-09T13:50:30.406701+0000","last_fullsized":"2023-08-09T13:50:30.406701+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:50:28.150437+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:50:28.150437+0000","last_clean_scrub_stamp":"2023-08-09T13:50:28.150437+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T21:39:39.502906+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.20","version":"0'0","reported_seq":11,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-08-09T13:50:30.408563+0000","last_change":"2023-08-09T13:50:29.245742+0000","last_active":"2023-08-09T13:50:30.408563+0000","last_peered":"2023-08-09T13:50:30.408563+0000","last_clean":"2023-08-09T13:50:30.408563+0000","last_became_active":"2023-08-09T13:50:29.242577+0000","last_became_peered":"2023-08-09T13:50:29.242577+0000","last_unstale":"2023-08-09T13:50:30.408563+0000","last_undegraded":"2023-08-09T13:50:30.408563+0000","last_fullsized":"2023-08-09T13:50:30.408563+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:50:28.150437+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:50:28.150437+0000","last_clean_scrub_stamp":"2023-08-09T13:50:28.150437+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T13:52:49.951981+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.1e","version":"0'0","reported_seq":11,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-08-09T13:50:30.411138+0000","last_change":"2023-08-09T13:50:29.245053+0000","last_active":"2023-08-09T13:50:30.411138+0000","last_peered":"2023-08-09T13:50:30.411138+0000","last_clean":"2023-08-09T13:50:30.411138+0000","last_became_active":"2023-08-09T13:50:29.244730+0000","last_became_peered":"2023-08-09T13:50:29.244730+0000","last_unstale":"2023-08-09T13:50:30.411138+0000","last_undegraded":"2023-08-09T13:50:30.411138+0000","last_fullsized":"2023-08-09T13:50:30.411138+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:50:28.150437+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:50:28.150437+0000","last_clean_scrub_stamp":"2023-08-09T13:50:28.150437+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T17:00:59.064421+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.1f","version":"89'4","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-08-09T13:50:30.411644+0000","last_change":"2023-08-09T13:50:28.198750+0000","last_active":"2023-08-09T13:50:30.411644+0000","last_peered":"2023-08-09T13:50:30.411644+0000","last_clean":"2023-08-09T13:50:30.411644+0000","last_became_active":"2023-08-09T13:50:28.198375+0000","last_became_peered":"2023-08-09T13:50:28.198375+0000","last_unstale":"2023-08-09T13:50:30.411644+0000","last_undegraded":"2023-08-09T13:50:30.411644+0000","last_fullsized":"2023-08-09T13:50:30.411644+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:50:27.140775+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:50:27.140775+0000","last_clean_scrub_stamp":"2023-08-09T13:50:27.140775+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T21:12:07.227144+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"11.1f","version":"0'0","reported_seq":11,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-08-09T13:50:30.406744+0000","last_change":"2023-08-09T13:50:29.235523+0000","last_active":"2023-08-09T13:50:30.406744+0000","last_peered":"2023-08-09T13:50:30.406744+0000","last_clean":"2023-08-09T13:50:30.406744+0000","last_became_active":"2023-08-09T13:50:29.235072+0000","last_became_peered":"2023-08-09T13:50:29.235072+0000","last_unstale":"2023-08-09T13:50:30.406744+0000","last_undegraded":"2023-08-09T13:50:30.406744+0000","last_fullsized":"2023-08-09T13:50:30.406744+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:50:28.150437+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:50:28.150437+0000","last_clean_scrub_stamp":"2023-08-09T13:50:28.150437+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T20:10:46.636697+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.1e","version":"89'1","reported_seq":14,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-08-09T13:50:30.407030+0000","last_change":"2023-08-09T13:50:28.189590+0000","last_active":"2023-08-09T13:50:30.407030+0000","last_peered":"2023-08-09T13:50:30.407030+0000","last_clean":"2023-08-09T13:50:30.407030+0000","last_became_active":"2023-08-09T13:50:28.189227+0000","last_became_peered":"2023-08-09T13:50:28.189227+0000","last_unstale":"2023-08-09T13:50:30.407030+0000","last_undegraded":"2023-08-09T13:50:30.407030+0000","last_fullsized":"2023-08-09T13:50:30.407030+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:50:27.140775+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:50:27.140775+0000","last_clean_scrub_stamp":"2023-08-09T13:50:27.140775+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T18:15:27.464113+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.1c","version":"0'0","reported_seq":11,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-08-09T13:50:30.408194+0000","last_change":"2023-08-09T13:50:29.251777+0000","last_active":"2023-08-09T13:50:30.408194+0000","last_peered":"2023-08-09T13:50:30.408194+0000","last_clean":"2023-08-09T13:50:30.408194+0000","last_became_active":"2023-08-09T13:50:29.251397+0000","last_became_peered":"2023-08-09T13:50:29.251397+0000","last_unstale":"2023-08-09T13:50:30.408194+0000","last_undegraded":"2023-08-09T13:50:30.408194+0000","last_fullsized":"2023-08-09T13:50:30.408194+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:50:28.150437+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:50:28.150437+0000","last_clean_scrub_stamp":"2023-08-09T13:50:28.150437+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T18:59:29.531912+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"10.1d","version":"89'2","reported_seq":15,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-08-09T13:50:30.210072+0000","last_change":"2023-08-09T13:50:28.206216+0000","last_active":"2023-08-09T13:50:30.210072+0000","last_peered":"2023-08-09T13:50:30.210072+0000","last_clean":"2023-08-09T13:50:30.210072+0000","last_became_active":"2023-08-09T13:50:28.205914+0000","last_became_peered":"2023-08-09T13:50:28.205914+0000","last_unstale":"2023-08-09T13:50:30.210072+0000","last_undegraded":"2023-08-09T13:50:30.210072+0000","last_fullsized":"2023-08-09T13:50:30.210072+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:50:27.140775+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:50:27.140775+0000","last_clean_scrub_stamp":"2023-08-09T13:50:27.140775+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-11T00:48:03.464197+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.1d","version":"0'0","reported_seq":11,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-08-09T13:50:30.410591+0000","last_change":"2023-08-09T13:50:29.231490+0000","last_active":"2023-08-09T13:50:30.410591+0000","last_peered":"2023-08-09T13:50:30.410591+0000","last_clean":"2023-08-09T13:50:30.410591+0000","last_became_active":"2023-08-09T13:50:29.231190+0000","last_became_peered":"2023-08-09T13:50:29.231190+0000","last_unstale":"2023-08-09T13:50:30.410591+0000","last_undegraded":"2023-08-09T13:50:30.410591+0000","last_fullsized":"2023-08-09T13:50:30.410591+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:50:28.150437+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:50:28.150437+0000","last_clean_scrub_stamp":"2023-08-09T13:50:28.150437+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T23:44:14.402860+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.1c","version":"89'2","reported_seq":15,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-08-09T13:50:30.408443+0000","last_change":"2023-08-09T13:50:28.195729+0000","last_active":"2023-08-09T13:50:30.408443+0000","last_peered":"2023-08-09T13:50:30.408443+0000","last_clean":"2023-08-09T13:50:30.408443+0000","last_became_active":"2023-08-09T13:50:28.195433+0000","last_became_peered":"2023-08-09T13:50:28.195433+0000","last_unstale":"2023-08-09T13:50:30.408443+0000","last_undegraded":"2023-08-09T13:50:30.408443+0000","last_fullsized":"2023-08-09T13:50:30.408443+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:50:27.140775+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:50:27.140775+0000","last_clean_scrub_stamp":"2023-08-09T13:50:27.140775+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-11T01:11:15.011327+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.1a","version":"0'0","reported_seq":11,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-08-09T13:50:30.406904+0000","last_change":"2023-08-09T13:50:29.234000+0000","last_active":"2023-08-09T13:50:30.406904+0000","last_peered":"2023-08-09T13:50:30.406904+0000","last_clean":"2023-08-09T13:50:30.406904+0000","last_became_active":"2023-08-09T13:50:29.233727+0000","last_became_peered":"2023-08-09T13:50:29.233727+0000","last_unstale":"2023-08-09T13:50:30.406904+0000","last_undegraded":"2023-08-09T13:50:30.406904+0000","last_fullsized":"2023-08-09T13:50:30.406904+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:50:28.150437+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:50:28.150437+0000","last_clean_scrub_stamp":"2023-08-09T13:50:28.150437+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T16:52:43.848278+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.1b","version":"89'1","reported_seq":14,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-08-09T13:50:30.208893+0000","last_change":"2023-08-09T13:50:28.196698+0000","last_active":"2023-08-09T13:50:30.208893+0000","last_peered":"2023-08-09T13:50:30.208893+0000","last_clean":"2023-08-09T13:50:30.208893+0000","last_became_active":"2023-08-09T13:50:28.196362+0000","last_became_peered":"2023-08-09T13:50:28.196362+0000","last_unstale":"2023-08-09T13:50:30.208893+0000","last_undegraded":"2023-08-09T13:50:30.208893+0000","last_fullsized":"2023-08-09T13:50:30.208893+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:50:27.140775+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:50:27.140775+0000","last_clean_scrub_stamp":"2023-08-09T13:50:27.140775+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T18:07:37.314159+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.1b","version":"0'0","reported_seq":11,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-08-09T13:50:30.209540+0000","last_change":"2023-08-09T13:50:29.243408+0000","last_active":"2023-08-09T13:50:30.209540+0000","last_peered":"2023-08-09T13:50:30.209540+0000","last_clean":"2023-08-09T13:50:30.209540+0000","last_became_active":"2023-08-09T13:50:29.243082+0000","last_became_peered":"2023-08-09T13:50:29.243082+0000","last_unstale":"2023-08-09T13:50:30.209540+0000","last_undegraded":"2023-08-09T13:50:30.209540+0000","last_fullsized":"2023-08-09T13:50:30.209540+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:50:28.150437+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:50:28.150437+0000","last_clean_scrub_stamp":"2023-08-09T13:50:28.150437+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T19:55:13.873621+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"10.1a","version":"89'1","reported_seq":14,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-08-09T13:50:30.407973+0000","last_change":"2023-08-09T13:50:28.205333+0000","last_active":"2023-08-09T13:50:30.407973+0000","last_peered":"2023-08-09T13:50:30.407973+0000","last_clean":"2023-08-09T13:50:30.407973+0000","last_became_active":"2023-08-09T13:50:28.204881+0000","last_became_peered":"2023-08-09T13:50:28.204881+0000","last_unstale":"2023-08-09T13:50:30.407973+0000","last_undegraded":"2023-08-09T13:50:30.407973+0000","last_fullsized":"2023-08-09T13:50:30.407973+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:50:27.140775+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:50:27.140775+0000","last_clean_scrub_stamp":"2023-08-09T13:50:27.140775+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-11T00:26:33.935584+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.18","version":"0'0","reported_seq":11,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-08-09T13:50:30.405579+0000","last_change":"2023-08-09T13:50:29.243428+0000","last_active":"2023-08-09T13:50:30.405579+0000","last_peered":"2023-08-09T13:50:30.405579+0000","last_clean":"2023-08-09T13:50:30.405579+0000","last_became_active":"2023-08-09T13:50:29.243134+0000","last_became_peered":"2023-08-09T13:50:29.243134+0000","last_unstale":"2023-08-09T13:50:30.405579+0000","last_undegraded":"2023-08-09T13:50:30.405579+0000","last_fullsized":"2023-08-09T13:50:30.405579+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:50:28.150437+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:50:28.150437+0000","last_clean_scrub_stamp":"2023-08-09T13:50:28.150437+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T18:19:56.894549+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.19","version":"0'0","reported_seq":13,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-08-09T13:50:30.207222+0000","last_change":"2023-08-09T13:50:28.204197+0000","last_active":"2023-08-09T13:50:30.207222+0000","last_peered":"2023-08-09T13:50:30.207222+0000","last_clean":"2023-08-09T13:50:30.207222+0000","last_became_active":"2023-08-09T13:50:28.203911+0000","last_became_peered":"2023-08-09T13:50:28.203911+0000","last_unstale":"2023-08-09T13:50:30.207222+0000","last_undegraded":"2023-08-09T13:50:30.207222+0000","last_fullsized":"2023-08-09T13:50:30.207222+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:50:27.140775+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:50:27.140775+0000","last_clean_scrub_stamp":"2023-08-09T13:50:27.140775+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T19:31:34.788085+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.19","version":"0'0","reported_seq":11,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-08-09T13:50:30.405358+0000","last_change":"2023-08-09T13:50:29.223620+0000","last_active":"2023-08-09T13:50:30.405358+0000","last_peered":"2023-08-09T13:50:30.405358+0000","last_clean":"2023-08-09T13:50:30.405358+0000","last_became_active":"2023-08-09T13:50:29.223063+0000","last_became_peered":"2023-08-09T13:50:29.223063+0000","last_unstale":"2023-08-09T13:50:30.405358+0000","last_undegraded":"2023-08-09T13:50:30.405358+0000","last_fullsized":"2023-08-09T13:50:30.405358+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:50:28.150437+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:50:28.150437+0000","last_clean_scrub_stamp":"2023-08-09T13:50:28.150437+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T16:41:09.645304+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.18","version":"0'0","reported_seq":13,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-08-09T13:50:30.405767+0000","last_change":"2023-08-09T13:50:28.189872+0000","last_active":"2023-08-09T13:50:30.405767+0000","last_peered":"2023-08-09T13:50:30.405767+0000","last_clean":"2023-08-09T13:50:30.405767+0000","last_became_active":"2023-08-09T13:50:28.189560+0000","last_became_peered":"2023-08-09T13:50:28.189560+0000","last_unstale":"2023-08-09T13:50:30.405767+0000","last_undegraded":"2023-08-09T13:50:30.405767+0000","last_fullsized":"2023-08-09T13:50:30.405767+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:50:27.140775+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:50:27.140775+0000","last_clean_scrub_stamp":"2023-08-09T13:50:27.140775+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T15:10:14.153697+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.16","version":"0'0","reported_seq":11,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-08-09T13:50:30.407495+0000","last_change":"2023-08-09T13:50:29.243980+0000","last_active":"2023-08-09T13:50:30.407495+0000","last_peered":"2023-08-09T13:50:30.407495+0000","last_clean":"2023-08-09T13:50:30.407495+0000","last_became_active":"2023-08-09T13:50:29.241148+0000","last_became_peered":"2023-08-09T13:50:29.241148+0000","last_unstale":"2023-08-09T13:50:30.407495+0000","last_undegraded":"2023-08-09T13:50:30.407495+0000","last_fullsized":"2023-08-09T13:50:30.407495+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:50:28.150437+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:50:28.150437+0000","last_clean_scrub_stamp":"2023-08-09T13:50:28.150437+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T16:15:06.185769+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.17","version":"0'0","reported_seq":13,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-08-09T13:50:30.210455+0000","last_change":"2023-08-09T13:50:28.198990+0000","last_active":"2023-08-09T13:50:30.210455+0000","last_peered":"2023-08-09T13:50:30.210455+0000","last_clean":"2023-08-09T13:50:30.210455+0000","last_became_active":"2023-08-09T13:50:28.197376+0000","last_became_peered":"2023-08-09T13:50:28.197376+0000","last_unstale":"2023-08-09T13:50:30.210455+0000","last_undegraded":"2023-08-09T13:50:30.210455+0000","last_fullsized":"2023-08-09T13:50:30.210455+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:50:27.140775+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:50:27.140775+0000","last_clean_scrub_stamp":"2023-08-09T13:50:27.140775+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T16:52:00.428747+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.17","version":"0'0","reported_seq":11,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-08-09T13:50:30.411432+0000","last_change":"2023-08-09T13:50:29.249548+0000","last_active":"2023-08-09T13:50:30.411432+0000","last_peered":"2023-08-09T13:50:30.411432+0000","last_clean":"2023-08-09T13:50:30.411432+0000","last_became_active":"2023-08-09T13:50:29.249264+0000","last_became_peered":"2023-08-09T13:50:29.249264+0000","last_unstale":"2023-08-09T13:50:30.411432+0000","last_undegraded":"2023-08-09T13:50:30.411432+0000","last_fullsized":"2023-08-09T13:50:30.411432+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:50:28.150437+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:50:28.150437+0000","last_clean_scrub_stamp":"2023-08-09T13:50:28.150437+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-11T00:32:07.349821+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.16","version":"0'0","reported_seq":13,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-08-09T13:50:30.410005+0000","last_change":"2023-08-09T13:50:28.203893+0000","last_active":"2023-08-09T13:50:30.410005+0000","last_peered":"2023-08-09T13:50:30.410005+0000","last_clean":"2023-08-09T13:50:30.410005+0000","last_became_active":"2023-08-09T13:50:28.203596+0000","last_became_peered":"2023-08-09T13:50:28.203596+0000","last_unstale":"2023-08-09T13:50:30.410005+0000","last_undegraded":"2023-08-09T13:50:30.410005+0000","last_fullsized":"2023-08-09T13:50:30.410005+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:50:27.140775+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:50:27.140775+0000","last_clean_scrub_stamp":"2023-08-09T13:50:27.140775+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T16:51:47.071487+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.14","version":"0'0","reported_seq":11,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-08-09T13:50:30.408413+0000","last_change":"2023-08-09T13:50:29.236992+0000","last_active":"2023-08-09T13:50:30.408413+0000","last_peered":"2023-08-09T13:50:30.408413+0000","last_clean":"2023-08-09T13:50:30.408413+0000","last_became_active":"2023-08-09T13:50:29.236708+0000","last_became_peered":"2023-08-09T13:50:29.236708+0000","last_unstale":"2023-08-09T13:50:30.408413+0000","last_undegraded":"2023-08-09T13:50:30.408413+0000","last_fullsized":"2023-08-09T13:50:30.408413+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:50:28.150437+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:50:28.150437+0000","last_clean_scrub_stamp":"2023-08-09T13:50:28.150437+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T23:49:15.426996+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"10.15","version":"89'3","reported_seq":16,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-08-09T13:50:30.209183+0000","last_change":"2023-08-09T13:50:28.198818+0000","last_active":"2023-08-09T13:50:30.209183+0000","last_peered":"2023-08-09T13:50:30.209183+0000","last_clean":"2023-08-09T13:50:30.209183+0000","last_became_active":"2023-08-09T13:50:28.198282+0000","last_became_peered":"2023-08-09T13:50:28.198282+0000","last_unstale":"2023-08-09T13:50:30.209183+0000","last_undegraded":"2023-08-09T13:50:30.209183+0000","last_fullsized":"2023-08-09T13:50:30.209183+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:50:27.140775+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:50:27.140775+0000","last_clean_scrub_stamp":"2023-08-09T13:50:27.140775+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T14:19:14.202902+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.15","version":"0'0","reported_seq":11,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-08-09T13:50:30.406181+0000","last_change":"2023-08-09T13:50:29.232045+0000","last_active":"2023-08-09T13:50:30.406181+0000","last_peered":"2023-08-09T13:50:30.406181+0000","last_clean":"2023-08-09T13:50:30.406181+0000","last_became_active":"2023-08-09T13:50:29.231691+0000","last_became_peered":"2023-08-09T13:50:29.231691+0000","last_unstale":"2023-08-09T13:50:30.406181+0000","last_undegraded":"2023-08-09T13:50:30.406181+0000","last_fullsized":"2023-08-09T13:50:30.406181+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:50:28.150437+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:50:28.150437+0000","last_clean_scrub_stamp":"2023-08-09T13:50:28.150437+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T15:55:00.516307+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.14","version":"89'6","reported_seq":19,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-08-09T13:50:30.207724+0000","last_change":"2023-08-09T13:50:28.194890+0000","last_active":"2023-08-09T13:50:30.207724+0000","last_peered":"2023-08-09T13:50:30.207724+0000","last_clean":"2023-08-09T13:50:30.207724+0000","last_became_active":"2023-08-09T13:50:28.194607+0000","last_became_peered":"2023-08-09T13:50:28.194607+0000","last_unstale":"2023-08-09T13:50:30.207724+0000","last_undegraded":"2023-08-09T13:50:30.207724+0000","last_fullsized":"2023-08-09T13:50:30.207724+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:50:27.140775+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:50:27.140775+0000","last_clean_scrub_stamp":"2023-08-09T13:50:27.140775+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T19:14:35.323306+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.12","version":"0'0","reported_seq":11,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-08-09T13:50:30.409406+0000","last_change":"2023-08-09T13:50:29.233976+0000","last_active":"2023-08-09T13:50:30.409406+0000","last_peered":"2023-08-09T13:50:30.409406+0000","last_clean":"2023-08-09T13:50:30.409406+0000","last_became_active":"2023-08-09T13:50:29.233696+0000","last_became_peered":"2023-08-09T13:50:29.233696+0000","last_unstale":"2023-08-09T13:50:30.409406+0000","last_undegraded":"2023-08-09T13:50:30.409406+0000","last_fullsized":"2023-08-09T13:50:30.409406+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:50:28.150437+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:50:28.150437+0000","last_clean_scrub_stamp":"2023-08-09T13:50:28.150437+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T16:47:25.470644+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.13","version":"89'2","reported_seq":15,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-08-09T13:50:30.207465+0000","last_change":"2023-08-09T13:50:28.198586+0000","last_active":"2023-08-09T13:50:30.207465+0000","last_peered":"2023-08-09T13:50:30.207465+0000","last_clean":"2023-08-09T13:50:30.207465+0000","last_became_active":"2023-08-09T13:50:28.198247+0000","last_became_peered":"2023-08-09T13:50:28.198247+0000","last_unstale":"2023-08-09T13:50:30.207465+0000","last_undegraded":"2023-08-09T13:50:30.207465+0000","last_fullsized":"2023-08-09T13:50:30.207465+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:50:27.140775+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:50:27.140775+0000","last_clean_scrub_stamp":"2023-08-09T13:50:27.140775+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T21:31:44.778069+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.13","version":"0'0","reported_seq":11,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-08-09T13:50:30.407626+0000","last_change":"2023-08-09T13:50:29.236800+0000","last_active":"2023-08-09T13:50:30.407626+0000","last_peered":"2023-08-09T13:50:30.407626+0000","last_clean":"2023-08-09T13:50:30.407626+0000","last_became_active":"2023-08-09T13:50:29.236527+0000","last_became_peered":"2023-08-09T13:50:29.236527+0000","last_unstale":"2023-08-09T13:50:30.407626+0000","last_undegraded":"2023-08-09T13:50:30.407626+0000","last_fullsized":"2023-08-09T13:50:30.407626+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:50:28.150437+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:50:28.150437+0000","last_clean_scrub_stamp":"2023-08-09T13:50:28.150437+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T16:15:14.514651+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.12","version":"0'0","reported_seq":13,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-08-09T13:50:30.408010+0000","last_change":"2023-08-09T13:50:28.201822+0000","last_active":"2023-08-09T13:50:30.408010+0000","last_peered":"2023-08-09T13:50:30.408010+0000","last_clean":"2023-08-09T13:50:30.408010+0000","last_became_active":"2023-08-09T13:50:28.201187+0000","last_became_peered":"2023-08-09T13:50:28.201187+0000","last_unstale":"2023-08-09T13:50:30.408010+0000","last_undegraded":"2023-08-09T13:50:30.408010+0000","last_fullsized":"2023-08-09T13:50:30.408010+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:50:27.140775+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:50:27.140775+0000","last_clean_scrub_stamp":"2023-08-09T13:50:27.140775+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T14:10:08.301216+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.10","version":"0'0","reported_seq":11,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-08-09T13:50:30.405345+0000","last_change":"2023-08-09T13:50:29.246513+0000","last_active":"2023-08-09T13:50:30.405345+0000","last_peered":"2023-08-09T13:50:30.405345+0000","last_clean":"2023-08-09T13:50:30.405345+0000","last_became_active":"2023-08-09T13:50:29.246176+0000","last_became_peered":"2023-08-09T13:50:29.246176+0000","last_unstale":"2023-08-09T13:50:30.405345+0000","last_undegraded":"2023-08-09T13:50:30.405345+0000","last_fullsized":"2023-08-09T13:50:30.405345+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:50:28.150437+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:50:28.150437+0000","last_clean_scrub_stamp":"2023-08-09T13:50:28.150437+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T19:41:45.324636+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.11","version":"0'0","reported_seq":13,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-08-09T13:50:30.406453+0000","last_change":"2023-08-09T13:50:28.193484+0000","last_active":"2023-08-09T13:50:30.406453+0000","last_peered":"2023-08-09T13:50:30.406453+0000","last_clean":"2023-08-09T13:50:30.406453+0000","last_became_active":"2023-08-09T13:50:28.193098+0000","last_became_peered":"2023-08-09T13:50:28.193098+0000","last_unstale":"2023-08-09T13:50:30.406453+0000","last_undegraded":"2023-08-09T13:50:30.406453+0000","last_fullsized":"2023-08-09T13:50:30.406453+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:50:27.140775+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:50:27.140775+0000","last_clean_scrub_stamp":"2023-08-09T13:50:27.140775+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T22:25:23.284935+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.11","version":"0'0","reported_seq":11,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-08-09T13:50:30.407690+0000","last_change":"2023-08-09T13:50:29.238791+0000","last_active":"2023-08-09T13:50:30.407690+0000","last_peered":"2023-08-09T13:50:30.407690+0000","last_clean":"2023-08-09T13:50:30.407690+0000","last_became_active":"2023-08-09T13:50:29.238163+0000","last_became_peered":"2023-08-09T13:50:29.238163+0000","last_unstale":"2023-08-09T13:50:30.407690+0000","last_undegraded":"2023-08-09T13:50:30.407690+0000","last_fullsized":"2023-08-09T13:50:30.407690+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:50:28.150437+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:50:28.150437+0000","last_clean_scrub_stamp":"2023-08-09T13:50:28.150437+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T20:18:58.144613+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.10","version":"89'8","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-08-09T13:50:30.208893+0000","last_change":"2023-08-09T13:50:28.200670+0000","last_active":"2023-08-09T13:50:30.208893+0000","last_peered":"2023-08-09T13:50:30.208893+0000","last_clean":"2023-08-09T13:50:30.208893+0000","last_became_active":"2023-08-09T13:50:28.200419+0000","last_became_peered":"2023-08-09T13:50:28.200419+0000","last_unstale":"2023-08-09T13:50:30.208893+0000","last_undegraded":"2023-08-09T13:50:30.208893+0000","last_fullsized":"2023-08-09T13:50:30.208893+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:50:27.140775+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:50:27.140775+0000","last_clean_scrub_stamp":"2023-08-09T13:50:27.140775+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T20:20:11.928550+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.6","version":"0'0","reported_seq":11,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-08-09T13:50:30.406773+0000","last_change":"2023-08-09T13:50:29.229974+0000","last_active":"2023-08-09T13:50:30.406773+0000","last_peered":"2023-08-09T13:50:30.406773+0000","last_clean":"2023-08-09T13:50:30.406773+0000","last_became_active":"2023-08-09T13:50:29.229589+0000","last_became_peered":"2023-08-09T13:50:29.229589+0000","last_unstale":"2023-08-09T13:50:30.406773+0000","last_undegraded":"2023-08-09T13:50:30.406773+0000","last_fullsized":"2023-08-09T13:50:30.406773+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:50:28.150437+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:50:28.150437+0000","last_clean_scrub_stamp":"2023-08-09T13:50:28.150437+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T23:11:20.894653+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.7","version":"89'2","reported_seq":15,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-08-09T13:50:30.410925+0000","last_change":"2023-08-09T13:50:28.204446+0000","last_active":"2023-08-09T13:50:30.410925+0000","last_peered":"2023-08-09T13:50:30.410925+0000","last_clean":"2023-08-09T13:50:30.410925+0000","last_became_active":"2023-08-09T13:50:28.204093+0000","last_became_peered":"2023-08-09T13:50:28.204093+0000","last_unstale":"2023-08-09T13:50:30.410925+0000","last_undegraded":"2023-08-09T13:50:30.410925+0000","last_fullsized":"2023-08-09T13:50:30.410925+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:50:27.140775+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:50:27.140775+0000","last_clean_scrub_stamp":"2023-08-09T13:50:27.140775+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T13:57:35.751807+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.7","version":"0'0","reported_seq":11,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-08-09T13:50:30.409140+0000","last_change":"2023-08-09T13:50:29.238476+0000","last_active":"2023-08-09T13:50:30.409140+0000","last_peered":"2023-08-09T13:50:30.409140+0000","last_clean":"2023-08-09T13:50:30.409140+0000","last_became_active":"2023-08-09T13:50:29.238194+0000","last_became_peered":"2023-08-09T13:50:29.238194+0000","last_unstale":"2023-08-09T13:50:30.409140+0000","last_undegraded":"2023-08-09T13:50:30.409140+0000","last_fullsized":"2023-08-09T13:50:30.409140+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:50:28.150437+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:50:28.150437+0000","last_clean_scrub_stamp":"2023-08-09T13:50:28.150437+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T22:10:03.532985+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.6","version":"90'1","reported_seq":14,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-08-09T13:50:30.407725+0000","last_change":"2023-08-09T13:50:28.191955+0000","last_active":"2023-08-09T13:50:30.407725+0000","last_peered":"2023-08-09T13:50:30.407725+0000","last_clean":"2023-08-09T13:50:30.407725+0000","last_became_active":"2023-08-09T13:50:28.191264+0000","last_became_peered":"2023-08-09T13:50:28.191264+0000","last_unstale":"2023-08-09T13:50:30.407725+0000","last_undegraded":"2023-08-09T13:50:30.407725+0000","last_fullsized":"2023-08-09T13:50:30.407725+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:50:27.140775+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:50:27.140775+0000","last_clean_scrub_stamp":"2023-08-09T13:50:27.140775+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T19:33:57.823801+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.4","version":"0'0","reported_seq":11,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-08-09T13:50:30.409899+0000","last_change":"2023-08-09T13:50:29.262188+0000","last_active":"2023-08-09T13:50:30.409899+0000","last_peered":"2023-08-09T13:50:30.409899+0000","last_clean":"2023-08-09T13:50:30.409899+0000","last_became_active":"2023-08-09T13:50:29.261805+0000","last_became_peered":"2023-08-09T13:50:29.261805+0000","last_unstale":"2023-08-09T13:50:30.409899+0000","last_undegraded":"2023-08-09T13:50:30.409899+0000","last_fullsized":"2023-08-09T13:50:30.409899+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:50:28.150437+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:50:28.150437+0000","last_clean_scrub_stamp":"2023-08-09T13:50:28.150437+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-11T00:46:38.079458+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.5","version":"0'0","reported_seq":13,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-08-09T13:50:30.406043+0000","last_change":"2023-08-09T13:50:28.193912+0000","last_active":"2023-08-09T13:50:30.406043+0000","last_peered":"2023-08-09T13:50:30.406043+0000","last_clean":"2023-08-09T13:50:30.406043+0000","last_became_active":"2023-08-09T13:50:28.193602+0000","last_became_peered":"2023-08-09T13:50:28.193602+0000","last_unstale":"2023-08-09T13:50:30.406043+0000","last_undegraded":"2023-08-09T13:50:30.406043+0000","last_fullsized":"2023-08-09T13:50:30.406043+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:50:27.140775+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:50:27.140775+0000","last_clean_scrub_stamp":"2023-08-09T13:50:27.140775+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T21:14:26.795628+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.5","version":"0'0","reported_seq":11,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-08-09T13:50:30.410734+0000","last_change":"2023-08-09T13:50:29.237250+0000","last_active":"2023-08-09T13:50:30.410734+0000","last_peered":"2023-08-09T13:50:30.410734+0000","last_clean":"2023-08-09T13:50:30.410734+0000","last_became_active":"2023-08-09T13:50:29.236821+0000","last_became_peered":"2023-08-09T13:50:29.236821+0000","last_unstale":"2023-08-09T13:50:30.410734+0000","last_undegraded":"2023-08-09T13:50:30.410734+0000","last_fullsized":"2023-08-09T13:50:30.410734+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:50:28.150437+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:50:28.150437+0000","last_clean_scrub_stamp":"2023-08-09T13:50:28.150437+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T21:30:34.061849+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.4","version":"0'0","reported_seq":13,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-08-09T13:50:30.406255+0000","last_change":"2023-08-09T13:50:28.197934+0000","last_active":"2023-08-09T13:50:30.406255+0000","last_peered":"2023-08-09T13:50:30.406255+0000","last_clean":"2023-08-09T13:50:30.406255+0000","last_became_active":"2023-08-09T13:50:28.197639+0000","last_became_peered":"2023-08-09T13:50:28.197639+0000","last_unstale":"2023-08-09T13:50:30.406255+0000","last_undegraded":"2023-08-09T13:50:30.406255+0000","last_fullsized":"2023-08-09T13:50:30.406255+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:50:27.140775+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:50:27.140775+0000","last_clean_scrub_stamp":"2023-08-09T13:50:27.140775+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T23:10:53.423417+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.2","version":"0'0","reported_seq":11,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-08-09T13:50:30.406822+0000","last_change":"2023-08-09T13:50:29.235605+0000","last_active":"2023-08-09T13:50:30.406822+0000","last_peered":"2023-08-09T13:50:30.406822+0000","last_clean":"2023-08-09T13:50:30.406822+0000","last_became_active":"2023-08-09T13:50:29.235033+0000","last_became_peered":"2023-08-09T13:50:29.235033+0000","last_unstale":"2023-08-09T13:50:30.406822+0000","last_undegraded":"2023-08-09T13:50:30.406822+0000","last_fullsized":"2023-08-09T13:50:30.406822+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:50:28.150437+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:50:28.150437+0000","last_clean_scrub_stamp":"2023-08-09T13:50:28.150437+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T16:56:38.589913+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.3","version":"0'0","reported_seq":13,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-08-09T13:50:30.407055+0000","last_change":"2023-08-09T13:50:28.189872+0000","last_active":"2023-08-09T13:50:30.407055+0000","last_peered":"2023-08-09T13:50:30.407055+0000","last_clean":"2023-08-09T13:50:30.407055+0000","last_became_active":"2023-08-09T13:50:28.189588+0000","last_became_peered":"2023-08-09T13:50:28.189588+0000","last_unstale":"2023-08-09T13:50:30.407055+0000","last_undegraded":"2023-08-09T13:50:30.407055+0000","last_fullsized":"2023-08-09T13:50:30.407055+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:50:27.140775+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:50:27.140775+0000","last_clean_scrub_stamp":"2023-08-09T13:50:27.140775+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T21:59:35.338331+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.3","version":"0'0","reported_seq":11,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-08-09T13:50:30.408701+0000","last_change":"2023-08-09T13:50:29.250246+0000","last_active":"2023-08-09T13:50:30.408701+0000","last_peered":"2023-08-09T13:50:30.408701+0000","last_clean":"2023-08-09T13:50:30.408701+0000","last_became_active":"2023-08-09T13:50:29.249885+0000","last_became_peered":"2023-08-09T13:50:29.249885+0000","last_unstale":"2023-08-09T13:50:30.408701+0000","last_undegraded":"2023-08-09T13:50:30.408701+0000","last_fullsized":"2023-08-09T13:50:30.408701+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:50:28.150437+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:50:28.150437+0000","last_clean_scrub_stamp":"2023-08-09T13:50:28.150437+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T19:55:56.624445+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"10.2","version":"0'0","reported_seq":13,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-08-09T13:50:30.207205+0000","last_change":"2023-08-09T13:50:28.202029+0000","last_active":"2023-08-09T13:50:30.207205+0000","last_peered":"2023-08-09T13:50:30.207205+0000","last_clean":"2023-08-09T13:50:30.207205+0000","last_became_active":"2023-08-09T13:50:28.201720+0000","last_became_peered":"2023-08-09T13:50:28.201720+0000","last_unstale":"2023-08-09T13:50:30.207205+0000","last_undegraded":"2023-08-09T13:50:30.207205+0000","last_fullsized":"2023-08-09T13:50:30.207205+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:50:27.140775+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:50:27.140775+0000","last_clean_scrub_stamp":"2023-08-09T13:50:27.140775+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T16:25:36.557353+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.0","version":"0'0","reported_seq":11,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-08-09T13:50:30.207491+0000","last_change":"2023-08-09T13:50:29.233938+0000","last_active":"2023-08-09T13:50:30.207491+0000","last_peered":"2023-08-09T13:50:30.207491+0000","last_clean":"2023-08-09T13:50:30.207491+0000","last_became_active":"2023-08-09T13:50:29.233536+0000","last_became_peered":"2023-08-09T13:50:29.233536+0000","last_unstale":"2023-08-09T13:50:30.207491+0000","last_undegraded":"2023-08-09T13:50:30.207491+0000","last_fullsized":"2023-08-09T13:50:30.207491+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:50:28.150437+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:50:28.150437+0000","last_clean_scrub_stamp":"2023-08-09T13:50:28.150437+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T16:15:00.918668+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.1","version":"0'0","reported_seq":13,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-08-09T13:50:30.208726+0000","last_change":"2023-08-09T13:50:28.200080+0000","last_active":"2023-08-09T13:50:30.208726+0000","last_peered":"2023-08-09T13:50:30.208726+0000","last_clean":"2023-08-09T13:50:30.208726+0000","last_became_active":"2023-08-09T13:50:28.199831+0000","last_became_peered":"2023-08-09T13:50:28.199831+0000","last_unstale":"2023-08-09T13:50:30.208726+0000","last_undegraded":"2023-08-09T13:50:30.208726+0000","last_fullsized":"2023-08-09T13:50:30.208726+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:50:27.140775+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:50:27.140775+0000","last_clean_scrub_stamp":"2023-08-09T13:50:27.140775+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T19:06:24.132488+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.1","version":"0'0","reported_seq":11,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-08-09T13:50:30.408716+0000","last_change":"2023-08-09T13:50:29.249892+0000","last_active":"2023-08-09T13:50:30.408716+0000","last_peered":"2023-08-09T13:50:30.408716+0000","last_clean":"2023-08-09T13:50:30.408716+0000","last_became_active":"2023-08-09T13:50:29.242554+0000","last_became_peered":"2023-08-09T13:50:29.242554+0000","last_unstale":"2023-08-09T13:50:30.408716+0000","last_undegraded":"2023-08-09T13:50:30.408716+0000","last_fullsized":"2023-08-09T13:50:30.408716+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:50:28.150437+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:50:28.150437+0000","last_clean_scrub_stamp":"2023-08-09T13:50:28.150437+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T16:42:26.559652+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"10.0","version":"0'0","reported_seq":13,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-08-09T13:50:30.408785+0000","last_change":"2023-08-09T13:50:28.197319+0000","last_active":"2023-08-09T13:50:30.408785+0000","last_peered":"2023-08-09T13:50:30.408785+0000","last_clean":"2023-08-09T13:50:30.408785+0000","last_became_active":"2023-08-09T13:50:28.197031+0000","last_became_peered":"2023-08-09T13:50:28.197031+0000","last_unstale":"2023-08-09T13:50:30.408785+0000","last_undegraded":"2023-08-09T13:50:30.408785+0000","last_fullsized":"2023-08-09T13:50:30.408785+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:50:27.140775+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:50:27.140775+0000","last_clean_scrub_stamp":"2023-08-09T13:50:27.140775+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-11T00:35:53.449568+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.a","version":"0'0","reported_seq":11,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-08-09T13:50:30.208584+0000","last_change":"2023-08-09T13:50:29.243164+0000","last_active":"2023-08-09T13:50:30.208584+0000","last_peered":"2023-08-09T13:50:30.208584+0000","last_clean":"2023-08-09T13:50:30.208584+0000","last_became_active":"2023-08-09T13:50:29.242860+0000","last_became_peered":"2023-08-09T13:50:29.242860+0000","last_unstale":"2023-08-09T13:50:30.208584+0000","last_undegraded":"2023-08-09T13:50:30.208584+0000","last_fullsized":"2023-08-09T13:50:30.208584+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:50:28.150437+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:50:28.150437+0000","last_clean_scrub_stamp":"2023-08-09T13:50:28.150437+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T16:30:25.964647+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.b","version":"89'2","reported_seq":15,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-08-09T13:50:30.409044+0000","last_change":"2023-08-09T13:50:28.203783+0000","last_active":"2023-08-09T13:50:30.409044+0000","last_peered":"2023-08-09T13:50:30.409044+0000","last_clean":"2023-08-09T13:50:30.409044+0000","last_became_active":"2023-08-09T13:50:28.202391+0000","last_became_peered":"2023-08-09T13:50:28.202391+0000","last_unstale":"2023-08-09T13:50:30.409044+0000","last_undegraded":"2023-08-09T13:50:30.409044+0000","last_fullsized":"2023-08-09T13:50:30.409044+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:50:27.140775+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:50:27.140775+0000","last_clean_scrub_stamp":"2023-08-09T13:50:27.140775+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-11T00:41:12.464656+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":151,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-08-09T13:50:30.409102+0000","last_change":"2023-08-09T13:33:56.401406+0000","last_active":"2023-08-09T13:50:30.409102+0000","last_peered":"2023-08-09T13:50:30.409102+0000","last_clean":"2023-08-09T13:50:30.409102+0000","last_became_active":"2023-08-09T13:33:56.400487+0000","last_became_peered":"2023-08-09T13:33:56.400487+0000","last_unstale":"2023-08-09T13:50:30.409102+0000","last_undegraded":"2023-08-09T13:50:30.409102+0000","last_fullsized":"2023-08-09T13:50:30.409102+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:33:54.845869+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:33:54.845869+0000","last_clean_scrub_stamp":"2023-08-09T13:33:54.845869+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T22:11:37.661887+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"11.b","version":"0'0","reported_seq":11,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-08-09T13:50:30.208779+0000","last_change":"2023-08-09T13:50:29.237832+0000","last_active":"2023-08-09T13:50:30.208779+0000","last_peered":"2023-08-09T13:50:30.208779+0000","last_clean":"2023-08-09T13:50:30.208779+0000","last_became_active":"2023-08-09T13:50:29.237523+0000","last_became_peered":"2023-08-09T13:50:29.237523+0000","last_unstale":"2023-08-09T13:50:30.208779+0000","last_undegraded":"2023-08-09T13:50:30.208779+0000","last_fullsized":"2023-08-09T13:50:30.208779+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:50:28.150437+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:50:28.150437+0000","last_clean_scrub_stamp":"2023-08-09T13:50:28.150437+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T19:14:28.632849+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.a","version":"0'0","reported_seq":13,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-08-09T13:50:30.408217+0000","last_change":"2023-08-09T13:50:28.205148+0000","last_active":"2023-08-09T13:50:30.408217+0000","last_peered":"2023-08-09T13:50:30.408217+0000","last_clean":"2023-08-09T13:50:30.408217+0000","last_became_active":"2023-08-09T13:50:28.204528+0000","last_became_peered":"2023-08-09T13:50:28.204528+0000","last_unstale":"2023-08-09T13:50:30.408217+0000","last_undegraded":"2023-08-09T13:50:30.408217+0000","last_fullsized":"2023-08-09T13:50:30.408217+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:50:27.140775+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:50:27.140775+0000","last_clean_scrub_stamp":"2023-08-09T13:50:27.140775+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T15:47:16.715572+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":150,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-08-09T13:50:30.407021+0000","last_change":"2023-08-09T13:33:56.401123+0000","last_active":"2023-08-09T13:50:30.407021+0000","last_peered":"2023-08-09T13:50:30.407021+0000","last_clean":"2023-08-09T13:50:30.407021+0000","last_became_active":"2023-08-09T13:33:56.399933+0000","last_became_peered":"2023-08-09T13:33:56.399933+0000","last_unstale":"2023-08-09T13:50:30.407021+0000","last_undegraded":"2023-08-09T13:50:30.407021+0000","last_fullsized":"2023-08-09T13:50:30.407021+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:33:54.845869+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:33:54.845869+0000","last_clean_scrub_stamp":"2023-08-09T13:33:54.845869+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-11T00:50:51.859858+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.8","version":"0'0","reported_seq":11,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-08-09T13:50:30.208522+0000","last_change":"2023-08-09T13:50:29.250361+0000","last_active":"2023-08-09T13:50:30.208522+0000","last_peered":"2023-08-09T13:50:30.208522+0000","last_clean":"2023-08-09T13:50:30.208522+0000","last_became_active":"2023-08-09T13:50:29.250039+0000","last_became_peered":"2023-08-09T13:50:29.250039+0000","last_unstale":"2023-08-09T13:50:30.208522+0000","last_undegraded":"2023-08-09T13:50:30.208522+0000","last_fullsized":"2023-08-09T13:50:30.208522+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:50:28.150437+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:50:28.150437+0000","last_clean_scrub_stamp":"2023-08-09T13:50:28.150437+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T15:47:57.974583+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.9","version":"0'0","reported_seq":13,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-08-09T13:50:30.406797+0000","last_change":"2023-08-09T13:50:28.192232+0000","last_active":"2023-08-09T13:50:30.406797+0000","last_peered":"2023-08-09T13:50:30.406797+0000","last_clean":"2023-08-09T13:50:30.406797+0000","last_became_active":"2023-08-09T13:50:28.191350+0000","last_became_peered":"2023-08-09T13:50:28.191350+0000","last_unstale":"2023-08-09T13:50:30.406797+0000","last_undegraded":"2023-08-09T13:50:30.406797+0000","last_fullsized":"2023-08-09T13:50:30.406797+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:50:27.140775+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:50:27.140775+0000","last_clean_scrub_stamp":"2023-08-09T13:50:27.140775+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T23:52:03.500625+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.2","version":"0'0","reported_seq":150,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-08-09T13:50:30.409511+0000","last_change":"2023-08-09T13:33:56.402794+0000","last_active":"2023-08-09T13:50:30.409511+0000","last_peered":"2023-08-09T13:50:30.409511+0000","last_clean":"2023-08-09T13:50:30.409511+0000","last_became_active":"2023-08-09T13:33:56.402038+0000","last_became_peered":"2023-08-09T13:33:56.402038+0000","last_unstale":"2023-08-09T13:50:30.409511+0000","last_undegraded":"2023-08-09T13:50:30.409511+0000","last_fullsized":"2023-08-09T13:50:30.409511+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:33:54.845869+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:33:54.845869+0000","last_clean_scrub_stamp":"2023-08-09T13:33:54.845869+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T16:16:23.385071+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"11.9","version":"0'0","reported_seq":11,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-08-09T13:50:30.208772+0000","last_change":"2023-08-09T13:50:29.246693+0000","last_active":"2023-08-09T13:50:30.208772+0000","last_peered":"2023-08-09T13:50:30.208772+0000","last_clean":"2023-08-09T13:50:30.208772+0000","last_became_active":"2023-08-09T13:50:29.246366+0000","last_became_peered":"2023-08-09T13:50:29.246366+0000","last_unstale":"2023-08-09T13:50:30.208772+0000","last_undegraded":"2023-08-09T13:50:30.208772+0000","last_fullsized":"2023-08-09T13:50:30.208772+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:50:28.150437+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:50:28.150437+0000","last_clean_scrub_stamp":"2023-08-09T13:50:28.150437+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T14:30:30.875587+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.8","version":"0'0","reported_seq":13,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-08-09T13:50:30.406875+0000","last_change":"2023-08-09T13:50:28.190965+0000","last_active":"2023-08-09T13:50:30.406875+0000","last_peered":"2023-08-09T13:50:30.406875+0000","last_clean":"2023-08-09T13:50:30.406875+0000","last_became_active":"2023-08-09T13:50:28.190696+0000","last_became_peered":"2023-08-09T13:50:28.190696+0000","last_unstale":"2023-08-09T13:50:30.406875+0000","last_undegraded":"2023-08-09T13:50:30.406875+0000","last_fullsized":"2023-08-09T13:50:30.406875+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:50:27.140775+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:50:27.140775+0000","last_clean_scrub_stamp":"2023-08-09T13:50:27.140775+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T22:28:09.956793+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.3","version":"0'0","reported_seq":150,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-08-09T13:50:30.207472+0000","last_change":"2023-08-09T13:33:55.886749+0000","last_active":"2023-08-09T13:50:30.207472+0000","last_peered":"2023-08-09T13:50:30.207472+0000","last_clean":"2023-08-09T13:50:30.207472+0000","last_became_active":"2023-08-09T13:33:55.886116+0000","last_became_peered":"2023-08-09T13:33:55.886116+0000","last_unstale":"2023-08-09T13:50:30.207472+0000","last_undegraded":"2023-08-09T13:50:30.207472+0000","last_fullsized":"2023-08-09T13:50:30.207472+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:33:54.845869+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:33:54.845869+0000","last_clean_scrub_stamp":"2023-08-09T13:33:54.845869+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T23:29:59.861309+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"77'161","reported_seq":457,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-08-09T13:50:30.408826+0000","last_change":"2023-08-09T13:33:55.886345+0000","last_active":"2023-08-09T13:50:30.408826+0000","last_peered":"2023-08-09T13:50:30.408826+0000","last_clean":"2023-08-09T13:50:30.408826+0000","last_became_active":"2023-08-09T13:33:55.885781+0000","last_became_peered":"2023-08-09T13:33:55.885781+0000","last_unstale":"2023-08-09T13:50:30.408826+0000","last_undegraded":"2023-08-09T13:50:30.408826+0000","last_fullsized":"2023-08-09T13:50:30.408826+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:33:54.845869+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:33:54.845869+0000","last_clean_scrub_stamp":"2023-08-09T13:33:54.845869+0000","objects_scrubbed":0,"log_size":161,"log_dups_size":0,"ondisk_log_size":161,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T22:54:47.590086+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":500,"num_read_kb":924,"num_write":239,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"11.e","version":"0'0","reported_seq":11,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-08-09T13:50:30.210115+0000","last_change":"2023-08-09T13:50:29.227243+0000","last_active":"2023-08-09T13:50:30.210115+0000","last_peered":"2023-08-09T13:50:30.210115+0000","last_clean":"2023-08-09T13:50:30.210115+0000","last_became_active":"2023-08-09T13:50:29.226756+0000","last_became_peered":"2023-08-09T13:50:29.226756+0000","last_unstale":"2023-08-09T13:50:30.210115+0000","last_undegraded":"2023-08-09T13:50:30.210115+0000","last_fullsized":"2023-08-09T13:50:30.210115+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:50:28.150437+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:50:28.150437+0000","last_clean_scrub_stamp":"2023-08-09T13:50:28.150437+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T16:22:47.844581+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"10.f","version":"89'1","reported_seq":14,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-08-09T13:50:30.410599+0000","last_change":"2023-08-09T13:50:28.205120+0000","last_active":"2023-08-09T13:50:30.410599+0000","last_peered":"2023-08-09T13:50:30.410599+0000","last_clean":"2023-08-09T13:50:30.410599+0000","last_became_active":"2023-08-09T13:50:28.204771+0000","last_became_peered":"2023-08-09T13:50:28.204771+0000","last_unstale":"2023-08-09T13:50:30.410599+0000","last_undegraded":"2023-08-09T13:50:30.410599+0000","last_fullsized":"2023-08-09T13:50:30.410599+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:50:27.140775+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:50:27.140775+0000","last_clean_scrub_stamp":"2023-08-09T13:50:27.140775+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T15:54:30.903742+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":160,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-08-09T13:50:30.208900+0000","last_change":"2023-08-09T13:33:55.889565+0000","last_active":"2023-08-09T13:50:30.208900+0000","last_peered":"2023-08-09T13:50:30.208900+0000","last_clean":"2023-08-09T13:50:30.208900+0000","last_became_active":"2023-08-09T13:33:55.889164+0000","last_became_peered":"2023-08-09T13:33:55.889164+0000","last_unstale":"2023-08-09T13:50:30.208900+0000","last_undegraded":"2023-08-09T13:50:30.208900+0000","last_fullsized":"2023-08-09T13:50:30.208900+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:33:54.845869+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:33:54.845869+0000","last_clean_scrub_stamp":"2023-08-09T13:33:54.845869+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T23:47:22.155576+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.f","version":"0'0","reported_seq":11,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-08-09T13:50:30.411502+0000","last_change":"2023-08-09T13:50:29.238559+0000","last_active":"2023-08-09T13:50:30.411502+0000","last_peered":"2023-08-09T13:50:30.411502+0000","last_clean":"2023-08-09T13:50:30.411502+0000","last_became_active":"2023-08-09T13:50:29.238287+0000","last_became_peered":"2023-08-09T13:50:29.238287+0000","last_unstale":"2023-08-09T13:50:30.411502+0000","last_undegraded":"2023-08-09T13:50:30.411502+0000","last_fullsized":"2023-08-09T13:50:30.411502+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:50:28.150437+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:50:28.150437+0000","last_clean_scrub_stamp":"2023-08-09T13:50:28.150437+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T23:16:13.975884+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.e","version":"89'2","reported_seq":15,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-08-09T13:50:30.210099+0000","last_change":"2023-08-09T13:50:28.206561+0000","last_active":"2023-08-09T13:50:30.210099+0000","last_peered":"2023-08-09T13:50:30.210099+0000","last_clean":"2023-08-09T13:50:30.210099+0000","last_became_active":"2023-08-09T13:50:28.206264+0000","last_became_peered":"2023-08-09T13:50:28.206264+0000","last_unstale":"2023-08-09T13:50:30.210099+0000","last_undegraded":"2023-08-09T13:50:30.210099+0000","last_fullsized":"2023-08-09T13:50:30.210099+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:50:27.140775+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:50:27.140775+0000","last_clean_scrub_stamp":"2023-08-09T13:50:27.140775+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-11T01:17:38.740286+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":150,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-08-09T13:50:30.210081+0000","last_change":"2023-08-09T13:33:56.404236+0000","last_active":"2023-08-09T13:50:30.210081+0000","last_peered":"2023-08-09T13:50:30.210081+0000","last_clean":"2023-08-09T13:50:30.210081+0000","last_became_active":"2023-08-09T13:33:56.402485+0000","last_became_peered":"2023-08-09T13:33:56.402485+0000","last_unstale":"2023-08-09T13:50:30.210081+0000","last_undegraded":"2023-08-09T13:50:30.210081+0000","last_fullsized":"2023-08-09T13:50:30.210081+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:33:54.845869+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:33:54.845869+0000","last_clean_scrub_stamp":"2023-08-09T13:33:54.845869+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T23:29:37.526528+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.c","version":"0'0","reported_seq":11,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-08-09T13:50:30.207708+0000","last_change":"2023-08-09T13:50:29.230700+0000","last_active":"2023-08-09T13:50:30.207708+0000","last_peered":"2023-08-09T13:50:30.207708+0000","last_clean":"2023-08-09T13:50:30.207708+0000","last_became_active":"2023-08-09T13:50:29.230389+0000","last_became_peered":"2023-08-09T13:50:29.230389+0000","last_unstale":"2023-08-09T13:50:30.207708+0000","last_undegraded":"2023-08-09T13:50:30.207708+0000","last_fullsized":"2023-08-09T13:50:30.207708+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:50:28.150437+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:50:28.150437+0000","last_clean_scrub_stamp":"2023-08-09T13:50:28.150437+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-11T00:40:44.937679+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.d","version":"89'1","reported_seq":14,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-08-09T13:50:30.408375+0000","last_change":"2023-08-09T13:50:28.199480+0000","last_active":"2023-08-09T13:50:30.408375+0000","last_peered":"2023-08-09T13:50:30.408375+0000","last_clean":"2023-08-09T13:50:30.408375+0000","last_became_active":"2023-08-09T13:50:28.199211+0000","last_became_peered":"2023-08-09T13:50:28.199211+0000","last_unstale":"2023-08-09T13:50:30.408375+0000","last_undegraded":"2023-08-09T13:50:30.408375+0000","last_fullsized":"2023-08-09T13:50:30.408375+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:50:27.140775+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:50:27.140775+0000","last_clean_scrub_stamp":"2023-08-09T13:50:27.140775+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T21:27:45.703707+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":148,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-08-09T13:50:30.410244+0000","last_change":"2023-08-09T13:33:56.401781+0000","last_active":"2023-08-09T13:50:30.410244+0000","last_peered":"2023-08-09T13:50:30.410244+0000","last_clean":"2023-08-09T13:50:30.410244+0000","last_became_active":"2023-08-09T13:33:56.400336+0000","last_became_peered":"2023-08-09T13:33:56.400336+0000","last_unstale":"2023-08-09T13:50:30.410244+0000","last_undegraded":"2023-08-09T13:50:30.410244+0000","last_fullsized":"2023-08-09T13:50:30.410244+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:33:54.845869+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:33:54.845869+0000","last_clean_scrub_stamp":"2023-08-09T13:33:54.845869+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T16:34:18.011530+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.d","version":"0'0","reported_seq":11,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-08-09T13:50:30.405923+0000","last_change":"2023-08-09T13:50:29.243494+0000","last_active":"2023-08-09T13:50:30.405923+0000","last_peered":"2023-08-09T13:50:30.405923+0000","last_clean":"2023-08-09T13:50:30.405923+0000","last_became_active":"2023-08-09T13:50:29.243154+0000","last_became_peered":"2023-08-09T13:50:29.243154+0000","last_unstale":"2023-08-09T13:50:30.405923+0000","last_undegraded":"2023-08-09T13:50:30.405923+0000","last_fullsized":"2023-08-09T13:50:30.405923+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:50:28.150437+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:50:28.150437+0000","last_clean_scrub_stamp":"2023-08-09T13:50:28.150437+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T14:21:14.527166+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.c","version":"0'0","reported_seq":13,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-08-09T13:50:30.410095+0000","last_change":"2023-08-09T13:50:28.203868+0000","last_active":"2023-08-09T13:50:30.410095+0000","last_peered":"2023-08-09T13:50:30.410095+0000","last_clean":"2023-08-09T13:50:30.410095+0000","last_became_active":"2023-08-09T13:50:28.203323+0000","last_became_peered":"2023-08-09T13:50:28.203323+0000","last_unstale":"2023-08-09T13:50:30.410095+0000","last_undegraded":"2023-08-09T13:50:30.410095+0000","last_fullsized":"2023-08-09T13:50:30.410095+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:50:27.140775+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:50:27.140775+0000","last_clean_scrub_stamp":"2023-08-09T13:50:27.140775+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T22:03:21.452695+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":148,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-08-09T13:50:30.411241+0000","last_change":"2023-08-09T13:33:55.886389+0000","last_active":"2023-08-09T13:50:30.411241+0000","last_peered":"2023-08-09T13:50:30.411241+0000","last_clean":"2023-08-09T13:50:30.411241+0000","last_became_active":"2023-08-09T13:33:55.885638+0000","last_became_peered":"2023-08-09T13:33:55.885638+0000","last_unstale":"2023-08-09T13:50:30.411241+0000","last_undegraded":"2023-08-09T13:50:30.411241+0000","last_fullsized":"2023-08-09T13:50:30.411241+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:33:54.845869+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:33:54.845869+0000","last_clean_scrub_stamp":"2023-08-09T13:33:54.845869+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-11T01:09:14.155690+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]}],"pool_stats":[{"poolid":11,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":10,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":39,"ondisk_log_size":39,"up":64,"acting":64,"num_store_stats":8},{"poolid":2,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":500,"num_read_kb":924,"num_write":239,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":161,"ondisk_log_size":161,"up":2,"acting":2,"num_store_stats":2},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":8}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834575048,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":36868,"kb_used_data":1656,"kb_used_omap":1,"kb_used_meta":35198,"kb_avail":94334972,"statfs":{"total":96636764160,"available":96599011328,"internally_reserved":0,"allocated":1695744,"data_stored":1174761,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1781,"internal_metadata":36043019},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Aug 9 13:50:16 2023","interfaces":[{"interface":"back","average":{"1min":1.181,"5min":1.222,"15min":1.232},"min":{"1min":0.69499999999999995,"5min":0.63400000000000001,"15min":0.61099999999999999},"max":{"1min":1.633,"5min":1.8919999999999999,"15min":2.1280000000000001},"last":0.92000000000000004},{"interface":"front","average":{"1min":1.1819999999999999,"5min":1.232,"15min":1.24},"min":{"1min":0.72399999999999998,"5min":0.72399999999999998,"15min":0.65000000000000002},"max":{"1min":1.6100000000000001,"5min":1.839,"15min":2.0099999999999998},"last":1.103}]},{"osd":1,"last update":"Wed Aug 9 13:50:16 2023","interfaces":[{"interface":"back","average":{"1min":1.2010000000000001,"5min":1.2450000000000001,"15min":1.256},"min":{"1min":0.72799999999999998,"5min":0.72799999999999998,"15min":0.59799999999999998},"max":{"1min":1.8,"5min":1.8,"15min":1.9470000000000001},"last":1.181},{"interface":"front","average":{"1min":1.2569999999999999,"5min":1.248,"15min":1.2869999999999999},"min":{"1min":0.83099999999999996,"5min":0.59299999999999997,"15min":0.59299999999999997},"max":{"1min":1.726,"5min":1.9410000000000001,"15min":1.9410000000000001},"last":0.99199999999999999}]},{"osd":2,"last update":"Wed Aug 9 13:50:16 2023","interfaces":[{"interface":"back","average":{"1min":1.278,"5min":1.2470000000000001,"15min":1.2430000000000001},"min":{"1min":0.81000000000000005,"5min":0.65700000000000003,"15min":0.65700000000000003},"max":{"1min":1.742,"5min":2.0129999999999999,"15min":2.0129999999999999},"last":1.635},{"interface":"front","average":{"1min":1.2889999999999999,"5min":1.323,"15min":1.3759999999999999},"min":{"1min":0.92600000000000005,"5min":0.80900000000000005,"15min":0.67800000000000005},"max":{"1min":1.7130000000000001,"5min":1.8740000000000001,"15min":2.4140000000000001},"last":1.0620000000000001}]},{"osd":3,"last update":"Wed Aug 9 13:50:16 2023","interfaces":[{"interface":"back","average":{"1min":1.2529999999999999,"5min":1.3180000000000001,"15min":1.331},"min":{"1min":0.98799999999999999,"5min":0.86799999999999999,"15min":0.77500000000000002},"max":{"1min":1.7809999999999999,"5min":1.97,"15min":1.97},"last":1.462},{"interface":"front","average":{"1min":1.3740000000000001,"5min":1.419,"15min":1.4159999999999999},"min":{"1min":0.93899999999999995,"5min":0.88300000000000001,"15min":0.82099999999999995},"max":{"1min":1.8160000000000001,"5min":2.0790000000000002,"15min":2.2480000000000002},"last":1.337}]},{"osd":4,"last update":"Wed Aug 9 13:50:16 2023","interfaces":[{"interface":"back","average":{"1min":1.343,"5min":1.2649999999999999,"15min":1.2769999999999999},"min":{"1min":0.86399999999999999,"5min":0.59499999999999997,"15min":0.59499999999999997},"max":{"1min":1.772,"5min":2.032,"15min":2.343},"last":1.506},{"interface":"front","average":{"1min":1.296,"5min":1.3160000000000001,"15min":1.339},"min":{"1min":0.93600000000000005,"5min":0.77100000000000002,"15min":0.56799999999999995},"max":{"1min":1.6739999999999999,"5min":2.1419999999999999,"15min":2.1419999999999999},"last":1.276}]},{"osd":5,"last update":"Wed Aug 9 13:50:16 2023","interfaces":[{"interface":"back","average":{"1min":1.4430000000000001,"5min":1.4350000000000001,"15min":1.419},"min":{"1min":1.048,"5min":0.88600000000000001,"15min":0.71499999999999997},"max":{"1min":1.9039999999999999,"5min":2.1480000000000001,"15min":2.6749999999999998},"last":1.54},{"interface":"front","average":{"1min":1.4810000000000001,"5min":1.4319999999999999,"15min":1.4510000000000001},"min":{"1min":0.95999999999999996,"5min":0.85699999999999998,"15min":0.69199999999999995},"max":{"1min":1.877,"5min":2.113,"15min":2.9569999999999999},"last":1.226}]},{"osd":6,"last update":"Wed Aug 9 13:50:16 2023","interfaces":[{"interface":"back","average":{"1min":1.6020000000000001,"5min":1.518,"15min":1.5489999999999999},"min":{"1min":1.2,"5min":0.85499999999999998,"15min":0.85499999999999998},"max":{"1min":1.8879999999999999,"5min":2.2069999999999999,"15min":2.8149999999999999},"last":1.397},{"interface":"front","average":{"1min":1.4139999999999999,"5min":1.4570000000000001,"15min":1.474},"min":{"1min":1.111,"5min":0.90600000000000003,"15min":0.60799999999999998},"max":{"1min":1.9419999999999999,"5min":2.1320000000000001,"15min":2.2530000000000001},"last":1.736}]}]},{"osd":6,"up_from":13,"seq":55834575048,"num_pgs":38,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32196,"kb_used_data":1072,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":94339644,"statfs":{"total":96636764160,"available":96603795456,"internally_reserved":0,"allocated":1097728,"data_stored":582508,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1781,"internal_metadata":31848715},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Aug 9 13:50:20 2023","interfaces":[{"interface":"back","average":{"1min":1.1160000000000001,"5min":1.143,"15min":1.161},"min":{"1min":0.75,"5min":0.72499999999999998,"15min":0.56599999999999995},"max":{"1min":1.5489999999999999,"5min":1.633,"15min":1.9870000000000001},"last":0.91500000000000004},{"interface":"front","average":{"1min":1.21,"5min":1.532,"15min":1.3080000000000001},"min":{"1min":0.79300000000000004,"5min":0.79300000000000004,"15min":0.76000000000000001},"max":{"1min":1.6519999999999999,"5min":28.036000000000001,"15min":28.036000000000001},"last":0.86399999999999999}]},{"osd":1,"last update":"Wed Aug 9 13:50:20 2023","interfaces":[{"interface":"back","average":{"1min":1.147,"5min":1.272,"15min":1.226},"min":{"1min":0.70399999999999996,"5min":0.70399999999999996,"15min":0.65500000000000003},"max":{"1min":1.659,"5min":7.742,"15min":7.742},"last":0.97699999999999998},{"interface":"front","average":{"1min":1.252,"5min":1.2849999999999999,"15min":1.276},"min":{"1min":0.84199999999999997,"5min":0.84199999999999997,"15min":0.68100000000000005},"max":{"1min":1.8899999999999999,"5min":1.8899999999999999,"15min":1.8899999999999999},"last":1.645}]},{"osd":2,"last update":"Wed Aug 9 13:50:20 2023","interfaces":[{"interface":"back","average":{"1min":1.2989999999999999,"5min":1.3480000000000001,"15min":1.3220000000000001},"min":{"1min":0.877,"5min":0.81999999999999995,"15min":0.68700000000000006},"max":{"1min":1.73,"5min":4.0640000000000001,"15min":4.0640000000000001},"last":1.5129999999999999},{"interface":"front","average":{"1min":1.244,"5min":1.4430000000000001,"15min":1.3460000000000001},"min":{"1min":0.76800000000000002,"5min":0.70999999999999996,"15min":0.70999999999999996},"max":{"1min":2.0390000000000001,"5min":13.154999999999999,"15min":13.154999999999999},"last":1.458}]},{"osd":3,"last update":"Wed Aug 9 13:50:20 2023","interfaces":[{"interface":"back","average":{"1min":1.3939999999999999,"5min":1.52,"15min":1.419},"min":{"1min":0.73099999999999998,"5min":0.73099999999999998,"15min":0.73099999999999998},"max":{"1min":2.2519999999999998,"5min":13.349,"15min":13.349},"last":1.222},{"interface":"front","average":{"1min":1.258,"5min":1.3620000000000001,"15min":1.325},"min":{"1min":0.88400000000000001,"5min":0.76400000000000001,"15min":0.74299999999999999},"max":{"1min":1.8560000000000001,"5min":7.9029999999999996,"15min":7.9029999999999996},"last":1.3280000000000001}]},{"osd":4,"last update":"Wed Aug 9 13:50:20 2023","interfaces":[{"interface":"back","average":{"1min":1.302,"5min":1.335,"15min":1.28},"min":{"1min":0.90700000000000003,"5min":0.78500000000000003,"15min":0.64300000000000002},"max":{"1min":1.992,"5min":5.2290000000000001,"15min":5.2290000000000001},"last":1.8069999999999999},{"interface":"front","average":{"1min":1.4019999999999999,"5min":1.498,"15min":1.389},"min":{"1min":0.88900000000000001,"5min":0.78700000000000003,"15min":0.69099999999999995},"max":{"1min":1.734,"5min":12.1,"15min":12.1},"last":1.5720000000000001}]},{"osd":5,"last update":"Wed Aug 9 13:50:20 2023","interfaces":[{"interface":"back","average":{"1min":1.3169999999999999,"5min":1.4399999999999999,"15min":1.357},"min":{"1min":0.93999999999999995,"5min":0.83799999999999997,"15min":0.65400000000000003},"max":{"1min":2.1139999999999999,"5min":8.1899999999999995,"15min":8.1899999999999995},"last":1.756},{"interface":"front","average":{"1min":1.5649999999999999,"5min":1.6639999999999999,"15min":1.589},"min":{"1min":1.2010000000000001,"5min":1.2010000000000001,"15min":0.83799999999999997},"max":{"1min":2.2000000000000002,"5min":8.8889999999999993,"15min":8.8889999999999993},"last":1.881}]},{"osd":7,"last update":"Wed Aug 9 13:50:20 2023","interfaces":[{"interface":"back","average":{"1min":1.577,"5min":1.6970000000000001,"15min":1.5880000000000001},"min":{"1min":1.1850000000000001,"5min":1.0029999999999999,"15min":0.62},"max":{"1min":1.8240000000000001,"5min":9.3320000000000007,"15min":9.3320000000000007},"last":1.7110000000000001},{"interface":"front","average":{"1min":1.4410000000000001,"5min":1.6060000000000001,"15min":1.548},"min":{"1min":0.93899999999999995,"5min":0.88500000000000001,"15min":0.88500000000000001},"max":{"1min":1.9470000000000001,"5min":8.0399999999999991,"15min":8.0399999999999991},"last":1.8380000000000001}]}]},{"osd":5,"up_from":13,"seq":55834575048,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":36284,"kb_used_data":1072,"kb_used_omap":1,"kb_used_meta":35198,"kb_avail":94335556,"statfs":{"total":96636764160,"available":96599609344,"internally_reserved":0,"allocated":1097728,"data_stored":583867,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1781,"internal_metadata":36043019},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Aug 9 13:50:19 2023","interfaces":[{"interface":"back","average":{"1min":1.2509999999999999,"5min":1.1759999999999999,"15min":1.179},"min":{"1min":0.63500000000000001,"5min":0.63500000000000001,"15min":0.63500000000000001},"max":{"1min":1.518,"5min":2.1869999999999998,"15min":2.1869999999999998},"last":0.59099999999999997},{"interface":"front","average":{"1min":1.1719999999999999,"5min":1.2,"15min":1.198},"min":{"1min":0.68600000000000005,"5min":0.68600000000000005,"15min":0.64200000000000002},"max":{"1min":1.579,"5min":1.6619999999999999,"15min":1.7989999999999999},"last":0.72199999999999998}]},{"osd":1,"last update":"Wed Aug 9 13:50:19 2023","interfaces":[{"interface":"back","average":{"1min":1.165,"5min":1.1990000000000001,"15min":1.2190000000000001},"min":{"1min":0.81599999999999995,"5min":0.73099999999999998,"15min":0.64100000000000001},"max":{"1min":1.631,"5min":1.6739999999999999,"15min":1.736},"last":0.68799999999999994},{"interface":"front","average":{"1min":1.264,"5min":1.296,"15min":1.2769999999999999},"min":{"1min":0.93300000000000005,"5min":0.66900000000000004,"15min":0.66900000000000004},"max":{"1min":1.6699999999999999,"5min":1.887,"15min":1.887},"last":0.94999999999999996}]},{"osd":2,"last update":"Wed Aug 9 13:50:19 2023","interfaces":[{"interface":"back","average":{"1min":1.2410000000000001,"5min":1.2569999999999999,"15min":1.2849999999999999},"min":{"1min":0.98199999999999998,"5min":0.68999999999999995,"15min":0.61599999999999999},"max":{"1min":1.5760000000000001,"5min":1.756,"15min":1.899},"last":1.036},{"interface":"front","average":{"1min":1.3049999999999999,"5min":1.302,"15min":1.3149999999999999},"min":{"1min":0.78400000000000003,"5min":0.78400000000000003,"15min":0.66800000000000004},"max":{"1min":1.6599999999999999,"5min":1.7190000000000001,"15min":1.7969999999999999},"last":1.0009999999999999}]},{"osd":3,"last update":"Wed Aug 9 13:50:19 2023","interfaces":[{"interface":"back","average":{"1min":1.365,"5min":1.3720000000000001,"15min":1.3600000000000001},"min":{"1min":0.95399999999999996,"5min":0.95399999999999996,"15min":0.59799999999999998},"max":{"1min":1.6890000000000001,"5min":2.1840000000000002,"15min":2.2690000000000001},"last":1.117},{"interface":"front","average":{"1min":1.4259999999999999,"5min":1.4379999999999999,"15min":1.4139999999999999},"min":{"1min":1.087,"5min":0.94899999999999995,"15min":0.70999999999999996},"max":{"1min":1.73,"5min":2.3780000000000001,"15min":2.3780000000000001},"last":1.218}]},{"osd":4,"last update":"Wed Aug 9 13:50:19 2023","interfaces":[{"interface":"back","average":{"1min":1.262,"5min":1.2729999999999999,"15min":1.2889999999999999},"min":{"1min":0.92700000000000005,"5min":0.84699999999999998,"15min":0.56599999999999995},"max":{"1min":1.673,"5min":2.0800000000000001,"15min":2.121},"last":1.0760000000000001},{"interface":"front","average":{"1min":1.3919999999999999,"5min":1.3400000000000001,"15min":1.3779999999999999},"min":{"1min":0.81999999999999995,"5min":0.76100000000000001,"15min":0.58599999999999997},"max":{"1min":1.944,"5min":2.1230000000000002,"15min":2.3679999999999999},"last":1.2749999999999999}]},{"osd":6,"last update":"Wed Aug 9 13:50:19 2023","interfaces":[{"interface":"back","average":{"1min":1.4299999999999999,"5min":1.3520000000000001,"15min":1.391},"min":{"1min":0.99099999999999999,"5min":0.90300000000000002,"15min":0.66700000000000004},"max":{"1min":1.9810000000000001,"5min":2.306,"15min":2.306},"last":1.3120000000000001},{"interface":"front","average":{"1min":1.49,"5min":1.4399999999999999,"15min":1.4710000000000001},"min":{"1min":1.1220000000000001,"5min":1.034,"15min":0.71199999999999997},"max":{"1min":1.827,"5min":2.2320000000000002,"15min":2.2320000000000002},"last":1.1659999999999999}]},{"osd":7,"last update":"Wed Aug 9 13:50:19 2023","interfaces":[{"interface":"back","average":{"1min":1.577,"5min":1.5580000000000001,"15min":1.5529999999999999},"min":{"1min":1.1160000000000001,"5min":1.0589999999999999,"15min":0.70899999999999996},"max":{"1min":1.9610000000000001,"5min":2.452,"15min":2.4729999999999999},"last":1.405},{"interface":"front","average":{"1min":1.611,"5min":1.585,"15min":1.6080000000000001},"min":{"1min":1.2110000000000001,"5min":1.163,"15min":0.77600000000000002},"max":{"1min":2.097,"5min":2.1070000000000002,"15min":2.3820000000000001},"last":1.369}]}]},{"osd":4,"up_from":13,"seq":55834575048,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32196,"kb_used_data":1072,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":94339644,"statfs":{"total":96636764160,"available":96603795456,"internally_reserved":0,"allocated":1097728,"data_stored":583847,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1781,"internal_metadata":31848715},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Aug 9 13:50:21 2023","interfaces":[{"interface":"back","average":{"1min":1.2330000000000001,"5min":1.2390000000000001,"15min":1.2030000000000001},"min":{"1min":0.875,"5min":0.70699999999999996,"15min":0.67700000000000005},"max":{"1min":1.5940000000000001,"5min":2.524,"15min":2.524},"last":1.5640000000000001},{"interface":"front","average":{"1min":1.206,"5min":1.236,"15min":1.2569999999999999},"min":{"1min":0.62,"5min":0.62,"15min":0.62},"max":{"1min":1.5980000000000001,"5min":1.863,"15min":1.863},"last":1.1200000000000001}]},{"osd":1,"last update":"Wed Aug 9 13:50:21 2023","interfaces":[{"interface":"back","average":{"1min":1.244,"5min":1.3169999999999999,"15min":1.2929999999999999},"min":{"1min":0.70999999999999996,"5min":0.67800000000000005,"15min":0.67800000000000005},"max":{"1min":1.48,"5min":1.8480000000000001,"15min":1.8480000000000001},"last":1.4950000000000001},{"interface":"front","average":{"1min":1.242,"5min":1.2649999999999999,"15min":1.248},"min":{"1min":0.81799999999999995,"5min":0.81799999999999995,"15min":0.64900000000000002},"max":{"1min":1.9330000000000001,"5min":1.9330000000000001,"15min":1.9330000000000001},"last":0.84899999999999998}]},{"osd":2,"last update":"Wed Aug 9 13:50:21 2023","interfaces":[{"interface":"back","average":{"1min":1.306,"5min":1.2869999999999999,"15min":1.2809999999999999},"min":{"1min":1.022,"5min":0.621,"15min":0.621},"max":{"1min":1.619,"5min":2.6429999999999998,"15min":2.6429999999999998},"last":0.90600000000000003},{"interface":"front","average":{"1min":1.343,"5min":1.3720000000000001,"15min":1.3660000000000001},"min":{"1min":0.88300000000000001,"5min":0.88300000000000001,"15min":0.74299999999999999},"max":{"1min":1.7569999999999999,"5min":1.7569999999999999,"15min":1.8149999999999999},"last":1.613}]},{"osd":3,"last update":"Wed Aug 9 13:50:21 2023","interfaces":[{"interface":"back","average":{"1min":1.377,"5min":1.3300000000000001,"15min":1.298},"min":{"1min":1.083,"5min":0.86399999999999999,"15min":0.84699999999999998},"max":{"1min":1.6739999999999999,"5min":2.6099999999999999,"15min":2.6099999999999999},"last":1.226},{"interface":"front","average":{"1min":1.4099999999999999,"5min":1.3979999999999999,"15min":1.3799999999999999},"min":{"1min":0.94799999999999995,"5min":0.88800000000000001,"15min":0.63100000000000001},"max":{"1min":1.6890000000000001,"5min":2.7629999999999999,"15min":2.7629999999999999},"last":1.7010000000000001}]},{"osd":5,"last update":"Wed Aug 9 13:50:21 2023","interfaces":[{"interface":"back","average":{"1min":1.288,"5min":1.3400000000000001,"15min":1.3300000000000001},"min":{"1min":0.876,"5min":0.73199999999999998,"15min":0.63200000000000001},"max":{"1min":1.734,"5min":1.8,"15min":2.0099999999999998},"last":1.758},{"interface":"front","average":{"1min":1.2509999999999999,"5min":1.323,"15min":1.3180000000000001},"min":{"1min":0.92000000000000004,"5min":0.72399999999999998,"15min":0.626},"max":{"1min":1.71,"5min":2.7090000000000001,"15min":2.7090000000000001},"last":1.821}]},{"osd":6,"last update":"Wed Aug 9 13:50:21 2023","interfaces":[{"interface":"back","average":{"1min":1.379,"5min":1.3160000000000001,"15min":1.3360000000000001},"min":{"1min":0.94699999999999995,"5min":0.74399999999999999,"15min":0.59099999999999997},"max":{"1min":1.7509999999999999,"5min":2.8559999999999999,"15min":2.8559999999999999},"last":1.883},{"interface":"front","average":{"1min":1.4430000000000001,"5min":1.518,"15min":1.504},"min":{"1min":1.0760000000000001,"5min":0.97799999999999998,"15min":0.76400000000000001},"max":{"1min":1.669,"5min":2.2719999999999998,"15min":2.2719999999999998},"last":2.1000000000000001}]},{"osd":7,"last update":"Wed Aug 9 13:50:21 2023","interfaces":[{"interface":"back","average":{"1min":1.46,"5min":1.4830000000000001,"15min":1.474},"min":{"1min":1.1499999999999999,"5min":0.85999999999999999,"15min":0.73899999999999999},"max":{"1min":1.8460000000000001,"5min":2.8050000000000002,"15min":2.8050000000000002},"last":1.9530000000000001},{"interface":"front","average":{"1min":1.474,"5min":1.522,"15min":1.514},"min":{"1min":1.236,"5min":0.97199999999999998,"15min":0.72899999999999998},"max":{"1min":1.754,"5min":3.415,"15min":3.415},"last":2.0339999999999998}]}]},{"osd":3,"up_from":13,"seq":55834575049,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":36292,"kb_used_data":1076,"kb_used_omap":1,"kb_used_meta":35198,"kb_avail":94335548,"statfs":{"total":96636764160,"available":96599601152,"internally_reserved":0,"allocated":1101824,"data_stored":583889,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1781,"internal_metadata":36043019},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Aug 9 13:50:25 2023","interfaces":[{"interface":"back","average":{"1min":0.84599999999999997,"5min":0.88300000000000001,"15min":0.91600000000000004},"min":{"1min":0.43099999999999999,"5min":0.27200000000000002,"15min":0.27200000000000002},"max":{"1min":1.3939999999999999,"5min":1.8500000000000001,"15min":1.968},"last":0.496},{"interface":"front","average":{"1min":1.071,"5min":1.0920000000000001,"15min":1.1060000000000001},"min":{"1min":0.48199999999999998,"5min":0.36399999999999999,"15min":0.36399999999999999},"max":{"1min":1.6910000000000001,"5min":1.6910000000000001,"15min":2.2360000000000002},"last":0.78900000000000003}]},{"osd":1,"last update":"Wed Aug 9 13:50:25 2023","interfaces":[{"interface":"back","average":{"1min":1.042,"5min":1.0089999999999999,"15min":1.012},"min":{"1min":0.59899999999999998,"5min":0.496,"15min":0.438},"max":{"1min":1.518,"5min":1.7110000000000001,"15min":2.5649999999999999},"last":0.63900000000000001},{"interface":"front","average":{"1min":1.091,"5min":1.1279999999999999,"15min":1.1639999999999999},"min":{"1min":0.51800000000000002,"5min":0.51800000000000002,"15min":0.42099999999999999},"max":{"1min":1.6839999999999999,"5min":1.6839999999999999,"15min":2.6760000000000002},"last":0.92900000000000005}]},{"osd":2,"last update":"Wed Aug 9 13:50:25 2023","interfaces":[{"interface":"back","average":{"1min":1.121,"5min":1.177,"15min":1.1930000000000001},"min":{"1min":0.53600000000000003,"5min":0.53600000000000003,"15min":0.53500000000000003},"max":{"1min":1.6240000000000001,"5min":1.913,"15min":1.923},"last":1.2330000000000001},{"interface":"front","average":{"1min":1.1899999999999999,"5min":1.2929999999999999,"15min":1.3080000000000001},"min":{"1min":0.625,"5min":0.52300000000000002,"15min":0.52000000000000002},"max":{"1min":1.9399999999999999,"5min":2.0259999999999998,"15min":2.0259999999999998},"last":0.73799999999999999}]},{"osd":4,"last update":"Wed Aug 9 13:50:25 2023","interfaces":[{"interface":"back","average":{"1min":1.232,"5min":1.258,"15min":1.282},"min":{"1min":0.91500000000000004,"5min":0.57899999999999996,"15min":0.57899999999999996},"max":{"1min":1.742,"5min":1.8160000000000001,"15min":1.8160000000000001},"last":0.96799999999999997},{"interface":"front","average":{"1min":1.349,"5min":1.397,"15min":1.3919999999999999},"min":{"1min":0.67600000000000005,"5min":0.628,"15min":0.628},"max":{"1min":1.8149999999999999,"5min":2.2040000000000002,"15min":2.2370000000000001},"last":1.1839999999999999}]},{"osd":5,"last update":"Wed Aug 9 13:50:25 2023","interfaces":[{"interface":"back","average":{"1min":1.456,"5min":1.468,"15min":1.474},"min":{"1min":0.67600000000000005,"5min":0.67600000000000005,"15min":0.67200000000000004},"max":{"1min":1.855,"5min":2.254,"15min":2.254},"last":1.04},{"interface":"front","average":{"1min":1.4850000000000001,"5min":1.4910000000000001,"15min":1.49},"min":{"1min":0.77600000000000002,"5min":0.65900000000000003,"15min":0.65900000000000003},"max":{"1min":2.3069999999999999,"5min":2.3069999999999999,"15min":2.3069999999999999},"last":1.0700000000000001}]},{"osd":6,"last update":"Wed Aug 9 13:50:25 2023","interfaces":[{"interface":"back","average":{"1min":1.3759999999999999,"5min":1.4370000000000001,"15min":1.4590000000000001},"min":{"1min":0.878,"5min":0.68200000000000005,"15min":0.68200000000000005},"max":{"1min":1.8759999999999999,"5min":2.1829999999999998,"15min":2.242},"last":1.3680000000000001},{"interface":"front","average":{"1min":1.466,"5min":1.5580000000000001,"15min":1.5529999999999999},"min":{"1min":1.0600000000000001,"5min":0.71099999999999997,"15min":0.68600000000000005},"max":{"1min":1.825,"5min":2.3420000000000001,"15min":2.3420000000000001},"last":1.099}]},{"osd":7,"last update":"Wed Aug 9 13:50:25 2023","interfaces":[{"interface":"back","average":{"1min":1.601,"5min":1.627,"15min":1.6160000000000001},"min":{"1min":1.115,"5min":0.69399999999999995,"15min":0.69399999999999995},"max":{"1min":2.46,"5min":2.46,"15min":2.46},"last":1.159},{"interface":"front","average":{"1min":1.4910000000000001,"5min":1.546,"15min":1.5600000000000001},"min":{"1min":0.78200000000000003,"5min":0.64200000000000002,"15min":0.61599999999999999},"max":{"1min":2.0579999999999998,"5min":2.359,"15min":2.484},"last":1.2709999999999999}]}]},{"osd":2,"up_from":13,"seq":55834575048,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":36300,"kb_used_data":1080,"kb_used_omap":1,"kb_used_meta":35198,"kb_avail":94335540,"statfs":{"total":96636764160,"available":96599592960,"internally_reserved":0,"allocated":1105920,"data_stored":585389,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1781,"internal_metadata":36043019},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Aug 9 13:50:17 2023","interfaces":[{"interface":"back","average":{"1min":0.96999999999999997,"5min":0.95299999999999996,"15min":0.95499999999999996},"min":{"1min":0.47799999999999998,"5min":0.36499999999999999,"15min":0.36499999999999999},"max":{"1min":1.5760000000000001,"5min":1.6180000000000001,"15min":1.8380000000000001},"last":0.91900000000000004},{"interface":"front","average":{"1min":0.97099999999999997,"5min":0.998,"15min":1.0049999999999999},"min":{"1min":0.45300000000000001,"5min":0.374,"15min":0.374},"max":{"1min":1.4199999999999999,"5min":2.3929999999999998,"15min":2.3929999999999998},"last":0.67400000000000004}]},{"osd":1,"last update":"Wed Aug 9 13:50:17 2023","interfaces":[{"interface":"back","average":{"1min":1.2529999999999999,"5min":1.1739999999999999,"15min":1.1870000000000001},"min":{"1min":0.61299999999999999,"5min":0.55200000000000005,"15min":0.55200000000000005},"max":{"1min":1.6870000000000001,"5min":1.77,"15min":1.8420000000000001},"last":0.72699999999999998},{"interface":"front","average":{"1min":1.216,"5min":1.2749999999999999,"15min":1.335},"min":{"1min":0.52500000000000002,"5min":0.52500000000000002,"15min":0.52500000000000002},"max":{"1min":1.7390000000000001,"5min":2.4870000000000001,"15min":2.4870000000000001},"last":0.626}]},{"osd":3,"last update":"Wed Aug 9 13:50:17 2023","interfaces":[{"interface":"back","average":{"1min":0.98099999999999998,"5min":1.006,"15min":1.0469999999999999},"min":{"1min":0.44800000000000001,"5min":0.38900000000000001,"15min":0.38900000000000001},"max":{"1min":1.3109999999999999,"5min":1.708,"15min":2.0409999999999999},"last":1.417},{"interface":"front","average":{"1min":1.2110000000000001,"5min":1.304,"15min":1.3089999999999999},"min":{"1min":0.55500000000000005,"5min":0.46200000000000002,"15min":0.46200000000000002},"max":{"1min":1.8009999999999999,"5min":8.8130000000000006,"15min":8.8130000000000006},"last":1.28}]},{"osd":4,"last update":"Wed Aug 9 13:50:17 2023","interfaces":[{"interface":"back","average":{"1min":1.415,"5min":1.4179999999999999,"15min":1.4510000000000001},"min":{"1min":0.83999999999999997,"5min":0.628,"15min":0.628},"max":{"1min":1.958,"5min":2.089,"15min":2.089},"last":1.3640000000000001},{"interface":"front","average":{"1min":1.462,"5min":1.359,"15min":1.3979999999999999},"min":{"1min":0.73799999999999999,"5min":0.68600000000000005,"15min":0.68600000000000005},"max":{"1min":2.472,"5min":2.472,"15min":2.472},"last":1.327}]},{"osd":5,"last update":"Wed Aug 9 13:50:17 2023","interfaces":[{"interface":"back","average":{"1min":1.5189999999999999,"5min":1.4379999999999999,"15min":1.458},"min":{"1min":0.76600000000000001,"5min":0.59999999999999998,"15min":0.59999999999999998},"max":{"1min":2.27,"5min":2.5840000000000001,"15min":2.5840000000000001},"last":1.468},{"interface":"front","average":{"1min":1.304,"5min":1.3540000000000001,"15min":1.349},"min":{"1min":0.77100000000000002,"5min":0.56799999999999995,"15min":0.56799999999999995},"max":{"1min":1.8859999999999999,"5min":3.5659999999999998,"15min":3.5659999999999998},"last":1.107}]},{"osd":6,"last update":"Wed Aug 9 13:50:17 2023","interfaces":[{"interface":"back","average":{"1min":1.4019999999999999,"5min":1.425,"15min":1.4259999999999999},"min":{"1min":0.79800000000000004,"5min":0.73799999999999999,"15min":0.73799999999999999},"max":{"1min":2.222,"5min":3.5099999999999998,"15min":3.5099999999999998},"last":1.538},{"interface":"front","average":{"1min":1.4790000000000001,"5min":1.4990000000000001,"15min":1.5409999999999999},"min":{"1min":0.82199999999999995,"5min":0.66300000000000003,"15min":0.66300000000000003},"max":{"1min":2.3769999999999998,"5min":2.3769999999999998,"15min":2.3769999999999998},"last":1.502}]},{"osd":7,"last update":"Wed Aug 9 13:50:17 2023","interfaces":[{"interface":"back","average":{"1min":1.54,"5min":1.51,"15min":1.548},"min":{"1min":0.78800000000000003,"5min":0.64600000000000002,"15min":0.64600000000000002},"max":{"1min":2.1970000000000001,"5min":2.5249999999999999,"15min":2.5249999999999999},"last":1.3899999999999999},{"interface":"front","average":{"1min":1.4970000000000001,"5min":1.52,"15min":1.51},"min":{"1min":0.99199999999999999,"5min":0.70499999999999996,"15min":0.70499999999999996},"max":{"1min":2.089,"5min":3.609,"15min":3.609},"last":1.647}]}]},{"osd":1,"up_from":13,"seq":55834575048,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28696,"kb_used_data":1668,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343144,"statfs":{"total":96636764160,"available":96607379456,"internally_reserved":0,"allocated":1708032,"data_stored":1176373,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1781,"internal_metadata":27654411},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Aug 9 13:50:31 2023","interfaces":[{"interface":"back","average":{"1min":0.99099999999999999,"5min":0.92000000000000004,"15min":0.93999999999999995},"min":{"1min":0.50600000000000001,"5min":0.50600000000000001,"15min":0.32400000000000001},"max":{"1min":1.7110000000000001,"5min":1.7110000000000001,"15min":2.1080000000000001},"last":0.66600000000000004},{"interface":"front","average":{"1min":0.98999999999999999,"5min":1.0740000000000001,"15min":1.095},"min":{"1min":0.59799999999999998,"5min":0.432,"15min":0.42499999999999999},"max":{"1min":1.7190000000000001,"5min":2.468,"15min":2.468},"last":0.63400000000000001}]},{"osd":2,"last update":"Wed Aug 9 13:50:31 2023","interfaces":[{"interface":"back","average":{"1min":1.008,"5min":0.97799999999999998,"15min":0.995},"min":{"1min":0.56699999999999995,"5min":0.41099999999999998,"15min":0.41099999999999998},"max":{"1min":1.6679999999999999,"5min":1.907,"15min":1.907},"last":0.60799999999999998},{"interface":"front","average":{"1min":1.1459999999999999,"5min":1.159,"15min":1.1879999999999999},"min":{"1min":0.80600000000000005,"5min":0.54400000000000004,"15min":0.46500000000000002},"max":{"1min":1.5189999999999999,"5min":2.1280000000000001,"15min":2.145},"last":0.81999999999999995}]},{"osd":3,"last update":"Wed Aug 9 13:50:31 2023","interfaces":[{"interface":"back","average":{"1min":1.218,"5min":1.2629999999999999,"15min":1.3260000000000001},"min":{"1min":0.63200000000000001,"5min":0.54400000000000004,"15min":0.432},"max":{"1min":1.7430000000000001,"5min":2.0270000000000001,"15min":2.3210000000000002},"last":0.86199999999999999},{"interface":"front","average":{"1min":1.1779999999999999,"5min":1.234,"15min":1.2450000000000001},"min":{"1min":0.70399999999999996,"5min":0.49299999999999999,"15min":0.44400000000000001},"max":{"1min":1.786,"5min":2.3540000000000001,"15min":2.7669999999999999},"last":0.72899999999999998}]},{"osd":4,"last update":"Wed Aug 9 13:50:31 2023","interfaces":[{"interface":"back","average":{"1min":1.3149999999999999,"5min":1.399,"15min":1.4079999999999999},"min":{"1min":0.94399999999999995,"5min":0.79800000000000004,"15min":0.58099999999999996},"max":{"1min":1.696,"5min":1.97,"15min":2.5},"last":1.173},{"interface":"front","average":{"1min":1.2689999999999999,"5min":1.3180000000000001,"15min":1.333},"min":{"1min":0.88200000000000001,"5min":0.70099999999999996,"15min":0.70099999999999996},"max":{"1min":1.6950000000000001,"5min":1.901,"15min":1.901},"last":0.80600000000000005}]},{"osd":5,"last update":"Wed Aug 9 13:50:31 2023","interfaces":[{"interface":"back","average":{"1min":1.3200000000000001,"5min":1.323,"15min":1.371},"min":{"1min":0.90700000000000003,"5min":0.76900000000000002,"15min":0.76900000000000002},"max":{"1min":1.837,"5min":2.294,"15min":2.294},"last":0.76800000000000002},{"interface":"front","average":{"1min":1.427,"5min":1.464,"15min":1.516},"min":{"1min":0.97899999999999998,"5min":0.91000000000000003,"15min":0.77700000000000002},"max":{"1min":1.837,"5min":2.1389999999999998,"15min":2.4649999999999999},"last":1.2010000000000001}]},{"osd":6,"last update":"Wed Aug 9 13:50:31 2023","interfaces":[{"interface":"back","average":{"1min":1.4219999999999999,"5min":1.48,"15min":1.514},"min":{"1min":1.026,"5min":0.77500000000000002,"15min":0.67400000000000004},"max":{"1min":1.925,"5min":2.2080000000000002,"15min":2.5270000000000001},"last":1.004},{"interface":"front","average":{"1min":1.4370000000000001,"5min":1.45,"15min":1.5},"min":{"1min":1.1379999999999999,"5min":0.82499999999999996,"15min":0.79800000000000004},"max":{"1min":2.02,"5min":2.02,"15min":2.1040000000000001},"last":0.92400000000000004}]},{"osd":7,"last update":"Wed Aug 9 13:50:31 2023","interfaces":[{"interface":"back","average":{"1min":1.552,"5min":1.5649999999999999,"15min":1.619},"min":{"1min":1.0389999999999999,"5min":0.86899999999999999,"15min":0.77100000000000002},"max":{"1min":2.4169999999999998,"5min":2.4169999999999998,"15min":2.5790000000000002},"last":1.252},{"interface":"front","average":{"1min":1.5740000000000001,"5min":1.5549999999999999,"15min":1.6000000000000001},"min":{"1min":1.2589999999999999,"5min":0.91300000000000003,"15min":0.75900000000000001},"max":{"1min":2.036,"5min":2.1040000000000001,"15min":2.6509999999999998},"last":0.96899999999999997}]}]},{"osd":0,"up_from":13,"seq":55834575048,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":40380,"kb_used_data":1064,"kb_used_omap":1,"kb_used_meta":39294,"kb_avail":94331460,"statfs":{"total":96636764160,"available":96595415040,"internally_reserved":0,"allocated":1089536,"data_stored":582416,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1781,"internal_metadata":40237323},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Wed Aug 9 13:50:27 2023","interfaces":[{"interface":"back","average":{"1min":0.91800000000000004,"5min":0.92200000000000004,"15min":0.91600000000000004},"min":{"1min":0.32500000000000001,"5min":0.32500000000000001,"15min":0.32500000000000001},"max":{"1min":1.5820000000000001,"5min":1.756,"15min":1.756},"last":0.76400000000000001},{"interface":"front","average":{"1min":1.0680000000000001,"5min":1.1020000000000001,"15min":1.113},"min":{"1min":0.47099999999999997,"5min":0.45300000000000001,"15min":0.45300000000000001},"max":{"1min":1.9019999999999999,"5min":1.9019999999999999,"15min":1.9019999999999999},"last":1.171}]},{"osd":2,"last update":"Wed Aug 9 13:50:27 2023","interfaces":[{"interface":"back","average":{"1min":1.048,"5min":1.1000000000000001,"15min":1.135},"min":{"1min":0.53800000000000003,"5min":0.51400000000000001,"15min":0.44800000000000001},"max":{"1min":1.702,"5min":1.776,"15min":1.889},"last":0.95199999999999996},{"interface":"front","average":{"1min":1.1819999999999999,"5min":1.149,"15min":1.179},"min":{"1min":0.58499999999999996,"5min":0.56399999999999995,"15min":0.52700000000000002},"max":{"1min":2.0129999999999999,"5min":2.0129999999999999,"15min":2.2519999999999998},"last":1.3620000000000001}]},{"osd":3,"last update":"Wed Aug 9 13:50:27 2023","interfaces":[{"interface":"back","average":{"1min":0.98499999999999999,"5min":1.0509999999999999,"15min":1.105},"min":{"1min":0.54900000000000004,"5min":0.46300000000000002,"15min":0.46300000000000002},"max":{"1min":1.345,"5min":1.823,"15min":2.5190000000000001},"last":1.1040000000000001},{"interface":"front","average":{"1min":1.272,"5min":1.2629999999999999,"15min":1.3140000000000001},"min":{"1min":0.56399999999999995,"5min":0.54700000000000004,"15min":0.54700000000000004},"max":{"1min":2.347,"5min":2.347,"15min":2.4649999999999999},"last":1.0740000000000001}]},{"osd":4,"last update":"Wed Aug 9 13:50:27 2023","interfaces":[{"interface":"back","average":{"1min":1.302,"5min":1.3129999999999999,"15min":1.347},"min":{"1min":0.84699999999999998,"5min":0.84699999999999998,"15min":0.74299999999999999},"max":{"1min":1.9630000000000001,"5min":2.2309999999999999,"15min":2.2309999999999999},"last":1.323},{"interface":"front","average":{"1min":1.3999999999999999,"5min":1.4390000000000001,"15min":1.452},"min":{"1min":0.82699999999999996,"5min":0.77600000000000002,"15min":0.74099999999999999},"max":{"1min":1.8440000000000001,"5min":2.5390000000000001,"15min":2.5390000000000001},"last":1.248}]},{"osd":5,"last update":"Wed Aug 9 13:50:27 2023","interfaces":[{"interface":"back","average":{"1min":1.387,"5min":1.4219999999999999,"15min":1.4650000000000001},"min":{"1min":0.66300000000000003,"5min":0.66300000000000003,"15min":0.66300000000000003},"max":{"1min":2.6669999999999998,"5min":2.6669999999999998,"15min":2.6669999999999998},"last":1.649},{"interface":"front","average":{"1min":1.3400000000000001,"5min":1.415,"15min":1.4490000000000001},"min":{"1min":0.70099999999999996,"5min":0.629,"15min":0.629},"max":{"1min":1.6160000000000001,"5min":2.1499999999999999,"15min":2.1499999999999999},"last":1.444}]},{"osd":6,"last update":"Wed Aug 9 13:50:27 2023","interfaces":[{"interface":"back","average":{"1min":1.51,"5min":1.532,"15min":1.5469999999999999},"min":{"1min":0.73599999999999999,"5min":0.73599999999999999,"15min":0.69799999999999995},"max":{"1min":2.3100000000000001,"5min":2.7250000000000001,"15min":2.7250000000000001},"last":1.397},{"interface":"front","average":{"1min":1.5169999999999999,"5min":1.5389999999999999,"15min":1.5600000000000001},"min":{"1min":0.86699999999999999,"5min":0.86699999999999999,"15min":0.86699999999999999},"max":{"1min":2.3199999999999998,"5min":2.399,"15min":2.399},"last":1.4670000000000001}]},{"osd":7,"last update":"Wed Aug 9 13:50:27 2023","interfaces":[{"interface":"back","average":{"1min":1.532,"5min":1.595,"15min":1.625},"min":{"1min":0.72299999999999998,"5min":0.72299999999999998,"15min":0.72299999999999998},"max":{"1min":2.3900000000000001,"5min":2.3900000000000001,"15min":2.464},"last":1.5600000000000001},{"interface":"front","average":{"1min":1.4350000000000001,"5min":1.512,"15min":1.55},"min":{"1min":0.84499999999999997,"5min":0.76700000000000002,"15min":0.76700000000000002},"max":{"1min":2.274,"5min":2.274,"15min":2.274},"last":1.5049999999999999}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":2228,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":112,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":70,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":92,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":616,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-08-09T13:50:39.382 INFO:tasks.cephfs.filesystem.ceph_manager:clean! 2023-08-09T13:50:39.383 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth caps client.0 mon 'allow r' osd 'allow rw tag cephfs data=cephfs' mds allow 2023-08-09T13:50:39.660 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:39.659+0000 7f2c39c05640 1 Processor -- start 2023-08-09T13:50:39.661 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:39.659+0000 7f2c39c05640 1 -- start start 2023-08-09T13:50:39.662 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:39.661+0000 7f2c39c05640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f2c3415c2d0 0x7f2c340b7df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:50:39.662 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:39.661+0000 7f2c39c05640 1 -- --> v1:172.21.15.43:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2c340d16a0 con 0x7f2c340d1b30 2023-08-09T13:50:39.663 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:39.661+0000 7f2c39c05640 1 -- --> v1:172.21.15.203:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2c340d1f70 con 0x7f2c340cebf0 2023-08-09T13:50:39.663 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:39.661+0000 7f2c39c05640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2c340d0d70 con 0x7f2c3415c2d0 2023-08-09T13:50:39.666 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:39.661+0000 7f2c33fff640 1 --1- >> v1:172.21.15.43:6789/0 conn(0x7f2c340d1b30 0x7f2c340cd020 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.43:6789/0 says I am v1:172.21.15.43:55216/0 (socket says 172.21.15.43:55216) 2023-08-09T13:50:39.666 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:39.661+0000 7f2c33fff640 1 -- 172.21.15.43:0/821424675 learned_addr learned my addr 172.21.15.43:0/821424675 (peer_addr_for_me v1:172.21.15.43:0/0) 2023-08-09T13:50:39.666 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:39.661+0000 7f2c38c03640 1 --2- 172.21.15.43:0/821424675 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f2c3415c2d0 0x7f2c340b7df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:50:39.667 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:39.662+0000 7f2c337fe640 1 -- 172.21.15.43:0/821424675 <== mon.0 v1:172.21.15.43:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1998173677 0 0) 0x7f2c340d16a0 con 0x7f2c340d1b30 2023-08-09T13:50:39.667 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:39.662+0000 7f2c337fe640 1 -- 172.21.15.43:0/821424675 --> v1:172.21.15.43:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f2c24003520 con 0x7f2c340d1b30 2023-08-09T13:50:39.668 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:39.662+0000 7f2c337fe640 1 -- 172.21.15.43:0/821424675 <== mon.0 v1:172.21.15.43:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2817700097 0 0) 0x7f2c24003520 con 0x7f2c340d1b30 2023-08-09T13:50:39.668 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:39.662+0000 7f2c337fe640 1 -- 172.21.15.43:0/821424675 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f2c3415c2d0 msgr2=0x7f2c340b7df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-09T13:50:39.668 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:39.662+0000 7f2c337fe640 1 --2- 172.21.15.43:0/821424675 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f2c3415c2d0 0x7f2c340b7df0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:50:39.668 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:39.662+0000 7f2c337fe640 1 -- 172.21.15.43:0/821424675 >> v1:172.21.15.203:6789/0 conn(0x7f2c340cebf0 legacy=0x7f2c340b5270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:50:39.669 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:39.662+0000 7f2c337fe640 1 -- 172.21.15.43:0/821424675 --> v1:172.21.15.43:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2c340d1050 con 0x7f2c340d1b30 2023-08-09T13:50:39.669 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:39.662+0000 7f2c337fe640 1 -- 172.21.15.43:0/821424675 <== mon.0 v1:172.21.15.43:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1285150740 0 0) 0x7f2c20001670 con 0x7f2c340d1b30 2023-08-09T13:50:39.669 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:39.662+0000 7f2c337fe640 1 -- 172.21.15.43:0/821424675 >> v1:172.21.15.43:6789/0 conn(0x7f2c340d1b30 legacy=0x7f2c340cd020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:50:39.669 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:39.662+0000 7f2c337fe640 1 --2- 172.21.15.43:0/821424675 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f2c3415c2d0 0x7f2c240039a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:50:39.669 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:39.662+0000 7f2c337fe640 1 --2- 172.21.15.43:0/821424675 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f2c24003f20 0x7f2c240047c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:50:39.669 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:39.662+0000 7f2c337fe640 1 --2- 172.21.15.43:0/821424675 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f2c24004d00 0x7f2c24005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:50:39.669 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:39.662+0000 7f2c337fe640 1 -- 172.21.15.43:0/821424675 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2c340d0d70 con 0x7f2c24004d00 2023-08-09T13:50:39.670 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:39.662+0000 7f2c337fe640 1 -- 172.21.15.43:0/821424675 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2c2400ba60 con 0x7f2c24003f20 2023-08-09T13:50:39.670 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:39.662+0000 7f2c337fe640 1 -- 172.21.15.43:0/821424675 --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2c2400bc20 con 0x7f2c3415c2d0 2023-08-09T13:50:39.670 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:39.663+0000 7f2c39404640 1 --2- 172.21.15.43:0/821424675 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f2c24004d00 0x7f2c24005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:50:39.670 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:39.663+0000 7f2c38c03640 1 --2- 172.21.15.43:0/821424675 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f2c3415c2d0 0x7f2c240039a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:50:39.670 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:39.663+0000 7f2c33fff640 1 --2- 172.21.15.43:0/821424675 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f2c24003f20 0x7f2c240047c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:50:39.670 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:39.663+0000 7f2c39404640 1 -- 172.21.15.43:0/821424675 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f2c3415c2d0 msgr2=0x7f2c240039a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-09T13:50:39.670 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:39.663+0000 7f2c39404640 1 --2- 172.21.15.43:0/821424675 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f2c3415c2d0 0x7f2c240039a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:50:39.671 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:39.663+0000 7f2c39404640 1 -- 172.21.15.43:0/821424675 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f2c24003f20 msgr2=0x7f2c240047c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-09T13:50:39.671 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:39.663+0000 7f2c39404640 1 --2- 172.21.15.43:0/821424675 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f2c24003f20 0x7f2c240047c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:50:39.671 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:39.663+0000 7f2c39404640 1 -- 172.21.15.43:0/821424675 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2c24003520 con 0x7f2c24004d00 2023-08-09T13:50:39.671 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:39.664+0000 7f2c39404640 1 --2- 172.21.15.43:0/821424675 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f2c24004d00 0x7f2c24005180 secure :-1 s=READY pgs=1033 cs=0 l=1 rev1=1 crypto rx=0x7f2c2c001e50 tx=0x7f2c240057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=753440119de53a06 server_cookie=0 in_seq=0 out_seq=0 2023-08-09T13:50:39.671 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:39.665+0000 7f2c337fe640 1 -- 172.21.15.43:0/821424675 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2c2c003b70 con 0x7f2c24004d00 2023-08-09T13:50:39.671 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:39.665+0000 7f2c337fe640 1 -- 172.21.15.43:0/821424675 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2c2c003cd0 con 0x7f2c24004d00 2023-08-09T13:50:39.671 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:39.665+0000 7f2c337fe640 1 -- 172.21.15.43:0/821424675 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2c2c005310 con 0x7f2c24004d00 2023-08-09T13:50:39.674 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:39.665+0000 7f2c39c05640 1 -- 172.21.15.43:0/821424675 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f2c24004d00 msgr2=0x7f2c24005180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:50:39.674 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:39.665+0000 7f2c39c05640 1 --2- 172.21.15.43:0/821424675 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f2c24004d00 0x7f2c24005180 secure :-1 s=READY pgs=1033 cs=0 l=1 rev1=1 crypto rx=0x7f2c2c001e50 tx=0x7f2c240057f0 comp rx=0 tx=0).stop 2023-08-09T13:50:39.674 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:39.665+0000 7f2c39404640 1 -- 172.21.15.43:0/821424675 reap_dead start 2023-08-09T13:50:39.674 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:39.665+0000 7f2c39c05640 1 -- 172.21.15.43:0/821424675 shutdown_connections 2023-08-09T13:50:39.674 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:39.665+0000 7f2c39c05640 1 -- 172.21.15.43:0/821424675 >> 172.21.15.43:0/821424675 conn(0x7f2c3406a230 msgr2=0x7f2c3406a640 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-09T13:50:39.675 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:39.666+0000 7f2c39c05640 1 -- 172.21.15.43:0/821424675 shutdown_connections 2023-08-09T13:50:39.675 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:39.666+0000 7f2c39c05640 1 -- 172.21.15.43:0/821424675 wait complete. 2023-08-09T13:50:39.675 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:39.667+0000 7f2c39c05640 1 Processor -- start 2023-08-09T13:50:39.675 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:39.668+0000 7f2c39c05640 1 -- start start 2023-08-09T13:50:39.675 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:39.668+0000 7f2c39c05640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f2c24004d00 0x7f2c341569e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:50:39.675 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:39.668+0000 7f2c39c05640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f2c34156060 0x7f2c34158b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:50:39.675 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:39.668+0000 7f2c39c05640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f2c34159090 0x7f2c340b5270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:50:39.676 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:39.669+0000 7f2c39c05640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2c34156fc0 con 0x7f2c34156060 2023-08-09T13:50:39.676 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:39.669+0000 7f2c39c05640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2c341f1cf0 con 0x7f2c24004d00 2023-08-09T13:50:39.676 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:39.669+0000 7f2c39c05640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2c341f1fd0 con 0x7f2c34159090 2023-08-09T13:50:39.676 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:39.669+0000 7f2c33fff640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f2c34156060 0x7f2c34158b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:50:39.676 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:39.669+0000 7f2c33fff640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f2c34156060 0x7f2c34158b50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:45028/0 (socket says 172.21.15.43:45028) 2023-08-09T13:50:39.676 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:39.669+0000 7f2c33fff640 1 -- 172.21.15.43:0/3556817550 learned_addr learned my addr 172.21.15.43:0/3556817550 (peer_addr_for_me v2:172.21.15.43:0/0) 2023-08-09T13:50:39.676 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:39.669+0000 7f2c39404640 1 --2- 172.21.15.43:0/3556817550 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f2c34159090 0x7f2c340b5270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:50:39.677 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:39.669+0000 7f2c33fff640 1 -- 172.21.15.43:0/3556817550 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f2c34159090 msgr2=0x7f2c340b5270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:50:39.677 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:39.669+0000 7f2c33fff640 1 --2- 172.21.15.43:0/3556817550 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f2c34159090 0x7f2c340b5270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:50:39.677 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:39.670+0000 7f2c33fff640 1 -- 172.21.15.43:0/3556817550 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f2c24004d00 msgr2=0x7f2c341569e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:50:39.677 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:39.670+0000 7f2c38c03640 1 --2- 172.21.15.43:0/3556817550 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f2c24004d00 0x7f2c341569e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:50:39.677 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:39.670+0000 7f2c33fff640 1 --2- 172.21.15.43:0/3556817550 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f2c24004d00 0x7f2c341569e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:50:39.677 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:39.670+0000 7f2c33fff640 1 -- 172.21.15.43:0/3556817550 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2c341f22b0 con 0x7f2c34156060 2023-08-09T13:50:39.677 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:39.670+0000 7f2c33fff640 1 --2- 172.21.15.43:0/3556817550 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f2c34156060 0x7f2c34158b50 secure :-1 s=READY pgs=1034 cs=0 l=1 rev1=1 crypto rx=0x7f2c2000e820 tx=0x7f2c2000ece0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-09T13:50:39.678 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:39.670+0000 7f2c317fa640 1 -- 172.21.15.43:0/3556817550 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2c2000c870 con 0x7f2c34156060 2023-08-09T13:50:39.678 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:39.670+0000 7f2c317fa640 1 -- 172.21.15.43:0/3556817550 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2c2000c9d0 con 0x7f2c34156060 2023-08-09T13:50:39.678 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:39.670+0000 7f2c317fa640 1 -- 172.21.15.43:0/3556817550 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2c2000cd80 con 0x7f2c34156060 2023-08-09T13:50:39.678 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:39.670+0000 7f2c39c05640 1 -- 172.21.15.43:0/3556817550 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2c341f19a0 con 0x7f2c34156060 2023-08-09T13:50:39.678 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:39.671+0000 7f2c39c05640 1 -- 172.21.15.43:0/3556817550 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2c340b57b0 con 0x7f2c34156060 2023-08-09T13:50:39.678 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:39.672+0000 7f2c39c05640 1 -- 172.21.15.43:0/3556817550 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2c340c8c60 con 0x7f2c34156060 2023-08-09T13:50:39.679 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:39.673+0000 7f2c317fa640 1 -- 172.21.15.43:0/3556817550 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 29) v1 ==== 88443+0+0 (secure 0 0 0) 0x7f2c2001d460 con 0x7f2c34156060 2023-08-09T13:50:39.679 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:39.674+0000 7f2c317fa640 1 --2- 172.21.15.43:0/3556817550 >> [v2:172.21.15.203:6832/3325141227,v1:172.21.15.203:6833/3325141227] conn(0x7f2c1806a170 0x7f2c1806c630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:50:39.679 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:39.674+0000 7f2c317fa640 1 -- 172.21.15.43:0/3556817550 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(90..90 src has 1..90) v4 ==== 6426+0+0 (secure 0 0 0) 0x7f2c20088970 con 0x7f2c34156060 2023-08-09T13:50:39.679 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:39.675+0000 7f2c38c03640 1 --2- 172.21.15.43:0/3556817550 >> [v2:172.21.15.203:6832/3325141227,v1:172.21.15.203:6833/3325141227] conn(0x7f2c1806a170 0x7f2c1806c630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:50:39.680 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:39.676+0000 7f2c38c03640 1 --2- 172.21.15.43:0/3556817550 >> [v2:172.21.15.203:6832/3325141227,v1:172.21.15.203:6833/3325141227] conn(0x7f2c1806a170 0x7f2c1806c630 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f2c28009950 tx=0x7f2c28008040 comp rx=0 tx=0).ready entity=mgr.5519 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-09T13:50:39.680 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:39.678+0000 7f2c317fa640 1 -- 172.21.15.43:0/3556817550 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178066 (secure 0 0 0) 0x7f2c20053de0 con 0x7f2c34156060 2023-08-09T13:50:39.964 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:39.963+0000 7f2c39c05640 1 -- 172.21.15.43:0/3556817550 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]} v 0) v1 -- 0x7f2c34157100 con 0x7f2c34156060 2023-08-09T13:50:39.973 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:39.971+0000 7f2c317fa640 1 -- 172.21.15.43:0/3556817550 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]}]=0 updated caps for client.0 v15) v1 ==== 190+0+147 (secure 0 0 0) 0x7f2c20057400 con 0x7f2c34156060 2023-08-09T13:50:39.973 INFO:teuthology.orchestra.run.smithi043.stdout:[client.0] 2023-08-09T13:50:39.973 INFO:teuthology.orchestra.run.smithi043.stdout: key = AQCPldNkg8zrDxAAM/hnJaE6S9HU/fRwQeBV5w== 2023-08-09T13:50:39.974 INFO:teuthology.orchestra.run.smithi043.stdout: caps mds = "allow" 2023-08-09T13:50:39.974 INFO:teuthology.orchestra.run.smithi043.stdout: caps mon = "allow r" 2023-08-09T13:50:39.974 INFO:teuthology.orchestra.run.smithi043.stdout: caps osd = "allow rw tag cephfs data=cephfs" 2023-08-09T13:50:39.974 INFO:teuthology.orchestra.run.smithi043.stderr:updated caps for client.0 2023-08-09T13:50:39.975 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:39.974+0000 7f2c39c05640 1 -- 172.21.15.43:0/3556817550 >> [v2:172.21.15.203:6832/3325141227,v1:172.21.15.203:6833/3325141227] conn(0x7f2c1806a170 msgr2=0x7f2c1806c630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:50:39.976 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:39.974+0000 7f2c39c05640 1 --2- 172.21.15.43:0/3556817550 >> [v2:172.21.15.203:6832/3325141227,v1:172.21.15.203:6833/3325141227] conn(0x7f2c1806a170 0x7f2c1806c630 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f2c28009950 tx=0x7f2c28008040 comp rx=0 tx=0).stop 2023-08-09T13:50:39.976 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:39.974+0000 7f2c39c05640 1 -- 172.21.15.43:0/3556817550 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f2c34156060 msgr2=0x7f2c34158b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:50:39.976 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:39.974+0000 7f2c39c05640 1 --2- 172.21.15.43:0/3556817550 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f2c34156060 0x7f2c34158b50 secure :-1 s=READY pgs=1034 cs=0 l=1 rev1=1 crypto rx=0x7f2c2000e820 tx=0x7f2c2000ece0 comp rx=0 tx=0).stop 2023-08-09T13:50:39.976 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:39.974+0000 7f2c39c05640 1 -- 172.21.15.43:0/3556817550 shutdown_connections 2023-08-09T13:50:39.977 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:39.974+0000 7f2c39c05640 1 --2- 172.21.15.43:0/3556817550 >> [v2:172.21.15.203:6832/3325141227,v1:172.21.15.203:6833/3325141227] conn(0x7f2c1806a170 0x7f2c1806c630 unknown :-1 s=CLOSED pgs=71 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:50:39.977 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:39.974+0000 7f2c39c05640 1 --2- 172.21.15.43:0/3556817550 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f2c34159090 0x7f2c340b5270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:50:39.977 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:39.974+0000 7f2c39c05640 1 --2- 172.21.15.43:0/3556817550 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f2c34156060 0x7f2c34158b50 unknown :-1 s=CLOSED pgs=1034 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:50:39.977 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:39.974+0000 7f2c39c05640 1 --2- 172.21.15.43:0/3556817550 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f2c24004d00 0x7f2c341569e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:50:39.977 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:39.974+0000 7f2c39c05640 1 -- 172.21.15.43:0/3556817550 >> 172.21.15.43:0/3556817550 conn(0x7f2c3406a230 msgr2=0x7f2c340cfaf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-09T13:50:39.977 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:39.974+0000 7f2c39c05640 1 -- 172.21.15.43:0/3556817550 shutdown_connections 2023-08-09T13:50:39.978 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:39.974+0000 7f2c39c05640 1 -- 172.21.15.43:0/3556817550 wait complete. 2023-08-09T13:50:39.996 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-08-09T13:50:40.220 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:40.219+0000 7f032cd5b640 1 Processor -- start 2023-08-09T13:50:40.221 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:40.219+0000 7f032cd5b640 1 -- start start 2023-08-09T13:50:40.223 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:40.221+0000 7f032cd5b640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f03280c86f0 0x7f03280c8b50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:50:40.223 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:40.221+0000 7f032cd5b640 1 -- --> v1:172.21.15.43:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f03280c9090 con 0x7f0328143c20 2023-08-09T13:50:40.223 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:40.221+0000 7f032cd5b640 1 -- --> v1:172.21.15.203:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f032806c440 con 0x7f032806c8d0 2023-08-09T13:50:40.224 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:40.221+0000 7f032cd5b640 1 -- --> v2:172.21.15.43:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0328144000 con 0x7f03280c86f0 2023-08-09T13:50:40.224 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:40.221+0000 7f0325d9b640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f03280c86f0 0x7f03280c8b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:50:40.224 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:40.221+0000 7f032659c640 1 --1- >> v1:172.21.15.43:6789/0 conn(0x7f0328143c20 0x7f032806f540 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.43:6789/0 says I am v1:172.21.15.43:55218/0 (socket says 172.21.15.43:55218) 2023-08-09T13:50:40.224 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:40.221+0000 7f032659c640 1 -- 172.21.15.43:0/239634148 learned_addr learned my addr 172.21.15.43:0/239634148 (peer_addr_for_me v1:172.21.15.43:0/0) 2023-08-09T13:50:40.224 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:40.222+0000 7f0325d9b640 1 -- 172.21.15.43:0/239634148 >> v1:172.21.15.203:6789/0 conn(0x7f032806c8d0 legacy=0x7f032806cd00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-09T13:50:40.224 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:40.222+0000 7f0325d9b640 1 -- 172.21.15.43:0/239634148 >> v1:172.21.15.43:6789/0 conn(0x7f0328143c20 legacy=0x7f032806f540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:50:40.225 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:40.222+0000 7f0325d9b640 1 -- 172.21.15.43:0/239634148 --> v2:172.21.15.43:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f032806fc50 con 0x7f03280c86f0 2023-08-09T13:50:40.225 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:40.222+0000 7f032559a640 1 -- 172.21.15.43:0/239634148 <== mon.0 v1:172.21.15.43:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 145329517 0 0) 0x7f03280c9090 con 0x7f0328143c20 2023-08-09T13:50:40.226 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:40.223+0000 7f0325d9b640 1 --2- 172.21.15.43:0/239634148 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f03280c86f0 0x7f03280c8b50 secure :-1 s=READY pgs=1036 cs=0 l=1 rev1=1 crypto rx=0x7f031c0097a0 tx=0x7f031c02b7a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b6b09f9330b963b9 server_cookie=0 in_seq=0 out_seq=0 2023-08-09T13:50:40.226 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:40.223+0000 7f032559a640 1 -- 172.21.15.43:0/239634148 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f031c02f050 con 0x7f03280c86f0 2023-08-09T13:50:40.226 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:40.223+0000 7f032559a640 1 -- 172.21.15.43:0/239634148 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f031c031700 con 0x7f03280c86f0 2023-08-09T13:50:40.226 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:40.223+0000 7f032559a640 1 -- 172.21.15.43:0/239634148 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f031c031a90 con 0x7f03280c86f0 2023-08-09T13:50:40.226 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:40.224+0000 7f032cd5b640 1 -- 172.21.15.43:0/239634148 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f03280c86f0 msgr2=0x7f03280c8b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:50:40.226 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:40.224+0000 7f032cd5b640 1 --2- 172.21.15.43:0/239634148 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f03280c86f0 0x7f03280c8b50 secure :-1 s=READY pgs=1036 cs=0 l=1 rev1=1 crypto rx=0x7f031c0097a0 tx=0x7f031c02b7a0 comp rx=0 tx=0).stop 2023-08-09T13:50:40.226 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:40.224+0000 7f032cd5b640 1 -- 172.21.15.43:0/239634148 shutdown_connections 2023-08-09T13:50:40.227 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:40.224+0000 7f032cd5b640 1 --2- 172.21.15.43:0/239634148 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f03280c86f0 0x7f03280c8b50 unknown :-1 s=CLOSED pgs=1036 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:50:40.227 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:40.224+0000 7f032cd5b640 1 -- 172.21.15.43:0/239634148 >> 172.21.15.43:0/239634148 conn(0x7f03280ce0a0 msgr2=0x7f03280ce480 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-09T13:50:40.227 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:40.224+0000 7f032cd5b640 1 -- 172.21.15.43:0/239634148 shutdown_connections 2023-08-09T13:50:40.228 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:40.224+0000 7f032cd5b640 1 -- 172.21.15.43:0/239634148 wait complete. 2023-08-09T13:50:40.229 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:40.227+0000 7f032cd5b640 1 Processor -- start 2023-08-09T13:50:40.229 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:40.227+0000 7f032cd5b640 1 -- start start 2023-08-09T13:50:40.230 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:40.228+0000 7f032cd5b640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f032806c8d0 0x7f032815bd30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:50:40.231 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:40.228+0000 7f032cd5b640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f03280c86f0 0x7f03280c7d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:50:40.231 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:40.228+0000 7f032cd5b640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f0328143c20 0x7f03281f4df0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:50:40.231 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:40.228+0000 7f032cd5b640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f032813ee70 con 0x7f032806c8d0 2023-08-09T13:50:40.231 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:40.228+0000 7f032cd5b640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f032815c270 con 0x7f0328143c20 2023-08-09T13:50:40.231 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:40.228+0000 7f032cd5b640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f032815ac90 con 0x7f03280c86f0 2023-08-09T13:50:40.231 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:40.228+0000 7f032659c640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f032806c8d0 0x7f032815bd30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:50:40.231 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:40.228+0000 7f032659c640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f032806c8d0 0x7f032815bd30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:45058/0 (socket says 172.21.15.43:45058) 2023-08-09T13:50:40.232 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:40.228+0000 7f032659c640 1 -- 172.21.15.43:0/1783368490 learned_addr learned my addr 172.21.15.43:0/1783368490 (peer_addr_for_me v2:172.21.15.43:0/0) 2023-08-09T13:50:40.232 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:40.229+0000 7f0326d9d640 1 --2- 172.21.15.43:0/1783368490 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f0328143c20 0x7f03281f4df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:50:40.232 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:40.229+0000 7f032659c640 1 -- 172.21.15.43:0/1783368490 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f03280c86f0 msgr2=0x7f03280c7d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:50:40.232 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:40.229+0000 7f032659c640 1 --2- 172.21.15.43:0/1783368490 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f03280c86f0 0x7f03280c7d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:50:40.232 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:40.229+0000 7f032659c640 1 -- 172.21.15.43:0/1783368490 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f0328143c20 msgr2=0x7f03281f4df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:50:40.232 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:40.229+0000 7f032659c640 1 --2- 172.21.15.43:0/1783368490 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f0328143c20 0x7f03281f4df0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:50:40.232 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:40.229+0000 7f032659c640 1 -- 172.21.15.43:0/1783368490 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f031c009400 con 0x7f032806c8d0 2023-08-09T13:50:40.233 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:40.229+0000 7f032659c640 1 --2- 172.21.15.43:0/1783368490 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f032806c8d0 0x7f032815bd30 secure :-1 s=READY pgs=1037 cs=0 l=1 rev1=1 crypto rx=0x7f031800c890 tx=0x7f031800cd50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-09T13:50:40.233 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:40.230+0000 7f0316ffd640 1 -- 172.21.15.43:0/1783368490 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f031800a5e0 con 0x7f032806c8d0 2023-08-09T13:50:40.234 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:40.230+0000 7f0316ffd640 1 -- 172.21.15.43:0/1783368490 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f031800a760 con 0x7f032806c8d0 2023-08-09T13:50:40.234 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:40.230+0000 7f0316ffd640 1 -- 172.21.15.43:0/1783368490 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f031800ab10 con 0x7f032806c8d0 2023-08-09T13:50:40.234 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:40.230+0000 7f032cd5b640 1 -- 172.21.15.43:0/1783368490 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f032815af70 con 0x7f032806c8d0 2023-08-09T13:50:40.234 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:40.230+0000 7f032cd5b640 1 -- 172.21.15.43:0/1783368490 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f032815b530 con 0x7f032806c8d0 2023-08-09T13:50:40.234 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:40.231+0000 7f032cd5b640 1 -- 172.21.15.43:0/1783368490 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f03280739f0 con 0x7f032806c8d0 2023-08-09T13:50:40.234 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:40.232+0000 7f0316ffd640 1 -- 172.21.15.43:0/1783368490 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 29) v1 ==== 88443+0+0 (secure 0 0 0) 0x7f031800ac70 con 0x7f032806c8d0 2023-08-09T13:50:40.235 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:40.233+0000 7f0316ffd640 1 --2- 172.21.15.43:0/1783368490 >> [v2:172.21.15.203:6832/3325141227,v1:172.21.15.203:6833/3325141227] conn(0x7f030806a170 0x7f030806c630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:50:40.235 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:40.233+0000 7f0316ffd640 1 -- 172.21.15.43:0/1783368490 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(90..90 src has 1..90) v4 ==== 6426+0+0 (secure 0 0 0) 0x7f0318087e10 con 0x7f032806c8d0 2023-08-09T13:50:40.235 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:40.234+0000 7f0325d9b640 1 --2- 172.21.15.43:0/1783368490 >> [v2:172.21.15.203:6832/3325141227,v1:172.21.15.203:6833/3325141227] conn(0x7f030806a170 0x7f030806c630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:50:40.236 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:40.235+0000 7f0325d9b640 1 --2- 172.21.15.43:0/1783368490 >> [v2:172.21.15.203:6832/3325141227,v1:172.21.15.203:6833/3325141227] conn(0x7f030806a170 0x7f030806c630 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f031c035000 tx=0x7f031c030000 comp rx=0 tx=0).ready entity=mgr.5519 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-09T13:50:40.238 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:40.237+0000 7f0316ffd640 1 -- 172.21.15.43:0/1783368490 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178066 (secure 0 0 0) 0x7f031808b030 con 0x7f032806c8d0 2023-08-09T13:50:40.469 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:40.467+0000 7f032cd5b640 1 -- 172.21.15.43:0/1783368490 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f03280cec90 con 0x7f032806c8d0 2023-08-09T13:50:40.470 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:40.469+0000 7f0316ffd640 1 -- 172.21.15.43:0/1783368490 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 35 v35) v1 ==== 94+0+4880 (secure 0 0 0) 0x7f0318018090 con 0x7f032806c8d0 2023-08-09T13:50:40.471 INFO:teuthology.orchestra.run.smithi043.stdout: 2023-08-09T13:50:40.471 INFO:teuthology.orchestra.run.smithi043.stdout:{"epoch":35,"default_fscid":4,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5654,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.203:6837/1227342478","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6835","nonce":1227342478},{"type":"v1","addr":"172.21.15.203:6837","nonce":1227342478}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":30},{"gid":7005,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.43:6834/1431309163","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6832","nonce":1431309163},{"type":"v1","addr":"172.21.15.43:6834","nonce":1431309163}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":29},{"gid":7008,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.43:6835/611328204","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6833","nonce":611328204},{"type":"v1","addr":"172.21.15.43:6835","nonce":611328204}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":30}],"filesystems":[{"mdsmap":{"epoch":35,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-08-09T13:50:28.672581+0000","modified":"2023-08-09T13:50:32.253466+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":7011},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7011":{"gid":7011,"name":"d","rank":0,"incarnation":31,"state":"up:active","state_seq":3,"addr":"172.21.15.203:6836/3061388209","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6834","nonce":3061388209},{"type":"v1","addr":"172.21.15.203:6836","nonce":3061388209}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[11],"metadata_pool":10,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":4}]} 2023-08-09T13:50:40.471 INFO:teuthology.orchestra.run.smithi043.stderr:dumped fsmap epoch 35 2023-08-09T13:50:40.473 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:40.471+0000 7f032cd5b640 1 -- 172.21.15.43:0/1783368490 >> [v2:172.21.15.203:6832/3325141227,v1:172.21.15.203:6833/3325141227] conn(0x7f030806a170 msgr2=0x7f030806c630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:50:40.473 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:40.471+0000 7f032cd5b640 1 --2- 172.21.15.43:0/1783368490 >> [v2:172.21.15.203:6832/3325141227,v1:172.21.15.203:6833/3325141227] conn(0x7f030806a170 0x7f030806c630 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f031c035000 tx=0x7f031c030000 comp rx=0 tx=0).stop 2023-08-09T13:50:40.473 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:40.471+0000 7f032cd5b640 1 -- 172.21.15.43:0/1783368490 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f032806c8d0 msgr2=0x7f032815bd30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:50:40.474 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:40.471+0000 7f032cd5b640 1 --2- 172.21.15.43:0/1783368490 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f032806c8d0 0x7f032815bd30 secure :-1 s=READY pgs=1037 cs=0 l=1 rev1=1 crypto rx=0x7f031800c890 tx=0x7f031800cd50 comp rx=0 tx=0).stop 2023-08-09T13:50:40.474 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:40.471+0000 7f032cd5b640 1 -- 172.21.15.43:0/1783368490 shutdown_connections 2023-08-09T13:50:40.474 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:40.472+0000 7f032cd5b640 1 --2- 172.21.15.43:0/1783368490 >> [v2:172.21.15.203:6832/3325141227,v1:172.21.15.203:6833/3325141227] conn(0x7f030806a170 0x7f030806c630 unknown :-1 s=CLOSED pgs=72 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:50:40.474 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:40.472+0000 7f032cd5b640 1 --2- 172.21.15.43:0/1783368490 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f0328143c20 0x7f03281f4df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:50:40.474 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:40.472+0000 7f032cd5b640 1 --2- 172.21.15.43:0/1783368490 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f03280c86f0 0x7f03280c7d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:50:40.474 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:40.472+0000 7f032cd5b640 1 --2- 172.21.15.43:0/1783368490 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f032806c8d0 0x7f032815bd30 unknown :-1 s=CLOSED pgs=1037 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:50:40.474 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:40.472+0000 7f032cd5b640 1 -- 172.21.15.43:0/1783368490 >> 172.21.15.43:0/1783368490 conn(0x7f03280ce0a0 msgr2=0x7f0328070d60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-09T13:50:40.475 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:40.472+0000 7f032cd5b640 1 -- 172.21.15.43:0/1783368490 shutdown_connections 2023-08-09T13:50:40.475 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:40.472+0000 7f032cd5b640 1 -- 172.21.15.43:0/1783368490 wait complete. 2023-08-09T13:50:40.896 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 35, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2023-08-09T13:50:28.672581+0000', 'modified': '2023-08-09T13:50:32.253466+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 7011}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_7011': {'gid': 7011, 'name': 'd', 'rank': 0, 'incarnation': 31, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.203:6836/3061388209', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.203:6834', 'nonce': 3061388209}, {'type': 'v1', 'addr': '172.21.15.203:6836', 'nonce': 3061388209}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138322906710015, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}}, 'data_pools': [11], 'metadata_pool': 10, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0} 2023-08-09T13:50:40.897 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2023-08-09T13:50:40.897 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.d status 2023-08-09T13:50:41.138 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:41.137+0000 7f1ba9287640 1 Processor -- start 2023-08-09T13:50:41.139 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:41.137+0000 7f1ba9287640 1 -- start start 2023-08-09T13:50:41.139 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:41.138+0000 7f1ba9287640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f1ba415c670 0x7f1ba40cd7d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:50:41.140 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:41.138+0000 7f1ba9287640 1 -- --> v1:172.21.15.43:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1ba40ca920 con 0x7f1ba40cd260 2023-08-09T13:50:41.140 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:41.138+0000 7f1ba9287640 1 -- --> v1:172.21.15.203:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1ba40cb1c0 con 0x7f1ba40cf020 2023-08-09T13:50:41.140 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:41.138+0000 7f1ba9287640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1ba40afc30 con 0x7f1ba415c670 2023-08-09T13:50:41.142 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:41.139+0000 7f1ba37fe640 1 --1- >> v1:172.21.15.43:6789/0 conn(0x7f1ba40cd260 0x7f1ba40b0e20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.43:6789/0 says I am v1:172.21.15.43:55220/0 (socket says 172.21.15.43:55220) 2023-08-09T13:50:41.142 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:41.139+0000 7f1ba37fe640 1 -- 172.21.15.43:0/343459385 learned_addr learned my addr 172.21.15.43:0/343459385 (peer_addr_for_me v1:172.21.15.43:0/0) 2023-08-09T13:50:41.142 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:41.139+0000 7f1ba2ffd640 1 --2- 172.21.15.43:0/343459385 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f1ba415c670 0x7f1ba40cd7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:50:41.142 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:41.139+0000 7f1ba1ffb640 1 -- 172.21.15.43:0/343459385 <== mon.0 v1:172.21.15.43:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3889360449 0 0) 0x7f1ba40ca920 con 0x7f1ba40cd260 2023-08-09T13:50:41.143 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:41.139+0000 7f1ba1ffb640 1 -- 172.21.15.43:0/343459385 --> v1:172.21.15.43:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f1b8c003520 con 0x7f1ba40cd260 2023-08-09T13:50:41.143 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:41.139+0000 7f1ba1ffb640 1 -- 172.21.15.43:0/343459385 <== mon.0 v1:172.21.15.43:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 148137278 0 0) 0x7f1b8c003520 con 0x7f1ba40cd260 2023-08-09T13:50:41.143 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:41.139+0000 7f1ba1ffb640 1 -- 172.21.15.43:0/343459385 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f1ba415c670 msgr2=0x7f1ba40cd7d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-09T13:50:41.143 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:41.139+0000 7f1ba1ffb640 1 --2- 172.21.15.43:0/343459385 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f1ba415c670 0x7f1ba40cd7d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:50:41.143 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:41.139+0000 7f1ba1ffb640 1 -- 172.21.15.43:0/343459385 >> v1:172.21.15.203:6789/0 conn(0x7f1ba40cf020 legacy=0x7f1ba40ccb50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:50:41.143 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:41.139+0000 7f1ba1ffb640 1 -- 172.21.15.43:0/343459385 --> v1:172.21.15.43:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1ba40cf460 con 0x7f1ba40cd260 2023-08-09T13:50:41.144 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:41.139+0000 7f1ba1ffb640 1 -- 172.21.15.43:0/343459385 <== mon.0 v1:172.21.15.43:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1285150740 0 0) 0x7f1b9c001670 con 0x7f1ba40cd260 2023-08-09T13:50:41.144 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:41.140+0000 7f1ba1ffb640 1 -- 172.21.15.43:0/343459385 >> v1:172.21.15.43:6789/0 conn(0x7f1ba40cd260 legacy=0x7f1ba40b0e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:50:41.144 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:41.140+0000 7f1ba1ffb640 1 --2- 172.21.15.43:0/343459385 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f1b8c003a10 0x7f1b8c003e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:50:41.144 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:41.140+0000 7f1ba1ffb640 1 --2- 172.21.15.43:0/343459385 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f1b8c0047c0 0x7f1b8c004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:50:41.144 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:41.140+0000 7f1ba1ffb640 1 --2- 172.21.15.43:0/343459385 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f1ba415c670 0x7f1b8c005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:50:41.144 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:41.140+0000 7f1ba1ffb640 1 -- 172.21.15.43:0/343459385 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1ba40afc30 con 0x7f1b8c003a10 2023-08-09T13:50:41.145 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:41.140+0000 7f1ba1ffb640 1 -- 172.21.15.43:0/343459385 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1b8c00ba80 con 0x7f1b8c0047c0 2023-08-09T13:50:41.145 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:41.140+0000 7f1ba1ffb640 1 -- 172.21.15.43:0/343459385 --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1b8c00bc40 con 0x7f1ba415c670 2023-08-09T13:50:41.145 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:41.140+0000 7f1ba2ffd640 1 --2- 172.21.15.43:0/343459385 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f1b8c003a10 0x7f1b8c003e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:50:41.145 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:41.140+0000 7f1ba2ffd640 1 -- 172.21.15.43:0/343459385 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f1ba415c670 msgr2=0x7f1b8c005180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-09T13:50:41.146 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:41.140+0000 7f1ba37fe640 1 --2- 172.21.15.43:0/343459385 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f1ba415c670 0x7f1b8c005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:50:41.146 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:41.140+0000 7f1ba2ffd640 1 --2- 172.21.15.43:0/343459385 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f1ba415c670 0x7f1b8c005180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:50:41.146 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:41.140+0000 7f1ba2ffd640 1 -- 172.21.15.43:0/343459385 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f1b8c0047c0 msgr2=0x7f1b8c004c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-09T13:50:41.147 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:41.140+0000 7f1ba2ffd640 1 --2- 172.21.15.43:0/343459385 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f1b8c0047c0 0x7f1b8c004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:50:41.147 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:41.140+0000 7f1ba2ffd640 1 -- 172.21.15.43:0/343459385 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1b8c003520 con 0x7f1b8c003a10 2023-08-09T13:50:41.151 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:41.142+0000 7f1ba2ffd640 1 --2- 172.21.15.43:0/343459385 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f1b8c003a10 0x7f1b8c003e30 secure :-1 s=READY pgs=1039 cs=0 l=1 rev1=1 crypto rx=0x7f1b94009fc0 tx=0x7f1b8c0057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=711a88034546d7c6 server_cookie=0 in_seq=0 out_seq=0 2023-08-09T13:50:41.152 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:41.142+0000 7f1ba1ffb640 1 -- 172.21.15.43:0/343459385 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1b9402cc20 con 0x7f1b8c003a10 2023-08-09T13:50:41.152 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:41.142+0000 7f1ba1ffb640 1 -- 172.21.15.43:0/343459385 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1b9400f040 con 0x7f1b8c003a10 2023-08-09T13:50:41.152 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:41.142+0000 7f1ba1ffb640 1 -- 172.21.15.43:0/343459385 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1b94034440 con 0x7f1b8c003a10 2023-08-09T13:50:41.152 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:41.142+0000 7f1ba9287640 1 -- 172.21.15.43:0/343459385 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f1b8c003a10 msgr2=0x7f1b8c003e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:50:41.153 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:41.142+0000 7f1ba9287640 1 --2- 172.21.15.43:0/343459385 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f1b8c003a10 0x7f1b8c003e30 secure :-1 s=READY pgs=1039 cs=0 l=1 rev1=1 crypto rx=0x7f1b94009fc0 tx=0x7f1b8c0057f0 comp rx=0 tx=0).stop 2023-08-09T13:50:41.153 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:41.143+0000 7f1ba37fe640 1 -- 172.21.15.43:0/343459385 reap_dead start 2023-08-09T13:50:41.153 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:41.143+0000 7f1ba9287640 1 -- 172.21.15.43:0/343459385 shutdown_connections 2023-08-09T13:50:41.153 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:41.143+0000 7f1ba9287640 1 -- 172.21.15.43:0/343459385 >> 172.21.15.43:0/343459385 conn(0x7f1ba4089fe0 msgr2=0x7f1ba40d02d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-09T13:50:41.153 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:41.143+0000 7f1ba9287640 1 -- 172.21.15.43:0/343459385 shutdown_connections 2023-08-09T13:50:41.154 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:41.143+0000 7f1ba9287640 1 -- 172.21.15.43:0/343459385 wait complete. 2023-08-09T13:50:41.154 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:41.144+0000 7f1ba9287640 1 Processor -- start 2023-08-09T13:50:41.154 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:41.145+0000 7f1ba9287640 1 -- start start 2023-08-09T13:50:41.154 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:41.146+0000 7f1ba9287640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f1b8c003a10 0x7f1ba40b27f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:50:41.154 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:41.146+0000 7f1ba9287640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f1ba40c8460 0x7f1ba40c9c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:50:41.154 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:41.146+0000 7f1ba9287640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f1ba4158b10 0x7f1ba40d0030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:50:41.154 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:41.146+0000 7f1ba9287640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1ba40c9850 con 0x7f1ba4158b10 2023-08-09T13:50:41.154 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:41.146+0000 7f1ba9287640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1ba40ca1b0 con 0x7f1ba40c8460 2023-08-09T13:50:41.155 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:41.146+0000 7f1ba9287640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1ba40c8bd0 con 0x7f1b8c003a10 2023-08-09T13:50:41.155 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:41.146+0000 7f1ba37fe640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f1ba4158b10 0x7f1ba40d0030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:50:41.155 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:41.146+0000 7f1ba37fe640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f1ba4158b10 0x7f1ba40d0030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:45078/0 (socket says 172.21.15.43:45078) 2023-08-09T13:50:41.155 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:41.146+0000 7f1ba37fe640 1 -- 172.21.15.43:0/3715578418 learned_addr learned my addr 172.21.15.43:0/3715578418 (peer_addr_for_me v2:172.21.15.43:0/0) 2023-08-09T13:50:41.155 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:41.146+0000 7f1ba2ffd640 1 --2- 172.21.15.43:0/3715578418 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f1b8c003a10 0x7f1ba40b27f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:50:41.155 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:41.146+0000 7f1ba27fc640 1 --2- 172.21.15.43:0/3715578418 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f1ba40c8460 0x7f1ba40c9c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:50:41.155 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:41.146+0000 7f1ba37fe640 1 -- 172.21.15.43:0/3715578418 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f1b8c003a10 msgr2=0x7f1ba40b27f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:50:41.156 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:41.146+0000 7f1ba37fe640 1 --2- 172.21.15.43:0/3715578418 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f1b8c003a10 0x7f1ba40b27f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:50:41.156 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:41.146+0000 7f1ba37fe640 1 -- 172.21.15.43:0/3715578418 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f1ba40c8460 msgr2=0x7f1ba40c9c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:50:41.156 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:41.146+0000 7f1ba37fe640 1 --2- 172.21.15.43:0/3715578418 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f1ba40c8460 0x7f1ba40c9c70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:50:41.156 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:41.146+0000 7f1ba37fe640 1 -- 172.21.15.43:0/3715578418 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1ba40c8eb0 con 0x7f1ba4158b10 2023-08-09T13:50:41.157 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:41.147+0000 7f1ba37fe640 1 --2- 172.21.15.43:0/3715578418 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f1ba4158b10 0x7f1ba40d0030 secure :-1 s=READY pgs=1040 cs=0 l=1 rev1=1 crypto rx=0x7f1b9c000c00 tx=0x7f1b9c00dc00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-09T13:50:41.157 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:41.147+0000 7f1b8b7fe640 1 -- 172.21.15.43:0/3715578418 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1b9c007230 con 0x7f1ba4158b10 2023-08-09T13:50:41.157 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:41.147+0000 7f1b8b7fe640 1 -- 172.21.15.43:0/3715578418 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1b9c007390 con 0x7f1ba4158b10 2023-08-09T13:50:41.157 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:41.147+0000 7f1b8b7fe640 1 -- 172.21.15.43:0/3715578418 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1b9c0047e0 con 0x7f1ba4158b10 2023-08-09T13:50:41.157 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:41.147+0000 7f1ba9287640 1 -- 172.21.15.43:0/3715578418 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1ba40c9190 con 0x7f1ba4158b10 2023-08-09T13:50:41.157 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:41.147+0000 7f1ba9287640 1 -- 172.21.15.43:0/3715578418 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1ba40c88e0 con 0x7f1ba4158b10 2023-08-09T13:50:41.165 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:41.151+0000 7f1b8b7fe640 1 -- 172.21.15.43:0/3715578418 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 29) v1 ==== 88443+0+0 (secure 0 0 0) 0x7f1b9c004940 con 0x7f1ba4158b10 2023-08-09T13:50:41.165 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:41.152+0000 7f1b8b7fe640 1 --2- 172.21.15.43:0/3715578418 >> [v2:172.21.15.203:6832/3325141227,v1:172.21.15.203:6833/3325141227] conn(0x7f1b9006a1a0 0x7f1b9006c660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:50:41.165 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:41.152+0000 7f1b8b7fe640 1 -- 172.21.15.43:0/3715578418 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(90..90 src has 1..90) v4 ==== 6426+0+0 (secure 0 0 0) 0x7f1b9c088750 con 0x7f1ba4158b10 2023-08-09T13:50:41.166 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:41.153+0000 7f1babe00740 1 Processor -- start 2023-08-09T13:50:41.166 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:41.153+0000 7f1babe00740 1 -- start start 2023-08-09T13:50:41.166 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:41.153+0000 7f1ba2ffd640 1 --2- 172.21.15.43:0/3715578418 >> [v2:172.21.15.203:6832/3325141227,v1:172.21.15.203:6833/3325141227] conn(0x7f1b9006a1a0 0x7f1b9006c660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:50:41.166 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:41.154+0000 7f1babe00740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5598e8394c50 0x5598e8395030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:50:41.166 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:41.154+0000 7f1babe00740 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x5598e8364bd0 0x5598e8391ec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:50:41.166 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:41.154+0000 7f1babe00740 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5598e83687d0 0x5598e8392400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:50:41.166 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:41.154+0000 7f1babe00740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x5598e82d68f0 con 0x5598e8394c50 2023-08-09T13:50:41.167 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:41.154+0000 7f1babe00740 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x5598e82d3fe0 con 0x5598e83687d0 2023-08-09T13:50:41.167 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:41.154+0000 7f1babe00740 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x5598e82cf8d0 con 0x5598e8364bd0 2023-08-09T13:50:41.167 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:41.154+0000 7f1ba2ffd640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5598e8394c50 0x5598e8395030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:50:41.167 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:41.154+0000 7f1ba2ffd640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5598e8394c50 0x5598e8395030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:45090/0 (socket says 172.21.15.43:45090) 2023-08-09T13:50:41.167 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:41.154+0000 7f1ba2ffd640 1 -- 172.21.15.43:0/595223534 learned_addr learned my addr 172.21.15.43:0/595223534 (peer_addr_for_me v2:172.21.15.43:0/0) 2023-08-09T13:50:41.167 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:41.154+0000 7f1ba2ffd640 1 --2- 172.21.15.43:0/3715578418 >> [v2:172.21.15.203:6832/3325141227,v1:172.21.15.203:6833/3325141227] conn(0x7f1b9006a1a0 0x7f1b9006c660 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f1b94036000 tx=0x7f1b9402e000 comp rx=0 tx=0).ready entity=mgr.5519 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-09T13:50:41.168 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:41.155+0000 7f1ba37fe640 1 --2- 172.21.15.43:0/595223534 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5598e83687d0 0x5598e8392400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:50:41.168 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:41.155+0000 7f1ba27fc640 1 --2- 172.21.15.43:0/595223534 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x5598e8364bd0 0x5598e8391ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:50:41.168 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:41.155+0000 7f1ba2ffd640 1 -- 172.21.15.43:0/595223534 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x5598e8364bd0 msgr2=0x5598e8391ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-09T13:50:41.168 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:41.155+0000 7f1ba2ffd640 1 --2- 172.21.15.43:0/595223534 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x5598e8364bd0 0x5598e8391ec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:50:41.168 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:41.155+0000 7f1ba2ffd640 1 -- 172.21.15.43:0/595223534 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5598e83687d0 msgr2=0x5598e8392400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-09T13:50:41.169 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:41.155+0000 7f1ba2ffd640 1 --2- 172.21.15.43:0/595223534 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5598e83687d0 0x5598e8392400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:50:41.169 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:41.155+0000 7f1ba2ffd640 1 -- 172.21.15.43:0/595223534 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5598e8288ac0 con 0x5598e8394c50 2023-08-09T13:50:41.169 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:41.155+0000 7f1ba37fe640 1 --2- 172.21.15.43:0/595223534 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5598e83687d0 0x5598e8392400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-08-09T13:50:41.169 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:41.155+0000 7f1ba2ffd640 1 --2- 172.21.15.43:0/595223534 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5598e8394c50 0x5598e8395030 secure :-1 s=READY pgs=1041 cs=0 l=1 rev1=1 crypto rx=0x7f1b9400e230 tx=0x7f1b94030000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=25773d00d8af8d5f server_cookie=0 in_seq=0 out_seq=0 2023-08-09T13:50:41.169 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:41.155+0000 7f1b88ff9640 1 -- 172.21.15.43:0/595223534 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1b940041a0 con 0x5598e8394c50 2023-08-09T13:50:41.170 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:41.155+0000 7f1b88ff9640 1 -- 172.21.15.43:0/595223534 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1b94004300 con 0x5598e8394c50 2023-08-09T13:50:41.170 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:41.155+0000 7f1b88ff9640 1 -- 172.21.15.43:0/595223534 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1b94004670 con 0x5598e8394c50 2023-08-09T13:50:41.170 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:41.156+0000 7f1babe00740 1 -- 172.21.15.43:0/595223534 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5598e8394c50 msgr2=0x5598e8395030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:50:41.170 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:41.156+0000 7f1babe00740 1 --2- 172.21.15.43:0/595223534 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5598e8394c50 0x5598e8395030 secure :-1 s=READY pgs=1041 cs=0 l=1 rev1=1 crypto rx=0x7f1b9400e230 tx=0x7f1b94030000 comp rx=0 tx=0).stop 2023-08-09T13:50:41.170 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:41.156+0000 7f1babe00740 1 -- 172.21.15.43:0/595223534 shutdown_connections 2023-08-09T13:50:41.170 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:41.156+0000 7f1babe00740 1 --2- 172.21.15.43:0/595223534 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5598e83687d0 0x5598e8392400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:50:41.170 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:41.156+0000 7f1babe00740 1 --2- 172.21.15.43:0/595223534 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x5598e8364bd0 0x5598e8391ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:50:41.171 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:41.156+0000 7f1babe00740 1 --2- 172.21.15.43:0/595223534 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5598e8394c50 0x5598e8395030 unknown :-1 s=CLOSED pgs=1041 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:50:41.171 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:41.156+0000 7f1babe00740 1 -- 172.21.15.43:0/595223534 >> 172.21.15.43:0/595223534 conn(0x5598e8366ee0 msgr2=0x5598e83afa60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-09T13:50:41.171 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:41.156+0000 7f1babe00740 1 -- 172.21.15.43:0/595223534 shutdown_connections 2023-08-09T13:50:41.171 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:41.156+0000 7f1babe00740 1 -- 172.21.15.43:0/595223534 wait complete. 2023-08-09T13:50:41.171 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:41.157+0000 7f1babe00740 1 -- start start 2023-08-09T13:50:41.171 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:41.157+0000 7f1babe00740 1 Processor -- start 2023-08-09T13:50:41.171 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:41.158+0000 7f1babe00740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5598e83687d0 0x5598e838a170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:50:41.172 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:41.158+0000 7f1babe00740 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x5598e8394c50 0x5598e83afe50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:50:41.172 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:41.158+0000 7f1babe00740 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5598e838d680 0x5598e83c3420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:50:41.172 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:41.158+0000 7f1babe00740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x5598e826fd40 con 0x5598e83687d0 2023-08-09T13:50:41.172 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:41.158+0000 7f1babe00740 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x5598e826f9e0 con 0x5598e838d680 2023-08-09T13:50:41.172 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:41.158+0000 7f1babe00740 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x5598e8035f20 con 0x5598e8394c50 2023-08-09T13:50:41.173 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:41.158+0000 7f1babe00740 10 client.0 _ll_register_callbacks cb 0x5598e8288160 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-08-09T13:50:41.173 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:41.159+0000 7f1ba2ffd640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5598e83687d0 0x5598e838a170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:50:41.173 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:41.159+0000 7f1ba2ffd640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5598e83687d0 0x5598e838a170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:45094/0 (socket says 172.21.15.43:45094) 2023-08-09T13:50:41.173 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:41.159+0000 7f1ba2ffd640 1 -- 172.21.15.43:0/3072041915 learned_addr learned my addr 172.21.15.43:0/3072041915 (peer_addr_for_me v2:172.21.15.43:0/0) 2023-08-09T13:50:41.173 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:41.159+0000 7f1ba27fc640 1 --2- 172.21.15.43:0/3072041915 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x5598e8394c50 0x5598e83afe50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:50:41.174 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:41.159+0000 7f1ba37fe640 1 --2- 172.21.15.43:0/3072041915 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5598e838d680 0x5598e83c3420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:50:41.174 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:41.159+0000 7f1ba2ffd640 1 -- 172.21.15.43:0/3072041915 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x5598e8394c50 msgr2=0x5598e83afe50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-09T13:50:41.174 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:41.159+0000 7f1ba2ffd640 1 --2- 172.21.15.43:0/3072041915 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x5598e8394c50 0x5598e83afe50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:50:41.174 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:41.159+0000 7f1ba2ffd640 1 -- 172.21.15.43:0/3072041915 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5598e838d680 msgr2=0x5598e83c3420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-09T13:50:41.174 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:41.159+0000 7f1ba2ffd640 1 --2- 172.21.15.43:0/3072041915 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5598e838d680 0x5598e83c3420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:50:41.174 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:41.159+0000 7f1ba2ffd640 1 -- 172.21.15.43:0/3072041915 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1b8c003520 con 0x5598e83687d0 2023-08-09T13:50:41.174 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:41.159+0000 7f1ba27fc640 1 --2- 172.21.15.43:0/3072041915 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x5598e8394c50 0x5598e83afe50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-08-09T13:50:41.174 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:41.159+0000 7f1ba2ffd640 1 --2- 172.21.15.43:0/3072041915 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5598e83687d0 0x5598e838a170 secure :-1 s=READY pgs=1042 cs=0 l=1 rev1=1 crypto rx=0x7f1b94000f80 tx=0x7f1b94004ab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ef58071dbb19eeab server_cookie=0 in_seq=0 out_seq=0 2023-08-09T13:50:41.175 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:41.159+0000 7f1b81ffb640 10 client.0 ms_handle_connect on v2:172.21.15.43:3300/0 2023-08-09T13:50:41.175 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:41.160+0000 7f1b81ffb640 1 -- 172.21.15.43:0/3072041915 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1b9403f890 con 0x5598e83687d0 2023-08-09T13:50:41.175 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:41.160+0000 7f1babe00740 1 -- 172.21.15.43:0/3072041915 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5598e82a7500 con 0x5598e83687d0 2023-08-09T13:50:41.175 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:41.160+0000 7f1b81ffb640 1 -- 172.21.15.43:0/3072041915 <== mon.0 v2:172.21.15.43:3300/0 2 ==== osd_map(90..90 src has 1..90) v4 ==== 6426+0+0 (secure 0 0 0) 0x7f1b9403f9f0 con 0x5598e83687d0 2023-08-09T13:50:41.175 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:41.160+0000 7f1b81ffb640 1 -- 172.21.15.43:0/3072041915 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_get_version_reply(handle=1 version=35) v2 ==== 24+0+0 (secure 0 0 0) 0x7f1b9403fbb0 con 0x5598e83687d0 2023-08-09T13:50:41.175 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:41.160+0000 7f1babe00740 10 client.7179 fetch_fsmap learned FSMap version 35 2023-08-09T13:50:41.175 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:41.160+0000 7f1babe00740 1 -- 172.21.15.43:0/3072041915 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,fsmap=35,monmap=2+}) v3 -- 0x5598e82cdf50 con 0x5598e83687d0 2023-08-09T13:50:41.176 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:41.160+0000 7f1b81ffb640 1 -- 172.21.15.43:0/3072041915 <== mon.0 v2:172.21.15.43:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x5598e82a7500 con 0x5598e83687d0 2023-08-09T13:50:41.176 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:41.160+0000 7f1b81ffb640 1 -- 172.21.15.43:0/3072041915 <== mon.0 v2:172.21.15.43:3300/0 5 ==== fsmap(e 35) v1 ==== 3294+0+0 (secure 0 0 0) 0x7f1b9403fd10 con 0x5598e83687d0 2023-08-09T13:50:41.176 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:41.160+0000 7f1babe00740 10 client.7179 fetch_fsmap finished waiting for FSMap version 35 2023-08-09T13:50:41.176 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:41.161+0000 7f1babe00740 10 client.7179 resolve_mds: resolved name 'd' to daemon mds.d 2023-08-09T13:50:41.176 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:41.161+0000 7f1babe00740 20 client.7179 populate_metadata read hostname 'smithi043' 2023-08-09T13:50:41.176 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:41.161+0000 7f1babe00740 1 --2- 172.21.15.43:0/3072041915 >> [v2:172.21.15.203:6834/3061388209,v1:172.21.15.203:6836/3061388209] conn(0x5598e8397e30 0x5598e838ed60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:50:41.176 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:41.161+0000 7f1babe00740 4 client.7179 mds_command: new command op to 7011 tid=0[{"prefix": "get_command_descriptions"}] 2023-08-09T13:50:41.176 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:41.161+0000 7f1babe00740 1 -- 172.21.15.43:0/3072041915 --> [v2:172.21.15.203:6834/3061388209,v1:172.21.15.203:6836/3061388209] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5598e82a4bc0 con 0x5598e8397e30 2023-08-09T13:50:41.177 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:41.161+0000 7f1ba27fc640 1 --2- 172.21.15.43:0/3072041915 >> [v2:172.21.15.203:6834/3061388209,v1:172.21.15.203:6836/3061388209] conn(0x5598e8397e30 0x5598e838ed60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:50:41.177 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:41.162+0000 7f1ba27fc640 1 --2- 172.21.15.43:0/3072041915 >> [v2:172.21.15.203:6834/3061388209,v1:172.21.15.203:6836/3061388209] conn(0x5598e8397e30 0x5598e838ed60 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d71102ca5f5bdb21 server_cookie=9af69f7cfee97ba1 in_seq=0 out_seq=0 2023-08-09T13:50:41.177 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:41.162+0000 7f1b81ffb640 10 client.7179 ms_handle_connect on v2:172.21.15.203:6834/3061388209 2023-08-09T13:50:41.178 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:41.164+0000 7f1b81ffb640 1 -- 172.21.15.43:0/3072041915 <== mds.0 v2:172.21.15.203:6834/3061388209 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18017 (crc 0 0 0) 0x5598e82a4bc0 con 0x5598e8397e30 2023-08-09T13:50:41.178 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:41.164+0000 7f1b81ffb640 10 client.7179 handle_command_reply: tid=0 2023-08-09T13:50:41.178 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:41.164+0000 7f1babe00740 1 client.7179 shutdown 2023-08-09T13:50:41.178 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:41.165+0000 7f1babe00740 1 -- 172.21.15.43:0/3072041915 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5598e83687d0 msgr2=0x5598e838a170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:50:41.178 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:41.165+0000 7f1babe00740 1 --2- 172.21.15.43:0/3072041915 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5598e83687d0 0x5598e838a170 secure :-1 s=READY pgs=1042 cs=0 l=1 rev1=1 crypto rx=0x7f1b94000f80 tx=0x7f1b94004ab0 comp rx=0 tx=0).stop 2023-08-09T13:50:41.179 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:41.165+0000 7f1babe00740 1 -- 172.21.15.43:0/3072041915 shutdown_connections 2023-08-09T13:50:41.179 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:41.165+0000 7f1babe00740 1 --2- 172.21.15.43:0/3072041915 >> [v2:172.21.15.203:6834/3061388209,v1:172.21.15.203:6836/3061388209] conn(0x5598e8397e30 0x5598e838ed60 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:50:41.179 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:41.165+0000 7f1babe00740 1 --2- 172.21.15.43:0/3072041915 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5598e838d680 0x5598e83c3420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:50:41.179 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:41.165+0000 7f1babe00740 1 --2- 172.21.15.43:0/3072041915 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x5598e8394c50 0x5598e83afe50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:50:41.179 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:41.165+0000 7f1babe00740 1 --2- 172.21.15.43:0/3072041915 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5598e83687d0 0x5598e838a170 unknown :-1 s=CLOSED pgs=1042 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:50:41.179 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:41.165+0000 7f1b81ffb640 0 client.7179 ms_handle_reset on v2:172.21.15.203:6834/3061388209 2023-08-09T13:50:41.179 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:41.165+0000 7f1babe00740 1 -- 172.21.15.43:0/3072041915 >> 172.21.15.43:0/3072041915 conn(0x5598e8364bd0 msgr2=0x5598e8369050 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-09T13:50:41.180 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:41.165+0000 7f1babe00740 1 -- 172.21.15.43:0/3072041915 shutdown_connections 2023-08-09T13:50:41.180 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:41.165+0000 7f1babe00740 1 -- 172.21.15.43:0/3072041915 wait complete. 2023-08-09T13:50:41.180 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:41.165+0000 7f1babe00740 20 client.7179 trim_cache size 0 max 16384 2023-08-09T13:50:41.183 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:41.182+0000 7f1babe00740 1 Processor -- start 2023-08-09T13:50:41.183 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:41.182+0000 7f1babe00740 1 -- start start 2023-08-09T13:50:41.184 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:41.183+0000 7f1babe00740 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5598e83687d0 0x5598e83af670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:50:41.185 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:41.183+0000 7f1babe00740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5598e838d680 0x5598e83b2f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:50:41.185 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:41.183+0000 7f1babe00740 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x5598e8394c50 0x5598e83b34b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:50:41.185 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:41.183+0000 7f1babe00740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x5598e8301010 con 0x5598e838d680 2023-08-09T13:50:41.185 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:41.183+0000 7f1babe00740 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x5598e83983b0 con 0x5598e83687d0 2023-08-09T13:50:41.185 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:41.183+0000 7f1babe00740 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x5598e8366550 con 0x5598e8394c50 2023-08-09T13:50:41.185 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:41.183+0000 7f1ba27fc640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5598e838d680 0x5598e83b2f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:50:41.185 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:41.183+0000 7f1ba27fc640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5598e838d680 0x5598e83b2f70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:45106/0 (socket says 172.21.15.43:45106) 2023-08-09T13:50:41.186 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:41.183+0000 7f1ba27fc640 1 -- 172.21.15.43:0/2584293650 learned_addr learned my addr 172.21.15.43:0/2584293650 (peer_addr_for_me v2:172.21.15.43:0/0) 2023-08-09T13:50:41.186 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:41.184+0000 7f1ba27fc640 1 -- 172.21.15.43:0/2584293650 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x5598e8394c50 msgr2=0x5598e83b34b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-09T13:50:41.187 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:41.184+0000 7f1ba27fc640 1 --2- 172.21.15.43:0/2584293650 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x5598e8394c50 0x5598e83b34b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:50:41.187 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:41.184+0000 7f1ba27fc640 1 -- 172.21.15.43:0/2584293650 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5598e83687d0 msgr2=0x5598e83af670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-09T13:50:41.187 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:41.184+0000 7f1ba27fc640 1 --2- 172.21.15.43:0/2584293650 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5598e83687d0 0x5598e83af670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:50:41.187 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:41.184+0000 7f1ba27fc640 1 -- 172.21.15.43:0/2584293650 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5598e8301300 con 0x5598e838d680 2023-08-09T13:50:41.187 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:41.184+0000 7f1ba27fc640 1 --2- 172.21.15.43:0/2584293650 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5598e838d680 0x5598e83b2f70 secure :-1 s=READY pgs=1043 cs=0 l=1 rev1=1 crypto rx=0x7f1b980011d0 tx=0x7f1b98018490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7086142fa317dfe5 server_cookie=0 in_seq=0 out_seq=0 2023-08-09T13:50:41.187 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:41.184+0000 7f1b897fa640 1 -- 172.21.15.43:0/2584293650 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1b980050d0 con 0x5598e838d680 2023-08-09T13:50:41.188 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:41.184+0000 7f1b897fa640 1 -- 172.21.15.43:0/2584293650 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1b9800d040 con 0x5598e838d680 2023-08-09T13:50:41.188 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:41.184+0000 7f1b897fa640 1 -- 172.21.15.43:0/2584293650 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1b98005470 con 0x5598e838d680 2023-08-09T13:50:41.188 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:41.185+0000 7f1babe00740 1 -- 172.21.15.43:0/2584293650 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5598e838d680 msgr2=0x5598e83b2f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:50:41.188 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:41.185+0000 7f1babe00740 1 --2- 172.21.15.43:0/2584293650 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5598e838d680 0x5598e83b2f70 secure :-1 s=READY pgs=1043 cs=0 l=1 rev1=1 crypto rx=0x7f1b980011d0 tx=0x7f1b98018490 comp rx=0 tx=0).stop 2023-08-09T13:50:41.188 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:41.185+0000 7f1babe00740 1 -- 172.21.15.43:0/2584293650 shutdown_connections 2023-08-09T13:50:41.188 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:41.185+0000 7f1babe00740 1 --2- 172.21.15.43:0/2584293650 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x5598e8394c50 0x5598e83b34b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:50:41.188 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:41.185+0000 7f1babe00740 1 --2- 172.21.15.43:0/2584293650 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5598e838d680 0x5598e83b2f70 unknown :-1 s=CLOSED pgs=1043 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:50:41.188 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:41.185+0000 7f1babe00740 1 --2- 172.21.15.43:0/2584293650 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5598e83687d0 0x5598e83af670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:50:41.189 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:41.185+0000 7f1babe00740 1 -- 172.21.15.43:0/2584293650 >> 172.21.15.43:0/2584293650 conn(0x5598e8364bd0 msgr2=0x5598e8363ce0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-09T13:50:41.189 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:41.185+0000 7f1babe00740 1 -- 172.21.15.43:0/2584293650 shutdown_connections 2023-08-09T13:50:41.189 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:41.185+0000 7f1babe00740 1 -- 172.21.15.43:0/2584293650 wait complete. 2023-08-09T13:50:41.191 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:41.186+0000 7f1babe00740 1 -- start start 2023-08-09T13:50:41.191 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:41.186+0000 7f1babe00740 1 Processor -- start 2023-08-09T13:50:41.191 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:41.187+0000 7f1babe00740 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x5598e83687d0 0x5598e8412480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:50:41.191 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:41.187+0000 7f1babe00740 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5598e838d680 0x5598e841baf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:50:41.191 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:41.187+0000 7f1babe00740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5598e8394c50 0x5598e83c9d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:50:41.192 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:41.187+0000 7f1babe00740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x5598e84129c0 con 0x5598e8394c50 2023-08-09T13:50:41.192 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:41.187+0000 7f1babe00740 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x5598e84194b0 con 0x5598e838d680 2023-08-09T13:50:41.192 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:41.187+0000 7f1babe00740 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x5598e8398ae0 con 0x5598e83687d0 2023-08-09T13:50:41.192 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:41.187+0000 7f1babe00740 10 client.0 _ll_register_callbacks cb 0x7f1b94050070 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-08-09T13:50:41.192 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:41.187+0000 7f1ba37fe640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5598e8394c50 0x5598e83c9d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:50:41.192 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:41.187+0000 7f1ba37fe640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5598e8394c50 0x5598e83c9d90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:45120/0 (socket says 172.21.15.43:45120) 2023-08-09T13:50:41.192 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:41.187+0000 7f1ba37fe640 1 -- 172.21.15.43:0/3170445914 learned_addr learned my addr 172.21.15.43:0/3170445914 (peer_addr_for_me v2:172.21.15.43:0/0) 2023-08-09T13:50:41.193 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:41.188+0000 7f1ba27fc640 1 --2- 172.21.15.43:0/3170445914 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5598e838d680 0x5598e841baf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:50:41.193 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:41.188+0000 7f1ba2ffd640 1 --2- 172.21.15.43:0/3170445914 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x5598e83687d0 0x5598e8412480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:50:41.193 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:41.188+0000 7f1ba37fe640 1 -- 172.21.15.43:0/3170445914 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x5598e83687d0 msgr2=0x5598e8412480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-09T13:50:41.193 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:41.188+0000 7f1ba37fe640 1 --2- 172.21.15.43:0/3170445914 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x5598e83687d0 0x5598e8412480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:50:41.193 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:41.188+0000 7f1ba37fe640 1 -- 172.21.15.43:0/3170445914 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5598e838d680 msgr2=0x5598e841baf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-09T13:50:41.193 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:41.188+0000 7f1ba37fe640 1 --2- 172.21.15.43:0/3170445914 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5598e838d680 0x5598e841baf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:50:41.193 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:41.188+0000 7f1ba37fe640 1 -- 172.21.15.43:0/3170445914 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1ba40cb1c0 con 0x5598e8394c50 2023-08-09T13:50:41.193 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:41.188+0000 7f1ba27fc640 1 --2- 172.21.15.43:0/3170445914 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5598e838d680 0x5598e841baf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-08-09T13:50:41.194 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:41.188+0000 7f1ba37fe640 1 --2- 172.21.15.43:0/3170445914 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5598e8394c50 0x5598e83c9d90 secure :-1 s=READY pgs=1044 cs=0 l=1 rev1=1 crypto rx=0x7f1b9c00e010 tx=0x7f1b9c01b3d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=999e38535e916bc1 server_cookie=0 in_seq=0 out_seq=0 2023-08-09T13:50:41.194 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:41.188+0000 7f1b81ffb640 10 client.0 ms_handle_connect on v2:172.21.15.43:3300/0 2023-08-09T13:50:41.194 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:41.188+0000 7f1b81ffb640 1 -- 172.21.15.43:0/3170445914 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1b9c05f3e0 con 0x5598e8394c50 2023-08-09T13:50:41.194 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:41.188+0000 7f1b81ffb640 1 -- 172.21.15.43:0/3170445914 <== mon.0 v2:172.21.15.43:3300/0 2 ==== osd_map(90..90 src has 1..90) v4 ==== 6426+0+0 (secure 0 0 0) 0x7f1b9c05ef50 con 0x5598e8394c50 2023-08-09T13:50:41.194 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:41.188+0000 7f1babe00740 1 -- 172.21.15.43:0/3170445914 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5598e82f7800 con 0x5598e8394c50 2023-08-09T13:50:41.194 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:41.188+0000 7f1b81ffb640 1 -- 172.21.15.43:0/3170445914 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_get_version_reply(handle=1 version=35) v2 ==== 24+0+0 (secure 0 0 0) 0x7f1b9c05f130 con 0x5598e8394c50 2023-08-09T13:50:41.194 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:41.188+0000 7f1babe00740 10 client.7185 fetch_fsmap learned FSMap version 35 2023-08-09T13:50:41.194 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:41.188+0000 7f1babe00740 1 -- 172.21.15.43:0/3170445914 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,fsmap=35,monmap=2+}) v3 -- 0x5598e826ea10 con 0x5598e8394c50 2023-08-09T13:50:41.195 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:41.189+0000 7f1b81ffb640 1 -- 172.21.15.43:0/3170445914 <== mon.0 v2:172.21.15.43:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x5598e82f7800 con 0x5598e8394c50 2023-08-09T13:50:41.195 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:41.189+0000 7f1b81ffb640 1 -- 172.21.15.43:0/3170445914 <== mon.0 v2:172.21.15.43:3300/0 5 ==== fsmap(e 35) v1 ==== 3294+0+0 (secure 0 0 0) 0x7f1b9c086170 con 0x5598e8394c50 2023-08-09T13:50:41.195 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:41.189+0000 7f1babe00740 10 client.7185 fetch_fsmap finished waiting for FSMap version 35 2023-08-09T13:50:41.195 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:41.189+0000 7f1babe00740 10 client.7185 resolve_mds: resolved name 'd' to daemon mds.d 2023-08-09T13:50:41.195 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:41.189+0000 7f1babe00740 20 client.7185 populate_metadata read hostname 'smithi043' 2023-08-09T13:50:41.195 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:41.189+0000 7f1babe00740 1 --2- 172.21.15.43:0/3170445914 >> [v2:172.21.15.203:6834/3061388209,v1:172.21.15.203:6836/3061388209] conn(0x5598e8391d90 0x5598e83b10c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:50:41.195 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:41.189+0000 7f1babe00740 4 client.7185 mds_command: new command op to 7011 tid=0[{"prefix": "status"}] 2023-08-09T13:50:41.196 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:41.189+0000 7f1babe00740 1 -- 172.21.15.43:0/3170445914 --> [v2:172.21.15.203:6834/3061388209,v1:172.21.15.203:6836/3061388209] -- command(tid 0: {"prefix": "status"}) v1 -- 0x5598e82c88b0 con 0x5598e8391d90 2023-08-09T13:50:41.196 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:41.190+0000 7f1ba2ffd640 1 --2- 172.21.15.43:0/3170445914 >> [v2:172.21.15.203:6834/3061388209,v1:172.21.15.203:6836/3061388209] conn(0x5598e8391d90 0x5598e83b10c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:50:41.197 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:41.191+0000 7f1ba2ffd640 1 --2- 172.21.15.43:0/3170445914 >> [v2:172.21.15.203:6834/3061388209,v1:172.21.15.203:6836/3061388209] conn(0x5598e8391d90 0x5598e83b10c0 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=3548091343785417 server_cookie=b969900b1d3c5134 in_seq=0 out_seq=0 2023-08-09T13:50:41.197 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:41.191+0000 7f1b81ffb640 10 client.7185 ms_handle_connect on v2:172.21.15.203:6834/3061388209 2023-08-09T13:50:41.197 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:41.191+0000 7f1b81ffb640 1 -- 172.21.15.43:0/3170445914 <== mds.0 v2:172.21.15.203:6834/3061388209 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+331 (crc 0 0 0) 0x5598e82c88b0 con 0x5598e8391d90 2023-08-09T13:50:41.197 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:41.191+0000 7f1b81ffb640 10 client.7185 handle_command_reply: tid=0 2023-08-09T13:50:41.197 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:41.191+0000 7f1babe00740 1 client.7185 shutdown 2023-08-09T13:50:41.198 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:41.191+0000 7f1babe00740 1 -- 172.21.15.43:0/3170445914 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5598e8394c50 msgr2=0x5598e83c9d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:50:41.198 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:41.191+0000 7f1babe00740 1 --2- 172.21.15.43:0/3170445914 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5598e8394c50 0x5598e83c9d90 secure :-1 s=READY pgs=1044 cs=0 l=1 rev1=1 crypto rx=0x7f1b9c00e010 tx=0x7f1b9c01b3d0 comp rx=0 tx=0).stop 2023-08-09T13:50:41.198 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:41.192+0000 7f1babe00740 1 -- 172.21.15.43:0/3170445914 shutdown_connections 2023-08-09T13:50:41.198 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:41.192+0000 7f1babe00740 1 --2- 172.21.15.43:0/3170445914 >> [v2:172.21.15.203:6834/3061388209,v1:172.21.15.203:6836/3061388209] conn(0x5598e8391d90 0x5598e83b10c0 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:50:41.198 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:41.192+0000 7f1babe00740 1 --2- 172.21.15.43:0/3170445914 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5598e8394c50 0x5598e83c9d90 unknown :-1 s=CLOSED pgs=1044 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:50:41.198 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:41.192+0000 7f1babe00740 1 --2- 172.21.15.43:0/3170445914 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5598e838d680 0x5598e841baf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:50:41.198 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:41.192+0000 7f1babe00740 1 --2- 172.21.15.43:0/3170445914 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x5598e83687d0 0x5598e8412480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:50:41.198 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:41.192+0000 7f1b81ffb640 0 client.7185 ms_handle_reset on v2:172.21.15.203:6834/3061388209 2023-08-09T13:50:41.199 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:41.192+0000 7f1babe00740 1 -- 172.21.15.43:0/3170445914 >> 172.21.15.43:0/3170445914 conn(0x5598e8364bd0 msgr2=0x5598e83b2f70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-09T13:50:41.199 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:41.192+0000 7f1babe00740 1 -- 172.21.15.43:0/3170445914 shutdown_connections 2023-08-09T13:50:41.199 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:41.192+0000 7f1babe00740 1 -- 172.21.15.43:0/3170445914 wait complete. 2023-08-09T13:50:41.199 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:41.192+0000 7f1babe00740 20 client.7185 trim_cache size 0 max 16384 2023-08-09T13:50:41.199 INFO:teuthology.orchestra.run.smithi043.stdout:{ 2023-08-09T13:50:41.200 INFO:teuthology.orchestra.run.smithi043.stdout: "cluster_fsid": "daac87ee-1337-4250-a955-aea4e408a39b", 2023-08-09T13:50:41.200 INFO:teuthology.orchestra.run.smithi043.stdout: "whoami": 0, 2023-08-09T13:50:41.200 INFO:teuthology.orchestra.run.smithi043.stdout: "id": 7011, 2023-08-09T13:50:41.200 INFO:teuthology.orchestra.run.smithi043.stdout: "want_state": "up:active", 2023-08-09T13:50:41.200 INFO:teuthology.orchestra.run.smithi043.stdout: "state": "up:active", 2023-08-09T13:50:41.200 INFO:teuthology.orchestra.run.smithi043.stdout: "fs_name": "cephfs", 2023-08-09T13:50:41.200 INFO:teuthology.orchestra.run.smithi043.stdout: "rank_uptime": 11.955658338999999, 2023-08-09T13:50:41.201 INFO:teuthology.orchestra.run.smithi043.stdout: "mdsmap_epoch": 35, 2023-08-09T13:50:41.201 INFO:teuthology.orchestra.run.smithi043.stdout: "osdmap_epoch": 90, 2023-08-09T13:50:41.201 INFO:teuthology.orchestra.run.smithi043.stdout: "osdmap_epoch_barrier": 90, 2023-08-09T13:50:41.201 INFO:teuthology.orchestra.run.smithi043.stdout: "uptime": 18.335344074999998 2023-08-09T13:50:41.201 INFO:teuthology.orchestra.run.smithi043.stdout:} 2023-08-09T13:50:41.202 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:41.193+0000 7f1ba9287640 1 -- 172.21.15.43:0/3715578418 >> [v2:172.21.15.203:6832/3325141227,v1:172.21.15.203:6833/3325141227] conn(0x7f1b9006a1a0 msgr2=0x7f1b9006c660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:50:41.202 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:41.193+0000 7f1ba9287640 1 --2- 172.21.15.43:0/3715578418 >> [v2:172.21.15.203:6832/3325141227,v1:172.21.15.203:6833/3325141227] conn(0x7f1b9006a1a0 0x7f1b9006c660 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f1b94036000 tx=0x7f1b9402e000 comp rx=0 tx=0).stop 2023-08-09T13:50:41.202 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:41.193+0000 7f1ba9287640 1 -- 172.21.15.43:0/3715578418 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f1ba4158b10 msgr2=0x7f1ba40d0030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:50:41.202 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:41.193+0000 7f1ba9287640 1 --2- 172.21.15.43:0/3715578418 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f1ba4158b10 0x7f1ba40d0030 secure :-1 s=READY pgs=1040 cs=0 l=1 rev1=1 crypto rx=0x7f1b9c000c00 tx=0x7f1b9c00dc00 comp rx=0 tx=0).stop 2023-08-09T13:50:41.202 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:41.193+0000 7f1ba9287640 1 -- 172.21.15.43:0/3715578418 shutdown_connections 2023-08-09T13:50:41.203 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:41.193+0000 7f1ba9287640 1 --2- 172.21.15.43:0/3715578418 >> [v2:172.21.15.203:6832/3325141227,v1:172.21.15.203:6833/3325141227] conn(0x7f1b9006a1a0 0x7f1b9006c660 unknown :-1 s=CLOSED pgs=73 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:50:41.203 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:41.193+0000 7f1ba9287640 1 --2- 172.21.15.43:0/3715578418 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f1ba4158b10 0x7f1ba40d0030 unknown :-1 s=CLOSED pgs=1040 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:50:41.203 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:41.193+0000 7f1ba9287640 1 --2- 172.21.15.43:0/3715578418 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f1ba40c8460 0x7f1ba40c9c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:50:41.203 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:41.193+0000 7f1ba9287640 1 --2- 172.21.15.43:0/3715578418 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f1b8c003a10 0x7f1ba40b27f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:50:41.203 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:41.193+0000 7f1ba9287640 1 -- 172.21.15.43:0/3715578418 >> 172.21.15.43:0/3715578418 conn(0x7f1ba4089fe0 msgr2=0x7f1ba40ccb50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-09T13:50:41.203 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:41.193+0000 7f1ba9287640 1 -- 172.21.15.43:0/3715578418 shutdown_connections 2023-08-09T13:50:41.203 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:41.193+0000 7f1ba9287640 1 -- 172.21.15.43:0/3715578418 wait complete. 2023-08-09T13:50:41.211 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2023-08-09T13:50:41.432 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:41.430+0000 7fc2f900f640 1 Processor -- start 2023-08-09T13:50:41.433 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:41.431+0000 7fc2f900f640 1 -- start start 2023-08-09T13:50:41.434 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:41.432+0000 7fc2f900f640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fc2f40c8750 0x7fc2f40ad340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:50:41.434 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:41.433+0000 7fc2f900f640 1 -- --> v1:172.21.15.43:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc2f40cbd90 con 0x7fc2f40acdf0 2023-08-09T13:50:41.434 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:41.433+0000 7fc2f900f640 1 -- --> v1:172.21.15.203:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc2f40ca2b0 con 0x7fc2f40c8b30 2023-08-09T13:50:41.434 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:41.433+0000 7fc2f900f640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc2f415e810 con 0x7fc2f40c8750 2023-08-09T13:50:41.436 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:41.433+0000 7fc2f880e640 1 --1- >> v1:172.21.15.43:6789/0 conn(0x7fc2f40acdf0 0x7fc2f4070f00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.43:6789/0 says I am v1:172.21.15.43:55228/0 (socket says 172.21.15.43:55228) 2023-08-09T13:50:41.436 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:41.433+0000 7fc2f880e640 1 -- 172.21.15.43:0/3945549605 learned_addr learned my addr 172.21.15.43:0/3945549605 (peer_addr_for_me v1:172.21.15.43:0/0) 2023-08-09T13:50:41.436 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:41.433+0000 7fc2f3fff640 1 --2- 172.21.15.43:0/3945549605 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fc2f40c8750 0x7fc2f40ad340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:50:41.436 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:41.433+0000 7fc2f2ffd640 1 -- 172.21.15.43:0/3945549605 <== mon.0 v1:172.21.15.43:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3557569407 0 0) 0x7fc2f40cbd90 con 0x7fc2f40acdf0 2023-08-09T13:50:41.437 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:41.434+0000 7fc2f2ffd640 1 -- 172.21.15.43:0/3945549605 --> v1:172.21.15.43:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fc2dc003520 con 0x7fc2f40acdf0 2023-08-09T13:50:41.437 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:41.434+0000 7fc2f2ffd640 1 -- 172.21.15.43:0/3945549605 <== mon.0 v1:172.21.15.43:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2785290581 0 0) 0x7fc2dc003520 con 0x7fc2f40acdf0 2023-08-09T13:50:41.437 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:41.434+0000 7fc2f2ffd640 1 -- 172.21.15.43:0/3945549605 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fc2f40c8750 msgr2=0x7fc2f40ad340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-09T13:50:41.437 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:41.434+0000 7fc2f2ffd640 1 --2- 172.21.15.43:0/3945549605 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fc2f40c8750 0x7fc2f40ad340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:50:41.437 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:41.434+0000 7fc2f2ffd640 1 -- 172.21.15.43:0/3945549605 >> v1:172.21.15.203:6789/0 conn(0x7fc2f40c8b30 legacy=0x7fc2f406f410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:50:41.437 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:41.434+0000 7fc2f2ffd640 1 -- 172.21.15.43:0/3945549605 --> v1:172.21.15.43:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc2f40c9a20 con 0x7fc2f40acdf0 2023-08-09T13:50:41.437 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:41.434+0000 7fc2f2ffd640 1 -- 172.21.15.43:0/3945549605 <== mon.0 v1:172.21.15.43:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1285150740 0 0) 0x7fc2ec0019f0 con 0x7fc2f40acdf0 2023-08-09T13:50:41.437 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:41.434+0000 7fc2f2ffd640 1 -- 172.21.15.43:0/3945549605 >> v1:172.21.15.43:6789/0 conn(0x7fc2f40acdf0 legacy=0x7fc2f4070f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:50:41.438 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:41.434+0000 7fc2f2ffd640 1 --2- 172.21.15.43:0/3945549605 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fc2dc0039a0 0x7fc2dc003dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:50:41.438 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:41.434+0000 7fc2f2ffd640 1 --2- 172.21.15.43:0/3945549605 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fc2f40c8750 0x7fc2dc0047c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:50:41.438 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:41.434+0000 7fc2f2ffd640 1 --2- 172.21.15.43:0/3945549605 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc2dc004d00 0x7fc2dc005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:50:41.438 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:41.434+0000 7fc2f2ffd640 1 -- 172.21.15.43:0/3945549605 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc2f415e810 con 0x7fc2dc004d00 2023-08-09T13:50:41.438 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:41.434+0000 7fc2f2ffd640 1 -- 172.21.15.43:0/3945549605 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc2dc00bab0 con 0x7fc2dc0039a0 2023-08-09T13:50:41.438 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:41.434+0000 7fc2f2ffd640 1 -- 172.21.15.43:0/3945549605 --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc2dc00bc70 con 0x7fc2f40c8750 2023-08-09T13:50:41.438 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:41.435+0000 7fc2f880e640 1 --2- 172.21.15.43:0/3945549605 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc2dc004d00 0x7fc2dc005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:50:41.440 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:41.435+0000 7fc2f3fff640 1 --2- 172.21.15.43:0/3945549605 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fc2dc0039a0 0x7fc2dc003dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:50:41.440 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:41.435+0000 7fc2f37fe640 1 --2- 172.21.15.43:0/3945549605 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fc2f40c8750 0x7fc2dc0047c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:50:41.440 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:41.435+0000 7fc2f880e640 1 -- 172.21.15.43:0/3945549605 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fc2f40c8750 msgr2=0x7fc2dc0047c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-09T13:50:41.440 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:41.435+0000 7fc2f880e640 1 --2- 172.21.15.43:0/3945549605 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fc2f40c8750 0x7fc2dc0047c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:50:41.440 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:41.435+0000 7fc2f880e640 1 -- 172.21.15.43:0/3945549605 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fc2dc0039a0 msgr2=0x7fc2dc003dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-09T13:50:41.440 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:41.435+0000 7fc2f880e640 1 --2- 172.21.15.43:0/3945549605 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fc2dc0039a0 0x7fc2dc003dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:50:41.441 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:41.435+0000 7fc2f880e640 1 -- 172.21.15.43:0/3945549605 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc2dc003520 con 0x7fc2dc004d00 2023-08-09T13:50:41.441 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:41.436+0000 7fc2f880e640 1 --2- 172.21.15.43:0/3945549605 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc2dc004d00 0x7fc2dc005180 secure :-1 s=READY pgs=1046 cs=0 l=1 rev1=1 crypto rx=0x7fc2ec00a820 tx=0x7fc2dc0057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d52c32213c710d56 server_cookie=0 in_seq=0 out_seq=0 2023-08-09T13:50:41.441 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:41.436+0000 7fc2f2ffd640 1 -- 172.21.15.43:0/3945549605 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc2ec0072d0 con 0x7fc2dc004d00 2023-08-09T13:50:41.441 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:41.437+0000 7fc2f2ffd640 1 -- 172.21.15.43:0/3945549605 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc2ec02f070 con 0x7fc2dc004d00 2023-08-09T13:50:41.441 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:41.437+0000 7fc2f2ffd640 1 -- 172.21.15.43:0/3945549605 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc2ec0076b0 con 0x7fc2dc004d00 2023-08-09T13:50:41.441 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:41.437+0000 7fc2f900f640 1 -- 172.21.15.43:0/3945549605 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc2dc004d00 msgr2=0x7fc2dc005180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:50:41.441 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:41.437+0000 7fc2f900f640 1 --2- 172.21.15.43:0/3945549605 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc2dc004d00 0x7fc2dc005180 secure :-1 s=READY pgs=1046 cs=0 l=1 rev1=1 crypto rx=0x7fc2ec00a820 tx=0x7fc2dc0057f0 comp rx=0 tx=0).stop 2023-08-09T13:50:41.442 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:41.437+0000 7fc2f880e640 1 -- 172.21.15.43:0/3945549605 reap_dead start 2023-08-09T13:50:41.442 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:41.437+0000 7fc2f900f640 1 -- 172.21.15.43:0/3945549605 shutdown_connections 2023-08-09T13:50:41.442 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:41.437+0000 7fc2f900f640 1 -- 172.21.15.43:0/3945549605 >> 172.21.15.43:0/3945549605 conn(0x7fc2f40cde90 msgr2=0x7fc2f40ce2a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-09T13:50:41.442 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:41.437+0000 7fc2f900f640 1 -- 172.21.15.43:0/3945549605 shutdown_connections 2023-08-09T13:50:41.442 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:41.437+0000 7fc2f900f640 1 -- 172.21.15.43:0/3945549605 wait complete. 2023-08-09T13:50:41.443 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:41.439+0000 7fc2f900f640 1 Processor -- start 2023-08-09T13:50:41.444 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:41.439+0000 7fc2f900f640 1 -- start start 2023-08-09T13:50:41.444 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:41.440+0000 7fc2f900f640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc2dc004d00 0x7fc2f414c1e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:50:41.444 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:41.440+0000 7fc2f900f640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fc2f4148f20 0x7fc2f4099aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:50:41.444 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:41.440+0000 7fc2f900f640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fc2f40c4f60 0x7fc2f40c53f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:50:41.444 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:41.440+0000 7fc2f900f640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc2f40c9f70 con 0x7fc2dc004d00 2023-08-09T13:50:41.444 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:41.440+0000 7fc2f900f640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc2f4099fe0 con 0x7fc2f4148f20 2023-08-09T13:50:41.444 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:41.440+0000 7fc2f900f640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc2f40c5930 con 0x7fc2f40c4f60 2023-08-09T13:50:41.444 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:41.440+0000 7fc2f3fff640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc2dc004d00 0x7fc2f414c1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:50:41.445 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:41.441+0000 7fc2f3fff640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc2dc004d00 0x7fc2f414c1e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:45134/0 (socket says 172.21.15.43:45134) 2023-08-09T13:50:41.445 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:41.441+0000 7fc2f3fff640 1 -- 172.21.15.43:0/2552136676 learned_addr learned my addr 172.21.15.43:0/2552136676 (peer_addr_for_me v2:172.21.15.43:0/0) 2023-08-09T13:50:41.445 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:41.441+0000 7fc2f3fff640 1 -- 172.21.15.43:0/2552136676 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fc2f40c4f60 msgr2=0x7fc2f40c53f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:50:41.445 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:41.441+0000 7fc2f3fff640 1 --2- 172.21.15.43:0/2552136676 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fc2f40c4f60 0x7fc2f40c53f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:50:41.445 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:41.441+0000 7fc2f3fff640 1 -- 172.21.15.43:0/2552136676 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fc2f4148f20 msgr2=0x7fc2f4099aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:50:41.445 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:41.441+0000 7fc2f3fff640 1 --2- 172.21.15.43:0/2552136676 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fc2f4148f20 0x7fc2f4099aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:50:41.445 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:41.441+0000 7fc2f3fff640 1 -- 172.21.15.43:0/2552136676 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc2f41493a0 con 0x7fc2dc004d00 2023-08-09T13:50:41.446 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:41.441+0000 7fc2f3fff640 1 --2- 172.21.15.43:0/2552136676 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc2dc004d00 0x7fc2f414c1e0 secure :-1 s=READY pgs=1047 cs=0 l=1 rev1=1 crypto rx=0x7fc2e4011600 tx=0x7fc2e4011ac0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-09T13:50:41.446 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:41.442+0000 7fc2f0ff9640 1 -- 172.21.15.43:0/2552136676 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc2e4015870 con 0x7fc2dc004d00 2023-08-09T13:50:41.446 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:41.442+0000 7fc2f0ff9640 1 -- 172.21.15.43:0/2552136676 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc2e40159d0 con 0x7fc2dc004d00 2023-08-09T13:50:41.446 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:41.442+0000 7fc2f0ff9640 1 -- 172.21.15.43:0/2552136676 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc2e4015d40 con 0x7fc2dc004d00 2023-08-09T13:50:41.446 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:41.442+0000 7fc2f900f640 1 -- 172.21.15.43:0/2552136676 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc2f409a1d0 con 0x7fc2dc004d00 2023-08-09T13:50:41.446 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:41.442+0000 7fc2f900f640 1 -- 172.21.15.43:0/2552136676 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc2f409a790 con 0x7fc2dc004d00 2023-08-09T13:50:41.447 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:41.443+0000 7fc2f900f640 1 -- 172.21.15.43:0/2552136676 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc2f40c9230 con 0x7fc2dc004d00 2023-08-09T13:50:41.447 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:41.444+0000 7fc2f0ff9640 1 -- 172.21.15.43:0/2552136676 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 29) v1 ==== 88443+0+0 (secure 0 0 0) 0x7fc2e400f490 con 0x7fc2dc004d00 2023-08-09T13:50:41.447 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:41.445+0000 7fc2f0ff9640 1 --2- 172.21.15.43:0/2552136676 >> [v2:172.21.15.203:6832/3325141227,v1:172.21.15.203:6833/3325141227] conn(0x7fc2e006a170 0x7fc2e006c630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:50:41.447 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:41.445+0000 7fc2f0ff9640 1 -- 172.21.15.43:0/2552136676 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(90..90 src has 1..90) v4 ==== 6426+0+0 (secure 0 0 0) 0x7fc2e4088d00 con 0x7fc2dc004d00 2023-08-09T13:50:41.448 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:41.446+0000 7fc2f37fe640 1 --2- 172.21.15.43:0/2552136676 >> [v2:172.21.15.203:6832/3325141227,v1:172.21.15.203:6833/3325141227] conn(0x7fc2e006a170 0x7fc2e006c630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:50:41.448 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:41.447+0000 7fc2f37fe640 1 --2- 172.21.15.43:0/2552136676 >> [v2:172.21.15.203:6832/3325141227,v1:172.21.15.203:6833/3325141227] conn(0x7fc2e006a170 0x7fc2e006c630 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7fc2e8007fd0 tx=0x7fc2e8009040 comp rx=0 tx=0).ready entity=mgr.5519 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-09T13:50:41.451 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:41.449+0000 7fc2f0ff9640 1 -- 172.21.15.43:0/2552136676 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178066 (secure 0 0 0) 0x7fc2e4057bf0 con 0x7fc2dc004d00 2023-08-09T13:50:41.686 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:41.684+0000 7fc2f900f640 1 -- 172.21.15.43:0/2552136676 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7fc2f40739f0 con 0x7fc2dc004d00 2023-08-09T13:50:41.687 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:41.686+0000 7fc2f0ff9640 1 -- 172.21.15.43:0/2552136676 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v35) v1 ==== 53+0+73 (secure 0 0 0) 0x7fc2e4057780 con 0x7fc2dc004d00 2023-08-09T13:50:41.687 INFO:teuthology.orchestra.run.smithi043.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2023-08-09T13:50:41.689 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:41.688+0000 7fc2f900f640 1 -- 172.21.15.43:0/2552136676 >> [v2:172.21.15.203:6832/3325141227,v1:172.21.15.203:6833/3325141227] conn(0x7fc2e006a170 msgr2=0x7fc2e006c630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:50:41.690 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:41.688+0000 7fc2f900f640 1 --2- 172.21.15.43:0/2552136676 >> [v2:172.21.15.203:6832/3325141227,v1:172.21.15.203:6833/3325141227] conn(0x7fc2e006a170 0x7fc2e006c630 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7fc2e8007fd0 tx=0x7fc2e8009040 comp rx=0 tx=0).stop 2023-08-09T13:50:41.690 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:41.688+0000 7fc2f900f640 1 -- 172.21.15.43:0/2552136676 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc2dc004d00 msgr2=0x7fc2f414c1e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:50:41.690 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:41.688+0000 7fc2f900f640 1 --2- 172.21.15.43:0/2552136676 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc2dc004d00 0x7fc2f414c1e0 secure :-1 s=READY pgs=1047 cs=0 l=1 rev1=1 crypto rx=0x7fc2e4011600 tx=0x7fc2e4011ac0 comp rx=0 tx=0).stop 2023-08-09T13:50:41.690 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:41.688+0000 7fc2f900f640 1 -- 172.21.15.43:0/2552136676 shutdown_connections 2023-08-09T13:50:41.691 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:41.688+0000 7fc2f900f640 1 --2- 172.21.15.43:0/2552136676 >> [v2:172.21.15.203:6832/3325141227,v1:172.21.15.203:6833/3325141227] conn(0x7fc2e006a170 0x7fc2e006c630 unknown :-1 s=CLOSED pgs=74 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:50:41.691 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:41.688+0000 7fc2f900f640 1 --2- 172.21.15.43:0/2552136676 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fc2f40c4f60 0x7fc2f40c53f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:50:41.691 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:41.688+0000 7fc2f900f640 1 --2- 172.21.15.43:0/2552136676 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fc2f4148f20 0x7fc2f4099aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:50:41.691 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:41.688+0000 7fc2f900f640 1 --2- 172.21.15.43:0/2552136676 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc2dc004d00 0x7fc2f414c1e0 unknown :-1 s=CLOSED pgs=1047 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:50:41.691 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:41.688+0000 7fc2f900f640 1 -- 172.21.15.43:0/2552136676 >> 172.21.15.43:0/2552136676 conn(0x7fc2f40cde90 msgr2=0x7fc2f415d5d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-09T13:50:41.691 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:41.689+0000 7fc2f900f640 1 -- 172.21.15.43:0/2552136676 shutdown_connections 2023-08-09T13:50:41.691 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:41.689+0000 7fc2f900f640 1 -- 172.21.15.43:0/2552136676 wait complete. 2023-08-09T13:50:41.710 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2023-08-09T13:50:41.711 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2023-08-09T13:50:41.711 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi043.front.sepia.ceph.com 2023-08-09T13:50:41.711 INFO:tasks.cephfs.mount:self.client.name = client.0 2023-08-09T13:50:41.711 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2023-08-09T13:50:41.712 INFO:tasks.cephfs.mount:self.cephfs_name = None 2023-08-09T13:50:41.712 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2023-08-09T13:50:41.712 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2023-08-09T13:50:41.712 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2023-08-09T13:50:41.713 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-09T13:50:41.713 DEBUG:teuthology.orchestra.run.smithi043:> ip addr 2023-08-09T13:50:41.771 INFO:teuthology.orchestra.run.smithi043.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2023-08-09T13:50:41.771 INFO:teuthology.orchestra.run.smithi043.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2023-08-09T13:50:41.772 INFO:teuthology.orchestra.run.smithi043.stdout: inet 127.0.0.1/8 scope host lo 2023-08-09T13:50:41.772 INFO:teuthology.orchestra.run.smithi043.stdout: valid_lft forever preferred_lft forever 2023-08-09T13:50:41.772 INFO:teuthology.orchestra.run.smithi043.stdout: inet6 ::1/128 scope host 2023-08-09T13:50:41.772 INFO:teuthology.orchestra.run.smithi043.stdout: valid_lft forever preferred_lft forever 2023-08-09T13:50:41.772 INFO:teuthology.orchestra.run.smithi043.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-08-09T13:50:41.772 INFO:teuthology.orchestra.run.smithi043.stdout: link/ether 0c:c4:7a:6b:fd:9c brd ff:ff:ff:ff:ff:ff 2023-08-09T13:50:41.772 INFO:teuthology.orchestra.run.smithi043.stdout: altname enp7s0f0 2023-08-09T13:50:41.773 INFO:teuthology.orchestra.run.smithi043.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-08-09T13:50:41.773 INFO:teuthology.orchestra.run.smithi043.stdout: link/ether 0c:c4:7a:6b:fd:9d brd ff:ff:ff:ff:ff:ff 2023-08-09T13:50:41.773 INFO:teuthology.orchestra.run.smithi043.stdout: altname enp7s0f1 2023-08-09T13:50:41.773 INFO:teuthology.orchestra.run.smithi043.stdout:4: ens1f0: mtu 1500 qdisc mq state UP group default qlen 1000 2023-08-09T13:50:41.773 INFO:teuthology.orchestra.run.smithi043.stdout: link/ether 0c:c4:7a:bd:13:58 brd ff:ff:ff:ff:ff:ff 2023-08-09T13:50:41.773 INFO:teuthology.orchestra.run.smithi043.stdout: altname enp1s0f0 2023-08-09T13:50:41.773 INFO:teuthology.orchestra.run.smithi043.stdout: inet 172.21.15.43/20 brd 172.21.15.255 scope global dynamic noprefixroute ens1f0 2023-08-09T13:50:41.773 INFO:teuthology.orchestra.run.smithi043.stdout: valid_lft 41262sec preferred_lft 41262sec 2023-08-09T13:50:41.774 INFO:teuthology.orchestra.run.smithi043.stdout: inet6 fe80::ec4:7aff:febd:1358/64 scope link 2023-08-09T13:50:41.774 INFO:teuthology.orchestra.run.smithi043.stdout: valid_lft forever preferred_lft forever 2023-08-09T13:50:41.774 INFO:teuthology.orchestra.run.smithi043.stdout:5: ens1f1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-08-09T13:50:41.774 INFO:teuthology.orchestra.run.smithi043.stdout: link/ether 0c:c4:7a:bd:13:59 brd ff:ff:ff:ff:ff:ff 2023-08-09T13:50:41.774 INFO:teuthology.orchestra.run.smithi043.stdout: altname enp1s0f1 2023-08-09T13:50:41.774 INFO:teuthology.orchestra.run.smithi043.stdout:8: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2023-08-09T13:50:41.774 INFO:teuthology.orchestra.run.smithi043.stdout: link/ether 56:43:50:19:f4:0c brd ff:ff:ff:ff:ff:ff 2023-08-09T13:50:41.774 INFO:teuthology.orchestra.run.smithi043.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2023-08-09T13:50:41.775 INFO:teuthology.orchestra.run.smithi043.stdout: valid_lft forever preferred_lft forever 2023-08-09T13:50:41.775 INFO:teuthology.orchestra.run.smithi043.stdout: inet6 fe80::e494:65ff:fe92:a182/64 scope link 2023-08-09T13:50:41.775 INFO:teuthology.orchestra.run.smithi043.stdout: valid_lft forever preferred_lft forever 2023-08-09T13:50:41.775 INFO:teuthology.orchestra.run.smithi043.stdout:9: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2023-08-09T13:50:41.775 INFO:teuthology.orchestra.run.smithi043.stdout: link/ether 7e:93:26:fa:07:8f brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.0 2023-08-09T13:50:41.775 INFO:teuthology.orchestra.run.smithi043.stdout: inet6 fe80::7c93:26ff:fefa:78f/64 scope link 2023-08-09T13:50:41.775 INFO:teuthology.orchestra.run.smithi043.stdout: valid_lft forever preferred_lft forever 2023-08-09T13:50:41.776 INFO:teuthology.orchestra.run.smithi043.stdout:10: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2023-08-09T13:50:41.776 INFO:teuthology.orchestra.run.smithi043.stdout: link/ether 56:43:50:19:f4:0c brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.2 2023-08-09T13:50:41.776 INFO:teuthology.orchestra.run.smithi043.stdout: inet6 fe80::5443:50ff:fe19:f40c/64 scope link 2023-08-09T13:50:41.776 INFO:teuthology.orchestra.run.smithi043.stdout: valid_lft forever preferred_lft forever 2023-08-09T13:50:41.777 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-09T13:50:41.777 DEBUG:teuthology.orchestra.run.smithi043:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2023-08-09T13:50:41.884 INFO:teuthology.orchestra.run.smithi043.stdout:1 2023-08-09T13:50:41.888 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-09T13:50:41.889 DEBUG:teuthology.orchestra.run.smithi043:> route 2023-08-09T13:50:41.950 INFO:teuthology.orchestra.run.smithi043.stdout:Kernel IP routing table 2023-08-09T13:50:41.950 INFO:teuthology.orchestra.run.smithi043.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2023-08-09T13:50:41.950 INFO:teuthology.orchestra.run.smithi043.stdout:default _gateway 0.0.0.0 UG 100 0 0 ens1f0 2023-08-09T13:50:41.950 INFO:teuthology.orchestra.run.smithi043.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 ens1f0 2023-08-09T13:50:41.950 INFO:teuthology.orchestra.run.smithi043.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2023-08-09T13:50:41.951 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-09T13:50:41.951 DEBUG:teuthology.orchestra.run.smithi043:> (cd / && exec bash -c ' 2023-08-09T13:50:41.952 DEBUG:teuthology.orchestra.run.smithi043:> set -e 2023-08-09T13:50:41.952 DEBUG:teuthology.orchestra.run.smithi043:> sudo iptables -A FORWARD -o ens1f0 -i ceph-brx -j ACCEPT 2023-08-09T13:50:41.953 DEBUG:teuthology.orchestra.run.smithi043:> sudo iptables -A FORWARD -i ens1f0 -o ceph-brx -j ACCEPT 2023-08-09T13:50:41.953 DEBUG:teuthology.orchestra.run.smithi043:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o ens1f0 -j MASQUERADE 2023-08-09T13:50:41.954 DEBUG:teuthology.orchestra.run.smithi043:> ') 2023-08-09T13:50:42.158 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-09T13:50:42.158 DEBUG:teuthology.orchestra.run.smithi043:> ip netns list 2023-08-09T13:50:42.217 INFO:teuthology.orchestra.run.smithi043.stdout:ceph-ns--home-ubuntu-cephtest-mnt.2 (id: 1) 2023-08-09T13:50:42.217 INFO:teuthology.orchestra.run.smithi043.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2023-08-09T13:50:42.218 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2023-08-09T13:50:42.218 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-09T13:50:42.218 DEBUG:teuthology.orchestra.run.smithi043:> sudo ip link set brx.0 up 2023-08-09T13:50:42.319 INFO:tasks.cephfs.fuse_mount:Client client.0 config is {} 2023-08-09T13:50:42.319 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-08-09T13:50:42.320 DEBUG:teuthology.orchestra.run.smithi043:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2023-08-09T13:50:42.380 INFO:teuthology.orchestra.run.smithi043.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2023-08-09T13:50:42.381 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-08-09T13:50:42.381 DEBUG:teuthology.orchestra.run.smithi043:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2023-08-09T13:50:42.439 DEBUG:teuthology.orchestra.run.smithi043:> sudo modprobe fuse 2023-08-09T13:50:42.545 INFO:teuthology.orchestra.run:Running command with timeout 30 2023-08-09T13:50:42.546 DEBUG:teuthology.orchestra.run.smithi043:> sudo mount -t fusectl /sys/fs/fuse/connections /sys/fs/fuse/connections 2023-08-09T13:50:42.651 INFO:teuthology.orchestra.run.smithi043.stderr:mount: /sys/fs/fuse/connections: /sys/fs/fuse/connections already mounted or mount point busy. 2023-08-09T13:50:42.655 DEBUG:teuthology.orchestra.run:got remote process result: 32 2023-08-09T13:50:42.656 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-09T13:50:42.656 DEBUG:teuthology.orchestra.run.smithi043:> ls /sys/fs/fuse/connections 2023-08-09T13:50:42.716 INFO:tasks.cephfs.fuse_mount:Pre-mount connections: [] 2023-08-09T13:50:42.717 DEBUG:teuthology.orchestra.run.smithi043:> (cd /home/ubuntu/cephtest && exec sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-fuse -f --admin-socket '/var/run/ceph/$cluster-$name.$pid.asok' /home/ubuntu/cephtest/mnt.0 --id 0) 2023-08-09T13:50:42.723 DEBUG:teuthology.orchestra.run.smithi043:> sudo modprobe fuse 2023-08-09T13:50:42.809 INFO:teuthology.orchestra.run:Running command with timeout 30 2023-08-09T13:50:42.809 DEBUG:teuthology.orchestra.run.smithi043:> sudo mount -t fusectl /sys/fs/fuse/connections /sys/fs/fuse/connections 2023-08-09T13:50:42.883 INFO:teuthology.orchestra.run.smithi043.stderr:mount: /sys/fs/fuse/connections: /sys/fs/fuse/connections already mounted or mount point busy. 2023-08-09T13:50:42.887 DEBUG:teuthology.orchestra.run:got remote process result: 32 2023-08-09T13:50:42.888 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-09T13:50:42.889 DEBUG:teuthology.orchestra.run.smithi043:> ls /sys/fs/fuse/connections 2023-08-09T13:50:42.923 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi043.stderr:2023-08-09T13:50:42.921+0000 7fe7ffc45480 -1 init, newargv = 0x55c64527c4d0 newargc=15 2023-08-09T13:50:42.923 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi043.stderr:ceph-fuse[118491]: starting ceph client 2023-08-09T13:50:42.936 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi043.stderr:ceph-fuse[118491]: starting fuse 2023-08-09T13:50:42.937 INFO:teuthology.orchestra.run.smithi043.stdout:40 2023-08-09T13:50:42.938 INFO:tasks.cephfs.fuse_mount:Post-mount connections: [40] 2023-08-09T13:50:42.938 DEBUG:teuthology.orchestra.run.smithi043:> sudo stdin-killer -- python3 -c ' 2023-08-09T13:50:42.939 DEBUG:teuthology.orchestra.run.smithi043:> import glob 2023-08-09T13:50:42.939 DEBUG:teuthology.orchestra.run.smithi043:> import re 2023-08-09T13:50:42.939 DEBUG:teuthology.orchestra.run.smithi043:> import os 2023-08-09T13:50:42.940 DEBUG:teuthology.orchestra.run.smithi043:> import subprocess 2023-08-09T13:50:42.940 DEBUG:teuthology.orchestra.run.smithi043:> 2023-08-09T13:50:42.940 DEBUG:teuthology.orchestra.run.smithi043:> def _find_admin_socket(client_name): 2023-08-09T13:50:42.941 DEBUG:teuthology.orchestra.run.smithi043:> asok_path = "/var/run/ceph/ceph-client.0.*.asok" 2023-08-09T13:50:42.941 DEBUG:teuthology.orchestra.run.smithi043:> files = glob.glob(asok_path) 2023-08-09T13:50:42.942 DEBUG:teuthology.orchestra.run.smithi043:> mountpoint = "/home/ubuntu/cephtest/mnt.0" 2023-08-09T13:50:42.942 DEBUG:teuthology.orchestra.run.smithi043:> 2023-08-09T13:50:42.942 DEBUG:teuthology.orchestra.run.smithi043:> # Given a non-glob path, it better be there 2023-08-09T13:50:42.943 DEBUG:teuthology.orchestra.run.smithi043:> if "*" not in asok_path: 2023-08-09T13:50:42.943 DEBUG:teuthology.orchestra.run.smithi043:> assert(len(files) == 1) 2023-08-09T13:50:42.944 DEBUG:teuthology.orchestra.run.smithi043:> return files[0] 2023-08-09T13:50:42.945 DEBUG:teuthology.orchestra.run.smithi043:> 2023-08-09T13:50:42.945 DEBUG:teuthology.orchestra.run.smithi043:> for f in files: 2023-08-09T13:50:42.946 DEBUG:teuthology.orchestra.run.smithi043:> pid = re.match(".*\.(\d+)\.asok$", f).group(1) 2023-08-09T13:50:42.946 DEBUG:teuthology.orchestra.run.smithi043:> if os.path.exists("/proc/{0}".format(pid)): 2023-08-09T13:50:42.947 DEBUG:teuthology.orchestra.run.smithi043:> with open("/proc/{0}/cmdline".format(pid), '"'"'r'"'"') as proc_f: 2023-08-09T13:50:42.948 DEBUG:teuthology.orchestra.run.smithi043:> contents = proc_f.read() 2023-08-09T13:50:42.948 DEBUG:teuthology.orchestra.run.smithi043:> if mountpoint in contents: 2023-08-09T13:50:42.949 DEBUG:teuthology.orchestra.run.smithi043:> return f 2023-08-09T13:50:42.950 DEBUG:teuthology.orchestra.run.smithi043:> raise RuntimeError("Client socket {0} not found".format(client_name)) 2023-08-09T13:50:42.951 DEBUG:teuthology.orchestra.run.smithi043:> 2023-08-09T13:50:42.951 DEBUG:teuthology.orchestra.run.smithi043:> print(_find_admin_socket("client.0")) 2023-08-09T13:50:42.951 DEBUG:teuthology.orchestra.run.smithi043:> ' 2023-08-09T13:50:43.150 INFO:teuthology.orchestra.run.smithi043.stdout:/var/run/ceph/ceph-client.0.118491.asok 2023-08-09T13:50:43.154 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:43 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2023-08-09T13:50:43.167 INFO:tasks.cephfs.fuse_mount:Found client admin socket at /var/run/ceph/ceph-client.0.118491.asok 2023-08-09T13:50:43.168 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-09T13:50:43.169 DEBUG:teuthology.orchestra.run.smithi043:> sudo ceph --admin-daemon /var/run/ceph/ceph-client.0.118491.asok status 2023-08-09T13:50:43.421 INFO:teuthology.orchestra.run.smithi043.stdout:{ 2023-08-09T13:50:43.421 INFO:teuthology.orchestra.run.smithi043.stdout: "metadata": { 2023-08-09T13:50:43.421 INFO:teuthology.orchestra.run.smithi043.stdout: "ceph_sha1": "668a86f232c9454b2ce5928d1432a29a72964d85", 2023-08-09T13:50:43.422 INFO:teuthology.orchestra.run.smithi043.stdout: "ceph_version": "ceph version 18.0.0-5389-g668a86f2 (668a86f232c9454b2ce5928d1432a29a72964d85) reef (dev)", 2023-08-09T13:50:43.422 INFO:teuthology.orchestra.run.smithi043.stdout: "entity_id": "0", 2023-08-09T13:50:43.422 INFO:teuthology.orchestra.run.smithi043.stdout: "hostname": "smithi043", 2023-08-09T13:50:43.422 INFO:teuthology.orchestra.run.smithi043.stdout: "mount_point": "/home/ubuntu/cephtest/mnt.0", 2023-08-09T13:50:43.422 INFO:teuthology.orchestra.run.smithi043.stdout: "pid": "118491", 2023-08-09T13:50:43.422 INFO:teuthology.orchestra.run.smithi043.stdout: "root": "/" 2023-08-09T13:50:43.422 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2023-08-09T13:50:43.423 INFO:teuthology.orchestra.run.smithi043.stdout: "dentry_count": 0, 2023-08-09T13:50:43.423 INFO:teuthology.orchestra.run.smithi043.stdout: "dentry_pinned_count": 0, 2023-08-09T13:50:43.423 INFO:teuthology.orchestra.run.smithi043.stdout: "id": 7194, 2023-08-09T13:50:43.423 INFO:teuthology.orchestra.run.smithi043.stdout: "inst": { 2023-08-09T13:50:43.423 INFO:teuthology.orchestra.run.smithi043.stdout: "name": { 2023-08-09T13:50:43.423 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "client", 2023-08-09T13:50:43.423 INFO:teuthology.orchestra.run.smithi043.stdout: "num": 7194 2023-08-09T13:50:43.424 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2023-08-09T13:50:43.424 INFO:teuthology.orchestra.run.smithi043.stdout: "addr": { 2023-08-09T13:50:43.424 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "v1", 2023-08-09T13:50:43.424 INFO:teuthology.orchestra.run.smithi043.stdout: "addr": "192.168.0.1:0", 2023-08-09T13:50:43.424 INFO:teuthology.orchestra.run.smithi043.stdout: "nonce": 2495236434 2023-08-09T13:50:43.424 INFO:teuthology.orchestra.run.smithi043.stdout: } 2023-08-09T13:50:43.424 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2023-08-09T13:50:43.424 INFO:teuthology.orchestra.run.smithi043.stdout: "addr": { 2023-08-09T13:50:43.425 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "v1", 2023-08-09T13:50:43.425 INFO:teuthology.orchestra.run.smithi043.stdout: "addr": "192.168.0.1:0", 2023-08-09T13:50:43.425 INFO:teuthology.orchestra.run.smithi043.stdout: "nonce": 2495236434 2023-08-09T13:50:43.425 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2023-08-09T13:50:43.425 INFO:teuthology.orchestra.run.smithi043.stdout: "inst_str": "client.7194 192.168.0.1:0/2495236434", 2023-08-09T13:50:43.425 INFO:teuthology.orchestra.run.smithi043.stdout: "addr_str": "192.168.0.1:0/2495236434", 2023-08-09T13:50:43.425 INFO:teuthology.orchestra.run.smithi043.stdout: "inode_count": 1, 2023-08-09T13:50:43.426 INFO:teuthology.orchestra.run.smithi043.stdout: "mds_epoch": 35, 2023-08-09T13:50:43.426 INFO:teuthology.orchestra.run.smithi043.stdout: "osd_epoch": 90, 2023-08-09T13:50:43.426 INFO:teuthology.orchestra.run.smithi043.stdout: "osd_epoch_barrier": 0, 2023-08-09T13:50:43.426 INFO:teuthology.orchestra.run.smithi043.stdout: "blocklisted": false, 2023-08-09T13:50:43.426 INFO:teuthology.orchestra.run.smithi043.stdout: "fs_name": "cephfs" 2023-08-09T13:50:43.426 INFO:teuthology.orchestra.run.smithi043.stdout:} 2023-08-09T13:50:43.436 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-09T13:50:43.436 DEBUG:teuthology.orchestra.run.smithi043:> stat --file-system '--printf=%T 2023-08-09T13:50:43.437 DEBUG:teuthology.orchestra.run.smithi043:> ' -- /home/ubuntu/cephtest/mnt.0 2023-08-09T13:50:43.495 INFO:teuthology.orchestra.run.smithi043.stdout:fuseblk 2023-08-09T13:50:43.496 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.0 2023-08-09T13:50:43.496 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-09T13:50:43.497 DEBUG:teuthology.orchestra.run.smithi043:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2023-08-09T13:50:43.604 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs volume ls 2023-08-09T13:50:43.841 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:43.837+0000 7fcf351b5640 1 Processor -- start 2023-08-09T13:50:43.841 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:43.837+0000 7fcf351b5640 1 -- start start 2023-08-09T13:50:43.841 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:43.839+0000 7fcf351b5640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fcf30143c20 0x7fcf3006f540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:50:43.841 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:43.839+0000 7fcf351b5640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7fcf3006cfe0 0x7fcf300c8b50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:50:43.842 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:43.839+0000 7fcf351b5640 1 -- --> v1:172.21.15.203:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fcf300c9090 con 0x7fcf300c86f0 2023-08-09T13:50:43.842 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:43.839+0000 7fcf351b5640 1 -- --> v2:172.21.15.43:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fcf30144000 con 0x7fcf3006cfe0 2023-08-09T13:50:43.842 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:43.839+0000 7fcf351b5640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcf3006c440 con 0x7fcf30143c20 2023-08-09T13:50:43.842 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:43.839+0000 7fcf2f59e640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7fcf3006cfe0 0x7fcf300c8b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:50:43.842 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:43.839+0000 7fcf2f59e640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7fcf3006cfe0 0x7fcf300c8b50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:45144/0 (socket says 172.21.15.43:45144) 2023-08-09T13:50:43.842 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:43.839+0000 7fcf2f59e640 1 -- 172.21.15.43:0/2632782701 learned_addr learned my addr 172.21.15.43:0/2632782701 (peer_addr_for_me v2:172.21.15.43:0/0) 2023-08-09T13:50:43.842 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:43.840+0000 7fcf2ed9d640 1 --2- 172.21.15.43:0/2632782701 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fcf30143c20 0x7fcf3006f540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:50:43.843 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:43.840+0000 7fcf2f59e640 1 -- 172.21.15.43:0/2632782701 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fcf30143c20 msgr2=0x7fcf3006f540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-09T13:50:43.844 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:43.840+0000 7fcf2f59e640 1 --2- 172.21.15.43:0/2632782701 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fcf30143c20 0x7fcf3006f540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:50:43.844 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:43.840+0000 7fcf2f59e640 1 -- 172.21.15.43:0/2632782701 >> v1:172.21.15.203:6789/0 conn(0x7fcf300c86f0 legacy=0x7fcf3006c8d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-09T13:50:43.844 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:43.840+0000 7fcf2f59e640 1 -- 172.21.15.43:0/2632782701 --> v2:172.21.15.43:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcf3006b830 con 0x7fcf3006cfe0 2023-08-09T13:50:43.844 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:43.841+0000 7fcf2f59e640 1 --2- 172.21.15.43:0/2632782701 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fcf3006cfe0 0x7fcf300c8b50 secure :-1 s=READY pgs=1049 cs=0 l=1 rev1=1 crypto rx=0x7fcf28009e50 tx=0x7fcf2802be50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ecfda4fe4cc3b24c server_cookie=0 in_seq=0 out_seq=0 2023-08-09T13:50:43.844 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:43.841+0000 7fcf2dd9b640 1 -- 172.21.15.43:0/2632782701 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcf2802c9f0 con 0x7fcf3006cfe0 2023-08-09T13:50:43.844 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:43.841+0000 7fcf2dd9b640 1 -- 172.21.15.43:0/2632782701 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcf2802cb50 con 0x7fcf3006cfe0 2023-08-09T13:50:43.845 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:43.841+0000 7fcf2dd9b640 1 -- 172.21.15.43:0/2632782701 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcf280343f0 con 0x7fcf3006cfe0 2023-08-09T13:50:43.845 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:43.842+0000 7fcf351b5640 1 -- 172.21.15.43:0/2632782701 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fcf3006cfe0 msgr2=0x7fcf300c8b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:50:43.845 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:43.842+0000 7fcf351b5640 1 --2- 172.21.15.43:0/2632782701 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fcf3006cfe0 0x7fcf300c8b50 secure :-1 s=READY pgs=1049 cs=0 l=1 rev1=1 crypto rx=0x7fcf28009e50 tx=0x7fcf2802be50 comp rx=0 tx=0).stop 2023-08-09T13:50:43.845 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:43.842+0000 7fcf351b5640 1 -- 172.21.15.43:0/2632782701 shutdown_connections 2023-08-09T13:50:43.845 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:43.842+0000 7fcf351b5640 1 --2- 172.21.15.43:0/2632782701 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fcf3006cfe0 0x7fcf300c8b50 unknown :-1 s=CLOSED pgs=1049 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:50:43.845 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:43.842+0000 7fcf351b5640 1 --2- 172.21.15.43:0/2632782701 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fcf30143c20 0x7fcf3006f540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:50:43.845 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:43.842+0000 7fcf351b5640 1 -- 172.21.15.43:0/2632782701 >> 172.21.15.43:0/2632782701 conn(0x7fcf300ce0a0 msgr2=0x7fcf300ce480 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-09T13:50:43.846 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:43.843+0000 7fcf351b5640 1 -- 172.21.15.43:0/2632782701 shutdown_connections 2023-08-09T13:50:43.846 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:43.843+0000 7fcf351b5640 1 -- 172.21.15.43:0/2632782701 wait complete. 2023-08-09T13:50:43.846 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:43.844+0000 7fcf351b5640 1 Processor -- start 2023-08-09T13:50:43.846 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:43.845+0000 7fcf351b5640 1 -- start start 2023-08-09T13:50:43.847 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:43.845+0000 7fcf351b5640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fcf3006cfe0 0x7fcf300c7a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:50:43.847 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:43.845+0000 7fcf351b5640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fcf300c86f0 0x7fcf300c43e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:50:43.848 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:43.845+0000 7fcf351b5640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fcf30143c20 0x7fcf301598e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:50:43.848 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:43.846+0000 7fcf351b5640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcf300c7f90 con 0x7fcf300c86f0 2023-08-09T13:50:43.848 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:43.846+0000 7fcf351b5640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcf300c58e0 con 0x7fcf3006cfe0 2023-08-09T13:50:43.848 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:43.846+0000 7fcf351b5640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcf300c5600 con 0x7fcf30143c20 2023-08-09T13:50:43.848 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:43.846+0000 7fcf2e59c640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fcf300c86f0 0x7fcf300c43e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:50:43.848 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:43.846+0000 7fcf2e59c640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fcf300c86f0 0x7fcf300c43e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:45160/0 (socket says 172.21.15.43:45160) 2023-08-09T13:50:43.848 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:43.846+0000 7fcf2e59c640 1 -- 172.21.15.43:0/741190909 learned_addr learned my addr 172.21.15.43:0/741190909 (peer_addr_for_me v2:172.21.15.43:0/0) 2023-08-09T13:50:43.848 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:43.846+0000 7fcf2ed9d640 1 --2- 172.21.15.43:0/741190909 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fcf3006cfe0 0x7fcf300c7a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:50:43.849 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:43.846+0000 7fcf2f59e640 1 --2- 172.21.15.43:0/741190909 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fcf30143c20 0x7fcf301598e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:50:43.850 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:43.846+0000 7fcf2e59c640 1 -- 172.21.15.43:0/741190909 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fcf30143c20 msgr2=0x7fcf301598e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:50:43.850 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:43.846+0000 7fcf2e59c640 1 --2- 172.21.15.43:0/741190909 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fcf30143c20 0x7fcf301598e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:50:43.850 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:43.846+0000 7fcf2e59c640 1 -- 172.21.15.43:0/741190909 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fcf3006cfe0 msgr2=0x7fcf300c7a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:50:43.850 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:43.846+0000 7fcf2e59c640 1 --2- 172.21.15.43:0/741190909 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fcf3006cfe0 0x7fcf300c7a50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:50:43.850 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:43.846+0000 7fcf2e59c640 1 -- 172.21.15.43:0/741190909 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcf28009b30 con 0x7fcf300c86f0 2023-08-09T13:50:43.850 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:43.846+0000 7fcf2f59e640 1 --2- 172.21.15.43:0/741190909 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fcf30143c20 0x7fcf301598e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-08-09T13:50:43.850 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:43.847+0000 7fcf2e59c640 1 --2- 172.21.15.43:0/741190909 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fcf300c86f0 0x7fcf300c43e0 secure :-1 s=READY pgs=1050 cs=0 l=1 rev1=1 crypto rx=0x7fcf1800b3f0 tx=0x7fcf1800b8b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-09T13:50:43.851 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:43.847+0000 7fcf277fe640 1 -- 172.21.15.43:0/741190909 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcf18011830 con 0x7fcf300c86f0 2023-08-09T13:50:43.851 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:43.847+0000 7fcf277fe640 1 -- 172.21.15.43:0/741190909 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcf18011990 con 0x7fcf300c86f0 2023-08-09T13:50:43.851 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:43.847+0000 7fcf277fe640 1 -- 172.21.15.43:0/741190909 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcf18011d40 con 0x7fcf300c86f0 2023-08-09T13:50:43.851 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:43.847+0000 7fcf351b5640 1 -- 172.21.15.43:0/741190909 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcf300b1ec0 con 0x7fcf300c86f0 2023-08-09T13:50:43.851 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:43.847+0000 7fcf351b5640 1 -- 172.21.15.43:0/741190909 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcf300b2480 con 0x7fcf300c86f0 2023-08-09T13:50:43.852 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:43.848+0000 7fcf351b5640 1 -- 172.21.15.43:0/741190909 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcf300739f0 con 0x7fcf300c86f0 2023-08-09T13:50:43.852 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:43.849+0000 7fcf277fe640 1 -- 172.21.15.43:0/741190909 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 29) v1 ==== 88443+0+0 (secure 0 0 0) 0x7fcf1801a400 con 0x7fcf300c86f0 2023-08-09T13:50:43.852 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:43.850+0000 7fcf277fe640 1 --2- 172.21.15.43:0/741190909 >> [v2:172.21.15.203:6832/3325141227,v1:172.21.15.203:6833/3325141227] conn(0x7fcf1006a170 0x7fcf1006c630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:50:43.852 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:43.850+0000 7fcf277fe640 1 -- 172.21.15.43:0/741190909 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(90..90 src has 1..90) v4 ==== 6426+0+0 (secure 0 0 0) 0x7fcf180886f0 con 0x7fcf300c86f0 2023-08-09T13:50:43.853 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:43.851+0000 7fcf2ed9d640 1 --2- 172.21.15.43:0/741190909 >> [v2:172.21.15.203:6832/3325141227,v1:172.21.15.203:6833/3325141227] conn(0x7fcf1006a170 0x7fcf1006c630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:50:43.853 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:43.852+0000 7fcf2ed9d640 1 --2- 172.21.15.43:0/741190909 >> [v2:172.21.15.203:6832/3325141227,v1:172.21.15.203:6833/3325141227] conn(0x7fcf1006a170 0x7fcf1006c630 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7fcf20006fd0 tx=0x7fcf20008040 comp rx=0 tx=0).ready entity=mgr.5519 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-09T13:50:43.857 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:43.855+0000 7fcf277fe640 1 -- 172.21.15.43:0/741190909 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178066 (secure 0 0 0) 0x7fcf18053b50 con 0x7fcf300c86f0 2023-08-09T13:50:44.076 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:44.074+0000 7fcf351b5640 1 -- 172.21.15.43:0/741190909 --> [v2:172.21.15.203:6832/3325141227,v1:172.21.15.203:6833/3325141227] -- mgr_command(tid 0: {"prefix": "fs volume ls", "target": ["mon-mgr", ""]}) v1 -- 0x7fcf3006bdf0 con 0x7fcf1006a170 2023-08-09T13:50:44.079 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:44.078+0000 7fcf277fe640 1 -- 172.21.15.43:0/741190909 <== mgr.5519 v2:172.21.15.203:6832/3325141227 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+41 (secure 0 0 0) 0x7fcf2000b200 con 0x7fcf1006a170 2023-08-09T13:50:44.080 INFO:teuthology.orchestra.run.smithi043.stdout:[ 2023-08-09T13:50:44.080 INFO:teuthology.orchestra.run.smithi043.stdout: { 2023-08-09T13:50:44.080 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "cephfs" 2023-08-09T13:50:44.080 INFO:teuthology.orchestra.run.smithi043.stdout: } 2023-08-09T13:50:44.080 INFO:teuthology.orchestra.run.smithi043.stdout:] 2023-08-09T13:50:44.082 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:44.080+0000 7fcf351b5640 1 -- 172.21.15.43:0/741190909 >> [v2:172.21.15.203:6832/3325141227,v1:172.21.15.203:6833/3325141227] conn(0x7fcf1006a170 msgr2=0x7fcf1006c630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:50:44.083 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:44.080+0000 7fcf351b5640 1 --2- 172.21.15.43:0/741190909 >> [v2:172.21.15.203:6832/3325141227,v1:172.21.15.203:6833/3325141227] conn(0x7fcf1006a170 0x7fcf1006c630 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7fcf20006fd0 tx=0x7fcf20008040 comp rx=0 tx=0).stop 2023-08-09T13:50:44.083 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:44.080+0000 7fcf351b5640 1 -- 172.21.15.43:0/741190909 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fcf300c86f0 msgr2=0x7fcf300c43e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:50:44.083 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:44.080+0000 7fcf351b5640 1 --2- 172.21.15.43:0/741190909 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fcf300c86f0 0x7fcf300c43e0 secure :-1 s=READY pgs=1050 cs=0 l=1 rev1=1 crypto rx=0x7fcf1800b3f0 tx=0x7fcf1800b8b0 comp rx=0 tx=0).stop 2023-08-09T13:50:44.083 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:44.081+0000 7fcf351b5640 1 -- 172.21.15.43:0/741190909 shutdown_connections 2023-08-09T13:50:44.083 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:44.081+0000 7fcf351b5640 1 --2- 172.21.15.43:0/741190909 >> [v2:172.21.15.203:6832/3325141227,v1:172.21.15.203:6833/3325141227] conn(0x7fcf1006a170 0x7fcf1006c630 unknown :-1 s=CLOSED pgs=75 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:50:44.083 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:44.081+0000 7fcf351b5640 1 --2- 172.21.15.43:0/741190909 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fcf30143c20 0x7fcf301598e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:50:44.084 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:44.081+0000 7fcf351b5640 1 --2- 172.21.15.43:0/741190909 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fcf300c86f0 0x7fcf300c43e0 unknown :-1 s=CLOSED pgs=1050 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:50:44.084 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:44.081+0000 7fcf351b5640 1 --2- 172.21.15.43:0/741190909 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fcf3006cfe0 0x7fcf300c7a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:50:44.084 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:44.081+0000 7fcf351b5640 1 -- 172.21.15.43:0/741190909 >> 172.21.15.43:0/741190909 conn(0x7fcf300ce0a0 msgr2=0x7fcf300c9220 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-09T13:50:44.084 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:44.081+0000 7fcf351b5640 1 -- 172.21.15.43:0/741190909 shutdown_connections 2023-08-09T13:50:44.084 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:44.081+0000 7fcf351b5640 1 -- 172.21.15.43:0/741190909 wait complete. 2023-08-09T13:50:44.101 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph mgr module enable snap_schedule 2023-08-09T13:50:44.333 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:44.332+0000 7f08b5ccc640 1 Processor -- start 2023-08-09T13:50:44.334 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:44.332+0000 7f08b5ccc640 1 -- start start 2023-08-09T13:50:44.335 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:44.334+0000 7f08b5ccc640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f08b0143c20 0x7f08b006f540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:50:44.336 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:44.334+0000 7f08b5ccc640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f08b006cfe0 0x7f08b00c8b50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:50:44.336 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:44.334+0000 7f08b5ccc640 1 -- --> v1:172.21.15.43:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f08b00c9090 con 0x7f08b00c86f0 2023-08-09T13:50:44.336 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:44.334+0000 7f08b5ccc640 1 -- --> v2:172.21.15.43:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f08b0144000 con 0x7f08b006cfe0 2023-08-09T13:50:44.336 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:44.334+0000 7f08b5ccc640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f08b006c440 con 0x7f08b0143c20 2023-08-09T13:50:44.336 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:44.334+0000 7f08affff640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f08b006cfe0 0x7f08b00c8b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:50:44.337 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:44.334+0000 7f08a7fff640 1 --1- >> v1:172.21.15.43:6789/0 conn(0x7f08b00c86f0 0x7f08b006c8d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.43:6789/0 says I am v1:172.21.15.43:55238/0 (socket says 172.21.15.43:55238) 2023-08-09T13:50:44.337 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:44.334+0000 7f08a7fff640 1 -- 172.21.15.43:0/3245281646 learned_addr learned my addr 172.21.15.43:0/3245281646 (peer_addr_for_me v1:172.21.15.43:0/0) 2023-08-09T13:50:44.338 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:44.334+0000 7f08af7fe640 1 --2- 172.21.15.43:0/3245281646 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f08b0143c20 0x7f08b006f540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:50:44.338 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:44.335+0000 7f08affff640 1 -- 172.21.15.43:0/3245281646 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f08b0143c20 msgr2=0x7f08b006f540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-09T13:50:44.338 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:44.335+0000 7f08affff640 1 --2- 172.21.15.43:0/3245281646 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f08b0143c20 0x7f08b006f540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:50:44.338 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:44.335+0000 7f08affff640 1 -- 172.21.15.43:0/3245281646 >> v1:172.21.15.43:6789/0 conn(0x7f08b00c86f0 legacy=0x7f08b006c8d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:50:44.338 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:44.335+0000 7f08affff640 1 -- 172.21.15.43:0/3245281646 --> v2:172.21.15.43:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f08b006b830 con 0x7f08b006cfe0 2023-08-09T13:50:44.339 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:44.336+0000 7f08affff640 1 --2- 172.21.15.43:0/3245281646 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f08b006cfe0 0x7f08b00c8b50 secure :-1 s=READY pgs=1052 cs=0 l=1 rev1=1 crypto rx=0x7f08a8009d90 tx=0x7f08a802be20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fb059022e86166bd server_cookie=0 in_seq=0 out_seq=0 2023-08-09T13:50:44.339 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:44.336+0000 7f08aeffd640 1 -- 172.21.15.43:0/3245281646 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f08a802cbd0 con 0x7f08b006cfe0 2023-08-09T13:50:44.339 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:44.336+0000 7f08aeffd640 1 -- 172.21.15.43:0/3245281646 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f08a802cd30 con 0x7f08b006cfe0 2023-08-09T13:50:44.339 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:44.336+0000 7f08aeffd640 1 -- 172.21.15.43:0/3245281646 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f08a802cea0 con 0x7f08b006cfe0 2023-08-09T13:50:44.340 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:44.337+0000 7f08b5ccc640 1 -- 172.21.15.43:0/3245281646 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f08b006cfe0 msgr2=0x7f08b00c8b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:50:44.341 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:44.337+0000 7f08b5ccc640 1 --2- 172.21.15.43:0/3245281646 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f08b006cfe0 0x7f08b00c8b50 secure :-1 s=READY pgs=1052 cs=0 l=1 rev1=1 crypto rx=0x7f08a8009d90 tx=0x7f08a802be20 comp rx=0 tx=0).stop 2023-08-09T13:50:44.341 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:44.337+0000 7f08b5ccc640 1 -- 172.21.15.43:0/3245281646 shutdown_connections 2023-08-09T13:50:44.341 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:44.337+0000 7f08b5ccc640 1 --2- 172.21.15.43:0/3245281646 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f08b006cfe0 0x7f08b00c8b50 unknown :-1 s=CLOSED pgs=1052 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:50:44.341 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:44.337+0000 7f08b5ccc640 1 --2- 172.21.15.43:0/3245281646 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f08b0143c20 0x7f08b006f540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:50:44.341 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:44.337+0000 7f08b5ccc640 1 -- 172.21.15.43:0/3245281646 >> 172.21.15.43:0/3245281646 conn(0x7f08b00ce0a0 msgr2=0x7f08b00ce480 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-09T13:50:44.341 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:44.337+0000 7f08b5ccc640 1 -- 172.21.15.43:0/3245281646 shutdown_connections 2023-08-09T13:50:44.341 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:44.337+0000 7f08b5ccc640 1 -- 172.21.15.43:0/3245281646 wait complete. 2023-08-09T13:50:44.343 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:44.339+0000 7f08b5ccc640 1 Processor -- start 2023-08-09T13:50:44.344 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:44.339+0000 7f08b5ccc640 1 -- start start 2023-08-09T13:50:44.344 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:44.340+0000 7f08b5ccc640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f08b006cfe0 0x7f08b00c68b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:50:44.344 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:44.340+0000 7f08b5ccc640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f08b00c86f0 0x7f08b00c78c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:50:44.344 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:44.340+0000 7f08b5ccc640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f08b0143c20 0x7f08b013dac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:50:44.344 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:44.340+0000 7f08b5ccc640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f08b00c7e00 con 0x7f08b00c86f0 2023-08-09T13:50:44.344 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:44.340+0000 7f08b5ccc640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f08b00ab8c0 con 0x7f08b0143c20 2023-08-09T13:50:44.344 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:44.340+0000 7f08b5ccc640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f08b00abb60 con 0x7f08b006cfe0 2023-08-09T13:50:44.345 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:44.340+0000 7f08a7fff640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f08b00c86f0 0x7f08b00c78c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:50:44.345 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:44.341+0000 7f08a7fff640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f08b00c86f0 0x7f08b00c78c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:45190/0 (socket says 172.21.15.43:45190) 2023-08-09T13:50:44.345 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:44.341+0000 7f08a7fff640 1 -- 172.21.15.43:0/2077118964 learned_addr learned my addr 172.21.15.43:0/2077118964 (peer_addr_for_me v2:172.21.15.43:0/0) 2023-08-09T13:50:44.345 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:44.341+0000 7f08af7fe640 1 --2- 172.21.15.43:0/2077118964 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f08b006cfe0 0x7f08b00c68b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:50:44.345 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:44.341+0000 7f08affff640 1 --2- 172.21.15.43:0/2077118964 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f08b0143c20 0x7f08b013dac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:50:44.345 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:44.341+0000 7f08a7fff640 1 -- 172.21.15.43:0/2077118964 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f08b006cfe0 msgr2=0x7f08b00c68b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:50:44.345 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:44.341+0000 7f08a7fff640 1 --2- 172.21.15.43:0/2077118964 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f08b006cfe0 0x7f08b00c68b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:50:44.346 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:44.341+0000 7f08a7fff640 1 -- 172.21.15.43:0/2077118964 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f08b0143c20 msgr2=0x7f08b013dac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:50:44.346 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:44.341+0000 7f08a7fff640 1 --2- 172.21.15.43:0/2077118964 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f08b0143c20 0x7f08b013dac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:50:44.346 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:44.341+0000 7f08a7fff640 1 -- 172.21.15.43:0/2077118964 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f08a8009a70 con 0x7f08b00c86f0 2023-08-09T13:50:44.346 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:44.341+0000 7f08a7fff640 1 --2- 172.21.15.43:0/2077118964 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f08b00c86f0 0x7f08b00c78c0 secure :-1 s=READY pgs=1053 cs=0 l=1 rev1=1 crypto rx=0x7f089800c4a0 tx=0x7f089800c960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-09T13:50:44.346 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:44.342+0000 7f08acff9640 1 -- 172.21.15.43:0/2077118964 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0898005ce0 con 0x7f08b00c86f0 2023-08-09T13:50:44.346 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:44.342+0000 7f08acff9640 1 -- 172.21.15.43:0/2077118964 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0898005e40 con 0x7f08b00c86f0 2023-08-09T13:50:44.347 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:44.342+0000 7f08b5ccc640 1 -- 172.21.15.43:0/2077118964 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f08b00abe40 con 0x7f08b00c86f0 2023-08-09T13:50:44.347 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:44.342+0000 7f08acff9640 1 -- 172.21.15.43:0/2077118964 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0898007270 con 0x7f08b00c86f0 2023-08-09T13:50:44.347 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:44.342+0000 7f08b5ccc640 1 -- 172.21.15.43:0/2077118964 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f08b00ac400 con 0x7f08b00c86f0 2023-08-09T13:50:44.347 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:44.343+0000 7f08b5ccc640 1 -- 172.21.15.43:0/2077118964 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f08b00739f0 con 0x7f08b00c86f0 2023-08-09T13:50:44.348 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:44.344+0000 7f08acff9640 1 -- 172.21.15.43:0/2077118964 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 29) v1 ==== 88443+0+0 (secure 0 0 0) 0x7f0898007490 con 0x7f08b00c86f0 2023-08-09T13:50:44.348 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:44.345+0000 7f08acff9640 1 --2- 172.21.15.43:0/2077118964 >> [v2:172.21.15.203:6832/3325141227,v1:172.21.15.203:6833/3325141227] conn(0x7f089006a170 0x7f089006c630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:50:44.348 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:44.345+0000 7f08acff9640 1 -- 172.21.15.43:0/2077118964 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(90..90 src has 1..90) v4 ==== 6426+0+0 (secure 0 0 0) 0x7f0898087da0 con 0x7f08b00c86f0 2023-08-09T13:50:44.348 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:44.346+0000 7f08af7fe640 1 --2- 172.21.15.43:0/2077118964 >> [v2:172.21.15.203:6832/3325141227,v1:172.21.15.203:6833/3325141227] conn(0x7f089006a170 0x7f089006c630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:50:44.349 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:44.347+0000 7f08af7fe640 1 --2- 172.21.15.43:0/2077118964 >> [v2:172.21.15.203:6832/3325141227,v1:172.21.15.203:6833/3325141227] conn(0x7f089006a170 0x7f089006c630 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f08a0006fd0 tx=0x7f08a0008040 comp rx=0 tx=0).ready entity=mgr.5519 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-09T13:50:44.350 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:44.349+0000 7f08acff9640 1 -- 172.21.15.43:0/2077118964 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178066 (secure 0 0 0) 0x7f0898053220 con 0x7f08b00c86f0 2023-08-09T13:50:44.598 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:44.597+0000 7f08b5ccc640 1 -- 172.21.15.43:0/2077118964 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "mgr module enable", "module": "snap_schedule"} v 0) v1 -- 0x7f08b00cec90 con 0x7f08b00c86f0 2023-08-09T13:50:45.142 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:45.140+0000 7f08acff9640 1 -- 172.21.15.43:0/2077118964 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mgrmap(e 30) v1 ==== 88460+0+0 (secure 0 0 0) 0x7f08980535f0 con 0x7f08b00c86f0 2023-08-09T13:50:45.145 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:45.143+0000 7f08acff9640 1 -- 172.21.15.43:0/2077118964 <== mon.0 v2:172.21.15.43:3300/0 8 ==== mon_command_ack([{"prefix": "mgr module enable", "module": "snap_schedule"}]=0 v30) v1 ==== 92+0+0 (secure 0 0 0) 0x7f0898053220 con 0x7f08b00c86f0 2023-08-09T13:50:45.149 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:45.147+0000 7f08b5ccc640 1 -- 172.21.15.43:0/2077118964 >> [v2:172.21.15.203:6832/3325141227,v1:172.21.15.203:6833/3325141227] conn(0x7f089006a170 msgr2=0x7f089006c630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:50:45.149 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:45.147+0000 7f08b5ccc640 1 --2- 172.21.15.43:0/2077118964 >> [v2:172.21.15.203:6832/3325141227,v1:172.21.15.203:6833/3325141227] conn(0x7f089006a170 0x7f089006c630 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f08a0006fd0 tx=0x7f08a0008040 comp rx=0 tx=0).stop 2023-08-09T13:50:45.149 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:45.147+0000 7f08b5ccc640 1 -- 172.21.15.43:0/2077118964 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f08b00c86f0 msgr2=0x7f08b00c78c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:50:45.150 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:45.147+0000 7f08b5ccc640 1 --2- 172.21.15.43:0/2077118964 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f08b00c86f0 0x7f08b00c78c0 secure :-1 s=READY pgs=1053 cs=0 l=1 rev1=1 crypto rx=0x7f089800c4a0 tx=0x7f089800c960 comp rx=0 tx=0).stop 2023-08-09T13:50:45.150 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:45.147+0000 7f08b5ccc640 1 -- 172.21.15.43:0/2077118964 shutdown_connections 2023-08-09T13:50:45.150 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:45.147+0000 7f08b5ccc640 1 --2- 172.21.15.43:0/2077118964 >> [v2:172.21.15.203:6832/3325141227,v1:172.21.15.203:6833/3325141227] conn(0x7f089006a170 0x7f089006c630 unknown :-1 s=CLOSED pgs=76 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:50:45.150 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:45.147+0000 7f08b5ccc640 1 --2- 172.21.15.43:0/2077118964 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f08b0143c20 0x7f08b013dac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:50:45.150 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:45.147+0000 7f08b5ccc640 1 --2- 172.21.15.43:0/2077118964 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f08b00c86f0 0x7f08b00c78c0 unknown :-1 s=CLOSED pgs=1053 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:50:45.150 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:45.147+0000 7f08b5ccc640 1 --2- 172.21.15.43:0/2077118964 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f08b006cfe0 0x7f08b00c68b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:50:45.151 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:45.147+0000 7f08b5ccc640 1 -- 172.21.15.43:0/2077118964 >> 172.21.15.43:0/2077118964 conn(0x7f08b00ce0a0 msgr2=0x7f08b006f440 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-09T13:50:45.151 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:45.148+0000 7f08b5ccc640 1 -- 172.21.15.43:0/2077118964 shutdown_connections 2023-08-09T13:50:45.151 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:45.148+0000 7f08b5ccc640 1 -- 172.21.15.43:0/2077118964 wait complete. 2023-08-09T13:50:45.170 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mgr mgr/snap_schedule/allow_m_granularity True 2023-08-09T13:50:45.361 INFO:tasks.ceph.mgr.x.smithi203.stderr:2023-08-09T13:50:45.358+0000 7fde51e10040 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2023-08-09T13:50:45.386 INFO:tasks.ceph.mgr.y.smithi043.stderr:2023-08-09T13:50:45.384+0000 7f986f630040 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2023-08-09T13:50:45.396 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:45.395+0000 7f2ad2e06640 1 Processor -- start 2023-08-09T13:50:45.397 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:45.395+0000 7f2ad2e06640 1 -- start start 2023-08-09T13:50:45.398 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:45.397+0000 7f2ad2e06640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f2acc15c550 0x7f2acc0cd6d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:50:45.398 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:45.397+0000 7f2ad2e06640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f2acc0b45f0 0x7f2acc0b59d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:50:45.399 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:45.397+0000 7f2ad2e06640 1 -- --> v1:172.21.15.203:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2acc0b7bd0 con 0x7f2acc0d04b0 2023-08-09T13:50:45.399 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:45.397+0000 7f2ad2e06640 1 -- --> v2:172.21.15.43:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2acc0d1c00 con 0x7f2acc0b45f0 2023-08-09T13:50:45.399 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:45.397+0000 7f2ad2e06640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2acc15c930 con 0x7f2acc15c550 2023-08-09T13:50:45.400 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:45.397+0000 7f2ad13a3640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f2acc0b45f0 0x7f2acc0b59d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:50:45.400 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:45.397+0000 7f2ad13a3640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f2acc0b45f0 0x7f2acc0b59d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:52004/0 (socket says 172.21.15.43:52004) 2023-08-09T13:50:45.400 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:45.397+0000 7f2ad13a3640 1 -- 172.21.15.43:0/1342929035 learned_addr learned my addr 172.21.15.43:0/1342929035 (peer_addr_for_me v2:172.21.15.43:0/0) 2023-08-09T13:50:45.400 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:45.397+0000 7f2ad0ba2640 1 --2- 172.21.15.43:0/1342929035 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f2acc15c550 0x7f2acc0cd6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:50:45.400 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:45.398+0000 7f2ad13a3640 1 -- 172.21.15.43:0/1342929035 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f2acc15c550 msgr2=0x7f2acc0cd6d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-09T13:50:45.401 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:45.398+0000 7f2ad13a3640 1 --2- 172.21.15.43:0/1342929035 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f2acc15c550 0x7f2acc0cd6d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:50:45.401 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:45.398+0000 7f2ad13a3640 1 -- 172.21.15.43:0/1342929035 >> v1:172.21.15.203:6789/0 conn(0x7f2acc0d04b0 legacy=0x7f2acc0b3ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:50:45.401 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:45.398+0000 7f2ad13a3640 1 -- 172.21.15.43:0/1342929035 --> v2:172.21.15.43:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2acc0b54f0 con 0x7f2acc0b45f0 2023-08-09T13:50:45.401 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:45.398+0000 7f2ad0ba2640 1 --2- 172.21.15.43:0/1342929035 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f2acc15c550 0x7f2acc0cd6d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-08-09T13:50:45.402 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:45.399+0000 7f2ad13a3640 1 --2- 172.21.15.43:0/1342929035 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f2acc0b45f0 0x7f2acc0b59d0 secure :-1 s=READY pgs=1058 cs=0 l=1 rev1=1 crypto rx=0x7f2ac4009e50 tx=0x7f2ac402bf60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6800cfebfaa39125 server_cookie=0 in_seq=0 out_seq=0 2023-08-09T13:50:45.402 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:45.399+0000 7f2acb7fe640 1 -- 172.21.15.43:0/1342929035 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2ac402cb00 con 0x7f2acc0b45f0 2023-08-09T13:50:45.402 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:45.399+0000 7f2acb7fe640 1 -- 172.21.15.43:0/1342929035 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2ac402cc60 con 0x7f2acc0b45f0 2023-08-09T13:50:45.403 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:45.400+0000 7f2acb7fe640 1 -- 172.21.15.43:0/1342929035 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2ac40343f0 con 0x7f2acc0b45f0 2023-08-09T13:50:45.403 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:45.400+0000 7f2ad2e06640 1 -- 172.21.15.43:0/1342929035 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f2acc0b45f0 msgr2=0x7f2acc0b59d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:50:45.403 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:45.400+0000 7f2ad2e06640 1 --2- 172.21.15.43:0/1342929035 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f2acc0b45f0 0x7f2acc0b59d0 secure :-1 s=READY pgs=1058 cs=0 l=1 rev1=1 crypto rx=0x7f2ac4009e50 tx=0x7f2ac402bf60 comp rx=0 tx=0).stop 2023-08-09T13:50:45.403 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:45.400+0000 7f2ad2e06640 1 -- 172.21.15.43:0/1342929035 shutdown_connections 2023-08-09T13:50:45.403 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:45.400+0000 7f2ad2e06640 1 --2- 172.21.15.43:0/1342929035 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f2acc0b45f0 0x7f2acc0b59d0 unknown :-1 s=CLOSED pgs=1058 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:50:45.403 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:45.400+0000 7f2ad2e06640 1 --2- 172.21.15.43:0/1342929035 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f2acc15c550 0x7f2acc0cd6d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:50:45.403 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:45.400+0000 7f2ad2e06640 1 -- 172.21.15.43:0/1342929035 >> 172.21.15.43:0/1342929035 conn(0x7f2acc071520 msgr2=0x7f2acc0ceee0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-09T13:50:45.404 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:45.400+0000 7f2ad2e06640 1 -- 172.21.15.43:0/1342929035 shutdown_connections 2023-08-09T13:50:45.404 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:45.401+0000 7f2ad2e06640 1 -- 172.21.15.43:0/1342929035 wait complete. 2023-08-09T13:50:45.405 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:45.402+0000 7f2ad2e06640 1 Processor -- start 2023-08-09T13:50:45.405 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:45.402+0000 7f2ad2e06640 1 -- start start 2023-08-09T13:50:45.405 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:45.403+0000 7f2ad2e06640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f2acc0b45f0 0x7f2acc1f1060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:50:45.405 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:45.403+0000 7f2ad2e06640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f2acc0d04b0 0x7f2acc0a8dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:50:45.406 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:45.403+0000 7f2ad2e06640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f2acc15c550 0x7f2acc0a9300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:50:45.406 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:45.403+0000 7f2ad2e06640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2acc0a9930 con 0x7f2acc0d04b0 2023-08-09T13:50:45.406 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:45.403+0000 7f2ad2e06640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2acc0a9c10 con 0x7f2acc0b45f0 2023-08-09T13:50:45.406 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:45.403+0000 7f2ad2e06640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2acc0a9ef0 con 0x7f2acc15c550 2023-08-09T13:50:45.407 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:45.404+0000 7f2acbfff640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f2acc0d04b0 0x7f2acc0a8dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:50:45.407 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:45.404+0000 7f2acbfff640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f2acc0d04b0 0x7f2acc0a8dc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:52016/0 (socket says 172.21.15.43:52016) 2023-08-09T13:50:45.407 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:45.404+0000 7f2acbfff640 1 -- 172.21.15.43:0/3050789868 learned_addr learned my addr 172.21.15.43:0/3050789868 (peer_addr_for_me v2:172.21.15.43:0/0) 2023-08-09T13:50:45.408 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:45.404+0000 7f2ad0ba2640 1 --2- 172.21.15.43:0/3050789868 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f2acc0b45f0 0x7f2acc1f1060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:50:45.408 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:45.404+0000 7f2ad13a3640 1 --2- 172.21.15.43:0/3050789868 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f2acc15c550 0x7f2acc0a9300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:50:45.408 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:45.404+0000 7f2acbfff640 1 -- 172.21.15.43:0/3050789868 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f2acc15c550 msgr2=0x7f2acc0a9300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:50:45.408 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:45.404+0000 7f2acbfff640 1 --2- 172.21.15.43:0/3050789868 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f2acc15c550 0x7f2acc0a9300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:50:45.408 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:45.404+0000 7f2acbfff640 1 -- 172.21.15.43:0/3050789868 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f2acc0b45f0 msgr2=0x7f2acc1f1060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:50:45.409 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:45.404+0000 7f2acbfff640 1 --2- 172.21.15.43:0/3050789868 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f2acc0b45f0 0x7f2acc1f1060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:50:45.409 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:45.404+0000 7f2acbfff640 1 -- 172.21.15.43:0/3050789868 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2ac4009b30 con 0x7f2acc0d04b0 2023-08-09T13:50:45.409 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:45.405+0000 7f2acbfff640 1 --2- 172.21.15.43:0/3050789868 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f2acc0d04b0 0x7f2acc0a8dc0 secure :-1 s=READY pgs=1059 cs=0 l=1 rev1=1 crypto rx=0x7f2abc00c450 tx=0x7f2abc00c910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-09T13:50:45.409 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:45.405+0000 7f2ac97fa640 1 -- 172.21.15.43:0/3050789868 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2abc0052d0 con 0x7f2acc0d04b0 2023-08-09T13:50:45.409 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:45.405+0000 7f2ac97fa640 1 -- 172.21.15.43:0/3050789868 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2abc005430 con 0x7f2acc0d04b0 2023-08-09T13:50:45.409 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:45.405+0000 7f2ac97fa640 1 -- 172.21.15.43:0/3050789868 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2abc012870 con 0x7f2acc0d04b0 2023-08-09T13:50:45.409 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:45.405+0000 7f2ad2e06640 1 -- 172.21.15.43:0/3050789868 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2acc0aa1d0 con 0x7f2acc0d04b0 2023-08-09T13:50:45.410 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:45.406+0000 7f2ad2e06640 1 -- 172.21.15.43:0/3050789868 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2acc1f15a0 con 0x7f2acc0d04b0 2023-08-09T13:50:45.410 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:45.407+0000 7f2ad2e06640 1 -- 172.21.15.43:0/3050789868 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2acc0aa3f0 con 0x7f2acc0d04b0 2023-08-09T13:50:45.411 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:45.408+0000 7f2ac97fa640 1 -- 172.21.15.43:0/3050789868 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 30) v1 ==== 88460+0+0 (secure 0 0 0) 0x7f2abc0129d0 con 0x7f2acc0d04b0 2023-08-09T13:50:45.411 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:45.409+0000 7f2ac97fa640 1 --2- 172.21.15.43:0/3050789868 >> [v2:172.21.15.203:6832/3325141227,v1:172.21.15.203:6833/3325141227] conn(0x7f2ab806a1c0 0x7f2ab806c680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:50:45.411 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:45.409+0000 7f2ac97fa640 1 -- 172.21.15.43:0/3050789868 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(90..90 src has 1..90) v4 ==== 6426+0+0 (secure 0 0 0) 0x7f2abc0883a0 con 0x7f2acc0d04b0 2023-08-09T13:50:45.411 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:45.409+0000 7f2ad0ba2640 1 -- 172.21.15.43:0/3050789868 >> [v2:172.21.15.203:6832/3325141227,v1:172.21.15.203:6833/3325141227] conn(0x7f2ab806a1c0 msgr2=0x7f2ab806c680 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.203:6832/3325141227 2023-08-09T13:50:45.411 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:45.409+0000 7f2ad0ba2640 1 --2- 172.21.15.43:0/3050789868 >> [v2:172.21.15.203:6832/3325141227,v1:172.21.15.203:6833/3325141227] conn(0x7f2ab806a1c0 0x7f2ab806c680 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-08-09T13:50:45.414 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:45.413+0000 7f2ac97fa640 1 -- 172.21.15.43:0/3050789868 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178066 (secure 0 0 0) 0x7f2abc053800 con 0x7f2acc0d04b0 2023-08-09T13:50:45.608 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:45.606+0000 7f2ad2e06640 1 -- 172.21.15.43:0/3050789868 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command([{prefix=config set, name=mgr/snap_schedule/allow_m_granularity}] v 0) v1 -- 0x7f2acc0739f0 con 0x7f2acc0d04b0 2023-08-09T13:50:45.611 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:45.610+0000 7f2ad0ba2640 1 -- 172.21.15.43:0/3050789868 >> [v2:172.21.15.203:6832/3325141227,v1:172.21.15.203:6833/3325141227] conn(0x7f2ab806a1c0 msgr2=0x7f2ab806c680 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.203:6832/3325141227 2023-08-09T13:50:45.611 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:45.610+0000 7f2ad0ba2640 1 --2- 172.21.15.43:0/3050789868 >> [v2:172.21.15.203:6832/3325141227,v1:172.21.15.203:6833/3325141227] conn(0x7f2ab806a1c0 0x7f2ab806c680 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2023-08-09T13:50:45.613 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:45.612+0000 7f2ac97fa640 1 -- 172.21.15.43:0/3050789868 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mgr/snap_schedule/allow_m_granularity}]=0 v23)=0 v23) v1 ==== 138+0+0 (secure 0 0 0) 0x7f2abc056e20 con 0x7f2acc0d04b0 2023-08-09T13:50:45.616 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:45.614+0000 7f2ad2e06640 1 -- 172.21.15.43:0/3050789868 >> [v2:172.21.15.203:6832/3325141227,v1:172.21.15.203:6833/3325141227] conn(0x7f2ab806a1c0 msgr2=0x7f2ab806c680 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-08-09T13:50:45.617 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:45.614+0000 7f2ad2e06640 1 --2- 172.21.15.43:0/3050789868 >> [v2:172.21.15.203:6832/3325141227,v1:172.21.15.203:6833/3325141227] conn(0x7f2ab806a1c0 0x7f2ab806c680 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:50:45.617 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:45.615+0000 7f2ad2e06640 1 -- 172.21.15.43:0/3050789868 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f2acc0d04b0 msgr2=0x7f2acc0a8dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:50:45.617 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:45.615+0000 7f2ad2e06640 1 --2- 172.21.15.43:0/3050789868 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f2acc0d04b0 0x7f2acc0a8dc0 secure :-1 s=READY pgs=1059 cs=0 l=1 rev1=1 crypto rx=0x7f2abc00c450 tx=0x7f2abc00c910 comp rx=0 tx=0).stop 2023-08-09T13:50:45.617 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:45.615+0000 7f2ad2e06640 1 -- 172.21.15.43:0/3050789868 shutdown_connections 2023-08-09T13:50:45.617 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:45.615+0000 7f2ad2e06640 1 --2- 172.21.15.43:0/3050789868 >> [v2:172.21.15.203:6832/3325141227,v1:172.21.15.203:6833/3325141227] conn(0x7f2ab806a1c0 0x7f2ab806c680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:50:45.618 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:45.615+0000 7f2ad2e06640 1 --2- 172.21.15.43:0/3050789868 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f2acc15c550 0x7f2acc0a9300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:50:45.618 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:45.615+0000 7f2ad2e06640 1 --2- 172.21.15.43:0/3050789868 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f2acc0d04b0 0x7f2acc0a8dc0 unknown :-1 s=CLOSED pgs=1059 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:50:45.618 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:45.615+0000 7f2ad2e06640 1 --2- 172.21.15.43:0/3050789868 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f2acc0b45f0 0x7f2acc1f1060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:50:45.618 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:45.615+0000 7f2ad2e06640 1 -- 172.21.15.43:0/3050789868 >> 172.21.15.43:0/3050789868 conn(0x7f2acc071520 msgr2=0x7f2acc15ee20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-09T13:50:45.618 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:45.615+0000 7f2ad2e06640 1 -- 172.21.15.43:0/3050789868 shutdown_connections 2023-08-09T13:50:45.618 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:45.615+0000 7f2ad2e06640 1 -- 172.21.15.43:0/3050789868 wait complete. 2023-08-09T13:50:45.635 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mgr mgr/snap_schedule/dump_on_update True 2023-08-09T13:50:45.847 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:45.845+0000 7f9dcb65e640 1 Processor -- start 2023-08-09T13:50:45.847 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:45.845+0000 7f9dcb65e640 1 -- start start 2023-08-09T13:50:45.849 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:45.847+0000 7f9dcb65e640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f9dc415c6f0 0x7f9dc40ac630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:50:45.849 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:45.847+0000 7f9dcb65e640 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7f9dc40acb70 0x7f9dc40b23a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:50:45.849 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:45.847+0000 7f9dcb65e640 1 -- --> v1:172.21.15.203:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9dc40b1ef0 con 0x7f9dc415c310 2023-08-09T13:50:45.849 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:45.847+0000 7f9dcb65e640 1 -- --> v2:172.21.15.203:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9dc40ad000 con 0x7f9dc40acb70 2023-08-09T13:50:45.849 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:45.847+0000 7f9dcb65e640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9dc406c8a0 con 0x7f9dc415c6f0 2023-08-09T13:50:45.850 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:45.848+0000 7f9dc9bfb640 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7f9dc40acb70 0x7f9dc40b23a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:50:45.850 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:45.848+0000 7f9dc8bf9640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f9dc415c6f0 0x7f9dc40ac630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:50:45.851 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:45.848+0000 7f9dc9bfb640 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7f9dc40acb70 0x7f9dc40b23a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.203:3300/0 says I am v2:172.21.15.43:54686/0 (socket says 172.21.15.43:54686) 2023-08-09T13:50:45.851 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:45.848+0000 7f9dc9bfb640 1 -- 172.21.15.43:0/1584156113 learned_addr learned my addr 172.21.15.43:0/1584156113 (peer_addr_for_me v2:172.21.15.43:0/0) 2023-08-09T13:50:45.851 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:45.849+0000 7f9dc9bfb640 1 -- 172.21.15.43:0/1584156113 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f9dc415c6f0 msgr2=0x7f9dc40ac630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-09T13:50:45.851 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:45.849+0000 7f9dc9bfb640 1 --2- 172.21.15.43:0/1584156113 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f9dc415c6f0 0x7f9dc40ac630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:50:45.851 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:45.849+0000 7f9dc9bfb640 1 -- 172.21.15.43:0/1584156113 >> v1:172.21.15.203:6789/0 conn(0x7f9dc415c310 legacy=0x7f9dc406c190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:50:45.851 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:45.849+0000 7f9dc9bfb640 1 -- 172.21.15.43:0/1584156113 --> v2:172.21.15.203:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9dc406cb70 con 0x7f9dc40acb70 2023-08-09T13:50:45.853 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:45.850+0000 7f9dc9bfb640 1 --2- 172.21.15.43:0/1584156113 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f9dc40acb70 0x7f9dc40b23a0 secure :-1 s=READY pgs=381 cs=0 l=1 rev1=1 crypto rx=0x7f9dbc009870 tx=0x7f9dbc02b980 comp rx=0 tx=0).ready entity=mon.1 client_cookie=ca7dac1357651855 server_cookie=0 in_seq=0 out_seq=0 2023-08-09T13:50:45.853 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:45.850+0000 7f9dbbfff640 1 -- 172.21.15.43:0/1584156113 <== mon.1 v2:172.21.15.203:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9dbc030840 con 0x7f9dc40acb70 2023-08-09T13:50:45.853 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:45.850+0000 7f9dbbfff640 1 -- 172.21.15.43:0/1584156113 <== mon.1 v2:172.21.15.203:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9dbc0309a0 con 0x7f9dc40acb70 2023-08-09T13:50:45.854 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:45.850+0000 7f9dbbfff640 1 -- 172.21.15.43:0/1584156113 <== mon.1 v2:172.21.15.203:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9dbc030d30 con 0x7f9dc40acb70 2023-08-09T13:50:45.854 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:45.851+0000 7f9dcb65e640 1 -- 172.21.15.43:0/1584156113 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f9dc40acb70 msgr2=0x7f9dc40b23a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:50:45.854 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:45.851+0000 7f9dcb65e640 1 --2- 172.21.15.43:0/1584156113 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f9dc40acb70 0x7f9dc40b23a0 secure :-1 s=READY pgs=381 cs=0 l=1 rev1=1 crypto rx=0x7f9dbc009870 tx=0x7f9dbc02b980 comp rx=0 tx=0).stop 2023-08-09T13:50:45.854 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:45.851+0000 7f9dcb65e640 1 -- 172.21.15.43:0/1584156113 shutdown_connections 2023-08-09T13:50:45.854 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:45.851+0000 7f9dcb65e640 1 --2- 172.21.15.43:0/1584156113 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f9dc40acb70 0x7f9dc40b23a0 unknown :-1 s=CLOSED pgs=381 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:50:45.854 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:45.851+0000 7f9dcb65e640 1 --2- 172.21.15.43:0/1584156113 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f9dc415c6f0 0x7f9dc40ac630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:50:45.854 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:45.851+0000 7f9dcb65e640 1 -- 172.21.15.43:0/1584156113 >> 172.21.15.43:0/1584156113 conn(0x7f9dc40697c0 msgr2=0x7f9dc4069ba0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-09T13:50:45.854 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:45.851+0000 7f9dcb65e640 1 -- 172.21.15.43:0/1584156113 shutdown_connections 2023-08-09T13:50:45.855 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:45.851+0000 7f9dcb65e640 1 -- 172.21.15.43:0/1584156113 wait complete. 2023-08-09T13:50:45.855 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:45.853+0000 7f9dcb65e640 1 Processor -- start 2023-08-09T13:50:45.855 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:45.853+0000 7f9dcb65e640 1 -- start start 2023-08-09T13:50:45.856 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:45.854+0000 7f9dcb65e640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f9dc40acb70 0x7f9dc41f14c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:50:45.856 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:45.854+0000 7f9dcb65e640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f9dc415c310 0x7f9dc41561b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:50:45.857 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:45.854+0000 7f9dcb65e640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f9dc415c6f0 0x7f9dc41566f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:50:45.857 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:45.854+0000 7f9dcb65e640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9dc4154cf0 con 0x7f9dc40acb70 2023-08-09T13:50:45.857 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:45.854+0000 7f9dcb65e640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9dc40b0f10 con 0x7f9dc415c310 2023-08-09T13:50:45.857 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:45.854+0000 7f9dcb65e640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9dc4154f00 con 0x7f9dc415c6f0 2023-08-09T13:50:45.857 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:45.855+0000 7f9dc93fa640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f9dc40acb70 0x7f9dc41f14c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:50:45.857 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:45.855+0000 7f9dc93fa640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f9dc40acb70 0x7f9dc41f14c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:52024/0 (socket says 172.21.15.43:52024) 2023-08-09T13:50:45.858 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:45.855+0000 7f9dc93fa640 1 -- 172.21.15.43:0/1319721579 learned_addr learned my addr 172.21.15.43:0/1319721579 (peer_addr_for_me v2:172.21.15.43:0/0) 2023-08-09T13:50:45.858 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:45.855+0000 7f9dc8bf9640 1 --2- 172.21.15.43:0/1319721579 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f9dc415c310 0x7f9dc41561b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:50:45.858 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:45.855+0000 7f9dc9bfb640 1 --2- 172.21.15.43:0/1319721579 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f9dc415c6f0 0x7f9dc41566f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:50:45.859 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:45.855+0000 7f9dc93fa640 1 -- 172.21.15.43:0/1319721579 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f9dc415c6f0 msgr2=0x7f9dc41566f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:50:45.859 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:45.855+0000 7f9dc93fa640 1 --2- 172.21.15.43:0/1319721579 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f9dc415c6f0 0x7f9dc41566f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:50:45.859 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:45.855+0000 7f9dc93fa640 1 -- 172.21.15.43:0/1319721579 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f9dc415c310 msgr2=0x7f9dc41561b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:50:45.859 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:45.855+0000 7f9dc93fa640 1 --2- 172.21.15.43:0/1319721579 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f9dc415c310 0x7f9dc41561b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:50:45.859 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:45.855+0000 7f9dc93fa640 1 -- 172.21.15.43:0/1319721579 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9dbc0094d0 con 0x7f9dc40acb70 2023-08-09T13:50:45.860 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:45.856+0000 7f9dc93fa640 1 --2- 172.21.15.43:0/1319721579 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f9dc40acb70 0x7f9dc41f14c0 secure :-1 s=READY pgs=1060 cs=0 l=1 rev1=1 crypto rx=0x7f9dc000c870 tx=0x7f9dc000cd30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-09T13:50:45.860 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:45.856+0000 7f9db9ffb640 1 -- 172.21.15.43:0/1319721579 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9dc0003b70 con 0x7f9dc40acb70 2023-08-09T13:50:45.860 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:45.856+0000 7f9db9ffb640 1 -- 172.21.15.43:0/1319721579 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9dc0003cf0 con 0x7f9dc40acb70 2023-08-09T13:50:45.860 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:45.856+0000 7f9db9ffb640 1 -- 172.21.15.43:0/1319721579 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9dc000aa90 con 0x7f9dc40acb70 2023-08-09T13:50:45.860 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:45.856+0000 7f9dcb65e640 1 -- 172.21.15.43:0/1319721579 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9dc41551e0 con 0x7f9dc40acb70 2023-08-09T13:50:45.860 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:45.856+0000 7f9dcb65e640 1 -- 172.21.15.43:0/1319721579 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9dc4155ef0 con 0x7f9dc40acb70 2023-08-09T13:50:45.860 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:45.857+0000 7f9dcb65e640 1 -- 172.21.15.43:0/1319721579 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9dc406a7d0 con 0x7f9dc40acb70 2023-08-09T13:50:45.861 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:45.858+0000 7f9db9ffb640 1 -- 172.21.15.43:0/1319721579 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 30) v1 ==== 88460+0+0 (secure 0 0 0) 0x7f9dc000abf0 con 0x7f9dc40acb70 2023-08-09T13:50:45.861 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:45.859+0000 7f9db9ffb640 1 --2- 172.21.15.43:0/1319721579 >> [v2:172.21.15.203:6832/3325141227,v1:172.21.15.203:6833/3325141227] conn(0x7f9da806a1c0 0x7f9da806c680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:50:45.861 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:45.859+0000 7f9db9ffb640 1 -- 172.21.15.43:0/1319721579 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(90..90 src has 1..90) v4 ==== 6426+0+0 (secure 0 0 0) 0x7f9dc0088850 con 0x7f9dc40acb70 2023-08-09T13:50:45.862 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:45.860+0000 7f9dc8bf9640 1 -- 172.21.15.43:0/1319721579 >> [v2:172.21.15.203:6832/3325141227,v1:172.21.15.203:6833/3325141227] conn(0x7f9da806a1c0 msgr2=0x7f9da806c680 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.203:6832/3325141227 2023-08-09T13:50:45.862 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:45.860+0000 7f9dc8bf9640 1 --2- 172.21.15.43:0/1319721579 >> [v2:172.21.15.203:6832/3325141227,v1:172.21.15.203:6833/3325141227] conn(0x7f9da806a1c0 0x7f9da806c680 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-08-09T13:50:45.865 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:45.864+0000 7f9db9ffb640 1 -- 172.21.15.43:0/1319721579 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178066 (secure 0 0 0) 0x7f9dc0053c20 con 0x7f9dc40acb70 2023-08-09T13:50:46.059 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:46.057+0000 7f9dcb65e640 1 -- 172.21.15.43:0/1319721579 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command([{prefix=config set, name=mgr/snap_schedule/dump_on_update}] v 0) v1 -- 0x7f9dc40739f0 con 0x7f9dc40acb70 2023-08-09T13:50:46.062 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:46.061+0000 7f9dc8bf9640 1 -- 172.21.15.43:0/1319721579 >> [v2:172.21.15.203:6832/3325141227,v1:172.21.15.203:6833/3325141227] conn(0x7f9da806a1c0 msgr2=0x7f9da806c680 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.203:6832/3325141227 2023-08-09T13:50:46.063 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:46.061+0000 7f9dc8bf9640 1 --2- 172.21.15.43:0/1319721579 >> [v2:172.21.15.203:6832/3325141227,v1:172.21.15.203:6833/3325141227] conn(0x7f9da806a1c0 0x7f9da806c680 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2023-08-09T13:50:46.064 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:46.063+0000 7f9db9ffb640 1 -- 172.21.15.43:0/1319721579 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mgr/snap_schedule/dump_on_update}]=0 v24)=0 v24) v1 ==== 133+0+0 (secure 0 0 0) 0x7f9dc0003e60 con 0x7f9dc40acb70 2023-08-09T13:50:46.067 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:46.066+0000 7f9dcb65e640 1 -- 172.21.15.43:0/1319721579 >> [v2:172.21.15.203:6832/3325141227,v1:172.21.15.203:6833/3325141227] conn(0x7f9da806a1c0 msgr2=0x7f9da806c680 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-08-09T13:50:46.068 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:46.066+0000 7f9dcb65e640 1 --2- 172.21.15.43:0/1319721579 >> [v2:172.21.15.203:6832/3325141227,v1:172.21.15.203:6833/3325141227] conn(0x7f9da806a1c0 0x7f9da806c680 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:50:46.068 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:46.066+0000 7f9dcb65e640 1 -- 172.21.15.43:0/1319721579 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f9dc40acb70 msgr2=0x7f9dc41f14c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:50:46.068 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:46.066+0000 7f9dcb65e640 1 --2- 172.21.15.43:0/1319721579 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f9dc40acb70 0x7f9dc41f14c0 secure :-1 s=READY pgs=1060 cs=0 l=1 rev1=1 crypto rx=0x7f9dc000c870 tx=0x7f9dc000cd30 comp rx=0 tx=0).stop 2023-08-09T13:50:46.068 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:46.066+0000 7f9dcb65e640 1 -- 172.21.15.43:0/1319721579 shutdown_connections 2023-08-09T13:50:46.068 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:46.066+0000 7f9dcb65e640 1 --2- 172.21.15.43:0/1319721579 >> [v2:172.21.15.203:6832/3325141227,v1:172.21.15.203:6833/3325141227] conn(0x7f9da806a1c0 0x7f9da806c680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:50:46.068 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:46.066+0000 7f9dcb65e640 1 --2- 172.21.15.43:0/1319721579 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f9dc415c6f0 0x7f9dc41566f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:50:46.069 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:46.066+0000 7f9dcb65e640 1 --2- 172.21.15.43:0/1319721579 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f9dc415c310 0x7f9dc41561b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:50:46.069 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:46.066+0000 7f9dcb65e640 1 --2- 172.21.15.43:0/1319721579 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f9dc40acb70 0x7f9dc41f14c0 unknown :-1 s=CLOSED pgs=1060 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:50:46.069 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:46.066+0000 7f9dcb65e640 1 -- 172.21.15.43:0/1319721579 >> 172.21.15.43:0/1319721579 conn(0x7f9dc40697c0 msgr2=0x7f9dc4157390 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-09T13:50:46.069 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:46.066+0000 7f9dcb65e640 1 -- 172.21.15.43:0/1319721579 shutdown_connections 2023-08-09T13:50:46.069 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:46.066+0000 7f9dcb65e640 1 -- 172.21.15.43:0/1319721579 wait complete. 2023-08-09T13:50:46.069 INFO:tasks.ceph.mgr.x.smithi203.stderr:2023-08-09T13:50:46.065+0000 7fde51e10040 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2023-08-09T13:50:46.087 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-09T13:50:46.088 DEBUG:teuthology.orchestra.run.smithi043:> (cd /home/ubuntu/cephtest/mnt.0 && exec mkdir -p snap_test_dir1) 2023-08-09T13:50:46.129 INFO:tasks.ceph.mgr.y.smithi043.stderr:2023-08-09T13:50:46.127+0000 7f986f630040 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2023-08-09T13:50:46.153 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs snap-schedule list --fs cephfs snap_test_dir1 2023-08-09T13:50:46.191 INFO:tasks.ceph.mgr.x.smithi203.stderr:2023-08-09T13:50:46.189+0000 7fde51e10040 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2023-08-09T13:50:46.248 INFO:tasks.ceph.mgr.x.smithi203.stderr:2023-08-09T13:50:46.245+0000 7fde51e10040 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2023-08-09T13:50:46.257 INFO:tasks.ceph.mgr.y.smithi043.stderr:2023-08-09T13:50:46.256+0000 7f986f630040 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2023-08-09T13:50:46.298 INFO:tasks.ceph.mgr.x.smithi203.stderr:2023-08-09T13:50:46.296+0000 7fde51e10040 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2023-08-09T13:50:46.318 INFO:tasks.ceph.mgr.y.smithi043.stderr:2023-08-09T13:50:46.316+0000 7f986f630040 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2023-08-09T13:50:46.370 INFO:tasks.ceph.mgr.y.smithi043.stderr:2023-08-09T13:50:46.369+0000 7f986f630040 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2023-08-09T13:50:46.419 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:46.417+0000 7f1aeca4d640 1 Processor -- start 2023-08-09T13:50:46.420 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:46.418+0000 7f1aeca4d640 1 -- start start 2023-08-09T13:50:46.421 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:46.419+0000 7f1aeca4d640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f1ae815fc70 0x7f1ae80b10c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:50:46.421 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:46.419+0000 7f1aeca4d640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f1ae80d3020 0x7f1ae80af0a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:50:46.421 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:46.419+0000 7f1aeca4d640 1 -- --> v1:172.21.15.43:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1ae80b0c30 con 0x7f1ae80d00b0 2023-08-09T13:50:46.421 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:46.419+0000 7f1aeca4d640 1 -- --> v2:172.21.15.43:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1ae80b0300 con 0x7f1ae80d3020 2023-08-09T13:50:46.422 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:46.419+0000 7f1aeca4d640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1ae80b05e0 con 0x7f1ae815fc70 2023-08-09T13:50:46.422 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:46.420+0000 7f1ae5d9b640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f1ae80d3020 0x7f1ae80af0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:50:46.422 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:46.420+0000 7f1ae6d9d640 1 --1- >> v1:172.21.15.43:6789/0 conn(0x7f1ae80d00b0 0x7f1ae80af5e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.43:6789/0 says I am v1:172.21.15.43:59872/0 (socket says 172.21.15.43:59872) 2023-08-09T13:50:46.422 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:46.420+0000 7f1ae6d9d640 1 -- 172.21.15.43:0/2705538054 learned_addr learned my addr 172.21.15.43:0/2705538054 (peer_addr_for_me v1:172.21.15.43:0/0) 2023-08-09T13:50:46.423 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:46.420+0000 7f1ae659c640 1 --2- 172.21.15.43:0/2705538054 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f1ae815fc70 0x7f1ae80b10c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:50:46.423 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:46.420+0000 7f1ae5d9b640 1 -- 172.21.15.43:0/2705538054 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f1ae815fc70 msgr2=0x7f1ae80b10c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-09T13:50:46.423 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:46.420+0000 7f1ae5d9b640 1 --2- 172.21.15.43:0/2705538054 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f1ae815fc70 0x7f1ae80b10c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:50:46.423 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:46.420+0000 7f1ae5d9b640 1 -- 172.21.15.43:0/2705538054 >> v1:172.21.15.43:6789/0 conn(0x7f1ae80d00b0 legacy=0x7f1ae80af5e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:50:46.423 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:46.420+0000 7f1ae5d9b640 1 -- 172.21.15.43:0/2705538054 --> v2:172.21.15.43:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1ae80b08c0 con 0x7f1ae80d3020 2023-08-09T13:50:46.423 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:46.420+0000 7f1ae559a640 1 -- 172.21.15.43:0/2705538054 <== mon.0 v1:172.21.15.43:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2958436429 0 0) 0x7f1ae80b0c30 con 0x7f1ae80d00b0 2023-08-09T13:50:46.424 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:46.422+0000 7f1ae5d9b640 1 --2- 172.21.15.43:0/2705538054 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f1ae80d3020 0x7f1ae80af0a0 secure :-1 s=READY pgs=1062 cs=0 l=1 rev1=1 crypto rx=0x7f1adc0097a0 tx=0x7f1adc02b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ca7d5d69af0d5d30 server_cookie=0 in_seq=0 out_seq=0 2023-08-09T13:50:46.425 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:46.422+0000 7f1ae559a640 1 -- 172.21.15.43:0/2705538054 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1adc031830 con 0x7f1ae80d3020 2023-08-09T13:50:46.425 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:46.422+0000 7f1ae559a640 1 -- 172.21.15.43:0/2705538054 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1adc031990 con 0x7f1ae80d3020 2023-08-09T13:50:46.425 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:46.422+0000 7f1ae559a640 1 -- 172.21.15.43:0/2705538054 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1adc031d20 con 0x7f1ae80d3020 2023-08-09T13:50:46.425 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:46.423+0000 7f1aeca4d640 1 -- 172.21.15.43:0/2705538054 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f1ae80d3020 msgr2=0x7f1ae80af0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:50:46.425 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:46.423+0000 7f1aeca4d640 1 --2- 172.21.15.43:0/2705538054 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f1ae80d3020 0x7f1ae80af0a0 secure :-1 s=READY pgs=1062 cs=0 l=1 rev1=1 crypto rx=0x7f1adc0097a0 tx=0x7f1adc02b8d0 comp rx=0 tx=0).stop 2023-08-09T13:50:46.425 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:46.423+0000 7f1aeca4d640 1 -- 172.21.15.43:0/2705538054 shutdown_connections 2023-08-09T13:50:46.425 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:46.423+0000 7f1aeca4d640 1 --2- 172.21.15.43:0/2705538054 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f1ae80d3020 0x7f1ae80af0a0 unknown :-1 s=CLOSED pgs=1062 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:50:46.426 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:46.423+0000 7f1aeca4d640 1 --2- 172.21.15.43:0/2705538054 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f1ae815fc70 0x7f1ae80b10c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:50:46.426 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:46.423+0000 7f1aeca4d640 1 -- 172.21.15.43:0/2705538054 >> 172.21.15.43:0/2705538054 conn(0x7f1ae809b7b0 msgr2=0x7f1ae80d2470 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-09T13:50:46.426 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:46.423+0000 7f1aeca4d640 1 -- 172.21.15.43:0/2705538054 shutdown_connections 2023-08-09T13:50:46.426 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:46.423+0000 7f1aeca4d640 1 -- 172.21.15.43:0/2705538054 wait complete. 2023-08-09T13:50:46.426 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:46.425+0000 7f1aeca4d640 1 Processor -- start 2023-08-09T13:50:46.427 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:46.425+0000 7f1aeca4d640 1 -- start start 2023-08-09T13:50:46.428 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:46.426+0000 7f1aeca4d640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f1ae80d00b0 0x7f1ae81f1b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:50:46.428 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:46.426+0000 7f1aeca4d640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f1ae815fc70 0x7f1ae80b7940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:50:46.428 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:46.426+0000 7f1aeca4d640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f1ae80cf390 0x7f1ae80cf820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:50:46.428 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:46.426+0000 7f1aeca4d640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1ae80b7520 con 0x7f1ae80cf390 2023-08-09T13:50:46.429 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:46.426+0000 7f1aeca4d640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1ae80cfd60 con 0x7f1ae815fc70 2023-08-09T13:50:46.429 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:46.426+0000 7f1aeca4d640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1ae81f2f70 con 0x7f1ae80d00b0 2023-08-09T13:50:46.429 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:46.426+0000 7f1ae6d9d640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f1ae80cf390 0x7f1ae80cf820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:50:46.429 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:46.426+0000 7f1ae6d9d640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f1ae80cf390 0x7f1ae80cf820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:52032/0 (socket says 172.21.15.43:52032) 2023-08-09T13:50:46.429 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:46.426+0000 7f1ae6d9d640 1 -- 172.21.15.43:0/3108568456 learned_addr learned my addr 172.21.15.43:0/3108568456 (peer_addr_for_me v2:172.21.15.43:0/0) 2023-08-09T13:50:46.429 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:46.426+0000 7f1ae5d9b640 1 --2- 172.21.15.43:0/3108568456 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f1ae815fc70 0x7f1ae80b7940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:50:46.429 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:46.426+0000 7f1ae659c640 1 --2- 172.21.15.43:0/3108568456 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f1ae80d00b0 0x7f1ae81f1b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:50:46.430 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:46.427+0000 7f1ae6d9d640 1 -- 172.21.15.43:0/3108568456 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f1ae80d00b0 msgr2=0x7f1ae81f1b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:50:46.430 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:46.427+0000 7f1ae6d9d640 1 --2- 172.21.15.43:0/3108568456 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f1ae80d00b0 0x7f1ae81f1b60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:50:46.430 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:46.427+0000 7f1ae6d9d640 1 -- 172.21.15.43:0/3108568456 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f1ae815fc70 msgr2=0x7f1ae80b7940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:50:46.430 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:46.427+0000 7f1ae6d9d640 1 --2- 172.21.15.43:0/3108568456 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f1ae815fc70 0x7f1ae80b7940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:50:46.430 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:46.427+0000 7f1ae6d9d640 1 -- 172.21.15.43:0/3108568456 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1adc009400 con 0x7f1ae80cf390 2023-08-09T13:50:46.431 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:46.427+0000 7f1ae6d9d640 1 --2- 172.21.15.43:0/3108568456 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f1ae80cf390 0x7f1ae80cf820 secure :-1 s=READY pgs=1063 cs=0 l=1 rev1=1 crypto rx=0x7f1ae000c780 tx=0x7f1ae000cc40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-09T13:50:46.431 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:46.427+0000 7f1ad6ffd640 1 -- 172.21.15.43:0/3108568456 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1ae000ce90 con 0x7f1ae80cf390 2023-08-09T13:50:46.431 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:46.427+0000 7f1ad6ffd640 1 -- 172.21.15.43:0/3108568456 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1ae000a640 con 0x7f1ae80cf390 2023-08-09T13:50:46.431 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:46.427+0000 7f1ad6ffd640 1 -- 172.21.15.43:0/3108568456 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1ae000aa30 con 0x7f1ae80cf390 2023-08-09T13:50:46.432 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:46.427+0000 7f1aeca4d640 1 -- 172.21.15.43:0/3108568456 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1ae80b6e60 con 0x7f1ae80cf390 2023-08-09T13:50:46.432 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:46.427+0000 7f1aeca4d640 1 -- 172.21.15.43:0/3108568456 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1ae80b6b10 con 0x7f1ae80cf390 2023-08-09T13:50:46.432 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:46.428+0000 7f1ad4ff9640 1 -- 172.21.15.43:0/3108568456 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1ae80b7660 con 0x7f1ae80cf390 2023-08-09T13:50:46.432 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:46.430+0000 7f1ad6ffd640 1 -- 172.21.15.43:0/3108568456 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 30) v1 ==== 88460+0+0 (secure 0 0 0) 0x7f1ae000ab90 con 0x7f1ae80cf390 2023-08-09T13:50:46.433 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:46.430+0000 7f1ad6ffd640 1 --2- 172.21.15.43:0/3108568456 >> [v2:172.21.15.203:6832/3325141227,v1:172.21.15.203:6833/3325141227] conn(0x7f1ac806a1f0 0x7f1ac806c6b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:50:46.433 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:46.431+0000 7f1ad6ffd640 1 -- 172.21.15.43:0/3108568456 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(90..90 src has 1..90) v4 ==== 6426+0+0 (secure 0 0 0) 0x7f1ae0087d70 con 0x7f1ae80cf390 2023-08-09T13:50:46.433 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:46.431+0000 7f1ae659c640 1 -- 172.21.15.43:0/3108568456 >> [v2:172.21.15.203:6832/3325141227,v1:172.21.15.203:6833/3325141227] conn(0x7f1ac806a1f0 msgr2=0x7f1ac806c6b0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.203:6832/3325141227 2023-08-09T13:50:46.433 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:46.431+0000 7f1ae659c640 1 --2- 172.21.15.43:0/3108568456 >> [v2:172.21.15.203:6832/3325141227,v1:172.21.15.203:6833/3325141227] conn(0x7f1ac806a1f0 0x7f1ac806c6b0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-08-09T13:50:46.436 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:46.435+0000 7f1ad6ffd640 1 -- 172.21.15.43:0/3108568456 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178066 (secure 0 0 0) 0x7f1ae80b7660 con 0x7f1ae80cf390 2023-08-09T13:50:46.633 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:46.632+0000 7f1ae659c640 1 -- 172.21.15.43:0/3108568456 >> [v2:172.21.15.203:6832/3325141227,v1:172.21.15.203:6833/3325141227] conn(0x7f1ac806a1f0 msgr2=0x7f1ac806c6b0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.203:6832/3325141227 2023-08-09T13:50:46.634 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:46.632+0000 7f1ae659c640 1 --2- 172.21.15.43:0/3108568456 >> [v2:172.21.15.203:6832/3325141227,v1:172.21.15.203:6833/3325141227] conn(0x7f1ac806a1f0 0x7f1ac806c6b0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2023-08-09T13:50:46.648 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:46.646+0000 7f1ad4ff9640 1 -- 172.21.15.43:0/3108568456 --> [v2:172.21.15.203:6832/3325141227,v1:172.21.15.203:6833/3325141227] -- mgr_command(tid 0: {"prefix": "fs snap-schedule list", "fs": "cephfs", "path": "snap_test_dir1", "target": ["mon-mgr", ""]}) v1 -- 0x7f1ae8160050 con 0x7f1ac806a1f0 2023-08-09T13:50:46.999 INFO:tasks.ceph.mgr.x.smithi203.stderr:2023-08-09T13:50:46.996+0000 7fde51e10040 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2023-08-09T13:50:47.034 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:47.033+0000 7f1ae659c640 1 -- 172.21.15.43:0/3108568456 >> [v2:172.21.15.203:6832/3325141227,v1:172.21.15.203:6833/3325141227] conn(0x7f1ac806a1f0 msgr2=0x7f1ac806c6b0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.203:6832/3325141227 2023-08-09T13:50:47.035 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:47.033+0000 7f1ae659c640 1 --2- 172.21.15.43:0/3108568456 >> [v2:172.21.15.203:6832/3325141227,v1:172.21.15.203:6833/3325141227] conn(0x7f1ac806a1f0 0x7f1ac806c6b0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2023-08-09T13:50:47.059 INFO:tasks.ceph.mgr.x.smithi203.stderr:2023-08-09T13:50:47.056+0000 7fde51e10040 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2023-08-09T13:50:47.109 INFO:tasks.ceph.mgr.x.smithi203.stderr:2023-08-09T13:50:47.106+0000 7fde51e10040 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2023-08-09T13:50:47.124 INFO:tasks.ceph.mgr.y.smithi043.stderr:2023-08-09T13:50:47.122+0000 7f986f630040 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2023-08-09T13:50:47.188 INFO:tasks.ceph.mgr.y.smithi043.stderr:2023-08-09T13:50:47.186+0000 7f986f630040 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2023-08-09T13:50:47.226 INFO:tasks.ceph.mgr.x.smithi203.stderr:2023-08-09T13:50:47.224+0000 7fde51e10040 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2023-08-09T13:50:47.239 INFO:tasks.ceph.mgr.y.smithi043.stderr:2023-08-09T13:50:47.238+0000 7f986f630040 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2023-08-09T13:50:47.362 INFO:tasks.ceph.mgr.y.smithi043.stderr:2023-08-09T13:50:47.360+0000 7f986f630040 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2023-08-09T13:50:47.385 INFO:tasks.ceph.mgr.x.smithi203.stderr:2023-08-09T13:50:47.380+0000 7fde51e10040 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2023-08-09T13:50:47.489 INFO:tasks.ceph.mgr.x.smithi203.stderr:2023-08-09T13:50:47.486+0000 7fde51e10040 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2023-08-09T13:50:47.525 INFO:tasks.ceph.mgr.y.smithi043.stderr:2023-08-09T13:50:47.523+0000 7f986f630040 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2023-08-09T13:50:47.537 INFO:tasks.ceph.mgr.x.smithi203.stderr:2023-08-09T13:50:47.535+0000 7fde51e10040 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2023-08-09T13:50:47.637 INFO:tasks.ceph.mgr.y.smithi043.stderr:2023-08-09T13:50:47.636+0000 7f986f630040 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2023-08-09T13:50:47.688 INFO:tasks.ceph.mgr.y.smithi043.stderr:2023-08-09T13:50:47.687+0000 7f986f630040 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2023-08-09T13:50:47.696 INFO:tasks.ceph.mgr.x.smithi203.stderr:2023-08-09T13:50:47.694+0000 7fde51e10040 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2023-08-09T13:50:47.835 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:47.834+0000 7f1ae659c640 1 -- 172.21.15.43:0/3108568456 >> [v2:172.21.15.203:6832/3325141227,v1:172.21.15.203:6833/3325141227] conn(0x7f1ac806a1f0 msgr2=0x7f1ac806c6b0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.203:6832/3325141227 2023-08-09T13:50:47.836 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:47.834+0000 7f1ae659c640 1 --2- 172.21.15.43:0/3108568456 >> [v2:172.21.15.203:6832/3325141227,v1:172.21.15.203:6833/3325141227] conn(0x7f1ac806a1f0 0x7f1ac806c6b0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2023-08-09T13:50:47.856 INFO:tasks.ceph.mgr.y.smithi043.stderr:2023-08-09T13:50:47.855+0000 7f986f630040 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2023-08-09T13:50:48.075 INFO:tasks.ceph.mgr.x.smithi203.stderr:2023-08-09T13:50:48.073+0000 7fde51e10040 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2023-08-09T13:50:48.256 INFO:tasks.ceph.mgr.y.smithi043.stderr:2023-08-09T13:50:48.253+0000 7f986f630040 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2023-08-09T13:50:48.296 INFO:tasks.ceph.mgr.x.smithi203.stderr:2023-08-09T13:50:48.293+0000 7fde51e10040 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2023-08-09T13:50:48.353 INFO:tasks.ceph.mgr.x.smithi203.stderr:2023-08-09T13:50:48.350+0000 7fde51e10040 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2023-08-09T13:50:48.402 INFO:tasks.ceph.mgr.x.smithi203.stderr:2023-08-09T13:50:48.399+0000 7fde51e10040 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2023-08-09T13:50:48.454 INFO:tasks.ceph.mgr.x.smithi203.stderr:2023-08-09T13:50:48.452+0000 7fde51e10040 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2023-08-09T13:50:48.485 INFO:tasks.ceph.mgr.y.smithi043.stderr:2023-08-09T13:50:48.482+0000 7f986f630040 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2023-08-09T13:50:48.544 INFO:tasks.ceph.mgr.y.smithi043.stderr:2023-08-09T13:50:48.542+0000 7f986f630040 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2023-08-09T13:50:48.597 INFO:tasks.ceph.mgr.y.smithi043.stderr:2023-08-09T13:50:48.594+0000 7f986f630040 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2023-08-09T13:50:48.651 INFO:tasks.ceph.mgr.y.smithi043.stderr:2023-08-09T13:50:48.648+0000 7f986f630040 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2023-08-09T13:50:48.747 INFO:tasks.ceph.mgr.x.smithi203.stderr:2023-08-09T13:50:48.745+0000 7fde51e10040 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2023-08-09T13:50:48.804 INFO:tasks.ceph.mgr.x.smithi203.stderr:2023-08-09T13:50:48.802+0000 7fde51e10040 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2023-08-09T13:50:48.856 INFO:tasks.ceph.mgr.x.smithi203.stderr:2023-08-09T13:50:48.853+0000 7fde51e10040 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2023-08-09T13:50:48.957 INFO:tasks.ceph.mgr.y.smithi043.stderr:2023-08-09T13:50:48.955+0000 7f986f630040 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2023-08-09T13:50:49.014 INFO:tasks.ceph.mgr.x.smithi203.stderr:2023-08-09T13:50:49.012+0000 7fde51e10040 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2023-08-09T13:50:49.018 INFO:tasks.ceph.mgr.y.smithi043.stderr:2023-08-09T13:50:49.015+0000 7f986f630040 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2023-08-09T13:50:49.063 INFO:tasks.ceph.mgr.x.smithi203.stderr:2023-08-09T13:50:49.061+0000 7fde51e10040 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2023-08-09T13:50:49.072 INFO:tasks.ceph.mgr.y.smithi043.stderr:2023-08-09T13:50:49.069+0000 7f986f630040 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2023-08-09T13:50:49.115 INFO:tasks.ceph.mgr.x.smithi203.stderr:2023-08-09T13:50:49.112+0000 7fde51e10040 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2023-08-09T13:50:49.165 INFO:tasks.ceph.mgr.x.smithi203.stderr:2023-08-09T13:50:49.162+0000 7fde51e10040 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2023-08-09T13:50:49.191 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:49.189+0000 7f1ad6ffd640 1 -- 172.21.15.43:0/3108568456 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mgrmap(e 31) v1 ==== 88069+0+0 (secure 0 0 0) 0x7f1ae0053510 con 0x7f1ae80cf390 2023-08-09T13:50:49.192 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:49.190+0000 7f1ad6ffd640 1 -- 172.21.15.43:0/3108568456 >> [v2:172.21.15.203:6832/3325141227,v1:172.21.15.203:6833/3325141227] conn(0x7f1ac806a1f0 msgr2=0x7f1ac806c6b0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-08-09T13:50:49.192 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:49.190+0000 7f1ad6ffd640 1 --2- 172.21.15.43:0/3108568456 >> [v2:172.21.15.203:6832/3325141227,v1:172.21.15.203:6833/3325141227] conn(0x7f1ac806a1f0 0x7f1ac806c6b0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:50:49.246 INFO:tasks.ceph.mgr.y.smithi043.stderr:2023-08-09T13:50:49.244+0000 7f986f630040 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2023-08-09T13:50:49.299 INFO:tasks.ceph.mgr.y.smithi043.stderr:2023-08-09T13:50:49.297+0000 7f986f630040 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2023-08-09T13:50:49.335 INFO:tasks.ceph.mgr.x.smithi203.stderr:2023-08-09T13:50:49.333+0000 7fddce68c640 -1 client.0 error registering admin socket command: (17) File exists 2023-08-09T13:50:49.336 INFO:tasks.ceph.mgr.x.smithi203.stderr:2023-08-09T13:50:49.333+0000 7fddce68c640 -1 client.0 error registering admin socket command: (17) File exists 2023-08-09T13:50:49.336 INFO:tasks.ceph.mgr.x.smithi203.stderr:2023-08-09T13:50:49.333+0000 7fddce68c640 -1 client.0 error registering admin socket command: (17) File exists 2023-08-09T13:50:49.336 INFO:tasks.ceph.mgr.x.smithi203.stderr:2023-08-09T13:50:49.333+0000 7fddce68c640 -1 client.0 error registering admin socket command: (17) File exists 2023-08-09T13:50:49.336 INFO:tasks.ceph.mgr.x.smithi203.stderr:2023-08-09T13:50:49.333+0000 7fddce68c640 -1 client.0 error registering admin socket command: (17) File exists 2023-08-09T13:50:49.337 INFO:tasks.ceph.mgr.x.smithi203.stderr:2023-08-09T13:50:49.334+0000 7fddd1692640 -1 client.0 error registering admin socket command: (17) File exists 2023-08-09T13:50:49.337 INFO:tasks.ceph.mgr.x.smithi203.stderr:2023-08-09T13:50:49.334+0000 7fddd1692640 -1 client.0 error registering admin socket command: (17) File exists 2023-08-09T13:50:49.337 INFO:tasks.ceph.mgr.x.smithi203.stderr:2023-08-09T13:50:49.334+0000 7fddd1692640 -1 client.0 error registering admin socket command: (17) File exists 2023-08-09T13:50:49.337 INFO:tasks.ceph.mgr.x.smithi203.stderr:2023-08-09T13:50:49.334+0000 7fddd1692640 -1 client.0 error registering admin socket command: (17) File exists 2023-08-09T13:50:49.337 INFO:tasks.ceph.mgr.x.smithi203.stderr:2023-08-09T13:50:49.334+0000 7fddd1692640 -1 client.0 error registering admin socket command: (17) File exists 2023-08-09T13:50:49.358 INFO:tasks.ceph.mgr.y.smithi043.stderr:2023-08-09T13:50:49.355+0000 7f986f630040 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2023-08-09T13:50:49.410 INFO:tasks.ceph.mgr.y.smithi043.stderr:2023-08-09T13:50:49.407+0000 7f986f630040 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2023-08-09T13:50:50.204 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:50.201+0000 7f1ad6ffd640 1 -- 172.21.15.43:0/3108568456 <== mon.0 v2:172.21.15.43:3300/0 8 ==== mgrmap(e 32) v1 ==== 88196+0+0 (secure 0 0 0) 0x7f1ae0085f20 con 0x7f1ae80cf390 2023-08-09T13:50:50.205 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:50.202+0000 7f1ad6ffd640 1 --2- 172.21.15.43:0/3108568456 >> [v2:172.21.15.203:6832/3851880723,v1:172.21.15.203:6833/3851880723] conn(0x7f1ac8072ce0 0x7f1ac80750d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:50:50.205 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:50.202+0000 7f1ad6ffd640 1 -- 172.21.15.43:0/3108568456 --> [v2:172.21.15.203:6832/3851880723,v1:172.21.15.203:6833/3851880723] -- mgr_command(tid 0: {"prefix": "fs snap-schedule list", "fs": "cephfs", "path": "snap_test_dir1", "target": ["mon-mgr", ""]}) v1 -- 0x7f1ae000ce90 con 0x7f1ac8072ce0 2023-08-09T13:50:50.210 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:50.207+0000 7f1ae659c640 1 --2- 172.21.15.43:0/3108568456 >> [v2:172.21.15.203:6832/3851880723,v1:172.21.15.203:6833/3851880723] conn(0x7f1ac8072ce0 0x7f1ac80750d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:50:50.221 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:50.218+0000 7f1ae659c640 1 --2- 172.21.15.43:0/3108568456 >> [v2:172.21.15.203:6832/3851880723,v1:172.21.15.203:6833/3851880723] conn(0x7f1ac8072ce0 0x7f1ac80750d0 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7f1ad8007040 tx=0x7f1ad8008040 comp rx=0 tx=0).ready entity=mgr.5750 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-09T13:50:50.242 INFO:tasks.ceph.mgr.x.smithi203.stderr:2023-08-09T13:50:50.239+0000 7fddd86a0640 -1 mgr.server reply reply (2) No such file or directory SnapSchedule for snap_test_dir1 not found 2023-08-09T13:50:50.243 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:50.239+0000 7f1ad6ffd640 1 -- 172.21.15.43:0/3108568456 <== mgr.5750 v2:172.21.15.203:6832/3851880723 1 ==== mgr_command_reply(tid 0: -2 SnapSchedule for snap_test_dir1 not found) v1 ==== 49+0+0 (secure 0 0 0) 0x7f1ae000ce90 con 0x7f1ac8072ce0 2023-08-09T13:50:50.243 INFO:teuthology.orchestra.run.smithi043.stderr:Error ENOENT: SnapSchedule for snap_test_dir1 not found 2023-08-09T13:50:50.261 DEBUG:teuthology.orchestra.run:got remote process result: 2 2023-08-09T13:50:50.261 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-09T13:50:50.262 DEBUG:teuthology.orchestra.run.smithi043:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1) 2023-08-09T13:50:50.331 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph mgr module disable snap_schedule 2023-08-09T13:50:50.562 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:50.559+0000 7ff79a98b640 1 Processor -- start 2023-08-09T13:50:50.562 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:50.559+0000 7ff79a98b640 1 -- start start 2023-08-09T13:50:50.563 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:50.561+0000 7ff79a98b640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7ff794142ca0 0x7ff7940c96e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:50:50.564 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:50.561+0000 7ff79a98b640 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7ff79415ce40 0x7ff7940cce00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:50:50.564 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:50.561+0000 7ff79a98b640 1 -- --> v1:172.21.15.203:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff794143080 con 0x7ff7940cd340 2023-08-09T13:50:50.564 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:50.561+0000 7ff79a98b640 1 -- --> v2:172.21.15.203:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff7940cd7b0 con 0x7ff79415ce40 2023-08-09T13:50:50.564 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:50.561+0000 7ff79a98b640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff79406d400 con 0x7ff794142ca0 2023-08-09T13:50:50.565 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:50.562+0000 7ff7937fe640 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7ff79415ce40 0x7ff7940cce00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:50:50.565 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:50.562+0000 7ff793fff640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7ff794142ca0 0x7ff7940c96e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:50:50.565 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:50.562+0000 7ff7937fe640 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7ff79415ce40 0x7ff7940cce00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.203:3300/0 says I am v2:172.21.15.43:54710/0 (socket says 172.21.15.43:54710) 2023-08-09T13:50:50.565 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:50.562+0000 7ff793fff640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7ff794142ca0 0x7ff7940c96e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.203:3301/0 says I am v2:172.21.15.43:40690/0 (socket says 172.21.15.43:40690) 2023-08-09T13:50:50.566 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:50.562+0000 7ff7937fe640 1 -- 172.21.15.43:0/4155457221 learned_addr learned my addr 172.21.15.43:0/4155457221 (peer_addr_for_me v2:172.21.15.43:0/0) 2023-08-09T13:50:50.566 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:50.562+0000 7ff798f28640 1 --1- 172.21.15.43:0/4155457221 >> v1:172.21.15.203:6789/0 conn(0x7ff7940cd340 0x7ff7940cbc70 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.203:6789/0 says I am v1:172.21.15.43:60476/0 (socket says 172.21.15.43:60476) 2023-08-09T13:50:50.566 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:50.563+0000 7ff793fff640 1 -- 172.21.15.43:0/4155457221 >> v2:172.21.15.203:3300/0 conn(0x7ff79415ce40 msgr2=0x7ff7940cce00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-09T13:50:50.566 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:50.563+0000 7ff793fff640 1 --2- 172.21.15.43:0/4155457221 >> v2:172.21.15.203:3300/0 conn(0x7ff79415ce40 0x7ff7940cce00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:50:50.566 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:50.563+0000 7ff793fff640 1 -- 172.21.15.43:0/4155457221 >> v1:172.21.15.203:6789/0 conn(0x7ff7940cd340 legacy=0x7ff7940cbc70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:50:50.566 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:50.563+0000 7ff793fff640 1 -- 172.21.15.43:0/4155457221 --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff7940c9d90 con 0x7ff794142ca0 2023-08-09T13:50:50.566 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:50.563+0000 7ff7937fe640 1 --2- 172.21.15.43:0/4155457221 >> v2:172.21.15.203:3300/0 conn(0x7ff79415ce40 0x7ff7940cce00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-08-09T13:50:50.567 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:50.564+0000 7ff793fff640 1 --2- 172.21.15.43:0/4155457221 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7ff794142ca0 0x7ff7940c96e0 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7ff784009d90 tx=0x7ff78402bea0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f87a9dd21237e5b7 server_cookie=0 in_seq=0 out_seq=0 2023-08-09T13:50:50.567 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:50.564+0000 7ff792ffd640 1 -- 172.21.15.43:0/4155457221 <== mon.2 v2:172.21.15.203:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff78402cc50 con 0x7ff794142ca0 2023-08-09T13:50:50.567 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:50.565+0000 7ff792ffd640 1 -- 172.21.15.43:0/4155457221 <== mon.2 v2:172.21.15.203:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff78402cdb0 con 0x7ff794142ca0 2023-08-09T13:50:50.568 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:50.565+0000 7ff792ffd640 1 -- 172.21.15.43:0/4155457221 <== mon.2 v2:172.21.15.203:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff784034660 con 0x7ff794142ca0 2023-08-09T13:50:50.568 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:50.565+0000 7ff79a98b640 1 -- 172.21.15.43:0/4155457221 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7ff794142ca0 msgr2=0x7ff7940c96e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:50:50.568 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:50.565+0000 7ff79a98b640 1 --2- 172.21.15.43:0/4155457221 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7ff794142ca0 0x7ff7940c96e0 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7ff784009d90 tx=0x7ff78402bea0 comp rx=0 tx=0).stop 2023-08-09T13:50:50.569 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:50.566+0000 7ff79a98b640 1 -- 172.21.15.43:0/4155457221 shutdown_connections 2023-08-09T13:50:50.569 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:50.566+0000 7ff79a98b640 1 --2- 172.21.15.43:0/4155457221 >> v2:172.21.15.203:3300/0 conn(0x7ff79415ce40 0x7ff7940cce00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:50:50.569 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:50.566+0000 7ff79a98b640 1 --2- 172.21.15.43:0/4155457221 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7ff794142ca0 0x7ff7940c96e0 unknown :-1 s=CLOSED pgs=147 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:50:50.569 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:50.566+0000 7ff79a98b640 1 -- 172.21.15.43:0/4155457221 >> 172.21.15.43:0/4155457221 conn(0x7ff7940ce0a0 msgr2=0x7ff7940ce480 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-09T13:50:50.569 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:50.566+0000 7ff79a98b640 1 -- 172.21.15.43:0/4155457221 shutdown_connections 2023-08-09T13:50:50.569 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:50.566+0000 7ff79a98b640 1 -- 172.21.15.43:0/4155457221 wait complete. 2023-08-09T13:50:50.571 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:50.568+0000 7ff79a98b640 1 Processor -- start 2023-08-09T13:50:50.571 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:50.569+0000 7ff79a98b640 1 -- start start 2023-08-09T13:50:50.573 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:50.570+0000 7ff79a98b640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7ff7940cd340 0x7ff794158c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:50:50.573 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:50.570+0000 7ff79a98b640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7ff794142ca0 0x7ff794137dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:50:50.573 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:50.570+0000 7ff79a98b640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff79415ce40 0x7ff7940ca870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:50:50.573 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:50.570+0000 7ff79a98b640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff79415abf0 con 0x7ff79415ce40 2023-08-09T13:50:50.573 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:50.570+0000 7ff79a98b640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff7941591b0 con 0x7ff7940cd340 2023-08-09T13:50:50.574 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:50.570+0000 7ff79a98b640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff79415a910 con 0x7ff794142ca0 2023-08-09T13:50:50.574 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:50.571+0000 7ff798f28640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff79415ce40 0x7ff7940ca870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:50:50.575 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:50.571+0000 7ff798f28640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff79415ce40 0x7ff7940ca870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:52038/0 (socket says 172.21.15.43:52038) 2023-08-09T13:50:50.575 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:50.571+0000 7ff798f28640 1 -- 172.21.15.43:0/2736148710 learned_addr learned my addr 172.21.15.43:0/2736148710 (peer_addr_for_me v2:172.21.15.43:0/0) 2023-08-09T13:50:50.575 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:50.571+0000 7ff793fff640 1 --2- 172.21.15.43:0/2736148710 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7ff7940cd340 0x7ff794158c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:50:50.575 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:50.571+0000 7ff7937fe640 1 --2- 172.21.15.43:0/2736148710 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7ff794142ca0 0x7ff794137dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:50:50.575 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:50.571+0000 7ff798f28640 1 -- 172.21.15.43:0/2736148710 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7ff794142ca0 msgr2=0x7ff794137dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:50:50.576 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:50.571+0000 7ff798f28640 1 --2- 172.21.15.43:0/2736148710 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7ff794142ca0 0x7ff794137dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:50:50.576 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:50.571+0000 7ff798f28640 1 -- 172.21.15.43:0/2736148710 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7ff7940cd340 msgr2=0x7ff794158c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:50:50.576 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:50.571+0000 7ff798f28640 1 --2- 172.21.15.43:0/2736148710 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7ff7940cd340 0x7ff794158c70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:50:50.576 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:50.571+0000 7ff798f28640 1 -- 172.21.15.43:0/2736148710 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff77c0094d0 con 0x7ff79415ce40 2023-08-09T13:50:50.576 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:50.571+0000 7ff793fff640 1 --2- 172.21.15.43:0/2736148710 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7ff7940cd340 0x7ff794158c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-08-09T13:50:50.576 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:50.572+0000 7ff798f28640 1 --2- 172.21.15.43:0/2736148710 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff79415ce40 0x7ff7940ca870 secure :-1 s=READY pgs=1064 cs=0 l=1 rev1=1 crypto rx=0x7ff78c00c3c0 tx=0x7ff78c00c880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-09T13:50:50.576 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:50.572+0000 7ff790ff9640 1 -- 172.21.15.43:0/2736148710 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff78c0074e0 con 0x7ff79415ce40 2023-08-09T13:50:50.577 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:50.572+0000 7ff790ff9640 1 -- 172.21.15.43:0/2736148710 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff78c007640 con 0x7ff79415ce40 2023-08-09T13:50:50.577 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:50.572+0000 7ff790ff9640 1 -- 172.21.15.43:0/2736148710 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff78c004a00 con 0x7ff79415ce40 2023-08-09T13:50:50.577 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:50.572+0000 7ff79a98b640 1 -- 172.21.15.43:0/2736148710 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff784009a70 con 0x7ff79415ce40 2023-08-09T13:50:50.577 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:50.572+0000 7ff79a98b640 1 -- 172.21.15.43:0/2736148710 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff7940c55f0 con 0x7ff79415ce40 2023-08-09T13:50:50.578 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:50.573+0000 7ff79a98b640 1 -- 172.21.15.43:0/2736148710 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff7940c5170 con 0x7ff79415ce40 2023-08-09T13:50:50.578 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:50.574+0000 7ff790ff9640 1 -- 172.21.15.43:0/2736148710 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 32) v1 ==== 88196+0+0 (secure 0 0 0) 0x7ff78c004b60 con 0x7ff79415ce40 2023-08-09T13:50:50.578 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:50.575+0000 7ff790ff9640 1 --2- 172.21.15.43:0/2736148710 >> [v2:172.21.15.203:6832/3851880723,v1:172.21.15.203:6833/3851880723] conn(0x7ff774069ef0 0x7ff77406c3b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:50:50.578 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:50.575+0000 7ff790ff9640 1 -- 172.21.15.43:0/2736148710 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(91..91 src has 1..91) v4 ==== 6770+0+0 (secure 0 0 0) 0x7ff78c087de0 con 0x7ff79415ce40 2023-08-09T13:50:50.579 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:50.576+0000 7ff793fff640 1 --2- 172.21.15.43:0/2736148710 >> [v2:172.21.15.203:6832/3851880723,v1:172.21.15.203:6833/3851880723] conn(0x7ff774069ef0 0x7ff77406c3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:50:50.579 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:50.577+0000 7ff793fff640 1 --2- 172.21.15.43:0/2736148710 >> [v2:172.21.15.203:6832/3851880723,v1:172.21.15.203:6833/3851880723] conn(0x7ff774069ef0 0x7ff77406c3b0 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7ff784036000 tx=0x7ff784030000 comp rx=0 tx=0).ready entity=mgr.5750 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-09T13:50:50.582 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:50.580+0000 7ff790ff9640 1 -- 172.21.15.43:0/2736148710 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178066 (secure 0 0 0) 0x7ff7940c5170 con 0x7ff79415ce40 2023-08-09T13:50:50.831 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:50.829+0000 7ff79a98b640 1 -- 172.21.15.43:0/2736148710 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "mgr module disable", "module": "snap_schedule"} v 0) v1 -- 0x7ff7940c5170 con 0x7ff79415ce40 2023-08-09T13:50:51.245 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:51.242+0000 7ff790ff9640 1 -- 172.21.15.43:0/2736148710 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mgrmap(e 33) v1 ==== 88444+0+0 (secure 0 0 0) 0x7ff78c087030 con 0x7ff79415ce40 2023-08-09T13:50:51.248 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:51.246+0000 7ff790ff9640 1 -- 172.21.15.43:0/2736148710 <== mon.0 v2:172.21.15.43:3300/0 8 ==== mon_command_ack([{"prefix": "mgr module disable", "module": "snap_schedule"}]=0 v33) v1 ==== 93+0+0 (secure 0 0 0) 0x7ff78c056f00 con 0x7ff79415ce40 2023-08-09T13:50:51.251 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:51.249+0000 7ff79a98b640 1 -- 172.21.15.43:0/2736148710 >> [v2:172.21.15.203:6832/3851880723,v1:172.21.15.203:6833/3851880723] conn(0x7ff774069ef0 msgr2=0x7ff77406c3b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:50:51.252 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:51.249+0000 7ff79a98b640 1 --2- 172.21.15.43:0/2736148710 >> [v2:172.21.15.203:6832/3851880723,v1:172.21.15.203:6833/3851880723] conn(0x7ff774069ef0 0x7ff77406c3b0 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7ff784036000 tx=0x7ff784030000 comp rx=0 tx=0).stop 2023-08-09T13:50:51.252 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:51.249+0000 7ff79a98b640 1 -- 172.21.15.43:0/2736148710 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff79415ce40 msgr2=0x7ff7940ca870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:50:51.252 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:51.249+0000 7ff79a98b640 1 --2- 172.21.15.43:0/2736148710 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff79415ce40 0x7ff7940ca870 secure :-1 s=READY pgs=1064 cs=0 l=1 rev1=1 crypto rx=0x7ff78c00c3c0 tx=0x7ff78c00c880 comp rx=0 tx=0).stop 2023-08-09T13:50:51.252 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:51.249+0000 7ff79a98b640 1 -- 172.21.15.43:0/2736148710 shutdown_connections 2023-08-09T13:50:51.253 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:51.249+0000 7ff79a98b640 1 --2- 172.21.15.43:0/2736148710 >> [v2:172.21.15.203:6832/3851880723,v1:172.21.15.203:6833/3851880723] conn(0x7ff774069ef0 0x7ff77406c3b0 unknown :-1 s=CLOSED pgs=23 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:50:51.253 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:51.249+0000 7ff79a98b640 1 --2- 172.21.15.43:0/2736148710 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff79415ce40 0x7ff7940ca870 unknown :-1 s=CLOSED pgs=1064 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:50:51.253 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:51.249+0000 7ff79a98b640 1 --2- 172.21.15.43:0/2736148710 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7ff794142ca0 0x7ff794137dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:50:51.253 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:51.249+0000 7ff79a98b640 1 --2- 172.21.15.43:0/2736148710 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7ff7940cd340 0x7ff794158c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:50:51.253 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:51.249+0000 7ff79a98b640 1 -- 172.21.15.43:0/2736148710 >> 172.21.15.43:0/2736148710 conn(0x7ff7940ce0a0 msgr2=0x7ff7940c7380 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-09T13:50:51.253 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:51.249+0000 7ff79a98b640 1 -- 172.21.15.43:0/2736148710 shutdown_connections 2023-08-09T13:50:51.253 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:51.250+0000 7ff79a98b640 1 -- 172.21.15.43:0/2736148710 wait complete. 2023-08-09T13:50:51.276 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2023-08-09T13:50:51.276 DEBUG:teuthology.orchestra.run.smithi043:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2023-08-09T13:50:51.387 DEBUG:teuthology.orchestra.run.smithi203:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2023-08-09T13:50:51.464 INFO:tasks.ceph.mgr.x.smithi203.stderr:2023-08-09T13:50:51.461+0000 7f4b1542c040 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2023-08-09T13:50:51.478 INFO:tasks.ceph.mgr.y.smithi043.stderr:2023-08-09T13:50:51.475+0000 7fe84c03a040 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2023-08-09T13:50:51.489 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2023-08-09T13:50:51.489 DEBUG:teuthology.orchestra.run.smithi043:> set -ex 2023-08-09T13:50:51.490 DEBUG:teuthology.orchestra.run.smithi043:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-09T13:50:51.548 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-08-09T13:50:51.764 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:51.761+0000 7f13a820d640 1 Processor -- start 2023-08-09T13:50:51.764 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:51.761+0000 7f13a820d640 1 -- start start 2023-08-09T13:50:51.767 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:51.763+0000 7f13a820d640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f13a00697c0 0x7f13a015cb20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:50:51.767 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:51.763+0000 7f13a820d640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f13a00ccdd0 0x7f13a00cd260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:50:51.767 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:51.763+0000 7f13a820d640 1 -- --> v1:172.21.15.43:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f13a015d060 con 0x7f13a006ed20 2023-08-09T13:50:51.767 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:51.763+0000 7f13a820d640 1 -- --> v2:172.21.15.43:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f13a015db30 con 0x7f13a00697c0 2023-08-09T13:50:51.768 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:51.763+0000 7f13a820d640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f13a00cd7a0 con 0x7f13a00ccdd0 2023-08-09T13:50:51.768 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:51.763+0000 7f13a57a8640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f13a00697c0 0x7f13a015cb20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:50:51.768 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:51.763+0000 7f13a5fa9640 1 --1- >> v1:172.21.15.43:6789/0 conn(0x7f13a006ed20 0x7f13a006dbc0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.43:6789/0 says I am v1:172.21.15.43:59878/0 (socket says 172.21.15.43:59878) 2023-08-09T13:50:51.768 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:51.763+0000 7f13a5fa9640 1 -- 172.21.15.43:0/3052647666 learned_addr learned my addr 172.21.15.43:0/3052647666 (peer_addr_for_me v1:172.21.15.43:0/0) 2023-08-09T13:50:51.768 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:51.764+0000 7f13a67aa640 1 --2- 172.21.15.43:0/3052647666 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f13a00ccdd0 0x7f13a00cd260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:50:51.769 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:51.764+0000 7f13a57a8640 1 -- 172.21.15.43:0/3052647666 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f13a00ccdd0 msgr2=0x7f13a00cd260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-09T13:50:51.769 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:51.764+0000 7f13a57a8640 1 --2- 172.21.15.43:0/3052647666 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f13a00ccdd0 0x7f13a00cd260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:50:51.769 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:51.764+0000 7f13a57a8640 1 -- 172.21.15.43:0/3052647666 >> v1:172.21.15.43:6789/0 conn(0x7f13a006ed20 legacy=0x7f13a006dbc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:50:51.769 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:51.764+0000 7f13a4fa7640 1 -- 172.21.15.43:0/3052647666 <== mon.0 v1:172.21.15.43:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1462084324 0 0) 0x7f13a015d060 con 0x7f13a006ed20 2023-08-09T13:50:51.769 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:51.764+0000 7f13a57a8640 1 -- 172.21.15.43:0/3052647666 --> v2:172.21.15.43:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f13a0070910 con 0x7f13a00697c0 2023-08-09T13:50:51.770 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:51.765+0000 7f13a57a8640 1 --2- 172.21.15.43:0/3052647666 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f13a00697c0 0x7f13a015cb20 secure :-1 s=READY pgs=1068 cs=0 l=1 rev1=1 crypto rx=0x7f13900097a0 tx=0x7f139002b840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7a5511aae4a36228 server_cookie=0 in_seq=0 out_seq=0 2023-08-09T13:50:51.770 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:51.765+0000 7f13a4fa7640 1 -- 172.21.15.43:0/3052647666 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f139002e070 con 0x7f13a00697c0 2023-08-09T13:50:51.771 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:51.765+0000 7f13a4fa7640 1 -- 172.21.15.43:0/3052647666 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f13900315a0 con 0x7f13a00697c0 2023-08-09T13:50:51.771 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:51.765+0000 7f13a4fa7640 1 -- 172.21.15.43:0/3052647666 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1390031930 con 0x7f13a00697c0 2023-08-09T13:50:51.771 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:51.766+0000 7f13a820d640 1 -- 172.21.15.43:0/3052647666 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f13a00697c0 msgr2=0x7f13a015cb20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:50:51.771 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:51.766+0000 7f13a820d640 1 --2- 172.21.15.43:0/3052647666 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f13a00697c0 0x7f13a015cb20 secure :-1 s=READY pgs=1068 cs=0 l=1 rev1=1 crypto rx=0x7f13900097a0 tx=0x7f139002b840 comp rx=0 tx=0).stop 2023-08-09T13:50:51.771 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:51.766+0000 7f13a820d640 1 -- 172.21.15.43:0/3052647666 shutdown_connections 2023-08-09T13:50:51.771 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:51.766+0000 7f13a820d640 1 --2- 172.21.15.43:0/3052647666 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f13a00ccdd0 0x7f13a00cd260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:50:51.771 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:51.766+0000 7f13a820d640 1 --2- 172.21.15.43:0/3052647666 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f13a00697c0 0x7f13a015cb20 unknown :-1 s=CLOSED pgs=1068 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:50:51.772 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:51.766+0000 7f13a820d640 1 -- 172.21.15.43:0/3052647666 >> 172.21.15.43:0/3052647666 conn(0x7f13a00cbc80 msgr2=0x7f13a00cc090 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-09T13:50:51.772 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:51.766+0000 7f13a820d640 1 -- 172.21.15.43:0/3052647666 shutdown_connections 2023-08-09T13:50:51.772 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:51.766+0000 7f13a820d640 1 -- 172.21.15.43:0/3052647666 wait complete. 2023-08-09T13:50:51.772 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:51.768+0000 7f13a820d640 1 Processor -- start 2023-08-09T13:50:51.773 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:51.768+0000 7f13a820d640 1 -- start start 2023-08-09T13:50:51.774 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:51.769+0000 7f13a820d640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f13a00697c0 0x7f13a01596b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:50:51.774 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:51.769+0000 7f13a820d640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f13a006ed20 0x7f13a00b4710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:50:51.774 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:51.769+0000 7f13a820d640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f13a00ccdd0 0x7f13a0155e00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:50:51.774 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:51.769+0000 7f13a820d640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f13a00ac920 con 0x7f13a006ed20 2023-08-09T13:50:51.774 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:51.769+0000 7f13a820d640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f13a00acc00 con 0x7f13a00697c0 2023-08-09T13:50:51.774 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:51.769+0000 7f13a820d640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f13a0158220 con 0x7f13a00ccdd0 2023-08-09T13:50:51.775 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:51.770+0000 7f13a57a8640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f13a006ed20 0x7f13a00b4710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:50:51.775 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:51.770+0000 7f13a57a8640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f13a006ed20 0x7f13a00b4710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:52076/0 (socket says 172.21.15.43:52076) 2023-08-09T13:50:51.775 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:51.770+0000 7f13a57a8640 1 -- 172.21.15.43:0/1941728160 learned_addr learned my addr 172.21.15.43:0/1941728160 (peer_addr_for_me v2:172.21.15.43:0/0) 2023-08-09T13:50:51.775 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:51.770+0000 7f13a5fa9640 1 --2- 172.21.15.43:0/1941728160 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f13a00697c0 0x7f13a01596b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:50:51.775 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:51.770+0000 7f13a67aa640 1 --2- 172.21.15.43:0/1941728160 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f13a00ccdd0 0x7f13a0155e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:50:51.775 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:51.770+0000 7f13a57a8640 1 -- 172.21.15.43:0/1941728160 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f13a00ccdd0 msgr2=0x7f13a0155e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:50:51.775 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:51.770+0000 7f13a57a8640 1 --2- 172.21.15.43:0/1941728160 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f13a00ccdd0 0x7f13a0155e00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:50:51.776 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:51.770+0000 7f13a57a8640 1 -- 172.21.15.43:0/1941728160 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f13a00697c0 msgr2=0x7f13a01596b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:50:51.776 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:51.770+0000 7f13a57a8640 1 --2- 172.21.15.43:0/1941728160 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f13a00697c0 0x7f13a01596b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:50:51.776 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:51.770+0000 7f13a57a8640 1 -- 172.21.15.43:0/1941728160 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1390009400 con 0x7f13a006ed20 2023-08-09T13:50:51.776 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:51.770+0000 7f13a57a8640 1 --2- 172.21.15.43:0/1941728160 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f13a006ed20 0x7f13a00b4710 secure :-1 s=READY pgs=1069 cs=0 l=1 rev1=1 crypto rx=0x7f13900098c0 tx=0x7f13900046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-09T13:50:51.776 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:51.771+0000 7f13967fc640 1 -- 172.21.15.43:0/1941728160 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1390004230 con 0x7f13a006ed20 2023-08-09T13:50:51.776 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:51.771+0000 7f13967fc640 1 -- 172.21.15.43:0/1941728160 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1390031f00 con 0x7f13a006ed20 2023-08-09T13:50:51.776 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:51.771+0000 7f13a820d640 1 -- 172.21.15.43:0/1941728160 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f13a0158500 con 0x7f13a006ed20 2023-08-09T13:50:51.777 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:51.771+0000 7f13967fc640 1 -- 172.21.15.43:0/1941728160 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1390032640 con 0x7f13a006ed20 2023-08-09T13:50:51.777 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:51.771+0000 7f13a820d640 1 -- 172.21.15.43:0/1941728160 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f13a0157f10 con 0x7f13a006ed20 2023-08-09T13:50:51.778 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:51.772+0000 7f13a820d640 1 -- 172.21.15.43:0/1941728160 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f13a0158a20 con 0x7f13a006ed20 2023-08-09T13:50:51.778 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:51.773+0000 7f13967fc640 1 -- 172.21.15.43:0/1941728160 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 33) v1 ==== 88444+0+0 (secure 0 0 0) 0x7f13900327a0 con 0x7f13a006ed20 2023-08-09T13:50:51.778 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:51.774+0000 7f13967fc640 1 --2- 172.21.15.43:0/1941728160 >> [v2:172.21.15.203:6832/3851880723,v1:172.21.15.203:6833/3851880723] conn(0x7f138406a170 0x7f138406c630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:50:51.778 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:51.774+0000 7f13967fc640 1 -- 172.21.15.43:0/1941728160 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(91..91 src has 1..91) v4 ==== 6770+0+0 (secure 0 0 0) 0x7f13900aac50 con 0x7f13a006ed20 2023-08-09T13:50:51.778 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:51.774+0000 7f13a5fa9640 1 -- 172.21.15.43:0/1941728160 >> [v2:172.21.15.203:6832/3851880723,v1:172.21.15.203:6833/3851880723] conn(0x7f138406a170 msgr2=0x7f138406c630 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.203:6832/3851880723 2023-08-09T13:50:51.778 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:51.774+0000 7f13a5fa9640 1 --2- 172.21.15.43:0/1941728160 >> [v2:172.21.15.203:6832/3851880723,v1:172.21.15.203:6833/3851880723] conn(0x7f138406a170 0x7f138406c630 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-08-09T13:50:51.781 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:51.779+0000 7f13967fc640 1 -- 172.21.15.43:0/1941728160 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178066 (secure 0 0 0) 0x7f13a0158a20 con 0x7f13a006ed20 2023-08-09T13:50:51.978 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:51.975+0000 7f13a5fa9640 1 -- 172.21.15.43:0/1941728160 >> [v2:172.21.15.203:6832/3851880723,v1:172.21.15.203:6833/3851880723] conn(0x7f138406a170 msgr2=0x7f138406c630 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.203:6832/3851880723 2023-08-09T13:50:51.978 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:51.975+0000 7f13a5fa9640 1 --2- 172.21.15.43:0/1941728160 >> [v2:172.21.15.203:6832/3851880723,v1:172.21.15.203:6833/3851880723] conn(0x7f138406a170 0x7f138406c630 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2023-08-09T13:50:51.979 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:51.975+0000 7f13a820d640 1 -- 172.21.15.43:0/1941728160 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f13a0158a20 con 0x7f13a006ed20 2023-08-09T13:50:51.981 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:51.978+0000 7f13967fc640 1 -- 172.21.15.43:0/1941728160 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v91) v1 ==== 74+0+19064 (secure 0 0 0) 0x7f13a0158a20 con 0x7f13a006ed20 2023-08-09T13:50:51.981 INFO:teuthology.orchestra.run.smithi043.stdout: 2023-08-09T13:50:51.981 INFO:teuthology.orchestra.run.smithi043.stdout:{"epoch":91,"fsid":"daac87ee-1337-4250-a955-aea4e408a39b","created":"2023-08-09T13:33:41.701198+0000","modified":"2023-08-09T13:50:49.168264+0000","last_up_change":"2023-08-09T13:33:53.783453+0000","last_in_change":"2023-08-09T13:33:45.148904+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":11,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":"rbd","create_time":"2023-08-09T13:33:54.365137+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":".mgr","create_time":"2023-08-09T13:33:54.831803+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":10,"pool_name":"cephfs_metadata","create_time":"2023-08-09T13:50:27.051781+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"89","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":2.5,"score_stable":2.5,"optimal_score":1,"raw_score_acting":2.5,"raw_score_stable":2.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":11,"pool_name":"cephfs_data","create_time":"2023-08-09T13:50:27.692933+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"90","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"cbd49ecd-e896-4fbd-864e-5c37a829b165","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6800","nonce":502667335},{"type":"v1","addr":"172.21.15.43:6801","nonce":502667335}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6802","nonce":502667335},{"type":"v1","addr":"172.21.15.43:6803","nonce":502667335}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6806","nonce":502667335},{"type":"v1","addr":"172.21.15.43:6807","nonce":502667335}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6804","nonce":502667335},{"type":"v1","addr":"172.21.15.43:6805","nonce":502667335}]},"public_addr":"172.21.15.43:6801/502667335","cluster_addr":"172.21.15.43:6803/502667335","heartbeat_back_addr":"172.21.15.43:6807/502667335","heartbeat_front_addr":"172.21.15.43:6805/502667335","state":["exists","up"]},{"osd":1,"uuid":"75c7e0fe-1ec7-4fe8-a2f2-3661a65b8500","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6808","nonce":3289134610},{"type":"v1","addr":"172.21.15.43:6809","nonce":3289134610}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6810","nonce":3289134610},{"type":"v1","addr":"172.21.15.43:6812","nonce":3289134610}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6821","nonce":3289134610},{"type":"v1","addr":"172.21.15.43:6824","nonce":3289134610}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6815","nonce":3289134610},{"type":"v1","addr":"172.21.15.43:6818","nonce":3289134610}]},"public_addr":"172.21.15.43:6809/3289134610","cluster_addr":"172.21.15.43:6812/3289134610","heartbeat_back_addr":"172.21.15.43:6824/3289134610","heartbeat_front_addr":"172.21.15.43:6818/3289134610","state":["exists","up"]},{"osd":2,"uuid":"bb5c41e9-7f5f-4125-9b01-3bb8bfc2ea83","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6811","nonce":4079709736},{"type":"v1","addr":"172.21.15.43:6814","nonce":4079709736}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6817","nonce":4079709736},{"type":"v1","addr":"172.21.15.43:6820","nonce":4079709736}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6829","nonce":4079709736},{"type":"v1","addr":"172.21.15.43:6831","nonce":4079709736}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6823","nonce":4079709736},{"type":"v1","addr":"172.21.15.43:6826","nonce":4079709736}]},"public_addr":"172.21.15.43:6814/4079709736","cluster_addr":"172.21.15.43:6820/4079709736","heartbeat_back_addr":"172.21.15.43:6831/4079709736","heartbeat_front_addr":"172.21.15.43:6826/4079709736","state":["exists","up"]},{"osd":3,"uuid":"bfe4c8b4-694a-465c-87e6-2b2aab852613","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6813","nonce":1709538759},{"type":"v1","addr":"172.21.15.43:6816","nonce":1709538759}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6819","nonce":1709538759},{"type":"v1","addr":"172.21.15.43:6822","nonce":1709538759}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6828","nonce":1709538759},{"type":"v1","addr":"172.21.15.43:6830","nonce":1709538759}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6825","nonce":1709538759},{"type":"v1","addr":"172.21.15.43:6827","nonce":1709538759}]},"public_addr":"172.21.15.43:6816/1709538759","cluster_addr":"172.21.15.43:6822/1709538759","heartbeat_back_addr":"172.21.15.43:6830/1709538759","heartbeat_front_addr":"172.21.15.43:6827/1709538759","state":["exists","up"]},{"osd":4,"uuid":"fa2e8a77-aba1-4756-853e-0d174a5bdeba","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6800","nonce":4196866481},{"type":"v1","addr":"172.21.15.203:6801","nonce":4196866481}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6802","nonce":4196866481},{"type":"v1","addr":"172.21.15.203:6803","nonce":4196866481}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6806","nonce":4196866481},{"type":"v1","addr":"172.21.15.203:6807","nonce":4196866481}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6804","nonce":4196866481},{"type":"v1","addr":"172.21.15.203:6805","nonce":4196866481}]},"public_addr":"172.21.15.203:6801/4196866481","cluster_addr":"172.21.15.203:6803/4196866481","heartbeat_back_addr":"172.21.15.203:6807/4196866481","heartbeat_front_addr":"172.21.15.203:6805/4196866481","state":["exists","up"]},{"osd":5,"uuid":"81811e4b-4d24-4f51-a82c-f1d700a2a00d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6816","nonce":3059290081},{"type":"v1","addr":"172.21.15.203:6817","nonce":3059290081}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6818","nonce":3059290081},{"type":"v1","addr":"172.21.15.203:6819","nonce":3059290081}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6822","nonce":3059290081},{"type":"v1","addr":"172.21.15.203:6823","nonce":3059290081}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6820","nonce":3059290081},{"type":"v1","addr":"172.21.15.203:6821","nonce":3059290081}]},"public_addr":"172.21.15.203:6817/3059290081","cluster_addr":"172.21.15.203:6819/3059290081","heartbeat_back_addr":"172.21.15.203:6823/3059290081","heartbeat_front_addr":"172.21.15.203:6821/3059290081","state":["exists","up"]},{"osd":6,"uuid":"307bac4d-6a5b-443d-af75-56d695d7674f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6824","nonce":512155569},{"type":"v1","addr":"172.21.15.203:6825","nonce":512155569}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6826","nonce":512155569},{"type":"v1","addr":"172.21.15.203:6827","nonce":512155569}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6830","nonce":512155569},{"type":"v1","addr":"172.21.15.203:6831","nonce":512155569}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6828","nonce":512155569},{"type":"v1","addr":"172.21.15.203:6829","nonce":512155569}]},"public_addr":"172.21.15.203:6825/512155569","cluster_addr":"172.21.15.203:6827/512155569","heartbeat_back_addr":"172.21.15.203:6831/512155569","heartbeat_front_addr":"172.21.15.203:6829/512155569","state":["exists","up"]},{"osd":7,"uuid":"a95dc7ca-9f1b-4295-af70-68a22a2d7d5e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6808","nonce":3331450438},{"type":"v1","addr":"172.21.15.203:6809","nonce":3331450438}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6810","nonce":3331450438},{"type":"v1","addr":"172.21.15.203:6811","nonce":3331450438}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6814","nonce":3331450438},{"type":"v1","addr":"172.21.15.203:6815","nonce":3331450438}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6812","nonce":3331450438},{"type":"v1","addr":"172.21.15.203:6813","nonce":3331450438}]},"public_addr":"172.21.15.203:6809/3331450438","cluster_addr":"172.21.15.203:6811/3331450438","heartbeat_back_addr":"172.21.15.203:6815/3331450438","heartbeat_front_addr":"172.21.15.203:6813/3331450438","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-09T13:33:50.547652+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-09T13:33:50.589469+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-09T13:33:50.561724+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-09T13:33:50.599650+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-09T13:33:50.411855+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-09T13:33:50.219681+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-09T13:33:50.504664+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-09T13:33:50.441248+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.203:6833/3325141227":"2023-08-10T13:50:49.168196+0000","172.21.15.203:6832/3325141227":"2023-08-10T13:50:49.168196+0000","172.21.15.203:0/3732830113":"2023-08-10T13:50:49.168196+0000","172.21.15.203:0/2753554568":"2023-08-10T13:50:49.168196+0000","172.21.15.203:0/2468196984":"2023-08-10T13:50:49.168196+0000","172.21.15.203:0/3375551329":"2023-08-10T13:50:49.168196+0000","172.21.15.203:0/1259705306":"2023-08-10T13:50:49.168196+0000","172.21.15.203:0/1322546009":"2023-08-10T13:50:49.168196+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":7,"snaps":[{"begin":104,"length":1},{"begin":221,"length":1}]}],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-08-09T13:50:51.984 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:51.980+0000 7f13a820d640 1 -- 172.21.15.43:0/1941728160 >> [v2:172.21.15.203:6832/3851880723,v1:172.21.15.203:6833/3851880723] conn(0x7f138406a170 msgr2=0x7f138406c630 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-08-09T13:50:51.984 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:51.980+0000 7f13a820d640 1 --2- 172.21.15.43:0/1941728160 >> [v2:172.21.15.203:6832/3851880723,v1:172.21.15.203:6833/3851880723] conn(0x7f138406a170 0x7f138406c630 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:50:51.984 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:51.980+0000 7f13a820d640 1 -- 172.21.15.43:0/1941728160 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f13a006ed20 msgr2=0x7f13a00b4710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:50:51.984 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:51.980+0000 7f13a820d640 1 --2- 172.21.15.43:0/1941728160 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f13a006ed20 0x7f13a00b4710 secure :-1 s=READY pgs=1069 cs=0 l=1 rev1=1 crypto rx=0x7f13900098c0 tx=0x7f13900046c0 comp rx=0 tx=0).stop 2023-08-09T13:50:51.984 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:51.980+0000 7f13a820d640 1 -- 172.21.15.43:0/1941728160 shutdown_connections 2023-08-09T13:50:51.985 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:51.980+0000 7f13a820d640 1 --2- 172.21.15.43:0/1941728160 >> [v2:172.21.15.203:6832/3851880723,v1:172.21.15.203:6833/3851880723] conn(0x7f138406a170 0x7f138406c630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:50:51.985 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:51.980+0000 7f13a820d640 1 --2- 172.21.15.43:0/1941728160 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f13a00ccdd0 0x7f13a0155e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:50:51.985 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:51.980+0000 7f13a820d640 1 --2- 172.21.15.43:0/1941728160 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f13a006ed20 0x7f13a00b4710 unknown :-1 s=CLOSED pgs=1069 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:50:51.985 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:51.980+0000 7f13a820d640 1 --2- 172.21.15.43:0/1941728160 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f13a00697c0 0x7f13a01596b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:50:51.985 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:51.980+0000 7f13a820d640 1 -- 172.21.15.43:0/1941728160 >> 172.21.15.43:0/1941728160 conn(0x7f13a00cbc80 msgr2=0x7f13a00c5a80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-09T13:50:51.985 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:51.981+0000 7f13a820d640 1 -- 172.21.15.43:0/1941728160 shutdown_connections 2023-08-09T13:50:51.985 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:51.981+0000 7f13a820d640 1 -- 172.21.15.43:0/1941728160 wait complete. 2023-08-09T13:50:52.002 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': 'rbd', 'create_time': '2023-08-09T13:33:54.365137+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}, 'read_balance': {'score_acting': 2, 'score_stable': 2, 'optimal_score': 1, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': '.mgr', 'create_time': '2023-08-09T13:33:54.831803+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 10, 'pool_name': 'cephfs_metadata', 'create_time': '2023-08-09T13:50:27.051781+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '89', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 2.5, 'score_stable': 2.5, 'optimal_score': 1, 'raw_score_acting': 2.5, 'raw_score_stable': 2.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 11, 'pool_name': 'cephfs_data', 'create_time': '2023-08-09T13:50:27.692933+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '90', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.3799999952316284, 'score_stable': 1.3799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.3799999952316284, 'raw_score_stable': 1.3799999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-08-09T13:50:52.004 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get rbd pg_num 2023-08-09T13:50:52.190 INFO:tasks.ceph.mgr.x.smithi203.stderr:2023-08-09T13:50:52.187+0000 7f4b1542c040 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2023-08-09T13:50:52.207 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:52.205+0000 7f4bf8fe8640 1 Processor -- start 2023-08-09T13:50:52.208 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:52.205+0000 7f4bf8fe8640 1 -- start start 2023-08-09T13:50:52.209 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:52.207+0000 7f4bf8fe8640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f4bf40cbb80 0x7f4bf40cb2b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:50:52.209 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:52.207+0000 7f4bf8fe8640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f4bf40ce1c0 0x7f4bf40ce5f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:50:52.210 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:52.207+0000 7f4bf8fe8640 1 -- --> v1:172.21.15.43:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4bf40ceb30 con 0x7f4bf415a0c0 2023-08-09T13:50:52.210 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:52.207+0000 7f4bf8fe8640 1 -- --> v2:172.21.15.43:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4bf415a4a0 con 0x7f4bf40ce1c0 2023-08-09T13:50:52.210 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:52.207+0000 7f4bf8fe8640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4bf40cc000 con 0x7f4bf40cbb80 2023-08-09T13:50:52.210 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:52.207+0000 7f4bf259c640 1 --1- >> v1:172.21.15.43:6789/0 conn(0x7f4bf415a0c0 0x7f4bf415d510 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.43:6789/0 says I am v1:172.21.15.43:59894/0 (socket says 172.21.15.43:59894) 2023-08-09T13:50:52.210 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:52.207+0000 7f4bf259c640 1 -- 172.21.15.43:0/2578375823 learned_addr learned my addr 172.21.15.43:0/2578375823 (peer_addr_for_me v1:172.21.15.43:0/0) 2023-08-09T13:50:52.211 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:52.207+0000 7f4bf2d9d640 1 --2- 172.21.15.43:0/2578375823 >> v2:172.21.15.43:3300/0 conn(0x7f4bf40ce1c0 0x7f4bf40ce5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:50:52.212 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:52.207+0000 7f4bf1d9b640 1 --2- 172.21.15.43:0/2578375823 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f4bf40cbb80 0x7f4bf40cb2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:50:52.212 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:52.208+0000 7f4bf159a640 1 -- 172.21.15.43:0/2578375823 <== mon.0 v1:172.21.15.43:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3926959438 0 0) 0x7f4bf40ceb30 con 0x7f4bf415a0c0 2023-08-09T13:50:52.212 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:52.208+0000 7f4bf2d9d640 1 -- 172.21.15.43:0/2578375823 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f4bf40cbb80 msgr2=0x7f4bf40cb2b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-09T13:50:52.212 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:52.208+0000 7f4bf2d9d640 1 --2- 172.21.15.43:0/2578375823 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f4bf40cbb80 0x7f4bf40cb2b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:50:52.212 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:52.208+0000 7f4bf2d9d640 1 -- 172.21.15.43:0/2578375823 >> v1:172.21.15.43:6789/0 conn(0x7f4bf415a0c0 legacy=0x7f4bf415d510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:50:52.212 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:52.208+0000 7f4bf2d9d640 1 -- 172.21.15.43:0/2578375823 --> v2:172.21.15.43:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4bf40cc2d0 con 0x7f4bf40ce1c0 2023-08-09T13:50:52.213 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:52.209+0000 7f4bf2d9d640 1 --2- 172.21.15.43:0/2578375823 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f4bf40ce1c0 0x7f4bf40ce5f0 secure :-1 s=READY pgs=1071 cs=0 l=1 rev1=1 crypto rx=0x7f4bec009d90 tx=0x7f4bec02bea0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=678b7740cc582c99 server_cookie=0 in_seq=0 out_seq=0 2023-08-09T13:50:52.213 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:52.209+0000 7f4bf159a640 1 -- 172.21.15.43:0/2578375823 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4bec02cc50 con 0x7f4bf40ce1c0 2023-08-09T13:50:52.213 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:52.209+0000 7f4bf159a640 1 -- 172.21.15.43:0/2578375823 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4bec02cdb0 con 0x7f4bf40ce1c0 2023-08-09T13:50:52.213 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:52.209+0000 7f4bf159a640 1 -- 172.21.15.43:0/2578375823 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4bec034660 con 0x7f4bf40ce1c0 2023-08-09T13:50:52.214 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:52.210+0000 7f4bf8fe8640 1 -- 172.21.15.43:0/2578375823 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f4bf40ce1c0 msgr2=0x7f4bf40ce5f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:50:52.214 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:52.210+0000 7f4bf8fe8640 1 --2- 172.21.15.43:0/2578375823 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f4bf40ce1c0 0x7f4bf40ce5f0 secure :-1 s=READY pgs=1071 cs=0 l=1 rev1=1 crypto rx=0x7f4bec009d90 tx=0x7f4bec02bea0 comp rx=0 tx=0).stop 2023-08-09T13:50:52.214 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:52.210+0000 7f4bf8fe8640 1 -- 172.21.15.43:0/2578375823 shutdown_connections 2023-08-09T13:50:52.215 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:52.210+0000 7f4bf8fe8640 1 --2- 172.21.15.43:0/2578375823 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f4bf40ce1c0 0x7f4bf40ce5f0 unknown :-1 s=CLOSED pgs=1071 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:50:52.215 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:52.210+0000 7f4bf8fe8640 1 --2- 172.21.15.43:0/2578375823 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f4bf40cbb80 0x7f4bf40cb2b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:50:52.215 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:52.210+0000 7f4bf8fe8640 1 -- 172.21.15.43:0/2578375823 >> 172.21.15.43:0/2578375823 conn(0x7f4bf40aff50 msgr2=0x7f4bf40b0360 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-09T13:50:52.215 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:52.210+0000 7f4bf8fe8640 1 -- 172.21.15.43:0/2578375823 shutdown_connections 2023-08-09T13:50:52.215 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:52.210+0000 7f4bf8fe8640 1 -- 172.21.15.43:0/2578375823 wait complete. 2023-08-09T13:50:52.215 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:52.212+0000 7f4bf8fe8640 1 Processor -- start 2023-08-09T13:50:52.216 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:52.212+0000 7f4bf8fe8640 1 -- start start 2023-08-09T13:50:52.218 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:52.213+0000 7f4bf8fe8640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f4bf40cbb80 0x7f4bf415fea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:50:52.218 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:52.213+0000 7f4bf8fe8640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f4bf40ce1c0 0x7f4bf41f1880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:50:52.218 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:52.213+0000 7f4bf8fe8640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f4bf415a0c0 0x7f4bf40c1ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:50:52.219 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:52.213+0000 7f4bf8fe8640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4bf40c2020 con 0x7f4bf40ce1c0 2023-08-09T13:50:52.219 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:52.213+0000 7f4bf8fe8640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4bf40c2250 con 0x7f4bf415a0c0 2023-08-09T13:50:52.219 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:52.213+0000 7f4bf8fe8640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4bf40c2530 con 0x7f4bf40cbb80 2023-08-09T13:50:52.219 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:52.213+0000 7f4bf1d9b640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f4bf40ce1c0 0x7f4bf41f1880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:50:52.219 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:52.213+0000 7f4bf1d9b640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f4bf40ce1c0 0x7f4bf41f1880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:52092/0 (socket says 172.21.15.43:52092) 2023-08-09T13:50:52.219 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:52.213+0000 7f4bf1d9b640 1 -- 172.21.15.43:0/4078768218 learned_addr learned my addr 172.21.15.43:0/4078768218 (peer_addr_for_me v2:172.21.15.43:0/0) 2023-08-09T13:50:52.219 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:52.213+0000 7f4bf2d9d640 1 --2- 172.21.15.43:0/4078768218 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f4bf415a0c0 0x7f4bf40c1ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:50:52.220 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:52.213+0000 7f4bf259c640 1 --2- 172.21.15.43:0/4078768218 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f4bf40cbb80 0x7f4bf415fea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:50:52.220 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:52.213+0000 7f4bf1d9b640 1 -- 172.21.15.43:0/4078768218 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f4bf40cbb80 msgr2=0x7f4bf415fea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:50:52.220 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:52.213+0000 7f4bf1d9b640 1 --2- 172.21.15.43:0/4078768218 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f4bf40cbb80 0x7f4bf415fea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:50:52.220 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:52.214+0000 7f4bf1d9b640 1 -- 172.21.15.43:0/4078768218 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f4bf415a0c0 msgr2=0x7f4bf40c1ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:50:52.220 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:52.214+0000 7f4bf1d9b640 1 --2- 172.21.15.43:0/4078768218 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f4bf415a0c0 0x7f4bf40c1ae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:50:52.220 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:52.214+0000 7f4bf1d9b640 1 -- 172.21.15.43:0/4078768218 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4bec009a70 con 0x7f4bf40ce1c0 2023-08-09T13:50:52.220 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:52.214+0000 7f4bf1d9b640 1 --2- 172.21.15.43:0/4078768218 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f4bf40ce1c0 0x7f4bf41f1880 secure :-1 s=READY pgs=1072 cs=0 l=1 rev1=1 crypto rx=0x7f4be800d800 tx=0x7f4be800dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-09T13:50:52.221 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:52.214+0000 7f4be2ffd640 1 -- 172.21.15.43:0/4078768218 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4be800b6c0 con 0x7f4bf40ce1c0 2023-08-09T13:50:52.221 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:52.214+0000 7f4be2ffd640 1 -- 172.21.15.43:0/4078768218 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4be800b820 con 0x7f4bf40ce1c0 2023-08-09T13:50:52.221 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:52.214+0000 7f4bf8fe8640 1 -- 172.21.15.43:0/4078768218 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4bf40c2810 con 0x7f4bf40ce1c0 2023-08-09T13:50:52.221 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:52.214+0000 7f4be2ffd640 1 -- 172.21.15.43:0/4078768218 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4be800bbb0 con 0x7f4bf40ce1c0 2023-08-09T13:50:52.221 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:52.214+0000 7f4bf8fe8640 1 -- 172.21.15.43:0/4078768218 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4bf40c2dd0 con 0x7f4bf40ce1c0 2023-08-09T13:50:52.221 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:52.215+0000 7f4bf8fe8640 1 -- 172.21.15.43:0/4078768218 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4bf40739f0 con 0x7f4bf40ce1c0 2023-08-09T13:50:52.222 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:52.217+0000 7f4be2ffd640 1 -- 172.21.15.43:0/4078768218 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 33) v1 ==== 88444+0+0 (secure 0 0 0) 0x7f4be800bd10 con 0x7f4bf40ce1c0 2023-08-09T13:50:52.222 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:52.218+0000 7f4be2ffd640 1 --2- 172.21.15.43:0/4078768218 >> [v2:172.21.15.203:6832/3851880723,v1:172.21.15.203:6833/3851880723] conn(0x7f4bd406a170 0x7f4bd406c630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:50:52.222 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:52.218+0000 7f4be2ffd640 1 -- 172.21.15.43:0/4078768218 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(91..91 src has 1..91) v4 ==== 6770+0+0 (secure 0 0 0) 0x7f4be8088ea0 con 0x7f4bf40ce1c0 2023-08-09T13:50:52.222 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:52.218+0000 7f4bf259c640 1 -- 172.21.15.43:0/4078768218 >> [v2:172.21.15.203:6832/3851880723,v1:172.21.15.203:6833/3851880723] conn(0x7f4bd406a170 msgr2=0x7f4bd406c630 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.203:6832/3851880723 2023-08-09T13:50:52.223 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:52.218+0000 7f4bf259c640 1 --2- 172.21.15.43:0/4078768218 >> [v2:172.21.15.203:6832/3851880723,v1:172.21.15.203:6833/3851880723] conn(0x7f4bd406a170 0x7f4bd406c630 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-08-09T13:50:52.226 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:52.224+0000 7f4be2ffd640 1 -- 172.21.15.43:0/4078768218 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178066 (secure 0 0 0) 0x7f4be8054240 con 0x7f4bf40ce1c0 2023-08-09T13:50:52.257 INFO:tasks.ceph.mgr.y.smithi043.stderr:2023-08-09T13:50:52.254+0000 7fe84c03a040 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2023-08-09T13:50:52.316 INFO:tasks.ceph.mgr.x.smithi203.stderr:2023-08-09T13:50:52.313+0000 7f4b1542c040 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2023-08-09T13:50:52.371 INFO:tasks.ceph.mgr.x.smithi203.stderr:2023-08-09T13:50:52.369+0000 7f4b1542c040 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2023-08-09T13:50:52.383 INFO:tasks.ceph.mgr.y.smithi043.stderr:2023-08-09T13:50:52.380+0000 7fe84c03a040 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2023-08-09T13:50:52.417 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:52.414+0000 7f4bf8fe8640 1 -- 172.21.15.43:0/4078768218 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"} v 0) v1 -- 0x7f4bf40c3ab0 con 0x7f4bf40ce1c0 2023-08-09T13:50:52.418 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:52.415+0000 7f4be2ffd640 1 -- 172.21.15.43:0/4078768218 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"}]=0 v91) v1 ==== 92+0+10 (secure 0 0 0) 0x7f4be8057860 con 0x7f4bf40ce1c0 2023-08-09T13:50:52.418 INFO:teuthology.orchestra.run.smithi043.stdout:pg_num: 8 2023-08-09T13:50:52.420 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:52.417+0000 7f4bf8fe8640 1 -- 172.21.15.43:0/4078768218 >> [v2:172.21.15.203:6832/3851880723,v1:172.21.15.203:6833/3851880723] conn(0x7f4bd406a170 msgr2=0x7f4bd406c630 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-08-09T13:50:52.420 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:52.417+0000 7f4bf8fe8640 1 --2- 172.21.15.43:0/4078768218 >> [v2:172.21.15.203:6832/3851880723,v1:172.21.15.203:6833/3851880723] conn(0x7f4bd406a170 0x7f4bd406c630 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:50:52.421 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:52.417+0000 7f4bf8fe8640 1 -- 172.21.15.43:0/4078768218 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f4bf40ce1c0 msgr2=0x7f4bf41f1880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:50:52.421 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:52.417+0000 7f4bf8fe8640 1 --2- 172.21.15.43:0/4078768218 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f4bf40ce1c0 0x7f4bf41f1880 secure :-1 s=READY pgs=1072 cs=0 l=1 rev1=1 crypto rx=0x7f4be800d800 tx=0x7f4be800dcc0 comp rx=0 tx=0).stop 2023-08-09T13:50:52.421 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:52.418+0000 7f4bf8fe8640 1 -- 172.21.15.43:0/4078768218 shutdown_connections 2023-08-09T13:50:52.421 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:52.418+0000 7f4bf8fe8640 1 --2- 172.21.15.43:0/4078768218 >> [v2:172.21.15.203:6832/3851880723,v1:172.21.15.203:6833/3851880723] conn(0x7f4bd406a170 0x7f4bd406c630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:50:52.421 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:52.418+0000 7f4bf8fe8640 1 --2- 172.21.15.43:0/4078768218 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f4bf415a0c0 0x7f4bf40c1ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:50:52.421 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:52.418+0000 7f4bf8fe8640 1 --2- 172.21.15.43:0/4078768218 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f4bf40ce1c0 0x7f4bf41f1880 unknown :-1 s=CLOSED pgs=1072 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:50:52.421 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:52.418+0000 7f4bf8fe8640 1 --2- 172.21.15.43:0/4078768218 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f4bf40cbb80 0x7f4bf415fea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:50:52.422 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:52.418+0000 7f4bf8fe8640 1 -- 172.21.15.43:0/4078768218 >> 172.21.15.43:0/4078768218 conn(0x7f4bf40aff50 msgr2=0x7f4bf4160a40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-09T13:50:52.422 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:52.418+0000 7f4bf8fe8640 1 -- 172.21.15.43:0/4078768218 shutdown_connections 2023-08-09T13:50:52.422 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:52.418+0000 7f4bf8fe8640 1 -- 172.21.15.43:0/4078768218 wait complete. 2023-08-09T13:50:52.422 INFO:tasks.ceph.mgr.x.smithi203.stderr:2023-08-09T13:50:52.419+0000 7f4b1542c040 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2023-08-09T13:50:52.437 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-08-09T13:50:52.444 INFO:tasks.ceph.mgr.y.smithi043.stderr:2023-08-09T13:50:52.442+0000 7fe84c03a040 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2023-08-09T13:50:52.498 INFO:tasks.ceph.mgr.y.smithi043.stderr:2023-08-09T13:50:52.495+0000 7fe84c03a040 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2023-08-09T13:50:52.648 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:52.645+0000 7f8942f7f640 1 Processor -- start 2023-08-09T13:50:52.648 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:52.646+0000 7f8942f7f640 1 -- start start 2023-08-09T13:50:52.650 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:52.647+0000 7f8942f7f640 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7f893c15f970 0x7f893c0cf1d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:50:52.650 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:52.647+0000 7f8942f7f640 1 -- --> v1:172.21.15.43:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f893c0d24a0 con 0x7f893c0d1ec0 2023-08-09T13:50:52.650 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:52.647+0000 7f8942f7f640 1 -- --> v1:172.21.15.203:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f893c0d2780 con 0x7f893c0cf710 2023-08-09T13:50:52.650 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:52.647+0000 7f8942f7f640 1 -- --> v2:172.21.15.203:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f893c0d1280 con 0x7f893c15f970 2023-08-09T13:50:52.652 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:52.648+0000 7f894277e640 1 --1- >> v1:172.21.15.43:6789/0 conn(0x7f893c0d1ec0 0x7f893c0b5670 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.43:6789/0 says I am v1:172.21.15.43:59906/0 (socket says 172.21.15.43:59906) 2023-08-09T13:50:52.652 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:52.648+0000 7f894277e640 1 -- 172.21.15.43:0/1496998495 learned_addr learned my addr 172.21.15.43:0/1496998495 (peer_addr_for_me v1:172.21.15.43:0/0) 2023-08-09T13:50:52.652 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:52.648+0000 7f8941f7d640 1 --2- 172.21.15.43:0/1496998495 >> v2:172.21.15.203:3300/0 conn(0x7f893c15f970 0x7f893c0cf1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:50:52.652 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:52.648+0000 7f8940f7b640 1 -- 172.21.15.43:0/1496998495 <== mon.0 v1:172.21.15.43:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 432780965 0 0) 0x7f893c0d24a0 con 0x7f893c0d1ec0 2023-08-09T13:50:52.653 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:52.648+0000 7f8940f7b640 1 -- 172.21.15.43:0/1496998495 --> v1:172.21.15.43:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f8928003520 con 0x7f893c0d1ec0 2023-08-09T13:50:52.653 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:52.649+0000 7f8940f7b640 1 -- 172.21.15.43:0/1496998495 <== mon.0 v1:172.21.15.43:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1407408774 0 0) 0x7f8928003520 con 0x7f893c0d1ec0 2023-08-09T13:50:52.653 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:52.649+0000 7f8940f7b640 1 -- 172.21.15.43:0/1496998495 >> v2:172.21.15.203:3300/0 conn(0x7f893c15f970 msgr2=0x7f893c0cf1d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-09T13:50:52.653 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:52.649+0000 7f8940f7b640 1 --2- 172.21.15.43:0/1496998495 >> v2:172.21.15.203:3300/0 conn(0x7f893c15f970 0x7f893c0cf1d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:50:52.653 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:52.649+0000 7f8940f7b640 1 -- 172.21.15.43:0/1496998495 >> v1:172.21.15.203:6789/0 conn(0x7f893c0cf710 legacy=0x7f893c0d17b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:50:52.653 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:52.649+0000 7f8940f7b640 1 -- 172.21.15.43:0/1496998495 --> v1:172.21.15.43:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f893c0d2a60 con 0x7f893c0d1ec0 2023-08-09T13:50:52.653 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:52.649+0000 7f8940f7b640 1 -- 172.21.15.43:0/1496998495 <== mon.0 v1:172.21.15.43:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1285150740 0 0) 0x7f89340019f0 con 0x7f893c0d1ec0 2023-08-09T13:50:52.654 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:52.649+0000 7f8940f7b640 1 -- 172.21.15.43:0/1496998495 >> v1:172.21.15.43:6789/0 conn(0x7f893c0d1ec0 legacy=0x7f893c0b5670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:50:52.654 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:52.649+0000 7f8940f7b640 1 --2- 172.21.15.43:0/1496998495 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f8928003970 0x7f8928003d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:50:52.654 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:52.649+0000 7f8940f7b640 1 --2- 172.21.15.43:0/1496998495 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f89280047c0 0x7f8928004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:50:52.654 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:52.649+0000 7f8940f7b640 1 --2- 172.21.15.43:0/1496998495 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f8928005180 0x7f892800b980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:50:52.654 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:52.649+0000 7f8940f7b640 1 -- 172.21.15.43:0/1496998495 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f892800c0f0 con 0x7f8928003970 2023-08-09T13:50:52.654 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:52.649+0000 7f8940f7b640 1 -- 172.21.15.43:0/1496998495 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f892800c260 con 0x7f89280047c0 2023-08-09T13:50:52.654 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:52.649+0000 7f8940f7b640 1 -- 172.21.15.43:0/1496998495 --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f892800c420 con 0x7f8928005180 2023-08-09T13:50:52.655 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:52.649+0000 7f8941f7d640 1 --2- 172.21.15.43:0/1496998495 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f8928003970 0x7f8928003d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:50:52.655 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:52.649+0000 7f8941f7d640 1 -- 172.21.15.43:0/1496998495 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f8928005180 msgr2=0x7f892800b980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-09T13:50:52.655 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:52.649+0000 7f8941f7d640 1 --2- 172.21.15.43:0/1496998495 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f8928005180 0x7f892800b980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:50:52.655 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:52.649+0000 7f8941f7d640 1 -- 172.21.15.43:0/1496998495 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f89280047c0 msgr2=0x7f8928004c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-09T13:50:52.655 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:52.649+0000 7f8941f7d640 1 --2- 172.21.15.43:0/1496998495 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f89280047c0 0x7f8928004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:50:52.655 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:52.649+0000 7f8941f7d640 1 -- 172.21.15.43:0/1496998495 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8928003520 con 0x7f8928003970 2023-08-09T13:50:52.655 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:52.650+0000 7f894277e640 1 -- 172.21.15.43:0/1496998495 reap_dead start 2023-08-09T13:50:52.656 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:52.651+0000 7f8941f7d640 1 --2- 172.21.15.43:0/1496998495 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f8928003970 0x7f8928003d90 secure :-1 s=READY pgs=1074 cs=0 l=1 rev1=1 crypto rx=0x7f8938009ac0 tx=0x7f89280057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c3ca3d2395c23015 server_cookie=0 in_seq=0 out_seq=0 2023-08-09T13:50:52.657 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:52.651+0000 7f8940f7b640 1 -- 172.21.15.43:0/1496998495 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f893802c6b0 con 0x7f8928003970 2023-08-09T13:50:52.657 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:52.651+0000 7f8940f7b640 1 -- 172.21.15.43:0/1496998495 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f893802c810 con 0x7f8928003970 2023-08-09T13:50:52.657 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:52.651+0000 7f8940f7b640 1 -- 172.21.15.43:0/1496998495 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f893802cba0 con 0x7f8928003970 2023-08-09T13:50:52.657 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:52.651+0000 7f8942f7f640 1 -- 172.21.15.43:0/1496998495 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f8928003970 msgr2=0x7f8928003d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:50:52.657 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:52.651+0000 7f8942f7f640 1 --2- 172.21.15.43:0/1496998495 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f8928003970 0x7f8928003d90 secure :-1 s=READY pgs=1074 cs=0 l=1 rev1=1 crypto rx=0x7f8938009ac0 tx=0x7f89280057f0 comp rx=0 tx=0).stop 2023-08-09T13:50:52.658 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:52.651+0000 7f8942f7f640 1 -- 172.21.15.43:0/1496998495 shutdown_connections 2023-08-09T13:50:52.658 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:52.651+0000 7f8942f7f640 1 --2- 172.21.15.43:0/1496998495 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f8928003970 0x7f8928003d90 unknown :-1 s=CLOSED pgs=1074 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:50:52.658 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:52.651+0000 7f8942f7f640 1 -- 172.21.15.43:0/1496998495 >> 172.21.15.43:0/1496998495 conn(0x7f893c0712e0 msgr2=0x7f893c0716f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-09T13:50:52.658 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:52.652+0000 7f8942f7f640 1 -- 172.21.15.43:0/1496998495 shutdown_connections 2023-08-09T13:50:52.658 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:52.652+0000 7f8942f7f640 1 -- 172.21.15.43:0/1496998495 wait complete. 2023-08-09T13:50:52.658 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:52.654+0000 7f8942f7f640 1 Processor -- start 2023-08-09T13:50:52.658 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:52.654+0000 7f8942f7f640 1 -- start start 2023-08-09T13:50:52.660 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:52.655+0000 7f8942f7f640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f8928003970 0x7f893c154250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:50:52.660 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:52.655+0000 7f8942f7f640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f893c1548b0 0x7f893c0b19d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:50:52.660 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:52.655+0000 7f8942f7f640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f893c15d5d0 0x7f893c15da50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:50:52.660 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:52.655+0000 7f8942f7f640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f893c15df90 con 0x7f893c1548b0 2023-08-09T13:50:52.660 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:52.655+0000 7f8942f7f640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f893c1f5b40 con 0x7f8928003970 2023-08-09T13:50:52.661 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:52.655+0000 7f8942f7f640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f893c157090 con 0x7f893c15d5d0 2023-08-09T13:50:52.661 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:52.655+0000 7f894177c640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f893c1548b0 0x7f893c0b19d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:50:52.661 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:52.655+0000 7f894177c640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f893c1548b0 0x7f893c0b19d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:52102/0 (socket says 172.21.15.43:52102) 2023-08-09T13:50:52.661 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:52.655+0000 7f894177c640 1 -- 172.21.15.43:0/2995141822 learned_addr learned my addr 172.21.15.43:0/2995141822 (peer_addr_for_me v2:172.21.15.43:0/0) 2023-08-09T13:50:52.661 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:52.655+0000 7f8941f7d640 1 --2- 172.21.15.43:0/2995141822 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f8928003970 0x7f893c154250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:50:52.661 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:52.655+0000 7f894177c640 1 -- 172.21.15.43:0/2995141822 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f893c15d5d0 msgr2=0x7f893c15da50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:50:52.661 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:52.655+0000 7f894177c640 1 --2- 172.21.15.43:0/2995141822 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f893c15d5d0 0x7f893c15da50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:50:52.662 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:52.655+0000 7f894177c640 1 -- 172.21.15.43:0/2995141822 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f8928003970 msgr2=0x7f893c154250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:50:52.662 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:52.655+0000 7f894177c640 1 --2- 172.21.15.43:0/2995141822 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f8928003970 0x7f893c154250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:50:52.662 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:52.655+0000 7f894177c640 1 -- 172.21.15.43:0/2995141822 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f893c156480 con 0x7f893c1548b0 2023-08-09T13:50:52.662 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:52.656+0000 7f894177c640 1 --2- 172.21.15.43:0/2995141822 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f893c1548b0 0x7f893c0b19d0 secure :-1 s=READY pgs=1075 cs=0 l=1 rev1=1 crypto rx=0x7f892c00c530 tx=0x7f892c00c9f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-09T13:50:52.663 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:52.656+0000 7f89327fc640 1 -- 172.21.15.43:0/2995141822 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f892c0052d0 con 0x7f893c1548b0 2023-08-09T13:50:52.663 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:52.656+0000 7f89327fc640 1 -- 172.21.15.43:0/2995141822 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f892c005430 con 0x7f893c1548b0 2023-08-09T13:50:52.663 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:52.656+0000 7f8942f7f640 1 -- 172.21.15.43:0/2995141822 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f893c156760 con 0x7f893c1548b0 2023-08-09T13:50:52.663 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:52.656+0000 7f89327fc640 1 -- 172.21.15.43:0/2995141822 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f892c012870 con 0x7f893c1548b0 2023-08-09T13:50:52.663 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:52.656+0000 7f8942f7f640 1 -- 172.21.15.43:0/2995141822 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f893c156d20 con 0x7f893c1548b0 2023-08-09T13:50:52.663 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:52.657+0000 7f8942f7f640 1 -- 172.21.15.43:0/2995141822 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f893c156ee0 con 0x7f893c1548b0 2023-08-09T13:50:52.664 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:52.658+0000 7f89327fc640 1 -- 172.21.15.43:0/2995141822 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 33) v1 ==== 88444+0+0 (secure 0 0 0) 0x7f892c0129d0 con 0x7f893c1548b0 2023-08-09T13:50:52.664 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:52.659+0000 7f89327fc640 1 --2- 172.21.15.43:0/2995141822 >> [v2:172.21.15.203:6832/3851880723,v1:172.21.15.203:6833/3851880723] conn(0x7f892006a1a0 0x7f892006c660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:50:52.664 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:52.659+0000 7f89327fc640 1 -- 172.21.15.43:0/2995141822 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(91..91 src has 1..91) v4 ==== 6770+0+0 (secure 0 0 0) 0x7f892c088620 con 0x7f893c1548b0 2023-08-09T13:50:52.664 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:52.660+0000 7f8941f7d640 1 -- 172.21.15.43:0/2995141822 >> [v2:172.21.15.203:6832/3851880723,v1:172.21.15.203:6833/3851880723] conn(0x7f892006a1a0 msgr2=0x7f892006c660 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.203:6832/3851880723 2023-08-09T13:50:52.664 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:52.660+0000 7f8941f7d640 1 --2- 172.21.15.43:0/2995141822 >> [v2:172.21.15.203:6832/3851880723,v1:172.21.15.203:6833/3851880723] conn(0x7f892006a1a0 0x7f892006c660 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-08-09T13:50:52.670 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:52.668+0000 7f89327fc640 1 -- 172.21.15.43:0/2995141822 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178066 (secure 0 0 0) 0x7f892c053890 con 0x7f893c1548b0 2023-08-09T13:50:52.861 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:52.859+0000 7f8942f7f640 1 -- 172.21.15.43:0/2995141822 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f893c04e390 con 0x7f893c1548b0 2023-08-09T13:50:52.862 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:52.860+0000 7f89327fc640 1 -- 172.21.15.43:0/2995141822 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v91) v1 ==== 93+0+10 (secure 0 0 0) 0x7f892c056eb0 con 0x7f893c1548b0 2023-08-09T13:50:52.863 INFO:teuthology.orchestra.run.smithi043.stdout:pg_num: 1 2023-08-09T13:50:52.863 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:52.861+0000 7f8941f7d640 1 -- 172.21.15.43:0/2995141822 >> [v2:172.21.15.203:6832/3851880723,v1:172.21.15.203:6833/3851880723] conn(0x7f892006a1a0 msgr2=0x7f892006c660 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.203:6832/3851880723 2023-08-09T13:50:52.864 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:52.861+0000 7f8941f7d640 1 --2- 172.21.15.43:0/2995141822 >> [v2:172.21.15.203:6832/3851880723,v1:172.21.15.203:6833/3851880723] conn(0x7f892006a1a0 0x7f892006c660 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2023-08-09T13:50:52.865 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:52.863+0000 7f8942f7f640 1 -- 172.21.15.43:0/2995141822 >> [v2:172.21.15.203:6832/3851880723,v1:172.21.15.203:6833/3851880723] conn(0x7f892006a1a0 msgr2=0x7f892006c660 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-08-09T13:50:52.866 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:52.863+0000 7f8942f7f640 1 --2- 172.21.15.43:0/2995141822 >> [v2:172.21.15.203:6832/3851880723,v1:172.21.15.203:6833/3851880723] conn(0x7f892006a1a0 0x7f892006c660 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:50:52.866 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:52.863+0000 7f8942f7f640 1 -- 172.21.15.43:0/2995141822 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f893c1548b0 msgr2=0x7f893c0b19d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:50:52.866 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:52.863+0000 7f8942f7f640 1 --2- 172.21.15.43:0/2995141822 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f893c1548b0 0x7f893c0b19d0 secure :-1 s=READY pgs=1075 cs=0 l=1 rev1=1 crypto rx=0x7f892c00c530 tx=0x7f892c00c9f0 comp rx=0 tx=0).stop 2023-08-09T13:50:52.866 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:52.863+0000 7f8942f7f640 1 -- 172.21.15.43:0/2995141822 shutdown_connections 2023-08-09T13:50:52.866 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:52.863+0000 7f8942f7f640 1 --2- 172.21.15.43:0/2995141822 >> [v2:172.21.15.203:6832/3851880723,v1:172.21.15.203:6833/3851880723] conn(0x7f892006a1a0 0x7f892006c660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:50:52.867 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:52.863+0000 7f8942f7f640 1 --2- 172.21.15.43:0/2995141822 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f893c15d5d0 0x7f893c15da50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:50:52.867 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:52.863+0000 7f8942f7f640 1 --2- 172.21.15.43:0/2995141822 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f893c1548b0 0x7f893c0b19d0 unknown :-1 s=CLOSED pgs=1075 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:50:52.867 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:52.863+0000 7f8942f7f640 1 --2- 172.21.15.43:0/2995141822 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f8928003970 0x7f893c154250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:50:52.867 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:52.863+0000 7f8942f7f640 1 -- 172.21.15.43:0/2995141822 >> 172.21.15.43:0/2995141822 conn(0x7f893c0712e0 msgr2=0x7f893c0b5690 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-09T13:50:52.867 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:52.863+0000 7f8942f7f640 1 -- 172.21.15.43:0/2995141822 shutdown_connections 2023-08-09T13:50:52.868 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:52.863+0000 7f8942f7f640 1 -- 172.21.15.43:0/2995141822 wait complete. 2023-08-09T13:50:52.883 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-08-09T13:50:53.087 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:53.084+0000 7fca711ab640 1 Processor -- start 2023-08-09T13:50:53.088 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:53.084+0000 7fca711ab640 1 -- start start 2023-08-09T13:50:53.089 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:53.086+0000 7fca711ab640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fca6c15b140 0x7fca6c13d7e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:50:53.089 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:53.086+0000 7fca711ab640 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7fca6c06de70 0x7fca6c0ce3b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:50:53.089 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:53.086+0000 7fca711ab640 1 -- --> v1:172.21.15.203:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fca6c071320 con 0x7fca6c13dd20 2023-08-09T13:50:53.089 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:53.086+0000 7fca711ab640 1 -- --> v2:172.21.15.203:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fca6c071600 con 0x7fca6c06de70 2023-08-09T13:50:53.090 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:53.086+0000 7fca711ab640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fca6c0cdbb0 con 0x7fca6c15b140 2023-08-09T13:50:53.090 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:53.087+0000 7fca6bfff640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fca6c15b140 0x7fca6c13d7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:50:53.091 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:53.087+0000 7fca6bfff640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fca6c15b140 0x7fca6c13d7e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.203:3301/0 says I am v2:172.21.15.43:40770/0 (socket says 172.21.15.43:40770) 2023-08-09T13:50:53.091 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:53.087+0000 7fca6bfff640 1 -- 172.21.15.43:0/3531994299 learned_addr learned my addr 172.21.15.43:0/3531994299 (peer_addr_for_me v2:172.21.15.43:0/0) 2023-08-09T13:50:53.091 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:53.087+0000 7fca709aa640 1 --2- 172.21.15.43:0/3531994299 >> v2:172.21.15.203:3300/0 conn(0x7fca6c06de70 0x7fca6c0ce3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:50:53.091 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:53.088+0000 7fca6bfff640 1 -- 172.21.15.43:0/3531994299 >> v2:172.21.15.203:3300/0 conn(0x7fca6c06de70 msgr2=0x7fca6c0ce3b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-09T13:50:53.091 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:53.088+0000 7fca6bfff640 1 --2- 172.21.15.43:0/3531994299 >> v2:172.21.15.203:3300/0 conn(0x7fca6c06de70 0x7fca6c0ce3b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:50:53.092 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:53.088+0000 7fca6bfff640 1 -- 172.21.15.43:0/3531994299 >> v1:172.21.15.203:6789/0 conn(0x7fca6c13dd20 legacy=0x7fca6c06d760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:50:53.092 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:53.088+0000 7fca6bfff640 1 -- 172.21.15.43:0/3531994299 --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fca6c0cde90 con 0x7fca6c15b140 2023-08-09T13:50:53.093 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:53.089+0000 7fca6bfff640 1 --2- 172.21.15.43:0/3531994299 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fca6c15b140 0x7fca6c13d7e0 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7fca5c009870 tx=0x7fca5c02b980 comp rx=0 tx=0).ready entity=mon.2 client_cookie=7218d3862f6c7403 server_cookie=0 in_seq=0 out_seq=0 2023-08-09T13:50:53.093 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:53.089+0000 7fca6b7fe640 1 -- 172.21.15.43:0/3531994299 <== mon.2 v2:172.21.15.203:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fca5c030840 con 0x7fca6c15b140 2023-08-09T13:50:53.093 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:53.090+0000 7fca6b7fe640 1 -- 172.21.15.43:0/3531994299 <== mon.2 v2:172.21.15.203:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fca5c0309a0 con 0x7fca6c15b140 2023-08-09T13:50:53.093 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:53.090+0000 7fca6b7fe640 1 -- 172.21.15.43:0/3531994299 <== mon.2 v2:172.21.15.203:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fca5c030d30 con 0x7fca6c15b140 2023-08-09T13:50:53.093 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:53.090+0000 7fca711ab640 1 -- 172.21.15.43:0/3531994299 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fca6c15b140 msgr2=0x7fca6c13d7e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:50:53.094 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:53.090+0000 7fca711ab640 1 --2- 172.21.15.43:0/3531994299 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fca6c15b140 0x7fca6c13d7e0 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7fca5c009870 tx=0x7fca5c02b980 comp rx=0 tx=0).stop 2023-08-09T13:50:53.094 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:53.090+0000 7fca711ab640 1 -- 172.21.15.43:0/3531994299 shutdown_connections 2023-08-09T13:50:53.094 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:53.090+0000 7fca711ab640 1 --2- 172.21.15.43:0/3531994299 >> v2:172.21.15.203:3300/0 conn(0x7fca6c06de70 0x7fca6c0ce3b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:50:53.094 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:53.090+0000 7fca711ab640 1 --2- 172.21.15.43:0/3531994299 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fca6c15b140 0x7fca6c13d7e0 unknown :-1 s=CLOSED pgs=149 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:50:53.094 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:53.090+0000 7fca711ab640 1 -- 172.21.15.43:0/3531994299 >> 172.21.15.43:0/3531994299 conn(0x7fca6c06ce10 msgr2=0x7fca6c0cef70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-09T13:50:53.094 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:53.090+0000 7fca711ab640 1 -- 172.21.15.43:0/3531994299 shutdown_connections 2023-08-09T13:50:53.094 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:53.090+0000 7fca711ab640 1 -- 172.21.15.43:0/3531994299 wait complete. 2023-08-09T13:50:53.095 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:53.092+0000 7fca711ab640 1 Processor -- start 2023-08-09T13:50:53.095 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:53.092+0000 7fca711ab640 1 -- start start 2023-08-09T13:50:53.096 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:53.093+0000 7fca711ab640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fca6c06de70 0x7fca6c1e2480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:50:53.096 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:53.093+0000 7fca711ab640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fca6c13dd20 0x7fca6c157880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:50:53.097 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:53.093+0000 7fca711ab640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fca6c15b140 0x7fca6c13ae30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:50:53.097 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:53.093+0000 7fca711ab640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fca6c1e2060 con 0x7fca6c15b140 2023-08-09T13:50:53.097 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:53.093+0000 7fca711ab640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fca6c1e13e0 con 0x7fca6c06de70 2023-08-09T13:50:53.097 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:53.093+0000 7fca711ab640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fca6c1e16c0 con 0x7fca6c13dd20 2023-08-09T13:50:53.097 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:53.093+0000 7fca709aa640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fca6c15b140 0x7fca6c13ae30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:50:53.097 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:53.094+0000 7fca709aa640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fca6c15b140 0x7fca6c13ae30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:52118/0 (socket says 172.21.15.43:52118) 2023-08-09T13:50:53.097 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:53.094+0000 7fca709aa640 1 -- 172.21.15.43:0/4241832072 learned_addr learned my addr 172.21.15.43:0/4241832072 (peer_addr_for_me v2:172.21.15.43:0/0) 2023-08-09T13:50:53.098 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:53.094+0000 7fca63fff640 1 --2- 172.21.15.43:0/4241832072 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fca6c13dd20 0x7fca6c157880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:50:53.098 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:53.094+0000 7fca6bfff640 1 --2- 172.21.15.43:0/4241832072 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fca6c06de70 0x7fca6c1e2480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:50:53.098 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:53.094+0000 7fca709aa640 1 -- 172.21.15.43:0/4241832072 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fca6c13dd20 msgr2=0x7fca6c157880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:50:53.098 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:53.094+0000 7fca709aa640 1 --2- 172.21.15.43:0/4241832072 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fca6c13dd20 0x7fca6c157880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:50:53.098 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:53.094+0000 7fca709aa640 1 -- 172.21.15.43:0/4241832072 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fca6c06de70 msgr2=0x7fca6c1e2480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:50:53.098 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:53.094+0000 7fca709aa640 1 --2- 172.21.15.43:0/4241832072 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fca6c06de70 0x7fca6c1e2480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:50:53.098 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:53.094+0000 7fca709aa640 1 -- 172.21.15.43:0/4241832072 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fca5c0094d0 con 0x7fca6c15b140 2023-08-09T13:50:53.099 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:53.094+0000 7fca709aa640 1 --2- 172.21.15.43:0/4241832072 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fca6c15b140 0x7fca6c13ae30 secure :-1 s=READY pgs=1076 cs=0 l=1 rev1=1 crypto rx=0x7fca6400efc0 tx=0x7fca6400c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-09T13:50:53.100 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:53.095+0000 7fca697fa640 1 -- 172.21.15.43:0/4241832072 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fca64009b20 con 0x7fca6c15b140 2023-08-09T13:50:53.100 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:53.095+0000 7fca697fa640 1 -- 172.21.15.43:0/4241832072 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fca6400f040 con 0x7fca6c15b140 2023-08-09T13:50:53.100 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:53.095+0000 7fca697fa640 1 -- 172.21.15.43:0/4241832072 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fca640146c0 con 0x7fca6c15b140 2023-08-09T13:50:53.101 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:53.095+0000 7fca711ab640 1 -- 172.21.15.43:0/4241832072 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fca6c15b6a0 con 0x7fca6c15b140 2023-08-09T13:50:53.101 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:53.095+0000 7fca711ab640 1 -- 172.21.15.43:0/4241832072 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fca6c1e1c80 con 0x7fca6c15b140 2023-08-09T13:50:53.102 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:53.096+0000 7fca711ab640 1 -- 172.21.15.43:0/4241832072 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fca6c04e390 con 0x7fca6c15b140 2023-08-09T13:50:53.102 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:53.097+0000 7fca697fa640 1 -- 172.21.15.43:0/4241832072 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 33) v1 ==== 88444+0+0 (secure 0 0 0) 0x7fca64014820 con 0x7fca6c15b140 2023-08-09T13:50:53.103 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:53.098+0000 7fca697fa640 1 --2- 172.21.15.43:0/4241832072 >> [v2:172.21.15.203:6832/3851880723,v1:172.21.15.203:6833/3851880723] conn(0x7fca4c06a170 0x7fca4c06c630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:50:53.103 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:53.098+0000 7fca697fa640 1 -- 172.21.15.43:0/4241832072 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(91..91 src has 1..91) v4 ==== 6770+0+0 (secure 0 0 0) 0x7fca64089890 con 0x7fca6c15b140 2023-08-09T13:50:53.103 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:53.098+0000 7fca6bfff640 1 -- 172.21.15.43:0/4241832072 >> [v2:172.21.15.203:6832/3851880723,v1:172.21.15.203:6833/3851880723] conn(0x7fca4c06a170 msgr2=0x7fca4c06c630 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.203:6832/3851880723 2023-08-09T13:50:53.103 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:53.098+0000 7fca6bfff640 1 --2- 172.21.15.43:0/4241832072 >> [v2:172.21.15.203:6832/3851880723,v1:172.21.15.203:6833/3851880723] conn(0x7fca4c06a170 0x7fca4c06c630 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-08-09T13:50:53.108 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:53.106+0000 7fca697fa640 1 -- 172.21.15.43:0/4241832072 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178066 (secure 0 0 0) 0x7fca640585b0 con 0x7fca6c15b140 2023-08-09T13:50:53.148 INFO:tasks.ceph.mgr.x.smithi203.stderr:2023-08-09T13:50:53.146+0000 7f4b1542c040 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2023-08-09T13:50:53.215 INFO:tasks.ceph.mgr.x.smithi203.stderr:2023-08-09T13:50:53.213+0000 7f4b1542c040 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2023-08-09T13:50:53.243 INFO:tasks.ceph.mgr.y.smithi043.stderr:2023-08-09T13:50:53.240+0000 7fe84c03a040 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2023-08-09T13:50:53.275 INFO:tasks.ceph.mgr.x.smithi203.stderr:2023-08-09T13:50:53.273+0000 7f4b1542c040 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2023-08-09T13:50:53.302 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:53.299+0000 7fca6bfff640 1 -- 172.21.15.43:0/4241832072 >> [v2:172.21.15.203:6832/3851880723,v1:172.21.15.203:6833/3851880723] conn(0x7fca4c06a170 msgr2=0x7fca4c06c630 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.203:6832/3851880723 2023-08-09T13:50:53.303 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:53.299+0000 7fca6bfff640 1 --2- 172.21.15.43:0/4241832072 >> [v2:172.21.15.203:6832/3851880723,v1:172.21.15.203:6833/3851880723] conn(0x7fca4c06a170 0x7fca4c06c630 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2023-08-09T13:50:53.309 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:53.306+0000 7fca711ab640 1 -- 172.21.15.43:0/4241832072 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7fca6c0af8e0 con 0x7fca6c15b140 2023-08-09T13:50:53.310 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:53.307+0000 7fca697fa640 1 -- 172.21.15.43:0/4241832072 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v91) v1 ==== 104+0+11 (secure 0 0 0) 0x7fca64009e60 con 0x7fca6c15b140 2023-08-09T13:50:53.310 INFO:teuthology.orchestra.run.smithi043.stdout:pg_num: 32 2023-08-09T13:50:53.311 INFO:tasks.ceph.mgr.y.smithi043.stderr:2023-08-09T13:50:53.308+0000 7fe84c03a040 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2023-08-09T13:50:53.312 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:53.309+0000 7fca711ab640 1 -- 172.21.15.43:0/4241832072 >> [v2:172.21.15.203:6832/3851880723,v1:172.21.15.203:6833/3851880723] conn(0x7fca4c06a170 msgr2=0x7fca4c06c630 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-08-09T13:50:53.313 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:53.309+0000 7fca711ab640 1 --2- 172.21.15.43:0/4241832072 >> [v2:172.21.15.203:6832/3851880723,v1:172.21.15.203:6833/3851880723] conn(0x7fca4c06a170 0x7fca4c06c630 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:50:53.313 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:53.309+0000 7fca711ab640 1 -- 172.21.15.43:0/4241832072 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fca6c15b140 msgr2=0x7fca6c13ae30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:50:53.313 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:53.309+0000 7fca711ab640 1 --2- 172.21.15.43:0/4241832072 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fca6c15b140 0x7fca6c13ae30 secure :-1 s=READY pgs=1076 cs=0 l=1 rev1=1 crypto rx=0x7fca6400efc0 tx=0x7fca6400c490 comp rx=0 tx=0).stop 2023-08-09T13:50:53.313 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:53.310+0000 7fca711ab640 1 -- 172.21.15.43:0/4241832072 shutdown_connections 2023-08-09T13:50:53.313 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:53.310+0000 7fca711ab640 1 --2- 172.21.15.43:0/4241832072 >> [v2:172.21.15.203:6832/3851880723,v1:172.21.15.203:6833/3851880723] conn(0x7fca4c06a170 0x7fca4c06c630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:50:53.313 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:53.310+0000 7fca711ab640 1 --2- 172.21.15.43:0/4241832072 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fca6c15b140 0x7fca6c13ae30 unknown :-1 s=CLOSED pgs=1076 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:50:53.313 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:53.310+0000 7fca711ab640 1 --2- 172.21.15.43:0/4241832072 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fca6c13dd20 0x7fca6c157880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:50:53.314 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:53.310+0000 7fca711ab640 1 --2- 172.21.15.43:0/4241832072 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fca6c06de70 0x7fca6c1e2480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:50:53.314 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:53.310+0000 7fca711ab640 1 -- 172.21.15.43:0/4241832072 >> 172.21.15.43:0/4241832072 conn(0x7fca6c06ce10 msgr2=0x7fca6c0b00e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-09T13:50:53.314 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:53.310+0000 7fca711ab640 1 -- 172.21.15.43:0/4241832072 shutdown_connections 2023-08-09T13:50:53.314 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:53.310+0000 7fca711ab640 1 -- 172.21.15.43:0/4241832072 wait complete. 2023-08-09T13:50:53.329 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-08-09T13:50:53.364 INFO:tasks.ceph.mgr.y.smithi043.stderr:2023-08-09T13:50:53.361+0000 7fe84c03a040 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2023-08-09T13:50:53.404 INFO:tasks.ceph.mgr.x.smithi203.stderr:2023-08-09T13:50:53.401+0000 7f4b1542c040 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2023-08-09T13:50:53.491 INFO:tasks.ceph.mgr.y.smithi043.stderr:2023-08-09T13:50:53.488+0000 7fe84c03a040 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2023-08-09T13:50:53.534 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:53.531+0000 7f6ca82ad640 1 Processor -- start 2023-08-09T13:50:53.534 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:53.531+0000 7f6ca82ad640 1 -- start start 2023-08-09T13:50:53.535 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:53.533+0000 7f6ca82ad640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f6ca015c550 0x7f6ca00cd6d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:50:53.536 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:53.533+0000 7f6ca82ad640 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7f6ca00b45f0 0x7f6ca00b59d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:50:53.536 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:53.533+0000 7f6ca82ad640 1 -- --> v1:172.21.15.43:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6ca00b7bd0 con 0x7f6ca00d04b0 2023-08-09T13:50:53.536 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:53.533+0000 7f6ca82ad640 1 -- --> v2:172.21.15.203:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6ca00d1c00 con 0x7f6ca00b45f0 2023-08-09T13:50:53.536 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:53.533+0000 7f6ca82ad640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6ca015c930 con 0x7f6ca015c550 2023-08-09T13:50:53.538 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:53.533+0000 7f6ca5848640 1 --1- >> v1:172.21.15.43:6789/0 conn(0x7f6ca00d04b0 0x7f6ca00b3ee0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.43:6789/0 says I am v1:172.21.15.43:59920/0 (socket says 172.21.15.43:59920) 2023-08-09T13:50:53.539 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:53.533+0000 7f6ca5848640 1 -- 172.21.15.43:0/3042488415 learned_addr learned my addr 172.21.15.43:0/3042488415 (peer_addr_for_me v1:172.21.15.43:0/0) 2023-08-09T13:50:53.539 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:53.534+0000 7f6ca6049640 1 --2- 172.21.15.43:0/3042488415 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f6ca015c550 0x7f6ca00cd6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:50:53.539 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:53.534+0000 7f6ca684a640 1 --2- 172.21.15.43:0/3042488415 >> v2:172.21.15.203:3300/0 conn(0x7f6ca00b45f0 0x7f6ca00b59d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:50:53.539 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:53.534+0000 7f6ca5047640 1 -- 172.21.15.43:0/3042488415 <== mon.0 v1:172.21.15.43:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3419530987 0 0) 0x7f6ca00b7bd0 con 0x7f6ca00d04b0 2023-08-09T13:50:53.539 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:53.534+0000 7f6ca5047640 1 -- 172.21.15.43:0/3042488415 --> v1:172.21.15.43:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f6c8c003520 con 0x7f6ca00d04b0 2023-08-09T13:50:53.539 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:53.534+0000 7f6ca5047640 1 -- 172.21.15.43:0/3042488415 <== mon.0 v1:172.21.15.43:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2896270962 0 0) 0x7f6c8c003520 con 0x7f6ca00d04b0 2023-08-09T13:50:53.540 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:53.534+0000 7f6ca5047640 1 -- 172.21.15.43:0/3042488415 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f6ca015c550 msgr2=0x7f6ca00cd6d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-09T13:50:53.540 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:53.534+0000 7f6ca5047640 1 --2- 172.21.15.43:0/3042488415 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f6ca015c550 0x7f6ca00cd6d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:50:53.540 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:53.534+0000 7f6ca5047640 1 -- 172.21.15.43:0/3042488415 >> v2:172.21.15.203:3300/0 conn(0x7f6ca00b45f0 msgr2=0x7f6ca00b59d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-09T13:50:53.540 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:53.534+0000 7f6ca5047640 1 --2- 172.21.15.43:0/3042488415 >> v2:172.21.15.203:3300/0 conn(0x7f6ca00b45f0 0x7f6ca00b59d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:50:53.540 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:53.534+0000 7f6ca5047640 1 -- 172.21.15.43:0/3042488415 --> v1:172.21.15.43:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6ca00b54f0 con 0x7f6ca00d04b0 2023-08-09T13:50:53.540 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:53.534+0000 7f6ca5047640 1 -- 172.21.15.43:0/3042488415 <== mon.0 v1:172.21.15.43:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1285150740 0 0) 0x7f6c94001670 con 0x7f6ca00d04b0 2023-08-09T13:50:53.540 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:53.535+0000 7f6ca5047640 1 -- 172.21.15.43:0/3042488415 >> v1:172.21.15.43:6789/0 conn(0x7f6ca00d04b0 legacy=0x7f6ca00b3ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:50:53.541 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:53.535+0000 7f6ca5047640 1 --2- 172.21.15.43:0/3042488415 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f6ca015c550 0x7f6c8c003970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:50:53.541 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:53.535+0000 7f6ca5047640 1 --2- 172.21.15.43:0/3042488415 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f6c8c003ef0 0x7f6c8c0047c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:50:53.541 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:53.535+0000 7f6ca5047640 1 --2- 172.21.15.43:0/3042488415 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f6c8c004d00 0x7f6c8c005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:50:53.541 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:53.535+0000 7f6ca5047640 1 -- 172.21.15.43:0/3042488415 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6ca00d1c00 con 0x7f6c8c003ef0 2023-08-09T13:50:53.541 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:53.535+0000 7f6ca5047640 1 -- 172.21.15.43:0/3042488415 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6ca015c930 con 0x7f6c8c004d00 2023-08-09T13:50:53.541 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:53.535+0000 7f6ca5047640 1 -- 172.21.15.43:0/3042488415 --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6c8c00b980 con 0x7f6ca015c550 2023-08-09T13:50:53.541 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:53.535+0000 7f6ca5848640 1 --2- 172.21.15.43:0/3042488415 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f6c8c003ef0 0x7f6c8c0047c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:50:53.541 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:53.535+0000 7f6ca6049640 1 --2- 172.21.15.43:0/3042488415 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f6ca015c550 0x7f6c8c003970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:50:53.542 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:53.535+0000 7f6ca684a640 1 --2- 172.21.15.43:0/3042488415 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f6c8c004d00 0x7f6c8c005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:50:53.542 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:53.535+0000 7f6ca5848640 1 -- 172.21.15.43:0/3042488415 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f6ca015c550 msgr2=0x7f6c8c003970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-09T13:50:53.542 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:53.535+0000 7f6ca5848640 1 --2- 172.21.15.43:0/3042488415 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f6ca015c550 0x7f6c8c003970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:50:53.542 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:53.535+0000 7f6ca5848640 1 -- 172.21.15.43:0/3042488415 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f6c8c004d00 msgr2=0x7f6c8c005180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-09T13:50:53.543 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:53.535+0000 7f6ca5848640 1 --2- 172.21.15.43:0/3042488415 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f6c8c004d00 0x7f6c8c005180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:50:53.543 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:53.535+0000 7f6ca5848640 1 -- 172.21.15.43:0/3042488415 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6c8c003520 con 0x7f6c8c003ef0 2023-08-09T13:50:53.548 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:53.537+0000 7f6ca5848640 1 --2- 172.21.15.43:0/3042488415 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f6c8c003ef0 0x7f6c8c0047c0 secure :-1 s=READY pgs=1078 cs=0 l=1 rev1=1 crypto rx=0x7f6c9400c240 tx=0x7f6c8c0057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8d0ed49ddd18abe8 server_cookie=0 in_seq=0 out_seq=0 2023-08-09T13:50:53.549 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:53.537+0000 7f6ca5047640 1 -- 172.21.15.43:0/3042488415 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6c94005ad0 con 0x7f6c8c003ef0 2023-08-09T13:50:53.549 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:53.537+0000 7f6ca5047640 1 -- 172.21.15.43:0/3042488415 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6c94005c30 con 0x7f6c8c003ef0 2023-08-09T13:50:53.550 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:53.537+0000 7f6ca5047640 1 -- 172.21.15.43:0/3042488415 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6c9402c8c0 con 0x7f6c8c003ef0 2023-08-09T13:50:53.550 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:53.538+0000 7f6ca82ad640 1 -- 172.21.15.43:0/3042488415 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f6c8c003ef0 msgr2=0x7f6c8c0047c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:50:53.550 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:53.538+0000 7f6ca82ad640 1 --2- 172.21.15.43:0/3042488415 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f6c8c003ef0 0x7f6c8c0047c0 secure :-1 s=READY pgs=1078 cs=0 l=1 rev1=1 crypto rx=0x7f6c9400c240 tx=0x7f6c8c0057f0 comp rx=0 tx=0).stop 2023-08-09T13:50:53.550 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:53.538+0000 7f6ca684a640 1 -- 172.21.15.43:0/3042488415 reap_dead start 2023-08-09T13:50:53.551 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:53.538+0000 7f6ca82ad640 1 -- 172.21.15.43:0/3042488415 shutdown_connections 2023-08-09T13:50:53.551 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:53.538+0000 7f6ca82ad640 1 -- 172.21.15.43:0/3042488415 >> 172.21.15.43:0/3042488415 conn(0x7f6ca0071520 msgr2=0x7f6ca00ceee0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-09T13:50:53.551 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:53.538+0000 7f6ca82ad640 1 -- 172.21.15.43:0/3042488415 shutdown_connections 2023-08-09T13:50:53.551 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:53.538+0000 7f6ca82ad640 1 -- 172.21.15.43:0/3042488415 wait complete. 2023-08-09T13:50:53.551 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:53.540+0000 7f6ca82ad640 1 Processor -- start 2023-08-09T13:50:53.552 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:53.540+0000 7f6ca82ad640 1 -- start start 2023-08-09T13:50:53.552 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:53.541+0000 7f6ca82ad640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f6c8c003ef0 0x7f6ca01f5980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:50:53.552 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:53.541+0000 7f6ca82ad640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f6ca00c9f40 0x7f6ca01f1290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:50:53.553 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:53.541+0000 7f6ca82ad640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f6ca00cdd20 0x7f6ca015a4b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:50:53.553 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:53.541+0000 7f6ca82ad640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6ca01f5ec0 con 0x7f6ca00cdd20 2023-08-09T13:50:53.553 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:53.541+0000 7f6ca82ad640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6ca01f1bb0 con 0x7f6ca00c9f40 2023-08-09T13:50:53.553 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:53.541+0000 7f6ca82ad640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6ca01f1e90 con 0x7f6c8c003ef0 2023-08-09T13:50:53.553 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:53.541+0000 7f6ca684a640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f6ca00cdd20 0x7f6ca015a4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:50:53.554 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:53.541+0000 7f6ca684a640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f6ca00cdd20 0x7f6ca015a4b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:52128/0 (socket says 172.21.15.43:52128) 2023-08-09T13:50:53.554 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:53.541+0000 7f6ca684a640 1 -- 172.21.15.43:0/3820528936 learned_addr learned my addr 172.21.15.43:0/3820528936 (peer_addr_for_me v2:172.21.15.43:0/0) 2023-08-09T13:50:53.554 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:53.541+0000 7f6ca5848640 1 --2- 172.21.15.43:0/3820528936 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f6ca00c9f40 0x7f6ca01f1290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:50:53.554 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:53.541+0000 7f6ca6049640 1 --2- 172.21.15.43:0/3820528936 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f6c8c003ef0 0x7f6ca01f5980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:50:53.554 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:53.541+0000 7f6ca684a640 1 -- 172.21.15.43:0/3820528936 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f6c8c003ef0 msgr2=0x7f6ca01f5980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:50:53.554 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:53.541+0000 7f6ca684a640 1 --2- 172.21.15.43:0/3820528936 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f6c8c003ef0 0x7f6ca01f5980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:50:53.554 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:53.542+0000 7f6ca684a640 1 -- 172.21.15.43:0/3820528936 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f6ca00c9f40 msgr2=0x7f6ca01f1290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:50:53.555 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:53.542+0000 7f6ca684a640 1 --2- 172.21.15.43:0/3820528936 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f6ca00c9f40 0x7f6ca01f1290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:50:53.555 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:53.542+0000 7f6ca684a640 1 -- 172.21.15.43:0/3820528936 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6ca01f2170 con 0x7f6ca00cdd20 2023-08-09T13:50:53.555 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:53.542+0000 7f6ca684a640 1 --2- 172.21.15.43:0/3820528936 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f6ca00cdd20 0x7f6ca015a4b0 secure :-1 s=READY pgs=1079 cs=0 l=1 rev1=1 crypto rx=0x7f6ca00ce4e0 tx=0x7f6c980115a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-09T13:50:53.555 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:53.542+0000 7f6c92ffd640 1 -- 172.21.15.43:0/3820528936 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6c9800e070 con 0x7f6ca00cdd20 2023-08-09T13:50:53.555 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:53.542+0000 7f6c92ffd640 1 -- 172.21.15.43:0/3820528936 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6c98011e10 con 0x7f6ca00cdd20 2023-08-09T13:50:53.555 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:53.542+0000 7f6ca82ad640 1 -- 172.21.15.43:0/3820528936 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6ca01f2450 con 0x7f6ca00cdd20 2023-08-09T13:50:53.555 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:53.542+0000 7f6c92ffd640 1 -- 172.21.15.43:0/3820528936 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6c98003990 con 0x7f6ca00cdd20 2023-08-09T13:50:53.555 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:53.542+0000 7f6ca82ad640 1 -- 172.21.15.43:0/3820528936 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6ca01f17d0 con 0x7f6ca00cdd20 2023-08-09T13:50:53.556 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:53.544+0000 7f6ca82ad640 1 -- 172.21.15.43:0/3820528936 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6ca00739f0 con 0x7f6ca00cdd20 2023-08-09T13:50:53.556 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:53.544+0000 7f6c92ffd640 1 -- 172.21.15.43:0/3820528936 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 33) v1 ==== 88444+0+0 (secure 0 0 0) 0x7f6c98003af0 con 0x7f6ca00cdd20 2023-08-09T13:50:53.556 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:53.545+0000 7f6c92ffd640 1 --2- 172.21.15.43:0/3820528936 >> [v2:172.21.15.203:6832/3851880723,v1:172.21.15.203:6833/3851880723] conn(0x7f6c8406a170 0x7f6c8406c630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:50:53.556 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:53.546+0000 7f6c92ffd640 1 -- 172.21.15.43:0/3820528936 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(91..91 src has 1..91) v4 ==== 6770+0+0 (secure 0 0 0) 0x7f6c980891b0 con 0x7f6ca00cdd20 2023-08-09T13:50:53.556 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:53.546+0000 7f6ca6049640 1 -- 172.21.15.43:0/3820528936 >> [v2:172.21.15.203:6832/3851880723,v1:172.21.15.203:6833/3851880723] conn(0x7f6c8406a170 msgr2=0x7f6c8406c630 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.203:6832/3851880723 2023-08-09T13:50:53.556 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:53.546+0000 7f6ca6049640 1 --2- 172.21.15.43:0/3820528936 >> [v2:172.21.15.203:6832/3851880723,v1:172.21.15.203:6833/3851880723] conn(0x7f6c8406a170 0x7f6c8406c630 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-08-09T13:50:53.557 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:53.553+0000 7f6c92ffd640 1 -- 172.21.15.43:0/3820528936 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178066 (secure 0 0 0) 0x7f6c980543a0 con 0x7f6ca00cdd20 2023-08-09T13:50:53.563 INFO:tasks.ceph.mgr.x.smithi203.stderr:2023-08-09T13:50:53.560+0000 7f4b1542c040 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2023-08-09T13:50:53.664 INFO:tasks.ceph.mgr.y.smithi043.stderr:2023-08-09T13:50:53.660+0000 7fe84c03a040 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2023-08-09T13:50:53.670 INFO:tasks.ceph.mgr.x.smithi203.stderr:2023-08-09T13:50:53.667+0000 7f4b1542c040 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2023-08-09T13:50:53.719 INFO:tasks.ceph.mgr.x.smithi203.stderr:2023-08-09T13:50:53.716+0000 7f4b1542c040 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2023-08-09T13:50:53.747 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:53.744+0000 7f6ca82ad640 1 -- 172.21.15.43:0/3820528936 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f6ca01f2970 con 0x7f6ca00cdd20 2023-08-09T13:50:53.747 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:53.745+0000 7f6c92ffd640 1 -- 172.21.15.43:0/3820528936 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v91) v1 ==== 100+0+11 (secure 0 0 0) 0x7f6c980579c0 con 0x7f6ca00cdd20 2023-08-09T13:50:53.748 INFO:teuthology.orchestra.run.smithi043.stdout:pg_num: 64 2023-08-09T13:50:53.749 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:53.747+0000 7f6ca6049640 1 -- 172.21.15.43:0/3820528936 >> [v2:172.21.15.203:6832/3851880723,v1:172.21.15.203:6833/3851880723] conn(0x7f6c8406a170 msgr2=0x7f6c8406c630 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.203:6832/3851880723 2023-08-09T13:50:53.750 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:53.747+0000 7f6ca6049640 1 --2- 172.21.15.43:0/3820528936 >> [v2:172.21.15.203:6832/3851880723,v1:172.21.15.203:6833/3851880723] conn(0x7f6c8406a170 0x7f6c8406c630 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2023-08-09T13:50:53.750 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:53.747+0000 7f6ca82ad640 1 -- 172.21.15.43:0/3820528936 >> [v2:172.21.15.203:6832/3851880723,v1:172.21.15.203:6833/3851880723] conn(0x7f6c8406a170 msgr2=0x7f6c8406c630 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-08-09T13:50:53.751 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:53.747+0000 7f6ca82ad640 1 --2- 172.21.15.43:0/3820528936 >> [v2:172.21.15.203:6832/3851880723,v1:172.21.15.203:6833/3851880723] conn(0x7f6c8406a170 0x7f6c8406c630 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:50:53.751 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:53.747+0000 7f6ca82ad640 1 -- 172.21.15.43:0/3820528936 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f6ca00cdd20 msgr2=0x7f6ca015a4b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:50:53.751 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:53.747+0000 7f6ca82ad640 1 --2- 172.21.15.43:0/3820528936 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f6ca00cdd20 0x7f6ca015a4b0 secure :-1 s=READY pgs=1079 cs=0 l=1 rev1=1 crypto rx=0x7f6ca00ce4e0 tx=0x7f6c980115a0 comp rx=0 tx=0).stop 2023-08-09T13:50:53.751 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:53.748+0000 7f6ca82ad640 1 -- 172.21.15.43:0/3820528936 shutdown_connections 2023-08-09T13:50:53.751 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:53.748+0000 7f6ca82ad640 1 --2- 172.21.15.43:0/3820528936 >> [v2:172.21.15.203:6832/3851880723,v1:172.21.15.203:6833/3851880723] conn(0x7f6c8406a170 0x7f6c8406c630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:50:53.751 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:53.748+0000 7f6ca82ad640 1 --2- 172.21.15.43:0/3820528936 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f6ca00cdd20 0x7f6ca015a4b0 unknown :-1 s=CLOSED pgs=1079 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:50:53.751 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:53.748+0000 7f6ca82ad640 1 --2- 172.21.15.43:0/3820528936 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f6ca00c9f40 0x7f6ca01f1290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:50:53.752 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:53.748+0000 7f6ca82ad640 1 --2- 172.21.15.43:0/3820528936 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f6c8c003ef0 0x7f6ca01f5980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:50:53.752 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:53.748+0000 7f6ca82ad640 1 -- 172.21.15.43:0/3820528936 >> 172.21.15.43:0/3820528936 conn(0x7f6ca0071520 msgr2=0x7f6ca015bbb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-09T13:50:53.752 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:53.748+0000 7f6ca82ad640 1 -- 172.21.15.43:0/3820528936 shutdown_connections 2023-08-09T13:50:53.752 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:53.748+0000 7f6ca82ad640 1 -- 172.21.15.43:0/3820528936 wait complete. 2023-08-09T13:50:53.770 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2023-08-09T13:50:53.778 INFO:tasks.ceph.mgr.y.smithi043.stderr:2023-08-09T13:50:53.776+0000 7fe84c03a040 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2023-08-09T13:50:53.832 INFO:tasks.ceph.mgr.y.smithi043.stderr:2023-08-09T13:50:53.829+0000 7fe84c03a040 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2023-08-09T13:50:53.878 INFO:tasks.ceph.mgr.x.smithi203.stderr:2023-08-09T13:50:53.875+0000 7f4b1542c040 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2023-08-09T13:50:53.993 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:53.990+0000 7f5fe5b0f640 1 Processor -- start 2023-08-09T13:50:53.993 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:53.990+0000 7f5fe5b0f640 1 -- start start 2023-08-09T13:50:53.994 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:53.991+0000 7f5fe5b0f640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f5fe00cacb0 0x7f5fe0160d50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:50:53.994 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:53.992+0000 7f5fe5b0f640 1 -- --> v1:172.21.15.43:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5fe006e080 con 0x7f5fe006d5c0 2023-08-09T13:50:53.994 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:53.992+0000 7f5fe5b0f640 1 -- --> v1:172.21.15.203:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5fe006e360 con 0x7f5fe0069ed0 2023-08-09T13:50:53.994 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:53.992+0000 7f5fe5b0f640 1 -- --> v2:172.21.15.43:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5fe0160270 con 0x7f5fe00cacb0 2023-08-09T13:50:53.995 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:53.992+0000 7f5fdffff640 1 --1- >> v1:172.21.15.43:6789/0 conn(0x7f5fe006d5c0 0x7f5fe00697c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.43:6789/0 says I am v1:172.21.15.43:59930/0 (socket says 172.21.15.43:59930) 2023-08-09T13:50:53.995 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:53.992+0000 7f5fdffff640 1 -- 172.21.15.43:0/3229371252 learned_addr learned my addr 172.21.15.43:0/3229371252 (peer_addr_for_me v1:172.21.15.43:0/0) 2023-08-09T13:50:53.995 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:53.992+0000 7f5fe4b0d640 1 --2- 172.21.15.43:0/3229371252 >> v2:172.21.15.43:3300/0 conn(0x7f5fe00cacb0 0x7f5fe0160d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:50:53.996 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:53.992+0000 7f5fe4b0d640 1 -- 172.21.15.43:0/3229371252 >> v1:172.21.15.203:6789/0 conn(0x7f5fe0069ed0 legacy=0x7f5fe0070590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-09T13:50:53.996 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:53.992+0000 7f5fe4b0d640 1 -- 172.21.15.43:0/3229371252 >> v1:172.21.15.43:6789/0 conn(0x7f5fe006d5c0 legacy=0x7f5fe00697c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:50:53.996 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:53.992+0000 7f5fe4b0d640 1 -- 172.21.15.43:0/3229371252 --> v2:172.21.15.43:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5fe0160550 con 0x7f5fe00cacb0 2023-08-09T13:50:53.996 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:53.994+0000 7f5fe4b0d640 1 --2- 172.21.15.43:0/3229371252 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f5fe00cacb0 0x7f5fe0160d50 secure :-1 s=READY pgs=1081 cs=0 l=1 rev1=1 crypto rx=0x7f5fd40097a0 tx=0x7f5fd402b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6a25f4529b008c79 server_cookie=0 in_seq=0 out_seq=0 2023-08-09T13:50:53.997 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:53.994+0000 7f5fdf7fe640 1 -- 172.21.15.43:0/3229371252 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5fd4031830 con 0x7f5fe00cacb0 2023-08-09T13:50:53.997 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:53.994+0000 7f5fdf7fe640 1 -- 172.21.15.43:0/3229371252 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5fd4031990 con 0x7f5fe00cacb0 2023-08-09T13:50:53.997 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:53.994+0000 7f5fdf7fe640 1 -- 172.21.15.43:0/3229371252 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5fd4031d20 con 0x7f5fe00cacb0 2023-08-09T13:50:53.997 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:53.994+0000 7f5fe5b0f640 1 -- 172.21.15.43:0/3229371252 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f5fe00cacb0 msgr2=0x7f5fe0160d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:50:53.997 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:53.994+0000 7f5fe5b0f640 1 --2- 172.21.15.43:0/3229371252 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f5fe00cacb0 0x7f5fe0160d50 secure :-1 s=READY pgs=1081 cs=0 l=1 rev1=1 crypto rx=0x7f5fd40097a0 tx=0x7f5fd402b8d0 comp rx=0 tx=0).stop 2023-08-09T13:50:53.998 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:53.995+0000 7f5fe5b0f640 1 -- 172.21.15.43:0/3229371252 shutdown_connections 2023-08-09T13:50:53.998 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:53.995+0000 7f5fe5b0f640 1 --2- 172.21.15.43:0/3229371252 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f5fe00cacb0 0x7f5fe0160d50 unknown :-1 s=CLOSED pgs=1081 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:50:53.998 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:53.995+0000 7f5fe5b0f640 1 -- 172.21.15.43:0/3229371252 >> 172.21.15.43:0/3229371252 conn(0x7f5fe00cf000 msgr2=0x7f5fe00cf410 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-09T13:50:53.998 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:53.995+0000 7f5fe5b0f640 1 -- 172.21.15.43:0/3229371252 shutdown_connections 2023-08-09T13:50:53.999 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:53.995+0000 7f5fe5b0f640 1 -- 172.21.15.43:0/3229371252 wait complete. 2023-08-09T13:50:53.999 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:53.996+0000 7f5fe5b0f640 1 Processor -- start 2023-08-09T13:50:53.999 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:53.996+0000 7f5fe5b0f640 1 -- start start 2023-08-09T13:50:54.000 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:53.997+0000 7f5fe5b0f640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f5fe0069ed0 0x7f5fe00af230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:50:54.001 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:53.997+0000 7f5fe5b0f640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f5fe006d5c0 0x7f5fe00ad760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:50:54.001 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:53.997+0000 7f5fe5b0f640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f5fe00cacb0 0x7f5fe01e7e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:50:54.001 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:53.997+0000 7f5fe5b0f640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5fe014c390 con 0x7f5fe0069ed0 2023-08-09T13:50:54.001 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:53.997+0000 7f5fe5b0f640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5fe014cf10 con 0x7f5fe00cacb0 2023-08-09T13:50:54.001 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:53.997+0000 7f5fe5b0f640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5fe014c5e0 con 0x7f5fe006d5c0 2023-08-09T13:50:54.001 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:53.998+0000 7f5fe4b0d640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f5fe0069ed0 0x7f5fe00af230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:50:54.001 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:53.998+0000 7f5fe4b0d640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f5fe0069ed0 0x7f5fe00af230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:52144/0 (socket says 172.21.15.43:52144) 2023-08-09T13:50:54.001 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:53.998+0000 7f5fe4b0d640 1 -- 172.21.15.43:0/2953325184 learned_addr learned my addr 172.21.15.43:0/2953325184 (peer_addr_for_me v2:172.21.15.43:0/0) 2023-08-09T13:50:54.002 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:53.998+0000 7f5fdffff640 1 --2- 172.21.15.43:0/2953325184 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f5fe006d5c0 0x7f5fe00ad760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:50:54.002 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:53.998+0000 7f5fe530e640 1 --2- 172.21.15.43:0/2953325184 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f5fe00cacb0 0x7f5fe01e7e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:50:54.002 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:53.998+0000 7f5fe4b0d640 1 -- 172.21.15.43:0/2953325184 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f5fe006d5c0 msgr2=0x7f5fe00ad760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:50:54.002 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:53.998+0000 7f5fe4b0d640 1 --2- 172.21.15.43:0/2953325184 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f5fe006d5c0 0x7f5fe00ad760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:50:54.002 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:53.998+0000 7f5fe4b0d640 1 -- 172.21.15.43:0/2953325184 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f5fe00cacb0 msgr2=0x7f5fe01e7e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:50:54.002 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:53.998+0000 7f5fe4b0d640 1 --2- 172.21.15.43:0/2953325184 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f5fe00cacb0 0x7f5fe01e7e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:50:54.003 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:53.998+0000 7f5fe4b0d640 1 -- 172.21.15.43:0/2953325184 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5fd4009400 con 0x7f5fe0069ed0 2023-08-09T13:50:54.003 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:53.998+0000 7f5fe4b0d640 1 --2- 172.21.15.43:0/2953325184 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f5fe0069ed0 0x7f5fe00af230 secure :-1 s=READY pgs=1082 cs=0 l=1 rev1=1 crypto rx=0x7f5fd40098c0 tx=0x7f5fd40046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-09T13:50:54.004 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:53.999+0000 7f5fdd7fa640 1 -- 172.21.15.43:0/2953325184 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5fd403aa60 con 0x7f5fe0069ed0 2023-08-09T13:50:54.004 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:53.999+0000 7f5fdd7fa640 1 -- 172.21.15.43:0/2953325184 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5fd403abc0 con 0x7f5fe0069ed0 2023-08-09T13:50:54.004 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:53.999+0000 7f5fdd7fa640 1 -- 172.21.15.43:0/2953325184 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5fd40423f0 con 0x7f5fe0069ed0 2023-08-09T13:50:54.004 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:53.999+0000 7f5fe5b0f640 1 -- 172.21.15.43:0/2953325184 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5fe014c8c0 con 0x7f5fe0069ed0 2023-08-09T13:50:54.004 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:53.999+0000 7f5fe5b0f640 1 -- 172.21.15.43:0/2953325184 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5fe00adca0 con 0x7f5fe0069ed0 2023-08-09T13:50:54.004 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:54.000+0000 7f5fe5b0f640 1 -- 172.21.15.43:0/2953325184 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5fe0071e40 con 0x7f5fe0069ed0 2023-08-09T13:50:54.005 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:54.002+0000 7f5fdd7fa640 1 -- 172.21.15.43:0/2953325184 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 33) v1 ==== 88444+0+0 (secure 0 0 0) 0x7f5fd4030380 con 0x7f5fe0069ed0 2023-08-09T13:50:54.006 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:54.003+0000 7f5fdd7fa640 1 --2- 172.21.15.43:0/2953325184 >> [v2:172.21.15.203:6832/3851880723,v1:172.21.15.203:6833/3851880723] conn(0x7f5fc406a1a0 0x7f5fc406c660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:50:54.006 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:54.003+0000 7f5fdd7fa640 1 -- 172.21.15.43:0/2953325184 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(91..91 src has 1..91) v4 ==== 6770+0+0 (secure 0 0 0) 0x7f5fd40aa6c0 con 0x7f5fe0069ed0 2023-08-09T13:50:54.006 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:54.003+0000 7f5fdffff640 1 -- 172.21.15.43:0/2953325184 >> [v2:172.21.15.203:6832/3851880723,v1:172.21.15.203:6833/3851880723] conn(0x7f5fc406a1a0 msgr2=0x7f5fc406c660 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.203:6832/3851880723 2023-08-09T13:50:54.006 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:54.003+0000 7f5fdffff640 1 --2- 172.21.15.43:0/2953325184 >> [v2:172.21.15.203:6832/3851880723,v1:172.21.15.203:6833/3851880723] conn(0x7f5fc406a1a0 0x7f5fc406c660 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-08-09T13:50:54.006 INFO:tasks.ceph.mgr.y.smithi043.stderr:2023-08-09T13:50:54.004+0000 7fe84c03a040 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2023-08-09T13:50:54.014 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:54.011+0000 7f5fdd7fa640 1 -- 172.21.15.43:0/2953325184 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178066 (secure 0 0 0) 0x7f5fd40759d0 con 0x7f5fe0069ed0 2023-08-09T13:50:54.207 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:54.204+0000 7f5fdffff640 1 -- 172.21.15.43:0/2953325184 >> [v2:172.21.15.203:6832/3851880723,v1:172.21.15.203:6833/3851880723] conn(0x7f5fc406a1a0 msgr2=0x7f5fc406c660 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.203:6832/3851880723 2023-08-09T13:50:54.207 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:54.204+0000 7f5fdffff640 1 --2- 172.21.15.43:0/2953325184 >> [v2:172.21.15.203:6832/3851880723,v1:172.21.15.203:6833/3851880723] conn(0x7f5fc406a1a0 0x7f5fc406c660 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2023-08-09T13:50:54.220 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:54.218+0000 7f5fe5b0f640 1 -- 172.21.15.43:0/2953325184 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7f5fe00739f0 con 0x7f5fe0069ed0 2023-08-09T13:50:54.221 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:54.219+0000 7f5fdd7fa640 1 -- 172.21.15.43:0/2953325184 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 8 entries v91) v1 ==== 80+0+478 (secure 0 0 0) 0x7f5fd4078ff0 con 0x7f5fe0069ed0 2023-08-09T13:50:54.221 INFO:teuthology.orchestra.run.smithi043.stdout:172.21.15.203:6833/3325141227 2023-08-10T13:50:49.168196+0000 2023-08-09T13:50:54.222 INFO:teuthology.orchestra.run.smithi043.stdout:172.21.15.203:6832/3325141227 2023-08-10T13:50:49.168196+0000 2023-08-09T13:50:54.222 INFO:teuthology.orchestra.run.smithi043.stdout:172.21.15.203:0/3732830113 2023-08-10T13:50:49.168196+0000 2023-08-09T13:50:54.222 INFO:teuthology.orchestra.run.smithi043.stdout:172.21.15.203:0/2753554568 2023-08-10T13:50:49.168196+0000 2023-08-09T13:50:54.222 INFO:teuthology.orchestra.run.smithi043.stdout:172.21.15.203:0/2468196984 2023-08-10T13:50:49.168196+0000 2023-08-09T13:50:54.222 INFO:teuthology.orchestra.run.smithi043.stdout:172.21.15.203:0/3375551329 2023-08-10T13:50:49.168196+0000 2023-08-09T13:50:54.222 INFO:teuthology.orchestra.run.smithi043.stdout:172.21.15.203:0/1259705306 2023-08-10T13:50:49.168196+0000 2023-08-09T13:50:54.222 INFO:teuthology.orchestra.run.smithi043.stdout:172.21.15.203:0/1322546009 2023-08-10T13:50:49.168196+0000 2023-08-09T13:50:54.223 INFO:teuthology.orchestra.run.smithi043.stderr:listed 8 entries 2023-08-09T13:50:54.224 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:54.221+0000 7f5fe5b0f640 1 -- 172.21.15.43:0/2953325184 >> [v2:172.21.15.203:6832/3851880723,v1:172.21.15.203:6833/3851880723] conn(0x7f5fc406a1a0 msgr2=0x7f5fc406c660 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-08-09T13:50:54.224 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:54.221+0000 7f5fe5b0f640 1 --2- 172.21.15.43:0/2953325184 >> [v2:172.21.15.203:6832/3851880723,v1:172.21.15.203:6833/3851880723] conn(0x7f5fc406a1a0 0x7f5fc406c660 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:50:54.224 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:54.221+0000 7f5fe5b0f640 1 -- 172.21.15.43:0/2953325184 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f5fe0069ed0 msgr2=0x7f5fe00af230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:50:54.224 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:54.221+0000 7f5fe5b0f640 1 --2- 172.21.15.43:0/2953325184 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f5fe0069ed0 0x7f5fe00af230 secure :-1 s=READY pgs=1082 cs=0 l=1 rev1=1 crypto rx=0x7f5fd40098c0 tx=0x7f5fd40046c0 comp rx=0 tx=0).stop 2023-08-09T13:50:54.224 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:54.221+0000 7f5fe5b0f640 1 -- 172.21.15.43:0/2953325184 shutdown_connections 2023-08-09T13:50:54.225 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:54.221+0000 7f5fe5b0f640 1 --2- 172.21.15.43:0/2953325184 >> [v2:172.21.15.203:6832/3851880723,v1:172.21.15.203:6833/3851880723] conn(0x7f5fc406a1a0 0x7f5fc406c660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:50:54.225 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:54.221+0000 7f5fe5b0f640 1 --2- 172.21.15.43:0/2953325184 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f5fe00cacb0 0x7f5fe01e7e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:50:54.225 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:54.221+0000 7f5fe5b0f640 1 --2- 172.21.15.43:0/2953325184 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f5fe006d5c0 0x7f5fe00ad760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:50:54.225 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:54.221+0000 7f5fe5b0f640 1 --2- 172.21.15.43:0/2953325184 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f5fe0069ed0 0x7f5fe00af230 unknown :-1 s=CLOSED pgs=1082 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:50:54.225 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:54.221+0000 7f5fe5b0f640 1 -- 172.21.15.43:0/2953325184 >> 172.21.15.43:0/2953325184 conn(0x7f5fe00cf000 msgr2=0x7f5fe006cba0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-09T13:50:54.225 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:54.221+0000 7f5fe5b0f640 1 -- 172.21.15.43:0/2953325184 shutdown_connections 2023-08-09T13:50:54.225 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:54.221+0000 7f5fe5b0f640 1 -- 172.21.15.43:0/2953325184 wait complete. 2023-08-09T13:50:54.241 INFO:tasks.cephfs.fuse_mount:Running fusermount -u on ubuntu@smithi043.front.sepia.ceph.com... 2023-08-09T13:50:54.242 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-09T13:50:54.242 DEBUG:teuthology.orchestra.run.smithi043:> sudo fusermount -u /home/ubuntu/cephtest/mnt.0 2023-08-09T13:50:54.259 INFO:tasks.ceph.mgr.x.smithi203.stderr:2023-08-09T13:50:54.256+0000 7f4b1542c040 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2023-08-09T13:50:54.333 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi043.stderr:ceph-fuse[118491]: fuse finished with error 0 and tester_r 0 2023-08-09T13:50:54.339 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi043.front.sepia.ceph.com 2023-08-09T13:50:54.340 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-09T13:50:54.340 DEBUG:teuthology.orchestra.run.smithi043:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2023-08-09T13:50:54.427 INFO:tasks.ceph.mgr.y.smithi043.stderr:2023-08-09T13:50:54.424+0000 7fe84c03a040 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2023-08-09T13:50:54.488 INFO:tasks.ceph.mgr.x.smithi203.stderr:2023-08-09T13:50:54.486+0000 7f4b1542c040 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2023-08-09T13:50:54.511 DEBUG:teuthology.orchestra.run.smithi203:> set -ex 2023-08-09T13:50:54.512 DEBUG:teuthology.orchestra.run.smithi203:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-09T13:50:54.547 INFO:tasks.ceph.mgr.x.smithi203.stderr:2023-08-09T13:50:54.545+0000 7f4b1542c040 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2023-08-09T13:50:54.567 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4053648,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497576k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=27,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248784k,nr_inodes=812196,mode=700,uid=1000,gid=1275 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497576k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2023-08-09T13:50:54.568 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi203.front.sepia.ceph.com 2023-08-09T13:50:54.568 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-09T13:50:54.568 DEBUG:teuthology.orchestra.run.smithi203:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.1) 2023-08-09T13:50:54.599 INFO:tasks.ceph.mgr.x.smithi203.stderr:2023-08-09T13:50:54.597+0000 7f4b1542c040 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2023-08-09T13:50:54.652 INFO:teuthology.orchestra.run.smithi203.stderr:rmdir: failed to remove '/home/ubuntu/cephtest/mnt.1': No such file or directory 2023-08-09T13:50:54.653 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-08-09T13:50:54.654 DEBUG:teuthology.orchestra.run.smithi043:> set -ex 2023-08-09T13:50:54.654 DEBUG:teuthology.orchestra.run.smithi043:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-09T13:50:54.656 INFO:tasks.ceph.mgr.x.smithi203.stderr:2023-08-09T13:50:54.654+0000 7f4b1542c040 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2023-08-09T13:50:54.664 INFO:tasks.ceph.mgr.y.smithi043.stderr:2023-08-09T13:50:54.657+0000 7fe84c03a040 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2023-08-09T13:50:54.713 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4054441,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6498844k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=27,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3249420k,nr_inodes=812355,mode=700,uid=1000,gid=1275 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6498844k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2023-08-09T13:50:54.713 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi043.front.sepia.ceph.com 2023-08-09T13:50:54.713 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-09T13:50:54.714 DEBUG:teuthology.orchestra.run.smithi043:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.2) 2023-08-09T13:50:54.721 INFO:tasks.ceph.mgr.y.smithi043.stderr:2023-08-09T13:50:54.718+0000 7fe84c03a040 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2023-08-09T13:50:54.766 INFO:teuthology.orchestra.run.smithi043.stderr:rmdir: failed to remove '/home/ubuntu/cephtest/mnt.2': No such file or directory 2023-08-09T13:50:54.767 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-08-09T13:50:54.768 DEBUG:teuthology.orchestra.run.smithi203:> set -ex 2023-08-09T13:50:54.768 DEBUG:teuthology.orchestra.run.smithi203:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-09T13:50:54.774 INFO:tasks.ceph.mgr.y.smithi043.stderr:2023-08-09T13:50:54.772+0000 7fe84c03a040 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2023-08-09T13:50:54.824 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4053648,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497576k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=27,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248784k,nr_inodes=812196,mode=700,uid=1000,gid=1275 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497576k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2023-08-09T13:50:54.824 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi203.front.sepia.ceph.com 2023-08-09T13:50:54.825 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-09T13:50:54.825 DEBUG:teuthology.orchestra.run.smithi203:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.3) 2023-08-09T13:50:54.830 INFO:tasks.ceph.mgr.y.smithi043.stderr:2023-08-09T13:50:54.827+0000 7fe84c03a040 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2023-08-09T13:50:54.875 INFO:teuthology.orchestra.run.smithi203.stderr:rmdir: failed to remove '/home/ubuntu/cephtest/mnt.3': No such file or directory 2023-08-09T13:50:54.876 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-08-09T13:50:54.877 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-08-09T13:50:54.962 INFO:tasks.ceph.mgr.x.smithi203.stderr:2023-08-09T13:50:54.959+0000 7f4b1542c040 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2023-08-09T13:50:55.019 INFO:tasks.ceph.mgr.x.smithi203.stderr:2023-08-09T13:50:55.017+0000 7f4b1542c040 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2023-08-09T13:50:55.071 INFO:tasks.ceph.mgr.x.smithi203.stderr:2023-08-09T13:50:55.068+0000 7f4b1542c040 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2023-08-09T13:50:55.098 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:55.096+0000 7f8114384640 1 Processor -- start 2023-08-09T13:50:55.098 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:55.096+0000 7f8114384640 1 -- start start 2023-08-09T13:50:55.100 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:55.097+0000 7f8114384640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f810c0b2790 0x7f810c0d2ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:50:55.100 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:55.098+0000 7f8114384640 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7f810c0b2b70 0x7f810c0b5330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:50:55.100 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:55.098+0000 7f8114384640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f810c0b3050 0x7f810c0b3f60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:50:55.100 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:55.098+0000 7f8114384640 1 -- --> v2:172.21.15.43:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f810c0b3950 con 0x7f810c0b3050 2023-08-09T13:50:55.101 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:55.098+0000 7f8114384640 1 -- --> v2:172.21.15.203:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f810c0ce090 con 0x7f810c0b2b70 2023-08-09T13:50:55.101 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:55.098+0000 7f8114384640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f810c0cd7f0 con 0x7f810c0b2790 2023-08-09T13:50:55.101 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:55.098+0000 7f8112921640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f810c0b3050 0x7f810c0b3f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:50:55.102 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:55.098+0000 7f8112921640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f810c0b3050 0x7f810c0b3f60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:46492/0 (socket says 172.21.15.43:46492) 2023-08-09T13:50:55.102 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:55.098+0000 7f8112921640 1 -- 172.21.15.43:0/3311477809 learned_addr learned my addr 172.21.15.43:0/3311477809 (peer_addr_for_me v2:172.21.15.43:0/0) 2023-08-09T13:50:55.102 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:55.098+0000 7f811191f640 1 --2- 172.21.15.43:0/3311477809 >> v2:172.21.15.203:3300/0 conn(0x7f810c0b2b70 0x7f810c0b5330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:50:55.102 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:55.098+0000 7f8112120640 1 --2- 172.21.15.43:0/3311477809 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f810c0b2790 0x7f810c0d2ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:50:55.102 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:55.099+0000 7f8112921640 1 -- 172.21.15.43:0/3311477809 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f810c0b2790 msgr2=0x7f810c0d2ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-09T13:50:55.103 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:55.099+0000 7f8112921640 1 --2- 172.21.15.43:0/3311477809 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f810c0b2790 0x7f810c0d2ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:50:55.103 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:55.099+0000 7f8112921640 1 -- 172.21.15.43:0/3311477809 >> v2:172.21.15.203:3300/0 conn(0x7f810c0b2b70 msgr2=0x7f810c0b5330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-09T13:50:55.104 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:55.099+0000 7f8112921640 1 --2- 172.21.15.43:0/3311477809 >> v2:172.21.15.203:3300/0 conn(0x7f810c0b2b70 0x7f810c0b5330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:50:55.104 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:55.099+0000 7f8112921640 1 -- 172.21.15.43:0/3311477809 --> v2:172.21.15.43:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f810c0cdad0 con 0x7f810c0b3050 2023-08-09T13:50:55.105 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:55.100+0000 7f8112921640 1 --2- 172.21.15.43:0/3311477809 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f810c0b3050 0x7f810c0b3f60 secure :-1 s=READY pgs=1083 cs=0 l=1 rev1=1 crypto rx=0x7f810c09bee0 tx=0x7f810402bee0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4489c16d4dcdfd68 server_cookie=0 in_seq=0 out_seq=0 2023-08-09T13:50:55.105 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:55.100+0000 7f811111e640 1 -- 172.21.15.43:0/3311477809 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f810402ca80 con 0x7f810c0b3050 2023-08-09T13:50:55.105 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:55.100+0000 7f811111e640 1 -- 172.21.15.43:0/3311477809 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f810402cbe0 con 0x7f810c0b3050 2023-08-09T13:50:55.106 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:55.100+0000 7f811111e640 1 -- 172.21.15.43:0/3311477809 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f81040343f0 con 0x7f810c0b3050 2023-08-09T13:50:55.106 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:55.101+0000 7f8114384640 1 -- 172.21.15.43:0/3311477809 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f810c0b3050 msgr2=0x7f810c0b3f60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:50:55.106 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:55.101+0000 7f8114384640 1 --2- 172.21.15.43:0/3311477809 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f810c0b3050 0x7f810c0b3f60 secure :-1 s=READY pgs=1083 cs=0 l=1 rev1=1 crypto rx=0x7f810c09bee0 tx=0x7f810402bee0 comp rx=0 tx=0).stop 2023-08-09T13:50:55.106 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:55.101+0000 7f8114384640 1 -- 172.21.15.43:0/3311477809 shutdown_connections 2023-08-09T13:50:55.106 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:55.101+0000 7f8114384640 1 --2- 172.21.15.43:0/3311477809 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f810c0b3050 0x7f810c0b3f60 unknown :-1 s=CLOSED pgs=1083 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:50:55.106 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:55.101+0000 7f8114384640 1 --2- 172.21.15.43:0/3311477809 >> v2:172.21.15.203:3300/0 conn(0x7f810c0b2b70 0x7f810c0b5330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:50:55.106 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:55.101+0000 7f8114384640 1 --2- 172.21.15.43:0/3311477809 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f810c0b2790 0x7f810c0d2ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:50:55.106 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:55.101+0000 7f8114384640 1 -- 172.21.15.43:0/3311477809 >> 172.21.15.43:0/3311477809 conn(0x7f810c09b7b0 msgr2=0x7f810c0d24e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-09T13:50:55.107 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:55.101+0000 7f8114384640 1 -- 172.21.15.43:0/3311477809 shutdown_connections 2023-08-09T13:50:55.107 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:55.101+0000 7f8114384640 1 -- 172.21.15.43:0/3311477809 wait complete. 2023-08-09T13:50:55.107 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:55.103+0000 7f8114384640 1 Processor -- start 2023-08-09T13:50:55.107 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:55.103+0000 7f8114384640 1 -- start start 2023-08-09T13:50:55.108 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:55.104+0000 7f8114384640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f810c0b2790 0x7f810c15a980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:50:55.109 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:55.104+0000 7f8114384640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f810c0b2b70 0x7f810c0c6010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:50:55.109 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:55.104+0000 7f8114384640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f810c0b3050 0x7f810c15a1d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:50:55.109 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:55.104+0000 7f8114384640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f810c0a01f0 con 0x7f810c0b2790 2023-08-09T13:50:55.109 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:55.104+0000 7f8114384640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f810c15b280 con 0x7f810c0b3050 2023-08-09T13:50:55.109 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:55.104+0000 7f8114384640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f810c09f670 con 0x7f810c0b2b70 2023-08-09T13:50:55.109 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:55.104+0000 7f8112120640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f810c0b2790 0x7f810c15a980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:50:55.110 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:55.104+0000 7f8112120640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f810c0b2790 0x7f810c15a980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:46502/0 (socket says 172.21.15.43:46502) 2023-08-09T13:50:55.110 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:55.104+0000 7f8112120640 1 -- 172.21.15.43:0/1877928352 learned_addr learned my addr 172.21.15.43:0/1877928352 (peer_addr_for_me v2:172.21.15.43:0/0) 2023-08-09T13:50:55.110 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:55.104+0000 7f8112921640 1 --2- 172.21.15.43:0/1877928352 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f810c0b3050 0x7f810c15a1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:50:55.110 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:55.104+0000 7f8112120640 1 -- 172.21.15.43:0/1877928352 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f810c0b2b70 msgr2=0x7f810c0c6010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:50:55.110 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:55.104+0000 7f8112120640 1 --2- 172.21.15.43:0/1877928352 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f810c0b2b70 0x7f810c0c6010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:50:55.110 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:55.104+0000 7f8112120640 1 -- 172.21.15.43:0/1877928352 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f810c0b3050 msgr2=0x7f810c15a1d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:50:55.110 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:55.104+0000 7f8112120640 1 --2- 172.21.15.43:0/1877928352 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f810c0b3050 0x7f810c15a1d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:50:55.110 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:55.104+0000 7f8112120640 1 -- 172.21.15.43:0/1877928352 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8104009b30 con 0x7f810c0b2790 2023-08-09T13:50:55.111 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:55.105+0000 7f8112120640 1 --2- 172.21.15.43:0/1877928352 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f810c0b2790 0x7f810c15a980 secure :-1 s=READY pgs=1084 cs=0 l=1 rev1=1 crypto rx=0x7f810800d800 tx=0x7f810800dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-09T13:50:55.111 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:55.105+0000 7f8102ffd640 1 -- 172.21.15.43:0/1877928352 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f81080096a0 con 0x7f810c0b2790 2023-08-09T13:50:55.111 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:55.105+0000 7f8102ffd640 1 -- 172.21.15.43:0/1877928352 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8108009800 con 0x7f810c0b2790 2023-08-09T13:50:55.111 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:55.105+0000 7f8102ffd640 1 -- 172.21.15.43:0/1877928352 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8108009990 con 0x7f810c0b2790 2023-08-09T13:50:55.111 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:55.105+0000 7f8114384640 1 -- 172.21.15.43:0/1877928352 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f810c09f950 con 0x7f810c0b2790 2023-08-09T13:50:55.111 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:55.105+0000 7f8114384640 1 -- 172.21.15.43:0/1877928352 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f810c09ff10 con 0x7f810c0b2790 2023-08-09T13:50:55.111 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:55.106+0000 7f8114384640 1 -- 172.21.15.43:0/1877928352 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f810c0739f0 con 0x7f810c0b2790 2023-08-09T13:50:55.112 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:55.108+0000 7f8102ffd640 1 -- 172.21.15.43:0/1877928352 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 33) v1 ==== 88444+0+0 (secure 0 0 0) 0x7f8108009af0 con 0x7f810c0b2790 2023-08-09T13:50:55.112 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:55.108+0000 7f8102ffd640 1 --2- 172.21.15.43:0/1877928352 >> [v2:172.21.15.203:6832/3851880723,v1:172.21.15.203:6833/3851880723] conn(0x7f80f006a170 0x7f80f006c630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:50:55.113 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:55.109+0000 7f8102ffd640 1 -- 172.21.15.43:0/1877928352 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(91..91 src has 1..91) v4 ==== 6770+0+0 (secure 0 0 0) 0x7f81080892b0 con 0x7f810c0b2790 2023-08-09T13:50:55.113 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:55.109+0000 7f811191f640 1 -- 172.21.15.43:0/1877928352 >> [v2:172.21.15.203:6832/3851880723,v1:172.21.15.203:6833/3851880723] conn(0x7f80f006a170 msgr2=0x7f80f006c630 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.203:6832/3851880723 2023-08-09T13:50:55.113 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:55.109+0000 7f811191f640 1 --2- 172.21.15.43:0/1877928352 >> [v2:172.21.15.203:6832/3851880723,v1:172.21.15.203:6833/3851880723] conn(0x7f80f006a170 0x7f80f006c630 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-08-09T13:50:55.113 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:55.111+0000 7f8102ffd640 1 -- 172.21.15.43:0/1877928352 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178066 (secure 0 0 0) 0x7f81080545a0 con 0x7f810c0b2790 2023-08-09T13:50:55.146 INFO:tasks.ceph.mgr.y.smithi043.stderr:2023-08-09T13:50:55.143+0000 7fe84c03a040 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2023-08-09T13:50:55.208 INFO:tasks.ceph.mgr.y.smithi043.stderr:2023-08-09T13:50:55.205+0000 7fe84c03a040 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2023-08-09T13:50:55.232 INFO:tasks.ceph.mgr.x.smithi203.stderr:2023-08-09T13:50:55.229+0000 7f4b1542c040 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2023-08-09T13:50:55.263 INFO:tasks.ceph.mgr.y.smithi043.stderr:2023-08-09T13:50:55.261+0000 7fe84c03a040 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2023-08-09T13:50:55.281 INFO:tasks.ceph.mgr.x.smithi203.stderr:2023-08-09T13:50:55.279+0000 7f4b1542c040 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2023-08-09T13:50:55.312 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:55.310+0000 7f811191f640 1 -- 172.21.15.43:0/1877928352 >> [v2:172.21.15.203:6832/3851880723,v1:172.21.15.203:6833/3851880723] conn(0x7f80f006a170 msgr2=0x7f80f006c630 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.203:6832/3851880723 2023-08-09T13:50:55.313 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:55.310+0000 7f811191f640 1 --2- 172.21.15.43:0/1877928352 >> [v2:172.21.15.203:6832/3851880723,v1:172.21.15.203:6833/3851880723] conn(0x7f80f006a170 0x7f80f006c630 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2023-08-09T13:50:55.334 INFO:tasks.ceph.mgr.x.smithi203.stderr:2023-08-09T13:50:55.332+0000 7f4b1542c040 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2023-08-09T13:50:55.343 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:55.340+0000 7f8114384640 1 -- 172.21.15.43:0/1877928352 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f810c0b0730 con 0x7f810c0b2790 2023-08-09T13:50:55.344 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:55.341+0000 7f8102ffd640 1 -- 172.21.15.43:0/1877928352 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 35 v35) v1 ==== 94+0+4880 (secure 0 0 0) 0x7f8108057bc0 con 0x7f810c0b2790 2023-08-09T13:50:55.344 INFO:teuthology.orchestra.run.smithi043.stdout: 2023-08-09T13:50:55.344 INFO:teuthology.orchestra.run.smithi043.stdout:{"epoch":35,"default_fscid":4,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5654,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.203:6837/1227342478","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6835","nonce":1227342478},{"type":"v1","addr":"172.21.15.203:6837","nonce":1227342478}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":30},{"gid":7005,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.43:6834/1431309163","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6832","nonce":1431309163},{"type":"v1","addr":"172.21.15.43:6834","nonce":1431309163}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":29},{"gid":7008,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.43:6835/611328204","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6833","nonce":611328204},{"type":"v1","addr":"172.21.15.43:6835","nonce":611328204}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":30}],"filesystems":[{"mdsmap":{"epoch":35,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-08-09T13:50:28.672581+0000","modified":"2023-08-09T13:50:32.253466+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":7011},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7011":{"gid":7011,"name":"d","rank":0,"incarnation":31,"state":"up:active","state_seq":3,"addr":"172.21.15.203:6836/3061388209","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6834","nonce":3061388209},{"type":"v1","addr":"172.21.15.203:6836","nonce":3061388209}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[11],"metadata_pool":10,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":4}]} 2023-08-09T13:50:55.345 INFO:teuthology.orchestra.run.smithi043.stderr:dumped fsmap epoch 35 2023-08-09T13:50:55.346 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:55.344+0000 7f8114384640 1 -- 172.21.15.43:0/1877928352 >> [v2:172.21.15.203:6832/3851880723,v1:172.21.15.203:6833/3851880723] conn(0x7f80f006a170 msgr2=0x7f80f006c630 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-08-09T13:50:55.347 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:55.344+0000 7f8114384640 1 --2- 172.21.15.43:0/1877928352 >> [v2:172.21.15.203:6832/3851880723,v1:172.21.15.203:6833/3851880723] conn(0x7f80f006a170 0x7f80f006c630 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:50:55.347 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:55.344+0000 7f8114384640 1 -- 172.21.15.43:0/1877928352 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f810c0b2790 msgr2=0x7f810c15a980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:50:55.347 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:55.344+0000 7f8114384640 1 --2- 172.21.15.43:0/1877928352 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f810c0b2790 0x7f810c15a980 secure :-1 s=READY pgs=1084 cs=0 l=1 rev1=1 crypto rx=0x7f810800d800 tx=0x7f810800dcc0 comp rx=0 tx=0).stop 2023-08-09T13:50:55.347 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:55.344+0000 7f8114384640 1 -- 172.21.15.43:0/1877928352 shutdown_connections 2023-08-09T13:50:55.347 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:55.344+0000 7f8114384640 1 --2- 172.21.15.43:0/1877928352 >> [v2:172.21.15.203:6832/3851880723,v1:172.21.15.203:6833/3851880723] conn(0x7f80f006a170 0x7f80f006c630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:50:55.348 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:55.344+0000 7f8114384640 1 --2- 172.21.15.43:0/1877928352 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f810c0b3050 0x7f810c15a1d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:50:55.348 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:55.344+0000 7f8114384640 1 --2- 172.21.15.43:0/1877928352 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f810c0b2b70 0x7f810c0c6010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:50:55.348 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:55.344+0000 7f8114384640 1 --2- 172.21.15.43:0/1877928352 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f810c0b2790 0x7f810c15a980 unknown :-1 s=CLOSED pgs=1084 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:50:55.348 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:55.344+0000 7f8114384640 1 -- 172.21.15.43:0/1877928352 >> 172.21.15.43:0/1877928352 conn(0x7f810c09b7b0 msgr2=0x7f810c06a5f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-09T13:50:55.348 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:55.344+0000 7f8114384640 1 -- 172.21.15.43:0/1877928352 shutdown_connections 2023-08-09T13:50:55.348 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:55.344+0000 7f8114384640 1 -- 172.21.15.43:0/1877928352 wait complete. 2023-08-09T13:50:55.365 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-08-09T13:50:55.384 INFO:tasks.ceph.mgr.x.smithi203.stderr:2023-08-09T13:50:55.382+0000 7f4b1542c040 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2023-08-09T13:50:55.445 INFO:tasks.ceph.mgr.y.smithi043.stderr:2023-08-09T13:50:55.443+0000 7fe84c03a040 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2023-08-09T13:50:55.481 INFO:tasks.ceph.mgr.x.smithi203.stderr:2023-08-09T13:50:55.478+0000 7f4a97886640 -1 client.0 error registering admin socket command: (17) File exists 2023-08-09T13:50:55.482 INFO:tasks.ceph.mgr.x.smithi203.stderr:2023-08-09T13:50:55.478+0000 7f4a97886640 -1 client.0 error registering admin socket command: (17) File exists 2023-08-09T13:50:55.482 INFO:tasks.ceph.mgr.x.smithi203.stderr:2023-08-09T13:50:55.478+0000 7f4a97886640 -1 client.0 error registering admin socket command: (17) File exists 2023-08-09T13:50:55.482 INFO:tasks.ceph.mgr.x.smithi203.stderr:2023-08-09T13:50:55.478+0000 7f4a97886640 -1 client.0 error registering admin socket command: (17) File exists 2023-08-09T13:50:55.482 INFO:tasks.ceph.mgr.x.smithi203.stderr:2023-08-09T13:50:55.478+0000 7f4a97886640 -1 client.0 error registering admin socket command: (17) File exists 2023-08-09T13:50:55.482 INFO:tasks.ceph.mgr.x.smithi203.stderr:2023-08-09T13:50:55.478+0000 7f4a9a88c640 -1 client.0 error registering admin socket command: (17) File exists 2023-08-09T13:50:55.482 INFO:tasks.ceph.mgr.x.smithi203.stderr:2023-08-09T13:50:55.478+0000 7f4a9a88c640 -1 client.0 error registering admin socket command: (17) File exists 2023-08-09T13:50:55.483 INFO:tasks.ceph.mgr.x.smithi203.stderr:2023-08-09T13:50:55.478+0000 7f4a9a88c640 -1 client.0 error registering admin socket command: (17) File exists 2023-08-09T13:50:55.483 INFO:tasks.ceph.mgr.x.smithi203.stderr:2023-08-09T13:50:55.478+0000 7f4a9a88c640 -1 client.0 error registering admin socket command: (17) File exists 2023-08-09T13:50:55.484 INFO:tasks.ceph.mgr.x.smithi203.stderr:2023-08-09T13:50:55.478+0000 7f4a9a88c640 -1 client.0 error registering admin socket command: (17) File exists 2023-08-09T13:50:55.502 INFO:tasks.ceph.mgr.y.smithi043.stderr:2023-08-09T13:50:55.499+0000 7fe84c03a040 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2023-08-09T13:50:55.559 INFO:tasks.ceph.mgr.y.smithi043.stderr:2023-08-09T13:50:55.556+0000 7fe84c03a040 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2023-08-09T13:50:55.574 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:55.572+0000 7f93df179640 1 Processor -- start 2023-08-09T13:50:55.575 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:55.572+0000 7f93df179640 1 -- start start 2023-08-09T13:50:55.576 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:55.574+0000 7f93df179640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f93d80b2790 0x7f93d80d2ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:50:55.577 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:55.574+0000 7f93df179640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f93d80b2b70 0x7f93d80b5330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:50:55.577 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:55.574+0000 7f93df179640 1 -- --> v1:172.21.15.203:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f93d80ce090 con 0x7f93d80b3050 2023-08-09T13:50:55.577 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:55.574+0000 7f93df179640 1 -- --> v2:172.21.15.43:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f93d80b3950 con 0x7f93d80b2790 2023-08-09T13:50:55.577 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:55.574+0000 7f93df179640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f93d80cd7f0 con 0x7f93d80b2b70 2023-08-09T13:50:55.577 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:55.574+0000 7f93dcf15640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f93d80b2790 0x7f93d80d2ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:50:55.577 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:55.574+0000 7f93dcf15640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f93d80b2790 0x7f93d80d2ed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:46506/0 (socket says 172.21.15.43:46506) 2023-08-09T13:50:55.577 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:55.574+0000 7f93dcf15640 1 -- 172.21.15.43:0/426432150 learned_addr learned my addr 172.21.15.43:0/426432150 (peer_addr_for_me v2:172.21.15.43:0/0) 2023-08-09T13:50:55.578 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:55.574+0000 7f93d7fff640 1 --2- 172.21.15.43:0/426432150 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f93d80b2b70 0x7f93d80b5330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:50:55.578 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:55.574+0000 7f93dcf15640 1 -- 172.21.15.43:0/426432150 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f93d80b2b70 msgr2=0x7f93d80b5330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-09T13:50:55.579 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:55.574+0000 7f93dcf15640 1 --2- 172.21.15.43:0/426432150 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f93d80b2b70 0x7f93d80b5330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:50:55.579 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:55.574+0000 7f93dcf15640 1 -- 172.21.15.43:0/426432150 >> v1:172.21.15.203:6789/0 conn(0x7f93d80b3050 legacy=0x7f93d80b3f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-09T13:50:55.579 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:55.574+0000 7f93dcf15640 1 -- 172.21.15.43:0/426432150 --> v2:172.21.15.43:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f93d80cdad0 con 0x7f93d80b2790 2023-08-09T13:50:55.579 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:55.574+0000 7f93d7fff640 1 --2- 172.21.15.43:0/426432150 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f93d80b2b70 0x7f93d80b5330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-08-09T13:50:55.579 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:55.575+0000 7f93dcf15640 1 --2- 172.21.15.43:0/426432150 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f93d80b2790 0x7f93d80d2ed0 secure :-1 s=READY pgs=1086 cs=0 l=1 rev1=1 crypto rx=0x7f93cc009870 tx=0x7f93cc02b8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8c3cdd67aee651cf server_cookie=0 in_seq=0 out_seq=0 2023-08-09T13:50:55.579 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:55.576+0000 7f93d77fe640 1 -- 172.21.15.43:0/426432150 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f93cc031640 con 0x7f93d80b2790 2023-08-09T13:50:55.579 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:55.576+0000 7f93d77fe640 1 -- 172.21.15.43:0/426432150 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f93cc0317a0 con 0x7f93d80b2790 2023-08-09T13:50:55.579 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:55.576+0000 7f93d77fe640 1 -- 172.21.15.43:0/426432150 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f93cc031b30 con 0x7f93d80b2790 2023-08-09T13:50:55.580 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:55.576+0000 7f93df179640 1 -- 172.21.15.43:0/426432150 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f93d80b2790 msgr2=0x7f93d80d2ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:50:55.580 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:55.576+0000 7f93df179640 1 --2- 172.21.15.43:0/426432150 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f93d80b2790 0x7f93d80d2ed0 secure :-1 s=READY pgs=1086 cs=0 l=1 rev1=1 crypto rx=0x7f93cc009870 tx=0x7f93cc02b8f0 comp rx=0 tx=0).stop 2023-08-09T13:50:55.580 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:55.577+0000 7f93df179640 1 -- 172.21.15.43:0/426432150 shutdown_connections 2023-08-09T13:50:55.581 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:55.577+0000 7f93df179640 1 --2- 172.21.15.43:0/426432150 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f93d80b2b70 0x7f93d80b5330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:50:55.581 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:55.577+0000 7f93df179640 1 --2- 172.21.15.43:0/426432150 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f93d80b2790 0x7f93d80d2ed0 unknown :-1 s=CLOSED pgs=1086 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:50:55.581 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:55.577+0000 7f93df179640 1 -- 172.21.15.43:0/426432150 >> 172.21.15.43:0/426432150 conn(0x7f93d809b7b0 msgr2=0x7f93d80d24e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-09T13:50:55.581 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:55.577+0000 7f93df179640 1 -- 172.21.15.43:0/426432150 shutdown_connections 2023-08-09T13:50:55.581 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:55.577+0000 7f93df179640 1 -- 172.21.15.43:0/426432150 wait complete. 2023-08-09T13:50:55.582 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:55.578+0000 7f93df179640 1 Processor -- start 2023-08-09T13:50:55.582 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:55.578+0000 7f93df179640 1 -- start start 2023-08-09T13:50:55.582 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:55.579+0000 7f93df179640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f93d80b2790 0x7f93d80b7330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:50:55.583 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:55.579+0000 7f93df179640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f93d80b2b70 0x7f93d80c6010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:50:55.583 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:55.579+0000 7f93df179640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f93d80b3050 0x7f93d815b650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:50:55.583 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:55.579+0000 7f93df179640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f93d80c6ad0 con 0x7f93d80b2b70 2023-08-09T13:50:55.583 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:55.579+0000 7f93df179640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f93d80c6db0 con 0x7f93d80b3050 2023-08-09T13:50:55.583 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:55.579+0000 7f93df179640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f93d80c7090 con 0x7f93d80b2790 2023-08-09T13:50:55.583 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:55.580+0000 7f93d7fff640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f93d80b2b70 0x7f93d80c6010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:50:55.583 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:55.580+0000 7f93d7fff640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f93d80b2b70 0x7f93d80c6010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:46520/0 (socket says 172.21.15.43:46520) 2023-08-09T13:50:55.584 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:55.580+0000 7f93d7fff640 1 -- 172.21.15.43:0/1157531317 learned_addr learned my addr 172.21.15.43:0/1157531317 (peer_addr_for_me v2:172.21.15.43:0/0) 2023-08-09T13:50:55.584 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:55.580+0000 7f93dcf15640 1 --2- 172.21.15.43:0/1157531317 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f93d80b2790 0x7f93d80b7330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:50:55.584 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:55.580+0000 7f93dd716640 1 --2- 172.21.15.43:0/1157531317 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f93d80b3050 0x7f93d815b650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:50:55.585 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:55.580+0000 7f93d7fff640 1 -- 172.21.15.43:0/1157531317 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f93d80b2790 msgr2=0x7f93d80b7330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:50:55.585 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:55.580+0000 7f93d7fff640 1 --2- 172.21.15.43:0/1157531317 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f93d80b2790 0x7f93d80b7330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:50:55.585 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:55.580+0000 7f93d7fff640 1 -- 172.21.15.43:0/1157531317 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f93d80b3050 msgr2=0x7f93d815b650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:50:55.585 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:55.580+0000 7f93d7fff640 1 --2- 172.21.15.43:0/1157531317 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f93d80b3050 0x7f93d815b650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:50:55.586 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:55.580+0000 7f93d7fff640 1 -- 172.21.15.43:0/1157531317 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f93cc0094d0 con 0x7f93d80b2b70 2023-08-09T13:50:55.586 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:55.580+0000 7f93d7fff640 1 --2- 172.21.15.43:0/1157531317 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f93d80b2b70 0x7f93d80c6010 secure :-1 s=READY pgs=1087 cs=0 l=1 rev1=1 crypto rx=0x7f93c400c7f0 tx=0x7f93c400ccb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-09T13:50:55.586 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:55.581+0000 7f93d57fa640 1 -- 172.21.15.43:0/1157531317 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f93c4007820 con 0x7f93d80b2b70 2023-08-09T13:50:55.586 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:55.581+0000 7f93d57fa640 1 -- 172.21.15.43:0/1157531317 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f93c4007980 con 0x7f93d80b2b70 2023-08-09T13:50:55.587 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:55.581+0000 7f93d57fa640 1 -- 172.21.15.43:0/1157531317 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f93c4007cf0 con 0x7f93d80b2b70 2023-08-09T13:50:55.587 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:55.581+0000 7f93df179640 1 -- 172.21.15.43:0/1157531317 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f93d80c7370 con 0x7f93d80b2b70 2023-08-09T13:50:55.587 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:55.581+0000 7f93df179640 1 -- 172.21.15.43:0/1157531317 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f93d80c8090 con 0x7f93d80b2b70 2023-08-09T13:50:55.587 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:55.582+0000 7f93df179640 1 -- 172.21.15.43:0/1157531317 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f93d80739f0 con 0x7f93d80b2b70 2023-08-09T13:50:55.588 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:55.583+0000 7f93d57fa640 1 -- 172.21.15.43:0/1157531317 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 34) v1 ==== 88052+0+0 (secure 0 0 0) 0x7f93c40113f0 con 0x7f93d80b2b70 2023-08-09T13:50:55.588 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:55.584+0000 7f93d57fa640 1 -- 172.21.15.43:0/1157531317 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(92..92 src has 1..92) v4 ==== 7114+0+0 (secure 0 0 0) 0x7f93c4088800 con 0x7f93d80b2b70 2023-08-09T13:50:55.590 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:55.588+0000 7f93d57fa640 1 -- 172.21.15.43:0/1157531317 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178066 (secure 0 0 0) 0x7f93c4053eb0 con 0x7f93d80b2b70 2023-08-09T13:50:55.615 INFO:tasks.ceph.mgr.y.smithi043.stderr:2023-08-09T13:50:55.612+0000 7fe84c03a040 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2023-08-09T13:50:55.785 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:55.782+0000 7f93df179640 1 -- 172.21.15.43:0/1157531317 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f93d809d500 con 0x7f93d80b2b70 2023-08-09T13:50:55.787 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:55.784+0000 7f93d57fa640 1 -- 172.21.15.43:0/1157531317 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v92) v1 ==== 74+0+19573 (secure 0 0 0) 0x7f93c40574d0 con 0x7f93d80b2b70 2023-08-09T13:50:55.788 INFO:teuthology.orchestra.run.smithi043.stdout: 2023-08-09T13:50:55.788 INFO:teuthology.orchestra.run.smithi043.stdout:{"epoch":92,"fsid":"daac87ee-1337-4250-a955-aea4e408a39b","created":"2023-08-09T13:33:41.701198+0000","modified":"2023-08-09T13:50:55.386567+0000","last_up_change":"2023-08-09T13:33:53.783453+0000","last_in_change":"2023-08-09T13:33:45.148904+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":11,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":"rbd","create_time":"2023-08-09T13:33:54.365137+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":".mgr","create_time":"2023-08-09T13:33:54.831803+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":10,"pool_name":"cephfs_metadata","create_time":"2023-08-09T13:50:27.051781+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"89","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":2.5,"score_stable":2.5,"optimal_score":1,"raw_score_acting":2.5,"raw_score_stable":2.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":11,"pool_name":"cephfs_data","create_time":"2023-08-09T13:50:27.692933+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"90","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"cbd49ecd-e896-4fbd-864e-5c37a829b165","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6800","nonce":502667335},{"type":"v1","addr":"172.21.15.43:6801","nonce":502667335}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6802","nonce":502667335},{"type":"v1","addr":"172.21.15.43:6803","nonce":502667335}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6806","nonce":502667335},{"type":"v1","addr":"172.21.15.43:6807","nonce":502667335}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6804","nonce":502667335},{"type":"v1","addr":"172.21.15.43:6805","nonce":502667335}]},"public_addr":"172.21.15.43:6801/502667335","cluster_addr":"172.21.15.43:6803/502667335","heartbeat_back_addr":"172.21.15.43:6807/502667335","heartbeat_front_addr":"172.21.15.43:6805/502667335","state":["exists","up"]},{"osd":1,"uuid":"75c7e0fe-1ec7-4fe8-a2f2-3661a65b8500","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6808","nonce":3289134610},{"type":"v1","addr":"172.21.15.43:6809","nonce":3289134610}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6810","nonce":3289134610},{"type":"v1","addr":"172.21.15.43:6812","nonce":3289134610}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6821","nonce":3289134610},{"type":"v1","addr":"172.21.15.43:6824","nonce":3289134610}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6815","nonce":3289134610},{"type":"v1","addr":"172.21.15.43:6818","nonce":3289134610}]},"public_addr":"172.21.15.43:6809/3289134610","cluster_addr":"172.21.15.43:6812/3289134610","heartbeat_back_addr":"172.21.15.43:6824/3289134610","heartbeat_front_addr":"172.21.15.43:6818/3289134610","state":["exists","up"]},{"osd":2,"uuid":"bb5c41e9-7f5f-4125-9b01-3bb8bfc2ea83","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6811","nonce":4079709736},{"type":"v1","addr":"172.21.15.43:6814","nonce":4079709736}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6817","nonce":4079709736},{"type":"v1","addr":"172.21.15.43:6820","nonce":4079709736}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6829","nonce":4079709736},{"type":"v1","addr":"172.21.15.43:6831","nonce":4079709736}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6823","nonce":4079709736},{"type":"v1","addr":"172.21.15.43:6826","nonce":4079709736}]},"public_addr":"172.21.15.43:6814/4079709736","cluster_addr":"172.21.15.43:6820/4079709736","heartbeat_back_addr":"172.21.15.43:6831/4079709736","heartbeat_front_addr":"172.21.15.43:6826/4079709736","state":["exists","up"]},{"osd":3,"uuid":"bfe4c8b4-694a-465c-87e6-2b2aab852613","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6813","nonce":1709538759},{"type":"v1","addr":"172.21.15.43:6816","nonce":1709538759}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6819","nonce":1709538759},{"type":"v1","addr":"172.21.15.43:6822","nonce":1709538759}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6828","nonce":1709538759},{"type":"v1","addr":"172.21.15.43:6830","nonce":1709538759}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6825","nonce":1709538759},{"type":"v1","addr":"172.21.15.43:6827","nonce":1709538759}]},"public_addr":"172.21.15.43:6816/1709538759","cluster_addr":"172.21.15.43:6822/1709538759","heartbeat_back_addr":"172.21.15.43:6830/1709538759","heartbeat_front_addr":"172.21.15.43:6827/1709538759","state":["exists","up"]},{"osd":4,"uuid":"fa2e8a77-aba1-4756-853e-0d174a5bdeba","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6800","nonce":4196866481},{"type":"v1","addr":"172.21.15.203:6801","nonce":4196866481}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6802","nonce":4196866481},{"type":"v1","addr":"172.21.15.203:6803","nonce":4196866481}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6806","nonce":4196866481},{"type":"v1","addr":"172.21.15.203:6807","nonce":4196866481}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6804","nonce":4196866481},{"type":"v1","addr":"172.21.15.203:6805","nonce":4196866481}]},"public_addr":"172.21.15.203:6801/4196866481","cluster_addr":"172.21.15.203:6803/4196866481","heartbeat_back_addr":"172.21.15.203:6807/4196866481","heartbeat_front_addr":"172.21.15.203:6805/4196866481","state":["exists","up"]},{"osd":5,"uuid":"81811e4b-4d24-4f51-a82c-f1d700a2a00d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6816","nonce":3059290081},{"type":"v1","addr":"172.21.15.203:6817","nonce":3059290081}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6818","nonce":3059290081},{"type":"v1","addr":"172.21.15.203:6819","nonce":3059290081}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6822","nonce":3059290081},{"type":"v1","addr":"172.21.15.203:6823","nonce":3059290081}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6820","nonce":3059290081},{"type":"v1","addr":"172.21.15.203:6821","nonce":3059290081}]},"public_addr":"172.21.15.203:6817/3059290081","cluster_addr":"172.21.15.203:6819/3059290081","heartbeat_back_addr":"172.21.15.203:6823/3059290081","heartbeat_front_addr":"172.21.15.203:6821/3059290081","state":["exists","up"]},{"osd":6,"uuid":"307bac4d-6a5b-443d-af75-56d695d7674f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6824","nonce":512155569},{"type":"v1","addr":"172.21.15.203:6825","nonce":512155569}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6826","nonce":512155569},{"type":"v1","addr":"172.21.15.203:6827","nonce":512155569}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6830","nonce":512155569},{"type":"v1","addr":"172.21.15.203:6831","nonce":512155569}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6828","nonce":512155569},{"type":"v1","addr":"172.21.15.203:6829","nonce":512155569}]},"public_addr":"172.21.15.203:6825/512155569","cluster_addr":"172.21.15.203:6827/512155569","heartbeat_back_addr":"172.21.15.203:6831/512155569","heartbeat_front_addr":"172.21.15.203:6829/512155569","state":["exists","up"]},{"osd":7,"uuid":"a95dc7ca-9f1b-4295-af70-68a22a2d7d5e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6808","nonce":3331450438},{"type":"v1","addr":"172.21.15.203:6809","nonce":3331450438}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6810","nonce":3331450438},{"type":"v1","addr":"172.21.15.203:6811","nonce":3331450438}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6814","nonce":3331450438},{"type":"v1","addr":"172.21.15.203:6815","nonce":3331450438}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6812","nonce":3331450438},{"type":"v1","addr":"172.21.15.203:6813","nonce":3331450438}]},"public_addr":"172.21.15.203:6809/3331450438","cluster_addr":"172.21.15.203:6811/3331450438","heartbeat_back_addr":"172.21.15.203:6815/3331450438","heartbeat_front_addr":"172.21.15.203:6813/3331450438","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-09T13:33:50.547652+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-09T13:33:50.589469+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-09T13:33:50.561724+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-09T13:33:50.599650+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-09T13:33:50.411855+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-09T13:33:50.219681+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-09T13:33:50.504664+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-09T13:33:50.441248+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.203:0/375511470":"2023-08-10T13:50:55.386522+0000","172.21.15.203:0/4256110232":"2023-08-10T13:50:55.386522+0000","172.21.15.203:0/3102402128":"2023-08-10T13:50:55.386522+0000","172.21.15.203:0/1112175379":"2023-08-10T13:50:55.386522+0000","172.21.15.203:6833/3851880723":"2023-08-10T13:50:55.386522+0000","172.21.15.203:6833/3325141227":"2023-08-10T13:50:49.168196+0000","172.21.15.203:6832/3325141227":"2023-08-10T13:50:49.168196+0000","172.21.15.203:0/1343982026":"2023-08-10T13:50:55.386522+0000","172.21.15.203:0/3732830113":"2023-08-10T13:50:49.168196+0000","172.21.15.203:0/2753554568":"2023-08-10T13:50:49.168196+0000","172.21.15.203:0/3841129997":"2023-08-10T13:50:55.386522+0000","172.21.15.203:0/2468196984":"2023-08-10T13:50:49.168196+0000","172.21.15.203:6832/3851880723":"2023-08-10T13:50:55.386522+0000","172.21.15.203:0/3375551329":"2023-08-10T13:50:49.168196+0000","172.21.15.203:0/1259705306":"2023-08-10T13:50:49.168196+0000","172.21.15.203:0/1322546009":"2023-08-10T13:50:49.168196+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":7,"snaps":[{"begin":104,"length":1},{"begin":221,"length":1}]}],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-08-09T13:50:55.790 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:55.787+0000 7f93df179640 1 -- 172.21.15.43:0/1157531317 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f93d80b2b70 msgr2=0x7f93d80c6010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:50:55.790 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:55.787+0000 7f93df179640 1 --2- 172.21.15.43:0/1157531317 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f93d80b2b70 0x7f93d80c6010 secure :-1 s=READY pgs=1087 cs=0 l=1 rev1=1 crypto rx=0x7f93c400c7f0 tx=0x7f93c400ccb0 comp rx=0 tx=0).stop 2023-08-09T13:50:55.791 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:55.788+0000 7f93df179640 1 -- 172.21.15.43:0/1157531317 shutdown_connections 2023-08-09T13:50:55.791 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:55.788+0000 7f93df179640 1 --2- 172.21.15.43:0/1157531317 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f93d80b3050 0x7f93d815b650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:50:55.791 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:55.788+0000 7f93df179640 1 --2- 172.21.15.43:0/1157531317 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f93d80b2b70 0x7f93d80c6010 unknown :-1 s=CLOSED pgs=1087 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:50:55.791 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:55.788+0000 7f93df179640 1 --2- 172.21.15.43:0/1157531317 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f93d80b2790 0x7f93d80b7330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:50:55.791 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:55.788+0000 7f93df179640 1 -- 172.21.15.43:0/1157531317 >> 172.21.15.43:0/1157531317 conn(0x7f93d809b7b0 msgr2=0x7f93d80b6650 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-09T13:50:55.791 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:55.788+0000 7f93df179640 1 -- 172.21.15.43:0/1157531317 shutdown_connections 2023-08-09T13:50:55.791 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:55.788+0000 7f93df179640 1 -- 172.21.15.43:0/1157531317 wait complete. 2023-08-09T13:50:55.809 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': 'rbd', 'create_time': '2023-08-09T13:33:54.365137+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}, 'read_balance': {'score_acting': 2, 'score_stable': 2, 'optimal_score': 1, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': '.mgr', 'create_time': '2023-08-09T13:33:54.831803+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 10, 'pool_name': 'cephfs_metadata', 'create_time': '2023-08-09T13:50:27.051781+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '89', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 2.5, 'score_stable': 2.5, 'optimal_score': 1, 'raw_score_acting': 2.5, 'raw_score_stable': 2.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 11, 'pool_name': 'cephfs_data', 'create_time': '2023-08-09T13:50:27.692933+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '90', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.3799999952316284, 'score_stable': 1.3799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.3799999952316284, 'raw_score_stable': 1.3799999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-08-09T13:50:55.810 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get rbd pg_num 2023-08-09T13:50:56.044 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:56.042+0000 7f8871537640 1 Processor -- start 2023-08-09T13:50:56.045 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:56.042+0000 7f8871537640 1 -- start start 2023-08-09T13:50:56.047 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:56.043+0000 7f8871537640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f886c159400 0x7f886c0cb9a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:50:56.047 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:56.043+0000 7f8871537640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f886c0c8900 0x7f886c15a270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:50:56.048 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:56.044+0000 7f8871537640 1 -- --> v1:172.21.15.43:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f886c15a7b0 con 0x7f886c0c8e50 2023-08-09T13:50:56.048 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:56.044+0000 7f8871537640 1 -- --> v2:172.21.15.43:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f886c070250 con 0x7f886c0c8900 2023-08-09T13:50:56.048 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:56.044+0000 7f8871537640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f886c0c86c0 con 0x7f886c159400 2023-08-09T13:50:56.048 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:56.044+0000 7f8863fff640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f886c0c8900 0x7f886c15a270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:50:56.048 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:56.044+0000 7f8870d36640 1 --1- >> v1:172.21.15.43:6789/0 conn(0x7f886c0c8e50 0x7f886c0706e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.43:6789/0 says I am v1:172.21.15.43:35036/0 (socket says 172.21.15.43:35036) 2023-08-09T13:50:56.049 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:56.044+0000 7f8870d36640 1 -- 172.21.15.43:0/2222145350 learned_addr learned my addr 172.21.15.43:0/2222145350 (peer_addr_for_me v1:172.21.15.43:0/0) 2023-08-09T13:50:56.049 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:56.044+0000 7f8863fff640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f886c0c8900 0x7f886c15a270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:46532/0 (socket says 172.21.15.43:46532) 2023-08-09T13:50:56.049 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:56.044+0000 7f886bfff640 1 --2- 172.21.15.43:0/2222145350 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f886c159400 0x7f886c0cb9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:50:56.049 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:56.044+0000 7f886b7fe640 1 -- 172.21.15.43:0/2222145350 <== mon.0 v1:172.21.15.43:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3192914929 0 0) 0x7f8864001490 con 0x7f886c0c8e50 2023-08-09T13:50:56.049 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:56.045+0000 7f886b7fe640 1 -- 172.21.15.43:0/2222145350 --> v1:172.21.15.43:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f8858003520 con 0x7f886c0c8e50 2023-08-09T13:50:56.050 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:56.045+0000 7f8863fff640 1 -- 172.21.15.43:0/2222145350 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f886c159400 msgr2=0x7f886c0cb9a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-09T13:50:56.050 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:56.045+0000 7f8863fff640 1 --2- 172.21.15.43:0/2222145350 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f886c159400 0x7f886c0cb9a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:50:56.050 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:56.045+0000 7f8863fff640 1 -- 172.21.15.43:0/2222145350 >> v1:172.21.15.43:6789/0 conn(0x7f886c0c8e50 legacy=0x7f886c0706e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:50:56.050 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:56.045+0000 7f8863fff640 1 -- 172.21.15.43:0/2222145350 --> v2:172.21.15.43:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f886c06fc60 con 0x7f886c0c8900 2023-08-09T13:50:56.052 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:56.046+0000 7f8863fff640 1 --2- 172.21.15.43:0/2222145350 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f886c0c8900 0x7f886c15a270 secure :-1 s=READY pgs=1089 cs=0 l=1 rev1=1 crypto rx=0x7f88540097a0 tx=0x7f885402b820 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bfadbc4bcf9623c6 server_cookie=0 in_seq=0 out_seq=0 2023-08-09T13:50:56.053 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:56.046+0000 7f886b7fe640 1 -- 172.21.15.43:0/2222145350 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8854031780 con 0x7f886c0c8900 2023-08-09T13:50:56.053 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:56.046+0000 7f886b7fe640 1 -- 172.21.15.43:0/2222145350 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f88540318e0 con 0x7f886c0c8900 2023-08-09T13:50:56.053 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:56.046+0000 7f886b7fe640 1 -- 172.21.15.43:0/2222145350 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8854031c70 con 0x7f886c0c8900 2023-08-09T13:50:56.053 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:56.047+0000 7f8871537640 1 -- 172.21.15.43:0/2222145350 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f886c0c8900 msgr2=0x7f886c15a270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:50:56.054 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:56.047+0000 7f8871537640 1 --2- 172.21.15.43:0/2222145350 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f886c0c8900 0x7f886c15a270 secure :-1 s=READY pgs=1089 cs=0 l=1 rev1=1 crypto rx=0x7f88540097a0 tx=0x7f885402b820 comp rx=0 tx=0).stop 2023-08-09T13:50:56.054 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:56.047+0000 7f8871537640 1 -- 172.21.15.43:0/2222145350 shutdown_connections 2023-08-09T13:50:56.054 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:56.047+0000 7f8871537640 1 --2- 172.21.15.43:0/2222145350 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f886c0c8900 0x7f886c15a270 unknown :-1 s=CLOSED pgs=1089 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:50:56.054 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:56.047+0000 7f8871537640 1 --2- 172.21.15.43:0/2222145350 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f886c159400 0x7f886c0cb9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:50:56.054 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:56.047+0000 7f8871537640 1 -- 172.21.15.43:0/2222145350 >> 172.21.15.43:0/2222145350 conn(0x7f886c0cddd0 msgr2=0x7f886c0ce1e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-09T13:50:56.055 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:56.047+0000 7f8871537640 1 -- 172.21.15.43:0/2222145350 shutdown_connections 2023-08-09T13:50:56.055 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:56.047+0000 7f8871537640 1 -- 172.21.15.43:0/2222145350 wait complete. 2023-08-09T13:50:56.055 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:56.049+0000 7f8871537640 1 Processor -- start 2023-08-09T13:50:56.055 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:56.049+0000 7f8871537640 1 -- start start 2023-08-09T13:50:56.057 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:56.050+0000 7f8871537640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f886c0c8900 0x7f886c0c5440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:50:56.057 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:56.050+0000 7f8871537640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f886c0c8e50 0x7f886c0ab8d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:50:56.057 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:56.050+0000 7f8871537640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f886c0abe10 0x7f886c1f54f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:50:56.057 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:56.050+0000 7f8871537640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f886c0ac2c0 con 0x7f886c0abe10 2023-08-09T13:50:56.057 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:56.050+0000 7f8871537640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f886c0c7220 con 0x7f886c0c8900 2023-08-09T13:50:56.057 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:56.050+0000 7f8871537640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f886c0c7500 con 0x7f886c0c8e50 2023-08-09T13:50:56.058 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:56.050+0000 7f8870d36640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f886c0abe10 0x7f886c1f54f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:50:56.058 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:56.050+0000 7f8870d36640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f886c0abe10 0x7f886c1f54f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:46538/0 (socket says 172.21.15.43:46538) 2023-08-09T13:50:56.058 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:56.050+0000 7f8870d36640 1 -- 172.21.15.43:0/1898652392 learned_addr learned my addr 172.21.15.43:0/1898652392 (peer_addr_for_me v2:172.21.15.43:0/0) 2023-08-09T13:50:56.058 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:56.051+0000 7f886bfff640 1 --2- 172.21.15.43:0/1898652392 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f886c0c8900 0x7f886c0c5440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:50:56.058 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:56.051+0000 7f8870d36640 1 -- 172.21.15.43:0/1898652392 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f886c0c8e50 msgr2=0x7f886c0ab8d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:50:56.058 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:56.051+0000 7f8863fff640 1 --2- 172.21.15.43:0/1898652392 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f886c0c8e50 0x7f886c0ab8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:50:56.059 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:56.051+0000 7f8870d36640 1 --2- 172.21.15.43:0/1898652392 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f886c0c8e50 0x7f886c0ab8d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:50:56.059 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:56.051+0000 7f8870d36640 1 -- 172.21.15.43:0/1898652392 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f886c0c8900 msgr2=0x7f886c0c5440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:50:56.059 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:56.051+0000 7f8870d36640 1 --2- 172.21.15.43:0/1898652392 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f886c0c8900 0x7f886c0c5440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:50:56.059 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:56.051+0000 7f8870d36640 1 -- 172.21.15.43:0/1898652392 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8854009400 con 0x7f886c0abe10 2023-08-09T13:50:56.060 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:56.051+0000 7f886bfff640 1 --2- 172.21.15.43:0/1898652392 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f886c0c8900 0x7f886c0c5440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-08-09T13:50:56.060 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:56.051+0000 7f8863fff640 1 --2- 172.21.15.43:0/1898652392 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f886c0c8e50 0x7f886c0ab8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-08-09T13:50:56.061 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:56.051+0000 7f8870d36640 1 --2- 172.21.15.43:0/1898652392 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f886c0abe10 0x7f886c1f54f0 secure :-1 s=READY pgs=1090 cs=0 l=1 rev1=1 crypto rx=0x7f886400cb40 tx=0x7f886400a060 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-09T13:50:56.061 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:56.051+0000 7f88697fa640 1 -- 172.21.15.43:0/1898652392 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f886400aca0 con 0x7f886c0abe10 2023-08-09T13:50:56.061 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:56.051+0000 7f88697fa640 1 -- 172.21.15.43:0/1898652392 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f886400ae00 con 0x7f886c0abe10 2023-08-09T13:50:56.061 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:56.051+0000 7f88697fa640 1 -- 172.21.15.43:0/1898652392 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8864012620 con 0x7f886c0abe10 2023-08-09T13:50:56.062 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:56.051+0000 7f8871537640 1 -- 172.21.15.43:0/1898652392 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f886c0c77e0 con 0x7f886c0abe10 2023-08-09T13:50:56.062 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:56.052+0000 7f8871537640 1 -- 172.21.15.43:0/1898652392 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f886c0c6ed0 con 0x7f886c0abe10 2023-08-09T13:50:56.062 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:56.053+0000 7f8871537640 1 -- 172.21.15.43:0/1898652392 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f886c04e390 con 0x7f886c0abe10 2023-08-09T13:50:56.062 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:56.054+0000 7f88697fa640 1 -- 172.21.15.43:0/1898652392 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 34) v1 ==== 88052+0+0 (secure 0 0 0) 0x7f8864007070 con 0x7f886c0abe10 2023-08-09T13:50:56.063 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:56.055+0000 7f88697fa640 1 -- 172.21.15.43:0/1898652392 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(92..92 src has 1..92) v4 ==== 7114+0+0 (secure 0 0 0) 0x7f8864088280 con 0x7f886c0abe10 2023-08-09T13:50:56.064 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:56.059+0000 7f88697fa640 1 -- 172.21.15.43:0/1898652392 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178066 (secure 0 0 0) 0x7f88640539a0 con 0x7f886c0abe10 2023-08-09T13:50:56.253 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:56.250+0000 7f8871537640 1 -- 172.21.15.43:0/1898652392 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"} v 0) v1 -- 0x7f886c15a7b0 con 0x7f886c0abe10 2023-08-09T13:50:56.254 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:56.251+0000 7f88697fa640 1 -- 172.21.15.43:0/1898652392 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"}]=0 v92) v1 ==== 92+0+10 (secure 0 0 0) 0x7f8864017020 con 0x7f886c0abe10 2023-08-09T13:50:56.254 INFO:teuthology.orchestra.run.smithi043.stdout:pg_num: 8 2023-08-09T13:50:56.256 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:56.254+0000 7f8871537640 1 -- 172.21.15.43:0/1898652392 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f886c0abe10 msgr2=0x7f886c1f54f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:50:56.256 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:56.254+0000 7f8871537640 1 --2- 172.21.15.43:0/1898652392 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f886c0abe10 0x7f886c1f54f0 secure :-1 s=READY pgs=1090 cs=0 l=1 rev1=1 crypto rx=0x7f886400cb40 tx=0x7f886400a060 comp rx=0 tx=0).stop 2023-08-09T13:50:56.257 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:56.254+0000 7f8871537640 1 -- 172.21.15.43:0/1898652392 shutdown_connections 2023-08-09T13:50:56.257 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:56.254+0000 7f8871537640 1 --2- 172.21.15.43:0/1898652392 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f886c0abe10 0x7f886c1f54f0 unknown :-1 s=CLOSED pgs=1090 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:50:56.257 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:56.254+0000 7f8871537640 1 --2- 172.21.15.43:0/1898652392 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f886c0c8e50 0x7f886c0ab8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:50:56.257 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:56.254+0000 7f8871537640 1 --2- 172.21.15.43:0/1898652392 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f886c0c8900 0x7f886c0c5440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:50:56.257 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:56.254+0000 7f8871537640 1 -- 172.21.15.43:0/1898652392 >> 172.21.15.43:0/1898652392 conn(0x7f886c0cddd0 msgr2=0x7f886c0705e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-09T13:50:56.258 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:56.254+0000 7f8871537640 1 -- 172.21.15.43:0/1898652392 shutdown_connections 2023-08-09T13:50:56.258 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:56.254+0000 7f8871537640 1 -- 172.21.15.43:0/1898652392 wait complete. 2023-08-09T13:50:56.274 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-08-09T13:50:56.514 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:56.512+0000 7f5be408e640 1 Processor -- start 2023-08-09T13:50:56.515 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:56.512+0000 7f5be408e640 1 -- start start 2023-08-09T13:50:56.516 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:56.513+0000 7f5be408e640 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7f5bdc0b45f0 0x7f5bdc0cd6d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:50:56.516 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:56.513+0000 7f5be408e640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f5bdc0d04b0 0x7f5bdc0b59d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:50:56.517 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:56.513+0000 7f5be408e640 1 -- --> v1:172.21.15.203:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5bdc0b7bd0 con 0x7f5bdc15c550 2023-08-09T13:50:56.517 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:56.514+0000 7f5be408e640 1 -- --> v2:172.21.15.203:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5bdc0d1c00 con 0x7f5bdc0b45f0 2023-08-09T13:50:56.517 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:56.514+0000 7f5be408e640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5bdc15c930 con 0x7f5bdc0d04b0 2023-08-09T13:50:56.518 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:56.514+0000 7f5be1629640 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7f5bdc0b45f0 0x7f5bdc0cd6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:50:56.518 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:56.514+0000 7f5be262b640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f5bdc0d04b0 0x7f5bdc0b59d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:50:56.519 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:56.514+0000 7f5be1629640 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7f5bdc0b45f0 0x7f5bdc0cd6d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.203:3300/0 says I am v2:172.21.15.43:52856/0 (socket says 172.21.15.43:52856) 2023-08-09T13:50:56.519 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:56.514+0000 7f5be1629640 1 -- 172.21.15.43:0/3679202167 learned_addr learned my addr 172.21.15.43:0/3679202167 (peer_addr_for_me v2:172.21.15.43:0/0) 2023-08-09T13:50:56.519 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:56.514+0000 7f5be262b640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f5bdc0d04b0 0x7f5bdc0b59d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.203:3301/0 says I am v2:172.21.15.43:60200/0 (socket says 172.21.15.43:60200) 2023-08-09T13:50:56.520 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:56.515+0000 7f5be1629640 1 -- 172.21.15.43:0/3679202167 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f5bdc0d04b0 msgr2=0x7f5bdc0b59d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-09T13:50:56.520 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:56.515+0000 7f5be1629640 1 --2- 172.21.15.43:0/3679202167 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f5bdc0d04b0 0x7f5bdc0b59d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:50:56.520 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:56.515+0000 7f5be1629640 1 -- 172.21.15.43:0/3679202167 >> v1:172.21.15.203:6789/0 conn(0x7f5bdc15c550 legacy=0x7f5bdc0b3ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:50:56.520 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:56.515+0000 7f5be1629640 1 -- 172.21.15.43:0/3679202167 --> v2:172.21.15.203:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5bdc0b54f0 con 0x7f5bdc0b45f0 2023-08-09T13:50:56.523 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:56.517+0000 7f5be1629640 1 --2- 172.21.15.43:0/3679202167 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f5bdc0b45f0 0x7f5bdc0cd6d0 secure :-1 s=READY pgs=399 cs=0 l=1 rev1=1 crypto rx=0x7f5bcc009870 tx=0x7f5bcc02b980 comp rx=0 tx=0).ready entity=mon.1 client_cookie=f4ac536c68846ff9 server_cookie=0 in_seq=0 out_seq=0 2023-08-09T13:50:56.523 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:56.517+0000 7f5be0e28640 1 -- 172.21.15.43:0/3679202167 <== mon.1 v2:172.21.15.203:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5bcc030840 con 0x7f5bdc0b45f0 2023-08-09T13:50:56.523 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:56.517+0000 7f5be0e28640 1 -- 172.21.15.43:0/3679202167 <== mon.1 v2:172.21.15.203:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5bcc0309a0 con 0x7f5bdc0b45f0 2023-08-09T13:50:56.523 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:56.517+0000 7f5be0e28640 1 -- 172.21.15.43:0/3679202167 <== mon.1 v2:172.21.15.203:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5bcc030d30 con 0x7f5bdc0b45f0 2023-08-09T13:50:56.523 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:56.518+0000 7f5be408e640 1 -- 172.21.15.43:0/3679202167 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f5bdc0b45f0 msgr2=0x7f5bdc0cd6d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:50:56.523 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:56.518+0000 7f5be408e640 1 --2- 172.21.15.43:0/3679202167 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f5bdc0b45f0 0x7f5bdc0cd6d0 secure :-1 s=READY pgs=399 cs=0 l=1 rev1=1 crypto rx=0x7f5bcc009870 tx=0x7f5bcc02b980 comp rx=0 tx=0).stop 2023-08-09T13:50:56.524 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:56.518+0000 7f5be408e640 1 -- 172.21.15.43:0/3679202167 shutdown_connections 2023-08-09T13:50:56.524 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:56.518+0000 7f5be408e640 1 --2- 172.21.15.43:0/3679202167 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f5bdc0d04b0 0x7f5bdc0b59d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:50:56.524 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:56.518+0000 7f5be408e640 1 --2- 172.21.15.43:0/3679202167 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f5bdc0b45f0 0x7f5bdc0cd6d0 unknown :-1 s=CLOSED pgs=399 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:50:56.524 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:56.518+0000 7f5be408e640 1 -- 172.21.15.43:0/3679202167 >> 172.21.15.43:0/3679202167 conn(0x7f5bdc071520 msgr2=0x7f5bdc0ceee0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-09T13:50:56.524 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:56.518+0000 7f5be408e640 1 -- 172.21.15.43:0/3679202167 shutdown_connections 2023-08-09T13:50:56.524 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:56.518+0000 7f5be408e640 1 -- 172.21.15.43:0/3679202167 wait complete. 2023-08-09T13:50:56.524 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:56.520+0000 7f5be408e640 1 Processor -- start 2023-08-09T13:50:56.525 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:56.520+0000 7f5be408e640 1 -- start start 2023-08-09T13:50:56.526 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:56.521+0000 7f5be408e640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f5bdc0b45f0 0x7f5bdc159e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:50:56.526 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:56.521+0000 7f5be408e640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f5bdc0d04b0 0x7f5bdc0a8dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:50:56.526 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:56.521+0000 7f5be408e640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f5bdc15c550 0x7f5bdc0b3330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:50:56.526 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:56.521+0000 7f5be408e640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5bdc0c9530 con 0x7f5bdc0b45f0 2023-08-09T13:50:56.527 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:56.521+0000 7f5be408e640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5bdc15a390 con 0x7f5bdc15c550 2023-08-09T13:50:56.527 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:56.521+0000 7f5be408e640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5bdc0a9710 con 0x7f5bdc0d04b0 2023-08-09T13:50:56.527 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:56.521+0000 7f5be1e2a640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f5bdc0b45f0 0x7f5bdc159e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:50:56.527 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:56.521+0000 7f5be1e2a640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f5bdc0b45f0 0x7f5bdc159e50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:46554/0 (socket says 172.21.15.43:46554) 2023-08-09T13:50:56.527 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:56.521+0000 7f5be1e2a640 1 -- 172.21.15.43:0/1036417741 learned_addr learned my addr 172.21.15.43:0/1036417741 (peer_addr_for_me v2:172.21.15.43:0/0) 2023-08-09T13:50:56.527 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:56.522+0000 7f5be1629640 1 --2- 172.21.15.43:0/1036417741 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f5bdc0d04b0 0x7f5bdc0a8dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:50:56.527 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:56.522+0000 7f5be262b640 1 --2- 172.21.15.43:0/1036417741 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f5bdc15c550 0x7f5bdc0b3330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:50:56.527 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:56.522+0000 7f5be1e2a640 1 -- 172.21.15.43:0/1036417741 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f5bdc0d04b0 msgr2=0x7f5bdc0a8dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:50:56.528 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:56.522+0000 7f5be1e2a640 1 --2- 172.21.15.43:0/1036417741 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f5bdc0d04b0 0x7f5bdc0a8dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:50:56.528 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:56.522+0000 7f5be1e2a640 1 -- 172.21.15.43:0/1036417741 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f5bdc15c550 msgr2=0x7f5bdc0b3330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:50:56.528 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:56.522+0000 7f5be1e2a640 1 --2- 172.21.15.43:0/1036417741 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f5bdc15c550 0x7f5bdc0b3330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:50:56.528 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:56.522+0000 7f5be1e2a640 1 -- 172.21.15.43:0/1036417741 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5bcc0094d0 con 0x7f5bdc0b45f0 2023-08-09T13:50:56.528 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:56.522+0000 7f5be1e2a640 1 --2- 172.21.15.43:0/1036417741 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f5bdc0b45f0 0x7f5bdc159e50 secure :-1 s=READY pgs=1091 cs=0 l=1 rev1=1 crypto rx=0x7f5bd800c400 tx=0x7f5bd800c8c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-09T13:50:56.528 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:56.523+0000 7f5bd27fc640 1 -- 172.21.15.43:0/1036417741 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5bd80052d0 con 0x7f5bdc0b45f0 2023-08-09T13:50:56.528 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:56.523+0000 7f5bd27fc640 1 -- 172.21.15.43:0/1036417741 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5bd8005430 con 0x7f5bdc0b45f0 2023-08-09T13:50:56.529 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:56.523+0000 7f5bd27fc640 1 -- 172.21.15.43:0/1036417741 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5bd8003830 con 0x7f5bdc0b45f0 2023-08-09T13:50:56.529 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:56.523+0000 7f5be408e640 1 -- 172.21.15.43:0/1036417741 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5bdc0a99f0 con 0x7f5bdc0b45f0 2023-08-09T13:50:56.529 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:56.523+0000 7f5be408e640 1 -- 172.21.15.43:0/1036417741 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5bdc0a9fb0 con 0x7f5bdc0b45f0 2023-08-09T13:50:56.530 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:56.524+0000 7f5be408e640 1 -- 172.21.15.43:0/1036417741 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5bdc0739f0 con 0x7f5bdc0b45f0 2023-08-09T13:50:56.530 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:56.526+0000 7f5bd27fc640 1 -- 172.21.15.43:0/1036417741 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 35) v1 ==== 88179+0+0 (secure 0 0 0) 0x7f5bd8003990 con 0x7f5bdc0b45f0 2023-08-09T13:50:56.530 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:56.527+0000 7f5bd27fc640 1 --2- 172.21.15.43:0/1036417741 >> [v2:172.21.15.203:6832/1778614695,v1:172.21.15.203:6833/1778614695] conn(0x7f5bc0069ea0 0x7f5bc006c360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:50:56.530 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:56.527+0000 7f5bd27fc640 1 -- 172.21.15.43:0/1036417741 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(92..92 src has 1..92) v4 ==== 7114+0+0 (secure 0 0 0) 0x7f5bd8087c30 con 0x7f5bdc0b45f0 2023-08-09T13:50:56.530 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:56.528+0000 7f5be1629640 1 --2- 172.21.15.43:0/1036417741 >> [v2:172.21.15.203:6832/1778614695,v1:172.21.15.203:6833/1778614695] conn(0x7f5bc0069ea0 0x7f5bc006c360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:50:56.531 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:56.529+0000 7f5be1629640 1 --2- 172.21.15.43:0/1036417741 >> [v2:172.21.15.203:6832/1778614695,v1:172.21.15.203:6833/1778614695] conn(0x7f5bc0069ea0 0x7f5bc006c360 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7f5bcc036000 tx=0x7f5bcc008030 comp rx=0 tx=0).ready entity=mgr.5819 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-09T13:50:56.534 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:56.532+0000 7f5bd27fc640 1 -- 172.21.15.43:0/1036417741 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178066 (secure 0 0 0) 0x7f5bd8053170 con 0x7f5bdc0b45f0 2023-08-09T13:50:56.725 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:56.723+0000 7f5be408e640 1 -- 172.21.15.43:0/1036417741 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f5bdc0c8bc0 con 0x7f5bdc0b45f0 2023-08-09T13:50:56.726 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:56.723+0000 7f5bd27fc640 1 -- 172.21.15.43:0/1036417741 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v92) v1 ==== 93+0+10 (secure 0 0 0) 0x7f5bd8056790 con 0x7f5bdc0b45f0 2023-08-09T13:50:56.726 INFO:teuthology.orchestra.run.smithi043.stdout:pg_num: 1 2023-08-09T13:50:56.729 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:56.726+0000 7f5be408e640 1 -- 172.21.15.43:0/1036417741 >> [v2:172.21.15.203:6832/1778614695,v1:172.21.15.203:6833/1778614695] conn(0x7f5bc0069ea0 msgr2=0x7f5bc006c360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:50:56.729 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:56.726+0000 7f5be408e640 1 --2- 172.21.15.43:0/1036417741 >> [v2:172.21.15.203:6832/1778614695,v1:172.21.15.203:6833/1778614695] conn(0x7f5bc0069ea0 0x7f5bc006c360 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7f5bcc036000 tx=0x7f5bcc008030 comp rx=0 tx=0).stop 2023-08-09T13:50:56.729 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:56.726+0000 7f5be408e640 1 -- 172.21.15.43:0/1036417741 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f5bdc0b45f0 msgr2=0x7f5bdc159e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:50:56.729 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:56.726+0000 7f5be408e640 1 --2- 172.21.15.43:0/1036417741 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f5bdc0b45f0 0x7f5bdc159e50 secure :-1 s=READY pgs=1091 cs=0 l=1 rev1=1 crypto rx=0x7f5bd800c400 tx=0x7f5bd800c8c0 comp rx=0 tx=0).stop 2023-08-09T13:50:56.729 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:56.726+0000 7f5be408e640 1 -- 172.21.15.43:0/1036417741 shutdown_connections 2023-08-09T13:50:56.729 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:56.726+0000 7f5be408e640 1 --2- 172.21.15.43:0/1036417741 >> [v2:172.21.15.203:6832/1778614695,v1:172.21.15.203:6833/1778614695] conn(0x7f5bc0069ea0 0x7f5bc006c360 unknown :-1 s=CLOSED pgs=20 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:50:56.730 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:56.726+0000 7f5be408e640 1 --2- 172.21.15.43:0/1036417741 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f5bdc15c550 0x7f5bdc0b3330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:50:56.730 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:56.726+0000 7f5be408e640 1 --2- 172.21.15.43:0/1036417741 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f5bdc0d04b0 0x7f5bdc0a8dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:50:56.730 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:56.726+0000 7f5be408e640 1 --2- 172.21.15.43:0/1036417741 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f5bdc0b45f0 0x7f5bdc159e50 unknown :-1 s=CLOSED pgs=1091 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:50:56.730 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:56.726+0000 7f5be408e640 1 -- 172.21.15.43:0/1036417741 >> 172.21.15.43:0/1036417741 conn(0x7f5bdc071520 msgr2=0x7f5bdc15ee70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-09T13:50:56.730 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:56.726+0000 7f5be408e640 1 -- 172.21.15.43:0/1036417741 shutdown_connections 2023-08-09T13:50:56.730 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:56.727+0000 7f5be408e640 1 -- 172.21.15.43:0/1036417741 wait complete. 2023-08-09T13:50:56.748 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-08-09T13:50:56.978 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:56.976+0000 7f1be823f640 1 Processor -- start 2023-08-09T13:50:56.979 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:56.976+0000 7f1be823f640 1 -- start start 2023-08-09T13:50:56.980 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:56.977+0000 7f1be823f640 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7f1be00ce080 0x7f1be00cafa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:50:56.980 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:56.977+0000 7f1be823f640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f1be00cb4e0 0x7f1be00cb970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:50:56.980 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:56.977+0000 7f1be823f640 1 -- --> v1:172.21.15.43:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1be00cbeb0 con 0x7f1be00ce720 2023-08-09T13:50:56.980 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:56.977+0000 7f1be823f640 1 -- --> v2:172.21.15.43:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1be00cd8a0 con 0x7f1be00cb4e0 2023-08-09T13:50:56.980 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:56.977+0000 7f1be823f640 1 -- --> v2:172.21.15.203:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1be00cf350 con 0x7f1be00ce080 2023-08-09T13:50:56.981 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:56.978+0000 7f1be5fdb640 1 --1- >> v1:172.21.15.43:6789/0 conn(0x7f1be00ce720 0x7f1be00ceb00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.43:6789/0 says I am v1:172.21.15.43:35042/0 (socket says 172.21.15.43:35042) 2023-08-09T13:50:56.982 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:56.978+0000 7f1be5fdb640 1 -- 172.21.15.43:0/2504096425 learned_addr learned my addr 172.21.15.43:0/2504096425 (peer_addr_for_me v1:172.21.15.43:0/0) 2023-08-09T13:50:56.982 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:56.978+0000 7f1be57da640 1 --2- 172.21.15.43:0/2504096425 >> v2:172.21.15.203:3300/0 conn(0x7f1be00ce080 0x7f1be00cafa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:50:56.982 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:56.978+0000 7f1be67dc640 1 --2- 172.21.15.43:0/2504096425 >> v2:172.21.15.43:3300/0 conn(0x7f1be00cb4e0 0x7f1be00cb970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:50:56.982 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:56.979+0000 7f1be4fd9640 1 -- 172.21.15.43:0/2504096425 <== mon.0 v1:172.21.15.43:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2040109640 0 0) 0x7f1be00cbeb0 con 0x7f1be00ce720 2023-08-09T13:50:56.982 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:56.979+0000 7f1be4fd9640 1 -- 172.21.15.43:0/2504096425 --> v1:172.21.15.43:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f1bcc003520 con 0x7f1be00ce720 2023-08-09T13:50:56.982 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:56.979+0000 7f1be57da640 1 -- 172.21.15.43:0/2504096425 >> v2:172.21.15.43:3300/0 conn(0x7f1be00cb4e0 msgr2=0x7f1be00cb970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-09T13:50:56.982 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:56.979+0000 7f1be57da640 1 --2- 172.21.15.43:0/2504096425 >> v2:172.21.15.43:3300/0 conn(0x7f1be00cb4e0 0x7f1be00cb970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:50:56.983 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:56.979+0000 7f1be57da640 1 -- 172.21.15.43:0/2504096425 >> v1:172.21.15.43:6789/0 conn(0x7f1be00ce720 legacy=0x7f1be00ceb00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:50:56.983 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:56.979+0000 7f1be57da640 1 -- 172.21.15.43:0/2504096425 --> v2:172.21.15.203:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1be0071a70 con 0x7f1be00ce080 2023-08-09T13:50:56.983 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:56.979+0000 7f1be67dc640 1 --2- 172.21.15.43:0/2504096425 >> v2:172.21.15.43:3300/0 conn(0x7f1be00cb4e0 0x7f1be00cb970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-08-09T13:50:56.984 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:56.980+0000 7f1be57da640 1 --2- 172.21.15.43:0/2504096425 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f1be00ce080 0x7f1be00cafa0 secure :-1 s=READY pgs=400 cs=0 l=1 rev1=1 crypto rx=0x7f1bd00097a0 tx=0x7f1bd002b840 comp rx=0 tx=0).ready entity=mon.1 client_cookie=594a1956ba643a56 server_cookie=0 in_seq=0 out_seq=0 2023-08-09T13:50:56.985 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:56.981+0000 7f1be4fd9640 1 -- 172.21.15.43:0/2504096425 <== mon.1 v2:172.21.15.203:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1bd002e070 con 0x7f1be00ce080 2023-08-09T13:50:56.985 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:56.981+0000 7f1be4fd9640 1 -- 172.21.15.43:0/2504096425 <== mon.1 v2:172.21.15.203:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1bd00315a0 con 0x7f1be00ce080 2023-08-09T13:50:56.985 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:56.981+0000 7f1be4fd9640 1 -- 172.21.15.43:0/2504096425 <== mon.1 v2:172.21.15.203:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1bd0031910 con 0x7f1be00ce080 2023-08-09T13:50:56.985 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:56.981+0000 7f1be823f640 1 -- 172.21.15.43:0/2504096425 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f1be00ce080 msgr2=0x7f1be00cafa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:50:56.985 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:56.981+0000 7f1be823f640 1 --2- 172.21.15.43:0/2504096425 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f1be00ce080 0x7f1be00cafa0 secure :-1 s=READY pgs=400 cs=0 l=1 rev1=1 crypto rx=0x7f1bd00097a0 tx=0x7f1bd002b840 comp rx=0 tx=0).stop 2023-08-09T13:50:56.986 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:56.981+0000 7f1be823f640 1 -- 172.21.15.43:0/2504096425 shutdown_connections 2023-08-09T13:50:56.986 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:56.981+0000 7f1be823f640 1 --2- 172.21.15.43:0/2504096425 >> v2:172.21.15.43:3300/0 conn(0x7f1be00cb4e0 0x7f1be00cb970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:50:56.986 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:56.981+0000 7f1be823f640 1 --2- 172.21.15.43:0/2504096425 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f1be00ce080 0x7f1be00cafa0 unknown :-1 s=CLOSED pgs=400 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:50:56.987 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:56.981+0000 7f1be823f640 1 -- 172.21.15.43:0/2504096425 >> 172.21.15.43:0/2504096425 conn(0x7f1be00ccd80 msgr2=0x7f1be00cd190 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-09T13:50:56.987 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:56.981+0000 7f1be823f640 1 -- 172.21.15.43:0/2504096425 shutdown_connections 2023-08-09T13:50:56.987 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:56.982+0000 7f1be823f640 1 -- 172.21.15.43:0/2504096425 wait complete. 2023-08-09T13:50:56.989 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:56.983+0000 7f1be823f640 1 Processor -- start 2023-08-09T13:50:56.989 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:56.983+0000 7f1be823f640 1 -- start start 2023-08-09T13:50:56.989 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:56.984+0000 7f1be823f640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f1be00cb4e0 0x7f1be00b59a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:50:56.989 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:56.984+0000 7f1be823f640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f1be00ce080 0x7f1be006c2c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:50:56.989 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:56.984+0000 7f1be823f640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f1be00ce720 0x7f1be015b590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:50:56.989 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:56.984+0000 7f1be823f640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1be00c2dd0 con 0x7f1be00cb4e0 2023-08-09T13:50:56.990 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:56.984+0000 7f1be823f640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1be006c800 con 0x7f1be00ce080 2023-08-09T13:50:56.990 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:56.984+0000 7f1be823f640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1be015bad0 con 0x7f1be00ce720 2023-08-09T13:50:56.990 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:56.985+0000 7f1be5fdb640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f1be00cb4e0 0x7f1be00b59a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:50:56.990 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:56.985+0000 7f1be5fdb640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f1be00cb4e0 0x7f1be00b59a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:46568/0 (socket says 172.21.15.43:46568) 2023-08-09T13:50:56.990 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:56.985+0000 7f1be5fdb640 1 -- 172.21.15.43:0/4144365781 learned_addr learned my addr 172.21.15.43:0/4144365781 (peer_addr_for_me v2:172.21.15.43:0/0) 2023-08-09T13:50:56.990 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:56.985+0000 7f1be5fdb640 1 -- 172.21.15.43:0/4144365781 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f1be00ce720 msgr2=0x7f1be015b590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:50:56.990 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:56.985+0000 7f1be57da640 1 --2- 172.21.15.43:0/4144365781 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f1be00ce080 0x7f1be006c2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:50:56.991 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:56.985+0000 7f1be5fdb640 1 --2- 172.21.15.43:0/4144365781 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f1be00ce720 0x7f1be015b590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:50:56.991 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:56.985+0000 7f1be5fdb640 1 -- 172.21.15.43:0/4144365781 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f1be00ce080 msgr2=0x7f1be006c2c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:50:56.991 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:56.985+0000 7f1be5fdb640 1 --2- 172.21.15.43:0/4144365781 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f1be00ce080 0x7f1be006c2c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:50:56.991 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:56.985+0000 7f1be5fdb640 1 -- 172.21.15.43:0/4144365781 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1bd8009400 con 0x7f1be00cb4e0 2023-08-09T13:50:56.991 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:56.985+0000 7f1be57da640 1 --2- 172.21.15.43:0/4144365781 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f1be00ce080 0x7f1be006c2c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-08-09T13:50:56.991 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:56.985+0000 7f1be5fdb640 1 --2- 172.21.15.43:0/4144365781 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f1be00cb4e0 0x7f1be00b59a0 secure :-1 s=READY pgs=1093 cs=0 l=1 rev1=1 crypto rx=0x7f1bdc00c840 tx=0x7f1bdc00cd00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-09T13:50:56.991 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:56.986+0000 7f1bd67fc640 1 -- 172.21.15.43:0/4144365781 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1bdc00a640 con 0x7f1be00cb4e0 2023-08-09T13:50:56.992 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:56.986+0000 7f1bd67fc640 1 -- 172.21.15.43:0/4144365781 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1bdc00a7e0 con 0x7f1be00cb4e0 2023-08-09T13:50:56.992 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:56.986+0000 7f1be823f640 1 -- 172.21.15.43:0/4144365781 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1bd0009400 con 0x7f1be00cb4e0 2023-08-09T13:50:56.992 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:56.986+0000 7f1bd67fc640 1 -- 172.21.15.43:0/4144365781 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1bdc00ab90 con 0x7f1be00cb4e0 2023-08-09T13:50:56.992 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:56.986+0000 7f1be823f640 1 -- 172.21.15.43:0/4144365781 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1be006aa00 con 0x7f1be00cb4e0 2023-08-09T13:50:56.993 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:56.987+0000 7f1be823f640 1 -- 172.21.15.43:0/4144365781 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1be006ac00 con 0x7f1be00cb4e0 2023-08-09T13:50:56.993 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:56.988+0000 7f1bd67fc640 1 -- 172.21.15.43:0/4144365781 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 35) v1 ==== 88179+0+0 (secure 0 0 0) 0x7f1bdc00acf0 con 0x7f1be00cb4e0 2023-08-09T13:50:56.993 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:56.989+0000 7f1bd67fc640 1 --2- 172.21.15.43:0/4144365781 >> [v2:172.21.15.203:6832/1778614695,v1:172.21.15.203:6833/1778614695] conn(0x7f1bc4069ed0 0x7f1bc406c390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:50:56.994 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:56.989+0000 7f1bd67fc640 1 -- 172.21.15.43:0/4144365781 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(92..92 src has 1..92) v4 ==== 7114+0+0 (secure 0 0 0) 0x7f1bdc087ef0 con 0x7f1be00cb4e0 2023-08-09T13:50:56.994 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:56.990+0000 7f1be57da640 1 --2- 172.21.15.43:0/4144365781 >> [v2:172.21.15.203:6832/1778614695,v1:172.21.15.203:6833/1778614695] conn(0x7f1bc4069ed0 0x7f1bc406c390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:50:56.994 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:56.991+0000 7f1be57da640 1 --2- 172.21.15.43:0/4144365781 >> [v2:172.21.15.203:6832/1778614695,v1:172.21.15.203:6833/1778614695] conn(0x7f1bc4069ed0 0x7f1bc406c390 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7f1bd0036000 tx=0x7f1bd003d040 comp rx=0 tx=0).ready entity=mgr.5819 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-09T13:50:56.996 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:56.994+0000 7f1bd67fc640 1 -- 172.21.15.43:0/4144365781 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178066 (secure 0 0 0) 0x7f1be006ac00 con 0x7f1be00cb4e0 2023-08-09T13:50:57.193 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:57.189+0000 7f1be823f640 1 -- 172.21.15.43:0/4144365781 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f1be006ac00 con 0x7f1be00cb4e0 2023-08-09T13:50:57.193 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:57.190+0000 7f1bd67fc640 1 -- 172.21.15.43:0/4144365781 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v92) v1 ==== 104+0+11 (secure 0 0 0) 0x7f1be006ac00 con 0x7f1be00cb4e0 2023-08-09T13:50:57.193 INFO:teuthology.orchestra.run.smithi043.stdout:pg_num: 32 2023-08-09T13:50:57.195 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:57.192+0000 7f1be823f640 1 -- 172.21.15.43:0/4144365781 >> [v2:172.21.15.203:6832/1778614695,v1:172.21.15.203:6833/1778614695] conn(0x7f1bc4069ed0 msgr2=0x7f1bc406c390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:50:57.196 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:57.192+0000 7f1be823f640 1 --2- 172.21.15.43:0/4144365781 >> [v2:172.21.15.203:6832/1778614695,v1:172.21.15.203:6833/1778614695] conn(0x7f1bc4069ed0 0x7f1bc406c390 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7f1bd0036000 tx=0x7f1bd003d040 comp rx=0 tx=0).stop 2023-08-09T13:50:57.196 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:57.193+0000 7f1be823f640 1 -- 172.21.15.43:0/4144365781 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f1be00cb4e0 msgr2=0x7f1be00b59a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:50:57.196 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:57.193+0000 7f1be823f640 1 --2- 172.21.15.43:0/4144365781 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f1be00cb4e0 0x7f1be00b59a0 secure :-1 s=READY pgs=1093 cs=0 l=1 rev1=1 crypto rx=0x7f1bdc00c840 tx=0x7f1bdc00cd00 comp rx=0 tx=0).stop 2023-08-09T13:50:57.196 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:57.193+0000 7f1be823f640 1 -- 172.21.15.43:0/4144365781 shutdown_connections 2023-08-09T13:50:57.196 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:57.193+0000 7f1be823f640 1 --2- 172.21.15.43:0/4144365781 >> [v2:172.21.15.203:6832/1778614695,v1:172.21.15.203:6833/1778614695] conn(0x7f1bc4069ed0 0x7f1bc406c390 unknown :-1 s=CLOSED pgs=22 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:50:57.196 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:57.193+0000 7f1be823f640 1 --2- 172.21.15.43:0/4144365781 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f1be00ce720 0x7f1be015b590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:50:57.197 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:57.193+0000 7f1be823f640 1 --2- 172.21.15.43:0/4144365781 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f1be00ce080 0x7f1be006c2c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:50:57.197 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:57.193+0000 7f1be823f640 1 --2- 172.21.15.43:0/4144365781 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f1be00cb4e0 0x7f1be00b59a0 unknown :-1 s=CLOSED pgs=1093 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:50:57.197 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:57.193+0000 7f1be823f640 1 -- 172.21.15.43:0/4144365781 >> 172.21.15.43:0/4144365781 conn(0x7f1be00ccd80 msgr2=0x7f1be015da00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-09T13:50:57.197 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:57.193+0000 7f1be823f640 1 -- 172.21.15.43:0/4144365781 shutdown_connections 2023-08-09T13:50:57.197 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:57.193+0000 7f1be823f640 1 -- 172.21.15.43:0/4144365781 wait complete. 2023-08-09T13:50:57.216 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-08-09T13:50:57.428 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:57.426+0000 7f6b51fc4640 1 Processor -- start 2023-08-09T13:50:57.429 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:57.426+0000 7f6b51fc4640 1 -- start start 2023-08-09T13:50:57.430 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:57.428+0000 7f6b51fc4640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f6b4c15c550 0x7f6b4c0cd6d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:50:57.430 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:57.428+0000 7f6b51fc4640 1 -- --> v1:172.21.15.43:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6b4c0b7bd0 con 0x7f6b4c0b45f0 2023-08-09T13:50:57.431 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:57.428+0000 7f6b51fc4640 1 -- --> v1:172.21.15.203:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6b4c0d1c00 con 0x7f6b4c0d04b0 2023-08-09T13:50:57.431 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:57.428+0000 7f6b51fc4640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6b4c15c930 con 0x7f6b4c15c550 2023-08-09T13:50:57.432 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:57.428+0000 7f6b4bfff640 1 --1- >> v1:172.21.15.43:6789/0 conn(0x7f6b4c0b45f0 0x7f6b4c0b59d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.43:6789/0 says I am v1:172.21.15.43:35046/0 (socket says 172.21.15.43:35046) 2023-08-09T13:50:57.432 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:57.428+0000 7f6b4bfff640 1 -- 172.21.15.43:0/2135755186 learned_addr learned my addr 172.21.15.43:0/2135755186 (peer_addr_for_me v1:172.21.15.43:0/0) 2023-08-09T13:50:57.432 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:57.429+0000 7f6b4b7fe640 1 --2- 172.21.15.43:0/2135755186 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f6b4c15c550 0x7f6b4c0cd6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:50:57.432 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:57.429+0000 7f6b4a7fc640 1 -- 172.21.15.43:0/2135755186 <== mon.0 v1:172.21.15.43:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1395672798 0 0) 0x7f6b4c0b7bd0 con 0x7f6b4c0b45f0 2023-08-09T13:50:57.432 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:57.429+0000 7f6b4a7fc640 1 -- 172.21.15.43:0/2135755186 --> v1:172.21.15.43:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f6b38003520 con 0x7f6b4c0b45f0 2023-08-09T13:50:57.433 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:57.429+0000 7f6b4a7fc640 1 -- 172.21.15.43:0/2135755186 <== mon.0 v1:172.21.15.43:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3155484606 0 0) 0x7f6b38003520 con 0x7f6b4c0b45f0 2023-08-09T13:50:57.433 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:57.429+0000 7f6b4a7fc640 1 -- 172.21.15.43:0/2135755186 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f6b4c15c550 msgr2=0x7f6b4c0cd6d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-09T13:50:57.433 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:57.429+0000 7f6b4a7fc640 1 --2- 172.21.15.43:0/2135755186 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f6b4c15c550 0x7f6b4c0cd6d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:50:57.433 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:57.429+0000 7f6b4a7fc640 1 -- 172.21.15.43:0/2135755186 >> v1:172.21.15.203:6789/0 conn(0x7f6b4c0d04b0 legacy=0x7f6b4c0b3ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:50:57.433 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:57.429+0000 7f6b4a7fc640 1 -- 172.21.15.43:0/2135755186 --> v1:172.21.15.43:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6b4c0b54f0 con 0x7f6b4c0b45f0 2023-08-09T13:50:57.433 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:57.429+0000 7f6b4a7fc640 1 -- 172.21.15.43:0/2135755186 <== mon.0 v1:172.21.15.43:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1285150740 0 0) 0x7f6b440019f0 con 0x7f6b4c0b45f0 2023-08-09T13:50:57.433 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:57.429+0000 7f6b4a7fc640 1 -- 172.21.15.43:0/2135755186 >> v1:172.21.15.43:6789/0 conn(0x7f6b4c0b45f0 legacy=0x7f6b4c0b59d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:50:57.433 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:57.429+0000 7f6b4a7fc640 1 --2- 172.21.15.43:0/2135755186 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f6b4c15c550 0x7f6b380039a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:50:57.434 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:57.429+0000 7f6b4a7fc640 1 --2- 172.21.15.43:0/2135755186 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f6b38003f20 0x7f6b380047c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:50:57.434 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:57.429+0000 7f6b4a7fc640 1 --2- 172.21.15.43:0/2135755186 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f6b38004d00 0x7f6b38005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:50:57.434 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:57.429+0000 7f6b4a7fc640 1 -- 172.21.15.43:0/2135755186 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6b4c15c930 con 0x7f6b38003f20 2023-08-09T13:50:57.434 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:57.429+0000 7f6b4a7fc640 1 -- 172.21.15.43:0/2135755186 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6b3800ba60 con 0x7f6b38004d00 2023-08-09T13:50:57.434 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:57.429+0000 7f6b4a7fc640 1 -- 172.21.15.43:0/2135755186 --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6b3800bc20 con 0x7f6b4c15c550 2023-08-09T13:50:57.435 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:57.430+0000 7f6b4affd640 1 --2- 172.21.15.43:0/2135755186 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f6b38003f20 0x7f6b380047c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:50:57.435 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:57.430+0000 7f6b4b7fe640 1 --2- 172.21.15.43:0/2135755186 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f6b4c15c550 0x7f6b380039a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:50:57.435 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:57.430+0000 7f6b4bfff640 1 --2- 172.21.15.43:0/2135755186 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f6b38004d00 0x7f6b38005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:50:57.435 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:57.430+0000 7f6b4affd640 1 -- 172.21.15.43:0/2135755186 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f6b4c15c550 msgr2=0x7f6b380039a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-09T13:50:57.436 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:57.430+0000 7f6b4affd640 1 --2- 172.21.15.43:0/2135755186 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f6b4c15c550 0x7f6b380039a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:50:57.436 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:57.430+0000 7f6b4affd640 1 -- 172.21.15.43:0/2135755186 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f6b38004d00 msgr2=0x7f6b38005180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-09T13:50:57.436 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:57.430+0000 7f6b4affd640 1 --2- 172.21.15.43:0/2135755186 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f6b38004d00 0x7f6b38005180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:50:57.436 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:57.430+0000 7f6b4affd640 1 -- 172.21.15.43:0/2135755186 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6b38003520 con 0x7f6b38003f20 2023-08-09T13:50:57.436 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:57.431+0000 7f6b4affd640 1 --2- 172.21.15.43:0/2135755186 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f6b38003f20 0x7f6b380047c0 secure :-1 s=READY pgs=1095 cs=0 l=1 rev1=1 crypto rx=0x7f6b34001750 tx=0x7f6b380057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c71979ec96c4a094 server_cookie=0 in_seq=0 out_seq=0 2023-08-09T13:50:57.437 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:57.433+0000 7f6b4a7fc640 1 -- 172.21.15.43:0/2135755186 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6b340052d0 con 0x7f6b38003f20 2023-08-09T13:50:57.437 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:57.433+0000 7f6b4a7fc640 1 -- 172.21.15.43:0/2135755186 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6b3402e050 con 0x7f6b38003f20 2023-08-09T13:50:57.438 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:57.433+0000 7f6b4a7fc640 1 -- 172.21.15.43:0/2135755186 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6b34003830 con 0x7f6b38003f20 2023-08-09T13:50:57.438 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:57.434+0000 7f6b51fc4640 1 -- 172.21.15.43:0/2135755186 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f6b38003f20 msgr2=0x7f6b380047c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:50:57.438 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:57.434+0000 7f6b51fc4640 1 --2- 172.21.15.43:0/2135755186 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f6b38003f20 0x7f6b380047c0 secure :-1 s=READY pgs=1095 cs=0 l=1 rev1=1 crypto rx=0x7f6b34001750 tx=0x7f6b380057f0 comp rx=0 tx=0).stop 2023-08-09T13:50:57.438 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:57.434+0000 7f6b4bfff640 1 -- 172.21.15.43:0/2135755186 reap_dead start 2023-08-09T13:50:57.438 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:57.434+0000 7f6b51fc4640 1 -- 172.21.15.43:0/2135755186 shutdown_connections 2023-08-09T13:50:57.438 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:57.434+0000 7f6b51fc4640 1 -- 172.21.15.43:0/2135755186 >> 172.21.15.43:0/2135755186 conn(0x7f6b4c071520 msgr2=0x7f6b4c0ceee0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-09T13:50:57.438 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:57.434+0000 7f6b51fc4640 1 -- 172.21.15.43:0/2135755186 shutdown_connections 2023-08-09T13:50:57.438 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:57.434+0000 7f6b51fc4640 1 -- 172.21.15.43:0/2135755186 wait complete. 2023-08-09T13:50:57.439 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:57.436+0000 7f6b51fc4640 1 Processor -- start 2023-08-09T13:50:57.439 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:57.436+0000 7f6b51fc4640 1 -- start start 2023-08-09T13:50:57.440 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:57.437+0000 7f6b51fc4640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f6b38003f20 0x7f6b4c0b3850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:50:57.440 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:57.437+0000 7f6b51fc4640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f6b4c1f5f40 0x7f6b4c0ccd20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:50:57.441 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:57.437+0000 7f6b51fc4640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f6b4c0cd260 0x7f6b4c0b5bb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:50:57.441 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:57.437+0000 7f6b51fc4640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6b4c1f63c0 con 0x7f6b38003f20 2023-08-09T13:50:57.441 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:57.437+0000 7f6b51fc4640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6b4c1f54f0 con 0x7f6b4c1f5f40 2023-08-09T13:50:57.441 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:57.437+0000 7f6b51fc4640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6b4c0cd880 con 0x7f6b4c0cd260 2023-08-09T13:50:57.441 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:57.437+0000 7f6b4b7fe640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f6b38003f20 0x7f6b4c0b3850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:50:57.441 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:57.437+0000 7f6b4b7fe640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f6b38003f20 0x7f6b4c0b3850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:46578/0 (socket says 172.21.15.43:46578) 2023-08-09T13:50:57.442 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:57.437+0000 7f6b4b7fe640 1 -- 172.21.15.43:0/4157598081 learned_addr learned my addr 172.21.15.43:0/4157598081 (peer_addr_for_me v2:172.21.15.43:0/0) 2023-08-09T13:50:57.442 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:57.438+0000 7f6b4affd640 1 --2- 172.21.15.43:0/4157598081 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f6b4c1f5f40 0x7f6b4c0ccd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:50:57.442 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:57.438+0000 7f6b4bfff640 1 --2- 172.21.15.43:0/4157598081 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f6b4c0cd260 0x7f6b4c0b5bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:50:57.442 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:57.438+0000 7f6b4b7fe640 1 -- 172.21.15.43:0/4157598081 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f6b4c0cd260 msgr2=0x7f6b4c0b5bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:50:57.442 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:57.438+0000 7f6b4b7fe640 1 --2- 172.21.15.43:0/4157598081 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f6b4c0cd260 0x7f6b4c0b5bb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:50:57.442 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:57.438+0000 7f6b4b7fe640 1 -- 172.21.15.43:0/4157598081 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f6b4c1f5f40 msgr2=0x7f6b4c0ccd20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:50:57.443 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:57.438+0000 7f6b4b7fe640 1 --2- 172.21.15.43:0/4157598081 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f6b4c1f5f40 0x7f6b4c0ccd20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:50:57.443 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:57.438+0000 7f6b4b7fe640 1 -- 172.21.15.43:0/4157598081 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6b4c0cdb60 con 0x7f6b38003f20 2023-08-09T13:50:57.444 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:57.438+0000 7f6b4bfff640 1 --2- 172.21.15.43:0/4157598081 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f6b4c0cd260 0x7f6b4c0b5bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-08-09T13:50:57.444 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:57.438+0000 7f6b4b7fe640 1 --2- 172.21.15.43:0/4157598081 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f6b38003f20 0x7f6b4c0b3850 secure :-1 s=READY pgs=1096 cs=0 l=1 rev1=1 crypto rx=0x7f6b3c011600 tx=0x7f6b3c011ac0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-09T13:50:57.444 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:57.440+0000 7f6b43fff640 1 -- 172.21.15.43:0/4157598081 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6b3c00f8c0 con 0x7f6b38003f20 2023-08-09T13:50:57.444 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:57.440+0000 7f6b51fc4640 1 -- 172.21.15.43:0/4157598081 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6b4c1f5740 con 0x7f6b38003f20 2023-08-09T13:50:57.444 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:57.440+0000 7f6b43fff640 1 -- 172.21.15.43:0/4157598081 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6b3c00fa20 con 0x7f6b38003f20 2023-08-09T13:50:57.445 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:57.440+0000 7f6b43fff640 1 -- 172.21.15.43:0/4157598081 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6b3c00fd90 con 0x7f6b38003f20 2023-08-09T13:50:57.445 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:57.440+0000 7f6b51fc4640 1 -- 172.21.15.43:0/4157598081 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6b4c1519f0 con 0x7f6b38003f20 2023-08-09T13:50:57.445 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:57.441+0000 7f6b51fc4640 1 -- 172.21.15.43:0/4157598081 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6b4c0739f0 con 0x7f6b38003f20 2023-08-09T13:50:57.446 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:57.442+0000 7f6b43fff640 1 -- 172.21.15.43:0/4157598081 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 35) v1 ==== 88179+0+0 (secure 0 0 0) 0x7f6b3c015460 con 0x7f6b38003f20 2023-08-09T13:50:57.446 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:57.443+0000 7f6b43fff640 1 --2- 172.21.15.43:0/4157598081 >> [v2:172.21.15.203:6832/1778614695,v1:172.21.15.203:6833/1778614695] conn(0x7f6b2c069ed0 0x7f6b2c06c390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:50:57.446 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:57.443+0000 7f6b43fff640 1 -- 172.21.15.43:0/4157598081 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(92..92 src has 1..92) v4 ==== 7114+0+0 (secure 0 0 0) 0x7f6b3c088f40 con 0x7f6b38003f20 2023-08-09T13:50:57.447 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:57.444+0000 7f6b4affd640 1 --2- 172.21.15.43:0/4157598081 >> [v2:172.21.15.203:6832/1778614695,v1:172.21.15.203:6833/1778614695] conn(0x7f6b2c069ed0 0x7f6b2c06c390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:50:57.448 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:57.445+0000 7f6b4affd640 1 --2- 172.21.15.43:0/4157598081 >> [v2:172.21.15.203:6832/1778614695,v1:172.21.15.203:6833/1778614695] conn(0x7f6b2c069ed0 0x7f6b2c06c390 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7f6b34033000 tx=0x7f6b34009030 comp rx=0 tx=0).ready entity=mgr.5819 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-09T13:50:57.451 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:57.448+0000 7f6b43fff640 1 -- 172.21.15.43:0/4157598081 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178066 (secure 0 0 0) 0x7f6b3c054480 con 0x7f6b38003f20 2023-08-09T13:50:57.654 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:57.651+0000 7f6b51fc4640 1 -- 172.21.15.43:0/4157598081 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f6b4c0cdcf0 con 0x7f6b38003f20 2023-08-09T13:50:57.655 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:57.652+0000 7f6b43fff640 1 -- 172.21.15.43:0/4157598081 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v92) v1 ==== 100+0+11 (secure 0 0 0) 0x7f6b3c057aa0 con 0x7f6b38003f20 2023-08-09T13:50:57.655 INFO:teuthology.orchestra.run.smithi043.stdout:pg_num: 64 2023-08-09T13:50:57.658 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:57.655+0000 7f6b51fc4640 1 -- 172.21.15.43:0/4157598081 >> [v2:172.21.15.203:6832/1778614695,v1:172.21.15.203:6833/1778614695] conn(0x7f6b2c069ed0 msgr2=0x7f6b2c06c390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:50:57.658 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:57.655+0000 7f6b51fc4640 1 --2- 172.21.15.43:0/4157598081 >> [v2:172.21.15.203:6832/1778614695,v1:172.21.15.203:6833/1778614695] conn(0x7f6b2c069ed0 0x7f6b2c06c390 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7f6b34033000 tx=0x7f6b34009030 comp rx=0 tx=0).stop 2023-08-09T13:50:57.658 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:57.655+0000 7f6b51fc4640 1 -- 172.21.15.43:0/4157598081 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f6b38003f20 msgr2=0x7f6b4c0b3850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:50:57.658 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:57.655+0000 7f6b51fc4640 1 --2- 172.21.15.43:0/4157598081 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f6b38003f20 0x7f6b4c0b3850 secure :-1 s=READY pgs=1096 cs=0 l=1 rev1=1 crypto rx=0x7f6b3c011600 tx=0x7f6b3c011ac0 comp rx=0 tx=0).stop 2023-08-09T13:50:57.658 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:57.655+0000 7f6b51fc4640 1 -- 172.21.15.43:0/4157598081 shutdown_connections 2023-08-09T13:50:57.658 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:57.655+0000 7f6b51fc4640 1 --2- 172.21.15.43:0/4157598081 >> [v2:172.21.15.203:6832/1778614695,v1:172.21.15.203:6833/1778614695] conn(0x7f6b2c069ed0 0x7f6b2c06c390 unknown :-1 s=CLOSED pgs=23 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:50:57.659 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:57.655+0000 7f6b51fc4640 1 --2- 172.21.15.43:0/4157598081 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f6b4c0cd260 0x7f6b4c0b5bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:50:57.659 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:57.655+0000 7f6b51fc4640 1 --2- 172.21.15.43:0/4157598081 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f6b4c1f5f40 0x7f6b4c0ccd20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:50:57.659 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:57.655+0000 7f6b51fc4640 1 --2- 172.21.15.43:0/4157598081 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f6b38003f20 0x7f6b4c0b3850 unknown :-1 s=CLOSED pgs=1096 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:50:57.659 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:57.655+0000 7f6b51fc4640 1 -- 172.21.15.43:0/4157598081 >> 172.21.15.43:0/4157598081 conn(0x7f6b4c071520 msgr2=0x7f6b4c0ceee0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-09T13:50:57.659 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:57.655+0000 7f6b51fc4640 1 -- 172.21.15.43:0/4157598081 shutdown_connections 2023-08-09T13:50:57.659 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:57.655+0000 7f6b51fc4640 1 -- 172.21.15.43:0/4157598081 wait complete. 2023-08-09T13:50:57.677 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-08-09T13:50:57.906 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:57.904+0000 7fb1992da640 1 Processor -- start 2023-08-09T13:50:57.907 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:57.904+0000 7fb1992da640 1 -- start start 2023-08-09T13:50:57.908 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:57.905+0000 7fb1992da640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fb1940b2b70 0x7fb1940d2ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:50:57.908 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:57.905+0000 7fb1992da640 1 -- --> v1:172.21.15.43:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb1940ce090 con 0x7fb1940b3050 2023-08-09T13:50:57.908 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:57.905+0000 7fb1992da640 1 -- --> v1:172.21.15.203:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb1940cd7f0 con 0x7fb1940b2790 2023-08-09T13:50:57.908 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:57.905+0000 7fb1992da640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb1940b3950 con 0x7fb1940b2b70 2023-08-09T13:50:57.910 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:57.906+0000 7fb1937fe640 1 --1- >> v1:172.21.15.43:6789/0 conn(0x7fb1940b3050 0x7fb1940b3f60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.43:6789/0 says I am v1:172.21.15.43:35060/0 (socket says 172.21.15.43:35060) 2023-08-09T13:50:57.910 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:57.906+0000 7fb1937fe640 1 -- 172.21.15.43:0/612672126 learned_addr learned my addr 172.21.15.43:0/612672126 (peer_addr_for_me v1:172.21.15.43:0/0) 2023-08-09T13:50:57.910 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:57.906+0000 7fb1927fc640 1 --2- 172.21.15.43:0/612672126 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fb1940b2b70 0x7fb1940d2ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:50:57.910 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:57.906+0000 7fb191ffb640 1 -- 172.21.15.43:0/612672126 <== mon.0 v1:172.21.15.43:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3309747601 0 0) 0x7fb1940ce090 con 0x7fb1940b3050 2023-08-09T13:50:57.910 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:57.906+0000 7fb191ffb640 1 -- 172.21.15.43:0/612672126 --> v1:172.21.15.43:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fb17c003520 con 0x7fb1940b3050 2023-08-09T13:50:57.911 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:57.906+0000 7fb191ffb640 1 -- 172.21.15.43:0/612672126 <== mon.0 v1:172.21.15.43:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1709220514 0 0) 0x7fb17c003520 con 0x7fb1940b3050 2023-08-09T13:50:57.911 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:57.906+0000 7fb191ffb640 1 -- 172.21.15.43:0/612672126 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fb1940b2b70 msgr2=0x7fb1940d2ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-09T13:50:57.911 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:57.906+0000 7fb191ffb640 1 --2- 172.21.15.43:0/612672126 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fb1940b2b70 0x7fb1940d2ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:50:57.911 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:57.906+0000 7fb191ffb640 1 -- 172.21.15.43:0/612672126 >> v1:172.21.15.203:6789/0 conn(0x7fb1940b2790 legacy=0x7fb1940b5330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:50:57.911 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:57.907+0000 7fb191ffb640 1 -- 172.21.15.43:0/612672126 --> v1:172.21.15.43:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb1940cdad0 con 0x7fb1940b3050 2023-08-09T13:50:57.912 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:57.907+0000 7fb191ffb640 1 -- 172.21.15.43:0/612672126 <== mon.0 v1:172.21.15.43:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1285150740 0 0) 0x7fb18c001670 con 0x7fb1940b3050 2023-08-09T13:50:57.912 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:57.907+0000 7fb191ffb640 1 -- 172.21.15.43:0/612672126 >> v1:172.21.15.43:6789/0 conn(0x7fb1940b3050 legacy=0x7fb1940b3f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:50:57.912 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:57.907+0000 7fb191ffb640 1 --2- 172.21.15.43:0/612672126 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fb17c003a10 0x7fb17c003e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:50:57.912 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:57.907+0000 7fb191ffb640 1 --2- 172.21.15.43:0/612672126 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb17c0047c0 0x7fb17c004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:50:57.912 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:57.907+0000 7fb191ffb640 1 --2- 172.21.15.43:0/612672126 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fb17c0051d0 0x7fb17c00b980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:50:57.913 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:57.907+0000 7fb191ffb640 1 -- 172.21.15.43:0/612672126 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb1940b3950 con 0x7fb17c0047c0 2023-08-09T13:50:57.913 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:57.907+0000 7fb191ffb640 1 -- 172.21.15.43:0/612672126 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb17c00c0b0 con 0x7fb17c003a10 2023-08-09T13:50:57.913 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:57.907+0000 7fb191ffb640 1 -- 172.21.15.43:0/612672126 --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb17c00c270 con 0x7fb17c0051d0 2023-08-09T13:50:57.913 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:57.907+0000 7fb1927fc640 1 --2- 172.21.15.43:0/612672126 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb17c0047c0 0x7fb17c004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:50:57.913 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:57.907+0000 7fb1937fe640 1 --2- 172.21.15.43:0/612672126 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fb17c0051d0 0x7fb17c00b980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:50:57.913 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:57.907+0000 7fb192ffd640 1 --2- 172.21.15.43:0/612672126 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fb17c003a10 0x7fb17c003e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:50:57.914 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:57.907+0000 7fb1927fc640 1 -- 172.21.15.43:0/612672126 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fb17c0051d0 msgr2=0x7fb17c00b980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-09T13:50:57.914 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:57.907+0000 7fb1927fc640 1 --2- 172.21.15.43:0/612672126 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fb17c0051d0 0x7fb17c00b980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:50:57.914 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:57.908+0000 7fb1927fc640 1 -- 172.21.15.43:0/612672126 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fb17c003a10 msgr2=0x7fb17c003e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-09T13:50:57.914 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:57.908+0000 7fb1927fc640 1 --2- 172.21.15.43:0/612672126 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fb17c003a10 0x7fb17c003e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:50:57.914 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:57.908+0000 7fb1927fc640 1 -- 172.21.15.43:0/612672126 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb17c003520 con 0x7fb17c0047c0 2023-08-09T13:50:57.914 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:57.908+0000 7fb192ffd640 1 --2- 172.21.15.43:0/612672126 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fb17c003a10 0x7fb17c003e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-08-09T13:50:57.916 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:57.909+0000 7fb1927fc640 1 --2- 172.21.15.43:0/612672126 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb17c0047c0 0x7fb17c004c40 secure :-1 s=READY pgs=1098 cs=0 l=1 rev1=1 crypto rx=0x7fb1940a03f0 tx=0x7fb17c0057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6e7f8806b37a5ef7 server_cookie=0 in_seq=0 out_seq=0 2023-08-09T13:50:57.917 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:57.909+0000 7fb191ffb640 1 -- 172.21.15.43:0/612672126 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb18802cbb0 con 0x7fb17c0047c0 2023-08-09T13:50:57.917 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:57.909+0000 7fb191ffb640 1 -- 172.21.15.43:0/612672126 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb18802cd10 con 0x7fb17c0047c0 2023-08-09T13:50:57.917 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:57.909+0000 7fb191ffb640 1 -- 172.21.15.43:0/612672126 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb188034660 con 0x7fb17c0047c0 2023-08-09T13:50:57.917 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:57.910+0000 7fb1992da640 1 -- 172.21.15.43:0/612672126 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb17c0047c0 msgr2=0x7fb17c004c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:50:57.917 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:57.910+0000 7fb1992da640 1 --2- 172.21.15.43:0/612672126 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb17c0047c0 0x7fb17c004c40 secure :-1 s=READY pgs=1098 cs=0 l=1 rev1=1 crypto rx=0x7fb1940a03f0 tx=0x7fb17c0057f0 comp rx=0 tx=0).stop 2023-08-09T13:50:57.918 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:57.910+0000 7fb1937fe640 1 -- 172.21.15.43:0/612672126 reap_dead start 2023-08-09T13:50:57.918 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:57.910+0000 7fb1992da640 1 -- 172.21.15.43:0/612672126 shutdown_connections 2023-08-09T13:50:57.918 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:57.910+0000 7fb1992da640 1 -- 172.21.15.43:0/612672126 >> 172.21.15.43:0/612672126 conn(0x7fb19409b7b0 msgr2=0x7fb1940d24e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-09T13:50:57.918 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:57.910+0000 7fb1992da640 1 -- 172.21.15.43:0/612672126 shutdown_connections 2023-08-09T13:50:57.918 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:57.910+0000 7fb1992da640 1 -- 172.21.15.43:0/612672126 wait complete. 2023-08-09T13:50:57.919 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:57.912+0000 7fb1992da640 1 Processor -- start 2023-08-09T13:50:57.919 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:57.912+0000 7fb1992da640 1 -- start start 2023-08-09T13:50:57.919 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:57.913+0000 7fb1992da640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb17c0047c0 0x7fb19415f570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:50:57.919 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:57.913+0000 7fb1992da640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fb194159d20 0x7fb194148d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:50:57.919 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:57.913+0000 7fb1992da640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fb19406a5f0 0x7fb19406aa80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:50:57.919 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:57.913+0000 7fb1992da640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb19406afc0 con 0x7fb17c0047c0 2023-08-09T13:50:57.919 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:57.913+0000 7fb1992da640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb19414c5f0 con 0x7fb194159d20 2023-08-09T13:50:57.919 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:57.913+0000 7fb1992da640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb19414ba70 con 0x7fb19406a5f0 2023-08-09T13:50:57.920 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:57.913+0000 7fb192ffd640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb17c0047c0 0x7fb19415f570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:50:57.920 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:57.913+0000 7fb192ffd640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb17c0047c0 0x7fb19415f570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:46592/0 (socket says 172.21.15.43:46592) 2023-08-09T13:50:57.920 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:57.913+0000 7fb192ffd640 1 -- 172.21.15.43:0/881629386 learned_addr learned my addr 172.21.15.43:0/881629386 (peer_addr_for_me v2:172.21.15.43:0/0) 2023-08-09T13:50:57.920 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:57.914+0000 7fb1937fe640 1 --2- 172.21.15.43:0/881629386 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fb19406a5f0 0x7fb19406aa80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:50:57.920 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:57.914+0000 7fb192ffd640 1 -- 172.21.15.43:0/881629386 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fb19406a5f0 msgr2=0x7fb19406aa80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:50:57.920 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:57.914+0000 7fb192ffd640 1 --2- 172.21.15.43:0/881629386 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fb19406a5f0 0x7fb19406aa80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:50:57.920 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:57.914+0000 7fb192ffd640 1 -- 172.21.15.43:0/881629386 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fb194159d20 msgr2=0x7fb194148d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:50:57.921 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:57.914+0000 7fb192ffd640 1 --2- 172.21.15.43:0/881629386 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fb194159d20 0x7fb194148d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:50:57.921 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:57.914+0000 7fb192ffd640 1 -- 172.21.15.43:0/881629386 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb19414bd50 con 0x7fb17c0047c0 2023-08-09T13:50:57.921 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:57.914+0000 7fb192ffd640 1 --2- 172.21.15.43:0/881629386 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb17c0047c0 0x7fb19415f570 secure :-1 s=READY pgs=1099 cs=0 l=1 rev1=1 crypto rx=0x7fb18400d7a0 tx=0x7fb18400dc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-09T13:50:57.922 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:57.914+0000 7fb17b7fe640 1 -- 172.21.15.43:0/881629386 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb184017070 con 0x7fb17c0047c0 2023-08-09T13:50:57.922 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:57.915+0000 7fb17b7fe640 1 -- 172.21.15.43:0/881629386 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb184005070 con 0x7fb17c0047c0 2023-08-09T13:50:57.923 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:57.915+0000 7fb17b7fe640 1 -- 172.21.15.43:0/881629386 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb184005420 con 0x7fb17c0047c0 2023-08-09T13:50:57.923 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:57.915+0000 7fb1992da640 1 -- 172.21.15.43:0/881629386 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb19414c030 con 0x7fb17c0047c0 2023-08-09T13:50:57.923 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:57.915+0000 7fb1992da640 1 -- 172.21.15.43:0/881629386 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb194149240 con 0x7fb17c0047c0 2023-08-09T13:50:57.923 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:57.916+0000 7fb1992da640 1 -- 172.21.15.43:0/881629386 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb1940739f0 con 0x7fb17c0047c0 2023-08-09T13:50:57.923 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:57.917+0000 7fb17b7fe640 1 -- 172.21.15.43:0/881629386 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 35) v1 ==== 88179+0+0 (secure 0 0 0) 0x7fb184007030 con 0x7fb17c0047c0 2023-08-09T13:50:57.923 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:57.918+0000 7fb17b7fe640 1 --2- 172.21.15.43:0/881629386 >> [v2:172.21.15.203:6832/1778614695,v1:172.21.15.203:6833/1778614695] conn(0x7fb180069ea0 0x7fb18006c360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:50:57.923 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:57.918+0000 7fb17b7fe640 1 -- 172.21.15.43:0/881629386 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(92..92 src has 1..92) v4 ==== 7114+0+0 (secure 0 0 0) 0x7fb184087f50 con 0x7fb17c0047c0 2023-08-09T13:50:57.923 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:57.919+0000 7fb1927fc640 1 --2- 172.21.15.43:0/881629386 >> [v2:172.21.15.203:6832/1778614695,v1:172.21.15.203:6833/1778614695] conn(0x7fb180069ea0 0x7fb18006c360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:50:57.924 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:57.920+0000 7fb1927fc640 1 --2- 172.21.15.43:0/881629386 >> [v2:172.21.15.203:6832/1778614695,v1:172.21.15.203:6833/1778614695] conn(0x7fb180069ea0 0x7fb18006c360 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7fb188036000 tx=0x7fb18802e000 comp rx=0 tx=0).ready entity=mgr.5819 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-09T13:50:57.924 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:57.922+0000 7fb17b7fe640 1 -- 172.21.15.43:0/881629386 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178066 (secure 0 0 0) 0x7fb184053490 con 0x7fb17c0047c0 2023-08-09T13:50:58.116 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:58.113+0000 7fb17b7fe640 1 -- 172.21.15.43:0/881629386 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mgrmap(e 36) v1 ==== 88387+0+0 (secure 0 0 0) 0x7fb184053860 con 0x7fb17c0047c0 2023-08-09T13:50:58.158 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:58.156+0000 7fb1992da640 1 -- 172.21.15.43:0/881629386 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fb19406c1b0 con 0x7fb17c0047c0 2023-08-09T13:50:58.159 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:58.157+0000 7fb17b7fe640 1 -- 172.21.15.43:0/881629386 <== mon.0 v2:172.21.15.43:3300/0 8 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 35 v35) v1 ==== 94+0+4880 (secure 0 0 0) 0x7fb184056f20 con 0x7fb17c0047c0 2023-08-09T13:50:58.160 INFO:teuthology.orchestra.run.smithi043.stdout: 2023-08-09T13:50:58.160 INFO:teuthology.orchestra.run.smithi043.stdout:{"epoch":35,"default_fscid":4,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5654,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.203:6837/1227342478","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6835","nonce":1227342478},{"type":"v1","addr":"172.21.15.203:6837","nonce":1227342478}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":30},{"gid":7005,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.43:6834/1431309163","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6832","nonce":1431309163},{"type":"v1","addr":"172.21.15.43:6834","nonce":1431309163}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":29},{"gid":7008,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.43:6835/611328204","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6833","nonce":611328204},{"type":"v1","addr":"172.21.15.43:6835","nonce":611328204}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":30}],"filesystems":[{"mdsmap":{"epoch":35,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-08-09T13:50:28.672581+0000","modified":"2023-08-09T13:50:32.253466+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":7011},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7011":{"gid":7011,"name":"d","rank":0,"incarnation":31,"state":"up:active","state_seq":3,"addr":"172.21.15.203:6836/3061388209","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6834","nonce":3061388209},{"type":"v1","addr":"172.21.15.203:6836","nonce":3061388209}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[11],"metadata_pool":10,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":4}]} 2023-08-09T13:50:58.161 INFO:teuthology.orchestra.run.smithi043.stderr:dumped fsmap epoch 35 2023-08-09T13:50:58.163 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:58.160+0000 7fb1992da640 1 -- 172.21.15.43:0/881629386 >> [v2:172.21.15.203:6832/1778614695,v1:172.21.15.203:6833/1778614695] conn(0x7fb180069ea0 msgr2=0x7fb18006c360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:50:58.163 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:58.160+0000 7fb1992da640 1 --2- 172.21.15.43:0/881629386 >> [v2:172.21.15.203:6832/1778614695,v1:172.21.15.203:6833/1778614695] conn(0x7fb180069ea0 0x7fb18006c360 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7fb188036000 tx=0x7fb18802e000 comp rx=0 tx=0).stop 2023-08-09T13:50:58.163 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:58.160+0000 7fb1992da640 1 -- 172.21.15.43:0/881629386 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb17c0047c0 msgr2=0x7fb19415f570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:50:58.163 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:58.160+0000 7fb1992da640 1 --2- 172.21.15.43:0/881629386 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb17c0047c0 0x7fb19415f570 secure :-1 s=READY pgs=1099 cs=0 l=1 rev1=1 crypto rx=0x7fb18400d7a0 tx=0x7fb18400dc60 comp rx=0 tx=0).stop 2023-08-09T13:50:58.163 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:58.160+0000 7fb1992da640 1 -- 172.21.15.43:0/881629386 shutdown_connections 2023-08-09T13:50:58.164 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:58.160+0000 7fb1992da640 1 --2- 172.21.15.43:0/881629386 >> [v2:172.21.15.203:6832/1778614695,v1:172.21.15.203:6833/1778614695] conn(0x7fb180069ea0 0x7fb18006c360 unknown :-1 s=CLOSED pgs=25 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:50:58.164 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:58.160+0000 7fb1992da640 1 --2- 172.21.15.43:0/881629386 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fb19406a5f0 0x7fb19406aa80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:50:58.164 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:58.160+0000 7fb1992da640 1 --2- 172.21.15.43:0/881629386 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fb194159d20 0x7fb194148d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:50:58.164 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:58.160+0000 7fb1992da640 1 --2- 172.21.15.43:0/881629386 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb17c0047c0 0x7fb19415f570 unknown :-1 s=CLOSED pgs=1099 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:50:58.164 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:58.160+0000 7fb1992da640 1 -- 172.21.15.43:0/881629386 >> 172.21.15.43:0/881629386 conn(0x7fb19409b7b0 msgr2=0x7fb19415a6f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-09T13:50:58.164 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:58.160+0000 7fb1992da640 1 -- 172.21.15.43:0/881629386 shutdown_connections 2023-08-09T13:50:58.164 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:58.160+0000 7fb1992da640 1 -- 172.21.15.43:0/881629386 wait complete. 2023-08-09T13:50:58.182 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-08-09T13:50:58.400 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:58.398+0000 7f2d719b5640 1 Processor -- start 2023-08-09T13:50:58.401 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:58.398+0000 7f2d719b5640 1 -- start start 2023-08-09T13:50:58.402 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:58.399+0000 7f2d719b5640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f2d6c14a5c0 0x7f2d6c160f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:50:58.402 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:58.399+0000 7f2d719b5640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f2d6c0cc380 0x7f2d6c06bb50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:50:58.402 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:58.399+0000 7f2d719b5640 1 -- --> v1:172.21.15.203:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2d6c0c35e0 con 0x7f2d6c0c3130 2023-08-09T13:50:58.402 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:58.399+0000 7f2d719b5640 1 -- --> v2:172.21.15.43:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2d6c0c4420 con 0x7f2d6c14a5c0 2023-08-09T13:50:58.403 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:58.399+0000 7f2d719b5640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2d6c0c38a0 con 0x7f2d6c0cc380 2023-08-09T13:50:58.403 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:58.400+0000 7f2d6affd640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f2d6c14a5c0 0x7f2d6c160f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:50:58.403 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:58.400+0000 7f2d6affd640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f2d6c14a5c0 0x7f2d6c160f20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:46608/0 (socket says 172.21.15.43:46608) 2023-08-09T13:50:58.403 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:58.400+0000 7f2d6affd640 1 -- 172.21.15.43:0/3758292314 learned_addr learned my addr 172.21.15.43:0/3758292314 (peer_addr_for_me v2:172.21.15.43:0/0) 2023-08-09T13:50:58.404 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:58.400+0000 7f2d6b7fe640 1 --2- 172.21.15.43:0/3758292314 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f2d6c0cc380 0x7f2d6c06bb50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:50:58.404 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:58.400+0000 7f2d6affd640 1 -- 172.21.15.43:0/3758292314 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f2d6c0cc380 msgr2=0x7f2d6c06bb50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-09T13:50:58.404 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:58.400+0000 7f2d6affd640 1 --2- 172.21.15.43:0/3758292314 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f2d6c0cc380 0x7f2d6c06bb50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:50:58.404 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:58.400+0000 7f2d6affd640 1 -- 172.21.15.43:0/3758292314 >> v1:172.21.15.203:6789/0 conn(0x7f2d6c0c3130 legacy=0x7f2d6c0cbc70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:50:58.404 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:58.400+0000 7f2d6affd640 1 -- 172.21.15.43:0/3758292314 --> v2:172.21.15.43:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2d6c0c3b80 con 0x7f2d6c14a5c0 2023-08-09T13:50:58.405 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:58.402+0000 7f2d6affd640 1 --2- 172.21.15.43:0/3758292314 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f2d6c14a5c0 0x7f2d6c160f20 secure :-1 s=READY pgs=1100 cs=0 l=1 rev1=1 crypto rx=0x7f2d5c0098a0 tx=0x7f2d5c02b8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4c6af57552876973 server_cookie=0 in_seq=0 out_seq=0 2023-08-09T13:50:58.405 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:58.402+0000 7f2d69ffb640 1 -- 172.21.15.43:0/3758292314 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2d5c031640 con 0x7f2d6c14a5c0 2023-08-09T13:50:58.405 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:58.402+0000 7f2d69ffb640 1 -- 172.21.15.43:0/3758292314 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2d5c0317a0 con 0x7f2d6c14a5c0 2023-08-09T13:50:58.405 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:58.402+0000 7f2d69ffb640 1 -- 172.21.15.43:0/3758292314 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2d5c031b30 con 0x7f2d6c14a5c0 2023-08-09T13:50:58.406 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:58.403+0000 7f2d719b5640 1 -- 172.21.15.43:0/3758292314 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f2d6c14a5c0 msgr2=0x7f2d6c160f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:50:58.406 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:58.403+0000 7f2d719b5640 1 --2- 172.21.15.43:0/3758292314 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f2d6c14a5c0 0x7f2d6c160f20 secure :-1 s=READY pgs=1100 cs=0 l=1 rev1=1 crypto rx=0x7f2d5c0098a0 tx=0x7f2d5c02b8f0 comp rx=0 tx=0).stop 2023-08-09T13:50:58.406 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:58.403+0000 7f2d719b5640 1 -- 172.21.15.43:0/3758292314 shutdown_connections 2023-08-09T13:50:58.406 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:58.403+0000 7f2d719b5640 1 --2- 172.21.15.43:0/3758292314 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f2d6c0cc380 0x7f2d6c06bb50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:50:58.407 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:58.403+0000 7f2d719b5640 1 --2- 172.21.15.43:0/3758292314 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f2d6c14a5c0 0x7f2d6c160f20 unknown :-1 s=CLOSED pgs=1100 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:50:58.407 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:58.403+0000 7f2d719b5640 1 -- 172.21.15.43:0/3758292314 >> 172.21.15.43:0/3758292314 conn(0x7f2d6c0ce0a0 msgr2=0x7f2d6c0ce480 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-09T13:50:58.407 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:58.403+0000 7f2d719b5640 1 -- 172.21.15.43:0/3758292314 shutdown_connections 2023-08-09T13:50:58.407 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:58.403+0000 7f2d719b5640 1 -- 172.21.15.43:0/3758292314 wait complete. 2023-08-09T13:50:58.407 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:58.405+0000 7f2d719b5640 1 Processor -- start 2023-08-09T13:50:58.408 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:58.405+0000 7f2d719b5640 1 -- start start 2023-08-09T13:50:58.408 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:58.406+0000 7f2d719b5640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f2d6c0c3130 0x7f2d6c1438f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:50:58.409 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:58.406+0000 7f2d719b5640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f2d6c0cc380 0x7f2d6c0b1900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:50:58.409 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:58.406+0000 7f2d719b5640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f2d6c14a5c0 0x7f2d6c1620c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:50:58.409 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:58.406+0000 7f2d719b5640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2d6c146fe0 con 0x7f2d6c0cc380 2023-08-09T13:50:58.409 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:58.406+0000 7f2d719b5640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2d6c162f10 con 0x7f2d6c14a5c0 2023-08-09T13:50:58.409 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:58.406+0000 7f2d719b5640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2d6c162c30 con 0x7f2d6c0c3130 2023-08-09T13:50:58.410 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:58.406+0000 7f2d6a7fc640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f2d6c0cc380 0x7f2d6c0b1900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:50:58.411 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:58.407+0000 7f2d6a7fc640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f2d6c0cc380 0x7f2d6c0b1900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:46610/0 (socket says 172.21.15.43:46610) 2023-08-09T13:50:58.411 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:58.407+0000 7f2d6a7fc640 1 -- 172.21.15.43:0/1194582920 learned_addr learned my addr 172.21.15.43:0/1194582920 (peer_addr_for_me v2:172.21.15.43:0/0) 2023-08-09T13:50:58.411 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:58.407+0000 7f2d6affd640 1 --2- 172.21.15.43:0/1194582920 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f2d6c0c3130 0x7f2d6c1438f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:50:58.411 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:58.407+0000 7f2d6b7fe640 1 --2- 172.21.15.43:0/1194582920 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f2d6c14a5c0 0x7f2d6c1620c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:50:58.411 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:58.407+0000 7f2d6a7fc640 1 -- 172.21.15.43:0/1194582920 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f2d6c0c3130 msgr2=0x7f2d6c1438f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:50:58.411 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:58.407+0000 7f2d6a7fc640 1 --2- 172.21.15.43:0/1194582920 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f2d6c0c3130 0x7f2d6c1438f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:50:58.412 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:58.407+0000 7f2d6a7fc640 1 -- 172.21.15.43:0/1194582920 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f2d6c14a5c0 msgr2=0x7f2d6c1620c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:50:58.412 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:58.407+0000 7f2d6a7fc640 1 --2- 172.21.15.43:0/1194582920 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f2d6c14a5c0 0x7f2d6c1620c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:50:58.412 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:58.407+0000 7f2d6a7fc640 1 -- 172.21.15.43:0/1194582920 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2d5c0094d0 con 0x7f2d6c0cc380 2023-08-09T13:50:58.412 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:58.407+0000 7f2d6a7fc640 1 --2- 172.21.15.43:0/1194582920 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f2d6c0cc380 0x7f2d6c0b1900 secure :-1 s=READY pgs=1101 cs=0 l=1 rev1=1 crypto rx=0x7f2d6000c790 tx=0x7f2d6000cc50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-09T13:50:58.412 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:58.408+0000 7f2d53fff640 1 -- 172.21.15.43:0/1194582920 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2d6000cea0 con 0x7f2d6c0cc380 2023-08-09T13:50:58.412 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:58.408+0000 7f2d53fff640 1 -- 172.21.15.43:0/1194582920 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2d60006d50 con 0x7f2d6c0cc380 2023-08-09T13:50:58.412 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:58.408+0000 7f2d53fff640 1 -- 172.21.15.43:0/1194582920 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2d60003bb0 con 0x7f2d6c0cc380 2023-08-09T13:50:58.413 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:58.408+0000 7f2d719b5640 1 -- 172.21.15.43:0/1194582920 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2d6c0b0e20 con 0x7f2d6c0cc380 2023-08-09T13:50:58.413 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:58.408+0000 7f2d719b5640 1 -- 172.21.15.43:0/1194582920 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2d6c0b13e0 con 0x7f2d6c0cc380 2023-08-09T13:50:58.413 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:58.409+0000 7f2d719b5640 1 -- 172.21.15.43:0/1194582920 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2d6c0739f0 con 0x7f2d6c0cc380 2023-08-09T13:50:58.413 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:58.410+0000 7f2d53fff640 1 -- 172.21.15.43:0/1194582920 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 36) v1 ==== 88387+0+0 (secure 0 0 0) 0x7f2d60003d10 con 0x7f2d6c0cc380 2023-08-09T13:50:58.414 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:58.411+0000 7f2d53fff640 1 --2- 172.21.15.43:0/1194582920 >> [v2:172.21.15.203:6832/1778614695,v1:172.21.15.203:6833/1778614695] conn(0x7f2d5806a0e0 0x7f2d5806c5a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:50:58.414 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:58.411+0000 7f2d53fff640 1 -- 172.21.15.43:0/1194582920 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(92..92 src has 1..92) v4 ==== 7114+0+0 (secure 0 0 0) 0x7f2d600885b0 con 0x7f2d6c0cc380 2023-08-09T13:50:58.414 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:58.412+0000 7f2d6affd640 1 --2- 172.21.15.43:0/1194582920 >> [v2:172.21.15.203:6832/1778614695,v1:172.21.15.203:6833/1778614695] conn(0x7f2d5806a0e0 0x7f2d5806c5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:50:58.415 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:58.413+0000 7f2d6affd640 1 --2- 172.21.15.43:0/1194582920 >> [v2:172.21.15.203:6832/1778614695,v1:172.21.15.203:6833/1778614695] conn(0x7f2d5806a0e0 0x7f2d5806c5a0 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7f2d5c036000 tx=0x7f2d5c03c040 comp rx=0 tx=0).ready entity=mgr.5819 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-09T13:50:58.418 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:58.415+0000 7f2d53fff640 1 -- 172.21.15.43:0/1194582920 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178066 (secure 0 0 0) 0x7f2d600573b0 con 0x7f2d6c0cc380 2023-08-09T13:50:58.615 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:58.612+0000 7f2d719b5640 1 -- 172.21.15.43:0/1194582920 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f2d6c0cc830 con 0x7f2d6c0cc380 2023-08-09T13:50:58.616 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:58.614+0000 7f2d53fff640 1 -- 172.21.15.43:0/1194582920 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v92) v1 ==== 74+0+19573 (secure 0 0 0) 0x7f2d60056f40 con 0x7f2d6c0cc380 2023-08-09T13:50:58.617 INFO:teuthology.orchestra.run.smithi043.stdout: 2023-08-09T13:50:58.618 INFO:teuthology.orchestra.run.smithi043.stdout:{"epoch":92,"fsid":"daac87ee-1337-4250-a955-aea4e408a39b","created":"2023-08-09T13:33:41.701198+0000","modified":"2023-08-09T13:50:55.386567+0000","last_up_change":"2023-08-09T13:33:53.783453+0000","last_in_change":"2023-08-09T13:33:45.148904+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":11,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":"rbd","create_time":"2023-08-09T13:33:54.365137+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":".mgr","create_time":"2023-08-09T13:33:54.831803+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":10,"pool_name":"cephfs_metadata","create_time":"2023-08-09T13:50:27.051781+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"89","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":2.5,"score_stable":2.5,"optimal_score":1,"raw_score_acting":2.5,"raw_score_stable":2.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":11,"pool_name":"cephfs_data","create_time":"2023-08-09T13:50:27.692933+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"90","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"cbd49ecd-e896-4fbd-864e-5c37a829b165","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6800","nonce":502667335},{"type":"v1","addr":"172.21.15.43:6801","nonce":502667335}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6802","nonce":502667335},{"type":"v1","addr":"172.21.15.43:6803","nonce":502667335}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6806","nonce":502667335},{"type":"v1","addr":"172.21.15.43:6807","nonce":502667335}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6804","nonce":502667335},{"type":"v1","addr":"172.21.15.43:6805","nonce":502667335}]},"public_addr":"172.21.15.43:6801/502667335","cluster_addr":"172.21.15.43:6803/502667335","heartbeat_back_addr":"172.21.15.43:6807/502667335","heartbeat_front_addr":"172.21.15.43:6805/502667335","state":["exists","up"]},{"osd":1,"uuid":"75c7e0fe-1ec7-4fe8-a2f2-3661a65b8500","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6808","nonce":3289134610},{"type":"v1","addr":"172.21.15.43:6809","nonce":3289134610}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6810","nonce":3289134610},{"type":"v1","addr":"172.21.15.43:6812","nonce":3289134610}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6821","nonce":3289134610},{"type":"v1","addr":"172.21.15.43:6824","nonce":3289134610}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6815","nonce":3289134610},{"type":"v1","addr":"172.21.15.43:6818","nonce":3289134610}]},"public_addr":"172.21.15.43:6809/3289134610","cluster_addr":"172.21.15.43:6812/3289134610","heartbeat_back_addr":"172.21.15.43:6824/3289134610","heartbeat_front_addr":"172.21.15.43:6818/3289134610","state":["exists","up"]},{"osd":2,"uuid":"bb5c41e9-7f5f-4125-9b01-3bb8bfc2ea83","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6811","nonce":4079709736},{"type":"v1","addr":"172.21.15.43:6814","nonce":4079709736}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6817","nonce":4079709736},{"type":"v1","addr":"172.21.15.43:6820","nonce":4079709736}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6829","nonce":4079709736},{"type":"v1","addr":"172.21.15.43:6831","nonce":4079709736}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6823","nonce":4079709736},{"type":"v1","addr":"172.21.15.43:6826","nonce":4079709736}]},"public_addr":"172.21.15.43:6814/4079709736","cluster_addr":"172.21.15.43:6820/4079709736","heartbeat_back_addr":"172.21.15.43:6831/4079709736","heartbeat_front_addr":"172.21.15.43:6826/4079709736","state":["exists","up"]},{"osd":3,"uuid":"bfe4c8b4-694a-465c-87e6-2b2aab852613","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6813","nonce":1709538759},{"type":"v1","addr":"172.21.15.43:6816","nonce":1709538759}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6819","nonce":1709538759},{"type":"v1","addr":"172.21.15.43:6822","nonce":1709538759}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6828","nonce":1709538759},{"type":"v1","addr":"172.21.15.43:6830","nonce":1709538759}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6825","nonce":1709538759},{"type":"v1","addr":"172.21.15.43:6827","nonce":1709538759}]},"public_addr":"172.21.15.43:6816/1709538759","cluster_addr":"172.21.15.43:6822/1709538759","heartbeat_back_addr":"172.21.15.43:6830/1709538759","heartbeat_front_addr":"172.21.15.43:6827/1709538759","state":["exists","up"]},{"osd":4,"uuid":"fa2e8a77-aba1-4756-853e-0d174a5bdeba","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6800","nonce":4196866481},{"type":"v1","addr":"172.21.15.203:6801","nonce":4196866481}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6802","nonce":4196866481},{"type":"v1","addr":"172.21.15.203:6803","nonce":4196866481}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6806","nonce":4196866481},{"type":"v1","addr":"172.21.15.203:6807","nonce":4196866481}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6804","nonce":4196866481},{"type":"v1","addr":"172.21.15.203:6805","nonce":4196866481}]},"public_addr":"172.21.15.203:6801/4196866481","cluster_addr":"172.21.15.203:6803/4196866481","heartbeat_back_addr":"172.21.15.203:6807/4196866481","heartbeat_front_addr":"172.21.15.203:6805/4196866481","state":["exists","up"]},{"osd":5,"uuid":"81811e4b-4d24-4f51-a82c-f1d700a2a00d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6816","nonce":3059290081},{"type":"v1","addr":"172.21.15.203:6817","nonce":3059290081}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6818","nonce":3059290081},{"type":"v1","addr":"172.21.15.203:6819","nonce":3059290081}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6822","nonce":3059290081},{"type":"v1","addr":"172.21.15.203:6823","nonce":3059290081}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6820","nonce":3059290081},{"type":"v1","addr":"172.21.15.203:6821","nonce":3059290081}]},"public_addr":"172.21.15.203:6817/3059290081","cluster_addr":"172.21.15.203:6819/3059290081","heartbeat_back_addr":"172.21.15.203:6823/3059290081","heartbeat_front_addr":"172.21.15.203:6821/3059290081","state":["exists","up"]},{"osd":6,"uuid":"307bac4d-6a5b-443d-af75-56d695d7674f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6824","nonce":512155569},{"type":"v1","addr":"172.21.15.203:6825","nonce":512155569}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6826","nonce":512155569},{"type":"v1","addr":"172.21.15.203:6827","nonce":512155569}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6830","nonce":512155569},{"type":"v1","addr":"172.21.15.203:6831","nonce":512155569}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6828","nonce":512155569},{"type":"v1","addr":"172.21.15.203:6829","nonce":512155569}]},"public_addr":"172.21.15.203:6825/512155569","cluster_addr":"172.21.15.203:6827/512155569","heartbeat_back_addr":"172.21.15.203:6831/512155569","heartbeat_front_addr":"172.21.15.203:6829/512155569","state":["exists","up"]},{"osd":7,"uuid":"a95dc7ca-9f1b-4295-af70-68a22a2d7d5e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6808","nonce":3331450438},{"type":"v1","addr":"172.21.15.203:6809","nonce":3331450438}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6810","nonce":3331450438},{"type":"v1","addr":"172.21.15.203:6811","nonce":3331450438}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6814","nonce":3331450438},{"type":"v1","addr":"172.21.15.203:6815","nonce":3331450438}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6812","nonce":3331450438},{"type":"v1","addr":"172.21.15.203:6813","nonce":3331450438}]},"public_addr":"172.21.15.203:6809/3331450438","cluster_addr":"172.21.15.203:6811/3331450438","heartbeat_back_addr":"172.21.15.203:6815/3331450438","heartbeat_front_addr":"172.21.15.203:6813/3331450438","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-09T13:33:50.547652+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-09T13:33:50.589469+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-09T13:33:50.561724+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-09T13:33:50.599650+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-09T13:33:50.411855+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-09T13:33:50.219681+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-09T13:33:50.504664+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-09T13:33:50.441248+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.203:0/375511470":"2023-08-10T13:50:55.386522+0000","172.21.15.203:0/4256110232":"2023-08-10T13:50:55.386522+0000","172.21.15.203:0/3102402128":"2023-08-10T13:50:55.386522+0000","172.21.15.203:0/1112175379":"2023-08-10T13:50:55.386522+0000","172.21.15.203:6833/3851880723":"2023-08-10T13:50:55.386522+0000","172.21.15.203:6833/3325141227":"2023-08-10T13:50:49.168196+0000","172.21.15.203:6832/3325141227":"2023-08-10T13:50:49.168196+0000","172.21.15.203:0/1343982026":"2023-08-10T13:50:55.386522+0000","172.21.15.203:0/3732830113":"2023-08-10T13:50:49.168196+0000","172.21.15.203:0/2753554568":"2023-08-10T13:50:49.168196+0000","172.21.15.203:0/3841129997":"2023-08-10T13:50:55.386522+0000","172.21.15.203:0/2468196984":"2023-08-10T13:50:49.168196+0000","172.21.15.203:6832/3851880723":"2023-08-10T13:50:55.386522+0000","172.21.15.203:0/3375551329":"2023-08-10T13:50:49.168196+0000","172.21.15.203:0/1259705306":"2023-08-10T13:50:49.168196+0000","172.21.15.203:0/1322546009":"2023-08-10T13:50:49.168196+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":7,"snaps":[{"begin":104,"length":1},{"begin":221,"length":1}]}],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-08-09T13:50:58.620 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:58.616+0000 7f2d719b5640 1 -- 172.21.15.43:0/1194582920 >> [v2:172.21.15.203:6832/1778614695,v1:172.21.15.203:6833/1778614695] conn(0x7f2d5806a0e0 msgr2=0x7f2d5806c5a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:50:58.620 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:58.616+0000 7f2d719b5640 1 --2- 172.21.15.43:0/1194582920 >> [v2:172.21.15.203:6832/1778614695,v1:172.21.15.203:6833/1778614695] conn(0x7f2d5806a0e0 0x7f2d5806c5a0 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7f2d5c036000 tx=0x7f2d5c03c040 comp rx=0 tx=0).stop 2023-08-09T13:50:58.621 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:58.616+0000 7f2d719b5640 1 -- 172.21.15.43:0/1194582920 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f2d6c0cc380 msgr2=0x7f2d6c0b1900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:50:58.621 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:58.616+0000 7f2d719b5640 1 --2- 172.21.15.43:0/1194582920 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f2d6c0cc380 0x7f2d6c0b1900 secure :-1 s=READY pgs=1101 cs=0 l=1 rev1=1 crypto rx=0x7f2d6000c790 tx=0x7f2d6000cc50 comp rx=0 tx=0).stop 2023-08-09T13:50:58.621 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:58.617+0000 7f2d719b5640 1 -- 172.21.15.43:0/1194582920 shutdown_connections 2023-08-09T13:50:58.622 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:58.617+0000 7f2d719b5640 1 --2- 172.21.15.43:0/1194582920 >> [v2:172.21.15.203:6832/1778614695,v1:172.21.15.203:6833/1778614695] conn(0x7f2d5806a0e0 0x7f2d5806c5a0 unknown :-1 s=CLOSED pgs=26 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:50:58.622 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:58.617+0000 7f2d719b5640 1 --2- 172.21.15.43:0/1194582920 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f2d6c14a5c0 0x7f2d6c1620c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:50:58.622 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:58.617+0000 7f2d719b5640 1 --2- 172.21.15.43:0/1194582920 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f2d6c0cc380 0x7f2d6c0b1900 unknown :-1 s=CLOSED pgs=1101 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:50:58.622 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:58.617+0000 7f2d719b5640 1 --2- 172.21.15.43:0/1194582920 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f2d6c0c3130 0x7f2d6c1438f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:50:58.623 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:58.617+0000 7f2d719b5640 1 -- 172.21.15.43:0/1194582920 >> 172.21.15.43:0/1194582920 conn(0x7f2d6c0ce0a0 msgr2=0x7f2d6c071150 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-09T13:50:58.623 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:58.617+0000 7f2d719b5640 1 -- 172.21.15.43:0/1194582920 shutdown_connections 2023-08-09T13:50:58.623 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:58.617+0000 7f2d719b5640 1 -- 172.21.15.43:0/1194582920 wait complete. 2023-08-09T13:50:58.639 INFO:tasks.cephfs.filesystem:Destroying file system cephfs and related pools 2023-08-09T13:50:58.640 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-08-09T13:50:58.868 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:58.865+0000 7f7b6ef0f640 1 Processor -- start 2023-08-09T13:50:58.868 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:58.866+0000 7f7b6ef0f640 1 -- start start 2023-08-09T13:50:58.870 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:58.867+0000 7f7b6ef0f640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f7b680ccf20 0x7f7b680cd550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:50:58.870 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:58.867+0000 7f7b6ef0f640 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7f7b680cda90 0x7f7b6815ca10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:50:58.870 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:58.867+0000 7f7b6ef0f640 1 -- --> v1:172.21.15.43:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7b680cca90 con 0x7f7b6815c630 2023-08-09T13:50:58.870 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:58.867+0000 7f7b6ef0f640 1 -- --> v2:172.21.15.203:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7b680ac600 con 0x7f7b680cda90 2023-08-09T13:50:58.870 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:58.867+0000 7f7b6ef0f640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7b680cbc70 con 0x7f7b680ccf20 2023-08-09T13:50:58.871 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:58.868+0000 7f7b6ccab640 1 --1- >> v1:172.21.15.43:6789/0 conn(0x7f7b6815c630 0x7f7b680abef0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.43:6789/0 says I am v1:172.21.15.43:35070/0 (socket says 172.21.15.43:35070) 2023-08-09T13:50:58.871 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:58.868+0000 7f7b6ccab640 1 -- 172.21.15.43:0/4103286343 learned_addr learned my addr 172.21.15.43:0/4103286343 (peer_addr_for_me v1:172.21.15.43:0/0) 2023-08-09T13:50:58.872 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:58.868+0000 7f7b67fff640 1 --2- 172.21.15.43:0/4103286343 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f7b680ccf20 0x7f7b680cd550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:50:58.873 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:58.868+0000 7f7b677fe640 1 -- 172.21.15.43:0/4103286343 <== mon.0 v1:172.21.15.43:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4127704152 0 0) 0x7f7b680cca90 con 0x7f7b6815c630 2023-08-09T13:50:58.873 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:58.868+0000 7f7b677fe640 1 -- 172.21.15.43:0/4103286343 --> v1:172.21.15.43:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f7b50003520 con 0x7f7b6815c630 2023-08-09T13:50:58.873 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:58.868+0000 7f7b6d4ac640 1 --2- 172.21.15.43:0/4103286343 >> v2:172.21.15.203:3300/0 conn(0x7f7b680cda90 0x7f7b6815ca10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:50:58.873 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:58.868+0000 7f7b677fe640 1 -- 172.21.15.43:0/4103286343 <== mon.0 v1:172.21.15.43:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1258673050 0 0) 0x7f7b50003520 con 0x7f7b6815c630 2023-08-09T13:50:58.873 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:58.868+0000 7f7b677fe640 1 -- 172.21.15.43:0/4103286343 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f7b680ccf20 msgr2=0x7f7b680cd550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-09T13:50:58.873 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:58.868+0000 7f7b677fe640 1 --2- 172.21.15.43:0/4103286343 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f7b680ccf20 0x7f7b680cd550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:50:58.874 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:58.868+0000 7f7b677fe640 1 -- 172.21.15.43:0/4103286343 >> v2:172.21.15.203:3300/0 conn(0x7f7b680cda90 msgr2=0x7f7b6815ca10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-09T13:50:58.874 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:58.868+0000 7f7b677fe640 1 --2- 172.21.15.43:0/4103286343 >> v2:172.21.15.203:3300/0 conn(0x7f7b680cda90 0x7f7b6815ca10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:50:58.874 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:58.868+0000 7f7b677fe640 1 -- 172.21.15.43:0/4103286343 --> v1:172.21.15.43:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7b680cbe80 con 0x7f7b6815c630 2023-08-09T13:50:58.874 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:58.869+0000 7f7b677fe640 1 -- 172.21.15.43:0/4103286343 <== mon.0 v1:172.21.15.43:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1285150740 0 0) 0x7f7b58001670 con 0x7f7b6815c630 2023-08-09T13:50:58.874 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:58.869+0000 7f7b677fe640 1 -- 172.21.15.43:0/4103286343 >> v1:172.21.15.43:6789/0 conn(0x7f7b6815c630 legacy=0x7f7b680abef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:50:58.875 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:58.869+0000 7f7b677fe640 1 --2- 172.21.15.43:0/4103286343 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f7b500039e0 0x7f7b50003e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:50:58.875 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:58.869+0000 7f7b677fe640 1 --2- 172.21.15.43:0/4103286343 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f7b680ccf20 0x7f7b500047c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:50:58.875 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:58.869+0000 7f7b677fe640 1 --2- 172.21.15.43:0/4103286343 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f7b50004d00 0x7f7b50005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:50:58.875 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:58.869+0000 7f7b677fe640 1 -- 172.21.15.43:0/4103286343 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7b680ac600 con 0x7f7b50004d00 2023-08-09T13:50:58.876 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:58.869+0000 7f7b677fe640 1 -- 172.21.15.43:0/4103286343 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7b680cbc70 con 0x7f7b500039e0 2023-08-09T13:50:58.876 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:58.869+0000 7f7b677fe640 1 -- 172.21.15.43:0/4103286343 --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7b5000ba00 con 0x7f7b680ccf20 2023-08-09T13:50:58.876 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:58.869+0000 7f7b6d4ac640 1 --2- 172.21.15.43:0/4103286343 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f7b50004d00 0x7f7b50005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:50:58.876 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:58.869+0000 7f7b6ccab640 1 --2- 172.21.15.43:0/4103286343 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f7b500039e0 0x7f7b50003e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:50:58.877 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:58.869+0000 7f7b6d4ac640 1 -- 172.21.15.43:0/4103286343 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f7b680ccf20 msgr2=0x7f7b500047c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-09T13:50:58.877 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:58.869+0000 7f7b6d4ac640 1 --2- 172.21.15.43:0/4103286343 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f7b680ccf20 0x7f7b500047c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:50:58.877 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:58.869+0000 7f7b6d4ac640 1 -- 172.21.15.43:0/4103286343 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f7b500039e0 msgr2=0x7f7b50003e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-09T13:50:58.877 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:58.869+0000 7f7b6d4ac640 1 --2- 172.21.15.43:0/4103286343 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f7b500039e0 0x7f7b50003e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:50:58.877 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:58.869+0000 7f7b6d4ac640 1 -- 172.21.15.43:0/4103286343 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7b50003520 con 0x7f7b50004d00 2023-08-09T13:50:58.877 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:58.869+0000 7f7b6ccab640 1 --2- 172.21.15.43:0/4103286343 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f7b500039e0 0x7f7b50003e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-08-09T13:50:58.879 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:58.871+0000 7f7b6d4ac640 1 --2- 172.21.15.43:0/4103286343 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f7b50004d00 0x7f7b50005180 secure :-1 s=READY pgs=1103 cs=0 l=1 rev1=1 crypto rx=0x7f7b60007360 tx=0x7f7b500057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9ead2a55fabaad2a server_cookie=0 in_seq=0 out_seq=0 2023-08-09T13:50:58.880 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:58.871+0000 7f7b677fe640 1 -- 172.21.15.43:0/4103286343 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7b6002ccf0 con 0x7f7b50004d00 2023-08-09T13:50:58.880 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:58.871+0000 7f7b677fe640 1 -- 172.21.15.43:0/4103286343 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7b6002ce50 con 0x7f7b50004d00 2023-08-09T13:50:58.880 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:58.871+0000 7f7b677fe640 1 -- 172.21.15.43:0/4103286343 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7b60034660 con 0x7f7b50004d00 2023-08-09T13:50:58.880 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:58.872+0000 7f7b6ef0f640 1 -- 172.21.15.43:0/4103286343 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f7b50004d00 msgr2=0x7f7b50005180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:50:58.880 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:58.872+0000 7f7b6ef0f640 1 --2- 172.21.15.43:0/4103286343 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f7b50004d00 0x7f7b50005180 secure :-1 s=READY pgs=1103 cs=0 l=1 rev1=1 crypto rx=0x7f7b60007360 tx=0x7f7b500057f0 comp rx=0 tx=0).stop 2023-08-09T13:50:58.880 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:58.872+0000 7f7b6d4ac640 1 -- 172.21.15.43:0/4103286343 reap_dead start 2023-08-09T13:50:58.880 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:58.872+0000 7f7b6ef0f640 1 -- 172.21.15.43:0/4103286343 shutdown_connections 2023-08-09T13:50:58.881 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:58.872+0000 7f7b6ef0f640 1 -- 172.21.15.43:0/4103286343 >> 172.21.15.43:0/4103286343 conn(0x7f7b680ce0a0 msgr2=0x7f7b680ce480 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-09T13:50:58.881 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:58.872+0000 7f7b6ef0f640 1 -- 172.21.15.43:0/4103286343 shutdown_connections 2023-08-09T13:50:58.881 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:58.872+0000 7f7b6ef0f640 1 -- 172.21.15.43:0/4103286343 wait complete. 2023-08-09T13:50:58.881 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:58.874+0000 7f7b6ef0f640 1 Processor -- start 2023-08-09T13:50:58.881 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:58.874+0000 7f7b6ef0f640 1 -- start start 2023-08-09T13:50:58.881 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:58.875+0000 7f7b6ef0f640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f7b50004d00 0x7f7b680b2330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:50:58.881 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:58.875+0000 7f7b6ef0f640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f7b680aedc0 0x7f7b6815ed80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:50:58.882 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:58.875+0000 7f7b6ef0f640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f7b680af240 0x7f7b6815f2c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:50:58.882 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:58.875+0000 7f7b6ef0f640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7b6815f800 con 0x7f7b680aedc0 2023-08-09T13:50:58.882 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:58.875+0000 7f7b6ef0f640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7b680b1600 con 0x7f7b50004d00 2023-08-09T13:50:58.882 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:58.875+0000 7f7b6ef0f640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7b681f2820 con 0x7f7b680af240 2023-08-09T13:50:58.882 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:58.875+0000 7f7b67fff640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f7b680aedc0 0x7f7b6815ed80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:50:58.882 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:58.875+0000 7f7b67fff640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f7b680aedc0 0x7f7b6815ed80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:46628/0 (socket says 172.21.15.43:46628) 2023-08-09T13:50:58.882 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:58.875+0000 7f7b67fff640 1 -- 172.21.15.43:0/512332348 learned_addr learned my addr 172.21.15.43:0/512332348 (peer_addr_for_me v2:172.21.15.43:0/0) 2023-08-09T13:50:58.883 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:58.875+0000 7f7b6d4ac640 1 --2- 172.21.15.43:0/512332348 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f7b680af240 0x7f7b6815f2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:50:58.883 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:58.875+0000 7f7b6ccab640 1 --2- 172.21.15.43:0/512332348 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f7b50004d00 0x7f7b680b2330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:50:58.883 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:58.876+0000 7f7b67fff640 1 -- 172.21.15.43:0/512332348 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f7b680af240 msgr2=0x7f7b6815f2c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:50:58.883 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:58.876+0000 7f7b67fff640 1 --2- 172.21.15.43:0/512332348 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f7b680af240 0x7f7b6815f2c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:50:58.883 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:58.876+0000 7f7b67fff640 1 -- 172.21.15.43:0/512332348 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f7b50004d00 msgr2=0x7f7b680b2330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:50:58.883 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:58.876+0000 7f7b67fff640 1 --2- 172.21.15.43:0/512332348 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f7b50004d00 0x7f7b680b2330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:50:58.884 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:58.876+0000 7f7b67fff640 1 -- 172.21.15.43:0/512332348 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7b680b1850 con 0x7f7b680aedc0 2023-08-09T13:50:58.884 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:58.876+0000 7f7b67fff640 1 --2- 172.21.15.43:0/512332348 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f7b680aedc0 0x7f7b6815ed80 secure :-1 s=READY pgs=1104 cs=0 l=1 rev1=1 crypto rx=0x7f7b5c00e7f0 tx=0x7f7b5c00ecb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-09T13:50:58.884 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:58.876+0000 7f7b657fa640 1 -- 172.21.15.43:0/512332348 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7b5c00eea0 con 0x7f7b680aedc0 2023-08-09T13:50:58.884 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:58.876+0000 7f7b657fa640 1 -- 172.21.15.43:0/512332348 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7b5c00c940 con 0x7f7b680aedc0 2023-08-09T13:50:58.884 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:58.876+0000 7f7b657fa640 1 -- 172.21.15.43:0/512332348 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7b5c00ccd0 con 0x7f7b680aedc0 2023-08-09T13:50:58.885 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:58.876+0000 7f7b6ef0f640 1 -- 172.21.15.43:0/512332348 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7b680b1b30 con 0x7f7b680aedc0 2023-08-09T13:50:58.886 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:58.876+0000 7f7b6ef0f640 1 -- 172.21.15.43:0/512332348 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7b681f1af0 con 0x7f7b680aedc0 2023-08-09T13:50:58.887 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:58.878+0000 7f7b6ef0f640 1 -- 172.21.15.43:0/512332348 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7b680739f0 con 0x7f7b680aedc0 2023-08-09T13:50:58.888 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:58.878+0000 7f7b657fa640 1 -- 172.21.15.43:0/512332348 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 36) v1 ==== 88387+0+0 (secure 0 0 0) 0x7f7b5c013470 con 0x7f7b680aedc0 2023-08-09T13:50:58.888 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:58.879+0000 7f7b657fa640 1 --2- 172.21.15.43:0/512332348 >> [v2:172.21.15.203:6832/1778614695,v1:172.21.15.203:6833/1778614695] conn(0x7f7b5406a0e0 0x7f7b5406c5a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:50:58.888 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:58.879+0000 7f7b657fa640 1 -- 172.21.15.43:0/512332348 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(92..92 src has 1..92) v4 ==== 7114+0+0 (secure 0 0 0) 0x7f7b5c088a60 con 0x7f7b680aedc0 2023-08-09T13:50:58.888 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:58.880+0000 7f7b6ccab640 1 --2- 172.21.15.43:0/512332348 >> [v2:172.21.15.203:6832/1778614695,v1:172.21.15.203:6833/1778614695] conn(0x7f7b5406a0e0 0x7f7b5406c5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:50:58.889 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:58.881+0000 7f7b6ccab640 1 --2- 172.21.15.43:0/512332348 >> [v2:172.21.15.203:6832/1778614695,v1:172.21.15.203:6833/1778614695] conn(0x7f7b5406a0e0 0x7f7b5406c5a0 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7f7b58000ff0 tx=0x7f7b58009040 comp rx=0 tx=0).ready entity=mgr.5819 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-09T13:50:58.889 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:58.884+0000 7f7b657fa640 1 -- 172.21.15.43:0/512332348 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178066 (secure 0 0 0) 0x7f7b5c053dd0 con 0x7f7b680aedc0 2023-08-09T13:50:59.119 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:59.117+0000 7f7b6ef0f640 1 -- 172.21.15.43:0/512332348 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f7b680cec90 con 0x7f7b680aedc0 2023-08-09T13:50:59.120 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:59.118+0000 7f7b657fa640 1 -- 172.21.15.43:0/512332348 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 35 v35) v1 ==== 94+0+4880 (secure 0 0 0) 0x7f7b5c017090 con 0x7f7b680aedc0 2023-08-09T13:50:59.120 INFO:teuthology.orchestra.run.smithi043.stdout: 2023-08-09T13:50:59.121 INFO:teuthology.orchestra.run.smithi043.stdout:{"epoch":35,"default_fscid":4,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5654,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.203:6837/1227342478","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6835","nonce":1227342478},{"type":"v1","addr":"172.21.15.203:6837","nonce":1227342478}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":30},{"gid":7005,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.43:6834/1431309163","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6832","nonce":1431309163},{"type":"v1","addr":"172.21.15.43:6834","nonce":1431309163}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":29},{"gid":7008,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.43:6835/611328204","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6833","nonce":611328204},{"type":"v1","addr":"172.21.15.43:6835","nonce":611328204}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":30}],"filesystems":[{"mdsmap":{"epoch":35,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-08-09T13:50:28.672581+0000","modified":"2023-08-09T13:50:32.253466+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":7011},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7011":{"gid":7011,"name":"d","rank":0,"incarnation":31,"state":"up:active","state_seq":3,"addr":"172.21.15.203:6836/3061388209","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6834","nonce":3061388209},{"type":"v1","addr":"172.21.15.203:6836","nonce":3061388209}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[11],"metadata_pool":10,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":4}]} 2023-08-09T13:50:59.121 INFO:teuthology.orchestra.run.smithi043.stderr:dumped fsmap epoch 35 2023-08-09T13:50:59.123 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:59.120+0000 7f7b6ef0f640 1 -- 172.21.15.43:0/512332348 >> [v2:172.21.15.203:6832/1778614695,v1:172.21.15.203:6833/1778614695] conn(0x7f7b5406a0e0 msgr2=0x7f7b5406c5a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:50:59.123 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:59.120+0000 7f7b6ef0f640 1 --2- 172.21.15.43:0/512332348 >> [v2:172.21.15.203:6832/1778614695,v1:172.21.15.203:6833/1778614695] conn(0x7f7b5406a0e0 0x7f7b5406c5a0 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7f7b58000ff0 tx=0x7f7b58009040 comp rx=0 tx=0).stop 2023-08-09T13:50:59.123 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:59.120+0000 7f7b6ef0f640 1 -- 172.21.15.43:0/512332348 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f7b680aedc0 msgr2=0x7f7b6815ed80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:50:59.123 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:59.120+0000 7f7b6ef0f640 1 --2- 172.21.15.43:0/512332348 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f7b680aedc0 0x7f7b6815ed80 secure :-1 s=READY pgs=1104 cs=0 l=1 rev1=1 crypto rx=0x7f7b5c00e7f0 tx=0x7f7b5c00ecb0 comp rx=0 tx=0).stop 2023-08-09T13:50:59.123 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:59.121+0000 7f7b6ef0f640 1 -- 172.21.15.43:0/512332348 shutdown_connections 2023-08-09T13:50:59.124 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:59.121+0000 7f7b6ef0f640 1 --2- 172.21.15.43:0/512332348 >> [v2:172.21.15.203:6832/1778614695,v1:172.21.15.203:6833/1778614695] conn(0x7f7b5406a0e0 0x7f7b5406c5a0 unknown :-1 s=CLOSED pgs=27 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:50:59.124 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:59.121+0000 7f7b6ef0f640 1 --2- 172.21.15.43:0/512332348 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f7b680af240 0x7f7b6815f2c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:50:59.124 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:59.121+0000 7f7b6ef0f640 1 --2- 172.21.15.43:0/512332348 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f7b680aedc0 0x7f7b6815ed80 unknown :-1 s=CLOSED pgs=1104 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:50:59.124 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:59.121+0000 7f7b6ef0f640 1 --2- 172.21.15.43:0/512332348 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f7b50004d00 0x7f7b680b2330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:50:59.124 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:59.121+0000 7f7b6ef0f640 1 -- 172.21.15.43:0/512332348 >> 172.21.15.43:0/512332348 conn(0x7f7b680ce0a0 msgr2=0x7f7b680caa80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-09T13:50:59.125 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:59.121+0000 7f7b6ef0f640 1 -- 172.21.15.43:0/512332348 shutdown_connections 2023-08-09T13:50:59.125 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:59.121+0000 7f7b6ef0f640 1 -- 172.21.15.43:0/512332348 wait complete. 2023-08-09T13:50:59.143 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-08-09T13:50:59.368 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:59.366+0000 7fcb7708e640 1 Processor -- start 2023-08-09T13:50:59.368 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:59.366+0000 7fcb7708e640 1 -- start start 2023-08-09T13:50:59.370 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:59.367+0000 7fcb7708e640 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7fcb700b2790 0x7fcb700d2ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:50:59.370 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:59.367+0000 7fcb7708e640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fcb700b2b70 0x7fcb700b5330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:50:59.370 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:59.367+0000 7fcb7708e640 1 -- --> v1:172.21.15.43:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fcb700ce090 con 0x7fcb700b3050 2023-08-09T13:50:59.370 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:59.367+0000 7fcb7708e640 1 -- --> v2:172.21.15.203:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fcb700b3950 con 0x7fcb700b2790 2023-08-09T13:50:59.370 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:59.367+0000 7fcb7708e640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcb700cd7f0 con 0x7fcb700b2b70 2023-08-09T13:50:59.371 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:59.368+0000 7fcb7562b640 1 --1- >> v1:172.21.15.43:6789/0 conn(0x7fcb700b3050 0x7fcb700b3f60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.43:6789/0 says I am v1:172.21.15.43:35080/0 (socket says 172.21.15.43:35080) 2023-08-09T13:50:59.372 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:59.368+0000 7fcb7562b640 1 -- 172.21.15.43:0/128341310 learned_addr learned my addr 172.21.15.43:0/128341310 (peer_addr_for_me v1:172.21.15.43:0/0) 2023-08-09T13:50:59.372 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:59.368+0000 7fcb74e2a640 1 --2- 172.21.15.43:0/128341310 >> v2:172.21.15.203:3300/0 conn(0x7fcb700b2790 0x7fcb700d2ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:50:59.372 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:59.368+0000 7fcb6ffff640 1 --2- 172.21.15.43:0/128341310 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fcb700b2b70 0x7fcb700b5330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:50:59.372 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:59.368+0000 7fcb6f7fe640 1 -- 172.21.15.43:0/128341310 <== mon.0 v1:172.21.15.43:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 659704613 0 0) 0x7fcb700ce090 con 0x7fcb700b3050 2023-08-09T13:50:59.372 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:59.368+0000 7fcb6f7fe640 1 -- 172.21.15.43:0/128341310 --> v1:172.21.15.43:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fcb58003520 con 0x7fcb700b3050 2023-08-09T13:50:59.372 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:59.369+0000 7fcb6f7fe640 1 -- 172.21.15.43:0/128341310 <== mon.0 v1:172.21.15.43:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3110158341 0 0) 0x7fcb58003520 con 0x7fcb700b3050 2023-08-09T13:50:59.373 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:59.369+0000 7fcb6f7fe640 1 -- 172.21.15.43:0/128341310 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fcb700b2b70 msgr2=0x7fcb700b5330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-09T13:50:59.373 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:59.369+0000 7fcb6f7fe640 1 --2- 172.21.15.43:0/128341310 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fcb700b2b70 0x7fcb700b5330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:50:59.373 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:59.369+0000 7fcb6f7fe640 1 -- 172.21.15.43:0/128341310 >> v2:172.21.15.203:3300/0 conn(0x7fcb700b2790 msgr2=0x7fcb700d2ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-09T13:50:59.373 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:59.369+0000 7fcb6f7fe640 1 --2- 172.21.15.43:0/128341310 >> v2:172.21.15.203:3300/0 conn(0x7fcb700b2790 0x7fcb700d2ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:50:59.373 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:59.369+0000 7fcb6f7fe640 1 -- 172.21.15.43:0/128341310 --> v1:172.21.15.43:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcb700cdad0 con 0x7fcb700b3050 2023-08-09T13:50:59.373 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:59.369+0000 7fcb6f7fe640 1 -- 172.21.15.43:0/128341310 <== mon.0 v1:172.21.15.43:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1285150740 0 0) 0x7fcb68001670 con 0x7fcb700b3050 2023-08-09T13:50:59.373 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:59.369+0000 7fcb6f7fe640 1 -- 172.21.15.43:0/128341310 >> v1:172.21.15.43:6789/0 conn(0x7fcb700b3050 legacy=0x7fcb700b3f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:50:59.373 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:59.369+0000 7fcb6f7fe640 1 --2- 172.21.15.43:0/128341310 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fcb580039e0 0x7fcb58003e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:50:59.374 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:59.369+0000 7fcb6f7fe640 1 --2- 172.21.15.43:0/128341310 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fcb700b2b70 0x7fcb580047c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:50:59.374 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:59.369+0000 7fcb6f7fe640 1 --2- 172.21.15.43:0/128341310 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fcb58004d00 0x7fcb58005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:50:59.374 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:59.369+0000 7fcb6f7fe640 1 -- 172.21.15.43:0/128341310 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcb700b3950 con 0x7fcb58004d00 2023-08-09T13:50:59.374 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:59.369+0000 7fcb6f7fe640 1 -- 172.21.15.43:0/128341310 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcb700cd7f0 con 0x7fcb580039e0 2023-08-09T13:50:59.374 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:59.369+0000 7fcb6f7fe640 1 -- 172.21.15.43:0/128341310 --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcb5800b980 con 0x7fcb700b2b70 2023-08-09T13:50:59.374 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:59.369+0000 7fcb7562b640 1 --2- 172.21.15.43:0/128341310 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fcb58004d00 0x7fcb58005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:50:59.375 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:59.369+0000 7fcb74e2a640 1 --2- 172.21.15.43:0/128341310 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fcb580039e0 0x7fcb58003e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:50:59.376 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:59.370+0000 7fcb6ffff640 1 --2- 172.21.15.43:0/128341310 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fcb700b2b70 0x7fcb580047c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:50:59.376 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:59.370+0000 7fcb7562b640 1 -- 172.21.15.43:0/128341310 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fcb700b2b70 msgr2=0x7fcb580047c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-09T13:50:59.376 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:59.370+0000 7fcb7562b640 1 --2- 172.21.15.43:0/128341310 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fcb700b2b70 0x7fcb580047c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:50:59.376 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:59.370+0000 7fcb7562b640 1 -- 172.21.15.43:0/128341310 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fcb580039e0 msgr2=0x7fcb58003e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-09T13:50:59.376 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:59.370+0000 7fcb7562b640 1 --2- 172.21.15.43:0/128341310 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fcb580039e0 0x7fcb58003e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:50:59.376 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:59.370+0000 7fcb7562b640 1 -- 172.21.15.43:0/128341310 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcb58003520 con 0x7fcb58004d00 2023-08-09T13:50:59.376 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:59.371+0000 7fcb7562b640 1 --2- 172.21.15.43:0/128341310 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fcb58004d00 0x7fcb58005180 secure :-1 s=READY pgs=1106 cs=0 l=1 rev1=1 crypto rx=0x7fcb6800c290 tx=0x7fcb580057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=926b9718616d9851 server_cookie=0 in_seq=0 out_seq=0 2023-08-09T13:50:59.377 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:59.371+0000 7fcb6f7fe640 1 -- 172.21.15.43:0/128341310 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcb68005ad0 con 0x7fcb58004d00 2023-08-09T13:50:59.377 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:59.371+0000 7fcb6f7fe640 1 -- 172.21.15.43:0/128341310 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcb68005c30 con 0x7fcb58004d00 2023-08-09T13:50:59.377 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:59.371+0000 7fcb6f7fe640 1 -- 172.21.15.43:0/128341310 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcb6802c8d0 con 0x7fcb58004d00 2023-08-09T13:50:59.377 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:59.372+0000 7fcb7708e640 1 -- 172.21.15.43:0/128341310 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fcb58004d00 msgr2=0x7fcb58005180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:50:59.377 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:59.372+0000 7fcb7708e640 1 --2- 172.21.15.43:0/128341310 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fcb58004d00 0x7fcb58005180 secure :-1 s=READY pgs=1106 cs=0 l=1 rev1=1 crypto rx=0x7fcb6800c290 tx=0x7fcb580057f0 comp rx=0 tx=0).stop 2023-08-09T13:50:59.377 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:59.372+0000 7fcb7562b640 1 -- 172.21.15.43:0/128341310 reap_dead start 2023-08-09T13:50:59.377 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:59.372+0000 7fcb7708e640 1 -- 172.21.15.43:0/128341310 shutdown_connections 2023-08-09T13:50:59.377 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:59.372+0000 7fcb7708e640 1 -- 172.21.15.43:0/128341310 >> 172.21.15.43:0/128341310 conn(0x7fcb7009b7b0 msgr2=0x7fcb700d24e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-09T13:50:59.378 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:59.372+0000 7fcb7708e640 1 -- 172.21.15.43:0/128341310 shutdown_connections 2023-08-09T13:50:59.378 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:59.372+0000 7fcb7708e640 1 -- 172.21.15.43:0/128341310 wait complete. 2023-08-09T13:50:59.379 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:59.374+0000 7fcb7708e640 1 Processor -- start 2023-08-09T13:50:59.379 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:59.374+0000 7fcb7708e640 1 -- start start 2023-08-09T13:50:59.380 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:59.375+0000 7fcb7708e640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fcb7015a5b0 0x7fcb700c6010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:50:59.380 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:59.375+0000 7fcb7708e640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fcb7015a990 0x7fcb700c7720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:50:59.380 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:59.375+0000 7fcb7708e640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fcb700b0b10 0x7fcb7006b580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:50:59.380 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:59.375+0000 7fcb7708e640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcb7015ff40 con 0x7fcb7015a990 2023-08-09T13:50:59.380 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:59.375+0000 7fcb7708e640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcb700c7290 con 0x7fcb7015a5b0 2023-08-09T13:50:59.380 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:59.375+0000 7fcb7708e640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcb700c7c60 con 0x7fcb700b0b10 2023-08-09T13:50:59.380 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:59.375+0000 7fcb6ffff640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fcb7015a990 0x7fcb700c7720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:50:59.381 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:59.375+0000 7fcb6ffff640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fcb7015a990 0x7fcb700c7720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:46654/0 (socket says 172.21.15.43:46654) 2023-08-09T13:50:59.381 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:59.375+0000 7fcb6ffff640 1 -- 172.21.15.43:0/724624159 learned_addr learned my addr 172.21.15.43:0/724624159 (peer_addr_for_me v2:172.21.15.43:0/0) 2023-08-09T13:50:59.381 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:59.376+0000 7fcb6ffff640 1 -- 172.21.15.43:0/724624159 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fcb700b0b10 msgr2=0x7fcb7006b580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:50:59.381 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:59.376+0000 7fcb7562b640 1 --2- 172.21.15.43:0/724624159 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fcb700b0b10 0x7fcb7006b580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:50:59.381 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:59.376+0000 7fcb6ffff640 1 --2- 172.21.15.43:0/724624159 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fcb700b0b10 0x7fcb7006b580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:50:59.381 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:59.376+0000 7fcb6ffff640 1 -- 172.21.15.43:0/724624159 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fcb7015a5b0 msgr2=0x7fcb700c6010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:50:59.381 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:59.376+0000 7fcb6ffff640 1 --2- 172.21.15.43:0/724624159 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fcb7015a5b0 0x7fcb700c6010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:50:59.382 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:59.376+0000 7fcb6ffff640 1 -- 172.21.15.43:0/724624159 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcb700c6680 con 0x7fcb7015a990 2023-08-09T13:50:59.382 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:59.376+0000 7fcb6ffff640 1 --2- 172.21.15.43:0/724624159 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fcb7015a990 0x7fcb700c7720 secure :-1 s=READY pgs=1107 cs=0 l=1 rev1=1 crypto rx=0x7fcb64011630 tx=0x7fcb64011af0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-09T13:50:59.382 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:59.376+0000 7fcb6d7fa640 1 -- 172.21.15.43:0/724624159 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcb6400f920 con 0x7fcb7015a990 2023-08-09T13:50:59.382 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:59.376+0000 7fcb6d7fa640 1 -- 172.21.15.43:0/724624159 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcb6400fa80 con 0x7fcb7015a990 2023-08-09T13:50:59.382 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:59.376+0000 7fcb6d7fa640 1 -- 172.21.15.43:0/724624159 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcb6400fdf0 con 0x7fcb7015a990 2023-08-09T13:50:59.382 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:59.376+0000 7fcb7708e640 1 -- 172.21.15.43:0/724624159 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcb700c6960 con 0x7fcb7015a990 2023-08-09T13:50:59.382 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:59.377+0000 7fcb7708e640 1 -- 172.21.15.43:0/724624159 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcb700c6f20 con 0x7fcb7015a990 2023-08-09T13:50:59.383 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:59.378+0000 7fcb7708e640 1 -- 172.21.15.43:0/724624159 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcb700739f0 con 0x7fcb7015a990 2023-08-09T13:50:59.383 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:59.379+0000 7fcb6d7fa640 1 -- 172.21.15.43:0/724624159 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 36) v1 ==== 88387+0+0 (secure 0 0 0) 0x7fcb64015460 con 0x7fcb7015a990 2023-08-09T13:50:59.383 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:59.379+0000 7fcb6d7fa640 1 --2- 172.21.15.43:0/724624159 >> [v2:172.21.15.203:6832/1778614695,v1:172.21.15.203:6833/1778614695] conn(0x7fcb5c06a110 0x7fcb5c06c5d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:50:59.383 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:59.380+0000 7fcb6d7fa640 1 -- 172.21.15.43:0/724624159 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(92..92 src has 1..92) v4 ==== 7114+0+0 (secure 0 0 0) 0x7fcb64088f30 con 0x7fcb7015a990 2023-08-09T13:50:59.384 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:59.380+0000 7fcb74e2a640 1 --2- 172.21.15.43:0/724624159 >> [v2:172.21.15.203:6832/1778614695,v1:172.21.15.203:6833/1778614695] conn(0x7fcb5c06a110 0x7fcb5c06c5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:50:59.384 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:59.381+0000 7fcb74e2a640 1 --2- 172.21.15.43:0/724624159 >> [v2:172.21.15.203:6832/1778614695,v1:172.21.15.203:6833/1778614695] conn(0x7fcb5c06a110 0x7fcb5c06c5d0 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7fcb60009fa0 tx=0x7fcb60009750 comp rx=0 tx=0).ready entity=mgr.5819 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-09T13:50:59.386 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:59.384+0000 7fcb6d7fa640 1 -- 172.21.15.43:0/724624159 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178066 (secure 0 0 0) 0x7fcb640541a0 con 0x7fcb7015a990 2023-08-09T13:50:59.465 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:59.462+0000 7fcb6d7fa640 1 -- 172.21.15.43:0/724624159 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mgrmap(e 37) v1 ==== 88443+0+0 (secure 0 0 0) 0x7fcb64054570 con 0x7fcb7015a990 2023-08-09T13:50:59.626 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:59.624+0000 7fcb7708e640 1 -- 172.21.15.43:0/724624159 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fcb700b4b00 con 0x7fcb7015a990 2023-08-09T13:50:59.627 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:59.625+0000 7fcb6d7fa640 1 -- 172.21.15.43:0/724624159 <== mon.0 v2:172.21.15.43:3300/0 8 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 35 v35) v1 ==== 94+0+4880 (secure 0 0 0) 0x7fcb64057c30 con 0x7fcb7015a990 2023-08-09T13:50:59.628 INFO:teuthology.orchestra.run.smithi043.stdout: 2023-08-09T13:50:59.628 INFO:teuthology.orchestra.run.smithi043.stdout:{"epoch":35,"default_fscid":4,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5654,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.203:6837/1227342478","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6835","nonce":1227342478},{"type":"v1","addr":"172.21.15.203:6837","nonce":1227342478}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":30},{"gid":7005,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.43:6834/1431309163","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6832","nonce":1431309163},{"type":"v1","addr":"172.21.15.43:6834","nonce":1431309163}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":29},{"gid":7008,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.43:6835/611328204","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6833","nonce":611328204},{"type":"v1","addr":"172.21.15.43:6835","nonce":611328204}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":30}],"filesystems":[{"mdsmap":{"epoch":35,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-08-09T13:50:28.672581+0000","modified":"2023-08-09T13:50:32.253466+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":7011},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7011":{"gid":7011,"name":"d","rank":0,"incarnation":31,"state":"up:active","state_seq":3,"addr":"172.21.15.203:6836/3061388209","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6834","nonce":3061388209},{"type":"v1","addr":"172.21.15.203:6836","nonce":3061388209}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[11],"metadata_pool":10,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":4}]} 2023-08-09T13:50:59.629 INFO:teuthology.orchestra.run.smithi043.stderr:dumped fsmap epoch 35 2023-08-09T13:50:59.630 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:59.627+0000 7fcb7708e640 1 -- 172.21.15.43:0/724624159 >> [v2:172.21.15.203:6832/1778614695,v1:172.21.15.203:6833/1778614695] conn(0x7fcb5c06a110 msgr2=0x7fcb5c06c5d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:50:59.630 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:59.627+0000 7fcb7708e640 1 --2- 172.21.15.43:0/724624159 >> [v2:172.21.15.203:6832/1778614695,v1:172.21.15.203:6833/1778614695] conn(0x7fcb5c06a110 0x7fcb5c06c5d0 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7fcb60009fa0 tx=0x7fcb60009750 comp rx=0 tx=0).stop 2023-08-09T13:50:59.630 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:59.627+0000 7fcb7708e640 1 -- 172.21.15.43:0/724624159 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fcb7015a990 msgr2=0x7fcb700c7720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:50:59.630 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:59.627+0000 7fcb7708e640 1 --2- 172.21.15.43:0/724624159 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fcb7015a990 0x7fcb700c7720 secure :-1 s=READY pgs=1107 cs=0 l=1 rev1=1 crypto rx=0x7fcb64011630 tx=0x7fcb64011af0 comp rx=0 tx=0).stop 2023-08-09T13:50:59.631 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:59.627+0000 7fcb7708e640 1 -- 172.21.15.43:0/724624159 shutdown_connections 2023-08-09T13:50:59.631 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:59.628+0000 7fcb7708e640 1 --2- 172.21.15.43:0/724624159 >> [v2:172.21.15.203:6832/1778614695,v1:172.21.15.203:6833/1778614695] conn(0x7fcb5c06a110 0x7fcb5c06c5d0 unknown :-1 s=CLOSED pgs=28 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:50:59.631 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:59.628+0000 7fcb7708e640 1 --2- 172.21.15.43:0/724624159 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fcb700b0b10 0x7fcb7006b580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:50:59.631 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:59.628+0000 7fcb7708e640 1 --2- 172.21.15.43:0/724624159 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fcb7015a990 0x7fcb700c7720 unknown :-1 s=CLOSED pgs=1107 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:50:59.631 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:59.628+0000 7fcb7708e640 1 --2- 172.21.15.43:0/724624159 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fcb7015a5b0 0x7fcb700c6010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:50:59.632 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:59.628+0000 7fcb7708e640 1 -- 172.21.15.43:0/724624159 >> 172.21.15.43:0/724624159 conn(0x7fcb7009b7b0 msgr2=0x7fcb700b54c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-09T13:50:59.632 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:59.628+0000 7fcb7708e640 1 -- 172.21.15.43:0/724624159 shutdown_connections 2023-08-09T13:50:59.632 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:59.628+0000 7fcb7708e640 1 -- 172.21.15.43:0/724624159 wait complete. 2023-08-09T13:50:59.650 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-08-09T13:50:59.876 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:59.873+0000 7fc4b620a640 1 Processor -- start 2023-08-09T13:50:59.876 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:59.873+0000 7fc4b620a640 1 -- start start 2023-08-09T13:50:59.878 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:59.875+0000 7fc4b620a640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7fc4b0160e70 0x7fc4b0149e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:50:59.878 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:59.875+0000 7fc4b620a640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fc4b00c9620 0x7fc4b00c9a50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:50:59.878 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:59.875+0000 7fc4b620a640 1 -- --> v1:172.21.15.203:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc4b01614b0 con 0x7fc4b00c0e60 2023-08-09T13:50:59.878 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:59.875+0000 7fc4b620a640 1 -- --> v2:172.21.15.43:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc4b0164540 con 0x7fc4b0160e70 2023-08-09T13:50:59.878 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:59.875+0000 7fc4b620a640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc4b0161790 con 0x7fc4b00c9620 2023-08-09T13:50:59.879 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:59.875+0000 7fc4b4a07640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7fc4b0160e70 0x7fc4b0149e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:50:59.879 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:59.876+0000 7fc4b4a07640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7fc4b0160e70 0x7fc4b0149e10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:46658/0 (socket says 172.21.15.43:46658) 2023-08-09T13:50:59.879 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:59.876+0000 7fc4b4a07640 1 -- 172.21.15.43:0/3516203307 learned_addr learned my addr 172.21.15.43:0/3516203307 (peer_addr_for_me v2:172.21.15.43:0/0) 2023-08-09T13:50:59.880 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:59.876+0000 7fc4b5a09640 1 --2- 172.21.15.43:0/3516203307 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fc4b00c9620 0x7fc4b00c9a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:50:59.880 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:59.876+0000 7fc4b4a07640 1 -- 172.21.15.43:0/3516203307 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fc4b00c9620 msgr2=0x7fc4b00c9a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-09T13:50:59.880 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:59.876+0000 7fc4b4a07640 1 --2- 172.21.15.43:0/3516203307 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fc4b00c9620 0x7fc4b00c9a50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:50:59.880 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:59.876+0000 7fc4b4a07640 1 -- 172.21.15.43:0/3516203307 >> v1:172.21.15.203:6789/0 conn(0x7fc4b00c0e60 legacy=0x7fc4b00c1240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-09T13:50:59.880 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:59.876+0000 7fc4b4a07640 1 -- 172.21.15.43:0/3516203307 --> v2:172.21.15.43:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc4b0161a70 con 0x7fc4b0160e70 2023-08-09T13:50:59.881 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:59.877+0000 7fc4b4a07640 1 --2- 172.21.15.43:0/3516203307 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc4b0160e70 0x7fc4b0149e10 secure :-1 s=READY pgs=1108 cs=0 l=1 rev1=1 crypto rx=0x7fc4b00ad010 tx=0x7fc4a002b840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5ab67a4d9956a832 server_cookie=0 in_seq=0 out_seq=0 2023-08-09T13:50:59.881 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:59.878+0000 7fc4a7fff640 1 -- 172.21.15.43:0/3516203307 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc4a002e070 con 0x7fc4b0160e70 2023-08-09T13:50:59.881 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:59.878+0000 7fc4a7fff640 1 -- 172.21.15.43:0/3516203307 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc4a00315a0 con 0x7fc4b0160e70 2023-08-09T13:50:59.881 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:59.878+0000 7fc4a7fff640 1 -- 172.21.15.43:0/3516203307 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc4a0031930 con 0x7fc4b0160e70 2023-08-09T13:50:59.882 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:59.878+0000 7fc4b620a640 1 -- 172.21.15.43:0/3516203307 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc4b0160e70 msgr2=0x7fc4b0149e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:50:59.882 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:59.878+0000 7fc4b620a640 1 --2- 172.21.15.43:0/3516203307 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc4b0160e70 0x7fc4b0149e10 secure :-1 s=READY pgs=1108 cs=0 l=1 rev1=1 crypto rx=0x7fc4b00ad010 tx=0x7fc4a002b840 comp rx=0 tx=0).stop 2023-08-09T13:50:59.882 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:59.878+0000 7fc4b620a640 1 -- 172.21.15.43:0/3516203307 shutdown_connections 2023-08-09T13:50:59.882 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:59.878+0000 7fc4b620a640 1 --2- 172.21.15.43:0/3516203307 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fc4b00c9620 0x7fc4b00c9a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:50:59.883 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:59.878+0000 7fc4b620a640 1 --2- 172.21.15.43:0/3516203307 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc4b0160e70 0x7fc4b0149e10 unknown :-1 s=CLOSED pgs=1108 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:50:59.883 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:59.878+0000 7fc4b620a640 1 -- 172.21.15.43:0/3516203307 >> 172.21.15.43:0/3516203307 conn(0x7fc4b00cde90 msgr2=0x7fc4b00ce2a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-09T13:50:59.883 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:59.879+0000 7fc4b620a640 1 -- 172.21.15.43:0/3516203307 shutdown_connections 2023-08-09T13:50:59.883 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:59.879+0000 7fc4b620a640 1 -- 172.21.15.43:0/3516203307 wait complete. 2023-08-09T13:50:59.883 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:59.880+0000 7fc4b620a640 1 Processor -- start 2023-08-09T13:50:59.884 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:59.881+0000 7fc4b620a640 1 -- start start 2023-08-09T13:50:59.885 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:59.881+0000 7fc4b620a640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc4b00c0e60 0x7fc4b0098cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:50:59.885 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:59.882+0000 7fc4b620a640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fc4b00c9620 0x7fc4b009aa40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:50:59.886 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:59.882+0000 7fc4b620a640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fc4b0160e70 0x7fc4b009af80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:50:59.886 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:59.882+0000 7fc4b620a640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc4b00c1a60 con 0x7fc4b00c0e60 2023-08-09T13:50:59.886 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:59.882+0000 7fc4b620a640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc4b009b640 con 0x7fc4b0160e70 2023-08-09T13:50:59.886 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:59.882+0000 7fc4b620a640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc4b00c2030 con 0x7fc4b00c9620 2023-08-09T13:50:59.886 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:59.882+0000 7fc4b5208640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc4b00c0e60 0x7fc4b0098cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:50:59.886 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:59.882+0000 7fc4b5208640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc4b00c0e60 0x7fc4b0098cc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:46662/0 (socket says 172.21.15.43:46662) 2023-08-09T13:50:59.887 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:59.882+0000 7fc4b5208640 1 -- 172.21.15.43:0/2957767438 learned_addr learned my addr 172.21.15.43:0/2957767438 (peer_addr_for_me v2:172.21.15.43:0/0) 2023-08-09T13:50:59.887 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:59.882+0000 7fc4b4a07640 1 --2- 172.21.15.43:0/2957767438 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fc4b00c9620 0x7fc4b009aa40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:50:59.887 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:59.882+0000 7fc4b5208640 1 -- 172.21.15.43:0/2957767438 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fc4b00c9620 msgr2=0x7fc4b009aa40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:50:59.887 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:59.882+0000 7fc4b5208640 1 --2- 172.21.15.43:0/2957767438 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fc4b00c9620 0x7fc4b009aa40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:50:59.887 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:59.882+0000 7fc4b5208640 1 -- 172.21.15.43:0/2957767438 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fc4b0160e70 msgr2=0x7fc4b009af80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:50:59.887 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:59.882+0000 7fc4b5208640 1 --2- 172.21.15.43:0/2957767438 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fc4b0160e70 0x7fc4b009af80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:50:59.887 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:59.882+0000 7fc4b5208640 1 -- 172.21.15.43:0/2957767438 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc4a00094d0 con 0x7fc4b00c0e60 2023-08-09T13:50:59.888 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:59.883+0000 7fc4b5208640 1 --2- 172.21.15.43:0/2957767438 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc4b00c0e60 0x7fc4b0098cc0 secure :-1 s=READY pgs=1109 cs=0 l=1 rev1=1 crypto rx=0x7fc4ac00b3f0 tx=0x7fc4ac00b8b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-09T13:50:59.889 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:59.883+0000 7fc4a5ffb640 1 -- 172.21.15.43:0/2957767438 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc4ac011830 con 0x7fc4b00c0e60 2023-08-09T13:50:59.889 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:59.883+0000 7fc4a5ffb640 1 -- 172.21.15.43:0/2957767438 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc4ac011990 con 0x7fc4b00c0e60 2023-08-09T13:50:59.889 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:59.883+0000 7fc4a5ffb640 1 -- 172.21.15.43:0/2957767438 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc4ac011d40 con 0x7fc4b00c0e60 2023-08-09T13:50:59.889 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:59.883+0000 7fc4b620a640 1 -- 172.21.15.43:0/2957767438 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc4b00c2310 con 0x7fc4b00c0e60 2023-08-09T13:50:59.889 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:59.883+0000 7fc4b620a640 1 -- 172.21.15.43:0/2957767438 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc4b0099200 con 0x7fc4b00c0e60 2023-08-09T13:50:59.889 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:59.884+0000 7fc4b620a640 1 -- 172.21.15.43:0/2957767438 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc4b00993f0 con 0x7fc4b00c0e60 2023-08-09T13:50:59.890 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:59.885+0000 7fc4a5ffb640 1 -- 172.21.15.43:0/2957767438 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 37) v1 ==== 88443+0+0 (secure 0 0 0) 0x7fc4ac01a400 con 0x7fc4b00c0e60 2023-08-09T13:50:59.890 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:59.886+0000 7fc4a5ffb640 1 --2- 172.21.15.43:0/2957767438 >> [v2:172.21.15.203:6832/1778614695,v1:172.21.15.203:6833/1778614695] conn(0x7fc49406a170 0x7fc49406c630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:50:59.890 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:59.886+0000 7fc4a5ffb640 1 -- 172.21.15.43:0/2957767438 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(92..92 src has 1..92) v4 ==== 7114+0+0 (secure 0 0 0) 0x7fc4ac088a40 con 0x7fc4b00c0e60 2023-08-09T13:50:59.890 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:59.887+0000 7fc4b4a07640 1 --2- 172.21.15.43:0/2957767438 >> [v2:172.21.15.203:6832/1778614695,v1:172.21.15.203:6833/1778614695] conn(0x7fc49406a170 0x7fc49406c630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:50:59.891 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:59.888+0000 7fc4b4a07640 1 --2- 172.21.15.43:0/2957767438 >> [v2:172.21.15.203:6832/1778614695,v1:172.21.15.203:6833/1778614695] conn(0x7fc49406a170 0x7fc49406c630 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7fc4a0036000 tx=0x7fc4a003d040 comp rx=0 tx=0).ready entity=mgr.5819 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-09T13:50:59.893 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:50:59.891+0000 7fc4a5ffb640 1 -- 172.21.15.43:0/2957767438 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178066 (secure 0 0 0) 0x7fc4ac053bf0 con 0x7fc4b00c0e60 2023-08-09T13:51:00.087 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:00.084+0000 7fc4b620a640 1 -- 172.21.15.43:0/2957767438 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fc4b00c1e90 con 0x7fc4b00c0e60 2023-08-09T13:51:00.090 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:00.088+0000 7fc4a5ffb640 1 -- 172.21.15.43:0/2957767438 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v92) v1 ==== 74+0+19573 (secure 0 0 0) 0x7fc4b00c1e90 con 0x7fc4b00c0e60 2023-08-09T13:51:00.091 INFO:teuthology.orchestra.run.smithi043.stdout: 2023-08-09T13:51:00.091 INFO:teuthology.orchestra.run.smithi043.stdout:{"epoch":92,"fsid":"daac87ee-1337-4250-a955-aea4e408a39b","created":"2023-08-09T13:33:41.701198+0000","modified":"2023-08-09T13:50:55.386567+0000","last_up_change":"2023-08-09T13:33:53.783453+0000","last_in_change":"2023-08-09T13:33:45.148904+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":11,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":"rbd","create_time":"2023-08-09T13:33:54.365137+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":".mgr","create_time":"2023-08-09T13:33:54.831803+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":10,"pool_name":"cephfs_metadata","create_time":"2023-08-09T13:50:27.051781+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"89","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":2.5,"score_stable":2.5,"optimal_score":1,"raw_score_acting":2.5,"raw_score_stable":2.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":11,"pool_name":"cephfs_data","create_time":"2023-08-09T13:50:27.692933+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"90","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"cbd49ecd-e896-4fbd-864e-5c37a829b165","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6800","nonce":502667335},{"type":"v1","addr":"172.21.15.43:6801","nonce":502667335}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6802","nonce":502667335},{"type":"v1","addr":"172.21.15.43:6803","nonce":502667335}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6806","nonce":502667335},{"type":"v1","addr":"172.21.15.43:6807","nonce":502667335}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6804","nonce":502667335},{"type":"v1","addr":"172.21.15.43:6805","nonce":502667335}]},"public_addr":"172.21.15.43:6801/502667335","cluster_addr":"172.21.15.43:6803/502667335","heartbeat_back_addr":"172.21.15.43:6807/502667335","heartbeat_front_addr":"172.21.15.43:6805/502667335","state":["exists","up"]},{"osd":1,"uuid":"75c7e0fe-1ec7-4fe8-a2f2-3661a65b8500","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6808","nonce":3289134610},{"type":"v1","addr":"172.21.15.43:6809","nonce":3289134610}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6810","nonce":3289134610},{"type":"v1","addr":"172.21.15.43:6812","nonce":3289134610}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6821","nonce":3289134610},{"type":"v1","addr":"172.21.15.43:6824","nonce":3289134610}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6815","nonce":3289134610},{"type":"v1","addr":"172.21.15.43:6818","nonce":3289134610}]},"public_addr":"172.21.15.43:6809/3289134610","cluster_addr":"172.21.15.43:6812/3289134610","heartbeat_back_addr":"172.21.15.43:6824/3289134610","heartbeat_front_addr":"172.21.15.43:6818/3289134610","state":["exists","up"]},{"osd":2,"uuid":"bb5c41e9-7f5f-4125-9b01-3bb8bfc2ea83","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6811","nonce":4079709736},{"type":"v1","addr":"172.21.15.43:6814","nonce":4079709736}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6817","nonce":4079709736},{"type":"v1","addr":"172.21.15.43:6820","nonce":4079709736}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6829","nonce":4079709736},{"type":"v1","addr":"172.21.15.43:6831","nonce":4079709736}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6823","nonce":4079709736},{"type":"v1","addr":"172.21.15.43:6826","nonce":4079709736}]},"public_addr":"172.21.15.43:6814/4079709736","cluster_addr":"172.21.15.43:6820/4079709736","heartbeat_back_addr":"172.21.15.43:6831/4079709736","heartbeat_front_addr":"172.21.15.43:6826/4079709736","state":["exists","up"]},{"osd":3,"uuid":"bfe4c8b4-694a-465c-87e6-2b2aab852613","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6813","nonce":1709538759},{"type":"v1","addr":"172.21.15.43:6816","nonce":1709538759}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6819","nonce":1709538759},{"type":"v1","addr":"172.21.15.43:6822","nonce":1709538759}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6828","nonce":1709538759},{"type":"v1","addr":"172.21.15.43:6830","nonce":1709538759}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6825","nonce":1709538759},{"type":"v1","addr":"172.21.15.43:6827","nonce":1709538759}]},"public_addr":"172.21.15.43:6816/1709538759","cluster_addr":"172.21.15.43:6822/1709538759","heartbeat_back_addr":"172.21.15.43:6830/1709538759","heartbeat_front_addr":"172.21.15.43:6827/1709538759","state":["exists","up"]},{"osd":4,"uuid":"fa2e8a77-aba1-4756-853e-0d174a5bdeba","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6800","nonce":4196866481},{"type":"v1","addr":"172.21.15.203:6801","nonce":4196866481}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6802","nonce":4196866481},{"type":"v1","addr":"172.21.15.203:6803","nonce":4196866481}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6806","nonce":4196866481},{"type":"v1","addr":"172.21.15.203:6807","nonce":4196866481}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6804","nonce":4196866481},{"type":"v1","addr":"172.21.15.203:6805","nonce":4196866481}]},"public_addr":"172.21.15.203:6801/4196866481","cluster_addr":"172.21.15.203:6803/4196866481","heartbeat_back_addr":"172.21.15.203:6807/4196866481","heartbeat_front_addr":"172.21.15.203:6805/4196866481","state":["exists","up"]},{"osd":5,"uuid":"81811e4b-4d24-4f51-a82c-f1d700a2a00d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6816","nonce":3059290081},{"type":"v1","addr":"172.21.15.203:6817","nonce":3059290081}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6818","nonce":3059290081},{"type":"v1","addr":"172.21.15.203:6819","nonce":3059290081}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6822","nonce":3059290081},{"type":"v1","addr":"172.21.15.203:6823","nonce":3059290081}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6820","nonce":3059290081},{"type":"v1","addr":"172.21.15.203:6821","nonce":3059290081}]},"public_addr":"172.21.15.203:6817/3059290081","cluster_addr":"172.21.15.203:6819/3059290081","heartbeat_back_addr":"172.21.15.203:6823/3059290081","heartbeat_front_addr":"172.21.15.203:6821/3059290081","state":["exists","up"]},{"osd":6,"uuid":"307bac4d-6a5b-443d-af75-56d695d7674f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6824","nonce":512155569},{"type":"v1","addr":"172.21.15.203:6825","nonce":512155569}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6826","nonce":512155569},{"type":"v1","addr":"172.21.15.203:6827","nonce":512155569}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6830","nonce":512155569},{"type":"v1","addr":"172.21.15.203:6831","nonce":512155569}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6828","nonce":512155569},{"type":"v1","addr":"172.21.15.203:6829","nonce":512155569}]},"public_addr":"172.21.15.203:6825/512155569","cluster_addr":"172.21.15.203:6827/512155569","heartbeat_back_addr":"172.21.15.203:6831/512155569","heartbeat_front_addr":"172.21.15.203:6829/512155569","state":["exists","up"]},{"osd":7,"uuid":"a95dc7ca-9f1b-4295-af70-68a22a2d7d5e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6808","nonce":3331450438},{"type":"v1","addr":"172.21.15.203:6809","nonce":3331450438}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6810","nonce":3331450438},{"type":"v1","addr":"172.21.15.203:6811","nonce":3331450438}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6814","nonce":3331450438},{"type":"v1","addr":"172.21.15.203:6815","nonce":3331450438}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6812","nonce":3331450438},{"type":"v1","addr":"172.21.15.203:6813","nonce":3331450438}]},"public_addr":"172.21.15.203:6809/3331450438","cluster_addr":"172.21.15.203:6811/3331450438","heartbeat_back_addr":"172.21.15.203:6815/3331450438","heartbeat_front_addr":"172.21.15.203:6813/3331450438","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-09T13:33:50.547652+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-09T13:33:50.589469+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-09T13:33:50.561724+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-09T13:33:50.599650+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-09T13:33:50.411855+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-09T13:33:50.219681+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-09T13:33:50.504664+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-09T13:33:50.441248+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.203:0/375511470":"2023-08-10T13:50:55.386522+0000","172.21.15.203:0/4256110232":"2023-08-10T13:50:55.386522+0000","172.21.15.203:0/3102402128":"2023-08-10T13:50:55.386522+0000","172.21.15.203:0/1112175379":"2023-08-10T13:50:55.386522+0000","172.21.15.203:6833/3851880723":"2023-08-10T13:50:55.386522+0000","172.21.15.203:6833/3325141227":"2023-08-10T13:50:49.168196+0000","172.21.15.203:6832/3325141227":"2023-08-10T13:50:49.168196+0000","172.21.15.203:0/1343982026":"2023-08-10T13:50:55.386522+0000","172.21.15.203:0/3732830113":"2023-08-10T13:50:49.168196+0000","172.21.15.203:0/2753554568":"2023-08-10T13:50:49.168196+0000","172.21.15.203:0/3841129997":"2023-08-10T13:50:55.386522+0000","172.21.15.203:0/2468196984":"2023-08-10T13:50:49.168196+0000","172.21.15.203:6832/3851880723":"2023-08-10T13:50:55.386522+0000","172.21.15.203:0/3375551329":"2023-08-10T13:50:49.168196+0000","172.21.15.203:0/1259705306":"2023-08-10T13:50:49.168196+0000","172.21.15.203:0/1322546009":"2023-08-10T13:50:49.168196+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":7,"snaps":[{"begin":104,"length":1},{"begin":221,"length":1}]}],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-08-09T13:51:00.092 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:00.090+0000 7fc4b620a640 1 -- 172.21.15.43:0/2957767438 >> [v2:172.21.15.203:6832/1778614695,v1:172.21.15.203:6833/1778614695] conn(0x7fc49406a170 msgr2=0x7fc49406c630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:51:00.093 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:00.090+0000 7fc4b620a640 1 --2- 172.21.15.43:0/2957767438 >> [v2:172.21.15.203:6832/1778614695,v1:172.21.15.203:6833/1778614695] conn(0x7fc49406a170 0x7fc49406c630 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7fc4a0036000 tx=0x7fc4a003d040 comp rx=0 tx=0).stop 2023-08-09T13:51:00.093 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:00.090+0000 7fc4b620a640 1 -- 172.21.15.43:0/2957767438 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc4b00c0e60 msgr2=0x7fc4b0098cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:51:00.093 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:00.090+0000 7fc4b620a640 1 --2- 172.21.15.43:0/2957767438 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc4b00c0e60 0x7fc4b0098cc0 secure :-1 s=READY pgs=1109 cs=0 l=1 rev1=1 crypto rx=0x7fc4ac00b3f0 tx=0x7fc4ac00b8b0 comp rx=0 tx=0).stop 2023-08-09T13:51:00.093 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:00.090+0000 7fc4b620a640 1 -- 172.21.15.43:0/2957767438 shutdown_connections 2023-08-09T13:51:00.094 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:00.090+0000 7fc4b620a640 1 --2- 172.21.15.43:0/2957767438 >> [v2:172.21.15.203:6832/1778614695,v1:172.21.15.203:6833/1778614695] conn(0x7fc49406a170 0x7fc49406c630 unknown :-1 s=CLOSED pgs=29 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:51:00.094 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:00.090+0000 7fc4b620a640 1 --2- 172.21.15.43:0/2957767438 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fc4b0160e70 0x7fc4b009af80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:51:00.094 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:00.090+0000 7fc4b620a640 1 --2- 172.21.15.43:0/2957767438 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fc4b00c9620 0x7fc4b009aa40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:51:00.094 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:00.090+0000 7fc4b620a640 1 --2- 172.21.15.43:0/2957767438 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc4b00c0e60 0x7fc4b0098cc0 unknown :-1 s=CLOSED pgs=1109 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:51:00.094 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:00.090+0000 7fc4b620a640 1 -- 172.21.15.43:0/2957767438 >> 172.21.15.43:0/2957767438 conn(0x7fc4b00cde90 msgr2=0x7fc4b00aaf10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-09T13:51:00.094 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:00.091+0000 7fc4b620a640 1 -- 172.21.15.43:0/2957767438 shutdown_connections 2023-08-09T13:51:00.095 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:00.091+0000 7fc4b620a640 1 -- 172.21.15.43:0/2957767438 wait complete. 2023-08-09T13:51:00.112 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2023-08-09T13:51:00.352 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:00.349+0000 7ff4fcc60640 1 Processor -- start 2023-08-09T13:51:00.352 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:00.349+0000 7ff4fcc60640 1 -- start start 2023-08-09T13:51:00.353 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:00.351+0000 7ff4fcc60640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7ff4f80d0a70 0x7ff4f80cd7f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:51:00.354 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:00.351+0000 7ff4fcc60640 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7ff4f80cdd30 0x7ff4f80b3fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:51:00.354 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:00.351+0000 7ff4fcc60640 1 -- --> v1:172.21.15.43:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff4f80d0f70 con 0x7ff4f815c680 2023-08-09T13:51:00.354 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:00.351+0000 7ff4fcc60640 1 -- --> v2:172.21.15.203:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff4f80ccd10 con 0x7ff4f80cdd30 2023-08-09T13:51:00.354 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:00.351+0000 7ff4fcc60640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff4f80ccff0 con 0x7ff4f80d0a70 2023-08-09T13:51:00.354 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:00.351+0000 7ff4f659c640 1 --1- >> v1:172.21.15.43:6789/0 conn(0x7ff4f815c680 0x7ff4f80ce760 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.43:6789/0 says I am v1:172.21.15.43:35082/0 (socket says 172.21.15.43:35082) 2023-08-09T13:51:00.354 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:00.351+0000 7ff4f659c640 1 -- 172.21.15.43:0/2722106587 learned_addr learned my addr 172.21.15.43:0/2722106587 (peer_addr_for_me v1:172.21.15.43:0/0) 2023-08-09T13:51:00.356 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:00.352+0000 7ff4f5d9b640 1 --2- 172.21.15.43:0/2722106587 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7ff4f80d0a70 0x7ff4f80cd7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:51:00.356 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:00.352+0000 7ff4f6d9d640 1 --2- 172.21.15.43:0/2722106587 >> v2:172.21.15.203:3300/0 conn(0x7ff4f80cdd30 0x7ff4f80b3fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:51:00.356 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:00.352+0000 7ff4f559a640 1 -- 172.21.15.43:0/2722106587 <== mon.0 v1:172.21.15.43:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1611536557 0 0) 0x7ff4f80d0f70 con 0x7ff4f815c680 2023-08-09T13:51:00.357 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:00.352+0000 7ff4f559a640 1 -- 172.21.15.43:0/2722106587 --> v1:172.21.15.43:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ff4e0003520 con 0x7ff4f815c680 2023-08-09T13:51:00.357 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:00.352+0000 7ff4f559a640 1 -- 172.21.15.43:0/2722106587 <== mon.0 v1:172.21.15.43:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 541139725 0 0) 0x7ff4e0003520 con 0x7ff4f815c680 2023-08-09T13:51:00.357 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:00.352+0000 7ff4f559a640 1 -- 172.21.15.43:0/2722106587 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7ff4f80d0a70 msgr2=0x7ff4f80cd7f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-09T13:51:00.357 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:00.352+0000 7ff4f559a640 1 --2- 172.21.15.43:0/2722106587 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7ff4f80d0a70 0x7ff4f80cd7f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:51:00.357 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:00.352+0000 7ff4f559a640 1 -- 172.21.15.43:0/2722106587 >> v2:172.21.15.203:3300/0 conn(0x7ff4f80cdd30 msgr2=0x7ff4f80b3fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-09T13:51:00.357 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:00.352+0000 7ff4f559a640 1 --2- 172.21.15.43:0/2722106587 >> v2:172.21.15.203:3300/0 conn(0x7ff4f80cdd30 0x7ff4f80b3fa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:51:00.357 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:00.352+0000 7ff4f559a640 1 -- 172.21.15.43:0/2722106587 --> v1:172.21.15.43:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff4f80cd2d0 con 0x7ff4f815c680 2023-08-09T13:51:00.357 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:00.352+0000 7ff4f559a640 1 -- 172.21.15.43:0/2722106587 <== mon.0 v1:172.21.15.43:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1285150740 0 0) 0x7ff4e80019f0 con 0x7ff4f815c680 2023-08-09T13:51:00.358 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:00.352+0000 7ff4f559a640 1 -- 172.21.15.43:0/2722106587 >> v1:172.21.15.43:6789/0 conn(0x7ff4f815c680 legacy=0x7ff4f80ce760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:51:00.358 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:00.352+0000 7ff4f559a640 1 --2- 172.21.15.43:0/2722106587 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7ff4f80d0a70 0x7ff4e00039e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:51:00.358 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:00.352+0000 7ff4f559a640 1 --2- 172.21.15.43:0/2722106587 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff4e0003f60 0x7ff4e00047c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:51:00.358 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:00.352+0000 7ff4f559a640 1 --2- 172.21.15.43:0/2722106587 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7ff4e0004d00 0x7ff4e0005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:51:00.358 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:00.352+0000 7ff4f559a640 1 -- 172.21.15.43:0/2722106587 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff4f80ccd10 con 0x7ff4e0003f60 2023-08-09T13:51:00.358 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:00.352+0000 7ff4f559a640 1 -- 172.21.15.43:0/2722106587 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff4f80ccff0 con 0x7ff4e0004d00 2023-08-09T13:51:00.359 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:00.353+0000 7ff4f559a640 1 -- 172.21.15.43:0/2722106587 --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff4e000ba60 con 0x7ff4f80d0a70 2023-08-09T13:51:00.359 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:00.353+0000 7ff4f5d9b640 1 --2- 172.21.15.43:0/2722106587 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff4e0003f60 0x7ff4e00047c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:51:00.359 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:00.353+0000 7ff4f659c640 1 --2- 172.21.15.43:0/2722106587 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7ff4f80d0a70 0x7ff4e00039e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:51:00.359 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:00.353+0000 7ff4f6d9d640 1 --2- 172.21.15.43:0/2722106587 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7ff4e0004d00 0x7ff4e0005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:51:00.359 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:00.353+0000 7ff4f5d9b640 1 -- 172.21.15.43:0/2722106587 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7ff4f80d0a70 msgr2=0x7ff4e00039e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-09T13:51:00.359 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:00.353+0000 7ff4f5d9b640 1 --2- 172.21.15.43:0/2722106587 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7ff4f80d0a70 0x7ff4e00039e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:51:00.360 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:00.353+0000 7ff4f5d9b640 1 -- 172.21.15.43:0/2722106587 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7ff4e0004d00 msgr2=0x7ff4e0005180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-09T13:51:00.360 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:00.353+0000 7ff4f5d9b640 1 --2- 172.21.15.43:0/2722106587 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7ff4e0004d00 0x7ff4e0005180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:51:00.360 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:00.353+0000 7ff4f5d9b640 1 -- 172.21.15.43:0/2722106587 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff4e0003520 con 0x7ff4e0003f60 2023-08-09T13:51:00.360 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:00.353+0000 7ff4f6d9d640 1 --2- 172.21.15.43:0/2722106587 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7ff4e0004d00 0x7ff4e0005180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-08-09T13:51:00.361 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:00.355+0000 7ff4f5d9b640 1 --2- 172.21.15.43:0/2722106587 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff4e0003f60 0x7ff4e00047c0 secure :-1 s=READY pgs=1111 cs=0 l=1 rev1=1 crypto rx=0x7ff4ec00ba30 tx=0x7ff4e00057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=15e53afe21735f88 server_cookie=0 in_seq=0 out_seq=0 2023-08-09T13:51:00.361 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:00.355+0000 7ff4f559a640 1 -- 172.21.15.43:0/2722106587 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff4ec032640 con 0x7ff4e0003f60 2023-08-09T13:51:00.362 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:00.355+0000 7ff4f559a640 1 -- 172.21.15.43:0/2722106587 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff4ec0327a0 con 0x7ff4e0003f60 2023-08-09T13:51:00.362 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:00.355+0000 7ff4f559a640 1 -- 172.21.15.43:0/2722106587 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff4ec032b30 con 0x7ff4e0003f60 2023-08-09T13:51:00.362 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:00.356+0000 7ff4fcc60640 1 -- 172.21.15.43:0/2722106587 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff4e0003f60 msgr2=0x7ff4e00047c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:51:00.362 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:00.356+0000 7ff4fcc60640 1 --2- 172.21.15.43:0/2722106587 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff4e0003f60 0x7ff4e00047c0 secure :-1 s=READY pgs=1111 cs=0 l=1 rev1=1 crypto rx=0x7ff4ec00ba30 tx=0x7ff4e00057f0 comp rx=0 tx=0).stop 2023-08-09T13:51:00.362 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:00.356+0000 7ff4f6d9d640 1 -- 172.21.15.43:0/2722106587 reap_dead start 2023-08-09T13:51:00.362 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:00.356+0000 7ff4fcc60640 1 -- 172.21.15.43:0/2722106587 shutdown_connections 2023-08-09T13:51:00.362 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:00.356+0000 7ff4fcc60640 1 -- 172.21.15.43:0/2722106587 >> 172.21.15.43:0/2722106587 conn(0x7ff4f8071520 msgr2=0x7ff4f80d2520 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-09T13:51:00.363 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:00.356+0000 7ff4fcc60640 1 -- 172.21.15.43:0/2722106587 shutdown_connections 2023-08-09T13:51:00.363 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:00.356+0000 7ff4fcc60640 1 -- 172.21.15.43:0/2722106587 wait complete. 2023-08-09T13:51:00.363 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:00.359+0000 7ff4fcc60640 1 Processor -- start 2023-08-09T13:51:00.363 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:00.359+0000 7ff4fcc60640 1 -- start start 2023-08-09T13:51:00.364 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:00.360+0000 7ff4fcc60640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff4e0003f60 0x7ff4f815dec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:51:00.364 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:00.360+0000 7ff4fcc60640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7ff4f815f390 0x7ff4f81564b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:51:00.365 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:00.360+0000 7ff4fcc60640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7ff4f815f810 0x7ff4f81f11a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:51:00.365 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:00.360+0000 7ff4fcc60640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff4f815daa0 con 0x7ff4e0003f60 2023-08-09T13:51:00.365 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:00.360+0000 7ff4fcc60640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff4f815e400 con 0x7ff4f815f810 2023-08-09T13:51:00.365 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:00.360+0000 7ff4fcc60640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff4f815ce20 con 0x7ff4f815f390 2023-08-09T13:51:00.365 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:00.360+0000 7ff4f659c640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff4e0003f60 0x7ff4f815dec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:51:00.365 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:00.360+0000 7ff4f659c640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff4e0003f60 0x7ff4f815dec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:46682/0 (socket says 172.21.15.43:46682) 2023-08-09T13:51:00.365 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:00.360+0000 7ff4f659c640 1 -- 172.21.15.43:0/2974860595 learned_addr learned my addr 172.21.15.43:0/2974860595 (peer_addr_for_me v2:172.21.15.43:0/0) 2023-08-09T13:51:00.366 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:00.361+0000 7ff4f659c640 1 -- 172.21.15.43:0/2974860595 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7ff4f815f390 msgr2=0x7ff4f81564b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:51:00.366 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:00.361+0000 7ff4f659c640 1 --2- 172.21.15.43:0/2974860595 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7ff4f815f390 0x7ff4f81564b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:51:00.366 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:00.361+0000 7ff4f659c640 1 -- 172.21.15.43:0/2974860595 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7ff4f815f810 msgr2=0x7ff4f81f11a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:51:00.366 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:00.361+0000 7ff4f659c640 1 --2- 172.21.15.43:0/2974860595 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7ff4f815f810 0x7ff4f81f11a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:51:00.366 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:00.361+0000 7ff4f659c640 1 -- 172.21.15.43:0/2974860595 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff4f815d100 con 0x7ff4e0003f60 2023-08-09T13:51:00.366 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:00.361+0000 7ff4f659c640 1 --2- 172.21.15.43:0/2974860595 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff4e0003f60 0x7ff4f815dec0 secure :-1 s=READY pgs=1112 cs=0 l=1 rev1=1 crypto rx=0x7ff4e8007a10 tx=0x7ff4e800e530 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-09T13:51:00.367 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:00.361+0000 7ff4e6ffd640 1 -- 172.21.15.43:0/2974860595 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff4e800ee50 con 0x7ff4e0003f60 2023-08-09T13:51:00.367 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:00.362+0000 7ff4e6ffd640 1 -- 172.21.15.43:0/2974860595 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff4e8013420 con 0x7ff4e0003f60 2023-08-09T13:51:00.367 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:00.362+0000 7ff4e6ffd640 1 -- 172.21.15.43:0/2974860595 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff4e80137d0 con 0x7ff4e0003f60 2023-08-09T13:51:00.367 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:00.362+0000 7ff4fcc60640 1 -- 172.21.15.43:0/2974860595 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff4f815d3e0 con 0x7ff4e0003f60 2023-08-09T13:51:00.367 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:00.362+0000 7ff4fcc60640 1 -- 172.21.15.43:0/2974860595 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff4f815ca60 con 0x7ff4e0003f60 2023-08-09T13:51:00.368 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:00.363+0000 7ff4fcc60640 1 -- 172.21.15.43:0/2974860595 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff4f815dd10 con 0x7ff4e0003f60 2023-08-09T13:51:00.368 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:00.364+0000 7ff4e6ffd640 1 -- 172.21.15.43:0/2974860595 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 37) v1 ==== 88443+0+0 (secure 0 0 0) 0x7ff4e8013930 con 0x7ff4e0003f60 2023-08-09T13:51:00.369 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:00.366+0000 7ff4e6ffd640 1 --2- 172.21.15.43:0/2974860595 >> [v2:172.21.15.203:6832/1778614695,v1:172.21.15.203:6833/1778614695] conn(0x7ff4d806a170 0x7ff4d806c630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:51:00.369 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:00.366+0000 7ff4e6ffd640 1 -- 172.21.15.43:0/2974860595 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(92..92 src has 1..92) v4 ==== 7114+0+0 (secure 0 0 0) 0x7ff4e80892f0 con 0x7ff4e0003f60 2023-08-09T13:51:00.369 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:00.367+0000 7ff4f5d9b640 1 --2- 172.21.15.43:0/2974860595 >> [v2:172.21.15.203:6832/1778614695,v1:172.21.15.203:6833/1778614695] conn(0x7ff4d806a170 0x7ff4d806c630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:51:00.370 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:00.368+0000 7ff4f5d9b640 1 --2- 172.21.15.43:0/2974860595 >> [v2:172.21.15.203:6832/1778614695,v1:172.21.15.203:6833/1778614695] conn(0x7ff4d806a170 0x7ff4d806c630 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0x7ff4ec00ba00 tx=0x7ff4ec032000 comp rx=0 tx=0).ready entity=mgr.5819 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-09T13:51:00.372 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:00.370+0000 7ff4e6ffd640 1 -- 172.21.15.43:0/2974860595 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178066 (secure 0 0 0) 0x7ff4e8054410 con 0x7ff4e0003f60 2023-08-09T13:51:00.609 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:00.607+0000 7ff4fcc60640 1 -- 172.21.15.43:0/2974860595 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) v1 -- 0x7ff4f815cbf0 con 0x7ff4e0003f60 2023-08-09T13:51:00.610 INFO:tasks.ceph.mon.a.smithi043.stderr:2023-08-09T13:51:00.608+0000 7f32867f3640 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2023-08-09T13:51:00.617 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:00.614+0000 7ff4e6ffd640 1 -- 172.21.15.43:0/2974860595 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v36) v1 ==== 161+0+0 (secure 0 0 0) 0x7ff4e8057a30 con 0x7ff4e0003f60 2023-08-09T13:51:00.617 INFO:teuthology.orchestra.run.smithi043.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2023-08-09T13:51:00.620 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:00.617+0000 7ff4fcc60640 1 -- 172.21.15.43:0/2974860595 >> [v2:172.21.15.203:6832/1778614695,v1:172.21.15.203:6833/1778614695] conn(0x7ff4d806a170 msgr2=0x7ff4d806c630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:51:00.621 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:00.617+0000 7ff4fcc60640 1 --2- 172.21.15.43:0/2974860595 >> [v2:172.21.15.203:6832/1778614695,v1:172.21.15.203:6833/1778614695] conn(0x7ff4d806a170 0x7ff4d806c630 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0x7ff4ec00ba00 tx=0x7ff4ec032000 comp rx=0 tx=0).stop 2023-08-09T13:51:00.621 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:00.618+0000 7ff4fcc60640 1 -- 172.21.15.43:0/2974860595 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff4e0003f60 msgr2=0x7ff4f815dec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:51:00.621 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:00.618+0000 7ff4fcc60640 1 --2- 172.21.15.43:0/2974860595 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff4e0003f60 0x7ff4f815dec0 secure :-1 s=READY pgs=1112 cs=0 l=1 rev1=1 crypto rx=0x7ff4e8007a10 tx=0x7ff4e800e530 comp rx=0 tx=0).stop 2023-08-09T13:51:00.621 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:00.618+0000 7ff4fcc60640 1 -- 172.21.15.43:0/2974860595 shutdown_connections 2023-08-09T13:51:00.621 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:00.618+0000 7ff4fcc60640 1 --2- 172.21.15.43:0/2974860595 >> [v2:172.21.15.203:6832/1778614695,v1:172.21.15.203:6833/1778614695] conn(0x7ff4d806a170 0x7ff4d806c630 unknown :-1 s=CLOSED pgs=30 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:51:00.621 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:00.618+0000 7ff4fcc60640 1 --2- 172.21.15.43:0/2974860595 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7ff4f815f810 0x7ff4f81f11a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:51:00.621 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:00.618+0000 7ff4fcc60640 1 --2- 172.21.15.43:0/2974860595 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7ff4f815f390 0x7ff4f81564b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:51:00.622 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:00.618+0000 7ff4fcc60640 1 --2- 172.21.15.43:0/2974860595 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff4e0003f60 0x7ff4f815dec0 unknown :-1 s=CLOSED pgs=1112 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:51:00.622 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:00.618+0000 7ff4fcc60640 1 -- 172.21.15.43:0/2974860595 >> 172.21.15.43:0/2974860595 conn(0x7ff4f8071520 msgr2=0x7ff4f80b65e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-09T13:51:00.622 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:00.618+0000 7ff4fcc60640 1 -- 172.21.15.43:0/2974860595 shutdown_connections 2023-08-09T13:51:00.622 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:00.618+0000 7ff4fcc60640 1 -- 172.21.15.43:0/2974860595 wait complete. 2023-08-09T13:51:00.640 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs rm cephfs --yes-i-really-mean-it 2023-08-09T13:51:00.642 INFO:tasks.ceph.mds.d.smithi203.stderr: -3250> 2023-08-09T13:50:34.123+0000 7f4eed864640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2023-08-09T13:51:00.742 INFO:tasks.ceph.mds.d.smithi203.stdout:starting mds.d at 2023-08-09T13:51:00.868 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:00.865+0000 7f4ddb18b640 1 Processor -- start 2023-08-09T13:51:00.868 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:00.865+0000 7f4ddb18b640 1 -- start start 2023-08-09T13:51:00.869 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:00.867+0000 7f4ddb18b640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f4dd415df60 0x7f4dd40ce1c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:51:00.869 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:00.867+0000 7f4ddb18b640 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7f4dd415e3d0 0x7f4dd40ce700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:51:00.870 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:00.867+0000 7f4ddb18b640 1 -- --> v1:172.21.15.203:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4dd415aef0 con 0x7f4dd40cb2d0 2023-08-09T13:51:00.870 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:00.867+0000 7f4ddb18b640 1 -- --> v2:172.21.15.43:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4dd415a970 con 0x7f4dd415df60 2023-08-09T13:51:00.870 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:00.867+0000 7f4ddb18b640 1 -- --> v2:172.21.15.203:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4dd415e940 con 0x7f4dd415e3d0 2023-08-09T13:51:00.873 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:00.867+0000 7f4dd3fff640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f4dd415df60 0x7f4dd40ce1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:51:00.873 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:00.867+0000 7f4dd3fff640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f4dd415df60 0x7f4dd40ce1c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:46694/0 (socket says 172.21.15.43:46694) 2023-08-09T13:51:00.874 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:00.867+0000 7f4dd3fff640 1 -- 172.21.15.43:0/995255627 learned_addr learned my addr 172.21.15.43:0/995255627 (peer_addr_for_me v2:172.21.15.43:0/0) 2023-08-09T13:51:00.874 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:00.868+0000 7f4dd9728640 1 --2- 172.21.15.43:0/995255627 >> v2:172.21.15.203:3300/0 conn(0x7f4dd415e3d0 0x7f4dd40ce700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:51:00.874 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:00.868+0000 7f4dd3fff640 1 -- 172.21.15.43:0/995255627 >> v2:172.21.15.203:3300/0 conn(0x7f4dd415e3d0 msgr2=0x7f4dd40ce700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-09T13:51:00.874 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:00.868+0000 7f4dd3fff640 1 --2- 172.21.15.43:0/995255627 >> v2:172.21.15.203:3300/0 conn(0x7f4dd415e3d0 0x7f4dd40ce700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:51:00.874 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:00.868+0000 7f4dd3fff640 1 -- 172.21.15.43:0/995255627 >> v1:172.21.15.203:6789/0 conn(0x7f4dd40cb2d0 legacy=0x7f4dd415b380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-09T13:51:00.874 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:00.868+0000 7f4dd3fff640 1 -- 172.21.15.43:0/995255627 --> v2:172.21.15.43:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4dd415ec20 con 0x7f4dd415df60 2023-08-09T13:51:00.874 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:00.869+0000 7f4dd3fff640 1 --2- 172.21.15.43:0/995255627 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f4dd415df60 0x7f4dd40ce1c0 secure :-1 s=READY pgs=1114 cs=0 l=1 rev1=1 crypto rx=0x7f4dd415de90 tx=0x7f4dc002b880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=491a9d84c04da762 server_cookie=0 in_seq=0 out_seq=0 2023-08-09T13:51:00.875 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:00.869+0000 7f4dd37fe640 1 -- 172.21.15.43:0/995255627 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4dc00317e0 con 0x7f4dd415df60 2023-08-09T13:51:00.875 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:00.869+0000 7f4dd37fe640 1 -- 172.21.15.43:0/995255627 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4dc0031940 con 0x7f4dd415df60 2023-08-09T13:51:00.875 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:00.869+0000 7f4dd37fe640 1 -- 172.21.15.43:0/995255627 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4dc0031cd0 con 0x7f4dd415df60 2023-08-09T13:51:00.875 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:00.870+0000 7f4ddb18b640 1 -- 172.21.15.43:0/995255627 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f4dd415df60 msgr2=0x7f4dd40ce1c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:51:00.875 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:00.870+0000 7f4ddb18b640 1 --2- 172.21.15.43:0/995255627 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f4dd415df60 0x7f4dd40ce1c0 secure :-1 s=READY pgs=1114 cs=0 l=1 rev1=1 crypto rx=0x7f4dd415de90 tx=0x7f4dc002b880 comp rx=0 tx=0).stop 2023-08-09T13:51:00.875 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:00.870+0000 7f4ddb18b640 1 -- 172.21.15.43:0/995255627 shutdown_connections 2023-08-09T13:51:00.875 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:00.870+0000 7f4ddb18b640 1 --2- 172.21.15.43:0/995255627 >> v2:172.21.15.203:3300/0 conn(0x7f4dd415e3d0 0x7f4dd40ce700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:51:00.876 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:00.870+0000 7f4ddb18b640 1 --2- 172.21.15.43:0/995255627 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f4dd415df60 0x7f4dd40ce1c0 unknown :-1 s=CLOSED pgs=1114 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:51:00.876 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:00.870+0000 7f4ddb18b640 1 -- 172.21.15.43:0/995255627 >> 172.21.15.43:0/995255627 conn(0x7f4dd40aff50 msgr2=0x7f4dd40b0360 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-09T13:51:00.876 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:00.870+0000 7f4ddb18b640 1 -- 172.21.15.43:0/995255627 shutdown_connections 2023-08-09T13:51:00.876 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:00.870+0000 7f4ddb18b640 1 -- 172.21.15.43:0/995255627 wait complete. 2023-08-09T13:51:00.877 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:00.873+0000 7f4ddb18b640 1 Processor -- start 2023-08-09T13:51:00.878 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:00.873+0000 7f4ddb18b640 1 -- start start 2023-08-09T13:51:00.878 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:00.874+0000 7f4ddb18b640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f4dd40cb2d0 0x7f4dd4151830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:51:00.878 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:00.874+0000 7f4ddb18b640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f4dd415df60 0x7f4dd40c7990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:51:00.878 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:00.874+0000 7f4ddb18b640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f4dd415e3d0 0x7f4dd40cbda0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:51:00.878 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:00.874+0000 7f4ddb18b640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4dd4151d70 con 0x7f4dd415e3d0 2023-08-09T13:51:00.878 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:00.874+0000 7f4ddb18b640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4dd40b24a0 con 0x7f4dd40cb2d0 2023-08-09T13:51:00.879 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:00.874+0000 7f4ddb18b640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4dd40b2780 con 0x7f4dd415df60 2023-08-09T13:51:00.879 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:00.875+0000 7f4dd9728640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f4dd415e3d0 0x7f4dd40cbda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:51:00.879 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:00.875+0000 7f4dd9728640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f4dd415e3d0 0x7f4dd40cbda0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:46702/0 (socket says 172.21.15.43:46702) 2023-08-09T13:51:00.879 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:00.875+0000 7f4dd9728640 1 -- 172.21.15.43:0/2468965364 learned_addr learned my addr 172.21.15.43:0/2468965364 (peer_addr_for_me v2:172.21.15.43:0/0) 2023-08-09T13:51:00.879 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:00.875+0000 7f4dd8f27640 1 --2- 172.21.15.43:0/2468965364 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f4dd40cb2d0 0x7f4dd4151830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:51:00.879 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:00.875+0000 7f4dd3fff640 1 --2- 172.21.15.43:0/2468965364 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f4dd415df60 0x7f4dd40c7990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:51:00.880 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:00.875+0000 7f4dd9728640 1 -- 172.21.15.43:0/2468965364 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f4dd415df60 msgr2=0x7f4dd40c7990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:51:00.880 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:00.875+0000 7f4dd9728640 1 --2- 172.21.15.43:0/2468965364 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f4dd415df60 0x7f4dd40c7990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:51:00.880 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:00.875+0000 7f4dd9728640 1 -- 172.21.15.43:0/2468965364 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f4dd40cb2d0 msgr2=0x7f4dd4151830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:51:00.880 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:00.875+0000 7f4dd9728640 1 --2- 172.21.15.43:0/2468965364 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f4dd40cb2d0 0x7f4dd4151830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:51:00.880 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:00.875+0000 7f4dd9728640 1 -- 172.21.15.43:0/2468965364 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4dc00094d0 con 0x7f4dd415e3d0 2023-08-09T13:51:00.881 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:00.876+0000 7f4dd9728640 1 --2- 172.21.15.43:0/2468965364 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f4dd415e3d0 0x7f4dd40cbda0 secure :-1 s=READY pgs=1115 cs=0 l=1 rev1=1 crypto rx=0x7f4dcc007e60 tx=0x7f4dcc00e4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-09T13:51:00.881 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:00.876+0000 7f4dd17fa640 1 -- 172.21.15.43:0/2468965364 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4dcc014420 con 0x7f4dd415e3d0 2023-08-09T13:51:00.882 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:00.876+0000 7f4dd17fa640 1 -- 172.21.15.43:0/2468965364 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4dcc0145a0 con 0x7f4dd415e3d0 2023-08-09T13:51:00.882 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:00.876+0000 7f4dd17fa640 1 -- 172.21.15.43:0/2468965364 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4dcc014950 con 0x7f4dd415e3d0 2023-08-09T13:51:00.882 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:00.876+0000 7f4ddb18b640 1 -- 172.21.15.43:0/2468965364 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4dd40b2a60 con 0x7f4dd415e3d0 2023-08-09T13:51:00.882 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:00.876+0000 7f4ddb18b640 1 -- 172.21.15.43:0/2468965364 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4dd40b3020 con 0x7f4dd415e3d0 2023-08-09T13:51:00.882 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:00.877+0000 7f4ddb18b640 1 -- 172.21.15.43:0/2468965364 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4dd40739f0 con 0x7f4dd415e3d0 2023-08-09T13:51:00.882 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:00.878+0000 7f4dd17fa640 1 -- 172.21.15.43:0/2468965364 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 37) v1 ==== 88443+0+0 (secure 0 0 0) 0x7f4dcc014ab0 con 0x7f4dd415e3d0 2023-08-09T13:51:00.883 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:00.879+0000 7f4dd17fa640 1 --2- 172.21.15.43:0/2468965364 >> [v2:172.21.15.203:6832/1778614695,v1:172.21.15.203:6833/1778614695] conn(0x7f4db806a170 0x7f4db806c630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:51:00.883 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:00.880+0000 7f4dd17fa640 1 -- 172.21.15.43:0/2468965364 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(93..93 src has 1..93) v4 ==== 7200+0+0 (secure 0 0 0) 0x7f4dcc089550 con 0x7f4dd415e3d0 2023-08-09T13:51:00.883 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:00.880+0000 7f4dd8f27640 1 --2- 172.21.15.43:0/2468965364 >> [v2:172.21.15.203:6832/1778614695,v1:172.21.15.203:6833/1778614695] conn(0x7f4db806a170 0x7f4db806c630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:51:00.884 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:00.881+0000 7f4dd8f27640 1 --2- 172.21.15.43:0/2468965364 >> [v2:172.21.15.203:6832/1778614695,v1:172.21.15.203:6833/1778614695] conn(0x7f4db806a170 0x7f4db806c630 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0x7f4dc8004ea0 tx=0x7f4dc8009040 comp rx=0 tx=0).ready entity=mgr.5819 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-09T13:51:00.886 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:00.883+0000 7f4dd17fa640 1 -- 172.21.15.43:0/2468965364 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178066 (secure 0 0 0) 0x7f4dcc0580d0 con 0x7f4dd415e3d0 2023-08-09T13:51:01.118 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:01.116+0000 7f4ddb18b640 1 -- 172.21.15.43:0/2468965364 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} v 0) v1 -- 0x7f4dd4154d80 con 0x7f4dd415e3d0 2023-08-09T13:51:01.632 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:01.629+0000 7f4dd17fa640 1 -- 172.21.15.43:0/2468965364 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]=0 v37) v1 ==== 104+0+0 (secure 0 0 0) 0x7f4dcc057c60 con 0x7f4dd415e3d0 2023-08-09T13:51:01.635 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:01.632+0000 7f4ddb18b640 1 -- 172.21.15.43:0/2468965364 >> [v2:172.21.15.203:6832/1778614695,v1:172.21.15.203:6833/1778614695] conn(0x7f4db806a170 msgr2=0x7f4db806c630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:51:01.636 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:01.632+0000 7f4ddb18b640 1 --2- 172.21.15.43:0/2468965364 >> [v2:172.21.15.203:6832/1778614695,v1:172.21.15.203:6833/1778614695] conn(0x7f4db806a170 0x7f4db806c630 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0x7f4dc8004ea0 tx=0x7f4dc8009040 comp rx=0 tx=0).stop 2023-08-09T13:51:01.636 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:01.632+0000 7f4ddb18b640 1 -- 172.21.15.43:0/2468965364 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f4dd415e3d0 msgr2=0x7f4dd40cbda0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:51:01.637 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:01.632+0000 7f4ddb18b640 1 --2- 172.21.15.43:0/2468965364 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f4dd415e3d0 0x7f4dd40cbda0 secure :-1 s=READY pgs=1115 cs=0 l=1 rev1=1 crypto rx=0x7f4dcc007e60 tx=0x7f4dcc00e4b0 comp rx=0 tx=0).stop 2023-08-09T13:51:01.637 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:01.633+0000 7f4ddb18b640 1 -- 172.21.15.43:0/2468965364 shutdown_connections 2023-08-09T13:51:01.637 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:01.633+0000 7f4ddb18b640 1 --2- 172.21.15.43:0/2468965364 >> [v2:172.21.15.203:6832/1778614695,v1:172.21.15.203:6833/1778614695] conn(0x7f4db806a170 0x7f4db806c630 unknown :-1 s=CLOSED pgs=31 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:51:01.638 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:01.633+0000 7f4ddb18b640 1 --2- 172.21.15.43:0/2468965364 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f4dd415e3d0 0x7f4dd40cbda0 unknown :-1 s=CLOSED pgs=1115 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:51:01.638 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:01.633+0000 7f4ddb18b640 1 --2- 172.21.15.43:0/2468965364 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f4dd415df60 0x7f4dd40c7990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:51:01.638 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:01.633+0000 7f4ddb18b640 1 --2- 172.21.15.43:0/2468965364 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f4dd40cb2d0 0x7f4dd4151830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:51:01.639 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:01.633+0000 7f4ddb18b640 1 -- 172.21.15.43:0/2468965364 >> 172.21.15.43:0/2468965364 conn(0x7f4dd40aff50 msgr2=0x7f4dd415b380 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-09T13:51:01.639 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:01.633+0000 7f4ddb18b640 1 -- 172.21.15.43:0/2468965364 shutdown_connections 2023-08-09T13:51:01.639 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:01.633+0000 7f4ddb18b640 1 -- 172.21.15.43:0/2468965364 wait complete. 2023-08-09T13:51:01.657 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_metadata cephfs_metadata --yes-i-really-really-mean-it 2023-08-09T13:51:01.889 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:01.886+0000 7f9877fff640 1 Processor -- start 2023-08-09T13:51:01.889 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:01.886+0000 7f9877fff640 1 -- start start 2023-08-09T13:51:01.890 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:01.888+0000 7f9877fff640 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7f98780cf370 0x7f9878167220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:51:01.890 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:01.888+0000 7f9877fff640 1 -- --> v1:172.21.15.43:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f98780cdce0 con 0x7f987815c720 2023-08-09T13:51:01.890 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:01.888+0000 7f9877fff640 1 -- --> v1:172.21.15.203:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f98780b5660 con 0x7f98780cd850 2023-08-09T13:51:01.891 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:01.888+0000 7f9877fff640 1 -- --> v2:172.21.15.203:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f987815cb00 con 0x7f98780cf370 2023-08-09T13:51:01.892 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:01.888+0000 7f9876ffd640 1 --1- >> v1:172.21.15.43:6789/0 conn(0x7f987815c720 0x7f98780d0900 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.43:6789/0 says I am v1:172.21.15.43:35096/0 (socket says 172.21.15.43:35096) 2023-08-09T13:51:01.892 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:01.888+0000 7f9876ffd640 1 -- 172.21.15.43:0/2411518398 learned_addr learned my addr 172.21.15.43:0/2411518398 (peer_addr_for_me v1:172.21.15.43:0/0) 2023-08-09T13:51:01.892 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:01.889+0000 7f98767fc640 1 -- 172.21.15.43:0/2411518398 <== mon.0 v1:172.21.15.43:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3520420201 0 0) 0x7f98780cdce0 con 0x7f987815c720 2023-08-09T13:51:01.892 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:01.889+0000 7f98777fe640 1 --2- 172.21.15.43:0/2411518398 >> v2:172.21.15.203:3300/0 conn(0x7f98780cf370 0x7f9878167220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:51:01.892 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:01.889+0000 7f98767fc640 1 -- 172.21.15.43:0/2411518398 --> v1:172.21.15.43:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f9864003520 con 0x7f987815c720 2023-08-09T13:51:01.893 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:01.889+0000 7f98767fc640 1 -- 172.21.15.43:0/2411518398 <== mon.0 v1:172.21.15.43:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 143402146 0 0) 0x7f9864003520 con 0x7f987815c720 2023-08-09T13:51:01.893 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:01.889+0000 7f98767fc640 1 -- 172.21.15.43:0/2411518398 >> v2:172.21.15.203:3300/0 conn(0x7f98780cf370 msgr2=0x7f9878167220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-09T13:51:01.893 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:01.889+0000 7f98767fc640 1 --2- 172.21.15.43:0/2411518398 >> v2:172.21.15.203:3300/0 conn(0x7f98780cf370 0x7f9878167220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:51:01.893 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:01.889+0000 7f98767fc640 1 -- 172.21.15.43:0/2411518398 >> v1:172.21.15.203:6789/0 conn(0x7f98780cd850 legacy=0x7f98780cec60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:51:01.893 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:01.889+0000 7f98767fc640 1 -- 172.21.15.43:0/2411518398 --> v1:172.21.15.43:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f98780b5900 con 0x7f987815c720 2023-08-09T13:51:01.893 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:01.889+0000 7f98767fc640 1 -- 172.21.15.43:0/2411518398 <== mon.0 v1:172.21.15.43:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1285150740 0 0) 0x7f9868001670 con 0x7f987815c720 2023-08-09T13:51:01.893 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:01.889+0000 7f98767fc640 1 -- 172.21.15.43:0/2411518398 >> v1:172.21.15.43:6789/0 conn(0x7f987815c720 legacy=0x7f98780d0900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:51:01.894 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:01.889+0000 7f98767fc640 1 --2- 172.21.15.43:0/2411518398 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f9864003970 0x7f9864003d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:51:01.894 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:01.889+0000 7f98767fc640 1 --2- 172.21.15.43:0/2411518398 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f98640047c0 0x7f9864004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:51:01.894 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:01.889+0000 7f98767fc640 1 --2- 172.21.15.43:0/2411518398 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f9864005180 0x7f986400b980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:51:01.894 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:01.889+0000 7f98767fc640 1 -- 172.21.15.43:0/2411518398 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f987815cb00 con 0x7f98640047c0 2023-08-09T13:51:01.894 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:01.889+0000 7f98767fc640 1 -- 172.21.15.43:0/2411518398 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f986400c020 con 0x7f9864003970 2023-08-09T13:51:01.894 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:01.889+0000 7f98767fc640 1 -- 172.21.15.43:0/2411518398 --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f986400c1e0 con 0x7f9864005180 2023-08-09T13:51:01.894 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:01.890+0000 7f986ffff640 1 --2- 172.21.15.43:0/2411518398 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f98640047c0 0x7f9864004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:51:01.897 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:01.890+0000 7f9876ffd640 1 --2- 172.21.15.43:0/2411518398 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f9864003970 0x7f9864003d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:51:01.898 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:01.890+0000 7f986ffff640 1 -- 172.21.15.43:0/2411518398 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f9864005180 msgr2=0x7f986400b980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-09T13:51:01.899 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:01.890+0000 7f98777fe640 1 --2- 172.21.15.43:0/2411518398 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f9864005180 0x7f986400b980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:51:01.899 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:01.890+0000 7f986ffff640 1 --2- 172.21.15.43:0/2411518398 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f9864005180 0x7f986400b980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:51:01.899 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:01.890+0000 7f986ffff640 1 -- 172.21.15.43:0/2411518398 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f9864003970 msgr2=0x7f9864003d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-09T13:51:01.899 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:01.890+0000 7f986ffff640 1 --2- 172.21.15.43:0/2411518398 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f9864003970 0x7f9864003d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:51:01.899 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:01.890+0000 7f986ffff640 1 -- 172.21.15.43:0/2411518398 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9864003520 con 0x7f98640047c0 2023-08-09T13:51:01.899 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:01.890+0000 7f9876ffd640 1 --2- 172.21.15.43:0/2411518398 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f9864003970 0x7f9864003d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-08-09T13:51:01.899 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:01.890+0000 7f98777fe640 1 -- 172.21.15.43:0/2411518398 reap_dead start 2023-08-09T13:51:01.900 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:01.891+0000 7f986ffff640 1 --2- 172.21.15.43:0/2411518398 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f98640047c0 0x7f9864004c40 secure :-1 s=READY pgs=1117 cs=0 l=1 rev1=1 crypto rx=0x7f9860001750 tx=0x7f98640057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ae428f0d6e740257 server_cookie=0 in_seq=0 out_seq=0 2023-08-09T13:51:01.900 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:01.892+0000 7f98767fc640 1 -- 172.21.15.43:0/2411518398 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f98600052d0 con 0x7f98640047c0 2023-08-09T13:51:01.900 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:01.892+0000 7f98767fc640 1 -- 172.21.15.43:0/2411518398 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f986002e050 con 0x7f98640047c0 2023-08-09T13:51:01.900 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:01.892+0000 7f98767fc640 1 -- 172.21.15.43:0/2411518398 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9860003830 con 0x7f98640047c0 2023-08-09T13:51:01.900 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:01.892+0000 7f9877fff640 1 -- 172.21.15.43:0/2411518398 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f98640047c0 msgr2=0x7f9864004c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:51:01.900 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:01.892+0000 7f9877fff640 1 --2- 172.21.15.43:0/2411518398 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f98640047c0 0x7f9864004c40 secure :-1 s=READY pgs=1117 cs=0 l=1 rev1=1 crypto rx=0x7f9860001750 tx=0x7f98640057f0 comp rx=0 tx=0).stop 2023-08-09T13:51:01.900 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:01.892+0000 7f9877fff640 1 -- 172.21.15.43:0/2411518398 shutdown_connections 2023-08-09T13:51:01.901 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:01.892+0000 7f9877fff640 1 --2- 172.21.15.43:0/2411518398 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f98640047c0 0x7f9864004c40 unknown :-1 s=CLOSED pgs=1117 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:51:01.901 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:01.892+0000 7f9877fff640 1 -- 172.21.15.43:0/2411518398 >> 172.21.15.43:0/2411518398 conn(0x7f987806a2a0 msgr2=0x7f987806a680 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-09T13:51:01.901 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:01.892+0000 7f9877fff640 1 -- 172.21.15.43:0/2411518398 shutdown_connections 2023-08-09T13:51:01.901 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:01.892+0000 7f9877fff640 1 -- 172.21.15.43:0/2411518398 wait complete. 2023-08-09T13:51:01.901 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:01.894+0000 7f9877fff640 1 Processor -- start 2023-08-09T13:51:01.901 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:01.894+0000 7f9877fff640 1 -- start start 2023-08-09T13:51:01.903 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:01.895+0000 7f9877fff640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f98640047c0 0x7f9878155940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:51:01.903 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:01.895+0000 7f9877fff640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f987815ec50 0x7f987815f0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:51:01.903 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:01.895+0000 7f9877fff640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f98780c6fa0 0x7f98780c7420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:51:01.904 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:01.895+0000 7f9877fff640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f987815f610 con 0x7f98780c6fa0 2023-08-09T13:51:01.904 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:01.895+0000 7f9877fff640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f98780c7960 con 0x7f98640047c0 2023-08-09T13:51:01.904 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:01.895+0000 7f9877fff640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f98780b4ad0 con 0x7f987815ec50 2023-08-09T13:51:01.904 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:01.895+0000 7f98777fe640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f98780c6fa0 0x7f98780c7420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:51:01.904 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:01.896+0000 7f98777fe640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f98780c6fa0 0x7f98780c7420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:46712/0 (socket says 172.21.15.43:46712) 2023-08-09T13:51:01.904 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:01.896+0000 7f98777fe640 1 -- 172.21.15.43:0/2451802102 learned_addr learned my addr 172.21.15.43:0/2451802102 (peer_addr_for_me v2:172.21.15.43:0/0) 2023-08-09T13:51:01.904 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:01.896+0000 7f9876ffd640 1 --2- 172.21.15.43:0/2451802102 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f98640047c0 0x7f9878155940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:51:01.905 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:01.896+0000 7f986ffff640 1 --2- 172.21.15.43:0/2451802102 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f987815ec50 0x7f987815f0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:51:01.905 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:01.896+0000 7f98777fe640 1 -- 172.21.15.43:0/2451802102 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f987815ec50 msgr2=0x7f987815f0d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:51:01.905 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:01.896+0000 7f98777fe640 1 --2- 172.21.15.43:0/2451802102 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f987815ec50 0x7f987815f0d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:51:01.905 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:01.896+0000 7f98777fe640 1 -- 172.21.15.43:0/2451802102 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f98640047c0 msgr2=0x7f9878155940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:51:01.905 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:01.896+0000 7f98777fe640 1 --2- 172.21.15.43:0/2451802102 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f98640047c0 0x7f9878155940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:51:01.905 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:01.896+0000 7f98777fe640 1 -- 172.21.15.43:0/2451802102 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9878155e80 con 0x7f98780c6fa0 2023-08-09T13:51:01.905 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:01.896+0000 7f9876ffd640 1 --2- 172.21.15.43:0/2451802102 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f98640047c0 0x7f9878155940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-08-09T13:51:01.906 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:01.896+0000 7f986ffff640 1 --2- 172.21.15.43:0/2451802102 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f987815ec50 0x7f987815f0d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-08-09T13:51:01.906 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:01.896+0000 7f98777fe640 1 --2- 172.21.15.43:0/2451802102 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f98780c6fa0 0x7f98780c7420 secure :-1 s=READY pgs=1118 cs=0 l=1 rev1=1 crypto rx=0x7f98780cc7c0 tx=0x7f987000f520 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-09T13:51:01.906 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:01.896+0000 7f986f7fe640 1 -- 172.21.15.43:0/2451802102 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f987001a070 con 0x7f98780c6fa0 2023-08-09T13:51:01.906 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:01.897+0000 7f986f7fe640 1 -- 172.21.15.43:0/2451802102 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f987000fdc0 con 0x7f98780c6fa0 2023-08-09T13:51:01.906 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:01.897+0000 7f986f7fe640 1 -- 172.21.15.43:0/2451802102 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9870003bb0 con 0x7f98780c6fa0 2023-08-09T13:51:01.906 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:01.897+0000 7f9877fff640 1 -- 172.21.15.43:0/2451802102 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f98780c7b80 con 0x7f98780c6fa0 2023-08-09T13:51:01.907 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:01.897+0000 7f9877fff640 1 -- 172.21.15.43:0/2451802102 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f98780c8140 con 0x7f98780c6fa0 2023-08-09T13:51:01.907 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:01.898+0000 7f9877fff640 1 -- 172.21.15.43:0/2451802102 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f98780739f0 con 0x7f98780c6fa0 2023-08-09T13:51:01.907 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:01.899+0000 7f986f7fe640 1 -- 172.21.15.43:0/2451802102 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 37) v1 ==== 88443+0+0 (secure 0 0 0) 0x7f9870016420 con 0x7f98780c6fa0 2023-08-09T13:51:01.907 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:01.900+0000 7f986f7fe640 1 --2- 172.21.15.43:0/2451802102 >> [v2:172.21.15.203:6832/1778614695,v1:172.21.15.203:6833/1778614695] conn(0x7f985806a1a0 0x7f985806c660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:51:01.907 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:01.900+0000 7f986f7fe640 1 -- 172.21.15.43:0/2451802102 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(93..93 src has 1..93) v4 ==== 7200+0+0 (secure 0 0 0) 0x7f98700895a0 con 0x7f98780c6fa0 2023-08-09T13:51:01.907 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:01.900+0000 7f9876ffd640 1 --2- 172.21.15.43:0/2451802102 >> [v2:172.21.15.203:6832/1778614695,v1:172.21.15.203:6833/1778614695] conn(0x7f985806a1a0 0x7f985806c660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:51:01.908 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:01.902+0000 7f9876ffd640 1 --2- 172.21.15.43:0/2451802102 >> [v2:172.21.15.203:6832/1778614695,v1:172.21.15.203:6833/1778614695] conn(0x7f985806a1a0 0x7f985806c660 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0x7f9868001200 tx=0x7f9868009040 comp rx=0 tx=0).ready entity=mgr.5819 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-09T13:51:01.908 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:01.904+0000 7f986f7fe640 1 -- 172.21.15.43:0/2451802102 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178066 (secure 0 0 0) 0x7f9870054610 con 0x7f98780c6fa0 2023-08-09T13:51:02.102 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:02.099+0000 7f9877fff640 1 -- 172.21.15.43:0/2451802102 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7f98780cbc60 con 0x7f98780c6fa0 2023-08-09T13:51:03.115 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:03.111+0000 7f986f7fe640 1 -- 172.21.15.43:0/2451802102 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_metadata' removed v94) v1 ==== 181+0+0 (secure 0 0 0) 0x7f9870057c30 con 0x7f98780c6fa0 2023-08-09T13:51:03.115 INFO:teuthology.orchestra.run.smithi043.stderr:pool 'cephfs_metadata' removed 2023-08-09T13:51:03.119 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:03.116+0000 7f9877fff640 1 -- 172.21.15.43:0/2451802102 >> [v2:172.21.15.203:6832/1778614695,v1:172.21.15.203:6833/1778614695] conn(0x7f985806a1a0 msgr2=0x7f985806c660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:51:03.119 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:03.116+0000 7f9877fff640 1 --2- 172.21.15.43:0/2451802102 >> [v2:172.21.15.203:6832/1778614695,v1:172.21.15.203:6833/1778614695] conn(0x7f985806a1a0 0x7f985806c660 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0x7f9868001200 tx=0x7f9868009040 comp rx=0 tx=0).stop 2023-08-09T13:51:03.119 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:03.116+0000 7f9877fff640 1 -- 172.21.15.43:0/2451802102 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f98780c6fa0 msgr2=0x7f98780c7420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:51:03.120 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:03.116+0000 7f9877fff640 1 --2- 172.21.15.43:0/2451802102 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f98780c6fa0 0x7f98780c7420 secure :-1 s=READY pgs=1118 cs=0 l=1 rev1=1 crypto rx=0x7f98780cc7c0 tx=0x7f987000f520 comp rx=0 tx=0).stop 2023-08-09T13:51:03.120 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:03.117+0000 7f9877fff640 1 -- 172.21.15.43:0/2451802102 shutdown_connections 2023-08-09T13:51:03.121 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:03.117+0000 7f9877fff640 1 --2- 172.21.15.43:0/2451802102 >> [v2:172.21.15.203:6832/1778614695,v1:172.21.15.203:6833/1778614695] conn(0x7f985806a1a0 0x7f985806c660 unknown :-1 s=CLOSED pgs=33 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:51:03.121 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:03.117+0000 7f9877fff640 1 --2- 172.21.15.43:0/2451802102 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f98780c6fa0 0x7f98780c7420 unknown :-1 s=CLOSED pgs=1118 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:51:03.121 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:03.117+0000 7f9877fff640 1 --2- 172.21.15.43:0/2451802102 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f987815ec50 0x7f987815f0d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:51:03.121 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:03.117+0000 7f9877fff640 1 --2- 172.21.15.43:0/2451802102 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f98640047c0 0x7f9878155940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:51:03.121 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:03.117+0000 7f9877fff640 1 -- 172.21.15.43:0/2451802102 >> 172.21.15.43:0/2451802102 conn(0x7f987806a2a0 msgr2=0x7f98780c95e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-09T13:51:03.121 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:03.117+0000 7f9877fff640 1 -- 172.21.15.43:0/2451802102 shutdown_connections 2023-08-09T13:51:03.121 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:03.118+0000 7f9877fff640 1 -- 172.21.15.43:0/2451802102 wait complete. 2023-08-09T13:51:03.144 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_data cephfs_data --yes-i-really-really-mean-it 2023-08-09T13:51:03.383 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:03.380+0000 7fb396214640 1 Processor -- start 2023-08-09T13:51:03.383 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:03.380+0000 7fb396214640 1 -- start start 2023-08-09T13:51:03.384 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:03.382+0000 7fb396214640 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7fb3900ce4c0 0x7fb3900ce950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:51:03.385 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:03.382+0000 7fb396214640 1 -- --> v1:172.21.15.43:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb39015b9b0 con 0x7fb3900ca370 2023-08-09T13:51:03.385 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:03.382+0000 7fb396214640 1 -- --> v1:172.21.15.203:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb39015ae10 con 0x7fb39006fb70 2023-08-09T13:51:03.385 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:03.382+0000 7fb396214640 1 -- --> v2:172.21.15.203:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb3900cee90 con 0x7fb3900ce4c0 2023-08-09T13:51:03.386 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:03.382+0000 7fb386ffd640 1 --1- >> v1:172.21.15.43:6789/0 conn(0x7fb3900ca370 0x7fb39015b2a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.43:6789/0 says I am v1:172.21.15.43:35100/0 (socket says 172.21.15.43:35100) 2023-08-09T13:51:03.386 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:03.382+0000 7fb386ffd640 1 -- 172.21.15.43:0/622681529 learned_addr learned my addr 172.21.15.43:0/622681529 (peer_addr_for_me v1:172.21.15.43:0/0) 2023-08-09T13:51:03.387 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:03.383+0000 7fb38ffff640 1 --2- 172.21.15.43:0/622681529 >> v2:172.21.15.203:3300/0 conn(0x7fb3900ce4c0 0x7fb3900ce950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:51:03.387 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:03.383+0000 7fb38effd640 1 -- 172.21.15.43:0/622681529 <== mon.0 v1:172.21.15.43:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2446538873 0 0) 0x7fb378001110 con 0x7fb3900ca370 2023-08-09T13:51:03.387 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:03.383+0000 7fb38effd640 1 -- 172.21.15.43:0/622681529 --> v1:172.21.15.43:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fb37c003520 con 0x7fb3900ca370 2023-08-09T13:51:03.387 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:03.383+0000 7fb38effd640 1 -- 172.21.15.43:0/622681529 <== mon.0 v1:172.21.15.43:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3156774433 0 0) 0x7fb37c003520 con 0x7fb3900ca370 2023-08-09T13:51:03.387 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:03.383+0000 7fb38effd640 1 -- 172.21.15.43:0/622681529 >> v2:172.21.15.203:3300/0 conn(0x7fb3900ce4c0 msgr2=0x7fb3900ce950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-09T13:51:03.387 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:03.383+0000 7fb38effd640 1 --2- 172.21.15.43:0/622681529 >> v2:172.21.15.203:3300/0 conn(0x7fb3900ce4c0 0x7fb3900ce950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:51:03.387 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:03.383+0000 7fb38effd640 1 -- 172.21.15.43:0/622681529 >> v1:172.21.15.203:6789/0 conn(0x7fb39006fb70 legacy=0x7fb3900caab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:51:03.388 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:03.383+0000 7fb38effd640 1 -- 172.21.15.43:0/622681529 --> v1:172.21.15.43:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb390154e70 con 0x7fb3900ca370 2023-08-09T13:51:03.388 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:03.383+0000 7fb38effd640 1 -- 172.21.15.43:0/622681529 <== mon.0 v1:172.21.15.43:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1285150740 0 0) 0x7fb378001800 con 0x7fb3900ca370 2023-08-09T13:51:03.388 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:03.383+0000 7fb38effd640 1 -- 172.21.15.43:0/622681529 >> v1:172.21.15.43:6789/0 conn(0x7fb3900ca370 legacy=0x7fb39015b2a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:51:03.388 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:03.384+0000 7fb38effd640 1 --2- 172.21.15.43:0/622681529 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb37c0039e0 0x7fb37c003e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:51:03.388 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:03.384+0000 7fb38effd640 1 --2- 172.21.15.43:0/622681529 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fb37c0047c0 0x7fb37c004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:51:03.388 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:03.384+0000 7fb38effd640 1 --2- 172.21.15.43:0/622681529 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fb37c005180 0x7fb37c00b980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:51:03.388 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:03.384+0000 7fb38effd640 1 -- 172.21.15.43:0/622681529 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb3900cee90 con 0x7fb37c0039e0 2023-08-09T13:51:03.389 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:03.384+0000 7fb38effd640 1 -- 172.21.15.43:0/622681529 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb37c00c0b0 con 0x7fb37c005180 2023-08-09T13:51:03.389 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:03.384+0000 7fb38effd640 1 -- 172.21.15.43:0/622681529 --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb37c00c270 con 0x7fb37c0047c0 2023-08-09T13:51:03.389 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:03.384+0000 7fb38f7fe640 1 --2- 172.21.15.43:0/622681529 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb37c0039e0 0x7fb37c003e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:51:03.390 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:03.384+0000 7fb38f7fe640 1 -- 172.21.15.43:0/622681529 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fb37c0047c0 msgr2=0x7fb37c004c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-09T13:51:03.390 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:03.384+0000 7fb38f7fe640 1 --2- 172.21.15.43:0/622681529 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fb37c0047c0 0x7fb37c004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:51:03.390 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:03.384+0000 7fb38f7fe640 1 -- 172.21.15.43:0/622681529 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fb37c005180 msgr2=0x7fb37c00b980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-09T13:51:03.390 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:03.384+0000 7fb38f7fe640 1 --2- 172.21.15.43:0/622681529 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fb37c005180 0x7fb37c00b980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:51:03.391 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:03.384+0000 7fb38f7fe640 1 -- 172.21.15.43:0/622681529 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb37c003520 con 0x7fb37c0039e0 2023-08-09T13:51:03.391 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:03.384+0000 7fb38ffff640 1 -- 172.21.15.43:0/622681529 reap_dead start 2023-08-09T13:51:03.391 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:03.385+0000 7fb38f7fe640 1 --2- 172.21.15.43:0/622681529 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb37c0039e0 0x7fb37c003e00 secure :-1 s=READY pgs=1120 cs=0 l=1 rev1=1 crypto rx=0x7fb380001e50 tx=0x7fb37c0057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c22705c817405b74 server_cookie=0 in_seq=0 out_seq=0 2023-08-09T13:51:03.391 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:03.386+0000 7fb38effd640 1 -- 172.21.15.43:0/622681529 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb380003b70 con 0x7fb37c0039e0 2023-08-09T13:51:03.391 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:03.386+0000 7fb38effd640 1 -- 172.21.15.43:0/622681529 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb380003cd0 con 0x7fb37c0039e0 2023-08-09T13:51:03.391 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:03.386+0000 7fb38effd640 1 -- 172.21.15.43:0/622681529 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb380005310 con 0x7fb37c0039e0 2023-08-09T13:51:03.391 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:03.386+0000 7fb396214640 1 -- 172.21.15.43:0/622681529 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb37c0039e0 msgr2=0x7fb37c003e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:51:03.392 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:03.386+0000 7fb396214640 1 --2- 172.21.15.43:0/622681529 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb37c0039e0 0x7fb37c003e00 secure :-1 s=READY pgs=1120 cs=0 l=1 rev1=1 crypto rx=0x7fb380001e50 tx=0x7fb37c0057f0 comp rx=0 tx=0).stop 2023-08-09T13:51:03.392 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:03.386+0000 7fb396214640 1 -- 172.21.15.43:0/622681529 shutdown_connections 2023-08-09T13:51:03.392 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:03.386+0000 7fb396214640 1 --2- 172.21.15.43:0/622681529 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb37c0039e0 0x7fb37c003e00 unknown :-1 s=CLOSED pgs=1120 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:51:03.392 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:03.386+0000 7fb396214640 1 -- 172.21.15.43:0/622681529 >> 172.21.15.43:0/622681529 conn(0x7fb39006b990 msgr2=0x7fb39006bda0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-09T13:51:03.392 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:03.386+0000 7fb396214640 1 -- 172.21.15.43:0/622681529 shutdown_connections 2023-08-09T13:51:03.392 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:03.386+0000 7fb396214640 1 -- 172.21.15.43:0/622681529 wait complete. 2023-08-09T13:51:03.394 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:03.388+0000 7fb396214640 1 Processor -- start 2023-08-09T13:51:03.394 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:03.388+0000 7fb396214640 1 -- start start 2023-08-09T13:51:03.394 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:03.389+0000 7fb396214640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fb37c0039e0 0x7fb390148fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:51:03.395 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:03.389+0000 7fb396214640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fb3901ef5d0 0x7fb390149500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:51:03.395 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:03.389+0000 7fb396214640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb390149a40 0x7fb390149ec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:51:03.395 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:03.389+0000 7fb396214640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb390147fb0 con 0x7fb390149a40 2023-08-09T13:51:03.395 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:03.389+0000 7fb396214640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb390148200 con 0x7fb37c0039e0 2023-08-09T13:51:03.395 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:03.389+0000 7fb396214640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb3901484e0 con 0x7fb3901ef5d0 2023-08-09T13:51:03.395 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:03.390+0000 7fb38ffff640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb390149a40 0x7fb390149ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:51:03.395 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:03.390+0000 7fb38ffff640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb390149a40 0x7fb390149ec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:46726/0 (socket says 172.21.15.43:46726) 2023-08-09T13:51:03.395 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:03.390+0000 7fb38ffff640 1 -- 172.21.15.43:0/3793457751 learned_addr learned my addr 172.21.15.43:0/3793457751 (peer_addr_for_me v2:172.21.15.43:0/0) 2023-08-09T13:51:03.396 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:03.390+0000 7fb386ffd640 1 --2- 172.21.15.43:0/3793457751 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fb3901ef5d0 0x7fb390149500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:51:03.396 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:03.390+0000 7fb38f7fe640 1 --2- 172.21.15.43:0/3793457751 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fb37c0039e0 0x7fb390148fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:51:03.396 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:03.390+0000 7fb38ffff640 1 -- 172.21.15.43:0/3793457751 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fb3901ef5d0 msgr2=0x7fb390149500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:51:03.396 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:03.390+0000 7fb38ffff640 1 --2- 172.21.15.43:0/3793457751 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fb3901ef5d0 0x7fb390149500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:51:03.396 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:03.390+0000 7fb38ffff640 1 -- 172.21.15.43:0/3793457751 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fb37c0039e0 msgr2=0x7fb390148fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:51:03.396 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:03.390+0000 7fb38ffff640 1 --2- 172.21.15.43:0/3793457751 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fb37c0039e0 0x7fb390148fc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:51:03.396 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:03.390+0000 7fb38ffff640 1 -- 172.21.15.43:0/3793457751 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb3901487c0 con 0x7fb390149a40 2023-08-09T13:51:03.397 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:03.391+0000 7fb38ffff640 1 --2- 172.21.15.43:0/3793457751 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb390149a40 0x7fb390149ec0 secure :-1 s=READY pgs=1121 cs=0 l=1 rev1=1 crypto rx=0x7fb3900ce8a0 tx=0x7fb38800ec00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-09T13:51:03.397 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:03.391+0000 7fb38cff9640 1 -- 172.21.15.43:0/3793457751 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb388004030 con 0x7fb390149a40 2023-08-09T13:51:03.397 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:03.391+0000 7fb38cff9640 1 -- 172.21.15.43:0/3793457751 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb388004190 con 0x7fb390149a40 2023-08-09T13:51:03.397 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:03.391+0000 7fb396214640 1 -- 172.21.15.43:0/3793457751 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb390148aa0 con 0x7fb390149a40 2023-08-09T13:51:03.397 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:03.391+0000 7fb38cff9640 1 -- 172.21.15.43:0/3793457751 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb388003610 con 0x7fb390149a40 2023-08-09T13:51:03.397 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:03.391+0000 7fb396214640 1 -- 172.21.15.43:0/3793457751 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb3900c61c0 con 0x7fb390149a40 2023-08-09T13:51:03.398 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:03.392+0000 7fb396214640 1 -- 172.21.15.43:0/3793457751 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb390148ce0 con 0x7fb390149a40 2023-08-09T13:51:03.398 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:03.393+0000 7fb38cff9640 1 -- 172.21.15.43:0/3793457751 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 37) v1 ==== 88443+0+0 (secure 0 0 0) 0x7fb388003770 con 0x7fb390149a40 2023-08-09T13:51:03.398 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:03.394+0000 7fb38cff9640 1 --2- 172.21.15.43:0/3793457751 >> [v2:172.21.15.203:6832/1778614695,v1:172.21.15.203:6833/1778614695] conn(0x7fb37006a1a0 0x7fb37006c660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:51:03.398 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:03.394+0000 7fb38cff9640 1 -- 172.21.15.43:0/3793457751 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(94..94 src has 1..94) v4 ==== 6762+0+0 (secure 0 0 0) 0x7fb388088900 con 0x7fb390149a40 2023-08-09T13:51:03.399 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:03.395+0000 7fb38f7fe640 1 --2- 172.21.15.43:0/3793457751 >> [v2:172.21.15.203:6832/1778614695,v1:172.21.15.203:6833/1778614695] conn(0x7fb37006a1a0 0x7fb37006c660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:51:03.399 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:03.396+0000 7fb38f7fe640 1 --2- 172.21.15.43:0/3793457751 >> [v2:172.21.15.203:6832/1778614695,v1:172.21.15.203:6833/1778614695] conn(0x7fb37006a1a0 0x7fb37006c660 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0x7fb380034000 tx=0x7fb380009030 comp rx=0 tx=0).ready entity=mgr.5819 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-09T13:51:03.402 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:03.399+0000 7fb38cff9640 1 -- 172.21.15.43:0/3793457751 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178066 (secure 0 0 0) 0x7fb390148ce0 con 0x7fb390149a40 2023-08-09T13:51:03.599 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:03.597+0000 7fb396214640 1 -- 172.21.15.43:0/3793457751 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7fb390148c90 con 0x7fb390149a40 2023-08-09T13:51:03.826 DEBUG:teuthology.orchestra.run.smithi043:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-08-09T13:51:03.835 DEBUG:teuthology.orchestra.run.smithi203:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-08-09T13:51:04.129 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:04.126+0000 7fb38cff9640 1 -- 172.21.15.43:0/3793457751 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data' removed v95) v1 ==== 169+0+0 (secure 0 0 0) 0x7fb388017020 con 0x7fb390149a40 2023-08-09T13:51:04.131 INFO:teuthology.orchestra.run.smithi043.stderr:pool 'cephfs_data' removed 2023-08-09T13:51:04.147 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:04.143+0000 7fb396214640 1 -- 172.21.15.43:0/3793457751 >> [v2:172.21.15.203:6832/1778614695,v1:172.21.15.203:6833/1778614695] conn(0x7fb37006a1a0 msgr2=0x7fb37006c660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:51:04.147 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:04.144+0000 7fb396214640 1 --2- 172.21.15.43:0/3793457751 >> [v2:172.21.15.203:6832/1778614695,v1:172.21.15.203:6833/1778614695] conn(0x7fb37006a1a0 0x7fb37006c660 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0x7fb380034000 tx=0x7fb380009030 comp rx=0 tx=0).stop 2023-08-09T13:51:04.147 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:04.144+0000 7fb396214640 1 -- 172.21.15.43:0/3793457751 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb390149a40 msgr2=0x7fb390149ec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:51:04.148 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:04.144+0000 7fb396214640 1 --2- 172.21.15.43:0/3793457751 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb390149a40 0x7fb390149ec0 secure :-1 s=READY pgs=1121 cs=0 l=1 rev1=1 crypto rx=0x7fb3900ce8a0 tx=0x7fb38800ec00 comp rx=0 tx=0).stop 2023-08-09T13:51:04.148 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:04.145+0000 7fb396214640 1 -- 172.21.15.43:0/3793457751 shutdown_connections 2023-08-09T13:51:04.148 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:04.145+0000 7fb396214640 1 --2- 172.21.15.43:0/3793457751 >> [v2:172.21.15.203:6832/1778614695,v1:172.21.15.203:6833/1778614695] conn(0x7fb37006a1a0 0x7fb37006c660 unknown :-1 s=CLOSED pgs=34 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:51:04.148 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:04.145+0000 7fb396214640 1 --2- 172.21.15.43:0/3793457751 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb390149a40 0x7fb390149ec0 unknown :-1 s=CLOSED pgs=1121 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:51:04.149 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:04.145+0000 7fb396214640 1 --2- 172.21.15.43:0/3793457751 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fb3901ef5d0 0x7fb390149500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:51:04.149 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:04.145+0000 7fb396214640 1 --2- 172.21.15.43:0/3793457751 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fb37c0039e0 0x7fb390148fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:51:04.149 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:04.145+0000 7fb396214640 1 -- 172.21.15.43:0/3793457751 >> 172.21.15.43:0/3793457751 conn(0x7fb39006b990 msgr2=0x7fb3901585b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-09T13:51:04.149 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:04.145+0000 7fb396214640 1 -- 172.21.15.43:0/3793457751 shutdown_connections 2023-08-09T13:51:04.149 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:04.145+0000 7fb396214640 1 -- 172.21.15.43:0/3793457751 wait complete. 2023-08-09T13:51:04.174 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config rm mgr mgr/snap_schedule/dump_on_update 2023-08-09T13:51:04.382 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:04.379+0000 7f853a6d3640 1 Processor -- start 2023-08-09T13:51:04.383 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:04.380+0000 7f853a6d3640 1 -- start start 2023-08-09T13:51:04.384 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:04.381+0000 7f853a6d3640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f85340ce1c0 0x7f85340ce5a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:51:04.384 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:04.381+0000 7f853a6d3640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f85340c1620 0x7f85340cbb80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:51:04.384 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:04.381+0000 7f853a6d3640 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7f8534160e20 0x7f85340cadd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:51:04.384 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:04.381+0000 7f853a6d3640 1 -- --> v2:172.21.15.43:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f85340cd0c0 con 0x7f85340c1620 2023-08-09T13:51:04.384 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:04.381+0000 7f853a6d3640 1 -- --> v2:172.21.15.203:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f85340ceae0 con 0x7f8534160e20 2023-08-09T13:51:04.385 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:04.381+0000 7f853a6d3640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8534161790 con 0x7f85340ce1c0 2023-08-09T13:51:04.389 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:04.382+0000 7f852bfff640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f85340c1620 0x7f85340cbb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:51:04.390 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:04.382+0000 7f852bfff640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f85340c1620 0x7f85340cbb80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:46736/0 (socket says 172.21.15.43:46736) 2023-08-09T13:51:04.390 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:04.382+0000 7f852bfff640 1 -- 172.21.15.43:0/829008501 learned_addr learned my addr 172.21.15.43:0/829008501 (peer_addr_for_me v2:172.21.15.43:0/0) 2023-08-09T13:51:04.390 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:04.382+0000 7f8538c70640 1 --2- 172.21.15.43:0/829008501 >> v2:172.21.15.203:3300/0 conn(0x7f8534160e20 0x7f85340cadd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:51:04.390 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:04.382+0000 7f8533fff640 1 --2- 172.21.15.43:0/829008501 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f85340ce1c0 0x7f85340ce5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:51:04.390 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:04.382+0000 7f852bfff640 1 -- 172.21.15.43:0/829008501 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f85340ce1c0 msgr2=0x7f85340ce5a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-09T13:51:04.390 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:04.382+0000 7f852bfff640 1 --2- 172.21.15.43:0/829008501 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f85340ce1c0 0x7f85340ce5a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:51:04.390 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:04.382+0000 7f852bfff640 1 -- 172.21.15.43:0/829008501 >> v2:172.21.15.203:3300/0 conn(0x7f8534160e20 msgr2=0x7f85340cadd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-09T13:51:04.391 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:04.382+0000 7f852bfff640 1 --2- 172.21.15.43:0/829008501 >> v2:172.21.15.203:3300/0 conn(0x7f8534160e20 0x7f85340cadd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:51:04.391 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:04.382+0000 7f852bfff640 1 -- 172.21.15.43:0/829008501 --> v2:172.21.15.43:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8534161a70 con 0x7f85340c1620 2023-08-09T13:51:04.391 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:04.382+0000 7f8533fff640 1 --2- 172.21.15.43:0/829008501 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f85340ce1c0 0x7f85340ce5a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-08-09T13:51:04.391 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:04.383+0000 7f852bfff640 1 --2- 172.21.15.43:0/829008501 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f85340c1620 0x7f85340cbb80 secure :-1 s=READY pgs=1122 cs=0 l=1 rev1=1 crypto rx=0x7f85340c0ed0 tx=0x7f851c02b870 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ec75388c55d602cc server_cookie=0 in_seq=0 out_seq=0 2023-08-09T13:51:04.391 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:04.384+0000 7f85337fe640 1 -- 172.21.15.43:0/829008501 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f851c0317d0 con 0x7f85340c1620 2023-08-09T13:51:04.391 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:04.384+0000 7f85337fe640 1 -- 172.21.15.43:0/829008501 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f851c031930 con 0x7f85340c1620 2023-08-09T13:51:04.391 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:04.384+0000 7f85337fe640 1 -- 172.21.15.43:0/829008501 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f851c031cc0 con 0x7f85340c1620 2023-08-09T13:51:04.392 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:04.384+0000 7f853a6d3640 1 -- 172.21.15.43:0/829008501 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f85340c1620 msgr2=0x7f85340cbb80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:51:04.392 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:04.384+0000 7f853a6d3640 1 --2- 172.21.15.43:0/829008501 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f85340c1620 0x7f85340cbb80 secure :-1 s=READY pgs=1122 cs=0 l=1 rev1=1 crypto rx=0x7f85340c0ed0 tx=0x7f851c02b870 comp rx=0 tx=0).stop 2023-08-09T13:51:04.392 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:04.384+0000 7f853a6d3640 1 -- 172.21.15.43:0/829008501 shutdown_connections 2023-08-09T13:51:04.392 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:04.384+0000 7f853a6d3640 1 --2- 172.21.15.43:0/829008501 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f85340c1620 0x7f85340cbb80 unknown :-1 s=CLOSED pgs=1122 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:51:04.392 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:04.384+0000 7f853a6d3640 1 --2- 172.21.15.43:0/829008501 >> v2:172.21.15.203:3300/0 conn(0x7f8534160e20 0x7f85340cadd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:51:04.392 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:04.384+0000 7f853a6d3640 1 --2- 172.21.15.43:0/829008501 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f85340ce1c0 0x7f85340ce5a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:51:04.392 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:04.384+0000 7f853a6d3640 1 -- 172.21.15.43:0/829008501 >> 172.21.15.43:0/829008501 conn(0x7f85340aff50 msgr2=0x7f85340b0360 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-09T13:51:04.393 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:04.384+0000 7f853a6d3640 1 -- 172.21.15.43:0/829008501 shutdown_connections 2023-08-09T13:51:04.393 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:04.385+0000 7f853a6d3640 1 -- 172.21.15.43:0/829008501 wait complete. 2023-08-09T13:51:04.393 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:04.386+0000 7f853a6d3640 1 Processor -- start 2023-08-09T13:51:04.393 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:04.386+0000 7f853a6d3640 1 -- start start 2023-08-09T13:51:04.395 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:04.387+0000 7f853a6d3640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f85340c1620 0x7f853414df60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:51:04.395 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:04.387+0000 7f853a6d3640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f85340ce1c0 0x7f853414c4e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:51:04.396 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:04.387+0000 7f853a6d3640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f8534160e20 0x7f853414e750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:51:04.396 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:04.387+0000 7f853a6d3640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f853414cec0 con 0x7f85340c1620 2023-08-09T13:51:04.396 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:04.387+0000 7f853a6d3640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f853414d1a0 con 0x7f85340ce1c0 2023-08-09T13:51:04.396 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:04.387+0000 7f853a6d3640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f853414d480 con 0x7f8534160e20 2023-08-09T13:51:04.396 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:04.388+0000 7f8533fff640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f85340c1620 0x7f853414df60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:51:04.396 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:04.388+0000 7f8533fff640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f85340c1620 0x7f853414df60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:46750/0 (socket says 172.21.15.43:46750) 2023-08-09T13:51:04.396 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:04.388+0000 7f8533fff640 1 -- 172.21.15.43:0/2124539686 learned_addr learned my addr 172.21.15.43:0/2124539686 (peer_addr_for_me v2:172.21.15.43:0/0) 2023-08-09T13:51:04.397 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:04.388+0000 7f8538c70640 1 --2- 172.21.15.43:0/2124539686 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f8534160e20 0x7f853414e750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:51:04.397 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:04.388+0000 7f852bfff640 1 --2- 172.21.15.43:0/2124539686 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f85340ce1c0 0x7f853414c4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:51:04.397 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:04.388+0000 7f8533fff640 1 -- 172.21.15.43:0/2124539686 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f8534160e20 msgr2=0x7f853414e750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:51:04.397 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:04.388+0000 7f8533fff640 1 --2- 172.21.15.43:0/2124539686 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f8534160e20 0x7f853414e750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:51:04.397 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:04.388+0000 7f8533fff640 1 -- 172.21.15.43:0/2124539686 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f85340ce1c0 msgr2=0x7f853414c4e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:51:04.397 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:04.388+0000 7f8533fff640 1 --2- 172.21.15.43:0/2124539686 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f85340ce1c0 0x7f853414c4e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:51:04.397 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:04.388+0000 7f8533fff640 1 -- 172.21.15.43:0/2124539686 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f851c0094d0 con 0x7f85340c1620 2023-08-09T13:51:04.398 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:04.389+0000 7f8533fff640 1 --2- 172.21.15.43:0/2124539686 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f85340c1620 0x7f853414df60 secure :-1 s=READY pgs=1123 cs=0 l=1 rev1=1 crypto rx=0x7f852400c7f0 tx=0x7f852400ccb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-09T13:51:04.398 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:04.389+0000 7f85317fa640 1 -- 172.21.15.43:0/2124539686 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8524007820 con 0x7f85340c1620 2023-08-09T13:51:04.398 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:04.389+0000 7f85317fa640 1 -- 172.21.15.43:0/2124539686 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8524007980 con 0x7f85340c1620 2023-08-09T13:51:04.398 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:04.389+0000 7f85317fa640 1 -- 172.21.15.43:0/2124539686 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8524007d30 con 0x7f85340c1620 2023-08-09T13:51:04.398 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:04.389+0000 7f853a6d3640 1 -- 172.21.15.43:0/2124539686 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f853414d760 con 0x7f85340c1620 2023-08-09T13:51:04.398 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:04.389+0000 7f853a6d3640 1 -- 172.21.15.43:0/2124539686 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f85341526d0 con 0x7f85340c1620 2023-08-09T13:51:04.398 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:04.391+0000 7f853a6d3640 1 -- 172.21.15.43:0/2124539686 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f853414dc80 con 0x7f85340c1620 2023-08-09T13:51:04.398 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:04.392+0000 7f85317fa640 1 -- 172.21.15.43:0/2124539686 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 37) v1 ==== 88443+0+0 (secure 0 0 0) 0x7f85240113f0 con 0x7f85340c1620 2023-08-09T13:51:04.399 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:04.392+0000 7f85317fa640 1 --2- 172.21.15.43:0/2124539686 >> [v2:172.21.15.203:6832/1778614695,v1:172.21.15.203:6833/1778614695] conn(0x7f851406a170 0x7f851406c630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:51:04.399 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:04.393+0000 7f85317fa640 1 -- 172.21.15.43:0/2124539686 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(95..95 src has 1..95) v4 ==== 6348+0+0 (secure 0 0 0) 0x7f8524088cd0 con 0x7f85340c1620 2023-08-09T13:51:04.399 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:04.393+0000 7f852bfff640 1 --2- 172.21.15.43:0/2124539686 >> [v2:172.21.15.203:6832/1778614695,v1:172.21.15.203:6833/1778614695] conn(0x7f851406a170 0x7f851406c630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:51:04.400 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:04.394+0000 7f852bfff640 1 --2- 172.21.15.43:0/2124539686 >> [v2:172.21.15.203:6832/1778614695,v1:172.21.15.203:6833/1778614695] conn(0x7f851406a170 0x7f851406c630 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7f851c036000 tx=0x7f851c03d040 comp rx=0 tx=0).ready entity=mgr.5819 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-09T13:51:04.400 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:04.397+0000 7f85317fa640 1 -- 172.21.15.43:0/2124539686 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178066 (secure 0 0 0) 0x7f8524053f60 con 0x7f85340c1620 2023-08-09T13:51:04.604 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:04.601+0000 7f853a6d3640 1 -- 172.21.15.43:0/2124539686 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "config rm", "who": "mgr", "name": "mgr/snap_schedule/dump_on_update"} v 0) v1 -- 0x7f85341528c0 con 0x7f85340c1620 2023-08-09T13:51:04.612 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:04.610+0000 7f85317fa640 1 -- 172.21.15.43:0/2124539686 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "config rm", "who": "mgr", "name": "mgr/snap_schedule/dump_on_update"}]=0 v25) v1 ==== 115+0+0 (secure 0 0 0) 0x7f8524057580 con 0x7f85340c1620 2023-08-09T13:51:04.615 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:04.612+0000 7f853a6d3640 1 -- 172.21.15.43:0/2124539686 >> [v2:172.21.15.203:6832/1778614695,v1:172.21.15.203:6833/1778614695] conn(0x7f851406a170 msgr2=0x7f851406c630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:51:04.615 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:04.612+0000 7f853a6d3640 1 --2- 172.21.15.43:0/2124539686 >> [v2:172.21.15.203:6832/1778614695,v1:172.21.15.203:6833/1778614695] conn(0x7f851406a170 0x7f851406c630 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7f851c036000 tx=0x7f851c03d040 comp rx=0 tx=0).stop 2023-08-09T13:51:04.615 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:04.613+0000 7f853a6d3640 1 -- 172.21.15.43:0/2124539686 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f85340c1620 msgr2=0x7f853414df60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:51:04.616 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:04.613+0000 7f853a6d3640 1 --2- 172.21.15.43:0/2124539686 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f85340c1620 0x7f853414df60 secure :-1 s=READY pgs=1123 cs=0 l=1 rev1=1 crypto rx=0x7f852400c7f0 tx=0x7f852400ccb0 comp rx=0 tx=0).stop 2023-08-09T13:51:04.616 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:04.613+0000 7f853a6d3640 1 -- 172.21.15.43:0/2124539686 shutdown_connections 2023-08-09T13:51:04.616 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:04.613+0000 7f853a6d3640 1 --2- 172.21.15.43:0/2124539686 >> [v2:172.21.15.203:6832/1778614695,v1:172.21.15.203:6833/1778614695] conn(0x7f851406a170 0x7f851406c630 unknown :-1 s=CLOSED pgs=35 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:51:04.616 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:04.613+0000 7f853a6d3640 1 --2- 172.21.15.43:0/2124539686 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f8534160e20 0x7f853414e750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:51:04.616 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:04.613+0000 7f853a6d3640 1 --2- 172.21.15.43:0/2124539686 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f85340ce1c0 0x7f853414c4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:51:04.616 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:04.613+0000 7f853a6d3640 1 --2- 172.21.15.43:0/2124539686 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f85340c1620 0x7f853414df60 unknown :-1 s=CLOSED pgs=1123 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:51:04.616 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:04.613+0000 7f853a6d3640 1 -- 172.21.15.43:0/2124539686 >> 172.21.15.43:0/2124539686 conn(0x7f85340aff50 msgr2=0x7f85340b0330 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-09T13:51:04.617 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:04.613+0000 7f853a6d3640 1 -- 172.21.15.43:0/2124539686 shutdown_connections 2023-08-09T13:51:04.617 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:04.613+0000 7f853a6d3640 1 -- 172.21.15.43:0/2124539686 wait complete. 2023-08-09T13:51:04.634 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config rm mgr mgr/snap_schedule/allow_m_granularity 2023-08-09T13:51:04.862 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:04.860+0000 7f98edb64640 1 Processor -- start 2023-08-09T13:51:04.862 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:04.860+0000 7f98edb64640 1 -- start start 2023-08-09T13:51:04.864 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:04.861+0000 7f98edb64640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f98e8143730 0x7f98e80cc4f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:51:04.864 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:04.861+0000 7f98edb64640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f98e8071840 0x7f98e80697c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:51:04.865 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:04.861+0000 7f98edb64640 1 -- --> v1:172.21.15.203:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f98e815de90 con 0x7f98e8071460 2023-08-09T13:51:04.865 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:04.862+0000 7f98edb64640 1 -- --> v2:172.21.15.43:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f98e8070d40 con 0x7f98e8143730 2023-08-09T13:51:04.865 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:04.862+0000 7f98edb64640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f98e815a740 con 0x7f98e8071840 2023-08-09T13:51:04.865 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:04.862+0000 7f98deffd640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f98e8143730 0x7f98e80cc4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:51:04.865 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:04.862+0000 7f98deffd640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f98e8143730 0x7f98e80cc4f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:46760/0 (socket says 172.21.15.43:46760) 2023-08-09T13:51:04.865 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:04.862+0000 7f98deffd640 1 -- 172.21.15.43:0/4134584181 learned_addr learned my addr 172.21.15.43:0/4134584181 (peer_addr_for_me v2:172.21.15.43:0/0) 2023-08-09T13:51:04.866 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:04.862+0000 7f98e7fff640 1 --2- 172.21.15.43:0/4134584181 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f98e8071840 0x7f98e80697c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:51:04.866 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:04.862+0000 7f98deffd640 1 -- 172.21.15.43:0/4134584181 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f98e8071840 msgr2=0x7f98e80697c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-09T13:51:04.866 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:04.862+0000 7f98deffd640 1 --2- 172.21.15.43:0/4134584181 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f98e8071840 0x7f98e80697c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:51:04.866 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:04.863+0000 7f98deffd640 1 -- 172.21.15.43:0/4134584181 >> v1:172.21.15.203:6789/0 conn(0x7f98e8071460 legacy=0x7f98e8070630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-09T13:51:04.867 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:04.863+0000 7f98deffd640 1 -- 172.21.15.43:0/4134584181 --> v2:172.21.15.43:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f98e80cb680 con 0x7f98e8143730 2023-08-09T13:51:04.867 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:04.864+0000 7f98deffd640 1 --2- 172.21.15.43:0/4134584181 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f98e8143730 0x7f98e80cc4f0 secure :-1 s=READY pgs=1124 cs=0 l=1 rev1=1 crypto rx=0x7f98d0009850 tx=0x7f98d002b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e6ede99bff541a1b server_cookie=0 in_seq=0 out_seq=0 2023-08-09T13:51:04.868 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:04.864+0000 7f98e6ffd640 1 -- 172.21.15.43:0/4134584181 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f98d0004230 con 0x7f98e8143730 2023-08-09T13:51:04.868 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:04.864+0000 7f98e6ffd640 1 -- 172.21.15.43:0/4134584181 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f98d0004390 con 0x7f98e8143730 2023-08-09T13:51:04.868 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:04.864+0000 7f98e6ffd640 1 -- 172.21.15.43:0/4134584181 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f98d0031830 con 0x7f98e8143730 2023-08-09T13:51:04.868 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:04.865+0000 7f98edb64640 1 -- 172.21.15.43:0/4134584181 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f98e8143730 msgr2=0x7f98e80cc4f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:51:04.868 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:04.865+0000 7f98edb64640 1 --2- 172.21.15.43:0/4134584181 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f98e8143730 0x7f98e80cc4f0 secure :-1 s=READY pgs=1124 cs=0 l=1 rev1=1 crypto rx=0x7f98d0009850 tx=0x7f98d002b8d0 comp rx=0 tx=0).stop 2023-08-09T13:51:04.868 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:04.865+0000 7f98edb64640 1 -- 172.21.15.43:0/4134584181 shutdown_connections 2023-08-09T13:51:04.868 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:04.865+0000 7f98edb64640 1 --2- 172.21.15.43:0/4134584181 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f98e8071840 0x7f98e80697c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:51:04.869 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:04.865+0000 7f98edb64640 1 --2- 172.21.15.43:0/4134584181 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f98e8143730 0x7f98e80cc4f0 unknown :-1 s=CLOSED pgs=1124 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:51:04.869 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:04.865+0000 7f98edb64640 1 -- 172.21.15.43:0/4134584181 >> 172.21.15.43:0/4134584181 conn(0x7f98e80b0cc0 msgr2=0x7f98e80b10d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-09T13:51:04.869 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:04.865+0000 7f98edb64640 1 -- 172.21.15.43:0/4134584181 shutdown_connections 2023-08-09T13:51:04.869 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:04.865+0000 7f98edb64640 1 -- 172.21.15.43:0/4134584181 wait complete. 2023-08-09T13:51:04.869 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:04.867+0000 7f98edb64640 1 Processor -- start 2023-08-09T13:51:04.870 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:04.867+0000 7f98edb64640 1 -- start start 2023-08-09T13:51:04.870 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:04.868+0000 7f98edb64640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f98e8071460 0x7f98e80c6bb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:51:04.870 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:04.868+0000 7f98edb64640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f98e8071840 0x7f98e815cae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:51:04.870 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:04.868+0000 7f98edb64640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f98e8143730 0x7f98e815d020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:51:04.871 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:04.868+0000 7f98edb64640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f98e80c70f0 con 0x7f98e8143730 2023-08-09T13:51:04.871 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:04.868+0000 7f98edb64640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f98e80c5b40 con 0x7f98e8071840 2023-08-09T13:51:04.871 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:04.868+0000 7f98edb64640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f98e80c3400 con 0x7f98e8071460 2023-08-09T13:51:04.872 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:04.868+0000 7f98e7fff640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f98e8143730 0x7f98e815d020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:51:04.872 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:04.868+0000 7f98e7fff640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f98e8143730 0x7f98e815d020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:46766/0 (socket says 172.21.15.43:46766) 2023-08-09T13:51:04.872 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:04.868+0000 7f98e7fff640 1 -- 172.21.15.43:0/2490498446 learned_addr learned my addr 172.21.15.43:0/2490498446 (peer_addr_for_me v2:172.21.15.43:0/0) 2023-08-09T13:51:04.873 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:04.869+0000 7f98deffd640 1 --2- 172.21.15.43:0/2490498446 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f98e8071840 0x7f98e815cae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:51:04.873 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:04.869+0000 7f98e77fe640 1 --2- 172.21.15.43:0/2490498446 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f98e8071460 0x7f98e80c6bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:51:04.873 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:04.869+0000 7f98e7fff640 1 -- 172.21.15.43:0/2490498446 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f98e8071460 msgr2=0x7f98e80c6bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:51:04.873 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:04.869+0000 7f98e7fff640 1 --2- 172.21.15.43:0/2490498446 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f98e8071460 0x7f98e80c6bb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:51:04.873 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:04.869+0000 7f98e7fff640 1 -- 172.21.15.43:0/2490498446 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f98e8071840 msgr2=0x7f98e815cae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:51:04.873 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:04.869+0000 7f98e7fff640 1 --2- 172.21.15.43:0/2490498446 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f98e8071840 0x7f98e815cae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:51:04.873 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:04.869+0000 7f98e7fff640 1 -- 172.21.15.43:0/2490498446 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f98d0009460 con 0x7f98e8143730 2023-08-09T13:51:04.874 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:04.869+0000 7f98e77fe640 1 --2- 172.21.15.43:0/2490498446 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f98e8071460 0x7f98e80c6bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-08-09T13:51:04.874 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:04.869+0000 7f98deffd640 1 --2- 172.21.15.43:0/2490498446 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f98e8071840 0x7f98e815cae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-08-09T13:51:04.874 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:04.869+0000 7f98e7fff640 1 --2- 172.21.15.43:0/2490498446 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f98e8143730 0x7f98e815d020 secure :-1 s=READY pgs=1125 cs=0 l=1 rev1=1 crypto rx=0x7f98e000d800 tx=0x7f98e000dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-09T13:51:04.874 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:04.869+0000 7f98e4ff9640 1 -- 172.21.15.43:0/2490498446 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f98e000b6c0 con 0x7f98e8143730 2023-08-09T13:51:04.874 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:04.869+0000 7f98e4ff9640 1 -- 172.21.15.43:0/2490498446 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f98e000b840 con 0x7f98e8143730 2023-08-09T13:51:04.874 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:04.869+0000 7f98edb64640 1 -- 172.21.15.43:0/2490498446 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f98e80c36e0 con 0x7f98e8143730 2023-08-09T13:51:04.874 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:04.869+0000 7f98e4ff9640 1 -- 172.21.15.43:0/2490498446 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f98e000bbf0 con 0x7f98e8143730 2023-08-09T13:51:04.875 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:04.870+0000 7f98edb64640 1 -- 172.21.15.43:0/2490498446 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f98e80c3ca0 con 0x7f98e8143730 2023-08-09T13:51:04.875 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:04.871+0000 7f98edb64640 1 -- 172.21.15.43:0/2490498446 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f98e80c43a0 con 0x7f98e8143730 2023-08-09T13:51:04.875 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:04.872+0000 7f98e4ff9640 1 -- 172.21.15.43:0/2490498446 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 37) v1 ==== 88443+0+0 (secure 0 0 0) 0x7f98e000bd50 con 0x7f98e8143730 2023-08-09T13:51:04.875 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:04.872+0000 7f98e4ff9640 1 --2- 172.21.15.43:0/2490498446 >> [v2:172.21.15.203:6832/1778614695,v1:172.21.15.203:6833/1778614695] conn(0x7f98c806a170 0x7f98c806c630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:51:04.876 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:04.873+0000 7f98e4ff9640 1 -- 172.21.15.43:0/2490498446 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(95..95 src has 1..95) v4 ==== 6348+0+0 (secure 0 0 0) 0x7f98e0088e50 con 0x7f98e8143730 2023-08-09T13:51:04.876 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:04.873+0000 7f98e77fe640 1 --2- 172.21.15.43:0/2490498446 >> [v2:172.21.15.203:6832/1778614695,v1:172.21.15.203:6833/1778614695] conn(0x7f98c806a170 0x7f98c806c630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:51:04.876 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:04.874+0000 7f98e77fe640 1 --2- 172.21.15.43:0/2490498446 >> [v2:172.21.15.203:6832/1778614695,v1:172.21.15.203:6833/1778614695] conn(0x7f98c806a170 0x7f98c806c630 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7f98e80cebd0 tx=0x7f98d8009040 comp rx=0 tx=0).ready entity=mgr.5819 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-09T13:51:04.879 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:04.877+0000 7f98e4ff9640 1 -- 172.21.15.43:0/2490498446 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178066 (secure 0 0 0) 0x7f98e0054270 con 0x7f98e8143730 2023-08-09T13:51:05.070 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:05.067+0000 7f98edb64640 1 -- 172.21.15.43:0/2490498446 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "config rm", "who": "mgr", "name": "mgr/snap_schedule/allow_m_granularity"} v 0) v1 -- 0x7f98e80c41c0 con 0x7f98e8143730 2023-08-09T13:51:05.077 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:05.075+0000 7f98e4ff9640 1 -- 172.21.15.43:0/2490498446 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "config rm", "who": "mgr", "name": "mgr/snap_schedule/allow_m_granularity"}]=0 v26) v1 ==== 120+0+0 (secure 0 0 0) 0x7f98e80c41c0 con 0x7f98e8143730 2023-08-09T13:51:05.080 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:05.078+0000 7f98edb64640 1 -- 172.21.15.43:0/2490498446 >> [v2:172.21.15.203:6832/1778614695,v1:172.21.15.203:6833/1778614695] conn(0x7f98c806a170 msgr2=0x7f98c806c630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:51:05.081 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:05.078+0000 7f98edb64640 1 --2- 172.21.15.43:0/2490498446 >> [v2:172.21.15.203:6832/1778614695,v1:172.21.15.203:6833/1778614695] conn(0x7f98c806a170 0x7f98c806c630 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7f98e80cebd0 tx=0x7f98d8009040 comp rx=0 tx=0).stop 2023-08-09T13:51:05.081 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:05.078+0000 7f98edb64640 1 -- 172.21.15.43:0/2490498446 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f98e8143730 msgr2=0x7f98e815d020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:51:05.081 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:05.078+0000 7f98edb64640 1 --2- 172.21.15.43:0/2490498446 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f98e8143730 0x7f98e815d020 secure :-1 s=READY pgs=1125 cs=0 l=1 rev1=1 crypto rx=0x7f98e000d800 tx=0x7f98e000dcc0 comp rx=0 tx=0).stop 2023-08-09T13:51:05.081 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:05.078+0000 7f98edb64640 1 -- 172.21.15.43:0/2490498446 shutdown_connections 2023-08-09T13:51:05.081 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:05.078+0000 7f98edb64640 1 --2- 172.21.15.43:0/2490498446 >> [v2:172.21.15.203:6832/1778614695,v1:172.21.15.203:6833/1778614695] conn(0x7f98c806a170 0x7f98c806c630 unknown :-1 s=CLOSED pgs=36 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:51:05.081 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:05.078+0000 7f98edb64640 1 --2- 172.21.15.43:0/2490498446 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f98e8143730 0x7f98e815d020 unknown :-1 s=CLOSED pgs=1125 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:51:05.082 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:05.078+0000 7f98edb64640 1 --2- 172.21.15.43:0/2490498446 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f98e8071840 0x7f98e815cae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:51:05.082 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:05.078+0000 7f98edb64640 1 --2- 172.21.15.43:0/2490498446 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f98e8071460 0x7f98e80c6bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:51:05.082 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:05.078+0000 7f98edb64640 1 -- 172.21.15.43:0/2490498446 >> 172.21.15.43:0/2490498446 conn(0x7f98e80b0cc0 msgr2=0x7f98e80ae010 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-09T13:51:05.082 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:05.078+0000 7f98edb64640 1 -- 172.21.15.43:0/2490498446 shutdown_connections 2023-08-09T13:51:05.082 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:05.078+0000 7f98edb64640 1 -- 172.21.15.43:0/2490498446 wait complete. 2023-08-09T13:51:05.104 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config rm mon mon_allow_pool_delete 2023-08-09T13:51:05.320 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:05.318+0000 7f1085e38640 1 Processor -- start 2023-08-09T13:51:05.321 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:05.318+0000 7f1085e38640 1 -- start start 2023-08-09T13:51:05.322 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:05.320+0000 7f1085e38640 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7f108015df60 0x7f10800ce1c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:51:05.323 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:05.320+0000 7f1085e38640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f108015e3d0 0x7f10800ce700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:51:05.323 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:05.320+0000 7f1085e38640 1 -- --> v1:172.21.15.43:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f108015aef0 con 0x7f10800cb2d0 2023-08-09T13:51:05.323 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:05.320+0000 7f1085e38640 1 -- --> v2:172.21.15.43:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f108015a970 con 0x7f108015e3d0 2023-08-09T13:51:05.323 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:05.320+0000 7f1085e38640 1 -- --> v2:172.21.15.203:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f108015e940 con 0x7f108015df60 2023-08-09T13:51:05.324 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:05.320+0000 7f107f7fe640 1 --1- >> v1:172.21.15.43:6789/0 conn(0x7f10800cb2d0 0x7f108015b380 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.43:6789/0 says I am v1:172.21.15.43:45652/0 (socket says 172.21.15.43:45652) 2023-08-09T13:51:05.324 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:05.320+0000 7f107f7fe640 1 -- 172.21.15.43:0/3322856906 learned_addr learned my addr 172.21.15.43:0/3322856906 (peer_addr_for_me v1:172.21.15.43:0/0) 2023-08-09T13:51:05.324 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:05.320+0000 7f107ffff640 1 --2- 172.21.15.43:0/3322856906 >> v2:172.21.15.43:3300/0 conn(0x7f108015e3d0 0x7f10800ce700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:51:05.325 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:05.321+0000 7f107effd640 1 --2- 172.21.15.43:0/3322856906 >> v2:172.21.15.203:3300/0 conn(0x7f108015df60 0x7f10800ce1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:51:05.325 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:05.321+0000 7f107e7fc640 1 -- 172.21.15.43:0/3322856906 <== mon.0 v1:172.21.15.43:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3097058043 0 0) 0x7f108015aef0 con 0x7f10800cb2d0 2023-08-09T13:51:05.325 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:05.321+0000 7f107e7fc640 1 -- 172.21.15.43:0/3322856906 --> v1:172.21.15.43:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f1068003520 con 0x7f10800cb2d0 2023-08-09T13:51:05.326 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:05.321+0000 7f107ffff640 1 -- 172.21.15.43:0/3322856906 >> v2:172.21.15.203:3300/0 conn(0x7f108015df60 msgr2=0x7f10800ce1c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-09T13:51:05.326 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:05.321+0000 7f107ffff640 1 --2- 172.21.15.43:0/3322856906 >> v2:172.21.15.203:3300/0 conn(0x7f108015df60 0x7f10800ce1c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:51:05.326 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:05.321+0000 7f107ffff640 1 -- 172.21.15.43:0/3322856906 >> v1:172.21.15.43:6789/0 conn(0x7f10800cb2d0 legacy=0x7f108015b380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:51:05.326 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:05.321+0000 7f107ffff640 1 -- 172.21.15.43:0/3322856906 --> v2:172.21.15.43:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f108015ec20 con 0x7f108015e3d0 2023-08-09T13:51:05.326 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:05.322+0000 7f107ffff640 1 --2- 172.21.15.43:0/3322856906 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f108015e3d0 0x7f10800ce700 secure :-1 s=READY pgs=1127 cs=0 l=1 rev1=1 crypto rx=0x7f10800cf0b0 tx=0x7f107802be40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8985a9a4a6748fa8 server_cookie=0 in_seq=0 out_seq=0 2023-08-09T13:51:05.326 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:05.322+0000 7f107e7fc640 1 -- 172.21.15.43:0/3322856906 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f107802cbf0 con 0x7f108015e3d0 2023-08-09T13:51:05.326 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:05.322+0000 7f107e7fc640 1 -- 172.21.15.43:0/3322856906 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f107802cd50 con 0x7f108015e3d0 2023-08-09T13:51:05.326 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:05.322+0000 7f107e7fc640 1 -- 172.21.15.43:0/3322856906 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1078034640 con 0x7f108015e3d0 2023-08-09T13:51:05.327 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:05.323+0000 7f1085e38640 1 -- 172.21.15.43:0/3322856906 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f108015e3d0 msgr2=0x7f10800ce700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:51:05.327 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:05.323+0000 7f1085e38640 1 --2- 172.21.15.43:0/3322856906 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f108015e3d0 0x7f10800ce700 secure :-1 s=READY pgs=1127 cs=0 l=1 rev1=1 crypto rx=0x7f10800cf0b0 tx=0x7f107802be40 comp rx=0 tx=0).stop 2023-08-09T13:51:05.327 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:05.323+0000 7f1085e38640 1 -- 172.21.15.43:0/3322856906 shutdown_connections 2023-08-09T13:51:05.328 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:05.323+0000 7f1085e38640 1 --2- 172.21.15.43:0/3322856906 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f108015e3d0 0x7f10800ce700 unknown :-1 s=CLOSED pgs=1127 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:51:05.328 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:05.323+0000 7f1085e38640 1 --2- 172.21.15.43:0/3322856906 >> v2:172.21.15.203:3300/0 conn(0x7f108015df60 0x7f10800ce1c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:51:05.328 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:05.323+0000 7f1085e38640 1 -- 172.21.15.43:0/3322856906 >> 172.21.15.43:0/3322856906 conn(0x7f10800aff50 msgr2=0x7f10800b0360 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-09T13:51:05.328 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:05.323+0000 7f1085e38640 1 -- 172.21.15.43:0/3322856906 shutdown_connections 2023-08-09T13:51:05.328 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:05.323+0000 7f1085e38640 1 -- 172.21.15.43:0/3322856906 wait complete. 2023-08-09T13:51:05.329 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:05.325+0000 7f1085e38640 1 Processor -- start 2023-08-09T13:51:05.329 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:05.325+0000 7f1085e38640 1 -- start start 2023-08-09T13:51:05.330 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:05.326+0000 7f1085e38640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f10800cb2d0 0x7f10801f26e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:51:05.331 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:05.326+0000 7f1085e38640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f108015df60 0x7f10801f2c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:51:05.331 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:05.326+0000 7f1085e38640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f108015e3d0 0x7f1080153730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:51:05.331 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:05.326+0000 7f1085e38640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f10800c5b80 con 0x7f10800cb2d0 2023-08-09T13:51:05.331 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:05.326+0000 7f1085e38640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f10800b2cc0 con 0x7f108015e3d0 2023-08-09T13:51:05.332 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:05.326+0000 7f1085e38640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f10800c5220 con 0x7f108015df60 2023-08-09T13:51:05.332 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:05.327+0000 7f107f7fe640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f10800cb2d0 0x7f10801f26e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:51:05.332 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:05.327+0000 7f107f7fe640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f10800cb2d0 0x7f10801f26e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:36482/0 (socket says 172.21.15.43:36482) 2023-08-09T13:51:05.333 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:05.327+0000 7f107f7fe640 1 -- 172.21.15.43:0/3767679947 learned_addr learned my addr 172.21.15.43:0/3767679947 (peer_addr_for_me v2:172.21.15.43:0/0) 2023-08-09T13:51:05.333 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:05.327+0000 7f107effd640 1 --2- 172.21.15.43:0/3767679947 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f108015df60 0x7f10801f2c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:51:05.333 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:05.327+0000 7f107f7fe640 1 -- 172.21.15.43:0/3767679947 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f108015df60 msgr2=0x7f10801f2c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:51:05.333 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:05.327+0000 7f107f7fe640 1 --2- 172.21.15.43:0/3767679947 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f108015df60 0x7f10801f2c70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:51:05.334 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:05.327+0000 7f107f7fe640 1 -- 172.21.15.43:0/3767679947 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f108015e3d0 msgr2=0x7f1080153730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:51:05.334 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:05.327+0000 7f107ffff640 1 --2- 172.21.15.43:0/3767679947 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f108015e3d0 0x7f1080153730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:51:05.334 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:05.327+0000 7f107f7fe640 1 --2- 172.21.15.43:0/3767679947 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f108015e3d0 0x7f1080153730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:51:05.334 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:05.327+0000 7f107f7fe640 1 -- 172.21.15.43:0/3767679947 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1078009a70 con 0x7f10800cb2d0 2023-08-09T13:51:05.335 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:05.327+0000 7f107f7fe640 1 --2- 172.21.15.43:0/3767679947 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f10800cb2d0 0x7f10801f26e0 secure :-1 s=READY pgs=1128 cs=0 l=1 rev1=1 crypto rx=0x7f107000c460 tx=0x7f107000c920 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-09T13:51:05.335 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:05.328+0000 7f1067fff640 1 -- 172.21.15.43:0/3767679947 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1070005ce0 con 0x7f10800cb2d0 2023-08-09T13:51:05.335 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:05.328+0000 7f1067fff640 1 -- 172.21.15.43:0/3767679947 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1070005e40 con 0x7f10800cb2d0 2023-08-09T13:51:05.336 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:05.328+0000 7f1067fff640 1 -- 172.21.15.43:0/3767679947 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1070007290 con 0x7f10800cb2d0 2023-08-09T13:51:05.336 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:05.328+0000 7f1085e38640 1 -- 172.21.15.43:0/3767679947 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f10800c54c0 con 0x7f10800cb2d0 2023-08-09T13:51:05.336 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:05.328+0000 7f1085e38640 1 -- 172.21.15.43:0/3767679947 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f10800b2470 con 0x7f10800cb2d0 2023-08-09T13:51:05.338 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:05.329+0000 7f1085e38640 1 -- 172.21.15.43:0/3767679947 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f10800739f0 con 0x7f10800cb2d0 2023-08-09T13:51:05.338 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:05.330+0000 7f1067fff640 1 -- 172.21.15.43:0/3767679947 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 37) v1 ==== 88443+0+0 (secure 0 0 0) 0x7f10700073f0 con 0x7f10800cb2d0 2023-08-09T13:51:05.339 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:05.331+0000 7f1067fff640 1 --2- 172.21.15.43:0/3767679947 >> [v2:172.21.15.203:6832/1778614695,v1:172.21.15.203:6833/1778614695] conn(0x7f106c06a1a0 0x7f106c06c660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:51:05.339 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:05.331+0000 7f1067fff640 1 -- 172.21.15.43:0/3767679947 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(95..95 src has 1..95) v4 ==== 6348+0+0 (secure 0 0 0) 0x7f1070087e00 con 0x7f10800cb2d0 2023-08-09T13:51:05.339 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:05.332+0000 7f107effd640 1 --2- 172.21.15.43:0/3767679947 >> [v2:172.21.15.203:6832/1778614695,v1:172.21.15.203:6833/1778614695] conn(0x7f106c06a1a0 0x7f106c06c660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:51:05.339 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:05.333+0000 7f107effd640 1 --2- 172.21.15.43:0/3767679947 >> [v2:172.21.15.203:6832/1778614695,v1:172.21.15.203:6833/1778614695] conn(0x7f106c06a1a0 0x7f106c06c660 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7f10801f2c20 tx=0x7f1074008040 comp rx=0 tx=0).ready entity=mgr.5819 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-09T13:51:05.340 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:05.335+0000 7f1067fff640 1 -- 172.21.15.43:0/3767679947 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178066 (secure 0 0 0) 0x7f1070053220 con 0x7f10800cb2d0 2023-08-09T13:51:05.534 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:05.532+0000 7f1085e38640 1 -- 172.21.15.43:0/3767679947 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"} v 0) v1 -- 0x7f10800cbb80 con 0x7f10800cb2d0 2023-08-09T13:51:05.540 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:05.538+0000 7f1067fff640 1 -- 172.21.15.43:0/3767679947 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"}]=0 v27) v1 ==== 104+0+0 (secure 0 0 0) 0x7f10700170a0 con 0x7f10800cb2d0 2023-08-09T13:51:05.543 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:05.540+0000 7f1085e38640 1 -- 172.21.15.43:0/3767679947 >> [v2:172.21.15.203:6832/1778614695,v1:172.21.15.203:6833/1778614695] conn(0x7f106c06a1a0 msgr2=0x7f106c06c660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:51:05.544 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:05.540+0000 7f1085e38640 1 --2- 172.21.15.43:0/3767679947 >> [v2:172.21.15.203:6832/1778614695,v1:172.21.15.203:6833/1778614695] conn(0x7f106c06a1a0 0x7f106c06c660 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7f10801f2c20 tx=0x7f1074008040 comp rx=0 tx=0).stop 2023-08-09T13:51:05.544 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:05.541+0000 7f1085e38640 1 -- 172.21.15.43:0/3767679947 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f10800cb2d0 msgr2=0x7f10801f26e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:51:05.544 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:05.541+0000 7f1085e38640 1 --2- 172.21.15.43:0/3767679947 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f10800cb2d0 0x7f10801f26e0 secure :-1 s=READY pgs=1128 cs=0 l=1 rev1=1 crypto rx=0x7f107000c460 tx=0x7f107000c920 comp rx=0 tx=0).stop 2023-08-09T13:51:05.545 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:05.541+0000 7f1085e38640 1 -- 172.21.15.43:0/3767679947 shutdown_connections 2023-08-09T13:51:05.546 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:05.541+0000 7f1085e38640 1 --2- 172.21.15.43:0/3767679947 >> [v2:172.21.15.203:6832/1778614695,v1:172.21.15.203:6833/1778614695] conn(0x7f106c06a1a0 0x7f106c06c660 unknown :-1 s=CLOSED pgs=37 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:51:05.546 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:05.541+0000 7f1085e38640 1 --2- 172.21.15.43:0/3767679947 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f108015e3d0 0x7f1080153730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:51:05.546 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:05.541+0000 7f1085e38640 1 --2- 172.21.15.43:0/3767679947 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f108015df60 0x7f10801f2c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:51:05.547 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:05.541+0000 7f1085e38640 1 --2- 172.21.15.43:0/3767679947 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f10800cb2d0 0x7f10801f26e0 unknown :-1 s=CLOSED pgs=1128 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:51:05.547 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:05.541+0000 7f1085e38640 1 -- 172.21.15.43:0/3767679947 >> 172.21.15.43:0/3767679947 conn(0x7f10800aff50 msgr2=0x7f1080154540 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-09T13:51:05.547 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:05.541+0000 7f1085e38640 1 -- 172.21.15.43:0/3767679947 shutdown_connections 2023-08-09T13:51:05.548 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:05.541+0000 7f1085e38640 1 -- 172.21.15.43:0/3767679947 wait complete. 2023-08-09T13:51:05.562 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Ended test tasks.cephfs.test_snap_schedules.TestSnapSchedules.test_non_existent_schedule' 2023-08-09T13:51:05.792 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:05.789+0000 7f5b810ad640 1 Processor -- start 2023-08-09T13:51:05.792 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:05.790+0000 7f5b810ad640 1 -- start start 2023-08-09T13:51:05.793 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:05.791+0000 7f5b810ad640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f5b7c143c20 0x7f5b7c06f540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:51:05.793 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:05.791+0000 7f5b810ad640 1 -- --> v1:172.21.15.43:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5b7c06c440 con 0x7f5b7c0c86f0 2023-08-09T13:51:05.794 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:05.791+0000 7f5b810ad640 1 -- --> v1:172.21.15.203:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5b7c06b830 con 0x7f5b7c06cfe0 2023-08-09T13:51:05.794 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:05.791+0000 7f5b810ad640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5b7c144000 con 0x7f5b7c143c20 2023-08-09T13:51:05.795 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:05.791+0000 7f5b7a59c640 1 --1- >> v1:172.21.15.43:6789/0 conn(0x7f5b7c0c86f0 0x7f5b7c06c8d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.43:6789/0 says I am v1:172.21.15.43:45666/0 (socket says 172.21.15.43:45666) 2023-08-09T13:51:05.795 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:05.791+0000 7f5b7a59c640 1 -- 172.21.15.43:0/1564757696 learned_addr learned my addr 172.21.15.43:0/1564757696 (peer_addr_for_me v1:172.21.15.43:0/0) 2023-08-09T13:51:05.795 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:05.792+0000 7f5b7ad9d640 1 --2- 172.21.15.43:0/1564757696 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f5b7c143c20 0x7f5b7c06f540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:51:05.796 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:05.792+0000 7f5b79d9b640 1 -- 172.21.15.43:0/1564757696 <== mon.0 v1:172.21.15.43:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1138306871 0 0) 0x7f5b7c06c440 con 0x7f5b7c0c86f0 2023-08-09T13:51:05.796 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:05.792+0000 7f5b79d9b640 1 -- 172.21.15.43:0/1564757696 --> v1:172.21.15.43:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f5b64003520 con 0x7f5b7c0c86f0 2023-08-09T13:51:05.796 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:05.792+0000 7f5b79d9b640 1 -- 172.21.15.43:0/1564757696 <== mon.0 v1:172.21.15.43:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3040567539 0 0) 0x7f5b64003520 con 0x7f5b7c0c86f0 2023-08-09T13:51:05.796 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:05.792+0000 7f5b79d9b640 1 -- 172.21.15.43:0/1564757696 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f5b7c143c20 msgr2=0x7f5b7c06f540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-09T13:51:05.796 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:05.792+0000 7f5b79d9b640 1 --2- 172.21.15.43:0/1564757696 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f5b7c143c20 0x7f5b7c06f540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:51:05.796 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:05.792+0000 7f5b79d9b640 1 -- 172.21.15.43:0/1564757696 >> v1:172.21.15.203:6789/0 conn(0x7f5b7c06cfe0 legacy=0x7f5b7c0ccc00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:51:05.796 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:05.792+0000 7f5b79d9b640 1 -- 172.21.15.43:0/1564757696 --> v1:172.21.15.43:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5b7c06bb10 con 0x7f5b7c0c86f0 2023-08-09T13:51:05.797 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:05.792+0000 7f5b79d9b640 1 -- 172.21.15.43:0/1564757696 <== mon.0 v1:172.21.15.43:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1285150740 0 0) 0x7f5b70001670 con 0x7f5b7c0c86f0 2023-08-09T13:51:05.797 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:05.792+0000 7f5b79d9b640 1 -- 172.21.15.43:0/1564757696 >> v1:172.21.15.43:6789/0 conn(0x7f5b7c0c86f0 legacy=0x7f5b7c06c8d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:51:05.797 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:05.793+0000 7f5b79d9b640 1 --2- 172.21.15.43:0/1564757696 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f5b7c143c20 0x7f5b64003a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:51:05.797 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:05.793+0000 7f5b79d9b640 1 --2- 172.21.15.43:0/1564757696 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f5b64003f90 0x7f5b640047c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:51:05.798 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:05.793+0000 7f5b79d9b640 1 --2- 172.21.15.43:0/1564757696 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f5b64004d00 0x7f5b64005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:51:05.798 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:05.793+0000 7f5b79d9b640 1 -- 172.21.15.43:0/1564757696 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5b7c144000 con 0x7f5b64004d00 2023-08-09T13:51:05.798 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:05.793+0000 7f5b79d9b640 1 -- 172.21.15.43:0/1564757696 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5b6400bb70 con 0x7f5b64003f90 2023-08-09T13:51:05.798 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:05.793+0000 7f5b79d9b640 1 -- 172.21.15.43:0/1564757696 --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5b6400bd30 con 0x7f5b7c143c20 2023-08-09T13:51:05.798 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:05.793+0000 7f5b7b59e640 1 --2- 172.21.15.43:0/1564757696 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f5b64004d00 0x7f5b64005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:51:05.799 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:05.793+0000 7f5b7ad9d640 1 --2- 172.21.15.43:0/1564757696 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f5b7c143c20 0x7f5b64003a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:51:05.799 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:05.793+0000 7f5b7b59e640 1 -- 172.21.15.43:0/1564757696 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f5b7c143c20 msgr2=0x7f5b64003a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-09T13:51:05.800 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:05.793+0000 7f5b7b59e640 1 --2- 172.21.15.43:0/1564757696 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f5b7c143c20 0x7f5b64003a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:51:05.800 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:05.793+0000 7f5b7b59e640 1 -- 172.21.15.43:0/1564757696 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f5b64003f90 msgr2=0x7f5b640047c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-09T13:51:05.800 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:05.793+0000 7f5b7a59c640 1 --2- 172.21.15.43:0/1564757696 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f5b64003f90 0x7f5b640047c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:51:05.800 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:05.794+0000 7f5b7b59e640 1 --2- 172.21.15.43:0/1564757696 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f5b64003f90 0x7f5b640047c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:51:05.800 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:05.794+0000 7f5b7b59e640 1 -- 172.21.15.43:0/1564757696 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5b64003520 con 0x7f5b64004d00 2023-08-09T13:51:05.800 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:05.794+0000 7f5b7ad9d640 1 --2- 172.21.15.43:0/1564757696 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f5b7c143c20 0x7f5b64003a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-08-09T13:51:05.800 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:05.795+0000 7f5b7b59e640 1 --2- 172.21.15.43:0/1564757696 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f5b64004d00 0x7f5b64005180 secure :-1 s=READY pgs=1130 cs=0 l=1 rev1=1 crypto rx=0x7f5b74001720 tx=0x7f5b640057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5b8355ec28f04055 server_cookie=0 in_seq=0 out_seq=0 2023-08-09T13:51:05.801 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:05.795+0000 7f5b79d9b640 1 -- 172.21.15.43:0/1564757696 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5b740052d0 con 0x7f5b64004d00 2023-08-09T13:51:05.801 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:05.795+0000 7f5b79d9b640 1 -- 172.21.15.43:0/1564757696 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5b7402e050 con 0x7f5b64004d00 2023-08-09T13:51:05.801 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:05.795+0000 7f5b79d9b640 1 -- 172.21.15.43:0/1564757696 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5b74003830 con 0x7f5b64004d00 2023-08-09T13:51:05.801 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:05.795+0000 7f5b810ad640 1 -- 172.21.15.43:0/1564757696 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f5b64004d00 msgr2=0x7f5b64005180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:51:05.801 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:05.795+0000 7f5b810ad640 1 --2- 172.21.15.43:0/1564757696 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f5b64004d00 0x7f5b64005180 secure :-1 s=READY pgs=1130 cs=0 l=1 rev1=1 crypto rx=0x7f5b74001720 tx=0x7f5b640057f0 comp rx=0 tx=0).stop 2023-08-09T13:51:05.801 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:05.795+0000 7f5b7b59e640 1 -- 172.21.15.43:0/1564757696 reap_dead start 2023-08-09T13:51:05.801 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:05.796+0000 7f5b810ad640 1 -- 172.21.15.43:0/1564757696 shutdown_connections 2023-08-09T13:51:05.802 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:05.796+0000 7f5b810ad640 1 -- 172.21.15.43:0/1564757696 >> 172.21.15.43:0/1564757696 conn(0x7f5b7c0ce0a0 msgr2=0x7f5b7c0ce480 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-09T13:51:05.802 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:05.796+0000 7f5b810ad640 1 -- 172.21.15.43:0/1564757696 shutdown_connections 2023-08-09T13:51:05.802 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:05.796+0000 7f5b810ad640 1 -- 172.21.15.43:0/1564757696 wait complete. 2023-08-09T13:51:05.803 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:05.798+0000 7f5b810ad640 1 Processor -- start 2023-08-09T13:51:05.804 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:05.798+0000 7f5b810ad640 1 -- start start 2023-08-09T13:51:05.804 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:05.799+0000 7f5b810ad640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f5b64004d00 0x7f5b7c1f0690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:51:05.804 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:05.799+0000 7f5b810ad640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f5b7c13bd50 0x7f5b7c15a680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:51:05.804 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:05.799+0000 7f5b810ad640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f5b7c0c6d00 0x7f5b7c0c71b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:51:05.804 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:05.799+0000 7f5b810ad640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5b7c0c76f0 con 0x7f5b64004d00 2023-08-09T13:51:05.804 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:05.799+0000 7f5b810ad640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5b7c1f0bd0 con 0x7f5b7c0c6d00 2023-08-09T13:51:05.804 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:05.799+0000 7f5b810ad640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5b7c1f3360 con 0x7f5b7c13bd50 2023-08-09T13:51:05.805 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:05.799+0000 7f5b7ad9d640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f5b64004d00 0x7f5b7c1f0690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:51:05.805 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:05.799+0000 7f5b7ad9d640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f5b64004d00 0x7f5b7c1f0690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:36492/0 (socket says 172.21.15.43:36492) 2023-08-09T13:51:05.805 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:05.799+0000 7f5b7ad9d640 1 -- 172.21.15.43:0/1522905266 learned_addr learned my addr 172.21.15.43:0/1522905266 (peer_addr_for_me v2:172.21.15.43:0/0) 2023-08-09T13:51:05.805 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:05.799+0000 7f5b7b59e640 1 --2- 172.21.15.43:0/1522905266 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f5b7c0c6d00 0x7f5b7c0c71b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:51:05.805 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:05.799+0000 7f5b7a59c640 1 --2- 172.21.15.43:0/1522905266 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f5b7c13bd50 0x7f5b7c15a680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:51:05.805 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:05.799+0000 7f5b7ad9d640 1 -- 172.21.15.43:0/1522905266 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f5b7c13bd50 msgr2=0x7f5b7c15a680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:51:05.805 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:05.799+0000 7f5b7ad9d640 1 --2- 172.21.15.43:0/1522905266 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f5b7c13bd50 0x7f5b7c15a680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:51:05.806 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:05.800+0000 7f5b7ad9d640 1 -- 172.21.15.43:0/1522905266 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f5b7c0c6d00 msgr2=0x7f5b7c0c71b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:51:05.806 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:05.800+0000 7f5b7ad9d640 1 --2- 172.21.15.43:0/1522905266 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f5b7c0c6d00 0x7f5b7c0c71b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:51:05.806 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:05.800+0000 7f5b7ad9d640 1 -- 172.21.15.43:0/1522905266 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5b7c1f3640 con 0x7f5b64004d00 2023-08-09T13:51:05.806 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:05.800+0000 7f5b7a59c640 1 --2- 172.21.15.43:0/1522905266 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f5b7c13bd50 0x7f5b7c15a680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-08-09T13:51:05.806 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:05.800+0000 7f5b7ad9d640 1 --2- 172.21.15.43:0/1522905266 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f5b64004d00 0x7f5b7c1f0690 secure :-1 s=READY pgs=1131 cs=0 l=1 rev1=1 crypto rx=0x7f5b6c00f350 tx=0x7f5b6c00f810 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-09T13:51:05.806 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:05.800+0000 7f5b637fe640 1 -- 172.21.15.43:0/1522905266 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5b6c003b90 con 0x7f5b64004d00 2023-08-09T13:51:05.806 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:05.800+0000 7f5b637fe640 1 -- 172.21.15.43:0/1522905266 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5b6c011040 con 0x7f5b64004d00 2023-08-09T13:51:05.806 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:05.800+0000 7f5b637fe640 1 -- 172.21.15.43:0/1522905266 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5b6c004020 con 0x7f5b64004d00 2023-08-09T13:51:05.807 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:05.800+0000 7f5b810ad640 1 -- 172.21.15.43:0/1522905266 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5b7c1f3920 con 0x7f5b64004d00 2023-08-09T13:51:05.807 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:05.800+0000 7f5b810ad640 1 -- 172.21.15.43:0/1522905266 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5b7c1f3ee0 con 0x7f5b64004d00 2023-08-09T13:51:05.808 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:05.802+0000 7f5b810ad640 1 -- 172.21.15.43:0/1522905266 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5b7c1f40a0 con 0x7f5b64004d00 2023-08-09T13:51:05.808 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:05.802+0000 7f5b637fe640 1 -- 172.21.15.43:0/1522905266 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 37) v1 ==== 88443+0+0 (secure 0 0 0) 0x7f5b6c004180 con 0x7f5b64004d00 2023-08-09T13:51:05.808 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:05.803+0000 7f5b637fe640 1 --2- 172.21.15.43:0/1522905266 >> [v2:172.21.15.203:6832/1778614695,v1:172.21.15.203:6833/1778614695] conn(0x7f5b6806a170 0x7f5b6806c630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:51:05.809 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:05.803+0000 7f5b637fe640 1 -- 172.21.15.43:0/1522905266 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(95..95 src has 1..95) v4 ==== 6348+0+0 (secure 0 0 0) 0x7f5b6c089200 con 0x7f5b64004d00 2023-08-09T13:51:05.809 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:05.804+0000 7f5b7a59c640 1 --2- 172.21.15.43:0/1522905266 >> [v2:172.21.15.203:6832/1778614695,v1:172.21.15.203:6833/1778614695] conn(0x7f5b6806a170 0x7f5b6806c630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:51:05.809 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:05.805+0000 7f5b7a59c640 1 --2- 172.21.15.43:0/1522905266 >> [v2:172.21.15.203:6832/1778614695,v1:172.21.15.203:6833/1778614695] conn(0x7f5b6806a170 0x7f5b6806c630 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7f5b70001200 tx=0x7f5b70001510 comp rx=0 tx=0).ready entity=mgr.5819 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-09T13:51:05.810 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:05.808+0000 7f5b637fe640 1 -- 172.21.15.43:0/1522905266 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178066 (secure 0 0 0) 0x7f5b6c054510 con 0x7f5b64004d00 2023-08-09T13:51:06.078 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:06.076+0000 7f5b810ad640 1 -- 172.21.15.43:0/1522905266 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_snap_schedules.TestSnapSchedules.test_non_existent_schedule"]} v 0) v1 -- 0x7f5b7c0739f0 con 0x7f5b64004d00 2023-08-09T13:51:06.132 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:06.129+0000 7f5b637fe640 1 -- 172.21.15.43:0/1522905266 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_snap_schedules.TestSnapSchedules.test_non_existent_schedule"]}]=0 v653) v1 ==== 156+0+0 (secure 0 0 0) 0x7f5b6c057b30 con 0x7f5b64004d00 2023-08-09T13:51:06.136 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:06.132+0000 7f5b810ad640 1 -- 172.21.15.43:0/1522905266 >> [v2:172.21.15.203:6832/1778614695,v1:172.21.15.203:6833/1778614695] conn(0x7f5b6806a170 msgr2=0x7f5b6806c630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:51:06.136 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:06.132+0000 7f5b810ad640 1 --2- 172.21.15.43:0/1522905266 >> [v2:172.21.15.203:6832/1778614695,v1:172.21.15.203:6833/1778614695] conn(0x7f5b6806a170 0x7f5b6806c630 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7f5b70001200 tx=0x7f5b70001510 comp rx=0 tx=0).stop 2023-08-09T13:51:06.136 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:06.132+0000 7f5b810ad640 1 -- 172.21.15.43:0/1522905266 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f5b64004d00 msgr2=0x7f5b7c1f0690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:51:06.136 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:06.132+0000 7f5b810ad640 1 --2- 172.21.15.43:0/1522905266 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f5b64004d00 0x7f5b7c1f0690 secure :-1 s=READY pgs=1131 cs=0 l=1 rev1=1 crypto rx=0x7f5b6c00f350 tx=0x7f5b6c00f810 comp rx=0 tx=0).stop 2023-08-09T13:51:06.137 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:06.133+0000 7f5b810ad640 1 -- 172.21.15.43:0/1522905266 shutdown_connections 2023-08-09T13:51:06.137 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:06.133+0000 7f5b810ad640 1 --2- 172.21.15.43:0/1522905266 >> [v2:172.21.15.203:6832/1778614695,v1:172.21.15.203:6833/1778614695] conn(0x7f5b6806a170 0x7f5b6806c630 unknown :-1 s=CLOSED pgs=38 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:51:06.137 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:06.133+0000 7f5b810ad640 1 --2- 172.21.15.43:0/1522905266 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f5b7c0c6d00 0x7f5b7c0c71b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:51:06.137 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:06.133+0000 7f5b810ad640 1 --2- 172.21.15.43:0/1522905266 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f5b7c13bd50 0x7f5b7c15a680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:51:06.137 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:06.133+0000 7f5b810ad640 1 --2- 172.21.15.43:0/1522905266 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f5b64004d00 0x7f5b7c1f0690 unknown :-1 s=CLOSED pgs=1131 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:51:06.137 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:06.133+0000 7f5b810ad640 1 -- 172.21.15.43:0/1522905266 >> 172.21.15.43:0/1522905266 conn(0x7f5b7c0ce0a0 msgr2=0x7f5b7c0cb0d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-09T13:51:06.137 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:06.133+0000 7f5b810ad640 1 -- 172.21.15.43:0/1522905266 shutdown_connections 2023-08-09T13:51:06.138 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:06.133+0000 7f5b810ad640 1 -- 172.21.15.43:0/1522905266 wait complete. 2023-08-09T13:51:06.154 INFO:tasks.cephfs_test_runner:Test listing non-existing snap schedules failure ... ok 2023-08-09T13:51:06.155 INFO:tasks.cephfs_test_runner:Starting test: test_non_existent_snap_schedule_list (tasks.cephfs.test_snap_schedules.TestSnapSchedules) Test listing snap schedules on a non-existing filesystem path failure 2023-08-09T13:51:06.155 INFO:tasks.cephfs_test_runner:test_non_existent_snap_schedule_list (tasks.cephfs.test_snap_schedules.TestSnapSchedules) 2023-08-09T13:51:06.155 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Starting test tasks.cephfs.test_snap_schedules.TestSnapSchedules.test_non_existent_snap_schedule_list' 2023-08-09T13:51:06.381 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:06.379+0000 7f26d3dee640 1 Processor -- start 2023-08-09T13:51:06.382 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:06.379+0000 7f26d3dee640 1 -- start start 2023-08-09T13:51:06.383 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:06.381+0000 7f26d3dee640 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7f26cc15c630 0x7f26cc0abef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:51:06.383 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:06.381+0000 7f26d3dee640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f26cc0cdc60 0x7f26cc15ca10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:51:06.384 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:06.381+0000 7f26d3dee640 1 -- --> v1:172.21.15.203:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f26cc0cca90 con 0x7f26cc0ccf20 2023-08-09T13:51:06.384 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:06.381+0000 7f26d3dee640 1 -- --> v2:172.21.15.203:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f26cc0cbc70 con 0x7f26cc15c630 2023-08-09T13:51:06.384 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:06.381+0000 7f26d3dee640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f26cc0cbe80 con 0x7f26cc0cdc60 2023-08-09T13:51:06.384 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:06.382+0000 7f26d238b640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f26cc0cdc60 0x7f26cc15ca10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:51:06.385 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:06.382+0000 7f26d1389640 1 --1- >> v1:172.21.15.203:6789/0 conn(0x7f26cc0ccf20 0x7f26cc0cd550 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.203:6789/0 says I am v1:172.21.15.43:46828/0 (socket says 172.21.15.43:46828) 2023-08-09T13:51:06.385 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:06.382+0000 7f26d1389640 1 -- 172.21.15.43:0/4289558938 learned_addr learned my addr 172.21.15.43:0/4289558938 (peer_addr_for_me v1:172.21.15.43:0/0) 2023-08-09T13:51:06.385 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:06.382+0000 7f26d1b8a640 1 --2- 172.21.15.43:0/4289558938 >> v2:172.21.15.203:3300/0 conn(0x7f26cc15c630 0x7f26cc0abef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:51:06.386 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:06.383+0000 7f26d238b640 1 -- 172.21.15.43:0/4289558938 >> v2:172.21.15.203:3300/0 conn(0x7f26cc15c630 msgr2=0x7f26cc0abef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-09T13:51:06.386 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:06.383+0000 7f26d238b640 1 --2- 172.21.15.43:0/4289558938 >> v2:172.21.15.203:3300/0 conn(0x7f26cc15c630 0x7f26cc0abef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:51:06.386 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:06.383+0000 7f26d238b640 1 -- 172.21.15.43:0/4289558938 >> v1:172.21.15.203:6789/0 conn(0x7f26cc0ccf20 legacy=0x7f26cc0cd550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:51:06.386 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:06.383+0000 7f26d238b640 1 -- 172.21.15.43:0/4289558938 --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f26cc0cc160 con 0x7f26cc0cdc60 2023-08-09T13:51:06.386 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:06.383+0000 7f26d0b88640 1 -- 172.21.15.43:0/4289558938 <== mon.1 v1:172.21.15.203:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2368956600 0 0) 0x7f26cc0cca90 con 0x7f26cc0ccf20 2023-08-09T13:51:06.387 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:06.384+0000 7f26d238b640 1 --2- 172.21.15.43:0/4289558938 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f26cc0cdc60 0x7f26cc15ca10 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7f26c4009d90 tx=0x7f26c402bea0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ab402d2b57ad0ded server_cookie=0 in_seq=0 out_seq=0 2023-08-09T13:51:06.387 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:06.384+0000 7f26d0b88640 1 -- 172.21.15.43:0/4289558938 <== mon.2 v2:172.21.15.203:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f26c402cc50 con 0x7f26cc0cdc60 2023-08-09T13:51:06.387 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:06.384+0000 7f26d0b88640 1 -- 172.21.15.43:0/4289558938 <== mon.2 v2:172.21.15.203:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f26c402cdb0 con 0x7f26cc0cdc60 2023-08-09T13:51:06.387 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:06.384+0000 7f26d0b88640 1 -- 172.21.15.43:0/4289558938 <== mon.2 v2:172.21.15.203:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f26c4034660 con 0x7f26cc0cdc60 2023-08-09T13:51:06.388 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:06.385+0000 7f26d3dee640 1 -- 172.21.15.43:0/4289558938 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f26cc0cdc60 msgr2=0x7f26cc15ca10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:51:06.388 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:06.385+0000 7f26d3dee640 1 --2- 172.21.15.43:0/4289558938 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f26cc0cdc60 0x7f26cc15ca10 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7f26c4009d90 tx=0x7f26c402bea0 comp rx=0 tx=0).stop 2023-08-09T13:51:06.388 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:06.385+0000 7f26d3dee640 1 -- 172.21.15.43:0/4289558938 shutdown_connections 2023-08-09T13:51:06.388 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:06.385+0000 7f26d3dee640 1 --2- 172.21.15.43:0/4289558938 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f26cc0cdc60 0x7f26cc15ca10 unknown :-1 s=CLOSED pgs=158 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:51:06.389 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:06.385+0000 7f26d3dee640 1 --2- 172.21.15.43:0/4289558938 >> v2:172.21.15.203:3300/0 conn(0x7f26cc15c630 0x7f26cc0abef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:51:06.389 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:06.385+0000 7f26d3dee640 1 -- 172.21.15.43:0/4289558938 >> 172.21.15.43:0/4289558938 conn(0x7f26cc0ce0a0 msgr2=0x7f26cc0ce480 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-09T13:51:06.389 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:06.385+0000 7f26d3dee640 1 -- 172.21.15.43:0/4289558938 shutdown_connections 2023-08-09T13:51:06.389 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:06.385+0000 7f26d3dee640 1 -- 172.21.15.43:0/4289558938 wait complete. 2023-08-09T13:51:06.389 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:06.387+0000 7f26d3dee640 1 Processor -- start 2023-08-09T13:51:06.390 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:06.387+0000 7f26d3dee640 1 -- start start 2023-08-09T13:51:06.391 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:06.388+0000 7f26d3dee640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f26cc0ccf20 0x7f26cc0c7550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:51:06.391 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:06.388+0000 7f26d3dee640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f26cc0cdc60 0x7f26cc0adb00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:51:06.391 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:06.388+0000 7f26d3dee640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f26cc15c630 0x7f26cc1f14f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:51:06.391 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:06.388+0000 7f26d3dee640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f26cc0ae040 con 0x7f26cc0cdc60 2023-08-09T13:51:06.391 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:06.388+0000 7f26d3dee640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f26cc0ae250 con 0x7f26cc15c630 2023-08-09T13:51:06.391 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:06.388+0000 7f26d3dee640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f26cc0ae530 con 0x7f26cc0ccf20 2023-08-09T13:51:06.392 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:06.389+0000 7f26d1389640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f26cc0cdc60 0x7f26cc0adb00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:51:06.393 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:06.389+0000 7f26d1389640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f26cc0cdc60 0x7f26cc0adb00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:36494/0 (socket says 172.21.15.43:36494) 2023-08-09T13:51:06.393 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:06.389+0000 7f26d1389640 1 -- 172.21.15.43:0/2975484226 learned_addr learned my addr 172.21.15.43:0/2975484226 (peer_addr_for_me v2:172.21.15.43:0/0) 2023-08-09T13:51:06.393 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:06.389+0000 7f26d238b640 1 --2- 172.21.15.43:0/2975484226 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f26cc15c630 0x7f26cc1f14f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:51:06.393 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:06.389+0000 7f26d1b8a640 1 --2- 172.21.15.43:0/2975484226 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f26cc0ccf20 0x7f26cc0c7550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:51:06.393 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:06.389+0000 7f26d1389640 1 -- 172.21.15.43:0/2975484226 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f26cc0ccf20 msgr2=0x7f26cc0c7550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:51:06.393 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:06.389+0000 7f26d1389640 1 --2- 172.21.15.43:0/2975484226 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f26cc0ccf20 0x7f26cc0c7550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:51:06.394 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:06.389+0000 7f26d1389640 1 -- 172.21.15.43:0/2975484226 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f26cc15c630 msgr2=0x7f26cc1f14f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:51:06.394 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:06.389+0000 7f26d1389640 1 --2- 172.21.15.43:0/2975484226 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f26cc15c630 0x7f26cc1f14f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:51:06.394 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:06.389+0000 7f26d1389640 1 -- 172.21.15.43:0/2975484226 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f26c4009a70 con 0x7f26cc0cdc60 2023-08-09T13:51:06.394 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:06.390+0000 7f26d1389640 1 --2- 172.21.15.43:0/2975484226 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f26cc0cdc60 0x7f26cc0adb00 secure :-1 s=READY pgs=1132 cs=0 l=1 rev1=1 crypto rx=0x7f26bc00c4d0 tx=0x7f26bc00c990 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-09T13:51:06.394 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:06.390+0000 7f26c27fc640 1 -- 172.21.15.43:0/2975484226 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f26bc0074e0 con 0x7f26cc0cdc60 2023-08-09T13:51:06.394 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:06.390+0000 7f26c27fc640 1 -- 172.21.15.43:0/2975484226 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f26bc007660 con 0x7f26cc0cdc60 2023-08-09T13:51:06.394 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:06.390+0000 7f26d3dee640 1 -- 172.21.15.43:0/2975484226 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f26cc0ae810 con 0x7f26cc0cdc60 2023-08-09T13:51:06.394 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:06.390+0000 7f26c27fc640 1 -- 172.21.15.43:0/2975484226 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f26bc004a20 con 0x7f26cc0cdc60 2023-08-09T13:51:06.395 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:06.390+0000 7f26d3dee640 1 -- 172.21.15.43:0/2975484226 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f26cc1f1a30 con 0x7f26cc0cdc60 2023-08-09T13:51:06.395 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:06.391+0000 7f26d3dee640 1 -- 172.21.15.43:0/2975484226 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f26cc15d640 con 0x7f26cc0cdc60 2023-08-09T13:51:06.395 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:06.392+0000 7f26c27fc640 1 -- 172.21.15.43:0/2975484226 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 37) v1 ==== 88443+0+0 (secure 0 0 0) 0x7f26bc004b80 con 0x7f26cc0cdc60 2023-08-09T13:51:06.396 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:06.393+0000 7f26c27fc640 1 --2- 172.21.15.43:0/2975484226 >> [v2:172.21.15.203:6832/1778614695,v1:172.21.15.203:6833/1778614695] conn(0x7f26b006a170 0x7f26b006c630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:51:06.396 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:06.394+0000 7f26c27fc640 1 -- 172.21.15.43:0/2975484226 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(95..95 src has 1..95) v4 ==== 6348+0+0 (secure 0 0 0) 0x7f26bc087e70 con 0x7f26cc0cdc60 2023-08-09T13:51:06.396 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:06.394+0000 7f26d1b8a640 1 --2- 172.21.15.43:0/2975484226 >> [v2:172.21.15.203:6832/1778614695,v1:172.21.15.203:6833/1778614695] conn(0x7f26b006a170 0x7f26b006c630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:51:06.397 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:06.395+0000 7f26d1b8a640 1 --2- 172.21.15.43:0/2975484226 >> [v2:172.21.15.203:6832/1778614695,v1:172.21.15.203:6833/1778614695] conn(0x7f26b006a170 0x7f26b006c630 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7f26c8009a30 tx=0x7f26c8008040 comp rx=0 tx=0).ready entity=mgr.5819 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-09T13:51:06.400 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:06.398+0000 7f26c27fc640 1 -- 172.21.15.43:0/2975484226 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178066 (secure 0 0 0) 0x7f26cc15d640 con 0x7f26cc0cdc60 2023-08-09T13:51:06.666 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:06.664+0000 7f26d3dee640 1 -- 172.21.15.43:0/2975484226 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_snap_schedules.TestSnapSchedules.test_non_existent_snap_schedule_list"]} v 0) v1 -- 0x7f26cc15da20 con 0x7f26cc0cdc60 2023-08-09T13:51:07.136 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:07.133+0000 7f26c27fc640 1 -- 172.21.15.43:0/2975484226 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_snap_schedules.TestSnapSchedules.test_non_existent_snap_schedule_list"]}]=0 v654) v1 ==== 169+0+0 (secure 0 0 0) 0x7f26cc15da20 con 0x7f26cc0cdc60 2023-08-09T13:51:07.139 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:07.137+0000 7f26d3dee640 1 -- 172.21.15.43:0/2975484226 >> [v2:172.21.15.203:6832/1778614695,v1:172.21.15.203:6833/1778614695] conn(0x7f26b006a170 msgr2=0x7f26b006c630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:51:07.139 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:07.137+0000 7f26d3dee640 1 --2- 172.21.15.43:0/2975484226 >> [v2:172.21.15.203:6832/1778614695,v1:172.21.15.203:6833/1778614695] conn(0x7f26b006a170 0x7f26b006c630 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7f26c8009a30 tx=0x7f26c8008040 comp rx=0 tx=0).stop 2023-08-09T13:51:07.139 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:07.137+0000 7f26d3dee640 1 -- 172.21.15.43:0/2975484226 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f26cc0cdc60 msgr2=0x7f26cc0adb00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:51:07.139 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:07.137+0000 7f26d3dee640 1 --2- 172.21.15.43:0/2975484226 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f26cc0cdc60 0x7f26cc0adb00 secure :-1 s=READY pgs=1132 cs=0 l=1 rev1=1 crypto rx=0x7f26bc00c4d0 tx=0x7f26bc00c990 comp rx=0 tx=0).stop 2023-08-09T13:51:07.140 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:07.137+0000 7f26d3dee640 1 -- 172.21.15.43:0/2975484226 shutdown_connections 2023-08-09T13:51:07.140 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:07.137+0000 7f26d3dee640 1 --2- 172.21.15.43:0/2975484226 >> [v2:172.21.15.203:6832/1778614695,v1:172.21.15.203:6833/1778614695] conn(0x7f26b006a170 0x7f26b006c630 unknown :-1 s=CLOSED pgs=39 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:51:07.140 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:07.137+0000 7f26d3dee640 1 --2- 172.21.15.43:0/2975484226 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f26cc15c630 0x7f26cc1f14f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:51:07.141 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:07.137+0000 7f26d3dee640 1 --2- 172.21.15.43:0/2975484226 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f26cc0cdc60 0x7f26cc0adb00 unknown :-1 s=CLOSED pgs=1132 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:51:07.141 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:07.137+0000 7f26d3dee640 1 --2- 172.21.15.43:0/2975484226 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f26cc0ccf20 0x7f26cc0c7550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:51:07.141 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:07.137+0000 7f26d3dee640 1 -- 172.21.15.43:0/2975484226 >> 172.21.15.43:0/2975484226 conn(0x7f26cc0ce0a0 msgr2=0x7f26cc160900 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-09T13:51:07.141 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:07.137+0000 7f26d3dee640 1 -- 172.21.15.43:0/2975484226 shutdown_connections 2023-08-09T13:51:07.141 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:07.137+0000 7f26d3dee640 1 -- 172.21.15.43:0/2975484226 wait complete. 2023-08-09T13:51:07.161 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-08-09T13:51:07.395 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:07.392+0000 7feb0f59e640 1 Processor -- start 2023-08-09T13:51:07.395 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:07.393+0000 7feb0f59e640 1 -- start start 2023-08-09T13:51:07.396 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:07.394+0000 7feb0f59e640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7feb100c8b30 0x7feb100ad340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:51:07.396 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:07.394+0000 7feb0f59e640 1 -- --> v1:172.21.15.43:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7feb100cbd90 con 0x7feb100acdf0 2023-08-09T13:51:07.396 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:07.394+0000 7feb0f59e640 1 -- --> v1:172.21.15.203:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7feb100ca2b0 con 0x7feb100c8750 2023-08-09T13:51:07.397 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:07.394+0000 7feb0f59e640 1 -- --> v2:172.21.15.43:3300/0 -- mon_getmap magic: 0 v1 -- 0x7feb1015e810 con 0x7feb100c8b30 2023-08-09T13:51:07.397 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:07.394+0000 7feb05d9b640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7feb100c8b30 0x7feb100ad340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:51:07.397 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:07.394+0000 7feb0ed9d640 1 --1- >> v1:172.21.15.43:6789/0 conn(0x7feb100acdf0 0x7feb10070f00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.43:6789/0 says I am v1:172.21.15.43:45670/0 (socket says 172.21.15.43:45670) 2023-08-09T13:51:07.398 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:07.394+0000 7feb05d9b640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7feb100c8b30 0x7feb100ad340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:36500/0 (socket says 172.21.15.43:36500) 2023-08-09T13:51:07.398 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:07.394+0000 7feb0ed9d640 1 -- 172.21.15.43:0/1616651773 learned_addr learned my addr 172.21.15.43:0/1616651773 (peer_addr_for_me v1:172.21.15.43:0/0) 2023-08-09T13:51:07.398 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:07.395+0000 7feb05d9b640 1 -- 172.21.15.43:0/1616651773 >> v1:172.21.15.203:6789/0 conn(0x7feb100c8750 legacy=0x7feb1006f410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-09T13:51:07.398 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:07.395+0000 7feb05d9b640 1 -- 172.21.15.43:0/1616651773 >> v1:172.21.15.43:6789/0 conn(0x7feb100acdf0 legacy=0x7feb10070f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:51:07.398 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:07.395+0000 7feb05d9b640 1 -- 172.21.15.43:0/1616651773 --> v2:172.21.15.43:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7feb100c9a20 con 0x7feb100c8b30 2023-08-09T13:51:07.399 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:07.396+0000 7feb05d9b640 1 --2- 172.21.15.43:0/1616651773 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7feb100c8b30 0x7feb100ad340 secure :-1 s=READY pgs=1134 cs=0 l=1 rev1=1 crypto rx=0x7feaf80097a0 tx=0x7feaf802b840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=749b321a9f81cb03 server_cookie=0 in_seq=0 out_seq=0 2023-08-09T13:51:07.399 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:07.396+0000 7feb0dd9b640 1 -- 172.21.15.43:0/1616651773 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feaf802e070 con 0x7feb100c8b30 2023-08-09T13:51:07.399 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:07.396+0000 7feb0dd9b640 1 -- 172.21.15.43:0/1616651773 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7feaf80315a0 con 0x7feb100c8b30 2023-08-09T13:51:07.399 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:07.396+0000 7feb0dd9b640 1 -- 172.21.15.43:0/1616651773 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feaf8031930 con 0x7feb100c8b30 2023-08-09T13:51:07.400 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:07.397+0000 7feb0f59e640 1 -- 172.21.15.43:0/1616651773 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7feb100c8b30 msgr2=0x7feb100ad340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:51:07.400 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:07.397+0000 7feb0f59e640 1 --2- 172.21.15.43:0/1616651773 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7feb100c8b30 0x7feb100ad340 secure :-1 s=READY pgs=1134 cs=0 l=1 rev1=1 crypto rx=0x7feaf80097a0 tx=0x7feaf802b840 comp rx=0 tx=0).stop 2023-08-09T13:51:07.400 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:07.397+0000 7feb0f59e640 1 -- 172.21.15.43:0/1616651773 shutdown_connections 2023-08-09T13:51:07.400 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:07.397+0000 7feb0f59e640 1 --2- 172.21.15.43:0/1616651773 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7feb100c8b30 0x7feb100ad340 unknown :-1 s=CLOSED pgs=1134 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:51:07.401 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:07.397+0000 7feb0f59e640 1 -- 172.21.15.43:0/1616651773 >> 172.21.15.43:0/1616651773 conn(0x7feb100cde90 msgr2=0x7feb100ce2a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-09T13:51:07.401 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:07.397+0000 7feb0f59e640 1 -- 172.21.15.43:0/1616651773 shutdown_connections 2023-08-09T13:51:07.401 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:07.397+0000 7feb0f59e640 1 -- 172.21.15.43:0/1616651773 wait complete. 2023-08-09T13:51:07.401 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:07.399+0000 7feb0f59e640 1 Processor -- start 2023-08-09T13:51:07.402 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:07.399+0000 7feb0f59e640 1 -- start start 2023-08-09T13:51:07.403 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:07.400+0000 7feb0f59e640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7feb100acdf0 0x7feb100b3520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:51:07.403 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:07.400+0000 7feb0f59e640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7feb100c8750 0x7feb100b1410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:51:07.403 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:07.400+0000 7feb0f59e640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7feb100c8b30 0x7feb101580f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:51:07.403 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:07.400+0000 7feb0f59e640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feb100c9f70 con 0x7feb100acdf0 2023-08-09T13:51:07.403 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:07.400+0000 7feb0f59e640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feb100c4410 con 0x7feb100c8b30 2023-08-09T13:51:07.404 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:07.400+0000 7feb0f59e640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7feb1006bc00 con 0x7feb100c8750 2023-08-09T13:51:07.404 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:07.400+0000 7feb0e59c640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7feb100acdf0 0x7feb100b3520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:51:07.404 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:07.401+0000 7feb0e59c640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7feb100acdf0 0x7feb100b3520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:36510/0 (socket says 172.21.15.43:36510) 2023-08-09T13:51:07.404 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:07.401+0000 7feb0e59c640 1 -- 172.21.15.43:0/2799515686 learned_addr learned my addr 172.21.15.43:0/2799515686 (peer_addr_for_me v2:172.21.15.43:0/0) 2023-08-09T13:51:07.405 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:07.401+0000 7feb05d9b640 1 --2- 172.21.15.43:0/2799515686 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7feb100c8750 0x7feb100b1410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:51:07.405 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:07.401+0000 7feb0ed9d640 1 --2- 172.21.15.43:0/2799515686 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7feb100c8b30 0x7feb101580f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:51:07.405 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:07.401+0000 7feb0e59c640 1 -- 172.21.15.43:0/2799515686 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7feb100c8750 msgr2=0x7feb100b1410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:51:07.405 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:07.401+0000 7feb0e59c640 1 --2- 172.21.15.43:0/2799515686 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7feb100c8750 0x7feb100b1410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:51:07.406 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:07.401+0000 7feb0e59c640 1 -- 172.21.15.43:0/2799515686 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7feb100c8b30 msgr2=0x7feb101580f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:51:07.406 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:07.401+0000 7feb0e59c640 1 --2- 172.21.15.43:0/2799515686 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7feb100c8b30 0x7feb101580f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:51:07.406 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:07.401+0000 7feb0e59c640 1 -- 172.21.15.43:0/2799515686 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7feaf8009400 con 0x7feb100acdf0 2023-08-09T13:51:07.406 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:07.401+0000 7feb0e59c640 1 --2- 172.21.15.43:0/2799515686 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7feb100acdf0 0x7feb100b3520 secure :-1 s=READY pgs=1135 cs=0 l=1 rev1=1 crypto rx=0x7feb0000c430 tx=0x7feb0000c8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-09T13:51:07.406 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:07.402+0000 7feb077fe640 1 -- 172.21.15.43:0/2799515686 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feb00011790 con 0x7feb100acdf0 2023-08-09T13:51:07.406 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:07.402+0000 7feb077fe640 1 -- 172.21.15.43:0/2799515686 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7feb000118f0 con 0x7feb100acdf0 2023-08-09T13:51:07.406 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:07.402+0000 7feb077fe640 1 -- 172.21.15.43:0/2799515686 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feb00011ce0 con 0x7feb100acdf0 2023-08-09T13:51:07.407 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:07.402+0000 7feb0f59e640 1 -- 172.21.15.43:0/2799515686 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7feb100b3a60 con 0x7feb100acdf0 2023-08-09T13:51:07.407 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:07.402+0000 7feb0f59e640 1 -- 172.21.15.43:0/2799515686 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7feb1006b540 con 0x7feb100acdf0 2023-08-09T13:51:07.407 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:07.404+0000 7feb0f59e640 1 -- 172.21.15.43:0/2799515686 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7feb1006b3a0 con 0x7feb100acdf0 2023-08-09T13:51:07.407 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:07.404+0000 7feb077fe640 1 -- 172.21.15.43:0/2799515686 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 37) v1 ==== 88443+0+0 (secure 0 0 0) 0x7feb0001c460 con 0x7feb100acdf0 2023-08-09T13:51:07.408 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:07.405+0000 7feb077fe640 1 --2- 172.21.15.43:0/2799515686 >> [v2:172.21.15.203:6832/1778614695,v1:172.21.15.203:6833/1778614695] conn(0x7feaf006a170 0x7feaf006c630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:51:07.408 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:07.405+0000 7feb077fe640 1 -- 172.21.15.43:0/2799515686 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(95..95 src has 1..95) v4 ==== 6348+0+0 (secure 0 0 0) 0x7feb00088730 con 0x7feb100acdf0 2023-08-09T13:51:07.408 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:07.406+0000 7feb05d9b640 1 --2- 172.21.15.43:0/2799515686 >> [v2:172.21.15.203:6832/1778614695,v1:172.21.15.203:6833/1778614695] conn(0x7feaf006a170 0x7feaf006c630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:51:07.409 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:07.407+0000 7feb05d9b640 1 --2- 172.21.15.43:0/2799515686 >> [v2:172.21.15.203:6832/1778614695,v1:172.21.15.203:6833/1778614695] conn(0x7feaf006a170 0x7feaf006c630 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7feaf8036000 tx=0x7feaf803d040 comp rx=0 tx=0).ready entity=mgr.5819 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-09T13:51:07.412 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:07.410+0000 7feb077fe640 1 -- 172.21.15.43:0/2799515686 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178066 (secure 0 0 0) 0x7feb1006b3a0 con 0x7feb100acdf0 2023-08-09T13:51:07.606 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:07.604+0000 7feb0f59e640 1 -- 172.21.15.43:0/2799515686 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7feb1006bd40 con 0x7feb100acdf0 2023-08-09T13:51:07.608 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:07.606+0000 7feb077fe640 1 -- 172.21.15.43:0/2799515686 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v95) v1 ==== 74+0+15995 (secure 0 0 0) 0x7feb1006bd40 con 0x7feb100acdf0 2023-08-09T13:51:07.609 INFO:teuthology.orchestra.run.smithi043.stdout: 2023-08-09T13:51:07.609 INFO:teuthology.orchestra.run.smithi043.stdout:{"epoch":95,"fsid":"daac87ee-1337-4250-a955-aea4e408a39b","created":"2023-08-09T13:33:41.701198+0000","modified":"2023-08-09T13:51:04.106481+0000","last_up_change":"2023-08-09T13:33:53.783453+0000","last_in_change":"2023-08-09T13:33:45.148904+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":11,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":"rbd","create_time":"2023-08-09T13:33:54.365137+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":".mgr","create_time":"2023-08-09T13:33:54.831803+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"cbd49ecd-e896-4fbd-864e-5c37a829b165","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6800","nonce":502667335},{"type":"v1","addr":"172.21.15.43:6801","nonce":502667335}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6802","nonce":502667335},{"type":"v1","addr":"172.21.15.43:6803","nonce":502667335}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6806","nonce":502667335},{"type":"v1","addr":"172.21.15.43:6807","nonce":502667335}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6804","nonce":502667335},{"type":"v1","addr":"172.21.15.43:6805","nonce":502667335}]},"public_addr":"172.21.15.43:6801/502667335","cluster_addr":"172.21.15.43:6803/502667335","heartbeat_back_addr":"172.21.15.43:6807/502667335","heartbeat_front_addr":"172.21.15.43:6805/502667335","state":["exists","up"]},{"osd":1,"uuid":"75c7e0fe-1ec7-4fe8-a2f2-3661a65b8500","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6808","nonce":3289134610},{"type":"v1","addr":"172.21.15.43:6809","nonce":3289134610}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6810","nonce":3289134610},{"type":"v1","addr":"172.21.15.43:6812","nonce":3289134610}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6821","nonce":3289134610},{"type":"v1","addr":"172.21.15.43:6824","nonce":3289134610}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6815","nonce":3289134610},{"type":"v1","addr":"172.21.15.43:6818","nonce":3289134610}]},"public_addr":"172.21.15.43:6809/3289134610","cluster_addr":"172.21.15.43:6812/3289134610","heartbeat_back_addr":"172.21.15.43:6824/3289134610","heartbeat_front_addr":"172.21.15.43:6818/3289134610","state":["exists","up"]},{"osd":2,"uuid":"bb5c41e9-7f5f-4125-9b01-3bb8bfc2ea83","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6811","nonce":4079709736},{"type":"v1","addr":"172.21.15.43:6814","nonce":4079709736}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6817","nonce":4079709736},{"type":"v1","addr":"172.21.15.43:6820","nonce":4079709736}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6829","nonce":4079709736},{"type":"v1","addr":"172.21.15.43:6831","nonce":4079709736}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6823","nonce":4079709736},{"type":"v1","addr":"172.21.15.43:6826","nonce":4079709736}]},"public_addr":"172.21.15.43:6814/4079709736","cluster_addr":"172.21.15.43:6820/4079709736","heartbeat_back_addr":"172.21.15.43:6831/4079709736","heartbeat_front_addr":"172.21.15.43:6826/4079709736","state":["exists","up"]},{"osd":3,"uuid":"bfe4c8b4-694a-465c-87e6-2b2aab852613","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6813","nonce":1709538759},{"type":"v1","addr":"172.21.15.43:6816","nonce":1709538759}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6819","nonce":1709538759},{"type":"v1","addr":"172.21.15.43:6822","nonce":1709538759}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6828","nonce":1709538759},{"type":"v1","addr":"172.21.15.43:6830","nonce":1709538759}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6825","nonce":1709538759},{"type":"v1","addr":"172.21.15.43:6827","nonce":1709538759}]},"public_addr":"172.21.15.43:6816/1709538759","cluster_addr":"172.21.15.43:6822/1709538759","heartbeat_back_addr":"172.21.15.43:6830/1709538759","heartbeat_front_addr":"172.21.15.43:6827/1709538759","state":["exists","up"]},{"osd":4,"uuid":"fa2e8a77-aba1-4756-853e-0d174a5bdeba","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6800","nonce":4196866481},{"type":"v1","addr":"172.21.15.203:6801","nonce":4196866481}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6802","nonce":4196866481},{"type":"v1","addr":"172.21.15.203:6803","nonce":4196866481}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6806","nonce":4196866481},{"type":"v1","addr":"172.21.15.203:6807","nonce":4196866481}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6804","nonce":4196866481},{"type":"v1","addr":"172.21.15.203:6805","nonce":4196866481}]},"public_addr":"172.21.15.203:6801/4196866481","cluster_addr":"172.21.15.203:6803/4196866481","heartbeat_back_addr":"172.21.15.203:6807/4196866481","heartbeat_front_addr":"172.21.15.203:6805/4196866481","state":["exists","up"]},{"osd":5,"uuid":"81811e4b-4d24-4f51-a82c-f1d700a2a00d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6816","nonce":3059290081},{"type":"v1","addr":"172.21.15.203:6817","nonce":3059290081}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6818","nonce":3059290081},{"type":"v1","addr":"172.21.15.203:6819","nonce":3059290081}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6822","nonce":3059290081},{"type":"v1","addr":"172.21.15.203:6823","nonce":3059290081}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6820","nonce":3059290081},{"type":"v1","addr":"172.21.15.203:6821","nonce":3059290081}]},"public_addr":"172.21.15.203:6817/3059290081","cluster_addr":"172.21.15.203:6819/3059290081","heartbeat_back_addr":"172.21.15.203:6823/3059290081","heartbeat_front_addr":"172.21.15.203:6821/3059290081","state":["exists","up"]},{"osd":6,"uuid":"307bac4d-6a5b-443d-af75-56d695d7674f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6824","nonce":512155569},{"type":"v1","addr":"172.21.15.203:6825","nonce":512155569}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6826","nonce":512155569},{"type":"v1","addr":"172.21.15.203:6827","nonce":512155569}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6830","nonce":512155569},{"type":"v1","addr":"172.21.15.203:6831","nonce":512155569}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6828","nonce":512155569},{"type":"v1","addr":"172.21.15.203:6829","nonce":512155569}]},"public_addr":"172.21.15.203:6825/512155569","cluster_addr":"172.21.15.203:6827/512155569","heartbeat_back_addr":"172.21.15.203:6831/512155569","heartbeat_front_addr":"172.21.15.203:6829/512155569","state":["exists","up"]},{"osd":7,"uuid":"a95dc7ca-9f1b-4295-af70-68a22a2d7d5e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6808","nonce":3331450438},{"type":"v1","addr":"172.21.15.203:6809","nonce":3331450438}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6810","nonce":3331450438},{"type":"v1","addr":"172.21.15.203:6811","nonce":3331450438}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6814","nonce":3331450438},{"type":"v1","addr":"172.21.15.203:6815","nonce":3331450438}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6812","nonce":3331450438},{"type":"v1","addr":"172.21.15.203:6813","nonce":3331450438}]},"public_addr":"172.21.15.203:6809/3331450438","cluster_addr":"172.21.15.203:6811/3331450438","heartbeat_back_addr":"172.21.15.203:6815/3331450438","heartbeat_front_addr":"172.21.15.203:6813/3331450438","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-09T13:33:50.547652+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-09T13:33:50.589469+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-09T13:33:50.561724+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-09T13:33:50.599650+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-09T13:33:50.411855+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-09T13:33:50.219681+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-09T13:33:50.504664+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-09T13:33:50.441248+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.203:6836/3061388209":"2023-08-10T13:51:00.609442+0000","172.21.15.203:0/375511470":"2023-08-10T13:50:55.386522+0000","172.21.15.203:0/4256110232":"2023-08-10T13:50:55.386522+0000","172.21.15.203:0/3102402128":"2023-08-10T13:50:55.386522+0000","172.21.15.203:0/1112175379":"2023-08-10T13:50:55.386522+0000","172.21.15.203:6833/3851880723":"2023-08-10T13:50:55.386522+0000","172.21.15.203:6833/3325141227":"2023-08-10T13:50:49.168196+0000","172.21.15.203:6832/3325141227":"2023-08-10T13:50:49.168196+0000","172.21.15.203:0/1343982026":"2023-08-10T13:50:55.386522+0000","172.21.15.203:0/3732830113":"2023-08-10T13:50:49.168196+0000","172.21.15.203:6834/3061388209":"2023-08-10T13:51:00.609442+0000","172.21.15.203:0/2753554568":"2023-08-10T13:50:49.168196+0000","172.21.15.203:0/3841129997":"2023-08-10T13:50:55.386522+0000","172.21.15.203:0/2468196984":"2023-08-10T13:50:49.168196+0000","172.21.15.203:6832/3851880723":"2023-08-10T13:50:55.386522+0000","172.21.15.203:0/3375551329":"2023-08-10T13:50:49.168196+0000","172.21.15.203:0/1259705306":"2023-08-10T13:50:49.168196+0000","172.21.15.203:0/1322546009":"2023-08-10T13:50:49.168196+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":7,"snaps":[{"begin":104,"length":1},{"begin":221,"length":1}]}],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-08-09T13:51:07.612 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:07.609+0000 7feb0f59e640 1 -- 172.21.15.43:0/2799515686 >> [v2:172.21.15.203:6832/1778614695,v1:172.21.15.203:6833/1778614695] conn(0x7feaf006a170 msgr2=0x7feaf006c630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:51:07.612 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:07.609+0000 7feb0f59e640 1 --2- 172.21.15.43:0/2799515686 >> [v2:172.21.15.203:6832/1778614695,v1:172.21.15.203:6833/1778614695] conn(0x7feaf006a170 0x7feaf006c630 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7feaf8036000 tx=0x7feaf803d040 comp rx=0 tx=0).stop 2023-08-09T13:51:07.612 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:07.609+0000 7feb0f59e640 1 -- 172.21.15.43:0/2799515686 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7feb100acdf0 msgr2=0x7feb100b3520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:51:07.613 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:07.609+0000 7feb0f59e640 1 --2- 172.21.15.43:0/2799515686 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7feb100acdf0 0x7feb100b3520 secure :-1 s=READY pgs=1135 cs=0 l=1 rev1=1 crypto rx=0x7feb0000c430 tx=0x7feb0000c8f0 comp rx=0 tx=0).stop 2023-08-09T13:51:07.613 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:07.609+0000 7feb0f59e640 1 -- 172.21.15.43:0/2799515686 shutdown_connections 2023-08-09T13:51:07.613 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:07.609+0000 7feb0f59e640 1 --2- 172.21.15.43:0/2799515686 >> [v2:172.21.15.203:6832/1778614695,v1:172.21.15.203:6833/1778614695] conn(0x7feaf006a170 0x7feaf006c630 unknown :-1 s=CLOSED pgs=40 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:51:07.613 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:07.609+0000 7feb0f59e640 1 --2- 172.21.15.43:0/2799515686 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7feb100c8b30 0x7feb101580f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:51:07.613 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:07.609+0000 7feb0f59e640 1 --2- 172.21.15.43:0/2799515686 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7feb100c8750 0x7feb100b1410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:51:07.613 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:07.609+0000 7feb0f59e640 1 --2- 172.21.15.43:0/2799515686 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7feb100acdf0 0x7feb100b3520 unknown :-1 s=CLOSED pgs=1135 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:51:07.614 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:07.609+0000 7feb0f59e640 1 -- 172.21.15.43:0/2799515686 >> 172.21.15.43:0/2799515686 conn(0x7feb100cde90 msgr2=0x7feb100cc7b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-09T13:51:07.614 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:07.609+0000 7feb0f59e640 1 -- 172.21.15.43:0/2799515686 shutdown_connections 2023-08-09T13:51:07.614 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:07.609+0000 7feb0f59e640 1 -- 172.21.15.43:0/2799515686 wait complete. 2023-08-09T13:51:07.633 INFO:tasks.cephfs.cephfs_test_case.ceph_manager:[{'pool': 1, 'pool_name': 'rbd', 'create_time': '2023-08-09T13:33:54.365137+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}, 'read_balance': {'score_acting': 2, 'score_stable': 2, 'optimal_score': 1, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': '.mgr', 'create_time': '2023-08-09T13:33:54.831803+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-08-09T13:51:07.634 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get rbd pg_num 2023-08-09T13:51:07.867 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:07.865+0000 7fbd38fb5640 1 Processor -- start 2023-08-09T13:51:07.867 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:07.865+0000 7fbd38fb5640 1 -- start start 2023-08-09T13:51:07.869 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:07.866+0000 7fbd38fb5640 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7fbd3415c550 0x7fbd340cd6d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:51:07.869 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:07.866+0000 7fbd38fb5640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fbd340d04b0 0x7fbd340b3ee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:51:07.869 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:07.866+0000 7fbd38fb5640 1 -- --> v1:172.21.15.43:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbd340b4870 con 0x7fbd340b4420 2023-08-09T13:51:07.869 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:07.866+0000 7fbd38fb5640 1 -- --> v2:172.21.15.203:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbd340b7bd0 con 0x7fbd3415c550 2023-08-09T13:51:07.869 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:07.866+0000 7fbd38fb5640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbd340d1c00 con 0x7fbd340d04b0 2023-08-09T13:51:07.870 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:07.867+0000 7fbd32d9d640 1 --1- >> v1:172.21.15.43:6789/0 conn(0x7fbd340b4420 0x7fbd3406a760 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.43:6789/0 says I am v1:172.21.15.43:45678/0 (socket says 172.21.15.43:45678) 2023-08-09T13:51:07.871 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:07.867+0000 7fbd32d9d640 1 -- 172.21.15.43:0/660220546 learned_addr learned my addr 172.21.15.43:0/660220546 (peer_addr_for_me v1:172.21.15.43:0/0) 2023-08-09T13:51:07.871 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:07.867+0000 7fbd3259c640 1 --2- 172.21.15.43:0/660220546 >> v2:172.21.15.203:3300/0 conn(0x7fbd3415c550 0x7fbd340cd6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:51:07.871 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:07.867+0000 7fbd31d9b640 1 --2- 172.21.15.43:0/660220546 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fbd340d04b0 0x7fbd340b3ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:51:07.871 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:07.867+0000 7fbd3159a640 1 -- 172.21.15.43:0/660220546 <== mon.0 v1:172.21.15.43:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3910106894 0 0) 0x7fbd2c001110 con 0x7fbd340b4420 2023-08-09T13:51:07.871 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:07.868+0000 7fbd3159a640 1 -- 172.21.15.43:0/660220546 --> v1:172.21.15.43:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fbd20003520 con 0x7fbd340b4420 2023-08-09T13:51:07.871 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:07.868+0000 7fbd3159a640 1 -- 172.21.15.43:0/660220546 <== mon.0 v1:172.21.15.43:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3558660904 0 0) 0x7fbd20003520 con 0x7fbd340b4420 2023-08-09T13:51:07.871 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:07.868+0000 7fbd3259c640 1 -- 172.21.15.43:0/660220546 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fbd340d04b0 msgr2=0x7fbd340b3ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-09T13:51:07.872 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:07.868+0000 7fbd3259c640 1 --2- 172.21.15.43:0/660220546 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fbd340d04b0 0x7fbd340b3ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:51:07.872 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:07.868+0000 7fbd3259c640 1 -- 172.21.15.43:0/660220546 >> v1:172.21.15.43:6789/0 conn(0x7fbd340b4420 legacy=0x7fbd3406a760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:51:07.872 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:07.868+0000 7fbd3259c640 1 -- 172.21.15.43:0/660220546 --> v2:172.21.15.203:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbd3415c930 con 0x7fbd3415c550 2023-08-09T13:51:07.873 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:07.869+0000 7fbd3259c640 1 --2- 172.21.15.43:0/660220546 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fbd3415c550 0x7fbd340cd6d0 secure :-1 s=READY pgs=414 cs=0 l=1 rev1=1 crypto rx=0x7fbd24009d90 tx=0x7fbd2402bd70 comp rx=0 tx=0).ready entity=mon.1 client_cookie=3145a678e252bfd1 server_cookie=0 in_seq=0 out_seq=0 2023-08-09T13:51:07.873 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:07.870+0000 7fbd3159a640 1 -- 172.21.15.43:0/660220546 <== mon.1 v2:172.21.15.203:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbd2c001800 con 0x7fbd3415c550 2023-08-09T13:51:07.873 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:07.870+0000 7fbd3159a640 1 -- 172.21.15.43:0/660220546 <== mon.1 v2:172.21.15.203:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbd2402c8a0 con 0x7fbd3415c550 2023-08-09T13:51:07.873 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:07.870+0000 7fbd3159a640 1 -- 172.21.15.43:0/660220546 <== mon.1 v2:172.21.15.203:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbd2402cc30 con 0x7fbd3415c550 2023-08-09T13:51:07.873 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:07.870+0000 7fbd38fb5640 1 -- 172.21.15.43:0/660220546 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fbd3415c550 msgr2=0x7fbd340cd6d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:51:07.873 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:07.870+0000 7fbd38fb5640 1 --2- 172.21.15.43:0/660220546 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fbd3415c550 0x7fbd340cd6d0 secure :-1 s=READY pgs=414 cs=0 l=1 rev1=1 crypto rx=0x7fbd24009d90 tx=0x7fbd2402bd70 comp rx=0 tx=0).stop 2023-08-09T13:51:07.874 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:07.871+0000 7fbd38fb5640 1 -- 172.21.15.43:0/660220546 shutdown_connections 2023-08-09T13:51:07.874 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:07.871+0000 7fbd38fb5640 1 --2- 172.21.15.43:0/660220546 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fbd340d04b0 0x7fbd340b3ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:51:07.874 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:07.871+0000 7fbd38fb5640 1 --2- 172.21.15.43:0/660220546 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fbd3415c550 0x7fbd340cd6d0 unknown :-1 s=CLOSED pgs=414 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:51:07.875 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:07.871+0000 7fbd38fb5640 1 -- 172.21.15.43:0/660220546 >> 172.21.15.43:0/660220546 conn(0x7fbd34071520 msgr2=0x7fbd340ceee0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-09T13:51:07.875 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:07.871+0000 7fbd38fb5640 1 -- 172.21.15.43:0/660220546 shutdown_connections 2023-08-09T13:51:07.875 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:07.871+0000 7fbd38fb5640 1 -- 172.21.15.43:0/660220546 wait complete. 2023-08-09T13:51:07.875 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:07.872+0000 7fbd38fb5640 1 Processor -- start 2023-08-09T13:51:07.876 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:07.873+0000 7fbd38fb5640 1 -- start start 2023-08-09T13:51:07.876 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:07.874+0000 7fbd38fb5640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fbd340b4420 0x7fbd3415e350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:51:07.877 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:07.874+0000 7fbd38fb5640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fbd340d04b0 0x7fbd3415a8e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:51:07.877 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:07.874+0000 7fbd38fb5640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fbd3415c550 0x7fbd34155ba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:51:07.877 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:07.874+0000 7fbd38fb5640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbd340a8f60 con 0x7fbd340d04b0 2023-08-09T13:51:07.879 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:07.874+0000 7fbd38fb5640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbd340a9240 con 0x7fbd340b4420 2023-08-09T13:51:07.879 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:07.874+0000 7fbd38fb5640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbd340a9520 con 0x7fbd3415c550 2023-08-09T13:51:07.879 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:07.874+0000 7fbd31d9b640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fbd340d04b0 0x7fbd3415a8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:51:07.879 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:07.874+0000 7fbd31d9b640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fbd340d04b0 0x7fbd3415a8e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:36520/0 (socket says 172.21.15.43:36520) 2023-08-09T13:51:07.879 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:07.874+0000 7fbd31d9b640 1 -- 172.21.15.43:0/3956538380 learned_addr learned my addr 172.21.15.43:0/3956538380 (peer_addr_for_me v2:172.21.15.43:0/0) 2023-08-09T13:51:07.881 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:07.874+0000 7fbd32d9d640 1 --2- 172.21.15.43:0/3956538380 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fbd3415c550 0x7fbd34155ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:51:07.881 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:07.874+0000 7fbd31d9b640 1 -- 172.21.15.43:0/3956538380 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fbd3415c550 msgr2=0x7fbd34155ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:51:07.881 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:07.874+0000 7fbd3259c640 1 --2- 172.21.15.43:0/3956538380 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fbd340b4420 0x7fbd3415e350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:51:07.881 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:07.875+0000 7fbd31d9b640 1 --2- 172.21.15.43:0/3956538380 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fbd3415c550 0x7fbd34155ba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:51:07.881 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:07.875+0000 7fbd31d9b640 1 -- 172.21.15.43:0/3956538380 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fbd340b4420 msgr2=0x7fbd3415e350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:51:07.881 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:07.875+0000 7fbd31d9b640 1 --2- 172.21.15.43:0/3956538380 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fbd340b4420 0x7fbd3415e350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:51:07.882 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:07.875+0000 7fbd31d9b640 1 -- 172.21.15.43:0/3956538380 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbd24009a70 con 0x7fbd340d04b0 2023-08-09T13:51:07.882 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:07.875+0000 7fbd32d9d640 1 --2- 172.21.15.43:0/3956538380 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fbd3415c550 0x7fbd34155ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-08-09T13:51:07.882 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:07.875+0000 7fbd3259c640 1 --2- 172.21.15.43:0/3956538380 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fbd340b4420 0x7fbd3415e350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-08-09T13:51:07.882 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:07.875+0000 7fbd31d9b640 1 --2- 172.21.15.43:0/3956538380 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fbd340d04b0 0x7fbd3415a8e0 secure :-1 s=READY pgs=1137 cs=0 l=1 rev1=1 crypto rx=0x7fbd1c00e9b0 tx=0x7fbd1c00ee70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-09T13:51:07.882 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:07.875+0000 7fbd2affd640 1 -- 172.21.15.43:0/3956538380 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbd1c009670 con 0x7fbd340d04b0 2023-08-09T13:51:07.882 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:07.875+0000 7fbd2affd640 1 -- 172.21.15.43:0/3956538380 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbd1c00c9a0 con 0x7fbd340d04b0 2023-08-09T13:51:07.882 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:07.875+0000 7fbd2affd640 1 -- 172.21.15.43:0/3956538380 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbd1c00cd50 con 0x7fbd340d04b0 2023-08-09T13:51:07.883 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:07.875+0000 7fbd38fb5640 1 -- 172.21.15.43:0/3956538380 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbd340a9800 con 0x7fbd340d04b0 2023-08-09T13:51:07.883 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:07.876+0000 7fbd38fb5640 1 -- 172.21.15.43:0/3956538380 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbd340ac850 con 0x7fbd340d04b0 2023-08-09T13:51:07.883 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:07.877+0000 7fbd38fb5640 1 -- 172.21.15.43:0/3956538380 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbd340739f0 con 0x7fbd340d04b0 2023-08-09T13:51:07.883 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:07.878+0000 7fbd2affd640 1 -- 172.21.15.43:0/3956538380 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 37) v1 ==== 88443+0+0 (secure 0 0 0) 0x7fbd1c013470 con 0x7fbd340d04b0 2023-08-09T13:51:07.883 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:07.878+0000 7fbd2affd640 1 --2- 172.21.15.43:0/3956538380 >> [v2:172.21.15.203:6832/1778614695,v1:172.21.15.203:6833/1778614695] conn(0x7fbd1406a170 0x7fbd1406c630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:51:07.883 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:07.879+0000 7fbd2affd640 1 -- 172.21.15.43:0/3956538380 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(95..95 src has 1..95) v4 ==== 6348+0+0 (secure 0 0 0) 0x7fbd1c0889c0 con 0x7fbd340d04b0 2023-08-09T13:51:07.884 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:07.879+0000 7fbd3259c640 1 --2- 172.21.15.43:0/3956538380 >> [v2:172.21.15.203:6832/1778614695,v1:172.21.15.203:6833/1778614695] conn(0x7fbd1406a170 0x7fbd1406c630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:51:07.884 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:07.880+0000 7fbd3259c640 1 --2- 172.21.15.43:0/3956538380 >> [v2:172.21.15.203:6832/1778614695,v1:172.21.15.203:6833/1778614695] conn(0x7fbd1406a170 0x7fbd1406c630 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7fbd24036000 tx=0x7fbd24030000 comp rx=0 tx=0).ready entity=mgr.5819 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-09T13:51:07.885 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:07.883+0000 7fbd2affd640 1 -- 172.21.15.43:0/3956538380 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178066 (secure 0 0 0) 0x7fbd1c053f10 con 0x7fbd340d04b0 2023-08-09T13:51:08.078 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:08.075+0000 7fbd38fb5640 1 -- 172.21.15.43:0/3956538380 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"} v 0) v1 -- 0x7fbd340b65f0 con 0x7fbd340d04b0 2023-08-09T13:51:08.079 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:08.076+0000 7fbd2affd640 1 -- 172.21.15.43:0/3956538380 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"}]=0 v95) v1 ==== 92+0+10 (secure 0 0 0) 0x7fbd1c057530 con 0x7fbd340d04b0 2023-08-09T13:51:08.079 INFO:teuthology.orchestra.run.smithi043.stdout:pg_num: 8 2023-08-09T13:51:08.082 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:08.079+0000 7fbd38fb5640 1 -- 172.21.15.43:0/3956538380 >> [v2:172.21.15.203:6832/1778614695,v1:172.21.15.203:6833/1778614695] conn(0x7fbd1406a170 msgr2=0x7fbd1406c630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:51:08.082 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:08.079+0000 7fbd38fb5640 1 --2- 172.21.15.43:0/3956538380 >> [v2:172.21.15.203:6832/1778614695,v1:172.21.15.203:6833/1778614695] conn(0x7fbd1406a170 0x7fbd1406c630 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7fbd24036000 tx=0x7fbd24030000 comp rx=0 tx=0).stop 2023-08-09T13:51:08.082 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:08.079+0000 7fbd38fb5640 1 -- 172.21.15.43:0/3956538380 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fbd340d04b0 msgr2=0x7fbd3415a8e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:51:08.082 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:08.079+0000 7fbd38fb5640 1 --2- 172.21.15.43:0/3956538380 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fbd340d04b0 0x7fbd3415a8e0 secure :-1 s=READY pgs=1137 cs=0 l=1 rev1=1 crypto rx=0x7fbd1c00e9b0 tx=0x7fbd1c00ee70 comp rx=0 tx=0).stop 2023-08-09T13:51:08.083 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:08.079+0000 7fbd38fb5640 1 -- 172.21.15.43:0/3956538380 shutdown_connections 2023-08-09T13:51:08.083 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:08.079+0000 7fbd38fb5640 1 --2- 172.21.15.43:0/3956538380 >> [v2:172.21.15.203:6832/1778614695,v1:172.21.15.203:6833/1778614695] conn(0x7fbd1406a170 0x7fbd1406c630 unknown :-1 s=CLOSED pgs=41 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:51:08.083 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:08.079+0000 7fbd38fb5640 1 --2- 172.21.15.43:0/3956538380 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fbd3415c550 0x7fbd34155ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:51:08.083 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:08.079+0000 7fbd38fb5640 1 --2- 172.21.15.43:0/3956538380 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fbd340d04b0 0x7fbd3415a8e0 unknown :-1 s=CLOSED pgs=1137 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:51:08.083 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:08.079+0000 7fbd38fb5640 1 --2- 172.21.15.43:0/3956538380 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fbd340b4420 0x7fbd3415e350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:51:08.083 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:08.079+0000 7fbd38fb5640 1 -- 172.21.15.43:0/3956538380 >> 172.21.15.43:0/3956538380 conn(0x7fbd34071520 msgr2=0x7fbd3415ced0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-09T13:51:08.083 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:08.080+0000 7fbd38fb5640 1 -- 172.21.15.43:0/3956538380 shutdown_connections 2023-08-09T13:51:08.084 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:08.080+0000 7fbd38fb5640 1 -- 172.21.15.43:0/3956538380 wait complete. 2023-08-09T13:51:08.100 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-08-09T13:51:08.314 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:08.311+0000 7f2d1db74640 1 Processor -- start 2023-08-09T13:51:08.315 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:08.312+0000 7f2d1db74640 1 -- start start 2023-08-09T13:51:08.316 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:08.313+0000 7f2d1db74640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f2d180b7df0 0x7f2d180d1b30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:51:08.316 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:08.313+0000 7f2d1db74640 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7f2d180cebf0 0x7f2d180cf060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:51:08.316 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:08.313+0000 7f2d1db74640 1 -- --> v1:172.21.15.43:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2d180d16a0 con 0x7f2d1815c2d0 2023-08-09T13:51:08.316 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:08.313+0000 7f2d1db74640 1 -- --> v2:172.21.15.203:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2d180b8230 con 0x7f2d180cebf0 2023-08-09T13:51:08.317 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:08.313+0000 7f2d1db74640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2d180cf5a0 con 0x7f2d180b7df0 2023-08-09T13:51:08.317 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:08.314+0000 7f2d1cb72640 1 --1- >> v1:172.21.15.43:6789/0 conn(0x7f2d1815c2d0 0x7f2d180cd020 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.43:6789/0 says I am v1:172.21.15.43:45684/0 (socket says 172.21.15.43:45684) 2023-08-09T13:51:08.317 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:08.314+0000 7f2d1cb72640 1 -- 172.21.15.43:0/1892542025 learned_addr learned my addr 172.21.15.43:0/1892542025 (peer_addr_for_me v1:172.21.15.43:0/0) 2023-08-09T13:51:08.320 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:08.314+0000 7f2d1d373640 1 --2- 172.21.15.43:0/1892542025 >> v2:172.21.15.203:3300/0 conn(0x7f2d180cebf0 0x7f2d180cf060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:51:08.321 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:08.314+0000 7f2d17fff640 1 --2- 172.21.15.43:0/1892542025 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f2d180b7df0 0x7f2d180d1b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:51:08.321 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:08.314+0000 7f2d177fe640 1 -- 172.21.15.43:0/1892542025 <== mon.0 v1:172.21.15.43:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4093135214 0 0) 0x7f2d180d16a0 con 0x7f2d1815c2d0 2023-08-09T13:51:08.321 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:08.314+0000 7f2d177fe640 1 -- 172.21.15.43:0/1892542025 --> v1:172.21.15.43:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f2d08003520 con 0x7f2d1815c2d0 2023-08-09T13:51:08.321 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:08.315+0000 7f2d177fe640 1 -- 172.21.15.43:0/1892542025 <== mon.0 v1:172.21.15.43:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 810297145 0 0) 0x7f2d08003520 con 0x7f2d1815c2d0 2023-08-09T13:51:08.322 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:08.315+0000 7f2d177fe640 1 -- 172.21.15.43:0/1892542025 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f2d180b7df0 msgr2=0x7f2d180d1b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-09T13:51:08.322 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:08.315+0000 7f2d177fe640 1 --2- 172.21.15.43:0/1892542025 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f2d180b7df0 0x7f2d180d1b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:51:08.322 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:08.315+0000 7f2d177fe640 1 -- 172.21.15.43:0/1892542025 >> v2:172.21.15.203:3300/0 conn(0x7f2d180cebf0 msgr2=0x7f2d180cf060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-09T13:51:08.322 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:08.315+0000 7f2d177fe640 1 --2- 172.21.15.43:0/1892542025 >> v2:172.21.15.203:3300/0 conn(0x7f2d180cebf0 0x7f2d180cf060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:51:08.322 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:08.315+0000 7f2d177fe640 1 -- 172.21.15.43:0/1892542025 --> v1:172.21.15.43:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2d180d0d70 con 0x7f2d1815c2d0 2023-08-09T13:51:08.322 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:08.315+0000 7f2d177fe640 1 -- 172.21.15.43:0/1892542025 <== mon.0 v1:172.21.15.43:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1285150740 0 0) 0x7f2d0c001670 con 0x7f2d1815c2d0 2023-08-09T13:51:08.323 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:08.315+0000 7f2d177fe640 1 -- 172.21.15.43:0/1892542025 >> v1:172.21.15.43:6789/0 conn(0x7f2d1815c2d0 legacy=0x7f2d180cd020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:51:08.323 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:08.315+0000 7f2d177fe640 1 --2- 172.21.15.43:0/1892542025 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f2d08003970 0x7f2d08003d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:51:08.323 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:08.315+0000 7f2d177fe640 1 --2- 172.21.15.43:0/1892542025 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f2d180b7df0 0x7f2d080047c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:51:08.323 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:08.315+0000 7f2d177fe640 1 --2- 172.21.15.43:0/1892542025 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f2d08004d00 0x7f2d08005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:51:08.323 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:08.315+0000 7f2d177fe640 1 -- 172.21.15.43:0/1892542025 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2d180cf5a0 con 0x7f2d08004d00 2023-08-09T13:51:08.324 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:08.315+0000 7f2d177fe640 1 -- 172.21.15.43:0/1892542025 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2d0800b980 con 0x7f2d08003970 2023-08-09T13:51:08.324 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:08.315+0000 7f2d177fe640 1 -- 172.21.15.43:0/1892542025 --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2d0800bac0 con 0x7f2d180b7df0 2023-08-09T13:51:08.324 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:08.315+0000 7f2d1d373640 1 --2- 172.21.15.43:0/1892542025 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f2d08004d00 0x7f2d08005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:51:08.324 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:08.315+0000 7f2d1cb72640 1 --2- 172.21.15.43:0/1892542025 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f2d08003970 0x7f2d08003d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:51:08.324 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:08.315+0000 7f2d17fff640 1 --2- 172.21.15.43:0/1892542025 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f2d180b7df0 0x7f2d080047c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:51:08.324 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:08.316+0000 7f2d1d373640 1 -- 172.21.15.43:0/1892542025 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f2d180b7df0 msgr2=0x7f2d080047c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-09T13:51:08.325 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:08.316+0000 7f2d1d373640 1 --2- 172.21.15.43:0/1892542025 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f2d180b7df0 0x7f2d080047c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:51:08.325 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:08.316+0000 7f2d1d373640 1 -- 172.21.15.43:0/1892542025 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f2d08003970 msgr2=0x7f2d08003d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-09T13:51:08.325 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:08.316+0000 7f2d1d373640 1 --2- 172.21.15.43:0/1892542025 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f2d08003970 0x7f2d08003d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:51:08.325 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:08.316+0000 7f2d1d373640 1 -- 172.21.15.43:0/1892542025 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2d08003520 con 0x7f2d08004d00 2023-08-09T13:51:08.325 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:08.316+0000 7f2d17fff640 1 --2- 172.21.15.43:0/1892542025 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f2d180b7df0 0x7f2d080047c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-08-09T13:51:08.326 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:08.317+0000 7f2d1d373640 1 --2- 172.21.15.43:0/1892542025 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f2d08004d00 0x7f2d08005180 secure :-1 s=READY pgs=1139 cs=0 l=1 rev1=1 crypto rx=0x7f2d10009fc0 tx=0x7f2d080057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f2a05d0631fb0fa server_cookie=0 in_seq=0 out_seq=0 2023-08-09T13:51:08.326 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:08.317+0000 7f2d177fe640 1 -- 172.21.15.43:0/1892542025 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2d1002cc30 con 0x7f2d08004d00 2023-08-09T13:51:08.326 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:08.317+0000 7f2d177fe640 1 -- 172.21.15.43:0/1892542025 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2d1000f040 con 0x7f2d08004d00 2023-08-09T13:51:08.326 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:08.317+0000 7f2d177fe640 1 -- 172.21.15.43:0/1892542025 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2d10034440 con 0x7f2d08004d00 2023-08-09T13:51:08.326 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:08.318+0000 7f2d1db74640 1 -- 172.21.15.43:0/1892542025 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f2d08004d00 msgr2=0x7f2d08005180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:51:08.326 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:08.318+0000 7f2d1db74640 1 --2- 172.21.15.43:0/1892542025 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f2d08004d00 0x7f2d08005180 secure :-1 s=READY pgs=1139 cs=0 l=1 rev1=1 crypto rx=0x7f2d10009fc0 tx=0x7f2d080057f0 comp rx=0 tx=0).stop 2023-08-09T13:51:08.326 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:08.318+0000 7f2d1d373640 1 -- 172.21.15.43:0/1892542025 reap_dead start 2023-08-09T13:51:08.327 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:08.318+0000 7f2d1db74640 1 -- 172.21.15.43:0/1892542025 shutdown_connections 2023-08-09T13:51:08.327 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:08.318+0000 7f2d1db74640 1 -- 172.21.15.43:0/1892542025 >> 172.21.15.43:0/1892542025 conn(0x7f2d1806a230 msgr2=0x7f2d1806a640 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-09T13:51:08.327 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:08.318+0000 7f2d1db74640 1 -- 172.21.15.43:0/1892542025 shutdown_connections 2023-08-09T13:51:08.327 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:08.318+0000 7f2d1db74640 1 -- 172.21.15.43:0/1892542025 wait complete. 2023-08-09T13:51:08.329 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:08.320+0000 7f2d1db74640 1 Processor -- start 2023-08-09T13:51:08.329 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:08.320+0000 7f2d1db74640 1 -- start start 2023-08-09T13:51:08.329 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:08.321+0000 7f2d1db74640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f2d180b3da0 0x7f2d18154fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:51:08.330 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:08.321+0000 7f2d1db74640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f2d180b4180 0x7f2d180b2a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:51:08.330 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:08.321+0000 7f2d1db74640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f2d180b2f60 0x7f2d180b3410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:51:08.330 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:08.321+0000 7f2d1db74640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2d181554f0 con 0x7f2d180b2f60 2023-08-09T13:51:08.330 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:08.321+0000 7f2d1db74640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2d181581c0 con 0x7f2d180b3da0 2023-08-09T13:51:08.330 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:08.321+0000 7f2d1db74640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2d181584a0 con 0x7f2d180b4180 2023-08-09T13:51:08.330 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:08.321+0000 7f2d1d373640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f2d180b2f60 0x7f2d180b3410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:51:08.330 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:08.322+0000 7f2d1d373640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f2d180b2f60 0x7f2d180b3410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:36534/0 (socket says 172.21.15.43:36534) 2023-08-09T13:51:08.331 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:08.322+0000 7f2d1d373640 1 -- 172.21.15.43:0/4271970148 learned_addr learned my addr 172.21.15.43:0/4271970148 (peer_addr_for_me v2:172.21.15.43:0/0) 2023-08-09T13:51:08.331 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:08.322+0000 7f2d1cb72640 1 --2- 172.21.15.43:0/4271970148 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f2d180b3da0 0x7f2d18154fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:51:08.331 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:08.322+0000 7f2d17fff640 1 --2- 172.21.15.43:0/4271970148 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f2d180b4180 0x7f2d180b2a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:51:08.331 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:08.322+0000 7f2d1d373640 1 -- 172.21.15.43:0/4271970148 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f2d180b4180 msgr2=0x7f2d180b2a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:51:08.331 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:08.322+0000 7f2d1d373640 1 --2- 172.21.15.43:0/4271970148 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f2d180b4180 0x7f2d180b2a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:51:08.331 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:08.322+0000 7f2d1d373640 1 -- 172.21.15.43:0/4271970148 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f2d180b3da0 msgr2=0x7f2d18154fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:51:08.332 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:08.322+0000 7f2d1d373640 1 --2- 172.21.15.43:0/4271970148 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f2d180b3da0 0x7f2d18154fb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:51:08.332 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:08.322+0000 7f2d1d373640 1 -- 172.21.15.43:0/4271970148 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2d181526c0 con 0x7f2d180b2f60 2023-08-09T13:51:08.332 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:08.322+0000 7f2d1cb72640 1 --2- 172.21.15.43:0/4271970148 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f2d180b3da0 0x7f2d18154fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-08-09T13:51:08.332 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:08.323+0000 7f2d1d373640 1 --2- 172.21.15.43:0/4271970148 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f2d180b2f60 0x7f2d180b3410 secure :-1 s=READY pgs=1140 cs=0 l=1 rev1=1 crypto rx=0x7f2d10037040 tx=0x7f2d10009750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-09T13:51:08.332 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:08.323+0000 7f2d157fa640 1 -- 172.21.15.43:0/4271970148 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2d10043070 con 0x7f2d180b2f60 2023-08-09T13:51:08.332 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:08.323+0000 7f2d157fa640 1 -- 172.21.15.43:0/4271970148 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2d10034b50 con 0x7f2d180b2f60 2023-08-09T13:51:08.333 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:08.323+0000 7f2d157fa640 1 -- 172.21.15.43:0/4271970148 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2d10003990 con 0x7f2d180b2f60 2023-08-09T13:51:08.333 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:08.323+0000 7f2d1db74640 1 -- 172.21.15.43:0/4271970148 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2d181529a0 con 0x7f2d180b2f60 2023-08-09T13:51:08.333 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:08.323+0000 7f2d1db74640 1 -- 172.21.15.43:0/4271970148 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2d18156f40 con 0x7f2d180b2f60 2023-08-09T13:51:08.333 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:08.324+0000 7f2d1db74640 1 -- 172.21.15.43:0/4271970148 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2d18152ec0 con 0x7f2d180b2f60 2023-08-09T13:51:08.333 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:08.326+0000 7f2d157fa640 1 -- 172.21.15.43:0/4271970148 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 37) v1 ==== 88443+0+0 (secure 0 0 0) 0x7f2d1003e440 con 0x7f2d180b2f60 2023-08-09T13:51:08.333 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:08.327+0000 7f2d157fa640 1 --2- 172.21.15.43:0/4271970148 >> [v2:172.21.15.203:6832/1778614695,v1:172.21.15.203:6833/1778614695] conn(0x7f2cfc06a170 0x7f2cfc06c630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:51:08.334 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:08.327+0000 7f2d157fa640 1 -- 172.21.15.43:0/4271970148 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(95..95 src has 1..95) v4 ==== 6348+0+0 (secure 0 0 0) 0x7f2d100aa1d0 con 0x7f2d180b2f60 2023-08-09T13:51:08.334 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:08.327+0000 7f2d1cb72640 1 --2- 172.21.15.43:0/4271970148 >> [v2:172.21.15.203:6832/1778614695,v1:172.21.15.203:6833/1778614695] conn(0x7f2cfc06a170 0x7f2cfc06c630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:51:08.334 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:08.329+0000 7f2d1cb72640 1 --2- 172.21.15.43:0/4271970148 >> [v2:172.21.15.203:6832/1778614695,v1:172.21.15.203:6833/1778614695] conn(0x7f2cfc06a170 0x7f2cfc06c630 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7f2d0c001eb0 tx=0x7f2d0c009040 comp rx=0 tx=0).ready entity=mgr.5819 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-09T13:51:08.335 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:08.331+0000 7f2d157fa640 1 -- 172.21.15.43:0/4271970148 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178066 (secure 0 0 0) 0x7f2d18152ec0 con 0x7f2d180b2f60 2023-08-09T13:51:08.529 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:08.527+0000 7f2d1db74640 1 -- 172.21.15.43:0/4271970148 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f2d18152ec0 con 0x7f2d180b2f60 2023-08-09T13:51:08.531 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:08.528+0000 7f2d157fa640 1 -- 172.21.15.43:0/4271970148 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v95) v1 ==== 93+0+10 (secure 0 0 0) 0x7f2d18152ec0 con 0x7f2d180b2f60 2023-08-09T13:51:08.531 INFO:teuthology.orchestra.run.smithi043.stdout:pg_num: 1 2023-08-09T13:51:08.534 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:08.531+0000 7f2d1db74640 1 -- 172.21.15.43:0/4271970148 >> [v2:172.21.15.203:6832/1778614695,v1:172.21.15.203:6833/1778614695] conn(0x7f2cfc06a170 msgr2=0x7f2cfc06c630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:51:08.534 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:08.531+0000 7f2d1db74640 1 --2- 172.21.15.43:0/4271970148 >> [v2:172.21.15.203:6832/1778614695,v1:172.21.15.203:6833/1778614695] conn(0x7f2cfc06a170 0x7f2cfc06c630 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7f2d0c001eb0 tx=0x7f2d0c009040 comp rx=0 tx=0).stop 2023-08-09T13:51:08.535 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:08.531+0000 7f2d1db74640 1 -- 172.21.15.43:0/4271970148 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f2d180b2f60 msgr2=0x7f2d180b3410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:51:08.535 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:08.531+0000 7f2d1db74640 1 --2- 172.21.15.43:0/4271970148 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f2d180b2f60 0x7f2d180b3410 secure :-1 s=READY pgs=1140 cs=0 l=1 rev1=1 crypto rx=0x7f2d10037040 tx=0x7f2d10009750 comp rx=0 tx=0).stop 2023-08-09T13:51:08.535 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:08.531+0000 7f2d1db74640 1 -- 172.21.15.43:0/4271970148 shutdown_connections 2023-08-09T13:51:08.536 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:08.531+0000 7f2d1db74640 1 --2- 172.21.15.43:0/4271970148 >> [v2:172.21.15.203:6832/1778614695,v1:172.21.15.203:6833/1778614695] conn(0x7f2cfc06a170 0x7f2cfc06c630 unknown :-1 s=CLOSED pgs=42 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:51:08.536 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:08.531+0000 7f2d1db74640 1 --2- 172.21.15.43:0/4271970148 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f2d180b2f60 0x7f2d180b3410 unknown :-1 s=CLOSED pgs=1140 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:51:08.536 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:08.531+0000 7f2d1db74640 1 --2- 172.21.15.43:0/4271970148 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f2d180b4180 0x7f2d180b2a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:51:08.537 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:08.531+0000 7f2d1db74640 1 --2- 172.21.15.43:0/4271970148 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f2d180b3da0 0x7f2d18154fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:51:08.537 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:08.531+0000 7f2d1db74640 1 -- 172.21.15.43:0/4271970148 >> 172.21.15.43:0/4271970148 conn(0x7f2d1806a230 msgr2=0x7f2d1815d290 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-09T13:51:08.537 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:08.531+0000 7f2d1db74640 1 -- 172.21.15.43:0/4271970148 shutdown_connections 2023-08-09T13:51:08.537 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:08.532+0000 7f2d1db74640 1 -- 172.21.15.43:0/4271970148 wait complete. 2023-08-09T13:51:08.553 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mon mon_allow_pool_delete True 2023-08-09T13:51:08.781 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:08.778+0000 7fe526367640 1 Processor -- start 2023-08-09T13:51:08.781 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:08.779+0000 7fe526367640 1 -- start start 2023-08-09T13:51:08.783 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:08.780+0000 7fe526367640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7fe520157ce0 0x7fe52015b0f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:51:08.783 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:08.780+0000 7fe526367640 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7fe5200c8470 0x7fe5200c8880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:51:08.783 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:08.780+0000 7fe526367640 1 -- --> v1:172.21.15.203:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe5200c8dc0 con 0x7fe52006f2c0 2023-08-09T13:51:08.783 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:08.780+0000 7fe526367640 1 -- --> v2:172.21.15.43:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe5201580c0 con 0x7fe520157ce0 2023-08-09T13:51:08.783 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:08.780+0000 7fe526367640 1 -- --> v2:172.21.15.203:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe5200c92c0 con 0x7fe5200c8470 2023-08-09T13:51:08.784 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:08.781+0000 7fe525365640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7fe520157ce0 0x7fe52015b0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:51:08.784 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:08.781+0000 7fe525365640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7fe520157ce0 0x7fe52015b0f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:36548/0 (socket says 172.21.15.43:36548) 2023-08-09T13:51:08.784 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:08.781+0000 7fe525365640 1 -- 172.21.15.43:0/83889847 learned_addr learned my addr 172.21.15.43:0/83889847 (peer_addr_for_me v2:172.21.15.43:0/0) 2023-08-09T13:51:08.784 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:08.781+0000 7fe525365640 1 -- 172.21.15.43:0/83889847 >> v2:172.21.15.203:3300/0 conn(0x7fe5200c8470 msgr2=0x7fe5200c8880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-09T13:51:08.784 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:08.781+0000 7fe525365640 1 --2- 172.21.15.43:0/83889847 >> v2:172.21.15.203:3300/0 conn(0x7fe5200c8470 0x7fe5200c8880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:51:08.785 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:08.781+0000 7fe525365640 1 -- 172.21.15.43:0/83889847 >> v1:172.21.15.203:6789/0 conn(0x7fe52006f2c0 legacy=0x7fe52006bc40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-09T13:51:08.785 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:08.781+0000 7fe525365640 1 -- 172.21.15.43:0/83889847 --> v2:172.21.15.43:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe52006b480 con 0x7fe520157ce0 2023-08-09T13:51:08.785 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:08.782+0000 7fe525365640 1 --2- 172.21.15.43:0/83889847 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fe520157ce0 0x7fe52015b0f0 secure :-1 s=READY pgs=1141 cs=0 l=1 rev1=1 crypto rx=0x7fe5200c9400 tx=0x7fe51c02b880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=caf24957a5945a74 server_cookie=0 in_seq=0 out_seq=0 2023-08-09T13:51:08.786 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:08.783+0000 7fe517fff640 1 -- 172.21.15.43:0/83889847 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe51c0317e0 con 0x7fe520157ce0 2023-08-09T13:51:08.786 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:08.783+0000 7fe517fff640 1 -- 172.21.15.43:0/83889847 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe51c031940 con 0x7fe520157ce0 2023-08-09T13:51:08.786 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:08.783+0000 7fe517fff640 1 -- 172.21.15.43:0/83889847 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe51c031cd0 con 0x7fe520157ce0 2023-08-09T13:51:08.786 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:08.783+0000 7fe526367640 1 -- 172.21.15.43:0/83889847 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fe520157ce0 msgr2=0x7fe52015b0f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:51:08.786 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:08.783+0000 7fe526367640 1 --2- 172.21.15.43:0/83889847 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fe520157ce0 0x7fe52015b0f0 secure :-1 s=READY pgs=1141 cs=0 l=1 rev1=1 crypto rx=0x7fe5200c9400 tx=0x7fe51c02b880 comp rx=0 tx=0).stop 2023-08-09T13:51:08.787 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:08.784+0000 7fe526367640 1 -- 172.21.15.43:0/83889847 shutdown_connections 2023-08-09T13:51:08.787 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:08.784+0000 7fe526367640 1 --2- 172.21.15.43:0/83889847 >> v2:172.21.15.203:3300/0 conn(0x7fe5200c8470 0x7fe5200c8880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:51:08.787 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:08.784+0000 7fe526367640 1 --2- 172.21.15.43:0/83889847 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fe520157ce0 0x7fe52015b0f0 unknown :-1 s=CLOSED pgs=1141 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:51:08.787 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:08.784+0000 7fe526367640 1 -- 172.21.15.43:0/83889847 >> 172.21.15.43:0/83889847 conn(0x7fe5200af9f0 msgr2=0x7fe5200afdd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-09T13:51:08.787 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:08.784+0000 7fe526367640 1 -- 172.21.15.43:0/83889847 shutdown_connections 2023-08-09T13:51:08.787 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:08.784+0000 7fe526367640 1 -- 172.21.15.43:0/83889847 wait complete. 2023-08-09T13:51:08.788 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:08.785+0000 7fe526367640 1 Processor -- start 2023-08-09T13:51:08.788 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:08.785+0000 7fe526367640 1 -- start start 2023-08-09T13:51:08.789 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:08.786+0000 7fe526367640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fe52006f2c0 0x7fe5201ed1d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:51:08.789 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:08.786+0000 7fe526367640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fe5200c8470 0x7fe5200c4120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:51:08.789 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:08.786+0000 7fe526367640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fe520157ce0 0x7fe5200c54c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:51:08.789 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:08.786+0000 7fe526367640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe5200c3080 con 0x7fe52006f2c0 2023-08-09T13:51:08.789 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:08.786+0000 7fe526367640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe5200c3360 con 0x7fe520157ce0 2023-08-09T13:51:08.790 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:08.786+0000 7fe526367640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe5200c3640 con 0x7fe5200c8470 2023-08-09T13:51:08.790 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:08.787+0000 7fe525365640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fe52006f2c0 0x7fe5201ed1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:51:08.790 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:08.787+0000 7fe525365640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fe52006f2c0 0x7fe5201ed1d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:36562/0 (socket says 172.21.15.43:36562) 2023-08-09T13:51:08.790 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:08.787+0000 7fe525365640 1 -- 172.21.15.43:0/2920827727 learned_addr learned my addr 172.21.15.43:0/2920827727 (peer_addr_for_me v2:172.21.15.43:0/0) 2023-08-09T13:51:08.791 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:08.787+0000 7fe525365640 1 -- 172.21.15.43:0/2920827727 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fe5200c8470 msgr2=0x7fe5200c4120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:51:08.791 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:08.787+0000 7fe524b64640 1 --2- 172.21.15.43:0/2920827727 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fe5200c8470 0x7fe5200c4120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:51:08.791 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:08.787+0000 7fe525b66640 1 --2- 172.21.15.43:0/2920827727 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fe520157ce0 0x7fe5200c54c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:51:08.792 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:08.787+0000 7fe525365640 1 --2- 172.21.15.43:0/2920827727 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fe5200c8470 0x7fe5200c4120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:51:08.792 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:08.787+0000 7fe525365640 1 -- 172.21.15.43:0/2920827727 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fe520157ce0 msgr2=0x7fe5200c54c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:51:08.792 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:08.787+0000 7fe525365640 1 --2- 172.21.15.43:0/2920827727 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fe520157ce0 0x7fe5200c54c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:51:08.792 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:08.787+0000 7fe525365640 1 -- 172.21.15.43:0/2920827727 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe51c0094d0 con 0x7fe52006f2c0 2023-08-09T13:51:08.792 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:08.787+0000 7fe524b64640 1 --2- 172.21.15.43:0/2920827727 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fe5200c8470 0x7fe5200c4120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-08-09T13:51:08.792 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:08.787+0000 7fe525b66640 1 --2- 172.21.15.43:0/2920827727 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fe520157ce0 0x7fe5200c54c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-08-09T13:51:08.792 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:08.788+0000 7fe525365640 1 --2- 172.21.15.43:0/2920827727 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fe52006f2c0 0x7fe5201ed1d0 secure :-1 s=READY pgs=1142 cs=0 l=1 rev1=1 crypto rx=0x7fe51c000c00 tx=0x7fe51c0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-09T13:51:08.792 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:08.788+0000 7fe515ffb640 1 -- 172.21.15.43:0/2920827727 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe51c032880 con 0x7fe52006f2c0 2023-08-09T13:51:08.793 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:08.788+0000 7fe515ffb640 1 -- 172.21.15.43:0/2920827727 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe51c0329e0 con 0x7fe52006f2c0 2023-08-09T13:51:08.793 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:08.788+0000 7fe515ffb640 1 -- 172.21.15.43:0/2920827727 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe51c032d90 con 0x7fe52006f2c0 2023-08-09T13:51:08.793 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:08.788+0000 7fe526367640 1 -- 172.21.15.43:0/2920827727 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe5200c3920 con 0x7fe52006f2c0 2023-08-09T13:51:08.793 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:08.788+0000 7fe526367640 1 -- 172.21.15.43:0/2920827727 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe5200c4660 con 0x7fe52006f2c0 2023-08-09T13:51:08.794 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:08.789+0000 7fe526367640 1 -- 172.21.15.43:0/2920827727 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe5200c3e40 con 0x7fe52006f2c0 2023-08-09T13:51:08.794 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:08.790+0000 7fe515ffb640 1 -- 172.21.15.43:0/2920827727 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 37) v1 ==== 88443+0+0 (secure 0 0 0) 0x7fe51c04d3f0 con 0x7fe52006f2c0 2023-08-09T13:51:08.794 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:08.791+0000 7fe515ffb640 1 --2- 172.21.15.43:0/2920827727 >> [v2:172.21.15.203:6832/1778614695,v1:172.21.15.203:6833/1778614695] conn(0x7fe50406a170 0x7fe50406c630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:51:08.794 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:08.791+0000 7fe515ffb640 1 -- 172.21.15.43:0/2920827727 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(95..95 src has 1..95) v4 ==== 6348+0+0 (secure 0 0 0) 0x7fe51c0aa860 con 0x7fe52006f2c0 2023-08-09T13:51:08.794 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:08.792+0000 7fe524b64640 1 --2- 172.21.15.43:0/2920827727 >> [v2:172.21.15.203:6832/1778614695,v1:172.21.15.203:6833/1778614695] conn(0x7fe50406a170 0x7fe50406c630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:51:08.795 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:08.793+0000 7fe524b64640 1 --2- 172.21.15.43:0/2920827727 >> [v2:172.21.15.203:6832/1778614695,v1:172.21.15.203:6833/1778614695] conn(0x7fe50406a170 0x7fe50406c630 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7fe5200c5030 tx=0x7fe510008040 comp rx=0 tx=0).ready entity=mgr.5819 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-09T13:51:08.797 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:08.795+0000 7fe515ffb640 1 -- 172.21.15.43:0/2920827727 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178066 (secure 0 0 0) 0x7fe5200c3e40 con 0x7fe52006f2c0 2023-08-09T13:51:08.981 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:08.978+0000 7fe526367640 1 -- 172.21.15.43:0/2920827727 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command([{prefix=config set, name=mon_allow_pool_delete}] v 0) v1 -- 0x7fe5200c3e40 con 0x7fe52006f2c0 2023-08-09T13:51:08.988 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:08.986+0000 7fe515ffb640 1 -- 172.21.15.43:0/2920827727 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_allow_pool_delete}]=0 v28)=0 v28) v1 ==== 122+0+0 (secure 0 0 0) 0x7fe51c075c10 con 0x7fe52006f2c0 2023-08-09T13:51:08.991 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:08.988+0000 7fe526367640 1 -- 172.21.15.43:0/2920827727 >> [v2:172.21.15.203:6832/1778614695,v1:172.21.15.203:6833/1778614695] conn(0x7fe50406a170 msgr2=0x7fe50406c630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:51:08.991 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:08.988+0000 7fe526367640 1 --2- 172.21.15.43:0/2920827727 >> [v2:172.21.15.203:6832/1778614695,v1:172.21.15.203:6833/1778614695] conn(0x7fe50406a170 0x7fe50406c630 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7fe5200c5030 tx=0x7fe510008040 comp rx=0 tx=0).stop 2023-08-09T13:51:08.991 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:08.988+0000 7fe526367640 1 -- 172.21.15.43:0/2920827727 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fe52006f2c0 msgr2=0x7fe5201ed1d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:51:08.991 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:08.988+0000 7fe526367640 1 --2- 172.21.15.43:0/2920827727 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fe52006f2c0 0x7fe5201ed1d0 secure :-1 s=READY pgs=1142 cs=0 l=1 rev1=1 crypto rx=0x7fe51c000c00 tx=0x7fe51c0046c0 comp rx=0 tx=0).stop 2023-08-09T13:51:08.992 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:08.988+0000 7fe526367640 1 -- 172.21.15.43:0/2920827727 shutdown_connections 2023-08-09T13:51:08.992 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:08.988+0000 7fe526367640 1 --2- 172.21.15.43:0/2920827727 >> [v2:172.21.15.203:6832/1778614695,v1:172.21.15.203:6833/1778614695] conn(0x7fe50406a170 0x7fe50406c630 unknown :-1 s=CLOSED pgs=43 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:51:08.992 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:08.988+0000 7fe526367640 1 --2- 172.21.15.43:0/2920827727 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fe520157ce0 0x7fe5200c54c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:51:08.992 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:08.988+0000 7fe526367640 1 --2- 172.21.15.43:0/2920827727 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fe5200c8470 0x7fe5200c4120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:51:08.992 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:08.988+0000 7fe526367640 1 --2- 172.21.15.43:0/2920827727 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fe52006f2c0 0x7fe5201ed1d0 unknown :-1 s=CLOSED pgs=1142 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:51:08.993 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:08.988+0000 7fe526367640 1 -- 172.21.15.43:0/2920827727 >> 172.21.15.43:0/2920827727 conn(0x7fe5200af9f0 msgr2=0x7fe5200cba20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-09T13:51:08.993 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:08.989+0000 7fe526367640 1 -- 172.21.15.43:0/2920827727 shutdown_connections 2023-08-09T13:51:08.993 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:08.989+0000 7fe526367640 1 -- 172.21.15.43:0/2920827727 wait complete. 2023-08-09T13:51:09.011 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2023-08-09T13:51:09.012 DEBUG:teuthology.orchestra.run.smithi043:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2023-08-09T13:51:09.115 DEBUG:teuthology.orchestra.run.smithi203:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2023-08-09T13:51:09.238 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2023-08-09T13:51:09.239 DEBUG:teuthology.orchestra.run.smithi043:> set -ex 2023-08-09T13:51:09.239 DEBUG:teuthology.orchestra.run.smithi043:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-09T13:51:09.302 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4054441,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6498844k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=27,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3249420k,nr_inodes=812355,mode=700,uid=1000,gid=1275 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6498844k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2023-08-09T13:51:09.303 DEBUG:teuthology.orchestra.run.smithi203:> set -ex 2023-08-09T13:51:09.303 DEBUG:teuthology.orchestra.run.smithi203:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-09T13:51:09.365 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4053648,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497576k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=27,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248784k,nr_inodes=812196,mode=700,uid=1000,gid=1275 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497576k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2023-08-09T13:51:09.365 DEBUG:teuthology.orchestra.run.smithi043:> set -ex 2023-08-09T13:51:09.365 DEBUG:teuthology.orchestra.run.smithi043:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-09T13:51:09.418 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4054441,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6498844k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=27,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3249420k,nr_inodes=812355,mode=700,uid=1000,gid=1275 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6498844k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2023-08-09T13:51:09.418 DEBUG:teuthology.orchestra.run.smithi203:> set -ex 2023-08-09T13:51:09.419 DEBUG:teuthology.orchestra.run.smithi203:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-09T13:51:09.480 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4053648,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497576k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=27,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248784k,nr_inodes=812196,mode=700,uid=1000,gid=1275 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497576k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2023-08-09T13:51:09.480 INFO:tasks.cephfs.cephfs_test_case:[, , , ] 2023-08-09T13:51:09.481 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-08-09T13:51:09.716 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:09.713+0000 7f4ff0f84640 1 Processor -- start 2023-08-09T13:51:09.716 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:09.713+0000 7f4ff0f84640 1 -- start start 2023-08-09T13:51:09.717 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:09.715+0000 7f4ff0f84640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f4fec0b2b70 0x7f4fec0d2ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:51:09.717 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:09.715+0000 7f4ff0f84640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f4fec0b3050 0x7f4fec0b3f60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:51:09.717 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:09.715+0000 7f4ff0f84640 1 -- --> v1:172.21.15.43:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4fec0ce090 con 0x7f4fec0b2790 2023-08-09T13:51:09.718 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:09.715+0000 7f4ff0f84640 1 -- --> v2:172.21.15.43:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4fec0b3950 con 0x7f4fec0b3050 2023-08-09T13:51:09.718 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:09.715+0000 7f4ff0f84640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4fec0cd7f0 con 0x7f4fec0b2b70 2023-08-09T13:51:09.719 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:09.715+0000 7f4fea59c640 1 --1- >> v1:172.21.15.43:6789/0 conn(0x7f4fec0b2790 0x7f4fec0b5330 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.43:6789/0 says I am v1:172.21.15.43:45696/0 (socket says 172.21.15.43:45696) 2023-08-09T13:51:09.719 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:09.715+0000 7f4fea59c640 1 -- 172.21.15.43:0/1644475532 learned_addr learned my addr 172.21.15.43:0/1644475532 (peer_addr_for_me v1:172.21.15.43:0/0) 2023-08-09T13:51:09.719 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:09.715+0000 7f4fead9d640 1 --2- 172.21.15.43:0/1644475532 >> v2:172.21.15.43:3300/0 conn(0x7f4fec0b3050 0x7f4fec0b3f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:51:09.719 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:09.716+0000 7f4fe9d9b640 1 --2- 172.21.15.43:0/1644475532 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f4fec0b2b70 0x7f4fec0d2ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:51:09.719 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:09.716+0000 7f4fead9d640 1 -- 172.21.15.43:0/1644475532 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f4fec0b2b70 msgr2=0x7f4fec0d2ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-09T13:51:09.719 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:09.716+0000 7f4fe959a640 1 -- 172.21.15.43:0/1644475532 <== mon.0 v1:172.21.15.43:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2691635485 0 0) 0x7f4fec0ce090 con 0x7f4fec0b2790 2023-08-09T13:51:09.719 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:09.716+0000 7f4fead9d640 1 --2- 172.21.15.43:0/1644475532 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f4fec0b2b70 0x7f4fec0d2ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:51:09.720 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:09.716+0000 7f4fead9d640 1 -- 172.21.15.43:0/1644475532 >> v1:172.21.15.43:6789/0 conn(0x7f4fec0b2790 legacy=0x7f4fec0b5330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:51:09.720 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:09.716+0000 7f4fead9d640 1 -- 172.21.15.43:0/1644475532 --> v2:172.21.15.43:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4fec0cdad0 con 0x7f4fec0b3050 2023-08-09T13:51:09.721 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:09.717+0000 7f4fead9d640 1 --2- 172.21.15.43:0/1644475532 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f4fec0b3050 0x7f4fec0b3f60 secure :-1 s=READY pgs=1144 cs=0 l=1 rev1=1 crypto rx=0x7f4fe4009d90 tx=0x7f4fe402bea0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7ca080bfa8493f95 server_cookie=0 in_seq=0 out_seq=0 2023-08-09T13:51:09.721 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:09.717+0000 7f4fe959a640 1 -- 172.21.15.43:0/1644475532 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4fe402cc50 con 0x7f4fec0b3050 2023-08-09T13:51:09.721 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:09.717+0000 7f4fe959a640 1 -- 172.21.15.43:0/1644475532 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4fe402cdb0 con 0x7f4fec0b3050 2023-08-09T13:51:09.721 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:09.717+0000 7f4fe959a640 1 -- 172.21.15.43:0/1644475532 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4fe4034660 con 0x7f4fec0b3050 2023-08-09T13:51:09.721 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:09.718+0000 7f4ff0f84640 1 -- 172.21.15.43:0/1644475532 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f4fec0b3050 msgr2=0x7f4fec0b3f60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:51:09.721 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:09.718+0000 7f4ff0f84640 1 --2- 172.21.15.43:0/1644475532 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f4fec0b3050 0x7f4fec0b3f60 secure :-1 s=READY pgs=1144 cs=0 l=1 rev1=1 crypto rx=0x7f4fe4009d90 tx=0x7f4fe402bea0 comp rx=0 tx=0).stop 2023-08-09T13:51:09.722 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:09.718+0000 7f4ff0f84640 1 -- 172.21.15.43:0/1644475532 shutdown_connections 2023-08-09T13:51:09.722 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:09.718+0000 7f4ff0f84640 1 --2- 172.21.15.43:0/1644475532 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f4fec0b3050 0x7f4fec0b3f60 unknown :-1 s=CLOSED pgs=1144 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:51:09.722 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:09.718+0000 7f4ff0f84640 1 --2- 172.21.15.43:0/1644475532 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f4fec0b2b70 0x7f4fec0d2ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:51:09.722 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:09.718+0000 7f4ff0f84640 1 -- 172.21.15.43:0/1644475532 >> 172.21.15.43:0/1644475532 conn(0x7f4fec09b7b0 msgr2=0x7f4fec0d24e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-09T13:51:09.722 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:09.718+0000 7f4ff0f84640 1 -- 172.21.15.43:0/1644475532 shutdown_connections 2023-08-09T13:51:09.722 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:09.718+0000 7f4ff0f84640 1 -- 172.21.15.43:0/1644475532 wait complete. 2023-08-09T13:51:09.723 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:09.720+0000 7f4ff0f84640 1 Processor -- start 2023-08-09T13:51:09.723 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:09.720+0000 7f4ff0f84640 1 -- start start 2023-08-09T13:51:09.724 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:09.721+0000 7f4ff0f84640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f4fec0b2790 0x7f4fec154870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:51:09.724 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:09.721+0000 7f4ff0f84640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f4fec0b2b70 0x7f4fec0c8260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:51:09.724 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:09.721+0000 7f4ff0f84640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f4fec0b3050 0x7f4fec0afb60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:51:09.724 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:09.721+0000 7f4ff0f84640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4fec0af0a0 con 0x7f4fec0b2b70 2023-08-09T13:51:09.724 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:09.721+0000 7f4ff0f84640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4fec0c8b60 con 0x7f4fec0b2790 2023-08-09T13:51:09.725 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:09.721+0000 7f4ff0f84640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4fec0c8e40 con 0x7f4fec0b3050 2023-08-09T13:51:09.725 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:09.722+0000 7f4fe9d9b640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f4fec0b2b70 0x7f4fec0c8260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:51:09.725 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:09.722+0000 7f4fe9d9b640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f4fec0b2b70 0x7f4fec0c8260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:36584/0 (socket says 172.21.15.43:36584) 2023-08-09T13:51:09.725 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:09.722+0000 7f4fe9d9b640 1 -- 172.21.15.43:0/1856776311 learned_addr learned my addr 172.21.15.43:0/1856776311 (peer_addr_for_me v2:172.21.15.43:0/0) 2023-08-09T13:51:09.726 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:09.722+0000 7f4fead9d640 1 --2- 172.21.15.43:0/1856776311 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f4fec0b3050 0x7f4fec0afb60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:51:09.726 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:09.722+0000 7f4fea59c640 1 --2- 172.21.15.43:0/1856776311 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f4fec0b2790 0x7f4fec154870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:51:09.726 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:09.722+0000 7f4fe9d9b640 1 -- 172.21.15.43:0/1856776311 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f4fec0b3050 msgr2=0x7f4fec0afb60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:51:09.727 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:09.722+0000 7f4fe9d9b640 1 --2- 172.21.15.43:0/1856776311 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f4fec0b3050 0x7f4fec0afb60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:51:09.727 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:09.722+0000 7f4fe9d9b640 1 -- 172.21.15.43:0/1856776311 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f4fec0b2790 msgr2=0x7f4fec154870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:51:09.727 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:09.722+0000 7f4fe9d9b640 1 --2- 172.21.15.43:0/1856776311 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f4fec0b2790 0x7f4fec154870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:51:09.727 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:09.722+0000 7f4fe9d9b640 1 -- 172.21.15.43:0/1856776311 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4fe4009a70 con 0x7f4fec0b2b70 2023-08-09T13:51:09.727 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:09.722+0000 7f4fead9d640 1 --2- 172.21.15.43:0/1856776311 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f4fec0b3050 0x7f4fec0afb60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-08-09T13:51:09.727 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:09.722+0000 7f4fea59c640 1 --2- 172.21.15.43:0/1856776311 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f4fec0b2790 0x7f4fec154870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-08-09T13:51:09.727 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:09.722+0000 7f4fe9d9b640 1 --2- 172.21.15.43:0/1856776311 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f4fec0b2b70 0x7f4fec0c8260 secure :-1 s=READY pgs=1145 cs=0 l=1 rev1=1 crypto rx=0x7f4fe000d5f0 tx=0x7f4fe000dab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-09T13:51:09.727 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:09.723+0000 7f4fdaffd640 1 -- 172.21.15.43:0/1856776311 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4fe00125f0 con 0x7f4fec0b2b70 2023-08-09T13:51:09.728 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:09.723+0000 7f4fdaffd640 1 -- 172.21.15.43:0/1856776311 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4fe0012750 con 0x7f4fec0b2b70 2023-08-09T13:51:09.728 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:09.723+0000 7f4fdaffd640 1 -- 172.21.15.43:0/1856776311 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4fe0012b00 con 0x7f4fec0b2b70 2023-08-09T13:51:09.728 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:09.723+0000 7f4ff0f84640 1 -- 172.21.15.43:0/1856776311 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4fec0af360 con 0x7f4fec0b2b70 2023-08-09T13:51:09.728 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:09.723+0000 7f4ff0f84640 1 -- 172.21.15.43:0/1856776311 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4fec154db0 con 0x7f4fec0b2b70 2023-08-09T13:51:09.729 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:09.724+0000 7f4ff0f84640 1 -- 172.21.15.43:0/1856776311 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4fec0739f0 con 0x7f4fec0b2b70 2023-08-09T13:51:09.729 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:09.725+0000 7f4fdaffd640 1 -- 172.21.15.43:0/1856776311 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 37) v1 ==== 88443+0+0 (secure 0 0 0) 0x7f4fe0012c60 con 0x7f4fec0b2b70 2023-08-09T13:51:09.729 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:09.726+0000 7f4fdaffd640 1 --2- 172.21.15.43:0/1856776311 >> [v2:172.21.15.203:6832/1778614695,v1:172.21.15.203:6833/1778614695] conn(0x7f4fcc06a1a0 0x7f4fcc06c660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:51:09.729 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:09.726+0000 7f4fdaffd640 1 -- 172.21.15.43:0/1856776311 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(95..95 src has 1..95) v4 ==== 6348+0+0 (secure 0 0 0) 0x7f4fe0088c50 con 0x7f4fec0b2b70 2023-08-09T13:51:09.729 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:09.727+0000 7f4fea59c640 1 --2- 172.21.15.43:0/1856776311 >> [v2:172.21.15.203:6832/1778614695,v1:172.21.15.203:6833/1778614695] conn(0x7f4fcc06a1a0 0x7f4fcc06c660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:51:09.730 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:09.728+0000 7f4fea59c640 1 --2- 172.21.15.43:0/1856776311 >> [v2:172.21.15.203:6832/1778614695,v1:172.21.15.203:6833/1778614695] conn(0x7f4fcc06a1a0 0x7f4fcc06c660 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7f4fdc000f60 tx=0x7f4fdc009040 comp rx=0 tx=0).ready entity=mgr.5819 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-09T13:51:09.733 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:09.730+0000 7f4fdaffd640 1 -- 172.21.15.43:0/1856776311 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178066 (secure 0 0 0) 0x7f4fe0054100 con 0x7f4fec0b2b70 2023-08-09T13:51:09.967 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:09.964+0000 7f4ff0f84640 1 -- 172.21.15.43:0/1856776311 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f4fec0b4b00 con 0x7f4fec0b2b70 2023-08-09T13:51:09.968 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:09.966+0000 7f4fdaffd640 1 -- 172.21.15.43:0/1856776311 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 37 v37) v1 ==== 94+0+3702 (secure 0 0 0) 0x7f4fe0057720 con 0x7f4fec0b2b70 2023-08-09T13:51:09.969 INFO:teuthology.orchestra.run.smithi043.stdout: 2023-08-09T13:51:09.969 INFO:teuthology.orchestra.run.smithi043.stdout:{"epoch":37,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5654,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.203:6837/1227342478","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6835","nonce":1227342478},{"type":"v1","addr":"172.21.15.203:6837","nonce":1227342478}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":30},{"gid":5924,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.203:6836/2299373191","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6834","nonce":2299373191},{"type":"v1","addr":"172.21.15.203:6836","nonce":2299373191}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":37},{"gid":7005,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.43:6834/1431309163","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6832","nonce":1431309163},{"type":"v1","addr":"172.21.15.43:6834","nonce":1431309163}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":29},{"gid":7008,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.43:6835/611328204","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6833","nonce":611328204},{"type":"v1","addr":"172.21.15.43:6835","nonce":611328204}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":30}],"filesystems":[]} 2023-08-09T13:51:09.969 INFO:teuthology.orchestra.run.smithi043.stderr:dumped fsmap epoch 37 2023-08-09T13:51:09.971 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:09.968+0000 7f4ff0f84640 1 -- 172.21.15.43:0/1856776311 >> [v2:172.21.15.203:6832/1778614695,v1:172.21.15.203:6833/1778614695] conn(0x7f4fcc06a1a0 msgr2=0x7f4fcc06c660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:51:09.971 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:09.968+0000 7f4ff0f84640 1 --2- 172.21.15.43:0/1856776311 >> [v2:172.21.15.203:6832/1778614695,v1:172.21.15.203:6833/1778614695] conn(0x7f4fcc06a1a0 0x7f4fcc06c660 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7f4fdc000f60 tx=0x7f4fdc009040 comp rx=0 tx=0).stop 2023-08-09T13:51:09.972 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:09.968+0000 7f4ff0f84640 1 -- 172.21.15.43:0/1856776311 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f4fec0b2b70 msgr2=0x7f4fec0c8260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:51:09.972 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:09.969+0000 7f4ff0f84640 1 --2- 172.21.15.43:0/1856776311 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f4fec0b2b70 0x7f4fec0c8260 secure :-1 s=READY pgs=1145 cs=0 l=1 rev1=1 crypto rx=0x7f4fe000d5f0 tx=0x7f4fe000dab0 comp rx=0 tx=0).stop 2023-08-09T13:51:09.972 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:09.969+0000 7f4ff0f84640 1 -- 172.21.15.43:0/1856776311 shutdown_connections 2023-08-09T13:51:09.972 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:09.969+0000 7f4ff0f84640 1 --2- 172.21.15.43:0/1856776311 >> [v2:172.21.15.203:6832/1778614695,v1:172.21.15.203:6833/1778614695] conn(0x7f4fcc06a1a0 0x7f4fcc06c660 unknown :-1 s=CLOSED pgs=44 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:51:09.972 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:09.969+0000 7f4ff0f84640 1 --2- 172.21.15.43:0/1856776311 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f4fec0b3050 0x7f4fec0afb60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:51:09.972 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:09.969+0000 7f4ff0f84640 1 --2- 172.21.15.43:0/1856776311 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f4fec0b2b70 0x7f4fec0c8260 unknown :-1 s=CLOSED pgs=1145 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:51:09.972 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:09.969+0000 7f4ff0f84640 1 --2- 172.21.15.43:0/1856776311 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f4fec0b2790 0x7f4fec154870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:51:09.973 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:09.969+0000 7f4ff0f84640 1 -- 172.21.15.43:0/1856776311 >> 172.21.15.43:0/1856776311 conn(0x7f4fec09b7b0 msgr2=0x7f4fec0b6110 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-09T13:51:09.973 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:09.969+0000 7f4ff0f84640 1 -- 172.21.15.43:0/1856776311 shutdown_connections 2023-08-09T13:51:09.973 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:09.969+0000 7f4ff0f84640 1 -- 172.21.15.43:0/1856776311 wait complete. 2023-08-09T13:51:09.989 INFO:tasks.ceph.mds.a:Restarting daemon 2023-08-09T13:51:09.989 INFO:tasks.ceph.mds.a:Stopping old one... 2023-08-09T13:51:09.990 DEBUG:tasks.ceph.mds.a:waiting for process to exit 2023-08-09T13:51:09.990 INFO:teuthology.orchestra.run:waiting for 300 2023-08-09T13:51:09.990 INFO:tasks.ceph.mds.c:Restarting daemon 2023-08-09T13:51:09.990 INFO:tasks.ceph.mds.c:Stopping old one... 2023-08-09T13:51:09.990 DEBUG:tasks.ceph.mds.c:waiting for process to exit 2023-08-09T13:51:09.991 INFO:teuthology.orchestra.run:waiting for 300 2023-08-09T13:51:09.991 INFO:tasks.ceph.mds.b:Restarting daemon 2023-08-09T13:51:09.991 INFO:tasks.ceph.mds.b:Stopping old one... 2023-08-09T13:51:09.991 DEBUG:tasks.ceph.mds.b:waiting for process to exit 2023-08-09T13:51:09.991 INFO:teuthology.orchestra.run:waiting for 300 2023-08-09T13:51:09.992 INFO:tasks.ceph.mds.d:Restarting daemon 2023-08-09T13:51:09.992 INFO:tasks.ceph.mds.d:Stopping old one... 2023-08-09T13:51:09.992 DEBUG:tasks.ceph.mds.d:waiting for process to exit 2023-08-09T13:51:09.992 INFO:teuthology.orchestra.run:waiting for 300 2023-08-09T13:51:10.016 INFO:tasks.ceph.mds.c:Stopped 2023-08-09T13:51:10.016 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i c 2023-08-09T13:51:10.020 INFO:tasks.ceph.mds.a:Stopped 2023-08-09T13:51:10.020 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i a 2023-08-09T13:51:10.027 INFO:tasks.ceph.mds.d:Stopped 2023-08-09T13:51:10.027 DEBUG:teuthology.orchestra.run.smithi203:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i d 2023-08-09T13:51:10.029 INFO:tasks.ceph.mds.b:Stopped 2023-08-09T13:51:10.029 DEBUG:teuthology.orchestra.run.smithi203:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i b 2023-08-09T13:51:10.068 INFO:tasks.ceph.mds.c:Started 2023-08-09T13:51:10.069 DEBUG:teuthology.parallel:result is None 2023-08-09T13:51:10.077 INFO:tasks.ceph.mds.a:Started 2023-08-09T13:51:10.077 DEBUG:teuthology.parallel:result is None 2023-08-09T13:51:10.079 INFO:tasks.ceph.mds.d:Started 2023-08-09T13:51:10.079 DEBUG:teuthology.parallel:result is None 2023-08-09T13:51:10.087 INFO:tasks.ceph.mds.b:Started 2023-08-09T13:51:10.087 DEBUG:teuthology.parallel:result is None 2023-08-09T13:51:10.087 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist clear 2023-08-09T13:51:10.238 INFO:tasks.ceph.mds.c.smithi043.stdout:starting mds.c at 2023-08-09T13:51:10.248 INFO:tasks.ceph.mds.b.smithi203.stdout:starting mds.b at 2023-08-09T13:51:10.249 INFO:tasks.ceph.mds.d.smithi203.stdout:starting mds.d at 2023-08-09T13:51:10.253 INFO:tasks.ceph.mds.a.smithi043.stdout:starting mds.a at 2023-08-09T13:51:10.294 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:10.292+0000 7f8ac2f57640 1 Processor -- start 2023-08-09T13:51:10.295 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:10.292+0000 7f8ac2f57640 1 -- start start 2023-08-09T13:51:10.299 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:10.293+0000 7f8ac2f57640 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7f8abc0d0a70 0x7f8abc0cd7f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:51:10.299 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:10.293+0000 7f8ac2f57640 1 -- --> v1:172.21.15.43:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8abc0d0f70 con 0x7f8abc15c680 2023-08-09T13:51:10.300 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:10.293+0000 7f8ac2f57640 1 -- --> v1:172.21.15.203:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8abc0ccd10 con 0x7f8abc0cdd30 2023-08-09T13:51:10.300 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:10.293+0000 7f8ac2f57640 1 -- --> v2:172.21.15.203:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8abc0ccff0 con 0x7f8abc0d0a70 2023-08-09T13:51:10.300 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:10.293+0000 7f8ac0cf3640 1 --1- >> v1:172.21.15.43:6789/0 conn(0x7f8abc15c680 0x7f8abc0ce760 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.43:6789/0 says I am v1:172.21.15.43:45718/0 (socket says 172.21.15.43:45718) 2023-08-09T13:51:10.301 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:10.293+0000 7f8ac0cf3640 1 -- 172.21.15.43:0/2143658299 learned_addr learned my addr 172.21.15.43:0/2143658299 (peer_addr_for_me v1:172.21.15.43:0/0) 2023-08-09T13:51:10.301 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:10.294+0000 7f8abbfff640 1 --2- 172.21.15.43:0/2143658299 >> v2:172.21.15.203:3300/0 conn(0x7f8abc0d0a70 0x7f8abc0cd7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:51:10.301 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:10.294+0000 7f8abb7fe640 1 -- 172.21.15.43:0/2143658299 <== mon.0 v1:172.21.15.43:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2433167979 0 0) 0x7f8abc0d0f70 con 0x7f8abc15c680 2023-08-09T13:51:10.301 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:10.294+0000 7f8abb7fe640 1 -- 172.21.15.43:0/2143658299 --> v1:172.21.15.43:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f8aac003520 con 0x7f8abc15c680 2023-08-09T13:51:10.302 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:10.294+0000 7f8abb7fe640 1 -- 172.21.15.43:0/2143658299 <== mon.0 v1:172.21.15.43:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3985341661 0 0) 0x7f8aac003520 con 0x7f8abc15c680 2023-08-09T13:51:10.302 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:10.294+0000 7f8abb7fe640 1 -- 172.21.15.43:0/2143658299 >> v2:172.21.15.203:3300/0 conn(0x7f8abc0d0a70 msgr2=0x7f8abc0cd7f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-09T13:51:10.302 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:10.294+0000 7f8abb7fe640 1 --2- 172.21.15.43:0/2143658299 >> v2:172.21.15.203:3300/0 conn(0x7f8abc0d0a70 0x7f8abc0cd7f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:51:10.303 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:10.294+0000 7f8abb7fe640 1 -- 172.21.15.43:0/2143658299 >> v1:172.21.15.203:6789/0 conn(0x7f8abc0cdd30 legacy=0x7f8abc0b3fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:51:10.303 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:10.294+0000 7f8abb7fe640 1 -- 172.21.15.43:0/2143658299 --> v1:172.21.15.43:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8abc0cd2d0 con 0x7f8abc15c680 2023-08-09T13:51:10.303 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:10.294+0000 7f8abb7fe640 1 -- 172.21.15.43:0/2143658299 <== mon.0 v1:172.21.15.43:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1285150740 0 0) 0x7f8ab0001670 con 0x7f8abc15c680 2023-08-09T13:51:10.303 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:10.294+0000 7f8abb7fe640 1 -- 172.21.15.43:0/2143658299 >> v1:172.21.15.43:6789/0 conn(0x7f8abc15c680 legacy=0x7f8abc0ce760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:51:10.303 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:10.294+0000 7f8abb7fe640 1 --2- 172.21.15.43:0/2143658299 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f8aac0039e0 0x7f8aac003e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:51:10.304 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:10.294+0000 7f8abb7fe640 1 --2- 172.21.15.43:0/2143658299 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f8aac0047c0 0x7f8aac004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:51:10.304 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:10.294+0000 7f8abb7fe640 1 --2- 172.21.15.43:0/2143658299 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f8aac005180 0x7f8aac00b980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:51:10.304 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:10.295+0000 7f8abb7fe640 1 -- 172.21.15.43:0/2143658299 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8abc0ccff0 con 0x7f8aac005180 2023-08-09T13:51:10.304 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:10.295+0000 7f8abb7fe640 1 -- 172.21.15.43:0/2143658299 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8aac00c0b0 con 0x7f8aac0039e0 2023-08-09T13:51:10.305 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:10.295+0000 7f8abb7fe640 1 -- 172.21.15.43:0/2143658299 --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8aac00c270 con 0x7f8aac0047c0 2023-08-09T13:51:10.305 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:10.295+0000 7f8ac14f4640 1 --2- 172.21.15.43:0/2143658299 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f8aac005180 0x7f8aac00b980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:51:10.305 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:10.295+0000 7f8ac0cf3640 1 --2- 172.21.15.43:0/2143658299 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f8aac0039e0 0x7f8aac003e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:51:10.305 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:10.295+0000 7f8abbfff640 1 --2- 172.21.15.43:0/2143658299 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f8aac0047c0 0x7f8aac004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:51:10.306 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:10.295+0000 7f8ac0cf3640 1 -- 172.21.15.43:0/2143658299 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f8aac0047c0 msgr2=0x7f8aac004c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-09T13:51:10.306 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:10.295+0000 7f8ac0cf3640 1 --2- 172.21.15.43:0/2143658299 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f8aac0047c0 0x7f8aac004c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:51:10.306 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:10.295+0000 7f8ac0cf3640 1 -- 172.21.15.43:0/2143658299 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f8aac005180 msgr2=0x7f8aac00b980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-09T13:51:10.307 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:10.295+0000 7f8ac0cf3640 1 --2- 172.21.15.43:0/2143658299 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f8aac005180 0x7f8aac00b980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:51:10.307 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:10.295+0000 7f8ac0cf3640 1 -- 172.21.15.43:0/2143658299 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8aac003520 con 0x7f8aac0039e0 2023-08-09T13:51:10.307 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:10.295+0000 7f8abbfff640 1 --2- 172.21.15.43:0/2143658299 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f8aac0047c0 0x7f8aac004c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-08-09T13:51:10.307 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:10.295+0000 7f8ac14f4640 1 --2- 172.21.15.43:0/2143658299 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f8aac005180 0x7f8aac00b980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-08-09T13:51:10.307 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:10.295+0000 7f8ac14f4640 1 -- 172.21.15.43:0/2143658299 reap_dead start 2023-08-09T13:51:10.308 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:10.296+0000 7f8ac0cf3640 1 --2- 172.21.15.43:0/2143658299 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f8aac0039e0 0x7f8aac003e00 secure :-1 s=READY pgs=421 cs=0 l=1 rev1=1 crypto rx=0x7f8ab000c210 tx=0x7f8aac0057f0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=cc5a0ad1a593b729 server_cookie=0 in_seq=0 out_seq=0 2023-08-09T13:51:10.311 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:10.297+0000 7f8abb7fe640 1 -- 172.21.15.43:0/2143658299 <== mon.1 v2:172.21.15.203:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8ab0005ad0 con 0x7f8aac0039e0 2023-08-09T13:51:10.311 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:10.297+0000 7f8abb7fe640 1 -- 172.21.15.43:0/2143658299 <== mon.1 v2:172.21.15.203:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8ab0005c30 con 0x7f8aac0039e0 2023-08-09T13:51:10.311 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:10.297+0000 7f8abb7fe640 1 -- 172.21.15.43:0/2143658299 <== mon.1 v2:172.21.15.203:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8ab002c8c0 con 0x7f8aac0039e0 2023-08-09T13:51:10.311 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:10.297+0000 7f8ac2f57640 1 -- 172.21.15.43:0/2143658299 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f8aac0039e0 msgr2=0x7f8aac003e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:51:10.312 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:10.297+0000 7f8ac2f57640 1 --2- 172.21.15.43:0/2143658299 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f8aac0039e0 0x7f8aac003e00 secure :-1 s=READY pgs=421 cs=0 l=1 rev1=1 crypto rx=0x7f8ab000c210 tx=0x7f8aac0057f0 comp rx=0 tx=0).stop 2023-08-09T13:51:10.312 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:10.297+0000 7f8ac2f57640 1 -- 172.21.15.43:0/2143658299 shutdown_connections 2023-08-09T13:51:10.312 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:10.297+0000 7f8ac2f57640 1 --2- 172.21.15.43:0/2143658299 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f8aac0039e0 0x7f8aac003e00 unknown :-1 s=CLOSED pgs=421 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:51:10.312 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:10.297+0000 7f8ac2f57640 1 -- 172.21.15.43:0/2143658299 >> 172.21.15.43:0/2143658299 conn(0x7f8abc071520 msgr2=0x7f8abc0d2520 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-09T13:51:10.312 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:10.298+0000 7f8ac2f57640 1 -- 172.21.15.43:0/2143658299 shutdown_connections 2023-08-09T13:51:10.312 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:10.298+0000 7f8ac2f57640 1 -- 172.21.15.43:0/2143658299 wait complete. 2023-08-09T13:51:10.313 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:10.299+0000 7f8ac2f57640 1 Processor -- start 2023-08-09T13:51:10.313 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:10.299+0000 7f8ac2f57640 1 -- start start 2023-08-09T13:51:10.313 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:10.300+0000 7f8ac2f57640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f8aac0039e0 0x7f8abc14dc00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:51:10.313 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:10.300+0000 7f8ac2f57640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f8abc15ee00 0x7f8abc15a7d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:51:10.313 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:10.300+0000 7f8ac2f57640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f8abc15f280 0x7f8abc15ad10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:51:10.313 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:10.300+0000 7f8ac2f57640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8abc15e970 con 0x7f8abc15ee00 2023-08-09T13:51:10.314 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:10.300+0000 7f8ac2f57640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8abc15dd60 con 0x7f8abc15f280 2023-08-09T13:51:10.314 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:10.300+0000 7f8ac2f57640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8abc15e040 con 0x7f8aac0039e0 2023-08-09T13:51:10.314 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:10.301+0000 7f8abbfff640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f8abc15ee00 0x7f8abc15a7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:51:10.314 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:10.301+0000 7f8abbfff640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f8abc15ee00 0x7f8abc15a7d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:36630/0 (socket says 172.21.15.43:36630) 2023-08-09T13:51:10.314 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:10.301+0000 7f8abbfff640 1 -- 172.21.15.43:0/3060456063 learned_addr learned my addr 172.21.15.43:0/3060456063 (peer_addr_for_me v2:172.21.15.43:0/0) 2023-08-09T13:51:10.314 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:10.301+0000 7f8ac0cf3640 1 --2- 172.21.15.43:0/3060456063 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f8aac0039e0 0x7f8abc14dc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:51:10.314 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:10.301+0000 7f8ac14f4640 1 --2- 172.21.15.43:0/3060456063 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f8abc15f280 0x7f8abc15ad10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:51:10.315 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:10.301+0000 7f8abbfff640 1 -- 172.21.15.43:0/3060456063 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f8aac0039e0 msgr2=0x7f8abc14dc00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:51:10.315 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:10.301+0000 7f8abbfff640 1 --2- 172.21.15.43:0/3060456063 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f8aac0039e0 0x7f8abc14dc00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:51:10.315 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:10.301+0000 7f8abbfff640 1 -- 172.21.15.43:0/3060456063 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f8abc15f280 msgr2=0x7f8abc15ad10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:51:10.315 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:10.301+0000 7f8abbfff640 1 --2- 172.21.15.43:0/3060456063 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f8abc15f280 0x7f8abc15ad10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:51:10.315 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:10.301+0000 7f8abbfff640 1 -- 172.21.15.43:0/3060456063 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8abc15e320 con 0x7f8abc15ee00 2023-08-09T13:51:10.315 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:10.302+0000 7f8abbfff640 1 --2- 172.21.15.43:0/3060456063 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f8abc15ee00 0x7f8abc15a7d0 secure :-1 s=READY pgs=1153 cs=0 l=1 rev1=1 crypto rx=0x7f8aa80114c0 tx=0x7f8aa80118c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-09T13:51:10.316 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:10.302+0000 7f8ab97fa640 1 -- 172.21.15.43:0/3060456063 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8aa8015870 con 0x7f8abc15ee00 2023-08-09T13:51:10.316 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:10.302+0000 7f8ab97fa640 1 -- 172.21.15.43:0/3060456063 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8aa80159d0 con 0x7f8abc15ee00 2023-08-09T13:51:10.316 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:10.302+0000 7f8ab97fa640 1 -- 172.21.15.43:0/3060456063 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8aa8015d40 con 0x7f8abc15ee00 2023-08-09T13:51:10.316 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:10.302+0000 7f8ac2f57640 1 -- 172.21.15.43:0/3060456063 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8abc15e600 con 0x7f8abc15ee00 2023-08-09T13:51:10.316 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:10.302+0000 7f8ac2f57640 1 -- 172.21.15.43:0/3060456063 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8abc150d30 con 0x7f8abc15ee00 2023-08-09T13:51:10.316 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:10.304+0000 7f8ac2f57640 1 -- 172.21.15.43:0/3060456063 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8abc15e790 con 0x7f8abc15ee00 2023-08-09T13:51:10.316 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:10.305+0000 7f8ab97fa640 1 -- 172.21.15.43:0/3060456063 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 37) v1 ==== 88443+0+0 (secure 0 0 0) 0x7f8aa8013400 con 0x7f8abc15ee00 2023-08-09T13:51:10.317 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:10.305+0000 7f8ab97fa640 1 --2- 172.21.15.43:0/3060456063 >> [v2:172.21.15.203:6832/1778614695,v1:172.21.15.203:6833/1778614695] conn(0x7f8aa006a1f0 0x7f8aa006c6b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:51:10.317 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:10.306+0000 7f8ab97fa640 1 -- 172.21.15.43:0/3060456063 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(95..95 src has 1..95) v4 ==== 6348+0+0 (secure 0 0 0) 0x7f8aa8088750 con 0x7f8abc15ee00 2023-08-09T13:51:10.317 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:10.306+0000 7f8ac0cf3640 1 --2- 172.21.15.43:0/3060456063 >> [v2:172.21.15.203:6832/1778614695,v1:172.21.15.203:6833/1778614695] conn(0x7f8aa006a1f0 0x7f8aa006c6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:51:10.317 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:10.307+0000 7f8ac0cf3640 1 --2- 172.21.15.43:0/3060456063 >> [v2:172.21.15.203:6832/1778614695,v1:172.21.15.203:6833/1778614695] conn(0x7f8aa006a1f0 0x7f8aa006c6b0 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7f8ab0034000 tx=0x7f8ab0009030 comp rx=0 tx=0).ready entity=mgr.5819 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-09T13:51:10.317 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:10.310+0000 7f8ab97fa640 1 -- 172.21.15.43:0/3060456063 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178066 (secure 0 0 0) 0x7f8aa8053cf0 con 0x7f8abc15ee00 2023-08-09T13:51:10.518 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:10.515+0000 7f8ac2f57640 1 -- 172.21.15.43:0/3060456063 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "osd blocklist clear"} v 0) v1 -- 0x7f8abc0739f0 con 0x7f8abc15ee00 2023-08-09T13:51:11.182 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:11.179+0000 7f8ab97fa640 1 -- 172.21.15.43:0/3060456063 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist clear"}]=0 removed all blocklist entries v97) v1 ==== 97+0+0 (secure 0 0 0) 0x7f8aa8057310 con 0x7f8abc15ee00 2023-08-09T13:51:11.182 INFO:teuthology.orchestra.run.smithi043.stderr: removed all blocklist entries 2023-08-09T13:51:11.186 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:11.183+0000 7f8ac2f57640 1 -- 172.21.15.43:0/3060456063 >> [v2:172.21.15.203:6832/1778614695,v1:172.21.15.203:6833/1778614695] conn(0x7f8aa006a1f0 msgr2=0x7f8aa006c6b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:51:11.186 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:11.183+0000 7f8ac2f57640 1 --2- 172.21.15.43:0/3060456063 >> [v2:172.21.15.203:6832/1778614695,v1:172.21.15.203:6833/1778614695] conn(0x7f8aa006a1f0 0x7f8aa006c6b0 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7f8ab0034000 tx=0x7f8ab0009030 comp rx=0 tx=0).stop 2023-08-09T13:51:11.186 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:11.183+0000 7f8ac2f57640 1 -- 172.21.15.43:0/3060456063 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f8abc15ee00 msgr2=0x7f8abc15a7d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:51:11.186 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:11.183+0000 7f8ac2f57640 1 --2- 172.21.15.43:0/3060456063 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f8abc15ee00 0x7f8abc15a7d0 secure :-1 s=READY pgs=1153 cs=0 l=1 rev1=1 crypto rx=0x7f8aa80114c0 tx=0x7f8aa80118c0 comp rx=0 tx=0).stop 2023-08-09T13:51:11.187 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:11.183+0000 7f8ac2f57640 1 -- 172.21.15.43:0/3060456063 shutdown_connections 2023-08-09T13:51:11.187 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:11.183+0000 7f8ac2f57640 1 --2- 172.21.15.43:0/3060456063 >> [v2:172.21.15.203:6832/1778614695,v1:172.21.15.203:6833/1778614695] conn(0x7f8aa006a1f0 0x7f8aa006c6b0 unknown :-1 s=CLOSED pgs=45 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:51:11.187 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:11.183+0000 7f8ac2f57640 1 --2- 172.21.15.43:0/3060456063 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f8abc15f280 0x7f8abc15ad10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:51:11.187 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:11.183+0000 7f8ac2f57640 1 --2- 172.21.15.43:0/3060456063 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f8abc15ee00 0x7f8abc15a7d0 unknown :-1 s=CLOSED pgs=1153 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:51:11.187 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:11.183+0000 7f8ac2f57640 1 --2- 172.21.15.43:0/3060456063 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f8aac0039e0 0x7f8abc14dc00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:51:11.187 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:11.183+0000 7f8ac2f57640 1 -- 172.21.15.43:0/3060456063 >> 172.21.15.43:0/3060456063 conn(0x7f8abc071520 msgr2=0x7f8abc0b6640 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-09T13:51:11.187 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:11.184+0000 7f8ac2f57640 1 -- 172.21.15.43:0/3060456063 shutdown_connections 2023-08-09T13:51:11.188 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:11.184+0000 7f8ac2f57640 1 -- 172.21.15.43:0/3060456063 wait complete. 2023-08-09T13:51:11.212 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth ls --format=json-pretty 2023-08-09T13:51:11.440 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:11.437+0000 7fa5b1f86640 1 Processor -- start 2023-08-09T13:51:11.440 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:11.437+0000 7fa5b1f86640 1 -- start start 2023-08-09T13:51:11.441 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:11.439+0000 7fa5b1f86640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7fa5ac160cb0 0x7fa5ac15c070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:51:11.442 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:11.439+0000 7fa5b1f86640 1 -- --> v1:172.21.15.43:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa5ac0c8680 con 0x7fa5ac160380 2023-08-09T13:51:11.442 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:11.439+0000 7fa5b1f86640 1 -- --> v1:172.21.15.203:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa5ac0c8960 con 0x7fa5ac160760 2023-08-09T13:51:11.442 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:11.439+0000 7fa5b1f86640 1 -- --> v2:172.21.15.43:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa5ac0ca7d0 con 0x7fa5ac160cb0 2023-08-09T13:51:11.443 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:11.439+0000 7fa5abfff640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7fa5ac160cb0 0x7fa5ac15c070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:51:11.443 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:11.439+0000 7fa5b0f84640 1 --1- >> v1:172.21.15.43:6789/0 conn(0x7fa5ac160380 0x7fa5ac0ca0c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.43:6789/0 says I am v1:172.21.15.43:45730/0 (socket says 172.21.15.43:45730) 2023-08-09T13:51:11.443 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:11.439+0000 7fa5abfff640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7fa5ac160cb0 0x7fa5ac15c070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:36644/0 (socket says 172.21.15.43:36644) 2023-08-09T13:51:11.443 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:11.439+0000 7fa5b0f84640 1 -- 172.21.15.43:0/1806459020 learned_addr learned my addr 172.21.15.43:0/1806459020 (peer_addr_for_me v1:172.21.15.43:0/0) 2023-08-09T13:51:11.443 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:11.440+0000 7fa5abfff640 1 -- 172.21.15.43:0/1806459020 >> v1:172.21.15.203:6789/0 conn(0x7fa5ac160760 legacy=0x7fa5ac0ab700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-09T13:51:11.443 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:11.440+0000 7fa5abfff640 1 -- 172.21.15.43:0/1806459020 >> v1:172.21.15.43:6789/0 conn(0x7fa5ac160380 legacy=0x7fa5ac0ca0c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:51:11.444 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:11.440+0000 7fa5abfff640 1 -- 172.21.15.43:0/1806459020 --> v2:172.21.15.43:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa5ac0aaf00 con 0x7fa5ac160cb0 2023-08-09T13:51:11.444 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:11.441+0000 7fa5abfff640 1 --2- 172.21.15.43:0/1806459020 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa5ac160cb0 0x7fa5ac15c070 secure :-1 s=READY pgs=1155 cs=0 l=1 rev1=1 crypto rx=0x7fa5980097a0 tx=0x7fa59802b790 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c295e225720fa5c7 server_cookie=0 in_seq=0 out_seq=0 2023-08-09T13:51:11.444 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:11.442+0000 7fa5ab7fe640 1 -- 172.21.15.43:0/1806459020 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa598030710 con 0x7fa5ac160cb0 2023-08-09T13:51:11.445 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:11.442+0000 7fa5ab7fe640 1 -- 172.21.15.43:0/1806459020 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa59802e050 con 0x7fa5ac160cb0 2023-08-09T13:51:11.445 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:11.442+0000 7fa5ab7fe640 1 -- 172.21.15.43:0/1806459020 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa598030a90 con 0x7fa5ac160cb0 2023-08-09T13:51:11.445 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:11.442+0000 7fa5b1f86640 1 -- 172.21.15.43:0/1806459020 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa5ac160cb0 msgr2=0x7fa5ac15c070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:51:11.446 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:11.442+0000 7fa5b1f86640 1 --2- 172.21.15.43:0/1806459020 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa5ac160cb0 0x7fa5ac15c070 secure :-1 s=READY pgs=1155 cs=0 l=1 rev1=1 crypto rx=0x7fa5980097a0 tx=0x7fa59802b790 comp rx=0 tx=0).stop 2023-08-09T13:51:11.446 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:11.443+0000 7fa5b1f86640 1 -- 172.21.15.43:0/1806459020 shutdown_connections 2023-08-09T13:51:11.446 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:11.443+0000 7fa5b1f86640 1 --2- 172.21.15.43:0/1806459020 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa5ac160cb0 0x7fa5ac15c070 unknown :-1 s=CLOSED pgs=1155 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:51:11.446 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:11.443+0000 7fa5b1f86640 1 -- 172.21.15.43:0/1806459020 >> 172.21.15.43:0/1806459020 conn(0x7fa5ac0cde90 msgr2=0x7fa5ac0ce2a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-09T13:51:11.446 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:11.443+0000 7fa5b1f86640 1 -- 172.21.15.43:0/1806459020 shutdown_connections 2023-08-09T13:51:11.446 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:11.443+0000 7fa5b1f86640 1 -- 172.21.15.43:0/1806459020 wait complete. 2023-08-09T13:51:11.447 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:11.445+0000 7fa5b1f86640 1 Processor -- start 2023-08-09T13:51:11.447 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:11.445+0000 7fa5b1f86640 1 -- start start 2023-08-09T13:51:11.448 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:11.446+0000 7fa5b1f86640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa5ac160380 0x7fa5ac15e320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:51:11.448 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:11.446+0000 7fa5b1f86640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fa5ac160760 0x7fa5ac098a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:51:11.449 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:11.446+0000 7fa5b1f86640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fa5ac160cb0 0x7fa5ac145d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:51:11.449 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:11.446+0000 7fa5b1f86640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa5ac0c5aa0 con 0x7fa5ac160380 2023-08-09T13:51:11.449 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:11.446+0000 7fa5b1f86640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa5ac0c5100 con 0x7fa5ac160760 2023-08-09T13:51:11.449 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:11.446+0000 7fa5b1f86640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa5ac0c53e0 con 0x7fa5ac160cb0 2023-08-09T13:51:11.449 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:11.446+0000 7fa5b0f84640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa5ac160380 0x7fa5ac15e320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:51:11.449 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:11.446+0000 7fa5b0f84640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa5ac160380 0x7fa5ac15e320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:36660/0 (socket says 172.21.15.43:36660) 2023-08-09T13:51:11.449 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:11.446+0000 7fa5b0f84640 1 -- 172.21.15.43:0/3376720483 learned_addr learned my addr 172.21.15.43:0/3376720483 (peer_addr_for_me v2:172.21.15.43:0/0) 2023-08-09T13:51:11.451 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:11.446+0000 7fa5b0f84640 1 -- 172.21.15.43:0/3376720483 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fa5ac160cb0 msgr2=0x7fa5ac145d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:51:11.451 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:11.446+0000 7fa5b0f84640 1 --2- 172.21.15.43:0/3376720483 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fa5ac160cb0 0x7fa5ac145d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:51:11.451 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:11.447+0000 7fa5b0f84640 1 -- 172.21.15.43:0/3376720483 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fa5ac160760 msgr2=0x7fa5ac098a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:51:11.451 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:11.447+0000 7fa5abfff640 1 --2- 172.21.15.43:0/3376720483 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fa5ac160760 0x7fa5ac098a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:51:11.451 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:11.447+0000 7fa5b0f84640 1 --2- 172.21.15.43:0/3376720483 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fa5ac160760 0x7fa5ac098a50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:51:11.451 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:11.447+0000 7fa5b0f84640 1 -- 172.21.15.43:0/3376720483 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa598009400 con 0x7fa5ac160380 2023-08-09T13:51:11.451 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:11.447+0000 7fa5b0f84640 1 --2- 172.21.15.43:0/3376720483 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa5ac160380 0x7fa5ac15e320 secure :-1 s=READY pgs=1156 cs=0 l=1 rev1=1 crypto rx=0x7fa5a000c780 tx=0x7fa5a000cc40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-09T13:51:11.452 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:11.447+0000 7fa5a97fa640 1 -- 172.21.15.43:0/3376720483 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa5a00072d0 con 0x7fa5ac160380 2023-08-09T13:51:11.452 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:11.447+0000 7fa5a97fa640 1 -- 172.21.15.43:0/3376720483 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa5a0007430 con 0x7fa5ac160380 2023-08-09T13:51:11.452 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:11.447+0000 7fa5b1f86640 1 -- 172.21.15.43:0/3376720483 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa5ac0c56c0 con 0x7fa5ac160380 2023-08-09T13:51:11.452 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:11.447+0000 7fa5a97fa640 1 -- 172.21.15.43:0/3376720483 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa5a00047e0 con 0x7fa5ac160380 2023-08-09T13:51:11.452 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:11.447+0000 7fa5b1f86640 1 -- 172.21.15.43:0/3376720483 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa5ac150430 con 0x7fa5ac160380 2023-08-09T13:51:11.453 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:11.449+0000 7fa5b1f86640 1 -- 172.21.15.43:0/3376720483 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa5ac0c4ec0 con 0x7fa5ac160380 2023-08-09T13:51:11.453 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:11.450+0000 7fa5a97fa640 1 -- 172.21.15.43:0/3376720483 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 37) v1 ==== 88443+0+0 (secure 0 0 0) 0x7fa5a0004940 con 0x7fa5ac160380 2023-08-09T13:51:11.453 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:11.451+0000 7fa5a97fa640 1 --2- 172.21.15.43:0/3376720483 >> [v2:172.21.15.203:6832/1778614695,v1:172.21.15.203:6833/1778614695] conn(0x7fa59006a170 0x7fa59006c630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:51:11.454 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:11.451+0000 7fa5a97fa640 1 -- 172.21.15.43:0/3376720483 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(99..99 src has 1..99) v4 ==== 5574+0+0 (secure 0 0 0) 0x7fa5a0087a10 con 0x7fa5ac160380 2023-08-09T13:51:11.454 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:11.451+0000 7fa5abfff640 1 --2- 172.21.15.43:0/3376720483 >> [v2:172.21.15.203:6832/1778614695,v1:172.21.15.203:6833/1778614695] conn(0x7fa59006a170 0x7fa59006c630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:51:11.455 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:11.453+0000 7fa5abfff640 1 --2- 172.21.15.43:0/3376720483 >> [v2:172.21.15.203:6832/1778614695,v1:172.21.15.203:6833/1778614695] conn(0x7fa59006a170 0x7fa59006c630 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7fa598034000 tx=0x7fa59802f000 comp rx=0 tx=0).ready entity=mgr.5819 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-09T13:51:11.457 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:11.455+0000 7fa5a97fa640 1 -- 172.21.15.43:0/3376720483 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178066 (secure 0 0 0) 0x7fa5ac0c4ec0 con 0x7fa5ac160380 2023-08-09T13:51:11.724 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:11.722+0000 7fa5b1f86640 1 -- 172.21.15.43:0/3376720483 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "auth ls", "format": "json-pretty"} v 0) v1 -- 0x7fa5ac150430 con 0x7fa5ac160380 2023-08-09T13:51:11.725 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:11.723+0000 7fa5a97fa640 1 -- 172.21.15.43:0/3376720483 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "auth ls", "format": "json-pretty"}]=0 v15) v1 ==== 80+0+5304 (secure 0 0 0) 0x7fa5ac150430 con 0x7fa5ac160380 2023-08-09T13:51:11.726 INFO:teuthology.orchestra.run.smithi043.stdout: 2023-08-09T13:51:11.726 INFO:teuthology.orchestra.run.smithi043.stdout:{ 2023-08-09T13:51:11.726 INFO:teuthology.orchestra.run.smithi043.stdout: "auth_dump": [ 2023-08-09T13:51:11.726 INFO:teuthology.orchestra.run.smithi043.stdout: { 2023-08-09T13:51:11.726 INFO:teuthology.orchestra.run.smithi043.stdout: "entity": "mds.a", 2023-08-09T13:51:11.726 INFO:teuthology.orchestra.run.smithi043.stdout: "key": "AQCOldNkPu6GCRAAZecwctvMji0Zubb60KMlyQ==", 2023-08-09T13:51:11.727 INFO:teuthology.orchestra.run.smithi043.stdout: "caps": { 2023-08-09T13:51:11.727 INFO:teuthology.orchestra.run.smithi043.stdout: "mds": "allow", 2023-08-09T13:51:11.727 INFO:teuthology.orchestra.run.smithi043.stdout: "mgr": "allow *", 2023-08-09T13:51:11.727 INFO:teuthology.orchestra.run.smithi043.stdout: "mon": "allow *", 2023-08-09T13:51:11.727 INFO:teuthology.orchestra.run.smithi043.stdout: "osd": "allow *" 2023-08-09T13:51:11.727 INFO:teuthology.orchestra.run.smithi043.stdout: } 2023-08-09T13:51:11.727 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2023-08-09T13:51:11.728 INFO:teuthology.orchestra.run.smithi043.stdout: { 2023-08-09T13:51:11.728 INFO:teuthology.orchestra.run.smithi043.stdout: "entity": "mds.b", 2023-08-09T13:51:11.728 INFO:teuthology.orchestra.run.smithi043.stdout: "key": "AQCOldNkt2+5KxAAHcckB+S5NZHsF5z22StXmw==", 2023-08-09T13:51:11.728 INFO:teuthology.orchestra.run.smithi043.stdout: "caps": { 2023-08-09T13:51:11.728 INFO:teuthology.orchestra.run.smithi043.stdout: "mds": "allow", 2023-08-09T13:51:11.728 INFO:teuthology.orchestra.run.smithi043.stdout: "mgr": "allow *", 2023-08-09T13:51:11.728 INFO:teuthology.orchestra.run.smithi043.stdout: "mon": "allow *", 2023-08-09T13:51:11.729 INFO:teuthology.orchestra.run.smithi043.stdout: "osd": "allow *" 2023-08-09T13:51:11.729 INFO:teuthology.orchestra.run.smithi043.stdout: } 2023-08-09T13:51:11.729 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2023-08-09T13:51:11.729 INFO:teuthology.orchestra.run.smithi043.stdout: { 2023-08-09T13:51:11.729 INFO:teuthology.orchestra.run.smithi043.stdout: "entity": "mds.c", 2023-08-09T13:51:11.729 INFO:teuthology.orchestra.run.smithi043.stdout: "key": "AQCOldNk5V46GhAAQBhkvfjVSJCSHLS/YkhgCA==", 2023-08-09T13:51:11.729 INFO:teuthology.orchestra.run.smithi043.stdout: "caps": { 2023-08-09T13:51:11.730 INFO:teuthology.orchestra.run.smithi043.stdout: "mds": "allow", 2023-08-09T13:51:11.730 INFO:teuthology.orchestra.run.smithi043.stdout: "mgr": "allow *", 2023-08-09T13:51:11.730 INFO:teuthology.orchestra.run.smithi043.stdout: "mon": "allow *", 2023-08-09T13:51:11.730 INFO:teuthology.orchestra.run.smithi043.stdout: "osd": "allow *" 2023-08-09T13:51:11.730 INFO:teuthology.orchestra.run.smithi043.stdout: } 2023-08-09T13:51:11.730 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2023-08-09T13:51:11.730 INFO:teuthology.orchestra.run.smithi043.stdout: { 2023-08-09T13:51:11.730 INFO:teuthology.orchestra.run.smithi043.stdout: "entity": "mds.d", 2023-08-09T13:51:11.731 INFO:teuthology.orchestra.run.smithi043.stdout: "key": "AQCPldNkNBoTARAAMVW+rSyXQKaau6I90fUX9Q==", 2023-08-09T13:51:11.731 INFO:teuthology.orchestra.run.smithi043.stdout: "caps": { 2023-08-09T13:51:11.731 INFO:teuthology.orchestra.run.smithi043.stdout: "mds": "allow", 2023-08-09T13:51:11.731 INFO:teuthology.orchestra.run.smithi043.stdout: "mgr": "allow *", 2023-08-09T13:51:11.731 INFO:teuthology.orchestra.run.smithi043.stdout: "mon": "allow *", 2023-08-09T13:51:11.731 INFO:teuthology.orchestra.run.smithi043.stdout: "osd": "allow *" 2023-08-09T13:51:11.731 INFO:teuthology.orchestra.run.smithi043.stdout: } 2023-08-09T13:51:11.732 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2023-08-09T13:51:11.732 INFO:teuthology.orchestra.run.smithi043.stdout: { 2023-08-09T13:51:11.732 INFO:teuthology.orchestra.run.smithi043.stdout: "entity": "osd.0", 2023-08-09T13:51:11.732 INFO:teuthology.orchestra.run.smithi043.stdout: "key": "AQCRldNkltPlMhAAAt3rFvraCtI0aSqGvt6UxA==", 2023-08-09T13:51:11.732 INFO:teuthology.orchestra.run.smithi043.stdout: "caps": { 2023-08-09T13:51:11.732 INFO:teuthology.orchestra.run.smithi043.stdout: "mgr": "allow profile osd", 2023-08-09T13:51:11.732 INFO:teuthology.orchestra.run.smithi043.stdout: "mon": "allow profile osd", 2023-08-09T13:51:11.733 INFO:teuthology.orchestra.run.smithi043.stdout: "osd": "allow *" 2023-08-09T13:51:11.733 INFO:teuthology.orchestra.run.smithi043.stdout: } 2023-08-09T13:51:11.733 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2023-08-09T13:51:11.733 INFO:teuthology.orchestra.run.smithi043.stdout: { 2023-08-09T13:51:11.733 INFO:teuthology.orchestra.run.smithi043.stdout: "entity": "osd.1", 2023-08-09T13:51:11.734 INFO:teuthology.orchestra.run.smithi043.stdout: "key": "AQCVldNkzpKYEhAA+wen0kV/vUDLQztq3vhfOg==", 2023-08-09T13:51:11.734 INFO:teuthology.orchestra.run.smithi043.stdout: "caps": { 2023-08-09T13:51:11.734 INFO:teuthology.orchestra.run.smithi043.stdout: "mgr": "allow profile osd", 2023-08-09T13:51:11.734 INFO:teuthology.orchestra.run.smithi043.stdout: "mon": "allow profile osd", 2023-08-09T13:51:11.734 INFO:teuthology.orchestra.run.smithi043.stdout: "osd": "allow *" 2023-08-09T13:51:11.734 INFO:teuthology.orchestra.run.smithi043.stdout: } 2023-08-09T13:51:11.734 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2023-08-09T13:51:11.735 INFO:teuthology.orchestra.run.smithi043.stdout: { 2023-08-09T13:51:11.735 INFO:teuthology.orchestra.run.smithi043.stdout: "entity": "osd.2", 2023-08-09T13:51:11.735 INFO:teuthology.orchestra.run.smithi043.stdout: "key": "AQCYldNkYbwNHhAAQY8g9dldb1V9M2MM2SHKDg==", 2023-08-09T13:51:11.735 INFO:teuthology.orchestra.run.smithi043.stdout: "caps": { 2023-08-09T13:51:11.735 INFO:teuthology.orchestra.run.smithi043.stdout: "mgr": "allow profile osd", 2023-08-09T13:51:11.735 INFO:teuthology.orchestra.run.smithi043.stdout: "mon": "allow profile osd", 2023-08-09T13:51:11.735 INFO:teuthology.orchestra.run.smithi043.stdout: "osd": "allow *" 2023-08-09T13:51:11.735 INFO:teuthology.orchestra.run.smithi043.stdout: } 2023-08-09T13:51:11.736 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2023-08-09T13:51:11.736 INFO:teuthology.orchestra.run.smithi043.stdout: { 2023-08-09T13:51:11.736 INFO:teuthology.orchestra.run.smithi043.stdout: "entity": "osd.3", 2023-08-09T13:51:11.736 INFO:teuthology.orchestra.run.smithi043.stdout: "key": "AQCbldNkeyW6KRAAsd+kgqik2aD4U7PXwccWtA==", 2023-08-09T13:51:11.736 INFO:teuthology.orchestra.run.smithi043.stdout: "caps": { 2023-08-09T13:51:11.736 INFO:teuthology.orchestra.run.smithi043.stdout: "mgr": "allow profile osd", 2023-08-09T13:51:11.736 INFO:teuthology.orchestra.run.smithi043.stdout: "mon": "allow profile osd", 2023-08-09T13:51:11.737 INFO:teuthology.orchestra.run.smithi043.stdout: "osd": "allow *" 2023-08-09T13:51:11.737 INFO:teuthology.orchestra.run.smithi043.stdout: } 2023-08-09T13:51:11.737 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2023-08-09T13:51:11.737 INFO:teuthology.orchestra.run.smithi043.stdout: { 2023-08-09T13:51:11.737 INFO:teuthology.orchestra.run.smithi043.stdout: "entity": "osd.4", 2023-08-09T13:51:11.737 INFO:teuthology.orchestra.run.smithi043.stdout: "key": "AQCildNk80UkOBAAtX+4JGpOVcfZEHetJoeZKw==", 2023-08-09T13:51:11.737 INFO:teuthology.orchestra.run.smithi043.stdout: "caps": { 2023-08-09T13:51:11.738 INFO:teuthology.orchestra.run.smithi043.stdout: "mgr": "allow profile osd", 2023-08-09T13:51:11.738 INFO:teuthology.orchestra.run.smithi043.stdout: "mon": "allow profile osd", 2023-08-09T13:51:11.738 INFO:teuthology.orchestra.run.smithi043.stdout: "osd": "allow *" 2023-08-09T13:51:11.738 INFO:teuthology.orchestra.run.smithi043.stdout: } 2023-08-09T13:51:11.738 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2023-08-09T13:51:11.738 INFO:teuthology.orchestra.run.smithi043.stdout: { 2023-08-09T13:51:11.738 INFO:teuthology.orchestra.run.smithi043.stdout: "entity": "osd.5", 2023-08-09T13:51:11.739 INFO:teuthology.orchestra.run.smithi043.stdout: "key": "AQCmldNkCA9SBxAAZj4So6MRj2Eq1I9H9JzLDA==", 2023-08-09T13:51:11.739 INFO:teuthology.orchestra.run.smithi043.stdout: "caps": { 2023-08-09T13:51:11.739 INFO:teuthology.orchestra.run.smithi043.stdout: "mgr": "allow profile osd", 2023-08-09T13:51:11.739 INFO:teuthology.orchestra.run.smithi043.stdout: "mon": "allow profile osd", 2023-08-09T13:51:11.739 INFO:teuthology.orchestra.run.smithi043.stdout: "osd": "allow *" 2023-08-09T13:51:11.739 INFO:teuthology.orchestra.run.smithi043.stdout: } 2023-08-09T13:51:11.739 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2023-08-09T13:51:11.740 INFO:teuthology.orchestra.run.smithi043.stdout: { 2023-08-09T13:51:11.740 INFO:teuthology.orchestra.run.smithi043.stdout: "entity": "osd.6", 2023-08-09T13:51:11.741 INFO:teuthology.orchestra.run.smithi043.stdout: "key": "AQCpldNkaOwAExAAraV1tUwr9uU6GheZ+hvdqQ==", 2023-08-09T13:51:11.741 INFO:teuthology.orchestra.run.smithi043.stdout: "caps": { 2023-08-09T13:51:11.741 INFO:teuthology.orchestra.run.smithi043.stdout: "mgr": "allow profile osd", 2023-08-09T13:51:11.741 INFO:teuthology.orchestra.run.smithi043.stdout: "mon": "allow profile osd", 2023-08-09T13:51:11.741 INFO:teuthology.orchestra.run.smithi043.stdout: "osd": "allow *" 2023-08-09T13:51:11.741 INFO:teuthology.orchestra.run.smithi043.stdout: } 2023-08-09T13:51:11.742 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2023-08-09T13:51:11.742 INFO:teuthology.orchestra.run.smithi043.stdout: { 2023-08-09T13:51:11.742 INFO:teuthology.orchestra.run.smithi043.stdout: "entity": "osd.7", 2023-08-09T13:51:11.742 INFO:teuthology.orchestra.run.smithi043.stdout: "key": "AQCsldNk2/GwLBAAP42EXPivzLk44X8aMEVVCg==", 2023-08-09T13:51:11.742 INFO:teuthology.orchestra.run.smithi043.stdout: "caps": { 2023-08-09T13:51:11.742 INFO:teuthology.orchestra.run.smithi043.stdout: "mgr": "allow profile osd", 2023-08-09T13:51:11.742 INFO:teuthology.orchestra.run.smithi043.stdout: "mon": "allow profile osd", 2023-08-09T13:51:11.742 INFO:teuthology.orchestra.run.smithi043.stdout: "osd": "allow *" 2023-08-09T13:51:11.743 INFO:teuthology.orchestra.run.smithi043.stdout: } 2023-08-09T13:51:11.743 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2023-08-09T13:51:11.743 INFO:teuthology.orchestra.run.smithi043.stdout: { 2023-08-09T13:51:11.743 INFO:teuthology.orchestra.run.smithi043.stdout: "entity": "client.0", 2023-08-09T13:51:11.743 INFO:teuthology.orchestra.run.smithi043.stdout: "key": "AQCPldNkg8zrDxAAM/hnJaE6S9HU/fRwQeBV5w==", 2023-08-09T13:51:11.743 INFO:teuthology.orchestra.run.smithi043.stdout: "caps": { 2023-08-09T13:51:11.744 INFO:teuthology.orchestra.run.smithi043.stdout: "mds": "allow", 2023-08-09T13:51:11.744 INFO:teuthology.orchestra.run.smithi043.stdout: "mon": "allow r", 2023-08-09T13:51:11.744 INFO:teuthology.orchestra.run.smithi043.stdout: "osd": "allow rw tag cephfs data=cephfs" 2023-08-09T13:51:11.744 INFO:teuthology.orchestra.run.smithi043.stdout: } 2023-08-09T13:51:11.744 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2023-08-09T13:51:11.745 INFO:teuthology.orchestra.run.smithi043.stdout: { 2023-08-09T13:51:11.745 INFO:teuthology.orchestra.run.smithi043.stdout: "entity": "client.1", 2023-08-09T13:51:11.745 INFO:teuthology.orchestra.run.smithi043.stdout: "key": "AQCPldNklfFxJxAA19+L78rW6jU1uOyjey6WNA==", 2023-08-09T13:51:11.745 INFO:teuthology.orchestra.run.smithi043.stdout: "caps": { 2023-08-09T13:51:11.745 INFO:teuthology.orchestra.run.smithi043.stdout: "mds": "allow", 2023-08-09T13:51:11.745 INFO:teuthology.orchestra.run.smithi043.stdout: "mgr": "allow r", 2023-08-09T13:51:11.745 INFO:teuthology.orchestra.run.smithi043.stdout: "mon": "allow rw", 2023-08-09T13:51:11.746 INFO:teuthology.orchestra.run.smithi043.stdout: "osd": "allow rwx" 2023-08-09T13:51:11.746 INFO:teuthology.orchestra.run.smithi043.stdout: } 2023-08-09T13:51:11.746 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2023-08-09T13:51:11.746 INFO:teuthology.orchestra.run.smithi043.stdout: { 2023-08-09T13:51:11.746 INFO:teuthology.orchestra.run.smithi043.stdout: "entity": "client.2", 2023-08-09T13:51:11.746 INFO:teuthology.orchestra.run.smithi043.stdout: "key": "AQCPldNkC2lgGxAAZ8qZboOCUaQlcpkPL1ozHA==", 2023-08-09T13:51:11.746 INFO:teuthology.orchestra.run.smithi043.stdout: "caps": { 2023-08-09T13:51:11.746 INFO:teuthology.orchestra.run.smithi043.stdout: "mds": "allow", 2023-08-09T13:51:11.747 INFO:teuthology.orchestra.run.smithi043.stdout: "mgr": "allow r", 2023-08-09T13:51:11.747 INFO:teuthology.orchestra.run.smithi043.stdout: "mon": "allow rw", 2023-08-09T13:51:11.747 INFO:teuthology.orchestra.run.smithi043.stdout: "osd": "allow rwx" 2023-08-09T13:51:11.747 INFO:teuthology.orchestra.run.smithi043.stdout: } 2023-08-09T13:51:11.747 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2023-08-09T13:51:11.747 INFO:teuthology.orchestra.run.smithi043.stdout: { 2023-08-09T13:51:11.747 INFO:teuthology.orchestra.run.smithi043.stdout: "entity": "client.3", 2023-08-09T13:51:11.747 INFO:teuthology.orchestra.run.smithi043.stdout: "key": "AQCPldNkl+rXMhAA9rf3KdnSYbvngqmLYnIGGA==", 2023-08-09T13:51:11.748 INFO:teuthology.orchestra.run.smithi043.stdout: "caps": { 2023-08-09T13:51:11.748 INFO:teuthology.orchestra.run.smithi043.stdout: "mds": "allow", 2023-08-09T13:51:11.748 INFO:teuthology.orchestra.run.smithi043.stdout: "mgr": "allow r", 2023-08-09T13:51:11.748 INFO:teuthology.orchestra.run.smithi043.stdout: "mon": "allow rw", 2023-08-09T13:51:11.748 INFO:teuthology.orchestra.run.smithi043.stdout: "osd": "allow rwx" 2023-08-09T13:51:11.748 INFO:teuthology.orchestra.run.smithi043.stdout: } 2023-08-09T13:51:11.748 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2023-08-09T13:51:11.748 INFO:teuthology.orchestra.run.smithi043.stdout: { 2023-08-09T13:51:11.749 INFO:teuthology.orchestra.run.smithi043.stdout: "entity": "client.admin", 2023-08-09T13:51:11.749 INFO:teuthology.orchestra.run.smithi043.stdout: "key": "AQCMldNkUae6NRAA2FBTr0QPBdKNDoe1WK5zyg==", 2023-08-09T13:51:11.749 INFO:teuthology.orchestra.run.smithi043.stdout: "caps": { 2023-08-09T13:51:11.749 INFO:teuthology.orchestra.run.smithi043.stdout: "mds": "allow *", 2023-08-09T13:51:11.749 INFO:teuthology.orchestra.run.smithi043.stdout: "mgr": "allow *", 2023-08-09T13:51:11.749 INFO:teuthology.orchestra.run.smithi043.stdout: "mon": "allow *", 2023-08-09T13:51:11.749 INFO:teuthology.orchestra.run.smithi043.stdout: "osd": "allow *" 2023-08-09T13:51:11.750 INFO:teuthology.orchestra.run.smithi043.stdout: } 2023-08-09T13:51:11.750 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2023-08-09T13:51:11.750 INFO:teuthology.orchestra.run.smithi043.stdout: { 2023-08-09T13:51:11.750 INFO:teuthology.orchestra.run.smithi043.stdout: "entity": "mgr.x", 2023-08-09T13:51:11.750 INFO:teuthology.orchestra.run.smithi043.stdout: "key": "AQCNldNk2ufSNxAAU7msRkwLaoLZHTIsk0ck/g==", 2023-08-09T13:51:11.750 INFO:teuthology.orchestra.run.smithi043.stdout: "caps": { 2023-08-09T13:51:11.750 INFO:teuthology.orchestra.run.smithi043.stdout: "mds": "allow *", 2023-08-09T13:51:11.751 INFO:teuthology.orchestra.run.smithi043.stdout: "mon": "allow profile mgr", 2023-08-09T13:51:11.751 INFO:teuthology.orchestra.run.smithi043.stdout: "osd": "allow *" 2023-08-09T13:51:11.751 INFO:teuthology.orchestra.run.smithi043.stdout: } 2023-08-09T13:51:11.751 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2023-08-09T13:51:11.751 INFO:teuthology.orchestra.run.smithi043.stdout: { 2023-08-09T13:51:11.751 INFO:teuthology.orchestra.run.smithi043.stdout: "entity": "mgr.y", 2023-08-09T13:51:11.751 INFO:teuthology.orchestra.run.smithi043.stdout: "key": "AQCNldNk2YgzLBAAA11Q8uAMEgE/bUni+tXIKA==", 2023-08-09T13:51:11.751 INFO:teuthology.orchestra.run.smithi043.stdout: "caps": { 2023-08-09T13:51:11.752 INFO:teuthology.orchestra.run.smithi043.stdout: "mds": "allow *", 2023-08-09T13:51:11.752 INFO:teuthology.orchestra.run.smithi043.stdout: "mon": "allow profile mgr", 2023-08-09T13:51:11.752 INFO:teuthology.orchestra.run.smithi043.stdout: "osd": "allow *" 2023-08-09T13:51:11.752 INFO:teuthology.orchestra.run.smithi043.stdout: } 2023-08-09T13:51:11.752 INFO:teuthology.orchestra.run.smithi043.stdout: } 2023-08-09T13:51:11.752 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2023-08-09T13:51:11.752 INFO:teuthology.orchestra.run.smithi043.stdout:} 2023-08-09T13:51:11.754 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:11.726+0000 7fa5b1f86640 1 -- 172.21.15.43:0/3376720483 >> [v2:172.21.15.203:6832/1778614695,v1:172.21.15.203:6833/1778614695] conn(0x7fa59006a170 msgr2=0x7fa59006c630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:51:11.754 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:11.726+0000 7fa5b1f86640 1 --2- 172.21.15.43:0/3376720483 >> [v2:172.21.15.203:6832/1778614695,v1:172.21.15.203:6833/1778614695] conn(0x7fa59006a170 0x7fa59006c630 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7fa598034000 tx=0x7fa59802f000 comp rx=0 tx=0).stop 2023-08-09T13:51:11.754 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:11.726+0000 7fa5b1f86640 1 -- 172.21.15.43:0/3376720483 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa5ac160380 msgr2=0x7fa5ac15e320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:51:11.754 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:11.726+0000 7fa5b1f86640 1 --2- 172.21.15.43:0/3376720483 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa5ac160380 0x7fa5ac15e320 secure :-1 s=READY pgs=1156 cs=0 l=1 rev1=1 crypto rx=0x7fa5a000c780 tx=0x7fa5a000cc40 comp rx=0 tx=0).stop 2023-08-09T13:51:11.754 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:11.726+0000 7fa5b1f86640 1 -- 172.21.15.43:0/3376720483 shutdown_connections 2023-08-09T13:51:11.755 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:11.726+0000 7fa5b1f86640 1 --2- 172.21.15.43:0/3376720483 >> [v2:172.21.15.203:6832/1778614695,v1:172.21.15.203:6833/1778614695] conn(0x7fa59006a170 0x7fa59006c630 unknown :-1 s=CLOSED pgs=47 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:51:11.755 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:11.726+0000 7fa5b1f86640 1 --2- 172.21.15.43:0/3376720483 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fa5ac160cb0 0x7fa5ac145d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:51:11.755 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:11.726+0000 7fa5b1f86640 1 --2- 172.21.15.43:0/3376720483 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fa5ac160760 0x7fa5ac098a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:51:11.755 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:11.726+0000 7fa5b1f86640 1 --2- 172.21.15.43:0/3376720483 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa5ac160380 0x7fa5ac15e320 unknown :-1 s=CLOSED pgs=1156 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:51:11.755 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:11.726+0000 7fa5b1f86640 1 -- 172.21.15.43:0/3376720483 >> 172.21.15.43:0/3376720483 conn(0x7fa5ac0cde90 msgr2=0x7fa5ac0ca910 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-09T13:51:11.755 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:11.726+0000 7fa5b1f86640 1 -- 172.21.15.43:0/3376720483 shutdown_connections 2023-08-09T13:51:11.755 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:11.726+0000 7fa5b1f86640 1 -- 172.21.15.43:0/3376720483 wait complete. 2023-08-09T13:51:11.756 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-08-09T13:51:11.980 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:11.978+0000 7f69a63d2640 1 Processor -- start 2023-08-09T13:51:11.981 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:11.978+0000 7f69a63d2640 1 -- start start 2023-08-09T13:51:11.982 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:11.980+0000 7f69a63d2640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f69a015a340 0x7f69a00cac60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:51:11.982 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:11.980+0000 7f69a63d2640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f69a00ce620 0x7f69a00ceab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:51:11.982 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:11.980+0000 7f69a63d2640 1 -- --> v1:172.21.15.203:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f69a015a720 con 0x7f69a00cb1a0 2023-08-09T13:51:11.982 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:11.980+0000 7f69a63d2640 1 -- --> v2:172.21.15.43:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f69a00cd970 con 0x7f69a00ce620 2023-08-09T13:51:11.983 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:11.980+0000 7f69a63d2640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f69a00b0190 con 0x7f69a015a340 2023-08-09T13:51:11.983 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:11.980+0000 7f69a5bd1640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f69a00ce620 0x7f69a00ceab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:51:11.984 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:11.980+0000 7f69a5bd1640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f69a00ce620 0x7f69a00ceab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:36670/0 (socket says 172.21.15.43:36670) 2023-08-09T13:51:11.984 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:11.980+0000 7f69a5bd1640 1 -- 172.21.15.43:0/1457036932 learned_addr learned my addr 172.21.15.43:0/1457036932 (peer_addr_for_me v2:172.21.15.43:0/0) 2023-08-09T13:51:11.984 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:11.981+0000 7f69a5bd1640 1 -- 172.21.15.43:0/1457036932 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f69a015a340 msgr2=0x7f69a00cac60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-09T13:51:11.984 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:11.981+0000 7f69a53d0640 1 --2- 172.21.15.43:0/1457036932 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f69a015a340 0x7f69a00cac60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:51:11.984 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:11.981+0000 7f69a5bd1640 1 --2- 172.21.15.43:0/1457036932 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f69a015a340 0x7f69a00cac60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:51:11.984 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:11.981+0000 7f69a5bd1640 1 -- 172.21.15.43:0/1457036932 >> v1:172.21.15.203:6789/0 conn(0x7f69a00cb1a0 legacy=0x7f69a00cc980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-09T13:51:11.985 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:11.981+0000 7f69a5bd1640 1 -- 172.21.15.43:0/1457036932 --> v2:172.21.15.43:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f69a00ceff0 con 0x7f69a00ce620 2023-08-09T13:51:11.985 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:11.982+0000 7f69a5bd1640 1 --2- 172.21.15.43:0/1457036932 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f69a00ce620 0x7f69a00ceab0 secure :-1 s=READY pgs=1157 cs=0 l=1 rev1=1 crypto rx=0x7f6998009870 tx=0x7f699802b7e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8ca39ebbe1887226 server_cookie=0 in_seq=0 out_seq=0 2023-08-09T13:51:11.986 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:11.982+0000 7f6997fff640 1 -- 172.21.15.43:0/1457036932 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6998031740 con 0x7f69a00ce620 2023-08-09T13:51:11.986 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:11.982+0000 7f6997fff640 1 -- 172.21.15.43:0/1457036932 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f69980318a0 con 0x7f69a00ce620 2023-08-09T13:51:11.986 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:11.982+0000 7f6997fff640 1 -- 172.21.15.43:0/1457036932 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6998031c30 con 0x7f69a00ce620 2023-08-09T13:51:11.986 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:11.983+0000 7f69a63d2640 1 -- 172.21.15.43:0/1457036932 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f69a00ce620 msgr2=0x7f69a00ceab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:51:11.986 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:11.983+0000 7f69a63d2640 1 --2- 172.21.15.43:0/1457036932 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f69a00ce620 0x7f69a00ceab0 secure :-1 s=READY pgs=1157 cs=0 l=1 rev1=1 crypto rx=0x7f6998009870 tx=0x7f699802b7e0 comp rx=0 tx=0).stop 2023-08-09T13:51:11.986 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:11.983+0000 7f69a63d2640 1 -- 172.21.15.43:0/1457036932 shutdown_connections 2023-08-09T13:51:11.987 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:11.983+0000 7f69a63d2640 1 --2- 172.21.15.43:0/1457036932 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f69a00ce620 0x7f69a00ceab0 unknown :-1 s=CLOSED pgs=1157 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:51:11.987 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:11.983+0000 7f69a63d2640 1 --2- 172.21.15.43:0/1457036932 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f69a015a340 0x7f69a00cac60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:51:11.987 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:11.983+0000 7f69a63d2640 1 -- 172.21.15.43:0/1457036932 >> 172.21.15.43:0/1457036932 conn(0x7f69a0089fe0 msgr2=0x7f69a0099470 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-09T13:51:11.987 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:11.983+0000 7f69a63d2640 1 -- 172.21.15.43:0/1457036932 shutdown_connections 2023-08-09T13:51:11.987 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:11.983+0000 7f69a63d2640 1 -- 172.21.15.43:0/1457036932 wait complete. 2023-08-09T13:51:11.988 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:11.985+0000 7f69a63d2640 1 Processor -- start 2023-08-09T13:51:11.988 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:11.985+0000 7f69a63d2640 1 -- start start 2023-08-09T13:51:11.988 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:11.986+0000 7f69a63d2640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f69a00cb1a0 0x7f69a006a8d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:51:11.989 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:11.986+0000 7f69a63d2640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f69a00ce620 0x7f69a006bd10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:51:11.989 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:11.986+0000 7f69a63d2640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f69a015a340 0x7f69a014e6d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:51:11.989 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:11.986+0000 7f69a63d2640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f69a00c4770 con 0x7f69a015a340 2023-08-09T13:51:11.989 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:11.986+0000 7f69a63d2640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f69a015aa80 con 0x7f69a00cb1a0 2023-08-09T13:51:11.989 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:11.986+0000 7f69a63d2640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f69a006b870 con 0x7f69a00ce620 2023-08-09T13:51:11.990 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:11.986+0000 7f69a5bd1640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f69a015a340 0x7f69a014e6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:51:11.990 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:11.986+0000 7f69a5bd1640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f69a015a340 0x7f69a014e6d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:36680/0 (socket says 172.21.15.43:36680) 2023-08-09T13:51:11.990 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:11.986+0000 7f69a5bd1640 1 -- 172.21.15.43:0/4075643717 learned_addr learned my addr 172.21.15.43:0/4075643717 (peer_addr_for_me v2:172.21.15.43:0/0) 2023-08-09T13:51:11.991 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:11.987+0000 7f69a53d0640 1 --2- 172.21.15.43:0/4075643717 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f69a00cb1a0 0x7f69a006a8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:51:11.991 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:11.987+0000 7f69a4bcf640 1 --2- 172.21.15.43:0/4075643717 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f69a00ce620 0x7f69a006bd10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:51:11.992 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:11.987+0000 7f69a5bd1640 1 -- 172.21.15.43:0/4075643717 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f69a00ce620 msgr2=0x7f69a006bd10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:51:11.992 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:11.987+0000 7f69a5bd1640 1 --2- 172.21.15.43:0/4075643717 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f69a00ce620 0x7f69a006bd10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:51:11.992 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:11.987+0000 7f69a5bd1640 1 -- 172.21.15.43:0/4075643717 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f69a00cb1a0 msgr2=0x7f69a006a8d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:51:11.992 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:11.987+0000 7f69a5bd1640 1 --2- 172.21.15.43:0/4075643717 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f69a00cb1a0 0x7f69a006a8d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:51:11.992 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:11.987+0000 7f69a5bd1640 1 -- 172.21.15.43:0/4075643717 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f69980094d0 con 0x7f69a015a340 2023-08-09T13:51:11.992 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:11.987+0000 7f69a5bd1640 1 --2- 172.21.15.43:0/4075643717 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f69a015a340 0x7f69a014e6d0 secure :-1 s=READY pgs=1158 cs=0 l=1 rev1=1 crypto rx=0x7f699802b820 tx=0x7f69980046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-09T13:51:11.993 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:11.987+0000 7f6995ffb640 1 -- 172.21.15.43:0/4075643717 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6998033d60 con 0x7f69a015a340 2023-08-09T13:51:11.993 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:11.987+0000 7f6995ffb640 1 -- 172.21.15.43:0/4075643717 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f699803c400 con 0x7f69a015a340 2023-08-09T13:51:11.993 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:11.987+0000 7f6995ffb640 1 -- 172.21.15.43:0/4075643717 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f699803c7b0 con 0x7f69a015a340 2023-08-09T13:51:11.993 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:11.987+0000 7f69a63d2640 1 -- 172.21.15.43:0/4075643717 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f69a006afd0 con 0x7f69a015a340 2023-08-09T13:51:11.993 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:11.988+0000 7f69a63d2640 1 -- 172.21.15.43:0/4075643717 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f69a006b590 con 0x7f69a015a340 2023-08-09T13:51:11.993 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:11.989+0000 7f69a63d2640 1 -- 172.21.15.43:0/4075643717 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f69a015ac10 con 0x7f69a015a340 2023-08-09T13:51:11.994 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:11.990+0000 7f6995ffb640 1 -- 172.21.15.43:0/4075643717 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 37) v1 ==== 88443+0+0 (secure 0 0 0) 0x7f699803a040 con 0x7f69a015a340 2023-08-09T13:51:11.994 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:11.991+0000 7f6995ffb640 1 --2- 172.21.15.43:0/4075643717 >> [v2:172.21.15.203:6832/1778614695,v1:172.21.15.203:6833/1778614695] conn(0x7f698406a170 0x7f698406c630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:51:11.994 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:11.991+0000 7f6995ffb640 1 -- 172.21.15.43:0/4075643717 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(99..99 src has 1..99) v4 ==== 5574+0+0 (secure 0 0 0) 0x7f69980aa010 con 0x7f69a015a340 2023-08-09T13:51:11.994 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:11.991+0000 7f69a53d0640 1 --2- 172.21.15.43:0/4075643717 >> [v2:172.21.15.203:6832/1778614695,v1:172.21.15.203:6833/1778614695] conn(0x7f698406a170 0x7f698406c630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:51:11.996 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:11.992+0000 7f69a53d0640 1 --2- 172.21.15.43:0/4075643717 >> [v2:172.21.15.203:6832/1778614695,v1:172.21.15.203:6833/1778614695] conn(0x7f698406a170 0x7f698406c630 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7f69a00c4560 tx=0x7f699c0083c0 comp rx=0 tx=0).ready entity=mgr.5819 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-09T13:51:11.997 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:11.995+0000 7f6995ffb640 1 -- 172.21.15.43:0/4075643717 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178066 (secure 0 0 0) 0x7f6998075870 con 0x7f69a015a340 2023-08-09T13:51:12.204 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:12.202+0000 7f69a63d2640 1 -- 172.21.15.43:0/4075643717 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f69a015a720 con 0x7f69a015a340 2023-08-09T13:51:12.206 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:12.204+0000 7f6995ffb640 1 -- 172.21.15.43:0/4075643717 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v99) v1 ==== 74+0+14845 (secure 0 0 0) 0x7f69a015a720 con 0x7f69a015a340 2023-08-09T13:51:12.207 INFO:teuthology.orchestra.run.smithi043.stdout: 2023-08-09T13:51:12.207 INFO:teuthology.orchestra.run.smithi043.stdout:{"epoch":99,"fsid":"daac87ee-1337-4250-a955-aea4e408a39b","created":"2023-08-09T13:33:41.701198+0000","modified":"2023-08-09T13:51:11.189713+0000","last_up_change":"2023-08-09T13:33:53.783453+0000","last_in_change":"2023-08-09T13:33:45.148904+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":11,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":"rbd","create_time":"2023-08-09T13:33:54.365137+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":".mgr","create_time":"2023-08-09T13:33:54.831803+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"cbd49ecd-e896-4fbd-864e-5c37a829b165","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6800","nonce":502667335},{"type":"v1","addr":"172.21.15.43:6801","nonce":502667335}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6802","nonce":502667335},{"type":"v1","addr":"172.21.15.43:6803","nonce":502667335}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6806","nonce":502667335},{"type":"v1","addr":"172.21.15.43:6807","nonce":502667335}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6804","nonce":502667335},{"type":"v1","addr":"172.21.15.43:6805","nonce":502667335}]},"public_addr":"172.21.15.43:6801/502667335","cluster_addr":"172.21.15.43:6803/502667335","heartbeat_back_addr":"172.21.15.43:6807/502667335","heartbeat_front_addr":"172.21.15.43:6805/502667335","state":["exists","up"]},{"osd":1,"uuid":"75c7e0fe-1ec7-4fe8-a2f2-3661a65b8500","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6808","nonce":3289134610},{"type":"v1","addr":"172.21.15.43:6809","nonce":3289134610}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6810","nonce":3289134610},{"type":"v1","addr":"172.21.15.43:6812","nonce":3289134610}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6821","nonce":3289134610},{"type":"v1","addr":"172.21.15.43:6824","nonce":3289134610}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6815","nonce":3289134610},{"type":"v1","addr":"172.21.15.43:6818","nonce":3289134610}]},"public_addr":"172.21.15.43:6809/3289134610","cluster_addr":"172.21.15.43:6812/3289134610","heartbeat_back_addr":"172.21.15.43:6824/3289134610","heartbeat_front_addr":"172.21.15.43:6818/3289134610","state":["exists","up"]},{"osd":2,"uuid":"bb5c41e9-7f5f-4125-9b01-3bb8bfc2ea83","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6811","nonce":4079709736},{"type":"v1","addr":"172.21.15.43:6814","nonce":4079709736}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6817","nonce":4079709736},{"type":"v1","addr":"172.21.15.43:6820","nonce":4079709736}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6829","nonce":4079709736},{"type":"v1","addr":"172.21.15.43:6831","nonce":4079709736}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6823","nonce":4079709736},{"type":"v1","addr":"172.21.15.43:6826","nonce":4079709736}]},"public_addr":"172.21.15.43:6814/4079709736","cluster_addr":"172.21.15.43:6820/4079709736","heartbeat_back_addr":"172.21.15.43:6831/4079709736","heartbeat_front_addr":"172.21.15.43:6826/4079709736","state":["exists","up"]},{"osd":3,"uuid":"bfe4c8b4-694a-465c-87e6-2b2aab852613","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6813","nonce":1709538759},{"type":"v1","addr":"172.21.15.43:6816","nonce":1709538759}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6819","nonce":1709538759},{"type":"v1","addr":"172.21.15.43:6822","nonce":1709538759}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6828","nonce":1709538759},{"type":"v1","addr":"172.21.15.43:6830","nonce":1709538759}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6825","nonce":1709538759},{"type":"v1","addr":"172.21.15.43:6827","nonce":1709538759}]},"public_addr":"172.21.15.43:6816/1709538759","cluster_addr":"172.21.15.43:6822/1709538759","heartbeat_back_addr":"172.21.15.43:6830/1709538759","heartbeat_front_addr":"172.21.15.43:6827/1709538759","state":["exists","up"]},{"osd":4,"uuid":"fa2e8a77-aba1-4756-853e-0d174a5bdeba","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6800","nonce":4196866481},{"type":"v1","addr":"172.21.15.203:6801","nonce":4196866481}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6802","nonce":4196866481},{"type":"v1","addr":"172.21.15.203:6803","nonce":4196866481}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6806","nonce":4196866481},{"type":"v1","addr":"172.21.15.203:6807","nonce":4196866481}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6804","nonce":4196866481},{"type":"v1","addr":"172.21.15.203:6805","nonce":4196866481}]},"public_addr":"172.21.15.203:6801/4196866481","cluster_addr":"172.21.15.203:6803/4196866481","heartbeat_back_addr":"172.21.15.203:6807/4196866481","heartbeat_front_addr":"172.21.15.203:6805/4196866481","state":["exists","up"]},{"osd":5,"uuid":"81811e4b-4d24-4f51-a82c-f1d700a2a00d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6816","nonce":3059290081},{"type":"v1","addr":"172.21.15.203:6817","nonce":3059290081}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6818","nonce":3059290081},{"type":"v1","addr":"172.21.15.203:6819","nonce":3059290081}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6822","nonce":3059290081},{"type":"v1","addr":"172.21.15.203:6823","nonce":3059290081}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6820","nonce":3059290081},{"type":"v1","addr":"172.21.15.203:6821","nonce":3059290081}]},"public_addr":"172.21.15.203:6817/3059290081","cluster_addr":"172.21.15.203:6819/3059290081","heartbeat_back_addr":"172.21.15.203:6823/3059290081","heartbeat_front_addr":"172.21.15.203:6821/3059290081","state":["exists","up"]},{"osd":6,"uuid":"307bac4d-6a5b-443d-af75-56d695d7674f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6824","nonce":512155569},{"type":"v1","addr":"172.21.15.203:6825","nonce":512155569}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6826","nonce":512155569},{"type":"v1","addr":"172.21.15.203:6827","nonce":512155569}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6830","nonce":512155569},{"type":"v1","addr":"172.21.15.203:6831","nonce":512155569}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6828","nonce":512155569},{"type":"v1","addr":"172.21.15.203:6829","nonce":512155569}]},"public_addr":"172.21.15.203:6825/512155569","cluster_addr":"172.21.15.203:6827/512155569","heartbeat_back_addr":"172.21.15.203:6831/512155569","heartbeat_front_addr":"172.21.15.203:6829/512155569","state":["exists","up"]},{"osd":7,"uuid":"a95dc7ca-9f1b-4295-af70-68a22a2d7d5e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6808","nonce":3331450438},{"type":"v1","addr":"172.21.15.203:6809","nonce":3331450438}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6810","nonce":3331450438},{"type":"v1","addr":"172.21.15.203:6811","nonce":3331450438}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6814","nonce":3331450438},{"type":"v1","addr":"172.21.15.203:6815","nonce":3331450438}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6812","nonce":3331450438},{"type":"v1","addr":"172.21.15.203:6813","nonce":3331450438}]},"public_addr":"172.21.15.203:6809/3331450438","cluster_addr":"172.21.15.203:6811/3331450438","heartbeat_back_addr":"172.21.15.203:6815/3331450438","heartbeat_front_addr":"172.21.15.203:6813/3331450438","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-09T13:33:50.547652+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-09T13:33:50.589469+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-09T13:33:50.561724+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-09T13:33:50.599650+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-09T13:33:50.411855+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-09T13:33:50.219681+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-09T13:33:50.504664+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-09T13:33:50.441248+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":7,"snaps":[{"begin":104,"length":1},{"begin":221,"length":1}]}],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-08-09T13:51:12.209 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:12.206+0000 7f69a63d2640 1 -- 172.21.15.43:0/4075643717 >> [v2:172.21.15.203:6832/1778614695,v1:172.21.15.203:6833/1778614695] conn(0x7f698406a170 msgr2=0x7f698406c630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:51:12.209 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:12.206+0000 7f69a63d2640 1 --2- 172.21.15.43:0/4075643717 >> [v2:172.21.15.203:6832/1778614695,v1:172.21.15.203:6833/1778614695] conn(0x7f698406a170 0x7f698406c630 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7f69a00c4560 tx=0x7f699c0083c0 comp rx=0 tx=0).stop 2023-08-09T13:51:12.210 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:12.206+0000 7f69a63d2640 1 -- 172.21.15.43:0/4075643717 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f69a015a340 msgr2=0x7f69a014e6d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:51:12.210 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:12.206+0000 7f69a63d2640 1 --2- 172.21.15.43:0/4075643717 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f69a015a340 0x7f69a014e6d0 secure :-1 s=READY pgs=1158 cs=0 l=1 rev1=1 crypto rx=0x7f699802b820 tx=0x7f69980046c0 comp rx=0 tx=0).stop 2023-08-09T13:51:12.210 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:12.207+0000 7f69a63d2640 1 -- 172.21.15.43:0/4075643717 shutdown_connections 2023-08-09T13:51:12.210 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:12.207+0000 7f69a63d2640 1 --2- 172.21.15.43:0/4075643717 >> [v2:172.21.15.203:6832/1778614695,v1:172.21.15.203:6833/1778614695] conn(0x7f698406a170 0x7f698406c630 unknown :-1 s=CLOSED pgs=48 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:51:12.210 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:12.207+0000 7f69a63d2640 1 --2- 172.21.15.43:0/4075643717 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f69a015a340 0x7f69a014e6d0 unknown :-1 s=CLOSED pgs=1158 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:51:12.210 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:12.207+0000 7f69a63d2640 1 --2- 172.21.15.43:0/4075643717 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f69a00ce620 0x7f69a006bd10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:51:12.211 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:12.207+0000 7f69a63d2640 1 --2- 172.21.15.43:0/4075643717 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f69a00cb1a0 0x7f69a006a8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:51:12.211 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:12.207+0000 7f69a63d2640 1 -- 172.21.15.43:0/4075643717 >> 172.21.15.43:0/4075643717 conn(0x7f69a0089fe0 msgr2=0x7f69a0159550 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-09T13:51:12.211 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:12.207+0000 7f69a63d2640 1 -- 172.21.15.43:0/4075643717 shutdown_connections 2023-08-09T13:51:12.211 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:12.207+0000 7f69a63d2640 1 -- 172.21.15.43:0/4075643717 wait complete. 2023-08-09T13:51:12.228 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': 'rbd', 'create_time': '2023-08-09T13:33:54.365137+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}, 'read_balance': {'score_acting': 2, 'score_stable': 2, 'optimal_score': 1, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': '.mgr', 'create_time': '2023-08-09T13:33:54.831803+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-08-09T13:51:12.229 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get rbd pg_num 2023-08-09T13:51:12.463 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:12.461+0000 7f4daafdd640 1 Processor -- start 2023-08-09T13:51:12.463 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:12.461+0000 7f4daafdd640 1 -- start start 2023-08-09T13:51:12.465 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:12.462+0000 7f4daafdd640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f4da40cf300 0x7f4da40d1b30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:51:12.465 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:12.462+0000 7f4daafdd640 1 -- --> v1:172.21.15.43:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4da40d16a0 con 0x7f4da415c2d0 2023-08-09T13:51:12.466 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:12.462+0000 7f4daafdd640 1 -- --> v1:172.21.15.203:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4da40d0d70 con 0x7f4da40b7df0 2023-08-09T13:51:12.466 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:12.462+0000 7f4daafdd640 1 -- --> v2:172.21.15.43:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4da40b8230 con 0x7f4da40cf300 2023-08-09T13:51:12.466 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:12.463+0000 7f4da9fdb640 1 --1- >> v1:172.21.15.43:6789/0 conn(0x7f4da415c2d0 0x7f4da40cd020 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.43:6789/0 says I am v1:172.21.15.43:45732/0 (socket says 172.21.15.43:45732) 2023-08-09T13:51:12.466 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:12.463+0000 7f4da9fdb640 1 -- 172.21.15.43:0/80437281 learned_addr learned my addr 172.21.15.43:0/80437281 (peer_addr_for_me v1:172.21.15.43:0/0) 2023-08-09T13:51:12.467 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:12.463+0000 7f4daa7dc640 1 --2- 172.21.15.43:0/80437281 >> v2:172.21.15.43:3300/0 conn(0x7f4da40cf300 0x7f4da40d1b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:51:12.469 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:12.463+0000 7f4daa7dc640 1 -- 172.21.15.43:0/80437281 >> v1:172.21.15.203:6789/0 conn(0x7f4da40b7df0 legacy=0x7f4da40cebf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-09T13:51:12.469 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:12.463+0000 7f4daa7dc640 1 -- 172.21.15.43:0/80437281 >> v1:172.21.15.43:6789/0 conn(0x7f4da415c2d0 legacy=0x7f4da40cd020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:51:12.469 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:12.463+0000 7f4daa7dc640 1 -- 172.21.15.43:0/80437281 --> v2:172.21.15.43:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4da40d1050 con 0x7f4da40cf300 2023-08-09T13:51:12.470 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:12.465+0000 7f4daa7dc640 1 --2- 172.21.15.43:0/80437281 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f4da40cf300 0x7f4da40d1b30 secure :-1 s=READY pgs=1160 cs=0 l=1 rev1=1 crypto rx=0x7f4d9c0097a0 tx=0x7f4d9c02b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f01e92bf0df1f0a6 server_cookie=0 in_seq=0 out_seq=0 2023-08-09T13:51:12.470 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:12.465+0000 7f4da8fd9640 1 -- 172.21.15.43:0/80437281 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4d9c031830 con 0x7f4da40cf300 2023-08-09T13:51:12.470 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:12.465+0000 7f4da8fd9640 1 -- 172.21.15.43:0/80437281 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4d9c031990 con 0x7f4da40cf300 2023-08-09T13:51:12.471 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:12.465+0000 7f4da8fd9640 1 -- 172.21.15.43:0/80437281 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4d9c031d20 con 0x7f4da40cf300 2023-08-09T13:51:12.471 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:12.466+0000 7f4daafdd640 1 -- 172.21.15.43:0/80437281 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f4da40cf300 msgr2=0x7f4da40d1b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:51:12.471 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:12.466+0000 7f4daafdd640 1 --2- 172.21.15.43:0/80437281 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f4da40cf300 0x7f4da40d1b30 secure :-1 s=READY pgs=1160 cs=0 l=1 rev1=1 crypto rx=0x7f4d9c0097a0 tx=0x7f4d9c02b8d0 comp rx=0 tx=0).stop 2023-08-09T13:51:12.471 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:12.466+0000 7f4daafdd640 1 -- 172.21.15.43:0/80437281 shutdown_connections 2023-08-09T13:51:12.472 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:12.466+0000 7f4daafdd640 1 --2- 172.21.15.43:0/80437281 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f4da40cf300 0x7f4da40d1b30 unknown :-1 s=CLOSED pgs=1160 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:51:12.472 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:12.466+0000 7f4daafdd640 1 -- 172.21.15.43:0/80437281 >> 172.21.15.43:0/80437281 conn(0x7f4da406a230 msgr2=0x7f4da406a640 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-09T13:51:12.472 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:12.466+0000 7f4daafdd640 1 -- 172.21.15.43:0/80437281 shutdown_connections 2023-08-09T13:51:12.472 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:12.466+0000 7f4daafdd640 1 -- 172.21.15.43:0/80437281 wait complete. 2023-08-09T13:51:12.475 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:12.468+0000 7f4daafdd640 1 Processor -- start 2023-08-09T13:51:12.476 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:12.468+0000 7f4daafdd640 1 -- start start 2023-08-09T13:51:12.476 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:12.469+0000 7f4daafdd640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f4da40b7df0 0x7f4da41f1c90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:51:12.476 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:12.469+0000 7f4daafdd640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f4da40cf300 0x7f4da41ef2a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:51:12.477 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:12.469+0000 7f4daafdd640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f4da415c2d0 0x7f4da40c92b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:51:12.477 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:12.469+0000 7f4daafdd640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4da40b7600 con 0x7f4da415c2d0 2023-08-09T13:51:12.477 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:12.469+0000 7f4daafdd640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4da40c99b0 con 0x7f4da40cf300 2023-08-09T13:51:12.478 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:12.469+0000 7f4daafdd640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4da40c9c90 con 0x7f4da40b7df0 2023-08-09T13:51:12.478 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:12.470+0000 7f4daa7dc640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f4da415c2d0 0x7f4da40c92b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:51:12.478 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:12.470+0000 7f4daa7dc640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f4da415c2d0 0x7f4da40c92b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:36698/0 (socket says 172.21.15.43:36698) 2023-08-09T13:51:12.478 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:12.470+0000 7f4daa7dc640 1 -- 172.21.15.43:0/103128433 learned_addr learned my addr 172.21.15.43:0/103128433 (peer_addr_for_me v2:172.21.15.43:0/0) 2023-08-09T13:51:12.479 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:12.470+0000 7f4da9fdb640 1 --2- 172.21.15.43:0/103128433 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f4da40b7df0 0x7f4da41f1c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:51:12.479 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:12.470+0000 7f4da97da640 1 --2- 172.21.15.43:0/103128433 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f4da40cf300 0x7f4da41ef2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:51:12.479 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:12.470+0000 7f4daa7dc640 1 -- 172.21.15.43:0/103128433 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f4da40b7df0 msgr2=0x7f4da41f1c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:51:12.479 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:12.470+0000 7f4daa7dc640 1 --2- 172.21.15.43:0/103128433 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f4da40b7df0 0x7f4da41f1c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:51:12.480 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:12.470+0000 7f4daa7dc640 1 -- 172.21.15.43:0/103128433 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f4da40cf300 msgr2=0x7f4da41ef2a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:51:12.480 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:12.470+0000 7f4daa7dc640 1 --2- 172.21.15.43:0/103128433 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f4da40cf300 0x7f4da41ef2a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:51:12.480 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:12.470+0000 7f4daa7dc640 1 -- 172.21.15.43:0/103128433 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4d9c009400 con 0x7f4da415c2d0 2023-08-09T13:51:12.481 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:12.470+0000 7f4da9fdb640 1 --2- 172.21.15.43:0/103128433 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f4da40b7df0 0x7f4da41f1c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-08-09T13:51:12.481 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:12.470+0000 7f4da97da640 1 --2- 172.21.15.43:0/103128433 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f4da40cf300 0x7f4da41ef2a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-08-09T13:51:12.481 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:12.471+0000 7f4daa7dc640 1 --2- 172.21.15.43:0/103128433 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f4da415c2d0 0x7f4da40c92b0 secure :-1 s=READY pgs=1161 cs=0 l=1 rev1=1 crypto rx=0x7f4d9c0098c0 tx=0x7f4d9c0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-09T13:51:12.481 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:12.471+0000 7f4d9a7fc640 1 -- 172.21.15.43:0/103128433 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4d9c03b9f0 con 0x7f4da415c2d0 2023-08-09T13:51:12.482 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:12.471+0000 7f4d9a7fc640 1 -- 172.21.15.43:0/103128433 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4d9c03bb50 con 0x7f4da415c2d0 2023-08-09T13:51:12.482 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:12.471+0000 7f4d9a7fc640 1 -- 172.21.15.43:0/103128433 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4d9c0453f0 con 0x7f4da415c2d0 2023-08-09T13:51:12.482 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:12.471+0000 7f4daafdd640 1 -- 172.21.15.43:0/103128433 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4da40c9f70 con 0x7f4da415c2d0 2023-08-09T13:51:12.482 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:12.471+0000 7f4daafdd640 1 -- 172.21.15.43:0/103128433 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4da40ca530 con 0x7f4da415c2d0 2023-08-09T13:51:12.483 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:12.473+0000 7f4daafdd640 1 -- 172.21.15.43:0/103128433 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4da40ca720 con 0x7f4da415c2d0 2023-08-09T13:51:12.484 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:12.473+0000 7f4d9a7fc640 1 -- 172.21.15.43:0/103128433 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 37) v1 ==== 88443+0+0 (secure 0 0 0) 0x7f4d9c045550 con 0x7f4da415c2d0 2023-08-09T13:51:12.485 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:12.474+0000 7f4d9a7fc640 1 --2- 172.21.15.43:0/103128433 >> [v2:172.21.15.203:6832/1778614695,v1:172.21.15.203:6833/1778614695] conn(0x7f4d8806a170 0x7f4d8806c630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:51:12.485 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:12.475+0000 7f4d9a7fc640 1 -- 172.21.15.43:0/103128433 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(99..99 src has 1..99) v4 ==== 5574+0+0 (secure 0 0 0) 0x7f4d9c0aa960 con 0x7f4da415c2d0 2023-08-09T13:51:12.485 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:12.475+0000 7f4da9fdb640 1 --2- 172.21.15.43:0/103128433 >> [v2:172.21.15.203:6832/1778614695,v1:172.21.15.203:6833/1778614695] conn(0x7f4d8806a170 0x7f4d8806c630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:51:12.485 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:12.476+0000 7f4da9fdb640 1 --2- 172.21.15.43:0/103128433 >> [v2:172.21.15.203:6832/1778614695,v1:172.21.15.203:6833/1778614695] conn(0x7f4d8806a170 0x7f4d8806c630 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7f4da0005c00 tx=0x7f4da0009040 comp rx=0 tx=0).ready entity=mgr.5819 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-09T13:51:12.486 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:12.479+0000 7f4d9a7fc640 1 -- 172.21.15.43:0/103128433 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178066 (secure 0 0 0) 0x7f4d9c075f70 con 0x7f4da415c2d0 2023-08-09T13:51:12.674 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:12.672+0000 7f4daafdd640 1 -- 172.21.15.43:0/103128433 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"} v 0) v1 -- 0x7f4da404e390 con 0x7f4da415c2d0 2023-08-09T13:51:12.675 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:12.673+0000 7f4d9a7fc640 1 -- 172.21.15.43:0/103128433 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"}]=0 v99) v1 ==== 92+0+10 (secure 0 0 0) 0x7f4d9c0350a0 con 0x7f4da415c2d0 2023-08-09T13:51:12.676 INFO:teuthology.orchestra.run.smithi043.stdout:pg_num: 8 2023-08-09T13:51:12.678 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:12.675+0000 7f4daafdd640 1 -- 172.21.15.43:0/103128433 >> [v2:172.21.15.203:6832/1778614695,v1:172.21.15.203:6833/1778614695] conn(0x7f4d8806a170 msgr2=0x7f4d8806c630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:51:12.678 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:12.675+0000 7f4daafdd640 1 --2- 172.21.15.43:0/103128433 >> [v2:172.21.15.203:6832/1778614695,v1:172.21.15.203:6833/1778614695] conn(0x7f4d8806a170 0x7f4d8806c630 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7f4da0005c00 tx=0x7f4da0009040 comp rx=0 tx=0).stop 2023-08-09T13:51:12.679 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:12.675+0000 7f4daafdd640 1 -- 172.21.15.43:0/103128433 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f4da415c2d0 msgr2=0x7f4da40c92b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:51:12.679 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:12.675+0000 7f4daafdd640 1 --2- 172.21.15.43:0/103128433 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f4da415c2d0 0x7f4da40c92b0 secure :-1 s=READY pgs=1161 cs=0 l=1 rev1=1 crypto rx=0x7f4d9c0098c0 tx=0x7f4d9c0046c0 comp rx=0 tx=0).stop 2023-08-09T13:51:12.679 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:12.676+0000 7f4daafdd640 1 -- 172.21.15.43:0/103128433 shutdown_connections 2023-08-09T13:51:12.679 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:12.676+0000 7f4daafdd640 1 --2- 172.21.15.43:0/103128433 >> [v2:172.21.15.203:6832/1778614695,v1:172.21.15.203:6833/1778614695] conn(0x7f4d8806a170 0x7f4d8806c630 unknown :-1 s=CLOSED pgs=52 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:51:12.679 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:12.676+0000 7f4daafdd640 1 --2- 172.21.15.43:0/103128433 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f4da415c2d0 0x7f4da40c92b0 unknown :-1 s=CLOSED pgs=1161 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:51:12.679 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:12.676+0000 7f4daafdd640 1 --2- 172.21.15.43:0/103128433 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f4da40cf300 0x7f4da41ef2a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:51:12.679 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:12.676+0000 7f4daafdd640 1 --2- 172.21.15.43:0/103128433 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f4da40b7df0 0x7f4da41f1c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:51:12.680 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:12.676+0000 7f4daafdd640 1 -- 172.21.15.43:0/103128433 >> 172.21.15.43:0/103128433 conn(0x7f4da406a230 msgr2=0x7f4da415e290 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-09T13:51:12.680 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:12.676+0000 7f4daafdd640 1 -- 172.21.15.43:0/103128433 shutdown_connections 2023-08-09T13:51:12.680 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:12.676+0000 7f4daafdd640 1 -- 172.21.15.43:0/103128433 wait complete. 2023-08-09T13:51:12.699 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-08-09T13:51:12.936 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:12.933+0000 7fae9cc91640 1 Processor -- start 2023-08-09T13:51:12.936 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:12.933+0000 7fae9cc91640 1 -- start start 2023-08-09T13:51:12.938 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:12.935+0000 7fae9cc91640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7fae98157ea0 0x7fae980c97d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:51:12.938 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:12.935+0000 7fae9cc91640 1 -- --> v1:172.21.15.43:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fae98158280 con 0x7fae980c90a0 2023-08-09T13:51:12.938 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:12.935+0000 7fae9cc91640 1 -- --> v1:172.21.15.203:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fae9815b560 con 0x7fae9815af10 2023-08-09T13:51:12.938 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:12.935+0000 7fae9cc91640 1 -- --> v2:172.21.15.43:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fae9815b840 con 0x7fae98157ea0 2023-08-09T13:51:12.939 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:12.935+0000 7fae8ffff640 1 --1- >> v1:172.21.15.43:6789/0 conn(0x7fae980c90a0 0x7fae9815a800 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.43:6789/0 says I am v1:172.21.15.43:45746/0 (socket says 172.21.15.43:45746) 2023-08-09T13:51:12.939 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:12.935+0000 7fae977fe640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7fae98157ea0 0x7fae980c97d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:51:12.939 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:12.935+0000 7fae8ffff640 1 -- 172.21.15.43:0/2484786840 learned_addr learned my addr 172.21.15.43:0/2484786840 (peer_addr_for_me v1:172.21.15.43:0/0) 2023-08-09T13:51:12.940 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:12.936+0000 7fae96ffd640 1 -- 172.21.15.43:0/2484786840 <== mon.0 v1:172.21.15.43:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2426305374 0 0) 0x7fae80001110 con 0x7fae980c90a0 2023-08-09T13:51:12.941 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:12.936+0000 7fae96ffd640 1 -- 172.21.15.43:0/2484786840 --> v1:172.21.15.43:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fae84003520 con 0x7fae980c90a0 2023-08-09T13:51:12.941 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:12.936+0000 7fae977fe640 1 -- 172.21.15.43:0/2484786840 >> v1:172.21.15.203:6789/0 conn(0x7fae9815af10 legacy=0x7fae980ad730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:51:12.941 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:12.936+0000 7fae977fe640 1 -- 172.21.15.43:0/2484786840 >> v1:172.21.15.43:6789/0 conn(0x7fae980c90a0 legacy=0x7fae9815a800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:51:12.941 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:12.936+0000 7fae977fe640 1 -- 172.21.15.43:0/2484786840 --> v2:172.21.15.43:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fae980c85c0 con 0x7fae98157ea0 2023-08-09T13:51:12.941 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:12.938+0000 7fae977fe640 1 --2- 172.21.15.43:0/2484786840 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fae98157ea0 0x7fae980c97d0 secure :-1 s=READY pgs=1163 cs=0 l=1 rev1=1 crypto rx=0x7fae880097a0 tx=0x7fae8802b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9eb67dd8c328da12 server_cookie=0 in_seq=0 out_seq=0 2023-08-09T13:51:12.942 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:12.938+0000 7fae96ffd640 1 -- 172.21.15.43:0/2484786840 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fae88031830 con 0x7fae98157ea0 2023-08-09T13:51:12.942 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:12.938+0000 7fae96ffd640 1 -- 172.21.15.43:0/2484786840 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fae88031990 con 0x7fae98157ea0 2023-08-09T13:51:12.942 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:12.938+0000 7fae96ffd640 1 -- 172.21.15.43:0/2484786840 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fae88031d20 con 0x7fae98157ea0 2023-08-09T13:51:12.943 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:12.939+0000 7fae9cc91640 1 -- 172.21.15.43:0/2484786840 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fae98157ea0 msgr2=0x7fae980c97d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:51:12.943 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:12.939+0000 7fae9cc91640 1 --2- 172.21.15.43:0/2484786840 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fae98157ea0 0x7fae980c97d0 secure :-1 s=READY pgs=1163 cs=0 l=1 rev1=1 crypto rx=0x7fae880097a0 tx=0x7fae8802b8d0 comp rx=0 tx=0).stop 2023-08-09T13:51:12.943 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:12.939+0000 7fae9cc91640 1 -- 172.21.15.43:0/2484786840 shutdown_connections 2023-08-09T13:51:12.943 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:12.939+0000 7fae9cc91640 1 --2- 172.21.15.43:0/2484786840 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fae98157ea0 0x7fae980c97d0 unknown :-1 s=CLOSED pgs=1163 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:51:12.943 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:12.939+0000 7fae9cc91640 1 -- 172.21.15.43:0/2484786840 >> 172.21.15.43:0/2484786840 conn(0x7fae980cddd0 msgr2=0x7fae980ce1e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-09T13:51:12.943 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:12.939+0000 7fae9cc91640 1 -- 172.21.15.43:0/2484786840 shutdown_connections 2023-08-09T13:51:12.943 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:12.939+0000 7fae9cc91640 1 -- 172.21.15.43:0/2484786840 wait complete. 2023-08-09T13:51:12.944 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:12.941+0000 7fae9cc91640 1 Processor -- start 2023-08-09T13:51:12.944 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:12.941+0000 7fae9cc91640 1 -- start start 2023-08-09T13:51:12.945 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:12.942+0000 7fae9cc91640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fae980c90a0 0x7fae980c7720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:51:12.945 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:12.942+0000 7fae9cc91640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fae98157ea0 0x7fae980c5b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:51:12.945 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:12.942+0000 7fae9cc91640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fae9815af10 0x7fae980aebc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:51:12.946 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:12.943+0000 7fae9cc91640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fae981f6600 con 0x7fae9815af10 2023-08-09T13:51:12.946 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:12.943+0000 7fae9cc91640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fae981f5730 con 0x7fae98157ea0 2023-08-09T13:51:12.946 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:12.943+0000 7fae9cc91640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fae981f5980 con 0x7fae980c90a0 2023-08-09T13:51:12.947 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:12.943+0000 7fae97fff640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fae9815af10 0x7fae980aebc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:51:12.947 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:12.943+0000 7fae97fff640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fae9815af10 0x7fae980aebc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:36722/0 (socket says 172.21.15.43:36722) 2023-08-09T13:51:12.947 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:12.943+0000 7fae97fff640 1 -- 172.21.15.43:0/2675128905 learned_addr learned my addr 172.21.15.43:0/2675128905 (peer_addr_for_me v2:172.21.15.43:0/0) 2023-08-09T13:51:12.950 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:12.943+0000 7fae8ffff640 1 --2- 172.21.15.43:0/2675128905 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fae98157ea0 0x7fae980c5b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:51:12.950 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:12.943+0000 7fae977fe640 1 --2- 172.21.15.43:0/2675128905 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fae980c90a0 0x7fae980c7720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:51:12.950 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:12.943+0000 7fae97fff640 1 -- 172.21.15.43:0/2675128905 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fae980c90a0 msgr2=0x7fae980c7720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:51:12.951 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:12.943+0000 7fae97fff640 1 --2- 172.21.15.43:0/2675128905 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fae980c90a0 0x7fae980c7720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:51:12.951 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:12.943+0000 7fae97fff640 1 -- 172.21.15.43:0/2675128905 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fae98157ea0 msgr2=0x7fae980c5b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:51:12.951 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:12.943+0000 7fae97fff640 1 --2- 172.21.15.43:0/2675128905 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fae98157ea0 0x7fae980c5b20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:51:12.951 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:12.943+0000 7fae97fff640 1 -- 172.21.15.43:0/2675128905 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fae88009400 con 0x7fae9815af10 2023-08-09T13:51:12.952 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:12.944+0000 7fae97fff640 1 --2- 172.21.15.43:0/2675128905 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fae9815af10 0x7fae980aebc0 secure :-1 s=READY pgs=1164 cs=0 l=1 rev1=1 crypto rx=0x7fae9000c6b0 tx=0x7fae9000cb70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-09T13:51:12.952 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:12.944+0000 7fae94ff9640 1 -- 172.21.15.43:0/2675128905 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fae90003b70 con 0x7fae9815af10 2023-08-09T13:51:12.952 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:12.944+0000 7fae94ff9640 1 -- 172.21.15.43:0/2675128905 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fae90003cf0 con 0x7fae9815af10 2023-08-09T13:51:12.953 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:12.944+0000 7fae94ff9640 1 -- 172.21.15.43:0/2675128905 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fae90003ea0 con 0x7fae9815af10 2023-08-09T13:51:12.953 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:12.944+0000 7fae9cc91640 1 -- 172.21.15.43:0/2675128905 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fae981f5c60 con 0x7fae9815af10 2023-08-09T13:51:12.953 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:12.944+0000 7fae9cc91640 1 -- 172.21.15.43:0/2675128905 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fae981f6220 con 0x7fae9815af10 2023-08-09T13:51:12.953 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:12.945+0000 7fae9cc91640 1 -- 172.21.15.43:0/2675128905 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fae980cbb00 con 0x7fae9815af10 2023-08-09T13:51:12.954 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:12.946+0000 7fae94ff9640 1 -- 172.21.15.43:0/2675128905 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 37) v1 ==== 88443+0+0 (secure 0 0 0) 0x7fae90005310 con 0x7fae9815af10 2023-08-09T13:51:12.954 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:12.947+0000 7fae94ff9640 1 --2- 172.21.15.43:0/2675128905 >> [v2:172.21.15.203:6832/1778614695,v1:172.21.15.203:6833/1778614695] conn(0x7fae7806a1a0 0x7fae7806c660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:51:12.954 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:12.947+0000 7fae94ff9640 1 -- 172.21.15.43:0/2675128905 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(99..99 src has 1..99) v4 ==== 5574+0+0 (secure 0 0 0) 0x7fae90088360 con 0x7fae9815af10 2023-08-09T13:51:12.955 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:12.948+0000 7fae977fe640 1 --2- 172.21.15.43:0/2675128905 >> [v2:172.21.15.203:6832/1778614695,v1:172.21.15.203:6833/1778614695] conn(0x7fae7806a1a0 0x7fae7806c660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:51:12.955 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:12.949+0000 7fae977fe640 1 --2- 172.21.15.43:0/2675128905 >> [v2:172.21.15.203:6832/1778614695,v1:172.21.15.203:6833/1778614695] conn(0x7fae7806a1a0 0x7fae7806c660 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7fae88036000 tx=0x7fae88008030 comp rx=0 tx=0).ready entity=mgr.5819 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-09T13:51:12.956 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:12.952+0000 7fae94ff9640 1 -- 172.21.15.43:0/2675128905 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178066 (secure 0 0 0) 0x7fae980cbb00 con 0x7fae9815af10 2023-08-09T13:51:13.150 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:13.148+0000 7fae9cc91640 1 -- 172.21.15.43:0/2675128905 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7fae980cbb00 con 0x7fae9815af10 2023-08-09T13:51:13.151 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:13.149+0000 7fae94ff9640 1 -- 172.21.15.43:0/2675128905 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v99) v1 ==== 93+0+10 (secure 0 0 0) 0x7fae980cbb00 con 0x7fae9815af10 2023-08-09T13:51:13.151 INFO:teuthology.orchestra.run.smithi043.stdout:pg_num: 1 2023-08-09T13:51:13.154 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:13.151+0000 7fae9cc91640 1 -- 172.21.15.43:0/2675128905 >> [v2:172.21.15.203:6832/1778614695,v1:172.21.15.203:6833/1778614695] conn(0x7fae7806a1a0 msgr2=0x7fae7806c660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:51:13.154 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:13.151+0000 7fae9cc91640 1 --2- 172.21.15.43:0/2675128905 >> [v2:172.21.15.203:6832/1778614695,v1:172.21.15.203:6833/1778614695] conn(0x7fae7806a1a0 0x7fae7806c660 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7fae88036000 tx=0x7fae88008030 comp rx=0 tx=0).stop 2023-08-09T13:51:13.155 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:13.152+0000 7fae9cc91640 1 -- 172.21.15.43:0/2675128905 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fae9815af10 msgr2=0x7fae980aebc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:51:13.155 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:13.152+0000 7fae9cc91640 1 --2- 172.21.15.43:0/2675128905 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fae9815af10 0x7fae980aebc0 secure :-1 s=READY pgs=1164 cs=0 l=1 rev1=1 crypto rx=0x7fae9000c6b0 tx=0x7fae9000cb70 comp rx=0 tx=0).stop 2023-08-09T13:51:13.155 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:13.152+0000 7fae9cc91640 1 -- 172.21.15.43:0/2675128905 shutdown_connections 2023-08-09T13:51:13.155 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:13.152+0000 7fae9cc91640 1 --2- 172.21.15.43:0/2675128905 >> [v2:172.21.15.203:6832/1778614695,v1:172.21.15.203:6833/1778614695] conn(0x7fae7806a1a0 0x7fae7806c660 unknown :-1 s=CLOSED pgs=53 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:51:13.155 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:13.152+0000 7fae9cc91640 1 --2- 172.21.15.43:0/2675128905 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fae9815af10 0x7fae980aebc0 unknown :-1 s=CLOSED pgs=1164 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:51:13.155 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:13.152+0000 7fae9cc91640 1 --2- 172.21.15.43:0/2675128905 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fae98157ea0 0x7fae980c5b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:51:13.155 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:13.152+0000 7fae9cc91640 1 --2- 172.21.15.43:0/2675128905 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fae980c90a0 0x7fae980c7720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:51:13.156 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:13.152+0000 7fae9cc91640 1 -- 172.21.15.43:0/2675128905 >> 172.21.15.43:0/2675128905 conn(0x7fae980cddd0 msgr2=0x7fae9806c630 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-09T13:51:13.156 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:13.152+0000 7fae9cc91640 1 -- 172.21.15.43:0/2675128905 shutdown_connections 2023-08-09T13:51:13.156 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:13.152+0000 7fae9cc91640 1 -- 172.21.15.43:0/2675128905 wait complete. 2023-08-09T13:51:13.174 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --format=json-pretty osd lspools 2023-08-09T13:51:13.405 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:13.403+0000 7fdbcf1d6640 1 Processor -- start 2023-08-09T13:51:13.406 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:13.403+0000 7fdbcf1d6640 1 -- start start 2023-08-09T13:51:13.407 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:13.404+0000 7fdbcf1d6640 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7fdbc815f7b0 0x7fdbc80b4760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:51:13.407 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:13.405+0000 7fdbcf1d6640 1 -- --> v1:172.21.15.43:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fdbc80b7bf0 con 0x7fdbc80b1050 2023-08-09T13:51:13.407 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:13.405+0000 7fdbcf1d6640 1 -- --> v1:172.21.15.203:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fdbc80b7ed0 con 0x7fdbc80cd6b0 2023-08-09T13:51:13.408 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:13.405+0000 7fdbcf1d6640 1 -- --> v2:172.21.15.203:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fdbc80d0370 con 0x7fdbc815f7b0 2023-08-09T13:51:13.408 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:13.405+0000 7fdbcd9d3640 1 --1- >> v1:172.21.15.43:6789/0 conn(0x7fdbc80b1050 0x7fdbc80ccfa0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.43:6789/0 says I am v1:172.21.15.43:45760/0 (socket says 172.21.15.43:45760) 2023-08-09T13:51:13.408 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:13.405+0000 7fdbcd9d3640 1 -- 172.21.15.43:0/2395634152 learned_addr learned my addr 172.21.15.43:0/2395634152 (peer_addr_for_me v1:172.21.15.43:0/0) 2023-08-09T13:51:13.410 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:13.405+0000 7fdbce1d4640 1 --2- 172.21.15.43:0/2395634152 >> v2:172.21.15.203:3300/0 conn(0x7fdbc815f7b0 0x7fdbc80b4760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:51:13.410 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:13.405+0000 7fdbcd1d2640 1 -- 172.21.15.43:0/2395634152 <== mon.0 v1:172.21.15.43:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1347570295 0 0) 0x7fdbc80b7bf0 con 0x7fdbc80b1050 2023-08-09T13:51:13.410 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:13.406+0000 7fdbcd1d2640 1 -- 172.21.15.43:0/2395634152 --> v1:172.21.15.43:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fdbb4003520 con 0x7fdbc80b1050 2023-08-09T13:51:13.410 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:13.406+0000 7fdbcd1d2640 1 -- 172.21.15.43:0/2395634152 <== mon.0 v1:172.21.15.43:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 198129862 0 0) 0x7fdbb4003520 con 0x7fdbc80b1050 2023-08-09T13:51:13.410 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:13.406+0000 7fdbcd1d2640 1 -- 172.21.15.43:0/2395634152 >> v2:172.21.15.203:3300/0 conn(0x7fdbc815f7b0 msgr2=0x7fdbc80b4760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-09T13:51:13.410 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:13.406+0000 7fdbcd1d2640 1 --2- 172.21.15.43:0/2395634152 >> v2:172.21.15.203:3300/0 conn(0x7fdbc815f7b0 0x7fdbc80b4760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:51:13.411 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:13.406+0000 7fdbcd1d2640 1 -- 172.21.15.43:0/2395634152 >> v1:172.21.15.203:6789/0 conn(0x7fdbc80cd6b0 legacy=0x7fdbc80d0e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:51:13.411 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:13.406+0000 7fdbcd1d2640 1 -- 172.21.15.43:0/2395634152 --> v1:172.21.15.43:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdbc80d0650 con 0x7fdbc80b1050 2023-08-09T13:51:13.411 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:13.406+0000 7fdbcd1d2640 1 -- 172.21.15.43:0/2395634152 <== mon.0 v1:172.21.15.43:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1285150740 0 0) 0x7fdbb8001670 con 0x7fdbc80b1050 2023-08-09T13:51:13.411 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:13.406+0000 7fdbcd1d2640 1 -- 172.21.15.43:0/2395634152 >> v1:172.21.15.43:6789/0 conn(0x7fdbc80b1050 legacy=0x7fdbc80ccfa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:51:13.411 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:13.406+0000 7fdbcd1d2640 1 --2- 172.21.15.43:0/2395634152 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fdbb40039c0 0x7fdbb4003de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:51:13.411 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:13.406+0000 7fdbcd1d2640 1 --2- 172.21.15.43:0/2395634152 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fdbb40047c0 0x7fdbb4004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:51:13.411 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:13.406+0000 7fdbcd1d2640 1 --2- 172.21.15.43:0/2395634152 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fdbb4005180 0x7fdbb400b980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:51:13.412 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:13.406+0000 7fdbcd1d2640 1 -- 172.21.15.43:0/2395634152 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdbc80d0370 con 0x7fdbb40039c0 2023-08-09T13:51:13.412 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:13.406+0000 7fdbcd1d2640 1 -- 172.21.15.43:0/2395634152 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdbb400c080 con 0x7fdbb4005180 2023-08-09T13:51:13.412 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:13.406+0000 7fdbcd1d2640 1 -- 172.21.15.43:0/2395634152 --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdbb400c240 con 0x7fdbb40047c0 2023-08-09T13:51:13.412 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:13.406+0000 7fdbce1d4640 1 --2- 172.21.15.43:0/2395634152 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fdbb40039c0 0x7fdbb4003de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:51:13.412 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:13.407+0000 7fdbce9d5640 1 --2- 172.21.15.43:0/2395634152 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fdbb4005180 0x7fdbb400b980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:51:13.413 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:13.407+0000 7fdbcd9d3640 1 --2- 172.21.15.43:0/2395634152 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fdbb40047c0 0x7fdbb4004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:51:13.413 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:13.407+0000 7fdbce1d4640 1 -- 172.21.15.43:0/2395634152 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fdbb40047c0 msgr2=0x7fdbb4004c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-09T13:51:13.413 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:13.407+0000 7fdbce1d4640 1 --2- 172.21.15.43:0/2395634152 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fdbb40047c0 0x7fdbb4004c40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:51:13.413 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:13.407+0000 7fdbce1d4640 1 -- 172.21.15.43:0/2395634152 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fdbb4005180 msgr2=0x7fdbb400b980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-09T13:51:13.413 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:13.407+0000 7fdbce1d4640 1 --2- 172.21.15.43:0/2395634152 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fdbb4005180 0x7fdbb400b980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:51:13.413 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:13.407+0000 7fdbce1d4640 1 -- 172.21.15.43:0/2395634152 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdbb4003520 con 0x7fdbb40039c0 2023-08-09T13:51:13.413 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:13.407+0000 7fdbce9d5640 1 --2- 172.21.15.43:0/2395634152 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fdbb4005180 0x7fdbb400b980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-08-09T13:51:13.414 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:13.407+0000 7fdbce9d5640 1 -- 172.21.15.43:0/2395634152 reap_dead start 2023-08-09T13:51:13.416 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:13.408+0000 7fdbce1d4640 1 --2- 172.21.15.43:0/2395634152 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fdbb40039c0 0x7fdbb4003de0 secure :-1 s=READY pgs=1166 cs=0 l=1 rev1=1 crypto rx=0x7fdbc4009ac0 tx=0x7fdbb40057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3224b17a456db3a5 server_cookie=0 in_seq=0 out_seq=0 2023-08-09T13:51:13.417 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:13.408+0000 7fdbcd1d2640 1 -- 172.21.15.43:0/2395634152 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdbc402c6b0 con 0x7fdbb40039c0 2023-08-09T13:51:13.417 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:13.408+0000 7fdbcd1d2640 1 -- 172.21.15.43:0/2395634152 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdbc402c810 con 0x7fdbb40039c0 2023-08-09T13:51:13.417 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:13.408+0000 7fdbcd1d2640 1 -- 172.21.15.43:0/2395634152 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdbc402cba0 con 0x7fdbb40039c0 2023-08-09T13:51:13.417 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:13.409+0000 7fdbcf1d6640 1 -- 172.21.15.43:0/2395634152 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fdbb40039c0 msgr2=0x7fdbb4003de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:51:13.417 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:13.409+0000 7fdbcf1d6640 1 --2- 172.21.15.43:0/2395634152 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fdbb40039c0 0x7fdbb4003de0 secure :-1 s=READY pgs=1166 cs=0 l=1 rev1=1 crypto rx=0x7fdbc4009ac0 tx=0x7fdbb40057f0 comp rx=0 tx=0).stop 2023-08-09T13:51:13.417 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:13.409+0000 7fdbcf1d6640 1 -- 172.21.15.43:0/2395634152 shutdown_connections 2023-08-09T13:51:13.417 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:13.409+0000 7fdbcf1d6640 1 --2- 172.21.15.43:0/2395634152 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fdbb40039c0 0x7fdbb4003de0 unknown :-1 s=CLOSED pgs=1166 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:51:13.418 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:13.409+0000 7fdbcf1d6640 1 -- 172.21.15.43:0/2395634152 >> 172.21.15.43:0/2395634152 conn(0x7fdbc806a2b0 msgr2=0x7fdbc806a690 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-09T13:51:13.418 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:13.409+0000 7fdbcf1d6640 1 -- 172.21.15.43:0/2395634152 shutdown_connections 2023-08-09T13:51:13.419 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:13.409+0000 7fdbcf1d6640 1 -- 172.21.15.43:0/2395634152 wait complete. 2023-08-09T13:51:13.419 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:13.411+0000 7fdbcf1d6640 1 Processor -- start 2023-08-09T13:51:13.419 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:13.411+0000 7fdbcf1d6640 1 -- start start 2023-08-09T13:51:13.419 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:13.412+0000 7fdbcf1d6640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fdbb40039c0 0x7fdbc80ca7e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:51:13.419 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:13.412+0000 7fdbcf1d6640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fdbc8152b90 0x7fdbc8156350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:51:13.419 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:13.413+0000 7fdbcf1d6640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fdbc8156890 0x7fdbc8157f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:51:13.419 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:13.413+0000 7fdbcf1d6640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdbc8156d10 con 0x7fdbb40039c0 2023-08-09T13:51:13.420 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:13.413+0000 7fdbcf1d6640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdbc8158fd0 con 0x7fdbc8152b90 2023-08-09T13:51:13.420 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:13.413+0000 7fdbcf1d6640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdbc81541b0 con 0x7fdbc8156890 2023-08-09T13:51:13.420 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:13.413+0000 7fdbce1d4640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fdbb40039c0 0x7fdbc80ca7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:51:13.420 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:13.413+0000 7fdbce1d4640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fdbb40039c0 0x7fdbc80ca7e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:36742/0 (socket says 172.21.15.43:36742) 2023-08-09T13:51:13.420 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:13.413+0000 7fdbce1d4640 1 -- 172.21.15.43:0/2134916465 learned_addr learned my addr 172.21.15.43:0/2134916465 (peer_addr_for_me v2:172.21.15.43:0/0) 2023-08-09T13:51:13.420 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:13.413+0000 7fdbcd9d3640 1 --2- 172.21.15.43:0/2134916465 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fdbc8152b90 0x7fdbc8156350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:51:13.420 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:13.413+0000 7fdbce9d5640 1 --2- 172.21.15.43:0/2134916465 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fdbc8156890 0x7fdbc8157f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:51:13.421 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:13.413+0000 7fdbce1d4640 1 -- 172.21.15.43:0/2134916465 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fdbc8156890 msgr2=0x7fdbc8157f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:51:13.421 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:13.413+0000 7fdbce1d4640 1 --2- 172.21.15.43:0/2134916465 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fdbc8156890 0x7fdbc8157f70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:51:13.421 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:13.413+0000 7fdbce1d4640 1 -- 172.21.15.43:0/2134916465 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fdbc8152b90 msgr2=0x7fdbc8156350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:51:13.421 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:13.413+0000 7fdbce1d4640 1 --2- 172.21.15.43:0/2134916465 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fdbc8152b90 0x7fdbc8156350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:51:13.421 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:13.413+0000 7fdbce1d4640 1 -- 172.21.15.43:0/2134916465 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdbc8158cf0 con 0x7fdbb40039c0 2023-08-09T13:51:13.421 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:13.413+0000 7fdbcd9d3640 1 --2- 172.21.15.43:0/2134916465 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fdbc8152b90 0x7fdbc8156350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-08-09T13:51:13.422 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:13.414+0000 7fdbce1d4640 1 --2- 172.21.15.43:0/2134916465 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fdbb40039c0 0x7fdbc80ca7e0 secure :-1 s=READY pgs=1167 cs=0 l=1 rev1=1 crypto rx=0x7fdbc402c250 tx=0x7fdbc4004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-09T13:51:13.422 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:13.414+0000 7fdbbeffd640 1 -- 172.21.15.43:0/2134916465 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdbc4003810 con 0x7fdbb40039c0 2023-08-09T13:51:13.422 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:13.414+0000 7fdbbeffd640 1 -- 172.21.15.43:0/2134916465 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdbc4003970 con 0x7fdbb40039c0 2023-08-09T13:51:13.422 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:13.414+0000 7fdbbeffd640 1 -- 172.21.15.43:0/2134916465 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdbc4003d00 con 0x7fdbb40039c0 2023-08-09T13:51:13.422 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:13.414+0000 7fdbcf1d6640 1 -- 172.21.15.43:0/2134916465 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdbc8153b60 con 0x7fdbb40039c0 2023-08-09T13:51:13.423 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:13.414+0000 7fdbcf1d6640 1 -- 172.21.15.43:0/2134916465 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdbc8153010 con 0x7fdbb40039c0 2023-08-09T13:51:13.423 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:13.416+0000 7fdbcf1d6640 1 -- 172.21.15.43:0/2134916465 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdbc81539c0 con 0x7fdbb40039c0 2023-08-09T13:51:13.424 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:13.416+0000 7fdbbeffd640 1 -- 172.21.15.43:0/2134916465 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 37) v1 ==== 88443+0+0 (secure 0 0 0) 0x7fdbc4034440 con 0x7fdbb40039c0 2023-08-09T13:51:13.424 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:13.417+0000 7fdbbeffd640 1 --2- 172.21.15.43:0/2134916465 >> [v2:172.21.15.203:6832/1778614695,v1:172.21.15.203:6833/1778614695] conn(0x7fdbac06a170 0x7fdbac06c630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:51:13.424 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:13.417+0000 7fdbbeffd640 1 -- 172.21.15.43:0/2134916465 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(99..99 src has 1..99) v4 ==== 5574+0+0 (secure 0 0 0) 0x7fdbc40aa890 con 0x7fdbb40039c0 2023-08-09T13:51:13.425 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:13.418+0000 7fdbcd9d3640 1 --2- 172.21.15.43:0/2134916465 >> [v2:172.21.15.203:6832/1778614695,v1:172.21.15.203:6833/1778614695] conn(0x7fdbac06a170 0x7fdbac06c630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:51:13.425 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:13.419+0000 7fdbcd9d3640 1 --2- 172.21.15.43:0/2134916465 >> [v2:172.21.15.203:6832/1778614695,v1:172.21.15.203:6833/1778614695] conn(0x7fdbac06a170 0x7fdbac06c630 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7fdbc8156ef0 tx=0x7fdbb8009040 comp rx=0 tx=0).ready entity=mgr.5819 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-09T13:51:13.426 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:13.422+0000 7fdbbeffd640 1 -- 172.21.15.43:0/2134916465 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178066 (secure 0 0 0) 0x7fdbc81539c0 con 0x7fdbb40039c0 2023-08-09T13:51:13.618 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:13.616+0000 7fdbcf1d6640 1 -- 172.21.15.43:0/2134916465 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) v1 -- 0x7fdbc81539c0 con 0x7fdbb40039c0 2023-08-09T13:51:13.619 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:13.617+0000 7fdbbeffd640 1 -- 172.21.15.43:0/2134916465 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v99) v1 ==== 84+0+126 (secure 0 0 0) 0x7fdbc404c090 con 0x7fdbb40039c0 2023-08-09T13:51:13.620 INFO:teuthology.orchestra.run.smithi043.stdout: 2023-08-09T13:51:13.620 INFO:teuthology.orchestra.run.smithi043.stdout:[ 2023-08-09T13:51:13.620 INFO:teuthology.orchestra.run.smithi043.stdout: { 2023-08-09T13:51:13.620 INFO:teuthology.orchestra.run.smithi043.stdout: "poolnum": 1, 2023-08-09T13:51:13.620 INFO:teuthology.orchestra.run.smithi043.stdout: "poolname": "rbd" 2023-08-09T13:51:13.620 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2023-08-09T13:51:13.621 INFO:teuthology.orchestra.run.smithi043.stdout: { 2023-08-09T13:51:13.621 INFO:teuthology.orchestra.run.smithi043.stdout: "poolnum": 2, 2023-08-09T13:51:13.621 INFO:teuthology.orchestra.run.smithi043.stdout: "poolname": ".mgr" 2023-08-09T13:51:13.621 INFO:teuthology.orchestra.run.smithi043.stdout: } 2023-08-09T13:51:13.621 INFO:teuthology.orchestra.run.smithi043.stdout:] 2023-08-09T13:51:13.622 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:13.620+0000 7fdbcf1d6640 1 -- 172.21.15.43:0/2134916465 >> [v2:172.21.15.203:6832/1778614695,v1:172.21.15.203:6833/1778614695] conn(0x7fdbac06a170 msgr2=0x7fdbac06c630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:51:13.623 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:13.620+0000 7fdbcf1d6640 1 --2- 172.21.15.43:0/2134916465 >> [v2:172.21.15.203:6832/1778614695,v1:172.21.15.203:6833/1778614695] conn(0x7fdbac06a170 0x7fdbac06c630 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7fdbc8156ef0 tx=0x7fdbb8009040 comp rx=0 tx=0).stop 2023-08-09T13:51:13.623 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:13.620+0000 7fdbcf1d6640 1 -- 172.21.15.43:0/2134916465 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fdbb40039c0 msgr2=0x7fdbc80ca7e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:51:13.623 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:13.620+0000 7fdbcf1d6640 1 --2- 172.21.15.43:0/2134916465 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fdbb40039c0 0x7fdbc80ca7e0 secure :-1 s=READY pgs=1167 cs=0 l=1 rev1=1 crypto rx=0x7fdbc402c250 tx=0x7fdbc4004730 comp rx=0 tx=0).stop 2023-08-09T13:51:13.623 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:13.620+0000 7fdbcf1d6640 1 -- 172.21.15.43:0/2134916465 shutdown_connections 2023-08-09T13:51:13.623 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:13.620+0000 7fdbcf1d6640 1 --2- 172.21.15.43:0/2134916465 >> [v2:172.21.15.203:6832/1778614695,v1:172.21.15.203:6833/1778614695] conn(0x7fdbac06a170 0x7fdbac06c630 unknown :-1 s=CLOSED pgs=54 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:51:13.623 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:13.620+0000 7fdbcf1d6640 1 --2- 172.21.15.43:0/2134916465 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fdbc8156890 0x7fdbc8157f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:51:13.623 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:13.620+0000 7fdbcf1d6640 1 --2- 172.21.15.43:0/2134916465 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fdbc8152b90 0x7fdbc8156350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:51:13.623 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:13.620+0000 7fdbcf1d6640 1 --2- 172.21.15.43:0/2134916465 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fdbb40039c0 0x7fdbc80ca7e0 unknown :-1 s=CLOSED pgs=1167 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:51:13.624 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:13.620+0000 7fdbcf1d6640 1 -- 172.21.15.43:0/2134916465 >> 172.21.15.43:0/2134916465 conn(0x7fdbc806a2b0 msgr2=0x7fdbc80b6620 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-09T13:51:13.624 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:13.620+0000 7fdbcf1d6640 1 -- 172.21.15.43:0/2134916465 shutdown_connections 2023-08-09T13:51:13.624 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:13.620+0000 7fdbcf1d6640 1 -- 172.21.15.43:0/2134916465 wait complete. 2023-08-09T13:51:13.643 DEBUG:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2023-08-09T13:51:13.643 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_metadata --pg_num_min 64 2023-08-09T13:51:13.876 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:13.873+0000 7f2082f5d640 1 Processor -- start 2023-08-09T13:51:13.876 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:13.874+0000 7f2082f5d640 1 -- start start 2023-08-09T13:51:13.877 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:13.875+0000 7f2082f5d640 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7f207c06fb70 0x7f207c0caab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:51:13.878 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:13.875+0000 7f2082f5d640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f207c0ca370 0x7f207c15b2a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:51:13.878 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:13.875+0000 7f2082f5d640 1 -- --> v1:172.21.15.203:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f207c15ae10 con 0x7f207c0ce4c0 2023-08-09T13:51:13.878 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:13.875+0000 7f2082f5d640 1 -- --> v2:172.21.15.43:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f207c154e70 con 0x7f207c0ca370 2023-08-09T13:51:13.878 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:13.875+0000 7f2082f5d640 1 -- --> v2:172.21.15.203:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f207c06ff50 con 0x7f207c06fb70 2023-08-09T13:51:13.879 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:13.876+0000 7f207bfff640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f207c0ca370 0x7f207c15b2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:51:13.879 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:13.876+0000 7f207bfff640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f207c0ca370 0x7f207c15b2a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:36750/0 (socket says 172.21.15.43:36750) 2023-08-09T13:51:13.879 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:13.876+0000 7f207bfff640 1 -- 172.21.15.43:0/2669525179 learned_addr learned my addr 172.21.15.43:0/2669525179 (peer_addr_for_me v2:172.21.15.43:0/0) 2023-08-09T13:51:13.879 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:13.876+0000 7f2080cf9640 1 --2- 172.21.15.43:0/2669525179 >> v2:172.21.15.203:3300/0 conn(0x7f207c06fb70 0x7f207c0caab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:51:13.879 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:13.876+0000 7f207bfff640 1 -- 172.21.15.43:0/2669525179 >> v2:172.21.15.203:3300/0 conn(0x7f207c06fb70 msgr2=0x7f207c0caab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-09T13:51:13.880 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:13.876+0000 7f207bfff640 1 --2- 172.21.15.43:0/2669525179 >> v2:172.21.15.203:3300/0 conn(0x7f207c06fb70 0x7f207c0caab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:51:13.880 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:13.876+0000 7f207bfff640 1 -- 172.21.15.43:0/2669525179 >> v1:172.21.15.203:6789/0 conn(0x7f207c0ce4c0 legacy=0x7f207c0c86f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-09T13:51:13.880 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:13.876+0000 7f207bfff640 1 -- 172.21.15.43:0/2669525179 --> v2:172.21.15.43:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f207c0cb320 con 0x7f207c0ca370 2023-08-09T13:51:13.880 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:13.877+0000 7f207bfff640 1 --2- 172.21.15.43:0/2669525179 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f207c0ca370 0x7f207c15b2a0 secure :-1 s=READY pgs=1168 cs=0 l=1 rev1=1 crypto rx=0x7f207c0cbb80 tx=0x7f206802b910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3d6d93a3543a1a12 server_cookie=0 in_seq=0 out_seq=0 2023-08-09T13:51:13.881 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:13.878+0000 7f207b7fe640 1 -- 172.21.15.43:0/2669525179 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2068031870 con 0x7f207c0ca370 2023-08-09T13:51:13.881 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:13.878+0000 7f207b7fe640 1 -- 172.21.15.43:0/2669525179 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f20680319d0 con 0x7f207c0ca370 2023-08-09T13:51:13.881 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:13.878+0000 7f207b7fe640 1 -- 172.21.15.43:0/2669525179 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2068031d60 con 0x7f207c0ca370 2023-08-09T13:51:13.881 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:13.878+0000 7f2082f5d640 1 -- 172.21.15.43:0/2669525179 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f207c0ca370 msgr2=0x7f207c15b2a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:51:13.881 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:13.878+0000 7f2082f5d640 1 --2- 172.21.15.43:0/2669525179 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f207c0ca370 0x7f207c15b2a0 secure :-1 s=READY pgs=1168 cs=0 l=1 rev1=1 crypto rx=0x7f207c0cbb80 tx=0x7f206802b910 comp rx=0 tx=0).stop 2023-08-09T13:51:13.882 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:13.879+0000 7f2082f5d640 1 -- 172.21.15.43:0/2669525179 shutdown_connections 2023-08-09T13:51:13.882 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:13.879+0000 7f2082f5d640 1 --2- 172.21.15.43:0/2669525179 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f207c0ca370 0x7f207c15b2a0 unknown :-1 s=CLOSED pgs=1168 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:51:13.882 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:13.879+0000 7f2082f5d640 1 --2- 172.21.15.43:0/2669525179 >> v2:172.21.15.203:3300/0 conn(0x7f207c06fb70 0x7f207c0caab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:51:13.882 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:13.879+0000 7f2082f5d640 1 -- 172.21.15.43:0/2669525179 >> 172.21.15.43:0/2669525179 conn(0x7f207c06b990 msgr2=0x7f207c06bda0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-09T13:51:13.882 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:13.879+0000 7f2082f5d640 1 -- 172.21.15.43:0/2669525179 shutdown_connections 2023-08-09T13:51:13.883 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:13.879+0000 7f2082f5d640 1 -- 172.21.15.43:0/2669525179 wait complete. 2023-08-09T13:51:13.883 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:13.880+0000 7f2082f5d640 1 Processor -- start 2023-08-09T13:51:13.883 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:13.881+0000 7f2082f5d640 1 -- start start 2023-08-09T13:51:13.884 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:13.882+0000 7f2082f5d640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f207c06fb70 0x7f207c153700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:51:13.884 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:13.882+0000 7f2082f5d640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f207c0ca370 0x7f207c1ee0f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:51:13.884 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:13.882+0000 7f2082f5d640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f207c0ce4c0 0x7f207c1f0d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:51:13.885 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:13.882+0000 7f2082f5d640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f207c1edcd0 con 0x7f207c06fb70 2023-08-09T13:51:13.885 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:13.882+0000 7f2082f5d640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f207c1ed6d0 con 0x7f207c0ca370 2023-08-09T13:51:13.885 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:13.882+0000 7f2082f5d640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f207c1f1c70 con 0x7f207c0ce4c0 2023-08-09T13:51:13.885 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:13.882+0000 7f2080cf9640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f207c06fb70 0x7f207c153700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:51:13.885 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:13.882+0000 7f2080cf9640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f207c06fb70 0x7f207c153700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:36756/0 (socket says 172.21.15.43:36756) 2023-08-09T13:51:13.885 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:13.882+0000 7f2080cf9640 1 -- 172.21.15.43:0/411565393 learned_addr learned my addr 172.21.15.43:0/411565393 (peer_addr_for_me v2:172.21.15.43:0/0) 2023-08-09T13:51:13.886 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:13.882+0000 7f2080cf9640 1 -- 172.21.15.43:0/411565393 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f207c0ce4c0 msgr2=0x7f207c1f0d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:51:13.887 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:13.882+0000 7f2080cf9640 1 --2- 172.21.15.43:0/411565393 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f207c0ce4c0 0x7f207c1f0d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:51:13.887 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:13.882+0000 7f2080cf9640 1 -- 172.21.15.43:0/411565393 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f207c0ca370 msgr2=0x7f207c1ee0f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:51:13.887 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:13.882+0000 7f2080cf9640 1 --2- 172.21.15.43:0/411565393 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f207c0ca370 0x7f207c1ee0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:51:13.887 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:13.882+0000 7f2080cf9640 1 -- 172.21.15.43:0/411565393 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f20680094d0 con 0x7f207c06fb70 2023-08-09T13:51:13.887 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:13.883+0000 7f2080cf9640 1 --2- 172.21.15.43:0/411565393 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f207c06fb70 0x7f207c153700 secure :-1 s=READY pgs=1169 cs=0 l=1 rev1=1 crypto rx=0x7f207000d620 tx=0x7f207000dae0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-09T13:51:13.887 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:13.883+0000 7f20797fa640 1 -- 172.21.15.43:0/411565393 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f207000b620 con 0x7f207c06fb70 2023-08-09T13:51:13.887 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:13.883+0000 7f20797fa640 1 -- 172.21.15.43:0/411565393 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f207000b780 con 0x7f207c06fb70 2023-08-09T13:51:13.888 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:13.883+0000 7f20797fa640 1 -- 172.21.15.43:0/411565393 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f207000bb30 con 0x7f207c06fb70 2023-08-09T13:51:13.888 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:13.883+0000 7f2082f5d640 1 -- 172.21.15.43:0/411565393 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f207c1f16b0 con 0x7f207c06fb70 2023-08-09T13:51:13.888 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:13.883+0000 7f2082f5d640 1 -- 172.21.15.43:0/411565393 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f207c1ed8f0 con 0x7f207c06fb70 2023-08-09T13:51:13.888 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:13.884+0000 7f2082f5d640 1 -- 172.21.15.43:0/411565393 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f207c1ede10 con 0x7f207c06fb70 2023-08-09T13:51:13.889 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:13.885+0000 7f20797fa640 1 -- 172.21.15.43:0/411565393 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 37) v1 ==== 88443+0+0 (secure 0 0 0) 0x7f207000bc90 con 0x7f207c06fb70 2023-08-09T13:51:13.889 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:13.886+0000 7f20797fa640 1 --2- 172.21.15.43:0/411565393 >> [v2:172.21.15.203:6832/1778614695,v1:172.21.15.203:6833/1778614695] conn(0x7f206006a170 0x7f206006c630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:51:13.889 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:13.886+0000 7f20797fa640 1 -- 172.21.15.43:0/411565393 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(99..99 src has 1..99) v4 ==== 5574+0+0 (secure 0 0 0) 0x7f20700889f0 con 0x7f207c06fb70 2023-08-09T13:51:13.889 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:13.887+0000 7f207bfff640 1 --2- 172.21.15.43:0/411565393 >> [v2:172.21.15.203:6832/1778614695,v1:172.21.15.203:6833/1778614695] conn(0x7f206006a170 0x7f206006c630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:51:13.890 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:13.888+0000 7f207bfff640 1 --2- 172.21.15.43:0/411565393 >> [v2:172.21.15.203:6832/1778614695,v1:172.21.15.203:6833/1778614695] conn(0x7f206006a170 0x7f206006c630 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f2068036000 tx=0x7f2068030000 comp rx=0 tx=0).ready entity=mgr.5819 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-09T13:51:13.892 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:13.890+0000 7f20797fa640 1 -- 172.21.15.43:0/411565393 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178066 (secure 0 0 0) 0x7f20700541a0 con 0x7f207c06fb70 2023-08-09T13:51:14.086 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:14.083+0000 7f2082f5d640 1 -- 172.21.15.43:0/411565393 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64} v 0) v1 -- 0x7f207c0739f0 con 0x7f207c06fb70 2023-08-09T13:51:14.196 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:14.193+0000 7f20797fa640 1 -- 172.21.15.43:0/411565393 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]=0 pool 'cephfs_metadata' created v100) v1 ==== 138+0+0 (secure 0 0 0) 0x7f20700577c0 con 0x7f207c06fb70 2023-08-09T13:51:14.196 INFO:teuthology.orchestra.run.smithi043.stderr:pool 'cephfs_metadata' created 2023-08-09T13:51:14.201 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:14.198+0000 7f2082f5d640 1 -- 172.21.15.43:0/411565393 >> [v2:172.21.15.203:6832/1778614695,v1:172.21.15.203:6833/1778614695] conn(0x7f206006a170 msgr2=0x7f206006c630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:51:14.202 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:14.198+0000 7f2082f5d640 1 --2- 172.21.15.43:0/411565393 >> [v2:172.21.15.203:6832/1778614695,v1:172.21.15.203:6833/1778614695] conn(0x7f206006a170 0x7f206006c630 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f2068036000 tx=0x7f2068030000 comp rx=0 tx=0).stop 2023-08-09T13:51:14.202 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:14.198+0000 7f2082f5d640 1 -- 172.21.15.43:0/411565393 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f207c06fb70 msgr2=0x7f207c153700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:51:14.202 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:14.198+0000 7f2082f5d640 1 --2- 172.21.15.43:0/411565393 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f207c06fb70 0x7f207c153700 secure :-1 s=READY pgs=1169 cs=0 l=1 rev1=1 crypto rx=0x7f207000d620 tx=0x7f207000dae0 comp rx=0 tx=0).stop 2023-08-09T13:51:14.202 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:14.199+0000 7f2082f5d640 1 -- 172.21.15.43:0/411565393 shutdown_connections 2023-08-09T13:51:14.202 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:14.199+0000 7f2082f5d640 1 --2- 172.21.15.43:0/411565393 >> [v2:172.21.15.203:6832/1778614695,v1:172.21.15.203:6833/1778614695] conn(0x7f206006a170 0x7f206006c630 unknown :-1 s=CLOSED pgs=55 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:51:14.202 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:14.199+0000 7f2082f5d640 1 --2- 172.21.15.43:0/411565393 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f207c0ce4c0 0x7f207c1f0d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:51:14.203 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:14.199+0000 7f2082f5d640 1 --2- 172.21.15.43:0/411565393 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f207c0ca370 0x7f207c1ee0f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:51:14.203 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:14.199+0000 7f2082f5d640 1 --2- 172.21.15.43:0/411565393 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f207c06fb70 0x7f207c153700 unknown :-1 s=CLOSED pgs=1169 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:51:14.203 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:14.200+0000 7f2082f5d640 1 -- 172.21.15.43:0/411565393 >> 172.21.15.43:0/411565393 conn(0x7f207c06b990 msgr2=0x7f207c0cd680 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-09T13:51:14.204 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:14.202+0000 7f2082f5d640 1 -- 172.21.15.43:0/411565393 shutdown_connections 2023-08-09T13:51:14.205 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:14.202+0000 7f2082f5d640 1 -- 172.21.15.43:0/411565393 wait complete. 2023-08-09T13:51:14.230 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data 64 --pg_num_min 64 --target_size_ratio 0.9 2023-08-09T13:51:14.445 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:14.442+0000 7f8a0e4fa640 1 Processor -- start 2023-08-09T13:51:14.446 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:14.443+0000 7f8a0e4fa640 1 -- start start 2023-08-09T13:51:14.447 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:14.444+0000 7f8a0e4fa640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f8a0815c550 0x7f8a080cd6d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:51:14.448 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:14.445+0000 7f8a0e4fa640 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7f8a080b45f0 0x7f8a080b59d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:51:14.448 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:14.445+0000 7f8a0e4fa640 1 -- --> v1:172.21.15.43:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8a080b7bd0 con 0x7f8a080d04b0 2023-08-09T13:51:14.449 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:14.445+0000 7f8a0e4fa640 1 -- --> v2:172.21.15.203:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8a080d1c00 con 0x7f8a080b45f0 2023-08-09T13:51:14.449 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:14.445+0000 7f8a0e4fa640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8a0815c930 con 0x7f8a0815c550 2023-08-09T13:51:14.449 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:14.445+0000 7f8a077fe640 1 --1- >> v1:172.21.15.43:6789/0 conn(0x7f8a080d04b0 0x7f8a080b3ee0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.43:6789/0 says I am v1:172.21.15.43:45772/0 (socket says 172.21.15.43:45772) 2023-08-09T13:51:14.449 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:14.445+0000 7f8a077fe640 1 -- 172.21.15.43:0/2914936746 learned_addr learned my addr 172.21.15.43:0/2914936746 (peer_addr_for_me v1:172.21.15.43:0/0) 2023-08-09T13:51:14.454 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:14.445+0000 7f8a07fff640 1 --2- 172.21.15.43:0/2914936746 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f8a0815c550 0x7f8a080cd6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:51:14.455 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:14.446+0000 7f8a0ca97640 1 --2- 172.21.15.43:0/2914936746 >> v2:172.21.15.203:3300/0 conn(0x7f8a080b45f0 0x7f8a080b59d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:51:14.455 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:14.446+0000 7f8a06ffd640 1 -- 172.21.15.43:0/2914936746 <== mon.0 v1:172.21.15.43:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 747332515 0 0) 0x7f8a080b7bd0 con 0x7f8a080d04b0 2023-08-09T13:51:14.455 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:14.446+0000 7f8a06ffd640 1 -- 172.21.15.43:0/2914936746 --> v1:172.21.15.43:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f89f0003520 con 0x7f8a080d04b0 2023-08-09T13:51:14.455 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:14.446+0000 7f8a06ffd640 1 -- 172.21.15.43:0/2914936746 <== mon.0 v1:172.21.15.43:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 456485440 0 0) 0x7f89f0003520 con 0x7f8a080d04b0 2023-08-09T13:51:14.455 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:14.446+0000 7f8a06ffd640 1 -- 172.21.15.43:0/2914936746 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f8a0815c550 msgr2=0x7f8a080cd6d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-09T13:51:14.455 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:14.446+0000 7f8a06ffd640 1 --2- 172.21.15.43:0/2914936746 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f8a0815c550 0x7f8a080cd6d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:51:14.455 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:14.446+0000 7f8a06ffd640 1 -- 172.21.15.43:0/2914936746 >> v2:172.21.15.203:3300/0 conn(0x7f8a080b45f0 msgr2=0x7f8a080b59d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-09T13:51:14.456 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:14.446+0000 7f8a06ffd640 1 --2- 172.21.15.43:0/2914936746 >> v2:172.21.15.203:3300/0 conn(0x7f8a080b45f0 0x7f8a080b59d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:51:14.456 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:14.446+0000 7f8a06ffd640 1 -- 172.21.15.43:0/2914936746 --> v1:172.21.15.43:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8a080b54f0 con 0x7f8a080d04b0 2023-08-09T13:51:14.456 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:14.446+0000 7f8a06ffd640 1 -- 172.21.15.43:0/2914936746 <== mon.0 v1:172.21.15.43:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1285150740 0 0) 0x7f89fc001670 con 0x7f8a080d04b0 2023-08-09T13:51:14.456 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:14.446+0000 7f8a06ffd640 1 -- 172.21.15.43:0/2914936746 >> v1:172.21.15.43:6789/0 conn(0x7f8a080d04b0 legacy=0x7f8a080b3ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:51:14.456 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:14.446+0000 7f8a06ffd640 1 --2- 172.21.15.43:0/2914936746 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f8a0815c550 0x7f89f0003970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:51:14.456 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:14.446+0000 7f8a06ffd640 1 --2- 172.21.15.43:0/2914936746 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f89f0003ef0 0x7f89f00047c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:51:14.456 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:14.446+0000 7f8a06ffd640 1 --2- 172.21.15.43:0/2914936746 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f89f0004d00 0x7f89f0005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:51:14.457 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:14.446+0000 7f8a06ffd640 1 -- 172.21.15.43:0/2914936746 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8a080d1c00 con 0x7f89f0003ef0 2023-08-09T13:51:14.457 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:14.446+0000 7f8a06ffd640 1 -- 172.21.15.43:0/2914936746 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8a0815c930 con 0x7f89f0004d00 2023-08-09T13:51:14.457 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:14.446+0000 7f8a06ffd640 1 -- 172.21.15.43:0/2914936746 --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f89f000b980 con 0x7f8a0815c550 2023-08-09T13:51:14.457 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:14.447+0000 7f8a077fe640 1 --2- 172.21.15.43:0/2914936746 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f89f0003ef0 0x7f89f00047c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:51:14.459 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:14.447+0000 7f8a07fff640 1 --2- 172.21.15.43:0/2914936746 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f8a0815c550 0x7f89f0003970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:51:14.459 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:14.447+0000 7f8a0ca97640 1 --2- 172.21.15.43:0/2914936746 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f89f0004d00 0x7f89f0005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:51:14.459 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:14.447+0000 7f8a077fe640 1 -- 172.21.15.43:0/2914936746 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f8a0815c550 msgr2=0x7f89f0003970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-09T13:51:14.459 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:14.447+0000 7f8a077fe640 1 --2- 172.21.15.43:0/2914936746 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f8a0815c550 0x7f89f0003970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:51:14.460 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:14.447+0000 7f8a077fe640 1 -- 172.21.15.43:0/2914936746 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f89f0004d00 msgr2=0x7f89f0005180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-09T13:51:14.460 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:14.447+0000 7f8a077fe640 1 --2- 172.21.15.43:0/2914936746 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f89f0004d00 0x7f89f0005180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:51:14.460 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:14.447+0000 7f8a077fe640 1 -- 172.21.15.43:0/2914936746 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f89f0003520 con 0x7f89f0003ef0 2023-08-09T13:51:14.460 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:14.448+0000 7f8a077fe640 1 --2- 172.21.15.43:0/2914936746 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f89f0003ef0 0x7f89f00047c0 secure :-1 s=READY pgs=1171 cs=0 l=1 rev1=1 crypto rx=0x7f89fc00c240 tx=0x7f89f00057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c8e261006916aa31 server_cookie=0 in_seq=0 out_seq=0 2023-08-09T13:51:14.460 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:14.449+0000 7f8a06ffd640 1 -- 172.21.15.43:0/2914936746 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f89fc005ad0 con 0x7f89f0003ef0 2023-08-09T13:51:14.460 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:14.449+0000 7f8a06ffd640 1 -- 172.21.15.43:0/2914936746 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f89fc005c50 con 0x7f89f0003ef0 2023-08-09T13:51:14.460 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:14.449+0000 7f8a06ffd640 1 -- 172.21.15.43:0/2914936746 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f89fc02c8c0 con 0x7f89f0003ef0 2023-08-09T13:51:14.461 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:14.449+0000 7f8a0e4fa640 1 -- 172.21.15.43:0/2914936746 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f89f0003ef0 msgr2=0x7f89f00047c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:51:14.461 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:14.449+0000 7f8a0e4fa640 1 --2- 172.21.15.43:0/2914936746 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f89f0003ef0 0x7f89f00047c0 secure :-1 s=READY pgs=1171 cs=0 l=1 rev1=1 crypto rx=0x7f89fc00c240 tx=0x7f89f00057f0 comp rx=0 tx=0).stop 2023-08-09T13:51:14.461 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:14.449+0000 7f8a0ca97640 1 -- 172.21.15.43:0/2914936746 reap_dead start 2023-08-09T13:51:14.461 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:14.449+0000 7f8a0e4fa640 1 -- 172.21.15.43:0/2914936746 shutdown_connections 2023-08-09T13:51:14.461 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:14.449+0000 7f8a0e4fa640 1 -- 172.21.15.43:0/2914936746 >> 172.21.15.43:0/2914936746 conn(0x7f8a08071520 msgr2=0x7f8a080ceee0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-09T13:51:14.461 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:14.449+0000 7f8a0e4fa640 1 -- 172.21.15.43:0/2914936746 shutdown_connections 2023-08-09T13:51:14.461 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:14.449+0000 7f8a0e4fa640 1 -- 172.21.15.43:0/2914936746 wait complete. 2023-08-09T13:51:14.462 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:14.451+0000 7f8a0e4fa640 1 Processor -- start 2023-08-09T13:51:14.462 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:14.451+0000 7f8a0e4fa640 1 -- start start 2023-08-09T13:51:14.464 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:14.452+0000 7f8a0e4fa640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f89f0003ef0 0x7f8a081f45f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:51:14.464 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:14.452+0000 7f8a0e4fa640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f8a081f2980 0x7f8a081f4db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:51:14.464 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:14.452+0000 7f8a0e4fa640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f8a081f52f0 0x7f8a081f1ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:51:14.464 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:14.452+0000 7f8a0e4fa640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8a081f5770 con 0x7f8a081f52f0 2023-08-09T13:51:14.465 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:14.452+0000 7f8a0e4fa640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8a081490a0 con 0x7f8a081f2980 2023-08-09T13:51:14.466 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:14.452+0000 7f8a0e4fa640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8a081f4b30 con 0x7f89f0003ef0 2023-08-09T13:51:14.466 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:14.453+0000 7f8a0ca97640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f8a081f52f0 0x7f8a081f1ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:51:14.466 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:14.453+0000 7f8a0ca97640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f8a081f52f0 0x7f8a081f1ae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:36778/0 (socket says 172.21.15.43:36778) 2023-08-09T13:51:14.466 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:14.453+0000 7f8a0ca97640 1 -- 172.21.15.43:0/2350698217 learned_addr learned my addr 172.21.15.43:0/2350698217 (peer_addr_for_me v2:172.21.15.43:0/0) 2023-08-09T13:51:14.466 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:14.453+0000 7f8a07fff640 1 --2- 172.21.15.43:0/2350698217 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f89f0003ef0 0x7f8a081f45f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:51:14.466 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:14.453+0000 7f8a077fe640 1 --2- 172.21.15.43:0/2350698217 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f8a081f2980 0x7f8a081f4db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:51:14.467 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:14.453+0000 7f8a0ca97640 1 -- 172.21.15.43:0/2350698217 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f89f0003ef0 msgr2=0x7f8a081f45f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:51:14.467 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:14.453+0000 7f8a0ca97640 1 --2- 172.21.15.43:0/2350698217 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f89f0003ef0 0x7f8a081f45f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:51:14.467 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:14.453+0000 7f8a0ca97640 1 -- 172.21.15.43:0/2350698217 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f8a081f2980 msgr2=0x7f8a081f4db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:51:14.467 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:14.453+0000 7f8a0ca97640 1 --2- 172.21.15.43:0/2350698217 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f8a081f2980 0x7f8a081f4db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:51:14.467 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:14.453+0000 7f8a0ca97640 1 -- 172.21.15.43:0/2350698217 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8a081492f0 con 0x7f8a081f52f0 2023-08-09T13:51:14.467 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:14.454+0000 7f8a0ca97640 1 --2- 172.21.15.43:0/2350698217 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f8a081f52f0 0x7f8a081f1ae0 secure :-1 s=READY pgs=1172 cs=0 l=1 rev1=1 crypto rx=0x7f8a080ce4e0 tx=0x7f8a0000f4a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-09T13:51:14.467 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:14.454+0000 7f8a04ff9640 1 -- 172.21.15.43:0/2350698217 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8a00017400 con 0x7f8a081f52f0 2023-08-09T13:51:14.467 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:14.454+0000 7f8a04ff9640 1 -- 172.21.15.43:0/2350698217 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8a00011040 con 0x7f8a081f52f0 2023-08-09T13:51:14.468 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:14.454+0000 7f8a04ff9640 1 -- 172.21.15.43:0/2350698217 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8a000177c0 con 0x7f8a081f52f0 2023-08-09T13:51:14.468 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:14.454+0000 7f8a0e4fa640 1 -- 172.21.15.43:0/2350698217 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8a081495d0 con 0x7f8a081f52f0 2023-08-09T13:51:14.468 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:14.454+0000 7f8a0e4fa640 1 -- 172.21.15.43:0/2350698217 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8a08149b90 con 0x7f8a081f52f0 2023-08-09T13:51:14.468 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:14.455+0000 7f8a0e4fa640 1 -- 172.21.15.43:0/2350698217 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8a080739f0 con 0x7f8a081f52f0 2023-08-09T13:51:14.468 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:14.457+0000 7f8a04ff9640 1 -- 172.21.15.43:0/2350698217 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 37) v1 ==== 88443+0+0 (secure 0 0 0) 0x7f8a00017920 con 0x7f8a081f52f0 2023-08-09T13:51:14.469 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:14.458+0000 7f8a04ff9640 1 --2- 172.21.15.43:0/2350698217 >> [v2:172.21.15.203:6832/1778614695,v1:172.21.15.203:6833/1778614695] conn(0x7f89f406a170 0x7f89f406c630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:51:14.469 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:14.458+0000 7f8a04ff9640 1 -- 172.21.15.43:0/2350698217 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(100..100 src has 1..100) v4 ==== 5944+0+0 (secure 0 0 0) 0x7f8a000895b0 con 0x7f8a081f52f0 2023-08-09T13:51:14.469 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:14.459+0000 7f8a07fff640 1 --2- 172.21.15.43:0/2350698217 >> [v2:172.21.15.203:6832/1778614695,v1:172.21.15.203:6833/1778614695] conn(0x7f89f406a170 0x7f89f406c630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:51:14.469 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:14.460+0000 7f8a07fff640 1 --2- 172.21.15.43:0/2350698217 >> [v2:172.21.15.203:6832/1778614695,v1:172.21.15.203:6833/1778614695] conn(0x7f89f406a170 0x7f89f406c630 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f89f8009930 tx=0x7f89f8008040 comp rx=0 tx=0).ready entity=mgr.5819 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-09T13:51:14.470 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:14.462+0000 7f8a04ff9640 1 -- 172.21.15.43:0/2350698217 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178066 (secure 0 0 0) 0x7f8a00054ad0 con 0x7f8a081f52f0 2023-08-09T13:51:14.659 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:14.657+0000 7f8a0e4fa640 1 -- 172.21.15.43:0/2350698217 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9} v 0) v1 -- 0x7f8a080b6030 con 0x7f8a081f52f0 2023-08-09T13:51:15.214 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:15.211+0000 7f8a04ff9640 1 -- 172.21.15.43:0/2350698217 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]=0 pool 'cephfs_data' created v101) v1 ==== 170+0+0 (secure 0 0 0) 0x7f8a000580f0 con 0x7f8a081f52f0 2023-08-09T13:51:15.215 INFO:teuthology.orchestra.run.smithi043.stderr:pool 'cephfs_data' created 2023-08-09T13:51:15.220 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:15.218+0000 7f8a0e4fa640 1 -- 172.21.15.43:0/2350698217 >> [v2:172.21.15.203:6832/1778614695,v1:172.21.15.203:6833/1778614695] conn(0x7f89f406a170 msgr2=0x7f89f406c630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:51:15.220 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:15.218+0000 7f8a0e4fa640 1 --2- 172.21.15.43:0/2350698217 >> [v2:172.21.15.203:6832/1778614695,v1:172.21.15.203:6833/1778614695] conn(0x7f89f406a170 0x7f89f406c630 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f89f8009930 tx=0x7f89f8008040 comp rx=0 tx=0).stop 2023-08-09T13:51:15.221 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:15.218+0000 7f8a0e4fa640 1 -- 172.21.15.43:0/2350698217 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f8a081f52f0 msgr2=0x7f8a081f1ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:51:15.221 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:15.218+0000 7f8a0e4fa640 1 --2- 172.21.15.43:0/2350698217 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f8a081f52f0 0x7f8a081f1ae0 secure :-1 s=READY pgs=1172 cs=0 l=1 rev1=1 crypto rx=0x7f8a080ce4e0 tx=0x7f8a0000f4a0 comp rx=0 tx=0).stop 2023-08-09T13:51:15.221 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:15.219+0000 7f8a0e4fa640 1 -- 172.21.15.43:0/2350698217 shutdown_connections 2023-08-09T13:51:15.221 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:15.219+0000 7f8a0e4fa640 1 --2- 172.21.15.43:0/2350698217 >> [v2:172.21.15.203:6832/1778614695,v1:172.21.15.203:6833/1778614695] conn(0x7f89f406a170 0x7f89f406c630 unknown :-1 s=CLOSED pgs=56 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:51:15.221 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:15.219+0000 7f8a0e4fa640 1 --2- 172.21.15.43:0/2350698217 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f8a081f52f0 0x7f8a081f1ae0 unknown :-1 s=CLOSED pgs=1172 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:51:15.222 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:15.219+0000 7f8a0e4fa640 1 --2- 172.21.15.43:0/2350698217 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f8a081f2980 0x7f8a081f4db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:51:15.222 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:15.219+0000 7f8a0e4fa640 1 --2- 172.21.15.43:0/2350698217 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f89f0003ef0 0x7f8a081f45f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:51:15.222 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:15.219+0000 7f8a0e4fa640 1 -- 172.21.15.43:0/2350698217 >> 172.21.15.43:0/2350698217 conn(0x7f8a08071520 msgr2=0x7f8a080d2440 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-09T13:51:15.222 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:15.220+0000 7f8a0e4fa640 1 -- 172.21.15.43:0/2350698217 shutdown_connections 2023-08-09T13:51:15.223 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:15.221+0000 7f8a0e4fa640 1 -- 172.21.15.43:0/2350698217 wait complete. 2023-08-09T13:51:15.262 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs new cephfs cephfs_metadata cephfs_data 2023-08-09T13:51:15.465 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:15.463+0000 7f6c63942640 1 Processor -- start 2023-08-09T13:51:15.466 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:15.463+0000 7f6c63942640 1 -- start start 2023-08-09T13:51:15.467 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:15.464+0000 7f6c63942640 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7f6c5c0cf790 0x7f6c5c0c3d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:51:15.467 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:15.464+0000 7f6c63942640 1 -- --> v1:172.21.15.43:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6c5c15e470 con 0x7f6c5c0ce850 2023-08-09T13:51:15.467 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:15.464+0000 7f6c63942640 1 -- --> v1:172.21.15.203:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6c5c15e750 con 0x7f6c5c15a550 2023-08-09T13:51:15.468 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:15.464+0000 7f6c63942640 1 -- --> v2:172.21.15.203:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6c5c15ea30 con 0x7f6c5c0cf790 2023-08-09T13:51:15.468 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:15.465+0000 7f6c61edf640 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7f6c5c0cf790 0x7f6c5c0c3d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:51:15.469 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:15.465+0000 7f6c60edd640 1 --1- >> v1:172.21.15.43:6789/0 conn(0x7f6c5c0ce850 0x7f6c5c06de00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.43:6789/0 says I am v1:172.21.15.43:59364/0 (socket says 172.21.15.43:59364) 2023-08-09T13:51:15.469 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:15.465+0000 7f6c60edd640 1 -- 172.21.15.43:0/2793217731 learned_addr learned my addr 172.21.15.43:0/2793217731 (peer_addr_for_me v1:172.21.15.43:0/0) 2023-08-09T13:51:15.469 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:15.466+0000 7f6c5bfff640 1 -- 172.21.15.43:0/2793217731 <== mon.0 v1:172.21.15.43:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3890919253 0 0) 0x7f6c5c15e470 con 0x7f6c5c0ce850 2023-08-09T13:51:15.469 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:15.466+0000 7f6c5bfff640 1 -- 172.21.15.43:0/2793217731 --> v1:172.21.15.43:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f6c44003520 con 0x7f6c5c0ce850 2023-08-09T13:51:15.469 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:15.466+0000 7f6c5bfff640 1 -- 172.21.15.43:0/2793217731 <== mon.1 v1:172.21.15.203:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3032802623 0 0) 0x7f6c5c15e750 con 0x7f6c5c15a550 2023-08-09T13:51:15.469 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:15.466+0000 7f6c5bfff640 1 -- 172.21.15.43:0/2793217731 --> v1:172.21.15.203:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f6c5c15e470 con 0x7f6c5c15a550 2023-08-09T13:51:15.470 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:15.466+0000 7f6c61edf640 1 -- 172.21.15.43:0/2793217731 >> v1:172.21.15.203:6789/0 conn(0x7f6c5c15a550 legacy=0x7f6c5c0ce140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:51:15.470 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:15.466+0000 7f6c61edf640 1 -- 172.21.15.43:0/2793217731 >> v1:172.21.15.43:6789/0 conn(0x7f6c5c0ce850 legacy=0x7f6c5c06de00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:51:15.470 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:15.466+0000 7f6c5bfff640 1 -- 172.21.15.43:0/2793217731 <== mon.0 v1:172.21.15.43:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3991720283 0 0) 0x7f6c44003520 con 0x7f6c5c0ce850 2023-08-09T13:51:15.470 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:15.466+0000 7f6c61edf640 1 -- 172.21.15.43:0/2793217731 --> v2:172.21.15.203:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6c5c15ed10 con 0x7f6c5c0cf790 2023-08-09T13:51:15.471 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:15.467+0000 7f6c61edf640 1 --2- 172.21.15.43:0/2793217731 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f6c5c0cf790 0x7f6c5c0c3d90 secure :-1 s=READY pgs=428 cs=0 l=1 rev1=1 crypto rx=0x7f6c540097a0 tx=0x7f6c5402b710 comp rx=0 tx=0).ready entity=mon.1 client_cookie=bd0106c7a8c71ce0 server_cookie=0 in_seq=0 out_seq=0 2023-08-09T13:51:15.471 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:15.468+0000 7f6c5bfff640 1 -- 172.21.15.43:0/2793217731 <== mon.1 v2:172.21.15.203:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6c50001670 con 0x7f6c5c0cf790 2023-08-09T13:51:15.471 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:15.468+0000 7f6c5bfff640 1 -- 172.21.15.43:0/2793217731 <== mon.1 v2:172.21.15.203:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6c5402e050 con 0x7f6c5c0cf790 2023-08-09T13:51:15.472 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:15.468+0000 7f6c5bfff640 1 -- 172.21.15.43:0/2793217731 <== mon.1 v2:172.21.15.203:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6c54030730 con 0x7f6c5c0cf790 2023-08-09T13:51:15.473 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:15.469+0000 7f6c63942640 1 -- 172.21.15.43:0/2793217731 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f6c5c0cf790 msgr2=0x7f6c5c0c3d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:51:15.473 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:15.469+0000 7f6c63942640 1 --2- 172.21.15.43:0/2793217731 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f6c5c0cf790 0x7f6c5c0c3d90 secure :-1 s=READY pgs=428 cs=0 l=1 rev1=1 crypto rx=0x7f6c540097a0 tx=0x7f6c5402b710 comp rx=0 tx=0).stop 2023-08-09T13:51:15.473 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:15.469+0000 7f6c63942640 1 -- 172.21.15.43:0/2793217731 shutdown_connections 2023-08-09T13:51:15.473 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:15.469+0000 7f6c63942640 1 --2- 172.21.15.43:0/2793217731 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f6c5c0cf790 0x7f6c5c0c3d90 unknown :-1 s=CLOSED pgs=428 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:51:15.473 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:15.469+0000 7f6c63942640 1 -- 172.21.15.43:0/2793217731 >> 172.21.15.43:0/2793217731 conn(0x7f6c5c0b1dd0 msgr2=0x7f6c5c0b21e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-09T13:51:15.474 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:15.469+0000 7f6c63942640 1 -- 172.21.15.43:0/2793217731 shutdown_connections 2023-08-09T13:51:15.474 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:15.469+0000 7f6c63942640 1 -- 172.21.15.43:0/2793217731 wait complete. 2023-08-09T13:51:15.474 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:15.471+0000 7f6c63942640 1 Processor -- start 2023-08-09T13:51:15.475 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:15.471+0000 7f6c63942640 1 -- start start 2023-08-09T13:51:15.476 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:15.472+0000 7f6c63942640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f6c5c0ce850 0x7f6c5c0b0c30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:51:15.476 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:15.472+0000 7f6c63942640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f6c5c0cf790 0x7f6c5c0c9960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:51:15.476 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:15.472+0000 7f6c63942640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f6c5c15a550 0x7f6c5c1f51b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:51:15.476 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:15.473+0000 7f6c63942640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6c5c1f1b80 con 0x7f6c5c15a550 2023-08-09T13:51:15.476 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:15.473+0000 7f6c63942640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6c5c0b1170 con 0x7f6c5c0ce850 2023-08-09T13:51:15.477 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:15.473+0000 7f6c63942640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6c5c0aba70 con 0x7f6c5c0cf790 2023-08-09T13:51:15.477 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:15.473+0000 7f6c61edf640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f6c5c15a550 0x7f6c5c1f51b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:51:15.477 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:15.473+0000 7f6c61edf640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f6c5c15a550 0x7f6c5c1f51b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:42820/0 (socket says 172.21.15.43:42820) 2023-08-09T13:51:15.477 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:15.473+0000 7f6c61edf640 1 -- 172.21.15.43:0/2703799643 learned_addr learned my addr 172.21.15.43:0/2703799643 (peer_addr_for_me v2:172.21.15.43:0/0) 2023-08-09T13:51:15.477 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:15.473+0000 7f6c60edd640 1 --2- 172.21.15.43:0/2703799643 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f6c5c0cf790 0x7f6c5c0c9960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:51:15.477 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:15.473+0000 7f6c616de640 1 --2- 172.21.15.43:0/2703799643 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f6c5c0ce850 0x7f6c5c0b0c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:51:15.477 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:15.473+0000 7f6c61edf640 1 -- 172.21.15.43:0/2703799643 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f6c5c0cf790 msgr2=0x7f6c5c0c9960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:51:15.478 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:15.473+0000 7f6c61edf640 1 --2- 172.21.15.43:0/2703799643 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f6c5c0cf790 0x7f6c5c0c9960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:51:15.478 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:15.473+0000 7f6c61edf640 1 -- 172.21.15.43:0/2703799643 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f6c5c0ce850 msgr2=0x7f6c5c0b0c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:51:15.478 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:15.473+0000 7f6c61edf640 1 --2- 172.21.15.43:0/2703799643 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f6c5c0ce850 0x7f6c5c0b0c30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:51:15.478 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:15.473+0000 7f6c61edf640 1 -- 172.21.15.43:0/2703799643 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6c54009400 con 0x7f6c5c15a550 2023-08-09T13:51:15.478 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:15.473+0000 7f6c60edd640 1 --2- 172.21.15.43:0/2703799643 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f6c5c0cf790 0x7f6c5c0c9960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-08-09T13:51:15.479 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:15.474+0000 7f6c61edf640 1 --2- 172.21.15.43:0/2703799643 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f6c5c15a550 0x7f6c5c1f51b0 secure :-1 s=READY pgs=1174 cs=0 l=1 rev1=1 crypto rx=0x7f6c5402b750 tx=0x7f6c540046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-09T13:51:15.479 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:15.474+0000 7f6c59ffb640 1 -- 172.21.15.43:0/2703799643 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6c54004230 con 0x7f6c5c15a550 2023-08-09T13:51:15.479 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:15.474+0000 7f6c59ffb640 1 -- 172.21.15.43:0/2703799643 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6c54004390 con 0x7f6c5c15a550 2023-08-09T13:51:15.479 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:15.474+0000 7f6c59ffb640 1 -- 172.21.15.43:0/2703799643 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6c54003830 con 0x7f6c5c15a550 2023-08-09T13:51:15.480 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:15.474+0000 7f6c63942640 1 -- 172.21.15.43:0/2703799643 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6c5c0abd50 con 0x7f6c5c15a550 2023-08-09T13:51:15.480 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:15.474+0000 7f6c63942640 1 -- 172.21.15.43:0/2703799643 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6c5c0ac310 con 0x7f6c5c15a550 2023-08-09T13:51:15.480 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:15.476+0000 7f6c63942640 1 -- 172.21.15.43:0/2703799643 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6c5c1f1d50 con 0x7f6c5c15a550 2023-08-09T13:51:15.481 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:15.477+0000 7f6c59ffb640 1 -- 172.21.15.43:0/2703799643 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 37) v1 ==== 88443+0+0 (secure 0 0 0) 0x7f6c54039040 con 0x7f6c5c15a550 2023-08-09T13:51:15.481 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:15.478+0000 7f6c59ffb640 1 --2- 172.21.15.43:0/2703799643 >> [v2:172.21.15.203:6832/1778614695,v1:172.21.15.203:6833/1778614695] conn(0x7f6c4806a1a0 0x7f6c4806c660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:51:15.481 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:15.478+0000 7f6c59ffb640 1 -- 172.21.15.43:0/2703799643 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(101..101 src has 1..101) v4 ==== 6326+0+0 (secure 0 0 0) 0x7f6c540aa180 con 0x7f6c5c15a550 2023-08-09T13:51:15.481 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:15.479+0000 7f6c616de640 1 --2- 172.21.15.43:0/2703799643 >> [v2:172.21.15.203:6832/1778614695,v1:172.21.15.203:6833/1778614695] conn(0x7f6c4806a1a0 0x7f6c4806c660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:51:15.482 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:15.480+0000 7f6c616de640 1 --2- 172.21.15.43:0/2703799643 >> [v2:172.21.15.203:6832/1778614695,v1:172.21.15.203:6833/1778614695] conn(0x7f6c4806a1a0 0x7f6c4806c660 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f6c4c001460 tx=0x7f6c4c009040 comp rx=0 tx=0).ready entity=mgr.5819 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-09T13:51:15.484 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:15.482+0000 7f6c59ffb640 1 -- 172.21.15.43:0/2703799643 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178066 (secure 0 0 0) 0x7f6c54075640 con 0x7f6c5c15a550 2023-08-09T13:51:15.716 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:15.714+0000 7f6c63942640 1 -- 172.21.15.43:0/2703799643 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"} v 0) v1 -- 0x7f6c5c1f1d50 con 0x7f6c5c15a550 2023-08-09T13:51:16.219 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:16.216+0000 7f6c59ffb640 1 -- 172.21.15.43:0/2703799643 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]=0 new fs with metadata pool 12 and data pool 13 v40) v1 ==== 174+0+0 (secure 0 0 0) 0x7f6c5c1f1d50 con 0x7f6c5c15a550 2023-08-09T13:51:16.220 INFO:teuthology.orchestra.run.smithi043.stderr:new fs with metadata pool 12 and data pool 13 2023-08-09T13:51:16.226 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:16.223+0000 7f6c63942640 1 -- 172.21.15.43:0/2703799643 >> [v2:172.21.15.203:6832/1778614695,v1:172.21.15.203:6833/1778614695] conn(0x7f6c4806a1a0 msgr2=0x7f6c4806c660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:51:16.226 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:16.223+0000 7f6c63942640 1 --2- 172.21.15.43:0/2703799643 >> [v2:172.21.15.203:6832/1778614695,v1:172.21.15.203:6833/1778614695] conn(0x7f6c4806a1a0 0x7f6c4806c660 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f6c4c001460 tx=0x7f6c4c009040 comp rx=0 tx=0).stop 2023-08-09T13:51:16.226 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:16.223+0000 7f6c63942640 1 -- 172.21.15.43:0/2703799643 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f6c5c15a550 msgr2=0x7f6c5c1f51b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:51:16.226 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:16.223+0000 7f6c63942640 1 --2- 172.21.15.43:0/2703799643 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f6c5c15a550 0x7f6c5c1f51b0 secure :-1 s=READY pgs=1174 cs=0 l=1 rev1=1 crypto rx=0x7f6c5402b750 tx=0x7f6c540046c0 comp rx=0 tx=0).stop 2023-08-09T13:51:16.226 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:16.223+0000 7f6c63942640 1 -- 172.21.15.43:0/2703799643 shutdown_connections 2023-08-09T13:51:16.227 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:16.223+0000 7f6c63942640 1 --2- 172.21.15.43:0/2703799643 >> [v2:172.21.15.203:6832/1778614695,v1:172.21.15.203:6833/1778614695] conn(0x7f6c4806a1a0 0x7f6c4806c660 secure :-1 s=CLOSED pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f6c4c001460 tx=0x7f6c4c009040 comp rx=0 tx=0).stop 2023-08-09T13:51:16.227 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:16.223+0000 7f6c63942640 1 --2- 172.21.15.43:0/2703799643 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f6c5c15a550 0x7f6c5c1f51b0 unknown :-1 s=CLOSED pgs=1174 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:51:16.227 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:16.223+0000 7f6c63942640 1 --2- 172.21.15.43:0/2703799643 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f6c5c0cf790 0x7f6c5c0c9960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:51:16.227 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:16.223+0000 7f6c63942640 1 --2- 172.21.15.43:0/2703799643 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f6c5c0ce850 0x7f6c5c0b0c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:51:16.227 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:16.223+0000 7f6c63942640 1 -- 172.21.15.43:0/2703799643 >> 172.21.15.43:0/2703799643 conn(0x7f6c5c0b1dd0 msgr2=0x7f6c5c0aee50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-09T13:51:16.229 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:16.226+0000 7f6c63942640 1 -- 172.21.15.43:0/2703799643 shutdown_connections 2023-08-09T13:51:16.229 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:16.227+0000 7f6c63942640 1 -- 172.21.15.43:0/2703799643 wait complete. 2023-08-09T13:51:16.275 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-08-09T13:51:16.465 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:16.463+0000 7f256aec0640 1 Processor -- start 2023-08-09T13:51:16.465 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:16.463+0000 7f256aec0640 1 -- start start 2023-08-09T13:51:16.467 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:16.464+0000 7f256aec0640 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7f25640c8750 0x7f25640ad340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:51:16.467 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:16.464+0000 7f256aec0640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f25640acdf0 0x7f256415b2c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:51:16.467 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:16.464+0000 7f256aec0640 1 -- --> v1:172.21.15.203:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f25640cbd90 con 0x7f25640c8b30 2023-08-09T13:51:16.467 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:16.464+0000 7f256aec0640 1 -- --> v2:172.21.15.43:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f256415e810 con 0x7f25640acdf0 2023-08-09T13:51:16.467 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:16.464+0000 7f256aec0640 1 -- --> v2:172.21.15.203:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f25640c9f70 con 0x7f25640c8750 2023-08-09T13:51:16.468 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:16.465+0000 7f256a6bf640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f25640acdf0 0x7f256415b2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:51:16.468 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:16.465+0000 7f256a6bf640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f25640acdf0 0x7f256415b2c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:42830/0 (socket says 172.21.15.43:42830) 2023-08-09T13:51:16.468 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:16.465+0000 7f256a6bf640 1 -- 172.21.15.43:0/356282047 learned_addr learned my addr 172.21.15.43:0/356282047 (peer_addr_for_me v2:172.21.15.43:0/0) 2023-08-09T13:51:16.469 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:16.465+0000 7f2569ebe640 1 --2- 172.21.15.43:0/356282047 >> v2:172.21.15.203:3300/0 conn(0x7f25640c8750 0x7f25640ad340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:51:16.469 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:16.465+0000 7f256a6bf640 1 -- 172.21.15.43:0/356282047 >> v2:172.21.15.203:3300/0 conn(0x7f25640c8750 msgr2=0x7f25640ad340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-09T13:51:16.469 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:16.465+0000 7f256a6bf640 1 --2- 172.21.15.43:0/356282047 >> v2:172.21.15.203:3300/0 conn(0x7f25640c8750 0x7f25640ad340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:51:16.469 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:16.465+0000 7f256a6bf640 1 -- 172.21.15.43:0/356282047 >> v1:172.21.15.203:6789/0 conn(0x7f25640c8b30 legacy=0x7f256406f410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-09T13:51:16.469 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:16.465+0000 7f256a6bf640 1 -- 172.21.15.43:0/356282047 --> v2:172.21.15.43:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f25640ca2b0 con 0x7f25640acdf0 2023-08-09T13:51:16.470 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:16.466+0000 7f256a6bf640 1 --2- 172.21.15.43:0/356282047 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f25640acdf0 0x7f256415b2c0 secure :-1 s=READY pgs=1175 cs=0 l=1 rev1=1 crypto rx=0x7f25640cb9a0 tx=0x7f255c02be50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8561ce691ae7b9b6 server_cookie=0 in_seq=0 out_seq=0 2023-08-09T13:51:16.471 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:16.467+0000 7f2568ebc640 1 -- 172.21.15.43:0/356282047 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f255c02c9f0 con 0x7f25640acdf0 2023-08-09T13:51:16.471 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:16.467+0000 7f2568ebc640 1 -- 172.21.15.43:0/356282047 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f255c02cb50 con 0x7f25640acdf0 2023-08-09T13:51:16.471 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:16.467+0000 7f2568ebc640 1 -- 172.21.15.43:0/356282047 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f255c0343f0 con 0x7f25640acdf0 2023-08-09T13:51:16.471 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:16.467+0000 7f256aec0640 1 -- 172.21.15.43:0/356282047 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f25640acdf0 msgr2=0x7f256415b2c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:51:16.471 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:16.467+0000 7f256aec0640 1 --2- 172.21.15.43:0/356282047 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f25640acdf0 0x7f256415b2c0 secure :-1 s=READY pgs=1175 cs=0 l=1 rev1=1 crypto rx=0x7f25640cb9a0 tx=0x7f255c02be50 comp rx=0 tx=0).stop 2023-08-09T13:51:16.471 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:16.467+0000 7f256aec0640 1 -- 172.21.15.43:0/356282047 shutdown_connections 2023-08-09T13:51:16.472 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:16.467+0000 7f256aec0640 1 --2- 172.21.15.43:0/356282047 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f25640acdf0 0x7f256415b2c0 unknown :-1 s=CLOSED pgs=1175 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:51:16.472 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:16.467+0000 7f256aec0640 1 --2- 172.21.15.43:0/356282047 >> v2:172.21.15.203:3300/0 conn(0x7f25640c8750 0x7f25640ad340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:51:16.472 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:16.467+0000 7f256aec0640 1 -- 172.21.15.43:0/356282047 >> 172.21.15.43:0/356282047 conn(0x7f25640cde90 msgr2=0x7f25640ce2a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-09T13:51:16.472 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:16.467+0000 7f256aec0640 1 -- 172.21.15.43:0/356282047 shutdown_connections 2023-08-09T13:51:16.472 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:16.467+0000 7f256aec0640 1 -- 172.21.15.43:0/356282047 wait complete. 2023-08-09T13:51:16.474 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:16.469+0000 7f256aec0640 1 Processor -- start 2023-08-09T13:51:16.474 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:16.469+0000 7f256aec0640 1 -- start start 2023-08-09T13:51:16.474 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:16.470+0000 7f256aec0640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f25640acdf0 0x7f25641594e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:51:16.474 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:16.470+0000 7f256aec0640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f25640c8750 0x7f256406bc80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:51:16.475 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:16.470+0000 7f256aec0640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f25640c8b30 0x7f25640cbf20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:51:16.475 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:16.470+0000 7f256aec0640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f25641f5fa0 con 0x7f25640c8b30 2023-08-09T13:51:16.475 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:16.470+0000 7f256aec0640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f25641f53d0 con 0x7f25640acdf0 2023-08-09T13:51:16.475 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:16.470+0000 7f256aec0640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f25641f5670 con 0x7f25640c8750 2023-08-09T13:51:16.475 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:16.471+0000 7f256a6bf640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f25640c8b30 0x7f25640cbf20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:51:16.476 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:16.471+0000 7f256a6bf640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f25640c8b30 0x7f25640cbf20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:42834/0 (socket says 172.21.15.43:42834) 2023-08-09T13:51:16.476 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:16.471+0000 7f256a6bf640 1 -- 172.21.15.43:0/868722269 learned_addr learned my addr 172.21.15.43:0/868722269 (peer_addr_for_me v2:172.21.15.43:0/0) 2023-08-09T13:51:16.476 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:16.471+0000 7f256a6bf640 1 -- 172.21.15.43:0/868722269 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f25640c8750 msgr2=0x7f256406bc80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:51:16.476 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:16.471+0000 7f256a6bf640 1 --2- 172.21.15.43:0/868722269 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f25640c8750 0x7f256406bc80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:51:16.476 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:16.471+0000 7f256a6bf640 1 -- 172.21.15.43:0/868722269 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f25640acdf0 msgr2=0x7f25641594e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:51:16.477 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:16.471+0000 7f2569ebe640 1 --2- 172.21.15.43:0/868722269 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f25640acdf0 0x7f25641594e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:51:16.477 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:16.471+0000 7f256a6bf640 1 --2- 172.21.15.43:0/868722269 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f25640acdf0 0x7f25641594e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:51:16.477 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:16.471+0000 7f256a6bf640 1 -- 172.21.15.43:0/868722269 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f255c009b30 con 0x7f25640c8b30 2023-08-09T13:51:16.477 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:16.471+0000 7f256a6bf640 1 --2- 172.21.15.43:0/868722269 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f25640c8b30 0x7f25640cbf20 secure :-1 s=READY pgs=1176 cs=0 l=1 rev1=1 crypto rx=0x7f255c036040 tx=0x7f255c004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-09T13:51:16.478 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:16.472+0000 7f255a7fc640 1 -- 172.21.15.43:0/868722269 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f255c03e450 con 0x7f25640c8b30 2023-08-09T13:51:16.479 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:16.472+0000 7f255a7fc640 1 -- 172.21.15.43:0/868722269 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f255c03e5d0 con 0x7f25640c8b30 2023-08-09T13:51:16.479 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:16.472+0000 7f256aec0640 1 -- 172.21.15.43:0/868722269 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f25641f5950 con 0x7f25640c8b30 2023-08-09T13:51:16.479 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:16.472+0000 7f255a7fc640 1 -- 172.21.15.43:0/868722269 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f255c03e980 con 0x7f25640c8b30 2023-08-09T13:51:16.479 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:16.472+0000 7f256aec0640 1 -- 172.21.15.43:0/868722269 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f256406ceb0 con 0x7f25640c8b30 2023-08-09T13:51:16.479 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:16.473+0000 7f256aec0640 1 -- 172.21.15.43:0/868722269 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f25641f6150 con 0x7f25640c8b30 2023-08-09T13:51:16.479 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:16.474+0000 7f255a7fc640 1 -- 172.21.15.43:0/868722269 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 37) v1 ==== 88443+0+0 (secure 0 0 0) 0x7f255c03eae0 con 0x7f25640c8b30 2023-08-09T13:51:16.480 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:16.475+0000 7f255a7fc640 1 --2- 172.21.15.43:0/868722269 >> [v2:172.21.15.203:6832/1778614695,v1:172.21.15.203:6833/1778614695] conn(0x7f254806a1a0 0x7f254806c660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:51:16.480 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:16.475+0000 7f255a7fc640 1 -- 172.21.15.43:0/868722269 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(102..102 src has 1..102) v4 ==== 6426+0+0 (secure 0 0 0) 0x7f255c0ab540 con 0x7f25640c8b30 2023-08-09T13:51:16.480 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:16.475+0000 7f2569ebe640 1 --2- 172.21.15.43:0/868722269 >> [v2:172.21.15.203:6832/1778614695,v1:172.21.15.203:6833/1778614695] conn(0x7f254806a1a0 0x7f254806c660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:51:16.481 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:16.477+0000 7f2569ebe640 1 --2- 172.21.15.43:0/868722269 >> [v2:172.21.15.203:6832/1778614695,v1:172.21.15.203:6833/1778614695] conn(0x7f254806a1a0 0x7f254806c660 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f2560006fd0 tx=0x7f2560008040 comp rx=0 tx=0).ready entity=mgr.5819 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-09T13:51:16.481 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:16.479+0000 7f255a7fc640 1 -- 172.21.15.43:0/868722269 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178066 (secure 0 0 0) 0x7f25641f6150 con 0x7f25640c8b30 2023-08-09T13:51:16.676 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:16.673+0000 7f256aec0640 1 -- 172.21.15.43:0/868722269 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f25640c9230 con 0x7f25640c8b30 2023-08-09T13:51:16.677 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:16.675+0000 7f255a7fc640 1 -- 172.21.15.43:0/868722269 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v102) v1 ==== 74+0+18571 (secure 0 0 0) 0x7f255c076910 con 0x7f25640c8b30 2023-08-09T13:51:16.678 INFO:teuthology.orchestra.run.smithi043.stdout: 2023-08-09T13:51:16.678 INFO:teuthology.orchestra.run.smithi043.stdout:{"epoch":102,"fsid":"daac87ee-1337-4250-a955-aea4e408a39b","created":"2023-08-09T13:33:41.701198+0000","modified":"2023-08-09T13:51:15.716408+0000","last_up_change":"2023-08-09T13:33:53.783453+0000","last_in_change":"2023-08-09T13:33:45.148904+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":13,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":"rbd","create_time":"2023-08-09T13:33:54.365137+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":".mgr","create_time":"2023-08-09T13:33:54.831803+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":12,"pool_name":"cephfs_metadata","create_time":"2023-08-09T13:51:14.142563+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"102","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":13,"pool_name":"cephfs_data","create_time":"2023-08-09T13:51:14.712668+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"102","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"cbd49ecd-e896-4fbd-864e-5c37a829b165","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":101,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6800","nonce":502667335},{"type":"v1","addr":"172.21.15.43:6801","nonce":502667335}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6802","nonce":502667335},{"type":"v1","addr":"172.21.15.43:6803","nonce":502667335}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6806","nonce":502667335},{"type":"v1","addr":"172.21.15.43:6807","nonce":502667335}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6804","nonce":502667335},{"type":"v1","addr":"172.21.15.43:6805","nonce":502667335}]},"public_addr":"172.21.15.43:6801/502667335","cluster_addr":"172.21.15.43:6803/502667335","heartbeat_back_addr":"172.21.15.43:6807/502667335","heartbeat_front_addr":"172.21.15.43:6805/502667335","state":["exists","up"]},{"osd":1,"uuid":"75c7e0fe-1ec7-4fe8-a2f2-3661a65b8500","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":101,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6808","nonce":3289134610},{"type":"v1","addr":"172.21.15.43:6809","nonce":3289134610}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6810","nonce":3289134610},{"type":"v1","addr":"172.21.15.43:6812","nonce":3289134610}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6821","nonce":3289134610},{"type":"v1","addr":"172.21.15.43:6824","nonce":3289134610}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6815","nonce":3289134610},{"type":"v1","addr":"172.21.15.43:6818","nonce":3289134610}]},"public_addr":"172.21.15.43:6809/3289134610","cluster_addr":"172.21.15.43:6812/3289134610","heartbeat_back_addr":"172.21.15.43:6824/3289134610","heartbeat_front_addr":"172.21.15.43:6818/3289134610","state":["exists","up"]},{"osd":2,"uuid":"bb5c41e9-7f5f-4125-9b01-3bb8bfc2ea83","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":101,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6811","nonce":4079709736},{"type":"v1","addr":"172.21.15.43:6814","nonce":4079709736}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6817","nonce":4079709736},{"type":"v1","addr":"172.21.15.43:6820","nonce":4079709736}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6829","nonce":4079709736},{"type":"v1","addr":"172.21.15.43:6831","nonce":4079709736}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6823","nonce":4079709736},{"type":"v1","addr":"172.21.15.43:6826","nonce":4079709736}]},"public_addr":"172.21.15.43:6814/4079709736","cluster_addr":"172.21.15.43:6820/4079709736","heartbeat_back_addr":"172.21.15.43:6831/4079709736","heartbeat_front_addr":"172.21.15.43:6826/4079709736","state":["exists","up"]},{"osd":3,"uuid":"bfe4c8b4-694a-465c-87e6-2b2aab852613","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":101,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6813","nonce":1709538759},{"type":"v1","addr":"172.21.15.43:6816","nonce":1709538759}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6819","nonce":1709538759},{"type":"v1","addr":"172.21.15.43:6822","nonce":1709538759}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6828","nonce":1709538759},{"type":"v1","addr":"172.21.15.43:6830","nonce":1709538759}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6825","nonce":1709538759},{"type":"v1","addr":"172.21.15.43:6827","nonce":1709538759}]},"public_addr":"172.21.15.43:6816/1709538759","cluster_addr":"172.21.15.43:6822/1709538759","heartbeat_back_addr":"172.21.15.43:6830/1709538759","heartbeat_front_addr":"172.21.15.43:6827/1709538759","state":["exists","up"]},{"osd":4,"uuid":"fa2e8a77-aba1-4756-853e-0d174a5bdeba","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":101,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6800","nonce":4196866481},{"type":"v1","addr":"172.21.15.203:6801","nonce":4196866481}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6802","nonce":4196866481},{"type":"v1","addr":"172.21.15.203:6803","nonce":4196866481}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6806","nonce":4196866481},{"type":"v1","addr":"172.21.15.203:6807","nonce":4196866481}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6804","nonce":4196866481},{"type":"v1","addr":"172.21.15.203:6805","nonce":4196866481}]},"public_addr":"172.21.15.203:6801/4196866481","cluster_addr":"172.21.15.203:6803/4196866481","heartbeat_back_addr":"172.21.15.203:6807/4196866481","heartbeat_front_addr":"172.21.15.203:6805/4196866481","state":["exists","up"]},{"osd":5,"uuid":"81811e4b-4d24-4f51-a82c-f1d700a2a00d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":101,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6816","nonce":3059290081},{"type":"v1","addr":"172.21.15.203:6817","nonce":3059290081}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6818","nonce":3059290081},{"type":"v1","addr":"172.21.15.203:6819","nonce":3059290081}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6822","nonce":3059290081},{"type":"v1","addr":"172.21.15.203:6823","nonce":3059290081}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6820","nonce":3059290081},{"type":"v1","addr":"172.21.15.203:6821","nonce":3059290081}]},"public_addr":"172.21.15.203:6817/3059290081","cluster_addr":"172.21.15.203:6819/3059290081","heartbeat_back_addr":"172.21.15.203:6823/3059290081","heartbeat_front_addr":"172.21.15.203:6821/3059290081","state":["exists","up"]},{"osd":6,"uuid":"307bac4d-6a5b-443d-af75-56d695d7674f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":101,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6824","nonce":512155569},{"type":"v1","addr":"172.21.15.203:6825","nonce":512155569}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6826","nonce":512155569},{"type":"v1","addr":"172.21.15.203:6827","nonce":512155569}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6830","nonce":512155569},{"type":"v1","addr":"172.21.15.203:6831","nonce":512155569}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6828","nonce":512155569},{"type":"v1","addr":"172.21.15.203:6829","nonce":512155569}]},"public_addr":"172.21.15.203:6825/512155569","cluster_addr":"172.21.15.203:6827/512155569","heartbeat_back_addr":"172.21.15.203:6831/512155569","heartbeat_front_addr":"172.21.15.203:6829/512155569","state":["exists","up"]},{"osd":7,"uuid":"a95dc7ca-9f1b-4295-af70-68a22a2d7d5e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":101,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6808","nonce":3331450438},{"type":"v1","addr":"172.21.15.203:6809","nonce":3331450438}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6810","nonce":3331450438},{"type":"v1","addr":"172.21.15.203:6811","nonce":3331450438}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6814","nonce":3331450438},{"type":"v1","addr":"172.21.15.203:6815","nonce":3331450438}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6812","nonce":3331450438},{"type":"v1","addr":"172.21.15.203:6813","nonce":3331450438}]},"public_addr":"172.21.15.203:6809/3331450438","cluster_addr":"172.21.15.203:6811/3331450438","heartbeat_back_addr":"172.21.15.203:6815/3331450438","heartbeat_front_addr":"172.21.15.203:6813/3331450438","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-09T13:33:50.547652+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-09T13:33:50.589469+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-09T13:33:50.561724+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-09T13:33:50.599650+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-09T13:33:50.411855+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-09T13:33:50.219681+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-09T13:33:50.504664+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-09T13:33:50.441248+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":7,"snaps":[{"begin":104,"length":1},{"begin":221,"length":1}]}],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-08-09T13:51:16.680 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:16.677+0000 7f256aec0640 1 -- 172.21.15.43:0/868722269 >> [v2:172.21.15.203:6832/1778614695,v1:172.21.15.203:6833/1778614695] conn(0x7f254806a1a0 msgr2=0x7f254806c660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:51:16.680 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:16.677+0000 7f256aec0640 1 --2- 172.21.15.43:0/868722269 >> [v2:172.21.15.203:6832/1778614695,v1:172.21.15.203:6833/1778614695] conn(0x7f254806a1a0 0x7f254806c660 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f2560006fd0 tx=0x7f2560008040 comp rx=0 tx=0).stop 2023-08-09T13:51:16.681 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:16.677+0000 7f256aec0640 1 -- 172.21.15.43:0/868722269 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f25640c8b30 msgr2=0x7f25640cbf20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:51:16.681 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:16.677+0000 7f256aec0640 1 --2- 172.21.15.43:0/868722269 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f25640c8b30 0x7f25640cbf20 secure :-1 s=READY pgs=1176 cs=0 l=1 rev1=1 crypto rx=0x7f255c036040 tx=0x7f255c004a40 comp rx=0 tx=0).stop 2023-08-09T13:51:16.681 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:16.678+0000 7f256aec0640 1 -- 172.21.15.43:0/868722269 shutdown_connections 2023-08-09T13:51:16.681 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:16.678+0000 7f256aec0640 1 --2- 172.21.15.43:0/868722269 >> [v2:172.21.15.203:6832/1778614695,v1:172.21.15.203:6833/1778614695] conn(0x7f254806a1a0 0x7f254806c660 unknown :-1 s=CLOSED pgs=58 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:51:16.681 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:16.678+0000 7f256aec0640 1 --2- 172.21.15.43:0/868722269 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f25640c8b30 0x7f25640cbf20 unknown :-1 s=CLOSED pgs=1176 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:51:16.681 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:16.678+0000 7f256aec0640 1 --2- 172.21.15.43:0/868722269 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f25640c8750 0x7f256406bc80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:51:16.681 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:16.678+0000 7f256aec0640 1 --2- 172.21.15.43:0/868722269 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f25640acdf0 0x7f25641594e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:51:16.682 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:16.678+0000 7f256aec0640 1 -- 172.21.15.43:0/868722269 >> 172.21.15.43:0/868722269 conn(0x7f25640cde90 msgr2=0x7f25640adf60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-09T13:51:16.682 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:16.678+0000 7f256aec0640 1 -- 172.21.15.43:0/868722269 shutdown_connections 2023-08-09T13:51:16.682 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:16.678+0000 7f256aec0640 1 -- 172.21.15.43:0/868722269 wait complete. 2023-08-09T13:51:16.697 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-08-09T13:51:16.920 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:16.918+0000 7f1c30dd6640 1 Processor -- start 2023-08-09T13:51:16.921 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:16.918+0000 7f1c30dd6640 1 -- start start 2023-08-09T13:51:16.922 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:16.919+0000 7f1c30dd6640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f1c2c0b2790 0x7f1c2c0d2ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:51:16.922 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:16.920+0000 7f1c30dd6640 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7f1c2c0b3050 0x7f1c2c0b3f60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:51:16.922 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:16.920+0000 7f1c30dd6640 1 -- --> v1:172.21.15.43:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1c2c0ce090 con 0x7f1c2c0b2b70 2023-08-09T13:51:16.922 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:16.920+0000 7f1c30dd6640 1 -- --> v2:172.21.15.43:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1c2c0b3950 con 0x7f1c2c0b2790 2023-08-09T13:51:16.923 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:16.920+0000 7f1c30dd6640 1 -- --> v2:172.21.15.203:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1c2c0cd7f0 con 0x7f1c2c0b3050 2023-08-09T13:51:16.923 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:16.920+0000 7f1c2a59c640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f1c2c0b2790 0x7f1c2c0d2ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:51:16.924 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:16.920+0000 7f1c2a59c640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f1c2c0b2790 0x7f1c2c0d2ed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:42850/0 (socket says 172.21.15.43:42850) 2023-08-09T13:51:16.924 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:16.920+0000 7f1c2a59c640 1 -- 172.21.15.43:0/1225261513 learned_addr learned my addr 172.21.15.43:0/1225261513 (peer_addr_for_me v2:172.21.15.43:0/0) 2023-08-09T13:51:16.924 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:16.920+0000 7f1c21d9b640 1 --1- 172.21.15.43:0/1225261513 >> v1:172.21.15.43:6789/0 conn(0x7f1c2c0b2b70 0x7f1c2c0b5330 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.43:6789/0 says I am v1:172.21.15.43:59366/0 (socket says 172.21.15.43:59366) 2023-08-09T13:51:16.924 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:16.920+0000 7f1c2ad9d640 1 --2- 172.21.15.43:0/1225261513 >> v2:172.21.15.203:3300/0 conn(0x7f1c2c0b3050 0x7f1c2c0b3f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:51:16.924 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:16.921+0000 7f1c2a59c640 1 -- 172.21.15.43:0/1225261513 >> v2:172.21.15.203:3300/0 conn(0x7f1c2c0b3050 msgr2=0x7f1c2c0b3f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-09T13:51:16.924 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:16.921+0000 7f1c2a59c640 1 --2- 172.21.15.43:0/1225261513 >> v2:172.21.15.203:3300/0 conn(0x7f1c2c0b3050 0x7f1c2c0b3f60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:51:16.924 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:16.921+0000 7f1c2a59c640 1 -- 172.21.15.43:0/1225261513 >> v1:172.21.15.43:6789/0 conn(0x7f1c2c0b2b70 legacy=0x7f1c2c0b5330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:51:16.925 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:16.921+0000 7f1c2a59c640 1 -- 172.21.15.43:0/1225261513 --> v2:172.21.15.43:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1c2c0cdad0 con 0x7f1c2c0b2790 2023-08-09T13:51:16.925 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:16.921+0000 7f1c29d9b640 1 -- 172.21.15.43:0/1225261513 <== mon.0 v1:172.21.15.43:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2069516998 0 0) 0x7f1c2c0ce090 con 0x7f1c2c0b2b70 2023-08-09T13:51:16.926 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:16.922+0000 7f1c2a59c640 1 --2- 172.21.15.43:0/1225261513 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f1c2c0b2790 0x7f1c2c0d2ed0 secure :-1 s=READY pgs=1178 cs=0 l=1 rev1=1 crypto rx=0x7f1c2c08a270 tx=0x7f1c1c02b9b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d17f24019e309450 server_cookie=0 in_seq=0 out_seq=0 2023-08-09T13:51:16.926 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:16.922+0000 7f1c29d9b640 1 -- 172.21.15.43:0/1225261513 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1c1c030870 con 0x7f1c2c0b2790 2023-08-09T13:51:16.926 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:16.922+0000 7f1c29d9b640 1 -- 172.21.15.43:0/1225261513 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1c1c0309d0 con 0x7f1c2c0b2790 2023-08-09T13:51:16.926 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:16.922+0000 7f1c29d9b640 1 -- 172.21.15.43:0/1225261513 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1c1c030d60 con 0x7f1c2c0b2790 2023-08-09T13:51:16.926 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:16.923+0000 7f1c30dd6640 1 -- 172.21.15.43:0/1225261513 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f1c2c0b2790 msgr2=0x7f1c2c0d2ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:51:16.926 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:16.923+0000 7f1c30dd6640 1 --2- 172.21.15.43:0/1225261513 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f1c2c0b2790 0x7f1c2c0d2ed0 secure :-1 s=READY pgs=1178 cs=0 l=1 rev1=1 crypto rx=0x7f1c2c08a270 tx=0x7f1c1c02b9b0 comp rx=0 tx=0).stop 2023-08-09T13:51:16.927 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:16.923+0000 7f1c30dd6640 1 -- 172.21.15.43:0/1225261513 shutdown_connections 2023-08-09T13:51:16.927 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:16.923+0000 7f1c30dd6640 1 --2- 172.21.15.43:0/1225261513 >> v2:172.21.15.203:3300/0 conn(0x7f1c2c0b3050 0x7f1c2c0b3f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:51:16.927 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:16.923+0000 7f1c30dd6640 1 --2- 172.21.15.43:0/1225261513 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f1c2c0b2790 0x7f1c2c0d2ed0 unknown :-1 s=CLOSED pgs=1178 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:51:16.927 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:16.923+0000 7f1c30dd6640 1 -- 172.21.15.43:0/1225261513 >> 172.21.15.43:0/1225261513 conn(0x7f1c2c09b7b0 msgr2=0x7f1c2c0d24e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-09T13:51:16.927 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:16.923+0000 7f1c30dd6640 1 -- 172.21.15.43:0/1225261513 shutdown_connections 2023-08-09T13:51:16.927 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:16.923+0000 7f1c30dd6640 1 -- 172.21.15.43:0/1225261513 wait complete. 2023-08-09T13:51:16.928 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:16.925+0000 7f1c30dd6640 1 Processor -- start 2023-08-09T13:51:16.928 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:16.925+0000 7f1c30dd6640 1 -- start start 2023-08-09T13:51:16.928 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:16.926+0000 7f1c30dd6640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f1c2c0b2790 0x7f1c2c1f1b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:51:16.929 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:16.926+0000 7f1c30dd6640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f1c2c0b2b70 0x7f1c2c0cb030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:51:16.929 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:16.926+0000 7f1c30dd6640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f1c2c0b3050 0x7f1c2c06b770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:51:16.929 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:16.926+0000 7f1c30dd6640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1c2c159be0 con 0x7f1c2c0b3050 2023-08-09T13:51:16.929 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:16.926+0000 7f1c30dd6640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1c2c159ec0 con 0x7f1c2c0b2b70 2023-08-09T13:51:16.929 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:16.926+0000 7f1c30dd6640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1c2c15a1a0 con 0x7f1c2c0b2790 2023-08-09T13:51:16.929 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:16.926+0000 7f1c2ad9d640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f1c2c0b3050 0x7f1c2c06b770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:51:16.929 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:16.926+0000 7f1c2ad9d640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f1c2c0b3050 0x7f1c2c06b770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:42864/0 (socket says 172.21.15.43:42864) 2023-08-09T13:51:16.930 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:16.926+0000 7f1c2ad9d640 1 -- 172.21.15.43:0/30712542 learned_addr learned my addr 172.21.15.43:0/30712542 (peer_addr_for_me v2:172.21.15.43:0/0) 2023-08-09T13:51:16.931 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:16.926+0000 7f1c21d9b640 1 --2- 172.21.15.43:0/30712542 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f1c2c0b2b70 0x7f1c2c0cb030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:51:16.931 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:16.926+0000 7f1c2a59c640 1 --2- 172.21.15.43:0/30712542 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f1c2c0b2790 0x7f1c2c1f1b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:51:16.931 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:16.926+0000 7f1c2ad9d640 1 -- 172.21.15.43:0/30712542 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f1c2c0b2790 msgr2=0x7f1c2c1f1b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:51:16.932 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:16.926+0000 7f1c2ad9d640 1 --2- 172.21.15.43:0/30712542 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f1c2c0b2790 0x7f1c2c1f1b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:51:16.932 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:16.926+0000 7f1c2ad9d640 1 -- 172.21.15.43:0/30712542 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f1c2c0b2b70 msgr2=0x7f1c2c0cb030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:51:16.932 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:16.926+0000 7f1c2ad9d640 1 --2- 172.21.15.43:0/30712542 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f1c2c0b2b70 0x7f1c2c0cb030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:51:16.932 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:16.927+0000 7f1c2ad9d640 1 -- 172.21.15.43:0/30712542 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1c1c0094d0 con 0x7f1c2c0b3050 2023-08-09T13:51:16.932 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:16.927+0000 7f1c2ad9d640 1 --2- 172.21.15.43:0/30712542 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f1c2c0b3050 0x7f1c2c06b770 secure :-1 s=READY pgs=1179 cs=0 l=1 rev1=1 crypto rx=0x7f1c2400cc30 tx=0x7f1c2400a5c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-09T13:51:16.932 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:16.927+0000 7f1c237fe640 1 -- 172.21.15.43:0/30712542 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1c2400ce40 con 0x7f1c2c0b3050 2023-08-09T13:51:16.932 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:16.927+0000 7f1c237fe640 1 -- 172.21.15.43:0/30712542 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1c24003990 con 0x7f1c2c0b3050 2023-08-09T13:51:16.933 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:16.927+0000 7f1c237fe640 1 -- 172.21.15.43:0/30712542 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1c24003d20 con 0x7f1c2c0b3050 2023-08-09T13:51:16.933 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:16.927+0000 7f1c30dd6640 1 -- 172.21.15.43:0/30712542 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1c2c15a480 con 0x7f1c2c0b3050 2023-08-09T13:51:16.933 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:16.927+0000 7f1c30dd6640 1 -- 172.21.15.43:0/30712542 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1c2c1f20b0 con 0x7f1c2c0b3050 2023-08-09T13:51:16.933 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:16.929+0000 7f1c30dd6640 1 -- 172.21.15.43:0/30712542 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1c2c15a9a0 con 0x7f1c2c0b3050 2023-08-09T13:51:16.934 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:16.930+0000 7f1c237fe640 1 -- 172.21.15.43:0/30712542 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 37) v1 ==== 88443+0+0 (secure 0 0 0) 0x7f1c24003e80 con 0x7f1c2c0b3050 2023-08-09T13:51:16.934 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:16.930+0000 7f1c237fe640 1 --2- 172.21.15.43:0/30712542 >> [v2:172.21.15.203:6832/1778614695,v1:172.21.15.203:6833/1778614695] conn(0x7f1c0c06a170 0x7f1c0c06c630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:51:16.934 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:16.931+0000 7f1c237fe640 1 -- 172.21.15.43:0/30712542 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(102..102 src has 1..102) v4 ==== 6426+0+0 (secure 0 0 0) 0x7f1c24088ce0 con 0x7f1c2c0b3050 2023-08-09T13:51:16.934 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:16.931+0000 7f1c2a59c640 1 --2- 172.21.15.43:0/30712542 >> [v2:172.21.15.203:6832/1778614695,v1:172.21.15.203:6833/1778614695] conn(0x7f1c0c06a170 0x7f1c0c06c630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:51:16.935 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:16.932+0000 7f1c2a59c640 1 --2- 172.21.15.43:0/30712542 >> [v2:172.21.15.203:6832/1778614695,v1:172.21.15.203:6833/1778614695] conn(0x7f1c0c06a170 0x7f1c0c06c630 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f1c1c036000 tx=0x7f1c1c03b040 comp rx=0 tx=0).ready entity=mgr.5819 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-09T13:51:16.937 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:16.935+0000 7f1c237fe640 1 -- 172.21.15.43:0/30712542 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178066 (secure 0 0 0) 0x7f1c2c15a9a0 con 0x7f1c2c0b3050 2023-08-09T13:51:17.128 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:17.126+0000 7f1c30dd6640 1 -- 172.21.15.43:0/30712542 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f1c2c15a9a0 con 0x7f1c2c0b3050 2023-08-09T13:51:17.130 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:17.128+0000 7f1c237fe640 1 -- 172.21.15.43:0/30712542 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v102) v1 ==== 74+0+18571 (secure 0 0 0) 0x7f1c2c15a9a0 con 0x7f1c2c0b3050 2023-08-09T13:51:17.131 INFO:teuthology.orchestra.run.smithi043.stdout: 2023-08-09T13:51:17.131 INFO:teuthology.orchestra.run.smithi043.stdout:{"epoch":102,"fsid":"daac87ee-1337-4250-a955-aea4e408a39b","created":"2023-08-09T13:33:41.701198+0000","modified":"2023-08-09T13:51:15.716408+0000","last_up_change":"2023-08-09T13:33:53.783453+0000","last_in_change":"2023-08-09T13:33:45.148904+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":13,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":"rbd","create_time":"2023-08-09T13:33:54.365137+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":".mgr","create_time":"2023-08-09T13:33:54.831803+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":12,"pool_name":"cephfs_metadata","create_time":"2023-08-09T13:51:14.142563+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"102","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":13,"pool_name":"cephfs_data","create_time":"2023-08-09T13:51:14.712668+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"102","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"cbd49ecd-e896-4fbd-864e-5c37a829b165","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":101,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6800","nonce":502667335},{"type":"v1","addr":"172.21.15.43:6801","nonce":502667335}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6802","nonce":502667335},{"type":"v1","addr":"172.21.15.43:6803","nonce":502667335}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6806","nonce":502667335},{"type":"v1","addr":"172.21.15.43:6807","nonce":502667335}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6804","nonce":502667335},{"type":"v1","addr":"172.21.15.43:6805","nonce":502667335}]},"public_addr":"172.21.15.43:6801/502667335","cluster_addr":"172.21.15.43:6803/502667335","heartbeat_back_addr":"172.21.15.43:6807/502667335","heartbeat_front_addr":"172.21.15.43:6805/502667335","state":["exists","up"]},{"osd":1,"uuid":"75c7e0fe-1ec7-4fe8-a2f2-3661a65b8500","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":101,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6808","nonce":3289134610},{"type":"v1","addr":"172.21.15.43:6809","nonce":3289134610}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6810","nonce":3289134610},{"type":"v1","addr":"172.21.15.43:6812","nonce":3289134610}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6821","nonce":3289134610},{"type":"v1","addr":"172.21.15.43:6824","nonce":3289134610}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6815","nonce":3289134610},{"type":"v1","addr":"172.21.15.43:6818","nonce":3289134610}]},"public_addr":"172.21.15.43:6809/3289134610","cluster_addr":"172.21.15.43:6812/3289134610","heartbeat_back_addr":"172.21.15.43:6824/3289134610","heartbeat_front_addr":"172.21.15.43:6818/3289134610","state":["exists","up"]},{"osd":2,"uuid":"bb5c41e9-7f5f-4125-9b01-3bb8bfc2ea83","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":101,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6811","nonce":4079709736},{"type":"v1","addr":"172.21.15.43:6814","nonce":4079709736}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6817","nonce":4079709736},{"type":"v1","addr":"172.21.15.43:6820","nonce":4079709736}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6829","nonce":4079709736},{"type":"v1","addr":"172.21.15.43:6831","nonce":4079709736}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6823","nonce":4079709736},{"type":"v1","addr":"172.21.15.43:6826","nonce":4079709736}]},"public_addr":"172.21.15.43:6814/4079709736","cluster_addr":"172.21.15.43:6820/4079709736","heartbeat_back_addr":"172.21.15.43:6831/4079709736","heartbeat_front_addr":"172.21.15.43:6826/4079709736","state":["exists","up"]},{"osd":3,"uuid":"bfe4c8b4-694a-465c-87e6-2b2aab852613","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":101,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6813","nonce":1709538759},{"type":"v1","addr":"172.21.15.43:6816","nonce":1709538759}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6819","nonce":1709538759},{"type":"v1","addr":"172.21.15.43:6822","nonce":1709538759}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6828","nonce":1709538759},{"type":"v1","addr":"172.21.15.43:6830","nonce":1709538759}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6825","nonce":1709538759},{"type":"v1","addr":"172.21.15.43:6827","nonce":1709538759}]},"public_addr":"172.21.15.43:6816/1709538759","cluster_addr":"172.21.15.43:6822/1709538759","heartbeat_back_addr":"172.21.15.43:6830/1709538759","heartbeat_front_addr":"172.21.15.43:6827/1709538759","state":["exists","up"]},{"osd":4,"uuid":"fa2e8a77-aba1-4756-853e-0d174a5bdeba","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":101,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6800","nonce":4196866481},{"type":"v1","addr":"172.21.15.203:6801","nonce":4196866481}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6802","nonce":4196866481},{"type":"v1","addr":"172.21.15.203:6803","nonce":4196866481}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6806","nonce":4196866481},{"type":"v1","addr":"172.21.15.203:6807","nonce":4196866481}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6804","nonce":4196866481},{"type":"v1","addr":"172.21.15.203:6805","nonce":4196866481}]},"public_addr":"172.21.15.203:6801/4196866481","cluster_addr":"172.21.15.203:6803/4196866481","heartbeat_back_addr":"172.21.15.203:6807/4196866481","heartbeat_front_addr":"172.21.15.203:6805/4196866481","state":["exists","up"]},{"osd":5,"uuid":"81811e4b-4d24-4f51-a82c-f1d700a2a00d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":101,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6816","nonce":3059290081},{"type":"v1","addr":"172.21.15.203:6817","nonce":3059290081}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6818","nonce":3059290081},{"type":"v1","addr":"172.21.15.203:6819","nonce":3059290081}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6822","nonce":3059290081},{"type":"v1","addr":"172.21.15.203:6823","nonce":3059290081}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6820","nonce":3059290081},{"type":"v1","addr":"172.21.15.203:6821","nonce":3059290081}]},"public_addr":"172.21.15.203:6817/3059290081","cluster_addr":"172.21.15.203:6819/3059290081","heartbeat_back_addr":"172.21.15.203:6823/3059290081","heartbeat_front_addr":"172.21.15.203:6821/3059290081","state":["exists","up"]},{"osd":6,"uuid":"307bac4d-6a5b-443d-af75-56d695d7674f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":101,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6824","nonce":512155569},{"type":"v1","addr":"172.21.15.203:6825","nonce":512155569}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6826","nonce":512155569},{"type":"v1","addr":"172.21.15.203:6827","nonce":512155569}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6830","nonce":512155569},{"type":"v1","addr":"172.21.15.203:6831","nonce":512155569}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6828","nonce":512155569},{"type":"v1","addr":"172.21.15.203:6829","nonce":512155569}]},"public_addr":"172.21.15.203:6825/512155569","cluster_addr":"172.21.15.203:6827/512155569","heartbeat_back_addr":"172.21.15.203:6831/512155569","heartbeat_front_addr":"172.21.15.203:6829/512155569","state":["exists","up"]},{"osd":7,"uuid":"a95dc7ca-9f1b-4295-af70-68a22a2d7d5e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":101,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6808","nonce":3331450438},{"type":"v1","addr":"172.21.15.203:6809","nonce":3331450438}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6810","nonce":3331450438},{"type":"v1","addr":"172.21.15.203:6811","nonce":3331450438}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6814","nonce":3331450438},{"type":"v1","addr":"172.21.15.203:6815","nonce":3331450438}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6812","nonce":3331450438},{"type":"v1","addr":"172.21.15.203:6813","nonce":3331450438}]},"public_addr":"172.21.15.203:6809/3331450438","cluster_addr":"172.21.15.203:6811/3331450438","heartbeat_back_addr":"172.21.15.203:6815/3331450438","heartbeat_front_addr":"172.21.15.203:6813/3331450438","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-09T13:33:50.547652+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-09T13:33:50.589469+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-09T13:33:50.561724+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-09T13:33:50.599650+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-09T13:33:50.411855+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-09T13:33:50.219681+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-09T13:33:50.504664+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-09T13:33:50.441248+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":7,"snaps":[{"begin":104,"length":1},{"begin":221,"length":1}]}],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-08-09T13:51:17.132 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:17.130+0000 7f1c30dd6640 1 -- 172.21.15.43:0/30712542 >> [v2:172.21.15.203:6832/1778614695,v1:172.21.15.203:6833/1778614695] conn(0x7f1c0c06a170 msgr2=0x7f1c0c06c630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:51:17.133 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:17.130+0000 7f1c30dd6640 1 --2- 172.21.15.43:0/30712542 >> [v2:172.21.15.203:6832/1778614695,v1:172.21.15.203:6833/1778614695] conn(0x7f1c0c06a170 0x7f1c0c06c630 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f1c1c036000 tx=0x7f1c1c03b040 comp rx=0 tx=0).stop 2023-08-09T13:51:17.133 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:17.130+0000 7f1c30dd6640 1 -- 172.21.15.43:0/30712542 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f1c2c0b3050 msgr2=0x7f1c2c06b770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:51:17.133 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:17.130+0000 7f1c30dd6640 1 --2- 172.21.15.43:0/30712542 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f1c2c0b3050 0x7f1c2c06b770 secure :-1 s=READY pgs=1179 cs=0 l=1 rev1=1 crypto rx=0x7f1c2400cc30 tx=0x7f1c2400a5c0 comp rx=0 tx=0).stop 2023-08-09T13:51:17.133 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:17.130+0000 7f1c30dd6640 1 -- 172.21.15.43:0/30712542 shutdown_connections 2023-08-09T13:51:17.133 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:17.130+0000 7f1c30dd6640 1 --2- 172.21.15.43:0/30712542 >> [v2:172.21.15.203:6832/1778614695,v1:172.21.15.203:6833/1778614695] conn(0x7f1c0c06a170 0x7f1c0c06c630 unknown :-1 s=CLOSED pgs=59 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:51:17.133 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:17.130+0000 7f1c30dd6640 1 --2- 172.21.15.43:0/30712542 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f1c2c0b3050 0x7f1c2c06b770 unknown :-1 s=CLOSED pgs=1179 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:51:17.134 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:17.130+0000 7f1c30dd6640 1 --2- 172.21.15.43:0/30712542 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f1c2c0b2b70 0x7f1c2c0cb030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:51:17.134 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:17.130+0000 7f1c30dd6640 1 --2- 172.21.15.43:0/30712542 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f1c2c0b2790 0x7f1c2c1f1b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:51:17.134 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:17.130+0000 7f1c30dd6640 1 -- 172.21.15.43:0/30712542 >> 172.21.15.43:0/30712542 conn(0x7f1c2c09b7b0 msgr2=0x7f1c2c0b7980 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-09T13:51:17.134 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:17.130+0000 7f1c30dd6640 1 -- 172.21.15.43:0/30712542 shutdown_connections 2023-08-09T13:51:17.135 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:17.130+0000 7f1c30dd6640 1 -- 172.21.15.43:0/30712542 wait complete. 2023-08-09T13:51:17.153 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs standby_count_wanted 0 2023-08-09T13:51:17.182 INFO:tasks.ceph.mds.a.smithi043.stderr:2023-08-09T13:51:17.180+0000 7f4b02b15640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2023-08-09T13:51:17.355 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:17.353+0000 7fe1f9d64640 1 Processor -- start 2023-08-09T13:51:17.356 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:17.353+0000 7fe1f9d64640 1 -- start start 2023-08-09T13:51:17.356 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:17.354+0000 7fe1f9d64640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7fe1f40ce720 0x7fe1f40ceb00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:51:17.357 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:17.354+0000 7fe1f9d64640 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7fe1f40ce080 0x7fe1f415e5b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:51:17.357 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:17.354+0000 7fe1f9d64640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fe1f415eaf0 0x7fe1f415ef80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:51:17.357 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:17.354+0000 7fe1f9d64640 1 -- --> v2:172.21.15.43:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe1f40cd8a0 con 0x7fe1f40ce720 2023-08-09T13:51:17.357 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:17.354+0000 7fe1f9d64640 1 -- --> v2:172.21.15.203:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe1f40cf350 con 0x7fe1f40ce080 2023-08-09T13:51:17.357 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:17.354+0000 7fe1f9d64640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe1f40cf040 con 0x7fe1f415eaf0 2023-08-09T13:51:17.358 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:17.354+0000 7fe1f37fe640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7fe1f40ce720 0x7fe1f40ceb00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:51:17.358 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:17.355+0000 7fe1f37fe640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7fe1f40ce720 0x7fe1f40ceb00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:42872/0 (socket says 172.21.15.43:42872) 2023-08-09T13:51:17.358 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:17.355+0000 7fe1f37fe640 1 -- 172.21.15.43:0/1894738885 learned_addr learned my addr 172.21.15.43:0/1894738885 (peer_addr_for_me v2:172.21.15.43:0/0) 2023-08-09T13:51:17.359 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:17.355+0000 7fe1f37fe640 1 -- 172.21.15.43:0/1894738885 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fe1f415eaf0 msgr2=0x7fe1f415ef80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-09T13:51:17.359 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:17.355+0000 7fe1f37fe640 1 --2- 172.21.15.43:0/1894738885 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fe1f415eaf0 0x7fe1f415ef80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:51:17.359 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:17.355+0000 7fe1f37fe640 1 -- 172.21.15.43:0/1894738885 >> v2:172.21.15.203:3300/0 conn(0x7fe1f40ce080 msgr2=0x7fe1f415e5b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-09T13:51:17.359 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:17.355+0000 7fe1f37fe640 1 --2- 172.21.15.43:0/1894738885 >> v2:172.21.15.203:3300/0 conn(0x7fe1f40ce080 0x7fe1f415e5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:51:17.359 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:17.355+0000 7fe1f37fe640 1 -- 172.21.15.43:0/1894738885 --> v2:172.21.15.43:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe1f4071a70 con 0x7fe1f40ce720 2023-08-09T13:51:17.359 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:17.356+0000 7fe1f37fe640 1 --2- 172.21.15.43:0/1894738885 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fe1f40ce720 0x7fe1f40ceb00 secure :-1 s=READY pgs=1180 cs=0 l=1 rev1=1 crypto rx=0x7fe1f40708f0 tx=0x7fe1e402be90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e5e9d13d9bacb23f server_cookie=0 in_seq=0 out_seq=0 2023-08-09T13:51:17.360 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:17.356+0000 7fe1f27fc640 1 -- 172.21.15.43:0/1894738885 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe1e402ca30 con 0x7fe1f40ce720 2023-08-09T13:51:17.360 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:17.356+0000 7fe1f27fc640 1 -- 172.21.15.43:0/1894738885 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe1e402cb90 con 0x7fe1f40ce720 2023-08-09T13:51:17.361 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:17.356+0000 7fe1f27fc640 1 -- 172.21.15.43:0/1894738885 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe1e40343f0 con 0x7fe1f40ce720 2023-08-09T13:51:17.361 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:17.357+0000 7fe1f9d64640 1 -- 172.21.15.43:0/1894738885 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fe1f40ce720 msgr2=0x7fe1f40ceb00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:51:17.361 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:17.357+0000 7fe1f9d64640 1 --2- 172.21.15.43:0/1894738885 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fe1f40ce720 0x7fe1f40ceb00 secure :-1 s=READY pgs=1180 cs=0 l=1 rev1=1 crypto rx=0x7fe1f40708f0 tx=0x7fe1e402be90 comp rx=0 tx=0).stop 2023-08-09T13:51:17.361 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:17.357+0000 7fe1f9d64640 1 -- 172.21.15.43:0/1894738885 shutdown_connections 2023-08-09T13:51:17.361 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:17.357+0000 7fe1f9d64640 1 --2- 172.21.15.43:0/1894738885 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fe1f415eaf0 0x7fe1f415ef80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:51:17.361 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:17.357+0000 7fe1f9d64640 1 --2- 172.21.15.43:0/1894738885 >> v2:172.21.15.203:3300/0 conn(0x7fe1f40ce080 0x7fe1f415e5b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:51:17.362 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:17.357+0000 7fe1f9d64640 1 --2- 172.21.15.43:0/1894738885 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fe1f40ce720 0x7fe1f40ceb00 unknown :-1 s=CLOSED pgs=1180 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:51:17.362 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:17.357+0000 7fe1f9d64640 1 -- 172.21.15.43:0/1894738885 >> 172.21.15.43:0/1894738885 conn(0x7fe1f40ccd80 msgr2=0x7fe1f40cd190 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-09T13:51:17.362 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:17.357+0000 7fe1f9d64640 1 -- 172.21.15.43:0/1894738885 shutdown_connections 2023-08-09T13:51:17.362 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:17.357+0000 7fe1f9d64640 1 -- 172.21.15.43:0/1894738885 wait complete. 2023-08-09T13:51:17.363 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:17.359+0000 7fe1f9d64640 1 Processor -- start 2023-08-09T13:51:17.363 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:17.359+0000 7fe1f9d64640 1 -- start start 2023-08-09T13:51:17.363 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:17.360+0000 7fe1f9d64640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fe1f40ce080 0x7fe1f414f290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:51:17.364 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:17.360+0000 7fe1f9d64640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fe1f40ce720 0x7fe1f40c59e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:51:17.364 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:17.360+0000 7fe1f9d64640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fe1f415eaf0 0x7fe1f4159040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:51:17.364 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:17.360+0000 7fe1f9d64640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe1f40c62e0 con 0x7fe1f40ce720 2023-08-09T13:51:17.364 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:17.360+0000 7fe1f9d64640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe1f40c3d50 con 0x7fe1f40ce080 2023-08-09T13:51:17.364 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:17.360+0000 7fe1f9d64640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe1f40c48d0 con 0x7fe1f415eaf0 2023-08-09T13:51:17.364 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:17.360+0000 7fe1f2ffd640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fe1f40ce720 0x7fe1f40c59e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:51:17.364 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:17.360+0000 7fe1f2ffd640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fe1f40ce720 0x7fe1f40c59e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:42878/0 (socket says 172.21.15.43:42878) 2023-08-09T13:51:17.365 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:17.360+0000 7fe1f2ffd640 1 -- 172.21.15.43:0/878464795 learned_addr learned my addr 172.21.15.43:0/878464795 (peer_addr_for_me v2:172.21.15.43:0/0) 2023-08-09T13:51:17.365 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:17.361+0000 7fe1f37fe640 1 --2- 172.21.15.43:0/878464795 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fe1f40ce080 0x7fe1f414f290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:51:17.365 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:17.361+0000 7fe1f3fff640 1 --2- 172.21.15.43:0/878464795 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fe1f415eaf0 0x7fe1f4159040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:51:17.365 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:17.361+0000 7fe1f2ffd640 1 -- 172.21.15.43:0/878464795 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fe1f415eaf0 msgr2=0x7fe1f4159040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:51:17.365 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:17.361+0000 7fe1f2ffd640 1 --2- 172.21.15.43:0/878464795 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fe1f415eaf0 0x7fe1f4159040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:51:17.366 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:17.361+0000 7fe1f2ffd640 1 -- 172.21.15.43:0/878464795 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fe1f40ce080 msgr2=0x7fe1f414f290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:51:17.366 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:17.361+0000 7fe1f2ffd640 1 --2- 172.21.15.43:0/878464795 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fe1f40ce080 0x7fe1f414f290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:51:17.366 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:17.361+0000 7fe1f2ffd640 1 -- 172.21.15.43:0/878464795 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe1e4009b30 con 0x7fe1f40ce720 2023-08-09T13:51:17.366 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:17.361+0000 7fe1f3fff640 1 --2- 172.21.15.43:0/878464795 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fe1f415eaf0 0x7fe1f4159040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-08-09T13:51:17.367 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:17.361+0000 7fe1f2ffd640 1 --2- 172.21.15.43:0/878464795 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fe1f40ce720 0x7fe1f40c59e0 secure :-1 s=READY pgs=1181 cs=0 l=1 rev1=1 crypto rx=0x7fe1e800b440 tx=0x7fe1e800b900 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-09T13:51:17.367 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:17.361+0000 7fe1dbfff640 1 -- 172.21.15.43:0/878464795 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe1e8011890 con 0x7fe1f40ce720 2023-08-09T13:51:17.367 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:17.361+0000 7fe1dbfff640 1 -- 172.21.15.43:0/878464795 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe1e80119f0 con 0x7fe1f40ce720 2023-08-09T13:51:17.367 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:17.361+0000 7fe1dbfff640 1 -- 172.21.15.43:0/878464795 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe1e8011da0 con 0x7fe1f40ce720 2023-08-09T13:51:17.367 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:17.362+0000 7fe1f9d64640 1 -- 172.21.15.43:0/878464795 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe1f40c4d60 con 0x7fe1f40ce720 2023-08-09T13:51:17.368 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:17.362+0000 7fe1f9d64640 1 -- 172.21.15.43:0/878464795 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe1f40c4280 con 0x7fe1f40ce720 2023-08-09T13:51:17.368 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:17.363+0000 7fe1f9d64640 1 -- 172.21.15.43:0/878464795 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe1f40739f0 con 0x7fe1f40ce720 2023-08-09T13:51:17.368 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:17.364+0000 7fe1dbfff640 1 -- 172.21.15.43:0/878464795 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 37) v1 ==== 88443+0+0 (secure 0 0 0) 0x7fe1e801a400 con 0x7fe1f40ce720 2023-08-09T13:51:17.368 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:17.365+0000 7fe1dbfff640 1 --2- 172.21.15.43:0/878464795 >> [v2:172.21.15.203:6832/1778614695,v1:172.21.15.203:6833/1778614695] conn(0x7fe1e006a170 0x7fe1e006c630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:51:17.369 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:17.365+0000 7fe1dbfff640 1 -- 172.21.15.43:0/878464795 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(103..103 src has 1..103) v4 ==== 6426+0+0 (secure 0 0 0) 0x7fe1e8088a90 con 0x7fe1f40ce720 2023-08-09T13:51:17.369 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:17.366+0000 7fe1f37fe640 1 --2- 172.21.15.43:0/878464795 >> [v2:172.21.15.203:6832/1778614695,v1:172.21.15.203:6833/1778614695] conn(0x7fe1e006a170 0x7fe1e006c630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:51:17.369 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:17.367+0000 7fe1f37fe640 1 --2- 172.21.15.43:0/878464795 >> [v2:172.21.15.203:6832/1778614695,v1:172.21.15.203:6833/1778614695] conn(0x7fe1e006a170 0x7fe1e006c630 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7fe1e4035000 tx=0x7fe1e4030000 comp rx=0 tx=0).ready entity=mgr.5819 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-09T13:51:17.371 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:17.369+0000 7fe1dbfff640 1 -- 172.21.15.43:0/878464795 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178066 (secure 0 0 0) 0x7fe1e8053f90 con 0x7fe1f40ce720 2023-08-09T13:51:17.607 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:17.605+0000 7fe1f9d64640 1 -- 172.21.15.43:0/878464795 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} v 0) v1 -- 0x7fe1f40c4a10 con 0x7fe1f40ce720 2023-08-09T13:51:18.256 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:18.253+0000 7fe1dbfff640 1 -- 172.21.15.43:0/878464795 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]=0 v42) v1 ==== 118+0+0 (secure 0 0 0) 0x7fe1e80150b0 con 0x7fe1f40ce720 2023-08-09T13:51:18.259 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:18.256+0000 7fe1f9d64640 1 -- 172.21.15.43:0/878464795 >> [v2:172.21.15.203:6832/1778614695,v1:172.21.15.203:6833/1778614695] conn(0x7fe1e006a170 msgr2=0x7fe1e006c630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:51:18.259 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:18.256+0000 7fe1f9d64640 1 --2- 172.21.15.43:0/878464795 >> [v2:172.21.15.203:6832/1778614695,v1:172.21.15.203:6833/1778614695] conn(0x7fe1e006a170 0x7fe1e006c630 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7fe1e4035000 tx=0x7fe1e4030000 comp rx=0 tx=0).stop 2023-08-09T13:51:18.259 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:18.256+0000 7fe1f9d64640 1 -- 172.21.15.43:0/878464795 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fe1f40ce720 msgr2=0x7fe1f40c59e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:51:18.259 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:18.256+0000 7fe1f9d64640 1 --2- 172.21.15.43:0/878464795 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fe1f40ce720 0x7fe1f40c59e0 secure :-1 s=READY pgs=1181 cs=0 l=1 rev1=1 crypto rx=0x7fe1e800b440 tx=0x7fe1e800b900 comp rx=0 tx=0).stop 2023-08-09T13:51:18.260 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:18.256+0000 7fe1f9d64640 1 -- 172.21.15.43:0/878464795 shutdown_connections 2023-08-09T13:51:18.260 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:18.256+0000 7fe1f9d64640 1 --2- 172.21.15.43:0/878464795 >> [v2:172.21.15.203:6832/1778614695,v1:172.21.15.203:6833/1778614695] conn(0x7fe1e006a170 0x7fe1e006c630 unknown :-1 s=CLOSED pgs=60 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:51:18.260 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:18.256+0000 7fe1f9d64640 1 --2- 172.21.15.43:0/878464795 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fe1f415eaf0 0x7fe1f4159040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:51:18.260 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:18.256+0000 7fe1f9d64640 1 --2- 172.21.15.43:0/878464795 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fe1f40ce720 0x7fe1f40c59e0 unknown :-1 s=CLOSED pgs=1181 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:51:18.260 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:18.256+0000 7fe1f9d64640 1 --2- 172.21.15.43:0/878464795 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fe1f40ce080 0x7fe1f414f290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:51:18.260 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:18.256+0000 7fe1f9d64640 1 -- 172.21.15.43:0/878464795 >> 172.21.15.43:0/878464795 conn(0x7fe1f40ccd80 msgr2=0x7fe1f40ae5c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-09T13:51:18.260 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:18.257+0000 7fe1f9d64640 1 -- 172.21.15.43:0/878464795 shutdown_connections 2023-08-09T13:51:18.261 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:18.257+0000 7fe1f9d64640 1 -- 172.21.15.43:0/878464795 wait complete. 2023-08-09T13:51:18.281 DEBUG:tasks.cephfs.filesystem:fs_config: {} 2023-08-09T13:51:18.282 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs allow_standby_replay false 2023-08-09T13:51:18.565 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:18.562+0000 7f8314301640 1 Processor -- start 2023-08-09T13:51:18.565 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:18.562+0000 7f8314301640 1 -- start start 2023-08-09T13:51:18.567 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:18.564+0000 7f8314301640 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7f830c0c86f0 0x7f830c0c8b50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:51:18.567 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:18.564+0000 7f8314301640 1 -- --> v1:172.21.15.43:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f830c0c9090 con 0x7f830c143c20 2023-08-09T13:51:18.567 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:18.564+0000 7f8314301640 1 -- --> v1:172.21.15.203:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f830c06c440 con 0x7f830c06c8d0 2023-08-09T13:51:18.567 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:18.564+0000 7f8314301640 1 -- --> v2:172.21.15.203:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f830c144000 con 0x7f830c0c86f0 2023-08-09T13:51:18.568 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:18.565+0000 7f831209d640 1 --1- >> v1:172.21.15.43:6789/0 conn(0x7f830c143c20 0x7f830c06f540 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.43:6789/0 says I am v1:172.21.15.43:59376/0 (socket says 172.21.15.43:59376) 2023-08-09T13:51:18.568 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:18.565+0000 7f831209d640 1 -- 172.21.15.43:0/923870681 learned_addr learned my addr 172.21.15.43:0/923870681 (peer_addr_for_me v1:172.21.15.43:0/0) 2023-08-09T13:51:18.568 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:18.565+0000 7f831189c640 1 --2- 172.21.15.43:0/923870681 >> v2:172.21.15.203:3300/0 conn(0x7f830c0c86f0 0x7f830c0c8b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:51:18.568 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:18.565+0000 7f831109b640 1 -- 172.21.15.43:0/923870681 <== mon.0 v1:172.21.15.43:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2558370637 0 0) 0x7f830c0c9090 con 0x7f830c143c20 2023-08-09T13:51:18.568 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:18.565+0000 7f831109b640 1 -- 172.21.15.43:0/923870681 --> v1:172.21.15.43:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f82f8003520 con 0x7f830c143c20 2023-08-09T13:51:18.570 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:18.566+0000 7f831109b640 1 -- 172.21.15.43:0/923870681 <== mon.0 v1:172.21.15.43:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3078982004 0 0) 0x7f82f8003520 con 0x7f830c143c20 2023-08-09T13:51:18.570 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:18.566+0000 7f831109b640 1 -- 172.21.15.43:0/923870681 >> v2:172.21.15.203:3300/0 conn(0x7f830c0c86f0 msgr2=0x7f830c0c8b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-09T13:51:18.570 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:18.566+0000 7f831109b640 1 --2- 172.21.15.43:0/923870681 >> v2:172.21.15.203:3300/0 conn(0x7f830c0c86f0 0x7f830c0c8b50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:51:18.570 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:18.566+0000 7f831109b640 1 -- 172.21.15.43:0/923870681 >> v1:172.21.15.203:6789/0 conn(0x7f830c06c8d0 legacy=0x7f830c06cd00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:51:18.570 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:18.566+0000 7f831109b640 1 -- 172.21.15.43:0/923870681 --> v1:172.21.15.43:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f830c06fc50 con 0x7f830c143c20 2023-08-09T13:51:18.571 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:18.566+0000 7f831109b640 1 -- 172.21.15.43:0/923870681 <== mon.0 v1:172.21.15.43:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1285150740 0 0) 0x7f8308001670 con 0x7f830c143c20 2023-08-09T13:51:18.571 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:18.566+0000 7f831109b640 1 -- 172.21.15.43:0/923870681 >> v1:172.21.15.43:6789/0 conn(0x7f830c143c20 legacy=0x7f830c06f540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:51:18.571 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:18.566+0000 7f831109b640 1 --2- 172.21.15.43:0/923870681 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f82f80039e0 0x7f82f8003e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:51:18.571 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:18.566+0000 7f831109b640 1 --2- 172.21.15.43:0/923870681 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f82f80047c0 0x7f82f8004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:51:18.571 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:18.566+0000 7f831109b640 1 --2- 172.21.15.43:0/923870681 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f82f8005280 0x7f82f800b980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:51:18.571 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:18.566+0000 7f831109b640 1 -- 172.21.15.43:0/923870681 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f830c144000 con 0x7f82f80039e0 2023-08-09T13:51:18.571 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:18.566+0000 7f831109b640 1 -- 172.21.15.43:0/923870681 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f82f800c1c0 con 0x7f82f8005280 2023-08-09T13:51:18.572 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:18.566+0000 7f831109b640 1 -- 172.21.15.43:0/923870681 --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f82f800c380 con 0x7f82f80047c0 2023-08-09T13:51:18.572 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:18.566+0000 7f831209d640 1 --2- 172.21.15.43:0/923870681 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f82f80039e0 0x7f82f8003e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:51:18.572 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:18.567+0000 7f831289e640 1 --2- 172.21.15.43:0/923870681 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f82f8005280 0x7f82f800b980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:51:18.572 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:18.567+0000 7f831189c640 1 --2- 172.21.15.43:0/923870681 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f82f80047c0 0x7f82f8004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:51:18.572 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:18.567+0000 7f831209d640 1 -- 172.21.15.43:0/923870681 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f82f80047c0 msgr2=0x7f82f8004c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-09T13:51:18.572 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:18.567+0000 7f831209d640 1 --2- 172.21.15.43:0/923870681 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f82f80047c0 0x7f82f8004c40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:51:18.573 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:18.567+0000 7f831209d640 1 -- 172.21.15.43:0/923870681 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f82f8005280 msgr2=0x7f82f800b980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-09T13:51:18.573 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:18.567+0000 7f831209d640 1 --2- 172.21.15.43:0/923870681 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f82f8005280 0x7f82f800b980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:51:18.573 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:18.567+0000 7f831209d640 1 -- 172.21.15.43:0/923870681 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f82f8003520 con 0x7f82f80039e0 2023-08-09T13:51:18.573 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:18.567+0000 7f831289e640 1 -- 172.21.15.43:0/923870681 reap_dead start 2023-08-09T13:51:18.574 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:18.568+0000 7f831209d640 1 --2- 172.21.15.43:0/923870681 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f82f80039e0 0x7f82f8003e00 secure :-1 s=READY pgs=1183 cs=0 l=1 rev1=1 crypto rx=0x7f830800c240 tx=0x7f82f80057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5c9242bc9a6f6730 server_cookie=0 in_seq=0 out_seq=0 2023-08-09T13:51:18.575 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:18.568+0000 7f831109b640 1 -- 172.21.15.43:0/923870681 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8308005ad0 con 0x7f82f80039e0 2023-08-09T13:51:18.575 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:18.569+0000 7f831109b640 1 -- 172.21.15.43:0/923870681 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8308005c30 con 0x7f82f80039e0 2023-08-09T13:51:18.575 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:18.569+0000 7f831109b640 1 -- 172.21.15.43:0/923870681 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f830802caf0 con 0x7f82f80039e0 2023-08-09T13:51:18.576 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:18.569+0000 7f8314301640 1 -- 172.21.15.43:0/923870681 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f82f80039e0 msgr2=0x7f82f8003e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:51:18.576 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:18.569+0000 7f8314301640 1 --2- 172.21.15.43:0/923870681 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f82f80039e0 0x7f82f8003e00 secure :-1 s=READY pgs=1183 cs=0 l=1 rev1=1 crypto rx=0x7f830800c240 tx=0x7f82f80057f0 comp rx=0 tx=0).stop 2023-08-09T13:51:18.576 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:18.569+0000 7f8314301640 1 -- 172.21.15.43:0/923870681 shutdown_connections 2023-08-09T13:51:18.577 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:18.569+0000 7f8314301640 1 --2- 172.21.15.43:0/923870681 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f82f80039e0 0x7f82f8003e00 unknown :-1 s=CLOSED pgs=1183 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:51:18.577 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:18.570+0000 7f8314301640 1 -- 172.21.15.43:0/923870681 >> 172.21.15.43:0/923870681 conn(0x7f830c0ce0a0 msgr2=0x7f830c0ce480 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-09T13:51:18.577 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:18.570+0000 7f8314301640 1 -- 172.21.15.43:0/923870681 shutdown_connections 2023-08-09T13:51:18.577 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:18.570+0000 7f8314301640 1 -- 172.21.15.43:0/923870681 wait complete. 2023-08-09T13:51:18.578 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:18.572+0000 7f8314301640 1 Processor -- start 2023-08-09T13:51:18.578 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:18.572+0000 7f8314301640 1 -- start start 2023-08-09T13:51:18.582 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:18.574+0000 7f8314301640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f82f80039e0 0x7f830c159fa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:51:18.582 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:18.574+0000 7f8314301640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f830c0aca90 0x7f830c0b1840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:51:18.582 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:18.574+0000 7f8314301640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f830c0acf10 0x7f830c0b1d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:51:18.582 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:18.574+0000 7f8314301640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f830c15d0b0 con 0x7f830c0aca90 2023-08-09T13:51:18.583 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:18.574+0000 7f8314301640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f830c15c430 con 0x7f82f80039e0 2023-08-09T13:51:18.583 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:18.574+0000 7f8314301640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f830c15c710 con 0x7f830c0acf10 2023-08-09T13:51:18.583 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:18.574+0000 7f831189c640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f830c0aca90 0x7f830c0b1840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:51:18.583 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:18.574+0000 7f831189c640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f830c0aca90 0x7f830c0b1840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:42902/0 (socket says 172.21.15.43:42902) 2023-08-09T13:51:18.584 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:18.574+0000 7f831189c640 1 -- 172.21.15.43:0/2827664297 learned_addr learned my addr 172.21.15.43:0/2827664297 (peer_addr_for_me v2:172.21.15.43:0/0) 2023-08-09T13:51:18.584 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:18.574+0000 7f831209d640 1 --2- 172.21.15.43:0/2827664297 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f82f80039e0 0x7f830c159fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:51:18.584 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:18.574+0000 7f831289e640 1 --2- 172.21.15.43:0/2827664297 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f830c0acf10 0x7f830c0b1d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:51:18.584 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:18.575+0000 7f831189c640 1 -- 172.21.15.43:0/2827664297 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f830c0acf10 msgr2=0x7f830c0b1d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:51:18.585 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:18.575+0000 7f831189c640 1 --2- 172.21.15.43:0/2827664297 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f830c0acf10 0x7f830c0b1d80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:51:18.585 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:18.575+0000 7f831189c640 1 -- 172.21.15.43:0/2827664297 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f82f80039e0 msgr2=0x7f830c159fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:51:18.585 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:18.575+0000 7f831189c640 1 --2- 172.21.15.43:0/2827664297 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f82f80039e0 0x7f830c159fa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:51:18.585 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:18.575+0000 7f831189c640 1 -- 172.21.15.43:0/2827664297 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f830c15c9f0 con 0x7f830c0aca90 2023-08-09T13:51:18.585 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:18.575+0000 7f831189c640 1 --2- 172.21.15.43:0/2827664297 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f830c0aca90 0x7f830c0b1840 secure :-1 s=READY pgs=1184 cs=0 l=1 rev1=1 crypto rx=0x7f82fc0108e0 tx=0x7f82fc010da0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-09T13:51:18.585 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:18.575+0000 7f8302ffd640 1 -- 172.21.15.43:0/2827664297 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f82fc00eb60 con 0x7f830c0aca90 2023-08-09T13:51:18.585 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:18.575+0000 7f8302ffd640 1 -- 172.21.15.43:0/2827664297 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f82fc00ecc0 con 0x7f830c0aca90 2023-08-09T13:51:18.586 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:18.575+0000 7f8302ffd640 1 -- 172.21.15.43:0/2827664297 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f82fc01c400 con 0x7f830c0aca90 2023-08-09T13:51:18.586 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:18.575+0000 7f8314301640 1 -- 172.21.15.43:0/2827664297 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f830c15ccd0 con 0x7f830c0aca90 2023-08-09T13:51:18.586 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:18.576+0000 7f8314301640 1 -- 172.21.15.43:0/2827664297 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f830c15c090 con 0x7f830c0aca90 2023-08-09T13:51:18.586 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:18.577+0000 7f8314301640 1 -- 172.21.15.43:0/2827664297 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f830c0739f0 con 0x7f830c0aca90 2023-08-09T13:51:18.586 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:18.578+0000 7f8302ffd640 1 -- 172.21.15.43:0/2827664297 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 37) v1 ==== 88443+0+0 (secure 0 0 0) 0x7f82fc01c560 con 0x7f830c0aca90 2023-08-09T13:51:18.586 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:18.579+0000 7f8302ffd640 1 --2- 172.21.15.43:0/2827664297 >> [v2:172.21.15.203:6832/1778614695,v1:172.21.15.203:6833/1778614695] conn(0x7f82f006a1a0 0x7f82f006c660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:51:18.586 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:18.579+0000 7f8302ffd640 1 -- 172.21.15.43:0/2827664297 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(103..103 src has 1..103) v4 ==== 6426+0+0 (secure 0 0 0) 0x7f82fc088e50 con 0x7f830c0aca90 2023-08-09T13:51:18.587 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:18.580+0000 7f831209d640 1 --2- 172.21.15.43:0/2827664297 >> [v2:172.21.15.203:6832/1778614695,v1:172.21.15.203:6833/1778614695] conn(0x7f82f006a1a0 0x7f82f006c660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:51:18.587 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:18.581+0000 7f831209d640 1 --2- 172.21.15.43:0/2827664297 >> [v2:172.21.15.203:6832/1778614695,v1:172.21.15.203:6833/1778614695] conn(0x7f82f006a1a0 0x7f82f006c660 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f8308034000 tx=0x7f8308009030 comp rx=0 tx=0).ready entity=mgr.5819 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-09T13:51:18.587 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:18.583+0000 7f8302ffd640 1 -- 172.21.15.43:0/2827664297 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178066 (secure 0 0 0) 0x7f82fc054360 con 0x7f830c0aca90 2023-08-09T13:51:18.821 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:18.818+0000 7f8314301640 1 -- 172.21.15.43:0/2827664297 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"} v 0) v1 -- 0x7f830c06c0c0 con 0x7f830c0aca90 2023-08-09T13:51:19.260 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:19.257+0000 7f8302ffd640 1 -- 172.21.15.43:0/2827664297 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"}]=0 v43) v1 ==== 122+0+0 (secure 0 0 0) 0x7f82fc057980 con 0x7f830c0aca90 2023-08-09T13:51:19.263 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:19.261+0000 7f8314301640 1 -- 172.21.15.43:0/2827664297 >> [v2:172.21.15.203:6832/1778614695,v1:172.21.15.203:6833/1778614695] conn(0x7f82f006a1a0 msgr2=0x7f82f006c660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:51:19.263 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:19.261+0000 7f8314301640 1 --2- 172.21.15.43:0/2827664297 >> [v2:172.21.15.203:6832/1778614695,v1:172.21.15.203:6833/1778614695] conn(0x7f82f006a1a0 0x7f82f006c660 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f8308034000 tx=0x7f8308009030 comp rx=0 tx=0).stop 2023-08-09T13:51:19.264 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:19.261+0000 7f8314301640 1 -- 172.21.15.43:0/2827664297 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f830c0aca90 msgr2=0x7f830c0b1840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:51:19.264 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:19.261+0000 7f8314301640 1 --2- 172.21.15.43:0/2827664297 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f830c0aca90 0x7f830c0b1840 secure :-1 s=READY pgs=1184 cs=0 l=1 rev1=1 crypto rx=0x7f82fc0108e0 tx=0x7f82fc010da0 comp rx=0 tx=0).stop 2023-08-09T13:51:19.264 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:19.261+0000 7f8314301640 1 -- 172.21.15.43:0/2827664297 shutdown_connections 2023-08-09T13:51:19.264 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:19.261+0000 7f8314301640 1 --2- 172.21.15.43:0/2827664297 >> [v2:172.21.15.203:6832/1778614695,v1:172.21.15.203:6833/1778614695] conn(0x7f82f006a1a0 0x7f82f006c660 unknown :-1 s=CLOSED pgs=61 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:51:19.264 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:19.261+0000 7f8314301640 1 --2- 172.21.15.43:0/2827664297 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f830c0acf10 0x7f830c0b1d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:51:19.264 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:19.261+0000 7f8314301640 1 --2- 172.21.15.43:0/2827664297 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f830c0aca90 0x7f830c0b1840 unknown :-1 s=CLOSED pgs=1184 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:51:19.265 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:19.261+0000 7f8314301640 1 --2- 172.21.15.43:0/2827664297 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f82f80039e0 0x7f830c159fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:51:19.265 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:19.261+0000 7f8314301640 1 -- 172.21.15.43:0/2827664297 >> 172.21.15.43:0/2827664297 conn(0x7f830c0ce0a0 msgr2=0x7f830c0ae9d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-09T13:51:19.265 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:19.261+0000 7f8314301640 1 -- 172.21.15.43:0/2827664297 shutdown_connections 2023-08-09T13:51:19.265 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:19.261+0000 7f8314301640 1 -- 172.21.15.43:0/2827664297 wait complete. 2023-08-09T13:51:19.289 DEBUG:tasks.cephfs.filesystem:Not Creating any subvols for filesystem 'cephfs' 2023-08-09T13:51:19.290 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-08-09T13:51:19.523 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:19.521+0000 7f2ed4c71640 1 Processor -- start 2023-08-09T13:51:19.524 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:19.521+0000 7f2ed4c71640 1 -- start start 2023-08-09T13:51:19.525 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:19.523+0000 7f2ed4c71640 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7f2ed00c8750 0x7f2ed00ad340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:51:19.525 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:19.523+0000 7f2ed4c71640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f2ed00c8b30 0x7f2ed006f410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:51:19.525 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:19.523+0000 7f2ed4c71640 1 -- --> v1:172.21.15.43:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2ed00cbd90 con 0x7f2ed00acdf0 2023-08-09T13:51:19.525 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:19.523+0000 7f2ed4c71640 1 -- --> v2:172.21.15.203:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2ed015e810 con 0x7f2ed00c8750 2023-08-09T13:51:19.526 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:19.523+0000 7f2ed4c71640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2ed00c9f70 con 0x7f2ed00c8b30 2023-08-09T13:51:19.527 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:19.523+0000 7f2ecffff640 1 --1- >> v1:172.21.15.43:6789/0 conn(0x7f2ed00acdf0 0x7f2ed0070f00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.43:6789/0 says I am v1:172.21.15.43:59386/0 (socket says 172.21.15.43:59386) 2023-08-09T13:51:19.527 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:19.523+0000 7f2ecffff640 1 -- 172.21.15.43:0/2573412203 learned_addr learned my addr 172.21.15.43:0/2573412203 (peer_addr_for_me v1:172.21.15.43:0/0) 2023-08-09T13:51:19.527 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:19.523+0000 7f2ecf7fe640 1 --2- 172.21.15.43:0/2573412203 >> v2:172.21.15.203:3300/0 conn(0x7f2ed00c8750 0x7f2ed00ad340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:51:19.527 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:19.523+0000 7f2eceffd640 1 --2- 172.21.15.43:0/2573412203 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f2ed00c8b30 0x7f2ed006f410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:51:19.527 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:19.524+0000 7f2ece7fc640 1 -- 172.21.15.43:0/2573412203 <== mon.0 v1:172.21.15.43:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1432183057 0 0) 0x7f2ed00cbd90 con 0x7f2ed00acdf0 2023-08-09T13:51:19.528 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:19.524+0000 7f2ece7fc640 1 -- 172.21.15.43:0/2573412203 --> v1:172.21.15.43:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f2eb8003520 con 0x7f2ed00acdf0 2023-08-09T13:51:19.528 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:19.524+0000 7f2ece7fc640 1 -- 172.21.15.43:0/2573412203 <== mon.0 v1:172.21.15.43:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1279065371 0 0) 0x7f2eb8003520 con 0x7f2ed00acdf0 2023-08-09T13:51:19.528 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:19.524+0000 7f2ece7fc640 1 -- 172.21.15.43:0/2573412203 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f2ed00c8b30 msgr2=0x7f2ed006f410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-09T13:51:19.528 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:19.524+0000 7f2ece7fc640 1 --2- 172.21.15.43:0/2573412203 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f2ed00c8b30 0x7f2ed006f410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:51:19.528 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:19.524+0000 7f2ece7fc640 1 -- 172.21.15.43:0/2573412203 >> v2:172.21.15.203:3300/0 conn(0x7f2ed00c8750 msgr2=0x7f2ed00ad340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-09T13:51:19.528 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:19.524+0000 7f2ece7fc640 1 --2- 172.21.15.43:0/2573412203 >> v2:172.21.15.203:3300/0 conn(0x7f2ed00c8750 0x7f2ed00ad340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:51:19.528 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:19.524+0000 7f2ece7fc640 1 -- 172.21.15.43:0/2573412203 --> v1:172.21.15.43:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2ed00ca2b0 con 0x7f2ed00acdf0 2023-08-09T13:51:19.528 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:19.524+0000 7f2ece7fc640 1 -- 172.21.15.43:0/2573412203 <== mon.0 v1:172.21.15.43:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1285150740 0 0) 0x7f2ec80019f0 con 0x7f2ed00acdf0 2023-08-09T13:51:19.529 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:19.524+0000 7f2ece7fc640 1 -- 172.21.15.43:0/2573412203 >> v1:172.21.15.43:6789/0 conn(0x7f2ed00acdf0 legacy=0x7f2ed0070f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:51:19.529 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:19.524+0000 7f2ece7fc640 1 --2- 172.21.15.43:0/2573412203 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f2eb8003970 0x7f2eb8003d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:51:19.529 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:19.524+0000 7f2ece7fc640 1 --2- 172.21.15.43:0/2573412203 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f2eb8004360 0x7f2eb8004840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:51:19.529 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:19.524+0000 7f2ece7fc640 1 --2- 172.21.15.43:0/2573412203 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f2eb8004eb0 0x7f2eb800b980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:51:19.529 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:19.525+0000 7f2ece7fc640 1 -- 172.21.15.43:0/2573412203 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2ed015e810 con 0x7f2eb8004eb0 2023-08-09T13:51:19.529 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:19.525+0000 7f2ece7fc640 1 -- 172.21.15.43:0/2573412203 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2ed00c9f70 con 0x7f2eb8004360 2023-08-09T13:51:19.529 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:19.525+0000 7f2ece7fc640 1 -- 172.21.15.43:0/2573412203 --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2eb800bec0 con 0x7f2eb8003970 2023-08-09T13:51:19.531 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:19.525+0000 7f2ecffff640 1 --2- 172.21.15.43:0/2573412203 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f2eb8004eb0 0x7f2eb800b980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:51:19.531 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:19.525+0000 7f2ecf7fe640 1 --2- 172.21.15.43:0/2573412203 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f2eb8003970 0x7f2eb8003d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:51:19.531 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:19.525+0000 7f2eceffd640 1 --2- 172.21.15.43:0/2573412203 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f2eb8004360 0x7f2eb8004840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:51:19.531 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:19.525+0000 7f2ecffff640 1 -- 172.21.15.43:0/2573412203 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f2eb8003970 msgr2=0x7f2eb8003d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-09T13:51:19.531 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:19.525+0000 7f2ecffff640 1 --2- 172.21.15.43:0/2573412203 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f2eb8003970 0x7f2eb8003d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:51:19.532 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:19.525+0000 7f2ecffff640 1 -- 172.21.15.43:0/2573412203 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f2eb8004360 msgr2=0x7f2eb8004840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-09T13:51:19.532 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:19.525+0000 7f2ecffff640 1 --2- 172.21.15.43:0/2573412203 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f2eb8004360 0x7f2eb8004840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:51:19.532 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:19.525+0000 7f2ecffff640 1 -- 172.21.15.43:0/2573412203 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2eb8003520 con 0x7f2eb8004eb0 2023-08-09T13:51:19.532 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:19.526+0000 7f2ecffff640 1 --2- 172.21.15.43:0/2573412203 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f2eb8004eb0 0x7f2eb800b980 secure :-1 s=READY pgs=1186 cs=0 l=1 rev1=1 crypto rx=0x7f2ec800a860 tx=0x7f2eb80057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e7bcb4678ce0dc42 server_cookie=0 in_seq=0 out_seq=0 2023-08-09T13:51:19.532 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:19.527+0000 7f2ece7fc640 1 -- 172.21.15.43:0/2573412203 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2ec8034630 con 0x7f2eb8004eb0 2023-08-09T13:51:19.532 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:19.527+0000 7f2ece7fc640 1 -- 172.21.15.43:0/2573412203 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2ec802f070 con 0x7f2eb8004eb0 2023-08-09T13:51:19.532 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:19.527+0000 7f2ece7fc640 1 -- 172.21.15.43:0/2573412203 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2ec8034a30 con 0x7f2eb8004eb0 2023-08-09T13:51:19.532 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:19.527+0000 7f2ed4c71640 1 -- 172.21.15.43:0/2573412203 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f2eb8004eb0 msgr2=0x7f2eb800b980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:51:19.533 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:19.527+0000 7f2ed4c71640 1 --2- 172.21.15.43:0/2573412203 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f2eb8004eb0 0x7f2eb800b980 secure :-1 s=READY pgs=1186 cs=0 l=1 rev1=1 crypto rx=0x7f2ec800a860 tx=0x7f2eb80057f0 comp rx=0 tx=0).stop 2023-08-09T13:51:19.533 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:19.527+0000 7f2ecffff640 1 -- 172.21.15.43:0/2573412203 reap_dead start 2023-08-09T13:51:19.533 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:19.527+0000 7f2ed4c71640 1 -- 172.21.15.43:0/2573412203 shutdown_connections 2023-08-09T13:51:19.533 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:19.527+0000 7f2ed4c71640 1 -- 172.21.15.43:0/2573412203 >> 172.21.15.43:0/2573412203 conn(0x7f2ed00cde90 msgr2=0x7f2ed00ce2a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-09T13:51:19.533 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:19.528+0000 7f2ed4c71640 1 -- 172.21.15.43:0/2573412203 shutdown_connections 2023-08-09T13:51:19.533 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:19.528+0000 7f2ed4c71640 1 -- 172.21.15.43:0/2573412203 wait complete. 2023-08-09T13:51:19.535 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:19.529+0000 7f2ed4c71640 1 Processor -- start 2023-08-09T13:51:19.535 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:19.530+0000 7f2ed4c71640 1 -- start start 2023-08-09T13:51:19.535 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:19.531+0000 7f2ed4c71640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f2eb8004eb0 0x7f2ed00c3690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:51:19.535 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:19.531+0000 7f2ed4c71640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f2ed01496d0 0x7f2ed00c19c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:51:19.535 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:19.531+0000 7f2ed4c71640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f2ed00c51d0 0x7f2ed00ae000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:51:19.536 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:19.531+0000 7f2ed4c71640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2ed0157c70 con 0x7f2ed01496d0 2023-08-09T13:51:19.536 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:19.531+0000 7f2ed4c71640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2ed00c2330 con 0x7f2eb8004eb0 2023-08-09T13:51:19.536 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:19.531+0000 7f2ed4c71640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2ed00c2610 con 0x7f2ed00c51d0 2023-08-09T13:51:19.536 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:19.531+0000 7f2eceffd640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f2ed01496d0 0x7f2ed00c19c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:51:19.536 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:19.531+0000 7f2eceffd640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f2ed01496d0 0x7f2ed00c19c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:42932/0 (socket says 172.21.15.43:42932) 2023-08-09T13:51:19.536 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:19.531+0000 7f2eceffd640 1 -- 172.21.15.43:0/3123225959 learned_addr learned my addr 172.21.15.43:0/3123225959 (peer_addr_for_me v2:172.21.15.43:0/0) 2023-08-09T13:51:19.536 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:19.531+0000 7f2ecf7fe640 1 --2- 172.21.15.43:0/3123225959 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f2eb8004eb0 0x7f2ed00c3690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:51:19.536 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:19.531+0000 7f2ecffff640 1 --2- 172.21.15.43:0/3123225959 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f2ed00c51d0 0x7f2ed00ae000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:51:19.537 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:19.532+0000 7f2eceffd640 1 -- 172.21.15.43:0/3123225959 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f2ed00c51d0 msgr2=0x7f2ed00ae000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:51:19.537 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:19.532+0000 7f2eceffd640 1 --2- 172.21.15.43:0/3123225959 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f2ed00c51d0 0x7f2ed00ae000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:51:19.537 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:19.532+0000 7f2eceffd640 1 -- 172.21.15.43:0/3123225959 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f2eb8004eb0 msgr2=0x7f2ed00c3690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:51:19.537 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:19.532+0000 7f2eceffd640 1 --2- 172.21.15.43:0/3123225959 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f2eb8004eb0 0x7f2ed00c3690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:51:19.537 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:19.532+0000 7f2eceffd640 1 -- 172.21.15.43:0/3123225959 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2ed00c28f0 con 0x7f2ed01496d0 2023-08-09T13:51:19.537 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:19.532+0000 7f2ecffff640 1 --2- 172.21.15.43:0/3123225959 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f2ed00c51d0 0x7f2ed00ae000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-08-09T13:51:19.537 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:19.532+0000 7f2eceffd640 1 --2- 172.21.15.43:0/3123225959 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f2ed01496d0 0x7f2ed00c19c0 secure :-1 s=READY pgs=1187 cs=0 l=1 rev1=1 crypto rx=0x7f2ec4011590 tx=0x7f2ec4011a50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-09T13:51:19.538 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:19.532+0000 7f2eb7fff640 1 -- 172.21.15.43:0/3123225959 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2ec4004440 con 0x7f2ed01496d0 2023-08-09T13:51:19.538 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:19.532+0000 7f2eb7fff640 1 -- 172.21.15.43:0/3123225959 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2ec4003610 con 0x7f2ed01496d0 2023-08-09T13:51:19.538 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:19.532+0000 7f2eb7fff640 1 -- 172.21.15.43:0/3123225959 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2ec4003980 con 0x7f2ed01496d0 2023-08-09T13:51:19.538 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:19.532+0000 7f2ed4c71640 1 -- 172.21.15.43:0/3123225959 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2ed0157f10 con 0x7f2ed01496d0 2023-08-09T13:51:19.538 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:19.533+0000 7f2ed4c71640 1 -- 172.21.15.43:0/3123225959 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2ed00c20e0 con 0x7f2ed01496d0 2023-08-09T13:51:19.539 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:19.534+0000 7f2ed4c71640 1 -- 172.21.15.43:0/3123225959 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2ed00c9230 con 0x7f2ed01496d0 2023-08-09T13:51:19.539 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:19.535+0000 7f2eb7fff640 1 -- 172.21.15.43:0/3123225959 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 37) v1 ==== 88443+0+0 (secure 0 0 0) 0x7f2ec4003ae0 con 0x7f2ed01496d0 2023-08-09T13:51:19.539 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:19.536+0000 7f2eb7fff640 1 --2- 172.21.15.43:0/3123225959 >> [v2:172.21.15.203:6832/1778614695,v1:172.21.15.203:6833/1778614695] conn(0x7f2ebc06a1a0 0x7f2ebc06c660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:51:19.539 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:19.536+0000 7f2eb7fff640 1 -- 172.21.15.43:0/3123225959 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(103..103 src has 1..103) v4 ==== 6426+0+0 (secure 0 0 0) 0x7f2ec4088c10 con 0x7f2ed01496d0 2023-08-09T13:51:19.539 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:19.537+0000 7f2ecf7fe640 1 --2- 172.21.15.43:0/3123225959 >> [v2:172.21.15.203:6832/1778614695,v1:172.21.15.203:6833/1778614695] conn(0x7f2ebc06a1a0 0x7f2ebc06c660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:51:19.540 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:19.538+0000 7f2ecf7fe640 1 --2- 172.21.15.43:0/3123225959 >> [v2:172.21.15.203:6832/1778614695,v1:172.21.15.203:6833/1778614695] conn(0x7f2ebc06a1a0 0x7f2ebc06c660 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f2ec0009a10 tx=0x7f2ec0008040 comp rx=0 tx=0).ready entity=mgr.5819 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-09T13:51:19.542 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:19.540+0000 7f2eb7fff640 1 -- 172.21.15.43:0/3123225959 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178066 (secure 0 0 0) 0x7f2ec4053fe0 con 0x7f2ed01496d0 2023-08-09T13:51:19.775 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:19.772+0000 7f2ed4c71640 1 -- 172.21.15.43:0/3123225959 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f2ed00739f0 con 0x7f2ed01496d0 2023-08-09T13:51:19.776 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:19.773+0000 7f2eb7fff640 1 -- 172.21.15.43:0/3123225959 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 43 v43) v1 ==== 94+0+4880 (secure 0 0 0) 0x7f2ec4057600 con 0x7f2ed01496d0 2023-08-09T13:51:19.776 INFO:teuthology.orchestra.run.smithi043.stdout: 2023-08-09T13:51:19.777 INFO:teuthology.orchestra.run.smithi043.stdout:{"epoch":43,"default_fscid":5,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5966,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.203:6835/4101044758","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6834","nonce":4101044758},{"type":"v1","addr":"172.21.15.203:6835","nonce":4101044758}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":39},{"gid":6460,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.203:6837/2974664437","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6836","nonce":2974664437},{"type":"v1","addr":"172.21.15.203:6837","nonce":2974664437}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":39},{"gid":7527,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.43:6833/795245144","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6832","nonce":795245144},{"type":"v1","addr":"172.21.15.43:6833","nonce":795245144}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":38}],"filesystems":[{"mdsmap":{"epoch":43,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-08-09T13:51:15.716741+0000","modified":"2023-08-09T13:51:19.249966+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":7542},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7542":{"gid":7542,"name":"a","rank":0,"incarnation":40,"state":"up:active","state_seq":3,"addr":"172.21.15.43:6835/3008795294","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6834","nonce":3008795294},{"type":"v1","addr":"172.21.15.43:6835","nonce":3008795294}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[13],"metadata_pool":12,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":5}]} 2023-08-09T13:51:19.777 INFO:teuthology.orchestra.run.smithi043.stderr:dumped fsmap epoch 43 2023-08-09T13:51:19.779 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:19.776+0000 7f2ed4c71640 1 -- 172.21.15.43:0/3123225959 >> [v2:172.21.15.203:6832/1778614695,v1:172.21.15.203:6833/1778614695] conn(0x7f2ebc06a1a0 msgr2=0x7f2ebc06c660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:51:19.779 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:19.776+0000 7f2ed4c71640 1 --2- 172.21.15.43:0/3123225959 >> [v2:172.21.15.203:6832/1778614695,v1:172.21.15.203:6833/1778614695] conn(0x7f2ebc06a1a0 0x7f2ebc06c660 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f2ec0009a10 tx=0x7f2ec0008040 comp rx=0 tx=0).stop 2023-08-09T13:51:19.780 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:19.777+0000 7f2ed4c71640 1 -- 172.21.15.43:0/3123225959 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f2ed01496d0 msgr2=0x7f2ed00c19c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:51:19.780 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:19.777+0000 7f2ed4c71640 1 --2- 172.21.15.43:0/3123225959 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f2ed01496d0 0x7f2ed00c19c0 secure :-1 s=READY pgs=1187 cs=0 l=1 rev1=1 crypto rx=0x7f2ec4011590 tx=0x7f2ec4011a50 comp rx=0 tx=0).stop 2023-08-09T13:51:19.780 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:19.777+0000 7f2ed4c71640 1 -- 172.21.15.43:0/3123225959 shutdown_connections 2023-08-09T13:51:19.780 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:19.777+0000 7f2ed4c71640 1 --2- 172.21.15.43:0/3123225959 >> [v2:172.21.15.203:6832/1778614695,v1:172.21.15.203:6833/1778614695] conn(0x7f2ebc06a1a0 0x7f2ebc06c660 unknown :-1 s=CLOSED pgs=62 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:51:19.780 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:19.777+0000 7f2ed4c71640 1 --2- 172.21.15.43:0/3123225959 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f2ed00c51d0 0x7f2ed00ae000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:51:19.780 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:19.777+0000 7f2ed4c71640 1 --2- 172.21.15.43:0/3123225959 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f2ed01496d0 0x7f2ed00c19c0 unknown :-1 s=CLOSED pgs=1187 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:51:19.780 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:19.777+0000 7f2ed4c71640 1 --2- 172.21.15.43:0/3123225959 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f2eb8004eb0 0x7f2ed00c3690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:51:19.781 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:19.777+0000 7f2ed4c71640 1 -- 172.21.15.43:0/3123225959 >> 172.21.15.43:0/3123225959 conn(0x7f2ed00cde90 msgr2=0x7f2ed01591d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-09T13:51:19.781 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:19.777+0000 7f2ed4c71640 1 -- 172.21.15.43:0/3123225959 shutdown_connections 2023-08-09T13:51:19.781 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:19.777+0000 7f2ed4c71640 1 -- 172.21.15.43:0/3123225959 wait complete. 2023-08-09T13:51:19.799 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-08-09T13:51:20.029 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:20.026+0000 7f08777cb640 1 Processor -- start 2023-08-09T13:51:20.030 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:20.027+0000 7f08777cb640 1 -- start start 2023-08-09T13:51:20.031 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:20.029+0000 7f08777cb640 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7f08700c86f0 0x7f08700c8b50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:51:20.031 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:20.029+0000 7f08777cb640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f087006c8d0 0x7f087006cd00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:51:20.032 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:20.029+0000 7f08777cb640 1 -- --> v1:172.21.15.203:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f08700c9090 con 0x7f0870143c20 2023-08-09T13:51:20.032 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:20.029+0000 7f08777cb640 1 -- --> v2:172.21.15.43:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0870144000 con 0x7f087006c8d0 2023-08-09T13:51:20.032 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:20.029+0000 7f08777cb640 1 -- --> v2:172.21.15.203:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f087006c440 con 0x7f08700c86f0 2023-08-09T13:51:20.032 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:20.029+0000 7f0875d68640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f087006c8d0 0x7f087006cd00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:51:20.032 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:20.029+0000 7f0875d68640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f087006c8d0 0x7f087006cd00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:42938/0 (socket says 172.21.15.43:42938) 2023-08-09T13:51:20.032 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:20.029+0000 7f0875d68640 1 -- 172.21.15.43:0/3530066923 learned_addr learned my addr 172.21.15.43:0/3530066923 (peer_addr_for_me v2:172.21.15.43:0/0) 2023-08-09T13:51:20.033 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:20.029+0000 7f0874d66640 1 --2- 172.21.15.43:0/3530066923 >> v2:172.21.15.203:3300/0 conn(0x7f08700c86f0 0x7f08700c8b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:51:20.033 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:20.030+0000 7f0875d68640 1 -- 172.21.15.43:0/3530066923 >> v2:172.21.15.203:3300/0 conn(0x7f08700c86f0 msgr2=0x7f08700c8b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-09T13:51:20.033 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:20.030+0000 7f0875d68640 1 --2- 172.21.15.43:0/3530066923 >> v2:172.21.15.203:3300/0 conn(0x7f08700c86f0 0x7f08700c8b50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:51:20.034 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:20.030+0000 7f0875d68640 1 -- 172.21.15.43:0/3530066923 >> v1:172.21.15.203:6789/0 conn(0x7f0870143c20 legacy=0x7f087006f540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-09T13:51:20.034 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:20.030+0000 7f0875d68640 1 -- 172.21.15.43:0/3530066923 --> v2:172.21.15.43:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f087006fc50 con 0x7f087006c8d0 2023-08-09T13:51:20.034 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:20.030+0000 7f0874d66640 1 --2- 172.21.15.43:0/3530066923 >> v2:172.21.15.203:3300/0 conn(0x7f08700c86f0 0x7f08700c8b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-08-09T13:51:20.034 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:20.031+0000 7f0875d68640 1 --2- 172.21.15.43:0/3530066923 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f087006c8d0 0x7f087006cd00 secure :-1 s=READY pgs=1188 cs=0 l=1 rev1=1 crypto rx=0x7f08700c9b70 tx=0x7f086802bee0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4acd7df046d98ab3 server_cookie=0 in_seq=0 out_seq=0 2023-08-09T13:51:20.035 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:20.031+0000 7f0867fff640 1 -- 172.21.15.43:0/3530066923 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f086802ca80 con 0x7f087006c8d0 2023-08-09T13:51:20.035 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:20.031+0000 7f0867fff640 1 -- 172.21.15.43:0/3530066923 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f086802cbe0 con 0x7f087006c8d0 2023-08-09T13:51:20.035 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:20.031+0000 7f0867fff640 1 -- 172.21.15.43:0/3530066923 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f08680343f0 con 0x7f087006c8d0 2023-08-09T13:51:20.035 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:20.032+0000 7f08777cb640 1 -- 172.21.15.43:0/3530066923 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f087006c8d0 msgr2=0x7f087006cd00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:51:20.035 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:20.032+0000 7f08777cb640 1 --2- 172.21.15.43:0/3530066923 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f087006c8d0 0x7f087006cd00 secure :-1 s=READY pgs=1188 cs=0 l=1 rev1=1 crypto rx=0x7f08700c9b70 tx=0x7f086802bee0 comp rx=0 tx=0).stop 2023-08-09T13:51:20.036 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:20.032+0000 7f08777cb640 1 -- 172.21.15.43:0/3530066923 shutdown_connections 2023-08-09T13:51:20.036 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:20.032+0000 7f08777cb640 1 --2- 172.21.15.43:0/3530066923 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f087006c8d0 0x7f087006cd00 unknown :-1 s=CLOSED pgs=1188 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:51:20.036 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:20.032+0000 7f08777cb640 1 --2- 172.21.15.43:0/3530066923 >> v2:172.21.15.203:3300/0 conn(0x7f08700c86f0 0x7f08700c8b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:51:20.036 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:20.032+0000 7f08777cb640 1 -- 172.21.15.43:0/3530066923 >> 172.21.15.43:0/3530066923 conn(0x7f08700ce0a0 msgr2=0x7f08700ce480 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-09T13:51:20.036 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:20.032+0000 7f08777cb640 1 -- 172.21.15.43:0/3530066923 shutdown_connections 2023-08-09T13:51:20.036 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:20.032+0000 7f08777cb640 1 -- 172.21.15.43:0/3530066923 wait complete. 2023-08-09T13:51:20.037 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:20.034+0000 7f08777cb640 1 Processor -- start 2023-08-09T13:51:20.037 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:20.034+0000 7f08777cb640 1 -- start start 2023-08-09T13:51:20.038 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:20.035+0000 7f08777cb640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f087006c8d0 0x7f0870159cd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:51:20.038 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:20.035+0000 7f08777cb640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f08700c86f0 0x7f087015d2b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:51:20.038 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:20.035+0000 7f08777cb640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f0870143c20 0x7f08700c3020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:51:20.038 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:20.035+0000 7f08777cb640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f08700c5210 con 0x7f08700c86f0 2023-08-09T13:51:20.038 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:20.035+0000 7f08777cb640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f087015b620 con 0x7f0870143c20 2023-08-09T13:51:20.038 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:20.035+0000 7f08777cb640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f087015a210 con 0x7f087006c8d0 2023-08-09T13:51:20.039 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:20.035+0000 7f0874d66640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f08700c86f0 0x7f087015d2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:51:20.039 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:20.035+0000 7f0874d66640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f08700c86f0 0x7f087015d2b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:42954/0 (socket says 172.21.15.43:42954) 2023-08-09T13:51:20.039 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:20.035+0000 7f0874d66640 1 -- 172.21.15.43:0/3652919653 learned_addr learned my addr 172.21.15.43:0/3652919653 (peer_addr_for_me v2:172.21.15.43:0/0) 2023-08-09T13:51:20.040 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:20.036+0000 7f0875567640 1 --2- 172.21.15.43:0/3652919653 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f087006c8d0 0x7f0870159cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:51:20.040 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:20.036+0000 7f0875d68640 1 --2- 172.21.15.43:0/3652919653 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f0870143c20 0x7f08700c3020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:51:20.040 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:20.036+0000 7f0874d66640 1 -- 172.21.15.43:0/3652919653 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f087006c8d0 msgr2=0x7f0870159cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:51:20.040 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:20.036+0000 7f0874d66640 1 --2- 172.21.15.43:0/3652919653 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f087006c8d0 0x7f0870159cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:51:20.041 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:20.036+0000 7f0874d66640 1 -- 172.21.15.43:0/3652919653 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f0870143c20 msgr2=0x7f08700c3020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:51:20.041 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:20.036+0000 7f0874d66640 1 --2- 172.21.15.43:0/3652919653 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f0870143c20 0x7f08700c3020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:51:20.041 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:20.036+0000 7f0874d66640 1 -- 172.21.15.43:0/3652919653 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0868009b30 con 0x7f08700c86f0 2023-08-09T13:51:20.041 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:20.036+0000 7f0875567640 1 --2- 172.21.15.43:0/3652919653 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f087006c8d0 0x7f0870159cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-08-09T13:51:20.041 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:20.036+0000 7f0875d68640 1 --2- 172.21.15.43:0/3652919653 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f0870143c20 0x7f08700c3020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-08-09T13:51:20.041 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:20.036+0000 7f0874d66640 1 --2- 172.21.15.43:0/3652919653 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f08700c86f0 0x7f087015d2b0 secure :-1 s=READY pgs=1189 cs=0 l=1 rev1=1 crypto rx=0x7f086000c7f0 tx=0x7f086000ccb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-09T13:51:20.041 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:20.037+0000 7f0865ffb640 1 -- 172.21.15.43:0/3652919653 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0860007820 con 0x7f08700c86f0 2023-08-09T13:51:20.042 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:20.037+0000 7f0865ffb640 1 -- 172.21.15.43:0/3652919653 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0860007980 con 0x7f08700c86f0 2023-08-09T13:51:20.042 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:20.037+0000 7f08777cb640 1 -- 172.21.15.43:0/3652919653 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f087015d7f0 con 0x7f08700c86f0 2023-08-09T13:51:20.042 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:20.037+0000 7f0865ffb640 1 -- 172.21.15.43:0/3652919653 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0860007cf0 con 0x7f08700c86f0 2023-08-09T13:51:20.042 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:20.037+0000 7f08777cb640 1 -- 172.21.15.43:0/3652919653 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f08700c5790 con 0x7f08700c86f0 2023-08-09T13:51:20.043 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:20.038+0000 7f08777cb640 1 -- 172.21.15.43:0/3652919653 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f08700739f0 con 0x7f08700c86f0 2023-08-09T13:51:20.043 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:20.039+0000 7f0865ffb640 1 -- 172.21.15.43:0/3652919653 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 37) v1 ==== 88443+0+0 (secure 0 0 0) 0x7f08600113f0 con 0x7f08700c86f0 2023-08-09T13:51:20.043 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:20.040+0000 7f0865ffb640 1 --2- 172.21.15.43:0/3652919653 >> [v2:172.21.15.203:6832/1778614695,v1:172.21.15.203:6833/1778614695] conn(0x7f085406a170 0x7f085406c630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:51:20.043 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:20.040+0000 7f0865ffb640 1 -- 172.21.15.43:0/3652919653 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(103..103 src has 1..103) v4 ==== 6426+0+0 (secure 0 0 0) 0x7f0860088d70 con 0x7f08700c86f0 2023-08-09T13:51:20.043 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:20.041+0000 7f0875567640 1 --2- 172.21.15.43:0/3652919653 >> [v2:172.21.15.203:6832/1778614695,v1:172.21.15.203:6833/1778614695] conn(0x7f085406a170 0x7f085406c630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:51:20.044 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:20.042+0000 7f0875567640 1 --2- 172.21.15.43:0/3652919653 >> [v2:172.21.15.203:6832/1778614695,v1:172.21.15.203:6833/1778614695] conn(0x7f085406a170 0x7f085406c630 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f086c000f60 tx=0x7f086c009040 comp rx=0 tx=0).ready entity=mgr.5819 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-09T13:51:20.047 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:20.044+0000 7f0865ffb640 1 -- 172.21.15.43:0/3652919653 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178066 (secure 0 0 0) 0x7f0860054040 con 0x7f08700c86f0 2023-08-09T13:51:20.240 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:20.237+0000 7f08777cb640 1 -- 172.21.15.43:0/3652919653 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f08700cec90 con 0x7f08700c86f0 2023-08-09T13:51:20.242 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:20.240+0000 7f0865ffb640 1 -- 172.21.15.43:0/3652919653 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v103) v1 ==== 74+0+18558 (secure 0 0 0) 0x7f086001f020 con 0x7f08700c86f0 2023-08-09T13:51:20.243 INFO:teuthology.orchestra.run.smithi043.stdout: 2023-08-09T13:51:20.243 INFO:teuthology.orchestra.run.smithi043.stdout:{"epoch":103,"fsid":"daac87ee-1337-4250-a955-aea4e408a39b","created":"2023-08-09T13:33:41.701198+0000","modified":"2023-08-09T13:51:17.210912+0000","last_up_change":"2023-08-09T13:33:53.783453+0000","last_in_change":"2023-08-09T13:33:45.148904+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":13,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":"rbd","create_time":"2023-08-09T13:33:54.365137+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":".mgr","create_time":"2023-08-09T13:33:54.831803+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":12,"pool_name":"cephfs_metadata","create_time":"2023-08-09T13:51:14.142563+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"102","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":13,"pool_name":"cephfs_data","create_time":"2023-08-09T13:51:14.712668+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"103","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"cbd49ecd-e896-4fbd-864e-5c37a829b165","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":101,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6800","nonce":502667335},{"type":"v1","addr":"172.21.15.43:6801","nonce":502667335}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6802","nonce":502667335},{"type":"v1","addr":"172.21.15.43:6803","nonce":502667335}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6806","nonce":502667335},{"type":"v1","addr":"172.21.15.43:6807","nonce":502667335}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6804","nonce":502667335},{"type":"v1","addr":"172.21.15.43:6805","nonce":502667335}]},"public_addr":"172.21.15.43:6801/502667335","cluster_addr":"172.21.15.43:6803/502667335","heartbeat_back_addr":"172.21.15.43:6807/502667335","heartbeat_front_addr":"172.21.15.43:6805/502667335","state":["exists","up"]},{"osd":1,"uuid":"75c7e0fe-1ec7-4fe8-a2f2-3661a65b8500","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":101,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6808","nonce":3289134610},{"type":"v1","addr":"172.21.15.43:6809","nonce":3289134610}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6810","nonce":3289134610},{"type":"v1","addr":"172.21.15.43:6812","nonce":3289134610}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6821","nonce":3289134610},{"type":"v1","addr":"172.21.15.43:6824","nonce":3289134610}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6815","nonce":3289134610},{"type":"v1","addr":"172.21.15.43:6818","nonce":3289134610}]},"public_addr":"172.21.15.43:6809/3289134610","cluster_addr":"172.21.15.43:6812/3289134610","heartbeat_back_addr":"172.21.15.43:6824/3289134610","heartbeat_front_addr":"172.21.15.43:6818/3289134610","state":["exists","up"]},{"osd":2,"uuid":"bb5c41e9-7f5f-4125-9b01-3bb8bfc2ea83","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":101,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6811","nonce":4079709736},{"type":"v1","addr":"172.21.15.43:6814","nonce":4079709736}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6817","nonce":4079709736},{"type":"v1","addr":"172.21.15.43:6820","nonce":4079709736}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6829","nonce":4079709736},{"type":"v1","addr":"172.21.15.43:6831","nonce":4079709736}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6823","nonce":4079709736},{"type":"v1","addr":"172.21.15.43:6826","nonce":4079709736}]},"public_addr":"172.21.15.43:6814/4079709736","cluster_addr":"172.21.15.43:6820/4079709736","heartbeat_back_addr":"172.21.15.43:6831/4079709736","heartbeat_front_addr":"172.21.15.43:6826/4079709736","state":["exists","up"]},{"osd":3,"uuid":"bfe4c8b4-694a-465c-87e6-2b2aab852613","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":101,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6813","nonce":1709538759},{"type":"v1","addr":"172.21.15.43:6816","nonce":1709538759}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6819","nonce":1709538759},{"type":"v1","addr":"172.21.15.43:6822","nonce":1709538759}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6828","nonce":1709538759},{"type":"v1","addr":"172.21.15.43:6830","nonce":1709538759}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6825","nonce":1709538759},{"type":"v1","addr":"172.21.15.43:6827","nonce":1709538759}]},"public_addr":"172.21.15.43:6816/1709538759","cluster_addr":"172.21.15.43:6822/1709538759","heartbeat_back_addr":"172.21.15.43:6830/1709538759","heartbeat_front_addr":"172.21.15.43:6827/1709538759","state":["exists","up"]},{"osd":4,"uuid":"fa2e8a77-aba1-4756-853e-0d174a5bdeba","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":101,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6800","nonce":4196866481},{"type":"v1","addr":"172.21.15.203:6801","nonce":4196866481}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6802","nonce":4196866481},{"type":"v1","addr":"172.21.15.203:6803","nonce":4196866481}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6806","nonce":4196866481},{"type":"v1","addr":"172.21.15.203:6807","nonce":4196866481}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6804","nonce":4196866481},{"type":"v1","addr":"172.21.15.203:6805","nonce":4196866481}]},"public_addr":"172.21.15.203:6801/4196866481","cluster_addr":"172.21.15.203:6803/4196866481","heartbeat_back_addr":"172.21.15.203:6807/4196866481","heartbeat_front_addr":"172.21.15.203:6805/4196866481","state":["exists","up"]},{"osd":5,"uuid":"81811e4b-4d24-4f51-a82c-f1d700a2a00d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":101,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6816","nonce":3059290081},{"type":"v1","addr":"172.21.15.203:6817","nonce":3059290081}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6818","nonce":3059290081},{"type":"v1","addr":"172.21.15.203:6819","nonce":3059290081}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6822","nonce":3059290081},{"type":"v1","addr":"172.21.15.203:6823","nonce":3059290081}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6820","nonce":3059290081},{"type":"v1","addr":"172.21.15.203:6821","nonce":3059290081}]},"public_addr":"172.21.15.203:6817/3059290081","cluster_addr":"172.21.15.203:6819/3059290081","heartbeat_back_addr":"172.21.15.203:6823/3059290081","heartbeat_front_addr":"172.21.15.203:6821/3059290081","state":["exists","up"]},{"osd":6,"uuid":"307bac4d-6a5b-443d-af75-56d695d7674f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":101,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6824","nonce":512155569},{"type":"v1","addr":"172.21.15.203:6825","nonce":512155569}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6826","nonce":512155569},{"type":"v1","addr":"172.21.15.203:6827","nonce":512155569}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6830","nonce":512155569},{"type":"v1","addr":"172.21.15.203:6831","nonce":512155569}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6828","nonce":512155569},{"type":"v1","addr":"172.21.15.203:6829","nonce":512155569}]},"public_addr":"172.21.15.203:6825/512155569","cluster_addr":"172.21.15.203:6827/512155569","heartbeat_back_addr":"172.21.15.203:6831/512155569","heartbeat_front_addr":"172.21.15.203:6829/512155569","state":["exists","up"]},{"osd":7,"uuid":"a95dc7ca-9f1b-4295-af70-68a22a2d7d5e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":101,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6808","nonce":3331450438},{"type":"v1","addr":"172.21.15.203:6809","nonce":3331450438}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6810","nonce":3331450438},{"type":"v1","addr":"172.21.15.203:6811","nonce":3331450438}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6814","nonce":3331450438},{"type":"v1","addr":"172.21.15.203:6815","nonce":3331450438}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6812","nonce":3331450438},{"type":"v1","addr":"172.21.15.203:6813","nonce":3331450438}]},"public_addr":"172.21.15.203:6809/3331450438","cluster_addr":"172.21.15.203:6811/3331450438","heartbeat_back_addr":"172.21.15.203:6815/3331450438","heartbeat_front_addr":"172.21.15.203:6813/3331450438","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-09T13:33:50.547652+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-09T13:33:50.589469+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-09T13:33:50.561724+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-09T13:33:50.599650+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-09T13:33:50.411855+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-09T13:33:50.219681+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-09T13:33:50.504664+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-09T13:33:50.441248+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":7,"snaps":[{"begin":104,"length":1},{"begin":221,"length":1}]}],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-08-09T13:51:20.245 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:20.243+0000 7f08777cb640 1 -- 172.21.15.43:0/3652919653 >> [v2:172.21.15.203:6832/1778614695,v1:172.21.15.203:6833/1778614695] conn(0x7f085406a170 msgr2=0x7f085406c630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:51:20.246 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:20.243+0000 7f08777cb640 1 --2- 172.21.15.43:0/3652919653 >> [v2:172.21.15.203:6832/1778614695,v1:172.21.15.203:6833/1778614695] conn(0x7f085406a170 0x7f085406c630 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f086c000f60 tx=0x7f086c009040 comp rx=0 tx=0).stop 2023-08-09T13:51:20.246 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:20.243+0000 7f08777cb640 1 -- 172.21.15.43:0/3652919653 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f08700c86f0 msgr2=0x7f087015d2b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:51:20.246 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:20.243+0000 7f08777cb640 1 --2- 172.21.15.43:0/3652919653 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f08700c86f0 0x7f087015d2b0 secure :-1 s=READY pgs=1189 cs=0 l=1 rev1=1 crypto rx=0x7f086000c7f0 tx=0x7f086000ccb0 comp rx=0 tx=0).stop 2023-08-09T13:51:20.246 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:20.243+0000 7f08777cb640 1 -- 172.21.15.43:0/3652919653 shutdown_connections 2023-08-09T13:51:20.246 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:20.243+0000 7f08777cb640 1 --2- 172.21.15.43:0/3652919653 >> [v2:172.21.15.203:6832/1778614695,v1:172.21.15.203:6833/1778614695] conn(0x7f085406a170 0x7f085406c630 unknown :-1 s=CLOSED pgs=63 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:51:20.246 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:20.243+0000 7f08777cb640 1 --2- 172.21.15.43:0/3652919653 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f0870143c20 0x7f08700c3020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:51:20.247 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:20.243+0000 7f08777cb640 1 --2- 172.21.15.43:0/3652919653 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f08700c86f0 0x7f087015d2b0 unknown :-1 s=CLOSED pgs=1189 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:51:20.247 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:20.243+0000 7f08777cb640 1 --2- 172.21.15.43:0/3652919653 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f087006c8d0 0x7f0870159cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:51:20.247 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:20.243+0000 7f08777cb640 1 -- 172.21.15.43:0/3652919653 >> 172.21.15.43:0/3652919653 conn(0x7f08700ce0a0 msgr2=0x7f08700c40e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-09T13:51:20.247 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:20.243+0000 7f08777cb640 1 -- 172.21.15.43:0/3652919653 shutdown_connections 2023-08-09T13:51:20.247 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:20.243+0000 7f08777cb640 1 -- 172.21.15.43:0/3652919653 wait complete. 2023-08-09T13:51:20.265 INFO:tasks.cephfs.filesystem.ceph_manager:waiting for clean 2023-08-09T13:51:20.265 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-08-09T13:51:20.483 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:20.481+0000 7f3946db5640 1 Processor -- start 2023-08-09T13:51:20.484 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:20.481+0000 7f3946db5640 1 -- start start 2023-08-09T13:51:20.485 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:20.483+0000 7f3946db5640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f3940160f20 0x7f39400c3130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:51:20.486 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:20.483+0000 7f3946db5640 1 -- --> v1:172.21.15.43:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f39400c3670 con 0x7f394014a5c0 2023-08-09T13:51:20.486 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:20.483+0000 7f3946db5640 1 -- --> v1:172.21.15.203:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f39400c38a0 con 0x7f39400cc380 2023-08-09T13:51:20.486 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:20.483+0000 7f3946db5640 1 -- --> v2:172.21.15.43:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f39400c4420 con 0x7f3940160f20 2023-08-09T13:51:20.487 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:20.484+0000 7f3944b51640 1 --1- >> v1:172.21.15.43:6789/0 conn(0x7f394014a5c0 0x7f39400cbc70 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.43:6789/0 says I am v1:172.21.15.43:59392/0 (socket says 172.21.15.43:59392) 2023-08-09T13:51:20.487 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:20.484+0000 7f3944b51640 1 -- 172.21.15.43:0/1957241756 learned_addr learned my addr 172.21.15.43:0/1957241756 (peer_addr_for_me v1:172.21.15.43:0/0) 2023-08-09T13:51:20.488 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:20.484+0000 7f3945352640 1 --2- 172.21.15.43:0/1957241756 >> v2:172.21.15.43:3300/0 conn(0x7f3940160f20 0x7f39400c3130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:51:20.488 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:20.484+0000 7f393f7fe640 1 -- 172.21.15.43:0/1957241756 <== mon.0 v1:172.21.15.43:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2214227674 0 0) 0x7f39400c3670 con 0x7f394014a5c0 2023-08-09T13:51:20.489 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:20.484+0000 7f393f7fe640 1 -- 172.21.15.43:0/1957241756 --> v1:172.21.15.43:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f3930003520 con 0x7f394014a5c0 2023-08-09T13:51:20.489 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:20.484+0000 7f3945352640 1 -- 172.21.15.43:0/1957241756 >> v1:172.21.15.203:6789/0 conn(0x7f39400cc380 legacy=0x7f394006bb50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-09T13:51:20.489 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:20.484+0000 7f3945352640 1 -- 172.21.15.43:0/1957241756 >> v1:172.21.15.43:6789/0 conn(0x7f394014a5c0 legacy=0x7f39400cbc70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:51:20.489 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:20.484+0000 7f3945352640 1 -- 172.21.15.43:0/1957241756 --> v2:172.21.15.43:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f39400c3b80 con 0x7f3940160f20 2023-08-09T13:51:20.490 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:20.485+0000 7f3945352640 1 --2- 172.21.15.43:0/1957241756 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f3940160f20 0x7f39400c3130 secure :-1 s=READY pgs=1191 cs=0 l=1 rev1=1 crypto rx=0x7f3938009d90 tx=0x7f393802bec0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a47f71ebc425267f server_cookie=0 in_seq=0 out_seq=0 2023-08-09T13:51:20.490 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:20.486+0000 7f393f7fe640 1 -- 172.21.15.43:0/1957241756 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f393802cc70 con 0x7f3940160f20 2023-08-09T13:51:20.490 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:20.486+0000 7f393f7fe640 1 -- 172.21.15.43:0/1957241756 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f393802cdd0 con 0x7f3940160f20 2023-08-09T13:51:20.490 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:20.486+0000 7f393f7fe640 1 -- 172.21.15.43:0/1957241756 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3938034640 con 0x7f3940160f20 2023-08-09T13:51:20.491 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:20.486+0000 7f3946db5640 1 -- 172.21.15.43:0/1957241756 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f3940160f20 msgr2=0x7f39400c3130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:51:20.491 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:20.486+0000 7f3946db5640 1 --2- 172.21.15.43:0/1957241756 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f3940160f20 0x7f39400c3130 secure :-1 s=READY pgs=1191 cs=0 l=1 rev1=1 crypto rx=0x7f3938009d90 tx=0x7f393802bec0 comp rx=0 tx=0).stop 2023-08-09T13:51:20.491 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:20.487+0000 7f3946db5640 1 -- 172.21.15.43:0/1957241756 shutdown_connections 2023-08-09T13:51:20.491 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:20.487+0000 7f3946db5640 1 --2- 172.21.15.43:0/1957241756 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f3940160f20 0x7f39400c3130 unknown :-1 s=CLOSED pgs=1191 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:51:20.491 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:20.487+0000 7f3946db5640 1 -- 172.21.15.43:0/1957241756 >> 172.21.15.43:0/1957241756 conn(0x7f39400ce0a0 msgr2=0x7f39400ce480 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-09T13:51:20.491 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:20.487+0000 7f3946db5640 1 -- 172.21.15.43:0/1957241756 shutdown_connections 2023-08-09T13:51:20.491 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:20.487+0000 7f3946db5640 1 -- 172.21.15.43:0/1957241756 wait complete. 2023-08-09T13:51:20.493 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:20.488+0000 7f3946db5640 1 Processor -- start 2023-08-09T13:51:20.493 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:20.489+0000 7f3946db5640 1 -- start start 2023-08-09T13:51:20.493 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:20.490+0000 7f3946db5640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f39400cc380 0x7f3940070990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:51:20.493 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:20.490+0000 7f3946db5640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f394014a5c0 0x7f39400b31f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:51:20.493 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:20.490+0000 7f3946db5640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f3940160f20 0x7f39400b3730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:51:20.494 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:20.490+0000 7f3946db5640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f394006fef0 con 0x7f3940160f20 2023-08-09T13:51:20.494 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:20.490+0000 7f3946db5640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3940164c90 con 0x7f39400cc380 2023-08-09T13:51:20.494 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:20.490+0000 7f3946db5640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3940070190 con 0x7f394014a5c0 2023-08-09T13:51:20.494 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:20.490+0000 7f3945352640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f3940160f20 0x7f39400b3730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:51:20.494 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:20.490+0000 7f3945352640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f3940160f20 0x7f39400b3730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:42984/0 (socket says 172.21.15.43:42984) 2023-08-09T13:51:20.494 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:20.490+0000 7f3945352640 1 -- 172.21.15.43:0/2886849529 learned_addr learned my addr 172.21.15.43:0/2886849529 (peer_addr_for_me v2:172.21.15.43:0/0) 2023-08-09T13:51:20.494 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:20.490+0000 7f3944b51640 1 --2- 172.21.15.43:0/2886849529 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f39400cc380 0x7f3940070990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:51:20.495 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:20.490+0000 7f393ffff640 1 --2- 172.21.15.43:0/2886849529 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f394014a5c0 0x7f39400b31f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:51:20.495 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:20.490+0000 7f3945352640 1 -- 172.21.15.43:0/2886849529 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f394014a5c0 msgr2=0x7f39400b31f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:51:20.495 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:20.490+0000 7f3945352640 1 --2- 172.21.15.43:0/2886849529 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f394014a5c0 0x7f39400b31f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:51:20.495 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:20.491+0000 7f3945352640 1 -- 172.21.15.43:0/2886849529 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f39400cc380 msgr2=0x7f3940070990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:51:20.495 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:20.491+0000 7f3945352640 1 --2- 172.21.15.43:0/2886849529 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f39400cc380 0x7f3940070990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:51:20.495 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:20.491+0000 7f3945352640 1 -- 172.21.15.43:0/2886849529 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3938009a70 con 0x7f3940160f20 2023-08-09T13:51:20.496 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:20.491+0000 7f3944b51640 1 --2- 172.21.15.43:0/2886849529 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f39400cc380 0x7f3940070990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-08-09T13:51:20.496 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:20.491+0000 7f3945352640 1 --2- 172.21.15.43:0/2886849529 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f3940160f20 0x7f39400b3730 secure :-1 s=READY pgs=1192 cs=0 l=1 rev1=1 crypto rx=0x7f3938009d90 tx=0x7f3938004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-09T13:51:20.497 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:20.491+0000 7f393d7fa640 1 -- 172.21.15.43:0/2886849529 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3938003b90 con 0x7f3940160f20 2023-08-09T13:51:20.497 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:20.491+0000 7f393d7fa640 1 -- 172.21.15.43:0/2886849529 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3938003d10 con 0x7f3940160f20 2023-08-09T13:51:20.497 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:20.491+0000 7f393d7fa640 1 -- 172.21.15.43:0/2886849529 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3938004270 con 0x7f3940160f20 2023-08-09T13:51:20.497 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:20.491+0000 7f3946db5640 1 -- 172.21.15.43:0/2886849529 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3940070470 con 0x7f3940160f20 2023-08-09T13:51:20.497 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:20.492+0000 7f3946db5640 1 -- 172.21.15.43:0/2886849529 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f39401601a0 con 0x7f3940160f20 2023-08-09T13:51:20.497 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:20.493+0000 7f3946db5640 1 -- 172.21.15.43:0/2886849529 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3940070ed0 con 0x7f3940160f20 2023-08-09T13:51:20.498 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:20.494+0000 7f393d7fa640 1 -- 172.21.15.43:0/2886849529 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 37) v1 ==== 88443+0+0 (secure 0 0 0) 0x7f393803b050 con 0x7f3940160f20 2023-08-09T13:51:20.498 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:20.495+0000 7f393d7fa640 1 --2- 172.21.15.43:0/2886849529 >> [v2:172.21.15.203:6832/1778614695,v1:172.21.15.203:6833/1778614695] conn(0x7f392406a170 0x7f392406c630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:51:20.499 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:20.495+0000 7f393d7fa640 1 -- 172.21.15.43:0/2886849529 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(103..103 src has 1..103) v4 ==== 6426+0+0 (secure 0 0 0) 0x7f39380aacb0 con 0x7f3940160f20 2023-08-09T13:51:20.499 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:20.495+0000 7f3944b51640 1 --2- 172.21.15.43:0/2886849529 >> [v2:172.21.15.203:6832/1778614695,v1:172.21.15.203:6833/1778614695] conn(0x7f392406a170 0x7f392406c630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:51:20.499 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:20.496+0000 7f3944b51640 1 --2- 172.21.15.43:0/2886849529 >> [v2:172.21.15.203:6832/1778614695,v1:172.21.15.203:6833/1778614695] conn(0x7f392406a170 0x7f392406c630 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f39340011e0 tx=0x7f3934009040 comp rx=0 tx=0).ready entity=mgr.5819 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-09T13:51:20.501 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:20.499+0000 7f393d7fa640 1 -- 172.21.15.43:0/2886849529 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178066 (secure 0 0 0) 0x7f3938076120 con 0x7f3940160f20 2023-08-09T13:51:20.692 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:20.689+0000 7f3946db5640 1 -- 172.21.15.43:0/2886849529 --> [v2:172.21.15.203:6832/1778614695,v1:172.21.15.203:6833/1778614695] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f39400c4140 con 0x7f392406a170 2023-08-09T13:51:20.722 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:20.719+0000 7f393d7fa640 1 -- 172.21.15.43:0/2886849529 <== mgr.5819 v2:172.21.15.203:6832/1778614695 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+309486 (secure 0 0 0) 0x7f39340013f0 con 0x7f392406a170 2023-08-09T13:51:20.726 INFO:teuthology.orchestra.run.smithi043.stdout: 2023-08-09T13:51:20.727 INFO:teuthology.orchestra.run.smithi043.stderr:dumped all 2023-08-09T13:51:20.728 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:20.725+0000 7f3946db5640 1 -- 172.21.15.43:0/2886849529 >> [v2:172.21.15.203:6832/1778614695,v1:172.21.15.203:6833/1778614695] conn(0x7f392406a170 msgr2=0x7f392406c630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:51:20.729 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:20.725+0000 7f3946db5640 1 --2- 172.21.15.43:0/2886849529 >> [v2:172.21.15.203:6832/1778614695,v1:172.21.15.203:6833/1778614695] conn(0x7f392406a170 0x7f392406c630 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f39340011e0 tx=0x7f3934009040 comp rx=0 tx=0).stop 2023-08-09T13:51:20.729 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:20.725+0000 7f3946db5640 1 -- 172.21.15.43:0/2886849529 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f3940160f20 msgr2=0x7f39400b3730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:51:20.729 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:20.725+0000 7f3946db5640 1 --2- 172.21.15.43:0/2886849529 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f3940160f20 0x7f39400b3730 secure :-1 s=READY pgs=1192 cs=0 l=1 rev1=1 crypto rx=0x7f3938009d90 tx=0x7f3938004a40 comp rx=0 tx=0).stop 2023-08-09T13:51:20.729 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:20.725+0000 7f3946db5640 1 -- 172.21.15.43:0/2886849529 shutdown_connections 2023-08-09T13:51:20.729 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:20.725+0000 7f3946db5640 1 --2- 172.21.15.43:0/2886849529 >> [v2:172.21.15.203:6832/1778614695,v1:172.21.15.203:6833/1778614695] conn(0x7f392406a170 0x7f392406c630 unknown :-1 s=CLOSED pgs=64 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:51:20.729 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:20.725+0000 7f3946db5640 1 --2- 172.21.15.43:0/2886849529 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f3940160f20 0x7f39400b3730 unknown :-1 s=CLOSED pgs=1192 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:51:20.730 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:20.725+0000 7f3946db5640 1 --2- 172.21.15.43:0/2886849529 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f394014a5c0 0x7f39400b31f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:51:20.730 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:20.725+0000 7f3946db5640 1 --2- 172.21.15.43:0/2886849529 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f39400cc380 0x7f3940070990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:51:20.730 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:20.725+0000 7f3946db5640 1 -- 172.21.15.43:0/2886849529 >> 172.21.15.43:0/2886849529 conn(0x7f39400ce0a0 msgr2=0x7f39400adc20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-09T13:51:20.730 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:20.725+0000 7f3946db5640 1 -- 172.21.15.43:0/2886849529 shutdown_connections 2023-08-09T13:51:20.730 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:20.725+0000 7f3946db5640 1 -- 172.21.15.43:0/2886849529 wait complete. 2023-08-09T13:51:20.747 INFO:teuthology.orchestra.run.smithi043.stdout:{"pg_ready":true,"pg_map":{"version":26,"stamp":"2023-08-09T13:51:19.433961+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":23,"num_object_clones":0,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":0,"num_read":619,"num_read_kb":1212,"num_write":307,"num_write_kb":2430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":213,"ondisk_log_size":213,"up":210,"acting":210,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":69,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":306520,"kb_used_data":10844,"kb_used_omap":13,"kb_used_meta":295538,"kb_avail":754668200,"statfs":{"total":773094113280,"available":772780236800,"internally_reserved":0,"allocated":11104256,"data_stored":6456252,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":14248,"internal_metadata":302631000},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":32,"apply_latency_ms":32,"commit_latency_ns":32000000,"apply_latency_ns":32000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"5.245225"},"pg_stats":[{"pgid":"13.3f","version":"0'0","reported_seq":9,"reported_epoch":102,"state":"active+clean","last_fresh":"2023-08-09T13:51:16.258593+0000","last_change":"2023-08-09T13:51:16.248603+0000","last_active":"2023-08-09T13:51:16.258593+0000","last_peered":"2023-08-09T13:51:16.258593+0000","last_clean":"2023-08-09T13:51:16.258593+0000","last_became_active":"2023-08-09T13:51:16.248037+0000","last_became_peered":"2023-08-09T13:51:16.248037+0000","last_unstale":"2023-08-09T13:51:16.258593+0000","last_undegraded":"2023-08-09T13:51:16.258593+0000","last_fullsized":"2023-08-09T13:51:16.258593+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:51:15.188798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:51:15.188798+0000","last_clean_scrub_stamp":"2023-08-09T13:51:15.188798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T16:48:12.509006+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.3e","version":"0'0","reported_seq":9,"reported_epoch":102,"state":"active+clean","last_fresh":"2023-08-09T13:51:16.262721+0000","last_change":"2023-08-09T13:51:16.257259+0000","last_active":"2023-08-09T13:51:16.262721+0000","last_peered":"2023-08-09T13:51:16.262721+0000","last_clean":"2023-08-09T13:51:16.262721+0000","last_became_active":"2023-08-09T13:51:16.256916+0000","last_became_peered":"2023-08-09T13:51:16.256916+0000","last_unstale":"2023-08-09T13:51:16.262721+0000","last_undegraded":"2023-08-09T13:51:16.262721+0000","last_fullsized":"2023-08-09T13:51:16.262721+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:51:15.188798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:51:15.188798+0000","last_clean_scrub_stamp":"2023-08-09T13:51:15.188798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T16:59:10.173876+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.3d","version":"0'0","reported_seq":9,"reported_epoch":102,"state":"active+clean","last_fresh":"2023-08-09T13:51:16.258432+0000","last_change":"2023-08-09T13:51:16.257154+0000","last_active":"2023-08-09T13:51:16.258432+0000","last_peered":"2023-08-09T13:51:16.258432+0000","last_clean":"2023-08-09T13:51:16.258432+0000","last_became_active":"2023-08-09T13:51:16.256519+0000","last_became_peered":"2023-08-09T13:51:16.256519+0000","last_unstale":"2023-08-09T13:51:16.258432+0000","last_undegraded":"2023-08-09T13:51:16.258432+0000","last_fullsized":"2023-08-09T13:51:16.258432+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:51:15.188798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:51:15.188798+0000","last_clean_scrub_stamp":"2023-08-09T13:51:15.188798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T22:23:55.883603+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"13.3c","version":"0'0","reported_seq":9,"reported_epoch":102,"state":"active+clean","last_fresh":"2023-08-09T13:51:16.260995+0000","last_change":"2023-08-09T13:51:16.257966+0000","last_active":"2023-08-09T13:51:16.260995+0000","last_peered":"2023-08-09T13:51:16.260995+0000","last_clean":"2023-08-09T13:51:16.260995+0000","last_became_active":"2023-08-09T13:51:16.257491+0000","last_became_peered":"2023-08-09T13:51:16.257491+0000","last_unstale":"2023-08-09T13:51:16.260995+0000","last_undegraded":"2023-08-09T13:51:16.260995+0000","last_fullsized":"2023-08-09T13:51:16.260995+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:51:15.188798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:51:15.188798+0000","last_clean_scrub_stamp":"2023-08-09T13:51:15.188798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T15:13:09.199403+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.3b","version":"0'0","reported_seq":9,"reported_epoch":102,"state":"active+clean","last_fresh":"2023-08-09T13:51:16.255556+0000","last_change":"2023-08-09T13:51:16.251679+0000","last_active":"2023-08-09T13:51:16.255556+0000","last_peered":"2023-08-09T13:51:16.255556+0000","last_clean":"2023-08-09T13:51:16.255556+0000","last_became_active":"2023-08-09T13:51:16.251416+0000","last_became_peered":"2023-08-09T13:51:16.251416+0000","last_unstale":"2023-08-09T13:51:16.255556+0000","last_undegraded":"2023-08-09T13:51:16.255556+0000","last_fullsized":"2023-08-09T13:51:16.255556+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:51:15.188798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:51:15.188798+0000","last_clean_scrub_stamp":"2023-08-09T13:51:15.188798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T15:52:45.214559+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.3a","version":"0'0","reported_seq":9,"reported_epoch":102,"state":"active+clean","last_fresh":"2023-08-09T13:51:16.253888+0000","last_change":"2023-08-09T13:51:16.247397+0000","last_active":"2023-08-09T13:51:16.253888+0000","last_peered":"2023-08-09T13:51:16.253888+0000","last_clean":"2023-08-09T13:51:16.253888+0000","last_became_active":"2023-08-09T13:51:16.247132+0000","last_became_peered":"2023-08-09T13:51:16.247132+0000","last_unstale":"2023-08-09T13:51:16.253888+0000","last_undegraded":"2023-08-09T13:51:16.253888+0000","last_fullsized":"2023-08-09T13:51:16.253888+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:51:15.188798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:51:15.188798+0000","last_clean_scrub_stamp":"2023-08-09T13:51:15.188798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T17:34:10.406118+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"13.39","version":"0'0","reported_seq":9,"reported_epoch":102,"state":"active+clean","last_fresh":"2023-08-09T13:51:16.252061+0000","last_change":"2023-08-09T13:51:16.245360+0000","last_active":"2023-08-09T13:51:16.252061+0000","last_peered":"2023-08-09T13:51:16.252061+0000","last_clean":"2023-08-09T13:51:16.252061+0000","last_became_active":"2023-08-09T13:51:16.245025+0000","last_became_peered":"2023-08-09T13:51:16.245025+0000","last_unstale":"2023-08-09T13:51:16.252061+0000","last_undegraded":"2023-08-09T13:51:16.252061+0000","last_fullsized":"2023-08-09T13:51:16.252061+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:51:15.188798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:51:15.188798+0000","last_clean_scrub_stamp":"2023-08-09T13:51:15.188798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T16:44:53.436136+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.38","version":"0'0","reported_seq":9,"reported_epoch":102,"state":"active+clean","last_fresh":"2023-08-09T13:51:16.254444+0000","last_change":"2023-08-09T13:51:16.253769+0000","last_active":"2023-08-09T13:51:16.254444+0000","last_peered":"2023-08-09T13:51:16.254444+0000","last_clean":"2023-08-09T13:51:16.254444+0000","last_became_active":"2023-08-09T13:51:16.253464+0000","last_became_peered":"2023-08-09T13:51:16.253464+0000","last_unstale":"2023-08-09T13:51:16.254444+0000","last_undegraded":"2023-08-09T13:51:16.254444+0000","last_fullsized":"2023-08-09T13:51:16.254444+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:51:15.188798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:51:15.188798+0000","last_clean_scrub_stamp":"2023-08-09T13:51:15.188798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T18:17:41.906539+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"13.37","version":"0'0","reported_seq":9,"reported_epoch":102,"state":"active+clean","last_fresh":"2023-08-09T13:51:16.235673+0000","last_change":"2023-08-09T13:51:16.233810+0000","last_active":"2023-08-09T13:51:16.235673+0000","last_peered":"2023-08-09T13:51:16.235673+0000","last_clean":"2023-08-09T13:51:16.235673+0000","last_became_active":"2023-08-09T13:51:16.232392+0000","last_became_peered":"2023-08-09T13:51:16.232392+0000","last_unstale":"2023-08-09T13:51:16.235673+0000","last_undegraded":"2023-08-09T13:51:16.235673+0000","last_fullsized":"2023-08-09T13:51:16.235673+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:51:15.188798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:51:15.188798+0000","last_clean_scrub_stamp":"2023-08-09T13:51:15.188798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T18:35:59.659869+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"13.36","version":"0'0","reported_seq":9,"reported_epoch":102,"state":"active+clean","last_fresh":"2023-08-09T13:51:16.259336+0000","last_change":"2023-08-09T13:51:16.253575+0000","last_active":"2023-08-09T13:51:16.259336+0000","last_peered":"2023-08-09T13:51:16.259336+0000","last_clean":"2023-08-09T13:51:16.259336+0000","last_became_active":"2023-08-09T13:51:16.253189+0000","last_became_peered":"2023-08-09T13:51:16.253189+0000","last_unstale":"2023-08-09T13:51:16.259336+0000","last_undegraded":"2023-08-09T13:51:16.259336+0000","last_fullsized":"2023-08-09T13:51:16.259336+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:51:15.188798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:51:15.188798+0000","last_clean_scrub_stamp":"2023-08-09T13:51:15.188798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T22:29:06.334182+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.35","version":"0'0","reported_seq":9,"reported_epoch":102,"state":"active+clean","last_fresh":"2023-08-09T13:51:16.262632+0000","last_change":"2023-08-09T13:51:16.256458+0000","last_active":"2023-08-09T13:51:16.262632+0000","last_peered":"2023-08-09T13:51:16.262632+0000","last_clean":"2023-08-09T13:51:16.262632+0000","last_became_active":"2023-08-09T13:51:16.256188+0000","last_became_peered":"2023-08-09T13:51:16.256188+0000","last_unstale":"2023-08-09T13:51:16.262632+0000","last_undegraded":"2023-08-09T13:51:16.262632+0000","last_fullsized":"2023-08-09T13:51:16.262632+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:51:15.188798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:51:15.188798+0000","last_clean_scrub_stamp":"2023-08-09T13:51:15.188798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T22:52:37.102129+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.34","version":"0'0","reported_seq":9,"reported_epoch":102,"state":"active+clean","last_fresh":"2023-08-09T13:51:16.259450+0000","last_change":"2023-08-09T13:51:16.254964+0000","last_active":"2023-08-09T13:51:16.259450+0000","last_peered":"2023-08-09T13:51:16.259450+0000","last_clean":"2023-08-09T13:51:16.259450+0000","last_became_active":"2023-08-09T13:51:16.254499+0000","last_became_peered":"2023-08-09T13:51:16.254499+0000","last_unstale":"2023-08-09T13:51:16.259450+0000","last_undegraded":"2023-08-09T13:51:16.259450+0000","last_fullsized":"2023-08-09T13:51:16.259450+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:51:15.188798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:51:15.188798+0000","last_clean_scrub_stamp":"2023-08-09T13:51:15.188798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T23:45:01.441221+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.33","version":"0'0","reported_seq":9,"reported_epoch":102,"state":"active+clean","last_fresh":"2023-08-09T13:51:16.243344+0000","last_change":"2023-08-09T13:51:16.240086+0000","last_active":"2023-08-09T13:51:16.243344+0000","last_peered":"2023-08-09T13:51:16.243344+0000","last_clean":"2023-08-09T13:51:16.243344+0000","last_became_active":"2023-08-09T13:51:16.239792+0000","last_became_peered":"2023-08-09T13:51:16.239792+0000","last_unstale":"2023-08-09T13:51:16.243344+0000","last_undegraded":"2023-08-09T13:51:16.243344+0000","last_fullsized":"2023-08-09T13:51:16.243344+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:51:15.188798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:51:15.188798+0000","last_clean_scrub_stamp":"2023-08-09T13:51:15.188798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T15:20:43.751333+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"13.32","version":"0'0","reported_seq":9,"reported_epoch":102,"state":"active+clean","last_fresh":"2023-08-09T13:51:16.257888+0000","last_change":"2023-08-09T13:51:16.253437+0000","last_active":"2023-08-09T13:51:16.257888+0000","last_peered":"2023-08-09T13:51:16.257888+0000","last_clean":"2023-08-09T13:51:16.257888+0000","last_became_active":"2023-08-09T13:51:16.253106+0000","last_became_peered":"2023-08-09T13:51:16.253106+0000","last_unstale":"2023-08-09T13:51:16.257888+0000","last_undegraded":"2023-08-09T13:51:16.257888+0000","last_fullsized":"2023-08-09T13:51:16.257888+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:51:15.188798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:51:15.188798+0000","last_clean_scrub_stamp":"2023-08-09T13:51:15.188798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T15:24:24.668964+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.31","version":"0'0","reported_seq":9,"reported_epoch":102,"state":"active+clean","last_fresh":"2023-08-09T13:51:16.253948+0000","last_change":"2023-08-09T13:51:16.247068+0000","last_active":"2023-08-09T13:51:16.253948+0000","last_peered":"2023-08-09T13:51:16.253948+0000","last_clean":"2023-08-09T13:51:16.253948+0000","last_became_active":"2023-08-09T13:51:16.246689+0000","last_became_peered":"2023-08-09T13:51:16.246689+0000","last_unstale":"2023-08-09T13:51:16.253948+0000","last_undegraded":"2023-08-09T13:51:16.253948+0000","last_fullsized":"2023-08-09T13:51:16.253948+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:51:15.188798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:51:15.188798+0000","last_clean_scrub_stamp":"2023-08-09T13:51:15.188798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-11T00:34:32.632449+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"13.30","version":"0'0","reported_seq":9,"reported_epoch":102,"state":"active+clean","last_fresh":"2023-08-09T13:51:16.255493+0000","last_change":"2023-08-09T13:51:16.250688+0000","last_active":"2023-08-09T13:51:16.255493+0000","last_peered":"2023-08-09T13:51:16.255493+0000","last_clean":"2023-08-09T13:51:16.255493+0000","last_became_active":"2023-08-09T13:51:16.250402+0000","last_became_peered":"2023-08-09T13:51:16.250402+0000","last_unstale":"2023-08-09T13:51:16.255493+0000","last_undegraded":"2023-08-09T13:51:16.255493+0000","last_fullsized":"2023-08-09T13:51:16.255493+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:51:15.188798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:51:15.188798+0000","last_clean_scrub_stamp":"2023-08-09T13:51:15.188798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T19:40:21.994081+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.2f","version":"0'0","reported_seq":9,"reported_epoch":102,"state":"active+clean","last_fresh":"2023-08-09T13:51:16.251253+0000","last_change":"2023-08-09T13:51:16.244707+0000","last_active":"2023-08-09T13:51:16.251253+0000","last_peered":"2023-08-09T13:51:16.251253+0000","last_clean":"2023-08-09T13:51:16.251253+0000","last_became_active":"2023-08-09T13:51:16.244438+0000","last_became_peered":"2023-08-09T13:51:16.244438+0000","last_unstale":"2023-08-09T13:51:16.251253+0000","last_undegraded":"2023-08-09T13:51:16.251253+0000","last_fullsized":"2023-08-09T13:51:16.251253+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:51:15.188798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:51:15.188798+0000","last_clean_scrub_stamp":"2023-08-09T13:51:15.188798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T15:37:01.831464+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.2e","version":"0'0","reported_seq":9,"reported_epoch":102,"state":"active+clean","last_fresh":"2023-08-09T13:51:16.258072+0000","last_change":"2023-08-09T13:51:16.254207+0000","last_active":"2023-08-09T13:51:16.258072+0000","last_peered":"2023-08-09T13:51:16.258072+0000","last_clean":"2023-08-09T13:51:16.258072+0000","last_became_active":"2023-08-09T13:51:16.253931+0000","last_became_peered":"2023-08-09T13:51:16.253931+0000","last_unstale":"2023-08-09T13:51:16.258072+0000","last_undegraded":"2023-08-09T13:51:16.258072+0000","last_fullsized":"2023-08-09T13:51:16.258072+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:51:15.188798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:51:15.188798+0000","last_clean_scrub_stamp":"2023-08-09T13:51:15.188798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T23:00:52.771901+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.2d","version":"0'0","reported_seq":9,"reported_epoch":102,"state":"active+clean","last_fresh":"2023-08-09T13:51:16.254573+0000","last_change":"2023-08-09T13:51:16.246483+0000","last_active":"2023-08-09T13:51:16.254573+0000","last_peered":"2023-08-09T13:51:16.254573+0000","last_clean":"2023-08-09T13:51:16.254573+0000","last_became_active":"2023-08-09T13:51:16.246199+0000","last_became_peered":"2023-08-09T13:51:16.246199+0000","last_unstale":"2023-08-09T13:51:16.254573+0000","last_undegraded":"2023-08-09T13:51:16.254573+0000","last_fullsized":"2023-08-09T13:51:16.254573+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:51:15.188798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:51:15.188798+0000","last_clean_scrub_stamp":"2023-08-09T13:51:15.188798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T15:52:45.214559+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.2c","version":"0'0","reported_seq":9,"reported_epoch":102,"state":"active+clean","last_fresh":"2023-08-09T13:51:16.250906+0000","last_change":"2023-08-09T13:51:16.241540+0000","last_active":"2023-08-09T13:51:16.250906+0000","last_peered":"2023-08-09T13:51:16.250906+0000","last_clean":"2023-08-09T13:51:16.250906+0000","last_became_active":"2023-08-09T13:51:16.241056+0000","last_became_peered":"2023-08-09T13:51:16.241056+0000","last_unstale":"2023-08-09T13:51:16.250906+0000","last_undegraded":"2023-08-09T13:51:16.250906+0000","last_fullsized":"2023-08-09T13:51:16.250906+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:51:15.188798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:51:15.188798+0000","last_clean_scrub_stamp":"2023-08-09T13:51:15.188798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T14:26:39.561465+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.2b","version":"0'0","reported_seq":9,"reported_epoch":102,"state":"active+clean","last_fresh":"2023-08-09T13:51:16.250700+0000","last_change":"2023-08-09T13:51:16.241383+0000","last_active":"2023-08-09T13:51:16.250700+0000","last_peered":"2023-08-09T13:51:16.250700+0000","last_clean":"2023-08-09T13:51:16.250700+0000","last_became_active":"2023-08-09T13:51:16.240841+0000","last_became_peered":"2023-08-09T13:51:16.240841+0000","last_unstale":"2023-08-09T13:51:16.250700+0000","last_undegraded":"2023-08-09T13:51:16.250700+0000","last_fullsized":"2023-08-09T13:51:16.250700+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:51:15.188798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:51:15.188798+0000","last_clean_scrub_stamp":"2023-08-09T13:51:15.188798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-11T01:16:57.226514+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.2a","version":"0'0","reported_seq":9,"reported_epoch":102,"state":"active+clean","last_fresh":"2023-08-09T13:51:16.243532+0000","last_change":"2023-08-09T13:51:16.227586+0000","last_active":"2023-08-09T13:51:16.243532+0000","last_peered":"2023-08-09T13:51:16.243532+0000","last_clean":"2023-08-09T13:51:16.243532+0000","last_became_active":"2023-08-09T13:51:16.227304+0000","last_became_peered":"2023-08-09T13:51:16.227304+0000","last_unstale":"2023-08-09T13:51:16.243532+0000","last_undegraded":"2023-08-09T13:51:16.243532+0000","last_fullsized":"2023-08-09T13:51:16.243532+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:51:15.188798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:51:15.188798+0000","last_clean_scrub_stamp":"2023-08-09T13:51:15.188798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T16:18:24.664649+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.29","version":"0'0","reported_seq":9,"reported_epoch":102,"state":"active+clean","last_fresh":"2023-08-09T13:51:16.259341+0000","last_change":"2023-08-09T13:51:16.253681+0000","last_active":"2023-08-09T13:51:16.259341+0000","last_peered":"2023-08-09T13:51:16.259341+0000","last_clean":"2023-08-09T13:51:16.259341+0000","last_became_active":"2023-08-09T13:51:16.253385+0000","last_became_peered":"2023-08-09T13:51:16.253385+0000","last_unstale":"2023-08-09T13:51:16.259341+0000","last_undegraded":"2023-08-09T13:51:16.259341+0000","last_fullsized":"2023-08-09T13:51:16.259341+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:51:15.188798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:51:15.188798+0000","last_clean_scrub_stamp":"2023-08-09T13:51:15.188798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-11T00:47:25.117819+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.28","version":"0'0","reported_seq":9,"reported_epoch":102,"state":"active+clean","last_fresh":"2023-08-09T13:51:16.250957+0000","last_change":"2023-08-09T13:51:16.242027+0000","last_active":"2023-08-09T13:51:16.250957+0000","last_peered":"2023-08-09T13:51:16.250957+0000","last_clean":"2023-08-09T13:51:16.250957+0000","last_became_active":"2023-08-09T13:51:16.241728+0000","last_became_peered":"2023-08-09T13:51:16.241728+0000","last_unstale":"2023-08-09T13:51:16.250957+0000","last_undegraded":"2023-08-09T13:51:16.250957+0000","last_fullsized":"2023-08-09T13:51:16.250957+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:51:15.188798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:51:15.188798+0000","last_clean_scrub_stamp":"2023-08-09T13:51:15.188798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T16:32:37.597359+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.27","version":"0'0","reported_seq":9,"reported_epoch":102,"state":"active+clean","last_fresh":"2023-08-09T13:51:16.261482+0000","last_change":"2023-08-09T13:51:16.251716+0000","last_active":"2023-08-09T13:51:16.261482+0000","last_peered":"2023-08-09T13:51:16.261482+0000","last_clean":"2023-08-09T13:51:16.261482+0000","last_became_active":"2023-08-09T13:51:16.251298+0000","last_became_peered":"2023-08-09T13:51:16.251298+0000","last_unstale":"2023-08-09T13:51:16.261482+0000","last_undegraded":"2023-08-09T13:51:16.261482+0000","last_fullsized":"2023-08-09T13:51:16.261482+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:51:15.188798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:51:15.188798+0000","last_clean_scrub_stamp":"2023-08-09T13:51:15.188798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-11T01:28:14.454002+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.26","version":"0'0","reported_seq":9,"reported_epoch":102,"state":"active+clean","last_fresh":"2023-08-09T13:51:16.257487+0000","last_change":"2023-08-09T13:51:16.252554+0000","last_active":"2023-08-09T13:51:16.257487+0000","last_peered":"2023-08-09T13:51:16.257487+0000","last_clean":"2023-08-09T13:51:16.257487+0000","last_became_active":"2023-08-09T13:51:16.252241+0000","last_became_peered":"2023-08-09T13:51:16.252241+0000","last_unstale":"2023-08-09T13:51:16.257487+0000","last_undegraded":"2023-08-09T13:51:16.257487+0000","last_fullsized":"2023-08-09T13:51:16.257487+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:51:15.188798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:51:15.188798+0000","last_clean_scrub_stamp":"2023-08-09T13:51:15.188798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T20:43:45.377919+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"13.25","version":"0'0","reported_seq":9,"reported_epoch":102,"state":"active+clean","last_fresh":"2023-08-09T13:51:16.255617+0000","last_change":"2023-08-09T13:51:16.247370+0000","last_active":"2023-08-09T13:51:16.255617+0000","last_peered":"2023-08-09T13:51:16.255617+0000","last_clean":"2023-08-09T13:51:16.255617+0000","last_became_active":"2023-08-09T13:51:16.247041+0000","last_became_peered":"2023-08-09T13:51:16.247041+0000","last_unstale":"2023-08-09T13:51:16.255617+0000","last_undegraded":"2023-08-09T13:51:16.255617+0000","last_fullsized":"2023-08-09T13:51:16.255617+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:51:15.188798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:51:15.188798+0000","last_clean_scrub_stamp":"2023-08-09T13:51:15.188798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T21:23:46.771956+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.24","version":"0'0","reported_seq":9,"reported_epoch":102,"state":"active+clean","last_fresh":"2023-08-09T13:51:16.259470+0000","last_change":"2023-08-09T13:51:16.255148+0000","last_active":"2023-08-09T13:51:16.259470+0000","last_peered":"2023-08-09T13:51:16.259470+0000","last_clean":"2023-08-09T13:51:16.259470+0000","last_became_active":"2023-08-09T13:51:16.254681+0000","last_became_peered":"2023-08-09T13:51:16.254681+0000","last_unstale":"2023-08-09T13:51:16.259470+0000","last_undegraded":"2023-08-09T13:51:16.259470+0000","last_fullsized":"2023-08-09T13:51:16.259470+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:51:15.188798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:51:15.188798+0000","last_clean_scrub_stamp":"2023-08-09T13:51:15.188798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T21:12:55.275167+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.23","version":"0'0","reported_seq":9,"reported_epoch":102,"state":"active+clean","last_fresh":"2023-08-09T13:51:16.262319+0000","last_change":"2023-08-09T13:51:16.255251+0000","last_active":"2023-08-09T13:51:16.262319+0000","last_peered":"2023-08-09T13:51:16.262319+0000","last_clean":"2023-08-09T13:51:16.262319+0000","last_became_active":"2023-08-09T13:51:16.254974+0000","last_became_peered":"2023-08-09T13:51:16.254974+0000","last_unstale":"2023-08-09T13:51:16.262319+0000","last_undegraded":"2023-08-09T13:51:16.262319+0000","last_fullsized":"2023-08-09T13:51:16.262319+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:51:15.188798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:51:15.188798+0000","last_clean_scrub_stamp":"2023-08-09T13:51:15.188798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T18:10:20.996377+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.22","version":"0'0","reported_seq":9,"reported_epoch":102,"state":"active+clean","last_fresh":"2023-08-09T13:51:16.250243+0000","last_change":"2023-08-09T13:51:16.243113+0000","last_active":"2023-08-09T13:51:16.250243+0000","last_peered":"2023-08-09T13:51:16.250243+0000","last_clean":"2023-08-09T13:51:16.250243+0000","last_became_active":"2023-08-09T13:51:16.242518+0000","last_became_peered":"2023-08-09T13:51:16.242518+0000","last_unstale":"2023-08-09T13:51:16.250243+0000","last_undegraded":"2023-08-09T13:51:16.250243+0000","last_fullsized":"2023-08-09T13:51:16.250243+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:51:15.188798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:51:15.188798+0000","last_clean_scrub_stamp":"2023-08-09T13:51:15.188798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T16:37:00.528301+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.21","version":"0'0","reported_seq":9,"reported_epoch":102,"state":"active+clean","last_fresh":"2023-08-09T13:51:16.245497+0000","last_change":"2023-08-09T13:51:16.240818+0000","last_active":"2023-08-09T13:51:16.245497+0000","last_peered":"2023-08-09T13:51:16.245497+0000","last_clean":"2023-08-09T13:51:16.245497+0000","last_became_active":"2023-08-09T13:51:16.240492+0000","last_became_peered":"2023-08-09T13:51:16.240492+0000","last_unstale":"2023-08-09T13:51:16.245497+0000","last_undegraded":"2023-08-09T13:51:16.245497+0000","last_fullsized":"2023-08-09T13:51:16.245497+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:51:15.188798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:51:15.188798+0000","last_clean_scrub_stamp":"2023-08-09T13:51:15.188798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T19:11:07.709369+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.20","version":"0'0","reported_seq":9,"reported_epoch":102,"state":"active+clean","last_fresh":"2023-08-09T13:51:16.258316+0000","last_change":"2023-08-09T13:51:16.251983+0000","last_active":"2023-08-09T13:51:16.258316+0000","last_peered":"2023-08-09T13:51:16.258316+0000","last_clean":"2023-08-09T13:51:16.258316+0000","last_became_active":"2023-08-09T13:51:16.251594+0000","last_became_peered":"2023-08-09T13:51:16.251594+0000","last_unstale":"2023-08-09T13:51:16.258316+0000","last_undegraded":"2023-08-09T13:51:16.258316+0000","last_fullsized":"2023-08-09T13:51:16.258316+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:51:15.188798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:51:15.188798+0000","last_clean_scrub_stamp":"2023-08-09T13:51:15.188798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T23:12:07.933015+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.1e","version":"0'0","reported_seq":9,"reported_epoch":102,"state":"active+clean","last_fresh":"2023-08-09T13:51:16.257697+0000","last_change":"2023-08-09T13:51:16.254978+0000","last_active":"2023-08-09T13:51:16.257697+0000","last_peered":"2023-08-09T13:51:16.257697+0000","last_clean":"2023-08-09T13:51:16.257697+0000","last_became_active":"2023-08-09T13:51:16.254646+0000","last_became_peered":"2023-08-09T13:51:16.254646+0000","last_unstale":"2023-08-09T13:51:16.257697+0000","last_undegraded":"2023-08-09T13:51:16.257697+0000","last_fullsized":"2023-08-09T13:51:16.257697+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:51:15.188798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:51:15.188798+0000","last_clean_scrub_stamp":"2023-08-09T13:51:15.188798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T19:37:14.081662+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"12.1f","version":"102'4","reported_seq":15,"reported_epoch":102,"state":"active+clean","last_fresh":"2023-08-09T13:51:16.258227+0000","last_change":"2023-08-09T13:51:15.242826+0000","last_active":"2023-08-09T13:51:16.258227+0000","last_peered":"2023-08-09T13:51:16.258227+0000","last_clean":"2023-08-09T13:51:16.258227+0000","last_became_active":"2023-08-09T13:51:15.242507+0000","last_became_peered":"2023-08-09T13:51:15.242507+0000","last_unstale":"2023-08-09T13:51:16.258227+0000","last_undegraded":"2023-08-09T13:51:16.258227+0000","last_fullsized":"2023-08-09T13:51:16.258227+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:51:14.171648+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:51:14.171648+0000","last_clean_scrub_stamp":"2023-08-09T13:51:14.171648+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T13:52:18.768602+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.1f","version":"0'0","reported_seq":9,"reported_epoch":102,"state":"active+clean","last_fresh":"2023-08-09T13:51:16.259855+0000","last_change":"2023-08-09T13:51:16.258195+0000","last_active":"2023-08-09T13:51:16.259855+0000","last_peered":"2023-08-09T13:51:16.259855+0000","last_clean":"2023-08-09T13:51:16.259855+0000","last_became_active":"2023-08-09T13:51:16.257074+0000","last_became_peered":"2023-08-09T13:51:16.257074+0000","last_unstale":"2023-08-09T13:51:16.259855+0000","last_undegraded":"2023-08-09T13:51:16.259855+0000","last_fullsized":"2023-08-09T13:51:16.259855+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:51:15.188798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:51:15.188798+0000","last_clean_scrub_stamp":"2023-08-09T13:51:15.188798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T22:28:58.004817+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.1e","version":"102'1","reported_seq":12,"reported_epoch":102,"state":"active+clean","last_fresh":"2023-08-09T13:51:16.250829+0000","last_change":"2023-08-09T13:51:15.243566+0000","last_active":"2023-08-09T13:51:16.250829+0000","last_peered":"2023-08-09T13:51:16.250829+0000","last_clean":"2023-08-09T13:51:16.250829+0000","last_became_active":"2023-08-09T13:51:15.242859+0000","last_became_peered":"2023-08-09T13:51:15.242859+0000","last_unstale":"2023-08-09T13:51:16.250829+0000","last_undegraded":"2023-08-09T13:51:16.250829+0000","last_fullsized":"2023-08-09T13:51:16.250829+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:51:14.171648+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:51:14.171648+0000","last_clean_scrub_stamp":"2023-08-09T13:51:14.171648+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T15:31:24.677331+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.1c","version":"0'0","reported_seq":9,"reported_epoch":102,"state":"active+clean","last_fresh":"2023-08-09T13:51:16.262914+0000","last_change":"2023-08-09T13:51:16.262328+0000","last_active":"2023-08-09T13:51:16.262914+0000","last_peered":"2023-08-09T13:51:16.262914+0000","last_clean":"2023-08-09T13:51:16.262914+0000","last_became_active":"2023-08-09T13:51:16.262009+0000","last_became_peered":"2023-08-09T13:51:16.262009+0000","last_unstale":"2023-08-09T13:51:16.262914+0000","last_undegraded":"2023-08-09T13:51:16.262914+0000","last_fullsized":"2023-08-09T13:51:16.262914+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:51:15.188798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:51:15.188798+0000","last_clean_scrub_stamp":"2023-08-09T13:51:15.188798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T14:29:26.408639+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.1d","version":"102'2","reported_seq":13,"reported_epoch":102,"state":"active+clean","last_fresh":"2023-08-09T13:51:16.239417+0000","last_change":"2023-08-09T13:51:15.231538+0000","last_active":"2023-08-09T13:51:16.239417+0000","last_peered":"2023-08-09T13:51:16.239417+0000","last_clean":"2023-08-09T13:51:16.239417+0000","last_became_active":"2023-08-09T13:51:15.231233+0000","last_became_peered":"2023-08-09T13:51:15.231233+0000","last_unstale":"2023-08-09T13:51:16.239417+0000","last_undegraded":"2023-08-09T13:51:16.239417+0000","last_fullsized":"2023-08-09T13:51:16.239417+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:51:14.171648+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:51:14.171648+0000","last_clean_scrub_stamp":"2023-08-09T13:51:14.171648+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T15:37:00.814313+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.1d","version":"0'0","reported_seq":9,"reported_epoch":102,"state":"active+clean","last_fresh":"2023-08-09T13:51:16.258451+0000","last_change":"2023-08-09T13:51:16.257359+0000","last_active":"2023-08-09T13:51:16.258451+0000","last_peered":"2023-08-09T13:51:16.258451+0000","last_clean":"2023-08-09T13:51:16.258451+0000","last_became_active":"2023-08-09T13:51:16.256890+0000","last_became_peered":"2023-08-09T13:51:16.256890+0000","last_unstale":"2023-08-09T13:51:16.258451+0000","last_undegraded":"2023-08-09T13:51:16.258451+0000","last_fullsized":"2023-08-09T13:51:16.258451+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:51:15.188798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:51:15.188798+0000","last_clean_scrub_stamp":"2023-08-09T13:51:15.188798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T20:57:55.297812+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.1c","version":"102'2","reported_seq":13,"reported_epoch":102,"state":"active+clean","last_fresh":"2023-08-09T13:51:16.239352+0000","last_change":"2023-08-09T13:51:15.237921+0000","last_active":"2023-08-09T13:51:16.239352+0000","last_peered":"2023-08-09T13:51:16.239352+0000","last_clean":"2023-08-09T13:51:16.239352+0000","last_became_active":"2023-08-09T13:51:15.237645+0000","last_became_peered":"2023-08-09T13:51:15.237645+0000","last_unstale":"2023-08-09T13:51:16.239352+0000","last_undegraded":"2023-08-09T13:51:16.239352+0000","last_fullsized":"2023-08-09T13:51:16.239352+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:51:14.171648+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:51:14.171648+0000","last_clean_scrub_stamp":"2023-08-09T13:51:14.171648+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T19:11:06.692218+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.1a","version":"0'0","reported_seq":9,"reported_epoch":102,"state":"active+clean","last_fresh":"2023-08-09T13:51:16.250222+0000","last_change":"2023-08-09T13:51:16.242848+0000","last_active":"2023-08-09T13:51:16.250222+0000","last_peered":"2023-08-09T13:51:16.250222+0000","last_clean":"2023-08-09T13:51:16.250222+0000","last_became_active":"2023-08-09T13:51:16.242115+0000","last_became_peered":"2023-08-09T13:51:16.242115+0000","last_unstale":"2023-08-09T13:51:16.250222+0000","last_undegraded":"2023-08-09T13:51:16.250222+0000","last_fullsized":"2023-08-09T13:51:16.250222+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:51:15.188798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:51:15.188798+0000","last_clean_scrub_stamp":"2023-08-09T13:51:15.188798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T22:25:07.345536+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.1b","version":"102'1","reported_seq":12,"reported_epoch":102,"state":"active+clean","last_fresh":"2023-08-09T13:51:16.261706+0000","last_change":"2023-08-09T13:51:15.238074+0000","last_active":"2023-08-09T13:51:16.261706+0000","last_peered":"2023-08-09T13:51:16.261706+0000","last_clean":"2023-08-09T13:51:16.261706+0000","last_became_active":"2023-08-09T13:51:15.237763+0000","last_became_peered":"2023-08-09T13:51:15.237763+0000","last_unstale":"2023-08-09T13:51:16.261706+0000","last_undegraded":"2023-08-09T13:51:16.261706+0000","last_fullsized":"2023-08-09T13:51:16.261706+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:51:14.171648+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:51:14.171648+0000","last_clean_scrub_stamp":"2023-08-09T13:51:14.171648+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T23:42:00.578777+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.1b","version":"0'0","reported_seq":9,"reported_epoch":102,"state":"active+clean","last_fresh":"2023-08-09T13:51:16.251871+0000","last_change":"2023-08-09T13:51:16.244386+0000","last_active":"2023-08-09T13:51:16.251871+0000","last_peered":"2023-08-09T13:51:16.251871+0000","last_clean":"2023-08-09T13:51:16.251871+0000","last_became_active":"2023-08-09T13:51:16.244037+0000","last_became_peered":"2023-08-09T13:51:16.244037+0000","last_unstale":"2023-08-09T13:51:16.251871+0000","last_undegraded":"2023-08-09T13:51:16.251871+0000","last_fullsized":"2023-08-09T13:51:16.251871+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:51:15.188798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:51:15.188798+0000","last_clean_scrub_stamp":"2023-08-09T13:51:15.188798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T15:46:24.190152+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"12.1a","version":"102'1","reported_seq":12,"reported_epoch":102,"state":"active+clean","last_fresh":"2023-08-09T13:51:16.251021+0000","last_change":"2023-08-09T13:51:15.244108+0000","last_active":"2023-08-09T13:51:16.251021+0000","last_peered":"2023-08-09T13:51:16.251021+0000","last_clean":"2023-08-09T13:51:16.251021+0000","last_became_active":"2023-08-09T13:51:15.243803+0000","last_became_peered":"2023-08-09T13:51:15.243803+0000","last_unstale":"2023-08-09T13:51:16.251021+0000","last_undegraded":"2023-08-09T13:51:16.251021+0000","last_fullsized":"2023-08-09T13:51:16.251021+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:51:14.171648+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:51:14.171648+0000","last_clean_scrub_stamp":"2023-08-09T13:51:14.171648+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T14:45:06.425833+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.18","version":"0'0","reported_seq":9,"reported_epoch":102,"state":"active+clean","last_fresh":"2023-08-09T13:51:16.246674+0000","last_change":"2023-08-09T13:51:16.244753+0000","last_active":"2023-08-09T13:51:16.246674+0000","last_peered":"2023-08-09T13:51:16.246674+0000","last_clean":"2023-08-09T13:51:16.246674+0000","last_became_active":"2023-08-09T13:51:16.243791+0000","last_became_peered":"2023-08-09T13:51:16.243791+0000","last_unstale":"2023-08-09T13:51:16.246674+0000","last_undegraded":"2023-08-09T13:51:16.246674+0000","last_fullsized":"2023-08-09T13:51:16.246674+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:51:15.188798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:51:15.188798+0000","last_clean_scrub_stamp":"2023-08-09T13:51:15.188798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T21:06:31.859283+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.19","version":"0'0","reported_seq":11,"reported_epoch":102,"state":"active+clean","last_fresh":"2023-08-09T13:51:16.215477+0000","last_change":"2023-08-09T13:51:15.240282+0000","last_active":"2023-08-09T13:51:16.215477+0000","last_peered":"2023-08-09T13:51:16.215477+0000","last_clean":"2023-08-09T13:51:16.215477+0000","last_became_active":"2023-08-09T13:51:15.239951+0000","last_became_peered":"2023-08-09T13:51:15.239951+0000","last_unstale":"2023-08-09T13:51:16.215477+0000","last_undegraded":"2023-08-09T13:51:16.215477+0000","last_fullsized":"2023-08-09T13:51:16.215477+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:51:14.171648+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:51:14.171648+0000","last_clean_scrub_stamp":"2023-08-09T13:51:14.171648+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-11T00:20:32.090823+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.19","version":"0'0","reported_seq":9,"reported_epoch":102,"state":"active+clean","last_fresh":"2023-08-09T13:51:16.253928+0000","last_change":"2023-08-09T13:51:16.246746+0000","last_active":"2023-08-09T13:51:16.253928+0000","last_peered":"2023-08-09T13:51:16.253928+0000","last_clean":"2023-08-09T13:51:16.253928+0000","last_became_active":"2023-08-09T13:51:16.246460+0000","last_became_peered":"2023-08-09T13:51:16.246460+0000","last_unstale":"2023-08-09T13:51:16.253928+0000","last_undegraded":"2023-08-09T13:51:16.253928+0000","last_fullsized":"2023-08-09T13:51:16.253928+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:51:15.188798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:51:15.188798+0000","last_clean_scrub_stamp":"2023-08-09T13:51:15.188798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T19:19:19.019619+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"12.18","version":"0'0","reported_seq":11,"reported_epoch":102,"state":"active+clean","last_fresh":"2023-08-09T13:51:16.220440+0000","last_change":"2023-08-09T13:51:15.241410+0000","last_active":"2023-08-09T13:51:16.220440+0000","last_peered":"2023-08-09T13:51:16.220440+0000","last_clean":"2023-08-09T13:51:16.220440+0000","last_became_active":"2023-08-09T13:51:15.241032+0000","last_became_peered":"2023-08-09T13:51:15.241032+0000","last_unstale":"2023-08-09T13:51:16.220440+0000","last_undegraded":"2023-08-09T13:51:16.220440+0000","last_fullsized":"2023-08-09T13:51:16.220440+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:51:14.171648+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:51:14.171648+0000","last_clean_scrub_stamp":"2023-08-09T13:51:14.171648+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-11T01:06:02.017780+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.16","version":"0'0","reported_seq":9,"reported_epoch":102,"state":"active+clean","last_fresh":"2023-08-09T13:51:16.255954+0000","last_change":"2023-08-09T13:51:16.252395+0000","last_active":"2023-08-09T13:51:16.255954+0000","last_peered":"2023-08-09T13:51:16.255954+0000","last_clean":"2023-08-09T13:51:16.255954+0000","last_became_active":"2023-08-09T13:51:16.252109+0000","last_became_peered":"2023-08-09T13:51:16.252109+0000","last_unstale":"2023-08-09T13:51:16.255954+0000","last_undegraded":"2023-08-09T13:51:16.255954+0000","last_fullsized":"2023-08-09T13:51:16.255954+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:51:15.188798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:51:15.188798+0000","last_clean_scrub_stamp":"2023-08-09T13:51:15.188798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T19:31:46.435562+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"12.17","version":"0'0","reported_seq":11,"reported_epoch":102,"state":"active+clean","last_fresh":"2023-08-09T13:51:16.238565+0000","last_change":"2023-08-09T13:51:15.238456+0000","last_active":"2023-08-09T13:51:16.238565+0000","last_peered":"2023-08-09T13:51:16.238565+0000","last_clean":"2023-08-09T13:51:16.238565+0000","last_became_active":"2023-08-09T13:51:15.238161+0000","last_became_peered":"2023-08-09T13:51:15.238161+0000","last_unstale":"2023-08-09T13:51:16.238565+0000","last_undegraded":"2023-08-09T13:51:16.238565+0000","last_fullsized":"2023-08-09T13:51:16.238565+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:51:14.171648+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:51:14.171648+0000","last_clean_scrub_stamp":"2023-08-09T13:51:14.171648+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T21:31:20.083059+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.17","version":"0'0","reported_seq":9,"reported_epoch":102,"state":"active+clean","last_fresh":"2023-08-09T13:51:16.233986+0000","last_change":"2023-08-09T13:51:16.230647+0000","last_active":"2023-08-09T13:51:16.233986+0000","last_peered":"2023-08-09T13:51:16.233986+0000","last_clean":"2023-08-09T13:51:16.233986+0000","last_became_active":"2023-08-09T13:51:16.230189+0000","last_became_peered":"2023-08-09T13:51:16.230189+0000","last_unstale":"2023-08-09T13:51:16.233986+0000","last_undegraded":"2023-08-09T13:51:16.233986+0000","last_fullsized":"2023-08-09T13:51:16.233986+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:51:15.188798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:51:15.188798+0000","last_clean_scrub_stamp":"2023-08-09T13:51:15.188798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T21:33:15.159821+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.16","version":"0'0","reported_seq":11,"reported_epoch":102,"state":"active+clean","last_fresh":"2023-08-09T13:51:16.242556+0000","last_change":"2023-08-09T13:51:15.245448+0000","last_active":"2023-08-09T13:51:16.242556+0000","last_peered":"2023-08-09T13:51:16.242556+0000","last_clean":"2023-08-09T13:51:16.242556+0000","last_became_active":"2023-08-09T13:51:15.245025+0000","last_became_peered":"2023-08-09T13:51:15.245025+0000","last_unstale":"2023-08-09T13:51:16.242556+0000","last_undegraded":"2023-08-09T13:51:16.242556+0000","last_fullsized":"2023-08-09T13:51:16.242556+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:51:14.171648+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:51:14.171648+0000","last_clean_scrub_stamp":"2023-08-09T13:51:14.171648+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-11T00:15:29.718414+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.14","version":"0'0","reported_seq":9,"reported_epoch":102,"state":"active+clean","last_fresh":"2023-08-09T13:51:16.259370+0000","last_change":"2023-08-09T13:51:16.257396+0000","last_active":"2023-08-09T13:51:16.259370+0000","last_peered":"2023-08-09T13:51:16.259370+0000","last_clean":"2023-08-09T13:51:16.259370+0000","last_became_active":"2023-08-09T13:51:16.257066+0000","last_became_peered":"2023-08-09T13:51:16.257066+0000","last_unstale":"2023-08-09T13:51:16.259370+0000","last_undegraded":"2023-08-09T13:51:16.259370+0000","last_fullsized":"2023-08-09T13:51:16.259370+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:51:15.188798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:51:15.188798+0000","last_clean_scrub_stamp":"2023-08-09T13:51:15.188798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T14:45:07.442984+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.15","version":"102'3","reported_seq":14,"reported_epoch":102,"state":"active+clean","last_fresh":"2023-08-09T13:51:16.262948+0000","last_change":"2023-08-09T13:51:15.242040+0000","last_active":"2023-08-09T13:51:16.262948+0000","last_peered":"2023-08-09T13:51:16.262948+0000","last_clean":"2023-08-09T13:51:16.262948+0000","last_became_active":"2023-08-09T13:51:15.241758+0000","last_became_peered":"2023-08-09T13:51:15.241758+0000","last_unstale":"2023-08-09T13:51:16.262948+0000","last_undegraded":"2023-08-09T13:51:16.262948+0000","last_fullsized":"2023-08-09T13:51:16.262948+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:51:14.171648+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:51:14.171648+0000","last_clean_scrub_stamp":"2023-08-09T13:51:14.171648+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-11T01:20:04.488085+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.15","version":"0'0","reported_seq":9,"reported_epoch":102,"state":"active+clean","last_fresh":"2023-08-09T13:51:16.245065+0000","last_change":"2023-08-09T13:51:16.241568+0000","last_active":"2023-08-09T13:51:16.245065+0000","last_peered":"2023-08-09T13:51:16.245065+0000","last_clean":"2023-08-09T13:51:16.245065+0000","last_became_active":"2023-08-09T13:51:16.240969+0000","last_became_peered":"2023-08-09T13:51:16.240969+0000","last_unstale":"2023-08-09T13:51:16.245065+0000","last_undegraded":"2023-08-09T13:51:16.245065+0000","last_fullsized":"2023-08-09T13:51:16.245065+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:51:15.188798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:51:15.188798+0000","last_clean_scrub_stamp":"2023-08-09T13:51:15.188798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-11T00:07:45.660155+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"12.14","version":"102'6","reported_seq":17,"reported_epoch":102,"state":"active+clean","last_fresh":"2023-08-09T13:51:16.263913+0000","last_change":"2023-08-09T13:51:15.244541+0000","last_active":"2023-08-09T13:51:16.263913+0000","last_peered":"2023-08-09T13:51:16.263913+0000","last_clean":"2023-08-09T13:51:16.263913+0000","last_became_active":"2023-08-09T13:51:15.244282+0000","last_became_peered":"2023-08-09T13:51:15.244282+0000","last_unstale":"2023-08-09T13:51:16.263913+0000","last_undegraded":"2023-08-09T13:51:16.263913+0000","last_fullsized":"2023-08-09T13:51:16.263913+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:51:14.171648+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:51:14.171648+0000","last_clean_scrub_stamp":"2023-08-09T13:51:14.171648+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T17:00:56.192797+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"13.12","version":"0'0","reported_seq":9,"reported_epoch":102,"state":"active+clean","last_fresh":"2023-08-09T13:51:16.239672+0000","last_change":"2023-08-09T13:51:16.232809+0000","last_active":"2023-08-09T13:51:16.239672+0000","last_peered":"2023-08-09T13:51:16.239672+0000","last_clean":"2023-08-09T13:51:16.239672+0000","last_became_active":"2023-08-09T13:51:16.232525+0000","last_became_peered":"2023-08-09T13:51:16.232525+0000","last_unstale":"2023-08-09T13:51:16.239672+0000","last_undegraded":"2023-08-09T13:51:16.239672+0000","last_fullsized":"2023-08-09T13:51:16.239672+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:51:15.188798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:51:15.188798+0000","last_clean_scrub_stamp":"2023-08-09T13:51:15.188798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T19:40:21.994081+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.13","version":"102'2","reported_seq":13,"reported_epoch":102,"state":"active+clean","last_fresh":"2023-08-09T13:51:16.259408+0000","last_change":"2023-08-09T13:51:15.226969+0000","last_active":"2023-08-09T13:51:16.259408+0000","last_peered":"2023-08-09T13:51:16.259408+0000","last_clean":"2023-08-09T13:51:16.259408+0000","last_became_active":"2023-08-09T13:51:15.226656+0000","last_became_peered":"2023-08-09T13:51:15.226656+0000","last_unstale":"2023-08-09T13:51:16.259408+0000","last_undegraded":"2023-08-09T13:51:16.259408+0000","last_fullsized":"2023-08-09T13:51:16.259408+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:51:14.171648+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:51:14.171648+0000","last_clean_scrub_stamp":"2023-08-09T13:51:14.171648+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-11T01:08:20.115321+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.13","version":"0'0","reported_seq":9,"reported_epoch":102,"state":"active+clean","last_fresh":"2023-08-09T13:51:16.247101+0000","last_change":"2023-08-09T13:51:16.244883+0000","last_active":"2023-08-09T13:51:16.247101+0000","last_peered":"2023-08-09T13:51:16.247101+0000","last_clean":"2023-08-09T13:51:16.247101+0000","last_became_active":"2023-08-09T13:51:16.244568+0000","last_became_peered":"2023-08-09T13:51:16.244568+0000","last_unstale":"2023-08-09T13:51:16.247101+0000","last_undegraded":"2023-08-09T13:51:16.247101+0000","last_fullsized":"2023-08-09T13:51:16.247101+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:51:15.188798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:51:15.188798+0000","last_clean_scrub_stamp":"2023-08-09T13:51:15.188798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T21:31:37.211388+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.12","version":"0'0","reported_seq":11,"reported_epoch":102,"state":"active+clean","last_fresh":"2023-08-09T13:51:16.231275+0000","last_change":"2023-08-09T13:51:15.234514+0000","last_active":"2023-08-09T13:51:16.231275+0000","last_peered":"2023-08-09T13:51:16.231275+0000","last_clean":"2023-08-09T13:51:16.231275+0000","last_became_active":"2023-08-09T13:51:15.234183+0000","last_became_peered":"2023-08-09T13:51:15.234183+0000","last_unstale":"2023-08-09T13:51:16.231275+0000","last_undegraded":"2023-08-09T13:51:16.231275+0000","last_fullsized":"2023-08-09T13:51:16.231275+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:51:14.171648+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:51:14.171648+0000","last_clean_scrub_stamp":"2023-08-09T13:51:14.171648+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T19:56:40.377835+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"13.10","version":"0'0","reported_seq":9,"reported_epoch":102,"state":"active+clean","last_fresh":"2023-08-09T13:51:16.249779+0000","last_change":"2023-08-09T13:51:16.241729+0000","last_active":"2023-08-09T13:51:16.249779+0000","last_peered":"2023-08-09T13:51:16.249779+0000","last_clean":"2023-08-09T13:51:16.249779+0000","last_became_active":"2023-08-09T13:51:16.241367+0000","last_became_peered":"2023-08-09T13:51:16.241367+0000","last_unstale":"2023-08-09T13:51:16.249779+0000","last_undegraded":"2023-08-09T13:51:16.249779+0000","last_fullsized":"2023-08-09T13:51:16.249779+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:51:15.188798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:51:15.188798+0000","last_clean_scrub_stamp":"2023-08-09T13:51:15.188798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T18:12:30.400222+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.11","version":"0'0","reported_seq":11,"reported_epoch":102,"state":"active+clean","last_fresh":"2023-08-09T13:51:16.220629+0000","last_change":"2023-08-09T13:51:15.243824+0000","last_active":"2023-08-09T13:51:16.220629+0000","last_peered":"2023-08-09T13:51:16.220629+0000","last_clean":"2023-08-09T13:51:16.220629+0000","last_became_active":"2023-08-09T13:51:15.243491+0000","last_became_peered":"2023-08-09T13:51:15.243491+0000","last_unstale":"2023-08-09T13:51:16.220629+0000","last_undegraded":"2023-08-09T13:51:16.220629+0000","last_fullsized":"2023-08-09T13:51:16.220629+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:51:14.171648+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:51:14.171648+0000","last_clean_scrub_stamp":"2023-08-09T13:51:14.171648+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T19:04:31.316491+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"13.11","version":"0'0","reported_seq":9,"reported_epoch":102,"state":"active+clean","last_fresh":"2023-08-09T13:51:16.262715+0000","last_change":"2023-08-09T13:51:16.257254+0000","last_active":"2023-08-09T13:51:16.262715+0000","last_peered":"2023-08-09T13:51:16.262715+0000","last_clean":"2023-08-09T13:51:16.262715+0000","last_became_active":"2023-08-09T13:51:16.256967+0000","last_became_peered":"2023-08-09T13:51:16.256967+0000","last_unstale":"2023-08-09T13:51:16.262715+0000","last_undegraded":"2023-08-09T13:51:16.262715+0000","last_fullsized":"2023-08-09T13:51:16.262715+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:51:15.188798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:51:15.188798+0000","last_clean_scrub_stamp":"2023-08-09T13:51:15.188798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T23:46:26.444683+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.10","version":"102'8","reported_seq":19,"reported_epoch":102,"state":"active+clean","last_fresh":"2023-08-09T13:51:16.250969+0000","last_change":"2023-08-09T13:51:15.233631+0000","last_active":"2023-08-09T13:51:16.250969+0000","last_peered":"2023-08-09T13:51:16.250969+0000","last_clean":"2023-08-09T13:51:16.250969+0000","last_became_active":"2023-08-09T13:51:15.233323+0000","last_became_peered":"2023-08-09T13:51:15.233323+0000","last_unstale":"2023-08-09T13:51:16.250969+0000","last_undegraded":"2023-08-09T13:51:16.250969+0000","last_fullsized":"2023-08-09T13:51:16.250969+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:51:14.171648+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:51:14.171648+0000","last_clean_scrub_stamp":"2023-08-09T13:51:14.171648+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T17:29:03.609676+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.6","version":"0'0","reported_seq":9,"reported_epoch":102,"state":"active+clean","last_fresh":"2023-08-09T13:51:16.254026+0000","last_change":"2023-08-09T13:51:16.246989+0000","last_active":"2023-08-09T13:51:16.254026+0000","last_peered":"2023-08-09T13:51:16.254026+0000","last_clean":"2023-08-09T13:51:16.254026+0000","last_became_active":"2023-08-09T13:51:16.246731+0000","last_became_peered":"2023-08-09T13:51:16.246731+0000","last_unstale":"2023-08-09T13:51:16.254026+0000","last_undegraded":"2023-08-09T13:51:16.254026+0000","last_fullsized":"2023-08-09T13:51:16.254026+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:51:15.188798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:51:15.188798+0000","last_clean_scrub_stamp":"2023-08-09T13:51:15.188798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-11T00:47:48.385325+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"12.7","version":"102'2","reported_seq":13,"reported_epoch":102,"state":"active+clean","last_fresh":"2023-08-09T13:51:16.253593+0000","last_change":"2023-08-09T13:51:15.228814+0000","last_active":"2023-08-09T13:51:16.253593+0000","last_peered":"2023-08-09T13:51:16.253593+0000","last_clean":"2023-08-09T13:51:16.253593+0000","last_became_active":"2023-08-09T13:51:15.227947+0000","last_became_peered":"2023-08-09T13:51:15.227947+0000","last_unstale":"2023-08-09T13:51:16.253593+0000","last_undegraded":"2023-08-09T13:51:16.253593+0000","last_fullsized":"2023-08-09T13:51:16.253593+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:51:14.171648+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:51:14.171648+0000","last_clean_scrub_stamp":"2023-08-09T13:51:14.171648+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-11T01:50:49.808694+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.7","version":"0'0","reported_seq":9,"reported_epoch":102,"state":"active+clean","last_fresh":"2023-08-09T13:51:16.257386+0000","last_change":"2023-08-09T13:51:16.250017+0000","last_active":"2023-08-09T13:51:16.257386+0000","last_peered":"2023-08-09T13:51:16.257386+0000","last_clean":"2023-08-09T13:51:16.257386+0000","last_became_active":"2023-08-09T13:51:16.249750+0000","last_became_peered":"2023-08-09T13:51:16.249750+0000","last_unstale":"2023-08-09T13:51:16.257386+0000","last_undegraded":"2023-08-09T13:51:16.257386+0000","last_fullsized":"2023-08-09T13:51:16.257386+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:51:15.188798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:51:15.188798+0000","last_clean_scrub_stamp":"2023-08-09T13:51:15.188798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T15:17:36.104232+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.6","version":"0'0","reported_seq":11,"reported_epoch":102,"state":"active+clean","last_fresh":"2023-08-09T13:51:16.223060+0000","last_change":"2023-08-09T13:51:15.227422+0000","last_active":"2023-08-09T13:51:16.223060+0000","last_peered":"2023-08-09T13:51:16.223060+0000","last_clean":"2023-08-09T13:51:16.223060+0000","last_became_active":"2023-08-09T13:51:15.227114+0000","last_became_peered":"2023-08-09T13:51:15.227114+0000","last_unstale":"2023-08-09T13:51:16.223060+0000","last_undegraded":"2023-08-09T13:51:16.223060+0000","last_fullsized":"2023-08-09T13:51:16.223060+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:51:14.171648+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:51:14.171648+0000","last_clean_scrub_stamp":"2023-08-09T13:51:14.171648+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T18:00:32.163746+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.4","version":"0'0","reported_seq":9,"reported_epoch":102,"state":"active+clean","last_fresh":"2023-08-09T13:51:16.259425+0000","last_change":"2023-08-09T13:51:16.254767+0000","last_active":"2023-08-09T13:51:16.259425+0000","last_peered":"2023-08-09T13:51:16.259425+0000","last_clean":"2023-08-09T13:51:16.259425+0000","last_became_active":"2023-08-09T13:51:16.254146+0000","last_became_peered":"2023-08-09T13:51:16.254146+0000","last_unstale":"2023-08-09T13:51:16.259425+0000","last_undegraded":"2023-08-09T13:51:16.259425+0000","last_fullsized":"2023-08-09T13:51:16.259425+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:51:15.188798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:51:15.188798+0000","last_clean_scrub_stamp":"2023-08-09T13:51:15.188798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T22:04:09.500719+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.5","version":"0'0","reported_seq":11,"reported_epoch":102,"state":"active+clean","last_fresh":"2023-08-09T13:51:16.233809+0000","last_change":"2023-08-09T13:51:15.241210+0000","last_active":"2023-08-09T13:51:16.233809+0000","last_peered":"2023-08-09T13:51:16.233809+0000","last_clean":"2023-08-09T13:51:16.233809+0000","last_became_active":"2023-08-09T13:51:15.240939+0000","last_became_peered":"2023-08-09T13:51:15.240939+0000","last_unstale":"2023-08-09T13:51:16.233809+0000","last_undegraded":"2023-08-09T13:51:16.233809+0000","last_fullsized":"2023-08-09T13:51:16.233809+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:51:14.171648+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:51:14.171648+0000","last_clean_scrub_stamp":"2023-08-09T13:51:14.171648+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T23:11:40.454290+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.5","version":"0'0","reported_seq":9,"reported_epoch":102,"state":"active+clean","last_fresh":"2023-08-09T13:51:16.245044+0000","last_change":"2023-08-09T13:51:16.241394+0000","last_active":"2023-08-09T13:51:16.245044+0000","last_peered":"2023-08-09T13:51:16.245044+0000","last_clean":"2023-08-09T13:51:16.245044+0000","last_became_active":"2023-08-09T13:51:16.240570+0000","last_became_peered":"2023-08-09T13:51:16.240570+0000","last_unstale":"2023-08-09T13:51:16.245044+0000","last_undegraded":"2023-08-09T13:51:16.245044+0000","last_fullsized":"2023-08-09T13:51:16.245044+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:51:15.188798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:51:15.188798+0000","last_clean_scrub_stamp":"2023-08-09T13:51:15.188798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T18:43:48.957746+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"12.4","version":"0'0","reported_seq":11,"reported_epoch":102,"state":"active+clean","last_fresh":"2023-08-09T13:51:16.240345+0000","last_change":"2023-08-09T13:51:15.235875+0000","last_active":"2023-08-09T13:51:16.240345+0000","last_peered":"2023-08-09T13:51:16.240345+0000","last_clean":"2023-08-09T13:51:16.240345+0000","last_became_active":"2023-08-09T13:51:15.235611+0000","last_became_peered":"2023-08-09T13:51:15.235611+0000","last_unstale":"2023-08-09T13:51:16.240345+0000","last_undegraded":"2023-08-09T13:51:16.240345+0000","last_fullsized":"2023-08-09T13:51:16.240345+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:51:14.171648+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:51:14.171648+0000","last_clean_scrub_stamp":"2023-08-09T13:51:14.171648+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T20:19:11.130074+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.2","version":"0'0","reported_seq":9,"reported_epoch":102,"state":"active+clean","last_fresh":"2023-08-09T13:51:16.253866+0000","last_change":"2023-08-09T13:51:16.246505+0000","last_active":"2023-08-09T13:51:16.253866+0000","last_peered":"2023-08-09T13:51:16.253866+0000","last_clean":"2023-08-09T13:51:16.253866+0000","last_became_active":"2023-08-09T13:51:16.246118+0000","last_became_peered":"2023-08-09T13:51:16.246118+0000","last_unstale":"2023-08-09T13:51:16.253866+0000","last_undegraded":"2023-08-09T13:51:16.253866+0000","last_fullsized":"2023-08-09T13:51:16.253866+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:51:15.188798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:51:15.188798+0000","last_clean_scrub_stamp":"2023-08-09T13:51:15.188798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T18:27:52.202303+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.3","version":"0'0","reported_seq":11,"reported_epoch":102,"state":"active+clean","last_fresh":"2023-08-09T13:51:16.216741+0000","last_change":"2023-08-09T13:51:15.235344+0000","last_active":"2023-08-09T13:51:16.216741+0000","last_peered":"2023-08-09T13:51:16.216741+0000","last_clean":"2023-08-09T13:51:16.216741+0000","last_became_active":"2023-08-09T13:51:15.235037+0000","last_became_peered":"2023-08-09T13:51:15.235037+0000","last_unstale":"2023-08-09T13:51:16.216741+0000","last_undegraded":"2023-08-09T13:51:16.216741+0000","last_fullsized":"2023-08-09T13:51:16.216741+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:51:14.171648+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:51:14.171648+0000","last_clean_scrub_stamp":"2023-08-09T13:51:14.171648+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T23:42:00.578777+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.3","version":"0'0","reported_seq":9,"reported_epoch":102,"state":"active+clean","last_fresh":"2023-08-09T13:51:16.251027+0000","last_change":"2023-08-09T13:51:16.240636+0000","last_active":"2023-08-09T13:51:16.251027+0000","last_peered":"2023-08-09T13:51:16.251027+0000","last_clean":"2023-08-09T13:51:16.251027+0000","last_became_active":"2023-08-09T13:51:16.240337+0000","last_became_peered":"2023-08-09T13:51:16.240337+0000","last_unstale":"2023-08-09T13:51:16.251027+0000","last_undegraded":"2023-08-09T13:51:16.251027+0000","last_fullsized":"2023-08-09T13:51:16.251027+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:51:15.188798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:51:15.188798+0000","last_clean_scrub_stamp":"2023-08-09T13:51:15.188798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T18:12:35.312144+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.2","version":"0'0","reported_seq":11,"reported_epoch":102,"state":"active+clean","last_fresh":"2023-08-09T13:51:16.224695+0000","last_change":"2023-08-09T13:51:15.241593+0000","last_active":"2023-08-09T13:51:16.224695+0000","last_peered":"2023-08-09T13:51:16.224695+0000","last_clean":"2023-08-09T13:51:16.224695+0000","last_became_active":"2023-08-09T13:51:15.241319+0000","last_became_peered":"2023-08-09T13:51:15.241319+0000","last_unstale":"2023-08-09T13:51:16.224695+0000","last_undegraded":"2023-08-09T13:51:16.224695+0000","last_fullsized":"2023-08-09T13:51:16.224695+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:51:14.171648+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:51:14.171648+0000","last_clean_scrub_stamp":"2023-08-09T13:51:14.171648+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T18:12:34.294994+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.0","version":"0'0","reported_seq":9,"reported_epoch":102,"state":"active+clean","last_fresh":"2023-08-09T13:51:16.257346+0000","last_change":"2023-08-09T13:51:16.252480+0000","last_active":"2023-08-09T13:51:16.257346+0000","last_peered":"2023-08-09T13:51:16.257346+0000","last_clean":"2023-08-09T13:51:16.257346+0000","last_became_active":"2023-08-09T13:51:16.252206+0000","last_became_peered":"2023-08-09T13:51:16.252206+0000","last_unstale":"2023-08-09T13:51:16.257346+0000","last_undegraded":"2023-08-09T13:51:16.257346+0000","last_fullsized":"2023-08-09T13:51:16.257346+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:51:15.188798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:51:15.188798+0000","last_clean_scrub_stamp":"2023-08-09T13:51:15.188798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T17:04:13.279977+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.1","version":"0'0","reported_seq":11,"reported_epoch":102,"state":"active+clean","last_fresh":"2023-08-09T13:51:16.238942+0000","last_change":"2023-08-09T13:51:15.239117+0000","last_active":"2023-08-09T13:51:16.238942+0000","last_peered":"2023-08-09T13:51:16.238942+0000","last_clean":"2023-08-09T13:51:16.238942+0000","last_became_active":"2023-08-09T13:51:15.238862+0000","last_became_peered":"2023-08-09T13:51:15.238862+0000","last_unstale":"2023-08-09T13:51:16.238942+0000","last_undegraded":"2023-08-09T13:51:16.238942+0000","last_fullsized":"2023-08-09T13:51:16.238942+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:51:14.171648+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:51:14.171648+0000","last_clean_scrub_stamp":"2023-08-09T13:51:14.171648+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-11T00:32:53.371031+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.1","version":"0'0","reported_seq":9,"reported_epoch":102,"state":"active+clean","last_fresh":"2023-08-09T13:51:16.253203+0000","last_change":"2023-08-09T13:51:16.248941+0000","last_active":"2023-08-09T13:51:16.253203+0000","last_peered":"2023-08-09T13:51:16.253203+0000","last_clean":"2023-08-09T13:51:16.253203+0000","last_became_active":"2023-08-09T13:51:16.248630+0000","last_became_peered":"2023-08-09T13:51:16.248630+0000","last_unstale":"2023-08-09T13:51:16.253203+0000","last_undegraded":"2023-08-09T13:51:16.253203+0000","last_fullsized":"2023-08-09T13:51:16.253203+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:51:15.188798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:51:15.188798+0000","last_clean_scrub_stamp":"2023-08-09T13:51:15.188798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T16:33:43.728000+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.0","version":"0'0","reported_seq":11,"reported_epoch":102,"state":"active+clean","last_fresh":"2023-08-09T13:51:16.222583+0000","last_change":"2023-08-09T13:51:15.245201+0000","last_active":"2023-08-09T13:51:16.222583+0000","last_peered":"2023-08-09T13:51:16.222583+0000","last_clean":"2023-08-09T13:51:16.222583+0000","last_became_active":"2023-08-09T13:51:15.244899+0000","last_became_peered":"2023-08-09T13:51:15.244899+0000","last_unstale":"2023-08-09T13:51:16.222583+0000","last_undegraded":"2023-08-09T13:51:16.222583+0000","last_fullsized":"2023-08-09T13:51:16.222583+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:51:14.171648+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:51:14.171648+0000","last_clean_scrub_stamp":"2023-08-09T13:51:14.171648+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T18:26:02.112097+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.c","version":"0'0","reported_seq":9,"reported_epoch":102,"state":"active+clean","last_fresh":"2023-08-09T13:51:16.252537+0000","last_change":"2023-08-09T13:51:16.232600+0000","last_active":"2023-08-09T13:51:16.252537+0000","last_peered":"2023-08-09T13:51:16.252537+0000","last_clean":"2023-08-09T13:51:16.252537+0000","last_became_active":"2023-08-09T13:51:16.232287+0000","last_became_peered":"2023-08-09T13:51:16.232287+0000","last_unstale":"2023-08-09T13:51:16.252537+0000","last_undegraded":"2023-08-09T13:51:16.252537+0000","last_fullsized":"2023-08-09T13:51:16.252537+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:51:15.188798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:51:15.188798+0000","last_clean_scrub_stamp":"2023-08-09T13:51:15.188798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T18:52:12.499231+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.d","version":"102'1","reported_seq":12,"reported_epoch":102,"state":"active+clean","last_fresh":"2023-08-09T13:51:16.262913+0000","last_change":"2023-08-09T13:51:15.233806+0000","last_active":"2023-08-09T13:51:16.262913+0000","last_peered":"2023-08-09T13:51:16.262913+0000","last_clean":"2023-08-09T13:51:16.262913+0000","last_became_active":"2023-08-09T13:51:15.233316+0000","last_became_peered":"2023-08-09T13:51:15.233316+0000","last_unstale":"2023-08-09T13:51:16.262913+0000","last_undegraded":"2023-08-09T13:51:16.262913+0000","last_fullsized":"2023-08-09T13:51:16.262913+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:51:14.171648+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:51:14.171648+0000","last_clean_scrub_stamp":"2023-08-09T13:51:14.171648+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T23:05:57.606883+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":169,"reported_epoch":102,"state":"active+clean","last_fresh":"2023-08-09T13:51:16.229542+0000","last_change":"2023-08-09T13:33:56.401406+0000","last_active":"2023-08-09T13:51:16.229542+0000","last_peered":"2023-08-09T13:51:16.229542+0000","last_clean":"2023-08-09T13:51:16.229542+0000","last_became_active":"2023-08-09T13:33:56.400487+0000","last_became_peered":"2023-08-09T13:33:56.400487+0000","last_unstale":"2023-08-09T13:51:16.229542+0000","last_undegraded":"2023-08-09T13:51:16.229542+0000","last_fullsized":"2023-08-09T13:51:16.229542+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:33:54.845869+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:33:54.845869+0000","last_clean_scrub_stamp":"2023-08-09T13:33:54.845869+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T23:03:24.702690+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"13.d","version":"0'0","reported_seq":9,"reported_epoch":102,"state":"active+clean","last_fresh":"2023-08-09T13:51:16.257651+0000","last_change":"2023-08-09T13:51:16.255187+0000","last_active":"2023-08-09T13:51:16.257651+0000","last_peered":"2023-08-09T13:51:16.257651+0000","last_clean":"2023-08-09T13:51:16.257651+0000","last_became_active":"2023-08-09T13:51:16.254860+0000","last_became_peered":"2023-08-09T13:51:16.254860+0000","last_unstale":"2023-08-09T13:51:16.257651+0000","last_undegraded":"2023-08-09T13:51:16.257651+0000","last_fullsized":"2023-08-09T13:51:16.257651+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:51:15.188798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:51:15.188798+0000","last_clean_scrub_stamp":"2023-08-09T13:51:15.188798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T17:50:15.038705+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"12.c","version":"0'0","reported_seq":11,"reported_epoch":102,"state":"active+clean","last_fresh":"2023-08-09T13:51:16.215940+0000","last_change":"2023-08-09T13:51:15.241814+0000","last_active":"2023-08-09T13:51:16.215940+0000","last_peered":"2023-08-09T13:51:16.215940+0000","last_clean":"2023-08-09T13:51:16.215940+0000","last_became_active":"2023-08-09T13:51:15.241491+0000","last_became_peered":"2023-08-09T13:51:15.241491+0000","last_unstale":"2023-08-09T13:51:16.215940+0000","last_undegraded":"2023-08-09T13:51:16.215940+0000","last_fullsized":"2023-08-09T13:51:16.215940+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:51:14.171648+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:51:14.171648+0000","last_clean_scrub_stamp":"2023-08-09T13:51:14.171648+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T18:26:02.112097+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":173,"reported_epoch":102,"state":"active+clean","last_fresh":"2023-08-09T13:51:16.220850+0000","last_change":"2023-08-09T13:33:56.401123+0000","last_active":"2023-08-09T13:51:16.220850+0000","last_peered":"2023-08-09T13:51:16.220850+0000","last_clean":"2023-08-09T13:51:16.220850+0000","last_became_active":"2023-08-09T13:33:56.399933+0000","last_became_peered":"2023-08-09T13:33:56.399933+0000","last_unstale":"2023-08-09T13:51:16.220850+0000","last_undegraded":"2023-08-09T13:51:16.220850+0000","last_fullsized":"2023-08-09T13:51:16.220850+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:33:54.845869+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:33:54.845869+0000","last_clean_scrub_stamp":"2023-08-09T13:33:54.845869+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T18:30:45.254362+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"13.e","version":"0'0","reported_seq":9,"reported_epoch":102,"state":"active+clean","last_fresh":"2023-08-09T13:51:16.257418+0000","last_change":"2023-08-09T13:51:16.251088+0000","last_active":"2023-08-09T13:51:16.257418+0000","last_peered":"2023-08-09T13:51:16.257418+0000","last_clean":"2023-08-09T13:51:16.257418+0000","last_became_active":"2023-08-09T13:51:16.250820+0000","last_became_peered":"2023-08-09T13:51:16.250820+0000","last_unstale":"2023-08-09T13:51:16.257418+0000","last_undegraded":"2023-08-09T13:51:16.257418+0000","last_fullsized":"2023-08-09T13:51:16.257418+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:51:15.188798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:51:15.188798+0000","last_clean_scrub_stamp":"2023-08-09T13:51:15.188798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T14:17:16.947840+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.f","version":"102'1","reported_seq":12,"reported_epoch":102,"state":"active+clean","last_fresh":"2023-08-09T13:51:16.258359+0000","last_change":"2023-08-09T13:51:15.243510+0000","last_active":"2023-08-09T13:51:16.258359+0000","last_peered":"2023-08-09T13:51:16.258359+0000","last_clean":"2023-08-09T13:51:16.258359+0000","last_became_active":"2023-08-09T13:51:15.242763+0000","last_became_peered":"2023-08-09T13:51:15.242763+0000","last_unstale":"2023-08-09T13:51:16.258359+0000","last_undegraded":"2023-08-09T13:51:16.258359+0000","last_fullsized":"2023-08-09T13:51:16.258359+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:51:14.171648+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:51:14.171648+0000","last_clean_scrub_stamp":"2023-08-09T13:51:14.171648+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T17:01:45.085632+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"1.2","version":"0'0","reported_seq":169,"reported_epoch":102,"state":"active+clean","last_fresh":"2023-08-09T13:51:16.228756+0000","last_change":"2023-08-09T13:33:56.402794+0000","last_active":"2023-08-09T13:51:16.228756+0000","last_peered":"2023-08-09T13:51:16.228756+0000","last_clean":"2023-08-09T13:51:16.228756+0000","last_became_active":"2023-08-09T13:33:56.402038+0000","last_became_peered":"2023-08-09T13:33:56.402038+0000","last_unstale":"2023-08-09T13:51:16.228756+0000","last_undegraded":"2023-08-09T13:51:16.228756+0000","last_fullsized":"2023-08-09T13:51:16.228756+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:33:54.845869+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:33:54.845869+0000","last_clean_scrub_stamp":"2023-08-09T13:33:54.845869+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T19:04:24.852783+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"13.f","version":"0'0","reported_seq":9,"reported_epoch":102,"state":"active+clean","last_fresh":"2023-08-09T13:51:16.258516+0000","last_change":"2023-08-09T13:51:16.248412+0000","last_active":"2023-08-09T13:51:16.258516+0000","last_peered":"2023-08-09T13:51:16.258516+0000","last_clean":"2023-08-09T13:51:16.258516+0000","last_became_active":"2023-08-09T13:51:16.247925+0000","last_became_peered":"2023-08-09T13:51:16.247925+0000","last_unstale":"2023-08-09T13:51:16.258516+0000","last_undegraded":"2023-08-09T13:51:16.258516+0000","last_fullsized":"2023-08-09T13:51:16.258516+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:51:15.188798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:51:15.188798+0000","last_clean_scrub_stamp":"2023-08-09T13:51:15.188798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T18:16:15.512137+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.e","version":"102'2","reported_seq":13,"reported_epoch":102,"state":"active+clean","last_fresh":"2023-08-09T13:51:16.240072+0000","last_change":"2023-08-09T13:51:15.240950+0000","last_active":"2023-08-09T13:51:16.240072+0000","last_peered":"2023-08-09T13:51:16.240072+0000","last_clean":"2023-08-09T13:51:16.240072+0000","last_became_active":"2023-08-09T13:51:15.240595+0000","last_became_peered":"2023-08-09T13:51:15.240595+0000","last_unstale":"2023-08-09T13:51:16.240072+0000","last_undegraded":"2023-08-09T13:51:16.240072+0000","last_fullsized":"2023-08-09T13:51:16.240072+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:51:14.171648+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:51:14.171648+0000","last_clean_scrub_stamp":"2023-08-09T13:51:14.171648+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T15:46:23.173001+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.3","version":"0'0","reported_seq":169,"reported_epoch":102,"state":"active+clean","last_fresh":"2023-08-09T13:51:16.221138+0000","last_change":"2023-08-09T13:33:55.886749+0000","last_active":"2023-08-09T13:51:16.221138+0000","last_peered":"2023-08-09T13:51:16.221138+0000","last_clean":"2023-08-09T13:51:16.221138+0000","last_became_active":"2023-08-09T13:33:55.886116+0000","last_became_peered":"2023-08-09T13:33:55.886116+0000","last_unstale":"2023-08-09T13:51:16.221138+0000","last_undegraded":"2023-08-09T13:51:16.221138+0000","last_fullsized":"2023-08-09T13:51:16.221138+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:33:54.845869+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:33:54.845869+0000","last_clean_scrub_stamp":"2023-08-09T13:33:54.845869+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T21:23:05.575670+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"92'175","reported_seq":524,"reported_epoch":102,"state":"active+clean","last_fresh":"2023-08-09T13:51:16.229403+0000","last_change":"2023-08-09T13:33:55.886345+0000","last_active":"2023-08-09T13:51:16.229403+0000","last_peered":"2023-08-09T13:51:16.229403+0000","last_clean":"2023-08-09T13:51:16.229403+0000","last_became_active":"2023-08-09T13:33:55.885781+0000","last_became_peered":"2023-08-09T13:33:55.885781+0000","last_unstale":"2023-08-09T13:51:16.229403+0000","last_undegraded":"2023-08-09T13:51:16.229403+0000","last_fullsized":"2023-08-09T13:51:16.229403+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:33:54.845869+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:33:54.845869+0000","last_clean_scrub_stamp":"2023-08-09T13:33:54.845869+0000","objects_scrubbed":0,"log_size":175,"log_dups_size":0,"ondisk_log_size":175,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T13:41:45.285281+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":619,"num_read_kb":1212,"num_write":263,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"13.8","version":"0'0","reported_seq":9,"reported_epoch":102,"state":"active+clean","last_fresh":"2023-08-09T13:51:16.246718+0000","last_change":"2023-08-09T13:51:16.239728+0000","last_active":"2023-08-09T13:51:16.246718+0000","last_peered":"2023-08-09T13:51:16.246718+0000","last_clean":"2023-08-09T13:51:16.246718+0000","last_became_active":"2023-08-09T13:51:16.239354+0000","last_became_peered":"2023-08-09T13:51:16.239354+0000","last_unstale":"2023-08-09T13:51:16.246718+0000","last_undegraded":"2023-08-09T13:51:16.246718+0000","last_fullsized":"2023-08-09T13:51:16.246718+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:51:15.188798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:51:15.188798+0000","last_clean_scrub_stamp":"2023-08-09T13:51:15.188798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T16:12:43.536183+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.9","version":"0'0","reported_seq":11,"reported_epoch":102,"state":"active+clean","last_fresh":"2023-08-09T13:51:16.237041+0000","last_change":"2023-08-09T13:51:15.243807+0000","last_active":"2023-08-09T13:51:16.237041+0000","last_peered":"2023-08-09T13:51:16.237041+0000","last_clean":"2023-08-09T13:51:16.237041+0000","last_became_active":"2023-08-09T13:51:15.243444+0000","last_became_peered":"2023-08-09T13:51:15.243444+0000","last_unstale":"2023-08-09T13:51:16.237041+0000","last_undegraded":"2023-08-09T13:51:16.237041+0000","last_fullsized":"2023-08-09T13:51:16.237041+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:51:14.171648+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:51:14.171648+0000","last_clean_scrub_stamp":"2023-08-09T13:51:14.171648+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-11T00:45:40.422997+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":179,"reported_epoch":102,"state":"active+clean","last_fresh":"2023-08-09T13:51:16.230041+0000","last_change":"2023-08-09T13:33:55.889565+0000","last_active":"2023-08-09T13:51:16.230041+0000","last_peered":"2023-08-09T13:51:16.230041+0000","last_clean":"2023-08-09T13:51:16.230041+0000","last_became_active":"2023-08-09T13:33:55.889164+0000","last_became_peered":"2023-08-09T13:33:55.889164+0000","last_unstale":"2023-08-09T13:51:16.230041+0000","last_undegraded":"2023-08-09T13:51:16.230041+0000","last_fullsized":"2023-08-09T13:51:16.230041+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:33:54.845869+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:33:54.845869+0000","last_clean_scrub_stamp":"2023-08-09T13:33:54.845869+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T18:48:05.967165+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"13.9","version":"0'0","reported_seq":9,"reported_epoch":102,"state":"active+clean","last_fresh":"2023-08-09T13:51:16.234817+0000","last_change":"2023-08-09T13:51:16.232672+0000","last_active":"2023-08-09T13:51:16.234817+0000","last_peered":"2023-08-09T13:51:16.234817+0000","last_clean":"2023-08-09T13:51:16.234817+0000","last_became_active":"2023-08-09T13:51:16.232084+0000","last_became_peered":"2023-08-09T13:51:16.232084+0000","last_unstale":"2023-08-09T13:51:16.234817+0000","last_undegraded":"2023-08-09T13:51:16.234817+0000","last_fullsized":"2023-08-09T13:51:16.234817+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:51:15.188798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:51:15.188798+0000","last_clean_scrub_stamp":"2023-08-09T13:51:15.188798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T17:25:36.877280+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.8","version":"0'0","reported_seq":11,"reported_epoch":102,"state":"active+clean","last_fresh":"2023-08-09T13:51:16.229763+0000","last_change":"2023-08-09T13:51:15.231846+0000","last_active":"2023-08-09T13:51:16.229763+0000","last_peered":"2023-08-09T13:51:16.229763+0000","last_clean":"2023-08-09T13:51:16.229763+0000","last_became_active":"2023-08-09T13:51:15.231559+0000","last_became_peered":"2023-08-09T13:51:15.231559+0000","last_unstale":"2023-08-09T13:51:16.229763+0000","last_undegraded":"2023-08-09T13:51:16.229763+0000","last_fullsized":"2023-08-09T13:51:16.229763+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:51:14.171648+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:51:14.171648+0000","last_clean_scrub_stamp":"2023-08-09T13:51:14.171648+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T17:12:30.175796+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":173,"reported_epoch":102,"state":"active+clean","last_fresh":"2023-08-09T13:51:16.228648+0000","last_change":"2023-08-09T13:33:56.404236+0000","last_active":"2023-08-09T13:51:16.228648+0000","last_peered":"2023-08-09T13:51:16.228648+0000","last_clean":"2023-08-09T13:51:16.228648+0000","last_became_active":"2023-08-09T13:33:56.402485+0000","last_became_peered":"2023-08-09T13:33:56.402485+0000","last_unstale":"2023-08-09T13:51:16.228648+0000","last_undegraded":"2023-08-09T13:51:16.228648+0000","last_fullsized":"2023-08-09T13:51:16.228648+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:33:54.845869+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:33:54.845869+0000","last_clean_scrub_stamp":"2023-08-09T13:33:54.845869+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T22:56:55.178189+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.a","version":"0'0","reported_seq":9,"reported_epoch":102,"state":"active+clean","last_fresh":"2023-08-09T13:51:16.258171+0000","last_change":"2023-08-09T13:51:16.254430+0000","last_active":"2023-08-09T13:51:16.258171+0000","last_peered":"2023-08-09T13:51:16.258171+0000","last_clean":"2023-08-09T13:51:16.258171+0000","last_became_active":"2023-08-09T13:51:16.252692+0000","last_became_peered":"2023-08-09T13:51:16.252692+0000","last_unstale":"2023-08-09T13:51:16.258171+0000","last_undegraded":"2023-08-09T13:51:16.258171+0000","last_fullsized":"2023-08-09T13:51:16.258171+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:51:15.188798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:51:15.188798+0000","last_clean_scrub_stamp":"2023-08-09T13:51:15.188798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T17:51:29.452605+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.b","version":"102'2","reported_seq":13,"reported_epoch":102,"state":"active+clean","last_fresh":"2023-08-09T13:51:16.251780+0000","last_change":"2023-08-09T13:51:15.236904+0000","last_active":"2023-08-09T13:51:16.251780+0000","last_peered":"2023-08-09T13:51:16.251780+0000","last_clean":"2023-08-09T13:51:16.251780+0000","last_became_active":"2023-08-09T13:51:15.236532+0000","last_became_peered":"2023-08-09T13:51:15.236532+0000","last_unstale":"2023-08-09T13:51:16.251780+0000","last_undegraded":"2023-08-09T13:51:16.251780+0000","last_fullsized":"2023-08-09T13:51:16.251780+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:51:14.171648+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:51:14.171648+0000","last_clean_scrub_stamp":"2023-08-09T13:51:14.171648+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T17:59:38.896262+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":167,"reported_epoch":102,"state":"active+clean","last_fresh":"2023-08-09T13:51:16.242693+0000","last_change":"2023-08-09T13:33:56.401781+0000","last_active":"2023-08-09T13:51:16.242693+0000","last_peered":"2023-08-09T13:51:16.242693+0000","last_clean":"2023-08-09T13:51:16.242693+0000","last_became_active":"2023-08-09T13:33:56.400336+0000","last_became_peered":"2023-08-09T13:33:56.400336+0000","last_unstale":"2023-08-09T13:51:16.242693+0000","last_undegraded":"2023-08-09T13:51:16.242693+0000","last_fullsized":"2023-08-09T13:51:16.242693+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:33:54.845869+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:33:54.845869+0000","last_clean_scrub_stamp":"2023-08-09T13:33:54.845869+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T16:50:41.367492+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.b","version":"0'0","reported_seq":9,"reported_epoch":102,"state":"active+clean","last_fresh":"2023-08-09T13:51:16.253879+0000","last_change":"2023-08-09T13:51:16.246308+0000","last_active":"2023-08-09T13:51:16.253879+0000","last_peered":"2023-08-09T13:51:16.253879+0000","last_clean":"2023-08-09T13:51:16.253879+0000","last_became_active":"2023-08-09T13:51:16.245989+0000","last_became_peered":"2023-08-09T13:51:16.245989+0000","last_unstale":"2023-08-09T13:51:16.253879+0000","last_undegraded":"2023-08-09T13:51:16.253879+0000","last_fullsized":"2023-08-09T13:51:16.253879+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:51:15.188798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:51:15.188798+0000","last_clean_scrub_stamp":"2023-08-09T13:51:15.188798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T20:02:29.796576+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.a","version":"0'0","reported_seq":11,"reported_epoch":102,"state":"active+clean","last_fresh":"2023-08-09T13:51:16.240880+0000","last_change":"2023-08-09T13:51:15.236061+0000","last_active":"2023-08-09T13:51:16.240880+0000","last_peered":"2023-08-09T13:51:16.240880+0000","last_clean":"2023-08-09T13:51:16.240880+0000","last_became_active":"2023-08-09T13:51:15.235777+0000","last_became_peered":"2023-08-09T13:51:15.235777+0000","last_unstale":"2023-08-09T13:51:16.240880+0000","last_undegraded":"2023-08-09T13:51:16.240880+0000","last_fullsized":"2023-08-09T13:51:16.240880+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:51:14.171648+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:51:14.171648+0000","last_clean_scrub_stamp":"2023-08-09T13:51:14.171648+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T20:17:36.716867+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":166,"reported_epoch":102,"state":"active+clean","last_fresh":"2023-08-09T13:51:16.238987+0000","last_change":"2023-08-09T13:33:55.886389+0000","last_active":"2023-08-09T13:51:16.238987+0000","last_peered":"2023-08-09T13:51:16.238987+0000","last_clean":"2023-08-09T13:51:16.238987+0000","last_became_active":"2023-08-09T13:33:55.885638+0000","last_became_peered":"2023-08-09T13:33:55.885638+0000","last_unstale":"2023-08-09T13:51:16.238987+0000","last_undegraded":"2023-08-09T13:51:16.238987+0000","last_fullsized":"2023-08-09T13:51:16.238987+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:33:54.845869+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:33:54.845869+0000","last_clean_scrub_stamp":"2023-08-09T13:33:54.845869+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T22:59:40.671316+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]}],"pool_stats":[{"poolid":13,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":12,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":38,"ondisk_log_size":38,"up":64,"acting":64,"num_store_stats":8},{"poolid":2,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":619,"num_read_kb":1212,"num_write":263,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":175,"ondisk_log_size":175,"up":2,"acting":2,"num_store_stats":2},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":8}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834575056,"num_pgs":16,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":37000,"kb_used_data":1788,"kb_used_omap":1,"kb_used_meta":35198,"kb_avail":94334840,"statfs":{"total":96636764160,"available":96598876160,"internally_reserved":0,"allocated":1830912,"data_stored":1249007,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1781,"internal_metadata":36043019},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Aug 9 13:50:16 2023","interfaces":[{"interface":"back","average":{"1min":1.181,"5min":1.222,"15min":1.232},"min":{"1min":0.69499999999999995,"5min":0.63400000000000001,"15min":0.61099999999999999},"max":{"1min":1.633,"5min":1.8919999999999999,"15min":2.1280000000000001},"last":0.84499999999999997},{"interface":"front","average":{"1min":1.1819999999999999,"5min":1.232,"15min":1.24},"min":{"1min":0.72399999999999998,"5min":0.72399999999999998,"15min":0.65000000000000002},"max":{"1min":1.6100000000000001,"5min":1.839,"15min":2.0099999999999998},"last":1.022}]},{"osd":1,"last update":"Wed Aug 9 13:50:16 2023","interfaces":[{"interface":"back","average":{"1min":1.2010000000000001,"5min":1.2450000000000001,"15min":1.256},"min":{"1min":0.72799999999999998,"5min":0.72799999999999998,"15min":0.59799999999999998},"max":{"1min":1.8,"5min":1.8,"15min":1.9470000000000001},"last":1.105},{"interface":"front","average":{"1min":1.2569999999999999,"5min":1.248,"15min":1.2869999999999999},"min":{"1min":0.83099999999999996,"5min":0.59299999999999997,"15min":0.59299999999999997},"max":{"1min":1.726,"5min":1.9410000000000001,"15min":1.9410000000000001},"last":1.2789999999999999}]},{"osd":2,"last update":"Wed Aug 9 13:50:16 2023","interfaces":[{"interface":"back","average":{"1min":1.278,"5min":1.2470000000000001,"15min":1.2430000000000001},"min":{"1min":0.81000000000000005,"5min":0.65700000000000003,"15min":0.65700000000000003},"max":{"1min":1.742,"5min":2.0129999999999999,"15min":2.0129999999999999},"last":1.4810000000000001},{"interface":"front","average":{"1min":1.2889999999999999,"5min":1.323,"15min":1.3759999999999999},"min":{"1min":0.92600000000000005,"5min":0.80900000000000005,"15min":0.67800000000000005},"max":{"1min":1.7130000000000001,"5min":1.8740000000000001,"15min":2.4140000000000001},"last":0.96199999999999997}]},{"osd":3,"last update":"Wed Aug 9 13:50:16 2023","interfaces":[{"interface":"back","average":{"1min":1.2529999999999999,"5min":1.3180000000000001,"15min":1.331},"min":{"1min":0.98799999999999999,"5min":0.86799999999999999,"15min":0.77500000000000002},"max":{"1min":1.7809999999999999,"5min":1.97,"15min":1.97},"last":1.3149999999999999},{"interface":"front","average":{"1min":1.3740000000000001,"5min":1.419,"15min":1.4159999999999999},"min":{"1min":0.93899999999999995,"5min":0.88300000000000001,"15min":0.82099999999999995},"max":{"1min":1.8160000000000001,"5min":2.0790000000000002,"15min":2.2480000000000002},"last":1.224}]},{"osd":4,"last update":"Wed Aug 9 13:50:16 2023","interfaces":[{"interface":"back","average":{"1min":1.343,"5min":1.2649999999999999,"15min":1.2769999999999999},"min":{"1min":0.86399999999999999,"5min":0.59499999999999997,"15min":0.59499999999999997},"max":{"1min":1.772,"5min":2.032,"15min":2.343},"last":1.466},{"interface":"front","average":{"1min":1.296,"5min":1.3160000000000001,"15min":1.339},"min":{"1min":0.93600000000000005,"5min":0.77100000000000002,"15min":0.56799999999999995},"max":{"1min":1.6739999999999999,"5min":2.1419999999999999,"15min":2.1419999999999999},"last":1.585}]},{"osd":5,"last update":"Wed Aug 9 13:50:16 2023","interfaces":[{"interface":"back","average":{"1min":1.4430000000000001,"5min":1.4350000000000001,"15min":1.419},"min":{"1min":1.048,"5min":0.88600000000000001,"15min":0.71499999999999997},"max":{"1min":1.9039999999999999,"5min":2.1480000000000001,"15min":2.6749999999999998},"last":1.3859999999999999},{"interface":"front","average":{"1min":1.4810000000000001,"5min":1.4319999999999999,"15min":1.4510000000000001},"min":{"1min":0.95999999999999996,"5min":0.85699999999999998,"15min":0.69199999999999995},"max":{"1min":1.877,"5min":2.113,"15min":2.9569999999999999},"last":1.1719999999999999}]},{"osd":6,"last update":"Wed Aug 9 13:50:16 2023","interfaces":[{"interface":"back","average":{"1min":1.6020000000000001,"5min":1.518,"15min":1.5489999999999999},"min":{"1min":1.2,"5min":0.85499999999999998,"15min":0.85499999999999998},"max":{"1min":1.8879999999999999,"5min":2.2069999999999999,"15min":2.8149999999999999},"last":1.367},{"interface":"front","average":{"1min":1.4139999999999999,"5min":1.4570000000000001,"15min":1.474},"min":{"1min":1.111,"5min":0.90600000000000003,"15min":0.60799999999999998},"max":{"1min":1.9419999999999999,"5min":2.1320000000000001,"15min":2.2530000000000001},"last":1.663}]}]},{"osd":6,"up_from":13,"seq":55834575056,"num_pgs":11,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":36420,"kb_used_data":1204,"kb_used_omap":1,"kb_used_meta":35198,"kb_avail":94335420,"statfs":{"total":96636764160,"available":96599470080,"internally_reserved":0,"allocated":1232896,"data_stored":657278,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1781,"internal_metadata":36043019},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Aug 9 13:50:20 2023","interfaces":[{"interface":"back","average":{"1min":1.1160000000000001,"5min":1.143,"15min":1.161},"min":{"1min":0.75,"5min":0.72499999999999998,"15min":0.56599999999999995},"max":{"1min":1.5489999999999999,"5min":1.633,"15min":1.9870000000000001},"last":0.65800000000000003},{"interface":"front","average":{"1min":1.21,"5min":1.532,"15min":1.3080000000000001},"min":{"1min":0.79300000000000004,"5min":0.79300000000000004,"15min":0.76000000000000001},"max":{"1min":1.6519999999999999,"5min":28.036000000000001,"15min":28.036000000000001},"last":1.0049999999999999}]},{"osd":1,"last update":"Wed Aug 9 13:50:20 2023","interfaces":[{"interface":"back","average":{"1min":1.147,"5min":1.272,"15min":1.226},"min":{"1min":0.70399999999999996,"5min":0.70399999999999996,"15min":0.65500000000000003},"max":{"1min":1.659,"5min":7.742,"15min":7.742},"last":0.79700000000000004},{"interface":"front","average":{"1min":1.252,"5min":1.2849999999999999,"15min":1.276},"min":{"1min":0.84199999999999997,"5min":0.84199999999999997,"15min":0.68100000000000005},"max":{"1min":1.8899999999999999,"5min":1.8899999999999999,"15min":1.8899999999999999},"last":1.3129999999999999}]},{"osd":2,"last update":"Wed Aug 9 13:50:20 2023","interfaces":[{"interface":"back","average":{"1min":1.2989999999999999,"5min":1.3480000000000001,"15min":1.3220000000000001},"min":{"1min":0.877,"5min":0.81999999999999995,"15min":0.68700000000000006},"max":{"1min":1.73,"5min":4.0640000000000001,"15min":4.0640000000000001},"last":1.0620000000000001},{"interface":"front","average":{"1min":1.244,"5min":1.4430000000000001,"15min":1.3460000000000001},"min":{"1min":0.76800000000000002,"5min":0.70999999999999996,"15min":0.70999999999999996},"max":{"1min":2.0390000000000001,"5min":13.154999999999999,"15min":13.154999999999999},"last":1.1160000000000001}]},{"osd":3,"last update":"Wed Aug 9 13:50:20 2023","interfaces":[{"interface":"back","average":{"1min":1.3939999999999999,"5min":1.52,"15min":1.419},"min":{"1min":0.73099999999999998,"5min":0.73099999999999998,"15min":0.73099999999999998},"max":{"1min":2.2519999999999998,"5min":13.349,"15min":13.349},"last":0.879},{"interface":"front","average":{"1min":1.258,"5min":1.3620000000000001,"15min":1.325},"min":{"1min":0.88400000000000001,"5min":0.76400000000000001,"15min":0.74299999999999999},"max":{"1min":1.8560000000000001,"5min":7.9029999999999996,"15min":7.9029999999999996},"last":0.94299999999999995}]},{"osd":4,"last update":"Wed Aug 9 13:50:20 2023","interfaces":[{"interface":"back","average":{"1min":1.302,"5min":1.335,"15min":1.28},"min":{"1min":0.90700000000000003,"5min":0.78500000000000003,"15min":0.64300000000000002},"max":{"1min":1.992,"5min":5.2290000000000001,"15min":5.2290000000000001},"last":1.4570000000000001},{"interface":"front","average":{"1min":1.4019999999999999,"5min":1.498,"15min":1.389},"min":{"1min":0.88900000000000001,"5min":0.78700000000000003,"15min":0.69099999999999995},"max":{"1min":1.734,"5min":12.1,"15min":12.1},"last":1.2669999999999999}]},{"osd":5,"last update":"Wed Aug 9 13:50:20 2023","interfaces":[{"interface":"back","average":{"1min":1.3169999999999999,"5min":1.4399999999999999,"15min":1.357},"min":{"1min":0.93999999999999995,"5min":0.83799999999999997,"15min":0.65400000000000003},"max":{"1min":2.1139999999999999,"5min":8.1899999999999995,"15min":8.1899999999999995},"last":1.395},{"interface":"front","average":{"1min":1.5649999999999999,"5min":1.6639999999999999,"15min":1.589},"min":{"1min":1.2010000000000001,"5min":1.2010000000000001,"15min":0.83799999999999997},"max":{"1min":2.2000000000000002,"5min":8.8889999999999993,"15min":8.8889999999999993},"last":1.552}]},{"osd":7,"last update":"Wed Aug 9 13:50:20 2023","interfaces":[{"interface":"back","average":{"1min":1.577,"5min":1.6970000000000001,"15min":1.5880000000000001},"min":{"1min":1.1850000000000001,"5min":1.0029999999999999,"15min":0.62},"max":{"1min":1.8240000000000001,"5min":9.3320000000000007,"15min":9.3320000000000007},"last":1.363},{"interface":"front","average":{"1min":1.4410000000000001,"5min":1.6060000000000001,"15min":1.548},"min":{"1min":0.93899999999999995,"5min":0.88500000000000001,"15min":0.88500000000000001},"max":{"1min":1.9470000000000001,"5min":8.0399999999999991,"15min":8.0399999999999991},"last":1.1819999999999999}]}]},{"osd":5,"up_from":13,"seq":55834575056,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":40548,"kb_used_data":1236,"kb_used_omap":1,"kb_used_meta":39294,"kb_avail":94331292,"statfs":{"total":96636764160,"available":96595243008,"internally_reserved":0,"allocated":1265664,"data_stored":667765,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1781,"internal_metadata":40237323},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Aug 9 13:50:19 2023","interfaces":[{"interface":"back","average":{"1min":1.2509999999999999,"5min":1.1759999999999999,"15min":1.179},"min":{"1min":0.63500000000000001,"5min":0.63500000000000001,"15min":0.63500000000000001},"max":{"1min":1.518,"5min":2.1869999999999998,"15min":2.1869999999999998},"last":1.337},{"interface":"front","average":{"1min":1.1719999999999999,"5min":1.2,"15min":1.198},"min":{"1min":0.68600000000000005,"5min":0.68600000000000005,"15min":0.64200000000000002},"max":{"1min":1.579,"5min":1.6619999999999999,"15min":1.7989999999999999},"last":1.2450000000000001}]},{"osd":1,"last update":"Wed Aug 9 13:50:19 2023","interfaces":[{"interface":"back","average":{"1min":1.165,"5min":1.1990000000000001,"15min":1.2190000000000001},"min":{"1min":0.81599999999999995,"5min":0.73099999999999998,"15min":0.64100000000000001},"max":{"1min":1.631,"5min":1.6739999999999999,"15min":1.736},"last":0.75700000000000001},{"interface":"front","average":{"1min":1.264,"5min":1.296,"15min":1.2769999999999999},"min":{"1min":0.93300000000000005,"5min":0.66900000000000004,"15min":0.66900000000000004},"max":{"1min":1.6699999999999999,"5min":1.887,"15min":1.887},"last":1.694}]},{"osd":2,"last update":"Wed Aug 9 13:50:19 2023","interfaces":[{"interface":"back","average":{"1min":1.2410000000000001,"5min":1.2569999999999999,"15min":1.2849999999999999},"min":{"1min":0.98199999999999998,"5min":0.68999999999999995,"15min":0.61599999999999999},"max":{"1min":1.5760000000000001,"5min":1.756,"15min":1.899},"last":0.81799999999999995},{"interface":"front","average":{"1min":1.3049999999999999,"5min":1.302,"15min":1.3149999999999999},"min":{"1min":0.78400000000000003,"5min":0.78400000000000003,"15min":0.66800000000000004},"max":{"1min":1.6599999999999999,"5min":1.7190000000000001,"15min":1.7969999999999999},"last":1.1719999999999999}]},{"osd":3,"last update":"Wed Aug 9 13:50:19 2023","interfaces":[{"interface":"back","average":{"1min":1.365,"5min":1.3720000000000001,"15min":1.3600000000000001},"min":{"1min":0.95399999999999996,"5min":0.95399999999999996,"15min":0.59799999999999998},"max":{"1min":1.6890000000000001,"5min":2.1840000000000002,"15min":2.2690000000000001},"last":1.534},{"interface":"front","average":{"1min":1.4259999999999999,"5min":1.4379999999999999,"15min":1.4139999999999999},"min":{"1min":1.087,"5min":0.94899999999999995,"15min":0.70999999999999996},"max":{"1min":1.73,"5min":2.3780000000000001,"15min":2.3780000000000001},"last":1.0629999999999999}]},{"osd":4,"last update":"Wed Aug 9 13:50:19 2023","interfaces":[{"interface":"back","average":{"1min":1.262,"5min":1.2729999999999999,"15min":1.2889999999999999},"min":{"1min":0.92700000000000005,"5min":0.84699999999999998,"15min":0.56599999999999995},"max":{"1min":1.673,"5min":2.0800000000000001,"15min":2.121},"last":1.2929999999999999},{"interface":"front","average":{"1min":1.3919999999999999,"5min":1.3400000000000001,"15min":1.3779999999999999},"min":{"1min":0.81999999999999995,"5min":0.76100000000000001,"15min":0.58599999999999997},"max":{"1min":1.944,"5min":2.1230000000000002,"15min":2.3679999999999999},"last":1.613}]},{"osd":6,"last update":"Wed Aug 9 13:50:19 2023","interfaces":[{"interface":"back","average":{"1min":1.4299999999999999,"5min":1.3520000000000001,"15min":1.391},"min":{"1min":0.99099999999999999,"5min":0.90300000000000002,"15min":0.66700000000000004},"max":{"1min":1.9810000000000001,"5min":2.306,"15min":2.306},"last":1.4410000000000001},{"interface":"front","average":{"1min":1.49,"5min":1.4399999999999999,"15min":1.4710000000000001},"min":{"1min":1.1220000000000001,"5min":1.034,"15min":0.71199999999999997},"max":{"1min":1.827,"5min":2.2320000000000002,"15min":2.2320000000000002},"last":1.49}]},{"osd":7,"last update":"Wed Aug 9 13:50:19 2023","interfaces":[{"interface":"back","average":{"1min":1.577,"5min":1.5580000000000001,"15min":1.5529999999999999},"min":{"1min":1.1160000000000001,"5min":1.0589999999999999,"15min":0.70899999999999996},"max":{"1min":1.9610000000000001,"5min":2.452,"15min":2.4729999999999999},"last":1.7509999999999999},{"interface":"front","average":{"1min":1.611,"5min":1.585,"15min":1.6080000000000001},"min":{"1min":1.2110000000000001,"5min":1.163,"15min":0.77600000000000002},"max":{"1min":2.097,"5min":2.1070000000000002,"15min":2.3820000000000001},"last":1.6619999999999999}]}]},{"osd":4,"up_from":13,"seq":55834575056,"num_pgs":6,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32328,"kb_used_data":1208,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":94339512,"statfs":{"total":96636764160,"available":96603660288,"internally_reserved":0,"allocated":1236992,"data_stored":658639,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1781,"internal_metadata":31848715},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Aug 9 13:50:21 2023","interfaces":[{"interface":"back","average":{"1min":1.2330000000000001,"5min":1.2390000000000001,"15min":1.2030000000000001},"min":{"1min":0.875,"5min":0.70699999999999996,"15min":0.67700000000000005},"max":{"1min":1.5940000000000001,"5min":2.524,"15min":2.524},"last":0.68799999999999994},{"interface":"front","average":{"1min":1.206,"5min":1.236,"15min":1.2569999999999999},"min":{"1min":0.62,"5min":0.62,"15min":0.62},"max":{"1min":1.5980000000000001,"5min":1.863,"15min":1.863},"last":0.80200000000000005}]},{"osd":1,"last update":"Wed Aug 9 13:50:21 2023","interfaces":[{"interface":"back","average":{"1min":1.244,"5min":1.3169999999999999,"15min":1.2929999999999999},"min":{"1min":0.70999999999999996,"5min":0.67800000000000005,"15min":0.67800000000000005},"max":{"1min":1.48,"5min":1.8480000000000001,"15min":1.8480000000000001},"last":1.1890000000000001},{"interface":"front","average":{"1min":1.242,"5min":1.2649999999999999,"15min":1.248},"min":{"1min":0.81799999999999995,"5min":0.81799999999999995,"15min":0.64900000000000002},"max":{"1min":1.9330000000000001,"5min":1.9330000000000001,"15min":1.9330000000000001},"last":0.75}]},{"osd":2,"last update":"Wed Aug 9 13:50:21 2023","interfaces":[{"interface":"back","average":{"1min":1.306,"5min":1.2869999999999999,"15min":1.2809999999999999},"min":{"1min":1.022,"5min":0.621,"15min":0.621},"max":{"1min":1.619,"5min":2.6429999999999998,"15min":2.6429999999999998},"last":0.96599999999999997},{"interface":"front","average":{"1min":1.343,"5min":1.3720000000000001,"15min":1.3660000000000001},"min":{"1min":0.88300000000000001,"5min":0.88300000000000001,"15min":0.74299999999999999},"max":{"1min":1.7569999999999999,"5min":1.7569999999999999,"15min":1.8149999999999999},"last":1.3460000000000001}]},{"osd":3,"last update":"Wed Aug 9 13:50:21 2023","interfaces":[{"interface":"back","average":{"1min":1.377,"5min":1.3300000000000001,"15min":1.298},"min":{"1min":1.083,"5min":0.86399999999999999,"15min":0.84699999999999998},"max":{"1min":1.6739999999999999,"5min":2.6099999999999999,"15min":2.6099999999999999},"last":0.86499999999999999},{"interface":"front","average":{"1min":1.4099999999999999,"5min":1.3979999999999999,"15min":1.3799999999999999},"min":{"1min":0.94799999999999995,"5min":0.88800000000000001,"15min":0.63100000000000001},"max":{"1min":1.6890000000000001,"5min":2.7629999999999999,"15min":2.7629999999999999},"last":1.2709999999999999}]},{"osd":5,"last update":"Wed Aug 9 13:50:21 2023","interfaces":[{"interface":"back","average":{"1min":1.288,"5min":1.3400000000000001,"15min":1.3300000000000001},"min":{"1min":0.876,"5min":0.73199999999999998,"15min":0.63200000000000001},"max":{"1min":1.734,"5min":1.8,"15min":2.0099999999999998},"last":1.4219999999999999},{"interface":"front","average":{"1min":1.2509999999999999,"5min":1.323,"15min":1.3180000000000001},"min":{"1min":0.92000000000000004,"5min":0.72399999999999998,"15min":0.626},"max":{"1min":1.71,"5min":2.7090000000000001,"15min":2.7090000000000001},"last":1.0269999999999999}]},{"osd":6,"last update":"Wed Aug 9 13:50:21 2023","interfaces":[{"interface":"back","average":{"1min":1.379,"5min":1.3160000000000001,"15min":1.3360000000000001},"min":{"1min":0.94699999999999995,"5min":0.74399999999999999,"15min":0.59099999999999997},"max":{"1min":1.7509999999999999,"5min":2.8559999999999999,"15min":2.8559999999999999},"last":1.3720000000000001},{"interface":"front","average":{"1min":1.4430000000000001,"5min":1.518,"15min":1.504},"min":{"1min":1.0760000000000001,"5min":0.97799999999999998,"15min":0.76400000000000001},"max":{"1min":1.669,"5min":2.2719999999999998,"15min":2.2719999999999998},"last":1.4850000000000001}]},{"osd":7,"last update":"Wed Aug 9 13:50:21 2023","interfaces":[{"interface":"back","average":{"1min":1.46,"5min":1.4830000000000001,"15min":1.474},"min":{"1min":1.1499999999999999,"5min":0.85999999999999999,"15min":0.73899999999999999},"max":{"1min":1.8460000000000001,"5min":2.8050000000000002,"15min":2.8050000000000002},"last":1.1200000000000001},{"interface":"front","average":{"1min":1.474,"5min":1.522,"15min":1.514},"min":{"1min":1.236,"5min":0.97199999999999998,"15min":0.72899999999999998},"max":{"1min":1.754,"5min":3.415,"15min":3.415},"last":1.4570000000000001}]}]},{"osd":3,"up_from":13,"seq":55834575057,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":36424,"kb_used_data":1208,"kb_used_omap":1,"kb_used_meta":35198,"kb_avail":94335416,"statfs":{"total":96636764160,"available":96599465984,"internally_reserved":0,"allocated":1236992,"data_stored":658639,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1781,"internal_metadata":36043019},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Aug 9 13:50:25 2023","interfaces":[{"interface":"back","average":{"1min":0.84599999999999997,"5min":0.88300000000000001,"15min":0.91600000000000004},"min":{"1min":0.43099999999999999,"5min":0.27200000000000002,"15min":0.27200000000000002},"max":{"1min":1.3939999999999999,"5min":1.8500000000000001,"15min":1.968},"last":0.72599999999999998},{"interface":"front","average":{"1min":1.071,"5min":1.0920000000000001,"15min":1.1060000000000001},"min":{"1min":0.48199999999999998,"5min":0.36399999999999999,"15min":0.36399999999999999},"max":{"1min":1.6910000000000001,"5min":1.6910000000000001,"15min":2.2360000000000002},"last":0.96099999999999997}]},{"osd":1,"last update":"Wed Aug 9 13:50:25 2023","interfaces":[{"interface":"back","average":{"1min":1.042,"5min":1.0089999999999999,"15min":1.012},"min":{"1min":0.59899999999999998,"5min":0.496,"15min":0.438},"max":{"1min":1.518,"5min":1.7110000000000001,"15min":2.5649999999999999},"last":1.171},{"interface":"front","average":{"1min":1.091,"5min":1.1279999999999999,"15min":1.1639999999999999},"min":{"1min":0.51800000000000002,"5min":0.51800000000000002,"15min":0.42099999999999999},"max":{"1min":1.6839999999999999,"5min":1.6839999999999999,"15min":2.6760000000000002},"last":1.024}]},{"osd":2,"last update":"Wed Aug 9 13:50:25 2023","interfaces":[{"interface":"back","average":{"1min":1.121,"5min":1.177,"15min":1.1930000000000001},"min":{"1min":0.53600000000000003,"5min":0.53600000000000003,"15min":0.53500000000000003},"max":{"1min":1.6240000000000001,"5min":1.913,"15min":1.923},"last":0.89300000000000002},{"interface":"front","average":{"1min":1.1899999999999999,"5min":1.2929999999999999,"15min":1.3080000000000001},"min":{"1min":0.625,"5min":0.52300000000000002,"15min":0.52000000000000002},"max":{"1min":1.9399999999999999,"5min":2.0259999999999998,"15min":2.0259999999999998},"last":1.2909999999999999}]},{"osd":4,"last update":"Wed Aug 9 13:50:25 2023","interfaces":[{"interface":"back","average":{"1min":1.232,"5min":1.258,"15min":1.282},"min":{"1min":0.91500000000000004,"5min":0.57899999999999996,"15min":0.57899999999999996},"max":{"1min":1.742,"5min":1.8160000000000001,"15min":1.8160000000000001},"last":1.242},{"interface":"front","average":{"1min":1.349,"5min":1.397,"15min":1.3919999999999999},"min":{"1min":0.67600000000000005,"5min":0.628,"15min":0.628},"max":{"1min":1.8149999999999999,"5min":2.2040000000000002,"15min":2.2370000000000001},"last":2.0960000000000001}]},{"osd":5,"last update":"Wed Aug 9 13:50:25 2023","interfaces":[{"interface":"back","average":{"1min":1.456,"5min":1.468,"15min":1.474},"min":{"1min":0.67600000000000005,"5min":0.67600000000000005,"15min":0.67200000000000004},"max":{"1min":1.855,"5min":2.254,"15min":2.254},"last":1.833},{"interface":"front","average":{"1min":1.4850000000000001,"5min":1.4910000000000001,"15min":1.49},"min":{"1min":0.77600000000000002,"5min":0.65900000000000003,"15min":0.65900000000000003},"max":{"1min":2.3069999999999999,"5min":2.3069999999999999,"15min":2.3069999999999999},"last":1.387}]},{"osd":6,"last update":"Wed Aug 9 13:50:25 2023","interfaces":[{"interface":"back","average":{"1min":1.3759999999999999,"5min":1.4370000000000001,"15min":1.4590000000000001},"min":{"1min":0.878,"5min":0.68200000000000005,"15min":0.68200000000000005},"max":{"1min":1.8759999999999999,"5min":2.1829999999999998,"15min":2.242},"last":1.631},{"interface":"front","average":{"1min":1.466,"5min":1.5580000000000001,"15min":1.5529999999999999},"min":{"1min":1.0600000000000001,"5min":0.71099999999999997,"15min":0.68600000000000005},"max":{"1min":1.825,"5min":2.3420000000000001,"15min":2.3420000000000001},"last":1.966}]},{"osd":7,"last update":"Wed Aug 9 13:50:25 2023","interfaces":[{"interface":"back","average":{"1min":1.601,"5min":1.627,"15min":1.6160000000000001},"min":{"1min":1.115,"5min":0.69399999999999995,"15min":0.69399999999999995},"max":{"1min":2.46,"5min":2.46,"15min":2.46},"last":1.4610000000000001},{"interface":"front","average":{"1min":1.4910000000000001,"5min":1.546,"15min":1.5600000000000001},"min":{"1min":0.78200000000000003,"5min":0.64200000000000002,"15min":0.61599999999999999},"max":{"1min":2.0579999999999998,"5min":2.359,"15min":2.484},"last":1.552}]}]},{"osd":2,"up_from":13,"seq":55834575056,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":36424,"kb_used_data":1208,"kb_used_omap":1,"kb_used_meta":35198,"kb_avail":94335416,"statfs":{"total":96636764160,"available":96599465984,"internally_reserved":0,"allocated":1236992,"data_stored":658639,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1781,"internal_metadata":36043019},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Aug 9 13:50:17 2023","interfaces":[{"interface":"back","average":{"1min":0.96999999999999997,"5min":0.95299999999999996,"15min":0.95499999999999996},"min":{"1min":0.47799999999999998,"5min":0.36499999999999999,"15min":0.36499999999999999},"max":{"1min":1.5760000000000001,"5min":1.6180000000000001,"15min":1.8380000000000001},"last":0.626},{"interface":"front","average":{"1min":0.97099999999999997,"5min":0.998,"15min":1.0049999999999999},"min":{"1min":0.45300000000000001,"5min":0.374,"15min":0.374},"max":{"1min":1.4199999999999999,"5min":2.3929999999999998,"15min":2.3929999999999998},"last":0.79400000000000004}]},{"osd":1,"last update":"Wed Aug 9 13:50:17 2023","interfaces":[{"interface":"back","average":{"1min":1.2529999999999999,"5min":1.1739999999999999,"15min":1.1870000000000001},"min":{"1min":0.61299999999999999,"5min":0.55200000000000005,"15min":0.55200000000000005},"max":{"1min":1.6870000000000001,"5min":1.77,"15min":1.8420000000000001},"last":1.321},{"interface":"front","average":{"1min":1.216,"5min":1.2749999999999999,"15min":1.335},"min":{"1min":0.52500000000000002,"5min":0.52500000000000002,"15min":0.52500000000000002},"max":{"1min":1.7390000000000001,"5min":2.4870000000000001,"15min":2.4870000000000001},"last":1.659}]},{"osd":3,"last update":"Wed Aug 9 13:50:17 2023","interfaces":[{"interface":"back","average":{"1min":0.98099999999999998,"5min":1.006,"15min":1.0469999999999999},"min":{"1min":0.44800000000000001,"5min":0.38900000000000001,"15min":0.38900000000000001},"max":{"1min":1.3109999999999999,"5min":1.708,"15min":2.0409999999999999},"last":1.244},{"interface":"front","average":{"1min":1.2110000000000001,"5min":1.304,"15min":1.3089999999999999},"min":{"1min":0.55500000000000005,"5min":0.46200000000000002,"15min":0.46200000000000002},"max":{"1min":1.8009999999999999,"5min":8.8130000000000006,"15min":8.8130000000000006},"last":0.873}]},{"osd":4,"last update":"Wed Aug 9 13:50:17 2023","interfaces":[{"interface":"back","average":{"1min":1.415,"5min":1.4179999999999999,"15min":1.4510000000000001},"min":{"1min":0.83999999999999997,"5min":0.628,"15min":0.628},"max":{"1min":1.958,"5min":2.089,"15min":2.089},"last":1.488},{"interface":"front","average":{"1min":1.462,"5min":1.359,"15min":1.3979999999999999},"min":{"1min":0.73799999999999999,"5min":0.68600000000000005,"15min":0.68600000000000005},"max":{"1min":2.472,"5min":2.472,"15min":2.472},"last":1.7509999999999999}]},{"osd":5,"last update":"Wed Aug 9 13:50:17 2023","interfaces":[{"interface":"back","average":{"1min":1.5189999999999999,"5min":1.4379999999999999,"15min":1.458},"min":{"1min":0.76600000000000001,"5min":0.59999999999999998,"15min":0.59999999999999998},"max":{"1min":2.27,"5min":2.5840000000000001,"15min":2.5840000000000001},"last":1.54},{"interface":"front","average":{"1min":1.304,"5min":1.3540000000000001,"15min":1.349},"min":{"1min":0.77100000000000002,"5min":0.56799999999999995,"15min":0.56799999999999995},"max":{"1min":1.8859999999999999,"5min":3.5659999999999998,"15min":3.5659999999999998},"last":1.4590000000000001}]},{"osd":6,"last update":"Wed Aug 9 13:50:17 2023","interfaces":[{"interface":"back","average":{"1min":1.4019999999999999,"5min":1.425,"15min":1.4259999999999999},"min":{"1min":0.79800000000000004,"5min":0.73799999999999999,"15min":0.73799999999999999},"max":{"1min":2.222,"5min":3.5099999999999998,"15min":3.5099999999999998},"last":1.3839999999999999},{"interface":"front","average":{"1min":1.4790000000000001,"5min":1.4990000000000001,"15min":1.5409999999999999},"min":{"1min":0.82199999999999995,"5min":0.66300000000000003,"15min":0.66300000000000003},"max":{"1min":2.3769999999999998,"5min":2.3769999999999998,"15min":2.3769999999999998},"last":1.589}]},{"osd":7,"last update":"Wed Aug 9 13:50:17 2023","interfaces":[{"interface":"back","average":{"1min":1.54,"5min":1.51,"15min":1.548},"min":{"1min":0.78800000000000003,"5min":0.64600000000000002,"15min":0.64600000000000002},"max":{"1min":2.1970000000000001,"5min":2.5249999999999999,"15min":2.5249999999999999},"last":1.4099999999999999},{"interface":"front","average":{"1min":1.4970000000000001,"5min":1.52,"15min":1.51},"min":{"1min":0.99199999999999999,"5min":0.70499999999999996,"15min":0.70499999999999996},"max":{"1min":2.089,"5min":3.609,"15min":3.609},"last":1.5149999999999999}]}]},{"osd":1,"up_from":13,"seq":55834575056,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":46860,"kb_used_data":1788,"kb_used_omap":1,"kb_used_meta":45054,"kb_avail":94324980,"statfs":{"total":96636764160,"available":96588779520,"internally_reserved":0,"allocated":1830912,"data_stored":1249007,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1781,"internal_metadata":46135563},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Aug 9 13:50:31 2023","interfaces":[{"interface":"back","average":{"1min":0.99099999999999999,"5min":0.92000000000000004,"15min":0.93999999999999995},"min":{"1min":0.50600000000000001,"5min":0.50600000000000001,"15min":0.32400000000000001},"max":{"1min":1.7110000000000001,"5min":1.7110000000000001,"15min":2.1080000000000001},"last":0.88800000000000001},{"interface":"front","average":{"1min":0.98999999999999999,"5min":1.0740000000000001,"15min":1.095},"min":{"1min":0.59799999999999998,"5min":0.432,"15min":0.42499999999999999},"max":{"1min":1.7190000000000001,"5min":2.468,"15min":2.468},"last":1.6519999999999999}]},{"osd":2,"last update":"Wed Aug 9 13:50:31 2023","interfaces":[{"interface":"back","average":{"1min":1.008,"5min":0.97799999999999998,"15min":0.995},"min":{"1min":0.56699999999999995,"5min":0.41099999999999998,"15min":0.41099999999999998},"max":{"1min":1.6679999999999999,"5min":1.907,"15min":1.907},"last":0.78200000000000003},{"interface":"front","average":{"1min":1.1459999999999999,"5min":1.159,"15min":1.1879999999999999},"min":{"1min":0.80600000000000005,"5min":0.54400000000000004,"15min":0.46500000000000002},"max":{"1min":1.5189999999999999,"5min":2.1280000000000001,"15min":2.145},"last":1.149}]},{"osd":3,"last update":"Wed Aug 9 13:50:31 2023","interfaces":[{"interface":"back","average":{"1min":1.218,"5min":1.2629999999999999,"15min":1.3260000000000001},"min":{"1min":0.63200000000000001,"5min":0.54400000000000004,"15min":0.432},"max":{"1min":1.7430000000000001,"5min":2.0270000000000001,"15min":2.3210000000000002},"last":1.3580000000000001},{"interface":"front","average":{"1min":1.1779999999999999,"5min":1.234,"15min":1.2450000000000001},"min":{"1min":0.70399999999999996,"5min":0.49299999999999999,"15min":0.44400000000000001},"max":{"1min":1.786,"5min":2.3540000000000001,"15min":2.7669999999999999},"last":0.94599999999999995}]},{"osd":4,"last update":"Wed Aug 9 13:50:31 2023","interfaces":[{"interface":"back","average":{"1min":1.3149999999999999,"5min":1.399,"15min":1.4079999999999999},"min":{"1min":0.94399999999999995,"5min":0.79800000000000004,"15min":0.58099999999999996},"max":{"1min":1.696,"5min":1.97,"15min":2.5},"last":1.214},{"interface":"front","average":{"1min":1.2689999999999999,"5min":1.3180000000000001,"15min":1.333},"min":{"1min":0.88200000000000001,"5min":0.70099999999999996,"15min":0.70099999999999996},"max":{"1min":1.6950000000000001,"5min":1.901,"15min":1.901},"last":1.2629999999999999}]},{"osd":5,"last update":"Wed Aug 9 13:50:31 2023","interfaces":[{"interface":"back","average":{"1min":1.3200000000000001,"5min":1.323,"15min":1.371},"min":{"1min":0.90700000000000003,"5min":0.76900000000000002,"15min":0.76900000000000002},"max":{"1min":1.837,"5min":2.294,"15min":2.294},"last":1.474},{"interface":"front","average":{"1min":1.427,"5min":1.464,"15min":1.516},"min":{"1min":0.97899999999999998,"5min":0.91000000000000003,"15min":0.77700000000000002},"max":{"1min":1.837,"5min":2.1389999999999998,"15min":2.4649999999999999},"last":1.502}]},{"osd":6,"last update":"Wed Aug 9 13:50:31 2023","interfaces":[{"interface":"back","average":{"1min":1.4219999999999999,"5min":1.48,"15min":1.514},"min":{"1min":1.026,"5min":0.77500000000000002,"15min":0.67400000000000004},"max":{"1min":1.925,"5min":2.2080000000000002,"15min":2.5270000000000001},"last":1.5780000000000001},{"interface":"front","average":{"1min":1.4370000000000001,"5min":1.45,"15min":1.5},"min":{"1min":1.1379999999999999,"5min":0.82499999999999996,"15min":0.79800000000000004},"max":{"1min":2.02,"5min":2.02,"15min":2.1040000000000001},"last":1.5489999999999999}]},{"osd":7,"last update":"Wed Aug 9 13:50:31 2023","interfaces":[{"interface":"back","average":{"1min":1.552,"5min":1.5649999999999999,"15min":1.619},"min":{"1min":1.0389999999999999,"5min":0.86899999999999999,"15min":0.77100000000000002},"max":{"1min":2.4169999999999998,"5min":2.4169999999999998,"15min":2.5790000000000002},"last":1.613},{"interface":"front","average":{"1min":1.5740000000000001,"5min":1.5549999999999999,"15min":1.6000000000000001},"min":{"1min":1.2589999999999999,"5min":0.91300000000000003,"15min":0.75900000000000001},"max":{"1min":2.036,"5min":2.1040000000000001,"15min":2.6509999999999998},"last":1.4299999999999999}]}]},{"osd":0,"up_from":13,"seq":55834575056,"num_pgs":11,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":40516,"kb_used_data":1204,"kb_used_omap":1,"kb_used_meta":39294,"kb_avail":94331324,"statfs":{"total":96636764160,"available":96595275776,"internally_reserved":0,"allocated":1232896,"data_stored":657278,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1781,"internal_metadata":40237323},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Wed Aug 9 13:50:27 2023","interfaces":[{"interface":"back","average":{"1min":0.91800000000000004,"5min":0.92200000000000004,"15min":0.91600000000000004},"min":{"1min":0.32500000000000001,"5min":0.32500000000000001,"15min":0.32500000000000001},"max":{"1min":1.5820000000000001,"5min":1.756,"15min":1.756},"last":1.0129999999999999},{"interface":"front","average":{"1min":1.0680000000000001,"5min":1.1020000000000001,"15min":1.113},"min":{"1min":0.47099999999999997,"5min":0.45300000000000001,"15min":0.45300000000000001},"max":{"1min":1.9019999999999999,"5min":1.9019999999999999,"15min":1.9019999999999999},"last":0.82199999999999995}]},{"osd":2,"last update":"Wed Aug 9 13:50:27 2023","interfaces":[{"interface":"back","average":{"1min":1.048,"5min":1.1000000000000001,"15min":1.135},"min":{"1min":0.53800000000000003,"5min":0.51400000000000001,"15min":0.44800000000000001},"max":{"1min":1.702,"5min":1.776,"15min":1.889},"last":0.745},{"interface":"front","average":{"1min":1.1819999999999999,"5min":1.149,"15min":1.179},"min":{"1min":0.58499999999999996,"5min":0.56399999999999995,"15min":0.52700000000000002},"max":{"1min":2.0129999999999999,"5min":2.0129999999999999,"15min":2.2519999999999998},"last":0.93799999999999994}]},{"osd":3,"last update":"Wed Aug 9 13:50:27 2023","interfaces":[{"interface":"back","average":{"1min":0.98499999999999999,"5min":1.0509999999999999,"15min":1.105},"min":{"1min":0.54900000000000004,"5min":0.46300000000000002,"15min":0.46300000000000002},"max":{"1min":1.345,"5min":1.823,"15min":2.5190000000000001},"last":1.0600000000000001},{"interface":"front","average":{"1min":1.272,"5min":1.2629999999999999,"15min":1.3140000000000001},"min":{"1min":0.56399999999999995,"5min":0.54700000000000004,"15min":0.54700000000000004},"max":{"1min":2.347,"5min":2.347,"15min":2.4649999999999999},"last":1.1419999999999999}]},{"osd":4,"last update":"Wed Aug 9 13:50:27 2023","interfaces":[{"interface":"back","average":{"1min":1.302,"5min":1.3129999999999999,"15min":1.347},"min":{"1min":0.84699999999999998,"5min":0.84699999999999998,"15min":0.74299999999999999},"max":{"1min":1.9630000000000001,"5min":2.2309999999999999,"15min":2.2309999999999999},"last":1.3280000000000001},{"interface":"front","average":{"1min":1.3999999999999999,"5min":1.4390000000000001,"15min":1.452},"min":{"1min":0.82699999999999996,"5min":0.77600000000000002,"15min":0.74099999999999999},"max":{"1min":1.8440000000000001,"5min":2.5390000000000001,"15min":2.5390000000000001},"last":1.3080000000000001}]},{"osd":5,"last update":"Wed Aug 9 13:50:27 2023","interfaces":[{"interface":"back","average":{"1min":1.387,"5min":1.4219999999999999,"15min":1.4650000000000001},"min":{"1min":0.66300000000000003,"5min":0.66300000000000003,"15min":0.66300000000000003},"max":{"1min":2.6669999999999998,"5min":2.6669999999999998,"15min":2.6669999999999998},"last":1.603},{"interface":"front","average":{"1min":1.3400000000000001,"5min":1.415,"15min":1.4490000000000001},"min":{"1min":0.70099999999999996,"5min":0.629,"15min":0.629},"max":{"1min":1.6160000000000001,"5min":2.1499999999999999,"15min":2.1499999999999999},"last":1.2450000000000001}]},{"osd":6,"last update":"Wed Aug 9 13:50:27 2023","interfaces":[{"interface":"back","average":{"1min":1.51,"5min":1.532,"15min":1.5469999999999999},"min":{"1min":0.73599999999999999,"5min":0.73599999999999999,"15min":0.69799999999999995},"max":{"1min":2.3100000000000001,"5min":2.7250000000000001,"15min":2.7250000000000001},"last":1.4219999999999999},{"interface":"front","average":{"1min":1.5169999999999999,"5min":1.5389999999999999,"15min":1.5600000000000001},"min":{"1min":0.86699999999999999,"5min":0.86699999999999999,"15min":0.86699999999999999},"max":{"1min":2.3199999999999998,"5min":2.399,"15min":2.399},"last":1.7829999999999999}]},{"osd":7,"last update":"Wed Aug 9 13:50:27 2023","interfaces":[{"interface":"back","average":{"1min":1.532,"5min":1.595,"15min":1.625},"min":{"1min":0.72299999999999998,"5min":0.72299999999999998,"15min":0.72299999999999998},"max":{"1min":2.3900000000000001,"5min":2.3900000000000001,"15min":2.464},"last":1.7130000000000001},{"interface":"front","average":{"1min":1.4350000000000001,"5min":1.512,"15min":1.55},"min":{"1min":0.84499999999999997,"5min":0.76700000000000002,"15min":0.76700000000000002},"max":{"1min":2.274,"5min":2.274,"15min":2.274},"last":1.389}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":34,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":194,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1634,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":694,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":70,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-08-09T13:51:20.754 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-08-09T13:51:21.022 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:21.019+0000 7f234ffff640 1 Processor -- start 2023-08-09T13:51:21.022 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:21.019+0000 7f234ffff640 1 -- start start 2023-08-09T13:51:21.023 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:21.021+0000 7f234ffff640 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7f235015a6a0 0x7f23500713b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:51:21.023 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:21.021+0000 7f234ffff640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f23500cbd90 0x7f23500c8fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:51:21.023 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:21.021+0000 7f234ffff640 1 -- --> v1:172.21.15.43:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f23500cdc70 con 0x7f23500c94f0 2023-08-09T13:51:21.023 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:21.021+0000 7f234ffff640 1 -- --> v2:172.21.15.203:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f23500c9960 con 0x7f235015a6a0 2023-08-09T13:51:21.024 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:21.021+0000 7f234ffff640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2350070910 con 0x7f23500cbd90 2023-08-09T13:51:21.025 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:21.021+0000 7f234f7fe640 1 --1- >> v1:172.21.15.43:6789/0 conn(0x7f23500c94f0 0x7f235015ad20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.43:6789/0 says I am v1:172.21.15.43:59408/0 (socket says 172.21.15.43:59408) 2023-08-09T13:51:21.026 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:21.021+0000 7f234f7fe640 1 -- 172.21.15.43:0/2160654534 learned_addr learned my addr 172.21.15.43:0/2160654534 (peer_addr_for_me v1:172.21.15.43:0/0) 2023-08-09T13:51:21.026 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:21.021+0000 7f23467fc640 1 --2- 172.21.15.43:0/2160654534 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f23500cbd90 0x7f23500c8fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:51:21.026 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:21.021+0000 7f234effd640 1 --2- 172.21.15.43:0/2160654534 >> v2:172.21.15.203:3300/0 conn(0x7f235015a6a0 0x7f23500713b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:51:21.026 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:21.022+0000 7f234e7fc640 1 -- 172.21.15.43:0/2160654534 <== mon.0 v1:172.21.15.43:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3697712982 0 0) 0x7f23500cdc70 con 0x7f23500c94f0 2023-08-09T13:51:21.026 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:21.022+0000 7f234e7fc640 1 -- 172.21.15.43:0/2160654534 --> v1:172.21.15.43:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f233c005730 con 0x7f23500c94f0 2023-08-09T13:51:21.026 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:21.022+0000 7f234e7fc640 1 -- 172.21.15.43:0/2160654534 <== mon.0 v1:172.21.15.43:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3903287227 0 0) 0x7f233c005730 con 0x7f23500c94f0 2023-08-09T13:51:21.026 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:21.022+0000 7f234e7fc640 1 -- 172.21.15.43:0/2160654534 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f23500cbd90 msgr2=0x7f23500c8fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-09T13:51:21.027 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:21.022+0000 7f234e7fc640 1 --2- 172.21.15.43:0/2160654534 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f23500cbd90 0x7f23500c8fb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:51:21.027 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:21.022+0000 7f234e7fc640 1 -- 172.21.15.43:0/2160654534 >> v2:172.21.15.203:3300/0 conn(0x7f235015a6a0 msgr2=0x7f23500713b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-09T13:51:21.027 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:21.022+0000 7f234e7fc640 1 --2- 172.21.15.43:0/2160654534 >> v2:172.21.15.203:3300/0 conn(0x7f235015a6a0 0x7f23500713b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:51:21.027 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:21.022+0000 7f234e7fc640 1 -- 172.21.15.43:0/2160654534 --> v1:172.21.15.43:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f23500c88c0 con 0x7f23500c94f0 2023-08-09T13:51:21.027 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:21.022+0000 7f234e7fc640 1 -- 172.21.15.43:0/2160654534 <== mon.0 v1:172.21.15.43:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1285150740 0 0) 0x7f2348001670 con 0x7f23500c94f0 2023-08-09T13:51:21.027 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:21.022+0000 7f234e7fc640 1 -- 172.21.15.43:0/2160654534 >> v1:172.21.15.43:6789/0 conn(0x7f23500c94f0 legacy=0x7f235015ad20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:51:21.027 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:21.022+0000 7f234e7fc640 1 --2- 172.21.15.43:0/2160654534 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f233c005b80 0x7f233c005fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:51:21.028 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:21.022+0000 7f234e7fc640 1 --2- 172.21.15.43:0/2160654534 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f233c0069d0 0x7f233c006e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:51:21.028 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:21.022+0000 7f234e7fc640 1 --2- 172.21.15.43:0/2160654534 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f23500cbd90 0x7f233c007390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:51:21.028 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:21.022+0000 7f234e7fc640 1 -- 172.21.15.43:0/2160654534 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f23500c9960 con 0x7f233c005b80 2023-08-09T13:51:21.028 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:21.022+0000 7f234e7fc640 1 -- 172.21.15.43:0/2160654534 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2350070910 con 0x7f233c0069d0 2023-08-09T13:51:21.029 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:21.022+0000 7f234e7fc640 1 -- 172.21.15.43:0/2160654534 --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f233c00dbc0 con 0x7f23500cbd90 2023-08-09T13:51:21.029 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:21.023+0000 7f234effd640 1 --2- 172.21.15.43:0/2160654534 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f233c005b80 0x7f233c005fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:51:21.029 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:21.023+0000 7f234effd640 1 -- 172.21.15.43:0/2160654534 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f23500cbd90 msgr2=0x7f233c007390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-09T13:51:21.029 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:21.023+0000 7f234effd640 1 --2- 172.21.15.43:0/2160654534 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f23500cbd90 0x7f233c007390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:51:21.030 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:21.023+0000 7f234effd640 1 -- 172.21.15.43:0/2160654534 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f233c0069d0 msgr2=0x7f233c006e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-09T13:51:21.030 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:21.023+0000 7f234effd640 1 --2- 172.21.15.43:0/2160654534 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f233c0069d0 0x7f233c006e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:51:21.030 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:21.023+0000 7f234effd640 1 -- 172.21.15.43:0/2160654534 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f233c005730 con 0x7f233c005b80 2023-08-09T13:51:21.035 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:21.024+0000 7f234effd640 1 --2- 172.21.15.43:0/2160654534 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f233c005b80 0x7f233c005fa0 secure :-1 s=READY pgs=1194 cs=0 l=1 rev1=1 crypto rx=0x7f2340009fc0 tx=0x7f233c007a00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a24977860168d264 server_cookie=0 in_seq=0 out_seq=0 2023-08-09T13:51:21.035 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:21.024+0000 7f234e7fc640 1 -- 172.21.15.43:0/2160654534 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f234002cc40 con 0x7f233c005b80 2023-08-09T13:51:21.035 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:21.024+0000 7f234e7fc640 1 -- 172.21.15.43:0/2160654534 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f234000f040 con 0x7f233c005b80 2023-08-09T13:51:21.036 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:21.024+0000 7f234e7fc640 1 -- 172.21.15.43:0/2160654534 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2340034440 con 0x7f233c005b80 2023-08-09T13:51:21.036 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:21.025+0000 7f234ffff640 1 -- 172.21.15.43:0/2160654534 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f233c005b80 msgr2=0x7f233c005fa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:51:21.036 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:21.025+0000 7f234ffff640 1 --2- 172.21.15.43:0/2160654534 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f233c005b80 0x7f233c005fa0 secure :-1 s=READY pgs=1194 cs=0 l=1 rev1=1 crypto rx=0x7f2340009fc0 tx=0x7f233c007a00 comp rx=0 tx=0).stop 2023-08-09T13:51:21.037 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:21.025+0000 7f234f7fe640 1 -- 172.21.15.43:0/2160654534 reap_dead start 2023-08-09T13:51:21.037 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:21.025+0000 7f234ffff640 1 -- 172.21.15.43:0/2160654534 shutdown_connections 2023-08-09T13:51:21.037 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:21.025+0000 7f234ffff640 1 -- 172.21.15.43:0/2160654534 >> 172.21.15.43:0/2160654534 conn(0x7f23500cde90 msgr2=0x7f23500ce2a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-09T13:51:21.037 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:21.025+0000 7f234ffff640 1 -- 172.21.15.43:0/2160654534 shutdown_connections 2023-08-09T13:51:21.038 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:21.025+0000 7f234ffff640 1 -- 172.21.15.43:0/2160654534 wait complete. 2023-08-09T13:51:21.038 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:21.027+0000 7f234ffff640 1 Processor -- start 2023-08-09T13:51:21.038 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:21.027+0000 7f234ffff640 1 -- start start 2023-08-09T13:51:21.038 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:21.028+0000 7f234ffff640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f233c005b80 0x7f2350154400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:51:21.039 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:21.028+0000 7f234ffff640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f23500b1770 0x7f23500b1bc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:51:21.039 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:21.028+0000 7f234ffff640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f23500b2100 0x7f235006c1e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:51:21.039 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:21.028+0000 7f234ffff640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f23501f51e0 con 0x7f23500b2100 2023-08-09T13:51:21.039 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:21.028+0000 7f234ffff640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f23501f5a80 con 0x7f233c005b80 2023-08-09T13:51:21.040 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:21.028+0000 7f234ffff640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f23500b25a0 con 0x7f23500b1770 2023-08-09T13:51:21.040 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:21.028+0000 7f234f7fe640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f23500b2100 0x7f235006c1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:51:21.040 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:21.029+0000 7f234f7fe640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f23500b2100 0x7f235006c1e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:43004/0 (socket says 172.21.15.43:43004) 2023-08-09T13:51:21.041 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:21.029+0000 7f234f7fe640 1 -- 172.21.15.43:0/103967003 learned_addr learned my addr 172.21.15.43:0/103967003 (peer_addr_for_me v2:172.21.15.43:0/0) 2023-08-09T13:51:21.041 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:21.029+0000 7f234effd640 1 --2- 172.21.15.43:0/103967003 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f233c005b80 0x7f2350154400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:51:21.041 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:21.029+0000 7f23467fc640 1 --2- 172.21.15.43:0/103967003 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f23500b1770 0x7f23500b1bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:51:21.042 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:21.029+0000 7f234f7fe640 1 -- 172.21.15.43:0/103967003 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f23500b1770 msgr2=0x7f23500b1bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:51:21.042 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:21.029+0000 7f234f7fe640 1 --2- 172.21.15.43:0/103967003 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f23500b1770 0x7f23500b1bc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:51:21.042 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:21.029+0000 7f234f7fe640 1 -- 172.21.15.43:0/103967003 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f233c005b80 msgr2=0x7f2350154400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:51:21.043 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:21.029+0000 7f234f7fe640 1 --2- 172.21.15.43:0/103967003 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f233c005b80 0x7f2350154400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:51:21.043 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:21.029+0000 7f234f7fe640 1 -- 172.21.15.43:0/103967003 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f23501f5430 con 0x7f23500b2100 2023-08-09T13:51:21.043 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:21.029+0000 7f234f7fe640 1 --2- 172.21.15.43:0/103967003 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f23500b2100 0x7f235006c1e0 secure :-1 s=READY pgs=1195 cs=0 l=1 rev1=1 crypto rx=0x7f2348000c00 tx=0x7f234800cb90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-09T13:51:21.044 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:21.030+0000 7f2347fff640 1 -- 172.21.15.43:0/103967003 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2348007230 con 0x7f23500b2100 2023-08-09T13:51:21.044 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:21.030+0000 7f2347fff640 1 -- 172.21.15.43:0/103967003 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f234800ced0 con 0x7f23500b2100 2023-08-09T13:51:21.044 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:21.030+0000 7f2347fff640 1 -- 172.21.15.43:0/103967003 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f23480075f0 con 0x7f23500b2100 2023-08-09T13:51:21.044 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:21.030+0000 7f234ffff640 1 -- 172.21.15.43:0/103967003 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f23501f5710 con 0x7f23500b2100 2023-08-09T13:51:21.045 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:21.030+0000 7f234ffff640 1 -- 172.21.15.43:0/103967003 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f23501f5f10 con 0x7f23500b2100 2023-08-09T13:51:21.045 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:21.031+0000 7f234ffff640 1 -- 172.21.15.43:0/103967003 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f23501f5c10 con 0x7f23500b2100 2023-08-09T13:51:21.046 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:21.032+0000 7f2347fff640 1 -- 172.21.15.43:0/103967003 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 37) v1 ==== 88443+0+0 (secure 0 0 0) 0x7f23480047e0 con 0x7f23500b2100 2023-08-09T13:51:21.047 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:21.033+0000 7f2347fff640 1 --2- 172.21.15.43:0/103967003 >> [v2:172.21.15.203:6832/1778614695,v1:172.21.15.203:6833/1778614695] conn(0x7f233006a170 0x7f233006c630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:51:21.047 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:21.033+0000 7f2347fff640 1 -- 172.21.15.43:0/103967003 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(103..103 src has 1..103) v4 ==== 6426+0+0 (secure 0 0 0) 0x7f2348087790 con 0x7f23500b2100 2023-08-09T13:51:21.047 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:21.034+0000 7f234effd640 1 --2- 172.21.15.43:0/103967003 >> [v2:172.21.15.203:6832/1778614695,v1:172.21.15.203:6833/1778614695] conn(0x7f233006a170 0x7f233006c630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:51:21.047 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:21.035+0000 7f234effd640 1 --2- 172.21.15.43:0/103967003 >> [v2:172.21.15.203:6832/1778614695,v1:172.21.15.203:6833/1778614695] conn(0x7f233006a170 0x7f233006c630 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f2340036000 tx=0x7f234002e000 comp rx=0 tx=0).ready entity=mgr.5819 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-09T13:51:21.047 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:21.038+0000 7f2347fff640 1 -- 172.21.15.43:0/103967003 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178066 (secure 0 0 0) 0x7f2348052b60 con 0x7f23500b2100 2023-08-09T13:51:21.228 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:21.225+0000 7f234ffff640 1 -- 172.21.15.43:0/103967003 --> [v2:172.21.15.203:6832/1778614695,v1:172.21.15.203:6833/1778614695] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f2350070e90 con 0x7f233006a170 2023-08-09T13:51:21.251 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:21.248+0000 7f2347fff640 1 -- 172.21.15.43:0/103967003 <== mgr.5819 v2:172.21.15.203:6832/1778614695 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+309486 (secure 0 0 0) 0x7f234002cdb0 con 0x7f233006a170 2023-08-09T13:51:21.256 INFO:teuthology.orchestra.run.smithi043.stdout: 2023-08-09T13:51:21.257 INFO:teuthology.orchestra.run.smithi043.stderr:dumped all 2023-08-09T13:51:21.260 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:21.254+0000 7f234ffff640 1 -- 172.21.15.43:0/103967003 >> [v2:172.21.15.203:6832/1778614695,v1:172.21.15.203:6833/1778614695] conn(0x7f233006a170 msgr2=0x7f233006c630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:51:21.261 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:21.254+0000 7f234ffff640 1 --2- 172.21.15.43:0/103967003 >> [v2:172.21.15.203:6832/1778614695,v1:172.21.15.203:6833/1778614695] conn(0x7f233006a170 0x7f233006c630 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f2340036000 tx=0x7f234002e000 comp rx=0 tx=0).stop 2023-08-09T13:51:21.261 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:21.254+0000 7f234ffff640 1 -- 172.21.15.43:0/103967003 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f23500b2100 msgr2=0x7f235006c1e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:51:21.261 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:21.254+0000 7f234ffff640 1 --2- 172.21.15.43:0/103967003 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f23500b2100 0x7f235006c1e0 secure :-1 s=READY pgs=1195 cs=0 l=1 rev1=1 crypto rx=0x7f2348000c00 tx=0x7f234800cb90 comp rx=0 tx=0).stop 2023-08-09T13:51:21.261 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:21.254+0000 7f234ffff640 1 -- 172.21.15.43:0/103967003 shutdown_connections 2023-08-09T13:51:21.262 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:21.254+0000 7f234ffff640 1 --2- 172.21.15.43:0/103967003 >> [v2:172.21.15.203:6832/1778614695,v1:172.21.15.203:6833/1778614695] conn(0x7f233006a170 0x7f233006c630 unknown :-1 s=CLOSED pgs=65 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:51:21.262 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:21.254+0000 7f234ffff640 1 --2- 172.21.15.43:0/103967003 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f23500b2100 0x7f235006c1e0 unknown :-1 s=CLOSED pgs=1195 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:51:21.262 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:21.254+0000 7f234ffff640 1 --2- 172.21.15.43:0/103967003 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f23500b1770 0x7f23500b1bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:51:21.262 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:21.254+0000 7f234ffff640 1 --2- 172.21.15.43:0/103967003 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f233c005b80 0x7f2350154400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:51:21.262 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:21.254+0000 7f234ffff640 1 -- 172.21.15.43:0/103967003 >> 172.21.15.43:0/103967003 conn(0x7f23500cde90 msgr2=0x7f23500c9aa0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-09T13:51:21.262 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:21.254+0000 7f234ffff640 1 -- 172.21.15.43:0/103967003 shutdown_connections 2023-08-09T13:51:21.262 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:21.254+0000 7f234ffff640 1 -- 172.21.15.43:0/103967003 wait complete. 2023-08-09T13:51:21.277 INFO:teuthology.orchestra.run.smithi043.stdout:{"pg_ready":true,"pg_map":{"version":26,"stamp":"2023-08-09T13:51:19.433961+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":23,"num_object_clones":0,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":0,"num_read":619,"num_read_kb":1212,"num_write":307,"num_write_kb":2430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":213,"ondisk_log_size":213,"up":210,"acting":210,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":69,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":306520,"kb_used_data":10844,"kb_used_omap":13,"kb_used_meta":295538,"kb_avail":754668200,"statfs":{"total":773094113280,"available":772780236800,"internally_reserved":0,"allocated":11104256,"data_stored":6456252,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":14248,"internal_metadata":302631000},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":32,"apply_latency_ms":32,"commit_latency_ns":32000000,"apply_latency_ns":32000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"5.245225"},"pg_stats":[{"pgid":"13.3f","version":"0'0","reported_seq":9,"reported_epoch":102,"state":"active+clean","last_fresh":"2023-08-09T13:51:16.258593+0000","last_change":"2023-08-09T13:51:16.248603+0000","last_active":"2023-08-09T13:51:16.258593+0000","last_peered":"2023-08-09T13:51:16.258593+0000","last_clean":"2023-08-09T13:51:16.258593+0000","last_became_active":"2023-08-09T13:51:16.248037+0000","last_became_peered":"2023-08-09T13:51:16.248037+0000","last_unstale":"2023-08-09T13:51:16.258593+0000","last_undegraded":"2023-08-09T13:51:16.258593+0000","last_fullsized":"2023-08-09T13:51:16.258593+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:51:15.188798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:51:15.188798+0000","last_clean_scrub_stamp":"2023-08-09T13:51:15.188798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T16:48:12.509006+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.3e","version":"0'0","reported_seq":9,"reported_epoch":102,"state":"active+clean","last_fresh":"2023-08-09T13:51:16.262721+0000","last_change":"2023-08-09T13:51:16.257259+0000","last_active":"2023-08-09T13:51:16.262721+0000","last_peered":"2023-08-09T13:51:16.262721+0000","last_clean":"2023-08-09T13:51:16.262721+0000","last_became_active":"2023-08-09T13:51:16.256916+0000","last_became_peered":"2023-08-09T13:51:16.256916+0000","last_unstale":"2023-08-09T13:51:16.262721+0000","last_undegraded":"2023-08-09T13:51:16.262721+0000","last_fullsized":"2023-08-09T13:51:16.262721+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:51:15.188798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:51:15.188798+0000","last_clean_scrub_stamp":"2023-08-09T13:51:15.188798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T16:59:10.173876+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.3d","version":"0'0","reported_seq":9,"reported_epoch":102,"state":"active+clean","last_fresh":"2023-08-09T13:51:16.258432+0000","last_change":"2023-08-09T13:51:16.257154+0000","last_active":"2023-08-09T13:51:16.258432+0000","last_peered":"2023-08-09T13:51:16.258432+0000","last_clean":"2023-08-09T13:51:16.258432+0000","last_became_active":"2023-08-09T13:51:16.256519+0000","last_became_peered":"2023-08-09T13:51:16.256519+0000","last_unstale":"2023-08-09T13:51:16.258432+0000","last_undegraded":"2023-08-09T13:51:16.258432+0000","last_fullsized":"2023-08-09T13:51:16.258432+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:51:15.188798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:51:15.188798+0000","last_clean_scrub_stamp":"2023-08-09T13:51:15.188798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T22:23:55.883603+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"13.3c","version":"0'0","reported_seq":9,"reported_epoch":102,"state":"active+clean","last_fresh":"2023-08-09T13:51:16.260995+0000","last_change":"2023-08-09T13:51:16.257966+0000","last_active":"2023-08-09T13:51:16.260995+0000","last_peered":"2023-08-09T13:51:16.260995+0000","last_clean":"2023-08-09T13:51:16.260995+0000","last_became_active":"2023-08-09T13:51:16.257491+0000","last_became_peered":"2023-08-09T13:51:16.257491+0000","last_unstale":"2023-08-09T13:51:16.260995+0000","last_undegraded":"2023-08-09T13:51:16.260995+0000","last_fullsized":"2023-08-09T13:51:16.260995+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:51:15.188798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:51:15.188798+0000","last_clean_scrub_stamp":"2023-08-09T13:51:15.188798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T15:13:09.199403+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.3b","version":"0'0","reported_seq":9,"reported_epoch":102,"state":"active+clean","last_fresh":"2023-08-09T13:51:16.255556+0000","last_change":"2023-08-09T13:51:16.251679+0000","last_active":"2023-08-09T13:51:16.255556+0000","last_peered":"2023-08-09T13:51:16.255556+0000","last_clean":"2023-08-09T13:51:16.255556+0000","last_became_active":"2023-08-09T13:51:16.251416+0000","last_became_peered":"2023-08-09T13:51:16.251416+0000","last_unstale":"2023-08-09T13:51:16.255556+0000","last_undegraded":"2023-08-09T13:51:16.255556+0000","last_fullsized":"2023-08-09T13:51:16.255556+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:51:15.188798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:51:15.188798+0000","last_clean_scrub_stamp":"2023-08-09T13:51:15.188798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T15:52:45.214559+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.3a","version":"0'0","reported_seq":9,"reported_epoch":102,"state":"active+clean","last_fresh":"2023-08-09T13:51:16.253888+0000","last_change":"2023-08-09T13:51:16.247397+0000","last_active":"2023-08-09T13:51:16.253888+0000","last_peered":"2023-08-09T13:51:16.253888+0000","last_clean":"2023-08-09T13:51:16.253888+0000","last_became_active":"2023-08-09T13:51:16.247132+0000","last_became_peered":"2023-08-09T13:51:16.247132+0000","last_unstale":"2023-08-09T13:51:16.253888+0000","last_undegraded":"2023-08-09T13:51:16.253888+0000","last_fullsized":"2023-08-09T13:51:16.253888+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:51:15.188798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:51:15.188798+0000","last_clean_scrub_stamp":"2023-08-09T13:51:15.188798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T17:34:10.406118+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"13.39","version":"0'0","reported_seq":9,"reported_epoch":102,"state":"active+clean","last_fresh":"2023-08-09T13:51:16.252061+0000","last_change":"2023-08-09T13:51:16.245360+0000","last_active":"2023-08-09T13:51:16.252061+0000","last_peered":"2023-08-09T13:51:16.252061+0000","last_clean":"2023-08-09T13:51:16.252061+0000","last_became_active":"2023-08-09T13:51:16.245025+0000","last_became_peered":"2023-08-09T13:51:16.245025+0000","last_unstale":"2023-08-09T13:51:16.252061+0000","last_undegraded":"2023-08-09T13:51:16.252061+0000","last_fullsized":"2023-08-09T13:51:16.252061+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:51:15.188798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:51:15.188798+0000","last_clean_scrub_stamp":"2023-08-09T13:51:15.188798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T16:44:53.436136+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.38","version":"0'0","reported_seq":9,"reported_epoch":102,"state":"active+clean","last_fresh":"2023-08-09T13:51:16.254444+0000","last_change":"2023-08-09T13:51:16.253769+0000","last_active":"2023-08-09T13:51:16.254444+0000","last_peered":"2023-08-09T13:51:16.254444+0000","last_clean":"2023-08-09T13:51:16.254444+0000","last_became_active":"2023-08-09T13:51:16.253464+0000","last_became_peered":"2023-08-09T13:51:16.253464+0000","last_unstale":"2023-08-09T13:51:16.254444+0000","last_undegraded":"2023-08-09T13:51:16.254444+0000","last_fullsized":"2023-08-09T13:51:16.254444+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:51:15.188798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:51:15.188798+0000","last_clean_scrub_stamp":"2023-08-09T13:51:15.188798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T18:17:41.906539+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"13.37","version":"0'0","reported_seq":9,"reported_epoch":102,"state":"active+clean","last_fresh":"2023-08-09T13:51:16.235673+0000","last_change":"2023-08-09T13:51:16.233810+0000","last_active":"2023-08-09T13:51:16.235673+0000","last_peered":"2023-08-09T13:51:16.235673+0000","last_clean":"2023-08-09T13:51:16.235673+0000","last_became_active":"2023-08-09T13:51:16.232392+0000","last_became_peered":"2023-08-09T13:51:16.232392+0000","last_unstale":"2023-08-09T13:51:16.235673+0000","last_undegraded":"2023-08-09T13:51:16.235673+0000","last_fullsized":"2023-08-09T13:51:16.235673+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:51:15.188798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:51:15.188798+0000","last_clean_scrub_stamp":"2023-08-09T13:51:15.188798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T18:35:59.659869+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"13.36","version":"0'0","reported_seq":9,"reported_epoch":102,"state":"active+clean","last_fresh":"2023-08-09T13:51:16.259336+0000","last_change":"2023-08-09T13:51:16.253575+0000","last_active":"2023-08-09T13:51:16.259336+0000","last_peered":"2023-08-09T13:51:16.259336+0000","last_clean":"2023-08-09T13:51:16.259336+0000","last_became_active":"2023-08-09T13:51:16.253189+0000","last_became_peered":"2023-08-09T13:51:16.253189+0000","last_unstale":"2023-08-09T13:51:16.259336+0000","last_undegraded":"2023-08-09T13:51:16.259336+0000","last_fullsized":"2023-08-09T13:51:16.259336+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:51:15.188798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:51:15.188798+0000","last_clean_scrub_stamp":"2023-08-09T13:51:15.188798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T22:29:06.334182+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.35","version":"0'0","reported_seq":9,"reported_epoch":102,"state":"active+clean","last_fresh":"2023-08-09T13:51:16.262632+0000","last_change":"2023-08-09T13:51:16.256458+0000","last_active":"2023-08-09T13:51:16.262632+0000","last_peered":"2023-08-09T13:51:16.262632+0000","last_clean":"2023-08-09T13:51:16.262632+0000","last_became_active":"2023-08-09T13:51:16.256188+0000","last_became_peered":"2023-08-09T13:51:16.256188+0000","last_unstale":"2023-08-09T13:51:16.262632+0000","last_undegraded":"2023-08-09T13:51:16.262632+0000","last_fullsized":"2023-08-09T13:51:16.262632+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:51:15.188798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:51:15.188798+0000","last_clean_scrub_stamp":"2023-08-09T13:51:15.188798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T22:52:37.102129+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.34","version":"0'0","reported_seq":9,"reported_epoch":102,"state":"active+clean","last_fresh":"2023-08-09T13:51:16.259450+0000","last_change":"2023-08-09T13:51:16.254964+0000","last_active":"2023-08-09T13:51:16.259450+0000","last_peered":"2023-08-09T13:51:16.259450+0000","last_clean":"2023-08-09T13:51:16.259450+0000","last_became_active":"2023-08-09T13:51:16.254499+0000","last_became_peered":"2023-08-09T13:51:16.254499+0000","last_unstale":"2023-08-09T13:51:16.259450+0000","last_undegraded":"2023-08-09T13:51:16.259450+0000","last_fullsized":"2023-08-09T13:51:16.259450+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:51:15.188798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:51:15.188798+0000","last_clean_scrub_stamp":"2023-08-09T13:51:15.188798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T23:45:01.441221+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.33","version":"0'0","reported_seq":9,"reported_epoch":102,"state":"active+clean","last_fresh":"2023-08-09T13:51:16.243344+0000","last_change":"2023-08-09T13:51:16.240086+0000","last_active":"2023-08-09T13:51:16.243344+0000","last_peered":"2023-08-09T13:51:16.243344+0000","last_clean":"2023-08-09T13:51:16.243344+0000","last_became_active":"2023-08-09T13:51:16.239792+0000","last_became_peered":"2023-08-09T13:51:16.239792+0000","last_unstale":"2023-08-09T13:51:16.243344+0000","last_undegraded":"2023-08-09T13:51:16.243344+0000","last_fullsized":"2023-08-09T13:51:16.243344+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:51:15.188798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:51:15.188798+0000","last_clean_scrub_stamp":"2023-08-09T13:51:15.188798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T15:20:43.751333+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"13.32","version":"0'0","reported_seq":9,"reported_epoch":102,"state":"active+clean","last_fresh":"2023-08-09T13:51:16.257888+0000","last_change":"2023-08-09T13:51:16.253437+0000","last_active":"2023-08-09T13:51:16.257888+0000","last_peered":"2023-08-09T13:51:16.257888+0000","last_clean":"2023-08-09T13:51:16.257888+0000","last_became_active":"2023-08-09T13:51:16.253106+0000","last_became_peered":"2023-08-09T13:51:16.253106+0000","last_unstale":"2023-08-09T13:51:16.257888+0000","last_undegraded":"2023-08-09T13:51:16.257888+0000","last_fullsized":"2023-08-09T13:51:16.257888+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:51:15.188798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:51:15.188798+0000","last_clean_scrub_stamp":"2023-08-09T13:51:15.188798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T15:24:24.668964+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.31","version":"0'0","reported_seq":9,"reported_epoch":102,"state":"active+clean","last_fresh":"2023-08-09T13:51:16.253948+0000","last_change":"2023-08-09T13:51:16.247068+0000","last_active":"2023-08-09T13:51:16.253948+0000","last_peered":"2023-08-09T13:51:16.253948+0000","last_clean":"2023-08-09T13:51:16.253948+0000","last_became_active":"2023-08-09T13:51:16.246689+0000","last_became_peered":"2023-08-09T13:51:16.246689+0000","last_unstale":"2023-08-09T13:51:16.253948+0000","last_undegraded":"2023-08-09T13:51:16.253948+0000","last_fullsized":"2023-08-09T13:51:16.253948+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:51:15.188798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:51:15.188798+0000","last_clean_scrub_stamp":"2023-08-09T13:51:15.188798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-11T00:34:32.632449+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"13.30","version":"0'0","reported_seq":9,"reported_epoch":102,"state":"active+clean","last_fresh":"2023-08-09T13:51:16.255493+0000","last_change":"2023-08-09T13:51:16.250688+0000","last_active":"2023-08-09T13:51:16.255493+0000","last_peered":"2023-08-09T13:51:16.255493+0000","last_clean":"2023-08-09T13:51:16.255493+0000","last_became_active":"2023-08-09T13:51:16.250402+0000","last_became_peered":"2023-08-09T13:51:16.250402+0000","last_unstale":"2023-08-09T13:51:16.255493+0000","last_undegraded":"2023-08-09T13:51:16.255493+0000","last_fullsized":"2023-08-09T13:51:16.255493+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:51:15.188798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:51:15.188798+0000","last_clean_scrub_stamp":"2023-08-09T13:51:15.188798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T19:40:21.994081+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.2f","version":"0'0","reported_seq":9,"reported_epoch":102,"state":"active+clean","last_fresh":"2023-08-09T13:51:16.251253+0000","last_change":"2023-08-09T13:51:16.244707+0000","last_active":"2023-08-09T13:51:16.251253+0000","last_peered":"2023-08-09T13:51:16.251253+0000","last_clean":"2023-08-09T13:51:16.251253+0000","last_became_active":"2023-08-09T13:51:16.244438+0000","last_became_peered":"2023-08-09T13:51:16.244438+0000","last_unstale":"2023-08-09T13:51:16.251253+0000","last_undegraded":"2023-08-09T13:51:16.251253+0000","last_fullsized":"2023-08-09T13:51:16.251253+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:51:15.188798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:51:15.188798+0000","last_clean_scrub_stamp":"2023-08-09T13:51:15.188798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T15:37:01.831464+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.2e","version":"0'0","reported_seq":9,"reported_epoch":102,"state":"active+clean","last_fresh":"2023-08-09T13:51:16.258072+0000","last_change":"2023-08-09T13:51:16.254207+0000","last_active":"2023-08-09T13:51:16.258072+0000","last_peered":"2023-08-09T13:51:16.258072+0000","last_clean":"2023-08-09T13:51:16.258072+0000","last_became_active":"2023-08-09T13:51:16.253931+0000","last_became_peered":"2023-08-09T13:51:16.253931+0000","last_unstale":"2023-08-09T13:51:16.258072+0000","last_undegraded":"2023-08-09T13:51:16.258072+0000","last_fullsized":"2023-08-09T13:51:16.258072+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:51:15.188798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:51:15.188798+0000","last_clean_scrub_stamp":"2023-08-09T13:51:15.188798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T23:00:52.771901+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.2d","version":"0'0","reported_seq":9,"reported_epoch":102,"state":"active+clean","last_fresh":"2023-08-09T13:51:16.254573+0000","last_change":"2023-08-09T13:51:16.246483+0000","last_active":"2023-08-09T13:51:16.254573+0000","last_peered":"2023-08-09T13:51:16.254573+0000","last_clean":"2023-08-09T13:51:16.254573+0000","last_became_active":"2023-08-09T13:51:16.246199+0000","last_became_peered":"2023-08-09T13:51:16.246199+0000","last_unstale":"2023-08-09T13:51:16.254573+0000","last_undegraded":"2023-08-09T13:51:16.254573+0000","last_fullsized":"2023-08-09T13:51:16.254573+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:51:15.188798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:51:15.188798+0000","last_clean_scrub_stamp":"2023-08-09T13:51:15.188798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T15:52:45.214559+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.2c","version":"0'0","reported_seq":9,"reported_epoch":102,"state":"active+clean","last_fresh":"2023-08-09T13:51:16.250906+0000","last_change":"2023-08-09T13:51:16.241540+0000","last_active":"2023-08-09T13:51:16.250906+0000","last_peered":"2023-08-09T13:51:16.250906+0000","last_clean":"2023-08-09T13:51:16.250906+0000","last_became_active":"2023-08-09T13:51:16.241056+0000","last_became_peered":"2023-08-09T13:51:16.241056+0000","last_unstale":"2023-08-09T13:51:16.250906+0000","last_undegraded":"2023-08-09T13:51:16.250906+0000","last_fullsized":"2023-08-09T13:51:16.250906+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:51:15.188798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:51:15.188798+0000","last_clean_scrub_stamp":"2023-08-09T13:51:15.188798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T14:26:39.561465+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.2b","version":"0'0","reported_seq":9,"reported_epoch":102,"state":"active+clean","last_fresh":"2023-08-09T13:51:16.250700+0000","last_change":"2023-08-09T13:51:16.241383+0000","last_active":"2023-08-09T13:51:16.250700+0000","last_peered":"2023-08-09T13:51:16.250700+0000","last_clean":"2023-08-09T13:51:16.250700+0000","last_became_active":"2023-08-09T13:51:16.240841+0000","last_became_peered":"2023-08-09T13:51:16.240841+0000","last_unstale":"2023-08-09T13:51:16.250700+0000","last_undegraded":"2023-08-09T13:51:16.250700+0000","last_fullsized":"2023-08-09T13:51:16.250700+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:51:15.188798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:51:15.188798+0000","last_clean_scrub_stamp":"2023-08-09T13:51:15.188798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-11T01:16:57.226514+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.2a","version":"0'0","reported_seq":9,"reported_epoch":102,"state":"active+clean","last_fresh":"2023-08-09T13:51:16.243532+0000","last_change":"2023-08-09T13:51:16.227586+0000","last_active":"2023-08-09T13:51:16.243532+0000","last_peered":"2023-08-09T13:51:16.243532+0000","last_clean":"2023-08-09T13:51:16.243532+0000","last_became_active":"2023-08-09T13:51:16.227304+0000","last_became_peered":"2023-08-09T13:51:16.227304+0000","last_unstale":"2023-08-09T13:51:16.243532+0000","last_undegraded":"2023-08-09T13:51:16.243532+0000","last_fullsized":"2023-08-09T13:51:16.243532+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:51:15.188798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:51:15.188798+0000","last_clean_scrub_stamp":"2023-08-09T13:51:15.188798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T16:18:24.664649+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.29","version":"0'0","reported_seq":9,"reported_epoch":102,"state":"active+clean","last_fresh":"2023-08-09T13:51:16.259341+0000","last_change":"2023-08-09T13:51:16.253681+0000","last_active":"2023-08-09T13:51:16.259341+0000","last_peered":"2023-08-09T13:51:16.259341+0000","last_clean":"2023-08-09T13:51:16.259341+0000","last_became_active":"2023-08-09T13:51:16.253385+0000","last_became_peered":"2023-08-09T13:51:16.253385+0000","last_unstale":"2023-08-09T13:51:16.259341+0000","last_undegraded":"2023-08-09T13:51:16.259341+0000","last_fullsized":"2023-08-09T13:51:16.259341+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:51:15.188798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:51:15.188798+0000","last_clean_scrub_stamp":"2023-08-09T13:51:15.188798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-11T00:47:25.117819+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.28","version":"0'0","reported_seq":9,"reported_epoch":102,"state":"active+clean","last_fresh":"2023-08-09T13:51:16.250957+0000","last_change":"2023-08-09T13:51:16.242027+0000","last_active":"2023-08-09T13:51:16.250957+0000","last_peered":"2023-08-09T13:51:16.250957+0000","last_clean":"2023-08-09T13:51:16.250957+0000","last_became_active":"2023-08-09T13:51:16.241728+0000","last_became_peered":"2023-08-09T13:51:16.241728+0000","last_unstale":"2023-08-09T13:51:16.250957+0000","last_undegraded":"2023-08-09T13:51:16.250957+0000","last_fullsized":"2023-08-09T13:51:16.250957+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:51:15.188798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:51:15.188798+0000","last_clean_scrub_stamp":"2023-08-09T13:51:15.188798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T16:32:37.597359+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.27","version":"0'0","reported_seq":9,"reported_epoch":102,"state":"active+clean","last_fresh":"2023-08-09T13:51:16.261482+0000","last_change":"2023-08-09T13:51:16.251716+0000","last_active":"2023-08-09T13:51:16.261482+0000","last_peered":"2023-08-09T13:51:16.261482+0000","last_clean":"2023-08-09T13:51:16.261482+0000","last_became_active":"2023-08-09T13:51:16.251298+0000","last_became_peered":"2023-08-09T13:51:16.251298+0000","last_unstale":"2023-08-09T13:51:16.261482+0000","last_undegraded":"2023-08-09T13:51:16.261482+0000","last_fullsized":"2023-08-09T13:51:16.261482+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:51:15.188798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:51:15.188798+0000","last_clean_scrub_stamp":"2023-08-09T13:51:15.188798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-11T01:28:14.454002+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.26","version":"0'0","reported_seq":9,"reported_epoch":102,"state":"active+clean","last_fresh":"2023-08-09T13:51:16.257487+0000","last_change":"2023-08-09T13:51:16.252554+0000","last_active":"2023-08-09T13:51:16.257487+0000","last_peered":"2023-08-09T13:51:16.257487+0000","last_clean":"2023-08-09T13:51:16.257487+0000","last_became_active":"2023-08-09T13:51:16.252241+0000","last_became_peered":"2023-08-09T13:51:16.252241+0000","last_unstale":"2023-08-09T13:51:16.257487+0000","last_undegraded":"2023-08-09T13:51:16.257487+0000","last_fullsized":"2023-08-09T13:51:16.257487+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:51:15.188798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:51:15.188798+0000","last_clean_scrub_stamp":"2023-08-09T13:51:15.188798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T20:43:45.377919+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"13.25","version":"0'0","reported_seq":9,"reported_epoch":102,"state":"active+clean","last_fresh":"2023-08-09T13:51:16.255617+0000","last_change":"2023-08-09T13:51:16.247370+0000","last_active":"2023-08-09T13:51:16.255617+0000","last_peered":"2023-08-09T13:51:16.255617+0000","last_clean":"2023-08-09T13:51:16.255617+0000","last_became_active":"2023-08-09T13:51:16.247041+0000","last_became_peered":"2023-08-09T13:51:16.247041+0000","last_unstale":"2023-08-09T13:51:16.255617+0000","last_undegraded":"2023-08-09T13:51:16.255617+0000","last_fullsized":"2023-08-09T13:51:16.255617+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:51:15.188798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:51:15.188798+0000","last_clean_scrub_stamp":"2023-08-09T13:51:15.188798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T21:23:46.771956+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.24","version":"0'0","reported_seq":9,"reported_epoch":102,"state":"active+clean","last_fresh":"2023-08-09T13:51:16.259470+0000","last_change":"2023-08-09T13:51:16.255148+0000","last_active":"2023-08-09T13:51:16.259470+0000","last_peered":"2023-08-09T13:51:16.259470+0000","last_clean":"2023-08-09T13:51:16.259470+0000","last_became_active":"2023-08-09T13:51:16.254681+0000","last_became_peered":"2023-08-09T13:51:16.254681+0000","last_unstale":"2023-08-09T13:51:16.259470+0000","last_undegraded":"2023-08-09T13:51:16.259470+0000","last_fullsized":"2023-08-09T13:51:16.259470+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:51:15.188798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:51:15.188798+0000","last_clean_scrub_stamp":"2023-08-09T13:51:15.188798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T21:12:55.275167+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.23","version":"0'0","reported_seq":9,"reported_epoch":102,"state":"active+clean","last_fresh":"2023-08-09T13:51:16.262319+0000","last_change":"2023-08-09T13:51:16.255251+0000","last_active":"2023-08-09T13:51:16.262319+0000","last_peered":"2023-08-09T13:51:16.262319+0000","last_clean":"2023-08-09T13:51:16.262319+0000","last_became_active":"2023-08-09T13:51:16.254974+0000","last_became_peered":"2023-08-09T13:51:16.254974+0000","last_unstale":"2023-08-09T13:51:16.262319+0000","last_undegraded":"2023-08-09T13:51:16.262319+0000","last_fullsized":"2023-08-09T13:51:16.262319+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:51:15.188798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:51:15.188798+0000","last_clean_scrub_stamp":"2023-08-09T13:51:15.188798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T18:10:20.996377+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.22","version":"0'0","reported_seq":9,"reported_epoch":102,"state":"active+clean","last_fresh":"2023-08-09T13:51:16.250243+0000","last_change":"2023-08-09T13:51:16.243113+0000","last_active":"2023-08-09T13:51:16.250243+0000","last_peered":"2023-08-09T13:51:16.250243+0000","last_clean":"2023-08-09T13:51:16.250243+0000","last_became_active":"2023-08-09T13:51:16.242518+0000","last_became_peered":"2023-08-09T13:51:16.242518+0000","last_unstale":"2023-08-09T13:51:16.250243+0000","last_undegraded":"2023-08-09T13:51:16.250243+0000","last_fullsized":"2023-08-09T13:51:16.250243+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:51:15.188798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:51:15.188798+0000","last_clean_scrub_stamp":"2023-08-09T13:51:15.188798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T16:37:00.528301+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.21","version":"0'0","reported_seq":9,"reported_epoch":102,"state":"active+clean","last_fresh":"2023-08-09T13:51:16.245497+0000","last_change":"2023-08-09T13:51:16.240818+0000","last_active":"2023-08-09T13:51:16.245497+0000","last_peered":"2023-08-09T13:51:16.245497+0000","last_clean":"2023-08-09T13:51:16.245497+0000","last_became_active":"2023-08-09T13:51:16.240492+0000","last_became_peered":"2023-08-09T13:51:16.240492+0000","last_unstale":"2023-08-09T13:51:16.245497+0000","last_undegraded":"2023-08-09T13:51:16.245497+0000","last_fullsized":"2023-08-09T13:51:16.245497+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:51:15.188798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:51:15.188798+0000","last_clean_scrub_stamp":"2023-08-09T13:51:15.188798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T19:11:07.709369+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.20","version":"0'0","reported_seq":9,"reported_epoch":102,"state":"active+clean","last_fresh":"2023-08-09T13:51:16.258316+0000","last_change":"2023-08-09T13:51:16.251983+0000","last_active":"2023-08-09T13:51:16.258316+0000","last_peered":"2023-08-09T13:51:16.258316+0000","last_clean":"2023-08-09T13:51:16.258316+0000","last_became_active":"2023-08-09T13:51:16.251594+0000","last_became_peered":"2023-08-09T13:51:16.251594+0000","last_unstale":"2023-08-09T13:51:16.258316+0000","last_undegraded":"2023-08-09T13:51:16.258316+0000","last_fullsized":"2023-08-09T13:51:16.258316+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:51:15.188798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:51:15.188798+0000","last_clean_scrub_stamp":"2023-08-09T13:51:15.188798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T23:12:07.933015+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.1e","version":"0'0","reported_seq":9,"reported_epoch":102,"state":"active+clean","last_fresh":"2023-08-09T13:51:16.257697+0000","last_change":"2023-08-09T13:51:16.254978+0000","last_active":"2023-08-09T13:51:16.257697+0000","last_peered":"2023-08-09T13:51:16.257697+0000","last_clean":"2023-08-09T13:51:16.257697+0000","last_became_active":"2023-08-09T13:51:16.254646+0000","last_became_peered":"2023-08-09T13:51:16.254646+0000","last_unstale":"2023-08-09T13:51:16.257697+0000","last_undegraded":"2023-08-09T13:51:16.257697+0000","last_fullsized":"2023-08-09T13:51:16.257697+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:51:15.188798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:51:15.188798+0000","last_clean_scrub_stamp":"2023-08-09T13:51:15.188798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T19:37:14.081662+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"12.1f","version":"102'4","reported_seq":15,"reported_epoch":102,"state":"active+clean","last_fresh":"2023-08-09T13:51:16.258227+0000","last_change":"2023-08-09T13:51:15.242826+0000","last_active":"2023-08-09T13:51:16.258227+0000","last_peered":"2023-08-09T13:51:16.258227+0000","last_clean":"2023-08-09T13:51:16.258227+0000","last_became_active":"2023-08-09T13:51:15.242507+0000","last_became_peered":"2023-08-09T13:51:15.242507+0000","last_unstale":"2023-08-09T13:51:16.258227+0000","last_undegraded":"2023-08-09T13:51:16.258227+0000","last_fullsized":"2023-08-09T13:51:16.258227+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:51:14.171648+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:51:14.171648+0000","last_clean_scrub_stamp":"2023-08-09T13:51:14.171648+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T13:52:18.768602+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.1f","version":"0'0","reported_seq":9,"reported_epoch":102,"state":"active+clean","last_fresh":"2023-08-09T13:51:16.259855+0000","last_change":"2023-08-09T13:51:16.258195+0000","last_active":"2023-08-09T13:51:16.259855+0000","last_peered":"2023-08-09T13:51:16.259855+0000","last_clean":"2023-08-09T13:51:16.259855+0000","last_became_active":"2023-08-09T13:51:16.257074+0000","last_became_peered":"2023-08-09T13:51:16.257074+0000","last_unstale":"2023-08-09T13:51:16.259855+0000","last_undegraded":"2023-08-09T13:51:16.259855+0000","last_fullsized":"2023-08-09T13:51:16.259855+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:51:15.188798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:51:15.188798+0000","last_clean_scrub_stamp":"2023-08-09T13:51:15.188798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T22:28:58.004817+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.1e","version":"102'1","reported_seq":12,"reported_epoch":102,"state":"active+clean","last_fresh":"2023-08-09T13:51:16.250829+0000","last_change":"2023-08-09T13:51:15.243566+0000","last_active":"2023-08-09T13:51:16.250829+0000","last_peered":"2023-08-09T13:51:16.250829+0000","last_clean":"2023-08-09T13:51:16.250829+0000","last_became_active":"2023-08-09T13:51:15.242859+0000","last_became_peered":"2023-08-09T13:51:15.242859+0000","last_unstale":"2023-08-09T13:51:16.250829+0000","last_undegraded":"2023-08-09T13:51:16.250829+0000","last_fullsized":"2023-08-09T13:51:16.250829+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:51:14.171648+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:51:14.171648+0000","last_clean_scrub_stamp":"2023-08-09T13:51:14.171648+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T15:31:24.677331+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.1c","version":"0'0","reported_seq":9,"reported_epoch":102,"state":"active+clean","last_fresh":"2023-08-09T13:51:16.262914+0000","last_change":"2023-08-09T13:51:16.262328+0000","last_active":"2023-08-09T13:51:16.262914+0000","last_peered":"2023-08-09T13:51:16.262914+0000","last_clean":"2023-08-09T13:51:16.262914+0000","last_became_active":"2023-08-09T13:51:16.262009+0000","last_became_peered":"2023-08-09T13:51:16.262009+0000","last_unstale":"2023-08-09T13:51:16.262914+0000","last_undegraded":"2023-08-09T13:51:16.262914+0000","last_fullsized":"2023-08-09T13:51:16.262914+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:51:15.188798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:51:15.188798+0000","last_clean_scrub_stamp":"2023-08-09T13:51:15.188798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T14:29:26.408639+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.1d","version":"102'2","reported_seq":13,"reported_epoch":102,"state":"active+clean","last_fresh":"2023-08-09T13:51:16.239417+0000","last_change":"2023-08-09T13:51:15.231538+0000","last_active":"2023-08-09T13:51:16.239417+0000","last_peered":"2023-08-09T13:51:16.239417+0000","last_clean":"2023-08-09T13:51:16.239417+0000","last_became_active":"2023-08-09T13:51:15.231233+0000","last_became_peered":"2023-08-09T13:51:15.231233+0000","last_unstale":"2023-08-09T13:51:16.239417+0000","last_undegraded":"2023-08-09T13:51:16.239417+0000","last_fullsized":"2023-08-09T13:51:16.239417+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:51:14.171648+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:51:14.171648+0000","last_clean_scrub_stamp":"2023-08-09T13:51:14.171648+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T15:37:00.814313+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.1d","version":"0'0","reported_seq":9,"reported_epoch":102,"state":"active+clean","last_fresh":"2023-08-09T13:51:16.258451+0000","last_change":"2023-08-09T13:51:16.257359+0000","last_active":"2023-08-09T13:51:16.258451+0000","last_peered":"2023-08-09T13:51:16.258451+0000","last_clean":"2023-08-09T13:51:16.258451+0000","last_became_active":"2023-08-09T13:51:16.256890+0000","last_became_peered":"2023-08-09T13:51:16.256890+0000","last_unstale":"2023-08-09T13:51:16.258451+0000","last_undegraded":"2023-08-09T13:51:16.258451+0000","last_fullsized":"2023-08-09T13:51:16.258451+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:51:15.188798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:51:15.188798+0000","last_clean_scrub_stamp":"2023-08-09T13:51:15.188798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T20:57:55.297812+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.1c","version":"102'2","reported_seq":13,"reported_epoch":102,"state":"active+clean","last_fresh":"2023-08-09T13:51:16.239352+0000","last_change":"2023-08-09T13:51:15.237921+0000","last_active":"2023-08-09T13:51:16.239352+0000","last_peered":"2023-08-09T13:51:16.239352+0000","last_clean":"2023-08-09T13:51:16.239352+0000","last_became_active":"2023-08-09T13:51:15.237645+0000","last_became_peered":"2023-08-09T13:51:15.237645+0000","last_unstale":"2023-08-09T13:51:16.239352+0000","last_undegraded":"2023-08-09T13:51:16.239352+0000","last_fullsized":"2023-08-09T13:51:16.239352+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:51:14.171648+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:51:14.171648+0000","last_clean_scrub_stamp":"2023-08-09T13:51:14.171648+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T19:11:06.692218+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.1a","version":"0'0","reported_seq":9,"reported_epoch":102,"state":"active+clean","last_fresh":"2023-08-09T13:51:16.250222+0000","last_change":"2023-08-09T13:51:16.242848+0000","last_active":"2023-08-09T13:51:16.250222+0000","last_peered":"2023-08-09T13:51:16.250222+0000","last_clean":"2023-08-09T13:51:16.250222+0000","last_became_active":"2023-08-09T13:51:16.242115+0000","last_became_peered":"2023-08-09T13:51:16.242115+0000","last_unstale":"2023-08-09T13:51:16.250222+0000","last_undegraded":"2023-08-09T13:51:16.250222+0000","last_fullsized":"2023-08-09T13:51:16.250222+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:51:15.188798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:51:15.188798+0000","last_clean_scrub_stamp":"2023-08-09T13:51:15.188798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T22:25:07.345536+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.1b","version":"102'1","reported_seq":12,"reported_epoch":102,"state":"active+clean","last_fresh":"2023-08-09T13:51:16.261706+0000","last_change":"2023-08-09T13:51:15.238074+0000","last_active":"2023-08-09T13:51:16.261706+0000","last_peered":"2023-08-09T13:51:16.261706+0000","last_clean":"2023-08-09T13:51:16.261706+0000","last_became_active":"2023-08-09T13:51:15.237763+0000","last_became_peered":"2023-08-09T13:51:15.237763+0000","last_unstale":"2023-08-09T13:51:16.261706+0000","last_undegraded":"2023-08-09T13:51:16.261706+0000","last_fullsized":"2023-08-09T13:51:16.261706+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:51:14.171648+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:51:14.171648+0000","last_clean_scrub_stamp":"2023-08-09T13:51:14.171648+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T23:42:00.578777+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.1b","version":"0'0","reported_seq":9,"reported_epoch":102,"state":"active+clean","last_fresh":"2023-08-09T13:51:16.251871+0000","last_change":"2023-08-09T13:51:16.244386+0000","last_active":"2023-08-09T13:51:16.251871+0000","last_peered":"2023-08-09T13:51:16.251871+0000","last_clean":"2023-08-09T13:51:16.251871+0000","last_became_active":"2023-08-09T13:51:16.244037+0000","last_became_peered":"2023-08-09T13:51:16.244037+0000","last_unstale":"2023-08-09T13:51:16.251871+0000","last_undegraded":"2023-08-09T13:51:16.251871+0000","last_fullsized":"2023-08-09T13:51:16.251871+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:51:15.188798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:51:15.188798+0000","last_clean_scrub_stamp":"2023-08-09T13:51:15.188798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T15:46:24.190152+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"12.1a","version":"102'1","reported_seq":12,"reported_epoch":102,"state":"active+clean","last_fresh":"2023-08-09T13:51:16.251021+0000","last_change":"2023-08-09T13:51:15.244108+0000","last_active":"2023-08-09T13:51:16.251021+0000","last_peered":"2023-08-09T13:51:16.251021+0000","last_clean":"2023-08-09T13:51:16.251021+0000","last_became_active":"2023-08-09T13:51:15.243803+0000","last_became_peered":"2023-08-09T13:51:15.243803+0000","last_unstale":"2023-08-09T13:51:16.251021+0000","last_undegraded":"2023-08-09T13:51:16.251021+0000","last_fullsized":"2023-08-09T13:51:16.251021+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:51:14.171648+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:51:14.171648+0000","last_clean_scrub_stamp":"2023-08-09T13:51:14.171648+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T14:45:06.425833+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.18","version":"0'0","reported_seq":9,"reported_epoch":102,"state":"active+clean","last_fresh":"2023-08-09T13:51:16.246674+0000","last_change":"2023-08-09T13:51:16.244753+0000","last_active":"2023-08-09T13:51:16.246674+0000","last_peered":"2023-08-09T13:51:16.246674+0000","last_clean":"2023-08-09T13:51:16.246674+0000","last_became_active":"2023-08-09T13:51:16.243791+0000","last_became_peered":"2023-08-09T13:51:16.243791+0000","last_unstale":"2023-08-09T13:51:16.246674+0000","last_undegraded":"2023-08-09T13:51:16.246674+0000","last_fullsized":"2023-08-09T13:51:16.246674+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:51:15.188798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:51:15.188798+0000","last_clean_scrub_stamp":"2023-08-09T13:51:15.188798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T21:06:31.859283+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.19","version":"0'0","reported_seq":11,"reported_epoch":102,"state":"active+clean","last_fresh":"2023-08-09T13:51:16.215477+0000","last_change":"2023-08-09T13:51:15.240282+0000","last_active":"2023-08-09T13:51:16.215477+0000","last_peered":"2023-08-09T13:51:16.215477+0000","last_clean":"2023-08-09T13:51:16.215477+0000","last_became_active":"2023-08-09T13:51:15.239951+0000","last_became_peered":"2023-08-09T13:51:15.239951+0000","last_unstale":"2023-08-09T13:51:16.215477+0000","last_undegraded":"2023-08-09T13:51:16.215477+0000","last_fullsized":"2023-08-09T13:51:16.215477+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:51:14.171648+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:51:14.171648+0000","last_clean_scrub_stamp":"2023-08-09T13:51:14.171648+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-11T00:20:32.090823+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.19","version":"0'0","reported_seq":9,"reported_epoch":102,"state":"active+clean","last_fresh":"2023-08-09T13:51:16.253928+0000","last_change":"2023-08-09T13:51:16.246746+0000","last_active":"2023-08-09T13:51:16.253928+0000","last_peered":"2023-08-09T13:51:16.253928+0000","last_clean":"2023-08-09T13:51:16.253928+0000","last_became_active":"2023-08-09T13:51:16.246460+0000","last_became_peered":"2023-08-09T13:51:16.246460+0000","last_unstale":"2023-08-09T13:51:16.253928+0000","last_undegraded":"2023-08-09T13:51:16.253928+0000","last_fullsized":"2023-08-09T13:51:16.253928+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:51:15.188798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:51:15.188798+0000","last_clean_scrub_stamp":"2023-08-09T13:51:15.188798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T19:19:19.019619+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"12.18","version":"0'0","reported_seq":11,"reported_epoch":102,"state":"active+clean","last_fresh":"2023-08-09T13:51:16.220440+0000","last_change":"2023-08-09T13:51:15.241410+0000","last_active":"2023-08-09T13:51:16.220440+0000","last_peered":"2023-08-09T13:51:16.220440+0000","last_clean":"2023-08-09T13:51:16.220440+0000","last_became_active":"2023-08-09T13:51:15.241032+0000","last_became_peered":"2023-08-09T13:51:15.241032+0000","last_unstale":"2023-08-09T13:51:16.220440+0000","last_undegraded":"2023-08-09T13:51:16.220440+0000","last_fullsized":"2023-08-09T13:51:16.220440+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:51:14.171648+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:51:14.171648+0000","last_clean_scrub_stamp":"2023-08-09T13:51:14.171648+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-11T01:06:02.017780+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.16","version":"0'0","reported_seq":9,"reported_epoch":102,"state":"active+clean","last_fresh":"2023-08-09T13:51:16.255954+0000","last_change":"2023-08-09T13:51:16.252395+0000","last_active":"2023-08-09T13:51:16.255954+0000","last_peered":"2023-08-09T13:51:16.255954+0000","last_clean":"2023-08-09T13:51:16.255954+0000","last_became_active":"2023-08-09T13:51:16.252109+0000","last_became_peered":"2023-08-09T13:51:16.252109+0000","last_unstale":"2023-08-09T13:51:16.255954+0000","last_undegraded":"2023-08-09T13:51:16.255954+0000","last_fullsized":"2023-08-09T13:51:16.255954+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:51:15.188798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:51:15.188798+0000","last_clean_scrub_stamp":"2023-08-09T13:51:15.188798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T19:31:46.435562+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"12.17","version":"0'0","reported_seq":11,"reported_epoch":102,"state":"active+clean","last_fresh":"2023-08-09T13:51:16.238565+0000","last_change":"2023-08-09T13:51:15.238456+0000","last_active":"2023-08-09T13:51:16.238565+0000","last_peered":"2023-08-09T13:51:16.238565+0000","last_clean":"2023-08-09T13:51:16.238565+0000","last_became_active":"2023-08-09T13:51:15.238161+0000","last_became_peered":"2023-08-09T13:51:15.238161+0000","last_unstale":"2023-08-09T13:51:16.238565+0000","last_undegraded":"2023-08-09T13:51:16.238565+0000","last_fullsized":"2023-08-09T13:51:16.238565+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:51:14.171648+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:51:14.171648+0000","last_clean_scrub_stamp":"2023-08-09T13:51:14.171648+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T21:31:20.083059+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.17","version":"0'0","reported_seq":9,"reported_epoch":102,"state":"active+clean","last_fresh":"2023-08-09T13:51:16.233986+0000","last_change":"2023-08-09T13:51:16.230647+0000","last_active":"2023-08-09T13:51:16.233986+0000","last_peered":"2023-08-09T13:51:16.233986+0000","last_clean":"2023-08-09T13:51:16.233986+0000","last_became_active":"2023-08-09T13:51:16.230189+0000","last_became_peered":"2023-08-09T13:51:16.230189+0000","last_unstale":"2023-08-09T13:51:16.233986+0000","last_undegraded":"2023-08-09T13:51:16.233986+0000","last_fullsized":"2023-08-09T13:51:16.233986+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:51:15.188798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:51:15.188798+0000","last_clean_scrub_stamp":"2023-08-09T13:51:15.188798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T21:33:15.159821+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.16","version":"0'0","reported_seq":11,"reported_epoch":102,"state":"active+clean","last_fresh":"2023-08-09T13:51:16.242556+0000","last_change":"2023-08-09T13:51:15.245448+0000","last_active":"2023-08-09T13:51:16.242556+0000","last_peered":"2023-08-09T13:51:16.242556+0000","last_clean":"2023-08-09T13:51:16.242556+0000","last_became_active":"2023-08-09T13:51:15.245025+0000","last_became_peered":"2023-08-09T13:51:15.245025+0000","last_unstale":"2023-08-09T13:51:16.242556+0000","last_undegraded":"2023-08-09T13:51:16.242556+0000","last_fullsized":"2023-08-09T13:51:16.242556+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:51:14.171648+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:51:14.171648+0000","last_clean_scrub_stamp":"2023-08-09T13:51:14.171648+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-11T00:15:29.718414+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.14","version":"0'0","reported_seq":9,"reported_epoch":102,"state":"active+clean","last_fresh":"2023-08-09T13:51:16.259370+0000","last_change":"2023-08-09T13:51:16.257396+0000","last_active":"2023-08-09T13:51:16.259370+0000","last_peered":"2023-08-09T13:51:16.259370+0000","last_clean":"2023-08-09T13:51:16.259370+0000","last_became_active":"2023-08-09T13:51:16.257066+0000","last_became_peered":"2023-08-09T13:51:16.257066+0000","last_unstale":"2023-08-09T13:51:16.259370+0000","last_undegraded":"2023-08-09T13:51:16.259370+0000","last_fullsized":"2023-08-09T13:51:16.259370+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:51:15.188798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:51:15.188798+0000","last_clean_scrub_stamp":"2023-08-09T13:51:15.188798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T14:45:07.442984+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.15","version":"102'3","reported_seq":14,"reported_epoch":102,"state":"active+clean","last_fresh":"2023-08-09T13:51:16.262948+0000","last_change":"2023-08-09T13:51:15.242040+0000","last_active":"2023-08-09T13:51:16.262948+0000","last_peered":"2023-08-09T13:51:16.262948+0000","last_clean":"2023-08-09T13:51:16.262948+0000","last_became_active":"2023-08-09T13:51:15.241758+0000","last_became_peered":"2023-08-09T13:51:15.241758+0000","last_unstale":"2023-08-09T13:51:16.262948+0000","last_undegraded":"2023-08-09T13:51:16.262948+0000","last_fullsized":"2023-08-09T13:51:16.262948+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:51:14.171648+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:51:14.171648+0000","last_clean_scrub_stamp":"2023-08-09T13:51:14.171648+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-11T01:20:04.488085+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.15","version":"0'0","reported_seq":9,"reported_epoch":102,"state":"active+clean","last_fresh":"2023-08-09T13:51:16.245065+0000","last_change":"2023-08-09T13:51:16.241568+0000","last_active":"2023-08-09T13:51:16.245065+0000","last_peered":"2023-08-09T13:51:16.245065+0000","last_clean":"2023-08-09T13:51:16.245065+0000","last_became_active":"2023-08-09T13:51:16.240969+0000","last_became_peered":"2023-08-09T13:51:16.240969+0000","last_unstale":"2023-08-09T13:51:16.245065+0000","last_undegraded":"2023-08-09T13:51:16.245065+0000","last_fullsized":"2023-08-09T13:51:16.245065+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:51:15.188798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:51:15.188798+0000","last_clean_scrub_stamp":"2023-08-09T13:51:15.188798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-11T00:07:45.660155+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"12.14","version":"102'6","reported_seq":17,"reported_epoch":102,"state":"active+clean","last_fresh":"2023-08-09T13:51:16.263913+0000","last_change":"2023-08-09T13:51:15.244541+0000","last_active":"2023-08-09T13:51:16.263913+0000","last_peered":"2023-08-09T13:51:16.263913+0000","last_clean":"2023-08-09T13:51:16.263913+0000","last_became_active":"2023-08-09T13:51:15.244282+0000","last_became_peered":"2023-08-09T13:51:15.244282+0000","last_unstale":"2023-08-09T13:51:16.263913+0000","last_undegraded":"2023-08-09T13:51:16.263913+0000","last_fullsized":"2023-08-09T13:51:16.263913+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:51:14.171648+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:51:14.171648+0000","last_clean_scrub_stamp":"2023-08-09T13:51:14.171648+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T17:00:56.192797+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"13.12","version":"0'0","reported_seq":9,"reported_epoch":102,"state":"active+clean","last_fresh":"2023-08-09T13:51:16.239672+0000","last_change":"2023-08-09T13:51:16.232809+0000","last_active":"2023-08-09T13:51:16.239672+0000","last_peered":"2023-08-09T13:51:16.239672+0000","last_clean":"2023-08-09T13:51:16.239672+0000","last_became_active":"2023-08-09T13:51:16.232525+0000","last_became_peered":"2023-08-09T13:51:16.232525+0000","last_unstale":"2023-08-09T13:51:16.239672+0000","last_undegraded":"2023-08-09T13:51:16.239672+0000","last_fullsized":"2023-08-09T13:51:16.239672+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:51:15.188798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:51:15.188798+0000","last_clean_scrub_stamp":"2023-08-09T13:51:15.188798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T19:40:21.994081+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.13","version":"102'2","reported_seq":13,"reported_epoch":102,"state":"active+clean","last_fresh":"2023-08-09T13:51:16.259408+0000","last_change":"2023-08-09T13:51:15.226969+0000","last_active":"2023-08-09T13:51:16.259408+0000","last_peered":"2023-08-09T13:51:16.259408+0000","last_clean":"2023-08-09T13:51:16.259408+0000","last_became_active":"2023-08-09T13:51:15.226656+0000","last_became_peered":"2023-08-09T13:51:15.226656+0000","last_unstale":"2023-08-09T13:51:16.259408+0000","last_undegraded":"2023-08-09T13:51:16.259408+0000","last_fullsized":"2023-08-09T13:51:16.259408+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:51:14.171648+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:51:14.171648+0000","last_clean_scrub_stamp":"2023-08-09T13:51:14.171648+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-11T01:08:20.115321+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.13","version":"0'0","reported_seq":9,"reported_epoch":102,"state":"active+clean","last_fresh":"2023-08-09T13:51:16.247101+0000","last_change":"2023-08-09T13:51:16.244883+0000","last_active":"2023-08-09T13:51:16.247101+0000","last_peered":"2023-08-09T13:51:16.247101+0000","last_clean":"2023-08-09T13:51:16.247101+0000","last_became_active":"2023-08-09T13:51:16.244568+0000","last_became_peered":"2023-08-09T13:51:16.244568+0000","last_unstale":"2023-08-09T13:51:16.247101+0000","last_undegraded":"2023-08-09T13:51:16.247101+0000","last_fullsized":"2023-08-09T13:51:16.247101+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:51:15.188798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:51:15.188798+0000","last_clean_scrub_stamp":"2023-08-09T13:51:15.188798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T21:31:37.211388+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.12","version":"0'0","reported_seq":11,"reported_epoch":102,"state":"active+clean","last_fresh":"2023-08-09T13:51:16.231275+0000","last_change":"2023-08-09T13:51:15.234514+0000","last_active":"2023-08-09T13:51:16.231275+0000","last_peered":"2023-08-09T13:51:16.231275+0000","last_clean":"2023-08-09T13:51:16.231275+0000","last_became_active":"2023-08-09T13:51:15.234183+0000","last_became_peered":"2023-08-09T13:51:15.234183+0000","last_unstale":"2023-08-09T13:51:16.231275+0000","last_undegraded":"2023-08-09T13:51:16.231275+0000","last_fullsized":"2023-08-09T13:51:16.231275+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:51:14.171648+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:51:14.171648+0000","last_clean_scrub_stamp":"2023-08-09T13:51:14.171648+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T19:56:40.377835+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"13.10","version":"0'0","reported_seq":9,"reported_epoch":102,"state":"active+clean","last_fresh":"2023-08-09T13:51:16.249779+0000","last_change":"2023-08-09T13:51:16.241729+0000","last_active":"2023-08-09T13:51:16.249779+0000","last_peered":"2023-08-09T13:51:16.249779+0000","last_clean":"2023-08-09T13:51:16.249779+0000","last_became_active":"2023-08-09T13:51:16.241367+0000","last_became_peered":"2023-08-09T13:51:16.241367+0000","last_unstale":"2023-08-09T13:51:16.249779+0000","last_undegraded":"2023-08-09T13:51:16.249779+0000","last_fullsized":"2023-08-09T13:51:16.249779+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:51:15.188798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:51:15.188798+0000","last_clean_scrub_stamp":"2023-08-09T13:51:15.188798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T18:12:30.400222+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.11","version":"0'0","reported_seq":11,"reported_epoch":102,"state":"active+clean","last_fresh":"2023-08-09T13:51:16.220629+0000","last_change":"2023-08-09T13:51:15.243824+0000","last_active":"2023-08-09T13:51:16.220629+0000","last_peered":"2023-08-09T13:51:16.220629+0000","last_clean":"2023-08-09T13:51:16.220629+0000","last_became_active":"2023-08-09T13:51:15.243491+0000","last_became_peered":"2023-08-09T13:51:15.243491+0000","last_unstale":"2023-08-09T13:51:16.220629+0000","last_undegraded":"2023-08-09T13:51:16.220629+0000","last_fullsized":"2023-08-09T13:51:16.220629+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:51:14.171648+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:51:14.171648+0000","last_clean_scrub_stamp":"2023-08-09T13:51:14.171648+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T19:04:31.316491+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"13.11","version":"0'0","reported_seq":9,"reported_epoch":102,"state":"active+clean","last_fresh":"2023-08-09T13:51:16.262715+0000","last_change":"2023-08-09T13:51:16.257254+0000","last_active":"2023-08-09T13:51:16.262715+0000","last_peered":"2023-08-09T13:51:16.262715+0000","last_clean":"2023-08-09T13:51:16.262715+0000","last_became_active":"2023-08-09T13:51:16.256967+0000","last_became_peered":"2023-08-09T13:51:16.256967+0000","last_unstale":"2023-08-09T13:51:16.262715+0000","last_undegraded":"2023-08-09T13:51:16.262715+0000","last_fullsized":"2023-08-09T13:51:16.262715+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:51:15.188798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:51:15.188798+0000","last_clean_scrub_stamp":"2023-08-09T13:51:15.188798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T23:46:26.444683+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.10","version":"102'8","reported_seq":19,"reported_epoch":102,"state":"active+clean","last_fresh":"2023-08-09T13:51:16.250969+0000","last_change":"2023-08-09T13:51:15.233631+0000","last_active":"2023-08-09T13:51:16.250969+0000","last_peered":"2023-08-09T13:51:16.250969+0000","last_clean":"2023-08-09T13:51:16.250969+0000","last_became_active":"2023-08-09T13:51:15.233323+0000","last_became_peered":"2023-08-09T13:51:15.233323+0000","last_unstale":"2023-08-09T13:51:16.250969+0000","last_undegraded":"2023-08-09T13:51:16.250969+0000","last_fullsized":"2023-08-09T13:51:16.250969+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:51:14.171648+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:51:14.171648+0000","last_clean_scrub_stamp":"2023-08-09T13:51:14.171648+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T17:29:03.609676+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.6","version":"0'0","reported_seq":9,"reported_epoch":102,"state":"active+clean","last_fresh":"2023-08-09T13:51:16.254026+0000","last_change":"2023-08-09T13:51:16.246989+0000","last_active":"2023-08-09T13:51:16.254026+0000","last_peered":"2023-08-09T13:51:16.254026+0000","last_clean":"2023-08-09T13:51:16.254026+0000","last_became_active":"2023-08-09T13:51:16.246731+0000","last_became_peered":"2023-08-09T13:51:16.246731+0000","last_unstale":"2023-08-09T13:51:16.254026+0000","last_undegraded":"2023-08-09T13:51:16.254026+0000","last_fullsized":"2023-08-09T13:51:16.254026+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:51:15.188798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:51:15.188798+0000","last_clean_scrub_stamp":"2023-08-09T13:51:15.188798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-11T00:47:48.385325+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"12.7","version":"102'2","reported_seq":13,"reported_epoch":102,"state":"active+clean","last_fresh":"2023-08-09T13:51:16.253593+0000","last_change":"2023-08-09T13:51:15.228814+0000","last_active":"2023-08-09T13:51:16.253593+0000","last_peered":"2023-08-09T13:51:16.253593+0000","last_clean":"2023-08-09T13:51:16.253593+0000","last_became_active":"2023-08-09T13:51:15.227947+0000","last_became_peered":"2023-08-09T13:51:15.227947+0000","last_unstale":"2023-08-09T13:51:16.253593+0000","last_undegraded":"2023-08-09T13:51:16.253593+0000","last_fullsized":"2023-08-09T13:51:16.253593+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:51:14.171648+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:51:14.171648+0000","last_clean_scrub_stamp":"2023-08-09T13:51:14.171648+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-11T01:50:49.808694+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.7","version":"0'0","reported_seq":9,"reported_epoch":102,"state":"active+clean","last_fresh":"2023-08-09T13:51:16.257386+0000","last_change":"2023-08-09T13:51:16.250017+0000","last_active":"2023-08-09T13:51:16.257386+0000","last_peered":"2023-08-09T13:51:16.257386+0000","last_clean":"2023-08-09T13:51:16.257386+0000","last_became_active":"2023-08-09T13:51:16.249750+0000","last_became_peered":"2023-08-09T13:51:16.249750+0000","last_unstale":"2023-08-09T13:51:16.257386+0000","last_undegraded":"2023-08-09T13:51:16.257386+0000","last_fullsized":"2023-08-09T13:51:16.257386+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:51:15.188798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:51:15.188798+0000","last_clean_scrub_stamp":"2023-08-09T13:51:15.188798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T15:17:36.104232+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.6","version":"0'0","reported_seq":11,"reported_epoch":102,"state":"active+clean","last_fresh":"2023-08-09T13:51:16.223060+0000","last_change":"2023-08-09T13:51:15.227422+0000","last_active":"2023-08-09T13:51:16.223060+0000","last_peered":"2023-08-09T13:51:16.223060+0000","last_clean":"2023-08-09T13:51:16.223060+0000","last_became_active":"2023-08-09T13:51:15.227114+0000","last_became_peered":"2023-08-09T13:51:15.227114+0000","last_unstale":"2023-08-09T13:51:16.223060+0000","last_undegraded":"2023-08-09T13:51:16.223060+0000","last_fullsized":"2023-08-09T13:51:16.223060+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:51:14.171648+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:51:14.171648+0000","last_clean_scrub_stamp":"2023-08-09T13:51:14.171648+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T18:00:32.163746+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.4","version":"0'0","reported_seq":9,"reported_epoch":102,"state":"active+clean","last_fresh":"2023-08-09T13:51:16.259425+0000","last_change":"2023-08-09T13:51:16.254767+0000","last_active":"2023-08-09T13:51:16.259425+0000","last_peered":"2023-08-09T13:51:16.259425+0000","last_clean":"2023-08-09T13:51:16.259425+0000","last_became_active":"2023-08-09T13:51:16.254146+0000","last_became_peered":"2023-08-09T13:51:16.254146+0000","last_unstale":"2023-08-09T13:51:16.259425+0000","last_undegraded":"2023-08-09T13:51:16.259425+0000","last_fullsized":"2023-08-09T13:51:16.259425+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:51:15.188798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:51:15.188798+0000","last_clean_scrub_stamp":"2023-08-09T13:51:15.188798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T22:04:09.500719+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.5","version":"0'0","reported_seq":11,"reported_epoch":102,"state":"active+clean","last_fresh":"2023-08-09T13:51:16.233809+0000","last_change":"2023-08-09T13:51:15.241210+0000","last_active":"2023-08-09T13:51:16.233809+0000","last_peered":"2023-08-09T13:51:16.233809+0000","last_clean":"2023-08-09T13:51:16.233809+0000","last_became_active":"2023-08-09T13:51:15.240939+0000","last_became_peered":"2023-08-09T13:51:15.240939+0000","last_unstale":"2023-08-09T13:51:16.233809+0000","last_undegraded":"2023-08-09T13:51:16.233809+0000","last_fullsized":"2023-08-09T13:51:16.233809+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:51:14.171648+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:51:14.171648+0000","last_clean_scrub_stamp":"2023-08-09T13:51:14.171648+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T23:11:40.454290+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.5","version":"0'0","reported_seq":9,"reported_epoch":102,"state":"active+clean","last_fresh":"2023-08-09T13:51:16.245044+0000","last_change":"2023-08-09T13:51:16.241394+0000","last_active":"2023-08-09T13:51:16.245044+0000","last_peered":"2023-08-09T13:51:16.245044+0000","last_clean":"2023-08-09T13:51:16.245044+0000","last_became_active":"2023-08-09T13:51:16.240570+0000","last_became_peered":"2023-08-09T13:51:16.240570+0000","last_unstale":"2023-08-09T13:51:16.245044+0000","last_undegraded":"2023-08-09T13:51:16.245044+0000","last_fullsized":"2023-08-09T13:51:16.245044+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:51:15.188798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:51:15.188798+0000","last_clean_scrub_stamp":"2023-08-09T13:51:15.188798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T18:43:48.957746+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"12.4","version":"0'0","reported_seq":11,"reported_epoch":102,"state":"active+clean","last_fresh":"2023-08-09T13:51:16.240345+0000","last_change":"2023-08-09T13:51:15.235875+0000","last_active":"2023-08-09T13:51:16.240345+0000","last_peered":"2023-08-09T13:51:16.240345+0000","last_clean":"2023-08-09T13:51:16.240345+0000","last_became_active":"2023-08-09T13:51:15.235611+0000","last_became_peered":"2023-08-09T13:51:15.235611+0000","last_unstale":"2023-08-09T13:51:16.240345+0000","last_undegraded":"2023-08-09T13:51:16.240345+0000","last_fullsized":"2023-08-09T13:51:16.240345+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:51:14.171648+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:51:14.171648+0000","last_clean_scrub_stamp":"2023-08-09T13:51:14.171648+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T20:19:11.130074+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.2","version":"0'0","reported_seq":9,"reported_epoch":102,"state":"active+clean","last_fresh":"2023-08-09T13:51:16.253866+0000","last_change":"2023-08-09T13:51:16.246505+0000","last_active":"2023-08-09T13:51:16.253866+0000","last_peered":"2023-08-09T13:51:16.253866+0000","last_clean":"2023-08-09T13:51:16.253866+0000","last_became_active":"2023-08-09T13:51:16.246118+0000","last_became_peered":"2023-08-09T13:51:16.246118+0000","last_unstale":"2023-08-09T13:51:16.253866+0000","last_undegraded":"2023-08-09T13:51:16.253866+0000","last_fullsized":"2023-08-09T13:51:16.253866+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:51:15.188798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:51:15.188798+0000","last_clean_scrub_stamp":"2023-08-09T13:51:15.188798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T18:27:52.202303+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.3","version":"0'0","reported_seq":11,"reported_epoch":102,"state":"active+clean","last_fresh":"2023-08-09T13:51:16.216741+0000","last_change":"2023-08-09T13:51:15.235344+0000","last_active":"2023-08-09T13:51:16.216741+0000","last_peered":"2023-08-09T13:51:16.216741+0000","last_clean":"2023-08-09T13:51:16.216741+0000","last_became_active":"2023-08-09T13:51:15.235037+0000","last_became_peered":"2023-08-09T13:51:15.235037+0000","last_unstale":"2023-08-09T13:51:16.216741+0000","last_undegraded":"2023-08-09T13:51:16.216741+0000","last_fullsized":"2023-08-09T13:51:16.216741+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:51:14.171648+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:51:14.171648+0000","last_clean_scrub_stamp":"2023-08-09T13:51:14.171648+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T23:42:00.578777+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.3","version":"0'0","reported_seq":9,"reported_epoch":102,"state":"active+clean","last_fresh":"2023-08-09T13:51:16.251027+0000","last_change":"2023-08-09T13:51:16.240636+0000","last_active":"2023-08-09T13:51:16.251027+0000","last_peered":"2023-08-09T13:51:16.251027+0000","last_clean":"2023-08-09T13:51:16.251027+0000","last_became_active":"2023-08-09T13:51:16.240337+0000","last_became_peered":"2023-08-09T13:51:16.240337+0000","last_unstale":"2023-08-09T13:51:16.251027+0000","last_undegraded":"2023-08-09T13:51:16.251027+0000","last_fullsized":"2023-08-09T13:51:16.251027+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:51:15.188798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:51:15.188798+0000","last_clean_scrub_stamp":"2023-08-09T13:51:15.188798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T18:12:35.312144+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.2","version":"0'0","reported_seq":11,"reported_epoch":102,"state":"active+clean","last_fresh":"2023-08-09T13:51:16.224695+0000","last_change":"2023-08-09T13:51:15.241593+0000","last_active":"2023-08-09T13:51:16.224695+0000","last_peered":"2023-08-09T13:51:16.224695+0000","last_clean":"2023-08-09T13:51:16.224695+0000","last_became_active":"2023-08-09T13:51:15.241319+0000","last_became_peered":"2023-08-09T13:51:15.241319+0000","last_unstale":"2023-08-09T13:51:16.224695+0000","last_undegraded":"2023-08-09T13:51:16.224695+0000","last_fullsized":"2023-08-09T13:51:16.224695+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:51:14.171648+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:51:14.171648+0000","last_clean_scrub_stamp":"2023-08-09T13:51:14.171648+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T18:12:34.294994+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.0","version":"0'0","reported_seq":9,"reported_epoch":102,"state":"active+clean","last_fresh":"2023-08-09T13:51:16.257346+0000","last_change":"2023-08-09T13:51:16.252480+0000","last_active":"2023-08-09T13:51:16.257346+0000","last_peered":"2023-08-09T13:51:16.257346+0000","last_clean":"2023-08-09T13:51:16.257346+0000","last_became_active":"2023-08-09T13:51:16.252206+0000","last_became_peered":"2023-08-09T13:51:16.252206+0000","last_unstale":"2023-08-09T13:51:16.257346+0000","last_undegraded":"2023-08-09T13:51:16.257346+0000","last_fullsized":"2023-08-09T13:51:16.257346+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:51:15.188798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:51:15.188798+0000","last_clean_scrub_stamp":"2023-08-09T13:51:15.188798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T17:04:13.279977+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.1","version":"0'0","reported_seq":11,"reported_epoch":102,"state":"active+clean","last_fresh":"2023-08-09T13:51:16.238942+0000","last_change":"2023-08-09T13:51:15.239117+0000","last_active":"2023-08-09T13:51:16.238942+0000","last_peered":"2023-08-09T13:51:16.238942+0000","last_clean":"2023-08-09T13:51:16.238942+0000","last_became_active":"2023-08-09T13:51:15.238862+0000","last_became_peered":"2023-08-09T13:51:15.238862+0000","last_unstale":"2023-08-09T13:51:16.238942+0000","last_undegraded":"2023-08-09T13:51:16.238942+0000","last_fullsized":"2023-08-09T13:51:16.238942+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:51:14.171648+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:51:14.171648+0000","last_clean_scrub_stamp":"2023-08-09T13:51:14.171648+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-11T00:32:53.371031+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.1","version":"0'0","reported_seq":9,"reported_epoch":102,"state":"active+clean","last_fresh":"2023-08-09T13:51:16.253203+0000","last_change":"2023-08-09T13:51:16.248941+0000","last_active":"2023-08-09T13:51:16.253203+0000","last_peered":"2023-08-09T13:51:16.253203+0000","last_clean":"2023-08-09T13:51:16.253203+0000","last_became_active":"2023-08-09T13:51:16.248630+0000","last_became_peered":"2023-08-09T13:51:16.248630+0000","last_unstale":"2023-08-09T13:51:16.253203+0000","last_undegraded":"2023-08-09T13:51:16.253203+0000","last_fullsized":"2023-08-09T13:51:16.253203+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:51:15.188798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:51:15.188798+0000","last_clean_scrub_stamp":"2023-08-09T13:51:15.188798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T16:33:43.728000+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.0","version":"0'0","reported_seq":11,"reported_epoch":102,"state":"active+clean","last_fresh":"2023-08-09T13:51:16.222583+0000","last_change":"2023-08-09T13:51:15.245201+0000","last_active":"2023-08-09T13:51:16.222583+0000","last_peered":"2023-08-09T13:51:16.222583+0000","last_clean":"2023-08-09T13:51:16.222583+0000","last_became_active":"2023-08-09T13:51:15.244899+0000","last_became_peered":"2023-08-09T13:51:15.244899+0000","last_unstale":"2023-08-09T13:51:16.222583+0000","last_undegraded":"2023-08-09T13:51:16.222583+0000","last_fullsized":"2023-08-09T13:51:16.222583+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:51:14.171648+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:51:14.171648+0000","last_clean_scrub_stamp":"2023-08-09T13:51:14.171648+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T18:26:02.112097+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.c","version":"0'0","reported_seq":9,"reported_epoch":102,"state":"active+clean","last_fresh":"2023-08-09T13:51:16.252537+0000","last_change":"2023-08-09T13:51:16.232600+0000","last_active":"2023-08-09T13:51:16.252537+0000","last_peered":"2023-08-09T13:51:16.252537+0000","last_clean":"2023-08-09T13:51:16.252537+0000","last_became_active":"2023-08-09T13:51:16.232287+0000","last_became_peered":"2023-08-09T13:51:16.232287+0000","last_unstale":"2023-08-09T13:51:16.252537+0000","last_undegraded":"2023-08-09T13:51:16.252537+0000","last_fullsized":"2023-08-09T13:51:16.252537+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:51:15.188798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:51:15.188798+0000","last_clean_scrub_stamp":"2023-08-09T13:51:15.188798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T18:52:12.499231+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.d","version":"102'1","reported_seq":12,"reported_epoch":102,"state":"active+clean","last_fresh":"2023-08-09T13:51:16.262913+0000","last_change":"2023-08-09T13:51:15.233806+0000","last_active":"2023-08-09T13:51:16.262913+0000","last_peered":"2023-08-09T13:51:16.262913+0000","last_clean":"2023-08-09T13:51:16.262913+0000","last_became_active":"2023-08-09T13:51:15.233316+0000","last_became_peered":"2023-08-09T13:51:15.233316+0000","last_unstale":"2023-08-09T13:51:16.262913+0000","last_undegraded":"2023-08-09T13:51:16.262913+0000","last_fullsized":"2023-08-09T13:51:16.262913+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:51:14.171648+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:51:14.171648+0000","last_clean_scrub_stamp":"2023-08-09T13:51:14.171648+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T23:05:57.606883+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":169,"reported_epoch":102,"state":"active+clean","last_fresh":"2023-08-09T13:51:16.229542+0000","last_change":"2023-08-09T13:33:56.401406+0000","last_active":"2023-08-09T13:51:16.229542+0000","last_peered":"2023-08-09T13:51:16.229542+0000","last_clean":"2023-08-09T13:51:16.229542+0000","last_became_active":"2023-08-09T13:33:56.400487+0000","last_became_peered":"2023-08-09T13:33:56.400487+0000","last_unstale":"2023-08-09T13:51:16.229542+0000","last_undegraded":"2023-08-09T13:51:16.229542+0000","last_fullsized":"2023-08-09T13:51:16.229542+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:33:54.845869+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:33:54.845869+0000","last_clean_scrub_stamp":"2023-08-09T13:33:54.845869+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T23:03:24.702690+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"13.d","version":"0'0","reported_seq":9,"reported_epoch":102,"state":"active+clean","last_fresh":"2023-08-09T13:51:16.257651+0000","last_change":"2023-08-09T13:51:16.255187+0000","last_active":"2023-08-09T13:51:16.257651+0000","last_peered":"2023-08-09T13:51:16.257651+0000","last_clean":"2023-08-09T13:51:16.257651+0000","last_became_active":"2023-08-09T13:51:16.254860+0000","last_became_peered":"2023-08-09T13:51:16.254860+0000","last_unstale":"2023-08-09T13:51:16.257651+0000","last_undegraded":"2023-08-09T13:51:16.257651+0000","last_fullsized":"2023-08-09T13:51:16.257651+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:51:15.188798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:51:15.188798+0000","last_clean_scrub_stamp":"2023-08-09T13:51:15.188798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T17:50:15.038705+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"12.c","version":"0'0","reported_seq":11,"reported_epoch":102,"state":"active+clean","last_fresh":"2023-08-09T13:51:16.215940+0000","last_change":"2023-08-09T13:51:15.241814+0000","last_active":"2023-08-09T13:51:16.215940+0000","last_peered":"2023-08-09T13:51:16.215940+0000","last_clean":"2023-08-09T13:51:16.215940+0000","last_became_active":"2023-08-09T13:51:15.241491+0000","last_became_peered":"2023-08-09T13:51:15.241491+0000","last_unstale":"2023-08-09T13:51:16.215940+0000","last_undegraded":"2023-08-09T13:51:16.215940+0000","last_fullsized":"2023-08-09T13:51:16.215940+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:51:14.171648+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:51:14.171648+0000","last_clean_scrub_stamp":"2023-08-09T13:51:14.171648+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T18:26:02.112097+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":173,"reported_epoch":102,"state":"active+clean","last_fresh":"2023-08-09T13:51:16.220850+0000","last_change":"2023-08-09T13:33:56.401123+0000","last_active":"2023-08-09T13:51:16.220850+0000","last_peered":"2023-08-09T13:51:16.220850+0000","last_clean":"2023-08-09T13:51:16.220850+0000","last_became_active":"2023-08-09T13:33:56.399933+0000","last_became_peered":"2023-08-09T13:33:56.399933+0000","last_unstale":"2023-08-09T13:51:16.220850+0000","last_undegraded":"2023-08-09T13:51:16.220850+0000","last_fullsized":"2023-08-09T13:51:16.220850+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:33:54.845869+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:33:54.845869+0000","last_clean_scrub_stamp":"2023-08-09T13:33:54.845869+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T18:30:45.254362+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"13.e","version":"0'0","reported_seq":9,"reported_epoch":102,"state":"active+clean","last_fresh":"2023-08-09T13:51:16.257418+0000","last_change":"2023-08-09T13:51:16.251088+0000","last_active":"2023-08-09T13:51:16.257418+0000","last_peered":"2023-08-09T13:51:16.257418+0000","last_clean":"2023-08-09T13:51:16.257418+0000","last_became_active":"2023-08-09T13:51:16.250820+0000","last_became_peered":"2023-08-09T13:51:16.250820+0000","last_unstale":"2023-08-09T13:51:16.257418+0000","last_undegraded":"2023-08-09T13:51:16.257418+0000","last_fullsized":"2023-08-09T13:51:16.257418+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:51:15.188798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:51:15.188798+0000","last_clean_scrub_stamp":"2023-08-09T13:51:15.188798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T14:17:16.947840+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.f","version":"102'1","reported_seq":12,"reported_epoch":102,"state":"active+clean","last_fresh":"2023-08-09T13:51:16.258359+0000","last_change":"2023-08-09T13:51:15.243510+0000","last_active":"2023-08-09T13:51:16.258359+0000","last_peered":"2023-08-09T13:51:16.258359+0000","last_clean":"2023-08-09T13:51:16.258359+0000","last_became_active":"2023-08-09T13:51:15.242763+0000","last_became_peered":"2023-08-09T13:51:15.242763+0000","last_unstale":"2023-08-09T13:51:16.258359+0000","last_undegraded":"2023-08-09T13:51:16.258359+0000","last_fullsized":"2023-08-09T13:51:16.258359+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:51:14.171648+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:51:14.171648+0000","last_clean_scrub_stamp":"2023-08-09T13:51:14.171648+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T17:01:45.085632+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"1.2","version":"0'0","reported_seq":169,"reported_epoch":102,"state":"active+clean","last_fresh":"2023-08-09T13:51:16.228756+0000","last_change":"2023-08-09T13:33:56.402794+0000","last_active":"2023-08-09T13:51:16.228756+0000","last_peered":"2023-08-09T13:51:16.228756+0000","last_clean":"2023-08-09T13:51:16.228756+0000","last_became_active":"2023-08-09T13:33:56.402038+0000","last_became_peered":"2023-08-09T13:33:56.402038+0000","last_unstale":"2023-08-09T13:51:16.228756+0000","last_undegraded":"2023-08-09T13:51:16.228756+0000","last_fullsized":"2023-08-09T13:51:16.228756+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:33:54.845869+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:33:54.845869+0000","last_clean_scrub_stamp":"2023-08-09T13:33:54.845869+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T19:04:24.852783+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"13.f","version":"0'0","reported_seq":9,"reported_epoch":102,"state":"active+clean","last_fresh":"2023-08-09T13:51:16.258516+0000","last_change":"2023-08-09T13:51:16.248412+0000","last_active":"2023-08-09T13:51:16.258516+0000","last_peered":"2023-08-09T13:51:16.258516+0000","last_clean":"2023-08-09T13:51:16.258516+0000","last_became_active":"2023-08-09T13:51:16.247925+0000","last_became_peered":"2023-08-09T13:51:16.247925+0000","last_unstale":"2023-08-09T13:51:16.258516+0000","last_undegraded":"2023-08-09T13:51:16.258516+0000","last_fullsized":"2023-08-09T13:51:16.258516+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:51:15.188798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:51:15.188798+0000","last_clean_scrub_stamp":"2023-08-09T13:51:15.188798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T18:16:15.512137+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.e","version":"102'2","reported_seq":13,"reported_epoch":102,"state":"active+clean","last_fresh":"2023-08-09T13:51:16.240072+0000","last_change":"2023-08-09T13:51:15.240950+0000","last_active":"2023-08-09T13:51:16.240072+0000","last_peered":"2023-08-09T13:51:16.240072+0000","last_clean":"2023-08-09T13:51:16.240072+0000","last_became_active":"2023-08-09T13:51:15.240595+0000","last_became_peered":"2023-08-09T13:51:15.240595+0000","last_unstale":"2023-08-09T13:51:16.240072+0000","last_undegraded":"2023-08-09T13:51:16.240072+0000","last_fullsized":"2023-08-09T13:51:16.240072+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:51:14.171648+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:51:14.171648+0000","last_clean_scrub_stamp":"2023-08-09T13:51:14.171648+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T15:46:23.173001+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.3","version":"0'0","reported_seq":169,"reported_epoch":102,"state":"active+clean","last_fresh":"2023-08-09T13:51:16.221138+0000","last_change":"2023-08-09T13:33:55.886749+0000","last_active":"2023-08-09T13:51:16.221138+0000","last_peered":"2023-08-09T13:51:16.221138+0000","last_clean":"2023-08-09T13:51:16.221138+0000","last_became_active":"2023-08-09T13:33:55.886116+0000","last_became_peered":"2023-08-09T13:33:55.886116+0000","last_unstale":"2023-08-09T13:51:16.221138+0000","last_undegraded":"2023-08-09T13:51:16.221138+0000","last_fullsized":"2023-08-09T13:51:16.221138+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:33:54.845869+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:33:54.845869+0000","last_clean_scrub_stamp":"2023-08-09T13:33:54.845869+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T21:23:05.575670+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"92'175","reported_seq":524,"reported_epoch":102,"state":"active+clean","last_fresh":"2023-08-09T13:51:16.229403+0000","last_change":"2023-08-09T13:33:55.886345+0000","last_active":"2023-08-09T13:51:16.229403+0000","last_peered":"2023-08-09T13:51:16.229403+0000","last_clean":"2023-08-09T13:51:16.229403+0000","last_became_active":"2023-08-09T13:33:55.885781+0000","last_became_peered":"2023-08-09T13:33:55.885781+0000","last_unstale":"2023-08-09T13:51:16.229403+0000","last_undegraded":"2023-08-09T13:51:16.229403+0000","last_fullsized":"2023-08-09T13:51:16.229403+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:33:54.845869+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:33:54.845869+0000","last_clean_scrub_stamp":"2023-08-09T13:33:54.845869+0000","objects_scrubbed":0,"log_size":175,"log_dups_size":0,"ondisk_log_size":175,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T13:41:45.285281+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":619,"num_read_kb":1212,"num_write":263,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"13.8","version":"0'0","reported_seq":9,"reported_epoch":102,"state":"active+clean","last_fresh":"2023-08-09T13:51:16.246718+0000","last_change":"2023-08-09T13:51:16.239728+0000","last_active":"2023-08-09T13:51:16.246718+0000","last_peered":"2023-08-09T13:51:16.246718+0000","last_clean":"2023-08-09T13:51:16.246718+0000","last_became_active":"2023-08-09T13:51:16.239354+0000","last_became_peered":"2023-08-09T13:51:16.239354+0000","last_unstale":"2023-08-09T13:51:16.246718+0000","last_undegraded":"2023-08-09T13:51:16.246718+0000","last_fullsized":"2023-08-09T13:51:16.246718+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:51:15.188798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:51:15.188798+0000","last_clean_scrub_stamp":"2023-08-09T13:51:15.188798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T16:12:43.536183+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.9","version":"0'0","reported_seq":11,"reported_epoch":102,"state":"active+clean","last_fresh":"2023-08-09T13:51:16.237041+0000","last_change":"2023-08-09T13:51:15.243807+0000","last_active":"2023-08-09T13:51:16.237041+0000","last_peered":"2023-08-09T13:51:16.237041+0000","last_clean":"2023-08-09T13:51:16.237041+0000","last_became_active":"2023-08-09T13:51:15.243444+0000","last_became_peered":"2023-08-09T13:51:15.243444+0000","last_unstale":"2023-08-09T13:51:16.237041+0000","last_undegraded":"2023-08-09T13:51:16.237041+0000","last_fullsized":"2023-08-09T13:51:16.237041+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:51:14.171648+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:51:14.171648+0000","last_clean_scrub_stamp":"2023-08-09T13:51:14.171648+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-11T00:45:40.422997+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":179,"reported_epoch":102,"state":"active+clean","last_fresh":"2023-08-09T13:51:16.230041+0000","last_change":"2023-08-09T13:33:55.889565+0000","last_active":"2023-08-09T13:51:16.230041+0000","last_peered":"2023-08-09T13:51:16.230041+0000","last_clean":"2023-08-09T13:51:16.230041+0000","last_became_active":"2023-08-09T13:33:55.889164+0000","last_became_peered":"2023-08-09T13:33:55.889164+0000","last_unstale":"2023-08-09T13:51:16.230041+0000","last_undegraded":"2023-08-09T13:51:16.230041+0000","last_fullsized":"2023-08-09T13:51:16.230041+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:33:54.845869+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:33:54.845869+0000","last_clean_scrub_stamp":"2023-08-09T13:33:54.845869+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T18:48:05.967165+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"13.9","version":"0'0","reported_seq":9,"reported_epoch":102,"state":"active+clean","last_fresh":"2023-08-09T13:51:16.234817+0000","last_change":"2023-08-09T13:51:16.232672+0000","last_active":"2023-08-09T13:51:16.234817+0000","last_peered":"2023-08-09T13:51:16.234817+0000","last_clean":"2023-08-09T13:51:16.234817+0000","last_became_active":"2023-08-09T13:51:16.232084+0000","last_became_peered":"2023-08-09T13:51:16.232084+0000","last_unstale":"2023-08-09T13:51:16.234817+0000","last_undegraded":"2023-08-09T13:51:16.234817+0000","last_fullsized":"2023-08-09T13:51:16.234817+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:51:15.188798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:51:15.188798+0000","last_clean_scrub_stamp":"2023-08-09T13:51:15.188798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T17:25:36.877280+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.8","version":"0'0","reported_seq":11,"reported_epoch":102,"state":"active+clean","last_fresh":"2023-08-09T13:51:16.229763+0000","last_change":"2023-08-09T13:51:15.231846+0000","last_active":"2023-08-09T13:51:16.229763+0000","last_peered":"2023-08-09T13:51:16.229763+0000","last_clean":"2023-08-09T13:51:16.229763+0000","last_became_active":"2023-08-09T13:51:15.231559+0000","last_became_peered":"2023-08-09T13:51:15.231559+0000","last_unstale":"2023-08-09T13:51:16.229763+0000","last_undegraded":"2023-08-09T13:51:16.229763+0000","last_fullsized":"2023-08-09T13:51:16.229763+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:51:14.171648+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:51:14.171648+0000","last_clean_scrub_stamp":"2023-08-09T13:51:14.171648+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T17:12:30.175796+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":173,"reported_epoch":102,"state":"active+clean","last_fresh":"2023-08-09T13:51:16.228648+0000","last_change":"2023-08-09T13:33:56.404236+0000","last_active":"2023-08-09T13:51:16.228648+0000","last_peered":"2023-08-09T13:51:16.228648+0000","last_clean":"2023-08-09T13:51:16.228648+0000","last_became_active":"2023-08-09T13:33:56.402485+0000","last_became_peered":"2023-08-09T13:33:56.402485+0000","last_unstale":"2023-08-09T13:51:16.228648+0000","last_undegraded":"2023-08-09T13:51:16.228648+0000","last_fullsized":"2023-08-09T13:51:16.228648+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:33:54.845869+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:33:54.845869+0000","last_clean_scrub_stamp":"2023-08-09T13:33:54.845869+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T22:56:55.178189+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.a","version":"0'0","reported_seq":9,"reported_epoch":102,"state":"active+clean","last_fresh":"2023-08-09T13:51:16.258171+0000","last_change":"2023-08-09T13:51:16.254430+0000","last_active":"2023-08-09T13:51:16.258171+0000","last_peered":"2023-08-09T13:51:16.258171+0000","last_clean":"2023-08-09T13:51:16.258171+0000","last_became_active":"2023-08-09T13:51:16.252692+0000","last_became_peered":"2023-08-09T13:51:16.252692+0000","last_unstale":"2023-08-09T13:51:16.258171+0000","last_undegraded":"2023-08-09T13:51:16.258171+0000","last_fullsized":"2023-08-09T13:51:16.258171+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:51:15.188798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:51:15.188798+0000","last_clean_scrub_stamp":"2023-08-09T13:51:15.188798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T17:51:29.452605+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.b","version":"102'2","reported_seq":13,"reported_epoch":102,"state":"active+clean","last_fresh":"2023-08-09T13:51:16.251780+0000","last_change":"2023-08-09T13:51:15.236904+0000","last_active":"2023-08-09T13:51:16.251780+0000","last_peered":"2023-08-09T13:51:16.251780+0000","last_clean":"2023-08-09T13:51:16.251780+0000","last_became_active":"2023-08-09T13:51:15.236532+0000","last_became_peered":"2023-08-09T13:51:15.236532+0000","last_unstale":"2023-08-09T13:51:16.251780+0000","last_undegraded":"2023-08-09T13:51:16.251780+0000","last_fullsized":"2023-08-09T13:51:16.251780+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:51:14.171648+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:51:14.171648+0000","last_clean_scrub_stamp":"2023-08-09T13:51:14.171648+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T17:59:38.896262+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":167,"reported_epoch":102,"state":"active+clean","last_fresh":"2023-08-09T13:51:16.242693+0000","last_change":"2023-08-09T13:33:56.401781+0000","last_active":"2023-08-09T13:51:16.242693+0000","last_peered":"2023-08-09T13:51:16.242693+0000","last_clean":"2023-08-09T13:51:16.242693+0000","last_became_active":"2023-08-09T13:33:56.400336+0000","last_became_peered":"2023-08-09T13:33:56.400336+0000","last_unstale":"2023-08-09T13:51:16.242693+0000","last_undegraded":"2023-08-09T13:51:16.242693+0000","last_fullsized":"2023-08-09T13:51:16.242693+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:33:54.845869+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:33:54.845869+0000","last_clean_scrub_stamp":"2023-08-09T13:33:54.845869+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T16:50:41.367492+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.b","version":"0'0","reported_seq":9,"reported_epoch":102,"state":"active+clean","last_fresh":"2023-08-09T13:51:16.253879+0000","last_change":"2023-08-09T13:51:16.246308+0000","last_active":"2023-08-09T13:51:16.253879+0000","last_peered":"2023-08-09T13:51:16.253879+0000","last_clean":"2023-08-09T13:51:16.253879+0000","last_became_active":"2023-08-09T13:51:16.245989+0000","last_became_peered":"2023-08-09T13:51:16.245989+0000","last_unstale":"2023-08-09T13:51:16.253879+0000","last_undegraded":"2023-08-09T13:51:16.253879+0000","last_fullsized":"2023-08-09T13:51:16.253879+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:51:15.188798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:51:15.188798+0000","last_clean_scrub_stamp":"2023-08-09T13:51:15.188798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T20:02:29.796576+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.a","version":"0'0","reported_seq":11,"reported_epoch":102,"state":"active+clean","last_fresh":"2023-08-09T13:51:16.240880+0000","last_change":"2023-08-09T13:51:15.236061+0000","last_active":"2023-08-09T13:51:16.240880+0000","last_peered":"2023-08-09T13:51:16.240880+0000","last_clean":"2023-08-09T13:51:16.240880+0000","last_became_active":"2023-08-09T13:51:15.235777+0000","last_became_peered":"2023-08-09T13:51:15.235777+0000","last_unstale":"2023-08-09T13:51:16.240880+0000","last_undegraded":"2023-08-09T13:51:16.240880+0000","last_fullsized":"2023-08-09T13:51:16.240880+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:51:14.171648+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:51:14.171648+0000","last_clean_scrub_stamp":"2023-08-09T13:51:14.171648+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T20:17:36.716867+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":166,"reported_epoch":102,"state":"active+clean","last_fresh":"2023-08-09T13:51:16.238987+0000","last_change":"2023-08-09T13:33:55.886389+0000","last_active":"2023-08-09T13:51:16.238987+0000","last_peered":"2023-08-09T13:51:16.238987+0000","last_clean":"2023-08-09T13:51:16.238987+0000","last_became_active":"2023-08-09T13:33:55.885638+0000","last_became_peered":"2023-08-09T13:33:55.885638+0000","last_unstale":"2023-08-09T13:51:16.238987+0000","last_undegraded":"2023-08-09T13:51:16.238987+0000","last_fullsized":"2023-08-09T13:51:16.238987+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:33:54.845869+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:33:54.845869+0000","last_clean_scrub_stamp":"2023-08-09T13:33:54.845869+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T22:59:40.671316+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]}],"pool_stats":[{"poolid":13,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":12,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":38,"ondisk_log_size":38,"up":64,"acting":64,"num_store_stats":8},{"poolid":2,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":619,"num_read_kb":1212,"num_write":263,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":175,"ondisk_log_size":175,"up":2,"acting":2,"num_store_stats":2},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":8}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834575056,"num_pgs":16,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":37000,"kb_used_data":1788,"kb_used_omap":1,"kb_used_meta":35198,"kb_avail":94334840,"statfs":{"total":96636764160,"available":96598876160,"internally_reserved":0,"allocated":1830912,"data_stored":1249007,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1781,"internal_metadata":36043019},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Aug 9 13:50:16 2023","interfaces":[{"interface":"back","average":{"1min":1.181,"5min":1.222,"15min":1.232},"min":{"1min":0.69499999999999995,"5min":0.63400000000000001,"15min":0.61099999999999999},"max":{"1min":1.633,"5min":1.8919999999999999,"15min":2.1280000000000001},"last":0.84499999999999997},{"interface":"front","average":{"1min":1.1819999999999999,"5min":1.232,"15min":1.24},"min":{"1min":0.72399999999999998,"5min":0.72399999999999998,"15min":0.65000000000000002},"max":{"1min":1.6100000000000001,"5min":1.839,"15min":2.0099999999999998},"last":1.022}]},{"osd":1,"last update":"Wed Aug 9 13:50:16 2023","interfaces":[{"interface":"back","average":{"1min":1.2010000000000001,"5min":1.2450000000000001,"15min":1.256},"min":{"1min":0.72799999999999998,"5min":0.72799999999999998,"15min":0.59799999999999998},"max":{"1min":1.8,"5min":1.8,"15min":1.9470000000000001},"last":1.105},{"interface":"front","average":{"1min":1.2569999999999999,"5min":1.248,"15min":1.2869999999999999},"min":{"1min":0.83099999999999996,"5min":0.59299999999999997,"15min":0.59299999999999997},"max":{"1min":1.726,"5min":1.9410000000000001,"15min":1.9410000000000001},"last":1.2789999999999999}]},{"osd":2,"last update":"Wed Aug 9 13:50:16 2023","interfaces":[{"interface":"back","average":{"1min":1.278,"5min":1.2470000000000001,"15min":1.2430000000000001},"min":{"1min":0.81000000000000005,"5min":0.65700000000000003,"15min":0.65700000000000003},"max":{"1min":1.742,"5min":2.0129999999999999,"15min":2.0129999999999999},"last":1.4810000000000001},{"interface":"front","average":{"1min":1.2889999999999999,"5min":1.323,"15min":1.3759999999999999},"min":{"1min":0.92600000000000005,"5min":0.80900000000000005,"15min":0.67800000000000005},"max":{"1min":1.7130000000000001,"5min":1.8740000000000001,"15min":2.4140000000000001},"last":0.96199999999999997}]},{"osd":3,"last update":"Wed Aug 9 13:50:16 2023","interfaces":[{"interface":"back","average":{"1min":1.2529999999999999,"5min":1.3180000000000001,"15min":1.331},"min":{"1min":0.98799999999999999,"5min":0.86799999999999999,"15min":0.77500000000000002},"max":{"1min":1.7809999999999999,"5min":1.97,"15min":1.97},"last":1.3149999999999999},{"interface":"front","average":{"1min":1.3740000000000001,"5min":1.419,"15min":1.4159999999999999},"min":{"1min":0.93899999999999995,"5min":0.88300000000000001,"15min":0.82099999999999995},"max":{"1min":1.8160000000000001,"5min":2.0790000000000002,"15min":2.2480000000000002},"last":1.224}]},{"osd":4,"last update":"Wed Aug 9 13:50:16 2023","interfaces":[{"interface":"back","average":{"1min":1.343,"5min":1.2649999999999999,"15min":1.2769999999999999},"min":{"1min":0.86399999999999999,"5min":0.59499999999999997,"15min":0.59499999999999997},"max":{"1min":1.772,"5min":2.032,"15min":2.343},"last":1.466},{"interface":"front","average":{"1min":1.296,"5min":1.3160000000000001,"15min":1.339},"min":{"1min":0.93600000000000005,"5min":0.77100000000000002,"15min":0.56799999999999995},"max":{"1min":1.6739999999999999,"5min":2.1419999999999999,"15min":2.1419999999999999},"last":1.585}]},{"osd":5,"last update":"Wed Aug 9 13:50:16 2023","interfaces":[{"interface":"back","average":{"1min":1.4430000000000001,"5min":1.4350000000000001,"15min":1.419},"min":{"1min":1.048,"5min":0.88600000000000001,"15min":0.71499999999999997},"max":{"1min":1.9039999999999999,"5min":2.1480000000000001,"15min":2.6749999999999998},"last":1.3859999999999999},{"interface":"front","average":{"1min":1.4810000000000001,"5min":1.4319999999999999,"15min":1.4510000000000001},"min":{"1min":0.95999999999999996,"5min":0.85699999999999998,"15min":0.69199999999999995},"max":{"1min":1.877,"5min":2.113,"15min":2.9569999999999999},"last":1.1719999999999999}]},{"osd":6,"last update":"Wed Aug 9 13:50:16 2023","interfaces":[{"interface":"back","average":{"1min":1.6020000000000001,"5min":1.518,"15min":1.5489999999999999},"min":{"1min":1.2,"5min":0.85499999999999998,"15min":0.85499999999999998},"max":{"1min":1.8879999999999999,"5min":2.2069999999999999,"15min":2.8149999999999999},"last":1.367},{"interface":"front","average":{"1min":1.4139999999999999,"5min":1.4570000000000001,"15min":1.474},"min":{"1min":1.111,"5min":0.90600000000000003,"15min":0.60799999999999998},"max":{"1min":1.9419999999999999,"5min":2.1320000000000001,"15min":2.2530000000000001},"last":1.663}]}]},{"osd":6,"up_from":13,"seq":55834575056,"num_pgs":11,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":36420,"kb_used_data":1204,"kb_used_omap":1,"kb_used_meta":35198,"kb_avail":94335420,"statfs":{"total":96636764160,"available":96599470080,"internally_reserved":0,"allocated":1232896,"data_stored":657278,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1781,"internal_metadata":36043019},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Aug 9 13:50:20 2023","interfaces":[{"interface":"back","average":{"1min":1.1160000000000001,"5min":1.143,"15min":1.161},"min":{"1min":0.75,"5min":0.72499999999999998,"15min":0.56599999999999995},"max":{"1min":1.5489999999999999,"5min":1.633,"15min":1.9870000000000001},"last":0.65800000000000003},{"interface":"front","average":{"1min":1.21,"5min":1.532,"15min":1.3080000000000001},"min":{"1min":0.79300000000000004,"5min":0.79300000000000004,"15min":0.76000000000000001},"max":{"1min":1.6519999999999999,"5min":28.036000000000001,"15min":28.036000000000001},"last":1.0049999999999999}]},{"osd":1,"last update":"Wed Aug 9 13:50:20 2023","interfaces":[{"interface":"back","average":{"1min":1.147,"5min":1.272,"15min":1.226},"min":{"1min":0.70399999999999996,"5min":0.70399999999999996,"15min":0.65500000000000003},"max":{"1min":1.659,"5min":7.742,"15min":7.742},"last":0.79700000000000004},{"interface":"front","average":{"1min":1.252,"5min":1.2849999999999999,"15min":1.276},"min":{"1min":0.84199999999999997,"5min":0.84199999999999997,"15min":0.68100000000000005},"max":{"1min":1.8899999999999999,"5min":1.8899999999999999,"15min":1.8899999999999999},"last":1.3129999999999999}]},{"osd":2,"last update":"Wed Aug 9 13:50:20 2023","interfaces":[{"interface":"back","average":{"1min":1.2989999999999999,"5min":1.3480000000000001,"15min":1.3220000000000001},"min":{"1min":0.877,"5min":0.81999999999999995,"15min":0.68700000000000006},"max":{"1min":1.73,"5min":4.0640000000000001,"15min":4.0640000000000001},"last":1.0620000000000001},{"interface":"front","average":{"1min":1.244,"5min":1.4430000000000001,"15min":1.3460000000000001},"min":{"1min":0.76800000000000002,"5min":0.70999999999999996,"15min":0.70999999999999996},"max":{"1min":2.0390000000000001,"5min":13.154999999999999,"15min":13.154999999999999},"last":1.1160000000000001}]},{"osd":3,"last update":"Wed Aug 9 13:50:20 2023","interfaces":[{"interface":"back","average":{"1min":1.3939999999999999,"5min":1.52,"15min":1.419},"min":{"1min":0.73099999999999998,"5min":0.73099999999999998,"15min":0.73099999999999998},"max":{"1min":2.2519999999999998,"5min":13.349,"15min":13.349},"last":0.879},{"interface":"front","average":{"1min":1.258,"5min":1.3620000000000001,"15min":1.325},"min":{"1min":0.88400000000000001,"5min":0.76400000000000001,"15min":0.74299999999999999},"max":{"1min":1.8560000000000001,"5min":7.9029999999999996,"15min":7.9029999999999996},"last":0.94299999999999995}]},{"osd":4,"last update":"Wed Aug 9 13:50:20 2023","interfaces":[{"interface":"back","average":{"1min":1.302,"5min":1.335,"15min":1.28},"min":{"1min":0.90700000000000003,"5min":0.78500000000000003,"15min":0.64300000000000002},"max":{"1min":1.992,"5min":5.2290000000000001,"15min":5.2290000000000001},"last":1.4570000000000001},{"interface":"front","average":{"1min":1.4019999999999999,"5min":1.498,"15min":1.389},"min":{"1min":0.88900000000000001,"5min":0.78700000000000003,"15min":0.69099999999999995},"max":{"1min":1.734,"5min":12.1,"15min":12.1},"last":1.2669999999999999}]},{"osd":5,"last update":"Wed Aug 9 13:50:20 2023","interfaces":[{"interface":"back","average":{"1min":1.3169999999999999,"5min":1.4399999999999999,"15min":1.357},"min":{"1min":0.93999999999999995,"5min":0.83799999999999997,"15min":0.65400000000000003},"max":{"1min":2.1139999999999999,"5min":8.1899999999999995,"15min":8.1899999999999995},"last":1.395},{"interface":"front","average":{"1min":1.5649999999999999,"5min":1.6639999999999999,"15min":1.589},"min":{"1min":1.2010000000000001,"5min":1.2010000000000001,"15min":0.83799999999999997},"max":{"1min":2.2000000000000002,"5min":8.8889999999999993,"15min":8.8889999999999993},"last":1.552}]},{"osd":7,"last update":"Wed Aug 9 13:50:20 2023","interfaces":[{"interface":"back","average":{"1min":1.577,"5min":1.6970000000000001,"15min":1.5880000000000001},"min":{"1min":1.1850000000000001,"5min":1.0029999999999999,"15min":0.62},"max":{"1min":1.8240000000000001,"5min":9.3320000000000007,"15min":9.3320000000000007},"last":1.363},{"interface":"front","average":{"1min":1.4410000000000001,"5min":1.6060000000000001,"15min":1.548},"min":{"1min":0.93899999999999995,"5min":0.88500000000000001,"15min":0.88500000000000001},"max":{"1min":1.9470000000000001,"5min":8.0399999999999991,"15min":8.0399999999999991},"last":1.1819999999999999}]}]},{"osd":5,"up_from":13,"seq":55834575056,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":40548,"kb_used_data":1236,"kb_used_omap":1,"kb_used_meta":39294,"kb_avail":94331292,"statfs":{"total":96636764160,"available":96595243008,"internally_reserved":0,"allocated":1265664,"data_stored":667765,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1781,"internal_metadata":40237323},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Aug 9 13:50:19 2023","interfaces":[{"interface":"back","average":{"1min":1.2509999999999999,"5min":1.1759999999999999,"15min":1.179},"min":{"1min":0.63500000000000001,"5min":0.63500000000000001,"15min":0.63500000000000001},"max":{"1min":1.518,"5min":2.1869999999999998,"15min":2.1869999999999998},"last":1.337},{"interface":"front","average":{"1min":1.1719999999999999,"5min":1.2,"15min":1.198},"min":{"1min":0.68600000000000005,"5min":0.68600000000000005,"15min":0.64200000000000002},"max":{"1min":1.579,"5min":1.6619999999999999,"15min":1.7989999999999999},"last":1.2450000000000001}]},{"osd":1,"last update":"Wed Aug 9 13:50:19 2023","interfaces":[{"interface":"back","average":{"1min":1.165,"5min":1.1990000000000001,"15min":1.2190000000000001},"min":{"1min":0.81599999999999995,"5min":0.73099999999999998,"15min":0.64100000000000001},"max":{"1min":1.631,"5min":1.6739999999999999,"15min":1.736},"last":0.75700000000000001},{"interface":"front","average":{"1min":1.264,"5min":1.296,"15min":1.2769999999999999},"min":{"1min":0.93300000000000005,"5min":0.66900000000000004,"15min":0.66900000000000004},"max":{"1min":1.6699999999999999,"5min":1.887,"15min":1.887},"last":1.694}]},{"osd":2,"last update":"Wed Aug 9 13:50:19 2023","interfaces":[{"interface":"back","average":{"1min":1.2410000000000001,"5min":1.2569999999999999,"15min":1.2849999999999999},"min":{"1min":0.98199999999999998,"5min":0.68999999999999995,"15min":0.61599999999999999},"max":{"1min":1.5760000000000001,"5min":1.756,"15min":1.899},"last":0.81799999999999995},{"interface":"front","average":{"1min":1.3049999999999999,"5min":1.302,"15min":1.3149999999999999},"min":{"1min":0.78400000000000003,"5min":0.78400000000000003,"15min":0.66800000000000004},"max":{"1min":1.6599999999999999,"5min":1.7190000000000001,"15min":1.7969999999999999},"last":1.1719999999999999}]},{"osd":3,"last update":"Wed Aug 9 13:50:19 2023","interfaces":[{"interface":"back","average":{"1min":1.365,"5min":1.3720000000000001,"15min":1.3600000000000001},"min":{"1min":0.95399999999999996,"5min":0.95399999999999996,"15min":0.59799999999999998},"max":{"1min":1.6890000000000001,"5min":2.1840000000000002,"15min":2.2690000000000001},"last":1.534},{"interface":"front","average":{"1min":1.4259999999999999,"5min":1.4379999999999999,"15min":1.4139999999999999},"min":{"1min":1.087,"5min":0.94899999999999995,"15min":0.70999999999999996},"max":{"1min":1.73,"5min":2.3780000000000001,"15min":2.3780000000000001},"last":1.0629999999999999}]},{"osd":4,"last update":"Wed Aug 9 13:50:19 2023","interfaces":[{"interface":"back","average":{"1min":1.262,"5min":1.2729999999999999,"15min":1.2889999999999999},"min":{"1min":0.92700000000000005,"5min":0.84699999999999998,"15min":0.56599999999999995},"max":{"1min":1.673,"5min":2.0800000000000001,"15min":2.121},"last":1.2929999999999999},{"interface":"front","average":{"1min":1.3919999999999999,"5min":1.3400000000000001,"15min":1.3779999999999999},"min":{"1min":0.81999999999999995,"5min":0.76100000000000001,"15min":0.58599999999999997},"max":{"1min":1.944,"5min":2.1230000000000002,"15min":2.3679999999999999},"last":1.613}]},{"osd":6,"last update":"Wed Aug 9 13:50:19 2023","interfaces":[{"interface":"back","average":{"1min":1.4299999999999999,"5min":1.3520000000000001,"15min":1.391},"min":{"1min":0.99099999999999999,"5min":0.90300000000000002,"15min":0.66700000000000004},"max":{"1min":1.9810000000000001,"5min":2.306,"15min":2.306},"last":1.4410000000000001},{"interface":"front","average":{"1min":1.49,"5min":1.4399999999999999,"15min":1.4710000000000001},"min":{"1min":1.1220000000000001,"5min":1.034,"15min":0.71199999999999997},"max":{"1min":1.827,"5min":2.2320000000000002,"15min":2.2320000000000002},"last":1.49}]},{"osd":7,"last update":"Wed Aug 9 13:50:19 2023","interfaces":[{"interface":"back","average":{"1min":1.577,"5min":1.5580000000000001,"15min":1.5529999999999999},"min":{"1min":1.1160000000000001,"5min":1.0589999999999999,"15min":0.70899999999999996},"max":{"1min":1.9610000000000001,"5min":2.452,"15min":2.4729999999999999},"last":1.7509999999999999},{"interface":"front","average":{"1min":1.611,"5min":1.585,"15min":1.6080000000000001},"min":{"1min":1.2110000000000001,"5min":1.163,"15min":0.77600000000000002},"max":{"1min":2.097,"5min":2.1070000000000002,"15min":2.3820000000000001},"last":1.6619999999999999}]}]},{"osd":4,"up_from":13,"seq":55834575056,"num_pgs":6,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32328,"kb_used_data":1208,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":94339512,"statfs":{"total":96636764160,"available":96603660288,"internally_reserved":0,"allocated":1236992,"data_stored":658639,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1781,"internal_metadata":31848715},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Aug 9 13:50:21 2023","interfaces":[{"interface":"back","average":{"1min":1.2330000000000001,"5min":1.2390000000000001,"15min":1.2030000000000001},"min":{"1min":0.875,"5min":0.70699999999999996,"15min":0.67700000000000005},"max":{"1min":1.5940000000000001,"5min":2.524,"15min":2.524},"last":0.68799999999999994},{"interface":"front","average":{"1min":1.206,"5min":1.236,"15min":1.2569999999999999},"min":{"1min":0.62,"5min":0.62,"15min":0.62},"max":{"1min":1.5980000000000001,"5min":1.863,"15min":1.863},"last":0.80200000000000005}]},{"osd":1,"last update":"Wed Aug 9 13:50:21 2023","interfaces":[{"interface":"back","average":{"1min":1.244,"5min":1.3169999999999999,"15min":1.2929999999999999},"min":{"1min":0.70999999999999996,"5min":0.67800000000000005,"15min":0.67800000000000005},"max":{"1min":1.48,"5min":1.8480000000000001,"15min":1.8480000000000001},"last":1.1890000000000001},{"interface":"front","average":{"1min":1.242,"5min":1.2649999999999999,"15min":1.248},"min":{"1min":0.81799999999999995,"5min":0.81799999999999995,"15min":0.64900000000000002},"max":{"1min":1.9330000000000001,"5min":1.9330000000000001,"15min":1.9330000000000001},"last":0.75}]},{"osd":2,"last update":"Wed Aug 9 13:50:21 2023","interfaces":[{"interface":"back","average":{"1min":1.306,"5min":1.2869999999999999,"15min":1.2809999999999999},"min":{"1min":1.022,"5min":0.621,"15min":0.621},"max":{"1min":1.619,"5min":2.6429999999999998,"15min":2.6429999999999998},"last":0.96599999999999997},{"interface":"front","average":{"1min":1.343,"5min":1.3720000000000001,"15min":1.3660000000000001},"min":{"1min":0.88300000000000001,"5min":0.88300000000000001,"15min":0.74299999999999999},"max":{"1min":1.7569999999999999,"5min":1.7569999999999999,"15min":1.8149999999999999},"last":1.3460000000000001}]},{"osd":3,"last update":"Wed Aug 9 13:50:21 2023","interfaces":[{"interface":"back","average":{"1min":1.377,"5min":1.3300000000000001,"15min":1.298},"min":{"1min":1.083,"5min":0.86399999999999999,"15min":0.84699999999999998},"max":{"1min":1.6739999999999999,"5min":2.6099999999999999,"15min":2.6099999999999999},"last":0.86499999999999999},{"interface":"front","average":{"1min":1.4099999999999999,"5min":1.3979999999999999,"15min":1.3799999999999999},"min":{"1min":0.94799999999999995,"5min":0.88800000000000001,"15min":0.63100000000000001},"max":{"1min":1.6890000000000001,"5min":2.7629999999999999,"15min":2.7629999999999999},"last":1.2709999999999999}]},{"osd":5,"last update":"Wed Aug 9 13:50:21 2023","interfaces":[{"interface":"back","average":{"1min":1.288,"5min":1.3400000000000001,"15min":1.3300000000000001},"min":{"1min":0.876,"5min":0.73199999999999998,"15min":0.63200000000000001},"max":{"1min":1.734,"5min":1.8,"15min":2.0099999999999998},"last":1.4219999999999999},{"interface":"front","average":{"1min":1.2509999999999999,"5min":1.323,"15min":1.3180000000000001},"min":{"1min":0.92000000000000004,"5min":0.72399999999999998,"15min":0.626},"max":{"1min":1.71,"5min":2.7090000000000001,"15min":2.7090000000000001},"last":1.0269999999999999}]},{"osd":6,"last update":"Wed Aug 9 13:50:21 2023","interfaces":[{"interface":"back","average":{"1min":1.379,"5min":1.3160000000000001,"15min":1.3360000000000001},"min":{"1min":0.94699999999999995,"5min":0.74399999999999999,"15min":0.59099999999999997},"max":{"1min":1.7509999999999999,"5min":2.8559999999999999,"15min":2.8559999999999999},"last":1.3720000000000001},{"interface":"front","average":{"1min":1.4430000000000001,"5min":1.518,"15min":1.504},"min":{"1min":1.0760000000000001,"5min":0.97799999999999998,"15min":0.76400000000000001},"max":{"1min":1.669,"5min":2.2719999999999998,"15min":2.2719999999999998},"last":1.4850000000000001}]},{"osd":7,"last update":"Wed Aug 9 13:50:21 2023","interfaces":[{"interface":"back","average":{"1min":1.46,"5min":1.4830000000000001,"15min":1.474},"min":{"1min":1.1499999999999999,"5min":0.85999999999999999,"15min":0.73899999999999999},"max":{"1min":1.8460000000000001,"5min":2.8050000000000002,"15min":2.8050000000000002},"last":1.1200000000000001},{"interface":"front","average":{"1min":1.474,"5min":1.522,"15min":1.514},"min":{"1min":1.236,"5min":0.97199999999999998,"15min":0.72899999999999998},"max":{"1min":1.754,"5min":3.415,"15min":3.415},"last":1.4570000000000001}]}]},{"osd":3,"up_from":13,"seq":55834575057,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":36424,"kb_used_data":1208,"kb_used_omap":1,"kb_used_meta":35198,"kb_avail":94335416,"statfs":{"total":96636764160,"available":96599465984,"internally_reserved":0,"allocated":1236992,"data_stored":658639,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1781,"internal_metadata":36043019},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Aug 9 13:50:25 2023","interfaces":[{"interface":"back","average":{"1min":0.84599999999999997,"5min":0.88300000000000001,"15min":0.91600000000000004},"min":{"1min":0.43099999999999999,"5min":0.27200000000000002,"15min":0.27200000000000002},"max":{"1min":1.3939999999999999,"5min":1.8500000000000001,"15min":1.968},"last":0.72599999999999998},{"interface":"front","average":{"1min":1.071,"5min":1.0920000000000001,"15min":1.1060000000000001},"min":{"1min":0.48199999999999998,"5min":0.36399999999999999,"15min":0.36399999999999999},"max":{"1min":1.6910000000000001,"5min":1.6910000000000001,"15min":2.2360000000000002},"last":0.96099999999999997}]},{"osd":1,"last update":"Wed Aug 9 13:50:25 2023","interfaces":[{"interface":"back","average":{"1min":1.042,"5min":1.0089999999999999,"15min":1.012},"min":{"1min":0.59899999999999998,"5min":0.496,"15min":0.438},"max":{"1min":1.518,"5min":1.7110000000000001,"15min":2.5649999999999999},"last":1.171},{"interface":"front","average":{"1min":1.091,"5min":1.1279999999999999,"15min":1.1639999999999999},"min":{"1min":0.51800000000000002,"5min":0.51800000000000002,"15min":0.42099999999999999},"max":{"1min":1.6839999999999999,"5min":1.6839999999999999,"15min":2.6760000000000002},"last":1.024}]},{"osd":2,"last update":"Wed Aug 9 13:50:25 2023","interfaces":[{"interface":"back","average":{"1min":1.121,"5min":1.177,"15min":1.1930000000000001},"min":{"1min":0.53600000000000003,"5min":0.53600000000000003,"15min":0.53500000000000003},"max":{"1min":1.6240000000000001,"5min":1.913,"15min":1.923},"last":0.89300000000000002},{"interface":"front","average":{"1min":1.1899999999999999,"5min":1.2929999999999999,"15min":1.3080000000000001},"min":{"1min":0.625,"5min":0.52300000000000002,"15min":0.52000000000000002},"max":{"1min":1.9399999999999999,"5min":2.0259999999999998,"15min":2.0259999999999998},"last":1.2909999999999999}]},{"osd":4,"last update":"Wed Aug 9 13:50:25 2023","interfaces":[{"interface":"back","average":{"1min":1.232,"5min":1.258,"15min":1.282},"min":{"1min":0.91500000000000004,"5min":0.57899999999999996,"15min":0.57899999999999996},"max":{"1min":1.742,"5min":1.8160000000000001,"15min":1.8160000000000001},"last":1.242},{"interface":"front","average":{"1min":1.349,"5min":1.397,"15min":1.3919999999999999},"min":{"1min":0.67600000000000005,"5min":0.628,"15min":0.628},"max":{"1min":1.8149999999999999,"5min":2.2040000000000002,"15min":2.2370000000000001},"last":2.0960000000000001}]},{"osd":5,"last update":"Wed Aug 9 13:50:25 2023","interfaces":[{"interface":"back","average":{"1min":1.456,"5min":1.468,"15min":1.474},"min":{"1min":0.67600000000000005,"5min":0.67600000000000005,"15min":0.67200000000000004},"max":{"1min":1.855,"5min":2.254,"15min":2.254},"last":1.833},{"interface":"front","average":{"1min":1.4850000000000001,"5min":1.4910000000000001,"15min":1.49},"min":{"1min":0.77600000000000002,"5min":0.65900000000000003,"15min":0.65900000000000003},"max":{"1min":2.3069999999999999,"5min":2.3069999999999999,"15min":2.3069999999999999},"last":1.387}]},{"osd":6,"last update":"Wed Aug 9 13:50:25 2023","interfaces":[{"interface":"back","average":{"1min":1.3759999999999999,"5min":1.4370000000000001,"15min":1.4590000000000001},"min":{"1min":0.878,"5min":0.68200000000000005,"15min":0.68200000000000005},"max":{"1min":1.8759999999999999,"5min":2.1829999999999998,"15min":2.242},"last":1.631},{"interface":"front","average":{"1min":1.466,"5min":1.5580000000000001,"15min":1.5529999999999999},"min":{"1min":1.0600000000000001,"5min":0.71099999999999997,"15min":0.68600000000000005},"max":{"1min":1.825,"5min":2.3420000000000001,"15min":2.3420000000000001},"last":1.966}]},{"osd":7,"last update":"Wed Aug 9 13:50:25 2023","interfaces":[{"interface":"back","average":{"1min":1.601,"5min":1.627,"15min":1.6160000000000001},"min":{"1min":1.115,"5min":0.69399999999999995,"15min":0.69399999999999995},"max":{"1min":2.46,"5min":2.46,"15min":2.46},"last":1.4610000000000001},{"interface":"front","average":{"1min":1.4910000000000001,"5min":1.546,"15min":1.5600000000000001},"min":{"1min":0.78200000000000003,"5min":0.64200000000000002,"15min":0.61599999999999999},"max":{"1min":2.0579999999999998,"5min":2.359,"15min":2.484},"last":1.552}]}]},{"osd":2,"up_from":13,"seq":55834575056,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":36424,"kb_used_data":1208,"kb_used_omap":1,"kb_used_meta":35198,"kb_avail":94335416,"statfs":{"total":96636764160,"available":96599465984,"internally_reserved":0,"allocated":1236992,"data_stored":658639,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1781,"internal_metadata":36043019},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Aug 9 13:50:17 2023","interfaces":[{"interface":"back","average":{"1min":0.96999999999999997,"5min":0.95299999999999996,"15min":0.95499999999999996},"min":{"1min":0.47799999999999998,"5min":0.36499999999999999,"15min":0.36499999999999999},"max":{"1min":1.5760000000000001,"5min":1.6180000000000001,"15min":1.8380000000000001},"last":0.626},{"interface":"front","average":{"1min":0.97099999999999997,"5min":0.998,"15min":1.0049999999999999},"min":{"1min":0.45300000000000001,"5min":0.374,"15min":0.374},"max":{"1min":1.4199999999999999,"5min":2.3929999999999998,"15min":2.3929999999999998},"last":0.79400000000000004}]},{"osd":1,"last update":"Wed Aug 9 13:50:17 2023","interfaces":[{"interface":"back","average":{"1min":1.2529999999999999,"5min":1.1739999999999999,"15min":1.1870000000000001},"min":{"1min":0.61299999999999999,"5min":0.55200000000000005,"15min":0.55200000000000005},"max":{"1min":1.6870000000000001,"5min":1.77,"15min":1.8420000000000001},"last":1.321},{"interface":"front","average":{"1min":1.216,"5min":1.2749999999999999,"15min":1.335},"min":{"1min":0.52500000000000002,"5min":0.52500000000000002,"15min":0.52500000000000002},"max":{"1min":1.7390000000000001,"5min":2.4870000000000001,"15min":2.4870000000000001},"last":1.659}]},{"osd":3,"last update":"Wed Aug 9 13:50:17 2023","interfaces":[{"interface":"back","average":{"1min":0.98099999999999998,"5min":1.006,"15min":1.0469999999999999},"min":{"1min":0.44800000000000001,"5min":0.38900000000000001,"15min":0.38900000000000001},"max":{"1min":1.3109999999999999,"5min":1.708,"15min":2.0409999999999999},"last":1.244},{"interface":"front","average":{"1min":1.2110000000000001,"5min":1.304,"15min":1.3089999999999999},"min":{"1min":0.55500000000000005,"5min":0.46200000000000002,"15min":0.46200000000000002},"max":{"1min":1.8009999999999999,"5min":8.8130000000000006,"15min":8.8130000000000006},"last":0.873}]},{"osd":4,"last update":"Wed Aug 9 13:50:17 2023","interfaces":[{"interface":"back","average":{"1min":1.415,"5min":1.4179999999999999,"15min":1.4510000000000001},"min":{"1min":0.83999999999999997,"5min":0.628,"15min":0.628},"max":{"1min":1.958,"5min":2.089,"15min":2.089},"last":1.488},{"interface":"front","average":{"1min":1.462,"5min":1.359,"15min":1.3979999999999999},"min":{"1min":0.73799999999999999,"5min":0.68600000000000005,"15min":0.68600000000000005},"max":{"1min":2.472,"5min":2.472,"15min":2.472},"last":1.7509999999999999}]},{"osd":5,"last update":"Wed Aug 9 13:50:17 2023","interfaces":[{"interface":"back","average":{"1min":1.5189999999999999,"5min":1.4379999999999999,"15min":1.458},"min":{"1min":0.76600000000000001,"5min":0.59999999999999998,"15min":0.59999999999999998},"max":{"1min":2.27,"5min":2.5840000000000001,"15min":2.5840000000000001},"last":1.54},{"interface":"front","average":{"1min":1.304,"5min":1.3540000000000001,"15min":1.349},"min":{"1min":0.77100000000000002,"5min":0.56799999999999995,"15min":0.56799999999999995},"max":{"1min":1.8859999999999999,"5min":3.5659999999999998,"15min":3.5659999999999998},"last":1.4590000000000001}]},{"osd":6,"last update":"Wed Aug 9 13:50:17 2023","interfaces":[{"interface":"back","average":{"1min":1.4019999999999999,"5min":1.425,"15min":1.4259999999999999},"min":{"1min":0.79800000000000004,"5min":0.73799999999999999,"15min":0.73799999999999999},"max":{"1min":2.222,"5min":3.5099999999999998,"15min":3.5099999999999998},"last":1.3839999999999999},{"interface":"front","average":{"1min":1.4790000000000001,"5min":1.4990000000000001,"15min":1.5409999999999999},"min":{"1min":0.82199999999999995,"5min":0.66300000000000003,"15min":0.66300000000000003},"max":{"1min":2.3769999999999998,"5min":2.3769999999999998,"15min":2.3769999999999998},"last":1.589}]},{"osd":7,"last update":"Wed Aug 9 13:50:17 2023","interfaces":[{"interface":"back","average":{"1min":1.54,"5min":1.51,"15min":1.548},"min":{"1min":0.78800000000000003,"5min":0.64600000000000002,"15min":0.64600000000000002},"max":{"1min":2.1970000000000001,"5min":2.5249999999999999,"15min":2.5249999999999999},"last":1.4099999999999999},{"interface":"front","average":{"1min":1.4970000000000001,"5min":1.52,"15min":1.51},"min":{"1min":0.99199999999999999,"5min":0.70499999999999996,"15min":0.70499999999999996},"max":{"1min":2.089,"5min":3.609,"15min":3.609},"last":1.5149999999999999}]}]},{"osd":1,"up_from":13,"seq":55834575056,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":46860,"kb_used_data":1788,"kb_used_omap":1,"kb_used_meta":45054,"kb_avail":94324980,"statfs":{"total":96636764160,"available":96588779520,"internally_reserved":0,"allocated":1830912,"data_stored":1249007,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1781,"internal_metadata":46135563},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Aug 9 13:50:31 2023","interfaces":[{"interface":"back","average":{"1min":0.99099999999999999,"5min":0.92000000000000004,"15min":0.93999999999999995},"min":{"1min":0.50600000000000001,"5min":0.50600000000000001,"15min":0.32400000000000001},"max":{"1min":1.7110000000000001,"5min":1.7110000000000001,"15min":2.1080000000000001},"last":0.88800000000000001},{"interface":"front","average":{"1min":0.98999999999999999,"5min":1.0740000000000001,"15min":1.095},"min":{"1min":0.59799999999999998,"5min":0.432,"15min":0.42499999999999999},"max":{"1min":1.7190000000000001,"5min":2.468,"15min":2.468},"last":1.6519999999999999}]},{"osd":2,"last update":"Wed Aug 9 13:50:31 2023","interfaces":[{"interface":"back","average":{"1min":1.008,"5min":0.97799999999999998,"15min":0.995},"min":{"1min":0.56699999999999995,"5min":0.41099999999999998,"15min":0.41099999999999998},"max":{"1min":1.6679999999999999,"5min":1.907,"15min":1.907},"last":0.78200000000000003},{"interface":"front","average":{"1min":1.1459999999999999,"5min":1.159,"15min":1.1879999999999999},"min":{"1min":0.80600000000000005,"5min":0.54400000000000004,"15min":0.46500000000000002},"max":{"1min":1.5189999999999999,"5min":2.1280000000000001,"15min":2.145},"last":1.149}]},{"osd":3,"last update":"Wed Aug 9 13:50:31 2023","interfaces":[{"interface":"back","average":{"1min":1.218,"5min":1.2629999999999999,"15min":1.3260000000000001},"min":{"1min":0.63200000000000001,"5min":0.54400000000000004,"15min":0.432},"max":{"1min":1.7430000000000001,"5min":2.0270000000000001,"15min":2.3210000000000002},"last":1.3580000000000001},{"interface":"front","average":{"1min":1.1779999999999999,"5min":1.234,"15min":1.2450000000000001},"min":{"1min":0.70399999999999996,"5min":0.49299999999999999,"15min":0.44400000000000001},"max":{"1min":1.786,"5min":2.3540000000000001,"15min":2.7669999999999999},"last":0.94599999999999995}]},{"osd":4,"last update":"Wed Aug 9 13:50:31 2023","interfaces":[{"interface":"back","average":{"1min":1.3149999999999999,"5min":1.399,"15min":1.4079999999999999},"min":{"1min":0.94399999999999995,"5min":0.79800000000000004,"15min":0.58099999999999996},"max":{"1min":1.696,"5min":1.97,"15min":2.5},"last":1.214},{"interface":"front","average":{"1min":1.2689999999999999,"5min":1.3180000000000001,"15min":1.333},"min":{"1min":0.88200000000000001,"5min":0.70099999999999996,"15min":0.70099999999999996},"max":{"1min":1.6950000000000001,"5min":1.901,"15min":1.901},"last":1.2629999999999999}]},{"osd":5,"last update":"Wed Aug 9 13:50:31 2023","interfaces":[{"interface":"back","average":{"1min":1.3200000000000001,"5min":1.323,"15min":1.371},"min":{"1min":0.90700000000000003,"5min":0.76900000000000002,"15min":0.76900000000000002},"max":{"1min":1.837,"5min":2.294,"15min":2.294},"last":1.474},{"interface":"front","average":{"1min":1.427,"5min":1.464,"15min":1.516},"min":{"1min":0.97899999999999998,"5min":0.91000000000000003,"15min":0.77700000000000002},"max":{"1min":1.837,"5min":2.1389999999999998,"15min":2.4649999999999999},"last":1.502}]},{"osd":6,"last update":"Wed Aug 9 13:50:31 2023","interfaces":[{"interface":"back","average":{"1min":1.4219999999999999,"5min":1.48,"15min":1.514},"min":{"1min":1.026,"5min":0.77500000000000002,"15min":0.67400000000000004},"max":{"1min":1.925,"5min":2.2080000000000002,"15min":2.5270000000000001},"last":1.5780000000000001},{"interface":"front","average":{"1min":1.4370000000000001,"5min":1.45,"15min":1.5},"min":{"1min":1.1379999999999999,"5min":0.82499999999999996,"15min":0.79800000000000004},"max":{"1min":2.02,"5min":2.02,"15min":2.1040000000000001},"last":1.5489999999999999}]},{"osd":7,"last update":"Wed Aug 9 13:50:31 2023","interfaces":[{"interface":"back","average":{"1min":1.552,"5min":1.5649999999999999,"15min":1.619},"min":{"1min":1.0389999999999999,"5min":0.86899999999999999,"15min":0.77100000000000002},"max":{"1min":2.4169999999999998,"5min":2.4169999999999998,"15min":2.5790000000000002},"last":1.613},{"interface":"front","average":{"1min":1.5740000000000001,"5min":1.5549999999999999,"15min":1.6000000000000001},"min":{"1min":1.2589999999999999,"5min":0.91300000000000003,"15min":0.75900000000000001},"max":{"1min":2.036,"5min":2.1040000000000001,"15min":2.6509999999999998},"last":1.4299999999999999}]}]},{"osd":0,"up_from":13,"seq":55834575056,"num_pgs":11,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":40516,"kb_used_data":1204,"kb_used_omap":1,"kb_used_meta":39294,"kb_avail":94331324,"statfs":{"total":96636764160,"available":96595275776,"internally_reserved":0,"allocated":1232896,"data_stored":657278,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1781,"internal_metadata":40237323},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Wed Aug 9 13:50:27 2023","interfaces":[{"interface":"back","average":{"1min":0.91800000000000004,"5min":0.92200000000000004,"15min":0.91600000000000004},"min":{"1min":0.32500000000000001,"5min":0.32500000000000001,"15min":0.32500000000000001},"max":{"1min":1.5820000000000001,"5min":1.756,"15min":1.756},"last":1.0129999999999999},{"interface":"front","average":{"1min":1.0680000000000001,"5min":1.1020000000000001,"15min":1.113},"min":{"1min":0.47099999999999997,"5min":0.45300000000000001,"15min":0.45300000000000001},"max":{"1min":1.9019999999999999,"5min":1.9019999999999999,"15min":1.9019999999999999},"last":0.82199999999999995}]},{"osd":2,"last update":"Wed Aug 9 13:50:27 2023","interfaces":[{"interface":"back","average":{"1min":1.048,"5min":1.1000000000000001,"15min":1.135},"min":{"1min":0.53800000000000003,"5min":0.51400000000000001,"15min":0.44800000000000001},"max":{"1min":1.702,"5min":1.776,"15min":1.889},"last":0.745},{"interface":"front","average":{"1min":1.1819999999999999,"5min":1.149,"15min":1.179},"min":{"1min":0.58499999999999996,"5min":0.56399999999999995,"15min":0.52700000000000002},"max":{"1min":2.0129999999999999,"5min":2.0129999999999999,"15min":2.2519999999999998},"last":0.93799999999999994}]},{"osd":3,"last update":"Wed Aug 9 13:50:27 2023","interfaces":[{"interface":"back","average":{"1min":0.98499999999999999,"5min":1.0509999999999999,"15min":1.105},"min":{"1min":0.54900000000000004,"5min":0.46300000000000002,"15min":0.46300000000000002},"max":{"1min":1.345,"5min":1.823,"15min":2.5190000000000001},"last":1.0600000000000001},{"interface":"front","average":{"1min":1.272,"5min":1.2629999999999999,"15min":1.3140000000000001},"min":{"1min":0.56399999999999995,"5min":0.54700000000000004,"15min":0.54700000000000004},"max":{"1min":2.347,"5min":2.347,"15min":2.4649999999999999},"last":1.1419999999999999}]},{"osd":4,"last update":"Wed Aug 9 13:50:27 2023","interfaces":[{"interface":"back","average":{"1min":1.302,"5min":1.3129999999999999,"15min":1.347},"min":{"1min":0.84699999999999998,"5min":0.84699999999999998,"15min":0.74299999999999999},"max":{"1min":1.9630000000000001,"5min":2.2309999999999999,"15min":2.2309999999999999},"last":1.3280000000000001},{"interface":"front","average":{"1min":1.3999999999999999,"5min":1.4390000000000001,"15min":1.452},"min":{"1min":0.82699999999999996,"5min":0.77600000000000002,"15min":0.74099999999999999},"max":{"1min":1.8440000000000001,"5min":2.5390000000000001,"15min":2.5390000000000001},"last":1.3080000000000001}]},{"osd":5,"last update":"Wed Aug 9 13:50:27 2023","interfaces":[{"interface":"back","average":{"1min":1.387,"5min":1.4219999999999999,"15min":1.4650000000000001},"min":{"1min":0.66300000000000003,"5min":0.66300000000000003,"15min":0.66300000000000003},"max":{"1min":2.6669999999999998,"5min":2.6669999999999998,"15min":2.6669999999999998},"last":1.603},{"interface":"front","average":{"1min":1.3400000000000001,"5min":1.415,"15min":1.4490000000000001},"min":{"1min":0.70099999999999996,"5min":0.629,"15min":0.629},"max":{"1min":1.6160000000000001,"5min":2.1499999999999999,"15min":2.1499999999999999},"last":1.2450000000000001}]},{"osd":6,"last update":"Wed Aug 9 13:50:27 2023","interfaces":[{"interface":"back","average":{"1min":1.51,"5min":1.532,"15min":1.5469999999999999},"min":{"1min":0.73599999999999999,"5min":0.73599999999999999,"15min":0.69799999999999995},"max":{"1min":2.3100000000000001,"5min":2.7250000000000001,"15min":2.7250000000000001},"last":1.4219999999999999},{"interface":"front","average":{"1min":1.5169999999999999,"5min":1.5389999999999999,"15min":1.5600000000000001},"min":{"1min":0.86699999999999999,"5min":0.86699999999999999,"15min":0.86699999999999999},"max":{"1min":2.3199999999999998,"5min":2.399,"15min":2.399},"last":1.7829999999999999}]},{"osd":7,"last update":"Wed Aug 9 13:50:27 2023","interfaces":[{"interface":"back","average":{"1min":1.532,"5min":1.595,"15min":1.625},"min":{"1min":0.72299999999999998,"5min":0.72299999999999998,"15min":0.72299999999999998},"max":{"1min":2.3900000000000001,"5min":2.3900000000000001,"15min":2.464},"last":1.7130000000000001},{"interface":"front","average":{"1min":1.4350000000000001,"5min":1.512,"15min":1.55},"min":{"1min":0.84499999999999997,"5min":0.76700000000000002,"15min":0.76700000000000002},"max":{"1min":2.274,"5min":2.274,"15min":2.274},"last":1.389}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":34,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":194,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1634,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":694,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":70,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-08-09T13:51:21.285 INFO:tasks.cephfs.filesystem.ceph_manager:clean! 2023-08-09T13:51:21.285 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth caps client.0 mon 'allow r' osd 'allow rw tag cephfs data=cephfs' mds allow 2023-08-09T13:51:21.565 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:21.563+0000 7fe7a6a63640 1 Processor -- start 2023-08-09T13:51:21.566 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:21.563+0000 7fe7a6a63640 1 -- start start 2023-08-09T13:51:21.567 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:21.565+0000 7fe7a6a63640 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7fe7a0070ef0 0x7fe7a00ab8d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:51:21.568 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:21.565+0000 7fe7a6a63640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7fe7a00abe10 0x7fe7a00cb9a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:51:21.568 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:21.565+0000 7fe7a6a63640 1 -- --> v1:172.21.15.43:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe7a00cc330 con 0x7fe7a00cbee0 2023-08-09T13:51:21.568 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:21.565+0000 7fe7a6a63640 1 -- --> v2:172.21.15.43:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe7a00acfa0 con 0x7fe7a00abe10 2023-08-09T13:51:21.568 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:21.565+0000 7fe7a6a63640 1 -- --> v2:172.21.15.203:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe7a015c940 con 0x7fe7a0070ef0 2023-08-09T13:51:21.568 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:21.565+0000 7fe7a5260640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7fe7a00abe10 0x7fe7a00cb9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:51:21.569 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:21.565+0000 7fe7a6262640 1 --1- >> v1:172.21.15.43:6789/0 conn(0x7fe7a00cbee0 0x7fe7a0141e50 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.43:6789/0 says I am v1:172.21.15.43:59418/0 (socket says 172.21.15.43:59418) 2023-08-09T13:51:21.569 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:21.565+0000 7fe7a6262640 1 -- 172.21.15.43:0/2144693556 learned_addr learned my addr 172.21.15.43:0/2144693556 (peer_addr_for_me v1:172.21.15.43:0/0) 2023-08-09T13:51:21.570 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:21.566+0000 7fe7a5a61640 1 --2- 172.21.15.43:0/2144693556 >> v2:172.21.15.203:3300/0 conn(0x7fe7a0070ef0 0x7fe7a00ab8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:51:21.570 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:21.566+0000 7fe7a5260640 1 -- 172.21.15.43:0/2144693556 >> v2:172.21.15.203:3300/0 conn(0x7fe7a0070ef0 msgr2=0x7fe7a00ab8d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-09T13:51:21.570 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:21.566+0000 7fe7a5260640 1 --2- 172.21.15.43:0/2144693556 >> v2:172.21.15.203:3300/0 conn(0x7fe7a0070ef0 0x7fe7a00ab8d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:51:21.570 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:21.566+0000 7fe7a5260640 1 -- 172.21.15.43:0/2144693556 >> v1:172.21.15.43:6789/0 conn(0x7fe7a00cbee0 legacy=0x7fe7a0141e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:51:21.570 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:21.566+0000 7fe7a5260640 1 -- 172.21.15.43:0/2144693556 --> v2:172.21.15.43:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe7a00ac390 con 0x7fe7a00abe10 2023-08-09T13:51:21.570 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:21.567+0000 7fe7a5260640 1 --2- 172.21.15.43:0/2144693556 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fe7a00abe10 0x7fe7a00cb9a0 secure :-1 s=READY pgs=1197 cs=0 l=1 rev1=1 crypto rx=0x7fe7a00cb920 tx=0x7fe79002b960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=97ca91fa8943c927 server_cookie=0 in_seq=0 out_seq=0 2023-08-09T13:51:21.571 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:21.567+0000 7fe7a4a5f640 1 -- 172.21.15.43:0/2144693556 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe790030610 con 0x7fe7a00abe10 2023-08-09T13:51:21.571 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:21.567+0000 7fe7a4a5f640 1 -- 172.21.15.43:0/2144693556 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe790030770 con 0x7fe7a00abe10 2023-08-09T13:51:21.571 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:21.567+0000 7fe7a4a5f640 1 -- 172.21.15.43:0/2144693556 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe790030b00 con 0x7fe7a00abe10 2023-08-09T13:51:21.572 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:21.568+0000 7fe7a6a63640 1 -- 172.21.15.43:0/2144693556 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fe7a00abe10 msgr2=0x7fe7a00cb9a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:51:21.572 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:21.568+0000 7fe7a6a63640 1 --2- 172.21.15.43:0/2144693556 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fe7a00abe10 0x7fe7a00cb9a0 secure :-1 s=READY pgs=1197 cs=0 l=1 rev1=1 crypto rx=0x7fe7a00cb920 tx=0x7fe79002b960 comp rx=0 tx=0).stop 2023-08-09T13:51:21.572 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:21.568+0000 7fe7a6a63640 1 -- 172.21.15.43:0/2144693556 shutdown_connections 2023-08-09T13:51:21.572 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:21.568+0000 7fe7a6a63640 1 --2- 172.21.15.43:0/2144693556 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fe7a00abe10 0x7fe7a00cb9a0 unknown :-1 s=CLOSED pgs=1197 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:51:21.572 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:21.568+0000 7fe7a6a63640 1 --2- 172.21.15.43:0/2144693556 >> v2:172.21.15.203:3300/0 conn(0x7fe7a0070ef0 0x7fe7a00ab8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:51:21.572 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:21.568+0000 7fe7a6a63640 1 -- 172.21.15.43:0/2144693556 >> 172.21.15.43:0/2144693556 conn(0x7fe7a00cddd0 msgr2=0x7fe7a00ce1e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-09T13:51:21.573 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:21.568+0000 7fe7a6a63640 1 -- 172.21.15.43:0/2144693556 shutdown_connections 2023-08-09T13:51:21.573 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:21.568+0000 7fe7a6a63640 1 -- 172.21.15.43:0/2144693556 wait complete. 2023-08-09T13:51:21.573 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:21.570+0000 7fe7a6a63640 1 Processor -- start 2023-08-09T13:51:21.574 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:21.570+0000 7fe7a6a63640 1 -- start start 2023-08-09T13:51:21.574 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:21.571+0000 7fe7a6a63640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fe7a0070ef0 0x7fe7a015de30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:51:21.574 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:21.571+0000 7fe7a6a63640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fe7a00abe10 0x7fe7a00c4710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:51:21.574 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:21.571+0000 7fe7a6a63640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fe7a00cbee0 0x7fe7a01f20e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:51:21.574 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:21.571+0000 7fe7a6a63640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe7a015e370 con 0x7fe7a00abe10 2023-08-09T13:51:21.574 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:21.571+0000 7fe7a6a63640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe7a01445d0 con 0x7fe7a0070ef0 2023-08-09T13:51:21.575 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:21.571+0000 7fe7a6a63640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe7a00c4c50 con 0x7fe7a00cbee0 2023-08-09T13:51:21.576 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:21.572+0000 7fe7a5260640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fe7a00abe10 0x7fe7a00c4710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:51:21.576 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:21.572+0000 7fe7a5260640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fe7a00abe10 0x7fe7a00c4710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:43030/0 (socket says 172.21.15.43:43030) 2023-08-09T13:51:21.576 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:21.572+0000 7fe7a5260640 1 -- 172.21.15.43:0/266665009 learned_addr learned my addr 172.21.15.43:0/266665009 (peer_addr_for_me v2:172.21.15.43:0/0) 2023-08-09T13:51:21.576 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:21.572+0000 7fe7a5a61640 1 --2- 172.21.15.43:0/266665009 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fe7a0070ef0 0x7fe7a015de30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:51:21.576 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:21.572+0000 7fe7a6262640 1 --2- 172.21.15.43:0/266665009 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fe7a00cbee0 0x7fe7a01f20e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:51:21.576 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:21.572+0000 7fe7a5260640 1 -- 172.21.15.43:0/266665009 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fe7a00cbee0 msgr2=0x7fe7a01f20e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:51:21.577 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:21.572+0000 7fe7a5260640 1 --2- 172.21.15.43:0/266665009 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fe7a00cbee0 0x7fe7a01f20e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:51:21.577 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:21.572+0000 7fe7a5260640 1 -- 172.21.15.43:0/266665009 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fe7a0070ef0 msgr2=0x7fe7a015de30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:51:21.577 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:21.572+0000 7fe7a5260640 1 --2- 172.21.15.43:0/266665009 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fe7a0070ef0 0x7fe7a015de30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:51:21.578 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:21.572+0000 7fe7a5260640 1 -- 172.21.15.43:0/266665009 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe790009400 con 0x7fe7a00abe10 2023-08-09T13:51:21.578 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:21.572+0000 7fe7a6262640 1 --2- 172.21.15.43:0/266665009 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fe7a00cbee0 0x7fe7a01f20e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-08-09T13:51:21.578 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:21.572+0000 7fe7a5a61640 1 --2- 172.21.15.43:0/266665009 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fe7a0070ef0 0x7fe7a015de30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-08-09T13:51:21.578 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:21.572+0000 7fe7a5260640 1 --2- 172.21.15.43:0/266665009 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fe7a00abe10 0x7fe7a00c4710 secure :-1 s=READY pgs=1198 cs=0 l=1 rev1=1 crypto rx=0x7fe79002bf70 tx=0x7fe790004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-09T13:51:21.578 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:21.573+0000 7fe7967fc640 1 -- 172.21.15.43:0/266665009 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe79003e040 con 0x7fe7a00abe10 2023-08-09T13:51:21.578 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:21.573+0000 7fe7967fc640 1 -- 172.21.15.43:0/266665009 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe790039cb0 con 0x7fe7a00abe10 2023-08-09T13:51:21.579 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:21.573+0000 7fe7967fc640 1 -- 172.21.15.43:0/266665009 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe790043680 con 0x7fe7a00abe10 2023-08-09T13:51:21.579 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:21.573+0000 7fe7a6a63640 1 -- 172.21.15.43:0/266665009 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe7a01f2620 con 0x7fe7a00abe10 2023-08-09T13:51:21.579 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:21.573+0000 7fe7a6a63640 1 -- 172.21.15.43:0/266665009 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe7a01f2ba0 con 0x7fe7a00abe10 2023-08-09T13:51:21.579 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:21.574+0000 7fe7a6a63640 1 -- 172.21.15.43:0/266665009 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe7a01443a0 con 0x7fe7a00abe10 2023-08-09T13:51:21.580 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:21.575+0000 7fe7967fc640 1 -- 172.21.15.43:0/266665009 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 37) v1 ==== 88443+0+0 (secure 0 0 0) 0x7fe7900437e0 con 0x7fe7a00abe10 2023-08-09T13:51:21.580 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:21.576+0000 7fe7967fc640 1 --2- 172.21.15.43:0/266665009 >> [v2:172.21.15.203:6832/1778614695,v1:172.21.15.203:6833/1778614695] conn(0x7fe78406a170 0x7fe78406c630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:51:21.580 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:21.576+0000 7fe7967fc640 1 -- 172.21.15.43:0/266665009 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(103..103 src has 1..103) v4 ==== 6426+0+0 (secure 0 0 0) 0x7fe7900aab80 con 0x7fe7a00abe10 2023-08-09T13:51:21.580 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:21.577+0000 7fe7a5a61640 1 --2- 172.21.15.43:0/266665009 >> [v2:172.21.15.203:6832/1778614695,v1:172.21.15.203:6833/1778614695] conn(0x7fe78406a170 0x7fe78406c630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:51:21.581 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:21.578+0000 7fe7a5a61640 1 --2- 172.21.15.43:0/266665009 >> [v2:172.21.15.203:6832/1778614695,v1:172.21.15.203:6833/1778614695] conn(0x7fe78406a170 0x7fe78406c630 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7fe79c006fd0 tx=0x7fe79c008040 comp rx=0 tx=0).ready entity=mgr.5819 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-09T13:51:21.583 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:21.580+0000 7fe7967fc640 1 -- 172.21.15.43:0/266665009 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178066 (secure 0 0 0) 0x7fe7a01443a0 con 0x7fe7a00abe10 2023-08-09T13:51:21.857 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:21.855+0000 7fe7a6a63640 1 -- 172.21.15.43:0/266665009 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]} v 0) v1 -- 0x7fe7a01443a0 con 0x7fe7a00abe10 2023-08-09T13:51:21.865 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:21.863+0000 7fe7967fc640 1 -- 172.21.15.43:0/266665009 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]}]=0 updated caps for client.0 v16) v1 ==== 190+0+147 (secure 0 0 0) 0x7fe7a01443a0 con 0x7fe7a00abe10 2023-08-09T13:51:21.866 INFO:teuthology.orchestra.run.smithi043.stdout:[client.0] 2023-08-09T13:51:21.866 INFO:teuthology.orchestra.run.smithi043.stdout: key = AQCPldNkg8zrDxAAM/hnJaE6S9HU/fRwQeBV5w== 2023-08-09T13:51:21.866 INFO:teuthology.orchestra.run.smithi043.stdout: caps mds = "allow" 2023-08-09T13:51:21.866 INFO:teuthology.orchestra.run.smithi043.stdout: caps mon = "allow r" 2023-08-09T13:51:21.867 INFO:teuthology.orchestra.run.smithi043.stdout: caps osd = "allow rw tag cephfs data=cephfs" 2023-08-09T13:51:21.867 INFO:teuthology.orchestra.run.smithi043.stderr:updated caps for client.0 2023-08-09T13:51:21.869 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:21.866+0000 7fe7a6a63640 1 -- 172.21.15.43:0/266665009 >> [v2:172.21.15.203:6832/1778614695,v1:172.21.15.203:6833/1778614695] conn(0x7fe78406a170 msgr2=0x7fe78406c630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:51:21.869 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:21.866+0000 7fe7a6a63640 1 --2- 172.21.15.43:0/266665009 >> [v2:172.21.15.203:6832/1778614695,v1:172.21.15.203:6833/1778614695] conn(0x7fe78406a170 0x7fe78406c630 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7fe79c006fd0 tx=0x7fe79c008040 comp rx=0 tx=0).stop 2023-08-09T13:51:21.869 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:21.866+0000 7fe7a6a63640 1 -- 172.21.15.43:0/266665009 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fe7a00abe10 msgr2=0x7fe7a00c4710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:51:21.869 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:21.866+0000 7fe7a6a63640 1 --2- 172.21.15.43:0/266665009 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fe7a00abe10 0x7fe7a00c4710 secure :-1 s=READY pgs=1198 cs=0 l=1 rev1=1 crypto rx=0x7fe79002bf70 tx=0x7fe790004730 comp rx=0 tx=0).stop 2023-08-09T13:51:21.869 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:21.866+0000 7fe7a6a63640 1 -- 172.21.15.43:0/266665009 shutdown_connections 2023-08-09T13:51:21.869 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:21.866+0000 7fe7a6a63640 1 --2- 172.21.15.43:0/266665009 >> [v2:172.21.15.203:6832/1778614695,v1:172.21.15.203:6833/1778614695] conn(0x7fe78406a170 0x7fe78406c630 unknown :-1 s=CLOSED pgs=66 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:51:21.869 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:21.866+0000 7fe7a6a63640 1 --2- 172.21.15.43:0/266665009 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fe7a00cbee0 0x7fe7a01f20e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:51:21.870 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:21.866+0000 7fe7a6a63640 1 --2- 172.21.15.43:0/266665009 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fe7a00abe10 0x7fe7a00c4710 unknown :-1 s=CLOSED pgs=1198 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:51:21.870 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:21.866+0000 7fe7a6a63640 1 --2- 172.21.15.43:0/266665009 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fe7a0070ef0 0x7fe7a015de30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:51:21.870 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:21.866+0000 7fe7a6a63640 1 -- 172.21.15.43:0/266665009 >> 172.21.15.43:0/266665009 conn(0x7fe7a00cddd0 msgr2=0x7fe7a00ad330 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-09T13:51:21.870 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:21.866+0000 7fe7a6a63640 1 -- 172.21.15.43:0/266665009 shutdown_connections 2023-08-09T13:51:21.870 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:21.866+0000 7fe7a6a63640 1 -- 172.21.15.43:0/266665009 wait complete. 2023-08-09T13:51:21.887 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-08-09T13:51:22.106 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:22.103+0000 7fe0ddc99640 1 Processor -- start 2023-08-09T13:51:22.106 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:22.104+0000 7fe0ddc99640 1 -- start start 2023-08-09T13:51:22.107 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:22.105+0000 7fe0ddc99640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7fe0d80b2b70 0x7fe0d80d2ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:51:22.108 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:22.105+0000 7fe0ddc99640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fe0d80b3050 0x7fe0d80b3f60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:51:22.108 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:22.105+0000 7fe0ddc99640 1 -- --> v1:172.21.15.43:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe0d80ce090 con 0x7fe0d80b2790 2023-08-09T13:51:22.108 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:22.105+0000 7fe0ddc99640 1 -- --> v2:172.21.15.43:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe0d80b3950 con 0x7fe0d80b2b70 2023-08-09T13:51:22.108 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:22.105+0000 7fe0ddc99640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe0d80cd7f0 con 0x7fe0d80b3050 2023-08-09T13:51:22.109 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:22.106+0000 7fe0cffff640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7fe0d80b2b70 0x7fe0d80d2ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:51:22.109 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:22.106+0000 7fe0cffff640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7fe0d80b2b70 0x7fe0d80d2ed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:43040/0 (socket says 172.21.15.43:43040) 2023-08-09T13:51:22.110 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:22.106+0000 7fe0d77fe640 1 --1- >> v1:172.21.15.43:6789/0 conn(0x7fe0d80b2790 0x7fe0d80b5330 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.43:6789/0 says I am v1:172.21.15.43:59430/0 (socket says 172.21.15.43:59430) 2023-08-09T13:51:22.110 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:22.106+0000 7fe0cffff640 1 -- 172.21.15.43:0/178444586 learned_addr learned my addr 172.21.15.43:0/178444586 (peer_addr_for_me v2:172.21.15.43:0/0) 2023-08-09T13:51:22.110 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:22.106+0000 7fe0d7fff640 1 --2- 172.21.15.43:0/178444586 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fe0d80b3050 0x7fe0d80b3f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:51:22.111 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:22.106+0000 7fe0cffff640 1 -- 172.21.15.43:0/178444586 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fe0d80b3050 msgr2=0x7fe0d80b3f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-09T13:51:22.111 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:22.106+0000 7fe0cffff640 1 --2- 172.21.15.43:0/178444586 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fe0d80b3050 0x7fe0d80b3f60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:51:22.111 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:22.106+0000 7fe0cffff640 1 -- 172.21.15.43:0/178444586 >> v1:172.21.15.43:6789/0 conn(0x7fe0d80b2790 legacy=0x7fe0d80b5330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:51:22.111 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:22.106+0000 7fe0d6ffd640 1 -- 172.21.15.43:0/178444586 <== mon.0 v1:172.21.15.43:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3425352945 0 0) 0x7fe0d80ce090 con 0x7fe0d80b2790 2023-08-09T13:51:22.111 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:22.106+0000 7fe0cffff640 1 -- 172.21.15.43:0/178444586 --> v2:172.21.15.43:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe0d80cdad0 con 0x7fe0d80b2b70 2023-08-09T13:51:22.113 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:22.107+0000 7fe0cffff640 1 --2- 172.21.15.43:0/178444586 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fe0d80b2b70 0x7fe0d80d2ed0 secure :-1 s=READY pgs=1200 cs=0 l=1 rev1=1 crypto rx=0x7fe0c0009870 tx=0x7fe0c002b980 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2ce60a65b1d1a331 server_cookie=0 in_seq=0 out_seq=0 2023-08-09T13:51:22.113 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:22.108+0000 7fe0d6ffd640 1 -- 172.21.15.43:0/178444586 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe0c0030840 con 0x7fe0d80b2b70 2023-08-09T13:51:22.113 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:22.108+0000 7fe0d6ffd640 1 -- 172.21.15.43:0/178444586 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe0c00309a0 con 0x7fe0d80b2b70 2023-08-09T13:51:22.113 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:22.108+0000 7fe0d6ffd640 1 -- 172.21.15.43:0/178444586 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe0c0030d30 con 0x7fe0d80b2b70 2023-08-09T13:51:22.113 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:22.108+0000 7fe0ddc99640 1 -- 172.21.15.43:0/178444586 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fe0d80b2b70 msgr2=0x7fe0d80d2ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:51:22.113 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:22.108+0000 7fe0ddc99640 1 --2- 172.21.15.43:0/178444586 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fe0d80b2b70 0x7fe0d80d2ed0 secure :-1 s=READY pgs=1200 cs=0 l=1 rev1=1 crypto rx=0x7fe0c0009870 tx=0x7fe0c002b980 comp rx=0 tx=0).stop 2023-08-09T13:51:22.113 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:22.109+0000 7fe0ddc99640 1 -- 172.21.15.43:0/178444586 shutdown_connections 2023-08-09T13:51:22.114 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:22.109+0000 7fe0ddc99640 1 --2- 172.21.15.43:0/178444586 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fe0d80b3050 0x7fe0d80b3f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:51:22.114 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:22.109+0000 7fe0ddc99640 1 --2- 172.21.15.43:0/178444586 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fe0d80b2b70 0x7fe0d80d2ed0 unknown :-1 s=CLOSED pgs=1200 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:51:22.114 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:22.109+0000 7fe0ddc99640 1 -- 172.21.15.43:0/178444586 >> 172.21.15.43:0/178444586 conn(0x7fe0d809b7b0 msgr2=0x7fe0d80d24e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-09T13:51:22.114 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:22.109+0000 7fe0ddc99640 1 -- 172.21.15.43:0/178444586 shutdown_connections 2023-08-09T13:51:22.114 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:22.109+0000 7fe0ddc99640 1 -- 172.21.15.43:0/178444586 wait complete. 2023-08-09T13:51:22.115 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:22.111+0000 7fe0ddc99640 1 Processor -- start 2023-08-09T13:51:22.116 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:22.111+0000 7fe0ddc99640 1 -- start start 2023-08-09T13:51:22.116 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:22.112+0000 7fe0ddc99640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fe0d80b2790 0x7fe0d8153860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:51:22.116 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:22.112+0000 7fe0ddc99640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fe0d80b2b70 0x7fe0d80cb5e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:51:22.116 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:22.112+0000 7fe0ddc99640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fe0d80b3050 0x7fe0d80b1860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:51:22.116 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:22.112+0000 7fe0ddc99640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe0d80cbb20 con 0x7fe0d80b3050 2023-08-09T13:51:22.116 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:22.112+0000 7fe0ddc99640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe0d80b1da0 con 0x7fe0d80b2790 2023-08-09T13:51:22.116 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:22.112+0000 7fe0ddc99640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe0d80b1fd0 con 0x7fe0d80b2b70 2023-08-09T13:51:22.117 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:22.112+0000 7fe0d7fff640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fe0d80b3050 0x7fe0d80b1860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:51:22.117 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:22.112+0000 7fe0d7fff640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fe0d80b3050 0x7fe0d80b1860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:43042/0 (socket says 172.21.15.43:43042) 2023-08-09T13:51:22.117 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:22.112+0000 7fe0d7fff640 1 -- 172.21.15.43:0/162393000 learned_addr learned my addr 172.21.15.43:0/162393000 (peer_addr_for_me v2:172.21.15.43:0/0) 2023-08-09T13:51:22.117 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:22.113+0000 7fe0d77fe640 1 --2- 172.21.15.43:0/162393000 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fe0d80b2790 0x7fe0d8153860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:51:22.117 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:22.113+0000 7fe0d7fff640 1 -- 172.21.15.43:0/162393000 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fe0d80b2b70 msgr2=0x7fe0d80cb5e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:51:22.117 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:22.113+0000 7fe0d7fff640 1 --2- 172.21.15.43:0/162393000 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fe0d80b2b70 0x7fe0d80cb5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:51:22.117 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:22.113+0000 7fe0d7fff640 1 -- 172.21.15.43:0/162393000 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fe0d80b2790 msgr2=0x7fe0d8153860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:51:22.117 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:22.113+0000 7fe0d7fff640 1 --2- 172.21.15.43:0/162393000 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fe0d80b2790 0x7fe0d8153860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:51:22.118 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:22.113+0000 7fe0d7fff640 1 -- 172.21.15.43:0/162393000 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe0c00094d0 con 0x7fe0d80b3050 2023-08-09T13:51:22.118 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:22.113+0000 7fe0d7fff640 1 --2- 172.21.15.43:0/162393000 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fe0d80b3050 0x7fe0d80b1860 secure :-1 s=READY pgs=1201 cs=0 l=1 rev1=1 crypto rx=0x7fe0d0007e60 tx=0x7fe0d000e4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-09T13:51:22.118 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:22.114+0000 7fe0d4ff9640 1 -- 172.21.15.43:0/162393000 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe0d0014420 con 0x7fe0d80b3050 2023-08-09T13:51:22.119 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:22.115+0000 7fe0d4ff9640 1 -- 172.21.15.43:0/162393000 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe0d00145a0 con 0x7fe0d80b3050 2023-08-09T13:51:22.119 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:22.115+0000 7fe0d4ff9640 1 -- 172.21.15.43:0/162393000 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe0d0014950 con 0x7fe0d80b3050 2023-08-09T13:51:22.119 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:22.115+0000 7fe0ddc99640 1 -- 172.21.15.43:0/162393000 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe0d80c96e0 con 0x7fe0d80b3050 2023-08-09T13:51:22.119 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:22.115+0000 7fe0ddc99640 1 -- 172.21.15.43:0/162393000 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe0d80c9ca0 con 0x7fe0d80b3050 2023-08-09T13:51:22.119 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:22.116+0000 7fe0ddc99640 1 -- 172.21.15.43:0/162393000 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe0d80739f0 con 0x7fe0d80b3050 2023-08-09T13:51:22.120 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:22.117+0000 7fe0d4ff9640 1 -- 172.21.15.43:0/162393000 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 37) v1 ==== 88443+0+0 (secure 0 0 0) 0x7fe0d0014ab0 con 0x7fe0d80b3050 2023-08-09T13:51:22.120 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:22.118+0000 7fe0d4ff9640 1 --2- 172.21.15.43:0/162393000 >> [v2:172.21.15.203:6832/1778614695,v1:172.21.15.203:6833/1778614695] conn(0x7fe0b806a170 0x7fe0b806c630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:51:22.120 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:22.118+0000 7fe0d4ff9640 1 -- 172.21.15.43:0/162393000 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(103..103 src has 1..103) v4 ==== 6426+0+0 (secure 0 0 0) 0x7fe0d0089130 con 0x7fe0d80b3050 2023-08-09T13:51:22.121 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:22.119+0000 7fe0d77fe640 1 --2- 172.21.15.43:0/162393000 >> [v2:172.21.15.203:6832/1778614695,v1:172.21.15.203:6833/1778614695] conn(0x7fe0b806a170 0x7fe0b806c630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:51:22.122 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:22.120+0000 7fe0d77fe640 1 --2- 172.21.15.43:0/162393000 >> [v2:172.21.15.203:6832/1778614695,v1:172.21.15.203:6833/1778614695] conn(0x7fe0b806a170 0x7fe0b806c630 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7fe0c8000f60 tx=0x7fe0c8009040 comp rx=0 tx=0).ready entity=mgr.5819 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-09T13:51:22.128 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:22.123+0000 7fe0d4ff9640 1 -- 172.21.15.43:0/162393000 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178066 (secure 0 0 0) 0x7fe0d00580c0 con 0x7fe0d80b3050 2023-08-09T13:51:22.368 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:22.365+0000 7fe0ddc99640 1 -- 172.21.15.43:0/162393000 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fe0d80b4b00 con 0x7fe0d80b3050 2023-08-09T13:51:22.368 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:22.366+0000 7fe0d4ff9640 1 -- 172.21.15.43:0/162393000 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 43 v43) v1 ==== 94+0+4880 (secure 0 0 0) 0x7fe0d0057c50 con 0x7fe0d80b3050 2023-08-09T13:51:22.369 INFO:teuthology.orchestra.run.smithi043.stdout: 2023-08-09T13:51:22.369 INFO:teuthology.orchestra.run.smithi043.stdout:{"epoch":43,"default_fscid":5,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5966,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.203:6835/4101044758","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6834","nonce":4101044758},{"type":"v1","addr":"172.21.15.203:6835","nonce":4101044758}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":39},{"gid":6460,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.203:6837/2974664437","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6836","nonce":2974664437},{"type":"v1","addr":"172.21.15.203:6837","nonce":2974664437}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":39},{"gid":7527,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.43:6833/795245144","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6832","nonce":795245144},{"type":"v1","addr":"172.21.15.43:6833","nonce":795245144}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":38}],"filesystems":[{"mdsmap":{"epoch":43,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-08-09T13:51:15.716741+0000","modified":"2023-08-09T13:51:19.249966+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":7542},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7542":{"gid":7542,"name":"a","rank":0,"incarnation":40,"state":"up:active","state_seq":3,"addr":"172.21.15.43:6835/3008795294","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6834","nonce":3008795294},{"type":"v1","addr":"172.21.15.43:6835","nonce":3008795294}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[13],"metadata_pool":12,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":5}]} 2023-08-09T13:51:22.369 INFO:teuthology.orchestra.run.smithi043.stderr:dumped fsmap epoch 43 2023-08-09T13:51:22.371 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:22.368+0000 7fe0ddc99640 1 -- 172.21.15.43:0/162393000 >> [v2:172.21.15.203:6832/1778614695,v1:172.21.15.203:6833/1778614695] conn(0x7fe0b806a170 msgr2=0x7fe0b806c630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:51:22.371 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:22.368+0000 7fe0ddc99640 1 --2- 172.21.15.43:0/162393000 >> [v2:172.21.15.203:6832/1778614695,v1:172.21.15.203:6833/1778614695] conn(0x7fe0b806a170 0x7fe0b806c630 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7fe0c8000f60 tx=0x7fe0c8009040 comp rx=0 tx=0).stop 2023-08-09T13:51:22.372 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:22.368+0000 7fe0ddc99640 1 -- 172.21.15.43:0/162393000 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fe0d80b3050 msgr2=0x7fe0d80b1860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:51:22.372 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:22.368+0000 7fe0ddc99640 1 --2- 172.21.15.43:0/162393000 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fe0d80b3050 0x7fe0d80b1860 secure :-1 s=READY pgs=1201 cs=0 l=1 rev1=1 crypto rx=0x7fe0d0007e60 tx=0x7fe0d000e4b0 comp rx=0 tx=0).stop 2023-08-09T13:51:22.372 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:22.369+0000 7fe0ddc99640 1 -- 172.21.15.43:0/162393000 shutdown_connections 2023-08-09T13:51:22.372 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:22.369+0000 7fe0ddc99640 1 --2- 172.21.15.43:0/162393000 >> [v2:172.21.15.203:6832/1778614695,v1:172.21.15.203:6833/1778614695] conn(0x7fe0b806a170 0x7fe0b806c630 unknown :-1 s=CLOSED pgs=67 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:51:22.372 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:22.369+0000 7fe0ddc99640 1 --2- 172.21.15.43:0/162393000 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fe0d80b3050 0x7fe0d80b1860 unknown :-1 s=CLOSED pgs=1201 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:51:22.372 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:22.369+0000 7fe0ddc99640 1 --2- 172.21.15.43:0/162393000 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fe0d80b2b70 0x7fe0d80cb5e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:51:22.372 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:22.369+0000 7fe0ddc99640 1 --2- 172.21.15.43:0/162393000 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fe0d80b2790 0x7fe0d8153860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:51:22.373 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:22.369+0000 7fe0ddc99640 1 -- 172.21.15.43:0/162393000 >> 172.21.15.43:0/162393000 conn(0x7fe0d809b7b0 msgr2=0x7fe0d80b7410 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-09T13:51:22.373 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:22.369+0000 7fe0ddc99640 1 -- 172.21.15.43:0/162393000 shutdown_connections 2023-08-09T13:51:22.373 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:22.369+0000 7fe0ddc99640 1 -- 172.21.15.43:0/162393000 wait complete. 2023-08-09T13:51:22.792 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 43, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2023-08-09T13:51:15.716741+0000', 'modified': '2023-08-09T13:51:19.249966+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 7542}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_7542': {'gid': 7542, 'name': 'a', 'rank': 0, 'incarnation': 40, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.43:6835/3008795294', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.43:6834', 'nonce': 3008795294}, {'type': 'v1', 'addr': '172.21.15.43:6835', 'nonce': 3008795294}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138322906710015, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}}, 'data_pools': [13], 'metadata_pool': 12, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0} 2023-08-09T13:51:22.793 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2023-08-09T13:51:22.793 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.a status 2023-08-09T13:51:23.032 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:23.030+0000 7fc1b15dd640 1 Processor -- start 2023-08-09T13:51:23.033 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:23.030+0000 7fc1b15dd640 1 -- start start 2023-08-09T13:51:23.034 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:23.032+0000 7fc1b15dd640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7fc1ac158980 0x7fc1ac0c8ec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:51:23.035 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:23.032+0000 7fc1b15dd640 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7fc1ac0cbfc0 0x7fc1ac070710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:51:23.035 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:23.032+0000 7fc1b15dd640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fc1ac0cc440 0x7fc1ac070c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:51:23.035 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:23.032+0000 7fc1b15dd640 1 -- --> v2:172.21.15.43:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc1ac158d60 con 0x7fc1ac158980 2023-08-09T13:51:23.035 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:23.032+0000 7fc1b15dd640 1 -- --> v2:172.21.15.203:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc1ac0c9710 con 0x7fc1ac0cbfc0 2023-08-09T13:51:23.035 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:23.032+0000 7fc1b15dd640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc1ac070280 con 0x7fc1ac0cc440 2023-08-09T13:51:23.035 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:23.032+0000 7fc1abfff640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7fc1ac158980 0x7fc1ac0c8ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:51:23.035 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:23.032+0000 7fc1abfff640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7fc1ac158980 0x7fc1ac0c8ec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:43056/0 (socket says 172.21.15.43:43056) 2023-08-09T13:51:23.035 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:23.032+0000 7fc1abfff640 1 -- 172.21.15.43:0/4197279483 learned_addr learned my addr 172.21.15.43:0/4197279483 (peer_addr_for_me v2:172.21.15.43:0/0) 2023-08-09T13:51:23.036 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:23.032+0000 7fc1abfff640 1 -- 172.21.15.43:0/4197279483 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fc1ac0cc440 msgr2=0x7fc1ac070c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-09T13:51:23.036 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:23.032+0000 7fc1abfff640 1 --2- 172.21.15.43:0/4197279483 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fc1ac0cc440 0x7fc1ac070c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:51:23.037 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:23.032+0000 7fc1abfff640 1 -- 172.21.15.43:0/4197279483 >> v2:172.21.15.203:3300/0 conn(0x7fc1ac0cbfc0 msgr2=0x7fc1ac070710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-09T13:51:23.037 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:23.032+0000 7fc1abfff640 1 --2- 172.21.15.43:0/4197279483 >> v2:172.21.15.203:3300/0 conn(0x7fc1ac0cbfc0 0x7fc1ac070710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:51:23.037 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:23.032+0000 7fc1abfff640 1 -- 172.21.15.43:0/4197279483 --> v2:172.21.15.43:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc1ac071190 con 0x7fc1ac158980 2023-08-09T13:51:23.037 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:23.034+0000 7fc1abfff640 1 --2- 172.21.15.43:0/4197279483 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc1ac158980 0x7fc1ac0c8ec0 secure :-1 s=READY pgs=1202 cs=0 l=1 rev1=1 crypto rx=0x7fc1ac06d810 tx=0x7fc19c02da90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c4e5e93410ce5f7d server_cookie=0 in_seq=0 out_seq=0 2023-08-09T13:51:23.037 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:23.034+0000 7fc1ab7fe640 1 -- 172.21.15.43:0/4197279483 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc19c02f850 con 0x7fc1ac158980 2023-08-09T13:51:23.037 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:23.034+0000 7fc1ab7fe640 1 -- 172.21.15.43:0/4197279483 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc19c02f9b0 con 0x7fc1ac158980 2023-08-09T13:51:23.037 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:23.034+0000 7fc1ab7fe640 1 -- 172.21.15.43:0/4197279483 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc19c02fd40 con 0x7fc1ac158980 2023-08-09T13:51:23.038 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:23.035+0000 7fc1b15dd640 1 -- 172.21.15.43:0/4197279483 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc1ac158980 msgr2=0x7fc1ac0c8ec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:51:23.038 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:23.035+0000 7fc1b15dd640 1 --2- 172.21.15.43:0/4197279483 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc1ac158980 0x7fc1ac0c8ec0 secure :-1 s=READY pgs=1202 cs=0 l=1 rev1=1 crypto rx=0x7fc1ac06d810 tx=0x7fc19c02da90 comp rx=0 tx=0).stop 2023-08-09T13:51:23.038 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:23.035+0000 7fc1b15dd640 1 -- 172.21.15.43:0/4197279483 shutdown_connections 2023-08-09T13:51:23.038 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:23.035+0000 7fc1b15dd640 1 --2- 172.21.15.43:0/4197279483 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fc1ac0cc440 0x7fc1ac070c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:51:23.039 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:23.035+0000 7fc1b15dd640 1 --2- 172.21.15.43:0/4197279483 >> v2:172.21.15.203:3300/0 conn(0x7fc1ac0cbfc0 0x7fc1ac070710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:51:23.039 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:23.035+0000 7fc1b15dd640 1 --2- 172.21.15.43:0/4197279483 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc1ac158980 0x7fc1ac0c8ec0 unknown :-1 s=CLOSED pgs=1202 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:51:23.039 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:23.035+0000 7fc1b15dd640 1 -- 172.21.15.43:0/4197279483 >> 172.21.15.43:0/4197279483 conn(0x7fc1ac0af950 msgr2=0x7fc1ac0afd60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-09T13:51:23.039 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:23.035+0000 7fc1b15dd640 1 -- 172.21.15.43:0/4197279483 shutdown_connections 2023-08-09T13:51:23.039 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:23.035+0000 7fc1b15dd640 1 -- 172.21.15.43:0/4197279483 wait complete. 2023-08-09T13:51:23.039 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:23.037+0000 7fc1b15dd640 1 Processor -- start 2023-08-09T13:51:23.040 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:23.037+0000 7fc1b15dd640 1 -- start start 2023-08-09T13:51:23.041 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:23.038+0000 7fc1b15dd640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fc1ac0cbfc0 0x7fc1ac153910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:51:23.041 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:23.038+0000 7fc1b15dd640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fc1ac0cc440 0x7fc1ac0b1970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:51:23.041 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:23.038+0000 7fc1b15dd640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc1ac158980 0x7fc1ac0b2f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:51:23.041 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:23.038+0000 7fc1b15dd640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc1ac0c51a0 con 0x7fc1ac158980 2023-08-09T13:51:23.041 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:23.038+0000 7fc1b15dd640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc1ac154db0 con 0x7fc1ac0cbfc0 2023-08-09T13:51:23.041 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:23.038+0000 7fc1b15dd640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc1ac155650 con 0x7fc1ac0cc440 2023-08-09T13:51:23.042 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:23.038+0000 7fc1b0ddc640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc1ac158980 0x7fc1ac0b2f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:51:23.042 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:23.038+0000 7fc1b0ddc640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc1ac158980 0x7fc1ac0b2f70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:43060/0 (socket says 172.21.15.43:43060) 2023-08-09T13:51:23.042 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:23.038+0000 7fc1b0ddc640 1 -- 172.21.15.43:0/3925603383 learned_addr learned my addr 172.21.15.43:0/3925603383 (peer_addr_for_me v2:172.21.15.43:0/0) 2023-08-09T13:51:23.043 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:23.039+0000 7fc1abfff640 1 --2- 172.21.15.43:0/3925603383 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fc1ac0cbfc0 0x7fc1ac153910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:51:23.044 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:23.039+0000 7fc1a37fe640 1 --2- 172.21.15.43:0/3925603383 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fc1ac0cc440 0x7fc1ac0b1970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:51:23.044 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:23.039+0000 7fc1b0ddc640 1 -- 172.21.15.43:0/3925603383 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fc1ac0cc440 msgr2=0x7fc1ac0b1970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:51:23.044 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:23.039+0000 7fc1b0ddc640 1 --2- 172.21.15.43:0/3925603383 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fc1ac0cc440 0x7fc1ac0b1970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:51:23.044 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:23.039+0000 7fc1b0ddc640 1 -- 172.21.15.43:0/3925603383 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fc1ac0cbfc0 msgr2=0x7fc1ac153910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:51:23.044 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:23.039+0000 7fc1b0ddc640 1 --2- 172.21.15.43:0/3925603383 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fc1ac0cbfc0 0x7fc1ac153910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:51:23.045 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:23.039+0000 7fc1b0ddc640 1 -- 172.21.15.43:0/3925603383 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc19c0094d0 con 0x7fc1ac158980 2023-08-09T13:51:23.045 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:23.039+0000 7fc1abfff640 1 --2- 172.21.15.43:0/3925603383 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fc1ac0cbfc0 0x7fc1ac153910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-08-09T13:51:23.045 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:23.039+0000 7fc1a37fe640 1 --2- 172.21.15.43:0/3925603383 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fc1ac0cc440 0x7fc1ac0b1970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-08-09T13:51:23.045 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:23.039+0000 7fc1b0ddc640 1 --2- 172.21.15.43:0/3925603383 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc1ac158980 0x7fc1ac0b2f70 secure :-1 s=READY pgs=1203 cs=0 l=1 rev1=1 crypto rx=0x7fc1a400b9e0 tx=0x7fc1a400bea0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-09T13:51:23.046 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:23.040+0000 7fc1a97fa640 1 -- 172.21.15.43:0/3925603383 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc1a400cac0 con 0x7fc1ac158980 2023-08-09T13:51:23.046 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:23.040+0000 7fc1a97fa640 1 -- 172.21.15.43:0/3925603383 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc1a400cc40 con 0x7fc1ac158980 2023-08-09T13:51:23.046 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:23.040+0000 7fc1a97fa640 1 -- 172.21.15.43:0/3925603383 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc1a4014440 con 0x7fc1ac158980 2023-08-09T13:51:23.047 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:23.040+0000 7fc1b15dd640 1 -- 172.21.15.43:0/3925603383 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc1ac0b21f0 con 0x7fc1ac158980 2023-08-09T13:51:23.047 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:23.040+0000 7fc1b15dd640 1 -- 172.21.15.43:0/3925603383 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc1ac1552e0 con 0x7fc1ac158980 2023-08-09T13:51:23.051 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:23.043+0000 7fc1a97fa640 1 -- 172.21.15.43:0/3925603383 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 37) v1 ==== 88443+0+0 (secure 0 0 0) 0x7fc1a401c360 con 0x7fc1ac158980 2023-08-09T13:51:23.051 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:23.044+0000 7fc1b53b6740 1 Processor -- start 2023-08-09T13:51:23.052 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:23.045+0000 7fc1b53b6740 1 -- start start 2023-08-09T13:51:23.052 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:23.045+0000 7fc1a97fa640 1 --2- 172.21.15.43:0/3925603383 >> [v2:172.21.15.203:6832/1778614695,v1:172.21.15.203:6833/1778614695] conn(0x7fc19006a170 0x7fc19006c630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:51:23.052 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:23.045+0000 7fc1a97fa640 1 -- 172.21.15.43:0/3925603383 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(103..103 src has 1..103) v4 ==== 6426+0+0 (secure 0 0 0) 0x7fc1a4088750 con 0x7fc1ac158980 2023-08-09T13:51:23.053 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:23.045+0000 7fc1a37fe640 1 --2- 172.21.15.43:0/3925603383 >> [v2:172.21.15.203:6832/1778614695,v1:172.21.15.203:6833/1778614695] conn(0x7fc19006a170 0x7fc19006c630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:51:23.053 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:23.046+0000 7fc1b53b6740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5607b88404a0 0x5607b884cbf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:51:23.053 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:23.046+0000 7fc1b53b6740 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5607b8840880 0x5607b8803f60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:51:23.053 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:23.046+0000 7fc1b53b6740 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x5607b88044a0 0x5607b8804920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:51:23.054 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:23.046+0000 7fc1b53b6740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x5607b87708f0 con 0x5607b88404a0 2023-08-09T13:51:23.054 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:23.046+0000 7fc1b53b6740 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x5607b876dfe0 con 0x5607b8840880 2023-08-09T13:51:23.054 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:23.046+0000 7fc1b53b6740 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x5607b87698d0 con 0x5607b88044a0 2023-08-09T13:51:23.055 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:23.046+0000 7fc1abfff640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5607b88404a0 0x5607b884cbf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:51:23.055 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:23.046+0000 7fc1abfff640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5607b88404a0 0x5607b884cbf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:43066/0 (socket says 172.21.15.43:43066) 2023-08-09T13:51:23.055 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:23.046+0000 7fc1abfff640 1 -- 172.21.15.43:0/230522148 learned_addr learned my addr 172.21.15.43:0/230522148 (peer_addr_for_me v2:172.21.15.43:0/0) 2023-08-09T13:51:23.056 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:23.046+0000 7fc1b0ddc640 1 --2- 172.21.15.43:0/230522148 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x5607b88044a0 0x5607b8804920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:51:23.056 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:23.046+0000 7fc1a37fe640 1 --2- 172.21.15.43:0/230522148 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5607b8840880 0x5607b8803f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:51:23.056 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:23.046+0000 7fc1abfff640 1 -- 172.21.15.43:0/230522148 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x5607b88044a0 msgr2=0x5607b8804920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-09T13:51:23.057 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:23.046+0000 7fc1abfff640 1 --2- 172.21.15.43:0/230522148 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x5607b88044a0 0x5607b8804920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:51:23.057 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:23.047+0000 7fc1abfff640 1 -- 172.21.15.43:0/230522148 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5607b8840880 msgr2=0x5607b8803f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-09T13:51:23.057 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:23.047+0000 7fc1abfff640 1 --2- 172.21.15.43:0/230522148 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5607b8840880 0x5607b8803f60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:51:23.057 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:23.047+0000 7fc1abfff640 1 -- 172.21.15.43:0/230522148 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5607b8722ac0 con 0x5607b88404a0 2023-08-09T13:51:23.058 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:23.047+0000 7fc1a37fe640 1 --2- 172.21.15.43:0/3925603383 >> [v2:172.21.15.203:6832/1778614695,v1:172.21.15.203:6833/1778614695] conn(0x7fc19006a170 0x7fc19006c630 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7fc194006fd0 tx=0x7fc19400ad10 comp rx=0 tx=0).ready entity=mgr.5819 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-09T13:51:23.058 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:23.047+0000 7fc1abfff640 1 --2- 172.21.15.43:0/230522148 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5607b88404a0 0x5607b884cbf0 secure :-1 s=READY pgs=1204 cs=0 l=1 rev1=1 crypto rx=0x7fc19c02e0a0 tx=0x7fc19c032000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3aab94dd42c72002 server_cookie=0 in_seq=0 out_seq=0 2023-08-09T13:51:23.058 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:23.047+0000 7fc1a1ffb640 1 -- 172.21.15.43:0/230522148 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc19c03e070 con 0x5607b88404a0 2023-08-09T13:51:23.058 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:23.047+0000 7fc1a1ffb640 1 -- 172.21.15.43:0/230522148 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc19c02f810 con 0x5607b88404a0 2023-08-09T13:51:23.059 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:23.047+0000 7fc1a1ffb640 1 -- 172.21.15.43:0/230522148 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc19c02fd40 con 0x5607b88404a0 2023-08-09T13:51:23.059 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:23.048+0000 7fc1b53b6740 1 -- 172.21.15.43:0/230522148 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5607b88404a0 msgr2=0x5607b884cbf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:51:23.059 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:23.048+0000 7fc1b53b6740 1 --2- 172.21.15.43:0/230522148 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5607b88404a0 0x5607b884cbf0 secure :-1 s=READY pgs=1204 cs=0 l=1 rev1=1 crypto rx=0x7fc19c02e0a0 tx=0x7fc19c032000 comp rx=0 tx=0).stop 2023-08-09T13:51:23.059 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:23.048+0000 7fc1b53b6740 1 -- 172.21.15.43:0/230522148 shutdown_connections 2023-08-09T13:51:23.060 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:23.048+0000 7fc1b53b6740 1 --2- 172.21.15.43:0/230522148 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x5607b88044a0 0x5607b8804920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:51:23.060 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:23.048+0000 7fc1b53b6740 1 --2- 172.21.15.43:0/230522148 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5607b8840880 0x5607b8803f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:51:23.060 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:23.048+0000 7fc1b53b6740 1 --2- 172.21.15.43:0/230522148 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5607b88404a0 0x5607b884cbf0 unknown :-1 s=CLOSED pgs=1204 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:51:23.060 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:23.048+0000 7fc1b53b6740 1 -- 172.21.15.43:0/230522148 >> 172.21.15.43:0/230522148 conn(0x5607b88010c0 msgr2=0x5607b8849c40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-09T13:51:23.061 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:23.048+0000 7fc1b53b6740 1 -- 172.21.15.43:0/230522148 shutdown_connections 2023-08-09T13:51:23.061 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:23.048+0000 7fc1b53b6740 1 -- 172.21.15.43:0/230522148 wait complete. 2023-08-09T13:51:23.061 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:23.049+0000 7fc1b53b6740 1 -- start start 2023-08-09T13:51:23.061 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:23.049+0000 7fc1b53b6740 1 Processor -- start 2023-08-09T13:51:23.067 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:23.050+0000 7fc1b53b6740 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x5607b88404a0 0x5607b882cba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:51:23.067 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:23.050+0000 7fc1b53b6740 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5607b8840880 0x5607b8831740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:51:23.068 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:23.050+0000 7fc1b53b6740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5607b8831c80 0x5607b8832100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:51:23.068 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:23.050+0000 7fc1b53b6740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x5607b8709d40 con 0x5607b8831c80 2023-08-09T13:51:23.068 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:23.050+0000 7fc1b53b6740 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x5607b87099e0 con 0x5607b8840880 2023-08-09T13:51:23.069 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:23.050+0000 7fc1b53b6740 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x5607b84cff20 con 0x5607b88404a0 2023-08-09T13:51:23.069 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:23.050+0000 7fc1b53b6740 10 client.0 _ll_register_callbacks cb 0x5607b8722160 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-08-09T13:51:23.069 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:23.050+0000 7fc1b0ddc640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5607b8831c80 0x5607b8832100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:51:23.069 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:23.050+0000 7fc1b0ddc640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5607b8831c80 0x5607b8832100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:43074/0 (socket says 172.21.15.43:43074) 2023-08-09T13:51:23.069 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:23.050+0000 7fc1b0ddc640 1 -- 172.21.15.43:0/862043952 learned_addr learned my addr 172.21.15.43:0/862043952 (peer_addr_for_me v2:172.21.15.43:0/0) 2023-08-09T13:51:23.070 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:23.051+0000 7fc1abfff640 1 --2- 172.21.15.43:0/862043952 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x5607b88404a0 0x5607b882cba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:51:23.070 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:23.051+0000 7fc1a37fe640 1 --2- 172.21.15.43:0/862043952 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5607b8840880 0x5607b8831740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:51:23.070 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:23.051+0000 7fc1b0ddc640 1 -- 172.21.15.43:0/862043952 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x5607b88404a0 msgr2=0x5607b882cba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-09T13:51:23.070 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:23.051+0000 7fc1b0ddc640 1 --2- 172.21.15.43:0/862043952 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x5607b88404a0 0x5607b882cba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:51:23.070 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:23.051+0000 7fc1b0ddc640 1 -- 172.21.15.43:0/862043952 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5607b8840880 msgr2=0x5607b8831740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-09T13:51:23.070 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:23.051+0000 7fc1b0ddc640 1 --2- 172.21.15.43:0/862043952 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5607b8840880 0x5607b8831740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:51:23.070 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:23.051+0000 7fc1b0ddc640 1 -- 172.21.15.43:0/862043952 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc1ac071190 con 0x5607b8831c80 2023-08-09T13:51:23.070 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:23.051+0000 7fc1b0ddc640 1 --2- 172.21.15.43:0/862043952 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5607b8831c80 0x5607b8832100 secure :-1 s=READY pgs=1205 cs=0 l=1 rev1=1 crypto rx=0x7fc1a40867c0 tx=0x7fc1a405f640 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a02e90cd219173c1 server_cookie=0 in_seq=0 out_seq=0 2023-08-09T13:51:23.071 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:23.051+0000 7fc19affd640 10 client.0 ms_handle_connect on v2:172.21.15.43:3300/0 2023-08-09T13:51:23.071 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:23.051+0000 7fc19affd640 1 -- 172.21.15.43:0/862043952 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc1a405b160 con 0x5607b8831c80 2023-08-09T13:51:23.071 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:23.052+0000 7fc19affd640 1 -- 172.21.15.43:0/862043952 <== mon.0 v2:172.21.15.43:3300/0 2 ==== osd_map(103..103 src has 1..103) v4 ==== 6426+0+0 (secure 0 0 0) 0x7fc1a405b2c0 con 0x5607b8831c80 2023-08-09T13:51:23.071 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:23.052+0000 7fc1b53b6740 1 -- 172.21.15.43:0/862043952 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5607b8741500 con 0x5607b8831c80 2023-08-09T13:51:23.071 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:23.052+0000 7fc19affd640 1 -- 172.21.15.43:0/862043952 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_get_version_reply(handle=1 version=43) v2 ==== 24+0+0 (secure 0 0 0) 0x7fc1a4053660 con 0x5607b8831c80 2023-08-09T13:51:23.071 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:23.052+0000 7fc1b53b6740 10 client.7689 fetch_fsmap learned FSMap version 43 2023-08-09T13:51:23.072 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:23.052+0000 7fc1b53b6740 1 -- 172.21.15.43:0/862043952 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,fsmap=43,monmap=2+}) v3 -- 0x5607b8767f50 con 0x5607b8831c80 2023-08-09T13:51:23.072 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:23.052+0000 7fc19affd640 1 -- 172.21.15.43:0/862043952 <== mon.0 v2:172.21.15.43:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x5607b8741500 con 0x5607b8831c80 2023-08-09T13:51:23.072 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:23.052+0000 7fc19affd640 1 -- 172.21.15.43:0/862043952 <== mon.0 v2:172.21.15.43:3300/0 5 ==== fsmap(e 43) v1 ==== 3294+0+0 (secure 0 0 0) 0x7fc1a40537c0 con 0x5607b8831c80 2023-08-09T13:51:23.072 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:23.052+0000 7fc1b53b6740 10 client.7689 fetch_fsmap finished waiting for FSMap version 43 2023-08-09T13:51:23.072 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:23.052+0000 7fc1b53b6740 10 client.7689 resolve_mds: resolved name 'a' to daemon mds.a 2023-08-09T13:51:23.072 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:23.052+0000 7fc1b53b6740 20 client.7689 populate_metadata read hostname 'smithi043' 2023-08-09T13:51:23.072 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:23.052+0000 7fc1b53b6740 1 --2- 172.21.15.43:0/862043952 >> [v2:172.21.15.43:6834/3008795294,v1:172.21.15.43:6835/3008795294] conn(0x5607b886e050 0x5607b8870440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:51:23.073 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:23.052+0000 7fc1b53b6740 4 client.7689 mds_command: new command op to 7542 tid=0[{"prefix": "get_command_descriptions"}] 2023-08-09T13:51:23.073 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:23.052+0000 7fc1b53b6740 1 -- 172.21.15.43:0/862043952 --> [v2:172.21.15.43:6834/3008795294,v1:172.21.15.43:6835/3008795294] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5607b873ebc0 con 0x5607b886e050 2023-08-09T13:51:23.073 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:23.053+0000 7fc1abfff640 1 --2- 172.21.15.43:0/862043952 >> [v2:172.21.15.43:6834/3008795294,v1:172.21.15.43:6835/3008795294] conn(0x5607b886e050 0x5607b8870440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:51:23.073 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:23.053+0000 7fc1abfff640 1 --2- 172.21.15.43:0/862043952 >> [v2:172.21.15.43:6834/3008795294,v1:172.21.15.43:6835/3008795294] conn(0x5607b886e050 0x5607b8870440 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ea651757fc2e8a01 server_cookie=4fe527e95badad7 in_seq=0 out_seq=0 2023-08-09T13:51:23.073 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:23.053+0000 7fc19affd640 10 client.7689 ms_handle_connect on v2:172.21.15.43:6834/3008795294 2023-08-09T13:51:23.073 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:23.054+0000 7fc19affd640 1 -- 172.21.15.43:0/862043952 <== mds.0 v2:172.21.15.43:6834/3008795294 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18017 (crc 0 0 0) 0x5607b873ebc0 con 0x5607b886e050 2023-08-09T13:51:23.073 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:23.054+0000 7fc19affd640 10 client.7689 handle_command_reply: tid=0 2023-08-09T13:51:23.074 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:23.054+0000 7fc1b53b6740 1 client.7689 shutdown 2023-08-09T13:51:23.074 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:23.054+0000 7fc1b53b6740 1 -- 172.21.15.43:0/862043952 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5607b8831c80 msgr2=0x5607b8832100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:51:23.074 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:23.054+0000 7fc1b53b6740 1 --2- 172.21.15.43:0/862043952 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5607b8831c80 0x5607b8832100 secure :-1 s=READY pgs=1205 cs=0 l=1 rev1=1 crypto rx=0x7fc1a40867c0 tx=0x7fc1a405f640 comp rx=0 tx=0).stop 2023-08-09T13:51:23.074 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:23.054+0000 7fc1b53b6740 1 -- 172.21.15.43:0/862043952 shutdown_connections 2023-08-09T13:51:23.074 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:23.054+0000 7fc1b53b6740 1 --2- 172.21.15.43:0/862043952 >> [v2:172.21.15.43:6834/3008795294,v1:172.21.15.43:6835/3008795294] conn(0x5607b886e050 0x5607b8870440 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:51:23.075 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:23.054+0000 7fc1b53b6740 1 --2- 172.21.15.43:0/862043952 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5607b8831c80 0x5607b8832100 unknown :-1 s=CLOSED pgs=1205 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:51:23.075 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:23.054+0000 7fc1b53b6740 1 --2- 172.21.15.43:0/862043952 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5607b8840880 0x5607b8831740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:51:23.075 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:23.054+0000 7fc1b53b6740 1 --2- 172.21.15.43:0/862043952 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x5607b88404a0 0x5607b882cba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:51:23.075 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:23.054+0000 7fc1b53b6740 1 -- 172.21.15.43:0/862043952 >> 172.21.15.43:0/862043952 conn(0x5607b88044a0 msgr2=0x5607b87fdf30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-09T13:51:23.075 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:23.054+0000 7fc19affd640 0 client.7689 ms_handle_reset on v2:172.21.15.43:6834/3008795294 2023-08-09T13:51:23.075 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:23.054+0000 7fc1b53b6740 1 -- 172.21.15.43:0/862043952 shutdown_connections 2023-08-09T13:51:23.076 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:23.054+0000 7fc1b53b6740 1 -- 172.21.15.43:0/862043952 wait complete. 2023-08-09T13:51:23.076 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:23.055+0000 7fc1b53b6740 20 client.7689 trim_cache size 0 max 16384 2023-08-09T13:51:23.077 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:23.071+0000 7fc1b53b6740 1 Processor -- start 2023-08-09T13:51:23.078 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:23.072+0000 7fc1b53b6740 1 -- start start 2023-08-09T13:51:23.078 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:23.072+0000 7fc1b53b6740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5607b8831c80 0x5607b883df70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:51:23.078 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:23.072+0000 7fc1b53b6740 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5607b88404a0 0x5607b883e4b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:51:23.078 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:23.072+0000 7fc1b53b6740 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x5607b8840880 0x5607b87fdf30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:51:23.078 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:23.072+0000 7fc1b53b6740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x5607b8721a20 con 0x5607b8831c80 2023-08-09T13:51:23.078 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:23.072+0000 7fc1b53b6740 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x5607b88302b0 con 0x5607b88404a0 2023-08-09T13:51:23.078 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:23.072+0000 7fc1b53b6740 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x5607b8803230 con 0x5607b8840880 2023-08-09T13:51:23.078 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:23.073+0000 7fc1abfff640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5607b8831c80 0x5607b883df70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:51:23.079 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:23.073+0000 7fc1abfff640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5607b8831c80 0x5607b883df70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:43084/0 (socket says 172.21.15.43:43084) 2023-08-09T13:51:23.079 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:23.073+0000 7fc1abfff640 1 -- 172.21.15.43:0/479391951 learned_addr learned my addr 172.21.15.43:0/479391951 (peer_addr_for_me v2:172.21.15.43:0/0) 2023-08-09T13:51:23.079 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:23.073+0000 7fc1abfff640 1 -- 172.21.15.43:0/479391951 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x5607b8840880 msgr2=0x5607b87fdf30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-09T13:51:23.079 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:23.073+0000 7fc1abfff640 1 --2- 172.21.15.43:0/479391951 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x5607b8840880 0x5607b87fdf30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:51:23.079 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:23.073+0000 7fc1abfff640 1 -- 172.21.15.43:0/479391951 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5607b88404a0 msgr2=0x5607b883e4b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-09T13:51:23.079 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:23.073+0000 7fc1abfff640 1 --2- 172.21.15.43:0/479391951 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5607b88404a0 0x5607b883e4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:51:23.079 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:23.073+0000 7fc1abfff640 1 -- 172.21.15.43:0/479391951 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5607b879b300 con 0x5607b8831c80 2023-08-09T13:51:23.080 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:23.073+0000 7fc1abfff640 1 --2- 172.21.15.43:0/479391951 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5607b8831c80 0x5607b883df70 secure :-1 s=READY pgs=1206 cs=0 l=1 rev1=1 crypto rx=0x7fc19c02dad0 tx=0x7fc19c033000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6e638a84d8a9d2ec server_cookie=0 in_seq=0 out_seq=0 2023-08-09T13:51:23.080 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:23.074+0000 7fc1a27fc640 1 -- 172.21.15.43:0/479391951 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc19c03e070 con 0x5607b8831c80 2023-08-09T13:51:23.080 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:23.074+0000 7fc1a27fc640 1 -- 172.21.15.43:0/479391951 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc19c02fc80 con 0x5607b8831c80 2023-08-09T13:51:23.080 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:23.074+0000 7fc1a27fc640 1 -- 172.21.15.43:0/479391951 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc19c04c080 con 0x5607b8831c80 2023-08-09T13:51:23.080 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:23.075+0000 7fc1b53b6740 1 -- 172.21.15.43:0/479391951 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5607b8831c80 msgr2=0x5607b883df70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:51:23.080 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:23.075+0000 7fc1b53b6740 1 --2- 172.21.15.43:0/479391951 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5607b8831c80 0x5607b883df70 secure :-1 s=READY pgs=1206 cs=0 l=1 rev1=1 crypto rx=0x7fc19c02dad0 tx=0x7fc19c033000 comp rx=0 tx=0).stop 2023-08-09T13:51:23.080 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:23.075+0000 7fc1b53b6740 1 -- 172.21.15.43:0/479391951 shutdown_connections 2023-08-09T13:51:23.081 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:23.075+0000 7fc1b53b6740 1 --2- 172.21.15.43:0/479391951 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x5607b8840880 0x5607b87fdf30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:51:23.081 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:23.075+0000 7fc1b53b6740 1 --2- 172.21.15.43:0/479391951 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5607b88404a0 0x5607b883e4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:51:23.081 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:23.075+0000 7fc1b53b6740 1 --2- 172.21.15.43:0/479391951 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5607b8831c80 0x5607b883df70 unknown :-1 s=CLOSED pgs=1206 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:51:23.081 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:23.075+0000 7fc1b53b6740 1 -- 172.21.15.43:0/479391951 >> 172.21.15.43:0/479391951 conn(0x5607b88044a0 msgr2=0x5607b87fa650 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-09T13:51:23.081 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:23.075+0000 7fc1b53b6740 1 -- 172.21.15.43:0/479391951 shutdown_connections 2023-08-09T13:51:23.081 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:23.075+0000 7fc1b53b6740 1 -- 172.21.15.43:0/479391951 wait complete. 2023-08-09T13:51:23.084 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:23.075+0000 7fc1b53b6740 1 -- start start 2023-08-09T13:51:23.084 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:23.076+0000 7fc1b53b6740 1 Processor -- start 2023-08-09T13:51:23.084 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:23.077+0000 7fc1b53b6740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5607b8831c80 0x5607b87fe580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:51:23.084 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:23.077+0000 7fc1b53b6740 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x5607b88404a0 0x5607b88294f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:51:23.085 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:23.077+0000 7fc1b53b6740 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5607b8840880 0x5607b883dd90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:51:23.085 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:23.077+0000 7fc1b53b6740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x5607b87fe160 con 0x5607b8831c80 2023-08-09T13:51:23.085 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:23.077+0000 7fc1b53b6740 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x5607b882a130 con 0x5607b8840880 2023-08-09T13:51:23.085 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:23.077+0000 7fc1b53b6740 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x5607b882c250 con 0x5607b88404a0 2023-08-09T13:51:23.085 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:23.077+0000 7fc1b53b6740 10 client.0 _ll_register_callbacks cb 0x5607b8805e70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-08-09T13:51:23.085 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:23.077+0000 7fc1abfff640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5607b8831c80 0x5607b87fe580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:51:23.085 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:23.077+0000 7fc1abfff640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5607b8831c80 0x5607b87fe580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:43096/0 (socket says 172.21.15.43:43096) 2023-08-09T13:51:23.085 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:23.077+0000 7fc1abfff640 1 -- 172.21.15.43:0/1798421485 learned_addr learned my addr 172.21.15.43:0/1798421485 (peer_addr_for_me v2:172.21.15.43:0/0) 2023-08-09T13:51:23.086 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:23.077+0000 7fc1a37fe640 1 --2- 172.21.15.43:0/1798421485 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x5607b88404a0 0x5607b88294f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:51:23.086 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:23.077+0000 7fc1b0ddc640 1 --2- 172.21.15.43:0/1798421485 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5607b8840880 0x5607b883dd90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:51:23.086 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:23.077+0000 7fc1abfff640 1 -- 172.21.15.43:0/1798421485 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x5607b88404a0 msgr2=0x5607b88294f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-09T13:51:23.086 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:23.077+0000 7fc1abfff640 1 --2- 172.21.15.43:0/1798421485 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x5607b88404a0 0x5607b88294f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:51:23.086 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:23.077+0000 7fc1abfff640 1 -- 172.21.15.43:0/1798421485 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5607b8840880 msgr2=0x5607b883dd90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-09T13:51:23.086 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:23.077+0000 7fc1abfff640 1 --2- 172.21.15.43:0/1798421485 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5607b8840880 0x5607b883dd90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:51:23.086 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:23.077+0000 7fc1abfff640 1 -- 172.21.15.43:0/1798421485 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5607b8722ac0 con 0x5607b8831c80 2023-08-09T13:51:23.087 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:23.078+0000 7fc1abfff640 1 --2- 172.21.15.43:0/1798421485 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5607b8831c80 0x5607b87fe580 secure :-1 s=READY pgs=1207 cs=0 l=1 rev1=1 crypto rx=0x7fc19c006ba0 tx=0x7fc19c004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1c81db86b0e02aa server_cookie=0 in_seq=0 out_seq=0 2023-08-09T13:51:23.087 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:23.078+0000 7fc19affd640 10 client.0 ms_handle_connect on v2:172.21.15.43:3300/0 2023-08-09T13:51:23.087 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:23.078+0000 7fc19affd640 1 -- 172.21.15.43:0/1798421485 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc19c03e070 con 0x5607b8831c80 2023-08-09T13:51:23.087 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:23.078+0000 7fc19affd640 1 -- 172.21.15.43:0/1798421485 <== mon.0 v2:172.21.15.43:3300/0 2 ==== osd_map(103..103 src has 1..103) v4 ==== 6426+0+0 (secure 0 0 0) 0x7fc19c034070 con 0x5607b8831c80 2023-08-09T13:51:23.087 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:23.078+0000 7fc1b53b6740 1 -- 172.21.15.43:0/1798421485 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5607b8791800 con 0x5607b8831c80 2023-08-09T13:51:23.087 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:23.078+0000 7fc19affd640 1 -- 172.21.15.43:0/1798421485 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_get_version_reply(handle=1 version=43) v2 ==== 24+0+0 (secure 0 0 0) 0x7fc19c04c3a0 con 0x5607b8831c80 2023-08-09T13:51:23.087 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:23.078+0000 7fc1b53b6740 10 client.7695 fetch_fsmap learned FSMap version 43 2023-08-09T13:51:23.087 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:23.078+0000 7fc1b53b6740 1 -- 172.21.15.43:0/1798421485 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,fsmap=43,monmap=2+}) v3 -- 0x5607b8708a10 con 0x5607b8831c80 2023-08-09T13:51:23.088 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:23.079+0000 7fc19affd640 1 -- 172.21.15.43:0/1798421485 <== mon.0 v2:172.21.15.43:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x5607b8791800 con 0x5607b8831c80 2023-08-09T13:51:23.088 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:23.079+0000 7fc19affd640 1 -- 172.21.15.43:0/1798421485 <== mon.0 v2:172.21.15.43:3300/0 5 ==== fsmap(e 43) v1 ==== 3294+0+0 (secure 0 0 0) 0x7fc19c053020 con 0x5607b8831c80 2023-08-09T13:51:23.088 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:23.079+0000 7fc1b53b6740 10 client.7695 fetch_fsmap finished waiting for FSMap version 43 2023-08-09T13:51:23.088 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:23.079+0000 7fc1b53b6740 10 client.7695 resolve_mds: resolved name 'a' to daemon mds.a 2023-08-09T13:51:23.088 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:23.079+0000 7fc1b53b6740 20 client.7695 populate_metadata read hostname 'smithi043' 2023-08-09T13:51:23.088 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:23.079+0000 7fc1b53b6740 1 --2- 172.21.15.43:0/1798421485 >> [v2:172.21.15.43:6834/3008795294,v1:172.21.15.43:6835/3008795294] conn(0x5607b87ff410 0x5607b883e2d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:51:23.088 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:23.079+0000 7fc1b53b6740 4 client.7695 mds_command: new command op to 7542 tid=0[{"prefix": "status"}] 2023-08-09T13:51:23.089 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:23.079+0000 7fc1b53b6740 1 -- 172.21.15.43:0/1798421485 --> [v2:172.21.15.43:6834/3008795294,v1:172.21.15.43:6835/3008795294] -- command(tid 0: {"prefix": "status"}) v1 -- 0x5607b87628b0 con 0x5607b87ff410 2023-08-09T13:51:23.089 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:23.079+0000 7fc1a37fe640 1 --2- 172.21.15.43:0/1798421485 >> [v2:172.21.15.43:6834/3008795294,v1:172.21.15.43:6835/3008795294] conn(0x5607b87ff410 0x5607b883e2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:51:23.089 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:23.080+0000 7fc1a37fe640 1 --2- 172.21.15.43:0/1798421485 >> [v2:172.21.15.43:6834/3008795294,v1:172.21.15.43:6835/3008795294] conn(0x5607b87ff410 0x5607b883e2d0 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=45618087f2f60d14 server_cookie=9f16b9d09daa1825 in_seq=0 out_seq=0 2023-08-09T13:51:23.089 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:23.080+0000 7fc19affd640 10 client.7695 ms_handle_connect on v2:172.21.15.43:6834/3008795294 2023-08-09T13:51:23.089 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:23.080+0000 7fc19affd640 1 -- 172.21.15.43:0/1798421485 <== mds.0 v2:172.21.15.43:6834/3008795294 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+327 (crc 0 0 0) 0x5607b87628b0 con 0x5607b87ff410 2023-08-09T13:51:23.089 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:23.080+0000 7fc19affd640 10 client.7695 handle_command_reply: tid=0 2023-08-09T13:51:23.089 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:23.080+0000 7fc1b53b6740 1 client.7695 shutdown 2023-08-09T13:51:23.089 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:23.080+0000 7fc1b53b6740 1 -- 172.21.15.43:0/1798421485 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5607b8831c80 msgr2=0x5607b87fe580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:51:23.090 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:23.080+0000 7fc1b53b6740 1 --2- 172.21.15.43:0/1798421485 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5607b8831c80 0x5607b87fe580 secure :-1 s=READY pgs=1207 cs=0 l=1 rev1=1 crypto rx=0x7fc19c006ba0 tx=0x7fc19c004730 comp rx=0 tx=0).stop 2023-08-09T13:51:23.090 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:23.080+0000 7fc1b53b6740 1 -- 172.21.15.43:0/1798421485 shutdown_connections 2023-08-09T13:51:23.090 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:23.080+0000 7fc1b53b6740 1 --2- 172.21.15.43:0/1798421485 >> [v2:172.21.15.43:6834/3008795294,v1:172.21.15.43:6835/3008795294] conn(0x5607b87ff410 0x5607b883e2d0 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:51:23.090 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:23.080+0000 7fc1b53b6740 1 --2- 172.21.15.43:0/1798421485 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5607b8840880 0x5607b883dd90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:51:23.090 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:23.080+0000 7fc1b53b6740 1 --2- 172.21.15.43:0/1798421485 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x5607b88404a0 0x5607b88294f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:51:23.090 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:23.080+0000 7fc1b53b6740 1 --2- 172.21.15.43:0/1798421485 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5607b8831c80 0x5607b87fe580 unknown :-1 s=CLOSED pgs=1207 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:51:23.090 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:23.080+0000 7fc19affd640 0 client.7695 ms_handle_reset on v2:172.21.15.43:6834/3008795294 2023-08-09T13:51:23.091 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:23.080+0000 7fc1b53b6740 1 -- 172.21.15.43:0/1798421485 >> 172.21.15.43:0/1798421485 conn(0x5607b88044a0 msgr2=0x5607b882c8c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-09T13:51:23.091 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:23.080+0000 7fc1b53b6740 1 -- 172.21.15.43:0/1798421485 shutdown_connections 2023-08-09T13:51:23.091 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:23.080+0000 7fc1b53b6740 1 -- 172.21.15.43:0/1798421485 wait complete. 2023-08-09T13:51:23.091 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:23.081+0000 7fc1b53b6740 20 client.7695 trim_cache size 0 max 16384 2023-08-09T13:51:23.092 INFO:teuthology.orchestra.run.smithi043.stdout:{ 2023-08-09T13:51:23.092 INFO:teuthology.orchestra.run.smithi043.stdout: "cluster_fsid": "daac87ee-1337-4250-a955-aea4e408a39b", 2023-08-09T13:51:23.092 INFO:teuthology.orchestra.run.smithi043.stdout: "whoami": 0, 2023-08-09T13:51:23.092 INFO:teuthology.orchestra.run.smithi043.stdout: "id": 7542, 2023-08-09T13:51:23.092 INFO:teuthology.orchestra.run.smithi043.stdout: "want_state": "up:active", 2023-08-09T13:51:23.093 INFO:teuthology.orchestra.run.smithi043.stdout: "state": "up:active", 2023-08-09T13:51:23.093 INFO:teuthology.orchestra.run.smithi043.stdout: "fs_name": "cephfs", 2023-08-09T13:51:23.093 INFO:teuthology.orchestra.run.smithi043.stdout: "rank_uptime": 6.8686379889999998, 2023-08-09T13:51:23.093 INFO:teuthology.orchestra.run.smithi043.stdout: "mdsmap_epoch": 43, 2023-08-09T13:51:23.093 INFO:teuthology.orchestra.run.smithi043.stdout: "osdmap_epoch": 103, 2023-08-09T13:51:23.094 INFO:teuthology.orchestra.run.smithi043.stdout: "osdmap_epoch_barrier": 103, 2023-08-09T13:51:23.094 INFO:teuthology.orchestra.run.smithi043.stdout: "uptime": 12.823877848 2023-08-09T13:51:23.094 INFO:teuthology.orchestra.run.smithi043.stdout:} 2023-08-09T13:51:23.095 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:23.081+0000 7fc1b15dd640 1 -- 172.21.15.43:0/3925603383 >> [v2:172.21.15.203:6832/1778614695,v1:172.21.15.203:6833/1778614695] conn(0x7fc19006a170 msgr2=0x7fc19006c630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:51:23.095 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:23.081+0000 7fc1b15dd640 1 --2- 172.21.15.43:0/3925603383 >> [v2:172.21.15.203:6832/1778614695,v1:172.21.15.203:6833/1778614695] conn(0x7fc19006a170 0x7fc19006c630 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7fc194006fd0 tx=0x7fc19400ad10 comp rx=0 tx=0).stop 2023-08-09T13:51:23.095 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:23.081+0000 7fc1b15dd640 1 -- 172.21.15.43:0/3925603383 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc1ac158980 msgr2=0x7fc1ac0b2f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:51:23.095 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:23.081+0000 7fc1b15dd640 1 --2- 172.21.15.43:0/3925603383 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc1ac158980 0x7fc1ac0b2f70 secure :-1 s=READY pgs=1203 cs=0 l=1 rev1=1 crypto rx=0x7fc1a400b9e0 tx=0x7fc1a400bea0 comp rx=0 tx=0).stop 2023-08-09T13:51:23.095 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:23.081+0000 7fc1b15dd640 1 -- 172.21.15.43:0/3925603383 shutdown_connections 2023-08-09T13:51:23.096 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:23.081+0000 7fc1b15dd640 1 --2- 172.21.15.43:0/3925603383 >> [v2:172.21.15.203:6832/1778614695,v1:172.21.15.203:6833/1778614695] conn(0x7fc19006a170 0x7fc19006c630 unknown :-1 s=CLOSED pgs=68 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:51:23.096 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:23.081+0000 7fc1b15dd640 1 --2- 172.21.15.43:0/3925603383 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc1ac158980 0x7fc1ac0b2f70 unknown :-1 s=CLOSED pgs=1203 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:51:23.096 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:23.081+0000 7fc1b15dd640 1 --2- 172.21.15.43:0/3925603383 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fc1ac0cc440 0x7fc1ac0b1970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:51:23.096 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:23.081+0000 7fc1b15dd640 1 --2- 172.21.15.43:0/3925603383 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fc1ac0cbfc0 0x7fc1ac153910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:51:23.096 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:23.081+0000 7fc1b15dd640 1 -- 172.21.15.43:0/3925603383 >> 172.21.15.43:0/3925603383 conn(0x7fc1ac0af950 msgr2=0x7fc1ac0cc8c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-09T13:51:23.096 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:23.081+0000 7fc1b15dd640 1 -- 172.21.15.43:0/3925603383 shutdown_connections 2023-08-09T13:51:23.096 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:23.082+0000 7fc1b15dd640 1 -- 172.21.15.43:0/3925603383 wait complete. 2023-08-09T13:51:23.102 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2023-08-09T13:51:23.330 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:23.327+0000 7f4891887640 1 Processor -- start 2023-08-09T13:51:23.331 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:23.328+0000 7f4891887640 1 -- start start 2023-08-09T13:51:23.333 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:23.329+0000 7f4891887640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f488c0b2b70 0x7f488c0d2ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:51:23.333 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:23.329+0000 7f4891887640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f488c0b3050 0x7f488c0b3f60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:51:23.334 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:23.329+0000 7f4891887640 1 -- --> v1:172.21.15.203:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f488c0ce090 con 0x7f488c0b2790 2023-08-09T13:51:23.334 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:23.329+0000 7f4891887640 1 -- --> v2:172.21.15.43:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f488c0b3950 con 0x7f488c0b2b70 2023-08-09T13:51:23.334 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:23.329+0000 7f4891887640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f488c0cd7f0 con 0x7f488c0b3050 2023-08-09T13:51:23.334 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:23.330+0000 7f4883fff640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f488c0b2b70 0x7f488c0d2ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:51:23.335 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:23.330+0000 7f4883fff640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f488c0b2b70 0x7f488c0d2ed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:43104/0 (socket says 172.21.15.43:43104) 2023-08-09T13:51:23.335 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:23.330+0000 7f4883fff640 1 -- 172.21.15.43:0/2105037671 learned_addr learned my addr 172.21.15.43:0/2105037671 (peer_addr_for_me v2:172.21.15.43:0/0) 2023-08-09T13:51:23.335 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:23.330+0000 7f488b7fe640 1 --2- 172.21.15.43:0/2105037671 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f488c0b3050 0x7f488c0b3f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:51:23.336 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:23.330+0000 7f4883fff640 1 -- 172.21.15.43:0/2105037671 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f488c0b3050 msgr2=0x7f488c0b3f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-09T13:51:23.336 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:23.330+0000 7f4883fff640 1 --2- 172.21.15.43:0/2105037671 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f488c0b3050 0x7f488c0b3f60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:51:23.336 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:23.330+0000 7f4883fff640 1 -- 172.21.15.43:0/2105037671 >> v1:172.21.15.203:6789/0 conn(0x7f488c0b2790 legacy=0x7f488c0b5330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:51:23.337 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:23.330+0000 7f4883fff640 1 -- 172.21.15.43:0/2105037671 --> v2:172.21.15.43:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f488c0cdad0 con 0x7f488c0b2b70 2023-08-09T13:51:23.339 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:23.332+0000 7f4883fff640 1 --2- 172.21.15.43:0/2105037671 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f488c0b2b70 0x7f488c0d2ed0 secure :-1 s=READY pgs=1208 cs=0 l=1 rev1=1 crypto rx=0x7f4874009870 tx=0x7f487402b980 comp rx=0 tx=0).ready entity=mon.0 client_cookie=97d46a9915f58282 server_cookie=0 in_seq=0 out_seq=0 2023-08-09T13:51:23.340 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:23.332+0000 7f488a7fc640 1 -- 172.21.15.43:0/2105037671 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4874030840 con 0x7f488c0b2b70 2023-08-09T13:51:23.340 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:23.332+0000 7f488a7fc640 1 -- 172.21.15.43:0/2105037671 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f48740309a0 con 0x7f488c0b2b70 2023-08-09T13:51:23.341 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:23.332+0000 7f488a7fc640 1 -- 172.21.15.43:0/2105037671 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4874030d30 con 0x7f488c0b2b70 2023-08-09T13:51:23.341 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:23.333+0000 7f4891887640 1 -- 172.21.15.43:0/2105037671 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f488c0b2b70 msgr2=0x7f488c0d2ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:51:23.341 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:23.333+0000 7f4891887640 1 --2- 172.21.15.43:0/2105037671 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f488c0b2b70 0x7f488c0d2ed0 secure :-1 s=READY pgs=1208 cs=0 l=1 rev1=1 crypto rx=0x7f4874009870 tx=0x7f487402b980 comp rx=0 tx=0).stop 2023-08-09T13:51:23.342 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:23.333+0000 7f4891887640 1 -- 172.21.15.43:0/2105037671 shutdown_connections 2023-08-09T13:51:23.342 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:23.333+0000 7f4891887640 1 --2- 172.21.15.43:0/2105037671 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f488c0b3050 0x7f488c0b3f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:51:23.342 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:23.333+0000 7f4891887640 1 --2- 172.21.15.43:0/2105037671 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f488c0b2b70 0x7f488c0d2ed0 unknown :-1 s=CLOSED pgs=1208 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:51:23.342 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:23.333+0000 7f4891887640 1 -- 172.21.15.43:0/2105037671 >> 172.21.15.43:0/2105037671 conn(0x7f488c09b7b0 msgr2=0x7f488c0d24e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-09T13:51:23.343 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:23.333+0000 7f4891887640 1 -- 172.21.15.43:0/2105037671 shutdown_connections 2023-08-09T13:51:23.343 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:23.333+0000 7f4891887640 1 -- 172.21.15.43:0/2105037671 wait complete. 2023-08-09T13:51:23.343 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:23.335+0000 7f4891887640 1 Processor -- start 2023-08-09T13:51:23.344 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:23.335+0000 7f4891887640 1 -- start start 2023-08-09T13:51:23.344 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:23.336+0000 7f4891887640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f488c0b2790 0x7f488c155510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:51:23.344 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:23.336+0000 7f4891887640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f488c0b2b70 0x7f488c1540b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:51:23.345 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:23.336+0000 7f4891887640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f488c0b3050 0x7f488c0cbd30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:51:23.345 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:23.336+0000 7f4891887640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f488c1588e0 con 0x7f488c0b3050 2023-08-09T13:51:23.345 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:23.336+0000 7f4891887640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f488c0b00f0 con 0x7f488c0b2790 2023-08-09T13:51:23.346 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:23.336+0000 7f4891887640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f488c158040 con 0x7f488c0b2b70 2023-08-09T13:51:23.346 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:23.336+0000 7f488b7fe640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f488c0b3050 0x7f488c0cbd30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:51:23.346 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:23.336+0000 7f488b7fe640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f488c0b3050 0x7f488c0cbd30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:43106/0 (socket says 172.21.15.43:43106) 2023-08-09T13:51:23.347 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:23.336+0000 7f488b7fe640 1 -- 172.21.15.43:0/204022594 learned_addr learned my addr 172.21.15.43:0/204022594 (peer_addr_for_me v2:172.21.15.43:0/0) 2023-08-09T13:51:23.347 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:23.337+0000 7f488affd640 1 --2- 172.21.15.43:0/204022594 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f488c0b2790 0x7f488c155510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:51:23.347 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:23.337+0000 7f4883fff640 1 --2- 172.21.15.43:0/204022594 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f488c0b2b70 0x7f488c1540b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:51:23.348 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:23.337+0000 7f488b7fe640 1 -- 172.21.15.43:0/204022594 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f488c0b2b70 msgr2=0x7f488c1540b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:51:23.348 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:23.337+0000 7f488b7fe640 1 --2- 172.21.15.43:0/204022594 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f488c0b2b70 0x7f488c1540b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:51:23.348 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:23.337+0000 7f488b7fe640 1 -- 172.21.15.43:0/204022594 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f488c0b2790 msgr2=0x7f488c155510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:51:23.348 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:23.337+0000 7f488b7fe640 1 --2- 172.21.15.43:0/204022594 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f488c0b2790 0x7f488c155510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:51:23.349 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:23.337+0000 7f488b7fe640 1 -- 172.21.15.43:0/204022594 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f48740094d0 con 0x7f488c0b3050 2023-08-09T13:51:23.349 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:23.337+0000 7f488b7fe640 1 --2- 172.21.15.43:0/204022594 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f488c0b3050 0x7f488c0cbd30 secure :-1 s=READY pgs=1209 cs=0 l=1 rev1=1 crypto rx=0x7f4884007e60 tx=0x7f488400e4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-09T13:51:23.351 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:23.337+0000 7f4888ff9640 1 -- 172.21.15.43:0/204022594 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4884014420 con 0x7f488c0b3050 2023-08-09T13:51:23.351 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:23.337+0000 7f4888ff9640 1 -- 172.21.15.43:0/204022594 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f48840145a0 con 0x7f488c0b3050 2023-08-09T13:51:23.351 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:23.337+0000 7f4888ff9640 1 -- 172.21.15.43:0/204022594 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4884014950 con 0x7f488c0b3050 2023-08-09T13:51:23.352 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:23.337+0000 7f4891887640 1 -- 172.21.15.43:0/204022594 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f488c158320 con 0x7f488c0b3050 2023-08-09T13:51:23.352 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:23.338+0000 7f4891887640 1 -- 172.21.15.43:0/204022594 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f488c0c9d90 con 0x7f488c0b3050 2023-08-09T13:51:23.352 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:23.339+0000 7f4891887640 1 -- 172.21.15.43:0/204022594 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f488c0c9bf0 con 0x7f488c0b3050 2023-08-09T13:51:23.352 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:23.340+0000 7f4888ff9640 1 -- 172.21.15.43:0/204022594 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 37) v1 ==== 88443+0+0 (secure 0 0 0) 0x7f4884014ab0 con 0x7f488c0b3050 2023-08-09T13:51:23.353 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:23.341+0000 7f4888ff9640 1 --2- 172.21.15.43:0/204022594 >> [v2:172.21.15.203:6832/1778614695,v1:172.21.15.203:6833/1778614695] conn(0x7f486c06a170 0x7f486c06c630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:51:23.353 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:23.341+0000 7f4888ff9640 1 -- 172.21.15.43:0/204022594 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(103..103 src has 1..103) v4 ==== 6426+0+0 (secure 0 0 0) 0x7f4884089270 con 0x7f488c0b3050 2023-08-09T13:51:23.353 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:23.342+0000 7f488affd640 1 --2- 172.21.15.43:0/204022594 >> [v2:172.21.15.203:6832/1778614695,v1:172.21.15.203:6833/1778614695] conn(0x7f486c06a170 0x7f486c06c630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:51:23.354 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:23.343+0000 7f488affd640 1 --2- 172.21.15.43:0/204022594 >> [v2:172.21.15.203:6832/1778614695,v1:172.21.15.203:6833/1778614695] conn(0x7f486c06a170 0x7f486c06c630 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f487c000f60 tx=0x7f487c009040 comp rx=0 tx=0).ready entity=mgr.5819 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-09T13:51:23.354 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:23.346+0000 7f4888ff9640 1 -- 172.21.15.43:0/204022594 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178066 (secure 0 0 0) 0x7f488c0c9bf0 con 0x7f488c0b3050 2023-08-09T13:51:23.583 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:23.580+0000 7f4891887640 1 -- 172.21.15.43:0/204022594 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7f488c0c9bf0 con 0x7f488c0b3050 2023-08-09T13:51:23.584 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:23.581+0000 7f4888ff9640 1 -- 172.21.15.43:0/204022594 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v43) v1 ==== 53+0+73 (secure 0 0 0) 0x7f488c0c9bf0 con 0x7f488c0b3050 2023-08-09T13:51:23.584 INFO:teuthology.orchestra.run.smithi043.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2023-08-09T13:51:23.587 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:23.584+0000 7f4891887640 1 -- 172.21.15.43:0/204022594 >> [v2:172.21.15.203:6832/1778614695,v1:172.21.15.203:6833/1778614695] conn(0x7f486c06a170 msgr2=0x7f486c06c630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:51:23.587 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:23.584+0000 7f4891887640 1 --2- 172.21.15.43:0/204022594 >> [v2:172.21.15.203:6832/1778614695,v1:172.21.15.203:6833/1778614695] conn(0x7f486c06a170 0x7f486c06c630 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f487c000f60 tx=0x7f487c009040 comp rx=0 tx=0).stop 2023-08-09T13:51:23.588 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:23.584+0000 7f4891887640 1 -- 172.21.15.43:0/204022594 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f488c0b3050 msgr2=0x7f488c0cbd30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:51:23.588 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:23.584+0000 7f4891887640 1 --2- 172.21.15.43:0/204022594 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f488c0b3050 0x7f488c0cbd30 secure :-1 s=READY pgs=1209 cs=0 l=1 rev1=1 crypto rx=0x7f4884007e60 tx=0x7f488400e4b0 comp rx=0 tx=0).stop 2023-08-09T13:51:23.588 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:23.584+0000 7f4891887640 1 -- 172.21.15.43:0/204022594 shutdown_connections 2023-08-09T13:51:23.589 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:23.584+0000 7f4891887640 1 --2- 172.21.15.43:0/204022594 >> [v2:172.21.15.203:6832/1778614695,v1:172.21.15.203:6833/1778614695] conn(0x7f486c06a170 0x7f486c06c630 unknown :-1 s=CLOSED pgs=69 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:51:23.589 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:23.584+0000 7f4891887640 1 --2- 172.21.15.43:0/204022594 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f488c0b3050 0x7f488c0cbd30 unknown :-1 s=CLOSED pgs=1209 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:51:23.589 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:23.584+0000 7f4891887640 1 --2- 172.21.15.43:0/204022594 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f488c0b2b70 0x7f488c1540b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:51:23.589 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:23.584+0000 7f4891887640 1 --2- 172.21.15.43:0/204022594 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f488c0b2790 0x7f488c155510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:51:23.590 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:23.584+0000 7f4891887640 1 -- 172.21.15.43:0/204022594 >> 172.21.15.43:0/204022594 conn(0x7f488c09b7b0 msgr2=0x7f488c0c8020 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-09T13:51:23.590 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:23.584+0000 7f4891887640 1 -- 172.21.15.43:0/204022594 shutdown_connections 2023-08-09T13:51:23.590 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:23.585+0000 7f4891887640 1 -- 172.21.15.43:0/204022594 wait complete. 2023-08-09T13:51:23.607 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2023-08-09T13:51:23.607 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2023-08-09T13:51:23.607 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi043.front.sepia.ceph.com 2023-08-09T13:51:23.608 INFO:tasks.cephfs.mount:self.client.name = client.0 2023-08-09T13:51:23.608 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2023-08-09T13:51:23.608 INFO:tasks.cephfs.mount:self.cephfs_name = None 2023-08-09T13:51:23.609 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2023-08-09T13:51:23.609 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2023-08-09T13:51:23.609 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2023-08-09T13:51:23.609 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-09T13:51:23.610 DEBUG:teuthology.orchestra.run.smithi043:> ip addr 2023-08-09T13:51:23.669 INFO:teuthology.orchestra.run.smithi043.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2023-08-09T13:51:23.670 INFO:teuthology.orchestra.run.smithi043.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2023-08-09T13:51:23.670 INFO:teuthology.orchestra.run.smithi043.stdout: inet 127.0.0.1/8 scope host lo 2023-08-09T13:51:23.670 INFO:teuthology.orchestra.run.smithi043.stdout: valid_lft forever preferred_lft forever 2023-08-09T13:51:23.670 INFO:teuthology.orchestra.run.smithi043.stdout: inet6 ::1/128 scope host 2023-08-09T13:51:23.671 INFO:teuthology.orchestra.run.smithi043.stdout: valid_lft forever preferred_lft forever 2023-08-09T13:51:23.671 INFO:teuthology.orchestra.run.smithi043.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-08-09T13:51:23.671 INFO:teuthology.orchestra.run.smithi043.stdout: link/ether 0c:c4:7a:6b:fd:9c brd ff:ff:ff:ff:ff:ff 2023-08-09T13:51:23.671 INFO:teuthology.orchestra.run.smithi043.stdout: altname enp7s0f0 2023-08-09T13:51:23.671 INFO:teuthology.orchestra.run.smithi043.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-08-09T13:51:23.671 INFO:teuthology.orchestra.run.smithi043.stdout: link/ether 0c:c4:7a:6b:fd:9d brd ff:ff:ff:ff:ff:ff 2023-08-09T13:51:23.671 INFO:teuthology.orchestra.run.smithi043.stdout: altname enp7s0f1 2023-08-09T13:51:23.672 INFO:teuthology.orchestra.run.smithi043.stdout:4: ens1f0: mtu 1500 qdisc mq state UP group default qlen 1000 2023-08-09T13:51:23.672 INFO:teuthology.orchestra.run.smithi043.stdout: link/ether 0c:c4:7a:bd:13:58 brd ff:ff:ff:ff:ff:ff 2023-08-09T13:51:23.672 INFO:teuthology.orchestra.run.smithi043.stdout: altname enp1s0f0 2023-08-09T13:51:23.672 INFO:teuthology.orchestra.run.smithi043.stdout: inet 172.21.15.43/20 brd 172.21.15.255 scope global dynamic noprefixroute ens1f0 2023-08-09T13:51:23.672 INFO:teuthology.orchestra.run.smithi043.stdout: valid_lft 41220sec preferred_lft 41220sec 2023-08-09T13:51:23.672 INFO:teuthology.orchestra.run.smithi043.stdout: inet6 fe80::ec4:7aff:febd:1358/64 scope link 2023-08-09T13:51:23.672 INFO:teuthology.orchestra.run.smithi043.stdout: valid_lft forever preferred_lft forever 2023-08-09T13:51:23.673 INFO:teuthology.orchestra.run.smithi043.stdout:5: ens1f1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-08-09T13:51:23.673 INFO:teuthology.orchestra.run.smithi043.stdout: link/ether 0c:c4:7a:bd:13:59 brd ff:ff:ff:ff:ff:ff 2023-08-09T13:51:23.673 INFO:teuthology.orchestra.run.smithi043.stdout: altname enp1s0f1 2023-08-09T13:51:23.673 INFO:teuthology.orchestra.run.smithi043.stdout:8: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2023-08-09T13:51:23.673 INFO:teuthology.orchestra.run.smithi043.stdout: link/ether 56:43:50:19:f4:0c brd ff:ff:ff:ff:ff:ff 2023-08-09T13:51:23.673 INFO:teuthology.orchestra.run.smithi043.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2023-08-09T13:51:23.674 INFO:teuthology.orchestra.run.smithi043.stdout: valid_lft forever preferred_lft forever 2023-08-09T13:51:23.674 INFO:teuthology.orchestra.run.smithi043.stdout: inet6 fe80::e494:65ff:fe92:a182/64 scope link 2023-08-09T13:51:23.674 INFO:teuthology.orchestra.run.smithi043.stdout: valid_lft forever preferred_lft forever 2023-08-09T13:51:23.674 INFO:teuthology.orchestra.run.smithi043.stdout:9: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2023-08-09T13:51:23.674 INFO:teuthology.orchestra.run.smithi043.stdout: link/ether 7e:93:26:fa:07:8f brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.0 2023-08-09T13:51:23.674 INFO:teuthology.orchestra.run.smithi043.stdout: inet6 fe80::7c93:26ff:fefa:78f/64 scope link 2023-08-09T13:51:23.674 INFO:teuthology.orchestra.run.smithi043.stdout: valid_lft forever preferred_lft forever 2023-08-09T13:51:23.675 INFO:teuthology.orchestra.run.smithi043.stdout:10: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2023-08-09T13:51:23.675 INFO:teuthology.orchestra.run.smithi043.stdout: link/ether 56:43:50:19:f4:0c brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.2 2023-08-09T13:51:23.675 INFO:teuthology.orchestra.run.smithi043.stdout: inet6 fe80::5443:50ff:fe19:f40c/64 scope link 2023-08-09T13:51:23.675 INFO:teuthology.orchestra.run.smithi043.stdout: valid_lft forever preferred_lft forever 2023-08-09T13:51:23.676 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-09T13:51:23.677 DEBUG:teuthology.orchestra.run.smithi043:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2023-08-09T13:51:23.774 INFO:teuthology.orchestra.run.smithi043.stdout:1 2023-08-09T13:51:23.778 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-09T13:51:23.778 DEBUG:teuthology.orchestra.run.smithi043:> route 2023-08-09T13:51:23.836 INFO:teuthology.orchestra.run.smithi043.stdout:Kernel IP routing table 2023-08-09T13:51:23.836 INFO:teuthology.orchestra.run.smithi043.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2023-08-09T13:51:23.836 INFO:teuthology.orchestra.run.smithi043.stdout:default _gateway 0.0.0.0 UG 100 0 0 ens1f0 2023-08-09T13:51:23.836 INFO:teuthology.orchestra.run.smithi043.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 ens1f0 2023-08-09T13:51:23.836 INFO:teuthology.orchestra.run.smithi043.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2023-08-09T13:51:23.837 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-09T13:51:23.837 DEBUG:teuthology.orchestra.run.smithi043:> (cd / && exec bash -c ' 2023-08-09T13:51:23.837 DEBUG:teuthology.orchestra.run.smithi043:> set -e 2023-08-09T13:51:23.838 DEBUG:teuthology.orchestra.run.smithi043:> sudo iptables -A FORWARD -o ens1f0 -i ceph-brx -j ACCEPT 2023-08-09T13:51:23.838 DEBUG:teuthology.orchestra.run.smithi043:> sudo iptables -A FORWARD -i ens1f0 -o ceph-brx -j ACCEPT 2023-08-09T13:51:23.838 DEBUG:teuthology.orchestra.run.smithi043:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o ens1f0 -j MASQUERADE 2023-08-09T13:51:23.839 DEBUG:teuthology.orchestra.run.smithi043:> ') 2023-08-09T13:51:24.050 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-09T13:51:24.050 DEBUG:teuthology.orchestra.run.smithi043:> ip netns list 2023-08-09T13:51:24.104 INFO:teuthology.orchestra.run.smithi043.stdout:ceph-ns--home-ubuntu-cephtest-mnt.2 (id: 1) 2023-08-09T13:51:24.105 INFO:teuthology.orchestra.run.smithi043.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2023-08-09T13:51:24.107 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2023-08-09T13:51:24.107 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-09T13:51:24.108 DEBUG:teuthology.orchestra.run.smithi043:> sudo ip link set brx.0 up 2023-08-09T13:51:24.217 INFO:tasks.cephfs.fuse_mount:Client client.0 config is {} 2023-08-09T13:51:24.218 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-08-09T13:51:24.218 DEBUG:teuthology.orchestra.run.smithi043:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2023-08-09T13:51:24.279 INFO:teuthology.orchestra.run.smithi043.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2023-08-09T13:51:24.280 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-08-09T13:51:24.280 DEBUG:teuthology.orchestra.run.smithi043:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2023-08-09T13:51:24.332 DEBUG:teuthology.orchestra.run.smithi043:> sudo modprobe fuse 2023-08-09T13:51:24.426 INFO:teuthology.orchestra.run:Running command with timeout 30 2023-08-09T13:51:24.426 DEBUG:teuthology.orchestra.run.smithi043:> sudo mount -t fusectl /sys/fs/fuse/connections /sys/fs/fuse/connections 2023-08-09T13:51:24.533 INFO:teuthology.orchestra.run.smithi043.stderr:mount: /sys/fs/fuse/connections: /sys/fs/fuse/connections already mounted or mount point busy. 2023-08-09T13:51:24.537 DEBUG:teuthology.orchestra.run:got remote process result: 32 2023-08-09T13:51:24.538 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-09T13:51:24.538 DEBUG:teuthology.orchestra.run.smithi043:> ls /sys/fs/fuse/connections 2023-08-09T13:51:24.599 INFO:tasks.cephfs.fuse_mount:Pre-mount connections: [] 2023-08-09T13:51:24.600 DEBUG:teuthology.orchestra.run.smithi043:> (cd /home/ubuntu/cephtest && exec sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-fuse -f --admin-socket '/var/run/ceph/$cluster-$name.$pid.asok' /home/ubuntu/cephtest/mnt.0 --id 0) 2023-08-09T13:51:24.609 DEBUG:teuthology.orchestra.run.smithi043:> sudo modprobe fuse 2023-08-09T13:51:24.701 INFO:teuthology.orchestra.run:Running command with timeout 30 2023-08-09T13:51:24.702 DEBUG:teuthology.orchestra.run.smithi043:> sudo mount -t fusectl /sys/fs/fuse/connections /sys/fs/fuse/connections 2023-08-09T13:51:24.787 INFO:teuthology.orchestra.run.smithi043.stderr:mount: /sys/fs/fuse/connections: /sys/fs/fuse/connections already mounted or mount point busy. 2023-08-09T13:51:24.789 DEBUG:teuthology.orchestra.run:got remote process result: 32 2023-08-09T13:51:24.790 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-09T13:51:24.791 DEBUG:teuthology.orchestra.run.smithi043:> ls /sys/fs/fuse/connections 2023-08-09T13:51:24.828 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi043.stderr:2023-08-09T13:51:24.825+0000 7f50dcf67480 -1 init, newargv = 0x564432096290 newargc=15 2023-08-09T13:51:24.828 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi043.stderr:ceph-fuse[121697]: starting ceph client 2023-08-09T13:51:24.837 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi043.stderr:ceph-fuse[121697]: starting fuse 2023-08-09T13:51:25.938 DEBUG:teuthology.orchestra.run.smithi043:> sudo modprobe fuse 2023-08-09T13:51:26.048 INFO:teuthology.orchestra.run:Running command with timeout 30 2023-08-09T13:51:26.049 DEBUG:teuthology.orchestra.run.smithi043:> sudo mount -t fusectl /sys/fs/fuse/connections /sys/fs/fuse/connections 2023-08-09T13:51:26.153 INFO:teuthology.orchestra.run.smithi043.stderr:mount: /sys/fs/fuse/connections: /sys/fs/fuse/connections already mounted or mount point busy. 2023-08-09T13:51:26.157 DEBUG:teuthology.orchestra.run:got remote process result: 32 2023-08-09T13:51:26.158 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-09T13:51:26.159 DEBUG:teuthology.orchestra.run.smithi043:> ls /sys/fs/fuse/connections 2023-08-09T13:51:26.215 INFO:teuthology.orchestra.run.smithi043.stdout:40 2023-08-09T13:51:26.216 INFO:tasks.cephfs.fuse_mount:Post-mount connections: [40] 2023-08-09T13:51:26.216 DEBUG:teuthology.orchestra.run.smithi043:> sudo stdin-killer -- python3 -c ' 2023-08-09T13:51:26.217 DEBUG:teuthology.orchestra.run.smithi043:> import glob 2023-08-09T13:51:26.217 DEBUG:teuthology.orchestra.run.smithi043:> import re 2023-08-09T13:51:26.217 DEBUG:teuthology.orchestra.run.smithi043:> import os 2023-08-09T13:51:26.218 DEBUG:teuthology.orchestra.run.smithi043:> import subprocess 2023-08-09T13:51:26.218 DEBUG:teuthology.orchestra.run.smithi043:> 2023-08-09T13:51:26.218 DEBUG:teuthology.orchestra.run.smithi043:> def _find_admin_socket(client_name): 2023-08-09T13:51:26.218 DEBUG:teuthology.orchestra.run.smithi043:> asok_path = "/var/run/ceph/ceph-client.0.*.asok" 2023-08-09T13:51:26.219 DEBUG:teuthology.orchestra.run.smithi043:> files = glob.glob(asok_path) 2023-08-09T13:51:26.219 DEBUG:teuthology.orchestra.run.smithi043:> mountpoint = "/home/ubuntu/cephtest/mnt.0" 2023-08-09T13:51:26.219 DEBUG:teuthology.orchestra.run.smithi043:> 2023-08-09T13:51:26.220 DEBUG:teuthology.orchestra.run.smithi043:> # Given a non-glob path, it better be there 2023-08-09T13:51:26.220 DEBUG:teuthology.orchestra.run.smithi043:> if "*" not in asok_path: 2023-08-09T13:51:26.220 DEBUG:teuthology.orchestra.run.smithi043:> assert(len(files) == 1) 2023-08-09T13:51:26.221 DEBUG:teuthology.orchestra.run.smithi043:> return files[0] 2023-08-09T13:51:26.221 DEBUG:teuthology.orchestra.run.smithi043:> 2023-08-09T13:51:26.221 DEBUG:teuthology.orchestra.run.smithi043:> for f in files: 2023-08-09T13:51:26.221 DEBUG:teuthology.orchestra.run.smithi043:> pid = re.match(".*\.(\d+)\.asok$", f).group(1) 2023-08-09T13:51:26.222 DEBUG:teuthology.orchestra.run.smithi043:> if os.path.exists("/proc/{0}".format(pid)): 2023-08-09T13:51:26.222 DEBUG:teuthology.orchestra.run.smithi043:> with open("/proc/{0}/cmdline".format(pid), '"'"'r'"'"') as proc_f: 2023-08-09T13:51:26.222 DEBUG:teuthology.orchestra.run.smithi043:> contents = proc_f.read() 2023-08-09T13:51:26.223 DEBUG:teuthology.orchestra.run.smithi043:> if mountpoint in contents: 2023-08-09T13:51:26.223 DEBUG:teuthology.orchestra.run.smithi043:> return f 2023-08-09T13:51:26.223 DEBUG:teuthology.orchestra.run.smithi043:> raise RuntimeError("Client socket {0} not found".format(client_name)) 2023-08-09T13:51:26.224 DEBUG:teuthology.orchestra.run.smithi043:> 2023-08-09T13:51:26.224 DEBUG:teuthology.orchestra.run.smithi043:> print(_find_admin_socket("client.0")) 2023-08-09T13:51:26.224 DEBUG:teuthology.orchestra.run.smithi043:> ' 2023-08-09T13:51:26.420 INFO:teuthology.orchestra.run.smithi043.stdout:/var/run/ceph/ceph-client.0.121697.asok 2023-08-09T13:51:26.424 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:26 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2023-08-09T13:51:26.438 INFO:tasks.cephfs.fuse_mount:Found client admin socket at /var/run/ceph/ceph-client.0.121697.asok 2023-08-09T13:51:26.438 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-09T13:51:26.439 DEBUG:teuthology.orchestra.run.smithi043:> sudo ceph --admin-daemon /var/run/ceph/ceph-client.0.121697.asok status 2023-08-09T13:51:26.691 INFO:teuthology.orchestra.run.smithi043.stdout:{ 2023-08-09T13:51:26.692 INFO:teuthology.orchestra.run.smithi043.stdout: "metadata": { 2023-08-09T13:51:26.692 INFO:teuthology.orchestra.run.smithi043.stdout: "ceph_sha1": "668a86f232c9454b2ce5928d1432a29a72964d85", 2023-08-09T13:51:26.692 INFO:teuthology.orchestra.run.smithi043.stdout: "ceph_version": "ceph version 18.0.0-5389-g668a86f2 (668a86f232c9454b2ce5928d1432a29a72964d85) reef (dev)", 2023-08-09T13:51:26.693 INFO:teuthology.orchestra.run.smithi043.stdout: "entity_id": "0", 2023-08-09T13:51:26.693 INFO:teuthology.orchestra.run.smithi043.stdout: "hostname": "smithi043", 2023-08-09T13:51:26.693 INFO:teuthology.orchestra.run.smithi043.stdout: "mount_point": "/home/ubuntu/cephtest/mnt.0", 2023-08-09T13:51:26.693 INFO:teuthology.orchestra.run.smithi043.stdout: "pid": "121697", 2023-08-09T13:51:26.694 INFO:teuthology.orchestra.run.smithi043.stdout: "root": "/" 2023-08-09T13:51:26.694 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2023-08-09T13:51:26.694 INFO:teuthology.orchestra.run.smithi043.stdout: "dentry_count": 0, 2023-08-09T13:51:26.694 INFO:teuthology.orchestra.run.smithi043.stdout: "dentry_pinned_count": 0, 2023-08-09T13:51:26.695 INFO:teuthology.orchestra.run.smithi043.stdout: "id": 7707, 2023-08-09T13:51:26.695 INFO:teuthology.orchestra.run.smithi043.stdout: "inst": { 2023-08-09T13:51:26.695 INFO:teuthology.orchestra.run.smithi043.stdout: "name": { 2023-08-09T13:51:26.695 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "client", 2023-08-09T13:51:26.696 INFO:teuthology.orchestra.run.smithi043.stdout: "num": 7707 2023-08-09T13:51:26.696 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2023-08-09T13:51:26.696 INFO:teuthology.orchestra.run.smithi043.stdout: "addr": { 2023-08-09T13:51:26.697 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "v1", 2023-08-09T13:51:26.697 INFO:teuthology.orchestra.run.smithi043.stdout: "addr": "192.168.0.1:0", 2023-08-09T13:51:26.697 INFO:teuthology.orchestra.run.smithi043.stdout: "nonce": 2396758877 2023-08-09T13:51:26.697 INFO:teuthology.orchestra.run.smithi043.stdout: } 2023-08-09T13:51:26.698 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2023-08-09T13:51:26.698 INFO:teuthology.orchestra.run.smithi043.stdout: "addr": { 2023-08-09T13:51:26.698 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "v1", 2023-08-09T13:51:26.698 INFO:teuthology.orchestra.run.smithi043.stdout: "addr": "192.168.0.1:0", 2023-08-09T13:51:26.699 INFO:teuthology.orchestra.run.smithi043.stdout: "nonce": 2396758877 2023-08-09T13:51:26.699 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2023-08-09T13:51:26.699 INFO:teuthology.orchestra.run.smithi043.stdout: "inst_str": "client.7707 192.168.0.1:0/2396758877", 2023-08-09T13:51:26.699 INFO:teuthology.orchestra.run.smithi043.stdout: "addr_str": "192.168.0.1:0/2396758877", 2023-08-09T13:51:26.700 INFO:teuthology.orchestra.run.smithi043.stdout: "inode_count": 1, 2023-08-09T13:51:26.700 INFO:teuthology.orchestra.run.smithi043.stdout: "mds_epoch": 43, 2023-08-09T13:51:26.700 INFO:teuthology.orchestra.run.smithi043.stdout: "osd_epoch": 103, 2023-08-09T13:51:26.700 INFO:teuthology.orchestra.run.smithi043.stdout: "osd_epoch_barrier": 0, 2023-08-09T13:51:26.701 INFO:teuthology.orchestra.run.smithi043.stdout: "blocklisted": false, 2023-08-09T13:51:26.701 INFO:teuthology.orchestra.run.smithi043.stdout: "fs_name": "cephfs" 2023-08-09T13:51:26.701 INFO:teuthology.orchestra.run.smithi043.stdout:} 2023-08-09T13:51:26.706 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-09T13:51:26.707 DEBUG:teuthology.orchestra.run.smithi043:> stat --file-system '--printf=%T 2023-08-09T13:51:26.707 DEBUG:teuthology.orchestra.run.smithi043:> ' -- /home/ubuntu/cephtest/mnt.0 2023-08-09T13:51:26.766 INFO:teuthology.orchestra.run.smithi043.stdout:fuseblk 2023-08-09T13:51:26.766 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.0 2023-08-09T13:51:26.767 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-09T13:51:26.767 DEBUG:teuthology.orchestra.run.smithi043:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2023-08-09T13:51:26.871 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs volume ls 2023-08-09T13:51:27.101 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:27.098+0000 7fbd5fa4c640 1 Processor -- start 2023-08-09T13:51:27.102 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:27.099+0000 7fbd5fa4c640 1 -- start start 2023-08-09T13:51:27.103 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:27.101+0000 7fbd5fa4c640 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7fbd580c86f0 0x7fbd580c8b50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:51:27.103 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:27.101+0000 7fbd5fa4c640 1 -- --> v1:172.21.15.43:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbd580c9090 con 0x7fbd58143c20 2023-08-09T13:51:27.104 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:27.101+0000 7fbd5fa4c640 1 -- --> v1:172.21.15.203:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbd5806c440 con 0x7fbd5806c8d0 2023-08-09T13:51:27.104 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:27.101+0000 7fbd5fa4c640 1 -- --> v2:172.21.15.203:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbd58144000 con 0x7fbd580c86f0 2023-08-09T13:51:27.104 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:27.101+0000 7fbd5d7e8640 1 --1- >> v1:172.21.15.43:6789/0 conn(0x7fbd58143c20 0x7fbd5806f540 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.43:6789/0 says I am v1:172.21.15.43:54392/0 (socket says 172.21.15.43:54392) 2023-08-09T13:51:27.104 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:27.101+0000 7fbd5d7e8640 1 -- 172.21.15.43:0/1912407327 learned_addr learned my addr 172.21.15.43:0/1912407327 (peer_addr_for_me v1:172.21.15.43:0/0) 2023-08-09T13:51:27.106 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:27.102+0000 7fbd5cfe7640 1 --2- 172.21.15.43:0/1912407327 >> v2:172.21.15.203:3300/0 conn(0x7fbd580c86f0 0x7fbd580c8b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:51:27.106 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:27.102+0000 7fbd4ffff640 1 -- 172.21.15.43:0/1912407327 <== mon.0 v1:172.21.15.43:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2141727200 0 0) 0x7fbd580c9090 con 0x7fbd58143c20 2023-08-09T13:51:27.106 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:27.102+0000 7fbd4ffff640 1 -- 172.21.15.43:0/1912407327 --> v1:172.21.15.43:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fbd44003520 con 0x7fbd58143c20 2023-08-09T13:51:27.106 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:27.102+0000 7fbd4ffff640 1 -- 172.21.15.43:0/1912407327 <== mon.1 v1:172.21.15.203:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3856684330 0 0) 0x7fbd5806c440 con 0x7fbd5806c8d0 2023-08-09T13:51:27.106 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:27.102+0000 7fbd4ffff640 1 -- 172.21.15.43:0/1912407327 --> v1:172.21.15.203:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fbd580c9090 con 0x7fbd5806c8d0 2023-08-09T13:51:27.106 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:27.102+0000 7fbd4ffff640 1 -- 172.21.15.43:0/1912407327 <== mon.0 v1:172.21.15.43:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3238710440 0 0) 0x7fbd44003520 con 0x7fbd58143c20 2023-08-09T13:51:27.107 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:27.102+0000 7fbd4ffff640 1 -- 172.21.15.43:0/1912407327 >> v2:172.21.15.203:3300/0 conn(0x7fbd580c86f0 msgr2=0x7fbd580c8b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-09T13:51:27.107 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:27.102+0000 7fbd4ffff640 1 --2- 172.21.15.43:0/1912407327 >> v2:172.21.15.203:3300/0 conn(0x7fbd580c86f0 0x7fbd580c8b50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:51:27.107 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:27.102+0000 7fbd4ffff640 1 -- 172.21.15.43:0/1912407327 >> v1:172.21.15.203:6789/0 conn(0x7fbd5806c8d0 legacy=0x7fbd5806cd00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:51:27.107 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:27.102+0000 7fbd4ffff640 1 -- 172.21.15.43:0/1912407327 --> v1:172.21.15.43:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbd5806fc50 con 0x7fbd58143c20 2023-08-09T13:51:27.107 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:27.102+0000 7fbd4ffff640 1 -- 172.21.15.43:0/1912407327 <== mon.0 v1:172.21.15.43:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1285150740 0 0) 0x7fbd540015f0 con 0x7fbd58143c20 2023-08-09T13:51:27.107 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:27.102+0000 7fbd4ffff640 1 -- 172.21.15.43:0/1912407327 >> v1:172.21.15.43:6789/0 conn(0x7fbd58143c20 legacy=0x7fbd5806f540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:51:27.107 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:27.102+0000 7fbd4ffff640 1 --2- 172.21.15.43:0/1912407327 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fbd44003bc0 0x7fbd44003fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:51:27.107 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:27.103+0000 7fbd4ffff640 1 --2- 172.21.15.43:0/1912407327 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fbd440047c0 0x7fbd44004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:51:27.108 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:27.103+0000 7fbd4ffff640 1 --2- 172.21.15.43:0/1912407327 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fbd440052b0 0x7fbd4400b980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:51:27.108 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:27.103+0000 7fbd4ffff640 1 -- 172.21.15.43:0/1912407327 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbd58144000 con 0x7fbd44003bc0 2023-08-09T13:51:27.108 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:27.103+0000 7fbd4ffff640 1 -- 172.21.15.43:0/1912407327 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbd4400c1f0 con 0x7fbd440047c0 2023-08-09T13:51:27.108 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:27.103+0000 7fbd4ffff640 1 -- 172.21.15.43:0/1912407327 --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbd4400c3b0 con 0x7fbd440052b0 2023-08-09T13:51:27.108 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:27.103+0000 7fbd5d7e8640 1 --2- 172.21.15.43:0/1912407327 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fbd44003bc0 0x7fbd44003fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:51:27.108 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:27.103+0000 7fbd5cfe7640 1 --2- 172.21.15.43:0/1912407327 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fbd440047c0 0x7fbd44004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:51:27.108 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:27.103+0000 7fbd5d7e8640 1 -- 172.21.15.43:0/1912407327 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fbd440052b0 msgr2=0x7fbd4400b980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-09T13:51:27.109 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:27.103+0000 7fbd5dfe9640 1 --2- 172.21.15.43:0/1912407327 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fbd440052b0 0x7fbd4400b980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:51:27.109 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:27.103+0000 7fbd5d7e8640 1 --2- 172.21.15.43:0/1912407327 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fbd440052b0 0x7fbd4400b980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:51:27.109 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:27.103+0000 7fbd5d7e8640 1 -- 172.21.15.43:0/1912407327 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fbd440047c0 msgr2=0x7fbd44004c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-09T13:51:27.109 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:27.103+0000 7fbd5d7e8640 1 --2- 172.21.15.43:0/1912407327 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fbd440047c0 0x7fbd44004c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:51:27.109 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:27.103+0000 7fbd5d7e8640 1 -- 172.21.15.43:0/1912407327 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbd44003520 con 0x7fbd44003bc0 2023-08-09T13:51:27.109 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:27.103+0000 7fbd5dfe9640 1 --2- 172.21.15.43:0/1912407327 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fbd440052b0 0x7fbd4400b980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-08-09T13:51:27.109 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:27.103+0000 7fbd5cfe7640 1 --2- 172.21.15.43:0/1912407327 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fbd440047c0 0x7fbd44004c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-08-09T13:51:27.110 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:27.103+0000 7fbd5dfe9640 1 -- 172.21.15.43:0/1912407327 reap_dead start 2023-08-09T13:51:27.111 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:27.104+0000 7fbd5d7e8640 1 --2- 172.21.15.43:0/1912407327 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fbd44003bc0 0x7fbd44003fc0 secure :-1 s=READY pgs=1213 cs=0 l=1 rev1=1 crypto rx=0x7fbd54007f40 tx=0x7fbd440057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=88bed4712de35f1 server_cookie=0 in_seq=0 out_seq=0 2023-08-09T13:51:27.111 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:27.105+0000 7fbd4ffff640 1 -- 172.21.15.43:0/1912407327 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbd54005b90 con 0x7fbd44003bc0 2023-08-09T13:51:27.111 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:27.105+0000 7fbd4ffff640 1 -- 172.21.15.43:0/1912407327 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbd54005d10 con 0x7fbd44003bc0 2023-08-09T13:51:27.111 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:27.105+0000 7fbd4ffff640 1 -- 172.21.15.43:0/1912407327 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbd540072b0 con 0x7fbd44003bc0 2023-08-09T13:51:27.111 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:27.105+0000 7fbd5fa4c640 1 -- 172.21.15.43:0/1912407327 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fbd44003bc0 msgr2=0x7fbd44003fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:51:27.111 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:27.105+0000 7fbd5fa4c640 1 --2- 172.21.15.43:0/1912407327 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fbd44003bc0 0x7fbd44003fc0 secure :-1 s=READY pgs=1213 cs=0 l=1 rev1=1 crypto rx=0x7fbd54007f40 tx=0x7fbd440057f0 comp rx=0 tx=0).stop 2023-08-09T13:51:27.112 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:27.105+0000 7fbd5fa4c640 1 -- 172.21.15.43:0/1912407327 shutdown_connections 2023-08-09T13:51:27.112 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:27.105+0000 7fbd5fa4c640 1 --2- 172.21.15.43:0/1912407327 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fbd44003bc0 0x7fbd44003fc0 unknown :-1 s=CLOSED pgs=1213 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:51:27.112 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:27.105+0000 7fbd5fa4c640 1 -- 172.21.15.43:0/1912407327 >> 172.21.15.43:0/1912407327 conn(0x7fbd580ce0a0 msgr2=0x7fbd580ce480 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-09T13:51:27.112 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:27.106+0000 7fbd5fa4c640 1 -- 172.21.15.43:0/1912407327 shutdown_connections 2023-08-09T13:51:27.112 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:27.106+0000 7fbd5fa4c640 1 -- 172.21.15.43:0/1912407327 wait complete. 2023-08-09T13:51:27.112 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:27.107+0000 7fbd5fa4c640 1 Processor -- start 2023-08-09T13:51:27.112 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:27.108+0000 7fbd5fa4c640 1 -- start start 2023-08-09T13:51:27.114 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:27.109+0000 7fbd5fa4c640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fbd44003bc0 0x7fbd580b2af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:51:27.114 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:27.109+0000 7fbd5fa4c640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fbd58159910 0x7fbd581591c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:51:27.115 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:27.109+0000 7fbd5fa4c640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fbd580c6d80 0x7fbd581423a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:51:27.115 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:27.109+0000 7fbd5fa4c640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbd580c5b30 con 0x7fbd58159910 2023-08-09T13:51:27.115 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:27.109+0000 7fbd5fa4c640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbd580b3030 con 0x7fbd580c6d80 2023-08-09T13:51:27.115 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:27.109+0000 7fbd5fa4c640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbd58159700 con 0x7fbd44003bc0 2023-08-09T13:51:27.115 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:27.109+0000 7fbd5cfe7640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fbd58159910 0x7fbd581591c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:51:27.115 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:27.109+0000 7fbd5cfe7640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fbd58159910 0x7fbd581591c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:58052/0 (socket says 172.21.15.43:58052) 2023-08-09T13:51:27.115 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:27.109+0000 7fbd5cfe7640 1 -- 172.21.15.43:0/2569979611 learned_addr learned my addr 172.21.15.43:0/2569979611 (peer_addr_for_me v2:172.21.15.43:0/0) 2023-08-09T13:51:27.116 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:27.109+0000 7fbd5d7e8640 1 --2- 172.21.15.43:0/2569979611 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fbd44003bc0 0x7fbd580b2af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:51:27.116 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:27.109+0000 7fbd5dfe9640 1 --2- 172.21.15.43:0/2569979611 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fbd580c6d80 0x7fbd581423a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:51:27.116 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:27.109+0000 7fbd5cfe7640 1 -- 172.21.15.43:0/2569979611 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fbd44003bc0 msgr2=0x7fbd580b2af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:51:27.116 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:27.109+0000 7fbd5cfe7640 1 --2- 172.21.15.43:0/2569979611 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fbd44003bc0 0x7fbd580b2af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:51:27.117 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:27.109+0000 7fbd5cfe7640 1 -- 172.21.15.43:0/2569979611 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fbd580c6d80 msgr2=0x7fbd581423a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:51:27.117 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:27.109+0000 7fbd5cfe7640 1 --2- 172.21.15.43:0/2569979611 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fbd580c6d80 0x7fbd581423a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:51:27.117 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:27.110+0000 7fbd5cfe7640 1 -- 172.21.15.43:0/2569979611 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbd580c5d20 con 0x7fbd58159910 2023-08-09T13:51:27.118 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:27.110+0000 7fbd5cfe7640 1 --2- 172.21.15.43:0/2569979611 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fbd58159910 0x7fbd581591c0 secure :-1 s=READY pgs=1214 cs=0 l=1 rev1=1 crypto rx=0x7fbd480107f0 tx=0x7fbd48010cb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-09T13:51:27.118 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:27.110+0000 7fbd4dffb640 1 -- 172.21.15.43:0/2569979611 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbd4800eb60 con 0x7fbd58159910 2023-08-09T13:51:27.118 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:27.110+0000 7fbd4dffb640 1 -- 172.21.15.43:0/2569979611 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbd4800ecc0 con 0x7fbd58159910 2023-08-09T13:51:27.118 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:27.110+0000 7fbd4dffb640 1 -- 172.21.15.43:0/2569979611 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbd4801c400 con 0x7fbd58159910 2023-08-09T13:51:27.119 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:27.110+0000 7fbd5fa4c640 1 -- 172.21.15.43:0/2569979611 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbd580c5fc0 con 0x7fbd58159910 2023-08-09T13:51:27.119 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:27.110+0000 7fbd5fa4c640 1 -- 172.21.15.43:0/2569979611 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbd580c6580 con 0x7fbd58159910 2023-08-09T13:51:27.119 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:27.111+0000 7fbd5fa4c640 1 -- 172.21.15.43:0/2569979611 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbd580739f0 con 0x7fbd58159910 2023-08-09T13:51:27.121 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:27.113+0000 7fbd4dffb640 1 -- 172.21.15.43:0/2569979611 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 37) v1 ==== 88443+0+0 (secure 0 0 0) 0x7fbd4801c580 con 0x7fbd58159910 2023-08-09T13:51:27.121 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:27.114+0000 7fbd4dffb640 1 --2- 172.21.15.43:0/2569979611 >> [v2:172.21.15.203:6832/1778614695,v1:172.21.15.203:6833/1778614695] conn(0x7fbd3c06a170 0x7fbd3c06c630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:51:27.122 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:27.114+0000 7fbd4dffb640 1 -- 172.21.15.43:0/2569979611 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(103..103 src has 1..103) v4 ==== 6426+0+0 (secure 0 0 0) 0x7fbd48088ea0 con 0x7fbd58159910 2023-08-09T13:51:27.122 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:27.115+0000 7fbd5d7e8640 1 --2- 172.21.15.43:0/2569979611 >> [v2:172.21.15.203:6832/1778614695,v1:172.21.15.203:6833/1778614695] conn(0x7fbd3c06a170 0x7fbd3c06c630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:51:27.122 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:27.116+0000 7fbd5d7e8640 1 --2- 172.21.15.43:0/2569979611 >> [v2:172.21.15.203:6832/1778614695,v1:172.21.15.203:6833/1778614695] conn(0x7fbd3c06a170 0x7fbd3c06c630 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7fbd54007f10 tx=0x7fbd54009030 comp rx=0 tx=0).ready entity=mgr.5819 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-09T13:51:27.123 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:27.119+0000 7fbd4dffb640 1 -- 172.21.15.43:0/2569979611 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178066 (secure 0 0 0) 0x7fbd480543c0 con 0x7fbd58159910 2023-08-09T13:51:27.345 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:27.342+0000 7fbd5fa4c640 1 -- 172.21.15.43:0/2569979611 --> [v2:172.21.15.203:6832/1778614695,v1:172.21.15.203:6833/1778614695] -- mgr_command(tid 0: {"prefix": "fs volume ls", "target": ["mon-mgr", ""]}) v1 -- 0x7fbd5806bb10 con 0x7fbd3c06a170 2023-08-09T13:51:27.349 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:27.347+0000 7fbd4dffb640 1 -- 172.21.15.43:0/2569979611 <== mgr.5819 v2:172.21.15.203:6832/1778614695 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+41 (secure 0 0 0) 0x7fbd5403d040 con 0x7fbd3c06a170 2023-08-09T13:51:27.350 INFO:teuthology.orchestra.run.smithi043.stdout:[ 2023-08-09T13:51:27.350 INFO:teuthology.orchestra.run.smithi043.stdout: { 2023-08-09T13:51:27.350 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "cephfs" 2023-08-09T13:51:27.350 INFO:teuthology.orchestra.run.smithi043.stdout: } 2023-08-09T13:51:27.350 INFO:teuthology.orchestra.run.smithi043.stdout:] 2023-08-09T13:51:27.351 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:27.349+0000 7fbd5fa4c640 1 -- 172.21.15.43:0/2569979611 >> [v2:172.21.15.203:6832/1778614695,v1:172.21.15.203:6833/1778614695] conn(0x7fbd3c06a170 msgr2=0x7fbd3c06c630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:51:27.352 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:27.349+0000 7fbd5fa4c640 1 --2- 172.21.15.43:0/2569979611 >> [v2:172.21.15.203:6832/1778614695,v1:172.21.15.203:6833/1778614695] conn(0x7fbd3c06a170 0x7fbd3c06c630 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7fbd54007f10 tx=0x7fbd54009030 comp rx=0 tx=0).stop 2023-08-09T13:51:27.352 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:27.349+0000 7fbd5fa4c640 1 -- 172.21.15.43:0/2569979611 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fbd58159910 msgr2=0x7fbd581591c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:51:27.352 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:27.349+0000 7fbd5fa4c640 1 --2- 172.21.15.43:0/2569979611 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fbd58159910 0x7fbd581591c0 secure :-1 s=READY pgs=1214 cs=0 l=1 rev1=1 crypto rx=0x7fbd480107f0 tx=0x7fbd48010cb0 comp rx=0 tx=0).stop 2023-08-09T13:51:27.352 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:27.349+0000 7fbd5fa4c640 1 -- 172.21.15.43:0/2569979611 shutdown_connections 2023-08-09T13:51:27.353 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:27.349+0000 7fbd5fa4c640 1 --2- 172.21.15.43:0/2569979611 >> [v2:172.21.15.203:6832/1778614695,v1:172.21.15.203:6833/1778614695] conn(0x7fbd3c06a170 0x7fbd3c06c630 unknown :-1 s=CLOSED pgs=70 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:51:27.353 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:27.349+0000 7fbd5fa4c640 1 --2- 172.21.15.43:0/2569979611 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fbd580c6d80 0x7fbd581423a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:51:27.353 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:27.349+0000 7fbd5fa4c640 1 --2- 172.21.15.43:0/2569979611 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fbd58159910 0x7fbd581591c0 unknown :-1 s=CLOSED pgs=1214 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:51:27.353 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:27.349+0000 7fbd5fa4c640 1 --2- 172.21.15.43:0/2569979611 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fbd44003bc0 0x7fbd580b2af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:51:27.353 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:27.349+0000 7fbd5fa4c640 1 -- 172.21.15.43:0/2569979611 >> 172.21.15.43:0/2569979611 conn(0x7fbd580ce0a0 msgr2=0x7fbd580cc3a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-09T13:51:27.353 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:27.349+0000 7fbd5fa4c640 1 -- 172.21.15.43:0/2569979611 shutdown_connections 2023-08-09T13:51:27.353 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:27.349+0000 7fbd5fa4c640 1 -- 172.21.15.43:0/2569979611 wait complete. 2023-08-09T13:51:27.372 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph mgr module enable snap_schedule 2023-08-09T13:51:27.607 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:27.605+0000 7f878ffff640 1 Processor -- start 2023-08-09T13:51:27.608 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:27.605+0000 7f878ffff640 1 -- start start 2023-08-09T13:51:27.609 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:27.607+0000 7f878ffff640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f87900c8b80 0x7f879015b0f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:51:27.609 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:27.607+0000 7f878ffff640 1 -- --> v1:172.21.15.43:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f879006b480 con 0x7f879006f2c0 2023-08-09T13:51:27.609 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:27.607+0000 7f878ffff640 1 -- --> v1:172.21.15.203:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8790071580 con 0x7f8790157ce0 2023-08-09T13:51:27.610 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:27.607+0000 7f878ffff640 1 -- --> v2:172.21.15.43:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f87901580c0 con 0x7f87900c8b80 2023-08-09T13:51:27.610 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:27.607+0000 7f878e7fc640 1 --1- >> v1:172.21.15.43:6789/0 conn(0x7f879006f2c0 0x7f87900c8470 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.43:6789/0 says I am v1:172.21.15.43:54394/0 (socket says 172.21.15.43:54394) 2023-08-09T13:51:27.610 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:27.607+0000 7f878e7fc640 1 -- 172.21.15.43:0/2378241453 learned_addr learned my addr 172.21.15.43:0/2378241453 (peer_addr_for_me v1:172.21.15.43:0/0) 2023-08-09T13:51:27.611 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:27.607+0000 7f878f7fe640 1 --2- 172.21.15.43:0/2378241453 >> v2:172.21.15.43:3300/0 conn(0x7f87900c8b80 0x7f879015b0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:51:27.611 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:27.608+0000 7f878dffb640 1 -- 172.21.15.43:0/2378241453 <== mon.0 v1:172.21.15.43:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3170361918 0 0) 0x7f879006b480 con 0x7f879006f2c0 2023-08-09T13:51:27.611 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:27.608+0000 7f878dffb640 1 -- 172.21.15.43:0/2378241453 --> v1:172.21.15.43:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f8778003520 con 0x7f879006f2c0 2023-08-09T13:51:27.611 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:27.608+0000 7f878f7fe640 1 -- 172.21.15.43:0/2378241453 >> v1:172.21.15.203:6789/0 conn(0x7f8790157ce0 legacy=0x7f879006bc40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:51:27.611 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:27.608+0000 7f878f7fe640 1 -- 172.21.15.43:0/2378241453 >> v1:172.21.15.43:6789/0 conn(0x7f879006f2c0 legacy=0x7f87900c8470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:51:27.611 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:27.608+0000 7f878f7fe640 1 -- 172.21.15.43:0/2378241453 --> v2:172.21.15.43:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f879006c350 con 0x7f87900c8b80 2023-08-09T13:51:27.612 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:27.609+0000 7f878f7fe640 1 --2- 172.21.15.43:0/2378241453 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f87900c8b80 0x7f879015b0f0 secure :-1 s=READY pgs=1216 cs=0 l=1 rev1=1 crypto rx=0x7f879006c540 tx=0x7f878802bea0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e614308e56b2712f server_cookie=0 in_seq=0 out_seq=0 2023-08-09T13:51:27.612 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:27.609+0000 7f878dffb640 1 -- 172.21.15.43:0/2378241453 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f878802cc50 con 0x7f87900c8b80 2023-08-09T13:51:27.612 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:27.609+0000 7f878dffb640 1 -- 172.21.15.43:0/2378241453 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f878802cdb0 con 0x7f87900c8b80 2023-08-09T13:51:27.613 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:27.609+0000 7f878dffb640 1 -- 172.21.15.43:0/2378241453 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8788034660 con 0x7f87900c8b80 2023-08-09T13:51:27.613 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:27.610+0000 7f878ffff640 1 -- 172.21.15.43:0/2378241453 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f87900c8b80 msgr2=0x7f879015b0f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:51:27.613 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:27.610+0000 7f878ffff640 1 --2- 172.21.15.43:0/2378241453 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f87900c8b80 0x7f879015b0f0 secure :-1 s=READY pgs=1216 cs=0 l=1 rev1=1 crypto rx=0x7f879006c540 tx=0x7f878802bea0 comp rx=0 tx=0).stop 2023-08-09T13:51:27.613 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:27.610+0000 7f878ffff640 1 -- 172.21.15.43:0/2378241453 shutdown_connections 2023-08-09T13:51:27.614 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:27.610+0000 7f878ffff640 1 --2- 172.21.15.43:0/2378241453 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f87900c8b80 0x7f879015b0f0 unknown :-1 s=CLOSED pgs=1216 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:51:27.614 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:27.610+0000 7f878ffff640 1 -- 172.21.15.43:0/2378241453 >> 172.21.15.43:0/2378241453 conn(0x7f87900af9f0 msgr2=0x7f87900afdd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-09T13:51:27.614 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:27.610+0000 7f878ffff640 1 -- 172.21.15.43:0/2378241453 shutdown_connections 2023-08-09T13:51:27.614 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:27.610+0000 7f878ffff640 1 -- 172.21.15.43:0/2378241453 wait complete. 2023-08-09T13:51:27.614 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:27.612+0000 7f878ffff640 1 Processor -- start 2023-08-09T13:51:27.615 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:27.612+0000 7f878ffff640 1 -- start start 2023-08-09T13:51:27.615 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:27.613+0000 7f878ffff640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f879006f2c0 0x7f87901f17f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:51:27.616 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:27.613+0000 7f878ffff640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f87900c8b80 0x7f8790146540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:51:27.616 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:27.613+0000 7f878ffff640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f8790157ce0 0x7f87901f5310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:51:27.616 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:27.613+0000 7f878ffff640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f87901433d0 con 0x7f87900c8b80 2023-08-09T13:51:27.616 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:27.613+0000 7f878ffff640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f87901436b0 con 0x7f879006f2c0 2023-08-09T13:51:27.616 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:27.613+0000 7f878ffff640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8790143990 con 0x7f8790157ce0 2023-08-09T13:51:27.616 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:27.613+0000 7f878e7fc640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f87900c8b80 0x7f8790146540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:51:27.618 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:27.613+0000 7f878e7fc640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f87900c8b80 0x7f8790146540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:58070/0 (socket says 172.21.15.43:58070) 2023-08-09T13:51:27.618 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:27.613+0000 7f878e7fc640 1 -- 172.21.15.43:0/3904161864 learned_addr learned my addr 172.21.15.43:0/3904161864 (peer_addr_for_me v2:172.21.15.43:0/0) 2023-08-09T13:51:27.618 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:27.614+0000 7f878effd640 1 --2- 172.21.15.43:0/3904161864 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f879006f2c0 0x7f87901f17f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:51:27.618 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:27.614+0000 7f878f7fe640 1 --2- 172.21.15.43:0/3904161864 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f8790157ce0 0x7f87901f5310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:51:27.618 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:27.614+0000 7f878e7fc640 1 -- 172.21.15.43:0/3904161864 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f8790157ce0 msgr2=0x7f87901f5310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:51:27.618 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:27.614+0000 7f878e7fc640 1 --2- 172.21.15.43:0/3904161864 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f8790157ce0 0x7f87901f5310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:51:27.618 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:27.614+0000 7f878e7fc640 1 -- 172.21.15.43:0/3904161864 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f879006f2c0 msgr2=0x7f87901f17f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:51:27.619 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:27.614+0000 7f878e7fc640 1 --2- 172.21.15.43:0/3904161864 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f879006f2c0 0x7f87901f17f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:51:27.619 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:27.614+0000 7f878e7fc640 1 -- 172.21.15.43:0/3904161864 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8788009a70 con 0x7f87900c8b80 2023-08-09T13:51:27.619 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:27.614+0000 7f878e7fc640 1 --2- 172.21.15.43:0/3904161864 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f87900c8b80 0x7f8790146540 secure :-1 s=READY pgs=1217 cs=0 l=1 rev1=1 crypto rx=0x7f878400c500 tx=0x7f878400c9c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-09T13:51:27.619 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:27.615+0000 7f87777fe640 1 -- 172.21.15.43:0/3904161864 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8784005ce0 con 0x7f87900c8b80 2023-08-09T13:51:27.619 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:27.615+0000 7f87777fe640 1 -- 172.21.15.43:0/3904161864 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8784005e40 con 0x7f87900c8b80 2023-08-09T13:51:27.619 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:27.615+0000 7f87777fe640 1 -- 172.21.15.43:0/3904161864 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8784007330 con 0x7f87900c8b80 2023-08-09T13:51:27.619 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:27.615+0000 7f878ffff640 1 -- 172.21.15.43:0/3904161864 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8790143c70 con 0x7f87900c8b80 2023-08-09T13:51:27.620 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:27.615+0000 7f878ffff640 1 -- 172.21.15.43:0/3904161864 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8790143050 con 0x7f87900c8b80 2023-08-09T13:51:27.620 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:27.616+0000 7f878ffff640 1 -- 172.21.15.43:0/3904161864 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8790144190 con 0x7f87900c8b80 2023-08-09T13:51:27.620 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:27.617+0000 7f87777fe640 1 -- 172.21.15.43:0/3904161864 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 37) v1 ==== 88443+0+0 (secure 0 0 0) 0x7f8784007550 con 0x7f87900c8b80 2023-08-09T13:51:27.621 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:27.618+0000 7f87777fe640 1 --2- 172.21.15.43:0/3904161864 >> [v2:172.21.15.203:6832/1778614695,v1:172.21.15.203:6833/1778614695] conn(0x7f877c06a170 0x7f877c06c630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:51:27.621 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:27.618+0000 7f87777fe640 1 -- 172.21.15.43:0/3904161864 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(103..103 src has 1..103) v4 ==== 6426+0+0 (secure 0 0 0) 0x7f8784087f10 con 0x7f87900c8b80 2023-08-09T13:51:27.621 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:27.619+0000 7f878effd640 1 --2- 172.21.15.43:0/3904161864 >> [v2:172.21.15.203:6832/1778614695,v1:172.21.15.203:6833/1778614695] conn(0x7f877c06a170 0x7f877c06c630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:51:27.622 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:27.620+0000 7f878effd640 1 --2- 172.21.15.43:0/3904161864 >> [v2:172.21.15.203:6832/1778614695,v1:172.21.15.203:6833/1778614695] conn(0x7f877c06a170 0x7f877c06c630 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f8780000f60 tx=0x7f8780009040 comp rx=0 tx=0).ready entity=mgr.5819 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-09T13:51:27.625 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:27.623+0000 7f87777fe640 1 -- 172.21.15.43:0/3904161864 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178066 (secure 0 0 0) 0x7f8790144190 con 0x7f87900c8b80 2023-08-09T13:51:27.889 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:27.886+0000 7f878ffff640 1 -- 172.21.15.43:0/3904161864 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "mgr module enable", "module": "snap_schedule"} v 0) v1 -- 0x7f8790144190 con 0x7f87900c8b80 2023-08-09T13:51:28.339 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:28.335+0000 7f87777fe640 1 -- 172.21.15.43:0/3904161864 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mgrmap(e 38) v1 ==== 88460+0+0 (secure 0 0 0) 0x7f8784053800 con 0x7f87900c8b80 2023-08-09T13:51:28.341 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:28.338+0000 7f87777fe640 1 -- 172.21.15.43:0/3904161864 <== mon.0 v2:172.21.15.43:3300/0 8 ==== mon_command_ack([{"prefix": "mgr module enable", "module": "snap_schedule"}]=0 v38) v1 ==== 92+0+0 (secure 0 0 0) 0x7f878405f050 con 0x7f87900c8b80 2023-08-09T13:51:28.344 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:28.341+0000 7f878ffff640 1 -- 172.21.15.43:0/3904161864 >> [v2:172.21.15.203:6832/1778614695,v1:172.21.15.203:6833/1778614695] conn(0x7f877c06a170 msgr2=0x7f877c06c630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:51:28.344 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:28.341+0000 7f878ffff640 1 --2- 172.21.15.43:0/3904161864 >> [v2:172.21.15.203:6832/1778614695,v1:172.21.15.203:6833/1778614695] conn(0x7f877c06a170 0x7f877c06c630 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f8780000f60 tx=0x7f8780009040 comp rx=0 tx=0).stop 2023-08-09T13:51:28.344 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:28.341+0000 7f878ffff640 1 -- 172.21.15.43:0/3904161864 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f87900c8b80 msgr2=0x7f8790146540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:51:28.344 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:28.341+0000 7f878ffff640 1 --2- 172.21.15.43:0/3904161864 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f87900c8b80 0x7f8790146540 secure :-1 s=READY pgs=1217 cs=0 l=1 rev1=1 crypto rx=0x7f878400c500 tx=0x7f878400c9c0 comp rx=0 tx=0).stop 2023-08-09T13:51:28.345 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:28.341+0000 7f878ffff640 1 -- 172.21.15.43:0/3904161864 shutdown_connections 2023-08-09T13:51:28.345 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:28.341+0000 7f878ffff640 1 --2- 172.21.15.43:0/3904161864 >> [v2:172.21.15.203:6832/1778614695,v1:172.21.15.203:6833/1778614695] conn(0x7f877c06a170 0x7f877c06c630 unknown :-1 s=CLOSED pgs=71 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:51:28.345 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:28.341+0000 7f878ffff640 1 --2- 172.21.15.43:0/3904161864 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f8790157ce0 0x7f87901f5310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:51:28.345 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:28.341+0000 7f878ffff640 1 --2- 172.21.15.43:0/3904161864 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f87900c8b80 0x7f8790146540 unknown :-1 s=CLOSED pgs=1217 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:51:28.345 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:28.341+0000 7f878ffff640 1 --2- 172.21.15.43:0/3904161864 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f879006f2c0 0x7f87901f17f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:51:28.345 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:28.341+0000 7f878ffff640 1 -- 172.21.15.43:0/3904161864 >> 172.21.15.43:0/3904161864 conn(0x7f87900af9f0 msgr2=0x7f87900cb9b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-09T13:51:28.345 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:28.341+0000 7f878ffff640 1 -- 172.21.15.43:0/3904161864 shutdown_connections 2023-08-09T13:51:28.345 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:28.342+0000 7f878ffff640 1 -- 172.21.15.43:0/3904161864 wait complete. 2023-08-09T13:51:28.364 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mgr mgr/snap_schedule/allow_m_granularity True 2023-08-09T13:51:28.566 INFO:tasks.ceph.mgr.x.smithi203.stderr:2023-08-09T13:51:28.563+0000 7f18f9c57040 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2023-08-09T13:51:28.577 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:28.574+0000 7f0e6359e640 1 Processor -- start 2023-08-09T13:51:28.578 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:28.575+0000 7f0e6359e640 1 -- start start 2023-08-09T13:51:28.579 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:28.576+0000 7f0e6359e640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f0e6415c2d0 0x7f0e640b7df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:51:28.580 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:28.576+0000 7f0e6359e640 1 -- --> v1:172.21.15.43:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0e640d16a0 con 0x7f0e640d1b30 2023-08-09T13:51:28.580 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:28.576+0000 7f0e6359e640 1 -- --> v1:172.21.15.203:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0e640d1fa0 con 0x7f0e640cebf0 2023-08-09T13:51:28.580 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:28.576+0000 7f0e6359e640 1 -- --> v2:172.21.15.43:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0e640d0d70 con 0x7f0e6415c2d0 2023-08-09T13:51:28.581 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:28.576+0000 7f0e6259c640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f0e6415c2d0 0x7f0e640b7df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:51:28.581 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:28.576+0000 7f0e6259c640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f0e6415c2d0 0x7f0e640b7df0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:58086/0 (socket says 172.21.15.43:58086) 2023-08-09T13:51:28.581 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:28.576+0000 7f0e6259c640 1 -- 172.21.15.43:0/4005939516 learned_addr learned my addr 172.21.15.43:0/4005939516 (peer_addr_for_me v2:172.21.15.43:0/0) 2023-08-09T13:51:28.582 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:28.576+0000 7f0e61d9b640 1 --1- 172.21.15.43:0/4005939516 >> v1:172.21.15.43:6789/0 conn(0x7f0e640d1b30 0x7f0e640cd020 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.43:6789/0 says I am v1:172.21.15.43:54406/0 (socket says 172.21.15.43:54406) 2023-08-09T13:51:28.582 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:28.577+0000 7f0e6259c640 1 -- 172.21.15.43:0/4005939516 >> v1:172.21.15.203:6789/0 conn(0x7f0e640cebf0 legacy=0x7f0e640b5270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-09T13:51:28.582 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:28.577+0000 7f0e6259c640 1 -- 172.21.15.43:0/4005939516 >> v1:172.21.15.43:6789/0 conn(0x7f0e640d1b30 legacy=0x7f0e640cd020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:51:28.582 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:28.577+0000 7f0e6259c640 1 -- 172.21.15.43:0/4005939516 --> v2:172.21.15.43:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0e640d1050 con 0x7f0e6415c2d0 2023-08-09T13:51:28.583 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:28.577+0000 7f0e6159a640 1 -- 172.21.15.43:0/4005939516 <== mon.0 v1:172.21.15.43:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4246168816 0 0) 0x7f0e640d16a0 con 0x7f0e640d1b30 2023-08-09T13:51:28.584 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:28.578+0000 7f0e6259c640 1 --2- 172.21.15.43:0/4005939516 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f0e6415c2d0 0x7f0e640b7df0 secure :-1 s=READY pgs=1222 cs=0 l=1 rev1=1 crypto rx=0x7f0e54009870 tx=0x7f0e5402b990 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ac83afd5410d90b7 server_cookie=0 in_seq=0 out_seq=0 2023-08-09T13:51:28.585 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:28.578+0000 7f0e6159a640 1 -- 172.21.15.43:0/4005939516 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0e54030850 con 0x7f0e6415c2d0 2023-08-09T13:51:28.585 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:28.578+0000 7f0e6159a640 1 -- 172.21.15.43:0/4005939516 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0e540309b0 con 0x7f0e6415c2d0 2023-08-09T13:51:28.585 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:28.578+0000 7f0e6159a640 1 -- 172.21.15.43:0/4005939516 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0e54030d40 con 0x7f0e6415c2d0 2023-08-09T13:51:28.586 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:28.579+0000 7f0e6359e640 1 -- 172.21.15.43:0/4005939516 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f0e6415c2d0 msgr2=0x7f0e640b7df0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:51:28.586 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:28.579+0000 7f0e6359e640 1 --2- 172.21.15.43:0/4005939516 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f0e6415c2d0 0x7f0e640b7df0 secure :-1 s=READY pgs=1222 cs=0 l=1 rev1=1 crypto rx=0x7f0e54009870 tx=0x7f0e5402b990 comp rx=0 tx=0).stop 2023-08-09T13:51:28.586 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:28.579+0000 7f0e6359e640 1 -- 172.21.15.43:0/4005939516 shutdown_connections 2023-08-09T13:51:28.586 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:28.579+0000 7f0e6359e640 1 --2- 172.21.15.43:0/4005939516 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f0e6415c2d0 0x7f0e640b7df0 unknown :-1 s=CLOSED pgs=1222 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:51:28.587 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:28.579+0000 7f0e6359e640 1 -- 172.21.15.43:0/4005939516 >> 172.21.15.43:0/4005939516 conn(0x7f0e6406a230 msgr2=0x7f0e6406a640 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-09T13:51:28.587 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:28.579+0000 7f0e6359e640 1 -- 172.21.15.43:0/4005939516 shutdown_connections 2023-08-09T13:51:28.587 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:28.579+0000 7f0e6359e640 1 -- 172.21.15.43:0/4005939516 wait complete. 2023-08-09T13:51:28.588 INFO:tasks.ceph.mgr.y.smithi043.stderr:2023-08-09T13:51:28.579+0000 7f1592e42040 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2023-08-09T13:51:28.591 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:28.581+0000 7f0e6359e640 1 Processor -- start 2023-08-09T13:51:28.591 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:28.581+0000 7f0e6359e640 1 -- start start 2023-08-09T13:51:28.591 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:28.582+0000 7f0e6359e640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f0e640cebf0 0x7f0e64153620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:51:28.592 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:28.582+0000 7f0e6359e640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f0e640d1b30 0x7f0e640b63f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:51:28.592 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:28.582+0000 7f0e6359e640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f0e6415c2d0 0x7f0e640b6930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:51:28.592 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:28.582+0000 7f0e6359e640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0e640caa20 con 0x7f0e640cebf0 2023-08-09T13:51:28.592 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:28.582+0000 7f0e6359e640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0e64153b60 con 0x7f0e640d1b30 2023-08-09T13:51:28.593 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:28.582+0000 7f0e6359e640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0e640b2a20 con 0x7f0e6415c2d0 2023-08-09T13:51:28.593 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:28.582+0000 7f0e6259c640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f0e640cebf0 0x7f0e64153620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:51:28.593 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:28.582+0000 7f0e6259c640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f0e640cebf0 0x7f0e64153620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:58090/0 (socket says 172.21.15.43:58090) 2023-08-09T13:51:28.593 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:28.582+0000 7f0e6259c640 1 -- 172.21.15.43:0/2892479302 learned_addr learned my addr 172.21.15.43:0/2892479302 (peer_addr_for_me v2:172.21.15.43:0/0) 2023-08-09T13:51:28.594 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:28.582+0000 7f0e6259c640 1 -- 172.21.15.43:0/2892479302 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f0e6415c2d0 msgr2=0x7f0e640b6930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:51:28.594 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:28.582+0000 7f0e6259c640 1 --2- 172.21.15.43:0/2892479302 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f0e6415c2d0 0x7f0e640b6930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:51:28.594 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:28.582+0000 7f0e6259c640 1 -- 172.21.15.43:0/2892479302 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f0e640d1b30 msgr2=0x7f0e640b63f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:51:28.595 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:28.582+0000 7f0e61d9b640 1 --2- 172.21.15.43:0/2892479302 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f0e640d1b30 0x7f0e640b63f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:51:28.595 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:28.582+0000 7f0e6259c640 1 --2- 172.21.15.43:0/2892479302 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f0e640d1b30 0x7f0e640b63f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:51:28.595 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:28.582+0000 7f0e6259c640 1 -- 172.21.15.43:0/2892479302 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0e540094d0 con 0x7f0e640cebf0 2023-08-09T13:51:28.595 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:28.583+0000 7f0e6259c640 1 --2- 172.21.15.43:0/2892479302 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f0e640cebf0 0x7f0e64153620 secure :-1 s=READY pgs=1223 cs=0 l=1 rev1=1 crypto rx=0x7f0e540090b0 tx=0x7f0e540046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-09T13:51:28.596 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:28.583+0000 7f0e52ffd640 1 -- 172.21.15.43:0/2892479302 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0e5403e070 con 0x7f0e640cebf0 2023-08-09T13:51:28.596 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:28.583+0000 7f0e52ffd640 1 -- 172.21.15.43:0/2892479302 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0e54004030 con 0x7f0e640cebf0 2023-08-09T13:51:28.596 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:28.583+0000 7f0e52ffd640 1 -- 172.21.15.43:0/2892479302 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0e540043e0 con 0x7f0e640cebf0 2023-08-09T13:51:28.597 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:28.583+0000 7f0e6359e640 1 -- 172.21.15.43:0/2892479302 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0e640b2c70 con 0x7f0e640cebf0 2023-08-09T13:51:28.597 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:28.583+0000 7f0e6359e640 1 -- 172.21.15.43:0/2892479302 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0e640b3230 con 0x7f0e640cebf0 2023-08-09T13:51:28.597 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:28.584+0000 7f0e6359e640 1 -- 172.21.15.43:0/2892479302 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0e6404e390 con 0x7f0e640cebf0 2023-08-09T13:51:28.598 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:28.585+0000 7f0e52ffd640 1 -- 172.21.15.43:0/2892479302 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 38) v1 ==== 88460+0+0 (secure 0 0 0) 0x7f0e54042070 con 0x7f0e640cebf0 2023-08-09T13:51:28.599 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:28.586+0000 7f0e52ffd640 1 --2- 172.21.15.43:0/2892479302 >> [v2:172.21.15.203:6832/1778614695,v1:172.21.15.203:6833/1778614695] conn(0x7f0e4406a1c0 0x7f0e4406c680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:51:28.599 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:28.586+0000 7f0e52ffd640 1 -- 172.21.15.43:0/2892479302 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(103..103 src has 1..103) v4 ==== 6426+0+0 (secure 0 0 0) 0x7f0e540aab80 con 0x7f0e640cebf0 2023-08-09T13:51:28.600 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:28.586+0000 7f0e61d9b640 1 -- 172.21.15.43:0/2892479302 >> [v2:172.21.15.203:6832/1778614695,v1:172.21.15.203:6833/1778614695] conn(0x7f0e4406a1c0 msgr2=0x7f0e4406c680 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.203:6832/1778614695 2023-08-09T13:51:28.600 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:28.586+0000 7f0e61d9b640 1 --2- 172.21.15.43:0/2892479302 >> [v2:172.21.15.203:6832/1778614695,v1:172.21.15.203:6833/1778614695] conn(0x7f0e4406a1c0 0x7f0e4406c680 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-08-09T13:51:28.600 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:28.590+0000 7f0e52ffd640 1 -- 172.21.15.43:0/2892479302 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178066 (secure 0 0 0) 0x7f0e54075e40 con 0x7f0e640cebf0 2023-08-09T13:51:28.789 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:28.786+0000 7f0e6359e640 1 -- 172.21.15.43:0/2892479302 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command([{prefix=config set, name=mgr/snap_schedule/allow_m_granularity}] v 0) v1 -- 0x7f0e640cd800 con 0x7f0e640cebf0 2023-08-09T13:51:28.789 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:28.787+0000 7f0e61d9b640 1 -- 172.21.15.43:0/2892479302 >> [v2:172.21.15.203:6832/1778614695,v1:172.21.15.203:6833/1778614695] conn(0x7f0e4406a1c0 msgr2=0x7f0e4406c680 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.203:6832/1778614695 2023-08-09T13:51:28.789 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:28.787+0000 7f0e61d9b640 1 --2- 172.21.15.43:0/2892479302 >> [v2:172.21.15.203:6832/1778614695,v1:172.21.15.203:6833/1778614695] conn(0x7f0e4406a1c0 0x7f0e4406c680 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2023-08-09T13:51:28.793 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:28.791+0000 7f0e52ffd640 1 -- 172.21.15.43:0/2892479302 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mgr/snap_schedule/allow_m_granularity}]=0 v29)=0 v29) v1 ==== 138+0+0 (secure 0 0 0) 0x7f0e54079460 con 0x7f0e640cebf0 2023-08-09T13:51:28.796 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:28.794+0000 7f0e6359e640 1 -- 172.21.15.43:0/2892479302 >> [v2:172.21.15.203:6832/1778614695,v1:172.21.15.203:6833/1778614695] conn(0x7f0e4406a1c0 msgr2=0x7f0e4406c680 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-08-09T13:51:28.797 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:28.794+0000 7f0e6359e640 1 --2- 172.21.15.43:0/2892479302 >> [v2:172.21.15.203:6832/1778614695,v1:172.21.15.203:6833/1778614695] conn(0x7f0e4406a1c0 0x7f0e4406c680 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:51:28.797 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:28.794+0000 7f0e6359e640 1 -- 172.21.15.43:0/2892479302 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f0e640cebf0 msgr2=0x7f0e64153620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:51:28.797 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:28.794+0000 7f0e6359e640 1 --2- 172.21.15.43:0/2892479302 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f0e640cebf0 0x7f0e64153620 secure :-1 s=READY pgs=1223 cs=0 l=1 rev1=1 crypto rx=0x7f0e540090b0 tx=0x7f0e540046c0 comp rx=0 tx=0).stop 2023-08-09T13:51:28.797 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:28.794+0000 7f0e6359e640 1 -- 172.21.15.43:0/2892479302 shutdown_connections 2023-08-09T13:51:28.797 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:28.794+0000 7f0e6359e640 1 --2- 172.21.15.43:0/2892479302 >> [v2:172.21.15.203:6832/1778614695,v1:172.21.15.203:6833/1778614695] conn(0x7f0e4406a1c0 0x7f0e4406c680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:51:28.797 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:28.794+0000 7f0e6359e640 1 --2- 172.21.15.43:0/2892479302 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f0e6415c2d0 0x7f0e640b6930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:51:28.798 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:28.794+0000 7f0e6359e640 1 --2- 172.21.15.43:0/2892479302 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f0e640d1b30 0x7f0e640b63f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:51:28.798 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:28.794+0000 7f0e6359e640 1 --2- 172.21.15.43:0/2892479302 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f0e640cebf0 0x7f0e64153620 unknown :-1 s=CLOSED pgs=1223 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:51:28.798 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:28.794+0000 7f0e6359e640 1 -- 172.21.15.43:0/2892479302 >> 172.21.15.43:0/2892479302 conn(0x7f0e6406a230 msgr2=0x7f0e6415f0f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-09T13:51:28.798 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:28.794+0000 7f0e6359e640 1 -- 172.21.15.43:0/2892479302 shutdown_connections 2023-08-09T13:51:28.798 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:28.794+0000 7f0e6359e640 1 -- 172.21.15.43:0/2892479302 wait complete. 2023-08-09T13:51:28.816 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mgr mgr/snap_schedule/dump_on_update True 2023-08-09T13:51:29.041 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:29.037+0000 7fdb254e5640 1 Processor -- start 2023-08-09T13:51:29.041 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:29.038+0000 7fdb254e5640 1 -- start start 2023-08-09T13:51:29.042 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:29.039+0000 7fdb254e5640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fdb20143c30 0x7fdb200cd980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:51:29.042 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:29.039+0000 7fdb254e5640 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7fdb200b0e20 0x7fdb200b1270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:51:29.042 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:29.039+0000 7fdb254e5640 1 -- --> v1:172.21.15.203:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fdb200b17b0 con 0x7fdb2015a7f0 2023-08-09T13:51:29.042 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:29.039+0000 7fdb254e5640 1 -- --> v2:172.21.15.203:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fdb2015ac30 con 0x7fdb200b0e20 2023-08-09T13:51:29.042 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:29.039+0000 7fdb254e5640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdb2006afd0 con 0x7fdb20143c30 2023-08-09T13:51:29.043 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:29.040+0000 7fdb24ce4640 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7fdb200b0e20 0x7fdb200b1270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:51:29.043 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:29.040+0000 7fdb24ce4640 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7fdb200b0e20 0x7fdb200b1270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.203:3300/0 says I am v2:172.21.15.43:54048/0 (socket says 172.21.15.43:54048) 2023-08-09T13:51:29.044 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:29.040+0000 7fdb24ce4640 1 -- 172.21.15.43:0/3318117222 learned_addr learned my addr 172.21.15.43:0/3318117222 (peer_addr_for_me v2:172.21.15.43:0/0) 2023-08-09T13:51:29.044 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:29.040+0000 7fdb1ffff640 1 --2- 172.21.15.43:0/3318117222 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fdb20143c30 0x7fdb200cd980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:51:29.044 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:29.041+0000 7fdb24ce4640 1 -- 172.21.15.43:0/3318117222 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fdb20143c30 msgr2=0x7fdb200cd980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-09T13:51:29.044 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:29.041+0000 7fdb24ce4640 1 --2- 172.21.15.43:0/3318117222 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fdb20143c30 0x7fdb200cd980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:51:29.044 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:29.041+0000 7fdb24ce4640 1 -- 172.21.15.43:0/3318117222 >> v1:172.21.15.203:6789/0 conn(0x7fdb2015a7f0 legacy=0x7fdb2006a8c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:51:29.044 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:29.041+0000 7fdb24ce4640 1 -- 172.21.15.43:0/3318117222 --> v2:172.21.15.203:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdb200cdf80 con 0x7fdb200b0e20 2023-08-09T13:51:29.045 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:29.042+0000 7fdb24ce4640 1 --2- 172.21.15.43:0/3318117222 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fdb200b0e20 0x7fdb200b1270 secure :-1 s=READY pgs=439 cs=0 l=1 rev1=1 crypto rx=0x7fdb18009e50 tx=0x7fdb1802bf60 comp rx=0 tx=0).ready entity=mon.1 client_cookie=eaf9a3d02b008d49 server_cookie=0 in_seq=0 out_seq=0 2023-08-09T13:51:29.045 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:29.042+0000 7fdb1effd640 1 -- 172.21.15.43:0/3318117222 <== mon.1 v2:172.21.15.203:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdb1802cb00 con 0x7fdb200b0e20 2023-08-09T13:51:29.045 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:29.042+0000 7fdb1effd640 1 -- 172.21.15.43:0/3318117222 <== mon.1 v2:172.21.15.203:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdb1802cc60 con 0x7fdb200b0e20 2023-08-09T13:51:29.045 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:29.042+0000 7fdb1effd640 1 -- 172.21.15.43:0/3318117222 <== mon.1 v2:172.21.15.203:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdb180343f0 con 0x7fdb200b0e20 2023-08-09T13:51:29.046 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:29.043+0000 7fdb254e5640 1 -- 172.21.15.43:0/3318117222 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fdb200b0e20 msgr2=0x7fdb200b1270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:51:29.046 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:29.043+0000 7fdb254e5640 1 --2- 172.21.15.43:0/3318117222 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fdb200b0e20 0x7fdb200b1270 secure :-1 s=READY pgs=439 cs=0 l=1 rev1=1 crypto rx=0x7fdb18009e50 tx=0x7fdb1802bf60 comp rx=0 tx=0).stop 2023-08-09T13:51:29.046 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:29.043+0000 7fdb254e5640 1 -- 172.21.15.43:0/3318117222 shutdown_connections 2023-08-09T13:51:29.047 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:29.043+0000 7fdb254e5640 1 --2- 172.21.15.43:0/3318117222 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fdb200b0e20 0x7fdb200b1270 unknown :-1 s=CLOSED pgs=439 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:51:29.047 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:29.043+0000 7fdb254e5640 1 --2- 172.21.15.43:0/3318117222 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fdb20143c30 0x7fdb200cd980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:51:29.047 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:29.043+0000 7fdb254e5640 1 -- 172.21.15.43:0/3318117222 >> 172.21.15.43:0/3318117222 conn(0x7fdb200b4400 msgr2=0x7fdb200b47e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-09T13:51:29.047 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:29.043+0000 7fdb254e5640 1 -- 172.21.15.43:0/3318117222 shutdown_connections 2023-08-09T13:51:29.047 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:29.043+0000 7fdb254e5640 1 -- 172.21.15.43:0/3318117222 wait complete. 2023-08-09T13:51:29.048 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:29.046+0000 7fdb254e5640 1 Processor -- start 2023-08-09T13:51:29.048 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:29.046+0000 7fdb254e5640 1 -- start start 2023-08-09T13:51:29.049 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:29.047+0000 7fdb254e5640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fdb200b0e20 0x7fdb2013cac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:51:29.050 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:29.047+0000 7fdb254e5640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fdb20143c30 0x7fdb20140640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:51:29.050 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:29.047+0000 7fdb254e5640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fdb2015a7f0 0x7fdb20140b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:51:29.050 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:29.047+0000 7fdb254e5640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdb201f5850 con 0x7fdb20143c30 2023-08-09T13:51:29.050 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:29.047+0000 7fdb254e5640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdb2013d000 con 0x7fdb200b0e20 2023-08-09T13:51:29.050 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:29.047+0000 7fdb254e5640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdb20069dd0 con 0x7fdb2015a7f0 2023-08-09T13:51:29.051 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:29.047+0000 7fdb1f7fe640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fdb20143c30 0x7fdb20140640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:51:29.051 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:29.047+0000 7fdb1f7fe640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fdb20143c30 0x7fdb20140640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:58106/0 (socket says 172.21.15.43:58106) 2023-08-09T13:51:29.052 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:29.047+0000 7fdb1f7fe640 1 -- 172.21.15.43:0/2884892238 learned_addr learned my addr 172.21.15.43:0/2884892238 (peer_addr_for_me v2:172.21.15.43:0/0) 2023-08-09T13:51:29.052 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:29.048+0000 7fdb1ffff640 1 --2- 172.21.15.43:0/2884892238 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fdb200b0e20 0x7fdb2013cac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:51:29.052 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:29.048+0000 7fdb24ce4640 1 --2- 172.21.15.43:0/2884892238 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fdb2015a7f0 0x7fdb20140b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:51:29.052 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:29.048+0000 7fdb1f7fe640 1 -- 172.21.15.43:0/2884892238 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fdb2015a7f0 msgr2=0x7fdb20140b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:51:29.052 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:29.048+0000 7fdb1f7fe640 1 --2- 172.21.15.43:0/2884892238 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fdb2015a7f0 0x7fdb20140b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:51:29.052 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:29.048+0000 7fdb1f7fe640 1 -- 172.21.15.43:0/2884892238 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fdb200b0e20 msgr2=0x7fdb2013cac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:51:29.052 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:29.048+0000 7fdb1f7fe640 1 --2- 172.21.15.43:0/2884892238 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fdb200b0e20 0x7fdb2013cac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:51:29.053 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:29.048+0000 7fdb1f7fe640 1 -- 172.21.15.43:0/2884892238 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdb18009b30 con 0x7fdb20143c30 2023-08-09T13:51:29.053 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:29.048+0000 7fdb1f7fe640 1 --2- 172.21.15.43:0/2884892238 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fdb20143c30 0x7fdb20140640 secure :-1 s=READY pgs=1224 cs=0 l=1 rev1=1 crypto rx=0x7fdb0800c450 tx=0x7fdb0800c910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-09T13:51:29.053 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:29.049+0000 7fdb1cff9640 1 -- 172.21.15.43:0/2884892238 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdb080052d0 con 0x7fdb20143c30 2023-08-09T13:51:29.053 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:29.049+0000 7fdb1cff9640 1 -- 172.21.15.43:0/2884892238 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdb08005430 con 0x7fdb20143c30 2023-08-09T13:51:29.053 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:29.049+0000 7fdb1cff9640 1 -- 172.21.15.43:0/2884892238 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdb08012870 con 0x7fdb20143c30 2023-08-09T13:51:29.053 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:29.049+0000 7fdb254e5640 1 -- 172.21.15.43:0/2884892238 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdb201f4c40 con 0x7fdb20143c30 2023-08-09T13:51:29.053 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:29.049+0000 7fdb254e5640 1 -- 172.21.15.43:0/2884892238 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdb201f5200 con 0x7fdb20143c30 2023-08-09T13:51:29.054 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:29.050+0000 7fdb254e5640 1 -- 172.21.15.43:0/2884892238 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdb2004e390 con 0x7fdb20143c30 2023-08-09T13:51:29.054 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:29.051+0000 7fdb1cff9640 1 -- 172.21.15.43:0/2884892238 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 38) v1 ==== 88460+0+0 (secure 0 0 0) 0x7fdb080129d0 con 0x7fdb20143c30 2023-08-09T13:51:29.055 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:29.052+0000 7fdb1cff9640 1 --2- 172.21.15.43:0/2884892238 >> [v2:172.21.15.203:6832/1778614695,v1:172.21.15.203:6833/1778614695] conn(0x7fdb0006a1f0 0x7fdb0006c6b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:51:29.055 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:29.052+0000 7fdb1cff9640 1 -- 172.21.15.43:0/2884892238 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(103..103 src has 1..103) v4 ==== 6426+0+0 (secure 0 0 0) 0x7fdb080882c0 con 0x7fdb20143c30 2023-08-09T13:51:29.055 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:29.052+0000 7fdb1ffff640 1 -- 172.21.15.43:0/2884892238 >> [v2:172.21.15.203:6832/1778614695,v1:172.21.15.203:6833/1778614695] conn(0x7fdb0006a1f0 msgr2=0x7fdb0006c6b0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.203:6832/1778614695 2023-08-09T13:51:29.055 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:29.052+0000 7fdb1ffff640 1 --2- 172.21.15.43:0/2884892238 >> [v2:172.21.15.203:6832/1778614695,v1:172.21.15.203:6833/1778614695] conn(0x7fdb0006a1f0 0x7fdb0006c6b0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-08-09T13:51:29.059 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:29.056+0000 7fdb1cff9640 1 -- 172.21.15.43:0/2884892238 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178066 (secure 0 0 0) 0x7fdb080537d0 con 0x7fdb20143c30 2023-08-09T13:51:29.256 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:29.253+0000 7fdb1ffff640 1 -- 172.21.15.43:0/2884892238 >> [v2:172.21.15.203:6832/1778614695,v1:172.21.15.203:6833/1778614695] conn(0x7fdb0006a1f0 msgr2=0x7fdb0006c6b0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.203:6832/1778614695 2023-08-09T13:51:29.256 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:29.253+0000 7fdb1ffff640 1 --2- 172.21.15.43:0/2884892238 >> [v2:172.21.15.203:6832/1778614695,v1:172.21.15.203:6833/1778614695] conn(0x7fdb0006a1f0 0x7fdb0006c6b0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2023-08-09T13:51:29.257 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:29.254+0000 7fdb254e5640 1 -- 172.21.15.43:0/2884892238 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command([{prefix=config set, name=mgr/snap_schedule/dump_on_update}] v 0) v1 -- 0x7fdb200708e0 con 0x7fdb20143c30 2023-08-09T13:51:29.265 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:29.263+0000 7fdb1cff9640 1 -- 172.21.15.43:0/2884892238 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mgr/snap_schedule/dump_on_update}]=0 v30)=0 v30) v1 ==== 133+0+0 (secure 0 0 0) 0x7fdb08056df0 con 0x7fdb20143c30 2023-08-09T13:51:29.268 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:29.265+0000 7fdb254e5640 1 -- 172.21.15.43:0/2884892238 >> [v2:172.21.15.203:6832/1778614695,v1:172.21.15.203:6833/1778614695] conn(0x7fdb0006a1f0 msgr2=0x7fdb0006c6b0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-08-09T13:51:29.269 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:29.265+0000 7fdb254e5640 1 --2- 172.21.15.43:0/2884892238 >> [v2:172.21.15.203:6832/1778614695,v1:172.21.15.203:6833/1778614695] conn(0x7fdb0006a1f0 0x7fdb0006c6b0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:51:29.269 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:29.265+0000 7fdb254e5640 1 -- 172.21.15.43:0/2884892238 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fdb20143c30 msgr2=0x7fdb20140640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:51:29.269 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:29.265+0000 7fdb254e5640 1 --2- 172.21.15.43:0/2884892238 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fdb20143c30 0x7fdb20140640 secure :-1 s=READY pgs=1224 cs=0 l=1 rev1=1 crypto rx=0x7fdb0800c450 tx=0x7fdb0800c910 comp rx=0 tx=0).stop 2023-08-09T13:51:29.269 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:29.265+0000 7fdb254e5640 1 -- 172.21.15.43:0/2884892238 shutdown_connections 2023-08-09T13:51:29.269 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:29.265+0000 7fdb254e5640 1 --2- 172.21.15.43:0/2884892238 >> [v2:172.21.15.203:6832/1778614695,v1:172.21.15.203:6833/1778614695] conn(0x7fdb0006a1f0 0x7fdb0006c6b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:51:29.269 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:29.265+0000 7fdb254e5640 1 --2- 172.21.15.43:0/2884892238 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fdb2015a7f0 0x7fdb20140b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:51:29.269 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:29.265+0000 7fdb254e5640 1 --2- 172.21.15.43:0/2884892238 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fdb20143c30 0x7fdb20140640 unknown :-1 s=CLOSED pgs=1224 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:51:29.270 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:29.265+0000 7fdb254e5640 1 --2- 172.21.15.43:0/2884892238 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fdb200b0e20 0x7fdb2013cac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:51:29.270 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:29.266+0000 7fdb254e5640 1 -- 172.21.15.43:0/2884892238 >> 172.21.15.43:0/2884892238 conn(0x7fdb200b4400 msgr2=0x7fdb200b47e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-09T13:51:29.270 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:29.266+0000 7fdb254e5640 1 -- 172.21.15.43:0/2884892238 shutdown_connections 2023-08-09T13:51:29.270 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:29.266+0000 7fdb254e5640 1 -- 172.21.15.43:0/2884892238 wait complete. 2023-08-09T13:51:29.287 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs snap-schedule list --fs cephfs snap_test_dir1 2023-08-09T13:51:29.294 INFO:tasks.ceph.mgr.x.smithi203.stderr:2023-08-09T13:51:29.292+0000 7f18f9c57040 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2023-08-09T13:51:29.368 INFO:tasks.ceph.mgr.y.smithi043.stderr:2023-08-09T13:51:29.366+0000 7f1592e42040 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2023-08-09T13:51:29.415 INFO:tasks.ceph.mgr.x.smithi203.stderr:2023-08-09T13:51:29.412+0000 7f18f9c57040 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2023-08-09T13:51:29.471 INFO:tasks.ceph.mgr.x.smithi203.stderr:2023-08-09T13:51:29.469+0000 7f18f9c57040 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2023-08-09T13:51:29.496 INFO:tasks.ceph.mgr.y.smithi043.stderr:2023-08-09T13:51:29.494+0000 7f1592e42040 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2023-08-09T13:51:29.507 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:29.504+0000 7f6ea759e640 1 Processor -- start 2023-08-09T13:51:29.507 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:29.505+0000 7f6ea759e640 1 -- start start 2023-08-09T13:51:29.509 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:29.506+0000 7f6ea759e640 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7f6ea8145750 0x7f6ea8070d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:51:29.509 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:29.506+0000 7f6ea759e640 1 -- --> v1:172.21.15.43:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6ea8145510 con 0x7f6ea80c5950 2023-08-09T13:51:29.510 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:29.506+0000 7f6ea759e640 1 -- --> v1:172.21.15.203:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6ea806e9a0 con 0x7f6ea80697c0 2023-08-09T13:51:29.510 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:29.506+0000 7f6ea759e640 1 -- --> v2:172.21.15.203:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6ea80cceb0 con 0x7f6ea8145750 2023-08-09T13:51:29.510 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:29.507+0000 7f6ea659c640 1 --1- >> v1:172.21.15.43:6789/0 conn(0x7f6ea80c5950 0x7f6ea806d7a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.43:6789/0 says I am v1:172.21.15.43:54416/0 (socket says 172.21.15.43:54416) 2023-08-09T13:51:29.510 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:29.507+0000 7f6ea659c640 1 -- 172.21.15.43:0/334231666 learned_addr learned my addr 172.21.15.43:0/334231666 (peer_addr_for_me v1:172.21.15.43:0/0) 2023-08-09T13:51:29.510 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:29.507+0000 7f6ea5d9b640 1 --2- 172.21.15.43:0/334231666 >> v2:172.21.15.203:3300/0 conn(0x7f6ea8145750 0x7f6ea8070d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:51:29.512 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:29.507+0000 7f6ea559a640 1 -- 172.21.15.43:0/334231666 <== mon.0 v1:172.21.15.43:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1814215173 0 0) 0x7f6ea8145510 con 0x7f6ea80c5950 2023-08-09T13:51:29.512 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:29.507+0000 7f6ea559a640 1 -- 172.21.15.43:0/334231666 --> v1:172.21.15.43:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f6e90003520 con 0x7f6ea80c5950 2023-08-09T13:51:29.513 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:29.508+0000 7f6ea5d9b640 1 -- 172.21.15.43:0/334231666 >> v1:172.21.15.203:6789/0 conn(0x7f6ea80697c0 legacy=0x7f6ea81665c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:51:29.513 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:29.508+0000 7f6ea5d9b640 1 -- 172.21.15.43:0/334231666 >> v1:172.21.15.43:6789/0 conn(0x7f6ea80c5950 legacy=0x7f6ea806d7a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:51:29.513 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:29.508+0000 7f6ea5d9b640 1 -- 172.21.15.43:0/334231666 --> v2:172.21.15.203:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6ea806ec60 con 0x7f6ea8145750 2023-08-09T13:51:29.513 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:29.509+0000 7f6ea5d9b640 1 --2- 172.21.15.43:0/334231666 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f6ea8145750 0x7f6ea8070d30 secure :-1 s=READY pgs=441 cs=0 l=1 rev1=1 crypto rx=0x7f6e9c0097d0 tx=0x7f6e9c02b960 comp rx=0 tx=0).ready entity=mon.1 client_cookie=836c03ee434194a server_cookie=0 in_seq=0 out_seq=0 2023-08-09T13:51:29.514 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:29.509+0000 7f6ea559a640 1 -- 172.21.15.43:0/334231666 <== mon.1 v2:172.21.15.203:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6e9c030610 con 0x7f6ea8145750 2023-08-09T13:51:29.514 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:29.509+0000 7f6ea559a640 1 -- 172.21.15.43:0/334231666 <== mon.1 v2:172.21.15.203:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6e9c030770 con 0x7f6ea8145750 2023-08-09T13:51:29.514 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:29.509+0000 7f6ea559a640 1 -- 172.21.15.43:0/334231666 <== mon.1 v2:172.21.15.203:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6e9c030ae0 con 0x7f6ea8145750 2023-08-09T13:51:29.516 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:29.510+0000 7f6ea759e640 1 -- 172.21.15.43:0/334231666 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f6ea8145750 msgr2=0x7f6ea8070d30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:51:29.516 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:29.510+0000 7f6ea759e640 1 --2- 172.21.15.43:0/334231666 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f6ea8145750 0x7f6ea8070d30 secure :-1 s=READY pgs=441 cs=0 l=1 rev1=1 crypto rx=0x7f6e9c0097d0 tx=0x7f6e9c02b960 comp rx=0 tx=0).stop 2023-08-09T13:51:29.516 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:29.510+0000 7f6ea759e640 1 -- 172.21.15.43:0/334231666 shutdown_connections 2023-08-09T13:51:29.517 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:29.510+0000 7f6ea759e640 1 --2- 172.21.15.43:0/334231666 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f6ea8145750 0x7f6ea8070d30 unknown :-1 s=CLOSED pgs=441 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:51:29.517 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:29.510+0000 7f6ea759e640 1 -- 172.21.15.43:0/334231666 >> 172.21.15.43:0/334231666 conn(0x7f6ea80cba30 msgr2=0x7f6ea80cbe40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-09T13:51:29.517 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:29.510+0000 7f6ea759e640 1 -- 172.21.15.43:0/334231666 shutdown_connections 2023-08-09T13:51:29.517 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:29.511+0000 7f6ea759e640 1 -- 172.21.15.43:0/334231666 wait complete. 2023-08-09T13:51:29.518 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:29.513+0000 7f6ea759e640 1 Processor -- start 2023-08-09T13:51:29.518 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:29.513+0000 7f6ea759e640 1 -- start start 2023-08-09T13:51:29.521 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:29.514+0000 7f6ea759e640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f6ea80697c0 0x7f6ea815bde0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:51:29.521 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:29.514+0000 7f6ea759e640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f6ea80c5950 0x7f6ea815b5e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:51:29.521 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:29.514+0000 7f6ea759e640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f6ea8145750 0x7f6ea806ac50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:51:29.522 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:29.514+0000 7f6ea759e640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6ea815ca50 con 0x7f6ea80697c0 2023-08-09T13:51:29.522 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:29.514+0000 7f6ea759e640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6ea8140c10 con 0x7f6ea80c5950 2023-08-09T13:51:29.522 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:29.514+0000 7f6ea759e640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6ea815c320 con 0x7f6ea8145750 2023-08-09T13:51:29.522 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:29.515+0000 7f6ea659c640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f6ea80697c0 0x7f6ea815bde0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:51:29.523 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:29.515+0000 7f6ea659c640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f6ea80697c0 0x7f6ea815bde0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:58118/0 (socket says 172.21.15.43:58118) 2023-08-09T13:51:29.523 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:29.515+0000 7f6ea659c640 1 -- 172.21.15.43:0/4075249924 learned_addr learned my addr 172.21.15.43:0/4075249924 (peer_addr_for_me v2:172.21.15.43:0/0) 2023-08-09T13:51:29.523 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:29.515+0000 7f6ea5d9b640 1 --2- 172.21.15.43:0/4075249924 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f6ea80c5950 0x7f6ea815b5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:51:29.523 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:29.515+0000 7f6ea659c640 1 -- 172.21.15.43:0/4075249924 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f6ea8145750 msgr2=0x7f6ea806ac50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:51:29.523 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:29.515+0000 7f6ea659c640 1 --2- 172.21.15.43:0/4075249924 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f6ea8145750 0x7f6ea806ac50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:51:29.523 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:29.515+0000 7f6ea659c640 1 -- 172.21.15.43:0/4075249924 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f6ea80c5950 msgr2=0x7f6ea815b5e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:51:29.523 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:29.515+0000 7f6ea659c640 1 --2- 172.21.15.43:0/4075249924 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f6ea80c5950 0x7f6ea815b5e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:51:29.524 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:29.515+0000 7f6ea659c640 1 -- 172.21.15.43:0/4075249924 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6e9c009400 con 0x7f6ea80697c0 2023-08-09T13:51:29.524 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:29.515+0000 7f6ea659c640 1 --2- 172.21.15.43:0/4075249924 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f6ea80697c0 0x7f6ea815bde0 secure :-1 s=READY pgs=1226 cs=0 l=1 rev1=1 crypto rx=0x7f6e9800c500 tx=0x7f6e9800c9c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-09T13:51:29.524 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:29.516+0000 7f6e96ffd640 1 -- 172.21.15.43:0/4075249924 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6e98005ce0 con 0x7f6ea80697c0 2023-08-09T13:51:29.524 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:29.516+0000 7f6e96ffd640 1 -- 172.21.15.43:0/4075249924 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6e98005e40 con 0x7f6ea80697c0 2023-08-09T13:51:29.525 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:29.516+0000 7f6e96ffd640 1 -- 172.21.15.43:0/4075249924 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6e98007330 con 0x7f6ea80697c0 2023-08-09T13:51:29.525 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:29.516+0000 7f6ea759e640 1 -- 172.21.15.43:0/4075249924 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6ea8140e60 con 0x7f6ea80697c0 2023-08-09T13:51:29.525 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:29.516+0000 7f6ea759e640 1 -- 172.21.15.43:0/4075249924 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6ea8141420 con 0x7f6ea80697c0 2023-08-09T13:51:29.525 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:29.517+0000 7f6ea759e640 1 -- 172.21.15.43:0/4075249924 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6ea80739f0 con 0x7f6ea80697c0 2023-08-09T13:51:29.526 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:29.518+0000 7f6e96ffd640 1 -- 172.21.15.43:0/4075249924 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 38) v1 ==== 88460+0+0 (secure 0 0 0) 0x7f6e98007550 con 0x7f6ea80697c0 2023-08-09T13:51:29.527 INFO:tasks.ceph.mgr.x.smithi203.stderr:2023-08-09T13:51:29.519+0000 7f18f9c57040 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2023-08-09T13:51:29.527 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:29.519+0000 7f6e96ffd640 1 --2- 172.21.15.43:0/4075249924 >> [v2:172.21.15.203:6832/1778614695,v1:172.21.15.203:6833/1778614695] conn(0x7f6e8806a1c0 0x7f6e8806c680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:51:29.528 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:29.519+0000 7f6e96ffd640 1 -- 172.21.15.43:0/4075249924 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(103..103 src has 1..103) v4 ==== 6426+0+0 (secure 0 0 0) 0x7f6e98088060 con 0x7f6ea80697c0 2023-08-09T13:51:29.528 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:29.519+0000 7f6ea5d9b640 1 -- 172.21.15.43:0/4075249924 >> [v2:172.21.15.203:6832/1778614695,v1:172.21.15.203:6833/1778614695] conn(0x7f6e8806a1c0 msgr2=0x7f6e8806c680 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.203:6832/1778614695 2023-08-09T13:51:29.528 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:29.519+0000 7f6ea5d9b640 1 --2- 172.21.15.43:0/4075249924 >> [v2:172.21.15.203:6832/1778614695,v1:172.21.15.203:6833/1778614695] conn(0x7f6e8806a1c0 0x7f6e8806c680 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-08-09T13:51:29.529 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:29.523+0000 7f6e96ffd640 1 -- 172.21.15.43:0/4075249924 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178066 (secure 0 0 0) 0x7f6e980ba7b0 con 0x7f6ea80697c0 2023-08-09T13:51:29.559 INFO:tasks.ceph.mgr.y.smithi043.stderr:2023-08-09T13:51:29.556+0000 7f1592e42040 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2023-08-09T13:51:29.612 INFO:tasks.ceph.mgr.y.smithi043.stderr:2023-08-09T13:51:29.609+0000 7f1592e42040 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2023-08-09T13:51:29.723 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:29.720+0000 7f6ea5d9b640 1 -- 172.21.15.43:0/4075249924 >> [v2:172.21.15.203:6832/1778614695,v1:172.21.15.203:6833/1778614695] conn(0x7f6e8806a1c0 msgr2=0x7f6e8806c680 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.203:6832/1778614695 2023-08-09T13:51:29.723 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:29.720+0000 7f6ea5d9b640 1 --2- 172.21.15.43:0/4075249924 >> [v2:172.21.15.203:6832/1778614695,v1:172.21.15.203:6833/1778614695] conn(0x7f6e8806a1c0 0x7f6e8806c680 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2023-08-09T13:51:29.737 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:29.735+0000 7f6ea759e640 1 -- 172.21.15.43:0/4075249924 --> [v2:172.21.15.203:6832/1778614695,v1:172.21.15.203:6833/1778614695] -- mgr_command(tid 0: {"prefix": "fs snap-schedule list", "fs": "cephfs", "path": "snap_test_dir1", "target": ["mon-mgr", ""]}) v1 -- 0x7f6ea80b3660 con 0x7f6e8806a1c0 2023-08-09T13:51:30.123 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:30.121+0000 7f6ea5d9b640 1 -- 172.21.15.43:0/4075249924 >> [v2:172.21.15.203:6832/1778614695,v1:172.21.15.203:6833/1778614695] conn(0x7f6e8806a1c0 msgr2=0x7f6e8806c680 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.203:6832/1778614695 2023-08-09T13:51:30.124 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:30.121+0000 7f6ea5d9b640 1 --2- 172.21.15.43:0/4075249924 >> [v2:172.21.15.203:6832/1778614695,v1:172.21.15.203:6833/1778614695] conn(0x7f6e8806a1c0 0x7f6e8806c680 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2023-08-09T13:51:30.227 INFO:tasks.ceph.mgr.x.smithi203.stderr:2023-08-09T13:51:30.224+0000 7f18f9c57040 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2023-08-09T13:51:30.289 INFO:tasks.ceph.mgr.x.smithi203.stderr:2023-08-09T13:51:30.287+0000 7f18f9c57040 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2023-08-09T13:51:30.338 INFO:tasks.ceph.mgr.x.smithi203.stderr:2023-08-09T13:51:30.336+0000 7f18f9c57040 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2023-08-09T13:51:30.346 INFO:tasks.ceph.mgr.y.smithi043.stderr:2023-08-09T13:51:30.344+0000 7f1592e42040 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2023-08-09T13:51:30.411 INFO:tasks.ceph.mgr.y.smithi043.stderr:2023-08-09T13:51:30.408+0000 7f1592e42040 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2023-08-09T13:51:30.455 INFO:tasks.ceph.mgr.x.smithi203.stderr:2023-08-09T13:51:30.452+0000 7f18f9c57040 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2023-08-09T13:51:30.463 INFO:tasks.ceph.mgr.y.smithi043.stderr:2023-08-09T13:51:30.461+0000 7f1592e42040 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2023-08-09T13:51:30.587 INFO:tasks.ceph.mgr.y.smithi043.stderr:2023-08-09T13:51:30.585+0000 7f1592e42040 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2023-08-09T13:51:30.611 INFO:tasks.ceph.mgr.x.smithi203.stderr:2023-08-09T13:51:30.608+0000 7f18f9c57040 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2023-08-09T13:51:30.718 INFO:tasks.ceph.mgr.x.smithi203.stderr:2023-08-09T13:51:30.716+0000 7f18f9c57040 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2023-08-09T13:51:30.754 INFO:tasks.ceph.mgr.y.smithi043.stderr:2023-08-09T13:51:30.751+0000 7f1592e42040 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2023-08-09T13:51:30.768 INFO:tasks.ceph.mgr.x.smithi203.stderr:2023-08-09T13:51:30.765+0000 7f18f9c57040 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2023-08-09T13:51:30.867 INFO:tasks.ceph.mgr.y.smithi043.stderr:2023-08-09T13:51:30.864+0000 7f1592e42040 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2023-08-09T13:51:30.919 INFO:tasks.ceph.mgr.y.smithi043.stderr:2023-08-09T13:51:30.916+0000 7f1592e42040 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2023-08-09T13:51:30.925 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:30.922+0000 7f6ea5d9b640 1 -- 172.21.15.43:0/4075249924 >> [v2:172.21.15.203:6832/1778614695,v1:172.21.15.203:6833/1778614695] conn(0x7f6e8806a1c0 msgr2=0x7f6e8806c680 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.203:6832/1778614695 2023-08-09T13:51:30.925 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:30.922+0000 7f6ea5d9b640 1 --2- 172.21.15.43:0/4075249924 >> [v2:172.21.15.203:6832/1778614695,v1:172.21.15.203:6833/1778614695] conn(0x7f6e8806a1c0 0x7f6e8806c680 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2023-08-09T13:51:30.926 INFO:tasks.ceph.mgr.x.smithi203.stderr:2023-08-09T13:51:30.923+0000 7f18f9c57040 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2023-08-09T13:51:31.088 INFO:tasks.ceph.mgr.y.smithi043.stderr:2023-08-09T13:51:31.086+0000 7f1592e42040 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2023-08-09T13:51:31.315 INFO:tasks.ceph.mgr.x.smithi203.stderr:2023-08-09T13:51:31.312+0000 7f18f9c57040 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2023-08-09T13:51:31.498 INFO:tasks.ceph.mgr.y.smithi043.stderr:2023-08-09T13:51:31.495+0000 7f1592e42040 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2023-08-09T13:51:31.537 INFO:tasks.ceph.mgr.x.smithi203.stderr:2023-08-09T13:51:31.534+0000 7f18f9c57040 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2023-08-09T13:51:31.594 INFO:tasks.ceph.mgr.x.smithi203.stderr:2023-08-09T13:51:31.591+0000 7f18f9c57040 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2023-08-09T13:51:31.644 INFO:tasks.ceph.mgr.x.smithi203.stderr:2023-08-09T13:51:31.641+0000 7f18f9c57040 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2023-08-09T13:51:31.696 INFO:tasks.ceph.mgr.x.smithi203.stderr:2023-08-09T13:51:31.693+0000 7f18f9c57040 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2023-08-09T13:51:31.732 INFO:tasks.ceph.mgr.y.smithi043.stderr:2023-08-09T13:51:31.727+0000 7f1592e42040 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2023-08-09T13:51:31.790 INFO:tasks.ceph.mgr.y.smithi043.stderr:2023-08-09T13:51:31.788+0000 7f1592e42040 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2023-08-09T13:51:31.843 INFO:tasks.ceph.mgr.y.smithi043.stderr:2023-08-09T13:51:31.840+0000 7f1592e42040 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2023-08-09T13:51:31.898 INFO:tasks.ceph.mgr.y.smithi043.stderr:2023-08-09T13:51:31.896+0000 7f1592e42040 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2023-08-09T13:51:31.989 INFO:tasks.ceph.mgr.x.smithi203.stderr:2023-08-09T13:51:31.986+0000 7f18f9c57040 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2023-08-09T13:51:32.047 INFO:tasks.ceph.mgr.x.smithi203.stderr:2023-08-09T13:51:32.044+0000 7f18f9c57040 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2023-08-09T13:51:32.098 INFO:tasks.ceph.mgr.x.smithi203.stderr:2023-08-09T13:51:32.096+0000 7f18f9c57040 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2023-08-09T13:51:32.210 INFO:tasks.ceph.mgr.y.smithi043.stderr:2023-08-09T13:51:32.207+0000 7f1592e42040 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2023-08-09T13:51:32.263 INFO:tasks.ceph.mgr.x.smithi203.stderr:2023-08-09T13:51:32.261+0000 7f18f9c57040 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2023-08-09T13:51:32.276 INFO:tasks.ceph.mgr.y.smithi043.stderr:2023-08-09T13:51:32.273+0000 7f1592e42040 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2023-08-09T13:51:32.314 INFO:tasks.ceph.mgr.x.smithi203.stderr:2023-08-09T13:51:32.311+0000 7f18f9c57040 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2023-08-09T13:51:32.331 INFO:tasks.ceph.mgr.y.smithi043.stderr:2023-08-09T13:51:32.328+0000 7f1592e42040 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2023-08-09T13:51:32.367 INFO:tasks.ceph.mgr.x.smithi203.stderr:2023-08-09T13:51:32.364+0000 7f18f9c57040 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2023-08-09T13:51:32.417 INFO:tasks.ceph.mgr.x.smithi203.stderr:2023-08-09T13:51:32.415+0000 7f18f9c57040 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2023-08-09T13:51:32.440 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:32.437+0000 7f6e96ffd640 1 -- 172.21.15.43:0/4075249924 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mgrmap(e 39) v1 ==== 88069+0+0 (secure 0 0 0) 0x7f6e980538c0 con 0x7f6ea80697c0 2023-08-09T13:51:32.441 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:32.438+0000 7f6e96ffd640 1 -- 172.21.15.43:0/4075249924 >> [v2:172.21.15.203:6832/1778614695,v1:172.21.15.203:6833/1778614695] conn(0x7f6e8806a1c0 msgr2=0x7f6e8806c680 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-08-09T13:51:32.441 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:32.438+0000 7f6e96ffd640 1 --2- 172.21.15.43:0/4075249924 >> [v2:172.21.15.203:6832/1778614695,v1:172.21.15.203:6833/1778614695] conn(0x7f6e8806a1c0 0x7f6e8806c680 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:51:32.513 INFO:tasks.ceph.mgr.y.smithi043.stderr:2023-08-09T13:51:32.510+0000 7f1592e42040 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2023-08-09T13:51:32.574 INFO:tasks.ceph.mgr.y.smithi043.stderr:2023-08-09T13:51:32.571+0000 7f1592e42040 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2023-08-09T13:51:32.620 INFO:tasks.ceph.mgr.x.smithi203.stderr:2023-08-09T13:51:32.618+0000 7f187689b640 -1 client.0 error registering admin socket command: (17) File exists 2023-08-09T13:51:32.620 INFO:tasks.ceph.mgr.x.smithi203.stderr:2023-08-09T13:51:32.618+0000 7f187689b640 -1 client.0 error registering admin socket command: (17) File exists 2023-08-09T13:51:32.621 INFO:tasks.ceph.mgr.x.smithi203.stderr:2023-08-09T13:51:32.618+0000 7f187689b640 -1 client.0 error registering admin socket command: (17) File exists 2023-08-09T13:51:32.621 INFO:tasks.ceph.mgr.x.smithi203.stderr:2023-08-09T13:51:32.618+0000 7f187689b640 -1 client.0 error registering admin socket command: (17) File exists 2023-08-09T13:51:32.621 INFO:tasks.ceph.mgr.x.smithi203.stderr:2023-08-09T13:51:32.618+0000 7f187689b640 -1 client.0 error registering admin socket command: (17) File exists 2023-08-09T13:51:32.621 INFO:tasks.ceph.mgr.x.smithi203.stderr:2023-08-09T13:51:32.619+0000 7f18798a1640 -1 client.0 error registering admin socket command: (17) File exists 2023-08-09T13:51:32.622 INFO:tasks.ceph.mgr.x.smithi203.stderr:2023-08-09T13:51:32.619+0000 7f18798a1640 -1 client.0 error registering admin socket command: (17) File exists 2023-08-09T13:51:32.622 INFO:tasks.ceph.mgr.x.smithi203.stderr:2023-08-09T13:51:32.619+0000 7f18798a1640 -1 client.0 error registering admin socket command: (17) File exists 2023-08-09T13:51:32.622 INFO:tasks.ceph.mgr.x.smithi203.stderr:2023-08-09T13:51:32.619+0000 7f18798a1640 -1 client.0 error registering admin socket command: (17) File exists 2023-08-09T13:51:32.622 INFO:tasks.ceph.mgr.x.smithi203.stderr:2023-08-09T13:51:32.619+0000 7f18798a1640 -1 client.0 error registering admin socket command: (17) File exists 2023-08-09T13:51:32.634 INFO:tasks.ceph.mgr.y.smithi043.stderr:2023-08-09T13:51:32.632+0000 7f1592e42040 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2023-08-09T13:51:32.687 INFO:tasks.ceph.mgr.y.smithi043.stderr:2023-08-09T13:51:32.685+0000 7f1592e42040 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2023-08-09T13:51:33.451 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:33.448+0000 7f6e96ffd640 1 -- 172.21.15.43:0/4075249924 <== mon.0 v2:172.21.15.43:3300/0 8 ==== mgrmap(e 40) v1 ==== 88196+0+0 (secure 0 0 0) 0x7f6e980753d0 con 0x7f6ea80697c0 2023-08-09T13:51:33.452 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:33.450+0000 7f6e96ffd640 1 --2- 172.21.15.43:0/4075249924 >> [v2:172.21.15.203:6832/2675832148,v1:172.21.15.203:6833/2675832148] conn(0x7f6e88072cb0 0x7f6e880750a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:51:33.453 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:33.450+0000 7f6e96ffd640 1 -- 172.21.15.43:0/4075249924 --> [v2:172.21.15.203:6832/2675832148,v1:172.21.15.203:6833/2675832148] -- mgr_command(tid 0: {"prefix": "fs snap-schedule list", "fs": "cephfs", "path": "snap_test_dir1", "target": ["mon-mgr", ""]}) v1 -- 0x7f6ea80b3660 con 0x7f6e88072cb0 2023-08-09T13:51:33.467 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:33.465+0000 7f6ea5d9b640 1 --2- 172.21.15.43:0/4075249924 >> [v2:172.21.15.203:6832/2675832148,v1:172.21.15.203:6833/2675832148] conn(0x7f6e88072cb0 0x7f6e880750a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:51:33.473 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:33.470+0000 7f6ea5d9b640 1 --2- 172.21.15.43:0/4075249924 >> [v2:172.21.15.203:6832/2675832148,v1:172.21.15.203:6833/2675832148] conn(0x7f6e88072cb0 0x7f6e880750a0 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7f6e9c036000 tx=0x7f6e9c03d040 comp rx=0 tx=0).ready entity=mgr.6547 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-09T13:51:33.491 INFO:tasks.ceph.mgr.x.smithi203.stderr:2023-08-09T13:51:33.489+0000 7f18808af640 -1 mgr.server reply reply (2) No such file or directory SnapSchedule for snap_test_dir1 not found 2023-08-09T13:51:33.492 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:33.489+0000 7f6e96ffd640 1 -- 172.21.15.43:0/4075249924 <== mgr.6547 v2:172.21.15.203:6832/2675832148 1 ==== mgr_command_reply(tid 0: -2 SnapSchedule for snap_test_dir1 not found) v1 ==== 49+0+0 (secure 0 0 0) 0x7f6ea80b3660 con 0x7f6e88072cb0 2023-08-09T13:51:33.492 INFO:teuthology.orchestra.run.smithi043.stderr:Error ENOENT: SnapSchedule for snap_test_dir1 not found 2023-08-09T13:51:33.505 DEBUG:teuthology.orchestra.run:got remote process result: 2 2023-08-09T13:51:33.506 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph mgr module disable snap_schedule 2023-08-09T13:51:33.741 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:33.739+0000 7fd1419b5640 1 Processor -- start 2023-08-09T13:51:33.741 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:33.739+0000 7fd1419b5640 1 -- start start 2023-08-09T13:51:33.743 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:33.741+0000 7fd1419b5640 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7fd13c0ccf20 0x7fd13c0cd550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:51:33.743 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:33.741+0000 7fd1419b5640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fd13c0cda90 0x7fd13c15ca10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:51:33.743 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:33.741+0000 7fd1419b5640 1 -- --> v1:172.21.15.203:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd13c0cdee0 con 0x7fd13c15c630 2023-08-09T13:51:33.744 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:33.741+0000 7fd1419b5640 1 -- --> v2:172.21.15.203:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd13c0ac600 con 0x7fd13c0ccf20 2023-08-09T13:51:33.744 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:33.741+0000 7fd1419b5640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd13c0cbc70 con 0x7fd13c0cda90 2023-08-09T13:51:33.744 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:33.741+0000 7fd13a7fc640 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7fd13c0ccf20 0x7fd13c0cd550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:51:33.748 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:33.741+0000 7fd13b7fe640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fd13c0cda90 0x7fd13c15ca10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:51:33.748 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:33.741+0000 7fd13a7fc640 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7fd13c0ccf20 0x7fd13c0cd550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.203:3300/0 says I am v2:172.21.15.43:54090/0 (socket says 172.21.15.43:54090) 2023-08-09T13:51:33.748 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:33.741+0000 7fd13a7fc640 1 -- 172.21.15.43:0/3708754272 learned_addr learned my addr 172.21.15.43:0/3708754272 (peer_addr_for_me v2:172.21.15.43:0/0) 2023-08-09T13:51:33.748 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:33.741+0000 7fd13b7fe640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fd13c0cda90 0x7fd13c15ca10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.203:3301/0 says I am v2:172.21.15.43:60582/0 (socket says 172.21.15.43:60582) 2023-08-09T13:51:33.748 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:33.742+0000 7fd13a7fc640 1 -- 172.21.15.43:0/3708754272 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fd13c0cda90 msgr2=0x7fd13c15ca10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-09T13:51:33.748 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:33.742+0000 7fd13a7fc640 1 --2- 172.21.15.43:0/3708754272 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fd13c0cda90 0x7fd13c15ca10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:51:33.748 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:33.742+0000 7fd13a7fc640 1 -- 172.21.15.43:0/3708754272 >> v1:172.21.15.203:6789/0 conn(0x7fd13c15c630 legacy=0x7fd13c0abef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:51:33.749 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:33.742+0000 7fd13a7fc640 1 -- 172.21.15.43:0/3708754272 --> v2:172.21.15.203:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd13c0cca90 con 0x7fd13c0ccf20 2023-08-09T13:51:33.750 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:33.744+0000 7fd13a7fc640 1 --2- 172.21.15.43:0/3708754272 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fd13c0ccf20 0x7fd13c0cd550 secure :-1 s=READY pgs=450 cs=0 l=1 rev1=1 crypto rx=0x7fd130009870 tx=0x7fd13002b980 comp rx=0 tx=0).ready entity=mon.1 client_cookie=375846682b8c8fc7 server_cookie=0 in_seq=0 out_seq=0 2023-08-09T13:51:33.750 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:33.744+0000 7fd139ffb640 1 -- 172.21.15.43:0/3708754272 <== mon.1 v2:172.21.15.203:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd130030840 con 0x7fd13c0ccf20 2023-08-09T13:51:33.750 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:33.744+0000 7fd139ffb640 1 -- 172.21.15.43:0/3708754272 <== mon.1 v2:172.21.15.203:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd1300309a0 con 0x7fd13c0ccf20 2023-08-09T13:51:33.750 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:33.744+0000 7fd139ffb640 1 -- 172.21.15.43:0/3708754272 <== mon.1 v2:172.21.15.203:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd130030d30 con 0x7fd13c0ccf20 2023-08-09T13:51:33.750 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:33.745+0000 7fd1419b5640 1 -- 172.21.15.43:0/3708754272 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fd13c0ccf20 msgr2=0x7fd13c0cd550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:51:33.750 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:33.745+0000 7fd1419b5640 1 --2- 172.21.15.43:0/3708754272 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fd13c0ccf20 0x7fd13c0cd550 secure :-1 s=READY pgs=450 cs=0 l=1 rev1=1 crypto rx=0x7fd130009870 tx=0x7fd13002b980 comp rx=0 tx=0).stop 2023-08-09T13:51:33.750 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:33.745+0000 7fd1419b5640 1 -- 172.21.15.43:0/3708754272 shutdown_connections 2023-08-09T13:51:33.751 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:33.745+0000 7fd1419b5640 1 --2- 172.21.15.43:0/3708754272 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fd13c0cda90 0x7fd13c15ca10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:51:33.751 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:33.745+0000 7fd1419b5640 1 --2- 172.21.15.43:0/3708754272 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fd13c0ccf20 0x7fd13c0cd550 unknown :-1 s=CLOSED pgs=450 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:51:33.751 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:33.745+0000 7fd1419b5640 1 -- 172.21.15.43:0/3708754272 >> 172.21.15.43:0/3708754272 conn(0x7fd13c0ce0a0 msgr2=0x7fd13c0ce480 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-09T13:51:33.751 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:33.745+0000 7fd1419b5640 1 -- 172.21.15.43:0/3708754272 shutdown_connections 2023-08-09T13:51:33.751 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:33.745+0000 7fd1419b5640 1 -- 172.21.15.43:0/3708754272 wait complete. 2023-08-09T13:51:33.751 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:33.747+0000 7fd1419b5640 1 Processor -- start 2023-08-09T13:51:33.752 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:33.748+0000 7fd1419b5640 1 -- start start 2023-08-09T13:51:33.753 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:33.749+0000 7fd1419b5640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fd13c0ccf20 0x7fd13c1f52a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:51:33.753 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:33.749+0000 7fd1419b5640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fd13c0cda90 0x7fd13c0b0d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:51:33.753 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:33.749+0000 7fd1419b5640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fd13c15c630 0x7fd13c0aeb50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:51:33.753 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:33.749+0000 7fd1419b5640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd13c0c5210 con 0x7fd13c0ccf20 2023-08-09T13:51:33.753 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:33.749+0000 7fd1419b5640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd13c1f5880 con 0x7fd13c0cda90 2023-08-09T13:51:33.753 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:33.749+0000 7fd1419b5640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd13c1f5b60 con 0x7fd13c15c630 2023-08-09T13:51:33.753 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:33.749+0000 7fd13affd640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fd13c0ccf20 0x7fd13c1f52a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:51:33.754 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:33.749+0000 7fd13affd640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fd13c0ccf20 0x7fd13c1f52a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:58134/0 (socket says 172.21.15.43:58134) 2023-08-09T13:51:33.754 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:33.749+0000 7fd13affd640 1 -- 172.21.15.43:0/291454579 learned_addr learned my addr 172.21.15.43:0/291454579 (peer_addr_for_me v2:172.21.15.43:0/0) 2023-08-09T13:51:33.754 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:33.750+0000 7fd13a7fc640 1 --2- 172.21.15.43:0/291454579 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fd13c0cda90 0x7fd13c0b0d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:51:33.754 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:33.750+0000 7fd13b7fe640 1 --2- 172.21.15.43:0/291454579 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fd13c15c630 0x7fd13c0aeb50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:51:33.754 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:33.750+0000 7fd13affd640 1 -- 172.21.15.43:0/291454579 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fd13c15c630 msgr2=0x7fd13c0aeb50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:51:33.754 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:33.750+0000 7fd13affd640 1 --2- 172.21.15.43:0/291454579 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fd13c15c630 0x7fd13c0aeb50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:51:33.754 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:33.750+0000 7fd13affd640 1 -- 172.21.15.43:0/291454579 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fd13c0cda90 msgr2=0x7fd13c0b0d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:51:33.754 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:33.750+0000 7fd13affd640 1 --2- 172.21.15.43:0/291454579 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fd13c0cda90 0x7fd13c0b0d80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:51:33.755 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:33.750+0000 7fd13affd640 1 -- 172.21.15.43:0/291454579 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd1300094d0 con 0x7fd13c0ccf20 2023-08-09T13:51:33.755 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:33.750+0000 7fd13affd640 1 --2- 172.21.15.43:0/291454579 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fd13c0ccf20 0x7fd13c1f52a0 secure :-1 s=READY pgs=1227 cs=0 l=1 rev1=1 crypto rx=0x7fd12c00c400 tx=0x7fd12c00c8c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-09T13:51:33.755 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:33.750+0000 7fd123fff640 1 -- 172.21.15.43:0/291454579 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd12c0052d0 con 0x7fd13c0ccf20 2023-08-09T13:51:33.756 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:33.751+0000 7fd123fff640 1 -- 172.21.15.43:0/291454579 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd12c005430 con 0x7fd13c0ccf20 2023-08-09T13:51:33.756 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:33.751+0000 7fd123fff640 1 -- 172.21.15.43:0/291454579 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd12c003830 con 0x7fd13c0ccf20 2023-08-09T13:51:33.756 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:33.751+0000 7fd1419b5640 1 -- 172.21.15.43:0/291454579 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd13c1f5e40 con 0x7fd13c0ccf20 2023-08-09T13:51:33.756 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:33.751+0000 7fd1419b5640 1 -- 172.21.15.43:0/291454579 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd13c1f6120 con 0x7fd13c0ccf20 2023-08-09T13:51:33.756 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:33.752+0000 7fd1419b5640 1 -- 172.21.15.43:0/291454579 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd13c0739f0 con 0x7fd13c0ccf20 2023-08-09T13:51:33.756 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:33.753+0000 7fd123fff640 1 -- 172.21.15.43:0/291454579 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 40) v1 ==== 88196+0+0 (secure 0 0 0) 0x7fd12c003990 con 0x7fd13c0ccf20 2023-08-09T13:51:33.757 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:33.754+0000 7fd123fff640 1 --2- 172.21.15.43:0/291454579 >> [v2:172.21.15.203:6832/2675832148,v1:172.21.15.203:6833/2675832148] conn(0x7fd128069ef0 0x7fd12806c3b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:51:33.757 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:33.754+0000 7fd123fff640 1 -- 172.21.15.43:0/291454579 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(104..104 src has 1..104) v4 ==== 6770+0+0 (secure 0 0 0) 0x7fd12c0879d0 con 0x7fd13c0ccf20 2023-08-09T13:51:33.757 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:33.755+0000 7fd13a7fc640 1 --2- 172.21.15.43:0/291454579 >> [v2:172.21.15.203:6832/2675832148,v1:172.21.15.203:6833/2675832148] conn(0x7fd128069ef0 0x7fd12806c3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:51:33.758 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:33.756+0000 7fd13a7fc640 1 --2- 172.21.15.43:0/291454579 >> [v2:172.21.15.203:6832/2675832148,v1:172.21.15.203:6833/2675832148] conn(0x7fd128069ef0 0x7fd12806c3b0 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7fd130036000 tx=0x7fd130008030 comp rx=0 tx=0).ready entity=mgr.6547 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-09T13:51:33.761 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:33.758+0000 7fd123fff640 1 -- 172.21.15.43:0/291454579 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178066 (secure 0 0 0) 0x7fd12c053060 con 0x7fd13c0ccf20 2023-08-09T13:51:33.969 DEBUG:teuthology.orchestra.run.smithi043:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-08-09T13:51:33.977 DEBUG:teuthology.orchestra.run.smithi203:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-08-09T13:51:34.000 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:33.998+0000 7fd1419b5640 1 -- 172.21.15.43:0/291454579 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "mgr module disable", "module": "snap_schedule"} v 0) v1 -- 0x7fd13c0cdee0 con 0x7fd13c0ccf20 2023-08-09T13:51:34.458 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:34.456+0000 7fd123fff640 1 -- 172.21.15.43:0/291454579 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mgrmap(e 41) v1 ==== 88444+0+0 (secure 0 0 0) 0x7fd12c053430 con 0x7fd13c0ccf20 2023-08-09T13:51:34.460 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:34.458+0000 7fd123fff640 1 -- 172.21.15.43:0/291454579 <== mon.0 v2:172.21.15.43:3300/0 8 ==== mon_command_ack([{"prefix": "mgr module disable", "module": "snap_schedule"}]=0 v41) v1 ==== 93+0+0 (secure 0 0 0) 0x7fd12c056af0 con 0x7fd13c0ccf20 2023-08-09T13:51:34.463 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:34.461+0000 7fd1419b5640 1 -- 172.21.15.43:0/291454579 >> [v2:172.21.15.203:6832/2675832148,v1:172.21.15.203:6833/2675832148] conn(0x7fd128069ef0 msgr2=0x7fd12806c3b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:51:34.464 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:34.461+0000 7fd1419b5640 1 --2- 172.21.15.43:0/291454579 >> [v2:172.21.15.203:6832/2675832148,v1:172.21.15.203:6833/2675832148] conn(0x7fd128069ef0 0x7fd12806c3b0 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7fd130036000 tx=0x7fd130008030 comp rx=0 tx=0).stop 2023-08-09T13:51:34.464 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:34.461+0000 7fd1419b5640 1 -- 172.21.15.43:0/291454579 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fd13c0ccf20 msgr2=0x7fd13c1f52a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:51:34.464 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:34.461+0000 7fd1419b5640 1 --2- 172.21.15.43:0/291454579 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fd13c0ccf20 0x7fd13c1f52a0 secure :-1 s=READY pgs=1227 cs=0 l=1 rev1=1 crypto rx=0x7fd12c00c400 tx=0x7fd12c00c8c0 comp rx=0 tx=0).stop 2023-08-09T13:51:34.464 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:34.461+0000 7fd1419b5640 1 -- 172.21.15.43:0/291454579 shutdown_connections 2023-08-09T13:51:34.464 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:34.461+0000 7fd1419b5640 1 --2- 172.21.15.43:0/291454579 >> [v2:172.21.15.203:6832/2675832148,v1:172.21.15.203:6833/2675832148] conn(0x7fd128069ef0 0x7fd12806c3b0 unknown :-1 s=CLOSED pgs=23 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:51:34.464 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:34.461+0000 7fd1419b5640 1 --2- 172.21.15.43:0/291454579 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fd13c15c630 0x7fd13c0aeb50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:51:34.465 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:34.461+0000 7fd1419b5640 1 --2- 172.21.15.43:0/291454579 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fd13c0cda90 0x7fd13c0b0d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:51:34.465 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:34.461+0000 7fd1419b5640 1 --2- 172.21.15.43:0/291454579 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fd13c0ccf20 0x7fd13c1f52a0 unknown :-1 s=CLOSED pgs=1227 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:51:34.465 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:34.461+0000 7fd1419b5640 1 -- 172.21.15.43:0/291454579 >> 172.21.15.43:0/291454579 conn(0x7fd13c0ce0a0 msgr2=0x7fd13c06c8a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-09T13:51:34.465 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:34.461+0000 7fd1419b5640 1 -- 172.21.15.43:0/291454579 shutdown_connections 2023-08-09T13:51:34.466 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:34.461+0000 7fd1419b5640 1 -- 172.21.15.43:0/291454579 wait complete. 2023-08-09T13:51:34.490 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2023-08-09T13:51:34.490 DEBUG:teuthology.orchestra.run.smithi043:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2023-08-09T13:51:34.593 DEBUG:teuthology.orchestra.run.smithi203:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2023-08-09T13:51:34.674 INFO:tasks.ceph.mgr.x.smithi203.stderr:2023-08-09T13:51:34.672+0000 7f67a7a3b040 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2023-08-09T13:51:34.691 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2023-08-09T13:51:34.692 DEBUG:teuthology.orchestra.run.smithi043:> set -ex 2023-08-09T13:51:34.692 DEBUG:teuthology.orchestra.run.smithi043:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-09T13:51:34.725 INFO:tasks.ceph.mgr.y.smithi043.stderr:2023-08-09T13:51:34.722+0000 7fc4e782c040 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2023-08-09T13:51:34.747 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-08-09T13:51:35.003 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:35.000+0000 7f4a2fdb5640 1 Processor -- start 2023-08-09T13:51:35.003 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:35.001+0000 7f4a2fdb5640 1 -- start start 2023-08-09T13:51:35.005 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:35.002+0000 7f4a2fdb5640 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7f4a281549c0 0x7f4a280712b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:51:35.005 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:35.002+0000 7f4a2fdb5640 1 -- --> v1:172.21.15.43:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4a28070e20 con 0x7f4a280697c0 2023-08-09T13:51:35.005 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:35.002+0000 7f4a2fdb5640 1 -- --> v1:172.21.15.203:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4a2815b5c0 con 0x7f4a2815af30 2023-08-09T13:51:35.005 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:35.002+0000 7f4a2fdb5640 1 -- --> v2:172.21.15.203:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4a28154da0 con 0x7f4a281549c0 2023-08-09T13:51:35.005 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:35.003+0000 7f4a2d350640 1 --1- >> v1:172.21.15.43:6789/0 conn(0x7f4a280697c0 0x7f4a2815a820 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.43:6789/0 says I am v1:172.21.15.43:60746/0 (socket says 172.21.15.43:60746) 2023-08-09T13:51:35.006 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:35.003+0000 7f4a2d350640 1 -- 172.21.15.43:0/1777307117 learned_addr learned my addr 172.21.15.43:0/1777307117 (peer_addr_for_me v1:172.21.15.43:0/0) 2023-08-09T13:51:35.007 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:35.003+0000 7f4a2db51640 1 --2- 172.21.15.43:0/1777307117 >> v2:172.21.15.203:3300/0 conn(0x7f4a281549c0 0x7f4a280712b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:51:35.007 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:35.003+0000 7f4a2cb4f640 1 -- 172.21.15.43:0/1777307117 <== mon.0 v1:172.21.15.43:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1185779403 0 0) 0x7f4a28070e20 con 0x7f4a280697c0 2023-08-09T13:51:35.007 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:35.003+0000 7f4a2cb4f640 1 -- 172.21.15.43:0/1777307117 --> v1:172.21.15.43:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f4a14003520 con 0x7f4a280697c0 2023-08-09T13:51:35.007 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:35.004+0000 7f4a2cb4f640 1 -- 172.21.15.43:0/1777307117 <== mon.0 v1:172.21.15.43:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1812616439 0 0) 0x7f4a14003520 con 0x7f4a280697c0 2023-08-09T13:51:35.008 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:35.004+0000 7f4a2cb4f640 1 -- 172.21.15.43:0/1777307117 >> v2:172.21.15.203:3300/0 conn(0x7f4a281549c0 msgr2=0x7f4a280712b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-09T13:51:35.008 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:35.004+0000 7f4a2cb4f640 1 --2- 172.21.15.43:0/1777307117 >> v2:172.21.15.203:3300/0 conn(0x7f4a281549c0 0x7f4a280712b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:51:35.008 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:35.004+0000 7f4a2cb4f640 1 -- 172.21.15.43:0/1777307117 >> v1:172.21.15.203:6789/0 conn(0x7f4a2815af30 legacy=0x7f4a280c97e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:51:35.008 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:35.004+0000 7f4a2cb4f640 1 -- 172.21.15.43:0/1777307117 --> v1:172.21.15.43:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4a280707d0 con 0x7f4a280697c0 2023-08-09T13:51:35.008 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:35.004+0000 7f4a2cb4f640 1 -- 172.21.15.43:0/1777307117 <== mon.0 v1:172.21.15.43:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1285150740 0 0) 0x7f4a18001670 con 0x7f4a280697c0 2023-08-09T13:51:35.008 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:35.004+0000 7f4a2cb4f640 1 -- 172.21.15.43:0/1777307117 >> v1:172.21.15.43:6789/0 conn(0x7f4a280697c0 legacy=0x7f4a2815a820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:51:35.008 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:35.004+0000 7f4a2cb4f640 1 --2- 172.21.15.43:0/1777307117 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f4a140039e0 0x7f4a14003e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:51:35.009 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:35.004+0000 7f4a2cb4f640 1 --2- 172.21.15.43:0/1777307117 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f4a140047c0 0x7f4a14004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:51:35.009 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:35.004+0000 7f4a2cb4f640 1 --2- 172.21.15.43:0/1777307117 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f4a140051f0 0x7f4a1400b980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:51:35.009 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:35.004+0000 7f4a2cb4f640 1 -- 172.21.15.43:0/1777307117 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4a28154da0 con 0x7f4a140047c0 2023-08-09T13:51:35.009 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:35.004+0000 7f4a2cb4f640 1 -- 172.21.15.43:0/1777307117 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4a1400c150 con 0x7f4a140051f0 2023-08-09T13:51:35.009 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:35.004+0000 7f4a2cb4f640 1 -- 172.21.15.43:0/1777307117 --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4a1400c310 con 0x7f4a140039e0 2023-08-09T13:51:35.009 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:35.004+0000 7f4a2d350640 1 --2- 172.21.15.43:0/1777307117 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f4a140047c0 0x7f4a14004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:51:35.009 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:35.004+0000 7f4a2e352640 1 --2- 172.21.15.43:0/1777307117 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f4a140051f0 0x7f4a1400b980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:51:35.010 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:35.005+0000 7f4a2d350640 1 -- 172.21.15.43:0/1777307117 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f4a140039e0 msgr2=0x7f4a14003e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-09T13:51:35.010 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:35.005+0000 7f4a2d350640 1 --2- 172.21.15.43:0/1777307117 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f4a140039e0 0x7f4a14003e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:51:35.010 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:35.005+0000 7f4a2d350640 1 -- 172.21.15.43:0/1777307117 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f4a140051f0 msgr2=0x7f4a1400b980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-09T13:51:35.010 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:35.005+0000 7f4a2d350640 1 --2- 172.21.15.43:0/1777307117 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f4a140051f0 0x7f4a1400b980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:51:35.010 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:35.005+0000 7f4a2d350640 1 -- 172.21.15.43:0/1777307117 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4a14003520 con 0x7f4a140047c0 2023-08-09T13:51:35.010 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:35.005+0000 7f4a2e352640 1 -- 172.21.15.43:0/1777307117 reap_dead start 2023-08-09T13:51:35.011 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:35.006+0000 7f4a2d350640 1 --2- 172.21.15.43:0/1777307117 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f4a140047c0 0x7f4a14004c40 secure :-1 s=READY pgs=1232 cs=0 l=1 rev1=1 crypto rx=0x7f4a1800c240 tx=0x7f4a140057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6e098bff3167add3 server_cookie=0 in_seq=0 out_seq=0 2023-08-09T13:51:35.011 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:35.006+0000 7f4a2cb4f640 1 -- 172.21.15.43:0/1777307117 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4a18005ad0 con 0x7f4a140047c0 2023-08-09T13:51:35.012 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:35.006+0000 7f4a2cb4f640 1 -- 172.21.15.43:0/1777307117 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4a18005c30 con 0x7f4a140047c0 2023-08-09T13:51:35.012 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:35.006+0000 7f4a2cb4f640 1 -- 172.21.15.43:0/1777307117 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4a1802caf0 con 0x7f4a140047c0 2023-08-09T13:51:35.012 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:35.007+0000 7f4a2fdb5640 1 -- 172.21.15.43:0/1777307117 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f4a140047c0 msgr2=0x7f4a14004c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:51:35.012 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:35.007+0000 7f4a2fdb5640 1 --2- 172.21.15.43:0/1777307117 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f4a140047c0 0x7f4a14004c40 secure :-1 s=READY pgs=1232 cs=0 l=1 rev1=1 crypto rx=0x7f4a1800c240 tx=0x7f4a140057f0 comp rx=0 tx=0).stop 2023-08-09T13:51:35.012 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:35.007+0000 7f4a2fdb5640 1 -- 172.21.15.43:0/1777307117 shutdown_connections 2023-08-09T13:51:35.012 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:35.007+0000 7f4a2fdb5640 1 --2- 172.21.15.43:0/1777307117 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f4a140047c0 0x7f4a14004c40 unknown :-1 s=CLOSED pgs=1232 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:51:35.012 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:35.007+0000 7f4a2fdb5640 1 -- 172.21.15.43:0/1777307117 >> 172.21.15.43:0/1777307117 conn(0x7f4a280cf190 msgr2=0x7f4a280cf570 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-09T13:51:35.013 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:35.007+0000 7f4a2fdb5640 1 -- 172.21.15.43:0/1777307117 shutdown_connections 2023-08-09T13:51:35.013 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:35.007+0000 7f4a2fdb5640 1 -- 172.21.15.43:0/1777307117 wait complete. 2023-08-09T13:51:35.013 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:35.009+0000 7f4a2fdb5640 1 Processor -- start 2023-08-09T13:51:35.013 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:35.009+0000 7f4a2fdb5640 1 -- start start 2023-08-09T13:51:35.014 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:35.010+0000 7f4a2fdb5640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f4a140047c0 0x7f4a280befb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:51:35.015 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:35.010+0000 7f4a2fdb5640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f4a280cd890 0x7f4a280cdd10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:51:35.015 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:35.010+0000 7f4a2fdb5640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f4a280c7990 0x7f4a280c7e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:51:35.015 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:35.010+0000 7f4a2fdb5640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4a280c6e10 con 0x7f4a280cd890 2023-08-09T13:51:35.015 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:35.010+0000 7f4a2fdb5640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4a280c6260 con 0x7f4a140047c0 2023-08-09T13:51:35.015 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:35.010+0000 7f4a2fdb5640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4a280ce250 con 0x7f4a280c7990 2023-08-09T13:51:35.015 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:35.010+0000 7f4a2d350640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f4a280cd890 0x7f4a280cdd10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:51:35.015 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:35.010+0000 7f4a2d350640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f4a280cd890 0x7f4a280cdd10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:41254/0 (socket says 172.21.15.43:41254) 2023-08-09T13:51:35.015 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:35.010+0000 7f4a2d350640 1 -- 172.21.15.43:0/3351091531 learned_addr learned my addr 172.21.15.43:0/3351091531 (peer_addr_for_me v2:172.21.15.43:0/0) 2023-08-09T13:51:35.016 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:35.011+0000 7f4a2e352640 1 --2- 172.21.15.43:0/3351091531 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f4a280c7990 0x7f4a280c7e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:51:35.016 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:35.011+0000 7f4a2db51640 1 --2- 172.21.15.43:0/3351091531 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f4a140047c0 0x7f4a280befb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:51:35.016 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:35.011+0000 7f4a2d350640 1 -- 172.21.15.43:0/3351091531 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f4a280c7990 msgr2=0x7f4a280c7e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:51:35.016 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:35.011+0000 7f4a2d350640 1 --2- 172.21.15.43:0/3351091531 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f4a280c7990 0x7f4a280c7e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:51:35.016 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:35.011+0000 7f4a2d350640 1 -- 172.21.15.43:0/3351091531 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f4a140047c0 msgr2=0x7f4a280befb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:51:35.016 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:35.011+0000 7f4a2d350640 1 --2- 172.21.15.43:0/3351091531 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f4a140047c0 0x7f4a280befb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:51:35.016 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:35.011+0000 7f4a2d350640 1 -- 172.21.15.43:0/3351091531 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4a280c8350 con 0x7f4a280cd890 2023-08-09T13:51:35.017 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:35.011+0000 7f4a2e352640 1 --2- 172.21.15.43:0/3351091531 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f4a280c7990 0x7f4a280c7e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-08-09T13:51:35.017 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:35.011+0000 7f4a2d350640 1 --2- 172.21.15.43:0/3351091531 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f4a280cd890 0x7f4a280cdd10 secure :-1 s=READY pgs=1233 cs=0 l=1 rev1=1 crypto rx=0x7f4a18001930 tx=0x7f4a1802e070 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-09T13:51:35.017 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:35.011+0000 7f4a1e7fc640 1 -- 172.21.15.43:0/3351091531 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4a1803c040 con 0x7f4a280cd890 2023-08-09T13:51:35.017 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:35.011+0000 7f4a1e7fc640 1 -- 172.21.15.43:0/3351091531 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4a1802ccc0 con 0x7f4a280cd890 2023-08-09T13:51:35.017 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:35.011+0000 7f4a1e7fc640 1 -- 172.21.15.43:0/3351091531 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4a180072b0 con 0x7f4a280cd890 2023-08-09T13:51:35.017 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:35.011+0000 7f4a2fdb5640 1 -- 172.21.15.43:0/3351091531 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4a280c6470 con 0x7f4a280cd890 2023-08-09T13:51:35.017 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:35.012+0000 7f4a2fdb5640 1 -- 172.21.15.43:0/3351091531 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4a280c6a30 con 0x7f4a280cd890 2023-08-09T13:51:35.018 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:35.013+0000 7f4a2fdb5640 1 -- 172.21.15.43:0/3351091531 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4a280c8d50 con 0x7f4a280cd890 2023-08-09T13:51:35.018 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:35.014+0000 7f4a1e7fc640 1 -- 172.21.15.43:0/3351091531 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 41) v1 ==== 88444+0+0 (secure 0 0 0) 0x7f4a18032070 con 0x7f4a280cd890 2023-08-09T13:51:35.018 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:35.015+0000 7f4a1e7fc640 1 --2- 172.21.15.43:0/3351091531 >> [v2:172.21.15.203:6832/2675832148,v1:172.21.15.203:6833/2675832148] conn(0x7f4a0c06a130 0x7f4a0c06c5f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:51:35.019 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:35.015+0000 7f4a1e7fc640 1 -- 172.21.15.43:0/3351091531 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(104..104 src has 1..104) v4 ==== 6770+0+0 (secure 0 0 0) 0x7f4a180a9ed0 con 0x7f4a280cd890 2023-08-09T13:51:35.019 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:35.015+0000 7f4a2db51640 1 -- 172.21.15.43:0/3351091531 >> [v2:172.21.15.203:6832/2675832148,v1:172.21.15.203:6833/2675832148] conn(0x7f4a0c06a130 msgr2=0x7f4a0c06c5f0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.203:6832/2675832148 2023-08-09T13:51:35.019 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:35.015+0000 7f4a2db51640 1 --2- 172.21.15.43:0/3351091531 >> [v2:172.21.15.203:6832/2675832148,v1:172.21.15.203:6833/2675832148] conn(0x7f4a0c06a130 0x7f4a0c06c5f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-08-09T13:51:35.021 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:35.019+0000 7f4a1e7fc640 1 -- 172.21.15.43:0/3351091531 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178066 (secure 0 0 0) 0x7f4a180751d0 con 0x7f4a280cd890 2023-08-09T13:51:35.217 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:35.214+0000 7f4a2fdb5640 1 -- 172.21.15.43:0/3351091531 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f4a280c6f50 con 0x7f4a280cd890 2023-08-09T13:51:35.219 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:35.216+0000 7f4a2db51640 1 -- 172.21.15.43:0/3351091531 >> [v2:172.21.15.203:6832/2675832148,v1:172.21.15.203:6833/2675832148] conn(0x7f4a0c06a130 msgr2=0x7f4a0c06c5f0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.203:6832/2675832148 2023-08-09T13:51:35.219 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:35.216+0000 7f4a2db51640 1 --2- 172.21.15.43:0/3351091531 >> [v2:172.21.15.203:6832/2675832148,v1:172.21.15.203:6833/2675832148] conn(0x7f4a0c06a130 0x7f4a0c06c5f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2023-08-09T13:51:35.220 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:35.218+0000 7f4a1e7fc640 1 -- 172.21.15.43:0/3351091531 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v104) v1 ==== 74+0+19066 (secure 0 0 0) 0x7f4a280c6f50 con 0x7f4a280cd890 2023-08-09T13:51:35.220 INFO:teuthology.orchestra.run.smithi043.stdout: 2023-08-09T13:51:35.221 INFO:teuthology.orchestra.run.smithi043.stdout:{"epoch":104,"fsid":"daac87ee-1337-4250-a955-aea4e408a39b","created":"2023-08-09T13:33:41.701198+0000","modified":"2023-08-09T13:51:32.420447+0000","last_up_change":"2023-08-09T13:33:53.783453+0000","last_in_change":"2023-08-09T13:33:45.148904+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":13,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":"rbd","create_time":"2023-08-09T13:33:54.365137+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":".mgr","create_time":"2023-08-09T13:33:54.831803+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":12,"pool_name":"cephfs_metadata","create_time":"2023-08-09T13:51:14.142563+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"102","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":13,"pool_name":"cephfs_data","create_time":"2023-08-09T13:51:14.712668+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"103","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"cbd49ecd-e896-4fbd-864e-5c37a829b165","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":101,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6800","nonce":502667335},{"type":"v1","addr":"172.21.15.43:6801","nonce":502667335}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6802","nonce":502667335},{"type":"v1","addr":"172.21.15.43:6803","nonce":502667335}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6806","nonce":502667335},{"type":"v1","addr":"172.21.15.43:6807","nonce":502667335}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6804","nonce":502667335},{"type":"v1","addr":"172.21.15.43:6805","nonce":502667335}]},"public_addr":"172.21.15.43:6801/502667335","cluster_addr":"172.21.15.43:6803/502667335","heartbeat_back_addr":"172.21.15.43:6807/502667335","heartbeat_front_addr":"172.21.15.43:6805/502667335","state":["exists","up"]},{"osd":1,"uuid":"75c7e0fe-1ec7-4fe8-a2f2-3661a65b8500","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":101,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6808","nonce":3289134610},{"type":"v1","addr":"172.21.15.43:6809","nonce":3289134610}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6810","nonce":3289134610},{"type":"v1","addr":"172.21.15.43:6812","nonce":3289134610}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6821","nonce":3289134610},{"type":"v1","addr":"172.21.15.43:6824","nonce":3289134610}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6815","nonce":3289134610},{"type":"v1","addr":"172.21.15.43:6818","nonce":3289134610}]},"public_addr":"172.21.15.43:6809/3289134610","cluster_addr":"172.21.15.43:6812/3289134610","heartbeat_back_addr":"172.21.15.43:6824/3289134610","heartbeat_front_addr":"172.21.15.43:6818/3289134610","state":["exists","up"]},{"osd":2,"uuid":"bb5c41e9-7f5f-4125-9b01-3bb8bfc2ea83","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":101,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6811","nonce":4079709736},{"type":"v1","addr":"172.21.15.43:6814","nonce":4079709736}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6817","nonce":4079709736},{"type":"v1","addr":"172.21.15.43:6820","nonce":4079709736}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6829","nonce":4079709736},{"type":"v1","addr":"172.21.15.43:6831","nonce":4079709736}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6823","nonce":4079709736},{"type":"v1","addr":"172.21.15.43:6826","nonce":4079709736}]},"public_addr":"172.21.15.43:6814/4079709736","cluster_addr":"172.21.15.43:6820/4079709736","heartbeat_back_addr":"172.21.15.43:6831/4079709736","heartbeat_front_addr":"172.21.15.43:6826/4079709736","state":["exists","up"]},{"osd":3,"uuid":"bfe4c8b4-694a-465c-87e6-2b2aab852613","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":101,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6813","nonce":1709538759},{"type":"v1","addr":"172.21.15.43:6816","nonce":1709538759}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6819","nonce":1709538759},{"type":"v1","addr":"172.21.15.43:6822","nonce":1709538759}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6828","nonce":1709538759},{"type":"v1","addr":"172.21.15.43:6830","nonce":1709538759}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6825","nonce":1709538759},{"type":"v1","addr":"172.21.15.43:6827","nonce":1709538759}]},"public_addr":"172.21.15.43:6816/1709538759","cluster_addr":"172.21.15.43:6822/1709538759","heartbeat_back_addr":"172.21.15.43:6830/1709538759","heartbeat_front_addr":"172.21.15.43:6827/1709538759","state":["exists","up"]},{"osd":4,"uuid":"fa2e8a77-aba1-4756-853e-0d174a5bdeba","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":101,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6800","nonce":4196866481},{"type":"v1","addr":"172.21.15.203:6801","nonce":4196866481}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6802","nonce":4196866481},{"type":"v1","addr":"172.21.15.203:6803","nonce":4196866481}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6806","nonce":4196866481},{"type":"v1","addr":"172.21.15.203:6807","nonce":4196866481}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6804","nonce":4196866481},{"type":"v1","addr":"172.21.15.203:6805","nonce":4196866481}]},"public_addr":"172.21.15.203:6801/4196866481","cluster_addr":"172.21.15.203:6803/4196866481","heartbeat_back_addr":"172.21.15.203:6807/4196866481","heartbeat_front_addr":"172.21.15.203:6805/4196866481","state":["exists","up"]},{"osd":5,"uuid":"81811e4b-4d24-4f51-a82c-f1d700a2a00d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":101,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6816","nonce":3059290081},{"type":"v1","addr":"172.21.15.203:6817","nonce":3059290081}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6818","nonce":3059290081},{"type":"v1","addr":"172.21.15.203:6819","nonce":3059290081}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6822","nonce":3059290081},{"type":"v1","addr":"172.21.15.203:6823","nonce":3059290081}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6820","nonce":3059290081},{"type":"v1","addr":"172.21.15.203:6821","nonce":3059290081}]},"public_addr":"172.21.15.203:6817/3059290081","cluster_addr":"172.21.15.203:6819/3059290081","heartbeat_back_addr":"172.21.15.203:6823/3059290081","heartbeat_front_addr":"172.21.15.203:6821/3059290081","state":["exists","up"]},{"osd":6,"uuid":"307bac4d-6a5b-443d-af75-56d695d7674f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":101,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6824","nonce":512155569},{"type":"v1","addr":"172.21.15.203:6825","nonce":512155569}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6826","nonce":512155569},{"type":"v1","addr":"172.21.15.203:6827","nonce":512155569}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6830","nonce":512155569},{"type":"v1","addr":"172.21.15.203:6831","nonce":512155569}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6828","nonce":512155569},{"type":"v1","addr":"172.21.15.203:6829","nonce":512155569}]},"public_addr":"172.21.15.203:6825/512155569","cluster_addr":"172.21.15.203:6827/512155569","heartbeat_back_addr":"172.21.15.203:6831/512155569","heartbeat_front_addr":"172.21.15.203:6829/512155569","state":["exists","up"]},{"osd":7,"uuid":"a95dc7ca-9f1b-4295-af70-68a22a2d7d5e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":101,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6808","nonce":3331450438},{"type":"v1","addr":"172.21.15.203:6809","nonce":3331450438}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6810","nonce":3331450438},{"type":"v1","addr":"172.21.15.203:6811","nonce":3331450438}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6814","nonce":3331450438},{"type":"v1","addr":"172.21.15.203:6815","nonce":3331450438}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6812","nonce":3331450438},{"type":"v1","addr":"172.21.15.203:6813","nonce":3331450438}]},"public_addr":"172.21.15.203:6809/3331450438","cluster_addr":"172.21.15.203:6811/3331450438","heartbeat_back_addr":"172.21.15.203:6815/3331450438","heartbeat_front_addr":"172.21.15.203:6813/3331450438","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-09T13:33:50.547652+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-09T13:33:50.589469+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-09T13:33:50.561724+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-09T13:33:50.599650+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-09T13:33:50.411855+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-09T13:33:50.219681+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-09T13:33:50.504664+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-09T13:33:50.441248+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.203:0/1343041460":"2023-08-10T13:51:32.420350+0000","172.21.15.203:6833/1778614695":"2023-08-10T13:51:32.420350+0000","172.21.15.203:0/3761332698":"2023-08-10T13:51:32.420350+0000","172.21.15.203:6832/1778614695":"2023-08-10T13:51:32.420350+0000","172.21.15.203:0/2054133158":"2023-08-10T13:51:32.420350+0000","172.21.15.203:0/1496080019":"2023-08-10T13:51:32.420350+0000","172.21.15.203:0/1894709586":"2023-08-10T13:51:32.420350+0000","172.21.15.203:0/270998287":"2023-08-10T13:51:32.420350+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":7,"snaps":[{"begin":104,"length":1},{"begin":221,"length":1}]}],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-08-09T13:51:35.223 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:35.220+0000 7f4a2fdb5640 1 -- 172.21.15.43:0/3351091531 >> [v2:172.21.15.203:6832/2675832148,v1:172.21.15.203:6833/2675832148] conn(0x7f4a0c06a130 msgr2=0x7f4a0c06c5f0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-08-09T13:51:35.223 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:35.220+0000 7f4a2fdb5640 1 --2- 172.21.15.43:0/3351091531 >> [v2:172.21.15.203:6832/2675832148,v1:172.21.15.203:6833/2675832148] conn(0x7f4a0c06a130 0x7f4a0c06c5f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:51:35.223 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:35.220+0000 7f4a2fdb5640 1 -- 172.21.15.43:0/3351091531 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f4a280cd890 msgr2=0x7f4a280cdd10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:51:35.223 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:35.220+0000 7f4a2fdb5640 1 --2- 172.21.15.43:0/3351091531 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f4a280cd890 0x7f4a280cdd10 secure :-1 s=READY pgs=1233 cs=0 l=1 rev1=1 crypto rx=0x7f4a18001930 tx=0x7f4a1802e070 comp rx=0 tx=0).stop 2023-08-09T13:51:35.223 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:35.220+0000 7f4a2fdb5640 1 -- 172.21.15.43:0/3351091531 shutdown_connections 2023-08-09T13:51:35.224 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:35.220+0000 7f4a2fdb5640 1 --2- 172.21.15.43:0/3351091531 >> [v2:172.21.15.203:6832/2675832148,v1:172.21.15.203:6833/2675832148] conn(0x7f4a0c06a130 0x7f4a0c06c5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:51:35.224 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:35.220+0000 7f4a2fdb5640 1 --2- 172.21.15.43:0/3351091531 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f4a280c7990 0x7f4a280c7e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:51:35.224 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:35.220+0000 7f4a2fdb5640 1 --2- 172.21.15.43:0/3351091531 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f4a280cd890 0x7f4a280cdd10 unknown :-1 s=CLOSED pgs=1233 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:51:35.224 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:35.220+0000 7f4a2fdb5640 1 --2- 172.21.15.43:0/3351091531 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f4a140047c0 0x7f4a280befb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:51:35.224 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:35.220+0000 7f4a2fdb5640 1 -- 172.21.15.43:0/3351091531 >> 172.21.15.43:0/3351091531 conn(0x7f4a280cf190 msgr2=0x7f4a2815d550 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-09T13:51:35.224 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:35.220+0000 7f4a2fdb5640 1 -- 172.21.15.43:0/3351091531 shutdown_connections 2023-08-09T13:51:35.224 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:35.221+0000 7f4a2fdb5640 1 -- 172.21.15.43:0/3351091531 wait complete. 2023-08-09T13:51:35.243 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': 'rbd', 'create_time': '2023-08-09T13:33:54.365137+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}, 'read_balance': {'score_acting': 2, 'score_stable': 2, 'optimal_score': 1, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': '.mgr', 'create_time': '2023-08-09T13:33:54.831803+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 12, 'pool_name': 'cephfs_metadata', 'create_time': '2023-08-09T13:51:14.142563+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '102', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 2, 'score_stable': 2, 'optimal_score': 1, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 13, 'pool_name': 'cephfs_data', 'create_time': '2023-08-09T13:51:14.712668+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '103', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.3799999952316284, 'score_stable': 1.3799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.3799999952316284, 'raw_score_stable': 1.3799999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-08-09T13:51:35.244 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get rbd pg_num 2023-08-09T13:51:35.376 INFO:tasks.ceph.mgr.x.smithi203.stderr:2023-08-09T13:51:35.374+0000 7f67a7a3b040 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2023-08-09T13:51:35.450 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:35.448+0000 7ff8ddf2f640 1 Processor -- start 2023-08-09T13:51:35.451 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:35.448+0000 7ff8ddf2f640 1 -- start start 2023-08-09T13:51:35.452 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:35.450+0000 7ff8ddf2f640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7ff8d80cee90 0x7ff8d80cdd90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:51:35.452 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:35.450+0000 7ff8ddf2f640 1 -- --> v1:172.21.15.43:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff8d80cb240 con 0x7ff8d80b0230 2023-08-09T13:51:35.452 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:35.450+0000 7ff8ddf2f640 1 -- --> v1:172.21.15.203:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff8d80cf270 con 0x7ff8d80ccbb0 2023-08-09T13:51:35.453 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:35.450+0000 7ff8ddf2f640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff8d80b1440 con 0x7ff8d80cee90 2023-08-09T13:51:35.453 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:35.450+0000 7ff8d7fff640 1 --1- >> v1:172.21.15.43:6789/0 conn(0x7ff8d80b0230 0x7ff8d80cb6d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.43:6789/0 says I am v1:172.21.15.43:60756/0 (socket says 172.21.15.43:60756) 2023-08-09T13:51:35.454 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:35.450+0000 7ff8d7fff640 1 -- 172.21.15.43:0/3504827532 learned_addr learned my addr 172.21.15.43:0/3504827532 (peer_addr_for_me v1:172.21.15.43:0/0) 2023-08-09T13:51:35.454 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:35.450+0000 7ff8d77fe640 1 --2- 172.21.15.43:0/3504827532 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7ff8d80cee90 0x7ff8d80cdd90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:51:35.454 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:35.451+0000 7ff8d67fc640 1 -- 172.21.15.43:0/3504827532 <== mon.0 v1:172.21.15.43:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1534540991 0 0) 0x7ff8d80cb240 con 0x7ff8d80b0230 2023-08-09T13:51:35.454 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:35.451+0000 7ff8d67fc640 1 -- 172.21.15.43:0/3504827532 --> v1:172.21.15.43:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ff8c0003520 con 0x7ff8d80b0230 2023-08-09T13:51:35.454 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:35.451+0000 7ff8d77fe640 1 -- 172.21.15.43:0/3504827532 >> v1:172.21.15.203:6789/0 conn(0x7ff8d80ccbb0 legacy=0x7ff8d80afb20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:51:35.454 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:35.451+0000 7ff8d67fc640 1 -- 172.21.15.43:0/3504827532 <== mon.0 v1:172.21.15.43:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 362775105 0 0) 0x7ff8c0003520 con 0x7ff8d80b0230 2023-08-09T13:51:35.455 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:35.451+0000 7ff8d77fe640 1 -- 172.21.15.43:0/3504827532 >> v1:172.21.15.43:6789/0 conn(0x7ff8d80b0230 legacy=0x7ff8d80cb6d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:51:35.455 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:35.451+0000 7ff8d77fe640 1 -- 172.21.15.43:0/3504827532 --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff8d80b0770 con 0x7ff8d80cee90 2023-08-09T13:51:35.455 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:35.452+0000 7ff8d77fe640 1 --2- 172.21.15.43:0/3504827532 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7ff8d80cee90 0x7ff8d80cdd90 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7ff8c80097c0 tx=0x7ff8c802b7f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=bc2d1465767aa40d server_cookie=0 in_seq=0 out_seq=0 2023-08-09T13:51:35.455 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:35.453+0000 7ff8d67fc640 1 -- 172.21.15.43:0/3504827532 <== mon.2 v2:172.21.15.203:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff8d00019f0 con 0x7ff8d80cee90 2023-08-09T13:51:35.456 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:35.453+0000 7ff8d67fc640 1 -- 172.21.15.43:0/3504827532 <== mon.2 v2:172.21.15.203:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff8c80314d0 con 0x7ff8d80cee90 2023-08-09T13:51:35.456 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:35.453+0000 7ff8d67fc640 1 -- 172.21.15.43:0/3504827532 <== mon.2 v2:172.21.15.203:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff8c8031840 con 0x7ff8d80cee90 2023-08-09T13:51:35.456 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:35.454+0000 7ff8ddf2f640 1 -- 172.21.15.43:0/3504827532 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7ff8d80cee90 msgr2=0x7ff8d80cdd90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:51:35.457 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:35.454+0000 7ff8ddf2f640 1 --2- 172.21.15.43:0/3504827532 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7ff8d80cee90 0x7ff8d80cdd90 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7ff8c80097c0 tx=0x7ff8c802b7f0 comp rx=0 tx=0).stop 2023-08-09T13:51:35.457 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:35.454+0000 7ff8ddf2f640 1 -- 172.21.15.43:0/3504827532 shutdown_connections 2023-08-09T13:51:35.457 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:35.454+0000 7ff8ddf2f640 1 --2- 172.21.15.43:0/3504827532 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7ff8d80cee90 0x7ff8d80cdd90 unknown :-1 s=CLOSED pgs=167 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:51:35.457 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:35.454+0000 7ff8ddf2f640 1 -- 172.21.15.43:0/3504827532 >> 172.21.15.43:0/3504827532 conn(0x7ff8d8089fe0 msgr2=0x7ff8d80d01c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-09T13:51:35.457 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:35.454+0000 7ff8ddf2f640 1 -- 172.21.15.43:0/3504827532 shutdown_connections 2023-08-09T13:51:35.457 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:35.454+0000 7ff8ddf2f640 1 -- 172.21.15.43:0/3504827532 wait complete. 2023-08-09T13:51:35.458 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:35.456+0000 7ff8ddf2f640 1 Processor -- start 2023-08-09T13:51:35.458 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:35.456+0000 7ff8ddf2f640 1 -- start start 2023-08-09T13:51:35.460 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:35.457+0000 7ff8ddf2f640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7ff8d80b0230 0x7ff8d80c7da0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:51:35.460 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:35.457+0000 7ff8ddf2f640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7ff8d80ccbb0 0x7ff8d8155610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:51:35.461 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:35.457+0000 7ff8ddf2f640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff8d80cee90 0x7ff8d80ae4e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:51:35.461 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:35.457+0000 7ff8ddf2f640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff8d80c7980 con 0x7ff8d80cee90 2023-08-09T13:51:35.461 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:35.457+0000 7ff8ddf2f640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff8d81574a0 con 0x7ff8d80b0230 2023-08-09T13:51:35.462 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:35.457+0000 7ff8ddf2f640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff8d80c82e0 con 0x7ff8d80ccbb0 2023-08-09T13:51:35.462 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:35.457+0000 7ff8d7fff640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff8d80cee90 0x7ff8d80ae4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:51:35.462 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:35.457+0000 7ff8d7fff640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff8d80cee90 0x7ff8d80ae4e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:41258/0 (socket says 172.21.15.43:41258) 2023-08-09T13:51:35.462 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:35.457+0000 7ff8d7fff640 1 -- 172.21.15.43:0/570470178 learned_addr learned my addr 172.21.15.43:0/570470178 (peer_addr_for_me v2:172.21.15.43:0/0) 2023-08-09T13:51:35.462 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:35.458+0000 7ff8d6ffd640 1 --2- 172.21.15.43:0/570470178 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7ff8d80ccbb0 0x7ff8d8155610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:51:35.462 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:35.458+0000 7ff8d77fe640 1 --2- 172.21.15.43:0/570470178 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7ff8d80b0230 0x7ff8d80c7da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:51:35.462 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:35.458+0000 7ff8d7fff640 1 -- 172.21.15.43:0/570470178 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7ff8d80ccbb0 msgr2=0x7ff8d8155610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:51:35.463 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:35.458+0000 7ff8d7fff640 1 --2- 172.21.15.43:0/570470178 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7ff8d80ccbb0 0x7ff8d8155610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:51:35.463 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:35.458+0000 7ff8d7fff640 1 -- 172.21.15.43:0/570470178 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7ff8d80b0230 msgr2=0x7ff8d80c7da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:51:35.463 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:35.458+0000 7ff8d7fff640 1 --2- 172.21.15.43:0/570470178 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7ff8d80b0230 0x7ff8d80c7da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:51:35.464 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:35.458+0000 7ff8d7fff640 1 -- 172.21.15.43:0/570470178 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff8c80093a0 con 0x7ff8d80cee90 2023-08-09T13:51:35.465 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:35.458+0000 7ff8d7fff640 1 --2- 172.21.15.43:0/570470178 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff8d80cee90 0x7ff8d80ae4e0 secure :-1 s=READY pgs=1235 cs=0 l=1 rev1=1 crypto rx=0x7ff8d0001fa0 tx=0x7ff8d000a430 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-09T13:51:35.465 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:35.458+0000 7ff8bffff640 1 -- 172.21.15.43:0/570470178 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff8d0012400 con 0x7ff8d80cee90 2023-08-09T13:51:35.465 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:35.458+0000 7ff8bffff640 1 -- 172.21.15.43:0/570470178 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff8d0012560 con 0x7ff8d80cee90 2023-08-09T13:51:35.466 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:35.458+0000 7ff8ddf2f640 1 -- 172.21.15.43:0/570470178 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff8d80c7330 con 0x7ff8d80cee90 2023-08-09T13:51:35.466 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:35.458+0000 7ff8bffff640 1 -- 172.21.15.43:0/570470178 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff8d00128f0 con 0x7ff8d80cee90 2023-08-09T13:51:35.466 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:35.459+0000 7ff8ddf2f640 1 -- 172.21.15.43:0/570470178 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff8d8154930 con 0x7ff8d80cee90 2023-08-09T13:51:35.466 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:35.460+0000 7ff8ddf2f640 1 -- 172.21.15.43:0/570470178 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff8d815dfc0 con 0x7ff8d80cee90 2023-08-09T13:51:35.466 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:35.461+0000 7ff8bffff640 1 -- 172.21.15.43:0/570470178 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 41) v1 ==== 88444+0+0 (secure 0 0 0) 0x7ff8d0012a50 con 0x7ff8d80cee90 2023-08-09T13:51:35.466 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:35.462+0000 7ff8bffff640 1 --2- 172.21.15.43:0/570470178 >> [v2:172.21.15.203:6832/2675832148,v1:172.21.15.203:6833/2675832148] conn(0x7ff8c406a170 0x7ff8c406c630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:51:35.466 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:35.462+0000 7ff8bffff640 1 -- 172.21.15.43:0/570470178 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(104..104 src has 1..104) v4 ==== 6770+0+0 (secure 0 0 0) 0x7ff8d0089160 con 0x7ff8d80cee90 2023-08-09T13:51:35.467 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:35.462+0000 7ff8d77fe640 1 -- 172.21.15.43:0/570470178 >> [v2:172.21.15.203:6832/2675832148,v1:172.21.15.203:6833/2675832148] conn(0x7ff8c406a170 msgr2=0x7ff8c406c630 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.203:6832/2675832148 2023-08-09T13:51:35.467 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:35.462+0000 7ff8d77fe640 1 --2- 172.21.15.43:0/570470178 >> [v2:172.21.15.203:6832/2675832148,v1:172.21.15.203:6833/2675832148] conn(0x7ff8c406a170 0x7ff8c406c630 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-08-09T13:51:35.468 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:35.466+0000 7ff8bffff640 1 -- 172.21.15.43:0/570470178 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178066 (secure 0 0 0) 0x7ff8d815dfc0 con 0x7ff8d80cee90 2023-08-09T13:51:35.498 INFO:tasks.ceph.mgr.x.smithi203.stderr:2023-08-09T13:51:35.495+0000 7f67a7a3b040 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2023-08-09T13:51:35.512 INFO:tasks.ceph.mgr.y.smithi043.stderr:2023-08-09T13:51:35.509+0000 7fc4e782c040 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2023-08-09T13:51:35.553 INFO:tasks.ceph.mgr.x.smithi203.stderr:2023-08-09T13:51:35.551+0000 7f67a7a3b040 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2023-08-09T13:51:35.602 INFO:tasks.ceph.mgr.x.smithi203.stderr:2023-08-09T13:51:35.599+0000 7f67a7a3b040 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2023-08-09T13:51:35.637 INFO:tasks.ceph.mgr.y.smithi043.stderr:2023-08-09T13:51:35.635+0000 7fc4e782c040 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2023-08-09T13:51:35.659 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:35.656+0000 7ff8ddf2f640 1 -- 172.21.15.43:0/570470178 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"} v 0) v1 -- 0x7ff8d815dfc0 con 0x7ff8d80cee90 2023-08-09T13:51:35.660 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:35.657+0000 7ff8bffff640 1 -- 172.21.15.43:0/570470178 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"}]=0 v104) v1 ==== 92+0+10 (secure 0 0 0) 0x7ff8d815dfc0 con 0x7ff8d80cee90 2023-08-09T13:51:35.660 INFO:teuthology.orchestra.run.smithi043.stdout:pg_num: 8 2023-08-09T13:51:35.662 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:35.660+0000 7ff8ddf2f640 1 -- 172.21.15.43:0/570470178 >> [v2:172.21.15.203:6832/2675832148,v1:172.21.15.203:6833/2675832148] conn(0x7ff8c406a170 msgr2=0x7ff8c406c630 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-08-09T13:51:35.662 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:35.660+0000 7ff8ddf2f640 1 --2- 172.21.15.43:0/570470178 >> [v2:172.21.15.203:6832/2675832148,v1:172.21.15.203:6833/2675832148] conn(0x7ff8c406a170 0x7ff8c406c630 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:51:35.662 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:35.660+0000 7ff8ddf2f640 1 -- 172.21.15.43:0/570470178 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff8d80cee90 msgr2=0x7ff8d80ae4e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:51:35.662 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:35.660+0000 7ff8ddf2f640 1 --2- 172.21.15.43:0/570470178 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff8d80cee90 0x7ff8d80ae4e0 secure :-1 s=READY pgs=1235 cs=0 l=1 rev1=1 crypto rx=0x7ff8d0001fa0 tx=0x7ff8d000a430 comp rx=0 tx=0).stop 2023-08-09T13:51:35.663 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:35.660+0000 7ff8ddf2f640 1 -- 172.21.15.43:0/570470178 shutdown_connections 2023-08-09T13:51:35.663 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:35.660+0000 7ff8ddf2f640 1 --2- 172.21.15.43:0/570470178 >> [v2:172.21.15.203:6832/2675832148,v1:172.21.15.203:6833/2675832148] conn(0x7ff8c406a170 0x7ff8c406c630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:51:35.664 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:35.660+0000 7ff8ddf2f640 1 --2- 172.21.15.43:0/570470178 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff8d80cee90 0x7ff8d80ae4e0 unknown :-1 s=CLOSED pgs=1235 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:51:35.664 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:35.660+0000 7ff8ddf2f640 1 --2- 172.21.15.43:0/570470178 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7ff8d80ccbb0 0x7ff8d8155610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:51:35.664 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:35.660+0000 7ff8ddf2f640 1 --2- 172.21.15.43:0/570470178 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7ff8d80b0230 0x7ff8d80c7da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:51:35.664 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:35.660+0000 7ff8ddf2f640 1 -- 172.21.15.43:0/570470178 >> 172.21.15.43:0/570470178 conn(0x7ff8d8089fe0 msgr2=0x7ff8d80ad6f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-09T13:51:35.665 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:35.660+0000 7ff8ddf2f640 1 -- 172.21.15.43:0/570470178 shutdown_connections 2023-08-09T13:51:35.665 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:35.660+0000 7ff8ddf2f640 1 -- 172.21.15.43:0/570470178 wait complete. 2023-08-09T13:51:35.681 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-08-09T13:51:35.700 INFO:tasks.ceph.mgr.y.smithi043.stderr:2023-08-09T13:51:35.697+0000 7fc4e782c040 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2023-08-09T13:51:35.752 INFO:tasks.ceph.mgr.y.smithi043.stderr:2023-08-09T13:51:35.750+0000 7fc4e782c040 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2023-08-09T13:51:35.894 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:35.891+0000 7f4eb549e640 1 Processor -- start 2023-08-09T13:51:35.894 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:35.892+0000 7f4eb549e640 1 -- start start 2023-08-09T13:51:35.895 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:35.893+0000 7f4eb549e640 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7f4eb015c2d0 0x7f4eb00b7df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:51:35.896 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:35.893+0000 7f4eb549e640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f4eb00d1b30 0x7f4eb00cd020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:51:35.896 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:35.893+0000 7f4eb549e640 1 -- --> v1:172.21.15.203:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4eb00d16a0 con 0x7f4eb00cebf0 2023-08-09T13:51:35.896 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:35.893+0000 7f4eb549e640 1 -- --> v2:172.21.15.203:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4eb00d1f70 con 0x7f4eb015c2d0 2023-08-09T13:51:35.896 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:35.893+0000 7f4eb549e640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4eb00cd560 con 0x7f4eb00d1b30 2023-08-09T13:51:35.897 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:35.894+0000 7f4eaf7fe640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f4eb00d1b30 0x7f4eb00cd020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:51:35.897 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:35.894+0000 7f4eaffff640 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7f4eb015c2d0 0x7f4eb00b7df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:51:35.898 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:35.894+0000 7f4eaf7fe640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f4eb00d1b30 0x7f4eb00cd020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.203:3301/0 says I am v2:172.21.15.43:36006/0 (socket says 172.21.15.43:36006) 2023-08-09T13:51:35.898 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:35.894+0000 7f4eaf7fe640 1 -- 172.21.15.43:0/2561731998 learned_addr learned my addr 172.21.15.43:0/2561731998 (peer_addr_for_me v2:172.21.15.43:0/0) 2023-08-09T13:51:35.899 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:35.895+0000 7f4eaf7fe640 1 -- 172.21.15.43:0/2561731998 >> v2:172.21.15.203:3300/0 conn(0x7f4eb015c2d0 msgr2=0x7f4eb00b7df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-09T13:51:35.899 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:35.895+0000 7f4eaf7fe640 1 --2- 172.21.15.43:0/2561731998 >> v2:172.21.15.203:3300/0 conn(0x7f4eb015c2d0 0x7f4eb00b7df0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:51:35.899 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:35.895+0000 7f4eaf7fe640 1 -- 172.21.15.43:0/2561731998 >> v1:172.21.15.203:6789/0 conn(0x7f4eb00cebf0 legacy=0x7f4eb00b3c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:51:35.899 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:35.895+0000 7f4eaf7fe640 1 -- 172.21.15.43:0/2561731998 --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4eb00d0d70 con 0x7f4eb00d1b30 2023-08-09T13:51:35.900 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:35.895+0000 7f4eaffff640 1 --2- 172.21.15.43:0/2561731998 >> v2:172.21.15.203:3300/0 conn(0x7f4eb015c2d0 0x7f4eb00b7df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-08-09T13:51:35.901 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:35.896+0000 7f4eaf7fe640 1 --2- 172.21.15.43:0/2561731998 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f4eb00d1b30 0x7f4eb00cd020 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7f4ea4009870 tx=0x7f4ea402b980 comp rx=0 tx=0).ready entity=mon.2 client_cookie=61d9626406314985 server_cookie=0 in_seq=0 out_seq=0 2023-08-09T13:51:35.902 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:35.896+0000 7f4eaeffd640 1 -- 172.21.15.43:0/2561731998 <== mon.2 v2:172.21.15.203:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4ea4030840 con 0x7f4eb00d1b30 2023-08-09T13:51:35.902 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:35.896+0000 7f4eaeffd640 1 -- 172.21.15.43:0/2561731998 <== mon.2 v2:172.21.15.203:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4ea40309a0 con 0x7f4eb00d1b30 2023-08-09T13:51:35.902 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:35.896+0000 7f4eaeffd640 1 -- 172.21.15.43:0/2561731998 <== mon.2 v2:172.21.15.203:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4ea4030d30 con 0x7f4eb00d1b30 2023-08-09T13:51:35.902 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:35.897+0000 7f4eb549e640 1 -- 172.21.15.43:0/2561731998 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f4eb00d1b30 msgr2=0x7f4eb00cd020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:51:35.903 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:35.897+0000 7f4eb549e640 1 --2- 172.21.15.43:0/2561731998 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f4eb00d1b30 0x7f4eb00cd020 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7f4ea4009870 tx=0x7f4ea402b980 comp rx=0 tx=0).stop 2023-08-09T13:51:35.903 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:35.897+0000 7f4eb549e640 1 -- 172.21.15.43:0/2561731998 shutdown_connections 2023-08-09T13:51:35.903 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:35.897+0000 7f4eb549e640 1 --2- 172.21.15.43:0/2561731998 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f4eb00d1b30 0x7f4eb00cd020 unknown :-1 s=CLOSED pgs=168 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:51:35.904 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:35.897+0000 7f4eb549e640 1 --2- 172.21.15.43:0/2561731998 >> v2:172.21.15.203:3300/0 conn(0x7f4eb015c2d0 0x7f4eb00b7df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:51:35.904 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:35.897+0000 7f4eb549e640 1 -- 172.21.15.43:0/2561731998 >> 172.21.15.43:0/2561731998 conn(0x7f4eb006a230 msgr2=0x7f4eb006a640 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-09T13:51:35.904 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:35.897+0000 7f4eb549e640 1 -- 172.21.15.43:0/2561731998 shutdown_connections 2023-08-09T13:51:35.904 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:35.897+0000 7f4eb549e640 1 -- 172.21.15.43:0/2561731998 wait complete. 2023-08-09T13:51:35.905 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:35.899+0000 7f4eb549e640 1 Processor -- start 2023-08-09T13:51:35.905 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:35.899+0000 7f4eb549e640 1 -- start start 2023-08-09T13:51:35.908 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:35.900+0000 7f4eb549e640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f4eb00cebf0 0x7f4eb01f5090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:51:35.909 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:35.900+0000 7f4eb549e640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f4eb00d1b30 0x7f4eb01f2560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:51:35.909 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:35.900+0000 7f4eb549e640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f4eb015c2d0 0x7f4eb014c4f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:51:35.909 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:35.900+0000 7f4eb549e640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4eb01f2aa0 con 0x7f4eb00d1b30 2023-08-09T13:51:35.910 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:35.900+0000 7f4eb549e640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4eb01562e0 con 0x7f4eb00cebf0 2023-08-09T13:51:35.910 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:35.900+0000 7f4eb549e640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4eb01f56e0 con 0x7f4eb015c2d0 2023-08-09T13:51:35.910 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:35.900+0000 7f4eaf7fe640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f4eb00d1b30 0x7f4eb01f2560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:51:35.910 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:35.900+0000 7f4eaf7fe640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f4eb00d1b30 0x7f4eb01f2560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:41270/0 (socket says 172.21.15.43:41270) 2023-08-09T13:51:35.911 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:35.900+0000 7f4eaf7fe640 1 -- 172.21.15.43:0/3085674082 learned_addr learned my addr 172.21.15.43:0/3085674082 (peer_addr_for_me v2:172.21.15.43:0/0) 2023-08-09T13:51:35.911 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:35.901+0000 7f4eaffff640 1 --2- 172.21.15.43:0/3085674082 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f4eb00cebf0 0x7f4eb01f5090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:51:35.911 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:35.901+0000 7f4eb4c9d640 1 --2- 172.21.15.43:0/3085674082 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f4eb015c2d0 0x7f4eb014c4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:51:35.912 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:35.901+0000 7f4eaf7fe640 1 -- 172.21.15.43:0/3085674082 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f4eb015c2d0 msgr2=0x7f4eb014c4f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:51:35.912 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:35.901+0000 7f4eaf7fe640 1 --2- 172.21.15.43:0/3085674082 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f4eb015c2d0 0x7f4eb014c4f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:51:35.912 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:35.901+0000 7f4eaf7fe640 1 -- 172.21.15.43:0/3085674082 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f4eb00cebf0 msgr2=0x7f4eb01f5090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:51:35.912 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:35.901+0000 7f4eaf7fe640 1 --2- 172.21.15.43:0/3085674082 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f4eb00cebf0 0x7f4eb01f5090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:51:35.913 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:35.901+0000 7f4eaf7fe640 1 -- 172.21.15.43:0/3085674082 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4ea0009400 con 0x7f4eb00d1b30 2023-08-09T13:51:35.913 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:35.901+0000 7f4eb4c9d640 1 --2- 172.21.15.43:0/3085674082 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f4eb015c2d0 0x7f4eb014c4f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-08-09T13:51:35.913 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:35.901+0000 7f4eaf7fe640 1 --2- 172.21.15.43:0/3085674082 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f4eb00d1b30 0x7f4eb01f2560 secure :-1 s=READY pgs=1236 cs=0 l=1 rev1=1 crypto rx=0x7f4ea4000c00 tx=0x7f4ea4004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-09T13:51:35.913 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:35.901+0000 7f4eacff9640 1 -- 172.21.15.43:0/3085674082 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4ea4039c90 con 0x7f4eb00d1b30 2023-08-09T13:51:35.914 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:35.902+0000 7f4eacff9640 1 -- 172.21.15.43:0/3085674082 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4ea4039df0 con 0x7f4eb00d1b30 2023-08-09T13:51:35.914 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:35.902+0000 7f4eb549e640 1 -- 172.21.15.43:0/3085674082 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4ea40094d0 con 0x7f4eb00d1b30 2023-08-09T13:51:35.914 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:35.902+0000 7f4eacff9640 1 -- 172.21.15.43:0/3085674082 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4ea4043640 con 0x7f4eb00d1b30 2023-08-09T13:51:35.914 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:35.902+0000 7f4eb549e640 1 -- 172.21.15.43:0/3085674082 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4eb01f5ca0 con 0x7f4eb00d1b30 2023-08-09T13:51:35.915 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:35.903+0000 7f4eb549e640 1 -- 172.21.15.43:0/3085674082 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4eb00cc8d0 con 0x7f4eb00d1b30 2023-08-09T13:51:35.915 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:35.904+0000 7f4eacff9640 1 -- 172.21.15.43:0/3085674082 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 41) v1 ==== 88444+0+0 (secure 0 0 0) 0x7f4ea40437a0 con 0x7f4eb00d1b30 2023-08-09T13:51:35.915 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:35.905+0000 7f4eacff9640 1 --2- 172.21.15.43:0/3085674082 >> [v2:172.21.15.203:6832/2675832148,v1:172.21.15.203:6833/2675832148] conn(0x7f4e9c06a170 0x7f4e9c06c630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:51:35.916 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:35.905+0000 7f4eacff9640 1 -- 172.21.15.43:0/3085674082 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(104..104 src has 1..104) v4 ==== 6770+0+0 (secure 0 0 0) 0x7f4ea403e070 con 0x7f4eb00d1b30 2023-08-09T13:51:35.916 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:35.905+0000 7f4eaffff640 1 -- 172.21.15.43:0/3085674082 >> [v2:172.21.15.203:6832/2675832148,v1:172.21.15.203:6833/2675832148] conn(0x7f4e9c06a170 msgr2=0x7f4e9c06c630 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.203:6832/2675832148 2023-08-09T13:51:35.916 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:35.906+0000 7f4eaffff640 1 --2- 172.21.15.43:0/3085674082 >> [v2:172.21.15.203:6832/2675832148,v1:172.21.15.203:6833/2675832148] conn(0x7f4e9c06a170 0x7f4e9c06c630 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-08-09T13:51:35.917 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:35.910+0000 7f4eacff9640 1 -- 172.21.15.43:0/3085674082 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178066 (secure 0 0 0) 0x7f4ea4076d40 con 0x7f4eb00d1b30 2023-08-09T13:51:36.103 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:36.101+0000 7f4eb549e640 1 -- 172.21.15.43:0/3085674082 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f4eb004e390 con 0x7f4eb00d1b30 2023-08-09T13:51:36.104 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:36.102+0000 7f4eacff9640 1 -- 172.21.15.43:0/3085674082 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v104) v1 ==== 93+0+10 (secure 0 0 0) 0x7f4ea40340a0 con 0x7f4eb00d1b30 2023-08-09T13:51:36.104 INFO:teuthology.orchestra.run.smithi043.stdout:pg_num: 1 2023-08-09T13:51:36.107 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:36.104+0000 7f4eb549e640 1 -- 172.21.15.43:0/3085674082 >> [v2:172.21.15.203:6832/2675832148,v1:172.21.15.203:6833/2675832148] conn(0x7f4e9c06a170 msgr2=0x7f4e9c06c630 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-08-09T13:51:36.108 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:36.104+0000 7f4eb549e640 1 --2- 172.21.15.43:0/3085674082 >> [v2:172.21.15.203:6832/2675832148,v1:172.21.15.203:6833/2675832148] conn(0x7f4e9c06a170 0x7f4e9c06c630 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:51:36.108 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:36.105+0000 7f4eb549e640 1 -- 172.21.15.43:0/3085674082 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f4eb00d1b30 msgr2=0x7f4eb01f2560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:51:36.108 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:36.105+0000 7f4eb549e640 1 --2- 172.21.15.43:0/3085674082 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f4eb00d1b30 0x7f4eb01f2560 secure :-1 s=READY pgs=1236 cs=0 l=1 rev1=1 crypto rx=0x7f4ea4000c00 tx=0x7f4ea4004730 comp rx=0 tx=0).stop 2023-08-09T13:51:36.108 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:36.105+0000 7f4eb549e640 1 -- 172.21.15.43:0/3085674082 shutdown_connections 2023-08-09T13:51:36.108 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:36.105+0000 7f4eb549e640 1 --2- 172.21.15.43:0/3085674082 >> [v2:172.21.15.203:6832/2675832148,v1:172.21.15.203:6833/2675832148] conn(0x7f4e9c06a170 0x7f4e9c06c630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:51:36.108 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:36.105+0000 7f4eb549e640 1 --2- 172.21.15.43:0/3085674082 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f4eb015c2d0 0x7f4eb014c4f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:51:36.109 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:36.105+0000 7f4eb549e640 1 --2- 172.21.15.43:0/3085674082 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f4eb00d1b30 0x7f4eb01f2560 unknown :-1 s=CLOSED pgs=1236 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:51:36.109 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:36.105+0000 7f4eb549e640 1 --2- 172.21.15.43:0/3085674082 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f4eb00cebf0 0x7f4eb01f5090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:51:36.109 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:36.105+0000 7f4eb549e640 1 -- 172.21.15.43:0/3085674082 >> 172.21.15.43:0/3085674082 conn(0x7f4eb006a230 msgr2=0x7f4eb00b3320 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-09T13:51:36.109 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:36.105+0000 7f4eb549e640 1 -- 172.21.15.43:0/3085674082 shutdown_connections 2023-08-09T13:51:36.109 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:36.105+0000 7f4eb549e640 1 -- 172.21.15.43:0/3085674082 wait complete. 2023-08-09T13:51:36.126 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-08-09T13:51:36.309 INFO:tasks.ceph.mgr.x.smithi203.stderr:2023-08-09T13:51:36.306+0000 7f67a7a3b040 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2023-08-09T13:51:36.375 INFO:tasks.ceph.mgr.x.smithi203.stderr:2023-08-09T13:51:36.373+0000 7f67a7a3b040 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2023-08-09T13:51:36.378 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:36.376+0000 7fa386b0f640 1 Processor -- start 2023-08-09T13:51:36.378 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:36.376+0000 7fa386b0f640 1 -- start start 2023-08-09T13:51:36.380 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:36.378+0000 7fa386b0f640 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7fa38015a360 0x7fa3800cb080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:51:36.380 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:36.378+0000 7fa386b0f640 1 -- --> v1:172.21.15.43:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa38015ded0 con 0x7fa3800cb5c0 2023-08-09T13:51:36.380 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:36.378+0000 7fa386b0f640 1 -- --> v1:172.21.15.203:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa38015e1b0 con 0x7fa380069ed0 2023-08-09T13:51:36.380 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:36.378+0000 7fa386b0f640 1 -- --> v2:172.21.15.203:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa38015e490 con 0x7fa38015a360 2023-08-09T13:51:36.382 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:36.378+0000 7fa38530c640 1 --1- >> v1:172.21.15.43:6789/0 conn(0x7fa3800cb5c0 0x7fa3800697c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.43:6789/0 says I am v1:172.21.15.43:60766/0 (socket says 172.21.15.43:60766) 2023-08-09T13:51:36.382 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:36.378+0000 7fa38530c640 1 -- 172.21.15.43:0/3685332677 learned_addr learned my addr 172.21.15.43:0/3685332677 (peer_addr_for_me v1:172.21.15.43:0/0) 2023-08-09T13:51:36.382 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:36.378+0000 7fa385b0d640 1 --2- 172.21.15.43:0/3685332677 >> v2:172.21.15.203:3300/0 conn(0x7fa38015a360 0x7fa3800cb080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:51:36.382 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:36.379+0000 7fa384b0b640 1 -- 172.21.15.43:0/3685332677 <== mon.0 v1:172.21.15.43:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 621487861 0 0) 0x7fa38015ded0 con 0x7fa3800cb5c0 2023-08-09T13:51:36.382 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:36.379+0000 7fa384b0b640 1 -- 172.21.15.43:0/3685332677 --> v1:172.21.15.43:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fa36c003520 con 0x7fa3800cb5c0 2023-08-09T13:51:36.382 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:36.379+0000 7fa384b0b640 1 -- 172.21.15.43:0/3685332677 <== mon.0 v1:172.21.15.43:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3415118745 0 0) 0x7fa36c003520 con 0x7fa3800cb5c0 2023-08-09T13:51:36.382 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:36.379+0000 7fa384b0b640 1 -- 172.21.15.43:0/3685332677 >> v2:172.21.15.203:3300/0 conn(0x7fa38015a360 msgr2=0x7fa3800cb080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-09T13:51:36.383 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:36.379+0000 7fa384b0b640 1 --2- 172.21.15.43:0/3685332677 >> v2:172.21.15.203:3300/0 conn(0x7fa38015a360 0x7fa3800cb080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:51:36.383 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:36.379+0000 7fa384b0b640 1 -- 172.21.15.43:0/3685332677 >> v1:172.21.15.203:6789/0 conn(0x7fa380069ed0 legacy=0x7fa380070670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:51:36.383 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:36.379+0000 7fa384b0b640 1 -- 172.21.15.43:0/3685332677 --> v1:172.21.15.43:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa38015e770 con 0x7fa3800cb5c0 2023-08-09T13:51:36.383 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:36.379+0000 7fa384b0b640 1 -- 172.21.15.43:0/3685332677 <== mon.0 v1:172.21.15.43:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1285150740 0 0) 0x7fa370001670 con 0x7fa3800cb5c0 2023-08-09T13:51:36.383 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:36.379+0000 7fa384b0b640 1 -- 172.21.15.43:0/3685332677 >> v1:172.21.15.43:6789/0 conn(0x7fa3800cb5c0 legacy=0x7fa3800697c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:51:36.384 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:36.379+0000 7fa384b0b640 1 --2- 172.21.15.43:0/3685332677 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fa36c0039c0 0x7fa36c003de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:51:36.384 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:36.379+0000 7fa384b0b640 1 --2- 172.21.15.43:0/3685332677 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa36c0047c0 0x7fa36c004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:51:36.384 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:36.379+0000 7fa384b0b640 1 --2- 172.21.15.43:0/3685332677 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fa36c005180 0x7fa36c00b980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:51:36.384 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:36.379+0000 7fa384b0b640 1 -- 172.21.15.43:0/3685332677 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa38015e490 con 0x7fa36c0047c0 2023-08-09T13:51:36.384 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:36.379+0000 7fa384b0b640 1 -- 172.21.15.43:0/3685332677 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa36c00c080 con 0x7fa36c005180 2023-08-09T13:51:36.384 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:36.379+0000 7fa384b0b640 1 -- 172.21.15.43:0/3685332677 --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa36c00c240 con 0x7fa36c0039c0 2023-08-09T13:51:36.384 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:36.379+0000 7fa38530c640 1 --2- 172.21.15.43:0/3685332677 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa36c0047c0 0x7fa36c004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:51:36.385 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:36.380+0000 7fa38630e640 1 --2- 172.21.15.43:0/3685332677 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fa36c005180 0x7fa36c00b980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:51:36.386 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:36.380+0000 7fa38530c640 1 -- 172.21.15.43:0/3685332677 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fa36c0039c0 msgr2=0x7fa36c003de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-09T13:51:36.386 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:36.380+0000 7fa38530c640 1 --2- 172.21.15.43:0/3685332677 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fa36c0039c0 0x7fa36c003de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:51:36.386 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:36.380+0000 7fa38530c640 1 -- 172.21.15.43:0/3685332677 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fa36c005180 msgr2=0x7fa36c00b980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-09T13:51:36.386 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:36.380+0000 7fa38530c640 1 --2- 172.21.15.43:0/3685332677 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fa36c005180 0x7fa36c00b980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:51:36.386 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:36.380+0000 7fa38530c640 1 -- 172.21.15.43:0/3685332677 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa36c003520 con 0x7fa36c0047c0 2023-08-09T13:51:36.386 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:36.380+0000 7fa38630e640 1 --2- 172.21.15.43:0/3685332677 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fa36c005180 0x7fa36c00b980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-08-09T13:51:36.387 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:36.380+0000 7fa38630e640 1 -- 172.21.15.43:0/3685332677 reap_dead start 2023-08-09T13:51:36.387 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:36.381+0000 7fa38530c640 1 --2- 172.21.15.43:0/3685332677 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa36c0047c0 0x7fa36c004c40 secure :-1 s=READY pgs=1238 cs=0 l=1 rev1=1 crypto rx=0x7fa37000c0e0 tx=0x7fa36c0057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5a2189af3727fe48 server_cookie=0 in_seq=0 out_seq=0 2023-08-09T13:51:36.387 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:36.381+0000 7fa384b0b640 1 -- 172.21.15.43:0/3685332677 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa370005ad0 con 0x7fa36c0047c0 2023-08-09T13:51:36.387 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:36.381+0000 7fa384b0b640 1 -- 172.21.15.43:0/3685332677 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa370005c50 con 0x7fa36c0047c0 2023-08-09T13:51:36.387 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:36.381+0000 7fa384b0b640 1 -- 172.21.15.43:0/3685332677 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa37002c810 con 0x7fa36c0047c0 2023-08-09T13:51:36.387 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:36.382+0000 7fa386b0f640 1 -- 172.21.15.43:0/3685332677 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa36c0047c0 msgr2=0x7fa36c004c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:51:36.387 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:36.382+0000 7fa386b0f640 1 --2- 172.21.15.43:0/3685332677 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa36c0047c0 0x7fa36c004c40 secure :-1 s=READY pgs=1238 cs=0 l=1 rev1=1 crypto rx=0x7fa37000c0e0 tx=0x7fa36c0057f0 comp rx=0 tx=0).stop 2023-08-09T13:51:36.387 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:36.382+0000 7fa386b0f640 1 -- 172.21.15.43:0/3685332677 shutdown_connections 2023-08-09T13:51:36.388 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:36.382+0000 7fa386b0f640 1 --2- 172.21.15.43:0/3685332677 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa36c0047c0 0x7fa36c004c40 unknown :-1 s=CLOSED pgs=1238 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:51:36.388 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:36.382+0000 7fa386b0f640 1 -- 172.21.15.43:0/3685332677 >> 172.21.15.43:0/3685332677 conn(0x7fa3800cef40 msgr2=0x7fa3800cf350 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-09T13:51:36.388 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:36.382+0000 7fa386b0f640 1 -- 172.21.15.43:0/3685332677 shutdown_connections 2023-08-09T13:51:36.388 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:36.382+0000 7fa386b0f640 1 -- 172.21.15.43:0/3685332677 wait complete. 2023-08-09T13:51:36.390 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:36.384+0000 7fa386b0f640 1 Processor -- start 2023-08-09T13:51:36.390 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:36.384+0000 7fa386b0f640 1 -- start start 2023-08-09T13:51:36.390 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:36.385+0000 7fa386b0f640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa36c0047c0 0x7fa38014c6f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:51:36.390 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:36.385+0000 7fa386b0f640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fa3800c6010 0x7fa3800c6470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:51:36.390 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:36.385+0000 7fa386b0f640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fa380157e00 0x7fa380158280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:51:36.390 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:36.385+0000 7fa386b0f640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa38014d3c0 con 0x7fa36c0047c0 2023-08-09T13:51:36.391 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:36.385+0000 7fa386b0f640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa3800c4490 con 0x7fa380157e00 2023-08-09T13:51:36.391 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:36.385+0000 7fa386b0f640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa3800c4730 con 0x7fa3800c6010 2023-08-09T13:51:36.391 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:36.385+0000 7fa385b0d640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa36c0047c0 0x7fa38014c6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:51:36.391 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:36.385+0000 7fa385b0d640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa36c0047c0 0x7fa38014c6f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:41290/0 (socket says 172.21.15.43:41290) 2023-08-09T13:51:36.391 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:36.385+0000 7fa385b0d640 1 -- 172.21.15.43:0/2164548093 learned_addr learned my addr 172.21.15.43:0/2164548093 (peer_addr_for_me v2:172.21.15.43:0/0) 2023-08-09T13:51:36.391 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:36.386+0000 7fa38630e640 1 --2- 172.21.15.43:0/2164548093 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fa380157e00 0x7fa380158280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:51:36.391 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:36.386+0000 7fa385b0d640 1 -- 172.21.15.43:0/2164548093 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fa3800c6010 msgr2=0x7fa3800c6470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:51:36.392 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:36.386+0000 7fa385b0d640 1 --2- 172.21.15.43:0/2164548093 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fa3800c6010 0x7fa3800c6470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:51:36.392 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:36.386+0000 7fa385b0d640 1 -- 172.21.15.43:0/2164548093 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fa380157e00 msgr2=0x7fa380158280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:51:36.392 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:36.386+0000 7fa385b0d640 1 --2- 172.21.15.43:0/2164548093 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fa380157e00 0x7fa380158280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:51:36.392 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:36.386+0000 7fa385b0d640 1 -- 172.21.15.43:0/2164548093 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa3800c4a10 con 0x7fa36c0047c0 2023-08-09T13:51:36.392 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:36.386+0000 7fa385b0d640 1 --2- 172.21.15.43:0/2164548093 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa36c0047c0 0x7fa38014c6f0 secure :-1 s=READY pgs=1239 cs=0 l=1 rev1=1 crypto rx=0x7fa37c00e860 tx=0x7fa37c00ed20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-09T13:51:36.392 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:36.386+0000 7fa3767fc640 1 -- 172.21.15.43:0/2164548093 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa37c009600 con 0x7fa36c0047c0 2023-08-09T13:51:36.392 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:36.386+0000 7fa3767fc640 1 -- 172.21.15.43:0/2164548093 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa37c00c940 con 0x7fa36c0047c0 2023-08-09T13:51:36.392 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:36.386+0000 7fa3767fc640 1 -- 172.21.15.43:0/2164548093 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa37c00ccd0 con 0x7fa36c0047c0 2023-08-09T13:51:36.393 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:36.386+0000 7fa386b0f640 1 -- 172.21.15.43:0/2164548093 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa3800c4cf0 con 0x7fa36c0047c0 2023-08-09T13:51:36.393 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:36.387+0000 7fa386b0f640 1 -- 172.21.15.43:0/2164548093 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa3800c52b0 con 0x7fa36c0047c0 2023-08-09T13:51:36.393 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:36.388+0000 7fa386b0f640 1 -- 172.21.15.43:0/2164548093 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa38004e390 con 0x7fa36c0047c0 2023-08-09T13:51:36.394 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:36.389+0000 7fa3767fc640 1 -- 172.21.15.43:0/2164548093 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 41) v1 ==== 88444+0+0 (secure 0 0 0) 0x7fa37c013470 con 0x7fa36c0047c0 2023-08-09T13:51:36.394 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:36.389+0000 7fa3767fc640 1 --2- 172.21.15.43:0/2164548093 >> [v2:172.21.15.203:6832/2675832148,v1:172.21.15.203:6833/2675832148] conn(0x7fa36406a170 0x7fa36406c630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:51:36.394 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:36.390+0000 7fa3767fc640 1 -- 172.21.15.43:0/2164548093 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(104..104 src has 1..104) v4 ==== 6770+0+0 (secure 0 0 0) 0x7fa37c088cc0 con 0x7fa36c0047c0 2023-08-09T13:51:36.394 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:36.390+0000 7fa38530c640 1 -- 172.21.15.43:0/2164548093 >> [v2:172.21.15.203:6832/2675832148,v1:172.21.15.203:6833/2675832148] conn(0x7fa36406a170 msgr2=0x7fa36406c630 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.203:6832/2675832148 2023-08-09T13:51:36.394 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:36.390+0000 7fa38530c640 1 --2- 172.21.15.43:0/2164548093 >> [v2:172.21.15.203:6832/2675832148,v1:172.21.15.203:6833/2675832148] conn(0x7fa36406a170 0x7fa36406c630 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-08-09T13:51:36.396 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:36.393+0000 7fa3767fc640 1 -- 172.21.15.43:0/2164548093 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178066 (secure 0 0 0) 0x7fa37c08f080 con 0x7fa36c0047c0 2023-08-09T13:51:36.430 INFO:tasks.ceph.mgr.x.smithi203.stderr:2023-08-09T13:51:36.427+0000 7f67a7a3b040 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2023-08-09T13:51:36.492 INFO:tasks.ceph.mgr.y.smithi043.stderr:2023-08-09T13:51:36.489+0000 7fc4e782c040 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2023-08-09T13:51:36.557 INFO:tasks.ceph.mgr.y.smithi043.stderr:2023-08-09T13:51:36.555+0000 7fc4e782c040 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2023-08-09T13:51:36.558 INFO:tasks.ceph.mgr.x.smithi203.stderr:2023-08-09T13:51:36.555+0000 7f67a7a3b040 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2023-08-09T13:51:36.587 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:36.584+0000 7fa386b0f640 1 -- 172.21.15.43:0/2164548093 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7fa38006a580 con 0x7fa36c0047c0 2023-08-09T13:51:36.588 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:36.585+0000 7fa3767fc640 1 -- 172.21.15.43:0/2164548093 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v104) v1 ==== 104+0+11 (secure 0 0 0) 0x7fa37c053f30 con 0x7fa36c0047c0 2023-08-09T13:51:36.588 INFO:teuthology.orchestra.run.smithi043.stdout:pg_num: 32 2023-08-09T13:51:36.591 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:36.588+0000 7fa386b0f640 1 -- 172.21.15.43:0/2164548093 >> [v2:172.21.15.203:6832/2675832148,v1:172.21.15.203:6833/2675832148] conn(0x7fa36406a170 msgr2=0x7fa36406c630 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-08-09T13:51:36.591 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:36.588+0000 7fa386b0f640 1 --2- 172.21.15.43:0/2164548093 >> [v2:172.21.15.203:6832/2675832148,v1:172.21.15.203:6833/2675832148] conn(0x7fa36406a170 0x7fa36406c630 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:51:36.591 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:36.588+0000 7fa386b0f640 1 -- 172.21.15.43:0/2164548093 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa36c0047c0 msgr2=0x7fa38014c6f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:51:36.592 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:36.588+0000 7fa386b0f640 1 --2- 172.21.15.43:0/2164548093 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa36c0047c0 0x7fa38014c6f0 secure :-1 s=READY pgs=1239 cs=0 l=1 rev1=1 crypto rx=0x7fa37c00e860 tx=0x7fa37c00ed20 comp rx=0 tx=0).stop 2023-08-09T13:51:36.592 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:36.588+0000 7fa386b0f640 1 -- 172.21.15.43:0/2164548093 shutdown_connections 2023-08-09T13:51:36.592 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:36.588+0000 7fa386b0f640 1 --2- 172.21.15.43:0/2164548093 >> [v2:172.21.15.203:6832/2675832148,v1:172.21.15.203:6833/2675832148] conn(0x7fa36406a170 0x7fa36406c630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:51:36.593 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:36.588+0000 7fa386b0f640 1 --2- 172.21.15.43:0/2164548093 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fa380157e00 0x7fa380158280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:51:36.593 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:36.588+0000 7fa386b0f640 1 --2- 172.21.15.43:0/2164548093 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fa3800c6010 0x7fa3800c6470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:51:36.593 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:36.588+0000 7fa386b0f640 1 --2- 172.21.15.43:0/2164548093 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa36c0047c0 0x7fa38014c6f0 unknown :-1 s=CLOSED pgs=1239 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:51:36.593 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:36.588+0000 7fa386b0f640 1 -- 172.21.15.43:0/2164548093 >> 172.21.15.43:0/2164548093 conn(0x7fa3800cef40 msgr2=0x7fa380070570 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-09T13:51:36.594 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:36.588+0000 7fa386b0f640 1 -- 172.21.15.43:0/2164548093 shutdown_connections 2023-08-09T13:51:36.594 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:36.588+0000 7fa386b0f640 1 -- 172.21.15.43:0/2164548093 wait complete. 2023-08-09T13:51:36.609 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-08-09T13:51:36.615 INFO:tasks.ceph.mgr.y.smithi043.stderr:2023-08-09T13:51:36.610+0000 7fc4e782c040 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2023-08-09T13:51:36.713 INFO:tasks.ceph.mgr.x.smithi203.stderr:2023-08-09T13:51:36.710+0000 7f67a7a3b040 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2023-08-09T13:51:36.738 INFO:tasks.ceph.mgr.y.smithi043.stderr:2023-08-09T13:51:36.735+0000 7fc4e782c040 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2023-08-09T13:51:36.817 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:36.814+0000 7f959886b640 1 Processor -- start 2023-08-09T13:51:36.817 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:36.815+0000 7f959886b640 1 -- start start 2023-08-09T13:51:36.819 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:36.816+0000 7f959886b640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f95940b21e0 0x7f95941672b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:51:36.819 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:36.816+0000 7f959886b640 1 -- --> v1:172.21.15.43:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f959406a8d0 con 0x7f95941601f0 2023-08-09T13:51:36.820 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:36.816+0000 7f959886b640 1 -- --> v1:172.21.15.203:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9594160e20 con 0x7f95941605d0 2023-08-09T13:51:36.820 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:36.816+0000 7f959886b640 1 -- --> v2:172.21.15.43:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f95940cdcc0 con 0x7f95940b21e0 2023-08-09T13:51:36.820 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:36.817+0000 7f9592d9d640 1 --1- >> v1:172.21.15.43:6789/0 conn(0x7f95941601f0 0x7f95940cf900 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.43:6789/0 says I am v1:172.21.15.43:60770/0 (socket says 172.21.15.43:60770) 2023-08-09T13:51:36.820 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:36.817+0000 7f9592d9d640 1 -- 172.21.15.43:0/1732254469 learned_addr learned my addr 172.21.15.43:0/1732254469 (peer_addr_for_me v1:172.21.15.43:0/0) 2023-08-09T13:51:36.820 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:36.817+0000 7f959359e640 1 --2- 172.21.15.43:0/1732254469 >> v2:172.21.15.43:3300/0 conn(0x7f95940b21e0 0x7f95941672b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:51:36.821 INFO:tasks.ceph.mgr.x.smithi203.stderr:2023-08-09T13:51:36.817+0000 7f67a7a3b040 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2023-08-09T13:51:36.821 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:36.817+0000 7f959359e640 1 -- 172.21.15.43:0/1732254469 >> v1:172.21.15.203:6789/0 conn(0x7f95941605d0 legacy=0x7f95940b1ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:51:36.822 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:36.817+0000 7f959359e640 1 -- 172.21.15.43:0/1732254469 >> v1:172.21.15.43:6789/0 conn(0x7f95941601f0 legacy=0x7f95940cf900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:51:36.822 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:36.817+0000 7f9591d9b640 1 -- 172.21.15.43:0/1732254469 <== mon.0 v1:172.21.15.43:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1332009200 0 0) 0x7f959406a8d0 con 0x7f95941601f0 2023-08-09T13:51:36.822 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:36.817+0000 7f959359e640 1 -- 172.21.15.43:0/1732254469 --> v2:172.21.15.43:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f95940cdfa0 con 0x7f95940b21e0 2023-08-09T13:51:36.822 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:36.819+0000 7f959359e640 1 --2- 172.21.15.43:0/1732254469 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f95940b21e0 0x7f95941672b0 secure :-1 s=READY pgs=1241 cs=0 l=1 rev1=1 crypto rx=0x7f958c009d90 tx=0x7f958c02bf50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=556a5c189de086cb server_cookie=0 in_seq=0 out_seq=0 2023-08-09T13:51:36.822 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:36.819+0000 7f9591d9b640 1 -- 172.21.15.43:0/1732254469 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f958c02cd00 con 0x7f95940b21e0 2023-08-09T13:51:36.822 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:36.819+0000 7f9591d9b640 1 -- 172.21.15.43:0/1732254469 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f958c02ce60 con 0x7f95940b21e0 2023-08-09T13:51:36.822 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:36.819+0000 7f9591d9b640 1 -- 172.21.15.43:0/1732254469 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f958c034660 con 0x7f95940b21e0 2023-08-09T13:51:36.823 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:36.820+0000 7f959886b640 1 -- 172.21.15.43:0/1732254469 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f95940b21e0 msgr2=0x7f95941672b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:51:36.823 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:36.820+0000 7f959886b640 1 --2- 172.21.15.43:0/1732254469 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f95940b21e0 0x7f95941672b0 secure :-1 s=READY pgs=1241 cs=0 l=1 rev1=1 crypto rx=0x7f958c009d90 tx=0x7f958c02bf50 comp rx=0 tx=0).stop 2023-08-09T13:51:36.823 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:36.820+0000 7f959886b640 1 -- 172.21.15.43:0/1732254469 shutdown_connections 2023-08-09T13:51:36.823 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:36.820+0000 7f959886b640 1 --2- 172.21.15.43:0/1732254469 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f95940b21e0 0x7f95941672b0 unknown :-1 s=CLOSED pgs=1241 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:51:36.824 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:36.820+0000 7f959886b640 1 -- 172.21.15.43:0/1732254469 >> 172.21.15.43:0/1732254469 conn(0x7f95940ccad0 msgr2=0x7f95940ccee0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-09T13:51:36.824 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:36.820+0000 7f959886b640 1 -- 172.21.15.43:0/1732254469 shutdown_connections 2023-08-09T13:51:36.824 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:36.820+0000 7f959886b640 1 -- 172.21.15.43:0/1732254469 wait complete. 2023-08-09T13:51:36.824 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:36.822+0000 7f959886b640 1 Processor -- start 2023-08-09T13:51:36.825 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:36.822+0000 7f959886b640 1 -- start start 2023-08-09T13:51:36.825 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:36.823+0000 7f959886b640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f95940b21e0 0x7f95941f4bc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:51:36.826 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:36.823+0000 7f959886b640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f95941601f0 0x7f95940c3b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:51:36.826 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:36.823+0000 7f959886b640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f95941605d0 0x7f95940c40c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:51:36.826 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:36.823+0000 7f959886b640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f95940c5300 con 0x7f95941605d0 2023-08-09T13:51:36.826 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:36.823+0000 7f959886b640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f95940c46f0 con 0x7f95940b21e0 2023-08-09T13:51:36.826 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:36.823+0000 7f959886b640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f95940c49d0 con 0x7f95941601f0 2023-08-09T13:51:36.826 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:36.823+0000 7f959359e640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f95941605d0 0x7f95940c40c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:51:36.826 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:36.823+0000 7f959359e640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f95941605d0 0x7f95940c40c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:41296/0 (socket says 172.21.15.43:41296) 2023-08-09T13:51:36.827 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:36.823+0000 7f959359e640 1 -- 172.21.15.43:0/3752535924 learned_addr learned my addr 172.21.15.43:0/3752535924 (peer_addr_for_me v2:172.21.15.43:0/0) 2023-08-09T13:51:36.827 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:36.823+0000 7f9592d9d640 1 --2- 172.21.15.43:0/3752535924 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f95940b21e0 0x7f95941f4bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:51:36.827 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:36.823+0000 7f959359e640 1 -- 172.21.15.43:0/3752535924 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f95941601f0 msgr2=0x7f95940c3b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:51:36.827 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:36.823+0000 7f959359e640 1 --2- 172.21.15.43:0/3752535924 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f95941601f0 0x7f95940c3b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:51:36.827 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:36.823+0000 7f959359e640 1 -- 172.21.15.43:0/3752535924 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f95940b21e0 msgr2=0x7f95941f4bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:51:36.827 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:36.823+0000 7f959359e640 1 --2- 172.21.15.43:0/3752535924 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f95940b21e0 0x7f95941f4bc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:51:36.827 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:36.823+0000 7f959359e640 1 -- 172.21.15.43:0/3752535924 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f958c009a70 con 0x7f95941605d0 2023-08-09T13:51:36.828 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:36.824+0000 7f959359e640 1 --2- 172.21.15.43:0/3752535924 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f95941605d0 0x7f95940c40c0 secure :-1 s=READY pgs=1242 cs=0 l=1 rev1=1 crypto rx=0x7f958c009d90 tx=0x7f958c004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-09T13:51:36.828 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:36.824+0000 7f957b7fe640 1 -- 172.21.15.43:0/3752535924 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f958c003b90 con 0x7f95941605d0 2023-08-09T13:51:36.828 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:36.824+0000 7f957b7fe640 1 -- 172.21.15.43:0/3752535924 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f958c003cf0 con 0x7f95941605d0 2023-08-09T13:51:36.829 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:36.824+0000 7f957b7fe640 1 -- 172.21.15.43:0/3752535924 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f958c004230 con 0x7f95941605d0 2023-08-09T13:51:36.829 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:36.824+0000 7f959886b640 1 -- 172.21.15.43:0/3752535924 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f95940c4cb0 con 0x7f95941605d0 2023-08-09T13:51:36.829 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:36.824+0000 7f959886b640 1 -- 172.21.15.43:0/3752535924 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f95940b59b0 con 0x7f95941605d0 2023-08-09T13:51:36.829 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:36.825+0000 7f959886b640 1 -- 172.21.15.43:0/3752535924 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f95940c5470 con 0x7f95941605d0 2023-08-09T13:51:36.830 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:36.826+0000 7f957b7fe640 1 -- 172.21.15.43:0/3752535924 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 41) v1 ==== 88444+0+0 (secure 0 0 0) 0x7f958c03b050 con 0x7f95941605d0 2023-08-09T13:51:36.830 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:36.827+0000 7f957b7fe640 1 --2- 172.21.15.43:0/3752535924 >> [v2:172.21.15.203:6832/2675832148,v1:172.21.15.203:6833/2675832148] conn(0x7f958006a1a0 0x7f958006c660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:51:36.830 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:36.827+0000 7f957b7fe640 1 -- 172.21.15.43:0/3752535924 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(104..104 src has 1..104) v4 ==== 6770+0+0 (secure 0 0 0) 0x7f958c0ab3d0 con 0x7f95941605d0 2023-08-09T13:51:36.830 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:36.827+0000 7f9592d9d640 1 -- 172.21.15.43:0/3752535924 >> [v2:172.21.15.203:6832/2675832148,v1:172.21.15.203:6833/2675832148] conn(0x7f958006a1a0 msgr2=0x7f958006c660 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.203:6832/2675832148 2023-08-09T13:51:36.830 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:36.828+0000 7f9592d9d640 1 --2- 172.21.15.43:0/3752535924 >> [v2:172.21.15.203:6832/2675832148,v1:172.21.15.203:6833/2675832148] conn(0x7f958006a1a0 0x7f958006c660 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-08-09T13:51:36.837 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:36.835+0000 7f957b7fe640 1 -- 172.21.15.43:0/3752535924 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178066 (secure 0 0 0) 0x7f958c076640 con 0x7f95941605d0 2023-08-09T13:51:36.870 INFO:tasks.ceph.mgr.x.smithi203.stderr:2023-08-09T13:51:36.868+0000 7f67a7a3b040 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2023-08-09T13:51:36.909 INFO:tasks.ceph.mgr.y.smithi043.stderr:2023-08-09T13:51:36.906+0000 7fc4e782c040 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2023-08-09T13:51:37.022 INFO:tasks.ceph.mgr.y.smithi043.stderr:2023-08-09T13:51:37.019+0000 7fc4e782c040 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2023-08-09T13:51:37.029 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:37.027+0000 7f959886b640 1 -- 172.21.15.43:0/3752535924 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f959404e390 con 0x7f95941605d0 2023-08-09T13:51:37.030 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:37.027+0000 7f957b7fe640 1 -- 172.21.15.43:0/3752535924 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v104) v1 ==== 100+0+11 (secure 0 0 0) 0x7f958c0390a0 con 0x7f95941605d0 2023-08-09T13:51:37.030 INFO:teuthology.orchestra.run.smithi043.stdout:pg_num: 64 2023-08-09T13:51:37.030 INFO:tasks.ceph.mgr.x.smithi203.stderr:2023-08-09T13:51:37.027+0000 7f67a7a3b040 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2023-08-09T13:51:37.031 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:37.029+0000 7f9592d9d640 1 -- 172.21.15.43:0/3752535924 >> [v2:172.21.15.203:6832/2675832148,v1:172.21.15.203:6833/2675832148] conn(0x7f958006a1a0 msgr2=0x7f958006c660 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.203:6832/2675832148 2023-08-09T13:51:37.031 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:37.029+0000 7f9592d9d640 1 --2- 172.21.15.43:0/3752535924 >> [v2:172.21.15.203:6832/2675832148,v1:172.21.15.203:6833/2675832148] conn(0x7f958006a1a0 0x7f958006c660 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2023-08-09T13:51:37.032 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:37.030+0000 7f959886b640 1 -- 172.21.15.43:0/3752535924 >> [v2:172.21.15.203:6832/2675832148,v1:172.21.15.203:6833/2675832148] conn(0x7f958006a1a0 msgr2=0x7f958006c660 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-08-09T13:51:37.033 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:37.030+0000 7f959886b640 1 --2- 172.21.15.43:0/3752535924 >> [v2:172.21.15.203:6832/2675832148,v1:172.21.15.203:6833/2675832148] conn(0x7f958006a1a0 0x7f958006c660 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:51:37.033 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:37.030+0000 7f959886b640 1 -- 172.21.15.43:0/3752535924 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f95941605d0 msgr2=0x7f95940c40c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:51:37.033 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:37.030+0000 7f959886b640 1 --2- 172.21.15.43:0/3752535924 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f95941605d0 0x7f95940c40c0 secure :-1 s=READY pgs=1242 cs=0 l=1 rev1=1 crypto rx=0x7f958c009d90 tx=0x7f958c004a40 comp rx=0 tx=0).stop 2023-08-09T13:51:37.033 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:37.030+0000 7f959886b640 1 -- 172.21.15.43:0/3752535924 shutdown_connections 2023-08-09T13:51:37.033 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:37.030+0000 7f959886b640 1 --2- 172.21.15.43:0/3752535924 >> [v2:172.21.15.203:6832/2675832148,v1:172.21.15.203:6833/2675832148] conn(0x7f958006a1a0 0x7f958006c660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:51:37.033 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:37.030+0000 7f959886b640 1 --2- 172.21.15.43:0/3752535924 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f95941605d0 0x7f95940c40c0 unknown :-1 s=CLOSED pgs=1242 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:51:37.034 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:37.030+0000 7f959886b640 1 --2- 172.21.15.43:0/3752535924 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f95941601f0 0x7f95940c3b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:51:37.034 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:37.030+0000 7f959886b640 1 --2- 172.21.15.43:0/3752535924 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f95940b21e0 0x7f95941f4bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:51:37.034 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:37.030+0000 7f959886b640 1 -- 172.21.15.43:0/3752535924 >> 172.21.15.43:0/3752535924 conn(0x7f95940ccad0 msgr2=0x7f95940b1640 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-09T13:51:37.034 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:37.030+0000 7f959886b640 1 -- 172.21.15.43:0/3752535924 shutdown_connections 2023-08-09T13:51:37.034 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:37.031+0000 7f959886b640 1 -- 172.21.15.43:0/3752535924 wait complete. 2023-08-09T13:51:37.050 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2023-08-09T13:51:37.075 INFO:tasks.ceph.mgr.y.smithi043.stderr:2023-08-09T13:51:37.073+0000 7fc4e782c040 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2023-08-09T13:51:37.246 INFO:tasks.ceph.mgr.y.smithi043.stderr:2023-08-09T13:51:37.244+0000 7fc4e782c040 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2023-08-09T13:51:37.264 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:37.261+0000 7fcc784a9640 1 Processor -- start 2023-08-09T13:51:37.264 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:37.261+0000 7fcc784a9640 1 -- start start 2023-08-09T13:51:37.266 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:37.262+0000 7fcc784a9640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fcc7015a570 0x7fcc700ccc30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:51:37.266 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:37.262+0000 7fcc784a9640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7fcc700ceeb0 0x7fcc700cd170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:51:37.266 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:37.263+0000 7fcc784a9640 1 -- --> v1:172.21.15.203:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fcc700acfd0 con 0x7fcc700af070 2023-08-09T13:51:37.266 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:37.263+0000 7fcc784a9640 1 -- --> v2:172.21.15.43:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fcc700cb990 con 0x7fcc700ceeb0 2023-08-09T13:51:37.266 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:37.263+0000 7fcc784a9640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcc700cbc70 con 0x7fcc7015a570 2023-08-09T13:51:37.267 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:37.263+0000 7fcc76a46640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7fcc700ceeb0 0x7fcc700cd170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:51:37.267 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:37.263+0000 7fcc76a46640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7fcc700ceeb0 0x7fcc700cd170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:41310/0 (socket says 172.21.15.43:41310) 2023-08-09T13:51:37.267 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:37.263+0000 7fcc76a46640 1 -- 172.21.15.43:0/2696762183 learned_addr learned my addr 172.21.15.43:0/2696762183 (peer_addr_for_me v2:172.21.15.43:0/0) 2023-08-09T13:51:37.267 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:37.263+0000 7fcc76245640 1 --2- 172.21.15.43:0/2696762183 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fcc7015a570 0x7fcc700ccc30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:51:37.267 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:37.263+0000 7fcc76a46640 1 -- 172.21.15.43:0/2696762183 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fcc7015a570 msgr2=0x7fcc700ccc30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-09T13:51:37.267 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:37.263+0000 7fcc76a46640 1 --2- 172.21.15.43:0/2696762183 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fcc7015a570 0x7fcc700ccc30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:51:37.267 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:37.263+0000 7fcc76a46640 1 -- 172.21.15.43:0/2696762183 >> v1:172.21.15.203:6789/0 conn(0x7fcc700af070 legacy=0x7fcc700ad460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-09T13:51:37.268 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:37.263+0000 7fcc76a46640 1 -- 172.21.15.43:0/2696762183 --> v2:172.21.15.43:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcc700cbf50 con 0x7fcc700ceeb0 2023-08-09T13:51:37.268 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:37.263+0000 7fcc76245640 1 --2- 172.21.15.43:0/2696762183 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fcc7015a570 0x7fcc700ccc30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-08-09T13:51:37.268 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:37.265+0000 7fcc76a46640 1 --2- 172.21.15.43:0/2696762183 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fcc700ceeb0 0x7fcc700cd170 secure :-1 s=READY pgs=1243 cs=0 l=1 rev1=1 crypto rx=0x7fcc68009e50 tx=0x7fcc6802bee0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2684290c4335ac46 server_cookie=0 in_seq=0 out_seq=0 2023-08-09T13:51:37.269 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:37.266+0000 7fcc75243640 1 -- 172.21.15.43:0/2696762183 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcc6802ca80 con 0x7fcc700ceeb0 2023-08-09T13:51:37.269 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:37.266+0000 7fcc75243640 1 -- 172.21.15.43:0/2696762183 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcc6802cbe0 con 0x7fcc700ceeb0 2023-08-09T13:51:37.269 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:37.266+0000 7fcc75243640 1 -- 172.21.15.43:0/2696762183 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcc680343f0 con 0x7fcc700ceeb0 2023-08-09T13:51:37.269 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:37.266+0000 7fcc784a9640 1 -- 172.21.15.43:0/2696762183 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fcc700ceeb0 msgr2=0x7fcc700cd170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:51:37.269 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:37.266+0000 7fcc784a9640 1 --2- 172.21.15.43:0/2696762183 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fcc700ceeb0 0x7fcc700cd170 secure :-1 s=READY pgs=1243 cs=0 l=1 rev1=1 crypto rx=0x7fcc68009e50 tx=0x7fcc6802bee0 comp rx=0 tx=0).stop 2023-08-09T13:51:37.270 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:37.266+0000 7fcc784a9640 1 -- 172.21.15.43:0/2696762183 shutdown_connections 2023-08-09T13:51:37.270 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:37.266+0000 7fcc784a9640 1 --2- 172.21.15.43:0/2696762183 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fcc700ceeb0 0x7fcc700cd170 unknown :-1 s=CLOSED pgs=1243 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:51:37.270 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:37.266+0000 7fcc784a9640 1 --2- 172.21.15.43:0/2696762183 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fcc7015a570 0x7fcc700ccc30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:51:37.270 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:37.266+0000 7fcc784a9640 1 -- 172.21.15.43:0/2696762183 >> 172.21.15.43:0/2696762183 conn(0x7fcc70089fe0 msgr2=0x7fcc700d02d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-09T13:51:37.270 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:37.267+0000 7fcc784a9640 1 -- 172.21.15.43:0/2696762183 shutdown_connections 2023-08-09T13:51:37.271 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:37.267+0000 7fcc784a9640 1 -- 172.21.15.43:0/2696762183 wait complete. 2023-08-09T13:51:37.271 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:37.268+0000 7fcc784a9640 1 Processor -- start 2023-08-09T13:51:37.271 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:37.269+0000 7fcc784a9640 1 -- start start 2023-08-09T13:51:37.272 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:37.269+0000 7fcc784a9640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fcc700af070 0x7fcc701f14d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:51:37.272 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:37.270+0000 7fcc784a9640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fcc700ceeb0 0x7fcc700b3f10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:51:37.273 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:37.270+0000 7fcc784a9640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fcc7015a570 0x7fcc700b4450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:51:37.273 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:37.270+0000 7fcc784a9640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcc700c73e0 con 0x7fcc700af070 2023-08-09T13:51:37.273 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:37.270+0000 7fcc784a9640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcc7015ae20 con 0x7fcc700ceeb0 2023-08-09T13:51:37.273 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:37.270+0000 7fcc784a9640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcc7015dc70 con 0x7fcc7015a570 2023-08-09T13:51:37.273 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:37.270+0000 7fcc76245640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fcc700af070 0x7fcc701f14d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:51:37.274 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:37.270+0000 7fcc76245640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fcc700af070 0x7fcc701f14d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:41314/0 (socket says 172.21.15.43:41314) 2023-08-09T13:51:37.274 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:37.270+0000 7fcc76245640 1 -- 172.21.15.43:0/1793183916 learned_addr learned my addr 172.21.15.43:0/1793183916 (peer_addr_for_me v2:172.21.15.43:0/0) 2023-08-09T13:51:37.275 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:37.270+0000 7fcc75a44640 1 --2- 172.21.15.43:0/1793183916 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fcc700ceeb0 0x7fcc700b3f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:51:37.275 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:37.270+0000 7fcc76245640 1 -- 172.21.15.43:0/1793183916 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fcc7015a570 msgr2=0x7fcc700b4450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:51:37.275 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:37.270+0000 7fcc76245640 1 --2- 172.21.15.43:0/1793183916 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fcc7015a570 0x7fcc700b4450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:51:37.275 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:37.270+0000 7fcc76245640 1 -- 172.21.15.43:0/1793183916 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fcc700ceeb0 msgr2=0x7fcc700b3f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:51:37.275 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:37.270+0000 7fcc76245640 1 --2- 172.21.15.43:0/1793183916 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fcc700ceeb0 0x7fcc700b3f10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:51:37.276 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:37.270+0000 7fcc76245640 1 -- 172.21.15.43:0/1793183916 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcc68009b30 con 0x7fcc700af070 2023-08-09T13:51:37.276 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:37.271+0000 7fcc76245640 1 --2- 172.21.15.43:0/1793183916 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fcc700af070 0x7fcc701f14d0 secure :-1 s=READY pgs=1244 cs=0 l=1 rev1=1 crypto rx=0x7fcc6c00c7f0 tx=0x7fcc6c00cbf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-09T13:51:37.276 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:37.271+0000 7fcc66ffd640 1 -- 172.21.15.43:0/1793183916 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcc6c007a30 con 0x7fcc700af070 2023-08-09T13:51:37.276 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:37.271+0000 7fcc66ffd640 1 -- 172.21.15.43:0/1793183916 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcc6c007b90 con 0x7fcc700af070 2023-08-09T13:51:37.276 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:37.271+0000 7fcc66ffd640 1 -- 172.21.15.43:0/1793183916 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcc6c00ce40 con 0x7fcc700af070 2023-08-09T13:51:37.276 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:37.271+0000 7fcc784a9640 1 -- 172.21.15.43:0/1793183916 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcc700c7630 con 0x7fcc700af070 2023-08-09T13:51:37.276 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:37.271+0000 7fcc784a9640 1 -- 172.21.15.43:0/1793183916 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcc700c7bf0 con 0x7fcc700af070 2023-08-09T13:51:37.277 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:37.272+0000 7fcc784a9640 1 -- 172.21.15.43:0/1793183916 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcc70071a60 con 0x7fcc700af070 2023-08-09T13:51:37.277 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:37.273+0000 7fcc66ffd640 1 -- 172.21.15.43:0/1793183916 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 41) v1 ==== 88444+0+0 (secure 0 0 0) 0x7fcc6c0113f0 con 0x7fcc700af070 2023-08-09T13:51:37.277 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:37.274+0000 7fcc66ffd640 1 --2- 172.21.15.43:0/1793183916 >> [v2:172.21.15.203:6832/2675832148,v1:172.21.15.203:6833/2675832148] conn(0x7fcc5406a170 0x7fcc5406c630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:51:37.278 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:37.274+0000 7fcc66ffd640 1 -- 172.21.15.43:0/1793183916 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(104..104 src has 1..104) v4 ==== 6770+0+0 (secure 0 0 0) 0x7fcc6c088cd0 con 0x7fcc700af070 2023-08-09T13:51:37.278 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:37.274+0000 7fcc75a44640 1 -- 172.21.15.43:0/1793183916 >> [v2:172.21.15.203:6832/2675832148,v1:172.21.15.203:6833/2675832148] conn(0x7fcc5406a170 msgr2=0x7fcc5406c630 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.203:6832/2675832148 2023-08-09T13:51:37.278 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:37.274+0000 7fcc75a44640 1 --2- 172.21.15.43:0/1793183916 >> [v2:172.21.15.203:6832/2675832148,v1:172.21.15.203:6833/2675832148] conn(0x7fcc5406a170 0x7fcc5406c630 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-08-09T13:51:37.281 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:37.279+0000 7fcc66ffd640 1 -- 172.21.15.43:0/1793183916 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178066 (secure 0 0 0) 0x7fcc6c053e30 con 0x7fcc700af070 2023-08-09T13:51:37.412 INFO:tasks.ceph.mgr.x.smithi203.stderr:2023-08-09T13:51:37.410+0000 7f67a7a3b040 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2023-08-09T13:51:37.477 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:37.475+0000 7fcc75a44640 1 -- 172.21.15.43:0/1793183916 >> [v2:172.21.15.203:6832/2675832148,v1:172.21.15.203:6833/2675832148] conn(0x7fcc5406a170 msgr2=0x7fcc5406c630 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.203:6832/2675832148 2023-08-09T13:51:37.478 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:37.475+0000 7fcc75a44640 1 --2- 172.21.15.43:0/1793183916 >> [v2:172.21.15.203:6832/2675832148,v1:172.21.15.203:6833/2675832148] conn(0x7fcc5406a170 0x7fcc5406c630 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2023-08-09T13:51:37.486 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:37.484+0000 7fcc784a9640 1 -- 172.21.15.43:0/1793183916 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7fcc700739f0 con 0x7fcc700af070 2023-08-09T13:51:37.488 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:37.485+0000 7fcc66ffd640 1 -- 172.21.15.43:0/1793183916 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 8 entries v104) v1 ==== 80+0+477 (secure 0 0 0) 0x7fcc6c01f1f0 con 0x7fcc700af070 2023-08-09T13:51:37.488 INFO:teuthology.orchestra.run.smithi043.stdout:172.21.15.203:0/1343041460 2023-08-10T13:51:32.420350+0000 2023-08-09T13:51:37.488 INFO:teuthology.orchestra.run.smithi043.stdout:172.21.15.203:6833/1778614695 2023-08-10T13:51:32.420350+0000 2023-08-09T13:51:37.488 INFO:teuthology.orchestra.run.smithi043.stdout:172.21.15.203:0/3761332698 2023-08-10T13:51:32.420350+0000 2023-08-09T13:51:37.488 INFO:teuthology.orchestra.run.smithi043.stdout:172.21.15.203:6832/1778614695 2023-08-10T13:51:32.420350+0000 2023-08-09T13:51:37.489 INFO:teuthology.orchestra.run.smithi043.stdout:172.21.15.203:0/2054133158 2023-08-10T13:51:32.420350+0000 2023-08-09T13:51:37.489 INFO:teuthology.orchestra.run.smithi043.stdout:172.21.15.203:0/1496080019 2023-08-10T13:51:32.420350+0000 2023-08-09T13:51:37.489 INFO:teuthology.orchestra.run.smithi043.stdout:172.21.15.203:0/1894709586 2023-08-10T13:51:32.420350+0000 2023-08-09T13:51:37.489 INFO:teuthology.orchestra.run.smithi043.stdout:172.21.15.203:0/270998287 2023-08-10T13:51:32.420350+0000 2023-08-09T13:51:37.489 INFO:teuthology.orchestra.run.smithi043.stderr:listed 8 entries 2023-08-09T13:51:37.490 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:37.487+0000 7fcc784a9640 1 -- 172.21.15.43:0/1793183916 >> [v2:172.21.15.203:6832/2675832148,v1:172.21.15.203:6833/2675832148] conn(0x7fcc5406a170 msgr2=0x7fcc5406c630 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-08-09T13:51:37.490 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:37.488+0000 7fcc784a9640 1 --2- 172.21.15.43:0/1793183916 >> [v2:172.21.15.203:6832/2675832148,v1:172.21.15.203:6833/2675832148] conn(0x7fcc5406a170 0x7fcc5406c630 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:51:37.491 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:37.488+0000 7fcc784a9640 1 -- 172.21.15.43:0/1793183916 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fcc700af070 msgr2=0x7fcc701f14d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:51:37.491 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:37.488+0000 7fcc784a9640 1 --2- 172.21.15.43:0/1793183916 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fcc700af070 0x7fcc701f14d0 secure :-1 s=READY pgs=1244 cs=0 l=1 rev1=1 crypto rx=0x7fcc6c00c7f0 tx=0x7fcc6c00cbf0 comp rx=0 tx=0).stop 2023-08-09T13:51:37.491 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:37.488+0000 7fcc784a9640 1 -- 172.21.15.43:0/1793183916 shutdown_connections 2023-08-09T13:51:37.491 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:37.488+0000 7fcc784a9640 1 --2- 172.21.15.43:0/1793183916 >> [v2:172.21.15.203:6832/2675832148,v1:172.21.15.203:6833/2675832148] conn(0x7fcc5406a170 0x7fcc5406c630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:51:37.491 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:37.488+0000 7fcc784a9640 1 --2- 172.21.15.43:0/1793183916 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fcc7015a570 0x7fcc700b4450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:51:37.491 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:37.488+0000 7fcc784a9640 1 --2- 172.21.15.43:0/1793183916 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fcc700ceeb0 0x7fcc700b3f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:51:37.491 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:37.488+0000 7fcc784a9640 1 --2- 172.21.15.43:0/1793183916 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fcc700af070 0x7fcc701f14d0 unknown :-1 s=CLOSED pgs=1244 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:51:37.491 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:37.488+0000 7fcc784a9640 1 -- 172.21.15.43:0/1793183916 >> 172.21.15.43:0/1793183916 conn(0x7fcc70089fe0 msgr2=0x7fcc7006b0b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-09T13:51:37.492 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:37.488+0000 7fcc784a9640 1 -- 172.21.15.43:0/1793183916 shutdown_connections 2023-08-09T13:51:37.492 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:37.488+0000 7fcc784a9640 1 -- 172.21.15.43:0/1793183916 wait complete. 2023-08-09T13:51:37.507 INFO:tasks.cephfs.fuse_mount:Running fusermount -u on ubuntu@smithi043.front.sepia.ceph.com... 2023-08-09T13:51:37.507 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-09T13:51:37.508 DEBUG:teuthology.orchestra.run.smithi043:> sudo fusermount -u /home/ubuntu/cephtest/mnt.0 2023-08-09T13:51:37.635 INFO:tasks.ceph.mgr.x.smithi203.stderr:2023-08-09T13:51:37.632+0000 7f67a7a3b040 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2023-08-09T13:51:37.641 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi043.stderr:ceph-fuse[121697]: fuse finished with error 0 and tester_r 0 2023-08-09T13:51:37.646 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi043.front.sepia.ceph.com 2023-08-09T13:51:37.646 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-09T13:51:37.647 DEBUG:teuthology.orchestra.run.smithi043:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2023-08-09T13:51:37.662 INFO:tasks.ceph.mgr.y.smithi043.stderr:2023-08-09T13:51:37.660+0000 7fc4e782c040 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2023-08-09T13:51:37.692 INFO:tasks.ceph.mgr.x.smithi203.stderr:2023-08-09T13:51:37.690+0000 7f67a7a3b040 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2023-08-09T13:51:37.741 INFO:tasks.ceph.mgr.x.smithi203.stderr:2023-08-09T13:51:37.739+0000 7f67a7a3b040 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2023-08-09T13:51:37.793 INFO:tasks.ceph.mgr.x.smithi203.stderr:2023-08-09T13:51:37.790+0000 7f67a7a3b040 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2023-08-09T13:51:37.821 DEBUG:teuthology.orchestra.run.smithi203:> set -ex 2023-08-09T13:51:37.821 DEBUG:teuthology.orchestra.run.smithi203:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-09T13:51:37.876 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4053648,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497576k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=27,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248784k,nr_inodes=812196,mode=700,uid=1000,gid=1275 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497576k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2023-08-09T13:51:37.876 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi203.front.sepia.ceph.com 2023-08-09T13:51:37.877 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-09T13:51:37.877 DEBUG:teuthology.orchestra.run.smithi203:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.1) 2023-08-09T13:51:37.894 INFO:tasks.ceph.mgr.y.smithi043.stderr:2023-08-09T13:51:37.892+0000 7fc4e782c040 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2023-08-09T13:51:37.925 INFO:teuthology.orchestra.run.smithi203.stderr:rmdir: failed to remove '/home/ubuntu/cephtest/mnt.1': No such file or directory 2023-08-09T13:51:37.925 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-08-09T13:51:37.926 DEBUG:teuthology.orchestra.run.smithi043:> set -ex 2023-08-09T13:51:37.927 DEBUG:teuthology.orchestra.run.smithi043:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-09T13:51:37.954 INFO:tasks.ceph.mgr.y.smithi043.stderr:2023-08-09T13:51:37.952+0000 7fc4e782c040 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2023-08-09T13:51:37.984 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4054441,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6498844k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=27,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3249420k,nr_inodes=812355,mode=700,uid=1000,gid=1275 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6498844k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2023-08-09T13:51:37.985 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi043.front.sepia.ceph.com 2023-08-09T13:51:37.985 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-09T13:51:37.985 DEBUG:teuthology.orchestra.run.smithi043:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.2) 2023-08-09T13:51:38.008 INFO:tasks.ceph.mgr.y.smithi043.stderr:2023-08-09T13:51:38.005+0000 7fc4e782c040 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2023-08-09T13:51:38.064 INFO:tasks.ceph.mgr.y.smithi043.stderr:2023-08-09T13:51:38.061+0000 7fc4e782c040 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2023-08-09T13:51:38.065 INFO:teuthology.orchestra.run.smithi043.stderr:rmdir: failed to remove '/home/ubuntu/cephtest/mnt.2': No such file or directory 2023-08-09T13:51:38.066 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-08-09T13:51:38.067 DEBUG:teuthology.orchestra.run.smithi203:> set -ex 2023-08-09T13:51:38.067 DEBUG:teuthology.orchestra.run.smithi203:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-09T13:51:38.095 INFO:tasks.ceph.mgr.x.smithi203.stderr:2023-08-09T13:51:38.093+0000 7f67a7a3b040 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2023-08-09T13:51:38.122 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4053648,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497576k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=27,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248784k,nr_inodes=812196,mode=700,uid=1000,gid=1275 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497576k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2023-08-09T13:51:38.122 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi203.front.sepia.ceph.com 2023-08-09T13:51:38.123 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-09T13:51:38.123 DEBUG:teuthology.orchestra.run.smithi203:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.3) 2023-08-09T13:51:38.156 INFO:tasks.ceph.mgr.x.smithi203.stderr:2023-08-09T13:51:38.154+0000 7f67a7a3b040 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2023-08-09T13:51:38.207 INFO:teuthology.orchestra.run.smithi203.stderr:rmdir: failed to remove '/home/ubuntu/cephtest/mnt.3': No such file or directory 2023-08-09T13:51:38.207 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-08-09T13:51:38.208 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-08-09T13:51:38.211 INFO:tasks.ceph.mgr.x.smithi203.stderr:2023-08-09T13:51:38.209+0000 7f67a7a3b040 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2023-08-09T13:51:38.373 INFO:tasks.ceph.mgr.x.smithi203.stderr:2023-08-09T13:51:38.370+0000 7f67a7a3b040 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2023-08-09T13:51:38.377 INFO:tasks.ceph.mgr.y.smithi043.stderr:2023-08-09T13:51:38.375+0000 7fc4e782c040 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2023-08-09T13:51:38.421 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:38.418+0000 7f7464538640 1 Processor -- start 2023-08-09T13:51:38.421 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:38.419+0000 7f7464538640 1 -- start start 2023-08-09T13:51:38.422 INFO:tasks.ceph.mgr.x.smithi203.stderr:2023-08-09T13:51:38.420+0000 7f67a7a3b040 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2023-08-09T13:51:38.423 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:38.420+0000 7f7464538640 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7f745c0b2790 0x7f745c0d2ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:51:38.423 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:38.420+0000 7f7464538640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f745c0b3050 0x7f745c0b3f60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:51:38.423 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:38.420+0000 7f7464538640 1 -- --> v1:172.21.15.43:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f745c0ce090 con 0x7f745c0b2b70 2023-08-09T13:51:38.423 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:38.420+0000 7f7464538640 1 -- --> v2:172.21.15.203:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f745c0b3950 con 0x7f745c0b2790 2023-08-09T13:51:38.423 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:38.420+0000 7f7464538640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f745c0cd7f0 con 0x7f745c0b3050 2023-08-09T13:51:38.425 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:38.421+0000 7f7461ad3640 1 --1- >> v1:172.21.15.43:6789/0 conn(0x7f745c0b2b70 0x7f745c0b5330 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.43:6789/0 says I am v1:172.21.15.43:60786/0 (socket says 172.21.15.43:60786) 2023-08-09T13:51:38.425 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:38.421+0000 7f7461ad3640 1 -- 172.21.15.43:0/342008161 learned_addr learned my addr 172.21.15.43:0/342008161 (peer_addr_for_me v1:172.21.15.43:0/0) 2023-08-09T13:51:38.425 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:38.421+0000 7f74622d4640 1 --2- 172.21.15.43:0/342008161 >> v2:172.21.15.203:3300/0 conn(0x7f745c0b2790 0x7f745c0d2ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:51:38.425 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:38.421+0000 7f74612d2640 1 -- 172.21.15.43:0/342008161 <== mon.0 v1:172.21.15.43:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1491878104 0 0) 0x7f745c0ce090 con 0x7f745c0b2b70 2023-08-09T13:51:38.425 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:38.421+0000 7f74612d2640 1 -- 172.21.15.43:0/342008161 --> v1:172.21.15.43:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f7448003520 con 0x7f745c0b2b70 2023-08-09T13:51:38.426 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:38.421+0000 7f7462ad5640 1 --2- 172.21.15.43:0/342008161 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f745c0b3050 0x7f745c0b3f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:51:38.426 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:38.421+0000 7f74612d2640 1 -- 172.21.15.43:0/342008161 <== mon.0 v1:172.21.15.43:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2842634398 0 0) 0x7f7448003520 con 0x7f745c0b2b70 2023-08-09T13:51:38.426 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:38.422+0000 7f74612d2640 1 -- 172.21.15.43:0/342008161 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f745c0b3050 msgr2=0x7f745c0b3f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-09T13:51:38.426 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:38.422+0000 7f74612d2640 1 --2- 172.21.15.43:0/342008161 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f745c0b3050 0x7f745c0b3f60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:51:38.426 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:38.422+0000 7f74612d2640 1 -- 172.21.15.43:0/342008161 >> v2:172.21.15.203:3300/0 conn(0x7f745c0b2790 msgr2=0x7f745c0d2ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-09T13:51:38.426 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:38.422+0000 7f74612d2640 1 --2- 172.21.15.43:0/342008161 >> v2:172.21.15.203:3300/0 conn(0x7f745c0b2790 0x7f745c0d2ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:51:38.426 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:38.422+0000 7f74612d2640 1 -- 172.21.15.43:0/342008161 --> v1:172.21.15.43:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f745c0cdad0 con 0x7f745c0b2b70 2023-08-09T13:51:38.426 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:38.422+0000 7f74612d2640 1 -- 172.21.15.43:0/342008161 <== mon.0 v1:172.21.15.43:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1285150740 0 0) 0x7f744c001670 con 0x7f745c0b2b70 2023-08-09T13:51:38.427 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:38.422+0000 7f74612d2640 1 -- 172.21.15.43:0/342008161 >> v1:172.21.15.43:6789/0 conn(0x7f745c0b2b70 legacy=0x7f745c0b5330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:51:38.427 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:38.422+0000 7f74612d2640 1 --2- 172.21.15.43:0/342008161 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f74480039e0 0x7f7448003e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:51:38.427 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:38.422+0000 7f74612d2640 1 --2- 172.21.15.43:0/342008161 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f745c0b3050 0x7f74480047c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:51:38.427 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:38.422+0000 7f74612d2640 1 --2- 172.21.15.43:0/342008161 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f7448004d00 0x7f7448005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:51:38.427 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:38.422+0000 7f74612d2640 1 -- 172.21.15.43:0/342008161 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f745c0b3950 con 0x7f7448004d00 2023-08-09T13:51:38.427 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:38.422+0000 7f74612d2640 1 -- 172.21.15.43:0/342008161 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f745c0cd7f0 con 0x7f74480039e0 2023-08-09T13:51:38.427 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:38.422+0000 7f74612d2640 1 -- 172.21.15.43:0/342008161 --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f744800ba30 con 0x7f745c0b3050 2023-08-09T13:51:38.427 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:38.422+0000 7f7462ad5640 1 --2- 172.21.15.43:0/342008161 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f7448004d00 0x7f7448005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:51:38.428 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:38.422+0000 7f74622d4640 1 --2- 172.21.15.43:0/342008161 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f74480039e0 0x7f7448003e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:51:38.428 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:38.422+0000 7f7461ad3640 1 --2- 172.21.15.43:0/342008161 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f745c0b3050 0x7f74480047c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:51:38.428 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:38.423+0000 7f7462ad5640 1 -- 172.21.15.43:0/342008161 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f745c0b3050 msgr2=0x7f74480047c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-09T13:51:38.428 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:38.423+0000 7f7462ad5640 1 --2- 172.21.15.43:0/342008161 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f745c0b3050 0x7f74480047c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:51:38.428 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:38.423+0000 7f7462ad5640 1 -- 172.21.15.43:0/342008161 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f74480039e0 msgr2=0x7f7448003e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-09T13:51:38.428 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:38.423+0000 7f7462ad5640 1 --2- 172.21.15.43:0/342008161 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f74480039e0 0x7f7448003e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:51:38.428 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:38.423+0000 7f7462ad5640 1 -- 172.21.15.43:0/342008161 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7448003520 con 0x7f7448004d00 2023-08-09T13:51:38.429 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:38.424+0000 7f7462ad5640 1 --2- 172.21.15.43:0/342008161 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f7448004d00 0x7f7448005180 secure :-1 s=READY pgs=1246 cs=0 l=1 rev1=1 crypto rx=0x7f7454007360 tx=0x7f74480057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=abfc51c354c96a9e server_cookie=0 in_seq=0 out_seq=0 2023-08-09T13:51:38.430 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:38.424+0000 7f74612d2640 1 -- 172.21.15.43:0/342008161 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f745402ccd0 con 0x7f7448004d00 2023-08-09T13:51:38.430 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:38.424+0000 7f74612d2640 1 -- 172.21.15.43:0/342008161 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f745402ce30 con 0x7f7448004d00 2023-08-09T13:51:38.430 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:38.424+0000 7f74612d2640 1 -- 172.21.15.43:0/342008161 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7454034660 con 0x7f7448004d00 2023-08-09T13:51:38.430 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:38.425+0000 7f7464538640 1 -- 172.21.15.43:0/342008161 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f7448004d00 msgr2=0x7f7448005180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:51:38.430 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:38.425+0000 7f7464538640 1 --2- 172.21.15.43:0/342008161 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f7448004d00 0x7f7448005180 secure :-1 s=READY pgs=1246 cs=0 l=1 rev1=1 crypto rx=0x7f7454007360 tx=0x7f74480057f0 comp rx=0 tx=0).stop 2023-08-09T13:51:38.430 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:38.425+0000 7f7462ad5640 1 -- 172.21.15.43:0/342008161 reap_dead start 2023-08-09T13:51:38.431 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:38.425+0000 7f7464538640 1 -- 172.21.15.43:0/342008161 shutdown_connections 2023-08-09T13:51:38.431 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:38.425+0000 7f7464538640 1 -- 172.21.15.43:0/342008161 >> 172.21.15.43:0/342008161 conn(0x7f745c09b7b0 msgr2=0x7f745c0d24e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-09T13:51:38.431 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:38.425+0000 7f7464538640 1 -- 172.21.15.43:0/342008161 shutdown_connections 2023-08-09T13:51:38.431 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:38.425+0000 7f7464538640 1 -- 172.21.15.43:0/342008161 wait complete. 2023-08-09T13:51:38.431 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:38.427+0000 7f7464538640 1 Processor -- start 2023-08-09T13:51:38.431 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:38.427+0000 7f7464538640 1 -- start start 2023-08-09T13:51:38.433 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:38.428+0000 7f7464538640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f7448004d00 0x7f745c0b1570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:51:38.433 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:38.428+0000 7f7464538640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f745c0b1ab0 0x7f745c157690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:51:38.433 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:38.428+0000 7f7464538640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f745c157bd0 0x7f745c0c4c90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:51:38.433 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:38.428+0000 7f7464538640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f745c0c7fd0 con 0x7f745c157bd0 2023-08-09T13:51:38.433 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:38.428+0000 7f7464538640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f745c1f63b0 con 0x7f745c0b1ab0 2023-08-09T13:51:38.433 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:38.428+0000 7f7464538640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f745c158080 con 0x7f7448004d00 2023-08-09T13:51:38.433 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:38.428+0000 7f7462ad5640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f745c157bd0 0x7f745c0c4c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:51:38.434 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:38.428+0000 7f7462ad5640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f745c157bd0 0x7f745c0c4c90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:41344/0 (socket says 172.21.15.43:41344) 2023-08-09T13:51:38.434 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:38.428+0000 7f7462ad5640 1 -- 172.21.15.43:0/4144923639 learned_addr learned my addr 172.21.15.43:0/4144923639 (peer_addr_for_me v2:172.21.15.43:0/0) 2023-08-09T13:51:38.434 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:38.429+0000 7f7461ad3640 1 --2- 172.21.15.43:0/4144923639 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f745c0b1ab0 0x7f745c157690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:51:38.434 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:38.429+0000 7f74622d4640 1 --2- 172.21.15.43:0/4144923639 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f7448004d00 0x7f745c0b1570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:51:38.434 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:38.429+0000 7f7462ad5640 1 -- 172.21.15.43:0/4144923639 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f7448004d00 msgr2=0x7f745c0b1570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:51:38.434 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:38.429+0000 7f7462ad5640 1 --2- 172.21.15.43:0/4144923639 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f7448004d00 0x7f745c0b1570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:51:38.434 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:38.429+0000 7f7462ad5640 1 -- 172.21.15.43:0/4144923639 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f745c0b1ab0 msgr2=0x7f745c157690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:51:38.435 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:38.429+0000 7f7462ad5640 1 --2- 172.21.15.43:0/4144923639 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f745c0b1ab0 0x7f745c157690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:51:38.435 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:38.429+0000 7f7462ad5640 1 -- 172.21.15.43:0/4144923639 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f745c0c8fe0 con 0x7f745c157bd0 2023-08-09T13:51:38.435 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:38.429+0000 7f7462ad5640 1 --2- 172.21.15.43:0/4144923639 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f745c157bd0 0x7f745c0c4c90 secure :-1 s=READY pgs=1247 cs=0 l=1 rev1=1 crypto rx=0x7f7454037040 tx=0x7f7454004ab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-09T13:51:38.435 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:38.430+0000 7f7452ffd640 1 -- 172.21.15.43:0/4144923639 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f745403e600 con 0x7f745c157bd0 2023-08-09T13:51:38.435 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:38.430+0000 7f7452ffd640 1 -- 172.21.15.43:0/4144923639 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f745403e760 con 0x7f745c157bd0 2023-08-09T13:51:38.435 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:38.430+0000 7f7464538640 1 -- 172.21.15.43:0/4144923639 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f745c0c8220 con 0x7f745c157bd0 2023-08-09T13:51:38.435 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:38.430+0000 7f7452ffd640 1 -- 172.21.15.43:0/4144923639 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f745403eaf0 con 0x7f745c157bd0 2023-08-09T13:51:38.435 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:38.430+0000 7f7464538640 1 -- 172.21.15.43:0/4144923639 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f745c0c87e0 con 0x7f745c157bd0 2023-08-09T13:51:38.436 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:38.431+0000 7f7464538640 1 -- 172.21.15.43:0/4144923639 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f745c0739f0 con 0x7f745c157bd0 2023-08-09T13:51:38.436 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:38.432+0000 7f7452ffd640 1 -- 172.21.15.43:0/4144923639 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 41) v1 ==== 88444+0+0 (secure 0 0 0) 0x7f745403ec50 con 0x7f745c157bd0 2023-08-09T13:51:38.437 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:38.433+0000 7f7452ffd640 1 --2- 172.21.15.43:0/4144923639 >> [v2:172.21.15.203:6832/2675832148,v1:172.21.15.203:6833/2675832148] conn(0x7f744006a1a0 0x7f744006c660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:51:38.437 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:38.433+0000 7f7452ffd640 1 -- 172.21.15.43:0/4144923639 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(104..104 src has 1..104) v4 ==== 6770+0+0 (secure 0 0 0) 0x7f74540aafe0 con 0x7f745c157bd0 2023-08-09T13:51:38.437 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:38.433+0000 7f74622d4640 1 -- 172.21.15.43:0/4144923639 >> [v2:172.21.15.203:6832/2675832148,v1:172.21.15.203:6833/2675832148] conn(0x7f744006a1a0 msgr2=0x7f744006c660 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.203:6832/2675832148 2023-08-09T13:51:38.437 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:38.433+0000 7f74622d4640 1 --2- 172.21.15.43:0/4144923639 >> [v2:172.21.15.203:6832/2675832148,v1:172.21.15.203:6833/2675832148] conn(0x7f744006a1a0 0x7f744006c660 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-08-09T13:51:38.440 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:38.438+0000 7f7452ffd640 1 -- 172.21.15.43:0/4144923639 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178066 (secure 0 0 0) 0x7f7454076300 con 0x7f745c157bd0 2023-08-09T13:51:38.440 INFO:tasks.ceph.mgr.y.smithi043.stderr:2023-08-09T13:51:38.438+0000 7fc4e782c040 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2023-08-09T13:51:38.476 INFO:tasks.ceph.mgr.x.smithi203.stderr:2023-08-09T13:51:38.474+0000 7f67a7a3b040 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2023-08-09T13:51:38.497 INFO:tasks.ceph.mgr.y.smithi043.stderr:2023-08-09T13:51:38.494+0000 7fc4e782c040 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2023-08-09T13:51:38.527 INFO:tasks.ceph.mgr.x.smithi203.stderr:2023-08-09T13:51:38.525+0000 7f67a7a3b040 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2023-08-09T13:51:38.556 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:38.553+0000 7f7452ffd640 1 -- 172.21.15.43:0/4144923639 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mgrmap(e 42) v1 ==== 88052+0+0 (secure 0 0 0) 0x7f74540766d0 con 0x7f745c157bd0 2023-08-09T13:51:38.556 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:38.554+0000 7f7452ffd640 1 -- 172.21.15.43:0/4144923639 >> [v2:172.21.15.203:6832/2675832148,v1:172.21.15.203:6833/2675832148] conn(0x7f744006a1a0 msgr2=0x7f744006c660 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-08-09T13:51:38.557 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:38.554+0000 7f7452ffd640 1 --2- 172.21.15.43:0/4144923639 >> [v2:172.21.15.203:6832/2675832148,v1:172.21.15.203:6833/2675832148] conn(0x7f744006a1a0 0x7f744006c660 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:51:38.620 INFO:tasks.ceph.mgr.x.smithi203.stderr:2023-08-09T13:51:38.618+0000 7f672dd2e640 -1 client.0 error registering admin socket command: (17) File exists 2023-08-09T13:51:38.620 INFO:tasks.ceph.mgr.x.smithi203.stderr:2023-08-09T13:51:38.618+0000 7f672dd2e640 -1 client.0 error registering admin socket command: (17) File exists 2023-08-09T13:51:38.621 INFO:tasks.ceph.mgr.x.smithi203.stderr:2023-08-09T13:51:38.618+0000 7f672dd2e640 -1 client.0 error registering admin socket command: (17) File exists 2023-08-09T13:51:38.621 INFO:tasks.ceph.mgr.x.smithi203.stderr:2023-08-09T13:51:38.618+0000 7f672dd2e640 -1 client.0 error registering admin socket command: (17) File exists 2023-08-09T13:51:38.621 INFO:tasks.ceph.mgr.x.smithi203.stderr:2023-08-09T13:51:38.618+0000 7f672dd2e640 -1 client.0 error registering admin socket command: (17) File exists 2023-08-09T13:51:38.621 INFO:tasks.ceph.mgr.x.smithi203.stderr:2023-08-09T13:51:38.619+0000 7f6729525640 -1 client.0 error registering admin socket command: (17) File exists 2023-08-09T13:51:38.621 INFO:tasks.ceph.mgr.x.smithi203.stderr:2023-08-09T13:51:38.619+0000 7f6729525640 -1 client.0 error registering admin socket command: (17) File exists 2023-08-09T13:51:38.622 INFO:tasks.ceph.mgr.x.smithi203.stderr:2023-08-09T13:51:38.619+0000 7f6729525640 -1 client.0 error registering admin socket command: (17) File exists 2023-08-09T13:51:38.622 INFO:tasks.ceph.mgr.x.smithi203.stderr:2023-08-09T13:51:38.619+0000 7f6729525640 -1 client.0 error registering admin socket command: (17) File exists 2023-08-09T13:51:38.622 INFO:tasks.ceph.mgr.x.smithi203.stderr:2023-08-09T13:51:38.619+0000 7f6729525640 -1 client.0 error registering admin socket command: (17) File exists 2023-08-09T13:51:38.678 INFO:tasks.ceph.mgr.y.smithi043.stderr:2023-08-09T13:51:38.675+0000 7fc4e782c040 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2023-08-09T13:51:38.685 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:38.683+0000 7f7464538640 1 -- 172.21.15.43:0/4144923639 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f745c0d05a0 con 0x7f745c157bd0 2023-08-09T13:51:38.686 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:38.683+0000 7f7452ffd640 1 -- 172.21.15.43:0/4144923639 <== mon.0 v2:172.21.15.43:3300/0 8 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 43 v43) v1 ==== 94+0+4880 (secure 0 0 0) 0x7f7454040b80 con 0x7f745c157bd0 2023-08-09T13:51:38.686 INFO:teuthology.orchestra.run.smithi043.stdout: 2023-08-09T13:51:38.686 INFO:teuthology.orchestra.run.smithi043.stdout:{"epoch":43,"default_fscid":5,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5966,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.203:6835/4101044758","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6834","nonce":4101044758},{"type":"v1","addr":"172.21.15.203:6835","nonce":4101044758}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":39},{"gid":6460,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.203:6837/2974664437","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6836","nonce":2974664437},{"type":"v1","addr":"172.21.15.203:6837","nonce":2974664437}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":39},{"gid":7527,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.43:6833/795245144","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6832","nonce":795245144},{"type":"v1","addr":"172.21.15.43:6833","nonce":795245144}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":38}],"filesystems":[{"mdsmap":{"epoch":43,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-08-09T13:51:15.716741+0000","modified":"2023-08-09T13:51:19.249966+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":7542},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7542":{"gid":7542,"name":"a","rank":0,"incarnation":40,"state":"up:active","state_seq":3,"addr":"172.21.15.43:6835/3008795294","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6834","nonce":3008795294},{"type":"v1","addr":"172.21.15.43:6835","nonce":3008795294}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[13],"metadata_pool":12,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":5}]} 2023-08-09T13:51:38.687 INFO:teuthology.orchestra.run.smithi043.stderr:dumped fsmap epoch 43 2023-08-09T13:51:38.689 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:38.687+0000 7f7464538640 1 -- 172.21.15.43:0/4144923639 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f745c157bd0 msgr2=0x7f745c0c4c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:51:38.690 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:38.687+0000 7f7464538640 1 --2- 172.21.15.43:0/4144923639 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f745c157bd0 0x7f745c0c4c90 secure :-1 s=READY pgs=1247 cs=0 l=1 rev1=1 crypto rx=0x7f7454037040 tx=0x7f7454004ab0 comp rx=0 tx=0).stop 2023-08-09T13:51:38.690 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:38.687+0000 7f7464538640 1 -- 172.21.15.43:0/4144923639 shutdown_connections 2023-08-09T13:51:38.690 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:38.687+0000 7f7464538640 1 --2- 172.21.15.43:0/4144923639 >> [v2:172.21.15.203:6832/2675832148,v1:172.21.15.203:6833/2675832148] conn(0x7f744006a1a0 0x7f744006c660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:51:38.690 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:38.687+0000 7f7464538640 1 --2- 172.21.15.43:0/4144923639 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f745c157bd0 0x7f745c0c4c90 unknown :-1 s=CLOSED pgs=1247 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:51:38.690 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:38.687+0000 7f7464538640 1 --2- 172.21.15.43:0/4144923639 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f745c0b1ab0 0x7f745c157690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:51:38.691 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:38.687+0000 7f7464538640 1 --2- 172.21.15.43:0/4144923639 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f7448004d00 0x7f745c0b1570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:51:38.691 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:38.687+0000 7f7464538640 1 -- 172.21.15.43:0/4144923639 >> 172.21.15.43:0/4144923639 conn(0x7f745c09b7b0 msgr2=0x7f745c1558d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-09T13:51:38.691 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:38.687+0000 7f7464538640 1 -- 172.21.15.43:0/4144923639 shutdown_connections 2023-08-09T13:51:38.691 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:38.687+0000 7f7464538640 1 -- 172.21.15.43:0/4144923639 wait complete. 2023-08-09T13:51:38.716 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-08-09T13:51:38.739 INFO:tasks.ceph.mgr.y.smithi043.stderr:2023-08-09T13:51:38.736+0000 7fc4e782c040 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2023-08-09T13:51:38.795 INFO:tasks.ceph.mgr.y.smithi043.stderr:2023-08-09T13:51:38.793+0000 7fc4e782c040 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2023-08-09T13:51:38.854 INFO:tasks.ceph.mgr.y.smithi043.stderr:2023-08-09T13:51:38.846+0000 7fc4e782c040 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2023-08-09T13:51:38.924 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:38.922+0000 7fc5df59e640 1 Processor -- start 2023-08-09T13:51:38.924 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:38.922+0000 7fc5df59e640 1 -- start start 2023-08-09T13:51:38.926 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:38.923+0000 7fc5df59e640 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7fc5e015a2c0 0x7fc5e00b1e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:51:38.926 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:38.923+0000 7fc5df59e640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7fc5e0070b00 0x7fc5e00b23b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:51:38.926 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:38.923+0000 7fc5df59e640 1 -- --> v1:172.21.15.203:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc5e00714c0 con 0x7fc5e006c530 2023-08-09T13:51:38.926 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:38.923+0000 7fc5df59e640 1 -- --> v2:172.21.15.43:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc5e00c9b50 con 0x7fc5e0070b00 2023-08-09T13:51:38.926 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:38.923+0000 7fc5df59e640 1 -- --> v2:172.21.15.203:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc5e00b1480 con 0x7fc5e015a2c0 2023-08-09T13:51:38.927 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:38.923+0000 7fc5ded9d640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7fc5e0070b00 0x7fc5e00b23b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:51:38.927 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:38.924+0000 7fc5ded9d640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7fc5e0070b00 0x7fc5e00b23b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:41358/0 (socket says 172.21.15.43:41358) 2023-08-09T13:51:38.927 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:38.924+0000 7fc5ded9d640 1 -- 172.21.15.43:0/923864036 learned_addr learned my addr 172.21.15.43:0/923864036 (peer_addr_for_me v2:172.21.15.43:0/0) 2023-08-09T13:51:38.927 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:38.924+0000 7fc5d5d9b640 1 --2- 172.21.15.43:0/923864036 >> v2:172.21.15.203:3300/0 conn(0x7fc5e015a2c0 0x7fc5e00b1e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:51:38.928 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:38.924+0000 7fc5ded9d640 1 -- 172.21.15.43:0/923864036 >> v2:172.21.15.203:3300/0 conn(0x7fc5e015a2c0 msgr2=0x7fc5e00b1e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-09T13:51:38.928 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:38.924+0000 7fc5ded9d640 1 --2- 172.21.15.43:0/923864036 >> v2:172.21.15.203:3300/0 conn(0x7fc5e015a2c0 0x7fc5e00b1e70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:51:38.928 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:38.924+0000 7fc5ded9d640 1 -- 172.21.15.43:0/923864036 >> v1:172.21.15.203:6789/0 conn(0x7fc5e006c530 legacy=0x7fc5e006c910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-09T13:51:38.928 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:38.924+0000 7fc5ded9d640 1 -- 172.21.15.43:0/923864036 --> v2:172.21.15.43:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc5e00711e0 con 0x7fc5e0070b00 2023-08-09T13:51:38.928 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:38.925+0000 7fc5ded9d640 1 --2- 172.21.15.43:0/923864036 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc5e0070b00 0x7fc5e00b23b0 secure :-1 s=READY pgs=1248 cs=0 l=1 rev1=1 crypto rx=0x7fc5e00b1c30 tx=0x7fc5d802bee0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ee7597a21a998190 server_cookie=0 in_seq=0 out_seq=0 2023-08-09T13:51:38.929 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:38.926+0000 7fc5ddd9b640 1 -- 172.21.15.43:0/923864036 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc5d802ca80 con 0x7fc5e0070b00 2023-08-09T13:51:38.929 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:38.926+0000 7fc5ddd9b640 1 -- 172.21.15.43:0/923864036 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc5d802cbe0 con 0x7fc5e0070b00 2023-08-09T13:51:38.929 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:38.926+0000 7fc5ddd9b640 1 -- 172.21.15.43:0/923864036 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc5d80343f0 con 0x7fc5e0070b00 2023-08-09T13:51:38.930 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:38.926+0000 7fc5df59e640 1 -- 172.21.15.43:0/923864036 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc5e0070b00 msgr2=0x7fc5e00b23b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:51:38.930 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:38.926+0000 7fc5df59e640 1 --2- 172.21.15.43:0/923864036 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc5e0070b00 0x7fc5e00b23b0 secure :-1 s=READY pgs=1248 cs=0 l=1 rev1=1 crypto rx=0x7fc5e00b1c30 tx=0x7fc5d802bee0 comp rx=0 tx=0).stop 2023-08-09T13:51:38.930 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:38.926+0000 7fc5df59e640 1 -- 172.21.15.43:0/923864036 shutdown_connections 2023-08-09T13:51:38.930 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:38.926+0000 7fc5df59e640 1 --2- 172.21.15.43:0/923864036 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc5e0070b00 0x7fc5e00b23b0 unknown :-1 s=CLOSED pgs=1248 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:51:38.930 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:38.926+0000 7fc5df59e640 1 --2- 172.21.15.43:0/923864036 >> v2:172.21.15.203:3300/0 conn(0x7fc5e015a2c0 0x7fc5e00b1e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:51:38.930 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:38.926+0000 7fc5df59e640 1 -- 172.21.15.43:0/923864036 >> 172.21.15.43:0/923864036 conn(0x7fc5e006a8d0 msgr2=0x7fc5e006acb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-09T13:51:38.930 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:38.927+0000 7fc5df59e640 1 -- 172.21.15.43:0/923864036 shutdown_connections 2023-08-09T13:51:38.931 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:38.927+0000 7fc5df59e640 1 -- 172.21.15.43:0/923864036 wait complete. 2023-08-09T13:51:38.931 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:38.928+0000 7fc5df59e640 1 Processor -- start 2023-08-09T13:51:38.931 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:38.929+0000 7fc5df59e640 1 -- start start 2023-08-09T13:51:38.932 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:38.930+0000 7fc5df59e640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fc5e006c530 0x7fc5e01549b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:51:38.932 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:38.930+0000 7fc5df59e640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc5e0070b00 0x7fc5e015b0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:51:38.933 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:38.930+0000 7fc5df59e640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fc5e015c930 0x7fc5e00ab8d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:51:38.933 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:38.930+0000 7fc5df59e640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc5e015cdc0 con 0x7fc5e0070b00 2023-08-09T13:51:38.933 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:38.930+0000 7fc5df59e640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc5e015b7c0 con 0x7fc5e015c930 2023-08-09T13:51:38.933 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:38.930+0000 7fc5df59e640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc5e015baa0 con 0x7fc5e006c530 2023-08-09T13:51:38.933 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:38.930+0000 7fc5d5d9b640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc5e0070b00 0x7fc5e015b0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:51:38.933 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:38.930+0000 7fc5d5d9b640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc5e0070b00 0x7fc5e015b0c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:41368/0 (socket says 172.21.15.43:41368) 2023-08-09T13:51:38.933 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:38.930+0000 7fc5d5d9b640 1 -- 172.21.15.43:0/1735504299 learned_addr learned my addr 172.21.15.43:0/1735504299 (peer_addr_for_me v2:172.21.15.43:0/0) 2023-08-09T13:51:38.934 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:38.930+0000 7fc5de59c640 1 --2- 172.21.15.43:0/1735504299 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fc5e006c530 0x7fc5e01549b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:51:38.935 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:38.930+0000 7fc5d5d9b640 1 -- 172.21.15.43:0/1735504299 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fc5e006c530 msgr2=0x7fc5e01549b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:51:38.935 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:38.930+0000 7fc5d5d9b640 1 --2- 172.21.15.43:0/1735504299 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fc5e006c530 0x7fc5e01549b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:51:38.935 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:38.930+0000 7fc5d5d9b640 1 -- 172.21.15.43:0/1735504299 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fc5e015c930 msgr2=0x7fc5e00ab8d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:51:38.935 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:38.930+0000 7fc5d5d9b640 1 --2- 172.21.15.43:0/1735504299 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fc5e015c930 0x7fc5e00ab8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:51:38.935 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:38.930+0000 7fc5d5d9b640 1 -- 172.21.15.43:0/1735504299 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc5d8009b30 con 0x7fc5e0070b00 2023-08-09T13:51:38.936 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:38.931+0000 7fc5d5d9b640 1 --2- 172.21.15.43:0/1735504299 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc5e0070b00 0x7fc5e015b0c0 secure :-1 s=READY pgs=1249 cs=0 l=1 rev1=1 crypto rx=0x7fc5c800c7f0 tx=0x7fc5c800ccb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-09T13:51:38.936 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:38.931+0000 7fc5d77fe640 1 -- 172.21.15.43:0/1735504299 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc5c8004220 con 0x7fc5e0070b00 2023-08-09T13:51:38.936 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:38.931+0000 7fc5d77fe640 1 -- 172.21.15.43:0/1735504299 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc5c8004380 con 0x7fc5e0070b00 2023-08-09T13:51:38.936 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:38.931+0000 7fc5d77fe640 1 -- 172.21.15.43:0/1735504299 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc5c8007840 con 0x7fc5e0070b00 2023-08-09T13:51:38.937 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:38.931+0000 7fc5df59e640 1 -- 172.21.15.43:0/1735504299 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc5e015bd80 con 0x7fc5e0070b00 2023-08-09T13:51:38.937 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:38.931+0000 7fc5df59e640 1 -- 172.21.15.43:0/1735504299 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc5e0152660 con 0x7fc5e0070b00 2023-08-09T13:51:38.937 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:38.933+0000 7fc5df59e640 1 -- 172.21.15.43:0/1735504299 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc5e0152910 con 0x7fc5e0070b00 2023-08-09T13:51:38.937 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:38.933+0000 7fc5d77fe640 1 -- 172.21.15.43:0/1735504299 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 42) v1 ==== 88052+0+0 (secure 0 0 0) 0x7fc5c8007bb0 con 0x7fc5e0070b00 2023-08-09T13:51:38.938 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:38.934+0000 7fc5d77fe640 1 -- 172.21.15.43:0/1735504299 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(105..105 src has 1..105) v4 ==== 7114+0+0 (secure 0 0 0) 0x7fc5c8088420 con 0x7fc5e0070b00 2023-08-09T13:51:38.941 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:38.938+0000 7fc5d77fe640 1 -- 172.21.15.43:0/1735504299 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178066 (secure 0 0 0) 0x7fc5e0152910 con 0x7fc5e0070b00 2023-08-09T13:51:39.135 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:39.133+0000 7fc5df59e640 1 -- 172.21.15.43:0/1735504299 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fc5e0152910 con 0x7fc5e0070b00 2023-08-09T13:51:39.138 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:39.135+0000 7fc5d77fe640 1 -- 172.21.15.43:0/1735504299 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v105) v1 ==== 74+0+19573 (secure 0 0 0) 0x7fc5e0152910 con 0x7fc5e0070b00 2023-08-09T13:51:39.138 INFO:teuthology.orchestra.run.smithi043.stdout: 2023-08-09T13:51:39.138 INFO:teuthology.orchestra.run.smithi043.stdout:{"epoch":105,"fsid":"daac87ee-1337-4250-a955-aea4e408a39b","created":"2023-08-09T13:33:41.701198+0000","modified":"2023-08-09T13:51:38.531070+0000","last_up_change":"2023-08-09T13:33:53.783453+0000","last_in_change":"2023-08-09T13:33:45.148904+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":13,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":"rbd","create_time":"2023-08-09T13:33:54.365137+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":".mgr","create_time":"2023-08-09T13:33:54.831803+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":12,"pool_name":"cephfs_metadata","create_time":"2023-08-09T13:51:14.142563+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"102","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":13,"pool_name":"cephfs_data","create_time":"2023-08-09T13:51:14.712668+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"103","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"cbd49ecd-e896-4fbd-864e-5c37a829b165","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":101,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6800","nonce":502667335},{"type":"v1","addr":"172.21.15.43:6801","nonce":502667335}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6802","nonce":502667335},{"type":"v1","addr":"172.21.15.43:6803","nonce":502667335}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6806","nonce":502667335},{"type":"v1","addr":"172.21.15.43:6807","nonce":502667335}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6804","nonce":502667335},{"type":"v1","addr":"172.21.15.43:6805","nonce":502667335}]},"public_addr":"172.21.15.43:6801/502667335","cluster_addr":"172.21.15.43:6803/502667335","heartbeat_back_addr":"172.21.15.43:6807/502667335","heartbeat_front_addr":"172.21.15.43:6805/502667335","state":["exists","up"]},{"osd":1,"uuid":"75c7e0fe-1ec7-4fe8-a2f2-3661a65b8500","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":101,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6808","nonce":3289134610},{"type":"v1","addr":"172.21.15.43:6809","nonce":3289134610}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6810","nonce":3289134610},{"type":"v1","addr":"172.21.15.43:6812","nonce":3289134610}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6821","nonce":3289134610},{"type":"v1","addr":"172.21.15.43:6824","nonce":3289134610}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6815","nonce":3289134610},{"type":"v1","addr":"172.21.15.43:6818","nonce":3289134610}]},"public_addr":"172.21.15.43:6809/3289134610","cluster_addr":"172.21.15.43:6812/3289134610","heartbeat_back_addr":"172.21.15.43:6824/3289134610","heartbeat_front_addr":"172.21.15.43:6818/3289134610","state":["exists","up"]},{"osd":2,"uuid":"bb5c41e9-7f5f-4125-9b01-3bb8bfc2ea83","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":101,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6811","nonce":4079709736},{"type":"v1","addr":"172.21.15.43:6814","nonce":4079709736}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6817","nonce":4079709736},{"type":"v1","addr":"172.21.15.43:6820","nonce":4079709736}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6829","nonce":4079709736},{"type":"v1","addr":"172.21.15.43:6831","nonce":4079709736}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6823","nonce":4079709736},{"type":"v1","addr":"172.21.15.43:6826","nonce":4079709736}]},"public_addr":"172.21.15.43:6814/4079709736","cluster_addr":"172.21.15.43:6820/4079709736","heartbeat_back_addr":"172.21.15.43:6831/4079709736","heartbeat_front_addr":"172.21.15.43:6826/4079709736","state":["exists","up"]},{"osd":3,"uuid":"bfe4c8b4-694a-465c-87e6-2b2aab852613","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":101,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6813","nonce":1709538759},{"type":"v1","addr":"172.21.15.43:6816","nonce":1709538759}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6819","nonce":1709538759},{"type":"v1","addr":"172.21.15.43:6822","nonce":1709538759}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6828","nonce":1709538759},{"type":"v1","addr":"172.21.15.43:6830","nonce":1709538759}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6825","nonce":1709538759},{"type":"v1","addr":"172.21.15.43:6827","nonce":1709538759}]},"public_addr":"172.21.15.43:6816/1709538759","cluster_addr":"172.21.15.43:6822/1709538759","heartbeat_back_addr":"172.21.15.43:6830/1709538759","heartbeat_front_addr":"172.21.15.43:6827/1709538759","state":["exists","up"]},{"osd":4,"uuid":"fa2e8a77-aba1-4756-853e-0d174a5bdeba","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":101,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6800","nonce":4196866481},{"type":"v1","addr":"172.21.15.203:6801","nonce":4196866481}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6802","nonce":4196866481},{"type":"v1","addr":"172.21.15.203:6803","nonce":4196866481}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6806","nonce":4196866481},{"type":"v1","addr":"172.21.15.203:6807","nonce":4196866481}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6804","nonce":4196866481},{"type":"v1","addr":"172.21.15.203:6805","nonce":4196866481}]},"public_addr":"172.21.15.203:6801/4196866481","cluster_addr":"172.21.15.203:6803/4196866481","heartbeat_back_addr":"172.21.15.203:6807/4196866481","heartbeat_front_addr":"172.21.15.203:6805/4196866481","state":["exists","up"]},{"osd":5,"uuid":"81811e4b-4d24-4f51-a82c-f1d700a2a00d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":101,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6816","nonce":3059290081},{"type":"v1","addr":"172.21.15.203:6817","nonce":3059290081}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6818","nonce":3059290081},{"type":"v1","addr":"172.21.15.203:6819","nonce":3059290081}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6822","nonce":3059290081},{"type":"v1","addr":"172.21.15.203:6823","nonce":3059290081}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6820","nonce":3059290081},{"type":"v1","addr":"172.21.15.203:6821","nonce":3059290081}]},"public_addr":"172.21.15.203:6817/3059290081","cluster_addr":"172.21.15.203:6819/3059290081","heartbeat_back_addr":"172.21.15.203:6823/3059290081","heartbeat_front_addr":"172.21.15.203:6821/3059290081","state":["exists","up"]},{"osd":6,"uuid":"307bac4d-6a5b-443d-af75-56d695d7674f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":101,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6824","nonce":512155569},{"type":"v1","addr":"172.21.15.203:6825","nonce":512155569}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6826","nonce":512155569},{"type":"v1","addr":"172.21.15.203:6827","nonce":512155569}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6830","nonce":512155569},{"type":"v1","addr":"172.21.15.203:6831","nonce":512155569}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6828","nonce":512155569},{"type":"v1","addr":"172.21.15.203:6829","nonce":512155569}]},"public_addr":"172.21.15.203:6825/512155569","cluster_addr":"172.21.15.203:6827/512155569","heartbeat_back_addr":"172.21.15.203:6831/512155569","heartbeat_front_addr":"172.21.15.203:6829/512155569","state":["exists","up"]},{"osd":7,"uuid":"a95dc7ca-9f1b-4295-af70-68a22a2d7d5e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":101,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6808","nonce":3331450438},{"type":"v1","addr":"172.21.15.203:6809","nonce":3331450438}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6810","nonce":3331450438},{"type":"v1","addr":"172.21.15.203:6811","nonce":3331450438}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6814","nonce":3331450438},{"type":"v1","addr":"172.21.15.203:6815","nonce":3331450438}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6812","nonce":3331450438},{"type":"v1","addr":"172.21.15.203:6813","nonce":3331450438}]},"public_addr":"172.21.15.203:6809/3331450438","cluster_addr":"172.21.15.203:6811/3331450438","heartbeat_back_addr":"172.21.15.203:6815/3331450438","heartbeat_front_addr":"172.21.15.203:6813/3331450438","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-09T13:33:50.547652+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-09T13:33:50.589469+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-09T13:33:50.561724+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-09T13:33:50.599650+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-09T13:33:50.411855+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-09T13:33:50.219681+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-09T13:33:50.504664+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-09T13:33:50.441248+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.203:0/996706743":"2023-08-10T13:51:38.530957+0000","172.21.15.203:0/969846951":"2023-08-10T13:51:38.530957+0000","172.21.15.203:6833/2675832148":"2023-08-10T13:51:38.530957+0000","172.21.15.203:6832/2675832148":"2023-08-10T13:51:38.530957+0000","172.21.15.203:0/822053193":"2023-08-10T13:51:38.530957+0000","172.21.15.203:0/1475395073":"2023-08-10T13:51:38.530957+0000","172.21.15.203:0/1343041460":"2023-08-10T13:51:32.420350+0000","172.21.15.203:6833/1778614695":"2023-08-10T13:51:32.420350+0000","172.21.15.203:0/3761332698":"2023-08-10T13:51:32.420350+0000","172.21.15.203:6832/1778614695":"2023-08-10T13:51:32.420350+0000","172.21.15.203:0/2293720318":"2023-08-10T13:51:38.530957+0000","172.21.15.203:0/2054133158":"2023-08-10T13:51:32.420350+0000","172.21.15.203:0/3225794224":"2023-08-10T13:51:38.530957+0000","172.21.15.203:0/1496080019":"2023-08-10T13:51:32.420350+0000","172.21.15.203:0/1894709586":"2023-08-10T13:51:32.420350+0000","172.21.15.203:0/270998287":"2023-08-10T13:51:32.420350+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":7,"snaps":[{"begin":104,"length":1},{"begin":221,"length":1}]}],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-08-09T13:51:39.140 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:39.138+0000 7fc5df59e640 1 -- 172.21.15.43:0/1735504299 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc5e0070b00 msgr2=0x7fc5e015b0c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:51:39.140 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:39.138+0000 7fc5df59e640 1 --2- 172.21.15.43:0/1735504299 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc5e0070b00 0x7fc5e015b0c0 secure :-1 s=READY pgs=1249 cs=0 l=1 rev1=1 crypto rx=0x7fc5c800c7f0 tx=0x7fc5c800ccb0 comp rx=0 tx=0).stop 2023-08-09T13:51:39.141 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:39.138+0000 7fc5df59e640 1 -- 172.21.15.43:0/1735504299 shutdown_connections 2023-08-09T13:51:39.141 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:39.138+0000 7fc5df59e640 1 --2- 172.21.15.43:0/1735504299 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fc5e015c930 0x7fc5e00ab8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:51:39.141 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:39.138+0000 7fc5df59e640 1 --2- 172.21.15.43:0/1735504299 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc5e0070b00 0x7fc5e015b0c0 unknown :-1 s=CLOSED pgs=1249 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:51:39.141 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:39.138+0000 7fc5df59e640 1 --2- 172.21.15.43:0/1735504299 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fc5e006c530 0x7fc5e01549b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:51:39.142 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:39.138+0000 7fc5df59e640 1 -- 172.21.15.43:0/1735504299 >> 172.21.15.43:0/1735504299 conn(0x7fc5e006a8d0 msgr2=0x7fc5e00cde30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-09T13:51:39.142 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:39.138+0000 7fc5df59e640 1 -- 172.21.15.43:0/1735504299 shutdown_connections 2023-08-09T13:51:39.142 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:39.138+0000 7fc5df59e640 1 -- 172.21.15.43:0/1735504299 wait complete. 2023-08-09T13:51:39.160 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': 'rbd', 'create_time': '2023-08-09T13:33:54.365137+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}, 'read_balance': {'score_acting': 2, 'score_stable': 2, 'optimal_score': 1, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': '.mgr', 'create_time': '2023-08-09T13:33:54.831803+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 12, 'pool_name': 'cephfs_metadata', 'create_time': '2023-08-09T13:51:14.142563+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '102', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 2, 'score_stable': 2, 'optimal_score': 1, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 13, 'pool_name': 'cephfs_data', 'create_time': '2023-08-09T13:51:14.712668+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '103', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.3799999952316284, 'score_stable': 1.3799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.3799999952316284, 'raw_score_stable': 1.3799999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-08-09T13:51:39.161 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get rbd pg_num 2023-08-09T13:51:39.444 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:39.442+0000 7fdc7c42f640 1 Processor -- start 2023-08-09T13:51:39.444 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:39.442+0000 7fdc7c42f640 1 -- start start 2023-08-09T13:51:39.446 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:39.443+0000 7fdc7c42f640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7fdc740b45f0 0x7fdc740cd6d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:51:39.446 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:39.443+0000 7fdc7c42f640 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7fdc740d04b0 0x7fdc740b59d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:51:39.446 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:39.443+0000 7fdc7c42f640 1 -- --> v1:172.21.15.203:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fdc740b7bd0 con 0x7fdc7415c550 2023-08-09T13:51:39.446 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:39.443+0000 7fdc7c42f640 1 -- --> v2:172.21.15.43:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fdc740d1c00 con 0x7fdc740b45f0 2023-08-09T13:51:39.447 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:39.443+0000 7fdc7c42f640 1 -- --> v2:172.21.15.203:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fdc7415c930 con 0x7fdc740d04b0 2023-08-09T13:51:39.447 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:39.444+0000 7fdc799ca640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7fdc740b45f0 0x7fdc740cd6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:51:39.447 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:39.444+0000 7fdc799ca640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7fdc740b45f0 0x7fdc740cd6d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:41382/0 (socket says 172.21.15.43:41382) 2023-08-09T13:51:39.447 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:39.444+0000 7fdc799ca640 1 -- 172.21.15.43:0/4040240212 learned_addr learned my addr 172.21.15.43:0/4040240212 (peer_addr_for_me v2:172.21.15.43:0/0) 2023-08-09T13:51:39.448 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:39.444+0000 7fdc7a9cc640 1 --2- 172.21.15.43:0/4040240212 >> v2:172.21.15.203:3300/0 conn(0x7fdc740d04b0 0x7fdc740b59d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:51:39.448 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:39.444+0000 7fdc799ca640 1 -- 172.21.15.43:0/4040240212 >> v2:172.21.15.203:3300/0 conn(0x7fdc740d04b0 msgr2=0x7fdc740b59d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-09T13:51:39.448 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:39.444+0000 7fdc799ca640 1 --2- 172.21.15.43:0/4040240212 >> v2:172.21.15.203:3300/0 conn(0x7fdc740d04b0 0x7fdc740b59d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:51:39.448 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:39.444+0000 7fdc799ca640 1 -- 172.21.15.43:0/4040240212 >> v1:172.21.15.203:6789/0 conn(0x7fdc7415c550 legacy=0x7fdc740b3ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-09T13:51:39.448 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:39.444+0000 7fdc799ca640 1 -- 172.21.15.43:0/4040240212 --> v2:172.21.15.43:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdc740b54f0 con 0x7fdc740b45f0 2023-08-09T13:51:39.449 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:39.446+0000 7fdc799ca640 1 --2- 172.21.15.43:0/4040240212 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fdc740b45f0 0x7fdc740cd6d0 secure :-1 s=READY pgs=1250 cs=0 l=1 rev1=1 crypto rx=0x7fdc740ce7f0 tx=0x7fdc6402b910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e4fb053e2441faef server_cookie=0 in_seq=0 out_seq=0 2023-08-09T13:51:39.449 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:39.446+0000 7fdc791c9640 1 -- 172.21.15.43:0/4040240212 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdc64031870 con 0x7fdc740b45f0 2023-08-09T13:51:39.450 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:39.446+0000 7fdc791c9640 1 -- 172.21.15.43:0/4040240212 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdc640319d0 con 0x7fdc740b45f0 2023-08-09T13:51:39.450 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:39.446+0000 7fdc791c9640 1 -- 172.21.15.43:0/4040240212 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdc64031d60 con 0x7fdc740b45f0 2023-08-09T13:51:39.450 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:39.447+0000 7fdc7c42f640 1 -- 172.21.15.43:0/4040240212 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fdc740b45f0 msgr2=0x7fdc740cd6d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:51:39.450 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:39.447+0000 7fdc7c42f640 1 --2- 172.21.15.43:0/4040240212 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fdc740b45f0 0x7fdc740cd6d0 secure :-1 s=READY pgs=1250 cs=0 l=1 rev1=1 crypto rx=0x7fdc740ce7f0 tx=0x7fdc6402b910 comp rx=0 tx=0).stop 2023-08-09T13:51:39.450 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:39.447+0000 7fdc7c42f640 1 -- 172.21.15.43:0/4040240212 shutdown_connections 2023-08-09T13:51:39.450 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:39.447+0000 7fdc7c42f640 1 --2- 172.21.15.43:0/4040240212 >> v2:172.21.15.203:3300/0 conn(0x7fdc740d04b0 0x7fdc740b59d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:51:39.450 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:39.447+0000 7fdc7c42f640 1 --2- 172.21.15.43:0/4040240212 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fdc740b45f0 0x7fdc740cd6d0 unknown :-1 s=CLOSED pgs=1250 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:51:39.451 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:39.447+0000 7fdc7c42f640 1 -- 172.21.15.43:0/4040240212 >> 172.21.15.43:0/4040240212 conn(0x7fdc74071520 msgr2=0x7fdc740ceee0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-09T13:51:39.451 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:39.447+0000 7fdc7c42f640 1 -- 172.21.15.43:0/4040240212 shutdown_connections 2023-08-09T13:51:39.451 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:39.447+0000 7fdc7c42f640 1 -- 172.21.15.43:0/4040240212 wait complete. 2023-08-09T13:51:39.451 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:39.449+0000 7fdc7c42f640 1 Processor -- start 2023-08-09T13:51:39.452 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:39.450+0000 7fdc7c42f640 1 -- start start 2023-08-09T13:51:39.453 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:39.451+0000 7fdc7c42f640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fdc740b45f0 0x7fdc740c9680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:51:39.453 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:39.451+0000 7fdc7c42f640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fdc740d04b0 0x7fdc740ca130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:51:39.453 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:39.451+0000 7fdc7c42f640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fdc7415c550 0x7fdc74155570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:51:39.454 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:39.451+0000 7fdc7c42f640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdc741564f0 con 0x7fdc740b45f0 2023-08-09T13:51:39.454 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:39.451+0000 7fdc7c42f640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdc74158df0 con 0x7fdc7415c550 2023-08-09T13:51:39.454 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:39.451+0000 7fdc7c42f640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdc741590d0 con 0x7fdc740d04b0 2023-08-09T13:51:39.455 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:39.451+0000 7fdc7a1cb640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fdc740b45f0 0x7fdc740c9680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:51:39.455 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:39.451+0000 7fdc7a1cb640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fdc740b45f0 0x7fdc740c9680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:41384/0 (socket says 172.21.15.43:41384) 2023-08-09T13:51:39.455 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:39.451+0000 7fdc7a1cb640 1 -- 172.21.15.43:0/3864417275 learned_addr learned my addr 172.21.15.43:0/3864417275 (peer_addr_for_me v2:172.21.15.43:0/0) 2023-08-09T13:51:39.456 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:39.452+0000 7fdc7a9cc640 1 --2- 172.21.15.43:0/3864417275 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fdc7415c550 0x7fdc74155570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:51:39.456 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:39.452+0000 7fdc799ca640 1 --2- 172.21.15.43:0/3864417275 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fdc740d04b0 0x7fdc740ca130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:51:39.456 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:39.452+0000 7fdc7a1cb640 1 -- 172.21.15.43:0/3864417275 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fdc740d04b0 msgr2=0x7fdc740ca130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:51:39.456 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:39.452+0000 7fdc7a1cb640 1 --2- 172.21.15.43:0/3864417275 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fdc740d04b0 0x7fdc740ca130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:51:39.456 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:39.452+0000 7fdc7a1cb640 1 -- 172.21.15.43:0/3864417275 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fdc7415c550 msgr2=0x7fdc74155570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:51:39.456 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:39.452+0000 7fdc7a1cb640 1 --2- 172.21.15.43:0/3864417275 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fdc7415c550 0x7fdc74155570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:51:39.456 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:39.452+0000 7fdc7a1cb640 1 -- 172.21.15.43:0/3864417275 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdc640094d0 con 0x7fdc740b45f0 2023-08-09T13:51:39.457 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:39.452+0000 7fdc799ca640 1 --2- 172.21.15.43:0/3864417275 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fdc740d04b0 0x7fdc740ca130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-08-09T13:51:39.457 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:39.452+0000 7fdc7a9cc640 1 --2- 172.21.15.43:0/3864417275 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fdc7415c550 0x7fdc74155570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-08-09T13:51:39.457 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:39.452+0000 7fdc7a1cb640 1 --2- 172.21.15.43:0/3864417275 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fdc740b45f0 0x7fdc740c9680 secure :-1 s=READY pgs=1251 cs=0 l=1 rev1=1 crypto rx=0x7fdc7000ba50 tx=0x7fdc7000bf10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-09T13:51:39.457 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:39.452+0000 7fdc6affd640 1 -- 172.21.15.43:0/3864417275 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdc7000cbe0 con 0x7fdc740b45f0 2023-08-09T13:51:39.457 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:39.452+0000 7fdc6affd640 1 -- 172.21.15.43:0/3864417275 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdc7000cd80 con 0x7fdc740b45f0 2023-08-09T13:51:39.457 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:39.452+0000 7fdc6affd640 1 -- 172.21.15.43:0/3864417275 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdc70014680 con 0x7fdc740b45f0 2023-08-09T13:51:39.457 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:39.452+0000 7fdc7c42f640 1 -- 172.21.15.43:0/3864417275 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdc741593b0 con 0x7fdc740b45f0 2023-08-09T13:51:39.458 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:39.453+0000 7fdc7c42f640 1 -- 172.21.15.43:0/3864417275 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdc740c9bc0 con 0x7fdc740b45f0 2023-08-09T13:51:39.458 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:39.454+0000 7fdc7c42f640 1 -- 172.21.15.43:0/3864417275 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdc740739f0 con 0x7fdc740b45f0 2023-08-09T13:51:39.458 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:39.455+0000 7fdc6affd640 1 -- 172.21.15.43:0/3864417275 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 42) v1 ==== 88052+0+0 (secure 0 0 0) 0x7fdc700147e0 con 0x7fdc740b45f0 2023-08-09T13:51:39.458 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:39.456+0000 7fdc6affd640 1 -- 172.21.15.43:0/3864417275 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(105..105 src has 1..105) v4 ==== 7114+0+0 (secure 0 0 0) 0x7fdc700883f0 con 0x7fdc740b45f0 2023-08-09T13:51:39.462 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:39.460+0000 7fdc6affd640 1 -- 172.21.15.43:0/3864417275 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178066 (secure 0 0 0) 0x7fdc70053ad0 con 0x7fdc740b45f0 2023-08-09T13:51:39.560 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:39.557+0000 7fdc6affd640 1 -- 172.21.15.43:0/3864417275 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mgrmap(e 43) v1 ==== 88179+0+0 (secure 0 0 0) 0x7fdc70053580 con 0x7fdc740b45f0 2023-08-09T13:51:39.560 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:39.557+0000 7fdc6affd640 1 --2- 172.21.15.43:0/3864417275 >> [v2:172.21.15.203:6832/1317108840,v1:172.21.15.203:6833/1317108840] conn(0x7fdc58070430 0x7fdc58072820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:51:39.563 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:39.560+0000 7fdc799ca640 1 --2- 172.21.15.43:0/3864417275 >> [v2:172.21.15.203:6832/1317108840,v1:172.21.15.203:6833/1317108840] conn(0x7fdc58070430 0x7fdc58072820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:51:39.567 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:39.565+0000 7fdc799ca640 1 --2- 172.21.15.43:0/3864417275 >> [v2:172.21.15.203:6832/1317108840,v1:172.21.15.203:6833/1317108840] conn(0x7fdc58070430 0x7fdc58072820 secure :-1 s=READY pgs=5 cs=0 l=1 rev1=1 crypto rx=0x7fdc64036000 tx=0x7fdc6403c040 comp rx=0 tx=0).ready entity=mgr.6619 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-09T13:51:39.667 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:39.664+0000 7fdc7c42f640 1 -- 172.21.15.43:0/3864417275 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"} v 0) v1 -- 0x7fdc740cd9e0 con 0x7fdc740b45f0 2023-08-09T13:51:39.668 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:39.665+0000 7fdc6affd640 1 -- 172.21.15.43:0/3864417275 <== mon.0 v2:172.21.15.43:3300/0 8 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"}]=0 v105) v1 ==== 92+0+10 (secure 0 0 0) 0x7fdc70070dc0 con 0x7fdc740b45f0 2023-08-09T13:51:39.668 INFO:teuthology.orchestra.run.smithi043.stdout:pg_num: 8 2023-08-09T13:51:39.670 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:39.668+0000 7fdc7c42f640 1 -- 172.21.15.43:0/3864417275 >> [v2:172.21.15.203:6832/1317108840,v1:172.21.15.203:6833/1317108840] conn(0x7fdc58070430 msgr2=0x7fdc58072820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:51:39.670 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:39.668+0000 7fdc7c42f640 1 --2- 172.21.15.43:0/3864417275 >> [v2:172.21.15.203:6832/1317108840,v1:172.21.15.203:6833/1317108840] conn(0x7fdc58070430 0x7fdc58072820 secure :-1 s=READY pgs=5 cs=0 l=1 rev1=1 crypto rx=0x7fdc64036000 tx=0x7fdc6403c040 comp rx=0 tx=0).stop 2023-08-09T13:51:39.671 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:39.668+0000 7fdc7c42f640 1 -- 172.21.15.43:0/3864417275 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fdc740b45f0 msgr2=0x7fdc740c9680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:51:39.671 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:39.668+0000 7fdc7c42f640 1 --2- 172.21.15.43:0/3864417275 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fdc740b45f0 0x7fdc740c9680 secure :-1 s=READY pgs=1251 cs=0 l=1 rev1=1 crypto rx=0x7fdc7000ba50 tx=0x7fdc7000bf10 comp rx=0 tx=0).stop 2023-08-09T13:51:39.671 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:39.668+0000 7fdc7c42f640 1 -- 172.21.15.43:0/3864417275 shutdown_connections 2023-08-09T13:51:39.671 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:39.668+0000 7fdc7c42f640 1 --2- 172.21.15.43:0/3864417275 >> [v2:172.21.15.203:6832/1317108840,v1:172.21.15.203:6833/1317108840] conn(0x7fdc58070430 0x7fdc58072820 unknown :-1 s=CLOSED pgs=5 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:51:39.671 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:39.668+0000 7fdc7c42f640 1 --2- 172.21.15.43:0/3864417275 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fdc7415c550 0x7fdc74155570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:51:39.671 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:39.668+0000 7fdc7c42f640 1 --2- 172.21.15.43:0/3864417275 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fdc740d04b0 0x7fdc740ca130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:51:39.671 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:39.668+0000 7fdc7c42f640 1 --2- 172.21.15.43:0/3864417275 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fdc740b45f0 0x7fdc740c9680 unknown :-1 s=CLOSED pgs=1251 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:51:39.672 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:39.668+0000 7fdc7c42f640 1 -- 172.21.15.43:0/3864417275 >> 172.21.15.43:0/3864417275 conn(0x7fdc74071520 msgr2=0x7fdc7415cb30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-09T13:51:39.672 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:39.668+0000 7fdc7c42f640 1 -- 172.21.15.43:0/3864417275 shutdown_connections 2023-08-09T13:51:39.672 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:39.668+0000 7fdc7c42f640 1 -- 172.21.15.43:0/3864417275 wait complete. 2023-08-09T13:51:39.693 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-08-09T13:51:39.926 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:39.924+0000 7fa1eecad640 1 Processor -- start 2023-08-09T13:51:39.926 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:39.924+0000 7fa1eecad640 1 -- start start 2023-08-09T13:51:39.927 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:39.925+0000 7fa1eecad640 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7fa1e815c2d0 0x7fa1e80b7df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:51:39.928 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:39.925+0000 7fa1eecad640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7fa1e80cebf0 0x7fa1e80cf040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:51:39.928 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:39.925+0000 7fa1eecad640 1 -- --> v1:172.21.15.43:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa1e80cf580 con 0x7fa1e80d1b30 2023-08-09T13:51:39.928 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:39.925+0000 7fa1eecad640 1 -- --> v2:172.21.15.43:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa1e80d16a0 con 0x7fa1e80cebf0 2023-08-09T13:51:39.928 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:39.925+0000 7fa1eecad640 1 -- --> v2:172.21.15.203:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa1e80d1fa0 con 0x7fa1e815c2d0 2023-08-09T13:51:39.929 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:39.926+0000 7fa1ee4ac640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7fa1e80cebf0 0x7fa1e80cf040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:51:39.929 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:39.926+0000 7fa1ee4ac640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7fa1e80cebf0 0x7fa1e80cf040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:41392/0 (socket says 172.21.15.43:41392) 2023-08-09T13:51:39.929 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:39.926+0000 7fa1ee4ac640 1 -- 172.21.15.43:0/2171518569 learned_addr learned my addr 172.21.15.43:0/2171518569 (peer_addr_for_me v2:172.21.15.43:0/0) 2023-08-09T13:51:39.929 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:39.926+0000 7fa1ed4aa640 1 --1- >> v1:172.21.15.43:6789/0 conn(0x7fa1e80d1b30 0x7fa1e80cd020 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.43:6789/0 says I am v1:172.21.15.43:60796/0 (socket says 172.21.15.43:60796) 2023-08-09T13:51:39.930 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:39.926+0000 7fa1edcab640 1 --2- 172.21.15.43:0/2171518569 >> v2:172.21.15.203:3300/0 conn(0x7fa1e815c2d0 0x7fa1e80b7df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:51:39.930 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:39.926+0000 7fa1ee4ac640 1 -- 172.21.15.43:0/2171518569 >> v2:172.21.15.203:3300/0 conn(0x7fa1e815c2d0 msgr2=0x7fa1e80b7df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-09T13:51:39.930 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:39.926+0000 7fa1ee4ac640 1 --2- 172.21.15.43:0/2171518569 >> v2:172.21.15.203:3300/0 conn(0x7fa1e815c2d0 0x7fa1e80b7df0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:51:39.930 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:39.926+0000 7fa1ee4ac640 1 -- 172.21.15.43:0/2171518569 >> v1:172.21.15.43:6789/0 conn(0x7fa1e80d1b30 legacy=0x7fa1e80cd020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:51:39.930 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:39.926+0000 7fa1ee4ac640 1 -- 172.21.15.43:0/2171518569 --> v2:172.21.15.43:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa1e80d0d70 con 0x7fa1e80cebf0 2023-08-09T13:51:39.931 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:39.928+0000 7fa1ee4ac640 1 --2- 172.21.15.43:0/2171518569 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa1e80cebf0 0x7fa1e80cf040 secure :-1 s=READY pgs=1253 cs=0 l=1 rev1=1 crypto rx=0x7fa1e806ae00 tx=0x7fa1e002b9a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4a5c6d1fca8ffc7a server_cookie=0 in_seq=0 out_seq=0 2023-08-09T13:51:39.931 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:39.928+0000 7fa1ecca9640 1 -- 172.21.15.43:0/2171518569 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa1e0030860 con 0x7fa1e80cebf0 2023-08-09T13:51:39.931 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:39.928+0000 7fa1ecca9640 1 -- 172.21.15.43:0/2171518569 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa1e00309c0 con 0x7fa1e80cebf0 2023-08-09T13:51:39.931 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:39.928+0000 7fa1ecca9640 1 -- 172.21.15.43:0/2171518569 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa1e0030d50 con 0x7fa1e80cebf0 2023-08-09T13:51:39.932 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:39.928+0000 7fa1eecad640 1 -- 172.21.15.43:0/2171518569 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa1e80cebf0 msgr2=0x7fa1e80cf040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:51:39.932 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:39.929+0000 7fa1eecad640 1 --2- 172.21.15.43:0/2171518569 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa1e80cebf0 0x7fa1e80cf040 secure :-1 s=READY pgs=1253 cs=0 l=1 rev1=1 crypto rx=0x7fa1e806ae00 tx=0x7fa1e002b9a0 comp rx=0 tx=0).stop 2023-08-09T13:51:39.932 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:39.929+0000 7fa1eecad640 1 -- 172.21.15.43:0/2171518569 shutdown_connections 2023-08-09T13:51:39.932 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:39.929+0000 7fa1eecad640 1 --2- 172.21.15.43:0/2171518569 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa1e80cebf0 0x7fa1e80cf040 unknown :-1 s=CLOSED pgs=1253 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:51:39.932 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:39.929+0000 7fa1eecad640 1 --2- 172.21.15.43:0/2171518569 >> v2:172.21.15.203:3300/0 conn(0x7fa1e815c2d0 0x7fa1e80b7df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:51:39.932 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:39.929+0000 7fa1eecad640 1 -- 172.21.15.43:0/2171518569 >> 172.21.15.43:0/2171518569 conn(0x7fa1e806a230 msgr2=0x7fa1e806a640 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-09T13:51:39.932 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:39.929+0000 7fa1eecad640 1 -- 172.21.15.43:0/2171518569 shutdown_connections 2023-08-09T13:51:39.933 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:39.929+0000 7fa1eecad640 1 -- 172.21.15.43:0/2171518569 wait complete. 2023-08-09T13:51:39.933 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:39.930+0000 7fa1eecad640 1 Processor -- start 2023-08-09T13:51:39.933 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:39.931+0000 7fa1eecad640 1 -- start start 2023-08-09T13:51:39.934 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:39.932+0000 7fa1eecad640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fa1e80cebf0 0x7fa1e8149f10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:51:39.934 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:39.932+0000 7fa1eecad640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa1e80d1b30 0x7fa1e80a91d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:51:39.934 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:39.932+0000 7fa1eecad640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fa1e815c2d0 0x7fa1e8148490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:51:39.935 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:39.932+0000 7fa1eecad640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa1e8148e70 con 0x7fa1e80d1b30 2023-08-09T13:51:39.935 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:39.932+0000 7fa1eecad640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa1e8149150 con 0x7fa1e80cebf0 2023-08-09T13:51:39.935 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:39.932+0000 7fa1eecad640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa1e8149430 con 0x7fa1e815c2d0 2023-08-09T13:51:39.935 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:39.932+0000 7fa1ed4aa640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa1e80d1b30 0x7fa1e80a91d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:51:39.935 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:39.932+0000 7fa1ed4aa640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa1e80d1b30 0x7fa1e80a91d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:41396/0 (socket says 172.21.15.43:41396) 2023-08-09T13:51:39.935 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:39.932+0000 7fa1ed4aa640 1 -- 172.21.15.43:0/3013495054 learned_addr learned my addr 172.21.15.43:0/3013495054 (peer_addr_for_me v2:172.21.15.43:0/0) 2023-08-09T13:51:39.936 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:39.932+0000 7fa1edcab640 1 --2- 172.21.15.43:0/3013495054 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fa1e80cebf0 0x7fa1e8149f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:51:39.936 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:39.932+0000 7fa1ed4aa640 1 -- 172.21.15.43:0/3013495054 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fa1e815c2d0 msgr2=0x7fa1e8148490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:51:39.937 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:39.932+0000 7fa1ed4aa640 1 --2- 172.21.15.43:0/3013495054 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fa1e815c2d0 0x7fa1e8148490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:51:39.937 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:39.932+0000 7fa1ed4aa640 1 -- 172.21.15.43:0/3013495054 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fa1e80cebf0 msgr2=0x7fa1e8149f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:51:39.937 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:39.932+0000 7fa1ed4aa640 1 --2- 172.21.15.43:0/3013495054 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fa1e80cebf0 0x7fa1e8149f10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:51:39.937 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:39.932+0000 7fa1ed4aa640 1 -- 172.21.15.43:0/3013495054 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa1e00094d0 con 0x7fa1e80d1b30 2023-08-09T13:51:39.937 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:39.933+0000 7fa1ed4aa640 1 --2- 172.21.15.43:0/3013495054 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa1e80d1b30 0x7fa1e80a91d0 secure :-1 s=READY pgs=1254 cs=0 l=1 rev1=1 crypto rx=0x7fa1d800c450 tx=0x7fa1d800c910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-09T13:51:39.937 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:39.933+0000 7fa1de7fc640 1 -- 172.21.15.43:0/3013495054 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa1d8007480 con 0x7fa1e80d1b30 2023-08-09T13:51:39.937 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:39.933+0000 7fa1de7fc640 1 -- 172.21.15.43:0/3013495054 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa1d80075e0 con 0x7fa1e80d1b30 2023-08-09T13:51:39.938 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:39.933+0000 7fa1de7fc640 1 -- 172.21.15.43:0/3013495054 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa1d8004a00 con 0x7fa1e80d1b30 2023-08-09T13:51:39.938 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:39.933+0000 7fa1eecad640 1 -- 172.21.15.43:0/3013495054 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa1e8149710 con 0x7fa1e80d1b30 2023-08-09T13:51:39.938 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:39.934+0000 7fa1eecad640 1 -- 172.21.15.43:0/3013495054 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa1e80a9710 con 0x7fa1e80d1b30 2023-08-09T13:51:39.938 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:39.935+0000 7fa1eecad640 1 -- 172.21.15.43:0/3013495054 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa1e8149c30 con 0x7fa1e80d1b30 2023-08-09T13:51:39.938 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:39.936+0000 7fa1de7fc640 1 -- 172.21.15.43:0/3013495054 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 43) v1 ==== 88179+0+0 (secure 0 0 0) 0x7fa1d8004b60 con 0x7fa1e80d1b30 2023-08-09T13:51:39.940 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:39.937+0000 7fa1de7fc640 1 --2- 172.21.15.43:0/3013495054 >> [v2:172.21.15.203:6832/1317108840,v1:172.21.15.203:6833/1317108840] conn(0x7fa1cc069ef0 0x7fa1cc06c3b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:51:39.940 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:39.937+0000 7fa1de7fc640 1 -- 172.21.15.43:0/3013495054 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(105..105 src has 1..105) v4 ==== 7114+0+0 (secure 0 0 0) 0x7fa1d8087a00 con 0x7fa1e80d1b30 2023-08-09T13:51:39.940 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:39.938+0000 7fa1edcab640 1 --2- 172.21.15.43:0/3013495054 >> [v2:172.21.15.203:6832/1317108840,v1:172.21.15.203:6833/1317108840] conn(0x7fa1cc069ef0 0x7fa1cc06c3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:51:39.941 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:39.939+0000 7fa1edcab640 1 --2- 172.21.15.43:0/3013495054 >> [v2:172.21.15.203:6832/1317108840,v1:172.21.15.203:6833/1317108840] conn(0x7fa1cc069ef0 0x7fa1cc06c3b0 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7fa1e4009f30 tx=0x7fa1e400b040 comp rx=0 tx=0).ready entity=mgr.6619 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-09T13:51:39.944 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:39.942+0000 7fa1de7fc640 1 -- 172.21.15.43:0/3013495054 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178066 (secure 0 0 0) 0x7fa1e8149c30 con 0x7fa1e80d1b30 2023-08-09T13:51:40.146 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:40.144+0000 7fa1eecad640 1 -- 172.21.15.43:0/3013495054 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7fa1e80cf580 con 0x7fa1e80d1b30 2023-08-09T13:51:40.148 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:40.145+0000 7fa1de7fc640 1 -- 172.21.15.43:0/3013495054 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v105) v1 ==== 93+0+10 (secure 0 0 0) 0x7fa1e80cf580 con 0x7fa1e80d1b30 2023-08-09T13:51:40.148 INFO:teuthology.orchestra.run.smithi043.stdout:pg_num: 1 2023-08-09T13:51:40.151 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:40.148+0000 7fa1eecad640 1 -- 172.21.15.43:0/3013495054 >> [v2:172.21.15.203:6832/1317108840,v1:172.21.15.203:6833/1317108840] conn(0x7fa1cc069ef0 msgr2=0x7fa1cc06c3b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:51:40.151 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:40.148+0000 7fa1eecad640 1 --2- 172.21.15.43:0/3013495054 >> [v2:172.21.15.203:6832/1317108840,v1:172.21.15.203:6833/1317108840] conn(0x7fa1cc069ef0 0x7fa1cc06c3b0 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7fa1e4009f30 tx=0x7fa1e400b040 comp rx=0 tx=0).stop 2023-08-09T13:51:40.151 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:40.148+0000 7fa1eecad640 1 -- 172.21.15.43:0/3013495054 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa1e80d1b30 msgr2=0x7fa1e80a91d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:51:40.151 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:40.148+0000 7fa1eecad640 1 --2- 172.21.15.43:0/3013495054 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa1e80d1b30 0x7fa1e80a91d0 secure :-1 s=READY pgs=1254 cs=0 l=1 rev1=1 crypto rx=0x7fa1d800c450 tx=0x7fa1d800c910 comp rx=0 tx=0).stop 2023-08-09T13:51:40.151 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:40.148+0000 7fa1eecad640 1 -- 172.21.15.43:0/3013495054 shutdown_connections 2023-08-09T13:51:40.152 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:40.148+0000 7fa1eecad640 1 --2- 172.21.15.43:0/3013495054 >> [v2:172.21.15.203:6832/1317108840,v1:172.21.15.203:6833/1317108840] conn(0x7fa1cc069ef0 0x7fa1cc06c3b0 unknown :-1 s=CLOSED pgs=22 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:51:40.152 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:40.148+0000 7fa1eecad640 1 --2- 172.21.15.43:0/3013495054 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fa1e815c2d0 0x7fa1e8148490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:51:40.152 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:40.148+0000 7fa1eecad640 1 --2- 172.21.15.43:0/3013495054 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa1e80d1b30 0x7fa1e80a91d0 unknown :-1 s=CLOSED pgs=1254 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:51:40.152 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:40.148+0000 7fa1eecad640 1 --2- 172.21.15.43:0/3013495054 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fa1e80cebf0 0x7fa1e8149f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:51:40.152 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:40.148+0000 7fa1eecad640 1 -- 172.21.15.43:0/3013495054 >> 172.21.15.43:0/3013495054 conn(0x7fa1e806a230 msgr2=0x7fa1e80b2980 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-09T13:51:40.152 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:40.148+0000 7fa1eecad640 1 -- 172.21.15.43:0/3013495054 shutdown_connections 2023-08-09T13:51:40.152 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:40.149+0000 7fa1eecad640 1 -- 172.21.15.43:0/3013495054 wait complete. 2023-08-09T13:51:40.169 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-08-09T13:51:40.392 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:40.390+0000 7fad2ae5e640 1 Processor -- start 2023-08-09T13:51:40.393 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:40.390+0000 7fad2ae5e640 1 -- start start 2023-08-09T13:51:40.394 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:40.392+0000 7fad2ae5e640 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7fad2406cfe0 0x7fad240c8b50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:51:40.395 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:40.392+0000 7fad2ae5e640 1 -- --> v1:172.21.15.43:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fad240c9090 con 0x7fad24143c20 2023-08-09T13:51:40.395 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:40.392+0000 7fad2ae5e640 1 -- --> v1:172.21.15.203:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fad2406c440 con 0x7fad240c86f0 2023-08-09T13:51:40.395 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:40.392+0000 7fad2ae5e640 1 -- --> v2:172.21.15.203:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fad24144000 con 0x7fad2406cfe0 2023-08-09T13:51:40.396 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:40.392+0000 7fad28bfa640 1 --1- >> v1:172.21.15.43:6789/0 conn(0x7fad24143c20 0x7fad2406f540 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.43:6789/0 says I am v1:172.21.15.43:60808/0 (socket says 172.21.15.43:60808) 2023-08-09T13:51:40.396 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:40.392+0000 7fad28bfa640 1 -- 172.21.15.43:0/1063502866 learned_addr learned my addr 172.21.15.43:0/1063502866 (peer_addr_for_me v1:172.21.15.43:0/0) 2023-08-09T13:51:40.396 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:40.393+0000 7fad293fb640 1 --2- 172.21.15.43:0/1063502866 >> v2:172.21.15.203:3300/0 conn(0x7fad2406cfe0 0x7fad240c8b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:51:40.397 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:40.393+0000 7fad237fe640 1 -- 172.21.15.43:0/1063502866 <== mon.0 v1:172.21.15.43:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3525398957 0 0) 0x7fad240c9090 con 0x7fad24143c20 2023-08-09T13:51:40.397 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:40.393+0000 7fad237fe640 1 -- 172.21.15.43:0/1063502866 --> v1:172.21.15.43:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fad14003520 con 0x7fad24143c20 2023-08-09T13:51:40.397 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:40.393+0000 7fad237fe640 1 -- 172.21.15.43:0/1063502866 <== mon.0 v1:172.21.15.43:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1080435284 0 0) 0x7fad14003520 con 0x7fad24143c20 2023-08-09T13:51:40.397 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:40.393+0000 7fad237fe640 1 -- 172.21.15.43:0/1063502866 >> v2:172.21.15.203:3300/0 conn(0x7fad2406cfe0 msgr2=0x7fad240c8b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-09T13:51:40.397 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:40.393+0000 7fad237fe640 1 --2- 172.21.15.43:0/1063502866 >> v2:172.21.15.203:3300/0 conn(0x7fad2406cfe0 0x7fad240c8b50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:51:40.398 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:40.393+0000 7fad237fe640 1 -- 172.21.15.43:0/1063502866 >> v1:172.21.15.203:6789/0 conn(0x7fad240c86f0 legacy=0x7fad2406c8d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:51:40.398 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:40.393+0000 7fad237fe640 1 -- 172.21.15.43:0/1063502866 --> v1:172.21.15.43:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fad2406fc50 con 0x7fad24143c20 2023-08-09T13:51:40.398 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:40.393+0000 7fad237fe640 1 -- 172.21.15.43:0/1063502866 <== mon.0 v1:172.21.15.43:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1285150740 0 0) 0x7fad18001670 con 0x7fad24143c20 2023-08-09T13:51:40.398 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:40.393+0000 7fad237fe640 1 -- 172.21.15.43:0/1063502866 >> v1:172.21.15.43:6789/0 conn(0x7fad24143c20 legacy=0x7fad2406f540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:51:40.398 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:40.393+0000 7fad237fe640 1 --2- 172.21.15.43:0/1063502866 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fad140039e0 0x7fad14003e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:51:40.398 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:40.393+0000 7fad237fe640 1 --2- 172.21.15.43:0/1063502866 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fad140047c0 0x7fad14004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:51:40.398 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:40.393+0000 7fad237fe640 1 --2- 172.21.15.43:0/1063502866 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fad14005280 0x7fad1400b980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:51:40.398 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:40.394+0000 7fad237fe640 1 -- 172.21.15.43:0/1063502866 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fad24144000 con 0x7fad140047c0 2023-08-09T13:51:40.399 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:40.394+0000 7fad237fe640 1 -- 172.21.15.43:0/1063502866 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fad1400c1c0 con 0x7fad14005280 2023-08-09T13:51:40.399 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:40.394+0000 7fad237fe640 1 -- 172.21.15.43:0/1063502866 --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fad1400c380 con 0x7fad140039e0 2023-08-09T13:51:40.399 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:40.394+0000 7fad23fff640 1 --2- 172.21.15.43:0/1063502866 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fad140047c0 0x7fad14004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:51:40.401 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:40.394+0000 7fad23fff640 1 -- 172.21.15.43:0/1063502866 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fad140039e0 msgr2=0x7fad14003e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-09T13:51:40.401 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:40.394+0000 7fad23fff640 1 --2- 172.21.15.43:0/1063502866 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fad140039e0 0x7fad14003e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:51:40.402 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:40.394+0000 7fad293fb640 1 --2- 172.21.15.43:0/1063502866 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fad14005280 0x7fad1400b980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:51:40.402 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:40.394+0000 7fad23fff640 1 -- 172.21.15.43:0/1063502866 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fad14005280 msgr2=0x7fad1400b980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-09T13:51:40.402 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:40.394+0000 7fad23fff640 1 --2- 172.21.15.43:0/1063502866 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fad14005280 0x7fad1400b980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:51:40.402 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:40.394+0000 7fad23fff640 1 -- 172.21.15.43:0/1063502866 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fad14003520 con 0x7fad140047c0 2023-08-09T13:51:40.402 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:40.394+0000 7fad293fb640 1 -- 172.21.15.43:0/1063502866 reap_dead start 2023-08-09T13:51:40.403 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:40.395+0000 7fad23fff640 1 --2- 172.21.15.43:0/1063502866 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fad140047c0 0x7fad14004c40 secure :-1 s=READY pgs=1256 cs=0 l=1 rev1=1 crypto rx=0x7fad10001750 tx=0x7fad140057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=98191434cc995de1 server_cookie=0 in_seq=0 out_seq=0 2023-08-09T13:51:40.403 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:40.396+0000 7fad237fe640 1 -- 172.21.15.43:0/1063502866 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fad100052d0 con 0x7fad140047c0 2023-08-09T13:51:40.403 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:40.396+0000 7fad237fe640 1 -- 172.21.15.43:0/1063502866 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fad1002e050 con 0x7fad140047c0 2023-08-09T13:51:40.403 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:40.396+0000 7fad237fe640 1 -- 172.21.15.43:0/1063502866 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fad10003830 con 0x7fad140047c0 2023-08-09T13:51:40.403 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:40.396+0000 7fad2ae5e640 1 -- 172.21.15.43:0/1063502866 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fad140047c0 msgr2=0x7fad14004c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:51:40.403 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:40.396+0000 7fad2ae5e640 1 --2- 172.21.15.43:0/1063502866 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fad140047c0 0x7fad14004c40 secure :-1 s=READY pgs=1256 cs=0 l=1 rev1=1 crypto rx=0x7fad10001750 tx=0x7fad140057f0 comp rx=0 tx=0).stop 2023-08-09T13:51:40.404 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:40.396+0000 7fad2ae5e640 1 -- 172.21.15.43:0/1063502866 shutdown_connections 2023-08-09T13:51:40.404 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:40.396+0000 7fad2ae5e640 1 --2- 172.21.15.43:0/1063502866 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fad140047c0 0x7fad14004c40 unknown :-1 s=CLOSED pgs=1256 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:51:40.404 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:40.396+0000 7fad2ae5e640 1 -- 172.21.15.43:0/1063502866 >> 172.21.15.43:0/1063502866 conn(0x7fad240ce0a0 msgr2=0x7fad240ce480 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-09T13:51:40.404 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:40.396+0000 7fad2ae5e640 1 -- 172.21.15.43:0/1063502866 shutdown_connections 2023-08-09T13:51:40.404 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:40.396+0000 7fad2ae5e640 1 -- 172.21.15.43:0/1063502866 wait complete. 2023-08-09T13:51:40.404 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:40.398+0000 7fad2ae5e640 1 Processor -- start 2023-08-09T13:51:40.404 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:40.399+0000 7fad2ae5e640 1 -- start start 2023-08-09T13:51:40.406 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:40.401+0000 7fad2ae5e640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fad140047c0 0x7fad2415d4a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:51:40.407 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:40.401+0000 7fad2ae5e640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fad24159a50 0x7fad240b2870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:51:40.407 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:40.401+0000 7fad2ae5e640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fad240aeb90 0x7fad240af010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:51:40.407 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:40.401+0000 7fad2ae5e640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fad24159ed0 con 0x7fad24159a50 2023-08-09T13:51:40.407 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:40.401+0000 7fad2ae5e640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fad240af550 con 0x7fad140047c0 2023-08-09T13:51:40.407 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:40.401+0000 7fad2ae5e640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fad240b30e0 con 0x7fad240aeb90 2023-08-09T13:51:40.408 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:40.401+0000 7fad23fff640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fad24159a50 0x7fad240b2870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:51:40.408 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:40.401+0000 7fad23fff640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fad24159a50 0x7fad240b2870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:41422/0 (socket says 172.21.15.43:41422) 2023-08-09T13:51:40.408 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:40.401+0000 7fad23fff640 1 -- 172.21.15.43:0/1925008767 learned_addr learned my addr 172.21.15.43:0/1925008767 (peer_addr_for_me v2:172.21.15.43:0/0) 2023-08-09T13:51:40.408 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:40.401+0000 7fad28bfa640 1 --2- 172.21.15.43:0/1925008767 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fad140047c0 0x7fad2415d4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:51:40.408 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:40.401+0000 7fad293fb640 1 --2- 172.21.15.43:0/1925008767 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fad240aeb90 0x7fad240af010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:51:40.408 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:40.401+0000 7fad23fff640 1 -- 172.21.15.43:0/1925008767 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fad240aeb90 msgr2=0x7fad240af010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:51:40.409 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:40.401+0000 7fad23fff640 1 --2- 172.21.15.43:0/1925008767 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fad240aeb90 0x7fad240af010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:51:40.409 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:40.401+0000 7fad23fff640 1 -- 172.21.15.43:0/1925008767 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fad140047c0 msgr2=0x7fad2415d4a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:51:40.409 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:40.401+0000 7fad23fff640 1 --2- 172.21.15.43:0/1925008767 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fad140047c0 0x7fad2415d4a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:51:40.409 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:40.401+0000 7fad23fff640 1 -- 172.21.15.43:0/1925008767 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fad240b33c0 con 0x7fad24159a50 2023-08-09T13:51:40.409 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:40.401+0000 7fad28bfa640 1 --2- 172.21.15.43:0/1925008767 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fad140047c0 0x7fad2415d4a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-08-09T13:51:40.410 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:40.401+0000 7fad23fff640 1 --2- 172.21.15.43:0/1925008767 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fad24159a50 0x7fad240b2870 secure :-1 s=READY pgs=1257 cs=0 l=1 rev1=1 crypto rx=0x7fad1002bc00 tx=0x7fad10001120 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-09T13:51:40.410 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:40.401+0000 7fad217fa640 1 -- 172.21.15.43:0/1925008767 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fad100076c0 con 0x7fad24159a50 2023-08-09T13:51:40.410 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:40.401+0000 7fad217fa640 1 -- 172.21.15.43:0/1925008767 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fad10007a30 con 0x7fad24159a50 2023-08-09T13:51:40.410 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:40.401+0000 7fad217fa640 1 -- 172.21.15.43:0/1925008767 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fad10006be0 con 0x7fad24159a50 2023-08-09T13:51:40.410 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:40.401+0000 7fad2ae5e640 1 -- 172.21.15.43:0/1925008767 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fad240b36a0 con 0x7fad24159a50 2023-08-09T13:51:40.410 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:40.401+0000 7fad2ae5e640 1 -- 172.21.15.43:0/1925008767 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fad240b2db0 con 0x7fad24159a50 2023-08-09T13:51:40.410 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:40.403+0000 7fad2ae5e640 1 -- 172.21.15.43:0/1925008767 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fad240739f0 con 0x7fad24159a50 2023-08-09T13:51:40.411 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:40.404+0000 7fad217fa640 1 -- 172.21.15.43:0/1925008767 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 43) v1 ==== 88179+0+0 (secure 0 0 0) 0x7fad10006d40 con 0x7fad24159a50 2023-08-09T13:51:40.412 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:40.405+0000 7fad217fa640 1 --2- 172.21.15.43:0/1925008767 >> [v2:172.21.15.203:6832/1317108840,v1:172.21.15.203:6833/1317108840] conn(0x7fad08069ea0 0x7fad0806c360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:51:40.412 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:40.405+0000 7fad217fa640 1 -- 172.21.15.43:0/1925008767 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(105..105 src has 1..105) v4 ==== 7114+0+0 (secure 0 0 0) 0x7fad100aa240 con 0x7fad24159a50 2023-08-09T13:51:40.412 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:40.405+0000 7fad28bfa640 1 --2- 172.21.15.43:0/1925008767 >> [v2:172.21.15.203:6832/1317108840,v1:172.21.15.203:6833/1317108840] conn(0x7fad08069ea0 0x7fad0806c360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:51:40.412 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:40.408+0000 7fad28bfa640 1 --2- 172.21.15.43:0/1925008767 >> [v2:172.21.15.203:6832/1317108840,v1:172.21.15.203:6833/1317108840] conn(0x7fad08069ea0 0x7fad0806c360 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7fad18005a70 tx=0x7fad180016b0 comp rx=0 tx=0).ready entity=mgr.6619 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-09T13:51:40.413 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:40.410+0000 7fad217fa640 1 -- 172.21.15.43:0/1925008767 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178066 (secure 0 0 0) 0x7fad100792e0 con 0x7fad24159a50 2023-08-09T13:51:40.571 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:40.569+0000 7fad217fa640 1 -- 172.21.15.43:0/1925008767 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mgrmap(e 44) v1 ==== 88387+0+0 (secure 0 0 0) 0x7fad10075c20 con 0x7fad24159a50 2023-08-09T13:51:40.610 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:40.608+0000 7fad2ae5e640 1 -- 172.21.15.43:0/1925008767 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7fad240cec90 con 0x7fad24159a50 2023-08-09T13:51:40.612 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:40.609+0000 7fad217fa640 1 -- 172.21.15.43:0/1925008767 <== mon.0 v2:172.21.15.43:3300/0 8 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v105) v1 ==== 104+0+11 (secure 0 0 0) 0x7fad10079050 con 0x7fad24159a50 2023-08-09T13:51:40.612 INFO:teuthology.orchestra.run.smithi043.stdout:pg_num: 32 2023-08-09T13:51:40.614 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:40.612+0000 7fad2ae5e640 1 -- 172.21.15.43:0/1925008767 >> [v2:172.21.15.203:6832/1317108840,v1:172.21.15.203:6833/1317108840] conn(0x7fad08069ea0 msgr2=0x7fad0806c360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:51:40.615 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:40.612+0000 7fad2ae5e640 1 --2- 172.21.15.43:0/1925008767 >> [v2:172.21.15.203:6832/1317108840,v1:172.21.15.203:6833/1317108840] conn(0x7fad08069ea0 0x7fad0806c360 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7fad18005a70 tx=0x7fad180016b0 comp rx=0 tx=0).stop 2023-08-09T13:51:40.615 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:40.612+0000 7fad2ae5e640 1 -- 172.21.15.43:0/1925008767 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fad24159a50 msgr2=0x7fad240b2870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:51:40.615 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:40.612+0000 7fad2ae5e640 1 --2- 172.21.15.43:0/1925008767 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fad24159a50 0x7fad240b2870 secure :-1 s=READY pgs=1257 cs=0 l=1 rev1=1 crypto rx=0x7fad1002bc00 tx=0x7fad10001120 comp rx=0 tx=0).stop 2023-08-09T13:51:40.615 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:40.612+0000 7fad2ae5e640 1 -- 172.21.15.43:0/1925008767 shutdown_connections 2023-08-09T13:51:40.615 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:40.612+0000 7fad2ae5e640 1 --2- 172.21.15.43:0/1925008767 >> [v2:172.21.15.203:6832/1317108840,v1:172.21.15.203:6833/1317108840] conn(0x7fad08069ea0 0x7fad0806c360 unknown :-1 s=CLOSED pgs=23 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:51:40.615 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:40.612+0000 7fad2ae5e640 1 --2- 172.21.15.43:0/1925008767 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fad240aeb90 0x7fad240af010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:51:40.616 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:40.612+0000 7fad2ae5e640 1 --2- 172.21.15.43:0/1925008767 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fad24159a50 0x7fad240b2870 unknown :-1 s=CLOSED pgs=1257 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:51:40.616 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:40.612+0000 7fad2ae5e640 1 --2- 172.21.15.43:0/1925008767 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fad140047c0 0x7fad2415d4a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:51:40.616 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:40.612+0000 7fad2ae5e640 1 -- 172.21.15.43:0/1925008767 >> 172.21.15.43:0/1925008767 conn(0x7fad240ce0a0 msgr2=0x7fad240c8ad0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-09T13:51:40.616 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:40.612+0000 7fad2ae5e640 1 -- 172.21.15.43:0/1925008767 shutdown_connections 2023-08-09T13:51:40.616 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:40.612+0000 7fad2ae5e640 1 -- 172.21.15.43:0/1925008767 wait complete. 2023-08-09T13:51:40.634 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-08-09T13:51:40.866 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:40.863+0000 7f10fd8f9640 1 Processor -- start 2023-08-09T13:51:40.866 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:40.863+0000 7f10fd8f9640 1 -- start start 2023-08-09T13:51:40.867 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:40.865+0000 7f10fd8f9640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f10f80b7df0 0x7f10f80d1b30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:51:40.868 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:40.865+0000 7f10fd8f9640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f10f80cebf0 0x7f10f80cf060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:51:40.868 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:40.865+0000 7f10fd8f9640 1 -- --> v1:172.21.15.43:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f10f80d16a0 con 0x7f10f815c2d0 2023-08-09T13:51:40.868 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:40.865+0000 7f10fd8f9640 1 -- --> v2:172.21.15.43:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f10f80b8230 con 0x7f10f80cebf0 2023-08-09T13:51:40.868 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:40.865+0000 7f10fd8f9640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f10f80cf5a0 con 0x7f10f80b7df0 2023-08-09T13:51:40.869 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:40.865+0000 7f10fd0f8640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f10f80cebf0 0x7f10f80cf060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:51:40.869 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:40.866+0000 7f10fd0f8640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f10f80cebf0 0x7f10f80cf060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:41436/0 (socket says 172.21.15.43:41436) 2023-08-09T13:51:40.869 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:40.866+0000 7f10fd0f8640 1 -- 172.21.15.43:0/3392721511 learned_addr learned my addr 172.21.15.43:0/3392721511 (peer_addr_for_me v2:172.21.15.43:0/0) 2023-08-09T13:51:40.870 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:40.866+0000 7f10f7fff640 1 --2- 172.21.15.43:0/3392721511 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f10f80b7df0 0x7f10f80d1b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:51:40.870 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:40.866+0000 7f10fd0f8640 1 -- 172.21.15.43:0/3392721511 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f10f80b7df0 msgr2=0x7f10f80d1b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-09T13:51:40.870 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:40.866+0000 7f10fd0f8640 1 --2- 172.21.15.43:0/3392721511 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f10f80b7df0 0x7f10f80d1b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:51:40.870 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:40.866+0000 7f10fd0f8640 1 -- 172.21.15.43:0/3392721511 >> v1:172.21.15.43:6789/0 conn(0x7f10f815c2d0 legacy=0x7f10f80cd020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:51:40.870 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:40.866+0000 7f10fd0f8640 1 -- 172.21.15.43:0/3392721511 --> v2:172.21.15.43:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f10f80d0d70 con 0x7f10f80cebf0 2023-08-09T13:51:40.870 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:40.866+0000 7f10f7fff640 1 --2- 172.21.15.43:0/3392721511 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f10f80b7df0 0x7f10f80d1b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-08-09T13:51:40.871 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:40.867+0000 7f10fd0f8640 1 --2- 172.21.15.43:0/3392721511 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f10f80cebf0 0x7f10f80cf060 secure :-1 s=READY pgs=1259 cs=0 l=1 rev1=1 crypto rx=0x7f10f0009e50 tx=0x7f10f002bf60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6a86f02957e85e03 server_cookie=0 in_seq=0 out_seq=0 2023-08-09T13:51:40.872 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:40.868+0000 7f10f77fe640 1 -- 172.21.15.43:0/3392721511 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f10f002cb00 con 0x7f10f80cebf0 2023-08-09T13:51:40.872 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:40.868+0000 7f10f77fe640 1 -- 172.21.15.43:0/3392721511 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f10f002cc60 con 0x7f10f80cebf0 2023-08-09T13:51:40.872 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:40.868+0000 7f10f77fe640 1 -- 172.21.15.43:0/3392721511 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f10f00343f0 con 0x7f10f80cebf0 2023-08-09T13:51:40.872 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:40.868+0000 7f10fd8f9640 1 -- 172.21.15.43:0/3392721511 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f10f80cebf0 msgr2=0x7f10f80cf060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:51:40.872 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:40.868+0000 7f10fd8f9640 1 --2- 172.21.15.43:0/3392721511 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f10f80cebf0 0x7f10f80cf060 secure :-1 s=READY pgs=1259 cs=0 l=1 rev1=1 crypto rx=0x7f10f0009e50 tx=0x7f10f002bf60 comp rx=0 tx=0).stop 2023-08-09T13:51:40.872 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:40.869+0000 7f10fd8f9640 1 -- 172.21.15.43:0/3392721511 shutdown_connections 2023-08-09T13:51:40.873 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:40.869+0000 7f10fd8f9640 1 --2- 172.21.15.43:0/3392721511 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f10f80cebf0 0x7f10f80cf060 unknown :-1 s=CLOSED pgs=1259 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:51:40.873 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:40.869+0000 7f10fd8f9640 1 --2- 172.21.15.43:0/3392721511 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f10f80b7df0 0x7f10f80d1b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:51:40.873 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:40.869+0000 7f10fd8f9640 1 -- 172.21.15.43:0/3392721511 >> 172.21.15.43:0/3392721511 conn(0x7f10f806a230 msgr2=0x7f10f806a640 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-09T13:51:40.873 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:40.869+0000 7f10fd8f9640 1 -- 172.21.15.43:0/3392721511 shutdown_connections 2023-08-09T13:51:40.873 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:40.869+0000 7f10fd8f9640 1 -- 172.21.15.43:0/3392721511 wait complete. 2023-08-09T13:51:40.874 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:40.872+0000 7f10fd8f9640 1 Processor -- start 2023-08-09T13:51:40.875 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:40.872+0000 7f10fd8f9640 1 -- start start 2023-08-09T13:51:40.875 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:40.873+0000 7f10fd8f9640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f10f80b7df0 0x7f10f8158dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:51:40.875 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:40.873+0000 7f10fd8f9640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f10f80cebf0 0x7f10f8153010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:51:40.875 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:40.873+0000 7f10fd8f9640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f10f815c2d0 0x7f10f81f18c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:51:40.875 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:40.873+0000 7f10fd8f9640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f10f81f51f0 con 0x7f10f80cebf0 2023-08-09T13:51:40.876 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:40.873+0000 7f10fd8f9640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f10f81f4ea0 con 0x7f10f815c2d0 2023-08-09T13:51:40.876 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:40.873+0000 7f10fd8f9640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f10f8153550 con 0x7f10f80b7df0 2023-08-09T13:51:40.876 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:40.873+0000 7f10f7fff640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f10f80cebf0 0x7f10f8153010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:51:40.876 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:40.873+0000 7f10f7fff640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f10f80cebf0 0x7f10f8153010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:41442/0 (socket says 172.21.15.43:41442) 2023-08-09T13:51:40.876 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:40.873+0000 7f10f7fff640 1 -- 172.21.15.43:0/794378051 learned_addr learned my addr 172.21.15.43:0/794378051 (peer_addr_for_me v2:172.21.15.43:0/0) 2023-08-09T13:51:40.878 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:40.873+0000 7f10fc8f7640 1 --2- 172.21.15.43:0/794378051 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f10f80b7df0 0x7f10f8158dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:51:40.878 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:40.873+0000 7f10f7fff640 1 -- 172.21.15.43:0/794378051 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f10f80b7df0 msgr2=0x7f10f8158dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:51:40.878 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:40.873+0000 7f10fd0f8640 1 --2- 172.21.15.43:0/794378051 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f10f815c2d0 0x7f10f81f18c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:51:40.878 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:40.873+0000 7f10f7fff640 1 --2- 172.21.15.43:0/794378051 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f10f80b7df0 0x7f10f8158dc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:51:40.878 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:40.873+0000 7f10f7fff640 1 -- 172.21.15.43:0/794378051 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f10f815c2d0 msgr2=0x7f10f81f18c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:51:40.879 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:40.873+0000 7f10f7fff640 1 --2- 172.21.15.43:0/794378051 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f10f815c2d0 0x7f10f81f18c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:51:40.879 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:40.873+0000 7f10f7fff640 1 -- 172.21.15.43:0/794378051 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f10f0009b30 con 0x7f10f80cebf0 2023-08-09T13:51:40.879 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:40.873+0000 7f10f7fff640 1 --2- 172.21.15.43:0/794378051 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f10f80cebf0 0x7f10f8153010 secure :-1 s=READY pgs=1260 cs=0 l=1 rev1=1 crypto rx=0x7f10e400c7f0 tx=0x7f10e400cbf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-09T13:51:40.879 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:40.873+0000 7f10f57fa640 1 -- 172.21.15.43:0/794378051 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f10e4007a30 con 0x7f10f80cebf0 2023-08-09T13:51:40.880 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:40.873+0000 7f10f57fa640 1 -- 172.21.15.43:0/794378051 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f10e4007b90 con 0x7f10f80cebf0 2023-08-09T13:51:40.880 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:40.873+0000 7f10f57fa640 1 -- 172.21.15.43:0/794378051 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f10e400ce40 con 0x7f10f80cebf0 2023-08-09T13:51:40.880 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:40.874+0000 7f10fd8f9640 1 -- 172.21.15.43:0/794378051 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f10f81f2190 con 0x7f10f80cebf0 2023-08-09T13:51:40.880 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:40.874+0000 7f10fd8f9640 1 -- 172.21.15.43:0/794378051 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f10f81f2750 con 0x7f10f80cebf0 2023-08-09T13:51:40.881 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:40.875+0000 7f10fd8f9640 1 -- 172.21.15.43:0/794378051 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f10f81f5360 con 0x7f10f80cebf0 2023-08-09T13:51:40.882 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:40.876+0000 7f10f57fa640 1 -- 172.21.15.43:0/794378051 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 44) v1 ==== 88387+0+0 (secure 0 0 0) 0x7f10e40113f0 con 0x7f10f80cebf0 2023-08-09T13:51:40.882 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:40.878+0000 7f10f57fa640 1 --2- 172.21.15.43:0/794378051 >> [v2:172.21.15.203:6832/1317108840,v1:172.21.15.203:6833/1317108840] conn(0x7f10dc06a0e0 0x7f10dc06c5a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:51:40.883 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:40.878+0000 7f10f57fa640 1 -- 172.21.15.43:0/794378051 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(105..105 src has 1..105) v4 ==== 7114+0+0 (secure 0 0 0) 0x7f10e4088d30 con 0x7f10f80cebf0 2023-08-09T13:51:40.883 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:40.878+0000 7f10fc8f7640 1 --2- 172.21.15.43:0/794378051 >> [v2:172.21.15.203:6832/1317108840,v1:172.21.15.203:6833/1317108840] conn(0x7f10dc06a0e0 0x7f10dc06c5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:51:40.883 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:40.879+0000 7f10fc8f7640 1 --2- 172.21.15.43:0/794378051 >> [v2:172.21.15.203:6832/1317108840,v1:172.21.15.203:6833/1317108840] conn(0x7f10dc06a0e0 0x7f10dc06c5a0 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7f10ec000f60 tx=0x7f10ec009040 comp rx=0 tx=0).ready entity=mgr.6619 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-09T13:51:40.884 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:40.882+0000 7f10f57fa640 1 -- 172.21.15.43:0/794378051 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178066 (secure 0 0 0) 0x7f10e4053e00 con 0x7f10f80cebf0 2023-08-09T13:51:41.077 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:41.074+0000 7f10fd8f9640 1 -- 172.21.15.43:0/794378051 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f10f804e390 con 0x7f10f80cebf0 2023-08-09T13:51:41.078 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:41.075+0000 7f10f57fa640 1 -- 172.21.15.43:0/794378051 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v105) v1 ==== 100+0+11 (secure 0 0 0) 0x7f10e401f0a0 con 0x7f10f80cebf0 2023-08-09T13:51:41.078 INFO:teuthology.orchestra.run.smithi043.stdout:pg_num: 64 2023-08-09T13:51:41.080 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:41.078+0000 7f10fd8f9640 1 -- 172.21.15.43:0/794378051 >> [v2:172.21.15.203:6832/1317108840,v1:172.21.15.203:6833/1317108840] conn(0x7f10dc06a0e0 msgr2=0x7f10dc06c5a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:51:41.080 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:41.078+0000 7f10fd8f9640 1 --2- 172.21.15.43:0/794378051 >> [v2:172.21.15.203:6832/1317108840,v1:172.21.15.203:6833/1317108840] conn(0x7f10dc06a0e0 0x7f10dc06c5a0 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7f10ec000f60 tx=0x7f10ec009040 comp rx=0 tx=0).stop 2023-08-09T13:51:41.081 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:41.078+0000 7f10fd8f9640 1 -- 172.21.15.43:0/794378051 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f10f80cebf0 msgr2=0x7f10f8153010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:51:41.081 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:41.078+0000 7f10fd8f9640 1 --2- 172.21.15.43:0/794378051 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f10f80cebf0 0x7f10f8153010 secure :-1 s=READY pgs=1260 cs=0 l=1 rev1=1 crypto rx=0x7f10e400c7f0 tx=0x7f10e400cbf0 comp rx=0 tx=0).stop 2023-08-09T13:51:41.081 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:41.078+0000 7f10fd8f9640 1 -- 172.21.15.43:0/794378051 shutdown_connections 2023-08-09T13:51:41.081 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:41.078+0000 7f10fd8f9640 1 --2- 172.21.15.43:0/794378051 >> [v2:172.21.15.203:6832/1317108840,v1:172.21.15.203:6833/1317108840] conn(0x7f10dc06a0e0 0x7f10dc06c5a0 unknown :-1 s=CLOSED pgs=25 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:51:41.081 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:41.078+0000 7f10fd8f9640 1 --2- 172.21.15.43:0/794378051 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f10f815c2d0 0x7f10f81f18c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:51:41.081 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:41.078+0000 7f10fd8f9640 1 --2- 172.21.15.43:0/794378051 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f10f80cebf0 0x7f10f8153010 unknown :-1 s=CLOSED pgs=1260 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:51:41.081 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:41.078+0000 7f10fd8f9640 1 --2- 172.21.15.43:0/794378051 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f10f80b7df0 0x7f10f8158dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:51:41.082 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:41.078+0000 7f10fd8f9640 1 -- 172.21.15.43:0/794378051 >> 172.21.15.43:0/794378051 conn(0x7f10f806a230 msgr2=0x7f10f80cccc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-09T13:51:41.082 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:41.078+0000 7f10fd8f9640 1 -- 172.21.15.43:0/794378051 shutdown_connections 2023-08-09T13:51:41.082 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:41.078+0000 7f10fd8f9640 1 -- 172.21.15.43:0/794378051 wait complete. 2023-08-09T13:51:41.098 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-08-09T13:51:41.329 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:41.328+0000 7f35fac5e640 1 Processor -- start 2023-08-09T13:51:41.330 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:41.328+0000 7f35fac5e640 1 -- start start 2023-08-09T13:51:41.331 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:41.328+0000 7f35fac5e640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f35f4143c20 0x7f35f406f540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:51:41.331 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:41.328+0000 7f35fac5e640 1 -- --> v1:172.21.15.43:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f35f406c440 con 0x7f35f40c86f0 2023-08-09T13:51:41.331 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:41.328+0000 7f35fac5e640 1 -- --> v1:172.21.15.203:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f35f406b830 con 0x7f35f406cfe0 2023-08-09T13:51:41.331 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:41.328+0000 7f35fac5e640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f35f4144000 con 0x7f35f4143c20 2023-08-09T13:51:41.332 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:41.329+0000 7f35f3fff640 1 --1- >> v1:172.21.15.43:6789/0 conn(0x7f35f40c86f0 0x7f35f406c8d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.43:6789/0 says I am v1:172.21.15.43:60828/0 (socket says 172.21.15.43:60828) 2023-08-09T13:51:41.332 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:41.329+0000 7f35f3fff640 1 -- 172.21.15.43:0/4225493206 learned_addr learned my addr 172.21.15.43:0/4225493206 (peer_addr_for_me v1:172.21.15.43:0/0) 2023-08-09T13:51:41.333 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:41.329+0000 7f35f89fa640 1 --2- 172.21.15.43:0/4225493206 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f35f4143c20 0x7f35f406f540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:51:41.333 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:41.330+0000 7f35f37fe640 1 -- 172.21.15.43:0/4225493206 <== mon.0 v1:172.21.15.43:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1741070976 0 0) 0x7f35f406c440 con 0x7f35f40c86f0 2023-08-09T13:51:41.333 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:41.330+0000 7f35f37fe640 1 -- 172.21.15.43:0/4225493206 --> v1:172.21.15.43:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f35dc003520 con 0x7f35f40c86f0 2023-08-09T13:51:41.334 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:41.330+0000 7f35f37fe640 1 -- 172.21.15.43:0/4225493206 <== mon.0 v1:172.21.15.43:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3498597016 0 0) 0x7f35dc003520 con 0x7f35f40c86f0 2023-08-09T13:51:41.335 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:41.330+0000 7f35f37fe640 1 -- 172.21.15.43:0/4225493206 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f35f4143c20 msgr2=0x7f35f406f540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-09T13:51:41.335 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:41.330+0000 7f35f37fe640 1 --2- 172.21.15.43:0/4225493206 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f35f4143c20 0x7f35f406f540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:51:41.335 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:41.330+0000 7f35f37fe640 1 -- 172.21.15.43:0/4225493206 >> v1:172.21.15.203:6789/0 conn(0x7f35f406cfe0 legacy=0x7f35f40ccc00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:51:41.335 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:41.330+0000 7f35f37fe640 1 -- 172.21.15.43:0/4225493206 --> v1:172.21.15.43:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f35f406bb10 con 0x7f35f40c86f0 2023-08-09T13:51:41.335 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:41.330+0000 7f35f37fe640 1 -- 172.21.15.43:0/4225493206 <== mon.0 v1:172.21.15.43:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1285150740 0 0) 0x7f35e80019f0 con 0x7f35f40c86f0 2023-08-09T13:51:41.336 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:41.330+0000 7f35f89fa640 1 --2- 172.21.15.43:0/4225493206 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f35f4143c20 0x7f35f406f540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-08-09T13:51:41.336 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:41.330+0000 7f35f37fe640 1 -- 172.21.15.43:0/4225493206 >> v1:172.21.15.43:6789/0 conn(0x7f35f40c86f0 legacy=0x7f35f406c8d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:51:41.336 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:41.330+0000 7f35f37fe640 1 --2- 172.21.15.43:0/4225493206 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f35f4143c20 0x7f35dc003a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:51:41.336 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:41.330+0000 7f35f37fe640 1 --2- 172.21.15.43:0/4225493206 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f35dc003f50 0x7f35dc0047c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:51:41.336 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:41.330+0000 7f35f37fe640 1 --2- 172.21.15.43:0/4225493206 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f35dc004d00 0x7f35dc005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:51:41.336 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:41.330+0000 7f35f37fe640 1 -- 172.21.15.43:0/4225493206 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f35f4144000 con 0x7f35dc004d00 2023-08-09T13:51:41.336 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:41.330+0000 7f35f37fe640 1 -- 172.21.15.43:0/4225493206 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f35dc00bb10 con 0x7f35dc003f50 2023-08-09T13:51:41.337 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:41.330+0000 7f35f37fe640 1 -- 172.21.15.43:0/4225493206 --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f35dc00bca0 con 0x7f35f4143c20 2023-08-09T13:51:41.337 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:41.331+0000 7f35f91fb640 1 --2- 172.21.15.43:0/4225493206 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f35dc004d00 0x7f35dc005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:51:41.337 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:41.331+0000 7f35f89fa640 1 --2- 172.21.15.43:0/4225493206 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f35f4143c20 0x7f35dc003a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:51:41.337 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:41.331+0000 7f35f3fff640 1 --2- 172.21.15.43:0/4225493206 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f35dc003f50 0x7f35dc0047c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:51:41.338 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:41.331+0000 7f35f91fb640 1 -- 172.21.15.43:0/4225493206 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f35f4143c20 msgr2=0x7f35dc003a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-09T13:51:41.338 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:41.331+0000 7f35f91fb640 1 --2- 172.21.15.43:0/4225493206 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f35f4143c20 0x7f35dc003a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:51:41.338 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:41.331+0000 7f35f91fb640 1 -- 172.21.15.43:0/4225493206 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f35dc003f50 msgr2=0x7f35dc0047c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-09T13:51:41.338 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:41.331+0000 7f35f91fb640 1 --2- 172.21.15.43:0/4225493206 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f35dc003f50 0x7f35dc0047c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:51:41.338 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:41.331+0000 7f35f91fb640 1 -- 172.21.15.43:0/4225493206 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f35e4009400 con 0x7f35dc004d00 2023-08-09T13:51:41.339 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:41.331+0000 7f35f3fff640 1 --2- 172.21.15.43:0/4225493206 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f35dc003f50 0x7f35dc0047c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-08-09T13:51:41.339 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:41.332+0000 7f35f91fb640 1 --2- 172.21.15.43:0/4225493206 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f35dc004d00 0x7f35dc005180 secure :-1 s=READY pgs=1262 cs=0 l=1 rev1=1 crypto rx=0x7f35ec001830 tx=0x7f35dc0057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=67e63a20d84d21b6 server_cookie=0 in_seq=0 out_seq=0 2023-08-09T13:51:41.341 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:41.333+0000 7f35f37fe640 1 -- 172.21.15.43:0/4225493206 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f35ec02f050 con 0x7f35dc004d00 2023-08-09T13:51:41.341 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:41.333+0000 7f35f37fe640 1 -- 172.21.15.43:0/4225493206 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f35ec033610 con 0x7f35dc004d00 2023-08-09T13:51:41.342 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:41.333+0000 7f35f37fe640 1 -- 172.21.15.43:0/4225493206 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f35ec0339c0 con 0x7f35dc004d00 2023-08-09T13:51:41.342 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:41.333+0000 7f35fac5e640 1 -- 172.21.15.43:0/4225493206 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f35dc004d00 msgr2=0x7f35dc005180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:51:41.342 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:41.333+0000 7f35fac5e640 1 --2- 172.21.15.43:0/4225493206 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f35dc004d00 0x7f35dc005180 secure :-1 s=READY pgs=1262 cs=0 l=1 rev1=1 crypto rx=0x7f35ec001830 tx=0x7f35dc0057f0 comp rx=0 tx=0).stop 2023-08-09T13:51:41.342 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:41.333+0000 7f35f91fb640 1 -- 172.21.15.43:0/4225493206 reap_dead start 2023-08-09T13:51:41.342 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:41.333+0000 7f35fac5e640 1 -- 172.21.15.43:0/4225493206 shutdown_connections 2023-08-09T13:51:41.342 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:41.333+0000 7f35fac5e640 1 -- 172.21.15.43:0/4225493206 >> 172.21.15.43:0/4225493206 conn(0x7f35f40ce0a0 msgr2=0x7f35f40ce480 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-09T13:51:41.342 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:41.333+0000 7f35fac5e640 1 -- 172.21.15.43:0/4225493206 shutdown_connections 2023-08-09T13:51:41.343 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:41.333+0000 7f35fac5e640 1 -- 172.21.15.43:0/4225493206 wait complete. 2023-08-09T13:51:41.343 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:41.335+0000 7f35fac5e640 1 Processor -- start 2023-08-09T13:51:41.343 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:41.335+0000 7f35fac5e640 1 -- start start 2023-08-09T13:51:41.343 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:41.336+0000 7f35fac5e640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f35dc004d00 0x7f35f41f0bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:51:41.343 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:41.337+0000 7f35fac5e640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f35f40b3850 0x7f35f40b3cd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:51:41.343 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:41.337+0000 7f35fac5e640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f35f40b0ff0 0x7f35f40b1470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:51:41.344 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:41.337+0000 7f35fac5e640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f35f40b4210 con 0x7f35f40b0ff0 2023-08-09T13:51:41.344 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:41.337+0000 7f35fac5e640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f35f40b1d60 con 0x7f35f40b3850 2023-08-09T13:51:41.344 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:41.337+0000 7f35fac5e640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f35f40b2040 con 0x7f35dc004d00 2023-08-09T13:51:41.344 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:41.337+0000 7f35f91fb640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f35f40b0ff0 0x7f35f40b1470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:51:41.344 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:41.337+0000 7f35f91fb640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f35f40b0ff0 0x7f35f40b1470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:41464/0 (socket says 172.21.15.43:41464) 2023-08-09T13:51:41.344 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:41.337+0000 7f35f91fb640 1 -- 172.21.15.43:0/39672789 learned_addr learned my addr 172.21.15.43:0/39672789 (peer_addr_for_me v2:172.21.15.43:0/0) 2023-08-09T13:51:41.344 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:41.337+0000 7f35f89fa640 1 --2- 172.21.15.43:0/39672789 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f35dc004d00 0x7f35f41f0bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:51:41.345 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:41.337+0000 7f35f3fff640 1 --2- 172.21.15.43:0/39672789 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f35f40b3850 0x7f35f40b3cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:51:41.345 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:41.337+0000 7f35f91fb640 1 -- 172.21.15.43:0/39672789 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f35dc004d00 msgr2=0x7f35f41f0bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:51:41.345 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:41.337+0000 7f35f91fb640 1 --2- 172.21.15.43:0/39672789 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f35dc004d00 0x7f35f41f0bd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:51:41.345 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:41.337+0000 7f35f91fb640 1 -- 172.21.15.43:0/39672789 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f35f40b3850 msgr2=0x7f35f40b3cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:51:41.345 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:41.337+0000 7f35f91fb640 1 --2- 172.21.15.43:0/39672789 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f35f40b3850 0x7f35f40b3cd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:51:41.345 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:41.337+0000 7f35f91fb640 1 -- 172.21.15.43:0/39672789 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f35f40b2320 con 0x7f35f40b0ff0 2023-08-09T13:51:41.345 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:41.338+0000 7f35f91fb640 1 --2- 172.21.15.43:0/39672789 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f35f40b0ff0 0x7f35f40b1470 secure :-1 s=READY pgs=1263 cs=0 l=1 rev1=1 crypto rx=0x7f35ec001950 tx=0x7f35ec001230 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-09T13:51:41.345 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:41.338+0000 7f35f17fa640 1 -- 172.21.15.43:0/39672789 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f35ec02f070 con 0x7f35f40b0ff0 2023-08-09T13:51:41.346 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:41.338+0000 7f35f17fa640 1 -- 172.21.15.43:0/39672789 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f35ec0054d0 con 0x7f35f40b0ff0 2023-08-09T13:51:41.346 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:41.338+0000 7f35f17fa640 1 -- 172.21.15.43:0/39672789 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f35ec007260 con 0x7f35f40b0ff0 2023-08-09T13:51:41.346 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:41.338+0000 7f35fac5e640 1 -- 172.21.15.43:0/39672789 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f35f40b2600 con 0x7f35f40b0ff0 2023-08-09T13:51:41.346 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:41.338+0000 7f35fac5e640 1 -- 172.21.15.43:0/39672789 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f35f40b19b0 con 0x7f35f40b0ff0 2023-08-09T13:51:41.347 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:41.340+0000 7f35fac5e640 1 -- 172.21.15.43:0/39672789 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f35f40b2b20 con 0x7f35f40b0ff0 2023-08-09T13:51:41.347 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:41.341+0000 7f35f17fa640 1 -- 172.21.15.43:0/39672789 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 44) v1 ==== 88387+0+0 (secure 0 0 0) 0x7f35ec0073c0 con 0x7f35f40b0ff0 2023-08-09T13:51:41.347 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:41.342+0000 7f35f17fa640 1 --2- 172.21.15.43:0/39672789 >> [v2:172.21.15.203:6832/1317108840,v1:172.21.15.203:6833/1317108840] conn(0x7f35e006a0e0 0x7f35e006c5a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:51:41.347 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:41.342+0000 7f35f17fa640 1 -- 172.21.15.43:0/39672789 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(105..105 src has 1..105) v4 ==== 7114+0+0 (secure 0 0 0) 0x7f35ec0a9340 con 0x7f35f40b0ff0 2023-08-09T13:51:41.347 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:41.342+0000 7f35f89fa640 1 --2- 172.21.15.43:0/39672789 >> [v2:172.21.15.203:6832/1317108840,v1:172.21.15.203:6833/1317108840] conn(0x7f35e006a0e0 0x7f35e006c5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:51:41.348 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:41.343+0000 7f35f89fa640 1 --2- 172.21.15.43:0/39672789 >> [v2:172.21.15.203:6832/1317108840,v1:172.21.15.203:6833/1317108840] conn(0x7f35e006a0e0 0x7f35e006c5a0 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7f35f40c8320 tx=0x7f35e4008040 comp rx=0 tx=0).ready entity=mgr.6619 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-09T13:51:41.348 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:41.346+0000 7f35f17fa640 1 -- 172.21.15.43:0/39672789 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178066 (secure 0 0 0) 0x7f35f40b2b20 con 0x7f35f40b0ff0 2023-08-09T13:51:41.585 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:41.581+0000 7f35fac5e640 1 -- 172.21.15.43:0/39672789 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f35f40b2b20 con 0x7f35f40b0ff0 2023-08-09T13:51:41.586 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:41.582+0000 7f35f17fa640 1 -- 172.21.15.43:0/39672789 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 43 v43) v1 ==== 94+0+4880 (secure 0 0 0) 0x7f35ec07b320 con 0x7f35f40b0ff0 2023-08-09T13:51:41.586 INFO:teuthology.orchestra.run.smithi043.stdout: 2023-08-09T13:51:41.586 INFO:teuthology.orchestra.run.smithi043.stdout:{"epoch":43,"default_fscid":5,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5966,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.203:6835/4101044758","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6834","nonce":4101044758},{"type":"v1","addr":"172.21.15.203:6835","nonce":4101044758}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":39},{"gid":6460,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.203:6837/2974664437","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6836","nonce":2974664437},{"type":"v1","addr":"172.21.15.203:6837","nonce":2974664437}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":39},{"gid":7527,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.43:6833/795245144","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6832","nonce":795245144},{"type":"v1","addr":"172.21.15.43:6833","nonce":795245144}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":38}],"filesystems":[{"mdsmap":{"epoch":43,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-08-09T13:51:15.716741+0000","modified":"2023-08-09T13:51:19.249966+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":7542},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7542":{"gid":7542,"name":"a","rank":0,"incarnation":40,"state":"up:active","state_seq":3,"addr":"172.21.15.43:6835/3008795294","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6834","nonce":3008795294},{"type":"v1","addr":"172.21.15.43:6835","nonce":3008795294}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[13],"metadata_pool":12,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":5}]} 2023-08-09T13:51:41.587 INFO:teuthology.orchestra.run.smithi043.stderr:dumped fsmap epoch 43 2023-08-09T13:51:41.587 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:41.585+0000 7f35fac5e640 1 -- 172.21.15.43:0/39672789 >> [v2:172.21.15.203:6832/1317108840,v1:172.21.15.203:6833/1317108840] conn(0x7f35e006a0e0 msgr2=0x7f35e006c5a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:51:41.587 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:41.585+0000 7f35fac5e640 1 --2- 172.21.15.43:0/39672789 >> [v2:172.21.15.203:6832/1317108840,v1:172.21.15.203:6833/1317108840] conn(0x7f35e006a0e0 0x7f35e006c5a0 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7f35f40c8320 tx=0x7f35e4008040 comp rx=0 tx=0).stop 2023-08-09T13:51:41.590 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:41.585+0000 7f35fac5e640 1 -- 172.21.15.43:0/39672789 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f35f40b0ff0 msgr2=0x7f35f40b1470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:51:41.590 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:41.585+0000 7f35fac5e640 1 --2- 172.21.15.43:0/39672789 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f35f40b0ff0 0x7f35f40b1470 secure :-1 s=READY pgs=1263 cs=0 l=1 rev1=1 crypto rx=0x7f35ec001950 tx=0x7f35ec001230 comp rx=0 tx=0).stop 2023-08-09T13:51:41.590 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:41.585+0000 7f35fac5e640 1 -- 172.21.15.43:0/39672789 shutdown_connections 2023-08-09T13:51:41.590 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:41.585+0000 7f35fac5e640 1 --2- 172.21.15.43:0/39672789 >> [v2:172.21.15.203:6832/1317108840,v1:172.21.15.203:6833/1317108840] conn(0x7f35e006a0e0 0x7f35e006c5a0 unknown :-1 s=CLOSED pgs=26 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:51:41.591 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:41.585+0000 7f35fac5e640 1 --2- 172.21.15.43:0/39672789 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f35f40b0ff0 0x7f35f40b1470 unknown :-1 s=CLOSED pgs=1263 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:51:41.591 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:41.585+0000 7f35fac5e640 1 --2- 172.21.15.43:0/39672789 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f35f40b3850 0x7f35f40b3cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:51:41.591 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:41.585+0000 7f35fac5e640 1 --2- 172.21.15.43:0/39672789 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f35dc004d00 0x7f35f41f0bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:51:41.591 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:41.585+0000 7f35fac5e640 1 -- 172.21.15.43:0/39672789 >> 172.21.15.43:0/39672789 conn(0x7f35f40ce0a0 msgr2=0x7f35f40c4430 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-09T13:51:41.591 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:41.585+0000 7f35fac5e640 1 -- 172.21.15.43:0/39672789 shutdown_connections 2023-08-09T13:51:41.591 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:41.585+0000 7f35fac5e640 1 -- 172.21.15.43:0/39672789 wait complete. 2023-08-09T13:51:41.608 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-08-09T13:51:41.833 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:41.831+0000 7f93a0f99640 1 Processor -- start 2023-08-09T13:51:41.833 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:41.831+0000 7f93a0f99640 1 -- start start 2023-08-09T13:51:41.836 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:41.833+0000 7f93a0f99640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f939c143c20 0x7f939c06f540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:51:41.836 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:41.833+0000 7f93a0f99640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f939c06cfe0 0x7f939c0c8b50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:51:41.837 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:41.833+0000 7f93a0f99640 1 -- --> v1:172.21.15.203:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f939c0c9090 con 0x7f939c0c86f0 2023-08-09T13:51:41.837 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:41.833+0000 7f93a0f99640 1 -- --> v2:172.21.15.43:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f939c144000 con 0x7f939c143c20 2023-08-09T13:51:41.837 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:41.833+0000 7f93a0f99640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f939c06c440 con 0x7f939c06cfe0 2023-08-09T13:51:41.837 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:41.833+0000 7f939a59c640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f939c143c20 0x7f939c06f540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:51:41.837 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:41.833+0000 7f939ad9d640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f939c06cfe0 0x7f939c0c8b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:51:41.838 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:41.833+0000 7f9391d9b640 1 --1- >> v1:172.21.15.203:6789/0 conn(0x7f939c0c86f0 0x7f939c06c8d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.203:6789/0 says I am v1:172.21.15.43:51418/0 (socket says 172.21.15.43:51418) 2023-08-09T13:51:41.838 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:41.833+0000 7f9391d9b640 1 -- 172.21.15.43:0/509291139 learned_addr learned my addr 172.21.15.43:0/509291139 (peer_addr_for_me v1:172.21.15.43:0/0) 2023-08-09T13:51:41.838 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:41.833+0000 7f939a59c640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f939c143c20 0x7f939c06f540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:41478/0 (socket says 172.21.15.43:41478) 2023-08-09T13:51:41.839 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:41.834+0000 7f939a59c640 1 -- 172.21.15.43:0/509291139 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f939c06cfe0 msgr2=0x7f939c0c8b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-09T13:51:41.839 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:41.834+0000 7f939a59c640 1 --2- 172.21.15.43:0/509291139 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f939c06cfe0 0x7f939c0c8b50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:51:41.839 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:41.834+0000 7f939a59c640 1 -- 172.21.15.43:0/509291139 >> v1:172.21.15.203:6789/0 conn(0x7f939c0c86f0 legacy=0x7f939c06c8d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:51:41.839 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:41.834+0000 7f939a59c640 1 -- 172.21.15.43:0/509291139 --> v2:172.21.15.43:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f939c06b830 con 0x7f939c143c20 2023-08-09T13:51:41.839 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:41.837+0000 7f939a59c640 1 --2- 172.21.15.43:0/509291139 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f939c143c20 0x7f939c06f540 secure :-1 s=READY pgs=1264 cs=0 l=1 rev1=1 crypto rx=0x7f938c0097a0 tx=0x7f938c02b840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=79e11a89eba80f27 server_cookie=0 in_seq=0 out_seq=0 2023-08-09T13:51:41.839 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:41.837+0000 7f9399d9b640 1 -- 172.21.15.43:0/509291139 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f938c02e070 con 0x7f939c143c20 2023-08-09T13:51:41.839 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:41.837+0000 7f9399d9b640 1 -- 172.21.15.43:0/509291139 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f938c0315a0 con 0x7f939c143c20 2023-08-09T13:51:41.840 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:41.837+0000 7f9399d9b640 1 -- 172.21.15.43:0/509291139 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f938c031930 con 0x7f939c143c20 2023-08-09T13:51:41.841 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:41.838+0000 7f93a0f99640 1 -- 172.21.15.43:0/509291139 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f939c143c20 msgr2=0x7f939c06f540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:51:41.841 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:41.838+0000 7f93a0f99640 1 --2- 172.21.15.43:0/509291139 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f939c143c20 0x7f939c06f540 secure :-1 s=READY pgs=1264 cs=0 l=1 rev1=1 crypto rx=0x7f938c0097a0 tx=0x7f938c02b840 comp rx=0 tx=0).stop 2023-08-09T13:51:41.842 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:41.838+0000 7f93a0f99640 1 -- 172.21.15.43:0/509291139 shutdown_connections 2023-08-09T13:51:41.842 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:41.838+0000 7f93a0f99640 1 --2- 172.21.15.43:0/509291139 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f939c06cfe0 0x7f939c0c8b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:51:41.842 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:41.838+0000 7f93a0f99640 1 --2- 172.21.15.43:0/509291139 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f939c143c20 0x7f939c06f540 unknown :-1 s=CLOSED pgs=1264 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:51:41.842 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:41.838+0000 7f93a0f99640 1 -- 172.21.15.43:0/509291139 >> 172.21.15.43:0/509291139 conn(0x7f939c0ce0a0 msgr2=0x7f939c0ce480 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-09T13:51:41.842 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:41.838+0000 7f93a0f99640 1 -- 172.21.15.43:0/509291139 shutdown_connections 2023-08-09T13:51:41.844 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:41.838+0000 7f93a0f99640 1 -- 172.21.15.43:0/509291139 wait complete. 2023-08-09T13:51:41.844 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:41.840+0000 7f93a0f99640 1 Processor -- start 2023-08-09T13:51:41.844 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:41.840+0000 7f93a0f99640 1 -- start start 2023-08-09T13:51:41.845 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:41.841+0000 7f93a0f99640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f939c06cfe0 0x7f939c13dd50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:51:41.845 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:41.841+0000 7f93a0f99640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f939c0c86f0 0x7f939c0cade0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:51:41.845 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:41.841+0000 7f93a0f99640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f939c143c20 0x7f939c0cb320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:51:41.845 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:41.841+0000 7f93a0f99640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f939c0b1940 con 0x7f939c143c20 2023-08-09T13:51:41.845 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:41.841+0000 7f93a0f99640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f939c13e290 con 0x7f939c0c86f0 2023-08-09T13:51:41.845 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:41.841+0000 7f93a0f99640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f939c0c7410 con 0x7f939c06cfe0 2023-08-09T13:51:41.845 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:41.841+0000 7f939ad9d640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f939c143c20 0x7f939c0cb320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:51:41.845 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:41.841+0000 7f939ad9d640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f939c143c20 0x7f939c0cb320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:41484/0 (socket says 172.21.15.43:41484) 2023-08-09T13:51:41.846 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:41.841+0000 7f939ad9d640 1 -- 172.21.15.43:0/4050010707 learned_addr learned my addr 172.21.15.43:0/4050010707 (peer_addr_for_me v2:172.21.15.43:0/0) 2023-08-09T13:51:41.846 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:41.842+0000 7f939a59c640 1 --2- 172.21.15.43:0/4050010707 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f939c06cfe0 0x7f939c13dd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:51:41.846 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:41.842+0000 7f9391d9b640 1 --2- 172.21.15.43:0/4050010707 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f939c0c86f0 0x7f939c0cade0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:51:41.846 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:41.842+0000 7f939ad9d640 1 -- 172.21.15.43:0/4050010707 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f939c06cfe0 msgr2=0x7f939c13dd50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:51:41.846 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:41.842+0000 7f939ad9d640 1 --2- 172.21.15.43:0/4050010707 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f939c06cfe0 0x7f939c13dd50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:51:41.846 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:41.842+0000 7f939ad9d640 1 -- 172.21.15.43:0/4050010707 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f939c0c86f0 msgr2=0x7f939c0cade0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:51:41.846 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:41.842+0000 7f939ad9d640 1 --2- 172.21.15.43:0/4050010707 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f939c0c86f0 0x7f939c0cade0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:51:41.847 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:41.842+0000 7f939ad9d640 1 -- 172.21.15.43:0/4050010707 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f938c009400 con 0x7f939c143c20 2023-08-09T13:51:41.847 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:41.842+0000 7f9391d9b640 1 --2- 172.21.15.43:0/4050010707 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f939c0c86f0 0x7f939c0cade0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-08-09T13:51:41.847 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:41.842+0000 7f939ad9d640 1 --2- 172.21.15.43:0/4050010707 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f939c143c20 0x7f939c0cb320 secure :-1 s=READY pgs=1265 cs=0 l=1 rev1=1 crypto rx=0x7f939400ec10 tx=0x7f939400c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-09T13:51:41.847 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:41.842+0000 7f93937fe640 1 -- 172.21.15.43:0/4050010707 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f939400cd20 con 0x7f939c143c20 2023-08-09T13:51:41.847 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:41.842+0000 7f93937fe640 1 -- 172.21.15.43:0/4050010707 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f939400f040 con 0x7f939c143c20 2023-08-09T13:51:41.847 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:41.842+0000 7f93937fe640 1 -- 172.21.15.43:0/4050010707 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f93940146c0 con 0x7f939c143c20 2023-08-09T13:51:41.847 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:41.842+0000 7f93a0f99640 1 -- 172.21.15.43:0/4050010707 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f939c0c76f0 con 0x7f939c143c20 2023-08-09T13:51:41.847 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:41.842+0000 7f93a0f99640 1 -- 172.21.15.43:0/4050010707 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f939c0c7cb0 con 0x7f939c143c20 2023-08-09T13:51:41.848 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:41.844+0000 7f93a0f99640 1 -- 172.21.15.43:0/4050010707 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f939c0b1ab0 con 0x7f939c143c20 2023-08-09T13:51:41.848 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:41.844+0000 7f93937fe640 1 -- 172.21.15.43:0/4050010707 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 44) v1 ==== 88387+0+0 (secure 0 0 0) 0x7f9394014a30 con 0x7f939c143c20 2023-08-09T13:51:41.849 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:41.846+0000 7f93937fe640 1 --2- 172.21.15.43:0/4050010707 >> [v2:172.21.15.203:6832/1317108840,v1:172.21.15.203:6833/1317108840] conn(0x7f937c06a0e0 0x7f937c06c5a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:51:41.849 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:41.846+0000 7f93937fe640 1 -- 172.21.15.43:0/4050010707 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(105..105 src has 1..105) v4 ==== 7114+0+0 (secure 0 0 0) 0x7f93940898c0 con 0x7f939c143c20 2023-08-09T13:51:41.849 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:41.846+0000 7f939a59c640 1 --2- 172.21.15.43:0/4050010707 >> [v2:172.21.15.203:6832/1317108840,v1:172.21.15.203:6833/1317108840] conn(0x7f937c06a0e0 0x7f937c06c5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:51:41.849 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:41.848+0000 7f939a59c640 1 --2- 172.21.15.43:0/4050010707 >> [v2:172.21.15.203:6832/1317108840,v1:172.21.15.203:6833/1317108840] conn(0x7f937c06a0e0 0x7f937c06c5a0 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7f938c036000 tx=0x7f938c03d040 comp rx=0 tx=0).ready entity=mgr.6619 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-09T13:51:41.851 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:41.850+0000 7f93937fe640 1 -- 172.21.15.43:0/4050010707 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178066 (secure 0 0 0) 0x7f9394054900 con 0x7f939c143c20 2023-08-09T13:51:42.049 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:42.047+0000 7f93a0f99640 1 -- 172.21.15.43:0/4050010707 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f939c0739f0 con 0x7f939c143c20 2023-08-09T13:51:42.052 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:42.050+0000 7f93937fe640 1 -- 172.21.15.43:0/4050010707 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v105) v1 ==== 74+0+19573 (secure 0 0 0) 0x7f9394018030 con 0x7f939c143c20 2023-08-09T13:51:42.053 INFO:teuthology.orchestra.run.smithi043.stdout: 2023-08-09T13:51:42.053 INFO:teuthology.orchestra.run.smithi043.stdout:{"epoch":105,"fsid":"daac87ee-1337-4250-a955-aea4e408a39b","created":"2023-08-09T13:33:41.701198+0000","modified":"2023-08-09T13:51:38.531070+0000","last_up_change":"2023-08-09T13:33:53.783453+0000","last_in_change":"2023-08-09T13:33:45.148904+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":13,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":"rbd","create_time":"2023-08-09T13:33:54.365137+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":".mgr","create_time":"2023-08-09T13:33:54.831803+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":12,"pool_name":"cephfs_metadata","create_time":"2023-08-09T13:51:14.142563+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"102","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":13,"pool_name":"cephfs_data","create_time":"2023-08-09T13:51:14.712668+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"103","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"cbd49ecd-e896-4fbd-864e-5c37a829b165","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":101,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6800","nonce":502667335},{"type":"v1","addr":"172.21.15.43:6801","nonce":502667335}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6802","nonce":502667335},{"type":"v1","addr":"172.21.15.43:6803","nonce":502667335}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6806","nonce":502667335},{"type":"v1","addr":"172.21.15.43:6807","nonce":502667335}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6804","nonce":502667335},{"type":"v1","addr":"172.21.15.43:6805","nonce":502667335}]},"public_addr":"172.21.15.43:6801/502667335","cluster_addr":"172.21.15.43:6803/502667335","heartbeat_back_addr":"172.21.15.43:6807/502667335","heartbeat_front_addr":"172.21.15.43:6805/502667335","state":["exists","up"]},{"osd":1,"uuid":"75c7e0fe-1ec7-4fe8-a2f2-3661a65b8500","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":101,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6808","nonce":3289134610},{"type":"v1","addr":"172.21.15.43:6809","nonce":3289134610}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6810","nonce":3289134610},{"type":"v1","addr":"172.21.15.43:6812","nonce":3289134610}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6821","nonce":3289134610},{"type":"v1","addr":"172.21.15.43:6824","nonce":3289134610}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6815","nonce":3289134610},{"type":"v1","addr":"172.21.15.43:6818","nonce":3289134610}]},"public_addr":"172.21.15.43:6809/3289134610","cluster_addr":"172.21.15.43:6812/3289134610","heartbeat_back_addr":"172.21.15.43:6824/3289134610","heartbeat_front_addr":"172.21.15.43:6818/3289134610","state":["exists","up"]},{"osd":2,"uuid":"bb5c41e9-7f5f-4125-9b01-3bb8bfc2ea83","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":101,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6811","nonce":4079709736},{"type":"v1","addr":"172.21.15.43:6814","nonce":4079709736}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6817","nonce":4079709736},{"type":"v1","addr":"172.21.15.43:6820","nonce":4079709736}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6829","nonce":4079709736},{"type":"v1","addr":"172.21.15.43:6831","nonce":4079709736}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6823","nonce":4079709736},{"type":"v1","addr":"172.21.15.43:6826","nonce":4079709736}]},"public_addr":"172.21.15.43:6814/4079709736","cluster_addr":"172.21.15.43:6820/4079709736","heartbeat_back_addr":"172.21.15.43:6831/4079709736","heartbeat_front_addr":"172.21.15.43:6826/4079709736","state":["exists","up"]},{"osd":3,"uuid":"bfe4c8b4-694a-465c-87e6-2b2aab852613","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":101,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6813","nonce":1709538759},{"type":"v1","addr":"172.21.15.43:6816","nonce":1709538759}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6819","nonce":1709538759},{"type":"v1","addr":"172.21.15.43:6822","nonce":1709538759}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6828","nonce":1709538759},{"type":"v1","addr":"172.21.15.43:6830","nonce":1709538759}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6825","nonce":1709538759},{"type":"v1","addr":"172.21.15.43:6827","nonce":1709538759}]},"public_addr":"172.21.15.43:6816/1709538759","cluster_addr":"172.21.15.43:6822/1709538759","heartbeat_back_addr":"172.21.15.43:6830/1709538759","heartbeat_front_addr":"172.21.15.43:6827/1709538759","state":["exists","up"]},{"osd":4,"uuid":"fa2e8a77-aba1-4756-853e-0d174a5bdeba","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":101,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6800","nonce":4196866481},{"type":"v1","addr":"172.21.15.203:6801","nonce":4196866481}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6802","nonce":4196866481},{"type":"v1","addr":"172.21.15.203:6803","nonce":4196866481}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6806","nonce":4196866481},{"type":"v1","addr":"172.21.15.203:6807","nonce":4196866481}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6804","nonce":4196866481},{"type":"v1","addr":"172.21.15.203:6805","nonce":4196866481}]},"public_addr":"172.21.15.203:6801/4196866481","cluster_addr":"172.21.15.203:6803/4196866481","heartbeat_back_addr":"172.21.15.203:6807/4196866481","heartbeat_front_addr":"172.21.15.203:6805/4196866481","state":["exists","up"]},{"osd":5,"uuid":"81811e4b-4d24-4f51-a82c-f1d700a2a00d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":101,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6816","nonce":3059290081},{"type":"v1","addr":"172.21.15.203:6817","nonce":3059290081}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6818","nonce":3059290081},{"type":"v1","addr":"172.21.15.203:6819","nonce":3059290081}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6822","nonce":3059290081},{"type":"v1","addr":"172.21.15.203:6823","nonce":3059290081}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6820","nonce":3059290081},{"type":"v1","addr":"172.21.15.203:6821","nonce":3059290081}]},"public_addr":"172.21.15.203:6817/3059290081","cluster_addr":"172.21.15.203:6819/3059290081","heartbeat_back_addr":"172.21.15.203:6823/3059290081","heartbeat_front_addr":"172.21.15.203:6821/3059290081","state":["exists","up"]},{"osd":6,"uuid":"307bac4d-6a5b-443d-af75-56d695d7674f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":101,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6824","nonce":512155569},{"type":"v1","addr":"172.21.15.203:6825","nonce":512155569}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6826","nonce":512155569},{"type":"v1","addr":"172.21.15.203:6827","nonce":512155569}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6830","nonce":512155569},{"type":"v1","addr":"172.21.15.203:6831","nonce":512155569}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6828","nonce":512155569},{"type":"v1","addr":"172.21.15.203:6829","nonce":512155569}]},"public_addr":"172.21.15.203:6825/512155569","cluster_addr":"172.21.15.203:6827/512155569","heartbeat_back_addr":"172.21.15.203:6831/512155569","heartbeat_front_addr":"172.21.15.203:6829/512155569","state":["exists","up"]},{"osd":7,"uuid":"a95dc7ca-9f1b-4295-af70-68a22a2d7d5e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":101,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6808","nonce":3331450438},{"type":"v1","addr":"172.21.15.203:6809","nonce":3331450438}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6810","nonce":3331450438},{"type":"v1","addr":"172.21.15.203:6811","nonce":3331450438}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6814","nonce":3331450438},{"type":"v1","addr":"172.21.15.203:6815","nonce":3331450438}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6812","nonce":3331450438},{"type":"v1","addr":"172.21.15.203:6813","nonce":3331450438}]},"public_addr":"172.21.15.203:6809/3331450438","cluster_addr":"172.21.15.203:6811/3331450438","heartbeat_back_addr":"172.21.15.203:6815/3331450438","heartbeat_front_addr":"172.21.15.203:6813/3331450438","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-09T13:33:50.547652+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-09T13:33:50.589469+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-09T13:33:50.561724+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-09T13:33:50.599650+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-09T13:33:50.411855+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-09T13:33:50.219681+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-09T13:33:50.504664+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-09T13:33:50.441248+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.203:0/996706743":"2023-08-10T13:51:38.530957+0000","172.21.15.203:0/969846951":"2023-08-10T13:51:38.530957+0000","172.21.15.203:6833/2675832148":"2023-08-10T13:51:38.530957+0000","172.21.15.203:6832/2675832148":"2023-08-10T13:51:38.530957+0000","172.21.15.203:0/822053193":"2023-08-10T13:51:38.530957+0000","172.21.15.203:0/1475395073":"2023-08-10T13:51:38.530957+0000","172.21.15.203:0/1343041460":"2023-08-10T13:51:32.420350+0000","172.21.15.203:6833/1778614695":"2023-08-10T13:51:32.420350+0000","172.21.15.203:0/3761332698":"2023-08-10T13:51:32.420350+0000","172.21.15.203:6832/1778614695":"2023-08-10T13:51:32.420350+0000","172.21.15.203:0/2293720318":"2023-08-10T13:51:38.530957+0000","172.21.15.203:0/2054133158":"2023-08-10T13:51:32.420350+0000","172.21.15.203:0/3225794224":"2023-08-10T13:51:38.530957+0000","172.21.15.203:0/1496080019":"2023-08-10T13:51:32.420350+0000","172.21.15.203:0/1894709586":"2023-08-10T13:51:32.420350+0000","172.21.15.203:0/270998287":"2023-08-10T13:51:32.420350+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":7,"snaps":[{"begin":104,"length":1},{"begin":221,"length":1}]}],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-08-09T13:51:42.054 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:42.053+0000 7f93a0f99640 1 -- 172.21.15.43:0/4050010707 >> [v2:172.21.15.203:6832/1317108840,v1:172.21.15.203:6833/1317108840] conn(0x7f937c06a0e0 msgr2=0x7f937c06c5a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:51:42.055 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:42.053+0000 7f93a0f99640 1 --2- 172.21.15.43:0/4050010707 >> [v2:172.21.15.203:6832/1317108840,v1:172.21.15.203:6833/1317108840] conn(0x7f937c06a0e0 0x7f937c06c5a0 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7f938c036000 tx=0x7f938c03d040 comp rx=0 tx=0).stop 2023-08-09T13:51:42.055 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:42.053+0000 7f93a0f99640 1 -- 172.21.15.43:0/4050010707 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f939c143c20 msgr2=0x7f939c0cb320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:51:42.055 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:42.053+0000 7f93a0f99640 1 --2- 172.21.15.43:0/4050010707 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f939c143c20 0x7f939c0cb320 secure :-1 s=READY pgs=1265 cs=0 l=1 rev1=1 crypto rx=0x7f939400ec10 tx=0x7f939400c490 comp rx=0 tx=0).stop 2023-08-09T13:51:42.056 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:42.053+0000 7f93a0f99640 1 -- 172.21.15.43:0/4050010707 shutdown_connections 2023-08-09T13:51:42.056 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:42.053+0000 7f93a0f99640 1 --2- 172.21.15.43:0/4050010707 >> [v2:172.21.15.203:6832/1317108840,v1:172.21.15.203:6833/1317108840] conn(0x7f937c06a0e0 0x7f937c06c5a0 unknown :-1 s=CLOSED pgs=27 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:51:42.056 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:42.053+0000 7f93a0f99640 1 --2- 172.21.15.43:0/4050010707 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f939c143c20 0x7f939c0cb320 unknown :-1 s=CLOSED pgs=1265 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:51:42.056 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:42.053+0000 7f93a0f99640 1 --2- 172.21.15.43:0/4050010707 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f939c0c86f0 0x7f939c0cade0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:51:42.056 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:42.053+0000 7f93a0f99640 1 --2- 172.21.15.43:0/4050010707 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f939c06cfe0 0x7f939c13dd50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:51:42.056 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:42.053+0000 7f93a0f99640 1 -- 172.21.15.43:0/4050010707 >> 172.21.15.43:0/4050010707 conn(0x7f939c0ce0a0 msgr2=0x7f939c15ac40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-09T13:51:42.057 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:42.053+0000 7f93a0f99640 1 -- 172.21.15.43:0/4050010707 shutdown_connections 2023-08-09T13:51:42.057 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:42.054+0000 7f93a0f99640 1 -- 172.21.15.43:0/4050010707 wait complete. 2023-08-09T13:51:42.075 INFO:tasks.cephfs.filesystem:Destroying file system cephfs and related pools 2023-08-09T13:51:42.075 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-08-09T13:51:42.307 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:42.305+0000 7f417ebcb640 1 Processor -- start 2023-08-09T13:51:42.307 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:42.305+0000 7f417ebcb640 1 -- start start 2023-08-09T13:51:42.309 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:42.307+0000 7f417ebcb640 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7f417815c630 0x7f41780abef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:51:42.310 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:42.307+0000 7f417ebcb640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f41780cdc60 0x7f417815ca10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:51:42.310 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:42.307+0000 7f417ebcb640 1 -- --> v1:172.21.15.203:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f41780cca90 con 0x7f41780ccf20 2023-08-09T13:51:42.310 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:42.307+0000 7f417ebcb640 1 -- --> v2:172.21.15.43:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f41780cbc70 con 0x7f41780cdc60 2023-08-09T13:51:42.311 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:42.307+0000 7f417ebcb640 1 -- --> v2:172.21.15.203:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f41780cbe80 con 0x7f417815c630 2023-08-09T13:51:42.311 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:42.307+0000 7f417d168640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f41780cdc60 0x7f417815ca10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:51:42.311 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:42.307+0000 7f417d168640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f41780cdc60 0x7f417815ca10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:41488/0 (socket says 172.21.15.43:41488) 2023-08-09T13:51:42.311 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:42.307+0000 7f417d168640 1 -- 172.21.15.43:0/3673373001 learned_addr learned my addr 172.21.15.43:0/3673373001 (peer_addr_for_me v2:172.21.15.43:0/0) 2023-08-09T13:51:42.312 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:42.308+0000 7f417c967640 1 --2- 172.21.15.43:0/3673373001 >> v2:172.21.15.203:3300/0 conn(0x7f417815c630 0x7f41780abef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:51:42.312 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:42.308+0000 7f417d168640 1 -- 172.21.15.43:0/3673373001 >> v2:172.21.15.203:3300/0 conn(0x7f417815c630 msgr2=0x7f41780abef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-09T13:51:42.312 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:42.308+0000 7f417d168640 1 --2- 172.21.15.43:0/3673373001 >> v2:172.21.15.203:3300/0 conn(0x7f417815c630 0x7f41780abef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:51:42.313 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:42.308+0000 7f417d168640 1 -- 172.21.15.43:0/3673373001 >> v1:172.21.15.203:6789/0 conn(0x7f41780ccf20 legacy=0x7f41780cd550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-09T13:51:42.313 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:42.308+0000 7f417d168640 1 -- 172.21.15.43:0/3673373001 --> v2:172.21.15.43:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f41780cc160 con 0x7f41780cdc60 2023-08-09T13:51:42.313 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:42.308+0000 7f417c967640 1 --2- 172.21.15.43:0/3673373001 >> v2:172.21.15.203:3300/0 conn(0x7f417815c630 0x7f41780abef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-08-09T13:51:42.318 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:42.309+0000 7f417d168640 1 --2- 172.21.15.43:0/3673373001 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f41780cdc60 0x7f417815ca10 secure :-1 s=READY pgs=1266 cs=0 l=1 rev1=1 crypto rx=0x7f41780cbbb0 tx=0x7f417002bee0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3580ea08ef06fbc8 server_cookie=0 in_seq=0 out_seq=0 2023-08-09T13:51:42.319 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:42.309+0000 7f41777fe640 1 -- 172.21.15.43:0/3673373001 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f417002ca80 con 0x7f41780cdc60 2023-08-09T13:51:42.319 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:42.309+0000 7f41777fe640 1 -- 172.21.15.43:0/3673373001 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f417002cbe0 con 0x7f41780cdc60 2023-08-09T13:51:42.319 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:42.309+0000 7f41777fe640 1 -- 172.21.15.43:0/3673373001 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f41700343f0 con 0x7f41780cdc60 2023-08-09T13:51:42.320 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:42.310+0000 7f417ebcb640 1 -- 172.21.15.43:0/3673373001 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f41780cdc60 msgr2=0x7f417815ca10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:51:42.320 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:42.310+0000 7f417ebcb640 1 --2- 172.21.15.43:0/3673373001 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f41780cdc60 0x7f417815ca10 secure :-1 s=READY pgs=1266 cs=0 l=1 rev1=1 crypto rx=0x7f41780cbbb0 tx=0x7f417002bee0 comp rx=0 tx=0).stop 2023-08-09T13:51:42.320 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:42.310+0000 7f417ebcb640 1 -- 172.21.15.43:0/3673373001 shutdown_connections 2023-08-09T13:51:42.321 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:42.310+0000 7f417ebcb640 1 --2- 172.21.15.43:0/3673373001 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f41780cdc60 0x7f417815ca10 unknown :-1 s=CLOSED pgs=1266 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:51:42.321 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:42.310+0000 7f417ebcb640 1 --2- 172.21.15.43:0/3673373001 >> v2:172.21.15.203:3300/0 conn(0x7f417815c630 0x7f41780abef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:51:42.321 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:42.310+0000 7f417ebcb640 1 -- 172.21.15.43:0/3673373001 >> 172.21.15.43:0/3673373001 conn(0x7f41780ce0a0 msgr2=0x7f41780ce480 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-09T13:51:42.322 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:42.310+0000 7f417ebcb640 1 -- 172.21.15.43:0/3673373001 shutdown_connections 2023-08-09T13:51:42.322 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:42.311+0000 7f417ebcb640 1 -- 172.21.15.43:0/3673373001 wait complete. 2023-08-09T13:51:42.322 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:42.313+0000 7f417ebcb640 1 Processor -- start 2023-08-09T13:51:42.322 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:42.313+0000 7f417ebcb640 1 -- start start 2023-08-09T13:51:42.323 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:42.314+0000 7f417ebcb640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f41780ccf20 0x7f41780adb00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:51:42.323 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:42.314+0000 7f417ebcb640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f41780cdc60 0x7f41781f1800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:51:42.323 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:42.314+0000 7f417ebcb640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f417815c630 0x7f41780705c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:51:42.324 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:42.314+0000 7f417ebcb640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f41781f1d40 con 0x7f41780ccf20 2023-08-09T13:51:42.324 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:42.314+0000 7f417ebcb640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f41780ae120 con 0x7f41780cdc60 2023-08-09T13:51:42.324 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:42.314+0000 7f417ebcb640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f41780ae400 con 0x7f417815c630 2023-08-09T13:51:42.324 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:42.314+0000 7f417c967640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f41780ccf20 0x7f41780adb00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:51:42.325 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:42.314+0000 7f417c967640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f41780ccf20 0x7f41780adb00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:41502/0 (socket says 172.21.15.43:41502) 2023-08-09T13:51:42.325 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:42.314+0000 7f417c967640 1 -- 172.21.15.43:0/1861395702 learned_addr learned my addr 172.21.15.43:0/1861395702 (peer_addr_for_me v2:172.21.15.43:0/0) 2023-08-09T13:51:42.326 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:42.314+0000 7f4177fff640 1 --2- 172.21.15.43:0/1861395702 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f41780cdc60 0x7f41781f1800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:51:42.326 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:42.315+0000 7f417d168640 1 --2- 172.21.15.43:0/1861395702 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f417815c630 0x7f41780705c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:51:42.326 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:42.315+0000 7f417c967640 1 -- 172.21.15.43:0/1861395702 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f417815c630 msgr2=0x7f41780705c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:51:42.326 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:42.315+0000 7f417c967640 1 --2- 172.21.15.43:0/1861395702 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f417815c630 0x7f41780705c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:51:42.326 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:42.315+0000 7f417c967640 1 -- 172.21.15.43:0/1861395702 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f41780cdc60 msgr2=0x7f41781f1800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:51:42.327 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:42.315+0000 7f417c967640 1 --2- 172.21.15.43:0/1861395702 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f41780cdc60 0x7f41781f1800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:51:42.327 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:42.315+0000 7f417c967640 1 -- 172.21.15.43:0/1861395702 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4170009b30 con 0x7f41780ccf20 2023-08-09T13:51:42.327 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:42.315+0000 7f417c967640 1 --2- 172.21.15.43:0/1861395702 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f41780ccf20 0x7f41780adb00 secure :-1 s=READY pgs=1267 cs=0 l=1 rev1=1 crypto rx=0x7f416c00c7f0 tx=0x7f416c00cbf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-09T13:51:42.327 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:42.315+0000 7f41757fa640 1 -- 172.21.15.43:0/1861395702 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f416c007a30 con 0x7f41780ccf20 2023-08-09T13:51:42.327 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:42.315+0000 7f41757fa640 1 -- 172.21.15.43:0/1861395702 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f416c007b90 con 0x7f41780ccf20 2023-08-09T13:51:42.327 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:42.315+0000 7f41757fa640 1 -- 172.21.15.43:0/1861395702 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f416c00ce40 con 0x7f41780ccf20 2023-08-09T13:51:42.328 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:42.315+0000 7f417ebcb640 1 -- 172.21.15.43:0/1861395702 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f41780ae6e0 con 0x7f41780ccf20 2023-08-09T13:51:42.328 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:42.316+0000 7f417ebcb640 1 -- 172.21.15.43:0/1861395702 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f417815d520 con 0x7f41780ccf20 2023-08-09T13:51:42.328 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:42.317+0000 7f417ebcb640 1 -- 172.21.15.43:0/1861395702 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f41780739f0 con 0x7f41780ccf20 2023-08-09T13:51:42.329 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:42.318+0000 7f41757fa640 1 -- 172.21.15.43:0/1861395702 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 44) v1 ==== 88387+0+0 (secure 0 0 0) 0x7f416c0113f0 con 0x7f41780ccf20 2023-08-09T13:51:42.329 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:42.319+0000 7f41757fa640 1 --2- 172.21.15.43:0/1861395702 >> [v2:172.21.15.203:6832/1317108840,v1:172.21.15.203:6833/1317108840] conn(0x7f415c06a0e0 0x7f415c06c5a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:51:42.329 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:42.319+0000 7f41757fa640 1 -- 172.21.15.43:0/1861395702 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(105..105 src has 1..105) v4 ==== 7114+0+0 (secure 0 0 0) 0x7f416c088cb0 con 0x7f41780ccf20 2023-08-09T13:51:42.329 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:42.320+0000 7f4177fff640 1 --2- 172.21.15.43:0/1861395702 >> [v2:172.21.15.203:6832/1317108840,v1:172.21.15.203:6833/1317108840] conn(0x7f415c06a0e0 0x7f415c06c5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:51:42.330 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:42.321+0000 7f4177fff640 1 --2- 172.21.15.43:0/1861395702 >> [v2:172.21.15.203:6832/1317108840,v1:172.21.15.203:6833/1317108840] conn(0x7f415c06a0e0 0x7f415c06c5a0 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7f4164006fd0 tx=0x7f4164008040 comp rx=0 tx=0).ready entity=mgr.6619 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-09T13:51:42.330 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:42.324+0000 7f41757fa640 1 -- 172.21.15.43:0/1861395702 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178066 (secure 0 0 0) 0x7f416c053e10 con 0x7f41780ccf20 2023-08-09T13:51:42.565 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:42.563+0000 7f417ebcb640 1 -- 172.21.15.43:0/1861395702 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f417815b870 con 0x7f41780ccf20 2023-08-09T13:51:42.566 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:42.565+0000 7f41757fa640 1 -- 172.21.15.43:0/1861395702 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 43 v43) v1 ==== 94+0+4880 (secure 0 0 0) 0x7f416c01f090 con 0x7f41780ccf20 2023-08-09T13:51:42.567 INFO:teuthology.orchestra.run.smithi043.stdout: 2023-08-09T13:51:42.567 INFO:teuthology.orchestra.run.smithi043.stdout:{"epoch":43,"default_fscid":5,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5966,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.203:6835/4101044758","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6834","nonce":4101044758},{"type":"v1","addr":"172.21.15.203:6835","nonce":4101044758}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":39},{"gid":6460,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.203:6837/2974664437","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6836","nonce":2974664437},{"type":"v1","addr":"172.21.15.203:6837","nonce":2974664437}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":39},{"gid":7527,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.43:6833/795245144","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6832","nonce":795245144},{"type":"v1","addr":"172.21.15.43:6833","nonce":795245144}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":38}],"filesystems":[{"mdsmap":{"epoch":43,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-08-09T13:51:15.716741+0000","modified":"2023-08-09T13:51:19.249966+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":7542},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7542":{"gid":7542,"name":"a","rank":0,"incarnation":40,"state":"up:active","state_seq":3,"addr":"172.21.15.43:6835/3008795294","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6834","nonce":3008795294},{"type":"v1","addr":"172.21.15.43:6835","nonce":3008795294}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[13],"metadata_pool":12,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":5}]} 2023-08-09T13:51:42.567 INFO:teuthology.orchestra.run.smithi043.stderr:dumped fsmap epoch 43 2023-08-09T13:51:42.569 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:42.568+0000 7f417ebcb640 1 -- 172.21.15.43:0/1861395702 >> [v2:172.21.15.203:6832/1317108840,v1:172.21.15.203:6833/1317108840] conn(0x7f415c06a0e0 msgr2=0x7f415c06c5a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:51:42.569 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:42.568+0000 7f417ebcb640 1 --2- 172.21.15.43:0/1861395702 >> [v2:172.21.15.203:6832/1317108840,v1:172.21.15.203:6833/1317108840] conn(0x7f415c06a0e0 0x7f415c06c5a0 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7f4164006fd0 tx=0x7f4164008040 comp rx=0 tx=0).stop 2023-08-09T13:51:42.570 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:42.568+0000 7f417ebcb640 1 -- 172.21.15.43:0/1861395702 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f41780ccf20 msgr2=0x7f41780adb00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:51:42.570 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:42.568+0000 7f417ebcb640 1 --2- 172.21.15.43:0/1861395702 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f41780ccf20 0x7f41780adb00 secure :-1 s=READY pgs=1267 cs=0 l=1 rev1=1 crypto rx=0x7f416c00c7f0 tx=0x7f416c00cbf0 comp rx=0 tx=0).stop 2023-08-09T13:51:42.570 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:42.568+0000 7f417ebcb640 1 -- 172.21.15.43:0/1861395702 shutdown_connections 2023-08-09T13:51:42.570 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:42.568+0000 7f417ebcb640 1 --2- 172.21.15.43:0/1861395702 >> [v2:172.21.15.203:6832/1317108840,v1:172.21.15.203:6833/1317108840] conn(0x7f415c06a0e0 0x7f415c06c5a0 unknown :-1 s=CLOSED pgs=28 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:51:42.570 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:42.568+0000 7f417ebcb640 1 --2- 172.21.15.43:0/1861395702 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f417815c630 0x7f41780705c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:51:42.570 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:42.568+0000 7f417ebcb640 1 --2- 172.21.15.43:0/1861395702 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f41780cdc60 0x7f41781f1800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:51:42.570 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:42.568+0000 7f417ebcb640 1 --2- 172.21.15.43:0/1861395702 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f41780ccf20 0x7f41780adb00 unknown :-1 s=CLOSED pgs=1267 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:51:42.571 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:42.568+0000 7f417ebcb640 1 -- 172.21.15.43:0/1861395702 >> 172.21.15.43:0/1861395702 conn(0x7f41780ce0a0 msgr2=0x7f41780c87d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-09T13:51:42.571 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:42.568+0000 7f417ebcb640 1 -- 172.21.15.43:0/1861395702 shutdown_connections 2023-08-09T13:51:42.571 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:42.568+0000 7f417ebcb640 1 -- 172.21.15.43:0/1861395702 wait complete. 2023-08-09T13:51:42.590 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-08-09T13:51:42.820 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:42.819+0000 7f739643f640 1 Processor -- start 2023-08-09T13:51:42.821 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:42.819+0000 7f739643f640 1 -- start start 2023-08-09T13:51:42.822 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:42.821+0000 7f739643f640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f73900b2790 0x7f73900d2ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:51:42.822 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:42.821+0000 7f739643f640 1 -- --> v1:172.21.15.43:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f73900ce090 con 0x7f73900b2b70 2023-08-09T13:51:42.822 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:42.821+0000 7f739643f640 1 -- --> v1:172.21.15.203:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f73900cd7f0 con 0x7f73900b3050 2023-08-09T13:51:42.823 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:42.821+0000 7f739643f640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f73900b3950 con 0x7f73900b2790 2023-08-09T13:51:42.823 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:42.821+0000 7f738f7fe640 1 --1- >> v1:172.21.15.43:6789/0 conn(0x7f73900b2b70 0x7f73900b5330 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.43:6789/0 says I am v1:172.21.15.43:60842/0 (socket says 172.21.15.43:60842) 2023-08-09T13:51:42.823 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:42.821+0000 7f738f7fe640 1 -- 172.21.15.43:0/3372903783 learned_addr learned my addr 172.21.15.43:0/3372903783 (peer_addr_for_me v1:172.21.15.43:0/0) 2023-08-09T13:51:42.824 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:42.822+0000 7f738ffff640 1 --2- 172.21.15.43:0/3372903783 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f73900b2790 0x7f73900d2ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:51:42.824 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:42.822+0000 7f738effd640 1 -- 172.21.15.43:0/3372903783 <== mon.0 v1:172.21.15.43:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3521656352 0 0) 0x7f73900ce090 con 0x7f73900b2b70 2023-08-09T13:51:42.824 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:42.822+0000 7f738effd640 1 -- 172.21.15.43:0/3372903783 --> v1:172.21.15.43:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f7378003520 con 0x7f73900b2b70 2023-08-09T13:51:42.825 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:42.822+0000 7f738effd640 1 -- 172.21.15.43:0/3372903783 <== mon.0 v1:172.21.15.43:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3062540092 0 0) 0x7f7378003520 con 0x7f73900b2b70 2023-08-09T13:51:42.825 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:42.822+0000 7f738effd640 1 -- 172.21.15.43:0/3372903783 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f73900b2790 msgr2=0x7f73900d2ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-09T13:51:42.825 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:42.822+0000 7f738effd640 1 --2- 172.21.15.43:0/3372903783 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f73900b2790 0x7f73900d2ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:51:42.825 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:42.822+0000 7f738effd640 1 -- 172.21.15.43:0/3372903783 >> v1:172.21.15.203:6789/0 conn(0x7f73900b3050 legacy=0x7f73900b3f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:51:42.825 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:42.822+0000 7f738effd640 1 -- 172.21.15.43:0/3372903783 --> v1:172.21.15.43:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f73900cdad0 con 0x7f73900b2b70 2023-08-09T13:51:42.826 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:42.822+0000 7f738effd640 1 -- 172.21.15.43:0/3372903783 <== mon.0 v1:172.21.15.43:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1285150740 0 0) 0x7f73840019f0 con 0x7f73900b2b70 2023-08-09T13:51:42.826 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:42.822+0000 7f738ffff640 1 --2- 172.21.15.43:0/3372903783 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f73900b2790 0x7f73900d2ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-08-09T13:51:42.826 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:42.822+0000 7f738effd640 1 -- 172.21.15.43:0/3372903783 >> v1:172.21.15.43:6789/0 conn(0x7f73900b2b70 legacy=0x7f73900b5330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:51:42.826 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:42.822+0000 7f738effd640 1 --2- 172.21.15.43:0/3372903783 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f7378003a10 0x7f7378003df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:51:42.826 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:42.823+0000 7f738effd640 1 --2- 172.21.15.43:0/3372903783 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f73900b2790 0x7f73780047c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:51:42.826 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:42.823+0000 7f738effd640 1 --2- 172.21.15.43:0/3372903783 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f7378004d00 0x7f7378005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:51:42.826 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:42.823+0000 7f738effd640 1 -- 172.21.15.43:0/3372903783 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f73900b3950 con 0x7f7378003a10 2023-08-09T13:51:42.827 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:42.823+0000 7f738effd640 1 -- 172.21.15.43:0/3372903783 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f737800ba30 con 0x7f7378004d00 2023-08-09T13:51:42.827 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:42.823+0000 7f738effd640 1 -- 172.21.15.43:0/3372903783 --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f737800bb70 con 0x7f73900b2790 2023-08-09T13:51:42.827 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:42.823+0000 7f738ffff640 1 --2- 172.21.15.43:0/3372903783 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f7378003a10 0x7f7378003df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:51:42.827 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:42.823+0000 7f738ffff640 1 -- 172.21.15.43:0/3372903783 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f73900b2790 msgr2=0x7f73780047c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-09T13:51:42.827 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:42.823+0000 7f738ffff640 1 --2- 172.21.15.43:0/3372903783 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f73900b2790 0x7f73780047c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:51:42.827 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:42.823+0000 7f738ffff640 1 -- 172.21.15.43:0/3372903783 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f7378004d00 msgr2=0x7f7378005180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-09T13:51:42.827 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:42.823+0000 7f738ffff640 1 --2- 172.21.15.43:0/3372903783 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f7378004d00 0x7f7378005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:51:42.827 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:42.823+0000 7f738ffff640 1 -- 172.21.15.43:0/3372903783 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7380009400 con 0x7f7378003a10 2023-08-09T13:51:42.828 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:42.824+0000 7f738ffff640 1 --2- 172.21.15.43:0/3372903783 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f7378003a10 0x7f7378003df0 secure :-1 s=READY pgs=1269 cs=0 l=1 rev1=1 crypto rx=0x7f7380009cc0 tx=0x7f73780057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f58813dd1b83a2df server_cookie=0 in_seq=0 out_seq=0 2023-08-09T13:51:42.829 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:42.825+0000 7f738effd640 1 -- 172.21.15.43:0/3372903783 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f738002c920 con 0x7f7378003a10 2023-08-09T13:51:42.829 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:42.825+0000 7f738effd640 1 -- 172.21.15.43:0/3372903783 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f738002ca80 con 0x7f7378003a10 2023-08-09T13:51:42.829 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:42.825+0000 7f738effd640 1 -- 172.21.15.43:0/3372903783 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f738002ce30 con 0x7f7378003a10 2023-08-09T13:51:42.829 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:42.825+0000 7f739643f640 1 -- 172.21.15.43:0/3372903783 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f7378003a10 msgr2=0x7f7378003df0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:51:42.829 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:42.825+0000 7f739643f640 1 --2- 172.21.15.43:0/3372903783 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f7378003a10 0x7f7378003df0 secure :-1 s=READY pgs=1269 cs=0 l=1 rev1=1 crypto rx=0x7f7380009cc0 tx=0x7f73780057f0 comp rx=0 tx=0).stop 2023-08-09T13:51:42.829 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:42.825+0000 7f73949dc640 1 -- 172.21.15.43:0/3372903783 reap_dead start 2023-08-09T13:51:42.829 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:42.825+0000 7f739643f640 1 -- 172.21.15.43:0/3372903783 shutdown_connections 2023-08-09T13:51:42.830 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:42.825+0000 7f739643f640 1 -- 172.21.15.43:0/3372903783 >> 172.21.15.43:0/3372903783 conn(0x7f739009b7b0 msgr2=0x7f73900d24e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-09T13:51:42.830 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:42.825+0000 7f739643f640 1 -- 172.21.15.43:0/3372903783 shutdown_connections 2023-08-09T13:51:42.830 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:42.825+0000 7f739643f640 1 -- 172.21.15.43:0/3372903783 wait complete. 2023-08-09T13:51:42.830 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:42.827+0000 7f739643f640 1 Processor -- start 2023-08-09T13:51:42.831 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:42.827+0000 7f739643f640 1 -- start start 2023-08-09T13:51:42.832 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:42.828+0000 7f739643f640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f7378003a10 0x7f7390154910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:51:42.832 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:42.828+0000 7f739643f640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f73900b1200 0x7f73900b1680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:51:42.832 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:42.829+0000 7f739643f640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f73900af420 0x7f73900af8d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:51:42.832 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:42.829+0000 7f739643f640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f73900b1bc0 con 0x7f73900b1200 2023-08-09T13:51:42.832 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:42.829+0000 7f739643f640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f73900b0170 con 0x7f73900af420 2023-08-09T13:51:42.832 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:42.829+0000 7f739643f640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f73900b0450 con 0x7f7378003a10 2023-08-09T13:51:42.832 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:42.829+0000 7f738f7fe640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f73900b1200 0x7f73900b1680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:51:42.833 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:42.829+0000 7f738f7fe640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f73900b1200 0x7f73900b1680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:41510/0 (socket says 172.21.15.43:41510) 2023-08-09T13:51:42.833 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:42.829+0000 7f738f7fe640 1 -- 172.21.15.43:0/3429265714 learned_addr learned my addr 172.21.15.43:0/3429265714 (peer_addr_for_me v2:172.21.15.43:0/0) 2023-08-09T13:51:42.833 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:42.829+0000 7f738ffff640 1 --2- 172.21.15.43:0/3429265714 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f7378003a10 0x7f7390154910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:51:42.833 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:42.829+0000 7f73949dc640 1 --2- 172.21.15.43:0/3429265714 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f73900af420 0x7f73900af8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:51:42.833 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:42.829+0000 7f738f7fe640 1 -- 172.21.15.43:0/3429265714 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f7378003a10 msgr2=0x7f7390154910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:51:42.833 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:42.829+0000 7f738f7fe640 1 --2- 172.21.15.43:0/3429265714 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f7378003a10 0x7f7390154910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:51:42.833 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:42.829+0000 7f738f7fe640 1 -- 172.21.15.43:0/3429265714 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f73900af420 msgr2=0x7f73900af8d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:51:42.833 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:42.829+0000 7f738f7fe640 1 --2- 172.21.15.43:0/3429265714 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f73900af420 0x7f73900af8d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:51:42.834 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:42.829+0000 7f738f7fe640 1 -- 172.21.15.43:0/3429265714 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f73900b0730 con 0x7f73900b1200 2023-08-09T13:51:42.834 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:42.830+0000 7f738f7fe640 1 --2- 172.21.15.43:0/3429265714 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f73900b1200 0x7f73900b1680 secure :-1 s=READY pgs=1270 cs=0 l=1 rev1=1 crypto rx=0x7f738400ca70 tx=0x7f738400cf30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-09T13:51:42.834 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:42.830+0000 7f738cff9640 1 -- 172.21.15.43:0/3429265714 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f738400a7c0 con 0x7f73900b1200 2023-08-09T13:51:42.834 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:42.830+0000 7f738cff9640 1 -- 172.21.15.43:0/3429265714 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f738400a920 con 0x7f73900b1200 2023-08-09T13:51:42.834 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:42.830+0000 7f739643f640 1 -- 172.21.15.43:0/3429265714 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f73900b0a10 con 0x7f73900b1200 2023-08-09T13:51:42.834 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:42.830+0000 7f738cff9640 1 -- 172.21.15.43:0/3429265714 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f738400ad50 con 0x7f73900b1200 2023-08-09T13:51:42.835 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:42.830+0000 7f739643f640 1 -- 172.21.15.43:0/3429265714 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7390154e50 con 0x7f73900b1200 2023-08-09T13:51:42.835 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:42.831+0000 7f739643f640 1 -- 172.21.15.43:0/3429265714 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f73900d05a0 con 0x7f73900b1200 2023-08-09T13:51:42.835 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:42.833+0000 7f738cff9640 1 -- 172.21.15.43:0/3429265714 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 44) v1 ==== 88387+0+0 (secure 0 0 0) 0x7f73840070d0 con 0x7f73900b1200 2023-08-09T13:51:42.835 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:42.833+0000 7f738cff9640 1 --2- 172.21.15.43:0/3429265714 >> [v2:172.21.15.203:6832/1317108840,v1:172.21.15.203:6833/1317108840] conn(0x7f737c06a110 0x7f737c06c5d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:51:42.835 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:42.834+0000 7f738cff9640 1 -- 172.21.15.43:0/3429265714 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(105..105 src has 1..105) v4 ==== 7114+0+0 (secure 0 0 0) 0x7f73840882e0 con 0x7f73900b1200 2023-08-09T13:51:42.836 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:42.834+0000 7f738ffff640 1 --2- 172.21.15.43:0/3429265714 >> [v2:172.21.15.203:6832/1317108840,v1:172.21.15.203:6833/1317108840] conn(0x7f737c06a110 0x7f737c06c5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:51:42.837 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:42.836+0000 7f738ffff640 1 --2- 172.21.15.43:0/3429265714 >> [v2:172.21.15.203:6832/1317108840,v1:172.21.15.203:6833/1317108840] conn(0x7f737c06a110 0x7f737c06c5d0 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7f7380036000 tx=0x7f7380030000 comp rx=0 tx=0).ready entity=mgr.6619 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-09T13:51:42.839 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:42.838+0000 7f738cff9640 1 -- 172.21.15.43:0/3429265714 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178066 (secure 0 0 0) 0x7f7384053430 con 0x7f73900b1200 2023-08-09T13:51:43.076 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:43.075+0000 7f739643f640 1 -- 172.21.15.43:0/3429265714 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f73900b0c00 con 0x7f73900b1200 2023-08-09T13:51:43.077 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:43.076+0000 7f738cff9640 1 -- 172.21.15.43:0/3429265714 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 43 v43) v1 ==== 94+0+4880 (secure 0 0 0) 0x7f73900b0c00 con 0x7f73900b1200 2023-08-09T13:51:43.078 INFO:teuthology.orchestra.run.smithi043.stdout: 2023-08-09T13:51:43.078 INFO:teuthology.orchestra.run.smithi043.stdout:{"epoch":43,"default_fscid":5,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5966,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.203:6835/4101044758","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6834","nonce":4101044758},{"type":"v1","addr":"172.21.15.203:6835","nonce":4101044758}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":39},{"gid":6460,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.203:6837/2974664437","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6836","nonce":2974664437},{"type":"v1","addr":"172.21.15.203:6837","nonce":2974664437}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":39},{"gid":7527,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.43:6833/795245144","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6832","nonce":795245144},{"type":"v1","addr":"172.21.15.43:6833","nonce":795245144}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":38}],"filesystems":[{"mdsmap":{"epoch":43,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-08-09T13:51:15.716741+0000","modified":"2023-08-09T13:51:19.249966+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":7542},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7542":{"gid":7542,"name":"a","rank":0,"incarnation":40,"state":"up:active","state_seq":3,"addr":"172.21.15.43:6835/3008795294","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6834","nonce":3008795294},{"type":"v1","addr":"172.21.15.43:6835","nonce":3008795294}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[13],"metadata_pool":12,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":5}]} 2023-08-09T13:51:43.078 INFO:teuthology.orchestra.run.smithi043.stderr:dumped fsmap epoch 43 2023-08-09T13:51:43.081 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:43.079+0000 7f739643f640 1 -- 172.21.15.43:0/3429265714 >> [v2:172.21.15.203:6832/1317108840,v1:172.21.15.203:6833/1317108840] conn(0x7f737c06a110 msgr2=0x7f737c06c5d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:51:43.081 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:43.079+0000 7f739643f640 1 --2- 172.21.15.43:0/3429265714 >> [v2:172.21.15.203:6832/1317108840,v1:172.21.15.203:6833/1317108840] conn(0x7f737c06a110 0x7f737c06c5d0 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7f7380036000 tx=0x7f7380030000 comp rx=0 tx=0).stop 2023-08-09T13:51:43.081 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:43.079+0000 7f739643f640 1 -- 172.21.15.43:0/3429265714 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f73900b1200 msgr2=0x7f73900b1680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:51:43.081 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:43.079+0000 7f739643f640 1 --2- 172.21.15.43:0/3429265714 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f73900b1200 0x7f73900b1680 secure :-1 s=READY pgs=1270 cs=0 l=1 rev1=1 crypto rx=0x7f738400ca70 tx=0x7f738400cf30 comp rx=0 tx=0).stop 2023-08-09T13:51:43.081 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:43.079+0000 7f739643f640 1 -- 172.21.15.43:0/3429265714 shutdown_connections 2023-08-09T13:51:43.081 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:43.079+0000 7f739643f640 1 --2- 172.21.15.43:0/3429265714 >> [v2:172.21.15.203:6832/1317108840,v1:172.21.15.203:6833/1317108840] conn(0x7f737c06a110 0x7f737c06c5d0 unknown :-1 s=CLOSED pgs=29 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:51:43.082 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:43.079+0000 7f739643f640 1 --2- 172.21.15.43:0/3429265714 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f73900af420 0x7f73900af8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:51:43.082 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:43.079+0000 7f739643f640 1 --2- 172.21.15.43:0/3429265714 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f73900b1200 0x7f73900b1680 unknown :-1 s=CLOSED pgs=1270 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:51:43.082 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:43.079+0000 7f739643f640 1 --2- 172.21.15.43:0/3429265714 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f7378003a10 0x7f7390154910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:51:43.082 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:43.079+0000 7f739643f640 1 -- 172.21.15.43:0/3429265714 >> 172.21.15.43:0/3429265714 conn(0x7f739009b7b0 msgr2=0x7f739006bca0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-09T13:51:43.082 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:43.079+0000 7f739643f640 1 -- 172.21.15.43:0/3429265714 shutdown_connections 2023-08-09T13:51:43.082 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:43.079+0000 7f739643f640 1 -- 172.21.15.43:0/3429265714 wait complete. 2023-08-09T13:51:43.099 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-08-09T13:51:43.333 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:43.331+0000 7fcf4d063640 1 Processor -- start 2023-08-09T13:51:43.333 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:43.331+0000 7fcf4d063640 1 -- start start 2023-08-09T13:51:43.334 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:43.333+0000 7fcf4d063640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fcf48160cb0 0x7fcf4815c070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:51:43.335 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:43.333+0000 7fcf4d063640 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7fcf48160760 0x7fcf4815c5b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:51:43.335 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:43.333+0000 7fcf4d063640 1 -- --> v1:172.21.15.203:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fcf480c8680 con 0x7fcf48160380 2023-08-09T13:51:43.335 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:43.333+0000 7fcf4d063640 1 -- --> v2:172.21.15.203:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fcf480ca7d0 con 0x7fcf48160760 2023-08-09T13:51:43.335 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:43.333+0000 7fcf4d063640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcf480c8960 con 0x7fcf48160cb0 2023-08-09T13:51:43.336 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:43.334+0000 7fcf4c862640 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7fcf48160760 0x7fcf4815c5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:51:43.336 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:43.334+0000 7fcf477fe640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fcf48160cb0 0x7fcf4815c070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:51:43.336 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:43.334+0000 7fcf477fe640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fcf48160cb0 0x7fcf4815c070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.203:3301/0 says I am v2:172.21.15.43:36298/0 (socket says 172.21.15.43:36298) 2023-08-09T13:51:43.336 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:43.334+0000 7fcf477fe640 1 -- 172.21.15.43:0/1178331792 learned_addr learned my addr 172.21.15.43:0/1178331792 (peer_addr_for_me v2:172.21.15.43:0/0) 2023-08-09T13:51:43.337 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:43.335+0000 7fcf477fe640 1 -- 172.21.15.43:0/1178331792 >> v2:172.21.15.203:3300/0 conn(0x7fcf48160760 msgr2=0x7fcf4815c5b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-09T13:51:43.337 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:43.335+0000 7fcf477fe640 1 --2- 172.21.15.43:0/1178331792 >> v2:172.21.15.203:3300/0 conn(0x7fcf48160760 0x7fcf4815c5b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:51:43.337 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:43.335+0000 7fcf477fe640 1 -- 172.21.15.43:0/1178331792 >> v1:172.21.15.203:6789/0 conn(0x7fcf48160380 legacy=0x7fcf480ca0c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:51:43.337 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:43.335+0000 7fcf477fe640 1 -- 172.21.15.43:0/1178331792 --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcf480aaf00 con 0x7fcf48160cb0 2023-08-09T13:51:43.338 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:43.336+0000 7fcf477fe640 1 --2- 172.21.15.43:0/1178331792 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fcf48160cb0 0x7fcf4815c070 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7fcf3c009870 tx=0x7fcf3c02ba10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=654eaf340c67f131 server_cookie=0 in_seq=0 out_seq=0 2023-08-09T13:51:43.338 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:43.336+0000 7fcf46ffd640 1 -- 172.21.15.43:0/1178331792 <== mon.2 v2:172.21.15.203:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcf3c02f830 con 0x7fcf48160cb0 2023-08-09T13:51:43.338 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:43.337+0000 7fcf46ffd640 1 -- 172.21.15.43:0/1178331792 <== mon.2 v2:172.21.15.203:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcf3c02f990 con 0x7fcf48160cb0 2023-08-09T13:51:43.338 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:43.337+0000 7fcf46ffd640 1 -- 172.21.15.43:0/1178331792 <== mon.2 v2:172.21.15.203:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcf3c02fd20 con 0x7fcf48160cb0 2023-08-09T13:51:43.339 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:43.337+0000 7fcf4d063640 1 -- 172.21.15.43:0/1178331792 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fcf48160cb0 msgr2=0x7fcf4815c070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:51:43.339 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:43.337+0000 7fcf4d063640 1 --2- 172.21.15.43:0/1178331792 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fcf48160cb0 0x7fcf4815c070 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7fcf3c009870 tx=0x7fcf3c02ba10 comp rx=0 tx=0).stop 2023-08-09T13:51:43.339 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:43.337+0000 7fcf4d063640 1 -- 172.21.15.43:0/1178331792 shutdown_connections 2023-08-09T13:51:43.340 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:43.337+0000 7fcf4d063640 1 --2- 172.21.15.43:0/1178331792 >> v2:172.21.15.203:3300/0 conn(0x7fcf48160760 0x7fcf4815c5b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:51:43.340 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:43.337+0000 7fcf4d063640 1 --2- 172.21.15.43:0/1178331792 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fcf48160cb0 0x7fcf4815c070 unknown :-1 s=CLOSED pgs=175 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:51:43.340 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:43.337+0000 7fcf4d063640 1 -- 172.21.15.43:0/1178331792 >> 172.21.15.43:0/1178331792 conn(0x7fcf480cde90 msgr2=0x7fcf480ce2a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-09T13:51:43.340 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:43.337+0000 7fcf4d063640 1 -- 172.21.15.43:0/1178331792 shutdown_connections 2023-08-09T13:51:43.340 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:43.337+0000 7fcf4d063640 1 -- 172.21.15.43:0/1178331792 wait complete. 2023-08-09T13:51:43.341 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:43.339+0000 7fcf4d063640 1 Processor -- start 2023-08-09T13:51:43.341 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:43.339+0000 7fcf4d063640 1 -- start start 2023-08-09T13:51:43.342 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:43.340+0000 7fcf4d063640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fcf48160380 0x7fcf480c73d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:51:43.342 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:43.340+0000 7fcf4d063640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fcf48160760 0x7fcf480aea20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:51:43.342 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:43.340+0000 7fcf4d063640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fcf48160cb0 0x7fcf480aef60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:51:43.342 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:43.340+0000 7fcf4d063640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcf480b0dd0 con 0x7fcf48160cb0 2023-08-09T13:51:43.342 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:43.340+0000 7fcf4d063640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcf480b10b0 con 0x7fcf48160760 2023-08-09T13:51:43.343 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:43.340+0000 7fcf4d063640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcf480b1390 con 0x7fcf48160380 2023-08-09T13:51:43.344 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:43.341+0000 7fcf4c862640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fcf48160cb0 0x7fcf480aef60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:51:43.345 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:43.341+0000 7fcf4c862640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fcf48160cb0 0x7fcf480aef60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:41512/0 (socket says 172.21.15.43:41512) 2023-08-09T13:51:43.345 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:43.341+0000 7fcf4c862640 1 -- 172.21.15.43:0/3845622789 learned_addr learned my addr 172.21.15.43:0/3845622789 (peer_addr_for_me v2:172.21.15.43:0/0) 2023-08-09T13:51:43.345 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:43.341+0000 7fcf47fff640 1 --2- 172.21.15.43:0/3845622789 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fcf48160380 0x7fcf480c73d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:51:43.345 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:43.341+0000 7fcf477fe640 1 --2- 172.21.15.43:0/3845622789 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fcf48160760 0x7fcf480aea20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:51:43.345 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:43.341+0000 7fcf4c862640 1 -- 172.21.15.43:0/3845622789 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fcf48160380 msgr2=0x7fcf480c73d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:51:43.345 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:43.341+0000 7fcf4c862640 1 --2- 172.21.15.43:0/3845622789 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fcf48160380 0x7fcf480c73d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:51:43.346 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:43.341+0000 7fcf4c862640 1 -- 172.21.15.43:0/3845622789 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fcf48160760 msgr2=0x7fcf480aea20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:51:43.346 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:43.341+0000 7fcf4c862640 1 --2- 172.21.15.43:0/3845622789 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fcf48160760 0x7fcf480aea20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:51:43.346 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:43.341+0000 7fcf4c862640 1 -- 172.21.15.43:0/3845622789 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcf3c0094d0 con 0x7fcf48160cb0 2023-08-09T13:51:43.346 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:43.342+0000 7fcf4c862640 1 --2- 172.21.15.43:0/3845622789 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fcf48160cb0 0x7fcf480aef60 secure :-1 s=READY pgs=1271 cs=0 l=1 rev1=1 crypto rx=0x7fcf4000efc0 tx=0x7fcf4000c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-09T13:51:43.346 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:43.342+0000 7fcf44ff9640 1 -- 172.21.15.43:0/3845622789 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcf40014480 con 0x7fcf48160cb0 2023-08-09T13:51:43.347 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:43.342+0000 7fcf44ff9640 1 -- 172.21.15.43:0/3845622789 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcf4000f040 con 0x7fcf48160cb0 2023-08-09T13:51:43.347 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:43.342+0000 7fcf44ff9640 1 -- 172.21.15.43:0/3845622789 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcf40014820 con 0x7fcf48160cb0 2023-08-09T13:51:43.347 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:43.342+0000 7fcf4d063640 1 -- 172.21.15.43:0/3845622789 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcf480b1670 con 0x7fcf48160cb0 2023-08-09T13:51:43.347 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:43.342+0000 7fcf4d063640 1 -- 172.21.15.43:0/3845622789 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcf480c7910 con 0x7fcf48160cb0 2023-08-09T13:51:43.348 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:43.343+0000 7fcf4d063640 1 -- 172.21.15.43:0/3845622789 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcf480739f0 con 0x7fcf48160cb0 2023-08-09T13:51:43.348 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:43.344+0000 7fcf44ff9640 1 -- 172.21.15.43:0/3845622789 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 45) v1 ==== 88443+0+0 (secure 0 0 0) 0x7fcf40014980 con 0x7fcf48160cb0 2023-08-09T13:51:43.349 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:43.345+0000 7fcf44ff9640 1 --2- 172.21.15.43:0/3845622789 >> [v2:172.21.15.203:6832/1317108840,v1:172.21.15.203:6833/1317108840] conn(0x7fcf3406a170 0x7fcf3406c630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:51:43.349 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:43.345+0000 7fcf44ff9640 1 -- 172.21.15.43:0/3845622789 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(105..105 src has 1..105) v4 ==== 7114+0+0 (secure 0 0 0) 0x7fcf400897d0 con 0x7fcf48160cb0 2023-08-09T13:51:43.349 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:43.346+0000 7fcf47fff640 1 --2- 172.21.15.43:0/3845622789 >> [v2:172.21.15.203:6832/1317108840,v1:172.21.15.203:6833/1317108840] conn(0x7fcf3406a170 0x7fcf3406c630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:51:43.349 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:43.347+0000 7fcf47fff640 1 --2- 172.21.15.43:0/3845622789 >> [v2:172.21.15.203:6832/1317108840,v1:172.21.15.203:6833/1317108840] conn(0x7fcf3406a170 0x7fcf3406c630 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0x7fcf480b3220 tx=0x7fcf38009040 comp rx=0 tx=0).ready entity=mgr.6619 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-09T13:51:43.351 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:43.349+0000 7fcf44ff9640 1 -- 172.21.15.43:0/3845622789 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178066 (secure 0 0 0) 0x7fcf40054870 con 0x7fcf48160cb0 2023-08-09T13:51:43.550 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:43.548+0000 7fcf4d063640 1 -- 172.21.15.43:0/3845622789 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fcf480b0f10 con 0x7fcf48160cb0 2023-08-09T13:51:43.552 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:43.551+0000 7fcf44ff9640 1 -- 172.21.15.43:0/3845622789 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v105) v1 ==== 74+0+19573 (secure 0 0 0) 0x7fcf480b0f10 con 0x7fcf48160cb0 2023-08-09T13:51:43.553 INFO:teuthology.orchestra.run.smithi043.stdout: 2023-08-09T13:51:43.553 INFO:teuthology.orchestra.run.smithi043.stdout:{"epoch":105,"fsid":"daac87ee-1337-4250-a955-aea4e408a39b","created":"2023-08-09T13:33:41.701198+0000","modified":"2023-08-09T13:51:38.531070+0000","last_up_change":"2023-08-09T13:33:53.783453+0000","last_in_change":"2023-08-09T13:33:45.148904+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":13,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":"rbd","create_time":"2023-08-09T13:33:54.365137+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":".mgr","create_time":"2023-08-09T13:33:54.831803+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":12,"pool_name":"cephfs_metadata","create_time":"2023-08-09T13:51:14.142563+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"102","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":13,"pool_name":"cephfs_data","create_time":"2023-08-09T13:51:14.712668+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"103","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"cbd49ecd-e896-4fbd-864e-5c37a829b165","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":101,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6800","nonce":502667335},{"type":"v1","addr":"172.21.15.43:6801","nonce":502667335}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6802","nonce":502667335},{"type":"v1","addr":"172.21.15.43:6803","nonce":502667335}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6806","nonce":502667335},{"type":"v1","addr":"172.21.15.43:6807","nonce":502667335}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6804","nonce":502667335},{"type":"v1","addr":"172.21.15.43:6805","nonce":502667335}]},"public_addr":"172.21.15.43:6801/502667335","cluster_addr":"172.21.15.43:6803/502667335","heartbeat_back_addr":"172.21.15.43:6807/502667335","heartbeat_front_addr":"172.21.15.43:6805/502667335","state":["exists","up"]},{"osd":1,"uuid":"75c7e0fe-1ec7-4fe8-a2f2-3661a65b8500","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":101,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6808","nonce":3289134610},{"type":"v1","addr":"172.21.15.43:6809","nonce":3289134610}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6810","nonce":3289134610},{"type":"v1","addr":"172.21.15.43:6812","nonce":3289134610}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6821","nonce":3289134610},{"type":"v1","addr":"172.21.15.43:6824","nonce":3289134610}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6815","nonce":3289134610},{"type":"v1","addr":"172.21.15.43:6818","nonce":3289134610}]},"public_addr":"172.21.15.43:6809/3289134610","cluster_addr":"172.21.15.43:6812/3289134610","heartbeat_back_addr":"172.21.15.43:6824/3289134610","heartbeat_front_addr":"172.21.15.43:6818/3289134610","state":["exists","up"]},{"osd":2,"uuid":"bb5c41e9-7f5f-4125-9b01-3bb8bfc2ea83","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":101,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6811","nonce":4079709736},{"type":"v1","addr":"172.21.15.43:6814","nonce":4079709736}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6817","nonce":4079709736},{"type":"v1","addr":"172.21.15.43:6820","nonce":4079709736}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6829","nonce":4079709736},{"type":"v1","addr":"172.21.15.43:6831","nonce":4079709736}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6823","nonce":4079709736},{"type":"v1","addr":"172.21.15.43:6826","nonce":4079709736}]},"public_addr":"172.21.15.43:6814/4079709736","cluster_addr":"172.21.15.43:6820/4079709736","heartbeat_back_addr":"172.21.15.43:6831/4079709736","heartbeat_front_addr":"172.21.15.43:6826/4079709736","state":["exists","up"]},{"osd":3,"uuid":"bfe4c8b4-694a-465c-87e6-2b2aab852613","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":101,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6813","nonce":1709538759},{"type":"v1","addr":"172.21.15.43:6816","nonce":1709538759}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6819","nonce":1709538759},{"type":"v1","addr":"172.21.15.43:6822","nonce":1709538759}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6828","nonce":1709538759},{"type":"v1","addr":"172.21.15.43:6830","nonce":1709538759}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6825","nonce":1709538759},{"type":"v1","addr":"172.21.15.43:6827","nonce":1709538759}]},"public_addr":"172.21.15.43:6816/1709538759","cluster_addr":"172.21.15.43:6822/1709538759","heartbeat_back_addr":"172.21.15.43:6830/1709538759","heartbeat_front_addr":"172.21.15.43:6827/1709538759","state":["exists","up"]},{"osd":4,"uuid":"fa2e8a77-aba1-4756-853e-0d174a5bdeba","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":101,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6800","nonce":4196866481},{"type":"v1","addr":"172.21.15.203:6801","nonce":4196866481}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6802","nonce":4196866481},{"type":"v1","addr":"172.21.15.203:6803","nonce":4196866481}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6806","nonce":4196866481},{"type":"v1","addr":"172.21.15.203:6807","nonce":4196866481}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6804","nonce":4196866481},{"type":"v1","addr":"172.21.15.203:6805","nonce":4196866481}]},"public_addr":"172.21.15.203:6801/4196866481","cluster_addr":"172.21.15.203:6803/4196866481","heartbeat_back_addr":"172.21.15.203:6807/4196866481","heartbeat_front_addr":"172.21.15.203:6805/4196866481","state":["exists","up"]},{"osd":5,"uuid":"81811e4b-4d24-4f51-a82c-f1d700a2a00d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":101,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6816","nonce":3059290081},{"type":"v1","addr":"172.21.15.203:6817","nonce":3059290081}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6818","nonce":3059290081},{"type":"v1","addr":"172.21.15.203:6819","nonce":3059290081}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6822","nonce":3059290081},{"type":"v1","addr":"172.21.15.203:6823","nonce":3059290081}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6820","nonce":3059290081},{"type":"v1","addr":"172.21.15.203:6821","nonce":3059290081}]},"public_addr":"172.21.15.203:6817/3059290081","cluster_addr":"172.21.15.203:6819/3059290081","heartbeat_back_addr":"172.21.15.203:6823/3059290081","heartbeat_front_addr":"172.21.15.203:6821/3059290081","state":["exists","up"]},{"osd":6,"uuid":"307bac4d-6a5b-443d-af75-56d695d7674f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":101,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6824","nonce":512155569},{"type":"v1","addr":"172.21.15.203:6825","nonce":512155569}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6826","nonce":512155569},{"type":"v1","addr":"172.21.15.203:6827","nonce":512155569}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6830","nonce":512155569},{"type":"v1","addr":"172.21.15.203:6831","nonce":512155569}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6828","nonce":512155569},{"type":"v1","addr":"172.21.15.203:6829","nonce":512155569}]},"public_addr":"172.21.15.203:6825/512155569","cluster_addr":"172.21.15.203:6827/512155569","heartbeat_back_addr":"172.21.15.203:6831/512155569","heartbeat_front_addr":"172.21.15.203:6829/512155569","state":["exists","up"]},{"osd":7,"uuid":"a95dc7ca-9f1b-4295-af70-68a22a2d7d5e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":101,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6808","nonce":3331450438},{"type":"v1","addr":"172.21.15.203:6809","nonce":3331450438}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6810","nonce":3331450438},{"type":"v1","addr":"172.21.15.203:6811","nonce":3331450438}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6814","nonce":3331450438},{"type":"v1","addr":"172.21.15.203:6815","nonce":3331450438}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6812","nonce":3331450438},{"type":"v1","addr":"172.21.15.203:6813","nonce":3331450438}]},"public_addr":"172.21.15.203:6809/3331450438","cluster_addr":"172.21.15.203:6811/3331450438","heartbeat_back_addr":"172.21.15.203:6815/3331450438","heartbeat_front_addr":"172.21.15.203:6813/3331450438","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-09T13:33:50.547652+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-09T13:33:50.589469+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-09T13:33:50.561724+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-09T13:33:50.599650+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-09T13:33:50.411855+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-09T13:33:50.219681+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-09T13:33:50.504664+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-09T13:33:50.441248+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.203:0/996706743":"2023-08-10T13:51:38.530957+0000","172.21.15.203:0/969846951":"2023-08-10T13:51:38.530957+0000","172.21.15.203:6833/2675832148":"2023-08-10T13:51:38.530957+0000","172.21.15.203:6832/2675832148":"2023-08-10T13:51:38.530957+0000","172.21.15.203:0/822053193":"2023-08-10T13:51:38.530957+0000","172.21.15.203:0/1475395073":"2023-08-10T13:51:38.530957+0000","172.21.15.203:0/1343041460":"2023-08-10T13:51:32.420350+0000","172.21.15.203:6833/1778614695":"2023-08-10T13:51:32.420350+0000","172.21.15.203:0/3761332698":"2023-08-10T13:51:32.420350+0000","172.21.15.203:6832/1778614695":"2023-08-10T13:51:32.420350+0000","172.21.15.203:0/2293720318":"2023-08-10T13:51:38.530957+0000","172.21.15.203:0/2054133158":"2023-08-10T13:51:32.420350+0000","172.21.15.203:0/3225794224":"2023-08-10T13:51:38.530957+0000","172.21.15.203:0/1496080019":"2023-08-10T13:51:32.420350+0000","172.21.15.203:0/1894709586":"2023-08-10T13:51:32.420350+0000","172.21.15.203:0/270998287":"2023-08-10T13:51:32.420350+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":7,"snaps":[{"begin":104,"length":1},{"begin":221,"length":1}]}],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-08-09T13:51:43.555 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:43.553+0000 7fcf4d063640 1 -- 172.21.15.43:0/3845622789 >> [v2:172.21.15.203:6832/1317108840,v1:172.21.15.203:6833/1317108840] conn(0x7fcf3406a170 msgr2=0x7fcf3406c630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:51:43.555 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:43.553+0000 7fcf4d063640 1 --2- 172.21.15.43:0/3845622789 >> [v2:172.21.15.203:6832/1317108840,v1:172.21.15.203:6833/1317108840] conn(0x7fcf3406a170 0x7fcf3406c630 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0x7fcf480b3220 tx=0x7fcf38009040 comp rx=0 tx=0).stop 2023-08-09T13:51:43.556 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:43.553+0000 7fcf4d063640 1 -- 172.21.15.43:0/3845622789 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fcf48160cb0 msgr2=0x7fcf480aef60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:51:43.556 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:43.553+0000 7fcf4d063640 1 --2- 172.21.15.43:0/3845622789 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fcf48160cb0 0x7fcf480aef60 secure :-1 s=READY pgs=1271 cs=0 l=1 rev1=1 crypto rx=0x7fcf4000efc0 tx=0x7fcf4000c490 comp rx=0 tx=0).stop 2023-08-09T13:51:43.556 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:43.554+0000 7fcf4d063640 1 -- 172.21.15.43:0/3845622789 shutdown_connections 2023-08-09T13:51:43.556 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:43.554+0000 7fcf4d063640 1 --2- 172.21.15.43:0/3845622789 >> [v2:172.21.15.203:6832/1317108840,v1:172.21.15.203:6833/1317108840] conn(0x7fcf3406a170 0x7fcf3406c630 unknown :-1 s=CLOSED pgs=30 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:51:43.557 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:43.554+0000 7fcf4d063640 1 --2- 172.21.15.43:0/3845622789 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fcf48160cb0 0x7fcf480aef60 unknown :-1 s=CLOSED pgs=1271 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:51:43.557 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:43.554+0000 7fcf4d063640 1 --2- 172.21.15.43:0/3845622789 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fcf48160760 0x7fcf480aea20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:51:43.557 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:43.554+0000 7fcf4d063640 1 --2- 172.21.15.43:0/3845622789 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fcf48160380 0x7fcf480c73d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:51:43.557 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:43.554+0000 7fcf4d063640 1 -- 172.21.15.43:0/3845622789 >> 172.21.15.43:0/3845622789 conn(0x7fcf480cde90 msgr2=0x7fcf4815a050 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-09T13:51:43.557 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:43.554+0000 7fcf4d063640 1 -- 172.21.15.43:0/3845622789 shutdown_connections 2023-08-09T13:51:43.557 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:43.554+0000 7fcf4d063640 1 -- 172.21.15.43:0/3845622789 wait complete. 2023-08-09T13:51:43.577 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2023-08-09T13:51:43.804 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:43.802+0000 7f0b2af5d640 1 Processor -- start 2023-08-09T13:51:43.804 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:43.802+0000 7f0b2af5d640 1 -- start start 2023-08-09T13:51:43.805 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:43.804+0000 7f0b2af5d640 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7f0b24142ca0 0x7f0b240c96e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:51:43.806 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:43.804+0000 7f0b2af5d640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f0b2415ce40 0x7f0b240cce00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:51:43.806 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:43.804+0000 7f0b2af5d640 1 -- --> v1:172.21.15.203:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0b24143080 con 0x7f0b240cd340 2023-08-09T13:51:43.806 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:43.804+0000 7f0b2af5d640 1 -- --> v2:172.21.15.203:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0b240cd7b0 con 0x7f0b24142ca0 2023-08-09T13:51:43.806 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:43.804+0000 7f0b2af5d640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0b2406d400 con 0x7f0b2415ce40 2023-08-09T13:51:43.807 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:43.805+0000 7f0b28cf9640 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7f0b24142ca0 0x7f0b240c96e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:51:43.807 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:43.805+0000 7f0b23fff640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f0b2415ce40 0x7f0b240cce00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:51:43.807 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:43.805+0000 7f0b28cf9640 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7f0b24142ca0 0x7f0b240c96e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.203:3300/0 says I am v2:172.21.15.43:35302/0 (socket says 172.21.15.43:35302) 2023-08-09T13:51:43.807 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:43.805+0000 7f0b23fff640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f0b2415ce40 0x7f0b240cce00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.203:3301/0 says I am v2:172.21.15.43:36320/0 (socket says 172.21.15.43:36320) 2023-08-09T13:51:43.807 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:43.805+0000 7f0b28cf9640 1 -- 172.21.15.43:0/3308454010 learned_addr learned my addr 172.21.15.43:0/3308454010 (peer_addr_for_me v2:172.21.15.43:0/0) 2023-08-09T13:51:43.808 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:43.806+0000 7f0b23fff640 1 -- 172.21.15.43:0/3308454010 >> v2:172.21.15.203:3300/0 conn(0x7f0b24142ca0 msgr2=0x7f0b240c96e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-09T13:51:43.808 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:43.806+0000 7f0b23fff640 1 --2- 172.21.15.43:0/3308454010 >> v2:172.21.15.203:3300/0 conn(0x7f0b24142ca0 0x7f0b240c96e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:51:43.808 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:43.806+0000 7f0b23fff640 1 -- 172.21.15.43:0/3308454010 >> v1:172.21.15.203:6789/0 conn(0x7f0b240cd340 legacy=0x7f0b240cbc70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:51:43.808 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:43.806+0000 7f0b23fff640 1 -- 172.21.15.43:0/3308454010 --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0b240c9d90 con 0x7f0b2415ce40 2023-08-09T13:51:43.808 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:43.806+0000 7f0b28cf9640 1 --2- 172.21.15.43:0/3308454010 >> v2:172.21.15.203:3300/0 conn(0x7f0b24142ca0 0x7f0b240c96e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-08-09T13:51:43.809 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:43.807+0000 7f0b23fff640 1 --2- 172.21.15.43:0/3308454010 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f0b2415ce40 0x7f0b240cce00 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7f0b100097a0 tx=0x7f0b1002b8b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=31f874361549f02a server_cookie=0 in_seq=0 out_seq=0 2023-08-09T13:51:43.809 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:43.808+0000 7f0b237fe640 1 -- 172.21.15.43:0/3308454010 <== mon.2 v2:172.21.15.203:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0b1002d040 con 0x7f0b2415ce40 2023-08-09T13:51:43.810 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:43.808+0000 7f0b237fe640 1 -- 172.21.15.43:0/3308454010 <== mon.2 v2:172.21.15.203:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0b10031810 con 0x7f0b2415ce40 2023-08-09T13:51:43.810 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:43.808+0000 7f0b237fe640 1 -- 172.21.15.43:0/3308454010 <== mon.2 v2:172.21.15.203:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0b10031ba0 con 0x7f0b2415ce40 2023-08-09T13:51:43.810 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:43.808+0000 7f0b2af5d640 1 -- 172.21.15.43:0/3308454010 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f0b2415ce40 msgr2=0x7f0b240cce00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:51:43.810 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:43.808+0000 7f0b2af5d640 1 --2- 172.21.15.43:0/3308454010 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f0b2415ce40 0x7f0b240cce00 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7f0b100097a0 tx=0x7f0b1002b8b0 comp rx=0 tx=0).stop 2023-08-09T13:51:43.811 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:43.808+0000 7f0b2af5d640 1 -- 172.21.15.43:0/3308454010 shutdown_connections 2023-08-09T13:51:43.811 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:43.808+0000 7f0b2af5d640 1 --2- 172.21.15.43:0/3308454010 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f0b2415ce40 0x7f0b240cce00 unknown :-1 s=CLOSED pgs=176 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:51:43.811 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:43.808+0000 7f0b2af5d640 1 --2- 172.21.15.43:0/3308454010 >> v2:172.21.15.203:3300/0 conn(0x7f0b24142ca0 0x7f0b240c96e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:51:43.811 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:43.808+0000 7f0b2af5d640 1 -- 172.21.15.43:0/3308454010 >> 172.21.15.43:0/3308454010 conn(0x7f0b240ce0a0 msgr2=0x7f0b240ce480 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-09T13:51:43.811 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:43.809+0000 7f0b2af5d640 1 -- 172.21.15.43:0/3308454010 shutdown_connections 2023-08-09T13:51:43.811 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:43.809+0000 7f0b2af5d640 1 -- 172.21.15.43:0/3308454010 wait complete. 2023-08-09T13:51:43.812 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:43.810+0000 7f0b2af5d640 1 Processor -- start 2023-08-09T13:51:43.812 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:43.811+0000 7f0b2af5d640 1 -- start start 2023-08-09T13:51:43.813 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:43.811+0000 7f0b2af5d640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f0b240cd340 0x7f0b2413eb60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:51:43.813 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:43.812+0000 7f0b2af5d640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f0b24142ca0 0x7f0b2415b010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:51:43.813 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:43.812+0000 7f0b2af5d640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f0b2415ce40 0x7f0b2415d320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:51:43.813 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:43.812+0000 7f0b2af5d640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0b2413f0a0 con 0x7f0b24142ca0 2023-08-09T13:51:43.813 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:43.812+0000 7f0b2af5d640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0b2415bad0 con 0x7f0b240cd340 2023-08-09T13:51:43.814 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:43.812+0000 7f0b2af5d640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0b240b2be0 con 0x7f0b2415ce40 2023-08-09T13:51:43.815 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:43.812+0000 7f0b23fff640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f0b24142ca0 0x7f0b2415b010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:51:43.815 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:43.812+0000 7f0b23fff640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f0b24142ca0 0x7f0b2415b010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:41522/0 (socket says 172.21.15.43:41522) 2023-08-09T13:51:43.815 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:43.812+0000 7f0b23fff640 1 -- 172.21.15.43:0/1465204554 learned_addr learned my addr 172.21.15.43:0/1465204554 (peer_addr_for_me v2:172.21.15.43:0/0) 2023-08-09T13:51:43.815 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:43.812+0000 7f0b294fa640 1 --2- 172.21.15.43:0/1465204554 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f0b2415ce40 0x7f0b2415d320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:51:43.815 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:43.812+0000 7f0b28cf9640 1 --2- 172.21.15.43:0/1465204554 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f0b240cd340 0x7f0b2413eb60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:51:43.816 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:43.812+0000 7f0b23fff640 1 -- 172.21.15.43:0/1465204554 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f0b2415ce40 msgr2=0x7f0b2415d320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:51:43.816 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:43.812+0000 7f0b23fff640 1 --2- 172.21.15.43:0/1465204554 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f0b2415ce40 0x7f0b2415d320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:51:43.816 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:43.812+0000 7f0b23fff640 1 -- 172.21.15.43:0/1465204554 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f0b240cd340 msgr2=0x7f0b2413eb60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:51:43.816 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:43.812+0000 7f0b23fff640 1 --2- 172.21.15.43:0/1465204554 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f0b240cd340 0x7f0b2413eb60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:51:43.816 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:43.813+0000 7f0b23fff640 1 -- 172.21.15.43:0/1465204554 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0b1800b770 con 0x7f0b24142ca0 2023-08-09T13:51:43.816 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:43.813+0000 7f0b23fff640 1 --2- 172.21.15.43:0/1465204554 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f0b24142ca0 0x7f0b2415b010 secure :-1 s=READY pgs=1272 cs=0 l=1 rev1=1 crypto rx=0x7f0b100098c0 tx=0x7f0b100046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-09T13:51:43.816 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:43.813+0000 7f0b217fa640 1 -- 172.21.15.43:0/1465204554 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0b10006e10 con 0x7f0b24142ca0 2023-08-09T13:51:43.816 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:43.813+0000 7f0b217fa640 1 -- 172.21.15.43:0/1465204554 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0b1003ad60 con 0x7f0b24142ca0 2023-08-09T13:51:43.817 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:43.813+0000 7f0b2af5d640 1 -- 172.21.15.43:0/1465204554 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0b10009400 con 0x7f0b24142ca0 2023-08-09T13:51:43.817 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:43.813+0000 7f0b217fa640 1 -- 172.21.15.43:0/1465204554 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0b10044620 con 0x7f0b24142ca0 2023-08-09T13:51:43.817 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:43.813+0000 7f0b2af5d640 1 -- 172.21.15.43:0/1465204554 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0b240b31a0 con 0x7f0b24142ca0 2023-08-09T13:51:43.817 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:43.815+0000 7f0b2af5d640 1 -- 172.21.15.43:0/1465204554 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0b240739f0 con 0x7f0b24142ca0 2023-08-09T13:51:43.817 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:43.816+0000 7f0b217fa640 1 -- 172.21.15.43:0/1465204554 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 45) v1 ==== 88443+0+0 (secure 0 0 0) 0x7f0b1004d440 con 0x7f0b24142ca0 2023-08-09T13:51:43.818 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:43.816+0000 7f0b217fa640 1 --2- 172.21.15.43:0/1465204554 >> [v2:172.21.15.203:6832/1317108840,v1:172.21.15.203:6833/1317108840] conn(0x7f0b0806a170 0x7f0b0806c630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:51:43.818 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:43.817+0000 7f0b217fa640 1 -- 172.21.15.43:0/1465204554 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(105..105 src has 1..105) v4 ==== 7114+0+0 (secure 0 0 0) 0x7f0b100ab060 con 0x7f0b24142ca0 2023-08-09T13:51:43.819 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:43.817+0000 7f0b28cf9640 1 --2- 172.21.15.43:0/1465204554 >> [v2:172.21.15.203:6832/1317108840,v1:172.21.15.203:6833/1317108840] conn(0x7f0b0806a170 0x7f0b0806c630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:51:43.819 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:43.818+0000 7f0b28cf9640 1 --2- 172.21.15.43:0/1465204554 >> [v2:172.21.15.203:6832/1317108840,v1:172.21.15.203:6833/1317108840] conn(0x7f0b0806a170 0x7f0b0806c630 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0x7f0b1800bef0 tx=0x7f0b1800b450 comp rx=0 tx=0).ready entity=mgr.6619 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-09T13:51:43.822 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:43.821+0000 7f0b217fa640 1 -- 172.21.15.43:0/1465204554 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178066 (secure 0 0 0) 0x7f0b100dd7b0 con 0x7f0b24142ca0 2023-08-09T13:51:44.053 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:44.052+0000 7f0b2af5d640 1 -- 172.21.15.43:0/1465204554 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) v1 -- 0x7f0b240cabc0 con 0x7f0b24142ca0 2023-08-09T13:51:44.054 INFO:tasks.ceph.mon.a.smithi043.stderr:2023-08-09T13:51:44.053+0000 7f32867f3640 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2023-08-09T13:51:44.066 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:44.064+0000 7f0b217fa640 1 -- 172.21.15.43:0/1465204554 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v44) v1 ==== 161+0+0 (secure 0 0 0) 0x7f0b10076070 con 0x7f0b24142ca0 2023-08-09T13:51:44.067 INFO:teuthology.orchestra.run.smithi043.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2023-08-09T13:51:44.069 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:44.067+0000 7f0b2af5d640 1 -- 172.21.15.43:0/1465204554 >> [v2:172.21.15.203:6832/1317108840,v1:172.21.15.203:6833/1317108840] conn(0x7f0b0806a170 msgr2=0x7f0b0806c630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:51:44.070 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:44.067+0000 7f0b2af5d640 1 --2- 172.21.15.43:0/1465204554 >> [v2:172.21.15.203:6832/1317108840,v1:172.21.15.203:6833/1317108840] conn(0x7f0b0806a170 0x7f0b0806c630 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0x7f0b1800bef0 tx=0x7f0b1800b450 comp rx=0 tx=0).stop 2023-08-09T13:51:44.070 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:44.067+0000 7f0b2af5d640 1 -- 172.21.15.43:0/1465204554 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f0b24142ca0 msgr2=0x7f0b2415b010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:51:44.070 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:44.067+0000 7f0b2af5d640 1 --2- 172.21.15.43:0/1465204554 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f0b24142ca0 0x7f0b2415b010 secure :-1 s=READY pgs=1272 cs=0 l=1 rev1=1 crypto rx=0x7f0b100098c0 tx=0x7f0b100046c0 comp rx=0 tx=0).stop 2023-08-09T13:51:44.071 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:44.067+0000 7f0b2af5d640 1 -- 172.21.15.43:0/1465204554 shutdown_connections 2023-08-09T13:51:44.071 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:44.067+0000 7f0b2af5d640 1 --2- 172.21.15.43:0/1465204554 >> [v2:172.21.15.203:6832/1317108840,v1:172.21.15.203:6833/1317108840] conn(0x7f0b0806a170 0x7f0b0806c630 unknown :-1 s=CLOSED pgs=31 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:51:44.071 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:44.067+0000 7f0b2af5d640 1 --2- 172.21.15.43:0/1465204554 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f0b2415ce40 0x7f0b2415d320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:51:44.072 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:44.067+0000 7f0b2af5d640 1 --2- 172.21.15.43:0/1465204554 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f0b24142ca0 0x7f0b2415b010 unknown :-1 s=CLOSED pgs=1272 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:51:44.072 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:44.067+0000 7f0b2af5d640 1 --2- 172.21.15.43:0/1465204554 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f0b240cd340 0x7f0b2413eb60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:51:44.072 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:44.067+0000 7f0b2af5d640 1 -- 172.21.15.43:0/1465204554 >> 172.21.15.43:0/1465204554 conn(0x7f0b240ce0a0 msgr2=0x7f0b24141d50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-09T13:51:44.073 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:44.068+0000 7f0b2af5d640 1 -- 172.21.15.43:0/1465204554 shutdown_connections 2023-08-09T13:51:44.073 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:44.068+0000 7f0b2af5d640 1 -- 172.21.15.43:0/1465204554 wait complete. 2023-08-09T13:51:44.076 INFO:tasks.ceph.mds.a.smithi043.stderr: -1878> 2023-08-09T13:51:17.180+0000 7f4b02b15640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2023-08-09T13:51:44.090 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs rm cephfs --yes-i-really-mean-it 2023-08-09T13:51:44.134 INFO:tasks.ceph.mds.a.smithi043.stdout:starting mds.a at 2023-08-09T13:51:44.293 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:44.291+0000 7f34b0c1a640 1 Processor -- start 2023-08-09T13:51:44.293 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:44.292+0000 7f34b0c1a640 1 -- start start 2023-08-09T13:51:44.294 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:44.293+0000 7f34b0c1a640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f34ac0b7df0 0x7f34ac0d1b30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:51:44.295 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:44.293+0000 7f34b0c1a640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f34ac0cebf0 0x7f34ac0cf040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:51:44.295 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:44.293+0000 7f34b0c1a640 1 -- --> v1:172.21.15.203:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f34ac0cf580 con 0x7f34ac15c2d0 2023-08-09T13:51:44.295 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:44.293+0000 7f34b0c1a640 1 -- --> v2:172.21.15.43:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f34ac0b8230 con 0x7f34ac0b7df0 2023-08-09T13:51:44.295 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:44.293+0000 7f34b0c1a640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f34ac0d16a0 con 0x7f34ac0cebf0 2023-08-09T13:51:44.296 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:44.294+0000 7f34aaffd640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f34ac0b7df0 0x7f34ac0d1b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:51:44.296 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:44.294+0000 7f34ab7fe640 1 --1- >> v1:172.21.15.203:6789/0 conn(0x7f34ac15c2d0 0x7f34ac0cd020 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.203:6789/0 says I am v1:172.21.15.43:51484/0 (socket says 172.21.15.43:51484) 2023-08-09T13:51:44.297 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:44.294+0000 7f34ab7fe640 1 -- 172.21.15.43:0/2080081221 learned_addr learned my addr 172.21.15.43:0/2080081221 (peer_addr_for_me v1:172.21.15.43:0/0) 2023-08-09T13:51:44.297 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:44.294+0000 7f34abfff640 1 --2- 172.21.15.43:0/2080081221 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f34ac0cebf0 0x7f34ac0cf040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:51:44.297 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:44.294+0000 7f34aaffd640 1 -- 172.21.15.43:0/2080081221 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f34ac0cebf0 msgr2=0x7f34ac0cf040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-09T13:51:44.297 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:44.294+0000 7f34aaffd640 1 --2- 172.21.15.43:0/2080081221 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f34ac0cebf0 0x7f34ac0cf040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:51:44.297 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:44.294+0000 7f34aaffd640 1 -- 172.21.15.43:0/2080081221 >> v1:172.21.15.203:6789/0 conn(0x7f34ac15c2d0 legacy=0x7f34ac0cd020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:51:44.297 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:44.294+0000 7f34aaffd640 1 -- 172.21.15.43:0/2080081221 --> v2:172.21.15.43:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f34ac0d0d70 con 0x7f34ac0b7df0 2023-08-09T13:51:44.299 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:44.295+0000 7f34aaffd640 1 --2- 172.21.15.43:0/2080081221 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f34ac0b7df0 0x7f34ac0d1b30 secure :-1 s=READY pgs=1274 cs=0 l=1 rev1=1 crypto rx=0x7f34a00097a0 tx=0x7f34a002b8b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=496959ec2179141e server_cookie=0 in_seq=0 out_seq=0 2023-08-09T13:51:44.299 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:44.296+0000 7f34aa7fc640 1 -- 172.21.15.43:0/2080081221 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f34a002d040 con 0x7f34ac0b7df0 2023-08-09T13:51:44.299 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:44.296+0000 7f34aa7fc640 1 -- 172.21.15.43:0/2080081221 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f34a0031810 con 0x7f34ac0b7df0 2023-08-09T13:51:44.299 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:44.296+0000 7f34aa7fc640 1 -- 172.21.15.43:0/2080081221 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f34a0031ba0 con 0x7f34ac0b7df0 2023-08-09T13:51:44.299 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:44.296+0000 7f34b0c1a640 1 -- 172.21.15.43:0/2080081221 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f34ac0b7df0 msgr2=0x7f34ac0d1b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:51:44.300 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:44.296+0000 7f34b0c1a640 1 --2- 172.21.15.43:0/2080081221 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f34ac0b7df0 0x7f34ac0d1b30 secure :-1 s=READY pgs=1274 cs=0 l=1 rev1=1 crypto rx=0x7f34a00097a0 tx=0x7f34a002b8b0 comp rx=0 tx=0).stop 2023-08-09T13:51:44.300 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:44.296+0000 7f34b0c1a640 1 -- 172.21.15.43:0/2080081221 shutdown_connections 2023-08-09T13:51:44.300 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:44.296+0000 7f34b0c1a640 1 --2- 172.21.15.43:0/2080081221 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f34ac0cebf0 0x7f34ac0cf040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:51:44.300 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:44.296+0000 7f34b0c1a640 1 --2- 172.21.15.43:0/2080081221 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f34ac0b7df0 0x7f34ac0d1b30 unknown :-1 s=CLOSED pgs=1274 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:51:44.300 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:44.296+0000 7f34b0c1a640 1 -- 172.21.15.43:0/2080081221 >> 172.21.15.43:0/2080081221 conn(0x7f34ac06a230 msgr2=0x7f34ac06a640 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-09T13:51:44.300 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:44.297+0000 7f34b0c1a640 1 -- 172.21.15.43:0/2080081221 shutdown_connections 2023-08-09T13:51:44.301 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:44.297+0000 7f34b0c1a640 1 -- 172.21.15.43:0/2080081221 wait complete. 2023-08-09T13:51:44.302 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:44.298+0000 7f34b0c1a640 1 Processor -- start 2023-08-09T13:51:44.302 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:44.298+0000 7f34b0c1a640 1 -- start start 2023-08-09T13:51:44.302 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:44.299+0000 7f34b0c1a640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f34ac0b7df0 0x7f34ac1f1a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:51:44.303 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:44.299+0000 7f34b0c1a640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f34ac0cebf0 0x7f34ac0a9fd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:51:44.303 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:44.299+0000 7f34b0c1a640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f34ac15c2d0 0x7f34ac1eefe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:51:44.303 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:44.299+0000 7f34b0c1a640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f34ac1591e0 con 0x7f34ac0cebf0 2023-08-09T13:51:44.303 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:44.299+0000 7f34b0c1a640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f34ac0aa510 con 0x7f34ac15c2d0 2023-08-09T13:51:44.303 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:44.299+0000 7f34b0c1a640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f34ac1ef520 con 0x7f34ac0b7df0 2023-08-09T13:51:44.303 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:44.300+0000 7f34aaffd640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f34ac0cebf0 0x7f34ac0a9fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:51:44.304 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:44.300+0000 7f34aaffd640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f34ac0cebf0 0x7f34ac0a9fd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:41538/0 (socket says 172.21.15.43:41538) 2023-08-09T13:51:44.304 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:44.300+0000 7f34aaffd640 1 -- 172.21.15.43:0/1399968610 learned_addr learned my addr 172.21.15.43:0/1399968610 (peer_addr_for_me v2:172.21.15.43:0/0) 2023-08-09T13:51:44.304 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:44.300+0000 7f34abfff640 1 --2- 172.21.15.43:0/1399968610 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f34ac15c2d0 0x7f34ac1eefe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:51:44.304 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:44.300+0000 7f34ab7fe640 1 --2- 172.21.15.43:0/1399968610 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f34ac0b7df0 0x7f34ac1f1a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:51:44.304 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:44.300+0000 7f34aaffd640 1 -- 172.21.15.43:0/1399968610 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f34ac0b7df0 msgr2=0x7f34ac1f1a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:51:44.304 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:44.300+0000 7f34aaffd640 1 --2- 172.21.15.43:0/1399968610 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f34ac0b7df0 0x7f34ac1f1a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:51:44.305 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:44.300+0000 7f34aaffd640 1 -- 172.21.15.43:0/1399968610 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f34ac15c2d0 msgr2=0x7f34ac1eefe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:51:44.305 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:44.300+0000 7f34aaffd640 1 --2- 172.21.15.43:0/1399968610 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f34ac15c2d0 0x7f34ac1eefe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:51:44.305 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:44.300+0000 7f34aaffd640 1 -- 172.21.15.43:0/1399968610 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f34a0009400 con 0x7f34ac0cebf0 2023-08-09T13:51:44.305 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:44.301+0000 7f34aaffd640 1 --2- 172.21.15.43:0/1399968610 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f34ac0cebf0 0x7f34ac0a9fd0 secure :-1 s=READY pgs=1275 cs=0 l=1 rev1=1 crypto rx=0x7f34a002bec0 tx=0x7f34a00046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-09T13:51:44.306 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:44.301+0000 7f3493fff640 1 -- 172.21.15.43:0/1399968610 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f34a003ad60 con 0x7f34ac0cebf0 2023-08-09T13:51:44.306 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:44.301+0000 7f3493fff640 1 -- 172.21.15.43:0/1399968610 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f34a00423f0 con 0x7f34ac0cebf0 2023-08-09T13:51:44.307 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:44.301+0000 7f3493fff640 1 -- 172.21.15.43:0/1399968610 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f34a00427a0 con 0x7f34ac0cebf0 2023-08-09T13:51:44.307 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:44.301+0000 7f34b0c1a640 1 -- 172.21.15.43:0/1399968610 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f34ac1f2ce0 con 0x7f34ac0cebf0 2023-08-09T13:51:44.307 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:44.301+0000 7f34b0c1a640 1 -- 172.21.15.43:0/1399968610 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f34ac1f23b0 con 0x7f34ac0cebf0 2023-08-09T13:51:44.307 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:44.302+0000 7f34b0c1a640 1 -- 172.21.15.43:0/1399968610 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f34ac1f2e70 con 0x7f34ac0cebf0 2023-08-09T13:51:44.307 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:44.303+0000 7f3493fff640 1 -- 172.21.15.43:0/1399968610 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 45) v1 ==== 88443+0+0 (secure 0 0 0) 0x7f34a0042900 con 0x7f34ac0cebf0 2023-08-09T13:51:44.307 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:44.304+0000 7f3493fff640 1 --2- 172.21.15.43:0/1399968610 >> [v2:172.21.15.203:6832/1317108840,v1:172.21.15.203:6833/1317108840] conn(0x7f349806a170 0x7f349806c630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:51:44.307 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:44.304+0000 7f3493fff640 1 -- 172.21.15.43:0/1399968610 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(106..106 src has 1..106) v4 ==== 7200+0+0 (secure 0 0 0) 0x7f34a00aac80 con 0x7f34ac0cebf0 2023-08-09T13:51:44.307 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:44.305+0000 7f34ab7fe640 1 --2- 172.21.15.43:0/1399968610 >> [v2:172.21.15.203:6832/1317108840,v1:172.21.15.203:6833/1317108840] conn(0x7f349806a170 0x7f349806c630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:51:44.308 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:44.306+0000 7f34ab7fe640 1 --2- 172.21.15.43:0/1399968610 >> [v2:172.21.15.203:6832/1317108840,v1:172.21.15.203:6833/1317108840] conn(0x7f349806a170 0x7f349806c630 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0x7f349c000f60 tx=0x7f349c009040 comp rx=0 tx=0).ready entity=mgr.6619 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-09T13:51:44.310 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:44.309+0000 7f3493fff640 1 -- 172.21.15.43:0/1399968610 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178066 (secure 0 0 0) 0x7f34a0075c60 con 0x7f34ac0cebf0 2023-08-09T13:51:44.540 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:44.539+0000 7f34b0c1a640 1 -- 172.21.15.43:0/1399968610 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} v 0) v1 -- 0x7f34ac04e390 con 0x7f34ac0cebf0 2023-08-09T13:51:45.067 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:45.065+0000 7f3493fff640 1 -- 172.21.15.43:0/1399968610 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]=0 v45) v1 ==== 104+0+0 (secure 0 0 0) 0x7f34a0079280 con 0x7f34ac0cebf0 2023-08-09T13:51:45.070 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:45.068+0000 7f34b0c1a640 1 -- 172.21.15.43:0/1399968610 >> [v2:172.21.15.203:6832/1317108840,v1:172.21.15.203:6833/1317108840] conn(0x7f349806a170 msgr2=0x7f349806c630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:51:45.070 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:45.068+0000 7f34b0c1a640 1 --2- 172.21.15.43:0/1399968610 >> [v2:172.21.15.203:6832/1317108840,v1:172.21.15.203:6833/1317108840] conn(0x7f349806a170 0x7f349806c630 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0x7f349c000f60 tx=0x7f349c009040 comp rx=0 tx=0).stop 2023-08-09T13:51:45.070 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:45.068+0000 7f34b0c1a640 1 -- 172.21.15.43:0/1399968610 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f34ac0cebf0 msgr2=0x7f34ac0a9fd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:51:45.070 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:45.068+0000 7f34b0c1a640 1 --2- 172.21.15.43:0/1399968610 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f34ac0cebf0 0x7f34ac0a9fd0 secure :-1 s=READY pgs=1275 cs=0 l=1 rev1=1 crypto rx=0x7f34a002bec0 tx=0x7f34a00046c0 comp rx=0 tx=0).stop 2023-08-09T13:51:45.071 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:45.069+0000 7f34b0c1a640 1 -- 172.21.15.43:0/1399968610 shutdown_connections 2023-08-09T13:51:45.071 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:45.069+0000 7f34b0c1a640 1 --2- 172.21.15.43:0/1399968610 >> [v2:172.21.15.203:6832/1317108840,v1:172.21.15.203:6833/1317108840] conn(0x7f349806a170 0x7f349806c630 unknown :-1 s=CLOSED pgs=32 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:51:45.071 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:45.069+0000 7f34b0c1a640 1 --2- 172.21.15.43:0/1399968610 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f34ac15c2d0 0x7f34ac1eefe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:51:45.071 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:45.069+0000 7f34b0c1a640 1 --2- 172.21.15.43:0/1399968610 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f34ac0cebf0 0x7f34ac0a9fd0 unknown :-1 s=CLOSED pgs=1275 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:51:45.072 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:45.069+0000 7f34b0c1a640 1 --2- 172.21.15.43:0/1399968610 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f34ac0b7df0 0x7f34ac1f1a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:51:45.072 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:45.069+0000 7f34b0c1a640 1 -- 172.21.15.43:0/1399968610 >> 172.21.15.43:0/1399968610 conn(0x7f34ac06a230 msgr2=0x7f34ac0cfaf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-09T13:51:45.072 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:45.069+0000 7f34b0c1a640 1 -- 172.21.15.43:0/1399968610 shutdown_connections 2023-08-09T13:51:45.072 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:45.069+0000 7f34b0c1a640 1 -- 172.21.15.43:0/1399968610 wait complete. 2023-08-09T13:51:45.092 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_metadata cephfs_metadata --yes-i-really-really-mean-it 2023-08-09T13:51:45.376 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:45.374+0000 7f49b286a640 1 Processor -- start 2023-08-09T13:51:45.376 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:45.374+0000 7f49b286a640 1 -- start start 2023-08-09T13:51:45.378 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:45.376+0000 7f49b286a640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f49ac15a330 0x7f49ac0c9780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:51:45.378 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:45.376+0000 7f49b286a640 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7f49ac0aff80 0x7f49ac070d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:51:45.378 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:45.376+0000 7f49b286a640 1 -- --> v1:172.21.15.203:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f49ac15dd30 con 0x7f49ac15ea60 2023-08-09T13:51:45.378 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:45.376+0000 7f49b286a640 1 -- --> v2:172.21.15.203:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f49ac0c9cc0 con 0x7f49ac0aff80 2023-08-09T13:51:45.379 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:45.376+0000 7f49b286a640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f49ac071240 con 0x7f49ac15a330 2023-08-09T13:51:45.380 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:45.377+0000 7f49b2069640 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7f49ac0aff80 0x7f49ac070d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:51:45.381 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:45.377+0000 7f49b1868640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f49ac15a330 0x7f49ac0c9780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:51:45.381 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:45.377+0000 7f49b1868640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f49ac15a330 0x7f49ac0c9780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.203:3301/0 says I am v2:172.21.15.43:44888/0 (socket says 172.21.15.43:44888) 2023-08-09T13:51:45.381 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:45.377+0000 7f49b1868640 1 -- 172.21.15.43:0/1412761528 learned_addr learned my addr 172.21.15.43:0/1412761528 (peer_addr_for_me v2:172.21.15.43:0/0) 2023-08-09T13:51:45.381 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:45.377+0000 7f49b2069640 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7f49ac0aff80 0x7f49ac070d00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.203:3300/0 says I am v2:172.21.15.43:43114/0 (socket says 172.21.15.43:43114) 2023-08-09T13:51:45.382 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:45.377+0000 7f49b1067640 1 --1- 172.21.15.43:0/1412761528 >> v1:172.21.15.203:6789/0 conn(0x7f49ac15ea60 0x7f49ac0af870 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.203:6789/0 says I am v1:172.21.15.43:59778/0 (socket says 172.21.15.43:59778) 2023-08-09T13:51:45.382 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:45.378+0000 7f49b1868640 1 -- 172.21.15.43:0/1412761528 >> v2:172.21.15.203:3300/0 conn(0x7f49ac0aff80 msgr2=0x7f49ac070d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-09T13:51:45.383 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:45.378+0000 7f49b1868640 1 --2- 172.21.15.43:0/1412761528 >> v2:172.21.15.203:3300/0 conn(0x7f49ac0aff80 0x7f49ac070d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:51:45.383 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:45.378+0000 7f49b1868640 1 -- 172.21.15.43:0/1412761528 >> v1:172.21.15.203:6789/0 conn(0x7f49ac15ea60 legacy=0x7f49ac0af870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:51:45.383 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:45.378+0000 7f49b1868640 1 -- 172.21.15.43:0/1412761528 --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f49ac071490 con 0x7f49ac15a330 2023-08-09T13:51:45.388 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:45.379+0000 7f49b1868640 1 --2- 172.21.15.43:0/1412761528 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f49ac15a330 0x7f49ac0c9780 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7f49a8009870 tx=0x7f49a802b980 comp rx=0 tx=0).ready entity=mon.2 client_cookie=3ffaaa6843a980a2 server_cookie=0 in_seq=0 out_seq=0 2023-08-09T13:51:45.388 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:45.380+0000 7f49b0866640 1 -- 172.21.15.43:0/1412761528 <== mon.2 v2:172.21.15.203:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f49a8030840 con 0x7f49ac15a330 2023-08-09T13:51:45.389 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:45.380+0000 7f49b0866640 1 -- 172.21.15.43:0/1412761528 <== mon.2 v2:172.21.15.203:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f49a80309a0 con 0x7f49ac15a330 2023-08-09T13:51:45.389 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:45.380+0000 7f49b0866640 1 -- 172.21.15.43:0/1412761528 <== mon.2 v2:172.21.15.203:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f49a8030d30 con 0x7f49ac15a330 2023-08-09T13:51:45.389 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:45.381+0000 7f49b286a640 1 -- 172.21.15.43:0/1412761528 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f49ac15a330 msgr2=0x7f49ac0c9780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:51:45.389 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:45.381+0000 7f49b286a640 1 --2- 172.21.15.43:0/1412761528 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f49ac15a330 0x7f49ac0c9780 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7f49a8009870 tx=0x7f49a802b980 comp rx=0 tx=0).stop 2023-08-09T13:51:45.389 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:45.381+0000 7f49b286a640 1 -- 172.21.15.43:0/1412761528 shutdown_connections 2023-08-09T13:51:45.389 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:45.381+0000 7f49b286a640 1 --2- 172.21.15.43:0/1412761528 >> v2:172.21.15.203:3300/0 conn(0x7f49ac0aff80 0x7f49ac070d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:51:45.390 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:45.381+0000 7f49b286a640 1 --2- 172.21.15.43:0/1412761528 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f49ac15a330 0x7f49ac0c9780 unknown :-1 s=CLOSED pgs=178 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:51:45.390 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:45.381+0000 7f49b286a640 1 -- 172.21.15.43:0/1412761528 >> 172.21.15.43:0/1412761528 conn(0x7f49ac0ca870 msgr2=0x7f49ac0cac50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-09T13:51:45.390 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:45.381+0000 7f49b286a640 1 -- 172.21.15.43:0/1412761528 shutdown_connections 2023-08-09T13:51:45.390 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:45.381+0000 7f49b286a640 1 -- 172.21.15.43:0/1412761528 wait complete. 2023-08-09T13:51:45.390 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:45.383+0000 7f49b286a640 1 Processor -- start 2023-08-09T13:51:45.391 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:45.383+0000 7f49b286a640 1 -- start start 2023-08-09T13:51:45.391 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:45.384+0000 7f49b286a640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f49ac0aff80 0x7f49ac1ef750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:51:45.391 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:45.384+0000 7f49b286a640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f49ac15a330 0x7f49ac0c77b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:51:45.391 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:45.384+0000 7f49b286a640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f49ac15ea60 0x7f49ac157fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:51:45.392 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:45.384+0000 7f49b286a640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f49ac0c7cf0 con 0x7f49ac15a330 2023-08-09T13:51:45.392 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:45.384+0000 7f49b286a640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f49ac1efe10 con 0x7f49ac15ea60 2023-08-09T13:51:45.392 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:45.384+0000 7f49b286a640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f49ac1f00f0 con 0x7f49ac0aff80 2023-08-09T13:51:45.393 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:45.384+0000 7f49b1067640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f49ac15a330 0x7f49ac0c77b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:51:45.393 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:45.385+0000 7f49b1067640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f49ac15a330 0x7f49ac0c77b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:48648/0 (socket says 172.21.15.43:48648) 2023-08-09T13:51:45.393 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:45.385+0000 7f49b1067640 1 -- 172.21.15.43:0/390158592 learned_addr learned my addr 172.21.15.43:0/390158592 (peer_addr_for_me v2:172.21.15.43:0/0) 2023-08-09T13:51:45.393 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:45.385+0000 7f49b1868640 1 --2- 172.21.15.43:0/390158592 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f49ac0aff80 0x7f49ac1ef750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:51:45.394 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:45.385+0000 7f49b2069640 1 --2- 172.21.15.43:0/390158592 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f49ac15ea60 0x7f49ac157fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:51:45.394 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:45.385+0000 7f49b1067640 1 -- 172.21.15.43:0/390158592 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f49ac0aff80 msgr2=0x7f49ac1ef750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:51:45.394 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:45.385+0000 7f49b1067640 1 --2- 172.21.15.43:0/390158592 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f49ac0aff80 0x7f49ac1ef750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:51:45.395 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:45.385+0000 7f49b1067640 1 -- 172.21.15.43:0/390158592 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f49ac15ea60 msgr2=0x7f49ac157fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:51:45.395 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:45.385+0000 7f49b1067640 1 --2- 172.21.15.43:0/390158592 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f49ac15ea60 0x7f49ac157fb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:51:45.395 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:45.385+0000 7f49b1067640 1 -- 172.21.15.43:0/390158592 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f49a80094d0 con 0x7f49ac15a330 2023-08-09T13:51:45.396 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:45.385+0000 7f49b2069640 1 --2- 172.21.15.43:0/390158592 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f49ac15ea60 0x7f49ac157fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-08-09T13:51:45.396 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:45.385+0000 7f49b1067640 1 --2- 172.21.15.43:0/390158592 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f49ac15a330 0x7f49ac0c77b0 secure :-1 s=READY pgs=1276 cs=0 l=1 rev1=1 crypto rx=0x7f499c00c400 tx=0x7f499c00c8c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-09T13:51:45.396 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:45.386+0000 7f49a27fc640 1 -- 172.21.15.43:0/390158592 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f499c0074e0 con 0x7f49ac15a330 2023-08-09T13:51:45.396 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:45.386+0000 7f49a27fc640 1 -- 172.21.15.43:0/390158592 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f499c007640 con 0x7f49ac15a330 2023-08-09T13:51:45.397 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:45.386+0000 7f49a27fc640 1 -- 172.21.15.43:0/390158592 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f499c004a00 con 0x7f49ac15a330 2023-08-09T13:51:45.397 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:45.386+0000 7f49b286a640 1 -- 172.21.15.43:0/390158592 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f49ac1f03d0 con 0x7f49ac15a330 2023-08-09T13:51:45.397 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:45.386+0000 7f49b286a640 1 -- 172.21.15.43:0/390158592 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f49ac0af660 con 0x7f49ac15a330 2023-08-09T13:51:45.399 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:45.387+0000 7f49b286a640 1 -- 172.21.15.43:0/390158592 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f49ac0739f0 con 0x7f49ac15a330 2023-08-09T13:51:45.399 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:45.388+0000 7f49a27fc640 1 -- 172.21.15.43:0/390158592 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 45) v1 ==== 88443+0+0 (secure 0 0 0) 0x7f499c004b60 con 0x7f49ac15a330 2023-08-09T13:51:45.399 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:45.389+0000 7f49a27fc640 1 --2- 172.21.15.43:0/390158592 >> [v2:172.21.15.203:6832/1317108840,v1:172.21.15.203:6833/1317108840] conn(0x7f499006a170 0x7f499006c630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:51:45.400 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:45.389+0000 7f49a27fc640 1 -- 172.21.15.43:0/390158592 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(106..106 src has 1..106) v4 ==== 7200+0+0 (secure 0 0 0) 0x7f499c088630 con 0x7f49ac15a330 2023-08-09T13:51:45.400 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:45.390+0000 7f49b1868640 1 --2- 172.21.15.43:0/390158592 >> [v2:172.21.15.203:6832/1317108840,v1:172.21.15.203:6833/1317108840] conn(0x7f499006a170 0x7f499006c630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:51:45.400 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:45.391+0000 7f49b1868640 1 --2- 172.21.15.43:0/390158592 >> [v2:172.21.15.203:6832/1317108840,v1:172.21.15.203:6833/1317108840] conn(0x7f499006a170 0x7f499006c630 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0x7f49a8036000 tx=0x7f49a8008030 comp rx=0 tx=0).ready entity=mgr.6619 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-09T13:51:45.401 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:45.393+0000 7f49a27fc640 1 -- 172.21.15.43:0/390158592 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178066 (secure 0 0 0) 0x7f499c0537d0 con 0x7f49ac15a330 2023-08-09T13:51:45.595 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:45.593+0000 7f49b286a640 1 -- 172.21.15.43:0/390158592 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7f49ac0afab0 con 0x7f49ac15a330 2023-08-09T13:51:46.612 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:46.607+0000 7f49a27fc640 1 -- 172.21.15.43:0/390158592 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_metadata' removed v107) v1 ==== 181+0+0 (secure 0 0 0) 0x7f499c056df0 con 0x7f49ac15a330 2023-08-09T13:51:46.613 INFO:teuthology.orchestra.run.smithi043.stderr:pool 'cephfs_metadata' removed 2023-08-09T13:51:46.618 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:46.616+0000 7f49b286a640 1 -- 172.21.15.43:0/390158592 >> [v2:172.21.15.203:6832/1317108840,v1:172.21.15.203:6833/1317108840] conn(0x7f499006a170 msgr2=0x7f499006c630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:51:46.619 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:46.616+0000 7f49b286a640 1 --2- 172.21.15.43:0/390158592 >> [v2:172.21.15.203:6832/1317108840,v1:172.21.15.203:6833/1317108840] conn(0x7f499006a170 0x7f499006c630 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0x7f49a8036000 tx=0x7f49a8008030 comp rx=0 tx=0).stop 2023-08-09T13:51:46.619 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:46.617+0000 7f49b286a640 1 -- 172.21.15.43:0/390158592 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f49ac15a330 msgr2=0x7f49ac0c77b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:51:46.619 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:46.617+0000 7f49b286a640 1 --2- 172.21.15.43:0/390158592 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f49ac15a330 0x7f49ac0c77b0 secure :-1 s=READY pgs=1276 cs=0 l=1 rev1=1 crypto rx=0x7f499c00c400 tx=0x7f499c00c8c0 comp rx=0 tx=0).stop 2023-08-09T13:51:46.621 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:46.618+0000 7f49b286a640 1 -- 172.21.15.43:0/390158592 shutdown_connections 2023-08-09T13:51:46.621 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:46.618+0000 7f49b286a640 1 --2- 172.21.15.43:0/390158592 >> [v2:172.21.15.203:6832/1317108840,v1:172.21.15.203:6833/1317108840] conn(0x7f499006a170 0x7f499006c630 unknown :-1 s=CLOSED pgs=34 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:51:46.621 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:46.618+0000 7f49b286a640 1 --2- 172.21.15.43:0/390158592 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f49ac15ea60 0x7f49ac157fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:51:46.621 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:46.618+0000 7f49b286a640 1 --2- 172.21.15.43:0/390158592 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f49ac15a330 0x7f49ac0c77b0 unknown :-1 s=CLOSED pgs=1276 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:51:46.622 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:46.618+0000 7f49b286a640 1 --2- 172.21.15.43:0/390158592 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f49ac0aff80 0x7f49ac1ef750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:51:46.622 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:46.618+0000 7f49b286a640 1 -- 172.21.15.43:0/390158592 >> 172.21.15.43:0/390158592 conn(0x7f49ac0ca870 msgr2=0x7f49ac159630 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-09T13:51:46.622 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:46.618+0000 7f49b286a640 1 -- 172.21.15.43:0/390158592 shutdown_connections 2023-08-09T13:51:46.623 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:46.618+0000 7f49b286a640 1 -- 172.21.15.43:0/390158592 wait complete. 2023-08-09T13:51:46.643 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_data cephfs_data --yes-i-really-really-mean-it 2023-08-09T13:51:46.866 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:46.865+0000 7f1f60938640 1 Processor -- start 2023-08-09T13:51:46.866 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:46.865+0000 7f1f60938640 1 -- start start 2023-08-09T13:51:46.868 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:46.866+0000 7f1f60938640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f1f5c06fb70 0x7f1f5c0caab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:51:46.868 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:46.867+0000 7f1f60938640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f1f5c0ce4c0 0x7f1f5c0ce950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:51:46.869 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:46.867+0000 7f1f60938640 1 -- --> v1:172.21.15.43:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1f5c15b9b0 con 0x7f1f5c0ca370 2023-08-09T13:51:46.869 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:46.867+0000 7f1f60938640 1 -- --> v2:172.21.15.43:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1f5c0cee90 con 0x7f1f5c06fb70 2023-08-09T13:51:46.869 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:46.867+0000 7f1f60938640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1f5c15ae10 con 0x7f1f5c0ce4c0 2023-08-09T13:51:46.869 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:46.867+0000 7f1f5a59c640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f1f5c06fb70 0x7f1f5c0caab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:51:46.869 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:46.867+0000 7f1f5a59c640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f1f5c06fb70 0x7f1f5c0caab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:48664/0 (socket says 172.21.15.43:48664) 2023-08-09T13:51:46.869 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:46.867+0000 7f1f5a59c640 1 -- 172.21.15.43:0/64277861 learned_addr learned my addr 172.21.15.43:0/64277861 (peer_addr_for_me v2:172.21.15.43:0/0) 2023-08-09T13:51:46.870 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:46.867+0000 7f1f5ad9d640 1 --2- 172.21.15.43:0/64277861 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f1f5c0ce4c0 0x7f1f5c0ce950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:51:46.870 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:46.868+0000 7f1f5a59c640 1 -- 172.21.15.43:0/64277861 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f1f5c0ce4c0 msgr2=0x7f1f5c0ce950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-09T13:51:46.870 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:46.868+0000 7f1f5a59c640 1 --2- 172.21.15.43:0/64277861 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f1f5c0ce4c0 0x7f1f5c0ce950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:51:46.870 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:46.868+0000 7f1f5a59c640 1 -- 172.21.15.43:0/64277861 >> v1:172.21.15.43:6789/0 conn(0x7f1f5c0ca370 legacy=0x7f1f5c15b2a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:51:46.870 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:46.868+0000 7f1f5a59c640 1 -- 172.21.15.43:0/64277861 --> v2:172.21.15.43:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1f5c154e70 con 0x7f1f5c06fb70 2023-08-09T13:51:46.871 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:46.868+0000 7f1f5ad9d640 1 --2- 172.21.15.43:0/64277861 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f1f5c0ce4c0 0x7f1f5c0ce950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-08-09T13:51:46.871 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:46.869+0000 7f1f5a59c640 1 --2- 172.21.15.43:0/64277861 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f1f5c06fb70 0x7f1f5c0caab0 secure :-1 s=READY pgs=1278 cs=0 l=1 rev1=1 crypto rx=0x7f1f4c009870 tx=0x7f1f4c02b980 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5f2544c812f9e3a7 server_cookie=0 in_seq=0 out_seq=0 2023-08-09T13:51:46.871 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:46.869+0000 7f1f5959a640 1 -- 172.21.15.43:0/64277861 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1f4c030840 con 0x7f1f5c06fb70 2023-08-09T13:51:46.871 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:46.870+0000 7f1f5959a640 1 -- 172.21.15.43:0/64277861 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1f4c0309a0 con 0x7f1f5c06fb70 2023-08-09T13:51:46.872 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:46.870+0000 7f1f5959a640 1 -- 172.21.15.43:0/64277861 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1f4c030d30 con 0x7f1f5c06fb70 2023-08-09T13:51:46.872 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:46.870+0000 7f1f60938640 1 -- 172.21.15.43:0/64277861 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f1f5c06fb70 msgr2=0x7f1f5c0caab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:51:46.872 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:46.870+0000 7f1f60938640 1 --2- 172.21.15.43:0/64277861 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f1f5c06fb70 0x7f1f5c0caab0 secure :-1 s=READY pgs=1278 cs=0 l=1 rev1=1 crypto rx=0x7f1f4c009870 tx=0x7f1f4c02b980 comp rx=0 tx=0).stop 2023-08-09T13:51:46.873 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:46.871+0000 7f1f60938640 1 -- 172.21.15.43:0/64277861 shutdown_connections 2023-08-09T13:51:46.873 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:46.871+0000 7f1f60938640 1 --2- 172.21.15.43:0/64277861 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f1f5c0ce4c0 0x7f1f5c0ce950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:51:46.873 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:46.871+0000 7f1f60938640 1 --2- 172.21.15.43:0/64277861 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f1f5c06fb70 0x7f1f5c0caab0 unknown :-1 s=CLOSED pgs=1278 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:51:46.873 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:46.871+0000 7f1f60938640 1 -- 172.21.15.43:0/64277861 >> 172.21.15.43:0/64277861 conn(0x7f1f5c06b990 msgr2=0x7f1f5c06bda0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-09T13:51:46.873 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:46.871+0000 7f1f60938640 1 -- 172.21.15.43:0/64277861 shutdown_connections 2023-08-09T13:51:46.873 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:46.871+0000 7f1f60938640 1 -- 172.21.15.43:0/64277861 wait complete. 2023-08-09T13:51:46.874 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:46.872+0000 7f1f60938640 1 Processor -- start 2023-08-09T13:51:46.874 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:46.873+0000 7f1f60938640 1 -- start start 2023-08-09T13:51:46.875 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:46.874+0000 7f1f60938640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f1f5c06fb70 0x7f1f5c0cc460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:51:46.876 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:46.874+0000 7f1f60938640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f1f5c0ca370 0x7f1f5c1546a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:51:46.876 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:46.874+0000 7f1f60938640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f1f5c0ce4c0 0x7f1f5c1f54b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:51:46.876 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:46.874+0000 7f1f60938640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1f5c154be0 con 0x7f1f5c06fb70 2023-08-09T13:51:46.876 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:46.874+0000 7f1f60938640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1f5c0ac650 con 0x7f1f5c0ce4c0 2023-08-09T13:51:46.876 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:46.874+0000 7f1f60938640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1f5c0ac930 con 0x7f1f5c0ca370 2023-08-09T13:51:46.876 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:46.874+0000 7f1f5a59c640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f1f5c06fb70 0x7f1f5c0cc460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:51:46.876 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:46.874+0000 7f1f5a59c640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f1f5c06fb70 0x7f1f5c0cc460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:48666/0 (socket says 172.21.15.43:48666) 2023-08-09T13:51:46.877 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:46.874+0000 7f1f5a59c640 1 -- 172.21.15.43:0/772453992 learned_addr learned my addr 172.21.15.43:0/772453992 (peer_addr_for_me v2:172.21.15.43:0/0) 2023-08-09T13:51:46.878 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:46.874+0000 7f1f5a59c640 1 -- 172.21.15.43:0/772453992 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f1f5c0ca370 msgr2=0x7f1f5c1546a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:51:46.878 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:46.874+0000 7f1f59d9b640 1 --2- 172.21.15.43:0/772453992 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f1f5c0ca370 0x7f1f5c1546a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:51:46.878 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:46.874+0000 7f1f5ad9d640 1 --2- 172.21.15.43:0/772453992 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f1f5c0ce4c0 0x7f1f5c1f54b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:51:46.879 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:46.874+0000 7f1f5a59c640 1 --2- 172.21.15.43:0/772453992 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f1f5c0ca370 0x7f1f5c1546a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:51:46.879 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:46.874+0000 7f1f5a59c640 1 -- 172.21.15.43:0/772453992 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f1f5c0ce4c0 msgr2=0x7f1f5c1f54b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:51:46.879 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:46.874+0000 7f1f5a59c640 1 --2- 172.21.15.43:0/772453992 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f1f5c0ce4c0 0x7f1f5c1f54b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:51:46.879 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:46.874+0000 7f1f5a59c640 1 -- 172.21.15.43:0/772453992 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1f4c0094d0 con 0x7f1f5c06fb70 2023-08-09T13:51:46.879 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:46.875+0000 7f1f5a59c640 1 --2- 172.21.15.43:0/772453992 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f1f5c06fb70 0x7f1f5c0cc460 secure :-1 s=READY pgs=1279 cs=0 l=1 rev1=1 crypto rx=0x7f1f4c000c00 tx=0x7f1f4c004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-09T13:51:46.879 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:46.875+0000 7f1f4affd640 1 -- 172.21.15.43:0/772453992 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1f4c030810 con 0x7f1f5c06fb70 2023-08-09T13:51:46.879 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:46.875+0000 7f1f4affd640 1 -- 172.21.15.43:0/772453992 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1f4c043400 con 0x7f1f5c06fb70 2023-08-09T13:51:46.880 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:46.875+0000 7f1f60938640 1 -- 172.21.15.43:0/772453992 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1f5c0acc10 con 0x7f1f5c06fb70 2023-08-09T13:51:46.880 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:46.875+0000 7f1f4affd640 1 -- 172.21.15.43:0/772453992 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1f4c0437d0 con 0x7f1f5c06fb70 2023-08-09T13:51:46.880 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:46.875+0000 7f1f60938640 1 -- 172.21.15.43:0/772453992 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1f5c0ad1d0 con 0x7f1f5c06fb70 2023-08-09T13:51:46.880 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:46.877+0000 7f1f60938640 1 -- 172.21.15.43:0/772453992 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1f5c06bd70 con 0x7f1f5c06fb70 2023-08-09T13:51:46.881 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:46.877+0000 7f1f4affd640 1 -- 172.21.15.43:0/772453992 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 45) v1 ==== 88443+0+0 (secure 0 0 0) 0x7f1f4c043930 con 0x7f1f5c06fb70 2023-08-09T13:51:46.881 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:46.878+0000 7f1f4affd640 1 --2- 172.21.15.43:0/772453992 >> [v2:172.21.15.203:6832/1317108840,v1:172.21.15.203:6833/1317108840] conn(0x7f1f3c06a170 0x7f1f3c06c630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:51:46.881 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:46.878+0000 7f1f4affd640 1 -- 172.21.15.43:0/772453992 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(107..107 src has 1..107) v4 ==== 6762+0+0 (secure 0 0 0) 0x7f1f4c03e070 con 0x7f1f5c06fb70 2023-08-09T13:51:46.882 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:46.879+0000 7f1f59d9b640 1 --2- 172.21.15.43:0/772453992 >> [v2:172.21.15.203:6832/1317108840,v1:172.21.15.203:6833/1317108840] conn(0x7f1f3c06a170 0x7f1f3c06c630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:51:46.882 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:46.880+0000 7f1f59d9b640 1 --2- 172.21.15.43:0/772453992 >> [v2:172.21.15.203:6832/1317108840,v1:172.21.15.203:6833/1317108840] conn(0x7f1f3c06a170 0x7f1f3c06c630 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7f1f50007fd0 tx=0x7f1f50009040 comp rx=0 tx=0).ready entity=mgr.6619 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-09T13:51:46.884 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:46.883+0000 7f1f4affd640 1 -- 172.21.15.43:0/772453992 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178066 (secure 0 0 0) 0x7f1f4c076fb0 con 0x7f1f5c06fb70 2023-08-09T13:51:47.085 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:47.084+0000 7f1f60938640 1 -- 172.21.15.43:0/772453992 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7f1f5c0739f0 con 0x7f1f5c06fb70 2023-08-09T13:51:47.620 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:47.618+0000 7f1f4affd640 1 -- 172.21.15.43:0/772453992 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data' removed v108) v1 ==== 169+0+0 (secure 0 0 0) 0x7f1f4c034020 con 0x7f1f5c06fb70 2023-08-09T13:51:47.621 INFO:teuthology.orchestra.run.smithi043.stderr:pool 'cephfs_data' removed 2023-08-09T13:51:47.626 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:47.625+0000 7f1f60938640 1 -- 172.21.15.43:0/772453992 >> [v2:172.21.15.203:6832/1317108840,v1:172.21.15.203:6833/1317108840] conn(0x7f1f3c06a170 msgr2=0x7f1f3c06c630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:51:47.627 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:47.625+0000 7f1f60938640 1 --2- 172.21.15.43:0/772453992 >> [v2:172.21.15.203:6832/1317108840,v1:172.21.15.203:6833/1317108840] conn(0x7f1f3c06a170 0x7f1f3c06c630 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7f1f50007fd0 tx=0x7f1f50009040 comp rx=0 tx=0).stop 2023-08-09T13:51:47.627 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:47.625+0000 7f1f60938640 1 -- 172.21.15.43:0/772453992 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f1f5c06fb70 msgr2=0x7f1f5c0cc460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:51:47.627 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:47.625+0000 7f1f60938640 1 --2- 172.21.15.43:0/772453992 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f1f5c06fb70 0x7f1f5c0cc460 secure :-1 s=READY pgs=1279 cs=0 l=1 rev1=1 crypto rx=0x7f1f4c000c00 tx=0x7f1f4c004730 comp rx=0 tx=0).stop 2023-08-09T13:51:47.628 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:47.625+0000 7f1f60938640 1 -- 172.21.15.43:0/772453992 shutdown_connections 2023-08-09T13:51:47.628 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:47.626+0000 7f1f60938640 1 --2- 172.21.15.43:0/772453992 >> [v2:172.21.15.203:6832/1317108840,v1:172.21.15.203:6833/1317108840] conn(0x7f1f3c06a170 0x7f1f3c06c630 unknown :-1 s=CLOSED pgs=35 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:51:47.628 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:47.626+0000 7f1f60938640 1 --2- 172.21.15.43:0/772453992 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f1f5c0ce4c0 0x7f1f5c1f54b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:51:47.628 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:47.626+0000 7f1f60938640 1 --2- 172.21.15.43:0/772453992 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f1f5c0ca370 0x7f1f5c1546a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:51:47.629 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:47.626+0000 7f1f60938640 1 --2- 172.21.15.43:0/772453992 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f1f5c06fb70 0x7f1f5c0cc460 unknown :-1 s=CLOSED pgs=1279 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:51:47.629 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:47.626+0000 7f1f60938640 1 -- 172.21.15.43:0/772453992 >> 172.21.15.43:0/772453992 conn(0x7f1f5c06b990 msgr2=0x7f1f5c0b04b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-09T13:51:47.629 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:47.626+0000 7f1f60938640 1 -- 172.21.15.43:0/772453992 shutdown_connections 2023-08-09T13:51:47.629 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:47.627+0000 7f1f60938640 1 -- 172.21.15.43:0/772453992 wait complete. 2023-08-09T13:51:47.655 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config rm mgr mgr/snap_schedule/dump_on_update 2023-08-09T13:51:47.878 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:47.876+0000 7f3752eec640 1 Processor -- start 2023-08-09T13:51:47.878 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:47.876+0000 7f3752eec640 1 -- start start 2023-08-09T13:51:47.879 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:47.878+0000 7f3752eec640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f374c0cb2d0 0x7f374c15b380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:51:47.880 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:47.878+0000 7f3752eec640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f374c0ce8d0 0x7f374c0c9f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:51:47.880 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:47.878+0000 7f3752eec640 1 -- --> v1:172.21.15.203:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f374c15aef0 con 0x7f374c15df60 2023-08-09T13:51:47.880 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:47.878+0000 7f3752eec640 1 -- --> v2:172.21.15.43:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f374c15a970 con 0x7f374c0cb2d0 2023-08-09T13:51:47.880 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:47.878+0000 7f3752eec640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f374c15b8c0 con 0x7f374c0ce8d0 2023-08-09T13:51:47.881 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:47.879+0000 7f3750c88640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f374c0cb2d0 0x7f374c15b380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:51:47.881 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:47.879+0000 7f3750c88640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f374c0cb2d0 0x7f374c15b380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:48672/0 (socket says 172.21.15.43:48672) 2023-08-09T13:51:47.881 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:47.879+0000 7f3750c88640 1 -- 172.21.15.43:0/39981218 learned_addr learned my addr 172.21.15.43:0/39981218 (peer_addr_for_me v2:172.21.15.43:0/0) 2023-08-09T13:51:47.881 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:47.879+0000 7f3751489640 1 --2- 172.21.15.43:0/39981218 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f374c0ce8d0 0x7f374c0c9f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:51:47.882 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:47.879+0000 7f3750c88640 1 -- 172.21.15.43:0/39981218 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f374c0ce8d0 msgr2=0x7f374c0c9f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-09T13:51:47.882 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:47.879+0000 7f3750c88640 1 --2- 172.21.15.43:0/39981218 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f374c0ce8d0 0x7f374c0c9f90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:51:47.882 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:47.879+0000 7f3750c88640 1 -- 172.21.15.43:0/39981218 >> v1:172.21.15.203:6789/0 conn(0x7f374c15df60 legacy=0x7f374c0ce1c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-09T13:51:47.882 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:47.879+0000 7f3750c88640 1 -- 172.21.15.43:0/39981218 --> v2:172.21.15.43:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f374c15e940 con 0x7f374c0cb2d0 2023-08-09T13:51:47.882 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:47.881+0000 7f3750c88640 1 --2- 172.21.15.43:0/39981218 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f374c0cb2d0 0x7f374c15b380 secure :-1 s=READY pgs=1280 cs=0 l=1 rev1=1 crypto rx=0x7f3740009870 tx=0x7f374002b8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=afb47ac7250213f1 server_cookie=0 in_seq=0 out_seq=0 2023-08-09T13:51:47.883 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:47.881+0000 7f374b7fe640 1 -- 172.21.15.43:0/39981218 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3740031640 con 0x7f374c0cb2d0 2023-08-09T13:51:47.883 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:47.881+0000 7f374b7fe640 1 -- 172.21.15.43:0/39981218 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f37400317a0 con 0x7f374c0cb2d0 2023-08-09T13:51:47.883 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:47.881+0000 7f374b7fe640 1 -- 172.21.15.43:0/39981218 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3740031b30 con 0x7f374c0cb2d0 2023-08-09T13:51:47.884 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:47.882+0000 7f3752eec640 1 -- 172.21.15.43:0/39981218 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f374c0cb2d0 msgr2=0x7f374c15b380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:51:47.884 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:47.882+0000 7f3752eec640 1 --2- 172.21.15.43:0/39981218 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f374c0cb2d0 0x7f374c15b380 secure :-1 s=READY pgs=1280 cs=0 l=1 rev1=1 crypto rx=0x7f3740009870 tx=0x7f374002b8f0 comp rx=0 tx=0).stop 2023-08-09T13:51:47.884 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:47.882+0000 7f3752eec640 1 -- 172.21.15.43:0/39981218 shutdown_connections 2023-08-09T13:51:47.885 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:47.882+0000 7f3752eec640 1 --2- 172.21.15.43:0/39981218 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f374c0ce8d0 0x7f374c0c9f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:51:47.885 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:47.882+0000 7f3752eec640 1 --2- 172.21.15.43:0/39981218 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f374c0cb2d0 0x7f374c15b380 unknown :-1 s=CLOSED pgs=1280 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:51:47.885 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:47.882+0000 7f3752eec640 1 -- 172.21.15.43:0/39981218 >> 172.21.15.43:0/39981218 conn(0x7f374c0aff50 msgr2=0x7f374c0b0360 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-09T13:51:47.885 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:47.882+0000 7f3752eec640 1 -- 172.21.15.43:0/39981218 shutdown_connections 2023-08-09T13:51:47.886 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:47.882+0000 7f3752eec640 1 -- 172.21.15.43:0/39981218 wait complete. 2023-08-09T13:51:47.888 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:47.884+0000 7f3752eec640 1 Processor -- start 2023-08-09T13:51:47.888 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:47.884+0000 7f3752eec640 1 -- start start 2023-08-09T13:51:47.889 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:47.885+0000 7f3752eec640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f374c0cb2d0 0x7f374c15bb30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:51:47.889 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:47.885+0000 7f3752eec640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f374c0ce8d0 0x7f374c0c6d60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:51:47.889 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:47.885+0000 7f3752eec640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f374c15df60 0x7f374c0c7ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:51:47.889 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:47.885+0000 7f3752eec640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f374c0c72a0 con 0x7f374c15df60 2023-08-09T13:51:47.889 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:47.885+0000 7f3752eec640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f374c15c1c0 con 0x7f374c0ce8d0 2023-08-09T13:51:47.889 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:47.885+0000 7f3752eec640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f374c15c4a0 con 0x7f374c0cb2d0 2023-08-09T13:51:47.889 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:47.885+0000 7f3751489640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f374c15df60 0x7f374c0c7ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:51:47.889 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:47.885+0000 7f3751489640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f374c15df60 0x7f374c0c7ab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:48678/0 (socket says 172.21.15.43:48678) 2023-08-09T13:51:47.890 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:47.885+0000 7f3751489640 1 -- 172.21.15.43:0/1282969476 learned_addr learned my addr 172.21.15.43:0/1282969476 (peer_addr_for_me v2:172.21.15.43:0/0) 2023-08-09T13:51:47.890 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:47.886+0000 7f374bfff640 1 --2- 172.21.15.43:0/1282969476 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f374c0ce8d0 0x7f374c0c6d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:51:47.890 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:47.886+0000 7f3750c88640 1 --2- 172.21.15.43:0/1282969476 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f374c0cb2d0 0x7f374c15bb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:51:47.890 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:47.886+0000 7f3751489640 1 -- 172.21.15.43:0/1282969476 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f374c0cb2d0 msgr2=0x7f374c15bb30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:51:47.890 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:47.886+0000 7f3751489640 1 --2- 172.21.15.43:0/1282969476 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f374c0cb2d0 0x7f374c15bb30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:51:47.890 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:47.886+0000 7f3751489640 1 -- 172.21.15.43:0/1282969476 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f374c0ce8d0 msgr2=0x7f374c0c6d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:51:47.890 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:47.886+0000 7f3751489640 1 --2- 172.21.15.43:0/1282969476 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f374c0ce8d0 0x7f374c0c6d60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:51:47.891 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:47.886+0000 7f3751489640 1 -- 172.21.15.43:0/1282969476 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f37400094d0 con 0x7f374c15df60 2023-08-09T13:51:47.891 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:47.886+0000 7f3751489640 1 --2- 172.21.15.43:0/1282969476 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f374c15df60 0x7f374c0c7ab0 secure :-1 s=READY pgs=1281 cs=0 l=1 rev1=1 crypto rx=0x7f3744007e40 tx=0x7f374400e4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-09T13:51:47.891 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:47.886+0000 7f37497fa640 1 -- 172.21.15.43:0/1282969476 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3744014620 con 0x7f374c15df60 2023-08-09T13:51:47.891 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:47.886+0000 7f37497fa640 1 -- 172.21.15.43:0/1282969476 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f374400ee90 con 0x7f374c15df60 2023-08-09T13:51:47.891 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:47.886+0000 7f37497fa640 1 -- 172.21.15.43:0/1282969476 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f37440149e0 con 0x7f374c15df60 2023-08-09T13:51:47.891 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:47.886+0000 7f3752eec640 1 -- 172.21.15.43:0/1282969476 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f374c15c780 con 0x7f374c15df60 2023-08-09T13:51:47.891 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:47.887+0000 7f3752eec640 1 -- 172.21.15.43:0/1282969476 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f374c15cd40 con 0x7f374c15df60 2023-08-09T13:51:47.892 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:47.888+0000 7f3752eec640 1 -- 172.21.15.43:0/1282969476 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f374c15cf60 con 0x7f374c15df60 2023-08-09T13:51:47.892 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:47.889+0000 7f37497fa640 1 -- 172.21.15.43:0/1282969476 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 45) v1 ==== 88443+0+0 (secure 0 0 0) 0x7f3744014b40 con 0x7f374c15df60 2023-08-09T13:51:47.892 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:47.890+0000 7f37497fa640 1 --2- 172.21.15.43:0/1282969476 >> [v2:172.21.15.203:6832/1317108840,v1:172.21.15.203:6833/1317108840] conn(0x7f373006a1a0 0x7f373006c660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:51:47.893 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:47.890+0000 7f37497fa640 1 -- 172.21.15.43:0/1282969476 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(108..108 src has 1..108) v4 ==== 6348+0+0 (secure 0 0 0) 0x7f3744089280 con 0x7f374c15df60 2023-08-09T13:51:47.893 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:47.891+0000 7f3750c88640 1 --2- 172.21.15.43:0/1282969476 >> [v2:172.21.15.203:6832/1317108840,v1:172.21.15.203:6833/1317108840] conn(0x7f373006a1a0 0x7f373006c660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:51:47.893 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:47.892+0000 7f3750c88640 1 --2- 172.21.15.43:0/1282969476 >> [v2:172.21.15.203:6832/1317108840,v1:172.21.15.203:6833/1317108840] conn(0x7f373006a1a0 0x7f373006c660 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7f3740036000 tx=0x7f374003c040 comp rx=0 tx=0).ready entity=mgr.6619 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-09T13:51:47.896 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:47.895+0000 7f37497fa640 1 -- 172.21.15.43:0/1282969476 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178066 (secure 0 0 0) 0x7f37440546b0 con 0x7f374c15df60 2023-08-09T13:51:48.112 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:48.111+0000 7f3752eec640 1 -- 172.21.15.43:0/1282969476 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "config rm", "who": "mgr", "name": "mgr/snap_schedule/dump_on_update"} v 0) v1 -- 0x7f374c0739f0 con 0x7f374c15df60 2023-08-09T13:51:48.118 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:48.117+0000 7f37497fa640 1 -- 172.21.15.43:0/1282969476 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "config rm", "who": "mgr", "name": "mgr/snap_schedule/dump_on_update"}]=0 v31) v1 ==== 115+0+0 (secure 0 0 0) 0x7f3744057cd0 con 0x7f374c15df60 2023-08-09T13:51:48.122 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:48.120+0000 7f3752eec640 1 -- 172.21.15.43:0/1282969476 >> [v2:172.21.15.203:6832/1317108840,v1:172.21.15.203:6833/1317108840] conn(0x7f373006a1a0 msgr2=0x7f373006c660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:51:48.122 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:48.120+0000 7f3752eec640 1 --2- 172.21.15.43:0/1282969476 >> [v2:172.21.15.203:6832/1317108840,v1:172.21.15.203:6833/1317108840] conn(0x7f373006a1a0 0x7f373006c660 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7f3740036000 tx=0x7f374003c040 comp rx=0 tx=0).stop 2023-08-09T13:51:48.123 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:48.120+0000 7f3752eec640 1 -- 172.21.15.43:0/1282969476 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f374c15df60 msgr2=0x7f374c0c7ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:51:48.124 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:48.120+0000 7f3752eec640 1 --2- 172.21.15.43:0/1282969476 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f374c15df60 0x7f374c0c7ab0 secure :-1 s=READY pgs=1281 cs=0 l=1 rev1=1 crypto rx=0x7f3744007e40 tx=0x7f374400e4b0 comp rx=0 tx=0).stop 2023-08-09T13:51:48.124 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:48.120+0000 7f3752eec640 1 -- 172.21.15.43:0/1282969476 shutdown_connections 2023-08-09T13:51:48.124 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:48.120+0000 7f3752eec640 1 --2- 172.21.15.43:0/1282969476 >> [v2:172.21.15.203:6832/1317108840,v1:172.21.15.203:6833/1317108840] conn(0x7f373006a1a0 0x7f373006c660 unknown :-1 s=CLOSED pgs=36 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:51:48.125 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:48.120+0000 7f3752eec640 1 --2- 172.21.15.43:0/1282969476 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f374c15df60 0x7f374c0c7ab0 unknown :-1 s=CLOSED pgs=1281 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:51:48.125 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:48.120+0000 7f3752eec640 1 --2- 172.21.15.43:0/1282969476 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f374c0ce8d0 0x7f374c0c6d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:51:48.125 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:48.120+0000 7f3752eec640 1 --2- 172.21.15.43:0/1282969476 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f374c0cb2d0 0x7f374c15bb30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:51:48.126 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:48.120+0000 7f3752eec640 1 -- 172.21.15.43:0/1282969476 >> 172.21.15.43:0/1282969476 conn(0x7f374c0aff50 msgr2=0x7f374c06f410 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-09T13:51:48.126 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:48.120+0000 7f3752eec640 1 -- 172.21.15.43:0/1282969476 shutdown_connections 2023-08-09T13:51:48.126 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:48.120+0000 7f3752eec640 1 -- 172.21.15.43:0/1282969476 wait complete. 2023-08-09T13:51:48.146 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config rm mgr mgr/snap_schedule/allow_m_granularity 2023-08-09T13:51:48.361 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:48.360+0000 7f4692339640 1 Processor -- start 2023-08-09T13:51:48.362 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:48.360+0000 7f4692339640 1 -- start start 2023-08-09T13:51:48.363 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:48.361+0000 7f4692339640 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7f468c15c550 0x7f468c0cd6d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:51:48.363 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:48.361+0000 7f4692339640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f468c0b45f0 0x7f468c0b59d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:51:48.364 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:48.362+0000 7f4692339640 1 -- --> v1:172.21.15.203:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f468c0b7bd0 con 0x7f468c0d04b0 2023-08-09T13:51:48.364 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:48.362+0000 7f4692339640 1 -- --> v2:172.21.15.203:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f468c0d1c00 con 0x7f468c15c550 2023-08-09T13:51:48.364 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:48.362+0000 7f4692339640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f468c15c930 con 0x7f468c0b45f0 2023-08-09T13:51:48.365 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:48.362+0000 7f468bfff640 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7f468c15c550 0x7f468c0cd6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:51:48.365 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:48.362+0000 7f46908d6640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f468c0b45f0 0x7f468c0b59d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:51:48.366 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:48.362+0000 7f468bfff640 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7f468c15c550 0x7f468c0cd6d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.203:3300/0 says I am v2:172.21.15.43:43146/0 (socket says 172.21.15.43:43146) 2023-08-09T13:51:48.366 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:48.362+0000 7f468bfff640 1 -- 172.21.15.43:0/1052160708 learned_addr learned my addr 172.21.15.43:0/1052160708 (peer_addr_for_me v2:172.21.15.43:0/0) 2023-08-09T13:51:48.366 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:48.363+0000 7f46908d6640 1 -- 172.21.15.43:0/1052160708 >> v2:172.21.15.203:3300/0 conn(0x7f468c15c550 msgr2=0x7f468c0cd6d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-09T13:51:48.367 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:48.363+0000 7f46908d6640 1 --2- 172.21.15.43:0/1052160708 >> v2:172.21.15.203:3300/0 conn(0x7f468c15c550 0x7f468c0cd6d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:51:48.367 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:48.363+0000 7f46908d6640 1 -- 172.21.15.43:0/1052160708 >> v1:172.21.15.203:6789/0 conn(0x7f468c0d04b0 legacy=0x7f468c0b3ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:51:48.367 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:48.363+0000 7f46908d6640 1 -- 172.21.15.43:0/1052160708 --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f468c0b54f0 con 0x7f468c0b45f0 2023-08-09T13:51:48.369 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:48.364+0000 7f46908d6640 1 --2- 172.21.15.43:0/1052160708 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f468c0b45f0 0x7f468c0b59d0 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7f46840097a0 tx=0x7f468402b820 comp rx=0 tx=0).ready entity=mon.2 client_cookie=fc4cc6ce37f0f820 server_cookie=0 in_seq=0 out_seq=0 2023-08-09T13:51:48.369 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:48.365+0000 7f468affd640 1 -- 172.21.15.43:0/1052160708 <== mon.2 v2:172.21.15.203:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4684031780 con 0x7f468c0b45f0 2023-08-09T13:51:48.369 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:48.365+0000 7f468affd640 1 -- 172.21.15.43:0/1052160708 <== mon.2 v2:172.21.15.203:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f46840318e0 con 0x7f468c0b45f0 2023-08-09T13:51:48.369 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:48.365+0000 7f468affd640 1 -- 172.21.15.43:0/1052160708 <== mon.2 v2:172.21.15.203:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4684031c70 con 0x7f468c0b45f0 2023-08-09T13:51:48.369 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:48.366+0000 7f4692339640 1 -- 172.21.15.43:0/1052160708 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f468c0b45f0 msgr2=0x7f468c0b59d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:51:48.370 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:48.366+0000 7f4692339640 1 --2- 172.21.15.43:0/1052160708 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f468c0b45f0 0x7f468c0b59d0 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7f46840097a0 tx=0x7f468402b820 comp rx=0 tx=0).stop 2023-08-09T13:51:48.370 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:48.366+0000 7f4692339640 1 -- 172.21.15.43:0/1052160708 shutdown_connections 2023-08-09T13:51:48.370 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:48.366+0000 7f4692339640 1 --2- 172.21.15.43:0/1052160708 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f468c0b45f0 0x7f468c0b59d0 unknown :-1 s=CLOSED pgs=179 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:51:48.370 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:48.366+0000 7f4692339640 1 --2- 172.21.15.43:0/1052160708 >> v2:172.21.15.203:3300/0 conn(0x7f468c15c550 0x7f468c0cd6d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:51:48.370 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:48.366+0000 7f4692339640 1 -- 172.21.15.43:0/1052160708 >> 172.21.15.43:0/1052160708 conn(0x7f468c071520 msgr2=0x7f468c0ceee0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-09T13:51:48.370 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:48.366+0000 7f4692339640 1 -- 172.21.15.43:0/1052160708 shutdown_connections 2023-08-09T13:51:48.370 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:48.366+0000 7f4692339640 1 -- 172.21.15.43:0/1052160708 wait complete. 2023-08-09T13:51:48.372 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:48.368+0000 7f4692339640 1 Processor -- start 2023-08-09T13:51:48.372 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:48.368+0000 7f4692339640 1 -- start start 2023-08-09T13:51:48.373 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:48.369+0000 7f4692339640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f468c0b45f0 0x7f468c0c50b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:51:48.373 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:48.369+0000 7f4692339640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f468c0d04b0 0x7f468c15ee70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:51:48.373 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:48.369+0000 7f4692339640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f468c15c550 0x7f468c15f3b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:51:48.373 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:48.369+0000 7f4692339640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f468c145620 con 0x7f468c15c550 2023-08-09T13:51:48.373 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:48.369+0000 7f4692339640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f468c0c55f0 con 0x7f468c0b45f0 2023-08-09T13:51:48.373 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:48.369+0000 7f4692339640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f468c145870 con 0x7f468c0d04b0 2023-08-09T13:51:48.373 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:48.369+0000 7f46908d6640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f468c15c550 0x7f468c15f3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:51:48.373 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:48.370+0000 7f46908d6640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f468c15c550 0x7f468c15f3b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:48684/0 (socket says 172.21.15.43:48684) 2023-08-09T13:51:48.374 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:48.370+0000 7f46908d6640 1 -- 172.21.15.43:0/3130248906 learned_addr learned my addr 172.21.15.43:0/3130248906 (peer_addr_for_me v2:172.21.15.43:0/0) 2023-08-09T13:51:48.374 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:48.370+0000 7f468b7fe640 1 --2- 172.21.15.43:0/3130248906 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f468c0d04b0 0x7f468c15ee70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:51:48.374 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:48.370+0000 7f468bfff640 1 --2- 172.21.15.43:0/3130248906 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f468c0b45f0 0x7f468c0c50b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:51:48.374 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:48.370+0000 7f46908d6640 1 -- 172.21.15.43:0/3130248906 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f468c0d04b0 msgr2=0x7f468c15ee70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:51:48.374 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:48.370+0000 7f46908d6640 1 --2- 172.21.15.43:0/3130248906 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f468c0d04b0 0x7f468c15ee70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:51:48.374 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:48.370+0000 7f46908d6640 1 -- 172.21.15.43:0/3130248906 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f468c0b45f0 msgr2=0x7f468c0c50b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:51:48.374 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:48.370+0000 7f46908d6640 1 --2- 172.21.15.43:0/3130248906 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f468c0b45f0 0x7f468c0c50b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:51:48.375 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:48.370+0000 7f46908d6640 1 -- 172.21.15.43:0/3130248906 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4684009400 con 0x7f468c15c550 2023-08-09T13:51:48.375 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:48.370+0000 7f468bfff640 1 --2- 172.21.15.43:0/3130248906 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f468c0b45f0 0x7f468c0c50b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-08-09T13:51:48.375 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:48.371+0000 7f46908d6640 1 --2- 172.21.15.43:0/3130248906 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f468c15c550 0x7f468c15f3b0 secure :-1 s=READY pgs=1282 cs=0 l=1 rev1=1 crypto rx=0x7f46840098c0 tx=0x7f46840046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-09T13:51:48.375 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:48.371+0000 7f4688ff9640 1 -- 172.21.15.43:0/3130248906 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4684032820 con 0x7f468c15c550 2023-08-09T13:51:48.375 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:48.371+0000 7f4688ff9640 1 -- 172.21.15.43:0/3130248906 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f46840329a0 con 0x7f468c15c550 2023-08-09T13:51:48.375 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:48.371+0000 7f4688ff9640 1 -- 172.21.15.43:0/3130248906 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4684032d50 con 0x7f468c15c550 2023-08-09T13:51:48.376 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:48.371+0000 7f4692339640 1 -- 172.21.15.43:0/3130248906 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f468c145b50 con 0x7f468c15c550 2023-08-09T13:51:48.376 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:48.371+0000 7f4692339640 1 -- 172.21.15.43:0/3130248906 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f468c146110 con 0x7f468c15c550 2023-08-09T13:51:48.376 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:48.372+0000 7f4692339640 1 -- 172.21.15.43:0/3130248906 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f468c0cdff0 con 0x7f468c15c550 2023-08-09T13:51:48.377 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:48.373+0000 7f4688ff9640 1 -- 172.21.15.43:0/3130248906 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 45) v1 ==== 88443+0+0 (secure 0 0 0) 0x7f46840453f0 con 0x7f468c15c550 2023-08-09T13:51:48.377 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:48.374+0000 7f4688ff9640 1 --2- 172.21.15.43:0/3130248906 >> [v2:172.21.15.203:6832/1317108840,v1:172.21.15.203:6833/1317108840] conn(0x7f466c06a170 0x7f466c06c630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:51:48.377 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:48.374+0000 7f4688ff9640 1 -- 172.21.15.43:0/3130248906 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(108..108 src has 1..108) v4 ==== 6348+0+0 (secure 0 0 0) 0x7f46840aab80 con 0x7f468c15c550 2023-08-09T13:51:48.377 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:48.375+0000 7f468bfff640 1 --2- 172.21.15.43:0/3130248906 >> [v2:172.21.15.203:6832/1317108840,v1:172.21.15.203:6833/1317108840] conn(0x7f466c06a170 0x7f466c06c630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:51:48.378 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:48.376+0000 7f468bfff640 1 --2- 172.21.15.43:0/3130248906 >> [v2:172.21.15.203:6832/1317108840,v1:172.21.15.203:6833/1317108840] conn(0x7f466c06a170 0x7f466c06c630 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7f467c009440 tx=0x7f467c00b040 comp rx=0 tx=0).ready entity=mgr.6619 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-09T13:51:48.380 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:48.379+0000 7f4688ff9640 1 -- 172.21.15.43:0/3130248906 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178066 (secure 0 0 0) 0x7f4684076060 con 0x7f468c15c550 2023-08-09T13:51:48.575 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:48.573+0000 7f4692339640 1 -- 172.21.15.43:0/3130248906 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "config rm", "who": "mgr", "name": "mgr/snap_schedule/allow_m_granularity"} v 0) v1 -- 0x7f468c146300 con 0x7f468c15c550 2023-08-09T13:51:48.581 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:48.579+0000 7f4688ff9640 1 -- 172.21.15.43:0/3130248906 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "config rm", "who": "mgr", "name": "mgr/snap_schedule/allow_m_granularity"}]=0 v32) v1 ==== 120+0+0 (secure 0 0 0) 0x7f4684031ed0 con 0x7f468c15c550 2023-08-09T13:51:48.583 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:48.582+0000 7f4692339640 1 -- 172.21.15.43:0/3130248906 >> [v2:172.21.15.203:6832/1317108840,v1:172.21.15.203:6833/1317108840] conn(0x7f466c06a170 msgr2=0x7f466c06c630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:51:48.584 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:48.582+0000 7f4692339640 1 --2- 172.21.15.43:0/3130248906 >> [v2:172.21.15.203:6832/1317108840,v1:172.21.15.203:6833/1317108840] conn(0x7f466c06a170 0x7f466c06c630 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7f467c009440 tx=0x7f467c00b040 comp rx=0 tx=0).stop 2023-08-09T13:51:48.584 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:48.582+0000 7f4692339640 1 -- 172.21.15.43:0/3130248906 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f468c15c550 msgr2=0x7f468c15f3b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:51:48.584 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:48.582+0000 7f4692339640 1 --2- 172.21.15.43:0/3130248906 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f468c15c550 0x7f468c15f3b0 secure :-1 s=READY pgs=1282 cs=0 l=1 rev1=1 crypto rx=0x7f46840098c0 tx=0x7f46840046c0 comp rx=0 tx=0).stop 2023-08-09T13:51:48.584 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:48.582+0000 7f4692339640 1 -- 172.21.15.43:0/3130248906 shutdown_connections 2023-08-09T13:51:48.584 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:48.582+0000 7f4692339640 1 --2- 172.21.15.43:0/3130248906 >> [v2:172.21.15.203:6832/1317108840,v1:172.21.15.203:6833/1317108840] conn(0x7f466c06a170 0x7f466c06c630 unknown :-1 s=CLOSED pgs=37 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:51:48.584 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:48.582+0000 7f4692339640 1 --2- 172.21.15.43:0/3130248906 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f468c15c550 0x7f468c15f3b0 unknown :-1 s=CLOSED pgs=1282 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:51:48.585 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:48.582+0000 7f4692339640 1 --2- 172.21.15.43:0/3130248906 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f468c0d04b0 0x7f468c15ee70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:51:48.585 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:48.582+0000 7f4692339640 1 --2- 172.21.15.43:0/3130248906 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f468c0b45f0 0x7f468c0c50b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:51:48.585 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:48.582+0000 7f4692339640 1 -- 172.21.15.43:0/3130248906 >> 172.21.15.43:0/3130248906 conn(0x7f468c071520 msgr2=0x7f468c0cc770 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-09T13:51:48.586 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:48.582+0000 7f4692339640 1 -- 172.21.15.43:0/3130248906 shutdown_connections 2023-08-09T13:51:48.587 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:48.582+0000 7f4692339640 1 -- 172.21.15.43:0/3130248906 wait complete. 2023-08-09T13:51:48.602 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config rm mon mon_allow_pool_delete 2023-08-09T13:51:48.842 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:48.840+0000 7fadc3626640 1 Processor -- start 2023-08-09T13:51:48.842 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:48.840+0000 7fadc3626640 1 -- start start 2023-08-09T13:51:48.843 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:48.842+0000 7fadc3626640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fadbc0cbb80 0x7fadbc0cb2b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:51:48.844 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:48.842+0000 7fadc3626640 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7fadbc0ce1c0 0x7fadbc0ce5f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:51:48.844 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:48.842+0000 7fadc3626640 1 -- --> v1:172.21.15.203:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fadbc0ceb30 con 0x7fadbc15a0c0 2023-08-09T13:51:48.844 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:48.842+0000 7fadc3626640 1 -- --> v2:172.21.15.203:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fadbc15a4a0 con 0x7fadbc0ce1c0 2023-08-09T13:51:48.845 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:48.842+0000 7fadc3626640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fadbc0cc000 con 0x7fadbc0cbb80 2023-08-09T13:51:48.846 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:48.842+0000 7fadc1bc3640 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7fadbc0ce1c0 0x7fadbc0ce5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:51:48.847 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:48.842+0000 7fadc1bc3640 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7fadbc0ce1c0 0x7fadbc0ce5f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.203:3300/0 says I am v2:172.21.15.43:43166/0 (socket says 172.21.15.43:43166) 2023-08-09T13:51:48.847 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:48.842+0000 7fadc1bc3640 1 -- 172.21.15.43:0/3661761217 learned_addr learned my addr 172.21.15.43:0/3661761217 (peer_addr_for_me v2:172.21.15.43:0/0) 2023-08-09T13:51:48.847 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:48.843+0000 7fadc0bc1640 1 --2- 172.21.15.43:0/3661761217 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fadbc0cbb80 0x7fadbc0cb2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:51:48.848 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:48.843+0000 7fadc1bc3640 1 -- 172.21.15.43:0/3661761217 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fadbc0cbb80 msgr2=0x7fadbc0cb2b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-09T13:51:48.848 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:48.843+0000 7fadc1bc3640 1 --2- 172.21.15.43:0/3661761217 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fadbc0cbb80 0x7fadbc0cb2b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:51:48.848 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:48.843+0000 7fadc1bc3640 1 -- 172.21.15.43:0/3661761217 >> v1:172.21.15.203:6789/0 conn(0x7fadbc15a0c0 legacy=0x7fadbc15d510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:51:48.848 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:48.843+0000 7fadc1bc3640 1 -- 172.21.15.43:0/3661761217 --> v2:172.21.15.203:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fadbc0cc2d0 con 0x7fadbc0ce1c0 2023-08-09T13:51:48.849 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:48.845+0000 7fadc1bc3640 1 --2- 172.21.15.43:0/3661761217 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fadbc0ce1c0 0x7fadbc0ce5f0 secure :-1 s=READY pgs=482 cs=0 l=1 rev1=1 crypto rx=0x7fadb4009e50 tx=0x7fadb402bf60 comp rx=0 tx=0).ready entity=mon.1 client_cookie=f03826c62c164fc6 server_cookie=0 in_seq=0 out_seq=0 2023-08-09T13:51:48.850 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:48.845+0000 7fadb3fff640 1 -- 172.21.15.43:0/3661761217 <== mon.1 v2:172.21.15.203:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fadb402cb00 con 0x7fadbc0ce1c0 2023-08-09T13:51:48.851 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:48.845+0000 7fadb3fff640 1 -- 172.21.15.43:0/3661761217 <== mon.1 v2:172.21.15.203:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fadb402cc60 con 0x7fadbc0ce1c0 2023-08-09T13:51:48.851 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:48.845+0000 7fadb3fff640 1 -- 172.21.15.43:0/3661761217 <== mon.1 v2:172.21.15.203:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fadb40343f0 con 0x7fadbc0ce1c0 2023-08-09T13:51:48.851 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:48.846+0000 7fadc3626640 1 -- 172.21.15.43:0/3661761217 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fadbc0ce1c0 msgr2=0x7fadbc0ce5f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:51:48.851 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:48.846+0000 7fadc3626640 1 --2- 172.21.15.43:0/3661761217 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fadbc0ce1c0 0x7fadbc0ce5f0 secure :-1 s=READY pgs=482 cs=0 l=1 rev1=1 crypto rx=0x7fadb4009e50 tx=0x7fadb402bf60 comp rx=0 tx=0).stop 2023-08-09T13:51:48.851 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:48.846+0000 7fadc3626640 1 -- 172.21.15.43:0/3661761217 shutdown_connections 2023-08-09T13:51:48.851 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:48.846+0000 7fadc3626640 1 --2- 172.21.15.43:0/3661761217 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fadbc0ce1c0 0x7fadbc0ce5f0 unknown :-1 s=CLOSED pgs=482 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:51:48.851 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:48.846+0000 7fadc3626640 1 --2- 172.21.15.43:0/3661761217 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fadbc0cbb80 0x7fadbc0cb2b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:51:48.851 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:48.846+0000 7fadc3626640 1 -- 172.21.15.43:0/3661761217 >> 172.21.15.43:0/3661761217 conn(0x7fadbc0aff50 msgr2=0x7fadbc0b0360 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-09T13:51:48.852 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:48.846+0000 7fadc3626640 1 -- 172.21.15.43:0/3661761217 shutdown_connections 2023-08-09T13:51:48.852 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:48.846+0000 7fadc3626640 1 -- 172.21.15.43:0/3661761217 wait complete. 2023-08-09T13:51:48.852 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:48.848+0000 7fadc3626640 1 Processor -- start 2023-08-09T13:51:48.852 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:48.848+0000 7fadc3626640 1 -- start start 2023-08-09T13:51:48.852 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:48.849+0000 7fadc3626640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fadbc0cbb80 0x7fadbc1e6b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:51:48.852 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:48.849+0000 7fadc3626640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fadbc0ce1c0 0x7fadbc1e7b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:51:48.852 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:48.849+0000 7fadc3626640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fadbc15a0c0 0x7fadbc1f14f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:51:48.853 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:48.849+0000 7fadc3626640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fadbc1e8080 con 0x7fadbc0cbb80 2023-08-09T13:51:48.853 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:48.849+0000 7fadc3626640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fadbc15d510 con 0x7fadbc15a0c0 2023-08-09T13:51:48.853 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:48.849+0000 7fadc3626640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fadbc1e8270 con 0x7fadbc0ce1c0 2023-08-09T13:51:48.854 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:48.849+0000 7fadc13c2640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fadbc0cbb80 0x7fadbc1e6b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:51:48.854 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:48.850+0000 7fadc13c2640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fadbc0cbb80 0x7fadbc1e6b20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:48690/0 (socket says 172.21.15.43:48690) 2023-08-09T13:51:48.854 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:48.850+0000 7fadc13c2640 1 -- 172.21.15.43:0/2469908345 learned_addr learned my addr 172.21.15.43:0/2469908345 (peer_addr_for_me v2:172.21.15.43:0/0) 2023-08-09T13:51:48.854 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:48.850+0000 7fadc13c2640 1 -- 172.21.15.43:0/2469908345 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fadbc0ce1c0 msgr2=0x7fadbc1e7b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:51:48.854 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:48.850+0000 7fadc13c2640 1 --2- 172.21.15.43:0/2469908345 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fadbc0ce1c0 0x7fadbc1e7b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:51:48.855 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:48.850+0000 7fadc13c2640 1 -- 172.21.15.43:0/2469908345 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fadbc15a0c0 msgr2=0x7fadbc1f14f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:51:48.855 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:48.850+0000 7fadc13c2640 1 --2- 172.21.15.43:0/2469908345 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fadbc15a0c0 0x7fadbc1f14f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:51:48.855 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:48.850+0000 7fadc13c2640 1 -- 172.21.15.43:0/2469908345 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fadb4009b30 con 0x7fadbc0cbb80 2023-08-09T13:51:48.855 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:48.850+0000 7fadc13c2640 1 --2- 172.21.15.43:0/2469908345 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fadbc0cbb80 0x7fadbc1e6b20 secure :-1 s=READY pgs=1283 cs=0 l=1 rev1=1 crypto rx=0x7fadb800c430 tx=0x7fadb800c8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-09T13:51:48.855 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:48.851+0000 7fadb1ffb640 1 -- 172.21.15.43:0/2469908345 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fadb80052d0 con 0x7fadbc0cbb80 2023-08-09T13:51:48.856 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:48.851+0000 7fadb1ffb640 1 -- 172.21.15.43:0/2469908345 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fadb8005450 con 0x7fadbc0cbb80 2023-08-09T13:51:48.856 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:48.851+0000 7fadb1ffb640 1 -- 172.21.15.43:0/2469908345 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fadb8003850 con 0x7fadbc0cbb80 2023-08-09T13:51:48.856 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:48.851+0000 7fadc3626640 1 -- 172.21.15.43:0/2469908345 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fadbc1e8550 con 0x7fadbc0cbb80 2023-08-09T13:51:48.856 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:48.851+0000 7fadc3626640 1 -- 172.21.15.43:0/2469908345 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fadbc1e8b10 con 0x7fadbc0cbb80 2023-08-09T13:51:48.856 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:48.852+0000 7fadc3626640 1 -- 172.21.15.43:0/2469908345 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fadbc15d6c0 con 0x7fadbc0cbb80 2023-08-09T13:51:48.857 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:48.853+0000 7fadb1ffb640 1 -- 172.21.15.43:0/2469908345 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 45) v1 ==== 88443+0+0 (secure 0 0 0) 0x7fadb80039b0 con 0x7fadbc0cbb80 2023-08-09T13:51:48.857 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:48.854+0000 7fadb1ffb640 1 --2- 172.21.15.43:0/2469908345 >> [v2:172.21.15.203:6832/1317108840,v1:172.21.15.203:6833/1317108840] conn(0x7fada006a170 0x7fada006c630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:51:48.857 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:48.854+0000 7fadb1ffb640 1 -- 172.21.15.43:0/2469908345 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(108..108 src has 1..108) v4 ==== 6348+0+0 (secure 0 0 0) 0x7fadb8087d50 con 0x7fadbc0cbb80 2023-08-09T13:51:48.857 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:48.855+0000 7fadc0bc1640 1 --2- 172.21.15.43:0/2469908345 >> [v2:172.21.15.203:6832/1317108840,v1:172.21.15.203:6833/1317108840] conn(0x7fada006a170 0x7fada006c630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:51:48.858 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:48.856+0000 7fadc0bc1640 1 --2- 172.21.15.43:0/2469908345 >> [v2:172.21.15.203:6832/1317108840,v1:172.21.15.203:6833/1317108840] conn(0x7fada006a170 0x7fada006c630 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7fadac006a90 tx=0x7fadac006a20 comp rx=0 tx=0).ready entity=mgr.6619 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-09T13:51:48.860 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:48.858+0000 7fadb1ffb640 1 -- 172.21.15.43:0/2469908345 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178066 (secure 0 0 0) 0x7fadbc15d6c0 con 0x7fadbc0cbb80 2023-08-09T13:51:49.054 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:49.052+0000 7fadc3626640 1 -- 172.21.15.43:0/2469908345 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"} v 0) v1 -- 0x7fadbc15d680 con 0x7fadbc0cbb80 2023-08-09T13:51:49.060 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:49.059+0000 7fadb1ffb640 1 -- 172.21.15.43:0/2469908345 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"}]=0 v33) v1 ==== 104+0+0 (secure 0 0 0) 0x7fadb8053200 con 0x7fadbc0cbb80 2023-08-09T13:51:49.063 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:49.061+0000 7fadc3626640 1 -- 172.21.15.43:0/2469908345 >> [v2:172.21.15.203:6832/1317108840,v1:172.21.15.203:6833/1317108840] conn(0x7fada006a170 msgr2=0x7fada006c630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:51:49.063 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:49.061+0000 7fadc3626640 1 --2- 172.21.15.43:0/2469908345 >> [v2:172.21.15.203:6832/1317108840,v1:172.21.15.203:6833/1317108840] conn(0x7fada006a170 0x7fada006c630 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7fadac006a90 tx=0x7fadac006a20 comp rx=0 tx=0).stop 2023-08-09T13:51:49.064 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:49.062+0000 7fadc3626640 1 -- 172.21.15.43:0/2469908345 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fadbc0cbb80 msgr2=0x7fadbc1e6b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:51:49.064 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:49.062+0000 7fadc3626640 1 --2- 172.21.15.43:0/2469908345 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fadbc0cbb80 0x7fadbc1e6b20 secure :-1 s=READY pgs=1283 cs=0 l=1 rev1=1 crypto rx=0x7fadb800c430 tx=0x7fadb800c8f0 comp rx=0 tx=0).stop 2023-08-09T13:51:49.064 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:49.062+0000 7fadc3626640 1 -- 172.21.15.43:0/2469908345 shutdown_connections 2023-08-09T13:51:49.064 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:49.062+0000 7fadc3626640 1 --2- 172.21.15.43:0/2469908345 >> [v2:172.21.15.203:6832/1317108840,v1:172.21.15.203:6833/1317108840] conn(0x7fada006a170 0x7fada006c630 unknown :-1 s=CLOSED pgs=38 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:51:49.064 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:49.062+0000 7fadc3626640 1 --2- 172.21.15.43:0/2469908345 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fadbc15a0c0 0x7fadbc1f14f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:51:49.064 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:49.062+0000 7fadc3626640 1 --2- 172.21.15.43:0/2469908345 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fadbc0ce1c0 0x7fadbc1e7b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:51:49.064 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:49.062+0000 7fadc3626640 1 --2- 172.21.15.43:0/2469908345 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fadbc0cbb80 0x7fadbc1e6b20 unknown :-1 s=CLOSED pgs=1283 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:51:49.065 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:49.062+0000 7fadc3626640 1 -- 172.21.15.43:0/2469908345 >> 172.21.15.43:0/2469908345 conn(0x7fadbc0aff50 msgr2=0x7fadbc15b730 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-09T13:51:49.065 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:49.062+0000 7fadc3626640 1 -- 172.21.15.43:0/2469908345 shutdown_connections 2023-08-09T13:51:49.065 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:49.062+0000 7fadc3626640 1 -- 172.21.15.43:0/2469908345 wait complete. 2023-08-09T13:51:49.084 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Ended test tasks.cephfs.test_snap_schedules.TestSnapSchedules.test_non_existent_snap_schedule_list' 2023-08-09T13:51:49.320 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:49.319+0000 7f5772237640 1 Processor -- start 2023-08-09T13:51:49.321 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:49.319+0000 7f5772237640 1 -- start start 2023-08-09T13:51:49.322 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:49.320+0000 7f5772237640 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7f576c0c8570 0x7f576c070ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:51:49.322 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:49.320+0000 7f5772237640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f576c0c9d70 0x7f576c15b210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:51:49.322 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:49.321+0000 7f5772237640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f576c0c8950 0x7f576c06b9e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:51:49.322 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:49.321+0000 7f5772237640 1 -- --> v2:172.21.15.43:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f576c154cd0 con 0x7f576c0c8950 2023-08-09T13:51:49.323 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:49.321+0000 7f5772237640 1 -- --> v2:172.21.15.203:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f576c15b750 con 0x7f576c0c8570 2023-08-09T13:51:49.323 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:49.321+0000 7f5772237640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f576c06f4f0 con 0x7f576c0c9d70 2023-08-09T13:51:49.324 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:49.321+0000 7f5771a36640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f576c0c8950 0x7f576c06b9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:51:49.324 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:49.321+0000 7f5771a36640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f576c0c8950 0x7f576c06b9e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:48696/0 (socket says 172.21.15.43:48696) 2023-08-09T13:51:49.324 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:49.321+0000 7f5771a36640 1 -- 172.21.15.43:0/1543341712 learned_addr learned my addr 172.21.15.43:0/1543341712 (peer_addr_for_me v2:172.21.15.43:0/0) 2023-08-09T13:51:49.324 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:49.321+0000 7f5771235640 1 --2- 172.21.15.43:0/1543341712 >> v2:172.21.15.203:3300/0 conn(0x7f576c0c8570 0x7f576c070ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:51:49.325 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:49.321+0000 7f5770a34640 1 --2- 172.21.15.43:0/1543341712 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f576c0c9d70 0x7f576c15b210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:51:49.325 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:49.321+0000 7f5771a36640 1 -- 172.21.15.43:0/1543341712 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f576c0c9d70 msgr2=0x7f576c15b210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-09T13:51:49.325 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:49.321+0000 7f5771a36640 1 --2- 172.21.15.43:0/1543341712 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f576c0c9d70 0x7f576c15b210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:51:49.325 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:49.322+0000 7f5771a36640 1 -- 172.21.15.43:0/1543341712 >> v2:172.21.15.203:3300/0 conn(0x7f576c0c8570 msgr2=0x7f576c070ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-09T13:51:49.325 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:49.322+0000 7f5771a36640 1 --2- 172.21.15.43:0/1543341712 >> v2:172.21.15.203:3300/0 conn(0x7f576c0c8570 0x7f576c070ff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:51:49.325 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:49.322+0000 7f5771a36640 1 -- 172.21.15.43:0/1543341712 --> v2:172.21.15.43:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f576c06f7d0 con 0x7f576c0c8950 2023-08-09T13:51:49.326 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:49.323+0000 7f5771a36640 1 --2- 172.21.15.43:0/1543341712 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f576c0c8950 0x7f576c06b9e0 secure :-1 s=READY pgs=1284 cs=0 l=1 rev1=1 crypto rx=0x7f576c0cefb0 tx=0x7f576402bed0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=90e683b6d80b1813 server_cookie=0 in_seq=0 out_seq=0 2023-08-09T13:51:49.326 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:49.323+0000 7f5763fff640 1 -- 172.21.15.43:0/1543341712 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f576402ca70 con 0x7f576c0c8950 2023-08-09T13:51:49.327 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:49.323+0000 7f5763fff640 1 -- 172.21.15.43:0/1543341712 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f576402cbd0 con 0x7f576c0c8950 2023-08-09T13:51:49.327 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:49.323+0000 7f5763fff640 1 -- 172.21.15.43:0/1543341712 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f57640343f0 con 0x7f576c0c8950 2023-08-09T13:51:49.327 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:49.324+0000 7f5772237640 1 -- 172.21.15.43:0/1543341712 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f576c0c8950 msgr2=0x7f576c06b9e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:51:49.327 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:49.324+0000 7f5772237640 1 --2- 172.21.15.43:0/1543341712 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f576c0c8950 0x7f576c06b9e0 secure :-1 s=READY pgs=1284 cs=0 l=1 rev1=1 crypto rx=0x7f576c0cefb0 tx=0x7f576402bed0 comp rx=0 tx=0).stop 2023-08-09T13:51:49.327 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:49.324+0000 7f5772237640 1 -- 172.21.15.43:0/1543341712 shutdown_connections 2023-08-09T13:51:49.327 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:49.324+0000 7f5772237640 1 --2- 172.21.15.43:0/1543341712 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f576c0c8950 0x7f576c06b9e0 unknown :-1 s=CLOSED pgs=1284 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:51:49.327 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:49.324+0000 7f5772237640 1 --2- 172.21.15.43:0/1543341712 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f576c0c9d70 0x7f576c15b210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:51:49.328 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:49.324+0000 7f5772237640 1 --2- 172.21.15.43:0/1543341712 >> v2:172.21.15.203:3300/0 conn(0x7f576c0c8570 0x7f576c070ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:51:49.328 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:49.324+0000 7f5772237640 1 -- 172.21.15.43:0/1543341712 >> 172.21.15.43:0/1543341712 conn(0x7f576c0afaf0 msgr2=0x7f576c0afed0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-09T13:51:49.328 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:49.324+0000 7f5772237640 1 -- 172.21.15.43:0/1543341712 shutdown_connections 2023-08-09T13:51:49.328 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:49.324+0000 7f5772237640 1 -- 172.21.15.43:0/1543341712 wait complete. 2023-08-09T13:51:49.329 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:49.326+0000 7f5772237640 1 Processor -- start 2023-08-09T13:51:49.329 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:49.326+0000 7f5772237640 1 -- start start 2023-08-09T13:51:49.330 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:49.327+0000 7f5772237640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f576c0c8570 0x7f576c0b2760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:51:49.330 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:49.327+0000 7f5772237640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f576c0c8950 0x7f576c0cde70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:51:49.330 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:49.327+0000 7f5772237640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f576c0c9d70 0x7f576c0ce3b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:51:49.330 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:49.327+0000 7f5772237640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f576c0b2ca0 con 0x7f576c0c9d70 2023-08-09T13:51:49.330 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:49.327+0000 7f5772237640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f576c1f2340 con 0x7f576c0c8950 2023-08-09T13:51:49.330 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:49.327+0000 7f5772237640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f576c1f1750 con 0x7f576c0c8570 2023-08-09T13:51:49.331 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:49.328+0000 7f5771a36640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f576c0c9d70 0x7f576c0ce3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:51:49.331 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:49.328+0000 7f5771a36640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f576c0c9d70 0x7f576c0ce3b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:48706/0 (socket says 172.21.15.43:48706) 2023-08-09T13:51:49.331 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:49.328+0000 7f5771a36640 1 -- 172.21.15.43:0/2617169143 learned_addr learned my addr 172.21.15.43:0/2617169143 (peer_addr_for_me v2:172.21.15.43:0/0) 2023-08-09T13:51:49.331 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:49.328+0000 7f5770a34640 1 --2- 172.21.15.43:0/2617169143 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f576c0c8950 0x7f576c0cde70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:51:49.331 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:49.328+0000 7f5771235640 1 --2- 172.21.15.43:0/2617169143 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f576c0c8570 0x7f576c0b2760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:51:49.331 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:49.328+0000 7f5771a36640 1 -- 172.21.15.43:0/2617169143 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f576c0c8570 msgr2=0x7f576c0b2760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:51:49.331 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:49.328+0000 7f5771a36640 1 --2- 172.21.15.43:0/2617169143 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f576c0c8570 0x7f576c0b2760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:51:49.331 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:49.328+0000 7f5771a36640 1 -- 172.21.15.43:0/2617169143 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f576c0c8950 msgr2=0x7f576c0cde70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:51:49.332 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:49.328+0000 7f5771a36640 1 --2- 172.21.15.43:0/2617169143 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f576c0c8950 0x7f576c0cde70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:51:49.332 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:49.328+0000 7f5771a36640 1 -- 172.21.15.43:0/2617169143 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5764009b30 con 0x7f576c0c9d70 2023-08-09T13:51:49.332 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:49.328+0000 7f5770a34640 1 --2- 172.21.15.43:0/2617169143 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f576c0c8950 0x7f576c0cde70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-08-09T13:51:49.332 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:49.328+0000 7f5771235640 1 --2- 172.21.15.43:0/2617169143 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f576c0c8570 0x7f576c0b2760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-08-09T13:51:49.333 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:49.328+0000 7f5771a36640 1 --2- 172.21.15.43:0/2617169143 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f576c0c9d70 0x7f576c0ce3b0 secure :-1 s=READY pgs=1285 cs=0 l=1 rev1=1 crypto rx=0x7f5764036040 tx=0x7f5764004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-09T13:51:49.333 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:49.329+0000 7f5761ffb640 1 -- 172.21.15.43:0/2617169143 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5764009510 con 0x7f576c0c9d70 2023-08-09T13:51:49.333 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:49.329+0000 7f5761ffb640 1 -- 172.21.15.43:0/2617169143 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f576403e450 con 0x7f576c0c9d70 2023-08-09T13:51:49.333 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:49.329+0000 7f5761ffb640 1 -- 172.21.15.43:0/2617169143 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f576403e800 con 0x7f576c0c9d70 2023-08-09T13:51:49.333 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:49.329+0000 7f5772237640 1 -- 172.21.15.43:0/2617169143 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f576c1f1a10 con 0x7f576c0c9d70 2023-08-09T13:51:49.334 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:49.329+0000 7f5772237640 1 -- 172.21.15.43:0/2617169143 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f576c1f1fd0 con 0x7f576c0c9d70 2023-08-09T13:51:49.334 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:49.330+0000 7f5772237640 1 -- 172.21.15.43:0/2617169143 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f576c0739f0 con 0x7f576c0c9d70 2023-08-09T13:51:49.334 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:49.331+0000 7f5761ffb640 1 -- 172.21.15.43:0/2617169143 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 45) v1 ==== 88443+0+0 (secure 0 0 0) 0x7f576403e960 con 0x7f576c0c9d70 2023-08-09T13:51:49.334 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:49.332+0000 7f5761ffb640 1 --2- 172.21.15.43:0/2617169143 >> [v2:172.21.15.203:6832/1317108840,v1:172.21.15.203:6833/1317108840] conn(0x7f575006a170 0x7f575006c630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:51:49.335 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:49.332+0000 7f5761ffb640 1 -- 172.21.15.43:0/2617169143 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(108..108 src has 1..108) v4 ==== 6348+0+0 (secure 0 0 0) 0x7f57640ab5c0 con 0x7f576c0c9d70 2023-08-09T13:51:49.335 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:49.333+0000 7f5771235640 1 --2- 172.21.15.43:0/2617169143 >> [v2:172.21.15.203:6832/1317108840,v1:172.21.15.203:6833/1317108840] conn(0x7f575006a170 0x7f575006c630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:51:49.335 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:49.334+0000 7f5771235640 1 --2- 172.21.15.43:0/2617169143 >> [v2:172.21.15.203:6832/1317108840,v1:172.21.15.203:6833/1317108840] conn(0x7f575006a170 0x7f575006c630 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7f5768006fd0 tx=0x7f5768008040 comp rx=0 tx=0).ready entity=mgr.6619 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-09T13:51:49.337 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:49.336+0000 7f5761ffb640 1 -- 172.21.15.43:0/2617169143 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178066 (secure 0 0 0) 0x7f5764076b90 con 0x7f576c0c9d70 2023-08-09T13:51:49.608 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:49.607+0000 7f5772237640 1 -- 172.21.15.43:0/2617169143 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_snap_schedules.TestSnapSchedules.test_non_existent_snap_schedule_list"]} v 0) v1 -- 0x7f576c06ff30 con 0x7f576c0c9d70 2023-08-09T13:51:49.621 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:49.620+0000 7f5761ffb640 1 -- 172.21.15.43:0/2617169143 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_snap_schedules.TestSnapSchedules.test_non_existent_snap_schedule_list"]}]=0 v692) v1 ==== 166+0+0 (secure 0 0 0) 0x7f576407a1b0 con 0x7f576c0c9d70 2023-08-09T13:51:49.624 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:49.622+0000 7f5772237640 1 -- 172.21.15.43:0/2617169143 >> [v2:172.21.15.203:6832/1317108840,v1:172.21.15.203:6833/1317108840] conn(0x7f575006a170 msgr2=0x7f575006c630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:51:49.624 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:49.622+0000 7f5772237640 1 --2- 172.21.15.43:0/2617169143 >> [v2:172.21.15.203:6832/1317108840,v1:172.21.15.203:6833/1317108840] conn(0x7f575006a170 0x7f575006c630 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7f5768006fd0 tx=0x7f5768008040 comp rx=0 tx=0).stop 2023-08-09T13:51:49.624 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:49.622+0000 7f5772237640 1 -- 172.21.15.43:0/2617169143 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f576c0c9d70 msgr2=0x7f576c0ce3b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:51:49.624 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:49.622+0000 7f5772237640 1 --2- 172.21.15.43:0/2617169143 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f576c0c9d70 0x7f576c0ce3b0 secure :-1 s=READY pgs=1285 cs=0 l=1 rev1=1 crypto rx=0x7f5764036040 tx=0x7f5764004a40 comp rx=0 tx=0).stop 2023-08-09T13:51:49.625 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:49.623+0000 7f5772237640 1 -- 172.21.15.43:0/2617169143 shutdown_connections 2023-08-09T13:51:49.625 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:49.623+0000 7f5772237640 1 --2- 172.21.15.43:0/2617169143 >> [v2:172.21.15.203:6832/1317108840,v1:172.21.15.203:6833/1317108840] conn(0x7f575006a170 0x7f575006c630 unknown :-1 s=CLOSED pgs=39 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:51:49.626 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:49.623+0000 7f5772237640 1 --2- 172.21.15.43:0/2617169143 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f576c0c9d70 0x7f576c0ce3b0 unknown :-1 s=CLOSED pgs=1285 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:51:49.626 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:49.623+0000 7f5772237640 1 --2- 172.21.15.43:0/2617169143 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f576c0c8950 0x7f576c0cde70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:51:49.626 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:49.623+0000 7f5772237640 1 --2- 172.21.15.43:0/2617169143 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f576c0c8570 0x7f576c0b2760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:51:49.626 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:49.623+0000 7f5772237640 1 -- 172.21.15.43:0/2617169143 >> 172.21.15.43:0/2617169143 conn(0x7f576c0afaf0 msgr2=0x7f576c0c0e50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-09T13:51:49.626 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:49.623+0000 7f5772237640 1 -- 172.21.15.43:0/2617169143 shutdown_connections 2023-08-09T13:51:49.626 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:49.623+0000 7f5772237640 1 -- 172.21.15.43:0/2617169143 wait complete. 2023-08-09T13:51:49.644 INFO:tasks.cephfs_test_runner:Test listing snap schedules on a non-existing filesystem path failure ... ok 2023-08-09T13:51:49.645 INFO:tasks.cephfs_test_runner:Starting test: test_schedule_auto_deactivation_for_non_existent_path (tasks.cephfs.test_snap_schedules.TestSnapSchedules) 2023-08-09T13:51:49.645 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Starting test tasks.cephfs.test_snap_schedules.TestSnapSchedules.test_schedule_auto_deactivation_for_non_existent_path' 2023-08-09T13:51:49.878 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:49.877+0000 7fb5fb314640 1 Processor -- start 2023-08-09T13:51:49.879 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:49.877+0000 7fb5fb314640 1 -- start start 2023-08-09T13:51:49.880 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:49.879+0000 7fb5fb314640 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7fb5f41591b0 0x7fb5f40ccfd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:51:49.880 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:49.879+0000 7fb5fb314640 1 -- --> v1:172.21.15.43:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb5f40cc070 con 0x7fb5f4159590 2023-08-09T13:51:49.881 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:49.879+0000 7fb5fb314640 1 -- --> v1:172.21.15.203:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb5f40cd8b0 con 0x7fb5f40c8bb0 2023-08-09T13:51:49.881 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:49.879+0000 7fb5fb314640 1 -- --> v2:172.21.15.203:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb5f40ccc10 con 0x7fb5f41591b0 2023-08-09T13:51:49.881 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:49.879+0000 7fb5f9b11640 1 --1- >> v1:172.21.15.43:6789/0 conn(0x7fb5f4159590 0x7fb5f40cc500 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.43:6789/0 says I am v1:172.21.15.43:37676/0 (socket says 172.21.15.43:37676) 2023-08-09T13:51:49.881 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:49.879+0000 7fb5f9b11640 1 -- 172.21.15.43:0/3159029733 learned_addr learned my addr 172.21.15.43:0/3159029733 (peer_addr_for_me v1:172.21.15.43:0/0) 2023-08-09T13:51:49.883 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:49.880+0000 7fb5fa312640 1 --2- 172.21.15.43:0/3159029733 >> v2:172.21.15.203:3300/0 conn(0x7fb5f41591b0 0x7fb5f40ccfd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:51:49.883 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:49.880+0000 7fb5f9310640 1 -- 172.21.15.43:0/3159029733 <== mon.0 v1:172.21.15.43:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2188035028 0 0) 0x7fb5f40cc070 con 0x7fb5f4159590 2023-08-09T13:51:49.883 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:49.880+0000 7fb5f9310640 1 -- 172.21.15.43:0/3159029733 --> v1:172.21.15.43:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fb5dc005730 con 0x7fb5f4159590 2023-08-09T13:51:49.883 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:49.880+0000 7fb5f9310640 1 -- 172.21.15.43:0/3159029733 <== mon.0 v1:172.21.15.43:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2097945388 0 0) 0x7fb5dc005730 con 0x7fb5f4159590 2023-08-09T13:51:49.883 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:49.880+0000 7fb5f9310640 1 -- 172.21.15.43:0/3159029733 >> v2:172.21.15.203:3300/0 conn(0x7fb5f41591b0 msgr2=0x7fb5f40ccfd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-09T13:51:49.883 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:49.880+0000 7fb5f9310640 1 --2- 172.21.15.43:0/3159029733 >> v2:172.21.15.203:3300/0 conn(0x7fb5f41591b0 0x7fb5f40ccfd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:51:49.884 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:49.880+0000 7fb5f9310640 1 -- 172.21.15.43:0/3159029733 >> v1:172.21.15.203:6789/0 conn(0x7fb5f40c8bb0 legacy=0x7fb5f40c9340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:51:49.884 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:49.880+0000 7fb5f9310640 1 -- 172.21.15.43:0/3159029733 --> v1:172.21.15.43:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb5f40cdb90 con 0x7fb5f4159590 2023-08-09T13:51:49.884 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:49.880+0000 7fb5f9310640 1 -- 172.21.15.43:0/3159029733 <== mon.0 v1:172.21.15.43:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1285150740 0 0) 0x7fb5e8001670 con 0x7fb5f4159590 2023-08-09T13:51:49.884 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:49.880+0000 7fb5f9310640 1 -- 172.21.15.43:0/3159029733 >> v1:172.21.15.43:6789/0 conn(0x7fb5f4159590 legacy=0x7fb5f40cc500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:51:49.884 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:49.880+0000 7fb5f9310640 1 --2- 172.21.15.43:0/3159029733 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb5dc005b80 0x7fb5dc005fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:51:49.884 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:49.880+0000 7fb5f9310640 1 --2- 172.21.15.43:0/3159029733 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fb5dc0069d0 0x7fb5dc006e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:51:49.884 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:49.880+0000 7fb5f9310640 1 --2- 172.21.15.43:0/3159029733 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fb5dc007390 0x7fb5dc00db90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:51:49.885 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:49.880+0000 7fb5f9310640 1 -- 172.21.15.43:0/3159029733 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb5f40ccc10 con 0x7fb5dc005b80 2023-08-09T13:51:49.885 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:49.880+0000 7fb5f9310640 1 -- 172.21.15.43:0/3159029733 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb5dc00e330 con 0x7fb5dc0069d0 2023-08-09T13:51:49.885 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:49.880+0000 7fb5f9310640 1 -- 172.21.15.43:0/3159029733 --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb5dc00e4f0 con 0x7fb5dc007390 2023-08-09T13:51:49.885 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:49.881+0000 7fb5fa312640 1 --2- 172.21.15.43:0/3159029733 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb5dc005b80 0x7fb5dc005fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:51:49.885 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:49.881+0000 7fb5f9b11640 1 --2- 172.21.15.43:0/3159029733 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fb5dc0069d0 0x7fb5dc006e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:51:49.885 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:49.881+0000 7fb5fa312640 1 -- 172.21.15.43:0/3159029733 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fb5dc007390 msgr2=0x7fb5dc00db90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-09T13:51:49.885 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:49.881+0000 7fb5fa312640 1 --2- 172.21.15.43:0/3159029733 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fb5dc007390 0x7fb5dc00db90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:51:49.886 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:49.881+0000 7fb5fa312640 1 -- 172.21.15.43:0/3159029733 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fb5dc0069d0 msgr2=0x7fb5dc006e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-09T13:51:49.886 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:49.881+0000 7fb5fa312640 1 --2- 172.21.15.43:0/3159029733 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fb5dc0069d0 0x7fb5dc006e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:51:49.886 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:49.881+0000 7fb5fa312640 1 -- 172.21.15.43:0/3159029733 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb5dc005730 con 0x7fb5dc005b80 2023-08-09T13:51:49.886 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:49.881+0000 7fb5f9b11640 1 --2- 172.21.15.43:0/3159029733 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fb5dc0069d0 0x7fb5dc006e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-08-09T13:51:49.886 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:49.881+0000 7fb5fab13640 1 -- 172.21.15.43:0/3159029733 reap_dead start 2023-08-09T13:51:49.887 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:49.882+0000 7fb5fa312640 1 --2- 172.21.15.43:0/3159029733 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb5dc005b80 0x7fb5dc005fa0 secure :-1 s=READY pgs=1287 cs=0 l=1 rev1=1 crypto rx=0x7fb5e4009fc0 tx=0x7fb5dc007a00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f73cbfff0dd2cc73 server_cookie=0 in_seq=0 out_seq=0 2023-08-09T13:51:49.888 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:49.882+0000 7fb5f9310640 1 -- 172.21.15.43:0/3159029733 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb5e402cc20 con 0x7fb5dc005b80 2023-08-09T13:51:49.888 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:49.882+0000 7fb5f9310640 1 -- 172.21.15.43:0/3159029733 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb5e400f040 con 0x7fb5dc005b80 2023-08-09T13:51:49.888 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:49.882+0000 7fb5f9310640 1 -- 172.21.15.43:0/3159029733 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb5e4034440 con 0x7fb5dc005b80 2023-08-09T13:51:49.888 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:49.883+0000 7fb5fb314640 1 -- 172.21.15.43:0/3159029733 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb5dc005b80 msgr2=0x7fb5dc005fa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:51:49.888 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:49.883+0000 7fb5fb314640 1 --2- 172.21.15.43:0/3159029733 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb5dc005b80 0x7fb5dc005fa0 secure :-1 s=READY pgs=1287 cs=0 l=1 rev1=1 crypto rx=0x7fb5e4009fc0 tx=0x7fb5dc007a00 comp rx=0 tx=0).stop 2023-08-09T13:51:49.888 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:49.883+0000 7fb5fb314640 1 -- 172.21.15.43:0/3159029733 shutdown_connections 2023-08-09T13:51:49.889 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:49.883+0000 7fb5fb314640 1 --2- 172.21.15.43:0/3159029733 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb5dc005b80 0x7fb5dc005fa0 unknown :-1 s=CLOSED pgs=1287 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:51:49.889 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:49.883+0000 7fb5fb314640 1 -- 172.21.15.43:0/3159029733 >> 172.21.15.43:0/3159029733 conn(0x7fb5f40afaf0 msgr2=0x7fb5f40afed0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-09T13:51:49.889 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:49.883+0000 7fb5fb314640 1 -- 172.21.15.43:0/3159029733 shutdown_connections 2023-08-09T13:51:49.889 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:49.883+0000 7fb5fb314640 1 -- 172.21.15.43:0/3159029733 wait complete. 2023-08-09T13:51:49.889 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:49.885+0000 7fb5fb314640 1 Processor -- start 2023-08-09T13:51:49.889 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:49.885+0000 7fb5fb314640 1 -- start start 2023-08-09T13:51:49.889 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:49.886+0000 7fb5fb314640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fb5dc005b80 0x7fb5f41573e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:51:49.890 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:49.886+0000 7fb5fb314640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb5f40ab8d0 0x7fb5f40c76d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:51:49.890 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:49.886+0000 7fb5fb314640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fb5f40c6db0 0x7fb5f40c7c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:51:49.890 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:49.886+0000 7fb5fb314640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb5f4157920 con 0x7fb5f40ab8d0 2023-08-09T13:51:49.890 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:49.886+0000 7fb5fb314640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb5f40abe40 con 0x7fb5f40c6db0 2023-08-09T13:51:49.890 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:49.886+0000 7fb5fb314640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb5f40ac120 con 0x7fb5dc005b80 2023-08-09T13:51:49.891 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:49.887+0000 7fb5f9b11640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb5f40ab8d0 0x7fb5f40c76d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:51:49.892 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:49.887+0000 7fb5f9b11640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb5f40ab8d0 0x7fb5f40c76d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:48724/0 (socket says 172.21.15.43:48724) 2023-08-09T13:51:49.892 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:49.887+0000 7fb5f9b11640 1 -- 172.21.15.43:0/2014162578 learned_addr learned my addr 172.21.15.43:0/2014162578 (peer_addr_for_me v2:172.21.15.43:0/0) 2023-08-09T13:51:49.892 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:49.887+0000 7fb5fa312640 1 --2- 172.21.15.43:0/2014162578 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fb5dc005b80 0x7fb5f41573e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:51:49.892 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:49.887+0000 7fb5f9b11640 1 -- 172.21.15.43:0/2014162578 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fb5dc005b80 msgr2=0x7fb5f41573e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:51:49.892 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:49.887+0000 7fb5f9b11640 1 --2- 172.21.15.43:0/2014162578 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fb5dc005b80 0x7fb5f41573e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:51:49.892 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:49.887+0000 7fb5fab13640 1 --2- 172.21.15.43:0/2014162578 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fb5f40c6db0 0x7fb5f40c7c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:51:49.893 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:49.887+0000 7fb5f9b11640 1 -- 172.21.15.43:0/2014162578 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fb5f40c6db0 msgr2=0x7fb5f40c7c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:51:49.893 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:49.887+0000 7fb5f9b11640 1 --2- 172.21.15.43:0/2014162578 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fb5f40c6db0 0x7fb5f40c7c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:51:49.893 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:49.887+0000 7fb5f9b11640 1 -- 172.21.15.43:0/2014162578 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb5f40ac400 con 0x7fb5f40ab8d0 2023-08-09T13:51:49.893 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:49.887+0000 7fb5fab13640 1 --2- 172.21.15.43:0/2014162578 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fb5f40c6db0 0x7fb5f40c7c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-08-09T13:51:49.893 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:49.888+0000 7fb5f9b11640 1 --2- 172.21.15.43:0/2014162578 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb5f40ab8d0 0x7fb5f40c76d0 secure :-1 s=READY pgs=1288 cs=0 l=1 rev1=1 crypto rx=0x7fb5e8001200 tx=0x7fb5e800dc50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-09T13:51:49.893 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:49.888+0000 7fb5f2ffd640 1 -- 172.21.15.43:0/2014162578 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb5e800dea0 con 0x7fb5f40ab8d0 2023-08-09T13:51:49.893 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:49.888+0000 7fb5f2ffd640 1 -- 172.21.15.43:0/2014162578 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb5e80074e0 con 0x7fb5f40ab8d0 2023-08-09T13:51:49.894 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:49.888+0000 7fb5fb314640 1 -- 172.21.15.43:0/2014162578 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb5f40ac6e0 con 0x7fb5f40ab8d0 2023-08-09T13:51:49.894 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:49.888+0000 7fb5f2ffd640 1 -- 172.21.15.43:0/2014162578 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb5e8004a20 con 0x7fb5f40ab8d0 2023-08-09T13:51:49.894 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:49.888+0000 7fb5fb314640 1 -- 172.21.15.43:0/2014162578 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb5f40c6a60 con 0x7fb5f40ab8d0 2023-08-09T13:51:49.894 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:49.889+0000 7fb5fb314640 1 -- 172.21.15.43:0/2014162578 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb5f40739f0 con 0x7fb5f40ab8d0 2023-08-09T13:51:49.894 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:49.890+0000 7fb5f2ffd640 1 -- 172.21.15.43:0/2014162578 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 45) v1 ==== 88443+0+0 (secure 0 0 0) 0x7fb5e8004b80 con 0x7fb5f40ab8d0 2023-08-09T13:51:49.895 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:49.892+0000 7fb5f2ffd640 1 --2- 172.21.15.43:0/2014162578 >> [v2:172.21.15.203:6832/1317108840,v1:172.21.15.203:6833/1317108840] conn(0x7fb5e006a1a0 0x7fb5e006c660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:51:49.895 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:49.892+0000 7fb5f2ffd640 1 -- 172.21.15.43:0/2014162578 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(108..108 src has 1..108) v4 ==== 6348+0+0 (secure 0 0 0) 0x7fb5e8088350 con 0x7fb5f40ab8d0 2023-08-09T13:51:49.895 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:49.892+0000 7fb5fa312640 1 --2- 172.21.15.43:0/2014162578 >> [v2:172.21.15.203:6832/1317108840,v1:172.21.15.203:6833/1317108840] conn(0x7fb5e006a1a0 0x7fb5e006c660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:51:49.895 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:49.894+0000 7fb5fa312640 1 --2- 172.21.15.43:0/2014162578 >> [v2:172.21.15.203:6832/1317108840,v1:172.21.15.203:6833/1317108840] conn(0x7fb5e006a1a0 0x7fb5e006c660 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7fb5e4036000 tx=0x7fb5e402e000 comp rx=0 tx=0).ready entity=mgr.6619 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-09T13:51:49.897 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:49.896+0000 7fb5f2ffd640 1 -- 172.21.15.43:0/2014162578 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178066 (secure 0 0 0) 0x7fb5e8053660 con 0x7fb5f40ab8d0 2023-08-09T13:51:50.162 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:50.160+0000 7fb5fb314640 1 -- 172.21.15.43:0/2014162578 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_snap_schedules.TestSnapSchedules.test_schedule_auto_deactivation_for_non_existent_path"]} v 0) v1 -- 0x7fb5f40cb180 con 0x7fb5f40ab8d0 2023-08-09T13:51:50.634 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:50.632+0000 7fb5f2ffd640 1 -- 172.21.15.43:0/2014162578 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_snap_schedules.TestSnapSchedules.test_schedule_auto_deactivation_for_non_existent_path"]}]=0 v693) v1 ==== 186+0+0 (secure 0 0 0) 0x7fb5e8056c80 con 0x7fb5f40ab8d0 2023-08-09T13:51:50.637 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:50.635+0000 7fb5fb314640 1 -- 172.21.15.43:0/2014162578 >> [v2:172.21.15.203:6832/1317108840,v1:172.21.15.203:6833/1317108840] conn(0x7fb5e006a1a0 msgr2=0x7fb5e006c660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:51:50.637 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:50.635+0000 7fb5fb314640 1 --2- 172.21.15.43:0/2014162578 >> [v2:172.21.15.203:6832/1317108840,v1:172.21.15.203:6833/1317108840] conn(0x7fb5e006a1a0 0x7fb5e006c660 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7fb5e4036000 tx=0x7fb5e402e000 comp rx=0 tx=0).stop 2023-08-09T13:51:50.637 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:50.635+0000 7fb5fb314640 1 -- 172.21.15.43:0/2014162578 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb5f40ab8d0 msgr2=0x7fb5f40c76d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:51:50.637 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:50.635+0000 7fb5fb314640 1 --2- 172.21.15.43:0/2014162578 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb5f40ab8d0 0x7fb5f40c76d0 secure :-1 s=READY pgs=1288 cs=0 l=1 rev1=1 crypto rx=0x7fb5e8001200 tx=0x7fb5e800dc50 comp rx=0 tx=0).stop 2023-08-09T13:51:50.638 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:50.635+0000 7fb5fb314640 1 -- 172.21.15.43:0/2014162578 shutdown_connections 2023-08-09T13:51:50.638 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:50.635+0000 7fb5fb314640 1 --2- 172.21.15.43:0/2014162578 >> [v2:172.21.15.203:6832/1317108840,v1:172.21.15.203:6833/1317108840] conn(0x7fb5e006a1a0 0x7fb5e006c660 unknown :-1 s=CLOSED pgs=40 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:51:50.638 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:50.635+0000 7fb5fb314640 1 --2- 172.21.15.43:0/2014162578 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fb5f40c6db0 0x7fb5f40c7c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:51:50.638 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:50.635+0000 7fb5fb314640 1 --2- 172.21.15.43:0/2014162578 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb5f40ab8d0 0x7fb5f40c76d0 unknown :-1 s=CLOSED pgs=1288 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:51:50.638 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:50.635+0000 7fb5fb314640 1 --2- 172.21.15.43:0/2014162578 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fb5dc005b80 0x7fb5f41573e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:51:50.638 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:50.635+0000 7fb5fb314640 1 -- 172.21.15.43:0/2014162578 >> 172.21.15.43:0/2014162578 conn(0x7fb5f40afaf0 msgr2=0x7fb5f40c41b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-09T13:51:50.638 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:50.636+0000 7fb5fb314640 1 -- 172.21.15.43:0/2014162578 shutdown_connections 2023-08-09T13:51:50.638 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:50.636+0000 7fb5fb314640 1 -- 172.21.15.43:0/2014162578 wait complete. 2023-08-09T13:51:50.659 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-08-09T13:51:50.890 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:50.889+0000 7fa56a799640 1 Processor -- start 2023-08-09T13:51:50.891 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:50.889+0000 7fa56a799640 1 -- start start 2023-08-09T13:51:50.892 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:50.891+0000 7fa56a799640 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7fa5640cd510 0x7fa5640c96e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:51:50.893 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:50.891+0000 7fa56a799640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7fa56415ce40 0x7fa5640c3290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:51:50.893 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:50.891+0000 7fa56a799640 1 -- --> v1:172.21.15.43:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa564143080 con 0x7fa564142ca0 2023-08-09T13:51:50.893 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:50.891+0000 7fa56a799640 1 -- --> v2:172.21.15.43:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa56406d400 con 0x7fa56415ce40 2023-08-09T13:51:50.893 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:50.891+0000 7fa56a799640 1 -- --> v2:172.21.15.203:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa5640c9d90 con 0x7fa5640cd510 2023-08-09T13:51:50.893 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:50.891+0000 7fa563fff640 1 --1- >> v1:172.21.15.43:6789/0 conn(0x7fa564142ca0 0x7fa5640cce00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.43:6789/0 says I am v1:172.21.15.43:37692/0 (socket says 172.21.15.43:37692) 2023-08-09T13:51:50.893 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:50.891+0000 7fa563fff640 1 -- 172.21.15.43:0/531296245 learned_addr learned my addr 172.21.15.43:0/531296245 (peer_addr_for_me v1:172.21.15.43:0/0) 2023-08-09T13:51:50.893 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:50.891+0000 7fa568d36640 1 --2- 172.21.15.43:0/531296245 >> v2:172.21.15.43:3300/0 conn(0x7fa56415ce40 0x7fa5640c3290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:51:50.894 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:50.892+0000 7fa55b7fe640 1 --2- 172.21.15.43:0/531296245 >> v2:172.21.15.203:3300/0 conn(0x7fa5640cd510 0x7fa5640c96e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:51:50.894 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:50.892+0000 7fa568d36640 1 -- 172.21.15.43:0/531296245 >> v2:172.21.15.203:3300/0 conn(0x7fa5640cd510 msgr2=0x7fa5640c96e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-09T13:51:50.895 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:50.892+0000 7fa568d36640 1 --2- 172.21.15.43:0/531296245 >> v2:172.21.15.203:3300/0 conn(0x7fa5640cd510 0x7fa5640c96e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:51:50.895 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:50.892+0000 7fa568d36640 1 -- 172.21.15.43:0/531296245 >> v1:172.21.15.43:6789/0 conn(0x7fa564142ca0 legacy=0x7fa5640cce00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:51:50.895 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:50.892+0000 7fa5637fe640 1 -- 172.21.15.43:0/531296245 <== mon.0 v1:172.21.15.43:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 206296098 0 0) 0x7fa564143080 con 0x7fa564142ca0 2023-08-09T13:51:50.895 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:50.892+0000 7fa568d36640 1 -- 172.21.15.43:0/531296245 --> v2:172.21.15.43:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa5640ca070 con 0x7fa56415ce40 2023-08-09T13:51:50.895 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:50.893+0000 7fa568d36640 1 --2- 172.21.15.43:0/531296245 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa56415ce40 0x7fa5640c3290 secure :-1 s=READY pgs=1290 cs=0 l=1 rev1=1 crypto rx=0x7fa56406e080 tx=0x7fa55c02b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4bbbb994743bf6b5 server_cookie=0 in_seq=0 out_seq=0 2023-08-09T13:51:50.895 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:50.893+0000 7fa5637fe640 1 -- 172.21.15.43:0/531296245 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa55c031830 con 0x7fa56415ce40 2023-08-09T13:51:50.896 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:50.893+0000 7fa5637fe640 1 -- 172.21.15.43:0/531296245 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa55c031990 con 0x7fa56415ce40 2023-08-09T13:51:50.896 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:50.893+0000 7fa5637fe640 1 -- 172.21.15.43:0/531296245 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa55c031d20 con 0x7fa56415ce40 2023-08-09T13:51:50.896 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:50.894+0000 7fa56a799640 1 -- 172.21.15.43:0/531296245 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa56415ce40 msgr2=0x7fa5640c3290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:51:50.897 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:50.894+0000 7fa56a799640 1 --2- 172.21.15.43:0/531296245 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa56415ce40 0x7fa5640c3290 secure :-1 s=READY pgs=1290 cs=0 l=1 rev1=1 crypto rx=0x7fa56406e080 tx=0x7fa55c02b8d0 comp rx=0 tx=0).stop 2023-08-09T13:51:50.897 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:50.894+0000 7fa56a799640 1 -- 172.21.15.43:0/531296245 shutdown_connections 2023-08-09T13:51:50.897 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:50.894+0000 7fa56a799640 1 --2- 172.21.15.43:0/531296245 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa56415ce40 0x7fa5640c3290 unknown :-1 s=CLOSED pgs=1290 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:51:50.897 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:50.894+0000 7fa56a799640 1 --2- 172.21.15.43:0/531296245 >> v2:172.21.15.203:3300/0 conn(0x7fa5640cd510 0x7fa5640c96e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:51:50.897 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:50.894+0000 7fa56a799640 1 -- 172.21.15.43:0/531296245 >> 172.21.15.43:0/531296245 conn(0x7fa5640ce0a0 msgr2=0x7fa5640ce480 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-09T13:51:50.897 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:50.894+0000 7fa56a799640 1 -- 172.21.15.43:0/531296245 shutdown_connections 2023-08-09T13:51:50.898 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:50.894+0000 7fa56a799640 1 -- 172.21.15.43:0/531296245 wait complete. 2023-08-09T13:51:50.898 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:50.896+0000 7fa56a799640 1 Processor -- start 2023-08-09T13:51:50.899 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:50.896+0000 7fa56a799640 1 -- start start 2023-08-09T13:51:50.899 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:50.897+0000 7fa56a799640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fa5640cd510 0x7fa56413bac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:51:50.899 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:50.897+0000 7fa56a799640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fa564142ca0 0x7fa5640c53e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:51:50.899 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:50.897+0000 7fa56a799640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa56415ce40 0x7fa5640c5920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:51:50.899 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:50.897+0000 7fa56a799640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa56413b630 con 0x7fa56415ce40 2023-08-09T13:51:50.899 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:50.897+0000 7fa56a799640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa5640c5e60 con 0x7fa564142ca0 2023-08-09T13:51:50.900 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:50.897+0000 7fa56a799640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa5640c60c0 con 0x7fa5640cd510 2023-08-09T13:51:50.900 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:50.897+0000 7fa568d36640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa56415ce40 0x7fa5640c5920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:51:50.900 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:50.897+0000 7fa568d36640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa56415ce40 0x7fa5640c5920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:48746/0 (socket says 172.21.15.43:48746) 2023-08-09T13:51:50.900 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:50.897+0000 7fa568d36640 1 -- 172.21.15.43:0/1778509655 learned_addr learned my addr 172.21.15.43:0/1778509655 (peer_addr_for_me v2:172.21.15.43:0/0) 2023-08-09T13:51:50.901 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:50.898+0000 7fa55b7fe640 1 --2- 172.21.15.43:0/1778509655 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fa564142ca0 0x7fa5640c53e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:51:50.901 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:50.898+0000 7fa563fff640 1 --2- 172.21.15.43:0/1778509655 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fa5640cd510 0x7fa56413bac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:51:50.901 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:50.898+0000 7fa568d36640 1 -- 172.21.15.43:0/1778509655 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fa5640cd510 msgr2=0x7fa56413bac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:51:50.901 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:50.898+0000 7fa568d36640 1 --2- 172.21.15.43:0/1778509655 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fa5640cd510 0x7fa56413bac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:51:50.902 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:50.898+0000 7fa568d36640 1 -- 172.21.15.43:0/1778509655 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fa564142ca0 msgr2=0x7fa5640c53e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:51:50.902 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:50.898+0000 7fa568d36640 1 --2- 172.21.15.43:0/1778509655 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fa564142ca0 0x7fa5640c53e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:51:50.902 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:50.898+0000 7fa568d36640 1 -- 172.21.15.43:0/1778509655 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa55c009400 con 0x7fa56415ce40 2023-08-09T13:51:50.902 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:50.898+0000 7fa568d36640 1 --2- 172.21.15.43:0/1778509655 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa56415ce40 0x7fa5640c5920 secure :-1 s=READY pgs=1291 cs=0 l=1 rev1=1 crypto rx=0x7fa55c034040 tx=0x7fa55c0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-09T13:51:50.902 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:50.898+0000 7fa5617fa640 1 -- 172.21.15.43:0/1778509655 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa55c0317b0 con 0x7fa56415ce40 2023-08-09T13:51:50.902 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:50.899+0000 7fa5617fa640 1 -- 172.21.15.43:0/1778509655 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa55c03a9e0 con 0x7fa56415ce40 2023-08-09T13:51:50.902 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:50.899+0000 7fa5617fa640 1 -- 172.21.15.43:0/1778509655 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa55c03ad90 con 0x7fa56415ce40 2023-08-09T13:51:50.903 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:50.899+0000 7fa56a799640 1 -- 172.21.15.43:0/1778509655 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa5640c63a0 con 0x7fa56415ce40 2023-08-09T13:51:50.903 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:50.899+0000 7fa56a799640 1 -- 172.21.15.43:0/1778509655 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa56413b2c0 con 0x7fa56415ce40 2023-08-09T13:51:50.903 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:50.900+0000 7fa56a799640 1 -- 172.21.15.43:0/1778509655 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa56413b7e0 con 0x7fa56415ce40 2023-08-09T13:51:50.904 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:50.901+0000 7fa5617fa640 1 -- 172.21.15.43:0/1778509655 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 45) v1 ==== 88443+0+0 (secure 0 0 0) 0x7fa55c04d450 con 0x7fa56415ce40 2023-08-09T13:51:50.904 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:50.902+0000 7fa5617fa640 1 --2- 172.21.15.43:0/1778509655 >> [v2:172.21.15.203:6832/1317108840,v1:172.21.15.203:6833/1317108840] conn(0x7fa54406a1a0 0x7fa54406c660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:51:50.904 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:50.902+0000 7fa5617fa640 1 -- 172.21.15.43:0/1778509655 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(108..108 src has 1..108) v4 ==== 6348+0+0 (secure 0 0 0) 0x7fa55c0aa9e0 con 0x7fa56415ce40 2023-08-09T13:51:50.904 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:50.903+0000 7fa563fff640 1 --2- 172.21.15.43:0/1778509655 >> [v2:172.21.15.203:6832/1317108840,v1:172.21.15.203:6833/1317108840] conn(0x7fa54406a1a0 0x7fa54406c660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:51:50.905 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:50.904+0000 7fa563fff640 1 --2- 172.21.15.43:0/1778509655 >> [v2:172.21.15.203:6832/1317108840,v1:172.21.15.203:6833/1317108840] conn(0x7fa54406a1a0 0x7fa54406c660 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7fa5540012e0 tx=0x7fa554007ed0 comp rx=0 tx=0).ready entity=mgr.6619 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-09T13:51:50.908 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:50.906+0000 7fa5617fa640 1 -- 172.21.15.43:0/1778509655 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178066 (secure 0 0 0) 0x7fa56413b7e0 con 0x7fa56415ce40 2023-08-09T13:51:51.102 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:51.100+0000 7fa56a799640 1 -- 172.21.15.43:0/1778509655 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fa56413b7e0 con 0x7fa56415ce40 2023-08-09T13:51:51.104 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:51.102+0000 7fa5617fa640 1 -- 172.21.15.43:0/1778509655 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v108) v1 ==== 74+0+15999 (secure 0 0 0) 0x7fa56413b7e0 con 0x7fa56415ce40 2023-08-09T13:51:51.104 INFO:teuthology.orchestra.run.smithi043.stdout: 2023-08-09T13:51:51.105 INFO:teuthology.orchestra.run.smithi043.stdout:{"epoch":108,"fsid":"daac87ee-1337-4250-a955-aea4e408a39b","created":"2023-08-09T13:33:41.701198+0000","modified":"2023-08-09T13:51:47.600280+0000","last_up_change":"2023-08-09T13:33:53.783453+0000","last_in_change":"2023-08-09T13:33:45.148904+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":13,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":"rbd","create_time":"2023-08-09T13:33:54.365137+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":".mgr","create_time":"2023-08-09T13:33:54.831803+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"cbd49ecd-e896-4fbd-864e-5c37a829b165","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":101,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6800","nonce":502667335},{"type":"v1","addr":"172.21.15.43:6801","nonce":502667335}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6802","nonce":502667335},{"type":"v1","addr":"172.21.15.43:6803","nonce":502667335}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6806","nonce":502667335},{"type":"v1","addr":"172.21.15.43:6807","nonce":502667335}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6804","nonce":502667335},{"type":"v1","addr":"172.21.15.43:6805","nonce":502667335}]},"public_addr":"172.21.15.43:6801/502667335","cluster_addr":"172.21.15.43:6803/502667335","heartbeat_back_addr":"172.21.15.43:6807/502667335","heartbeat_front_addr":"172.21.15.43:6805/502667335","state":["exists","up"]},{"osd":1,"uuid":"75c7e0fe-1ec7-4fe8-a2f2-3661a65b8500","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":101,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6808","nonce":3289134610},{"type":"v1","addr":"172.21.15.43:6809","nonce":3289134610}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6810","nonce":3289134610},{"type":"v1","addr":"172.21.15.43:6812","nonce":3289134610}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6821","nonce":3289134610},{"type":"v1","addr":"172.21.15.43:6824","nonce":3289134610}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6815","nonce":3289134610},{"type":"v1","addr":"172.21.15.43:6818","nonce":3289134610}]},"public_addr":"172.21.15.43:6809/3289134610","cluster_addr":"172.21.15.43:6812/3289134610","heartbeat_back_addr":"172.21.15.43:6824/3289134610","heartbeat_front_addr":"172.21.15.43:6818/3289134610","state":["exists","up"]},{"osd":2,"uuid":"bb5c41e9-7f5f-4125-9b01-3bb8bfc2ea83","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":101,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6811","nonce":4079709736},{"type":"v1","addr":"172.21.15.43:6814","nonce":4079709736}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6817","nonce":4079709736},{"type":"v1","addr":"172.21.15.43:6820","nonce":4079709736}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6829","nonce":4079709736},{"type":"v1","addr":"172.21.15.43:6831","nonce":4079709736}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6823","nonce":4079709736},{"type":"v1","addr":"172.21.15.43:6826","nonce":4079709736}]},"public_addr":"172.21.15.43:6814/4079709736","cluster_addr":"172.21.15.43:6820/4079709736","heartbeat_back_addr":"172.21.15.43:6831/4079709736","heartbeat_front_addr":"172.21.15.43:6826/4079709736","state":["exists","up"]},{"osd":3,"uuid":"bfe4c8b4-694a-465c-87e6-2b2aab852613","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":101,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6813","nonce":1709538759},{"type":"v1","addr":"172.21.15.43:6816","nonce":1709538759}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6819","nonce":1709538759},{"type":"v1","addr":"172.21.15.43:6822","nonce":1709538759}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6828","nonce":1709538759},{"type":"v1","addr":"172.21.15.43:6830","nonce":1709538759}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6825","nonce":1709538759},{"type":"v1","addr":"172.21.15.43:6827","nonce":1709538759}]},"public_addr":"172.21.15.43:6816/1709538759","cluster_addr":"172.21.15.43:6822/1709538759","heartbeat_back_addr":"172.21.15.43:6830/1709538759","heartbeat_front_addr":"172.21.15.43:6827/1709538759","state":["exists","up"]},{"osd":4,"uuid":"fa2e8a77-aba1-4756-853e-0d174a5bdeba","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":101,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6800","nonce":4196866481},{"type":"v1","addr":"172.21.15.203:6801","nonce":4196866481}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6802","nonce":4196866481},{"type":"v1","addr":"172.21.15.203:6803","nonce":4196866481}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6806","nonce":4196866481},{"type":"v1","addr":"172.21.15.203:6807","nonce":4196866481}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6804","nonce":4196866481},{"type":"v1","addr":"172.21.15.203:6805","nonce":4196866481}]},"public_addr":"172.21.15.203:6801/4196866481","cluster_addr":"172.21.15.203:6803/4196866481","heartbeat_back_addr":"172.21.15.203:6807/4196866481","heartbeat_front_addr":"172.21.15.203:6805/4196866481","state":["exists","up"]},{"osd":5,"uuid":"81811e4b-4d24-4f51-a82c-f1d700a2a00d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":101,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6816","nonce":3059290081},{"type":"v1","addr":"172.21.15.203:6817","nonce":3059290081}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6818","nonce":3059290081},{"type":"v1","addr":"172.21.15.203:6819","nonce":3059290081}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6822","nonce":3059290081},{"type":"v1","addr":"172.21.15.203:6823","nonce":3059290081}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6820","nonce":3059290081},{"type":"v1","addr":"172.21.15.203:6821","nonce":3059290081}]},"public_addr":"172.21.15.203:6817/3059290081","cluster_addr":"172.21.15.203:6819/3059290081","heartbeat_back_addr":"172.21.15.203:6823/3059290081","heartbeat_front_addr":"172.21.15.203:6821/3059290081","state":["exists","up"]},{"osd":6,"uuid":"307bac4d-6a5b-443d-af75-56d695d7674f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":101,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6824","nonce":512155569},{"type":"v1","addr":"172.21.15.203:6825","nonce":512155569}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6826","nonce":512155569},{"type":"v1","addr":"172.21.15.203:6827","nonce":512155569}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6830","nonce":512155569},{"type":"v1","addr":"172.21.15.203:6831","nonce":512155569}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6828","nonce":512155569},{"type":"v1","addr":"172.21.15.203:6829","nonce":512155569}]},"public_addr":"172.21.15.203:6825/512155569","cluster_addr":"172.21.15.203:6827/512155569","heartbeat_back_addr":"172.21.15.203:6831/512155569","heartbeat_front_addr":"172.21.15.203:6829/512155569","state":["exists","up"]},{"osd":7,"uuid":"a95dc7ca-9f1b-4295-af70-68a22a2d7d5e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":101,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6808","nonce":3331450438},{"type":"v1","addr":"172.21.15.203:6809","nonce":3331450438}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6810","nonce":3331450438},{"type":"v1","addr":"172.21.15.203:6811","nonce":3331450438}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6814","nonce":3331450438},{"type":"v1","addr":"172.21.15.203:6815","nonce":3331450438}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6812","nonce":3331450438},{"type":"v1","addr":"172.21.15.203:6813","nonce":3331450438}]},"public_addr":"172.21.15.203:6809/3331450438","cluster_addr":"172.21.15.203:6811/3331450438","heartbeat_back_addr":"172.21.15.203:6815/3331450438","heartbeat_front_addr":"172.21.15.203:6813/3331450438","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-09T13:33:50.547652+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-09T13:33:50.589469+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-09T13:33:50.561724+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-09T13:33:50.599650+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-09T13:33:50.411855+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-09T13:33:50.219681+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-09T13:33:50.504664+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-09T13:33:50.441248+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.43:6834/3008795294":"2023-08-10T13:51:44.053276+0000","172.21.15.43:6835/3008795294":"2023-08-10T13:51:44.053276+0000","172.21.15.203:0/996706743":"2023-08-10T13:51:38.530957+0000","172.21.15.203:0/969846951":"2023-08-10T13:51:38.530957+0000","172.21.15.203:6833/2675832148":"2023-08-10T13:51:38.530957+0000","172.21.15.203:6832/2675832148":"2023-08-10T13:51:38.530957+0000","172.21.15.203:0/822053193":"2023-08-10T13:51:38.530957+0000","172.21.15.203:0/1475395073":"2023-08-10T13:51:38.530957+0000","172.21.15.203:0/1343041460":"2023-08-10T13:51:32.420350+0000","172.21.15.203:6833/1778614695":"2023-08-10T13:51:32.420350+0000","172.21.15.203:0/3761332698":"2023-08-10T13:51:32.420350+0000","172.21.15.203:6832/1778614695":"2023-08-10T13:51:32.420350+0000","172.21.15.203:0/2293720318":"2023-08-10T13:51:38.530957+0000","172.21.15.203:0/2054133158":"2023-08-10T13:51:32.420350+0000","172.21.15.203:0/3225794224":"2023-08-10T13:51:38.530957+0000","172.21.15.203:0/1496080019":"2023-08-10T13:51:32.420350+0000","172.21.15.203:0/1894709586":"2023-08-10T13:51:32.420350+0000","172.21.15.203:0/270998287":"2023-08-10T13:51:32.420350+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":7,"snaps":[{"begin":104,"length":1},{"begin":221,"length":1}]}],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-08-09T13:51:51.107 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:51.105+0000 7fa56a799640 1 -- 172.21.15.43:0/1778509655 >> [v2:172.21.15.203:6832/1317108840,v1:172.21.15.203:6833/1317108840] conn(0x7fa54406a1a0 msgr2=0x7fa54406c660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:51:51.107 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:51.105+0000 7fa56a799640 1 --2- 172.21.15.43:0/1778509655 >> [v2:172.21.15.203:6832/1317108840,v1:172.21.15.203:6833/1317108840] conn(0x7fa54406a1a0 0x7fa54406c660 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7fa5540012e0 tx=0x7fa554007ed0 comp rx=0 tx=0).stop 2023-08-09T13:51:51.107 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:51.105+0000 7fa56a799640 1 -- 172.21.15.43:0/1778509655 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa56415ce40 msgr2=0x7fa5640c5920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:51:51.107 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:51.105+0000 7fa56a799640 1 --2- 172.21.15.43:0/1778509655 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa56415ce40 0x7fa5640c5920 secure :-1 s=READY pgs=1291 cs=0 l=1 rev1=1 crypto rx=0x7fa55c034040 tx=0x7fa55c0046c0 comp rx=0 tx=0).stop 2023-08-09T13:51:51.107 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:51.105+0000 7fa56a799640 1 -- 172.21.15.43:0/1778509655 shutdown_connections 2023-08-09T13:51:51.108 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:51.105+0000 7fa56a799640 1 --2- 172.21.15.43:0/1778509655 >> [v2:172.21.15.203:6832/1317108840,v1:172.21.15.203:6833/1317108840] conn(0x7fa54406a1a0 0x7fa54406c660 unknown :-1 s=CLOSED pgs=41 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:51:51.108 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:51.105+0000 7fa56a799640 1 --2- 172.21.15.43:0/1778509655 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa56415ce40 0x7fa5640c5920 unknown :-1 s=CLOSED pgs=1291 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:51:51.108 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:51.105+0000 7fa56a799640 1 --2- 172.21.15.43:0/1778509655 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fa564142ca0 0x7fa5640c53e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:51:51.108 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:51.105+0000 7fa56a799640 1 --2- 172.21.15.43:0/1778509655 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fa5640cd510 0x7fa56413bac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:51:51.108 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:51.105+0000 7fa56a799640 1 -- 172.21.15.43:0/1778509655 >> 172.21.15.43:0/1778509655 conn(0x7fa5640ce0a0 msgr2=0x7fa5640c8710 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-09T13:51:51.109 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:51.106+0000 7fa56a799640 1 -- 172.21.15.43:0/1778509655 shutdown_connections 2023-08-09T13:51:51.109 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:51.106+0000 7fa56a799640 1 -- 172.21.15.43:0/1778509655 wait complete. 2023-08-09T13:51:51.127 INFO:tasks.cephfs.cephfs_test_case.ceph_manager:[{'pool': 1, 'pool_name': 'rbd', 'create_time': '2023-08-09T13:33:54.365137+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}, 'read_balance': {'score_acting': 2, 'score_stable': 2, 'optimal_score': 1, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': '.mgr', 'create_time': '2023-08-09T13:33:54.831803+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-08-09T13:51:51.128 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get rbd pg_num 2023-08-09T13:51:51.358 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:51.357+0000 7f84f7221640 1 Processor -- start 2023-08-09T13:51:51.359 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:51.357+0000 7f84f7221640 1 -- start start 2023-08-09T13:51:51.360 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:51.359+0000 7f84f7221640 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7f84f00cb9a0 0x7f84f00cbde0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:51:51.360 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:51.359+0000 7f84f7221640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f84f006f310 0x7f84f006f7a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:51:51.361 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:51.359+0000 7f84f7221640 1 -- --> v1:172.21.15.43:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f84f00acfa0 con 0x7f84f0070ef0 2023-08-09T13:51:51.361 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:51.359+0000 7f84f7221640 1 -- --> v2:172.21.15.203:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f84f00cc320 con 0x7f84f00cb9a0 2023-08-09T13:51:51.361 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:51.359+0000 7f84f7221640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f84f015c940 con 0x7f84f006f310 2023-08-09T13:51:51.362 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:51.359+0000 7f84f621f640 1 --1- >> v1:172.21.15.43:6789/0 conn(0x7f84f0070ef0 0x7f84f00ab8d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.43:6789/0 says I am v1:172.21.15.43:37698/0 (socket says 172.21.15.43:37698) 2023-08-09T13:51:51.362 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:51.359+0000 7f84f621f640 1 -- 172.21.15.43:0/2253976356 learned_addr learned my addr 172.21.15.43:0/2253976356 (peer_addr_for_me v1:172.21.15.43:0/0) 2023-08-09T13:51:51.363 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:51.360+0000 7f84f6a20640 1 --2- 172.21.15.43:0/2253976356 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f84f006f310 0x7f84f006f7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:51:51.363 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:51.360+0000 7f84f5a1e640 1 --2- 172.21.15.43:0/2253976356 >> v2:172.21.15.203:3300/0 conn(0x7f84f00cb9a0 0x7f84f00cbde0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:51:51.363 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:51.360+0000 7f84f521d640 1 -- 172.21.15.43:0/2253976356 <== mon.0 v1:172.21.15.43:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1873452567 0 0) 0x7f84f00acfa0 con 0x7f84f0070ef0 2023-08-09T13:51:51.363 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:51.360+0000 7f84f521d640 1 -- 172.21.15.43:0/2253976356 --> v1:172.21.15.43:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f84dc003520 con 0x7f84f0070ef0 2023-08-09T13:51:51.363 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:51.360+0000 7f84f521d640 1 -- 172.21.15.43:0/2253976356 <== mon.0 v1:172.21.15.43:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 735187397 0 0) 0x7f84dc003520 con 0x7f84f0070ef0 2023-08-09T13:51:51.363 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:51.360+0000 7f84f521d640 1 -- 172.21.15.43:0/2253976356 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f84f006f310 msgr2=0x7f84f006f7a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-09T13:51:51.363 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:51.360+0000 7f84f521d640 1 --2- 172.21.15.43:0/2253976356 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f84f006f310 0x7f84f006f7a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:51:51.363 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:51.360+0000 7f84f521d640 1 -- 172.21.15.43:0/2253976356 >> v2:172.21.15.203:3300/0 conn(0x7f84f00cb9a0 msgr2=0x7f84f00cbde0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-09T13:51:51.364 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:51.360+0000 7f84f521d640 1 --2- 172.21.15.43:0/2253976356 >> v2:172.21.15.203:3300/0 conn(0x7f84f00cb9a0 0x7f84f00cbde0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:51:51.364 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:51.360+0000 7f84f521d640 1 -- 172.21.15.43:0/2253976356 --> v1:172.21.15.43:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f84f00ac390 con 0x7f84f0070ef0 2023-08-09T13:51:51.364 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:51.360+0000 7f84f521d640 1 -- 172.21.15.43:0/2253976356 <== mon.0 v1:172.21.15.43:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1285150740 0 0) 0x7f84ec001670 con 0x7f84f0070ef0 2023-08-09T13:51:51.364 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:51.360+0000 7f84f521d640 1 -- 172.21.15.43:0/2253976356 >> v1:172.21.15.43:6789/0 conn(0x7f84f0070ef0 legacy=0x7f84f00ab8d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:51:51.364 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:51.360+0000 7f84f521d640 1 --2- 172.21.15.43:0/2253976356 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f84dc003970 0x7f84dc003d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:51:51.364 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:51.360+0000 7f84f521d640 1 --2- 172.21.15.43:0/2253976356 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f84f006f310 0x7f84dc0047c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:51:51.364 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:51.360+0000 7f84f521d640 1 --2- 172.21.15.43:0/2253976356 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f84dc004d00 0x7f84dc005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:51:51.365 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:51.360+0000 7f84f521d640 1 -- 172.21.15.43:0/2253976356 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f84f00cc320 con 0x7f84dc004d00 2023-08-09T13:51:51.365 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:51.360+0000 7f84f521d640 1 -- 172.21.15.43:0/2253976356 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f84f015c940 con 0x7f84dc003970 2023-08-09T13:51:51.365 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:51.360+0000 7f84f521d640 1 -- 172.21.15.43:0/2253976356 --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f84dc00b980 con 0x7f84f006f310 2023-08-09T13:51:51.365 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:51.361+0000 7f84f6a20640 1 --2- 172.21.15.43:0/2253976356 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f84dc004d00 0x7f84dc005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:51:51.365 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:51.361+0000 7f84f621f640 1 --2- 172.21.15.43:0/2253976356 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f84dc003970 0x7f84dc003d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:51:51.365 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:51.361+0000 7f84f5a1e640 1 --2- 172.21.15.43:0/2253976356 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f84f006f310 0x7f84dc0047c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:51:51.366 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:51.361+0000 7f84f6a20640 1 -- 172.21.15.43:0/2253976356 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f84f006f310 msgr2=0x7f84dc0047c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-09T13:51:51.367 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:51.361+0000 7f84f6a20640 1 --2- 172.21.15.43:0/2253976356 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f84f006f310 0x7f84dc0047c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:51:51.367 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:51.361+0000 7f84f6a20640 1 -- 172.21.15.43:0/2253976356 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f84dc003970 msgr2=0x7f84dc003d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-09T13:51:51.367 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:51.361+0000 7f84f6a20640 1 --2- 172.21.15.43:0/2253976356 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f84dc003970 0x7f84dc003d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:51:51.367 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:51.361+0000 7f84f6a20640 1 -- 172.21.15.43:0/2253976356 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f84dc003520 con 0x7f84dc004d00 2023-08-09T13:51:51.367 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:51.362+0000 7f84f6a20640 1 --2- 172.21.15.43:0/2253976356 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f84dc004d00 0x7f84dc005180 secure :-1 s=READY pgs=1293 cs=0 l=1 rev1=1 crypto rx=0x7f84e8009fc0 tx=0x7f84dc0057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3f3e25bc954763ce server_cookie=0 in_seq=0 out_seq=0 2023-08-09T13:51:51.367 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:51.363+0000 7f84f521d640 1 -- 172.21.15.43:0/2253976356 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f84e802cbf0 con 0x7f84dc004d00 2023-08-09T13:51:51.367 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:51.363+0000 7f84f521d640 1 -- 172.21.15.43:0/2253976356 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f84e800f040 con 0x7f84dc004d00 2023-08-09T13:51:51.367 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:51.363+0000 7f84f521d640 1 -- 172.21.15.43:0/2253976356 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f84e8034440 con 0x7f84dc004d00 2023-08-09T13:51:51.368 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:51.363+0000 7f84f7221640 1 -- 172.21.15.43:0/2253976356 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f84dc004d00 msgr2=0x7f84dc005180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:51:51.368 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:51.363+0000 7f84f7221640 1 --2- 172.21.15.43:0/2253976356 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f84dc004d00 0x7f84dc005180 secure :-1 s=READY pgs=1293 cs=0 l=1 rev1=1 crypto rx=0x7f84e8009fc0 tx=0x7f84dc0057f0 comp rx=0 tx=0).stop 2023-08-09T13:51:51.368 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:51.363+0000 7f84f6a20640 1 -- 172.21.15.43:0/2253976356 reap_dead start 2023-08-09T13:51:51.368 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:51.363+0000 7f84f7221640 1 -- 172.21.15.43:0/2253976356 shutdown_connections 2023-08-09T13:51:51.368 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:51.363+0000 7f84f7221640 1 -- 172.21.15.43:0/2253976356 >> 172.21.15.43:0/2253976356 conn(0x7f84f00cddd0 msgr2=0x7f84f00ce1e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-09T13:51:51.368 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:51.363+0000 7f84f7221640 1 -- 172.21.15.43:0/2253976356 shutdown_connections 2023-08-09T13:51:51.368 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:51.363+0000 7f84f7221640 1 -- 172.21.15.43:0/2253976356 wait complete. 2023-08-09T13:51:51.369 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:51.365+0000 7f84f7221640 1 Processor -- start 2023-08-09T13:51:51.370 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:51.365+0000 7f84f7221640 1 -- start start 2023-08-09T13:51:51.370 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:51.366+0000 7f84f7221640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f84f00ca990 0x7f84f015e3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:51:51.370 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:51.366+0000 7f84f7221640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f84f015d860 0x7f84f006c830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:51:51.370 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:51.366+0000 7f84f7221640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f84f006cd70 0x7f84f0143fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:51:51.370 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:51.366+0000 7f84f7221640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f84f006d200 con 0x7f84f006cd70 2023-08-09T13:51:51.371 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:51.366+0000 7f84f7221640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f84f0143b30 con 0x7f84f015d860 2023-08-09T13:51:51.371 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:51.366+0000 7f84f7221640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f84f0142f20 con 0x7f84f00ca990 2023-08-09T13:51:51.371 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:51.367+0000 7f84f6a20640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f84f006cd70 0x7f84f0143fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:51:51.371 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:51.367+0000 7f84f6a20640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f84f006cd70 0x7f84f0143fc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:48764/0 (socket says 172.21.15.43:48764) 2023-08-09T13:51:51.371 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:51.367+0000 7f84f6a20640 1 -- 172.21.15.43:0/2398508188 learned_addr learned my addr 172.21.15.43:0/2398508188 (peer_addr_for_me v2:172.21.15.43:0/0) 2023-08-09T13:51:51.371 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:51.367+0000 7f84f621f640 1 --2- 172.21.15.43:0/2398508188 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f84f00ca990 0x7f84f015e3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:51:51.371 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:51.367+0000 7f84f5a1e640 1 --2- 172.21.15.43:0/2398508188 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f84f015d860 0x7f84f006c830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:51:51.372 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:51.367+0000 7f84f6a20640 1 -- 172.21.15.43:0/2398508188 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f84f00ca990 msgr2=0x7f84f015e3f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:51:51.372 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:51.367+0000 7f84f6a20640 1 --2- 172.21.15.43:0/2398508188 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f84f00ca990 0x7f84f015e3f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:51:51.372 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:51.367+0000 7f84f6a20640 1 -- 172.21.15.43:0/2398508188 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f84f015d860 msgr2=0x7f84f006c830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:51:51.372 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:51.367+0000 7f84f6a20640 1 --2- 172.21.15.43:0/2398508188 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f84f015d860 0x7f84f006c830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:51:51.372 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:51.367+0000 7f84f6a20640 1 -- 172.21.15.43:0/2398508188 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f84f0143200 con 0x7f84f006cd70 2023-08-09T13:51:51.372 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:51.367+0000 7f84f5a1e640 1 --2- 172.21.15.43:0/2398508188 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f84f015d860 0x7f84f006c830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-08-09T13:51:51.372 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:51.368+0000 7f84f6a20640 1 --2- 172.21.15.43:0/2398508188 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f84f006cd70 0x7f84f0143fc0 secure :-1 s=READY pgs=1294 cs=0 l=1 rev1=1 crypto rx=0x7f84e8002230 tx=0x7f84e8009750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-09T13:51:51.372 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:51.368+0000 7f84e6ffd640 1 -- 172.21.15.43:0/2398508188 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f84e8043070 con 0x7f84f006cd70 2023-08-09T13:51:51.373 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:51.368+0000 7f84e6ffd640 1 -- 172.21.15.43:0/2398508188 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f84e8003990 con 0x7f84f006cd70 2023-08-09T13:51:51.373 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:51.368+0000 7f84e6ffd640 1 -- 172.21.15.43:0/2398508188 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f84e8003d40 con 0x7f84f006cd70 2023-08-09T13:51:51.373 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:51.368+0000 7f84f7221640 1 -- 172.21.15.43:0/2398508188 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f84f01434e0 con 0x7f84f006cd70 2023-08-09T13:51:51.373 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:51.368+0000 7f84f7221640 1 -- 172.21.15.43:0/2398508188 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f84f00cad70 con 0x7f84f006cd70 2023-08-09T13:51:51.374 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:51.369+0000 7f84f7221640 1 -- 172.21.15.43:0/2398508188 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f84f00caf00 con 0x7f84f006cd70 2023-08-09T13:51:51.374 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:51.370+0000 7f84e6ffd640 1 -- 172.21.15.43:0/2398508188 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 45) v1 ==== 88443+0+0 (secure 0 0 0) 0x7f84e8034900 con 0x7f84f006cd70 2023-08-09T13:51:51.374 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:51.371+0000 7f84e6ffd640 1 --2- 172.21.15.43:0/2398508188 >> [v2:172.21.15.203:6832/1317108840,v1:172.21.15.203:6833/1317108840] conn(0x7f84d406a170 0x7f84d406c630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:51:51.374 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:51.371+0000 7f84e6ffd640 1 -- 172.21.15.43:0/2398508188 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(108..108 src has 1..108) v4 ==== 6348+0+0 (secure 0 0 0) 0x7f84e80a9fe0 con 0x7f84f006cd70 2023-08-09T13:51:51.374 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:51.372+0000 7f84f621f640 1 --2- 172.21.15.43:0/2398508188 >> [v2:172.21.15.203:6832/1317108840,v1:172.21.15.203:6833/1317108840] conn(0x7f84d406a170 0x7f84d406c630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:51:51.375 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:51.373+0000 7f84f621f640 1 --2- 172.21.15.43:0/2398508188 >> [v2:172.21.15.203:6832/1317108840,v1:172.21.15.203:6833/1317108840] conn(0x7f84d406a170 0x7f84d406c630 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7f84ec000c00 tx=0x7f84ec009040 comp rx=0 tx=0).ready entity=mgr.6619 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-09T13:51:51.377 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:51.376+0000 7f84e6ffd640 1 -- 172.21.15.43:0/2398508188 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178066 (secure 0 0 0) 0x7f84e80754f0 con 0x7f84f006cd70 2023-08-09T13:51:51.573 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:51.572+0000 7f84f7221640 1 -- 172.21.15.43:0/2398508188 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"} v 0) v1 -- 0x7f84f00ac930 con 0x7f84f006cd70 2023-08-09T13:51:51.574 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:51.573+0000 7f84e6ffd640 1 -- 172.21.15.43:0/2398508188 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"}]=0 v108) v1 ==== 92+0+10 (secure 0 0 0) 0x7f84f00ac930 con 0x7f84f006cd70 2023-08-09T13:51:51.575 INFO:teuthology.orchestra.run.smithi043.stdout:pg_num: 8 2023-08-09T13:51:51.577 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:51.575+0000 7f84f7221640 1 -- 172.21.15.43:0/2398508188 >> [v2:172.21.15.203:6832/1317108840,v1:172.21.15.203:6833/1317108840] conn(0x7f84d406a170 msgr2=0x7f84d406c630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:51:51.578 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:51.575+0000 7f84f7221640 1 --2- 172.21.15.43:0/2398508188 >> [v2:172.21.15.203:6832/1317108840,v1:172.21.15.203:6833/1317108840] conn(0x7f84d406a170 0x7f84d406c630 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7f84ec000c00 tx=0x7f84ec009040 comp rx=0 tx=0).stop 2023-08-09T13:51:51.578 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:51.575+0000 7f84f7221640 1 -- 172.21.15.43:0/2398508188 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f84f006cd70 msgr2=0x7f84f0143fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:51:51.578 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:51.575+0000 7f84f7221640 1 --2- 172.21.15.43:0/2398508188 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f84f006cd70 0x7f84f0143fc0 secure :-1 s=READY pgs=1294 cs=0 l=1 rev1=1 crypto rx=0x7f84e8002230 tx=0x7f84e8009750 comp rx=0 tx=0).stop 2023-08-09T13:51:51.578 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:51.576+0000 7f84f7221640 1 -- 172.21.15.43:0/2398508188 shutdown_connections 2023-08-09T13:51:51.579 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:51.576+0000 7f84f7221640 1 --2- 172.21.15.43:0/2398508188 >> [v2:172.21.15.203:6832/1317108840,v1:172.21.15.203:6833/1317108840] conn(0x7f84d406a170 0x7f84d406c630 unknown :-1 s=CLOSED pgs=42 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:51:51.579 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:51.576+0000 7f84f7221640 1 --2- 172.21.15.43:0/2398508188 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f84f006cd70 0x7f84f0143fc0 unknown :-1 s=CLOSED pgs=1294 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:51:51.579 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:51.576+0000 7f84f7221640 1 --2- 172.21.15.43:0/2398508188 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f84f015d860 0x7f84f006c830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:51:51.579 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:51.576+0000 7f84f7221640 1 --2- 172.21.15.43:0/2398508188 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f84f00ca990 0x7f84f015e3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:51:51.579 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:51.576+0000 7f84f7221640 1 -- 172.21.15.43:0/2398508188 >> 172.21.15.43:0/2398508188 conn(0x7f84f00cddd0 msgr2=0x7f84f00ab8d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-09T13:51:51.580 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:51.576+0000 7f84f7221640 1 -- 172.21.15.43:0/2398508188 shutdown_connections 2023-08-09T13:51:51.580 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:51.576+0000 7f84f7221640 1 -- 172.21.15.43:0/2398508188 wait complete. 2023-08-09T13:51:51.598 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-08-09T13:51:51.831 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:51.829+0000 7f785359e640 1 Processor -- start 2023-08-09T13:51:51.831 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:51.829+0000 7f785359e640 1 -- start start 2023-08-09T13:51:51.832 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:51.831+0000 7f785359e640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f7854157ea0 0x7f78540c97d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:51:51.832 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:51.831+0000 7f785359e640 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7f78540c90a0 0x7f78540c9d10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:51:51.833 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:51.831+0000 7f785359e640 1 -- --> v1:172.21.15.43:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7854158280 con 0x7f785415a800 2023-08-09T13:51:51.833 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:51.831+0000 7f785359e640 1 -- --> v2:172.21.15.203:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f785415b840 con 0x7f78540c90a0 2023-08-09T13:51:51.833 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:51.831+0000 7f785359e640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f785415b350 con 0x7f7854157ea0 2023-08-09T13:51:51.834 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:51.831+0000 7f7852d9d640 1 --1- >> v1:172.21.15.43:6789/0 conn(0x7f785415a800 0x7f785415ac40 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.43:6789/0 says I am v1:172.21.15.43:37708/0 (socket says 172.21.15.43:37708) 2023-08-09T13:51:51.834 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:51.831+0000 7f7852d9d640 1 -- 172.21.15.43:0/3796020543 learned_addr learned my addr 172.21.15.43:0/3796020543 (peer_addr_for_me v1:172.21.15.43:0/0) 2023-08-09T13:51:51.835 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:51.832+0000 7f785259c640 1 --2- 172.21.15.43:0/3796020543 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f7854157ea0 0x7f78540c97d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:51:51.835 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:51.832+0000 7f7851d9b640 1 --2- 172.21.15.43:0/3796020543 >> v2:172.21.15.203:3300/0 conn(0x7f78540c90a0 0x7f78540c9d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:51:51.835 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:51.832+0000 7f785159a640 1 -- 172.21.15.43:0/3796020543 <== mon.0 v1:172.21.15.43:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2369202554 0 0) 0x7f784c001490 con 0x7f785415a800 2023-08-09T13:51:51.835 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:51.832+0000 7f785159a640 1 -- 172.21.15.43:0/3796020543 --> v1:172.21.15.43:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f783c003520 con 0x7f785415a800 2023-08-09T13:51:51.835 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:51.832+0000 7f785159a640 1 -- 172.21.15.43:0/3796020543 <== mon.0 v1:172.21.15.43:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2193011641 0 0) 0x7f783c003520 con 0x7f785415a800 2023-08-09T13:51:51.835 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:51.832+0000 7f785159a640 1 -- 172.21.15.43:0/3796020543 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f7854157ea0 msgr2=0x7f78540c97d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-09T13:51:51.835 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:51.832+0000 7f785159a640 1 --2- 172.21.15.43:0/3796020543 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f7854157ea0 0x7f78540c97d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:51:51.835 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:51.832+0000 7f785159a640 1 -- 172.21.15.43:0/3796020543 >> v2:172.21.15.203:3300/0 conn(0x7f78540c90a0 msgr2=0x7f78540c9d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-09T13:51:51.836 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:51.832+0000 7f785159a640 1 --2- 172.21.15.43:0/3796020543 >> v2:172.21.15.203:3300/0 conn(0x7f78540c90a0 0x7f78540c9d10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:51:51.836 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:51.832+0000 7f785159a640 1 -- 172.21.15.43:0/3796020543 --> v1:172.21.15.43:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f785415b560 con 0x7f785415a800 2023-08-09T13:51:51.836 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:51.832+0000 7f785159a640 1 -- 172.21.15.43:0/3796020543 <== mon.0 v1:172.21.15.43:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1285150740 0 0) 0x7f784c001b80 con 0x7f785415a800 2023-08-09T13:51:51.836 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:51.833+0000 7f785159a640 1 -- 172.21.15.43:0/3796020543 >> v1:172.21.15.43:6789/0 conn(0x7f785415a800 legacy=0x7f785415ac40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:51:51.836 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:51.833+0000 7f785159a640 1 --2- 172.21.15.43:0/3796020543 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f783c003970 0x7f783c003d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:51:51.836 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:51.833+0000 7f785159a640 1 --2- 172.21.15.43:0/3796020543 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f7854157ea0 0x7f783c0047c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:51:51.836 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:51.833+0000 7f785159a640 1 --2- 172.21.15.43:0/3796020543 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f783c004d00 0x7f783c005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:51:51.837 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:51.833+0000 7f785159a640 1 -- 172.21.15.43:0/3796020543 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f785415b840 con 0x7f783c004d00 2023-08-09T13:51:51.837 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:51.833+0000 7f785159a640 1 -- 172.21.15.43:0/3796020543 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f785415b350 con 0x7f783c003970 2023-08-09T13:51:51.837 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:51.833+0000 7f785159a640 1 -- 172.21.15.43:0/3796020543 --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f783c00b980 con 0x7f7854157ea0 2023-08-09T13:51:51.837 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:51.833+0000 7f7852d9d640 1 --2- 172.21.15.43:0/3796020543 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f783c004d00 0x7f783c005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:51:51.838 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:51.833+0000 7f7851d9b640 1 --2- 172.21.15.43:0/3796020543 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f7854157ea0 0x7f783c0047c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:51:51.838 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:51.833+0000 7f785259c640 1 --2- 172.21.15.43:0/3796020543 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f783c003970 0x7f783c003d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:51:51.838 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:51.834+0000 7f7852d9d640 1 -- 172.21.15.43:0/3796020543 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f7854157ea0 msgr2=0x7f783c0047c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-09T13:51:51.839 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:51.834+0000 7f7852d9d640 1 --2- 172.21.15.43:0/3796020543 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f7854157ea0 0x7f783c0047c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:51:51.839 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:51.834+0000 7f7852d9d640 1 -- 172.21.15.43:0/3796020543 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f783c003970 msgr2=0x7f783c003d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-09T13:51:51.839 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:51.834+0000 7f7852d9d640 1 --2- 172.21.15.43:0/3796020543 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f783c003970 0x7f783c003d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:51:51.839 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:51.834+0000 7f7852d9d640 1 -- 172.21.15.43:0/3796020543 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f783c003520 con 0x7f783c004d00 2023-08-09T13:51:51.839 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:51.835+0000 7f7852d9d640 1 --2- 172.21.15.43:0/3796020543 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f783c004d00 0x7f783c005180 secure :-1 s=READY pgs=1296 cs=0 l=1 rev1=1 crypto rx=0x7f784c00a830 tx=0x7f783c0057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f49a8219454074e9 server_cookie=0 in_seq=0 out_seq=0 2023-08-09T13:51:51.839 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:51.835+0000 7f785159a640 1 -- 172.21.15.43:0/3796020543 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f784c034630 con 0x7f783c004d00 2023-08-09T13:51:51.839 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:51.835+0000 7f785159a640 1 -- 172.21.15.43:0/3796020543 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f784c02f070 con 0x7f783c004d00 2023-08-09T13:51:51.840 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:51.835+0000 7f785159a640 1 -- 172.21.15.43:0/3796020543 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f784c0349d0 con 0x7f783c004d00 2023-08-09T13:51:51.840 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:51.836+0000 7f785359e640 1 -- 172.21.15.43:0/3796020543 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f783c004d00 msgr2=0x7f783c005180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:51:51.840 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:51.836+0000 7f785359e640 1 --2- 172.21.15.43:0/3796020543 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f783c004d00 0x7f783c005180 secure :-1 s=READY pgs=1296 cs=0 l=1 rev1=1 crypto rx=0x7f784c00a830 tx=0x7f783c0057f0 comp rx=0 tx=0).stop 2023-08-09T13:51:51.840 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:51.836+0000 7f7852d9d640 1 -- 172.21.15.43:0/3796020543 reap_dead start 2023-08-09T13:51:51.840 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:51.836+0000 7f785359e640 1 -- 172.21.15.43:0/3796020543 shutdown_connections 2023-08-09T13:51:51.840 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:51.836+0000 7f785359e640 1 -- 172.21.15.43:0/3796020543 >> 172.21.15.43:0/3796020543 conn(0x7f78540cddd0 msgr2=0x7f78540ce1e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-09T13:51:51.840 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:51.836+0000 7f785359e640 1 -- 172.21.15.43:0/3796020543 shutdown_connections 2023-08-09T13:51:51.840 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:51.836+0000 7f785359e640 1 -- 172.21.15.43:0/3796020543 wait complete. 2023-08-09T13:51:51.842 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:51.838+0000 7f785359e640 1 Processor -- start 2023-08-09T13:51:51.842 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:51.838+0000 7f785359e640 1 -- start start 2023-08-09T13:51:51.843 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:51.839+0000 7f785359e640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f78540c3b00 0x7f78540c18e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:51:51.843 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:51.839+0000 7f785359e640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f78540c3ee0 0x7f78540c1e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:51:51.843 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:51.839+0000 7f785359e640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f785414c630 0x7f785414cae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:51:51.843 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:51.839+0000 7f785359e640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f785414d020 con 0x7f78540c3ee0 2023-08-09T13:51:51.843 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:51.839+0000 7f785359e640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f78540c2450 con 0x7f785414c630 2023-08-09T13:51:51.843 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:51.839+0000 7f785359e640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f78540c2730 con 0x7f78540c3b00 2023-08-09T13:51:51.843 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:51.839+0000 7f7851d9b640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f78540c3ee0 0x7f78540c1e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:51:51.844 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:51.840+0000 7f7851d9b640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f78540c3ee0 0x7f78540c1e20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:48794/0 (socket says 172.21.15.43:48794) 2023-08-09T13:51:51.844 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:51.840+0000 7f7851d9b640 1 -- 172.21.15.43:0/4120694819 learned_addr learned my addr 172.21.15.43:0/4120694819 (peer_addr_for_me v2:172.21.15.43:0/0) 2023-08-09T13:51:51.844 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:51.840+0000 7f785259c640 1 --2- 172.21.15.43:0/4120694819 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f78540c3b00 0x7f78540c18e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:51:51.844 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:51.840+0000 7f7852d9d640 1 --2- 172.21.15.43:0/4120694819 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f785414c630 0x7f785414cae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:51:51.844 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:51.840+0000 7f7851d9b640 1 -- 172.21.15.43:0/4120694819 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f78540c3b00 msgr2=0x7f78540c18e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:51:51.844 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:51.840+0000 7f7851d9b640 1 --2- 172.21.15.43:0/4120694819 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f78540c3b00 0x7f78540c18e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:51:51.844 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:51.840+0000 7f7851d9b640 1 -- 172.21.15.43:0/4120694819 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f785414c630 msgr2=0x7f785414cae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:51:51.844 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:51.840+0000 7f7851d9b640 1 --2- 172.21.15.43:0/4120694819 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f785414c630 0x7f785414cae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:51:51.845 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:51.840+0000 7f7851d9b640 1 -- 172.21.15.43:0/4120694819 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f78540c2a10 con 0x7f78540c3ee0 2023-08-09T13:51:51.845 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:51.840+0000 7f7851d9b640 1 --2- 172.21.15.43:0/4120694819 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f78540c3ee0 0x7f78540c1e20 secure :-1 s=READY pgs=1297 cs=0 l=1 rev1=1 crypto rx=0x7f78480116d0 tx=0x7f7848011b90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-09T13:51:51.845 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:51.841+0000 7f7842ffd640 1 -- 172.21.15.43:0/4120694819 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f784800f9c0 con 0x7f78540c3ee0 2023-08-09T13:51:51.845 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:51.841+0000 7f7842ffd640 1 -- 172.21.15.43:0/4120694819 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f784800fb20 con 0x7f78540c3ee0 2023-08-09T13:51:51.845 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:51.841+0000 7f7842ffd640 1 -- 172.21.15.43:0/4120694819 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f784800fe90 con 0x7f78540c3ee0 2023-08-09T13:51:51.845 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:51.841+0000 7f785359e640 1 -- 172.21.15.43:0/4120694819 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f78540c2cf0 con 0x7f78540c3ee0 2023-08-09T13:51:51.845 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:51.841+0000 7f785359e640 1 -- 172.21.15.43:0/4120694819 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f785406cc80 con 0x7f78540c3ee0 2023-08-09T13:51:51.846 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:51.842+0000 7f785359e640 1 -- 172.21.15.43:0/4120694819 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f785404e390 con 0x7f78540c3ee0 2023-08-09T13:51:51.846 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:51.843+0000 7f7842ffd640 1 -- 172.21.15.43:0/4120694819 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 45) v1 ==== 88443+0+0 (secure 0 0 0) 0x7f7848015460 con 0x7f78540c3ee0 2023-08-09T13:51:51.846 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:51.845+0000 7f7842ffd640 1 --2- 172.21.15.43:0/4120694819 >> [v2:172.21.15.203:6832/1317108840,v1:172.21.15.203:6833/1317108840] conn(0x7f783406a1a0 0x7f783406c660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:51:51.846 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:51.845+0000 7f7842ffd640 1 -- 172.21.15.43:0/4120694819 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(108..108 src has 1..108) v4 ==== 6348+0+0 (secure 0 0 0) 0x7f7848088d50 con 0x7f78540c3ee0 2023-08-09T13:51:51.847 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:51.845+0000 7f785259c640 1 --2- 172.21.15.43:0/4120694819 >> [v2:172.21.15.203:6832/1317108840,v1:172.21.15.203:6833/1317108840] conn(0x7f783406a1a0 0x7f783406c660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:51:51.848 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:51.847+0000 7f785259c640 1 --2- 172.21.15.43:0/4120694819 >> [v2:172.21.15.203:6832/1317108840,v1:172.21.15.203:6833/1317108840] conn(0x7f783406a1a0 0x7f783406c660 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7f7844009ac0 tx=0x7f7844008040 comp rx=0 tx=0).ready entity=mgr.6619 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-09T13:51:51.850 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:51.849+0000 7f7842ffd640 1 -- 172.21.15.43:0/4120694819 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178066 (secure 0 0 0) 0x7f7848054170 con 0x7f78540c3ee0 2023-08-09T13:51:52.051 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:52.050+0000 7f785359e640 1 -- 172.21.15.43:0/4120694819 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f7854073110 con 0x7f78540c3ee0 2023-08-09T13:51:52.052 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:52.051+0000 7f7842ffd640 1 -- 172.21.15.43:0/4120694819 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v108) v1 ==== 93+0+10 (secure 0 0 0) 0x7f7848057790 con 0x7f78540c3ee0 2023-08-09T13:51:52.052 INFO:teuthology.orchestra.run.smithi043.stdout:pg_num: 1 2023-08-09T13:51:52.054 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:52.053+0000 7f785359e640 1 -- 172.21.15.43:0/4120694819 >> [v2:172.21.15.203:6832/1317108840,v1:172.21.15.203:6833/1317108840] conn(0x7f783406a1a0 msgr2=0x7f783406c660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:51:52.055 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:52.053+0000 7f785359e640 1 --2- 172.21.15.43:0/4120694819 >> [v2:172.21.15.203:6832/1317108840,v1:172.21.15.203:6833/1317108840] conn(0x7f783406a1a0 0x7f783406c660 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7f7844009ac0 tx=0x7f7844008040 comp rx=0 tx=0).stop 2023-08-09T13:51:52.055 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:52.053+0000 7f785359e640 1 -- 172.21.15.43:0/4120694819 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f78540c3ee0 msgr2=0x7f78540c1e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:51:52.055 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:52.053+0000 7f785359e640 1 --2- 172.21.15.43:0/4120694819 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f78540c3ee0 0x7f78540c1e20 secure :-1 s=READY pgs=1297 cs=0 l=1 rev1=1 crypto rx=0x7f78480116d0 tx=0x7f7848011b90 comp rx=0 tx=0).stop 2023-08-09T13:51:52.055 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:52.053+0000 7f785359e640 1 -- 172.21.15.43:0/4120694819 shutdown_connections 2023-08-09T13:51:52.056 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:52.053+0000 7f785359e640 1 --2- 172.21.15.43:0/4120694819 >> [v2:172.21.15.203:6832/1317108840,v1:172.21.15.203:6833/1317108840] conn(0x7f783406a1a0 0x7f783406c660 unknown :-1 s=CLOSED pgs=43 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:51:52.056 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:52.053+0000 7f785359e640 1 --2- 172.21.15.43:0/4120694819 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f785414c630 0x7f785414cae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:51:52.056 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:52.053+0000 7f785359e640 1 --2- 172.21.15.43:0/4120694819 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f78540c3ee0 0x7f78540c1e20 unknown :-1 s=CLOSED pgs=1297 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:51:52.056 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:52.053+0000 7f785359e640 1 --2- 172.21.15.43:0/4120694819 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f78540c3b00 0x7f78540c18e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:51:52.056 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:52.053+0000 7f785359e640 1 -- 172.21.15.43:0/4120694819 >> 172.21.15.43:0/4120694819 conn(0x7f78540cddd0 msgr2=0x7f7854070be0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-09T13:51:52.056 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:52.054+0000 7f785359e640 1 -- 172.21.15.43:0/4120694819 shutdown_connections 2023-08-09T13:51:52.057 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:52.054+0000 7f785359e640 1 -- 172.21.15.43:0/4120694819 wait complete. 2023-08-09T13:51:52.075 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mon mon_allow_pool_delete True 2023-08-09T13:51:52.302 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:52.301+0000 7fbc07fff640 1 Processor -- start 2023-08-09T13:51:52.302 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:52.301+0000 7fbc07fff640 1 -- start start 2023-08-09T13:51:52.304 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:52.302+0000 7fbc07fff640 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7fbc0806f2c0 0x7fbc0815b0f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:51:52.304 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:52.302+0000 7fbc07fff640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7fbc080c8470 0x7fbc080c8880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:51:52.304 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:52.302+0000 7fbc07fff640 1 -- --> v1:172.21.15.43:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbc080c8dc0 con 0x7fbc08157ce0 2023-08-09T13:51:52.304 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:52.302+0000 7fbc07fff640 1 -- --> v2:172.21.15.43:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbc081580c0 con 0x7fbc080c8470 2023-08-09T13:51:52.305 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:52.302+0000 7fbc07fff640 1 -- --> v2:172.21.15.203:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbc080c92c0 con 0x7fbc0806f2c0 2023-08-09T13:51:52.305 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:52.303+0000 7fbc06ffd640 1 --1- >> v1:172.21.15.43:6789/0 conn(0x7fbc08157ce0 0x7fbc0806bc40 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.43:6789/0 says I am v1:172.21.15.43:37720/0 (socket says 172.21.15.43:37720) 2023-08-09T13:51:52.305 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:52.303+0000 7fbc06ffd640 1 -- 172.21.15.43:0/245260238 learned_addr learned my addr 172.21.15.43:0/245260238 (peer_addr_for_me v1:172.21.15.43:0/0) 2023-08-09T13:51:52.305 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:52.303+0000 7fbc077fe640 1 --2- 172.21.15.43:0/245260238 >> v2:172.21.15.43:3300/0 conn(0x7fbc080c8470 0x7fbc080c8880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:51:52.306 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:52.303+0000 7fbc077fe640 1 -- 172.21.15.43:0/245260238 >> v2:172.21.15.203:3300/0 conn(0x7fbc0806f2c0 msgr2=0x7fbc0815b0f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-09T13:51:52.306 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:52.303+0000 7fbc077fe640 1 --2- 172.21.15.43:0/245260238 >> v2:172.21.15.203:3300/0 conn(0x7fbc0806f2c0 0x7fbc0815b0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:51:52.306 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:52.303+0000 7fbc077fe640 1 -- 172.21.15.43:0/245260238 >> v1:172.21.15.43:6789/0 conn(0x7fbc08157ce0 legacy=0x7fbc0806bc40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:51:52.306 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:52.303+0000 7fbc077fe640 1 -- 172.21.15.43:0/245260238 --> v2:172.21.15.43:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbc0806b480 con 0x7fbc080c8470 2023-08-09T13:51:52.306 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:52.304+0000 7fbc077fe640 1 --2- 172.21.15.43:0/245260238 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fbc080c8470 0x7fbc080c8880 secure :-1 s=READY pgs=1299 cs=0 l=1 rev1=1 crypto rx=0x7fbc080c9400 tx=0x7fbc0002bc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=854d2088fff3d6e1 server_cookie=0 in_seq=0 out_seq=0 2023-08-09T13:51:52.313 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:52.305+0000 7fbc067fc640 1 -- 172.21.15.43:0/245260238 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbc0002c990 con 0x7fbc080c8470 2023-08-09T13:51:52.313 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:52.305+0000 7fbc067fc640 1 -- 172.21.15.43:0/245260238 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbc0002caf0 con 0x7fbc080c8470 2023-08-09T13:51:52.313 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:52.305+0000 7fbc067fc640 1 -- 172.21.15.43:0/245260238 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbc0002ce80 con 0x7fbc080c8470 2023-08-09T13:51:52.313 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:52.305+0000 7fbc07fff640 1 -- 172.21.15.43:0/245260238 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fbc080c8470 msgr2=0x7fbc080c8880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:51:52.314 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:52.305+0000 7fbc07fff640 1 --2- 172.21.15.43:0/245260238 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fbc080c8470 0x7fbc080c8880 secure :-1 s=READY pgs=1299 cs=0 l=1 rev1=1 crypto rx=0x7fbc080c9400 tx=0x7fbc0002bc60 comp rx=0 tx=0).stop 2023-08-09T13:51:52.314 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:52.306+0000 7fbc07fff640 1 -- 172.21.15.43:0/245260238 shutdown_connections 2023-08-09T13:51:52.314 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:52.306+0000 7fbc07fff640 1 --2- 172.21.15.43:0/245260238 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fbc080c8470 0x7fbc080c8880 unknown :-1 s=CLOSED pgs=1299 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:51:52.314 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:52.306+0000 7fbc07fff640 1 --2- 172.21.15.43:0/245260238 >> v2:172.21.15.203:3300/0 conn(0x7fbc0806f2c0 0x7fbc0815b0f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:51:52.314 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:52.306+0000 7fbc07fff640 1 -- 172.21.15.43:0/245260238 >> 172.21.15.43:0/245260238 conn(0x7fbc080af9f0 msgr2=0x7fbc080afdd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-09T13:51:52.314 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:52.306+0000 7fbc07fff640 1 -- 172.21.15.43:0/245260238 shutdown_connections 2023-08-09T13:51:52.314 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:52.306+0000 7fbc07fff640 1 -- 172.21.15.43:0/245260238 wait complete. 2023-08-09T13:51:52.315 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:52.307+0000 7fbc07fff640 1 Processor -- start 2023-08-09T13:51:52.315 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:52.308+0000 7fbc07fff640 1 -- start start 2023-08-09T13:51:52.315 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:52.308+0000 7fbc07fff640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fbc0806f2c0 0x7fbc0814c530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:51:52.315 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:52.308+0000 7fbc07fff640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fbc080c8470 0x7fbc0815da30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:51:52.315 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:52.308+0000 7fbc07fff640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fbc08157ce0 0x7fbc0815ba20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:51:52.315 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:52.309+0000 7fbc07fff640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbc080c32c0 con 0x7fbc08157ce0 2023-08-09T13:51:52.315 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:52.309+0000 7fbc07fff640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbc0814ca70 con 0x7fbc0806f2c0 2023-08-09T13:51:52.316 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:52.309+0000 7fbc07fff640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbc0815e4a0 con 0x7fbc080c8470 2023-08-09T13:51:52.316 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:52.309+0000 7fbc077fe640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fbc08157ce0 0x7fbc0815ba20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:51:52.316 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:52.309+0000 7fbc077fe640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fbc08157ce0 0x7fbc0815ba20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:48806/0 (socket says 172.21.15.43:48806) 2023-08-09T13:51:52.316 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:52.309+0000 7fbc077fe640 1 -- 172.21.15.43:0/1335710359 learned_addr learned my addr 172.21.15.43:0/1335710359 (peer_addr_for_me v2:172.21.15.43:0/0) 2023-08-09T13:51:52.316 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:52.309+0000 7fbc06ffd640 1 --2- 172.21.15.43:0/1335710359 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fbc0806f2c0 0x7fbc0814c530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:51:52.316 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:52.309+0000 7fbbfffff640 1 --2- 172.21.15.43:0/1335710359 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fbc080c8470 0x7fbc0815da30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:51:52.316 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:52.309+0000 7fbc077fe640 1 -- 172.21.15.43:0/1335710359 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fbc080c8470 msgr2=0x7fbc0815da30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:51:52.317 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:52.309+0000 7fbc077fe640 1 --2- 172.21.15.43:0/1335710359 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fbc080c8470 0x7fbc0815da30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:51:52.317 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:52.309+0000 7fbc077fe640 1 -- 172.21.15.43:0/1335710359 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fbc0806f2c0 msgr2=0x7fbc0814c530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:51:52.317 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:52.309+0000 7fbc077fe640 1 --2- 172.21.15.43:0/1335710359 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fbc0806f2c0 0x7fbc0814c530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:51:52.317 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:52.309+0000 7fbc077fe640 1 -- 172.21.15.43:0/1335710359 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbc00009a70 con 0x7fbc08157ce0 2023-08-09T13:51:52.317 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:52.310+0000 7fbc077fe640 1 --2- 172.21.15.43:0/1335710359 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fbc08157ce0 0x7fbc0815ba20 secure :-1 s=READY pgs=1300 cs=0 l=1 rev1=1 crypto rx=0x7fbc00009e50 tx=0x7fbc00004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-09T13:51:52.317 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:52.310+0000 7fbbff7fe640 1 -- 172.21.15.43:0/1335710359 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbc0003e610 con 0x7fbc08157ce0 2023-08-09T13:51:52.317 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:52.310+0000 7fbbff7fe640 1 -- 172.21.15.43:0/1335710359 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbc0003e790 con 0x7fbc08157ce0 2023-08-09T13:51:52.318 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:52.310+0000 7fbbff7fe640 1 -- 172.21.15.43:0/1335710359 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbc0003eb40 con 0x7fbc08157ce0 2023-08-09T13:51:52.318 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:52.310+0000 7fbc07fff640 1 -- 172.21.15.43:0/1335710359 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbc080c2c70 con 0x7fbc08157ce0 2023-08-09T13:51:52.318 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:52.310+0000 7fbc07fff640 1 -- 172.21.15.43:0/1335710359 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbc081e3960 con 0x7fbc08157ce0 2023-08-09T13:51:52.318 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:52.311+0000 7fbc07fff640 1 -- 172.21.15.43:0/1335710359 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbc080c2ad0 con 0x7fbc08157ce0 2023-08-09T13:51:52.319 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:52.312+0000 7fbbff7fe640 1 -- 172.21.15.43:0/1335710359 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 45) v1 ==== 88443+0+0 (secure 0 0 0) 0x7fbc0003b050 con 0x7fbc08157ce0 2023-08-09T13:51:52.319 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:52.313+0000 7fbbff7fe640 1 --2- 172.21.15.43:0/1335710359 >> [v2:172.21.15.203:6832/1317108840,v1:172.21.15.203:6833/1317108840] conn(0x7fbbe806a1a0 0x7fbbe806c660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:51:52.319 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:52.313+0000 7fbbff7fe640 1 -- 172.21.15.43:0/1335710359 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(108..108 src has 1..108) v4 ==== 6348+0+0 (secure 0 0 0) 0x7fbc000aafe0 con 0x7fbc08157ce0 2023-08-09T13:51:52.319 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:52.314+0000 7fbc06ffd640 1 --2- 172.21.15.43:0/1335710359 >> [v2:172.21.15.203:6832/1317108840,v1:172.21.15.203:6833/1317108840] conn(0x7fbbe806a1a0 0x7fbbe806c660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:51:52.319 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:52.315+0000 7fbc06ffd640 1 --2- 172.21.15.43:0/1335710359 >> [v2:172.21.15.203:6832/1317108840,v1:172.21.15.203:6833/1317108840] conn(0x7fbbe806a1a0 0x7fbbe806c660 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7fbbf8007fd0 tx=0x7fbbf8007c20 comp rx=0 tx=0).ready entity=mgr.6619 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-09T13:51:52.320 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:52.318+0000 7fbbff7fe640 1 -- 172.21.15.43:0/1335710359 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178066 (secure 0 0 0) 0x7fbc080c2ad0 con 0x7fbc08157ce0 2023-08-09T13:51:52.512 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:52.510+0000 7fbc07fff640 1 -- 172.21.15.43:0/1335710359 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command([{prefix=config set, name=mon_allow_pool_delete}] v 0) v1 -- 0x7fbc080c2ad0 con 0x7fbc08157ce0 2023-08-09T13:51:52.521 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:52.519+0000 7fbbff7fe640 1 -- 172.21.15.43:0/1335710359 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_allow_pool_delete}]=0 v34)=0 v34) v1 ==== 122+0+0 (secure 0 0 0) 0x7fbc080c2ad0 con 0x7fbc08157ce0 2023-08-09T13:51:52.523 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:52.522+0000 7fbc07fff640 1 -- 172.21.15.43:0/1335710359 >> [v2:172.21.15.203:6832/1317108840,v1:172.21.15.203:6833/1317108840] conn(0x7fbbe806a1a0 msgr2=0x7fbbe806c660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:51:52.524 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:52.522+0000 7fbc07fff640 1 --2- 172.21.15.43:0/1335710359 >> [v2:172.21.15.203:6832/1317108840,v1:172.21.15.203:6833/1317108840] conn(0x7fbbe806a1a0 0x7fbbe806c660 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7fbbf8007fd0 tx=0x7fbbf8007c20 comp rx=0 tx=0).stop 2023-08-09T13:51:52.524 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:52.522+0000 7fbc07fff640 1 -- 172.21.15.43:0/1335710359 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fbc08157ce0 msgr2=0x7fbc0815ba20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:51:52.524 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:52.522+0000 7fbc07fff640 1 --2- 172.21.15.43:0/1335710359 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fbc08157ce0 0x7fbc0815ba20 secure :-1 s=READY pgs=1300 cs=0 l=1 rev1=1 crypto rx=0x7fbc00009e50 tx=0x7fbc00004a40 comp rx=0 tx=0).stop 2023-08-09T13:51:52.524 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:52.522+0000 7fbc07fff640 1 -- 172.21.15.43:0/1335710359 shutdown_connections 2023-08-09T13:51:52.524 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:52.522+0000 7fbc07fff640 1 --2- 172.21.15.43:0/1335710359 >> [v2:172.21.15.203:6832/1317108840,v1:172.21.15.203:6833/1317108840] conn(0x7fbbe806a1a0 0x7fbbe806c660 unknown :-1 s=CLOSED pgs=44 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:51:52.524 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:52.522+0000 7fbc07fff640 1 --2- 172.21.15.43:0/1335710359 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fbc08157ce0 0x7fbc0815ba20 unknown :-1 s=CLOSED pgs=1300 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:51:52.525 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:52.522+0000 7fbc07fff640 1 --2- 172.21.15.43:0/1335710359 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fbc080c8470 0x7fbc0815da30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:51:52.525 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:52.522+0000 7fbc07fff640 1 --2- 172.21.15.43:0/1335710359 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fbc0806f2c0 0x7fbc0814c530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:51:52.525 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:52.522+0000 7fbc07fff640 1 -- 172.21.15.43:0/1335710359 >> 172.21.15.43:0/1335710359 conn(0x7fbc080af9f0 msgr2=0x7fbc080afdd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-09T13:51:52.525 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:52.522+0000 7fbc07fff640 1 -- 172.21.15.43:0/1335710359 shutdown_connections 2023-08-09T13:51:52.526 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:52.522+0000 7fbc07fff640 1 -- 172.21.15.43:0/1335710359 wait complete. 2023-08-09T13:51:52.542 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2023-08-09T13:51:52.542 DEBUG:teuthology.orchestra.run.smithi043:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2023-08-09T13:51:52.659 DEBUG:teuthology.orchestra.run.smithi203:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2023-08-09T13:51:52.773 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2023-08-09T13:51:52.774 DEBUG:teuthology.orchestra.run.smithi043:> set -ex 2023-08-09T13:51:52.774 DEBUG:teuthology.orchestra.run.smithi043:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-09T13:51:52.836 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4054441,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6498844k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=27,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3249420k,nr_inodes=812355,mode=700,uid=1000,gid=1275 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6498844k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2023-08-09T13:51:52.836 DEBUG:teuthology.orchestra.run.smithi203:> set -ex 2023-08-09T13:51:52.837 DEBUG:teuthology.orchestra.run.smithi203:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-09T13:51:52.897 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4053648,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497576k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=27,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248784k,nr_inodes=812196,mode=700,uid=1000,gid=1275 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497576k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2023-08-09T13:51:52.898 DEBUG:teuthology.orchestra.run.smithi043:> set -ex 2023-08-09T13:51:52.898 DEBUG:teuthology.orchestra.run.smithi043:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-09T13:51:52.964 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4054441,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6498844k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=27,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3249420k,nr_inodes=812355,mode=700,uid=1000,gid=1275 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6498844k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2023-08-09T13:51:52.964 DEBUG:teuthology.orchestra.run.smithi203:> set -ex 2023-08-09T13:51:52.965 DEBUG:teuthology.orchestra.run.smithi203:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-09T13:51:53.023 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4053648,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497576k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=27,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248784k,nr_inodes=812196,mode=700,uid=1000,gid=1275 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497576k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2023-08-09T13:51:53.024 INFO:tasks.cephfs.cephfs_test_case:[, , , ] 2023-08-09T13:51:53.024 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-08-09T13:51:53.264 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:53.262+0000 7f40c7478640 1 Processor -- start 2023-08-09T13:51:53.264 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:53.263+0000 7f40c7478640 1 -- start start 2023-08-09T13:51:53.266 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:53.264+0000 7f40c7478640 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7f40c00cd6b0 0x7f40c00b4760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:51:53.266 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:53.264+0000 7f40c7478640 1 -- --> v1:172.21.15.43:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f40c00b7bf0 con 0x7f40c00b1050 2023-08-09T13:51:53.266 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:53.264+0000 7f40c7478640 1 -- --> v1:172.21.15.203:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f40c00b7ed0 con 0x7f40c015f7b0 2023-08-09T13:51:53.266 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:53.264+0000 7f40c7478640 1 -- --> v2:172.21.15.203:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f40c00d0370 con 0x7f40c00cd6b0 2023-08-09T13:51:53.267 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:53.265+0000 7f40c5c75640 1 --1- >> v1:172.21.15.43:6789/0 conn(0x7f40c00b1050 0x7f40c00d0e50 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.43:6789/0 says I am v1:172.21.15.43:37732/0 (socket says 172.21.15.43:37732) 2023-08-09T13:51:53.267 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:53.265+0000 7f40c5c75640 1 -- 172.21.15.43:0/3561702344 learned_addr learned my addr 172.21.15.43:0/3561702344 (peer_addr_for_me v1:172.21.15.43:0/0) 2023-08-09T13:51:53.268 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:53.265+0000 7f40c6c77640 1 --2- 172.21.15.43:0/3561702344 >> v2:172.21.15.203:3300/0 conn(0x7f40c00cd6b0 0x7f40c00b4760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:51:53.268 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:53.265+0000 7f40c5474640 1 -- 172.21.15.43:0/3561702344 <== mon.0 v1:172.21.15.43:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2232574489 0 0) 0x7f40c00b7bf0 con 0x7f40c00b1050 2023-08-09T13:51:53.269 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:53.265+0000 7f40c5474640 1 -- 172.21.15.43:0/3561702344 --> v1:172.21.15.43:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f40ac003520 con 0x7f40c00b1050 2023-08-09T13:51:53.269 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:53.266+0000 7f40c5474640 1 -- 172.21.15.43:0/3561702344 <== mon.0 v1:172.21.15.43:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 718997407 0 0) 0x7f40ac003520 con 0x7f40c00b1050 2023-08-09T13:51:53.269 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:53.266+0000 7f40c5474640 1 -- 172.21.15.43:0/3561702344 >> v2:172.21.15.203:3300/0 conn(0x7f40c00cd6b0 msgr2=0x7f40c00b4760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-09T13:51:53.269 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:53.266+0000 7f40c5474640 1 --2- 172.21.15.43:0/3561702344 >> v2:172.21.15.203:3300/0 conn(0x7f40c00cd6b0 0x7f40c00b4760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:51:53.270 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:53.266+0000 7f40c5474640 1 -- 172.21.15.43:0/3561702344 >> v1:172.21.15.203:6789/0 conn(0x7f40c015f7b0 legacy=0x7f40c00ccfa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:51:53.270 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:53.266+0000 7f40c5474640 1 -- 172.21.15.43:0/3561702344 --> v1:172.21.15.43:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f40c00d0650 con 0x7f40c00b1050 2023-08-09T13:51:53.270 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:53.266+0000 7f40c5474640 1 -- 172.21.15.43:0/3561702344 <== mon.0 v1:172.21.15.43:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1285150740 0 0) 0x7f40b0001670 con 0x7f40c00b1050 2023-08-09T13:51:53.270 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:53.266+0000 7f40c5474640 1 -- 172.21.15.43:0/3561702344 >> v1:172.21.15.43:6789/0 conn(0x7f40c00b1050 legacy=0x7f40c00d0e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:51:53.270 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:53.266+0000 7f40c5474640 1 --2- 172.21.15.43:0/3561702344 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f40ac003970 0x7f40ac003d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:51:53.270 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:53.266+0000 7f40c5474640 1 --2- 172.21.15.43:0/3561702344 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f40ac0047c0 0x7f40ac004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:51:53.271 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:53.266+0000 7f40c5474640 1 --2- 172.21.15.43:0/3561702344 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f40ac005180 0x7f40ac00b980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:51:53.271 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:53.266+0000 7f40c5474640 1 -- 172.21.15.43:0/3561702344 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f40c00d0370 con 0x7f40ac003970 2023-08-09T13:51:53.271 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:53.266+0000 7f40c5474640 1 -- 172.21.15.43:0/3561702344 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f40ac00c020 con 0x7f40ac0047c0 2023-08-09T13:51:53.271 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:53.266+0000 7f40c5474640 1 -- 172.21.15.43:0/3561702344 --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f40ac00c1e0 con 0x7f40ac005180 2023-08-09T13:51:53.271 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:53.266+0000 7f40c6476640 1 --2- 172.21.15.43:0/3561702344 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f40ac003970 0x7f40ac003d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:51:53.271 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:53.266+0000 7f40c5c75640 1 --2- 172.21.15.43:0/3561702344 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f40ac0047c0 0x7f40ac004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:51:53.271 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:53.267+0000 7f40c6476640 1 -- 172.21.15.43:0/3561702344 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f40ac005180 msgr2=0x7f40ac00b980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-09T13:51:53.272 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:53.267+0000 7f40c6476640 1 --2- 172.21.15.43:0/3561702344 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f40ac005180 0x7f40ac00b980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:51:53.272 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:53.267+0000 7f40c6476640 1 -- 172.21.15.43:0/3561702344 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f40ac0047c0 msgr2=0x7f40ac004c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-09T13:51:53.272 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:53.267+0000 7f40c6476640 1 --2- 172.21.15.43:0/3561702344 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f40ac0047c0 0x7f40ac004c40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:51:53.272 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:53.267+0000 7f40c6476640 1 -- 172.21.15.43:0/3561702344 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f40ac003520 con 0x7f40ac003970 2023-08-09T13:51:53.272 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:53.267+0000 7f40c6c77640 1 -- 172.21.15.43:0/3561702344 reap_dead start 2023-08-09T13:51:53.274 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:53.268+0000 7f40c6476640 1 --2- 172.21.15.43:0/3561702344 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f40ac003970 0x7f40ac003d90 secure :-1 s=READY pgs=1302 cs=0 l=1 rev1=1 crypto rx=0x7f40bc001750 tx=0x7f40ac0057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c1836e4831700fcc server_cookie=0 in_seq=0 out_seq=0 2023-08-09T13:51:53.274 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:53.268+0000 7f40c5474640 1 -- 172.21.15.43:0/3561702344 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f40bc0052d0 con 0x7f40ac003970 2023-08-09T13:51:53.274 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:53.268+0000 7f40c5474640 1 -- 172.21.15.43:0/3561702344 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f40bc02e050 con 0x7f40ac003970 2023-08-09T13:51:53.275 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:53.268+0000 7f40c5474640 1 -- 172.21.15.43:0/3561702344 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f40bc003830 con 0x7f40ac003970 2023-08-09T13:51:53.275 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:53.269+0000 7f40c7478640 1 -- 172.21.15.43:0/3561702344 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f40ac003970 msgr2=0x7f40ac003d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:51:53.275 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:53.269+0000 7f40c7478640 1 --2- 172.21.15.43:0/3561702344 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f40ac003970 0x7f40ac003d90 secure :-1 s=READY pgs=1302 cs=0 l=1 rev1=1 crypto rx=0x7f40bc001750 tx=0x7f40ac0057f0 comp rx=0 tx=0).stop 2023-08-09T13:51:53.275 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:53.269+0000 7f40c7478640 1 -- 172.21.15.43:0/3561702344 shutdown_connections 2023-08-09T13:51:53.275 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:53.269+0000 7f40c7478640 1 --2- 172.21.15.43:0/3561702344 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f40ac003970 0x7f40ac003d90 unknown :-1 s=CLOSED pgs=1302 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:51:53.275 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:53.269+0000 7f40c7478640 1 -- 172.21.15.43:0/3561702344 >> 172.21.15.43:0/3561702344 conn(0x7f40c006a2b0 msgr2=0x7f40c006a690 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-09T13:51:53.275 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:53.269+0000 7f40c7478640 1 -- 172.21.15.43:0/3561702344 shutdown_connections 2023-08-09T13:51:53.276 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:53.269+0000 7f40c7478640 1 -- 172.21.15.43:0/3561702344 wait complete. 2023-08-09T13:51:53.276 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:53.271+0000 7f40c7478640 1 Processor -- start 2023-08-09T13:51:53.276 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:53.271+0000 7f40c7478640 1 -- start start 2023-08-09T13:51:53.276 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:53.272+0000 7f40c7478640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f40ac003970 0x7f40c0153960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:51:53.276 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:53.272+0000 7f40c7478640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f40c0158d10 0x7f40c0156340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:51:53.276 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:53.272+0000 7f40c7478640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f40c01f1260 0x7f40c01f16e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:51:53.277 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:53.272+0000 7f40c7478640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f40c0159190 con 0x7f40c01f1260 2023-08-09T13:51:53.277 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:53.272+0000 7f40c7478640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f40c0159660 con 0x7f40c0158d10 2023-08-09T13:51:53.277 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:53.272+0000 7f40c7478640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f40c01534d0 con 0x7f40ac003970 2023-08-09T13:51:53.277 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:53.272+0000 7f40c6c77640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f40c01f1260 0x7f40c01f16e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:51:53.277 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:53.272+0000 7f40c6c77640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f40c01f1260 0x7f40c01f16e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:48824/0 (socket says 172.21.15.43:48824) 2023-08-09T13:51:53.277 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:53.272+0000 7f40c6c77640 1 -- 172.21.15.43:0/4057168985 learned_addr learned my addr 172.21.15.43:0/4057168985 (peer_addr_for_me v2:172.21.15.43:0/0) 2023-08-09T13:51:53.277 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:53.272+0000 7f40c6476640 1 --2- 172.21.15.43:0/4057168985 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f40ac003970 0x7f40c0153960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:51:53.278 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:53.273+0000 7f40c6c77640 1 -- 172.21.15.43:0/4057168985 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f40ac003970 msgr2=0x7f40c0153960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:51:53.278 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:53.273+0000 7f40c5c75640 1 --2- 172.21.15.43:0/4057168985 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f40c0158d10 0x7f40c0156340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:51:53.278 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:53.273+0000 7f40c6c77640 1 --2- 172.21.15.43:0/4057168985 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f40ac003970 0x7f40c0153960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:51:53.278 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:53.273+0000 7f40c6c77640 1 -- 172.21.15.43:0/4057168985 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f40c0158d10 msgr2=0x7f40c0156340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:51:53.278 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:53.273+0000 7f40c6c77640 1 --2- 172.21.15.43:0/4057168985 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f40c0158d10 0x7f40c0156340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:51:53.278 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:53.273+0000 7f40c6c77640 1 -- 172.21.15.43:0/4057168985 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f40c0159380 con 0x7f40c01f1260 2023-08-09T13:51:53.278 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:53.273+0000 7f40c6476640 1 --2- 172.21.15.43:0/4057168985 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f40ac003970 0x7f40c0153960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-08-09T13:51:53.280 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:53.273+0000 7f40c6c77640 1 --2- 172.21.15.43:0/4057168985 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f40c01f1260 0x7f40c01f16e0 secure :-1 s=READY pgs=1303 cs=0 l=1 rev1=1 crypto rx=0x7f40c00cdbe0 tx=0x7f40b800ef30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-09T13:51:53.280 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:53.273+0000 7f40b6ffd640 1 -- 172.21.15.43:0/4057168985 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f40b8010040 con 0x7f40c01f1260 2023-08-09T13:51:53.280 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:53.273+0000 7f40b6ffd640 1 -- 172.21.15.43:0/4057168985 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f40b800bba0 con 0x7f40c01f1260 2023-08-09T13:51:53.280 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:53.273+0000 7f40b6ffd640 1 -- 172.21.15.43:0/4057168985 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f40b8014400 con 0x7f40c01f1260 2023-08-09T13:51:53.280 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:53.273+0000 7f40c7478640 1 -- 172.21.15.43:0/4057168985 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f40c0152e80 con 0x7f40c01f1260 2023-08-09T13:51:53.281 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:53.274+0000 7f40c7478640 1 -- 172.21.15.43:0/4057168985 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f40c0156880 con 0x7f40c01f1260 2023-08-09T13:51:53.281 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:53.275+0000 7f40c7478640 1 -- 172.21.15.43:0/4057168985 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f40c0152ce0 con 0x7f40c01f1260 2023-08-09T13:51:53.281 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:53.276+0000 7f40b6ffd640 1 -- 172.21.15.43:0/4057168985 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 45) v1 ==== 88443+0+0 (secure 0 0 0) 0x7f40b8014560 con 0x7f40c01f1260 2023-08-09T13:51:53.281 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:53.276+0000 7f40b6ffd640 1 --2- 172.21.15.43:0/4057168985 >> [v2:172.21.15.203:6832/1317108840,v1:172.21.15.203:6833/1317108840] conn(0x7f40a406a170 0x7f40a406c630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:51:53.281 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:53.277+0000 7f40b6ffd640 1 -- 172.21.15.43:0/4057168985 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(108..108 src has 1..108) v4 ==== 6348+0+0 (secure 0 0 0) 0x7f40b8088e40 con 0x7f40c01f1260 2023-08-09T13:51:53.281 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:53.277+0000 7f40c6476640 1 --2- 172.21.15.43:0/4057168985 >> [v2:172.21.15.203:6832/1317108840,v1:172.21.15.203:6833/1317108840] conn(0x7f40a406a170 0x7f40a406c630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:51:53.281 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:53.279+0000 7f40c6476640 1 --2- 172.21.15.43:0/4057168985 >> [v2:172.21.15.203:6832/1317108840,v1:172.21.15.203:6833/1317108840] conn(0x7f40a406a170 0x7f40a406c630 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7f40bc033000 tx=0x7f40bc009030 comp rx=0 tx=0).ready entity=mgr.6619 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-09T13:51:53.282 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:53.281+0000 7f40b6ffd640 1 -- 172.21.15.43:0/4057168985 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178066 (secure 0 0 0) 0x7f40c0152ce0 con 0x7f40c01f1260 2023-08-09T13:51:53.513 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:53.512+0000 7f40c7478640 1 -- 172.21.15.43:0/4057168985 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f40c0072ae0 con 0x7f40c01f1260 2023-08-09T13:51:53.515 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:53.513+0000 7f40b6ffd640 1 -- 172.21.15.43:0/4057168985 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 45 v45) v1 ==== 94+0+3702 (secure 0 0 0) 0x7f40c0072ae0 con 0x7f40c01f1260 2023-08-09T13:51:53.515 INFO:teuthology.orchestra.run.smithi043.stdout: 2023-08-09T13:51:53.515 INFO:teuthology.orchestra.run.smithi043.stdout:{"epoch":45,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5966,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.203:6835/4101044758","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6834","nonce":4101044758},{"type":"v1","addr":"172.21.15.203:6835","nonce":4101044758}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":39},{"gid":6460,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.203:6837/2974664437","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6836","nonce":2974664437},{"type":"v1","addr":"172.21.15.203:6837","nonce":2974664437}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":39},{"gid":7527,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.43:6833/795245144","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6832","nonce":795245144},{"type":"v1","addr":"172.21.15.43:6833","nonce":795245144}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":38},{"gid":7941,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.43:6835/3454393568","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6834","nonce":3454393568},{"type":"v1","addr":"172.21.15.43:6835","nonce":3454393568}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":45}],"filesystems":[]} 2023-08-09T13:51:53.516 INFO:teuthology.orchestra.run.smithi043.stderr:dumped fsmap epoch 45 2023-08-09T13:51:53.517 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:53.516+0000 7f40c7478640 1 -- 172.21.15.43:0/4057168985 >> [v2:172.21.15.203:6832/1317108840,v1:172.21.15.203:6833/1317108840] conn(0x7f40a406a170 msgr2=0x7f40a406c630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:51:53.518 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:53.516+0000 7f40c7478640 1 --2- 172.21.15.43:0/4057168985 >> [v2:172.21.15.203:6832/1317108840,v1:172.21.15.203:6833/1317108840] conn(0x7f40a406a170 0x7f40a406c630 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7f40bc033000 tx=0x7f40bc009030 comp rx=0 tx=0).stop 2023-08-09T13:51:53.518 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:53.516+0000 7f40c7478640 1 -- 172.21.15.43:0/4057168985 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f40c01f1260 msgr2=0x7f40c01f16e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:51:53.518 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:53.516+0000 7f40c7478640 1 --2- 172.21.15.43:0/4057168985 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f40c01f1260 0x7f40c01f16e0 secure :-1 s=READY pgs=1303 cs=0 l=1 rev1=1 crypto rx=0x7f40c00cdbe0 tx=0x7f40b800ef30 comp rx=0 tx=0).stop 2023-08-09T13:51:53.519 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:53.516+0000 7f40c7478640 1 -- 172.21.15.43:0/4057168985 shutdown_connections 2023-08-09T13:51:53.519 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:53.516+0000 7f40c7478640 1 --2- 172.21.15.43:0/4057168985 >> [v2:172.21.15.203:6832/1317108840,v1:172.21.15.203:6833/1317108840] conn(0x7f40a406a170 0x7f40a406c630 unknown :-1 s=CLOSED pgs=45 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:51:53.519 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:53.516+0000 7f40c7478640 1 --2- 172.21.15.43:0/4057168985 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f40c01f1260 0x7f40c01f16e0 unknown :-1 s=CLOSED pgs=1303 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:51:53.519 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:53.516+0000 7f40c7478640 1 --2- 172.21.15.43:0/4057168985 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f40c0158d10 0x7f40c0156340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:51:53.519 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:53.516+0000 7f40c7478640 1 --2- 172.21.15.43:0/4057168985 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f40ac003970 0x7f40c0153960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:51:53.519 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:53.517+0000 7f40c7478640 1 -- 172.21.15.43:0/4057168985 >> 172.21.15.43:0/4057168985 conn(0x7f40c006a2b0 msgr2=0x7f40c015e800 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-09T13:51:53.519 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:53.517+0000 7f40c7478640 1 -- 172.21.15.43:0/4057168985 shutdown_connections 2023-08-09T13:51:53.520 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:53.517+0000 7f40c7478640 1 -- 172.21.15.43:0/4057168985 wait complete. 2023-08-09T13:51:53.537 INFO:tasks.ceph.mds.a:Restarting daemon 2023-08-09T13:51:53.537 INFO:tasks.ceph.mds.a:Stopping old one... 2023-08-09T13:51:53.537 DEBUG:tasks.ceph.mds.a:waiting for process to exit 2023-08-09T13:51:53.537 INFO:teuthology.orchestra.run:waiting for 300 2023-08-09T13:51:53.538 INFO:tasks.ceph.mds.c:Restarting daemon 2023-08-09T13:51:53.538 INFO:tasks.ceph.mds.c:Stopping old one... 2023-08-09T13:51:53.538 DEBUG:tasks.ceph.mds.c:waiting for process to exit 2023-08-09T13:51:53.538 INFO:teuthology.orchestra.run:waiting for 300 2023-08-09T13:51:53.539 INFO:tasks.ceph.mds.b:Restarting daemon 2023-08-09T13:51:53.539 INFO:tasks.ceph.mds.b:Stopping old one... 2023-08-09T13:51:53.539 DEBUG:tasks.ceph.mds.b:waiting for process to exit 2023-08-09T13:51:53.539 INFO:teuthology.orchestra.run:waiting for 300 2023-08-09T13:51:53.539 INFO:tasks.ceph.mds.d:Restarting daemon 2023-08-09T13:51:53.540 INFO:tasks.ceph.mds.d:Stopping old one... 2023-08-09T13:51:53.540 DEBUG:tasks.ceph.mds.d:waiting for process to exit 2023-08-09T13:51:53.540 INFO:teuthology.orchestra.run:waiting for 300 2023-08-09T13:51:53.557 INFO:tasks.ceph.mds.a:Stopped 2023-08-09T13:51:53.557 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i a 2023-08-09T13:51:53.560 INFO:tasks.ceph.mds.b:Stopped 2023-08-09T13:51:53.561 DEBUG:teuthology.orchestra.run.smithi203:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i b 2023-08-09T13:51:53.564 INFO:tasks.ceph.mds.d:Stopped 2023-08-09T13:51:53.564 DEBUG:teuthology.orchestra.run.smithi203:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i d 2023-08-09T13:51:53.567 INFO:tasks.ceph.mds.c:Stopped 2023-08-09T13:51:53.568 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i c 2023-08-09T13:51:53.612 INFO:tasks.ceph.mds.b:Started 2023-08-09T13:51:53.613 DEBUG:teuthology.parallel:result is None 2023-08-09T13:51:53.615 INFO:tasks.ceph.mds.a:Started 2023-08-09T13:51:53.616 DEBUG:teuthology.parallel:result is None 2023-08-09T13:51:53.620 INFO:tasks.ceph.mds.d:Started 2023-08-09T13:51:53.620 DEBUG:teuthology.parallel:result is None 2023-08-09T13:51:53.623 INFO:tasks.ceph.mds.c:Started 2023-08-09T13:51:53.624 DEBUG:teuthology.parallel:result is None 2023-08-09T13:51:53.625 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist clear 2023-08-09T13:51:53.779 INFO:tasks.ceph.mds.b.smithi203.stdout:starting mds.b at 2023-08-09T13:51:53.787 INFO:tasks.ceph.mds.d.smithi203.stdout:starting mds.d at 2023-08-09T13:51:53.788 INFO:tasks.ceph.mds.c.smithi043.stdout:starting mds.c at 2023-08-09T13:51:53.788 INFO:tasks.ceph.mds.a.smithi043.stdout:starting mds.a at 2023-08-09T13:51:53.822 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:53.820+0000 7f3ca7b12640 1 Processor -- start 2023-08-09T13:51:53.822 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:53.821+0000 7f3ca7b12640 1 -- start start 2023-08-09T13:51:53.823 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:53.822+0000 7f3ca7b12640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f3ca015cb00 0x7f3ca0070b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:51:53.823 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:53.822+0000 7f3ca7b12640 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7f3ca0070470 0x7f3ca00cce90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:51:53.823 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:53.822+0000 7f3ca7b12640 1 -- --> v1:172.21.15.203:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3ca015cee0 con 0x7f3ca00c9a10 2023-08-09T13:51:53.824 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:53.822+0000 7f3ca7b12640 1 -- --> v2:172.21.15.43:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3ca00cca00 con 0x7f3ca015cb00 2023-08-09T13:51:53.824 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:53.822+0000 7f3ca7b12640 1 -- --> v2:172.21.15.203:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3ca0071040 con 0x7f3ca0070470 2023-08-09T13:51:53.824 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:53.822+0000 7f3ca58ae640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f3ca015cb00 0x7f3ca0070b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:51:53.825 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:53.822+0000 7f3ca58ae640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f3ca015cb00 0x7f3ca0070b00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:48862/0 (socket says 172.21.15.43:48862) 2023-08-09T13:51:53.825 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:53.822+0000 7f3ca58ae640 1 -- 172.21.15.43:0/2873962955 learned_addr learned my addr 172.21.15.43:0/2873962955 (peer_addr_for_me v2:172.21.15.43:0/0) 2023-08-09T13:51:53.825 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:53.822+0000 7f3ca60af640 1 --2- 172.21.15.43:0/2873962955 >> v2:172.21.15.203:3300/0 conn(0x7f3ca0070470 0x7f3ca00cce90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:51:53.825 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:53.823+0000 7f3ca58ae640 1 -- 172.21.15.43:0/2873962955 >> v2:172.21.15.203:3300/0 conn(0x7f3ca0070470 msgr2=0x7f3ca00cce90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-09T13:51:53.825 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:53.823+0000 7f3ca58ae640 1 --2- 172.21.15.43:0/2873962955 >> v2:172.21.15.203:3300/0 conn(0x7f3ca0070470 0x7f3ca00cce90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:51:53.825 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:53.823+0000 7f3ca58ae640 1 -- 172.21.15.43:0/2873962955 >> v1:172.21.15.203:6789/0 conn(0x7f3ca00c9a10 legacy=0x7f3ca006fd60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-09T13:51:53.826 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:53.823+0000 7f3ca58ae640 1 -- 172.21.15.43:0/2873962955 --> v2:172.21.15.43:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3ca00cc6d0 con 0x7f3ca015cb00 2023-08-09T13:51:53.826 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:53.824+0000 7f3ca58ae640 1 --2- 172.21.15.43:0/2873962955 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f3ca015cb00 0x7f3ca0070b00 secure :-1 s=READY pgs=1307 cs=0 l=1 rev1=1 crypto rx=0x7f3ca0072e90 tx=0x7f3c9c02b880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=444a09f72081930b server_cookie=0 in_seq=0 out_seq=0 2023-08-09T13:51:53.827 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:53.824+0000 7f3ca48ac640 1 -- 172.21.15.43:0/2873962955 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3c9c0317e0 con 0x7f3ca015cb00 2023-08-09T13:51:53.827 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:53.824+0000 7f3ca48ac640 1 -- 172.21.15.43:0/2873962955 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3c9c031940 con 0x7f3ca015cb00 2023-08-09T13:51:53.827 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:53.824+0000 7f3ca48ac640 1 -- 172.21.15.43:0/2873962955 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3c9c031cd0 con 0x7f3ca015cb00 2023-08-09T13:51:53.827 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:53.825+0000 7f3ca7b12640 1 -- 172.21.15.43:0/2873962955 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f3ca015cb00 msgr2=0x7f3ca0070b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:51:53.827 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:53.825+0000 7f3ca7b12640 1 --2- 172.21.15.43:0/2873962955 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f3ca015cb00 0x7f3ca0070b00 secure :-1 s=READY pgs=1307 cs=0 l=1 rev1=1 crypto rx=0x7f3ca0072e90 tx=0x7f3c9c02b880 comp rx=0 tx=0).stop 2023-08-09T13:51:53.827 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:53.825+0000 7f3ca7b12640 1 -- 172.21.15.43:0/2873962955 shutdown_connections 2023-08-09T13:51:53.827 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:53.825+0000 7f3ca7b12640 1 --2- 172.21.15.43:0/2873962955 >> v2:172.21.15.203:3300/0 conn(0x7f3ca0070470 0x7f3ca00cce90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:51:53.827 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:53.825+0000 7f3ca7b12640 1 --2- 172.21.15.43:0/2873962955 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f3ca015cb00 0x7f3ca0070b00 unknown :-1 s=CLOSED pgs=1307 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:51:53.828 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:53.825+0000 7f3ca7b12640 1 -- 172.21.15.43:0/2873962955 >> 172.21.15.43:0/2873962955 conn(0x7f3ca00ce9d0 msgr2=0x7f3ca00cedb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-09T13:51:53.828 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:53.825+0000 7f3ca7b12640 1 -- 172.21.15.43:0/2873962955 shutdown_connections 2023-08-09T13:51:53.828 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:53.825+0000 7f3ca7b12640 1 -- 172.21.15.43:0/2873962955 wait complete. 2023-08-09T13:51:53.828 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:53.827+0000 7f3ca7b12640 1 Processor -- start 2023-08-09T13:51:53.829 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:53.827+0000 7f3ca7b12640 1 -- start start 2023-08-09T13:51:53.829 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:53.828+0000 7f3ca7b12640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f3ca0070470 0x7f3ca0158ce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:51:53.830 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:53.828+0000 7f3ca7b12640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f3ca015cb00 0x7f3ca01593c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:51:53.830 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:53.828+0000 7f3ca7b12640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f3ca00cd0a0 0x7f3ca00cd550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:51:53.830 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:53.828+0000 7f3ca7b12640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3ca006cba0 con 0x7f3ca00cd0a0 2023-08-09T13:51:53.830 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:53.828+0000 7f3ca7b12640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3ca00cda90 con 0x7f3ca015cb00 2023-08-09T13:51:53.830 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:53.828+0000 7f3ca7b12640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3ca0159220 con 0x7f3ca0070470 2023-08-09T13:51:53.830 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:53.828+0000 7f3ca60af640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f3ca00cd0a0 0x7f3ca00cd550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:51:53.831 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:53.828+0000 7f3ca60af640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f3ca00cd0a0 0x7f3ca00cd550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:48866/0 (socket says 172.21.15.43:48866) 2023-08-09T13:51:53.831 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:53.828+0000 7f3ca60af640 1 -- 172.21.15.43:0/2267169035 learned_addr learned my addr 172.21.15.43:0/2267169035 (peer_addr_for_me v2:172.21.15.43:0/0) 2023-08-09T13:51:53.831 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:53.828+0000 7f3ca58ae640 1 --2- 172.21.15.43:0/2267169035 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f3ca0070470 0x7f3ca0158ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:51:53.831 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:53.828+0000 7f3ca50ad640 1 --2- 172.21.15.43:0/2267169035 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f3ca015cb00 0x7f3ca01593c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:51:53.832 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:53.829+0000 7f3ca60af640 1 -- 172.21.15.43:0/2267169035 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f3ca0070470 msgr2=0x7f3ca0158ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:51:53.832 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:53.829+0000 7f3ca60af640 1 --2- 172.21.15.43:0/2267169035 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f3ca0070470 0x7f3ca0158ce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:51:53.832 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:53.829+0000 7f3ca60af640 1 -- 172.21.15.43:0/2267169035 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f3ca015cb00 msgr2=0x7f3ca01593c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:51:53.833 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:53.829+0000 7f3ca60af640 1 --2- 172.21.15.43:0/2267169035 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f3ca015cb00 0x7f3ca01593c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:51:53.833 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:53.829+0000 7f3ca60af640 1 -- 172.21.15.43:0/2267169035 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3c9c0094d0 con 0x7f3ca00cd0a0 2023-08-09T13:51:53.833 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:53.829+0000 7f3ca50ad640 1 --2- 172.21.15.43:0/2267169035 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f3ca015cb00 0x7f3ca01593c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-08-09T13:51:53.833 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:53.829+0000 7f3ca60af640 1 --2- 172.21.15.43:0/2267169035 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f3ca00cd0a0 0x7f3ca00cd550 secure :-1 s=READY pgs=1308 cs=0 l=1 rev1=1 crypto rx=0x7f3c9800efc0 tx=0x7f3c9800c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-09T13:51:53.833 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:53.829+0000 7f3c967fc640 1 -- 172.21.15.43:0/2267169035 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3c98009ba0 con 0x7f3ca00cd0a0 2023-08-09T13:51:53.833 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:53.829+0000 7f3c967fc640 1 -- 172.21.15.43:0/2267169035 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3c9800f040 con 0x7f3ca00cd0a0 2023-08-09T13:51:53.833 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:53.829+0000 7f3c967fc640 1 -- 172.21.15.43:0/2267169035 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3c980146c0 con 0x7f3ca00cd0a0 2023-08-09T13:51:53.834 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:53.829+0000 7f3ca7b12640 1 -- 172.21.15.43:0/2267169035 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3ca006bcd0 con 0x7f3ca00cd0a0 2023-08-09T13:51:53.834 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:53.829+0000 7f3ca7b12640 1 -- 172.21.15.43:0/2267169035 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3ca006c200 con 0x7f3ca00cd0a0 2023-08-09T13:51:53.834 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:53.830+0000 7f3ca7b12640 1 -- 172.21.15.43:0/2267169035 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3ca00731e0 con 0x7f3ca00cd0a0 2023-08-09T13:51:53.834 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:53.832+0000 7f3c967fc640 1 -- 172.21.15.43:0/2267169035 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 45) v1 ==== 88443+0+0 (secure 0 0 0) 0x7f3c98014820 con 0x7f3ca00cd0a0 2023-08-09T13:51:53.835 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:53.832+0000 7f3c967fc640 1 --2- 172.21.15.43:0/2267169035 >> [v2:172.21.15.203:6832/1317108840,v1:172.21.15.203:6833/1317108840] conn(0x7f3c8406a1a0 0x7f3c8406c660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:51:53.835 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:53.833+0000 7f3c967fc640 1 -- 172.21.15.43:0/2267169035 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(108..108 src has 1..108) v4 ==== 6348+0+0 (secure 0 0 0) 0x7f3c980896f0 con 0x7f3ca00cd0a0 2023-08-09T13:51:53.835 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:53.833+0000 7f3ca58ae640 1 --2- 172.21.15.43:0/2267169035 >> [v2:172.21.15.203:6832/1317108840,v1:172.21.15.203:6833/1317108840] conn(0x7f3c8406a1a0 0x7f3c8406c660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:51:53.835 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:53.834+0000 7f3ca58ae640 1 --2- 172.21.15.43:0/2267169035 >> [v2:172.21.15.203:6832/1317108840,v1:172.21.15.203:6833/1317108840] conn(0x7f3c8406a1a0 0x7f3c8406c660 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7f3c9c036000 tx=0x7f3c9c03d040 comp rx=0 tx=0).ready entity=mgr.6619 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-09T13:51:53.838 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:53.837+0000 7f3c967fc640 1 -- 172.21.15.43:0/2267169035 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178066 (secure 0 0 0) 0x7f3c98054a00 con 0x7f3ca00cd0a0 2023-08-09T13:51:54.051 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:54.049+0000 7f3ca7b12640 1 -- 172.21.15.43:0/2267169035 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "osd blocklist clear"} v 0) v1 -- 0x7f3ca00739f0 con 0x7f3ca00cd0a0 2023-08-09T13:51:54.663 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:54.661+0000 7f3c967fc640 1 -- 172.21.15.43:0/2267169035 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist clear"}]=0 removed all blocklist entries v110) v1 ==== 97+0+0 (secure 0 0 0) 0x7f3c98058020 con 0x7f3ca00cd0a0 2023-08-09T13:51:54.663 INFO:teuthology.orchestra.run.smithi043.stderr: removed all blocklist entries 2023-08-09T13:51:54.666 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:54.665+0000 7f3ca7b12640 1 -- 172.21.15.43:0/2267169035 >> [v2:172.21.15.203:6832/1317108840,v1:172.21.15.203:6833/1317108840] conn(0x7f3c8406a1a0 msgr2=0x7f3c8406c660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:51:54.667 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:54.665+0000 7f3ca7b12640 1 --2- 172.21.15.43:0/2267169035 >> [v2:172.21.15.203:6832/1317108840,v1:172.21.15.203:6833/1317108840] conn(0x7f3c8406a1a0 0x7f3c8406c660 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7f3c9c036000 tx=0x7f3c9c03d040 comp rx=0 tx=0).stop 2023-08-09T13:51:54.667 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:54.665+0000 7f3ca7b12640 1 -- 172.21.15.43:0/2267169035 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f3ca00cd0a0 msgr2=0x7f3ca00cd550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:51:54.667 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:54.665+0000 7f3ca7b12640 1 --2- 172.21.15.43:0/2267169035 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f3ca00cd0a0 0x7f3ca00cd550 secure :-1 s=READY pgs=1308 cs=0 l=1 rev1=1 crypto rx=0x7f3c9800efc0 tx=0x7f3c9800c490 comp rx=0 tx=0).stop 2023-08-09T13:51:54.667 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:54.665+0000 7f3ca7b12640 1 -- 172.21.15.43:0/2267169035 shutdown_connections 2023-08-09T13:51:54.668 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:54.665+0000 7f3ca7b12640 1 --2- 172.21.15.43:0/2267169035 >> [v2:172.21.15.203:6832/1317108840,v1:172.21.15.203:6833/1317108840] conn(0x7f3c8406a1a0 0x7f3c8406c660 unknown :-1 s=CLOSED pgs=46 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:51:54.668 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:54.665+0000 7f3ca7b12640 1 --2- 172.21.15.43:0/2267169035 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f3ca00cd0a0 0x7f3ca00cd550 unknown :-1 s=CLOSED pgs=1308 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:51:54.668 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:54.665+0000 7f3ca7b12640 1 --2- 172.21.15.43:0/2267169035 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f3ca015cb00 0x7f3ca01593c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:51:54.669 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:54.665+0000 7f3ca7b12640 1 --2- 172.21.15.43:0/2267169035 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f3ca0070470 0x7f3ca0158ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:51:54.669 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:54.665+0000 7f3ca7b12640 1 -- 172.21.15.43:0/2267169035 >> 172.21.15.43:0/2267169035 conn(0x7f3ca00ce9d0 msgr2=0x7f3ca00ae4f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-09T13:51:54.669 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:54.666+0000 7f3ca7b12640 1 -- 172.21.15.43:0/2267169035 shutdown_connections 2023-08-09T13:51:54.669 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:54.666+0000 7f3ca7b12640 1 -- 172.21.15.43:0/2267169035 wait complete. 2023-08-09T13:51:54.692 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth ls --format=json-pretty 2023-08-09T13:51:54.922 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:54.920+0000 7f993ecd7640 1 Processor -- start 2023-08-09T13:51:54.922 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:54.920+0000 7f993ecd7640 1 -- start start 2023-08-09T13:51:54.923 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:54.922+0000 7f993ecd7640 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7f993815a6a0 0x7f99380713b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:51:54.923 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:54.922+0000 7f993ecd7640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f99380cbd90 0x7f99380c8fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:51:54.924 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:54.922+0000 7f993ecd7640 1 -- --> v1:172.21.15.203:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f99380cdc70 con 0x7f99380c94f0 2023-08-09T13:51:54.924 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:54.922+0000 7f993ecd7640 1 -- --> v2:172.21.15.43:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f99380c9900 con 0x7f99380cbd90 2023-08-09T13:51:54.924 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:54.922+0000 7f993ecd7640 1 -- --> v2:172.21.15.203:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9938070910 con 0x7f993815a6a0 2023-08-09T13:51:54.925 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:54.922+0000 7f993d4d4640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f99380cbd90 0x7f99380c8fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:51:54.925 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:54.922+0000 7f993d4d4640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f99380cbd90 0x7f99380c8fb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:48880/0 (socket says 172.21.15.43:48880) 2023-08-09T13:51:54.925 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:54.922+0000 7f993d4d4640 1 -- 172.21.15.43:0/380714679 learned_addr learned my addr 172.21.15.43:0/380714679 (peer_addr_for_me v2:172.21.15.43:0/0) 2023-08-09T13:51:54.925 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:54.923+0000 7f993dcd5640 1 --2- 172.21.15.43:0/380714679 >> v2:172.21.15.203:3300/0 conn(0x7f993815a6a0 0x7f99380713b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:51:54.925 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:54.923+0000 7f993d4d4640 1 -- 172.21.15.43:0/380714679 >> v2:172.21.15.203:3300/0 conn(0x7f993815a6a0 msgr2=0x7f99380713b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-09T13:51:54.925 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:54.923+0000 7f993d4d4640 1 --2- 172.21.15.43:0/380714679 >> v2:172.21.15.203:3300/0 conn(0x7f993815a6a0 0x7f99380713b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:51:54.925 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:54.923+0000 7f993d4d4640 1 -- 172.21.15.43:0/380714679 >> v1:172.21.15.203:6789/0 conn(0x7f99380c94f0 legacy=0x7f993815ad20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-09T13:51:54.926 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:54.923+0000 7f993d4d4640 1 -- 172.21.15.43:0/380714679 --> v2:172.21.15.43:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f99380c88c0 con 0x7f99380cbd90 2023-08-09T13:51:54.926 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:54.924+0000 7f993d4d4640 1 --2- 172.21.15.43:0/380714679 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f99380cbd90 0x7f99380c8fb0 secure :-1 s=READY pgs=1309 cs=0 l=1 rev1=1 crypto rx=0x7f993806d920 tx=0x7f992802b880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2233daea6c03ac05 server_cookie=0 in_seq=0 out_seq=0 2023-08-09T13:51:54.927 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:54.924+0000 7f993ccd3640 1 -- 172.21.15.43:0/380714679 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f99280317e0 con 0x7f99380cbd90 2023-08-09T13:51:54.927 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:54.924+0000 7f993ccd3640 1 -- 172.21.15.43:0/380714679 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9928031940 con 0x7f99380cbd90 2023-08-09T13:51:54.927 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:54.924+0000 7f993ccd3640 1 -- 172.21.15.43:0/380714679 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9928031cd0 con 0x7f99380cbd90 2023-08-09T13:51:54.927 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:54.925+0000 7f993ecd7640 1 -- 172.21.15.43:0/380714679 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f99380cbd90 msgr2=0x7f99380c8fb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:51:54.927 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:54.925+0000 7f993ecd7640 1 --2- 172.21.15.43:0/380714679 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f99380cbd90 0x7f99380c8fb0 secure :-1 s=READY pgs=1309 cs=0 l=1 rev1=1 crypto rx=0x7f993806d920 tx=0x7f992802b880 comp rx=0 tx=0).stop 2023-08-09T13:51:54.927 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:54.925+0000 7f993ecd7640 1 -- 172.21.15.43:0/380714679 shutdown_connections 2023-08-09T13:51:54.928 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:54.925+0000 7f993ecd7640 1 --2- 172.21.15.43:0/380714679 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f99380cbd90 0x7f99380c8fb0 unknown :-1 s=CLOSED pgs=1309 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:51:54.928 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:54.925+0000 7f993ecd7640 1 --2- 172.21.15.43:0/380714679 >> v2:172.21.15.203:3300/0 conn(0x7f993815a6a0 0x7f99380713b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:51:54.928 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:54.925+0000 7f993ecd7640 1 -- 172.21.15.43:0/380714679 >> 172.21.15.43:0/380714679 conn(0x7f99380cde90 msgr2=0x7f99380ce2a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-09T13:51:54.928 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:54.925+0000 7f993ecd7640 1 -- 172.21.15.43:0/380714679 shutdown_connections 2023-08-09T13:51:54.928 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:54.925+0000 7f993ecd7640 1 -- 172.21.15.43:0/380714679 wait complete. 2023-08-09T13:51:54.929 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:54.928+0000 7f993ecd7640 1 Processor -- start 2023-08-09T13:51:54.929 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:54.928+0000 7f993ecd7640 1 -- start start 2023-08-09T13:51:54.930 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:54.929+0000 7f993ecd7640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f99380c94f0 0x7f993806bba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:51:54.931 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:54.929+0000 7f993ecd7640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f99380cbd90 0x7f99380b0ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:51:54.931 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:54.929+0000 7f993ecd7640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f993815a6a0 0x7f99380b0ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:51:54.931 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:54.929+0000 7f993ecd7640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f99380c19c0 con 0x7f99380c94f0 2023-08-09T13:51:54.931 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:54.929+0000 7f993ecd7640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9938156cf0 con 0x7f993815a6a0 2023-08-09T13:51:54.931 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:54.929+0000 7f993ecd7640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f993806c0e0 con 0x7f99380cbd90 2023-08-09T13:51:54.932 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:54.930+0000 7f993dcd5640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f99380c94f0 0x7f993806bba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:51:54.933 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:54.930+0000 7f993dcd5640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f99380c94f0 0x7f993806bba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:48886/0 (socket says 172.21.15.43:48886) 2023-08-09T13:51:54.933 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:54.930+0000 7f993dcd5640 1 -- 172.21.15.43:0/2387355735 learned_addr learned my addr 172.21.15.43:0/2387355735 (peer_addr_for_me v2:172.21.15.43:0/0) 2023-08-09T13:51:54.933 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:54.930+0000 7f993e4d6640 1 --2- 172.21.15.43:0/2387355735 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f993815a6a0 0x7f99380b0ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:51:54.933 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:54.930+0000 7f993d4d4640 1 --2- 172.21.15.43:0/2387355735 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f99380cbd90 0x7f99380b0ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:51:54.933 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:54.930+0000 7f993dcd5640 1 -- 172.21.15.43:0/2387355735 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f99380cbd90 msgr2=0x7f99380b0ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:51:54.933 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:54.930+0000 7f993dcd5640 1 --2- 172.21.15.43:0/2387355735 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f99380cbd90 0x7f99380b0ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:51:54.933 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:54.930+0000 7f993dcd5640 1 -- 172.21.15.43:0/2387355735 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f993815a6a0 msgr2=0x7f99380b0ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:51:54.934 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:54.930+0000 7f993dcd5640 1 --2- 172.21.15.43:0/2387355735 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f993815a6a0 0x7f99380b0ff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:51:54.934 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:54.930+0000 7f993dcd5640 1 -- 172.21.15.43:0/2387355735 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f99280094d0 con 0x7f99380c94f0 2023-08-09T13:51:54.934 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:54.930+0000 7f993dcd5640 1 --2- 172.21.15.43:0/2387355735 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f99380c94f0 0x7f993806bba0 secure :-1 s=READY pgs=1310 cs=0 l=1 rev1=1 crypto rx=0x7f993400d800 tx=0x7f993400dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-09T13:51:54.934 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:54.931+0000 7f992e7fc640 1 -- 172.21.15.43:0/2387355735 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f993400b6c0 con 0x7f99380c94f0 2023-08-09T13:51:54.934 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:54.931+0000 7f992e7fc640 1 -- 172.21.15.43:0/2387355735 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f993400b820 con 0x7f99380c94f0 2023-08-09T13:51:54.934 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:54.931+0000 7f992e7fc640 1 -- 172.21.15.43:0/2387355735 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f993400bbb0 con 0x7f99380c94f0 2023-08-09T13:51:54.934 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:54.931+0000 7f993ecd7640 1 -- 172.21.15.43:0/2387355735 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f993806c3b0 con 0x7f99380c94f0 2023-08-09T13:51:54.934 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:54.931+0000 7f993ecd7640 1 -- 172.21.15.43:0/2387355735 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f993806c970 con 0x7f99380c94f0 2023-08-09T13:51:54.935 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:54.932+0000 7f993ecd7640 1 -- 172.21.15.43:0/2387355735 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f99380c9fe0 con 0x7f99380c94f0 2023-08-09T13:51:54.935 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:54.933+0000 7f992e7fc640 1 -- 172.21.15.43:0/2387355735 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 45) v1 ==== 88443+0+0 (secure 0 0 0) 0x7f993400bd10 con 0x7f99380c94f0 2023-08-09T13:51:54.936 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:54.934+0000 7f992e7fc640 1 --2- 172.21.15.43:0/2387355735 >> [v2:172.21.15.203:6832/1317108840,v1:172.21.15.203:6833/1317108840] conn(0x7f991c06a170 0x7f991c06c630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:51:54.936 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:54.934+0000 7f992e7fc640 1 -- 172.21.15.43:0/2387355735 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(112..112 src has 1..112) v4 ==== 5574+0+0 (secure 0 0 0) 0x7f9934088b50 con 0x7f99380c94f0 2023-08-09T13:51:54.936 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:54.935+0000 7f993d4d4640 1 --2- 172.21.15.43:0/2387355735 >> [v2:172.21.15.203:6832/1317108840,v1:172.21.15.203:6833/1317108840] conn(0x7f991c06a170 0x7f991c06c630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:51:54.937 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:54.936+0000 7f993d4d4640 1 --2- 172.21.15.43:0/2387355735 >> [v2:172.21.15.203:6832/1317108840,v1:172.21.15.203:6833/1317108840] conn(0x7f991c06a170 0x7f991c06c630 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7f9928036000 tx=0x7f992803d040 comp rx=0 tx=0).ready entity=mgr.6619 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-09T13:51:54.940 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:54.938+0000 7f992e7fc640 1 -- 172.21.15.43:0/2387355735 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178066 (secure 0 0 0) 0x7f9934054270 con 0x7f99380c94f0 2023-08-09T13:51:55.214 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:55.212+0000 7f993ecd7640 1 -- 172.21.15.43:0/2387355735 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "auth ls", "format": "json-pretty"} v 0) v1 -- 0x7f99380cc220 con 0x7f99380c94f0 2023-08-09T13:51:55.214 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:55.213+0000 7f992e7fc640 1 -- 172.21.15.43:0/2387355735 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "auth ls", "format": "json-pretty"}]=0 v16) v1 ==== 80+0+5304 (secure 0 0 0) 0x7f9934057890 con 0x7f99380c94f0 2023-08-09T13:51:55.215 INFO:teuthology.orchestra.run.smithi043.stdout: 2023-08-09T13:51:55.215 INFO:teuthology.orchestra.run.smithi043.stdout:{ 2023-08-09T13:51:55.215 INFO:teuthology.orchestra.run.smithi043.stdout: "auth_dump": [ 2023-08-09T13:51:55.215 INFO:teuthology.orchestra.run.smithi043.stdout: { 2023-08-09T13:51:55.216 INFO:teuthology.orchestra.run.smithi043.stdout: "entity": "mds.a", 2023-08-09T13:51:55.216 INFO:teuthology.orchestra.run.smithi043.stdout: "key": "AQCOldNkPu6GCRAAZecwctvMji0Zubb60KMlyQ==", 2023-08-09T13:51:55.216 INFO:teuthology.orchestra.run.smithi043.stdout: "caps": { 2023-08-09T13:51:55.216 INFO:teuthology.orchestra.run.smithi043.stdout: "mds": "allow", 2023-08-09T13:51:55.217 INFO:teuthology.orchestra.run.smithi043.stdout: "mgr": "allow *", 2023-08-09T13:51:55.217 INFO:teuthology.orchestra.run.smithi043.stdout: "mon": "allow *", 2023-08-09T13:51:55.217 INFO:teuthology.orchestra.run.smithi043.stdout: "osd": "allow *" 2023-08-09T13:51:55.217 INFO:teuthology.orchestra.run.smithi043.stdout: } 2023-08-09T13:51:55.217 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2023-08-09T13:51:55.217 INFO:teuthology.orchestra.run.smithi043.stdout: { 2023-08-09T13:51:55.217 INFO:teuthology.orchestra.run.smithi043.stdout: "entity": "mds.b", 2023-08-09T13:51:55.217 INFO:teuthology.orchestra.run.smithi043.stdout: "key": "AQCOldNkt2+5KxAAHcckB+S5NZHsF5z22StXmw==", 2023-08-09T13:51:55.218 INFO:teuthology.orchestra.run.smithi043.stdout: "caps": { 2023-08-09T13:51:55.218 INFO:teuthology.orchestra.run.smithi043.stdout: "mds": "allow", 2023-08-09T13:51:55.218 INFO:teuthology.orchestra.run.smithi043.stdout: "mgr": "allow *", 2023-08-09T13:51:55.218 INFO:teuthology.orchestra.run.smithi043.stdout: "mon": "allow *", 2023-08-09T13:51:55.218 INFO:teuthology.orchestra.run.smithi043.stdout: "osd": "allow *" 2023-08-09T13:51:55.218 INFO:teuthology.orchestra.run.smithi043.stdout: } 2023-08-09T13:51:55.218 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2023-08-09T13:51:55.218 INFO:teuthology.orchestra.run.smithi043.stdout: { 2023-08-09T13:51:55.219 INFO:teuthology.orchestra.run.smithi043.stdout: "entity": "mds.c", 2023-08-09T13:51:55.219 INFO:teuthology.orchestra.run.smithi043.stdout: "key": "AQCOldNk5V46GhAAQBhkvfjVSJCSHLS/YkhgCA==", 2023-08-09T13:51:55.219 INFO:teuthology.orchestra.run.smithi043.stdout: "caps": { 2023-08-09T13:51:55.219 INFO:teuthology.orchestra.run.smithi043.stdout: "mds": "allow", 2023-08-09T13:51:55.219 INFO:teuthology.orchestra.run.smithi043.stdout: "mgr": "allow *", 2023-08-09T13:51:55.219 INFO:teuthology.orchestra.run.smithi043.stdout: "mon": "allow *", 2023-08-09T13:51:55.219 INFO:teuthology.orchestra.run.smithi043.stdout: "osd": "allow *" 2023-08-09T13:51:55.220 INFO:teuthology.orchestra.run.smithi043.stdout: } 2023-08-09T13:51:55.220 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2023-08-09T13:51:55.220 INFO:teuthology.orchestra.run.smithi043.stdout: { 2023-08-09T13:51:55.220 INFO:teuthology.orchestra.run.smithi043.stdout: "entity": "mds.d", 2023-08-09T13:51:55.220 INFO:teuthology.orchestra.run.smithi043.stdout: "key": "AQCPldNkNBoTARAAMVW+rSyXQKaau6I90fUX9Q==", 2023-08-09T13:51:55.220 INFO:teuthology.orchestra.run.smithi043.stdout: "caps": { 2023-08-09T13:51:55.220 INFO:teuthology.orchestra.run.smithi043.stdout: "mds": "allow", 2023-08-09T13:51:55.220 INFO:teuthology.orchestra.run.smithi043.stdout: "mgr": "allow *", 2023-08-09T13:51:55.221 INFO:teuthology.orchestra.run.smithi043.stdout: "mon": "allow *", 2023-08-09T13:51:55.221 INFO:teuthology.orchestra.run.smithi043.stdout: "osd": "allow *" 2023-08-09T13:51:55.221 INFO:teuthology.orchestra.run.smithi043.stdout: } 2023-08-09T13:51:55.221 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2023-08-09T13:51:55.221 INFO:teuthology.orchestra.run.smithi043.stdout: { 2023-08-09T13:51:55.221 INFO:teuthology.orchestra.run.smithi043.stdout: "entity": "osd.0", 2023-08-09T13:51:55.221 INFO:teuthology.orchestra.run.smithi043.stdout: "key": "AQCRldNkltPlMhAAAt3rFvraCtI0aSqGvt6UxA==", 2023-08-09T13:51:55.221 INFO:teuthology.orchestra.run.smithi043.stdout: "caps": { 2023-08-09T13:51:55.222 INFO:teuthology.orchestra.run.smithi043.stdout: "mgr": "allow profile osd", 2023-08-09T13:51:55.222 INFO:teuthology.orchestra.run.smithi043.stdout: "mon": "allow profile osd", 2023-08-09T13:51:55.222 INFO:teuthology.orchestra.run.smithi043.stdout: "osd": "allow *" 2023-08-09T13:51:55.222 INFO:teuthology.orchestra.run.smithi043.stdout: } 2023-08-09T13:51:55.222 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2023-08-09T13:51:55.222 INFO:teuthology.orchestra.run.smithi043.stdout: { 2023-08-09T13:51:55.222 INFO:teuthology.orchestra.run.smithi043.stdout: "entity": "osd.1", 2023-08-09T13:51:55.222 INFO:teuthology.orchestra.run.smithi043.stdout: "key": "AQCVldNkzpKYEhAA+wen0kV/vUDLQztq3vhfOg==", 2023-08-09T13:51:55.223 INFO:teuthology.orchestra.run.smithi043.stdout: "caps": { 2023-08-09T13:51:55.223 INFO:teuthology.orchestra.run.smithi043.stdout: "mgr": "allow profile osd", 2023-08-09T13:51:55.223 INFO:teuthology.orchestra.run.smithi043.stdout: "mon": "allow profile osd", 2023-08-09T13:51:55.223 INFO:teuthology.orchestra.run.smithi043.stdout: "osd": "allow *" 2023-08-09T13:51:55.223 INFO:teuthology.orchestra.run.smithi043.stdout: } 2023-08-09T13:51:55.223 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2023-08-09T13:51:55.223 INFO:teuthology.orchestra.run.smithi043.stdout: { 2023-08-09T13:51:55.223 INFO:teuthology.orchestra.run.smithi043.stdout: "entity": "osd.2", 2023-08-09T13:51:55.224 INFO:teuthology.orchestra.run.smithi043.stdout: "key": "AQCYldNkYbwNHhAAQY8g9dldb1V9M2MM2SHKDg==", 2023-08-09T13:51:55.224 INFO:teuthology.orchestra.run.smithi043.stdout: "caps": { 2023-08-09T13:51:55.224 INFO:teuthology.orchestra.run.smithi043.stdout: "mgr": "allow profile osd", 2023-08-09T13:51:55.224 INFO:teuthology.orchestra.run.smithi043.stdout: "mon": "allow profile osd", 2023-08-09T13:51:55.224 INFO:teuthology.orchestra.run.smithi043.stdout: "osd": "allow *" 2023-08-09T13:51:55.224 INFO:teuthology.orchestra.run.smithi043.stdout: } 2023-08-09T13:51:55.224 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2023-08-09T13:51:55.225 INFO:teuthology.orchestra.run.smithi043.stdout: { 2023-08-09T13:51:55.225 INFO:teuthology.orchestra.run.smithi043.stdout: "entity": "osd.3", 2023-08-09T13:51:55.225 INFO:teuthology.orchestra.run.smithi043.stdout: "key": "AQCbldNkeyW6KRAAsd+kgqik2aD4U7PXwccWtA==", 2023-08-09T13:51:55.225 INFO:teuthology.orchestra.run.smithi043.stdout: "caps": { 2023-08-09T13:51:55.225 INFO:teuthology.orchestra.run.smithi043.stdout: "mgr": "allow profile osd", 2023-08-09T13:51:55.225 INFO:teuthology.orchestra.run.smithi043.stdout: "mon": "allow profile osd", 2023-08-09T13:51:55.225 INFO:teuthology.orchestra.run.smithi043.stdout: "osd": "allow *" 2023-08-09T13:51:55.225 INFO:teuthology.orchestra.run.smithi043.stdout: } 2023-08-09T13:51:55.226 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2023-08-09T13:51:55.226 INFO:teuthology.orchestra.run.smithi043.stdout: { 2023-08-09T13:51:55.226 INFO:teuthology.orchestra.run.smithi043.stdout: "entity": "osd.4", 2023-08-09T13:51:55.226 INFO:teuthology.orchestra.run.smithi043.stdout: "key": "AQCildNk80UkOBAAtX+4JGpOVcfZEHetJoeZKw==", 2023-08-09T13:51:55.226 INFO:teuthology.orchestra.run.smithi043.stdout: "caps": { 2023-08-09T13:51:55.226 INFO:teuthology.orchestra.run.smithi043.stdout: "mgr": "allow profile osd", 2023-08-09T13:51:55.226 INFO:teuthology.orchestra.run.smithi043.stdout: "mon": "allow profile osd", 2023-08-09T13:51:55.226 INFO:teuthology.orchestra.run.smithi043.stdout: "osd": "allow *" 2023-08-09T13:51:55.227 INFO:teuthology.orchestra.run.smithi043.stdout: } 2023-08-09T13:51:55.227 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2023-08-09T13:51:55.227 INFO:teuthology.orchestra.run.smithi043.stdout: { 2023-08-09T13:51:55.227 INFO:teuthology.orchestra.run.smithi043.stdout: "entity": "osd.5", 2023-08-09T13:51:55.227 INFO:teuthology.orchestra.run.smithi043.stdout: "key": "AQCmldNkCA9SBxAAZj4So6MRj2Eq1I9H9JzLDA==", 2023-08-09T13:51:55.227 INFO:teuthology.orchestra.run.smithi043.stdout: "caps": { 2023-08-09T13:51:55.227 INFO:teuthology.orchestra.run.smithi043.stdout: "mgr": "allow profile osd", 2023-08-09T13:51:55.227 INFO:teuthology.orchestra.run.smithi043.stdout: "mon": "allow profile osd", 2023-08-09T13:51:55.228 INFO:teuthology.orchestra.run.smithi043.stdout: "osd": "allow *" 2023-08-09T13:51:55.228 INFO:teuthology.orchestra.run.smithi043.stdout: } 2023-08-09T13:51:55.228 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2023-08-09T13:51:55.228 INFO:teuthology.orchestra.run.smithi043.stdout: { 2023-08-09T13:51:55.228 INFO:teuthology.orchestra.run.smithi043.stdout: "entity": "osd.6", 2023-08-09T13:51:55.228 INFO:teuthology.orchestra.run.smithi043.stdout: "key": "AQCpldNkaOwAExAAraV1tUwr9uU6GheZ+hvdqQ==", 2023-08-09T13:51:55.228 INFO:teuthology.orchestra.run.smithi043.stdout: "caps": { 2023-08-09T13:51:55.228 INFO:teuthology.orchestra.run.smithi043.stdout: "mgr": "allow profile osd", 2023-08-09T13:51:55.229 INFO:teuthology.orchestra.run.smithi043.stdout: "mon": "allow profile osd", 2023-08-09T13:51:55.229 INFO:teuthology.orchestra.run.smithi043.stdout: "osd": "allow *" 2023-08-09T13:51:55.229 INFO:teuthology.orchestra.run.smithi043.stdout: } 2023-08-09T13:51:55.229 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2023-08-09T13:51:55.229 INFO:teuthology.orchestra.run.smithi043.stdout: { 2023-08-09T13:51:55.229 INFO:teuthology.orchestra.run.smithi043.stdout: "entity": "osd.7", 2023-08-09T13:51:55.229 INFO:teuthology.orchestra.run.smithi043.stdout: "key": "AQCsldNk2/GwLBAAP42EXPivzLk44X8aMEVVCg==", 2023-08-09T13:51:55.229 INFO:teuthology.orchestra.run.smithi043.stdout: "caps": { 2023-08-09T13:51:55.230 INFO:teuthology.orchestra.run.smithi043.stdout: "mgr": "allow profile osd", 2023-08-09T13:51:55.230 INFO:teuthology.orchestra.run.smithi043.stdout: "mon": "allow profile osd", 2023-08-09T13:51:55.230 INFO:teuthology.orchestra.run.smithi043.stdout: "osd": "allow *" 2023-08-09T13:51:55.230 INFO:teuthology.orchestra.run.smithi043.stdout: } 2023-08-09T13:51:55.230 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2023-08-09T13:51:55.230 INFO:teuthology.orchestra.run.smithi043.stdout: { 2023-08-09T13:51:55.230 INFO:teuthology.orchestra.run.smithi043.stdout: "entity": "client.0", 2023-08-09T13:51:55.230 INFO:teuthology.orchestra.run.smithi043.stdout: "key": "AQCPldNkg8zrDxAAM/hnJaE6S9HU/fRwQeBV5w==", 2023-08-09T13:51:55.231 INFO:teuthology.orchestra.run.smithi043.stdout: "caps": { 2023-08-09T13:51:55.231 INFO:teuthology.orchestra.run.smithi043.stdout: "mds": "allow", 2023-08-09T13:51:55.231 INFO:teuthology.orchestra.run.smithi043.stdout: "mon": "allow r", 2023-08-09T13:51:55.231 INFO:teuthology.orchestra.run.smithi043.stdout: "osd": "allow rw tag cephfs data=cephfs" 2023-08-09T13:51:55.231 INFO:teuthology.orchestra.run.smithi043.stdout: } 2023-08-09T13:51:55.231 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2023-08-09T13:51:55.231 INFO:teuthology.orchestra.run.smithi043.stdout: { 2023-08-09T13:51:55.231 INFO:teuthology.orchestra.run.smithi043.stdout: "entity": "client.1", 2023-08-09T13:51:55.232 INFO:teuthology.orchestra.run.smithi043.stdout: "key": "AQCPldNklfFxJxAA19+L78rW6jU1uOyjey6WNA==", 2023-08-09T13:51:55.232 INFO:teuthology.orchestra.run.smithi043.stdout: "caps": { 2023-08-09T13:51:55.232 INFO:teuthology.orchestra.run.smithi043.stdout: "mds": "allow", 2023-08-09T13:51:55.232 INFO:teuthology.orchestra.run.smithi043.stdout: "mgr": "allow r", 2023-08-09T13:51:55.232 INFO:teuthology.orchestra.run.smithi043.stdout: "mon": "allow rw", 2023-08-09T13:51:55.232 INFO:teuthology.orchestra.run.smithi043.stdout: "osd": "allow rwx" 2023-08-09T13:51:55.232 INFO:teuthology.orchestra.run.smithi043.stdout: } 2023-08-09T13:51:55.232 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2023-08-09T13:51:55.233 INFO:teuthology.orchestra.run.smithi043.stdout: { 2023-08-09T13:51:55.233 INFO:teuthology.orchestra.run.smithi043.stdout: "entity": "client.2", 2023-08-09T13:51:55.233 INFO:teuthology.orchestra.run.smithi043.stdout: "key": "AQCPldNkC2lgGxAAZ8qZboOCUaQlcpkPL1ozHA==", 2023-08-09T13:51:55.233 INFO:teuthology.orchestra.run.smithi043.stdout: "caps": { 2023-08-09T13:51:55.233 INFO:teuthology.orchestra.run.smithi043.stdout: "mds": "allow", 2023-08-09T13:51:55.233 INFO:teuthology.orchestra.run.smithi043.stdout: "mgr": "allow r", 2023-08-09T13:51:55.233 INFO:teuthology.orchestra.run.smithi043.stdout: "mon": "allow rw", 2023-08-09T13:51:55.233 INFO:teuthology.orchestra.run.smithi043.stdout: "osd": "allow rwx" 2023-08-09T13:51:55.234 INFO:teuthology.orchestra.run.smithi043.stdout: } 2023-08-09T13:51:55.234 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2023-08-09T13:51:55.234 INFO:teuthology.orchestra.run.smithi043.stdout: { 2023-08-09T13:51:55.234 INFO:teuthology.orchestra.run.smithi043.stdout: "entity": "client.3", 2023-08-09T13:51:55.234 INFO:teuthology.orchestra.run.smithi043.stdout: "key": "AQCPldNkl+rXMhAA9rf3KdnSYbvngqmLYnIGGA==", 2023-08-09T13:51:55.234 INFO:teuthology.orchestra.run.smithi043.stdout: "caps": { 2023-08-09T13:51:55.234 INFO:teuthology.orchestra.run.smithi043.stdout: "mds": "allow", 2023-08-09T13:51:55.234 INFO:teuthology.orchestra.run.smithi043.stdout: "mgr": "allow r", 2023-08-09T13:51:55.235 INFO:teuthology.orchestra.run.smithi043.stdout: "mon": "allow rw", 2023-08-09T13:51:55.235 INFO:teuthology.orchestra.run.smithi043.stdout: "osd": "allow rwx" 2023-08-09T13:51:55.235 INFO:teuthology.orchestra.run.smithi043.stdout: } 2023-08-09T13:51:55.235 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2023-08-09T13:51:55.235 INFO:teuthology.orchestra.run.smithi043.stdout: { 2023-08-09T13:51:55.235 INFO:teuthology.orchestra.run.smithi043.stdout: "entity": "client.admin", 2023-08-09T13:51:55.235 INFO:teuthology.orchestra.run.smithi043.stdout: "key": "AQCMldNkUae6NRAA2FBTr0QPBdKNDoe1WK5zyg==", 2023-08-09T13:51:55.236 INFO:teuthology.orchestra.run.smithi043.stdout: "caps": { 2023-08-09T13:51:55.236 INFO:teuthology.orchestra.run.smithi043.stdout: "mds": "allow *", 2023-08-09T13:51:55.236 INFO:teuthology.orchestra.run.smithi043.stdout: "mgr": "allow *", 2023-08-09T13:51:55.236 INFO:teuthology.orchestra.run.smithi043.stdout: "mon": "allow *", 2023-08-09T13:51:55.236 INFO:teuthology.orchestra.run.smithi043.stdout: "osd": "allow *" 2023-08-09T13:51:55.236 INFO:teuthology.orchestra.run.smithi043.stdout: } 2023-08-09T13:51:55.236 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2023-08-09T13:51:55.236 INFO:teuthology.orchestra.run.smithi043.stdout: { 2023-08-09T13:51:55.237 INFO:teuthology.orchestra.run.smithi043.stdout: "entity": "mgr.x", 2023-08-09T13:51:55.237 INFO:teuthology.orchestra.run.smithi043.stdout: "key": "AQCNldNk2ufSNxAAU7msRkwLaoLZHTIsk0ck/g==", 2023-08-09T13:51:55.237 INFO:teuthology.orchestra.run.smithi043.stdout: "caps": { 2023-08-09T13:51:55.237 INFO:teuthology.orchestra.run.smithi043.stdout: "mds": "allow *", 2023-08-09T13:51:55.237 INFO:teuthology.orchestra.run.smithi043.stdout: "mon": "allow profile mgr", 2023-08-09T13:51:55.237 INFO:teuthology.orchestra.run.smithi043.stdout: "osd": "allow *" 2023-08-09T13:51:55.237 INFO:teuthology.orchestra.run.smithi043.stdout: } 2023-08-09T13:51:55.237 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2023-08-09T13:51:55.238 INFO:teuthology.orchestra.run.smithi043.stdout: { 2023-08-09T13:51:55.238 INFO:teuthology.orchestra.run.smithi043.stdout: "entity": "mgr.y", 2023-08-09T13:51:55.238 INFO:teuthology.orchestra.run.smithi043.stdout: "key": "AQCNldNk2YgzLBAAA11Q8uAMEgE/bUni+tXIKA==", 2023-08-09T13:51:55.238 INFO:teuthology.orchestra.run.smithi043.stdout: "caps": { 2023-08-09T13:51:55.238 INFO:teuthology.orchestra.run.smithi043.stdout: "mds": "allow *", 2023-08-09T13:51:55.238 INFO:teuthology.orchestra.run.smithi043.stdout: "mon": "allow profile mgr", 2023-08-09T13:51:55.238 INFO:teuthology.orchestra.run.smithi043.stdout: "osd": "allow *" 2023-08-09T13:51:55.238 INFO:teuthology.orchestra.run.smithi043.stdout: } 2023-08-09T13:51:55.239 INFO:teuthology.orchestra.run.smithi043.stdout: } 2023-08-09T13:51:55.239 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2023-08-09T13:51:55.239 INFO:teuthology.orchestra.run.smithi043.stdout:} 2023-08-09T13:51:55.240 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:55.216+0000 7f993ecd7640 1 -- 172.21.15.43:0/2387355735 >> [v2:172.21.15.203:6832/1317108840,v1:172.21.15.203:6833/1317108840] conn(0x7f991c06a170 msgr2=0x7f991c06c630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:51:55.240 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:55.216+0000 7f993ecd7640 1 --2- 172.21.15.43:0/2387355735 >> [v2:172.21.15.203:6832/1317108840,v1:172.21.15.203:6833/1317108840] conn(0x7f991c06a170 0x7f991c06c630 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7f9928036000 tx=0x7f992803d040 comp rx=0 tx=0).stop 2023-08-09T13:51:55.240 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:55.216+0000 7f993ecd7640 1 -- 172.21.15.43:0/2387355735 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f99380c94f0 msgr2=0x7f993806bba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:51:55.241 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:55.216+0000 7f993ecd7640 1 --2- 172.21.15.43:0/2387355735 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f99380c94f0 0x7f993806bba0 secure :-1 s=READY pgs=1310 cs=0 l=1 rev1=1 crypto rx=0x7f993400d800 tx=0x7f993400dcc0 comp rx=0 tx=0).stop 2023-08-09T13:51:55.241 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:55.216+0000 7f993ecd7640 1 -- 172.21.15.43:0/2387355735 shutdown_connections 2023-08-09T13:51:55.241 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:55.216+0000 7f993ecd7640 1 --2- 172.21.15.43:0/2387355735 >> [v2:172.21.15.203:6832/1317108840,v1:172.21.15.203:6833/1317108840] conn(0x7f991c06a170 0x7f991c06c630 unknown :-1 s=CLOSED pgs=48 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:51:55.241 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:55.216+0000 7f993ecd7640 1 --2- 172.21.15.43:0/2387355735 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f993815a6a0 0x7f99380b0ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:51:55.241 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:55.216+0000 7f993ecd7640 1 --2- 172.21.15.43:0/2387355735 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f99380cbd90 0x7f99380b0ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:51:55.241 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:55.216+0000 7f993ecd7640 1 --2- 172.21.15.43:0/2387355735 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f99380c94f0 0x7f993806bba0 unknown :-1 s=CLOSED pgs=1310 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:51:55.241 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:55.216+0000 7f993ecd7640 1 -- 172.21.15.43:0/2387355735 >> 172.21.15.43:0/2387355735 conn(0x7f99380cde90 msgr2=0x7f99380af280 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-09T13:51:55.242 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:55.216+0000 7f993ecd7640 1 -- 172.21.15.43:0/2387355735 shutdown_connections 2023-08-09T13:51:55.242 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:55.216+0000 7f993ecd7640 1 -- 172.21.15.43:0/2387355735 wait complete. 2023-08-09T13:51:55.242 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-08-09T13:51:55.476 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:55.474+0000 7fc3a5569640 1 Processor -- start 2023-08-09T13:51:55.476 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:55.475+0000 7fc3a5569640 1 -- start start 2023-08-09T13:51:55.478 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:55.476+0000 7fc3a5569640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fc3a015f7b0 0x7fc3a00b4760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:51:55.478 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:55.476+0000 7fc3a5569640 1 -- --> v1:172.21.15.43:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc3a00b7bf0 con 0x7fc3a00cd6b0 2023-08-09T13:51:55.479 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:55.476+0000 7fc3a5569640 1 -- --> v1:172.21.15.203:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc3a00b7ed0 con 0x7fc3a00b1050 2023-08-09T13:51:55.479 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:55.476+0000 7fc3a5569640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc3a00d0370 con 0x7fc3a015f7b0 2023-08-09T13:51:55.479 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:55.477+0000 7fc3a4d68640 1 --1- >> v1:172.21.15.43:6789/0 conn(0x7fc3a00cd6b0 0x7fc3a00d0e50 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.43:6789/0 says I am v1:172.21.15.43:43688/0 (socket says 172.21.15.43:43688) 2023-08-09T13:51:55.479 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:55.477+0000 7fc3a4d68640 1 -- 172.21.15.43:0/3460389547 learned_addr learned my addr 172.21.15.43:0/3460389547 (peer_addr_for_me v1:172.21.15.43:0/0) 2023-08-09T13:51:55.485 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:55.477+0000 7fc39ffff640 1 --2- 172.21.15.43:0/3460389547 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fc3a015f7b0 0x7fc3a00b4760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:51:55.485 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:55.477+0000 7fc39f7fe640 1 -- 172.21.15.43:0/3460389547 <== mon.0 v1:172.21.15.43:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 111710613 0 0) 0x7fc3a00b7bf0 con 0x7fc3a00cd6b0 2023-08-09T13:51:55.485 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:55.477+0000 7fc39f7fe640 1 -- 172.21.15.43:0/3460389547 --> v1:172.21.15.43:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fc38c003520 con 0x7fc3a00cd6b0 2023-08-09T13:51:55.485 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:55.478+0000 7fc39f7fe640 1 -- 172.21.15.43:0/3460389547 <== mon.0 v1:172.21.15.43:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2253424131 0 0) 0x7fc38c003520 con 0x7fc3a00cd6b0 2023-08-09T13:51:55.485 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:55.478+0000 7fc39f7fe640 1 -- 172.21.15.43:0/3460389547 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fc3a015f7b0 msgr2=0x7fc3a00b4760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-09T13:51:55.486 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:55.478+0000 7fc39f7fe640 1 --2- 172.21.15.43:0/3460389547 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fc3a015f7b0 0x7fc3a00b4760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:51:55.486 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:55.478+0000 7fc39f7fe640 1 -- 172.21.15.43:0/3460389547 >> v1:172.21.15.203:6789/0 conn(0x7fc3a00b1050 legacy=0x7fc3a00ccfa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:51:55.486 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:55.478+0000 7fc39f7fe640 1 -- 172.21.15.43:0/3460389547 --> v1:172.21.15.43:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc3a00d0650 con 0x7fc3a00cd6b0 2023-08-09T13:51:55.486 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:55.478+0000 7fc39f7fe640 1 -- 172.21.15.43:0/3460389547 <== mon.0 v1:172.21.15.43:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1285150740 0 0) 0x7fc3980019f0 con 0x7fc3a00cd6b0 2023-08-09T13:51:55.486 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:55.478+0000 7fc39f7fe640 1 -- 172.21.15.43:0/3460389547 >> v1:172.21.15.43:6789/0 conn(0x7fc3a00cd6b0 legacy=0x7fc3a00d0e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:51:55.486 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:55.478+0000 7fc39f7fe640 1 --2- 172.21.15.43:0/3460389547 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fc38c0039a0 0x7fc38c003dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:51:55.486 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:55.478+0000 7fc39f7fe640 1 --2- 172.21.15.43:0/3460389547 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc38c0047c0 0x7fc38c004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:51:55.487 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:55.478+0000 7fc39f7fe640 1 --2- 172.21.15.43:0/3460389547 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fc3a015f7b0 0x7fc38c005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:51:55.487 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:55.478+0000 7fc39f7fe640 1 -- 172.21.15.43:0/3460389547 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc3a00d0370 con 0x7fc38c0047c0 2023-08-09T13:51:55.487 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:55.478+0000 7fc39f7fe640 1 -- 172.21.15.43:0/3460389547 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc38c00ba30 con 0x7fc38c0039a0 2023-08-09T13:51:55.487 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:55.478+0000 7fc39f7fe640 1 -- 172.21.15.43:0/3460389547 --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc38c00bbf0 con 0x7fc3a015f7b0 2023-08-09T13:51:55.487 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:55.478+0000 7fc397fff640 1 --2- 172.21.15.43:0/3460389547 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc38c0047c0 0x7fc38c004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:51:55.488 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:55.478+0000 7fc39ffff640 1 --2- 172.21.15.43:0/3460389547 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fc38c0039a0 0x7fc38c003dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:51:55.488 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:55.478+0000 7fc3a4d68640 1 --2- 172.21.15.43:0/3460389547 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fc3a015f7b0 0x7fc38c005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:51:55.488 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:55.479+0000 7fc397fff640 1 -- 172.21.15.43:0/3460389547 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fc3a015f7b0 msgr2=0x7fc38c005180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-09T13:51:55.489 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:55.479+0000 7fc397fff640 1 --2- 172.21.15.43:0/3460389547 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fc3a015f7b0 0x7fc38c005180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:51:55.489 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:55.479+0000 7fc397fff640 1 -- 172.21.15.43:0/3460389547 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fc38c0039a0 msgr2=0x7fc38c003dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-09T13:51:55.489 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:55.479+0000 7fc397fff640 1 --2- 172.21.15.43:0/3460389547 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fc38c0039a0 0x7fc38c003dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:51:55.489 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:55.479+0000 7fc397fff640 1 -- 172.21.15.43:0/3460389547 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc38c003520 con 0x7fc38c0047c0 2023-08-09T13:51:55.489 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:55.479+0000 7fc3a4d68640 1 --2- 172.21.15.43:0/3460389547 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fc3a015f7b0 0x7fc38c005180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-08-09T13:51:55.489 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:55.480+0000 7fc397fff640 1 --2- 172.21.15.43:0/3460389547 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc38c0047c0 0x7fc38c004c40 secure :-1 s=READY pgs=1312 cs=0 l=1 rev1=1 crypto rx=0x7fc388001720 tx=0x7fc38c0057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=97402f3f90bd8d8b server_cookie=0 in_seq=0 out_seq=0 2023-08-09T13:51:55.490 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:55.480+0000 7fc39f7fe640 1 -- 172.21.15.43:0/3460389547 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc3880052d0 con 0x7fc38c0047c0 2023-08-09T13:51:55.490 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:55.480+0000 7fc39f7fe640 1 -- 172.21.15.43:0/3460389547 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc38802e050 con 0x7fc38c0047c0 2023-08-09T13:51:55.490 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:55.480+0000 7fc39f7fe640 1 -- 172.21.15.43:0/3460389547 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc3880035f0 con 0x7fc38c0047c0 2023-08-09T13:51:55.490 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:55.481+0000 7fc3a5569640 1 -- 172.21.15.43:0/3460389547 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc38c0047c0 msgr2=0x7fc38c004c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:51:55.490 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:55.481+0000 7fc3a5569640 1 --2- 172.21.15.43:0/3460389547 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc38c0047c0 0x7fc38c004c40 secure :-1 s=READY pgs=1312 cs=0 l=1 rev1=1 crypto rx=0x7fc388001720 tx=0x7fc38c0057f0 comp rx=0 tx=0).stop 2023-08-09T13:51:55.491 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:55.481+0000 7fc3a4d68640 1 -- 172.21.15.43:0/3460389547 reap_dead start 2023-08-09T13:51:55.491 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:55.481+0000 7fc3a5569640 1 -- 172.21.15.43:0/3460389547 shutdown_connections 2023-08-09T13:51:55.491 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:55.481+0000 7fc3a5569640 1 -- 172.21.15.43:0/3460389547 >> 172.21.15.43:0/3460389547 conn(0x7fc3a006a2b0 msgr2=0x7fc3a006a690 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-09T13:51:55.491 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:55.481+0000 7fc3a5569640 1 -- 172.21.15.43:0/3460389547 shutdown_connections 2023-08-09T13:51:55.491 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:55.481+0000 7fc3a5569640 1 -- 172.21.15.43:0/3460389547 wait complete. 2023-08-09T13:51:55.491 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:55.483+0000 7fc3a5569640 1 Processor -- start 2023-08-09T13:51:55.491 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:55.483+0000 7fc3a5569640 1 -- start start 2023-08-09T13:51:55.493 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:55.484+0000 7fc3a5569640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fc38c0047c0 0x7fc3a0155310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:51:55.494 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:55.484+0000 7fc3a5569640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc3a01ef6f0 0x7fc3a01f2950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:51:55.494 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:55.484+0000 7fc3a5569640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fc3a01568d0 0x7fc3a00cc850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:51:55.494 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:55.484+0000 7fc3a5569640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc3a0153ca0 con 0x7fc3a01ef6f0 2023-08-09T13:51:55.494 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:55.484+0000 7fc3a5569640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc3a01f2e90 con 0x7fc38c0047c0 2023-08-09T13:51:55.494 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:55.484+0000 7fc3a5569640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc3a01559f0 con 0x7fc3a01568d0 2023-08-09T13:51:55.494 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:55.484+0000 7fc397fff640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc3a01ef6f0 0x7fc3a01f2950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:51:55.495 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:55.484+0000 7fc397fff640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc3a01ef6f0 0x7fc3a01f2950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:32962/0 (socket says 172.21.15.43:32962) 2023-08-09T13:51:55.495 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:55.484+0000 7fc397fff640 1 -- 172.21.15.43:0/3251282110 learned_addr learned my addr 172.21.15.43:0/3251282110 (peer_addr_for_me v2:172.21.15.43:0/0) 2023-08-09T13:51:55.495 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:55.484+0000 7fc3a4d68640 1 --2- 172.21.15.43:0/3251282110 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fc3a01568d0 0x7fc3a00cc850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:51:55.495 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:55.484+0000 7fc39ffff640 1 --2- 172.21.15.43:0/3251282110 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fc38c0047c0 0x7fc3a0155310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:51:55.495 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:55.485+0000 7fc397fff640 1 -- 172.21.15.43:0/3251282110 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fc3a01568d0 msgr2=0x7fc3a00cc850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:51:55.495 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:55.485+0000 7fc397fff640 1 --2- 172.21.15.43:0/3251282110 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fc3a01568d0 0x7fc3a00cc850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:51:55.495 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:55.485+0000 7fc397fff640 1 -- 172.21.15.43:0/3251282110 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fc38c0047c0 msgr2=0x7fc3a0155310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:51:55.496 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:55.485+0000 7fc397fff640 1 --2- 172.21.15.43:0/3251282110 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fc38c0047c0 0x7fc3a0155310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:51:55.496 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:55.485+0000 7fc397fff640 1 -- 172.21.15.43:0/3251282110 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc3a0155cd0 con 0x7fc3a01ef6f0 2023-08-09T13:51:55.496 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:55.485+0000 7fc3a4d68640 1 --2- 172.21.15.43:0/3251282110 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fc3a01568d0 0x7fc3a00cc850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-08-09T13:51:55.496 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:55.485+0000 7fc397fff640 1 --2- 172.21.15.43:0/3251282110 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc3a01ef6f0 0x7fc3a01f2950 secure :-1 s=READY pgs=1313 cs=0 l=1 rev1=1 crypto rx=0x7fc388005040 tx=0x7fc388001120 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-09T13:51:55.496 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:55.485+0000 7fc39d7fa640 1 -- 172.21.15.43:0/3251282110 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc3880076c0 con 0x7fc3a01ef6f0 2023-08-09T13:51:55.496 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:55.485+0000 7fc39d7fa640 1 -- 172.21.15.43:0/3251282110 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc388007a30 con 0x7fc3a01ef6f0 2023-08-09T13:51:55.496 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:55.485+0000 7fc39d7fa640 1 -- 172.21.15.43:0/3251282110 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc388006be0 con 0x7fc3a01ef6f0 2023-08-09T13:51:55.496 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:55.485+0000 7fc3a5569640 1 -- 172.21.15.43:0/3251282110 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc3a01538c0 con 0x7fc3a01ef6f0 2023-08-09T13:51:55.497 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:55.486+0000 7fc3a5569640 1 -- 172.21.15.43:0/3251282110 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc3a0156d80 con 0x7fc3a01ef6f0 2023-08-09T13:51:55.497 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:55.487+0000 7fc3a5569640 1 -- 172.21.15.43:0/3251282110 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc3a00739f0 con 0x7fc3a01ef6f0 2023-08-09T13:51:55.497 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:55.488+0000 7fc39d7fa640 1 -- 172.21.15.43:0/3251282110 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 45) v1 ==== 88443+0+0 (secure 0 0 0) 0x7fc388006d40 con 0x7fc3a01ef6f0 2023-08-09T13:51:55.497 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:55.489+0000 7fc39d7fa640 1 --2- 172.21.15.43:0/3251282110 >> [v2:172.21.15.203:6832/1317108840,v1:172.21.15.203:6833/1317108840] conn(0x7fc38006a170 0x7fc38006c630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:51:55.497 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:55.489+0000 7fc39d7fa640 1 -- 172.21.15.43:0/3251282110 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(112..112 src has 1..112) v4 ==== 5574+0+0 (secure 0 0 0) 0x7fc3880aa6f0 con 0x7fc3a01ef6f0 2023-08-09T13:51:55.497 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:55.489+0000 7fc39ffff640 1 --2- 172.21.15.43:0/3251282110 >> [v2:172.21.15.203:6832/1317108840,v1:172.21.15.203:6833/1317108840] conn(0x7fc38006a170 0x7fc38006c630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:51:55.497 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:55.491+0000 7fc39ffff640 1 --2- 172.21.15.43:0/3251282110 >> [v2:172.21.15.203:6832/1317108840,v1:172.21.15.203:6833/1317108840] conn(0x7fc38006a170 0x7fc38006c630 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7fc390009a10 tx=0x7fc390008040 comp rx=0 tx=0).ready entity=mgr.6619 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-09T13:51:55.498 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:55.493+0000 7fc39d7fa640 1 -- 172.21.15.43:0/3251282110 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178066 (secure 0 0 0) 0x7fc3880798c0 con 0x7fc3a01ef6f0 2023-08-09T13:51:55.692 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:55.690+0000 7fc3a5569640 1 -- 172.21.15.43:0/3251282110 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fc3a0159c10 con 0x7fc3a01ef6f0 2023-08-09T13:51:55.695 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:55.693+0000 7fc39d7fa640 1 -- 172.21.15.43:0/3251282110 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v112) v1 ==== 74+0+14854 (secure 0 0 0) 0x7fc388079450 con 0x7fc3a01ef6f0 2023-08-09T13:51:55.695 INFO:teuthology.orchestra.run.smithi043.stdout: 2023-08-09T13:51:55.696 INFO:teuthology.orchestra.run.smithi043.stdout:{"epoch":112,"fsid":"daac87ee-1337-4250-a955-aea4e408a39b","created":"2023-08-09T13:33:41.701198+0000","modified":"2023-08-09T13:51:54.673053+0000","last_up_change":"2023-08-09T13:33:53.783453+0000","last_in_change":"2023-08-09T13:33:45.148904+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":13,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":"rbd","create_time":"2023-08-09T13:33:54.365137+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":".mgr","create_time":"2023-08-09T13:33:54.831803+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"cbd49ecd-e896-4fbd-864e-5c37a829b165","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":101,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6800","nonce":502667335},{"type":"v1","addr":"172.21.15.43:6801","nonce":502667335}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6802","nonce":502667335},{"type":"v1","addr":"172.21.15.43:6803","nonce":502667335}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6806","nonce":502667335},{"type":"v1","addr":"172.21.15.43:6807","nonce":502667335}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6804","nonce":502667335},{"type":"v1","addr":"172.21.15.43:6805","nonce":502667335}]},"public_addr":"172.21.15.43:6801/502667335","cluster_addr":"172.21.15.43:6803/502667335","heartbeat_back_addr":"172.21.15.43:6807/502667335","heartbeat_front_addr":"172.21.15.43:6805/502667335","state":["exists","up"]},{"osd":1,"uuid":"75c7e0fe-1ec7-4fe8-a2f2-3661a65b8500","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":101,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6808","nonce":3289134610},{"type":"v1","addr":"172.21.15.43:6809","nonce":3289134610}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6810","nonce":3289134610},{"type":"v1","addr":"172.21.15.43:6812","nonce":3289134610}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6821","nonce":3289134610},{"type":"v1","addr":"172.21.15.43:6824","nonce":3289134610}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6815","nonce":3289134610},{"type":"v1","addr":"172.21.15.43:6818","nonce":3289134610}]},"public_addr":"172.21.15.43:6809/3289134610","cluster_addr":"172.21.15.43:6812/3289134610","heartbeat_back_addr":"172.21.15.43:6824/3289134610","heartbeat_front_addr":"172.21.15.43:6818/3289134610","state":["exists","up"]},{"osd":2,"uuid":"bb5c41e9-7f5f-4125-9b01-3bb8bfc2ea83","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":101,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6811","nonce":4079709736},{"type":"v1","addr":"172.21.15.43:6814","nonce":4079709736}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6817","nonce":4079709736},{"type":"v1","addr":"172.21.15.43:6820","nonce":4079709736}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6829","nonce":4079709736},{"type":"v1","addr":"172.21.15.43:6831","nonce":4079709736}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6823","nonce":4079709736},{"type":"v1","addr":"172.21.15.43:6826","nonce":4079709736}]},"public_addr":"172.21.15.43:6814/4079709736","cluster_addr":"172.21.15.43:6820/4079709736","heartbeat_back_addr":"172.21.15.43:6831/4079709736","heartbeat_front_addr":"172.21.15.43:6826/4079709736","state":["exists","up"]},{"osd":3,"uuid":"bfe4c8b4-694a-465c-87e6-2b2aab852613","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":101,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6813","nonce":1709538759},{"type":"v1","addr":"172.21.15.43:6816","nonce":1709538759}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6819","nonce":1709538759},{"type":"v1","addr":"172.21.15.43:6822","nonce":1709538759}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6828","nonce":1709538759},{"type":"v1","addr":"172.21.15.43:6830","nonce":1709538759}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6825","nonce":1709538759},{"type":"v1","addr":"172.21.15.43:6827","nonce":1709538759}]},"public_addr":"172.21.15.43:6816/1709538759","cluster_addr":"172.21.15.43:6822/1709538759","heartbeat_back_addr":"172.21.15.43:6830/1709538759","heartbeat_front_addr":"172.21.15.43:6827/1709538759","state":["exists","up"]},{"osd":4,"uuid":"fa2e8a77-aba1-4756-853e-0d174a5bdeba","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":101,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6800","nonce":4196866481},{"type":"v1","addr":"172.21.15.203:6801","nonce":4196866481}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6802","nonce":4196866481},{"type":"v1","addr":"172.21.15.203:6803","nonce":4196866481}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6806","nonce":4196866481},{"type":"v1","addr":"172.21.15.203:6807","nonce":4196866481}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6804","nonce":4196866481},{"type":"v1","addr":"172.21.15.203:6805","nonce":4196866481}]},"public_addr":"172.21.15.203:6801/4196866481","cluster_addr":"172.21.15.203:6803/4196866481","heartbeat_back_addr":"172.21.15.203:6807/4196866481","heartbeat_front_addr":"172.21.15.203:6805/4196866481","state":["exists","up"]},{"osd":5,"uuid":"81811e4b-4d24-4f51-a82c-f1d700a2a00d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":101,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6816","nonce":3059290081},{"type":"v1","addr":"172.21.15.203:6817","nonce":3059290081}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6818","nonce":3059290081},{"type":"v1","addr":"172.21.15.203:6819","nonce":3059290081}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6822","nonce":3059290081},{"type":"v1","addr":"172.21.15.203:6823","nonce":3059290081}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6820","nonce":3059290081},{"type":"v1","addr":"172.21.15.203:6821","nonce":3059290081}]},"public_addr":"172.21.15.203:6817/3059290081","cluster_addr":"172.21.15.203:6819/3059290081","heartbeat_back_addr":"172.21.15.203:6823/3059290081","heartbeat_front_addr":"172.21.15.203:6821/3059290081","state":["exists","up"]},{"osd":6,"uuid":"307bac4d-6a5b-443d-af75-56d695d7674f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":101,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6824","nonce":512155569},{"type":"v1","addr":"172.21.15.203:6825","nonce":512155569}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6826","nonce":512155569},{"type":"v1","addr":"172.21.15.203:6827","nonce":512155569}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6830","nonce":512155569},{"type":"v1","addr":"172.21.15.203:6831","nonce":512155569}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6828","nonce":512155569},{"type":"v1","addr":"172.21.15.203:6829","nonce":512155569}]},"public_addr":"172.21.15.203:6825/512155569","cluster_addr":"172.21.15.203:6827/512155569","heartbeat_back_addr":"172.21.15.203:6831/512155569","heartbeat_front_addr":"172.21.15.203:6829/512155569","state":["exists","up"]},{"osd":7,"uuid":"a95dc7ca-9f1b-4295-af70-68a22a2d7d5e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":101,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6808","nonce":3331450438},{"type":"v1","addr":"172.21.15.203:6809","nonce":3331450438}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6810","nonce":3331450438},{"type":"v1","addr":"172.21.15.203:6811","nonce":3331450438}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6814","nonce":3331450438},{"type":"v1","addr":"172.21.15.203:6815","nonce":3331450438}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6812","nonce":3331450438},{"type":"v1","addr":"172.21.15.203:6813","nonce":3331450438}]},"public_addr":"172.21.15.203:6809/3331450438","cluster_addr":"172.21.15.203:6811/3331450438","heartbeat_back_addr":"172.21.15.203:6815/3331450438","heartbeat_front_addr":"172.21.15.203:6813/3331450438","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-09T13:33:50.547652+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-09T13:33:50.589469+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-09T13:33:50.561724+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-09T13:33:50.599650+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-09T13:33:50.411855+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-09T13:33:50.219681+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-09T13:33:50.504664+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-09T13:33:50.441248+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":7,"snaps":[{"begin":104,"length":1},{"begin":221,"length":1}]}],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-08-09T13:51:55.698 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:55.695+0000 7fc3a5569640 1 -- 172.21.15.43:0/3251282110 >> [v2:172.21.15.203:6832/1317108840,v1:172.21.15.203:6833/1317108840] conn(0x7fc38006a170 msgr2=0x7fc38006c630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:51:55.698 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:55.696+0000 7fc3a5569640 1 --2- 172.21.15.43:0/3251282110 >> [v2:172.21.15.203:6832/1317108840,v1:172.21.15.203:6833/1317108840] conn(0x7fc38006a170 0x7fc38006c630 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7fc390009a10 tx=0x7fc390008040 comp rx=0 tx=0).stop 2023-08-09T13:51:55.699 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:55.696+0000 7fc3a5569640 1 -- 172.21.15.43:0/3251282110 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc3a01ef6f0 msgr2=0x7fc3a01f2950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:51:55.699 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:55.696+0000 7fc3a5569640 1 --2- 172.21.15.43:0/3251282110 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc3a01ef6f0 0x7fc3a01f2950 secure :-1 s=READY pgs=1313 cs=0 l=1 rev1=1 crypto rx=0x7fc388005040 tx=0x7fc388001120 comp rx=0 tx=0).stop 2023-08-09T13:51:55.699 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:55.696+0000 7fc3a5569640 1 -- 172.21.15.43:0/3251282110 shutdown_connections 2023-08-09T13:51:55.700 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:55.696+0000 7fc3a5569640 1 --2- 172.21.15.43:0/3251282110 >> [v2:172.21.15.203:6832/1317108840,v1:172.21.15.203:6833/1317108840] conn(0x7fc38006a170 0x7fc38006c630 unknown :-1 s=CLOSED pgs=49 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:51:55.700 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:55.696+0000 7fc3a5569640 1 --2- 172.21.15.43:0/3251282110 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fc3a01568d0 0x7fc3a00cc850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:51:55.700 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:55.696+0000 7fc3a5569640 1 --2- 172.21.15.43:0/3251282110 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc3a01ef6f0 0x7fc3a01f2950 unknown :-1 s=CLOSED pgs=1313 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:51:55.701 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:55.696+0000 7fc3a5569640 1 --2- 172.21.15.43:0/3251282110 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fc38c0047c0 0x7fc3a0155310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:51:55.701 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:55.696+0000 7fc3a5569640 1 -- 172.21.15.43:0/3251282110 >> 172.21.15.43:0/3251282110 conn(0x7fc3a006a2b0 msgr2=0x7fc3a00d0d50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-09T13:51:55.701 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:55.696+0000 7fc3a5569640 1 -- 172.21.15.43:0/3251282110 shutdown_connections 2023-08-09T13:51:55.701 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:55.696+0000 7fc3a5569640 1 -- 172.21.15.43:0/3251282110 wait complete. 2023-08-09T13:51:55.720 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': 'rbd', 'create_time': '2023-08-09T13:33:54.365137+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}, 'read_balance': {'score_acting': 2, 'score_stable': 2, 'optimal_score': 1, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': '.mgr', 'create_time': '2023-08-09T13:33:54.831803+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-08-09T13:51:55.721 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get rbd pg_num 2023-08-09T13:51:55.959 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:55.958+0000 7fe062a08640 1 Processor -- start 2023-08-09T13:51:55.959 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:55.958+0000 7fe062a08640 1 -- start start 2023-08-09T13:51:55.961 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:55.959+0000 7fe062a08640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7fe05c15c2d0 0x7fe05c0b7df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:51:55.961 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:55.959+0000 7fe062a08640 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7fe05c0cebf0 0x7fe05c0cf040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:51:55.961 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:55.959+0000 7fe062a08640 1 -- --> v1:172.21.15.203:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe05c0cf580 con 0x7fe05c0d1b30 2023-08-09T13:51:55.961 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:55.959+0000 7fe062a08640 1 -- --> v2:172.21.15.43:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe05c0d16a0 con 0x7fe05c15c2d0 2023-08-09T13:51:55.962 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:55.959+0000 7fe062a08640 1 -- --> v2:172.21.15.203:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe05c0d1fa0 con 0x7fe05c0cebf0 2023-08-09T13:51:55.962 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:55.960+0000 7fe061a06640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7fe05c15c2d0 0x7fe05c0b7df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:51:55.962 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:55.960+0000 7fe061a06640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7fe05c15c2d0 0x7fe05c0b7df0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:32968/0 (socket says 172.21.15.43:32968) 2023-08-09T13:51:55.963 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:55.960+0000 7fe061a06640 1 -- 172.21.15.43:0/2280915172 learned_addr learned my addr 172.21.15.43:0/2280915172 (peer_addr_for_me v2:172.21.15.43:0/0) 2023-08-09T13:51:55.964 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:55.960+0000 7fe062207640 1 --2- 172.21.15.43:0/2280915172 >> v2:172.21.15.203:3300/0 conn(0x7fe05c0cebf0 0x7fe05c0cf040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:51:55.964 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:55.960+0000 7fe061a06640 1 -- 172.21.15.43:0/2280915172 >> v2:172.21.15.203:3300/0 conn(0x7fe05c0cebf0 msgr2=0x7fe05c0cf040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-09T13:51:55.964 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:55.960+0000 7fe061a06640 1 --2- 172.21.15.43:0/2280915172 >> v2:172.21.15.203:3300/0 conn(0x7fe05c0cebf0 0x7fe05c0cf040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:51:55.965 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:55.960+0000 7fe061a06640 1 -- 172.21.15.43:0/2280915172 >> v1:172.21.15.203:6789/0 conn(0x7fe05c0d1b30 legacy=0x7fe05c0cd020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-09T13:51:55.965 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:55.960+0000 7fe061a06640 1 -- 172.21.15.43:0/2280915172 --> v2:172.21.15.43:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe05c0d0d70 con 0x7fe05c15c2d0 2023-08-09T13:51:55.965 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:55.961+0000 7fe061a06640 1 --2- 172.21.15.43:0/2280915172 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fe05c15c2d0 0x7fe05c0b7df0 secure :-1 s=READY pgs=1314 cs=0 l=1 rev1=1 crypto rx=0x7fe05c0b8330 tx=0x7fe05802b910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=79769fc3d00d3f7e server_cookie=0 in_seq=0 out_seq=0 2023-08-09T13:51:55.965 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:55.962+0000 7fe060a04640 1 -- 172.21.15.43:0/2280915172 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe058031870 con 0x7fe05c15c2d0 2023-08-09T13:51:55.965 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:55.962+0000 7fe060a04640 1 -- 172.21.15.43:0/2280915172 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe0580319d0 con 0x7fe05c15c2d0 2023-08-09T13:51:55.965 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:55.962+0000 7fe060a04640 1 -- 172.21.15.43:0/2280915172 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe058031d60 con 0x7fe05c15c2d0 2023-08-09T13:51:55.966 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:55.962+0000 7fe062a08640 1 -- 172.21.15.43:0/2280915172 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fe05c15c2d0 msgr2=0x7fe05c0b7df0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:51:55.966 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:55.962+0000 7fe062a08640 1 --2- 172.21.15.43:0/2280915172 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fe05c15c2d0 0x7fe05c0b7df0 secure :-1 s=READY pgs=1314 cs=0 l=1 rev1=1 crypto rx=0x7fe05c0b8330 tx=0x7fe05802b910 comp rx=0 tx=0).stop 2023-08-09T13:51:55.967 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:55.963+0000 7fe062a08640 1 -- 172.21.15.43:0/2280915172 shutdown_connections 2023-08-09T13:51:55.967 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:55.963+0000 7fe062a08640 1 --2- 172.21.15.43:0/2280915172 >> v2:172.21.15.203:3300/0 conn(0x7fe05c0cebf0 0x7fe05c0cf040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:51:55.967 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:55.963+0000 7fe062a08640 1 --2- 172.21.15.43:0/2280915172 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fe05c15c2d0 0x7fe05c0b7df0 unknown :-1 s=CLOSED pgs=1314 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:51:55.967 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:55.963+0000 7fe062a08640 1 -- 172.21.15.43:0/2280915172 >> 172.21.15.43:0/2280915172 conn(0x7fe05c06a230 msgr2=0x7fe05c06a640 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-09T13:51:55.967 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:55.963+0000 7fe062a08640 1 -- 172.21.15.43:0/2280915172 shutdown_connections 2023-08-09T13:51:55.968 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:55.963+0000 7fe062a08640 1 -- 172.21.15.43:0/2280915172 wait complete. 2023-08-09T13:51:55.969 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:55.964+0000 7fe062a08640 1 Processor -- start 2023-08-09T13:51:55.969 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:55.965+0000 7fe062a08640 1 -- start start 2023-08-09T13:51:55.970 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:55.966+0000 7fe062a08640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fe05c0cebf0 0x7fe05c0c9c90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:51:55.970 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:55.966+0000 7fe062a08640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fe05c0d1b30 0x7fe05c0b37c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:51:55.970 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:55.966+0000 7fe062a08640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fe05c15c2d0 0x7fe05c0b3d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:51:55.970 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:55.966+0000 7fe062a08640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe05c0ca1d0 con 0x7fe05c0d1b30 2023-08-09T13:51:55.970 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:55.966+0000 7fe062a08640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe05c1f1dd0 con 0x7fe05c0cebf0 2023-08-09T13:51:55.970 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:55.966+0000 7fe062a08640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe05c1f20b0 con 0x7fe05c15c2d0 2023-08-09T13:51:55.970 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:55.966+0000 7fe061205640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fe05c0d1b30 0x7fe05c0b37c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:51:55.971 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:55.966+0000 7fe061205640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fe05c0d1b30 0x7fe05c0b37c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:32976/0 (socket says 172.21.15.43:32976) 2023-08-09T13:51:55.971 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:55.966+0000 7fe061205640 1 -- 172.21.15.43:0/1943403338 learned_addr learned my addr 172.21.15.43:0/1943403338 (peer_addr_for_me v2:172.21.15.43:0/0) 2023-08-09T13:51:55.971 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:55.967+0000 7fe061205640 1 -- 172.21.15.43:0/1943403338 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fe05c15c2d0 msgr2=0x7fe05c0b3d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:51:55.971 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:55.967+0000 7fe061205640 1 --2- 172.21.15.43:0/1943403338 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fe05c15c2d0 0x7fe05c0b3d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:51:55.971 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:55.967+0000 7fe061205640 1 -- 172.21.15.43:0/1943403338 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fe05c0cebf0 msgr2=0x7fe05c0c9c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:51:55.971 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:55.967+0000 7fe061205640 1 --2- 172.21.15.43:0/1943403338 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fe05c0cebf0 0x7fe05c0c9c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:51:55.971 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:55.967+0000 7fe061205640 1 -- 172.21.15.43:0/1943403338 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe0580094d0 con 0x7fe05c0d1b30 2023-08-09T13:51:55.972 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:55.967+0000 7fe061205640 1 --2- 172.21.15.43:0/1943403338 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fe05c0d1b30 0x7fe05c0b37c0 secure :-1 s=READY pgs=1315 cs=0 l=1 rev1=1 crypto rx=0x7fe04c00c400 tx=0x7fe04c00c8c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-09T13:51:55.972 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:55.967+0000 7fe0527fc640 1 -- 172.21.15.43:0/1943403338 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe04c0035f0 con 0x7fe05c0d1b30 2023-08-09T13:51:55.972 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:55.967+0000 7fe0527fc640 1 -- 172.21.15.43:0/1943403338 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe04c003750 con 0x7fe05c0d1b30 2023-08-09T13:51:55.972 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:55.967+0000 7fe062a08640 1 -- 172.21.15.43:0/1943403338 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe05c1f2390 con 0x7fe05c0d1b30 2023-08-09T13:51:55.972 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:55.967+0000 7fe0527fc640 1 -- 172.21.15.43:0/1943403338 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe04c003b40 con 0x7fe05c0d1b30 2023-08-09T13:51:55.972 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:55.967+0000 7fe062a08640 1 -- 172.21.15.43:0/1943403338 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe05c1f2950 con 0x7fe05c0d1b30 2023-08-09T13:51:55.973 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:55.969+0000 7fe062a08640 1 -- 172.21.15.43:0/1943403338 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe05c04e390 con 0x7fe05c0d1b30 2023-08-09T13:51:55.973 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:55.970+0000 7fe0527fc640 1 -- 172.21.15.43:0/1943403338 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 45) v1 ==== 88443+0+0 (secure 0 0 0) 0x7fe04c003ca0 con 0x7fe05c0d1b30 2023-08-09T13:51:55.973 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:55.971+0000 7fe0527fc640 1 --2- 172.21.15.43:0/1943403338 >> [v2:172.21.15.203:6832/1317108840,v1:172.21.15.203:6833/1317108840] conn(0x7fe04006a170 0x7fe04006c630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:51:55.973 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:55.971+0000 7fe0527fc640 1 -- 172.21.15.43:0/1943403338 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(112..112 src has 1..112) v4 ==== 5574+0+0 (secure 0 0 0) 0x7fe04c087700 con 0x7fe05c0d1b30 2023-08-09T13:51:55.974 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:55.972+0000 7fe061a06640 1 --2- 172.21.15.43:0/1943403338 >> [v2:172.21.15.203:6832/1317108840,v1:172.21.15.203:6833/1317108840] conn(0x7fe04006a170 0x7fe04006c630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:51:55.974 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:55.973+0000 7fe061a06640 1 --2- 172.21.15.43:0/1943403338 >> [v2:172.21.15.203:6832/1317108840,v1:172.21.15.203:6833/1317108840] conn(0x7fe04006a170 0x7fe04006c630 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7fe058036000 tx=0x7fe058030000 comp rx=0 tx=0).ready entity=mgr.6619 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-09T13:51:55.976 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:55.975+0000 7fe0527fc640 1 -- 172.21.15.43:0/1943403338 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178066 (secure 0 0 0) 0x7fe04c052f60 con 0x7fe05c0d1b30 2023-08-09T13:51:56.170 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:56.168+0000 7fe062a08640 1 -- 172.21.15.43:0/1943403338 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"} v 0) v1 -- 0x7fe05c15f6d0 con 0x7fe05c0d1b30 2023-08-09T13:51:56.171 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:56.169+0000 7fe0527fc640 1 -- 172.21.15.43:0/1943403338 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"}]=0 v112) v1 ==== 92+0+10 (secure 0 0 0) 0x7fe04c056580 con 0x7fe05c0d1b30 2023-08-09T13:51:56.171 INFO:teuthology.orchestra.run.smithi043.stdout:pg_num: 8 2023-08-09T13:51:56.174 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:56.172+0000 7fe062a08640 1 -- 172.21.15.43:0/1943403338 >> [v2:172.21.15.203:6832/1317108840,v1:172.21.15.203:6833/1317108840] conn(0x7fe04006a170 msgr2=0x7fe04006c630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:51:56.174 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:56.172+0000 7fe062a08640 1 --2- 172.21.15.43:0/1943403338 >> [v2:172.21.15.203:6832/1317108840,v1:172.21.15.203:6833/1317108840] conn(0x7fe04006a170 0x7fe04006c630 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7fe058036000 tx=0x7fe058030000 comp rx=0 tx=0).stop 2023-08-09T13:51:56.174 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:56.172+0000 7fe062a08640 1 -- 172.21.15.43:0/1943403338 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fe05c0d1b30 msgr2=0x7fe05c0b37c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:51:56.174 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:56.172+0000 7fe062a08640 1 --2- 172.21.15.43:0/1943403338 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fe05c0d1b30 0x7fe05c0b37c0 secure :-1 s=READY pgs=1315 cs=0 l=1 rev1=1 crypto rx=0x7fe04c00c400 tx=0x7fe04c00c8c0 comp rx=0 tx=0).stop 2023-08-09T13:51:56.175 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:56.172+0000 7fe062a08640 1 -- 172.21.15.43:0/1943403338 shutdown_connections 2023-08-09T13:51:56.175 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:56.172+0000 7fe062a08640 1 --2- 172.21.15.43:0/1943403338 >> [v2:172.21.15.203:6832/1317108840,v1:172.21.15.203:6833/1317108840] conn(0x7fe04006a170 0x7fe04006c630 unknown :-1 s=CLOSED pgs=53 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:51:56.175 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:56.172+0000 7fe062a08640 1 --2- 172.21.15.43:0/1943403338 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fe05c15c2d0 0x7fe05c0b3d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:51:56.175 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:56.172+0000 7fe062a08640 1 --2- 172.21.15.43:0/1943403338 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fe05c0d1b30 0x7fe05c0b37c0 unknown :-1 s=CLOSED pgs=1315 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:51:56.175 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:56.172+0000 7fe062a08640 1 --2- 172.21.15.43:0/1943403338 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fe05c0cebf0 0x7fe05c0c9c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:51:56.175 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:56.172+0000 7fe062a08640 1 -- 172.21.15.43:0/1943403338 >> 172.21.15.43:0/1943403338 conn(0x7fe05c06a230 msgr2=0x7fe05c0b19d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-09T13:51:56.175 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:56.173+0000 7fe062a08640 1 -- 172.21.15.43:0/1943403338 shutdown_connections 2023-08-09T13:51:56.176 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:56.173+0000 7fe062a08640 1 -- 172.21.15.43:0/1943403338 wait complete. 2023-08-09T13:51:56.193 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-08-09T13:51:56.428 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:56.426+0000 7fbe9759e640 1 Processor -- start 2023-08-09T13:51:56.428 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:56.426+0000 7fbe9759e640 1 -- start start 2023-08-09T13:51:56.429 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:56.428+0000 7fbe9759e640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fbe98157ea0 0x7fbe980c97d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:51:56.429 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:56.428+0000 7fbe9759e640 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7fbe980c90a0 0x7fbe980c9d10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:51:56.430 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:56.428+0000 7fbe9759e640 1 -- --> v1:172.21.15.43:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbe98158280 con 0x7fbe9815a800 2023-08-09T13:51:56.430 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:56.428+0000 7fbe9759e640 1 -- --> v2:172.21.15.203:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbe9815b840 con 0x7fbe980c90a0 2023-08-09T13:51:56.430 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:56.428+0000 7fbe9759e640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbe9815b350 con 0x7fbe98157ea0 2023-08-09T13:51:56.431 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:56.429+0000 7fbe96d9d640 1 --1- >> v1:172.21.15.43:6789/0 conn(0x7fbe9815a800 0x7fbe9815ac40 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.43:6789/0 says I am v1:172.21.15.43:43692/0 (socket says 172.21.15.43:43692) 2023-08-09T13:51:56.432 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:56.429+0000 7fbe96d9d640 1 -- 172.21.15.43:0/918930165 learned_addr learned my addr 172.21.15.43:0/918930165 (peer_addr_for_me v1:172.21.15.43:0/0) 2023-08-09T13:51:56.432 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:56.429+0000 7fbe95d9b640 1 --2- 172.21.15.43:0/918930165 >> v2:172.21.15.203:3300/0 conn(0x7fbe980c90a0 0x7fbe980c9d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:51:56.432 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:56.429+0000 7fbe9659c640 1 --2- 172.21.15.43:0/918930165 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fbe98157ea0 0x7fbe980c97d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:51:56.432 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:56.429+0000 7fbe9559a640 1 -- 172.21.15.43:0/918930165 <== mon.0 v1:172.21.15.43:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4031539945 0 0) 0x7fbe90001490 con 0x7fbe9815a800 2023-08-09T13:51:56.432 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:56.429+0000 7fbe9559a640 1 -- 172.21.15.43:0/918930165 --> v1:172.21.15.43:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fbe80003520 con 0x7fbe9815a800 2023-08-09T13:51:56.432 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:56.429+0000 7fbe9559a640 1 -- 172.21.15.43:0/918930165 <== mon.0 v1:172.21.15.43:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2640878476 0 0) 0x7fbe80003520 con 0x7fbe9815a800 2023-08-09T13:51:56.433 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:56.429+0000 7fbe9559a640 1 -- 172.21.15.43:0/918930165 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fbe98157ea0 msgr2=0x7fbe980c97d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-09T13:51:56.433 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:56.429+0000 7fbe9559a640 1 --2- 172.21.15.43:0/918930165 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fbe98157ea0 0x7fbe980c97d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:51:56.433 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:56.429+0000 7fbe9559a640 1 -- 172.21.15.43:0/918930165 >> v2:172.21.15.203:3300/0 conn(0x7fbe980c90a0 msgr2=0x7fbe980c9d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-09T13:51:56.433 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:56.429+0000 7fbe9559a640 1 --2- 172.21.15.43:0/918930165 >> v2:172.21.15.203:3300/0 conn(0x7fbe980c90a0 0x7fbe980c9d10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:51:56.433 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:56.430+0000 7fbe9559a640 1 -- 172.21.15.43:0/918930165 --> v1:172.21.15.43:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbe9815b560 con 0x7fbe9815a800 2023-08-09T13:51:56.433 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:56.430+0000 7fbe9559a640 1 -- 172.21.15.43:0/918930165 <== mon.0 v1:172.21.15.43:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1285150740 0 0) 0x7fbe90001b80 con 0x7fbe9815a800 2023-08-09T13:51:56.433 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:56.430+0000 7fbe95d9b640 1 --2- 172.21.15.43:0/918930165 >> v2:172.21.15.203:3300/0 conn(0x7fbe980c90a0 0x7fbe980c9d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-08-09T13:51:56.434 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:56.430+0000 7fbe9559a640 1 -- 172.21.15.43:0/918930165 >> v1:172.21.15.43:6789/0 conn(0x7fbe9815a800 legacy=0x7fbe9815ac40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:51:56.434 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:56.430+0000 7fbe9559a640 1 --2- 172.21.15.43:0/918930165 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fbe98157ea0 0x7fbe80003970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:51:56.435 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:56.430+0000 7fbe9559a640 1 --2- 172.21.15.43:0/918930165 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fbe80003ef0 0x7fbe800047c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:51:56.435 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:56.430+0000 7fbe9559a640 1 --2- 172.21.15.43:0/918930165 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fbe80004d00 0x7fbe80005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:51:56.435 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:56.430+0000 7fbe9559a640 1 -- 172.21.15.43:0/918930165 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbe9815b840 con 0x7fbe80003ef0 2023-08-09T13:51:56.435 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:56.430+0000 7fbe9559a640 1 -- 172.21.15.43:0/918930165 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbe9815b350 con 0x7fbe80004d00 2023-08-09T13:51:56.435 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:56.430+0000 7fbe9559a640 1 -- 172.21.15.43:0/918930165 --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbe8000b980 con 0x7fbe98157ea0 2023-08-09T13:51:56.435 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:56.430+0000 7fbe95d9b640 1 --2- 172.21.15.43:0/918930165 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fbe80003ef0 0x7fbe800047c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:51:56.437 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:56.430+0000 7fbe9659c640 1 --2- 172.21.15.43:0/918930165 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fbe98157ea0 0x7fbe80003970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:51:56.437 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:56.430+0000 7fbe95d9b640 1 -- 172.21.15.43:0/918930165 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fbe98157ea0 msgr2=0x7fbe80003970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-09T13:51:56.437 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:56.430+0000 7fbe95d9b640 1 --2- 172.21.15.43:0/918930165 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fbe98157ea0 0x7fbe80003970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:51:56.438 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:56.430+0000 7fbe95d9b640 1 -- 172.21.15.43:0/918930165 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fbe80004d00 msgr2=0x7fbe80005180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-09T13:51:56.438 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:56.430+0000 7fbe96d9d640 1 --2- 172.21.15.43:0/918930165 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fbe80004d00 0x7fbe80005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:51:56.438 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:56.431+0000 7fbe95d9b640 1 --2- 172.21.15.43:0/918930165 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fbe80004d00 0x7fbe80005180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:51:56.438 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:56.431+0000 7fbe95d9b640 1 -- 172.21.15.43:0/918930165 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbe80003520 con 0x7fbe80003ef0 2023-08-09T13:51:56.438 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:56.431+0000 7fbe96d9d640 1 --2- 172.21.15.43:0/918930165 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fbe80004d00 0x7fbe80005180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-08-09T13:51:56.438 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:56.432+0000 7fbe95d9b640 1 --2- 172.21.15.43:0/918930165 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fbe80003ef0 0x7fbe800047c0 secure :-1 s=READY pgs=1317 cs=0 l=1 rev1=1 crypto rx=0x7fbe8c009c90 tx=0x7fbe800057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8e4e8fed565e80ab server_cookie=0 in_seq=0 out_seq=0 2023-08-09T13:51:56.438 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:56.432+0000 7fbe9559a640 1 -- 172.21.15.43:0/918930165 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbe8c02c980 con 0x7fbe80003ef0 2023-08-09T13:51:56.439 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:56.432+0000 7fbe9559a640 1 -- 172.21.15.43:0/918930165 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbe8c02cae0 con 0x7fbe80003ef0 2023-08-09T13:51:56.439 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:56.432+0000 7fbe9559a640 1 -- 172.21.15.43:0/918930165 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbe8c034440 con 0x7fbe80003ef0 2023-08-09T13:51:56.439 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:56.432+0000 7fbe9759e640 1 -- 172.21.15.43:0/918930165 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fbe80003ef0 msgr2=0x7fbe800047c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:51:56.439 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:56.432+0000 7fbe9759e640 1 --2- 172.21.15.43:0/918930165 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fbe80003ef0 0x7fbe800047c0 secure :-1 s=READY pgs=1317 cs=0 l=1 rev1=1 crypto rx=0x7fbe8c009c90 tx=0x7fbe800057f0 comp rx=0 tx=0).stop 2023-08-09T13:51:56.439 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:56.433+0000 7fbe96d9d640 1 -- 172.21.15.43:0/918930165 reap_dead start 2023-08-09T13:51:56.439 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:56.433+0000 7fbe9759e640 1 -- 172.21.15.43:0/918930165 shutdown_connections 2023-08-09T13:51:56.439 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:56.433+0000 7fbe9759e640 1 -- 172.21.15.43:0/918930165 >> 172.21.15.43:0/918930165 conn(0x7fbe980cddd0 msgr2=0x7fbe980ce1e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-09T13:51:56.439 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:56.433+0000 7fbe9759e640 1 -- 172.21.15.43:0/918930165 shutdown_connections 2023-08-09T13:51:56.440 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:56.433+0000 7fbe9759e640 1 -- 172.21.15.43:0/918930165 wait complete. 2023-08-09T13:51:56.440 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:56.435+0000 7fbe9759e640 1 Processor -- start 2023-08-09T13:51:56.440 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:56.435+0000 7fbe9759e640 1 -- start start 2023-08-09T13:51:56.440 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:56.436+0000 7fbe9759e640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fbe80003ef0 0x7fbe980ab8d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:51:56.440 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:56.436+0000 7fbe9759e640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fbe9814b940 0x7fbe981482f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:51:56.440 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:56.436+0000 7fbe9759e640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fbe9815c9d0 0x7fbe9815ce50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:51:56.440 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:56.436+0000 7fbe9759e640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbe9815d390 con 0x7fbe9814b940 2023-08-09T13:51:56.441 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:56.436+0000 7fbe9759e640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbe980ad6b0 con 0x7fbe80003ef0 2023-08-09T13:51:56.441 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:56.436+0000 7fbe9759e640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbe980ad8e0 con 0x7fbe9815c9d0 2023-08-09T13:51:56.443 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:56.436+0000 7fbe95d9b640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fbe9814b940 0x7fbe981482f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:51:56.443 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:56.436+0000 7fbe95d9b640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fbe9814b940 0x7fbe981482f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:32988/0 (socket says 172.21.15.43:32988) 2023-08-09T13:51:56.443 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:56.436+0000 7fbe95d9b640 1 -- 172.21.15.43:0/3958900764 learned_addr learned my addr 172.21.15.43:0/3958900764 (peer_addr_for_me v2:172.21.15.43:0/0) 2023-08-09T13:51:56.443 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:56.437+0000 7fbe9659c640 1 --2- 172.21.15.43:0/3958900764 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fbe80003ef0 0x7fbe980ab8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:51:56.443 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:56.437+0000 7fbe95d9b640 1 -- 172.21.15.43:0/3958900764 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fbe9815c9d0 msgr2=0x7fbe9815ce50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:51:56.443 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:56.437+0000 7fbe95d9b640 1 --2- 172.21.15.43:0/3958900764 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fbe9815c9d0 0x7fbe9815ce50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:51:56.444 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:56.437+0000 7fbe95d9b640 1 -- 172.21.15.43:0/3958900764 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fbe80003ef0 msgr2=0x7fbe980ab8d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:51:56.444 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:56.437+0000 7fbe95d9b640 1 --2- 172.21.15.43:0/3958900764 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fbe80003ef0 0x7fbe980ab8d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:51:56.444 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:56.437+0000 7fbe95d9b640 1 -- 172.21.15.43:0/3958900764 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbe980adbc0 con 0x7fbe9814b940 2023-08-09T13:51:56.444 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:56.437+0000 7fbe9659c640 1 --2- 172.21.15.43:0/3958900764 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fbe80003ef0 0x7fbe980ab8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-08-09T13:51:56.444 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:56.437+0000 7fbe95d9b640 1 --2- 172.21.15.43:0/3958900764 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fbe9814b940 0x7fbe981482f0 secure :-1 s=READY pgs=1318 cs=0 l=1 rev1=1 crypto rx=0x7fbe8c037040 tx=0x7fbe8c004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-09T13:51:56.445 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:56.437+0000 7fbe86ffd640 1 -- 172.21.15.43:0/3958900764 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbe8c034a80 con 0x7fbe9814b940 2023-08-09T13:51:56.445 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:56.437+0000 7fbe86ffd640 1 -- 172.21.15.43:0/3958900764 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbe8c034c20 con 0x7fbe9814b940 2023-08-09T13:51:56.445 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:56.437+0000 7fbe86ffd640 1 -- 172.21.15.43:0/3958900764 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbe8c004020 con 0x7fbe9814b940 2023-08-09T13:51:56.445 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:56.437+0000 7fbe9759e640 1 -- 172.21.15.43:0/3958900764 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbe980adea0 con 0x7fbe9814b940 2023-08-09T13:51:56.445 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:56.438+0000 7fbe9759e640 1 -- 172.21.15.43:0/3958900764 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbe980ac1d0 con 0x7fbe9814b940 2023-08-09T13:51:56.445 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:56.439+0000 7fbe9759e640 1 -- 172.21.15.43:0/3958900764 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbe9804e390 con 0x7fbe9814b940 2023-08-09T13:51:56.445 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:56.440+0000 7fbe86ffd640 1 -- 172.21.15.43:0/3958900764 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 45) v1 ==== 88443+0+0 (secure 0 0 0) 0x7fbe8c004180 con 0x7fbe9814b940 2023-08-09T13:51:56.446 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:56.441+0000 7fbe86ffd640 1 --2- 172.21.15.43:0/3958900764 >> [v2:172.21.15.203:6832/1317108840,v1:172.21.15.203:6833/1317108840] conn(0x7fbe7806a170 0x7fbe7806c630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:51:56.446 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:56.441+0000 7fbe86ffd640 1 -- 172.21.15.43:0/3958900764 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(112..112 src has 1..112) v4 ==== 5574+0+0 (secure 0 0 0) 0x7fbe8c0aa8d0 con 0x7fbe9814b940 2023-08-09T13:51:56.446 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:56.441+0000 7fbe9659c640 1 --2- 172.21.15.43:0/3958900764 >> [v2:172.21.15.203:6832/1317108840,v1:172.21.15.203:6833/1317108840] conn(0x7fbe7806a170 0x7fbe7806c630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:51:56.446 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:56.442+0000 7fbe9659c640 1 --2- 172.21.15.43:0/3958900764 >> [v2:172.21.15.203:6832/1317108840,v1:172.21.15.203:6833/1317108840] conn(0x7fbe7806a170 0x7fbe7806c630 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7fbe88009910 tx=0x7fbe88008040 comp rx=0 tx=0).ready entity=mgr.6619 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-09T13:51:56.447 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:56.445+0000 7fbe86ffd640 1 -- 172.21.15.43:0/3958900764 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178066 (secure 0 0 0) 0x7fbe8c075ff0 con 0x7fbe9814b940 2023-08-09T13:51:56.640 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:56.638+0000 7fbe9759e640 1 -- 172.21.15.43:0/3958900764 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7fbe980af2a0 con 0x7fbe9814b940 2023-08-09T13:51:56.641 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:56.639+0000 7fbe86ffd640 1 -- 172.21.15.43:0/3958900764 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v112) v1 ==== 93+0+10 (secure 0 0 0) 0x7fbe8c079610 con 0x7fbe9814b940 2023-08-09T13:51:56.641 INFO:teuthology.orchestra.run.smithi043.stdout:pg_num: 1 2023-08-09T13:51:56.644 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:56.641+0000 7fbe9759e640 1 -- 172.21.15.43:0/3958900764 >> [v2:172.21.15.203:6832/1317108840,v1:172.21.15.203:6833/1317108840] conn(0x7fbe7806a170 msgr2=0x7fbe7806c630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:51:56.644 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:56.641+0000 7fbe9759e640 1 --2- 172.21.15.43:0/3958900764 >> [v2:172.21.15.203:6832/1317108840,v1:172.21.15.203:6833/1317108840] conn(0x7fbe7806a170 0x7fbe7806c630 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7fbe88009910 tx=0x7fbe88008040 comp rx=0 tx=0).stop 2023-08-09T13:51:56.644 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:56.641+0000 7fbe9759e640 1 -- 172.21.15.43:0/3958900764 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fbe9814b940 msgr2=0x7fbe981482f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:51:56.644 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:56.641+0000 7fbe9759e640 1 --2- 172.21.15.43:0/3958900764 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fbe9814b940 0x7fbe981482f0 secure :-1 s=READY pgs=1318 cs=0 l=1 rev1=1 crypto rx=0x7fbe8c037040 tx=0x7fbe8c004730 comp rx=0 tx=0).stop 2023-08-09T13:51:56.645 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:56.642+0000 7fbe9759e640 1 -- 172.21.15.43:0/3958900764 shutdown_connections 2023-08-09T13:51:56.645 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:56.642+0000 7fbe9759e640 1 --2- 172.21.15.43:0/3958900764 >> [v2:172.21.15.203:6832/1317108840,v1:172.21.15.203:6833/1317108840] conn(0x7fbe7806a170 0x7fbe7806c630 unknown :-1 s=CLOSED pgs=54 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:51:56.645 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:56.642+0000 7fbe9759e640 1 --2- 172.21.15.43:0/3958900764 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fbe9815c9d0 0x7fbe9815ce50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:51:56.645 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:56.642+0000 7fbe9759e640 1 --2- 172.21.15.43:0/3958900764 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fbe9814b940 0x7fbe981482f0 unknown :-1 s=CLOSED pgs=1318 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:51:56.646 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:56.642+0000 7fbe9759e640 1 --2- 172.21.15.43:0/3958900764 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fbe80003ef0 0x7fbe980ab8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:51:56.646 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:56.642+0000 7fbe9759e640 1 -- 172.21.15.43:0/3958900764 >> 172.21.15.43:0/3958900764 conn(0x7fbe980cddd0 msgr2=0x7fbe9815ac40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-09T13:51:56.647 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:56.642+0000 7fbe9759e640 1 -- 172.21.15.43:0/3958900764 shutdown_connections 2023-08-09T13:51:56.647 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:56.642+0000 7fbe9759e640 1 -- 172.21.15.43:0/3958900764 wait complete. 2023-08-09T13:51:56.663 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --format=json-pretty osd lspools 2023-08-09T13:51:56.898 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:56.896+0000 7f0975a0f640 1 Processor -- start 2023-08-09T13:51:56.898 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:56.897+0000 7f0975a0f640 1 -- start start 2023-08-09T13:51:56.899 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:56.898+0000 7f0975a0f640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f097015f7b0 0x7f09700b4760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:51:56.900 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:56.898+0000 7f0975a0f640 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7f09700cd6b0 0x7f09700d0e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:51:56.900 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:56.898+0000 7f0975a0f640 1 -- --> v1:172.21.15.43:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f09700b7bf0 con 0x7f09700b1050 2023-08-09T13:51:56.900 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:56.898+0000 7f0975a0f640 1 -- --> v2:172.21.15.203:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f09700b7ed0 con 0x7f09700cd6b0 2023-08-09T13:51:56.900 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:56.898+0000 7f0975a0f640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f09700d0370 con 0x7f097015f7b0 2023-08-09T13:51:56.902 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:56.898+0000 7f096ffff640 1 --1- >> v1:172.21.15.43:6789/0 conn(0x7f09700b1050 0x7f09700ccfa0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.43:6789/0 says I am v1:172.21.15.43:43702/0 (socket says 172.21.15.43:43702) 2023-08-09T13:51:56.902 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:56.898+0000 7f096ffff640 1 -- 172.21.15.43:0/749979750 learned_addr learned my addr 172.21.15.43:0/749979750 (peer_addr_for_me v1:172.21.15.43:0/0) 2023-08-09T13:51:56.902 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:56.899+0000 7f097520e640 1 --2- 172.21.15.43:0/749979750 >> v2:172.21.15.203:3300/0 conn(0x7f09700cd6b0 0x7f09700d0e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:51:56.902 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:56.899+0000 7f0974a0d640 1 --2- 172.21.15.43:0/749979750 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f097015f7b0 0x7f09700b4760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:51:56.903 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:56.899+0000 7f096f7fe640 1 -- 172.21.15.43:0/749979750 <== mon.0 v1:172.21.15.43:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3959010582 0 0) 0x7f09700b7bf0 con 0x7f09700b1050 2023-08-09T13:51:56.903 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:56.899+0000 7f096f7fe640 1 -- 172.21.15.43:0/749979750 --> v1:172.21.15.43:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f0960003520 con 0x7f09700b1050 2023-08-09T13:51:56.903 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:56.899+0000 7f096f7fe640 1 -- 172.21.15.43:0/749979750 <== mon.0 v1:172.21.15.43:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1054263935 0 0) 0x7f0960003520 con 0x7f09700b1050 2023-08-09T13:51:56.903 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:56.899+0000 7f096f7fe640 1 -- 172.21.15.43:0/749979750 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f097015f7b0 msgr2=0x7f09700b4760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-09T13:51:56.903 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:56.899+0000 7f096f7fe640 1 --2- 172.21.15.43:0/749979750 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f097015f7b0 0x7f09700b4760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:51:56.903 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:56.899+0000 7f096f7fe640 1 -- 172.21.15.43:0/749979750 >> v2:172.21.15.203:3300/0 conn(0x7f09700cd6b0 msgr2=0x7f09700d0e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-09T13:51:56.903 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:56.899+0000 7f096f7fe640 1 --2- 172.21.15.43:0/749979750 >> v2:172.21.15.203:3300/0 conn(0x7f09700cd6b0 0x7f09700d0e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:51:56.904 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:56.899+0000 7f096f7fe640 1 -- 172.21.15.43:0/749979750 --> v1:172.21.15.43:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f09700d0650 con 0x7f09700b1050 2023-08-09T13:51:56.904 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:56.899+0000 7f096f7fe640 1 -- 172.21.15.43:0/749979750 <== mon.0 v1:172.21.15.43:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1285150740 0 0) 0x7f095c001670 con 0x7f09700b1050 2023-08-09T13:51:56.904 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:56.899+0000 7f096f7fe640 1 -- 172.21.15.43:0/749979750 >> v1:172.21.15.43:6789/0 conn(0x7f09700b1050 legacy=0x7f09700ccfa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:51:56.904 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:56.900+0000 7f096f7fe640 1 --2- 172.21.15.43:0/749979750 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f0960003970 0x7f0960003d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:51:56.904 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:56.900+0000 7f096f7fe640 1 --2- 172.21.15.43:0/749979750 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f09600047c0 0x7f0960004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:51:56.904 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:56.900+0000 7f096f7fe640 1 --2- 172.21.15.43:0/749979750 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f097015f7b0 0x7f0960005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:51:56.904 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:56.900+0000 7f096f7fe640 1 -- 172.21.15.43:0/749979750 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f09700b7ed0 con 0x7f0960003970 2023-08-09T13:51:56.904 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:56.900+0000 7f096f7fe640 1 -- 172.21.15.43:0/749979750 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f09700d0370 con 0x7f09600047c0 2023-08-09T13:51:56.905 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:56.900+0000 7f096f7fe640 1 -- 172.21.15.43:0/749979750 --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f096000b980 con 0x7f097015f7b0 2023-08-09T13:51:56.905 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:56.900+0000 7f0974a0d640 1 --2- 172.21.15.43:0/749979750 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f0960003970 0x7f0960003d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:51:56.905 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:56.900+0000 7f096ffff640 1 --2- 172.21.15.43:0/749979750 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f09600047c0 0x7f0960004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:51:56.905 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:56.900+0000 7f0974a0d640 1 -- 172.21.15.43:0/749979750 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f097015f7b0 msgr2=0x7f0960005180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-09T13:51:56.905 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:56.900+0000 7f0974a0d640 1 --2- 172.21.15.43:0/749979750 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f097015f7b0 0x7f0960005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:51:56.905 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:56.900+0000 7f0974a0d640 1 -- 172.21.15.43:0/749979750 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f09600047c0 msgr2=0x7f0960004c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-09T13:51:56.905 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:56.900+0000 7f0974a0d640 1 --2- 172.21.15.43:0/749979750 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f09600047c0 0x7f0960004c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:51:56.906 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:56.900+0000 7f0974a0d640 1 -- 172.21.15.43:0/749979750 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0960003520 con 0x7f0960003970 2023-08-09T13:51:56.906 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:56.900+0000 7f096ffff640 1 --2- 172.21.15.43:0/749979750 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f09600047c0 0x7f0960004c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-08-09T13:51:56.907 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:56.901+0000 7f0974a0d640 1 --2- 172.21.15.43:0/749979750 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f0960003970 0x7f0960003d90 secure :-1 s=READY pgs=1320 cs=0 l=1 rev1=1 crypto rx=0x7f0964009ac0 tx=0x7f09600057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9e4a1e4caef0972b server_cookie=0 in_seq=0 out_seq=0 2023-08-09T13:51:56.907 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:56.902+0000 7f096f7fe640 1 -- 172.21.15.43:0/749979750 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f096402c710 con 0x7f0960003970 2023-08-09T13:51:56.907 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:56.902+0000 7f096f7fe640 1 -- 172.21.15.43:0/749979750 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f096402c870 con 0x7f0960003970 2023-08-09T13:51:56.907 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:56.902+0000 7f096f7fe640 1 -- 172.21.15.43:0/749979750 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f096402cc00 con 0x7f0960003970 2023-08-09T13:51:56.908 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:56.902+0000 7f0975a0f640 1 -- 172.21.15.43:0/749979750 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f0960003970 msgr2=0x7f0960003d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:51:56.908 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:56.902+0000 7f0975a0f640 1 --2- 172.21.15.43:0/749979750 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f0960003970 0x7f0960003d90 secure :-1 s=READY pgs=1320 cs=0 l=1 rev1=1 crypto rx=0x7f0964009ac0 tx=0x7f09600057f0 comp rx=0 tx=0).stop 2023-08-09T13:51:56.908 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:56.902+0000 7f097520e640 1 -- 172.21.15.43:0/749979750 reap_dead start 2023-08-09T13:51:56.908 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:56.902+0000 7f0975a0f640 1 -- 172.21.15.43:0/749979750 shutdown_connections 2023-08-09T13:51:56.908 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:56.902+0000 7f0975a0f640 1 -- 172.21.15.43:0/749979750 >> 172.21.15.43:0/749979750 conn(0x7f097006a2b0 msgr2=0x7f097006a690 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-09T13:51:56.908 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:56.903+0000 7f0975a0f640 1 -- 172.21.15.43:0/749979750 shutdown_connections 2023-08-09T13:51:56.908 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:56.903+0000 7f0975a0f640 1 -- 172.21.15.43:0/749979750 wait complete. 2023-08-09T13:51:56.909 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:56.904+0000 7f0975a0f640 1 Processor -- start 2023-08-09T13:51:56.909 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:56.905+0000 7f0975a0f640 1 -- start start 2023-08-09T13:51:56.909 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:56.906+0000 7f0975a0f640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f0960003970 0x7f0970157ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:51:56.909 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:56.906+0000 7f0975a0f640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f09700c98e0 0x7f0970156520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:51:56.909 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:56.906+0000 7f0975a0f640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f09700c9d60 0x7f0970156a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:51:56.909 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:56.906+0000 7f0975a0f640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0970156fa0 con 0x7f09700c9d60 2023-08-09T13:51:56.909 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:56.906+0000 7f0975a0f640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f09700cb780 con 0x7f09700c98e0 2023-08-09T13:51:56.909 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:56.906+0000 7f0975a0f640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f09700cb9f0 con 0x7f0960003970 2023-08-09T13:51:56.910 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:56.906+0000 7f097520e640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f09700c9d60 0x7f0970156a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:51:56.911 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:56.906+0000 7f097520e640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f09700c9d60 0x7f0970156a60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:32994/0 (socket says 172.21.15.43:32994) 2023-08-09T13:51:56.911 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:56.906+0000 7f097520e640 1 -- 172.21.15.43:0/4043486552 learned_addr learned my addr 172.21.15.43:0/4043486552 (peer_addr_for_me v2:172.21.15.43:0/0) 2023-08-09T13:51:56.911 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:56.906+0000 7f096ffff640 1 --2- 172.21.15.43:0/4043486552 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f09700c98e0 0x7f0970156520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:51:56.911 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:56.906+0000 7f097520e640 1 -- 172.21.15.43:0/4043486552 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f0960003970 msgr2=0x7f0970157ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:51:56.911 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:56.906+0000 7f097520e640 1 --2- 172.21.15.43:0/4043486552 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f0960003970 0x7f0970157ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:51:56.912 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:56.906+0000 7f097520e640 1 -- 172.21.15.43:0/4043486552 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f09700c98e0 msgr2=0x7f0970156520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:51:56.912 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:56.906+0000 7f097520e640 1 --2- 172.21.15.43:0/4043486552 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f09700c98e0 0x7f0970156520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:51:56.912 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:56.906+0000 7f097520e640 1 -- 172.21.15.43:0/4043486552 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f09700cbcd0 con 0x7f09700c9d60 2023-08-09T13:51:56.912 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:56.906+0000 7f096ffff640 1 --2- 172.21.15.43:0/4043486552 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f09700c98e0 0x7f0970156520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-08-09T13:51:56.912 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:56.907+0000 7f097520e640 1 --2- 172.21.15.43:0/4043486552 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f09700c9d60 0x7f0970156a60 secure :-1 s=READY pgs=1321 cs=0 l=1 rev1=1 crypto rx=0x7f097006bfe0 tx=0x7f096800b480 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-09T13:51:56.912 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:56.907+0000 7f096d7fa640 1 -- 172.21.15.43:0/4043486552 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0968010040 con 0x7f09700c9d60 2023-08-09T13:51:56.912 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:56.907+0000 7f096d7fa640 1 -- 172.21.15.43:0/4043486552 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f096800bae0 con 0x7f09700c9d60 2023-08-09T13:51:56.913 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:56.907+0000 7f0975a0f640 1 -- 172.21.15.43:0/4043486552 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f09700cbfb0 con 0x7f09700c9d60 2023-08-09T13:51:56.913 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:56.907+0000 7f096d7fa640 1 -- 172.21.15.43:0/4043486552 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f096800be70 con 0x7f09700c9d60 2023-08-09T13:51:56.913 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:56.907+0000 7f0975a0f640 1 -- 172.21.15.43:0/4043486552 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0970158530 con 0x7f09700c9d60 2023-08-09T13:51:56.913 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:56.909+0000 7f0975a0f640 1 -- 172.21.15.43:0/4043486552 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f09700cc4d0 con 0x7f09700c9d60 2023-08-09T13:51:56.913 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:56.910+0000 7f096d7fa640 1 -- 172.21.15.43:0/4043486552 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 45) v1 ==== 88443+0+0 (secure 0 0 0) 0x7f096801d400 con 0x7f09700c9d60 2023-08-09T13:51:56.914 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:56.910+0000 7f096d7fa640 1 --2- 172.21.15.43:0/4043486552 >> [v2:172.21.15.203:6832/1317108840,v1:172.21.15.203:6833/1317108840] conn(0x7f095406a1a0 0x7f095406c660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:51:56.914 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:56.910+0000 7f096d7fa640 1 -- 172.21.15.43:0/4043486552 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(112..112 src has 1..112) v4 ==== 5574+0+0 (secure 0 0 0) 0x7f0968088c70 con 0x7f09700c9d60 2023-08-09T13:51:56.914 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:56.911+0000 7f0974a0d640 1 --2- 172.21.15.43:0/4043486552 >> [v2:172.21.15.203:6832/1317108840,v1:172.21.15.203:6833/1317108840] conn(0x7f095406a1a0 0x7f095406c660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:51:56.914 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:56.912+0000 7f0974a0d640 1 --2- 172.21.15.43:0/4043486552 >> [v2:172.21.15.203:6832/1317108840,v1:172.21.15.203:6833/1317108840] conn(0x7f095406a1a0 0x7f095406c660 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f0964036000 tx=0x7f096402e000 comp rx=0 tx=0).ready entity=mgr.6619 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-09T13:51:56.916 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:56.915+0000 7f096d7fa640 1 -- 172.21.15.43:0/4043486552 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178066 (secure 0 0 0) 0x7f09700cc4d0 con 0x7f09700c9d60 2023-08-09T13:51:57.115 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:57.114+0000 7f0975a0f640 1 -- 172.21.15.43:0/4043486552 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) v1 -- 0x7f09700cc4d0 con 0x7f09700c9d60 2023-08-09T13:51:57.116 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:57.114+0000 7f096d7fa640 1 -- 172.21.15.43:0/4043486552 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v112) v1 ==== 84+0+126 (secure 0 0 0) 0x7f09700cc4d0 con 0x7f09700c9d60 2023-08-09T13:51:57.116 INFO:teuthology.orchestra.run.smithi043.stdout: 2023-08-09T13:51:57.116 INFO:teuthology.orchestra.run.smithi043.stdout:[ 2023-08-09T13:51:57.117 INFO:teuthology.orchestra.run.smithi043.stdout: { 2023-08-09T13:51:57.117 INFO:teuthology.orchestra.run.smithi043.stdout: "poolnum": 1, 2023-08-09T13:51:57.117 INFO:teuthology.orchestra.run.smithi043.stdout: "poolname": "rbd" 2023-08-09T13:51:57.117 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2023-08-09T13:51:57.117 INFO:teuthology.orchestra.run.smithi043.stdout: { 2023-08-09T13:51:57.117 INFO:teuthology.orchestra.run.smithi043.stdout: "poolnum": 2, 2023-08-09T13:51:57.117 INFO:teuthology.orchestra.run.smithi043.stdout: "poolname": ".mgr" 2023-08-09T13:51:57.117 INFO:teuthology.orchestra.run.smithi043.stdout: } 2023-08-09T13:51:57.118 INFO:teuthology.orchestra.run.smithi043.stdout:] 2023-08-09T13:51:57.119 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:57.117+0000 7f0975a0f640 1 -- 172.21.15.43:0/4043486552 >> [v2:172.21.15.203:6832/1317108840,v1:172.21.15.203:6833/1317108840] conn(0x7f095406a1a0 msgr2=0x7f095406c660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:51:57.119 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:57.117+0000 7f0975a0f640 1 --2- 172.21.15.43:0/4043486552 >> [v2:172.21.15.203:6832/1317108840,v1:172.21.15.203:6833/1317108840] conn(0x7f095406a1a0 0x7f095406c660 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f0964036000 tx=0x7f096402e000 comp rx=0 tx=0).stop 2023-08-09T13:51:57.119 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:57.117+0000 7f0975a0f640 1 -- 172.21.15.43:0/4043486552 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f09700c9d60 msgr2=0x7f0970156a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:51:57.120 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:57.117+0000 7f0975a0f640 1 --2- 172.21.15.43:0/4043486552 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f09700c9d60 0x7f0970156a60 secure :-1 s=READY pgs=1321 cs=0 l=1 rev1=1 crypto rx=0x7f097006bfe0 tx=0x7f096800b480 comp rx=0 tx=0).stop 2023-08-09T13:51:57.120 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:57.118+0000 7f0975a0f640 1 -- 172.21.15.43:0/4043486552 shutdown_connections 2023-08-09T13:51:57.120 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:57.118+0000 7f0975a0f640 1 --2- 172.21.15.43:0/4043486552 >> [v2:172.21.15.203:6832/1317108840,v1:172.21.15.203:6833/1317108840] conn(0x7f095406a1a0 0x7f095406c660 unknown :-1 s=CLOSED pgs=55 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:51:57.120 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:57.118+0000 7f0975a0f640 1 --2- 172.21.15.43:0/4043486552 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f09700c9d60 0x7f0970156a60 unknown :-1 s=CLOSED pgs=1321 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:51:57.120 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:57.118+0000 7f0975a0f640 1 --2- 172.21.15.43:0/4043486552 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f09700c98e0 0x7f0970156520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:51:57.120 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:57.118+0000 7f0975a0f640 1 --2- 172.21.15.43:0/4043486552 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f0960003970 0x7f0970157ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:51:57.120 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:57.118+0000 7f0975a0f640 1 -- 172.21.15.43:0/4043486552 >> 172.21.15.43:0/4043486552 conn(0x7f097006a2b0 msgr2=0x7f097015e1d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-09T13:51:57.121 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:57.118+0000 7f0975a0f640 1 -- 172.21.15.43:0/4043486552 shutdown_connections 2023-08-09T13:51:57.121 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:57.118+0000 7f0975a0f640 1 -- 172.21.15.43:0/4043486552 wait complete. 2023-08-09T13:51:57.139 DEBUG:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2023-08-09T13:51:57.139 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_metadata --pg_num_min 64 2023-08-09T13:51:57.369 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:57.367+0000 7fc38dbdb640 1 Processor -- start 2023-08-09T13:51:57.370 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:57.367+0000 7fc38dbdb640 1 -- start start 2023-08-09T13:51:57.373 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:57.369+0000 7fc38dbdb640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7fc38815ea60 0x7fc3880c9780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:51:57.377 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:57.369+0000 7fc38dbdb640 1 -- --> v1:172.21.15.43:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc38815dd30 con 0x7fc3880aff80 2023-08-09T13:51:57.377 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:57.369+0000 7fc38dbdb640 1 -- --> v1:172.21.15.203:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc388071490 con 0x7fc38815a330 2023-08-09T13:51:57.377 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:57.369+0000 7fc38dbdb640 1 -- --> v2:172.21.15.43:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc3880c9cc0 con 0x7fc38815ea60 2023-08-09T13:51:57.378 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:57.369+0000 7fc387fff640 1 --1- >> v1:172.21.15.43:6789/0 conn(0x7fc3880aff80 0x7fc38806b400 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.43:6789/0 says I am v1:172.21.15.43:43716/0 (socket says 172.21.15.43:43716) 2023-08-09T13:51:57.378 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:57.369+0000 7fc387fff640 1 -- 172.21.15.43:0/3498956414 learned_addr learned my addr 172.21.15.43:0/3498956414 (peer_addr_for_me v1:172.21.15.43:0/0) 2023-08-09T13:51:57.378 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:57.369+0000 7fc38d3da640 1 --2- 172.21.15.43:0/3498956414 >> v2:172.21.15.43:3300/0 conn(0x7fc38815ea60 0x7fc3880c9780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:51:57.378 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:57.369+0000 7fc3877fe640 1 -- 172.21.15.43:0/3498956414 <== mon.0 v1:172.21.15.43:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4144211252 0 0) 0x7fc38815dd30 con 0x7fc3880aff80 2023-08-09T13:51:57.378 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:57.370+0000 7fc3877fe640 1 -- 172.21.15.43:0/3498956414 --> v1:172.21.15.43:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fc370003520 con 0x7fc3880aff80 2023-08-09T13:51:57.379 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:57.370+0000 7fc38d3da640 1 -- 172.21.15.43:0/3498956414 >> v1:172.21.15.203:6789/0 conn(0x7fc38815a330 legacy=0x7fc3880af870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-09T13:51:57.379 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:57.370+0000 7fc38d3da640 1 -- 172.21.15.43:0/3498956414 >> v1:172.21.15.43:6789/0 conn(0x7fc3880aff80 legacy=0x7fc38806b400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:51:57.379 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:57.370+0000 7fc38d3da640 1 -- 172.21.15.43:0/3498956414 --> v2:172.21.15.43:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc38815df80 con 0x7fc38815ea60 2023-08-09T13:51:57.379 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:57.371+0000 7fc38d3da640 1 --2- 172.21.15.43:0/3498956414 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc38815ea60 0x7fc3880c9780 secure :-1 s=READY pgs=1323 cs=0 l=1 rev1=1 crypto rx=0x7fc3800097a0 tx=0x7fc38002b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1eba4a95b604b288 server_cookie=0 in_seq=0 out_seq=0 2023-08-09T13:51:57.380 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:57.371+0000 7fc3877fe640 1 -- 172.21.15.43:0/3498956414 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc380031830 con 0x7fc38815ea60 2023-08-09T13:51:57.380 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:57.371+0000 7fc3877fe640 1 -- 172.21.15.43:0/3498956414 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc380031990 con 0x7fc38815ea60 2023-08-09T13:51:57.380 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:57.371+0000 7fc3877fe640 1 -- 172.21.15.43:0/3498956414 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc380031d20 con 0x7fc38815ea60 2023-08-09T13:51:57.386 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:57.372+0000 7fc38dbdb640 1 -- 172.21.15.43:0/3498956414 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc38815ea60 msgr2=0x7fc3880c9780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:51:57.386 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:57.372+0000 7fc38dbdb640 1 --2- 172.21.15.43:0/3498956414 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc38815ea60 0x7fc3880c9780 secure :-1 s=READY pgs=1323 cs=0 l=1 rev1=1 crypto rx=0x7fc3800097a0 tx=0x7fc38002b8d0 comp rx=0 tx=0).stop 2023-08-09T13:51:57.386 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:57.372+0000 7fc38dbdb640 1 -- 172.21.15.43:0/3498956414 shutdown_connections 2023-08-09T13:51:57.386 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:57.372+0000 7fc38dbdb640 1 --2- 172.21.15.43:0/3498956414 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc38815ea60 0x7fc3880c9780 unknown :-1 s=CLOSED pgs=1323 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:51:57.387 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:57.372+0000 7fc38dbdb640 1 -- 172.21.15.43:0/3498956414 >> 172.21.15.43:0/3498956414 conn(0x7fc3880ca870 msgr2=0x7fc3880cac50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-09T13:51:57.387 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:57.372+0000 7fc38dbdb640 1 -- 172.21.15.43:0/3498956414 shutdown_connections 2023-08-09T13:51:57.387 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:57.372+0000 7fc38dbdb640 1 -- 172.21.15.43:0/3498956414 wait complete. 2023-08-09T13:51:57.387 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:57.374+0000 7fc38dbdb640 1 Processor -- start 2023-08-09T13:51:57.387 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:57.374+0000 7fc38dbdb640 1 -- start start 2023-08-09T13:51:57.388 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:57.375+0000 7fc38dbdb640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fc3880aff80 0x7fc3880ab8d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:51:57.388 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:57.375+0000 7fc38dbdb640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc38815a330 0x7fc3881f1380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:51:57.388 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:57.375+0000 7fc38dbdb640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fc38815ea60 0x7fc388156ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:51:57.388 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:57.375+0000 7fc38dbdb640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc3880abef0 con 0x7fc38815a330 2023-08-09T13:51:57.388 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:57.375+0000 7fc38dbdb640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc3880ac1d0 con 0x7fc3880aff80 2023-08-09T13:51:57.389 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:57.375+0000 7fc38dbdb640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc3880ac4b0 con 0x7fc38815ea60 2023-08-09T13:51:57.389 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:57.376+0000 7fc387fff640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc38815a330 0x7fc3881f1380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:51:57.389 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:57.376+0000 7fc387fff640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc38815a330 0x7fc3881f1380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:33014/0 (socket says 172.21.15.43:33014) 2023-08-09T13:51:57.389 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:57.376+0000 7fc387fff640 1 -- 172.21.15.43:0/456916313 learned_addr learned my addr 172.21.15.43:0/456916313 (peer_addr_for_me v2:172.21.15.43:0/0) 2023-08-09T13:51:57.389 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:57.376+0000 7fc38cbd9640 1 --2- 172.21.15.43:0/456916313 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fc3880aff80 0x7fc3880ab8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:51:57.389 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:57.376+0000 7fc387fff640 1 -- 172.21.15.43:0/456916313 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fc38815ea60 msgr2=0x7fc388156ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:51:57.389 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:57.376+0000 7fc387fff640 1 --2- 172.21.15.43:0/456916313 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fc38815ea60 0x7fc388156ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:51:57.390 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:57.376+0000 7fc387fff640 1 -- 172.21.15.43:0/456916313 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fc3880aff80 msgr2=0x7fc3880ab8d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:51:57.390 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:57.376+0000 7fc387fff640 1 --2- 172.21.15.43:0/456916313 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fc3880aff80 0x7fc3880ab8d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:51:57.390 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:57.376+0000 7fc387fff640 1 -- 172.21.15.43:0/456916313 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc380009400 con 0x7fc38815a330 2023-08-09T13:51:57.390 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:57.376+0000 7fc387fff640 1 --2- 172.21.15.43:0/456916313 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc38815a330 0x7fc3881f1380 secure :-1 s=READY pgs=1324 cs=0 l=1 rev1=1 crypto rx=0x7fc37c00c500 tx=0x7fc37c00c9c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-09T13:51:57.390 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:57.377+0000 7fc3857fa640 1 -- 172.21.15.43:0/456916313 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc37c005ce0 con 0x7fc38815a330 2023-08-09T13:51:57.390 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:57.377+0000 7fc3857fa640 1 -- 172.21.15.43:0/456916313 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc37c005e40 con 0x7fc38815a330 2023-08-09T13:51:57.390 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:57.377+0000 7fc3857fa640 1 -- 172.21.15.43:0/456916313 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc37c007330 con 0x7fc38815a330 2023-08-09T13:51:57.390 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:57.377+0000 7fc38dbdb640 1 -- 172.21.15.43:0/456916313 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc3880ac790 con 0x7fc38815a330 2023-08-09T13:51:57.391 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:57.377+0000 7fc38dbdb640 1 -- 172.21.15.43:0/456916313 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc3880c50e0 con 0x7fc38815a330 2023-08-09T13:51:57.391 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:57.378+0000 7fc38dbdb640 1 -- 172.21.15.43:0/456916313 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc3880739f0 con 0x7fc38815a330 2023-08-09T13:51:57.391 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:57.379+0000 7fc3857fa640 1 -- 172.21.15.43:0/456916313 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 45) v1 ==== 88443+0+0 (secure 0 0 0) 0x7fc37c007490 con 0x7fc38815a330 2023-08-09T13:51:57.391 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:57.380+0000 7fc3857fa640 1 --2- 172.21.15.43:0/456916313 >> [v2:172.21.15.203:6832/1317108840,v1:172.21.15.203:6833/1317108840] conn(0x7fc37406a1a0 0x7fc37406c660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:51:57.391 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:57.380+0000 7fc3857fa640 1 -- 172.21.15.43:0/456916313 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(112..112 src has 1..112) v4 ==== 5574+0+0 (secure 0 0 0) 0x7fc37c087aa0 con 0x7fc38815a330 2023-08-09T13:51:57.391 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:57.381+0000 7fc38cbd9640 1 --2- 172.21.15.43:0/456916313 >> [v2:172.21.15.203:6832/1317108840,v1:172.21.15.203:6833/1317108840] conn(0x7fc37406a1a0 0x7fc37406c660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:51:57.391 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:57.382+0000 7fc38cbd9640 1 --2- 172.21.15.43:0/456916313 >> [v2:172.21.15.203:6832/1317108840,v1:172.21.15.203:6833/1317108840] conn(0x7fc37406a1a0 0x7fc37406c660 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7fc378005520 tx=0x7fc378009040 comp rx=0 tx=0).ready entity=mgr.6619 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-09T13:51:57.392 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:57.385+0000 7fc3857fa640 1 -- 172.21.15.43:0/456916313 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178066 (secure 0 0 0) 0x7fc37c0533b0 con 0x7fc38815a330 2023-08-09T13:51:57.580 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:57.579+0000 7fc38dbdb640 1 -- 172.21.15.43:0/456916313 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64} v 0) v1 -- 0x7fc38806c550 con 0x7fc38815a330 2023-08-09T13:51:58.139 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:58.137+0000 7fc3857fa640 1 -- 172.21.15.43:0/456916313 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]=0 pool 'cephfs_metadata' created v113) v1 ==== 138+0+0 (secure 0 0 0) 0x7fc37c017020 con 0x7fc38815a330 2023-08-09T13:51:58.141 INFO:teuthology.orchestra.run.smithi043.stderr:pool 'cephfs_metadata' created 2023-08-09T13:51:58.149 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:58.147+0000 7fc38dbdb640 1 -- 172.21.15.43:0/456916313 >> [v2:172.21.15.203:6832/1317108840,v1:172.21.15.203:6833/1317108840] conn(0x7fc37406a1a0 msgr2=0x7fc37406c660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:51:58.149 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:58.147+0000 7fc38dbdb640 1 --2- 172.21.15.43:0/456916313 >> [v2:172.21.15.203:6832/1317108840,v1:172.21.15.203:6833/1317108840] conn(0x7fc37406a1a0 0x7fc37406c660 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7fc378005520 tx=0x7fc378009040 comp rx=0 tx=0).stop 2023-08-09T13:51:58.149 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:58.147+0000 7fc38dbdb640 1 -- 172.21.15.43:0/456916313 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc38815a330 msgr2=0x7fc3881f1380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:51:58.149 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:58.147+0000 7fc38dbdb640 1 --2- 172.21.15.43:0/456916313 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc38815a330 0x7fc3881f1380 secure :-1 s=READY pgs=1324 cs=0 l=1 rev1=1 crypto rx=0x7fc37c00c500 tx=0x7fc37c00c9c0 comp rx=0 tx=0).stop 2023-08-09T13:51:58.150 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:58.148+0000 7fc38dbdb640 1 -- 172.21.15.43:0/456916313 shutdown_connections 2023-08-09T13:51:58.151 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:58.148+0000 7fc38dbdb640 1 --2- 172.21.15.43:0/456916313 >> [v2:172.21.15.203:6832/1317108840,v1:172.21.15.203:6833/1317108840] conn(0x7fc37406a1a0 0x7fc37406c660 unknown :-1 s=CLOSED pgs=56 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:51:58.151 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:58.149+0000 7fc38dbdb640 1 --2- 172.21.15.43:0/456916313 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fc38815ea60 0x7fc388156ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:51:58.151 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:58.149+0000 7fc38dbdb640 1 --2- 172.21.15.43:0/456916313 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc38815a330 0x7fc3881f1380 unknown :-1 s=CLOSED pgs=1324 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:51:58.151 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:58.149+0000 7fc38dbdb640 1 --2- 172.21.15.43:0/456916313 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fc3880aff80 0x7fc3880ab8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:51:58.152 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:58.149+0000 7fc38dbdb640 1 -- 172.21.15.43:0/456916313 >> 172.21.15.43:0/456916313 conn(0x7fc3880ca870 msgr2=0x7fc388159550 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-09T13:51:58.152 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:58.150+0000 7fc38dbdb640 1 -- 172.21.15.43:0/456916313 shutdown_connections 2023-08-09T13:51:58.152 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:58.150+0000 7fc38dbdb640 1 -- 172.21.15.43:0/456916313 wait complete. 2023-08-09T13:51:58.173 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data 64 --pg_num_min 64 --target_size_ratio 0.9 2023-08-09T13:51:58.402 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:58.400+0000 7f5120953640 1 Processor -- start 2023-08-09T13:51:58.402 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:58.400+0000 7f5120953640 1 -- start start 2023-08-09T13:51:58.404 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:58.402+0000 7f5120953640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f511c15f970 0x7f511c0cf1d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:51:58.404 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:58.402+0000 7f5120953640 1 -- --> v1:172.21.15.43:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f511c0cfb70 con 0x7f511c0cf710 2023-08-09T13:51:58.404 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:58.402+0000 7f5120953640 1 -- --> v1:172.21.15.203:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f511c0d24a0 con 0x7f511c0d1ec0 2023-08-09T13:51:58.405 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:58.402+0000 7f5120953640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f511c0d1280 con 0x7f511c15f970 2023-08-09T13:51:58.406 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:58.403+0000 7f511affd640 1 --1- >> v1:172.21.15.43:6789/0 conn(0x7f511c0cf710 0x7f511c0d17b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.43:6789/0 says I am v1:172.21.15.43:43732/0 (socket says 172.21.15.43:43732) 2023-08-09T13:51:58.406 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:58.403+0000 7f511affd640 1 -- 172.21.15.43:0/894056519 learned_addr learned my addr 172.21.15.43:0/894056519 (peer_addr_for_me v1:172.21.15.43:0/0) 2023-08-09T13:51:58.406 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:58.403+0000 7f511b7fe640 1 --2- 172.21.15.43:0/894056519 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f511c15f970 0x7f511c0cf1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:51:58.406 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:58.404+0000 7f511a7fc640 1 -- 172.21.15.43:0/894056519 <== mon.0 v1:172.21.15.43:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 912146161 0 0) 0x7f511c0cfb70 con 0x7f511c0cf710 2023-08-09T13:51:58.406 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:58.404+0000 7f511a7fc640 1 -- 172.21.15.43:0/894056519 --> v1:172.21.15.43:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f5104003520 con 0x7f511c0cf710 2023-08-09T13:51:58.406 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:58.404+0000 7f511a7fc640 1 -- 172.21.15.43:0/894056519 <== mon.1 v1:172.21.15.203:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 29321338 0 0) 0x7f511c0d24a0 con 0x7f511c0d1ec0 2023-08-09T13:51:58.407 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:58.404+0000 7f511a7fc640 1 -- 172.21.15.43:0/894056519 --> v1:172.21.15.203:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f511c0cfb70 con 0x7f511c0d1ec0 2023-08-09T13:51:58.407 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:58.404+0000 7f511a7fc640 1 -- 172.21.15.43:0/894056519 <== mon.0 v1:172.21.15.43:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1027726782 0 0) 0x7f5104003520 con 0x7f511c0cf710 2023-08-09T13:51:58.407 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:58.404+0000 7f511b7fe640 1 -- 172.21.15.43:0/894056519 >> v1:172.21.15.203:6789/0 conn(0x7f511c0d1ec0 legacy=0x7f511c0b5670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:51:58.407 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:58.404+0000 7f511b7fe640 1 -- 172.21.15.43:0/894056519 >> v1:172.21.15.43:6789/0 conn(0x7f511c0cf710 legacy=0x7f511c0d17b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:51:58.407 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:58.404+0000 7f511b7fe640 1 -- 172.21.15.43:0/894056519 --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f511c0d2780 con 0x7f511c15f970 2023-08-09T13:51:58.408 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:58.406+0000 7f511b7fe640 1 --2- 172.21.15.43:0/894056519 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f511c15f970 0x7f511c0cf1d0 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7f510c0097d0 tx=0x7f510c02b800 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ce983193195ff1e5 server_cookie=0 in_seq=0 out_seq=0 2023-08-09T13:51:58.408 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:58.406+0000 7f511a7fc640 1 -- 172.21.15.43:0/894056519 <== mon.2 v2:172.21.15.203:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5110001670 con 0x7f511c15f970 2023-08-09T13:51:58.408 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:58.406+0000 7f511a7fc640 1 -- 172.21.15.43:0/894056519 <== mon.2 v2:172.21.15.203:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f510c02e050 con 0x7f511c15f970 2023-08-09T13:51:58.408 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:58.406+0000 7f511a7fc640 1 -- 172.21.15.43:0/894056519 <== mon.2 v2:172.21.15.203:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f510c030730 con 0x7f511c15f970 2023-08-09T13:51:58.409 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:58.407+0000 7f5120953640 1 -- 172.21.15.43:0/894056519 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f511c15f970 msgr2=0x7f511c0cf1d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:51:58.409 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:58.407+0000 7f5120953640 1 --2- 172.21.15.43:0/894056519 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f511c15f970 0x7f511c0cf1d0 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7f510c0097d0 tx=0x7f510c02b800 comp rx=0 tx=0).stop 2023-08-09T13:51:58.409 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:58.407+0000 7f5120953640 1 -- 172.21.15.43:0/894056519 shutdown_connections 2023-08-09T13:51:58.409 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:58.407+0000 7f5120953640 1 --2- 172.21.15.43:0/894056519 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f511c15f970 0x7f511c0cf1d0 unknown :-1 s=CLOSED pgs=183 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:51:58.409 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:58.407+0000 7f5120953640 1 -- 172.21.15.43:0/894056519 >> 172.21.15.43:0/894056519 conn(0x7f511c0712e0 msgr2=0x7f511c0716f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-09T13:51:58.409 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:58.407+0000 7f5120953640 1 -- 172.21.15.43:0/894056519 shutdown_connections 2023-08-09T13:51:58.410 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:58.408+0000 7f5120953640 1 -- 172.21.15.43:0/894056519 wait complete. 2023-08-09T13:51:58.411 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:58.410+0000 7f5120953640 1 Processor -- start 2023-08-09T13:51:58.411 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:58.410+0000 7f5120953640 1 -- start start 2023-08-09T13:51:58.413 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:58.411+0000 7f5120953640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f511c0cf710 0x7f511c1596f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:51:58.413 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:58.411+0000 7f5120953640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f511c0d1ec0 0x7f511c0ca230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:51:58.413 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:58.412+0000 7f5120953640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f511c15f970 0x7f511c0a9690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:51:58.413 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:58.412+0000 7f5120953640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f511c0c6a30 con 0x7f511c0cf710 2023-08-09T13:51:58.414 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:58.412+0000 7f5120953640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f511c0c5e20 con 0x7f511c15f970 2023-08-09T13:51:58.414 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:58.412+0000 7f5120953640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f511c0c6100 con 0x7f511c0d1ec0 2023-08-09T13:51:58.414 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:58.412+0000 7f511b7fe640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f511c0cf710 0x7f511c1596f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:51:58.414 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:58.412+0000 7f511b7fe640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f511c0cf710 0x7f511c1596f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:33022/0 (socket says 172.21.15.43:33022) 2023-08-09T13:51:58.414 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:58.412+0000 7f511b7fe640 1 -- 172.21.15.43:0/838159024 learned_addr learned my addr 172.21.15.43:0/838159024 (peer_addr_for_me v2:172.21.15.43:0/0) 2023-08-09T13:51:58.415 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:58.412+0000 7f511affd640 1 --2- 172.21.15.43:0/838159024 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f511c0d1ec0 0x7f511c0ca230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:51:58.415 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:58.412+0000 7f511bfff640 1 --2- 172.21.15.43:0/838159024 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f511c15f970 0x7f511c0a9690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:51:58.415 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:58.412+0000 7f511b7fe640 1 -- 172.21.15.43:0/838159024 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f511c0d1ec0 msgr2=0x7f511c0ca230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:51:58.416 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:58.412+0000 7f511b7fe640 1 --2- 172.21.15.43:0/838159024 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f511c0d1ec0 0x7f511c0ca230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:51:58.416 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:58.412+0000 7f511b7fe640 1 -- 172.21.15.43:0/838159024 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f511c15f970 msgr2=0x7f511c0a9690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:51:58.416 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:58.412+0000 7f511b7fe640 1 --2- 172.21.15.43:0/838159024 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f511c15f970 0x7f511c0a9690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:51:58.416 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:58.412+0000 7f511b7fe640 1 -- 172.21.15.43:0/838159024 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f510c009400 con 0x7f511c0cf710 2023-08-09T13:51:58.416 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:58.413+0000 7f511b7fe640 1 --2- 172.21.15.43:0/838159024 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f511c0cf710 0x7f511c1596f0 secure :-1 s=READY pgs=1326 cs=0 l=1 rev1=1 crypto rx=0x7f510c000c00 tx=0x7f510c0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-09T13:51:58.416 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:58.413+0000 7f5103fff640 1 -- 172.21.15.43:0/838159024 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f510c004230 con 0x7f511c0cf710 2023-08-09T13:51:58.416 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:58.413+0000 7f5103fff640 1 -- 172.21.15.43:0/838159024 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f510c0043b0 con 0x7f511c0cf710 2023-08-09T13:51:58.417 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:58.413+0000 7f5103fff640 1 -- 172.21.15.43:0/838159024 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f510c004520 con 0x7f511c0cf710 2023-08-09T13:51:58.417 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:58.413+0000 7f5120953640 1 -- 172.21.15.43:0/838159024 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f511c0c63e0 con 0x7f511c0cf710 2023-08-09T13:51:58.417 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:58.414+0000 7f5120953640 1 -- 172.21.15.43:0/838159024 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f511c159c30 con 0x7f511c0cf710 2023-08-09T13:51:58.417 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:58.415+0000 7f5120953640 1 -- 172.21.15.43:0/838159024 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f511c04e390 con 0x7f511c0cf710 2023-08-09T13:51:58.418 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:58.417+0000 7f5103fff640 1 -- 172.21.15.43:0/838159024 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 45) v1 ==== 88443+0+0 (secure 0 0 0) 0x7f510c039040 con 0x7f511c0cf710 2023-08-09T13:51:58.419 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:58.418+0000 7f5103fff640 1 --2- 172.21.15.43:0/838159024 >> [v2:172.21.15.203:6832/1317108840,v1:172.21.15.203:6833/1317108840] conn(0x7f510806a170 0x7f510806c630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:51:58.420 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:58.418+0000 7f5103fff640 1 -- 172.21.15.43:0/838159024 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(113..113 src has 1..113) v4 ==== 5944+0+0 (secure 0 0 0) 0x7f510c0aa1e0 con 0x7f511c0cf710 2023-08-09T13:51:58.420 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:58.419+0000 7f511affd640 1 --2- 172.21.15.43:0/838159024 >> [v2:172.21.15.203:6832/1317108840,v1:172.21.15.203:6833/1317108840] conn(0x7f510806a170 0x7f510806c630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:51:58.421 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:58.420+0000 7f511affd640 1 --2- 172.21.15.43:0/838159024 >> [v2:172.21.15.203:6832/1317108840,v1:172.21.15.203:6833/1317108840] conn(0x7f510806a170 0x7f510806c630 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f511c0b82e0 tx=0x7f5110009040 comp rx=0 tx=0).ready entity=mgr.6619 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-09T13:51:58.422 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:58.421+0000 7f5103fff640 1 -- 172.21.15.43:0/838159024 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178066 (secure 0 0 0) 0x7f510c0758b0 con 0x7f511c0cf710 2023-08-09T13:51:58.625 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:58.623+0000 7f5120953640 1 -- 172.21.15.43:0/838159024 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9} v 0) v1 -- 0x7f511c0b2cf0 con 0x7f511c0cf710 2023-08-09T13:51:59.162 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:59.159+0000 7f5103fff640 1 -- 172.21.15.43:0/838159024 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]=0 pool 'cephfs_data' created v114) v1 ==== 170+0+0 (secure 0 0 0) 0x7f510c0370d0 con 0x7f511c0cf710 2023-08-09T13:51:59.166 INFO:teuthology.orchestra.run.smithi043.stderr:pool 'cephfs_data' created 2023-08-09T13:51:59.172 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:59.171+0000 7f5120953640 1 -- 172.21.15.43:0/838159024 >> [v2:172.21.15.203:6832/1317108840,v1:172.21.15.203:6833/1317108840] conn(0x7f510806a170 msgr2=0x7f510806c630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:51:59.172 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:59.171+0000 7f5120953640 1 --2- 172.21.15.43:0/838159024 >> [v2:172.21.15.203:6832/1317108840,v1:172.21.15.203:6833/1317108840] conn(0x7f510806a170 0x7f510806c630 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f511c0b82e0 tx=0x7f5110009040 comp rx=0 tx=0).stop 2023-08-09T13:51:59.173 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:59.171+0000 7f5120953640 1 -- 172.21.15.43:0/838159024 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f511c0cf710 msgr2=0x7f511c1596f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:51:59.173 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:59.171+0000 7f5120953640 1 --2- 172.21.15.43:0/838159024 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f511c0cf710 0x7f511c1596f0 secure :-1 s=READY pgs=1326 cs=0 l=1 rev1=1 crypto rx=0x7f510c000c00 tx=0x7f510c0046c0 comp rx=0 tx=0).stop 2023-08-09T13:51:59.175 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:59.174+0000 7f5120953640 1 -- 172.21.15.43:0/838159024 shutdown_connections 2023-08-09T13:51:59.176 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:59.174+0000 7f5120953640 1 --2- 172.21.15.43:0/838159024 >> [v2:172.21.15.203:6832/1317108840,v1:172.21.15.203:6833/1317108840] conn(0x7f510806a170 0x7f510806c630 unknown :-1 s=CLOSED pgs=57 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:51:59.177 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:59.174+0000 7f5120953640 1 --2- 172.21.15.43:0/838159024 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f511c15f970 0x7f511c0a9690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:51:59.177 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:59.174+0000 7f5120953640 1 --2- 172.21.15.43:0/838159024 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f511c0d1ec0 0x7f511c0ca230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:51:59.177 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:59.175+0000 7f5120953640 1 --2- 172.21.15.43:0/838159024 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f511c0cf710 0x7f511c1596f0 unknown :-1 s=CLOSED pgs=1326 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:51:59.177 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:59.175+0000 7f5120953640 1 -- 172.21.15.43:0/838159024 >> 172.21.15.43:0/838159024 conn(0x7f511c0712e0 msgr2=0x7f511c06a2e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-09T13:51:59.179 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:59.178+0000 7f5120953640 1 -- 172.21.15.43:0/838159024 shutdown_connections 2023-08-09T13:51:59.181 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:59.180+0000 7f5120953640 1 -- 172.21.15.43:0/838159024 wait complete. 2023-08-09T13:51:59.202 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs new cephfs cephfs_metadata cephfs_data 2023-08-09T13:51:59.403 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:59.402+0000 7ff64a4ba640 1 Processor -- start 2023-08-09T13:51:59.403 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:59.402+0000 7ff64a4ba640 1 -- start start 2023-08-09T13:51:59.405 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:59.403+0000 7ff64a4ba640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7ff64415c550 0x7ff6440cd6d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:51:59.405 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:59.403+0000 7ff64a4ba640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7ff6440d04b0 0x7ff6440b3ee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:51:59.405 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:59.403+0000 7ff64a4ba640 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7ff6440b4420 0x7ff64406a760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:51:59.405 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:59.403+0000 7ff64a4ba640 1 -- --> v2:172.21.15.43:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff6440b4870 con 0x7ff64415c550 2023-08-09T13:51:59.406 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:59.403+0000 7ff64a4ba640 1 -- --> v2:172.21.15.203:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff6440b7bd0 con 0x7ff6440b4420 2023-08-09T13:51:59.406 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:59.403+0000 7ff64a4ba640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff6440d1c00 con 0x7ff6440d04b0 2023-08-09T13:51:59.407 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:59.404+0000 7ff643fff640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7ff64415c550 0x7ff6440cd6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:51:59.407 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:59.404+0000 7ff643fff640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7ff64415c550 0x7ff6440cd6d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:33026/0 (socket says 172.21.15.43:33026) 2023-08-09T13:51:59.407 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:59.404+0000 7ff643fff640 1 -- 172.21.15.43:0/1357164505 learned_addr learned my addr 172.21.15.43:0/1357164505 (peer_addr_for_me v2:172.21.15.43:0/0) 2023-08-09T13:51:59.407 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:59.404+0000 7ff648a57640 1 --2- 172.21.15.43:0/1357164505 >> v2:172.21.15.203:3300/0 conn(0x7ff6440b4420 0x7ff64406a760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:51:59.407 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:59.404+0000 7ff6437fe640 1 --2- 172.21.15.43:0/1357164505 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7ff6440d04b0 0x7ff6440b3ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:51:59.407 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:59.404+0000 7ff643fff640 1 -- 172.21.15.43:0/1357164505 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7ff6440d04b0 msgr2=0x7ff6440b3ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-09T13:51:59.407 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:59.404+0000 7ff643fff640 1 --2- 172.21.15.43:0/1357164505 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7ff6440d04b0 0x7ff6440b3ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:51:59.408 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:59.404+0000 7ff643fff640 1 -- 172.21.15.43:0/1357164505 >> v2:172.21.15.203:3300/0 conn(0x7ff6440b4420 msgr2=0x7ff64406a760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-09T13:51:59.408 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:59.405+0000 7ff643fff640 1 --2- 172.21.15.43:0/1357164505 >> v2:172.21.15.203:3300/0 conn(0x7ff6440b4420 0x7ff64406a760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:51:59.408 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:59.405+0000 7ff643fff640 1 -- 172.21.15.43:0/1357164505 --> v2:172.21.15.43:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff64415c930 con 0x7ff64415c550 2023-08-09T13:51:59.408 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:59.405+0000 7ff648a57640 1 --2- 172.21.15.43:0/1357164505 >> v2:172.21.15.203:3300/0 conn(0x7ff6440b4420 0x7ff64406a760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-08-09T13:51:59.408 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:59.405+0000 7ff6437fe640 1 --2- 172.21.15.43:0/1357164505 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7ff6440d04b0 0x7ff6440b3ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-08-09T13:51:59.409 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:59.406+0000 7ff643fff640 1 --2- 172.21.15.43:0/1357164505 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff64415c550 0x7ff6440cd6d0 secure :-1 s=READY pgs=1327 cs=0 l=1 rev1=1 crypto rx=0x7ff6440cee90 tx=0x7ff63402bee0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=442bc7959bca7e3d server_cookie=0 in_seq=0 out_seq=0 2023-08-09T13:51:59.409 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:59.406+0000 7ff642ffd640 1 -- 172.21.15.43:0/1357164505 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff63402ca80 con 0x7ff64415c550 2023-08-09T13:51:59.409 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:59.406+0000 7ff642ffd640 1 -- 172.21.15.43:0/1357164505 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff63402cbe0 con 0x7ff64415c550 2023-08-09T13:51:59.410 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:59.406+0000 7ff642ffd640 1 -- 172.21.15.43:0/1357164505 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff6340343f0 con 0x7ff64415c550 2023-08-09T13:51:59.410 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:59.407+0000 7ff64a4ba640 1 -- 172.21.15.43:0/1357164505 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff64415c550 msgr2=0x7ff6440cd6d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:51:59.410 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:59.407+0000 7ff64a4ba640 1 --2- 172.21.15.43:0/1357164505 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff64415c550 0x7ff6440cd6d0 secure :-1 s=READY pgs=1327 cs=0 l=1 rev1=1 crypto rx=0x7ff6440cee90 tx=0x7ff63402bee0 comp rx=0 tx=0).stop 2023-08-09T13:51:59.410 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:59.407+0000 7ff64a4ba640 1 -- 172.21.15.43:0/1357164505 shutdown_connections 2023-08-09T13:51:59.410 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:59.407+0000 7ff64a4ba640 1 --2- 172.21.15.43:0/1357164505 >> v2:172.21.15.203:3300/0 conn(0x7ff6440b4420 0x7ff64406a760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:51:59.411 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:59.407+0000 7ff64a4ba640 1 --2- 172.21.15.43:0/1357164505 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7ff6440d04b0 0x7ff6440b3ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:51:59.411 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:59.407+0000 7ff64a4ba640 1 --2- 172.21.15.43:0/1357164505 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff64415c550 0x7ff6440cd6d0 unknown :-1 s=CLOSED pgs=1327 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:51:59.411 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:59.407+0000 7ff64a4ba640 1 -- 172.21.15.43:0/1357164505 >> 172.21.15.43:0/1357164505 conn(0x7ff644071520 msgr2=0x7ff6440ceee0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-09T13:51:59.412 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:59.407+0000 7ff64a4ba640 1 -- 172.21.15.43:0/1357164505 shutdown_connections 2023-08-09T13:51:59.412 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:59.407+0000 7ff64a4ba640 1 -- 172.21.15.43:0/1357164505 wait complete. 2023-08-09T13:51:59.415 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:59.409+0000 7ff64a4ba640 1 Processor -- start 2023-08-09T13:51:59.416 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:59.409+0000 7ff64a4ba640 1 -- start start 2023-08-09T13:51:59.416 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:59.410+0000 7ff64a4ba640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff6440b4420 0x7ff64415a970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:51:59.416 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:59.410+0000 7ff64a4ba640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7ff6440d04b0 0x7ff644156280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:51:59.417 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:59.410+0000 7ff64a4ba640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7ff64415c550 0x7ff6441567c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:51:59.417 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:59.410+0000 7ff64a4ba640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff6440aab10 con 0x7ff6440b4420 2023-08-09T13:51:59.417 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:59.410+0000 7ff64a4ba640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff6440a8dc0 con 0x7ff64415c550 2023-08-09T13:51:59.417 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:59.410+0000 7ff64a4ba640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff6441f5ad0 con 0x7ff6440d04b0 2023-08-09T13:51:59.418 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:59.410+0000 7ff643fff640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff6440b4420 0x7ff64415a970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:51:59.418 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:59.411+0000 7ff643fff640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff6440b4420 0x7ff64415a970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:33042/0 (socket says 172.21.15.43:33042) 2023-08-09T13:51:59.418 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:59.411+0000 7ff643fff640 1 -- 172.21.15.43:0/3352172891 learned_addr learned my addr 172.21.15.43:0/3352172891 (peer_addr_for_me v2:172.21.15.43:0/0) 2023-08-09T13:51:59.419 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:59.411+0000 7ff6437fe640 1 --2- 172.21.15.43:0/3352172891 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7ff6440d04b0 0x7ff644156280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:51:59.419 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:59.411+0000 7ff648a57640 1 --2- 172.21.15.43:0/3352172891 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7ff64415c550 0x7ff6441567c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:51:59.419 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:59.411+0000 7ff643fff640 1 -- 172.21.15.43:0/3352172891 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7ff6440d04b0 msgr2=0x7ff644156280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:51:59.419 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:59.411+0000 7ff643fff640 1 --2- 172.21.15.43:0/3352172891 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7ff6440d04b0 0x7ff644156280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:51:59.419 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:59.411+0000 7ff643fff640 1 -- 172.21.15.43:0/3352172891 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7ff64415c550 msgr2=0x7ff6441567c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:51:59.420 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:59.411+0000 7ff643fff640 1 --2- 172.21.15.43:0/3352172891 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7ff64415c550 0x7ff6441567c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:51:59.420 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:59.411+0000 7ff643fff640 1 -- 172.21.15.43:0/3352172891 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff634009b30 con 0x7ff6440b4420 2023-08-09T13:51:59.420 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:59.411+0000 7ff648a57640 1 --2- 172.21.15.43:0/3352172891 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7ff64415c550 0x7ff6441567c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-08-09T13:51:59.420 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:59.411+0000 7ff643fff640 1 --2- 172.21.15.43:0/3352172891 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff6440b4420 0x7ff64415a970 secure :-1 s=READY pgs=1328 cs=0 l=1 rev1=1 crypto rx=0x7ff634036040 tx=0x7ff634004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-09T13:51:59.421 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:59.412+0000 7ff640ff9640 1 -- 172.21.15.43:0/3352172891 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff63403e450 con 0x7ff6440b4420 2023-08-09T13:51:59.421 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:59.412+0000 7ff640ff9640 1 -- 172.21.15.43:0/3352172891 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff63403e5f0 con 0x7ff6440b4420 2023-08-09T13:51:59.421 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:59.412+0000 7ff640ff9640 1 -- 172.21.15.43:0/3352172891 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff63403e9a0 con 0x7ff6440b4420 2023-08-09T13:51:59.422 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:59.412+0000 7ff64a4ba640 1 -- 172.21.15.43:0/3352172891 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff6440a9010 con 0x7ff6440b4420 2023-08-09T13:51:59.422 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:59.412+0000 7ff64a4ba640 1 -- 172.21.15.43:0/3352172891 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff6440a95d0 con 0x7ff6440b4420 2023-08-09T13:51:59.422 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:59.413+0000 7ff64a4ba640 1 -- 172.21.15.43:0/3352172891 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff6440739f0 con 0x7ff6440b4420 2023-08-09T13:51:59.423 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:59.415+0000 7ff640ff9640 1 -- 172.21.15.43:0/3352172891 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 45) v1 ==== 88443+0+0 (secure 0 0 0) 0x7ff63402cd50 con 0x7ff6440b4420 2023-08-09T13:51:59.423 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:59.416+0000 7ff640ff9640 1 --2- 172.21.15.43:0/3352172891 >> [v2:172.21.15.203:6832/1317108840,v1:172.21.15.203:6833/1317108840] conn(0x7ff63006a170 0x7ff63006c630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:51:59.423 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:59.416+0000 7ff640ff9640 1 -- 172.21.15.43:0/3352172891 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(114..114 src has 1..114) v4 ==== 6326+0+0 (secure 0 0 0) 0x7ff6340ab8e0 con 0x7ff6440b4420 2023-08-09T13:51:59.424 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:59.417+0000 7ff6437fe640 1 --2- 172.21.15.43:0/3352172891 >> [v2:172.21.15.203:6832/1317108840,v1:172.21.15.203:6833/1317108840] conn(0x7ff63006a170 0x7ff63006c630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:51:59.424 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:59.418+0000 7ff6437fe640 1 --2- 172.21.15.43:0/3352172891 >> [v2:172.21.15.203:6832/1317108840,v1:172.21.15.203:6833/1317108840] conn(0x7ff63006a170 0x7ff63006c630 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7ff6440cae90 tx=0x7ff638008040 comp rx=0 tx=0).ready entity=mgr.6619 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-09T13:51:59.424 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:59.420+0000 7ff640ff9640 1 -- 172.21.15.43:0/3352172891 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178066 (secure 0 0 0) 0x7ff634076eb0 con 0x7ff6440b4420 2023-08-09T13:51:59.655 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:51:59.653+0000 7ff64a4ba640 1 -- 172.21.15.43:0/3352172891 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"} v 0) v1 -- 0x7ff644158ed0 con 0x7ff6440b4420 2023-08-09T13:52:00.221 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:52:00.219+0000 7ff640ff9640 1 -- 172.21.15.43:0/3352172891 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]=0 new fs with metadata pool 14 and data pool 15 v48) v1 ==== 174+0+0 (secure 0 0 0) 0x7ff634033e80 con 0x7ff6440b4420 2023-08-09T13:52:00.225 INFO:teuthology.orchestra.run.smithi043.stderr:new fs with metadata pool 14 and data pool 15 2023-08-09T13:52:00.228 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:52:00.226+0000 7ff64a4ba640 1 -- 172.21.15.43:0/3352172891 >> [v2:172.21.15.203:6832/1317108840,v1:172.21.15.203:6833/1317108840] conn(0x7ff63006a170 msgr2=0x7ff63006c630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:52:00.228 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:52:00.226+0000 7ff64a4ba640 1 --2- 172.21.15.43:0/3352172891 >> [v2:172.21.15.203:6832/1317108840,v1:172.21.15.203:6833/1317108840] conn(0x7ff63006a170 0x7ff63006c630 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7ff6440cae90 tx=0x7ff638008040 comp rx=0 tx=0).stop 2023-08-09T13:52:00.228 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:52:00.226+0000 7ff64a4ba640 1 -- 172.21.15.43:0/3352172891 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff6440b4420 msgr2=0x7ff64415a970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:52:00.228 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:52:00.226+0000 7ff64a4ba640 1 --2- 172.21.15.43:0/3352172891 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff6440b4420 0x7ff64415a970 secure :-1 s=READY pgs=1328 cs=0 l=1 rev1=1 crypto rx=0x7ff634036040 tx=0x7ff634004a40 comp rx=0 tx=0).stop 2023-08-09T13:52:00.230 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:52:00.228+0000 7ff64a4ba640 1 -- 172.21.15.43:0/3352172891 shutdown_connections 2023-08-09T13:52:00.230 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:52:00.228+0000 7ff64a4ba640 1 --2- 172.21.15.43:0/3352172891 >> [v2:172.21.15.203:6832/1317108840,v1:172.21.15.203:6833/1317108840] conn(0x7ff63006a170 0x7ff63006c630 unknown :-1 s=CLOSED pgs=58 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:52:00.230 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:52:00.229+0000 7ff64a4ba640 1 --2- 172.21.15.43:0/3352172891 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7ff64415c550 0x7ff6441567c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:52:00.231 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:52:00.229+0000 7ff64a4ba640 1 --2- 172.21.15.43:0/3352172891 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7ff6440d04b0 0x7ff644156280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:52:00.231 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:52:00.229+0000 7ff64a4ba640 1 --2- 172.21.15.43:0/3352172891 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff6440b4420 0x7ff64415a970 unknown :-1 s=CLOSED pgs=1328 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:52:00.231 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:52:00.229+0000 7ff64a4ba640 1 -- 172.21.15.43:0/3352172891 >> 172.21.15.43:0/3352172891 conn(0x7ff644071520 msgr2=0x7ff6440d2440 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-09T13:52:00.240 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:52:00.238+0000 7ff64a4ba640 1 -- 172.21.15.43:0/3352172891 shutdown_connections 2023-08-09T13:52:00.241 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:52:00.239+0000 7ff64a4ba640 1 -- 172.21.15.43:0/3352172891 wait complete. 2023-08-09T13:52:00.261 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-08-09T13:52:00.453 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:52:00.451+0000 7f1bd37ab640 1 Processor -- start 2023-08-09T13:52:00.453 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:52:00.452+0000 7f1bd37ab640 1 -- start start 2023-08-09T13:52:00.454 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:52:00.453+0000 7f1bd37ab640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f1bcc0b2b70 0x7f1bcc0d2ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:52:00.454 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:52:00.453+0000 7f1bd37ab640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f1bcc0b3050 0x7f1bcc0b3f60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:52:00.455 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:52:00.453+0000 7f1bd37ab640 1 -- --> v1:172.21.15.43:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1bcc0ce090 con 0x7f1bcc0b2790 2023-08-09T13:52:00.455 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:52:00.453+0000 7f1bd37ab640 1 -- --> v2:172.21.15.43:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1bcc0b3950 con 0x7f1bcc0b2b70 2023-08-09T13:52:00.455 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:52:00.453+0000 7f1bd37ab640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1bcc0cd7f0 con 0x7f1bcc0b3050 2023-08-09T13:52:00.456 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:52:00.453+0000 7f1bd0d46640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f1bcc0b2b70 0x7f1bcc0d2ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:52:00.456 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:52:00.453+0000 7f1bd0d46640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f1bcc0b2b70 0x7f1bcc0d2ed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:33046/0 (socket says 172.21.15.43:33046) 2023-08-09T13:52:00.456 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:52:00.453+0000 7f1bd1547640 1 --1- >> v1:172.21.15.43:6789/0 conn(0x7f1bcc0b2790 0x7f1bcc0b5330 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.43:6789/0 says I am v1:172.21.15.43:43734/0 (socket says 172.21.15.43:43734) 2023-08-09T13:52:00.456 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:52:00.453+0000 7f1bd0d46640 1 -- 172.21.15.43:0/757446208 learned_addr learned my addr 172.21.15.43:0/757446208 (peer_addr_for_me v2:172.21.15.43:0/0) 2023-08-09T13:52:00.456 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:52:00.454+0000 7f1bd1d48640 1 --2- 172.21.15.43:0/757446208 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f1bcc0b3050 0x7f1bcc0b3f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:52:00.456 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:52:00.454+0000 7f1bd0d46640 1 -- 172.21.15.43:0/757446208 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f1bcc0b3050 msgr2=0x7f1bcc0b3f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-09T13:52:00.457 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:52:00.454+0000 7f1bd0d46640 1 --2- 172.21.15.43:0/757446208 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f1bcc0b3050 0x7f1bcc0b3f60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:52:00.457 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:52:00.454+0000 7f1bd0d46640 1 -- 172.21.15.43:0/757446208 >> v1:172.21.15.43:6789/0 conn(0x7f1bcc0b2790 legacy=0x7f1bcc0b5330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:52:00.457 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:52:00.454+0000 7f1bd0d46640 1 -- 172.21.15.43:0/757446208 --> v2:172.21.15.43:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1bcc0cdad0 con 0x7f1bcc0b2b70 2023-08-09T13:52:00.458 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:52:00.455+0000 7f1bd0d46640 1 --2- 172.21.15.43:0/757446208 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f1bcc0b2b70 0x7f1bcc0d2ed0 secure :-1 s=READY pgs=1330 cs=0 l=1 rev1=1 crypto rx=0x7f1bc0009870 tx=0x7f1bc002b980 comp rx=0 tx=0).ready entity=mon.0 client_cookie=546fde3e27115521 server_cookie=0 in_seq=0 out_seq=0 2023-08-09T13:52:00.458 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:52:00.455+0000 7f1bcbfff640 1 -- 172.21.15.43:0/757446208 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1bc0030840 con 0x7f1bcc0b2b70 2023-08-09T13:52:00.458 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:52:00.455+0000 7f1bcbfff640 1 -- 172.21.15.43:0/757446208 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1bc00309a0 con 0x7f1bcc0b2b70 2023-08-09T13:52:00.458 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:52:00.455+0000 7f1bcbfff640 1 -- 172.21.15.43:0/757446208 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1bc0030d30 con 0x7f1bcc0b2b70 2023-08-09T13:52:00.458 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:52:00.456+0000 7f1bd37ab640 1 -- 172.21.15.43:0/757446208 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f1bcc0b2b70 msgr2=0x7f1bcc0d2ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:52:00.458 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:52:00.456+0000 7f1bd37ab640 1 --2- 172.21.15.43:0/757446208 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f1bcc0b2b70 0x7f1bcc0d2ed0 secure :-1 s=READY pgs=1330 cs=0 l=1 rev1=1 crypto rx=0x7f1bc0009870 tx=0x7f1bc002b980 comp rx=0 tx=0).stop 2023-08-09T13:52:00.459 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:52:00.456+0000 7f1bd37ab640 1 -- 172.21.15.43:0/757446208 shutdown_connections 2023-08-09T13:52:00.459 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:52:00.456+0000 7f1bd37ab640 1 --2- 172.21.15.43:0/757446208 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f1bcc0b3050 0x7f1bcc0b3f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:52:00.459 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:52:00.456+0000 7f1bd37ab640 1 --2- 172.21.15.43:0/757446208 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f1bcc0b2b70 0x7f1bcc0d2ed0 unknown :-1 s=CLOSED pgs=1330 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:52:00.459 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:52:00.456+0000 7f1bd37ab640 1 -- 172.21.15.43:0/757446208 >> 172.21.15.43:0/757446208 conn(0x7f1bcc09b7b0 msgr2=0x7f1bcc0d24e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-09T13:52:00.459 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:52:00.456+0000 7f1bd37ab640 1 -- 172.21.15.43:0/757446208 shutdown_connections 2023-08-09T13:52:00.459 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:52:00.456+0000 7f1bd37ab640 1 -- 172.21.15.43:0/757446208 wait complete. 2023-08-09T13:52:00.460 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:52:00.458+0000 7f1bd37ab640 1 Processor -- start 2023-08-09T13:52:00.460 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:52:00.458+0000 7f1bd37ab640 1 -- start start 2023-08-09T13:52:00.460 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:52:00.459+0000 7f1bd37ab640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f1bcc0b2790 0x7f1bcc0c8c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:52:00.461 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:52:00.459+0000 7f1bd37ab640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f1bcc0b2b70 0x7f1bcc154aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:52:00.461 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:52:00.459+0000 7f1bd37ab640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f1bcc0b3050 0x7f1bcc0afc10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:52:00.461 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:52:00.459+0000 7f1bd37ab640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1bcc0c9150 con 0x7f1bcc0b2790 2023-08-09T13:52:00.461 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:52:00.459+0000 7f1bd37ab640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1bcc154fe0 con 0x7f1bcc0b3050 2023-08-09T13:52:00.461 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:52:00.459+0000 7f1bd37ab640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1bcc0b0150 con 0x7f1bcc0b2b70 2023-08-09T13:52:00.462 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:52:00.459+0000 7f1bd1547640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f1bcc0b2790 0x7f1bcc0c8c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:52:00.462 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:52:00.460+0000 7f1bd1547640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f1bcc0b2790 0x7f1bcc0c8c10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:33050/0 (socket says 172.21.15.43:33050) 2023-08-09T13:52:00.462 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:52:00.460+0000 7f1bd1547640 1 -- 172.21.15.43:0/3285726241 learned_addr learned my addr 172.21.15.43:0/3285726241 (peer_addr_for_me v2:172.21.15.43:0/0) 2023-08-09T13:52:00.463 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:52:00.460+0000 7f1bd1d48640 1 --2- 172.21.15.43:0/3285726241 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f1bcc0b3050 0x7f1bcc0afc10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:52:00.463 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:52:00.460+0000 7f1bd1547640 1 -- 172.21.15.43:0/3285726241 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f1bcc0b2b70 msgr2=0x7f1bcc154aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:52:00.463 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:52:00.460+0000 7f1bd0d46640 1 --2- 172.21.15.43:0/3285726241 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f1bcc0b2b70 0x7f1bcc154aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:52:00.463 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:52:00.460+0000 7f1bd1547640 1 --2- 172.21.15.43:0/3285726241 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f1bcc0b2b70 0x7f1bcc154aa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:52:00.463 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:52:00.460+0000 7f1bd1547640 1 -- 172.21.15.43:0/3285726241 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f1bcc0b3050 msgr2=0x7f1bcc0afc10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:52:00.464 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:52:00.460+0000 7f1bd1547640 1 --2- 172.21.15.43:0/3285726241 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f1bcc0b3050 0x7f1bcc0afc10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:52:00.464 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:52:00.460+0000 7f1bd1547640 1 -- 172.21.15.43:0/3285726241 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1bc00094d0 con 0x7f1bcc0b2790 2023-08-09T13:52:00.464 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:52:00.460+0000 7f1bd1547640 1 --2- 172.21.15.43:0/3285726241 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f1bcc0b2790 0x7f1bcc0c8c10 secure :-1 s=READY pgs=1331 cs=0 l=1 rev1=1 crypto rx=0x7f1bbc00c3c0 tx=0x7f1bbc00c880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-09T13:52:00.464 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:52:00.461+0000 7f1bc9ffb640 1 -- 172.21.15.43:0/3285726241 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1bbc0074e0 con 0x7f1bcc0b2790 2023-08-09T13:52:00.464 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:52:00.461+0000 7f1bc9ffb640 1 -- 172.21.15.43:0/3285726241 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1bbc007640 con 0x7f1bcc0b2790 2023-08-09T13:52:00.464 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:52:00.461+0000 7f1bc9ffb640 1 -- 172.21.15.43:0/3285726241 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1bbc004a00 con 0x7f1bcc0b2790 2023-08-09T13:52:00.465 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:52:00.461+0000 7f1bd37ab640 1 -- 172.21.15.43:0/3285726241 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1bcc0b0920 con 0x7f1bcc0b2790 2023-08-09T13:52:00.465 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:52:00.461+0000 7f1bd37ab640 1 -- 172.21.15.43:0/3285726241 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1bcc0b0640 con 0x7f1bcc0b2790 2023-08-09T13:52:00.465 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:52:00.462+0000 7f1bd37ab640 1 -- 172.21.15.43:0/3285726241 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1bcc0ce500 con 0x7f1bcc0b2790 2023-08-09T13:52:00.466 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:52:00.463+0000 7f1bc9ffb640 1 -- 172.21.15.43:0/3285726241 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 45) v1 ==== 88443+0+0 (secure 0 0 0) 0x7f1bbc004b60 con 0x7f1bcc0b2790 2023-08-09T13:52:00.466 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:52:00.464+0000 7f1bc9ffb640 1 --2- 172.21.15.43:0/3285726241 >> [v2:172.21.15.203:6832/1317108840,v1:172.21.15.203:6833/1317108840] conn(0x7f1bb806a170 0x7f1bb806c630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:52:00.466 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:52:00.464+0000 7f1bc9ffb640 1 -- 172.21.15.43:0/3285726241 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(115..115 src has 1..115) v4 ==== 6426+0+0 (secure 0 0 0) 0x7f1bbc088090 con 0x7f1bcc0b2790 2023-08-09T13:52:00.466 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:52:00.465+0000 7f1bd0d46640 1 --2- 172.21.15.43:0/3285726241 >> [v2:172.21.15.203:6832/1317108840,v1:172.21.15.203:6833/1317108840] conn(0x7f1bb806a170 0x7f1bb806c630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:52:00.468 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:52:00.467+0000 7f1bd0d46640 1 --2- 172.21.15.43:0/3285726241 >> [v2:172.21.15.203:6832/1317108840,v1:172.21.15.203:6833/1317108840] conn(0x7f1bb806a170 0x7f1bb806c630 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f1bc0036000 tx=0x7f1bc003b040 comp rx=0 tx=0).ready entity=mgr.6619 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-09T13:52:00.470 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:52:00.468+0000 7f1bc9ffb640 1 -- 172.21.15.43:0/3285726241 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178066 (secure 0 0 0) 0x7f1bbc0534f0 con 0x7f1bcc0b2790 2023-08-09T13:52:00.668 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:52:00.666+0000 7f1bd37ab640 1 -- 172.21.15.43:0/3285726241 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f1bcc0739f0 con 0x7f1bcc0b2790 2023-08-09T13:52:00.671 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:52:00.669+0000 7f1bc9ffb640 1 -- 172.21.15.43:0/3285726241 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v115) v1 ==== 74+0+18519 (secure 0 0 0) 0x7f1bbc056b10 con 0x7f1bcc0b2790 2023-08-09T13:52:00.671 INFO:teuthology.orchestra.run.smithi043.stdout: 2023-08-09T13:52:00.672 INFO:teuthology.orchestra.run.smithi043.stdout:{"epoch":115,"fsid":"daac87ee-1337-4250-a955-aea4e408a39b","created":"2023-08-09T13:33:41.701198+0000","modified":"2023-08-09T13:51:59.654793+0000","last_up_change":"2023-08-09T13:33:53.783453+0000","last_in_change":"2023-08-09T13:33:45.148904+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":15,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":"rbd","create_time":"2023-08-09T13:33:54.365137+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":".mgr","create_time":"2023-08-09T13:33:54.831803+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":14,"pool_name":"cephfs_metadata","create_time":"2023-08-09T13:51:57.637653+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"115","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":2.5,"score_stable":2.5,"optimal_score":1,"raw_score_acting":2.5,"raw_score_stable":2.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":15,"pool_name":"cephfs_data","create_time":"2023-08-09T13:51:58.676117+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"115","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"cbd49ecd-e896-4fbd-864e-5c37a829b165","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":114,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6800","nonce":502667335},{"type":"v1","addr":"172.21.15.43:6801","nonce":502667335}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6802","nonce":502667335},{"type":"v1","addr":"172.21.15.43:6803","nonce":502667335}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6806","nonce":502667335},{"type":"v1","addr":"172.21.15.43:6807","nonce":502667335}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6804","nonce":502667335},{"type":"v1","addr":"172.21.15.43:6805","nonce":502667335}]},"public_addr":"172.21.15.43:6801/502667335","cluster_addr":"172.21.15.43:6803/502667335","heartbeat_back_addr":"172.21.15.43:6807/502667335","heartbeat_front_addr":"172.21.15.43:6805/502667335","state":["exists","up"]},{"osd":1,"uuid":"75c7e0fe-1ec7-4fe8-a2f2-3661a65b8500","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":114,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6808","nonce":3289134610},{"type":"v1","addr":"172.21.15.43:6809","nonce":3289134610}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6810","nonce":3289134610},{"type":"v1","addr":"172.21.15.43:6812","nonce":3289134610}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6821","nonce":3289134610},{"type":"v1","addr":"172.21.15.43:6824","nonce":3289134610}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6815","nonce":3289134610},{"type":"v1","addr":"172.21.15.43:6818","nonce":3289134610}]},"public_addr":"172.21.15.43:6809/3289134610","cluster_addr":"172.21.15.43:6812/3289134610","heartbeat_back_addr":"172.21.15.43:6824/3289134610","heartbeat_front_addr":"172.21.15.43:6818/3289134610","state":["exists","up"]},{"osd":2,"uuid":"bb5c41e9-7f5f-4125-9b01-3bb8bfc2ea83","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":114,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6811","nonce":4079709736},{"type":"v1","addr":"172.21.15.43:6814","nonce":4079709736}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6817","nonce":4079709736},{"type":"v1","addr":"172.21.15.43:6820","nonce":4079709736}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6829","nonce":4079709736},{"type":"v1","addr":"172.21.15.43:6831","nonce":4079709736}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6823","nonce":4079709736},{"type":"v1","addr":"172.21.15.43:6826","nonce":4079709736}]},"public_addr":"172.21.15.43:6814/4079709736","cluster_addr":"172.21.15.43:6820/4079709736","heartbeat_back_addr":"172.21.15.43:6831/4079709736","heartbeat_front_addr":"172.21.15.43:6826/4079709736","state":["exists","up"]},{"osd":3,"uuid":"bfe4c8b4-694a-465c-87e6-2b2aab852613","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":114,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6813","nonce":1709538759},{"type":"v1","addr":"172.21.15.43:6816","nonce":1709538759}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6819","nonce":1709538759},{"type":"v1","addr":"172.21.15.43:6822","nonce":1709538759}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6828","nonce":1709538759},{"type":"v1","addr":"172.21.15.43:6830","nonce":1709538759}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6825","nonce":1709538759},{"type":"v1","addr":"172.21.15.43:6827","nonce":1709538759}]},"public_addr":"172.21.15.43:6816/1709538759","cluster_addr":"172.21.15.43:6822/1709538759","heartbeat_back_addr":"172.21.15.43:6830/1709538759","heartbeat_front_addr":"172.21.15.43:6827/1709538759","state":["exists","up"]},{"osd":4,"uuid":"fa2e8a77-aba1-4756-853e-0d174a5bdeba","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":114,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6800","nonce":4196866481},{"type":"v1","addr":"172.21.15.203:6801","nonce":4196866481}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6802","nonce":4196866481},{"type":"v1","addr":"172.21.15.203:6803","nonce":4196866481}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6806","nonce":4196866481},{"type":"v1","addr":"172.21.15.203:6807","nonce":4196866481}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6804","nonce":4196866481},{"type":"v1","addr":"172.21.15.203:6805","nonce":4196866481}]},"public_addr":"172.21.15.203:6801/4196866481","cluster_addr":"172.21.15.203:6803/4196866481","heartbeat_back_addr":"172.21.15.203:6807/4196866481","heartbeat_front_addr":"172.21.15.203:6805/4196866481","state":["exists","up"]},{"osd":5,"uuid":"81811e4b-4d24-4f51-a82c-f1d700a2a00d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":114,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6816","nonce":3059290081},{"type":"v1","addr":"172.21.15.203:6817","nonce":3059290081}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6818","nonce":3059290081},{"type":"v1","addr":"172.21.15.203:6819","nonce":3059290081}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6822","nonce":3059290081},{"type":"v1","addr":"172.21.15.203:6823","nonce":3059290081}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6820","nonce":3059290081},{"type":"v1","addr":"172.21.15.203:6821","nonce":3059290081}]},"public_addr":"172.21.15.203:6817/3059290081","cluster_addr":"172.21.15.203:6819/3059290081","heartbeat_back_addr":"172.21.15.203:6823/3059290081","heartbeat_front_addr":"172.21.15.203:6821/3059290081","state":["exists","up"]},{"osd":6,"uuid":"307bac4d-6a5b-443d-af75-56d695d7674f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":114,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6824","nonce":512155569},{"type":"v1","addr":"172.21.15.203:6825","nonce":512155569}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6826","nonce":512155569},{"type":"v1","addr":"172.21.15.203:6827","nonce":512155569}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6830","nonce":512155569},{"type":"v1","addr":"172.21.15.203:6831","nonce":512155569}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6828","nonce":512155569},{"type":"v1","addr":"172.21.15.203:6829","nonce":512155569}]},"public_addr":"172.21.15.203:6825/512155569","cluster_addr":"172.21.15.203:6827/512155569","heartbeat_back_addr":"172.21.15.203:6831/512155569","heartbeat_front_addr":"172.21.15.203:6829/512155569","state":["exists","up"]},{"osd":7,"uuid":"a95dc7ca-9f1b-4295-af70-68a22a2d7d5e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":114,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6808","nonce":3331450438},{"type":"v1","addr":"172.21.15.203:6809","nonce":3331450438}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6810","nonce":3331450438},{"type":"v1","addr":"172.21.15.203:6811","nonce":3331450438}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6814","nonce":3331450438},{"type":"v1","addr":"172.21.15.203:6815","nonce":3331450438}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6812","nonce":3331450438},{"type":"v1","addr":"172.21.15.203:6813","nonce":3331450438}]},"public_addr":"172.21.15.203:6809/3331450438","cluster_addr":"172.21.15.203:6811/3331450438","heartbeat_back_addr":"172.21.15.203:6815/3331450438","heartbeat_front_addr":"172.21.15.203:6813/3331450438","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-09T13:33:50.547652+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-09T13:33:50.589469+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-09T13:33:50.561724+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-09T13:33:50.599650+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-09T13:33:50.411855+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-09T13:33:50.219681+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-09T13:33:50.504664+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-09T13:33:50.441248+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":7,"snaps":[{"begin":104,"length":1},{"begin":221,"length":1}]}],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-08-09T13:52:00.674 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:52:00.672+0000 7f1bd37ab640 1 -- 172.21.15.43:0/3285726241 >> [v2:172.21.15.203:6832/1317108840,v1:172.21.15.203:6833/1317108840] conn(0x7f1bb806a170 msgr2=0x7f1bb806c630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:52:00.674 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:52:00.672+0000 7f1bd37ab640 1 --2- 172.21.15.43:0/3285726241 >> [v2:172.21.15.203:6832/1317108840,v1:172.21.15.203:6833/1317108840] conn(0x7f1bb806a170 0x7f1bb806c630 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f1bc0036000 tx=0x7f1bc003b040 comp rx=0 tx=0).stop 2023-08-09T13:52:00.674 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:52:00.672+0000 7f1bd37ab640 1 -- 172.21.15.43:0/3285726241 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f1bcc0b2790 msgr2=0x7f1bcc0c8c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:52:00.674 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:52:00.672+0000 7f1bd37ab640 1 --2- 172.21.15.43:0/3285726241 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f1bcc0b2790 0x7f1bcc0c8c10 secure :-1 s=READY pgs=1331 cs=0 l=1 rev1=1 crypto rx=0x7f1bbc00c3c0 tx=0x7f1bbc00c880 comp rx=0 tx=0).stop 2023-08-09T13:52:00.674 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:52:00.672+0000 7f1bd37ab640 1 -- 172.21.15.43:0/3285726241 shutdown_connections 2023-08-09T13:52:00.674 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:52:00.672+0000 7f1bd37ab640 1 --2- 172.21.15.43:0/3285726241 >> [v2:172.21.15.203:6832/1317108840,v1:172.21.15.203:6833/1317108840] conn(0x7f1bb806a170 0x7f1bb806c630 unknown :-1 s=CLOSED pgs=59 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:52:00.675 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:52:00.672+0000 7f1bd37ab640 1 --2- 172.21.15.43:0/3285726241 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f1bcc0b3050 0x7f1bcc0afc10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:52:00.675 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:52:00.672+0000 7f1bd37ab640 1 --2- 172.21.15.43:0/3285726241 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f1bcc0b2b70 0x7f1bcc154aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:52:00.675 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:52:00.672+0000 7f1bd37ab640 1 --2- 172.21.15.43:0/3285726241 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f1bcc0b2790 0x7f1bcc0c8c10 unknown :-1 s=CLOSED pgs=1331 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:52:00.675 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:52:00.672+0000 7f1bd37ab640 1 -- 172.21.15.43:0/3285726241 >> 172.21.15.43:0/3285726241 conn(0x7f1bcc09b7b0 msgr2=0x7f1bcc0b3f60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-09T13:52:00.675 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:52:00.672+0000 7f1bd37ab640 1 -- 172.21.15.43:0/3285726241 shutdown_connections 2023-08-09T13:52:00.675 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:52:00.672+0000 7f1bd37ab640 1 -- 172.21.15.43:0/3285726241 wait complete. 2023-08-09T13:52:00.695 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-08-09T13:52:00.928 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:52:00.926+0000 7fbbc326a640 1 Processor -- start 2023-08-09T13:52:00.928 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:52:00.927+0000 7fbbc326a640 1 -- start start 2023-08-09T13:52:00.931 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:52:00.928+0000 7fbbc326a640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7fbbbc0c86f0 0x7fbbbc0c8b50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:52:00.932 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:52:00.928+0000 7fbbc326a640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fbbbc06c8d0 0x7fbbbc06cd40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:52:00.932 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:52:00.928+0000 7fbbc326a640 1 -- --> v1:172.21.15.43:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbbbc0c9090 con 0x7fbbbc143c20 2023-08-09T13:52:00.932 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:52:00.928+0000 7fbbc326a640 1 -- --> v2:172.21.15.43:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbbbc144000 con 0x7fbbbc0c86f0 2023-08-09T13:52:00.933 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:52:00.928+0000 7fbbc326a640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbbbc06c440 con 0x7fbbbc06c8d0 2023-08-09T13:52:00.933 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:52:00.929+0000 7fbbc0805640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7fbbbc0c86f0 0x7fbbbc0c8b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:52:00.933 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:52:00.929+0000 7fbbc0805640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7fbbbc0c86f0 0x7fbbbc0c8b50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:33054/0 (socket says 172.21.15.43:33054) 2023-08-09T13:52:00.933 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:52:00.929+0000 7fbbc0805640 1 -- 172.21.15.43:0/4030925860 learned_addr learned my addr 172.21.15.43:0/4030925860 (peer_addr_for_me v2:172.21.15.43:0/0) 2023-08-09T13:52:00.934 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:52:00.929+0000 7fbbc1807640 1 --2- 172.21.15.43:0/4030925860 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fbbbc06c8d0 0x7fbbbc06cd40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:52:00.934 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:52:00.929+0000 7fbbc0805640 1 -- 172.21.15.43:0/4030925860 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fbbbc06c8d0 msgr2=0x7fbbbc06cd40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-09T13:52:00.934 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:52:00.929+0000 7fbbc0805640 1 --2- 172.21.15.43:0/4030925860 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fbbbc06c8d0 0x7fbbbc06cd40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:52:00.934 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:52:00.929+0000 7fbbc0805640 1 -- 172.21.15.43:0/4030925860 >> v1:172.21.15.43:6789/0 conn(0x7fbbbc143c20 legacy=0x7fbbbc06f540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:52:00.935 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:52:00.929+0000 7fbbc0805640 1 -- 172.21.15.43:0/4030925860 --> v2:172.21.15.43:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbbbc06fc50 con 0x7fbbbc0c86f0 2023-08-09T13:52:00.935 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:52:00.929+0000 7fbbb3fff640 1 -- 172.21.15.43:0/4030925860 <== mon.0 v1:172.21.15.43:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1709478380 0 0) 0x7fbbbc0c9090 con 0x7fbbbc143c20 2023-08-09T13:52:00.938 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:52:00.930+0000 7fbbc0805640 1 --2- 172.21.15.43:0/4030925860 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fbbbc0c86f0 0x7fbbbc0c8b50 secure :-1 s=READY pgs=1333 cs=0 l=1 rev1=1 crypto rx=0x7fbbac009870 tx=0x7fbbac02b8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e43803ba221cfdff server_cookie=0 in_seq=0 out_seq=0 2023-08-09T13:52:00.939 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:52:00.931+0000 7fbbb3fff640 1 -- 172.21.15.43:0/4030925860 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbbac031640 con 0x7fbbbc0c86f0 2023-08-09T13:52:00.939 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:52:00.931+0000 7fbbb3fff640 1 -- 172.21.15.43:0/4030925860 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbbac0317a0 con 0x7fbbbc0c86f0 2023-08-09T13:52:00.939 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:52:00.931+0000 7fbbb3fff640 1 -- 172.21.15.43:0/4030925860 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbbac031b30 con 0x7fbbbc0c86f0 2023-08-09T13:52:00.939 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:52:00.931+0000 7fbbc326a640 1 -- 172.21.15.43:0/4030925860 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fbbbc0c86f0 msgr2=0x7fbbbc0c8b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:52:00.940 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:52:00.931+0000 7fbbc326a640 1 --2- 172.21.15.43:0/4030925860 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fbbbc0c86f0 0x7fbbbc0c8b50 secure :-1 s=READY pgs=1333 cs=0 l=1 rev1=1 crypto rx=0x7fbbac009870 tx=0x7fbbac02b8f0 comp rx=0 tx=0).stop 2023-08-09T13:52:00.940 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:52:00.931+0000 7fbbc326a640 1 -- 172.21.15.43:0/4030925860 shutdown_connections 2023-08-09T13:52:00.940 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:52:00.931+0000 7fbbc326a640 1 --2- 172.21.15.43:0/4030925860 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fbbbc06c8d0 0x7fbbbc06cd40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:52:00.940 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:52:00.931+0000 7fbbc326a640 1 --2- 172.21.15.43:0/4030925860 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fbbbc0c86f0 0x7fbbbc0c8b50 unknown :-1 s=CLOSED pgs=1333 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:52:00.941 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:52:00.931+0000 7fbbc326a640 1 -- 172.21.15.43:0/4030925860 >> 172.21.15.43:0/4030925860 conn(0x7fbbbc0ce0a0 msgr2=0x7fbbbc0ce480 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-09T13:52:00.941 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:52:00.931+0000 7fbbc326a640 1 -- 172.21.15.43:0/4030925860 shutdown_connections 2023-08-09T13:52:00.941 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:52:00.931+0000 7fbbc326a640 1 -- 172.21.15.43:0/4030925860 wait complete. 2023-08-09T13:52:00.941 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:52:00.933+0000 7fbbc326a640 1 Processor -- start 2023-08-09T13:52:00.942 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:52:00.933+0000 7fbbc326a640 1 -- start start 2023-08-09T13:52:00.942 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:52:00.934+0000 7fbbc326a640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fbbbc06c8d0 0x7fbbbc0b3c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:52:00.942 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:52:00.934+0000 7fbbc326a640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fbbbc0c86f0 0x7fbbbc141e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:52:00.942 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:52:00.934+0000 7fbbc326a640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fbbbc143c20 0x7fbbbc15ba20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:52:00.942 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:52:00.934+0000 7fbbc326a640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbbbc142e10 con 0x7fbbbc143c20 2023-08-09T13:52:00.943 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:52:00.934+0000 7fbbc326a640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbbbc0b4140 con 0x7fbbbc0c86f0 2023-08-09T13:52:00.943 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:52:00.934+0000 7fbbc326a640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbbbc142360 con 0x7fbbbc06c8d0 2023-08-09T13:52:00.943 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:52:00.935+0000 7fbbc1807640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fbbbc143c20 0x7fbbbc15ba20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:52:00.943 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:52:00.935+0000 7fbbc1807640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fbbbc143c20 0x7fbbbc15ba20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:33064/0 (socket says 172.21.15.43:33064) 2023-08-09T13:52:00.944 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:52:00.935+0000 7fbbc1807640 1 -- 172.21.15.43:0/3833328709 learned_addr learned my addr 172.21.15.43:0/3833328709 (peer_addr_for_me v2:172.21.15.43:0/0) 2023-08-09T13:52:00.944 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:52:00.935+0000 7fbbc0805640 1 --2- 172.21.15.43:0/3833328709 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fbbbc0c86f0 0x7fbbbc141e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:52:00.944 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:52:00.935+0000 7fbbc1006640 1 --2- 172.21.15.43:0/3833328709 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fbbbc06c8d0 0x7fbbbc0b3c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:52:00.944 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:52:00.935+0000 7fbbc1807640 1 -- 172.21.15.43:0/3833328709 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fbbbc06c8d0 msgr2=0x7fbbbc0b3c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:52:00.945 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:52:00.935+0000 7fbbc1807640 1 --2- 172.21.15.43:0/3833328709 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fbbbc06c8d0 0x7fbbbc0b3c00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:52:00.945 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:52:00.935+0000 7fbbc1807640 1 -- 172.21.15.43:0/3833328709 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fbbbc0c86f0 msgr2=0x7fbbbc141e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:52:00.945 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:52:00.935+0000 7fbbc1807640 1 --2- 172.21.15.43:0/3833328709 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fbbbc0c86f0 0x7fbbbc141e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:52:00.945 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:52:00.935+0000 7fbbc1807640 1 -- 172.21.15.43:0/3833328709 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbbac0094d0 con 0x7fbbbc143c20 2023-08-09T13:52:00.946 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:52:00.935+0000 7fbbc0805640 1 --2- 172.21.15.43:0/3833328709 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fbbbc0c86f0 0x7fbbbc141e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-08-09T13:52:00.946 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:52:00.936+0000 7fbbc1807640 1 --2- 172.21.15.43:0/3833328709 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fbbbc143c20 0x7fbbbc15ba20 secure :-1 s=READY pgs=1334 cs=0 l=1 rev1=1 crypto rx=0x7fbbb4007cf0 tx=0x7fbbb400e4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-09T13:52:00.948 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:52:00.936+0000 7fbbb1ffb640 1 -- 172.21.15.43:0/3833328709 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbbb4014420 con 0x7fbbbc143c20 2023-08-09T13:52:00.948 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:52:00.936+0000 7fbbb1ffb640 1 -- 172.21.15.43:0/3833328709 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbbb40145a0 con 0x7fbbbc143c20 2023-08-09T13:52:00.948 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:52:00.936+0000 7fbbb1ffb640 1 -- 172.21.15.43:0/3833328709 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbbb4014950 con 0x7fbbbc143c20 2023-08-09T13:52:00.948 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:52:00.936+0000 7fbbc326a640 1 -- 172.21.15.43:0/3833328709 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbbbc1f2340 con 0x7fbbbc143c20 2023-08-09T13:52:00.948 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:52:00.936+0000 7fbbc326a640 1 -- 172.21.15.43:0/3833328709 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbbbc1f2900 con 0x7fbbbc143c20 2023-08-09T13:52:00.948 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:52:00.937+0000 7fbbc326a640 1 -- 172.21.15.43:0/3833328709 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbbbc0739f0 con 0x7fbbbc143c20 2023-08-09T13:52:00.949 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:52:00.938+0000 7fbbb1ffb640 1 -- 172.21.15.43:0/3833328709 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 45) v1 ==== 88443+0+0 (secure 0 0 0) 0x7fbbb4014ab0 con 0x7fbbbc143c20 2023-08-09T13:52:00.949 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:52:00.939+0000 7fbbb1ffb640 1 --2- 172.21.15.43:0/3833328709 >> [v2:172.21.15.203:6832/1317108840,v1:172.21.15.203:6833/1317108840] conn(0x7fbba006a170 0x7fbba006c630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:52:00.950 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:52:00.939+0000 7fbbb1ffb640 1 -- 172.21.15.43:0/3833328709 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(115..115 src has 1..115) v4 ==== 6426+0+0 (secure 0 0 0) 0x7fbbb40891d0 con 0x7fbbbc143c20 2023-08-09T13:52:00.950 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:52:00.940+0000 7fbbc1006640 1 --2- 172.21.15.43:0/3833328709 >> [v2:172.21.15.203:6832/1317108840,v1:172.21.15.203:6833/1317108840] conn(0x7fbba006a170 0x7fbba006c630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:52:00.950 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:52:00.941+0000 7fbbc1006640 1 --2- 172.21.15.43:0/3833328709 >> [v2:172.21.15.203:6832/1317108840,v1:172.21.15.203:6833/1317108840] conn(0x7fbba006a170 0x7fbba006c630 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7fbbb8000f60 tx=0x7fbbb8009040 comp rx=0 tx=0).ready entity=mgr.6619 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-09T13:52:00.950 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:52:00.943+0000 7fbbb1ffb640 1 -- 172.21.15.43:0/3833328709 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178066 (secure 0 0 0) 0x7fbbb40545a0 con 0x7fbbbc143c20 2023-08-09T13:52:01.140 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:52:01.139+0000 7fbbc326a640 1 -- 172.21.15.43:0/3833328709 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fbbbc0cec90 con 0x7fbbbc143c20 2023-08-09T13:52:01.143 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:52:01.142+0000 7fbbb1ffb640 1 -- 172.21.15.43:0/3833328709 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v115) v1 ==== 74+0+18519 (secure 0 0 0) 0x7fbbb4057bc0 con 0x7fbbbc143c20 2023-08-09T13:52:01.144 INFO:teuthology.orchestra.run.smithi043.stdout: 2023-08-09T13:52:01.144 INFO:teuthology.orchestra.run.smithi043.stdout:{"epoch":115,"fsid":"daac87ee-1337-4250-a955-aea4e408a39b","created":"2023-08-09T13:33:41.701198+0000","modified":"2023-08-09T13:51:59.654793+0000","last_up_change":"2023-08-09T13:33:53.783453+0000","last_in_change":"2023-08-09T13:33:45.148904+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":15,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":"rbd","create_time":"2023-08-09T13:33:54.365137+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":".mgr","create_time":"2023-08-09T13:33:54.831803+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":14,"pool_name":"cephfs_metadata","create_time":"2023-08-09T13:51:57.637653+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"115","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":2.5,"score_stable":2.5,"optimal_score":1,"raw_score_acting":2.5,"raw_score_stable":2.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":15,"pool_name":"cephfs_data","create_time":"2023-08-09T13:51:58.676117+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"115","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"cbd49ecd-e896-4fbd-864e-5c37a829b165","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":114,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6800","nonce":502667335},{"type":"v1","addr":"172.21.15.43:6801","nonce":502667335}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6802","nonce":502667335},{"type":"v1","addr":"172.21.15.43:6803","nonce":502667335}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6806","nonce":502667335},{"type":"v1","addr":"172.21.15.43:6807","nonce":502667335}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6804","nonce":502667335},{"type":"v1","addr":"172.21.15.43:6805","nonce":502667335}]},"public_addr":"172.21.15.43:6801/502667335","cluster_addr":"172.21.15.43:6803/502667335","heartbeat_back_addr":"172.21.15.43:6807/502667335","heartbeat_front_addr":"172.21.15.43:6805/502667335","state":["exists","up"]},{"osd":1,"uuid":"75c7e0fe-1ec7-4fe8-a2f2-3661a65b8500","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":114,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6808","nonce":3289134610},{"type":"v1","addr":"172.21.15.43:6809","nonce":3289134610}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6810","nonce":3289134610},{"type":"v1","addr":"172.21.15.43:6812","nonce":3289134610}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6821","nonce":3289134610},{"type":"v1","addr":"172.21.15.43:6824","nonce":3289134610}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6815","nonce":3289134610},{"type":"v1","addr":"172.21.15.43:6818","nonce":3289134610}]},"public_addr":"172.21.15.43:6809/3289134610","cluster_addr":"172.21.15.43:6812/3289134610","heartbeat_back_addr":"172.21.15.43:6824/3289134610","heartbeat_front_addr":"172.21.15.43:6818/3289134610","state":["exists","up"]},{"osd":2,"uuid":"bb5c41e9-7f5f-4125-9b01-3bb8bfc2ea83","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":114,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6811","nonce":4079709736},{"type":"v1","addr":"172.21.15.43:6814","nonce":4079709736}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6817","nonce":4079709736},{"type":"v1","addr":"172.21.15.43:6820","nonce":4079709736}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6829","nonce":4079709736},{"type":"v1","addr":"172.21.15.43:6831","nonce":4079709736}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6823","nonce":4079709736},{"type":"v1","addr":"172.21.15.43:6826","nonce":4079709736}]},"public_addr":"172.21.15.43:6814/4079709736","cluster_addr":"172.21.15.43:6820/4079709736","heartbeat_back_addr":"172.21.15.43:6831/4079709736","heartbeat_front_addr":"172.21.15.43:6826/4079709736","state":["exists","up"]},{"osd":3,"uuid":"bfe4c8b4-694a-465c-87e6-2b2aab852613","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":114,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6813","nonce":1709538759},{"type":"v1","addr":"172.21.15.43:6816","nonce":1709538759}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6819","nonce":1709538759},{"type":"v1","addr":"172.21.15.43:6822","nonce":1709538759}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6828","nonce":1709538759},{"type":"v1","addr":"172.21.15.43:6830","nonce":1709538759}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6825","nonce":1709538759},{"type":"v1","addr":"172.21.15.43:6827","nonce":1709538759}]},"public_addr":"172.21.15.43:6816/1709538759","cluster_addr":"172.21.15.43:6822/1709538759","heartbeat_back_addr":"172.21.15.43:6830/1709538759","heartbeat_front_addr":"172.21.15.43:6827/1709538759","state":["exists","up"]},{"osd":4,"uuid":"fa2e8a77-aba1-4756-853e-0d174a5bdeba","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":114,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6800","nonce":4196866481},{"type":"v1","addr":"172.21.15.203:6801","nonce":4196866481}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6802","nonce":4196866481},{"type":"v1","addr":"172.21.15.203:6803","nonce":4196866481}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6806","nonce":4196866481},{"type":"v1","addr":"172.21.15.203:6807","nonce":4196866481}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6804","nonce":4196866481},{"type":"v1","addr":"172.21.15.203:6805","nonce":4196866481}]},"public_addr":"172.21.15.203:6801/4196866481","cluster_addr":"172.21.15.203:6803/4196866481","heartbeat_back_addr":"172.21.15.203:6807/4196866481","heartbeat_front_addr":"172.21.15.203:6805/4196866481","state":["exists","up"]},{"osd":5,"uuid":"81811e4b-4d24-4f51-a82c-f1d700a2a00d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":114,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6816","nonce":3059290081},{"type":"v1","addr":"172.21.15.203:6817","nonce":3059290081}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6818","nonce":3059290081},{"type":"v1","addr":"172.21.15.203:6819","nonce":3059290081}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6822","nonce":3059290081},{"type":"v1","addr":"172.21.15.203:6823","nonce":3059290081}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6820","nonce":3059290081},{"type":"v1","addr":"172.21.15.203:6821","nonce":3059290081}]},"public_addr":"172.21.15.203:6817/3059290081","cluster_addr":"172.21.15.203:6819/3059290081","heartbeat_back_addr":"172.21.15.203:6823/3059290081","heartbeat_front_addr":"172.21.15.203:6821/3059290081","state":["exists","up"]},{"osd":6,"uuid":"307bac4d-6a5b-443d-af75-56d695d7674f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":114,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6824","nonce":512155569},{"type":"v1","addr":"172.21.15.203:6825","nonce":512155569}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6826","nonce":512155569},{"type":"v1","addr":"172.21.15.203:6827","nonce":512155569}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6830","nonce":512155569},{"type":"v1","addr":"172.21.15.203:6831","nonce":512155569}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6828","nonce":512155569},{"type":"v1","addr":"172.21.15.203:6829","nonce":512155569}]},"public_addr":"172.21.15.203:6825/512155569","cluster_addr":"172.21.15.203:6827/512155569","heartbeat_back_addr":"172.21.15.203:6831/512155569","heartbeat_front_addr":"172.21.15.203:6829/512155569","state":["exists","up"]},{"osd":7,"uuid":"a95dc7ca-9f1b-4295-af70-68a22a2d7d5e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":114,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6808","nonce":3331450438},{"type":"v1","addr":"172.21.15.203:6809","nonce":3331450438}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6810","nonce":3331450438},{"type":"v1","addr":"172.21.15.203:6811","nonce":3331450438}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6814","nonce":3331450438},{"type":"v1","addr":"172.21.15.203:6815","nonce":3331450438}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6812","nonce":3331450438},{"type":"v1","addr":"172.21.15.203:6813","nonce":3331450438}]},"public_addr":"172.21.15.203:6809/3331450438","cluster_addr":"172.21.15.203:6811/3331450438","heartbeat_back_addr":"172.21.15.203:6815/3331450438","heartbeat_front_addr":"172.21.15.203:6813/3331450438","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-09T13:33:50.547652+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-09T13:33:50.589469+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-09T13:33:50.561724+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-09T13:33:50.599650+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-09T13:33:50.411855+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-09T13:33:50.219681+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-09T13:33:50.504664+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-09T13:33:50.441248+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":7,"snaps":[{"begin":104,"length":1},{"begin":221,"length":1}]}],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-08-09T13:52:01.146 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:52:01.144+0000 7fbbc326a640 1 -- 172.21.15.43:0/3833328709 >> [v2:172.21.15.203:6832/1317108840,v1:172.21.15.203:6833/1317108840] conn(0x7fbba006a170 msgr2=0x7fbba006c630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:52:01.146 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:52:01.144+0000 7fbbc326a640 1 --2- 172.21.15.43:0/3833328709 >> [v2:172.21.15.203:6832/1317108840,v1:172.21.15.203:6833/1317108840] conn(0x7fbba006a170 0x7fbba006c630 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7fbbb8000f60 tx=0x7fbbb8009040 comp rx=0 tx=0).stop 2023-08-09T13:52:01.147 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:52:01.144+0000 7fbbc326a640 1 -- 172.21.15.43:0/3833328709 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fbbbc143c20 msgr2=0x7fbbbc15ba20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:52:01.147 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:52:01.144+0000 7fbbc326a640 1 --2- 172.21.15.43:0/3833328709 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fbbbc143c20 0x7fbbbc15ba20 secure :-1 s=READY pgs=1334 cs=0 l=1 rev1=1 crypto rx=0x7fbbb4007cf0 tx=0x7fbbb400e4b0 comp rx=0 tx=0).stop 2023-08-09T13:52:01.147 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:52:01.145+0000 7fbbc326a640 1 -- 172.21.15.43:0/3833328709 shutdown_connections 2023-08-09T13:52:01.147 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:52:01.145+0000 7fbbc326a640 1 --2- 172.21.15.43:0/3833328709 >> [v2:172.21.15.203:6832/1317108840,v1:172.21.15.203:6833/1317108840] conn(0x7fbba006a170 0x7fbba006c630 unknown :-1 s=CLOSED pgs=60 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:52:01.147 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:52:01.145+0000 7fbbc326a640 1 --2- 172.21.15.43:0/3833328709 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fbbbc143c20 0x7fbbbc15ba20 unknown :-1 s=CLOSED pgs=1334 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:52:01.147 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:52:01.145+0000 7fbbc326a640 1 --2- 172.21.15.43:0/3833328709 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fbbbc0c86f0 0x7fbbbc141e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:52:01.147 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:52:01.145+0000 7fbbc326a640 1 --2- 172.21.15.43:0/3833328709 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fbbbc06c8d0 0x7fbbbc0b3c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:52:01.148 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:52:01.145+0000 7fbbc326a640 1 -- 172.21.15.43:0/3833328709 >> 172.21.15.43:0/3833328709 conn(0x7fbbbc0ce0a0 msgr2=0x7fbbbc0ae7f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-09T13:52:01.148 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:52:01.145+0000 7fbbc326a640 1 -- 172.21.15.43:0/3833328709 shutdown_connections 2023-08-09T13:52:01.148 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:52:01.145+0000 7fbbc326a640 1 -- 172.21.15.43:0/3833328709 wait complete. 2023-08-09T13:52:01.167 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs standby_count_wanted 0 2023-08-09T13:52:01.397 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:52:01.396+0000 7fa6f0b53640 1 Processor -- start 2023-08-09T13:52:01.397 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:52:01.396+0000 7fa6f0b53640 1 -- start start 2023-08-09T13:52:01.399 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:52:01.398+0000 7fa6f0b53640 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7fa6ec1547a0 0x7fa6ec0b16b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:52:01.399 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:52:01.398+0000 7fa6f0b53640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fa6ec15b2c0 0x7fa6ec0b38a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:52:01.399 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:52:01.398+0000 7fa6f0b53640 1 -- --> v1:172.21.15.43:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa6ec154b80 con 0x7fa6ec15a700 2023-08-09T13:52:01.399 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:52:01.398+0000 7fa6f0b53640 1 -- --> v2:172.21.15.203:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa6ec0b1bf0 con 0x7fa6ec1547a0 2023-08-09T13:52:01.400 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:52:01.398+0000 7fa6f0b53640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa6ec0b1e50 con 0x7fa6ec15b2c0 2023-08-09T13:52:01.401 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:52:01.398+0000 7fa6e2ffd640 1 --1- >> v1:172.21.15.43:6789/0 conn(0x7fa6ec15a700 0x7fa6ec15abb0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.43:6789/0 says I am v1:172.21.15.43:43758/0 (socket says 172.21.15.43:43758) 2023-08-09T13:52:01.401 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:52:01.398+0000 7fa6e2ffd640 1 -- 172.21.15.43:0/1508099033 learned_addr learned my addr 172.21.15.43:0/1508099033 (peer_addr_for_me v1:172.21.15.43:0/0) 2023-08-09T13:52:01.401 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:52:01.398+0000 7fa6eb7fe640 1 --2- 172.21.15.43:0/1508099033 >> v2:172.21.15.203:3300/0 conn(0x7fa6ec1547a0 0x7fa6ec0b16b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:52:01.401 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:52:01.399+0000 7fa6ebfff640 1 --2- 172.21.15.43:0/1508099033 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fa6ec15b2c0 0x7fa6ec0b38a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:52:01.401 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:52:01.399+0000 7fa6eaffd640 1 -- 172.21.15.43:0/1508099033 <== mon.0 v1:172.21.15.43:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 756898967 0 0) 0x7fa6ec154b80 con 0x7fa6ec15a700 2023-08-09T13:52:01.402 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:52:01.399+0000 7fa6eaffd640 1 -- 172.21.15.43:0/1508099033 --> v1:172.21.15.43:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fa6d8003520 con 0x7fa6ec15a700 2023-08-09T13:52:01.402 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:52:01.399+0000 7fa6eaffd640 1 -- 172.21.15.43:0/1508099033 <== mon.0 v1:172.21.15.43:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2074149359 0 0) 0x7fa6d8003520 con 0x7fa6ec15a700 2023-08-09T13:52:01.402 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:52:01.399+0000 7fa6eaffd640 1 -- 172.21.15.43:0/1508099033 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fa6ec15b2c0 msgr2=0x7fa6ec0b38a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-09T13:52:01.402 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:52:01.399+0000 7fa6eaffd640 1 --2- 172.21.15.43:0/1508099033 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fa6ec15b2c0 0x7fa6ec0b38a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:52:01.402 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:52:01.399+0000 7fa6eaffd640 1 -- 172.21.15.43:0/1508099033 >> v2:172.21.15.203:3300/0 conn(0x7fa6ec1547a0 msgr2=0x7fa6ec0b16b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-09T13:52:01.402 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:52:01.399+0000 7fa6eaffd640 1 --2- 172.21.15.43:0/1508099033 >> v2:172.21.15.203:3300/0 conn(0x7fa6ec1547a0 0x7fa6ec0b16b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:52:01.402 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:52:01.399+0000 7fa6eaffd640 1 -- 172.21.15.43:0/1508099033 --> v1:172.21.15.43:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa6ec070760 con 0x7fa6ec15a700 2023-08-09T13:52:01.403 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:52:01.399+0000 7fa6eaffd640 1 -- 172.21.15.43:0/1508099033 <== mon.0 v1:172.21.15.43:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1285150740 0 0) 0x7fa6d4001670 con 0x7fa6ec15a700 2023-08-09T13:52:01.403 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:52:01.399+0000 7fa6eaffd640 1 -- 172.21.15.43:0/1508099033 >> v1:172.21.15.43:6789/0 conn(0x7fa6ec15a700 legacy=0x7fa6ec15abb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:52:01.403 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:52:01.399+0000 7fa6eaffd640 1 --2- 172.21.15.43:0/1508099033 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fa6d8003970 0x7fa6d8003d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:52:01.403 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:52:01.399+0000 7fa6eaffd640 1 --2- 172.21.15.43:0/1508099033 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fa6ec15b2c0 0x7fa6d80047c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:52:01.403 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:52:01.399+0000 7fa6eaffd640 1 --2- 172.21.15.43:0/1508099033 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa6d8004d00 0x7fa6d8005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:52:01.403 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:52:01.399+0000 7fa6eaffd640 1 -- 172.21.15.43:0/1508099033 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa6ec0b1bf0 con 0x7fa6d8004d00 2023-08-09T13:52:01.403 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:52:01.399+0000 7fa6eaffd640 1 -- 172.21.15.43:0/1508099033 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa6ec0b1e50 con 0x7fa6d8003970 2023-08-09T13:52:01.403 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:52:01.399+0000 7fa6eaffd640 1 -- 172.21.15.43:0/1508099033 --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa6d800b980 con 0x7fa6ec15b2c0 2023-08-09T13:52:01.404 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:52:01.400+0000 7fa6ebfff640 1 --2- 172.21.15.43:0/1508099033 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa6d8004d00 0x7fa6d8005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:52:01.405 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:52:01.400+0000 7fa6ebfff640 1 -- 172.21.15.43:0/1508099033 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fa6ec15b2c0 msgr2=0x7fa6d80047c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-09T13:52:01.405 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:52:01.400+0000 7fa6ebfff640 1 --2- 172.21.15.43:0/1508099033 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fa6ec15b2c0 0x7fa6d80047c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:52:01.405 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:52:01.400+0000 7fa6eb7fe640 1 --2- 172.21.15.43:0/1508099033 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fa6d8003970 0x7fa6d8003d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:52:01.405 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:52:01.400+0000 7fa6ebfff640 1 -- 172.21.15.43:0/1508099033 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fa6d8003970 msgr2=0x7fa6d8003d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-09T13:52:01.405 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:52:01.400+0000 7fa6ebfff640 1 --2- 172.21.15.43:0/1508099033 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fa6d8003970 0x7fa6d8003d90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:52:01.405 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:52:01.400+0000 7fa6ebfff640 1 -- 172.21.15.43:0/1508099033 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa6d8003520 con 0x7fa6d8004d00 2023-08-09T13:52:01.406 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:52:01.401+0000 7fa6ebfff640 1 --2- 172.21.15.43:0/1508099033 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa6d8004d00 0x7fa6d8005180 secure :-1 s=READY pgs=1336 cs=0 l=1 rev1=1 crypto rx=0x7fa6e4009f90 tx=0x7fa6d80057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fc166d68587bee6c server_cookie=0 in_seq=0 out_seq=0 2023-08-09T13:52:01.406 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:52:01.402+0000 7fa6eaffd640 1 -- 172.21.15.43:0/1508099033 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa6e402cc20 con 0x7fa6d8004d00 2023-08-09T13:52:01.406 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:52:01.402+0000 7fa6eaffd640 1 -- 172.21.15.43:0/1508099033 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa6e400f040 con 0x7fa6d8004d00 2023-08-09T13:52:01.406 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:52:01.402+0000 7fa6eaffd640 1 -- 172.21.15.43:0/1508099033 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa6e4034440 con 0x7fa6d8004d00 2023-08-09T13:52:01.406 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:52:01.402+0000 7fa6f0b53640 1 -- 172.21.15.43:0/1508099033 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa6d8004d00 msgr2=0x7fa6d8005180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:52:01.406 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:52:01.402+0000 7fa6f0b53640 1 --2- 172.21.15.43:0/1508099033 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa6d8004d00 0x7fa6d8005180 secure :-1 s=READY pgs=1336 cs=0 l=1 rev1=1 crypto rx=0x7fa6e4009f90 tx=0x7fa6d80057f0 comp rx=0 tx=0).stop 2023-08-09T13:52:01.406 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:52:01.402+0000 7fa6ebfff640 1 -- 172.21.15.43:0/1508099033 reap_dead start 2023-08-09T13:52:01.407 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:52:01.402+0000 7fa6f0b53640 1 -- 172.21.15.43:0/1508099033 shutdown_connections 2023-08-09T13:52:01.407 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:52:01.402+0000 7fa6f0b53640 1 -- 172.21.15.43:0/1508099033 >> 172.21.15.43:0/1508099033 conn(0x7fa6ec0ce640 msgr2=0x7fa6ec0cea50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-09T13:52:01.407 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:52:01.402+0000 7fa6f0b53640 1 -- 172.21.15.43:0/1508099033 shutdown_connections 2023-08-09T13:52:01.407 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:52:01.402+0000 7fa6f0b53640 1 -- 172.21.15.43:0/1508099033 wait complete. 2023-08-09T13:52:01.408 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:52:01.404+0000 7fa6f0b53640 1 Processor -- start 2023-08-09T13:52:01.409 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:52:01.404+0000 7fa6f0b53640 1 -- start start 2023-08-09T13:52:01.409 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:52:01.405+0000 7fa6f0b53640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa6d8004d00 0x7fa6ec15c7b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:52:01.409 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:52:01.405+0000 7fa6f0b53640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fa6ec06cc50 0x7fa6ec06d0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:52:01.409 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:52:01.405+0000 7fa6f0b53640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fa6ec06d610 0x7fa6ec154b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:52:01.409 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:52:01.405+0000 7fa6f0b53640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa6ec06da90 con 0x7fa6d8004d00 2023-08-09T13:52:01.409 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:52:01.405+0000 7fa6f0b53640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa6ec06bc40 con 0x7fa6ec06cc50 2023-08-09T13:52:01.410 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:52:01.405+0000 7fa6f0b53640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa6ec06c7c0 con 0x7fa6ec06d610 2023-08-09T13:52:01.410 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:52:01.406+0000 7fa6eb7fe640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa6d8004d00 0x7fa6ec15c7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:52:01.410 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:52:01.406+0000 7fa6eb7fe640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa6d8004d00 0x7fa6ec15c7b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:33090/0 (socket says 172.21.15.43:33090) 2023-08-09T13:52:01.410 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:52:01.406+0000 7fa6eb7fe640 1 -- 172.21.15.43:0/3001974300 learned_addr learned my addr 172.21.15.43:0/3001974300 (peer_addr_for_me v2:172.21.15.43:0/0) 2023-08-09T13:52:01.410 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:52:01.406+0000 7fa6e2ffd640 1 --2- 172.21.15.43:0/3001974300 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fa6ec06cc50 0x7fa6ec06d0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:52:01.410 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:52:01.406+0000 7fa6ebfff640 1 --2- 172.21.15.43:0/3001974300 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fa6ec06d610 0x7fa6ec154b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:52:01.410 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:52:01.406+0000 7fa6eb7fe640 1 -- 172.21.15.43:0/3001974300 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fa6ec06d610 msgr2=0x7fa6ec154b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:52:01.410 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:52:01.406+0000 7fa6eb7fe640 1 --2- 172.21.15.43:0/3001974300 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fa6ec06d610 0x7fa6ec154b80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:52:01.411 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:52:01.406+0000 7fa6eb7fe640 1 -- 172.21.15.43:0/3001974300 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fa6ec06cc50 msgr2=0x7fa6ec06d0d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:52:01.411 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:52:01.406+0000 7fa6eb7fe640 1 --2- 172.21.15.43:0/3001974300 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fa6ec06cc50 0x7fa6ec06d0d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:52:01.411 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:52:01.406+0000 7fa6eb7fe640 1 -- 172.21.15.43:0/3001974300 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa6ec06be90 con 0x7fa6d8004d00 2023-08-09T13:52:01.411 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:52:01.407+0000 7fa6eb7fe640 1 --2- 172.21.15.43:0/3001974300 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa6d8004d00 0x7fa6ec15c7b0 secure :-1 s=READY pgs=1337 cs=0 l=1 rev1=1 crypto rx=0x7fa6dc010910 tx=0x7fa6dc010dd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-09T13:52:01.411 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:52:01.407+0000 7fa6e8ff9640 1 -- 172.21.15.43:0/3001974300 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa6dc009690 con 0x7fa6d8004d00 2023-08-09T13:52:01.411 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:52:01.407+0000 7fa6e8ff9640 1 -- 172.21.15.43:0/3001974300 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa6dc00ec20 con 0x7fa6d8004d00 2023-08-09T13:52:01.411 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:52:01.407+0000 7fa6f0b53640 1 -- 172.21.15.43:0/3001974300 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa6ec06c170 con 0x7fa6d8004d00 2023-08-09T13:52:01.411 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:52:01.407+0000 7fa6e8ff9640 1 -- 172.21.15.43:0/3001974300 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa6dc01c400 con 0x7fa6d8004d00 2023-08-09T13:52:01.412 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:52:01.407+0000 7fa6f0b53640 1 -- 172.21.15.43:0/3001974300 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa6ec15e980 con 0x7fa6d8004d00 2023-08-09T13:52:01.412 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:52:01.408+0000 7fa6f0b53640 1 -- 172.21.15.43:0/3001974300 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa6ec04e390 con 0x7fa6d8004d00 2023-08-09T13:52:01.413 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:52:01.409+0000 7fa6e8ff9640 1 -- 172.21.15.43:0/3001974300 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 45) v1 ==== 88443+0+0 (secure 0 0 0) 0x7fa6dc01c560 con 0x7fa6d8004d00 2023-08-09T13:52:01.413 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:52:01.410+0000 7fa6e8ff9640 1 --2- 172.21.15.43:0/3001974300 >> [v2:172.21.15.203:6832/1317108840,v1:172.21.15.203:6833/1317108840] conn(0x7fa6cc06a170 0x7fa6cc06c630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:52:01.413 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:52:01.410+0000 7fa6e8ff9640 1 -- 172.21.15.43:0/3001974300 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(116..116 src has 1..116) v4 ==== 6426+0+0 (secure 0 0 0) 0x7fa6dc088e60 con 0x7fa6d8004d00 2023-08-09T13:52:01.413 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:52:01.411+0000 7fa6e2ffd640 1 --2- 172.21.15.43:0/3001974300 >> [v2:172.21.15.203:6832/1317108840,v1:172.21.15.203:6833/1317108840] conn(0x7fa6cc06a170 0x7fa6cc06c630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:52:01.413 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:52:01.412+0000 7fa6e2ffd640 1 --2- 172.21.15.43:0/3001974300 >> [v2:172.21.15.203:6832/1317108840,v1:172.21.15.203:6833/1317108840] conn(0x7fa6cc06a170 0x7fa6cc06c630 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7fa6d40017a0 tx=0x7fa6d40016b0 comp rx=0 tx=0).ready entity=mgr.6619 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-09T13:52:01.416 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:52:01.414+0000 7fa6e8ff9640 1 -- 172.21.15.43:0/3001974300 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178066 (secure 0 0 0) 0x7fa6dc054370 con 0x7fa6d8004d00 2023-08-09T13:52:01.653 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:52:01.651+0000 7fa6f0b53640 1 -- 172.21.15.43:0/3001974300 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} v 0) v1 -- 0x7fa6ec072ae0 con 0x7fa6d8004d00 2023-08-09T13:52:02.226 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:52:02.224+0000 7fa6e8ff9640 1 -- 172.21.15.43:0/3001974300 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]=0 v51) v1 ==== 118+0+0 (secure 0 0 0) 0x7fa6dc057990 con 0x7fa6d8004d00 2023-08-09T13:52:02.230 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:52:02.228+0000 7fa6f0b53640 1 -- 172.21.15.43:0/3001974300 >> [v2:172.21.15.203:6832/1317108840,v1:172.21.15.203:6833/1317108840] conn(0x7fa6cc06a170 msgr2=0x7fa6cc06c630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:52:02.230 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:52:02.228+0000 7fa6f0b53640 1 --2- 172.21.15.43:0/3001974300 >> [v2:172.21.15.203:6832/1317108840,v1:172.21.15.203:6833/1317108840] conn(0x7fa6cc06a170 0x7fa6cc06c630 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7fa6d40017a0 tx=0x7fa6d40016b0 comp rx=0 tx=0).stop 2023-08-09T13:52:02.230 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:52:02.228+0000 7fa6f0b53640 1 -- 172.21.15.43:0/3001974300 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa6d8004d00 msgr2=0x7fa6ec15c7b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:52:02.231 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:52:02.228+0000 7fa6f0b53640 1 --2- 172.21.15.43:0/3001974300 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa6d8004d00 0x7fa6ec15c7b0 secure :-1 s=READY pgs=1337 cs=0 l=1 rev1=1 crypto rx=0x7fa6dc010910 tx=0x7fa6dc010dd0 comp rx=0 tx=0).stop 2023-08-09T13:52:02.231 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:52:02.229+0000 7fa6f0b53640 1 -- 172.21.15.43:0/3001974300 shutdown_connections 2023-08-09T13:52:02.231 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:52:02.229+0000 7fa6f0b53640 1 --2- 172.21.15.43:0/3001974300 >> [v2:172.21.15.203:6832/1317108840,v1:172.21.15.203:6833/1317108840] conn(0x7fa6cc06a170 0x7fa6cc06c630 unknown :-1 s=CLOSED pgs=61 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:52:02.231 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:52:02.229+0000 7fa6f0b53640 1 --2- 172.21.15.43:0/3001974300 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fa6ec06d610 0x7fa6ec154b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:52:02.231 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:52:02.229+0000 7fa6f0b53640 1 --2- 172.21.15.43:0/3001974300 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fa6ec06cc50 0x7fa6ec06d0d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:52:02.231 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:52:02.229+0000 7fa6f0b53640 1 --2- 172.21.15.43:0/3001974300 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa6d8004d00 0x7fa6ec15c7b0 unknown :-1 s=CLOSED pgs=1337 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:52:02.231 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:52:02.229+0000 7fa6f0b53640 1 -- 172.21.15.43:0/3001974300 >> 172.21.15.43:0/3001974300 conn(0x7fa6ec0ce640 msgr2=0x7fa6ec14ea10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-09T13:52:02.232 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:52:02.229+0000 7fa6f0b53640 1 -- 172.21.15.43:0/3001974300 shutdown_connections 2023-08-09T13:52:02.232 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:52:02.229+0000 7fa6f0b53640 1 -- 172.21.15.43:0/3001974300 wait complete. 2023-08-09T13:52:02.253 DEBUG:tasks.cephfs.filesystem:fs_config: {} 2023-08-09T13:52:02.253 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs allow_standby_replay false 2023-08-09T13:52:02.489 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:52:02.488+0000 7fcef02ad640 1 Processor -- start 2023-08-09T13:52:02.489 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:52:02.488+0000 7fcef02ad640 1 -- start start 2023-08-09T13:52:02.491 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:52:02.490+0000 7fcef02ad640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7fcee8143c20 0x7fcee806f540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:52:02.491 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:52:02.490+0000 7fcef02ad640 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7fcee80c86f0 0x7fcee80c8b50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:52:02.491 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:52:02.490+0000 7fcef02ad640 1 -- --> v1:172.21.15.203:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fcee80c9090 con 0x7fcee806c8d0 2023-08-09T13:52:02.492 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:52:02.490+0000 7fcef02ad640 1 -- --> v2:172.21.15.43:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fcee8144000 con 0x7fcee8143c20 2023-08-09T13:52:02.492 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:52:02.490+0000 7fcef02ad640 1 -- --> v2:172.21.15.203:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fcee806c440 con 0x7fcee80c86f0 2023-08-09T13:52:02.494 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:52:02.490+0000 7fceee049640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7fcee8143c20 0x7fcee806f540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:52:02.494 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:52:02.490+0000 7fceee049640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7fcee8143c20 0x7fcee806f540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:33092/0 (socket says 172.21.15.43:33092) 2023-08-09T13:52:02.494 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:52:02.490+0000 7fceee049640 1 -- 172.21.15.43:0/471885338 learned_addr learned my addr 172.21.15.43:0/471885338 (peer_addr_for_me v2:172.21.15.43:0/0) 2023-08-09T13:52:02.494 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:52:02.490+0000 7fceed848640 1 --2- 172.21.15.43:0/471885338 >> v2:172.21.15.203:3300/0 conn(0x7fcee80c86f0 0x7fcee80c8b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:52:02.494 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:52:02.490+0000 7fceee049640 1 -- 172.21.15.43:0/471885338 >> v2:172.21.15.203:3300/0 conn(0x7fcee80c86f0 msgr2=0x7fcee80c8b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-09T13:52:02.494 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:52:02.490+0000 7fceee049640 1 --2- 172.21.15.43:0/471885338 >> v2:172.21.15.203:3300/0 conn(0x7fcee80c86f0 0x7fcee80c8b50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:52:02.495 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:52:02.491+0000 7fceee049640 1 -- 172.21.15.43:0/471885338 >> v1:172.21.15.203:6789/0 conn(0x7fcee806c8d0 legacy=0x7fcee806cd00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-09T13:52:02.495 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:52:02.491+0000 7fceee049640 1 -- 172.21.15.43:0/471885338 --> v2:172.21.15.43:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcee806b830 con 0x7fcee8143c20 2023-08-09T13:52:02.495 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:52:02.492+0000 7fceee049640 1 --2- 172.21.15.43:0/471885338 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fcee8143c20 0x7fcee806f540 secure :-1 s=READY pgs=1338 cs=0 l=1 rev1=1 crypto rx=0x7fcee80c9b70 tx=0x7fcee402b7f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5f3aceb79ec95178 server_cookie=0 in_seq=0 out_seq=0 2023-08-09T13:52:02.495 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:52:02.492+0000 7fceed047640 1 -- 172.21.15.43:0/471885338 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcee4031750 con 0x7fcee8143c20 2023-08-09T13:52:02.495 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:52:02.492+0000 7fceed047640 1 -- 172.21.15.43:0/471885338 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcee40318b0 con 0x7fcee8143c20 2023-08-09T13:52:02.496 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:52:02.492+0000 7fceed047640 1 -- 172.21.15.43:0/471885338 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcee4031c40 con 0x7fcee8143c20 2023-08-09T13:52:02.497 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:52:02.493+0000 7fcef02ad640 1 -- 172.21.15.43:0/471885338 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fcee8143c20 msgr2=0x7fcee806f540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:52:02.497 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:52:02.493+0000 7fcef02ad640 1 --2- 172.21.15.43:0/471885338 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fcee8143c20 0x7fcee806f540 secure :-1 s=READY pgs=1338 cs=0 l=1 rev1=1 crypto rx=0x7fcee80c9b70 tx=0x7fcee402b7f0 comp rx=0 tx=0).stop 2023-08-09T13:52:02.497 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:52:02.493+0000 7fcef02ad640 1 -- 172.21.15.43:0/471885338 shutdown_connections 2023-08-09T13:52:02.497 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:52:02.493+0000 7fcef02ad640 1 --2- 172.21.15.43:0/471885338 >> v2:172.21.15.203:3300/0 conn(0x7fcee80c86f0 0x7fcee80c8b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:52:02.497 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:52:02.493+0000 7fcef02ad640 1 --2- 172.21.15.43:0/471885338 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fcee8143c20 0x7fcee806f540 unknown :-1 s=CLOSED pgs=1338 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:52:02.498 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:52:02.493+0000 7fcef02ad640 1 -- 172.21.15.43:0/471885338 >> 172.21.15.43:0/471885338 conn(0x7fcee80ce0a0 msgr2=0x7fcee80ce480 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-09T13:52:02.498 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:52:02.493+0000 7fcef02ad640 1 -- 172.21.15.43:0/471885338 shutdown_connections 2023-08-09T13:52:02.498 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:52:02.493+0000 7fcef02ad640 1 -- 172.21.15.43:0/471885338 wait complete. 2023-08-09T13:52:02.498 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:52:02.496+0000 7fcef02ad640 1 Processor -- start 2023-08-09T13:52:02.500 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:52:02.496+0000 7fcef02ad640 1 -- start start 2023-08-09T13:52:02.500 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:52:02.497+0000 7fcef02ad640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fcee806c8d0 0x7fcee80c75b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:52:02.500 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:52:02.497+0000 7fcef02ad640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fcee80c86f0 0x7fcee81327a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:52:02.500 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:52:02.497+0000 7fcef02ad640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fcee8143c20 0x7fcee80b3430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:52:02.501 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:52:02.497+0000 7fcef02ad640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcee80b2990 con 0x7fcee80c86f0 2023-08-09T13:52:02.501 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:52:02.497+0000 7fcef02ad640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcee81e35b0 con 0x7fcee8143c20 2023-08-09T13:52:02.501 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:52:02.497+0000 7fcef02ad640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcee81e3890 con 0x7fcee806c8d0 2023-08-09T13:52:02.501 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:52:02.497+0000 7fceed848640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fcee80c86f0 0x7fcee81327a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:52:02.501 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:52:02.497+0000 7fceed848640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fcee80c86f0 0x7fcee81327a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:33102/0 (socket says 172.21.15.43:33102) 2023-08-09T13:52:02.501 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:52:02.497+0000 7fceed848640 1 -- 172.21.15.43:0/3379914525 learned_addr learned my addr 172.21.15.43:0/3379914525 (peer_addr_for_me v2:172.21.15.43:0/0) 2023-08-09T13:52:02.501 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:52:02.498+0000 7fceee84a640 1 --2- 172.21.15.43:0/3379914525 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fcee8143c20 0x7fcee80b3430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:52:02.501 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:52:02.498+0000 7fceee049640 1 --2- 172.21.15.43:0/3379914525 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fcee806c8d0 0x7fcee80c75b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:52:02.502 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:52:02.498+0000 7fceed848640 1 -- 172.21.15.43:0/3379914525 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fcee806c8d0 msgr2=0x7fcee80c75b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:52:02.502 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:52:02.498+0000 7fceed848640 1 --2- 172.21.15.43:0/3379914525 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fcee806c8d0 0x7fcee80c75b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:52:02.502 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:52:02.498+0000 7fceed848640 1 -- 172.21.15.43:0/3379914525 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fcee8143c20 msgr2=0x7fcee80b3430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:52:02.502 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:52:02.498+0000 7fceed848640 1 --2- 172.21.15.43:0/3379914525 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fcee8143c20 0x7fcee80b3430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:52:02.502 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:52:02.498+0000 7fceed848640 1 -- 172.21.15.43:0/3379914525 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcee40094d0 con 0x7fcee80c86f0 2023-08-09T13:52:02.502 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:52:02.498+0000 7fceee84a640 1 --2- 172.21.15.43:0/3379914525 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fcee8143c20 0x7fcee80b3430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-08-09T13:52:02.502 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:52:02.498+0000 7fceed848640 1 --2- 172.21.15.43:0/3379914525 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fcee80c86f0 0x7fcee81327a0 secure :-1 s=READY pgs=1339 cs=0 l=1 rev1=1 crypto rx=0x7fced800d5f0 tx=0x7fced800dab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-09T13:52:02.503 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:52:02.499+0000 7fcedeffd640 1 -- 172.21.15.43:0/3379914525 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fced80125f0 con 0x7fcee80c86f0 2023-08-09T13:52:02.503 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:52:02.499+0000 7fcedeffd640 1 -- 172.21.15.43:0/3379914525 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fced8012750 con 0x7fcee80c86f0 2023-08-09T13:52:02.503 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:52:02.499+0000 7fcedeffd640 1 -- 172.21.15.43:0/3379914525 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fced8012b00 con 0x7fcee80c86f0 2023-08-09T13:52:02.503 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:52:02.499+0000 7fcef02ad640 1 -- 172.21.15.43:0/3379914525 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcee80b2c30 con 0x7fcee80c86f0 2023-08-09T13:52:02.503 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:52:02.499+0000 7fcef02ad640 1 -- 172.21.15.43:0/3379914525 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcee80c7af0 con 0x7fcee80c86f0 2023-08-09T13:52:02.504 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:52:02.500+0000 7fcef02ad640 1 -- 172.21.15.43:0/3379914525 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcee80c7cb0 con 0x7fcee80c86f0 2023-08-09T13:52:02.504 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:52:02.501+0000 7fcedeffd640 1 -- 172.21.15.43:0/3379914525 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 45) v1 ==== 88443+0+0 (secure 0 0 0) 0x7fced8012c60 con 0x7fcee80c86f0 2023-08-09T13:52:02.504 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:52:02.502+0000 7fcedeffd640 1 --2- 172.21.15.43:0/3379914525 >> [v2:172.21.15.203:6832/1317108840,v1:172.21.15.203:6833/1317108840] conn(0x7fcecc06a1a0 0x7fcecc06c660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:52:02.504 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:52:02.502+0000 7fcedeffd640 1 -- 172.21.15.43:0/3379914525 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(116..116 src has 1..116) v4 ==== 6426+0+0 (secure 0 0 0) 0x7fced8088d80 con 0x7fcee80c86f0 2023-08-09T13:52:02.505 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:52:02.503+0000 7fceee049640 1 --2- 172.21.15.43:0/3379914525 >> [v2:172.21.15.203:6832/1317108840,v1:172.21.15.203:6833/1317108840] conn(0x7fcecc06a1a0 0x7fcecc06c660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:52:02.505 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:52:02.504+0000 7fceee049640 1 --2- 172.21.15.43:0/3379914525 >> [v2:172.21.15.203:6832/1317108840,v1:172.21.15.203:6833/1317108840] conn(0x7fcecc06a1a0 0x7fcecc06c660 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7fcee4035000 tx=0x7fcee4008030 comp rx=0 tx=0).ready entity=mgr.6619 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-09T13:52:02.508 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:52:02.507+0000 7fcedeffd640 1 -- 172.21.15.43:0/3379914525 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178066 (secure 0 0 0) 0x7fced8054150 con 0x7fcee80c86f0 2023-08-09T13:52:02.743 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:52:02.742+0000 7fcef02ad640 1 -- 172.21.15.43:0/3379914525 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"} v 0) v1 -- 0x7fcee806c0a0 con 0x7fcee80c86f0 2023-08-09T13:52:03.241 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:52:03.240+0000 7fcedeffd640 1 -- 172.21.15.43:0/3379914525 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"}]=0 v52) v1 ==== 122+0+0 (secure 0 0 0) 0x7fced8057770 con 0x7fcee80c86f0 2023-08-09T13:52:03.245 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:52:03.243+0000 7fcef02ad640 1 -- 172.21.15.43:0/3379914525 >> [v2:172.21.15.203:6832/1317108840,v1:172.21.15.203:6833/1317108840] conn(0x7fcecc06a1a0 msgr2=0x7fcecc06c660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:52:03.245 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:52:03.243+0000 7fcef02ad640 1 --2- 172.21.15.43:0/3379914525 >> [v2:172.21.15.203:6832/1317108840,v1:172.21.15.203:6833/1317108840] conn(0x7fcecc06a1a0 0x7fcecc06c660 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7fcee4035000 tx=0x7fcee4008030 comp rx=0 tx=0).stop 2023-08-09T13:52:03.245 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:52:03.243+0000 7fcef02ad640 1 -- 172.21.15.43:0/3379914525 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fcee80c86f0 msgr2=0x7fcee81327a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:52:03.246 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:52:03.243+0000 7fcef02ad640 1 --2- 172.21.15.43:0/3379914525 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fcee80c86f0 0x7fcee81327a0 secure :-1 s=READY pgs=1339 cs=0 l=1 rev1=1 crypto rx=0x7fced800d5f0 tx=0x7fced800dab0 comp rx=0 tx=0).stop 2023-08-09T13:52:03.246 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:52:03.243+0000 7fcef02ad640 1 -- 172.21.15.43:0/3379914525 shutdown_connections 2023-08-09T13:52:03.246 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:52:03.243+0000 7fcef02ad640 1 --2- 172.21.15.43:0/3379914525 >> [v2:172.21.15.203:6832/1317108840,v1:172.21.15.203:6833/1317108840] conn(0x7fcecc06a1a0 0x7fcecc06c660 unknown :-1 s=CLOSED pgs=62 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:52:03.246 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:52:03.243+0000 7fcef02ad640 1 --2- 172.21.15.43:0/3379914525 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fcee8143c20 0x7fcee80b3430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:52:03.246 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:52:03.243+0000 7fcef02ad640 1 --2- 172.21.15.43:0/3379914525 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fcee80c86f0 0x7fcee81327a0 unknown :-1 s=CLOSED pgs=1339 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:52:03.246 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:52:03.243+0000 7fcef02ad640 1 --2- 172.21.15.43:0/3379914525 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fcee806c8d0 0x7fcee80c75b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:52:03.246 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:52:03.243+0000 7fcef02ad640 1 -- 172.21.15.43:0/3379914525 >> 172.21.15.43:0/3379914525 conn(0x7fcee80ce0a0 msgr2=0x7fcee80cc260 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-09T13:52:03.247 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:52:03.244+0000 7fcef02ad640 1 -- 172.21.15.43:0/3379914525 shutdown_connections 2023-08-09T13:52:03.247 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:52:03.244+0000 7fcef02ad640 1 -- 172.21.15.43:0/3379914525 wait complete. 2023-08-09T13:52:03.267 DEBUG:tasks.cephfs.filesystem:Not Creating any subvols for filesystem 'cephfs' 2023-08-09T13:52:03.267 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-08-09T13:52:03.487 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:52:03.485+0000 7f75059b1640 1 Processor -- start 2023-08-09T13:52:03.487 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:52:03.485+0000 7f75059b1640 1 -- start start 2023-08-09T13:52:03.488 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:52:03.487+0000 7f75059b1640 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7f75000c86f0 0x7f75000c8b50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:52:03.489 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:52:03.487+0000 7f75059b1640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f750006c8d0 0x7f750006cd00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:52:03.489 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:52:03.487+0000 7f75059b1640 1 -- --> v1:172.21.15.43:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f75000c9090 con 0x7f7500143c20 2023-08-09T13:52:03.489 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:52:03.487+0000 7f75059b1640 1 -- --> v2:172.21.15.43:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7500144000 con 0x7f750006c8d0 2023-08-09T13:52:03.489 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:52:03.487+0000 7f75059b1640 1 -- --> v2:172.21.15.203:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f750006c440 con 0x7f75000c86f0 2023-08-09T13:52:03.490 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:52:03.488+0000 7f74feffd640 1 --1- >> v1:172.21.15.43:6789/0 conn(0x7f7500143c20 0x7f750006f540 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.43:6789/0 says I am v1:172.21.15.43:43772/0 (socket says 172.21.15.43:43772) 2023-08-09T13:52:03.490 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:52:03.488+0000 7f74feffd640 1 -- 172.21.15.43:0/570362623 learned_addr learned my addr 172.21.15.43:0/570362623 (peer_addr_for_me v1:172.21.15.43:0/0) 2023-08-09T13:52:03.490 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:52:03.488+0000 7f74ff7fe640 1 --2- 172.21.15.43:0/570362623 >> v2:172.21.15.43:3300/0 conn(0x7f750006c8d0 0x7f750006cd00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:52:03.491 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:52:03.488+0000 7f74fe7fc640 1 --2- 172.21.15.43:0/570362623 >> v2:172.21.15.203:3300/0 conn(0x7f75000c86f0 0x7f75000c8b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:52:03.491 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:52:03.488+0000 7f74fdffb640 1 -- 172.21.15.43:0/570362623 <== mon.0 v1:172.21.15.43:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1053131014 0 0) 0x7f75000c9090 con 0x7f7500143c20 2023-08-09T13:52:03.491 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:52:03.488+0000 7f74fdffb640 1 -- 172.21.15.43:0/570362623 --> v1:172.21.15.43:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f74e8003520 con 0x7f7500143c20 2023-08-09T13:52:03.491 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:52:03.488+0000 7f74ff7fe640 1 -- 172.21.15.43:0/570362623 >> v2:172.21.15.203:3300/0 conn(0x7f75000c86f0 msgr2=0x7f75000c8b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-09T13:52:03.491 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:52:03.488+0000 7f74ff7fe640 1 --2- 172.21.15.43:0/570362623 >> v2:172.21.15.203:3300/0 conn(0x7f75000c86f0 0x7f75000c8b50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:52:03.491 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:52:03.489+0000 7f74ff7fe640 1 -- 172.21.15.43:0/570362623 >> v1:172.21.15.43:6789/0 conn(0x7f7500143c20 legacy=0x7f750006f540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:52:03.491 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:52:03.489+0000 7f74ff7fe640 1 -- 172.21.15.43:0/570362623 --> v2:172.21.15.43:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f750006fc50 con 0x7f750006c8d0 2023-08-09T13:52:03.492 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:52:03.490+0000 7f74ff7fe640 1 --2- 172.21.15.43:0/570362623 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f750006c8d0 0x7f750006cd00 secure :-1 s=READY pgs=1341 cs=0 l=1 rev1=1 crypto rx=0x7f75000c9b70 tx=0x7f74f802be40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ceefe3209294ada6 server_cookie=0 in_seq=0 out_seq=0 2023-08-09T13:52:03.492 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:52:03.491+0000 7f74fdffb640 1 -- 172.21.15.43:0/570362623 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f74f802cbf0 con 0x7f750006c8d0 2023-08-09T13:52:03.492 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:52:03.491+0000 7f74fdffb640 1 -- 172.21.15.43:0/570362623 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f74f802cd50 con 0x7f750006c8d0 2023-08-09T13:52:03.493 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:52:03.491+0000 7f74fdffb640 1 -- 172.21.15.43:0/570362623 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f74f8034660 con 0x7f750006c8d0 2023-08-09T13:52:03.493 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:52:03.491+0000 7f75059b1640 1 -- 172.21.15.43:0/570362623 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f750006c8d0 msgr2=0x7f750006cd00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:52:03.493 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:52:03.491+0000 7f75059b1640 1 --2- 172.21.15.43:0/570362623 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f750006c8d0 0x7f750006cd00 secure :-1 s=READY pgs=1341 cs=0 l=1 rev1=1 crypto rx=0x7f75000c9b70 tx=0x7f74f802be40 comp rx=0 tx=0).stop 2023-08-09T13:52:03.494 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:52:03.491+0000 7f75059b1640 1 -- 172.21.15.43:0/570362623 shutdown_connections 2023-08-09T13:52:03.494 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:52:03.491+0000 7f75059b1640 1 --2- 172.21.15.43:0/570362623 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f750006c8d0 0x7f750006cd00 unknown :-1 s=CLOSED pgs=1341 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:52:03.494 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:52:03.491+0000 7f75059b1640 1 --2- 172.21.15.43:0/570362623 >> v2:172.21.15.203:3300/0 conn(0x7f75000c86f0 0x7f75000c8b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:52:03.494 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:52:03.491+0000 7f75059b1640 1 -- 172.21.15.43:0/570362623 >> 172.21.15.43:0/570362623 conn(0x7f75000ce0a0 msgr2=0x7f75000ce480 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-09T13:52:03.494 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:52:03.491+0000 7f75059b1640 1 -- 172.21.15.43:0/570362623 shutdown_connections 2023-08-09T13:52:03.494 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:52:03.491+0000 7f75059b1640 1 -- 172.21.15.43:0/570362623 wait complete. 2023-08-09T13:52:03.495 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:52:03.494+0000 7f75059b1640 1 Processor -- start 2023-08-09T13:52:03.495 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:52:03.494+0000 7f75059b1640 1 -- start start 2023-08-09T13:52:03.497 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:52:03.495+0000 7f75059b1640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f750006c8d0 0x7f75001f4ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:52:03.497 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:52:03.495+0000 7f75059b1640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f75000c86f0 0x7f75001f53e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:52:03.497 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:52:03.495+0000 7f75059b1640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f7500143c20 0x7f75000b2da0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:52:03.497 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:52:03.495+0000 7f75059b1640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f75001412e0 con 0x7f75000c86f0 2023-08-09T13:52:03.498 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:52:03.495+0000 7f75059b1640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f75001406d0 con 0x7f7500143c20 2023-08-09T13:52:03.498 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:52:03.495+0000 7f75059b1640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f75001409b0 con 0x7f750006c8d0 2023-08-09T13:52:03.498 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:52:03.495+0000 7f74fe7fc640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f75000c86f0 0x7f75001f53e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:52:03.498 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:52:03.495+0000 7f74fe7fc640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f75000c86f0 0x7f75001f53e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:33132/0 (socket says 172.21.15.43:33132) 2023-08-09T13:52:03.499 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:52:03.495+0000 7f74fe7fc640 1 -- 172.21.15.43:0/1961370034 learned_addr learned my addr 172.21.15.43:0/1961370034 (peer_addr_for_me v2:172.21.15.43:0/0) 2023-08-09T13:52:03.500 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:52:03.496+0000 7f74ff7fe640 1 --2- 172.21.15.43:0/1961370034 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f7500143c20 0x7f75000b2da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:52:03.501 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:52:03.496+0000 7f74feffd640 1 --2- 172.21.15.43:0/1961370034 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f750006c8d0 0x7f75001f4ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:52:03.501 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:52:03.496+0000 7f74fe7fc640 1 -- 172.21.15.43:0/1961370034 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f750006c8d0 msgr2=0x7f75001f4ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:52:03.501 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:52:03.496+0000 7f74fe7fc640 1 --2- 172.21.15.43:0/1961370034 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f750006c8d0 0x7f75001f4ea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:52:03.501 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:52:03.496+0000 7f74fe7fc640 1 -- 172.21.15.43:0/1961370034 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f7500143c20 msgr2=0x7f75000b2da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:52:03.502 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:52:03.496+0000 7f74fe7fc640 1 --2- 172.21.15.43:0/1961370034 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f7500143c20 0x7f75000b2da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:52:03.502 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:52:03.496+0000 7f74fe7fc640 1 -- 172.21.15.43:0/1961370034 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f74f8009a70 con 0x7f75000c86f0 2023-08-09T13:52:03.502 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:52:03.496+0000 7f74ff7fe640 1 --2- 172.21.15.43:0/1961370034 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f7500143c20 0x7f75000b2da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-08-09T13:52:03.502 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:52:03.496+0000 7f74feffd640 1 --2- 172.21.15.43:0/1961370034 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f750006c8d0 0x7f75001f4ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-08-09T13:52:03.502 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:52:03.496+0000 7f74fe7fc640 1 --2- 172.21.15.43:0/1961370034 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f75000c86f0 0x7f75001f53e0 secure :-1 s=READY pgs=1342 cs=0 l=1 rev1=1 crypto rx=0x7f74f400e930 tx=0x7f74f400edf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-09T13:52:03.502 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:52:03.496+0000 7f74e7fff640 1 -- 172.21.15.43:0/1961370034 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f74f4009690 con 0x7f75000c86f0 2023-08-09T13:52:03.502 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:52:03.497+0000 7f74e7fff640 1 -- 172.21.15.43:0/1961370034 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f74f400ca00 con 0x7f75000c86f0 2023-08-09T13:52:03.503 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:52:03.497+0000 7f74e7fff640 1 -- 172.21.15.43:0/1961370034 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f74f400cdb0 con 0x7f75000c86f0 2023-08-09T13:52:03.503 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:52:03.497+0000 7f75059b1640 1 -- 172.21.15.43:0/1961370034 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7500140c90 con 0x7f75000c86f0 2023-08-09T13:52:03.503 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:52:03.497+0000 7f75059b1640 1 -- 172.21.15.43:0/1961370034 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f75001f1650 con 0x7f75000c86f0 2023-08-09T13:52:03.503 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:52:03.498+0000 7f75059b1640 1 -- 172.21.15.43:0/1961370034 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f75000739f0 con 0x7f75000c86f0 2023-08-09T13:52:03.503 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:52:03.499+0000 7f74e7fff640 1 -- 172.21.15.43:0/1961370034 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 45) v1 ==== 88443+0+0 (secure 0 0 0) 0x7f74f4013470 con 0x7f75000c86f0 2023-08-09T13:52:03.504 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:52:03.500+0000 7f74e7fff640 1 --2- 172.21.15.43:0/1961370034 >> [v2:172.21.15.203:6832/1317108840,v1:172.21.15.203:6833/1317108840] conn(0x7f74ec06a170 0x7f74ec06c630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:52:03.504 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:52:03.500+0000 7f74e7fff640 1 -- 172.21.15.43:0/1961370034 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(116..116 src has 1..116) v4 ==== 6426+0+0 (secure 0 0 0) 0x7f74f4088b90 con 0x7f75000c86f0 2023-08-09T13:52:03.504 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:52:03.501+0000 7f74feffd640 1 --2- 172.21.15.43:0/1961370034 >> [v2:172.21.15.203:6832/1317108840,v1:172.21.15.203:6833/1317108840] conn(0x7f74ec06a170 0x7f74ec06c630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:52:03.505 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:52:03.502+0000 7f74feffd640 1 --2- 172.21.15.43:0/1961370034 >> [v2:172.21.15.203:6832/1317108840,v1:172.21.15.203:6833/1317108840] conn(0x7f74ec06a170 0x7f74ec06c630 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f74f00011e0 tx=0x7f74f0009040 comp rx=0 tx=0).ready entity=mgr.6619 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-09T13:52:03.505 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:52:03.504+0000 7f74e7fff640 1 -- 172.21.15.43:0/1961370034 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178066 (secure 0 0 0) 0x7f74f40540b0 con 0x7f75000c86f0 2023-08-09T13:52:03.741 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:52:03.740+0000 7f75059b1640 1 -- 172.21.15.43:0/1961370034 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f75000cec90 con 0x7f75000c86f0 2023-08-09T13:52:03.742 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:52:03.741+0000 7f74e7fff640 1 -- 172.21.15.43:0/1961370034 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 52 v52) v1 ==== 94+0+4883 (secure 0 0 0) 0x7f74f4088070 con 0x7f75000c86f0 2023-08-09T13:52:03.743 INFO:teuthology.orchestra.run.smithi043.stdout: 2023-08-09T13:52:03.743 INFO:teuthology.orchestra.run.smithi043.stdout:{"epoch":52,"default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6215,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.43:6835/2214346432","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6833","nonce":2214346432},{"type":"v1","addr":"172.21.15.43:6835","nonce":2214346432}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":47},{"gid":6823,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.203:6835/3236264045","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6834","nonce":3236264045},{"type":"v1","addr":"172.21.15.203:6835","nonce":3236264045}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":46},{"gid":6826,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.203:6837/3788289156","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6836","nonce":3788289156},{"type":"v1","addr":"172.21.15.203:6837","nonce":3788289156}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":47}],"filesystems":[{"mdsmap":{"epoch":52,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-08-09T13:51:59.655630+0000","modified":"2023-08-09T13:52:03.220527+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":8040},"failed":[],"damaged":[],"stopped":[],"info":{"gid_8040":{"gid":8040,"name":"c","rank":0,"incarnation":48,"state":"up:active","state_seq":3,"addr":"172.21.15.43:6834/2223580538","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6832","nonce":2223580538},{"type":"v1","addr":"172.21.15.43:6834","nonce":2223580538}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[15],"metadata_pool":14,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":6}]} 2023-08-09T13:52:03.744 INFO:teuthology.orchestra.run.smithi043.stderr:dumped fsmap epoch 52 2023-08-09T13:52:03.746 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:52:03.743+0000 7f75059b1640 1 -- 172.21.15.43:0/1961370034 >> [v2:172.21.15.203:6832/1317108840,v1:172.21.15.203:6833/1317108840] conn(0x7f74ec06a170 msgr2=0x7f74ec06c630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:52:03.746 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:52:03.743+0000 7f75059b1640 1 --2- 172.21.15.43:0/1961370034 >> [v2:172.21.15.203:6832/1317108840,v1:172.21.15.203:6833/1317108840] conn(0x7f74ec06a170 0x7f74ec06c630 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f74f00011e0 tx=0x7f74f0009040 comp rx=0 tx=0).stop 2023-08-09T13:52:03.746 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:52:03.744+0000 7f75059b1640 1 -- 172.21.15.43:0/1961370034 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f75000c86f0 msgr2=0x7f75001f53e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:52:03.747 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:52:03.744+0000 7f75059b1640 1 --2- 172.21.15.43:0/1961370034 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f75000c86f0 0x7f75001f53e0 secure :-1 s=READY pgs=1342 cs=0 l=1 rev1=1 crypto rx=0x7f74f400e930 tx=0x7f74f400edf0 comp rx=0 tx=0).stop 2023-08-09T13:52:03.747 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:52:03.744+0000 7f75059b1640 1 -- 172.21.15.43:0/1961370034 shutdown_connections 2023-08-09T13:52:03.747 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:52:03.744+0000 7f75059b1640 1 --2- 172.21.15.43:0/1961370034 >> [v2:172.21.15.203:6832/1317108840,v1:172.21.15.203:6833/1317108840] conn(0x7f74ec06a170 0x7f74ec06c630 unknown :-1 s=CLOSED pgs=63 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:52:03.748 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:52:03.744+0000 7f75059b1640 1 --2- 172.21.15.43:0/1961370034 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f7500143c20 0x7f75000b2da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:52:03.748 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:52:03.744+0000 7f75059b1640 1 --2- 172.21.15.43:0/1961370034 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f75000c86f0 0x7f75001f53e0 unknown :-1 s=CLOSED pgs=1342 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:52:03.748 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:52:03.744+0000 7f75059b1640 1 --2- 172.21.15.43:0/1961370034 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f750006c8d0 0x7f75001f4ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:52:03.748 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:52:03.744+0000 7f75059b1640 1 -- 172.21.15.43:0/1961370034 >> 172.21.15.43:0/1961370034 conn(0x7f75000ce0a0 msgr2=0x7f75000aeae0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-09T13:52:03.749 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:52:03.744+0000 7f75059b1640 1 -- 172.21.15.43:0/1961370034 shutdown_connections 2023-08-09T13:52:03.749 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:52:03.744+0000 7f75059b1640 1 -- 172.21.15.43:0/1961370034 wait complete. 2023-08-09T13:52:03.765 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-08-09T13:52:03.990 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:52:03.989+0000 7f3f7beb8640 1 Processor -- start 2023-08-09T13:52:03.991 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:52:03.989+0000 7f3f7beb8640 1 -- start start 2023-08-09T13:52:03.992 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:52:03.990+0000 7f3f7beb8640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f3f74143c20 0x7f3f7406f540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:52:03.992 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:52:03.990+0000 7f3f7beb8640 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7f3f740c86f0 0x7f3f740c8b50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:52:03.992 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:52:03.991+0000 7f3f7beb8640 1 -- --> v1:172.21.15.203:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3f740c9090 con 0x7f3f7406c8d0 2023-08-09T13:52:03.993 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:52:03.991+0000 7f3f7beb8640 1 -- --> v2:172.21.15.43:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3f74144000 con 0x7f3f74143c20 2023-08-09T13:52:03.993 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:52:03.991+0000 7f3f7beb8640 1 -- --> v2:172.21.15.203:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3f7406c440 con 0x7f3f740c86f0 2023-08-09T13:52:03.993 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:52:03.991+0000 7f3f79c54640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f3f74143c20 0x7f3f7406f540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:52:03.993 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:52:03.991+0000 7f3f79c54640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f3f74143c20 0x7f3f7406f540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:33144/0 (socket says 172.21.15.43:33144) 2023-08-09T13:52:03.993 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:52:03.991+0000 7f3f79c54640 1 -- 172.21.15.43:0/3184178150 learned_addr learned my addr 172.21.15.43:0/3184178150 (peer_addr_for_me v2:172.21.15.43:0/0) 2023-08-09T13:52:03.994 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:52:03.991+0000 7f3f79453640 1 --2- 172.21.15.43:0/3184178150 >> v2:172.21.15.203:3300/0 conn(0x7f3f740c86f0 0x7f3f740c8b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:52:03.994 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:52:03.991+0000 7f3f79c54640 1 -- 172.21.15.43:0/3184178150 >> v2:172.21.15.203:3300/0 conn(0x7f3f740c86f0 msgr2=0x7f3f740c8b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-09T13:52:03.994 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:52:03.991+0000 7f3f79c54640 1 --2- 172.21.15.43:0/3184178150 >> v2:172.21.15.203:3300/0 conn(0x7f3f740c86f0 0x7f3f740c8b50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:52:03.995 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:52:03.991+0000 7f3f79c54640 1 -- 172.21.15.43:0/3184178150 >> v1:172.21.15.203:6789/0 conn(0x7f3f7406c8d0 legacy=0x7f3f7406cd00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-09T13:52:03.995 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:52:03.991+0000 7f3f79c54640 1 -- 172.21.15.43:0/3184178150 --> v2:172.21.15.43:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3f7406b830 con 0x7f3f74143c20 2023-08-09T13:52:03.995 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:52:03.991+0000 7f3f79453640 1 --2- 172.21.15.43:0/3184178150 >> v2:172.21.15.203:3300/0 conn(0x7f3f740c86f0 0x7f3f740c8b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-08-09T13:52:03.995 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:52:03.993+0000 7f3f79c54640 1 --2- 172.21.15.43:0/3184178150 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f3f74143c20 0x7f3f7406f540 secure :-1 s=READY pgs=1343 cs=0 l=1 rev1=1 crypto rx=0x7f3f740c9b70 tx=0x7f3f7002b7f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=52458102d91d2edb server_cookie=0 in_seq=0 out_seq=0 2023-08-09T13:52:03.995 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:52:03.993+0000 7f3f78c52640 1 -- 172.21.15.43:0/3184178150 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3f70031750 con 0x7f3f74143c20 2023-08-09T13:52:03.995 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:52:03.993+0000 7f3f78c52640 1 -- 172.21.15.43:0/3184178150 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3f700318b0 con 0x7f3f74143c20 2023-08-09T13:52:03.995 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:52:03.993+0000 7f3f78c52640 1 -- 172.21.15.43:0/3184178150 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3f70031c40 con 0x7f3f74143c20 2023-08-09T13:52:03.996 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:52:03.993+0000 7f3f7beb8640 1 -- 172.21.15.43:0/3184178150 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f3f74143c20 msgr2=0x7f3f7406f540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:52:03.996 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:52:03.993+0000 7f3f7beb8640 1 --2- 172.21.15.43:0/3184178150 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f3f74143c20 0x7f3f7406f540 secure :-1 s=READY pgs=1343 cs=0 l=1 rev1=1 crypto rx=0x7f3f740c9b70 tx=0x7f3f7002b7f0 comp rx=0 tx=0).stop 2023-08-09T13:52:03.996 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:52:03.994+0000 7f3f7beb8640 1 -- 172.21.15.43:0/3184178150 shutdown_connections 2023-08-09T13:52:03.997 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:52:03.994+0000 7f3f7beb8640 1 --2- 172.21.15.43:0/3184178150 >> v2:172.21.15.203:3300/0 conn(0x7f3f740c86f0 0x7f3f740c8b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:52:03.997 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:52:03.994+0000 7f3f7beb8640 1 --2- 172.21.15.43:0/3184178150 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f3f74143c20 0x7f3f7406f540 unknown :-1 s=CLOSED pgs=1343 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:52:03.997 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:52:03.994+0000 7f3f7beb8640 1 -- 172.21.15.43:0/3184178150 >> 172.21.15.43:0/3184178150 conn(0x7f3f740ce0a0 msgr2=0x7f3f740ce480 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-09T13:52:03.997 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:52:03.994+0000 7f3f7beb8640 1 -- 172.21.15.43:0/3184178150 shutdown_connections 2023-08-09T13:52:03.997 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:52:03.994+0000 7f3f7beb8640 1 -- 172.21.15.43:0/3184178150 wait complete. 2023-08-09T13:52:03.998 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:52:03.995+0000 7f3f7beb8640 1 Processor -- start 2023-08-09T13:52:03.998 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:52:03.995+0000 7f3f7beb8640 1 -- start start 2023-08-09T13:52:03.998 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:52:03.996+0000 7f3f7beb8640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f3f7406c8d0 0x7f3f74141d20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:52:03.998 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:52:03.996+0000 7f3f7beb8640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f3f740c86f0 0x7f3f740c5b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:52:03.999 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:52:03.996+0000 7f3f7beb8640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f3f74143c20 0x7f3f7415cdd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:52:03.999 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:52:03.996+0000 7f3f7beb8640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3f740c58d0 con 0x7f3f740c86f0 2023-08-09T13:52:03.999 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:52:03.996+0000 7f3f7beb8640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3f7415c940 con 0x7f3f74143c20 2023-08-09T13:52:03.999 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:52:03.996+0000 7f3f7beb8640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3f740c5310 con 0x7f3f7406c8d0 2023-08-09T13:52:03.999 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:52:03.997+0000 7f3f79453640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f3f740c86f0 0x7f3f740c5b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:52:03.999 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:52:03.997+0000 7f3f79453640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f3f740c86f0 0x7f3f740c5b80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:33156/0 (socket says 172.21.15.43:33156) 2023-08-09T13:52:03.999 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:52:03.997+0000 7f3f79453640 1 -- 172.21.15.43:0/3485484102 learned_addr learned my addr 172.21.15.43:0/3485484102 (peer_addr_for_me v2:172.21.15.43:0/0) 2023-08-09T13:52:04.000 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:52:03.997+0000 7f3f79c54640 1 --2- 172.21.15.43:0/3485484102 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f3f7406c8d0 0x7f3f74141d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:52:04.001 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:52:03.997+0000 7f3f79453640 1 -- 172.21.15.43:0/3485484102 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f3f7406c8d0 msgr2=0x7f3f74141d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:52:04.001 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:52:03.997+0000 7f3f79453640 1 --2- 172.21.15.43:0/3485484102 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f3f7406c8d0 0x7f3f74141d20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:52:04.001 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:52:03.997+0000 7f3f79453640 1 -- 172.21.15.43:0/3485484102 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f3f74143c20 msgr2=0x7f3f7415cdd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:52:04.001 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:52:03.997+0000 7f3f79453640 1 --2- 172.21.15.43:0/3485484102 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f3f74143c20 0x7f3f7415cdd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:52:04.001 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:52:03.997+0000 7f3f79453640 1 -- 172.21.15.43:0/3485484102 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3f700094d0 con 0x7f3f740c86f0 2023-08-09T13:52:04.001 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:52:03.997+0000 7f3f79453640 1 --2- 172.21.15.43:0/3485484102 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f3f740c86f0 0x7f3f740c5b80 secure :-1 s=READY pgs=1344 cs=0 l=1 rev1=1 crypto rx=0x7f3f6400c7f0 tx=0x7f3f6400ccb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-09T13:52:04.001 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:52:03.998+0000 7f3f6a7fc640 1 -- 172.21.15.43:0/3485484102 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3f64007820 con 0x7f3f740c86f0 2023-08-09T13:52:04.001 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:52:03.998+0000 7f3f6a7fc640 1 -- 172.21.15.43:0/3485484102 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3f64007980 con 0x7f3f740c86f0 2023-08-09T13:52:04.002 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:52:03.998+0000 7f3f6a7fc640 1 -- 172.21.15.43:0/3485484102 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3f64007cf0 con 0x7f3f740c86f0 2023-08-09T13:52:04.002 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:52:03.998+0000 7f3f7beb8640 1 -- 172.21.15.43:0/3485484102 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3f740c55f0 con 0x7f3f740c86f0 2023-08-09T13:52:04.002 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:52:03.998+0000 7f3f7beb8640 1 -- 172.21.15.43:0/3485484102 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3f740c60c0 con 0x7f3f740c86f0 2023-08-09T13:52:04.002 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:52:03.999+0000 7f3f7beb8640 1 -- 172.21.15.43:0/3485484102 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3f7415cad0 con 0x7f3f740c86f0 2023-08-09T13:52:04.002 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:52:04.000+0000 7f3f6a7fc640 1 -- 172.21.15.43:0/3485484102 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 45) v1 ==== 88443+0+0 (secure 0 0 0) 0x7f3f640113f0 con 0x7f3f740c86f0 2023-08-09T13:52:04.003 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:52:04.001+0000 7f3f6a7fc640 1 --2- 172.21.15.43:0/3485484102 >> [v2:172.21.15.203:6832/1317108840,v1:172.21.15.203:6833/1317108840] conn(0x7f3f5806a170 0x7f3f5806c630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:52:04.003 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:52:04.002+0000 7f3f6a7fc640 1 -- 172.21.15.43:0/3485484102 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(116..116 src has 1..116) v4 ==== 6426+0+0 (secure 0 0 0) 0x7f3f64088d20 con 0x7f3f740c86f0 2023-08-09T13:52:04.003 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:52:04.002+0000 7f3f79c54640 1 --2- 172.21.15.43:0/3485484102 >> [v2:172.21.15.203:6832/1317108840,v1:172.21.15.203:6833/1317108840] conn(0x7f3f5806a170 0x7f3f5806c630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:52:04.005 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:52:04.003+0000 7f3f79c54640 1 --2- 172.21.15.43:0/3485484102 >> [v2:172.21.15.203:6832/1317108840,v1:172.21.15.203:6833/1317108840] conn(0x7f3f5806a170 0x7f3f5806c630 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f3f70035000 tx=0x7f3f70008030 comp rx=0 tx=0).ready entity=mgr.6619 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-09T13:52:04.008 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:52:04.006+0000 7f3f6a7fc640 1 -- 172.21.15.43:0/3485484102 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178066 (secure 0 0 0) 0x7f3f64053fe0 con 0x7f3f740c86f0 2023-08-09T13:52:04.123 DEBUG:teuthology.orchestra.run.smithi043:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-08-09T13:52:04.132 DEBUG:teuthology.orchestra.run.smithi203:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-08-09T13:52:04.203 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:52:04.202+0000 7f3f7beb8640 1 -- 172.21.15.43:0/3485484102 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f3f740739f0 con 0x7f3f740c86f0 2023-08-09T13:52:04.205 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:52:04.203+0000 7f3f6a7fc640 1 -- 172.21.15.43:0/3485484102 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v116) v1 ==== 74+0+18506 (secure 0 0 0) 0x7f3f64057600 con 0x7f3f740c86f0 2023-08-09T13:52:04.206 INFO:teuthology.orchestra.run.smithi043.stdout: 2023-08-09T13:52:04.206 INFO:teuthology.orchestra.run.smithi043.stdout:{"epoch":116,"fsid":"daac87ee-1337-4250-a955-aea4e408a39b","created":"2023-08-09T13:33:41.701198+0000","modified":"2023-08-09T13:52:01.206290+0000","last_up_change":"2023-08-09T13:33:53.783453+0000","last_in_change":"2023-08-09T13:33:45.148904+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":15,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":"rbd","create_time":"2023-08-09T13:33:54.365137+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":".mgr","create_time":"2023-08-09T13:33:54.831803+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":14,"pool_name":"cephfs_metadata","create_time":"2023-08-09T13:51:57.637653+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"115","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":2.5,"score_stable":2.5,"optimal_score":1,"raw_score_acting":2.5,"raw_score_stable":2.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":15,"pool_name":"cephfs_data","create_time":"2023-08-09T13:51:58.676117+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"116","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"cbd49ecd-e896-4fbd-864e-5c37a829b165","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":114,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6800","nonce":502667335},{"type":"v1","addr":"172.21.15.43:6801","nonce":502667335}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6802","nonce":502667335},{"type":"v1","addr":"172.21.15.43:6803","nonce":502667335}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6806","nonce":502667335},{"type":"v1","addr":"172.21.15.43:6807","nonce":502667335}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6804","nonce":502667335},{"type":"v1","addr":"172.21.15.43:6805","nonce":502667335}]},"public_addr":"172.21.15.43:6801/502667335","cluster_addr":"172.21.15.43:6803/502667335","heartbeat_back_addr":"172.21.15.43:6807/502667335","heartbeat_front_addr":"172.21.15.43:6805/502667335","state":["exists","up"]},{"osd":1,"uuid":"75c7e0fe-1ec7-4fe8-a2f2-3661a65b8500","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":114,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6808","nonce":3289134610},{"type":"v1","addr":"172.21.15.43:6809","nonce":3289134610}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6810","nonce":3289134610},{"type":"v1","addr":"172.21.15.43:6812","nonce":3289134610}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6821","nonce":3289134610},{"type":"v1","addr":"172.21.15.43:6824","nonce":3289134610}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6815","nonce":3289134610},{"type":"v1","addr":"172.21.15.43:6818","nonce":3289134610}]},"public_addr":"172.21.15.43:6809/3289134610","cluster_addr":"172.21.15.43:6812/3289134610","heartbeat_back_addr":"172.21.15.43:6824/3289134610","heartbeat_front_addr":"172.21.15.43:6818/3289134610","state":["exists","up"]},{"osd":2,"uuid":"bb5c41e9-7f5f-4125-9b01-3bb8bfc2ea83","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":114,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6811","nonce":4079709736},{"type":"v1","addr":"172.21.15.43:6814","nonce":4079709736}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6817","nonce":4079709736},{"type":"v1","addr":"172.21.15.43:6820","nonce":4079709736}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6829","nonce":4079709736},{"type":"v1","addr":"172.21.15.43:6831","nonce":4079709736}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6823","nonce":4079709736},{"type":"v1","addr":"172.21.15.43:6826","nonce":4079709736}]},"public_addr":"172.21.15.43:6814/4079709736","cluster_addr":"172.21.15.43:6820/4079709736","heartbeat_back_addr":"172.21.15.43:6831/4079709736","heartbeat_front_addr":"172.21.15.43:6826/4079709736","state":["exists","up"]},{"osd":3,"uuid":"bfe4c8b4-694a-465c-87e6-2b2aab852613","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":114,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6813","nonce":1709538759},{"type":"v1","addr":"172.21.15.43:6816","nonce":1709538759}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6819","nonce":1709538759},{"type":"v1","addr":"172.21.15.43:6822","nonce":1709538759}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6828","nonce":1709538759},{"type":"v1","addr":"172.21.15.43:6830","nonce":1709538759}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6825","nonce":1709538759},{"type":"v1","addr":"172.21.15.43:6827","nonce":1709538759}]},"public_addr":"172.21.15.43:6816/1709538759","cluster_addr":"172.21.15.43:6822/1709538759","heartbeat_back_addr":"172.21.15.43:6830/1709538759","heartbeat_front_addr":"172.21.15.43:6827/1709538759","state":["exists","up"]},{"osd":4,"uuid":"fa2e8a77-aba1-4756-853e-0d174a5bdeba","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":114,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6800","nonce":4196866481},{"type":"v1","addr":"172.21.15.203:6801","nonce":4196866481}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6802","nonce":4196866481},{"type":"v1","addr":"172.21.15.203:6803","nonce":4196866481}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6806","nonce":4196866481},{"type":"v1","addr":"172.21.15.203:6807","nonce":4196866481}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6804","nonce":4196866481},{"type":"v1","addr":"172.21.15.203:6805","nonce":4196866481}]},"public_addr":"172.21.15.203:6801/4196866481","cluster_addr":"172.21.15.203:6803/4196866481","heartbeat_back_addr":"172.21.15.203:6807/4196866481","heartbeat_front_addr":"172.21.15.203:6805/4196866481","state":["exists","up"]},{"osd":5,"uuid":"81811e4b-4d24-4f51-a82c-f1d700a2a00d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":114,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6816","nonce":3059290081},{"type":"v1","addr":"172.21.15.203:6817","nonce":3059290081}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6818","nonce":3059290081},{"type":"v1","addr":"172.21.15.203:6819","nonce":3059290081}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6822","nonce":3059290081},{"type":"v1","addr":"172.21.15.203:6823","nonce":3059290081}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6820","nonce":3059290081},{"type":"v1","addr":"172.21.15.203:6821","nonce":3059290081}]},"public_addr":"172.21.15.203:6817/3059290081","cluster_addr":"172.21.15.203:6819/3059290081","heartbeat_back_addr":"172.21.15.203:6823/3059290081","heartbeat_front_addr":"172.21.15.203:6821/3059290081","state":["exists","up"]},{"osd":6,"uuid":"307bac4d-6a5b-443d-af75-56d695d7674f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":114,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6824","nonce":512155569},{"type":"v1","addr":"172.21.15.203:6825","nonce":512155569}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6826","nonce":512155569},{"type":"v1","addr":"172.21.15.203:6827","nonce":512155569}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6830","nonce":512155569},{"type":"v1","addr":"172.21.15.203:6831","nonce":512155569}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6828","nonce":512155569},{"type":"v1","addr":"172.21.15.203:6829","nonce":512155569}]},"public_addr":"172.21.15.203:6825/512155569","cluster_addr":"172.21.15.203:6827/512155569","heartbeat_back_addr":"172.21.15.203:6831/512155569","heartbeat_front_addr":"172.21.15.203:6829/512155569","state":["exists","up"]},{"osd":7,"uuid":"a95dc7ca-9f1b-4295-af70-68a22a2d7d5e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":114,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6808","nonce":3331450438},{"type":"v1","addr":"172.21.15.203:6809","nonce":3331450438}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6810","nonce":3331450438},{"type":"v1","addr":"172.21.15.203:6811","nonce":3331450438}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6814","nonce":3331450438},{"type":"v1","addr":"172.21.15.203:6815","nonce":3331450438}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6812","nonce":3331450438},{"type":"v1","addr":"172.21.15.203:6813","nonce":3331450438}]},"public_addr":"172.21.15.203:6809/3331450438","cluster_addr":"172.21.15.203:6811/3331450438","heartbeat_back_addr":"172.21.15.203:6815/3331450438","heartbeat_front_addr":"172.21.15.203:6813/3331450438","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-09T13:33:50.547652+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-09T13:33:50.589469+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-09T13:33:50.561724+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-09T13:33:50.599650+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-09T13:33:50.411855+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-09T13:33:50.219681+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-09T13:33:50.504664+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-09T13:33:50.441248+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":7,"snaps":[{"begin":104,"length":1},{"begin":221,"length":1}]}],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-08-09T13:52:04.208 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:52:04.206+0000 7f3f7beb8640 1 -- 172.21.15.43:0/3485484102 >> [v2:172.21.15.203:6832/1317108840,v1:172.21.15.203:6833/1317108840] conn(0x7f3f5806a170 msgr2=0x7f3f5806c630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:52:04.208 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:52:04.206+0000 7f3f7beb8640 1 --2- 172.21.15.43:0/3485484102 >> [v2:172.21.15.203:6832/1317108840,v1:172.21.15.203:6833/1317108840] conn(0x7f3f5806a170 0x7f3f5806c630 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f3f70035000 tx=0x7f3f70008030 comp rx=0 tx=0).stop 2023-08-09T13:52:04.208 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:52:04.206+0000 7f3f7beb8640 1 -- 172.21.15.43:0/3485484102 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f3f740c86f0 msgr2=0x7f3f740c5b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:52:04.208 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:52:04.206+0000 7f3f7beb8640 1 --2- 172.21.15.43:0/3485484102 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f3f740c86f0 0x7f3f740c5b80 secure :-1 s=READY pgs=1344 cs=0 l=1 rev1=1 crypto rx=0x7f3f6400c7f0 tx=0x7f3f6400ccb0 comp rx=0 tx=0).stop 2023-08-09T13:52:04.208 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:52:04.206+0000 7f3f7beb8640 1 -- 172.21.15.43:0/3485484102 shutdown_connections 2023-08-09T13:52:04.208 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:52:04.206+0000 7f3f7beb8640 1 --2- 172.21.15.43:0/3485484102 >> [v2:172.21.15.203:6832/1317108840,v1:172.21.15.203:6833/1317108840] conn(0x7f3f5806a170 0x7f3f5806c630 unknown :-1 s=CLOSED pgs=64 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:52:04.209 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:52:04.206+0000 7f3f7beb8640 1 --2- 172.21.15.43:0/3485484102 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f3f74143c20 0x7f3f7415cdd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:52:04.209 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:52:04.206+0000 7f3f7beb8640 1 --2- 172.21.15.43:0/3485484102 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f3f740c86f0 0x7f3f740c5b80 unknown :-1 s=CLOSED pgs=1344 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:52:04.209 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:52:04.206+0000 7f3f7beb8640 1 --2- 172.21.15.43:0/3485484102 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f3f7406c8d0 0x7f3f74141d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:52:04.209 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:52:04.206+0000 7f3f7beb8640 1 -- 172.21.15.43:0/3485484102 >> 172.21.15.43:0/3485484102 conn(0x7f3f740ce0a0 msgr2=0x7f3f740c4470 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-09T13:52:04.209 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:52:04.206+0000 7f3f7beb8640 1 -- 172.21.15.43:0/3485484102 shutdown_connections 2023-08-09T13:52:04.209 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:52:04.206+0000 7f3f7beb8640 1 -- 172.21.15.43:0/3485484102 wait complete. 2023-08-09T13:52:04.231 INFO:tasks.cephfs.filesystem.ceph_manager:waiting for clean 2023-08-09T13:52:04.231 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-08-09T13:52:04.446 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:52:04.445+0000 7ff1553d6640 1 Processor -- start 2023-08-09T13:52:04.447 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:52:04.445+0000 7ff1553d6640 1 -- start start 2023-08-09T13:52:04.448 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:52:04.446+0000 7ff1553d6640 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7ff1500b4670 0x7ff1500d2ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:52:04.449 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:52:04.446+0000 7ff1553d6640 1 -- --> v1:172.21.15.43:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff1500ce090 con 0x7ff1500b2b70 2023-08-09T13:52:04.449 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:52:04.446+0000 7ff1553d6640 1 -- --> v1:172.21.15.203:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff1500cd7f0 con 0x7ff1500b2790 2023-08-09T13:52:04.450 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:52:04.446+0000 7ff1553d6640 1 -- --> v2:172.21.15.203:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff1500b3950 con 0x7ff1500b4670 2023-08-09T13:52:04.450 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:52:04.447+0000 7ff14e7fc640 1 --1- >> v1:172.21.15.43:6789/0 conn(0x7ff1500b2b70 0x7ff1500b3f60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.43:6789/0 says I am v1:172.21.15.43:43780/0 (socket says 172.21.15.43:43780) 2023-08-09T13:52:04.450 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:52:04.447+0000 7ff14e7fc640 1 -- 172.21.15.43:0/3633026337 learned_addr learned my addr 172.21.15.43:0/3633026337 (peer_addr_for_me v1:172.21.15.43:0/0) 2023-08-09T13:52:04.455 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:52:04.447+0000 7ff14f7fe640 1 --2- 172.21.15.43:0/3633026337 >> v2:172.21.15.203:3300/0 conn(0x7ff1500b4670 0x7ff1500d2ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:52:04.455 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:52:04.447+0000 7ff14dffb640 1 -- 172.21.15.43:0/3633026337 <== mon.0 v1:172.21.15.43:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2297576038 0 0) 0x7ff1500ce090 con 0x7ff1500b2b70 2023-08-09T13:52:04.455 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:52:04.447+0000 7ff14dffb640 1 -- 172.21.15.43:0/3633026337 --> v1:172.21.15.43:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ff138003520 con 0x7ff1500b2b70 2023-08-09T13:52:04.455 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:52:04.448+0000 7ff14dffb640 1 -- 172.21.15.43:0/3633026337 <== mon.0 v1:172.21.15.43:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2585970030 0 0) 0x7ff138003520 con 0x7ff1500b2b70 2023-08-09T13:52:04.455 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:52:04.448+0000 7ff14dffb640 1 -- 172.21.15.43:0/3633026337 >> v2:172.21.15.203:3300/0 conn(0x7ff1500b4670 msgr2=0x7ff1500d2ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-09T13:52:04.456 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:52:04.448+0000 7ff14dffb640 1 --2- 172.21.15.43:0/3633026337 >> v2:172.21.15.203:3300/0 conn(0x7ff1500b4670 0x7ff1500d2ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:52:04.456 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:52:04.448+0000 7ff14dffb640 1 -- 172.21.15.43:0/3633026337 >> v1:172.21.15.203:6789/0 conn(0x7ff1500b2790 legacy=0x7ff1500b5330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:52:04.456 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:52:04.448+0000 7ff14dffb640 1 -- 172.21.15.43:0/3633026337 --> v1:172.21.15.43:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff1500cdad0 con 0x7ff1500b2b70 2023-08-09T13:52:04.456 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:52:04.448+0000 7ff14dffb640 1 -- 172.21.15.43:0/3633026337 <== mon.0 v1:172.21.15.43:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1285150740 0 0) 0x7ff144001670 con 0x7ff1500b2b70 2023-08-09T13:52:04.456 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:52:04.448+0000 7ff14dffb640 1 -- 172.21.15.43:0/3633026337 >> v1:172.21.15.43:6789/0 conn(0x7ff1500b2b70 legacy=0x7ff1500b3f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:52:04.457 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:52:04.448+0000 7ff14dffb640 1 --2- 172.21.15.43:0/3633026337 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7ff1380039e0 0x7ff138003e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:52:04.457 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:52:04.448+0000 7ff14dffb640 1 --2- 172.21.15.43:0/3633026337 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff1380047c0 0x7ff138004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:52:04.457 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:52:04.448+0000 7ff14dffb640 1 --2- 172.21.15.43:0/3633026337 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7ff138005180 0x7ff13800b980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:52:04.457 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:52:04.448+0000 7ff14dffb640 1 -- 172.21.15.43:0/3633026337 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff1500b3950 con 0x7ff1380047c0 2023-08-09T13:52:04.457 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:52:04.448+0000 7ff14dffb640 1 -- 172.21.15.43:0/3633026337 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff13800c0b0 con 0x7ff138005180 2023-08-09T13:52:04.457 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:52:04.448+0000 7ff14dffb640 1 -- 172.21.15.43:0/3633026337 --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff13800c270 con 0x7ff1380039e0 2023-08-09T13:52:04.457 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:52:04.448+0000 7ff14e7fc640 1 --2- 172.21.15.43:0/3633026337 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff1380047c0 0x7ff138004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:52:04.458 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:52:04.449+0000 7ff14effd640 1 --2- 172.21.15.43:0/3633026337 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7ff1380039e0 0x7ff138003e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:52:04.458 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:52:04.449+0000 7ff14f7fe640 1 --2- 172.21.15.43:0/3633026337 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7ff138005180 0x7ff13800b980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:52:04.458 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:52:04.449+0000 7ff14e7fc640 1 -- 172.21.15.43:0/3633026337 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7ff1380039e0 msgr2=0x7ff138003e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-09T13:52:04.458 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:52:04.449+0000 7ff14e7fc640 1 --2- 172.21.15.43:0/3633026337 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7ff1380039e0 0x7ff138003e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:52:04.458 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:52:04.449+0000 7ff14e7fc640 1 -- 172.21.15.43:0/3633026337 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7ff138005180 msgr2=0x7ff13800b980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-09T13:52:04.458 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:52:04.449+0000 7ff14e7fc640 1 --2- 172.21.15.43:0/3633026337 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7ff138005180 0x7ff13800b980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:52:04.458 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:52:04.449+0000 7ff14e7fc640 1 -- 172.21.15.43:0/3633026337 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff138003520 con 0x7ff1380047c0 2023-08-09T13:52:04.459 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:52:04.449+0000 7ff14f7fe640 1 --2- 172.21.15.43:0/3633026337 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7ff138005180 0x7ff13800b980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-08-09T13:52:04.459 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:52:04.449+0000 7ff14f7fe640 1 -- 172.21.15.43:0/3633026337 reap_dead start 2023-08-09T13:52:04.459 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:52:04.450+0000 7ff14e7fc640 1 --2- 172.21.15.43:0/3633026337 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff1380047c0 0x7ff138004c40 secure :-1 s=READY pgs=1346 cs=0 l=1 rev1=1 crypto rx=0x7ff14400c240 tx=0x7ff1380057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f513c8faf7945714 server_cookie=0 in_seq=0 out_seq=0 2023-08-09T13:52:04.459 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:52:04.450+0000 7ff14dffb640 1 -- 172.21.15.43:0/3633026337 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff144005ad0 con 0x7ff1380047c0 2023-08-09T13:52:04.459 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:52:04.450+0000 7ff14dffb640 1 -- 172.21.15.43:0/3633026337 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff144005c30 con 0x7ff1380047c0 2023-08-09T13:52:04.459 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:52:04.450+0000 7ff14dffb640 1 -- 172.21.15.43:0/3633026337 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff14402ca80 con 0x7ff1380047c0 2023-08-09T13:52:04.459 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:52:04.451+0000 7ff1553d6640 1 -- 172.21.15.43:0/3633026337 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff1380047c0 msgr2=0x7ff138004c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:52:04.460 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:52:04.451+0000 7ff1553d6640 1 --2- 172.21.15.43:0/3633026337 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff1380047c0 0x7ff138004c40 secure :-1 s=READY pgs=1346 cs=0 l=1 rev1=1 crypto rx=0x7ff14400c240 tx=0x7ff1380057f0 comp rx=0 tx=0).stop 2023-08-09T13:52:04.460 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:52:04.451+0000 7ff1553d6640 1 -- 172.21.15.43:0/3633026337 shutdown_connections 2023-08-09T13:52:04.460 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:52:04.451+0000 7ff1553d6640 1 --2- 172.21.15.43:0/3633026337 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff1380047c0 0x7ff138004c40 unknown :-1 s=CLOSED pgs=1346 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:52:04.460 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:52:04.451+0000 7ff1553d6640 1 -- 172.21.15.43:0/3633026337 >> 172.21.15.43:0/3633026337 conn(0x7ff15009b7b0 msgr2=0x7ff1500d24e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-09T13:52:04.460 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:52:04.451+0000 7ff1553d6640 1 -- 172.21.15.43:0/3633026337 shutdown_connections 2023-08-09T13:52:04.460 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:52:04.451+0000 7ff1553d6640 1 -- 172.21.15.43:0/3633026337 wait complete. 2023-08-09T13:52:04.461 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:52:04.453+0000 7ff1553d6640 1 Processor -- start 2023-08-09T13:52:04.461 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:52:04.453+0000 7ff1553d6640 1 -- start start 2023-08-09T13:52:04.463 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:52:04.454+0000 7ff1553d6640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff1380047c0 0x7ff1500c98e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:52:04.463 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:52:04.454+0000 7ff1553d6640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7ff150152980 0x7ff1500cbc50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:52:04.463 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:52:04.454+0000 7ff1553d6640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7ff1500ceda0 0x7ff1500cf220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:52:04.463 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:52:04.454+0000 7ff1553d6640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff1500cf760 con 0x7ff1380047c0 2023-08-09T13:52:04.463 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:52:04.454+0000 7ff1553d6640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff1500cc4a0 con 0x7ff150152980 2023-08-09T13:52:04.464 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:52:04.454+0000 7ff1553d6640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff1500cc780 con 0x7ff1500ceda0 2023-08-09T13:52:04.464 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:52:04.454+0000 7ff14effd640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff1380047c0 0x7ff1500c98e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:52:04.464 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:52:04.454+0000 7ff14effd640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff1380047c0 0x7ff1500c98e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:33182/0 (socket says 172.21.15.43:33182) 2023-08-09T13:52:04.464 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:52:04.455+0000 7ff14effd640 1 -- 172.21.15.43:0/3436548983 learned_addr learned my addr 172.21.15.43:0/3436548983 (peer_addr_for_me v2:172.21.15.43:0/0) 2023-08-09T13:52:04.464 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:52:04.455+0000 7ff14e7fc640 1 --2- 172.21.15.43:0/3436548983 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7ff150152980 0x7ff1500cbc50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:52:04.465 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:52:04.455+0000 7ff14effd640 1 -- 172.21.15.43:0/3436548983 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7ff1500ceda0 msgr2=0x7ff1500cf220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:52:04.465 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:52:04.455+0000 7ff14effd640 1 --2- 172.21.15.43:0/3436548983 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7ff1500ceda0 0x7ff1500cf220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:52:04.465 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:52:04.455+0000 7ff14effd640 1 -- 172.21.15.43:0/3436548983 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7ff150152980 msgr2=0x7ff1500cbc50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:52:04.465 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:52:04.455+0000 7ff14effd640 1 --2- 172.21.15.43:0/3436548983 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7ff150152980 0x7ff1500cbc50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:52:04.465 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:52:04.455+0000 7ff14effd640 1 -- 172.21.15.43:0/3436548983 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff1500cca60 con 0x7ff1380047c0 2023-08-09T13:52:04.465 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:52:04.455+0000 7ff14effd640 1 --2- 172.21.15.43:0/3436548983 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff1380047c0 0x7ff1500c98e0 secure :-1 s=READY pgs=1347 cs=0 l=1 rev1=1 crypto rx=0x7ff14000e840 tx=0x7ff14000ed00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-09T13:52:04.466 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:52:04.456+0000 7ff1377fe640 1 -- 172.21.15.43:0/3436548983 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff140003b80 con 0x7ff1380047c0 2023-08-09T13:52:04.466 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:52:04.456+0000 7ff1377fe640 1 -- 172.21.15.43:0/3436548983 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff140003ce0 con 0x7ff1380047c0 2023-08-09T13:52:04.466 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:52:04.456+0000 7ff1553d6640 1 -- 172.21.15.43:0/3436548983 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff1500c63c0 con 0x7ff1380047c0 2023-08-09T13:52:04.466 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:52:04.456+0000 7ff1377fe640 1 -- 172.21.15.43:0/3436548983 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff140005310 con 0x7ff1380047c0 2023-08-09T13:52:04.466 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:52:04.456+0000 7ff1553d6640 1 -- 172.21.15.43:0/3436548983 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff1500cc190 con 0x7ff1380047c0 2023-08-09T13:52:04.466 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:52:04.457+0000 7ff1553d6640 1 -- 172.21.15.43:0/3436548983 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff1500739f0 con 0x7ff1380047c0 2023-08-09T13:52:04.467 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:52:04.458+0000 7ff1377fe640 1 -- 172.21.15.43:0/3436548983 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 45) v1 ==== 88443+0+0 (secure 0 0 0) 0x7ff140005470 con 0x7ff1380047c0 2023-08-09T13:52:04.467 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:52:04.459+0000 7ff1377fe640 1 --2- 172.21.15.43:0/3436548983 >> [v2:172.21.15.203:6832/1317108840,v1:172.21.15.203:6833/1317108840] conn(0x7ff13c06a1a0 0x7ff13c06c660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:52:04.467 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:52:04.459+0000 7ff1377fe640 1 -- 172.21.15.43:0/3436548983 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(116..116 src has 1..116) v4 ==== 6426+0+0 (secure 0 0 0) 0x7ff1400889b0 con 0x7ff1380047c0 2023-08-09T13:52:04.467 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:52:04.460+0000 7ff14e7fc640 1 --2- 172.21.15.43:0/3436548983 >> [v2:172.21.15.203:6832/1317108840,v1:172.21.15.203:6833/1317108840] conn(0x7ff13c06a1a0 0x7ff13c06c660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:52:04.467 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:52:04.461+0000 7ff14e7fc640 1 --2- 172.21.15.43:0/3436548983 >> [v2:172.21.15.203:6832/1317108840,v1:172.21.15.203:6833/1317108840] conn(0x7ff13c06a1a0 0x7ff13c06c660 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7ff144034000 tx=0x7ff144009030 comp rx=0 tx=0).ready entity=mgr.6619 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-09T13:52:04.468 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:52:04.463+0000 7ff1377fe640 1 -- 172.21.15.43:0/3436548983 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178066 (secure 0 0 0) 0x7ff140053d90 con 0x7ff1380047c0 2023-08-09T13:52:04.657 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:52:04.656+0000 7ff1553d6640 1 -- 172.21.15.43:0/3436548983 --> [v2:172.21.15.203:6832/1317108840,v1:172.21.15.203:6833/1317108840] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7ff1500d0930 con 0x7ff13c06a1a0 2023-08-09T13:52:04.687 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:52:04.685+0000 7ff1377fe640 1 -- 172.21.15.43:0/3436548983 <== mgr.6619 v2:172.21.15.203:6832/1317108840 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+304014 (secure 0 0 0) 0x7ff1440014f0 con 0x7ff13c06a1a0 2023-08-09T13:52:04.689 INFO:teuthology.orchestra.run.smithi043.stdout: 2023-08-09T13:52:04.692 INFO:teuthology.orchestra.run.smithi043.stderr:dumped all 2023-08-09T13:52:04.694 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:52:04.692+0000 7ff1553d6640 1 -- 172.21.15.43:0/3436548983 >> [v2:172.21.15.203:6832/1317108840,v1:172.21.15.203:6833/1317108840] conn(0x7ff13c06a1a0 msgr2=0x7ff13c06c660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:52:04.694 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:52:04.692+0000 7ff1553d6640 1 --2- 172.21.15.43:0/3436548983 >> [v2:172.21.15.203:6832/1317108840,v1:172.21.15.203:6833/1317108840] conn(0x7ff13c06a1a0 0x7ff13c06c660 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7ff144034000 tx=0x7ff144009030 comp rx=0 tx=0).stop 2023-08-09T13:52:04.695 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:52:04.692+0000 7ff1553d6640 1 -- 172.21.15.43:0/3436548983 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff1380047c0 msgr2=0x7ff1500c98e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:52:04.695 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:52:04.692+0000 7ff1553d6640 1 --2- 172.21.15.43:0/3436548983 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff1380047c0 0x7ff1500c98e0 secure :-1 s=READY pgs=1347 cs=0 l=1 rev1=1 crypto rx=0x7ff14000e840 tx=0x7ff14000ed00 comp rx=0 tx=0).stop 2023-08-09T13:52:04.695 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:52:04.692+0000 7ff1553d6640 1 -- 172.21.15.43:0/3436548983 shutdown_connections 2023-08-09T13:52:04.695 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:52:04.692+0000 7ff1553d6640 1 --2- 172.21.15.43:0/3436548983 >> [v2:172.21.15.203:6832/1317108840,v1:172.21.15.203:6833/1317108840] conn(0x7ff13c06a1a0 0x7ff13c06c660 unknown :-1 s=CLOSED pgs=65 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:52:04.696 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:52:04.692+0000 7ff1553d6640 1 --2- 172.21.15.43:0/3436548983 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7ff1500ceda0 0x7ff1500cf220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:52:04.696 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:52:04.692+0000 7ff1553d6640 1 --2- 172.21.15.43:0/3436548983 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7ff150152980 0x7ff1500cbc50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:52:04.696 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:52:04.692+0000 7ff1553d6640 1 --2- 172.21.15.43:0/3436548983 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff1380047c0 0x7ff1500c98e0 unknown :-1 s=CLOSED pgs=1347 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:52:04.697 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:52:04.692+0000 7ff1553d6640 1 -- 172.21.15.43:0/3436548983 >> 172.21.15.43:0/3436548983 conn(0x7ff15009b7b0 msgr2=0x7ff150160e30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-09T13:52:04.697 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:52:04.692+0000 7ff1553d6640 1 -- 172.21.15.43:0/3436548983 shutdown_connections 2023-08-09T13:52:04.697 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:52:04.693+0000 7ff1553d6640 1 -- 172.21.15.43:0/3436548983 wait complete. 2023-08-09T13:52:04.716 INFO:teuthology.orchestra.run.smithi043.stdout:{"pg_ready":true,"pg_map":{"version":27,"stamp":"2023-08-09T13:52:04.571627+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":715,"num_read_kb":1418,"num_write":283,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":186,"ondisk_log_size":186,"up":210,"acting":210,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":23,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":307720,"kb_used_data":12068,"kb_used_omap":13,"kb_used_meta":295538,"kb_avail":754667000,"statfs":{"total":773094113280,"available":772779008000,"internally_reserved":0,"allocated":12357632,"data_stored":7127134,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":14248,"internal_metadata":302631000},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":37,"apply_latency_ms":37,"commit_latency_ns":37000000,"apply_latency_ns":37000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.001865"},"pg_stats":[{"pgid":"15.3f","version":"0'0","reported_seq":1,"reported_epoch":114,"state":"creating+peering","last_fresh":"2023-08-09T13:51:59.173066+0000","last_change":"2023-08-09T13:51:59.173066+0000","last_active":"2023-08-09T13:51:59.130555+0000","last_peered":"2023-08-09T13:51:59.130555+0000","last_clean":"2023-08-09T13:51:59.130555+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-09T13:51:59.173066+0000","last_undegraded":"2023-08-09T13:51:59.173066+0000","last_fullsized":"2023-08-09T13:51:59.173066+0000","mapping_epoch":114,"log_start":"0'0","ondisk_log_start":"0'0","created":114,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:51:59.130555+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:51:59.130555+0000","last_clean_scrub_stamp":"2023-08-09T13:51:59.130555+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"15.3e","version":"0'0","reported_seq":1,"reported_epoch":114,"state":"creating+peering","last_fresh":"2023-08-09T13:51:59.161823+0000","last_change":"2023-08-09T13:51:59.161823+0000","last_active":"2023-08-09T13:51:59.130555+0000","last_peered":"2023-08-09T13:51:59.130555+0000","last_clean":"2023-08-09T13:51:59.130555+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-09T13:51:59.161823+0000","last_undegraded":"2023-08-09T13:51:59.161823+0000","last_fullsized":"2023-08-09T13:51:59.161823+0000","mapping_epoch":114,"log_start":"0'0","ondisk_log_start":"0'0","created":114,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:51:59.130555+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:51:59.130555+0000","last_clean_scrub_stamp":"2023-08-09T13:51:59.130555+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.3d","version":"0'0","reported_seq":1,"reported_epoch":114,"state":"creating+peering","last_fresh":"2023-08-09T13:51:59.169470+0000","last_change":"2023-08-09T13:51:59.169470+0000","last_active":"2023-08-09T13:51:59.130555+0000","last_peered":"2023-08-09T13:51:59.130555+0000","last_clean":"2023-08-09T13:51:59.130555+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-09T13:51:59.169470+0000","last_undegraded":"2023-08-09T13:51:59.169470+0000","last_fullsized":"2023-08-09T13:51:59.169470+0000","mapping_epoch":114,"log_start":"0'0","ondisk_log_start":"0'0","created":114,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:51:59.130555+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:51:59.130555+0000","last_clean_scrub_stamp":"2023-08-09T13:51:59.130555+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"15.3c","version":"0'0","reported_seq":1,"reported_epoch":114,"state":"creating+peering","last_fresh":"2023-08-09T13:51:59.150559+0000","last_change":"2023-08-09T13:51:59.150559+0000","last_active":"2023-08-09T13:51:59.130555+0000","last_peered":"2023-08-09T13:51:59.130555+0000","last_clean":"2023-08-09T13:51:59.130555+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-09T13:51:59.150559+0000","last_undegraded":"2023-08-09T13:51:59.150559+0000","last_fullsized":"2023-08-09T13:51:59.150559+0000","mapping_epoch":114,"log_start":"0'0","ondisk_log_start":"0'0","created":114,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:51:59.130555+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:51:59.130555+0000","last_clean_scrub_stamp":"2023-08-09T13:51:59.130555+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"15.3b","version":"0'0","reported_seq":1,"reported_epoch":114,"state":"creating+peering","last_fresh":"2023-08-09T13:51:59.155619+0000","last_change":"2023-08-09T13:51:59.155619+0000","last_active":"2023-08-09T13:51:59.130555+0000","last_peered":"2023-08-09T13:51:59.130555+0000","last_clean":"2023-08-09T13:51:59.130555+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-09T13:51:59.155619+0000","last_undegraded":"2023-08-09T13:51:59.155619+0000","last_fullsized":"2023-08-09T13:51:59.155619+0000","mapping_epoch":114,"log_start":"0'0","ondisk_log_start":"0'0","created":114,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:51:59.130555+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:51:59.130555+0000","last_clean_scrub_stamp":"2023-08-09T13:51:59.130555+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.3a","version":"0'0","reported_seq":1,"reported_epoch":114,"state":"creating+peering","last_fresh":"2023-08-09T13:51:59.155005+0000","last_change":"2023-08-09T13:51:59.155005+0000","last_active":"2023-08-09T13:51:59.130555+0000","last_peered":"2023-08-09T13:51:59.130555+0000","last_clean":"2023-08-09T13:51:59.130555+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-09T13:51:59.155005+0000","last_undegraded":"2023-08-09T13:51:59.155005+0000","last_fullsized":"2023-08-09T13:51:59.155005+0000","mapping_epoch":114,"log_start":"0'0","ondisk_log_start":"0'0","created":114,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:51:59.130555+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:51:59.130555+0000","last_clean_scrub_stamp":"2023-08-09T13:51:59.130555+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"15.39","version":"0'0","reported_seq":1,"reported_epoch":114,"state":"creating+peering","last_fresh":"2023-08-09T13:51:59.154799+0000","last_change":"2023-08-09T13:51:59.154799+0000","last_active":"2023-08-09T13:51:59.130555+0000","last_peered":"2023-08-09T13:51:59.130555+0000","last_clean":"2023-08-09T13:51:59.130555+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-09T13:51:59.154799+0000","last_undegraded":"2023-08-09T13:51:59.154799+0000","last_fullsized":"2023-08-09T13:51:59.154799+0000","mapping_epoch":114,"log_start":"0'0","ondisk_log_start":"0'0","created":114,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:51:59.130555+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:51:59.130555+0000","last_clean_scrub_stamp":"2023-08-09T13:51:59.130555+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"15.38","version":"0'0","reported_seq":1,"reported_epoch":114,"state":"creating+peering","last_fresh":"2023-08-09T13:51:59.161114+0000","last_change":"2023-08-09T13:51:59.161114+0000","last_active":"2023-08-09T13:51:59.130555+0000","last_peered":"2023-08-09T13:51:59.130555+0000","last_clean":"2023-08-09T13:51:59.130555+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-09T13:51:59.161114+0000","last_undegraded":"2023-08-09T13:51:59.161114+0000","last_fullsized":"2023-08-09T13:51:59.161114+0000","mapping_epoch":114,"log_start":"0'0","ondisk_log_start":"0'0","created":114,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:51:59.130555+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:51:59.130555+0000","last_clean_scrub_stamp":"2023-08-09T13:51:59.130555+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.37","version":"0'0","reported_seq":1,"reported_epoch":114,"state":"creating+peering","last_fresh":"2023-08-09T13:51:59.170489+0000","last_change":"2023-08-09T13:51:59.170489+0000","last_active":"2023-08-09T13:51:59.130555+0000","last_peered":"2023-08-09T13:51:59.130555+0000","last_clean":"2023-08-09T13:51:59.130555+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-09T13:51:59.170489+0000","last_undegraded":"2023-08-09T13:51:59.170489+0000","last_fullsized":"2023-08-09T13:51:59.170489+0000","mapping_epoch":114,"log_start":"0'0","ondisk_log_start":"0'0","created":114,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:51:59.130555+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:51:59.130555+0000","last_clean_scrub_stamp":"2023-08-09T13:51:59.130555+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"15.36","version":"0'0","reported_seq":1,"reported_epoch":114,"state":"creating+peering","last_fresh":"2023-08-09T13:51:59.157362+0000","last_change":"2023-08-09T13:51:59.157362+0000","last_active":"2023-08-09T13:51:59.130555+0000","last_peered":"2023-08-09T13:51:59.130555+0000","last_clean":"2023-08-09T13:51:59.130555+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-09T13:51:59.157362+0000","last_undegraded":"2023-08-09T13:51:59.157362+0000","last_fullsized":"2023-08-09T13:51:59.157362+0000","mapping_epoch":114,"log_start":"0'0","ondisk_log_start":"0'0","created":114,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:51:59.130555+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:51:59.130555+0000","last_clean_scrub_stamp":"2023-08-09T13:51:59.130555+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.35","version":"0'0","reported_seq":1,"reported_epoch":114,"state":"creating+peering","last_fresh":"2023-08-09T13:51:59.158636+0000","last_change":"2023-08-09T13:51:59.158636+0000","last_active":"2023-08-09T13:51:59.130555+0000","last_peered":"2023-08-09T13:51:59.130555+0000","last_clean":"2023-08-09T13:51:59.130555+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-09T13:51:59.158636+0000","last_undegraded":"2023-08-09T13:51:59.158636+0000","last_fullsized":"2023-08-09T13:51:59.158636+0000","mapping_epoch":114,"log_start":"0'0","ondisk_log_start":"0'0","created":114,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:51:59.130555+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:51:59.130555+0000","last_clean_scrub_stamp":"2023-08-09T13:51:59.130555+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"15.34","version":"0'0","reported_seq":1,"reported_epoch":114,"state":"creating+peering","last_fresh":"2023-08-09T13:51:59.175941+0000","last_change":"2023-08-09T13:51:59.175941+0000","last_active":"2023-08-09T13:51:59.130555+0000","last_peered":"2023-08-09T13:51:59.130555+0000","last_clean":"2023-08-09T13:51:59.130555+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-09T13:51:59.175941+0000","last_undegraded":"2023-08-09T13:51:59.175941+0000","last_fullsized":"2023-08-09T13:51:59.175941+0000","mapping_epoch":114,"log_start":"0'0","ondisk_log_start":"0'0","created":114,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:51:59.130555+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:51:59.130555+0000","last_clean_scrub_stamp":"2023-08-09T13:51:59.130555+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"15.33","version":"0'0","reported_seq":1,"reported_epoch":114,"state":"creating+peering","last_fresh":"2023-08-09T13:51:59.150804+0000","last_change":"2023-08-09T13:51:59.150804+0000","last_active":"2023-08-09T13:51:59.130555+0000","last_peered":"2023-08-09T13:51:59.130555+0000","last_clean":"2023-08-09T13:51:59.130555+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-09T13:51:59.150804+0000","last_undegraded":"2023-08-09T13:51:59.150804+0000","last_fullsized":"2023-08-09T13:51:59.150804+0000","mapping_epoch":114,"log_start":"0'0","ondisk_log_start":"0'0","created":114,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:51:59.130555+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:51:59.130555+0000","last_clean_scrub_stamp":"2023-08-09T13:51:59.130555+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"15.32","version":"0'0","reported_seq":1,"reported_epoch":114,"state":"creating+peering","last_fresh":"2023-08-09T13:51:59.165080+0000","last_change":"2023-08-09T13:51:59.165080+0000","last_active":"2023-08-09T13:51:59.130555+0000","last_peered":"2023-08-09T13:51:59.130555+0000","last_clean":"2023-08-09T13:51:59.130555+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-09T13:51:59.165080+0000","last_undegraded":"2023-08-09T13:51:59.165080+0000","last_fullsized":"2023-08-09T13:51:59.165080+0000","mapping_epoch":114,"log_start":"0'0","ondisk_log_start":"0'0","created":114,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:51:59.130555+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:51:59.130555+0000","last_clean_scrub_stamp":"2023-08-09T13:51:59.130555+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.31","version":"0'0","reported_seq":1,"reported_epoch":114,"state":"creating+peering","last_fresh":"2023-08-09T13:51:59.150713+0000","last_change":"2023-08-09T13:51:59.150713+0000","last_active":"2023-08-09T13:51:59.130555+0000","last_peered":"2023-08-09T13:51:59.130555+0000","last_clean":"2023-08-09T13:51:59.130555+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-09T13:51:59.150713+0000","last_undegraded":"2023-08-09T13:51:59.150713+0000","last_fullsized":"2023-08-09T13:51:59.150713+0000","mapping_epoch":114,"log_start":"0'0","ondisk_log_start":"0'0","created":114,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:51:59.130555+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:51:59.130555+0000","last_clean_scrub_stamp":"2023-08-09T13:51:59.130555+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"15.30","version":"0'0","reported_seq":1,"reported_epoch":114,"state":"creating+peering","last_fresh":"2023-08-09T13:51:59.157225+0000","last_change":"2023-08-09T13:51:59.157225+0000","last_active":"2023-08-09T13:51:59.130555+0000","last_peered":"2023-08-09T13:51:59.130555+0000","last_clean":"2023-08-09T13:51:59.130555+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-09T13:51:59.157225+0000","last_undegraded":"2023-08-09T13:51:59.157225+0000","last_fullsized":"2023-08-09T13:51:59.157225+0000","mapping_epoch":114,"log_start":"0'0","ondisk_log_start":"0'0","created":114,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:51:59.130555+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:51:59.130555+0000","last_clean_scrub_stamp":"2023-08-09T13:51:59.130555+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"15.2f","version":"0'0","reported_seq":1,"reported_epoch":114,"state":"creating+peering","last_fresh":"2023-08-09T13:51:59.157118+0000","last_change":"2023-08-09T13:51:59.157118+0000","last_active":"2023-08-09T13:51:59.130555+0000","last_peered":"2023-08-09T13:51:59.130555+0000","last_clean":"2023-08-09T13:51:59.130555+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-09T13:51:59.157118+0000","last_undegraded":"2023-08-09T13:51:59.157118+0000","last_fullsized":"2023-08-09T13:51:59.157118+0000","mapping_epoch":114,"log_start":"0'0","ondisk_log_start":"0'0","created":114,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:51:59.130555+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:51:59.130555+0000","last_clean_scrub_stamp":"2023-08-09T13:51:59.130555+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.2e","version":"0'0","reported_seq":1,"reported_epoch":114,"state":"creating+peering","last_fresh":"2023-08-09T13:51:59.153061+0000","last_change":"2023-08-09T13:51:59.153061+0000","last_active":"2023-08-09T13:51:59.130555+0000","last_peered":"2023-08-09T13:51:59.130555+0000","last_clean":"2023-08-09T13:51:59.130555+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-09T13:51:59.153061+0000","last_undegraded":"2023-08-09T13:51:59.153061+0000","last_fullsized":"2023-08-09T13:51:59.153061+0000","mapping_epoch":114,"log_start":"0'0","ondisk_log_start":"0'0","created":114,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:51:59.130555+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:51:59.130555+0000","last_clean_scrub_stamp":"2023-08-09T13:51:59.130555+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"15.2d","version":"0'0","reported_seq":1,"reported_epoch":114,"state":"creating+peering","last_fresh":"2023-08-09T13:51:59.156327+0000","last_change":"2023-08-09T13:51:59.156327+0000","last_active":"2023-08-09T13:51:59.130555+0000","last_peered":"2023-08-09T13:51:59.130555+0000","last_clean":"2023-08-09T13:51:59.130555+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-09T13:51:59.156327+0000","last_undegraded":"2023-08-09T13:51:59.156327+0000","last_fullsized":"2023-08-09T13:51:59.156327+0000","mapping_epoch":114,"log_start":"0'0","ondisk_log_start":"0'0","created":114,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:51:59.130555+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:51:59.130555+0000","last_clean_scrub_stamp":"2023-08-09T13:51:59.130555+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.2c","version":"0'0","reported_seq":1,"reported_epoch":114,"state":"creating+peering","last_fresh":"2023-08-09T13:51:59.175350+0000","last_change":"2023-08-09T13:51:59.175350+0000","last_active":"2023-08-09T13:51:59.130555+0000","last_peered":"2023-08-09T13:51:59.130555+0000","last_clean":"2023-08-09T13:51:59.130555+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-09T13:51:59.175350+0000","last_undegraded":"2023-08-09T13:51:59.175350+0000","last_fullsized":"2023-08-09T13:51:59.175350+0000","mapping_epoch":114,"log_start":"0'0","ondisk_log_start":"0'0","created":114,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:51:59.130555+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:51:59.130555+0000","last_clean_scrub_stamp":"2023-08-09T13:51:59.130555+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"15.2b","version":"0'0","reported_seq":1,"reported_epoch":114,"state":"creating+peering","last_fresh":"2023-08-09T13:51:59.163315+0000","last_change":"2023-08-09T13:51:59.163315+0000","last_active":"2023-08-09T13:51:59.130555+0000","last_peered":"2023-08-09T13:51:59.130555+0000","last_clean":"2023-08-09T13:51:59.130555+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-09T13:51:59.163315+0000","last_undegraded":"2023-08-09T13:51:59.163315+0000","last_fullsized":"2023-08-09T13:51:59.163315+0000","mapping_epoch":114,"log_start":"0'0","ondisk_log_start":"0'0","created":114,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:51:59.130555+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:51:59.130555+0000","last_clean_scrub_stamp":"2023-08-09T13:51:59.130555+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.2a","version":"0'0","reported_seq":1,"reported_epoch":114,"state":"creating+peering","last_fresh":"2023-08-09T13:51:59.158490+0000","last_change":"2023-08-09T13:51:59.158490+0000","last_active":"2023-08-09T13:51:59.130555+0000","last_peered":"2023-08-09T13:51:59.130555+0000","last_clean":"2023-08-09T13:51:59.130555+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-09T13:51:59.158490+0000","last_undegraded":"2023-08-09T13:51:59.158490+0000","last_fullsized":"2023-08-09T13:51:59.158490+0000","mapping_epoch":114,"log_start":"0'0","ondisk_log_start":"0'0","created":114,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:51:59.130555+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:51:59.130555+0000","last_clean_scrub_stamp":"2023-08-09T13:51:59.130555+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.29","version":"0'0","reported_seq":1,"reported_epoch":114,"state":"creating+peering","last_fresh":"2023-08-09T13:51:59.157791+0000","last_change":"2023-08-09T13:51:59.157791+0000","last_active":"2023-08-09T13:51:59.130555+0000","last_peered":"2023-08-09T13:51:59.130555+0000","last_clean":"2023-08-09T13:51:59.130555+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-09T13:51:59.157791+0000","last_undegraded":"2023-08-09T13:51:59.157791+0000","last_fullsized":"2023-08-09T13:51:59.157791+0000","mapping_epoch":114,"log_start":"0'0","ondisk_log_start":"0'0","created":114,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:51:59.130555+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:51:59.130555+0000","last_clean_scrub_stamp":"2023-08-09T13:51:59.130555+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.28","version":"0'0","reported_seq":1,"reported_epoch":114,"state":"creating+peering","last_fresh":"2023-08-09T13:51:59.156031+0000","last_change":"2023-08-09T13:51:59.156031+0000","last_active":"2023-08-09T13:51:59.130555+0000","last_peered":"2023-08-09T13:51:59.130555+0000","last_clean":"2023-08-09T13:51:59.130555+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-09T13:51:59.156031+0000","last_undegraded":"2023-08-09T13:51:59.156031+0000","last_fullsized":"2023-08-09T13:51:59.156031+0000","mapping_epoch":114,"log_start":"0'0","ondisk_log_start":"0'0","created":114,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:51:59.130555+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:51:59.130555+0000","last_clean_scrub_stamp":"2023-08-09T13:51:59.130555+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"15.27","version":"0'0","reported_seq":1,"reported_epoch":114,"state":"creating+peering","last_fresh":"2023-08-09T13:51:59.157867+0000","last_change":"2023-08-09T13:51:59.157867+0000","last_active":"2023-08-09T13:51:59.130555+0000","last_peered":"2023-08-09T13:51:59.130555+0000","last_clean":"2023-08-09T13:51:59.130555+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-09T13:51:59.157867+0000","last_undegraded":"2023-08-09T13:51:59.157867+0000","last_fullsized":"2023-08-09T13:51:59.157867+0000","mapping_epoch":114,"log_start":"0'0","ondisk_log_start":"0'0","created":114,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:51:59.130555+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:51:59.130555+0000","last_clean_scrub_stamp":"2023-08-09T13:51:59.130555+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.26","version":"0'0","reported_seq":1,"reported_epoch":114,"state":"creating+peering","last_fresh":"2023-08-09T13:51:59.159192+0000","last_change":"2023-08-09T13:51:59.159192+0000","last_active":"2023-08-09T13:51:59.130555+0000","last_peered":"2023-08-09T13:51:59.130555+0000","last_clean":"2023-08-09T13:51:59.130555+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-09T13:51:59.159192+0000","last_undegraded":"2023-08-09T13:51:59.159192+0000","last_fullsized":"2023-08-09T13:51:59.159192+0000","mapping_epoch":114,"log_start":"0'0","ondisk_log_start":"0'0","created":114,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:51:59.130555+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:51:59.130555+0000","last_clean_scrub_stamp":"2023-08-09T13:51:59.130555+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.25","version":"0'0","reported_seq":1,"reported_epoch":114,"state":"creating+peering","last_fresh":"2023-08-09T13:51:59.157367+0000","last_change":"2023-08-09T13:51:59.157367+0000","last_active":"2023-08-09T13:51:59.130555+0000","last_peered":"2023-08-09T13:51:59.130555+0000","last_clean":"2023-08-09T13:51:59.130555+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-09T13:51:59.157367+0000","last_undegraded":"2023-08-09T13:51:59.157367+0000","last_fullsized":"2023-08-09T13:51:59.157367+0000","mapping_epoch":114,"log_start":"0'0","ondisk_log_start":"0'0","created":114,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:51:59.130555+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:51:59.130555+0000","last_clean_scrub_stamp":"2023-08-09T13:51:59.130555+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"15.24","version":"0'0","reported_seq":1,"reported_epoch":114,"state":"creating+peering","last_fresh":"2023-08-09T13:51:59.162536+0000","last_change":"2023-08-09T13:51:59.162536+0000","last_active":"2023-08-09T13:51:59.130555+0000","last_peered":"2023-08-09T13:51:59.130555+0000","last_clean":"2023-08-09T13:51:59.130555+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-09T13:51:59.162536+0000","last_undegraded":"2023-08-09T13:51:59.162536+0000","last_fullsized":"2023-08-09T13:51:59.162536+0000","mapping_epoch":114,"log_start":"0'0","ondisk_log_start":"0'0","created":114,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:51:59.130555+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:51:59.130555+0000","last_clean_scrub_stamp":"2023-08-09T13:51:59.130555+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.23","version":"0'0","reported_seq":1,"reported_epoch":114,"state":"creating+peering","last_fresh":"2023-08-09T13:51:59.160575+0000","last_change":"2023-08-09T13:51:59.160575+0000","last_active":"2023-08-09T13:51:59.130555+0000","last_peered":"2023-08-09T13:51:59.130555+0000","last_clean":"2023-08-09T13:51:59.130555+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-09T13:51:59.160575+0000","last_undegraded":"2023-08-09T13:51:59.160575+0000","last_fullsized":"2023-08-09T13:51:59.160575+0000","mapping_epoch":114,"log_start":"0'0","ondisk_log_start":"0'0","created":114,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:51:59.130555+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:51:59.130555+0000","last_clean_scrub_stamp":"2023-08-09T13:51:59.130555+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.22","version":"0'0","reported_seq":1,"reported_epoch":114,"state":"creating+peering","last_fresh":"2023-08-09T13:51:59.164643+0000","last_change":"2023-08-09T13:51:59.164643+0000","last_active":"2023-08-09T13:51:59.130555+0000","last_peered":"2023-08-09T13:51:59.130555+0000","last_clean":"2023-08-09T13:51:59.130555+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-09T13:51:59.164643+0000","last_undegraded":"2023-08-09T13:51:59.164643+0000","last_fullsized":"2023-08-09T13:51:59.164643+0000","mapping_epoch":114,"log_start":"0'0","ondisk_log_start":"0'0","created":114,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:51:59.130555+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:51:59.130555+0000","last_clean_scrub_stamp":"2023-08-09T13:51:59.130555+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.21","version":"0'0","reported_seq":1,"reported_epoch":114,"state":"creating+peering","last_fresh":"2023-08-09T13:51:59.149533+0000","last_change":"2023-08-09T13:51:59.149533+0000","last_active":"2023-08-09T13:51:59.130555+0000","last_peered":"2023-08-09T13:51:59.130555+0000","last_clean":"2023-08-09T13:51:59.130555+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-09T13:51:59.149533+0000","last_undegraded":"2023-08-09T13:51:59.149533+0000","last_fullsized":"2023-08-09T13:51:59.149533+0000","mapping_epoch":114,"log_start":"0'0","ondisk_log_start":"0'0","created":114,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:51:59.130555+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:51:59.130555+0000","last_clean_scrub_stamp":"2023-08-09T13:51:59.130555+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"15.20","version":"0'0","reported_seq":1,"reported_epoch":114,"state":"creating+peering","last_fresh":"2023-08-09T13:51:59.168339+0000","last_change":"2023-08-09T13:51:59.168339+0000","last_active":"2023-08-09T13:51:59.130555+0000","last_peered":"2023-08-09T13:51:59.130555+0000","last_clean":"2023-08-09T13:51:59.130555+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-09T13:51:59.168339+0000","last_undegraded":"2023-08-09T13:51:59.168339+0000","last_fullsized":"2023-08-09T13:51:59.168339+0000","mapping_epoch":114,"log_start":"0'0","ondisk_log_start":"0'0","created":114,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:51:59.130555+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:51:59.130555+0000","last_clean_scrub_stamp":"2023-08-09T13:51:59.130555+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"15.1e","version":"0'0","reported_seq":1,"reported_epoch":114,"state":"creating+peering","last_fresh":"2023-08-09T13:51:59.162676+0000","last_change":"2023-08-09T13:51:59.162676+0000","last_active":"2023-08-09T13:51:59.130555+0000","last_peered":"2023-08-09T13:51:59.130555+0000","last_clean":"2023-08-09T13:51:59.130555+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-09T13:51:59.162676+0000","last_undegraded":"2023-08-09T13:51:59.162676+0000","last_fullsized":"2023-08-09T13:51:59.162676+0000","mapping_epoch":114,"log_start":"0'0","ondisk_log_start":"0'0","created":114,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:51:59.130555+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:51:59.130555+0000","last_clean_scrub_stamp":"2023-08-09T13:51:59.130555+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"14.1f","version":"0'0","reported_seq":9,"reported_epoch":114,"state":"active+clean","last_fresh":"2023-08-09T13:51:59.180137+0000","last_change":"2023-08-09T13:51:59.176081+0000","last_active":"2023-08-09T13:51:59.180137+0000","last_peered":"2023-08-09T13:51:59.180137+0000","last_clean":"2023-08-09T13:51:59.180137+0000","last_became_active":"2023-08-09T13:51:59.175790+0000","last_became_peered":"2023-08-09T13:51:59.175790+0000","last_unstale":"2023-08-09T13:51:59.180137+0000","last_undegraded":"2023-08-09T13:51:59.180137+0000","last_fullsized":"2023-08-09T13:51:59.180137+0000","mapping_epoch":113,"log_start":"0'0","ondisk_log_start":"0'0","created":113,"last_epoch_clean":114,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:51:58.125661+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:51:58.125661+0000","last_clean_scrub_stamp":"2023-08-09T13:51:58.125661+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T17:58:19.394820+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.1f","version":"0'0","reported_seq":1,"reported_epoch":114,"state":"creating+peering","last_fresh":"2023-08-09T13:51:59.151940+0000","last_change":"2023-08-09T13:51:59.151940+0000","last_active":"2023-08-09T13:51:59.130555+0000","last_peered":"2023-08-09T13:51:59.130555+0000","last_clean":"2023-08-09T13:51:59.130555+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-09T13:51:59.151940+0000","last_undegraded":"2023-08-09T13:51:59.151940+0000","last_fullsized":"2023-08-09T13:51:59.151940+0000","mapping_epoch":114,"log_start":"0'0","ondisk_log_start":"0'0","created":114,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:51:59.130555+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:51:59.130555+0000","last_clean_scrub_stamp":"2023-08-09T13:51:59.130555+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"14.1e","version":"0'0","reported_seq":9,"reported_epoch":114,"state":"active+clean","last_fresh":"2023-08-09T13:51:59.180102+0000","last_change":"2023-08-09T13:51:59.178857+0000","last_active":"2023-08-09T13:51:59.180102+0000","last_peered":"2023-08-09T13:51:59.180102+0000","last_clean":"2023-08-09T13:51:59.180102+0000","last_became_active":"2023-08-09T13:51:59.178483+0000","last_became_peered":"2023-08-09T13:51:59.178483+0000","last_unstale":"2023-08-09T13:51:59.180102+0000","last_undegraded":"2023-08-09T13:51:59.180102+0000","last_fullsized":"2023-08-09T13:51:59.180102+0000","mapping_epoch":113,"log_start":"0'0","ondisk_log_start":"0'0","created":113,"last_epoch_clean":114,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:51:58.125661+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:51:58.125661+0000","last_clean_scrub_stamp":"2023-08-09T13:51:58.125661+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T15:40:02.742070+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"15.1c","version":"0'0","reported_seq":1,"reported_epoch":114,"state":"creating+peering","last_fresh":"2023-08-09T13:51:59.166472+0000","last_change":"2023-08-09T13:51:59.166472+0000","last_active":"2023-08-09T13:51:59.130555+0000","last_peered":"2023-08-09T13:51:59.130555+0000","last_clean":"2023-08-09T13:51:59.130555+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-09T13:51:59.166472+0000","last_undegraded":"2023-08-09T13:51:59.166472+0000","last_fullsized":"2023-08-09T13:51:59.166472+0000","mapping_epoch":114,"log_start":"0'0","ondisk_log_start":"0'0","created":114,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:51:59.130555+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:51:59.130555+0000","last_clean_scrub_stamp":"2023-08-09T13:51:59.130555+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.1d","version":"0'0","reported_seq":9,"reported_epoch":114,"state":"active+clean","last_fresh":"2023-08-09T13:51:59.173649+0000","last_change":"2023-08-09T13:51:59.168942+0000","last_active":"2023-08-09T13:51:59.173649+0000","last_peered":"2023-08-09T13:51:59.173649+0000","last_clean":"2023-08-09T13:51:59.173649+0000","last_became_active":"2023-08-09T13:51:59.168599+0000","last_became_peered":"2023-08-09T13:51:59.168599+0000","last_unstale":"2023-08-09T13:51:59.173649+0000","last_undegraded":"2023-08-09T13:51:59.173649+0000","last_fullsized":"2023-08-09T13:51:59.173649+0000","mapping_epoch":113,"log_start":"0'0","ondisk_log_start":"0'0","created":113,"last_epoch_clean":114,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:51:58.125661+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:51:58.125661+0000","last_clean_scrub_stamp":"2023-08-09T13:51:58.125661+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T21:57:39.771411+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.1d","version":"0'0","reported_seq":1,"reported_epoch":114,"state":"creating+peering","last_fresh":"2023-08-09T13:51:59.170197+0000","last_change":"2023-08-09T13:51:59.170197+0000","last_active":"2023-08-09T13:51:59.130555+0000","last_peered":"2023-08-09T13:51:59.130555+0000","last_clean":"2023-08-09T13:51:59.130555+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-09T13:51:59.170197+0000","last_undegraded":"2023-08-09T13:51:59.170197+0000","last_fullsized":"2023-08-09T13:51:59.170197+0000","mapping_epoch":114,"log_start":"0'0","ondisk_log_start":"0'0","created":114,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:51:59.130555+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:51:59.130555+0000","last_clean_scrub_stamp":"2023-08-09T13:51:59.130555+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.1c","version":"0'0","reported_seq":9,"reported_epoch":114,"state":"active+clean","last_fresh":"2023-08-09T13:51:59.184792+0000","last_change":"2023-08-09T13:51:59.183710+0000","last_active":"2023-08-09T13:51:59.184792+0000","last_peered":"2023-08-09T13:51:59.184792+0000","last_clean":"2023-08-09T13:51:59.184792+0000","last_became_active":"2023-08-09T13:51:59.183401+0000","last_became_peered":"2023-08-09T13:51:59.183401+0000","last_unstale":"2023-08-09T13:51:59.184792+0000","last_undegraded":"2023-08-09T13:51:59.184792+0000","last_fullsized":"2023-08-09T13:51:59.184792+0000","mapping_epoch":113,"log_start":"0'0","ondisk_log_start":"0'0","created":113,"last_epoch_clean":114,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:51:58.125661+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:51:58.125661+0000","last_clean_scrub_stamp":"2023-08-09T13:51:58.125661+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T13:55:08.336317+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.1a","version":"0'0","reported_seq":1,"reported_epoch":114,"state":"creating+peering","last_fresh":"2023-08-09T13:51:59.165426+0000","last_change":"2023-08-09T13:51:59.165426+0000","last_active":"2023-08-09T13:51:59.130555+0000","last_peered":"2023-08-09T13:51:59.130555+0000","last_clean":"2023-08-09T13:51:59.130555+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-09T13:51:59.165426+0000","last_undegraded":"2023-08-09T13:51:59.165426+0000","last_fullsized":"2023-08-09T13:51:59.165426+0000","mapping_epoch":114,"log_start":"0'0","ondisk_log_start":"0'0","created":114,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:51:59.130555+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:51:59.130555+0000","last_clean_scrub_stamp":"2023-08-09T13:51:59.130555+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"14.1b","version":"0'0","reported_seq":9,"reported_epoch":114,"state":"active+clean","last_fresh":"2023-08-09T13:51:59.180443+0000","last_change":"2023-08-09T13:51:59.178308+0000","last_active":"2023-08-09T13:51:59.180443+0000","last_peered":"2023-08-09T13:51:59.180443+0000","last_clean":"2023-08-09T13:51:59.180443+0000","last_became_active":"2023-08-09T13:51:59.178022+0000","last_became_peered":"2023-08-09T13:51:59.178022+0000","last_unstale":"2023-08-09T13:51:59.180443+0000","last_undegraded":"2023-08-09T13:51:59.180443+0000","last_fullsized":"2023-08-09T13:51:59.180443+0000","mapping_epoch":113,"log_start":"0'0","ondisk_log_start":"0'0","created":113,"last_epoch_clean":114,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:51:58.125661+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:51:58.125661+0000","last_clean_scrub_stamp":"2023-08-09T13:51:58.125661+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T15:51:56.504946+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"15.1b","version":"0'0","reported_seq":1,"reported_epoch":114,"state":"creating+peering","last_fresh":"2023-08-09T13:51:59.157444+0000","last_change":"2023-08-09T13:51:59.157444+0000","last_active":"2023-08-09T13:51:59.130555+0000","last_peered":"2023-08-09T13:51:59.130555+0000","last_clean":"2023-08-09T13:51:59.130555+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-09T13:51:59.157444+0000","last_undegraded":"2023-08-09T13:51:59.157444+0000","last_fullsized":"2023-08-09T13:51:59.157444+0000","mapping_epoch":114,"log_start":"0'0","ondisk_log_start":"0'0","created":114,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:51:59.130555+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:51:59.130555+0000","last_clean_scrub_stamp":"2023-08-09T13:51:59.130555+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"14.1a","version":"0'0","reported_seq":9,"reported_epoch":114,"state":"active+clean","last_fresh":"2023-08-09T13:51:59.173191+0000","last_change":"2023-08-09T13:51:59.165522+0000","last_active":"2023-08-09T13:51:59.173191+0000","last_peered":"2023-08-09T13:51:59.173191+0000","last_clean":"2023-08-09T13:51:59.173191+0000","last_became_active":"2023-08-09T13:51:59.165145+0000","last_became_peered":"2023-08-09T13:51:59.165145+0000","last_unstale":"2023-08-09T13:51:59.173191+0000","last_undegraded":"2023-08-09T13:51:59.173191+0000","last_fullsized":"2023-08-09T13:51:59.173191+0000","mapping_epoch":113,"log_start":"0'0","ondisk_log_start":"0'0","created":113,"last_epoch_clean":114,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:51:58.125661+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:51:58.125661+0000","last_clean_scrub_stamp":"2023-08-09T13:51:58.125661+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T16:19:11.497849+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.18","version":"0'0","reported_seq":1,"reported_epoch":114,"state":"creating+peering","last_fresh":"2023-08-09T13:51:59.149114+0000","last_change":"2023-08-09T13:51:59.149114+0000","last_active":"2023-08-09T13:51:59.130555+0000","last_peered":"2023-08-09T13:51:59.130555+0000","last_clean":"2023-08-09T13:51:59.130555+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-09T13:51:59.149114+0000","last_undegraded":"2023-08-09T13:51:59.149114+0000","last_fullsized":"2023-08-09T13:51:59.149114+0000","mapping_epoch":114,"log_start":"0'0","ondisk_log_start":"0'0","created":114,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:51:59.130555+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:51:59.130555+0000","last_clean_scrub_stamp":"2023-08-09T13:51:59.130555+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"14.19","version":"0'0","reported_seq":9,"reported_epoch":114,"state":"active+clean","last_fresh":"2023-08-09T13:51:59.178533+0000","last_change":"2023-08-09T13:51:59.174362+0000","last_active":"2023-08-09T13:51:59.178533+0000","last_peered":"2023-08-09T13:51:59.178533+0000","last_clean":"2023-08-09T13:51:59.178533+0000","last_became_active":"2023-08-09T13:51:59.174002+0000","last_became_peered":"2023-08-09T13:51:59.174002+0000","last_unstale":"2023-08-09T13:51:59.178533+0000","last_undegraded":"2023-08-09T13:51:59.178533+0000","last_fullsized":"2023-08-09T13:51:59.178533+0000","mapping_epoch":113,"log_start":"0'0","ondisk_log_start":"0'0","created":113,"last_epoch_clean":114,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:51:58.125661+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:51:58.125661+0000","last_clean_scrub_stamp":"2023-08-09T13:51:58.125661+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T19:08:56.894026+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.19","version":"0'0","reported_seq":1,"reported_epoch":114,"state":"creating+peering","last_fresh":"2023-08-09T13:51:59.168819+0000","last_change":"2023-08-09T13:51:59.168819+0000","last_active":"2023-08-09T13:51:59.130555+0000","last_peered":"2023-08-09T13:51:59.130555+0000","last_clean":"2023-08-09T13:51:59.130555+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-09T13:51:59.168819+0000","last_undegraded":"2023-08-09T13:51:59.168819+0000","last_fullsized":"2023-08-09T13:51:59.168819+0000","mapping_epoch":114,"log_start":"0'0","ondisk_log_start":"0'0","created":114,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:51:59.130555+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:51:59.130555+0000","last_clean_scrub_stamp":"2023-08-09T13:51:59.130555+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.18","version":"0'0","reported_seq":9,"reported_epoch":114,"state":"active+clean","last_fresh":"2023-08-09T13:51:59.180392+0000","last_change":"2023-08-09T13:51:59.177991+0000","last_active":"2023-08-09T13:51:59.180392+0000","last_peered":"2023-08-09T13:51:59.180392+0000","last_clean":"2023-08-09T13:51:59.180392+0000","last_became_active":"2023-08-09T13:51:59.177651+0000","last_became_peered":"2023-08-09T13:51:59.177651+0000","last_unstale":"2023-08-09T13:51:59.180392+0000","last_undegraded":"2023-08-09T13:51:59.180392+0000","last_fullsized":"2023-08-09T13:51:59.180392+0000","mapping_epoch":113,"log_start":"0'0","ondisk_log_start":"0'0","created":113,"last_epoch_clean":114,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:51:58.125661+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:51:58.125661+0000","last_clean_scrub_stamp":"2023-08-09T13:51:58.125661+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T18:13:34.464631+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"15.16","version":"0'0","reported_seq":1,"reported_epoch":114,"state":"creating+peering","last_fresh":"2023-08-09T13:51:59.158359+0000","last_change":"2023-08-09T13:51:59.158359+0000","last_active":"2023-08-09T13:51:59.130555+0000","last_peered":"2023-08-09T13:51:59.130555+0000","last_clean":"2023-08-09T13:51:59.130555+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-09T13:51:59.158359+0000","last_undegraded":"2023-08-09T13:51:59.158359+0000","last_fullsized":"2023-08-09T13:51:59.158359+0000","mapping_epoch":114,"log_start":"0'0","ondisk_log_start":"0'0","created":114,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:51:59.130555+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:51:59.130555+0000","last_clean_scrub_stamp":"2023-08-09T13:51:59.130555+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"14.17","version":"0'0","reported_seq":9,"reported_epoch":114,"state":"active+clean","last_fresh":"2023-08-09T13:51:59.182321+0000","last_change":"2023-08-09T13:51:59.181245+0000","last_active":"2023-08-09T13:51:59.182321+0000","last_peered":"2023-08-09T13:51:59.182321+0000","last_clean":"2023-08-09T13:51:59.182321+0000","last_became_active":"2023-08-09T13:51:59.180911+0000","last_became_peered":"2023-08-09T13:51:59.180911+0000","last_unstale":"2023-08-09T13:51:59.182321+0000","last_undegraded":"2023-08-09T13:51:59.182321+0000","last_fullsized":"2023-08-09T13:51:59.182321+0000","mapping_epoch":113,"log_start":"0'0","ondisk_log_start":"0'0","created":113,"last_epoch_clean":114,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:51:58.125661+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:51:58.125661+0000","last_clean_scrub_stamp":"2023-08-09T13:51:58.125661+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T16:30:25.226241+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"15.17","version":"0'0","reported_seq":1,"reported_epoch":114,"state":"creating+peering","last_fresh":"2023-08-09T13:51:59.169791+0000","last_change":"2023-08-09T13:51:59.169791+0000","last_active":"2023-08-09T13:51:59.130555+0000","last_peered":"2023-08-09T13:51:59.130555+0000","last_clean":"2023-08-09T13:51:59.130555+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-09T13:51:59.169791+0000","last_undegraded":"2023-08-09T13:51:59.169791+0000","last_fullsized":"2023-08-09T13:51:59.169791+0000","mapping_epoch":114,"log_start":"0'0","ondisk_log_start":"0'0","created":114,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:51:59.130555+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:51:59.130555+0000","last_clean_scrub_stamp":"2023-08-09T13:51:59.130555+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.16","version":"0'0","reported_seq":9,"reported_epoch":114,"state":"active+clean","last_fresh":"2023-08-09T13:51:59.176606+0000","last_change":"2023-08-09T13:51:59.172670+0000","last_active":"2023-08-09T13:51:59.176606+0000","last_peered":"2023-08-09T13:51:59.176606+0000","last_clean":"2023-08-09T13:51:59.176606+0000","last_became_active":"2023-08-09T13:51:59.172395+0000","last_became_peered":"2023-08-09T13:51:59.172395+0000","last_unstale":"2023-08-09T13:51:59.176606+0000","last_undegraded":"2023-08-09T13:51:59.176606+0000","last_fullsized":"2023-08-09T13:51:59.176606+0000","mapping_epoch":113,"log_start":"0'0","ondisk_log_start":"0'0","created":113,"last_epoch_clean":114,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:51:58.125661+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:51:58.125661+0000","last_clean_scrub_stamp":"2023-08-09T13:51:58.125661+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T19:55:12.433811+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"15.14","version":"0'0","reported_seq":1,"reported_epoch":114,"state":"creating+peering","last_fresh":"2023-08-09T13:51:59.161549+0000","last_change":"2023-08-09T13:51:59.161549+0000","last_active":"2023-08-09T13:51:59.130555+0000","last_peered":"2023-08-09T13:51:59.130555+0000","last_clean":"2023-08-09T13:51:59.130555+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-09T13:51:59.161549+0000","last_undegraded":"2023-08-09T13:51:59.161549+0000","last_fullsized":"2023-08-09T13:51:59.161549+0000","mapping_epoch":114,"log_start":"0'0","ondisk_log_start":"0'0","created":114,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:51:59.130555+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:51:59.130555+0000","last_clean_scrub_stamp":"2023-08-09T13:51:59.130555+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"14.15","version":"0'0","reported_seq":9,"reported_epoch":114,"state":"active+clean","last_fresh":"2023-08-09T13:51:59.184646+0000","last_change":"2023-08-09T13:51:59.181964+0000","last_active":"2023-08-09T13:51:59.184646+0000","last_peered":"2023-08-09T13:51:59.184646+0000","last_clean":"2023-08-09T13:51:59.184646+0000","last_became_active":"2023-08-09T13:51:59.181630+0000","last_became_peered":"2023-08-09T13:51:59.181630+0000","last_unstale":"2023-08-09T13:51:59.184646+0000","last_undegraded":"2023-08-09T13:51:59.184646+0000","last_fullsized":"2023-08-09T13:51:59.184646+0000","mapping_epoch":113,"log_start":"0'0","ondisk_log_start":"0'0","created":113,"last_epoch_clean":114,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:51:58.125661+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:51:58.125661+0000","last_clean_scrub_stamp":"2023-08-09T13:51:58.125661+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T16:43:18.755379+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"15.15","version":"0'0","reported_seq":1,"reported_epoch":114,"state":"creating+peering","last_fresh":"2023-08-09T13:51:59.156075+0000","last_change":"2023-08-09T13:51:59.156075+0000","last_active":"2023-08-09T13:51:59.130555+0000","last_peered":"2023-08-09T13:51:59.130555+0000","last_clean":"2023-08-09T13:51:59.130555+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-09T13:51:59.156075+0000","last_undegraded":"2023-08-09T13:51:59.156075+0000","last_fullsized":"2023-08-09T13:51:59.156075+0000","mapping_epoch":114,"log_start":"0'0","ondisk_log_start":"0'0","created":114,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:51:59.130555+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:51:59.130555+0000","last_clean_scrub_stamp":"2023-08-09T13:51:59.130555+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"14.14","version":"0'0","reported_seq":9,"reported_epoch":114,"state":"active+clean","last_fresh":"2023-08-09T13:51:59.184692+0000","last_change":"2023-08-09T13:51:59.181967+0000","last_active":"2023-08-09T13:51:59.184692+0000","last_peered":"2023-08-09T13:51:59.184692+0000","last_clean":"2023-08-09T13:51:59.184692+0000","last_became_active":"2023-08-09T13:51:59.180664+0000","last_became_peered":"2023-08-09T13:51:59.180664+0000","last_unstale":"2023-08-09T13:51:59.184692+0000","last_undegraded":"2023-08-09T13:51:59.184692+0000","last_fullsized":"2023-08-09T13:51:59.184692+0000","mapping_epoch":113,"log_start":"0'0","ondisk_log_start":"0'0","created":113,"last_epoch_clean":114,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:51:58.125661+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:51:58.125661+0000","last_clean_scrub_stamp":"2023-08-09T13:51:58.125661+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-11T01:49:03.976153+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"15.12","version":"0'0","reported_seq":1,"reported_epoch":114,"state":"creating+peering","last_fresh":"2023-08-09T13:51:59.150320+0000","last_change":"2023-08-09T13:51:59.150320+0000","last_active":"2023-08-09T13:51:59.130555+0000","last_peered":"2023-08-09T13:51:59.130555+0000","last_clean":"2023-08-09T13:51:59.130555+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-09T13:51:59.150320+0000","last_undegraded":"2023-08-09T13:51:59.150320+0000","last_fullsized":"2023-08-09T13:51:59.150320+0000","mapping_epoch":114,"log_start":"0'0","ondisk_log_start":"0'0","created":114,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:51:59.130555+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:51:59.130555+0000","last_clean_scrub_stamp":"2023-08-09T13:51:59.130555+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"14.13","version":"0'0","reported_seq":9,"reported_epoch":114,"state":"active+clean","last_fresh":"2023-08-09T13:51:59.183063+0000","last_change":"2023-08-09T13:51:59.181856+0000","last_active":"2023-08-09T13:51:59.183063+0000","last_peered":"2023-08-09T13:51:59.183063+0000","last_clean":"2023-08-09T13:51:59.183063+0000","last_became_active":"2023-08-09T13:51:59.181568+0000","last_became_peered":"2023-08-09T13:51:59.181568+0000","last_unstale":"2023-08-09T13:51:59.183063+0000","last_undegraded":"2023-08-09T13:51:59.183063+0000","last_fullsized":"2023-08-09T13:51:59.183063+0000","mapping_epoch":113,"log_start":"0'0","ondisk_log_start":"0'0","created":113,"last_epoch_clean":114,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:51:58.125661+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:51:58.125661+0000","last_clean_scrub_stamp":"2023-08-09T13:51:58.125661+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T22:11:33.508209+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.13","version":"0'0","reported_seq":1,"reported_epoch":114,"state":"creating+peering","last_fresh":"2023-08-09T13:51:59.163775+0000","last_change":"2023-08-09T13:51:59.163775+0000","last_active":"2023-08-09T13:51:59.130555+0000","last_peered":"2023-08-09T13:51:59.130555+0000","last_clean":"2023-08-09T13:51:59.130555+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-09T13:51:59.163775+0000","last_undegraded":"2023-08-09T13:51:59.163775+0000","last_fullsized":"2023-08-09T13:51:59.163775+0000","mapping_epoch":114,"log_start":"0'0","ondisk_log_start":"0'0","created":114,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:51:59.130555+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:51:59.130555+0000","last_clean_scrub_stamp":"2023-08-09T13:51:59.130555+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.12","version":"0'0","reported_seq":9,"reported_epoch":114,"state":"active+clean","last_fresh":"2023-08-09T13:51:59.180137+0000","last_change":"2023-08-09T13:51:59.178710+0000","last_active":"2023-08-09T13:51:59.180137+0000","last_peered":"2023-08-09T13:51:59.180137+0000","last_clean":"2023-08-09T13:51:59.180137+0000","last_became_active":"2023-08-09T13:51:59.178402+0000","last_became_peered":"2023-08-09T13:51:59.178402+0000","last_unstale":"2023-08-09T13:51:59.180137+0000","last_undegraded":"2023-08-09T13:51:59.180137+0000","last_fullsized":"2023-08-09T13:51:59.180137+0000","mapping_epoch":113,"log_start":"0'0","ondisk_log_start":"0'0","created":113,"last_epoch_clean":114,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:51:58.125661+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:51:58.125661+0000","last_clean_scrub_stamp":"2023-08-09T13:51:58.125661+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T19:35:28.808687+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.10","version":"0'0","reported_seq":1,"reported_epoch":114,"state":"creating+peering","last_fresh":"2023-08-09T13:51:59.167863+0000","last_change":"2023-08-09T13:51:59.167863+0000","last_active":"2023-08-09T13:51:59.130555+0000","last_peered":"2023-08-09T13:51:59.130555+0000","last_clean":"2023-08-09T13:51:59.130555+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-09T13:51:59.167863+0000","last_undegraded":"2023-08-09T13:51:59.167863+0000","last_fullsized":"2023-08-09T13:51:59.167863+0000","mapping_epoch":114,"log_start":"0'0","ondisk_log_start":"0'0","created":114,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:51:59.130555+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:51:59.130555+0000","last_clean_scrub_stamp":"2023-08-09T13:51:59.130555+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.11","version":"0'0","reported_seq":9,"reported_epoch":114,"state":"active+clean","last_fresh":"2023-08-09T13:51:59.184619+0000","last_change":"2023-08-09T13:51:59.181121+0000","last_active":"2023-08-09T13:51:59.184619+0000","last_peered":"2023-08-09T13:51:59.184619+0000","last_clean":"2023-08-09T13:51:59.184619+0000","last_became_active":"2023-08-09T13:51:59.180844+0000","last_became_peered":"2023-08-09T13:51:59.180844+0000","last_unstale":"2023-08-09T13:51:59.184619+0000","last_undegraded":"2023-08-09T13:51:59.184619+0000","last_fullsized":"2023-08-09T13:51:59.184619+0000","mapping_epoch":113,"log_start":"0'0","ondisk_log_start":"0'0","created":113,"last_epoch_clean":114,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:51:58.125661+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:51:58.125661+0000","last_clean_scrub_stamp":"2023-08-09T13:51:58.125661+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T14:42:49.973080+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"15.11","version":"0'0","reported_seq":1,"reported_epoch":114,"state":"creating+peering","last_fresh":"2023-08-09T13:51:59.151102+0000","last_change":"2023-08-09T13:51:59.151102+0000","last_active":"2023-08-09T13:51:59.130555+0000","last_peered":"2023-08-09T13:51:59.130555+0000","last_clean":"2023-08-09T13:51:59.130555+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-09T13:51:59.151102+0000","last_undegraded":"2023-08-09T13:51:59.151102+0000","last_fullsized":"2023-08-09T13:51:59.151102+0000","mapping_epoch":114,"log_start":"0'0","ondisk_log_start":"0'0","created":114,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:51:59.130555+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:51:59.130555+0000","last_clean_scrub_stamp":"2023-08-09T13:51:59.130555+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"14.10","version":"0'0","reported_seq":9,"reported_epoch":114,"state":"active+clean","last_fresh":"2023-08-09T13:51:59.178095+0000","last_change":"2023-08-09T13:51:59.174790+0000","last_active":"2023-08-09T13:51:59.178095+0000","last_peered":"2023-08-09T13:51:59.178095+0000","last_clean":"2023-08-09T13:51:59.178095+0000","last_became_active":"2023-08-09T13:51:59.174453+0000","last_became_peered":"2023-08-09T13:51:59.174453+0000","last_unstale":"2023-08-09T13:51:59.178095+0000","last_undegraded":"2023-08-09T13:51:59.178095+0000","last_fullsized":"2023-08-09T13:51:59.178095+0000","mapping_epoch":113,"log_start":"0'0","ondisk_log_start":"0'0","created":113,"last_epoch_clean":114,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:51:58.125661+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:51:58.125661+0000","last_clean_scrub_stamp":"2023-08-09T13:51:58.125661+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T19:07:38.533940+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"15.6","version":"0'0","reported_seq":1,"reported_epoch":114,"state":"creating+peering","last_fresh":"2023-08-09T13:51:59.157299+0000","last_change":"2023-08-09T13:51:59.157299+0000","last_active":"2023-08-09T13:51:59.130555+0000","last_peered":"2023-08-09T13:51:59.130555+0000","last_clean":"2023-08-09T13:51:59.130555+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-09T13:51:59.157299+0000","last_undegraded":"2023-08-09T13:51:59.157299+0000","last_fullsized":"2023-08-09T13:51:59.157299+0000","mapping_epoch":114,"log_start":"0'0","ondisk_log_start":"0'0","created":114,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:51:59.130555+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:51:59.130555+0000","last_clean_scrub_stamp":"2023-08-09T13:51:59.130555+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"14.7","version":"0'0","reported_seq":9,"reported_epoch":114,"state":"active+clean","last_fresh":"2023-08-09T13:51:59.187317+0000","last_change":"2023-08-09T13:51:59.182730+0000","last_active":"2023-08-09T13:51:59.187317+0000","last_peered":"2023-08-09T13:51:59.187317+0000","last_clean":"2023-08-09T13:51:59.187317+0000","last_became_active":"2023-08-09T13:51:59.182469+0000","last_became_peered":"2023-08-09T13:51:59.182469+0000","last_unstale":"2023-08-09T13:51:59.187317+0000","last_undegraded":"2023-08-09T13:51:59.187317+0000","last_fullsized":"2023-08-09T13:51:59.187317+0000","mapping_epoch":113,"log_start":"0'0","ondisk_log_start":"0'0","created":113,"last_epoch_clean":114,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:51:58.125661+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:51:58.125661+0000","last_clean_scrub_stamp":"2023-08-09T13:51:58.125661+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T23:49:36.976561+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"15.7","version":"0'0","reported_seq":1,"reported_epoch":114,"state":"creating+peering","last_fresh":"2023-08-09T13:51:59.162752+0000","last_change":"2023-08-09T13:51:59.162752+0000","last_active":"2023-08-09T13:51:59.130555+0000","last_peered":"2023-08-09T13:51:59.130555+0000","last_clean":"2023-08-09T13:51:59.130555+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-09T13:51:59.162752+0000","last_undegraded":"2023-08-09T13:51:59.162752+0000","last_fullsized":"2023-08-09T13:51:59.162752+0000","mapping_epoch":114,"log_start":"0'0","ondisk_log_start":"0'0","created":114,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:51:59.130555+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:51:59.130555+0000","last_clean_scrub_stamp":"2023-08-09T13:51:59.130555+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.6","version":"0'0","reported_seq":9,"reported_epoch":114,"state":"active+clean","last_fresh":"2023-08-09T13:51:59.173431+0000","last_change":"2023-08-09T13:51:59.168280+0000","last_active":"2023-08-09T13:51:59.173431+0000","last_peered":"2023-08-09T13:51:59.173431+0000","last_clean":"2023-08-09T13:51:59.173431+0000","last_became_active":"2023-08-09T13:51:59.167980+0000","last_became_peered":"2023-08-09T13:51:59.167980+0000","last_unstale":"2023-08-09T13:51:59.173431+0000","last_undegraded":"2023-08-09T13:51:59.173431+0000","last_fullsized":"2023-08-09T13:51:59.173431+0000","mapping_epoch":113,"log_start":"0'0","ondisk_log_start":"0'0","created":113,"last_epoch_clean":114,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:51:58.125661+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:51:58.125661+0000","last_clean_scrub_stamp":"2023-08-09T13:51:58.125661+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-11T01:49:54.670225+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.4","version":"0'0","reported_seq":1,"reported_epoch":114,"state":"creating+peering","last_fresh":"2023-08-09T13:51:59.159598+0000","last_change":"2023-08-09T13:51:59.159598+0000","last_active":"2023-08-09T13:51:59.130555+0000","last_peered":"2023-08-09T13:51:59.130555+0000","last_clean":"2023-08-09T13:51:59.130555+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-09T13:51:59.159598+0000","last_undegraded":"2023-08-09T13:51:59.159598+0000","last_fullsized":"2023-08-09T13:51:59.159598+0000","mapping_epoch":114,"log_start":"0'0","ondisk_log_start":"0'0","created":114,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:51:59.130555+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:51:59.130555+0000","last_clean_scrub_stamp":"2023-08-09T13:51:59.130555+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.5","version":"0'0","reported_seq":9,"reported_epoch":114,"state":"active+clean","last_fresh":"2023-08-09T13:51:59.181921+0000","last_change":"2023-08-09T13:51:59.179649+0000","last_active":"2023-08-09T13:51:59.181921+0000","last_peered":"2023-08-09T13:51:59.181921+0000","last_clean":"2023-08-09T13:51:59.181921+0000","last_became_active":"2023-08-09T13:51:59.179348+0000","last_became_peered":"2023-08-09T13:51:59.179348+0000","last_unstale":"2023-08-09T13:51:59.181921+0000","last_undegraded":"2023-08-09T13:51:59.181921+0000","last_fullsized":"2023-08-09T13:51:59.181921+0000","mapping_epoch":113,"log_start":"0'0","ondisk_log_start":"0'0","created":113,"last_epoch_clean":114,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:51:58.125661+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:51:58.125661+0000","last_clean_scrub_stamp":"2023-08-09T13:51:58.125661+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T22:28:38.666057+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.5","version":"0'0","reported_seq":1,"reported_epoch":114,"state":"creating+peering","last_fresh":"2023-08-09T13:51:59.156952+0000","last_change":"2023-08-09T13:51:59.156952+0000","last_active":"2023-08-09T13:51:59.130555+0000","last_peered":"2023-08-09T13:51:59.130555+0000","last_clean":"2023-08-09T13:51:59.130555+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-09T13:51:59.156952+0000","last_undegraded":"2023-08-09T13:51:59.156952+0000","last_fullsized":"2023-08-09T13:51:59.156952+0000","mapping_epoch":114,"log_start":"0'0","ondisk_log_start":"0'0","created":114,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:51:59.130555+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:51:59.130555+0000","last_clean_scrub_stamp":"2023-08-09T13:51:59.130555+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"14.4","version":"0'0","reported_seq":9,"reported_epoch":114,"state":"active+clean","last_fresh":"2023-08-09T13:51:59.178493+0000","last_change":"2023-08-09T13:51:59.174311+0000","last_active":"2023-08-09T13:51:59.178493+0000","last_peered":"2023-08-09T13:51:59.178493+0000","last_clean":"2023-08-09T13:51:59.178493+0000","last_became_active":"2023-08-09T13:51:59.173990+0000","last_became_peered":"2023-08-09T13:51:59.173990+0000","last_unstale":"2023-08-09T13:51:59.178493+0000","last_undegraded":"2023-08-09T13:51:59.178493+0000","last_fullsized":"2023-08-09T13:51:59.178493+0000","mapping_epoch":113,"log_start":"0'0","ondisk_log_start":"0'0","created":113,"last_epoch_clean":114,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:51:58.125661+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:51:58.125661+0000","last_clean_scrub_stamp":"2023-08-09T13:51:58.125661+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T18:17:02.532080+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.2","version":"0'0","reported_seq":1,"reported_epoch":114,"state":"creating+peering","last_fresh":"2023-08-09T13:51:59.154951+0000","last_change":"2023-08-09T13:51:59.154951+0000","last_active":"2023-08-09T13:51:59.130555+0000","last_peered":"2023-08-09T13:51:59.130555+0000","last_clean":"2023-08-09T13:51:59.130555+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-09T13:51:59.154951+0000","last_undegraded":"2023-08-09T13:51:59.154951+0000","last_fullsized":"2023-08-09T13:51:59.154951+0000","mapping_epoch":114,"log_start":"0'0","ondisk_log_start":"0'0","created":114,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:51:59.130555+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:51:59.130555+0000","last_clean_scrub_stamp":"2023-08-09T13:51:59.130555+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"14.3","version":"0'0","reported_seq":9,"reported_epoch":114,"state":"active+clean","last_fresh":"2023-08-09T13:51:59.173345+0000","last_change":"2023-08-09T13:51:59.167523+0000","last_active":"2023-08-09T13:51:59.173345+0000","last_peered":"2023-08-09T13:51:59.173345+0000","last_clean":"2023-08-09T13:51:59.173345+0000","last_became_active":"2023-08-09T13:51:59.167059+0000","last_became_peered":"2023-08-09T13:51:59.167059+0000","last_unstale":"2023-08-09T13:51:59.173345+0000","last_undegraded":"2023-08-09T13:51:59.173345+0000","last_fullsized":"2023-08-09T13:51:59.173345+0000","mapping_epoch":113,"log_start":"0'0","ondisk_log_start":"0'0","created":113,"last_epoch_clean":114,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:51:58.125661+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:51:58.125661+0000","last_clean_scrub_stamp":"2023-08-09T13:51:58.125661+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T21:59:42.744613+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.3","version":"0'0","reported_seq":1,"reported_epoch":114,"state":"creating+peering","last_fresh":"2023-08-09T13:51:59.169608+0000","last_change":"2023-08-09T13:51:59.169608+0000","last_active":"2023-08-09T13:51:59.130555+0000","last_peered":"2023-08-09T13:51:59.130555+0000","last_clean":"2023-08-09T13:51:59.130555+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-09T13:51:59.169608+0000","last_undegraded":"2023-08-09T13:51:59.169608+0000","last_fullsized":"2023-08-09T13:51:59.169608+0000","mapping_epoch":114,"log_start":"0'0","ondisk_log_start":"0'0","created":114,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:51:59.130555+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:51:59.130555+0000","last_clean_scrub_stamp":"2023-08-09T13:51:59.130555+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.2","version":"0'0","reported_seq":9,"reported_epoch":114,"state":"active+clean","last_fresh":"2023-08-09T13:51:59.179577+0000","last_change":"2023-08-09T13:51:59.176919+0000","last_active":"2023-08-09T13:51:59.179577+0000","last_peered":"2023-08-09T13:51:59.179577+0000","last_clean":"2023-08-09T13:51:59.179577+0000","last_became_active":"2023-08-09T13:51:59.176503+0000","last_became_peered":"2023-08-09T13:51:59.176503+0000","last_unstale":"2023-08-09T13:51:59.179577+0000","last_undegraded":"2023-08-09T13:51:59.179577+0000","last_fullsized":"2023-08-09T13:51:59.179577+0000","mapping_epoch":113,"log_start":"0'0","ondisk_log_start":"0'0","created":113,"last_epoch_clean":114,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:51:58.125661+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:51:58.125661+0000","last_clean_scrub_stamp":"2023-08-09T13:51:58.125661+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T18:48:13.415415+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.0","version":"0'0","reported_seq":1,"reported_epoch":114,"state":"creating+peering","last_fresh":"2023-08-09T13:51:59.153156+0000","last_change":"2023-08-09T13:51:59.153156+0000","last_active":"2023-08-09T13:51:59.130555+0000","last_peered":"2023-08-09T13:51:59.130555+0000","last_clean":"2023-08-09T13:51:59.130555+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-09T13:51:59.153156+0000","last_undegraded":"2023-08-09T13:51:59.153156+0000","last_fullsized":"2023-08-09T13:51:59.153156+0000","mapping_epoch":114,"log_start":"0'0","ondisk_log_start":"0'0","created":114,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:51:59.130555+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:51:59.130555+0000","last_clean_scrub_stamp":"2023-08-09T13:51:59.130555+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"14.1","version":"0'0","reported_seq":9,"reported_epoch":114,"state":"active+clean","last_fresh":"2023-08-09T13:51:59.176213+0000","last_change":"2023-08-09T13:51:59.171070+0000","last_active":"2023-08-09T13:51:59.176213+0000","last_peered":"2023-08-09T13:51:59.176213+0000","last_clean":"2023-08-09T13:51:59.176213+0000","last_became_active":"2023-08-09T13:51:59.170788+0000","last_became_peered":"2023-08-09T13:51:59.170788+0000","last_unstale":"2023-08-09T13:51:59.176213+0000","last_undegraded":"2023-08-09T13:51:59.176213+0000","last_fullsized":"2023-08-09T13:51:59.176213+0000","mapping_epoch":113,"log_start":"0'0","ondisk_log_start":"0'0","created":113,"last_epoch_clean":114,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:51:58.125661+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:51:58.125661+0000","last_clean_scrub_stamp":"2023-08-09T13:51:58.125661+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-11T00:16:05.270843+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"15.1","version":"0'0","reported_seq":1,"reported_epoch":114,"state":"creating+peering","last_fresh":"2023-08-09T13:51:59.156158+0000","last_change":"2023-08-09T13:51:59.156158+0000","last_active":"2023-08-09T13:51:59.130555+0000","last_peered":"2023-08-09T13:51:59.130555+0000","last_clean":"2023-08-09T13:51:59.130555+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-09T13:51:59.156158+0000","last_undegraded":"2023-08-09T13:51:59.156158+0000","last_fullsized":"2023-08-09T13:51:59.156158+0000","mapping_epoch":114,"log_start":"0'0","ondisk_log_start":"0'0","created":114,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:51:59.130555+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:51:59.130555+0000","last_clean_scrub_stamp":"2023-08-09T13:51:59.130555+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"14.0","version":"0'0","reported_seq":9,"reported_epoch":114,"state":"active+clean","last_fresh":"2023-08-09T13:51:59.175060+0000","last_change":"2023-08-09T13:51:59.173223+0000","last_active":"2023-08-09T13:51:59.175060+0000","last_peered":"2023-08-09T13:51:59.175060+0000","last_clean":"2023-08-09T13:51:59.175060+0000","last_became_active":"2023-08-09T13:51:59.172948+0000","last_became_peered":"2023-08-09T13:51:59.172948+0000","last_unstale":"2023-08-09T13:51:59.175060+0000","last_undegraded":"2023-08-09T13:51:59.175060+0000","last_fullsized":"2023-08-09T13:51:59.175060+0000","mapping_epoch":113,"log_start":"0'0","ondisk_log_start":"0'0","created":113,"last_epoch_clean":114,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:51:58.125661+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:51:58.125661+0000","last_clean_scrub_stamp":"2023-08-09T13:51:58.125661+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T16:16:44.489876+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.e","version":"0'0","reported_seq":1,"reported_epoch":114,"state":"creating+peering","last_fresh":"2023-08-09T13:51:59.155432+0000","last_change":"2023-08-09T13:51:59.155432+0000","last_active":"2023-08-09T13:51:59.130555+0000","last_peered":"2023-08-09T13:51:59.130555+0000","last_clean":"2023-08-09T13:51:59.130555+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-09T13:51:59.155432+0000","last_undegraded":"2023-08-09T13:51:59.155432+0000","last_fullsized":"2023-08-09T13:51:59.155432+0000","mapping_epoch":114,"log_start":"0'0","ondisk_log_start":"0'0","created":114,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:51:59.130555+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:51:59.130555+0000","last_clean_scrub_stamp":"2023-08-09T13:51:59.130555+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"14.f","version":"0'0","reported_seq":9,"reported_epoch":114,"state":"active+clean","last_fresh":"2023-08-09T13:51:59.176216+0000","last_change":"2023-08-09T13:51:59.174685+0000","last_active":"2023-08-09T13:51:59.176216+0000","last_peered":"2023-08-09T13:51:59.176216+0000","last_clean":"2023-08-09T13:51:59.176216+0000","last_became_active":"2023-08-09T13:51:59.174370+0000","last_became_peered":"2023-08-09T13:51:59.174370+0000","last_unstale":"2023-08-09T13:51:59.176216+0000","last_undegraded":"2023-08-09T13:51:59.176216+0000","last_fullsized":"2023-08-09T13:51:59.176216+0000","mapping_epoch":113,"log_start":"0'0","ondisk_log_start":"0'0","created":113,"last_epoch_clean":114,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:51:58.125661+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:51:58.125661+0000","last_clean_scrub_stamp":"2023-08-09T13:51:58.125661+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T19:42:06.381104+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":189,"reported_epoch":114,"state":"active+clean","last_fresh":"2023-08-09T13:51:59.160631+0000","last_change":"2023-08-09T13:33:56.401406+0000","last_active":"2023-08-09T13:51:59.160631+0000","last_peered":"2023-08-09T13:51:59.160631+0000","last_clean":"2023-08-09T13:51:59.160631+0000","last_became_active":"2023-08-09T13:33:56.400487+0000","last_became_peered":"2023-08-09T13:33:56.400487+0000","last_unstale":"2023-08-09T13:51:59.160631+0000","last_undegraded":"2023-08-09T13:51:59.160631+0000","last_fullsized":"2023-08-09T13:51:59.160631+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:33:54.845869+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:33:54.845869+0000","last_clean_scrub_stamp":"2023-08-09T13:33:54.845869+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T16:56:32.266112+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"15.f","version":"0'0","reported_seq":1,"reported_epoch":114,"state":"creating+peering","last_fresh":"2023-08-09T13:51:59.156558+0000","last_change":"2023-08-09T13:51:59.156558+0000","last_active":"2023-08-09T13:51:59.130555+0000","last_peered":"2023-08-09T13:51:59.130555+0000","last_clean":"2023-08-09T13:51:59.130555+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-09T13:51:59.156558+0000","last_undegraded":"2023-08-09T13:51:59.156558+0000","last_fullsized":"2023-08-09T13:51:59.156558+0000","mapping_epoch":114,"log_start":"0'0","ondisk_log_start":"0'0","created":114,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:51:59.130555+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:51:59.130555+0000","last_clean_scrub_stamp":"2023-08-09T13:51:59.130555+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"14.e","version":"0'0","reported_seq":9,"reported_epoch":114,"state":"active+clean","last_fresh":"2023-08-09T13:51:59.182628+0000","last_change":"2023-08-09T13:51:59.180623+0000","last_active":"2023-08-09T13:51:59.182628+0000","last_peered":"2023-08-09T13:51:59.182628+0000","last_clean":"2023-08-09T13:51:59.182628+0000","last_became_active":"2023-08-09T13:51:59.178785+0000","last_became_peered":"2023-08-09T13:51:59.178785+0000","last_unstale":"2023-08-09T13:51:59.182628+0000","last_undegraded":"2023-08-09T13:51:59.182628+0000","last_fullsized":"2023-08-09T13:51:59.182628+0000","mapping_epoch":113,"log_start":"0'0","ondisk_log_start":"0'0","created":113,"last_epoch_clean":114,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:51:58.125661+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:51:58.125661+0000","last_clean_scrub_stamp":"2023-08-09T13:51:58.125661+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T21:35:50.978670+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":193,"reported_epoch":114,"state":"active+clean","last_fresh":"2023-08-09T13:51:59.171640+0000","last_change":"2023-08-09T13:33:56.401123+0000","last_active":"2023-08-09T13:51:59.171640+0000","last_peered":"2023-08-09T13:51:59.171640+0000","last_clean":"2023-08-09T13:51:59.171640+0000","last_became_active":"2023-08-09T13:33:56.399933+0000","last_became_peered":"2023-08-09T13:33:56.399933+0000","last_unstale":"2023-08-09T13:51:59.171640+0000","last_undegraded":"2023-08-09T13:51:59.171640+0000","last_fullsized":"2023-08-09T13:51:59.171640+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:33:54.845869+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:33:54.845869+0000","last_clean_scrub_stamp":"2023-08-09T13:33:54.845869+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T19:50:20.512873+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"15.c","version":"0'0","reported_seq":1,"reported_epoch":114,"state":"creating+peering","last_fresh":"2023-08-09T13:51:59.166603+0000","last_change":"2023-08-09T13:51:59.166603+0000","last_active":"2023-08-09T13:51:59.130555+0000","last_peered":"2023-08-09T13:51:59.130555+0000","last_clean":"2023-08-09T13:51:59.130555+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-09T13:51:59.166603+0000","last_undegraded":"2023-08-09T13:51:59.166603+0000","last_fullsized":"2023-08-09T13:51:59.166603+0000","mapping_epoch":114,"log_start":"0'0","ondisk_log_start":"0'0","created":114,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:51:59.130555+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:51:59.130555+0000","last_clean_scrub_stamp":"2023-08-09T13:51:59.130555+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"14.d","version":"0'0","reported_seq":9,"reported_epoch":114,"state":"active+clean","last_fresh":"2023-08-09T13:51:59.179009+0000","last_change":"2023-08-09T13:51:59.174909+0000","last_active":"2023-08-09T13:51:59.179009+0000","last_peered":"2023-08-09T13:51:59.179009+0000","last_clean":"2023-08-09T13:51:59.179009+0000","last_became_active":"2023-08-09T13:51:59.174527+0000","last_became_peered":"2023-08-09T13:51:59.174527+0000","last_unstale":"2023-08-09T13:51:59.179009+0000","last_undegraded":"2023-08-09T13:51:59.179009+0000","last_fullsized":"2023-08-09T13:51:59.179009+0000","mapping_epoch":113,"log_start":"0'0","ondisk_log_start":"0'0","created":113,"last_epoch_clean":114,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:51:58.125661+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:51:58.125661+0000","last_clean_scrub_stamp":"2023-08-09T13:51:58.125661+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T19:54:13.290921+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"1.2","version":"0'0","reported_seq":189,"reported_epoch":114,"state":"active+clean","last_fresh":"2023-08-09T13:51:59.161664+0000","last_change":"2023-08-09T13:33:56.402794+0000","last_active":"2023-08-09T13:51:59.161664+0000","last_peered":"2023-08-09T13:51:59.161664+0000","last_clean":"2023-08-09T13:51:59.161664+0000","last_became_active":"2023-08-09T13:33:56.402038+0000","last_became_peered":"2023-08-09T13:33:56.402038+0000","last_unstale":"2023-08-09T13:51:59.161664+0000","last_undegraded":"2023-08-09T13:51:59.161664+0000","last_fullsized":"2023-08-09T13:51:59.161664+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:33:54.845869+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:33:54.845869+0000","last_clean_scrub_stamp":"2023-08-09T13:33:54.845869+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T22:44:13.009942+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"15.d","version":"0'0","reported_seq":1,"reported_epoch":114,"state":"creating+peering","last_fresh":"2023-08-09T13:51:59.154844+0000","last_change":"2023-08-09T13:51:59.154844+0000","last_active":"2023-08-09T13:51:59.130555+0000","last_peered":"2023-08-09T13:51:59.130555+0000","last_clean":"2023-08-09T13:51:59.130555+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-09T13:51:59.154844+0000","last_undegraded":"2023-08-09T13:51:59.154844+0000","last_fullsized":"2023-08-09T13:51:59.154844+0000","mapping_epoch":114,"log_start":"0'0","ondisk_log_start":"0'0","created":114,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:51:59.130555+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:51:59.130555+0000","last_clean_scrub_stamp":"2023-08-09T13:51:59.130555+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"14.c","version":"0'0","reported_seq":9,"reported_epoch":114,"state":"active+clean","last_fresh":"2023-08-09T13:51:59.184772+0000","last_change":"2023-08-09T13:51:59.183702+0000","last_active":"2023-08-09T13:51:59.184772+0000","last_peered":"2023-08-09T13:51:59.184772+0000","last_clean":"2023-08-09T13:51:59.184772+0000","last_became_active":"2023-08-09T13:51:59.183387+0000","last_became_peered":"2023-08-09T13:51:59.183387+0000","last_unstale":"2023-08-09T13:51:59.184772+0000","last_undegraded":"2023-08-09T13:51:59.184772+0000","last_fullsized":"2023-08-09T13:51:59.184772+0000","mapping_epoch":113,"log_start":"0'0","ondisk_log_start":"0'0","created":113,"last_epoch_clean":114,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:51:58.125661+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:51:58.125661+0000","last_clean_scrub_stamp":"2023-08-09T13:51:58.125661+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T16:05:08.950799+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.3","version":"0'0","reported_seq":189,"reported_epoch":114,"state":"active+clean","last_fresh":"2023-08-09T13:51:59.169171+0000","last_change":"2023-08-09T13:33:55.886749+0000","last_active":"2023-08-09T13:51:59.169171+0000","last_peered":"2023-08-09T13:51:59.169171+0000","last_clean":"2023-08-09T13:51:59.169171+0000","last_became_active":"2023-08-09T13:33:55.886116+0000","last_became_peered":"2023-08-09T13:33:55.886116+0000","last_unstale":"2023-08-09T13:51:59.169171+0000","last_undegraded":"2023-08-09T13:51:59.169171+0000","last_fullsized":"2023-08-09T13:51:59.169171+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:33:54.845869+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:33:54.845869+0000","last_clean_scrub_stamp":"2023-08-09T13:33:54.845869+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T20:06:41.616544+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"105'186","reported_seq":583,"reported_epoch":114,"state":"active+clean","last_fresh":"2023-08-09T13:51:59.160322+0000","last_change":"2023-08-09T13:33:55.886345+0000","last_active":"2023-08-09T13:51:59.160322+0000","last_peered":"2023-08-09T13:51:59.160322+0000","last_clean":"2023-08-09T13:51:59.160322+0000","last_became_active":"2023-08-09T13:33:55.885781+0000","last_became_peered":"2023-08-09T13:33:55.885781+0000","last_unstale":"2023-08-09T13:51:59.160322+0000","last_undegraded":"2023-08-09T13:51:59.160322+0000","last_fullsized":"2023-08-09T13:51:59.160322+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:33:54.845869+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:33:54.845869+0000","last_clean_scrub_stamp":"2023-08-09T13:33:54.845869+0000","objects_scrubbed":0,"log_size":186,"log_dups_size":0,"ondisk_log_size":186,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T16:37:53.138245+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":715,"num_read_kb":1418,"num_write":283,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"15.a","version":"0'0","reported_seq":1,"reported_epoch":114,"state":"creating+peering","last_fresh":"2023-08-09T13:51:59.164187+0000","last_change":"2023-08-09T13:51:59.164187+0000","last_active":"2023-08-09T13:51:59.130555+0000","last_peered":"2023-08-09T13:51:59.130555+0000","last_clean":"2023-08-09T13:51:59.130555+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-09T13:51:59.164187+0000","last_undegraded":"2023-08-09T13:51:59.164187+0000","last_fullsized":"2023-08-09T13:51:59.164187+0000","mapping_epoch":114,"log_start":"0'0","ondisk_log_start":"0'0","created":114,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:51:59.130555+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:51:59.130555+0000","last_clean_scrub_stamp":"2023-08-09T13:51:59.130555+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"14.b","version":"0'0","reported_seq":9,"reported_epoch":114,"state":"active+clean","last_fresh":"2023-08-09T13:51:59.173500+0000","last_change":"2023-08-09T13:51:59.168603+0000","last_active":"2023-08-09T13:51:59.173500+0000","last_peered":"2023-08-09T13:51:59.173500+0000","last_clean":"2023-08-09T13:51:59.173500+0000","last_became_active":"2023-08-09T13:51:59.168273+0000","last_became_peered":"2023-08-09T13:51:59.168273+0000","last_unstale":"2023-08-09T13:51:59.173500+0000","last_undegraded":"2023-08-09T13:51:59.173500+0000","last_fullsized":"2023-08-09T13:51:59.173500+0000","mapping_epoch":113,"log_start":"0'0","ondisk_log_start":"0'0","created":113,"last_epoch_clean":114,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:51:58.125661+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:51:58.125661+0000","last_clean_scrub_stamp":"2023-08-09T13:51:58.125661+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T20:40:57.215042+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":199,"reported_epoch":114,"state":"active+clean","last_fresh":"2023-08-09T13:51:59.163213+0000","last_change":"2023-08-09T13:33:55.889565+0000","last_active":"2023-08-09T13:51:59.163213+0000","last_peered":"2023-08-09T13:51:59.163213+0000","last_clean":"2023-08-09T13:51:59.163213+0000","last_became_active":"2023-08-09T13:33:55.889164+0000","last_became_peered":"2023-08-09T13:33:55.889164+0000","last_unstale":"2023-08-09T13:51:59.163213+0000","last_undegraded":"2023-08-09T13:51:59.163213+0000","last_fullsized":"2023-08-09T13:51:59.163213+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:33:54.845869+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:33:54.845869+0000","last_clean_scrub_stamp":"2023-08-09T13:33:54.845869+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T19:12:54.459595+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"15.b","version":"0'0","reported_seq":1,"reported_epoch":114,"state":"creating+peering","last_fresh":"2023-08-09T13:51:59.156039+0000","last_change":"2023-08-09T13:51:59.156039+0000","last_active":"2023-08-09T13:51:59.130555+0000","last_peered":"2023-08-09T13:51:59.130555+0000","last_clean":"2023-08-09T13:51:59.130555+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-09T13:51:59.156039+0000","last_undegraded":"2023-08-09T13:51:59.156039+0000","last_fullsized":"2023-08-09T13:51:59.156039+0000","mapping_epoch":114,"log_start":"0'0","ondisk_log_start":"0'0","created":114,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:51:59.130555+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:51:59.130555+0000","last_clean_scrub_stamp":"2023-08-09T13:51:59.130555+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"14.a","version":"0'0","reported_seq":9,"reported_epoch":114,"state":"active+clean","last_fresh":"2023-08-09T13:51:59.182288+0000","last_change":"2023-08-09T13:51:59.179151+0000","last_active":"2023-08-09T13:51:59.182288+0000","last_peered":"2023-08-09T13:51:59.182288+0000","last_clean":"2023-08-09T13:51:59.182288+0000","last_became_active":"2023-08-09T13:51:59.178247+0000","last_became_peered":"2023-08-09T13:51:59.178247+0000","last_unstale":"2023-08-09T13:51:59.182288+0000","last_undegraded":"2023-08-09T13:51:59.182288+0000","last_fullsized":"2023-08-09T13:51:59.182288+0000","mapping_epoch":113,"log_start":"0'0","ondisk_log_start":"0'0","created":113,"last_epoch_clean":114,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:51:58.125661+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:51:58.125661+0000","last_clean_scrub_stamp":"2023-08-09T13:51:58.125661+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T14:19:12.218569+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":193,"reported_epoch":114,"state":"active+clean","last_fresh":"2023-08-09T13:51:59.180570+0000","last_change":"2023-08-09T13:33:56.404236+0000","last_active":"2023-08-09T13:51:59.180570+0000","last_peered":"2023-08-09T13:51:59.180570+0000","last_clean":"2023-08-09T13:51:59.180570+0000","last_became_active":"2023-08-09T13:33:56.402485+0000","last_became_peered":"2023-08-09T13:33:56.402485+0000","last_unstale":"2023-08-09T13:51:59.180570+0000","last_undegraded":"2023-08-09T13:51:59.180570+0000","last_fullsized":"2023-08-09T13:51:59.180570+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:33:54.845869+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:33:54.845869+0000","last_clean_scrub_stamp":"2023-08-09T13:33:54.845869+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T20:10:52.098147+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"15.8","version":"0'0","reported_seq":1,"reported_epoch":114,"state":"creating+peering","last_fresh":"2023-08-09T13:51:59.162674+0000","last_change":"2023-08-09T13:51:59.162674+0000","last_active":"2023-08-09T13:51:59.130555+0000","last_peered":"2023-08-09T13:51:59.130555+0000","last_clean":"2023-08-09T13:51:59.130555+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-09T13:51:59.162674+0000","last_undegraded":"2023-08-09T13:51:59.162674+0000","last_fullsized":"2023-08-09T13:51:59.162674+0000","mapping_epoch":114,"log_start":"0'0","ondisk_log_start":"0'0","created":114,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:51:59.130555+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:51:59.130555+0000","last_clean_scrub_stamp":"2023-08-09T13:51:59.130555+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"14.9","version":"0'0","reported_seq":9,"reported_epoch":114,"state":"active+clean","last_fresh":"2023-08-09T13:51:59.180017+0000","last_change":"2023-08-09T13:51:59.177964+0000","last_active":"2023-08-09T13:51:59.180017+0000","last_peered":"2023-08-09T13:51:59.180017+0000","last_clean":"2023-08-09T13:51:59.180017+0000","last_became_active":"2023-08-09T13:51:59.177186+0000","last_became_peered":"2023-08-09T13:51:59.177186+0000","last_unstale":"2023-08-09T13:51:59.180017+0000","last_undegraded":"2023-08-09T13:51:59.180017+0000","last_fullsized":"2023-08-09T13:51:59.180017+0000","mapping_epoch":113,"log_start":"0'0","ondisk_log_start":"0'0","created":113,"last_epoch_clean":114,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:51:58.125661+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:51:58.125661+0000","last_clean_scrub_stamp":"2023-08-09T13:51:58.125661+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T18:14:44.331723+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":189,"reported_epoch":114,"state":"active+clean","last_fresh":"2023-08-09T13:51:59.157955+0000","last_change":"2023-08-09T13:33:56.401781+0000","last_active":"2023-08-09T13:51:59.157955+0000","last_peered":"2023-08-09T13:51:59.157955+0000","last_clean":"2023-08-09T13:51:59.157955+0000","last_became_active":"2023-08-09T13:33:56.400336+0000","last_became_peered":"2023-08-09T13:33:56.400336+0000","last_unstale":"2023-08-09T13:51:59.157955+0000","last_undegraded":"2023-08-09T13:51:59.157955+0000","last_fullsized":"2023-08-09T13:51:59.157955+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:33:54.845869+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:33:54.845869+0000","last_clean_scrub_stamp":"2023-08-09T13:33:54.845869+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T16:05:37.049450+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.9","version":"0'0","reported_seq":1,"reported_epoch":114,"state":"creating+peering","last_fresh":"2023-08-09T13:51:59.157318+0000","last_change":"2023-08-09T13:51:59.157318+0000","last_active":"2023-08-09T13:51:59.130555+0000","last_peered":"2023-08-09T13:51:59.130555+0000","last_clean":"2023-08-09T13:51:59.130555+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-09T13:51:59.157318+0000","last_undegraded":"2023-08-09T13:51:59.157318+0000","last_fullsized":"2023-08-09T13:51:59.157318+0000","mapping_epoch":114,"log_start":"0'0","ondisk_log_start":"0'0","created":114,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:51:59.130555+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:51:59.130555+0000","last_clean_scrub_stamp":"2023-08-09T13:51:59.130555+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"14.8","version":"0'0","reported_seq":9,"reported_epoch":114,"state":"active+clean","last_fresh":"2023-08-09T13:51:59.174355+0000","last_change":"2023-08-09T13:51:59.170689+0000","last_active":"2023-08-09T13:51:59.174355+0000","last_peered":"2023-08-09T13:51:59.174355+0000","last_clean":"2023-08-09T13:51:59.174355+0000","last_became_active":"2023-08-09T13:51:59.170409+0000","last_became_peered":"2023-08-09T13:51:59.170409+0000","last_unstale":"2023-08-09T13:51:59.174355+0000","last_undegraded":"2023-08-09T13:51:59.174355+0000","last_fullsized":"2023-08-09T13:51:59.174355+0000","mapping_epoch":113,"log_start":"0'0","ondisk_log_start":"0'0","created":113,"last_epoch_clean":114,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:51:58.125661+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:51:58.125661+0000","last_clean_scrub_stamp":"2023-08-09T13:51:58.125661+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T17:00:34.835236+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":189,"reported_epoch":114,"state":"active+clean","last_fresh":"2023-08-09T13:51:59.159658+0000","last_change":"2023-08-09T13:33:55.886389+0000","last_active":"2023-08-09T13:51:59.159658+0000","last_peered":"2023-08-09T13:51:59.159658+0000","last_clean":"2023-08-09T13:51:59.159658+0000","last_became_active":"2023-08-09T13:33:55.885638+0000","last_became_peered":"2023-08-09T13:33:55.885638+0000","last_unstale":"2023-08-09T13:51:59.159658+0000","last_undegraded":"2023-08-09T13:51:59.159658+0000","last_fullsized":"2023-08-09T13:51:59.159658+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:33:54.845869+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:33:54.845869+0000","last_clean_scrub_stamp":"2023-08-09T13:33:54.845869+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T16:40:37.514883+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]}],"pool_stats":[{"poolid":15,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":2,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":715,"num_read_kb":1418,"num_write":283,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":186,"ondisk_log_size":186,"up":2,"acting":2,"num_store_stats":2},{"poolid":14,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":8}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834575064,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":37168,"kb_used_data":1952,"kb_used_omap":1,"kb_used_meta":35198,"kb_avail":94334672,"statfs":{"total":96636764160,"available":96598704128,"internally_reserved":0,"allocated":1998848,"data_stored":1338378,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1781,"internal_metadata":36043019},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Aug 9 13:51:20 2023","interfaces":[{"interface":"back","average":{"1min":1.101,"5min":1.1910000000000001,"15min":1.2370000000000001},"min":{"1min":0.61599999999999999,"5min":0.61599999999999999,"15min":0.61599999999999999},"max":{"1min":1.764,"5min":1.8919999999999999,"15min":2.1280000000000001},"last":1.1899999999999999},{"interface":"front","average":{"1min":1.153,"5min":1.224,"15min":1.2430000000000001},"min":{"1min":0.64200000000000002,"5min":0.64200000000000002,"15min":0.64200000000000002},"max":{"1min":1.8839999999999999,"5min":1.8839999999999999,"15min":2.0099999999999998},"last":1.1259999999999999}]},{"osd":1,"last update":"Wed Aug 9 13:51:20 2023","interfaces":[{"interface":"back","average":{"1min":1.1830000000000001,"5min":1.22,"15min":1.26},"min":{"1min":0.69199999999999995,"5min":0.69199999999999995,"15min":0.59799999999999998},"max":{"1min":1.8120000000000001,"5min":1.8120000000000001,"15min":1.9470000000000001},"last":1.4390000000000001},{"interface":"front","average":{"1min":1.1839999999999999,"5min":1.2310000000000001,"15min":1.2809999999999999},"min":{"1min":0.69099999999999995,"5min":0.59299999999999997,"15min":0.59299999999999997},"max":{"1min":1.998,"5min":1.998,"15min":1.998},"last":1.2789999999999999}]},{"osd":2,"last update":"Wed Aug 9 13:51:20 2023","interfaces":[{"interface":"back","average":{"1min":1.1990000000000001,"5min":1.234,"15min":1.252},"min":{"1min":0.79100000000000004,"5min":0.65700000000000003,"15min":0.65700000000000003},"max":{"1min":1.6539999999999999,"5min":2.0129999999999999,"15min":2.0129999999999999},"last":1.518},{"interface":"front","average":{"1min":1.22,"5min":1.2829999999999999,"15min":1.369},"min":{"1min":0.58199999999999996,"5min":0.58199999999999996,"15min":0.58199999999999996},"max":{"1min":1.7050000000000001,"5min":1.8740000000000001,"15min":2.4140000000000001},"last":1.5489999999999999}]},{"osd":3,"last update":"Wed Aug 9 13:51:20 2023","interfaces":[{"interface":"back","average":{"1min":1.272,"5min":1.29,"15min":1.3360000000000001},"min":{"1min":0.67500000000000004,"5min":0.67500000000000004,"15min":0.67500000000000004},"max":{"1min":1.8140000000000001,"5min":1.97,"15min":1.97},"last":1.6180000000000001},{"interface":"front","average":{"1min":1.363,"5min":1.3959999999999999,"15min":1.413},"min":{"1min":0.71499999999999997,"5min":0.71499999999999997,"15min":0.71499999999999997},"max":{"1min":1.615,"5min":2.0790000000000002,"15min":2.2480000000000002},"last":1.647}]},{"osd":4,"last update":"Wed Aug 9 13:51:20 2023","interfaces":[{"interface":"back","average":{"1min":1.2430000000000001,"5min":1.2669999999999999,"15min":1.2749999999999999},"min":{"1min":0.74299999999999999,"5min":0.59499999999999997,"15min":0.59499999999999997},"max":{"1min":2.097,"5min":2.097,"15min":2.343},"last":1.008},{"interface":"front","average":{"1min":1.2509999999999999,"5min":1.2969999999999999,"15min":1.3340000000000001},"min":{"1min":0.60799999999999998,"5min":0.60799999999999998,"15min":0.60799999999999998},"max":{"1min":2.0539999999999998,"5min":2.1419999999999999,"15min":2.1419999999999999},"last":1.333}]},{"osd":5,"last update":"Wed Aug 9 13:51:20 2023","interfaces":[{"interface":"back","average":{"1min":1.456,"5min":1.4350000000000001,"15min":1.4179999999999999},"min":{"1min":0.65800000000000003,"5min":0.65800000000000003,"15min":0.65800000000000003},"max":{"1min":1.7549999999999999,"5min":2.1480000000000001,"15min":2.6749999999999998},"last":1.052},{"interface":"front","average":{"1min":1.383,"5min":1.4099999999999999,"15min":1.4450000000000001},"min":{"1min":0.83999999999999997,"5min":0.83999999999999997,"15min":0.79700000000000004},"max":{"1min":2.157,"5min":2.157,"15min":2.9569999999999999},"last":1.3859999999999999}]},{"osd":6,"last update":"Wed Aug 9 13:51:20 2023","interfaces":[{"interface":"back","average":{"1min":1.4650000000000001,"5min":1.504,"15min":1.5429999999999999},"min":{"1min":0.88500000000000001,"5min":0.85499999999999998,"15min":0.85499999999999998},"max":{"1min":1.8280000000000001,"5min":2.2069999999999999,"15min":2.8149999999999999},"last":1.4870000000000001},{"interface":"front","average":{"1min":1.421,"5min":1.4430000000000001,"15min":1.474},"min":{"1min":0.79700000000000004,"5min":0.79700000000000004,"15min":0.79700000000000004},"max":{"1min":1.736,"5min":2.1320000000000001,"15min":2.2530000000000001},"last":1.7170000000000001}]}]},{"osd":6,"up_from":13,"seq":55834575064,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":36580,"kb_used_data":1368,"kb_used_omap":1,"kb_used_meta":35198,"kb_avail":94335260,"statfs":{"total":96636764160,"available":96599306240,"internally_reserved":0,"allocated":1400832,"data_stored":746649,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1781,"internal_metadata":36043019},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":8,"apply_latency_ms":8,"commit_latency_ns":8000000,"apply_latency_ns":8000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Aug 9 13:51:24 2023","interfaces":[{"interface":"back","average":{"1min":0.98499999999999999,"5min":1.1160000000000001,"15min":1.153},"min":{"1min":0.54000000000000004,"5min":0.54000000000000004,"15min":0.54000000000000004},"max":{"1min":1.484,"5min":1.633,"15min":1.9870000000000001},"last":0.96899999999999997},{"interface":"front","average":{"1min":1.0620000000000001,"5min":1.492,"15min":1.306},"min":{"1min":0.51800000000000002,"5min":0.51800000000000002,"15min":0.51800000000000002},"max":{"1min":1.6479999999999999,"5min":28.036000000000001,"15min":28.036000000000001},"last":1.0800000000000001}]},{"osd":1,"last update":"Wed Aug 9 13:51:24 2023","interfaces":[{"interface":"back","average":{"1min":1.0940000000000001,"5min":1.238,"15min":1.232},"min":{"1min":0.48999999999999999,"5min":0.48999999999999999,"15min":0.48999999999999999},"max":{"1min":1.6639999999999999,"5min":7.742,"15min":7.742},"last":0.79800000000000004},{"interface":"front","average":{"1min":1.1950000000000001,"5min":1.276,"15min":1.28},"min":{"1min":0.64600000000000002,"5min":0.64600000000000002,"15min":0.64600000000000002},"max":{"1min":2.056,"5min":2.056,"15min":2.056},"last":1.222}]},{"osd":2,"last update":"Wed Aug 9 13:51:24 2023","interfaces":[{"interface":"back","average":{"1min":1.206,"5min":1.323,"15min":1.3260000000000001},"min":{"1min":0.80600000000000005,"5min":0.80600000000000005,"15min":0.74099999999999999},"max":{"1min":1.8819999999999999,"5min":4.0640000000000001,"15min":4.0640000000000001},"last":1.147},{"interface":"front","average":{"1min":1.161,"5min":1.399,"15min":1.347},"min":{"1min":0.80700000000000005,"5min":0.70999999999999996,"15min":0.70999999999999996},"max":{"1min":1.6679999999999999,"5min":13.154999999999999,"15min":13.154999999999999},"last":1.5029999999999999}]},{"osd":3,"last update":"Wed Aug 9 13:51:24 2023","interfaces":[{"interface":"back","average":{"1min":1.196,"5min":1.452,"15min":1.415},"min":{"1min":0.67200000000000004,"5min":0.67200000000000004,"15min":0.67200000000000004},"max":{"1min":1.522,"5min":13.349,"15min":13.349},"last":1.401},{"interface":"front","average":{"1min":1.254,"5min":1.353,"15min":1.3280000000000001},"min":{"1min":0.75700000000000001,"5min":0.75700000000000001,"15min":0.74299999999999999},"max":{"1min":1.732,"5min":7.9029999999999996,"15min":7.9029999999999996},"last":1.319}]},{"osd":4,"last update":"Wed Aug 9 13:51:24 2023","interfaces":[{"interface":"back","average":{"1min":1.323,"5min":1.3540000000000001,"15min":1.284},"min":{"1min":0.86199999999999999,"5min":0.84499999999999997,"15min":0.64300000000000002},"max":{"1min":1.833,"5min":5.2290000000000001,"15min":5.2290000000000001},"last":1.3640000000000001},{"interface":"front","average":{"1min":1.367,"5min":1.506,"15min":1.3879999999999999},"min":{"1min":0.89900000000000002,"5min":0.78700000000000003,"15min":0.78700000000000003},"max":{"1min":1.714,"5min":12.1,"15min":12.1},"last":1.27}]},{"osd":5,"last update":"Wed Aug 9 13:51:24 2023","interfaces":[{"interface":"back","average":{"1min":1.423,"5min":1.446,"15min":1.3660000000000001},"min":{"1min":0.57699999999999996,"5min":0.57699999999999996,"15min":0.57699999999999996},"max":{"1min":1.843,"5min":8.1899999999999995,"15min":8.1899999999999995},"last":1.1850000000000001},{"interface":"front","average":{"1min":1.55,"5min":1.6679999999999999,"15min":1.593},"min":{"1min":0.93000000000000005,"5min":0.93000000000000005,"15min":0.83799999999999997},"max":{"1min":2.1459999999999999,"5min":8.8889999999999993,"15min":8.8889999999999993},"last":1.4379999999999999}]},{"osd":7,"last update":"Wed Aug 9 13:51:24 2023","interfaces":[{"interface":"back","average":{"1min":1.5269999999999999,"5min":1.6719999999999999,"15min":1.587},"min":{"1min":0.96099999999999997,"5min":0.96099999999999997,"15min":0.62},"max":{"1min":1.907,"5min":9.3320000000000007,"15min":9.3320000000000007},"last":1.6359999999999999},{"interface":"front","average":{"1min":1.556,"5min":1.613,"15min":1.55},"min":{"1min":0.88200000000000001,"5min":0.88200000000000001,"15min":0.88200000000000001},"max":{"1min":1.97,"5min":8.0399999999999991,"15min":8.0399999999999991},"last":1.4690000000000001}]}]},{"osd":5,"up_from":13,"seq":55834575064,"num_pgs":5,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":40668,"kb_used_data":1360,"kb_used_omap":1,"kb_used_meta":39294,"kb_avail":94331172,"statfs":{"total":96636764160,"available":96595120128,"internally_reserved":0,"allocated":1392640,"data_stored":741018,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1781,"internal_metadata":40237323},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Aug 9 13:51:21 2023","interfaces":[{"interface":"back","average":{"1min":1.542,"5min":1.25,"15min":1.206},"min":{"1min":0.59099999999999997,"5min":0.59099999999999997,"15min":0.59099999999999997},"max":{"1min":8.9299999999999997,"5min":8.9299999999999997,"15min":8.9299999999999997},"last":1.2350000000000001},{"interface":"front","average":{"1min":1.6080000000000001,"5min":1.264,"15min":1.2330000000000001},"min":{"1min":0.60499999999999998,"5min":0.60499999999999998,"15min":0.60499999999999998},"max":{"1min":8.7140000000000004,"5min":8.7140000000000004,"15min":8.7140000000000004},"last":1.6160000000000001}]},{"osd":1,"last update":"Wed Aug 9 13:51:21 2023","interfaces":[{"interface":"back","average":{"1min":1.476,"5min":1.2330000000000001,"15min":1.246},"min":{"1min":0.68799999999999994,"5min":0.68799999999999994,"15min":0.64100000000000001},"max":{"1min":8.2720000000000002,"5min":8.2720000000000002,"15min":8.2720000000000002},"last":0.82399999999999995},{"interface":"front","average":{"1min":1.6679999999999999,"5min":1.359,"15min":1.3100000000000001},"min":{"1min":0.80500000000000005,"5min":0.66900000000000004,"15min":0.66900000000000004},"max":{"1min":9.0020000000000007,"5min":9.0020000000000007,"15min":9.0020000000000007},"last":1.3720000000000001}]},{"osd":2,"last update":"Wed Aug 9 13:51:21 2023","interfaces":[{"interface":"back","average":{"1min":1.7190000000000001,"5min":1.335,"15min":1.319},"min":{"1min":0.81799999999999995,"5min":0.68999999999999995,"15min":0.61599999999999999},"max":{"1min":8.6050000000000004,"5min":8.6050000000000004,"15min":8.6050000000000004},"last":1.1439999999999999},{"interface":"front","average":{"1min":1.641,"5min":1.371,"15min":1.347},"min":{"1min":0.78300000000000003,"5min":0.78300000000000003,"15min":0.72099999999999997},"max":{"1min":8.4149999999999991,"5min":8.4149999999999991,"15min":8.4149999999999991},"last":1.4299999999999999}]},{"osd":3,"last update":"Wed Aug 9 13:51:21 2023","interfaces":[{"interface":"back","average":{"1min":1.8160000000000001,"5min":1.466,"15min":1.391},"min":{"1min":0.72599999999999998,"5min":0.72599999999999998,"15min":0.59799999999999998},"max":{"1min":9.0470000000000006,"5min":9.0470000000000006,"15min":9.0470000000000006},"last":1.573},{"interface":"front","average":{"1min":1.738,"5min":1.4970000000000001,"15min":1.4370000000000001},"min":{"1min":0.76300000000000001,"5min":0.76300000000000001,"15min":0.70999999999999996},"max":{"1min":8.7669999999999995,"5min":8.7669999999999995,"15min":8.7669999999999995},"last":1.302}]},{"osd":4,"last update":"Wed Aug 9 13:51:21 2023","interfaces":[{"interface":"back","average":{"1min":1.724,"5min":1.367,"15min":1.321},"min":{"1min":0.54300000000000004,"5min":0.54300000000000004,"15min":0.54300000000000004},"max":{"1min":8.4909999999999997,"5min":8.4909999999999997,"15min":8.4909999999999997},"last":1.55},{"interface":"front","average":{"1min":1.9079999999999999,"5min":1.4570000000000001,"15min":1.4119999999999999},"min":{"1min":0.68899999999999995,"5min":0.68899999999999995,"15min":0.68899999999999995},"max":{"1min":9.4589999999999996,"5min":9.4589999999999996,"15min":9.4589999999999996},"last":1.6579999999999999}]},{"osd":6,"last update":"Wed Aug 9 13:51:21 2023","interfaces":[{"interface":"back","average":{"1min":2.5259999999999998,"5min":1.5980000000000001,"15min":1.4610000000000001},"min":{"1min":0.51000000000000001,"5min":0.51000000000000001,"15min":0.51000000000000001},"max":{"1min":17.407,"5min":17.407,"15min":17.407},"last":1.5249999999999999},{"interface":"front","average":{"1min":2.351,"5min":1.6279999999999999,"15min":1.53},"min":{"1min":0.71199999999999997,"5min":0.71199999999999997,"15min":0.71199999999999997},"max":{"1min":16.356000000000002,"5min":16.356000000000002,"15min":16.356000000000002},"last":1.7350000000000001}]},{"osd":7,"last update":"Wed Aug 9 13:51:21 2023","interfaces":[{"interface":"back","average":{"1min":2.117,"5min":1.665,"15min":1.591},"min":{"1min":0.74299999999999999,"5min":0.74299999999999999,"15min":0.74299999999999999},"max":{"1min":9.9239999999999995,"5min":9.9239999999999995,"15min":9.9239999999999995},"last":1.482},{"interface":"front","average":{"1min":2.2400000000000002,"5min":1.716,"15min":1.645},"min":{"1min":0.65600000000000003,"5min":0.65600000000000003,"15min":0.65600000000000003},"max":{"1min":10.789,"5min":10.789,"15min":10.789},"last":1.6919999999999999}]}]},{"osd":4,"up_from":13,"seq":55834575064,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32488,"kb_used_data":1372,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":94339352,"statfs":{"total":96636764160,"available":96603496448,"internally_reserved":0,"allocated":1404928,"data_stored":748010,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1781,"internal_metadata":31848715},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Aug 9 13:51:23 2023","interfaces":[{"interface":"back","average":{"1min":1.131,"5min":1.2150000000000001,"15min":1.2070000000000001},"min":{"1min":0.66600000000000004,"5min":0.66600000000000004,"15min":0.66600000000000004},"max":{"1min":1.768,"5min":2.524,"15min":2.524},"last":1.349},{"interface":"front","average":{"1min":1.0609999999999999,"5min":1.181,"15min":1.25},"min":{"1min":0.66800000000000004,"5min":0.62,"15min":0.62},"max":{"1min":1.667,"5min":1.7390000000000001,"15min":1.863},"last":1.4770000000000001}]},{"osd":1,"last update":"Wed Aug 9 13:51:23 2023","interfaces":[{"interface":"back","average":{"1min":1.1839999999999999,"5min":1.264,"15min":1.2969999999999999},"min":{"1min":0.79700000000000004,"5min":0.67800000000000005,"15min":0.67800000000000005},"max":{"1min":1.609,"5min":1.8480000000000001,"15min":1.8480000000000001},"last":1.2},{"interface":"front","average":{"1min":1.1319999999999999,"5min":1.232,"15min":1.2470000000000001},"min":{"1min":0.75,"5min":0.75,"15min":0.65400000000000003},"max":{"1min":1.669,"5min":1.9330000000000001,"15min":1.9330000000000001},"last":1.153}]},{"osd":2,"last update":"Wed Aug 9 13:51:23 2023","interfaces":[{"interface":"back","average":{"1min":1.081,"5min":1.2509999999999999,"15min":1.272},"min":{"1min":0.69999999999999996,"5min":0.621,"15min":0.621},"max":{"1min":1.496,"5min":2.6429999999999998,"15min":2.6429999999999998},"last":1.522},{"interface":"front","average":{"1min":1.25,"5min":1.3220000000000001,"15min":1.365},"min":{"1min":0.76000000000000001,"5min":0.76000000000000001,"15min":0.75900000000000001},"max":{"1min":2.0910000000000002,"5min":2.0910000000000002,"15min":2.0910000000000002},"last":1.667}]},{"osd":3,"last update":"Wed Aug 9 13:51:23 2023","interfaces":[{"interface":"back","average":{"1min":1.1599999999999999,"5min":1.2889999999999999,"15min":1.298},"min":{"1min":0.754,"5min":0.754,"15min":0.754},"max":{"1min":1.921,"5min":2.6099999999999999,"15min":2.6099999999999999},"last":0.84499999999999997},{"interface":"front","average":{"1min":1.2509999999999999,"5min":1.3740000000000001,"15min":1.3819999999999999},"min":{"1min":0.89100000000000001,"5min":0.88800000000000001,"15min":0.63100000000000001},"max":{"1min":1.7010000000000001,"5min":2.7629999999999999,"15min":2.7629999999999999},"last":1.694}]},{"osd":5,"last update":"Wed Aug 9 13:51:23 2023","interfaces":[{"interface":"back","average":{"1min":1.294,"5min":1.3140000000000001,"15min":1.327},"min":{"1min":0.67300000000000004,"5min":0.67300000000000004,"15min":0.63200000000000001},"max":{"1min":1.758,"5min":1.8,"15min":2.0099999999999998},"last":1.73},{"interface":"front","average":{"1min":1.361,"5min":1.337,"15min":1.3169999999999999},"min":{"1min":0.65600000000000003,"5min":0.65600000000000003,"15min":0.626},"max":{"1min":2.0369999999999999,"5min":2.7090000000000001,"15min":2.7090000000000001},"last":1.298}]},{"osd":6,"last update":"Wed Aug 9 13:51:23 2023","interfaces":[{"interface":"back","average":{"1min":1.409,"5min":1.341,"15min":1.3340000000000001},"min":{"1min":0.90600000000000003,"5min":0.74399999999999999,"15min":0.59099999999999997},"max":{"1min":1.96,"5min":2.8559999999999999,"15min":2.8559999999999999},"last":1.758},{"interface":"front","average":{"1min":1.496,"5min":1.514,"15min":1.5029999999999999},"min":{"1min":0.878,"5min":0.878,"15min":0.76400000000000001},"max":{"1min":2.1890000000000001,"5min":2.2719999999999998,"15min":2.2719999999999998},"last":1.9410000000000001}]},{"osd":7,"last update":"Wed Aug 9 13:51:23 2023","interfaces":[{"interface":"back","average":{"1min":1.3859999999999999,"5min":1.468,"15min":1.4690000000000001},"min":{"1min":0.53100000000000003,"5min":0.53100000000000003,"15min":0.53100000000000003},"max":{"1min":1.9530000000000001,"5min":2.8050000000000002,"15min":2.8050000000000002},"last":1.5940000000000001},{"interface":"front","average":{"1min":1.5289999999999999,"5min":1.52,"15min":1.5129999999999999},"min":{"1min":0.78700000000000003,"5min":0.78700000000000003,"15min":0.72899999999999998},"max":{"1min":2.1419999999999999,"5min":3.415,"15min":3.415},"last":1.976}]}]},{"osd":3,"up_from":13,"seq":55834575065,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":36572,"kb_used_data":1360,"kb_used_omap":1,"kb_used_meta":35198,"kb_avail":94335268,"statfs":{"total":96636764160,"available":96599314432,"internally_reserved":0,"allocated":1392640,"data_stored":741018,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1781,"internal_metadata":36043019},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Aug 9 13:51:26 2023","interfaces":[{"interface":"back","average":{"1min":0.76900000000000002,"5min":0.85299999999999998,"15min":0.90500000000000003},"min":{"1min":0.496,"5min":0.27200000000000002,"15min":0.27200000000000002},"max":{"1min":1.5580000000000001,"5min":1.8500000000000001,"15min":1.968},"last":0.99099999999999999},{"interface":"front","average":{"1min":0.83499999999999996,"5min":1.0349999999999999,"15min":1.097},"min":{"1min":0.36299999999999999,"5min":0.36299999999999999,"15min":0.36299999999999999},"max":{"1min":1.4059999999999999,"5min":1.6910000000000001,"15min":2.0590000000000002},"last":1.3200000000000001}]},{"osd":1,"last update":"Wed Aug 9 13:51:26 2023","interfaces":[{"interface":"back","average":{"1min":0.82799999999999996,"5min":0.95399999999999996,"15min":1.0089999999999999},"min":{"1min":0.51500000000000001,"5min":0.496,"15min":0.438},"max":{"1min":1.27,"5min":1.6519999999999999,"15min":2.5649999999999999},"last":1.038},{"interface":"front","average":{"1min":0.97899999999999998,"5min":1.081,"15min":1.161},"min":{"1min":0.60499999999999998,"5min":0.51800000000000002,"15min":0.498},"max":{"1min":1.6779999999999999,"5min":1.6839999999999999,"15min":2.6760000000000002},"last":1.393}]},{"osd":2,"last update":"Wed Aug 9 13:51:26 2023","interfaces":[{"interface":"back","average":{"1min":0.995,"5min":1.115,"15min":1.194},"min":{"1min":0.42999999999999999,"5min":0.42999999999999999,"15min":0.42999999999999999},"max":{"1min":1.6879999999999999,"5min":1.913,"15min":1.923},"last":1.494},{"interface":"front","average":{"1min":1.103,"5min":1.2290000000000001,"15min":1.3069999999999999},"min":{"1min":0.61299999999999999,"5min":0.52300000000000002,"15min":0.52300000000000002},"max":{"1min":1.7290000000000001,"5min":1.9399999999999999,"15min":2.0259999999999998},"last":1.254}]},{"osd":4,"last update":"Wed Aug 9 13:51:26 2023","interfaces":[{"interface":"back","average":{"1min":1.1000000000000001,"5min":1.214,"15min":1.274},"min":{"1min":0.54900000000000004,"5min":0.54900000000000004,"15min":0.54900000000000004},"max":{"1min":1.732,"5min":1.8160000000000001,"15min":1.8160000000000001},"last":1.7070000000000001},{"interface":"front","average":{"1min":1.2769999999999999,"5min":1.3440000000000001,"15min":1.3959999999999999},"min":{"1min":0.76100000000000001,"5min":0.628,"15min":0.628},"max":{"1min":2.782,"5min":2.782,"15min":2.782},"last":1.7629999999999999}]},{"osd":5,"last update":"Wed Aug 9 13:51:26 2023","interfaces":[{"interface":"back","average":{"1min":1.266,"5min":1.4179999999999999,"15min":1.47},"min":{"1min":0.69899999999999995,"5min":0.67600000000000005,"15min":0.67200000000000004},"max":{"1min":1.833,"5min":2.254,"15min":2.254},"last":2.012},{"interface":"front","average":{"1min":1.258,"5min":1.431,"15min":1.484},"min":{"1min":0.66000000000000003,"5min":0.65900000000000003,"15min":0.65900000000000003},"max":{"1min":1.919,"5min":2.3069999999999999,"15min":2.3069999999999999},"last":1.6060000000000001}]},{"osd":6,"last update":"Wed Aug 9 13:51:26 2023","interfaces":[{"interface":"back","average":{"1min":1.381,"5min":1.419,"15min":1.4630000000000001},"min":{"1min":0.78200000000000003,"5min":0.68200000000000005,"15min":0.68200000000000005},"max":{"1min":2.8340000000000001,"5min":2.8340000000000001,"15min":2.8340000000000001},"last":1.5620000000000001},{"interface":"front","average":{"1min":1.389,"5min":1.5109999999999999,"15min":1.5580000000000001},"min":{"1min":0.64100000000000001,"5min":0.64100000000000001,"15min":0.64100000000000001},"max":{"1min":2.9060000000000001,"5min":2.9060000000000001,"15min":2.9060000000000001},"last":2.1480000000000001}]},{"osd":7,"last update":"Wed Aug 9 13:51:26 2023","interfaces":[{"interface":"back","average":{"1min":1.3580000000000001,"5min":1.5549999999999999,"15min":1.6100000000000001},"min":{"1min":0.77800000000000002,"5min":0.69399999999999995,"15min":0.69399999999999995},"max":{"1min":2.0419999999999998,"5min":2.46,"15min":2.46},"last":1.966},{"interface":"front","average":{"1min":1.403,"5min":1.5049999999999999,"15min":1.5620000000000001},"min":{"1min":0.93799999999999994,"5min":0.64200000000000002,"15min":0.64200000000000002},"max":{"1min":2.7200000000000002,"5min":2.7200000000000002,"15min":2.7200000000000002},"last":1.804}]}]},{"osd":2,"up_from":13,"seq":55834575064,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":36572,"kb_used_data":1360,"kb_used_omap":1,"kb_used_meta":35198,"kb_avail":94335268,"statfs":{"total":96636764160,"available":96599314432,"internally_reserved":0,"allocated":1392640,"data_stored":741018,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1781,"internal_metadata":36043019},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Aug 9 13:51:19 2023","interfaces":[{"interface":"back","average":{"1min":0.82699999999999996,"5min":0.91400000000000003,"15min":0.95599999999999996},"min":{"1min":0.47199999999999998,"5min":0.36499999999999999,"15min":0.36499999999999999},"max":{"1min":1.484,"5min":1.5940000000000001,"15min":1.8380000000000001},"last":0.749},{"interface":"front","average":{"1min":0.90200000000000002,"5min":0.96699999999999997,"15min":1.0009999999999999},"min":{"1min":0.38200000000000001,"5min":0.374,"15min":0.374},"max":{"1min":1.7,"5min":2.3929999999999998,"15min":2.3929999999999998},"last":0.995}]},{"osd":1,"last update":"Wed Aug 9 13:51:19 2023","interfaces":[{"interface":"back","average":{"1min":1.038,"5min":1.143,"15min":1.1839999999999999},"min":{"1min":0.39400000000000002,"5min":0.39400000000000002,"15min":0.39400000000000002},"max":{"1min":1.615,"5min":1.77,"15min":1.8420000000000001},"last":1.149},{"interface":"front","average":{"1min":0.98399999999999999,"5min":1.171,"15min":1.3220000000000001},"min":{"1min":0.435,"5min":0.435,"15min":0.435},"max":{"1min":1.659,"5min":2.4870000000000001,"15min":2.4870000000000001},"last":1.1879999999999999}]},{"osd":3,"last update":"Wed Aug 9 13:51:19 2023","interfaces":[{"interface":"back","average":{"1min":0.93100000000000005,"5min":0.97999999999999998,"15min":1.0509999999999999},"min":{"1min":0.55300000000000005,"5min":0.38900000000000001,"15min":0.38900000000000001},"max":{"1min":1.417,"5min":1.708,"15min":2.0409999999999999},"last":0.93400000000000005},{"interface":"front","average":{"1min":1.0820000000000001,"5min":1.2549999999999999,"15min":1.3080000000000001},"min":{"1min":0.52900000000000003,"5min":0.46200000000000002,"15min":0.46200000000000002},"max":{"1min":1.6279999999999999,"5min":8.8130000000000006,"15min":8.8130000000000006},"last":1.304}]},{"osd":4,"last update":"Wed Aug 9 13:51:19 2023","interfaces":[{"interface":"back","average":{"1min":1.335,"5min":1.3740000000000001,"15min":1.452},"min":{"1min":0.81000000000000005,"5min":0.628,"15min":0.628},"max":{"1min":1.8340000000000001,"5min":2.089,"15min":2.089},"last":1.5109999999999999},{"interface":"front","average":{"1min":1.2969999999999999,"5min":1.3360000000000001,"15min":1.3939999999999999},"min":{"1min":0.75900000000000001,"5min":0.68600000000000005,"15min":0.68600000000000005},"max":{"1min":1.7509999999999999,"5min":2.472,"15min":2.472},"last":1.5469999999999999}]},{"osd":5,"last update":"Wed Aug 9 13:51:19 2023","interfaces":[{"interface":"back","average":{"1min":1.361,"5min":1.4059999999999999,"15min":1.4590000000000001},"min":{"1min":0.86399999999999999,"5min":0.59999999999999998,"15min":0.59999999999999998},"max":{"1min":2.0640000000000001,"5min":2.5840000000000001,"15min":2.5840000000000001},"last":1.383},{"interface":"front","average":{"1min":1.2829999999999999,"5min":1.3260000000000001,"15min":1.351},"min":{"1min":0.89000000000000001,"5min":0.56799999999999995,"15min":0.56799999999999995},"max":{"1min":1.78,"5min":3.5659999999999998,"15min":3.5659999999999998},"last":1.2370000000000001}]},{"osd":6,"last update":"Wed Aug 9 13:51:19 2023","interfaces":[{"interface":"back","average":{"1min":1.3009999999999999,"5min":1.3879999999999999,"15min":1.429},"min":{"1min":0.82199999999999995,"5min":0.73799999999999999,"15min":0.73799999999999999},"max":{"1min":2.1200000000000001,"5min":3.5099999999999998,"15min":3.5099999999999998},"last":1.3540000000000001},{"interface":"front","average":{"1min":1.4099999999999999,"5min":1.458,"15min":1.54},"min":{"1min":0.84899999999999998,"5min":0.66300000000000003,"15min":0.66300000000000003},"max":{"1min":2.04,"5min":2.3769999999999998,"15min":2.3769999999999998},"last":1.4279999999999999}]},{"osd":7,"last update":"Wed Aug 9 13:51:19 2023","interfaces":[{"interface":"back","average":{"1min":1.3620000000000001,"5min":1.452,"15min":1.54},"min":{"1min":0.85099999999999998,"5min":0.64600000000000002,"15min":0.64600000000000002},"max":{"1min":1.865,"5min":2.5249999999999999,"15min":2.5249999999999999},"last":1.6200000000000001},{"interface":"front","average":{"1min":1.3939999999999999,"5min":1.476,"15min":1.514},"min":{"1min":0.94699999999999995,"5min":0.70499999999999996,"15min":0.70499999999999996},"max":{"1min":1.891,"5min":3.609,"15min":3.609},"last":1.4139999999999999}]}]},{"osd":1,"up_from":13,"seq":55834575064,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":47008,"kb_used_data":1940,"kb_used_omap":1,"kb_used_meta":45054,"kb_avail":94324832,"statfs":{"total":96636764160,"available":96588627968,"internally_reserved":0,"allocated":1986560,"data_stored":1331386,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1781,"internal_metadata":46135563},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Aug 9 13:51:34 2023","interfaces":[{"interface":"back","average":{"1min":0.77600000000000002,"5min":0.90000000000000002,"15min":0.93799999999999994},"min":{"1min":0.45800000000000002,"5min":0.45800000000000002,"15min":0.45800000000000002},"max":{"1min":1.1910000000000001,"5min":1.7110000000000001,"15min":2.1080000000000001},"last":0.57599999999999996},{"interface":"front","average":{"1min":0.83299999999999996,"5min":1.0129999999999999,"15min":1.095},"min":{"1min":0.60599999999999998,"5min":0.432,"15min":0.432},"max":{"1min":1.6519999999999999,"5min":2.468,"15min":2.468},"last":0.40699999999999997}]},{"osd":2,"last update":"Wed Aug 9 13:51:34 2023","interfaces":[{"interface":"back","average":{"1min":0.83399999999999996,"5min":0.93799999999999994,"15min":0.99299999999999999},"min":{"1min":0.57499999999999996,"5min":0.41099999999999998,"15min":0.41099999999999998},"max":{"1min":1.2130000000000001,"5min":1.907,"15min":1.907},"last":0.52200000000000002},{"interface":"front","average":{"1min":0.91100000000000003,"5min":1.0920000000000001,"15min":1.1859999999999999},"min":{"1min":0.59699999999999998,"5min":0.54400000000000004,"15min":0.54400000000000004},"max":{"1min":1.45,"5min":2.1280000000000001,"15min":2.145},"last":0.50700000000000001}]},{"osd":3,"last update":"Wed Aug 9 13:51:34 2023","interfaces":[{"interface":"back","average":{"1min":0.97199999999999998,"5min":1.1890000000000001,"15min":1.3220000000000001},"min":{"1min":0.60799999999999998,"5min":0.54400000000000004,"15min":0.54400000000000004},"max":{"1min":1.466,"5min":2.0270000000000001,"15min":2.3210000000000002},"last":0.54600000000000004},{"interface":"front","average":{"1min":1.042,"5min":1.163,"15min":1.25},"min":{"1min":0.625,"5min":0.49299999999999999,"15min":0.45300000000000001},"max":{"1min":1.6539999999999999,"5min":2.3540000000000001,"15min":2.7669999999999999},"last":0.48299999999999998}]},{"osd":4,"last update":"Wed Aug 9 13:51:34 2023","interfaces":[{"interface":"back","average":{"1min":1.1819999999999999,"5min":1.337,"15min":1.4059999999999999},"min":{"1min":0.79800000000000004,"5min":0.79800000000000004,"15min":0.58099999999999996},"max":{"1min":1.9259999999999999,"5min":1.97,"15min":2.5},"last":0.91100000000000003},{"interface":"front","average":{"1min":1.157,"5min":1.27,"15min":1.329},"min":{"1min":0.76600000000000001,"5min":0.70099999999999996,"15min":0.70099999999999996},"max":{"1min":1.895,"5min":1.895,"15min":1.901},"last":0.71099999999999997}]},{"osd":5,"last update":"Wed Aug 9 13:51:34 2023","interfaces":[{"interface":"back","average":{"1min":1.2150000000000001,"5min":1.29,"15min":1.3680000000000001},"min":{"1min":0.76800000000000002,"5min":0.76800000000000002,"15min":0.76800000000000002},"max":{"1min":1.73,"5min":2.294,"15min":2.294},"last":0.75900000000000001},{"interface":"front","average":{"1min":1.2949999999999999,"5min":1.3999999999999999,"15min":1.512},"min":{"1min":0.89200000000000002,"5min":0.89200000000000002,"15min":0.89200000000000002},"max":{"1min":1.6859999999999999,"5min":2.1389999999999998,"15min":2.4649999999999999},"last":0.78800000000000003}]},{"osd":6,"last update":"Wed Aug 9 13:51:34 2023","interfaces":[{"interface":"back","average":{"1min":1.325,"5min":1.4339999999999999,"15min":1.5149999999999999},"min":{"1min":0.96599999999999997,"5min":0.77500000000000002,"15min":0.67400000000000004},"max":{"1min":1.5780000000000001,"5min":2.2080000000000002,"15min":2.5270000000000001},"last":0.97599999999999998},{"interface":"front","average":{"1min":1.28,"5min":1.3979999999999999,"15min":1.4990000000000001},"min":{"1min":0.73599999999999999,"5min":0.73599999999999999,"15min":0.73599999999999999},"max":{"1min":1.591,"5min":2.02,"15min":2.1040000000000001},"last":0.80600000000000005}]},{"osd":7,"last update":"Wed Aug 9 13:51:34 2023","interfaces":[{"interface":"back","average":{"1min":1.3240000000000001,"5min":1.4930000000000001,"15min":1.615},"min":{"1min":0.93700000000000006,"5min":0.86899999999999999,"15min":0.86899999999999999},"max":{"1min":1.776,"5min":2.4169999999999998,"15min":2.5790000000000002},"last":0.81899999999999995},{"interface":"front","average":{"1min":1.3420000000000001,"5min":1.488,"15min":1.5980000000000001},"min":{"1min":0.96899999999999997,"5min":0.91300000000000003,"15min":0.89500000000000002},"max":{"1min":1.8320000000000001,"5min":2.1040000000000001,"15min":2.6509999999999998},"last":1.083}]}]},{"osd":0,"up_from":13,"seq":55834575064,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":40664,"kb_used_data":1356,"kb_used_omap":1,"kb_used_meta":39294,"kb_avail":94331176,"statfs":{"total":96636764160,"available":96595124224,"internally_reserved":0,"allocated":1388544,"data_stored":739657,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1781,"internal_metadata":40237323},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Wed Aug 9 13:51:27 2023","interfaces":[{"interface":"back","average":{"1min":0.95899999999999996,"5min":0.91300000000000003,"15min":0.92100000000000004},"min":{"1min":0.41199999999999998,"5min":0.32500000000000001,"15min":0.32500000000000001},"max":{"1min":1.4990000000000001,"5min":1.756,"15min":1.756},"last":1.528},{"interface":"front","average":{"1min":1.002,"5min":1.077,"15min":1.111},"min":{"1min":0.39200000000000002,"5min":0.39200000000000002,"15min":0.39200000000000002},"max":{"1min":1.546,"5min":1.9019999999999999,"15min":1.9019999999999999},"last":0.81999999999999995}]},{"osd":2,"last update":"Wed Aug 9 13:51:27 2023","interfaces":[{"interface":"back","average":{"1min":1,"5min":1.0680000000000001,"15min":1.1339999999999999},"min":{"1min":0.58399999999999996,"5min":0.51400000000000001,"15min":0.503},"max":{"1min":1.5089999999999999,"5min":1.776,"15min":1.889},"last":1.1639999999999999},{"interface":"front","average":{"1min":0.98799999999999999,"5min":1.1020000000000001,"15min":1.1739999999999999},"min":{"1min":0.439,"5min":0.439,"15min":0.439},"max":{"1min":1.6259999999999999,"5min":2.0129999999999999,"15min":2.0129999999999999},"last":1.704}]},{"osd":3,"last update":"Wed Aug 9 13:51:27 2023","interfaces":[{"interface":"back","average":{"1min":1,"5min":1.0349999999999999,"15min":1.101},"min":{"1min":0.50900000000000001,"5min":0.46300000000000002,"15min":0.46300000000000002},"max":{"1min":1.5189999999999999,"5min":1.823,"15min":1.8919999999999999},"last":0.67800000000000005},{"interface":"front","average":{"1min":1.2390000000000001,"5min":1.252,"15min":1.3140000000000001},"min":{"1min":0.63400000000000001,"5min":0.54700000000000004,"15min":0.54700000000000004},"max":{"1min":2.2050000000000001,"5min":2.347,"15min":2.4649999999999999},"last":1.022}]},{"osd":4,"last update":"Wed Aug 9 13:51:27 2023","interfaces":[{"interface":"back","average":{"1min":1.2509999999999999,"5min":1.2949999999999999,"15min":1.343},"min":{"1min":0.73899999999999999,"5min":0.73899999999999999,"15min":0.73899999999999999},"max":{"1min":1.581,"5min":2.2309999999999999,"15min":2.2309999999999999},"last":1.2150000000000001},{"interface":"front","average":{"1min":1.409,"5min":1.417,"15min":1.456},"min":{"1min":0.86399999999999999,"5min":0.77600000000000002,"15min":0.77600000000000002},"max":{"1min":2.3540000000000001,"5min":2.516,"15min":2.5390000000000001},"last":1.395}]},{"osd":5,"last update":"Wed Aug 9 13:51:27 2023","interfaces":[{"interface":"back","average":{"1min":1.3859999999999999,"5min":1.4019999999999999,"15min":1.462},"min":{"1min":0.875,"5min":0.66300000000000003,"15min":0.66300000000000003},"max":{"1min":1.764,"5min":2.6669999999999998,"15min":2.6669999999999998},"last":1.1020000000000001},{"interface":"front","average":{"1min":1.333,"5min":1.3859999999999999,"15min":1.4470000000000001},"min":{"1min":0.88400000000000001,"5min":0.629,"15min":0.629},"max":{"1min":1.645,"5min":2.1499999999999999,"15min":2.1499999999999999},"last":0.96099999999999997}]},{"osd":6,"last update":"Wed Aug 9 13:51:27 2023","interfaces":[{"interface":"back","average":{"1min":1.4870000000000001,"5min":1.5089999999999999,"15min":1.548},"min":{"1min":1.014,"5min":0.73599999999999999,"15min":0.73599999999999999},"max":{"1min":2.4670000000000001,"5min":2.4670000000000001,"15min":2.7250000000000001},"last":1.6319999999999999},{"interface":"front","average":{"1min":1.486,"5min":1.526,"15min":1.5569999999999999},"min":{"1min":0.95999999999999996,"5min":0.86699999999999999,"15min":0.86699999999999999},"max":{"1min":1.988,"5min":2.399,"15min":2.399},"last":1.573}]},{"osd":7,"last update":"Wed Aug 9 13:51:27 2023","interfaces":[{"interface":"back","average":{"1min":1.552,"5min":1.5720000000000001,"15min":1.621},"min":{"1min":0.89200000000000002,"5min":0.72299999999999998,"15min":0.72299999999999998},"max":{"1min":2.0880000000000001,"5min":2.3900000000000001,"15min":2.3900000000000001},"last":1.2490000000000001},{"interface":"front","average":{"1min":1.4179999999999999,"5min":1.484,"15min":1.54},"min":{"1min":0.84099999999999997,"5min":0.76700000000000002,"15min":0.76700000000000002},"max":{"1min":1.7969999999999999,"5min":2.274,"15min":2.274},"last":1.4890000000000001}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-08-09T13:52:04.723 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-08-09T13:52:04.996 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:52:04.994+0000 7f2a5805e640 1 Processor -- start 2023-08-09T13:52:04.996 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:52:04.994+0000 7f2a5805e640 1 -- start start 2023-08-09T13:52:04.998 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:52:04.996+0000 7f2a5805e640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f2a50143c20 0x7f2a5006f540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:52:04.999 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:52:04.996+0000 7f2a5805e640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f2a5006cfe0 0x7f2a500c8b50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:52:04.999 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:52:04.996+0000 7f2a5805e640 1 -- --> v1:172.21.15.203:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2a500c9090 con 0x7f2a500c86f0 2023-08-09T13:52:04.999 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:52:04.996+0000 7f2a5805e640 1 -- --> v2:172.21.15.43:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2a50144000 con 0x7f2a50143c20 2023-08-09T13:52:04.999 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:52:04.996+0000 7f2a5805e640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2a5006c440 con 0x7f2a5006cfe0 2023-08-09T13:52:05.000 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:52:04.996+0000 7f2a55dfa640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f2a50143c20 0x7f2a5006f540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:52:05.000 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:52:04.996+0000 7f2a55dfa640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f2a50143c20 0x7f2a5006f540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:54278/0 (socket says 172.21.15.43:54278) 2023-08-09T13:52:05.000 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:52:04.996+0000 7f2a55dfa640 1 -- 172.21.15.43:0/1607567552 learned_addr learned my addr 172.21.15.43:0/1607567552 (peer_addr_for_me v2:172.21.15.43:0/0) 2023-08-09T13:52:05.001 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:52:04.997+0000 7f2a565fb640 1 --2- 172.21.15.43:0/1607567552 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f2a5006cfe0 0x7f2a500c8b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:52:05.001 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:52:04.997+0000 7f2a55dfa640 1 -- 172.21.15.43:0/1607567552 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f2a5006cfe0 msgr2=0x7f2a500c8b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-09T13:52:05.001 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:52:04.997+0000 7f2a55dfa640 1 --2- 172.21.15.43:0/1607567552 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f2a5006cfe0 0x7f2a500c8b50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:52:05.001 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:52:04.997+0000 7f2a55dfa640 1 -- 172.21.15.43:0/1607567552 >> v1:172.21.15.203:6789/0 conn(0x7f2a500c86f0 legacy=0x7f2a5006c8d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-09T13:52:05.001 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:52:04.997+0000 7f2a55dfa640 1 -- 172.21.15.43:0/1607567552 --> v2:172.21.15.43:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2a5006b830 con 0x7f2a50143c20 2023-08-09T13:52:05.002 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:52:04.998+0000 7f2a55dfa640 1 --2- 172.21.15.43:0/1607567552 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f2a50143c20 0x7f2a5006f540 secure :-1 s=READY pgs=1348 cs=0 l=1 rev1=1 crypto rx=0x7f2a4c009870 tx=0x7f2a4c02b860 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d8a7703bced89411 server_cookie=0 in_seq=0 out_seq=0 2023-08-09T13:52:05.003 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:52:04.998+0000 7f2a54df8640 1 -- 172.21.15.43:0/1607567552 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2a4c0317c0 con 0x7f2a50143c20 2023-08-09T13:52:05.003 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:52:04.998+0000 7f2a54df8640 1 -- 172.21.15.43:0/1607567552 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2a4c031920 con 0x7f2a50143c20 2023-08-09T13:52:05.003 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:52:04.998+0000 7f2a54df8640 1 -- 172.21.15.43:0/1607567552 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2a4c031cb0 con 0x7f2a50143c20 2023-08-09T13:52:05.003 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:52:04.999+0000 7f2a5805e640 1 -- 172.21.15.43:0/1607567552 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f2a50143c20 msgr2=0x7f2a5006f540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:52:05.003 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:52:04.999+0000 7f2a5805e640 1 --2- 172.21.15.43:0/1607567552 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f2a50143c20 0x7f2a5006f540 secure :-1 s=READY pgs=1348 cs=0 l=1 rev1=1 crypto rx=0x7f2a4c009870 tx=0x7f2a4c02b860 comp rx=0 tx=0).stop 2023-08-09T13:52:05.004 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:52:04.999+0000 7f2a5805e640 1 -- 172.21.15.43:0/1607567552 shutdown_connections 2023-08-09T13:52:05.004 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:52:04.999+0000 7f2a5805e640 1 --2- 172.21.15.43:0/1607567552 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f2a5006cfe0 0x7f2a500c8b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:52:05.004 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:52:04.999+0000 7f2a5805e640 1 --2- 172.21.15.43:0/1607567552 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f2a50143c20 0x7f2a5006f540 unknown :-1 s=CLOSED pgs=1348 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:52:05.004 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:52:04.999+0000 7f2a5805e640 1 -- 172.21.15.43:0/1607567552 >> 172.21.15.43:0/1607567552 conn(0x7f2a500ce0a0 msgr2=0x7f2a500ce480 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-09T13:52:05.004 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:52:04.999+0000 7f2a5805e640 1 -- 172.21.15.43:0/1607567552 shutdown_connections 2023-08-09T13:52:05.005 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:52:04.999+0000 7f2a5805e640 1 -- 172.21.15.43:0/1607567552 wait complete. 2023-08-09T13:52:05.005 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:52:05.001+0000 7f2a5805e640 1 Processor -- start 2023-08-09T13:52:05.005 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:52:05.001+0000 7f2a5805e640 1 -- start start 2023-08-09T13:52:05.006 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:52:05.002+0000 7f2a5805e640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f2a5006cfe0 0x7f2a501598a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:52:05.007 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:52:05.002+0000 7f2a5805e640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f2a500c86f0 0x7f2a5015cc30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:52:05.007 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:52:05.002+0000 7f2a5805e640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f2a50143c20 0x7f2a500b30e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:52:05.007 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:52:05.002+0000 7f2a5805e640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2a5015c7a0 con 0x7f2a5006cfe0 2023-08-09T13:52:05.007 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:52:05.002+0000 7f2a5805e640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2a5015be70 con 0x7f2a500c86f0 2023-08-09T13:52:05.007 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:52:05.002+0000 7f2a5805e640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2a5015c150 con 0x7f2a50143c20 2023-08-09T13:52:05.007 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:52:05.003+0000 7f2a55dfa640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f2a5006cfe0 0x7f2a501598a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:52:05.008 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:52:05.003+0000 7f2a55dfa640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f2a5006cfe0 0x7f2a501598a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:54290/0 (socket says 172.21.15.43:54290) 2023-08-09T13:52:05.008 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:52:05.003+0000 7f2a55dfa640 1 -- 172.21.15.43:0/2698037561 learned_addr learned my addr 172.21.15.43:0/2698037561 (peer_addr_for_me v2:172.21.15.43:0/0) 2023-08-09T13:52:05.008 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:52:05.003+0000 7f2a565fb640 1 --2- 172.21.15.43:0/2698037561 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f2a50143c20 0x7f2a500b30e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:52:05.008 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:52:05.003+0000 7f2a555f9640 1 --2- 172.21.15.43:0/2698037561 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f2a500c86f0 0x7f2a5015cc30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:52:05.008 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:52:05.003+0000 7f2a55dfa640 1 -- 172.21.15.43:0/2698037561 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f2a50143c20 msgr2=0x7f2a500b30e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:52:05.008 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:52:05.003+0000 7f2a55dfa640 1 --2- 172.21.15.43:0/2698037561 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f2a50143c20 0x7f2a500b30e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:52:05.008 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:52:05.003+0000 7f2a55dfa640 1 -- 172.21.15.43:0/2698037561 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f2a500c86f0 msgr2=0x7f2a5015cc30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:52:05.009 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:52:05.003+0000 7f2a55dfa640 1 --2- 172.21.15.43:0/2698037561 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f2a500c86f0 0x7f2a5015cc30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:52:05.009 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:52:05.003+0000 7f2a55dfa640 1 -- 172.21.15.43:0/2698037561 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2a4c0094d0 con 0x7f2a5006cfe0 2023-08-09T13:52:05.009 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:52:05.003+0000 7f2a55dfa640 1 --2- 172.21.15.43:0/2698037561 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f2a5006cfe0 0x7f2a501598a0 secure :-1 s=READY pgs=1349 cs=0 l=1 rev1=1 crypto rx=0x7f2a4c000c00 tx=0x7f2a4c0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-09T13:52:05.009 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:52:05.004+0000 7f2a467fc640 1 -- 172.21.15.43:0/2698037561 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2a4c0329a0 con 0x7f2a5006cfe0 2023-08-09T13:52:05.009 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:52:05.004+0000 7f2a467fc640 1 -- 172.21.15.43:0/2698037561 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2a4c032b20 con 0x7f2a5006cfe0 2023-08-09T13:52:05.009 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:52:05.004+0000 7f2a467fc640 1 -- 172.21.15.43:0/2698037561 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2a4c0453f0 con 0x7f2a5006cfe0 2023-08-09T13:52:05.010 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:52:05.004+0000 7f2a5805e640 1 -- 172.21.15.43:0/2698037561 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2a5015c430 con 0x7f2a5006cfe0 2023-08-09T13:52:05.010 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:52:05.004+0000 7f2a5805e640 1 -- 172.21.15.43:0/2698037561 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2a5015a310 con 0x7f2a5006cfe0 2023-08-09T13:52:05.010 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:52:05.005+0000 7f2a5805e640 1 -- 172.21.15.43:0/2698037561 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2a500739f0 con 0x7f2a5006cfe0 2023-08-09T13:52:05.011 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:52:05.006+0000 7f2a467fc640 1 -- 172.21.15.43:0/2698037561 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 45) v1 ==== 88443+0+0 (secure 0 0 0) 0x7f2a4c04e450 con 0x7f2a5006cfe0 2023-08-09T13:52:05.011 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:52:05.007+0000 7f2a467fc640 1 --2- 172.21.15.43:0/2698037561 >> [v2:172.21.15.203:6832/1317108840,v1:172.21.15.203:6833/1317108840] conn(0x7f2a3406a170 0x7f2a3406c630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:52:05.011 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:52:05.007+0000 7f2a467fc640 1 -- 172.21.15.43:0/2698037561 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(116..116 src has 1..116) v4 ==== 6426+0+0 (secure 0 0 0) 0x7f2a4c0aacd0 con 0x7f2a5006cfe0 2023-08-09T13:52:05.011 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:52:05.008+0000 7f2a555f9640 1 --2- 172.21.15.43:0/2698037561 >> [v2:172.21.15.203:6832/1317108840,v1:172.21.15.203:6833/1317108840] conn(0x7f2a3406a170 0x7f2a3406c630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:52:05.011 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:52:05.009+0000 7f2a555f9640 1 --2- 172.21.15.43:0/2698037561 >> [v2:172.21.15.203:6832/1317108840,v1:172.21.15.203:6833/1317108840] conn(0x7f2a3406a170 0x7f2a3406c630 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f2a501402d0 tx=0x7f2a40009040 comp rx=0 tx=0).ready entity=mgr.6619 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-09T13:52:05.012 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:52:05.011+0000 7f2a467fc640 1 -- 172.21.15.43:0/2698037561 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178066 (secure 0 0 0) 0x7f2a4c0760a0 con 0x7f2a5006cfe0 2023-08-09T13:52:05.200 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:52:05.199+0000 7f2a5805e640 1 -- 172.21.15.43:0/2698037561 --> [v2:172.21.15.203:6832/1317108840,v1:172.21.15.203:6833/1317108840] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f2a5006bdf0 con 0x7f2a3406a170 2023-08-09T13:52:05.223 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:52:05.222+0000 7f2a467fc640 1 -- 172.21.15.43:0/2698037561 <== mgr.6619 v2:172.21.15.203:6832/1317108840 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+304014 (secure 0 0 0) 0x7f2a400053f0 con 0x7f2a3406a170 2023-08-09T13:52:05.227 INFO:teuthology.orchestra.run.smithi043.stdout: 2023-08-09T13:52:05.228 INFO:teuthology.orchestra.run.smithi043.stderr:dumped all 2023-08-09T13:52:05.229 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:52:05.227+0000 7f2a5805e640 1 -- 172.21.15.43:0/2698037561 >> [v2:172.21.15.203:6832/1317108840,v1:172.21.15.203:6833/1317108840] conn(0x7f2a3406a170 msgr2=0x7f2a3406c630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:52:05.229 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:52:05.227+0000 7f2a5805e640 1 --2- 172.21.15.43:0/2698037561 >> [v2:172.21.15.203:6832/1317108840,v1:172.21.15.203:6833/1317108840] conn(0x7f2a3406a170 0x7f2a3406c630 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f2a501402d0 tx=0x7f2a40009040 comp rx=0 tx=0).stop 2023-08-09T13:52:05.230 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:52:05.227+0000 7f2a5805e640 1 -- 172.21.15.43:0/2698037561 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f2a5006cfe0 msgr2=0x7f2a501598a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:52:05.230 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:52:05.227+0000 7f2a5805e640 1 --2- 172.21.15.43:0/2698037561 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f2a5006cfe0 0x7f2a501598a0 secure :-1 s=READY pgs=1349 cs=0 l=1 rev1=1 crypto rx=0x7f2a4c000c00 tx=0x7f2a4c0046c0 comp rx=0 tx=0).stop 2023-08-09T13:52:05.230 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:52:05.228+0000 7f2a5805e640 1 -- 172.21.15.43:0/2698037561 shutdown_connections 2023-08-09T13:52:05.230 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:52:05.228+0000 7f2a5805e640 1 --2- 172.21.15.43:0/2698037561 >> [v2:172.21.15.203:6832/1317108840,v1:172.21.15.203:6833/1317108840] conn(0x7f2a3406a170 0x7f2a3406c630 unknown :-1 s=CLOSED pgs=66 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:52:05.230 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:52:05.228+0000 7f2a5805e640 1 --2- 172.21.15.43:0/2698037561 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f2a50143c20 0x7f2a500b30e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:52:05.230 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:52:05.228+0000 7f2a5805e640 1 --2- 172.21.15.43:0/2698037561 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f2a500c86f0 0x7f2a5015cc30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:52:05.230 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:52:05.228+0000 7f2a5805e640 1 --2- 172.21.15.43:0/2698037561 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f2a5006cfe0 0x7f2a501598a0 unknown :-1 s=CLOSED pgs=1349 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:52:05.231 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:52:05.228+0000 7f2a5805e640 1 -- 172.21.15.43:0/2698037561 >> 172.21.15.43:0/2698037561 conn(0x7f2a500ce0a0 msgr2=0x7f2a500adc20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-09T13:52:05.231 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:52:05.228+0000 7f2a5805e640 1 -- 172.21.15.43:0/2698037561 shutdown_connections 2023-08-09T13:52:05.231 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:52:05.228+0000 7f2a5805e640 1 -- 172.21.15.43:0/2698037561 wait complete. 2023-08-09T13:52:05.252 INFO:teuthology.orchestra.run.smithi043.stdout:{"pg_ready":true,"pg_map":{"version":27,"stamp":"2023-08-09T13:52:04.571627+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":715,"num_read_kb":1418,"num_write":283,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":186,"ondisk_log_size":186,"up":210,"acting":210,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":23,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":307720,"kb_used_data":12068,"kb_used_omap":13,"kb_used_meta":295538,"kb_avail":754667000,"statfs":{"total":773094113280,"available":772779008000,"internally_reserved":0,"allocated":12357632,"data_stored":7127134,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":14248,"internal_metadata":302631000},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":37,"apply_latency_ms":37,"commit_latency_ns":37000000,"apply_latency_ns":37000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.001865"},"pg_stats":[{"pgid":"15.3f","version":"0'0","reported_seq":1,"reported_epoch":114,"state":"creating+peering","last_fresh":"2023-08-09T13:51:59.173066+0000","last_change":"2023-08-09T13:51:59.173066+0000","last_active":"2023-08-09T13:51:59.130555+0000","last_peered":"2023-08-09T13:51:59.130555+0000","last_clean":"2023-08-09T13:51:59.130555+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-09T13:51:59.173066+0000","last_undegraded":"2023-08-09T13:51:59.173066+0000","last_fullsized":"2023-08-09T13:51:59.173066+0000","mapping_epoch":114,"log_start":"0'0","ondisk_log_start":"0'0","created":114,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:51:59.130555+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:51:59.130555+0000","last_clean_scrub_stamp":"2023-08-09T13:51:59.130555+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"15.3e","version":"0'0","reported_seq":1,"reported_epoch":114,"state":"creating+peering","last_fresh":"2023-08-09T13:51:59.161823+0000","last_change":"2023-08-09T13:51:59.161823+0000","last_active":"2023-08-09T13:51:59.130555+0000","last_peered":"2023-08-09T13:51:59.130555+0000","last_clean":"2023-08-09T13:51:59.130555+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-09T13:51:59.161823+0000","last_undegraded":"2023-08-09T13:51:59.161823+0000","last_fullsized":"2023-08-09T13:51:59.161823+0000","mapping_epoch":114,"log_start":"0'0","ondisk_log_start":"0'0","created":114,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:51:59.130555+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:51:59.130555+0000","last_clean_scrub_stamp":"2023-08-09T13:51:59.130555+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.3d","version":"0'0","reported_seq":1,"reported_epoch":114,"state":"creating+peering","last_fresh":"2023-08-09T13:51:59.169470+0000","last_change":"2023-08-09T13:51:59.169470+0000","last_active":"2023-08-09T13:51:59.130555+0000","last_peered":"2023-08-09T13:51:59.130555+0000","last_clean":"2023-08-09T13:51:59.130555+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-09T13:51:59.169470+0000","last_undegraded":"2023-08-09T13:51:59.169470+0000","last_fullsized":"2023-08-09T13:51:59.169470+0000","mapping_epoch":114,"log_start":"0'0","ondisk_log_start":"0'0","created":114,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:51:59.130555+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:51:59.130555+0000","last_clean_scrub_stamp":"2023-08-09T13:51:59.130555+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"15.3c","version":"0'0","reported_seq":1,"reported_epoch":114,"state":"creating+peering","last_fresh":"2023-08-09T13:51:59.150559+0000","last_change":"2023-08-09T13:51:59.150559+0000","last_active":"2023-08-09T13:51:59.130555+0000","last_peered":"2023-08-09T13:51:59.130555+0000","last_clean":"2023-08-09T13:51:59.130555+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-09T13:51:59.150559+0000","last_undegraded":"2023-08-09T13:51:59.150559+0000","last_fullsized":"2023-08-09T13:51:59.150559+0000","mapping_epoch":114,"log_start":"0'0","ondisk_log_start":"0'0","created":114,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:51:59.130555+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:51:59.130555+0000","last_clean_scrub_stamp":"2023-08-09T13:51:59.130555+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"15.3b","version":"0'0","reported_seq":1,"reported_epoch":114,"state":"creating+peering","last_fresh":"2023-08-09T13:51:59.155619+0000","last_change":"2023-08-09T13:51:59.155619+0000","last_active":"2023-08-09T13:51:59.130555+0000","last_peered":"2023-08-09T13:51:59.130555+0000","last_clean":"2023-08-09T13:51:59.130555+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-09T13:51:59.155619+0000","last_undegraded":"2023-08-09T13:51:59.155619+0000","last_fullsized":"2023-08-09T13:51:59.155619+0000","mapping_epoch":114,"log_start":"0'0","ondisk_log_start":"0'0","created":114,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:51:59.130555+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:51:59.130555+0000","last_clean_scrub_stamp":"2023-08-09T13:51:59.130555+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.3a","version":"0'0","reported_seq":1,"reported_epoch":114,"state":"creating+peering","last_fresh":"2023-08-09T13:51:59.155005+0000","last_change":"2023-08-09T13:51:59.155005+0000","last_active":"2023-08-09T13:51:59.130555+0000","last_peered":"2023-08-09T13:51:59.130555+0000","last_clean":"2023-08-09T13:51:59.130555+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-09T13:51:59.155005+0000","last_undegraded":"2023-08-09T13:51:59.155005+0000","last_fullsized":"2023-08-09T13:51:59.155005+0000","mapping_epoch":114,"log_start":"0'0","ondisk_log_start":"0'0","created":114,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:51:59.130555+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:51:59.130555+0000","last_clean_scrub_stamp":"2023-08-09T13:51:59.130555+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"15.39","version":"0'0","reported_seq":1,"reported_epoch":114,"state":"creating+peering","last_fresh":"2023-08-09T13:51:59.154799+0000","last_change":"2023-08-09T13:51:59.154799+0000","last_active":"2023-08-09T13:51:59.130555+0000","last_peered":"2023-08-09T13:51:59.130555+0000","last_clean":"2023-08-09T13:51:59.130555+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-09T13:51:59.154799+0000","last_undegraded":"2023-08-09T13:51:59.154799+0000","last_fullsized":"2023-08-09T13:51:59.154799+0000","mapping_epoch":114,"log_start":"0'0","ondisk_log_start":"0'0","created":114,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:51:59.130555+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:51:59.130555+0000","last_clean_scrub_stamp":"2023-08-09T13:51:59.130555+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"15.38","version":"0'0","reported_seq":1,"reported_epoch":114,"state":"creating+peering","last_fresh":"2023-08-09T13:51:59.161114+0000","last_change":"2023-08-09T13:51:59.161114+0000","last_active":"2023-08-09T13:51:59.130555+0000","last_peered":"2023-08-09T13:51:59.130555+0000","last_clean":"2023-08-09T13:51:59.130555+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-09T13:51:59.161114+0000","last_undegraded":"2023-08-09T13:51:59.161114+0000","last_fullsized":"2023-08-09T13:51:59.161114+0000","mapping_epoch":114,"log_start":"0'0","ondisk_log_start":"0'0","created":114,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:51:59.130555+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:51:59.130555+0000","last_clean_scrub_stamp":"2023-08-09T13:51:59.130555+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.37","version":"0'0","reported_seq":1,"reported_epoch":114,"state":"creating+peering","last_fresh":"2023-08-09T13:51:59.170489+0000","last_change":"2023-08-09T13:51:59.170489+0000","last_active":"2023-08-09T13:51:59.130555+0000","last_peered":"2023-08-09T13:51:59.130555+0000","last_clean":"2023-08-09T13:51:59.130555+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-09T13:51:59.170489+0000","last_undegraded":"2023-08-09T13:51:59.170489+0000","last_fullsized":"2023-08-09T13:51:59.170489+0000","mapping_epoch":114,"log_start":"0'0","ondisk_log_start":"0'0","created":114,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:51:59.130555+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:51:59.130555+0000","last_clean_scrub_stamp":"2023-08-09T13:51:59.130555+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"15.36","version":"0'0","reported_seq":1,"reported_epoch":114,"state":"creating+peering","last_fresh":"2023-08-09T13:51:59.157362+0000","last_change":"2023-08-09T13:51:59.157362+0000","last_active":"2023-08-09T13:51:59.130555+0000","last_peered":"2023-08-09T13:51:59.130555+0000","last_clean":"2023-08-09T13:51:59.130555+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-09T13:51:59.157362+0000","last_undegraded":"2023-08-09T13:51:59.157362+0000","last_fullsized":"2023-08-09T13:51:59.157362+0000","mapping_epoch":114,"log_start":"0'0","ondisk_log_start":"0'0","created":114,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:51:59.130555+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:51:59.130555+0000","last_clean_scrub_stamp":"2023-08-09T13:51:59.130555+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.35","version":"0'0","reported_seq":1,"reported_epoch":114,"state":"creating+peering","last_fresh":"2023-08-09T13:51:59.158636+0000","last_change":"2023-08-09T13:51:59.158636+0000","last_active":"2023-08-09T13:51:59.130555+0000","last_peered":"2023-08-09T13:51:59.130555+0000","last_clean":"2023-08-09T13:51:59.130555+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-09T13:51:59.158636+0000","last_undegraded":"2023-08-09T13:51:59.158636+0000","last_fullsized":"2023-08-09T13:51:59.158636+0000","mapping_epoch":114,"log_start":"0'0","ondisk_log_start":"0'0","created":114,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:51:59.130555+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:51:59.130555+0000","last_clean_scrub_stamp":"2023-08-09T13:51:59.130555+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"15.34","version":"0'0","reported_seq":1,"reported_epoch":114,"state":"creating+peering","last_fresh":"2023-08-09T13:51:59.175941+0000","last_change":"2023-08-09T13:51:59.175941+0000","last_active":"2023-08-09T13:51:59.130555+0000","last_peered":"2023-08-09T13:51:59.130555+0000","last_clean":"2023-08-09T13:51:59.130555+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-09T13:51:59.175941+0000","last_undegraded":"2023-08-09T13:51:59.175941+0000","last_fullsized":"2023-08-09T13:51:59.175941+0000","mapping_epoch":114,"log_start":"0'0","ondisk_log_start":"0'0","created":114,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:51:59.130555+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:51:59.130555+0000","last_clean_scrub_stamp":"2023-08-09T13:51:59.130555+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"15.33","version":"0'0","reported_seq":1,"reported_epoch":114,"state":"creating+peering","last_fresh":"2023-08-09T13:51:59.150804+0000","last_change":"2023-08-09T13:51:59.150804+0000","last_active":"2023-08-09T13:51:59.130555+0000","last_peered":"2023-08-09T13:51:59.130555+0000","last_clean":"2023-08-09T13:51:59.130555+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-09T13:51:59.150804+0000","last_undegraded":"2023-08-09T13:51:59.150804+0000","last_fullsized":"2023-08-09T13:51:59.150804+0000","mapping_epoch":114,"log_start":"0'0","ondisk_log_start":"0'0","created":114,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:51:59.130555+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:51:59.130555+0000","last_clean_scrub_stamp":"2023-08-09T13:51:59.130555+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"15.32","version":"0'0","reported_seq":1,"reported_epoch":114,"state":"creating+peering","last_fresh":"2023-08-09T13:51:59.165080+0000","last_change":"2023-08-09T13:51:59.165080+0000","last_active":"2023-08-09T13:51:59.130555+0000","last_peered":"2023-08-09T13:51:59.130555+0000","last_clean":"2023-08-09T13:51:59.130555+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-09T13:51:59.165080+0000","last_undegraded":"2023-08-09T13:51:59.165080+0000","last_fullsized":"2023-08-09T13:51:59.165080+0000","mapping_epoch":114,"log_start":"0'0","ondisk_log_start":"0'0","created":114,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:51:59.130555+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:51:59.130555+0000","last_clean_scrub_stamp":"2023-08-09T13:51:59.130555+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.31","version":"0'0","reported_seq":1,"reported_epoch":114,"state":"creating+peering","last_fresh":"2023-08-09T13:51:59.150713+0000","last_change":"2023-08-09T13:51:59.150713+0000","last_active":"2023-08-09T13:51:59.130555+0000","last_peered":"2023-08-09T13:51:59.130555+0000","last_clean":"2023-08-09T13:51:59.130555+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-09T13:51:59.150713+0000","last_undegraded":"2023-08-09T13:51:59.150713+0000","last_fullsized":"2023-08-09T13:51:59.150713+0000","mapping_epoch":114,"log_start":"0'0","ondisk_log_start":"0'0","created":114,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:51:59.130555+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:51:59.130555+0000","last_clean_scrub_stamp":"2023-08-09T13:51:59.130555+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"15.30","version":"0'0","reported_seq":1,"reported_epoch":114,"state":"creating+peering","last_fresh":"2023-08-09T13:51:59.157225+0000","last_change":"2023-08-09T13:51:59.157225+0000","last_active":"2023-08-09T13:51:59.130555+0000","last_peered":"2023-08-09T13:51:59.130555+0000","last_clean":"2023-08-09T13:51:59.130555+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-09T13:51:59.157225+0000","last_undegraded":"2023-08-09T13:51:59.157225+0000","last_fullsized":"2023-08-09T13:51:59.157225+0000","mapping_epoch":114,"log_start":"0'0","ondisk_log_start":"0'0","created":114,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:51:59.130555+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:51:59.130555+0000","last_clean_scrub_stamp":"2023-08-09T13:51:59.130555+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"15.2f","version":"0'0","reported_seq":1,"reported_epoch":114,"state":"creating+peering","last_fresh":"2023-08-09T13:51:59.157118+0000","last_change":"2023-08-09T13:51:59.157118+0000","last_active":"2023-08-09T13:51:59.130555+0000","last_peered":"2023-08-09T13:51:59.130555+0000","last_clean":"2023-08-09T13:51:59.130555+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-09T13:51:59.157118+0000","last_undegraded":"2023-08-09T13:51:59.157118+0000","last_fullsized":"2023-08-09T13:51:59.157118+0000","mapping_epoch":114,"log_start":"0'0","ondisk_log_start":"0'0","created":114,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:51:59.130555+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:51:59.130555+0000","last_clean_scrub_stamp":"2023-08-09T13:51:59.130555+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.2e","version":"0'0","reported_seq":1,"reported_epoch":114,"state":"creating+peering","last_fresh":"2023-08-09T13:51:59.153061+0000","last_change":"2023-08-09T13:51:59.153061+0000","last_active":"2023-08-09T13:51:59.130555+0000","last_peered":"2023-08-09T13:51:59.130555+0000","last_clean":"2023-08-09T13:51:59.130555+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-09T13:51:59.153061+0000","last_undegraded":"2023-08-09T13:51:59.153061+0000","last_fullsized":"2023-08-09T13:51:59.153061+0000","mapping_epoch":114,"log_start":"0'0","ondisk_log_start":"0'0","created":114,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:51:59.130555+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:51:59.130555+0000","last_clean_scrub_stamp":"2023-08-09T13:51:59.130555+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"15.2d","version":"0'0","reported_seq":1,"reported_epoch":114,"state":"creating+peering","last_fresh":"2023-08-09T13:51:59.156327+0000","last_change":"2023-08-09T13:51:59.156327+0000","last_active":"2023-08-09T13:51:59.130555+0000","last_peered":"2023-08-09T13:51:59.130555+0000","last_clean":"2023-08-09T13:51:59.130555+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-09T13:51:59.156327+0000","last_undegraded":"2023-08-09T13:51:59.156327+0000","last_fullsized":"2023-08-09T13:51:59.156327+0000","mapping_epoch":114,"log_start":"0'0","ondisk_log_start":"0'0","created":114,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:51:59.130555+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:51:59.130555+0000","last_clean_scrub_stamp":"2023-08-09T13:51:59.130555+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.2c","version":"0'0","reported_seq":1,"reported_epoch":114,"state":"creating+peering","last_fresh":"2023-08-09T13:51:59.175350+0000","last_change":"2023-08-09T13:51:59.175350+0000","last_active":"2023-08-09T13:51:59.130555+0000","last_peered":"2023-08-09T13:51:59.130555+0000","last_clean":"2023-08-09T13:51:59.130555+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-09T13:51:59.175350+0000","last_undegraded":"2023-08-09T13:51:59.175350+0000","last_fullsized":"2023-08-09T13:51:59.175350+0000","mapping_epoch":114,"log_start":"0'0","ondisk_log_start":"0'0","created":114,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:51:59.130555+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:51:59.130555+0000","last_clean_scrub_stamp":"2023-08-09T13:51:59.130555+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"15.2b","version":"0'0","reported_seq":1,"reported_epoch":114,"state":"creating+peering","last_fresh":"2023-08-09T13:51:59.163315+0000","last_change":"2023-08-09T13:51:59.163315+0000","last_active":"2023-08-09T13:51:59.130555+0000","last_peered":"2023-08-09T13:51:59.130555+0000","last_clean":"2023-08-09T13:51:59.130555+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-09T13:51:59.163315+0000","last_undegraded":"2023-08-09T13:51:59.163315+0000","last_fullsized":"2023-08-09T13:51:59.163315+0000","mapping_epoch":114,"log_start":"0'0","ondisk_log_start":"0'0","created":114,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:51:59.130555+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:51:59.130555+0000","last_clean_scrub_stamp":"2023-08-09T13:51:59.130555+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.2a","version":"0'0","reported_seq":1,"reported_epoch":114,"state":"creating+peering","last_fresh":"2023-08-09T13:51:59.158490+0000","last_change":"2023-08-09T13:51:59.158490+0000","last_active":"2023-08-09T13:51:59.130555+0000","last_peered":"2023-08-09T13:51:59.130555+0000","last_clean":"2023-08-09T13:51:59.130555+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-09T13:51:59.158490+0000","last_undegraded":"2023-08-09T13:51:59.158490+0000","last_fullsized":"2023-08-09T13:51:59.158490+0000","mapping_epoch":114,"log_start":"0'0","ondisk_log_start":"0'0","created":114,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:51:59.130555+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:51:59.130555+0000","last_clean_scrub_stamp":"2023-08-09T13:51:59.130555+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.29","version":"0'0","reported_seq":1,"reported_epoch":114,"state":"creating+peering","last_fresh":"2023-08-09T13:51:59.157791+0000","last_change":"2023-08-09T13:51:59.157791+0000","last_active":"2023-08-09T13:51:59.130555+0000","last_peered":"2023-08-09T13:51:59.130555+0000","last_clean":"2023-08-09T13:51:59.130555+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-09T13:51:59.157791+0000","last_undegraded":"2023-08-09T13:51:59.157791+0000","last_fullsized":"2023-08-09T13:51:59.157791+0000","mapping_epoch":114,"log_start":"0'0","ondisk_log_start":"0'0","created":114,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:51:59.130555+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:51:59.130555+0000","last_clean_scrub_stamp":"2023-08-09T13:51:59.130555+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.28","version":"0'0","reported_seq":1,"reported_epoch":114,"state":"creating+peering","last_fresh":"2023-08-09T13:51:59.156031+0000","last_change":"2023-08-09T13:51:59.156031+0000","last_active":"2023-08-09T13:51:59.130555+0000","last_peered":"2023-08-09T13:51:59.130555+0000","last_clean":"2023-08-09T13:51:59.130555+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-09T13:51:59.156031+0000","last_undegraded":"2023-08-09T13:51:59.156031+0000","last_fullsized":"2023-08-09T13:51:59.156031+0000","mapping_epoch":114,"log_start":"0'0","ondisk_log_start":"0'0","created":114,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:51:59.130555+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:51:59.130555+0000","last_clean_scrub_stamp":"2023-08-09T13:51:59.130555+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"15.27","version":"0'0","reported_seq":1,"reported_epoch":114,"state":"creating+peering","last_fresh":"2023-08-09T13:51:59.157867+0000","last_change":"2023-08-09T13:51:59.157867+0000","last_active":"2023-08-09T13:51:59.130555+0000","last_peered":"2023-08-09T13:51:59.130555+0000","last_clean":"2023-08-09T13:51:59.130555+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-09T13:51:59.157867+0000","last_undegraded":"2023-08-09T13:51:59.157867+0000","last_fullsized":"2023-08-09T13:51:59.157867+0000","mapping_epoch":114,"log_start":"0'0","ondisk_log_start":"0'0","created":114,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:51:59.130555+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:51:59.130555+0000","last_clean_scrub_stamp":"2023-08-09T13:51:59.130555+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.26","version":"0'0","reported_seq":1,"reported_epoch":114,"state":"creating+peering","last_fresh":"2023-08-09T13:51:59.159192+0000","last_change":"2023-08-09T13:51:59.159192+0000","last_active":"2023-08-09T13:51:59.130555+0000","last_peered":"2023-08-09T13:51:59.130555+0000","last_clean":"2023-08-09T13:51:59.130555+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-09T13:51:59.159192+0000","last_undegraded":"2023-08-09T13:51:59.159192+0000","last_fullsized":"2023-08-09T13:51:59.159192+0000","mapping_epoch":114,"log_start":"0'0","ondisk_log_start":"0'0","created":114,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:51:59.130555+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:51:59.130555+0000","last_clean_scrub_stamp":"2023-08-09T13:51:59.130555+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.25","version":"0'0","reported_seq":1,"reported_epoch":114,"state":"creating+peering","last_fresh":"2023-08-09T13:51:59.157367+0000","last_change":"2023-08-09T13:51:59.157367+0000","last_active":"2023-08-09T13:51:59.130555+0000","last_peered":"2023-08-09T13:51:59.130555+0000","last_clean":"2023-08-09T13:51:59.130555+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-09T13:51:59.157367+0000","last_undegraded":"2023-08-09T13:51:59.157367+0000","last_fullsized":"2023-08-09T13:51:59.157367+0000","mapping_epoch":114,"log_start":"0'0","ondisk_log_start":"0'0","created":114,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:51:59.130555+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:51:59.130555+0000","last_clean_scrub_stamp":"2023-08-09T13:51:59.130555+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"15.24","version":"0'0","reported_seq":1,"reported_epoch":114,"state":"creating+peering","last_fresh":"2023-08-09T13:51:59.162536+0000","last_change":"2023-08-09T13:51:59.162536+0000","last_active":"2023-08-09T13:51:59.130555+0000","last_peered":"2023-08-09T13:51:59.130555+0000","last_clean":"2023-08-09T13:51:59.130555+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-09T13:51:59.162536+0000","last_undegraded":"2023-08-09T13:51:59.162536+0000","last_fullsized":"2023-08-09T13:51:59.162536+0000","mapping_epoch":114,"log_start":"0'0","ondisk_log_start":"0'0","created":114,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:51:59.130555+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:51:59.130555+0000","last_clean_scrub_stamp":"2023-08-09T13:51:59.130555+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.23","version":"0'0","reported_seq":1,"reported_epoch":114,"state":"creating+peering","last_fresh":"2023-08-09T13:51:59.160575+0000","last_change":"2023-08-09T13:51:59.160575+0000","last_active":"2023-08-09T13:51:59.130555+0000","last_peered":"2023-08-09T13:51:59.130555+0000","last_clean":"2023-08-09T13:51:59.130555+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-09T13:51:59.160575+0000","last_undegraded":"2023-08-09T13:51:59.160575+0000","last_fullsized":"2023-08-09T13:51:59.160575+0000","mapping_epoch":114,"log_start":"0'0","ondisk_log_start":"0'0","created":114,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:51:59.130555+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:51:59.130555+0000","last_clean_scrub_stamp":"2023-08-09T13:51:59.130555+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.22","version":"0'0","reported_seq":1,"reported_epoch":114,"state":"creating+peering","last_fresh":"2023-08-09T13:51:59.164643+0000","last_change":"2023-08-09T13:51:59.164643+0000","last_active":"2023-08-09T13:51:59.130555+0000","last_peered":"2023-08-09T13:51:59.130555+0000","last_clean":"2023-08-09T13:51:59.130555+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-09T13:51:59.164643+0000","last_undegraded":"2023-08-09T13:51:59.164643+0000","last_fullsized":"2023-08-09T13:51:59.164643+0000","mapping_epoch":114,"log_start":"0'0","ondisk_log_start":"0'0","created":114,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:51:59.130555+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:51:59.130555+0000","last_clean_scrub_stamp":"2023-08-09T13:51:59.130555+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.21","version":"0'0","reported_seq":1,"reported_epoch":114,"state":"creating+peering","last_fresh":"2023-08-09T13:51:59.149533+0000","last_change":"2023-08-09T13:51:59.149533+0000","last_active":"2023-08-09T13:51:59.130555+0000","last_peered":"2023-08-09T13:51:59.130555+0000","last_clean":"2023-08-09T13:51:59.130555+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-09T13:51:59.149533+0000","last_undegraded":"2023-08-09T13:51:59.149533+0000","last_fullsized":"2023-08-09T13:51:59.149533+0000","mapping_epoch":114,"log_start":"0'0","ondisk_log_start":"0'0","created":114,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:51:59.130555+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:51:59.130555+0000","last_clean_scrub_stamp":"2023-08-09T13:51:59.130555+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"15.20","version":"0'0","reported_seq":1,"reported_epoch":114,"state":"creating+peering","last_fresh":"2023-08-09T13:51:59.168339+0000","last_change":"2023-08-09T13:51:59.168339+0000","last_active":"2023-08-09T13:51:59.130555+0000","last_peered":"2023-08-09T13:51:59.130555+0000","last_clean":"2023-08-09T13:51:59.130555+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-09T13:51:59.168339+0000","last_undegraded":"2023-08-09T13:51:59.168339+0000","last_fullsized":"2023-08-09T13:51:59.168339+0000","mapping_epoch":114,"log_start":"0'0","ondisk_log_start":"0'0","created":114,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:51:59.130555+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:51:59.130555+0000","last_clean_scrub_stamp":"2023-08-09T13:51:59.130555+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"15.1e","version":"0'0","reported_seq":1,"reported_epoch":114,"state":"creating+peering","last_fresh":"2023-08-09T13:51:59.162676+0000","last_change":"2023-08-09T13:51:59.162676+0000","last_active":"2023-08-09T13:51:59.130555+0000","last_peered":"2023-08-09T13:51:59.130555+0000","last_clean":"2023-08-09T13:51:59.130555+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-09T13:51:59.162676+0000","last_undegraded":"2023-08-09T13:51:59.162676+0000","last_fullsized":"2023-08-09T13:51:59.162676+0000","mapping_epoch":114,"log_start":"0'0","ondisk_log_start":"0'0","created":114,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:51:59.130555+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:51:59.130555+0000","last_clean_scrub_stamp":"2023-08-09T13:51:59.130555+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"14.1f","version":"0'0","reported_seq":9,"reported_epoch":114,"state":"active+clean","last_fresh":"2023-08-09T13:51:59.180137+0000","last_change":"2023-08-09T13:51:59.176081+0000","last_active":"2023-08-09T13:51:59.180137+0000","last_peered":"2023-08-09T13:51:59.180137+0000","last_clean":"2023-08-09T13:51:59.180137+0000","last_became_active":"2023-08-09T13:51:59.175790+0000","last_became_peered":"2023-08-09T13:51:59.175790+0000","last_unstale":"2023-08-09T13:51:59.180137+0000","last_undegraded":"2023-08-09T13:51:59.180137+0000","last_fullsized":"2023-08-09T13:51:59.180137+0000","mapping_epoch":113,"log_start":"0'0","ondisk_log_start":"0'0","created":113,"last_epoch_clean":114,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:51:58.125661+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:51:58.125661+0000","last_clean_scrub_stamp":"2023-08-09T13:51:58.125661+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T17:58:19.394820+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.1f","version":"0'0","reported_seq":1,"reported_epoch":114,"state":"creating+peering","last_fresh":"2023-08-09T13:51:59.151940+0000","last_change":"2023-08-09T13:51:59.151940+0000","last_active":"2023-08-09T13:51:59.130555+0000","last_peered":"2023-08-09T13:51:59.130555+0000","last_clean":"2023-08-09T13:51:59.130555+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-09T13:51:59.151940+0000","last_undegraded":"2023-08-09T13:51:59.151940+0000","last_fullsized":"2023-08-09T13:51:59.151940+0000","mapping_epoch":114,"log_start":"0'0","ondisk_log_start":"0'0","created":114,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:51:59.130555+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:51:59.130555+0000","last_clean_scrub_stamp":"2023-08-09T13:51:59.130555+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"14.1e","version":"0'0","reported_seq":9,"reported_epoch":114,"state":"active+clean","last_fresh":"2023-08-09T13:51:59.180102+0000","last_change":"2023-08-09T13:51:59.178857+0000","last_active":"2023-08-09T13:51:59.180102+0000","last_peered":"2023-08-09T13:51:59.180102+0000","last_clean":"2023-08-09T13:51:59.180102+0000","last_became_active":"2023-08-09T13:51:59.178483+0000","last_became_peered":"2023-08-09T13:51:59.178483+0000","last_unstale":"2023-08-09T13:51:59.180102+0000","last_undegraded":"2023-08-09T13:51:59.180102+0000","last_fullsized":"2023-08-09T13:51:59.180102+0000","mapping_epoch":113,"log_start":"0'0","ondisk_log_start":"0'0","created":113,"last_epoch_clean":114,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:51:58.125661+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:51:58.125661+0000","last_clean_scrub_stamp":"2023-08-09T13:51:58.125661+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T15:40:02.742070+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"15.1c","version":"0'0","reported_seq":1,"reported_epoch":114,"state":"creating+peering","last_fresh":"2023-08-09T13:51:59.166472+0000","last_change":"2023-08-09T13:51:59.166472+0000","last_active":"2023-08-09T13:51:59.130555+0000","last_peered":"2023-08-09T13:51:59.130555+0000","last_clean":"2023-08-09T13:51:59.130555+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-09T13:51:59.166472+0000","last_undegraded":"2023-08-09T13:51:59.166472+0000","last_fullsized":"2023-08-09T13:51:59.166472+0000","mapping_epoch":114,"log_start":"0'0","ondisk_log_start":"0'0","created":114,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:51:59.130555+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:51:59.130555+0000","last_clean_scrub_stamp":"2023-08-09T13:51:59.130555+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.1d","version":"0'0","reported_seq":9,"reported_epoch":114,"state":"active+clean","last_fresh":"2023-08-09T13:51:59.173649+0000","last_change":"2023-08-09T13:51:59.168942+0000","last_active":"2023-08-09T13:51:59.173649+0000","last_peered":"2023-08-09T13:51:59.173649+0000","last_clean":"2023-08-09T13:51:59.173649+0000","last_became_active":"2023-08-09T13:51:59.168599+0000","last_became_peered":"2023-08-09T13:51:59.168599+0000","last_unstale":"2023-08-09T13:51:59.173649+0000","last_undegraded":"2023-08-09T13:51:59.173649+0000","last_fullsized":"2023-08-09T13:51:59.173649+0000","mapping_epoch":113,"log_start":"0'0","ondisk_log_start":"0'0","created":113,"last_epoch_clean":114,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:51:58.125661+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:51:58.125661+0000","last_clean_scrub_stamp":"2023-08-09T13:51:58.125661+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T21:57:39.771411+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.1d","version":"0'0","reported_seq":1,"reported_epoch":114,"state":"creating+peering","last_fresh":"2023-08-09T13:51:59.170197+0000","last_change":"2023-08-09T13:51:59.170197+0000","last_active":"2023-08-09T13:51:59.130555+0000","last_peered":"2023-08-09T13:51:59.130555+0000","last_clean":"2023-08-09T13:51:59.130555+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-09T13:51:59.170197+0000","last_undegraded":"2023-08-09T13:51:59.170197+0000","last_fullsized":"2023-08-09T13:51:59.170197+0000","mapping_epoch":114,"log_start":"0'0","ondisk_log_start":"0'0","created":114,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:51:59.130555+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:51:59.130555+0000","last_clean_scrub_stamp":"2023-08-09T13:51:59.130555+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.1c","version":"0'0","reported_seq":9,"reported_epoch":114,"state":"active+clean","last_fresh":"2023-08-09T13:51:59.184792+0000","last_change":"2023-08-09T13:51:59.183710+0000","last_active":"2023-08-09T13:51:59.184792+0000","last_peered":"2023-08-09T13:51:59.184792+0000","last_clean":"2023-08-09T13:51:59.184792+0000","last_became_active":"2023-08-09T13:51:59.183401+0000","last_became_peered":"2023-08-09T13:51:59.183401+0000","last_unstale":"2023-08-09T13:51:59.184792+0000","last_undegraded":"2023-08-09T13:51:59.184792+0000","last_fullsized":"2023-08-09T13:51:59.184792+0000","mapping_epoch":113,"log_start":"0'0","ondisk_log_start":"0'0","created":113,"last_epoch_clean":114,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:51:58.125661+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:51:58.125661+0000","last_clean_scrub_stamp":"2023-08-09T13:51:58.125661+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T13:55:08.336317+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.1a","version":"0'0","reported_seq":1,"reported_epoch":114,"state":"creating+peering","last_fresh":"2023-08-09T13:51:59.165426+0000","last_change":"2023-08-09T13:51:59.165426+0000","last_active":"2023-08-09T13:51:59.130555+0000","last_peered":"2023-08-09T13:51:59.130555+0000","last_clean":"2023-08-09T13:51:59.130555+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-09T13:51:59.165426+0000","last_undegraded":"2023-08-09T13:51:59.165426+0000","last_fullsized":"2023-08-09T13:51:59.165426+0000","mapping_epoch":114,"log_start":"0'0","ondisk_log_start":"0'0","created":114,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:51:59.130555+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:51:59.130555+0000","last_clean_scrub_stamp":"2023-08-09T13:51:59.130555+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"14.1b","version":"0'0","reported_seq":9,"reported_epoch":114,"state":"active+clean","last_fresh":"2023-08-09T13:51:59.180443+0000","last_change":"2023-08-09T13:51:59.178308+0000","last_active":"2023-08-09T13:51:59.180443+0000","last_peered":"2023-08-09T13:51:59.180443+0000","last_clean":"2023-08-09T13:51:59.180443+0000","last_became_active":"2023-08-09T13:51:59.178022+0000","last_became_peered":"2023-08-09T13:51:59.178022+0000","last_unstale":"2023-08-09T13:51:59.180443+0000","last_undegraded":"2023-08-09T13:51:59.180443+0000","last_fullsized":"2023-08-09T13:51:59.180443+0000","mapping_epoch":113,"log_start":"0'0","ondisk_log_start":"0'0","created":113,"last_epoch_clean":114,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:51:58.125661+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:51:58.125661+0000","last_clean_scrub_stamp":"2023-08-09T13:51:58.125661+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T15:51:56.504946+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"15.1b","version":"0'0","reported_seq":1,"reported_epoch":114,"state":"creating+peering","last_fresh":"2023-08-09T13:51:59.157444+0000","last_change":"2023-08-09T13:51:59.157444+0000","last_active":"2023-08-09T13:51:59.130555+0000","last_peered":"2023-08-09T13:51:59.130555+0000","last_clean":"2023-08-09T13:51:59.130555+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-09T13:51:59.157444+0000","last_undegraded":"2023-08-09T13:51:59.157444+0000","last_fullsized":"2023-08-09T13:51:59.157444+0000","mapping_epoch":114,"log_start":"0'0","ondisk_log_start":"0'0","created":114,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:51:59.130555+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:51:59.130555+0000","last_clean_scrub_stamp":"2023-08-09T13:51:59.130555+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"14.1a","version":"0'0","reported_seq":9,"reported_epoch":114,"state":"active+clean","last_fresh":"2023-08-09T13:51:59.173191+0000","last_change":"2023-08-09T13:51:59.165522+0000","last_active":"2023-08-09T13:51:59.173191+0000","last_peered":"2023-08-09T13:51:59.173191+0000","last_clean":"2023-08-09T13:51:59.173191+0000","last_became_active":"2023-08-09T13:51:59.165145+0000","last_became_peered":"2023-08-09T13:51:59.165145+0000","last_unstale":"2023-08-09T13:51:59.173191+0000","last_undegraded":"2023-08-09T13:51:59.173191+0000","last_fullsized":"2023-08-09T13:51:59.173191+0000","mapping_epoch":113,"log_start":"0'0","ondisk_log_start":"0'0","created":113,"last_epoch_clean":114,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:51:58.125661+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:51:58.125661+0000","last_clean_scrub_stamp":"2023-08-09T13:51:58.125661+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T16:19:11.497849+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.18","version":"0'0","reported_seq":1,"reported_epoch":114,"state":"creating+peering","last_fresh":"2023-08-09T13:51:59.149114+0000","last_change":"2023-08-09T13:51:59.149114+0000","last_active":"2023-08-09T13:51:59.130555+0000","last_peered":"2023-08-09T13:51:59.130555+0000","last_clean":"2023-08-09T13:51:59.130555+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-09T13:51:59.149114+0000","last_undegraded":"2023-08-09T13:51:59.149114+0000","last_fullsized":"2023-08-09T13:51:59.149114+0000","mapping_epoch":114,"log_start":"0'0","ondisk_log_start":"0'0","created":114,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:51:59.130555+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:51:59.130555+0000","last_clean_scrub_stamp":"2023-08-09T13:51:59.130555+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"14.19","version":"0'0","reported_seq":9,"reported_epoch":114,"state":"active+clean","last_fresh":"2023-08-09T13:51:59.178533+0000","last_change":"2023-08-09T13:51:59.174362+0000","last_active":"2023-08-09T13:51:59.178533+0000","last_peered":"2023-08-09T13:51:59.178533+0000","last_clean":"2023-08-09T13:51:59.178533+0000","last_became_active":"2023-08-09T13:51:59.174002+0000","last_became_peered":"2023-08-09T13:51:59.174002+0000","last_unstale":"2023-08-09T13:51:59.178533+0000","last_undegraded":"2023-08-09T13:51:59.178533+0000","last_fullsized":"2023-08-09T13:51:59.178533+0000","mapping_epoch":113,"log_start":"0'0","ondisk_log_start":"0'0","created":113,"last_epoch_clean":114,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:51:58.125661+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:51:58.125661+0000","last_clean_scrub_stamp":"2023-08-09T13:51:58.125661+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T19:08:56.894026+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.19","version":"0'0","reported_seq":1,"reported_epoch":114,"state":"creating+peering","last_fresh":"2023-08-09T13:51:59.168819+0000","last_change":"2023-08-09T13:51:59.168819+0000","last_active":"2023-08-09T13:51:59.130555+0000","last_peered":"2023-08-09T13:51:59.130555+0000","last_clean":"2023-08-09T13:51:59.130555+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-09T13:51:59.168819+0000","last_undegraded":"2023-08-09T13:51:59.168819+0000","last_fullsized":"2023-08-09T13:51:59.168819+0000","mapping_epoch":114,"log_start":"0'0","ondisk_log_start":"0'0","created":114,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:51:59.130555+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:51:59.130555+0000","last_clean_scrub_stamp":"2023-08-09T13:51:59.130555+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.18","version":"0'0","reported_seq":9,"reported_epoch":114,"state":"active+clean","last_fresh":"2023-08-09T13:51:59.180392+0000","last_change":"2023-08-09T13:51:59.177991+0000","last_active":"2023-08-09T13:51:59.180392+0000","last_peered":"2023-08-09T13:51:59.180392+0000","last_clean":"2023-08-09T13:51:59.180392+0000","last_became_active":"2023-08-09T13:51:59.177651+0000","last_became_peered":"2023-08-09T13:51:59.177651+0000","last_unstale":"2023-08-09T13:51:59.180392+0000","last_undegraded":"2023-08-09T13:51:59.180392+0000","last_fullsized":"2023-08-09T13:51:59.180392+0000","mapping_epoch":113,"log_start":"0'0","ondisk_log_start":"0'0","created":113,"last_epoch_clean":114,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:51:58.125661+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:51:58.125661+0000","last_clean_scrub_stamp":"2023-08-09T13:51:58.125661+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T18:13:34.464631+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"15.16","version":"0'0","reported_seq":1,"reported_epoch":114,"state":"creating+peering","last_fresh":"2023-08-09T13:51:59.158359+0000","last_change":"2023-08-09T13:51:59.158359+0000","last_active":"2023-08-09T13:51:59.130555+0000","last_peered":"2023-08-09T13:51:59.130555+0000","last_clean":"2023-08-09T13:51:59.130555+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-09T13:51:59.158359+0000","last_undegraded":"2023-08-09T13:51:59.158359+0000","last_fullsized":"2023-08-09T13:51:59.158359+0000","mapping_epoch":114,"log_start":"0'0","ondisk_log_start":"0'0","created":114,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:51:59.130555+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:51:59.130555+0000","last_clean_scrub_stamp":"2023-08-09T13:51:59.130555+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"14.17","version":"0'0","reported_seq":9,"reported_epoch":114,"state":"active+clean","last_fresh":"2023-08-09T13:51:59.182321+0000","last_change":"2023-08-09T13:51:59.181245+0000","last_active":"2023-08-09T13:51:59.182321+0000","last_peered":"2023-08-09T13:51:59.182321+0000","last_clean":"2023-08-09T13:51:59.182321+0000","last_became_active":"2023-08-09T13:51:59.180911+0000","last_became_peered":"2023-08-09T13:51:59.180911+0000","last_unstale":"2023-08-09T13:51:59.182321+0000","last_undegraded":"2023-08-09T13:51:59.182321+0000","last_fullsized":"2023-08-09T13:51:59.182321+0000","mapping_epoch":113,"log_start":"0'0","ondisk_log_start":"0'0","created":113,"last_epoch_clean":114,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:51:58.125661+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:51:58.125661+0000","last_clean_scrub_stamp":"2023-08-09T13:51:58.125661+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T16:30:25.226241+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"15.17","version":"0'0","reported_seq":1,"reported_epoch":114,"state":"creating+peering","last_fresh":"2023-08-09T13:51:59.169791+0000","last_change":"2023-08-09T13:51:59.169791+0000","last_active":"2023-08-09T13:51:59.130555+0000","last_peered":"2023-08-09T13:51:59.130555+0000","last_clean":"2023-08-09T13:51:59.130555+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-09T13:51:59.169791+0000","last_undegraded":"2023-08-09T13:51:59.169791+0000","last_fullsized":"2023-08-09T13:51:59.169791+0000","mapping_epoch":114,"log_start":"0'0","ondisk_log_start":"0'0","created":114,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:51:59.130555+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:51:59.130555+0000","last_clean_scrub_stamp":"2023-08-09T13:51:59.130555+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.16","version":"0'0","reported_seq":9,"reported_epoch":114,"state":"active+clean","last_fresh":"2023-08-09T13:51:59.176606+0000","last_change":"2023-08-09T13:51:59.172670+0000","last_active":"2023-08-09T13:51:59.176606+0000","last_peered":"2023-08-09T13:51:59.176606+0000","last_clean":"2023-08-09T13:51:59.176606+0000","last_became_active":"2023-08-09T13:51:59.172395+0000","last_became_peered":"2023-08-09T13:51:59.172395+0000","last_unstale":"2023-08-09T13:51:59.176606+0000","last_undegraded":"2023-08-09T13:51:59.176606+0000","last_fullsized":"2023-08-09T13:51:59.176606+0000","mapping_epoch":113,"log_start":"0'0","ondisk_log_start":"0'0","created":113,"last_epoch_clean":114,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:51:58.125661+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:51:58.125661+0000","last_clean_scrub_stamp":"2023-08-09T13:51:58.125661+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T19:55:12.433811+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"15.14","version":"0'0","reported_seq":1,"reported_epoch":114,"state":"creating+peering","last_fresh":"2023-08-09T13:51:59.161549+0000","last_change":"2023-08-09T13:51:59.161549+0000","last_active":"2023-08-09T13:51:59.130555+0000","last_peered":"2023-08-09T13:51:59.130555+0000","last_clean":"2023-08-09T13:51:59.130555+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-09T13:51:59.161549+0000","last_undegraded":"2023-08-09T13:51:59.161549+0000","last_fullsized":"2023-08-09T13:51:59.161549+0000","mapping_epoch":114,"log_start":"0'0","ondisk_log_start":"0'0","created":114,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:51:59.130555+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:51:59.130555+0000","last_clean_scrub_stamp":"2023-08-09T13:51:59.130555+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"14.15","version":"0'0","reported_seq":9,"reported_epoch":114,"state":"active+clean","last_fresh":"2023-08-09T13:51:59.184646+0000","last_change":"2023-08-09T13:51:59.181964+0000","last_active":"2023-08-09T13:51:59.184646+0000","last_peered":"2023-08-09T13:51:59.184646+0000","last_clean":"2023-08-09T13:51:59.184646+0000","last_became_active":"2023-08-09T13:51:59.181630+0000","last_became_peered":"2023-08-09T13:51:59.181630+0000","last_unstale":"2023-08-09T13:51:59.184646+0000","last_undegraded":"2023-08-09T13:51:59.184646+0000","last_fullsized":"2023-08-09T13:51:59.184646+0000","mapping_epoch":113,"log_start":"0'0","ondisk_log_start":"0'0","created":113,"last_epoch_clean":114,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:51:58.125661+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:51:58.125661+0000","last_clean_scrub_stamp":"2023-08-09T13:51:58.125661+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T16:43:18.755379+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"15.15","version":"0'0","reported_seq":1,"reported_epoch":114,"state":"creating+peering","last_fresh":"2023-08-09T13:51:59.156075+0000","last_change":"2023-08-09T13:51:59.156075+0000","last_active":"2023-08-09T13:51:59.130555+0000","last_peered":"2023-08-09T13:51:59.130555+0000","last_clean":"2023-08-09T13:51:59.130555+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-09T13:51:59.156075+0000","last_undegraded":"2023-08-09T13:51:59.156075+0000","last_fullsized":"2023-08-09T13:51:59.156075+0000","mapping_epoch":114,"log_start":"0'0","ondisk_log_start":"0'0","created":114,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:51:59.130555+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:51:59.130555+0000","last_clean_scrub_stamp":"2023-08-09T13:51:59.130555+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"14.14","version":"0'0","reported_seq":9,"reported_epoch":114,"state":"active+clean","last_fresh":"2023-08-09T13:51:59.184692+0000","last_change":"2023-08-09T13:51:59.181967+0000","last_active":"2023-08-09T13:51:59.184692+0000","last_peered":"2023-08-09T13:51:59.184692+0000","last_clean":"2023-08-09T13:51:59.184692+0000","last_became_active":"2023-08-09T13:51:59.180664+0000","last_became_peered":"2023-08-09T13:51:59.180664+0000","last_unstale":"2023-08-09T13:51:59.184692+0000","last_undegraded":"2023-08-09T13:51:59.184692+0000","last_fullsized":"2023-08-09T13:51:59.184692+0000","mapping_epoch":113,"log_start":"0'0","ondisk_log_start":"0'0","created":113,"last_epoch_clean":114,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:51:58.125661+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:51:58.125661+0000","last_clean_scrub_stamp":"2023-08-09T13:51:58.125661+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-11T01:49:03.976153+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"15.12","version":"0'0","reported_seq":1,"reported_epoch":114,"state":"creating+peering","last_fresh":"2023-08-09T13:51:59.150320+0000","last_change":"2023-08-09T13:51:59.150320+0000","last_active":"2023-08-09T13:51:59.130555+0000","last_peered":"2023-08-09T13:51:59.130555+0000","last_clean":"2023-08-09T13:51:59.130555+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-09T13:51:59.150320+0000","last_undegraded":"2023-08-09T13:51:59.150320+0000","last_fullsized":"2023-08-09T13:51:59.150320+0000","mapping_epoch":114,"log_start":"0'0","ondisk_log_start":"0'0","created":114,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:51:59.130555+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:51:59.130555+0000","last_clean_scrub_stamp":"2023-08-09T13:51:59.130555+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"14.13","version":"0'0","reported_seq":9,"reported_epoch":114,"state":"active+clean","last_fresh":"2023-08-09T13:51:59.183063+0000","last_change":"2023-08-09T13:51:59.181856+0000","last_active":"2023-08-09T13:51:59.183063+0000","last_peered":"2023-08-09T13:51:59.183063+0000","last_clean":"2023-08-09T13:51:59.183063+0000","last_became_active":"2023-08-09T13:51:59.181568+0000","last_became_peered":"2023-08-09T13:51:59.181568+0000","last_unstale":"2023-08-09T13:51:59.183063+0000","last_undegraded":"2023-08-09T13:51:59.183063+0000","last_fullsized":"2023-08-09T13:51:59.183063+0000","mapping_epoch":113,"log_start":"0'0","ondisk_log_start":"0'0","created":113,"last_epoch_clean":114,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:51:58.125661+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:51:58.125661+0000","last_clean_scrub_stamp":"2023-08-09T13:51:58.125661+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T22:11:33.508209+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.13","version":"0'0","reported_seq":1,"reported_epoch":114,"state":"creating+peering","last_fresh":"2023-08-09T13:51:59.163775+0000","last_change":"2023-08-09T13:51:59.163775+0000","last_active":"2023-08-09T13:51:59.130555+0000","last_peered":"2023-08-09T13:51:59.130555+0000","last_clean":"2023-08-09T13:51:59.130555+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-09T13:51:59.163775+0000","last_undegraded":"2023-08-09T13:51:59.163775+0000","last_fullsized":"2023-08-09T13:51:59.163775+0000","mapping_epoch":114,"log_start":"0'0","ondisk_log_start":"0'0","created":114,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:51:59.130555+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:51:59.130555+0000","last_clean_scrub_stamp":"2023-08-09T13:51:59.130555+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.12","version":"0'0","reported_seq":9,"reported_epoch":114,"state":"active+clean","last_fresh":"2023-08-09T13:51:59.180137+0000","last_change":"2023-08-09T13:51:59.178710+0000","last_active":"2023-08-09T13:51:59.180137+0000","last_peered":"2023-08-09T13:51:59.180137+0000","last_clean":"2023-08-09T13:51:59.180137+0000","last_became_active":"2023-08-09T13:51:59.178402+0000","last_became_peered":"2023-08-09T13:51:59.178402+0000","last_unstale":"2023-08-09T13:51:59.180137+0000","last_undegraded":"2023-08-09T13:51:59.180137+0000","last_fullsized":"2023-08-09T13:51:59.180137+0000","mapping_epoch":113,"log_start":"0'0","ondisk_log_start":"0'0","created":113,"last_epoch_clean":114,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:51:58.125661+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:51:58.125661+0000","last_clean_scrub_stamp":"2023-08-09T13:51:58.125661+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T19:35:28.808687+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.10","version":"0'0","reported_seq":1,"reported_epoch":114,"state":"creating+peering","last_fresh":"2023-08-09T13:51:59.167863+0000","last_change":"2023-08-09T13:51:59.167863+0000","last_active":"2023-08-09T13:51:59.130555+0000","last_peered":"2023-08-09T13:51:59.130555+0000","last_clean":"2023-08-09T13:51:59.130555+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-09T13:51:59.167863+0000","last_undegraded":"2023-08-09T13:51:59.167863+0000","last_fullsized":"2023-08-09T13:51:59.167863+0000","mapping_epoch":114,"log_start":"0'0","ondisk_log_start":"0'0","created":114,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:51:59.130555+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:51:59.130555+0000","last_clean_scrub_stamp":"2023-08-09T13:51:59.130555+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.11","version":"0'0","reported_seq":9,"reported_epoch":114,"state":"active+clean","last_fresh":"2023-08-09T13:51:59.184619+0000","last_change":"2023-08-09T13:51:59.181121+0000","last_active":"2023-08-09T13:51:59.184619+0000","last_peered":"2023-08-09T13:51:59.184619+0000","last_clean":"2023-08-09T13:51:59.184619+0000","last_became_active":"2023-08-09T13:51:59.180844+0000","last_became_peered":"2023-08-09T13:51:59.180844+0000","last_unstale":"2023-08-09T13:51:59.184619+0000","last_undegraded":"2023-08-09T13:51:59.184619+0000","last_fullsized":"2023-08-09T13:51:59.184619+0000","mapping_epoch":113,"log_start":"0'0","ondisk_log_start":"0'0","created":113,"last_epoch_clean":114,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:51:58.125661+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:51:58.125661+0000","last_clean_scrub_stamp":"2023-08-09T13:51:58.125661+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T14:42:49.973080+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"15.11","version":"0'0","reported_seq":1,"reported_epoch":114,"state":"creating+peering","last_fresh":"2023-08-09T13:51:59.151102+0000","last_change":"2023-08-09T13:51:59.151102+0000","last_active":"2023-08-09T13:51:59.130555+0000","last_peered":"2023-08-09T13:51:59.130555+0000","last_clean":"2023-08-09T13:51:59.130555+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-09T13:51:59.151102+0000","last_undegraded":"2023-08-09T13:51:59.151102+0000","last_fullsized":"2023-08-09T13:51:59.151102+0000","mapping_epoch":114,"log_start":"0'0","ondisk_log_start":"0'0","created":114,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:51:59.130555+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:51:59.130555+0000","last_clean_scrub_stamp":"2023-08-09T13:51:59.130555+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"14.10","version":"0'0","reported_seq":9,"reported_epoch":114,"state":"active+clean","last_fresh":"2023-08-09T13:51:59.178095+0000","last_change":"2023-08-09T13:51:59.174790+0000","last_active":"2023-08-09T13:51:59.178095+0000","last_peered":"2023-08-09T13:51:59.178095+0000","last_clean":"2023-08-09T13:51:59.178095+0000","last_became_active":"2023-08-09T13:51:59.174453+0000","last_became_peered":"2023-08-09T13:51:59.174453+0000","last_unstale":"2023-08-09T13:51:59.178095+0000","last_undegraded":"2023-08-09T13:51:59.178095+0000","last_fullsized":"2023-08-09T13:51:59.178095+0000","mapping_epoch":113,"log_start":"0'0","ondisk_log_start":"0'0","created":113,"last_epoch_clean":114,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:51:58.125661+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:51:58.125661+0000","last_clean_scrub_stamp":"2023-08-09T13:51:58.125661+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T19:07:38.533940+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"15.6","version":"0'0","reported_seq":1,"reported_epoch":114,"state":"creating+peering","last_fresh":"2023-08-09T13:51:59.157299+0000","last_change":"2023-08-09T13:51:59.157299+0000","last_active":"2023-08-09T13:51:59.130555+0000","last_peered":"2023-08-09T13:51:59.130555+0000","last_clean":"2023-08-09T13:51:59.130555+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-09T13:51:59.157299+0000","last_undegraded":"2023-08-09T13:51:59.157299+0000","last_fullsized":"2023-08-09T13:51:59.157299+0000","mapping_epoch":114,"log_start":"0'0","ondisk_log_start":"0'0","created":114,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:51:59.130555+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:51:59.130555+0000","last_clean_scrub_stamp":"2023-08-09T13:51:59.130555+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"14.7","version":"0'0","reported_seq":9,"reported_epoch":114,"state":"active+clean","last_fresh":"2023-08-09T13:51:59.187317+0000","last_change":"2023-08-09T13:51:59.182730+0000","last_active":"2023-08-09T13:51:59.187317+0000","last_peered":"2023-08-09T13:51:59.187317+0000","last_clean":"2023-08-09T13:51:59.187317+0000","last_became_active":"2023-08-09T13:51:59.182469+0000","last_became_peered":"2023-08-09T13:51:59.182469+0000","last_unstale":"2023-08-09T13:51:59.187317+0000","last_undegraded":"2023-08-09T13:51:59.187317+0000","last_fullsized":"2023-08-09T13:51:59.187317+0000","mapping_epoch":113,"log_start":"0'0","ondisk_log_start":"0'0","created":113,"last_epoch_clean":114,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:51:58.125661+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:51:58.125661+0000","last_clean_scrub_stamp":"2023-08-09T13:51:58.125661+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T23:49:36.976561+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"15.7","version":"0'0","reported_seq":1,"reported_epoch":114,"state":"creating+peering","last_fresh":"2023-08-09T13:51:59.162752+0000","last_change":"2023-08-09T13:51:59.162752+0000","last_active":"2023-08-09T13:51:59.130555+0000","last_peered":"2023-08-09T13:51:59.130555+0000","last_clean":"2023-08-09T13:51:59.130555+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-09T13:51:59.162752+0000","last_undegraded":"2023-08-09T13:51:59.162752+0000","last_fullsized":"2023-08-09T13:51:59.162752+0000","mapping_epoch":114,"log_start":"0'0","ondisk_log_start":"0'0","created":114,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:51:59.130555+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:51:59.130555+0000","last_clean_scrub_stamp":"2023-08-09T13:51:59.130555+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.6","version":"0'0","reported_seq":9,"reported_epoch":114,"state":"active+clean","last_fresh":"2023-08-09T13:51:59.173431+0000","last_change":"2023-08-09T13:51:59.168280+0000","last_active":"2023-08-09T13:51:59.173431+0000","last_peered":"2023-08-09T13:51:59.173431+0000","last_clean":"2023-08-09T13:51:59.173431+0000","last_became_active":"2023-08-09T13:51:59.167980+0000","last_became_peered":"2023-08-09T13:51:59.167980+0000","last_unstale":"2023-08-09T13:51:59.173431+0000","last_undegraded":"2023-08-09T13:51:59.173431+0000","last_fullsized":"2023-08-09T13:51:59.173431+0000","mapping_epoch":113,"log_start":"0'0","ondisk_log_start":"0'0","created":113,"last_epoch_clean":114,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:51:58.125661+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:51:58.125661+0000","last_clean_scrub_stamp":"2023-08-09T13:51:58.125661+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-11T01:49:54.670225+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.4","version":"0'0","reported_seq":1,"reported_epoch":114,"state":"creating+peering","last_fresh":"2023-08-09T13:51:59.159598+0000","last_change":"2023-08-09T13:51:59.159598+0000","last_active":"2023-08-09T13:51:59.130555+0000","last_peered":"2023-08-09T13:51:59.130555+0000","last_clean":"2023-08-09T13:51:59.130555+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-09T13:51:59.159598+0000","last_undegraded":"2023-08-09T13:51:59.159598+0000","last_fullsized":"2023-08-09T13:51:59.159598+0000","mapping_epoch":114,"log_start":"0'0","ondisk_log_start":"0'0","created":114,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:51:59.130555+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:51:59.130555+0000","last_clean_scrub_stamp":"2023-08-09T13:51:59.130555+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.5","version":"0'0","reported_seq":9,"reported_epoch":114,"state":"active+clean","last_fresh":"2023-08-09T13:51:59.181921+0000","last_change":"2023-08-09T13:51:59.179649+0000","last_active":"2023-08-09T13:51:59.181921+0000","last_peered":"2023-08-09T13:51:59.181921+0000","last_clean":"2023-08-09T13:51:59.181921+0000","last_became_active":"2023-08-09T13:51:59.179348+0000","last_became_peered":"2023-08-09T13:51:59.179348+0000","last_unstale":"2023-08-09T13:51:59.181921+0000","last_undegraded":"2023-08-09T13:51:59.181921+0000","last_fullsized":"2023-08-09T13:51:59.181921+0000","mapping_epoch":113,"log_start":"0'0","ondisk_log_start":"0'0","created":113,"last_epoch_clean":114,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:51:58.125661+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:51:58.125661+0000","last_clean_scrub_stamp":"2023-08-09T13:51:58.125661+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T22:28:38.666057+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.5","version":"0'0","reported_seq":1,"reported_epoch":114,"state":"creating+peering","last_fresh":"2023-08-09T13:51:59.156952+0000","last_change":"2023-08-09T13:51:59.156952+0000","last_active":"2023-08-09T13:51:59.130555+0000","last_peered":"2023-08-09T13:51:59.130555+0000","last_clean":"2023-08-09T13:51:59.130555+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-09T13:51:59.156952+0000","last_undegraded":"2023-08-09T13:51:59.156952+0000","last_fullsized":"2023-08-09T13:51:59.156952+0000","mapping_epoch":114,"log_start":"0'0","ondisk_log_start":"0'0","created":114,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:51:59.130555+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:51:59.130555+0000","last_clean_scrub_stamp":"2023-08-09T13:51:59.130555+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"14.4","version":"0'0","reported_seq":9,"reported_epoch":114,"state":"active+clean","last_fresh":"2023-08-09T13:51:59.178493+0000","last_change":"2023-08-09T13:51:59.174311+0000","last_active":"2023-08-09T13:51:59.178493+0000","last_peered":"2023-08-09T13:51:59.178493+0000","last_clean":"2023-08-09T13:51:59.178493+0000","last_became_active":"2023-08-09T13:51:59.173990+0000","last_became_peered":"2023-08-09T13:51:59.173990+0000","last_unstale":"2023-08-09T13:51:59.178493+0000","last_undegraded":"2023-08-09T13:51:59.178493+0000","last_fullsized":"2023-08-09T13:51:59.178493+0000","mapping_epoch":113,"log_start":"0'0","ondisk_log_start":"0'0","created":113,"last_epoch_clean":114,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:51:58.125661+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:51:58.125661+0000","last_clean_scrub_stamp":"2023-08-09T13:51:58.125661+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T18:17:02.532080+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.2","version":"0'0","reported_seq":1,"reported_epoch":114,"state":"creating+peering","last_fresh":"2023-08-09T13:51:59.154951+0000","last_change":"2023-08-09T13:51:59.154951+0000","last_active":"2023-08-09T13:51:59.130555+0000","last_peered":"2023-08-09T13:51:59.130555+0000","last_clean":"2023-08-09T13:51:59.130555+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-09T13:51:59.154951+0000","last_undegraded":"2023-08-09T13:51:59.154951+0000","last_fullsized":"2023-08-09T13:51:59.154951+0000","mapping_epoch":114,"log_start":"0'0","ondisk_log_start":"0'0","created":114,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:51:59.130555+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:51:59.130555+0000","last_clean_scrub_stamp":"2023-08-09T13:51:59.130555+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"14.3","version":"0'0","reported_seq":9,"reported_epoch":114,"state":"active+clean","last_fresh":"2023-08-09T13:51:59.173345+0000","last_change":"2023-08-09T13:51:59.167523+0000","last_active":"2023-08-09T13:51:59.173345+0000","last_peered":"2023-08-09T13:51:59.173345+0000","last_clean":"2023-08-09T13:51:59.173345+0000","last_became_active":"2023-08-09T13:51:59.167059+0000","last_became_peered":"2023-08-09T13:51:59.167059+0000","last_unstale":"2023-08-09T13:51:59.173345+0000","last_undegraded":"2023-08-09T13:51:59.173345+0000","last_fullsized":"2023-08-09T13:51:59.173345+0000","mapping_epoch":113,"log_start":"0'0","ondisk_log_start":"0'0","created":113,"last_epoch_clean":114,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:51:58.125661+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:51:58.125661+0000","last_clean_scrub_stamp":"2023-08-09T13:51:58.125661+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T21:59:42.744613+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.3","version":"0'0","reported_seq":1,"reported_epoch":114,"state":"creating+peering","last_fresh":"2023-08-09T13:51:59.169608+0000","last_change":"2023-08-09T13:51:59.169608+0000","last_active":"2023-08-09T13:51:59.130555+0000","last_peered":"2023-08-09T13:51:59.130555+0000","last_clean":"2023-08-09T13:51:59.130555+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-09T13:51:59.169608+0000","last_undegraded":"2023-08-09T13:51:59.169608+0000","last_fullsized":"2023-08-09T13:51:59.169608+0000","mapping_epoch":114,"log_start":"0'0","ondisk_log_start":"0'0","created":114,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:51:59.130555+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:51:59.130555+0000","last_clean_scrub_stamp":"2023-08-09T13:51:59.130555+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.2","version":"0'0","reported_seq":9,"reported_epoch":114,"state":"active+clean","last_fresh":"2023-08-09T13:51:59.179577+0000","last_change":"2023-08-09T13:51:59.176919+0000","last_active":"2023-08-09T13:51:59.179577+0000","last_peered":"2023-08-09T13:51:59.179577+0000","last_clean":"2023-08-09T13:51:59.179577+0000","last_became_active":"2023-08-09T13:51:59.176503+0000","last_became_peered":"2023-08-09T13:51:59.176503+0000","last_unstale":"2023-08-09T13:51:59.179577+0000","last_undegraded":"2023-08-09T13:51:59.179577+0000","last_fullsized":"2023-08-09T13:51:59.179577+0000","mapping_epoch":113,"log_start":"0'0","ondisk_log_start":"0'0","created":113,"last_epoch_clean":114,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:51:58.125661+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:51:58.125661+0000","last_clean_scrub_stamp":"2023-08-09T13:51:58.125661+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T18:48:13.415415+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.0","version":"0'0","reported_seq":1,"reported_epoch":114,"state":"creating+peering","last_fresh":"2023-08-09T13:51:59.153156+0000","last_change":"2023-08-09T13:51:59.153156+0000","last_active":"2023-08-09T13:51:59.130555+0000","last_peered":"2023-08-09T13:51:59.130555+0000","last_clean":"2023-08-09T13:51:59.130555+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-09T13:51:59.153156+0000","last_undegraded":"2023-08-09T13:51:59.153156+0000","last_fullsized":"2023-08-09T13:51:59.153156+0000","mapping_epoch":114,"log_start":"0'0","ondisk_log_start":"0'0","created":114,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:51:59.130555+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:51:59.130555+0000","last_clean_scrub_stamp":"2023-08-09T13:51:59.130555+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"14.1","version":"0'0","reported_seq":9,"reported_epoch":114,"state":"active+clean","last_fresh":"2023-08-09T13:51:59.176213+0000","last_change":"2023-08-09T13:51:59.171070+0000","last_active":"2023-08-09T13:51:59.176213+0000","last_peered":"2023-08-09T13:51:59.176213+0000","last_clean":"2023-08-09T13:51:59.176213+0000","last_became_active":"2023-08-09T13:51:59.170788+0000","last_became_peered":"2023-08-09T13:51:59.170788+0000","last_unstale":"2023-08-09T13:51:59.176213+0000","last_undegraded":"2023-08-09T13:51:59.176213+0000","last_fullsized":"2023-08-09T13:51:59.176213+0000","mapping_epoch":113,"log_start":"0'0","ondisk_log_start":"0'0","created":113,"last_epoch_clean":114,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:51:58.125661+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:51:58.125661+0000","last_clean_scrub_stamp":"2023-08-09T13:51:58.125661+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-11T00:16:05.270843+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"15.1","version":"0'0","reported_seq":1,"reported_epoch":114,"state":"creating+peering","last_fresh":"2023-08-09T13:51:59.156158+0000","last_change":"2023-08-09T13:51:59.156158+0000","last_active":"2023-08-09T13:51:59.130555+0000","last_peered":"2023-08-09T13:51:59.130555+0000","last_clean":"2023-08-09T13:51:59.130555+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-09T13:51:59.156158+0000","last_undegraded":"2023-08-09T13:51:59.156158+0000","last_fullsized":"2023-08-09T13:51:59.156158+0000","mapping_epoch":114,"log_start":"0'0","ondisk_log_start":"0'0","created":114,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:51:59.130555+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:51:59.130555+0000","last_clean_scrub_stamp":"2023-08-09T13:51:59.130555+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"14.0","version":"0'0","reported_seq":9,"reported_epoch":114,"state":"active+clean","last_fresh":"2023-08-09T13:51:59.175060+0000","last_change":"2023-08-09T13:51:59.173223+0000","last_active":"2023-08-09T13:51:59.175060+0000","last_peered":"2023-08-09T13:51:59.175060+0000","last_clean":"2023-08-09T13:51:59.175060+0000","last_became_active":"2023-08-09T13:51:59.172948+0000","last_became_peered":"2023-08-09T13:51:59.172948+0000","last_unstale":"2023-08-09T13:51:59.175060+0000","last_undegraded":"2023-08-09T13:51:59.175060+0000","last_fullsized":"2023-08-09T13:51:59.175060+0000","mapping_epoch":113,"log_start":"0'0","ondisk_log_start":"0'0","created":113,"last_epoch_clean":114,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:51:58.125661+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:51:58.125661+0000","last_clean_scrub_stamp":"2023-08-09T13:51:58.125661+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T16:16:44.489876+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.e","version":"0'0","reported_seq":1,"reported_epoch":114,"state":"creating+peering","last_fresh":"2023-08-09T13:51:59.155432+0000","last_change":"2023-08-09T13:51:59.155432+0000","last_active":"2023-08-09T13:51:59.130555+0000","last_peered":"2023-08-09T13:51:59.130555+0000","last_clean":"2023-08-09T13:51:59.130555+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-09T13:51:59.155432+0000","last_undegraded":"2023-08-09T13:51:59.155432+0000","last_fullsized":"2023-08-09T13:51:59.155432+0000","mapping_epoch":114,"log_start":"0'0","ondisk_log_start":"0'0","created":114,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:51:59.130555+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:51:59.130555+0000","last_clean_scrub_stamp":"2023-08-09T13:51:59.130555+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"14.f","version":"0'0","reported_seq":9,"reported_epoch":114,"state":"active+clean","last_fresh":"2023-08-09T13:51:59.176216+0000","last_change":"2023-08-09T13:51:59.174685+0000","last_active":"2023-08-09T13:51:59.176216+0000","last_peered":"2023-08-09T13:51:59.176216+0000","last_clean":"2023-08-09T13:51:59.176216+0000","last_became_active":"2023-08-09T13:51:59.174370+0000","last_became_peered":"2023-08-09T13:51:59.174370+0000","last_unstale":"2023-08-09T13:51:59.176216+0000","last_undegraded":"2023-08-09T13:51:59.176216+0000","last_fullsized":"2023-08-09T13:51:59.176216+0000","mapping_epoch":113,"log_start":"0'0","ondisk_log_start":"0'0","created":113,"last_epoch_clean":114,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:51:58.125661+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:51:58.125661+0000","last_clean_scrub_stamp":"2023-08-09T13:51:58.125661+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T19:42:06.381104+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":189,"reported_epoch":114,"state":"active+clean","last_fresh":"2023-08-09T13:51:59.160631+0000","last_change":"2023-08-09T13:33:56.401406+0000","last_active":"2023-08-09T13:51:59.160631+0000","last_peered":"2023-08-09T13:51:59.160631+0000","last_clean":"2023-08-09T13:51:59.160631+0000","last_became_active":"2023-08-09T13:33:56.400487+0000","last_became_peered":"2023-08-09T13:33:56.400487+0000","last_unstale":"2023-08-09T13:51:59.160631+0000","last_undegraded":"2023-08-09T13:51:59.160631+0000","last_fullsized":"2023-08-09T13:51:59.160631+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:33:54.845869+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:33:54.845869+0000","last_clean_scrub_stamp":"2023-08-09T13:33:54.845869+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T16:56:32.266112+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"15.f","version":"0'0","reported_seq":1,"reported_epoch":114,"state":"creating+peering","last_fresh":"2023-08-09T13:51:59.156558+0000","last_change":"2023-08-09T13:51:59.156558+0000","last_active":"2023-08-09T13:51:59.130555+0000","last_peered":"2023-08-09T13:51:59.130555+0000","last_clean":"2023-08-09T13:51:59.130555+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-09T13:51:59.156558+0000","last_undegraded":"2023-08-09T13:51:59.156558+0000","last_fullsized":"2023-08-09T13:51:59.156558+0000","mapping_epoch":114,"log_start":"0'0","ondisk_log_start":"0'0","created":114,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:51:59.130555+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:51:59.130555+0000","last_clean_scrub_stamp":"2023-08-09T13:51:59.130555+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"14.e","version":"0'0","reported_seq":9,"reported_epoch":114,"state":"active+clean","last_fresh":"2023-08-09T13:51:59.182628+0000","last_change":"2023-08-09T13:51:59.180623+0000","last_active":"2023-08-09T13:51:59.182628+0000","last_peered":"2023-08-09T13:51:59.182628+0000","last_clean":"2023-08-09T13:51:59.182628+0000","last_became_active":"2023-08-09T13:51:59.178785+0000","last_became_peered":"2023-08-09T13:51:59.178785+0000","last_unstale":"2023-08-09T13:51:59.182628+0000","last_undegraded":"2023-08-09T13:51:59.182628+0000","last_fullsized":"2023-08-09T13:51:59.182628+0000","mapping_epoch":113,"log_start":"0'0","ondisk_log_start":"0'0","created":113,"last_epoch_clean":114,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:51:58.125661+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:51:58.125661+0000","last_clean_scrub_stamp":"2023-08-09T13:51:58.125661+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T21:35:50.978670+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":193,"reported_epoch":114,"state":"active+clean","last_fresh":"2023-08-09T13:51:59.171640+0000","last_change":"2023-08-09T13:33:56.401123+0000","last_active":"2023-08-09T13:51:59.171640+0000","last_peered":"2023-08-09T13:51:59.171640+0000","last_clean":"2023-08-09T13:51:59.171640+0000","last_became_active":"2023-08-09T13:33:56.399933+0000","last_became_peered":"2023-08-09T13:33:56.399933+0000","last_unstale":"2023-08-09T13:51:59.171640+0000","last_undegraded":"2023-08-09T13:51:59.171640+0000","last_fullsized":"2023-08-09T13:51:59.171640+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:33:54.845869+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:33:54.845869+0000","last_clean_scrub_stamp":"2023-08-09T13:33:54.845869+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T19:50:20.512873+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"15.c","version":"0'0","reported_seq":1,"reported_epoch":114,"state":"creating+peering","last_fresh":"2023-08-09T13:51:59.166603+0000","last_change":"2023-08-09T13:51:59.166603+0000","last_active":"2023-08-09T13:51:59.130555+0000","last_peered":"2023-08-09T13:51:59.130555+0000","last_clean":"2023-08-09T13:51:59.130555+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-09T13:51:59.166603+0000","last_undegraded":"2023-08-09T13:51:59.166603+0000","last_fullsized":"2023-08-09T13:51:59.166603+0000","mapping_epoch":114,"log_start":"0'0","ondisk_log_start":"0'0","created":114,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:51:59.130555+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:51:59.130555+0000","last_clean_scrub_stamp":"2023-08-09T13:51:59.130555+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"14.d","version":"0'0","reported_seq":9,"reported_epoch":114,"state":"active+clean","last_fresh":"2023-08-09T13:51:59.179009+0000","last_change":"2023-08-09T13:51:59.174909+0000","last_active":"2023-08-09T13:51:59.179009+0000","last_peered":"2023-08-09T13:51:59.179009+0000","last_clean":"2023-08-09T13:51:59.179009+0000","last_became_active":"2023-08-09T13:51:59.174527+0000","last_became_peered":"2023-08-09T13:51:59.174527+0000","last_unstale":"2023-08-09T13:51:59.179009+0000","last_undegraded":"2023-08-09T13:51:59.179009+0000","last_fullsized":"2023-08-09T13:51:59.179009+0000","mapping_epoch":113,"log_start":"0'0","ondisk_log_start":"0'0","created":113,"last_epoch_clean":114,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:51:58.125661+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:51:58.125661+0000","last_clean_scrub_stamp":"2023-08-09T13:51:58.125661+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T19:54:13.290921+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"1.2","version":"0'0","reported_seq":189,"reported_epoch":114,"state":"active+clean","last_fresh":"2023-08-09T13:51:59.161664+0000","last_change":"2023-08-09T13:33:56.402794+0000","last_active":"2023-08-09T13:51:59.161664+0000","last_peered":"2023-08-09T13:51:59.161664+0000","last_clean":"2023-08-09T13:51:59.161664+0000","last_became_active":"2023-08-09T13:33:56.402038+0000","last_became_peered":"2023-08-09T13:33:56.402038+0000","last_unstale":"2023-08-09T13:51:59.161664+0000","last_undegraded":"2023-08-09T13:51:59.161664+0000","last_fullsized":"2023-08-09T13:51:59.161664+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:33:54.845869+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:33:54.845869+0000","last_clean_scrub_stamp":"2023-08-09T13:33:54.845869+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T22:44:13.009942+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"15.d","version":"0'0","reported_seq":1,"reported_epoch":114,"state":"creating+peering","last_fresh":"2023-08-09T13:51:59.154844+0000","last_change":"2023-08-09T13:51:59.154844+0000","last_active":"2023-08-09T13:51:59.130555+0000","last_peered":"2023-08-09T13:51:59.130555+0000","last_clean":"2023-08-09T13:51:59.130555+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-09T13:51:59.154844+0000","last_undegraded":"2023-08-09T13:51:59.154844+0000","last_fullsized":"2023-08-09T13:51:59.154844+0000","mapping_epoch":114,"log_start":"0'0","ondisk_log_start":"0'0","created":114,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:51:59.130555+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:51:59.130555+0000","last_clean_scrub_stamp":"2023-08-09T13:51:59.130555+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"14.c","version":"0'0","reported_seq":9,"reported_epoch":114,"state":"active+clean","last_fresh":"2023-08-09T13:51:59.184772+0000","last_change":"2023-08-09T13:51:59.183702+0000","last_active":"2023-08-09T13:51:59.184772+0000","last_peered":"2023-08-09T13:51:59.184772+0000","last_clean":"2023-08-09T13:51:59.184772+0000","last_became_active":"2023-08-09T13:51:59.183387+0000","last_became_peered":"2023-08-09T13:51:59.183387+0000","last_unstale":"2023-08-09T13:51:59.184772+0000","last_undegraded":"2023-08-09T13:51:59.184772+0000","last_fullsized":"2023-08-09T13:51:59.184772+0000","mapping_epoch":113,"log_start":"0'0","ondisk_log_start":"0'0","created":113,"last_epoch_clean":114,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:51:58.125661+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:51:58.125661+0000","last_clean_scrub_stamp":"2023-08-09T13:51:58.125661+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T16:05:08.950799+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.3","version":"0'0","reported_seq":189,"reported_epoch":114,"state":"active+clean","last_fresh":"2023-08-09T13:51:59.169171+0000","last_change":"2023-08-09T13:33:55.886749+0000","last_active":"2023-08-09T13:51:59.169171+0000","last_peered":"2023-08-09T13:51:59.169171+0000","last_clean":"2023-08-09T13:51:59.169171+0000","last_became_active":"2023-08-09T13:33:55.886116+0000","last_became_peered":"2023-08-09T13:33:55.886116+0000","last_unstale":"2023-08-09T13:51:59.169171+0000","last_undegraded":"2023-08-09T13:51:59.169171+0000","last_fullsized":"2023-08-09T13:51:59.169171+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:33:54.845869+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:33:54.845869+0000","last_clean_scrub_stamp":"2023-08-09T13:33:54.845869+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T20:06:41.616544+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"105'186","reported_seq":583,"reported_epoch":114,"state":"active+clean","last_fresh":"2023-08-09T13:51:59.160322+0000","last_change":"2023-08-09T13:33:55.886345+0000","last_active":"2023-08-09T13:51:59.160322+0000","last_peered":"2023-08-09T13:51:59.160322+0000","last_clean":"2023-08-09T13:51:59.160322+0000","last_became_active":"2023-08-09T13:33:55.885781+0000","last_became_peered":"2023-08-09T13:33:55.885781+0000","last_unstale":"2023-08-09T13:51:59.160322+0000","last_undegraded":"2023-08-09T13:51:59.160322+0000","last_fullsized":"2023-08-09T13:51:59.160322+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:33:54.845869+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:33:54.845869+0000","last_clean_scrub_stamp":"2023-08-09T13:33:54.845869+0000","objects_scrubbed":0,"log_size":186,"log_dups_size":0,"ondisk_log_size":186,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T16:37:53.138245+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":715,"num_read_kb":1418,"num_write":283,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"15.a","version":"0'0","reported_seq":1,"reported_epoch":114,"state":"creating+peering","last_fresh":"2023-08-09T13:51:59.164187+0000","last_change":"2023-08-09T13:51:59.164187+0000","last_active":"2023-08-09T13:51:59.130555+0000","last_peered":"2023-08-09T13:51:59.130555+0000","last_clean":"2023-08-09T13:51:59.130555+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-09T13:51:59.164187+0000","last_undegraded":"2023-08-09T13:51:59.164187+0000","last_fullsized":"2023-08-09T13:51:59.164187+0000","mapping_epoch":114,"log_start":"0'0","ondisk_log_start":"0'0","created":114,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:51:59.130555+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:51:59.130555+0000","last_clean_scrub_stamp":"2023-08-09T13:51:59.130555+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"14.b","version":"0'0","reported_seq":9,"reported_epoch":114,"state":"active+clean","last_fresh":"2023-08-09T13:51:59.173500+0000","last_change":"2023-08-09T13:51:59.168603+0000","last_active":"2023-08-09T13:51:59.173500+0000","last_peered":"2023-08-09T13:51:59.173500+0000","last_clean":"2023-08-09T13:51:59.173500+0000","last_became_active":"2023-08-09T13:51:59.168273+0000","last_became_peered":"2023-08-09T13:51:59.168273+0000","last_unstale":"2023-08-09T13:51:59.173500+0000","last_undegraded":"2023-08-09T13:51:59.173500+0000","last_fullsized":"2023-08-09T13:51:59.173500+0000","mapping_epoch":113,"log_start":"0'0","ondisk_log_start":"0'0","created":113,"last_epoch_clean":114,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:51:58.125661+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:51:58.125661+0000","last_clean_scrub_stamp":"2023-08-09T13:51:58.125661+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T20:40:57.215042+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":199,"reported_epoch":114,"state":"active+clean","last_fresh":"2023-08-09T13:51:59.163213+0000","last_change":"2023-08-09T13:33:55.889565+0000","last_active":"2023-08-09T13:51:59.163213+0000","last_peered":"2023-08-09T13:51:59.163213+0000","last_clean":"2023-08-09T13:51:59.163213+0000","last_became_active":"2023-08-09T13:33:55.889164+0000","last_became_peered":"2023-08-09T13:33:55.889164+0000","last_unstale":"2023-08-09T13:51:59.163213+0000","last_undegraded":"2023-08-09T13:51:59.163213+0000","last_fullsized":"2023-08-09T13:51:59.163213+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:33:54.845869+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:33:54.845869+0000","last_clean_scrub_stamp":"2023-08-09T13:33:54.845869+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T19:12:54.459595+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"15.b","version":"0'0","reported_seq":1,"reported_epoch":114,"state":"creating+peering","last_fresh":"2023-08-09T13:51:59.156039+0000","last_change":"2023-08-09T13:51:59.156039+0000","last_active":"2023-08-09T13:51:59.130555+0000","last_peered":"2023-08-09T13:51:59.130555+0000","last_clean":"2023-08-09T13:51:59.130555+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-09T13:51:59.156039+0000","last_undegraded":"2023-08-09T13:51:59.156039+0000","last_fullsized":"2023-08-09T13:51:59.156039+0000","mapping_epoch":114,"log_start":"0'0","ondisk_log_start":"0'0","created":114,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:51:59.130555+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:51:59.130555+0000","last_clean_scrub_stamp":"2023-08-09T13:51:59.130555+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"14.a","version":"0'0","reported_seq":9,"reported_epoch":114,"state":"active+clean","last_fresh":"2023-08-09T13:51:59.182288+0000","last_change":"2023-08-09T13:51:59.179151+0000","last_active":"2023-08-09T13:51:59.182288+0000","last_peered":"2023-08-09T13:51:59.182288+0000","last_clean":"2023-08-09T13:51:59.182288+0000","last_became_active":"2023-08-09T13:51:59.178247+0000","last_became_peered":"2023-08-09T13:51:59.178247+0000","last_unstale":"2023-08-09T13:51:59.182288+0000","last_undegraded":"2023-08-09T13:51:59.182288+0000","last_fullsized":"2023-08-09T13:51:59.182288+0000","mapping_epoch":113,"log_start":"0'0","ondisk_log_start":"0'0","created":113,"last_epoch_clean":114,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:51:58.125661+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:51:58.125661+0000","last_clean_scrub_stamp":"2023-08-09T13:51:58.125661+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T14:19:12.218569+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":193,"reported_epoch":114,"state":"active+clean","last_fresh":"2023-08-09T13:51:59.180570+0000","last_change":"2023-08-09T13:33:56.404236+0000","last_active":"2023-08-09T13:51:59.180570+0000","last_peered":"2023-08-09T13:51:59.180570+0000","last_clean":"2023-08-09T13:51:59.180570+0000","last_became_active":"2023-08-09T13:33:56.402485+0000","last_became_peered":"2023-08-09T13:33:56.402485+0000","last_unstale":"2023-08-09T13:51:59.180570+0000","last_undegraded":"2023-08-09T13:51:59.180570+0000","last_fullsized":"2023-08-09T13:51:59.180570+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:33:54.845869+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:33:54.845869+0000","last_clean_scrub_stamp":"2023-08-09T13:33:54.845869+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T20:10:52.098147+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"15.8","version":"0'0","reported_seq":1,"reported_epoch":114,"state":"creating+peering","last_fresh":"2023-08-09T13:51:59.162674+0000","last_change":"2023-08-09T13:51:59.162674+0000","last_active":"2023-08-09T13:51:59.130555+0000","last_peered":"2023-08-09T13:51:59.130555+0000","last_clean":"2023-08-09T13:51:59.130555+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-09T13:51:59.162674+0000","last_undegraded":"2023-08-09T13:51:59.162674+0000","last_fullsized":"2023-08-09T13:51:59.162674+0000","mapping_epoch":114,"log_start":"0'0","ondisk_log_start":"0'0","created":114,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:51:59.130555+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:51:59.130555+0000","last_clean_scrub_stamp":"2023-08-09T13:51:59.130555+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"14.9","version":"0'0","reported_seq":9,"reported_epoch":114,"state":"active+clean","last_fresh":"2023-08-09T13:51:59.180017+0000","last_change":"2023-08-09T13:51:59.177964+0000","last_active":"2023-08-09T13:51:59.180017+0000","last_peered":"2023-08-09T13:51:59.180017+0000","last_clean":"2023-08-09T13:51:59.180017+0000","last_became_active":"2023-08-09T13:51:59.177186+0000","last_became_peered":"2023-08-09T13:51:59.177186+0000","last_unstale":"2023-08-09T13:51:59.180017+0000","last_undegraded":"2023-08-09T13:51:59.180017+0000","last_fullsized":"2023-08-09T13:51:59.180017+0000","mapping_epoch":113,"log_start":"0'0","ondisk_log_start":"0'0","created":113,"last_epoch_clean":114,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:51:58.125661+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:51:58.125661+0000","last_clean_scrub_stamp":"2023-08-09T13:51:58.125661+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T18:14:44.331723+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":189,"reported_epoch":114,"state":"active+clean","last_fresh":"2023-08-09T13:51:59.157955+0000","last_change":"2023-08-09T13:33:56.401781+0000","last_active":"2023-08-09T13:51:59.157955+0000","last_peered":"2023-08-09T13:51:59.157955+0000","last_clean":"2023-08-09T13:51:59.157955+0000","last_became_active":"2023-08-09T13:33:56.400336+0000","last_became_peered":"2023-08-09T13:33:56.400336+0000","last_unstale":"2023-08-09T13:51:59.157955+0000","last_undegraded":"2023-08-09T13:51:59.157955+0000","last_fullsized":"2023-08-09T13:51:59.157955+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:33:54.845869+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:33:54.845869+0000","last_clean_scrub_stamp":"2023-08-09T13:33:54.845869+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T16:05:37.049450+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.9","version":"0'0","reported_seq":1,"reported_epoch":114,"state":"creating+peering","last_fresh":"2023-08-09T13:51:59.157318+0000","last_change":"2023-08-09T13:51:59.157318+0000","last_active":"2023-08-09T13:51:59.130555+0000","last_peered":"2023-08-09T13:51:59.130555+0000","last_clean":"2023-08-09T13:51:59.130555+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-09T13:51:59.157318+0000","last_undegraded":"2023-08-09T13:51:59.157318+0000","last_fullsized":"2023-08-09T13:51:59.157318+0000","mapping_epoch":114,"log_start":"0'0","ondisk_log_start":"0'0","created":114,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:51:59.130555+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:51:59.130555+0000","last_clean_scrub_stamp":"2023-08-09T13:51:59.130555+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"14.8","version":"0'0","reported_seq":9,"reported_epoch":114,"state":"active+clean","last_fresh":"2023-08-09T13:51:59.174355+0000","last_change":"2023-08-09T13:51:59.170689+0000","last_active":"2023-08-09T13:51:59.174355+0000","last_peered":"2023-08-09T13:51:59.174355+0000","last_clean":"2023-08-09T13:51:59.174355+0000","last_became_active":"2023-08-09T13:51:59.170409+0000","last_became_peered":"2023-08-09T13:51:59.170409+0000","last_unstale":"2023-08-09T13:51:59.174355+0000","last_undegraded":"2023-08-09T13:51:59.174355+0000","last_fullsized":"2023-08-09T13:51:59.174355+0000","mapping_epoch":113,"log_start":"0'0","ondisk_log_start":"0'0","created":113,"last_epoch_clean":114,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:51:58.125661+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:51:58.125661+0000","last_clean_scrub_stamp":"2023-08-09T13:51:58.125661+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T17:00:34.835236+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":189,"reported_epoch":114,"state":"active+clean","last_fresh":"2023-08-09T13:51:59.159658+0000","last_change":"2023-08-09T13:33:55.886389+0000","last_active":"2023-08-09T13:51:59.159658+0000","last_peered":"2023-08-09T13:51:59.159658+0000","last_clean":"2023-08-09T13:51:59.159658+0000","last_became_active":"2023-08-09T13:33:55.885638+0000","last_became_peered":"2023-08-09T13:33:55.885638+0000","last_unstale":"2023-08-09T13:51:59.159658+0000","last_undegraded":"2023-08-09T13:51:59.159658+0000","last_fullsized":"2023-08-09T13:51:59.159658+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:33:54.845869+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:33:54.845869+0000","last_clean_scrub_stamp":"2023-08-09T13:33:54.845869+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T16:40:37.514883+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]}],"pool_stats":[{"poolid":15,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":2,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":715,"num_read_kb":1418,"num_write":283,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":186,"ondisk_log_size":186,"up":2,"acting":2,"num_store_stats":2},{"poolid":14,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":8}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834575064,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":37168,"kb_used_data":1952,"kb_used_omap":1,"kb_used_meta":35198,"kb_avail":94334672,"statfs":{"total":96636764160,"available":96598704128,"internally_reserved":0,"allocated":1998848,"data_stored":1338378,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1781,"internal_metadata":36043019},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Aug 9 13:51:20 2023","interfaces":[{"interface":"back","average":{"1min":1.101,"5min":1.1910000000000001,"15min":1.2370000000000001},"min":{"1min":0.61599999999999999,"5min":0.61599999999999999,"15min":0.61599999999999999},"max":{"1min":1.764,"5min":1.8919999999999999,"15min":2.1280000000000001},"last":1.1899999999999999},{"interface":"front","average":{"1min":1.153,"5min":1.224,"15min":1.2430000000000001},"min":{"1min":0.64200000000000002,"5min":0.64200000000000002,"15min":0.64200000000000002},"max":{"1min":1.8839999999999999,"5min":1.8839999999999999,"15min":2.0099999999999998},"last":1.1259999999999999}]},{"osd":1,"last update":"Wed Aug 9 13:51:20 2023","interfaces":[{"interface":"back","average":{"1min":1.1830000000000001,"5min":1.22,"15min":1.26},"min":{"1min":0.69199999999999995,"5min":0.69199999999999995,"15min":0.59799999999999998},"max":{"1min":1.8120000000000001,"5min":1.8120000000000001,"15min":1.9470000000000001},"last":1.4390000000000001},{"interface":"front","average":{"1min":1.1839999999999999,"5min":1.2310000000000001,"15min":1.2809999999999999},"min":{"1min":0.69099999999999995,"5min":0.59299999999999997,"15min":0.59299999999999997},"max":{"1min":1.998,"5min":1.998,"15min":1.998},"last":1.2789999999999999}]},{"osd":2,"last update":"Wed Aug 9 13:51:20 2023","interfaces":[{"interface":"back","average":{"1min":1.1990000000000001,"5min":1.234,"15min":1.252},"min":{"1min":0.79100000000000004,"5min":0.65700000000000003,"15min":0.65700000000000003},"max":{"1min":1.6539999999999999,"5min":2.0129999999999999,"15min":2.0129999999999999},"last":1.518},{"interface":"front","average":{"1min":1.22,"5min":1.2829999999999999,"15min":1.369},"min":{"1min":0.58199999999999996,"5min":0.58199999999999996,"15min":0.58199999999999996},"max":{"1min":1.7050000000000001,"5min":1.8740000000000001,"15min":2.4140000000000001},"last":1.5489999999999999}]},{"osd":3,"last update":"Wed Aug 9 13:51:20 2023","interfaces":[{"interface":"back","average":{"1min":1.272,"5min":1.29,"15min":1.3360000000000001},"min":{"1min":0.67500000000000004,"5min":0.67500000000000004,"15min":0.67500000000000004},"max":{"1min":1.8140000000000001,"5min":1.97,"15min":1.97},"last":1.6180000000000001},{"interface":"front","average":{"1min":1.363,"5min":1.3959999999999999,"15min":1.413},"min":{"1min":0.71499999999999997,"5min":0.71499999999999997,"15min":0.71499999999999997},"max":{"1min":1.615,"5min":2.0790000000000002,"15min":2.2480000000000002},"last":1.647}]},{"osd":4,"last update":"Wed Aug 9 13:51:20 2023","interfaces":[{"interface":"back","average":{"1min":1.2430000000000001,"5min":1.2669999999999999,"15min":1.2749999999999999},"min":{"1min":0.74299999999999999,"5min":0.59499999999999997,"15min":0.59499999999999997},"max":{"1min":2.097,"5min":2.097,"15min":2.343},"last":1.008},{"interface":"front","average":{"1min":1.2509999999999999,"5min":1.2969999999999999,"15min":1.3340000000000001},"min":{"1min":0.60799999999999998,"5min":0.60799999999999998,"15min":0.60799999999999998},"max":{"1min":2.0539999999999998,"5min":2.1419999999999999,"15min":2.1419999999999999},"last":1.333}]},{"osd":5,"last update":"Wed Aug 9 13:51:20 2023","interfaces":[{"interface":"back","average":{"1min":1.456,"5min":1.4350000000000001,"15min":1.4179999999999999},"min":{"1min":0.65800000000000003,"5min":0.65800000000000003,"15min":0.65800000000000003},"max":{"1min":1.7549999999999999,"5min":2.1480000000000001,"15min":2.6749999999999998},"last":1.052},{"interface":"front","average":{"1min":1.383,"5min":1.4099999999999999,"15min":1.4450000000000001},"min":{"1min":0.83999999999999997,"5min":0.83999999999999997,"15min":0.79700000000000004},"max":{"1min":2.157,"5min":2.157,"15min":2.9569999999999999},"last":1.3859999999999999}]},{"osd":6,"last update":"Wed Aug 9 13:51:20 2023","interfaces":[{"interface":"back","average":{"1min":1.4650000000000001,"5min":1.504,"15min":1.5429999999999999},"min":{"1min":0.88500000000000001,"5min":0.85499999999999998,"15min":0.85499999999999998},"max":{"1min":1.8280000000000001,"5min":2.2069999999999999,"15min":2.8149999999999999},"last":1.4870000000000001},{"interface":"front","average":{"1min":1.421,"5min":1.4430000000000001,"15min":1.474},"min":{"1min":0.79700000000000004,"5min":0.79700000000000004,"15min":0.79700000000000004},"max":{"1min":1.736,"5min":2.1320000000000001,"15min":2.2530000000000001},"last":1.7170000000000001}]}]},{"osd":6,"up_from":13,"seq":55834575064,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":36580,"kb_used_data":1368,"kb_used_omap":1,"kb_used_meta":35198,"kb_avail":94335260,"statfs":{"total":96636764160,"available":96599306240,"internally_reserved":0,"allocated":1400832,"data_stored":746649,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1781,"internal_metadata":36043019},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":8,"apply_latency_ms":8,"commit_latency_ns":8000000,"apply_latency_ns":8000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Aug 9 13:51:24 2023","interfaces":[{"interface":"back","average":{"1min":0.98499999999999999,"5min":1.1160000000000001,"15min":1.153},"min":{"1min":0.54000000000000004,"5min":0.54000000000000004,"15min":0.54000000000000004},"max":{"1min":1.484,"5min":1.633,"15min":1.9870000000000001},"last":0.96899999999999997},{"interface":"front","average":{"1min":1.0620000000000001,"5min":1.492,"15min":1.306},"min":{"1min":0.51800000000000002,"5min":0.51800000000000002,"15min":0.51800000000000002},"max":{"1min":1.6479999999999999,"5min":28.036000000000001,"15min":28.036000000000001},"last":1.0800000000000001}]},{"osd":1,"last update":"Wed Aug 9 13:51:24 2023","interfaces":[{"interface":"back","average":{"1min":1.0940000000000001,"5min":1.238,"15min":1.232},"min":{"1min":0.48999999999999999,"5min":0.48999999999999999,"15min":0.48999999999999999},"max":{"1min":1.6639999999999999,"5min":7.742,"15min":7.742},"last":0.79800000000000004},{"interface":"front","average":{"1min":1.1950000000000001,"5min":1.276,"15min":1.28},"min":{"1min":0.64600000000000002,"5min":0.64600000000000002,"15min":0.64600000000000002},"max":{"1min":2.056,"5min":2.056,"15min":2.056},"last":1.222}]},{"osd":2,"last update":"Wed Aug 9 13:51:24 2023","interfaces":[{"interface":"back","average":{"1min":1.206,"5min":1.323,"15min":1.3260000000000001},"min":{"1min":0.80600000000000005,"5min":0.80600000000000005,"15min":0.74099999999999999},"max":{"1min":1.8819999999999999,"5min":4.0640000000000001,"15min":4.0640000000000001},"last":1.147},{"interface":"front","average":{"1min":1.161,"5min":1.399,"15min":1.347},"min":{"1min":0.80700000000000005,"5min":0.70999999999999996,"15min":0.70999999999999996},"max":{"1min":1.6679999999999999,"5min":13.154999999999999,"15min":13.154999999999999},"last":1.5029999999999999}]},{"osd":3,"last update":"Wed Aug 9 13:51:24 2023","interfaces":[{"interface":"back","average":{"1min":1.196,"5min":1.452,"15min":1.415},"min":{"1min":0.67200000000000004,"5min":0.67200000000000004,"15min":0.67200000000000004},"max":{"1min":1.522,"5min":13.349,"15min":13.349},"last":1.401},{"interface":"front","average":{"1min":1.254,"5min":1.353,"15min":1.3280000000000001},"min":{"1min":0.75700000000000001,"5min":0.75700000000000001,"15min":0.74299999999999999},"max":{"1min":1.732,"5min":7.9029999999999996,"15min":7.9029999999999996},"last":1.319}]},{"osd":4,"last update":"Wed Aug 9 13:51:24 2023","interfaces":[{"interface":"back","average":{"1min":1.323,"5min":1.3540000000000001,"15min":1.284},"min":{"1min":0.86199999999999999,"5min":0.84499999999999997,"15min":0.64300000000000002},"max":{"1min":1.833,"5min":5.2290000000000001,"15min":5.2290000000000001},"last":1.3640000000000001},{"interface":"front","average":{"1min":1.367,"5min":1.506,"15min":1.3879999999999999},"min":{"1min":0.89900000000000002,"5min":0.78700000000000003,"15min":0.78700000000000003},"max":{"1min":1.714,"5min":12.1,"15min":12.1},"last":1.27}]},{"osd":5,"last update":"Wed Aug 9 13:51:24 2023","interfaces":[{"interface":"back","average":{"1min":1.423,"5min":1.446,"15min":1.3660000000000001},"min":{"1min":0.57699999999999996,"5min":0.57699999999999996,"15min":0.57699999999999996},"max":{"1min":1.843,"5min":8.1899999999999995,"15min":8.1899999999999995},"last":1.1850000000000001},{"interface":"front","average":{"1min":1.55,"5min":1.6679999999999999,"15min":1.593},"min":{"1min":0.93000000000000005,"5min":0.93000000000000005,"15min":0.83799999999999997},"max":{"1min":2.1459999999999999,"5min":8.8889999999999993,"15min":8.8889999999999993},"last":1.4379999999999999}]},{"osd":7,"last update":"Wed Aug 9 13:51:24 2023","interfaces":[{"interface":"back","average":{"1min":1.5269999999999999,"5min":1.6719999999999999,"15min":1.587},"min":{"1min":0.96099999999999997,"5min":0.96099999999999997,"15min":0.62},"max":{"1min":1.907,"5min":9.3320000000000007,"15min":9.3320000000000007},"last":1.6359999999999999},{"interface":"front","average":{"1min":1.556,"5min":1.613,"15min":1.55},"min":{"1min":0.88200000000000001,"5min":0.88200000000000001,"15min":0.88200000000000001},"max":{"1min":1.97,"5min":8.0399999999999991,"15min":8.0399999999999991},"last":1.4690000000000001}]}]},{"osd":5,"up_from":13,"seq":55834575064,"num_pgs":5,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":40668,"kb_used_data":1360,"kb_used_omap":1,"kb_used_meta":39294,"kb_avail":94331172,"statfs":{"total":96636764160,"available":96595120128,"internally_reserved":0,"allocated":1392640,"data_stored":741018,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1781,"internal_metadata":40237323},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Aug 9 13:51:21 2023","interfaces":[{"interface":"back","average":{"1min":1.542,"5min":1.25,"15min":1.206},"min":{"1min":0.59099999999999997,"5min":0.59099999999999997,"15min":0.59099999999999997},"max":{"1min":8.9299999999999997,"5min":8.9299999999999997,"15min":8.9299999999999997},"last":1.2350000000000001},{"interface":"front","average":{"1min":1.6080000000000001,"5min":1.264,"15min":1.2330000000000001},"min":{"1min":0.60499999999999998,"5min":0.60499999999999998,"15min":0.60499999999999998},"max":{"1min":8.7140000000000004,"5min":8.7140000000000004,"15min":8.7140000000000004},"last":1.6160000000000001}]},{"osd":1,"last update":"Wed Aug 9 13:51:21 2023","interfaces":[{"interface":"back","average":{"1min":1.476,"5min":1.2330000000000001,"15min":1.246},"min":{"1min":0.68799999999999994,"5min":0.68799999999999994,"15min":0.64100000000000001},"max":{"1min":8.2720000000000002,"5min":8.2720000000000002,"15min":8.2720000000000002},"last":0.82399999999999995},{"interface":"front","average":{"1min":1.6679999999999999,"5min":1.359,"15min":1.3100000000000001},"min":{"1min":0.80500000000000005,"5min":0.66900000000000004,"15min":0.66900000000000004},"max":{"1min":9.0020000000000007,"5min":9.0020000000000007,"15min":9.0020000000000007},"last":1.3720000000000001}]},{"osd":2,"last update":"Wed Aug 9 13:51:21 2023","interfaces":[{"interface":"back","average":{"1min":1.7190000000000001,"5min":1.335,"15min":1.319},"min":{"1min":0.81799999999999995,"5min":0.68999999999999995,"15min":0.61599999999999999},"max":{"1min":8.6050000000000004,"5min":8.6050000000000004,"15min":8.6050000000000004},"last":1.1439999999999999},{"interface":"front","average":{"1min":1.641,"5min":1.371,"15min":1.347},"min":{"1min":0.78300000000000003,"5min":0.78300000000000003,"15min":0.72099999999999997},"max":{"1min":8.4149999999999991,"5min":8.4149999999999991,"15min":8.4149999999999991},"last":1.4299999999999999}]},{"osd":3,"last update":"Wed Aug 9 13:51:21 2023","interfaces":[{"interface":"back","average":{"1min":1.8160000000000001,"5min":1.466,"15min":1.391},"min":{"1min":0.72599999999999998,"5min":0.72599999999999998,"15min":0.59799999999999998},"max":{"1min":9.0470000000000006,"5min":9.0470000000000006,"15min":9.0470000000000006},"last":1.573},{"interface":"front","average":{"1min":1.738,"5min":1.4970000000000001,"15min":1.4370000000000001},"min":{"1min":0.76300000000000001,"5min":0.76300000000000001,"15min":0.70999999999999996},"max":{"1min":8.7669999999999995,"5min":8.7669999999999995,"15min":8.7669999999999995},"last":1.302}]},{"osd":4,"last update":"Wed Aug 9 13:51:21 2023","interfaces":[{"interface":"back","average":{"1min":1.724,"5min":1.367,"15min":1.321},"min":{"1min":0.54300000000000004,"5min":0.54300000000000004,"15min":0.54300000000000004},"max":{"1min":8.4909999999999997,"5min":8.4909999999999997,"15min":8.4909999999999997},"last":1.55},{"interface":"front","average":{"1min":1.9079999999999999,"5min":1.4570000000000001,"15min":1.4119999999999999},"min":{"1min":0.68899999999999995,"5min":0.68899999999999995,"15min":0.68899999999999995},"max":{"1min":9.4589999999999996,"5min":9.4589999999999996,"15min":9.4589999999999996},"last":1.6579999999999999}]},{"osd":6,"last update":"Wed Aug 9 13:51:21 2023","interfaces":[{"interface":"back","average":{"1min":2.5259999999999998,"5min":1.5980000000000001,"15min":1.4610000000000001},"min":{"1min":0.51000000000000001,"5min":0.51000000000000001,"15min":0.51000000000000001},"max":{"1min":17.407,"5min":17.407,"15min":17.407},"last":1.5249999999999999},{"interface":"front","average":{"1min":2.351,"5min":1.6279999999999999,"15min":1.53},"min":{"1min":0.71199999999999997,"5min":0.71199999999999997,"15min":0.71199999999999997},"max":{"1min":16.356000000000002,"5min":16.356000000000002,"15min":16.356000000000002},"last":1.7350000000000001}]},{"osd":7,"last update":"Wed Aug 9 13:51:21 2023","interfaces":[{"interface":"back","average":{"1min":2.117,"5min":1.665,"15min":1.591},"min":{"1min":0.74299999999999999,"5min":0.74299999999999999,"15min":0.74299999999999999},"max":{"1min":9.9239999999999995,"5min":9.9239999999999995,"15min":9.9239999999999995},"last":1.482},{"interface":"front","average":{"1min":2.2400000000000002,"5min":1.716,"15min":1.645},"min":{"1min":0.65600000000000003,"5min":0.65600000000000003,"15min":0.65600000000000003},"max":{"1min":10.789,"5min":10.789,"15min":10.789},"last":1.6919999999999999}]}]},{"osd":4,"up_from":13,"seq":55834575064,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32488,"kb_used_data":1372,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":94339352,"statfs":{"total":96636764160,"available":96603496448,"internally_reserved":0,"allocated":1404928,"data_stored":748010,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1781,"internal_metadata":31848715},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Aug 9 13:51:23 2023","interfaces":[{"interface":"back","average":{"1min":1.131,"5min":1.2150000000000001,"15min":1.2070000000000001},"min":{"1min":0.66600000000000004,"5min":0.66600000000000004,"15min":0.66600000000000004},"max":{"1min":1.768,"5min":2.524,"15min":2.524},"last":1.349},{"interface":"front","average":{"1min":1.0609999999999999,"5min":1.181,"15min":1.25},"min":{"1min":0.66800000000000004,"5min":0.62,"15min":0.62},"max":{"1min":1.667,"5min":1.7390000000000001,"15min":1.863},"last":1.4770000000000001}]},{"osd":1,"last update":"Wed Aug 9 13:51:23 2023","interfaces":[{"interface":"back","average":{"1min":1.1839999999999999,"5min":1.264,"15min":1.2969999999999999},"min":{"1min":0.79700000000000004,"5min":0.67800000000000005,"15min":0.67800000000000005},"max":{"1min":1.609,"5min":1.8480000000000001,"15min":1.8480000000000001},"last":1.2},{"interface":"front","average":{"1min":1.1319999999999999,"5min":1.232,"15min":1.2470000000000001},"min":{"1min":0.75,"5min":0.75,"15min":0.65400000000000003},"max":{"1min":1.669,"5min":1.9330000000000001,"15min":1.9330000000000001},"last":1.153}]},{"osd":2,"last update":"Wed Aug 9 13:51:23 2023","interfaces":[{"interface":"back","average":{"1min":1.081,"5min":1.2509999999999999,"15min":1.272},"min":{"1min":0.69999999999999996,"5min":0.621,"15min":0.621},"max":{"1min":1.496,"5min":2.6429999999999998,"15min":2.6429999999999998},"last":1.522},{"interface":"front","average":{"1min":1.25,"5min":1.3220000000000001,"15min":1.365},"min":{"1min":0.76000000000000001,"5min":0.76000000000000001,"15min":0.75900000000000001},"max":{"1min":2.0910000000000002,"5min":2.0910000000000002,"15min":2.0910000000000002},"last":1.667}]},{"osd":3,"last update":"Wed Aug 9 13:51:23 2023","interfaces":[{"interface":"back","average":{"1min":1.1599999999999999,"5min":1.2889999999999999,"15min":1.298},"min":{"1min":0.754,"5min":0.754,"15min":0.754},"max":{"1min":1.921,"5min":2.6099999999999999,"15min":2.6099999999999999},"last":0.84499999999999997},{"interface":"front","average":{"1min":1.2509999999999999,"5min":1.3740000000000001,"15min":1.3819999999999999},"min":{"1min":0.89100000000000001,"5min":0.88800000000000001,"15min":0.63100000000000001},"max":{"1min":1.7010000000000001,"5min":2.7629999999999999,"15min":2.7629999999999999},"last":1.694}]},{"osd":5,"last update":"Wed Aug 9 13:51:23 2023","interfaces":[{"interface":"back","average":{"1min":1.294,"5min":1.3140000000000001,"15min":1.327},"min":{"1min":0.67300000000000004,"5min":0.67300000000000004,"15min":0.63200000000000001},"max":{"1min":1.758,"5min":1.8,"15min":2.0099999999999998},"last":1.73},{"interface":"front","average":{"1min":1.361,"5min":1.337,"15min":1.3169999999999999},"min":{"1min":0.65600000000000003,"5min":0.65600000000000003,"15min":0.626},"max":{"1min":2.0369999999999999,"5min":2.7090000000000001,"15min":2.7090000000000001},"last":1.298}]},{"osd":6,"last update":"Wed Aug 9 13:51:23 2023","interfaces":[{"interface":"back","average":{"1min":1.409,"5min":1.341,"15min":1.3340000000000001},"min":{"1min":0.90600000000000003,"5min":0.74399999999999999,"15min":0.59099999999999997},"max":{"1min":1.96,"5min":2.8559999999999999,"15min":2.8559999999999999},"last":1.758},{"interface":"front","average":{"1min":1.496,"5min":1.514,"15min":1.5029999999999999},"min":{"1min":0.878,"5min":0.878,"15min":0.76400000000000001},"max":{"1min":2.1890000000000001,"5min":2.2719999999999998,"15min":2.2719999999999998},"last":1.9410000000000001}]},{"osd":7,"last update":"Wed Aug 9 13:51:23 2023","interfaces":[{"interface":"back","average":{"1min":1.3859999999999999,"5min":1.468,"15min":1.4690000000000001},"min":{"1min":0.53100000000000003,"5min":0.53100000000000003,"15min":0.53100000000000003},"max":{"1min":1.9530000000000001,"5min":2.8050000000000002,"15min":2.8050000000000002},"last":1.5940000000000001},{"interface":"front","average":{"1min":1.5289999999999999,"5min":1.52,"15min":1.5129999999999999},"min":{"1min":0.78700000000000003,"5min":0.78700000000000003,"15min":0.72899999999999998},"max":{"1min":2.1419999999999999,"5min":3.415,"15min":3.415},"last":1.976}]}]},{"osd":3,"up_from":13,"seq":55834575065,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":36572,"kb_used_data":1360,"kb_used_omap":1,"kb_used_meta":35198,"kb_avail":94335268,"statfs":{"total":96636764160,"available":96599314432,"internally_reserved":0,"allocated":1392640,"data_stored":741018,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1781,"internal_metadata":36043019},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Aug 9 13:51:26 2023","interfaces":[{"interface":"back","average":{"1min":0.76900000000000002,"5min":0.85299999999999998,"15min":0.90500000000000003},"min":{"1min":0.496,"5min":0.27200000000000002,"15min":0.27200000000000002},"max":{"1min":1.5580000000000001,"5min":1.8500000000000001,"15min":1.968},"last":0.99099999999999999},{"interface":"front","average":{"1min":0.83499999999999996,"5min":1.0349999999999999,"15min":1.097},"min":{"1min":0.36299999999999999,"5min":0.36299999999999999,"15min":0.36299999999999999},"max":{"1min":1.4059999999999999,"5min":1.6910000000000001,"15min":2.0590000000000002},"last":1.3200000000000001}]},{"osd":1,"last update":"Wed Aug 9 13:51:26 2023","interfaces":[{"interface":"back","average":{"1min":0.82799999999999996,"5min":0.95399999999999996,"15min":1.0089999999999999},"min":{"1min":0.51500000000000001,"5min":0.496,"15min":0.438},"max":{"1min":1.27,"5min":1.6519999999999999,"15min":2.5649999999999999},"last":1.038},{"interface":"front","average":{"1min":0.97899999999999998,"5min":1.081,"15min":1.161},"min":{"1min":0.60499999999999998,"5min":0.51800000000000002,"15min":0.498},"max":{"1min":1.6779999999999999,"5min":1.6839999999999999,"15min":2.6760000000000002},"last":1.393}]},{"osd":2,"last update":"Wed Aug 9 13:51:26 2023","interfaces":[{"interface":"back","average":{"1min":0.995,"5min":1.115,"15min":1.194},"min":{"1min":0.42999999999999999,"5min":0.42999999999999999,"15min":0.42999999999999999},"max":{"1min":1.6879999999999999,"5min":1.913,"15min":1.923},"last":1.494},{"interface":"front","average":{"1min":1.103,"5min":1.2290000000000001,"15min":1.3069999999999999},"min":{"1min":0.61299999999999999,"5min":0.52300000000000002,"15min":0.52300000000000002},"max":{"1min":1.7290000000000001,"5min":1.9399999999999999,"15min":2.0259999999999998},"last":1.254}]},{"osd":4,"last update":"Wed Aug 9 13:51:26 2023","interfaces":[{"interface":"back","average":{"1min":1.1000000000000001,"5min":1.214,"15min":1.274},"min":{"1min":0.54900000000000004,"5min":0.54900000000000004,"15min":0.54900000000000004},"max":{"1min":1.732,"5min":1.8160000000000001,"15min":1.8160000000000001},"last":1.7070000000000001},{"interface":"front","average":{"1min":1.2769999999999999,"5min":1.3440000000000001,"15min":1.3959999999999999},"min":{"1min":0.76100000000000001,"5min":0.628,"15min":0.628},"max":{"1min":2.782,"5min":2.782,"15min":2.782},"last":1.7629999999999999}]},{"osd":5,"last update":"Wed Aug 9 13:51:26 2023","interfaces":[{"interface":"back","average":{"1min":1.266,"5min":1.4179999999999999,"15min":1.47},"min":{"1min":0.69899999999999995,"5min":0.67600000000000005,"15min":0.67200000000000004},"max":{"1min":1.833,"5min":2.254,"15min":2.254},"last":2.012},{"interface":"front","average":{"1min":1.258,"5min":1.431,"15min":1.484},"min":{"1min":0.66000000000000003,"5min":0.65900000000000003,"15min":0.65900000000000003},"max":{"1min":1.919,"5min":2.3069999999999999,"15min":2.3069999999999999},"last":1.6060000000000001}]},{"osd":6,"last update":"Wed Aug 9 13:51:26 2023","interfaces":[{"interface":"back","average":{"1min":1.381,"5min":1.419,"15min":1.4630000000000001},"min":{"1min":0.78200000000000003,"5min":0.68200000000000005,"15min":0.68200000000000005},"max":{"1min":2.8340000000000001,"5min":2.8340000000000001,"15min":2.8340000000000001},"last":1.5620000000000001},{"interface":"front","average":{"1min":1.389,"5min":1.5109999999999999,"15min":1.5580000000000001},"min":{"1min":0.64100000000000001,"5min":0.64100000000000001,"15min":0.64100000000000001},"max":{"1min":2.9060000000000001,"5min":2.9060000000000001,"15min":2.9060000000000001},"last":2.1480000000000001}]},{"osd":7,"last update":"Wed Aug 9 13:51:26 2023","interfaces":[{"interface":"back","average":{"1min":1.3580000000000001,"5min":1.5549999999999999,"15min":1.6100000000000001},"min":{"1min":0.77800000000000002,"5min":0.69399999999999995,"15min":0.69399999999999995},"max":{"1min":2.0419999999999998,"5min":2.46,"15min":2.46},"last":1.966},{"interface":"front","average":{"1min":1.403,"5min":1.5049999999999999,"15min":1.5620000000000001},"min":{"1min":0.93799999999999994,"5min":0.64200000000000002,"15min":0.64200000000000002},"max":{"1min":2.7200000000000002,"5min":2.7200000000000002,"15min":2.7200000000000002},"last":1.804}]}]},{"osd":2,"up_from":13,"seq":55834575064,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":36572,"kb_used_data":1360,"kb_used_omap":1,"kb_used_meta":35198,"kb_avail":94335268,"statfs":{"total":96636764160,"available":96599314432,"internally_reserved":0,"allocated":1392640,"data_stored":741018,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1781,"internal_metadata":36043019},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Aug 9 13:51:19 2023","interfaces":[{"interface":"back","average":{"1min":0.82699999999999996,"5min":0.91400000000000003,"15min":0.95599999999999996},"min":{"1min":0.47199999999999998,"5min":0.36499999999999999,"15min":0.36499999999999999},"max":{"1min":1.484,"5min":1.5940000000000001,"15min":1.8380000000000001},"last":0.749},{"interface":"front","average":{"1min":0.90200000000000002,"5min":0.96699999999999997,"15min":1.0009999999999999},"min":{"1min":0.38200000000000001,"5min":0.374,"15min":0.374},"max":{"1min":1.7,"5min":2.3929999999999998,"15min":2.3929999999999998},"last":0.995}]},{"osd":1,"last update":"Wed Aug 9 13:51:19 2023","interfaces":[{"interface":"back","average":{"1min":1.038,"5min":1.143,"15min":1.1839999999999999},"min":{"1min":0.39400000000000002,"5min":0.39400000000000002,"15min":0.39400000000000002},"max":{"1min":1.615,"5min":1.77,"15min":1.8420000000000001},"last":1.149},{"interface":"front","average":{"1min":0.98399999999999999,"5min":1.171,"15min":1.3220000000000001},"min":{"1min":0.435,"5min":0.435,"15min":0.435},"max":{"1min":1.659,"5min":2.4870000000000001,"15min":2.4870000000000001},"last":1.1879999999999999}]},{"osd":3,"last update":"Wed Aug 9 13:51:19 2023","interfaces":[{"interface":"back","average":{"1min":0.93100000000000005,"5min":0.97999999999999998,"15min":1.0509999999999999},"min":{"1min":0.55300000000000005,"5min":0.38900000000000001,"15min":0.38900000000000001},"max":{"1min":1.417,"5min":1.708,"15min":2.0409999999999999},"last":0.93400000000000005},{"interface":"front","average":{"1min":1.0820000000000001,"5min":1.2549999999999999,"15min":1.3080000000000001},"min":{"1min":0.52900000000000003,"5min":0.46200000000000002,"15min":0.46200000000000002},"max":{"1min":1.6279999999999999,"5min":8.8130000000000006,"15min":8.8130000000000006},"last":1.304}]},{"osd":4,"last update":"Wed Aug 9 13:51:19 2023","interfaces":[{"interface":"back","average":{"1min":1.335,"5min":1.3740000000000001,"15min":1.452},"min":{"1min":0.81000000000000005,"5min":0.628,"15min":0.628},"max":{"1min":1.8340000000000001,"5min":2.089,"15min":2.089},"last":1.5109999999999999},{"interface":"front","average":{"1min":1.2969999999999999,"5min":1.3360000000000001,"15min":1.3939999999999999},"min":{"1min":0.75900000000000001,"5min":0.68600000000000005,"15min":0.68600000000000005},"max":{"1min":1.7509999999999999,"5min":2.472,"15min":2.472},"last":1.5469999999999999}]},{"osd":5,"last update":"Wed Aug 9 13:51:19 2023","interfaces":[{"interface":"back","average":{"1min":1.361,"5min":1.4059999999999999,"15min":1.4590000000000001},"min":{"1min":0.86399999999999999,"5min":0.59999999999999998,"15min":0.59999999999999998},"max":{"1min":2.0640000000000001,"5min":2.5840000000000001,"15min":2.5840000000000001},"last":1.383},{"interface":"front","average":{"1min":1.2829999999999999,"5min":1.3260000000000001,"15min":1.351},"min":{"1min":0.89000000000000001,"5min":0.56799999999999995,"15min":0.56799999999999995},"max":{"1min":1.78,"5min":3.5659999999999998,"15min":3.5659999999999998},"last":1.2370000000000001}]},{"osd":6,"last update":"Wed Aug 9 13:51:19 2023","interfaces":[{"interface":"back","average":{"1min":1.3009999999999999,"5min":1.3879999999999999,"15min":1.429},"min":{"1min":0.82199999999999995,"5min":0.73799999999999999,"15min":0.73799999999999999},"max":{"1min":2.1200000000000001,"5min":3.5099999999999998,"15min":3.5099999999999998},"last":1.3540000000000001},{"interface":"front","average":{"1min":1.4099999999999999,"5min":1.458,"15min":1.54},"min":{"1min":0.84899999999999998,"5min":0.66300000000000003,"15min":0.66300000000000003},"max":{"1min":2.04,"5min":2.3769999999999998,"15min":2.3769999999999998},"last":1.4279999999999999}]},{"osd":7,"last update":"Wed Aug 9 13:51:19 2023","interfaces":[{"interface":"back","average":{"1min":1.3620000000000001,"5min":1.452,"15min":1.54},"min":{"1min":0.85099999999999998,"5min":0.64600000000000002,"15min":0.64600000000000002},"max":{"1min":1.865,"5min":2.5249999999999999,"15min":2.5249999999999999},"last":1.6200000000000001},{"interface":"front","average":{"1min":1.3939999999999999,"5min":1.476,"15min":1.514},"min":{"1min":0.94699999999999995,"5min":0.70499999999999996,"15min":0.70499999999999996},"max":{"1min":1.891,"5min":3.609,"15min":3.609},"last":1.4139999999999999}]}]},{"osd":1,"up_from":13,"seq":55834575064,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":47008,"kb_used_data":1940,"kb_used_omap":1,"kb_used_meta":45054,"kb_avail":94324832,"statfs":{"total":96636764160,"available":96588627968,"internally_reserved":0,"allocated":1986560,"data_stored":1331386,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1781,"internal_metadata":46135563},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Aug 9 13:51:34 2023","interfaces":[{"interface":"back","average":{"1min":0.77600000000000002,"5min":0.90000000000000002,"15min":0.93799999999999994},"min":{"1min":0.45800000000000002,"5min":0.45800000000000002,"15min":0.45800000000000002},"max":{"1min":1.1910000000000001,"5min":1.7110000000000001,"15min":2.1080000000000001},"last":0.57599999999999996},{"interface":"front","average":{"1min":0.83299999999999996,"5min":1.0129999999999999,"15min":1.095},"min":{"1min":0.60599999999999998,"5min":0.432,"15min":0.432},"max":{"1min":1.6519999999999999,"5min":2.468,"15min":2.468},"last":0.40699999999999997}]},{"osd":2,"last update":"Wed Aug 9 13:51:34 2023","interfaces":[{"interface":"back","average":{"1min":0.83399999999999996,"5min":0.93799999999999994,"15min":0.99299999999999999},"min":{"1min":0.57499999999999996,"5min":0.41099999999999998,"15min":0.41099999999999998},"max":{"1min":1.2130000000000001,"5min":1.907,"15min":1.907},"last":0.52200000000000002},{"interface":"front","average":{"1min":0.91100000000000003,"5min":1.0920000000000001,"15min":1.1859999999999999},"min":{"1min":0.59699999999999998,"5min":0.54400000000000004,"15min":0.54400000000000004},"max":{"1min":1.45,"5min":2.1280000000000001,"15min":2.145},"last":0.50700000000000001}]},{"osd":3,"last update":"Wed Aug 9 13:51:34 2023","interfaces":[{"interface":"back","average":{"1min":0.97199999999999998,"5min":1.1890000000000001,"15min":1.3220000000000001},"min":{"1min":0.60799999999999998,"5min":0.54400000000000004,"15min":0.54400000000000004},"max":{"1min":1.466,"5min":2.0270000000000001,"15min":2.3210000000000002},"last":0.54600000000000004},{"interface":"front","average":{"1min":1.042,"5min":1.163,"15min":1.25},"min":{"1min":0.625,"5min":0.49299999999999999,"15min":0.45300000000000001},"max":{"1min":1.6539999999999999,"5min":2.3540000000000001,"15min":2.7669999999999999},"last":0.48299999999999998}]},{"osd":4,"last update":"Wed Aug 9 13:51:34 2023","interfaces":[{"interface":"back","average":{"1min":1.1819999999999999,"5min":1.337,"15min":1.4059999999999999},"min":{"1min":0.79800000000000004,"5min":0.79800000000000004,"15min":0.58099999999999996},"max":{"1min":1.9259999999999999,"5min":1.97,"15min":2.5},"last":0.91100000000000003},{"interface":"front","average":{"1min":1.157,"5min":1.27,"15min":1.329},"min":{"1min":0.76600000000000001,"5min":0.70099999999999996,"15min":0.70099999999999996},"max":{"1min":1.895,"5min":1.895,"15min":1.901},"last":0.71099999999999997}]},{"osd":5,"last update":"Wed Aug 9 13:51:34 2023","interfaces":[{"interface":"back","average":{"1min":1.2150000000000001,"5min":1.29,"15min":1.3680000000000001},"min":{"1min":0.76800000000000002,"5min":0.76800000000000002,"15min":0.76800000000000002},"max":{"1min":1.73,"5min":2.294,"15min":2.294},"last":0.75900000000000001},{"interface":"front","average":{"1min":1.2949999999999999,"5min":1.3999999999999999,"15min":1.512},"min":{"1min":0.89200000000000002,"5min":0.89200000000000002,"15min":0.89200000000000002},"max":{"1min":1.6859999999999999,"5min":2.1389999999999998,"15min":2.4649999999999999},"last":0.78800000000000003}]},{"osd":6,"last update":"Wed Aug 9 13:51:34 2023","interfaces":[{"interface":"back","average":{"1min":1.325,"5min":1.4339999999999999,"15min":1.5149999999999999},"min":{"1min":0.96599999999999997,"5min":0.77500000000000002,"15min":0.67400000000000004},"max":{"1min":1.5780000000000001,"5min":2.2080000000000002,"15min":2.5270000000000001},"last":0.97599999999999998},{"interface":"front","average":{"1min":1.28,"5min":1.3979999999999999,"15min":1.4990000000000001},"min":{"1min":0.73599999999999999,"5min":0.73599999999999999,"15min":0.73599999999999999},"max":{"1min":1.591,"5min":2.02,"15min":2.1040000000000001},"last":0.80600000000000005}]},{"osd":7,"last update":"Wed Aug 9 13:51:34 2023","interfaces":[{"interface":"back","average":{"1min":1.3240000000000001,"5min":1.4930000000000001,"15min":1.615},"min":{"1min":0.93700000000000006,"5min":0.86899999999999999,"15min":0.86899999999999999},"max":{"1min":1.776,"5min":2.4169999999999998,"15min":2.5790000000000002},"last":0.81899999999999995},{"interface":"front","average":{"1min":1.3420000000000001,"5min":1.488,"15min":1.5980000000000001},"min":{"1min":0.96899999999999997,"5min":0.91300000000000003,"15min":0.89500000000000002},"max":{"1min":1.8320000000000001,"5min":2.1040000000000001,"15min":2.6509999999999998},"last":1.083}]}]},{"osd":0,"up_from":13,"seq":55834575064,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":40664,"kb_used_data":1356,"kb_used_omap":1,"kb_used_meta":39294,"kb_avail":94331176,"statfs":{"total":96636764160,"available":96595124224,"internally_reserved":0,"allocated":1388544,"data_stored":739657,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1781,"internal_metadata":40237323},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Wed Aug 9 13:51:27 2023","interfaces":[{"interface":"back","average":{"1min":0.95899999999999996,"5min":0.91300000000000003,"15min":0.92100000000000004},"min":{"1min":0.41199999999999998,"5min":0.32500000000000001,"15min":0.32500000000000001},"max":{"1min":1.4990000000000001,"5min":1.756,"15min":1.756},"last":1.528},{"interface":"front","average":{"1min":1.002,"5min":1.077,"15min":1.111},"min":{"1min":0.39200000000000002,"5min":0.39200000000000002,"15min":0.39200000000000002},"max":{"1min":1.546,"5min":1.9019999999999999,"15min":1.9019999999999999},"last":0.81999999999999995}]},{"osd":2,"last update":"Wed Aug 9 13:51:27 2023","interfaces":[{"interface":"back","average":{"1min":1,"5min":1.0680000000000001,"15min":1.1339999999999999},"min":{"1min":0.58399999999999996,"5min":0.51400000000000001,"15min":0.503},"max":{"1min":1.5089999999999999,"5min":1.776,"15min":1.889},"last":1.1639999999999999},{"interface":"front","average":{"1min":0.98799999999999999,"5min":1.1020000000000001,"15min":1.1739999999999999},"min":{"1min":0.439,"5min":0.439,"15min":0.439},"max":{"1min":1.6259999999999999,"5min":2.0129999999999999,"15min":2.0129999999999999},"last":1.704}]},{"osd":3,"last update":"Wed Aug 9 13:51:27 2023","interfaces":[{"interface":"back","average":{"1min":1,"5min":1.0349999999999999,"15min":1.101},"min":{"1min":0.50900000000000001,"5min":0.46300000000000002,"15min":0.46300000000000002},"max":{"1min":1.5189999999999999,"5min":1.823,"15min":1.8919999999999999},"last":0.67800000000000005},{"interface":"front","average":{"1min":1.2390000000000001,"5min":1.252,"15min":1.3140000000000001},"min":{"1min":0.63400000000000001,"5min":0.54700000000000004,"15min":0.54700000000000004},"max":{"1min":2.2050000000000001,"5min":2.347,"15min":2.4649999999999999},"last":1.022}]},{"osd":4,"last update":"Wed Aug 9 13:51:27 2023","interfaces":[{"interface":"back","average":{"1min":1.2509999999999999,"5min":1.2949999999999999,"15min":1.343},"min":{"1min":0.73899999999999999,"5min":0.73899999999999999,"15min":0.73899999999999999},"max":{"1min":1.581,"5min":2.2309999999999999,"15min":2.2309999999999999},"last":1.2150000000000001},{"interface":"front","average":{"1min":1.409,"5min":1.417,"15min":1.456},"min":{"1min":0.86399999999999999,"5min":0.77600000000000002,"15min":0.77600000000000002},"max":{"1min":2.3540000000000001,"5min":2.516,"15min":2.5390000000000001},"last":1.395}]},{"osd":5,"last update":"Wed Aug 9 13:51:27 2023","interfaces":[{"interface":"back","average":{"1min":1.3859999999999999,"5min":1.4019999999999999,"15min":1.462},"min":{"1min":0.875,"5min":0.66300000000000003,"15min":0.66300000000000003},"max":{"1min":1.764,"5min":2.6669999999999998,"15min":2.6669999999999998},"last":1.1020000000000001},{"interface":"front","average":{"1min":1.333,"5min":1.3859999999999999,"15min":1.4470000000000001},"min":{"1min":0.88400000000000001,"5min":0.629,"15min":0.629},"max":{"1min":1.645,"5min":2.1499999999999999,"15min":2.1499999999999999},"last":0.96099999999999997}]},{"osd":6,"last update":"Wed Aug 9 13:51:27 2023","interfaces":[{"interface":"back","average":{"1min":1.4870000000000001,"5min":1.5089999999999999,"15min":1.548},"min":{"1min":1.014,"5min":0.73599999999999999,"15min":0.73599999999999999},"max":{"1min":2.4670000000000001,"5min":2.4670000000000001,"15min":2.7250000000000001},"last":1.6319999999999999},{"interface":"front","average":{"1min":1.486,"5min":1.526,"15min":1.5569999999999999},"min":{"1min":0.95999999999999996,"5min":0.86699999999999999,"15min":0.86699999999999999},"max":{"1min":1.988,"5min":2.399,"15min":2.399},"last":1.573}]},{"osd":7,"last update":"Wed Aug 9 13:51:27 2023","interfaces":[{"interface":"back","average":{"1min":1.552,"5min":1.5720000000000001,"15min":1.621},"min":{"1min":0.89200000000000002,"5min":0.72299999999999998,"15min":0.72299999999999998},"max":{"1min":2.0880000000000001,"5min":2.3900000000000001,"15min":2.3900000000000001},"last":1.2490000000000001},{"interface":"front","average":{"1min":1.4179999999999999,"5min":1.484,"15min":1.54},"min":{"1min":0.84099999999999997,"5min":0.76700000000000002,"15min":0.76700000000000002},"max":{"1min":1.7969999999999999,"5min":2.274,"15min":2.274},"last":1.4890000000000001}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-08-09T13:52:05.262 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-08-09T13:52:05.520 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:52:05.518+0000 7fe98e806640 1 Processor -- start 2023-08-09T13:52:05.520 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:52:05.518+0000 7fe98e806640 1 -- start start 2023-08-09T13:52:05.522 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:52:05.520+0000 7fe98e806640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7fe988143c20 0x7fe98806f540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:52:05.522 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:52:05.520+0000 7fe98e806640 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7fe98806cfe0 0x7fe9880c8b50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:52:05.522 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:52:05.520+0000 7fe98e806640 1 -- --> v1:172.21.15.43:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe9880c9090 con 0x7fe9880c86f0 2023-08-09T13:52:05.522 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:52:05.520+0000 7fe98e806640 1 -- --> v2:172.21.15.43:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe988144000 con 0x7fe988143c20 2023-08-09T13:52:05.522 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:52:05.520+0000 7fe98e806640 1 -- --> v2:172.21.15.203:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe98806c440 con 0x7fe98806cfe0 2023-08-09T13:52:05.523 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:52:05.521+0000 7fe987fff640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7fe988143c20 0x7fe98806f540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:52:05.523 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:52:05.521+0000 7fe97ffff640 1 --1- >> v1:172.21.15.43:6789/0 conn(0x7fe9880c86f0 0x7fe98806c8d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.43:6789/0 says I am v1:172.21.15.43:38082/0 (socket says 172.21.15.43:38082) 2023-08-09T13:52:05.524 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:52:05.521+0000 7fe97ffff640 1 -- 172.21.15.43:0/2977714785 learned_addr learned my addr 172.21.15.43:0/2977714785 (peer_addr_for_me v1:172.21.15.43:0/0) 2023-08-09T13:52:05.524 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:52:05.521+0000 7fe987fff640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7fe988143c20 0x7fe98806f540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:54304/0 (socket says 172.21.15.43:54304) 2023-08-09T13:52:05.524 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:52:05.521+0000 7fe98cda3640 1 --2- 172.21.15.43:0/2977714785 >> v2:172.21.15.203:3300/0 conn(0x7fe98806cfe0 0x7fe9880c8b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:52:05.525 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:52:05.522+0000 7fe987fff640 1 -- 172.21.15.43:0/2977714785 >> v2:172.21.15.203:3300/0 conn(0x7fe98806cfe0 msgr2=0x7fe9880c8b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-09T13:52:05.525 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:52:05.522+0000 7fe987fff640 1 --2- 172.21.15.43:0/2977714785 >> v2:172.21.15.203:3300/0 conn(0x7fe98806cfe0 0x7fe9880c8b50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:52:05.525 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:52:05.522+0000 7fe987fff640 1 -- 172.21.15.43:0/2977714785 >> v1:172.21.15.43:6789/0 conn(0x7fe9880c86f0 legacy=0x7fe98806c8d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:52:05.525 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:52:05.522+0000 7fe987fff640 1 -- 172.21.15.43:0/2977714785 --> v2:172.21.15.43:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe98806b830 con 0x7fe988143c20 2023-08-09T13:52:05.526 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:52:05.522+0000 7fe9877fe640 1 -- 172.21.15.43:0/2977714785 <== mon.0 v1:172.21.15.43:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2912691219 0 0) 0x7fe9880c9090 con 0x7fe9880c86f0 2023-08-09T13:52:05.528 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:52:05.523+0000 7fe987fff640 1 --2- 172.21.15.43:0/2977714785 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fe988143c20 0x7fe98806f540 secure :-1 s=READY pgs=1351 cs=0 l=1 rev1=1 crypto rx=0x7fe9880caad0 tx=0x7fe97802bda0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3673a480ea9c4b71 server_cookie=0 in_seq=0 out_seq=0 2023-08-09T13:52:05.528 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:52:05.523+0000 7fe9877fe640 1 -- 172.21.15.43:0/2977714785 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe97802cb50 con 0x7fe988143c20 2023-08-09T13:52:05.528 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:52:05.523+0000 7fe9877fe640 1 -- 172.21.15.43:0/2977714785 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe97802ccb0 con 0x7fe988143c20 2023-08-09T13:52:05.529 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:52:05.523+0000 7fe9877fe640 1 -- 172.21.15.43:0/2977714785 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe978034660 con 0x7fe988143c20 2023-08-09T13:52:05.529 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:52:05.524+0000 7fe98e806640 1 -- 172.21.15.43:0/2977714785 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fe988143c20 msgr2=0x7fe98806f540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:52:05.529 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:52:05.524+0000 7fe98e806640 1 --2- 172.21.15.43:0/2977714785 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fe988143c20 0x7fe98806f540 secure :-1 s=READY pgs=1351 cs=0 l=1 rev1=1 crypto rx=0x7fe9880caad0 tx=0x7fe97802bda0 comp rx=0 tx=0).stop 2023-08-09T13:52:05.529 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:52:05.524+0000 7fe98e806640 1 -- 172.21.15.43:0/2977714785 shutdown_connections 2023-08-09T13:52:05.530 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:52:05.524+0000 7fe98e806640 1 --2- 172.21.15.43:0/2977714785 >> v2:172.21.15.203:3300/0 conn(0x7fe98806cfe0 0x7fe9880c8b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:52:05.530 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:52:05.524+0000 7fe98e806640 1 --2- 172.21.15.43:0/2977714785 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fe988143c20 0x7fe98806f540 unknown :-1 s=CLOSED pgs=1351 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:52:05.530 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:52:05.524+0000 7fe98e806640 1 -- 172.21.15.43:0/2977714785 >> 172.21.15.43:0/2977714785 conn(0x7fe9880ce0a0 msgr2=0x7fe9880ce480 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-09T13:52:05.530 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:52:05.524+0000 7fe98e806640 1 -- 172.21.15.43:0/2977714785 shutdown_connections 2023-08-09T13:52:05.530 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:52:05.524+0000 7fe98e806640 1 -- 172.21.15.43:0/2977714785 wait complete. 2023-08-09T13:52:05.530 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:52:05.526+0000 7fe98e806640 1 Processor -- start 2023-08-09T13:52:05.530 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:52:05.526+0000 7fe98e806640 1 -- start start 2023-08-09T13:52:05.532 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:52:05.527+0000 7fe98e806640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fe98806cfe0 0x7fe9880b2440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:52:05.532 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:52:05.527+0000 7fe98e806640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fe9880c86f0 0x7fe9880adf00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:52:05.532 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:52:05.527+0000 7fe98e806640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fe988143c20 0x7fe988141a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:52:05.532 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:52:05.527+0000 7fe98e806640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe9880ac8d0 con 0x7fe988143c20 2023-08-09T13:52:05.532 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:52:05.527+0000 7fe98e806640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe9880abd50 con 0x7fe9880c86f0 2023-08-09T13:52:05.533 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:52:05.527+0000 7fe98e806640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe9880ac030 con 0x7fe98806cfe0 2023-08-09T13:52:05.533 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:52:05.527+0000 7fe98cda3640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fe988143c20 0x7fe988141a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:52:05.533 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:52:05.527+0000 7fe98cda3640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fe988143c20 0x7fe988141a20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:54318/0 (socket says 172.21.15.43:54318) 2023-08-09T13:52:05.533 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:52:05.527+0000 7fe98cda3640 1 -- 172.21.15.43:0/554100449 learned_addr learned my addr 172.21.15.43:0/554100449 (peer_addr_for_me v2:172.21.15.43:0/0) 2023-08-09T13:52:05.533 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:52:05.528+0000 7fe97ffff640 1 --2- 172.21.15.43:0/554100449 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fe9880c86f0 0x7fe9880adf00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:52:05.533 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:52:05.528+0000 7fe987fff640 1 --2- 172.21.15.43:0/554100449 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fe98806cfe0 0x7fe9880b2440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:52:05.533 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:52:05.528+0000 7fe98cda3640 1 -- 172.21.15.43:0/554100449 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fe98806cfe0 msgr2=0x7fe9880b2440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:52:05.534 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:52:05.528+0000 7fe98cda3640 1 --2- 172.21.15.43:0/554100449 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fe98806cfe0 0x7fe9880b2440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:52:05.534 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:52:05.528+0000 7fe98cda3640 1 -- 172.21.15.43:0/554100449 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fe9880c86f0 msgr2=0x7fe9880adf00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:52:05.534 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:52:05.528+0000 7fe98cda3640 1 --2- 172.21.15.43:0/554100449 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fe9880c86f0 0x7fe9880adf00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:52:05.534 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:52:05.528+0000 7fe98cda3640 1 -- 172.21.15.43:0/554100449 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe978009a70 con 0x7fe988143c20 2023-08-09T13:52:05.534 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:52:05.528+0000 7fe98cda3640 1 --2- 172.21.15.43:0/554100449 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fe988143c20 0x7fe988141a20 secure :-1 s=READY pgs=1352 cs=0 l=1 rev1=1 crypto rx=0x7fe98000e7f0 tx=0x7fe98000ecb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-09T13:52:05.534 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:52:05.528+0000 7fe9857fa640 1 -- 172.21.15.43:0/554100449 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe980009460 con 0x7fe988143c20 2023-08-09T13:52:05.534 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:52:05.528+0000 7fe9857fa640 1 -- 172.21.15.43:0/554100449 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe98000c730 con 0x7fe988143c20 2023-08-09T13:52:05.534 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:52:05.528+0000 7fe98e806640 1 -- 172.21.15.43:0/554100449 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe9880ac310 con 0x7fe988143c20 2023-08-09T13:52:05.535 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:52:05.528+0000 7fe9857fa640 1 -- 172.21.15.43:0/554100449 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe98000cae0 con 0x7fe988143c20 2023-08-09T13:52:05.535 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:52:05.529+0000 7fe98e806640 1 -- 172.21.15.43:0/554100449 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe9880b2980 con 0x7fe988143c20 2023-08-09T13:52:05.535 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:52:05.530+0000 7fe98e806640 1 -- 172.21.15.43:0/554100449 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe9880739f0 con 0x7fe988143c20 2023-08-09T13:52:05.535 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:52:05.531+0000 7fe9857fa640 1 -- 172.21.15.43:0/554100449 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 45) v1 ==== 88443+0+0 (secure 0 0 0) 0x7fe98000cc40 con 0x7fe988143c20 2023-08-09T13:52:05.536 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:52:05.532+0000 7fe9857fa640 1 --2- 172.21.15.43:0/554100449 >> [v2:172.21.15.203:6832/1317108840,v1:172.21.15.203:6833/1317108840] conn(0x7fe96806a170 0x7fe96806c630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:52:05.536 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:52:05.533+0000 7fe9857fa640 1 -- 172.21.15.43:0/554100449 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(116..116 src has 1..116) v4 ==== 6426+0+0 (secure 0 0 0) 0x7fe980088d00 con 0x7fe988143c20 2023-08-09T13:52:05.536 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:52:05.533+0000 7fe987fff640 1 --2- 172.21.15.43:0/554100449 >> [v2:172.21.15.203:6832/1317108840,v1:172.21.15.203:6833/1317108840] conn(0x7fe96806a170 0x7fe96806c630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:52:05.536 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:52:05.534+0000 7fe987fff640 1 --2- 172.21.15.43:0/554100449 >> [v2:172.21.15.203:6832/1317108840,v1:172.21.15.203:6833/1317108840] conn(0x7fe96806a170 0x7fe96806c630 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7fe978036000 tx=0x7fe978030000 comp rx=0 tx=0).ready entity=mgr.6619 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-09T13:52:05.538 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:52:05.537+0000 7fe9857fa640 1 -- 172.21.15.43:0/554100449 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178066 (secure 0 0 0) 0x7fe9800540d0 con 0x7fe988143c20 2023-08-09T13:52:05.670 INFO:tasks.ceph.mds.c.smithi043.stderr:2023-08-09T13:52:05.668+0000 7f0151030640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2023-08-09T13:52:05.729 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:52:05.728+0000 7fe98e806640 1 -- 172.21.15.43:0/554100449 --> [v2:172.21.15.203:6832/1317108840,v1:172.21.15.203:6833/1317108840] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7fe98806bdf0 con 0x7fe96806a170 2023-08-09T13:52:05.753 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:52:05.751+0000 7fe9857fa640 1 -- 172.21.15.43:0/554100449 <== mgr.6619 v2:172.21.15.203:6832/1317108840 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+304014 (secure 0 0 0) 0x7fe978034540 con 0x7fe96806a170 2023-08-09T13:52:05.755 INFO:teuthology.orchestra.run.smithi043.stdout: 2023-08-09T13:52:05.758 INFO:teuthology.orchestra.run.smithi043.stderr:dumped all 2023-08-09T13:52:05.760 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:52:05.758+0000 7fe98e806640 1 -- 172.21.15.43:0/554100449 >> [v2:172.21.15.203:6832/1317108840,v1:172.21.15.203:6833/1317108840] conn(0x7fe96806a170 msgr2=0x7fe96806c630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:52:05.760 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:52:05.758+0000 7fe98e806640 1 --2- 172.21.15.43:0/554100449 >> [v2:172.21.15.203:6832/1317108840,v1:172.21.15.203:6833/1317108840] conn(0x7fe96806a170 0x7fe96806c630 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7fe978036000 tx=0x7fe978030000 comp rx=0 tx=0).stop 2023-08-09T13:52:05.760 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:52:05.758+0000 7fe98e806640 1 -- 172.21.15.43:0/554100449 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fe988143c20 msgr2=0x7fe988141a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:52:05.760 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:52:05.758+0000 7fe98e806640 1 --2- 172.21.15.43:0/554100449 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fe988143c20 0x7fe988141a20 secure :-1 s=READY pgs=1352 cs=0 l=1 rev1=1 crypto rx=0x7fe98000e7f0 tx=0x7fe98000ecb0 comp rx=0 tx=0).stop 2023-08-09T13:52:05.760 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:52:05.758+0000 7fe98e806640 1 -- 172.21.15.43:0/554100449 shutdown_connections 2023-08-09T13:52:05.760 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:52:05.758+0000 7fe98e806640 1 --2- 172.21.15.43:0/554100449 >> [v2:172.21.15.203:6832/1317108840,v1:172.21.15.203:6833/1317108840] conn(0x7fe96806a170 0x7fe96806c630 unknown :-1 s=CLOSED pgs=67 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:52:05.761 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:52:05.758+0000 7fe98e806640 1 --2- 172.21.15.43:0/554100449 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fe988143c20 0x7fe988141a20 unknown :-1 s=CLOSED pgs=1352 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:52:05.761 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:52:05.758+0000 7fe98e806640 1 --2- 172.21.15.43:0/554100449 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fe9880c86f0 0x7fe9880adf00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:52:05.761 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:52:05.758+0000 7fe98e806640 1 --2- 172.21.15.43:0/554100449 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fe98806cfe0 0x7fe9880b2440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:52:05.761 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:52:05.758+0000 7fe98e806640 1 -- 172.21.15.43:0/554100449 >> 172.21.15.43:0/554100449 conn(0x7fe9880ce0a0 msgr2=0x7fe98815aad0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-09T13:52:05.761 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:52:05.758+0000 7fe98e806640 1 -- 172.21.15.43:0/554100449 shutdown_connections 2023-08-09T13:52:05.761 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:52:05.758+0000 7fe98e806640 1 -- 172.21.15.43:0/554100449 wait complete. 2023-08-09T13:52:05.779 INFO:teuthology.orchestra.run.smithi043.stdout:{"pg_ready":true,"pg_map":{"version":27,"stamp":"2023-08-09T13:52:04.571627+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":715,"num_read_kb":1418,"num_write":283,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":186,"ondisk_log_size":186,"up":210,"acting":210,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":23,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":307720,"kb_used_data":12068,"kb_used_omap":13,"kb_used_meta":295538,"kb_avail":754667000,"statfs":{"total":773094113280,"available":772779008000,"internally_reserved":0,"allocated":12357632,"data_stored":7127134,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":14248,"internal_metadata":302631000},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":37,"apply_latency_ms":37,"commit_latency_ns":37000000,"apply_latency_ns":37000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.001865"},"pg_stats":[{"pgid":"15.3f","version":"0'0","reported_seq":1,"reported_epoch":114,"state":"creating+peering","last_fresh":"2023-08-09T13:51:59.173066+0000","last_change":"2023-08-09T13:51:59.173066+0000","last_active":"2023-08-09T13:51:59.130555+0000","last_peered":"2023-08-09T13:51:59.130555+0000","last_clean":"2023-08-09T13:51:59.130555+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-09T13:51:59.173066+0000","last_undegraded":"2023-08-09T13:51:59.173066+0000","last_fullsized":"2023-08-09T13:51:59.173066+0000","mapping_epoch":114,"log_start":"0'0","ondisk_log_start":"0'0","created":114,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:51:59.130555+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:51:59.130555+0000","last_clean_scrub_stamp":"2023-08-09T13:51:59.130555+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"15.3e","version":"0'0","reported_seq":1,"reported_epoch":114,"state":"creating+peering","last_fresh":"2023-08-09T13:51:59.161823+0000","last_change":"2023-08-09T13:51:59.161823+0000","last_active":"2023-08-09T13:51:59.130555+0000","last_peered":"2023-08-09T13:51:59.130555+0000","last_clean":"2023-08-09T13:51:59.130555+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-09T13:51:59.161823+0000","last_undegraded":"2023-08-09T13:51:59.161823+0000","last_fullsized":"2023-08-09T13:51:59.161823+0000","mapping_epoch":114,"log_start":"0'0","ondisk_log_start":"0'0","created":114,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:51:59.130555+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:51:59.130555+0000","last_clean_scrub_stamp":"2023-08-09T13:51:59.130555+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.3d","version":"0'0","reported_seq":1,"reported_epoch":114,"state":"creating+peering","last_fresh":"2023-08-09T13:51:59.169470+0000","last_change":"2023-08-09T13:51:59.169470+0000","last_active":"2023-08-09T13:51:59.130555+0000","last_peered":"2023-08-09T13:51:59.130555+0000","last_clean":"2023-08-09T13:51:59.130555+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-09T13:51:59.169470+0000","last_undegraded":"2023-08-09T13:51:59.169470+0000","last_fullsized":"2023-08-09T13:51:59.169470+0000","mapping_epoch":114,"log_start":"0'0","ondisk_log_start":"0'0","created":114,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:51:59.130555+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:51:59.130555+0000","last_clean_scrub_stamp":"2023-08-09T13:51:59.130555+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"15.3c","version":"0'0","reported_seq":1,"reported_epoch":114,"state":"creating+peering","last_fresh":"2023-08-09T13:51:59.150559+0000","last_change":"2023-08-09T13:51:59.150559+0000","last_active":"2023-08-09T13:51:59.130555+0000","last_peered":"2023-08-09T13:51:59.130555+0000","last_clean":"2023-08-09T13:51:59.130555+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-09T13:51:59.150559+0000","last_undegraded":"2023-08-09T13:51:59.150559+0000","last_fullsized":"2023-08-09T13:51:59.150559+0000","mapping_epoch":114,"log_start":"0'0","ondisk_log_start":"0'0","created":114,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:51:59.130555+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:51:59.130555+0000","last_clean_scrub_stamp":"2023-08-09T13:51:59.130555+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"15.3b","version":"0'0","reported_seq":1,"reported_epoch":114,"state":"creating+peering","last_fresh":"2023-08-09T13:51:59.155619+0000","last_change":"2023-08-09T13:51:59.155619+0000","last_active":"2023-08-09T13:51:59.130555+0000","last_peered":"2023-08-09T13:51:59.130555+0000","last_clean":"2023-08-09T13:51:59.130555+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-09T13:51:59.155619+0000","last_undegraded":"2023-08-09T13:51:59.155619+0000","last_fullsized":"2023-08-09T13:51:59.155619+0000","mapping_epoch":114,"log_start":"0'0","ondisk_log_start":"0'0","created":114,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:51:59.130555+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:51:59.130555+0000","last_clean_scrub_stamp":"2023-08-09T13:51:59.130555+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.3a","version":"0'0","reported_seq":1,"reported_epoch":114,"state":"creating+peering","last_fresh":"2023-08-09T13:51:59.155005+0000","last_change":"2023-08-09T13:51:59.155005+0000","last_active":"2023-08-09T13:51:59.130555+0000","last_peered":"2023-08-09T13:51:59.130555+0000","last_clean":"2023-08-09T13:51:59.130555+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-09T13:51:59.155005+0000","last_undegraded":"2023-08-09T13:51:59.155005+0000","last_fullsized":"2023-08-09T13:51:59.155005+0000","mapping_epoch":114,"log_start":"0'0","ondisk_log_start":"0'0","created":114,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:51:59.130555+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:51:59.130555+0000","last_clean_scrub_stamp":"2023-08-09T13:51:59.130555+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"15.39","version":"0'0","reported_seq":1,"reported_epoch":114,"state":"creating+peering","last_fresh":"2023-08-09T13:51:59.154799+0000","last_change":"2023-08-09T13:51:59.154799+0000","last_active":"2023-08-09T13:51:59.130555+0000","last_peered":"2023-08-09T13:51:59.130555+0000","last_clean":"2023-08-09T13:51:59.130555+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-09T13:51:59.154799+0000","last_undegraded":"2023-08-09T13:51:59.154799+0000","last_fullsized":"2023-08-09T13:51:59.154799+0000","mapping_epoch":114,"log_start":"0'0","ondisk_log_start":"0'0","created":114,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:51:59.130555+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:51:59.130555+0000","last_clean_scrub_stamp":"2023-08-09T13:51:59.130555+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"15.38","version":"0'0","reported_seq":1,"reported_epoch":114,"state":"creating+peering","last_fresh":"2023-08-09T13:51:59.161114+0000","last_change":"2023-08-09T13:51:59.161114+0000","last_active":"2023-08-09T13:51:59.130555+0000","last_peered":"2023-08-09T13:51:59.130555+0000","last_clean":"2023-08-09T13:51:59.130555+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-09T13:51:59.161114+0000","last_undegraded":"2023-08-09T13:51:59.161114+0000","last_fullsized":"2023-08-09T13:51:59.161114+0000","mapping_epoch":114,"log_start":"0'0","ondisk_log_start":"0'0","created":114,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:51:59.130555+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:51:59.130555+0000","last_clean_scrub_stamp":"2023-08-09T13:51:59.130555+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.37","version":"0'0","reported_seq":1,"reported_epoch":114,"state":"creating+peering","last_fresh":"2023-08-09T13:51:59.170489+0000","last_change":"2023-08-09T13:51:59.170489+0000","last_active":"2023-08-09T13:51:59.130555+0000","last_peered":"2023-08-09T13:51:59.130555+0000","last_clean":"2023-08-09T13:51:59.130555+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-09T13:51:59.170489+0000","last_undegraded":"2023-08-09T13:51:59.170489+0000","last_fullsized":"2023-08-09T13:51:59.170489+0000","mapping_epoch":114,"log_start":"0'0","ondisk_log_start":"0'0","created":114,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:51:59.130555+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:51:59.130555+0000","last_clean_scrub_stamp":"2023-08-09T13:51:59.130555+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"15.36","version":"0'0","reported_seq":1,"reported_epoch":114,"state":"creating+peering","last_fresh":"2023-08-09T13:51:59.157362+0000","last_change":"2023-08-09T13:51:59.157362+0000","last_active":"2023-08-09T13:51:59.130555+0000","last_peered":"2023-08-09T13:51:59.130555+0000","last_clean":"2023-08-09T13:51:59.130555+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-09T13:51:59.157362+0000","last_undegraded":"2023-08-09T13:51:59.157362+0000","last_fullsized":"2023-08-09T13:51:59.157362+0000","mapping_epoch":114,"log_start":"0'0","ondisk_log_start":"0'0","created":114,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:51:59.130555+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:51:59.130555+0000","last_clean_scrub_stamp":"2023-08-09T13:51:59.130555+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.35","version":"0'0","reported_seq":1,"reported_epoch":114,"state":"creating+peering","last_fresh":"2023-08-09T13:51:59.158636+0000","last_change":"2023-08-09T13:51:59.158636+0000","last_active":"2023-08-09T13:51:59.130555+0000","last_peered":"2023-08-09T13:51:59.130555+0000","last_clean":"2023-08-09T13:51:59.130555+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-09T13:51:59.158636+0000","last_undegraded":"2023-08-09T13:51:59.158636+0000","last_fullsized":"2023-08-09T13:51:59.158636+0000","mapping_epoch":114,"log_start":"0'0","ondisk_log_start":"0'0","created":114,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:51:59.130555+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:51:59.130555+0000","last_clean_scrub_stamp":"2023-08-09T13:51:59.130555+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"15.34","version":"0'0","reported_seq":1,"reported_epoch":114,"state":"creating+peering","last_fresh":"2023-08-09T13:51:59.175941+0000","last_change":"2023-08-09T13:51:59.175941+0000","last_active":"2023-08-09T13:51:59.130555+0000","last_peered":"2023-08-09T13:51:59.130555+0000","last_clean":"2023-08-09T13:51:59.130555+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-09T13:51:59.175941+0000","last_undegraded":"2023-08-09T13:51:59.175941+0000","last_fullsized":"2023-08-09T13:51:59.175941+0000","mapping_epoch":114,"log_start":"0'0","ondisk_log_start":"0'0","created":114,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:51:59.130555+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:51:59.130555+0000","last_clean_scrub_stamp":"2023-08-09T13:51:59.130555+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"15.33","version":"0'0","reported_seq":1,"reported_epoch":114,"state":"creating+peering","last_fresh":"2023-08-09T13:51:59.150804+0000","last_change":"2023-08-09T13:51:59.150804+0000","last_active":"2023-08-09T13:51:59.130555+0000","last_peered":"2023-08-09T13:51:59.130555+0000","last_clean":"2023-08-09T13:51:59.130555+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-09T13:51:59.150804+0000","last_undegraded":"2023-08-09T13:51:59.150804+0000","last_fullsized":"2023-08-09T13:51:59.150804+0000","mapping_epoch":114,"log_start":"0'0","ondisk_log_start":"0'0","created":114,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:51:59.130555+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:51:59.130555+0000","last_clean_scrub_stamp":"2023-08-09T13:51:59.130555+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"15.32","version":"0'0","reported_seq":1,"reported_epoch":114,"state":"creating+peering","last_fresh":"2023-08-09T13:51:59.165080+0000","last_change":"2023-08-09T13:51:59.165080+0000","last_active":"2023-08-09T13:51:59.130555+0000","last_peered":"2023-08-09T13:51:59.130555+0000","last_clean":"2023-08-09T13:51:59.130555+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-09T13:51:59.165080+0000","last_undegraded":"2023-08-09T13:51:59.165080+0000","last_fullsized":"2023-08-09T13:51:59.165080+0000","mapping_epoch":114,"log_start":"0'0","ondisk_log_start":"0'0","created":114,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:51:59.130555+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:51:59.130555+0000","last_clean_scrub_stamp":"2023-08-09T13:51:59.130555+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.31","version":"0'0","reported_seq":1,"reported_epoch":114,"state":"creating+peering","last_fresh":"2023-08-09T13:51:59.150713+0000","last_change":"2023-08-09T13:51:59.150713+0000","last_active":"2023-08-09T13:51:59.130555+0000","last_peered":"2023-08-09T13:51:59.130555+0000","last_clean":"2023-08-09T13:51:59.130555+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-09T13:51:59.150713+0000","last_undegraded":"2023-08-09T13:51:59.150713+0000","last_fullsized":"2023-08-09T13:51:59.150713+0000","mapping_epoch":114,"log_start":"0'0","ondisk_log_start":"0'0","created":114,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:51:59.130555+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:51:59.130555+0000","last_clean_scrub_stamp":"2023-08-09T13:51:59.130555+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"15.30","version":"0'0","reported_seq":1,"reported_epoch":114,"state":"creating+peering","last_fresh":"2023-08-09T13:51:59.157225+0000","last_change":"2023-08-09T13:51:59.157225+0000","last_active":"2023-08-09T13:51:59.130555+0000","last_peered":"2023-08-09T13:51:59.130555+0000","last_clean":"2023-08-09T13:51:59.130555+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-09T13:51:59.157225+0000","last_undegraded":"2023-08-09T13:51:59.157225+0000","last_fullsized":"2023-08-09T13:51:59.157225+0000","mapping_epoch":114,"log_start":"0'0","ondisk_log_start":"0'0","created":114,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:51:59.130555+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:51:59.130555+0000","last_clean_scrub_stamp":"2023-08-09T13:51:59.130555+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"15.2f","version":"0'0","reported_seq":1,"reported_epoch":114,"state":"creating+peering","last_fresh":"2023-08-09T13:51:59.157118+0000","last_change":"2023-08-09T13:51:59.157118+0000","last_active":"2023-08-09T13:51:59.130555+0000","last_peered":"2023-08-09T13:51:59.130555+0000","last_clean":"2023-08-09T13:51:59.130555+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-09T13:51:59.157118+0000","last_undegraded":"2023-08-09T13:51:59.157118+0000","last_fullsized":"2023-08-09T13:51:59.157118+0000","mapping_epoch":114,"log_start":"0'0","ondisk_log_start":"0'0","created":114,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:51:59.130555+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:51:59.130555+0000","last_clean_scrub_stamp":"2023-08-09T13:51:59.130555+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.2e","version":"0'0","reported_seq":1,"reported_epoch":114,"state":"creating+peering","last_fresh":"2023-08-09T13:51:59.153061+0000","last_change":"2023-08-09T13:51:59.153061+0000","last_active":"2023-08-09T13:51:59.130555+0000","last_peered":"2023-08-09T13:51:59.130555+0000","last_clean":"2023-08-09T13:51:59.130555+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-09T13:51:59.153061+0000","last_undegraded":"2023-08-09T13:51:59.153061+0000","last_fullsized":"2023-08-09T13:51:59.153061+0000","mapping_epoch":114,"log_start":"0'0","ondisk_log_start":"0'0","created":114,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:51:59.130555+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:51:59.130555+0000","last_clean_scrub_stamp":"2023-08-09T13:51:59.130555+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"15.2d","version":"0'0","reported_seq":1,"reported_epoch":114,"state":"creating+peering","last_fresh":"2023-08-09T13:51:59.156327+0000","last_change":"2023-08-09T13:51:59.156327+0000","last_active":"2023-08-09T13:51:59.130555+0000","last_peered":"2023-08-09T13:51:59.130555+0000","last_clean":"2023-08-09T13:51:59.130555+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-09T13:51:59.156327+0000","last_undegraded":"2023-08-09T13:51:59.156327+0000","last_fullsized":"2023-08-09T13:51:59.156327+0000","mapping_epoch":114,"log_start":"0'0","ondisk_log_start":"0'0","created":114,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:51:59.130555+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:51:59.130555+0000","last_clean_scrub_stamp":"2023-08-09T13:51:59.130555+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.2c","version":"0'0","reported_seq":1,"reported_epoch":114,"state":"creating+peering","last_fresh":"2023-08-09T13:51:59.175350+0000","last_change":"2023-08-09T13:51:59.175350+0000","last_active":"2023-08-09T13:51:59.130555+0000","last_peered":"2023-08-09T13:51:59.130555+0000","last_clean":"2023-08-09T13:51:59.130555+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-09T13:51:59.175350+0000","last_undegraded":"2023-08-09T13:51:59.175350+0000","last_fullsized":"2023-08-09T13:51:59.175350+0000","mapping_epoch":114,"log_start":"0'0","ondisk_log_start":"0'0","created":114,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:51:59.130555+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:51:59.130555+0000","last_clean_scrub_stamp":"2023-08-09T13:51:59.130555+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"15.2b","version":"0'0","reported_seq":1,"reported_epoch":114,"state":"creating+peering","last_fresh":"2023-08-09T13:51:59.163315+0000","last_change":"2023-08-09T13:51:59.163315+0000","last_active":"2023-08-09T13:51:59.130555+0000","last_peered":"2023-08-09T13:51:59.130555+0000","last_clean":"2023-08-09T13:51:59.130555+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-09T13:51:59.163315+0000","last_undegraded":"2023-08-09T13:51:59.163315+0000","last_fullsized":"2023-08-09T13:51:59.163315+0000","mapping_epoch":114,"log_start":"0'0","ondisk_log_start":"0'0","created":114,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:51:59.130555+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:51:59.130555+0000","last_clean_scrub_stamp":"2023-08-09T13:51:59.130555+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.2a","version":"0'0","reported_seq":1,"reported_epoch":114,"state":"creating+peering","last_fresh":"2023-08-09T13:51:59.158490+0000","last_change":"2023-08-09T13:51:59.158490+0000","last_active":"2023-08-09T13:51:59.130555+0000","last_peered":"2023-08-09T13:51:59.130555+0000","last_clean":"2023-08-09T13:51:59.130555+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-09T13:51:59.158490+0000","last_undegraded":"2023-08-09T13:51:59.158490+0000","last_fullsized":"2023-08-09T13:51:59.158490+0000","mapping_epoch":114,"log_start":"0'0","ondisk_log_start":"0'0","created":114,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:51:59.130555+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:51:59.130555+0000","last_clean_scrub_stamp":"2023-08-09T13:51:59.130555+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.29","version":"0'0","reported_seq":1,"reported_epoch":114,"state":"creating+peering","last_fresh":"2023-08-09T13:51:59.157791+0000","last_change":"2023-08-09T13:51:59.157791+0000","last_active":"2023-08-09T13:51:59.130555+0000","last_peered":"2023-08-09T13:51:59.130555+0000","last_clean":"2023-08-09T13:51:59.130555+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-09T13:51:59.157791+0000","last_undegraded":"2023-08-09T13:51:59.157791+0000","last_fullsized":"2023-08-09T13:51:59.157791+0000","mapping_epoch":114,"log_start":"0'0","ondisk_log_start":"0'0","created":114,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:51:59.130555+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:51:59.130555+0000","last_clean_scrub_stamp":"2023-08-09T13:51:59.130555+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.28","version":"0'0","reported_seq":1,"reported_epoch":114,"state":"creating+peering","last_fresh":"2023-08-09T13:51:59.156031+0000","last_change":"2023-08-09T13:51:59.156031+0000","last_active":"2023-08-09T13:51:59.130555+0000","last_peered":"2023-08-09T13:51:59.130555+0000","last_clean":"2023-08-09T13:51:59.130555+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-09T13:51:59.156031+0000","last_undegraded":"2023-08-09T13:51:59.156031+0000","last_fullsized":"2023-08-09T13:51:59.156031+0000","mapping_epoch":114,"log_start":"0'0","ondisk_log_start":"0'0","created":114,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:51:59.130555+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:51:59.130555+0000","last_clean_scrub_stamp":"2023-08-09T13:51:59.130555+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"15.27","version":"0'0","reported_seq":1,"reported_epoch":114,"state":"creating+peering","last_fresh":"2023-08-09T13:51:59.157867+0000","last_change":"2023-08-09T13:51:59.157867+0000","last_active":"2023-08-09T13:51:59.130555+0000","last_peered":"2023-08-09T13:51:59.130555+0000","last_clean":"2023-08-09T13:51:59.130555+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-09T13:51:59.157867+0000","last_undegraded":"2023-08-09T13:51:59.157867+0000","last_fullsized":"2023-08-09T13:51:59.157867+0000","mapping_epoch":114,"log_start":"0'0","ondisk_log_start":"0'0","created":114,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:51:59.130555+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:51:59.130555+0000","last_clean_scrub_stamp":"2023-08-09T13:51:59.130555+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.26","version":"0'0","reported_seq":1,"reported_epoch":114,"state":"creating+peering","last_fresh":"2023-08-09T13:51:59.159192+0000","last_change":"2023-08-09T13:51:59.159192+0000","last_active":"2023-08-09T13:51:59.130555+0000","last_peered":"2023-08-09T13:51:59.130555+0000","last_clean":"2023-08-09T13:51:59.130555+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-09T13:51:59.159192+0000","last_undegraded":"2023-08-09T13:51:59.159192+0000","last_fullsized":"2023-08-09T13:51:59.159192+0000","mapping_epoch":114,"log_start":"0'0","ondisk_log_start":"0'0","created":114,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:51:59.130555+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:51:59.130555+0000","last_clean_scrub_stamp":"2023-08-09T13:51:59.130555+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.25","version":"0'0","reported_seq":1,"reported_epoch":114,"state":"creating+peering","last_fresh":"2023-08-09T13:51:59.157367+0000","last_change":"2023-08-09T13:51:59.157367+0000","last_active":"2023-08-09T13:51:59.130555+0000","last_peered":"2023-08-09T13:51:59.130555+0000","last_clean":"2023-08-09T13:51:59.130555+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-09T13:51:59.157367+0000","last_undegraded":"2023-08-09T13:51:59.157367+0000","last_fullsized":"2023-08-09T13:51:59.157367+0000","mapping_epoch":114,"log_start":"0'0","ondisk_log_start":"0'0","created":114,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:51:59.130555+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:51:59.130555+0000","last_clean_scrub_stamp":"2023-08-09T13:51:59.130555+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"15.24","version":"0'0","reported_seq":1,"reported_epoch":114,"state":"creating+peering","last_fresh":"2023-08-09T13:51:59.162536+0000","last_change":"2023-08-09T13:51:59.162536+0000","last_active":"2023-08-09T13:51:59.130555+0000","last_peered":"2023-08-09T13:51:59.130555+0000","last_clean":"2023-08-09T13:51:59.130555+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-09T13:51:59.162536+0000","last_undegraded":"2023-08-09T13:51:59.162536+0000","last_fullsized":"2023-08-09T13:51:59.162536+0000","mapping_epoch":114,"log_start":"0'0","ondisk_log_start":"0'0","created":114,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:51:59.130555+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:51:59.130555+0000","last_clean_scrub_stamp":"2023-08-09T13:51:59.130555+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.23","version":"0'0","reported_seq":1,"reported_epoch":114,"state":"creating+peering","last_fresh":"2023-08-09T13:51:59.160575+0000","last_change":"2023-08-09T13:51:59.160575+0000","last_active":"2023-08-09T13:51:59.130555+0000","last_peered":"2023-08-09T13:51:59.130555+0000","last_clean":"2023-08-09T13:51:59.130555+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-09T13:51:59.160575+0000","last_undegraded":"2023-08-09T13:51:59.160575+0000","last_fullsized":"2023-08-09T13:51:59.160575+0000","mapping_epoch":114,"log_start":"0'0","ondisk_log_start":"0'0","created":114,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:51:59.130555+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:51:59.130555+0000","last_clean_scrub_stamp":"2023-08-09T13:51:59.130555+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.22","version":"0'0","reported_seq":1,"reported_epoch":114,"state":"creating+peering","last_fresh":"2023-08-09T13:51:59.164643+0000","last_change":"2023-08-09T13:51:59.164643+0000","last_active":"2023-08-09T13:51:59.130555+0000","last_peered":"2023-08-09T13:51:59.130555+0000","last_clean":"2023-08-09T13:51:59.130555+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-09T13:51:59.164643+0000","last_undegraded":"2023-08-09T13:51:59.164643+0000","last_fullsized":"2023-08-09T13:51:59.164643+0000","mapping_epoch":114,"log_start":"0'0","ondisk_log_start":"0'0","created":114,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:51:59.130555+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:51:59.130555+0000","last_clean_scrub_stamp":"2023-08-09T13:51:59.130555+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.21","version":"0'0","reported_seq":1,"reported_epoch":114,"state":"creating+peering","last_fresh":"2023-08-09T13:51:59.149533+0000","last_change":"2023-08-09T13:51:59.149533+0000","last_active":"2023-08-09T13:51:59.130555+0000","last_peered":"2023-08-09T13:51:59.130555+0000","last_clean":"2023-08-09T13:51:59.130555+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-09T13:51:59.149533+0000","last_undegraded":"2023-08-09T13:51:59.149533+0000","last_fullsized":"2023-08-09T13:51:59.149533+0000","mapping_epoch":114,"log_start":"0'0","ondisk_log_start":"0'0","created":114,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:51:59.130555+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:51:59.130555+0000","last_clean_scrub_stamp":"2023-08-09T13:51:59.130555+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"15.20","version":"0'0","reported_seq":1,"reported_epoch":114,"state":"creating+peering","last_fresh":"2023-08-09T13:51:59.168339+0000","last_change":"2023-08-09T13:51:59.168339+0000","last_active":"2023-08-09T13:51:59.130555+0000","last_peered":"2023-08-09T13:51:59.130555+0000","last_clean":"2023-08-09T13:51:59.130555+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-09T13:51:59.168339+0000","last_undegraded":"2023-08-09T13:51:59.168339+0000","last_fullsized":"2023-08-09T13:51:59.168339+0000","mapping_epoch":114,"log_start":"0'0","ondisk_log_start":"0'0","created":114,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:51:59.130555+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:51:59.130555+0000","last_clean_scrub_stamp":"2023-08-09T13:51:59.130555+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"15.1e","version":"0'0","reported_seq":1,"reported_epoch":114,"state":"creating+peering","last_fresh":"2023-08-09T13:51:59.162676+0000","last_change":"2023-08-09T13:51:59.162676+0000","last_active":"2023-08-09T13:51:59.130555+0000","last_peered":"2023-08-09T13:51:59.130555+0000","last_clean":"2023-08-09T13:51:59.130555+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-09T13:51:59.162676+0000","last_undegraded":"2023-08-09T13:51:59.162676+0000","last_fullsized":"2023-08-09T13:51:59.162676+0000","mapping_epoch":114,"log_start":"0'0","ondisk_log_start":"0'0","created":114,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:51:59.130555+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:51:59.130555+0000","last_clean_scrub_stamp":"2023-08-09T13:51:59.130555+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"14.1f","version":"0'0","reported_seq":9,"reported_epoch":114,"state":"active+clean","last_fresh":"2023-08-09T13:51:59.180137+0000","last_change":"2023-08-09T13:51:59.176081+0000","last_active":"2023-08-09T13:51:59.180137+0000","last_peered":"2023-08-09T13:51:59.180137+0000","last_clean":"2023-08-09T13:51:59.180137+0000","last_became_active":"2023-08-09T13:51:59.175790+0000","last_became_peered":"2023-08-09T13:51:59.175790+0000","last_unstale":"2023-08-09T13:51:59.180137+0000","last_undegraded":"2023-08-09T13:51:59.180137+0000","last_fullsized":"2023-08-09T13:51:59.180137+0000","mapping_epoch":113,"log_start":"0'0","ondisk_log_start":"0'0","created":113,"last_epoch_clean":114,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:51:58.125661+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:51:58.125661+0000","last_clean_scrub_stamp":"2023-08-09T13:51:58.125661+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T17:58:19.394820+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.1f","version":"0'0","reported_seq":1,"reported_epoch":114,"state":"creating+peering","last_fresh":"2023-08-09T13:51:59.151940+0000","last_change":"2023-08-09T13:51:59.151940+0000","last_active":"2023-08-09T13:51:59.130555+0000","last_peered":"2023-08-09T13:51:59.130555+0000","last_clean":"2023-08-09T13:51:59.130555+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-09T13:51:59.151940+0000","last_undegraded":"2023-08-09T13:51:59.151940+0000","last_fullsized":"2023-08-09T13:51:59.151940+0000","mapping_epoch":114,"log_start":"0'0","ondisk_log_start":"0'0","created":114,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:51:59.130555+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:51:59.130555+0000","last_clean_scrub_stamp":"2023-08-09T13:51:59.130555+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"14.1e","version":"0'0","reported_seq":9,"reported_epoch":114,"state":"active+clean","last_fresh":"2023-08-09T13:51:59.180102+0000","last_change":"2023-08-09T13:51:59.178857+0000","last_active":"2023-08-09T13:51:59.180102+0000","last_peered":"2023-08-09T13:51:59.180102+0000","last_clean":"2023-08-09T13:51:59.180102+0000","last_became_active":"2023-08-09T13:51:59.178483+0000","last_became_peered":"2023-08-09T13:51:59.178483+0000","last_unstale":"2023-08-09T13:51:59.180102+0000","last_undegraded":"2023-08-09T13:51:59.180102+0000","last_fullsized":"2023-08-09T13:51:59.180102+0000","mapping_epoch":113,"log_start":"0'0","ondisk_log_start":"0'0","created":113,"last_epoch_clean":114,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:51:58.125661+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:51:58.125661+0000","last_clean_scrub_stamp":"2023-08-09T13:51:58.125661+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T15:40:02.742070+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"15.1c","version":"0'0","reported_seq":1,"reported_epoch":114,"state":"creating+peering","last_fresh":"2023-08-09T13:51:59.166472+0000","last_change":"2023-08-09T13:51:59.166472+0000","last_active":"2023-08-09T13:51:59.130555+0000","last_peered":"2023-08-09T13:51:59.130555+0000","last_clean":"2023-08-09T13:51:59.130555+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-09T13:51:59.166472+0000","last_undegraded":"2023-08-09T13:51:59.166472+0000","last_fullsized":"2023-08-09T13:51:59.166472+0000","mapping_epoch":114,"log_start":"0'0","ondisk_log_start":"0'0","created":114,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:51:59.130555+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:51:59.130555+0000","last_clean_scrub_stamp":"2023-08-09T13:51:59.130555+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.1d","version":"0'0","reported_seq":9,"reported_epoch":114,"state":"active+clean","last_fresh":"2023-08-09T13:51:59.173649+0000","last_change":"2023-08-09T13:51:59.168942+0000","last_active":"2023-08-09T13:51:59.173649+0000","last_peered":"2023-08-09T13:51:59.173649+0000","last_clean":"2023-08-09T13:51:59.173649+0000","last_became_active":"2023-08-09T13:51:59.168599+0000","last_became_peered":"2023-08-09T13:51:59.168599+0000","last_unstale":"2023-08-09T13:51:59.173649+0000","last_undegraded":"2023-08-09T13:51:59.173649+0000","last_fullsized":"2023-08-09T13:51:59.173649+0000","mapping_epoch":113,"log_start":"0'0","ondisk_log_start":"0'0","created":113,"last_epoch_clean":114,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:51:58.125661+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:51:58.125661+0000","last_clean_scrub_stamp":"2023-08-09T13:51:58.125661+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T21:57:39.771411+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.1d","version":"0'0","reported_seq":1,"reported_epoch":114,"state":"creating+peering","last_fresh":"2023-08-09T13:51:59.170197+0000","last_change":"2023-08-09T13:51:59.170197+0000","last_active":"2023-08-09T13:51:59.130555+0000","last_peered":"2023-08-09T13:51:59.130555+0000","last_clean":"2023-08-09T13:51:59.130555+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-09T13:51:59.170197+0000","last_undegraded":"2023-08-09T13:51:59.170197+0000","last_fullsized":"2023-08-09T13:51:59.170197+0000","mapping_epoch":114,"log_start":"0'0","ondisk_log_start":"0'0","created":114,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:51:59.130555+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:51:59.130555+0000","last_clean_scrub_stamp":"2023-08-09T13:51:59.130555+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.1c","version":"0'0","reported_seq":9,"reported_epoch":114,"state":"active+clean","last_fresh":"2023-08-09T13:51:59.184792+0000","last_change":"2023-08-09T13:51:59.183710+0000","last_active":"2023-08-09T13:51:59.184792+0000","last_peered":"2023-08-09T13:51:59.184792+0000","last_clean":"2023-08-09T13:51:59.184792+0000","last_became_active":"2023-08-09T13:51:59.183401+0000","last_became_peered":"2023-08-09T13:51:59.183401+0000","last_unstale":"2023-08-09T13:51:59.184792+0000","last_undegraded":"2023-08-09T13:51:59.184792+0000","last_fullsized":"2023-08-09T13:51:59.184792+0000","mapping_epoch":113,"log_start":"0'0","ondisk_log_start":"0'0","created":113,"last_epoch_clean":114,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:51:58.125661+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:51:58.125661+0000","last_clean_scrub_stamp":"2023-08-09T13:51:58.125661+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T13:55:08.336317+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.1a","version":"0'0","reported_seq":1,"reported_epoch":114,"state":"creating+peering","last_fresh":"2023-08-09T13:51:59.165426+0000","last_change":"2023-08-09T13:51:59.165426+0000","last_active":"2023-08-09T13:51:59.130555+0000","last_peered":"2023-08-09T13:51:59.130555+0000","last_clean":"2023-08-09T13:51:59.130555+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-09T13:51:59.165426+0000","last_undegraded":"2023-08-09T13:51:59.165426+0000","last_fullsized":"2023-08-09T13:51:59.165426+0000","mapping_epoch":114,"log_start":"0'0","ondisk_log_start":"0'0","created":114,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:51:59.130555+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:51:59.130555+0000","last_clean_scrub_stamp":"2023-08-09T13:51:59.130555+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"14.1b","version":"0'0","reported_seq":9,"reported_epoch":114,"state":"active+clean","last_fresh":"2023-08-09T13:51:59.180443+0000","last_change":"2023-08-09T13:51:59.178308+0000","last_active":"2023-08-09T13:51:59.180443+0000","last_peered":"2023-08-09T13:51:59.180443+0000","last_clean":"2023-08-09T13:51:59.180443+0000","last_became_active":"2023-08-09T13:51:59.178022+0000","last_became_peered":"2023-08-09T13:51:59.178022+0000","last_unstale":"2023-08-09T13:51:59.180443+0000","last_undegraded":"2023-08-09T13:51:59.180443+0000","last_fullsized":"2023-08-09T13:51:59.180443+0000","mapping_epoch":113,"log_start":"0'0","ondisk_log_start":"0'0","created":113,"last_epoch_clean":114,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:51:58.125661+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:51:58.125661+0000","last_clean_scrub_stamp":"2023-08-09T13:51:58.125661+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T15:51:56.504946+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"15.1b","version":"0'0","reported_seq":1,"reported_epoch":114,"state":"creating+peering","last_fresh":"2023-08-09T13:51:59.157444+0000","last_change":"2023-08-09T13:51:59.157444+0000","last_active":"2023-08-09T13:51:59.130555+0000","last_peered":"2023-08-09T13:51:59.130555+0000","last_clean":"2023-08-09T13:51:59.130555+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-09T13:51:59.157444+0000","last_undegraded":"2023-08-09T13:51:59.157444+0000","last_fullsized":"2023-08-09T13:51:59.157444+0000","mapping_epoch":114,"log_start":"0'0","ondisk_log_start":"0'0","created":114,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:51:59.130555+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:51:59.130555+0000","last_clean_scrub_stamp":"2023-08-09T13:51:59.130555+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"14.1a","version":"0'0","reported_seq":9,"reported_epoch":114,"state":"active+clean","last_fresh":"2023-08-09T13:51:59.173191+0000","last_change":"2023-08-09T13:51:59.165522+0000","last_active":"2023-08-09T13:51:59.173191+0000","last_peered":"2023-08-09T13:51:59.173191+0000","last_clean":"2023-08-09T13:51:59.173191+0000","last_became_active":"2023-08-09T13:51:59.165145+0000","last_became_peered":"2023-08-09T13:51:59.165145+0000","last_unstale":"2023-08-09T13:51:59.173191+0000","last_undegraded":"2023-08-09T13:51:59.173191+0000","last_fullsized":"2023-08-09T13:51:59.173191+0000","mapping_epoch":113,"log_start":"0'0","ondisk_log_start":"0'0","created":113,"last_epoch_clean":114,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:51:58.125661+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:51:58.125661+0000","last_clean_scrub_stamp":"2023-08-09T13:51:58.125661+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T16:19:11.497849+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.18","version":"0'0","reported_seq":1,"reported_epoch":114,"state":"creating+peering","last_fresh":"2023-08-09T13:51:59.149114+0000","last_change":"2023-08-09T13:51:59.149114+0000","last_active":"2023-08-09T13:51:59.130555+0000","last_peered":"2023-08-09T13:51:59.130555+0000","last_clean":"2023-08-09T13:51:59.130555+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-09T13:51:59.149114+0000","last_undegraded":"2023-08-09T13:51:59.149114+0000","last_fullsized":"2023-08-09T13:51:59.149114+0000","mapping_epoch":114,"log_start":"0'0","ondisk_log_start":"0'0","created":114,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:51:59.130555+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:51:59.130555+0000","last_clean_scrub_stamp":"2023-08-09T13:51:59.130555+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"14.19","version":"0'0","reported_seq":9,"reported_epoch":114,"state":"active+clean","last_fresh":"2023-08-09T13:51:59.178533+0000","last_change":"2023-08-09T13:51:59.174362+0000","last_active":"2023-08-09T13:51:59.178533+0000","last_peered":"2023-08-09T13:51:59.178533+0000","last_clean":"2023-08-09T13:51:59.178533+0000","last_became_active":"2023-08-09T13:51:59.174002+0000","last_became_peered":"2023-08-09T13:51:59.174002+0000","last_unstale":"2023-08-09T13:51:59.178533+0000","last_undegraded":"2023-08-09T13:51:59.178533+0000","last_fullsized":"2023-08-09T13:51:59.178533+0000","mapping_epoch":113,"log_start":"0'0","ondisk_log_start":"0'0","created":113,"last_epoch_clean":114,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:51:58.125661+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:51:58.125661+0000","last_clean_scrub_stamp":"2023-08-09T13:51:58.125661+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T19:08:56.894026+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.19","version":"0'0","reported_seq":1,"reported_epoch":114,"state":"creating+peering","last_fresh":"2023-08-09T13:51:59.168819+0000","last_change":"2023-08-09T13:51:59.168819+0000","last_active":"2023-08-09T13:51:59.130555+0000","last_peered":"2023-08-09T13:51:59.130555+0000","last_clean":"2023-08-09T13:51:59.130555+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-09T13:51:59.168819+0000","last_undegraded":"2023-08-09T13:51:59.168819+0000","last_fullsized":"2023-08-09T13:51:59.168819+0000","mapping_epoch":114,"log_start":"0'0","ondisk_log_start":"0'0","created":114,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:51:59.130555+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:51:59.130555+0000","last_clean_scrub_stamp":"2023-08-09T13:51:59.130555+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.18","version":"0'0","reported_seq":9,"reported_epoch":114,"state":"active+clean","last_fresh":"2023-08-09T13:51:59.180392+0000","last_change":"2023-08-09T13:51:59.177991+0000","last_active":"2023-08-09T13:51:59.180392+0000","last_peered":"2023-08-09T13:51:59.180392+0000","last_clean":"2023-08-09T13:51:59.180392+0000","last_became_active":"2023-08-09T13:51:59.177651+0000","last_became_peered":"2023-08-09T13:51:59.177651+0000","last_unstale":"2023-08-09T13:51:59.180392+0000","last_undegraded":"2023-08-09T13:51:59.180392+0000","last_fullsized":"2023-08-09T13:51:59.180392+0000","mapping_epoch":113,"log_start":"0'0","ondisk_log_start":"0'0","created":113,"last_epoch_clean":114,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:51:58.125661+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:51:58.125661+0000","last_clean_scrub_stamp":"2023-08-09T13:51:58.125661+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T18:13:34.464631+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"15.16","version":"0'0","reported_seq":1,"reported_epoch":114,"state":"creating+peering","last_fresh":"2023-08-09T13:51:59.158359+0000","last_change":"2023-08-09T13:51:59.158359+0000","last_active":"2023-08-09T13:51:59.130555+0000","last_peered":"2023-08-09T13:51:59.130555+0000","last_clean":"2023-08-09T13:51:59.130555+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-09T13:51:59.158359+0000","last_undegraded":"2023-08-09T13:51:59.158359+0000","last_fullsized":"2023-08-09T13:51:59.158359+0000","mapping_epoch":114,"log_start":"0'0","ondisk_log_start":"0'0","created":114,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:51:59.130555+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:51:59.130555+0000","last_clean_scrub_stamp":"2023-08-09T13:51:59.130555+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"14.17","version":"0'0","reported_seq":9,"reported_epoch":114,"state":"active+clean","last_fresh":"2023-08-09T13:51:59.182321+0000","last_change":"2023-08-09T13:51:59.181245+0000","last_active":"2023-08-09T13:51:59.182321+0000","last_peered":"2023-08-09T13:51:59.182321+0000","last_clean":"2023-08-09T13:51:59.182321+0000","last_became_active":"2023-08-09T13:51:59.180911+0000","last_became_peered":"2023-08-09T13:51:59.180911+0000","last_unstale":"2023-08-09T13:51:59.182321+0000","last_undegraded":"2023-08-09T13:51:59.182321+0000","last_fullsized":"2023-08-09T13:51:59.182321+0000","mapping_epoch":113,"log_start":"0'0","ondisk_log_start":"0'0","created":113,"last_epoch_clean":114,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:51:58.125661+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:51:58.125661+0000","last_clean_scrub_stamp":"2023-08-09T13:51:58.125661+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T16:30:25.226241+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"15.17","version":"0'0","reported_seq":1,"reported_epoch":114,"state":"creating+peering","last_fresh":"2023-08-09T13:51:59.169791+0000","last_change":"2023-08-09T13:51:59.169791+0000","last_active":"2023-08-09T13:51:59.130555+0000","last_peered":"2023-08-09T13:51:59.130555+0000","last_clean":"2023-08-09T13:51:59.130555+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-09T13:51:59.169791+0000","last_undegraded":"2023-08-09T13:51:59.169791+0000","last_fullsized":"2023-08-09T13:51:59.169791+0000","mapping_epoch":114,"log_start":"0'0","ondisk_log_start":"0'0","created":114,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:51:59.130555+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:51:59.130555+0000","last_clean_scrub_stamp":"2023-08-09T13:51:59.130555+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.16","version":"0'0","reported_seq":9,"reported_epoch":114,"state":"active+clean","last_fresh":"2023-08-09T13:51:59.176606+0000","last_change":"2023-08-09T13:51:59.172670+0000","last_active":"2023-08-09T13:51:59.176606+0000","last_peered":"2023-08-09T13:51:59.176606+0000","last_clean":"2023-08-09T13:51:59.176606+0000","last_became_active":"2023-08-09T13:51:59.172395+0000","last_became_peered":"2023-08-09T13:51:59.172395+0000","last_unstale":"2023-08-09T13:51:59.176606+0000","last_undegraded":"2023-08-09T13:51:59.176606+0000","last_fullsized":"2023-08-09T13:51:59.176606+0000","mapping_epoch":113,"log_start":"0'0","ondisk_log_start":"0'0","created":113,"last_epoch_clean":114,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:51:58.125661+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:51:58.125661+0000","last_clean_scrub_stamp":"2023-08-09T13:51:58.125661+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T19:55:12.433811+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"15.14","version":"0'0","reported_seq":1,"reported_epoch":114,"state":"creating+peering","last_fresh":"2023-08-09T13:51:59.161549+0000","last_change":"2023-08-09T13:51:59.161549+0000","last_active":"2023-08-09T13:51:59.130555+0000","last_peered":"2023-08-09T13:51:59.130555+0000","last_clean":"2023-08-09T13:51:59.130555+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-09T13:51:59.161549+0000","last_undegraded":"2023-08-09T13:51:59.161549+0000","last_fullsized":"2023-08-09T13:51:59.161549+0000","mapping_epoch":114,"log_start":"0'0","ondisk_log_start":"0'0","created":114,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:51:59.130555+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:51:59.130555+0000","last_clean_scrub_stamp":"2023-08-09T13:51:59.130555+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"14.15","version":"0'0","reported_seq":9,"reported_epoch":114,"state":"active+clean","last_fresh":"2023-08-09T13:51:59.184646+0000","last_change":"2023-08-09T13:51:59.181964+0000","last_active":"2023-08-09T13:51:59.184646+0000","last_peered":"2023-08-09T13:51:59.184646+0000","last_clean":"2023-08-09T13:51:59.184646+0000","last_became_active":"2023-08-09T13:51:59.181630+0000","last_became_peered":"2023-08-09T13:51:59.181630+0000","last_unstale":"2023-08-09T13:51:59.184646+0000","last_undegraded":"2023-08-09T13:51:59.184646+0000","last_fullsized":"2023-08-09T13:51:59.184646+0000","mapping_epoch":113,"log_start":"0'0","ondisk_log_start":"0'0","created":113,"last_epoch_clean":114,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:51:58.125661+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:51:58.125661+0000","last_clean_scrub_stamp":"2023-08-09T13:51:58.125661+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T16:43:18.755379+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"15.15","version":"0'0","reported_seq":1,"reported_epoch":114,"state":"creating+peering","last_fresh":"2023-08-09T13:51:59.156075+0000","last_change":"2023-08-09T13:51:59.156075+0000","last_active":"2023-08-09T13:51:59.130555+0000","last_peered":"2023-08-09T13:51:59.130555+0000","last_clean":"2023-08-09T13:51:59.130555+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-09T13:51:59.156075+0000","last_undegraded":"2023-08-09T13:51:59.156075+0000","last_fullsized":"2023-08-09T13:51:59.156075+0000","mapping_epoch":114,"log_start":"0'0","ondisk_log_start":"0'0","created":114,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:51:59.130555+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:51:59.130555+0000","last_clean_scrub_stamp":"2023-08-09T13:51:59.130555+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"14.14","version":"0'0","reported_seq":9,"reported_epoch":114,"state":"active+clean","last_fresh":"2023-08-09T13:51:59.184692+0000","last_change":"2023-08-09T13:51:59.181967+0000","last_active":"2023-08-09T13:51:59.184692+0000","last_peered":"2023-08-09T13:51:59.184692+0000","last_clean":"2023-08-09T13:51:59.184692+0000","last_became_active":"2023-08-09T13:51:59.180664+0000","last_became_peered":"2023-08-09T13:51:59.180664+0000","last_unstale":"2023-08-09T13:51:59.184692+0000","last_undegraded":"2023-08-09T13:51:59.184692+0000","last_fullsized":"2023-08-09T13:51:59.184692+0000","mapping_epoch":113,"log_start":"0'0","ondisk_log_start":"0'0","created":113,"last_epoch_clean":114,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:51:58.125661+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:51:58.125661+0000","last_clean_scrub_stamp":"2023-08-09T13:51:58.125661+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-11T01:49:03.976153+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"15.12","version":"0'0","reported_seq":1,"reported_epoch":114,"state":"creating+peering","last_fresh":"2023-08-09T13:51:59.150320+0000","last_change":"2023-08-09T13:51:59.150320+0000","last_active":"2023-08-09T13:51:59.130555+0000","last_peered":"2023-08-09T13:51:59.130555+0000","last_clean":"2023-08-09T13:51:59.130555+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-09T13:51:59.150320+0000","last_undegraded":"2023-08-09T13:51:59.150320+0000","last_fullsized":"2023-08-09T13:51:59.150320+0000","mapping_epoch":114,"log_start":"0'0","ondisk_log_start":"0'0","created":114,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:51:59.130555+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:51:59.130555+0000","last_clean_scrub_stamp":"2023-08-09T13:51:59.130555+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"14.13","version":"0'0","reported_seq":9,"reported_epoch":114,"state":"active+clean","last_fresh":"2023-08-09T13:51:59.183063+0000","last_change":"2023-08-09T13:51:59.181856+0000","last_active":"2023-08-09T13:51:59.183063+0000","last_peered":"2023-08-09T13:51:59.183063+0000","last_clean":"2023-08-09T13:51:59.183063+0000","last_became_active":"2023-08-09T13:51:59.181568+0000","last_became_peered":"2023-08-09T13:51:59.181568+0000","last_unstale":"2023-08-09T13:51:59.183063+0000","last_undegraded":"2023-08-09T13:51:59.183063+0000","last_fullsized":"2023-08-09T13:51:59.183063+0000","mapping_epoch":113,"log_start":"0'0","ondisk_log_start":"0'0","created":113,"last_epoch_clean":114,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:51:58.125661+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:51:58.125661+0000","last_clean_scrub_stamp":"2023-08-09T13:51:58.125661+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T22:11:33.508209+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.13","version":"0'0","reported_seq":1,"reported_epoch":114,"state":"creating+peering","last_fresh":"2023-08-09T13:51:59.163775+0000","last_change":"2023-08-09T13:51:59.163775+0000","last_active":"2023-08-09T13:51:59.130555+0000","last_peered":"2023-08-09T13:51:59.130555+0000","last_clean":"2023-08-09T13:51:59.130555+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-09T13:51:59.163775+0000","last_undegraded":"2023-08-09T13:51:59.163775+0000","last_fullsized":"2023-08-09T13:51:59.163775+0000","mapping_epoch":114,"log_start":"0'0","ondisk_log_start":"0'0","created":114,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:51:59.130555+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:51:59.130555+0000","last_clean_scrub_stamp":"2023-08-09T13:51:59.130555+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.12","version":"0'0","reported_seq":9,"reported_epoch":114,"state":"active+clean","last_fresh":"2023-08-09T13:51:59.180137+0000","last_change":"2023-08-09T13:51:59.178710+0000","last_active":"2023-08-09T13:51:59.180137+0000","last_peered":"2023-08-09T13:51:59.180137+0000","last_clean":"2023-08-09T13:51:59.180137+0000","last_became_active":"2023-08-09T13:51:59.178402+0000","last_became_peered":"2023-08-09T13:51:59.178402+0000","last_unstale":"2023-08-09T13:51:59.180137+0000","last_undegraded":"2023-08-09T13:51:59.180137+0000","last_fullsized":"2023-08-09T13:51:59.180137+0000","mapping_epoch":113,"log_start":"0'0","ondisk_log_start":"0'0","created":113,"last_epoch_clean":114,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:51:58.125661+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:51:58.125661+0000","last_clean_scrub_stamp":"2023-08-09T13:51:58.125661+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T19:35:28.808687+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.10","version":"0'0","reported_seq":1,"reported_epoch":114,"state":"creating+peering","last_fresh":"2023-08-09T13:51:59.167863+0000","last_change":"2023-08-09T13:51:59.167863+0000","last_active":"2023-08-09T13:51:59.130555+0000","last_peered":"2023-08-09T13:51:59.130555+0000","last_clean":"2023-08-09T13:51:59.130555+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-09T13:51:59.167863+0000","last_undegraded":"2023-08-09T13:51:59.167863+0000","last_fullsized":"2023-08-09T13:51:59.167863+0000","mapping_epoch":114,"log_start":"0'0","ondisk_log_start":"0'0","created":114,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:51:59.130555+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:51:59.130555+0000","last_clean_scrub_stamp":"2023-08-09T13:51:59.130555+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.11","version":"0'0","reported_seq":9,"reported_epoch":114,"state":"active+clean","last_fresh":"2023-08-09T13:51:59.184619+0000","last_change":"2023-08-09T13:51:59.181121+0000","last_active":"2023-08-09T13:51:59.184619+0000","last_peered":"2023-08-09T13:51:59.184619+0000","last_clean":"2023-08-09T13:51:59.184619+0000","last_became_active":"2023-08-09T13:51:59.180844+0000","last_became_peered":"2023-08-09T13:51:59.180844+0000","last_unstale":"2023-08-09T13:51:59.184619+0000","last_undegraded":"2023-08-09T13:51:59.184619+0000","last_fullsized":"2023-08-09T13:51:59.184619+0000","mapping_epoch":113,"log_start":"0'0","ondisk_log_start":"0'0","created":113,"last_epoch_clean":114,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:51:58.125661+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:51:58.125661+0000","last_clean_scrub_stamp":"2023-08-09T13:51:58.125661+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T14:42:49.973080+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"15.11","version":"0'0","reported_seq":1,"reported_epoch":114,"state":"creating+peering","last_fresh":"2023-08-09T13:51:59.151102+0000","last_change":"2023-08-09T13:51:59.151102+0000","last_active":"2023-08-09T13:51:59.130555+0000","last_peered":"2023-08-09T13:51:59.130555+0000","last_clean":"2023-08-09T13:51:59.130555+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-09T13:51:59.151102+0000","last_undegraded":"2023-08-09T13:51:59.151102+0000","last_fullsized":"2023-08-09T13:51:59.151102+0000","mapping_epoch":114,"log_start":"0'0","ondisk_log_start":"0'0","created":114,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:51:59.130555+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:51:59.130555+0000","last_clean_scrub_stamp":"2023-08-09T13:51:59.130555+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"14.10","version":"0'0","reported_seq":9,"reported_epoch":114,"state":"active+clean","last_fresh":"2023-08-09T13:51:59.178095+0000","last_change":"2023-08-09T13:51:59.174790+0000","last_active":"2023-08-09T13:51:59.178095+0000","last_peered":"2023-08-09T13:51:59.178095+0000","last_clean":"2023-08-09T13:51:59.178095+0000","last_became_active":"2023-08-09T13:51:59.174453+0000","last_became_peered":"2023-08-09T13:51:59.174453+0000","last_unstale":"2023-08-09T13:51:59.178095+0000","last_undegraded":"2023-08-09T13:51:59.178095+0000","last_fullsized":"2023-08-09T13:51:59.178095+0000","mapping_epoch":113,"log_start":"0'0","ondisk_log_start":"0'0","created":113,"last_epoch_clean":114,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:51:58.125661+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:51:58.125661+0000","last_clean_scrub_stamp":"2023-08-09T13:51:58.125661+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T19:07:38.533940+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"15.6","version":"0'0","reported_seq":1,"reported_epoch":114,"state":"creating+peering","last_fresh":"2023-08-09T13:51:59.157299+0000","last_change":"2023-08-09T13:51:59.157299+0000","last_active":"2023-08-09T13:51:59.130555+0000","last_peered":"2023-08-09T13:51:59.130555+0000","last_clean":"2023-08-09T13:51:59.130555+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-09T13:51:59.157299+0000","last_undegraded":"2023-08-09T13:51:59.157299+0000","last_fullsized":"2023-08-09T13:51:59.157299+0000","mapping_epoch":114,"log_start":"0'0","ondisk_log_start":"0'0","created":114,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:51:59.130555+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:51:59.130555+0000","last_clean_scrub_stamp":"2023-08-09T13:51:59.130555+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"14.7","version":"0'0","reported_seq":9,"reported_epoch":114,"state":"active+clean","last_fresh":"2023-08-09T13:51:59.187317+0000","last_change":"2023-08-09T13:51:59.182730+0000","last_active":"2023-08-09T13:51:59.187317+0000","last_peered":"2023-08-09T13:51:59.187317+0000","last_clean":"2023-08-09T13:51:59.187317+0000","last_became_active":"2023-08-09T13:51:59.182469+0000","last_became_peered":"2023-08-09T13:51:59.182469+0000","last_unstale":"2023-08-09T13:51:59.187317+0000","last_undegraded":"2023-08-09T13:51:59.187317+0000","last_fullsized":"2023-08-09T13:51:59.187317+0000","mapping_epoch":113,"log_start":"0'0","ondisk_log_start":"0'0","created":113,"last_epoch_clean":114,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:51:58.125661+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:51:58.125661+0000","last_clean_scrub_stamp":"2023-08-09T13:51:58.125661+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T23:49:36.976561+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"15.7","version":"0'0","reported_seq":1,"reported_epoch":114,"state":"creating+peering","last_fresh":"2023-08-09T13:51:59.162752+0000","last_change":"2023-08-09T13:51:59.162752+0000","last_active":"2023-08-09T13:51:59.130555+0000","last_peered":"2023-08-09T13:51:59.130555+0000","last_clean":"2023-08-09T13:51:59.130555+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-09T13:51:59.162752+0000","last_undegraded":"2023-08-09T13:51:59.162752+0000","last_fullsized":"2023-08-09T13:51:59.162752+0000","mapping_epoch":114,"log_start":"0'0","ondisk_log_start":"0'0","created":114,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:51:59.130555+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:51:59.130555+0000","last_clean_scrub_stamp":"2023-08-09T13:51:59.130555+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.6","version":"0'0","reported_seq":9,"reported_epoch":114,"state":"active+clean","last_fresh":"2023-08-09T13:51:59.173431+0000","last_change":"2023-08-09T13:51:59.168280+0000","last_active":"2023-08-09T13:51:59.173431+0000","last_peered":"2023-08-09T13:51:59.173431+0000","last_clean":"2023-08-09T13:51:59.173431+0000","last_became_active":"2023-08-09T13:51:59.167980+0000","last_became_peered":"2023-08-09T13:51:59.167980+0000","last_unstale":"2023-08-09T13:51:59.173431+0000","last_undegraded":"2023-08-09T13:51:59.173431+0000","last_fullsized":"2023-08-09T13:51:59.173431+0000","mapping_epoch":113,"log_start":"0'0","ondisk_log_start":"0'0","created":113,"last_epoch_clean":114,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:51:58.125661+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:51:58.125661+0000","last_clean_scrub_stamp":"2023-08-09T13:51:58.125661+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-11T01:49:54.670225+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.4","version":"0'0","reported_seq":1,"reported_epoch":114,"state":"creating+peering","last_fresh":"2023-08-09T13:51:59.159598+0000","last_change":"2023-08-09T13:51:59.159598+0000","last_active":"2023-08-09T13:51:59.130555+0000","last_peered":"2023-08-09T13:51:59.130555+0000","last_clean":"2023-08-09T13:51:59.130555+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-09T13:51:59.159598+0000","last_undegraded":"2023-08-09T13:51:59.159598+0000","last_fullsized":"2023-08-09T13:51:59.159598+0000","mapping_epoch":114,"log_start":"0'0","ondisk_log_start":"0'0","created":114,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:51:59.130555+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:51:59.130555+0000","last_clean_scrub_stamp":"2023-08-09T13:51:59.130555+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.5","version":"0'0","reported_seq":9,"reported_epoch":114,"state":"active+clean","last_fresh":"2023-08-09T13:51:59.181921+0000","last_change":"2023-08-09T13:51:59.179649+0000","last_active":"2023-08-09T13:51:59.181921+0000","last_peered":"2023-08-09T13:51:59.181921+0000","last_clean":"2023-08-09T13:51:59.181921+0000","last_became_active":"2023-08-09T13:51:59.179348+0000","last_became_peered":"2023-08-09T13:51:59.179348+0000","last_unstale":"2023-08-09T13:51:59.181921+0000","last_undegraded":"2023-08-09T13:51:59.181921+0000","last_fullsized":"2023-08-09T13:51:59.181921+0000","mapping_epoch":113,"log_start":"0'0","ondisk_log_start":"0'0","created":113,"last_epoch_clean":114,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:51:58.125661+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:51:58.125661+0000","last_clean_scrub_stamp":"2023-08-09T13:51:58.125661+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T22:28:38.666057+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.5","version":"0'0","reported_seq":1,"reported_epoch":114,"state":"creating+peering","last_fresh":"2023-08-09T13:51:59.156952+0000","last_change":"2023-08-09T13:51:59.156952+0000","last_active":"2023-08-09T13:51:59.130555+0000","last_peered":"2023-08-09T13:51:59.130555+0000","last_clean":"2023-08-09T13:51:59.130555+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-09T13:51:59.156952+0000","last_undegraded":"2023-08-09T13:51:59.156952+0000","last_fullsized":"2023-08-09T13:51:59.156952+0000","mapping_epoch":114,"log_start":"0'0","ondisk_log_start":"0'0","created":114,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:51:59.130555+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:51:59.130555+0000","last_clean_scrub_stamp":"2023-08-09T13:51:59.130555+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"14.4","version":"0'0","reported_seq":9,"reported_epoch":114,"state":"active+clean","last_fresh":"2023-08-09T13:51:59.178493+0000","last_change":"2023-08-09T13:51:59.174311+0000","last_active":"2023-08-09T13:51:59.178493+0000","last_peered":"2023-08-09T13:51:59.178493+0000","last_clean":"2023-08-09T13:51:59.178493+0000","last_became_active":"2023-08-09T13:51:59.173990+0000","last_became_peered":"2023-08-09T13:51:59.173990+0000","last_unstale":"2023-08-09T13:51:59.178493+0000","last_undegraded":"2023-08-09T13:51:59.178493+0000","last_fullsized":"2023-08-09T13:51:59.178493+0000","mapping_epoch":113,"log_start":"0'0","ondisk_log_start":"0'0","created":113,"last_epoch_clean":114,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:51:58.125661+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:51:58.125661+0000","last_clean_scrub_stamp":"2023-08-09T13:51:58.125661+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T18:17:02.532080+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.2","version":"0'0","reported_seq":1,"reported_epoch":114,"state":"creating+peering","last_fresh":"2023-08-09T13:51:59.154951+0000","last_change":"2023-08-09T13:51:59.154951+0000","last_active":"2023-08-09T13:51:59.130555+0000","last_peered":"2023-08-09T13:51:59.130555+0000","last_clean":"2023-08-09T13:51:59.130555+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-09T13:51:59.154951+0000","last_undegraded":"2023-08-09T13:51:59.154951+0000","last_fullsized":"2023-08-09T13:51:59.154951+0000","mapping_epoch":114,"log_start":"0'0","ondisk_log_start":"0'0","created":114,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:51:59.130555+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:51:59.130555+0000","last_clean_scrub_stamp":"2023-08-09T13:51:59.130555+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"14.3","version":"0'0","reported_seq":9,"reported_epoch":114,"state":"active+clean","last_fresh":"2023-08-09T13:51:59.173345+0000","last_change":"2023-08-09T13:51:59.167523+0000","last_active":"2023-08-09T13:51:59.173345+0000","last_peered":"2023-08-09T13:51:59.173345+0000","last_clean":"2023-08-09T13:51:59.173345+0000","last_became_active":"2023-08-09T13:51:59.167059+0000","last_became_peered":"2023-08-09T13:51:59.167059+0000","last_unstale":"2023-08-09T13:51:59.173345+0000","last_undegraded":"2023-08-09T13:51:59.173345+0000","last_fullsized":"2023-08-09T13:51:59.173345+0000","mapping_epoch":113,"log_start":"0'0","ondisk_log_start":"0'0","created":113,"last_epoch_clean":114,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:51:58.125661+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:51:58.125661+0000","last_clean_scrub_stamp":"2023-08-09T13:51:58.125661+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T21:59:42.744613+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.3","version":"0'0","reported_seq":1,"reported_epoch":114,"state":"creating+peering","last_fresh":"2023-08-09T13:51:59.169608+0000","last_change":"2023-08-09T13:51:59.169608+0000","last_active":"2023-08-09T13:51:59.130555+0000","last_peered":"2023-08-09T13:51:59.130555+0000","last_clean":"2023-08-09T13:51:59.130555+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-09T13:51:59.169608+0000","last_undegraded":"2023-08-09T13:51:59.169608+0000","last_fullsized":"2023-08-09T13:51:59.169608+0000","mapping_epoch":114,"log_start":"0'0","ondisk_log_start":"0'0","created":114,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:51:59.130555+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:51:59.130555+0000","last_clean_scrub_stamp":"2023-08-09T13:51:59.130555+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.2","version":"0'0","reported_seq":9,"reported_epoch":114,"state":"active+clean","last_fresh":"2023-08-09T13:51:59.179577+0000","last_change":"2023-08-09T13:51:59.176919+0000","last_active":"2023-08-09T13:51:59.179577+0000","last_peered":"2023-08-09T13:51:59.179577+0000","last_clean":"2023-08-09T13:51:59.179577+0000","last_became_active":"2023-08-09T13:51:59.176503+0000","last_became_peered":"2023-08-09T13:51:59.176503+0000","last_unstale":"2023-08-09T13:51:59.179577+0000","last_undegraded":"2023-08-09T13:51:59.179577+0000","last_fullsized":"2023-08-09T13:51:59.179577+0000","mapping_epoch":113,"log_start":"0'0","ondisk_log_start":"0'0","created":113,"last_epoch_clean":114,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:51:58.125661+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:51:58.125661+0000","last_clean_scrub_stamp":"2023-08-09T13:51:58.125661+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T18:48:13.415415+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.0","version":"0'0","reported_seq":1,"reported_epoch":114,"state":"creating+peering","last_fresh":"2023-08-09T13:51:59.153156+0000","last_change":"2023-08-09T13:51:59.153156+0000","last_active":"2023-08-09T13:51:59.130555+0000","last_peered":"2023-08-09T13:51:59.130555+0000","last_clean":"2023-08-09T13:51:59.130555+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-09T13:51:59.153156+0000","last_undegraded":"2023-08-09T13:51:59.153156+0000","last_fullsized":"2023-08-09T13:51:59.153156+0000","mapping_epoch":114,"log_start":"0'0","ondisk_log_start":"0'0","created":114,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:51:59.130555+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:51:59.130555+0000","last_clean_scrub_stamp":"2023-08-09T13:51:59.130555+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"14.1","version":"0'0","reported_seq":9,"reported_epoch":114,"state":"active+clean","last_fresh":"2023-08-09T13:51:59.176213+0000","last_change":"2023-08-09T13:51:59.171070+0000","last_active":"2023-08-09T13:51:59.176213+0000","last_peered":"2023-08-09T13:51:59.176213+0000","last_clean":"2023-08-09T13:51:59.176213+0000","last_became_active":"2023-08-09T13:51:59.170788+0000","last_became_peered":"2023-08-09T13:51:59.170788+0000","last_unstale":"2023-08-09T13:51:59.176213+0000","last_undegraded":"2023-08-09T13:51:59.176213+0000","last_fullsized":"2023-08-09T13:51:59.176213+0000","mapping_epoch":113,"log_start":"0'0","ondisk_log_start":"0'0","created":113,"last_epoch_clean":114,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:51:58.125661+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:51:58.125661+0000","last_clean_scrub_stamp":"2023-08-09T13:51:58.125661+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-11T00:16:05.270843+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"15.1","version":"0'0","reported_seq":1,"reported_epoch":114,"state":"creating+peering","last_fresh":"2023-08-09T13:51:59.156158+0000","last_change":"2023-08-09T13:51:59.156158+0000","last_active":"2023-08-09T13:51:59.130555+0000","last_peered":"2023-08-09T13:51:59.130555+0000","last_clean":"2023-08-09T13:51:59.130555+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-09T13:51:59.156158+0000","last_undegraded":"2023-08-09T13:51:59.156158+0000","last_fullsized":"2023-08-09T13:51:59.156158+0000","mapping_epoch":114,"log_start":"0'0","ondisk_log_start":"0'0","created":114,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:51:59.130555+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:51:59.130555+0000","last_clean_scrub_stamp":"2023-08-09T13:51:59.130555+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"14.0","version":"0'0","reported_seq":9,"reported_epoch":114,"state":"active+clean","last_fresh":"2023-08-09T13:51:59.175060+0000","last_change":"2023-08-09T13:51:59.173223+0000","last_active":"2023-08-09T13:51:59.175060+0000","last_peered":"2023-08-09T13:51:59.175060+0000","last_clean":"2023-08-09T13:51:59.175060+0000","last_became_active":"2023-08-09T13:51:59.172948+0000","last_became_peered":"2023-08-09T13:51:59.172948+0000","last_unstale":"2023-08-09T13:51:59.175060+0000","last_undegraded":"2023-08-09T13:51:59.175060+0000","last_fullsized":"2023-08-09T13:51:59.175060+0000","mapping_epoch":113,"log_start":"0'0","ondisk_log_start":"0'0","created":113,"last_epoch_clean":114,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:51:58.125661+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:51:58.125661+0000","last_clean_scrub_stamp":"2023-08-09T13:51:58.125661+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T16:16:44.489876+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.e","version":"0'0","reported_seq":1,"reported_epoch":114,"state":"creating+peering","last_fresh":"2023-08-09T13:51:59.155432+0000","last_change":"2023-08-09T13:51:59.155432+0000","last_active":"2023-08-09T13:51:59.130555+0000","last_peered":"2023-08-09T13:51:59.130555+0000","last_clean":"2023-08-09T13:51:59.130555+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-09T13:51:59.155432+0000","last_undegraded":"2023-08-09T13:51:59.155432+0000","last_fullsized":"2023-08-09T13:51:59.155432+0000","mapping_epoch":114,"log_start":"0'0","ondisk_log_start":"0'0","created":114,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:51:59.130555+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:51:59.130555+0000","last_clean_scrub_stamp":"2023-08-09T13:51:59.130555+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"14.f","version":"0'0","reported_seq":9,"reported_epoch":114,"state":"active+clean","last_fresh":"2023-08-09T13:51:59.176216+0000","last_change":"2023-08-09T13:51:59.174685+0000","last_active":"2023-08-09T13:51:59.176216+0000","last_peered":"2023-08-09T13:51:59.176216+0000","last_clean":"2023-08-09T13:51:59.176216+0000","last_became_active":"2023-08-09T13:51:59.174370+0000","last_became_peered":"2023-08-09T13:51:59.174370+0000","last_unstale":"2023-08-09T13:51:59.176216+0000","last_undegraded":"2023-08-09T13:51:59.176216+0000","last_fullsized":"2023-08-09T13:51:59.176216+0000","mapping_epoch":113,"log_start":"0'0","ondisk_log_start":"0'0","created":113,"last_epoch_clean":114,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:51:58.125661+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:51:58.125661+0000","last_clean_scrub_stamp":"2023-08-09T13:51:58.125661+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T19:42:06.381104+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":189,"reported_epoch":114,"state":"active+clean","last_fresh":"2023-08-09T13:51:59.160631+0000","last_change":"2023-08-09T13:33:56.401406+0000","last_active":"2023-08-09T13:51:59.160631+0000","last_peered":"2023-08-09T13:51:59.160631+0000","last_clean":"2023-08-09T13:51:59.160631+0000","last_became_active":"2023-08-09T13:33:56.400487+0000","last_became_peered":"2023-08-09T13:33:56.400487+0000","last_unstale":"2023-08-09T13:51:59.160631+0000","last_undegraded":"2023-08-09T13:51:59.160631+0000","last_fullsized":"2023-08-09T13:51:59.160631+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:33:54.845869+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:33:54.845869+0000","last_clean_scrub_stamp":"2023-08-09T13:33:54.845869+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T16:56:32.266112+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"15.f","version":"0'0","reported_seq":1,"reported_epoch":114,"state":"creating+peering","last_fresh":"2023-08-09T13:51:59.156558+0000","last_change":"2023-08-09T13:51:59.156558+0000","last_active":"2023-08-09T13:51:59.130555+0000","last_peered":"2023-08-09T13:51:59.130555+0000","last_clean":"2023-08-09T13:51:59.130555+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-09T13:51:59.156558+0000","last_undegraded":"2023-08-09T13:51:59.156558+0000","last_fullsized":"2023-08-09T13:51:59.156558+0000","mapping_epoch":114,"log_start":"0'0","ondisk_log_start":"0'0","created":114,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:51:59.130555+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:51:59.130555+0000","last_clean_scrub_stamp":"2023-08-09T13:51:59.130555+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"14.e","version":"0'0","reported_seq":9,"reported_epoch":114,"state":"active+clean","last_fresh":"2023-08-09T13:51:59.182628+0000","last_change":"2023-08-09T13:51:59.180623+0000","last_active":"2023-08-09T13:51:59.182628+0000","last_peered":"2023-08-09T13:51:59.182628+0000","last_clean":"2023-08-09T13:51:59.182628+0000","last_became_active":"2023-08-09T13:51:59.178785+0000","last_became_peered":"2023-08-09T13:51:59.178785+0000","last_unstale":"2023-08-09T13:51:59.182628+0000","last_undegraded":"2023-08-09T13:51:59.182628+0000","last_fullsized":"2023-08-09T13:51:59.182628+0000","mapping_epoch":113,"log_start":"0'0","ondisk_log_start":"0'0","created":113,"last_epoch_clean":114,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:51:58.125661+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:51:58.125661+0000","last_clean_scrub_stamp":"2023-08-09T13:51:58.125661+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T21:35:50.978670+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":193,"reported_epoch":114,"state":"active+clean","last_fresh":"2023-08-09T13:51:59.171640+0000","last_change":"2023-08-09T13:33:56.401123+0000","last_active":"2023-08-09T13:51:59.171640+0000","last_peered":"2023-08-09T13:51:59.171640+0000","last_clean":"2023-08-09T13:51:59.171640+0000","last_became_active":"2023-08-09T13:33:56.399933+0000","last_became_peered":"2023-08-09T13:33:56.399933+0000","last_unstale":"2023-08-09T13:51:59.171640+0000","last_undegraded":"2023-08-09T13:51:59.171640+0000","last_fullsized":"2023-08-09T13:51:59.171640+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:33:54.845869+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:33:54.845869+0000","last_clean_scrub_stamp":"2023-08-09T13:33:54.845869+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T19:50:20.512873+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"15.c","version":"0'0","reported_seq":1,"reported_epoch":114,"state":"creating+peering","last_fresh":"2023-08-09T13:51:59.166603+0000","last_change":"2023-08-09T13:51:59.166603+0000","last_active":"2023-08-09T13:51:59.130555+0000","last_peered":"2023-08-09T13:51:59.130555+0000","last_clean":"2023-08-09T13:51:59.130555+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-09T13:51:59.166603+0000","last_undegraded":"2023-08-09T13:51:59.166603+0000","last_fullsized":"2023-08-09T13:51:59.166603+0000","mapping_epoch":114,"log_start":"0'0","ondisk_log_start":"0'0","created":114,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:51:59.130555+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:51:59.130555+0000","last_clean_scrub_stamp":"2023-08-09T13:51:59.130555+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"14.d","version":"0'0","reported_seq":9,"reported_epoch":114,"state":"active+clean","last_fresh":"2023-08-09T13:51:59.179009+0000","last_change":"2023-08-09T13:51:59.174909+0000","last_active":"2023-08-09T13:51:59.179009+0000","last_peered":"2023-08-09T13:51:59.179009+0000","last_clean":"2023-08-09T13:51:59.179009+0000","last_became_active":"2023-08-09T13:51:59.174527+0000","last_became_peered":"2023-08-09T13:51:59.174527+0000","last_unstale":"2023-08-09T13:51:59.179009+0000","last_undegraded":"2023-08-09T13:51:59.179009+0000","last_fullsized":"2023-08-09T13:51:59.179009+0000","mapping_epoch":113,"log_start":"0'0","ondisk_log_start":"0'0","created":113,"last_epoch_clean":114,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:51:58.125661+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:51:58.125661+0000","last_clean_scrub_stamp":"2023-08-09T13:51:58.125661+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T19:54:13.290921+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"1.2","version":"0'0","reported_seq":189,"reported_epoch":114,"state":"active+clean","last_fresh":"2023-08-09T13:51:59.161664+0000","last_change":"2023-08-09T13:33:56.402794+0000","last_active":"2023-08-09T13:51:59.161664+0000","last_peered":"2023-08-09T13:51:59.161664+0000","last_clean":"2023-08-09T13:51:59.161664+0000","last_became_active":"2023-08-09T13:33:56.402038+0000","last_became_peered":"2023-08-09T13:33:56.402038+0000","last_unstale":"2023-08-09T13:51:59.161664+0000","last_undegraded":"2023-08-09T13:51:59.161664+0000","last_fullsized":"2023-08-09T13:51:59.161664+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:33:54.845869+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:33:54.845869+0000","last_clean_scrub_stamp":"2023-08-09T13:33:54.845869+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T22:44:13.009942+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"15.d","version":"0'0","reported_seq":1,"reported_epoch":114,"state":"creating+peering","last_fresh":"2023-08-09T13:51:59.154844+0000","last_change":"2023-08-09T13:51:59.154844+0000","last_active":"2023-08-09T13:51:59.130555+0000","last_peered":"2023-08-09T13:51:59.130555+0000","last_clean":"2023-08-09T13:51:59.130555+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-09T13:51:59.154844+0000","last_undegraded":"2023-08-09T13:51:59.154844+0000","last_fullsized":"2023-08-09T13:51:59.154844+0000","mapping_epoch":114,"log_start":"0'0","ondisk_log_start":"0'0","created":114,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:51:59.130555+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:51:59.130555+0000","last_clean_scrub_stamp":"2023-08-09T13:51:59.130555+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"14.c","version":"0'0","reported_seq":9,"reported_epoch":114,"state":"active+clean","last_fresh":"2023-08-09T13:51:59.184772+0000","last_change":"2023-08-09T13:51:59.183702+0000","last_active":"2023-08-09T13:51:59.184772+0000","last_peered":"2023-08-09T13:51:59.184772+0000","last_clean":"2023-08-09T13:51:59.184772+0000","last_became_active":"2023-08-09T13:51:59.183387+0000","last_became_peered":"2023-08-09T13:51:59.183387+0000","last_unstale":"2023-08-09T13:51:59.184772+0000","last_undegraded":"2023-08-09T13:51:59.184772+0000","last_fullsized":"2023-08-09T13:51:59.184772+0000","mapping_epoch":113,"log_start":"0'0","ondisk_log_start":"0'0","created":113,"last_epoch_clean":114,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:51:58.125661+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:51:58.125661+0000","last_clean_scrub_stamp":"2023-08-09T13:51:58.125661+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T16:05:08.950799+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.3","version":"0'0","reported_seq":189,"reported_epoch":114,"state":"active+clean","last_fresh":"2023-08-09T13:51:59.169171+0000","last_change":"2023-08-09T13:33:55.886749+0000","last_active":"2023-08-09T13:51:59.169171+0000","last_peered":"2023-08-09T13:51:59.169171+0000","last_clean":"2023-08-09T13:51:59.169171+0000","last_became_active":"2023-08-09T13:33:55.886116+0000","last_became_peered":"2023-08-09T13:33:55.886116+0000","last_unstale":"2023-08-09T13:51:59.169171+0000","last_undegraded":"2023-08-09T13:51:59.169171+0000","last_fullsized":"2023-08-09T13:51:59.169171+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:33:54.845869+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:33:54.845869+0000","last_clean_scrub_stamp":"2023-08-09T13:33:54.845869+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T20:06:41.616544+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"105'186","reported_seq":583,"reported_epoch":114,"state":"active+clean","last_fresh":"2023-08-09T13:51:59.160322+0000","last_change":"2023-08-09T13:33:55.886345+0000","last_active":"2023-08-09T13:51:59.160322+0000","last_peered":"2023-08-09T13:51:59.160322+0000","last_clean":"2023-08-09T13:51:59.160322+0000","last_became_active":"2023-08-09T13:33:55.885781+0000","last_became_peered":"2023-08-09T13:33:55.885781+0000","last_unstale":"2023-08-09T13:51:59.160322+0000","last_undegraded":"2023-08-09T13:51:59.160322+0000","last_fullsized":"2023-08-09T13:51:59.160322+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:33:54.845869+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:33:54.845869+0000","last_clean_scrub_stamp":"2023-08-09T13:33:54.845869+0000","objects_scrubbed":0,"log_size":186,"log_dups_size":0,"ondisk_log_size":186,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T16:37:53.138245+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":715,"num_read_kb":1418,"num_write":283,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"15.a","version":"0'0","reported_seq":1,"reported_epoch":114,"state":"creating+peering","last_fresh":"2023-08-09T13:51:59.164187+0000","last_change":"2023-08-09T13:51:59.164187+0000","last_active":"2023-08-09T13:51:59.130555+0000","last_peered":"2023-08-09T13:51:59.130555+0000","last_clean":"2023-08-09T13:51:59.130555+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-09T13:51:59.164187+0000","last_undegraded":"2023-08-09T13:51:59.164187+0000","last_fullsized":"2023-08-09T13:51:59.164187+0000","mapping_epoch":114,"log_start":"0'0","ondisk_log_start":"0'0","created":114,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:51:59.130555+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:51:59.130555+0000","last_clean_scrub_stamp":"2023-08-09T13:51:59.130555+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"14.b","version":"0'0","reported_seq":9,"reported_epoch":114,"state":"active+clean","last_fresh":"2023-08-09T13:51:59.173500+0000","last_change":"2023-08-09T13:51:59.168603+0000","last_active":"2023-08-09T13:51:59.173500+0000","last_peered":"2023-08-09T13:51:59.173500+0000","last_clean":"2023-08-09T13:51:59.173500+0000","last_became_active":"2023-08-09T13:51:59.168273+0000","last_became_peered":"2023-08-09T13:51:59.168273+0000","last_unstale":"2023-08-09T13:51:59.173500+0000","last_undegraded":"2023-08-09T13:51:59.173500+0000","last_fullsized":"2023-08-09T13:51:59.173500+0000","mapping_epoch":113,"log_start":"0'0","ondisk_log_start":"0'0","created":113,"last_epoch_clean":114,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:51:58.125661+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:51:58.125661+0000","last_clean_scrub_stamp":"2023-08-09T13:51:58.125661+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T20:40:57.215042+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":199,"reported_epoch":114,"state":"active+clean","last_fresh":"2023-08-09T13:51:59.163213+0000","last_change":"2023-08-09T13:33:55.889565+0000","last_active":"2023-08-09T13:51:59.163213+0000","last_peered":"2023-08-09T13:51:59.163213+0000","last_clean":"2023-08-09T13:51:59.163213+0000","last_became_active":"2023-08-09T13:33:55.889164+0000","last_became_peered":"2023-08-09T13:33:55.889164+0000","last_unstale":"2023-08-09T13:51:59.163213+0000","last_undegraded":"2023-08-09T13:51:59.163213+0000","last_fullsized":"2023-08-09T13:51:59.163213+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:33:54.845869+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:33:54.845869+0000","last_clean_scrub_stamp":"2023-08-09T13:33:54.845869+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T19:12:54.459595+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"15.b","version":"0'0","reported_seq":1,"reported_epoch":114,"state":"creating+peering","last_fresh":"2023-08-09T13:51:59.156039+0000","last_change":"2023-08-09T13:51:59.156039+0000","last_active":"2023-08-09T13:51:59.130555+0000","last_peered":"2023-08-09T13:51:59.130555+0000","last_clean":"2023-08-09T13:51:59.130555+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-09T13:51:59.156039+0000","last_undegraded":"2023-08-09T13:51:59.156039+0000","last_fullsized":"2023-08-09T13:51:59.156039+0000","mapping_epoch":114,"log_start":"0'0","ondisk_log_start":"0'0","created":114,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:51:59.130555+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:51:59.130555+0000","last_clean_scrub_stamp":"2023-08-09T13:51:59.130555+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"14.a","version":"0'0","reported_seq":9,"reported_epoch":114,"state":"active+clean","last_fresh":"2023-08-09T13:51:59.182288+0000","last_change":"2023-08-09T13:51:59.179151+0000","last_active":"2023-08-09T13:51:59.182288+0000","last_peered":"2023-08-09T13:51:59.182288+0000","last_clean":"2023-08-09T13:51:59.182288+0000","last_became_active":"2023-08-09T13:51:59.178247+0000","last_became_peered":"2023-08-09T13:51:59.178247+0000","last_unstale":"2023-08-09T13:51:59.182288+0000","last_undegraded":"2023-08-09T13:51:59.182288+0000","last_fullsized":"2023-08-09T13:51:59.182288+0000","mapping_epoch":113,"log_start":"0'0","ondisk_log_start":"0'0","created":113,"last_epoch_clean":114,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:51:58.125661+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:51:58.125661+0000","last_clean_scrub_stamp":"2023-08-09T13:51:58.125661+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T14:19:12.218569+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":193,"reported_epoch":114,"state":"active+clean","last_fresh":"2023-08-09T13:51:59.180570+0000","last_change":"2023-08-09T13:33:56.404236+0000","last_active":"2023-08-09T13:51:59.180570+0000","last_peered":"2023-08-09T13:51:59.180570+0000","last_clean":"2023-08-09T13:51:59.180570+0000","last_became_active":"2023-08-09T13:33:56.402485+0000","last_became_peered":"2023-08-09T13:33:56.402485+0000","last_unstale":"2023-08-09T13:51:59.180570+0000","last_undegraded":"2023-08-09T13:51:59.180570+0000","last_fullsized":"2023-08-09T13:51:59.180570+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:33:54.845869+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:33:54.845869+0000","last_clean_scrub_stamp":"2023-08-09T13:33:54.845869+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T20:10:52.098147+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"15.8","version":"0'0","reported_seq":1,"reported_epoch":114,"state":"creating+peering","last_fresh":"2023-08-09T13:51:59.162674+0000","last_change":"2023-08-09T13:51:59.162674+0000","last_active":"2023-08-09T13:51:59.130555+0000","last_peered":"2023-08-09T13:51:59.130555+0000","last_clean":"2023-08-09T13:51:59.130555+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-09T13:51:59.162674+0000","last_undegraded":"2023-08-09T13:51:59.162674+0000","last_fullsized":"2023-08-09T13:51:59.162674+0000","mapping_epoch":114,"log_start":"0'0","ondisk_log_start":"0'0","created":114,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:51:59.130555+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:51:59.130555+0000","last_clean_scrub_stamp":"2023-08-09T13:51:59.130555+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"14.9","version":"0'0","reported_seq":9,"reported_epoch":114,"state":"active+clean","last_fresh":"2023-08-09T13:51:59.180017+0000","last_change":"2023-08-09T13:51:59.177964+0000","last_active":"2023-08-09T13:51:59.180017+0000","last_peered":"2023-08-09T13:51:59.180017+0000","last_clean":"2023-08-09T13:51:59.180017+0000","last_became_active":"2023-08-09T13:51:59.177186+0000","last_became_peered":"2023-08-09T13:51:59.177186+0000","last_unstale":"2023-08-09T13:51:59.180017+0000","last_undegraded":"2023-08-09T13:51:59.180017+0000","last_fullsized":"2023-08-09T13:51:59.180017+0000","mapping_epoch":113,"log_start":"0'0","ondisk_log_start":"0'0","created":113,"last_epoch_clean":114,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:51:58.125661+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:51:58.125661+0000","last_clean_scrub_stamp":"2023-08-09T13:51:58.125661+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T18:14:44.331723+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":189,"reported_epoch":114,"state":"active+clean","last_fresh":"2023-08-09T13:51:59.157955+0000","last_change":"2023-08-09T13:33:56.401781+0000","last_active":"2023-08-09T13:51:59.157955+0000","last_peered":"2023-08-09T13:51:59.157955+0000","last_clean":"2023-08-09T13:51:59.157955+0000","last_became_active":"2023-08-09T13:33:56.400336+0000","last_became_peered":"2023-08-09T13:33:56.400336+0000","last_unstale":"2023-08-09T13:51:59.157955+0000","last_undegraded":"2023-08-09T13:51:59.157955+0000","last_fullsized":"2023-08-09T13:51:59.157955+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:33:54.845869+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:33:54.845869+0000","last_clean_scrub_stamp":"2023-08-09T13:33:54.845869+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T16:05:37.049450+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.9","version":"0'0","reported_seq":1,"reported_epoch":114,"state":"creating+peering","last_fresh":"2023-08-09T13:51:59.157318+0000","last_change":"2023-08-09T13:51:59.157318+0000","last_active":"2023-08-09T13:51:59.130555+0000","last_peered":"2023-08-09T13:51:59.130555+0000","last_clean":"2023-08-09T13:51:59.130555+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-09T13:51:59.157318+0000","last_undegraded":"2023-08-09T13:51:59.157318+0000","last_fullsized":"2023-08-09T13:51:59.157318+0000","mapping_epoch":114,"log_start":"0'0","ondisk_log_start":"0'0","created":114,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:51:59.130555+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:51:59.130555+0000","last_clean_scrub_stamp":"2023-08-09T13:51:59.130555+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"14.8","version":"0'0","reported_seq":9,"reported_epoch":114,"state":"active+clean","last_fresh":"2023-08-09T13:51:59.174355+0000","last_change":"2023-08-09T13:51:59.170689+0000","last_active":"2023-08-09T13:51:59.174355+0000","last_peered":"2023-08-09T13:51:59.174355+0000","last_clean":"2023-08-09T13:51:59.174355+0000","last_became_active":"2023-08-09T13:51:59.170409+0000","last_became_peered":"2023-08-09T13:51:59.170409+0000","last_unstale":"2023-08-09T13:51:59.174355+0000","last_undegraded":"2023-08-09T13:51:59.174355+0000","last_fullsized":"2023-08-09T13:51:59.174355+0000","mapping_epoch":113,"log_start":"0'0","ondisk_log_start":"0'0","created":113,"last_epoch_clean":114,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:51:58.125661+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:51:58.125661+0000","last_clean_scrub_stamp":"2023-08-09T13:51:58.125661+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T17:00:34.835236+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":189,"reported_epoch":114,"state":"active+clean","last_fresh":"2023-08-09T13:51:59.159658+0000","last_change":"2023-08-09T13:33:55.886389+0000","last_active":"2023-08-09T13:51:59.159658+0000","last_peered":"2023-08-09T13:51:59.159658+0000","last_clean":"2023-08-09T13:51:59.159658+0000","last_became_active":"2023-08-09T13:33:55.885638+0000","last_became_peered":"2023-08-09T13:33:55.885638+0000","last_unstale":"2023-08-09T13:51:59.159658+0000","last_undegraded":"2023-08-09T13:51:59.159658+0000","last_fullsized":"2023-08-09T13:51:59.159658+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:33:54.845869+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:33:54.845869+0000","last_clean_scrub_stamp":"2023-08-09T13:33:54.845869+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T16:40:37.514883+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]}],"pool_stats":[{"poolid":15,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":2,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":715,"num_read_kb":1418,"num_write":283,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":186,"ondisk_log_size":186,"up":2,"acting":2,"num_store_stats":2},{"poolid":14,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":8}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834575064,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":37168,"kb_used_data":1952,"kb_used_omap":1,"kb_used_meta":35198,"kb_avail":94334672,"statfs":{"total":96636764160,"available":96598704128,"internally_reserved":0,"allocated":1998848,"data_stored":1338378,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1781,"internal_metadata":36043019},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Aug 9 13:51:20 2023","interfaces":[{"interface":"back","average":{"1min":1.101,"5min":1.1910000000000001,"15min":1.2370000000000001},"min":{"1min":0.61599999999999999,"5min":0.61599999999999999,"15min":0.61599999999999999},"max":{"1min":1.764,"5min":1.8919999999999999,"15min":2.1280000000000001},"last":1.1899999999999999},{"interface":"front","average":{"1min":1.153,"5min":1.224,"15min":1.2430000000000001},"min":{"1min":0.64200000000000002,"5min":0.64200000000000002,"15min":0.64200000000000002},"max":{"1min":1.8839999999999999,"5min":1.8839999999999999,"15min":2.0099999999999998},"last":1.1259999999999999}]},{"osd":1,"last update":"Wed Aug 9 13:51:20 2023","interfaces":[{"interface":"back","average":{"1min":1.1830000000000001,"5min":1.22,"15min":1.26},"min":{"1min":0.69199999999999995,"5min":0.69199999999999995,"15min":0.59799999999999998},"max":{"1min":1.8120000000000001,"5min":1.8120000000000001,"15min":1.9470000000000001},"last":1.4390000000000001},{"interface":"front","average":{"1min":1.1839999999999999,"5min":1.2310000000000001,"15min":1.2809999999999999},"min":{"1min":0.69099999999999995,"5min":0.59299999999999997,"15min":0.59299999999999997},"max":{"1min":1.998,"5min":1.998,"15min":1.998},"last":1.2789999999999999}]},{"osd":2,"last update":"Wed Aug 9 13:51:20 2023","interfaces":[{"interface":"back","average":{"1min":1.1990000000000001,"5min":1.234,"15min":1.252},"min":{"1min":0.79100000000000004,"5min":0.65700000000000003,"15min":0.65700000000000003},"max":{"1min":1.6539999999999999,"5min":2.0129999999999999,"15min":2.0129999999999999},"last":1.518},{"interface":"front","average":{"1min":1.22,"5min":1.2829999999999999,"15min":1.369},"min":{"1min":0.58199999999999996,"5min":0.58199999999999996,"15min":0.58199999999999996},"max":{"1min":1.7050000000000001,"5min":1.8740000000000001,"15min":2.4140000000000001},"last":1.5489999999999999}]},{"osd":3,"last update":"Wed Aug 9 13:51:20 2023","interfaces":[{"interface":"back","average":{"1min":1.272,"5min":1.29,"15min":1.3360000000000001},"min":{"1min":0.67500000000000004,"5min":0.67500000000000004,"15min":0.67500000000000004},"max":{"1min":1.8140000000000001,"5min":1.97,"15min":1.97},"last":1.6180000000000001},{"interface":"front","average":{"1min":1.363,"5min":1.3959999999999999,"15min":1.413},"min":{"1min":0.71499999999999997,"5min":0.71499999999999997,"15min":0.71499999999999997},"max":{"1min":1.615,"5min":2.0790000000000002,"15min":2.2480000000000002},"last":1.647}]},{"osd":4,"last update":"Wed Aug 9 13:51:20 2023","interfaces":[{"interface":"back","average":{"1min":1.2430000000000001,"5min":1.2669999999999999,"15min":1.2749999999999999},"min":{"1min":0.74299999999999999,"5min":0.59499999999999997,"15min":0.59499999999999997},"max":{"1min":2.097,"5min":2.097,"15min":2.343},"last":1.008},{"interface":"front","average":{"1min":1.2509999999999999,"5min":1.2969999999999999,"15min":1.3340000000000001},"min":{"1min":0.60799999999999998,"5min":0.60799999999999998,"15min":0.60799999999999998},"max":{"1min":2.0539999999999998,"5min":2.1419999999999999,"15min":2.1419999999999999},"last":1.333}]},{"osd":5,"last update":"Wed Aug 9 13:51:20 2023","interfaces":[{"interface":"back","average":{"1min":1.456,"5min":1.4350000000000001,"15min":1.4179999999999999},"min":{"1min":0.65800000000000003,"5min":0.65800000000000003,"15min":0.65800000000000003},"max":{"1min":1.7549999999999999,"5min":2.1480000000000001,"15min":2.6749999999999998},"last":1.052},{"interface":"front","average":{"1min":1.383,"5min":1.4099999999999999,"15min":1.4450000000000001},"min":{"1min":0.83999999999999997,"5min":0.83999999999999997,"15min":0.79700000000000004},"max":{"1min":2.157,"5min":2.157,"15min":2.9569999999999999},"last":1.3859999999999999}]},{"osd":6,"last update":"Wed Aug 9 13:51:20 2023","interfaces":[{"interface":"back","average":{"1min":1.4650000000000001,"5min":1.504,"15min":1.5429999999999999},"min":{"1min":0.88500000000000001,"5min":0.85499999999999998,"15min":0.85499999999999998},"max":{"1min":1.8280000000000001,"5min":2.2069999999999999,"15min":2.8149999999999999},"last":1.4870000000000001},{"interface":"front","average":{"1min":1.421,"5min":1.4430000000000001,"15min":1.474},"min":{"1min":0.79700000000000004,"5min":0.79700000000000004,"15min":0.79700000000000004},"max":{"1min":1.736,"5min":2.1320000000000001,"15min":2.2530000000000001},"last":1.7170000000000001}]}]},{"osd":6,"up_from":13,"seq":55834575064,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":36580,"kb_used_data":1368,"kb_used_omap":1,"kb_used_meta":35198,"kb_avail":94335260,"statfs":{"total":96636764160,"available":96599306240,"internally_reserved":0,"allocated":1400832,"data_stored":746649,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1781,"internal_metadata":36043019},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":8,"apply_latency_ms":8,"commit_latency_ns":8000000,"apply_latency_ns":8000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Aug 9 13:51:24 2023","interfaces":[{"interface":"back","average":{"1min":0.98499999999999999,"5min":1.1160000000000001,"15min":1.153},"min":{"1min":0.54000000000000004,"5min":0.54000000000000004,"15min":0.54000000000000004},"max":{"1min":1.484,"5min":1.633,"15min":1.9870000000000001},"last":0.96899999999999997},{"interface":"front","average":{"1min":1.0620000000000001,"5min":1.492,"15min":1.306},"min":{"1min":0.51800000000000002,"5min":0.51800000000000002,"15min":0.51800000000000002},"max":{"1min":1.6479999999999999,"5min":28.036000000000001,"15min":28.036000000000001},"last":1.0800000000000001}]},{"osd":1,"last update":"Wed Aug 9 13:51:24 2023","interfaces":[{"interface":"back","average":{"1min":1.0940000000000001,"5min":1.238,"15min":1.232},"min":{"1min":0.48999999999999999,"5min":0.48999999999999999,"15min":0.48999999999999999},"max":{"1min":1.6639999999999999,"5min":7.742,"15min":7.742},"last":0.79800000000000004},{"interface":"front","average":{"1min":1.1950000000000001,"5min":1.276,"15min":1.28},"min":{"1min":0.64600000000000002,"5min":0.64600000000000002,"15min":0.64600000000000002},"max":{"1min":2.056,"5min":2.056,"15min":2.056},"last":1.222}]},{"osd":2,"last update":"Wed Aug 9 13:51:24 2023","interfaces":[{"interface":"back","average":{"1min":1.206,"5min":1.323,"15min":1.3260000000000001},"min":{"1min":0.80600000000000005,"5min":0.80600000000000005,"15min":0.74099999999999999},"max":{"1min":1.8819999999999999,"5min":4.0640000000000001,"15min":4.0640000000000001},"last":1.147},{"interface":"front","average":{"1min":1.161,"5min":1.399,"15min":1.347},"min":{"1min":0.80700000000000005,"5min":0.70999999999999996,"15min":0.70999999999999996},"max":{"1min":1.6679999999999999,"5min":13.154999999999999,"15min":13.154999999999999},"last":1.5029999999999999}]},{"osd":3,"last update":"Wed Aug 9 13:51:24 2023","interfaces":[{"interface":"back","average":{"1min":1.196,"5min":1.452,"15min":1.415},"min":{"1min":0.67200000000000004,"5min":0.67200000000000004,"15min":0.67200000000000004},"max":{"1min":1.522,"5min":13.349,"15min":13.349},"last":1.401},{"interface":"front","average":{"1min":1.254,"5min":1.353,"15min":1.3280000000000001},"min":{"1min":0.75700000000000001,"5min":0.75700000000000001,"15min":0.74299999999999999},"max":{"1min":1.732,"5min":7.9029999999999996,"15min":7.9029999999999996},"last":1.319}]},{"osd":4,"last update":"Wed Aug 9 13:51:24 2023","interfaces":[{"interface":"back","average":{"1min":1.323,"5min":1.3540000000000001,"15min":1.284},"min":{"1min":0.86199999999999999,"5min":0.84499999999999997,"15min":0.64300000000000002},"max":{"1min":1.833,"5min":5.2290000000000001,"15min":5.2290000000000001},"last":1.3640000000000001},{"interface":"front","average":{"1min":1.367,"5min":1.506,"15min":1.3879999999999999},"min":{"1min":0.89900000000000002,"5min":0.78700000000000003,"15min":0.78700000000000003},"max":{"1min":1.714,"5min":12.1,"15min":12.1},"last":1.27}]},{"osd":5,"last update":"Wed Aug 9 13:51:24 2023","interfaces":[{"interface":"back","average":{"1min":1.423,"5min":1.446,"15min":1.3660000000000001},"min":{"1min":0.57699999999999996,"5min":0.57699999999999996,"15min":0.57699999999999996},"max":{"1min":1.843,"5min":8.1899999999999995,"15min":8.1899999999999995},"last":1.1850000000000001},{"interface":"front","average":{"1min":1.55,"5min":1.6679999999999999,"15min":1.593},"min":{"1min":0.93000000000000005,"5min":0.93000000000000005,"15min":0.83799999999999997},"max":{"1min":2.1459999999999999,"5min":8.8889999999999993,"15min":8.8889999999999993},"last":1.4379999999999999}]},{"osd":7,"last update":"Wed Aug 9 13:51:24 2023","interfaces":[{"interface":"back","average":{"1min":1.5269999999999999,"5min":1.6719999999999999,"15min":1.587},"min":{"1min":0.96099999999999997,"5min":0.96099999999999997,"15min":0.62},"max":{"1min":1.907,"5min":9.3320000000000007,"15min":9.3320000000000007},"last":1.6359999999999999},{"interface":"front","average":{"1min":1.556,"5min":1.613,"15min":1.55},"min":{"1min":0.88200000000000001,"5min":0.88200000000000001,"15min":0.88200000000000001},"max":{"1min":1.97,"5min":8.0399999999999991,"15min":8.0399999999999991},"last":1.4690000000000001}]}]},{"osd":5,"up_from":13,"seq":55834575064,"num_pgs":5,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":40668,"kb_used_data":1360,"kb_used_omap":1,"kb_used_meta":39294,"kb_avail":94331172,"statfs":{"total":96636764160,"available":96595120128,"internally_reserved":0,"allocated":1392640,"data_stored":741018,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1781,"internal_metadata":40237323},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Aug 9 13:51:21 2023","interfaces":[{"interface":"back","average":{"1min":1.542,"5min":1.25,"15min":1.206},"min":{"1min":0.59099999999999997,"5min":0.59099999999999997,"15min":0.59099999999999997},"max":{"1min":8.9299999999999997,"5min":8.9299999999999997,"15min":8.9299999999999997},"last":1.2350000000000001},{"interface":"front","average":{"1min":1.6080000000000001,"5min":1.264,"15min":1.2330000000000001},"min":{"1min":0.60499999999999998,"5min":0.60499999999999998,"15min":0.60499999999999998},"max":{"1min":8.7140000000000004,"5min":8.7140000000000004,"15min":8.7140000000000004},"last":1.6160000000000001}]},{"osd":1,"last update":"Wed Aug 9 13:51:21 2023","interfaces":[{"interface":"back","average":{"1min":1.476,"5min":1.2330000000000001,"15min":1.246},"min":{"1min":0.68799999999999994,"5min":0.68799999999999994,"15min":0.64100000000000001},"max":{"1min":8.2720000000000002,"5min":8.2720000000000002,"15min":8.2720000000000002},"last":0.82399999999999995},{"interface":"front","average":{"1min":1.6679999999999999,"5min":1.359,"15min":1.3100000000000001},"min":{"1min":0.80500000000000005,"5min":0.66900000000000004,"15min":0.66900000000000004},"max":{"1min":9.0020000000000007,"5min":9.0020000000000007,"15min":9.0020000000000007},"last":1.3720000000000001}]},{"osd":2,"last update":"Wed Aug 9 13:51:21 2023","interfaces":[{"interface":"back","average":{"1min":1.7190000000000001,"5min":1.335,"15min":1.319},"min":{"1min":0.81799999999999995,"5min":0.68999999999999995,"15min":0.61599999999999999},"max":{"1min":8.6050000000000004,"5min":8.6050000000000004,"15min":8.6050000000000004},"last":1.1439999999999999},{"interface":"front","average":{"1min":1.641,"5min":1.371,"15min":1.347},"min":{"1min":0.78300000000000003,"5min":0.78300000000000003,"15min":0.72099999999999997},"max":{"1min":8.4149999999999991,"5min":8.4149999999999991,"15min":8.4149999999999991},"last":1.4299999999999999}]},{"osd":3,"last update":"Wed Aug 9 13:51:21 2023","interfaces":[{"interface":"back","average":{"1min":1.8160000000000001,"5min":1.466,"15min":1.391},"min":{"1min":0.72599999999999998,"5min":0.72599999999999998,"15min":0.59799999999999998},"max":{"1min":9.0470000000000006,"5min":9.0470000000000006,"15min":9.0470000000000006},"last":1.573},{"interface":"front","average":{"1min":1.738,"5min":1.4970000000000001,"15min":1.4370000000000001},"min":{"1min":0.76300000000000001,"5min":0.76300000000000001,"15min":0.70999999999999996},"max":{"1min":8.7669999999999995,"5min":8.7669999999999995,"15min":8.7669999999999995},"last":1.302}]},{"osd":4,"last update":"Wed Aug 9 13:51:21 2023","interfaces":[{"interface":"back","average":{"1min":1.724,"5min":1.367,"15min":1.321},"min":{"1min":0.54300000000000004,"5min":0.54300000000000004,"15min":0.54300000000000004},"max":{"1min":8.4909999999999997,"5min":8.4909999999999997,"15min":8.4909999999999997},"last":1.55},{"interface":"front","average":{"1min":1.9079999999999999,"5min":1.4570000000000001,"15min":1.4119999999999999},"min":{"1min":0.68899999999999995,"5min":0.68899999999999995,"15min":0.68899999999999995},"max":{"1min":9.4589999999999996,"5min":9.4589999999999996,"15min":9.4589999999999996},"last":1.6579999999999999}]},{"osd":6,"last update":"Wed Aug 9 13:51:21 2023","interfaces":[{"interface":"back","average":{"1min":2.5259999999999998,"5min":1.5980000000000001,"15min":1.4610000000000001},"min":{"1min":0.51000000000000001,"5min":0.51000000000000001,"15min":0.51000000000000001},"max":{"1min":17.407,"5min":17.407,"15min":17.407},"last":1.5249999999999999},{"interface":"front","average":{"1min":2.351,"5min":1.6279999999999999,"15min":1.53},"min":{"1min":0.71199999999999997,"5min":0.71199999999999997,"15min":0.71199999999999997},"max":{"1min":16.356000000000002,"5min":16.356000000000002,"15min":16.356000000000002},"last":1.7350000000000001}]},{"osd":7,"last update":"Wed Aug 9 13:51:21 2023","interfaces":[{"interface":"back","average":{"1min":2.117,"5min":1.665,"15min":1.591},"min":{"1min":0.74299999999999999,"5min":0.74299999999999999,"15min":0.74299999999999999},"max":{"1min":9.9239999999999995,"5min":9.9239999999999995,"15min":9.9239999999999995},"last":1.482},{"interface":"front","average":{"1min":2.2400000000000002,"5min":1.716,"15min":1.645},"min":{"1min":0.65600000000000003,"5min":0.65600000000000003,"15min":0.65600000000000003},"max":{"1min":10.789,"5min":10.789,"15min":10.789},"last":1.6919999999999999}]}]},{"osd":4,"up_from":13,"seq":55834575064,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32488,"kb_used_data":1372,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":94339352,"statfs":{"total":96636764160,"available":96603496448,"internally_reserved":0,"allocated":1404928,"data_stored":748010,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1781,"internal_metadata":31848715},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Aug 9 13:51:23 2023","interfaces":[{"interface":"back","average":{"1min":1.131,"5min":1.2150000000000001,"15min":1.2070000000000001},"min":{"1min":0.66600000000000004,"5min":0.66600000000000004,"15min":0.66600000000000004},"max":{"1min":1.768,"5min":2.524,"15min":2.524},"last":1.349},{"interface":"front","average":{"1min":1.0609999999999999,"5min":1.181,"15min":1.25},"min":{"1min":0.66800000000000004,"5min":0.62,"15min":0.62},"max":{"1min":1.667,"5min":1.7390000000000001,"15min":1.863},"last":1.4770000000000001}]},{"osd":1,"last update":"Wed Aug 9 13:51:23 2023","interfaces":[{"interface":"back","average":{"1min":1.1839999999999999,"5min":1.264,"15min":1.2969999999999999},"min":{"1min":0.79700000000000004,"5min":0.67800000000000005,"15min":0.67800000000000005},"max":{"1min":1.609,"5min":1.8480000000000001,"15min":1.8480000000000001},"last":1.2},{"interface":"front","average":{"1min":1.1319999999999999,"5min":1.232,"15min":1.2470000000000001},"min":{"1min":0.75,"5min":0.75,"15min":0.65400000000000003},"max":{"1min":1.669,"5min":1.9330000000000001,"15min":1.9330000000000001},"last":1.153}]},{"osd":2,"last update":"Wed Aug 9 13:51:23 2023","interfaces":[{"interface":"back","average":{"1min":1.081,"5min":1.2509999999999999,"15min":1.272},"min":{"1min":0.69999999999999996,"5min":0.621,"15min":0.621},"max":{"1min":1.496,"5min":2.6429999999999998,"15min":2.6429999999999998},"last":1.522},{"interface":"front","average":{"1min":1.25,"5min":1.3220000000000001,"15min":1.365},"min":{"1min":0.76000000000000001,"5min":0.76000000000000001,"15min":0.75900000000000001},"max":{"1min":2.0910000000000002,"5min":2.0910000000000002,"15min":2.0910000000000002},"last":1.667}]},{"osd":3,"last update":"Wed Aug 9 13:51:23 2023","interfaces":[{"interface":"back","average":{"1min":1.1599999999999999,"5min":1.2889999999999999,"15min":1.298},"min":{"1min":0.754,"5min":0.754,"15min":0.754},"max":{"1min":1.921,"5min":2.6099999999999999,"15min":2.6099999999999999},"last":0.84499999999999997},{"interface":"front","average":{"1min":1.2509999999999999,"5min":1.3740000000000001,"15min":1.3819999999999999},"min":{"1min":0.89100000000000001,"5min":0.88800000000000001,"15min":0.63100000000000001},"max":{"1min":1.7010000000000001,"5min":2.7629999999999999,"15min":2.7629999999999999},"last":1.694}]},{"osd":5,"last update":"Wed Aug 9 13:51:23 2023","interfaces":[{"interface":"back","average":{"1min":1.294,"5min":1.3140000000000001,"15min":1.327},"min":{"1min":0.67300000000000004,"5min":0.67300000000000004,"15min":0.63200000000000001},"max":{"1min":1.758,"5min":1.8,"15min":2.0099999999999998},"last":1.73},{"interface":"front","average":{"1min":1.361,"5min":1.337,"15min":1.3169999999999999},"min":{"1min":0.65600000000000003,"5min":0.65600000000000003,"15min":0.626},"max":{"1min":2.0369999999999999,"5min":2.7090000000000001,"15min":2.7090000000000001},"last":1.298}]},{"osd":6,"last update":"Wed Aug 9 13:51:23 2023","interfaces":[{"interface":"back","average":{"1min":1.409,"5min":1.341,"15min":1.3340000000000001},"min":{"1min":0.90600000000000003,"5min":0.74399999999999999,"15min":0.59099999999999997},"max":{"1min":1.96,"5min":2.8559999999999999,"15min":2.8559999999999999},"last":1.758},{"interface":"front","average":{"1min":1.496,"5min":1.514,"15min":1.5029999999999999},"min":{"1min":0.878,"5min":0.878,"15min":0.76400000000000001},"max":{"1min":2.1890000000000001,"5min":2.2719999999999998,"15min":2.2719999999999998},"last":1.9410000000000001}]},{"osd":7,"last update":"Wed Aug 9 13:51:23 2023","interfaces":[{"interface":"back","average":{"1min":1.3859999999999999,"5min":1.468,"15min":1.4690000000000001},"min":{"1min":0.53100000000000003,"5min":0.53100000000000003,"15min":0.53100000000000003},"max":{"1min":1.9530000000000001,"5min":2.8050000000000002,"15min":2.8050000000000002},"last":1.5940000000000001},{"interface":"front","average":{"1min":1.5289999999999999,"5min":1.52,"15min":1.5129999999999999},"min":{"1min":0.78700000000000003,"5min":0.78700000000000003,"15min":0.72899999999999998},"max":{"1min":2.1419999999999999,"5min":3.415,"15min":3.415},"last":1.976}]}]},{"osd":3,"up_from":13,"seq":55834575065,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":36572,"kb_used_data":1360,"kb_used_omap":1,"kb_used_meta":35198,"kb_avail":94335268,"statfs":{"total":96636764160,"available":96599314432,"internally_reserved":0,"allocated":1392640,"data_stored":741018,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1781,"internal_metadata":36043019},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Aug 9 13:51:26 2023","interfaces":[{"interface":"back","average":{"1min":0.76900000000000002,"5min":0.85299999999999998,"15min":0.90500000000000003},"min":{"1min":0.496,"5min":0.27200000000000002,"15min":0.27200000000000002},"max":{"1min":1.5580000000000001,"5min":1.8500000000000001,"15min":1.968},"last":0.99099999999999999},{"interface":"front","average":{"1min":0.83499999999999996,"5min":1.0349999999999999,"15min":1.097},"min":{"1min":0.36299999999999999,"5min":0.36299999999999999,"15min":0.36299999999999999},"max":{"1min":1.4059999999999999,"5min":1.6910000000000001,"15min":2.0590000000000002},"last":1.3200000000000001}]},{"osd":1,"last update":"Wed Aug 9 13:51:26 2023","interfaces":[{"interface":"back","average":{"1min":0.82799999999999996,"5min":0.95399999999999996,"15min":1.0089999999999999},"min":{"1min":0.51500000000000001,"5min":0.496,"15min":0.438},"max":{"1min":1.27,"5min":1.6519999999999999,"15min":2.5649999999999999},"last":1.038},{"interface":"front","average":{"1min":0.97899999999999998,"5min":1.081,"15min":1.161},"min":{"1min":0.60499999999999998,"5min":0.51800000000000002,"15min":0.498},"max":{"1min":1.6779999999999999,"5min":1.6839999999999999,"15min":2.6760000000000002},"last":1.393}]},{"osd":2,"last update":"Wed Aug 9 13:51:26 2023","interfaces":[{"interface":"back","average":{"1min":0.995,"5min":1.115,"15min":1.194},"min":{"1min":0.42999999999999999,"5min":0.42999999999999999,"15min":0.42999999999999999},"max":{"1min":1.6879999999999999,"5min":1.913,"15min":1.923},"last":1.494},{"interface":"front","average":{"1min":1.103,"5min":1.2290000000000001,"15min":1.3069999999999999},"min":{"1min":0.61299999999999999,"5min":0.52300000000000002,"15min":0.52300000000000002},"max":{"1min":1.7290000000000001,"5min":1.9399999999999999,"15min":2.0259999999999998},"last":1.254}]},{"osd":4,"last update":"Wed Aug 9 13:51:26 2023","interfaces":[{"interface":"back","average":{"1min":1.1000000000000001,"5min":1.214,"15min":1.274},"min":{"1min":0.54900000000000004,"5min":0.54900000000000004,"15min":0.54900000000000004},"max":{"1min":1.732,"5min":1.8160000000000001,"15min":1.8160000000000001},"last":1.7070000000000001},{"interface":"front","average":{"1min":1.2769999999999999,"5min":1.3440000000000001,"15min":1.3959999999999999},"min":{"1min":0.76100000000000001,"5min":0.628,"15min":0.628},"max":{"1min":2.782,"5min":2.782,"15min":2.782},"last":1.7629999999999999}]},{"osd":5,"last update":"Wed Aug 9 13:51:26 2023","interfaces":[{"interface":"back","average":{"1min":1.266,"5min":1.4179999999999999,"15min":1.47},"min":{"1min":0.69899999999999995,"5min":0.67600000000000005,"15min":0.67200000000000004},"max":{"1min":1.833,"5min":2.254,"15min":2.254},"last":2.012},{"interface":"front","average":{"1min":1.258,"5min":1.431,"15min":1.484},"min":{"1min":0.66000000000000003,"5min":0.65900000000000003,"15min":0.65900000000000003},"max":{"1min":1.919,"5min":2.3069999999999999,"15min":2.3069999999999999},"last":1.6060000000000001}]},{"osd":6,"last update":"Wed Aug 9 13:51:26 2023","interfaces":[{"interface":"back","average":{"1min":1.381,"5min":1.419,"15min":1.4630000000000001},"min":{"1min":0.78200000000000003,"5min":0.68200000000000005,"15min":0.68200000000000005},"max":{"1min":2.8340000000000001,"5min":2.8340000000000001,"15min":2.8340000000000001},"last":1.5620000000000001},{"interface":"front","average":{"1min":1.389,"5min":1.5109999999999999,"15min":1.5580000000000001},"min":{"1min":0.64100000000000001,"5min":0.64100000000000001,"15min":0.64100000000000001},"max":{"1min":2.9060000000000001,"5min":2.9060000000000001,"15min":2.9060000000000001},"last":2.1480000000000001}]},{"osd":7,"last update":"Wed Aug 9 13:51:26 2023","interfaces":[{"interface":"back","average":{"1min":1.3580000000000001,"5min":1.5549999999999999,"15min":1.6100000000000001},"min":{"1min":0.77800000000000002,"5min":0.69399999999999995,"15min":0.69399999999999995},"max":{"1min":2.0419999999999998,"5min":2.46,"15min":2.46},"last":1.966},{"interface":"front","average":{"1min":1.403,"5min":1.5049999999999999,"15min":1.5620000000000001},"min":{"1min":0.93799999999999994,"5min":0.64200000000000002,"15min":0.64200000000000002},"max":{"1min":2.7200000000000002,"5min":2.7200000000000002,"15min":2.7200000000000002},"last":1.804}]}]},{"osd":2,"up_from":13,"seq":55834575064,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":36572,"kb_used_data":1360,"kb_used_omap":1,"kb_used_meta":35198,"kb_avail":94335268,"statfs":{"total":96636764160,"available":96599314432,"internally_reserved":0,"allocated":1392640,"data_stored":741018,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1781,"internal_metadata":36043019},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Aug 9 13:51:19 2023","interfaces":[{"interface":"back","average":{"1min":0.82699999999999996,"5min":0.91400000000000003,"15min":0.95599999999999996},"min":{"1min":0.47199999999999998,"5min":0.36499999999999999,"15min":0.36499999999999999},"max":{"1min":1.484,"5min":1.5940000000000001,"15min":1.8380000000000001},"last":0.749},{"interface":"front","average":{"1min":0.90200000000000002,"5min":0.96699999999999997,"15min":1.0009999999999999},"min":{"1min":0.38200000000000001,"5min":0.374,"15min":0.374},"max":{"1min":1.7,"5min":2.3929999999999998,"15min":2.3929999999999998},"last":0.995}]},{"osd":1,"last update":"Wed Aug 9 13:51:19 2023","interfaces":[{"interface":"back","average":{"1min":1.038,"5min":1.143,"15min":1.1839999999999999},"min":{"1min":0.39400000000000002,"5min":0.39400000000000002,"15min":0.39400000000000002},"max":{"1min":1.615,"5min":1.77,"15min":1.8420000000000001},"last":1.149},{"interface":"front","average":{"1min":0.98399999999999999,"5min":1.171,"15min":1.3220000000000001},"min":{"1min":0.435,"5min":0.435,"15min":0.435},"max":{"1min":1.659,"5min":2.4870000000000001,"15min":2.4870000000000001},"last":1.1879999999999999}]},{"osd":3,"last update":"Wed Aug 9 13:51:19 2023","interfaces":[{"interface":"back","average":{"1min":0.93100000000000005,"5min":0.97999999999999998,"15min":1.0509999999999999},"min":{"1min":0.55300000000000005,"5min":0.38900000000000001,"15min":0.38900000000000001},"max":{"1min":1.417,"5min":1.708,"15min":2.0409999999999999},"last":0.93400000000000005},{"interface":"front","average":{"1min":1.0820000000000001,"5min":1.2549999999999999,"15min":1.3080000000000001},"min":{"1min":0.52900000000000003,"5min":0.46200000000000002,"15min":0.46200000000000002},"max":{"1min":1.6279999999999999,"5min":8.8130000000000006,"15min":8.8130000000000006},"last":1.304}]},{"osd":4,"last update":"Wed Aug 9 13:51:19 2023","interfaces":[{"interface":"back","average":{"1min":1.335,"5min":1.3740000000000001,"15min":1.452},"min":{"1min":0.81000000000000005,"5min":0.628,"15min":0.628},"max":{"1min":1.8340000000000001,"5min":2.089,"15min":2.089},"last":1.5109999999999999},{"interface":"front","average":{"1min":1.2969999999999999,"5min":1.3360000000000001,"15min":1.3939999999999999},"min":{"1min":0.75900000000000001,"5min":0.68600000000000005,"15min":0.68600000000000005},"max":{"1min":1.7509999999999999,"5min":2.472,"15min":2.472},"last":1.5469999999999999}]},{"osd":5,"last update":"Wed Aug 9 13:51:19 2023","interfaces":[{"interface":"back","average":{"1min":1.361,"5min":1.4059999999999999,"15min":1.4590000000000001},"min":{"1min":0.86399999999999999,"5min":0.59999999999999998,"15min":0.59999999999999998},"max":{"1min":2.0640000000000001,"5min":2.5840000000000001,"15min":2.5840000000000001},"last":1.383},{"interface":"front","average":{"1min":1.2829999999999999,"5min":1.3260000000000001,"15min":1.351},"min":{"1min":0.89000000000000001,"5min":0.56799999999999995,"15min":0.56799999999999995},"max":{"1min":1.78,"5min":3.5659999999999998,"15min":3.5659999999999998},"last":1.2370000000000001}]},{"osd":6,"last update":"Wed Aug 9 13:51:19 2023","interfaces":[{"interface":"back","average":{"1min":1.3009999999999999,"5min":1.3879999999999999,"15min":1.429},"min":{"1min":0.82199999999999995,"5min":0.73799999999999999,"15min":0.73799999999999999},"max":{"1min":2.1200000000000001,"5min":3.5099999999999998,"15min":3.5099999999999998},"last":1.3540000000000001},{"interface":"front","average":{"1min":1.4099999999999999,"5min":1.458,"15min":1.54},"min":{"1min":0.84899999999999998,"5min":0.66300000000000003,"15min":0.66300000000000003},"max":{"1min":2.04,"5min":2.3769999999999998,"15min":2.3769999999999998},"last":1.4279999999999999}]},{"osd":7,"last update":"Wed Aug 9 13:51:19 2023","interfaces":[{"interface":"back","average":{"1min":1.3620000000000001,"5min":1.452,"15min":1.54},"min":{"1min":0.85099999999999998,"5min":0.64600000000000002,"15min":0.64600000000000002},"max":{"1min":1.865,"5min":2.5249999999999999,"15min":2.5249999999999999},"last":1.6200000000000001},{"interface":"front","average":{"1min":1.3939999999999999,"5min":1.476,"15min":1.514},"min":{"1min":0.94699999999999995,"5min":0.70499999999999996,"15min":0.70499999999999996},"max":{"1min":1.891,"5min":3.609,"15min":3.609},"last":1.4139999999999999}]}]},{"osd":1,"up_from":13,"seq":55834575064,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":47008,"kb_used_data":1940,"kb_used_omap":1,"kb_used_meta":45054,"kb_avail":94324832,"statfs":{"total":96636764160,"available":96588627968,"internally_reserved":0,"allocated":1986560,"data_stored":1331386,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1781,"internal_metadata":46135563},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Aug 9 13:51:34 2023","interfaces":[{"interface":"back","average":{"1min":0.77600000000000002,"5min":0.90000000000000002,"15min":0.93799999999999994},"min":{"1min":0.45800000000000002,"5min":0.45800000000000002,"15min":0.45800000000000002},"max":{"1min":1.1910000000000001,"5min":1.7110000000000001,"15min":2.1080000000000001},"last":0.57599999999999996},{"interface":"front","average":{"1min":0.83299999999999996,"5min":1.0129999999999999,"15min":1.095},"min":{"1min":0.60599999999999998,"5min":0.432,"15min":0.432},"max":{"1min":1.6519999999999999,"5min":2.468,"15min":2.468},"last":0.40699999999999997}]},{"osd":2,"last update":"Wed Aug 9 13:51:34 2023","interfaces":[{"interface":"back","average":{"1min":0.83399999999999996,"5min":0.93799999999999994,"15min":0.99299999999999999},"min":{"1min":0.57499999999999996,"5min":0.41099999999999998,"15min":0.41099999999999998},"max":{"1min":1.2130000000000001,"5min":1.907,"15min":1.907},"last":0.52200000000000002},{"interface":"front","average":{"1min":0.91100000000000003,"5min":1.0920000000000001,"15min":1.1859999999999999},"min":{"1min":0.59699999999999998,"5min":0.54400000000000004,"15min":0.54400000000000004},"max":{"1min":1.45,"5min":2.1280000000000001,"15min":2.145},"last":0.50700000000000001}]},{"osd":3,"last update":"Wed Aug 9 13:51:34 2023","interfaces":[{"interface":"back","average":{"1min":0.97199999999999998,"5min":1.1890000000000001,"15min":1.3220000000000001},"min":{"1min":0.60799999999999998,"5min":0.54400000000000004,"15min":0.54400000000000004},"max":{"1min":1.466,"5min":2.0270000000000001,"15min":2.3210000000000002},"last":0.54600000000000004},{"interface":"front","average":{"1min":1.042,"5min":1.163,"15min":1.25},"min":{"1min":0.625,"5min":0.49299999999999999,"15min":0.45300000000000001},"max":{"1min":1.6539999999999999,"5min":2.3540000000000001,"15min":2.7669999999999999},"last":0.48299999999999998}]},{"osd":4,"last update":"Wed Aug 9 13:51:34 2023","interfaces":[{"interface":"back","average":{"1min":1.1819999999999999,"5min":1.337,"15min":1.4059999999999999},"min":{"1min":0.79800000000000004,"5min":0.79800000000000004,"15min":0.58099999999999996},"max":{"1min":1.9259999999999999,"5min":1.97,"15min":2.5},"last":0.91100000000000003},{"interface":"front","average":{"1min":1.157,"5min":1.27,"15min":1.329},"min":{"1min":0.76600000000000001,"5min":0.70099999999999996,"15min":0.70099999999999996},"max":{"1min":1.895,"5min":1.895,"15min":1.901},"last":0.71099999999999997}]},{"osd":5,"last update":"Wed Aug 9 13:51:34 2023","interfaces":[{"interface":"back","average":{"1min":1.2150000000000001,"5min":1.29,"15min":1.3680000000000001},"min":{"1min":0.76800000000000002,"5min":0.76800000000000002,"15min":0.76800000000000002},"max":{"1min":1.73,"5min":2.294,"15min":2.294},"last":0.75900000000000001},{"interface":"front","average":{"1min":1.2949999999999999,"5min":1.3999999999999999,"15min":1.512},"min":{"1min":0.89200000000000002,"5min":0.89200000000000002,"15min":0.89200000000000002},"max":{"1min":1.6859999999999999,"5min":2.1389999999999998,"15min":2.4649999999999999},"last":0.78800000000000003}]},{"osd":6,"last update":"Wed Aug 9 13:51:34 2023","interfaces":[{"interface":"back","average":{"1min":1.325,"5min":1.4339999999999999,"15min":1.5149999999999999},"min":{"1min":0.96599999999999997,"5min":0.77500000000000002,"15min":0.67400000000000004},"max":{"1min":1.5780000000000001,"5min":2.2080000000000002,"15min":2.5270000000000001},"last":0.97599999999999998},{"interface":"front","average":{"1min":1.28,"5min":1.3979999999999999,"15min":1.4990000000000001},"min":{"1min":0.73599999999999999,"5min":0.73599999999999999,"15min":0.73599999999999999},"max":{"1min":1.591,"5min":2.02,"15min":2.1040000000000001},"last":0.80600000000000005}]},{"osd":7,"last update":"Wed Aug 9 13:51:34 2023","interfaces":[{"interface":"back","average":{"1min":1.3240000000000001,"5min":1.4930000000000001,"15min":1.615},"min":{"1min":0.93700000000000006,"5min":0.86899999999999999,"15min":0.86899999999999999},"max":{"1min":1.776,"5min":2.4169999999999998,"15min":2.5790000000000002},"last":0.81899999999999995},{"interface":"front","average":{"1min":1.3420000000000001,"5min":1.488,"15min":1.5980000000000001},"min":{"1min":0.96899999999999997,"5min":0.91300000000000003,"15min":0.89500000000000002},"max":{"1min":1.8320000000000001,"5min":2.1040000000000001,"15min":2.6509999999999998},"last":1.083}]}]},{"osd":0,"up_from":13,"seq":55834575064,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":40664,"kb_used_data":1356,"kb_used_omap":1,"kb_used_meta":39294,"kb_avail":94331176,"statfs":{"total":96636764160,"available":96595124224,"internally_reserved":0,"allocated":1388544,"data_stored":739657,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1781,"internal_metadata":40237323},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Wed Aug 9 13:51:27 2023","interfaces":[{"interface":"back","average":{"1min":0.95899999999999996,"5min":0.91300000000000003,"15min":0.92100000000000004},"min":{"1min":0.41199999999999998,"5min":0.32500000000000001,"15min":0.32500000000000001},"max":{"1min":1.4990000000000001,"5min":1.756,"15min":1.756},"last":1.528},{"interface":"front","average":{"1min":1.002,"5min":1.077,"15min":1.111},"min":{"1min":0.39200000000000002,"5min":0.39200000000000002,"15min":0.39200000000000002},"max":{"1min":1.546,"5min":1.9019999999999999,"15min":1.9019999999999999},"last":0.81999999999999995}]},{"osd":2,"last update":"Wed Aug 9 13:51:27 2023","interfaces":[{"interface":"back","average":{"1min":1,"5min":1.0680000000000001,"15min":1.1339999999999999},"min":{"1min":0.58399999999999996,"5min":0.51400000000000001,"15min":0.503},"max":{"1min":1.5089999999999999,"5min":1.776,"15min":1.889},"last":1.1639999999999999},{"interface":"front","average":{"1min":0.98799999999999999,"5min":1.1020000000000001,"15min":1.1739999999999999},"min":{"1min":0.439,"5min":0.439,"15min":0.439},"max":{"1min":1.6259999999999999,"5min":2.0129999999999999,"15min":2.0129999999999999},"last":1.704}]},{"osd":3,"last update":"Wed Aug 9 13:51:27 2023","interfaces":[{"interface":"back","average":{"1min":1,"5min":1.0349999999999999,"15min":1.101},"min":{"1min":0.50900000000000001,"5min":0.46300000000000002,"15min":0.46300000000000002},"max":{"1min":1.5189999999999999,"5min":1.823,"15min":1.8919999999999999},"last":0.67800000000000005},{"interface":"front","average":{"1min":1.2390000000000001,"5min":1.252,"15min":1.3140000000000001},"min":{"1min":0.63400000000000001,"5min":0.54700000000000004,"15min":0.54700000000000004},"max":{"1min":2.2050000000000001,"5min":2.347,"15min":2.4649999999999999},"last":1.022}]},{"osd":4,"last update":"Wed Aug 9 13:51:27 2023","interfaces":[{"interface":"back","average":{"1min":1.2509999999999999,"5min":1.2949999999999999,"15min":1.343},"min":{"1min":0.73899999999999999,"5min":0.73899999999999999,"15min":0.73899999999999999},"max":{"1min":1.581,"5min":2.2309999999999999,"15min":2.2309999999999999},"last":1.2150000000000001},{"interface":"front","average":{"1min":1.409,"5min":1.417,"15min":1.456},"min":{"1min":0.86399999999999999,"5min":0.77600000000000002,"15min":0.77600000000000002},"max":{"1min":2.3540000000000001,"5min":2.516,"15min":2.5390000000000001},"last":1.395}]},{"osd":5,"last update":"Wed Aug 9 13:51:27 2023","interfaces":[{"interface":"back","average":{"1min":1.3859999999999999,"5min":1.4019999999999999,"15min":1.462},"min":{"1min":0.875,"5min":0.66300000000000003,"15min":0.66300000000000003},"max":{"1min":1.764,"5min":2.6669999999999998,"15min":2.6669999999999998},"last":1.1020000000000001},{"interface":"front","average":{"1min":1.333,"5min":1.3859999999999999,"15min":1.4470000000000001},"min":{"1min":0.88400000000000001,"5min":0.629,"15min":0.629},"max":{"1min":1.645,"5min":2.1499999999999999,"15min":2.1499999999999999},"last":0.96099999999999997}]},{"osd":6,"last update":"Wed Aug 9 13:51:27 2023","interfaces":[{"interface":"back","average":{"1min":1.4870000000000001,"5min":1.5089999999999999,"15min":1.548},"min":{"1min":1.014,"5min":0.73599999999999999,"15min":0.73599999999999999},"max":{"1min":2.4670000000000001,"5min":2.4670000000000001,"15min":2.7250000000000001},"last":1.6319999999999999},{"interface":"front","average":{"1min":1.486,"5min":1.526,"15min":1.5569999999999999},"min":{"1min":0.95999999999999996,"5min":0.86699999999999999,"15min":0.86699999999999999},"max":{"1min":1.988,"5min":2.399,"15min":2.399},"last":1.573}]},{"osd":7,"last update":"Wed Aug 9 13:51:27 2023","interfaces":[{"interface":"back","average":{"1min":1.552,"5min":1.5720000000000001,"15min":1.621},"min":{"1min":0.89200000000000002,"5min":0.72299999999999998,"15min":0.72299999999999998},"max":{"1min":2.0880000000000001,"5min":2.3900000000000001,"15min":2.3900000000000001},"last":1.2490000000000001},{"interface":"front","average":{"1min":1.4179999999999999,"5min":1.484,"15min":1.54},"min":{"1min":0.84099999999999997,"5min":0.76700000000000002,"15min":0.76700000000000002},"max":{"1min":1.7969999999999999,"5min":2.274,"15min":2.274},"last":1.4890000000000001}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-08-09T13:52:05.791 INFO:tasks.cephfs.filesystem.ceph_manager:PG 15.3f is not active+clean 2023-08-09T13:52:05.792 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '15.3f', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 114, 'state': 'creating+peering', 'last_fresh': '2023-08-09T13:51:59.173066+0000', 'last_change': '2023-08-09T13:51:59.173066+0000', 'last_active': '2023-08-09T13:51:59.130555+0000', 'last_peered': '2023-08-09T13:51:59.130555+0000', 'last_clean': '2023-08-09T13:51:59.130555+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-08-09T13:51:59.173066+0000', 'last_undegraded': '2023-08-09T13:51:59.173066+0000', 'last_fullsized': '2023-08-09T13:51:59.173066+0000', 'mapping_epoch': 114, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 114, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-08-09T13:51:59.130555+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-08-09T13:51:59.130555+0000', 'last_clean_scrub_stamp': '2023-08-09T13:51:59.130555+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 5], 'acting': [1, 5], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 1, 'purged_snaps': []} 2023-08-09T13:52:05.793 INFO:tasks.cephfs.filesystem.ceph_manager:PG 15.3e is not active+clean 2023-08-09T13:52:05.793 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '15.3e', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 114, 'state': 'creating+peering', 'last_fresh': '2023-08-09T13:51:59.161823+0000', 'last_change': '2023-08-09T13:51:59.161823+0000', 'last_active': '2023-08-09T13:51:59.130555+0000', 'last_peered': '2023-08-09T13:51:59.130555+0000', 'last_clean': '2023-08-09T13:51:59.130555+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-08-09T13:51:59.161823+0000', 'last_undegraded': '2023-08-09T13:51:59.161823+0000', 'last_fullsized': '2023-08-09T13:51:59.161823+0000', 'mapping_epoch': 114, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 114, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-08-09T13:51:59.130555+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-08-09T13:51:59.130555+0000', 'last_clean_scrub_stamp': '2023-08-09T13:51:59.130555+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 5], 'acting': [0, 5], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2023-08-09T13:52:05.794 INFO:tasks.cephfs.filesystem.ceph_manager:PG 15.3d is not active+clean 2023-08-09T13:52:05.794 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '15.3d', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 114, 'state': 'creating+peering', 'last_fresh': '2023-08-09T13:51:59.169470+0000', 'last_change': '2023-08-09T13:51:59.169470+0000', 'last_active': '2023-08-09T13:51:59.130555+0000', 'last_peered': '2023-08-09T13:51:59.130555+0000', 'last_clean': '2023-08-09T13:51:59.130555+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-08-09T13:51:59.169470+0000', 'last_undegraded': '2023-08-09T13:51:59.169470+0000', 'last_fullsized': '2023-08-09T13:51:59.169470+0000', 'mapping_epoch': 114, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 114, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-08-09T13:51:59.130555+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-08-09T13:51:59.130555+0000', 'last_clean_scrub_stamp': '2023-08-09T13:51:59.130555+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 5], 'acting': [1, 5], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 1, 'purged_snaps': []} 2023-08-09T13:52:05.795 INFO:tasks.cephfs.filesystem.ceph_manager:PG 15.3c is not active+clean 2023-08-09T13:52:05.796 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '15.3c', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 114, 'state': 'creating+peering', 'last_fresh': '2023-08-09T13:51:59.150559+0000', 'last_change': '2023-08-09T13:51:59.150559+0000', 'last_active': '2023-08-09T13:51:59.130555+0000', 'last_peered': '2023-08-09T13:51:59.130555+0000', 'last_clean': '2023-08-09T13:51:59.130555+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-08-09T13:51:59.150559+0000', 'last_undegraded': '2023-08-09T13:51:59.150559+0000', 'last_fullsized': '2023-08-09T13:51:59.150559+0000', 'mapping_epoch': 114, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 114, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-08-09T13:51:59.130555+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-08-09T13:51:59.130555+0000', 'last_clean_scrub_stamp': '2023-08-09T13:51:59.130555+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 2], 'acting': [3, 2], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2023-08-09T13:52:05.797 INFO:tasks.cephfs.filesystem.ceph_manager:PG 15.3b is not active+clean 2023-08-09T13:52:05.797 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '15.3b', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 114, 'state': 'creating+peering', 'last_fresh': '2023-08-09T13:51:59.155619+0000', 'last_change': '2023-08-09T13:51:59.155619+0000', 'last_active': '2023-08-09T13:51:59.130555+0000', 'last_peered': '2023-08-09T13:51:59.130555+0000', 'last_clean': '2023-08-09T13:51:59.130555+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-08-09T13:51:59.155619+0000', 'last_undegraded': '2023-08-09T13:51:59.155619+0000', 'last_fullsized': '2023-08-09T13:51:59.155619+0000', 'mapping_epoch': 114, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 114, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-08-09T13:51:59.130555+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-08-09T13:51:59.130555+0000', 'last_clean_scrub_stamp': '2023-08-09T13:51:59.130555+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [6, 3], 'acting': [6, 3], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 6, 'acting_primary': 6, 'purged_snaps': []} 2023-08-09T13:52:05.797 INFO:tasks.cephfs.filesystem.ceph_manager:PG 15.3a is not active+clean 2023-08-09T13:52:05.798 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '15.3a', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 114, 'state': 'creating+peering', 'last_fresh': '2023-08-09T13:51:59.155005+0000', 'last_change': '2023-08-09T13:51:59.155005+0000', 'last_active': '2023-08-09T13:51:59.130555+0000', 'last_peered': '2023-08-09T13:51:59.130555+0000', 'last_clean': '2023-08-09T13:51:59.130555+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-08-09T13:51:59.155005+0000', 'last_undegraded': '2023-08-09T13:51:59.155005+0000', 'last_fullsized': '2023-08-09T13:51:59.155005+0000', 'mapping_epoch': 114, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 114, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-08-09T13:51:59.130555+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-08-09T13:51:59.130555+0000', 'last_clean_scrub_stamp': '2023-08-09T13:51:59.130555+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [7, 5], 'acting': [7, 5], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 7, 'acting_primary': 7, 'purged_snaps': []} 2023-08-09T13:52:05.798 INFO:tasks.cephfs.filesystem.ceph_manager:PG 15.39 is not active+clean 2023-08-09T13:52:05.798 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '15.39', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 114, 'state': 'creating+peering', 'last_fresh': '2023-08-09T13:51:59.154799+0000', 'last_change': '2023-08-09T13:51:59.154799+0000', 'last_active': '2023-08-09T13:51:59.130555+0000', 'last_peered': '2023-08-09T13:51:59.130555+0000', 'last_clean': '2023-08-09T13:51:59.130555+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-08-09T13:51:59.154799+0000', 'last_undegraded': '2023-08-09T13:51:59.154799+0000', 'last_fullsized': '2023-08-09T13:51:59.154799+0000', 'mapping_epoch': 114, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 114, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-08-09T13:51:59.130555+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-08-09T13:51:59.130555+0000', 'last_clean_scrub_stamp': '2023-08-09T13:51:59.130555+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [7, 5], 'acting': [7, 5], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 7, 'acting_primary': 7, 'purged_snaps': []} 2023-08-09T13:52:05.799 INFO:tasks.cephfs.filesystem.ceph_manager:PG 15.38 is not active+clean 2023-08-09T13:52:05.799 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '15.38', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 114, 'state': 'creating+peering', 'last_fresh': '2023-08-09T13:51:59.161114+0000', 'last_change': '2023-08-09T13:51:59.161114+0000', 'last_active': '2023-08-09T13:51:59.130555+0000', 'last_peered': '2023-08-09T13:51:59.130555+0000', 'last_clean': '2023-08-09T13:51:59.130555+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-08-09T13:51:59.161114+0000', 'last_undegraded': '2023-08-09T13:51:59.161114+0000', 'last_fullsized': '2023-08-09T13:51:59.161114+0000', 'mapping_epoch': 114, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 114, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-08-09T13:51:59.130555+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-08-09T13:51:59.130555+0000', 'last_clean_scrub_stamp': '2023-08-09T13:51:59.130555+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 7], 'acting': [0, 7], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2023-08-09T13:52:05.800 INFO:tasks.cephfs.filesystem.ceph_manager:PG 15.37 is not active+clean 2023-08-09T13:52:05.800 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '15.37', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 114, 'state': 'creating+peering', 'last_fresh': '2023-08-09T13:51:59.170489+0000', 'last_change': '2023-08-09T13:51:59.170489+0000', 'last_active': '2023-08-09T13:51:59.130555+0000', 'last_peered': '2023-08-09T13:51:59.130555+0000', 'last_clean': '2023-08-09T13:51:59.130555+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-08-09T13:51:59.170489+0000', 'last_undegraded': '2023-08-09T13:51:59.170489+0000', 'last_fullsized': '2023-08-09T13:51:59.170489+0000', 'mapping_epoch': 114, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 114, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-08-09T13:51:59.130555+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-08-09T13:51:59.130555+0000', 'last_clean_scrub_stamp': '2023-08-09T13:51:59.130555+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [4, 3], 'acting': [4, 3], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 4, 'acting_primary': 4, 'purged_snaps': []} 2023-08-09T13:52:05.800 INFO:tasks.cephfs.filesystem.ceph_manager:PG 15.36 is not active+clean 2023-08-09T13:52:05.801 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '15.36', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 114, 'state': 'creating+peering', 'last_fresh': '2023-08-09T13:51:59.157362+0000', 'last_change': '2023-08-09T13:51:59.157362+0000', 'last_active': '2023-08-09T13:51:59.130555+0000', 'last_peered': '2023-08-09T13:51:59.130555+0000', 'last_clean': '2023-08-09T13:51:59.130555+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-08-09T13:51:59.157362+0000', 'last_undegraded': '2023-08-09T13:51:59.157362+0000', 'last_fullsized': '2023-08-09T13:51:59.157362+0000', 'mapping_epoch': 114, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 114, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-08-09T13:51:59.130555+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-08-09T13:51:59.130555+0000', 'last_clean_scrub_stamp': '2023-08-09T13:51:59.130555+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [6, 1], 'acting': [6, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 6, 'acting_primary': 6, 'purged_snaps': []} 2023-08-09T13:52:05.801 INFO:tasks.cephfs.filesystem.ceph_manager:PG 15.35 is not active+clean 2023-08-09T13:52:05.801 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '15.35', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 114, 'state': 'creating+peering', 'last_fresh': '2023-08-09T13:51:59.158636+0000', 'last_change': '2023-08-09T13:51:59.158636+0000', 'last_active': '2023-08-09T13:51:59.130555+0000', 'last_peered': '2023-08-09T13:51:59.130555+0000', 'last_clean': '2023-08-09T13:51:59.130555+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-08-09T13:51:59.158636+0000', 'last_undegraded': '2023-08-09T13:51:59.158636+0000', 'last_fullsized': '2023-08-09T13:51:59.158636+0000', 'mapping_epoch': 114, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 114, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-08-09T13:51:59.130555+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-08-09T13:51:59.130555+0000', 'last_clean_scrub_stamp': '2023-08-09T13:51:59.130555+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [7, 4], 'acting': [7, 4], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 7, 'acting_primary': 7, 'purged_snaps': []} 2023-08-09T13:52:05.802 INFO:tasks.cephfs.filesystem.ceph_manager:PG 15.34 is not active+clean 2023-08-09T13:52:05.802 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '15.34', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 114, 'state': 'creating+peering', 'last_fresh': '2023-08-09T13:51:59.175941+0000', 'last_change': '2023-08-09T13:51:59.175941+0000', 'last_active': '2023-08-09T13:51:59.130555+0000', 'last_peered': '2023-08-09T13:51:59.130555+0000', 'last_clean': '2023-08-09T13:51:59.130555+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-08-09T13:51:59.175941+0000', 'last_undegraded': '2023-08-09T13:51:59.175941+0000', 'last_fullsized': '2023-08-09T13:51:59.175941+0000', 'mapping_epoch': 114, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 114, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-08-09T13:51:59.130555+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-08-09T13:51:59.130555+0000', 'last_clean_scrub_stamp': '2023-08-09T13:51:59.130555+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [4, 1], 'acting': [4, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 4, 'acting_primary': 4, 'purged_snaps': []} 2023-08-09T13:52:05.803 INFO:tasks.cephfs.filesystem.ceph_manager:PG 15.33 is not active+clean 2023-08-09T13:52:05.803 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '15.33', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 114, 'state': 'creating+peering', 'last_fresh': '2023-08-09T13:51:59.150804+0000', 'last_change': '2023-08-09T13:51:59.150804+0000', 'last_active': '2023-08-09T13:51:59.130555+0000', 'last_peered': '2023-08-09T13:51:59.130555+0000', 'last_clean': '2023-08-09T13:51:59.130555+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-08-09T13:51:59.150804+0000', 'last_undegraded': '2023-08-09T13:51:59.150804+0000', 'last_fullsized': '2023-08-09T13:51:59.150804+0000', 'mapping_epoch': 114, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 114, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-08-09T13:51:59.130555+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-08-09T13:51:59.130555+0000', 'last_clean_scrub_stamp': '2023-08-09T13:51:59.130555+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 4], 'acting': [3, 4], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2023-08-09T13:52:05.803 INFO:tasks.cephfs.filesystem.ceph_manager:PG 15.32 is not active+clean 2023-08-09T13:52:05.804 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '15.32', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 114, 'state': 'creating+peering', 'last_fresh': '2023-08-09T13:51:59.165080+0000', 'last_change': '2023-08-09T13:51:59.165080+0000', 'last_active': '2023-08-09T13:51:59.130555+0000', 'last_peered': '2023-08-09T13:51:59.130555+0000', 'last_clean': '2023-08-09T13:51:59.130555+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-08-09T13:51:59.165080+0000', 'last_undegraded': '2023-08-09T13:51:59.165080+0000', 'last_fullsized': '2023-08-09T13:51:59.165080+0000', 'mapping_epoch': 114, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 114, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-08-09T13:51:59.130555+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-08-09T13:51:59.130555+0000', 'last_clean_scrub_stamp': '2023-08-09T13:51:59.130555+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 6], 'acting': [2, 6], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2023-08-09T13:52:05.804 INFO:tasks.cephfs.filesystem.ceph_manager:PG 15.31 is not active+clean 2023-08-09T13:52:05.805 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '15.31', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 114, 'state': 'creating+peering', 'last_fresh': '2023-08-09T13:51:59.150713+0000', 'last_change': '2023-08-09T13:51:59.150713+0000', 'last_active': '2023-08-09T13:51:59.130555+0000', 'last_peered': '2023-08-09T13:51:59.130555+0000', 'last_clean': '2023-08-09T13:51:59.130555+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-08-09T13:51:59.150713+0000', 'last_undegraded': '2023-08-09T13:51:59.150713+0000', 'last_fullsized': '2023-08-09T13:51:59.150713+0000', 'mapping_epoch': 114, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 114, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-08-09T13:51:59.130555+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-08-09T13:51:59.130555+0000', 'last_clean_scrub_stamp': '2023-08-09T13:51:59.130555+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 1], 'acting': [3, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2023-08-09T13:52:05.805 INFO:tasks.cephfs.filesystem.ceph_manager:PG 15.30 is not active+clean 2023-08-09T13:52:05.806 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '15.30', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 114, 'state': 'creating+peering', 'last_fresh': '2023-08-09T13:51:59.157225+0000', 'last_change': '2023-08-09T13:51:59.157225+0000', 'last_active': '2023-08-09T13:51:59.130555+0000', 'last_peered': '2023-08-09T13:51:59.130555+0000', 'last_clean': '2023-08-09T13:51:59.130555+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-08-09T13:51:59.157225+0000', 'last_undegraded': '2023-08-09T13:51:59.157225+0000', 'last_fullsized': '2023-08-09T13:51:59.157225+0000', 'mapping_epoch': 114, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 114, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-08-09T13:51:59.130555+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-08-09T13:51:59.130555+0000', 'last_clean_scrub_stamp': '2023-08-09T13:51:59.130555+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [7, 0], 'acting': [7, 0], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 7, 'acting_primary': 7, 'purged_snaps': []} 2023-08-09T13:52:05.806 INFO:tasks.cephfs.filesystem.ceph_manager:PG 15.2f is not active+clean 2023-08-09T13:52:05.806 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '15.2f', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 114, 'state': 'creating+peering', 'last_fresh': '2023-08-09T13:51:59.157118+0000', 'last_change': '2023-08-09T13:51:59.157118+0000', 'last_active': '2023-08-09T13:51:59.130555+0000', 'last_peered': '2023-08-09T13:51:59.130555+0000', 'last_clean': '2023-08-09T13:51:59.130555+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-08-09T13:51:59.157118+0000', 'last_undegraded': '2023-08-09T13:51:59.157118+0000', 'last_fullsized': '2023-08-09T13:51:59.157118+0000', 'mapping_epoch': 114, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 114, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-08-09T13:51:59.130555+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-08-09T13:51:59.130555+0000', 'last_clean_scrub_stamp': '2023-08-09T13:51:59.130555+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 3], 'acting': [0, 3], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2023-08-09T13:52:05.807 INFO:tasks.cephfs.filesystem.ceph_manager:PG 15.2e is not active+clean 2023-08-09T13:52:05.807 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '15.2e', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 114, 'state': 'creating+peering', 'last_fresh': '2023-08-09T13:51:59.153061+0000', 'last_change': '2023-08-09T13:51:59.153061+0000', 'last_active': '2023-08-09T13:51:59.130555+0000', 'last_peered': '2023-08-09T13:51:59.130555+0000', 'last_clean': '2023-08-09T13:51:59.130555+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-08-09T13:51:59.153061+0000', 'last_undegraded': '2023-08-09T13:51:59.153061+0000', 'last_fullsized': '2023-08-09T13:51:59.153061+0000', 'mapping_epoch': 114, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 114, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-08-09T13:51:59.130555+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-08-09T13:51:59.130555+0000', 'last_clean_scrub_stamp': '2023-08-09T13:51:59.130555+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 2], 'acting': [3, 2], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2023-08-09T13:52:05.807 INFO:tasks.cephfs.filesystem.ceph_manager:PG 15.2d is not active+clean 2023-08-09T13:52:05.808 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '15.2d', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 114, 'state': 'creating+peering', 'last_fresh': '2023-08-09T13:51:59.156327+0000', 'last_change': '2023-08-09T13:51:59.156327+0000', 'last_active': '2023-08-09T13:51:59.130555+0000', 'last_peered': '2023-08-09T13:51:59.130555+0000', 'last_clean': '2023-08-09T13:51:59.130555+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-08-09T13:51:59.156327+0000', 'last_undegraded': '2023-08-09T13:51:59.156327+0000', 'last_fullsized': '2023-08-09T13:51:59.156327+0000', 'mapping_epoch': 114, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 114, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-08-09T13:51:59.130555+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-08-09T13:51:59.130555+0000', 'last_clean_scrub_stamp': '2023-08-09T13:51:59.130555+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 1], 'acting': [2, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2023-08-09T13:52:05.808 INFO:tasks.cephfs.filesystem.ceph_manager:PG 15.2c is not active+clean 2023-08-09T13:52:05.808 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '15.2c', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 114, 'state': 'creating+peering', 'last_fresh': '2023-08-09T13:51:59.175350+0000', 'last_change': '2023-08-09T13:51:59.175350+0000', 'last_active': '2023-08-09T13:51:59.130555+0000', 'last_peered': '2023-08-09T13:51:59.130555+0000', 'last_clean': '2023-08-09T13:51:59.130555+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-08-09T13:51:59.175350+0000', 'last_undegraded': '2023-08-09T13:51:59.175350+0000', 'last_fullsized': '2023-08-09T13:51:59.175350+0000', 'mapping_epoch': 114, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 114, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-08-09T13:51:59.130555+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-08-09T13:51:59.130555+0000', 'last_clean_scrub_stamp': '2023-08-09T13:51:59.130555+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [4, 3], 'acting': [4, 3], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 4, 'acting_primary': 4, 'purged_snaps': []} 2023-08-09T13:52:05.809 INFO:tasks.cephfs.filesystem.ceph_manager:PG 15.2b is not active+clean 2023-08-09T13:52:05.810 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '15.2b', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 114, 'state': 'creating+peering', 'last_fresh': '2023-08-09T13:51:59.163315+0000', 'last_change': '2023-08-09T13:51:59.163315+0000', 'last_active': '2023-08-09T13:51:59.130555+0000', 'last_peered': '2023-08-09T13:51:59.130555+0000', 'last_clean': '2023-08-09T13:51:59.130555+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-08-09T13:51:59.163315+0000', 'last_undegraded': '2023-08-09T13:51:59.163315+0000', 'last_fullsized': '2023-08-09T13:51:59.163315+0000', 'mapping_epoch': 114, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 114, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-08-09T13:51:59.130555+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-08-09T13:51:59.130555+0000', 'last_clean_scrub_stamp': '2023-08-09T13:51:59.130555+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 5], 'acting': [2, 5], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2023-08-09T13:52:05.810 INFO:tasks.cephfs.filesystem.ceph_manager:PG 15.2a is not active+clean 2023-08-09T13:52:05.811 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '15.2a', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 114, 'state': 'creating+peering', 'last_fresh': '2023-08-09T13:51:59.158490+0000', 'last_change': '2023-08-09T13:51:59.158490+0000', 'last_active': '2023-08-09T13:51:59.130555+0000', 'last_peered': '2023-08-09T13:51:59.130555+0000', 'last_clean': '2023-08-09T13:51:59.130555+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-08-09T13:51:59.158490+0000', 'last_undegraded': '2023-08-09T13:51:59.158490+0000', 'last_fullsized': '2023-08-09T13:51:59.158490+0000', 'mapping_epoch': 114, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 114, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-08-09T13:51:59.130555+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-08-09T13:51:59.130555+0000', 'last_clean_scrub_stamp': '2023-08-09T13:51:59.130555+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 5], 'acting': [0, 5], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2023-08-09T13:52:05.812 INFO:tasks.cephfs.filesystem.ceph_manager:PG 15.29 is not active+clean 2023-08-09T13:52:05.812 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '15.29', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 114, 'state': 'creating+peering', 'last_fresh': '2023-08-09T13:51:59.157791+0000', 'last_change': '2023-08-09T13:51:59.157791+0000', 'last_active': '2023-08-09T13:51:59.130555+0000', 'last_peered': '2023-08-09T13:51:59.130555+0000', 'last_clean': '2023-08-09T13:51:59.130555+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-08-09T13:51:59.157791+0000', 'last_undegraded': '2023-08-09T13:51:59.157791+0000', 'last_fullsized': '2023-08-09T13:51:59.157791+0000', 'mapping_epoch': 114, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 114, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-08-09T13:51:59.130555+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-08-09T13:51:59.130555+0000', 'last_clean_scrub_stamp': '2023-08-09T13:51:59.130555+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 5], 'acting': [0, 5], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2023-08-09T13:52:05.813 INFO:tasks.cephfs.filesystem.ceph_manager:PG 15.28 is not active+clean 2023-08-09T13:52:05.813 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '15.28', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 114, 'state': 'creating+peering', 'last_fresh': '2023-08-09T13:51:59.156031+0000', 'last_change': '2023-08-09T13:51:59.156031+0000', 'last_active': '2023-08-09T13:51:59.130555+0000', 'last_peered': '2023-08-09T13:51:59.130555+0000', 'last_clean': '2023-08-09T13:51:59.130555+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-08-09T13:51:59.156031+0000', 'last_undegraded': '2023-08-09T13:51:59.156031+0000', 'last_fullsized': '2023-08-09T13:51:59.156031+0000', 'mapping_epoch': 114, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 114, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-08-09T13:51:59.130555+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-08-09T13:51:59.130555+0000', 'last_clean_scrub_stamp': '2023-08-09T13:51:59.130555+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [7, 1], 'acting': [7, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 7, 'acting_primary': 7, 'purged_snaps': []} 2023-08-09T13:52:05.813 INFO:tasks.cephfs.filesystem.ceph_manager:PG 15.27 is not active+clean 2023-08-09T13:52:05.813 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '15.27', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 114, 'state': 'creating+peering', 'last_fresh': '2023-08-09T13:51:59.157867+0000', 'last_change': '2023-08-09T13:51:59.157867+0000', 'last_active': '2023-08-09T13:51:59.130555+0000', 'last_peered': '2023-08-09T13:51:59.130555+0000', 'last_clean': '2023-08-09T13:51:59.130555+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-08-09T13:51:59.157867+0000', 'last_undegraded': '2023-08-09T13:51:59.157867+0000', 'last_fullsized': '2023-08-09T13:51:59.157867+0000', 'mapping_epoch': 114, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 114, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-08-09T13:51:59.130555+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-08-09T13:51:59.130555+0000', 'last_clean_scrub_stamp': '2023-08-09T13:51:59.130555+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [6, 0], 'acting': [6, 0], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 6, 'acting_primary': 6, 'purged_snaps': []} 2023-08-09T13:52:05.814 INFO:tasks.cephfs.filesystem.ceph_manager:PG 15.26 is not active+clean 2023-08-09T13:52:05.814 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '15.26', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 114, 'state': 'creating+peering', 'last_fresh': '2023-08-09T13:51:59.159192+0000', 'last_change': '2023-08-09T13:51:59.159192+0000', 'last_active': '2023-08-09T13:51:59.130555+0000', 'last_peered': '2023-08-09T13:51:59.130555+0000', 'last_clean': '2023-08-09T13:51:59.130555+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-08-09T13:51:59.159192+0000', 'last_undegraded': '2023-08-09T13:51:59.159192+0000', 'last_fullsized': '2023-08-09T13:51:59.159192+0000', 'mapping_epoch': 114, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 114, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-08-09T13:51:59.130555+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-08-09T13:51:59.130555+0000', 'last_clean_scrub_stamp': '2023-08-09T13:51:59.130555+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2023-08-09T13:52:05.814 INFO:tasks.cephfs.filesystem.ceph_manager:PG 15.25 is not active+clean 2023-08-09T13:52:05.815 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '15.25', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 114, 'state': 'creating+peering', 'last_fresh': '2023-08-09T13:51:59.157367+0000', 'last_change': '2023-08-09T13:51:59.157367+0000', 'last_active': '2023-08-09T13:51:59.130555+0000', 'last_peered': '2023-08-09T13:51:59.130555+0000', 'last_clean': '2023-08-09T13:51:59.130555+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-08-09T13:51:59.157367+0000', 'last_undegraded': '2023-08-09T13:51:59.157367+0000', 'last_fullsized': '2023-08-09T13:51:59.157367+0000', 'mapping_epoch': 114, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 114, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-08-09T13:51:59.130555+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-08-09T13:51:59.130555+0000', 'last_clean_scrub_stamp': '2023-08-09T13:51:59.130555+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [7, 2], 'acting': [7, 2], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 7, 'acting_primary': 7, 'purged_snaps': []} 2023-08-09T13:52:05.815 INFO:tasks.cephfs.filesystem.ceph_manager:PG 15.24 is not active+clean 2023-08-09T13:52:05.815 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '15.24', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 114, 'state': 'creating+peering', 'last_fresh': '2023-08-09T13:51:59.162536+0000', 'last_change': '2023-08-09T13:51:59.162536+0000', 'last_active': '2023-08-09T13:51:59.130555+0000', 'last_peered': '2023-08-09T13:51:59.130555+0000', 'last_clean': '2023-08-09T13:51:59.130555+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-08-09T13:51:59.162536+0000', 'last_undegraded': '2023-08-09T13:51:59.162536+0000', 'last_fullsized': '2023-08-09T13:51:59.162536+0000', 'mapping_epoch': 114, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 114, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-08-09T13:51:59.130555+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-08-09T13:51:59.130555+0000', 'last_clean_scrub_stamp': '2023-08-09T13:51:59.130555+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [6, 3], 'acting': [6, 3], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 6, 'acting_primary': 6, 'purged_snaps': []} 2023-08-09T13:52:05.816 INFO:tasks.cephfs.filesystem.ceph_manager:PG 15.23 is not active+clean 2023-08-09T13:52:05.816 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '15.23', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 114, 'state': 'creating+peering', 'last_fresh': '2023-08-09T13:51:59.160575+0000', 'last_change': '2023-08-09T13:51:59.160575+0000', 'last_active': '2023-08-09T13:51:59.130555+0000', 'last_peered': '2023-08-09T13:51:59.130555+0000', 'last_clean': '2023-08-09T13:51:59.130555+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-08-09T13:51:59.160575+0000', 'last_undegraded': '2023-08-09T13:51:59.160575+0000', 'last_fullsized': '2023-08-09T13:51:59.160575+0000', 'mapping_epoch': 114, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 114, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-08-09T13:51:59.130555+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-08-09T13:51:59.130555+0000', 'last_clean_scrub_stamp': '2023-08-09T13:51:59.130555+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2023-08-09T13:52:05.816 INFO:tasks.cephfs.filesystem.ceph_manager:PG 15.22 is not active+clean 2023-08-09T13:52:05.817 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '15.22', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 114, 'state': 'creating+peering', 'last_fresh': '2023-08-09T13:51:59.164643+0000', 'last_change': '2023-08-09T13:51:59.164643+0000', 'last_active': '2023-08-09T13:51:59.130555+0000', 'last_peered': '2023-08-09T13:51:59.130555+0000', 'last_clean': '2023-08-09T13:51:59.130555+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-08-09T13:51:59.164643+0000', 'last_undegraded': '2023-08-09T13:51:59.164643+0000', 'last_fullsized': '2023-08-09T13:51:59.164643+0000', 'mapping_epoch': 114, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 114, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-08-09T13:51:59.130555+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-08-09T13:51:59.130555+0000', 'last_clean_scrub_stamp': '2023-08-09T13:51:59.130555+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 1], 'acting': [2, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2023-08-09T13:52:05.817 INFO:tasks.cephfs.filesystem.ceph_manager:PG 15.21 is not active+clean 2023-08-09T13:52:05.817 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '15.21', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 114, 'state': 'creating+peering', 'last_fresh': '2023-08-09T13:51:59.149533+0000', 'last_change': '2023-08-09T13:51:59.149533+0000', 'last_active': '2023-08-09T13:51:59.130555+0000', 'last_peered': '2023-08-09T13:51:59.130555+0000', 'last_clean': '2023-08-09T13:51:59.130555+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-08-09T13:51:59.149533+0000', 'last_undegraded': '2023-08-09T13:51:59.149533+0000', 'last_fullsized': '2023-08-09T13:51:59.149533+0000', 'mapping_epoch': 114, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 114, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-08-09T13:51:59.130555+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-08-09T13:51:59.130555+0000', 'last_clean_scrub_stamp': '2023-08-09T13:51:59.130555+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 6], 'acting': [3, 6], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2023-08-09T13:52:05.818 INFO:tasks.cephfs.filesystem.ceph_manager:PG 15.20 is not active+clean 2023-08-09T13:52:05.818 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '15.20', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 114, 'state': 'creating+peering', 'last_fresh': '2023-08-09T13:51:59.168339+0000', 'last_change': '2023-08-09T13:51:59.168339+0000', 'last_active': '2023-08-09T13:51:59.130555+0000', 'last_peered': '2023-08-09T13:51:59.130555+0000', 'last_clean': '2023-08-09T13:51:59.130555+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-08-09T13:51:59.168339+0000', 'last_undegraded': '2023-08-09T13:51:59.168339+0000', 'last_fullsized': '2023-08-09T13:51:59.168339+0000', 'mapping_epoch': 114, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 114, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-08-09T13:51:59.130555+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-08-09T13:51:59.130555+0000', 'last_clean_scrub_stamp': '2023-08-09T13:51:59.130555+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [1, 0], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 1, 'purged_snaps': []} 2023-08-09T13:52:05.818 INFO:tasks.cephfs.filesystem.ceph_manager:PG 15.1e is not active+clean 2023-08-09T13:52:05.819 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '15.1e', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 114, 'state': 'creating+peering', 'last_fresh': '2023-08-09T13:51:59.162676+0000', 'last_change': '2023-08-09T13:51:59.162676+0000', 'last_active': '2023-08-09T13:51:59.130555+0000', 'last_peered': '2023-08-09T13:51:59.130555+0000', 'last_clean': '2023-08-09T13:51:59.130555+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-08-09T13:51:59.162676+0000', 'last_undegraded': '2023-08-09T13:51:59.162676+0000', 'last_fullsized': '2023-08-09T13:51:59.162676+0000', 'mapping_epoch': 114, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 114, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-08-09T13:51:59.130555+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-08-09T13:51:59.130555+0000', 'last_clean_scrub_stamp': '2023-08-09T13:51:59.130555+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 7], 'acting': [2, 7], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2023-08-09T13:52:05.819 INFO:tasks.cephfs.filesystem.ceph_manager:PG 15.1f is not active+clean 2023-08-09T13:52:05.819 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '15.1f', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 114, 'state': 'creating+peering', 'last_fresh': '2023-08-09T13:51:59.151940+0000', 'last_change': '2023-08-09T13:51:59.151940+0000', 'last_active': '2023-08-09T13:51:59.130555+0000', 'last_peered': '2023-08-09T13:51:59.130555+0000', 'last_clean': '2023-08-09T13:51:59.130555+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-08-09T13:51:59.151940+0000', 'last_undegraded': '2023-08-09T13:51:59.151940+0000', 'last_fullsized': '2023-08-09T13:51:59.151940+0000', 'mapping_epoch': 114, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 114, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-08-09T13:51:59.130555+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-08-09T13:51:59.130555+0000', 'last_clean_scrub_stamp': '2023-08-09T13:51:59.130555+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 6], 'acting': [3, 6], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2023-08-09T13:52:05.820 INFO:tasks.cephfs.filesystem.ceph_manager:PG 15.1c is not active+clean 2023-08-09T13:52:05.820 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '15.1c', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 114, 'state': 'creating+peering', 'last_fresh': '2023-08-09T13:51:59.166472+0000', 'last_change': '2023-08-09T13:51:59.166472+0000', 'last_active': '2023-08-09T13:51:59.130555+0000', 'last_peered': '2023-08-09T13:51:59.130555+0000', 'last_clean': '2023-08-09T13:51:59.130555+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-08-09T13:51:59.166472+0000', 'last_undegraded': '2023-08-09T13:51:59.166472+0000', 'last_fullsized': '2023-08-09T13:51:59.166472+0000', 'mapping_epoch': 114, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 114, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-08-09T13:51:59.130555+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-08-09T13:51:59.130555+0000', 'last_clean_scrub_stamp': '2023-08-09T13:51:59.130555+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 4], 'acting': [1, 4], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 1, 'purged_snaps': []} 2023-08-09T13:52:05.820 INFO:tasks.cephfs.filesystem.ceph_manager:PG 15.1d is not active+clean 2023-08-09T13:52:05.821 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '15.1d', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 114, 'state': 'creating+peering', 'last_fresh': '2023-08-09T13:51:59.170197+0000', 'last_change': '2023-08-09T13:51:59.170197+0000', 'last_active': '2023-08-09T13:51:59.130555+0000', 'last_peered': '2023-08-09T13:51:59.130555+0000', 'last_clean': '2023-08-09T13:51:59.130555+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-08-09T13:51:59.170197+0000', 'last_undegraded': '2023-08-09T13:51:59.170197+0000', 'last_fullsized': '2023-08-09T13:51:59.170197+0000', 'mapping_epoch': 114, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 114, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-08-09T13:51:59.130555+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-08-09T13:51:59.130555+0000', 'last_clean_scrub_stamp': '2023-08-09T13:51:59.130555+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [4, 5], 'acting': [4, 5], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 4, 'acting_primary': 4, 'purged_snaps': []} 2023-08-09T13:52:05.821 INFO:tasks.cephfs.filesystem.ceph_manager:PG 15.1a is not active+clean 2023-08-09T13:52:05.821 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '15.1a', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 114, 'state': 'creating+peering', 'last_fresh': '2023-08-09T13:51:59.165426+0000', 'last_change': '2023-08-09T13:51:59.165426+0000', 'last_active': '2023-08-09T13:51:59.130555+0000', 'last_peered': '2023-08-09T13:51:59.130555+0000', 'last_clean': '2023-08-09T13:51:59.130555+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-08-09T13:51:59.165426+0000', 'last_undegraded': '2023-08-09T13:51:59.165426+0000', 'last_fullsized': '2023-08-09T13:51:59.165426+0000', 'mapping_epoch': 114, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 114, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-08-09T13:51:59.130555+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-08-09T13:51:59.130555+0000', 'last_clean_scrub_stamp': '2023-08-09T13:51:59.130555+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [5, 4], 'acting': [5, 4], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 5, 'acting_primary': 5, 'purged_snaps': []} 2023-08-09T13:52:05.822 INFO:tasks.cephfs.filesystem.ceph_manager:PG 15.1b is not active+clean 2023-08-09T13:52:05.822 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '15.1b', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 114, 'state': 'creating+peering', 'last_fresh': '2023-08-09T13:51:59.157444+0000', 'last_change': '2023-08-09T13:51:59.157444+0000', 'last_active': '2023-08-09T13:51:59.130555+0000', 'last_peered': '2023-08-09T13:51:59.130555+0000', 'last_clean': '2023-08-09T13:51:59.130555+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-08-09T13:51:59.157444+0000', 'last_undegraded': '2023-08-09T13:51:59.157444+0000', 'last_fullsized': '2023-08-09T13:51:59.157444+0000', 'mapping_epoch': 114, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 114, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-08-09T13:51:59.130555+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-08-09T13:51:59.130555+0000', 'last_clean_scrub_stamp': '2023-08-09T13:51:59.130555+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [7, 6], 'acting': [7, 6], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 7, 'acting_primary': 7, 'purged_snaps': []} 2023-08-09T13:52:05.822 INFO:tasks.cephfs.filesystem.ceph_manager:PG 15.18 is not active+clean 2023-08-09T13:52:05.822 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '15.18', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 114, 'state': 'creating+peering', 'last_fresh': '2023-08-09T13:51:59.149114+0000', 'last_change': '2023-08-09T13:51:59.149114+0000', 'last_active': '2023-08-09T13:51:59.130555+0000', 'last_peered': '2023-08-09T13:51:59.130555+0000', 'last_clean': '2023-08-09T13:51:59.130555+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-08-09T13:51:59.149114+0000', 'last_undegraded': '2023-08-09T13:51:59.149114+0000', 'last_fullsized': '2023-08-09T13:51:59.149114+0000', 'mapping_epoch': 114, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 114, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-08-09T13:51:59.130555+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-08-09T13:51:59.130555+0000', 'last_clean_scrub_stamp': '2023-08-09T13:51:59.130555+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 1], 'acting': [3, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2023-08-09T13:52:05.823 INFO:tasks.cephfs.filesystem.ceph_manager:PG 15.19 is not active+clean 2023-08-09T13:52:05.823 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '15.19', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 114, 'state': 'creating+peering', 'last_fresh': '2023-08-09T13:51:59.168819+0000', 'last_change': '2023-08-09T13:51:59.168819+0000', 'last_active': '2023-08-09T13:51:59.130555+0000', 'last_peered': '2023-08-09T13:51:59.130555+0000', 'last_clean': '2023-08-09T13:51:59.130555+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-08-09T13:51:59.168819+0000', 'last_undegraded': '2023-08-09T13:51:59.168819+0000', 'last_fullsized': '2023-08-09T13:51:59.168819+0000', 'mapping_epoch': 114, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 114, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-08-09T13:51:59.130555+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-08-09T13:51:59.130555+0000', 'last_clean_scrub_stamp': '2023-08-09T13:51:59.130555+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [4, 5], 'acting': [4, 5], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 4, 'acting_primary': 4, 'purged_snaps': []} 2023-08-09T13:52:05.823 INFO:tasks.cephfs.filesystem.ceph_manager:PG 15.16 is not active+clean 2023-08-09T13:52:05.824 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '15.16', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 114, 'state': 'creating+peering', 'last_fresh': '2023-08-09T13:51:59.158359+0000', 'last_change': '2023-08-09T13:51:59.158359+0000', 'last_active': '2023-08-09T13:51:59.130555+0000', 'last_peered': '2023-08-09T13:51:59.130555+0000', 'last_clean': '2023-08-09T13:51:59.130555+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-08-09T13:51:59.158359+0000', 'last_undegraded': '2023-08-09T13:51:59.158359+0000', 'last_fullsized': '2023-08-09T13:51:59.158359+0000', 'mapping_epoch': 114, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 114, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-08-09T13:51:59.130555+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-08-09T13:51:59.130555+0000', 'last_clean_scrub_stamp': '2023-08-09T13:51:59.130555+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 4], 'acting': [0, 4], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2023-08-09T13:52:05.824 INFO:tasks.cephfs.filesystem.ceph_manager:PG 15.17 is not active+clean 2023-08-09T13:52:05.824 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '15.17', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 114, 'state': 'creating+peering', 'last_fresh': '2023-08-09T13:51:59.169791+0000', 'last_change': '2023-08-09T13:51:59.169791+0000', 'last_active': '2023-08-09T13:51:59.130555+0000', 'last_peered': '2023-08-09T13:51:59.130555+0000', 'last_clean': '2023-08-09T13:51:59.130555+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-08-09T13:51:59.169791+0000', 'last_undegraded': '2023-08-09T13:51:59.169791+0000', 'last_fullsized': '2023-08-09T13:51:59.169791+0000', 'mapping_epoch': 114, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 114, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-08-09T13:51:59.130555+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-08-09T13:51:59.130555+0000', 'last_clean_scrub_stamp': '2023-08-09T13:51:59.130555+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [4, 5], 'acting': [4, 5], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 4, 'acting_primary': 4, 'purged_snaps': []} 2023-08-09T13:52:05.825 INFO:tasks.cephfs.filesystem.ceph_manager:PG 15.14 is not active+clean 2023-08-09T13:52:05.825 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '15.14', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 114, 'state': 'creating+peering', 'last_fresh': '2023-08-09T13:51:59.161549+0000', 'last_change': '2023-08-09T13:51:59.161549+0000', 'last_active': '2023-08-09T13:51:59.130555+0000', 'last_peered': '2023-08-09T13:51:59.130555+0000', 'last_clean': '2023-08-09T13:51:59.130555+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-08-09T13:51:59.161549+0000', 'last_undegraded': '2023-08-09T13:51:59.161549+0000', 'last_fullsized': '2023-08-09T13:51:59.161549+0000', 'mapping_epoch': 114, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 114, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-08-09T13:51:59.130555+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-08-09T13:51:59.130555+0000', 'last_clean_scrub_stamp': '2023-08-09T13:51:59.130555+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [6, 0], 'acting': [6, 0], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 6, 'acting_primary': 6, 'purged_snaps': []} 2023-08-09T13:52:05.825 INFO:tasks.cephfs.filesystem.ceph_manager:PG 15.15 is not active+clean 2023-08-09T13:52:05.826 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '15.15', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 114, 'state': 'creating+peering', 'last_fresh': '2023-08-09T13:51:59.156075+0000', 'last_change': '2023-08-09T13:51:59.156075+0000', 'last_active': '2023-08-09T13:51:59.130555+0000', 'last_peered': '2023-08-09T13:51:59.130555+0000', 'last_clean': '2023-08-09T13:51:59.130555+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-08-09T13:51:59.156075+0000', 'last_undegraded': '2023-08-09T13:51:59.156075+0000', 'last_fullsized': '2023-08-09T13:51:59.156075+0000', 'mapping_epoch': 114, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 114, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-08-09T13:51:59.130555+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-08-09T13:51:59.130555+0000', 'last_clean_scrub_stamp': '2023-08-09T13:51:59.130555+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [7, 0], 'acting': [7, 0], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 7, 'acting_primary': 7, 'purged_snaps': []} 2023-08-09T13:52:05.826 INFO:tasks.cephfs.filesystem.ceph_manager:PG 15.12 is not active+clean 2023-08-09T13:52:05.826 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '15.12', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 114, 'state': 'creating+peering', 'last_fresh': '2023-08-09T13:51:59.150320+0000', 'last_change': '2023-08-09T13:51:59.150320+0000', 'last_active': '2023-08-09T13:51:59.130555+0000', 'last_peered': '2023-08-09T13:51:59.130555+0000', 'last_clean': '2023-08-09T13:51:59.130555+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-08-09T13:51:59.150320+0000', 'last_undegraded': '2023-08-09T13:51:59.150320+0000', 'last_fullsized': '2023-08-09T13:51:59.150320+0000', 'mapping_epoch': 114, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 114, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-08-09T13:51:59.130555+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-08-09T13:51:59.130555+0000', 'last_clean_scrub_stamp': '2023-08-09T13:51:59.130555+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 7], 'acting': [3, 7], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2023-08-09T13:52:05.827 INFO:tasks.cephfs.filesystem.ceph_manager:PG 15.13 is not active+clean 2023-08-09T13:52:05.827 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '15.13', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 114, 'state': 'creating+peering', 'last_fresh': '2023-08-09T13:51:59.163775+0000', 'last_change': '2023-08-09T13:51:59.163775+0000', 'last_active': '2023-08-09T13:51:59.130555+0000', 'last_peered': '2023-08-09T13:51:59.130555+0000', 'last_clean': '2023-08-09T13:51:59.130555+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-08-09T13:51:59.163775+0000', 'last_undegraded': '2023-08-09T13:51:59.163775+0000', 'last_fullsized': '2023-08-09T13:51:59.163775+0000', 'mapping_epoch': 114, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 114, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-08-09T13:51:59.130555+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-08-09T13:51:59.130555+0000', 'last_clean_scrub_stamp': '2023-08-09T13:51:59.130555+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 2], 'acting': [1, 2], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 1, 'purged_snaps': []} 2023-08-09T13:52:05.827 INFO:tasks.cephfs.filesystem.ceph_manager:PG 15.10 is not active+clean 2023-08-09T13:52:05.828 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '15.10', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 114, 'state': 'creating+peering', 'last_fresh': '2023-08-09T13:51:59.167863+0000', 'last_change': '2023-08-09T13:51:59.167863+0000', 'last_active': '2023-08-09T13:51:59.130555+0000', 'last_peered': '2023-08-09T13:51:59.130555+0000', 'last_clean': '2023-08-09T13:51:59.130555+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-08-09T13:51:59.167863+0000', 'last_undegraded': '2023-08-09T13:51:59.167863+0000', 'last_fullsized': '2023-08-09T13:51:59.167863+0000', 'mapping_epoch': 114, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 114, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-08-09T13:51:59.130555+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-08-09T13:51:59.130555+0000', 'last_clean_scrub_stamp': '2023-08-09T13:51:59.130555+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [4, 5], 'acting': [4, 5], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 4, 'acting_primary': 4, 'purged_snaps': []} 2023-08-09T13:52:05.828 INFO:tasks.cephfs.filesystem.ceph_manager:PG 15.11 is not active+clean 2023-08-09T13:52:05.828 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '15.11', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 114, 'state': 'creating+peering', 'last_fresh': '2023-08-09T13:51:59.151102+0000', 'last_change': '2023-08-09T13:51:59.151102+0000', 'last_active': '2023-08-09T13:51:59.130555+0000', 'last_peered': '2023-08-09T13:51:59.130555+0000', 'last_clean': '2023-08-09T13:51:59.130555+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-08-09T13:51:59.151102+0000', 'last_undegraded': '2023-08-09T13:51:59.151102+0000', 'last_fullsized': '2023-08-09T13:51:59.151102+0000', 'mapping_epoch': 114, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 114, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-08-09T13:51:59.130555+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-08-09T13:51:59.130555+0000', 'last_clean_scrub_stamp': '2023-08-09T13:51:59.130555+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 4], 'acting': [2, 4], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2023-08-09T13:52:05.829 INFO:tasks.cephfs.filesystem.ceph_manager:PG 15.6 is not active+clean 2023-08-09T13:52:05.829 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '15.6', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 114, 'state': 'creating+peering', 'last_fresh': '2023-08-09T13:51:59.157299+0000', 'last_change': '2023-08-09T13:51:59.157299+0000', 'last_active': '2023-08-09T13:51:59.130555+0000', 'last_peered': '2023-08-09T13:51:59.130555+0000', 'last_clean': '2023-08-09T13:51:59.130555+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-08-09T13:51:59.157299+0000', 'last_undegraded': '2023-08-09T13:51:59.157299+0000', 'last_fullsized': '2023-08-09T13:51:59.157299+0000', 'mapping_epoch': 114, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 114, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-08-09T13:51:59.130555+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-08-09T13:51:59.130555+0000', 'last_clean_scrub_stamp': '2023-08-09T13:51:59.130555+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 5], 'acting': [0, 5], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2023-08-09T13:52:05.829 INFO:tasks.cephfs.filesystem.ceph_manager:PG 15.7 is not active+clean 2023-08-09T13:52:05.830 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '15.7', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 114, 'state': 'creating+peering', 'last_fresh': '2023-08-09T13:51:59.162752+0000', 'last_change': '2023-08-09T13:51:59.162752+0000', 'last_active': '2023-08-09T13:51:59.130555+0000', 'last_peered': '2023-08-09T13:51:59.130555+0000', 'last_clean': '2023-08-09T13:51:59.130555+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-08-09T13:51:59.162752+0000', 'last_undegraded': '2023-08-09T13:51:59.162752+0000', 'last_fullsized': '2023-08-09T13:51:59.162752+0000', 'mapping_epoch': 114, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 114, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-08-09T13:51:59.130555+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-08-09T13:51:59.130555+0000', 'last_clean_scrub_stamp': '2023-08-09T13:51:59.130555+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 2], 'acting': [1, 2], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 1, 'purged_snaps': []} 2023-08-09T13:52:05.830 INFO:tasks.cephfs.filesystem.ceph_manager:PG 15.4 is not active+clean 2023-08-09T13:52:05.830 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '15.4', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 114, 'state': 'creating+peering', 'last_fresh': '2023-08-09T13:51:59.159598+0000', 'last_change': '2023-08-09T13:51:59.159598+0000', 'last_active': '2023-08-09T13:51:59.130555+0000', 'last_peered': '2023-08-09T13:51:59.130555+0000', 'last_clean': '2023-08-09T13:51:59.130555+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-08-09T13:51:59.159598+0000', 'last_undegraded': '2023-08-09T13:51:59.159598+0000', 'last_fullsized': '2023-08-09T13:51:59.159598+0000', 'mapping_epoch': 114, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 114, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-08-09T13:51:59.130555+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-08-09T13:51:59.130555+0000', 'last_clean_scrub_stamp': '2023-08-09T13:51:59.130555+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 6], 'acting': [1, 6], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 1, 'purged_snaps': []} 2023-08-09T13:52:05.830 INFO:tasks.cephfs.filesystem.ceph_manager:PG 15.5 is not active+clean 2023-08-09T13:52:05.831 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '15.5', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 114, 'state': 'creating+peering', 'last_fresh': '2023-08-09T13:51:59.156952+0000', 'last_change': '2023-08-09T13:51:59.156952+0000', 'last_active': '2023-08-09T13:51:59.130555+0000', 'last_peered': '2023-08-09T13:51:59.130555+0000', 'last_clean': '2023-08-09T13:51:59.130555+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-08-09T13:51:59.156952+0000', 'last_undegraded': '2023-08-09T13:51:59.156952+0000', 'last_fullsized': '2023-08-09T13:51:59.156952+0000', 'mapping_epoch': 114, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 114, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-08-09T13:51:59.130555+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-08-09T13:51:59.130555+0000', 'last_clean_scrub_stamp': '2023-08-09T13:51:59.130555+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [6, 3], 'acting': [6, 3], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 6, 'acting_primary': 6, 'purged_snaps': []} 2023-08-09T13:52:05.831 INFO:tasks.cephfs.filesystem.ceph_manager:PG 15.2 is not active+clean 2023-08-09T13:52:05.831 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '15.2', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 114, 'state': 'creating+peering', 'last_fresh': '2023-08-09T13:51:59.154951+0000', 'last_change': '2023-08-09T13:51:59.154951+0000', 'last_active': '2023-08-09T13:51:59.130555+0000', 'last_peered': '2023-08-09T13:51:59.130555+0000', 'last_clean': '2023-08-09T13:51:59.130555+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-08-09T13:51:59.154951+0000', 'last_undegraded': '2023-08-09T13:51:59.154951+0000', 'last_fullsized': '2023-08-09T13:51:59.154951+0000', 'mapping_epoch': 114, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 114, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-08-09T13:51:59.130555+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-08-09T13:51:59.130555+0000', 'last_clean_scrub_stamp': '2023-08-09T13:51:59.130555+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 4], 'acting': [2, 4], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2023-08-09T13:52:05.832 INFO:tasks.cephfs.filesystem.ceph_manager:PG 15.3 is not active+clean 2023-08-09T13:52:05.832 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '15.3', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 114, 'state': 'creating+peering', 'last_fresh': '2023-08-09T13:51:59.169608+0000', 'last_change': '2023-08-09T13:51:59.169608+0000', 'last_active': '2023-08-09T13:51:59.130555+0000', 'last_peered': '2023-08-09T13:51:59.130555+0000', 'last_clean': '2023-08-09T13:51:59.130555+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-08-09T13:51:59.169608+0000', 'last_undegraded': '2023-08-09T13:51:59.169608+0000', 'last_fullsized': '2023-08-09T13:51:59.169608+0000', 'mapping_epoch': 114, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 114, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-08-09T13:51:59.130555+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-08-09T13:51:59.130555+0000', 'last_clean_scrub_stamp': '2023-08-09T13:51:59.130555+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [4, 6], 'acting': [4, 6], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 4, 'acting_primary': 4, 'purged_snaps': []} 2023-08-09T13:52:05.832 INFO:tasks.cephfs.filesystem.ceph_manager:PG 15.0 is not active+clean 2023-08-09T13:52:05.833 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '15.0', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 114, 'state': 'creating+peering', 'last_fresh': '2023-08-09T13:51:59.153156+0000', 'last_change': '2023-08-09T13:51:59.153156+0000', 'last_active': '2023-08-09T13:51:59.130555+0000', 'last_peered': '2023-08-09T13:51:59.130555+0000', 'last_clean': '2023-08-09T13:51:59.130555+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-08-09T13:51:59.153156+0000', 'last_undegraded': '2023-08-09T13:51:59.153156+0000', 'last_fullsized': '2023-08-09T13:51:59.153156+0000', 'mapping_epoch': 114, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 114, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-08-09T13:51:59.130555+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-08-09T13:51:59.130555+0000', 'last_clean_scrub_stamp': '2023-08-09T13:51:59.130555+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 4], 'acting': [2, 4], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2023-08-09T13:52:05.833 INFO:tasks.cephfs.filesystem.ceph_manager:PG 15.1 is not active+clean 2023-08-09T13:52:05.833 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '15.1', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 114, 'state': 'creating+peering', 'last_fresh': '2023-08-09T13:51:59.156158+0000', 'last_change': '2023-08-09T13:51:59.156158+0000', 'last_active': '2023-08-09T13:51:59.130555+0000', 'last_peered': '2023-08-09T13:51:59.130555+0000', 'last_clean': '2023-08-09T13:51:59.130555+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-08-09T13:51:59.156158+0000', 'last_undegraded': '2023-08-09T13:51:59.156158+0000', 'last_fullsized': '2023-08-09T13:51:59.156158+0000', 'mapping_epoch': 114, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 114, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-08-09T13:51:59.130555+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-08-09T13:51:59.130555+0000', 'last_clean_scrub_stamp': '2023-08-09T13:51:59.130555+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [6, 3], 'acting': [6, 3], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 6, 'acting_primary': 6, 'purged_snaps': []} 2023-08-09T13:52:05.834 INFO:tasks.cephfs.filesystem.ceph_manager:PG 15.e is not active+clean 2023-08-09T13:52:05.834 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '15.e', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 114, 'state': 'creating+peering', 'last_fresh': '2023-08-09T13:51:59.155432+0000', 'last_change': '2023-08-09T13:51:59.155432+0000', 'last_active': '2023-08-09T13:51:59.130555+0000', 'last_peered': '2023-08-09T13:51:59.130555+0000', 'last_clean': '2023-08-09T13:51:59.130555+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-08-09T13:51:59.155432+0000', 'last_undegraded': '2023-08-09T13:51:59.155432+0000', 'last_fullsized': '2023-08-09T13:51:59.155432+0000', 'mapping_epoch': 114, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 114, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-08-09T13:51:59.130555+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-08-09T13:51:59.130555+0000', 'last_clean_scrub_stamp': '2023-08-09T13:51:59.130555+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 3], 'acting': [2, 3], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2023-08-09T13:52:05.834 INFO:tasks.cephfs.filesystem.ceph_manager:PG 15.f is not active+clean 2023-08-09T13:52:05.835 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '15.f', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 114, 'state': 'creating+peering', 'last_fresh': '2023-08-09T13:51:59.156558+0000', 'last_change': '2023-08-09T13:51:59.156558+0000', 'last_active': '2023-08-09T13:51:59.130555+0000', 'last_peered': '2023-08-09T13:51:59.130555+0000', 'last_clean': '2023-08-09T13:51:59.130555+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-08-09T13:51:59.156558+0000', 'last_undegraded': '2023-08-09T13:51:59.156558+0000', 'last_fullsized': '2023-08-09T13:51:59.156558+0000', 'mapping_epoch': 114, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 114, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-08-09T13:51:59.130555+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-08-09T13:51:59.130555+0000', 'last_clean_scrub_stamp': '2023-08-09T13:51:59.130555+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [6, 4], 'acting': [6, 4], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 6, 'acting_primary': 6, 'purged_snaps': []} 2023-08-09T13:52:05.835 INFO:tasks.cephfs.filesystem.ceph_manager:PG 15.c is not active+clean 2023-08-09T13:52:05.835 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '15.c', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 114, 'state': 'creating+peering', 'last_fresh': '2023-08-09T13:51:59.166603+0000', 'last_change': '2023-08-09T13:51:59.166603+0000', 'last_active': '2023-08-09T13:51:59.130555+0000', 'last_peered': '2023-08-09T13:51:59.130555+0000', 'last_clean': '2023-08-09T13:51:59.130555+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-08-09T13:51:59.166603+0000', 'last_undegraded': '2023-08-09T13:51:59.166603+0000', 'last_fullsized': '2023-08-09T13:51:59.166603+0000', 'mapping_epoch': 114, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 114, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-08-09T13:51:59.130555+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-08-09T13:51:59.130555+0000', 'last_clean_scrub_stamp': '2023-08-09T13:51:59.130555+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [5, 6], 'acting': [5, 6], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 5, 'acting_primary': 5, 'purged_snaps': []} 2023-08-09T13:52:05.836 INFO:tasks.cephfs.filesystem.ceph_manager:PG 15.d is not active+clean 2023-08-09T13:52:05.836 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '15.d', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 114, 'state': 'creating+peering', 'last_fresh': '2023-08-09T13:51:59.154844+0000', 'last_change': '2023-08-09T13:51:59.154844+0000', 'last_active': '2023-08-09T13:51:59.130555+0000', 'last_peered': '2023-08-09T13:51:59.130555+0000', 'last_clean': '2023-08-09T13:51:59.130555+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-08-09T13:51:59.154844+0000', 'last_undegraded': '2023-08-09T13:51:59.154844+0000', 'last_fullsized': '2023-08-09T13:51:59.154844+0000', 'mapping_epoch': 114, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 114, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-08-09T13:51:59.130555+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-08-09T13:51:59.130555+0000', 'last_clean_scrub_stamp': '2023-08-09T13:51:59.130555+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [7, 4], 'acting': [7, 4], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 7, 'acting_primary': 7, 'purged_snaps': []} 2023-08-09T13:52:05.836 INFO:tasks.cephfs.filesystem.ceph_manager:PG 15.a is not active+clean 2023-08-09T13:52:05.836 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '15.a', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 114, 'state': 'creating+peering', 'last_fresh': '2023-08-09T13:51:59.164187+0000', 'last_change': '2023-08-09T13:51:59.164187+0000', 'last_active': '2023-08-09T13:51:59.130555+0000', 'last_peered': '2023-08-09T13:51:59.130555+0000', 'last_clean': '2023-08-09T13:51:59.130555+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-08-09T13:51:59.164187+0000', 'last_undegraded': '2023-08-09T13:51:59.164187+0000', 'last_fullsized': '2023-08-09T13:51:59.164187+0000', 'mapping_epoch': 114, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 114, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-08-09T13:51:59.130555+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-08-09T13:51:59.130555+0000', 'last_clean_scrub_stamp': '2023-08-09T13:51:59.130555+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 3], 'acting': [2, 3], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2023-08-09T13:52:05.837 INFO:tasks.cephfs.filesystem.ceph_manager:PG 15.b is not active+clean 2023-08-09T13:52:05.837 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '15.b', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 114, 'state': 'creating+peering', 'last_fresh': '2023-08-09T13:51:59.156039+0000', 'last_change': '2023-08-09T13:51:59.156039+0000', 'last_active': '2023-08-09T13:51:59.130555+0000', 'last_peered': '2023-08-09T13:51:59.130555+0000', 'last_clean': '2023-08-09T13:51:59.130555+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-08-09T13:51:59.156039+0000', 'last_undegraded': '2023-08-09T13:51:59.156039+0000', 'last_fullsized': '2023-08-09T13:51:59.156039+0000', 'mapping_epoch': 114, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 114, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-08-09T13:51:59.130555+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-08-09T13:51:59.130555+0000', 'last_clean_scrub_stamp': '2023-08-09T13:51:59.130555+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 0], 'acting': [2, 0], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2023-08-09T13:52:05.837 INFO:tasks.cephfs.filesystem.ceph_manager:PG 15.8 is not active+clean 2023-08-09T13:52:05.838 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '15.8', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 114, 'state': 'creating+peering', 'last_fresh': '2023-08-09T13:51:59.162674+0000', 'last_change': '2023-08-09T13:51:59.162674+0000', 'last_active': '2023-08-09T13:51:59.130555+0000', 'last_peered': '2023-08-09T13:51:59.130555+0000', 'last_clean': '2023-08-09T13:51:59.130555+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-08-09T13:51:59.162674+0000', 'last_undegraded': '2023-08-09T13:51:59.162674+0000', 'last_fullsized': '2023-08-09T13:51:59.162674+0000', 'mapping_epoch': 114, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 114, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-08-09T13:51:59.130555+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-08-09T13:51:59.130555+0000', 'last_clean_scrub_stamp': '2023-08-09T13:51:59.130555+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 3], 'acting': [2, 3], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2023-08-09T13:52:05.838 INFO:tasks.cephfs.filesystem.ceph_manager:PG 15.9 is not active+clean 2023-08-09T13:52:05.838 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '15.9', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 114, 'state': 'creating+peering', 'last_fresh': '2023-08-09T13:51:59.157318+0000', 'last_change': '2023-08-09T13:51:59.157318+0000', 'last_active': '2023-08-09T13:51:59.130555+0000', 'last_peered': '2023-08-09T13:51:59.130555+0000', 'last_clean': '2023-08-09T13:51:59.130555+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-08-09T13:51:59.157318+0000', 'last_undegraded': '2023-08-09T13:51:59.157318+0000', 'last_fullsized': '2023-08-09T13:51:59.157318+0000', 'mapping_epoch': 114, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 114, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-08-09T13:51:59.130555+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-08-09T13:51:59.130555+0000', 'last_clean_scrub_stamp': '2023-08-09T13:51:59.130555+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [6, 1], 'acting': [6, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 6, 'acting_primary': 6, 'purged_snaps': []} 2023-08-09T13:52:05.839 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2023-08-09T13:52:06.081 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:52:06.075+0000 7fe8ebfff640 1 Processor -- start 2023-08-09T13:52:06.081 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:52:06.075+0000 7fe8ebfff640 1 -- start start 2023-08-09T13:52:06.081 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:52:06.076+0000 7fe8ebfff640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fe8ec0c8750 0x7fe8ec0ad340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:52:06.081 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:52:06.076+0000 7fe8ebfff640 1 -- --> v1:172.21.15.43:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe8ec0cbd90 con 0x7fe8ec0acdf0 2023-08-09T13:52:06.081 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:52:06.077+0000 7fe8ebfff640 1 -- --> v1:172.21.15.203:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe8ec0ca2b0 con 0x7fe8ec0c8b30 2023-08-09T13:52:06.082 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:52:06.077+0000 7fe8ebfff640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe8ec15e810 con 0x7fe8ec0c8750 2023-08-09T13:52:06.082 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:52:06.077+0000 7fe8eb7fe640 1 --1- >> v1:172.21.15.43:6789/0 conn(0x7fe8ec0acdf0 0x7fe8ec070f00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.43:6789/0 says I am v1:172.21.15.43:38094/0 (socket says 172.21.15.43:38094) 2023-08-09T13:52:06.082 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:52:06.077+0000 7fe8eb7fe640 1 -- 172.21.15.43:0/1083786905 learned_addr learned my addr 172.21.15.43:0/1083786905 (peer_addr_for_me v1:172.21.15.43:0/0) 2023-08-09T13:52:06.082 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:52:06.077+0000 7fe8eaffd640 1 --2- 172.21.15.43:0/1083786905 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fe8ec0c8750 0x7fe8ec0ad340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:52:06.082 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:52:06.078+0000 7fe8e9ffb640 1 -- 172.21.15.43:0/1083786905 <== mon.0 v1:172.21.15.43:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 486119458 0 0) 0x7fe8ec0cbd90 con 0x7fe8ec0acdf0 2023-08-09T13:52:06.082 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:52:06.078+0000 7fe8e9ffb640 1 -- 172.21.15.43:0/1083786905 --> v1:172.21.15.43:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fe8d4003520 con 0x7fe8ec0acdf0 2023-08-09T13:52:06.082 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:52:06.078+0000 7fe8e9ffb640 1 -- 172.21.15.43:0/1083786905 <== mon.0 v1:172.21.15.43:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3997553491 0 0) 0x7fe8d4003520 con 0x7fe8ec0acdf0 2023-08-09T13:52:06.082 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:52:06.078+0000 7fe8e9ffb640 1 -- 172.21.15.43:0/1083786905 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fe8ec0c8750 msgr2=0x7fe8ec0ad340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-09T13:52:06.083 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:52:06.078+0000 7fe8e9ffb640 1 --2- 172.21.15.43:0/1083786905 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fe8ec0c8750 0x7fe8ec0ad340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:52:06.083 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:52:06.078+0000 7fe8e9ffb640 1 -- 172.21.15.43:0/1083786905 >> v1:172.21.15.203:6789/0 conn(0x7fe8ec0c8b30 legacy=0x7fe8ec06f410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:52:06.083 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:52:06.078+0000 7fe8e9ffb640 1 -- 172.21.15.43:0/1083786905 --> v1:172.21.15.43:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe8ec0c9a20 con 0x7fe8ec0acdf0 2023-08-09T13:52:06.083 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:52:06.078+0000 7fe8e9ffb640 1 -- 172.21.15.43:0/1083786905 <== mon.0 v1:172.21.15.43:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1285150740 0 0) 0x7fe8e40019f0 con 0x7fe8ec0acdf0 2023-08-09T13:52:06.083 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:52:06.078+0000 7fe8e9ffb640 1 -- 172.21.15.43:0/1083786905 >> v1:172.21.15.43:6789/0 conn(0x7fe8ec0acdf0 legacy=0x7fe8ec070f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:52:06.083 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:52:06.078+0000 7fe8e9ffb640 1 --2- 172.21.15.43:0/1083786905 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fe8d40039a0 0x7fe8d4003d80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:52:06.083 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:52:06.078+0000 7fe8e9ffb640 1 --2- 172.21.15.43:0/1083786905 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fe8d4004300 0x7fe8d40047e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:52:06.084 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:52:06.078+0000 7fe8e9ffb640 1 --2- 172.21.15.43:0/1083786905 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fe8d4005080 0x7fe8d400b980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:52:06.084 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:52:06.078+0000 7fe8e9ffb640 1 -- 172.21.15.43:0/1083786905 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe8ec15e810 con 0x7fe8d4004300 2023-08-09T13:52:06.084 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:52:06.078+0000 7fe8e9ffb640 1 -- 172.21.15.43:0/1083786905 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe8d400bfa0 con 0x7fe8d4005080 2023-08-09T13:52:06.084 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:52:06.078+0000 7fe8e9ffb640 1 -- 172.21.15.43:0/1083786905 --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe8d400c160 con 0x7fe8d40039a0 2023-08-09T13:52:06.084 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:52:06.078+0000 7fe8ea7fc640 1 --2- 172.21.15.43:0/1083786905 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fe8d4004300 0x7fe8d40047e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:52:06.084 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:52:06.079+0000 7fe8eb7fe640 1 --2- 172.21.15.43:0/1083786905 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fe8d4005080 0x7fe8d400b980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:52:06.084 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:52:06.079+0000 7fe8eaffd640 1 --2- 172.21.15.43:0/1083786905 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fe8d40039a0 0x7fe8d4003d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:52:06.085 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:52:06.079+0000 7fe8ea7fc640 1 -- 172.21.15.43:0/1083786905 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fe8d40039a0 msgr2=0x7fe8d4003d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-09T13:52:06.085 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:52:06.079+0000 7fe8ea7fc640 1 --2- 172.21.15.43:0/1083786905 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fe8d40039a0 0x7fe8d4003d80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:52:06.085 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:52:06.079+0000 7fe8ea7fc640 1 -- 172.21.15.43:0/1083786905 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fe8d4005080 msgr2=0x7fe8d400b980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-09T13:52:06.085 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:52:06.079+0000 7fe8ea7fc640 1 --2- 172.21.15.43:0/1083786905 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fe8d4005080 0x7fe8d400b980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:52:06.085 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:52:06.079+0000 7fe8ea7fc640 1 -- 172.21.15.43:0/1083786905 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe8d4003520 con 0x7fe8d4004300 2023-08-09T13:52:06.085 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:52:06.079+0000 7fe8eaffd640 1 --2- 172.21.15.43:0/1083786905 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fe8d40039a0 0x7fe8d4003d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-08-09T13:52:06.085 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:52:06.079+0000 7fe8eb7fe640 1 --2- 172.21.15.43:0/1083786905 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fe8d4005080 0x7fe8d400b980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-08-09T13:52:06.087 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:52:06.080+0000 7fe8ea7fc640 1 --2- 172.21.15.43:0/1083786905 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fe8d4004300 0x7fe8d40047e0 secure :-1 s=READY pgs=1354 cs=0 l=1 rev1=1 crypto rx=0x7fe8e0001860 tx=0x7fe8d40057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4e459e80c03ce987 server_cookie=0 in_seq=0 out_seq=0 2023-08-09T13:52:06.087 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:52:06.081+0000 7fe8e9ffb640 1 -- 172.21.15.43:0/1083786905 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe8e0033410 con 0x7fe8d4004300 2023-08-09T13:52:06.087 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:52:06.081+0000 7fe8e9ffb640 1 -- 172.21.15.43:0/1083786905 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe8e0033590 con 0x7fe8d4004300 2023-08-09T13:52:06.088 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:52:06.081+0000 7fe8e9ffb640 1 -- 172.21.15.43:0/1083786905 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe8e0033940 con 0x7fe8d4004300 2023-08-09T13:52:06.088 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:52:06.081+0000 7fe8ebfff640 1 -- 172.21.15.43:0/1083786905 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fe8d4004300 msgr2=0x7fe8d40047e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:52:06.088 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:52:06.081+0000 7fe8ebfff640 1 --2- 172.21.15.43:0/1083786905 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fe8d4004300 0x7fe8d40047e0 secure :-1 s=READY pgs=1354 cs=0 l=1 rev1=1 crypto rx=0x7fe8e0001860 tx=0x7fe8d40057f0 comp rx=0 tx=0).stop 2023-08-09T13:52:06.088 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:52:06.081+0000 7fe8eb7fe640 1 -- 172.21.15.43:0/1083786905 reap_dead start 2023-08-09T13:52:06.088 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:52:06.081+0000 7fe8ebfff640 1 -- 172.21.15.43:0/1083786905 shutdown_connections 2023-08-09T13:52:06.088 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:52:06.081+0000 7fe8ebfff640 1 -- 172.21.15.43:0/1083786905 >> 172.21.15.43:0/1083786905 conn(0x7fe8ec0cde90 msgr2=0x7fe8ec0ce2a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-09T13:52:06.089 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:52:06.081+0000 7fe8ebfff640 1 -- 172.21.15.43:0/1083786905 shutdown_connections 2023-08-09T13:52:06.089 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:52:06.081+0000 7fe8ebfff640 1 -- 172.21.15.43:0/1083786905 wait complete. 2023-08-09T13:52:06.089 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:52:06.083+0000 7fe8ebfff640 1 Processor -- start 2023-08-09T13:52:06.089 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:52:06.083+0000 7fe8ebfff640 1 -- start start 2023-08-09T13:52:06.089 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:52:06.084+0000 7fe8ebfff640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fe8d4004300 0x7fe8ec14c280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:52:06.089 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:52:06.084+0000 7fe8ebfff640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fe8ec0c3eb0 0x7fe8ec0c4330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:52:06.089 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:52:06.084+0000 7fe8ebfff640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fe8ec0b0ab0 0x7fe8ec0b0f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:52:06.089 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:52:06.084+0000 7fe8ebfff640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe8ec0c9f70 con 0x7fe8d4004300 2023-08-09T13:52:06.090 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:52:06.084+0000 7fe8ebfff640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe8ec0c4870 con 0x7fe8ec0c3eb0 2023-08-09T13:52:06.090 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:52:06.084+0000 7fe8ebfff640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe8ec0c1af0 con 0x7fe8ec0b0ab0 2023-08-09T13:52:06.090 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:52:06.085+0000 7fe8eaffd640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fe8d4004300 0x7fe8ec14c280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:52:06.090 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:52:06.085+0000 7fe8eaffd640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fe8d4004300 0x7fe8ec14c280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:54328/0 (socket says 172.21.15.43:54328) 2023-08-09T13:52:06.090 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:52:06.085+0000 7fe8eaffd640 1 -- 172.21.15.43:0/1595225953 learned_addr learned my addr 172.21.15.43:0/1595225953 (peer_addr_for_me v2:172.21.15.43:0/0) 2023-08-09T13:52:06.091 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:52:06.085+0000 7fe8eb7fe640 1 --2- 172.21.15.43:0/1595225953 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fe8ec0b0ab0 0x7fe8ec0b0f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:52:06.091 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:52:06.085+0000 7fe8ea7fc640 1 --2- 172.21.15.43:0/1595225953 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fe8ec0c3eb0 0x7fe8ec0c4330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:52:06.091 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:52:06.085+0000 7fe8eaffd640 1 -- 172.21.15.43:0/1595225953 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fe8ec0b0ab0 msgr2=0x7fe8ec0b0f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:52:06.091 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:52:06.085+0000 7fe8eaffd640 1 --2- 172.21.15.43:0/1595225953 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fe8ec0b0ab0 0x7fe8ec0b0f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:52:06.091 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:52:06.085+0000 7fe8eaffd640 1 -- 172.21.15.43:0/1595225953 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fe8ec0c3eb0 msgr2=0x7fe8ec0c4330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:52:06.091 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:52:06.085+0000 7fe8eaffd640 1 --2- 172.21.15.43:0/1595225953 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fe8ec0c3eb0 0x7fe8ec0c4330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:52:06.092 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:52:06.085+0000 7fe8eaffd640 1 -- 172.21.15.43:0/1595225953 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe8ec0b17e0 con 0x7fe8d4004300 2023-08-09T13:52:06.092 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:52:06.086+0000 7fe8eaffd640 1 --2- 172.21.15.43:0/1595225953 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fe8d4004300 0x7fe8ec14c280 secure :-1 s=READY pgs=1355 cs=0 l=1 rev1=1 crypto rx=0x7fe8dc0107f0 tx=0x7fe8dc010cb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-09T13:52:06.093 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:52:06.086+0000 7fe8d37fe640 1 -- 172.21.15.43:0/1595225953 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe8dc00e950 con 0x7fe8d4004300 2023-08-09T13:52:06.093 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:52:06.086+0000 7fe8d37fe640 1 -- 172.21.15.43:0/1595225953 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe8dc00eab0 con 0x7fe8d4004300 2023-08-09T13:52:06.093 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:52:06.086+0000 7fe8d37fe640 1 -- 172.21.15.43:0/1595225953 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe8dc00ee20 con 0x7fe8d4004300 2023-08-09T13:52:06.093 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:52:06.086+0000 7fe8ebfff640 1 -- 172.21.15.43:0/1595225953 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe8ec0b1ac0 con 0x7fe8d4004300 2023-08-09T13:52:06.093 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:52:06.086+0000 7fe8ebfff640 1 -- 172.21.15.43:0/1595225953 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe8ec0b2080 con 0x7fe8d4004300 2023-08-09T13:52:06.094 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:52:06.087+0000 7fe8ebfff640 1 -- 172.21.15.43:0/1595225953 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe8ec0c9230 con 0x7fe8d4004300 2023-08-09T13:52:06.094 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:52:06.088+0000 7fe8d37fe640 1 -- 172.21.15.43:0/1595225953 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 45) v1 ==== 88443+0+0 (secure 0 0 0) 0x7fe8dc01c400 con 0x7fe8d4004300 2023-08-09T13:52:06.094 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:52:06.089+0000 7fe8d37fe640 1 --2- 172.21.15.43:0/1595225953 >> [v2:172.21.15.203:6832/1317108840,v1:172.21.15.203:6833/1317108840] conn(0x7fe8d806a170 0x7fe8d806c630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:52:06.094 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:52:06.089+0000 7fe8d37fe640 1 -- 172.21.15.43:0/1595225953 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(116..116 src has 1..116) v4 ==== 6426+0+0 (secure 0 0 0) 0x7fe8dc088cb0 con 0x7fe8d4004300 2023-08-09T13:52:06.094 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:52:06.090+0000 7fe8ea7fc640 1 --2- 172.21.15.43:0/1595225953 >> [v2:172.21.15.203:6832/1317108840,v1:172.21.15.203:6833/1317108840] conn(0x7fe8d806a170 0x7fe8d806c630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:52:06.094 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:52:06.091+0000 7fe8ea7fc640 1 --2- 172.21.15.43:0/1595225953 >> [v2:172.21.15.203:6832/1317108840,v1:172.21.15.203:6833/1317108840] conn(0x7fe8d806a170 0x7fe8d806c630 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7fe8e0035000 tx=0x7fe8e0009030 comp rx=0 tx=0).ready entity=mgr.6619 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-09T13:52:06.095 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:52:06.094+0000 7fe8d37fe640 1 -- 172.21.15.43:0/1595225953 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178066 (secure 0 0 0) 0x7fe8dc0541c0 con 0x7fe8d4004300 2023-08-09T13:52:06.360 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:52:06.359+0000 7fe8ebfff640 1 -- 172.21.15.43:0/1595225953 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "status", "format": "json"} v 0) v1 -- 0x7fe8ec0739f0 con 0x7fe8d4004300 2023-08-09T13:52:06.361 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:52:06.360+0000 7fe8d37fe640 1 -- 172.21.15.43:0/1595225953 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json"}]=0 v0) v1 ==== 72+0+3232 (secure 0 0 0) 0x7fe8dc0577e0 con 0x7fe8d4004300 2023-08-09T13:52:06.362 INFO:teuthology.orchestra.run.smithi043.stdout: 2023-08-09T13:52:06.362 INFO:teuthology.orchestra.run.smithi043.stdout:{"fsid":"daac87ee-1337-4250-a955-aea4e408a39b","health":{"status":"HEALTH_OK","checks":{},"mutes":[]},"election_epoch":4,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":1104,"monmap":{"epoch":1,"min_mon_release_name":"reef","num_mons":3},"osdmap":{"epoch":116,"num_osds":8,"num_up_osds":8,"osd_up_since":1691588033,"num_in_osds":8,"osd_in_since":1691588025,"num_remapped_pgs":0},"pgmap":{"pgs_by_state":[{"state_name":"creating+peering","count":64},{"state_name":"active+clean","count":41}],"num_pgs":105,"num_pools":4,"num_objects":2,"data_bytes":590368,"bytes_used":315105280,"bytes_avail":772779008000,"bytes_total":773094113280,"inactive_pgs_ratio":0.60952383279800415},"fsmap":{"epoch":52,"id":6,"up":1,"in":1,"max":1,"by_rank":[{"filesystem_id":6,"rank":0,"name":"c","status":"up:active","gid":8040}],"up:standby":3},"mgrmap":{"available":true,"num_standbys":1,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":12,"modified":"2023-08-09T13:51:44.565148+0000","services":{"mds":{"daemons":{"summary":"","b":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}},"c":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}},"d":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}}}},"mgr":{"daemons":{"summary":"","x":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}},"y":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}}}},"mon":{"daemons":{"summary":"","a":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}},"b":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}},"c":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}}}},"osd":{"daemons":{"summary":"","0":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}},"1":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}},"2":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}},"3":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}},"4":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}},"5":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}},"6":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}},"7":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}}}}}},"progress_events":{}} 2023-08-09T13:52:06.365 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:52:06.362+0000 7fe8ebfff640 1 -- 172.21.15.43:0/1595225953 >> [v2:172.21.15.203:6832/1317108840,v1:172.21.15.203:6833/1317108840] conn(0x7fe8d806a170 msgr2=0x7fe8d806c630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:52:06.365 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:52:06.362+0000 7fe8ebfff640 1 --2- 172.21.15.43:0/1595225953 >> [v2:172.21.15.203:6832/1317108840,v1:172.21.15.203:6833/1317108840] conn(0x7fe8d806a170 0x7fe8d806c630 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7fe8e0035000 tx=0x7fe8e0009030 comp rx=0 tx=0).stop 2023-08-09T13:52:06.365 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:52:06.363+0000 7fe8ebfff640 1 -- 172.21.15.43:0/1595225953 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fe8d4004300 msgr2=0x7fe8ec14c280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:52:06.365 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:52:06.363+0000 7fe8ebfff640 1 --2- 172.21.15.43:0/1595225953 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fe8d4004300 0x7fe8ec14c280 secure :-1 s=READY pgs=1355 cs=0 l=1 rev1=1 crypto rx=0x7fe8dc0107f0 tx=0x7fe8dc010cb0 comp rx=0 tx=0).stop 2023-08-09T13:52:06.365 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:52:06.363+0000 7fe8ebfff640 1 -- 172.21.15.43:0/1595225953 shutdown_connections 2023-08-09T13:52:06.365 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:52:06.363+0000 7fe8ebfff640 1 --2- 172.21.15.43:0/1595225953 >> [v2:172.21.15.203:6832/1317108840,v1:172.21.15.203:6833/1317108840] conn(0x7fe8d806a170 0x7fe8d806c630 unknown :-1 s=CLOSED pgs=68 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:52:06.366 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:52:06.363+0000 7fe8ebfff640 1 --2- 172.21.15.43:0/1595225953 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fe8ec0b0ab0 0x7fe8ec0b0f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:52:06.366 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:52:06.363+0000 7fe8ebfff640 1 --2- 172.21.15.43:0/1595225953 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fe8ec0c3eb0 0x7fe8ec0c4330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:52:06.366 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:52:06.363+0000 7fe8ebfff640 1 --2- 172.21.15.43:0/1595225953 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fe8d4004300 0x7fe8ec14c280 unknown :-1 s=CLOSED pgs=1355 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:52:06.366 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:52:06.363+0000 7fe8ebfff640 1 -- 172.21.15.43:0/1595225953 >> 172.21.15.43:0/1595225953 conn(0x7fe8ec0cde90 msgr2=0x7fe8ec06c320 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-09T13:52:06.366 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:52:06.363+0000 7fe8ebfff640 1 -- 172.21.15.43:0/1595225953 shutdown_connections 2023-08-09T13:52:06.366 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:52:06.363+0000 7fe8ebfff640 1 -- 172.21.15.43:0/1595225953 wait complete. 2023-08-09T13:52:06.384 INFO:tasks.cephfs.filesystem.ceph_manager:no progress seen, keeping timeout for now 2023-08-09T13:52:06.385 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-08-09T13:52:06.616 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:52:06.614+0000 7fd25d737640 1 Processor -- start 2023-08-09T13:52:06.616 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:52:06.614+0000 7fd25d737640 1 -- start start 2023-08-09T13:52:06.617 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:52:06.616+0000 7fd25d737640 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7fd2580c8750 0x7fd2580ad340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:52:06.617 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:52:06.616+0000 7fd25d737640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7fd2580c8b30 0x7fd25806f410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:52:06.618 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:52:06.616+0000 7fd25d737640 1 -- --> v1:172.21.15.203:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd2580cbd90 con 0x7fd2580acdf0 2023-08-09T13:52:06.618 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:52:06.616+0000 7fd25d737640 1 -- --> v2:172.21.15.43:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd25815e810 con 0x7fd2580c8b30 2023-08-09T13:52:06.618 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:52:06.616+0000 7fd25d737640 1 -- --> v2:172.21.15.203:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd2580c9f70 con 0x7fd2580c8750 2023-08-09T13:52:06.619 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:52:06.616+0000 7fd2577fe640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7fd2580c8b30 0x7fd25806f410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:52:06.619 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:52:06.617+0000 7fd2577fe640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7fd2580c8b30 0x7fd25806f410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:54332/0 (socket says 172.21.15.43:54332) 2023-08-09T13:52:06.619 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:52:06.617+0000 7fd2577fe640 1 -- 172.21.15.43:0/1502348589 learned_addr learned my addr 172.21.15.43:0/1502348589 (peer_addr_for_me v2:172.21.15.43:0/0) 2023-08-09T13:52:06.619 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:52:06.617+0000 7fd257fff640 1 --2- 172.21.15.43:0/1502348589 >> v2:172.21.15.203:3300/0 conn(0x7fd2580c8750 0x7fd2580ad340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:52:06.619 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:52:06.617+0000 7fd2577fe640 1 -- 172.21.15.43:0/1502348589 >> v2:172.21.15.203:3300/0 conn(0x7fd2580c8750 msgr2=0x7fd2580ad340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-09T13:52:06.619 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:52:06.617+0000 7fd2577fe640 1 --2- 172.21.15.43:0/1502348589 >> v2:172.21.15.203:3300/0 conn(0x7fd2580c8750 0x7fd2580ad340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:52:06.620 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:52:06.617+0000 7fd2577fe640 1 -- 172.21.15.43:0/1502348589 >> v1:172.21.15.203:6789/0 conn(0x7fd2580acdf0 legacy=0x7fd258070f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-09T13:52:06.620 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:52:06.617+0000 7fd2577fe640 1 -- 172.21.15.43:0/1502348589 --> v2:172.21.15.43:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd2580ca2b0 con 0x7fd2580c8b30 2023-08-09T13:52:06.620 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:52:06.618+0000 7fd2577fe640 1 --2- 172.21.15.43:0/1502348589 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fd2580c8b30 0x7fd25806f410 secure :-1 s=READY pgs=1356 cs=0 l=1 rev1=1 crypto rx=0x7fd2580cb9a0 tx=0x7fd24c02b910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=df54318ca3b4ef54 server_cookie=0 in_seq=0 out_seq=0 2023-08-09T13:52:06.621 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:52:06.619+0000 7fd256ffd640 1 -- 172.21.15.43:0/1502348589 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd24c031870 con 0x7fd2580c8b30 2023-08-09T13:52:06.621 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:52:06.619+0000 7fd256ffd640 1 -- 172.21.15.43:0/1502348589 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd24c0319d0 con 0x7fd2580c8b30 2023-08-09T13:52:06.621 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:52:06.619+0000 7fd256ffd640 1 -- 172.21.15.43:0/1502348589 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd24c031d60 con 0x7fd2580c8b30 2023-08-09T13:52:06.621 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:52:06.619+0000 7fd25d737640 1 -- 172.21.15.43:0/1502348589 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fd2580c8b30 msgr2=0x7fd25806f410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:52:06.621 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:52:06.619+0000 7fd25d737640 1 --2- 172.21.15.43:0/1502348589 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fd2580c8b30 0x7fd25806f410 secure :-1 s=READY pgs=1356 cs=0 l=1 rev1=1 crypto rx=0x7fd2580cb9a0 tx=0x7fd24c02b910 comp rx=0 tx=0).stop 2023-08-09T13:52:06.622 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:52:06.619+0000 7fd25d737640 1 -- 172.21.15.43:0/1502348589 shutdown_connections 2023-08-09T13:52:06.622 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:52:06.619+0000 7fd25d737640 1 --2- 172.21.15.43:0/1502348589 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fd2580c8b30 0x7fd25806f410 unknown :-1 s=CLOSED pgs=1356 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:52:06.622 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:52:06.619+0000 7fd25d737640 1 --2- 172.21.15.43:0/1502348589 >> v2:172.21.15.203:3300/0 conn(0x7fd2580c8750 0x7fd2580ad340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:52:06.622 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:52:06.619+0000 7fd25d737640 1 -- 172.21.15.43:0/1502348589 >> 172.21.15.43:0/1502348589 conn(0x7fd2580cde90 msgr2=0x7fd2580ce2a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-09T13:52:06.622 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:52:06.620+0000 7fd25d737640 1 -- 172.21.15.43:0/1502348589 shutdown_connections 2023-08-09T13:52:06.623 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:52:06.620+0000 7fd25d737640 1 -- 172.21.15.43:0/1502348589 wait complete. 2023-08-09T13:52:06.623 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:52:06.621+0000 7fd25d737640 1 Processor -- start 2023-08-09T13:52:06.623 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:52:06.621+0000 7fd25d737640 1 -- start start 2023-08-09T13:52:06.624 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:52:06.622+0000 7fd25d737640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fd2580acdf0 0x7fd2580c7410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:52:06.625 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:52:06.622+0000 7fd25d737640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fd2580c8750 0x7fd2580c40a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:52:06.625 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:52:06.622+0000 7fd25d737640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fd2580c8b30 0x7fd2580c19c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:52:06.625 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:52:06.622+0000 7fd25d737640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd2580c69c0 con 0x7fd2580c8b30 2023-08-09T13:52:06.625 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:52:06.622+0000 7fd25d737640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd2580b0df0 con 0x7fd2580c8750 2023-08-09T13:52:06.625 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:52:06.622+0000 7fd25d737640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd2580c7950 con 0x7fd2580acdf0 2023-08-09T13:52:06.626 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:52:06.623+0000 7fd25cf36640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fd2580c8b30 0x7fd2580c19c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:52:06.626 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:52:06.623+0000 7fd25cf36640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fd2580c8b30 0x7fd2580c19c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:54346/0 (socket says 172.21.15.43:54346) 2023-08-09T13:52:06.626 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:52:06.623+0000 7fd25cf36640 1 -- 172.21.15.43:0/2280894015 learned_addr learned my addr 172.21.15.43:0/2280894015 (peer_addr_for_me v2:172.21.15.43:0/0) 2023-08-09T13:52:06.626 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:52:06.623+0000 7fd257fff640 1 --2- 172.21.15.43:0/2280894015 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fd2580acdf0 0x7fd2580c7410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:52:06.626 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:52:06.623+0000 7fd2577fe640 1 --2- 172.21.15.43:0/2280894015 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fd2580c8750 0x7fd2580c40a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:52:06.626 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:52:06.623+0000 7fd25cf36640 1 -- 172.21.15.43:0/2280894015 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fd2580acdf0 msgr2=0x7fd2580c7410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:52:06.626 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:52:06.623+0000 7fd25cf36640 1 --2- 172.21.15.43:0/2280894015 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fd2580acdf0 0x7fd2580c7410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:52:06.626 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:52:06.623+0000 7fd25cf36640 1 -- 172.21.15.43:0/2280894015 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fd2580c8750 msgr2=0x7fd2580c40a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:52:06.627 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:52:06.623+0000 7fd25cf36640 1 --2- 172.21.15.43:0/2280894015 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fd2580c8750 0x7fd2580c40a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:52:06.627 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:52:06.623+0000 7fd25cf36640 1 -- 172.21.15.43:0/2280894015 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd24c0094d0 con 0x7fd2580c8b30 2023-08-09T13:52:06.627 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:52:06.623+0000 7fd2577fe640 1 --2- 172.21.15.43:0/2280894015 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fd2580c8750 0x7fd2580c40a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-08-09T13:52:06.628 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:52:06.624+0000 7fd25cf36640 1 --2- 172.21.15.43:0/2280894015 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fd2580c8b30 0x7fd2580c19c0 secure :-1 s=READY pgs=1357 cs=0 l=1 rev1=1 crypto rx=0x7fd25000c780 tx=0x7fd25000cc40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-09T13:52:06.628 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:52:06.624+0000 7fd254ff9640 1 -- 172.21.15.43:0/2280894015 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd250004b60 con 0x7fd2580c8b30 2023-08-09T13:52:06.628 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:52:06.624+0000 7fd254ff9640 1 -- 172.21.15.43:0/2280894015 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd25000ce90 con 0x7fd2580c8b30 2023-08-09T13:52:06.628 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:52:06.624+0000 7fd25d737640 1 -- 172.21.15.43:0/2280894015 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd25815b2f0 con 0x7fd2580c8b30 2023-08-09T13:52:06.628 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:52:06.624+0000 7fd254ff9640 1 -- 172.21.15.43:0/2280894015 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd25000a850 con 0x7fd2580c8b30 2023-08-09T13:52:06.629 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:52:06.624+0000 7fd25d737640 1 -- 172.21.15.43:0/2280894015 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd2580c6c10 con 0x7fd2580c8b30 2023-08-09T13:52:06.629 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:52:06.625+0000 7fd25d737640 1 -- 172.21.15.43:0/2280894015 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd25815b810 con 0x7fd2580c8b30 2023-08-09T13:52:06.629 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:52:06.626+0000 7fd254ff9640 1 -- 172.21.15.43:0/2280894015 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 45) v1 ==== 88443+0+0 (secure 0 0 0) 0x7fd25000a9b0 con 0x7fd2580c8b30 2023-08-09T13:52:06.630 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:52:06.627+0000 7fd254ff9640 1 --2- 172.21.15.43:0/2280894015 >> [v2:172.21.15.203:6832/1317108840,v1:172.21.15.203:6833/1317108840] conn(0x7fd24406a170 0x7fd24406c630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:52:06.630 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:52:06.627+0000 7fd254ff9640 1 -- 172.21.15.43:0/2280894015 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(116..116 src has 1..116) v4 ==== 6426+0+0 (secure 0 0 0) 0x7fd250088a60 con 0x7fd2580c8b30 2023-08-09T13:52:06.630 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:52:06.628+0000 7fd257fff640 1 --2- 172.21.15.43:0/2280894015 >> [v2:172.21.15.203:6832/1317108840,v1:172.21.15.203:6833/1317108840] conn(0x7fd24406a170 0x7fd24406c630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:52:06.630 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:52:06.629+0000 7fd257fff640 1 --2- 172.21.15.43:0/2280894015 >> [v2:172.21.15.203:6832/1317108840,v1:172.21.15.203:6833/1317108840] conn(0x7fd24406a170 0x7fd24406c630 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7fd248006fd0 tx=0x7fd248008040 comp rx=0 tx=0).ready entity=mgr.6619 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-09T13:52:06.633 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:52:06.632+0000 7fd254ff9640 1 -- 172.21.15.43:0/2280894015 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178066 (secure 0 0 0) 0x7fd250053eb0 con 0x7fd2580c8b30 2023-08-09T13:52:06.827 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:52:06.825+0000 7fd25d737640 1 -- 172.21.15.43:0/2280894015 --> [v2:172.21.15.203:6832/1317108840,v1:172.21.15.203:6833/1317108840] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7fd2580c9a20 con 0x7fd24406a170 2023-08-09T13:52:06.851 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:52:06.849+0000 7fd254ff9640 1 -- 172.21.15.43:0/2280894015 <== mgr.6619 v2:172.21.15.203:6832/1317108840 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+309413 (secure 0 0 0) 0x7fd24800f830 con 0x7fd24406a170 2023-08-09T13:52:06.854 INFO:teuthology.orchestra.run.smithi043.stdout: 2023-08-09T13:52:06.856 INFO:teuthology.orchestra.run.smithi043.stderr:dumped all 2023-08-09T13:52:06.858 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:52:06.854+0000 7fd25d737640 1 -- 172.21.15.43:0/2280894015 >> [v2:172.21.15.203:6832/1317108840,v1:172.21.15.203:6833/1317108840] conn(0x7fd24406a170 msgr2=0x7fd24406c630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:52:06.858 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:52:06.854+0000 7fd25d737640 1 --2- 172.21.15.43:0/2280894015 >> [v2:172.21.15.203:6832/1317108840,v1:172.21.15.203:6833/1317108840] conn(0x7fd24406a170 0x7fd24406c630 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7fd248006fd0 tx=0x7fd248008040 comp rx=0 tx=0).stop 2023-08-09T13:52:06.858 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:52:06.855+0000 7fd25d737640 1 -- 172.21.15.43:0/2280894015 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fd2580c8b30 msgr2=0x7fd2580c19c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:52:06.859 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:52:06.855+0000 7fd25d737640 1 --2- 172.21.15.43:0/2280894015 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fd2580c8b30 0x7fd2580c19c0 secure :-1 s=READY pgs=1357 cs=0 l=1 rev1=1 crypto rx=0x7fd25000c780 tx=0x7fd25000cc40 comp rx=0 tx=0).stop 2023-08-09T13:52:06.859 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:52:06.855+0000 7fd25d737640 1 -- 172.21.15.43:0/2280894015 shutdown_connections 2023-08-09T13:52:06.859 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:52:06.855+0000 7fd25d737640 1 --2- 172.21.15.43:0/2280894015 >> [v2:172.21.15.203:6832/1317108840,v1:172.21.15.203:6833/1317108840] conn(0x7fd24406a170 0x7fd24406c630 unknown :-1 s=CLOSED pgs=69 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:52:06.859 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:52:06.855+0000 7fd25d737640 1 --2- 172.21.15.43:0/2280894015 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fd2580c8b30 0x7fd2580c19c0 unknown :-1 s=CLOSED pgs=1357 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:52:06.859 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:52:06.855+0000 7fd25d737640 1 --2- 172.21.15.43:0/2280894015 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fd2580c8750 0x7fd2580c40a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:52:06.859 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:52:06.855+0000 7fd25d737640 1 --2- 172.21.15.43:0/2280894015 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fd2580acdf0 0x7fd2580c7410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:52:06.859 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:52:06.855+0000 7fd25d737640 1 -- 172.21.15.43:0/2280894015 >> 172.21.15.43:0/2280894015 conn(0x7fd2580cde90 msgr2=0x7fd2580aad80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-09T13:52:06.859 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:52:06.855+0000 7fd25d737640 1 -- 172.21.15.43:0/2280894015 shutdown_connections 2023-08-09T13:52:06.860 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:52:06.855+0000 7fd25d737640 1 -- 172.21.15.43:0/2280894015 wait complete. 2023-08-09T13:52:06.877 INFO:teuthology.orchestra.run.smithi043.stdout:{"pg_ready":true,"pg_map":{"version":28,"stamp":"2023-08-09T13:52:06.572267+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":715,"num_read_kb":1418,"num_write":328,"num_write_kb":2430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":225,"ondisk_log_size":225,"up":210,"acting":210,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":210,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":308040,"kb_used_data":12384,"kb_used_omap":13,"kb_used_meta":295538,"kb_avail":754666680,"statfs":{"total":773094113280,"available":772778680320,"internally_reserved":0,"allocated":12681216,"data_stored":7282986,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":14248,"internal_metadata":302631000},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":32,"apply_latency_ms":32,"commit_latency_ns":32000000,"apply_latency_ns":32000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"7.430032"},"pg_stats":[{"pgid":"15.3f","version":"0'0","reported_seq":11,"reported_epoch":116,"state":"active+clean","last_fresh":"2023-08-09T13:52:01.220181+0000","last_change":"2023-08-09T13:52:00.224715+0000","last_active":"2023-08-09T13:52:01.220181+0000","last_peered":"2023-08-09T13:52:01.220181+0000","last_clean":"2023-08-09T13:52:01.220181+0000","last_became_active":"2023-08-09T13:52:00.224404+0000","last_became_peered":"2023-08-09T13:52:00.224404+0000","last_unstale":"2023-08-09T13:52:01.220181+0000","last_undegraded":"2023-08-09T13:52:01.220181+0000","last_fullsized":"2023-08-09T13:52:01.220181+0000","mapping_epoch":114,"log_start":"0'0","ondisk_log_start":"0'0","created":114,"last_epoch_clean":115,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:51:59.130555+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:51:59.130555+0000","last_clean_scrub_stamp":"2023-08-09T13:51:59.130555+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-11T00:54:55.831419+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"15.3e","version":"0'0","reported_seq":11,"reported_epoch":116,"state":"active+clean","last_fresh":"2023-08-09T13:52:02.227521+0000","last_change":"2023-08-09T13:52:00.218198+0000","last_active":"2023-08-09T13:52:02.227521+0000","last_peered":"2023-08-09T13:52:02.227521+0000","last_clean":"2023-08-09T13:52:02.227521+0000","last_became_active":"2023-08-09T13:52:00.217280+0000","last_became_peered":"2023-08-09T13:52:00.217280+0000","last_unstale":"2023-08-09T13:52:02.227521+0000","last_undegraded":"2023-08-09T13:52:02.227521+0000","last_fullsized":"2023-08-09T13:52:02.227521+0000","mapping_epoch":114,"log_start":"0'0","ondisk_log_start":"0'0","created":114,"last_epoch_clean":115,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:51:59.130555+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:51:59.130555+0000","last_clean_scrub_stamp":"2023-08-09T13:51:59.130555+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T16:40:23.268291+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.3d","version":"0'0","reported_seq":11,"reported_epoch":116,"state":"active+clean","last_fresh":"2023-08-09T13:52:01.220028+0000","last_change":"2023-08-09T13:52:00.224291+0000","last_active":"2023-08-09T13:52:01.220028+0000","last_peered":"2023-08-09T13:52:01.220028+0000","last_clean":"2023-08-09T13:52:01.220028+0000","last_became_active":"2023-08-09T13:52:00.223948+0000","last_became_peered":"2023-08-09T13:52:00.223948+0000","last_unstale":"2023-08-09T13:52:01.220028+0000","last_undegraded":"2023-08-09T13:52:01.220028+0000","last_fullsized":"2023-08-09T13:52:01.220028+0000","mapping_epoch":114,"log_start":"0'0","ondisk_log_start":"0'0","created":114,"last_epoch_clean":115,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:51:59.130555+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:51:59.130555+0000","last_clean_scrub_stamp":"2023-08-09T13:51:59.130555+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T16:56:55.245269+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"15.3c","version":"0'0","reported_seq":11,"reported_epoch":116,"state":"active+clean","last_fresh":"2023-08-09T13:52:01.221601+0000","last_change":"2023-08-09T13:52:00.211918+0000","last_active":"2023-08-09T13:52:01.221601+0000","last_peered":"2023-08-09T13:52:01.221601+0000","last_clean":"2023-08-09T13:52:01.221601+0000","last_became_active":"2023-08-09T13:52:00.210972+0000","last_became_peered":"2023-08-09T13:52:00.210972+0000","last_unstale":"2023-08-09T13:52:01.221601+0000","last_undegraded":"2023-08-09T13:52:01.221601+0000","last_fullsized":"2023-08-09T13:52:01.221601+0000","mapping_epoch":114,"log_start":"0'0","ondisk_log_start":"0'0","created":114,"last_epoch_clean":115,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:51:59.130555+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:51:59.130555+0000","last_clean_scrub_stamp":"2023-08-09T13:51:59.130555+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T23:44:30.232510+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"15.3b","version":"0'0","reported_seq":11,"reported_epoch":116,"state":"active+clean","last_fresh":"2023-08-09T13:52:02.229246+0000","last_change":"2023-08-09T13:52:00.215126+0000","last_active":"2023-08-09T13:52:02.229246+0000","last_peered":"2023-08-09T13:52:02.229246+0000","last_clean":"2023-08-09T13:52:02.229246+0000","last_became_active":"2023-08-09T13:52:00.214737+0000","last_became_peered":"2023-08-09T13:52:00.214737+0000","last_unstale":"2023-08-09T13:52:02.229246+0000","last_undegraded":"2023-08-09T13:52:02.229246+0000","last_fullsized":"2023-08-09T13:52:02.229246+0000","mapping_epoch":114,"log_start":"0'0","ondisk_log_start":"0'0","created":114,"last_epoch_clean":115,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:51:59.130555+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:51:59.130555+0000","last_clean_scrub_stamp":"2023-08-09T13:51:59.130555+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T14:03:42.929353+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.3a","version":"0'0","reported_seq":11,"reported_epoch":116,"state":"active+clean","last_fresh":"2023-08-09T13:52:02.228024+0000","last_change":"2023-08-09T13:52:00.226139+0000","last_active":"2023-08-09T13:52:02.228024+0000","last_peered":"2023-08-09T13:52:02.228024+0000","last_clean":"2023-08-09T13:52:02.228024+0000","last_became_active":"2023-08-09T13:52:00.225854+0000","last_became_peered":"2023-08-09T13:52:00.225854+0000","last_unstale":"2023-08-09T13:52:02.228024+0000","last_undegraded":"2023-08-09T13:52:02.228024+0000","last_fullsized":"2023-08-09T13:52:02.228024+0000","mapping_epoch":114,"log_start":"0'0","ondisk_log_start":"0'0","created":114,"last_epoch_clean":115,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:51:59.130555+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:51:59.130555+0000","last_clean_scrub_stamp":"2023-08-09T13:51:59.130555+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-11T00:12:54.873335+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"15.39","version":"0'0","reported_seq":11,"reported_epoch":116,"state":"active+clean","last_fresh":"2023-08-09T13:52:02.226730+0000","last_change":"2023-08-09T13:52:00.220933+0000","last_active":"2023-08-09T13:52:02.226730+0000","last_peered":"2023-08-09T13:52:02.226730+0000","last_clean":"2023-08-09T13:52:02.226730+0000","last_became_active":"2023-08-09T13:52:00.220647+0000","last_became_peered":"2023-08-09T13:52:00.220647+0000","last_unstale":"2023-08-09T13:52:02.226730+0000","last_undegraded":"2023-08-09T13:52:02.226730+0000","last_fullsized":"2023-08-09T13:52:02.226730+0000","mapping_epoch":114,"log_start":"0'0","ondisk_log_start":"0'0","created":114,"last_epoch_clean":115,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:51:59.130555+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:51:59.130555+0000","last_clean_scrub_stamp":"2023-08-09T13:51:59.130555+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T17:54:11.970806+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"15.38","version":"0'0","reported_seq":11,"reported_epoch":116,"state":"active+clean","last_fresh":"2023-08-09T13:52:02.226879+0000","last_change":"2023-08-09T13:52:00.204279+0000","last_active":"2023-08-09T13:52:02.226879+0000","last_peered":"2023-08-09T13:52:02.226879+0000","last_clean":"2023-08-09T13:52:02.226879+0000","last_became_active":"2023-08-09T13:52:00.203894+0000","last_became_peered":"2023-08-09T13:52:00.203894+0000","last_unstale":"2023-08-09T13:52:02.226879+0000","last_undegraded":"2023-08-09T13:52:02.226879+0000","last_fullsized":"2023-08-09T13:52:02.226879+0000","mapping_epoch":114,"log_start":"0'0","ondisk_log_start":"0'0","created":114,"last_epoch_clean":115,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:51:59.130555+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:51:59.130555+0000","last_clean_scrub_stamp":"2023-08-09T13:51:59.130555+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-11T00:36:05.454609+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.37","version":"0'0","reported_seq":11,"reported_epoch":116,"state":"active+clean","last_fresh":"2023-08-09T13:52:01.223149+0000","last_change":"2023-08-09T13:52:00.227036+0000","last_active":"2023-08-09T13:52:01.223149+0000","last_peered":"2023-08-09T13:52:01.223149+0000","last_clean":"2023-08-09T13:52:01.223149+0000","last_became_active":"2023-08-09T13:52:00.225864+0000","last_became_peered":"2023-08-09T13:52:00.225864+0000","last_unstale":"2023-08-09T13:52:01.223149+0000","last_undegraded":"2023-08-09T13:52:01.223149+0000","last_fullsized":"2023-08-09T13:52:01.223149+0000","mapping_epoch":114,"log_start":"0'0","ondisk_log_start":"0'0","created":114,"last_epoch_clean":115,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:51:59.130555+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:51:59.130555+0000","last_clean_scrub_stamp":"2023-08-09T13:51:59.130555+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T19:04:21.591286+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"15.36","version":"0'0","reported_seq":11,"reported_epoch":116,"state":"active+clean","last_fresh":"2023-08-09T13:52:02.230598+0000","last_change":"2023-08-09T13:52:00.227896+0000","last_active":"2023-08-09T13:52:02.230598+0000","last_peered":"2023-08-09T13:52:02.230598+0000","last_clean":"2023-08-09T13:52:02.230598+0000","last_became_active":"2023-08-09T13:52:00.227619+0000","last_became_peered":"2023-08-09T13:52:00.227619+0000","last_unstale":"2023-08-09T13:52:02.230598+0000","last_undegraded":"2023-08-09T13:52:02.230598+0000","last_fullsized":"2023-08-09T13:52:02.230598+0000","mapping_epoch":114,"log_start":"0'0","ondisk_log_start":"0'0","created":114,"last_epoch_clean":115,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:51:59.130555+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:51:59.130555+0000","last_clean_scrub_stamp":"2023-08-09T13:51:59.130555+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T16:11:26.030519+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.35","version":"0'0","reported_seq":11,"reported_epoch":116,"state":"active+clean","last_fresh":"2023-08-09T13:52:02.227069+0000","last_change":"2023-08-09T13:52:00.219649+0000","last_active":"2023-08-09T13:52:02.227069+0000","last_peered":"2023-08-09T13:52:02.227069+0000","last_clean":"2023-08-09T13:52:02.227069+0000","last_became_active":"2023-08-09T13:52:00.219213+0000","last_became_peered":"2023-08-09T13:52:00.219213+0000","last_unstale":"2023-08-09T13:52:02.227069+0000","last_undegraded":"2023-08-09T13:52:02.227069+0000","last_fullsized":"2023-08-09T13:52:02.227069+0000","mapping_epoch":114,"log_start":"0'0","ondisk_log_start":"0'0","created":114,"last_epoch_clean":115,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:51:59.130555+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:51:59.130555+0000","last_clean_scrub_stamp":"2023-08-09T13:51:59.130555+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T17:30:32.182529+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"15.34","version":"0'0","reported_seq":11,"reported_epoch":116,"state":"active+clean","last_fresh":"2023-08-09T13:52:01.222814+0000","last_change":"2023-08-09T13:52:00.233213+0000","last_active":"2023-08-09T13:52:01.222814+0000","last_peered":"2023-08-09T13:52:01.222814+0000","last_clean":"2023-08-09T13:52:01.222814+0000","last_became_active":"2023-08-09T13:52:00.232595+0000","last_became_peered":"2023-08-09T13:52:00.232595+0000","last_unstale":"2023-08-09T13:52:01.222814+0000","last_undegraded":"2023-08-09T13:52:01.222814+0000","last_fullsized":"2023-08-09T13:52:01.222814+0000","mapping_epoch":114,"log_start":"0'0","ondisk_log_start":"0'0","created":114,"last_epoch_clean":115,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:51:59.130555+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:51:59.130555+0000","last_clean_scrub_stamp":"2023-08-09T13:51:59.130555+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T18:35:08.302484+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"15.33","version":"0'0","reported_seq":11,"reported_epoch":116,"state":"active+clean","last_fresh":"2023-08-09T13:52:01.221386+0000","last_change":"2023-08-09T13:52:00.227803+0000","last_active":"2023-08-09T13:52:01.221386+0000","last_peered":"2023-08-09T13:52:01.221386+0000","last_clean":"2023-08-09T13:52:01.221386+0000","last_became_active":"2023-08-09T13:52:00.227454+0000","last_became_peered":"2023-08-09T13:52:00.227454+0000","last_unstale":"2023-08-09T13:52:01.221386+0000","last_undegraded":"2023-08-09T13:52:01.221386+0000","last_fullsized":"2023-08-09T13:52:01.221386+0000","mapping_epoch":114,"log_start":"0'0","ondisk_log_start":"0'0","created":114,"last_epoch_clean":115,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:51:59.130555+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:51:59.130555+0000","last_clean_scrub_stamp":"2023-08-09T13:51:59.130555+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T17:00:08.383368+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"15.32","version":"0'0","reported_seq":11,"reported_epoch":116,"state":"active+clean","last_fresh":"2023-08-09T13:52:01.418792+0000","last_change":"2023-08-09T13:52:00.228808+0000","last_active":"2023-08-09T13:52:01.418792+0000","last_peered":"2023-08-09T13:52:01.418792+0000","last_clean":"2023-08-09T13:52:01.418792+0000","last_became_active":"2023-08-09T13:52:00.228402+0000","last_became_peered":"2023-08-09T13:52:00.228402+0000","last_unstale":"2023-08-09T13:52:01.418792+0000","last_undegraded":"2023-08-09T13:52:01.418792+0000","last_fullsized":"2023-08-09T13:52:01.418792+0000","mapping_epoch":114,"log_start":"0'0","ondisk_log_start":"0'0","created":114,"last_epoch_clean":115,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:51:59.130555+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:51:59.130555+0000","last_clean_scrub_stamp":"2023-08-09T13:51:59.130555+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T13:59:07.741588+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.31","version":"0'0","reported_seq":11,"reported_epoch":116,"state":"active+clean","last_fresh":"2023-08-09T13:52:01.221161+0000","last_change":"2023-08-09T13:52:00.217670+0000","last_active":"2023-08-09T13:52:01.221161+0000","last_peered":"2023-08-09T13:52:01.221161+0000","last_clean":"2023-08-09T13:52:01.221161+0000","last_became_active":"2023-08-09T13:52:00.217394+0000","last_became_peered":"2023-08-09T13:52:00.217394+0000","last_unstale":"2023-08-09T13:52:01.221161+0000","last_undegraded":"2023-08-09T13:52:01.221161+0000","last_fullsized":"2023-08-09T13:52:01.221161+0000","mapping_epoch":114,"log_start":"0'0","ondisk_log_start":"0'0","created":114,"last_epoch_clean":115,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:51:59.130555+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:51:59.130555+0000","last_clean_scrub_stamp":"2023-08-09T13:51:59.130555+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T17:30:32.182529+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"15.30","version":"0'0","reported_seq":11,"reported_epoch":116,"state":"active+clean","last_fresh":"2023-08-09T13:52:02.226714+0000","last_change":"2023-08-09T13:52:00.207684+0000","last_active":"2023-08-09T13:52:02.226714+0000","last_peered":"2023-08-09T13:52:02.226714+0000","last_clean":"2023-08-09T13:52:02.226714+0000","last_became_active":"2023-08-09T13:52:00.207349+0000","last_became_peered":"2023-08-09T13:52:00.207349+0000","last_unstale":"2023-08-09T13:52:02.226714+0000","last_undegraded":"2023-08-09T13:52:02.226714+0000","last_fullsized":"2023-08-09T13:52:02.226714+0000","mapping_epoch":114,"log_start":"0'0","ondisk_log_start":"0'0","created":114,"last_epoch_clean":115,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:51:59.130555+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:51:59.130555+0000","last_clean_scrub_stamp":"2023-08-09T13:51:59.130555+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T18:47:38.882381+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"15.2f","version":"0'0","reported_seq":11,"reported_epoch":116,"state":"active+clean","last_fresh":"2023-08-09T13:52:02.227599+0000","last_change":"2023-08-09T13:52:00.207941+0000","last_active":"2023-08-09T13:52:02.227599+0000","last_peered":"2023-08-09T13:52:02.227599+0000","last_clean":"2023-08-09T13:52:02.227599+0000","last_became_active":"2023-08-09T13:52:00.207602+0000","last_became_peered":"2023-08-09T13:52:00.207602+0000","last_unstale":"2023-08-09T13:52:02.227599+0000","last_undegraded":"2023-08-09T13:52:02.227599+0000","last_fullsized":"2023-08-09T13:52:02.227599+0000","mapping_epoch":114,"log_start":"0'0","ondisk_log_start":"0'0","created":114,"last_epoch_clean":115,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:51:59.130555+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:51:59.130555+0000","last_clean_scrub_stamp":"2023-08-09T13:51:59.130555+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T21:35:51.983564+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.2e","version":"0'0","reported_seq":11,"reported_epoch":116,"state":"active+clean","last_fresh":"2023-08-09T13:52:01.221775+0000","last_change":"2023-08-09T13:52:00.211412+0000","last_active":"2023-08-09T13:52:01.221775+0000","last_peered":"2023-08-09T13:52:01.221775+0000","last_clean":"2023-08-09T13:52:01.221775+0000","last_became_active":"2023-08-09T13:52:00.211012+0000","last_became_peered":"2023-08-09T13:52:00.211012+0000","last_unstale":"2023-08-09T13:52:01.221775+0000","last_undegraded":"2023-08-09T13:52:01.221775+0000","last_fullsized":"2023-08-09T13:52:01.221775+0000","mapping_epoch":114,"log_start":"0'0","ondisk_log_start":"0'0","created":114,"last_epoch_clean":115,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:51:59.130555+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:51:59.130555+0000","last_clean_scrub_stamp":"2023-08-09T13:51:59.130555+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-11T00:12:54.873335+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"15.2d","version":"0'0","reported_seq":11,"reported_epoch":116,"state":"active+clean","last_fresh":"2023-08-09T13:52:01.419294+0000","last_change":"2023-08-09T13:52:00.220158+0000","last_active":"2023-08-09T13:52:01.419294+0000","last_peered":"2023-08-09T13:52:01.419294+0000","last_clean":"2023-08-09T13:52:01.419294+0000","last_became_active":"2023-08-09T13:52:00.219834+0000","last_became_peered":"2023-08-09T13:52:00.219834+0000","last_unstale":"2023-08-09T13:52:01.419294+0000","last_undegraded":"2023-08-09T13:52:01.419294+0000","last_fullsized":"2023-08-09T13:52:01.419294+0000","mapping_epoch":114,"log_start":"0'0","ondisk_log_start":"0'0","created":114,"last_epoch_clean":115,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:51:59.130555+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:51:59.130555+0000","last_clean_scrub_stamp":"2023-08-09T13:51:59.130555+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T19:02:16.257928+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.2c","version":"0'0","reported_seq":11,"reported_epoch":116,"state":"active+clean","last_fresh":"2023-08-09T13:52:01.222416+0000","last_change":"2023-08-09T13:52:00.227515+0000","last_active":"2023-08-09T13:52:01.222416+0000","last_peered":"2023-08-09T13:52:01.222416+0000","last_clean":"2023-08-09T13:52:01.222416+0000","last_became_active":"2023-08-09T13:52:00.227142+0000","last_became_peered":"2023-08-09T13:52:00.227142+0000","last_unstale":"2023-08-09T13:52:01.222416+0000","last_undegraded":"2023-08-09T13:52:01.222416+0000","last_fullsized":"2023-08-09T13:52:01.222416+0000","mapping_epoch":114,"log_start":"0'0","ondisk_log_start":"0'0","created":114,"last_epoch_clean":115,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:51:59.130555+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:51:59.130555+0000","last_clean_scrub_stamp":"2023-08-09T13:51:59.130555+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T14:22:06.615562+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"15.2b","version":"0'0","reported_seq":11,"reported_epoch":116,"state":"active+clean","last_fresh":"2023-08-09T13:52:01.419015+0000","last_change":"2023-08-09T13:52:00.226181+0000","last_active":"2023-08-09T13:52:01.419015+0000","last_peered":"2023-08-09T13:52:01.419015+0000","last_clean":"2023-08-09T13:52:01.419015+0000","last_became_active":"2023-08-09T13:52:00.225876+0000","last_became_peered":"2023-08-09T13:52:00.225876+0000","last_unstale":"2023-08-09T13:52:01.419015+0000","last_undegraded":"2023-08-09T13:52:01.419015+0000","last_fullsized":"2023-08-09T13:52:01.419015+0000","mapping_epoch":114,"log_start":"0'0","ondisk_log_start":"0'0","created":114,"last_epoch_clean":115,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:51:59.130555+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:51:59.130555+0000","last_clean_scrub_stamp":"2023-08-09T13:51:59.130555+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T17:38:57.970130+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.2a","version":"0'0","reported_seq":11,"reported_epoch":116,"state":"active+clean","last_fresh":"2023-08-09T13:52:02.226954+0000","last_change":"2023-08-09T13:52:00.217756+0000","last_active":"2023-08-09T13:52:02.226954+0000","last_peered":"2023-08-09T13:52:02.226954+0000","last_clean":"2023-08-09T13:52:02.226954+0000","last_became_active":"2023-08-09T13:52:00.217441+0000","last_became_peered":"2023-08-09T13:52:00.217441+0000","last_unstale":"2023-08-09T13:52:02.226954+0000","last_undegraded":"2023-08-09T13:52:02.226954+0000","last_fullsized":"2023-08-09T13:52:02.226954+0000","mapping_epoch":114,"log_start":"0'0","ondisk_log_start":"0'0","created":114,"last_epoch_clean":115,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:51:59.130555+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:51:59.130555+0000","last_clean_scrub_stamp":"2023-08-09T13:51:59.130555+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-11T00:34:01.999153+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.29","version":"0'0","reported_seq":11,"reported_epoch":116,"state":"active+clean","last_fresh":"2023-08-09T13:52:02.226935+0000","last_change":"2023-08-09T13:52:00.215869+0000","last_active":"2023-08-09T13:52:02.226935+0000","last_peered":"2023-08-09T13:52:02.226935+0000","last_clean":"2023-08-09T13:52:02.226935+0000","last_became_active":"2023-08-09T13:52:00.215592+0000","last_became_peered":"2023-08-09T13:52:00.215592+0000","last_unstale":"2023-08-09T13:52:02.226935+0000","last_undegraded":"2023-08-09T13:52:02.226935+0000","last_fullsized":"2023-08-09T13:52:02.226935+0000","mapping_epoch":114,"log_start":"0'0","ondisk_log_start":"0'0","created":114,"last_epoch_clean":115,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:51:59.130555+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:51:59.130555+0000","last_clean_scrub_stamp":"2023-08-09T13:51:59.130555+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T22:31:49.221924+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.28","version":"0'0","reported_seq":11,"reported_epoch":116,"state":"active+clean","last_fresh":"2023-08-09T13:52:02.226788+0000","last_change":"2023-08-09T13:52:00.230092+0000","last_active":"2023-08-09T13:52:02.226788+0000","last_peered":"2023-08-09T13:52:02.226788+0000","last_clean":"2023-08-09T13:52:02.226788+0000","last_became_active":"2023-08-09T13:52:00.229786+0000","last_became_peered":"2023-08-09T13:52:00.229786+0000","last_unstale":"2023-08-09T13:52:02.226788+0000","last_undegraded":"2023-08-09T13:52:02.226788+0000","last_fullsized":"2023-08-09T13:52:02.226788+0000","mapping_epoch":114,"log_start":"0'0","ondisk_log_start":"0'0","created":114,"last_epoch_clean":115,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:51:59.130555+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:51:59.130555+0000","last_clean_scrub_stamp":"2023-08-09T13:51:59.130555+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T23:11:37.527659+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"15.27","version":"0'0","reported_seq":11,"reported_epoch":116,"state":"active+clean","last_fresh":"2023-08-09T13:52:02.229935+0000","last_change":"2023-08-09T13:52:00.224333+0000","last_active":"2023-08-09T13:52:02.229935+0000","last_peered":"2023-08-09T13:52:02.229935+0000","last_clean":"2023-08-09T13:52:02.229935+0000","last_became_active":"2023-08-09T13:52:00.224023+0000","last_became_peered":"2023-08-09T13:52:00.224023+0000","last_unstale":"2023-08-09T13:52:02.229935+0000","last_undegraded":"2023-08-09T13:52:02.229935+0000","last_fullsized":"2023-08-09T13:52:02.229935+0000","mapping_epoch":114,"log_start":"0'0","ondisk_log_start":"0'0","created":114,"last_epoch_clean":115,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:51:59.130555+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:51:59.130555+0000","last_clean_scrub_stamp":"2023-08-09T13:51:59.130555+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-11T01:00:31.885923+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.26","version":"0'0","reported_seq":11,"reported_epoch":116,"state":"active+clean","last_fresh":"2023-08-09T13:52:02.227692+0000","last_change":"2023-08-09T13:52:00.221166+0000","last_active":"2023-08-09T13:52:02.227692+0000","last_peered":"2023-08-09T13:52:02.227692+0000","last_clean":"2023-08-09T13:52:02.227692+0000","last_became_active":"2023-08-09T13:52:00.220856+0000","last_became_peered":"2023-08-09T13:52:00.220856+0000","last_unstale":"2023-08-09T13:52:02.227692+0000","last_undegraded":"2023-08-09T13:52:02.227692+0000","last_fullsized":"2023-08-09T13:52:02.227692+0000","mapping_epoch":114,"log_start":"0'0","ondisk_log_start":"0'0","created":114,"last_epoch_clean":115,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:51:59.130555+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:51:59.130555+0000","last_clean_scrub_stamp":"2023-08-09T13:51:59.130555+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T14:42:50.977974+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.25","version":"0'0","reported_seq":11,"reported_epoch":116,"state":"active+clean","last_fresh":"2023-08-09T13:52:02.227139+0000","last_change":"2023-08-09T13:52:00.221719+0000","last_active":"2023-08-09T13:52:02.227139+0000","last_peered":"2023-08-09T13:52:02.227139+0000","last_clean":"2023-08-09T13:52:02.227139+0000","last_became_active":"2023-08-09T13:52:00.221398+0000","last_became_peered":"2023-08-09T13:52:00.221398+0000","last_unstale":"2023-08-09T13:52:02.227139+0000","last_undegraded":"2023-08-09T13:52:02.227139+0000","last_fullsized":"2023-08-09T13:52:02.227139+0000","mapping_epoch":114,"log_start":"0'0","ondisk_log_start":"0'0","created":114,"last_epoch_clean":115,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:51:59.130555+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:51:59.130555+0000","last_clean_scrub_stamp":"2023-08-09T13:51:59.130555+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T21:54:42.771899+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"15.24","version":"0'0","reported_seq":11,"reported_epoch":116,"state":"active+clean","last_fresh":"2023-08-09T13:52:02.229433+0000","last_change":"2023-08-09T13:52:00.218234+0000","last_active":"2023-08-09T13:52:02.229433+0000","last_peered":"2023-08-09T13:52:02.229433+0000","last_clean":"2023-08-09T13:52:02.229433+0000","last_became_active":"2023-08-09T13:52:00.217730+0000","last_became_peered":"2023-08-09T13:52:00.217730+0000","last_unstale":"2023-08-09T13:52:02.229433+0000","last_undegraded":"2023-08-09T13:52:02.229433+0000","last_fullsized":"2023-08-09T13:52:02.229433+0000","mapping_epoch":114,"log_start":"0'0","ondisk_log_start":"0'0","created":114,"last_epoch_clean":115,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:51:59.130555+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:51:59.130555+0000","last_clean_scrub_stamp":"2023-08-09T13:51:59.130555+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T17:28:44.810490+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.23","version":"0'0","reported_seq":11,"reported_epoch":116,"state":"active+clean","last_fresh":"2023-08-09T13:52:02.227717+0000","last_change":"2023-08-09T13:52:00.222136+0000","last_active":"2023-08-09T13:52:02.227717+0000","last_peered":"2023-08-09T13:52:02.227717+0000","last_clean":"2023-08-09T13:52:02.227717+0000","last_became_active":"2023-08-09T13:52:00.221485+0000","last_became_peered":"2023-08-09T13:52:00.221485+0000","last_unstale":"2023-08-09T13:52:02.227717+0000","last_undegraded":"2023-08-09T13:52:02.227717+0000","last_fullsized":"2023-08-09T13:52:02.227717+0000","mapping_epoch":114,"log_start":"0'0","ondisk_log_start":"0'0","created":114,"last_epoch_clean":115,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:51:59.130555+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:51:59.130555+0000","last_clean_scrub_stamp":"2023-08-09T13:51:59.130555+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-11T01:49:04.981047+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.22","version":"0'0","reported_seq":11,"reported_epoch":116,"state":"active+clean","last_fresh":"2023-08-09T13:52:01.418818+0000","last_change":"2023-08-09T13:52:00.220078+0000","last_active":"2023-08-09T13:52:01.418818+0000","last_peered":"2023-08-09T13:52:01.418818+0000","last_clean":"2023-08-09T13:52:01.418818+0000","last_became_active":"2023-08-09T13:52:00.219770+0000","last_became_peered":"2023-08-09T13:52:00.219770+0000","last_unstale":"2023-08-09T13:52:01.418818+0000","last_undegraded":"2023-08-09T13:52:01.418818+0000","last_fullsized":"2023-08-09T13:52:01.418818+0000","mapping_epoch":114,"log_start":"0'0","ondisk_log_start":"0'0","created":114,"last_epoch_clean":115,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:51:59.130555+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:51:59.130555+0000","last_clean_scrub_stamp":"2023-08-09T13:51:59.130555+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T18:22:00.956161+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.21","version":"0'0","reported_seq":11,"reported_epoch":116,"state":"active+clean","last_fresh":"2023-08-09T13:52:01.221780+0000","last_change":"2023-08-09T13:52:00.226969+0000","last_active":"2023-08-09T13:52:01.221780+0000","last_peered":"2023-08-09T13:52:01.221780+0000","last_clean":"2023-08-09T13:52:01.221780+0000","last_became_active":"2023-08-09T13:52:00.226594+0000","last_became_peered":"2023-08-09T13:52:00.226594+0000","last_unstale":"2023-08-09T13:52:01.221780+0000","last_undegraded":"2023-08-09T13:52:01.221780+0000","last_fullsized":"2023-08-09T13:52:01.221780+0000","mapping_epoch":114,"log_start":"0'0","ondisk_log_start":"0'0","created":114,"last_epoch_clean":115,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:51:59.130555+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:51:59.130555+0000","last_clean_scrub_stamp":"2023-08-09T13:51:59.130555+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T14:07:44.282339+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"15.20","version":"0'0","reported_seq":11,"reported_epoch":116,"state":"active+clean","last_fresh":"2023-08-09T13:52:01.219258+0000","last_change":"2023-08-09T13:52:00.217646+0000","last_active":"2023-08-09T13:52:01.219258+0000","last_peered":"2023-08-09T13:52:01.219258+0000","last_clean":"2023-08-09T13:52:01.219258+0000","last_became_active":"2023-08-09T13:52:00.217316+0000","last_became_peered":"2023-08-09T13:52:00.217316+0000","last_unstale":"2023-08-09T13:52:01.219258+0000","last_undegraded":"2023-08-09T13:52:01.219258+0000","last_fullsized":"2023-08-09T13:52:01.219258+0000","mapping_epoch":114,"log_start":"0'0","ondisk_log_start":"0'0","created":114,"last_epoch_clean":115,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:51:59.130555+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:51:59.130555+0000","last_clean_scrub_stamp":"2023-08-09T13:51:59.130555+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-11T00:58:17.334390+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"15.1e","version":"0'0","reported_seq":11,"reported_epoch":116,"state":"active+clean","last_fresh":"2023-08-09T13:52:01.419293+0000","last_change":"2023-08-09T13:52:00.211947+0000","last_active":"2023-08-09T13:52:01.419293+0000","last_peered":"2023-08-09T13:52:01.419293+0000","last_clean":"2023-08-09T13:52:01.419293+0000","last_became_active":"2023-08-09T13:52:00.211455+0000","last_became_peered":"2023-08-09T13:52:00.211455+0000","last_unstale":"2023-08-09T13:52:01.419293+0000","last_undegraded":"2023-08-09T13:52:01.419293+0000","last_fullsized":"2023-08-09T13:52:01.419293+0000","mapping_epoch":114,"log_start":"0'0","ondisk_log_start":"0'0","created":114,"last_epoch_clean":115,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:51:59.130555+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:51:59.130555+0000","last_clean_scrub_stamp":"2023-08-09T13:51:59.130555+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-11T01:31:43.361691+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"14.1f","version":"115'4","reported_seq":17,"reported_epoch":116,"state":"active+clean","last_fresh":"2023-08-09T13:52:02.227886+0000","last_change":"2023-08-09T13:51:59.176081+0000","last_active":"2023-08-09T13:52:02.227886+0000","last_peered":"2023-08-09T13:52:02.227886+0000","last_clean":"2023-08-09T13:52:02.227886+0000","last_became_active":"2023-08-09T13:51:59.175790+0000","last_became_peered":"2023-08-09T13:51:59.175790+0000","last_unstale":"2023-08-09T13:52:02.227886+0000","last_undegraded":"2023-08-09T13:52:02.227886+0000","last_fullsized":"2023-08-09T13:52:02.227886+0000","mapping_epoch":113,"log_start":"0'0","ondisk_log_start":"0'0","created":113,"last_epoch_clean":114,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:51:58.125661+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:51:58.125661+0000","last_clean_scrub_stamp":"2023-08-09T13:51:58.125661+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T22:25:52.108038+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.1f","version":"0'0","reported_seq":11,"reported_epoch":116,"state":"active+clean","last_fresh":"2023-08-09T13:52:01.221580+0000","last_change":"2023-08-09T13:52:00.226768+0000","last_active":"2023-08-09T13:52:01.221580+0000","last_peered":"2023-08-09T13:52:01.221580+0000","last_clean":"2023-08-09T13:52:01.221580+0000","last_became_active":"2023-08-09T13:52:00.226460+0000","last_became_peered":"2023-08-09T13:52:00.226460+0000","last_unstale":"2023-08-09T13:52:01.221580+0000","last_undegraded":"2023-08-09T13:52:01.221580+0000","last_fullsized":"2023-08-09T13:52:01.221580+0000","mapping_epoch":114,"log_start":"0'0","ondisk_log_start":"0'0","created":114,"last_epoch_clean":115,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:51:59.130555+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:51:59.130555+0000","last_clean_scrub_stamp":"2023-08-09T13:51:59.130555+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T23:42:47.388329+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"14.1e","version":"115'1","reported_seq":14,"reported_epoch":116,"state":"active+clean","last_fresh":"2023-08-09T13:52:02.227882+0000","last_change":"2023-08-09T13:51:59.178857+0000","last_active":"2023-08-09T13:52:02.227882+0000","last_peered":"2023-08-09T13:52:02.227882+0000","last_clean":"2023-08-09T13:52:02.227882+0000","last_became_active":"2023-08-09T13:51:59.178483+0000","last_became_peered":"2023-08-09T13:51:59.178483+0000","last_unstale":"2023-08-09T13:52:02.227882+0000","last_undegraded":"2023-08-09T13:52:02.227882+0000","last_fullsized":"2023-08-09T13:52:02.227882+0000","mapping_epoch":113,"log_start":"0'0","ondisk_log_start":"0'0","created":113,"last_epoch_clean":114,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:51:58.125661+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:51:58.125661+0000","last_clean_scrub_stamp":"2023-08-09T13:51:58.125661+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T22:50:15.633917+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"15.1c","version":"0'0","reported_seq":11,"reported_epoch":116,"state":"active+clean","last_fresh":"2023-08-09T13:52:01.219749+0000","last_change":"2023-08-09T13:52:00.228876+0000","last_active":"2023-08-09T13:52:01.219749+0000","last_peered":"2023-08-09T13:52:01.219749+0000","last_clean":"2023-08-09T13:52:01.219749+0000","last_became_active":"2023-08-09T13:52:00.227420+0000","last_became_peered":"2023-08-09T13:52:00.227420+0000","last_unstale":"2023-08-09T13:52:01.219749+0000","last_undegraded":"2023-08-09T13:52:01.219749+0000","last_fullsized":"2023-08-09T13:52:01.219749+0000","mapping_epoch":114,"log_start":"0'0","ondisk_log_start":"0'0","created":114,"last_epoch_clean":115,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:51:59.130555+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:51:59.130555+0000","last_clean_scrub_stamp":"2023-08-09T13:51:59.130555+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T22:09:46.463659+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.1d","version":"115'2","reported_seq":15,"reported_epoch":116,"state":"active+clean","last_fresh":"2023-08-09T13:52:02.227415+0000","last_change":"2023-08-09T13:51:59.168942+0000","last_active":"2023-08-09T13:52:02.227415+0000","last_peered":"2023-08-09T13:52:02.227415+0000","last_clean":"2023-08-09T13:52:02.227415+0000","last_became_active":"2023-08-09T13:51:59.168599+0000","last_became_peered":"2023-08-09T13:51:59.168599+0000","last_unstale":"2023-08-09T13:52:02.227415+0000","last_undegraded":"2023-08-09T13:52:02.227415+0000","last_fullsized":"2023-08-09T13:52:02.227415+0000","mapping_epoch":113,"log_start":"0'0","ondisk_log_start":"0'0","created":113,"last_epoch_clean":114,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:51:58.125661+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:51:58.125661+0000","last_clean_scrub_stamp":"2023-08-09T13:51:58.125661+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-11T00:44:59.042168+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.1d","version":"0'0","reported_seq":11,"reported_epoch":116,"state":"active+clean","last_fresh":"2023-08-09T13:52:01.222594+0000","last_change":"2023-08-09T13:52:00.223439+0000","last_active":"2023-08-09T13:52:01.222594+0000","last_peered":"2023-08-09T13:52:01.222594+0000","last_clean":"2023-08-09T13:52:01.222594+0000","last_became_active":"2023-08-09T13:52:00.222651+0000","last_became_peered":"2023-08-09T13:52:00.222651+0000","last_unstale":"2023-08-09T13:52:01.222594+0000","last_undegraded":"2023-08-09T13:52:01.222594+0000","last_fullsized":"2023-08-09T13:52:01.222594+0000","mapping_epoch":114,"log_start":"0'0","ondisk_log_start":"0'0","created":114,"last_epoch_clean":115,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:51:59.130555+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:51:59.130555+0000","last_clean_scrub_stamp":"2023-08-09T13:51:59.130555+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T16:15:13.180404+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.1c","version":"115'2","reported_seq":15,"reported_epoch":116,"state":"active+clean","last_fresh":"2023-08-09T13:52:02.227858+0000","last_change":"2023-08-09T13:51:59.183710+0000","last_active":"2023-08-09T13:52:02.227858+0000","last_peered":"2023-08-09T13:52:02.227858+0000","last_clean":"2023-08-09T13:52:02.227858+0000","last_became_active":"2023-08-09T13:51:59.183401+0000","last_became_peered":"2023-08-09T13:51:59.183401+0000","last_unstale":"2023-08-09T13:52:02.227858+0000","last_undegraded":"2023-08-09T13:52:02.227858+0000","last_fullsized":"2023-08-09T13:52:02.227858+0000","mapping_epoch":113,"log_start":"0'0","ondisk_log_start":"0'0","created":113,"last_epoch_clean":114,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:51:58.125661+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:51:58.125661+0000","last_clean_scrub_stamp":"2023-08-09T13:51:58.125661+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T17:51:46.462712+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.1a","version":"0'0","reported_seq":11,"reported_epoch":116,"state":"active+clean","last_fresh":"2023-08-09T13:52:02.228354+0000","last_change":"2023-08-09T13:52:00.214536+0000","last_active":"2023-08-09T13:52:02.228354+0000","last_peered":"2023-08-09T13:52:02.228354+0000","last_clean":"2023-08-09T13:52:02.228354+0000","last_became_active":"2023-08-09T13:52:00.214237+0000","last_became_peered":"2023-08-09T13:52:00.214237+0000","last_unstale":"2023-08-09T13:52:02.228354+0000","last_undegraded":"2023-08-09T13:52:02.228354+0000","last_fullsized":"2023-08-09T13:52:02.228354+0000","mapping_epoch":114,"log_start":"0'0","ondisk_log_start":"0'0","created":114,"last_epoch_clean":115,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:51:59.130555+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:51:59.130555+0000","last_clean_scrub_stamp":"2023-08-09T13:51:59.130555+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T14:23:12.776157+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"14.1b","version":"115'1","reported_seq":14,"reported_epoch":116,"state":"active+clean","last_fresh":"2023-08-09T13:52:01.219598+0000","last_change":"2023-08-09T13:51:59.178308+0000","last_active":"2023-08-09T13:52:01.219598+0000","last_peered":"2023-08-09T13:52:01.219598+0000","last_clean":"2023-08-09T13:52:01.219598+0000","last_became_active":"2023-08-09T13:51:59.178022+0000","last_became_peered":"2023-08-09T13:51:59.178022+0000","last_unstale":"2023-08-09T13:52:01.219598+0000","last_undegraded":"2023-08-09T13:52:01.219598+0000","last_fullsized":"2023-08-09T13:52:01.219598+0000","mapping_epoch":113,"log_start":"0'0","ondisk_log_start":"0'0","created":113,"last_epoch_clean":114,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:51:58.125661+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:51:58.125661+0000","last_clean_scrub_stamp":"2023-08-09T13:51:58.125661+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T15:02:20.908250+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"15.1b","version":"0'0","reported_seq":11,"reported_epoch":116,"state":"active+clean","last_fresh":"2023-08-09T13:52:02.226965+0000","last_change":"2023-08-09T13:52:00.216318+0000","last_active":"2023-08-09T13:52:02.226965+0000","last_peered":"2023-08-09T13:52:02.226965+0000","last_clean":"2023-08-09T13:52:02.226965+0000","last_became_active":"2023-08-09T13:52:00.216021+0000","last_became_peered":"2023-08-09T13:52:00.216021+0000","last_unstale":"2023-08-09T13:52:02.226965+0000","last_undegraded":"2023-08-09T13:52:02.226965+0000","last_fullsized":"2023-08-09T13:52:02.226965+0000","mapping_epoch":114,"log_start":"0'0","ondisk_log_start":"0'0","created":114,"last_epoch_clean":115,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:51:59.130555+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:51:59.130555+0000","last_clean_scrub_stamp":"2023-08-09T13:51:59.130555+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T18:50:44.479523+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"14.1a","version":"115'1","reported_seq":14,"reported_epoch":116,"state":"active+clean","last_fresh":"2023-08-09T13:52:02.226933+0000","last_change":"2023-08-09T13:51:59.165522+0000","last_active":"2023-08-09T13:52:02.226933+0000","last_peered":"2023-08-09T13:52:02.226933+0000","last_clean":"2023-08-09T13:52:02.226933+0000","last_became_active":"2023-08-09T13:51:59.165145+0000","last_became_peered":"2023-08-09T13:51:59.165145+0000","last_unstale":"2023-08-09T13:52:02.226933+0000","last_undegraded":"2023-08-09T13:52:02.226933+0000","last_fullsized":"2023-08-09T13:52:02.226933+0000","mapping_epoch":113,"log_start":"0'0","ondisk_log_start":"0'0","created":113,"last_epoch_clean":114,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:51:58.125661+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:51:58.125661+0000","last_clean_scrub_stamp":"2023-08-09T13:51:58.125661+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-11T01:20:41.335156+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.18","version":"0'0","reported_seq":11,"reported_epoch":116,"state":"active+clean","last_fresh":"2023-08-09T13:52:01.221194+0000","last_change":"2023-08-09T13:52:00.218747+0000","last_active":"2023-08-09T13:52:01.221194+0000","last_peered":"2023-08-09T13:52:01.221194+0000","last_clean":"2023-08-09T13:52:01.221194+0000","last_became_active":"2023-08-09T13:52:00.218470+0000","last_became_peered":"2023-08-09T13:52:00.218470+0000","last_unstale":"2023-08-09T13:52:01.221194+0000","last_undegraded":"2023-08-09T13:52:01.221194+0000","last_fullsized":"2023-08-09T13:52:01.221194+0000","mapping_epoch":114,"log_start":"0'0","ondisk_log_start":"0'0","created":114,"last_epoch_clean":115,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:51:59.130555+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:51:59.130555+0000","last_clean_scrub_stamp":"2023-08-09T13:51:59.130555+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T19:49:50.219295+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"14.19","version":"0'0","reported_seq":13,"reported_epoch":116,"state":"active+clean","last_fresh":"2023-08-09T13:52:02.229449+0000","last_change":"2023-08-09T13:51:59.174362+0000","last_active":"2023-08-09T13:52:02.229449+0000","last_peered":"2023-08-09T13:52:02.229449+0000","last_clean":"2023-08-09T13:52:02.229449+0000","last_became_active":"2023-08-09T13:51:59.174002+0000","last_became_peered":"2023-08-09T13:51:59.174002+0000","last_unstale":"2023-08-09T13:52:02.229449+0000","last_undegraded":"2023-08-09T13:52:02.229449+0000","last_fullsized":"2023-08-09T13:52:02.229449+0000","mapping_epoch":113,"log_start":"0'0","ondisk_log_start":"0'0","created":113,"last_epoch_clean":114,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:51:58.125661+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:51:58.125661+0000","last_clean_scrub_stamp":"2023-08-09T13:51:58.125661+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T20:09:44.063483+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.19","version":"0'0","reported_seq":11,"reported_epoch":116,"state":"active+clean","last_fresh":"2023-08-09T13:52:01.221492+0000","last_change":"2023-08-09T13:52:00.223692+0000","last_active":"2023-08-09T13:52:01.221492+0000","last_peered":"2023-08-09T13:52:01.221492+0000","last_clean":"2023-08-09T13:52:01.221492+0000","last_became_active":"2023-08-09T13:52:00.223359+0000","last_became_peered":"2023-08-09T13:52:00.223359+0000","last_unstale":"2023-08-09T13:52:01.221492+0000","last_undegraded":"2023-08-09T13:52:01.221492+0000","last_fullsized":"2023-08-09T13:52:01.221492+0000","mapping_epoch":114,"log_start":"0'0","ondisk_log_start":"0'0","created":114,"last_epoch_clean":115,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:51:59.130555+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:51:59.130555+0000","last_clean_scrub_stamp":"2023-08-09T13:51:59.130555+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T23:02:17.294628+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.18","version":"0'0","reported_seq":13,"reported_epoch":116,"state":"active+clean","last_fresh":"2023-08-09T13:52:01.219461+0000","last_change":"2023-08-09T13:51:59.177991+0000","last_active":"2023-08-09T13:52:01.219461+0000","last_peered":"2023-08-09T13:52:01.219461+0000","last_clean":"2023-08-09T13:52:01.219461+0000","last_became_active":"2023-08-09T13:51:59.177651+0000","last_became_peered":"2023-08-09T13:51:59.177651+0000","last_unstale":"2023-08-09T13:52:01.219461+0000","last_undegraded":"2023-08-09T13:52:01.219461+0000","last_fullsized":"2023-08-09T13:52:01.219461+0000","mapping_epoch":113,"log_start":"0'0","ondisk_log_start":"0'0","created":113,"last_epoch_clean":114,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:51:58.125661+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:51:58.125661+0000","last_clean_scrub_stamp":"2023-08-09T13:51:58.125661+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T15:55:33.249335+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"15.16","version":"0'0","reported_seq":11,"reported_epoch":116,"state":"active+clean","last_fresh":"2023-08-09T13:52:02.227886+0000","last_change":"2023-08-09T13:52:00.214111+0000","last_active":"2023-08-09T13:52:02.227886+0000","last_peered":"2023-08-09T13:52:02.227886+0000","last_clean":"2023-08-09T13:52:02.227886+0000","last_became_active":"2023-08-09T13:52:00.213831+0000","last_became_peered":"2023-08-09T13:52:00.213831+0000","last_unstale":"2023-08-09T13:52:02.227886+0000","last_undegraded":"2023-08-09T13:52:02.227886+0000","last_fullsized":"2023-08-09T13:52:02.227886+0000","mapping_epoch":114,"log_start":"0'0","ondisk_log_start":"0'0","created":114,"last_epoch_clean":115,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:51:59.130555+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:51:59.130555+0000","last_clean_scrub_stamp":"2023-08-09T13:51:59.130555+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T20:09:37.121769+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"14.17","version":"0'0","reported_seq":13,"reported_epoch":116,"state":"active+clean","last_fresh":"2023-08-09T13:52:01.220356+0000","last_change":"2023-08-09T13:51:59.181245+0000","last_active":"2023-08-09T13:52:01.220356+0000","last_peered":"2023-08-09T13:52:01.220356+0000","last_clean":"2023-08-09T13:52:01.220356+0000","last_became_active":"2023-08-09T13:51:59.180911+0000","last_became_peered":"2023-08-09T13:51:59.180911+0000","last_unstale":"2023-08-09T13:52:01.220356+0000","last_undegraded":"2023-08-09T13:52:01.220356+0000","last_fullsized":"2023-08-09T13:52:01.220356+0000","mapping_epoch":113,"log_start":"0'0","ondisk_log_start":"0'0","created":113,"last_epoch_clean":114,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:51:58.125661+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:51:58.125661+0000","last_clean_scrub_stamp":"2023-08-09T13:51:58.125661+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T21:59:54.047931+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"15.17","version":"0'0","reported_seq":11,"reported_epoch":116,"state":"active+clean","last_fresh":"2023-08-09T13:52:01.223360+0000","last_change":"2023-08-09T13:52:00.224692+0000","last_active":"2023-08-09T13:52:01.223360+0000","last_peered":"2023-08-09T13:52:01.223360+0000","last_clean":"2023-08-09T13:52:01.223360+0000","last_became_active":"2023-08-09T13:52:00.224355+0000","last_became_peered":"2023-08-09T13:52:00.224355+0000","last_unstale":"2023-08-09T13:52:01.223360+0000","last_undegraded":"2023-08-09T13:52:01.223360+0000","last_fullsized":"2023-08-09T13:52:01.223360+0000","mapping_epoch":114,"log_start":"0'0","ondisk_log_start":"0'0","created":114,"last_epoch_clean":115,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:51:59.130555+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:51:59.130555+0000","last_clean_scrub_stamp":"2023-08-09T13:51:59.130555+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T19:55:13.438705+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.16","version":"0'0","reported_seq":13,"reported_epoch":116,"state":"active+clean","last_fresh":"2023-08-09T13:52:01.221834+0000","last_change":"2023-08-09T13:51:59.172670+0000","last_active":"2023-08-09T13:52:01.221834+0000","last_peered":"2023-08-09T13:52:01.221834+0000","last_clean":"2023-08-09T13:52:01.221834+0000","last_became_active":"2023-08-09T13:51:59.172395+0000","last_became_peered":"2023-08-09T13:51:59.172395+0000","last_unstale":"2023-08-09T13:52:01.221834+0000","last_undegraded":"2023-08-09T13:52:01.221834+0000","last_fullsized":"2023-08-09T13:52:01.221834+0000","mapping_epoch":113,"log_start":"0'0","ondisk_log_start":"0'0","created":113,"last_epoch_clean":114,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:51:58.125661+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:51:58.125661+0000","last_clean_scrub_stamp":"2023-08-09T13:51:58.125661+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T15:33:24.805826+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"15.14","version":"0'0","reported_seq":11,"reported_epoch":116,"state":"active+clean","last_fresh":"2023-08-09T13:52:02.229499+0000","last_change":"2023-08-09T13:52:00.218051+0000","last_active":"2023-08-09T13:52:02.229499+0000","last_peered":"2023-08-09T13:52:02.229499+0000","last_clean":"2023-08-09T13:52:02.229499+0000","last_became_active":"2023-08-09T13:52:00.217381+0000","last_became_peered":"2023-08-09T13:52:00.217381+0000","last_unstale":"2023-08-09T13:52:02.229499+0000","last_undegraded":"2023-08-09T13:52:02.229499+0000","last_fullsized":"2023-08-09T13:52:02.229499+0000","mapping_epoch":114,"log_start":"0'0","ondisk_log_start":"0'0","created":114,"last_epoch_clean":115,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:51:59.130555+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:51:59.130555+0000","last_clean_scrub_stamp":"2023-08-09T13:51:59.130555+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T19:20:41.697718+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"14.15","version":"115'3","reported_seq":16,"reported_epoch":116,"state":"active+clean","last_fresh":"2023-08-09T13:52:02.227755+0000","last_change":"2023-08-09T13:51:59.181964+0000","last_active":"2023-08-09T13:52:02.227755+0000","last_peered":"2023-08-09T13:52:02.227755+0000","last_clean":"2023-08-09T13:52:02.227755+0000","last_became_active":"2023-08-09T13:51:59.181630+0000","last_became_peered":"2023-08-09T13:51:59.181630+0000","last_unstale":"2023-08-09T13:52:02.227755+0000","last_undegraded":"2023-08-09T13:52:02.227755+0000","last_fullsized":"2023-08-09T13:52:02.227755+0000","mapping_epoch":113,"log_start":"0'0","ondisk_log_start":"0'0","created":113,"last_epoch_clean":114,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:51:58.125661+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:51:58.125661+0000","last_clean_scrub_stamp":"2023-08-09T13:51:58.125661+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T23:44:29.227616+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"15.15","version":"0'0","reported_seq":11,"reported_epoch":116,"state":"active+clean","last_fresh":"2023-08-09T13:52:02.227429+0000","last_change":"2023-08-09T13:52:00.210997+0000","last_active":"2023-08-09T13:52:02.227429+0000","last_peered":"2023-08-09T13:52:02.227429+0000","last_clean":"2023-08-09T13:52:02.227429+0000","last_became_active":"2023-08-09T13:52:00.210530+0000","last_became_peered":"2023-08-09T13:52:00.210530+0000","last_unstale":"2023-08-09T13:52:02.227429+0000","last_undegraded":"2023-08-09T13:52:02.227429+0000","last_fullsized":"2023-08-09T13:52:02.227429+0000","mapping_epoch":114,"log_start":"0'0","ondisk_log_start":"0'0","created":114,"last_epoch_clean":115,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:51:59.130555+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:51:59.130555+0000","last_clean_scrub_stamp":"2023-08-09T13:51:59.130555+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T17:00:08.383368+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"14.14","version":"115'6","reported_seq":19,"reported_epoch":116,"state":"active+clean","last_fresh":"2023-08-09T13:52:01.223231+0000","last_change":"2023-08-09T13:51:59.181967+0000","last_active":"2023-08-09T13:52:01.223231+0000","last_peered":"2023-08-09T13:52:01.223231+0000","last_clean":"2023-08-09T13:52:01.223231+0000","last_became_active":"2023-08-09T13:51:59.180664+0000","last_became_peered":"2023-08-09T13:51:59.180664+0000","last_unstale":"2023-08-09T13:52:01.223231+0000","last_undegraded":"2023-08-09T13:52:01.223231+0000","last_fullsized":"2023-08-09T13:52:01.223231+0000","mapping_epoch":113,"log_start":"0'0","ondisk_log_start":"0'0","created":113,"last_epoch_clean":114,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:51:58.125661+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:51:58.125661+0000","last_clean_scrub_stamp":"2023-08-09T13:51:58.125661+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-11T00:16:05.270843+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"15.12","version":"0'0","reported_seq":11,"reported_epoch":116,"state":"active+clean","last_fresh":"2023-08-09T13:52:01.221294+0000","last_change":"2023-08-09T13:52:00.209000+0000","last_active":"2023-08-09T13:52:01.221294+0000","last_peered":"2023-08-09T13:52:01.221294+0000","last_clean":"2023-08-09T13:52:01.221294+0000","last_became_active":"2023-08-09T13:52:00.208722+0000","last_became_peered":"2023-08-09T13:52:00.208722+0000","last_unstale":"2023-08-09T13:52:01.221294+0000","last_undegraded":"2023-08-09T13:52:01.221294+0000","last_fullsized":"2023-08-09T13:52:01.221294+0000","mapping_epoch":114,"log_start":"0'0","ondisk_log_start":"0'0","created":114,"last_epoch_clean":115,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:51:59.130555+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:51:59.130555+0000","last_clean_scrub_stamp":"2023-08-09T13:51:59.130555+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T21:54:42.771899+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"14.13","version":"115'2","reported_seq":15,"reported_epoch":116,"state":"active+clean","last_fresh":"2023-08-09T13:52:01.419233+0000","last_change":"2023-08-09T13:51:59.181856+0000","last_active":"2023-08-09T13:52:01.419233+0000","last_peered":"2023-08-09T13:52:01.419233+0000","last_clean":"2023-08-09T13:52:01.419233+0000","last_became_active":"2023-08-09T13:51:59.181568+0000","last_became_peered":"2023-08-09T13:51:59.181568+0000","last_unstale":"2023-08-09T13:52:01.419233+0000","last_undegraded":"2023-08-09T13:52:01.419233+0000","last_fullsized":"2023-08-09T13:52:01.419233+0000","mapping_epoch":113,"log_start":"0'0","ondisk_log_start":"0'0","created":113,"last_epoch_clean":114,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:51:58.125661+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:51:58.125661+0000","last_clean_scrub_stamp":"2023-08-09T13:51:58.125661+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T16:40:16.690907+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.13","version":"0'0","reported_seq":11,"reported_epoch":116,"state":"active+clean","last_fresh":"2023-08-09T13:52:01.219693+0000","last_change":"2023-08-09T13:52:00.233817+0000","last_active":"2023-08-09T13:52:01.219693+0000","last_peered":"2023-08-09T13:52:01.219693+0000","last_clean":"2023-08-09T13:52:01.219693+0000","last_became_active":"2023-08-09T13:52:00.233557+0000","last_became_peered":"2023-08-09T13:52:00.233557+0000","last_unstale":"2023-08-09T13:52:01.219693+0000","last_undegraded":"2023-08-09T13:52:01.219693+0000","last_fullsized":"2023-08-09T13:52:01.219693+0000","mapping_epoch":114,"log_start":"0'0","ondisk_log_start":"0'0","created":114,"last_epoch_clean":115,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:51:59.130555+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:51:59.130555+0000","last_clean_scrub_stamp":"2023-08-09T13:51:59.130555+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T16:20:41.956734+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.12","version":"0'0","reported_seq":13,"reported_epoch":116,"state":"active+clean","last_fresh":"2023-08-09T13:52:01.419147+0000","last_change":"2023-08-09T13:51:59.178710+0000","last_active":"2023-08-09T13:52:01.419147+0000","last_peered":"2023-08-09T13:52:01.419147+0000","last_clean":"2023-08-09T13:52:01.419147+0000","last_became_active":"2023-08-09T13:51:59.178402+0000","last_became_peered":"2023-08-09T13:51:59.178402+0000","last_unstale":"2023-08-09T13:52:01.419147+0000","last_undegraded":"2023-08-09T13:52:01.419147+0000","last_fullsized":"2023-08-09T13:52:01.419147+0000","mapping_epoch":113,"log_start":"0'0","ondisk_log_start":"0'0","created":113,"last_epoch_clean":114,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:51:58.125661+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:51:58.125661+0000","last_clean_scrub_stamp":"2023-08-09T13:51:58.125661+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T20:03:43.329471+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.10","version":"0'0","reported_seq":11,"reported_epoch":116,"state":"active+clean","last_fresh":"2023-08-09T13:52:01.221493+0000","last_change":"2023-08-09T13:52:00.223810+0000","last_active":"2023-08-09T13:52:01.221493+0000","last_peered":"2023-08-09T13:52:01.221493+0000","last_clean":"2023-08-09T13:52:01.221493+0000","last_became_active":"2023-08-09T13:52:00.223492+0000","last_became_peered":"2023-08-09T13:52:00.223492+0000","last_unstale":"2023-08-09T13:52:01.221493+0000","last_undegraded":"2023-08-09T13:52:01.221493+0000","last_fullsized":"2023-08-09T13:52:01.221493+0000","mapping_epoch":114,"log_start":"0'0","ondisk_log_start":"0'0","created":114,"last_epoch_clean":115,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:51:59.130555+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:51:59.130555+0000","last_clean_scrub_stamp":"2023-08-09T13:51:59.130555+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T15:40:03.746964+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.11","version":"0'0","reported_seq":13,"reported_epoch":116,"state":"active+clean","last_fresh":"2023-08-09T13:52:01.221690+0000","last_change":"2023-08-09T13:51:59.181121+0000","last_active":"2023-08-09T13:52:01.221690+0000","last_peered":"2023-08-09T13:52:01.221690+0000","last_clean":"2023-08-09T13:52:01.221690+0000","last_became_active":"2023-08-09T13:51:59.180844+0000","last_became_peered":"2023-08-09T13:51:59.180844+0000","last_unstale":"2023-08-09T13:52:01.221690+0000","last_undegraded":"2023-08-09T13:52:01.221690+0000","last_fullsized":"2023-08-09T13:52:01.221690+0000","mapping_epoch":113,"log_start":"0'0","ondisk_log_start":"0'0","created":113,"last_epoch_clean":114,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:51:58.125661+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:51:58.125661+0000","last_clean_scrub_stamp":"2023-08-09T13:51:58.125661+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T16:43:18.755379+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"15.11","version":"0'0","reported_seq":11,"reported_epoch":116,"state":"active+clean","last_fresh":"2023-08-09T13:52:01.418790+0000","last_change":"2023-08-09T13:52:00.216007+0000","last_active":"2023-08-09T13:52:01.418790+0000","last_peered":"2023-08-09T13:52:01.418790+0000","last_clean":"2023-08-09T13:52:01.418790+0000","last_became_active":"2023-08-09T13:52:00.215714+0000","last_became_peered":"2023-08-09T13:52:00.215714+0000","last_unstale":"2023-08-09T13:52:01.418790+0000","last_undegraded":"2023-08-09T13:52:01.418790+0000","last_fullsized":"2023-08-09T13:52:01.418790+0000","mapping_epoch":114,"log_start":"0'0","ondisk_log_start":"0'0","created":114,"last_epoch_clean":115,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:51:59.130555+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:51:59.130555+0000","last_clean_scrub_stamp":"2023-08-09T13:51:59.130555+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-11T01:27:18.440376+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"14.10","version":"115'8","reported_seq":21,"reported_epoch":116,"state":"active+clean","last_fresh":"2023-08-09T13:52:01.219639+0000","last_change":"2023-08-09T13:51:59.174790+0000","last_active":"2023-08-09T13:52:01.219639+0000","last_peered":"2023-08-09T13:52:01.219639+0000","last_clean":"2023-08-09T13:52:01.219639+0000","last_became_active":"2023-08-09T13:51:59.174453+0000","last_became_peered":"2023-08-09T13:51:59.174453+0000","last_unstale":"2023-08-09T13:52:01.219639+0000","last_undegraded":"2023-08-09T13:52:01.219639+0000","last_fullsized":"2023-08-09T13:52:01.219639+0000","mapping_epoch":113,"log_start":"0'0","ondisk_log_start":"0'0","created":113,"last_epoch_clean":114,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:51:58.125661+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:51:58.125661+0000","last_clean_scrub_stamp":"2023-08-09T13:51:58.125661+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T15:05:53.316835+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"15.6","version":"0'0","reported_seq":11,"reported_epoch":116,"state":"active+clean","last_fresh":"2023-08-09T13:52:02.227981+0000","last_change":"2023-08-09T13:52:00.217311+0000","last_active":"2023-08-09T13:52:02.227981+0000","last_peered":"2023-08-09T13:52:02.227981+0000","last_clean":"2023-08-09T13:52:02.227981+0000","last_became_active":"2023-08-09T13:52:00.216850+0000","last_became_peered":"2023-08-09T13:52:00.216850+0000","last_unstale":"2023-08-09T13:52:02.227981+0000","last_undegraded":"2023-08-09T13:52:02.227981+0000","last_fullsized":"2023-08-09T13:52:02.227981+0000","mapping_epoch":114,"log_start":"0'0","ondisk_log_start":"0'0","created":114,"last_epoch_clean":115,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:51:59.130555+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:51:59.130555+0000","last_clean_scrub_stamp":"2023-08-09T13:51:59.130555+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T20:47:55.200313+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"14.7","version":"115'2","reported_seq":15,"reported_epoch":116,"state":"active+clean","last_fresh":"2023-08-09T13:52:02.231424+0000","last_change":"2023-08-09T13:51:59.182730+0000","last_active":"2023-08-09T13:52:02.231424+0000","last_peered":"2023-08-09T13:52:02.231424+0000","last_clean":"2023-08-09T13:52:02.231424+0000","last_became_active":"2023-08-09T13:51:59.182469+0000","last_became_peered":"2023-08-09T13:51:59.182469+0000","last_unstale":"2023-08-09T13:52:02.231424+0000","last_undegraded":"2023-08-09T13:52:02.231424+0000","last_fullsized":"2023-08-09T13:52:02.231424+0000","mapping_epoch":113,"log_start":"0'0","ondisk_log_start":"0'0","created":113,"last_epoch_clean":114,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:51:58.125661+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:51:58.125661+0000","last_clean_scrub_stamp":"2023-08-09T13:51:58.125661+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T19:15:58.608074+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"15.7","version":"0'0","reported_seq":11,"reported_epoch":116,"state":"active+clean","last_fresh":"2023-08-09T13:52:01.220232+0000","last_change":"2023-08-09T13:52:00.233211+0000","last_active":"2023-08-09T13:52:01.220232+0000","last_peered":"2023-08-09T13:52:01.220232+0000","last_clean":"2023-08-09T13:52:01.220232+0000","last_became_active":"2023-08-09T13:52:00.232860+0000","last_became_peered":"2023-08-09T13:52:00.232860+0000","last_unstale":"2023-08-09T13:52:01.220232+0000","last_undegraded":"2023-08-09T13:52:01.220232+0000","last_fullsized":"2023-08-09T13:52:01.220232+0000","mapping_epoch":114,"log_start":"0'0","ondisk_log_start":"0'0","created":114,"last_epoch_clean":115,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:51:59.130555+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:51:59.130555+0000","last_clean_scrub_stamp":"2023-08-09T13:51:59.130555+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T13:55:34.217882+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.6","version":"116'1","reported_seq":14,"reported_epoch":116,"state":"active+clean","last_fresh":"2023-08-09T13:52:02.228074+0000","last_change":"2023-08-09T13:51:59.168280+0000","last_active":"2023-08-09T13:52:02.228074+0000","last_peered":"2023-08-09T13:52:02.228074+0000","last_clean":"2023-08-09T13:52:02.228074+0000","last_became_active":"2023-08-09T13:51:59.167980+0000","last_became_peered":"2023-08-09T13:51:59.167980+0000","last_unstale":"2023-08-09T13:52:02.228074+0000","last_undegraded":"2023-08-09T13:52:02.228074+0000","last_fullsized":"2023-08-09T13:52:02.228074+0000","mapping_epoch":113,"log_start":"0'0","ondisk_log_start":"0'0","created":113,"last_epoch_clean":114,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:51:58.125661+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:51:58.125661+0000","last_clean_scrub_stamp":"2023-08-09T13:51:58.125661+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T16:44:35.804569+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.4","version":"0'0","reported_seq":11,"reported_epoch":116,"state":"active+clean","last_fresh":"2023-08-09T13:52:01.219941+0000","last_change":"2023-08-09T13:52:00.228211+0000","last_active":"2023-08-09T13:52:01.219941+0000","last_peered":"2023-08-09T13:52:01.219941+0000","last_clean":"2023-08-09T13:52:01.219941+0000","last_became_active":"2023-08-09T13:52:00.227911+0000","last_became_peered":"2023-08-09T13:52:00.227911+0000","last_unstale":"2023-08-09T13:52:01.219941+0000","last_undegraded":"2023-08-09T13:52:01.219941+0000","last_fullsized":"2023-08-09T13:52:01.219941+0000","mapping_epoch":114,"log_start":"0'0","ondisk_log_start":"0'0","created":114,"last_epoch_clean":115,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:51:59.130555+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:51:59.130555+0000","last_clean_scrub_stamp":"2023-08-09T13:51:59.130555+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T19:34:08.224464+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.5","version":"0'0","reported_seq":13,"reported_epoch":116,"state":"active+clean","last_fresh":"2023-08-09T13:52:02.227620+0000","last_change":"2023-08-09T13:51:59.179649+0000","last_active":"2023-08-09T13:52:02.227620+0000","last_peered":"2023-08-09T13:52:02.227620+0000","last_clean":"2023-08-09T13:52:02.227620+0000","last_became_active":"2023-08-09T13:51:59.179348+0000","last_became_peered":"2023-08-09T13:51:59.179348+0000","last_unstale":"2023-08-09T13:52:02.227620+0000","last_undegraded":"2023-08-09T13:52:02.227620+0000","last_fullsized":"2023-08-09T13:52:02.227620+0000","mapping_epoch":113,"log_start":"0'0","ondisk_log_start":"0'0","created":113,"last_epoch_clean":114,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:51:58.125661+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:51:58.125661+0000","last_clean_scrub_stamp":"2023-08-09T13:51:58.125661+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T20:57:33.648101+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.5","version":"0'0","reported_seq":11,"reported_epoch":116,"state":"active+clean","last_fresh":"2023-08-09T13:52:02.230093+0000","last_change":"2023-08-09T13:52:00.211669+0000","last_active":"2023-08-09T13:52:02.230093+0000","last_peered":"2023-08-09T13:52:02.230093+0000","last_clean":"2023-08-09T13:52:02.230093+0000","last_became_active":"2023-08-09T13:52:00.211314+0000","last_became_peered":"2023-08-09T13:52:00.211314+0000","last_unstale":"2023-08-09T13:52:02.230093+0000","last_undegraded":"2023-08-09T13:52:02.230093+0000","last_fullsized":"2023-08-09T13:52:02.230093+0000","mapping_epoch":114,"log_start":"0'0","ondisk_log_start":"0'0","created":114,"last_epoch_clean":115,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:51:59.130555+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:51:59.130555+0000","last_clean_scrub_stamp":"2023-08-09T13:51:59.130555+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T19:08:15.523370+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"14.4","version":"0'0","reported_seq":13,"reported_epoch":116,"state":"active+clean","last_fresh":"2023-08-09T13:52:02.229733+0000","last_change":"2023-08-09T13:51:59.174311+0000","last_active":"2023-08-09T13:52:02.229733+0000","last_peered":"2023-08-09T13:52:02.229733+0000","last_clean":"2023-08-09T13:52:02.229733+0000","last_became_active":"2023-08-09T13:51:59.173990+0000","last_became_peered":"2023-08-09T13:51:59.173990+0000","last_unstale":"2023-08-09T13:52:02.229733+0000","last_undegraded":"2023-08-09T13:52:02.229733+0000","last_fullsized":"2023-08-09T13:52:02.229733+0000","mapping_epoch":113,"log_start":"0'0","ondisk_log_start":"0'0","created":113,"last_epoch_clean":114,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:51:58.125661+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:51:58.125661+0000","last_clean_scrub_stamp":"2023-08-09T13:51:58.125661+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T21:53:48.212015+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.2","version":"0'0","reported_seq":11,"reported_epoch":116,"state":"active+clean","last_fresh":"2023-08-09T13:52:01.419005+0000","last_change":"2023-08-09T13:52:00.214396+0000","last_active":"2023-08-09T13:52:01.419005+0000","last_peered":"2023-08-09T13:52:01.419005+0000","last_clean":"2023-08-09T13:52:01.419005+0000","last_became_active":"2023-08-09T13:52:00.213395+0000","last_became_peered":"2023-08-09T13:52:00.213395+0000","last_unstale":"2023-08-09T13:52:01.419005+0000","last_undegraded":"2023-08-09T13:52:01.419005+0000","last_fullsized":"2023-08-09T13:52:01.419005+0000","mapping_epoch":114,"log_start":"0'0","ondisk_log_start":"0'0","created":114,"last_epoch_clean":115,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:51:59.130555+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:51:59.130555+0000","last_clean_scrub_stamp":"2023-08-09T13:51:59.130555+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T14:47:29.943812+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"14.3","version":"0'0","reported_seq":13,"reported_epoch":116,"state":"active+clean","last_fresh":"2023-08-09T13:52:02.227823+0000","last_change":"2023-08-09T13:51:59.167523+0000","last_active":"2023-08-09T13:52:02.227823+0000","last_peered":"2023-08-09T13:52:02.227823+0000","last_clean":"2023-08-09T13:52:02.227823+0000","last_became_active":"2023-08-09T13:51:59.167059+0000","last_became_peered":"2023-08-09T13:51:59.167059+0000","last_unstale":"2023-08-09T13:52:02.227823+0000","last_undegraded":"2023-08-09T13:52:02.227823+0000","last_fullsized":"2023-08-09T13:52:02.227823+0000","mapping_epoch":113,"log_start":"0'0","ondisk_log_start":"0'0","created":113,"last_epoch_clean":114,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:51:58.125661+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:51:58.125661+0000","last_clean_scrub_stamp":"2023-08-09T13:51:58.125661+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T20:28:55.377939+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.3","version":"0'0","reported_seq":11,"reported_epoch":116,"state":"active+clean","last_fresh":"2023-08-09T13:52:01.222209+0000","last_change":"2023-08-09T13:52:00.220968+0000","last_active":"2023-08-09T13:52:01.222209+0000","last_peered":"2023-08-09T13:52:01.222209+0000","last_clean":"2023-08-09T13:52:01.222209+0000","last_became_active":"2023-08-09T13:52:00.220710+0000","last_became_peered":"2023-08-09T13:52:00.220710+0000","last_unstale":"2023-08-09T13:52:01.222209+0000","last_undegraded":"2023-08-09T13:52:01.222209+0000","last_fullsized":"2023-08-09T13:52:01.222209+0000","mapping_epoch":114,"log_start":"0'0","ondisk_log_start":"0'0","created":114,"last_epoch_clean":115,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:51:59.130555+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:51:59.130555+0000","last_clean_scrub_stamp":"2023-08-09T13:51:59.130555+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T19:42:07.385997+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.2","version":"0'0","reported_seq":13,"reported_epoch":116,"state":"active+clean","last_fresh":"2023-08-09T13:52:02.228985+0000","last_change":"2023-08-09T13:51:59.176919+0000","last_active":"2023-08-09T13:52:02.228985+0000","last_peered":"2023-08-09T13:52:02.228985+0000","last_clean":"2023-08-09T13:52:02.228985+0000","last_became_active":"2023-08-09T13:51:59.176503+0000","last_became_peered":"2023-08-09T13:51:59.176503+0000","last_unstale":"2023-08-09T13:52:02.228985+0000","last_undegraded":"2023-08-09T13:52:02.228985+0000","last_fullsized":"2023-08-09T13:52:02.228985+0000","mapping_epoch":113,"log_start":"0'0","ondisk_log_start":"0'0","created":113,"last_epoch_clean":114,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:51:58.125661+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:51:58.125661+0000","last_clean_scrub_stamp":"2023-08-09T13:51:58.125661+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T21:36:20.093982+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.0","version":"0'0","reported_seq":11,"reported_epoch":116,"state":"active+clean","last_fresh":"2023-08-09T13:52:01.418787+0000","last_change":"2023-08-09T13:52:00.216790+0000","last_active":"2023-08-09T13:52:01.418787+0000","last_peered":"2023-08-09T13:52:01.418787+0000","last_clean":"2023-08-09T13:52:01.418787+0000","last_became_active":"2023-08-09T13:52:00.216456+0000","last_became_peered":"2023-08-09T13:52:00.216456+0000","last_unstale":"2023-08-09T13:52:01.418787+0000","last_undegraded":"2023-08-09T13:52:01.418787+0000","last_fullsized":"2023-08-09T13:52:01.418787+0000","mapping_epoch":114,"log_start":"0'0","ondisk_log_start":"0'0","created":114,"last_epoch_clean":115,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:51:59.130555+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:51:59.130555+0000","last_clean_scrub_stamp":"2023-08-09T13:51:59.130555+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T15:56:02.893523+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"14.1","version":"0'0","reported_seq":13,"reported_epoch":116,"state":"active+clean","last_fresh":"2023-08-09T13:52:01.221369+0000","last_change":"2023-08-09T13:51:59.171070+0000","last_active":"2023-08-09T13:52:01.221369+0000","last_peered":"2023-08-09T13:52:01.221369+0000","last_clean":"2023-08-09T13:52:01.221369+0000","last_became_active":"2023-08-09T13:51:59.170788+0000","last_became_peered":"2023-08-09T13:51:59.170788+0000","last_unstale":"2023-08-09T13:52:01.221369+0000","last_undegraded":"2023-08-09T13:52:01.221369+0000","last_fullsized":"2023-08-09T13:52:01.221369+0000","mapping_epoch":113,"log_start":"0'0","ondisk_log_start":"0'0","created":113,"last_epoch_clean":114,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:51:58.125661+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:51:58.125661+0000","last_clean_scrub_stamp":"2023-08-09T13:51:58.125661+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T20:53:08.597878+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"15.1","version":"0'0","reported_seq":11,"reported_epoch":116,"state":"active+clean","last_fresh":"2023-08-09T13:52:02.229115+0000","last_change":"2023-08-09T13:52:00.222037+0000","last_active":"2023-08-09T13:52:02.229115+0000","last_peered":"2023-08-09T13:52:02.229115+0000","last_clean":"2023-08-09T13:52:02.229115+0000","last_became_active":"2023-08-09T13:52:00.217448+0000","last_became_peered":"2023-08-09T13:52:00.217448+0000","last_unstale":"2023-08-09T13:52:02.229115+0000","last_undegraded":"2023-08-09T13:52:02.229115+0000","last_fullsized":"2023-08-09T13:52:02.229115+0000","mapping_epoch":114,"log_start":"0'0","ondisk_log_start":"0'0","created":114,"last_epoch_clean":115,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:51:59.130555+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:51:59.130555+0000","last_clean_scrub_stamp":"2023-08-09T13:51:59.130555+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T21:14:10.012130+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"14.0","version":"0'0","reported_seq":13,"reported_epoch":116,"state":"active+clean","last_fresh":"2023-08-09T13:52:01.418966+0000","last_change":"2023-08-09T13:51:59.173223+0000","last_active":"2023-08-09T13:52:01.418966+0000","last_peered":"2023-08-09T13:52:01.418966+0000","last_clean":"2023-08-09T13:52:01.418966+0000","last_became_active":"2023-08-09T13:51:59.172948+0000","last_became_peered":"2023-08-09T13:51:59.172948+0000","last_unstale":"2023-08-09T13:52:01.418966+0000","last_undegraded":"2023-08-09T13:52:01.418966+0000","last_fullsized":"2023-08-09T13:52:01.418966+0000","mapping_epoch":113,"log_start":"0'0","ondisk_log_start":"0'0","created":113,"last_epoch_clean":114,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:51:58.125661+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:51:58.125661+0000","last_clean_scrub_stamp":"2023-08-09T13:51:58.125661+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T23:28:53.795333+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.e","version":"0'0","reported_seq":11,"reported_epoch":116,"state":"active+clean","last_fresh":"2023-08-09T13:52:01.419454+0000","last_change":"2023-08-09T13:52:00.210798+0000","last_active":"2023-08-09T13:52:01.419454+0000","last_peered":"2023-08-09T13:52:01.419454+0000","last_clean":"2023-08-09T13:52:01.419454+0000","last_became_active":"2023-08-09T13:52:00.210358+0000","last_became_peered":"2023-08-09T13:52:00.210358+0000","last_unstale":"2023-08-09T13:52:01.419454+0000","last_undegraded":"2023-08-09T13:52:01.419454+0000","last_fullsized":"2023-08-09T13:52:01.419454+0000","mapping_epoch":114,"log_start":"0'0","ondisk_log_start":"0'0","created":114,"last_epoch_clean":115,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:51:59.130555+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:51:59.130555+0000","last_clean_scrub_stamp":"2023-08-09T13:51:59.130555+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T16:59:58.856242+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"14.f","version":"115'1","reported_seq":14,"reported_epoch":116,"state":"active+clean","last_fresh":"2023-08-09T13:52:02.227196+0000","last_change":"2023-08-09T13:51:59.174685+0000","last_active":"2023-08-09T13:52:02.227196+0000","last_peered":"2023-08-09T13:52:02.227196+0000","last_clean":"2023-08-09T13:52:02.227196+0000","last_became_active":"2023-08-09T13:51:59.174370+0000","last_became_peered":"2023-08-09T13:51:59.174370+0000","last_unstale":"2023-08-09T13:52:02.227196+0000","last_undegraded":"2023-08-09T13:52:02.227196+0000","last_fullsized":"2023-08-09T13:52:02.227196+0000","mapping_epoch":113,"log_start":"0'0","ondisk_log_start":"0'0","created":113,"last_epoch_clean":114,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:51:58.125661+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:51:58.125661+0000","last_clean_scrub_stamp":"2023-08-09T13:51:58.125661+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T20:53:08.597878+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":193,"reported_epoch":116,"state":"active+clean","last_fresh":"2023-08-09T13:52:02.228680+0000","last_change":"2023-08-09T13:33:56.401406+0000","last_active":"2023-08-09T13:52:02.228680+0000","last_peered":"2023-08-09T13:52:02.228680+0000","last_clean":"2023-08-09T13:52:02.228680+0000","last_became_active":"2023-08-09T13:33:56.400487+0000","last_became_peered":"2023-08-09T13:33:56.400487+0000","last_unstale":"2023-08-09T13:52:02.228680+0000","last_undegraded":"2023-08-09T13:52:02.228680+0000","last_fullsized":"2023-08-09T13:52:02.228680+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:33:54.845869+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:33:54.845869+0000","last_clean_scrub_stamp":"2023-08-09T13:33:54.845869+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T15:15:21.526034+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"15.f","version":"0'0","reported_seq":11,"reported_epoch":116,"state":"active+clean","last_fresh":"2023-08-09T13:52:02.230093+0000","last_change":"2023-08-09T13:52:00.217897+0000","last_active":"2023-08-09T13:52:02.230093+0000","last_peered":"2023-08-09T13:52:02.230093+0000","last_clean":"2023-08-09T13:52:02.230093+0000","last_became_active":"2023-08-09T13:52:00.217627+0000","last_became_peered":"2023-08-09T13:52:00.217627+0000","last_unstale":"2023-08-09T13:52:02.230093+0000","last_undegraded":"2023-08-09T13:52:02.230093+0000","last_fullsized":"2023-08-09T13:52:02.230093+0000","mapping_epoch":114,"log_start":"0'0","ondisk_log_start":"0'0","created":114,"last_epoch_clean":115,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:51:59.130555+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:51:59.130555+0000","last_clean_scrub_stamp":"2023-08-09T13:51:59.130555+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T16:54:25.108485+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"14.e","version":"115'2","reported_seq":15,"reported_epoch":116,"state":"active+clean","last_fresh":"2023-08-09T13:52:01.223136+0000","last_change":"2023-08-09T13:51:59.180623+0000","last_active":"2023-08-09T13:52:01.223136+0000","last_peered":"2023-08-09T13:52:01.223136+0000","last_clean":"2023-08-09T13:52:01.223136+0000","last_became_active":"2023-08-09T13:51:59.178785+0000","last_became_peered":"2023-08-09T13:51:59.178785+0000","last_unstale":"2023-08-09T13:52:01.223136+0000","last_undegraded":"2023-08-09T13:52:01.223136+0000","last_fullsized":"2023-08-09T13:52:01.223136+0000","mapping_epoch":113,"log_start":"0'0","ondisk_log_start":"0'0","created":113,"last_epoch_clean":114,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:51:58.125661+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:51:58.125661+0000","last_clean_scrub_stamp":"2023-08-09T13:51:58.125661+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T19:33:51.888915+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":197,"reported_epoch":116,"state":"active+clean","last_fresh":"2023-08-09T13:52:02.229066+0000","last_change":"2023-08-09T13:33:56.401123+0000","last_active":"2023-08-09T13:52:02.229066+0000","last_peered":"2023-08-09T13:52:02.229066+0000","last_clean":"2023-08-09T13:52:02.229066+0000","last_became_active":"2023-08-09T13:33:56.399933+0000","last_became_peered":"2023-08-09T13:33:56.399933+0000","last_unstale":"2023-08-09T13:52:02.229066+0000","last_undegraded":"2023-08-09T13:52:02.229066+0000","last_fullsized":"2023-08-09T13:52:02.229066+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:33:54.845869+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:33:54.845869+0000","last_clean_scrub_stamp":"2023-08-09T13:33:54.845869+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T16:13:52.660079+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"15.c","version":"0'0","reported_seq":11,"reported_epoch":116,"state":"active+clean","last_fresh":"2023-08-09T13:52:02.227915+0000","last_change":"2023-08-09T13:52:00.220774+0000","last_active":"2023-08-09T13:52:02.227915+0000","last_peered":"2023-08-09T13:52:02.227915+0000","last_clean":"2023-08-09T13:52:02.227915+0000","last_became_active":"2023-08-09T13:52:00.220505+0000","last_became_peered":"2023-08-09T13:52:00.220505+0000","last_unstale":"2023-08-09T13:52:02.227915+0000","last_undegraded":"2023-08-09T13:52:02.227915+0000","last_fullsized":"2023-08-09T13:52:02.227915+0000","mapping_epoch":114,"log_start":"0'0","ondisk_log_start":"0'0","created":114,"last_epoch_clean":115,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:51:59.130555+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:51:59.130555+0000","last_clean_scrub_stamp":"2023-08-09T13:51:59.130555+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T22:48:53.283499+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"14.d","version":"115'1","reported_seq":14,"reported_epoch":116,"state":"active+clean","last_fresh":"2023-08-09T13:52:02.229861+0000","last_change":"2023-08-09T13:51:59.174909+0000","last_active":"2023-08-09T13:52:02.229861+0000","last_peered":"2023-08-09T13:52:02.229861+0000","last_clean":"2023-08-09T13:52:02.229861+0000","last_became_active":"2023-08-09T13:51:59.174527+0000","last_became_peered":"2023-08-09T13:51:59.174527+0000","last_unstale":"2023-08-09T13:52:02.229861+0000","last_undegraded":"2023-08-09T13:52:02.229861+0000","last_fullsized":"2023-08-09T13:52:02.229861+0000","mapping_epoch":113,"log_start":"0'0","ondisk_log_start":"0'0","created":113,"last_epoch_clean":114,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:51:58.125661+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:51:58.125661+0000","last_clean_scrub_stamp":"2023-08-09T13:51:58.125661+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T14:11:59.228722+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"1.2","version":"0'0","reported_seq":193,"reported_epoch":116,"state":"active+clean","last_fresh":"2023-08-09T13:52:02.228361+0000","last_change":"2023-08-09T13:33:56.402794+0000","last_active":"2023-08-09T13:52:02.228361+0000","last_peered":"2023-08-09T13:52:02.228361+0000","last_clean":"2023-08-09T13:52:02.228361+0000","last_became_active":"2023-08-09T13:33:56.402038+0000","last_became_peered":"2023-08-09T13:33:56.402038+0000","last_unstale":"2023-08-09T13:52:02.228361+0000","last_undegraded":"2023-08-09T13:52:02.228361+0000","last_fullsized":"2023-08-09T13:52:02.228361+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:33:54.845869+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:33:54.845869+0000","last_clean_scrub_stamp":"2023-08-09T13:33:54.845869+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T13:49:39.997653+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"15.d","version":"0'0","reported_seq":11,"reported_epoch":116,"state":"active+clean","last_fresh":"2023-08-09T13:52:02.227529+0000","last_change":"2023-08-09T13:52:00.219473+0000","last_active":"2023-08-09T13:52:02.227529+0000","last_peered":"2023-08-09T13:52:02.227529+0000","last_clean":"2023-08-09T13:52:02.227529+0000","last_became_active":"2023-08-09T13:52:00.218836+0000","last_became_peered":"2023-08-09T13:52:00.218836+0000","last_unstale":"2023-08-09T13:52:02.227529+0000","last_undegraded":"2023-08-09T13:52:02.227529+0000","last_fullsized":"2023-08-09T13:52:02.227529+0000","mapping_epoch":114,"log_start":"0'0","ondisk_log_start":"0'0","created":114,"last_epoch_clean":115,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:51:59.130555+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:51:59.130555+0000","last_clean_scrub_stamp":"2023-08-09T13:51:59.130555+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T23:42:47.388329+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"14.c","version":"0'0","reported_seq":13,"reported_epoch":116,"state":"active+clean","last_fresh":"2023-08-09T13:52:02.227506+0000","last_change":"2023-08-09T13:51:59.183702+0000","last_active":"2023-08-09T13:52:02.227506+0000","last_peered":"2023-08-09T13:52:02.227506+0000","last_clean":"2023-08-09T13:52:02.227506+0000","last_became_active":"2023-08-09T13:51:59.183387+0000","last_became_peered":"2023-08-09T13:51:59.183387+0000","last_unstale":"2023-08-09T13:52:02.227506+0000","last_undegraded":"2023-08-09T13:52:02.227506+0000","last_fullsized":"2023-08-09T13:52:02.227506+0000","mapping_epoch":113,"log_start":"0'0","ondisk_log_start":"0'0","created":113,"last_epoch_clean":114,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:51:58.125661+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:51:58.125661+0000","last_clean_scrub_stamp":"2023-08-09T13:51:58.125661+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T16:41:06.536543+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.3","version":"0'0","reported_seq":193,"reported_epoch":116,"state":"active+clean","last_fresh":"2023-08-09T13:52:01.219784+0000","last_change":"2023-08-09T13:33:55.886749+0000","last_active":"2023-08-09T13:52:01.219784+0000","last_peered":"2023-08-09T13:52:01.219784+0000","last_clean":"2023-08-09T13:52:01.219784+0000","last_became_active":"2023-08-09T13:33:55.886116+0000","last_became_peered":"2023-08-09T13:33:55.886116+0000","last_unstale":"2023-08-09T13:52:01.219784+0000","last_undegraded":"2023-08-09T13:52:01.219784+0000","last_fullsized":"2023-08-09T13:52:01.219784+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:33:54.845869+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:33:54.845869+0000","last_clean_scrub_stamp":"2023-08-09T13:33:54.845869+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T23:07:12.228212+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"105'186","reported_seq":587,"reported_epoch":116,"state":"active+clean","last_fresh":"2023-08-09T13:52:02.227119+0000","last_change":"2023-08-09T13:33:55.886345+0000","last_active":"2023-08-09T13:52:02.227119+0000","last_peered":"2023-08-09T13:52:02.227119+0000","last_clean":"2023-08-09T13:52:02.227119+0000","last_became_active":"2023-08-09T13:33:55.885781+0000","last_became_peered":"2023-08-09T13:33:55.885781+0000","last_unstale":"2023-08-09T13:52:02.227119+0000","last_undegraded":"2023-08-09T13:52:02.227119+0000","last_fullsized":"2023-08-09T13:52:02.227119+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:33:54.845869+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:33:54.845869+0000","last_clean_scrub_stamp":"2023-08-09T13:33:54.845869+0000","objects_scrubbed":0,"log_size":186,"log_dups_size":0,"ondisk_log_size":186,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T19:31:45.934609+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":715,"num_read_kb":1418,"num_write":283,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"15.a","version":"0'0","reported_seq":11,"reported_epoch":116,"state":"active+clean","last_fresh":"2023-08-09T13:52:01.419121+0000","last_change":"2023-08-09T13:52:00.210575+0000","last_active":"2023-08-09T13:52:01.419121+0000","last_peered":"2023-08-09T13:52:01.419121+0000","last_clean":"2023-08-09T13:52:01.419121+0000","last_became_active":"2023-08-09T13:52:00.210182+0000","last_became_peered":"2023-08-09T13:52:00.210182+0000","last_unstale":"2023-08-09T13:52:01.419121+0000","last_undegraded":"2023-08-09T13:52:01.419121+0000","last_fullsized":"2023-08-09T13:52:01.419121+0000","mapping_epoch":114,"log_start":"0'0","ondisk_log_start":"0'0","created":114,"last_epoch_clean":115,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:51:59.130555+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:51:59.130555+0000","last_clean_scrub_stamp":"2023-08-09T13:51:59.130555+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T17:12:07.979144+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"14.b","version":"115'2","reported_seq":15,"reported_epoch":116,"state":"active+clean","last_fresh":"2023-08-09T13:52:02.227580+0000","last_change":"2023-08-09T13:51:59.168603+0000","last_active":"2023-08-09T13:52:02.227580+0000","last_peered":"2023-08-09T13:52:02.227580+0000","last_clean":"2023-08-09T13:52:02.227580+0000","last_became_active":"2023-08-09T13:51:59.168273+0000","last_became_peered":"2023-08-09T13:51:59.168273+0000","last_unstale":"2023-08-09T13:52:02.227580+0000","last_undegraded":"2023-08-09T13:52:02.227580+0000","last_fullsized":"2023-08-09T13:52:02.227580+0000","mapping_epoch":113,"log_start":"0'0","ondisk_log_start":"0'0","created":113,"last_epoch_clean":114,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:51:58.125661+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:51:58.125661+0000","last_clean_scrub_stamp":"2023-08-09T13:51:58.125661+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T14:50:42.716966+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":203,"reported_epoch":116,"state":"active+clean","last_fresh":"2023-08-09T13:52:01.221680+0000","last_change":"2023-08-09T13:33:55.889565+0000","last_active":"2023-08-09T13:52:01.221680+0000","last_peered":"2023-08-09T13:52:01.221680+0000","last_clean":"2023-08-09T13:52:01.221680+0000","last_became_active":"2023-08-09T13:33:55.889164+0000","last_became_peered":"2023-08-09T13:33:55.889164+0000","last_unstale":"2023-08-09T13:52:01.221680+0000","last_undegraded":"2023-08-09T13:52:01.221680+0000","last_fullsized":"2023-08-09T13:52:01.221680+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:33:54.845869+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:33:54.845869+0000","last_clean_scrub_stamp":"2023-08-09T13:33:54.845869+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T22:32:12.354125+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"15.b","version":"0'0","reported_seq":11,"reported_epoch":116,"state":"active+clean","last_fresh":"2023-08-09T13:52:01.419072+0000","last_change":"2023-08-09T13:52:00.214910+0000","last_active":"2023-08-09T13:52:01.419072+0000","last_peered":"2023-08-09T13:52:01.419072+0000","last_clean":"2023-08-09T13:52:01.419072+0000","last_became_active":"2023-08-09T13:52:00.214685+0000","last_became_peered":"2023-08-09T13:52:00.214685+0000","last_unstale":"2023-08-09T13:52:01.419072+0000","last_undegraded":"2023-08-09T13:52:01.419072+0000","last_fullsized":"2023-08-09T13:52:01.419072+0000","mapping_epoch":114,"log_start":"0'0","ondisk_log_start":"0'0","created":114,"last_epoch_clean":115,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:51:59.130555+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:51:59.130555+0000","last_clean_scrub_stamp":"2023-08-09T13:51:59.130555+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T22:56:47.012795+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"14.a","version":"0'0","reported_seq":13,"reported_epoch":116,"state":"active+clean","last_fresh":"2023-08-09T13:52:02.228593+0000","last_change":"2023-08-09T13:51:59.179151+0000","last_active":"2023-08-09T13:52:02.228593+0000","last_peered":"2023-08-09T13:52:02.228593+0000","last_clean":"2023-08-09T13:52:02.228593+0000","last_became_active":"2023-08-09T13:51:59.178247+0000","last_became_peered":"2023-08-09T13:51:59.178247+0000","last_unstale":"2023-08-09T13:52:02.228593+0000","last_undegraded":"2023-08-09T13:52:02.228593+0000","last_fullsized":"2023-08-09T13:52:02.228593+0000","mapping_epoch":113,"log_start":"0'0","ondisk_log_start":"0'0","created":113,"last_epoch_clean":114,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:51:58.125661+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:51:58.125661+0000","last_clean_scrub_stamp":"2023-08-09T13:51:58.125661+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T15:49:27.949808+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":197,"reported_epoch":116,"state":"active+clean","last_fresh":"2023-08-09T13:52:01.223079+0000","last_change":"2023-08-09T13:33:56.404236+0000","last_active":"2023-08-09T13:52:01.223079+0000","last_peered":"2023-08-09T13:52:01.223079+0000","last_clean":"2023-08-09T13:52:01.223079+0000","last_became_active":"2023-08-09T13:33:56.402485+0000","last_became_peered":"2023-08-09T13:33:56.402485+0000","last_unstale":"2023-08-09T13:52:01.223079+0000","last_undegraded":"2023-08-09T13:52:01.223079+0000","last_fullsized":"2023-08-09T13:52:01.223079+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:33:54.845869+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:33:54.845869+0000","last_clean_scrub_stamp":"2023-08-09T13:33:54.845869+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T16:52:26.468612+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"15.8","version":"0'0","reported_seq":11,"reported_epoch":116,"state":"active+clean","last_fresh":"2023-08-09T13:52:01.418796+0000","last_change":"2023-08-09T13:52:00.210861+0000","last_active":"2023-08-09T13:52:01.418796+0000","last_peered":"2023-08-09T13:52:01.418796+0000","last_clean":"2023-08-09T13:52:01.418796+0000","last_became_active":"2023-08-09T13:52:00.210529+0000","last_became_peered":"2023-08-09T13:52:00.210529+0000","last_unstale":"2023-08-09T13:52:01.418796+0000","last_undegraded":"2023-08-09T13:52:01.418796+0000","last_fullsized":"2023-08-09T13:52:01.418796+0000","mapping_epoch":114,"log_start":"0'0","ondisk_log_start":"0'0","created":114,"last_epoch_clean":115,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:51:59.130555+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:51:59.130555+0000","last_clean_scrub_stamp":"2023-08-09T13:51:59.130555+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T23:45:14.245991+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"14.9","version":"0'0","reported_seq":13,"reported_epoch":116,"state":"active+clean","last_fresh":"2023-08-09T13:52:02.228770+0000","last_change":"2023-08-09T13:51:59.177964+0000","last_active":"2023-08-09T13:52:02.228770+0000","last_peered":"2023-08-09T13:52:02.228770+0000","last_clean":"2023-08-09T13:52:02.228770+0000","last_became_active":"2023-08-09T13:51:59.177186+0000","last_became_peered":"2023-08-09T13:51:59.177186+0000","last_unstale":"2023-08-09T13:52:02.228770+0000","last_undegraded":"2023-08-09T13:52:02.228770+0000","last_fullsized":"2023-08-09T13:52:02.228770+0000","mapping_epoch":113,"log_start":"0'0","ondisk_log_start":"0'0","created":113,"last_epoch_clean":114,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:51:58.125661+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:51:58.125661+0000","last_clean_scrub_stamp":"2023-08-09T13:51:58.125661+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T14:17:42.189888+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":193,"reported_epoch":116,"state":"active+clean","last_fresh":"2023-08-09T13:52:02.229888+0000","last_change":"2023-08-09T13:33:56.401781+0000","last_active":"2023-08-09T13:52:02.229888+0000","last_peered":"2023-08-09T13:52:02.229888+0000","last_clean":"2023-08-09T13:52:02.229888+0000","last_became_active":"2023-08-09T13:33:56.400336+0000","last_became_peered":"2023-08-09T13:33:56.400336+0000","last_unstale":"2023-08-09T13:52:02.229888+0000","last_undegraded":"2023-08-09T13:52:02.229888+0000","last_fullsized":"2023-08-09T13:52:02.229888+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:33:54.845869+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:33:54.845869+0000","last_clean_scrub_stamp":"2023-08-09T13:33:54.845869+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T21:34:19.616633+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.9","version":"0'0","reported_seq":11,"reported_epoch":116,"state":"active+clean","last_fresh":"2023-08-09T13:52:02.228771+0000","last_change":"2023-08-09T13:52:00.222392+0000","last_active":"2023-08-09T13:52:02.228771+0000","last_peered":"2023-08-09T13:52:02.228771+0000","last_clean":"2023-08-09T13:52:02.228771+0000","last_became_active":"2023-08-09T13:52:00.222090+0000","last_became_peered":"2023-08-09T13:52:00.222090+0000","last_unstale":"2023-08-09T13:52:02.228771+0000","last_undegraded":"2023-08-09T13:52:02.228771+0000","last_fullsized":"2023-08-09T13:52:02.228771+0000","mapping_epoch":114,"log_start":"0'0","ondisk_log_start":"0'0","created":114,"last_epoch_clean":115,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:51:59.130555+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:51:59.130555+0000","last_clean_scrub_stamp":"2023-08-09T13:51:59.130555+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T22:13:05.076442+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"14.8","version":"0'0","reported_seq":13,"reported_epoch":116,"state":"active+clean","last_fresh":"2023-08-09T13:52:02.227331+0000","last_change":"2023-08-09T13:51:59.170689+0000","last_active":"2023-08-09T13:52:02.227331+0000","last_peered":"2023-08-09T13:52:02.227331+0000","last_clean":"2023-08-09T13:52:02.227331+0000","last_became_active":"2023-08-09T13:51:59.170409+0000","last_became_peered":"2023-08-09T13:51:59.170409+0000","last_unstale":"2023-08-09T13:52:02.227331+0000","last_undegraded":"2023-08-09T13:52:02.227331+0000","last_fullsized":"2023-08-09T13:52:02.227331+0000","mapping_epoch":113,"log_start":"0'0","ondisk_log_start":"0'0","created":113,"last_epoch_clean":114,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:51:58.125661+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:51:58.125661+0000","last_clean_scrub_stamp":"2023-08-09T13:51:58.125661+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-11T01:23:51.545812+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":193,"reported_epoch":116,"state":"active+clean","last_fresh":"2023-08-09T13:52:02.230489+0000","last_change":"2023-08-09T13:33:55.886389+0000","last_active":"2023-08-09T13:52:02.230489+0000","last_peered":"2023-08-09T13:52:02.230489+0000","last_clean":"2023-08-09T13:52:02.230489+0000","last_became_active":"2023-08-09T13:33:55.885638+0000","last_became_peered":"2023-08-09T13:33:55.885638+0000","last_unstale":"2023-08-09T13:52:02.230489+0000","last_undegraded":"2023-08-09T13:52:02.230489+0000","last_fullsized":"2023-08-09T13:52:02.230489+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:33:54.845869+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:33:54.845869+0000","last_clean_scrub_stamp":"2023-08-09T13:33:54.845869+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T19:59:29.993498+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]}],"pool_stats":[{"poolid":15,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":2,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":715,"num_read_kb":1418,"num_write":283,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":186,"ondisk_log_size":186,"up":2,"acting":2,"num_store_stats":2},{"poolid":14,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":39,"ondisk_log_size":39,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":8}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834575065,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":37196,"kb_used_data":1984,"kb_used_omap":1,"kb_used_meta":35198,"kb_avail":94334644,"statfs":{"total":96636764160,"available":96598675456,"internally_reserved":0,"allocated":2031616,"data_stored":1352999,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1781,"internal_metadata":36043019},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Aug 9 13:51:20 2023","interfaces":[{"interface":"back","average":{"1min":1.101,"5min":1.1910000000000001,"15min":1.2370000000000001},"min":{"1min":0.61599999999999999,"5min":0.61599999999999999,"15min":0.61599999999999999},"max":{"1min":1.764,"5min":1.8919999999999999,"15min":2.1280000000000001},"last":0.72599999999999998},{"interface":"front","average":{"1min":1.153,"5min":1.224,"15min":1.2430000000000001},"min":{"1min":0.64200000000000002,"5min":0.64200000000000002,"15min":0.64200000000000002},"max":{"1min":1.8839999999999999,"5min":1.8839999999999999,"15min":2.0099999999999998},"last":1.3500000000000001}]},{"osd":1,"last update":"Wed Aug 9 13:51:20 2023","interfaces":[{"interface":"back","average":{"1min":1.1830000000000001,"5min":1.22,"15min":1.26},"min":{"1min":0.69199999999999995,"5min":0.69199999999999995,"15min":0.59799999999999998},"max":{"1min":1.8120000000000001,"5min":1.8120000000000001,"15min":1.9470000000000001},"last":1.0189999999999999},{"interface":"front","average":{"1min":1.1839999999999999,"5min":1.2310000000000001,"15min":1.2809999999999999},"min":{"1min":0.69099999999999995,"5min":0.59299999999999997,"15min":0.59299999999999997},"max":{"1min":1.998,"5min":1.998,"15min":1.998},"last":0.97999999999999998}]},{"osd":2,"last update":"Wed Aug 9 13:51:20 2023","interfaces":[{"interface":"back","average":{"1min":1.1990000000000001,"5min":1.234,"15min":1.252},"min":{"1min":0.79100000000000004,"5min":0.65700000000000003,"15min":0.65700000000000003},"max":{"1min":1.6539999999999999,"5min":2.0129999999999999,"15min":2.0129999999999999},"last":1.1619999999999999},{"interface":"front","average":{"1min":1.22,"5min":1.2829999999999999,"15min":1.369},"min":{"1min":0.58199999999999996,"5min":0.58199999999999996,"15min":0.58199999999999996},"max":{"1min":1.7050000000000001,"5min":1.8740000000000001,"15min":2.4140000000000001},"last":0.91300000000000003}]},{"osd":3,"last update":"Wed Aug 9 13:51:20 2023","interfaces":[{"interface":"back","average":{"1min":1.272,"5min":1.29,"15min":1.3360000000000001},"min":{"1min":0.67500000000000004,"5min":0.67500000000000004,"15min":0.67500000000000004},"max":{"1min":1.8140000000000001,"5min":1.97,"15min":1.97},"last":1.5209999999999999},{"interface":"front","average":{"1min":1.363,"5min":1.3959999999999999,"15min":1.413},"min":{"1min":0.71499999999999997,"5min":0.71499999999999997,"15min":0.71499999999999997},"max":{"1min":1.615,"5min":2.0790000000000002,"15min":2.2480000000000002},"last":1.125}]},{"osd":4,"last update":"Wed Aug 9 13:51:20 2023","interfaces":[{"interface":"back","average":{"1min":1.2430000000000001,"5min":1.2669999999999999,"15min":1.2749999999999999},"min":{"1min":0.74299999999999999,"5min":0.59499999999999997,"15min":0.59499999999999997},"max":{"1min":2.097,"5min":2.097,"15min":2.343},"last":1.2350000000000001},{"interface":"front","average":{"1min":1.2509999999999999,"5min":1.2969999999999999,"15min":1.3340000000000001},"min":{"1min":0.60799999999999998,"5min":0.60799999999999998,"15min":0.60799999999999998},"max":{"1min":2.0539999999999998,"5min":2.1419999999999999,"15min":2.1419999999999999},"last":1.6399999999999999}]},{"osd":5,"last update":"Wed Aug 9 13:51:20 2023","interfaces":[{"interface":"back","average":{"1min":1.456,"5min":1.4350000000000001,"15min":1.4179999999999999},"min":{"1min":0.65800000000000003,"5min":0.65800000000000003,"15min":0.65800000000000003},"max":{"1min":1.7549999999999999,"5min":2.1480000000000001,"15min":2.6749999999999998},"last":1.2709999999999999},{"interface":"front","average":{"1min":1.383,"5min":1.4099999999999999,"15min":1.4450000000000001},"min":{"1min":0.83999999999999997,"5min":0.83999999999999997,"15min":0.79700000000000004},"max":{"1min":2.157,"5min":2.157,"15min":2.9569999999999999},"last":1.5660000000000001}]},{"osd":6,"last update":"Wed Aug 9 13:51:20 2023","interfaces":[{"interface":"back","average":{"1min":1.4650000000000001,"5min":1.504,"15min":1.5429999999999999},"min":{"1min":0.88500000000000001,"5min":0.85499999999999998,"15min":0.85499999999999998},"max":{"1min":1.8280000000000001,"5min":2.2069999999999999,"15min":2.8149999999999999},"last":1.464},{"interface":"front","average":{"1min":1.421,"5min":1.4430000000000001,"15min":1.474},"min":{"1min":0.79700000000000004,"5min":0.79700000000000004,"15min":0.79700000000000004},"max":{"1min":1.736,"5min":2.1320000000000001,"15min":2.2530000000000001},"last":1.7110000000000001}]}]},{"osd":6,"up_from":13,"seq":55834575065,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":36612,"kb_used_data":1400,"kb_used_omap":1,"kb_used_meta":35198,"kb_avail":94335228,"statfs":{"total":96636764160,"available":96599273472,"internally_reserved":0,"allocated":1433600,"data_stored":761810,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1781,"internal_metadata":36043019},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Aug 9 13:51:24 2023","interfaces":[{"interface":"back","average":{"1min":0.98499999999999999,"5min":1.1160000000000001,"15min":1.153},"min":{"1min":0.54000000000000004,"5min":0.54000000000000004,"15min":0.54000000000000004},"max":{"1min":1.484,"5min":1.633,"15min":1.9870000000000001},"last":1.3129999999999999},{"interface":"front","average":{"1min":1.0620000000000001,"5min":1.492,"15min":1.306},"min":{"1min":0.51800000000000002,"5min":0.51800000000000002,"15min":0.51800000000000002},"max":{"1min":1.6479999999999999,"5min":28.036000000000001,"15min":28.036000000000001},"last":0.745}]},{"osd":1,"last update":"Wed Aug 9 13:51:24 2023","interfaces":[{"interface":"back","average":{"1min":1.0940000000000001,"5min":1.238,"15min":1.232},"min":{"1min":0.48999999999999999,"5min":0.48999999999999999,"15min":0.48999999999999999},"max":{"1min":1.6639999999999999,"5min":7.742,"15min":7.742},"last":0.83399999999999996},{"interface":"front","average":{"1min":1.1950000000000001,"5min":1.276,"15min":1.28},"min":{"1min":0.64600000000000002,"5min":0.64600000000000002,"15min":0.64600000000000002},"max":{"1min":2.056,"5min":2.056,"15min":2.056},"last":1.1339999999999999}]},{"osd":2,"last update":"Wed Aug 9 13:51:24 2023","interfaces":[{"interface":"back","average":{"1min":1.206,"5min":1.323,"15min":1.3260000000000001},"min":{"1min":0.80600000000000005,"5min":0.80600000000000005,"15min":0.74099999999999999},"max":{"1min":1.8819999999999999,"5min":4.0640000000000001,"15min":4.0640000000000001},"last":1.3919999999999999},{"interface":"front","average":{"1min":1.161,"5min":1.399,"15min":1.347},"min":{"1min":0.80700000000000005,"5min":0.70999999999999996,"15min":0.70999999999999996},"max":{"1min":1.6679999999999999,"5min":13.154999999999999,"15min":13.154999999999999},"last":0.96699999999999997}]},{"osd":3,"last update":"Wed Aug 9 13:51:24 2023","interfaces":[{"interface":"back","average":{"1min":1.196,"5min":1.452,"15min":1.415},"min":{"1min":0.67200000000000004,"5min":0.67200000000000004,"15min":0.67200000000000004},"max":{"1min":1.522,"5min":13.349,"15min":13.349},"last":1.258},{"interface":"front","average":{"1min":1.254,"5min":1.353,"15min":1.3280000000000001},"min":{"1min":0.75700000000000001,"5min":0.75700000000000001,"15min":0.74299999999999999},"max":{"1min":1.732,"5min":7.9029999999999996,"15min":7.9029999999999996},"last":1.0740000000000001}]},{"osd":4,"last update":"Wed Aug 9 13:51:24 2023","interfaces":[{"interface":"back","average":{"1min":1.323,"5min":1.3540000000000001,"15min":1.284},"min":{"1min":0.86199999999999999,"5min":0.84499999999999997,"15min":0.64300000000000002},"max":{"1min":1.833,"5min":5.2290000000000001,"15min":5.2290000000000001},"last":1.4450000000000001},{"interface":"front","average":{"1min":1.367,"5min":1.506,"15min":1.3879999999999999},"min":{"1min":0.89900000000000002,"5min":0.78700000000000003,"15min":0.78700000000000003},"max":{"1min":1.714,"5min":12.1,"15min":12.1},"last":1.359}]},{"osd":5,"last update":"Wed Aug 9 13:51:24 2023","interfaces":[{"interface":"back","average":{"1min":1.423,"5min":1.446,"15min":1.3660000000000001},"min":{"1min":0.57699999999999996,"5min":0.57699999999999996,"15min":0.57699999999999996},"max":{"1min":1.843,"5min":8.1899999999999995,"15min":8.1899999999999995},"last":1.222},{"interface":"front","average":{"1min":1.55,"5min":1.6679999999999999,"15min":1.593},"min":{"1min":0.93000000000000005,"5min":0.93000000000000005,"15min":0.83799999999999997},"max":{"1min":2.1459999999999999,"5min":8.8889999999999993,"15min":8.8889999999999993},"last":1.496}]},{"osd":7,"last update":"Wed Aug 9 13:51:24 2023","interfaces":[{"interface":"back","average":{"1min":1.5269999999999999,"5min":1.6719999999999999,"15min":1.587},"min":{"1min":0.96099999999999997,"5min":0.96099999999999997,"15min":0.62},"max":{"1min":1.907,"5min":9.3320000000000007,"15min":9.3320000000000007},"last":1.415},{"interface":"front","average":{"1min":1.556,"5min":1.613,"15min":1.55},"min":{"1min":0.88200000000000001,"5min":0.88200000000000001,"15min":0.88200000000000001},"max":{"1min":1.97,"5min":8.0399999999999991,"15min":8.0399999999999991},"last":1.2929999999999999}]}]},{"osd":5,"up_from":13,"seq":55834575065,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":40712,"kb_used_data":1400,"kb_used_omap":1,"kb_used_meta":39294,"kb_avail":94331128,"statfs":{"total":96636764160,"available":96595075072,"internally_reserved":0,"allocated":1433600,"data_stored":762589,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1781,"internal_metadata":40237323},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Aug 9 13:51:21 2023","interfaces":[{"interface":"back","average":{"1min":1.542,"5min":1.25,"15min":1.206},"min":{"1min":0.59099999999999997,"5min":0.59099999999999997,"15min":0.59099999999999997},"max":{"1min":8.9299999999999997,"5min":8.9299999999999997,"15min":8.9299999999999997},"last":0.98299999999999998},{"interface":"front","average":{"1min":1.6080000000000001,"5min":1.264,"15min":1.2330000000000001},"min":{"1min":0.60499999999999998,"5min":0.60499999999999998,"15min":0.60499999999999998},"max":{"1min":8.7140000000000004,"5min":8.7140000000000004,"15min":8.7140000000000004},"last":0.79500000000000004}]},{"osd":1,"last update":"Wed Aug 9 13:51:21 2023","interfaces":[{"interface":"back","average":{"1min":1.476,"5min":1.2330000000000001,"15min":1.246},"min":{"1min":0.68799999999999994,"5min":0.68799999999999994,"15min":0.64100000000000001},"max":{"1min":8.2720000000000002,"5min":8.2720000000000002,"15min":8.2720000000000002},"last":0.86599999999999999},{"interface":"front","average":{"1min":1.6679999999999999,"5min":1.359,"15min":1.3100000000000001},"min":{"1min":0.80500000000000005,"5min":0.66900000000000004,"15min":0.66900000000000004},"max":{"1min":9.0020000000000007,"5min":9.0020000000000007,"15min":9.0020000000000007},"last":0.73399999999999999}]},{"osd":2,"last update":"Wed Aug 9 13:51:21 2023","interfaces":[{"interface":"back","average":{"1min":1.7190000000000001,"5min":1.335,"15min":1.319},"min":{"1min":0.81799999999999995,"5min":0.68999999999999995,"15min":0.61599999999999999},"max":{"1min":8.6050000000000004,"5min":8.6050000000000004,"15min":8.6050000000000004},"last":1.1379999999999999},{"interface":"front","average":{"1min":1.641,"5min":1.371,"15min":1.347},"min":{"1min":0.78300000000000003,"5min":0.78300000000000003,"15min":0.72099999999999997},"max":{"1min":8.4149999999999991,"5min":8.4149999999999991,"15min":8.4149999999999991},"last":1.048}]},{"osd":3,"last update":"Wed Aug 9 13:51:21 2023","interfaces":[{"interface":"back","average":{"1min":1.8160000000000001,"5min":1.466,"15min":1.391},"min":{"1min":0.72599999999999998,"5min":0.72599999999999998,"15min":0.59799999999999998},"max":{"1min":9.0470000000000006,"5min":9.0470000000000006,"15min":9.0470000000000006},"last":1.0960000000000001},{"interface":"front","average":{"1min":1.738,"5min":1.4970000000000001,"15min":1.4370000000000001},"min":{"1min":0.76300000000000001,"5min":0.76300000000000001,"15min":0.70999999999999996},"max":{"1min":8.7669999999999995,"5min":8.7669999999999995,"15min":8.7669999999999995},"last":1.256}]},{"osd":4,"last update":"Wed Aug 9 13:51:21 2023","interfaces":[{"interface":"back","average":{"1min":1.724,"5min":1.367,"15min":1.321},"min":{"1min":0.54300000000000004,"5min":0.54300000000000004,"15min":0.54300000000000004},"max":{"1min":8.4909999999999997,"5min":8.4909999999999997,"15min":8.4909999999999997},"last":1.3100000000000001},{"interface":"front","average":{"1min":1.9079999999999999,"5min":1.4570000000000001,"15min":1.4119999999999999},"min":{"1min":0.68899999999999995,"5min":0.68899999999999995,"15min":0.68899999999999995},"max":{"1min":9.4589999999999996,"5min":9.4589999999999996,"15min":9.4589999999999996},"last":1.2050000000000001}]},{"osd":6,"last update":"Wed Aug 9 13:51:21 2023","interfaces":[{"interface":"back","average":{"1min":2.5259999999999998,"5min":1.5980000000000001,"15min":1.4610000000000001},"min":{"1min":0.51000000000000001,"5min":0.51000000000000001,"15min":0.51000000000000001},"max":{"1min":17.407,"5min":17.407,"15min":17.407},"last":1.464},{"interface":"front","average":{"1min":2.351,"5min":1.6279999999999999,"15min":1.53},"min":{"1min":0.71199999999999997,"5min":0.71199999999999997,"15min":0.71199999999999997},"max":{"1min":16.356000000000002,"5min":16.356000000000002,"15min":16.356000000000002},"last":1.536}]},{"osd":7,"last update":"Wed Aug 9 13:51:21 2023","interfaces":[{"interface":"back","average":{"1min":2.117,"5min":1.665,"15min":1.591},"min":{"1min":0.74299999999999999,"5min":0.74299999999999999,"15min":0.74299999999999999},"max":{"1min":9.9239999999999995,"5min":9.9239999999999995,"15min":9.9239999999999995},"last":1.3879999999999999},{"interface":"front","average":{"1min":2.2400000000000002,"5min":1.716,"15min":1.645},"min":{"1min":0.65600000000000003,"5min":0.65600000000000003,"15min":0.65600000000000003},"max":{"1min":10.789,"5min":10.789,"15min":10.789},"last":1.6000000000000001}]}]},{"osd":4,"up_from":13,"seq":55834575065,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32532,"kb_used_data":1416,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":94339308,"statfs":{"total":96636764160,"available":96603451392,"internally_reserved":0,"allocated":1449984,"data_stored":764255,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1781,"internal_metadata":31848715},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Aug 9 13:51:23 2023","interfaces":[{"interface":"back","average":{"1min":1.131,"5min":1.2150000000000001,"15min":1.2070000000000001},"min":{"1min":0.66600000000000004,"5min":0.66600000000000004,"15min":0.66600000000000004},"max":{"1min":1.768,"5min":2.524,"15min":2.524},"last":1.0529999999999999},{"interface":"front","average":{"1min":1.0609999999999999,"5min":1.181,"15min":1.25},"min":{"1min":0.66800000000000004,"5min":0.62,"15min":0.62},"max":{"1min":1.667,"5min":1.7390000000000001,"15min":1.863},"last":1.238}]},{"osd":1,"last update":"Wed Aug 9 13:51:23 2023","interfaces":[{"interface":"back","average":{"1min":1.1839999999999999,"5min":1.264,"15min":1.2969999999999999},"min":{"1min":0.79700000000000004,"5min":0.67800000000000005,"15min":0.67800000000000005},"max":{"1min":1.609,"5min":1.8480000000000001,"15min":1.8480000000000001},"last":1.4059999999999999},{"interface":"front","average":{"1min":1.1319999999999999,"5min":1.232,"15min":1.2470000000000001},"min":{"1min":0.75,"5min":0.75,"15min":0.65400000000000003},"max":{"1min":1.669,"5min":1.9330000000000001,"15min":1.9330000000000001},"last":1.5049999999999999}]},{"osd":2,"last update":"Wed Aug 9 13:51:23 2023","interfaces":[{"interface":"back","average":{"1min":1.081,"5min":1.2509999999999999,"15min":1.272},"min":{"1min":0.69999999999999996,"5min":0.621,"15min":0.621},"max":{"1min":1.496,"5min":2.6429999999999998,"15min":2.6429999999999998},"last":0.81799999999999995},{"interface":"front","average":{"1min":1.25,"5min":1.3220000000000001,"15min":1.365},"min":{"1min":0.76000000000000001,"5min":0.76000000000000001,"15min":0.75900000000000001},"max":{"1min":2.0910000000000002,"5min":2.0910000000000002,"15min":2.0910000000000002},"last":1.6080000000000001}]},{"osd":3,"last update":"Wed Aug 9 13:51:23 2023","interfaces":[{"interface":"back","average":{"1min":1.1599999999999999,"5min":1.2889999999999999,"15min":1.298},"min":{"1min":0.754,"5min":0.754,"15min":0.754},"max":{"1min":1.921,"5min":2.6099999999999999,"15min":2.6099999999999999},"last":1.2809999999999999},{"interface":"front","average":{"1min":1.2509999999999999,"5min":1.3740000000000001,"15min":1.3819999999999999},"min":{"1min":0.89100000000000001,"5min":0.88800000000000001,"15min":0.63100000000000001},"max":{"1min":1.7010000000000001,"5min":2.7629999999999999,"15min":2.7629999999999999},"last":1.1639999999999999}]},{"osd":5,"last update":"Wed Aug 9 13:51:23 2023","interfaces":[{"interface":"back","average":{"1min":1.294,"5min":1.3140000000000001,"15min":1.327},"min":{"1min":0.67300000000000004,"5min":0.67300000000000004,"15min":0.63200000000000001},"max":{"1min":1.758,"5min":1.8,"15min":2.0099999999999998},"last":1.546},{"interface":"front","average":{"1min":1.361,"5min":1.337,"15min":1.3169999999999999},"min":{"1min":0.65600000000000003,"5min":0.65600000000000003,"15min":0.626},"max":{"1min":2.0369999999999999,"5min":2.7090000000000001,"15min":2.7090000000000001},"last":1.357}]},{"osd":6,"last update":"Wed Aug 9 13:51:23 2023","interfaces":[{"interface":"back","average":{"1min":1.409,"5min":1.341,"15min":1.3340000000000001},"min":{"1min":0.90600000000000003,"5min":0.74399999999999999,"15min":0.59099999999999997},"max":{"1min":1.96,"5min":2.8559999999999999,"15min":2.8559999999999999},"last":1.4530000000000001},{"interface":"front","average":{"1min":1.496,"5min":1.514,"15min":1.5029999999999999},"min":{"1min":0.878,"5min":0.878,"15min":0.76400000000000001},"max":{"1min":2.1890000000000001,"5min":2.2719999999999998,"15min":2.2719999999999998},"last":1.6559999999999999}]},{"osd":7,"last update":"Wed Aug 9 13:51:23 2023","interfaces":[{"interface":"back","average":{"1min":1.3859999999999999,"5min":1.468,"15min":1.4690000000000001},"min":{"1min":0.53100000000000003,"5min":0.53100000000000003,"15min":0.53100000000000003},"max":{"1min":1.9530000000000001,"5min":2.8050000000000002,"15min":2.8050000000000002},"last":1.5720000000000001},{"interface":"front","average":{"1min":1.5289999999999999,"5min":1.52,"15min":1.5129999999999999},"min":{"1min":0.78700000000000003,"5min":0.78700000000000003,"15min":0.72899999999999998},"max":{"1min":2.1419999999999999,"5min":3.415,"15min":3.415},"last":1.3109999999999999}]}]},{"osd":3,"up_from":13,"seq":55834575066,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":36628,"kb_used_data":1412,"kb_used_omap":1,"kb_used_meta":35198,"kb_avail":94335212,"statfs":{"total":96636764160,"available":96599257088,"internally_reserved":0,"allocated":1445888,"data_stored":764153,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1781,"internal_metadata":36043019},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Aug 9 13:51:26 2023","interfaces":[{"interface":"back","average":{"1min":0.76900000000000002,"5min":0.85299999999999998,"15min":0.90500000000000003},"min":{"1min":0.496,"5min":0.27200000000000002,"15min":0.27200000000000002},"max":{"1min":1.5580000000000001,"5min":1.8500000000000001,"15min":1.968},"last":0.91100000000000003},{"interface":"front","average":{"1min":0.83499999999999996,"5min":1.0349999999999999,"15min":1.097},"min":{"1min":0.36299999999999999,"5min":0.36299999999999999,"15min":0.36299999999999999},"max":{"1min":1.4059999999999999,"5min":1.6910000000000001,"15min":2.0590000000000002},"last":1.228}]},{"osd":1,"last update":"Wed Aug 9 13:51:26 2023","interfaces":[{"interface":"back","average":{"1min":0.82799999999999996,"5min":0.95399999999999996,"15min":1.0089999999999999},"min":{"1min":0.51500000000000001,"5min":0.496,"15min":0.438},"max":{"1min":1.27,"5min":1.6519999999999999,"15min":2.5649999999999999},"last":0.873},{"interface":"front","average":{"1min":0.97899999999999998,"5min":1.081,"15min":1.161},"min":{"1min":0.60499999999999998,"5min":0.51800000000000002,"15min":0.498},"max":{"1min":1.6779999999999999,"5min":1.6839999999999999,"15min":2.6760000000000002},"last":1.0289999999999999}]},{"osd":2,"last update":"Wed Aug 9 13:51:26 2023","interfaces":[{"interface":"back","average":{"1min":0.995,"5min":1.115,"15min":1.194},"min":{"1min":0.42999999999999999,"5min":0.42999999999999999,"15min":0.42999999999999999},"max":{"1min":1.6879999999999999,"5min":1.913,"15min":1.923},"last":1.585},{"interface":"front","average":{"1min":1.103,"5min":1.2290000000000001,"15min":1.3069999999999999},"min":{"1min":0.61299999999999999,"5min":0.52300000000000002,"15min":0.52300000000000002},"max":{"1min":1.7290000000000001,"5min":1.9399999999999999,"15min":2.0259999999999998},"last":1.177}]},{"osd":4,"last update":"Wed Aug 9 13:51:26 2023","interfaces":[{"interface":"back","average":{"1min":1.1000000000000001,"5min":1.214,"15min":1.274},"min":{"1min":0.54900000000000004,"5min":0.54900000000000004,"15min":0.54900000000000004},"max":{"1min":1.732,"5min":1.8160000000000001,"15min":1.8160000000000001},"last":1.456},{"interface":"front","average":{"1min":1.2769999999999999,"5min":1.3440000000000001,"15min":1.3959999999999999},"min":{"1min":0.76100000000000001,"5min":0.628,"15min":0.628},"max":{"1min":2.782,"5min":2.782,"15min":2.782},"last":1.2809999999999999}]},{"osd":5,"last update":"Wed Aug 9 13:51:26 2023","interfaces":[{"interface":"back","average":{"1min":1.266,"5min":1.4179999999999999,"15min":1.47},"min":{"1min":0.69899999999999995,"5min":0.67600000000000005,"15min":0.67200000000000004},"max":{"1min":1.833,"5min":2.254,"15min":2.254},"last":1.397},{"interface":"front","average":{"1min":1.258,"5min":1.431,"15min":1.484},"min":{"1min":0.66000000000000003,"5min":0.65900000000000003,"15min":0.65900000000000003},"max":{"1min":1.919,"5min":2.3069999999999999,"15min":2.3069999999999999},"last":1.508}]},{"osd":6,"last update":"Wed Aug 9 13:51:26 2023","interfaces":[{"interface":"back","average":{"1min":1.381,"5min":1.419,"15min":1.4630000000000001},"min":{"1min":0.78200000000000003,"5min":0.68200000000000005,"15min":0.68200000000000005},"max":{"1min":2.8340000000000001,"5min":2.8340000000000001,"15min":2.8340000000000001},"last":1.5489999999999999},{"interface":"front","average":{"1min":1.389,"5min":1.5109999999999999,"15min":1.5580000000000001},"min":{"1min":0.64100000000000001,"5min":0.64100000000000001,"15min":0.64100000000000001},"max":{"1min":2.9060000000000001,"5min":2.9060000000000001,"15min":2.9060000000000001},"last":1.7050000000000001}]},{"osd":7,"last update":"Wed Aug 9 13:51:26 2023","interfaces":[{"interface":"back","average":{"1min":1.3580000000000001,"5min":1.5549999999999999,"15min":1.6100000000000001},"min":{"1min":0.77800000000000002,"5min":0.69399999999999995,"15min":0.69399999999999995},"max":{"1min":2.0419999999999998,"5min":2.46,"15min":2.46},"last":1.339},{"interface":"front","average":{"1min":1.403,"5min":1.5049999999999999,"15min":1.5620000000000001},"min":{"1min":0.93799999999999994,"5min":0.64200000000000002,"15min":0.64200000000000002},"max":{"1min":2.7200000000000002,"5min":2.7200000000000002,"15min":2.7200000000000002},"last":1.643}]}]},{"osd":2,"up_from":13,"seq":55834575065,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":36612,"kb_used_data":1400,"kb_used_omap":1,"kb_used_meta":35198,"kb_avail":94335228,"statfs":{"total":96636764160,"available":96599273472,"internally_reserved":0,"allocated":1433600,"data_stored":762553,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1781,"internal_metadata":36043019},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Aug 9 13:51:19 2023","interfaces":[{"interface":"back","average":{"1min":0.82699999999999996,"5min":0.91400000000000003,"15min":0.95599999999999996},"min":{"1min":0.47199999999999998,"5min":0.36499999999999999,"15min":0.36499999999999999},"max":{"1min":1.484,"5min":1.5940000000000001,"15min":1.8380000000000001},"last":0.76600000000000001},{"interface":"front","average":{"1min":0.90200000000000002,"5min":0.96699999999999997,"15min":1.0009999999999999},"min":{"1min":0.38200000000000001,"5min":0.374,"15min":0.374},"max":{"1min":1.7,"5min":2.3929999999999998,"15min":2.3929999999999998},"last":1.2}]},{"osd":1,"last update":"Wed Aug 9 13:51:19 2023","interfaces":[{"interface":"back","average":{"1min":1.038,"5min":1.143,"15min":1.1839999999999999},"min":{"1min":0.39400000000000002,"5min":0.39400000000000002,"15min":0.39400000000000002},"max":{"1min":1.615,"5min":1.77,"15min":1.8420000000000001},"last":0.88300000000000001},{"interface":"front","average":{"1min":0.98399999999999999,"5min":1.171,"15min":1.3220000000000001},"min":{"1min":0.435,"5min":0.435,"15min":0.435},"max":{"1min":1.659,"5min":2.4870000000000001,"15min":2.4870000000000001},"last":1.298}]},{"osd":3,"last update":"Wed Aug 9 13:51:19 2023","interfaces":[{"interface":"back","average":{"1min":0.93100000000000005,"5min":0.97999999999999998,"15min":1.0509999999999999},"min":{"1min":0.55300000000000005,"5min":0.38900000000000001,"15min":0.38900000000000001},"max":{"1min":1.417,"5min":1.708,"15min":2.0409999999999999},"last":0.73399999999999999},{"interface":"front","average":{"1min":1.0820000000000001,"5min":1.2549999999999999,"15min":1.3080000000000001},"min":{"1min":0.52900000000000003,"5min":0.46200000000000002,"15min":0.46200000000000002},"max":{"1min":1.6279999999999999,"5min":8.8130000000000006,"15min":8.8130000000000006},"last":1.1639999999999999}]},{"osd":4,"last update":"Wed Aug 9 13:51:19 2023","interfaces":[{"interface":"back","average":{"1min":1.335,"5min":1.3740000000000001,"15min":1.452},"min":{"1min":0.81000000000000005,"5min":0.628,"15min":0.628},"max":{"1min":1.8340000000000001,"5min":2.089,"15min":2.089},"last":1.0740000000000001},{"interface":"front","average":{"1min":1.2969999999999999,"5min":1.3360000000000001,"15min":1.3939999999999999},"min":{"1min":0.75900000000000001,"5min":0.68600000000000005,"15min":0.68600000000000005},"max":{"1min":1.7509999999999999,"5min":2.472,"15min":2.472},"last":1.419}]},{"osd":5,"last update":"Wed Aug 9 13:51:19 2023","interfaces":[{"interface":"back","average":{"1min":1.361,"5min":1.4059999999999999,"15min":1.4590000000000001},"min":{"1min":0.86399999999999999,"5min":0.59999999999999998,"15min":0.59999999999999998},"max":{"1min":2.0640000000000001,"5min":2.5840000000000001,"15min":2.5840000000000001},"last":1.3620000000000001},{"interface":"front","average":{"1min":1.2829999999999999,"5min":1.3260000000000001,"15min":1.351},"min":{"1min":0.89000000000000001,"5min":0.56799999999999995,"15min":0.56799999999999995},"max":{"1min":1.78,"5min":3.5659999999999998,"15min":3.5659999999999998},"last":1.032}]},{"osd":6,"last update":"Wed Aug 9 13:51:19 2023","interfaces":[{"interface":"back","average":{"1min":1.3009999999999999,"5min":1.3879999999999999,"15min":1.429},"min":{"1min":0.82199999999999995,"5min":0.73799999999999999,"15min":0.73799999999999999},"max":{"1min":2.1200000000000001,"5min":3.5099999999999998,"15min":3.5099999999999998},"last":1.2749999999999999},{"interface":"front","average":{"1min":1.4099999999999999,"5min":1.458,"15min":1.54},"min":{"1min":0.84899999999999998,"5min":0.66300000000000003,"15min":0.66300000000000003},"max":{"1min":2.04,"5min":2.3769999999999998,"15min":2.3769999999999998},"last":1.248}]},{"osd":7,"last update":"Wed Aug 9 13:51:19 2023","interfaces":[{"interface":"back","average":{"1min":1.3620000000000001,"5min":1.452,"15min":1.54},"min":{"1min":0.85099999999999998,"5min":0.64600000000000002,"15min":0.64600000000000002},"max":{"1min":1.865,"5min":2.5249999999999999,"15min":2.5249999999999999},"last":1.476},{"interface":"front","average":{"1min":1.3939999999999999,"5min":1.476,"15min":1.514},"min":{"1min":0.94699999999999995,"5min":0.70499999999999996,"15min":0.70499999999999996},"max":{"1min":1.891,"5min":3.609,"15min":3.609},"last":1.339}]}]},{"osd":1,"up_from":13,"seq":55834575065,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":47044,"kb_used_data":1976,"kb_used_omap":1,"kb_used_meta":45054,"kb_avail":94324796,"statfs":{"total":96636764160,"available":96588591104,"internally_reserved":0,"allocated":2023424,"data_stored":1352887,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1781,"internal_metadata":46135563},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Aug 9 13:51:34 2023","interfaces":[{"interface":"back","average":{"1min":0.77600000000000002,"5min":0.90000000000000002,"15min":0.93799999999999994},"min":{"1min":0.45800000000000002,"5min":0.45800000000000002,"15min":0.45800000000000002},"max":{"1min":1.1910000000000001,"5min":1.7110000000000001,"15min":2.1080000000000001},"last":0.58799999999999997},{"interface":"front","average":{"1min":0.83299999999999996,"5min":1.0129999999999999,"15min":1.095},"min":{"1min":0.60599999999999998,"5min":0.432,"15min":0.432},"max":{"1min":1.6519999999999999,"5min":2.468,"15min":2.468},"last":1.214}]},{"osd":2,"last update":"Wed Aug 9 13:51:34 2023","interfaces":[{"interface":"back","average":{"1min":0.83399999999999996,"5min":0.93799999999999994,"15min":0.99299999999999999},"min":{"1min":0.57499999999999996,"5min":0.41099999999999998,"15min":0.41099999999999998},"max":{"1min":1.2130000000000001,"5min":1.907,"15min":1.907},"last":0.80900000000000005},{"interface":"front","average":{"1min":0.91100000000000003,"5min":1.0920000000000001,"15min":1.1859999999999999},"min":{"1min":0.59699999999999998,"5min":0.54400000000000004,"15min":0.54400000000000004},"max":{"1min":1.45,"5min":2.1280000000000001,"15min":2.145},"last":1.3700000000000001}]},{"osd":3,"last update":"Wed Aug 9 13:51:34 2023","interfaces":[{"interface":"back","average":{"1min":0.97199999999999998,"5min":1.1890000000000001,"15min":1.3220000000000001},"min":{"1min":0.60799999999999998,"5min":0.54400000000000004,"15min":0.54400000000000004},"max":{"1min":1.466,"5min":2.0270000000000001,"15min":2.3210000000000002},"last":1.131},{"interface":"front","average":{"1min":1.042,"5min":1.163,"15min":1.25},"min":{"1min":0.625,"5min":0.49299999999999999,"15min":0.45300000000000001},"max":{"1min":1.6539999999999999,"5min":2.3540000000000001,"15min":2.7669999999999999},"last":0.66700000000000004}]},{"osd":4,"last update":"Wed Aug 9 13:51:34 2023","interfaces":[{"interface":"back","average":{"1min":1.1819999999999999,"5min":1.337,"15min":1.4059999999999999},"min":{"1min":0.79800000000000004,"5min":0.79800000000000004,"15min":0.58099999999999996},"max":{"1min":1.9259999999999999,"5min":1.97,"15min":2.5},"last":1.0069999999999999},{"interface":"front","average":{"1min":1.157,"5min":1.27,"15min":1.329},"min":{"1min":0.76600000000000001,"5min":0.70099999999999996,"15min":0.70099999999999996},"max":{"1min":1.895,"5min":1.895,"15min":1.901},"last":1.048}]},{"osd":5,"last update":"Wed Aug 9 13:51:34 2023","interfaces":[{"interface":"back","average":{"1min":1.2150000000000001,"5min":1.29,"15min":1.3680000000000001},"min":{"1min":0.76800000000000002,"5min":0.76800000000000002,"15min":0.76800000000000002},"max":{"1min":1.73,"5min":2.294,"15min":2.294},"last":1.278},{"interface":"front","average":{"1min":1.2949999999999999,"5min":1.3999999999999999,"15min":1.512},"min":{"1min":0.89200000000000002,"5min":0.89200000000000002,"15min":0.89200000000000002},"max":{"1min":1.6859999999999999,"5min":2.1389999999999998,"15min":2.4649999999999999},"last":1.591}]},{"osd":6,"last update":"Wed Aug 9 13:51:34 2023","interfaces":[{"interface":"back","average":{"1min":1.325,"5min":1.4339999999999999,"15min":1.5149999999999999},"min":{"1min":0.96599999999999997,"5min":0.77500000000000002,"15min":0.67400000000000004},"max":{"1min":1.5780000000000001,"5min":2.2080000000000002,"15min":2.5270000000000001},"last":1.4390000000000001},{"interface":"front","average":{"1min":1.28,"5min":1.3979999999999999,"15min":1.4990000000000001},"min":{"1min":0.73599999999999999,"5min":0.73599999999999999,"15min":0.73599999999999999},"max":{"1min":1.591,"5min":2.02,"15min":2.1040000000000001},"last":1.4630000000000001}]},{"osd":7,"last update":"Wed Aug 9 13:51:34 2023","interfaces":[{"interface":"back","average":{"1min":1.3240000000000001,"5min":1.4930000000000001,"15min":1.615},"min":{"1min":0.93700000000000006,"5min":0.86899999999999999,"15min":0.86899999999999999},"max":{"1min":1.776,"5min":2.4169999999999998,"15min":2.5790000000000002},"last":1.4870000000000001},{"interface":"front","average":{"1min":1.3420000000000001,"5min":1.488,"15min":1.5980000000000001},"min":{"1min":0.96899999999999997,"5min":0.91300000000000003,"15min":0.89500000000000002},"max":{"1min":1.8320000000000001,"5min":2.1040000000000001,"15min":2.6509999999999998},"last":1.3320000000000001}]}]},{"osd":0,"up_from":13,"seq":55834575065,"num_pgs":32,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":40704,"kb_used_data":1396,"kb_used_omap":1,"kb_used_meta":39294,"kb_avail":94331136,"statfs":{"total":96636764160,"available":96595083264,"internally_reserved":0,"allocated":1429504,"data_stored":761740,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1781,"internal_metadata":40237323},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Wed Aug 9 13:51:27 2023","interfaces":[{"interface":"back","average":{"1min":0.95899999999999996,"5min":0.91300000000000003,"15min":0.92100000000000004},"min":{"1min":0.41199999999999998,"5min":0.32500000000000001,"15min":0.32500000000000001},"max":{"1min":1.4990000000000001,"5min":1.756,"15min":1.756},"last":0.97099999999999997},{"interface":"front","average":{"1min":1.002,"5min":1.077,"15min":1.111},"min":{"1min":0.39200000000000002,"5min":0.39200000000000002,"15min":0.39200000000000002},"max":{"1min":1.546,"5min":1.9019999999999999,"15min":1.9019999999999999},"last":1.0960000000000001}]},{"osd":2,"last update":"Wed Aug 9 13:51:27 2023","interfaces":[{"interface":"back","average":{"1min":1,"5min":1.0680000000000001,"15min":1.1339999999999999},"min":{"1min":0.58399999999999996,"5min":0.51400000000000001,"15min":0.503},"max":{"1min":1.5089999999999999,"5min":1.776,"15min":1.889},"last":0.79900000000000004},{"interface":"front","average":{"1min":0.98799999999999999,"5min":1.1020000000000001,"15min":1.1739999999999999},"min":{"1min":0.439,"5min":0.439,"15min":0.439},"max":{"1min":1.6259999999999999,"5min":2.0129999999999999,"15min":2.0129999999999999},"last":1.286}]},{"osd":3,"last update":"Wed Aug 9 13:51:27 2023","interfaces":[{"interface":"back","average":{"1min":1,"5min":1.0349999999999999,"15min":1.101},"min":{"1min":0.50900000000000001,"5min":0.46300000000000002,"15min":0.46300000000000002},"max":{"1min":1.5189999999999999,"5min":1.823,"15min":1.8919999999999999},"last":1.1559999999999999},{"interface":"front","average":{"1min":1.2390000000000001,"5min":1.252,"15min":1.3140000000000001},"min":{"1min":0.63400000000000001,"5min":0.54700000000000004,"15min":0.54700000000000004},"max":{"1min":2.2050000000000001,"5min":2.347,"15min":2.4649999999999999},"last":1.3640000000000001}]},{"osd":4,"last update":"Wed Aug 9 13:51:27 2023","interfaces":[{"interface":"back","average":{"1min":1.2509999999999999,"5min":1.2949999999999999,"15min":1.343},"min":{"1min":0.73899999999999999,"5min":0.73899999999999999,"15min":0.73899999999999999},"max":{"1min":1.581,"5min":2.2309999999999999,"15min":2.2309999999999999},"last":1.5369999999999999},{"interface":"front","average":{"1min":1.409,"5min":1.417,"15min":1.456},"min":{"1min":0.86399999999999999,"5min":0.77600000000000002,"15min":0.77600000000000002},"max":{"1min":2.3540000000000001,"5min":2.516,"15min":2.5390000000000001},"last":1.2490000000000001}]},{"osd":5,"last update":"Wed Aug 9 13:51:27 2023","interfaces":[{"interface":"back","average":{"1min":1.3859999999999999,"5min":1.4019999999999999,"15min":1.462},"min":{"1min":0.875,"5min":0.66300000000000003,"15min":0.66300000000000003},"max":{"1min":1.764,"5min":2.6669999999999998,"15min":2.6669999999999998},"last":1.407},{"interface":"front","average":{"1min":1.333,"5min":1.3859999999999999,"15min":1.4470000000000001},"min":{"1min":0.88400000000000001,"5min":0.629,"15min":0.629},"max":{"1min":1.645,"5min":2.1499999999999999,"15min":2.1499999999999999},"last":1.3149999999999999}]},{"osd":6,"last update":"Wed Aug 9 13:51:27 2023","interfaces":[{"interface":"back","average":{"1min":1.4870000000000001,"5min":1.5089999999999999,"15min":1.548},"min":{"1min":1.014,"5min":0.73599999999999999,"15min":0.73599999999999999},"max":{"1min":2.4670000000000001,"5min":2.4670000000000001,"15min":2.7250000000000001},"last":1.486},{"interface":"front","average":{"1min":1.486,"5min":1.526,"15min":1.5569999999999999},"min":{"1min":0.95999999999999996,"5min":0.86699999999999999,"15min":0.86699999999999999},"max":{"1min":1.988,"5min":2.399,"15min":2.399},"last":1.345}]},{"osd":7,"last update":"Wed Aug 9 13:51:27 2023","interfaces":[{"interface":"back","average":{"1min":1.552,"5min":1.5720000000000001,"15min":1.621},"min":{"1min":0.89200000000000002,"5min":0.72299999999999998,"15min":0.72299999999999998},"max":{"1min":2.0880000000000001,"5min":2.3900000000000001,"15min":2.3900000000000001},"last":1.458},{"interface":"front","average":{"1min":1.4179999999999999,"5min":1.484,"15min":1.54},"min":{"1min":0.84099999999999997,"5min":0.76700000000000002,"15min":0.76700000000000002},"max":{"1min":1.7969999999999999,"5min":2.274,"15min":2.274},"last":1.425}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":34,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1634,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":1736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":70,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":652,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":112,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-08-09T13:52:09.885 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-08-09T13:52:10.124 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:52:10.122+0000 7fb50fb76640 1 Processor -- start 2023-08-09T13:52:10.125 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:52:10.123+0000 7fb50fb76640 1 -- start start 2023-08-09T13:52:10.126 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:52:10.125+0000 7fb50fb76640 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7fb50806cfe0 0x7fb5080c8b50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:52:10.127 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:52:10.125+0000 7fb50fb76640 1 -- --> v1:172.21.15.43:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb5080c9090 con 0x7fb508143c20 2023-08-09T13:52:10.127 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:52:10.125+0000 7fb50fb76640 1 -- --> v1:172.21.15.203:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb50806c440 con 0x7fb5080c86f0 2023-08-09T13:52:10.127 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:52:10.125+0000 7fb50fb76640 1 -- --> v2:172.21.15.203:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb508144000 con 0x7fb50806cfe0 2023-08-09T13:52:10.128 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:52:10.125+0000 7fb50d912640 1 --1- >> v1:172.21.15.43:6789/0 conn(0x7fb508143c20 0x7fb50806f540 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.43:6789/0 says I am v1:172.21.15.43:38104/0 (socket says 172.21.15.43:38104) 2023-08-09T13:52:10.128 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:52:10.125+0000 7fb50d912640 1 -- 172.21.15.43:0/2291721388 learned_addr learned my addr 172.21.15.43:0/2291721388 (peer_addr_for_me v1:172.21.15.43:0/0) 2023-08-09T13:52:10.132 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:52:10.125+0000 7fb50c910640 1 -- 172.21.15.43:0/2291721388 <== mon.0 v1:172.21.15.43:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1720763243 0 0) 0x7fb5080c9090 con 0x7fb508143c20 2023-08-09T13:52:10.133 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:52:10.125+0000 7fb50e113640 1 --2- 172.21.15.43:0/2291721388 >> v2:172.21.15.203:3300/0 conn(0x7fb50806cfe0 0x7fb5080c8b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:52:10.133 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:52:10.125+0000 7fb50c910640 1 -- 172.21.15.43:0/2291721388 --> v1:172.21.15.43:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fb4f4003520 con 0x7fb508143c20 2023-08-09T13:52:10.133 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:52:10.126+0000 7fb50c910640 1 -- 172.21.15.43:0/2291721388 <== mon.0 v1:172.21.15.43:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 172086365 0 0) 0x7fb4f4003520 con 0x7fb508143c20 2023-08-09T13:52:10.134 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:52:10.126+0000 7fb50c910640 1 -- 172.21.15.43:0/2291721388 >> v2:172.21.15.203:3300/0 conn(0x7fb50806cfe0 msgr2=0x7fb5080c8b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-09T13:52:10.134 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:52:10.126+0000 7fb50c910640 1 --2- 172.21.15.43:0/2291721388 >> v2:172.21.15.203:3300/0 conn(0x7fb50806cfe0 0x7fb5080c8b50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:52:10.134 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:52:10.126+0000 7fb50c910640 1 -- 172.21.15.43:0/2291721388 >> v1:172.21.15.203:6789/0 conn(0x7fb5080c86f0 legacy=0x7fb50806c8d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:52:10.134 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:52:10.126+0000 7fb50c910640 1 -- 172.21.15.43:0/2291721388 --> v1:172.21.15.43:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb50806fc50 con 0x7fb508143c20 2023-08-09T13:52:10.134 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:52:10.126+0000 7fb50c910640 1 -- 172.21.15.43:0/2291721388 <== mon.0 v1:172.21.15.43:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1285150740 0 0) 0x7fb504001670 con 0x7fb508143c20 2023-08-09T13:52:10.135 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:52:10.126+0000 7fb50c910640 1 -- 172.21.15.43:0/2291721388 >> v1:172.21.15.43:6789/0 conn(0x7fb508143c20 legacy=0x7fb50806f540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:52:10.135 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:52:10.126+0000 7fb50c910640 1 --2- 172.21.15.43:0/2291721388 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb4f40039e0 0x7fb4f4003e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:52:10.135 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:52:10.126+0000 7fb50c910640 1 --2- 172.21.15.43:0/2291721388 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fb4f40047c0 0x7fb4f4004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:52:10.135 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:52:10.126+0000 7fb50c910640 1 --2- 172.21.15.43:0/2291721388 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fb4f4005330 0x7fb4f400b9a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:52:10.135 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:52:10.126+0000 7fb50c910640 1 -- 172.21.15.43:0/2291721388 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb508144000 con 0x7fb4f40039e0 2023-08-09T13:52:10.135 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:52:10.126+0000 7fb50c910640 1 -- 172.21.15.43:0/2291721388 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb4f400c280 con 0x7fb4f4005330 2023-08-09T13:52:10.135 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:52:10.126+0000 7fb50c910640 1 -- 172.21.15.43:0/2291721388 --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb4f400c440 con 0x7fb4f40047c0 2023-08-09T13:52:10.136 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:52:10.126+0000 7fb50d912640 1 --2- 172.21.15.43:0/2291721388 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb4f40039e0 0x7fb4f4003e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:52:10.136 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:52:10.127+0000 7fb50e113640 1 --2- 172.21.15.43:0/2291721388 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fb4f4005330 0x7fb4f400b9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:52:10.136 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:52:10.127+0000 7fb50d111640 1 --2- 172.21.15.43:0/2291721388 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fb4f40047c0 0x7fb4f4004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:52:10.136 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:52:10.127+0000 7fb50d912640 1 -- 172.21.15.43:0/2291721388 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fb4f40047c0 msgr2=0x7fb4f4004c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-09T13:52:10.136 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:52:10.127+0000 7fb50d912640 1 --2- 172.21.15.43:0/2291721388 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fb4f40047c0 0x7fb4f4004c40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:52:10.136 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:52:10.127+0000 7fb50d912640 1 -- 172.21.15.43:0/2291721388 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fb4f4005330 msgr2=0x7fb4f400b9a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-09T13:52:10.136 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:52:10.127+0000 7fb50d912640 1 --2- 172.21.15.43:0/2291721388 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fb4f4005330 0x7fb4f400b9a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:52:10.136 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:52:10.127+0000 7fb50d912640 1 -- 172.21.15.43:0/2291721388 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb4f4003520 con 0x7fb4f40039e0 2023-08-09T13:52:10.137 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:52:10.127+0000 7fb50e113640 1 --2- 172.21.15.43:0/2291721388 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fb4f4005330 0x7fb4f400b9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-08-09T13:52:10.137 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:52:10.127+0000 7fb50e113640 1 -- 172.21.15.43:0/2291721388 reap_dead start 2023-08-09T13:52:10.137 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:52:10.128+0000 7fb50d912640 1 --2- 172.21.15.43:0/2291721388 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb4f40039e0 0x7fb4f4003e00 secure :-1 s=READY pgs=1359 cs=0 l=1 rev1=1 crypto rx=0x7fb50400c240 tx=0x7fb4f40057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8a4cd3dacaa06610 server_cookie=0 in_seq=0 out_seq=0 2023-08-09T13:52:10.137 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:52:10.128+0000 7fb50c910640 1 -- 172.21.15.43:0/2291721388 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb504005ad0 con 0x7fb4f40039e0 2023-08-09T13:52:10.137 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:52:10.128+0000 7fb50c910640 1 -- 172.21.15.43:0/2291721388 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb504005c30 con 0x7fb4f40039e0 2023-08-09T13:52:10.137 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:52:10.128+0000 7fb50c910640 1 -- 172.21.15.43:0/2291721388 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb50402caf0 con 0x7fb4f40039e0 2023-08-09T13:52:10.137 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:52:10.129+0000 7fb50fb76640 1 -- 172.21.15.43:0/2291721388 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb4f40039e0 msgr2=0x7fb4f4003e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:52:10.137 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:52:10.129+0000 7fb50fb76640 1 --2- 172.21.15.43:0/2291721388 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb4f40039e0 0x7fb4f4003e00 secure :-1 s=READY pgs=1359 cs=0 l=1 rev1=1 crypto rx=0x7fb50400c240 tx=0x7fb4f40057f0 comp rx=0 tx=0).stop 2023-08-09T13:52:10.138 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:52:10.129+0000 7fb50fb76640 1 -- 172.21.15.43:0/2291721388 shutdown_connections 2023-08-09T13:52:10.138 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:52:10.129+0000 7fb50fb76640 1 --2- 172.21.15.43:0/2291721388 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb4f40039e0 0x7fb4f4003e00 unknown :-1 s=CLOSED pgs=1359 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:52:10.138 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:52:10.129+0000 7fb50fb76640 1 -- 172.21.15.43:0/2291721388 >> 172.21.15.43:0/2291721388 conn(0x7fb5080ce0a0 msgr2=0x7fb5080ce480 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-09T13:52:10.138 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:52:10.129+0000 7fb50fb76640 1 -- 172.21.15.43:0/2291721388 shutdown_connections 2023-08-09T13:52:10.138 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:52:10.129+0000 7fb50fb76640 1 -- 172.21.15.43:0/2291721388 wait complete. 2023-08-09T13:52:10.138 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:52:10.131+0000 7fb50fb76640 1 Processor -- start 2023-08-09T13:52:10.138 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:52:10.131+0000 7fb50fb76640 1 -- start start 2023-08-09T13:52:10.140 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:52:10.132+0000 7fb50fb76640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fb4f40039e0 0x7fb5080c5170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:52:10.141 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:52:10.132+0000 7fb50fb76640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb508159f10 0x7fb5080c3930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:52:10.141 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:52:10.132+0000 7fb50fb76640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fb5080c3e70 0x7fb5080b3130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:52:10.141 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:52:10.132+0000 7fb50fb76640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb5080ac190 con 0x7fb508159f10 2023-08-09T13:52:10.141 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:52:10.132+0000 7fb50fb76640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb5080ab8c0 con 0x7fb4f40039e0 2023-08-09T13:52:10.141 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:52:10.132+0000 7fb50fb76640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb5080c42f0 con 0x7fb5080c3e70 2023-08-09T13:52:10.141 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:52:10.132+0000 7fb50d111640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb508159f10 0x7fb5080c3930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:52:10.141 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:52:10.132+0000 7fb50d111640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb508159f10 0x7fb5080c3930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:54350/0 (socket says 172.21.15.43:54350) 2023-08-09T13:52:10.142 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:52:10.132+0000 7fb50d111640 1 -- 172.21.15.43:0/598860555 learned_addr learned my addr 172.21.15.43:0/598860555 (peer_addr_for_me v2:172.21.15.43:0/0) 2023-08-09T13:52:10.142 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:52:10.133+0000 7fb50e113640 1 --2- 172.21.15.43:0/598860555 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fb5080c3e70 0x7fb5080b3130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:52:10.142 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:52:10.133+0000 7fb50d912640 1 --2- 172.21.15.43:0/598860555 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fb4f40039e0 0x7fb5080c5170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:52:10.142 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:52:10.133+0000 7fb50d111640 1 -- 172.21.15.43:0/598860555 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fb5080c3e70 msgr2=0x7fb5080b3130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:52:10.142 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:52:10.133+0000 7fb50d111640 1 --2- 172.21.15.43:0/598860555 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fb5080c3e70 0x7fb5080b3130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:52:10.142 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:52:10.133+0000 7fb50d111640 1 -- 172.21.15.43:0/598860555 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fb4f40039e0 msgr2=0x7fb5080c5170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:52:10.142 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:52:10.133+0000 7fb50d111640 1 --2- 172.21.15.43:0/598860555 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fb4f40039e0 0x7fb5080c5170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:52:10.143 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:52:10.133+0000 7fb50d111640 1 -- 172.21.15.43:0/598860555 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb508140290 con 0x7fb508159f10 2023-08-09T13:52:10.143 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:52:10.133+0000 7fb50d111640 1 --2- 172.21.15.43:0/598860555 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb508159f10 0x7fb5080c3930 secure :-1 s=READY pgs=1360 cs=0 l=1 rev1=1 crypto rx=0x7fb4f8001eb0 tx=0x7fb4f800d9d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-09T13:52:10.143 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:52:10.133+0000 7fb4fe7fc640 1 -- 172.21.15.43:0/598860555 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb4f8005480 con 0x7fb508159f10 2023-08-09T13:52:10.143 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:52:10.134+0000 7fb4fe7fc640 1 -- 172.21.15.43:0/598860555 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb4f8003610 con 0x7fb508159f10 2023-08-09T13:52:10.143 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:52:10.134+0000 7fb50fb76640 1 -- 172.21.15.43:0/598860555 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb5080c4510 con 0x7fb508159f10 2023-08-09T13:52:10.143 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:52:10.134+0000 7fb4fe7fc640 1 -- 172.21.15.43:0/598860555 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb4f8003980 con 0x7fb508159f10 2023-08-09T13:52:10.143 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:52:10.134+0000 7fb50fb76640 1 -- 172.21.15.43:0/598860555 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb5080abdb0 con 0x7fb508159f10 2023-08-09T13:52:10.143 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:52:10.135+0000 7fb50fb76640 1 -- 172.21.15.43:0/598860555 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb5080ac400 con 0x7fb508159f10 2023-08-09T13:52:10.144 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:52:10.136+0000 7fb4fe7fc640 1 -- 172.21.15.43:0/598860555 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 45) v1 ==== 88443+0+0 (secure 0 0 0) 0x7fb4f8003ae0 con 0x7fb508159f10 2023-08-09T13:52:10.144 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:52:10.137+0000 7fb4fe7fc640 1 --2- 172.21.15.43:0/598860555 >> [v2:172.21.15.203:6832/1317108840,v1:172.21.15.203:6833/1317108840] conn(0x7fb4ec06a170 0x7fb4ec06c630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:52:10.144 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:52:10.137+0000 7fb4fe7fc640 1 -- 172.21.15.43:0/598860555 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(116..116 src has 1..116) v4 ==== 6426+0+0 (secure 0 0 0) 0x7fb4f8088890 con 0x7fb508159f10 2023-08-09T13:52:10.144 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:52:10.138+0000 7fb50d912640 1 --2- 172.21.15.43:0/598860555 >> [v2:172.21.15.203:6832/1317108840,v1:172.21.15.203:6833/1317108840] conn(0x7fb4ec06a170 0x7fb4ec06c630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:52:10.144 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:52:10.139+0000 7fb50d912640 1 --2- 172.21.15.43:0/598860555 >> [v2:172.21.15.203:6832/1317108840,v1:172.21.15.203:6833/1317108840] conn(0x7fb4ec06a170 0x7fb4ec06c630 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7fb504034000 tx=0x7fb504009030 comp rx=0 tx=0).ready entity=mgr.6619 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-09T13:52:10.145 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:52:10.141+0000 7fb4fe7fc640 1 -- 172.21.15.43:0/598860555 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178066 (secure 0 0 0) 0x7fb4f8053be0 con 0x7fb508159f10 2023-08-09T13:52:10.342 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:52:10.340+0000 7fb50fb76640 1 -- 172.21.15.43:0/598860555 --> [v2:172.21.15.203:6832/1317108840,v1:172.21.15.203:6833/1317108840] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7fb50806bb10 con 0x7fb4ec06a170 2023-08-09T13:52:10.367 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:52:10.365+0000 7fb4fe7fc640 1 -- 172.21.15.43:0/598860555 <== mgr.6619 v2:172.21.15.203:6832/1317108840 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+309413 (secure 0 0 0) 0x7fb50403d040 con 0x7fb4ec06a170 2023-08-09T13:52:10.372 INFO:teuthology.orchestra.run.smithi043.stdout: 2023-08-09T13:52:10.373 INFO:teuthology.orchestra.run.smithi043.stderr:dumped all 2023-08-09T13:52:10.375 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:52:10.372+0000 7fb50fb76640 1 -- 172.21.15.43:0/598860555 >> [v2:172.21.15.203:6832/1317108840,v1:172.21.15.203:6833/1317108840] conn(0x7fb4ec06a170 msgr2=0x7fb4ec06c630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:52:10.375 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:52:10.372+0000 7fb50fb76640 1 --2- 172.21.15.43:0/598860555 >> [v2:172.21.15.203:6832/1317108840,v1:172.21.15.203:6833/1317108840] conn(0x7fb4ec06a170 0x7fb4ec06c630 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7fb504034000 tx=0x7fb504009030 comp rx=0 tx=0).stop 2023-08-09T13:52:10.375 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:52:10.372+0000 7fb50fb76640 1 -- 172.21.15.43:0/598860555 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb508159f10 msgr2=0x7fb5080c3930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:52:10.375 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:52:10.372+0000 7fb50fb76640 1 --2- 172.21.15.43:0/598860555 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb508159f10 0x7fb5080c3930 secure :-1 s=READY pgs=1360 cs=0 l=1 rev1=1 crypto rx=0x7fb4f8001eb0 tx=0x7fb4f800d9d0 comp rx=0 tx=0).stop 2023-08-09T13:52:10.375 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:52:10.372+0000 7fb50fb76640 1 -- 172.21.15.43:0/598860555 shutdown_connections 2023-08-09T13:52:10.376 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:52:10.372+0000 7fb50fb76640 1 --2- 172.21.15.43:0/598860555 >> [v2:172.21.15.203:6832/1317108840,v1:172.21.15.203:6833/1317108840] conn(0x7fb4ec06a170 0x7fb4ec06c630 unknown :-1 s=CLOSED pgs=70 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:52:10.376 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:52:10.372+0000 7fb50fb76640 1 --2- 172.21.15.43:0/598860555 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fb5080c3e70 0x7fb5080b3130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:52:10.376 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:52:10.373+0000 7fb50fb76640 1 --2- 172.21.15.43:0/598860555 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb508159f10 0x7fb5080c3930 unknown :-1 s=CLOSED pgs=1360 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:52:10.376 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:52:10.373+0000 7fb50fb76640 1 --2- 172.21.15.43:0/598860555 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fb4f40039e0 0x7fb5080c5170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:52:10.376 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:52:10.373+0000 7fb50fb76640 1 -- 172.21.15.43:0/598860555 >> 172.21.15.43:0/598860555 conn(0x7fb5080ce0a0 msgr2=0x7fb5080adc00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-09T13:52:10.376 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:52:10.373+0000 7fb50fb76640 1 -- 172.21.15.43:0/598860555 shutdown_connections 2023-08-09T13:52:10.377 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:52:10.373+0000 7fb50fb76640 1 -- 172.21.15.43:0/598860555 wait complete. 2023-08-09T13:52:10.395 INFO:teuthology.orchestra.run.smithi043.stdout:{"pg_ready":true,"pg_map":{"version":29,"stamp":"2023-08-09T13:52:08.573533+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":715,"num_read_kb":1418,"num_write":328,"num_write_kb":2430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":225,"ondisk_log_size":225,"up":210,"acting":210,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":210,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":308040,"kb_used_data":12384,"kb_used_omap":13,"kb_used_meta":295538,"kb_avail":754666680,"statfs":{"total":773094113280,"available":772778680320,"internally_reserved":0,"allocated":12681216,"data_stored":7282986,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":14248,"internal_metadata":302631000},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":32,"apply_latency_ms":32,"commit_latency_ns":32000000,"apply_latency_ns":32000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"8.389283"},"pg_stats":[{"pgid":"15.3f","version":"0'0","reported_seq":11,"reported_epoch":116,"state":"active+clean","last_fresh":"2023-08-09T13:52:01.220181+0000","last_change":"2023-08-09T13:52:00.224715+0000","last_active":"2023-08-09T13:52:01.220181+0000","last_peered":"2023-08-09T13:52:01.220181+0000","last_clean":"2023-08-09T13:52:01.220181+0000","last_became_active":"2023-08-09T13:52:00.224404+0000","last_became_peered":"2023-08-09T13:52:00.224404+0000","last_unstale":"2023-08-09T13:52:01.220181+0000","last_undegraded":"2023-08-09T13:52:01.220181+0000","last_fullsized":"2023-08-09T13:52:01.220181+0000","mapping_epoch":114,"log_start":"0'0","ondisk_log_start":"0'0","created":114,"last_epoch_clean":115,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:51:59.130555+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:51:59.130555+0000","last_clean_scrub_stamp":"2023-08-09T13:51:59.130555+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-11T00:54:55.831419+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"15.3e","version":"0'0","reported_seq":11,"reported_epoch":116,"state":"active+clean","last_fresh":"2023-08-09T13:52:02.227521+0000","last_change":"2023-08-09T13:52:00.218198+0000","last_active":"2023-08-09T13:52:02.227521+0000","last_peered":"2023-08-09T13:52:02.227521+0000","last_clean":"2023-08-09T13:52:02.227521+0000","last_became_active":"2023-08-09T13:52:00.217280+0000","last_became_peered":"2023-08-09T13:52:00.217280+0000","last_unstale":"2023-08-09T13:52:02.227521+0000","last_undegraded":"2023-08-09T13:52:02.227521+0000","last_fullsized":"2023-08-09T13:52:02.227521+0000","mapping_epoch":114,"log_start":"0'0","ondisk_log_start":"0'0","created":114,"last_epoch_clean":115,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:51:59.130555+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:51:59.130555+0000","last_clean_scrub_stamp":"2023-08-09T13:51:59.130555+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T16:40:23.268291+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.3d","version":"0'0","reported_seq":11,"reported_epoch":116,"state":"active+clean","last_fresh":"2023-08-09T13:52:01.220028+0000","last_change":"2023-08-09T13:52:00.224291+0000","last_active":"2023-08-09T13:52:01.220028+0000","last_peered":"2023-08-09T13:52:01.220028+0000","last_clean":"2023-08-09T13:52:01.220028+0000","last_became_active":"2023-08-09T13:52:00.223948+0000","last_became_peered":"2023-08-09T13:52:00.223948+0000","last_unstale":"2023-08-09T13:52:01.220028+0000","last_undegraded":"2023-08-09T13:52:01.220028+0000","last_fullsized":"2023-08-09T13:52:01.220028+0000","mapping_epoch":114,"log_start":"0'0","ondisk_log_start":"0'0","created":114,"last_epoch_clean":115,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:51:59.130555+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:51:59.130555+0000","last_clean_scrub_stamp":"2023-08-09T13:51:59.130555+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T16:56:55.245269+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"15.3c","version":"0'0","reported_seq":11,"reported_epoch":116,"state":"active+clean","last_fresh":"2023-08-09T13:52:01.221601+0000","last_change":"2023-08-09T13:52:00.211918+0000","last_active":"2023-08-09T13:52:01.221601+0000","last_peered":"2023-08-09T13:52:01.221601+0000","last_clean":"2023-08-09T13:52:01.221601+0000","last_became_active":"2023-08-09T13:52:00.210972+0000","last_became_peered":"2023-08-09T13:52:00.210972+0000","last_unstale":"2023-08-09T13:52:01.221601+0000","last_undegraded":"2023-08-09T13:52:01.221601+0000","last_fullsized":"2023-08-09T13:52:01.221601+0000","mapping_epoch":114,"log_start":"0'0","ondisk_log_start":"0'0","created":114,"last_epoch_clean":115,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:51:59.130555+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:51:59.130555+0000","last_clean_scrub_stamp":"2023-08-09T13:51:59.130555+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T23:44:30.232510+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"15.3b","version":"0'0","reported_seq":11,"reported_epoch":116,"state":"active+clean","last_fresh":"2023-08-09T13:52:02.229246+0000","last_change":"2023-08-09T13:52:00.215126+0000","last_active":"2023-08-09T13:52:02.229246+0000","last_peered":"2023-08-09T13:52:02.229246+0000","last_clean":"2023-08-09T13:52:02.229246+0000","last_became_active":"2023-08-09T13:52:00.214737+0000","last_became_peered":"2023-08-09T13:52:00.214737+0000","last_unstale":"2023-08-09T13:52:02.229246+0000","last_undegraded":"2023-08-09T13:52:02.229246+0000","last_fullsized":"2023-08-09T13:52:02.229246+0000","mapping_epoch":114,"log_start":"0'0","ondisk_log_start":"0'0","created":114,"last_epoch_clean":115,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:51:59.130555+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:51:59.130555+0000","last_clean_scrub_stamp":"2023-08-09T13:51:59.130555+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T14:03:42.929353+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.3a","version":"0'0","reported_seq":11,"reported_epoch":116,"state":"active+clean","last_fresh":"2023-08-09T13:52:02.228024+0000","last_change":"2023-08-09T13:52:00.226139+0000","last_active":"2023-08-09T13:52:02.228024+0000","last_peered":"2023-08-09T13:52:02.228024+0000","last_clean":"2023-08-09T13:52:02.228024+0000","last_became_active":"2023-08-09T13:52:00.225854+0000","last_became_peered":"2023-08-09T13:52:00.225854+0000","last_unstale":"2023-08-09T13:52:02.228024+0000","last_undegraded":"2023-08-09T13:52:02.228024+0000","last_fullsized":"2023-08-09T13:52:02.228024+0000","mapping_epoch":114,"log_start":"0'0","ondisk_log_start":"0'0","created":114,"last_epoch_clean":115,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:51:59.130555+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:51:59.130555+0000","last_clean_scrub_stamp":"2023-08-09T13:51:59.130555+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-11T00:12:54.873335+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"15.39","version":"0'0","reported_seq":11,"reported_epoch":116,"state":"active+clean","last_fresh":"2023-08-09T13:52:02.226730+0000","last_change":"2023-08-09T13:52:00.220933+0000","last_active":"2023-08-09T13:52:02.226730+0000","last_peered":"2023-08-09T13:52:02.226730+0000","last_clean":"2023-08-09T13:52:02.226730+0000","last_became_active":"2023-08-09T13:52:00.220647+0000","last_became_peered":"2023-08-09T13:52:00.220647+0000","last_unstale":"2023-08-09T13:52:02.226730+0000","last_undegraded":"2023-08-09T13:52:02.226730+0000","last_fullsized":"2023-08-09T13:52:02.226730+0000","mapping_epoch":114,"log_start":"0'0","ondisk_log_start":"0'0","created":114,"last_epoch_clean":115,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:51:59.130555+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:51:59.130555+0000","last_clean_scrub_stamp":"2023-08-09T13:51:59.130555+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T17:54:11.970806+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"15.38","version":"0'0","reported_seq":11,"reported_epoch":116,"state":"active+clean","last_fresh":"2023-08-09T13:52:02.226879+0000","last_change":"2023-08-09T13:52:00.204279+0000","last_active":"2023-08-09T13:52:02.226879+0000","last_peered":"2023-08-09T13:52:02.226879+0000","last_clean":"2023-08-09T13:52:02.226879+0000","last_became_active":"2023-08-09T13:52:00.203894+0000","last_became_peered":"2023-08-09T13:52:00.203894+0000","last_unstale":"2023-08-09T13:52:02.226879+0000","last_undegraded":"2023-08-09T13:52:02.226879+0000","last_fullsized":"2023-08-09T13:52:02.226879+0000","mapping_epoch":114,"log_start":"0'0","ondisk_log_start":"0'0","created":114,"last_epoch_clean":115,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:51:59.130555+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:51:59.130555+0000","last_clean_scrub_stamp":"2023-08-09T13:51:59.130555+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-11T00:36:05.454609+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.37","version":"0'0","reported_seq":11,"reported_epoch":116,"state":"active+clean","last_fresh":"2023-08-09T13:52:01.223149+0000","last_change":"2023-08-09T13:52:00.227036+0000","last_active":"2023-08-09T13:52:01.223149+0000","last_peered":"2023-08-09T13:52:01.223149+0000","last_clean":"2023-08-09T13:52:01.223149+0000","last_became_active":"2023-08-09T13:52:00.225864+0000","last_became_peered":"2023-08-09T13:52:00.225864+0000","last_unstale":"2023-08-09T13:52:01.223149+0000","last_undegraded":"2023-08-09T13:52:01.223149+0000","last_fullsized":"2023-08-09T13:52:01.223149+0000","mapping_epoch":114,"log_start":"0'0","ondisk_log_start":"0'0","created":114,"last_epoch_clean":115,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:51:59.130555+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:51:59.130555+0000","last_clean_scrub_stamp":"2023-08-09T13:51:59.130555+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T19:04:21.591286+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"15.36","version":"0'0","reported_seq":11,"reported_epoch":116,"state":"active+clean","last_fresh":"2023-08-09T13:52:02.230598+0000","last_change":"2023-08-09T13:52:00.227896+0000","last_active":"2023-08-09T13:52:02.230598+0000","last_peered":"2023-08-09T13:52:02.230598+0000","last_clean":"2023-08-09T13:52:02.230598+0000","last_became_active":"2023-08-09T13:52:00.227619+0000","last_became_peered":"2023-08-09T13:52:00.227619+0000","last_unstale":"2023-08-09T13:52:02.230598+0000","last_undegraded":"2023-08-09T13:52:02.230598+0000","last_fullsized":"2023-08-09T13:52:02.230598+0000","mapping_epoch":114,"log_start":"0'0","ondisk_log_start":"0'0","created":114,"last_epoch_clean":115,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:51:59.130555+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:51:59.130555+0000","last_clean_scrub_stamp":"2023-08-09T13:51:59.130555+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T16:11:26.030519+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.35","version":"0'0","reported_seq":11,"reported_epoch":116,"state":"active+clean","last_fresh":"2023-08-09T13:52:02.227069+0000","last_change":"2023-08-09T13:52:00.219649+0000","last_active":"2023-08-09T13:52:02.227069+0000","last_peered":"2023-08-09T13:52:02.227069+0000","last_clean":"2023-08-09T13:52:02.227069+0000","last_became_active":"2023-08-09T13:52:00.219213+0000","last_became_peered":"2023-08-09T13:52:00.219213+0000","last_unstale":"2023-08-09T13:52:02.227069+0000","last_undegraded":"2023-08-09T13:52:02.227069+0000","last_fullsized":"2023-08-09T13:52:02.227069+0000","mapping_epoch":114,"log_start":"0'0","ondisk_log_start":"0'0","created":114,"last_epoch_clean":115,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:51:59.130555+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:51:59.130555+0000","last_clean_scrub_stamp":"2023-08-09T13:51:59.130555+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T17:30:32.182529+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"15.34","version":"0'0","reported_seq":11,"reported_epoch":116,"state":"active+clean","last_fresh":"2023-08-09T13:52:01.222814+0000","last_change":"2023-08-09T13:52:00.233213+0000","last_active":"2023-08-09T13:52:01.222814+0000","last_peered":"2023-08-09T13:52:01.222814+0000","last_clean":"2023-08-09T13:52:01.222814+0000","last_became_active":"2023-08-09T13:52:00.232595+0000","last_became_peered":"2023-08-09T13:52:00.232595+0000","last_unstale":"2023-08-09T13:52:01.222814+0000","last_undegraded":"2023-08-09T13:52:01.222814+0000","last_fullsized":"2023-08-09T13:52:01.222814+0000","mapping_epoch":114,"log_start":"0'0","ondisk_log_start":"0'0","created":114,"last_epoch_clean":115,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:51:59.130555+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:51:59.130555+0000","last_clean_scrub_stamp":"2023-08-09T13:51:59.130555+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T18:35:08.302484+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"15.33","version":"0'0","reported_seq":11,"reported_epoch":116,"state":"active+clean","last_fresh":"2023-08-09T13:52:01.221386+0000","last_change":"2023-08-09T13:52:00.227803+0000","last_active":"2023-08-09T13:52:01.221386+0000","last_peered":"2023-08-09T13:52:01.221386+0000","last_clean":"2023-08-09T13:52:01.221386+0000","last_became_active":"2023-08-09T13:52:00.227454+0000","last_became_peered":"2023-08-09T13:52:00.227454+0000","last_unstale":"2023-08-09T13:52:01.221386+0000","last_undegraded":"2023-08-09T13:52:01.221386+0000","last_fullsized":"2023-08-09T13:52:01.221386+0000","mapping_epoch":114,"log_start":"0'0","ondisk_log_start":"0'0","created":114,"last_epoch_clean":115,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:51:59.130555+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:51:59.130555+0000","last_clean_scrub_stamp":"2023-08-09T13:51:59.130555+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T17:00:08.383368+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"15.32","version":"0'0","reported_seq":11,"reported_epoch":116,"state":"active+clean","last_fresh":"2023-08-09T13:52:01.418792+0000","last_change":"2023-08-09T13:52:00.228808+0000","last_active":"2023-08-09T13:52:01.418792+0000","last_peered":"2023-08-09T13:52:01.418792+0000","last_clean":"2023-08-09T13:52:01.418792+0000","last_became_active":"2023-08-09T13:52:00.228402+0000","last_became_peered":"2023-08-09T13:52:00.228402+0000","last_unstale":"2023-08-09T13:52:01.418792+0000","last_undegraded":"2023-08-09T13:52:01.418792+0000","last_fullsized":"2023-08-09T13:52:01.418792+0000","mapping_epoch":114,"log_start":"0'0","ondisk_log_start":"0'0","created":114,"last_epoch_clean":115,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:51:59.130555+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:51:59.130555+0000","last_clean_scrub_stamp":"2023-08-09T13:51:59.130555+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T13:59:07.741588+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.31","version":"0'0","reported_seq":11,"reported_epoch":116,"state":"active+clean","last_fresh":"2023-08-09T13:52:01.221161+0000","last_change":"2023-08-09T13:52:00.217670+0000","last_active":"2023-08-09T13:52:01.221161+0000","last_peered":"2023-08-09T13:52:01.221161+0000","last_clean":"2023-08-09T13:52:01.221161+0000","last_became_active":"2023-08-09T13:52:00.217394+0000","last_became_peered":"2023-08-09T13:52:00.217394+0000","last_unstale":"2023-08-09T13:52:01.221161+0000","last_undegraded":"2023-08-09T13:52:01.221161+0000","last_fullsized":"2023-08-09T13:52:01.221161+0000","mapping_epoch":114,"log_start":"0'0","ondisk_log_start":"0'0","created":114,"last_epoch_clean":115,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:51:59.130555+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:51:59.130555+0000","last_clean_scrub_stamp":"2023-08-09T13:51:59.130555+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T17:30:32.182529+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"15.30","version":"0'0","reported_seq":11,"reported_epoch":116,"state":"active+clean","last_fresh":"2023-08-09T13:52:02.226714+0000","last_change":"2023-08-09T13:52:00.207684+0000","last_active":"2023-08-09T13:52:02.226714+0000","last_peered":"2023-08-09T13:52:02.226714+0000","last_clean":"2023-08-09T13:52:02.226714+0000","last_became_active":"2023-08-09T13:52:00.207349+0000","last_became_peered":"2023-08-09T13:52:00.207349+0000","last_unstale":"2023-08-09T13:52:02.226714+0000","last_undegraded":"2023-08-09T13:52:02.226714+0000","last_fullsized":"2023-08-09T13:52:02.226714+0000","mapping_epoch":114,"log_start":"0'0","ondisk_log_start":"0'0","created":114,"last_epoch_clean":115,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:51:59.130555+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:51:59.130555+0000","last_clean_scrub_stamp":"2023-08-09T13:51:59.130555+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T18:47:38.882381+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"15.2f","version":"0'0","reported_seq":11,"reported_epoch":116,"state":"active+clean","last_fresh":"2023-08-09T13:52:02.227599+0000","last_change":"2023-08-09T13:52:00.207941+0000","last_active":"2023-08-09T13:52:02.227599+0000","last_peered":"2023-08-09T13:52:02.227599+0000","last_clean":"2023-08-09T13:52:02.227599+0000","last_became_active":"2023-08-09T13:52:00.207602+0000","last_became_peered":"2023-08-09T13:52:00.207602+0000","last_unstale":"2023-08-09T13:52:02.227599+0000","last_undegraded":"2023-08-09T13:52:02.227599+0000","last_fullsized":"2023-08-09T13:52:02.227599+0000","mapping_epoch":114,"log_start":"0'0","ondisk_log_start":"0'0","created":114,"last_epoch_clean":115,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:51:59.130555+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:51:59.130555+0000","last_clean_scrub_stamp":"2023-08-09T13:51:59.130555+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T21:35:51.983564+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.2e","version":"0'0","reported_seq":11,"reported_epoch":116,"state":"active+clean","last_fresh":"2023-08-09T13:52:01.221775+0000","last_change":"2023-08-09T13:52:00.211412+0000","last_active":"2023-08-09T13:52:01.221775+0000","last_peered":"2023-08-09T13:52:01.221775+0000","last_clean":"2023-08-09T13:52:01.221775+0000","last_became_active":"2023-08-09T13:52:00.211012+0000","last_became_peered":"2023-08-09T13:52:00.211012+0000","last_unstale":"2023-08-09T13:52:01.221775+0000","last_undegraded":"2023-08-09T13:52:01.221775+0000","last_fullsized":"2023-08-09T13:52:01.221775+0000","mapping_epoch":114,"log_start":"0'0","ondisk_log_start":"0'0","created":114,"last_epoch_clean":115,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:51:59.130555+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:51:59.130555+0000","last_clean_scrub_stamp":"2023-08-09T13:51:59.130555+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-11T00:12:54.873335+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"15.2d","version":"0'0","reported_seq":11,"reported_epoch":116,"state":"active+clean","last_fresh":"2023-08-09T13:52:01.419294+0000","last_change":"2023-08-09T13:52:00.220158+0000","last_active":"2023-08-09T13:52:01.419294+0000","last_peered":"2023-08-09T13:52:01.419294+0000","last_clean":"2023-08-09T13:52:01.419294+0000","last_became_active":"2023-08-09T13:52:00.219834+0000","last_became_peered":"2023-08-09T13:52:00.219834+0000","last_unstale":"2023-08-09T13:52:01.419294+0000","last_undegraded":"2023-08-09T13:52:01.419294+0000","last_fullsized":"2023-08-09T13:52:01.419294+0000","mapping_epoch":114,"log_start":"0'0","ondisk_log_start":"0'0","created":114,"last_epoch_clean":115,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:51:59.130555+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:51:59.130555+0000","last_clean_scrub_stamp":"2023-08-09T13:51:59.130555+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T19:02:16.257928+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.2c","version":"0'0","reported_seq":11,"reported_epoch":116,"state":"active+clean","last_fresh":"2023-08-09T13:52:01.222416+0000","last_change":"2023-08-09T13:52:00.227515+0000","last_active":"2023-08-09T13:52:01.222416+0000","last_peered":"2023-08-09T13:52:01.222416+0000","last_clean":"2023-08-09T13:52:01.222416+0000","last_became_active":"2023-08-09T13:52:00.227142+0000","last_became_peered":"2023-08-09T13:52:00.227142+0000","last_unstale":"2023-08-09T13:52:01.222416+0000","last_undegraded":"2023-08-09T13:52:01.222416+0000","last_fullsized":"2023-08-09T13:52:01.222416+0000","mapping_epoch":114,"log_start":"0'0","ondisk_log_start":"0'0","created":114,"last_epoch_clean":115,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:51:59.130555+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:51:59.130555+0000","last_clean_scrub_stamp":"2023-08-09T13:51:59.130555+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T14:22:06.615562+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"15.2b","version":"0'0","reported_seq":11,"reported_epoch":116,"state":"active+clean","last_fresh":"2023-08-09T13:52:01.419015+0000","last_change":"2023-08-09T13:52:00.226181+0000","last_active":"2023-08-09T13:52:01.419015+0000","last_peered":"2023-08-09T13:52:01.419015+0000","last_clean":"2023-08-09T13:52:01.419015+0000","last_became_active":"2023-08-09T13:52:00.225876+0000","last_became_peered":"2023-08-09T13:52:00.225876+0000","last_unstale":"2023-08-09T13:52:01.419015+0000","last_undegraded":"2023-08-09T13:52:01.419015+0000","last_fullsized":"2023-08-09T13:52:01.419015+0000","mapping_epoch":114,"log_start":"0'0","ondisk_log_start":"0'0","created":114,"last_epoch_clean":115,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:51:59.130555+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:51:59.130555+0000","last_clean_scrub_stamp":"2023-08-09T13:51:59.130555+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T17:38:57.970130+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.2a","version":"0'0","reported_seq":11,"reported_epoch":116,"state":"active+clean","last_fresh":"2023-08-09T13:52:02.226954+0000","last_change":"2023-08-09T13:52:00.217756+0000","last_active":"2023-08-09T13:52:02.226954+0000","last_peered":"2023-08-09T13:52:02.226954+0000","last_clean":"2023-08-09T13:52:02.226954+0000","last_became_active":"2023-08-09T13:52:00.217441+0000","last_became_peered":"2023-08-09T13:52:00.217441+0000","last_unstale":"2023-08-09T13:52:02.226954+0000","last_undegraded":"2023-08-09T13:52:02.226954+0000","last_fullsized":"2023-08-09T13:52:02.226954+0000","mapping_epoch":114,"log_start":"0'0","ondisk_log_start":"0'0","created":114,"last_epoch_clean":115,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:51:59.130555+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:51:59.130555+0000","last_clean_scrub_stamp":"2023-08-09T13:51:59.130555+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-11T00:34:01.999153+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.29","version":"0'0","reported_seq":11,"reported_epoch":116,"state":"active+clean","last_fresh":"2023-08-09T13:52:02.226935+0000","last_change":"2023-08-09T13:52:00.215869+0000","last_active":"2023-08-09T13:52:02.226935+0000","last_peered":"2023-08-09T13:52:02.226935+0000","last_clean":"2023-08-09T13:52:02.226935+0000","last_became_active":"2023-08-09T13:52:00.215592+0000","last_became_peered":"2023-08-09T13:52:00.215592+0000","last_unstale":"2023-08-09T13:52:02.226935+0000","last_undegraded":"2023-08-09T13:52:02.226935+0000","last_fullsized":"2023-08-09T13:52:02.226935+0000","mapping_epoch":114,"log_start":"0'0","ondisk_log_start":"0'0","created":114,"last_epoch_clean":115,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:51:59.130555+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:51:59.130555+0000","last_clean_scrub_stamp":"2023-08-09T13:51:59.130555+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T22:31:49.221924+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.28","version":"0'0","reported_seq":11,"reported_epoch":116,"state":"active+clean","last_fresh":"2023-08-09T13:52:02.226788+0000","last_change":"2023-08-09T13:52:00.230092+0000","last_active":"2023-08-09T13:52:02.226788+0000","last_peered":"2023-08-09T13:52:02.226788+0000","last_clean":"2023-08-09T13:52:02.226788+0000","last_became_active":"2023-08-09T13:52:00.229786+0000","last_became_peered":"2023-08-09T13:52:00.229786+0000","last_unstale":"2023-08-09T13:52:02.226788+0000","last_undegraded":"2023-08-09T13:52:02.226788+0000","last_fullsized":"2023-08-09T13:52:02.226788+0000","mapping_epoch":114,"log_start":"0'0","ondisk_log_start":"0'0","created":114,"last_epoch_clean":115,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:51:59.130555+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:51:59.130555+0000","last_clean_scrub_stamp":"2023-08-09T13:51:59.130555+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T23:11:37.527659+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"15.27","version":"0'0","reported_seq":11,"reported_epoch":116,"state":"active+clean","last_fresh":"2023-08-09T13:52:02.229935+0000","last_change":"2023-08-09T13:52:00.224333+0000","last_active":"2023-08-09T13:52:02.229935+0000","last_peered":"2023-08-09T13:52:02.229935+0000","last_clean":"2023-08-09T13:52:02.229935+0000","last_became_active":"2023-08-09T13:52:00.224023+0000","last_became_peered":"2023-08-09T13:52:00.224023+0000","last_unstale":"2023-08-09T13:52:02.229935+0000","last_undegraded":"2023-08-09T13:52:02.229935+0000","last_fullsized":"2023-08-09T13:52:02.229935+0000","mapping_epoch":114,"log_start":"0'0","ondisk_log_start":"0'0","created":114,"last_epoch_clean":115,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:51:59.130555+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:51:59.130555+0000","last_clean_scrub_stamp":"2023-08-09T13:51:59.130555+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-11T01:00:31.885923+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.26","version":"0'0","reported_seq":11,"reported_epoch":116,"state":"active+clean","last_fresh":"2023-08-09T13:52:02.227692+0000","last_change":"2023-08-09T13:52:00.221166+0000","last_active":"2023-08-09T13:52:02.227692+0000","last_peered":"2023-08-09T13:52:02.227692+0000","last_clean":"2023-08-09T13:52:02.227692+0000","last_became_active":"2023-08-09T13:52:00.220856+0000","last_became_peered":"2023-08-09T13:52:00.220856+0000","last_unstale":"2023-08-09T13:52:02.227692+0000","last_undegraded":"2023-08-09T13:52:02.227692+0000","last_fullsized":"2023-08-09T13:52:02.227692+0000","mapping_epoch":114,"log_start":"0'0","ondisk_log_start":"0'0","created":114,"last_epoch_clean":115,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:51:59.130555+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:51:59.130555+0000","last_clean_scrub_stamp":"2023-08-09T13:51:59.130555+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T14:42:50.977974+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.25","version":"0'0","reported_seq":11,"reported_epoch":116,"state":"active+clean","last_fresh":"2023-08-09T13:52:02.227139+0000","last_change":"2023-08-09T13:52:00.221719+0000","last_active":"2023-08-09T13:52:02.227139+0000","last_peered":"2023-08-09T13:52:02.227139+0000","last_clean":"2023-08-09T13:52:02.227139+0000","last_became_active":"2023-08-09T13:52:00.221398+0000","last_became_peered":"2023-08-09T13:52:00.221398+0000","last_unstale":"2023-08-09T13:52:02.227139+0000","last_undegraded":"2023-08-09T13:52:02.227139+0000","last_fullsized":"2023-08-09T13:52:02.227139+0000","mapping_epoch":114,"log_start":"0'0","ondisk_log_start":"0'0","created":114,"last_epoch_clean":115,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:51:59.130555+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:51:59.130555+0000","last_clean_scrub_stamp":"2023-08-09T13:51:59.130555+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T21:54:42.771899+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"15.24","version":"0'0","reported_seq":11,"reported_epoch":116,"state":"active+clean","last_fresh":"2023-08-09T13:52:02.229433+0000","last_change":"2023-08-09T13:52:00.218234+0000","last_active":"2023-08-09T13:52:02.229433+0000","last_peered":"2023-08-09T13:52:02.229433+0000","last_clean":"2023-08-09T13:52:02.229433+0000","last_became_active":"2023-08-09T13:52:00.217730+0000","last_became_peered":"2023-08-09T13:52:00.217730+0000","last_unstale":"2023-08-09T13:52:02.229433+0000","last_undegraded":"2023-08-09T13:52:02.229433+0000","last_fullsized":"2023-08-09T13:52:02.229433+0000","mapping_epoch":114,"log_start":"0'0","ondisk_log_start":"0'0","created":114,"last_epoch_clean":115,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:51:59.130555+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:51:59.130555+0000","last_clean_scrub_stamp":"2023-08-09T13:51:59.130555+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T17:28:44.810490+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.23","version":"0'0","reported_seq":11,"reported_epoch":116,"state":"active+clean","last_fresh":"2023-08-09T13:52:02.227717+0000","last_change":"2023-08-09T13:52:00.222136+0000","last_active":"2023-08-09T13:52:02.227717+0000","last_peered":"2023-08-09T13:52:02.227717+0000","last_clean":"2023-08-09T13:52:02.227717+0000","last_became_active":"2023-08-09T13:52:00.221485+0000","last_became_peered":"2023-08-09T13:52:00.221485+0000","last_unstale":"2023-08-09T13:52:02.227717+0000","last_undegraded":"2023-08-09T13:52:02.227717+0000","last_fullsized":"2023-08-09T13:52:02.227717+0000","mapping_epoch":114,"log_start":"0'0","ondisk_log_start":"0'0","created":114,"last_epoch_clean":115,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:51:59.130555+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:51:59.130555+0000","last_clean_scrub_stamp":"2023-08-09T13:51:59.130555+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-11T01:49:04.981047+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.22","version":"0'0","reported_seq":11,"reported_epoch":116,"state":"active+clean","last_fresh":"2023-08-09T13:52:01.418818+0000","last_change":"2023-08-09T13:52:00.220078+0000","last_active":"2023-08-09T13:52:01.418818+0000","last_peered":"2023-08-09T13:52:01.418818+0000","last_clean":"2023-08-09T13:52:01.418818+0000","last_became_active":"2023-08-09T13:52:00.219770+0000","last_became_peered":"2023-08-09T13:52:00.219770+0000","last_unstale":"2023-08-09T13:52:01.418818+0000","last_undegraded":"2023-08-09T13:52:01.418818+0000","last_fullsized":"2023-08-09T13:52:01.418818+0000","mapping_epoch":114,"log_start":"0'0","ondisk_log_start":"0'0","created":114,"last_epoch_clean":115,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:51:59.130555+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:51:59.130555+0000","last_clean_scrub_stamp":"2023-08-09T13:51:59.130555+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T18:22:00.956161+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.21","version":"0'0","reported_seq":11,"reported_epoch":116,"state":"active+clean","last_fresh":"2023-08-09T13:52:01.221780+0000","last_change":"2023-08-09T13:52:00.226969+0000","last_active":"2023-08-09T13:52:01.221780+0000","last_peered":"2023-08-09T13:52:01.221780+0000","last_clean":"2023-08-09T13:52:01.221780+0000","last_became_active":"2023-08-09T13:52:00.226594+0000","last_became_peered":"2023-08-09T13:52:00.226594+0000","last_unstale":"2023-08-09T13:52:01.221780+0000","last_undegraded":"2023-08-09T13:52:01.221780+0000","last_fullsized":"2023-08-09T13:52:01.221780+0000","mapping_epoch":114,"log_start":"0'0","ondisk_log_start":"0'0","created":114,"last_epoch_clean":115,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:51:59.130555+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:51:59.130555+0000","last_clean_scrub_stamp":"2023-08-09T13:51:59.130555+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T14:07:44.282339+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"15.20","version":"0'0","reported_seq":11,"reported_epoch":116,"state":"active+clean","last_fresh":"2023-08-09T13:52:01.219258+0000","last_change":"2023-08-09T13:52:00.217646+0000","last_active":"2023-08-09T13:52:01.219258+0000","last_peered":"2023-08-09T13:52:01.219258+0000","last_clean":"2023-08-09T13:52:01.219258+0000","last_became_active":"2023-08-09T13:52:00.217316+0000","last_became_peered":"2023-08-09T13:52:00.217316+0000","last_unstale":"2023-08-09T13:52:01.219258+0000","last_undegraded":"2023-08-09T13:52:01.219258+0000","last_fullsized":"2023-08-09T13:52:01.219258+0000","mapping_epoch":114,"log_start":"0'0","ondisk_log_start":"0'0","created":114,"last_epoch_clean":115,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:51:59.130555+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:51:59.130555+0000","last_clean_scrub_stamp":"2023-08-09T13:51:59.130555+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-11T00:58:17.334390+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"15.1e","version":"0'0","reported_seq":11,"reported_epoch":116,"state":"active+clean","last_fresh":"2023-08-09T13:52:01.419293+0000","last_change":"2023-08-09T13:52:00.211947+0000","last_active":"2023-08-09T13:52:01.419293+0000","last_peered":"2023-08-09T13:52:01.419293+0000","last_clean":"2023-08-09T13:52:01.419293+0000","last_became_active":"2023-08-09T13:52:00.211455+0000","last_became_peered":"2023-08-09T13:52:00.211455+0000","last_unstale":"2023-08-09T13:52:01.419293+0000","last_undegraded":"2023-08-09T13:52:01.419293+0000","last_fullsized":"2023-08-09T13:52:01.419293+0000","mapping_epoch":114,"log_start":"0'0","ondisk_log_start":"0'0","created":114,"last_epoch_clean":115,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:51:59.130555+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:51:59.130555+0000","last_clean_scrub_stamp":"2023-08-09T13:51:59.130555+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-11T01:31:43.361691+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"14.1f","version":"115'4","reported_seq":17,"reported_epoch":116,"state":"active+clean","last_fresh":"2023-08-09T13:52:02.227886+0000","last_change":"2023-08-09T13:51:59.176081+0000","last_active":"2023-08-09T13:52:02.227886+0000","last_peered":"2023-08-09T13:52:02.227886+0000","last_clean":"2023-08-09T13:52:02.227886+0000","last_became_active":"2023-08-09T13:51:59.175790+0000","last_became_peered":"2023-08-09T13:51:59.175790+0000","last_unstale":"2023-08-09T13:52:02.227886+0000","last_undegraded":"2023-08-09T13:52:02.227886+0000","last_fullsized":"2023-08-09T13:52:02.227886+0000","mapping_epoch":113,"log_start":"0'0","ondisk_log_start":"0'0","created":113,"last_epoch_clean":114,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:51:58.125661+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:51:58.125661+0000","last_clean_scrub_stamp":"2023-08-09T13:51:58.125661+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T22:25:52.108038+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.1f","version":"0'0","reported_seq":11,"reported_epoch":116,"state":"active+clean","last_fresh":"2023-08-09T13:52:01.221580+0000","last_change":"2023-08-09T13:52:00.226768+0000","last_active":"2023-08-09T13:52:01.221580+0000","last_peered":"2023-08-09T13:52:01.221580+0000","last_clean":"2023-08-09T13:52:01.221580+0000","last_became_active":"2023-08-09T13:52:00.226460+0000","last_became_peered":"2023-08-09T13:52:00.226460+0000","last_unstale":"2023-08-09T13:52:01.221580+0000","last_undegraded":"2023-08-09T13:52:01.221580+0000","last_fullsized":"2023-08-09T13:52:01.221580+0000","mapping_epoch":114,"log_start":"0'0","ondisk_log_start":"0'0","created":114,"last_epoch_clean":115,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:51:59.130555+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:51:59.130555+0000","last_clean_scrub_stamp":"2023-08-09T13:51:59.130555+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T23:42:47.388329+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"14.1e","version":"115'1","reported_seq":14,"reported_epoch":116,"state":"active+clean","last_fresh":"2023-08-09T13:52:02.227882+0000","last_change":"2023-08-09T13:51:59.178857+0000","last_active":"2023-08-09T13:52:02.227882+0000","last_peered":"2023-08-09T13:52:02.227882+0000","last_clean":"2023-08-09T13:52:02.227882+0000","last_became_active":"2023-08-09T13:51:59.178483+0000","last_became_peered":"2023-08-09T13:51:59.178483+0000","last_unstale":"2023-08-09T13:52:02.227882+0000","last_undegraded":"2023-08-09T13:52:02.227882+0000","last_fullsized":"2023-08-09T13:52:02.227882+0000","mapping_epoch":113,"log_start":"0'0","ondisk_log_start":"0'0","created":113,"last_epoch_clean":114,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:51:58.125661+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:51:58.125661+0000","last_clean_scrub_stamp":"2023-08-09T13:51:58.125661+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T22:50:15.633917+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"15.1c","version":"0'0","reported_seq":11,"reported_epoch":116,"state":"active+clean","last_fresh":"2023-08-09T13:52:01.219749+0000","last_change":"2023-08-09T13:52:00.228876+0000","last_active":"2023-08-09T13:52:01.219749+0000","last_peered":"2023-08-09T13:52:01.219749+0000","last_clean":"2023-08-09T13:52:01.219749+0000","last_became_active":"2023-08-09T13:52:00.227420+0000","last_became_peered":"2023-08-09T13:52:00.227420+0000","last_unstale":"2023-08-09T13:52:01.219749+0000","last_undegraded":"2023-08-09T13:52:01.219749+0000","last_fullsized":"2023-08-09T13:52:01.219749+0000","mapping_epoch":114,"log_start":"0'0","ondisk_log_start":"0'0","created":114,"last_epoch_clean":115,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:51:59.130555+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:51:59.130555+0000","last_clean_scrub_stamp":"2023-08-09T13:51:59.130555+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T22:09:46.463659+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.1d","version":"115'2","reported_seq":15,"reported_epoch":116,"state":"active+clean","last_fresh":"2023-08-09T13:52:02.227415+0000","last_change":"2023-08-09T13:51:59.168942+0000","last_active":"2023-08-09T13:52:02.227415+0000","last_peered":"2023-08-09T13:52:02.227415+0000","last_clean":"2023-08-09T13:52:02.227415+0000","last_became_active":"2023-08-09T13:51:59.168599+0000","last_became_peered":"2023-08-09T13:51:59.168599+0000","last_unstale":"2023-08-09T13:52:02.227415+0000","last_undegraded":"2023-08-09T13:52:02.227415+0000","last_fullsized":"2023-08-09T13:52:02.227415+0000","mapping_epoch":113,"log_start":"0'0","ondisk_log_start":"0'0","created":113,"last_epoch_clean":114,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:51:58.125661+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:51:58.125661+0000","last_clean_scrub_stamp":"2023-08-09T13:51:58.125661+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-11T00:44:59.042168+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.1d","version":"0'0","reported_seq":11,"reported_epoch":116,"state":"active+clean","last_fresh":"2023-08-09T13:52:01.222594+0000","last_change":"2023-08-09T13:52:00.223439+0000","last_active":"2023-08-09T13:52:01.222594+0000","last_peered":"2023-08-09T13:52:01.222594+0000","last_clean":"2023-08-09T13:52:01.222594+0000","last_became_active":"2023-08-09T13:52:00.222651+0000","last_became_peered":"2023-08-09T13:52:00.222651+0000","last_unstale":"2023-08-09T13:52:01.222594+0000","last_undegraded":"2023-08-09T13:52:01.222594+0000","last_fullsized":"2023-08-09T13:52:01.222594+0000","mapping_epoch":114,"log_start":"0'0","ondisk_log_start":"0'0","created":114,"last_epoch_clean":115,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:51:59.130555+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:51:59.130555+0000","last_clean_scrub_stamp":"2023-08-09T13:51:59.130555+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T16:15:13.180404+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.1c","version":"115'2","reported_seq":15,"reported_epoch":116,"state":"active+clean","last_fresh":"2023-08-09T13:52:02.227858+0000","last_change":"2023-08-09T13:51:59.183710+0000","last_active":"2023-08-09T13:52:02.227858+0000","last_peered":"2023-08-09T13:52:02.227858+0000","last_clean":"2023-08-09T13:52:02.227858+0000","last_became_active":"2023-08-09T13:51:59.183401+0000","last_became_peered":"2023-08-09T13:51:59.183401+0000","last_unstale":"2023-08-09T13:52:02.227858+0000","last_undegraded":"2023-08-09T13:52:02.227858+0000","last_fullsized":"2023-08-09T13:52:02.227858+0000","mapping_epoch":113,"log_start":"0'0","ondisk_log_start":"0'0","created":113,"last_epoch_clean":114,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:51:58.125661+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:51:58.125661+0000","last_clean_scrub_stamp":"2023-08-09T13:51:58.125661+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T17:51:46.462712+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.1a","version":"0'0","reported_seq":11,"reported_epoch":116,"state":"active+clean","last_fresh":"2023-08-09T13:52:02.228354+0000","last_change":"2023-08-09T13:52:00.214536+0000","last_active":"2023-08-09T13:52:02.228354+0000","last_peered":"2023-08-09T13:52:02.228354+0000","last_clean":"2023-08-09T13:52:02.228354+0000","last_became_active":"2023-08-09T13:52:00.214237+0000","last_became_peered":"2023-08-09T13:52:00.214237+0000","last_unstale":"2023-08-09T13:52:02.228354+0000","last_undegraded":"2023-08-09T13:52:02.228354+0000","last_fullsized":"2023-08-09T13:52:02.228354+0000","mapping_epoch":114,"log_start":"0'0","ondisk_log_start":"0'0","created":114,"last_epoch_clean":115,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:51:59.130555+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:51:59.130555+0000","last_clean_scrub_stamp":"2023-08-09T13:51:59.130555+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T14:23:12.776157+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"14.1b","version":"115'1","reported_seq":14,"reported_epoch":116,"state":"active+clean","last_fresh":"2023-08-09T13:52:01.219598+0000","last_change":"2023-08-09T13:51:59.178308+0000","last_active":"2023-08-09T13:52:01.219598+0000","last_peered":"2023-08-09T13:52:01.219598+0000","last_clean":"2023-08-09T13:52:01.219598+0000","last_became_active":"2023-08-09T13:51:59.178022+0000","last_became_peered":"2023-08-09T13:51:59.178022+0000","last_unstale":"2023-08-09T13:52:01.219598+0000","last_undegraded":"2023-08-09T13:52:01.219598+0000","last_fullsized":"2023-08-09T13:52:01.219598+0000","mapping_epoch":113,"log_start":"0'0","ondisk_log_start":"0'0","created":113,"last_epoch_clean":114,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:51:58.125661+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:51:58.125661+0000","last_clean_scrub_stamp":"2023-08-09T13:51:58.125661+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T15:02:20.908250+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"15.1b","version":"0'0","reported_seq":11,"reported_epoch":116,"state":"active+clean","last_fresh":"2023-08-09T13:52:02.226965+0000","last_change":"2023-08-09T13:52:00.216318+0000","last_active":"2023-08-09T13:52:02.226965+0000","last_peered":"2023-08-09T13:52:02.226965+0000","last_clean":"2023-08-09T13:52:02.226965+0000","last_became_active":"2023-08-09T13:52:00.216021+0000","last_became_peered":"2023-08-09T13:52:00.216021+0000","last_unstale":"2023-08-09T13:52:02.226965+0000","last_undegraded":"2023-08-09T13:52:02.226965+0000","last_fullsized":"2023-08-09T13:52:02.226965+0000","mapping_epoch":114,"log_start":"0'0","ondisk_log_start":"0'0","created":114,"last_epoch_clean":115,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:51:59.130555+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:51:59.130555+0000","last_clean_scrub_stamp":"2023-08-09T13:51:59.130555+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T18:50:44.479523+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"14.1a","version":"115'1","reported_seq":14,"reported_epoch":116,"state":"active+clean","last_fresh":"2023-08-09T13:52:02.226933+0000","last_change":"2023-08-09T13:51:59.165522+0000","last_active":"2023-08-09T13:52:02.226933+0000","last_peered":"2023-08-09T13:52:02.226933+0000","last_clean":"2023-08-09T13:52:02.226933+0000","last_became_active":"2023-08-09T13:51:59.165145+0000","last_became_peered":"2023-08-09T13:51:59.165145+0000","last_unstale":"2023-08-09T13:52:02.226933+0000","last_undegraded":"2023-08-09T13:52:02.226933+0000","last_fullsized":"2023-08-09T13:52:02.226933+0000","mapping_epoch":113,"log_start":"0'0","ondisk_log_start":"0'0","created":113,"last_epoch_clean":114,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:51:58.125661+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:51:58.125661+0000","last_clean_scrub_stamp":"2023-08-09T13:51:58.125661+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-11T01:20:41.335156+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.18","version":"0'0","reported_seq":11,"reported_epoch":116,"state":"active+clean","last_fresh":"2023-08-09T13:52:01.221194+0000","last_change":"2023-08-09T13:52:00.218747+0000","last_active":"2023-08-09T13:52:01.221194+0000","last_peered":"2023-08-09T13:52:01.221194+0000","last_clean":"2023-08-09T13:52:01.221194+0000","last_became_active":"2023-08-09T13:52:00.218470+0000","last_became_peered":"2023-08-09T13:52:00.218470+0000","last_unstale":"2023-08-09T13:52:01.221194+0000","last_undegraded":"2023-08-09T13:52:01.221194+0000","last_fullsized":"2023-08-09T13:52:01.221194+0000","mapping_epoch":114,"log_start":"0'0","ondisk_log_start":"0'0","created":114,"last_epoch_clean":115,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:51:59.130555+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:51:59.130555+0000","last_clean_scrub_stamp":"2023-08-09T13:51:59.130555+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T19:49:50.219295+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"14.19","version":"0'0","reported_seq":13,"reported_epoch":116,"state":"active+clean","last_fresh":"2023-08-09T13:52:02.229449+0000","last_change":"2023-08-09T13:51:59.174362+0000","last_active":"2023-08-09T13:52:02.229449+0000","last_peered":"2023-08-09T13:52:02.229449+0000","last_clean":"2023-08-09T13:52:02.229449+0000","last_became_active":"2023-08-09T13:51:59.174002+0000","last_became_peered":"2023-08-09T13:51:59.174002+0000","last_unstale":"2023-08-09T13:52:02.229449+0000","last_undegraded":"2023-08-09T13:52:02.229449+0000","last_fullsized":"2023-08-09T13:52:02.229449+0000","mapping_epoch":113,"log_start":"0'0","ondisk_log_start":"0'0","created":113,"last_epoch_clean":114,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:51:58.125661+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:51:58.125661+0000","last_clean_scrub_stamp":"2023-08-09T13:51:58.125661+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T20:09:44.063483+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.19","version":"0'0","reported_seq":11,"reported_epoch":116,"state":"active+clean","last_fresh":"2023-08-09T13:52:01.221492+0000","last_change":"2023-08-09T13:52:00.223692+0000","last_active":"2023-08-09T13:52:01.221492+0000","last_peered":"2023-08-09T13:52:01.221492+0000","last_clean":"2023-08-09T13:52:01.221492+0000","last_became_active":"2023-08-09T13:52:00.223359+0000","last_became_peered":"2023-08-09T13:52:00.223359+0000","last_unstale":"2023-08-09T13:52:01.221492+0000","last_undegraded":"2023-08-09T13:52:01.221492+0000","last_fullsized":"2023-08-09T13:52:01.221492+0000","mapping_epoch":114,"log_start":"0'0","ondisk_log_start":"0'0","created":114,"last_epoch_clean":115,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:51:59.130555+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:51:59.130555+0000","last_clean_scrub_stamp":"2023-08-09T13:51:59.130555+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T23:02:17.294628+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.18","version":"0'0","reported_seq":13,"reported_epoch":116,"state":"active+clean","last_fresh":"2023-08-09T13:52:01.219461+0000","last_change":"2023-08-09T13:51:59.177991+0000","last_active":"2023-08-09T13:52:01.219461+0000","last_peered":"2023-08-09T13:52:01.219461+0000","last_clean":"2023-08-09T13:52:01.219461+0000","last_became_active":"2023-08-09T13:51:59.177651+0000","last_became_peered":"2023-08-09T13:51:59.177651+0000","last_unstale":"2023-08-09T13:52:01.219461+0000","last_undegraded":"2023-08-09T13:52:01.219461+0000","last_fullsized":"2023-08-09T13:52:01.219461+0000","mapping_epoch":113,"log_start":"0'0","ondisk_log_start":"0'0","created":113,"last_epoch_clean":114,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:51:58.125661+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:51:58.125661+0000","last_clean_scrub_stamp":"2023-08-09T13:51:58.125661+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T15:55:33.249335+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"15.16","version":"0'0","reported_seq":11,"reported_epoch":116,"state":"active+clean","last_fresh":"2023-08-09T13:52:02.227886+0000","last_change":"2023-08-09T13:52:00.214111+0000","last_active":"2023-08-09T13:52:02.227886+0000","last_peered":"2023-08-09T13:52:02.227886+0000","last_clean":"2023-08-09T13:52:02.227886+0000","last_became_active":"2023-08-09T13:52:00.213831+0000","last_became_peered":"2023-08-09T13:52:00.213831+0000","last_unstale":"2023-08-09T13:52:02.227886+0000","last_undegraded":"2023-08-09T13:52:02.227886+0000","last_fullsized":"2023-08-09T13:52:02.227886+0000","mapping_epoch":114,"log_start":"0'0","ondisk_log_start":"0'0","created":114,"last_epoch_clean":115,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:51:59.130555+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:51:59.130555+0000","last_clean_scrub_stamp":"2023-08-09T13:51:59.130555+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T20:09:37.121769+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"14.17","version":"0'0","reported_seq":13,"reported_epoch":116,"state":"active+clean","last_fresh":"2023-08-09T13:52:01.220356+0000","last_change":"2023-08-09T13:51:59.181245+0000","last_active":"2023-08-09T13:52:01.220356+0000","last_peered":"2023-08-09T13:52:01.220356+0000","last_clean":"2023-08-09T13:52:01.220356+0000","last_became_active":"2023-08-09T13:51:59.180911+0000","last_became_peered":"2023-08-09T13:51:59.180911+0000","last_unstale":"2023-08-09T13:52:01.220356+0000","last_undegraded":"2023-08-09T13:52:01.220356+0000","last_fullsized":"2023-08-09T13:52:01.220356+0000","mapping_epoch":113,"log_start":"0'0","ondisk_log_start":"0'0","created":113,"last_epoch_clean":114,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:51:58.125661+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:51:58.125661+0000","last_clean_scrub_stamp":"2023-08-09T13:51:58.125661+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T21:59:54.047931+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"15.17","version":"0'0","reported_seq":11,"reported_epoch":116,"state":"active+clean","last_fresh":"2023-08-09T13:52:01.223360+0000","last_change":"2023-08-09T13:52:00.224692+0000","last_active":"2023-08-09T13:52:01.223360+0000","last_peered":"2023-08-09T13:52:01.223360+0000","last_clean":"2023-08-09T13:52:01.223360+0000","last_became_active":"2023-08-09T13:52:00.224355+0000","last_became_peered":"2023-08-09T13:52:00.224355+0000","last_unstale":"2023-08-09T13:52:01.223360+0000","last_undegraded":"2023-08-09T13:52:01.223360+0000","last_fullsized":"2023-08-09T13:52:01.223360+0000","mapping_epoch":114,"log_start":"0'0","ondisk_log_start":"0'0","created":114,"last_epoch_clean":115,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:51:59.130555+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:51:59.130555+0000","last_clean_scrub_stamp":"2023-08-09T13:51:59.130555+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T19:55:13.438705+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.16","version":"0'0","reported_seq":13,"reported_epoch":116,"state":"active+clean","last_fresh":"2023-08-09T13:52:01.221834+0000","last_change":"2023-08-09T13:51:59.172670+0000","last_active":"2023-08-09T13:52:01.221834+0000","last_peered":"2023-08-09T13:52:01.221834+0000","last_clean":"2023-08-09T13:52:01.221834+0000","last_became_active":"2023-08-09T13:51:59.172395+0000","last_became_peered":"2023-08-09T13:51:59.172395+0000","last_unstale":"2023-08-09T13:52:01.221834+0000","last_undegraded":"2023-08-09T13:52:01.221834+0000","last_fullsized":"2023-08-09T13:52:01.221834+0000","mapping_epoch":113,"log_start":"0'0","ondisk_log_start":"0'0","created":113,"last_epoch_clean":114,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:51:58.125661+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:51:58.125661+0000","last_clean_scrub_stamp":"2023-08-09T13:51:58.125661+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T15:33:24.805826+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"15.14","version":"0'0","reported_seq":11,"reported_epoch":116,"state":"active+clean","last_fresh":"2023-08-09T13:52:02.229499+0000","last_change":"2023-08-09T13:52:00.218051+0000","last_active":"2023-08-09T13:52:02.229499+0000","last_peered":"2023-08-09T13:52:02.229499+0000","last_clean":"2023-08-09T13:52:02.229499+0000","last_became_active":"2023-08-09T13:52:00.217381+0000","last_became_peered":"2023-08-09T13:52:00.217381+0000","last_unstale":"2023-08-09T13:52:02.229499+0000","last_undegraded":"2023-08-09T13:52:02.229499+0000","last_fullsized":"2023-08-09T13:52:02.229499+0000","mapping_epoch":114,"log_start":"0'0","ondisk_log_start":"0'0","created":114,"last_epoch_clean":115,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:51:59.130555+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:51:59.130555+0000","last_clean_scrub_stamp":"2023-08-09T13:51:59.130555+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T19:20:41.697718+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"14.15","version":"115'3","reported_seq":16,"reported_epoch":116,"state":"active+clean","last_fresh":"2023-08-09T13:52:02.227755+0000","last_change":"2023-08-09T13:51:59.181964+0000","last_active":"2023-08-09T13:52:02.227755+0000","last_peered":"2023-08-09T13:52:02.227755+0000","last_clean":"2023-08-09T13:52:02.227755+0000","last_became_active":"2023-08-09T13:51:59.181630+0000","last_became_peered":"2023-08-09T13:51:59.181630+0000","last_unstale":"2023-08-09T13:52:02.227755+0000","last_undegraded":"2023-08-09T13:52:02.227755+0000","last_fullsized":"2023-08-09T13:52:02.227755+0000","mapping_epoch":113,"log_start":"0'0","ondisk_log_start":"0'0","created":113,"last_epoch_clean":114,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:51:58.125661+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:51:58.125661+0000","last_clean_scrub_stamp":"2023-08-09T13:51:58.125661+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T23:44:29.227616+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"15.15","version":"0'0","reported_seq":11,"reported_epoch":116,"state":"active+clean","last_fresh":"2023-08-09T13:52:02.227429+0000","last_change":"2023-08-09T13:52:00.210997+0000","last_active":"2023-08-09T13:52:02.227429+0000","last_peered":"2023-08-09T13:52:02.227429+0000","last_clean":"2023-08-09T13:52:02.227429+0000","last_became_active":"2023-08-09T13:52:00.210530+0000","last_became_peered":"2023-08-09T13:52:00.210530+0000","last_unstale":"2023-08-09T13:52:02.227429+0000","last_undegraded":"2023-08-09T13:52:02.227429+0000","last_fullsized":"2023-08-09T13:52:02.227429+0000","mapping_epoch":114,"log_start":"0'0","ondisk_log_start":"0'0","created":114,"last_epoch_clean":115,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:51:59.130555+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:51:59.130555+0000","last_clean_scrub_stamp":"2023-08-09T13:51:59.130555+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T17:00:08.383368+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"14.14","version":"115'6","reported_seq":19,"reported_epoch":116,"state":"active+clean","last_fresh":"2023-08-09T13:52:01.223231+0000","last_change":"2023-08-09T13:51:59.181967+0000","last_active":"2023-08-09T13:52:01.223231+0000","last_peered":"2023-08-09T13:52:01.223231+0000","last_clean":"2023-08-09T13:52:01.223231+0000","last_became_active":"2023-08-09T13:51:59.180664+0000","last_became_peered":"2023-08-09T13:51:59.180664+0000","last_unstale":"2023-08-09T13:52:01.223231+0000","last_undegraded":"2023-08-09T13:52:01.223231+0000","last_fullsized":"2023-08-09T13:52:01.223231+0000","mapping_epoch":113,"log_start":"0'0","ondisk_log_start":"0'0","created":113,"last_epoch_clean":114,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:51:58.125661+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:51:58.125661+0000","last_clean_scrub_stamp":"2023-08-09T13:51:58.125661+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-11T00:16:05.270843+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"15.12","version":"0'0","reported_seq":11,"reported_epoch":116,"state":"active+clean","last_fresh":"2023-08-09T13:52:01.221294+0000","last_change":"2023-08-09T13:52:00.209000+0000","last_active":"2023-08-09T13:52:01.221294+0000","last_peered":"2023-08-09T13:52:01.221294+0000","last_clean":"2023-08-09T13:52:01.221294+0000","last_became_active":"2023-08-09T13:52:00.208722+0000","last_became_peered":"2023-08-09T13:52:00.208722+0000","last_unstale":"2023-08-09T13:52:01.221294+0000","last_undegraded":"2023-08-09T13:52:01.221294+0000","last_fullsized":"2023-08-09T13:52:01.221294+0000","mapping_epoch":114,"log_start":"0'0","ondisk_log_start":"0'0","created":114,"last_epoch_clean":115,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:51:59.130555+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:51:59.130555+0000","last_clean_scrub_stamp":"2023-08-09T13:51:59.130555+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T21:54:42.771899+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"14.13","version":"115'2","reported_seq":15,"reported_epoch":116,"state":"active+clean","last_fresh":"2023-08-09T13:52:01.419233+0000","last_change":"2023-08-09T13:51:59.181856+0000","last_active":"2023-08-09T13:52:01.419233+0000","last_peered":"2023-08-09T13:52:01.419233+0000","last_clean":"2023-08-09T13:52:01.419233+0000","last_became_active":"2023-08-09T13:51:59.181568+0000","last_became_peered":"2023-08-09T13:51:59.181568+0000","last_unstale":"2023-08-09T13:52:01.419233+0000","last_undegraded":"2023-08-09T13:52:01.419233+0000","last_fullsized":"2023-08-09T13:52:01.419233+0000","mapping_epoch":113,"log_start":"0'0","ondisk_log_start":"0'0","created":113,"last_epoch_clean":114,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:51:58.125661+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:51:58.125661+0000","last_clean_scrub_stamp":"2023-08-09T13:51:58.125661+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T16:40:16.690907+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.13","version":"0'0","reported_seq":11,"reported_epoch":116,"state":"active+clean","last_fresh":"2023-08-09T13:52:01.219693+0000","last_change":"2023-08-09T13:52:00.233817+0000","last_active":"2023-08-09T13:52:01.219693+0000","last_peered":"2023-08-09T13:52:01.219693+0000","last_clean":"2023-08-09T13:52:01.219693+0000","last_became_active":"2023-08-09T13:52:00.233557+0000","last_became_peered":"2023-08-09T13:52:00.233557+0000","last_unstale":"2023-08-09T13:52:01.219693+0000","last_undegraded":"2023-08-09T13:52:01.219693+0000","last_fullsized":"2023-08-09T13:52:01.219693+0000","mapping_epoch":114,"log_start":"0'0","ondisk_log_start":"0'0","created":114,"last_epoch_clean":115,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:51:59.130555+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:51:59.130555+0000","last_clean_scrub_stamp":"2023-08-09T13:51:59.130555+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T16:20:41.956734+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.12","version":"0'0","reported_seq":13,"reported_epoch":116,"state":"active+clean","last_fresh":"2023-08-09T13:52:01.419147+0000","last_change":"2023-08-09T13:51:59.178710+0000","last_active":"2023-08-09T13:52:01.419147+0000","last_peered":"2023-08-09T13:52:01.419147+0000","last_clean":"2023-08-09T13:52:01.419147+0000","last_became_active":"2023-08-09T13:51:59.178402+0000","last_became_peered":"2023-08-09T13:51:59.178402+0000","last_unstale":"2023-08-09T13:52:01.419147+0000","last_undegraded":"2023-08-09T13:52:01.419147+0000","last_fullsized":"2023-08-09T13:52:01.419147+0000","mapping_epoch":113,"log_start":"0'0","ondisk_log_start":"0'0","created":113,"last_epoch_clean":114,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:51:58.125661+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:51:58.125661+0000","last_clean_scrub_stamp":"2023-08-09T13:51:58.125661+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T20:03:43.329471+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.10","version":"0'0","reported_seq":11,"reported_epoch":116,"state":"active+clean","last_fresh":"2023-08-09T13:52:01.221493+0000","last_change":"2023-08-09T13:52:00.223810+0000","last_active":"2023-08-09T13:52:01.221493+0000","last_peered":"2023-08-09T13:52:01.221493+0000","last_clean":"2023-08-09T13:52:01.221493+0000","last_became_active":"2023-08-09T13:52:00.223492+0000","last_became_peered":"2023-08-09T13:52:00.223492+0000","last_unstale":"2023-08-09T13:52:01.221493+0000","last_undegraded":"2023-08-09T13:52:01.221493+0000","last_fullsized":"2023-08-09T13:52:01.221493+0000","mapping_epoch":114,"log_start":"0'0","ondisk_log_start":"0'0","created":114,"last_epoch_clean":115,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:51:59.130555+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:51:59.130555+0000","last_clean_scrub_stamp":"2023-08-09T13:51:59.130555+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T15:40:03.746964+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.11","version":"0'0","reported_seq":13,"reported_epoch":116,"state":"active+clean","last_fresh":"2023-08-09T13:52:01.221690+0000","last_change":"2023-08-09T13:51:59.181121+0000","last_active":"2023-08-09T13:52:01.221690+0000","last_peered":"2023-08-09T13:52:01.221690+0000","last_clean":"2023-08-09T13:52:01.221690+0000","last_became_active":"2023-08-09T13:51:59.180844+0000","last_became_peered":"2023-08-09T13:51:59.180844+0000","last_unstale":"2023-08-09T13:52:01.221690+0000","last_undegraded":"2023-08-09T13:52:01.221690+0000","last_fullsized":"2023-08-09T13:52:01.221690+0000","mapping_epoch":113,"log_start":"0'0","ondisk_log_start":"0'0","created":113,"last_epoch_clean":114,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:51:58.125661+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:51:58.125661+0000","last_clean_scrub_stamp":"2023-08-09T13:51:58.125661+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T16:43:18.755379+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"15.11","version":"0'0","reported_seq":11,"reported_epoch":116,"state":"active+clean","last_fresh":"2023-08-09T13:52:01.418790+0000","last_change":"2023-08-09T13:52:00.216007+0000","last_active":"2023-08-09T13:52:01.418790+0000","last_peered":"2023-08-09T13:52:01.418790+0000","last_clean":"2023-08-09T13:52:01.418790+0000","last_became_active":"2023-08-09T13:52:00.215714+0000","last_became_peered":"2023-08-09T13:52:00.215714+0000","last_unstale":"2023-08-09T13:52:01.418790+0000","last_undegraded":"2023-08-09T13:52:01.418790+0000","last_fullsized":"2023-08-09T13:52:01.418790+0000","mapping_epoch":114,"log_start":"0'0","ondisk_log_start":"0'0","created":114,"last_epoch_clean":115,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:51:59.130555+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:51:59.130555+0000","last_clean_scrub_stamp":"2023-08-09T13:51:59.130555+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-11T01:27:18.440376+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"14.10","version":"115'8","reported_seq":21,"reported_epoch":116,"state":"active+clean","last_fresh":"2023-08-09T13:52:01.219639+0000","last_change":"2023-08-09T13:51:59.174790+0000","last_active":"2023-08-09T13:52:01.219639+0000","last_peered":"2023-08-09T13:52:01.219639+0000","last_clean":"2023-08-09T13:52:01.219639+0000","last_became_active":"2023-08-09T13:51:59.174453+0000","last_became_peered":"2023-08-09T13:51:59.174453+0000","last_unstale":"2023-08-09T13:52:01.219639+0000","last_undegraded":"2023-08-09T13:52:01.219639+0000","last_fullsized":"2023-08-09T13:52:01.219639+0000","mapping_epoch":113,"log_start":"0'0","ondisk_log_start":"0'0","created":113,"last_epoch_clean":114,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:51:58.125661+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:51:58.125661+0000","last_clean_scrub_stamp":"2023-08-09T13:51:58.125661+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T15:05:53.316835+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"15.6","version":"0'0","reported_seq":11,"reported_epoch":116,"state":"active+clean","last_fresh":"2023-08-09T13:52:02.227981+0000","last_change":"2023-08-09T13:52:00.217311+0000","last_active":"2023-08-09T13:52:02.227981+0000","last_peered":"2023-08-09T13:52:02.227981+0000","last_clean":"2023-08-09T13:52:02.227981+0000","last_became_active":"2023-08-09T13:52:00.216850+0000","last_became_peered":"2023-08-09T13:52:00.216850+0000","last_unstale":"2023-08-09T13:52:02.227981+0000","last_undegraded":"2023-08-09T13:52:02.227981+0000","last_fullsized":"2023-08-09T13:52:02.227981+0000","mapping_epoch":114,"log_start":"0'0","ondisk_log_start":"0'0","created":114,"last_epoch_clean":115,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:51:59.130555+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:51:59.130555+0000","last_clean_scrub_stamp":"2023-08-09T13:51:59.130555+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T20:47:55.200313+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"14.7","version":"115'2","reported_seq":15,"reported_epoch":116,"state":"active+clean","last_fresh":"2023-08-09T13:52:02.231424+0000","last_change":"2023-08-09T13:51:59.182730+0000","last_active":"2023-08-09T13:52:02.231424+0000","last_peered":"2023-08-09T13:52:02.231424+0000","last_clean":"2023-08-09T13:52:02.231424+0000","last_became_active":"2023-08-09T13:51:59.182469+0000","last_became_peered":"2023-08-09T13:51:59.182469+0000","last_unstale":"2023-08-09T13:52:02.231424+0000","last_undegraded":"2023-08-09T13:52:02.231424+0000","last_fullsized":"2023-08-09T13:52:02.231424+0000","mapping_epoch":113,"log_start":"0'0","ondisk_log_start":"0'0","created":113,"last_epoch_clean":114,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:51:58.125661+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:51:58.125661+0000","last_clean_scrub_stamp":"2023-08-09T13:51:58.125661+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T19:15:58.608074+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"15.7","version":"0'0","reported_seq":11,"reported_epoch":116,"state":"active+clean","last_fresh":"2023-08-09T13:52:01.220232+0000","last_change":"2023-08-09T13:52:00.233211+0000","last_active":"2023-08-09T13:52:01.220232+0000","last_peered":"2023-08-09T13:52:01.220232+0000","last_clean":"2023-08-09T13:52:01.220232+0000","last_became_active":"2023-08-09T13:52:00.232860+0000","last_became_peered":"2023-08-09T13:52:00.232860+0000","last_unstale":"2023-08-09T13:52:01.220232+0000","last_undegraded":"2023-08-09T13:52:01.220232+0000","last_fullsized":"2023-08-09T13:52:01.220232+0000","mapping_epoch":114,"log_start":"0'0","ondisk_log_start":"0'0","created":114,"last_epoch_clean":115,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:51:59.130555+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:51:59.130555+0000","last_clean_scrub_stamp":"2023-08-09T13:51:59.130555+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T13:55:34.217882+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.6","version":"116'1","reported_seq":14,"reported_epoch":116,"state":"active+clean","last_fresh":"2023-08-09T13:52:02.228074+0000","last_change":"2023-08-09T13:51:59.168280+0000","last_active":"2023-08-09T13:52:02.228074+0000","last_peered":"2023-08-09T13:52:02.228074+0000","last_clean":"2023-08-09T13:52:02.228074+0000","last_became_active":"2023-08-09T13:51:59.167980+0000","last_became_peered":"2023-08-09T13:51:59.167980+0000","last_unstale":"2023-08-09T13:52:02.228074+0000","last_undegraded":"2023-08-09T13:52:02.228074+0000","last_fullsized":"2023-08-09T13:52:02.228074+0000","mapping_epoch":113,"log_start":"0'0","ondisk_log_start":"0'0","created":113,"last_epoch_clean":114,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:51:58.125661+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:51:58.125661+0000","last_clean_scrub_stamp":"2023-08-09T13:51:58.125661+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T16:44:35.804569+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.4","version":"0'0","reported_seq":11,"reported_epoch":116,"state":"active+clean","last_fresh":"2023-08-09T13:52:01.219941+0000","last_change":"2023-08-09T13:52:00.228211+0000","last_active":"2023-08-09T13:52:01.219941+0000","last_peered":"2023-08-09T13:52:01.219941+0000","last_clean":"2023-08-09T13:52:01.219941+0000","last_became_active":"2023-08-09T13:52:00.227911+0000","last_became_peered":"2023-08-09T13:52:00.227911+0000","last_unstale":"2023-08-09T13:52:01.219941+0000","last_undegraded":"2023-08-09T13:52:01.219941+0000","last_fullsized":"2023-08-09T13:52:01.219941+0000","mapping_epoch":114,"log_start":"0'0","ondisk_log_start":"0'0","created":114,"last_epoch_clean":115,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:51:59.130555+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:51:59.130555+0000","last_clean_scrub_stamp":"2023-08-09T13:51:59.130555+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T19:34:08.224464+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.5","version":"0'0","reported_seq":13,"reported_epoch":116,"state":"active+clean","last_fresh":"2023-08-09T13:52:02.227620+0000","last_change":"2023-08-09T13:51:59.179649+0000","last_active":"2023-08-09T13:52:02.227620+0000","last_peered":"2023-08-09T13:52:02.227620+0000","last_clean":"2023-08-09T13:52:02.227620+0000","last_became_active":"2023-08-09T13:51:59.179348+0000","last_became_peered":"2023-08-09T13:51:59.179348+0000","last_unstale":"2023-08-09T13:52:02.227620+0000","last_undegraded":"2023-08-09T13:52:02.227620+0000","last_fullsized":"2023-08-09T13:52:02.227620+0000","mapping_epoch":113,"log_start":"0'0","ondisk_log_start":"0'0","created":113,"last_epoch_clean":114,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:51:58.125661+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:51:58.125661+0000","last_clean_scrub_stamp":"2023-08-09T13:51:58.125661+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T20:57:33.648101+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.5","version":"0'0","reported_seq":11,"reported_epoch":116,"state":"active+clean","last_fresh":"2023-08-09T13:52:02.230093+0000","last_change":"2023-08-09T13:52:00.211669+0000","last_active":"2023-08-09T13:52:02.230093+0000","last_peered":"2023-08-09T13:52:02.230093+0000","last_clean":"2023-08-09T13:52:02.230093+0000","last_became_active":"2023-08-09T13:52:00.211314+0000","last_became_peered":"2023-08-09T13:52:00.211314+0000","last_unstale":"2023-08-09T13:52:02.230093+0000","last_undegraded":"2023-08-09T13:52:02.230093+0000","last_fullsized":"2023-08-09T13:52:02.230093+0000","mapping_epoch":114,"log_start":"0'0","ondisk_log_start":"0'0","created":114,"last_epoch_clean":115,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:51:59.130555+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:51:59.130555+0000","last_clean_scrub_stamp":"2023-08-09T13:51:59.130555+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T19:08:15.523370+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"14.4","version":"0'0","reported_seq":13,"reported_epoch":116,"state":"active+clean","last_fresh":"2023-08-09T13:52:02.229733+0000","last_change":"2023-08-09T13:51:59.174311+0000","last_active":"2023-08-09T13:52:02.229733+0000","last_peered":"2023-08-09T13:52:02.229733+0000","last_clean":"2023-08-09T13:52:02.229733+0000","last_became_active":"2023-08-09T13:51:59.173990+0000","last_became_peered":"2023-08-09T13:51:59.173990+0000","last_unstale":"2023-08-09T13:52:02.229733+0000","last_undegraded":"2023-08-09T13:52:02.229733+0000","last_fullsized":"2023-08-09T13:52:02.229733+0000","mapping_epoch":113,"log_start":"0'0","ondisk_log_start":"0'0","created":113,"last_epoch_clean":114,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:51:58.125661+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:51:58.125661+0000","last_clean_scrub_stamp":"2023-08-09T13:51:58.125661+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T21:53:48.212015+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.2","version":"0'0","reported_seq":11,"reported_epoch":116,"state":"active+clean","last_fresh":"2023-08-09T13:52:01.419005+0000","last_change":"2023-08-09T13:52:00.214396+0000","last_active":"2023-08-09T13:52:01.419005+0000","last_peered":"2023-08-09T13:52:01.419005+0000","last_clean":"2023-08-09T13:52:01.419005+0000","last_became_active":"2023-08-09T13:52:00.213395+0000","last_became_peered":"2023-08-09T13:52:00.213395+0000","last_unstale":"2023-08-09T13:52:01.419005+0000","last_undegraded":"2023-08-09T13:52:01.419005+0000","last_fullsized":"2023-08-09T13:52:01.419005+0000","mapping_epoch":114,"log_start":"0'0","ondisk_log_start":"0'0","created":114,"last_epoch_clean":115,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:51:59.130555+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:51:59.130555+0000","last_clean_scrub_stamp":"2023-08-09T13:51:59.130555+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T14:47:29.943812+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"14.3","version":"0'0","reported_seq":13,"reported_epoch":116,"state":"active+clean","last_fresh":"2023-08-09T13:52:02.227823+0000","last_change":"2023-08-09T13:51:59.167523+0000","last_active":"2023-08-09T13:52:02.227823+0000","last_peered":"2023-08-09T13:52:02.227823+0000","last_clean":"2023-08-09T13:52:02.227823+0000","last_became_active":"2023-08-09T13:51:59.167059+0000","last_became_peered":"2023-08-09T13:51:59.167059+0000","last_unstale":"2023-08-09T13:52:02.227823+0000","last_undegraded":"2023-08-09T13:52:02.227823+0000","last_fullsized":"2023-08-09T13:52:02.227823+0000","mapping_epoch":113,"log_start":"0'0","ondisk_log_start":"0'0","created":113,"last_epoch_clean":114,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:51:58.125661+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:51:58.125661+0000","last_clean_scrub_stamp":"2023-08-09T13:51:58.125661+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T20:28:55.377939+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.3","version":"0'0","reported_seq":11,"reported_epoch":116,"state":"active+clean","last_fresh":"2023-08-09T13:52:01.222209+0000","last_change":"2023-08-09T13:52:00.220968+0000","last_active":"2023-08-09T13:52:01.222209+0000","last_peered":"2023-08-09T13:52:01.222209+0000","last_clean":"2023-08-09T13:52:01.222209+0000","last_became_active":"2023-08-09T13:52:00.220710+0000","last_became_peered":"2023-08-09T13:52:00.220710+0000","last_unstale":"2023-08-09T13:52:01.222209+0000","last_undegraded":"2023-08-09T13:52:01.222209+0000","last_fullsized":"2023-08-09T13:52:01.222209+0000","mapping_epoch":114,"log_start":"0'0","ondisk_log_start":"0'0","created":114,"last_epoch_clean":115,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:51:59.130555+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:51:59.130555+0000","last_clean_scrub_stamp":"2023-08-09T13:51:59.130555+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T19:42:07.385997+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.2","version":"0'0","reported_seq":13,"reported_epoch":116,"state":"active+clean","last_fresh":"2023-08-09T13:52:02.228985+0000","last_change":"2023-08-09T13:51:59.176919+0000","last_active":"2023-08-09T13:52:02.228985+0000","last_peered":"2023-08-09T13:52:02.228985+0000","last_clean":"2023-08-09T13:52:02.228985+0000","last_became_active":"2023-08-09T13:51:59.176503+0000","last_became_peered":"2023-08-09T13:51:59.176503+0000","last_unstale":"2023-08-09T13:52:02.228985+0000","last_undegraded":"2023-08-09T13:52:02.228985+0000","last_fullsized":"2023-08-09T13:52:02.228985+0000","mapping_epoch":113,"log_start":"0'0","ondisk_log_start":"0'0","created":113,"last_epoch_clean":114,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:51:58.125661+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:51:58.125661+0000","last_clean_scrub_stamp":"2023-08-09T13:51:58.125661+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T21:36:20.093982+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.0","version":"0'0","reported_seq":11,"reported_epoch":116,"state":"active+clean","last_fresh":"2023-08-09T13:52:01.418787+0000","last_change":"2023-08-09T13:52:00.216790+0000","last_active":"2023-08-09T13:52:01.418787+0000","last_peered":"2023-08-09T13:52:01.418787+0000","last_clean":"2023-08-09T13:52:01.418787+0000","last_became_active":"2023-08-09T13:52:00.216456+0000","last_became_peered":"2023-08-09T13:52:00.216456+0000","last_unstale":"2023-08-09T13:52:01.418787+0000","last_undegraded":"2023-08-09T13:52:01.418787+0000","last_fullsized":"2023-08-09T13:52:01.418787+0000","mapping_epoch":114,"log_start":"0'0","ondisk_log_start":"0'0","created":114,"last_epoch_clean":115,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:51:59.130555+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:51:59.130555+0000","last_clean_scrub_stamp":"2023-08-09T13:51:59.130555+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T15:56:02.893523+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"14.1","version":"0'0","reported_seq":13,"reported_epoch":116,"state":"active+clean","last_fresh":"2023-08-09T13:52:01.221369+0000","last_change":"2023-08-09T13:51:59.171070+0000","last_active":"2023-08-09T13:52:01.221369+0000","last_peered":"2023-08-09T13:52:01.221369+0000","last_clean":"2023-08-09T13:52:01.221369+0000","last_became_active":"2023-08-09T13:51:59.170788+0000","last_became_peered":"2023-08-09T13:51:59.170788+0000","last_unstale":"2023-08-09T13:52:01.221369+0000","last_undegraded":"2023-08-09T13:52:01.221369+0000","last_fullsized":"2023-08-09T13:52:01.221369+0000","mapping_epoch":113,"log_start":"0'0","ondisk_log_start":"0'0","created":113,"last_epoch_clean":114,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:51:58.125661+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:51:58.125661+0000","last_clean_scrub_stamp":"2023-08-09T13:51:58.125661+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T20:53:08.597878+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"15.1","version":"0'0","reported_seq":11,"reported_epoch":116,"state":"active+clean","last_fresh":"2023-08-09T13:52:02.229115+0000","last_change":"2023-08-09T13:52:00.222037+0000","last_active":"2023-08-09T13:52:02.229115+0000","last_peered":"2023-08-09T13:52:02.229115+0000","last_clean":"2023-08-09T13:52:02.229115+0000","last_became_active":"2023-08-09T13:52:00.217448+0000","last_became_peered":"2023-08-09T13:52:00.217448+0000","last_unstale":"2023-08-09T13:52:02.229115+0000","last_undegraded":"2023-08-09T13:52:02.229115+0000","last_fullsized":"2023-08-09T13:52:02.229115+0000","mapping_epoch":114,"log_start":"0'0","ondisk_log_start":"0'0","created":114,"last_epoch_clean":115,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:51:59.130555+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:51:59.130555+0000","last_clean_scrub_stamp":"2023-08-09T13:51:59.130555+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T21:14:10.012130+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"14.0","version":"0'0","reported_seq":13,"reported_epoch":116,"state":"active+clean","last_fresh":"2023-08-09T13:52:01.418966+0000","last_change":"2023-08-09T13:51:59.173223+0000","last_active":"2023-08-09T13:52:01.418966+0000","last_peered":"2023-08-09T13:52:01.418966+0000","last_clean":"2023-08-09T13:52:01.418966+0000","last_became_active":"2023-08-09T13:51:59.172948+0000","last_became_peered":"2023-08-09T13:51:59.172948+0000","last_unstale":"2023-08-09T13:52:01.418966+0000","last_undegraded":"2023-08-09T13:52:01.418966+0000","last_fullsized":"2023-08-09T13:52:01.418966+0000","mapping_epoch":113,"log_start":"0'0","ondisk_log_start":"0'0","created":113,"last_epoch_clean":114,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:51:58.125661+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:51:58.125661+0000","last_clean_scrub_stamp":"2023-08-09T13:51:58.125661+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T23:28:53.795333+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.e","version":"0'0","reported_seq":11,"reported_epoch":116,"state":"active+clean","last_fresh":"2023-08-09T13:52:01.419454+0000","last_change":"2023-08-09T13:52:00.210798+0000","last_active":"2023-08-09T13:52:01.419454+0000","last_peered":"2023-08-09T13:52:01.419454+0000","last_clean":"2023-08-09T13:52:01.419454+0000","last_became_active":"2023-08-09T13:52:00.210358+0000","last_became_peered":"2023-08-09T13:52:00.210358+0000","last_unstale":"2023-08-09T13:52:01.419454+0000","last_undegraded":"2023-08-09T13:52:01.419454+0000","last_fullsized":"2023-08-09T13:52:01.419454+0000","mapping_epoch":114,"log_start":"0'0","ondisk_log_start":"0'0","created":114,"last_epoch_clean":115,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:51:59.130555+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:51:59.130555+0000","last_clean_scrub_stamp":"2023-08-09T13:51:59.130555+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T16:59:58.856242+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"14.f","version":"115'1","reported_seq":14,"reported_epoch":116,"state":"active+clean","last_fresh":"2023-08-09T13:52:02.227196+0000","last_change":"2023-08-09T13:51:59.174685+0000","last_active":"2023-08-09T13:52:02.227196+0000","last_peered":"2023-08-09T13:52:02.227196+0000","last_clean":"2023-08-09T13:52:02.227196+0000","last_became_active":"2023-08-09T13:51:59.174370+0000","last_became_peered":"2023-08-09T13:51:59.174370+0000","last_unstale":"2023-08-09T13:52:02.227196+0000","last_undegraded":"2023-08-09T13:52:02.227196+0000","last_fullsized":"2023-08-09T13:52:02.227196+0000","mapping_epoch":113,"log_start":"0'0","ondisk_log_start":"0'0","created":113,"last_epoch_clean":114,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:51:58.125661+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:51:58.125661+0000","last_clean_scrub_stamp":"2023-08-09T13:51:58.125661+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T20:53:08.597878+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":193,"reported_epoch":116,"state":"active+clean","last_fresh":"2023-08-09T13:52:02.228680+0000","last_change":"2023-08-09T13:33:56.401406+0000","last_active":"2023-08-09T13:52:02.228680+0000","last_peered":"2023-08-09T13:52:02.228680+0000","last_clean":"2023-08-09T13:52:02.228680+0000","last_became_active":"2023-08-09T13:33:56.400487+0000","last_became_peered":"2023-08-09T13:33:56.400487+0000","last_unstale":"2023-08-09T13:52:02.228680+0000","last_undegraded":"2023-08-09T13:52:02.228680+0000","last_fullsized":"2023-08-09T13:52:02.228680+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:33:54.845869+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:33:54.845869+0000","last_clean_scrub_stamp":"2023-08-09T13:33:54.845869+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T15:15:21.526034+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"15.f","version":"0'0","reported_seq":11,"reported_epoch":116,"state":"active+clean","last_fresh":"2023-08-09T13:52:02.230093+0000","last_change":"2023-08-09T13:52:00.217897+0000","last_active":"2023-08-09T13:52:02.230093+0000","last_peered":"2023-08-09T13:52:02.230093+0000","last_clean":"2023-08-09T13:52:02.230093+0000","last_became_active":"2023-08-09T13:52:00.217627+0000","last_became_peered":"2023-08-09T13:52:00.217627+0000","last_unstale":"2023-08-09T13:52:02.230093+0000","last_undegraded":"2023-08-09T13:52:02.230093+0000","last_fullsized":"2023-08-09T13:52:02.230093+0000","mapping_epoch":114,"log_start":"0'0","ondisk_log_start":"0'0","created":114,"last_epoch_clean":115,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:51:59.130555+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:51:59.130555+0000","last_clean_scrub_stamp":"2023-08-09T13:51:59.130555+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T16:54:25.108485+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"14.e","version":"115'2","reported_seq":15,"reported_epoch":116,"state":"active+clean","last_fresh":"2023-08-09T13:52:01.223136+0000","last_change":"2023-08-09T13:51:59.180623+0000","last_active":"2023-08-09T13:52:01.223136+0000","last_peered":"2023-08-09T13:52:01.223136+0000","last_clean":"2023-08-09T13:52:01.223136+0000","last_became_active":"2023-08-09T13:51:59.178785+0000","last_became_peered":"2023-08-09T13:51:59.178785+0000","last_unstale":"2023-08-09T13:52:01.223136+0000","last_undegraded":"2023-08-09T13:52:01.223136+0000","last_fullsized":"2023-08-09T13:52:01.223136+0000","mapping_epoch":113,"log_start":"0'0","ondisk_log_start":"0'0","created":113,"last_epoch_clean":114,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:51:58.125661+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:51:58.125661+0000","last_clean_scrub_stamp":"2023-08-09T13:51:58.125661+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T19:33:51.888915+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":197,"reported_epoch":116,"state":"active+clean","last_fresh":"2023-08-09T13:52:02.229066+0000","last_change":"2023-08-09T13:33:56.401123+0000","last_active":"2023-08-09T13:52:02.229066+0000","last_peered":"2023-08-09T13:52:02.229066+0000","last_clean":"2023-08-09T13:52:02.229066+0000","last_became_active":"2023-08-09T13:33:56.399933+0000","last_became_peered":"2023-08-09T13:33:56.399933+0000","last_unstale":"2023-08-09T13:52:02.229066+0000","last_undegraded":"2023-08-09T13:52:02.229066+0000","last_fullsized":"2023-08-09T13:52:02.229066+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:33:54.845869+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:33:54.845869+0000","last_clean_scrub_stamp":"2023-08-09T13:33:54.845869+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T16:13:52.660079+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"15.c","version":"0'0","reported_seq":11,"reported_epoch":116,"state":"active+clean","last_fresh":"2023-08-09T13:52:02.227915+0000","last_change":"2023-08-09T13:52:00.220774+0000","last_active":"2023-08-09T13:52:02.227915+0000","last_peered":"2023-08-09T13:52:02.227915+0000","last_clean":"2023-08-09T13:52:02.227915+0000","last_became_active":"2023-08-09T13:52:00.220505+0000","last_became_peered":"2023-08-09T13:52:00.220505+0000","last_unstale":"2023-08-09T13:52:02.227915+0000","last_undegraded":"2023-08-09T13:52:02.227915+0000","last_fullsized":"2023-08-09T13:52:02.227915+0000","mapping_epoch":114,"log_start":"0'0","ondisk_log_start":"0'0","created":114,"last_epoch_clean":115,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:51:59.130555+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:51:59.130555+0000","last_clean_scrub_stamp":"2023-08-09T13:51:59.130555+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T22:48:53.283499+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"14.d","version":"115'1","reported_seq":14,"reported_epoch":116,"state":"active+clean","last_fresh":"2023-08-09T13:52:02.229861+0000","last_change":"2023-08-09T13:51:59.174909+0000","last_active":"2023-08-09T13:52:02.229861+0000","last_peered":"2023-08-09T13:52:02.229861+0000","last_clean":"2023-08-09T13:52:02.229861+0000","last_became_active":"2023-08-09T13:51:59.174527+0000","last_became_peered":"2023-08-09T13:51:59.174527+0000","last_unstale":"2023-08-09T13:52:02.229861+0000","last_undegraded":"2023-08-09T13:52:02.229861+0000","last_fullsized":"2023-08-09T13:52:02.229861+0000","mapping_epoch":113,"log_start":"0'0","ondisk_log_start":"0'0","created":113,"last_epoch_clean":114,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:51:58.125661+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:51:58.125661+0000","last_clean_scrub_stamp":"2023-08-09T13:51:58.125661+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T14:11:59.228722+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"1.2","version":"0'0","reported_seq":193,"reported_epoch":116,"state":"active+clean","last_fresh":"2023-08-09T13:52:02.228361+0000","last_change":"2023-08-09T13:33:56.402794+0000","last_active":"2023-08-09T13:52:02.228361+0000","last_peered":"2023-08-09T13:52:02.228361+0000","last_clean":"2023-08-09T13:52:02.228361+0000","last_became_active":"2023-08-09T13:33:56.402038+0000","last_became_peered":"2023-08-09T13:33:56.402038+0000","last_unstale":"2023-08-09T13:52:02.228361+0000","last_undegraded":"2023-08-09T13:52:02.228361+0000","last_fullsized":"2023-08-09T13:52:02.228361+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:33:54.845869+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:33:54.845869+0000","last_clean_scrub_stamp":"2023-08-09T13:33:54.845869+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T13:49:39.997653+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"15.d","version":"0'0","reported_seq":11,"reported_epoch":116,"state":"active+clean","last_fresh":"2023-08-09T13:52:02.227529+0000","last_change":"2023-08-09T13:52:00.219473+0000","last_active":"2023-08-09T13:52:02.227529+0000","last_peered":"2023-08-09T13:52:02.227529+0000","last_clean":"2023-08-09T13:52:02.227529+0000","last_became_active":"2023-08-09T13:52:00.218836+0000","last_became_peered":"2023-08-09T13:52:00.218836+0000","last_unstale":"2023-08-09T13:52:02.227529+0000","last_undegraded":"2023-08-09T13:52:02.227529+0000","last_fullsized":"2023-08-09T13:52:02.227529+0000","mapping_epoch":114,"log_start":"0'0","ondisk_log_start":"0'0","created":114,"last_epoch_clean":115,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:51:59.130555+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:51:59.130555+0000","last_clean_scrub_stamp":"2023-08-09T13:51:59.130555+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T23:42:47.388329+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"14.c","version":"0'0","reported_seq":13,"reported_epoch":116,"state":"active+clean","last_fresh":"2023-08-09T13:52:02.227506+0000","last_change":"2023-08-09T13:51:59.183702+0000","last_active":"2023-08-09T13:52:02.227506+0000","last_peered":"2023-08-09T13:52:02.227506+0000","last_clean":"2023-08-09T13:52:02.227506+0000","last_became_active":"2023-08-09T13:51:59.183387+0000","last_became_peered":"2023-08-09T13:51:59.183387+0000","last_unstale":"2023-08-09T13:52:02.227506+0000","last_undegraded":"2023-08-09T13:52:02.227506+0000","last_fullsized":"2023-08-09T13:52:02.227506+0000","mapping_epoch":113,"log_start":"0'0","ondisk_log_start":"0'0","created":113,"last_epoch_clean":114,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:51:58.125661+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:51:58.125661+0000","last_clean_scrub_stamp":"2023-08-09T13:51:58.125661+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T16:41:06.536543+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.3","version":"0'0","reported_seq":193,"reported_epoch":116,"state":"active+clean","last_fresh":"2023-08-09T13:52:01.219784+0000","last_change":"2023-08-09T13:33:55.886749+0000","last_active":"2023-08-09T13:52:01.219784+0000","last_peered":"2023-08-09T13:52:01.219784+0000","last_clean":"2023-08-09T13:52:01.219784+0000","last_became_active":"2023-08-09T13:33:55.886116+0000","last_became_peered":"2023-08-09T13:33:55.886116+0000","last_unstale":"2023-08-09T13:52:01.219784+0000","last_undegraded":"2023-08-09T13:52:01.219784+0000","last_fullsized":"2023-08-09T13:52:01.219784+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:33:54.845869+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:33:54.845869+0000","last_clean_scrub_stamp":"2023-08-09T13:33:54.845869+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T23:07:12.228212+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"105'186","reported_seq":587,"reported_epoch":116,"state":"active+clean","last_fresh":"2023-08-09T13:52:02.227119+0000","last_change":"2023-08-09T13:33:55.886345+0000","last_active":"2023-08-09T13:52:02.227119+0000","last_peered":"2023-08-09T13:52:02.227119+0000","last_clean":"2023-08-09T13:52:02.227119+0000","last_became_active":"2023-08-09T13:33:55.885781+0000","last_became_peered":"2023-08-09T13:33:55.885781+0000","last_unstale":"2023-08-09T13:52:02.227119+0000","last_undegraded":"2023-08-09T13:52:02.227119+0000","last_fullsized":"2023-08-09T13:52:02.227119+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:33:54.845869+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:33:54.845869+0000","last_clean_scrub_stamp":"2023-08-09T13:33:54.845869+0000","objects_scrubbed":0,"log_size":186,"log_dups_size":0,"ondisk_log_size":186,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T19:31:45.934609+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":715,"num_read_kb":1418,"num_write":283,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"15.a","version":"0'0","reported_seq":11,"reported_epoch":116,"state":"active+clean","last_fresh":"2023-08-09T13:52:01.419121+0000","last_change":"2023-08-09T13:52:00.210575+0000","last_active":"2023-08-09T13:52:01.419121+0000","last_peered":"2023-08-09T13:52:01.419121+0000","last_clean":"2023-08-09T13:52:01.419121+0000","last_became_active":"2023-08-09T13:52:00.210182+0000","last_became_peered":"2023-08-09T13:52:00.210182+0000","last_unstale":"2023-08-09T13:52:01.419121+0000","last_undegraded":"2023-08-09T13:52:01.419121+0000","last_fullsized":"2023-08-09T13:52:01.419121+0000","mapping_epoch":114,"log_start":"0'0","ondisk_log_start":"0'0","created":114,"last_epoch_clean":115,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:51:59.130555+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:51:59.130555+0000","last_clean_scrub_stamp":"2023-08-09T13:51:59.130555+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T17:12:07.979144+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"14.b","version":"115'2","reported_seq":15,"reported_epoch":116,"state":"active+clean","last_fresh":"2023-08-09T13:52:02.227580+0000","last_change":"2023-08-09T13:51:59.168603+0000","last_active":"2023-08-09T13:52:02.227580+0000","last_peered":"2023-08-09T13:52:02.227580+0000","last_clean":"2023-08-09T13:52:02.227580+0000","last_became_active":"2023-08-09T13:51:59.168273+0000","last_became_peered":"2023-08-09T13:51:59.168273+0000","last_unstale":"2023-08-09T13:52:02.227580+0000","last_undegraded":"2023-08-09T13:52:02.227580+0000","last_fullsized":"2023-08-09T13:52:02.227580+0000","mapping_epoch":113,"log_start":"0'0","ondisk_log_start":"0'0","created":113,"last_epoch_clean":114,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:51:58.125661+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:51:58.125661+0000","last_clean_scrub_stamp":"2023-08-09T13:51:58.125661+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T14:50:42.716966+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":203,"reported_epoch":116,"state":"active+clean","last_fresh":"2023-08-09T13:52:01.221680+0000","last_change":"2023-08-09T13:33:55.889565+0000","last_active":"2023-08-09T13:52:01.221680+0000","last_peered":"2023-08-09T13:52:01.221680+0000","last_clean":"2023-08-09T13:52:01.221680+0000","last_became_active":"2023-08-09T13:33:55.889164+0000","last_became_peered":"2023-08-09T13:33:55.889164+0000","last_unstale":"2023-08-09T13:52:01.221680+0000","last_undegraded":"2023-08-09T13:52:01.221680+0000","last_fullsized":"2023-08-09T13:52:01.221680+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:33:54.845869+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:33:54.845869+0000","last_clean_scrub_stamp":"2023-08-09T13:33:54.845869+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T22:32:12.354125+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"15.b","version":"0'0","reported_seq":11,"reported_epoch":116,"state":"active+clean","last_fresh":"2023-08-09T13:52:01.419072+0000","last_change":"2023-08-09T13:52:00.214910+0000","last_active":"2023-08-09T13:52:01.419072+0000","last_peered":"2023-08-09T13:52:01.419072+0000","last_clean":"2023-08-09T13:52:01.419072+0000","last_became_active":"2023-08-09T13:52:00.214685+0000","last_became_peered":"2023-08-09T13:52:00.214685+0000","last_unstale":"2023-08-09T13:52:01.419072+0000","last_undegraded":"2023-08-09T13:52:01.419072+0000","last_fullsized":"2023-08-09T13:52:01.419072+0000","mapping_epoch":114,"log_start":"0'0","ondisk_log_start":"0'0","created":114,"last_epoch_clean":115,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:51:59.130555+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:51:59.130555+0000","last_clean_scrub_stamp":"2023-08-09T13:51:59.130555+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T22:56:47.012795+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"14.a","version":"0'0","reported_seq":13,"reported_epoch":116,"state":"active+clean","last_fresh":"2023-08-09T13:52:02.228593+0000","last_change":"2023-08-09T13:51:59.179151+0000","last_active":"2023-08-09T13:52:02.228593+0000","last_peered":"2023-08-09T13:52:02.228593+0000","last_clean":"2023-08-09T13:52:02.228593+0000","last_became_active":"2023-08-09T13:51:59.178247+0000","last_became_peered":"2023-08-09T13:51:59.178247+0000","last_unstale":"2023-08-09T13:52:02.228593+0000","last_undegraded":"2023-08-09T13:52:02.228593+0000","last_fullsized":"2023-08-09T13:52:02.228593+0000","mapping_epoch":113,"log_start":"0'0","ondisk_log_start":"0'0","created":113,"last_epoch_clean":114,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:51:58.125661+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:51:58.125661+0000","last_clean_scrub_stamp":"2023-08-09T13:51:58.125661+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T15:49:27.949808+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":197,"reported_epoch":116,"state":"active+clean","last_fresh":"2023-08-09T13:52:01.223079+0000","last_change":"2023-08-09T13:33:56.404236+0000","last_active":"2023-08-09T13:52:01.223079+0000","last_peered":"2023-08-09T13:52:01.223079+0000","last_clean":"2023-08-09T13:52:01.223079+0000","last_became_active":"2023-08-09T13:33:56.402485+0000","last_became_peered":"2023-08-09T13:33:56.402485+0000","last_unstale":"2023-08-09T13:52:01.223079+0000","last_undegraded":"2023-08-09T13:52:01.223079+0000","last_fullsized":"2023-08-09T13:52:01.223079+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:33:54.845869+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:33:54.845869+0000","last_clean_scrub_stamp":"2023-08-09T13:33:54.845869+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T16:52:26.468612+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"15.8","version":"0'0","reported_seq":11,"reported_epoch":116,"state":"active+clean","last_fresh":"2023-08-09T13:52:01.418796+0000","last_change":"2023-08-09T13:52:00.210861+0000","last_active":"2023-08-09T13:52:01.418796+0000","last_peered":"2023-08-09T13:52:01.418796+0000","last_clean":"2023-08-09T13:52:01.418796+0000","last_became_active":"2023-08-09T13:52:00.210529+0000","last_became_peered":"2023-08-09T13:52:00.210529+0000","last_unstale":"2023-08-09T13:52:01.418796+0000","last_undegraded":"2023-08-09T13:52:01.418796+0000","last_fullsized":"2023-08-09T13:52:01.418796+0000","mapping_epoch":114,"log_start":"0'0","ondisk_log_start":"0'0","created":114,"last_epoch_clean":115,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:51:59.130555+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:51:59.130555+0000","last_clean_scrub_stamp":"2023-08-09T13:51:59.130555+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T23:45:14.245991+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"14.9","version":"0'0","reported_seq":13,"reported_epoch":116,"state":"active+clean","last_fresh":"2023-08-09T13:52:02.228770+0000","last_change":"2023-08-09T13:51:59.177964+0000","last_active":"2023-08-09T13:52:02.228770+0000","last_peered":"2023-08-09T13:52:02.228770+0000","last_clean":"2023-08-09T13:52:02.228770+0000","last_became_active":"2023-08-09T13:51:59.177186+0000","last_became_peered":"2023-08-09T13:51:59.177186+0000","last_unstale":"2023-08-09T13:52:02.228770+0000","last_undegraded":"2023-08-09T13:52:02.228770+0000","last_fullsized":"2023-08-09T13:52:02.228770+0000","mapping_epoch":113,"log_start":"0'0","ondisk_log_start":"0'0","created":113,"last_epoch_clean":114,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:51:58.125661+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:51:58.125661+0000","last_clean_scrub_stamp":"2023-08-09T13:51:58.125661+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T14:17:42.189888+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":193,"reported_epoch":116,"state":"active+clean","last_fresh":"2023-08-09T13:52:02.229888+0000","last_change":"2023-08-09T13:33:56.401781+0000","last_active":"2023-08-09T13:52:02.229888+0000","last_peered":"2023-08-09T13:52:02.229888+0000","last_clean":"2023-08-09T13:52:02.229888+0000","last_became_active":"2023-08-09T13:33:56.400336+0000","last_became_peered":"2023-08-09T13:33:56.400336+0000","last_unstale":"2023-08-09T13:52:02.229888+0000","last_undegraded":"2023-08-09T13:52:02.229888+0000","last_fullsized":"2023-08-09T13:52:02.229888+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:33:54.845869+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:33:54.845869+0000","last_clean_scrub_stamp":"2023-08-09T13:33:54.845869+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T21:34:19.616633+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.9","version":"0'0","reported_seq":11,"reported_epoch":116,"state":"active+clean","last_fresh":"2023-08-09T13:52:02.228771+0000","last_change":"2023-08-09T13:52:00.222392+0000","last_active":"2023-08-09T13:52:02.228771+0000","last_peered":"2023-08-09T13:52:02.228771+0000","last_clean":"2023-08-09T13:52:02.228771+0000","last_became_active":"2023-08-09T13:52:00.222090+0000","last_became_peered":"2023-08-09T13:52:00.222090+0000","last_unstale":"2023-08-09T13:52:02.228771+0000","last_undegraded":"2023-08-09T13:52:02.228771+0000","last_fullsized":"2023-08-09T13:52:02.228771+0000","mapping_epoch":114,"log_start":"0'0","ondisk_log_start":"0'0","created":114,"last_epoch_clean":115,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:51:59.130555+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:51:59.130555+0000","last_clean_scrub_stamp":"2023-08-09T13:51:59.130555+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T22:13:05.076442+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"14.8","version":"0'0","reported_seq":13,"reported_epoch":116,"state":"active+clean","last_fresh":"2023-08-09T13:52:02.227331+0000","last_change":"2023-08-09T13:51:59.170689+0000","last_active":"2023-08-09T13:52:02.227331+0000","last_peered":"2023-08-09T13:52:02.227331+0000","last_clean":"2023-08-09T13:52:02.227331+0000","last_became_active":"2023-08-09T13:51:59.170409+0000","last_became_peered":"2023-08-09T13:51:59.170409+0000","last_unstale":"2023-08-09T13:52:02.227331+0000","last_undegraded":"2023-08-09T13:52:02.227331+0000","last_fullsized":"2023-08-09T13:52:02.227331+0000","mapping_epoch":113,"log_start":"0'0","ondisk_log_start":"0'0","created":113,"last_epoch_clean":114,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:51:58.125661+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:51:58.125661+0000","last_clean_scrub_stamp":"2023-08-09T13:51:58.125661+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-11T01:23:51.545812+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":193,"reported_epoch":116,"state":"active+clean","last_fresh":"2023-08-09T13:52:02.230489+0000","last_change":"2023-08-09T13:33:55.886389+0000","last_active":"2023-08-09T13:52:02.230489+0000","last_peered":"2023-08-09T13:52:02.230489+0000","last_clean":"2023-08-09T13:52:02.230489+0000","last_became_active":"2023-08-09T13:33:55.885638+0000","last_became_peered":"2023-08-09T13:33:55.885638+0000","last_unstale":"2023-08-09T13:52:02.230489+0000","last_undegraded":"2023-08-09T13:52:02.230489+0000","last_fullsized":"2023-08-09T13:52:02.230489+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:33:54.845869+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:33:54.845869+0000","last_clean_scrub_stamp":"2023-08-09T13:33:54.845869+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T19:59:29.993498+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]}],"pool_stats":[{"poolid":15,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":2,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":715,"num_read_kb":1418,"num_write":283,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":186,"ondisk_log_size":186,"up":2,"acting":2,"num_store_stats":2},{"poolid":14,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":39,"ondisk_log_size":39,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":8}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834575065,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":37196,"kb_used_data":1984,"kb_used_omap":1,"kb_used_meta":35198,"kb_avail":94334644,"statfs":{"total":96636764160,"available":96598675456,"internally_reserved":0,"allocated":2031616,"data_stored":1352999,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1781,"internal_metadata":36043019},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Aug 9 13:51:20 2023","interfaces":[{"interface":"back","average":{"1min":1.101,"5min":1.1910000000000001,"15min":1.2370000000000001},"min":{"1min":0.61599999999999999,"5min":0.61599999999999999,"15min":0.61599999999999999},"max":{"1min":1.764,"5min":1.8919999999999999,"15min":2.1280000000000001},"last":0.72599999999999998},{"interface":"front","average":{"1min":1.153,"5min":1.224,"15min":1.2430000000000001},"min":{"1min":0.64200000000000002,"5min":0.64200000000000002,"15min":0.64200000000000002},"max":{"1min":1.8839999999999999,"5min":1.8839999999999999,"15min":2.0099999999999998},"last":1.3500000000000001}]},{"osd":1,"last update":"Wed Aug 9 13:51:20 2023","interfaces":[{"interface":"back","average":{"1min":1.1830000000000001,"5min":1.22,"15min":1.26},"min":{"1min":0.69199999999999995,"5min":0.69199999999999995,"15min":0.59799999999999998},"max":{"1min":1.8120000000000001,"5min":1.8120000000000001,"15min":1.9470000000000001},"last":1.0189999999999999},{"interface":"front","average":{"1min":1.1839999999999999,"5min":1.2310000000000001,"15min":1.2809999999999999},"min":{"1min":0.69099999999999995,"5min":0.59299999999999997,"15min":0.59299999999999997},"max":{"1min":1.998,"5min":1.998,"15min":1.998},"last":0.97999999999999998}]},{"osd":2,"last update":"Wed Aug 9 13:51:20 2023","interfaces":[{"interface":"back","average":{"1min":1.1990000000000001,"5min":1.234,"15min":1.252},"min":{"1min":0.79100000000000004,"5min":0.65700000000000003,"15min":0.65700000000000003},"max":{"1min":1.6539999999999999,"5min":2.0129999999999999,"15min":2.0129999999999999},"last":1.1619999999999999},{"interface":"front","average":{"1min":1.22,"5min":1.2829999999999999,"15min":1.369},"min":{"1min":0.58199999999999996,"5min":0.58199999999999996,"15min":0.58199999999999996},"max":{"1min":1.7050000000000001,"5min":1.8740000000000001,"15min":2.4140000000000001},"last":0.91300000000000003}]},{"osd":3,"last update":"Wed Aug 9 13:51:20 2023","interfaces":[{"interface":"back","average":{"1min":1.272,"5min":1.29,"15min":1.3360000000000001},"min":{"1min":0.67500000000000004,"5min":0.67500000000000004,"15min":0.67500000000000004},"max":{"1min":1.8140000000000001,"5min":1.97,"15min":1.97},"last":1.5209999999999999},{"interface":"front","average":{"1min":1.363,"5min":1.3959999999999999,"15min":1.413},"min":{"1min":0.71499999999999997,"5min":0.71499999999999997,"15min":0.71499999999999997},"max":{"1min":1.615,"5min":2.0790000000000002,"15min":2.2480000000000002},"last":1.125}]},{"osd":4,"last update":"Wed Aug 9 13:51:20 2023","interfaces":[{"interface":"back","average":{"1min":1.2430000000000001,"5min":1.2669999999999999,"15min":1.2749999999999999},"min":{"1min":0.74299999999999999,"5min":0.59499999999999997,"15min":0.59499999999999997},"max":{"1min":2.097,"5min":2.097,"15min":2.343},"last":1.2350000000000001},{"interface":"front","average":{"1min":1.2509999999999999,"5min":1.2969999999999999,"15min":1.3340000000000001},"min":{"1min":0.60799999999999998,"5min":0.60799999999999998,"15min":0.60799999999999998},"max":{"1min":2.0539999999999998,"5min":2.1419999999999999,"15min":2.1419999999999999},"last":1.6399999999999999}]},{"osd":5,"last update":"Wed Aug 9 13:51:20 2023","interfaces":[{"interface":"back","average":{"1min":1.456,"5min":1.4350000000000001,"15min":1.4179999999999999},"min":{"1min":0.65800000000000003,"5min":0.65800000000000003,"15min":0.65800000000000003},"max":{"1min":1.7549999999999999,"5min":2.1480000000000001,"15min":2.6749999999999998},"last":1.2709999999999999},{"interface":"front","average":{"1min":1.383,"5min":1.4099999999999999,"15min":1.4450000000000001},"min":{"1min":0.83999999999999997,"5min":0.83999999999999997,"15min":0.79700000000000004},"max":{"1min":2.157,"5min":2.157,"15min":2.9569999999999999},"last":1.5660000000000001}]},{"osd":6,"last update":"Wed Aug 9 13:51:20 2023","interfaces":[{"interface":"back","average":{"1min":1.4650000000000001,"5min":1.504,"15min":1.5429999999999999},"min":{"1min":0.88500000000000001,"5min":0.85499999999999998,"15min":0.85499999999999998},"max":{"1min":1.8280000000000001,"5min":2.2069999999999999,"15min":2.8149999999999999},"last":1.464},{"interface":"front","average":{"1min":1.421,"5min":1.4430000000000001,"15min":1.474},"min":{"1min":0.79700000000000004,"5min":0.79700000000000004,"15min":0.79700000000000004},"max":{"1min":1.736,"5min":2.1320000000000001,"15min":2.2530000000000001},"last":1.7110000000000001}]}]},{"osd":6,"up_from":13,"seq":55834575065,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":36612,"kb_used_data":1400,"kb_used_omap":1,"kb_used_meta":35198,"kb_avail":94335228,"statfs":{"total":96636764160,"available":96599273472,"internally_reserved":0,"allocated":1433600,"data_stored":761810,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1781,"internal_metadata":36043019},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Aug 9 13:51:24 2023","interfaces":[{"interface":"back","average":{"1min":0.98499999999999999,"5min":1.1160000000000001,"15min":1.153},"min":{"1min":0.54000000000000004,"5min":0.54000000000000004,"15min":0.54000000000000004},"max":{"1min":1.484,"5min":1.633,"15min":1.9870000000000001},"last":1.3129999999999999},{"interface":"front","average":{"1min":1.0620000000000001,"5min":1.492,"15min":1.306},"min":{"1min":0.51800000000000002,"5min":0.51800000000000002,"15min":0.51800000000000002},"max":{"1min":1.6479999999999999,"5min":28.036000000000001,"15min":28.036000000000001},"last":0.745}]},{"osd":1,"last update":"Wed Aug 9 13:51:24 2023","interfaces":[{"interface":"back","average":{"1min":1.0940000000000001,"5min":1.238,"15min":1.232},"min":{"1min":0.48999999999999999,"5min":0.48999999999999999,"15min":0.48999999999999999},"max":{"1min":1.6639999999999999,"5min":7.742,"15min":7.742},"last":0.83399999999999996},{"interface":"front","average":{"1min":1.1950000000000001,"5min":1.276,"15min":1.28},"min":{"1min":0.64600000000000002,"5min":0.64600000000000002,"15min":0.64600000000000002},"max":{"1min":2.056,"5min":2.056,"15min":2.056},"last":1.1339999999999999}]},{"osd":2,"last update":"Wed Aug 9 13:51:24 2023","interfaces":[{"interface":"back","average":{"1min":1.206,"5min":1.323,"15min":1.3260000000000001},"min":{"1min":0.80600000000000005,"5min":0.80600000000000005,"15min":0.74099999999999999},"max":{"1min":1.8819999999999999,"5min":4.0640000000000001,"15min":4.0640000000000001},"last":1.3919999999999999},{"interface":"front","average":{"1min":1.161,"5min":1.399,"15min":1.347},"min":{"1min":0.80700000000000005,"5min":0.70999999999999996,"15min":0.70999999999999996},"max":{"1min":1.6679999999999999,"5min":13.154999999999999,"15min":13.154999999999999},"last":0.96699999999999997}]},{"osd":3,"last update":"Wed Aug 9 13:51:24 2023","interfaces":[{"interface":"back","average":{"1min":1.196,"5min":1.452,"15min":1.415},"min":{"1min":0.67200000000000004,"5min":0.67200000000000004,"15min":0.67200000000000004},"max":{"1min":1.522,"5min":13.349,"15min":13.349},"last":1.258},{"interface":"front","average":{"1min":1.254,"5min":1.353,"15min":1.3280000000000001},"min":{"1min":0.75700000000000001,"5min":0.75700000000000001,"15min":0.74299999999999999},"max":{"1min":1.732,"5min":7.9029999999999996,"15min":7.9029999999999996},"last":1.0740000000000001}]},{"osd":4,"last update":"Wed Aug 9 13:51:24 2023","interfaces":[{"interface":"back","average":{"1min":1.323,"5min":1.3540000000000001,"15min":1.284},"min":{"1min":0.86199999999999999,"5min":0.84499999999999997,"15min":0.64300000000000002},"max":{"1min":1.833,"5min":5.2290000000000001,"15min":5.2290000000000001},"last":1.4450000000000001},{"interface":"front","average":{"1min":1.367,"5min":1.506,"15min":1.3879999999999999},"min":{"1min":0.89900000000000002,"5min":0.78700000000000003,"15min":0.78700000000000003},"max":{"1min":1.714,"5min":12.1,"15min":12.1},"last":1.359}]},{"osd":5,"last update":"Wed Aug 9 13:51:24 2023","interfaces":[{"interface":"back","average":{"1min":1.423,"5min":1.446,"15min":1.3660000000000001},"min":{"1min":0.57699999999999996,"5min":0.57699999999999996,"15min":0.57699999999999996},"max":{"1min":1.843,"5min":8.1899999999999995,"15min":8.1899999999999995},"last":1.222},{"interface":"front","average":{"1min":1.55,"5min":1.6679999999999999,"15min":1.593},"min":{"1min":0.93000000000000005,"5min":0.93000000000000005,"15min":0.83799999999999997},"max":{"1min":2.1459999999999999,"5min":8.8889999999999993,"15min":8.8889999999999993},"last":1.496}]},{"osd":7,"last update":"Wed Aug 9 13:51:24 2023","interfaces":[{"interface":"back","average":{"1min":1.5269999999999999,"5min":1.6719999999999999,"15min":1.587},"min":{"1min":0.96099999999999997,"5min":0.96099999999999997,"15min":0.62},"max":{"1min":1.907,"5min":9.3320000000000007,"15min":9.3320000000000007},"last":1.415},{"interface":"front","average":{"1min":1.556,"5min":1.613,"15min":1.55},"min":{"1min":0.88200000000000001,"5min":0.88200000000000001,"15min":0.88200000000000001},"max":{"1min":1.97,"5min":8.0399999999999991,"15min":8.0399999999999991},"last":1.2929999999999999}]}]},{"osd":5,"up_from":13,"seq":55834575065,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":40712,"kb_used_data":1400,"kb_used_omap":1,"kb_used_meta":39294,"kb_avail":94331128,"statfs":{"total":96636764160,"available":96595075072,"internally_reserved":0,"allocated":1433600,"data_stored":762589,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1781,"internal_metadata":40237323},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Aug 9 13:51:21 2023","interfaces":[{"interface":"back","average":{"1min":1.542,"5min":1.25,"15min":1.206},"min":{"1min":0.59099999999999997,"5min":0.59099999999999997,"15min":0.59099999999999997},"max":{"1min":8.9299999999999997,"5min":8.9299999999999997,"15min":8.9299999999999997},"last":0.98299999999999998},{"interface":"front","average":{"1min":1.6080000000000001,"5min":1.264,"15min":1.2330000000000001},"min":{"1min":0.60499999999999998,"5min":0.60499999999999998,"15min":0.60499999999999998},"max":{"1min":8.7140000000000004,"5min":8.7140000000000004,"15min":8.7140000000000004},"last":0.79500000000000004}]},{"osd":1,"last update":"Wed Aug 9 13:51:21 2023","interfaces":[{"interface":"back","average":{"1min":1.476,"5min":1.2330000000000001,"15min":1.246},"min":{"1min":0.68799999999999994,"5min":0.68799999999999994,"15min":0.64100000000000001},"max":{"1min":8.2720000000000002,"5min":8.2720000000000002,"15min":8.2720000000000002},"last":0.86599999999999999},{"interface":"front","average":{"1min":1.6679999999999999,"5min":1.359,"15min":1.3100000000000001},"min":{"1min":0.80500000000000005,"5min":0.66900000000000004,"15min":0.66900000000000004},"max":{"1min":9.0020000000000007,"5min":9.0020000000000007,"15min":9.0020000000000007},"last":0.73399999999999999}]},{"osd":2,"last update":"Wed Aug 9 13:51:21 2023","interfaces":[{"interface":"back","average":{"1min":1.7190000000000001,"5min":1.335,"15min":1.319},"min":{"1min":0.81799999999999995,"5min":0.68999999999999995,"15min":0.61599999999999999},"max":{"1min":8.6050000000000004,"5min":8.6050000000000004,"15min":8.6050000000000004},"last":1.1379999999999999},{"interface":"front","average":{"1min":1.641,"5min":1.371,"15min":1.347},"min":{"1min":0.78300000000000003,"5min":0.78300000000000003,"15min":0.72099999999999997},"max":{"1min":8.4149999999999991,"5min":8.4149999999999991,"15min":8.4149999999999991},"last":1.048}]},{"osd":3,"last update":"Wed Aug 9 13:51:21 2023","interfaces":[{"interface":"back","average":{"1min":1.8160000000000001,"5min":1.466,"15min":1.391},"min":{"1min":0.72599999999999998,"5min":0.72599999999999998,"15min":0.59799999999999998},"max":{"1min":9.0470000000000006,"5min":9.0470000000000006,"15min":9.0470000000000006},"last":1.0960000000000001},{"interface":"front","average":{"1min":1.738,"5min":1.4970000000000001,"15min":1.4370000000000001},"min":{"1min":0.76300000000000001,"5min":0.76300000000000001,"15min":0.70999999999999996},"max":{"1min":8.7669999999999995,"5min":8.7669999999999995,"15min":8.7669999999999995},"last":1.256}]},{"osd":4,"last update":"Wed Aug 9 13:51:21 2023","interfaces":[{"interface":"back","average":{"1min":1.724,"5min":1.367,"15min":1.321},"min":{"1min":0.54300000000000004,"5min":0.54300000000000004,"15min":0.54300000000000004},"max":{"1min":8.4909999999999997,"5min":8.4909999999999997,"15min":8.4909999999999997},"last":1.3100000000000001},{"interface":"front","average":{"1min":1.9079999999999999,"5min":1.4570000000000001,"15min":1.4119999999999999},"min":{"1min":0.68899999999999995,"5min":0.68899999999999995,"15min":0.68899999999999995},"max":{"1min":9.4589999999999996,"5min":9.4589999999999996,"15min":9.4589999999999996},"last":1.2050000000000001}]},{"osd":6,"last update":"Wed Aug 9 13:51:21 2023","interfaces":[{"interface":"back","average":{"1min":2.5259999999999998,"5min":1.5980000000000001,"15min":1.4610000000000001},"min":{"1min":0.51000000000000001,"5min":0.51000000000000001,"15min":0.51000000000000001},"max":{"1min":17.407,"5min":17.407,"15min":17.407},"last":1.464},{"interface":"front","average":{"1min":2.351,"5min":1.6279999999999999,"15min":1.53},"min":{"1min":0.71199999999999997,"5min":0.71199999999999997,"15min":0.71199999999999997},"max":{"1min":16.356000000000002,"5min":16.356000000000002,"15min":16.356000000000002},"last":1.536}]},{"osd":7,"last update":"Wed Aug 9 13:51:21 2023","interfaces":[{"interface":"back","average":{"1min":2.117,"5min":1.665,"15min":1.591},"min":{"1min":0.74299999999999999,"5min":0.74299999999999999,"15min":0.74299999999999999},"max":{"1min":9.9239999999999995,"5min":9.9239999999999995,"15min":9.9239999999999995},"last":1.3879999999999999},{"interface":"front","average":{"1min":2.2400000000000002,"5min":1.716,"15min":1.645},"min":{"1min":0.65600000000000003,"5min":0.65600000000000003,"15min":0.65600000000000003},"max":{"1min":10.789,"5min":10.789,"15min":10.789},"last":1.6000000000000001}]}]},{"osd":4,"up_from":13,"seq":55834575065,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32532,"kb_used_data":1416,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":94339308,"statfs":{"total":96636764160,"available":96603451392,"internally_reserved":0,"allocated":1449984,"data_stored":764255,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1781,"internal_metadata":31848715},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Aug 9 13:51:23 2023","interfaces":[{"interface":"back","average":{"1min":1.131,"5min":1.2150000000000001,"15min":1.2070000000000001},"min":{"1min":0.66600000000000004,"5min":0.66600000000000004,"15min":0.66600000000000004},"max":{"1min":1.768,"5min":2.524,"15min":2.524},"last":1.0529999999999999},{"interface":"front","average":{"1min":1.0609999999999999,"5min":1.181,"15min":1.25},"min":{"1min":0.66800000000000004,"5min":0.62,"15min":0.62},"max":{"1min":1.667,"5min":1.7390000000000001,"15min":1.863},"last":1.238}]},{"osd":1,"last update":"Wed Aug 9 13:51:23 2023","interfaces":[{"interface":"back","average":{"1min":1.1839999999999999,"5min":1.264,"15min":1.2969999999999999},"min":{"1min":0.79700000000000004,"5min":0.67800000000000005,"15min":0.67800000000000005},"max":{"1min":1.609,"5min":1.8480000000000001,"15min":1.8480000000000001},"last":1.4059999999999999},{"interface":"front","average":{"1min":1.1319999999999999,"5min":1.232,"15min":1.2470000000000001},"min":{"1min":0.75,"5min":0.75,"15min":0.65400000000000003},"max":{"1min":1.669,"5min":1.9330000000000001,"15min":1.9330000000000001},"last":1.5049999999999999}]},{"osd":2,"last update":"Wed Aug 9 13:51:23 2023","interfaces":[{"interface":"back","average":{"1min":1.081,"5min":1.2509999999999999,"15min":1.272},"min":{"1min":0.69999999999999996,"5min":0.621,"15min":0.621},"max":{"1min":1.496,"5min":2.6429999999999998,"15min":2.6429999999999998},"last":0.81799999999999995},{"interface":"front","average":{"1min":1.25,"5min":1.3220000000000001,"15min":1.365},"min":{"1min":0.76000000000000001,"5min":0.76000000000000001,"15min":0.75900000000000001},"max":{"1min":2.0910000000000002,"5min":2.0910000000000002,"15min":2.0910000000000002},"last":1.6080000000000001}]},{"osd":3,"last update":"Wed Aug 9 13:51:23 2023","interfaces":[{"interface":"back","average":{"1min":1.1599999999999999,"5min":1.2889999999999999,"15min":1.298},"min":{"1min":0.754,"5min":0.754,"15min":0.754},"max":{"1min":1.921,"5min":2.6099999999999999,"15min":2.6099999999999999},"last":1.2809999999999999},{"interface":"front","average":{"1min":1.2509999999999999,"5min":1.3740000000000001,"15min":1.3819999999999999},"min":{"1min":0.89100000000000001,"5min":0.88800000000000001,"15min":0.63100000000000001},"max":{"1min":1.7010000000000001,"5min":2.7629999999999999,"15min":2.7629999999999999},"last":1.1639999999999999}]},{"osd":5,"last update":"Wed Aug 9 13:51:23 2023","interfaces":[{"interface":"back","average":{"1min":1.294,"5min":1.3140000000000001,"15min":1.327},"min":{"1min":0.67300000000000004,"5min":0.67300000000000004,"15min":0.63200000000000001},"max":{"1min":1.758,"5min":1.8,"15min":2.0099999999999998},"last":1.546},{"interface":"front","average":{"1min":1.361,"5min":1.337,"15min":1.3169999999999999},"min":{"1min":0.65600000000000003,"5min":0.65600000000000003,"15min":0.626},"max":{"1min":2.0369999999999999,"5min":2.7090000000000001,"15min":2.7090000000000001},"last":1.357}]},{"osd":6,"last update":"Wed Aug 9 13:51:23 2023","interfaces":[{"interface":"back","average":{"1min":1.409,"5min":1.341,"15min":1.3340000000000001},"min":{"1min":0.90600000000000003,"5min":0.74399999999999999,"15min":0.59099999999999997},"max":{"1min":1.96,"5min":2.8559999999999999,"15min":2.8559999999999999},"last":1.4530000000000001},{"interface":"front","average":{"1min":1.496,"5min":1.514,"15min":1.5029999999999999},"min":{"1min":0.878,"5min":0.878,"15min":0.76400000000000001},"max":{"1min":2.1890000000000001,"5min":2.2719999999999998,"15min":2.2719999999999998},"last":1.6559999999999999}]},{"osd":7,"last update":"Wed Aug 9 13:51:23 2023","interfaces":[{"interface":"back","average":{"1min":1.3859999999999999,"5min":1.468,"15min":1.4690000000000001},"min":{"1min":0.53100000000000003,"5min":0.53100000000000003,"15min":0.53100000000000003},"max":{"1min":1.9530000000000001,"5min":2.8050000000000002,"15min":2.8050000000000002},"last":1.5720000000000001},{"interface":"front","average":{"1min":1.5289999999999999,"5min":1.52,"15min":1.5129999999999999},"min":{"1min":0.78700000000000003,"5min":0.78700000000000003,"15min":0.72899999999999998},"max":{"1min":2.1419999999999999,"5min":3.415,"15min":3.415},"last":1.3109999999999999}]}]},{"osd":3,"up_from":13,"seq":55834575066,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":36628,"kb_used_data":1412,"kb_used_omap":1,"kb_used_meta":35198,"kb_avail":94335212,"statfs":{"total":96636764160,"available":96599257088,"internally_reserved":0,"allocated":1445888,"data_stored":764153,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1781,"internal_metadata":36043019},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Aug 9 13:51:26 2023","interfaces":[{"interface":"back","average":{"1min":0.76900000000000002,"5min":0.85299999999999998,"15min":0.90500000000000003},"min":{"1min":0.496,"5min":0.27200000000000002,"15min":0.27200000000000002},"max":{"1min":1.5580000000000001,"5min":1.8500000000000001,"15min":1.968},"last":0.91100000000000003},{"interface":"front","average":{"1min":0.83499999999999996,"5min":1.0349999999999999,"15min":1.097},"min":{"1min":0.36299999999999999,"5min":0.36299999999999999,"15min":0.36299999999999999},"max":{"1min":1.4059999999999999,"5min":1.6910000000000001,"15min":2.0590000000000002},"last":1.228}]},{"osd":1,"last update":"Wed Aug 9 13:51:26 2023","interfaces":[{"interface":"back","average":{"1min":0.82799999999999996,"5min":0.95399999999999996,"15min":1.0089999999999999},"min":{"1min":0.51500000000000001,"5min":0.496,"15min":0.438},"max":{"1min":1.27,"5min":1.6519999999999999,"15min":2.5649999999999999},"last":0.873},{"interface":"front","average":{"1min":0.97899999999999998,"5min":1.081,"15min":1.161},"min":{"1min":0.60499999999999998,"5min":0.51800000000000002,"15min":0.498},"max":{"1min":1.6779999999999999,"5min":1.6839999999999999,"15min":2.6760000000000002},"last":1.0289999999999999}]},{"osd":2,"last update":"Wed Aug 9 13:51:26 2023","interfaces":[{"interface":"back","average":{"1min":0.995,"5min":1.115,"15min":1.194},"min":{"1min":0.42999999999999999,"5min":0.42999999999999999,"15min":0.42999999999999999},"max":{"1min":1.6879999999999999,"5min":1.913,"15min":1.923},"last":1.585},{"interface":"front","average":{"1min":1.103,"5min":1.2290000000000001,"15min":1.3069999999999999},"min":{"1min":0.61299999999999999,"5min":0.52300000000000002,"15min":0.52300000000000002},"max":{"1min":1.7290000000000001,"5min":1.9399999999999999,"15min":2.0259999999999998},"last":1.177}]},{"osd":4,"last update":"Wed Aug 9 13:51:26 2023","interfaces":[{"interface":"back","average":{"1min":1.1000000000000001,"5min":1.214,"15min":1.274},"min":{"1min":0.54900000000000004,"5min":0.54900000000000004,"15min":0.54900000000000004},"max":{"1min":1.732,"5min":1.8160000000000001,"15min":1.8160000000000001},"last":1.456},{"interface":"front","average":{"1min":1.2769999999999999,"5min":1.3440000000000001,"15min":1.3959999999999999},"min":{"1min":0.76100000000000001,"5min":0.628,"15min":0.628},"max":{"1min":2.782,"5min":2.782,"15min":2.782},"last":1.2809999999999999}]},{"osd":5,"last update":"Wed Aug 9 13:51:26 2023","interfaces":[{"interface":"back","average":{"1min":1.266,"5min":1.4179999999999999,"15min":1.47},"min":{"1min":0.69899999999999995,"5min":0.67600000000000005,"15min":0.67200000000000004},"max":{"1min":1.833,"5min":2.254,"15min":2.254},"last":1.397},{"interface":"front","average":{"1min":1.258,"5min":1.431,"15min":1.484},"min":{"1min":0.66000000000000003,"5min":0.65900000000000003,"15min":0.65900000000000003},"max":{"1min":1.919,"5min":2.3069999999999999,"15min":2.3069999999999999},"last":1.508}]},{"osd":6,"last update":"Wed Aug 9 13:51:26 2023","interfaces":[{"interface":"back","average":{"1min":1.381,"5min":1.419,"15min":1.4630000000000001},"min":{"1min":0.78200000000000003,"5min":0.68200000000000005,"15min":0.68200000000000005},"max":{"1min":2.8340000000000001,"5min":2.8340000000000001,"15min":2.8340000000000001},"last":1.5489999999999999},{"interface":"front","average":{"1min":1.389,"5min":1.5109999999999999,"15min":1.5580000000000001},"min":{"1min":0.64100000000000001,"5min":0.64100000000000001,"15min":0.64100000000000001},"max":{"1min":2.9060000000000001,"5min":2.9060000000000001,"15min":2.9060000000000001},"last":1.7050000000000001}]},{"osd":7,"last update":"Wed Aug 9 13:51:26 2023","interfaces":[{"interface":"back","average":{"1min":1.3580000000000001,"5min":1.5549999999999999,"15min":1.6100000000000001},"min":{"1min":0.77800000000000002,"5min":0.69399999999999995,"15min":0.69399999999999995},"max":{"1min":2.0419999999999998,"5min":2.46,"15min":2.46},"last":1.339},{"interface":"front","average":{"1min":1.403,"5min":1.5049999999999999,"15min":1.5620000000000001},"min":{"1min":0.93799999999999994,"5min":0.64200000000000002,"15min":0.64200000000000002},"max":{"1min":2.7200000000000002,"5min":2.7200000000000002,"15min":2.7200000000000002},"last":1.643}]}]},{"osd":2,"up_from":13,"seq":55834575065,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":36612,"kb_used_data":1400,"kb_used_omap":1,"kb_used_meta":35198,"kb_avail":94335228,"statfs":{"total":96636764160,"available":96599273472,"internally_reserved":0,"allocated":1433600,"data_stored":762553,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1781,"internal_metadata":36043019},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Aug 9 13:51:19 2023","interfaces":[{"interface":"back","average":{"1min":0.82699999999999996,"5min":0.91400000000000003,"15min":0.95599999999999996},"min":{"1min":0.47199999999999998,"5min":0.36499999999999999,"15min":0.36499999999999999},"max":{"1min":1.484,"5min":1.5940000000000001,"15min":1.8380000000000001},"last":0.76600000000000001},{"interface":"front","average":{"1min":0.90200000000000002,"5min":0.96699999999999997,"15min":1.0009999999999999},"min":{"1min":0.38200000000000001,"5min":0.374,"15min":0.374},"max":{"1min":1.7,"5min":2.3929999999999998,"15min":2.3929999999999998},"last":1.2}]},{"osd":1,"last update":"Wed Aug 9 13:51:19 2023","interfaces":[{"interface":"back","average":{"1min":1.038,"5min":1.143,"15min":1.1839999999999999},"min":{"1min":0.39400000000000002,"5min":0.39400000000000002,"15min":0.39400000000000002},"max":{"1min":1.615,"5min":1.77,"15min":1.8420000000000001},"last":0.88300000000000001},{"interface":"front","average":{"1min":0.98399999999999999,"5min":1.171,"15min":1.3220000000000001},"min":{"1min":0.435,"5min":0.435,"15min":0.435},"max":{"1min":1.659,"5min":2.4870000000000001,"15min":2.4870000000000001},"last":1.298}]},{"osd":3,"last update":"Wed Aug 9 13:51:19 2023","interfaces":[{"interface":"back","average":{"1min":0.93100000000000005,"5min":0.97999999999999998,"15min":1.0509999999999999},"min":{"1min":0.55300000000000005,"5min":0.38900000000000001,"15min":0.38900000000000001},"max":{"1min":1.417,"5min":1.708,"15min":2.0409999999999999},"last":0.73399999999999999},{"interface":"front","average":{"1min":1.0820000000000001,"5min":1.2549999999999999,"15min":1.3080000000000001},"min":{"1min":0.52900000000000003,"5min":0.46200000000000002,"15min":0.46200000000000002},"max":{"1min":1.6279999999999999,"5min":8.8130000000000006,"15min":8.8130000000000006},"last":1.1639999999999999}]},{"osd":4,"last update":"Wed Aug 9 13:51:19 2023","interfaces":[{"interface":"back","average":{"1min":1.335,"5min":1.3740000000000001,"15min":1.452},"min":{"1min":0.81000000000000005,"5min":0.628,"15min":0.628},"max":{"1min":1.8340000000000001,"5min":2.089,"15min":2.089},"last":1.0740000000000001},{"interface":"front","average":{"1min":1.2969999999999999,"5min":1.3360000000000001,"15min":1.3939999999999999},"min":{"1min":0.75900000000000001,"5min":0.68600000000000005,"15min":0.68600000000000005},"max":{"1min":1.7509999999999999,"5min":2.472,"15min":2.472},"last":1.419}]},{"osd":5,"last update":"Wed Aug 9 13:51:19 2023","interfaces":[{"interface":"back","average":{"1min":1.361,"5min":1.4059999999999999,"15min":1.4590000000000001},"min":{"1min":0.86399999999999999,"5min":0.59999999999999998,"15min":0.59999999999999998},"max":{"1min":2.0640000000000001,"5min":2.5840000000000001,"15min":2.5840000000000001},"last":1.3620000000000001},{"interface":"front","average":{"1min":1.2829999999999999,"5min":1.3260000000000001,"15min":1.351},"min":{"1min":0.89000000000000001,"5min":0.56799999999999995,"15min":0.56799999999999995},"max":{"1min":1.78,"5min":3.5659999999999998,"15min":3.5659999999999998},"last":1.032}]},{"osd":6,"last update":"Wed Aug 9 13:51:19 2023","interfaces":[{"interface":"back","average":{"1min":1.3009999999999999,"5min":1.3879999999999999,"15min":1.429},"min":{"1min":0.82199999999999995,"5min":0.73799999999999999,"15min":0.73799999999999999},"max":{"1min":2.1200000000000001,"5min":3.5099999999999998,"15min":3.5099999999999998},"last":1.2749999999999999},{"interface":"front","average":{"1min":1.4099999999999999,"5min":1.458,"15min":1.54},"min":{"1min":0.84899999999999998,"5min":0.66300000000000003,"15min":0.66300000000000003},"max":{"1min":2.04,"5min":2.3769999999999998,"15min":2.3769999999999998},"last":1.248}]},{"osd":7,"last update":"Wed Aug 9 13:51:19 2023","interfaces":[{"interface":"back","average":{"1min":1.3620000000000001,"5min":1.452,"15min":1.54},"min":{"1min":0.85099999999999998,"5min":0.64600000000000002,"15min":0.64600000000000002},"max":{"1min":1.865,"5min":2.5249999999999999,"15min":2.5249999999999999},"last":1.476},{"interface":"front","average":{"1min":1.3939999999999999,"5min":1.476,"15min":1.514},"min":{"1min":0.94699999999999995,"5min":0.70499999999999996,"15min":0.70499999999999996},"max":{"1min":1.891,"5min":3.609,"15min":3.609},"last":1.339}]}]},{"osd":1,"up_from":13,"seq":55834575065,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":47044,"kb_used_data":1976,"kb_used_omap":1,"kb_used_meta":45054,"kb_avail":94324796,"statfs":{"total":96636764160,"available":96588591104,"internally_reserved":0,"allocated":2023424,"data_stored":1352887,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1781,"internal_metadata":46135563},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Aug 9 13:51:34 2023","interfaces":[{"interface":"back","average":{"1min":0.77600000000000002,"5min":0.90000000000000002,"15min":0.93799999999999994},"min":{"1min":0.45800000000000002,"5min":0.45800000000000002,"15min":0.45800000000000002},"max":{"1min":1.1910000000000001,"5min":1.7110000000000001,"15min":2.1080000000000001},"last":0.58799999999999997},{"interface":"front","average":{"1min":0.83299999999999996,"5min":1.0129999999999999,"15min":1.095},"min":{"1min":0.60599999999999998,"5min":0.432,"15min":0.432},"max":{"1min":1.6519999999999999,"5min":2.468,"15min":2.468},"last":1.214}]},{"osd":2,"last update":"Wed Aug 9 13:51:34 2023","interfaces":[{"interface":"back","average":{"1min":0.83399999999999996,"5min":0.93799999999999994,"15min":0.99299999999999999},"min":{"1min":0.57499999999999996,"5min":0.41099999999999998,"15min":0.41099999999999998},"max":{"1min":1.2130000000000001,"5min":1.907,"15min":1.907},"last":0.80900000000000005},{"interface":"front","average":{"1min":0.91100000000000003,"5min":1.0920000000000001,"15min":1.1859999999999999},"min":{"1min":0.59699999999999998,"5min":0.54400000000000004,"15min":0.54400000000000004},"max":{"1min":1.45,"5min":2.1280000000000001,"15min":2.145},"last":1.3700000000000001}]},{"osd":3,"last update":"Wed Aug 9 13:51:34 2023","interfaces":[{"interface":"back","average":{"1min":0.97199999999999998,"5min":1.1890000000000001,"15min":1.3220000000000001},"min":{"1min":0.60799999999999998,"5min":0.54400000000000004,"15min":0.54400000000000004},"max":{"1min":1.466,"5min":2.0270000000000001,"15min":2.3210000000000002},"last":1.131},{"interface":"front","average":{"1min":1.042,"5min":1.163,"15min":1.25},"min":{"1min":0.625,"5min":0.49299999999999999,"15min":0.45300000000000001},"max":{"1min":1.6539999999999999,"5min":2.3540000000000001,"15min":2.7669999999999999},"last":0.66700000000000004}]},{"osd":4,"last update":"Wed Aug 9 13:51:34 2023","interfaces":[{"interface":"back","average":{"1min":1.1819999999999999,"5min":1.337,"15min":1.4059999999999999},"min":{"1min":0.79800000000000004,"5min":0.79800000000000004,"15min":0.58099999999999996},"max":{"1min":1.9259999999999999,"5min":1.97,"15min":2.5},"last":1.0069999999999999},{"interface":"front","average":{"1min":1.157,"5min":1.27,"15min":1.329},"min":{"1min":0.76600000000000001,"5min":0.70099999999999996,"15min":0.70099999999999996},"max":{"1min":1.895,"5min":1.895,"15min":1.901},"last":1.048}]},{"osd":5,"last update":"Wed Aug 9 13:51:34 2023","interfaces":[{"interface":"back","average":{"1min":1.2150000000000001,"5min":1.29,"15min":1.3680000000000001},"min":{"1min":0.76800000000000002,"5min":0.76800000000000002,"15min":0.76800000000000002},"max":{"1min":1.73,"5min":2.294,"15min":2.294},"last":1.278},{"interface":"front","average":{"1min":1.2949999999999999,"5min":1.3999999999999999,"15min":1.512},"min":{"1min":0.89200000000000002,"5min":0.89200000000000002,"15min":0.89200000000000002},"max":{"1min":1.6859999999999999,"5min":2.1389999999999998,"15min":2.4649999999999999},"last":1.591}]},{"osd":6,"last update":"Wed Aug 9 13:51:34 2023","interfaces":[{"interface":"back","average":{"1min":1.325,"5min":1.4339999999999999,"15min":1.5149999999999999},"min":{"1min":0.96599999999999997,"5min":0.77500000000000002,"15min":0.67400000000000004},"max":{"1min":1.5780000000000001,"5min":2.2080000000000002,"15min":2.5270000000000001},"last":1.4390000000000001},{"interface":"front","average":{"1min":1.28,"5min":1.3979999999999999,"15min":1.4990000000000001},"min":{"1min":0.73599999999999999,"5min":0.73599999999999999,"15min":0.73599999999999999},"max":{"1min":1.591,"5min":2.02,"15min":2.1040000000000001},"last":1.4630000000000001}]},{"osd":7,"last update":"Wed Aug 9 13:51:34 2023","interfaces":[{"interface":"back","average":{"1min":1.3240000000000001,"5min":1.4930000000000001,"15min":1.615},"min":{"1min":0.93700000000000006,"5min":0.86899999999999999,"15min":0.86899999999999999},"max":{"1min":1.776,"5min":2.4169999999999998,"15min":2.5790000000000002},"last":1.4870000000000001},{"interface":"front","average":{"1min":1.3420000000000001,"5min":1.488,"15min":1.5980000000000001},"min":{"1min":0.96899999999999997,"5min":0.91300000000000003,"15min":0.89500000000000002},"max":{"1min":1.8320000000000001,"5min":2.1040000000000001,"15min":2.6509999999999998},"last":1.3320000000000001}]}]},{"osd":0,"up_from":13,"seq":55834575065,"num_pgs":32,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":40704,"kb_used_data":1396,"kb_used_omap":1,"kb_used_meta":39294,"kb_avail":94331136,"statfs":{"total":96636764160,"available":96595083264,"internally_reserved":0,"allocated":1429504,"data_stored":761740,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1781,"internal_metadata":40237323},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Wed Aug 9 13:51:27 2023","interfaces":[{"interface":"back","average":{"1min":0.95899999999999996,"5min":0.91300000000000003,"15min":0.92100000000000004},"min":{"1min":0.41199999999999998,"5min":0.32500000000000001,"15min":0.32500000000000001},"max":{"1min":1.4990000000000001,"5min":1.756,"15min":1.756},"last":0.97099999999999997},{"interface":"front","average":{"1min":1.002,"5min":1.077,"15min":1.111},"min":{"1min":0.39200000000000002,"5min":0.39200000000000002,"15min":0.39200000000000002},"max":{"1min":1.546,"5min":1.9019999999999999,"15min":1.9019999999999999},"last":1.0960000000000001}]},{"osd":2,"last update":"Wed Aug 9 13:51:27 2023","interfaces":[{"interface":"back","average":{"1min":1,"5min":1.0680000000000001,"15min":1.1339999999999999},"min":{"1min":0.58399999999999996,"5min":0.51400000000000001,"15min":0.503},"max":{"1min":1.5089999999999999,"5min":1.776,"15min":1.889},"last":0.79900000000000004},{"interface":"front","average":{"1min":0.98799999999999999,"5min":1.1020000000000001,"15min":1.1739999999999999},"min":{"1min":0.439,"5min":0.439,"15min":0.439},"max":{"1min":1.6259999999999999,"5min":2.0129999999999999,"15min":2.0129999999999999},"last":1.286}]},{"osd":3,"last update":"Wed Aug 9 13:51:27 2023","interfaces":[{"interface":"back","average":{"1min":1,"5min":1.0349999999999999,"15min":1.101},"min":{"1min":0.50900000000000001,"5min":0.46300000000000002,"15min":0.46300000000000002},"max":{"1min":1.5189999999999999,"5min":1.823,"15min":1.8919999999999999},"last":1.1559999999999999},{"interface":"front","average":{"1min":1.2390000000000001,"5min":1.252,"15min":1.3140000000000001},"min":{"1min":0.63400000000000001,"5min":0.54700000000000004,"15min":0.54700000000000004},"max":{"1min":2.2050000000000001,"5min":2.347,"15min":2.4649999999999999},"last":1.3640000000000001}]},{"osd":4,"last update":"Wed Aug 9 13:51:27 2023","interfaces":[{"interface":"back","average":{"1min":1.2509999999999999,"5min":1.2949999999999999,"15min":1.343},"min":{"1min":0.73899999999999999,"5min":0.73899999999999999,"15min":0.73899999999999999},"max":{"1min":1.581,"5min":2.2309999999999999,"15min":2.2309999999999999},"last":1.5369999999999999},{"interface":"front","average":{"1min":1.409,"5min":1.417,"15min":1.456},"min":{"1min":0.86399999999999999,"5min":0.77600000000000002,"15min":0.77600000000000002},"max":{"1min":2.3540000000000001,"5min":2.516,"15min":2.5390000000000001},"last":1.2490000000000001}]},{"osd":5,"last update":"Wed Aug 9 13:51:27 2023","interfaces":[{"interface":"back","average":{"1min":1.3859999999999999,"5min":1.4019999999999999,"15min":1.462},"min":{"1min":0.875,"5min":0.66300000000000003,"15min":0.66300000000000003},"max":{"1min":1.764,"5min":2.6669999999999998,"15min":2.6669999999999998},"last":1.407},{"interface":"front","average":{"1min":1.333,"5min":1.3859999999999999,"15min":1.4470000000000001},"min":{"1min":0.88400000000000001,"5min":0.629,"15min":0.629},"max":{"1min":1.645,"5min":2.1499999999999999,"15min":2.1499999999999999},"last":1.3149999999999999}]},{"osd":6,"last update":"Wed Aug 9 13:51:27 2023","interfaces":[{"interface":"back","average":{"1min":1.4870000000000001,"5min":1.5089999999999999,"15min":1.548},"min":{"1min":1.014,"5min":0.73599999999999999,"15min":0.73599999999999999},"max":{"1min":2.4670000000000001,"5min":2.4670000000000001,"15min":2.7250000000000001},"last":1.486},{"interface":"front","average":{"1min":1.486,"5min":1.526,"15min":1.5569999999999999},"min":{"1min":0.95999999999999996,"5min":0.86699999999999999,"15min":0.86699999999999999},"max":{"1min":1.988,"5min":2.399,"15min":2.399},"last":1.345}]},{"osd":7,"last update":"Wed Aug 9 13:51:27 2023","interfaces":[{"interface":"back","average":{"1min":1.552,"5min":1.5720000000000001,"15min":1.621},"min":{"1min":0.89200000000000002,"5min":0.72299999999999998,"15min":0.72299999999999998},"max":{"1min":2.0880000000000001,"5min":2.3900000000000001,"15min":2.3900000000000001},"last":1.458},{"interface":"front","average":{"1min":1.4179999999999999,"5min":1.484,"15min":1.54},"min":{"1min":0.84099999999999997,"5min":0.76700000000000002,"15min":0.76700000000000002},"max":{"1min":1.7969999999999999,"5min":2.274,"15min":2.274},"last":1.425}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":34,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1634,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":1736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":70,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":652,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":112,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-08-09T13:52:10.403 INFO:tasks.cephfs.filesystem.ceph_manager:clean! 2023-08-09T13:52:10.405 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth caps client.0 mon 'allow r' osd 'allow rw tag cephfs data=cephfs' mds allow 2023-08-09T13:52:10.685 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:52:10.683+0000 7fd8fbf2f640 1 Processor -- start 2023-08-09T13:52:10.685 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:52:10.683+0000 7fd8fbf2f640 1 -- start start 2023-08-09T13:52:10.686 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:52:10.685+0000 7fd8fbf2f640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7fd8f40cb2d0 0x7fd8f415b380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:52:10.687 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:52:10.685+0000 7fd8fbf2f640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fd8f415df60 0x7fd8f40ce1c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:52:10.687 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:52:10.685+0000 7fd8fbf2f640 1 -- --> v1:172.21.15.203:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd8f415aef0 con 0x7fd8f415e3c0 2023-08-09T13:52:10.687 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:52:10.685+0000 7fd8fbf2f640 1 -- --> v2:172.21.15.43:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd8f415a970 con 0x7fd8f40cb2d0 2023-08-09T13:52:10.687 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:52:10.685+0000 7fd8fbf2f640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd8f415b8c0 con 0x7fd8f415df60 2023-08-09T13:52:10.687 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:52:10.685+0000 7fd8f9ccb640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7fd8f40cb2d0 0x7fd8f415b380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:52:10.688 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:52:10.685+0000 7fd8f9ccb640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7fd8f40cb2d0 0x7fd8f415b380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:54360/0 (socket says 172.21.15.43:54360) 2023-08-09T13:52:10.688 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:52:10.685+0000 7fd8f9ccb640 1 -- 172.21.15.43:0/1739607639 learned_addr learned my addr 172.21.15.43:0/1739607639 (peer_addr_for_me v2:172.21.15.43:0/0) 2023-08-09T13:52:10.688 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:52:10.686+0000 7fd8f9ccb640 1 -- 172.21.15.43:0/1739607639 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fd8f415df60 msgr2=0x7fd8f40ce1c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-09T13:52:10.688 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:52:10.686+0000 7fd8f9ccb640 1 --2- 172.21.15.43:0/1739607639 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fd8f415df60 0x7fd8f40ce1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:52:10.688 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:52:10.686+0000 7fd8f9ccb640 1 -- 172.21.15.43:0/1739607639 >> v1:172.21.15.203:6789/0 conn(0x7fd8f415e3c0 legacy=0x7fd8f40c9f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-09T13:52:10.688 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:52:10.686+0000 7fd8f9ccb640 1 -- 172.21.15.43:0/1739607639 --> v2:172.21.15.43:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd8f415e940 con 0x7fd8f40cb2d0 2023-08-09T13:52:10.689 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:52:10.687+0000 7fd8f9ccb640 1 --2- 172.21.15.43:0/1739607639 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fd8f40cb2d0 0x7fd8f415b380 secure :-1 s=READY pgs=1361 cs=0 l=1 rev1=1 crypto rx=0x7fd8f0009870 tx=0x7fd8f002b8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=325a8b0ee3559649 server_cookie=0 in_seq=0 out_seq=0 2023-08-09T13:52:10.690 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:52:10.687+0000 7fd8f8cc9640 1 -- 172.21.15.43:0/1739607639 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd8f0031640 con 0x7fd8f40cb2d0 2023-08-09T13:52:10.690 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:52:10.687+0000 7fd8f8cc9640 1 -- 172.21.15.43:0/1739607639 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd8f00317a0 con 0x7fd8f40cb2d0 2023-08-09T13:52:10.690 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:52:10.687+0000 7fd8f8cc9640 1 -- 172.21.15.43:0/1739607639 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd8f0031b30 con 0x7fd8f40cb2d0 2023-08-09T13:52:10.690 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:52:10.688+0000 7fd8fbf2f640 1 -- 172.21.15.43:0/1739607639 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fd8f40cb2d0 msgr2=0x7fd8f415b380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:52:10.690 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:52:10.688+0000 7fd8fbf2f640 1 --2- 172.21.15.43:0/1739607639 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fd8f40cb2d0 0x7fd8f415b380 secure :-1 s=READY pgs=1361 cs=0 l=1 rev1=1 crypto rx=0x7fd8f0009870 tx=0x7fd8f002b8f0 comp rx=0 tx=0).stop 2023-08-09T13:52:10.690 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:52:10.688+0000 7fd8fbf2f640 1 -- 172.21.15.43:0/1739607639 shutdown_connections 2023-08-09T13:52:10.690 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:52:10.688+0000 7fd8fbf2f640 1 --2- 172.21.15.43:0/1739607639 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fd8f415df60 0x7fd8f40ce1c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:52:10.690 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:52:10.688+0000 7fd8fbf2f640 1 --2- 172.21.15.43:0/1739607639 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fd8f40cb2d0 0x7fd8f415b380 unknown :-1 s=CLOSED pgs=1361 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:52:10.691 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:52:10.688+0000 7fd8fbf2f640 1 -- 172.21.15.43:0/1739607639 >> 172.21.15.43:0/1739607639 conn(0x7fd8f40aff50 msgr2=0x7fd8f40b0360 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-09T13:52:10.691 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:52:10.688+0000 7fd8fbf2f640 1 -- 172.21.15.43:0/1739607639 shutdown_connections 2023-08-09T13:52:10.691 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:52:10.688+0000 7fd8fbf2f640 1 -- 172.21.15.43:0/1739607639 wait complete. 2023-08-09T13:52:10.691 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:52:10.690+0000 7fd8fbf2f640 1 Processor -- start 2023-08-09T13:52:10.692 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:52:10.690+0000 7fd8fbf2f640 1 -- start start 2023-08-09T13:52:10.692 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:52:10.691+0000 7fd8fbf2f640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fd8f40cb2d0 0x7fd8f414b6c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:52:10.692 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:52:10.691+0000 7fd8fbf2f640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fd8f415df60 0x7fd8f4153760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:52:10.693 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:52:10.691+0000 7fd8fbf2f640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fd8f415e3c0 0x7fd8f40cc640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:52:10.693 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:52:10.691+0000 7fd8fbf2f640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd8f4152e70 con 0x7fd8f415e3c0 2023-08-09T13:52:10.693 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:52:10.691+0000 7fd8fbf2f640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd8f4153ca0 con 0x7fd8f415df60 2023-08-09T13:52:10.693 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:52:10.691+0000 7fd8fbf2f640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd8f4151f80 con 0x7fd8f40cb2d0 2023-08-09T13:52:10.693 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:52:10.691+0000 7fd8fa4cc640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fd8f415e3c0 0x7fd8f40cc640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:52:10.694 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:52:10.691+0000 7fd8fa4cc640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fd8f415e3c0 0x7fd8f40cc640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:54366/0 (socket says 172.21.15.43:54366) 2023-08-09T13:52:10.694 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:52:10.691+0000 7fd8fa4cc640 1 -- 172.21.15.43:0/2203112317 learned_addr learned my addr 172.21.15.43:0/2203112317 (peer_addr_for_me v2:172.21.15.43:0/0) 2023-08-09T13:52:10.694 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:52:10.692+0000 7fd8f9ccb640 1 --2- 172.21.15.43:0/2203112317 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fd8f40cb2d0 0x7fd8f414b6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:52:10.695 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:52:10.692+0000 7fd8f94ca640 1 --2- 172.21.15.43:0/2203112317 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fd8f415df60 0x7fd8f4153760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:52:10.695 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:52:10.692+0000 7fd8fa4cc640 1 -- 172.21.15.43:0/2203112317 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fd8f40cb2d0 msgr2=0x7fd8f414b6c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:52:10.695 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:52:10.692+0000 7fd8fa4cc640 1 --2- 172.21.15.43:0/2203112317 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fd8f40cb2d0 0x7fd8f414b6c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:52:10.695 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:52:10.692+0000 7fd8fa4cc640 1 -- 172.21.15.43:0/2203112317 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fd8f415df60 msgr2=0x7fd8f4153760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:52:10.695 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:52:10.692+0000 7fd8fa4cc640 1 --2- 172.21.15.43:0/2203112317 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fd8f415df60 0x7fd8f4153760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:52:10.695 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:52:10.692+0000 7fd8fa4cc640 1 -- 172.21.15.43:0/2203112317 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd8f00094d0 con 0x7fd8f415e3c0 2023-08-09T13:52:10.695 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:52:10.692+0000 7fd8fa4cc640 1 --2- 172.21.15.43:0/2203112317 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fd8f415e3c0 0x7fd8f40cc640 secure :-1 s=READY pgs=1362 cs=0 l=1 rev1=1 crypto rx=0x7fd8ec00b990 tx=0x7fd8ec00be50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-09T13:52:10.696 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:52:10.692+0000 7fd8ea7fc640 1 -- 172.21.15.43:0/2203112317 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd8ec00cba0 con 0x7fd8f415e3c0 2023-08-09T13:52:10.696 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:52:10.692+0000 7fd8ea7fc640 1 -- 172.21.15.43:0/2203112317 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd8ec00cd00 con 0x7fd8f415e3c0 2023-08-09T13:52:10.696 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:52:10.692+0000 7fd8ea7fc640 1 -- 172.21.15.43:0/2203112317 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd8ec014680 con 0x7fd8f415e3c0 2023-08-09T13:52:10.696 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:52:10.692+0000 7fd8fbf2f640 1 -- 172.21.15.43:0/2203112317 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd8f4152260 con 0x7fd8f415e3c0 2023-08-09T13:52:10.696 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:52:10.693+0000 7fd8fbf2f640 1 -- 172.21.15.43:0/2203112317 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd8f4152820 con 0x7fd8f415e3c0 2023-08-09T13:52:10.697 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:52:10.694+0000 7fd8fbf2f640 1 -- 172.21.15.43:0/2203112317 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd8f4149740 con 0x7fd8f415e3c0 2023-08-09T13:52:10.697 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:52:10.695+0000 7fd8ea7fc640 1 -- 172.21.15.43:0/2203112317 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 45) v1 ==== 88443+0+0 (secure 0 0 0) 0x7fd8ec01d3b0 con 0x7fd8f415e3c0 2023-08-09T13:52:10.697 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:52:10.696+0000 7fd8ea7fc640 1 --2- 172.21.15.43:0/2203112317 >> [v2:172.21.15.203:6832/1317108840,v1:172.21.15.203:6833/1317108840] conn(0x7fd8d806a1a0 0x7fd8d806c660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:52:10.697 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:52:10.696+0000 7fd8ea7fc640 1 -- 172.21.15.43:0/2203112317 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(116..116 src has 1..116) v4 ==== 6426+0+0 (secure 0 0 0) 0x7fd8ec0887a0 con 0x7fd8f415e3c0 2023-08-09T13:52:10.698 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:52:10.697+0000 7fd8f9ccb640 1 --2- 172.21.15.43:0/2203112317 >> [v2:172.21.15.203:6832/1317108840,v1:172.21.15.203:6833/1317108840] conn(0x7fd8d806a1a0 0x7fd8d806c660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:52:10.699 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:52:10.698+0000 7fd8f9ccb640 1 --2- 172.21.15.43:0/2203112317 >> [v2:172.21.15.203:6832/1317108840,v1:172.21.15.203:6833/1317108840] conn(0x7fd8d806a1a0 0x7fd8d806c660 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7fd8f0036000 tx=0x7fd8f003c040 comp rx=0 tx=0).ready entity=mgr.6619 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-09T13:52:10.701 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:52:10.700+0000 7fd8ea7fc640 1 -- 172.21.15.43:0/2203112317 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178066 (secure 0 0 0) 0x7fd8f4149740 con 0x7fd8f415e3c0 2023-08-09T13:52:10.969 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:52:10.968+0000 7fd8fbf2f640 1 -- 172.21.15.43:0/2203112317 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]} v 0) v1 -- 0x7fd8f4149740 con 0x7fd8f415e3c0 2023-08-09T13:52:10.975 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:52:10.973+0000 7fd8ea7fc640 1 -- 172.21.15.43:0/2203112317 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]}]=0 updated caps for client.0 v17) v1 ==== 190+0+147 (secure 0 0 0) 0x7fd8f4149740 con 0x7fd8f415e3c0 2023-08-09T13:52:10.975 INFO:teuthology.orchestra.run.smithi043.stdout:[client.0] 2023-08-09T13:52:10.976 INFO:teuthology.orchestra.run.smithi043.stdout: key = AQCPldNkg8zrDxAAM/hnJaE6S9HU/fRwQeBV5w== 2023-08-09T13:52:10.976 INFO:teuthology.orchestra.run.smithi043.stdout: caps mds = "allow" 2023-08-09T13:52:10.976 INFO:teuthology.orchestra.run.smithi043.stdout: caps mon = "allow r" 2023-08-09T13:52:10.976 INFO:teuthology.orchestra.run.smithi043.stdout: caps osd = "allow rw tag cephfs data=cephfs" 2023-08-09T13:52:10.976 INFO:teuthology.orchestra.run.smithi043.stderr:updated caps for client.0 2023-08-09T13:52:10.978 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:52:10.976+0000 7fd8fbf2f640 1 -- 172.21.15.43:0/2203112317 >> [v2:172.21.15.203:6832/1317108840,v1:172.21.15.203:6833/1317108840] conn(0x7fd8d806a1a0 msgr2=0x7fd8d806c660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:52:10.978 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:52:10.976+0000 7fd8fbf2f640 1 --2- 172.21.15.43:0/2203112317 >> [v2:172.21.15.203:6832/1317108840,v1:172.21.15.203:6833/1317108840] conn(0x7fd8d806a1a0 0x7fd8d806c660 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7fd8f0036000 tx=0x7fd8f003c040 comp rx=0 tx=0).stop 2023-08-09T13:52:10.978 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:52:10.976+0000 7fd8fbf2f640 1 -- 172.21.15.43:0/2203112317 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fd8f415e3c0 msgr2=0x7fd8f40cc640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:52:10.979 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:52:10.976+0000 7fd8fbf2f640 1 --2- 172.21.15.43:0/2203112317 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fd8f415e3c0 0x7fd8f40cc640 secure :-1 s=READY pgs=1362 cs=0 l=1 rev1=1 crypto rx=0x7fd8ec00b990 tx=0x7fd8ec00be50 comp rx=0 tx=0).stop 2023-08-09T13:52:10.979 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:52:10.977+0000 7fd8fbf2f640 1 -- 172.21.15.43:0/2203112317 shutdown_connections 2023-08-09T13:52:10.979 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:52:10.977+0000 7fd8fbf2f640 1 --2- 172.21.15.43:0/2203112317 >> [v2:172.21.15.203:6832/1317108840,v1:172.21.15.203:6833/1317108840] conn(0x7fd8d806a1a0 0x7fd8d806c660 unknown :-1 s=CLOSED pgs=71 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:52:10.979 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:52:10.977+0000 7fd8fbf2f640 1 --2- 172.21.15.43:0/2203112317 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fd8f415e3c0 0x7fd8f40cc640 unknown :-1 s=CLOSED pgs=1362 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:52:10.979 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:52:10.977+0000 7fd8fbf2f640 1 --2- 172.21.15.43:0/2203112317 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fd8f415df60 0x7fd8f4153760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:52:10.979 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:52:10.977+0000 7fd8fbf2f640 1 --2- 172.21.15.43:0/2203112317 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fd8f40cb2d0 0x7fd8f414b6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:52:10.979 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:52:10.977+0000 7fd8fbf2f640 1 -- 172.21.15.43:0/2203112317 >> 172.21.15.43:0/2203112317 conn(0x7fd8f40aff50 msgr2=0x7fd8f4154780 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-09T13:52:10.979 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:52:10.977+0000 7fd8fbf2f640 1 -- 172.21.15.43:0/2203112317 shutdown_connections 2023-08-09T13:52:10.980 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:52:10.977+0000 7fd8fbf2f640 1 -- 172.21.15.43:0/2203112317 wait complete. 2023-08-09T13:52:10.998 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-08-09T13:52:11.233 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:52:11.231+0000 7f3f4e444640 1 Processor -- start 2023-08-09T13:52:11.233 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:52:11.232+0000 7f3f4e444640 1 -- start start 2023-08-09T13:52:11.234 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:52:11.233+0000 7f3f4e444640 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7f3f4815f7b0 0x7f3f480b4760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:52:11.235 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:52:11.233+0000 7f3f4e444640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f3f480b1050 0x7f3f480ccfa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:52:11.235 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:52:11.233+0000 7f3f4e444640 1 -- --> v1:172.21.15.43:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3f480cd930 con 0x7f3f480cd4e0 2023-08-09T13:52:11.235 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:52:11.233+0000 7f3f4e444640 1 -- --> v2:172.21.15.203:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3f480b7bf0 con 0x7f3f4815f7b0 2023-08-09T13:52:11.235 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:52:11.233+0000 7f3f4e444640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3f480b7ed0 con 0x7f3f480b1050 2023-08-09T13:52:11.237 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:52:11.233+0000 7f3f4dc43640 1 --1- >> v1:172.21.15.43:6789/0 conn(0x7f3f480cd4e0 0x7f3f480ceae0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.43:6789/0 says I am v1:172.21.15.43:38116/0 (socket says 172.21.15.43:38116) 2023-08-09T13:52:11.237 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:52:11.233+0000 7f3f4dc43640 1 -- 172.21.15.43:0/50152317 learned_addr learned my addr 172.21.15.43:0/50152317 (peer_addr_for_me v1:172.21.15.43:0/0) 2023-08-09T13:52:11.237 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:52:11.234+0000 7f3f4cc41640 1 --2- 172.21.15.43:0/50152317 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f3f480b1050 0x7f3f480ccfa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:52:11.237 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:52:11.234+0000 7f3f4d442640 1 --2- 172.21.15.43:0/50152317 >> v2:172.21.15.203:3300/0 conn(0x7f3f4815f7b0 0x7f3f480b4760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:52:11.238 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:52:11.234+0000 7f3f3ffff640 1 -- 172.21.15.43:0/50152317 <== mon.0 v1:172.21.15.43:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 777515567 0 0) 0x7f3f40001110 con 0x7f3f480cd4e0 2023-08-09T13:52:11.238 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:52:11.234+0000 7f3f3ffff640 1 -- 172.21.15.43:0/50152317 --> v1:172.21.15.43:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f3f34003520 con 0x7f3f480cd4e0 2023-08-09T13:52:11.238 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:52:11.234+0000 7f3f3ffff640 1 -- 172.21.15.43:0/50152317 <== mon.0 v1:172.21.15.43:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3537791624 0 0) 0x7f3f34003520 con 0x7f3f480cd4e0 2023-08-09T13:52:11.238 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:52:11.234+0000 7f3f3ffff640 1 -- 172.21.15.43:0/50152317 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f3f480b1050 msgr2=0x7f3f480ccfa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-09T13:52:11.238 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:52:11.234+0000 7f3f3ffff640 1 --2- 172.21.15.43:0/50152317 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f3f480b1050 0x7f3f480ccfa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:52:11.238 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:52:11.234+0000 7f3f3ffff640 1 -- 172.21.15.43:0/50152317 >> v2:172.21.15.203:3300/0 conn(0x7f3f4815f7b0 msgr2=0x7f3f480b4760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-09T13:52:11.238 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:52:11.234+0000 7f3f3ffff640 1 --2- 172.21.15.43:0/50152317 >> v2:172.21.15.203:3300/0 conn(0x7f3f4815f7b0 0x7f3f480b4760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:52:11.239 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:52:11.234+0000 7f3f3ffff640 1 -- 172.21.15.43:0/50152317 --> v1:172.21.15.43:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3f480d0370 con 0x7f3f480cd4e0 2023-08-09T13:52:11.239 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:52:11.234+0000 7f3f3ffff640 1 -- 172.21.15.43:0/50152317 <== mon.0 v1:172.21.15.43:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1285150740 0 0) 0x7f3f40001800 con 0x7f3f480cd4e0 2023-08-09T13:52:11.239 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:52:11.234+0000 7f3f3ffff640 1 -- 172.21.15.43:0/50152317 >> v1:172.21.15.43:6789/0 conn(0x7f3f480cd4e0 legacy=0x7f3f480ceae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:52:11.239 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:52:11.235+0000 7f3f3ffff640 1 --2- 172.21.15.43:0/50152317 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f3f34003970 0x7f3f34003d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:52:11.239 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:52:11.235+0000 7f3f3ffff640 1 --2- 172.21.15.43:0/50152317 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f3f340047c0 0x7f3f34004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:52:11.239 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:52:11.235+0000 7f3f3ffff640 1 --2- 172.21.15.43:0/50152317 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f3f480b1050 0x7f3f34005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:52:11.239 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:52:11.235+0000 7f3f3ffff640 1 -- 172.21.15.43:0/50152317 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3f480b7bf0 con 0x7f3f34003970 2023-08-09T13:52:11.239 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:52:11.235+0000 7f3f3ffff640 1 -- 172.21.15.43:0/50152317 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3f480b7ed0 con 0x7f3f340047c0 2023-08-09T13:52:11.240 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:52:11.235+0000 7f3f3ffff640 1 -- 172.21.15.43:0/50152317 --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3f3400b980 con 0x7f3f480b1050 2023-08-09T13:52:11.240 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:52:11.235+0000 7f3f4d442640 1 --2- 172.21.15.43:0/50152317 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f3f34003970 0x7f3f34003d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:52:11.240 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:52:11.235+0000 7f3f4dc43640 1 --2- 172.21.15.43:0/50152317 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f3f480b1050 0x7f3f34005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:52:11.240 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:52:11.235+0000 7f3f4cc41640 1 --2- 172.21.15.43:0/50152317 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f3f340047c0 0x7f3f34004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:52:11.240 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:52:11.235+0000 7f3f4d442640 1 -- 172.21.15.43:0/50152317 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f3f480b1050 msgr2=0x7f3f34005180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-09T13:52:11.240 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:52:11.235+0000 7f3f4d442640 1 --2- 172.21.15.43:0/50152317 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f3f480b1050 0x7f3f34005180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:52:11.240 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:52:11.236+0000 7f3f4d442640 1 -- 172.21.15.43:0/50152317 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f3f340047c0 msgr2=0x7f3f34004c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-09T13:52:11.241 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:52:11.236+0000 7f3f4d442640 1 --2- 172.21.15.43:0/50152317 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f3f340047c0 0x7f3f34004c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:52:11.241 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:52:11.236+0000 7f3f4d442640 1 -- 172.21.15.43:0/50152317 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3f34003520 con 0x7f3f34003970 2023-08-09T13:52:11.242 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:52:11.237+0000 7f3f4d442640 1 --2- 172.21.15.43:0/50152317 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f3f34003970 0x7f3f34003d90 secure :-1 s=READY pgs=1364 cs=0 l=1 rev1=1 crypto rx=0x7f3f44009fc0 tx=0x7f3f340057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dbdcfd5a2389fdad server_cookie=0 in_seq=0 out_seq=0 2023-08-09T13:52:11.242 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:52:11.237+0000 7f3f3ffff640 1 -- 172.21.15.43:0/50152317 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3f4402cc60 con 0x7f3f34003970 2023-08-09T13:52:11.242 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:52:11.237+0000 7f3f3ffff640 1 -- 172.21.15.43:0/50152317 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3f4400f040 con 0x7f3f34003970 2023-08-09T13:52:11.242 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:52:11.238+0000 7f3f3ffff640 1 -- 172.21.15.43:0/50152317 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3f44034440 con 0x7f3f34003970 2023-08-09T13:52:11.242 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:52:11.238+0000 7f3f4e444640 1 -- 172.21.15.43:0/50152317 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f3f34003970 msgr2=0x7f3f34003d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:52:11.242 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:52:11.238+0000 7f3f4e444640 1 --2- 172.21.15.43:0/50152317 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f3f34003970 0x7f3f34003d90 secure :-1 s=READY pgs=1364 cs=0 l=1 rev1=1 crypto rx=0x7f3f44009fc0 tx=0x7f3f340057f0 comp rx=0 tx=0).stop 2023-08-09T13:52:11.243 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:52:11.238+0000 7f3f4dc43640 1 -- 172.21.15.43:0/50152317 reap_dead start 2023-08-09T13:52:11.243 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:52:11.238+0000 7f3f4e444640 1 -- 172.21.15.43:0/50152317 shutdown_connections 2023-08-09T13:52:11.243 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:52:11.238+0000 7f3f4e444640 1 -- 172.21.15.43:0/50152317 >> 172.21.15.43:0/50152317 conn(0x7f3f4806a2b0 msgr2=0x7f3f4806a690 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-09T13:52:11.243 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:52:11.238+0000 7f3f4e444640 1 -- 172.21.15.43:0/50152317 shutdown_connections 2023-08-09T13:52:11.243 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:52:11.238+0000 7f3f4e444640 1 -- 172.21.15.43:0/50152317 wait complete. 2023-08-09T13:52:11.243 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:52:11.240+0000 7f3f4e444640 1 Processor -- start 2023-08-09T13:52:11.243 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:52:11.240+0000 7f3f4e444640 1 -- start start 2023-08-09T13:52:11.245 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:52:11.241+0000 7f3f4e444640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f3f34003970 0x7f3f481f13d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:52:11.245 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:52:11.241+0000 7f3f4e444640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f3f481f1910 0x7f3f480b2bb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:52:11.245 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:52:11.241+0000 7f3f4e444640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f3f480b30f0 0x7f3f48153a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:52:11.245 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:52:11.241+0000 7f3f4e444640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3f480b3570 con 0x7f3f481f1910 2023-08-09T13:52:11.245 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:52:11.241+0000 7f3f4e444640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3f481533e0 con 0x7f3f480b30f0 2023-08-09T13:52:11.246 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:52:11.241+0000 7f3f4e444640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3f481f1ea0 con 0x7f3f34003970 2023-08-09T13:52:11.246 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:52:11.242+0000 7f3f4cc41640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f3f481f1910 0x7f3f480b2bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:52:11.246 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:52:11.242+0000 7f3f4cc41640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f3f481f1910 0x7f3f480b2bb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:54376/0 (socket says 172.21.15.43:54376) 2023-08-09T13:52:11.246 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:52:11.242+0000 7f3f4cc41640 1 -- 172.21.15.43:0/1766827693 learned_addr learned my addr 172.21.15.43:0/1766827693 (peer_addr_for_me v2:172.21.15.43:0/0) 2023-08-09T13:52:11.246 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:52:11.242+0000 7f3f4d442640 1 --2- 172.21.15.43:0/1766827693 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f3f34003970 0x7f3f481f13d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:52:11.246 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:52:11.242+0000 7f3f4dc43640 1 --2- 172.21.15.43:0/1766827693 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f3f480b30f0 0x7f3f48153a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:52:11.246 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:52:11.242+0000 7f3f4cc41640 1 -- 172.21.15.43:0/1766827693 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f3f34003970 msgr2=0x7f3f481f13d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:52:11.247 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:52:11.242+0000 7f3f4cc41640 1 --2- 172.21.15.43:0/1766827693 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f3f34003970 0x7f3f481f13d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:52:11.247 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:52:11.242+0000 7f3f4cc41640 1 -- 172.21.15.43:0/1766827693 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f3f480b30f0 msgr2=0x7f3f48153a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:52:11.247 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:52:11.242+0000 7f3f4cc41640 1 --2- 172.21.15.43:0/1766827693 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f3f480b30f0 0x7f3f48153a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:52:11.247 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:52:11.242+0000 7f3f4cc41640 1 -- 172.21.15.43:0/1766827693 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3f481f2180 con 0x7f3f481f1910 2023-08-09T13:52:11.247 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:52:11.242+0000 7f3f4dc43640 1 --2- 172.21.15.43:0/1766827693 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f3f480b30f0 0x7f3f48153a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-08-09T13:52:11.247 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:52:11.242+0000 7f3f4cc41640 1 --2- 172.21.15.43:0/1766827693 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f3f481f1910 0x7f3f480b2bb0 secure :-1 s=READY pgs=1365 cs=0 l=1 rev1=1 crypto rx=0x7f3f38011600 tx=0x7f3f38011ac0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-09T13:52:11.247 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:52:11.243+0000 7f3f3dffb640 1 -- 172.21.15.43:0/1766827693 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3f3800f920 con 0x7f3f481f1910 2023-08-09T13:52:11.247 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:52:11.243+0000 7f3f3dffb640 1 -- 172.21.15.43:0/1766827693 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3f3800fa80 con 0x7f3f481f1910 2023-08-09T13:52:11.248 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:52:11.243+0000 7f3f3dffb640 1 -- 172.21.15.43:0/1766827693 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3f3800fdf0 con 0x7f3f481f1910 2023-08-09T13:52:11.248 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:52:11.243+0000 7f3f4e444640 1 -- 172.21.15.43:0/1766827693 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3f481f2460 con 0x7f3f481f1910 2023-08-09T13:52:11.248 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:52:11.243+0000 7f3f4e444640 1 -- 172.21.15.43:0/1766827693 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3f480ca730 con 0x7f3f481f1910 2023-08-09T13:52:11.249 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:52:11.244+0000 7f3f4e444640 1 -- 172.21.15.43:0/1766827693 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3f480739f0 con 0x7f3f481f1910 2023-08-09T13:52:11.249 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:52:11.245+0000 7f3f3dffb640 1 -- 172.21.15.43:0/1766827693 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 45) v1 ==== 88443+0+0 (secure 0 0 0) 0x7f3f38015460 con 0x7f3f481f1910 2023-08-09T13:52:11.249 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:52:11.247+0000 7f3f3dffb640 1 --2- 172.21.15.43:0/1766827693 >> [v2:172.21.15.203:6832/1317108840,v1:172.21.15.203:6833/1317108840] conn(0x7f3f2c06a170 0x7f3f2c06c630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:52:11.249 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:52:11.247+0000 7f3f3dffb640 1 -- 172.21.15.43:0/1766827693 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(116..116 src has 1..116) v4 ==== 6426+0+0 (secure 0 0 0) 0x7f3f38088db0 con 0x7f3f481f1910 2023-08-09T13:52:11.249 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:52:11.247+0000 7f3f4d442640 1 --2- 172.21.15.43:0/1766827693 >> [v2:172.21.15.203:6832/1317108840,v1:172.21.15.203:6833/1317108840] conn(0x7f3f2c06a170 0x7f3f2c06c630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:52:11.250 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:52:11.248+0000 7f3f4d442640 1 --2- 172.21.15.43:0/1766827693 >> [v2:172.21.15.203:6832/1317108840,v1:172.21.15.203:6833/1317108840] conn(0x7f3f2c06a170 0x7f3f2c06c630 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f3f44036000 tx=0x7f3f4402e000 comp rx=0 tx=0).ready entity=mgr.6619 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-09T13:52:11.252 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:52:11.251+0000 7f3f3dffb640 1 -- 172.21.15.43:0/1766827693 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178066 (secure 0 0 0) 0x7f3f3808d070 con 0x7f3f481f1910 2023-08-09T13:52:11.488 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:52:11.487+0000 7f3f4e444640 1 -- 172.21.15.43:0/1766827693 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f3f480cd0c0 con 0x7f3f481f1910 2023-08-09T13:52:11.489 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:52:11.488+0000 7f3f3dffb640 1 -- 172.21.15.43:0/1766827693 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 52 v52) v1 ==== 94+0+4883 (secure 0 0 0) 0x7f3f38054210 con 0x7f3f481f1910 2023-08-09T13:52:11.490 INFO:teuthology.orchestra.run.smithi043.stdout: 2023-08-09T13:52:11.490 INFO:teuthology.orchestra.run.smithi043.stdout:{"epoch":52,"default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6215,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.43:6835/2214346432","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6833","nonce":2214346432},{"type":"v1","addr":"172.21.15.43:6835","nonce":2214346432}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":47},{"gid":6823,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.203:6835/3236264045","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6834","nonce":3236264045},{"type":"v1","addr":"172.21.15.203:6835","nonce":3236264045}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":46},{"gid":6826,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.203:6837/3788289156","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6836","nonce":3788289156},{"type":"v1","addr":"172.21.15.203:6837","nonce":3788289156}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":47}],"filesystems":[{"mdsmap":{"epoch":52,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-08-09T13:51:59.655630+0000","modified":"2023-08-09T13:52:03.220527+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":8040},"failed":[],"damaged":[],"stopped":[],"info":{"gid_8040":{"gid":8040,"name":"c","rank":0,"incarnation":48,"state":"up:active","state_seq":3,"addr":"172.21.15.43:6834/2223580538","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6832","nonce":2223580538},{"type":"v1","addr":"172.21.15.43:6834","nonce":2223580538}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[15],"metadata_pool":14,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":6}]} 2023-08-09T13:52:11.490 INFO:teuthology.orchestra.run.smithi043.stderr:dumped fsmap epoch 52 2023-08-09T13:52:11.492 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:52:11.491+0000 7f3f4e444640 1 -- 172.21.15.43:0/1766827693 >> [v2:172.21.15.203:6832/1317108840,v1:172.21.15.203:6833/1317108840] conn(0x7f3f2c06a170 msgr2=0x7f3f2c06c630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:52:11.493 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:52:11.491+0000 7f3f4e444640 1 --2- 172.21.15.43:0/1766827693 >> [v2:172.21.15.203:6832/1317108840,v1:172.21.15.203:6833/1317108840] conn(0x7f3f2c06a170 0x7f3f2c06c630 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f3f44036000 tx=0x7f3f4402e000 comp rx=0 tx=0).stop 2023-08-09T13:52:11.493 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:52:11.491+0000 7f3f4e444640 1 -- 172.21.15.43:0/1766827693 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f3f481f1910 msgr2=0x7f3f480b2bb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:52:11.493 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:52:11.491+0000 7f3f4e444640 1 --2- 172.21.15.43:0/1766827693 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f3f481f1910 0x7f3f480b2bb0 secure :-1 s=READY pgs=1365 cs=0 l=1 rev1=1 crypto rx=0x7f3f38011600 tx=0x7f3f38011ac0 comp rx=0 tx=0).stop 2023-08-09T13:52:11.493 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:52:11.491+0000 7f3f4e444640 1 -- 172.21.15.43:0/1766827693 shutdown_connections 2023-08-09T13:52:11.493 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:52:11.491+0000 7f3f4e444640 1 --2- 172.21.15.43:0/1766827693 >> [v2:172.21.15.203:6832/1317108840,v1:172.21.15.203:6833/1317108840] conn(0x7f3f2c06a170 0x7f3f2c06c630 unknown :-1 s=CLOSED pgs=72 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:52:11.493 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:52:11.491+0000 7f3f4e444640 1 --2- 172.21.15.43:0/1766827693 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f3f480b30f0 0x7f3f48153a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:52:11.493 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:52:11.491+0000 7f3f4e444640 1 --2- 172.21.15.43:0/1766827693 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f3f481f1910 0x7f3f480b2bb0 unknown :-1 s=CLOSED pgs=1365 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:52:11.494 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:52:11.491+0000 7f3f4e444640 1 --2- 172.21.15.43:0/1766827693 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f3f34003970 0x7f3f481f13d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:52:11.494 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:52:11.491+0000 7f3f4e444640 1 -- 172.21.15.43:0/1766827693 >> 172.21.15.43:0/1766827693 conn(0x7f3f4806a2b0 msgr2=0x7f3f480b4630 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-09T13:52:11.494 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:52:11.491+0000 7f3f4e444640 1 -- 172.21.15.43:0/1766827693 shutdown_connections 2023-08-09T13:52:11.494 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:52:11.491+0000 7f3f4e444640 1 -- 172.21.15.43:0/1766827693 wait complete. 2023-08-09T13:52:11.916 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 52, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2023-08-09T13:51:59.655630+0000', 'modified': '2023-08-09T13:52:03.220527+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 8040}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_8040': {'gid': 8040, 'name': 'c', 'rank': 0, 'incarnation': 48, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.43:6834/2223580538', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.43:6832', 'nonce': 2223580538}, {'type': 'v1', 'addr': '172.21.15.43:6834', 'nonce': 2223580538}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138322906710015, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}}, 'data_pools': [15], 'metadata_pool': 14, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0} 2023-08-09T13:52:11.916 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2023-08-09T13:52:11.917 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.c status 2023-08-09T13:52:12.158 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:52:12.156+0000 7f98ab9c4640 1 Processor -- start 2023-08-09T13:52:12.158 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:52:12.156+0000 7f98ab9c4640 1 -- start start 2023-08-09T13:52:12.159 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:52:12.158+0000 7f98ab9c4640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f98a415c630 0x7f98a40abef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:52:12.160 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:52:12.158+0000 7f98ab9c4640 1 -- --> v1:172.21.15.43:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f98a40cca90 con 0x7f98a40ccf20 2023-08-09T13:52:12.160 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:52:12.158+0000 7f98ab9c4640 1 -- --> v1:172.21.15.203:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f98a40cbe80 con 0x7f98a40cdc60 2023-08-09T13:52:12.160 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:52:12.158+0000 7f98ab9c4640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f98a40cbc70 con 0x7f98a415c630 2023-08-09T13:52:12.162 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:52:12.158+0000 7f98a8f5f640 1 --1- >> v1:172.21.15.43:6789/0 conn(0x7f98a40ccf20 0x7f98a40cd550 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.43:6789/0 says I am v1:172.21.15.43:38124/0 (socket says 172.21.15.43:38124) 2023-08-09T13:52:12.162 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:52:12.158+0000 7f98a8f5f640 1 -- 172.21.15.43:0/3818692633 learned_addr learned my addr 172.21.15.43:0/3818692633 (peer_addr_for_me v1:172.21.15.43:0/0) 2023-08-09T13:52:12.162 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:52:12.159+0000 7f98a9760640 1 --2- 172.21.15.43:0/3818692633 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f98a415c630 0x7f98a40abef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:52:12.162 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:52:12.159+0000 7f989bfff640 1 -- 172.21.15.43:0/3818692633 <== mon.0 v1:172.21.15.43:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 881269703 0 0) 0x7f98a40cca90 con 0x7f98a40ccf20 2023-08-09T13:52:12.162 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:52:12.159+0000 7f989bfff640 1 -- 172.21.15.43:0/3818692633 --> v1:172.21.15.43:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f9890003520 con 0x7f98a40ccf20 2023-08-09T13:52:12.163 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:52:12.159+0000 7f989bfff640 1 -- 172.21.15.43:0/3818692633 <== mon.0 v1:172.21.15.43:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 146816245 0 0) 0x7f9890003520 con 0x7f98a40ccf20 2023-08-09T13:52:12.163 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:52:12.159+0000 7f989bfff640 1 -- 172.21.15.43:0/3818692633 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f98a415c630 msgr2=0x7f98a40abef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-09T13:52:12.163 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:52:12.159+0000 7f989bfff640 1 --2- 172.21.15.43:0/3818692633 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f98a415c630 0x7f98a40abef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:52:12.163 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:52:12.159+0000 7f989bfff640 1 -- 172.21.15.43:0/3818692633 >> v1:172.21.15.203:6789/0 conn(0x7f98a40cdc60 legacy=0x7f98a415ca10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:52:12.163 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:52:12.159+0000 7f989bfff640 1 -- 172.21.15.43:0/3818692633 --> v1:172.21.15.43:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f98a40cc160 con 0x7f98a40ccf20 2023-08-09T13:52:12.163 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:52:12.159+0000 7f989bfff640 1 -- 172.21.15.43:0/3818692633 <== mon.0 v1:172.21.15.43:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1285150740 0 0) 0x7f9894001670 con 0x7f98a40ccf20 2023-08-09T13:52:12.164 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:52:12.159+0000 7f989bfff640 1 -- 172.21.15.43:0/3818692633 >> v1:172.21.15.43:6789/0 conn(0x7f98a40ccf20 legacy=0x7f98a40cd550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:52:12.164 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:52:12.159+0000 7f989bfff640 1 --2- 172.21.15.43:0/3818692633 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f9890003a10 0x7f9890003e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:52:12.164 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:52:12.159+0000 7f989bfff640 1 --2- 172.21.15.43:0/3818692633 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f98a415c630 0x7f98900047c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:52:12.164 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:52:12.159+0000 7f989bfff640 1 --2- 172.21.15.43:0/3818692633 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f9890004d00 0x7f9890005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:52:12.164 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:52:12.159+0000 7f989bfff640 1 -- 172.21.15.43:0/3818692633 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f98a40cbc70 con 0x7f9890003a10 2023-08-09T13:52:12.164 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:52:12.159+0000 7f989bfff640 1 -- 172.21.15.43:0/3818692633 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f989000ba90 con 0x7f9890004d00 2023-08-09T13:52:12.164 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:52:12.159+0000 7f989bfff640 1 -- 172.21.15.43:0/3818692633 --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f989000bc50 con 0x7f98a415c630 2023-08-09T13:52:12.165 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:52:12.160+0000 7f98a9760640 1 --2- 172.21.15.43:0/3818692633 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f9890003a10 0x7f9890003e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:52:12.165 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:52:12.160+0000 7f98a8f5f640 1 --2- 172.21.15.43:0/3818692633 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f98a415c630 0x7f98900047c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:52:12.165 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:52:12.160+0000 7f98a9760640 1 -- 172.21.15.43:0/3818692633 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f98a415c630 msgr2=0x7f98900047c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-09T13:52:12.165 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:52:12.160+0000 7f98a9760640 1 --2- 172.21.15.43:0/3818692633 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f98a415c630 0x7f98900047c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:52:12.165 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:52:12.160+0000 7f98a9760640 1 -- 172.21.15.43:0/3818692633 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f9890004d00 msgr2=0x7f9890005180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-09T13:52:12.165 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:52:12.160+0000 7f98a9f61640 1 --2- 172.21.15.43:0/3818692633 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f9890004d00 0x7f9890005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:52:12.166 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:52:12.160+0000 7f98a9760640 1 --2- 172.21.15.43:0/3818692633 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f9890004d00 0x7f9890005180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:52:12.166 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:52:12.160+0000 7f98a9760640 1 -- 172.21.15.43:0/3818692633 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9890003520 con 0x7f9890003a10 2023-08-09T13:52:12.166 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:52:12.160+0000 7f98a8f5f640 1 --2- 172.21.15.43:0/3818692633 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f98a415c630 0x7f98900047c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-08-09T13:52:12.166 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:52:12.160+0000 7f98a9f61640 1 --2- 172.21.15.43:0/3818692633 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f9890004d00 0x7f9890005180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-08-09T13:52:12.170 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:52:12.162+0000 7f98a9760640 1 --2- 172.21.15.43:0/3818692633 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f9890003a10 0x7f9890003e30 secure :-1 s=READY pgs=1367 cs=0 l=1 rev1=1 crypto rx=0x7f98a0009a90 tx=0x7f98900057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bd48fa098f0e343e server_cookie=0 in_seq=0 out_seq=0 2023-08-09T13:52:12.170 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:52:12.162+0000 7f989bfff640 1 -- 172.21.15.43:0/3818692633 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f98a002c5c0 con 0x7f9890003a10 2023-08-09T13:52:12.171 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:52:12.162+0000 7f989bfff640 1 -- 172.21.15.43:0/3818692633 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f98a002c720 con 0x7f9890003a10 2023-08-09T13:52:12.171 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:52:12.162+0000 7f989bfff640 1 -- 172.21.15.43:0/3818692633 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f98a002cab0 con 0x7f9890003a10 2023-08-09T13:52:12.171 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:52:12.162+0000 7f98ab9c4640 1 -- 172.21.15.43:0/3818692633 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f9890003a10 msgr2=0x7f9890003e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:52:12.171 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:52:12.162+0000 7f98ab9c4640 1 --2- 172.21.15.43:0/3818692633 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f9890003a10 0x7f9890003e30 secure :-1 s=READY pgs=1367 cs=0 l=1 rev1=1 crypto rx=0x7f98a0009a90 tx=0x7f98900057f0 comp rx=0 tx=0).stop 2023-08-09T13:52:12.172 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:52:12.162+0000 7f98a9f61640 1 -- 172.21.15.43:0/3818692633 reap_dead start 2023-08-09T13:52:12.172 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:52:12.163+0000 7f98ab9c4640 1 -- 172.21.15.43:0/3818692633 shutdown_connections 2023-08-09T13:52:12.172 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:52:12.163+0000 7f98ab9c4640 1 -- 172.21.15.43:0/3818692633 >> 172.21.15.43:0/3818692633 conn(0x7f98a40ce0a0 msgr2=0x7f98a40ce480 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-09T13:52:12.173 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:52:12.163+0000 7f98ab9c4640 1 -- 172.21.15.43:0/3818692633 shutdown_connections 2023-08-09T13:52:12.173 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:52:12.163+0000 7f98ab9c4640 1 -- 172.21.15.43:0/3818692633 wait complete. 2023-08-09T13:52:12.173 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:52:12.165+0000 7f98ab9c4640 1 Processor -- start 2023-08-09T13:52:12.174 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:52:12.165+0000 7f98ab9c4640 1 -- start start 2023-08-09T13:52:12.174 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:52:12.166+0000 7f98ab9c4640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f9890003a10 0x7f98a41f6100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:52:12.174 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:52:12.166+0000 7f98ab9c4640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f98a40c6d10 0x7f98a4153d20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:52:12.174 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:52:12.166+0000 7f98ab9c4640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f98a40c58e0 0x7f98a40c5d60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:52:12.175 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:52:12.166+0000 7f98ab9c4640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f98a40c5170 con 0x7f98a40c58e0 2023-08-09T13:52:12.175 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:52:12.166+0000 7f98ab9c4640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f98a40c5450 con 0x7f9890003a10 2023-08-09T13:52:12.175 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:52:12.166+0000 7f98ab9c4640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f98a41f6640 con 0x7f98a40c6d10 2023-08-09T13:52:12.175 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:52:12.166+0000 7f98a9f61640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f98a40c58e0 0x7f98a40c5d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:52:12.176 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:52:12.166+0000 7f98a9f61640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f98a40c58e0 0x7f98a40c5d60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:54390/0 (socket says 172.21.15.43:54390) 2023-08-09T13:52:12.176 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:52:12.166+0000 7f98a9f61640 1 -- 172.21.15.43:0/1406389778 learned_addr learned my addr 172.21.15.43:0/1406389778 (peer_addr_for_me v2:172.21.15.43:0/0) 2023-08-09T13:52:12.176 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:52:12.166+0000 7f98a9760640 1 --2- 172.21.15.43:0/1406389778 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f9890003a10 0x7f98a41f6100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:52:12.177 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:52:12.166+0000 7f98a9f61640 1 -- 172.21.15.43:0/1406389778 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f98a40c6d10 msgr2=0x7f98a4153d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:52:12.177 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:52:12.166+0000 7f98a9f61640 1 --2- 172.21.15.43:0/1406389778 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f98a40c6d10 0x7f98a4153d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:52:12.177 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:52:12.166+0000 7f98a9f61640 1 -- 172.21.15.43:0/1406389778 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f9890003a10 msgr2=0x7f98a41f6100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:52:12.178 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:52:12.166+0000 7f98a9f61640 1 --2- 172.21.15.43:0/1406389778 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f9890003a10 0x7f98a41f6100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:52:12.178 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:52:12.167+0000 7f98a9f61640 1 -- 172.21.15.43:0/1406389778 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f98a4154260 con 0x7f98a40c58e0 2023-08-09T13:52:12.178 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:52:12.167+0000 7f98a9f61640 1 --2- 172.21.15.43:0/1406389778 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f98a40c58e0 0x7f98a40c5d60 secure :-1 s=READY pgs=1368 cs=0 l=1 rev1=1 crypto rx=0x7f98a406d9d0 tx=0x7f989c00d9d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-09T13:52:12.179 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:52:12.167+0000 7f9899ffb640 1 -- 172.21.15.43:0/1406389778 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f989c003970 con 0x7f98a40c58e0 2023-08-09T13:52:12.179 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:52:12.167+0000 7f9899ffb640 1 -- 172.21.15.43:0/1406389778 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f989c003af0 con 0x7f98a40c58e0 2023-08-09T13:52:12.179 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:52:12.167+0000 7f98ab9c4640 1 -- 172.21.15.43:0/1406389778 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f98a40c7240 con 0x7f98a40c58e0 2023-08-09T13:52:12.179 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:52:12.167+0000 7f9899ffb640 1 -- 172.21.15.43:0/1406389778 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f989c003ea0 con 0x7f98a40c58e0 2023-08-09T13:52:12.179 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:52:12.167+0000 7f98ab9c4640 1 -- 172.21.15.43:0/1406389778 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f98a40c7800 con 0x7f98a40c58e0 2023-08-09T13:52:12.185 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:52:12.170+0000 7f9899ffb640 1 -- 172.21.15.43:0/1406389778 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 45) v1 ==== 88443+0+0 (secure 0 0 0) 0x7f989c005070 con 0x7f98a40c58e0 2023-08-09T13:52:12.185 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:52:12.171+0000 7f9899ffb640 1 --2- 172.21.15.43:0/1406389778 >> [v2:172.21.15.203:6832/1317108840,v1:172.21.15.203:6833/1317108840] conn(0x7f988806a1a0 0x7f988806c660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:52:12.185 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:52:12.171+0000 7f9899ffb640 1 -- 172.21.15.43:0/1406389778 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(116..116 src has 1..116) v4 ==== 6426+0+0 (secure 0 0 0) 0x7f989c088590 con 0x7f98a40c58e0 2023-08-09T13:52:12.185 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:52:12.171+0000 7f98ae608740 1 Processor -- start 2023-08-09T13:52:12.186 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:52:12.171+0000 7f98ae608740 1 -- start start 2023-08-09T13:52:12.186 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:52:12.171+0000 7f98a9760640 1 --2- 172.21.15.43:0/1406389778 >> [v2:172.21.15.203:6832/1317108840,v1:172.21.15.203:6833/1317108840] conn(0x7f988806a1a0 0x7f988806c660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:52:12.186 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:52:12.172+0000 7f98ae608740 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x55f5132b2c50 0x55f5132b3030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:52:12.186 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:52:12.172+0000 7f98ae608740 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55f513282bd0 0x55f5132afec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:52:12.186 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:52:12.172+0000 7f98ae608740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55f5132867d0 0x55f5132b0400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:52:12.186 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:52:12.172+0000 7f98ae608740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f5131f48f0 con 0x55f5132867d0 2023-08-09T13:52:12.186 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:52:12.172+0000 7f98ae608740 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f5131f1fe0 con 0x55f513282bd0 2023-08-09T13:52:12.187 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:52:12.172+0000 7f98ae608740 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x55f5131ed8d0 con 0x55f5132b2c50 2023-08-09T13:52:12.187 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:52:12.172+0000 7f98a9f61640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55f5132867d0 0x55f5132b0400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:52:12.187 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:52:12.173+0000 7f98a9f61640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55f5132867d0 0x55f5132b0400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:54406/0 (socket says 172.21.15.43:54406) 2023-08-09T13:52:12.187 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:52:12.173+0000 7f98a9f61640 1 -- 172.21.15.43:0/3562231344 learned_addr learned my addr 172.21.15.43:0/3562231344 (peer_addr_for_me v2:172.21.15.43:0/0) 2023-08-09T13:52:12.187 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:52:12.173+0000 7f98a9760640 1 --2- 172.21.15.43:0/1406389778 >> [v2:172.21.15.203:6832/1317108840,v1:172.21.15.203:6833/1317108840] conn(0x7f988806a1a0 0x7f988806c660 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f98a0036000 tx=0x7f98a0030000 comp rx=0 tx=0).ready entity=mgr.6619 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-09T13:52:12.187 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:52:12.173+0000 7f98a8f5f640 1 --2- 172.21.15.43:0/3562231344 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55f513282bd0 0x55f5132afec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:52:12.187 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:52:12.173+0000 7f98a9760640 1 --2- 172.21.15.43:0/3562231344 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x55f5132b2c50 0x55f5132b3030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:52:12.187 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:52:12.173+0000 7f98a9f61640 1 -- 172.21.15.43:0/3562231344 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x55f5132b2c50 msgr2=0x55f5132b3030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-09T13:52:12.188 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:52:12.173+0000 7f98a9f61640 1 --2- 172.21.15.43:0/3562231344 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x55f5132b2c50 0x55f5132b3030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:52:12.188 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:52:12.173+0000 7f98a9f61640 1 -- 172.21.15.43:0/3562231344 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55f513282bd0 msgr2=0x55f5132afec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-09T13:52:12.188 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:52:12.173+0000 7f98a9f61640 1 --2- 172.21.15.43:0/3562231344 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55f513282bd0 0x55f5132afec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:52:12.188 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:52:12.173+0000 7f98a9f61640 1 -- 172.21.15.43:0/3562231344 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55f5131a6ac0 con 0x55f5132867d0 2023-08-09T13:52:12.188 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:52:12.173+0000 7f98a8f5f640 1 --2- 172.21.15.43:0/3562231344 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55f513282bd0 0x55f5132afec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-08-09T13:52:12.188 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:52:12.173+0000 7f98a9760640 1 --2- 172.21.15.43:0/3562231344 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x55f5132b2c50 0x55f5132b3030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-08-09T13:52:12.189 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:52:12.173+0000 7f98a9f61640 1 --2- 172.21.15.43:0/3562231344 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55f5132867d0 0x55f5132b0400 secure :-1 s=READY pgs=1369 cs=0 l=1 rev1=1 crypto rx=0x7f989c00ded0 tx=0x7f989c011ed0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=90780c91b124764a server_cookie=0 in_seq=0 out_seq=0 2023-08-09T13:52:12.189 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:52:12.173+0000 7f988effd640 1 -- 172.21.15.43:0/3562231344 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f989c05f2b0 con 0x55f5132867d0 2023-08-09T13:52:12.189 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:52:12.174+0000 7f988effd640 1 -- 172.21.15.43:0/3562231344 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f989c055940 con 0x55f5132867d0 2023-08-09T13:52:12.189 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:52:12.174+0000 7f988effd640 1 -- 172.21.15.43:0/3562231344 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f989c053cb0 con 0x55f5132867d0 2023-08-09T13:52:12.189 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:52:12.174+0000 7f98ae608740 1 -- 172.21.15.43:0/3562231344 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55f5132867d0 msgr2=0x55f5132b0400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:52:12.190 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:52:12.174+0000 7f98ae608740 1 --2- 172.21.15.43:0/3562231344 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55f5132867d0 0x55f5132b0400 secure :-1 s=READY pgs=1369 cs=0 l=1 rev1=1 crypto rx=0x7f989c00ded0 tx=0x7f989c011ed0 comp rx=0 tx=0).stop 2023-08-09T13:52:12.190 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:52:12.174+0000 7f98ae608740 1 -- 172.21.15.43:0/3562231344 shutdown_connections 2023-08-09T13:52:12.190 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:52:12.174+0000 7f98ae608740 1 --2- 172.21.15.43:0/3562231344 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55f5132867d0 0x55f5132b0400 unknown :-1 s=CLOSED pgs=1369 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:52:12.190 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:52:12.174+0000 7f98ae608740 1 --2- 172.21.15.43:0/3562231344 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55f513282bd0 0x55f5132afec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:52:12.190 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:52:12.174+0000 7f98ae608740 1 --2- 172.21.15.43:0/3562231344 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x55f5132b2c50 0x55f5132b3030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:52:12.190 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:52:12.174+0000 7f98ae608740 1 -- 172.21.15.43:0/3562231344 >> 172.21.15.43:0/3562231344 conn(0x55f513284ee0 msgr2=0x55f5132cda60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-09T13:52:12.190 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:52:12.174+0000 7f98ae608740 1 -- 172.21.15.43:0/3562231344 shutdown_connections 2023-08-09T13:52:12.191 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:52:12.174+0000 7f98ae608740 1 -- 172.21.15.43:0/3562231344 wait complete. 2023-08-09T13:52:12.191 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:52:12.175+0000 7f98ae608740 1 -- start start 2023-08-09T13:52:12.191 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:52:12.175+0000 7f98ae608740 1 Processor -- start 2023-08-09T13:52:12.191 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:52:12.176+0000 7f98ae608740 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55f5132867d0 0x55f5132a88b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:52:12.191 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:52:12.176+0000 7f98ae608740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55f5132b2c50 0x55f5132aa4d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:52:12.191 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:52:12.176+0000 7f98ae608740 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x55f5132cdfe0 0x55f5132aaa10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:52:12.191 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:52:12.176+0000 7f98ae608740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f51318dd40 con 0x55f5132b2c50 2023-08-09T13:52:12.191 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:52:12.176+0000 7f98ae608740 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f51318d9e0 con 0x55f5132867d0 2023-08-09T13:52:12.192 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:52:12.176+0000 7f98ae608740 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x55f512f53f20 con 0x55f5132cdfe0 2023-08-09T13:52:12.192 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:52:12.176+0000 7f98ae608740 10 client.0 _ll_register_callbacks cb 0x55f5131a6160 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-08-09T13:52:12.192 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:52:12.177+0000 7f98a8f5f640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55f5132b2c50 0x55f5132aa4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:52:12.192 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:52:12.177+0000 7f98a8f5f640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55f5132b2c50 0x55f5132aa4d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:54410/0 (socket says 172.21.15.43:54410) 2023-08-09T13:52:12.192 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:52:12.177+0000 7f98a8f5f640 1 -- 172.21.15.43:0/2081588595 learned_addr learned my addr 172.21.15.43:0/2081588595 (peer_addr_for_me v2:172.21.15.43:0/0) 2023-08-09T13:52:12.192 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:52:12.177+0000 7f98a9f61640 1 --2- 172.21.15.43:0/2081588595 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x55f5132cdfe0 0x55f5132aaa10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:52:12.192 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:52:12.177+0000 7f98a9760640 1 --2- 172.21.15.43:0/2081588595 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55f5132867d0 0x55f5132a88b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:52:12.193 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:52:12.177+0000 7f98a8f5f640 1 -- 172.21.15.43:0/2081588595 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x55f5132cdfe0 msgr2=0x55f5132aaa10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-09T13:52:12.193 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:52:12.177+0000 7f98a8f5f640 1 --2- 172.21.15.43:0/2081588595 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x55f5132cdfe0 0x55f5132aaa10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:52:12.193 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:52:12.177+0000 7f98a8f5f640 1 -- 172.21.15.43:0/2081588595 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55f5132867d0 msgr2=0x55f5132a88b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-09T13:52:12.193 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:52:12.177+0000 7f98a8f5f640 1 --2- 172.21.15.43:0/2081588595 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55f5132867d0 0x55f5132a88b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:52:12.193 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:52:12.177+0000 7f98a8f5f640 1 -- 172.21.15.43:0/2081588595 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f98a40c7800 con 0x55f5132b2c50 2023-08-09T13:52:12.193 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:52:12.177+0000 7f98a9760640 1 --2- 172.21.15.43:0/2081588595 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55f5132867d0 0x55f5132a88b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-08-09T13:52:12.193 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:52:12.177+0000 7f98a8f5f640 1 --2- 172.21.15.43:0/2081588595 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55f5132b2c50 0x55f5132aa4d0 secure :-1 s=READY pgs=1370 cs=0 l=1 rev1=1 crypto rx=0x7f989400e240 tx=0x7f989400e700 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1bd0c2f55e1c59a6 server_cookie=0 in_seq=0 out_seq=0 2023-08-09T13:52:12.193 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:52:12.177+0000 7f9883fff640 10 client.0 ms_handle_connect on v2:172.21.15.43:3300/0 2023-08-09T13:52:12.194 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:52:12.178+0000 7f9883fff640 1 -- 172.21.15.43:0/2081588595 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f98940074e0 con 0x55f5132b2c50 2023-08-09T13:52:12.194 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:52:12.178+0000 7f9883fff640 1 -- 172.21.15.43:0/2081588595 <== mon.0 v2:172.21.15.43:3300/0 2 ==== osd_map(116..116 src has 1..116) v4 ==== 6426+0+0 (secure 0 0 0) 0x7f9894007640 con 0x55f5132b2c50 2023-08-09T13:52:12.194 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:52:12.178+0000 7f98ae608740 1 -- 172.21.15.43:0/2081588595 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55f5131c5500 con 0x55f5132b2c50 2023-08-09T13:52:12.194 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:52:12.178+0000 7f9883fff640 1 -- 172.21.15.43:0/2081588595 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_get_version_reply(handle=1 version=52) v2 ==== 24+0+0 (secure 0 0 0) 0x7f989400eed0 con 0x55f5132b2c50 2023-08-09T13:52:12.194 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:52:12.178+0000 7f98ae608740 10 client.8211 fetch_fsmap learned FSMap version 52 2023-08-09T13:52:12.194 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:52:12.178+0000 7f98ae608740 1 -- 172.21.15.43:0/2081588595 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,fsmap=52,monmap=2+}) v3 -- 0x55f5131ebf50 con 0x55f5132b2c50 2023-08-09T13:52:12.195 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:52:12.178+0000 7f9883fff640 1 -- 172.21.15.43:0/2081588595 <== mon.0 v2:172.21.15.43:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x55f5131c5500 con 0x55f5132b2c50 2023-08-09T13:52:12.195 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:52:12.178+0000 7f9883fff640 1 -- 172.21.15.43:0/2081588595 <== mon.0 v2:172.21.15.43:3300/0 5 ==== fsmap(e 52) v1 ==== 3294+0+0 (secure 0 0 0) 0x7f98940047e0 con 0x55f5132b2c50 2023-08-09T13:52:12.195 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:52:12.178+0000 7f98ae608740 10 client.8211 fetch_fsmap finished waiting for FSMap version 52 2023-08-09T13:52:12.195 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:52:12.179+0000 7f98ae608740 10 client.8211 resolve_mds: resolved name 'c' to daemon mds.c 2023-08-09T13:52:12.195 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:52:12.179+0000 7f98ae608740 20 client.8211 populate_metadata read hostname 'smithi043' 2023-08-09T13:52:12.195 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:52:12.179+0000 7f98ae608740 1 --2- 172.21.15.43:0/2081588595 >> [v2:172.21.15.43:6832/2223580538,v1:172.21.15.43:6834/2223580538] conn(0x55f5132b44d0 0x55f5132b48b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:52:12.195 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:52:12.179+0000 7f98ae608740 4 client.8211 mds_command: new command op to 8040 tid=0[{"prefix": "get_command_descriptions"}] 2023-08-09T13:52:12.196 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:52:12.179+0000 7f98ae608740 1 -- 172.21.15.43:0/2081588595 --> [v2:172.21.15.43:6832/2223580538,v1:172.21.15.43:6834/2223580538] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55f5131c2bc0 con 0x55f5132b44d0 2023-08-09T13:52:12.196 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:52:12.179+0000 7f98a9760640 1 --2- 172.21.15.43:0/2081588595 >> [v2:172.21.15.43:6832/2223580538,v1:172.21.15.43:6834/2223580538] conn(0x55f5132b44d0 0x55f5132b48b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:52:12.196 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:52:12.179+0000 7f98a9760640 1 --2- 172.21.15.43:0/2081588595 >> [v2:172.21.15.43:6832/2223580538,v1:172.21.15.43:6834/2223580538] conn(0x55f5132b44d0 0x55f5132b48b0 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=35547421a1da426 server_cookie=8b9a61f05ac0bbc3 in_seq=0 out_seq=0 2023-08-09T13:52:12.196 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:52:12.179+0000 7f9883fff640 10 client.8211 ms_handle_connect on v2:172.21.15.43:6832/2223580538 2023-08-09T13:52:12.196 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:52:12.180+0000 7f9883fff640 1 -- 172.21.15.43:0/2081588595 <== mds.0 v2:172.21.15.43:6832/2223580538 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18017 (crc 0 0 0) 0x55f5131c2bc0 con 0x55f5132b44d0 2023-08-09T13:52:12.196 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:52:12.180+0000 7f9883fff640 10 client.8211 handle_command_reply: tid=0 2023-08-09T13:52:12.196 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:52:12.180+0000 7f98ae608740 1 client.8211 shutdown 2023-08-09T13:52:12.197 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:52:12.181+0000 7f98ae608740 1 -- 172.21.15.43:0/2081588595 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55f5132b2c50 msgr2=0x55f5132aa4d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:52:12.197 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:52:12.181+0000 7f98ae608740 1 --2- 172.21.15.43:0/2081588595 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55f5132b2c50 0x55f5132aa4d0 secure :-1 s=READY pgs=1370 cs=0 l=1 rev1=1 crypto rx=0x7f989400e240 tx=0x7f989400e700 comp rx=0 tx=0).stop 2023-08-09T13:52:12.197 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:52:12.181+0000 7f98ae608740 1 -- 172.21.15.43:0/2081588595 shutdown_connections 2023-08-09T13:52:12.197 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:52:12.181+0000 7f98ae608740 1 --2- 172.21.15.43:0/2081588595 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x55f5132cdfe0 0x55f5132aaa10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:52:12.197 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:52:12.181+0000 7f98ae608740 1 --2- 172.21.15.43:0/2081588595 >> [v2:172.21.15.43:6832/2223580538,v1:172.21.15.43:6834/2223580538] conn(0x55f5132b44d0 0x55f5132b48b0 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:52:12.197 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:52:12.181+0000 7f98ae608740 1 --2- 172.21.15.43:0/2081588595 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55f5132b2c50 0x55f5132aa4d0 unknown :-1 s=CLOSED pgs=1370 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:52:12.197 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:52:12.181+0000 7f98ae608740 1 --2- 172.21.15.43:0/2081588595 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55f5132867d0 0x55f5132a88b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:52:12.198 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:52:12.181+0000 7f9883fff640 0 client.8211 ms_handle_reset on v2:172.21.15.43:6832/2223580538 2023-08-09T13:52:12.198 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:52:12.181+0000 7f98ae608740 1 -- 172.21.15.43:0/2081588595 >> 172.21.15.43:0/2081588595 conn(0x55f513282bd0 msgr2=0x55f5132b6960 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-09T13:52:12.198 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:52:12.181+0000 7f98ae608740 1 -- 172.21.15.43:0/2081588595 shutdown_connections 2023-08-09T13:52:12.198 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:52:12.181+0000 7f98ae608740 1 -- 172.21.15.43:0/2081588595 wait complete. 2023-08-09T13:52:12.198 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:52:12.181+0000 7f98ae608740 20 client.8211 trim_cache size 0 max 16384 2023-08-09T13:52:12.200 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:52:12.198+0000 7f98ae608740 1 Processor -- start 2023-08-09T13:52:12.200 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:52:12.198+0000 7f98ae608740 1 -- start start 2023-08-09T13:52:12.201 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:52:12.199+0000 7f98ae608740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55f5132867d0 0x55f5132a75b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:52:12.201 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:52:12.199+0000 7f98ae608740 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55f5132b2c50 0x55f51327e4a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:52:12.201 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:52:12.199+0000 7f98ae608740 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x55f5132cdfe0 0x55f5132afaf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:52:12.201 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:52:12.199+0000 7f98ae608740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f5132a95a0 con 0x55f5132867d0 2023-08-09T13:52:12.201 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:52:12.199+0000 7f98ae608740 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f5132a9270 con 0x55f5132b2c50 2023-08-09T13:52:12.201 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:52:12.199+0000 7f98ae608740 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x55f51321f150 con 0x55f5132cdfe0 2023-08-09T13:52:12.202 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:52:12.199+0000 7f98a9760640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55f5132867d0 0x55f5132a75b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:52:12.202 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:52:12.199+0000 7f98a9760640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55f5132867d0 0x55f5132a75b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:54420/0 (socket says 172.21.15.43:54420) 2023-08-09T13:52:12.202 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:52:12.199+0000 7f98a9760640 1 -- 172.21.15.43:0/1483531070 learned_addr learned my addr 172.21.15.43:0/1483531070 (peer_addr_for_me v2:172.21.15.43:0/0) 2023-08-09T13:52:12.203 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:52:12.200+0000 7f98a9f61640 1 --2- 172.21.15.43:0/1483531070 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x55f5132cdfe0 0x55f5132afaf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:52:12.203 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:52:12.200+0000 7f98a8f5f640 1 --2- 172.21.15.43:0/1483531070 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55f5132b2c50 0x55f51327e4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:52:12.204 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:52:12.200+0000 7f98a9760640 1 -- 172.21.15.43:0/1483531070 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x55f5132cdfe0 msgr2=0x55f5132afaf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-09T13:52:12.204 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:52:12.200+0000 7f98a9760640 1 --2- 172.21.15.43:0/1483531070 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x55f5132cdfe0 0x55f5132afaf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:52:12.204 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:52:12.200+0000 7f98a9760640 1 -- 172.21.15.43:0/1483531070 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55f5132b2c50 msgr2=0x55f51327e4a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-09T13:52:12.204 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:52:12.200+0000 7f98a9760640 1 --2- 172.21.15.43:0/1483531070 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55f5132b2c50 0x55f51327e4a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:52:12.204 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:52:12.200+0000 7f98a9760640 1 -- 172.21.15.43:0/1483531070 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55f51321f300 con 0x55f5132867d0 2023-08-09T13:52:12.204 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:52:12.200+0000 7f98a8f5f640 1 --2- 172.21.15.43:0/1483531070 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55f5132b2c50 0x55f51327e4a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-08-09T13:52:12.204 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:52:12.201+0000 7f98a9760640 1 --2- 172.21.15.43:0/1483531070 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55f5132867d0 0x55f5132a75b0 secure :-1 s=READY pgs=1371 cs=0 l=1 rev1=1 crypto rx=0x7f98a0004450 tx=0x7f98a002e000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=65ddcd2c721cc519 server_cookie=0 in_seq=0 out_seq=0 2023-08-09T13:52:12.205 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:52:12.201+0000 7f988f7fe640 1 -- 172.21.15.43:0/1483531070 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f98a002ce60 con 0x55f5132867d0 2023-08-09T13:52:12.205 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:52:12.201+0000 7f988f7fe640 1 -- 172.21.15.43:0/1483531070 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f98a0003610 con 0x55f5132867d0 2023-08-09T13:52:12.205 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:52:12.201+0000 7f988f7fe640 1 -- 172.21.15.43:0/1483531070 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f98a0003980 con 0x55f5132867d0 2023-08-09T13:52:12.205 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:52:12.201+0000 7f98ae608740 1 -- 172.21.15.43:0/1483531070 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55f5132867d0 msgr2=0x55f5132a75b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:52:12.205 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:52:12.201+0000 7f98ae608740 1 --2- 172.21.15.43:0/1483531070 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55f5132867d0 0x55f5132a75b0 secure :-1 s=READY pgs=1371 cs=0 l=1 rev1=1 crypto rx=0x7f98a0004450 tx=0x7f98a002e000 comp rx=0 tx=0).stop 2023-08-09T13:52:12.205 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:52:12.202+0000 7f98ae608740 1 -- 172.21.15.43:0/1483531070 shutdown_connections 2023-08-09T13:52:12.205 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:52:12.202+0000 7f98ae608740 1 --2- 172.21.15.43:0/1483531070 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x55f5132cdfe0 0x55f5132afaf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:52:12.205 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:52:12.202+0000 7f98ae608740 1 --2- 172.21.15.43:0/1483531070 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55f5132b2c50 0x55f51327e4a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:52:12.206 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:52:12.202+0000 7f98ae608740 1 --2- 172.21.15.43:0/1483531070 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55f5132867d0 0x55f5132a75b0 unknown :-1 s=CLOSED pgs=1371 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:52:12.206 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:52:12.202+0000 7f98ae608740 1 -- 172.21.15.43:0/1483531070 >> 172.21.15.43:0/1483531070 conn(0x55f513282bd0 msgr2=0x55f513288b40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-09T13:52:12.206 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:52:12.202+0000 7f98ae608740 1 -- 172.21.15.43:0/1483531070 shutdown_connections 2023-08-09T13:52:12.206 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:52:12.202+0000 7f98ae608740 1 -- 172.21.15.43:0/1483531070 wait complete. 2023-08-09T13:52:12.209 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:52:12.202+0000 7f98ae608740 1 -- start start 2023-08-09T13:52:12.210 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:52:12.203+0000 7f98ae608740 1 Processor -- start 2023-08-09T13:52:12.210 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:52:12.204+0000 7f98ae608740 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55f5132867d0 0x55f5132eaaf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:52:12.210 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:52:12.204+0000 7f98ae608740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55f5132b2c50 0x55f5132aa470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:52:12.210 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:52:12.204+0000 7f98ae608740 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x55f5132cdfe0 0x55f5132aa9b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:52:12.210 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:52:12.204+0000 7f98ae608740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f5132dc130 con 0x55f5132b2c50 2023-08-09T13:52:12.211 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:52:12.204+0000 7f98ae608740 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f5132eb030 con 0x55f5132867d0 2023-08-09T13:52:12.211 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:52:12.204+0000 7f98ae608740 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x55f513337f10 con 0x55f5132cdfe0 2023-08-09T13:52:12.211 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:52:12.204+0000 7f98ae608740 10 client.0 _ll_register_callbacks cb 0x7f9894001c80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-08-09T13:52:12.211 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:52:12.204+0000 7f98a8f5f640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55f5132b2c50 0x55f5132aa470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:52:12.211 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:52:12.204+0000 7f98a8f5f640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55f5132b2c50 0x55f5132aa470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:54426/0 (socket says 172.21.15.43:54426) 2023-08-09T13:52:12.211 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:52:12.204+0000 7f98a8f5f640 1 -- 172.21.15.43:0/3484253024 learned_addr learned my addr 172.21.15.43:0/3484253024 (peer_addr_for_me v2:172.21.15.43:0/0) 2023-08-09T13:52:12.211 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:52:12.204+0000 7f98a9760640 1 --2- 172.21.15.43:0/3484253024 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55f5132867d0 0x55f5132eaaf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:52:12.212 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:52:12.204+0000 7f98a9f61640 1 --2- 172.21.15.43:0/3484253024 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x55f5132cdfe0 0x55f5132aa9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:52:12.212 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:52:12.204+0000 7f98a8f5f640 1 -- 172.21.15.43:0/3484253024 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x55f5132cdfe0 msgr2=0x55f5132aa9b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-09T13:52:12.212 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:52:12.204+0000 7f98a8f5f640 1 --2- 172.21.15.43:0/3484253024 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x55f5132cdfe0 0x55f5132aa9b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:52:12.212 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:52:12.204+0000 7f98a8f5f640 1 -- 172.21.15.43:0/3484253024 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55f5132867d0 msgr2=0x55f5132eaaf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-09T13:52:12.212 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:52:12.204+0000 7f98a8f5f640 1 --2- 172.21.15.43:0/3484253024 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55f5132867d0 0x55f5132eaaf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:52:12.212 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:52:12.204+0000 7f98a8f5f640 1 -- 172.21.15.43:0/3484253024 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9890003520 con 0x55f5132b2c50 2023-08-09T13:52:12.213 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:52:12.205+0000 7f98a8f5f640 1 --2- 172.21.15.43:0/3484253024 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55f5132b2c50 0x55f5132aa470 secure :-1 s=READY pgs=1372 cs=0 l=1 rev1=1 crypto rx=0x7f9894001570 tx=0x7f9894009040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=40f8f427ace81329 server_cookie=0 in_seq=0 out_seq=0 2023-08-09T13:52:12.213 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:52:12.205+0000 7f9883fff640 10 client.0 ms_handle_connect on v2:172.21.15.43:3300/0 2023-08-09T13:52:12.213 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:52:12.205+0000 7f9883fff640 1 -- 172.21.15.43:0/3484253024 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f989400ec00 con 0x55f5132b2c50 2023-08-09T13:52:12.213 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:52:12.205+0000 7f9883fff640 1 -- 172.21.15.43:0/3484253024 <== mon.0 v2:172.21.15.43:3300/0 2 ==== osd_map(116..116 src has 1..116) v4 ==== 6426+0+0 (secure 0 0 0) 0x7f98940072e0 con 0x55f5132b2c50 2023-08-09T13:52:12.213 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:52:12.205+0000 7f98ae608740 1 -- 172.21.15.43:0/3484253024 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55f513215800 con 0x55f5132b2c50 2023-08-09T13:52:12.213 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:52:12.205+0000 7f9883fff640 1 -- 172.21.15.43:0/3484253024 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_get_version_reply(handle=1 version=52) v2 ==== 24+0+0 (secure 0 0 0) 0x7f98940074a0 con 0x55f5132b2c50 2023-08-09T13:52:12.213 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:52:12.205+0000 7f98ae608740 10 client.8217 fetch_fsmap learned FSMap version 52 2023-08-09T13:52:12.214 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:52:12.205+0000 7f98ae608740 1 -- 172.21.15.43:0/3484253024 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,fsmap=52,monmap=2+}) v3 -- 0x55f51318ca10 con 0x55f5132b2c50 2023-08-09T13:52:12.214 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:52:12.206+0000 7f9883fff640 1 -- 172.21.15.43:0/3484253024 <== mon.0 v2:172.21.15.43:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x55f513215800 con 0x55f5132b2c50 2023-08-09T13:52:12.214 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:52:12.206+0000 7f9883fff640 1 -- 172.21.15.43:0/3484253024 <== mon.0 v2:172.21.15.43:3300/0 5 ==== fsmap(e 52) v1 ==== 3294+0+0 (secure 0 0 0) 0x7f9894026070 con 0x55f5132b2c50 2023-08-09T13:52:12.214 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:52:12.206+0000 7f98ae608740 10 client.8217 fetch_fsmap finished waiting for FSMap version 52 2023-08-09T13:52:12.214 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:52:12.206+0000 7f98ae608740 10 client.8217 resolve_mds: resolved name 'c' to daemon mds.c 2023-08-09T13:52:12.214 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:52:12.206+0000 7f98ae608740 20 client.8217 populate_metadata read hostname 'smithi043' 2023-08-09T13:52:12.214 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:52:12.206+0000 7f98ae608740 1 --2- 172.21.15.43:0/3484253024 >> [v2:172.21.15.43:6832/2223580538,v1:172.21.15.43:6834/2223580538] conn(0x55f513287fc0 0x55f5132883a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:52:12.215 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:52:12.206+0000 7f98ae608740 4 client.8217 mds_command: new command op to 8040 tid=0[{"prefix": "status"}] 2023-08-09T13:52:12.215 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:52:12.206+0000 7f98ae608740 1 -- 172.21.15.43:0/3484253024 --> [v2:172.21.15.43:6832/2223580538,v1:172.21.15.43:6834/2223580538] -- command(tid 0: {"prefix": "status"}) v1 -- 0x55f5131e68b0 con 0x55f513287fc0 2023-08-09T13:52:12.215 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:52:12.206+0000 7f98a9760640 1 --2- 172.21.15.43:0/3484253024 >> [v2:172.21.15.43:6832/2223580538,v1:172.21.15.43:6834/2223580538] conn(0x55f513287fc0 0x55f5132883a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:52:12.215 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:52:12.207+0000 7f98a9760640 1 --2- 172.21.15.43:0/3484253024 >> [v2:172.21.15.43:6832/2223580538,v1:172.21.15.43:6834/2223580538] conn(0x55f513287fc0 0x55f5132883a0 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f870075d242aca80 server_cookie=be951482609985e7 in_seq=0 out_seq=0 2023-08-09T13:52:12.216 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:52:12.207+0000 7f9883fff640 10 client.8217 ms_handle_connect on v2:172.21.15.43:6832/2223580538 2023-08-09T13:52:12.216 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:52:12.207+0000 7f9883fff640 1 -- 172.21.15.43:0/3484253024 <== mds.0 v2:172.21.15.43:6832/2223580538 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+333 (crc 0 0 0) 0x55f5131e68b0 con 0x55f513287fc0 2023-08-09T13:52:12.216 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:52:12.207+0000 7f9883fff640 10 client.8217 handle_command_reply: tid=0 2023-08-09T13:52:12.216 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:52:12.207+0000 7f98ae608740 1 client.8217 shutdown 2023-08-09T13:52:12.216 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:52:12.207+0000 7f98ae608740 1 -- 172.21.15.43:0/3484253024 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55f5132b2c50 msgr2=0x55f5132aa470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:52:12.216 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:52:12.207+0000 7f98ae608740 1 --2- 172.21.15.43:0/3484253024 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55f5132b2c50 0x55f5132aa470 secure :-1 s=READY pgs=1372 cs=0 l=1 rev1=1 crypto rx=0x7f9894001570 tx=0x7f9894009040 comp rx=0 tx=0).stop 2023-08-09T13:52:12.217 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:52:12.207+0000 7f98ae608740 1 -- 172.21.15.43:0/3484253024 shutdown_connections 2023-08-09T13:52:12.217 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:52:12.207+0000 7f98ae608740 1 --2- 172.21.15.43:0/3484253024 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x55f5132cdfe0 0x55f5132aa9b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:52:12.217 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:52:12.207+0000 7f98ae608740 1 --2- 172.21.15.43:0/3484253024 >> [v2:172.21.15.43:6832/2223580538,v1:172.21.15.43:6834/2223580538] conn(0x55f513287fc0 0x55f5132883a0 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:52:12.217 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:52:12.207+0000 7f98ae608740 1 --2- 172.21.15.43:0/3484253024 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55f5132b2c50 0x55f5132aa470 unknown :-1 s=CLOSED pgs=1372 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:52:12.217 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:52:12.207+0000 7f98ae608740 1 --2- 172.21.15.43:0/3484253024 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55f5132867d0 0x55f5132eaaf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:52:12.217 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:52:12.207+0000 7f9883fff640 0 client.8217 ms_handle_reset on v2:172.21.15.43:6832/2223580538 2023-08-09T13:52:12.217 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:52:12.207+0000 7f98ae608740 1 -- 172.21.15.43:0/3484253024 >> 172.21.15.43:0/3484253024 conn(0x55f513282bd0 msgr2=0x55f513288d60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-09T13:52:12.217 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:52:12.208+0000 7f98ae608740 1 -- 172.21.15.43:0/3484253024 shutdown_connections 2023-08-09T13:52:12.218 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:52:12.208+0000 7f98ae608740 1 -- 172.21.15.43:0/3484253024 wait complete. 2023-08-09T13:52:12.218 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:52:12.208+0000 7f98ae608740 20 client.8217 trim_cache size 0 max 16384 2023-08-09T13:52:12.218 INFO:teuthology.orchestra.run.smithi043.stdout:{ 2023-08-09T13:52:12.218 INFO:teuthology.orchestra.run.smithi043.stdout: "cluster_fsid": "daac87ee-1337-4250-a955-aea4e408a39b", 2023-08-09T13:52:12.218 INFO:teuthology.orchestra.run.smithi043.stdout: "whoami": 0, 2023-08-09T13:52:12.219 INFO:teuthology.orchestra.run.smithi043.stdout: "id": 8040, 2023-08-09T13:52:12.219 INFO:teuthology.orchestra.run.smithi043.stdout: "want_state": "up:active", 2023-08-09T13:52:12.219 INFO:teuthology.orchestra.run.smithi043.stdout: "state": "up:active", 2023-08-09T13:52:12.219 INFO:teuthology.orchestra.run.smithi043.stdout: "fs_name": "cephfs", 2023-08-09T13:52:12.219 INFO:teuthology.orchestra.run.smithi043.stdout: "rank_uptime": 11.998835209999999, 2023-08-09T13:52:12.219 INFO:teuthology.orchestra.run.smithi043.stdout: "mdsmap_epoch": 52, 2023-08-09T13:52:12.219 INFO:teuthology.orchestra.run.smithi043.stdout: "osdmap_epoch": 116, 2023-08-09T13:52:12.220 INFO:teuthology.orchestra.run.smithi043.stdout: "osdmap_epoch_barrier": 116, 2023-08-09T13:52:12.220 INFO:teuthology.orchestra.run.smithi043.stdout: "uptime": 18.417460645999999 2023-08-09T13:52:12.220 INFO:teuthology.orchestra.run.smithi043.stdout:} 2023-08-09T13:52:12.221 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:52:12.208+0000 7f98ab9c4640 1 -- 172.21.15.43:0/1406389778 >> [v2:172.21.15.203:6832/1317108840,v1:172.21.15.203:6833/1317108840] conn(0x7f988806a1a0 msgr2=0x7f988806c660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:52:12.221 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:52:12.208+0000 7f98ab9c4640 1 --2- 172.21.15.43:0/1406389778 >> [v2:172.21.15.203:6832/1317108840,v1:172.21.15.203:6833/1317108840] conn(0x7f988806a1a0 0x7f988806c660 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f98a0036000 tx=0x7f98a0030000 comp rx=0 tx=0).stop 2023-08-09T13:52:12.221 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:52:12.208+0000 7f98ab9c4640 1 -- 172.21.15.43:0/1406389778 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f98a40c58e0 msgr2=0x7f98a40c5d60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:52:12.221 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:52:12.208+0000 7f98ab9c4640 1 --2- 172.21.15.43:0/1406389778 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f98a40c58e0 0x7f98a40c5d60 secure :-1 s=READY pgs=1368 cs=0 l=1 rev1=1 crypto rx=0x7f98a406d9d0 tx=0x7f989c00d9d0 comp rx=0 tx=0).stop 2023-08-09T13:52:12.221 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:52:12.208+0000 7f98ab9c4640 1 -- 172.21.15.43:0/1406389778 shutdown_connections 2023-08-09T13:52:12.221 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:52:12.208+0000 7f98ab9c4640 1 --2- 172.21.15.43:0/1406389778 >> [v2:172.21.15.203:6832/1317108840,v1:172.21.15.203:6833/1317108840] conn(0x7f988806a1a0 0x7f988806c660 unknown :-1 s=CLOSED pgs=73 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:52:12.222 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:52:12.208+0000 7f98ab9c4640 1 --2- 172.21.15.43:0/1406389778 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f98a40c58e0 0x7f98a40c5d60 unknown :-1 s=CLOSED pgs=1368 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:52:12.222 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:52:12.208+0000 7f98ab9c4640 1 --2- 172.21.15.43:0/1406389778 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f98a40c6d10 0x7f98a4153d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:52:12.222 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:52:12.208+0000 7f98ab9c4640 1 --2- 172.21.15.43:0/1406389778 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f9890003a10 0x7f98a41f6100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:52:12.222 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:52:12.208+0000 7f98ab9c4640 1 -- 172.21.15.43:0/1406389778 >> 172.21.15.43:0/1406389778 conn(0x7f98a40ce0a0 msgr2=0x7f98a40c87c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-09T13:52:12.222 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:52:12.209+0000 7f98ab9c4640 1 -- 172.21.15.43:0/1406389778 shutdown_connections 2023-08-09T13:52:12.222 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:52:12.209+0000 7f98ab9c4640 1 -- 172.21.15.43:0/1406389778 wait complete. 2023-08-09T13:52:12.230 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2023-08-09T13:52:12.460 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:52:12.459+0000 7fd052d49640 1 Processor -- start 2023-08-09T13:52:12.460 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:52:12.459+0000 7fd052d49640 1 -- start start 2023-08-09T13:52:12.461 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:52:12.460+0000 7fd052d49640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7fd04c15f7b0 0x7fd04c0b4760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:52:12.462 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:52:12.460+0000 7fd052d49640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fd04c0cd6b0 0x7fd04c0d0e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:52:12.462 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:52:12.460+0000 7fd052d49640 1 -- --> v1:172.21.15.203:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd04c0b7bf0 con 0x7fd04c0b1050 2023-08-09T13:52:12.462 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:52:12.460+0000 7fd052d49640 1 -- --> v2:172.21.15.43:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd04c0b7ed0 con 0x7fd04c15f7b0 2023-08-09T13:52:12.462 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:52:12.460+0000 7fd052d49640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd04c0d0370 con 0x7fd04c0cd6b0 2023-08-09T13:52:12.463 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:52:12.461+0000 7fd051d47640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7fd04c15f7b0 0x7fd04c0b4760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:52:12.463 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:52:12.461+0000 7fd051d47640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7fd04c15f7b0 0x7fd04c0b4760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:54430/0 (socket says 172.21.15.43:54430) 2023-08-09T13:52:12.463 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:52:12.461+0000 7fd051d47640 1 -- 172.21.15.43:0/1141605267 learned_addr learned my addr 172.21.15.43:0/1141605267 (peer_addr_for_me v2:172.21.15.43:0/0) 2023-08-09T13:52:12.463 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:52:12.461+0000 7fd052548640 1 --2- 172.21.15.43:0/1141605267 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fd04c0cd6b0 0x7fd04c0d0e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:52:12.464 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:52:12.461+0000 7fd051d47640 1 -- 172.21.15.43:0/1141605267 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fd04c0cd6b0 msgr2=0x7fd04c0d0e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-09T13:52:12.464 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:52:12.461+0000 7fd051d47640 1 --2- 172.21.15.43:0/1141605267 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fd04c0cd6b0 0x7fd04c0d0e50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:52:12.464 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:52:12.461+0000 7fd051d47640 1 -- 172.21.15.43:0/1141605267 >> v1:172.21.15.203:6789/0 conn(0x7fd04c0b1050 legacy=0x7fd04c0ccfa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-09T13:52:12.464 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:52:12.461+0000 7fd051d47640 1 -- 172.21.15.43:0/1141605267 --> v2:172.21.15.43:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd04c0d0650 con 0x7fd04c15f7b0 2023-08-09T13:52:12.465 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:52:12.462+0000 7fd051d47640 1 --2- 172.21.15.43:0/1141605267 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fd04c15f7b0 0x7fd04c0b4760 secure :-1 s=READY pgs=1373 cs=0 l=1 rev1=1 crypto rx=0x7fd0480098a0 tx=0x7fd04802b8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ad0b06452967dda2 server_cookie=0 in_seq=0 out_seq=0 2023-08-09T13:52:12.465 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:52:12.463+0000 7fd050d45640 1 -- 172.21.15.43:0/1141605267 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd048031640 con 0x7fd04c15f7b0 2023-08-09T13:52:12.465 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:52:12.463+0000 7fd050d45640 1 -- 172.21.15.43:0/1141605267 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd0480317a0 con 0x7fd04c15f7b0 2023-08-09T13:52:12.465 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:52:12.463+0000 7fd050d45640 1 -- 172.21.15.43:0/1141605267 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd048031b30 con 0x7fd04c15f7b0 2023-08-09T13:52:12.466 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:52:12.463+0000 7fd052d49640 1 -- 172.21.15.43:0/1141605267 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fd04c15f7b0 msgr2=0x7fd04c0b4760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:52:12.466 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:52:12.463+0000 7fd052d49640 1 --2- 172.21.15.43:0/1141605267 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fd04c15f7b0 0x7fd04c0b4760 secure :-1 s=READY pgs=1373 cs=0 l=1 rev1=1 crypto rx=0x7fd0480098a0 tx=0x7fd04802b8f0 comp rx=0 tx=0).stop 2023-08-09T13:52:12.466 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:52:12.463+0000 7fd052d49640 1 -- 172.21.15.43:0/1141605267 shutdown_connections 2023-08-09T13:52:12.466 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:52:12.463+0000 7fd052d49640 1 --2- 172.21.15.43:0/1141605267 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fd04c0cd6b0 0x7fd04c0d0e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:52:12.466 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:52:12.463+0000 7fd052d49640 1 --2- 172.21.15.43:0/1141605267 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fd04c15f7b0 0x7fd04c0b4760 unknown :-1 s=CLOSED pgs=1373 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:52:12.466 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:52:12.463+0000 7fd052d49640 1 -- 172.21.15.43:0/1141605267 >> 172.21.15.43:0/1141605267 conn(0x7fd04c06a2b0 msgr2=0x7fd04c06a690 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-09T13:52:12.466 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:52:12.464+0000 7fd052d49640 1 -- 172.21.15.43:0/1141605267 shutdown_connections 2023-08-09T13:52:12.467 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:52:12.464+0000 7fd052d49640 1 -- 172.21.15.43:0/1141605267 wait complete. 2023-08-09T13:52:12.467 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:52:12.465+0000 7fd052d49640 1 Processor -- start 2023-08-09T13:52:12.467 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:52:12.465+0000 7fd052d49640 1 -- start start 2023-08-09T13:52:12.468 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:52:12.466+0000 7fd052d49640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fd04c0b1050 0x7fd04c1461c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:52:12.468 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:52:12.466+0000 7fd052d49640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fd04c0cd6b0 0x7fd04c1f1500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:52:12.468 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:52:12.466+0000 7fd052d49640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fd04c15f7b0 0x7fd04c1f23d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:52:12.468 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:52:12.466+0000 7fd052d49640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd04c1471e0 con 0x7fd04c0b1050 2023-08-09T13:52:12.469 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:52:12.466+0000 7fd052d49640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd04c0c8cd0 con 0x7fd04c15f7b0 2023-08-09T13:52:12.469 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:52:12.466+0000 7fd052d49640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd04c146700 con 0x7fd04c0cd6b0 2023-08-09T13:52:12.469 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:52:12.467+0000 7fd051d47640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fd04c0b1050 0x7fd04c1461c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:52:12.469 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:52:12.467+0000 7fd051d47640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fd04c0b1050 0x7fd04c1461c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:54434/0 (socket says 172.21.15.43:54434) 2023-08-09T13:52:12.470 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:52:12.467+0000 7fd051d47640 1 -- 172.21.15.43:0/3826825728 learned_addr learned my addr 172.21.15.43:0/3826825728 (peer_addr_for_me v2:172.21.15.43:0/0) 2023-08-09T13:52:12.473 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:52:12.467+0000 7fd051d47640 1 -- 172.21.15.43:0/3826825728 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fd04c0cd6b0 msgr2=0x7fd04c1f1500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:52:12.473 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:52:12.467+0000 7fd051d47640 1 --2- 172.21.15.43:0/3826825728 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fd04c0cd6b0 0x7fd04c1f1500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:52:12.473 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:52:12.467+0000 7fd051d47640 1 -- 172.21.15.43:0/3826825728 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fd04c15f7b0 msgr2=0x7fd04c1f23d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:52:12.474 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:52:12.467+0000 7fd051d47640 1 --2- 172.21.15.43:0/3826825728 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fd04c15f7b0 0x7fd04c1f23d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:52:12.474 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:52:12.467+0000 7fd051d47640 1 -- 172.21.15.43:0/3826825728 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd0480094d0 con 0x7fd04c0b1050 2023-08-09T13:52:12.474 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:52:12.467+0000 7fd051d47640 1 --2- 172.21.15.43:0/3826825728 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fd04c0b1050 0x7fd04c1461c0 secure :-1 s=READY pgs=1374 cs=0 l=1 rev1=1 crypto rx=0x7fd0480098a0 tx=0x7fd0480046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-09T13:52:12.475 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:52:12.468+0000 7fd0427fc640 1 -- 172.21.15.43:0/3826825728 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd04803ac60 con 0x7fd04c0b1050 2023-08-09T13:52:12.475 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:52:12.468+0000 7fd0427fc640 1 -- 172.21.15.43:0/3826825728 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd04803ade0 con 0x7fd04c0b1050 2023-08-09T13:52:12.475 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:52:12.468+0000 7fd0427fc640 1 -- 172.21.15.43:0/3826825728 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd048044630 con 0x7fd04c0b1050 2023-08-09T13:52:12.476 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:52:12.468+0000 7fd052d49640 1 -- 172.21.15.43:0/3826825728 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd04c1f1f40 con 0x7fd04c0b1050 2023-08-09T13:52:12.476 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:52:12.468+0000 7fd052d49640 1 -- 172.21.15.43:0/3826825728 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd04c146c20 con 0x7fd04c0b1050 2023-08-09T13:52:12.476 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:52:12.469+0000 7fd052d49640 1 -- 172.21.15.43:0/3826825728 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd04c0739f0 con 0x7fd04c0b1050 2023-08-09T13:52:12.476 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:52:12.470+0000 7fd0427fc640 1 -- 172.21.15.43:0/3826825728 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 45) v1 ==== 88443+0+0 (secure 0 0 0) 0x7fd04802f050 con 0x7fd04c0b1050 2023-08-09T13:52:12.477 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:52:12.471+0000 7fd0427fc640 1 --2- 172.21.15.43:0/3826825728 >> [v2:172.21.15.203:6832/1317108840,v1:172.21.15.203:6833/1317108840] conn(0x7fd03006a170 0x7fd03006c630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:52:12.477 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:52:12.471+0000 7fd0427fc640 1 -- 172.21.15.43:0/3826825728 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(116..116 src has 1..116) v4 ==== 6426+0+0 (secure 0 0 0) 0x7fd0480aabf0 con 0x7fd04c0b1050 2023-08-09T13:52:12.478 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:52:12.472+0000 7fd051546640 1 --2- 172.21.15.43:0/3826825728 >> [v2:172.21.15.203:6832/1317108840,v1:172.21.15.203:6833/1317108840] conn(0x7fd03006a170 0x7fd03006c630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:52:12.478 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:52:12.473+0000 7fd051546640 1 --2- 172.21.15.43:0/3826825728 >> [v2:172.21.15.203:6832/1317108840,v1:172.21.15.203:6833/1317108840] conn(0x7fd03006a170 0x7fd03006c630 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7fd03c006fd0 tx=0x7fd03c006bc0 comp rx=0 tx=0).ready entity=mgr.6619 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-09T13:52:12.479 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:52:12.476+0000 7fd0427fc640 1 -- 172.21.15.43:0/3826825728 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178066 (secure 0 0 0) 0x7fd048075eb0 con 0x7fd04c0b1050 2023-08-09T13:52:12.716 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:52:12.714+0000 7fd052d49640 1 -- 172.21.15.43:0/3826825728 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7fd04c0b6eb0 con 0x7fd04c0b1050 2023-08-09T13:52:12.717 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:52:12.715+0000 7fd0427fc640 1 -- 172.21.15.43:0/3826825728 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v52) v1 ==== 53+0+73 (secure 0 0 0) 0x7fd0480794d0 con 0x7fd04c0b1050 2023-08-09T13:52:12.717 INFO:teuthology.orchestra.run.smithi043.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2023-08-09T13:52:12.720 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:52:12.718+0000 7fd052d49640 1 -- 172.21.15.43:0/3826825728 >> [v2:172.21.15.203:6832/1317108840,v1:172.21.15.203:6833/1317108840] conn(0x7fd03006a170 msgr2=0x7fd03006c630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:52:12.720 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:52:12.718+0000 7fd052d49640 1 --2- 172.21.15.43:0/3826825728 >> [v2:172.21.15.203:6832/1317108840,v1:172.21.15.203:6833/1317108840] conn(0x7fd03006a170 0x7fd03006c630 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7fd03c006fd0 tx=0x7fd03c006bc0 comp rx=0 tx=0).stop 2023-08-09T13:52:12.721 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:52:12.718+0000 7fd052d49640 1 -- 172.21.15.43:0/3826825728 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fd04c0b1050 msgr2=0x7fd04c1461c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:52:12.721 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:52:12.718+0000 7fd052d49640 1 --2- 172.21.15.43:0/3826825728 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fd04c0b1050 0x7fd04c1461c0 secure :-1 s=READY pgs=1374 cs=0 l=1 rev1=1 crypto rx=0x7fd0480098a0 tx=0x7fd0480046c0 comp rx=0 tx=0).stop 2023-08-09T13:52:12.721 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:52:12.719+0000 7fd052d49640 1 -- 172.21.15.43:0/3826825728 shutdown_connections 2023-08-09T13:52:12.721 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:52:12.719+0000 7fd052d49640 1 --2- 172.21.15.43:0/3826825728 >> [v2:172.21.15.203:6832/1317108840,v1:172.21.15.203:6833/1317108840] conn(0x7fd03006a170 0x7fd03006c630 unknown :-1 s=CLOSED pgs=74 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:52:12.721 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:52:12.719+0000 7fd052d49640 1 --2- 172.21.15.43:0/3826825728 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fd04c15f7b0 0x7fd04c1f23d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:52:12.721 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:52:12.719+0000 7fd052d49640 1 --2- 172.21.15.43:0/3826825728 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fd04c0cd6b0 0x7fd04c1f1500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:52:12.722 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:52:12.719+0000 7fd052d49640 1 --2- 172.21.15.43:0/3826825728 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fd04c0b1050 0x7fd04c1461c0 unknown :-1 s=CLOSED pgs=1374 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:52:12.722 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:52:12.719+0000 7fd052d49640 1 -- 172.21.15.43:0/3826825728 >> 172.21.15.43:0/3826825728 conn(0x7fd04c06a2b0 msgr2=0x7fd04c0d0e50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-09T13:52:12.722 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:52:12.719+0000 7fd052d49640 1 -- 172.21.15.43:0/3826825728 shutdown_connections 2023-08-09T13:52:12.722 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:52:12.719+0000 7fd052d49640 1 -- 172.21.15.43:0/3826825728 wait complete. 2023-08-09T13:52:12.741 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2023-08-09T13:52:12.741 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2023-08-09T13:52:12.741 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi043.front.sepia.ceph.com 2023-08-09T13:52:12.742 INFO:tasks.cephfs.mount:self.client.name = client.0 2023-08-09T13:52:12.742 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2023-08-09T13:52:12.742 INFO:tasks.cephfs.mount:self.cephfs_name = None 2023-08-09T13:52:12.743 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2023-08-09T13:52:12.743 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2023-08-09T13:52:12.743 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2023-08-09T13:52:12.743 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-09T13:52:12.744 DEBUG:teuthology.orchestra.run.smithi043:> ip addr 2023-08-09T13:52:12.805 INFO:teuthology.orchestra.run.smithi043.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2023-08-09T13:52:12.805 INFO:teuthology.orchestra.run.smithi043.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2023-08-09T13:52:12.805 INFO:teuthology.orchestra.run.smithi043.stdout: inet 127.0.0.1/8 scope host lo 2023-08-09T13:52:12.806 INFO:teuthology.orchestra.run.smithi043.stdout: valid_lft forever preferred_lft forever 2023-08-09T13:52:12.806 INFO:teuthology.orchestra.run.smithi043.stdout: inet6 ::1/128 scope host 2023-08-09T13:52:12.806 INFO:teuthology.orchestra.run.smithi043.stdout: valid_lft forever preferred_lft forever 2023-08-09T13:52:12.807 INFO:teuthology.orchestra.run.smithi043.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-08-09T13:52:12.807 INFO:teuthology.orchestra.run.smithi043.stdout: link/ether 0c:c4:7a:6b:fd:9c brd ff:ff:ff:ff:ff:ff 2023-08-09T13:52:12.807 INFO:teuthology.orchestra.run.smithi043.stdout: altname enp7s0f0 2023-08-09T13:52:12.807 INFO:teuthology.orchestra.run.smithi043.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-08-09T13:52:12.808 INFO:teuthology.orchestra.run.smithi043.stdout: link/ether 0c:c4:7a:6b:fd:9d brd ff:ff:ff:ff:ff:ff 2023-08-09T13:52:12.808 INFO:teuthology.orchestra.run.smithi043.stdout: altname enp7s0f1 2023-08-09T13:52:12.808 INFO:teuthology.orchestra.run.smithi043.stdout:4: ens1f0: mtu 1500 qdisc mq state UP group default qlen 1000 2023-08-09T13:52:12.808 INFO:teuthology.orchestra.run.smithi043.stdout: link/ether 0c:c4:7a:bd:13:58 brd ff:ff:ff:ff:ff:ff 2023-08-09T13:52:12.808 INFO:teuthology.orchestra.run.smithi043.stdout: altname enp1s0f0 2023-08-09T13:52:12.808 INFO:teuthology.orchestra.run.smithi043.stdout: inet 172.21.15.43/20 brd 172.21.15.255 scope global dynamic noprefixroute ens1f0 2023-08-09T13:52:12.808 INFO:teuthology.orchestra.run.smithi043.stdout: valid_lft 41171sec preferred_lft 41171sec 2023-08-09T13:52:12.809 INFO:teuthology.orchestra.run.smithi043.stdout: inet6 fe80::ec4:7aff:febd:1358/64 scope link 2023-08-09T13:52:12.809 INFO:teuthology.orchestra.run.smithi043.stdout: valid_lft forever preferred_lft forever 2023-08-09T13:52:12.809 INFO:teuthology.orchestra.run.smithi043.stdout:5: ens1f1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-08-09T13:52:12.809 INFO:teuthology.orchestra.run.smithi043.stdout: link/ether 0c:c4:7a:bd:13:59 brd ff:ff:ff:ff:ff:ff 2023-08-09T13:52:12.809 INFO:teuthology.orchestra.run.smithi043.stdout: altname enp1s0f1 2023-08-09T13:52:12.809 INFO:teuthology.orchestra.run.smithi043.stdout:8: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2023-08-09T13:52:12.809 INFO:teuthology.orchestra.run.smithi043.stdout: link/ether 56:43:50:19:f4:0c brd ff:ff:ff:ff:ff:ff 2023-08-09T13:52:12.809 INFO:teuthology.orchestra.run.smithi043.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2023-08-09T13:52:12.810 INFO:teuthology.orchestra.run.smithi043.stdout: valid_lft forever preferred_lft forever 2023-08-09T13:52:12.810 INFO:teuthology.orchestra.run.smithi043.stdout: inet6 fe80::e494:65ff:fe92:a182/64 scope link 2023-08-09T13:52:12.810 INFO:teuthology.orchestra.run.smithi043.stdout: valid_lft forever preferred_lft forever 2023-08-09T13:52:12.810 INFO:teuthology.orchestra.run.smithi043.stdout:9: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2023-08-09T13:52:12.810 INFO:teuthology.orchestra.run.smithi043.stdout: link/ether 7e:93:26:fa:07:8f brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.0 2023-08-09T13:52:12.810 INFO:teuthology.orchestra.run.smithi043.stdout: inet6 fe80::7c93:26ff:fefa:78f/64 scope link 2023-08-09T13:52:12.810 INFO:teuthology.orchestra.run.smithi043.stdout: valid_lft forever preferred_lft forever 2023-08-09T13:52:12.810 INFO:teuthology.orchestra.run.smithi043.stdout:10: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2023-08-09T13:52:12.811 INFO:teuthology.orchestra.run.smithi043.stdout: link/ether 56:43:50:19:f4:0c brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.2 2023-08-09T13:52:12.811 INFO:teuthology.orchestra.run.smithi043.stdout: inet6 fe80::5443:50ff:fe19:f40c/64 scope link 2023-08-09T13:52:12.811 INFO:teuthology.orchestra.run.smithi043.stdout: valid_lft forever preferred_lft forever 2023-08-09T13:52:12.812 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-09T13:52:12.812 DEBUG:teuthology.orchestra.run.smithi043:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2023-08-09T13:52:12.916 INFO:teuthology.orchestra.run.smithi043.stdout:1 2023-08-09T13:52:12.920 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-09T13:52:12.921 DEBUG:teuthology.orchestra.run.smithi043:> route 2023-08-09T13:52:12.980 INFO:teuthology.orchestra.run.smithi043.stdout:Kernel IP routing table 2023-08-09T13:52:12.980 INFO:teuthology.orchestra.run.smithi043.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2023-08-09T13:52:12.981 INFO:teuthology.orchestra.run.smithi043.stdout:default _gateway 0.0.0.0 UG 100 0 0 ens1f0 2023-08-09T13:52:12.981 INFO:teuthology.orchestra.run.smithi043.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 ens1f0 2023-08-09T13:52:12.981 INFO:teuthology.orchestra.run.smithi043.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2023-08-09T13:52:12.982 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-09T13:52:12.982 DEBUG:teuthology.orchestra.run.smithi043:> (cd / && exec bash -c ' 2023-08-09T13:52:12.983 DEBUG:teuthology.orchestra.run.smithi043:> set -e 2023-08-09T13:52:12.983 DEBUG:teuthology.orchestra.run.smithi043:> sudo iptables -A FORWARD -o ens1f0 -i ceph-brx -j ACCEPT 2023-08-09T13:52:12.983 DEBUG:teuthology.orchestra.run.smithi043:> sudo iptables -A FORWARD -i ens1f0 -o ceph-brx -j ACCEPT 2023-08-09T13:52:12.984 DEBUG:teuthology.orchestra.run.smithi043:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o ens1f0 -j MASQUERADE 2023-08-09T13:52:12.984 DEBUG:teuthology.orchestra.run.smithi043:> ') 2023-08-09T13:52:13.187 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-09T13:52:13.187 DEBUG:teuthology.orchestra.run.smithi043:> ip netns list 2023-08-09T13:52:13.245 INFO:teuthology.orchestra.run.smithi043.stdout:ceph-ns--home-ubuntu-cephtest-mnt.2 (id: 1) 2023-08-09T13:52:13.245 INFO:teuthology.orchestra.run.smithi043.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2023-08-09T13:52:13.246 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2023-08-09T13:52:13.246 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-09T13:52:13.247 DEBUG:teuthology.orchestra.run.smithi043:> sudo ip link set brx.0 up 2023-08-09T13:52:13.352 INFO:tasks.cephfs.fuse_mount:Client client.0 config is {} 2023-08-09T13:52:13.353 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-08-09T13:52:13.353 DEBUG:teuthology.orchestra.run.smithi043:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2023-08-09T13:52:13.412 INFO:teuthology.orchestra.run.smithi043.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2023-08-09T13:52:13.412 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-08-09T13:52:13.413 DEBUG:teuthology.orchestra.run.smithi043:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2023-08-09T13:52:13.470 DEBUG:teuthology.orchestra.run.smithi043:> sudo modprobe fuse 2023-08-09T13:52:13.572 INFO:teuthology.orchestra.run:Running command with timeout 30 2023-08-09T13:52:13.573 DEBUG:teuthology.orchestra.run.smithi043:> sudo mount -t fusectl /sys/fs/fuse/connections /sys/fs/fuse/connections 2023-08-09T13:52:13.673 INFO:teuthology.orchestra.run.smithi043.stderr:mount: /sys/fs/fuse/connections: /sys/fs/fuse/connections already mounted or mount point busy. 2023-08-09T13:52:13.676 DEBUG:teuthology.orchestra.run:got remote process result: 32 2023-08-09T13:52:13.677 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-09T13:52:13.677 DEBUG:teuthology.orchestra.run.smithi043:> ls /sys/fs/fuse/connections 2023-08-09T13:52:13.734 INFO:tasks.cephfs.fuse_mount:Pre-mount connections: [] 2023-08-09T13:52:13.734 DEBUG:teuthology.orchestra.run.smithi043:> (cd /home/ubuntu/cephtest && exec sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-fuse -f --admin-socket '/var/run/ceph/$cluster-$name.$pid.asok' /home/ubuntu/cephtest/mnt.0 --id 0) 2023-08-09T13:52:13.744 DEBUG:teuthology.orchestra.run.smithi043:> sudo modprobe fuse 2023-08-09T13:52:13.832 INFO:teuthology.orchestra.run:Running command with timeout 30 2023-08-09T13:52:13.832 DEBUG:teuthology.orchestra.run.smithi043:> sudo mount -t fusectl /sys/fs/fuse/connections /sys/fs/fuse/connections 2023-08-09T13:52:13.911 INFO:teuthology.orchestra.run.smithi043.stderr:mount: /sys/fs/fuse/connections: /sys/fs/fuse/connections already mounted or mount point busy. 2023-08-09T13:52:13.914 DEBUG:teuthology.orchestra.run:got remote process result: 32 2023-08-09T13:52:13.915 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-09T13:52:13.915 DEBUG:teuthology.orchestra.run.smithi043:> ls /sys/fs/fuse/connections 2023-08-09T13:52:13.948 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi043.stderr:2023-08-09T13:52:13.946+0000 7f3c460da480 -1 init, newargv = 0x562c539b31d0 newargc=15 2023-08-09T13:52:13.948 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi043.stderr:ceph-fuse[125126]: starting ceph client 2023-08-09T13:52:13.958 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi043.stderr:ceph-fuse[125126]: starting fuse 2023-08-09T13:52:13.963 INFO:teuthology.orchestra.run.smithi043.stdout:40 2023-08-09T13:52:13.963 INFO:tasks.cephfs.fuse_mount:Post-mount connections: [40] 2023-08-09T13:52:13.964 DEBUG:teuthology.orchestra.run.smithi043:> sudo stdin-killer -- python3 -c ' 2023-08-09T13:52:13.964 DEBUG:teuthology.orchestra.run.smithi043:> import glob 2023-08-09T13:52:13.964 DEBUG:teuthology.orchestra.run.smithi043:> import re 2023-08-09T13:52:13.965 DEBUG:teuthology.orchestra.run.smithi043:> import os 2023-08-09T13:52:13.965 DEBUG:teuthology.orchestra.run.smithi043:> import subprocess 2023-08-09T13:52:13.965 DEBUG:teuthology.orchestra.run.smithi043:> 2023-08-09T13:52:13.966 DEBUG:teuthology.orchestra.run.smithi043:> def _find_admin_socket(client_name): 2023-08-09T13:52:13.966 DEBUG:teuthology.orchestra.run.smithi043:> asok_path = "/var/run/ceph/ceph-client.0.*.asok" 2023-08-09T13:52:13.966 DEBUG:teuthology.orchestra.run.smithi043:> files = glob.glob(asok_path) 2023-08-09T13:52:13.967 DEBUG:teuthology.orchestra.run.smithi043:> mountpoint = "/home/ubuntu/cephtest/mnt.0" 2023-08-09T13:52:13.967 DEBUG:teuthology.orchestra.run.smithi043:> 2023-08-09T13:52:13.967 DEBUG:teuthology.orchestra.run.smithi043:> # Given a non-glob path, it better be there 2023-08-09T13:52:13.968 DEBUG:teuthology.orchestra.run.smithi043:> if "*" not in asok_path: 2023-08-09T13:52:13.968 DEBUG:teuthology.orchestra.run.smithi043:> assert(len(files) == 1) 2023-08-09T13:52:13.968 DEBUG:teuthology.orchestra.run.smithi043:> return files[0] 2023-08-09T13:52:13.968 DEBUG:teuthology.orchestra.run.smithi043:> 2023-08-09T13:52:13.969 DEBUG:teuthology.orchestra.run.smithi043:> for f in files: 2023-08-09T13:52:13.969 DEBUG:teuthology.orchestra.run.smithi043:> pid = re.match(".*\.(\d+)\.asok$", f).group(1) 2023-08-09T13:52:13.969 DEBUG:teuthology.orchestra.run.smithi043:> if os.path.exists("/proc/{0}".format(pid)): 2023-08-09T13:52:13.970 DEBUG:teuthology.orchestra.run.smithi043:> with open("/proc/{0}/cmdline".format(pid), '"'"'r'"'"') as proc_f: 2023-08-09T13:52:13.970 DEBUG:teuthology.orchestra.run.smithi043:> contents = proc_f.read() 2023-08-09T13:52:13.970 DEBUG:teuthology.orchestra.run.smithi043:> if mountpoint in contents: 2023-08-09T13:52:13.971 DEBUG:teuthology.orchestra.run.smithi043:> return f 2023-08-09T13:52:13.971 DEBUG:teuthology.orchestra.run.smithi043:> raise RuntimeError("Client socket {0} not found".format(client_name)) 2023-08-09T13:52:13.971 DEBUG:teuthology.orchestra.run.smithi043:> 2023-08-09T13:52:13.972 DEBUG:teuthology.orchestra.run.smithi043:> print(_find_admin_socket("client.0")) 2023-08-09T13:52:13.973 DEBUG:teuthology.orchestra.run.smithi043:> ' 2023-08-09T13:52:14.177 INFO:teuthology.orchestra.run.smithi043.stdout:/var/run/ceph/ceph-client.0.125126.asok 2023-08-09T13:52:14.182 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:52:14 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2023-08-09T13:52:14.194 INFO:tasks.cephfs.fuse_mount:Found client admin socket at /var/run/ceph/ceph-client.0.125126.asok 2023-08-09T13:52:14.195 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-09T13:52:14.195 DEBUG:teuthology.orchestra.run.smithi043:> sudo ceph --admin-daemon /var/run/ceph/ceph-client.0.125126.asok status 2023-08-09T13:52:14.442 INFO:teuthology.orchestra.run.smithi043.stdout:{ 2023-08-09T13:52:14.442 INFO:teuthology.orchestra.run.smithi043.stdout: "metadata": { 2023-08-09T13:52:14.442 INFO:teuthology.orchestra.run.smithi043.stdout: "ceph_sha1": "668a86f232c9454b2ce5928d1432a29a72964d85", 2023-08-09T13:52:14.442 INFO:teuthology.orchestra.run.smithi043.stdout: "ceph_version": "ceph version 18.0.0-5389-g668a86f2 (668a86f232c9454b2ce5928d1432a29a72964d85) reef (dev)", 2023-08-09T13:52:14.442 INFO:teuthology.orchestra.run.smithi043.stdout: "entity_id": "0", 2023-08-09T13:52:14.443 INFO:teuthology.orchestra.run.smithi043.stdout: "hostname": "smithi043", 2023-08-09T13:52:14.443 INFO:teuthology.orchestra.run.smithi043.stdout: "mount_point": "/home/ubuntu/cephtest/mnt.0", 2023-08-09T13:52:14.443 INFO:teuthology.orchestra.run.smithi043.stdout: "pid": "125126", 2023-08-09T13:52:14.443 INFO:teuthology.orchestra.run.smithi043.stdout: "root": "/" 2023-08-09T13:52:14.443 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2023-08-09T13:52:14.443 INFO:teuthology.orchestra.run.smithi043.stdout: "dentry_count": 0, 2023-08-09T13:52:14.443 INFO:teuthology.orchestra.run.smithi043.stdout: "dentry_pinned_count": 0, 2023-08-09T13:52:14.443 INFO:teuthology.orchestra.run.smithi043.stdout: "id": 8232, 2023-08-09T13:52:14.444 INFO:teuthology.orchestra.run.smithi043.stdout: "inst": { 2023-08-09T13:52:14.444 INFO:teuthology.orchestra.run.smithi043.stdout: "name": { 2023-08-09T13:52:14.444 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "client", 2023-08-09T13:52:14.444 INFO:teuthology.orchestra.run.smithi043.stdout: "num": 8232 2023-08-09T13:52:14.444 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2023-08-09T13:52:14.444 INFO:teuthology.orchestra.run.smithi043.stdout: "addr": { 2023-08-09T13:52:14.444 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "v1", 2023-08-09T13:52:14.445 INFO:teuthology.orchestra.run.smithi043.stdout: "addr": "192.168.0.1:0", 2023-08-09T13:52:14.445 INFO:teuthology.orchestra.run.smithi043.stdout: "nonce": 3541259421 2023-08-09T13:52:14.445 INFO:teuthology.orchestra.run.smithi043.stdout: } 2023-08-09T13:52:14.445 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2023-08-09T13:52:14.445 INFO:teuthology.orchestra.run.smithi043.stdout: "addr": { 2023-08-09T13:52:14.445 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "v1", 2023-08-09T13:52:14.445 INFO:teuthology.orchestra.run.smithi043.stdout: "addr": "192.168.0.1:0", 2023-08-09T13:52:14.445 INFO:teuthology.orchestra.run.smithi043.stdout: "nonce": 3541259421 2023-08-09T13:52:14.446 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2023-08-09T13:52:14.446 INFO:teuthology.orchestra.run.smithi043.stdout: "inst_str": "client.8232 192.168.0.1:0/3541259421", 2023-08-09T13:52:14.446 INFO:teuthology.orchestra.run.smithi043.stdout: "addr_str": "192.168.0.1:0/3541259421", 2023-08-09T13:52:14.446 INFO:teuthology.orchestra.run.smithi043.stdout: "inode_count": 1, 2023-08-09T13:52:14.446 INFO:teuthology.orchestra.run.smithi043.stdout: "mds_epoch": 52, 2023-08-09T13:52:14.446 INFO:teuthology.orchestra.run.smithi043.stdout: "osd_epoch": 116, 2023-08-09T13:52:14.446 INFO:teuthology.orchestra.run.smithi043.stdout: "osd_epoch_barrier": 0, 2023-08-09T13:52:14.446 INFO:teuthology.orchestra.run.smithi043.stdout: "blocklisted": false, 2023-08-09T13:52:14.447 INFO:teuthology.orchestra.run.smithi043.stdout: "fs_name": "cephfs" 2023-08-09T13:52:14.447 INFO:teuthology.orchestra.run.smithi043.stdout:} 2023-08-09T13:52:14.456 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-09T13:52:14.457 DEBUG:teuthology.orchestra.run.smithi043:> stat --file-system '--printf=%T 2023-08-09T13:52:14.457 DEBUG:teuthology.orchestra.run.smithi043:> ' -- /home/ubuntu/cephtest/mnt.0 2023-08-09T13:52:14.516 INFO:teuthology.orchestra.run.smithi043.stdout:fuseblk 2023-08-09T13:52:14.517 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.0 2023-08-09T13:52:14.517 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-09T13:52:14.517 DEBUG:teuthology.orchestra.run.smithi043:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2023-08-09T13:52:14.620 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs volume ls 2023-08-09T13:52:14.852 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:52:14.849+0000 7ff6f47b5640 1 Processor -- start 2023-08-09T13:52:14.852 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:52:14.850+0000 7ff6f47b5640 1 -- start start 2023-08-09T13:52:14.853 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:52:14.851+0000 7ff6f47b5640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7ff6ec15c680 0x7ff6ec0ce760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:52:14.854 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:52:14.851+0000 7ff6f47b5640 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7ff6ec0d0a70 0x7ff6ec0cd7f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:52:14.854 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:52:14.851+0000 7ff6f47b5640 1 -- --> v1:172.21.15.43:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff6ec0d0f70 con 0x7ff6ec0cdd30 2023-08-09T13:52:14.854 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:52:14.851+0000 7ff6f47b5640 1 -- --> v2:172.21.15.43:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff6ec0ccd10 con 0x7ff6ec15c680 2023-08-09T13:52:14.854 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:52:14.851+0000 7ff6f47b5640 1 -- --> v2:172.21.15.203:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff6ec0ccff0 con 0x7ff6ec0d0a70 2023-08-09T13:52:14.854 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:52:14.851+0000 7ff6f2551640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7ff6ec15c680 0x7ff6ec0ce760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:52:14.855 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:52:14.851+0000 7ff6f2d52640 1 --1- >> v1:172.21.15.43:6789/0 conn(0x7ff6ec0cdd30 0x7ff6ec0b3fa0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.43:6789/0 says I am v1:172.21.15.43:38132/0 (socket says 172.21.15.43:38132) 2023-08-09T13:52:14.855 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:52:14.851+0000 7ff6f2d52640 1 -- 172.21.15.43:0/1416218662 learned_addr learned my addr 172.21.15.43:0/1416218662 (peer_addr_for_me v1:172.21.15.43:0/0) 2023-08-09T13:52:14.856 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:52:14.852+0000 7ff6f1d50640 1 --2- 172.21.15.43:0/1416218662 >> v2:172.21.15.203:3300/0 conn(0x7ff6ec0d0a70 0x7ff6ec0cd7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:52:14.857 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:52:14.852+0000 7ff6f2551640 1 -- 172.21.15.43:0/1416218662 >> v2:172.21.15.203:3300/0 conn(0x7ff6ec0d0a70 msgr2=0x7ff6ec0cd7f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-09T13:52:14.857 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:52:14.852+0000 7ff6f2551640 1 --2- 172.21.15.43:0/1416218662 >> v2:172.21.15.203:3300/0 conn(0x7ff6ec0d0a70 0x7ff6ec0cd7f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:52:14.857 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:52:14.852+0000 7ff6f2551640 1 -- 172.21.15.43:0/1416218662 >> v1:172.21.15.43:6789/0 conn(0x7ff6ec0cdd30 legacy=0x7ff6ec0b3fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:52:14.857 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:52:14.852+0000 7ff6f2551640 1 -- 172.21.15.43:0/1416218662 --> v2:172.21.15.43:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff6ec0cd2d0 con 0x7ff6ec15c680 2023-08-09T13:52:14.857 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:52:14.852+0000 7ff6f154f640 1 -- 172.21.15.43:0/1416218662 <== mon.0 v1:172.21.15.43:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2231108438 0 0) 0x7ff6ec0d0f70 con 0x7ff6ec0cdd30 2023-08-09T13:52:14.857 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:52:14.852+0000 7ff6f1d50640 1 --2- 172.21.15.43:0/1416218662 >> v2:172.21.15.203:3300/0 conn(0x7ff6ec0d0a70 0x7ff6ec0cd7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-08-09T13:52:14.858 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:52:14.853+0000 7ff6f2551640 1 --2- 172.21.15.43:0/1416218662 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff6ec15c680 0x7ff6ec0ce760 secure :-1 s=READY pgs=1379 cs=0 l=1 rev1=1 crypto rx=0x7ff6ec0b5400 tx=0x7ff6e802b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=57ec3a714c325a18 server_cookie=0 in_seq=0 out_seq=0 2023-08-09T13:52:14.858 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:52:14.853+0000 7ff6f154f640 1 -- 172.21.15.43:0/1416218662 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff6e8031830 con 0x7ff6ec15c680 2023-08-09T13:52:14.858 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:52:14.853+0000 7ff6f154f640 1 -- 172.21.15.43:0/1416218662 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff6e8031990 con 0x7ff6ec15c680 2023-08-09T13:52:14.858 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:52:14.853+0000 7ff6f154f640 1 -- 172.21.15.43:0/1416218662 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff6e8031d20 con 0x7ff6ec15c680 2023-08-09T13:52:14.858 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:52:14.854+0000 7ff6f47b5640 1 -- 172.21.15.43:0/1416218662 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff6ec15c680 msgr2=0x7ff6ec0ce760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:52:14.859 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:52:14.854+0000 7ff6f47b5640 1 --2- 172.21.15.43:0/1416218662 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff6ec15c680 0x7ff6ec0ce760 secure :-1 s=READY pgs=1379 cs=0 l=1 rev1=1 crypto rx=0x7ff6ec0b5400 tx=0x7ff6e802b8d0 comp rx=0 tx=0).stop 2023-08-09T13:52:14.859 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:52:14.854+0000 7ff6f47b5640 1 -- 172.21.15.43:0/1416218662 shutdown_connections 2023-08-09T13:52:14.859 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:52:14.854+0000 7ff6f47b5640 1 --2- 172.21.15.43:0/1416218662 >> v2:172.21.15.203:3300/0 conn(0x7ff6ec0d0a70 0x7ff6ec0cd7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:52:14.859 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:52:14.854+0000 7ff6f47b5640 1 --2- 172.21.15.43:0/1416218662 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff6ec15c680 0x7ff6ec0ce760 unknown :-1 s=CLOSED pgs=1379 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:52:14.859 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:52:14.854+0000 7ff6f47b5640 1 -- 172.21.15.43:0/1416218662 >> 172.21.15.43:0/1416218662 conn(0x7ff6ec071520 msgr2=0x7ff6ec0d2520 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-09T13:52:14.860 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:52:14.854+0000 7ff6f47b5640 1 -- 172.21.15.43:0/1416218662 shutdown_connections 2023-08-09T13:52:14.860 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:52:14.854+0000 7ff6f47b5640 1 -- 172.21.15.43:0/1416218662 wait complete. 2023-08-09T13:52:14.862 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:52:14.856+0000 7ff6f47b5640 1 Processor -- start 2023-08-09T13:52:14.863 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:52:14.856+0000 7ff6f47b5640 1 -- start start 2023-08-09T13:52:14.863 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:52:14.857+0000 7ff6f47b5640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7ff6ec0d0a70 0x7ff6ec1f5bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:52:14.863 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:52:14.857+0000 7ff6f47b5640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff6ec15c680 0x7ff6ec1433a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:52:14.863 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:52:14.857+0000 7ff6f47b5640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7ff6ec154b50 0x7ff6ec1f1410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:52:14.863 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:52:14.857+0000 7ff6f47b5640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff6ec1497c0 con 0x7ff6ec15c680 2023-08-09T13:52:14.864 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:52:14.857+0000 7ff6f47b5640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff6ec15e480 con 0x7ff6ec154b50 2023-08-09T13:52:14.864 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:52:14.857+0000 7ff6f47b5640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff6ec1f6110 con 0x7ff6ec0d0a70 2023-08-09T13:52:14.864 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:52:14.857+0000 7ff6f1d50640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff6ec15c680 0x7ff6ec1433a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:52:14.864 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:52:14.858+0000 7ff6f1d50640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff6ec15c680 0x7ff6ec1433a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:54460/0 (socket says 172.21.15.43:54460) 2023-08-09T13:52:14.865 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:52:14.858+0000 7ff6f1d50640 1 -- 172.21.15.43:0/4006124147 learned_addr learned my addr 172.21.15.43:0/4006124147 (peer_addr_for_me v2:172.21.15.43:0/0) 2023-08-09T13:52:14.865 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:52:14.858+0000 7ff6f2551640 1 --2- 172.21.15.43:0/4006124147 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7ff6ec0d0a70 0x7ff6ec1f5bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:52:14.865 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:52:14.858+0000 7ff6f2d52640 1 --2- 172.21.15.43:0/4006124147 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7ff6ec154b50 0x7ff6ec1f1410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:52:14.865 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:52:14.858+0000 7ff6f1d50640 1 -- 172.21.15.43:0/4006124147 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7ff6ec0d0a70 msgr2=0x7ff6ec1f5bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:52:14.866 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:52:14.858+0000 7ff6f1d50640 1 --2- 172.21.15.43:0/4006124147 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7ff6ec0d0a70 0x7ff6ec1f5bd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:52:14.866 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:52:14.858+0000 7ff6f1d50640 1 -- 172.21.15.43:0/4006124147 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7ff6ec154b50 msgr2=0x7ff6ec1f1410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:52:14.866 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:52:14.858+0000 7ff6f1d50640 1 --2- 172.21.15.43:0/4006124147 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7ff6ec154b50 0x7ff6ec1f1410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:52:14.866 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:52:14.858+0000 7ff6f1d50640 1 -- 172.21.15.43:0/4006124147 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff6e8009400 con 0x7ff6ec15c680 2023-08-09T13:52:14.866 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:52:14.858+0000 7ff6f2d52640 1 --2- 172.21.15.43:0/4006124147 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7ff6ec154b50 0x7ff6ec1f1410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-08-09T13:52:14.867 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:52:14.858+0000 7ff6f1d50640 1 --2- 172.21.15.43:0/4006124147 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff6ec15c680 0x7ff6ec1433a0 secure :-1 s=READY pgs=1380 cs=0 l=1 rev1=1 crypto rx=0x7ff6dc00c820 tx=0x7ff6dc00cce0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-09T13:52:14.867 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:52:14.859+0000 7ff6e2ffd640 1 -- 172.21.15.43:0/4006124147 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff6dc007a30 con 0x7ff6ec15c680 2023-08-09T13:52:14.867 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:52:14.859+0000 7ff6e2ffd640 1 -- 172.21.15.43:0/4006124147 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff6dc007b90 con 0x7ff6ec15c680 2023-08-09T13:52:14.867 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:52:14.859+0000 7ff6e2ffd640 1 -- 172.21.15.43:0/4006124147 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff6dc0113f0 con 0x7ff6ec15c680 2023-08-09T13:52:14.868 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:52:14.859+0000 7ff6f47b5640 1 -- 172.21.15.43:0/4006124147 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff6ec155050 con 0x7ff6ec15c680 2023-08-09T13:52:14.868 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:52:14.859+0000 7ff6f47b5640 1 -- 172.21.15.43:0/4006124147 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff6ec15de30 con 0x7ff6ec15c680 2023-08-09T13:52:14.868 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:52:14.860+0000 7ff6f47b5640 1 -- 172.21.15.43:0/4006124147 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff6ec0b6530 con 0x7ff6ec15c680 2023-08-09T13:52:14.869 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:52:14.861+0000 7ff6e2ffd640 1 -- 172.21.15.43:0/4006124147 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 45) v1 ==== 88443+0+0 (secure 0 0 0) 0x7ff6dc011550 con 0x7ff6ec15c680 2023-08-09T13:52:14.869 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:52:14.862+0000 7ff6e2ffd640 1 --2- 172.21.15.43:0/4006124147 >> [v2:172.21.15.203:6832/1317108840,v1:172.21.15.203:6833/1317108840] conn(0x7ff6d006a170 0x7ff6d006c630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:52:14.870 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:52:14.862+0000 7ff6e2ffd640 1 -- 172.21.15.43:0/4006124147 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(116..116 src has 1..116) v4 ==== 6426+0+0 (secure 0 0 0) 0x7ff6dc088b40 con 0x7ff6ec15c680 2023-08-09T13:52:14.870 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:52:14.863+0000 7ff6f2551640 1 --2- 172.21.15.43:0/4006124147 >> [v2:172.21.15.203:6832/1317108840,v1:172.21.15.203:6833/1317108840] conn(0x7ff6d006a170 0x7ff6d006c630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:52:14.870 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:52:14.864+0000 7ff6f2551640 1 --2- 172.21.15.43:0/4006124147 >> [v2:172.21.15.203:6832/1317108840,v1:172.21.15.203:6833/1317108840] conn(0x7ff6d006a170 0x7ff6d006c630 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7ff6e8036000 tx=0x7ff6e8008030 comp rx=0 tx=0).ready entity=mgr.6619 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-09T13:52:14.870 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:52:14.866+0000 7ff6e2ffd640 1 -- 172.21.15.43:0/4006124147 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178066 (secure 0 0 0) 0x7ff6dc053e90 con 0x7ff6ec15c680 2023-08-09T13:52:15.087 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:52:15.084+0000 7ff6f47b5640 1 -- 172.21.15.43:0/4006124147 --> [v2:172.21.15.203:6832/1317108840,v1:172.21.15.203:6833/1317108840] -- mgr_command(tid 0: {"prefix": "fs volume ls", "target": ["mon-mgr", ""]}) v1 -- 0x7ff6ec1552c0 con 0x7ff6d006a170 2023-08-09T13:52:15.091 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:52:15.088+0000 7ff6e2ffd640 1 -- 172.21.15.43:0/4006124147 <== mgr.6619 v2:172.21.15.203:6832/1317108840 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+41 (secure 0 0 0) 0x7ff6ec1552c0 con 0x7ff6d006a170 2023-08-09T13:52:15.091 INFO:teuthology.orchestra.run.smithi043.stdout:[ 2023-08-09T13:52:15.092 INFO:teuthology.orchestra.run.smithi043.stdout: { 2023-08-09T13:52:15.092 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "cephfs" 2023-08-09T13:52:15.092 INFO:teuthology.orchestra.run.smithi043.stdout: } 2023-08-09T13:52:15.092 INFO:teuthology.orchestra.run.smithi043.stdout:] 2023-08-09T13:52:15.093 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:52:15.091+0000 7ff6f47b5640 1 -- 172.21.15.43:0/4006124147 >> [v2:172.21.15.203:6832/1317108840,v1:172.21.15.203:6833/1317108840] conn(0x7ff6d006a170 msgr2=0x7ff6d006c630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:52:15.094 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:52:15.091+0000 7ff6f47b5640 1 --2- 172.21.15.43:0/4006124147 >> [v2:172.21.15.203:6832/1317108840,v1:172.21.15.203:6833/1317108840] conn(0x7ff6d006a170 0x7ff6d006c630 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7ff6e8036000 tx=0x7ff6e8008030 comp rx=0 tx=0).stop 2023-08-09T13:52:15.094 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:52:15.091+0000 7ff6f47b5640 1 -- 172.21.15.43:0/4006124147 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff6ec15c680 msgr2=0x7ff6ec1433a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:52:15.094 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:52:15.091+0000 7ff6f47b5640 1 --2- 172.21.15.43:0/4006124147 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff6ec15c680 0x7ff6ec1433a0 secure :-1 s=READY pgs=1380 cs=0 l=1 rev1=1 crypto rx=0x7ff6dc00c820 tx=0x7ff6dc00cce0 comp rx=0 tx=0).stop 2023-08-09T13:52:15.094 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:52:15.091+0000 7ff6f47b5640 1 -- 172.21.15.43:0/4006124147 shutdown_connections 2023-08-09T13:52:15.094 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:52:15.091+0000 7ff6f47b5640 1 --2- 172.21.15.43:0/4006124147 >> [v2:172.21.15.203:6832/1317108840,v1:172.21.15.203:6833/1317108840] conn(0x7ff6d006a170 0x7ff6d006c630 unknown :-1 s=CLOSED pgs=75 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:52:15.094 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:52:15.091+0000 7ff6f47b5640 1 --2- 172.21.15.43:0/4006124147 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7ff6ec154b50 0x7ff6ec1f1410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:52:15.094 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:52:15.091+0000 7ff6f47b5640 1 --2- 172.21.15.43:0/4006124147 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff6ec15c680 0x7ff6ec1433a0 unknown :-1 s=CLOSED pgs=1380 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:52:15.095 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:52:15.091+0000 7ff6f47b5640 1 --2- 172.21.15.43:0/4006124147 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7ff6ec0d0a70 0x7ff6ec1f5bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:52:15.095 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:52:15.091+0000 7ff6f47b5640 1 -- 172.21.15.43:0/4006124147 >> 172.21.15.43:0/4006124147 conn(0x7ff6ec071520 msgr2=0x7ff6ec0b6fd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-09T13:52:15.095 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:52:15.091+0000 7ff6f47b5640 1 -- 172.21.15.43:0/4006124147 shutdown_connections 2023-08-09T13:52:15.095 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:52:15.091+0000 7ff6f47b5640 1 -- 172.21.15.43:0/4006124147 wait complete. 2023-08-09T13:52:15.112 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph mgr module enable snap_schedule 2023-08-09T13:52:15.349 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:52:15.346+0000 7f5ffdc4b640 1 Processor -- start 2023-08-09T13:52:15.349 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:52:15.346+0000 7f5ffdc4b640 1 -- start start 2023-08-09T13:52:15.350 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:52:15.348+0000 7f5ffdc4b640 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7f5ff80c8b80 0x7f5ff815b0f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:52:15.350 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:52:15.348+0000 7f5ffdc4b640 1 -- --> v1:172.21.15.43:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5ff806b480 con 0x7f5ff8157ce0 2023-08-09T13:52:15.351 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:52:15.348+0000 7f5ffdc4b640 1 -- --> v1:172.21.15.203:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5ff8071580 con 0x7f5ff806f2c0 2023-08-09T13:52:15.351 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:52:15.348+0000 7f5ffdc4b640 1 -- --> v2:172.21.15.203:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5ff81580c0 con 0x7f5ff80c8b80 2023-08-09T13:52:15.351 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:52:15.348+0000 7f5ffcc49640 1 --1- >> v1:172.21.15.43:6789/0 conn(0x7f5ff8157ce0 0x7f5ff806bc40 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.43:6789/0 says I am v1:172.21.15.43:60670/0 (socket says 172.21.15.43:60670) 2023-08-09T13:52:15.351 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:52:15.348+0000 7f5ffcc49640 1 -- 172.21.15.43:0/1340182040 learned_addr learned my addr 172.21.15.43:0/1340182040 (peer_addr_for_me v1:172.21.15.43:0/0) 2023-08-09T13:52:15.353 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:52:15.348+0000 7f5ffd44a640 1 --2- 172.21.15.43:0/1340182040 >> v2:172.21.15.203:3300/0 conn(0x7f5ff80c8b80 0x7f5ff815b0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:52:15.353 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:52:15.348+0000 7f5ff77fe640 1 -- 172.21.15.43:0/1340182040 <== mon.0 v1:172.21.15.43:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1379803771 0 0) 0x7f5ff806b480 con 0x7f5ff8157ce0 2023-08-09T13:52:15.353 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:52:15.349+0000 7f5ff77fe640 1 -- 172.21.15.43:0/1340182040 --> v1:172.21.15.43:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f5fe8003520 con 0x7f5ff8157ce0 2023-08-09T13:52:15.353 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:52:15.349+0000 7f5ff77fe640 1 -- 172.21.15.43:0/1340182040 <== mon.0 v1:172.21.15.43:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 130885331 0 0) 0x7f5fe8003520 con 0x7f5ff8157ce0 2023-08-09T13:52:15.353 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:52:15.349+0000 7f5ff77fe640 1 -- 172.21.15.43:0/1340182040 >> v2:172.21.15.203:3300/0 conn(0x7f5ff80c8b80 msgr2=0x7f5ff815b0f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-09T13:52:15.353 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:52:15.349+0000 7f5ff77fe640 1 --2- 172.21.15.43:0/1340182040 >> v2:172.21.15.203:3300/0 conn(0x7f5ff80c8b80 0x7f5ff815b0f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:52:15.354 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:52:15.349+0000 7f5ff77fe640 1 -- 172.21.15.43:0/1340182040 >> v1:172.21.15.203:6789/0 conn(0x7f5ff806f2c0 legacy=0x7f5ff80c8470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:52:15.354 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:52:15.349+0000 7f5ff77fe640 1 -- 172.21.15.43:0/1340182040 --> v1:172.21.15.43:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5ff806c350 con 0x7f5ff8157ce0 2023-08-09T13:52:15.354 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:52:15.349+0000 7f5ff77fe640 1 -- 172.21.15.43:0/1340182040 <== mon.0 v1:172.21.15.43:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1285150740 0 0) 0x7f5fec001670 con 0x7f5ff8157ce0 2023-08-09T13:52:15.354 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:52:15.349+0000 7f5ffd44a640 1 --2- 172.21.15.43:0/1340182040 >> v2:172.21.15.203:3300/0 conn(0x7f5ff80c8b80 0x7f5ff815b0f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-08-09T13:52:15.354 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:52:15.349+0000 7f5ff77fe640 1 -- 172.21.15.43:0/1340182040 >> v1:172.21.15.43:6789/0 conn(0x7f5ff8157ce0 legacy=0x7f5ff806bc40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:52:15.354 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:52:15.349+0000 7f5ff77fe640 1 --2- 172.21.15.43:0/1340182040 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f5fe8003970 0x7f5fe8003db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:52:15.354 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:52:15.349+0000 7f5ff77fe640 1 --2- 172.21.15.43:0/1340182040 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f5fe80047c0 0x7f5fe8004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:52:15.354 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:52:15.349+0000 7f5ff77fe640 1 --2- 172.21.15.43:0/1340182040 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f5fe80052a0 0x7f5fe800b980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:52:15.355 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:52:15.349+0000 7f5ff77fe640 1 -- 172.21.15.43:0/1340182040 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5ff81580c0 con 0x7f5fe80047c0 2023-08-09T13:52:15.355 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:52:15.349+0000 7f5ff77fe640 1 -- 172.21.15.43:0/1340182040 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5fe800c1a0 con 0x7f5fe80052a0 2023-08-09T13:52:15.355 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:52:15.349+0000 7f5ff77fe640 1 -- 172.21.15.43:0/1340182040 --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5fe800c360 con 0x7f5fe8003970 2023-08-09T13:52:15.355 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:52:15.349+0000 7f5ff7fff640 1 --2- 172.21.15.43:0/1340182040 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f5fe80047c0 0x7f5fe8004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:52:15.355 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:52:15.350+0000 7f5ffd44a640 1 --2- 172.21.15.43:0/1340182040 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f5fe80052a0 0x7f5fe800b980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:52:15.355 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:52:15.350+0000 7f5ffcc49640 1 --2- 172.21.15.43:0/1340182040 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f5fe8003970 0x7f5fe8003db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:52:15.355 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:52:15.350+0000 7f5ff7fff640 1 -- 172.21.15.43:0/1340182040 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f5fe8003970 msgr2=0x7f5fe8003db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-09T13:52:15.355 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:52:15.350+0000 7f5ff7fff640 1 --2- 172.21.15.43:0/1340182040 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f5fe8003970 0x7f5fe8003db0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:52:15.356 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:52:15.350+0000 7f5ff7fff640 1 -- 172.21.15.43:0/1340182040 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f5fe80052a0 msgr2=0x7f5fe800b980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-09T13:52:15.356 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:52:15.350+0000 7f5ff7fff640 1 --2- 172.21.15.43:0/1340182040 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f5fe80052a0 0x7f5fe800b980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:52:15.356 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:52:15.350+0000 7f5ff7fff640 1 -- 172.21.15.43:0/1340182040 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5fe8003520 con 0x7f5fe80047c0 2023-08-09T13:52:15.356 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:52:15.350+0000 7f5ffd44a640 1 -- 172.21.15.43:0/1340182040 reap_dead start 2023-08-09T13:52:15.358 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:52:15.351+0000 7f5ff7fff640 1 --2- 172.21.15.43:0/1340182040 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f5fe80047c0 0x7f5fe8004c40 secure :-1 s=READY pgs=1382 cs=0 l=1 rev1=1 crypto rx=0x7f5fe4001720 tx=0x7f5fe80057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d4ec35496b8f6ea4 server_cookie=0 in_seq=0 out_seq=0 2023-08-09T13:52:15.358 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:52:15.351+0000 7f5ff77fe640 1 -- 172.21.15.43:0/1340182040 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5fe40052d0 con 0x7f5fe80047c0 2023-08-09T13:52:15.358 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:52:15.351+0000 7f5ff77fe640 1 -- 172.21.15.43:0/1340182040 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5fe402e050 con 0x7f5fe80047c0 2023-08-09T13:52:15.358 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:52:15.351+0000 7f5ff77fe640 1 -- 172.21.15.43:0/1340182040 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5fe4003830 con 0x7f5fe80047c0 2023-08-09T13:52:15.358 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:52:15.352+0000 7f5ffdc4b640 1 -- 172.21.15.43:0/1340182040 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f5fe80047c0 msgr2=0x7f5fe8004c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:52:15.358 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:52:15.352+0000 7f5ffdc4b640 1 --2- 172.21.15.43:0/1340182040 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f5fe80047c0 0x7f5fe8004c40 secure :-1 s=READY pgs=1382 cs=0 l=1 rev1=1 crypto rx=0x7f5fe4001720 tx=0x7f5fe80057f0 comp rx=0 tx=0).stop 2023-08-09T13:52:15.358 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:52:15.352+0000 7f5ffdc4b640 1 -- 172.21.15.43:0/1340182040 shutdown_connections 2023-08-09T13:52:15.359 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:52:15.352+0000 7f5ffdc4b640 1 --2- 172.21.15.43:0/1340182040 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f5fe80047c0 0x7f5fe8004c40 unknown :-1 s=CLOSED pgs=1382 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:52:15.359 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:52:15.352+0000 7f5ffdc4b640 1 -- 172.21.15.43:0/1340182040 >> 172.21.15.43:0/1340182040 conn(0x7f5ff80af9f0 msgr2=0x7f5ff80afdd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-09T13:52:15.359 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:52:15.352+0000 7f5ffdc4b640 1 -- 172.21.15.43:0/1340182040 shutdown_connections 2023-08-09T13:52:15.359 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:52:15.352+0000 7f5ffdc4b640 1 -- 172.21.15.43:0/1340182040 wait complete. 2023-08-09T13:52:15.359 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:52:15.354+0000 7f5ffdc4b640 1 Processor -- start 2023-08-09T13:52:15.359 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:52:15.354+0000 7f5ffdc4b640 1 -- start start 2023-08-09T13:52:15.359 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:52:15.355+0000 7f5ffdc4b640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f5fe80047c0 0x7f5ff81f57c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:52:15.359 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:52:15.355+0000 7f5ffdc4b640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f5ff815af90 0x7f5ff815d080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:52:15.360 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:52:15.355+0000 7f5ffdc4b640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f5ff815d5c0 0x7f5ff80c1950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:52:15.360 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:52:15.355+0000 7f5ffdc4b640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5ff815da40 con 0x7f5ff815af90 2023-08-09T13:52:15.360 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:52:15.355+0000 7f5ffdc4b640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5ff815b710 con 0x7f5ff815d5c0 2023-08-09T13:52:15.360 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:52:15.355+0000 7f5ffdc4b640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5ff815b9f0 con 0x7f5fe80047c0 2023-08-09T13:52:15.360 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:52:15.355+0000 7f5ff7fff640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f5ff815af90 0x7f5ff815d080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:52:15.360 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:52:15.355+0000 7f5ff7fff640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f5ff815af90 0x7f5ff815d080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:57756/0 (socket says 172.21.15.43:57756) 2023-08-09T13:52:15.360 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:52:15.355+0000 7f5ff7fff640 1 -- 172.21.15.43:0/3865721993 learned_addr learned my addr 172.21.15.43:0/3865721993 (peer_addr_for_me v2:172.21.15.43:0/0) 2023-08-09T13:52:15.361 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:52:15.356+0000 7f5ffcc49640 1 --2- 172.21.15.43:0/3865721993 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f5fe80047c0 0x7f5ff81f57c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:52:15.362 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:52:15.356+0000 7f5ffd44a640 1 --2- 172.21.15.43:0/3865721993 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f5ff815d5c0 0x7f5ff80c1950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:52:15.362 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:52:15.356+0000 7f5ff7fff640 1 -- 172.21.15.43:0/3865721993 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f5fe80047c0 msgr2=0x7f5ff81f57c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:52:15.362 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:52:15.356+0000 7f5ff7fff640 1 --2- 172.21.15.43:0/3865721993 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f5fe80047c0 0x7f5ff81f57c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:52:15.362 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:52:15.356+0000 7f5ff7fff640 1 -- 172.21.15.43:0/3865721993 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f5ff815d5c0 msgr2=0x7f5ff80c1950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:52:15.362 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:52:15.356+0000 7f5ff7fff640 1 --2- 172.21.15.43:0/3865721993 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f5ff815d5c0 0x7f5ff80c1950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:52:15.362 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:52:15.356+0000 7f5ff7fff640 1 -- 172.21.15.43:0/3865721993 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5ff815bcd0 con 0x7f5ff815af90 2023-08-09T13:52:15.363 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:52:15.356+0000 7f5ffcc49640 1 --2- 172.21.15.43:0/3865721993 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f5fe80047c0 0x7f5ff81f57c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-08-09T13:52:15.363 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:52:15.356+0000 7f5ffd44a640 1 --2- 172.21.15.43:0/3865721993 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f5ff815d5c0 0x7f5ff80c1950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-08-09T13:52:15.363 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:52:15.356+0000 7f5ff7fff640 1 --2- 172.21.15.43:0/3865721993 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f5ff815af90 0x7f5ff815d080 secure :-1 s=READY pgs=1383 cs=0 l=1 rev1=1 crypto rx=0x7f5fe4005040 tx=0x7f5fe4001120 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-09T13:52:15.363 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:52:15.357+0000 7f5ff57fa640 1 -- 172.21.15.43:0/3865721993 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5fe4007a30 con 0x7f5ff815af90 2023-08-09T13:52:15.363 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:52:15.357+0000 7f5ff57fa640 1 -- 172.21.15.43:0/3865721993 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5fe40069e0 con 0x7f5ff815af90 2023-08-09T13:52:15.363 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:52:15.357+0000 7f5ff57fa640 1 -- 172.21.15.43:0/3865721993 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5fe4006d70 con 0x7f5ff815af90 2023-08-09T13:52:15.363 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:52:15.357+0000 7f5ffdc4b640 1 -- 172.21.15.43:0/3865721993 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5ff815bfb0 con 0x7f5ff815af90 2023-08-09T13:52:15.363 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:52:15.357+0000 7f5ffdc4b640 1 -- 172.21.15.43:0/3865721993 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5ff815b3f0 con 0x7f5ff815af90 2023-08-09T13:52:15.364 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:52:15.358+0000 7f5ffdc4b640 1 -- 172.21.15.43:0/3865721993 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5ff815d9a0 con 0x7f5ff815af90 2023-08-09T13:52:15.364 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:52:15.359+0000 7f5ff57fa640 1 -- 172.21.15.43:0/3865721993 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 45) v1 ==== 88443+0+0 (secure 0 0 0) 0x7f5fe40035f0 con 0x7f5ff815af90 2023-08-09T13:52:15.364 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:52:15.360+0000 7f5ff57fa640 1 --2- 172.21.15.43:0/3865721993 >> [v2:172.21.15.203:6832/1317108840,v1:172.21.15.203:6833/1317108840] conn(0x7f5fdc06a1a0 0x7f5fdc06c660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:52:15.364 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:52:15.360+0000 7f5ff57fa640 1 -- 172.21.15.43:0/3865721993 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(116..116 src has 1..116) v4 ==== 6426+0+0 (secure 0 0 0) 0x7f5fe40aab30 con 0x7f5ff815af90 2023-08-09T13:52:15.365 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:52:15.361+0000 7f5ffcc49640 1 --2- 172.21.15.43:0/3865721993 >> [v2:172.21.15.203:6832/1317108840,v1:172.21.15.203:6833/1317108840] conn(0x7f5fdc06a1a0 0x7f5fdc06c660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:52:15.365 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:52:15.362+0000 7f5ffcc49640 1 --2- 172.21.15.43:0/3865721993 >> [v2:172.21.15.203:6832/1317108840,v1:172.21.15.203:6833/1317108840] conn(0x7f5fdc06a1a0 0x7f5fdc06c660 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f5fec001200 tx=0x7f5fec009040 comp rx=0 tx=0).ready entity=mgr.6619 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-09T13:52:15.366 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:52:15.364+0000 7f5ff57fa640 1 -- 172.21.15.43:0/3865721993 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178066 (secure 0 0 0) 0x7f5fe4075df0 con 0x7f5ff815af90 2023-08-09T13:52:15.611 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:52:15.609+0000 7f5ffdc4b640 1 -- 172.21.15.43:0/3865721993 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "mgr module enable", "module": "snap_schedule"} v 0) v1 -- 0x7f5ff804e390 con 0x7f5ff815af90 2023-08-09T13:52:16.320 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:52:16.318+0000 7f5ff57fa640 1 -- 172.21.15.43:0/3865721993 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mgrmap(e 46) v1 ==== 88460+0+0 (secure 0 0 0) 0x7f5fe40761c0 con 0x7f5ff815af90 2023-08-09T13:52:16.322 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:52:16.320+0000 7f5ff57fa640 1 -- 172.21.15.43:0/3865721993 <== mon.0 v2:172.21.15.43:3300/0 8 ==== mon_command_ack([{"prefix": "mgr module enable", "module": "snap_schedule"}]=0 v46) v1 ==== 92+0+0 (secure 0 0 0) 0x7f5fe40a2c10 con 0x7f5ff815af90 2023-08-09T13:52:16.325 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:52:16.323+0000 7f5ffdc4b640 1 -- 172.21.15.43:0/3865721993 >> [v2:172.21.15.203:6832/1317108840,v1:172.21.15.203:6833/1317108840] conn(0x7f5fdc06a1a0 msgr2=0x7f5fdc06c660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:52:16.325 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:52:16.323+0000 7f5ffdc4b640 1 --2- 172.21.15.43:0/3865721993 >> [v2:172.21.15.203:6832/1317108840,v1:172.21.15.203:6833/1317108840] conn(0x7f5fdc06a1a0 0x7f5fdc06c660 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f5fec001200 tx=0x7f5fec009040 comp rx=0 tx=0).stop 2023-08-09T13:52:16.326 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:52:16.323+0000 7f5ffdc4b640 1 -- 172.21.15.43:0/3865721993 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f5ff815af90 msgr2=0x7f5ff815d080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:52:16.326 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:52:16.323+0000 7f5ffdc4b640 1 --2- 172.21.15.43:0/3865721993 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f5ff815af90 0x7f5ff815d080 secure :-1 s=READY pgs=1383 cs=0 l=1 rev1=1 crypto rx=0x7f5fe4005040 tx=0x7f5fe4001120 comp rx=0 tx=0).stop 2023-08-09T13:52:16.326 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:52:16.323+0000 7f5ffdc4b640 1 -- 172.21.15.43:0/3865721993 shutdown_connections 2023-08-09T13:52:16.327 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:52:16.323+0000 7f5ffdc4b640 1 --2- 172.21.15.43:0/3865721993 >> [v2:172.21.15.203:6832/1317108840,v1:172.21.15.203:6833/1317108840] conn(0x7f5fdc06a1a0 0x7f5fdc06c660 unknown :-1 s=CLOSED pgs=76 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:52:16.327 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:52:16.323+0000 7f5ffdc4b640 1 --2- 172.21.15.43:0/3865721993 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f5ff815d5c0 0x7f5ff80c1950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:52:16.327 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:52:16.323+0000 7f5ffdc4b640 1 --2- 172.21.15.43:0/3865721993 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f5ff815af90 0x7f5ff815d080 unknown :-1 s=CLOSED pgs=1383 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:52:16.327 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:52:16.323+0000 7f5ffdc4b640 1 --2- 172.21.15.43:0/3865721993 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f5fe80047c0 0x7f5ff81f57c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:52:16.327 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:52:16.323+0000 7f5ffdc4b640 1 -- 172.21.15.43:0/3865721993 >> 172.21.15.43:0/3865721993 conn(0x7f5ff80af9f0 msgr2=0x7f5ff80b0c00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-09T13:52:16.327 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:52:16.323+0000 7f5ffdc4b640 1 -- 172.21.15.43:0/3865721993 shutdown_connections 2023-08-09T13:52:16.327 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:52:16.323+0000 7f5ffdc4b640 1 -- 172.21.15.43:0/3865721993 wait complete. 2023-08-09T13:52:16.346 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mgr mgr/snap_schedule/allow_m_granularity True 2023-08-09T13:52:16.547 INFO:tasks.ceph.mgr.x.smithi203.stderr:2023-08-09T13:52:16.544+0000 7fafd242b040 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2023-08-09T13:52:16.565 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:52:16.562+0000 7f54e1077640 1 Processor -- start 2023-08-09T13:52:16.565 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:52:16.562+0000 7f54e1077640 1 -- start start 2023-08-09T13:52:16.566 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:52:16.563+0000 7f54e1077640 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7f54dc0d04b0 0x7f54dc0cd6d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:52:16.566 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:52:16.563+0000 7f54e1077640 1 -- --> v1:172.21.15.43:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f54dc0b7bd0 con 0x7f54dc0b45f0 2023-08-09T13:52:16.566 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:52:16.563+0000 7f54e1077640 1 -- --> v1:172.21.15.203:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f54dc0d1c00 con 0x7f54dc15c550 2023-08-09T13:52:16.566 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:52:16.563+0000 7f54e1077640 1 -- --> v2:172.21.15.203:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f54dc15c930 con 0x7f54dc0d04b0 2023-08-09T13:52:16.567 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:52:16.564+0000 7f54da59c640 1 --1- >> v1:172.21.15.43:6789/0 conn(0x7f54dc0b45f0 0x7f54dc0b59d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.43:6789/0 says I am v1:172.21.15.43:60678/0 (socket says 172.21.15.43:60678) 2023-08-09T13:52:16.567 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:52:16.564+0000 7f54da59c640 1 -- 172.21.15.43:0/4089188789 learned_addr learned my addr 172.21.15.43:0/4089188789 (peer_addr_for_me v1:172.21.15.43:0/0) 2023-08-09T13:52:16.567 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:52:16.564+0000 7f54db59e640 1 --2- 172.21.15.43:0/4089188789 >> v2:172.21.15.203:3300/0 conn(0x7f54dc0d04b0 0x7f54dc0cd6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:52:16.568 INFO:tasks.ceph.mgr.y.smithi043.stderr:2023-08-09T13:52:16.564+0000 7f6f07e0d040 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2023-08-09T13:52:16.569 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:52:16.564+0000 7f54d9d9b640 1 -- 172.21.15.43:0/4089188789 <== mon.0 v1:172.21.15.43:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4233099370 0 0) 0x7f54dc0b7bd0 con 0x7f54dc0b45f0 2023-08-09T13:52:16.569 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:52:16.564+0000 7f54d9d9b640 1 -- 172.21.15.43:0/4089188789 --> v1:172.21.15.43:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f54c4003520 con 0x7f54dc0b45f0 2023-08-09T13:52:16.569 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:52:16.565+0000 7f54db59e640 1 -- 172.21.15.43:0/4089188789 >> v1:172.21.15.203:6789/0 conn(0x7f54dc15c550 legacy=0x7f54dc0b3ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:52:16.569 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:52:16.565+0000 7f54db59e640 1 -- 172.21.15.43:0/4089188789 >> v1:172.21.15.43:6789/0 conn(0x7f54dc0b45f0 legacy=0x7f54dc0b59d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:52:16.569 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:52:16.565+0000 7f54db59e640 1 -- 172.21.15.43:0/4089188789 --> v2:172.21.15.203:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f54dc0b54f0 con 0x7f54dc0d04b0 2023-08-09T13:52:16.570 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:52:16.566+0000 7f54db59e640 1 --2- 172.21.15.43:0/4089188789 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f54dc0d04b0 0x7f54dc0cd6d0 secure :-1 s=READY pgs=509 cs=0 l=1 rev1=1 crypto rx=0x7f54d00097a0 tx=0x7f54d002b960 comp rx=0 tx=0).ready entity=mon.1 client_cookie=62eca48cb09d23d4 server_cookie=0 in_seq=0 out_seq=0 2023-08-09T13:52:16.570 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:52:16.566+0000 7f54d9d9b640 1 -- 172.21.15.43:0/4089188789 <== mon.1 v2:172.21.15.203:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f54d0030610 con 0x7f54dc0d04b0 2023-08-09T13:52:16.570 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:52:16.566+0000 7f54d9d9b640 1 -- 172.21.15.43:0/4089188789 <== mon.1 v2:172.21.15.203:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f54d0030770 con 0x7f54dc0d04b0 2023-08-09T13:52:16.570 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:52:16.566+0000 7f54d9d9b640 1 -- 172.21.15.43:0/4089188789 <== mon.1 v2:172.21.15.203:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f54d0030ae0 con 0x7f54dc0d04b0 2023-08-09T13:52:16.571 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:52:16.567+0000 7f54e1077640 1 -- 172.21.15.43:0/4089188789 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f54dc0d04b0 msgr2=0x7f54dc0cd6d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:52:16.571 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:52:16.567+0000 7f54e1077640 1 --2- 172.21.15.43:0/4089188789 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f54dc0d04b0 0x7f54dc0cd6d0 secure :-1 s=READY pgs=509 cs=0 l=1 rev1=1 crypto rx=0x7f54d00097a0 tx=0x7f54d002b960 comp rx=0 tx=0).stop 2023-08-09T13:52:16.571 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:52:16.567+0000 7f54e1077640 1 -- 172.21.15.43:0/4089188789 shutdown_connections 2023-08-09T13:52:16.572 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:52:16.567+0000 7f54e1077640 1 --2- 172.21.15.43:0/4089188789 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f54dc0d04b0 0x7f54dc0cd6d0 unknown :-1 s=CLOSED pgs=509 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:52:16.572 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:52:16.567+0000 7f54e1077640 1 -- 172.21.15.43:0/4089188789 >> 172.21.15.43:0/4089188789 conn(0x7f54dc071520 msgr2=0x7f54dc0ceee0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-09T13:52:16.572 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:52:16.567+0000 7f54e1077640 1 -- 172.21.15.43:0/4089188789 shutdown_connections 2023-08-09T13:52:16.572 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:52:16.567+0000 7f54e1077640 1 -- 172.21.15.43:0/4089188789 wait complete. 2023-08-09T13:52:16.574 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:52:16.569+0000 7f54e1077640 1 Processor -- start 2023-08-09T13:52:16.574 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:52:16.569+0000 7f54e1077640 1 -- start start 2023-08-09T13:52:16.574 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:52:16.570+0000 7f54e1077640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f54dc0b45f0 0x7f54dc159ec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:52:16.574 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:52:16.570+0000 7f54e1077640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f54dc0d04b0 0x7f54dc0ca450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:52:16.575 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:52:16.570+0000 7f54e1077640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f54dc15c550 0x7f54dc1f1d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:52:16.575 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:52:16.570+0000 7f54e1077640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f54dc15a400 con 0x7f54dc0d04b0 2023-08-09T13:52:16.575 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:52:16.570+0000 7f54e1077640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f54dc1f18d0 con 0x7f54dc15c550 2023-08-09T13:52:16.575 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:52:16.570+0000 7f54e1077640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f54dc0caea0 con 0x7f54dc0b45f0 2023-08-09T13:52:16.575 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:52:16.571+0000 7f54da59c640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f54dc0d04b0 0x7f54dc0ca450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:52:16.576 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:52:16.571+0000 7f54da59c640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f54dc0d04b0 0x7f54dc0ca450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:57796/0 (socket says 172.21.15.43:57796) 2023-08-09T13:52:16.576 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:52:16.571+0000 7f54da59c640 1 -- 172.21.15.43:0/347563317 learned_addr learned my addr 172.21.15.43:0/347563317 (peer_addr_for_me v2:172.21.15.43:0/0) 2023-08-09T13:52:16.576 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:52:16.571+0000 7f54dad9d640 1 --2- 172.21.15.43:0/347563317 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f54dc0b45f0 0x7f54dc159ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:52:16.576 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:52:16.571+0000 7f54db59e640 1 --2- 172.21.15.43:0/347563317 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f54dc15c550 0x7f54dc1f1d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:52:16.576 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:52:16.571+0000 7f54da59c640 1 -- 172.21.15.43:0/347563317 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f54dc0b45f0 msgr2=0x7f54dc159ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:52:16.576 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:52:16.571+0000 7f54da59c640 1 --2- 172.21.15.43:0/347563317 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f54dc0b45f0 0x7f54dc159ec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:52:16.577 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:52:16.571+0000 7f54da59c640 1 -- 172.21.15.43:0/347563317 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f54dc15c550 msgr2=0x7f54dc1f1d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:52:16.577 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:52:16.571+0000 7f54da59c640 1 --2- 172.21.15.43:0/347563317 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f54dc15c550 0x7f54dc1f1d80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:52:16.577 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:52:16.571+0000 7f54da59c640 1 -- 172.21.15.43:0/347563317 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f54d0009400 con 0x7f54dc0d04b0 2023-08-09T13:52:16.578 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:52:16.572+0000 7f54da59c640 1 --2- 172.21.15.43:0/347563317 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f54dc0d04b0 0x7f54dc0ca450 secure :-1 s=READY pgs=1388 cs=0 l=1 rev1=1 crypto rx=0x7f54cc00c500 tx=0x7f54cc00c9c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-09T13:52:16.579 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:52:16.572+0000 7f54c37fe640 1 -- 172.21.15.43:0/347563317 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f54cc005ce0 con 0x7f54dc0d04b0 2023-08-09T13:52:16.580 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:52:16.572+0000 7f54c37fe640 1 -- 172.21.15.43:0/347563317 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f54cc005e40 con 0x7f54dc0d04b0 2023-08-09T13:52:16.580 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:52:16.572+0000 7f54c37fe640 1 -- 172.21.15.43:0/347563317 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f54cc007330 con 0x7f54dc0d04b0 2023-08-09T13:52:16.580 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:52:16.572+0000 7f54e1077640 1 -- 172.21.15.43:0/347563317 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f54dc0cb180 con 0x7f54dc0d04b0 2023-08-09T13:52:16.580 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:52:16.572+0000 7f54e1077640 1 -- 172.21.15.43:0/347563317 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f54dc0cb740 con 0x7f54dc0d04b0 2023-08-09T13:52:16.580 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:52:16.573+0000 7f54e1077640 1 -- 172.21.15.43:0/347563317 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f54dc1f1aa0 con 0x7f54dc0d04b0 2023-08-09T13:52:16.580 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:52:16.574+0000 7f54c37fe640 1 -- 172.21.15.43:0/347563317 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 46) v1 ==== 88460+0+0 (secure 0 0 0) 0x7f54cc007550 con 0x7f54dc0d04b0 2023-08-09T13:52:16.580 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:52:16.575+0000 7f54c37fe640 1 --2- 172.21.15.43:0/347563317 >> [v2:172.21.15.203:6832/1317108840,v1:172.21.15.203:6833/1317108840] conn(0x7f54c806a1c0 0x7f54c806c680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:52:16.581 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:52:16.575+0000 7f54c37fe640 1 -- 172.21.15.43:0/347563317 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(116..116 src has 1..116) v4 ==== 6426+0+0 (secure 0 0 0) 0x7f54cc087f20 con 0x7f54dc0d04b0 2023-08-09T13:52:16.581 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:52:16.576+0000 7f54dad9d640 1 -- 172.21.15.43:0/347563317 >> [v2:172.21.15.203:6832/1317108840,v1:172.21.15.203:6833/1317108840] conn(0x7f54c806a1c0 msgr2=0x7f54c806c680 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.203:6832/1317108840 2023-08-09T13:52:16.581 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:52:16.576+0000 7f54dad9d640 1 --2- 172.21.15.43:0/347563317 >> [v2:172.21.15.203:6832/1317108840,v1:172.21.15.203:6833/1317108840] conn(0x7f54c806a1c0 0x7f54c806c680 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-08-09T13:52:16.582 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:52:16.579+0000 7f54c37fe640 1 -- 172.21.15.43:0/347563317 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178066 (secure 0 0 0) 0x7f54dc1f1aa0 con 0x7f54dc0d04b0 2023-08-09T13:52:16.780 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:52:16.777+0000 7f54dad9d640 1 -- 172.21.15.43:0/347563317 >> [v2:172.21.15.203:6832/1317108840,v1:172.21.15.203:6833/1317108840] conn(0x7f54c806a1c0 msgr2=0x7f54c806c680 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.203:6832/1317108840 2023-08-09T13:52:16.780 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:52:16.777+0000 7f54dad9d640 1 --2- 172.21.15.43:0/347563317 >> [v2:172.21.15.203:6832/1317108840,v1:172.21.15.203:6833/1317108840] conn(0x7f54c806a1c0 0x7f54c806c680 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2023-08-09T13:52:16.780 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:52:16.777+0000 7f54e1077640 1 -- 172.21.15.43:0/347563317 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command([{prefix=config set, name=mgr/snap_schedule/allow_m_granularity}] v 0) v1 -- 0x7f54dc1f1a40 con 0x7f54dc0d04b0 2023-08-09T13:52:16.786 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:52:16.783+0000 7f54c37fe640 1 -- 172.21.15.43:0/347563317 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mgr/snap_schedule/allow_m_granularity}]=0 v35)=0 v35) v1 ==== 138+0+0 (secure 0 0 0) 0x7f54cc017020 con 0x7f54dc0d04b0 2023-08-09T13:52:16.789 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:52:16.786+0000 7f54e1077640 1 -- 172.21.15.43:0/347563317 >> [v2:172.21.15.203:6832/1317108840,v1:172.21.15.203:6833/1317108840] conn(0x7f54c806a1c0 msgr2=0x7f54c806c680 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-08-09T13:52:16.789 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:52:16.786+0000 7f54e1077640 1 --2- 172.21.15.43:0/347563317 >> [v2:172.21.15.203:6832/1317108840,v1:172.21.15.203:6833/1317108840] conn(0x7f54c806a1c0 0x7f54c806c680 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:52:16.789 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:52:16.786+0000 7f54e1077640 1 -- 172.21.15.43:0/347563317 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f54dc0d04b0 msgr2=0x7f54dc0ca450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:52:16.789 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:52:16.786+0000 7f54e1077640 1 --2- 172.21.15.43:0/347563317 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f54dc0d04b0 0x7f54dc0ca450 secure :-1 s=READY pgs=1388 cs=0 l=1 rev1=1 crypto rx=0x7f54cc00c500 tx=0x7f54cc00c9c0 comp rx=0 tx=0).stop 2023-08-09T13:52:16.789 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:52:16.786+0000 7f54e1077640 1 -- 172.21.15.43:0/347563317 shutdown_connections 2023-08-09T13:52:16.790 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:52:16.786+0000 7f54e1077640 1 --2- 172.21.15.43:0/347563317 >> [v2:172.21.15.203:6832/1317108840,v1:172.21.15.203:6833/1317108840] conn(0x7f54c806a1c0 0x7f54c806c680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:52:16.790 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:52:16.786+0000 7f54e1077640 1 --2- 172.21.15.43:0/347563317 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f54dc15c550 0x7f54dc1f1d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:52:16.790 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:52:16.786+0000 7f54e1077640 1 --2- 172.21.15.43:0/347563317 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f54dc0d04b0 0x7f54dc0ca450 unknown :-1 s=CLOSED pgs=1388 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:52:16.790 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:52:16.786+0000 7f54e1077640 1 --2- 172.21.15.43:0/347563317 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f54dc0b45f0 0x7f54dc159ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:52:16.790 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:52:16.786+0000 7f54e1077640 1 -- 172.21.15.43:0/347563317 >> 172.21.15.43:0/347563317 conn(0x7f54dc071520 msgr2=0x7f54dc0c7f80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-09T13:52:16.791 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:52:16.786+0000 7f54e1077640 1 -- 172.21.15.43:0/347563317 shutdown_connections 2023-08-09T13:52:16.791 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:52:16.786+0000 7f54e1077640 1 -- 172.21.15.43:0/347563317 wait complete. 2023-08-09T13:52:16.809 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mgr mgr/snap_schedule/dump_on_update True 2023-08-09T13:52:17.033 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:52:17.030+0000 7fd25f2d9640 1 Processor -- start 2023-08-09T13:52:17.033 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:52:17.030+0000 7fd25f2d9640 1 -- start start 2023-08-09T13:52:17.034 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:52:17.032+0000 7fd25f2d9640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fd2580d04b0 0x7fd2580cd6d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:52:17.035 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:52:17.032+0000 7fd25f2d9640 1 -- --> v1:172.21.15.43:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd2580b7bd0 con 0x7fd2580b45f0 2023-08-09T13:52:17.035 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:52:17.032+0000 7fd25f2d9640 1 -- --> v1:172.21.15.203:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd2580d1c00 con 0x7fd25815c550 2023-08-09T13:52:17.035 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:52:17.032+0000 7fd25f2d9640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd25815c930 con 0x7fd2580d04b0 2023-08-09T13:52:17.035 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:52:17.032+0000 7fd25c874640 1 --1- >> v1:172.21.15.43:6789/0 conn(0x7fd2580b45f0 0x7fd2580b59d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.43:6789/0 says I am v1:172.21.15.43:60688/0 (socket says 172.21.15.43:60688) 2023-08-09T13:52:17.035 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:52:17.032+0000 7fd25c874640 1 -- 172.21.15.43:0/2907376211 learned_addr learned my addr 172.21.15.43:0/2907376211 (peer_addr_for_me v1:172.21.15.43:0/0) 2023-08-09T13:52:17.037 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:52:17.033+0000 7fd25d876640 1 --2- 172.21.15.43:0/2907376211 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fd2580d04b0 0x7fd2580cd6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:52:17.037 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:52:17.033+0000 7fd24ffff640 1 -- 172.21.15.43:0/2907376211 <== mon.0 v1:172.21.15.43:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3139710700 0 0) 0x7fd2580b7bd0 con 0x7fd2580b45f0 2023-08-09T13:52:17.037 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:52:17.033+0000 7fd24ffff640 1 -- 172.21.15.43:0/2907376211 --> v1:172.21.15.43:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fd244003520 con 0x7fd2580b45f0 2023-08-09T13:52:17.037 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:52:17.033+0000 7fd24ffff640 1 -- 172.21.15.43:0/2907376211 <== mon.0 v1:172.21.15.43:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3094028854 0 0) 0x7fd244003520 con 0x7fd2580b45f0 2023-08-09T13:52:17.037 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:52:17.033+0000 7fd24ffff640 1 -- 172.21.15.43:0/2907376211 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fd2580d04b0 msgr2=0x7fd2580cd6d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-09T13:52:17.037 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:52:17.033+0000 7fd24ffff640 1 --2- 172.21.15.43:0/2907376211 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fd2580d04b0 0x7fd2580cd6d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:52:17.038 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:52:17.033+0000 7fd24ffff640 1 -- 172.21.15.43:0/2907376211 >> v1:172.21.15.203:6789/0 conn(0x7fd25815c550 legacy=0x7fd2580b3ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:52:17.038 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:52:17.033+0000 7fd24ffff640 1 -- 172.21.15.43:0/2907376211 --> v1:172.21.15.43:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd2580b54f0 con 0x7fd2580b45f0 2023-08-09T13:52:17.038 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:52:17.033+0000 7fd24ffff640 1 -- 172.21.15.43:0/2907376211 <== mon.0 v1:172.21.15.43:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1285150740 0 0) 0x7fd248001670 con 0x7fd2580b45f0 2023-08-09T13:52:17.038 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:52:17.033+0000 7fd24ffff640 1 -- 172.21.15.43:0/2907376211 >> v1:172.21.15.43:6789/0 conn(0x7fd2580b45f0 legacy=0x7fd2580b59d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:52:17.039 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:52:17.033+0000 7fd24ffff640 1 --2- 172.21.15.43:0/2907376211 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fd2580d04b0 0x7fd2440039a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:52:17.039 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:52:17.033+0000 7fd24ffff640 1 --2- 172.21.15.43:0/2907376211 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fd244003f20 0x7fd2440047c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:52:17.039 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:52:17.034+0000 7fd24ffff640 1 --2- 172.21.15.43:0/2907376211 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fd244004d00 0x7fd244005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:52:17.039 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:52:17.034+0000 7fd24ffff640 1 -- 172.21.15.43:0/2907376211 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd25815c930 con 0x7fd244003f20 2023-08-09T13:52:17.039 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:52:17.034+0000 7fd24ffff640 1 -- 172.21.15.43:0/2907376211 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd24400ba60 con 0x7fd244004d00 2023-08-09T13:52:17.039 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:52:17.034+0000 7fd24ffff640 1 -- 172.21.15.43:0/2907376211 --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd24400bc20 con 0x7fd2580d04b0 2023-08-09T13:52:17.039 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:52:17.034+0000 7fd25c874640 1 --2- 172.21.15.43:0/2907376211 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fd244003f20 0x7fd2440047c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:52:17.039 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:52:17.034+0000 7fd25d075640 1 --2- 172.21.15.43:0/2907376211 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fd2580d04b0 0x7fd2440039a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:52:17.040 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:52:17.034+0000 7fd25d876640 1 --2- 172.21.15.43:0/2907376211 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fd244004d00 0x7fd244005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:52:17.040 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:52:17.034+0000 7fd25c874640 1 -- 172.21.15.43:0/2907376211 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fd2580d04b0 msgr2=0x7fd2440039a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-09T13:52:17.040 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:52:17.034+0000 7fd25c874640 1 --2- 172.21.15.43:0/2907376211 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fd2580d04b0 0x7fd2440039a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:52:17.040 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:52:17.034+0000 7fd25c874640 1 -- 172.21.15.43:0/2907376211 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fd244004d00 msgr2=0x7fd244005180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-09T13:52:17.040 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:52:17.034+0000 7fd25c874640 1 --2- 172.21.15.43:0/2907376211 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fd244004d00 0x7fd244005180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:52:17.040 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:52:17.034+0000 7fd25c874640 1 -- 172.21.15.43:0/2907376211 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd244003520 con 0x7fd244003f20 2023-08-09T13:52:17.041 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:52:17.036+0000 7fd25c874640 1 --2- 172.21.15.43:0/2907376211 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fd244003f20 0x7fd2440047c0 secure :-1 s=READY pgs=1390 cs=0 l=1 rev1=1 crypto rx=0x7fd24800c190 tx=0x7fd2440057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cb097a650a6270f3 server_cookie=0 in_seq=0 out_seq=0 2023-08-09T13:52:17.042 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:52:17.036+0000 7fd24ffff640 1 -- 172.21.15.43:0/2907376211 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd248005ad0 con 0x7fd244003f20 2023-08-09T13:52:17.042 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:52:17.036+0000 7fd24ffff640 1 -- 172.21.15.43:0/2907376211 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd248005c50 con 0x7fd244003f20 2023-08-09T13:52:17.042 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:52:17.036+0000 7fd24ffff640 1 -- 172.21.15.43:0/2907376211 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd24802c8a0 con 0x7fd244003f20 2023-08-09T13:52:17.042 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:52:17.037+0000 7fd25f2d9640 1 -- 172.21.15.43:0/2907376211 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fd244003f20 msgr2=0x7fd2440047c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:52:17.042 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:52:17.037+0000 7fd25f2d9640 1 --2- 172.21.15.43:0/2907376211 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fd244003f20 0x7fd2440047c0 secure :-1 s=READY pgs=1390 cs=0 l=1 rev1=1 crypto rx=0x7fd24800c190 tx=0x7fd2440057f0 comp rx=0 tx=0).stop 2023-08-09T13:52:17.042 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:52:17.037+0000 7fd25d876640 1 -- 172.21.15.43:0/2907376211 reap_dead start 2023-08-09T13:52:17.042 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:52:17.037+0000 7fd25f2d9640 1 -- 172.21.15.43:0/2907376211 shutdown_connections 2023-08-09T13:52:17.043 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:52:17.037+0000 7fd25f2d9640 1 -- 172.21.15.43:0/2907376211 >> 172.21.15.43:0/2907376211 conn(0x7fd258071520 msgr2=0x7fd2580ceee0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-09T13:52:17.044 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:52:17.037+0000 7fd25f2d9640 1 -- 172.21.15.43:0/2907376211 shutdown_connections 2023-08-09T13:52:17.044 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:52:17.037+0000 7fd25f2d9640 1 -- 172.21.15.43:0/2907376211 wait complete. 2023-08-09T13:52:17.044 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:52:17.039+0000 7fd25f2d9640 1 Processor -- start 2023-08-09T13:52:17.044 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:52:17.039+0000 7fd25f2d9640 1 -- start start 2023-08-09T13:52:17.048 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:52:17.040+0000 7fd25f2d9640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fd244003f20 0x7fd2580caaa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:52:17.048 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:52:17.040+0000 7fd25f2d9640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fd2580cc120 0x7fd258157190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:52:17.049 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:52:17.040+0000 7fd25f2d9640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fd258148730 0x7fd25815cec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:52:17.049 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:52:17.040+0000 7fd25f2d9640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd258159070 con 0x7fd244003f20 2023-08-09T13:52:17.049 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:52:17.040+0000 7fd25f2d9640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd2580cc5a0 con 0x7fd258148730 2023-08-09T13:52:17.050 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:52:17.040+0000 7fd25f2d9640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd2580cb080 con 0x7fd2580cc120 2023-08-09T13:52:17.050 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:52:17.040+0000 7fd25d075640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fd244003f20 0x7fd2580caaa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:52:17.050 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:52:17.040+0000 7fd25d075640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fd244003f20 0x7fd2580caaa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:57820/0 (socket says 172.21.15.43:57820) 2023-08-09T13:52:17.050 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:52:17.040+0000 7fd25d075640 1 -- 172.21.15.43:0/2980737255 learned_addr learned my addr 172.21.15.43:0/2980737255 (peer_addr_for_me v2:172.21.15.43:0/0) 2023-08-09T13:52:17.050 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:52:17.041+0000 7fd25c874640 1 --2- 172.21.15.43:0/2980737255 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fd2580cc120 0x7fd258157190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:52:17.050 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:52:17.041+0000 7fd25d876640 1 --2- 172.21.15.43:0/2980737255 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fd258148730 0x7fd25815cec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:52:17.050 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:52:17.041+0000 7fd25d075640 1 -- 172.21.15.43:0/2980737255 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fd2580cc120 msgr2=0x7fd258157190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:52:17.051 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:52:17.041+0000 7fd25d075640 1 --2- 172.21.15.43:0/2980737255 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fd2580cc120 0x7fd258157190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:52:17.051 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:52:17.041+0000 7fd25d075640 1 -- 172.21.15.43:0/2980737255 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fd258148730 msgr2=0x7fd25815cec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:52:17.051 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:52:17.041+0000 7fd25d075640 1 --2- 172.21.15.43:0/2980737255 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fd258148730 0x7fd25815cec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:52:17.051 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:52:17.041+0000 7fd25d075640 1 -- 172.21.15.43:0/2980737255 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd2580cb360 con 0x7fd244003f20 2023-08-09T13:52:17.051 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:52:17.041+0000 7fd25d876640 1 --2- 172.21.15.43:0/2980737255 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fd258148730 0x7fd25815cec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-08-09T13:52:17.051 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:52:17.041+0000 7fd25c874640 1 --2- 172.21.15.43:0/2980737255 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fd2580cc120 0x7fd258157190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-08-09T13:52:17.051 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:52:17.041+0000 7fd25d075640 1 --2- 172.21.15.43:0/2980737255 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fd244003f20 0x7fd2580caaa0 secure :-1 s=READY pgs=1391 cs=0 l=1 rev1=1 crypto rx=0x7fd25400e550 tx=0x7fd25400ea10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-09T13:52:17.052 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:52:17.042+0000 7fd24dffb640 1 -- 172.21.15.43:0/2980737255 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd2540052e0 con 0x7fd244003f20 2023-08-09T13:52:17.052 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:52:17.042+0000 7fd24dffb640 1 -- 172.21.15.43:0/2980737255 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd254005440 con 0x7fd244003f20 2023-08-09T13:52:17.052 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:52:17.042+0000 7fd24dffb640 1 -- 172.21.15.43:0/2980737255 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd254003810 con 0x7fd244003f20 2023-08-09T13:52:17.052 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:52:17.042+0000 7fd25f2d9640 1 -- 172.21.15.43:0/2980737255 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd2580cb640 con 0x7fd244003f20 2023-08-09T13:52:17.053 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:52:17.042+0000 7fd25f2d9640 1 -- 172.21.15.43:0/2980737255 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd2580cbc00 con 0x7fd244003f20 2023-08-09T13:52:17.053 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:52:17.043+0000 7fd25f2d9640 1 -- 172.21.15.43:0/2980737255 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd2580cbd90 con 0x7fd244003f20 2023-08-09T13:52:17.053 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:52:17.044+0000 7fd24dffb640 1 -- 172.21.15.43:0/2980737255 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 46) v1 ==== 88460+0+0 (secure 0 0 0) 0x7fd254003970 con 0x7fd244003f20 2023-08-09T13:52:17.053 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:52:17.045+0000 7fd24dffb640 1 --2- 172.21.15.43:0/2980737255 >> [v2:172.21.15.203:6832/1317108840,v1:172.21.15.203:6833/1317108840] conn(0x7fd23c06a1f0 0x7fd23c06c6b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:52:17.053 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:52:17.045+0000 7fd24dffb640 1 -- 172.21.15.43:0/2980737255 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(116..116 src has 1..116) v4 ==== 6426+0+0 (secure 0 0 0) 0x7fd254088660 con 0x7fd244003f20 2023-08-09T13:52:17.054 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:52:17.045+0000 7fd25c874640 1 -- 172.21.15.43:0/2980737255 >> [v2:172.21.15.203:6832/1317108840,v1:172.21.15.203:6833/1317108840] conn(0x7fd23c06a1f0 msgr2=0x7fd23c06c6b0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.203:6832/1317108840 2023-08-09T13:52:17.054 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:52:17.045+0000 7fd25c874640 1 --2- 172.21.15.43:0/2980737255 >> [v2:172.21.15.203:6832/1317108840,v1:172.21.15.203:6833/1317108840] conn(0x7fd23c06a1f0 0x7fd23c06c6b0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-08-09T13:52:17.055 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:52:17.053+0000 7fd24dffb640 1 -- 172.21.15.43:0/2980737255 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178066 (secure 0 0 0) 0x7fd254053ad0 con 0x7fd244003f20 2023-08-09T13:52:17.248 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:52:17.245+0000 7fd25f2d9640 1 -- 172.21.15.43:0/2980737255 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command([{prefix=config set, name=mgr/snap_schedule/dump_on_update}] v 0) v1 -- 0x7fd2580739f0 con 0x7fd244003f20 2023-08-09T13:52:17.248 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:52:17.246+0000 7fd25c874640 1 -- 172.21.15.43:0/2980737255 >> [v2:172.21.15.203:6832/1317108840,v1:172.21.15.203:6833/1317108840] conn(0x7fd23c06a1f0 msgr2=0x7fd23c06c6b0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.203:6832/1317108840 2023-08-09T13:52:17.248 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:52:17.246+0000 7fd25c874640 1 --2- 172.21.15.43:0/2980737255 >> [v2:172.21.15.203:6832/1317108840,v1:172.21.15.203:6833/1317108840] conn(0x7fd23c06a1f0 0x7fd23c06c6b0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2023-08-09T13:52:17.255 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:52:17.253+0000 7fd24dffb640 1 -- 172.21.15.43:0/2980737255 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mgr/snap_schedule/dump_on_update}]=0 v36)=0 v36) v1 ==== 133+0+0 (secure 0 0 0) 0x7fd2540570f0 con 0x7fd244003f20 2023-08-09T13:52:17.258 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:52:17.256+0000 7fd25f2d9640 1 -- 172.21.15.43:0/2980737255 >> [v2:172.21.15.203:6832/1317108840,v1:172.21.15.203:6833/1317108840] conn(0x7fd23c06a1f0 msgr2=0x7fd23c06c6b0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-08-09T13:52:17.258 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:52:17.256+0000 7fd25f2d9640 1 --2- 172.21.15.43:0/2980737255 >> [v2:172.21.15.203:6832/1317108840,v1:172.21.15.203:6833/1317108840] conn(0x7fd23c06a1f0 0x7fd23c06c6b0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:52:17.258 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:52:17.256+0000 7fd25f2d9640 1 -- 172.21.15.43:0/2980737255 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fd244003f20 msgr2=0x7fd2580caaa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:52:17.259 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:52:17.256+0000 7fd25f2d9640 1 --2- 172.21.15.43:0/2980737255 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fd244003f20 0x7fd2580caaa0 secure :-1 s=READY pgs=1391 cs=0 l=1 rev1=1 crypto rx=0x7fd25400e550 tx=0x7fd25400ea10 comp rx=0 tx=0).stop 2023-08-09T13:52:17.259 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:52:17.256+0000 7fd25f2d9640 1 -- 172.21.15.43:0/2980737255 shutdown_connections 2023-08-09T13:52:17.259 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:52:17.256+0000 7fd25f2d9640 1 --2- 172.21.15.43:0/2980737255 >> [v2:172.21.15.203:6832/1317108840,v1:172.21.15.203:6833/1317108840] conn(0x7fd23c06a1f0 0x7fd23c06c6b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:52:17.260 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:52:17.256+0000 7fd25f2d9640 1 --2- 172.21.15.43:0/2980737255 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fd258148730 0x7fd25815cec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:52:17.260 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:52:17.256+0000 7fd25f2d9640 1 --2- 172.21.15.43:0/2980737255 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fd2580cc120 0x7fd258157190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:52:17.260 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:52:17.256+0000 7fd25f2d9640 1 --2- 172.21.15.43:0/2980737255 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fd244003f20 0x7fd2580caaa0 unknown :-1 s=CLOSED pgs=1391 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:52:17.260 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:52:17.256+0000 7fd25f2d9640 1 -- 172.21.15.43:0/2980737255 >> 172.21.15.43:0/2980737255 conn(0x7fd258071520 msgr2=0x7fd2580b6cd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-09T13:52:17.260 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:52:17.256+0000 7fd25f2d9640 1 -- 172.21.15.43:0/2980737255 shutdown_connections 2023-08-09T13:52:17.261 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:52:17.256+0000 7fd25f2d9640 1 -- 172.21.15.43:0/2980737255 wait complete. 2023-08-09T13:52:17.266 INFO:tasks.ceph.mgr.x.smithi203.stderr:2023-08-09T13:52:17.263+0000 7fafd242b040 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2023-08-09T13:52:17.280 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs snap-schedule add --fs cephfs /bad-path 1M 2023-08-09T13:52:17.346 INFO:tasks.ceph.mgr.y.smithi043.stderr:2023-08-09T13:52:17.343+0000 7f6f07e0d040 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2023-08-09T13:52:17.387 INFO:tasks.ceph.mgr.x.smithi203.stderr:2023-08-09T13:52:17.384+0000 7fafd242b040 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2023-08-09T13:52:17.443 INFO:tasks.ceph.mgr.x.smithi203.stderr:2023-08-09T13:52:17.440+0000 7fafd242b040 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2023-08-09T13:52:17.473 INFO:tasks.ceph.mgr.y.smithi043.stderr:2023-08-09T13:52:17.470+0000 7f6f07e0d040 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2023-08-09T13:52:17.480 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:52:17.478+0000 7f68e4999640 1 Processor -- start 2023-08-09T13:52:17.481 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:52:17.478+0000 7f68e4999640 1 -- start start 2023-08-09T13:52:17.482 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:52:17.480+0000 7f68e4999640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f68e015a560 0x7f68e00d09d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:52:17.482 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:52:17.480+0000 7f68e4999640 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7f68e00d0f10 0x7f68e00cccd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:52:17.482 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:52:17.480+0000 7f68e4999640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f68e00cd210 0x7f68e00b0f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:52:17.482 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:52:17.480+0000 7f68e4999640 1 -- --> v2:172.21.15.43:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f68e015a940 con 0x7f68e015a560 2023-08-09T13:52:17.483 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:52:17.480+0000 7f68e4999640 1 -- --> v2:172.21.15.203:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f68e00cd6a0 con 0x7f68e00d0f10 2023-08-09T13:52:17.483 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:52:17.480+0000 7f68e4999640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f68e00cbde0 con 0x7f68e00cd210 2023-08-09T13:52:17.484 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:52:17.480+0000 7f68de59c640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f68e015a560 0x7f68e00d09d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:52:17.484 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:52:17.480+0000 7f68de59c640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f68e015a560 0x7f68e00d09d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:57832/0 (socket says 172.21.15.43:57832) 2023-08-09T13:52:17.484 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:52:17.480+0000 7f68de59c640 1 -- 172.21.15.43:0/3397256233 learned_addr learned my addr 172.21.15.43:0/3397256233 (peer_addr_for_me v2:172.21.15.43:0/0) 2023-08-09T13:52:17.484 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:52:17.480+0000 7f68ddd9b640 1 --2- 172.21.15.43:0/3397256233 >> v2:172.21.15.203:3300/0 conn(0x7f68e00d0f10 0x7f68e00cccd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:52:17.484 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:52:17.480+0000 7f68ded9d640 1 --2- 172.21.15.43:0/3397256233 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f68e00cd210 0x7f68e00b0f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:52:17.484 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:52:17.481+0000 7f68de59c640 1 -- 172.21.15.43:0/3397256233 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f68e00cd210 msgr2=0x7f68e00b0f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-09T13:52:17.485 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:52:17.481+0000 7f68de59c640 1 --2- 172.21.15.43:0/3397256233 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f68e00cd210 0x7f68e00b0f30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:52:17.485 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:52:17.481+0000 7f68de59c640 1 -- 172.21.15.43:0/3397256233 >> v2:172.21.15.203:3300/0 conn(0x7f68e00d0f10 msgr2=0x7f68e00cccd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-09T13:52:17.485 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:52:17.481+0000 7f68de59c640 1 --2- 172.21.15.43:0/3397256233 >> v2:172.21.15.203:3300/0 conn(0x7f68e00d0f10 0x7f68e00cccd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:52:17.485 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:52:17.481+0000 7f68de59c640 1 -- 172.21.15.43:0/3397256233 --> v2:172.21.15.43:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f68e00cb1d0 con 0x7f68e015a560 2023-08-09T13:52:17.486 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:52:17.482+0000 7f68de59c640 1 --2- 172.21.15.43:0/3397256233 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f68e015a560 0x7f68e00d09d0 secure :-1 s=READY pgs=1392 cs=0 l=1 rev1=1 crypto rx=0x7f68e015def0 tx=0x7f68d002b8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9b01e593c7098353 server_cookie=0 in_seq=0 out_seq=0 2023-08-09T13:52:17.486 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:52:17.482+0000 7f68dd59a640 1 -- 172.21.15.43:0/3397256233 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f68d0004230 con 0x7f68e015a560 2023-08-09T13:52:17.486 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:52:17.482+0000 7f68dd59a640 1 -- 172.21.15.43:0/3397256233 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f68d0004390 con 0x7f68e015a560 2023-08-09T13:52:17.486 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:52:17.482+0000 7f68dd59a640 1 -- 172.21.15.43:0/3397256233 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f68d0003810 con 0x7f68e015a560 2023-08-09T13:52:17.486 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:52:17.483+0000 7f68e4999640 1 -- 172.21.15.43:0/3397256233 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f68e015a560 msgr2=0x7f68e00d09d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:52:17.487 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:52:17.483+0000 7f68e4999640 1 --2- 172.21.15.43:0/3397256233 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f68e015a560 0x7f68e00d09d0 secure :-1 s=READY pgs=1392 cs=0 l=1 rev1=1 crypto rx=0x7f68e015def0 tx=0x7f68d002b8f0 comp rx=0 tx=0).stop 2023-08-09T13:52:17.487 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:52:17.483+0000 7f68e4999640 1 -- 172.21.15.43:0/3397256233 shutdown_connections 2023-08-09T13:52:17.487 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:52:17.483+0000 7f68e4999640 1 --2- 172.21.15.43:0/3397256233 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f68e00cd210 0x7f68e00b0f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:52:17.487 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:52:17.483+0000 7f68e4999640 1 --2- 172.21.15.43:0/3397256233 >> v2:172.21.15.203:3300/0 conn(0x7f68e00d0f10 0x7f68e00cccd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:52:17.487 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:52:17.483+0000 7f68e4999640 1 --2- 172.21.15.43:0/3397256233 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f68e015a560 0x7f68e00d09d0 unknown :-1 s=CLOSED pgs=1392 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:52:17.487 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:52:17.483+0000 7f68e4999640 1 -- 172.21.15.43:0/3397256233 >> 172.21.15.43:0/3397256233 conn(0x7f68e0099600 msgr2=0x7f68e00d02c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-09T13:52:17.487 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:52:17.483+0000 7f68e4999640 1 -- 172.21.15.43:0/3397256233 shutdown_connections 2023-08-09T13:52:17.488 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:52:17.483+0000 7f68e4999640 1 -- 172.21.15.43:0/3397256233 wait complete. 2023-08-09T13:52:17.488 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:52:17.485+0000 7f68e4999640 1 Processor -- start 2023-08-09T13:52:17.488 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:52:17.485+0000 7f68e4999640 1 -- start start 2023-08-09T13:52:17.489 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:52:17.486+0000 7f68e4999640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f68e00cd210 0x7f68e014d950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:52:17.490 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:52:17.486+0000 7f68e4999640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f68e00d0f10 0x7f68e01f31a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:52:17.490 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:52:17.486+0000 7f68e4999640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f68e015a560 0x7f68e006c480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:52:17.490 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:52:17.486+0000 7f68e4999640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f68e006ab50 con 0x7f68e00cd210 2023-08-09T13:52:17.490 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:52:17.486+0000 7f68e4999640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f68e01f27c0 con 0x7f68e015a560 2023-08-09T13:52:17.490 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:52:17.486+0000 7f68e4999640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f68e01f2aa0 con 0x7f68e00d0f10 2023-08-09T13:52:17.491 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:52:17.487+0000 7f68de59c640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f68e00cd210 0x7f68e014d950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:52:17.491 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:52:17.487+0000 7f68de59c640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f68e00cd210 0x7f68e014d950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:57834/0 (socket says 172.21.15.43:57834) 2023-08-09T13:52:17.491 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:52:17.487+0000 7f68de59c640 1 -- 172.21.15.43:0/2107578990 learned_addr learned my addr 172.21.15.43:0/2107578990 (peer_addr_for_me v2:172.21.15.43:0/0) 2023-08-09T13:52:17.491 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:52:17.487+0000 7f68ddd9b640 1 --2- 172.21.15.43:0/2107578990 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f68e00d0f10 0x7f68e01f31a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:52:17.491 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:52:17.487+0000 7f68ded9d640 1 --2- 172.21.15.43:0/2107578990 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f68e015a560 0x7f68e006c480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:52:17.491 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:52:17.487+0000 7f68de59c640 1 -- 172.21.15.43:0/2107578990 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f68e00d0f10 msgr2=0x7f68e01f31a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:52:17.491 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:52:17.487+0000 7f68de59c640 1 --2- 172.21.15.43:0/2107578990 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f68e00d0f10 0x7f68e01f31a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:52:17.492 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:52:17.487+0000 7f68de59c640 1 -- 172.21.15.43:0/2107578990 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f68e015a560 msgr2=0x7f68e006c480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:52:17.492 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:52:17.487+0000 7f68de59c640 1 --2- 172.21.15.43:0/2107578990 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f68e015a560 0x7f68e006c480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:52:17.492 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:52:17.487+0000 7f68de59c640 1 -- 172.21.15.43:0/2107578990 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f68d0009460 con 0x7f68e00cd210 2023-08-09T13:52:17.493 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:52:17.487+0000 7f68de59c640 1 --2- 172.21.15.43:0/2107578990 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f68e00cd210 0x7f68e014d950 secure :-1 s=READY pgs=1393 cs=0 l=1 rev1=1 crypto rx=0x7f68d0000c00 tx=0x7f68d00046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-09T13:52:17.493 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:52:17.488+0000 7f68ceffd640 1 -- 172.21.15.43:0/2107578990 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f68d0003e50 con 0x7f68e00cd210 2023-08-09T13:52:17.493 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:52:17.488+0000 7f68ceffd640 1 -- 172.21.15.43:0/2107578990 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f68d0004500 con 0x7f68e00cd210 2023-08-09T13:52:17.493 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:52:17.488+0000 7f68e4999640 1 -- 172.21.15.43:0/2107578990 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f68e01f2d80 con 0x7f68e00cd210 2023-08-09T13:52:17.493 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:52:17.488+0000 7f68ceffd640 1 -- 172.21.15.43:0/2107578990 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f68d0006280 con 0x7f68e00cd210 2023-08-09T13:52:17.493 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:52:17.488+0000 7f68e4999640 1 -- 172.21.15.43:0/2107578990 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f68e01f2190 con 0x7f68e00cd210 2023-08-09T13:52:17.493 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:52:17.489+0000 7f68e4999640 1 -- 172.21.15.43:0/2107578990 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f68e006ac90 con 0x7f68e00cd210 2023-08-09T13:52:17.494 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:52:17.490+0000 7f68ceffd640 1 -- 172.21.15.43:0/2107578990 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 46) v1 ==== 88460+0+0 (secure 0 0 0) 0x7f68d002f050 con 0x7f68e00cd210 2023-08-09T13:52:17.494 INFO:tasks.ceph.mgr.x.smithi203.stderr:2023-08-09T13:52:17.490+0000 7fafd242b040 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2023-08-09T13:52:17.495 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:52:17.491+0000 7f68ceffd640 1 --2- 172.21.15.43:0/2107578990 >> [v2:172.21.15.203:6832/1317108840,v1:172.21.15.203:6833/1317108840] conn(0x7f68c006a1c0 0x7f68c006c680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:52:17.495 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:52:17.491+0000 7f68ceffd640 1 -- 172.21.15.43:0/2107578990 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(116..116 src has 1..116) v4 ==== 6426+0+0 (secure 0 0 0) 0x7f68d00ab0d0 con 0x7f68e00cd210 2023-08-09T13:52:17.495 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:52:17.491+0000 7f68ddd9b640 1 -- 172.21.15.43:0/2107578990 >> [v2:172.21.15.203:6832/1317108840,v1:172.21.15.203:6833/1317108840] conn(0x7f68c006a1c0 msgr2=0x7f68c006c680 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.203:6832/1317108840 2023-08-09T13:52:17.495 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:52:17.491+0000 7f68ddd9b640 1 --2- 172.21.15.43:0/2107578990 >> [v2:172.21.15.203:6832/1317108840,v1:172.21.15.203:6833/1317108840] conn(0x7f68c006a1c0 0x7f68c006c680 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-08-09T13:52:17.497 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:52:17.495+0000 7f68ceffd640 1 -- 172.21.15.43:0/2107578990 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178066 (secure 0 0 0) 0x7f68d00764a0 con 0x7f68e00cd210 2023-08-09T13:52:17.534 INFO:tasks.ceph.mgr.y.smithi043.stderr:2023-08-09T13:52:17.532+0000 7f6f07e0d040 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2023-08-09T13:52:17.588 INFO:tasks.ceph.mgr.y.smithi043.stderr:2023-08-09T13:52:17.584+0000 7f6f07e0d040 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2023-08-09T13:52:17.702 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:52:17.692+0000 7f68ddd9b640 1 -- 172.21.15.43:0/2107578990 >> [v2:172.21.15.203:6832/1317108840,v1:172.21.15.203:6833/1317108840] conn(0x7f68c006a1c0 msgr2=0x7f68c006c680 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.203:6832/1317108840 2023-08-09T13:52:17.703 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:52:17.692+0000 7f68ddd9b640 1 --2- 172.21.15.43:0/2107578990 >> [v2:172.21.15.203:6832/1317108840,v1:172.21.15.203:6833/1317108840] conn(0x7f68c006a1c0 0x7f68c006c680 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2023-08-09T13:52:17.710 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:52:17.707+0000 7f68e4999640 1 -- 172.21.15.43:0/2107578990 --> [v2:172.21.15.203:6832/1317108840,v1:172.21.15.203:6833/1317108840] -- mgr_command(tid 0: {"prefix": "fs snap-schedule add", "fs": "cephfs", "path": "/bad-path", "snap_schedule": "1M", "target": ["mon-mgr", ""]}) v1 -- 0x7f68e006a8d0 con 0x7f68c006a1c0 2023-08-09T13:52:18.096 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:52:18.093+0000 7f68ddd9b640 1 -- 172.21.15.43:0/2107578990 >> [v2:172.21.15.203:6832/1317108840,v1:172.21.15.203:6833/1317108840] conn(0x7f68c006a1c0 msgr2=0x7f68c006c680 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.203:6832/1317108840 2023-08-09T13:52:18.096 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:52:18.093+0000 7f68ddd9b640 1 --2- 172.21.15.43:0/2107578990 >> [v2:172.21.15.203:6832/1317108840,v1:172.21.15.203:6833/1317108840] conn(0x7f68c006a1c0 0x7f68c006c680 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2023-08-09T13:52:18.179 INFO:tasks.ceph.mgr.x.smithi203.stderr:2023-08-09T13:52:18.176+0000 7fafd242b040 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2023-08-09T13:52:18.240 INFO:tasks.ceph.mgr.x.smithi203.stderr:2023-08-09T13:52:18.238+0000 7fafd242b040 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2023-08-09T13:52:18.291 INFO:tasks.ceph.mgr.x.smithi203.stderr:2023-08-09T13:52:18.288+0000 7fafd242b040 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2023-08-09T13:52:18.321 INFO:tasks.ceph.mgr.y.smithi043.stderr:2023-08-09T13:52:18.318+0000 7f6f07e0d040 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2023-08-09T13:52:18.386 INFO:tasks.ceph.mgr.y.smithi043.stderr:2023-08-09T13:52:18.383+0000 7f6f07e0d040 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2023-08-09T13:52:18.410 INFO:tasks.ceph.mgr.x.smithi203.stderr:2023-08-09T13:52:18.407+0000 7fafd242b040 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2023-08-09T13:52:18.437 INFO:tasks.ceph.mgr.y.smithi043.stderr:2023-08-09T13:52:18.435+0000 7f6f07e0d040 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2023-08-09T13:52:18.561 INFO:tasks.ceph.mgr.y.smithi043.stderr:2023-08-09T13:52:18.558+0000 7f6f07e0d040 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2023-08-09T13:52:18.566 INFO:tasks.ceph.mgr.x.smithi203.stderr:2023-08-09T13:52:18.564+0000 7fafd242b040 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2023-08-09T13:52:18.675 INFO:tasks.ceph.mgr.x.smithi203.stderr:2023-08-09T13:52:18.672+0000 7fafd242b040 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2023-08-09T13:52:18.724 INFO:tasks.ceph.mgr.x.smithi203.stderr:2023-08-09T13:52:18.721+0000 7fafd242b040 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2023-08-09T13:52:18.725 INFO:tasks.ceph.mgr.y.smithi043.stderr:2023-08-09T13:52:18.722+0000 7f6f07e0d040 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2023-08-09T13:52:18.840 INFO:tasks.ceph.mgr.y.smithi043.stderr:2023-08-09T13:52:18.837+0000 7f6f07e0d040 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2023-08-09T13:52:18.884 INFO:tasks.ceph.mgr.x.smithi203.stderr:2023-08-09T13:52:18.882+0000 7fafd242b040 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2023-08-09T13:52:18.891 INFO:tasks.ceph.mgr.y.smithi043.stderr:2023-08-09T13:52:18.889+0000 7f6f07e0d040 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2023-08-09T13:52:18.897 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:52:18.894+0000 7f68ddd9b640 1 -- 172.21.15.43:0/2107578990 >> [v2:172.21.15.203:6832/1317108840,v1:172.21.15.203:6833/1317108840] conn(0x7f68c006a1c0 msgr2=0x7f68c006c680 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.203:6832/1317108840 2023-08-09T13:52:18.897 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:52:18.894+0000 7f68ddd9b640 1 --2- 172.21.15.43:0/2107578990 >> [v2:172.21.15.203:6832/1317108840,v1:172.21.15.203:6833/1317108840] conn(0x7f68c006a1c0 0x7f68c006c680 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2023-08-09T13:52:19.060 INFO:tasks.ceph.mgr.y.smithi043.stderr:2023-08-09T13:52:19.058+0000 7f6f07e0d040 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2023-08-09T13:52:19.271 INFO:tasks.ceph.mgr.x.smithi203.stderr:2023-08-09T13:52:19.268+0000 7fafd242b040 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2023-08-09T13:52:19.462 INFO:tasks.ceph.mgr.y.smithi043.stderr:2023-08-09T13:52:19.460+0000 7f6f07e0d040 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2023-08-09T13:52:19.493 INFO:tasks.ceph.mgr.x.smithi203.stderr:2023-08-09T13:52:19.491+0000 7fafd242b040 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2023-08-09T13:52:19.551 INFO:tasks.ceph.mgr.x.smithi203.stderr:2023-08-09T13:52:19.549+0000 7fafd242b040 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2023-08-09T13:52:19.601 INFO:tasks.ceph.mgr.x.smithi203.stderr:2023-08-09T13:52:19.599+0000 7fafd242b040 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2023-08-09T13:52:19.654 INFO:tasks.ceph.mgr.x.smithi203.stderr:2023-08-09T13:52:19.652+0000 7fafd242b040 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2023-08-09T13:52:19.693 INFO:tasks.ceph.mgr.y.smithi043.stderr:2023-08-09T13:52:19.691+0000 7f6f07e0d040 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2023-08-09T13:52:19.753 INFO:tasks.ceph.mgr.y.smithi043.stderr:2023-08-09T13:52:19.751+0000 7f6f07e0d040 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2023-08-09T13:52:19.806 INFO:tasks.ceph.mgr.y.smithi043.stderr:2023-08-09T13:52:19.803+0000 7f6f07e0d040 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2023-08-09T13:52:19.860 INFO:tasks.ceph.mgr.y.smithi043.stderr:2023-08-09T13:52:19.858+0000 7f6f07e0d040 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2023-08-09T13:52:19.949 INFO:tasks.ceph.mgr.x.smithi203.stderr:2023-08-09T13:52:19.947+0000 7fafd242b040 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2023-08-09T13:52:20.007 INFO:tasks.ceph.mgr.x.smithi203.stderr:2023-08-09T13:52:20.005+0000 7fafd242b040 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2023-08-09T13:52:20.059 INFO:tasks.ceph.mgr.x.smithi203.stderr:2023-08-09T13:52:20.057+0000 7fafd242b040 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2023-08-09T13:52:20.170 INFO:tasks.ceph.mgr.y.smithi043.stderr:2023-08-09T13:52:20.167+0000 7f6f07e0d040 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2023-08-09T13:52:20.219 INFO:tasks.ceph.mgr.x.smithi203.stderr:2023-08-09T13:52:20.216+0000 7fafd242b040 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2023-08-09T13:52:20.231 INFO:tasks.ceph.mgr.y.smithi043.stderr:2023-08-09T13:52:20.228+0000 7f6f07e0d040 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2023-08-09T13:52:20.269 INFO:tasks.ceph.mgr.x.smithi203.stderr:2023-08-09T13:52:20.266+0000 7fafd242b040 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2023-08-09T13:52:20.285 INFO:tasks.ceph.mgr.y.smithi043.stderr:2023-08-09T13:52:20.283+0000 7f6f07e0d040 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2023-08-09T13:52:20.322 INFO:tasks.ceph.mgr.x.smithi203.stderr:2023-08-09T13:52:20.319+0000 7fafd242b040 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2023-08-09T13:52:20.372 INFO:tasks.ceph.mgr.x.smithi203.stderr:2023-08-09T13:52:20.370+0000 7fafd242b040 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2023-08-09T13:52:20.396 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:52:20.392+0000 7f68ceffd640 1 -- 172.21.15.43:0/2107578990 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mgrmap(e 47) v1 ==== 88069+0+0 (secure 0 0 0) 0x7f68d0076870 con 0x7f68e00cd210 2023-08-09T13:52:20.396 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:52:20.393+0000 7f68ceffd640 1 -- 172.21.15.43:0/2107578990 >> [v2:172.21.15.203:6832/1317108840,v1:172.21.15.203:6833/1317108840] conn(0x7f68c006a1c0 msgr2=0x7f68c006c680 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-08-09T13:52:20.396 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:52:20.393+0000 7f68ceffd640 1 --2- 172.21.15.43:0/2107578990 >> [v2:172.21.15.203:6832/1317108840,v1:172.21.15.203:6833/1317108840] conn(0x7f68c006a1c0 0x7f68c006c680 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:52:20.469 INFO:tasks.ceph.mgr.y.smithi043.stderr:2023-08-09T13:52:20.466+0000 7f6f07e0d040 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2023-08-09T13:52:20.526 INFO:tasks.ceph.mgr.y.smithi043.stderr:2023-08-09T13:52:20.524+0000 7f6f07e0d040 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2023-08-09T13:52:20.582 INFO:tasks.ceph.mgr.x.smithi203.stderr:2023-08-09T13:52:20.580+0000 7faf518d2640 -1 client.0 error registering admin socket command: (17) File exists 2023-08-09T13:52:20.582 INFO:tasks.ceph.mgr.x.smithi203.stderr:2023-08-09T13:52:20.580+0000 7faf518d2640 -1 client.0 error registering admin socket command: (17) File exists 2023-08-09T13:52:20.583 INFO:tasks.ceph.mgr.x.smithi203.stderr:2023-08-09T13:52:20.580+0000 7faf518d2640 -1 client.0 error registering admin socket command: (17) File exists 2023-08-09T13:52:20.583 INFO:tasks.ceph.mgr.x.smithi203.stderr:2023-08-09T13:52:20.580+0000 7faf518d2640 -1 client.0 error registering admin socket command: (17) File exists 2023-08-09T13:52:20.583 INFO:tasks.ceph.mgr.x.smithi203.stderr:2023-08-09T13:52:20.580+0000 7faf518d2640 -1 client.0 error registering admin socket command: (17) File exists 2023-08-09T13:52:20.583 INFO:tasks.ceph.mgr.x.smithi203.stderr:2023-08-09T13:52:20.580+0000 7faf4e8cc640 -1 client.0 error registering admin socket command: (17) File exists 2023-08-09T13:52:20.583 INFO:tasks.ceph.mgr.x.smithi203.stderr:2023-08-09T13:52:20.580+0000 7faf4e8cc640 -1 client.0 error registering admin socket command: (17) File exists 2023-08-09T13:52:20.584 INFO:tasks.ceph.mgr.x.smithi203.stderr:2023-08-09T13:52:20.580+0000 7faf4e8cc640 -1 client.0 error registering admin socket command: (17) File exists 2023-08-09T13:52:20.584 INFO:tasks.ceph.mgr.x.smithi203.stderr:2023-08-09T13:52:20.580+0000 7faf4e8cc640 -1 client.0 error registering admin socket command: (17) File exists 2023-08-09T13:52:20.584 INFO:tasks.ceph.mgr.x.smithi203.stderr:2023-08-09T13:52:20.580+0000 7faf4e8cc640 -1 client.0 error registering admin socket command: (17) File exists 2023-08-09T13:52:20.587 INFO:tasks.ceph.mgr.y.smithi043.stderr:2023-08-09T13:52:20.585+0000 7f6f07e0d040 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2023-08-09T13:52:20.641 INFO:tasks.ceph.mgr.y.smithi043.stderr:2023-08-09T13:52:20.638+0000 7f6f07e0d040 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2023-08-09T13:52:21.405 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:52:21.402+0000 7f68ceffd640 1 -- 172.21.15.43:0/2107578990 <== mon.0 v2:172.21.15.43:3300/0 8 ==== mgrmap(e 48) v1 ==== 88196+0+0 (secure 0 0 0) 0x7f68d00433c0 con 0x7f68e00cd210 2023-08-09T13:52:21.406 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:52:21.403+0000 7f68ceffd640 1 --2- 172.21.15.43:0/2107578990 >> [v2:172.21.15.203:6832/2041169533,v1:172.21.15.203:6833/2041169533] conn(0x7f68c0034fd0 0x7f68c0074c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:52:21.406 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:52:21.403+0000 7f68ceffd640 1 -- 172.21.15.43:0/2107578990 --> [v2:172.21.15.203:6832/2041169533,v1:172.21.15.203:6833/2041169533] -- mgr_command(tid 0: {"prefix": "fs snap-schedule add", "fs": "cephfs", "path": "/bad-path", "snap_schedule": "1M", "target": ["mon-mgr", ""]}) v1 -- 0x7f68c0036270 con 0x7f68c0034fd0 2023-08-09T13:52:21.408 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:52:21.406+0000 7f68ddd9b640 1 --2- 172.21.15.43:0/2107578990 >> [v2:172.21.15.203:6832/2041169533,v1:172.21.15.203:6833/2041169533] conn(0x7f68c0034fd0 0x7f68c0074c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:52:21.413 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:52:21.411+0000 7f68ddd9b640 1 --2- 172.21.15.43:0/2107578990 >> [v2:172.21.15.203:6832/2041169533,v1:172.21.15.203:6833/2041169533] conn(0x7f68c0034fd0 0x7f68c0074c80 secure :-1 s=READY pgs=4 cs=0 l=1 rev1=1 crypto rx=0x7f68e00b2b60 tx=0x7f68d4008040 comp rx=0 tx=0).ready entity=mgr.6919 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-09T13:52:21.581 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:52:21.578+0000 7f68ceffd640 1 -- 172.21.15.43:0/2107578990 <== mgr.6919 v2:172.21.15.203:6832/2041169533 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+32 (secure 0 0 0) 0x7f68d400c050 con 0x7f68c0034fd0 2023-08-09T13:52:21.581 INFO:teuthology.orchestra.run.smithi043.stdout:Schedule set for path /bad-path 2023-08-09T13:52:21.584 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:52:21.581+0000 7f68e4999640 1 -- 172.21.15.43:0/2107578990 >> [v2:172.21.15.203:6832/2041169533,v1:172.21.15.203:6833/2041169533] conn(0x7f68c0034fd0 msgr2=0x7f68c0074c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:52:21.585 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:52:21.581+0000 7f68e4999640 1 --2- 172.21.15.43:0/2107578990 >> [v2:172.21.15.203:6832/2041169533,v1:172.21.15.203:6833/2041169533] conn(0x7f68c0034fd0 0x7f68c0074c80 secure :-1 s=READY pgs=4 cs=0 l=1 rev1=1 crypto rx=0x7f68e00b2b60 tx=0x7f68d4008040 comp rx=0 tx=0).stop 2023-08-09T13:52:21.585 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:52:21.581+0000 7f68e4999640 1 -- 172.21.15.43:0/2107578990 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f68e00cd210 msgr2=0x7f68e014d950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:52:21.585 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:52:21.581+0000 7f68e4999640 1 --2- 172.21.15.43:0/2107578990 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f68e00cd210 0x7f68e014d950 secure :-1 s=READY pgs=1393 cs=0 l=1 rev1=1 crypto rx=0x7f68d0000c00 tx=0x7f68d00046c0 comp rx=0 tx=0).stop 2023-08-09T13:52:21.585 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:52:21.582+0000 7f68ded9d640 1 -- 172.21.15.43:0/2107578990 reap_dead start 2023-08-09T13:52:21.585 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:52:21.582+0000 7f68e4999640 1 -- 172.21.15.43:0/2107578990 shutdown_connections 2023-08-09T13:52:21.585 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:52:21.582+0000 7f68e4999640 1 -- 172.21.15.43:0/2107578990 >> 172.21.15.43:0/2107578990 conn(0x7f68e0099600 msgr2=0x7f68e00b5590 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-09T13:52:21.585 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:52:21.582+0000 7f68e4999640 1 -- 172.21.15.43:0/2107578990 shutdown_connections 2023-08-09T13:52:21.586 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:52:21.582+0000 7f68e4999640 1 -- 172.21.15.43:0/2107578990 wait complete. 2023-08-09T13:52:21.604 DEBUG:tasks.cephfs.test_snap_schedules:res=Schedule set for path /bad-path 2023-08-09T13:52:21.605 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs snap-schedule status --fs cephfs --format json /bad-path 2023-08-09T13:52:21.834 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:52:21.831+0000 7fa7e208a640 1 Processor -- start 2023-08-09T13:52:21.834 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:52:21.831+0000 7fa7e208a640 1 -- start start 2023-08-09T13:52:21.837 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:52:21.833+0000 7fa7e208a640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7fa7dc15a330 0x7fa7dc0c9780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:52:21.837 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:52:21.833+0000 7fa7e208a640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fa7dc0aff80 0x7fa7dc070d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:52:21.837 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:52:21.833+0000 7fa7e208a640 1 -- --> v1:172.21.15.203:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa7dc15dd30 con 0x7fa7dc15ea60 2023-08-09T13:52:21.837 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:52:21.833+0000 7fa7e208a640 1 -- --> v2:172.21.15.43:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa7dc0c9cc0 con 0x7fa7dc15a330 2023-08-09T13:52:21.838 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:52:21.833+0000 7fa7e208a640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa7dc071240 con 0x7fa7dc0aff80 2023-08-09T13:52:21.838 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:52:21.833+0000 7fa7e1088640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7fa7dc15a330 0x7fa7dc0c9780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:52:21.838 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:52:21.833+0000 7fa7e1088640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7fa7dc15a330 0x7fa7dc0c9780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:57836/0 (socket says 172.21.15.43:57836) 2023-08-09T13:52:21.838 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:52:21.833+0000 7fa7e1088640 1 -- 172.21.15.43:0/1713475899 learned_addr learned my addr 172.21.15.43:0/1713475899 (peer_addr_for_me v2:172.21.15.43:0/0) 2023-08-09T13:52:21.838 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:52:21.833+0000 7fa7e1889640 1 --2- 172.21.15.43:0/1713475899 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fa7dc0aff80 0x7fa7dc070d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:52:21.839 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:52:21.834+0000 7fa7e1088640 1 -- 172.21.15.43:0/1713475899 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fa7dc0aff80 msgr2=0x7fa7dc070d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-09T13:52:21.839 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:52:21.834+0000 7fa7e1088640 1 --2- 172.21.15.43:0/1713475899 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fa7dc0aff80 0x7fa7dc070d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:52:21.839 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:52:21.834+0000 7fa7e1088640 1 -- 172.21.15.43:0/1713475899 >> v1:172.21.15.203:6789/0 conn(0x7fa7dc15ea60 legacy=0x7fa7dc0af870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:52:21.839 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:52:21.834+0000 7fa7e1088640 1 -- 172.21.15.43:0/1713475899 --> v2:172.21.15.43:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa7dc071490 con 0x7fa7dc15a330 2023-08-09T13:52:21.843 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:52:21.835+0000 7fa7e1088640 1 --2- 172.21.15.43:0/1713475899 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa7dc15a330 0x7fa7dc0c9780 secure :-1 s=READY pgs=1394 cs=0 l=1 rev1=1 crypto rx=0x7fa7d8009870 tx=0x7fa7d802b980 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cd869cfdcb42bebb server_cookie=0 in_seq=0 out_seq=0 2023-08-09T13:52:21.843 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:52:21.835+0000 7fa7d3fff640 1 -- 172.21.15.43:0/1713475899 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa7d8030840 con 0x7fa7dc15a330 2023-08-09T13:52:21.843 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:52:21.835+0000 7fa7d3fff640 1 -- 172.21.15.43:0/1713475899 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa7d80309a0 con 0x7fa7dc15a330 2023-08-09T13:52:21.844 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:52:21.835+0000 7fa7d3fff640 1 -- 172.21.15.43:0/1713475899 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa7d8030d30 con 0x7fa7dc15a330 2023-08-09T13:52:21.844 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:52:21.836+0000 7fa7e208a640 1 -- 172.21.15.43:0/1713475899 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa7dc15a330 msgr2=0x7fa7dc0c9780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:52:21.844 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:52:21.836+0000 7fa7e208a640 1 --2- 172.21.15.43:0/1713475899 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa7dc15a330 0x7fa7dc0c9780 secure :-1 s=READY pgs=1394 cs=0 l=1 rev1=1 crypto rx=0x7fa7d8009870 tx=0x7fa7d802b980 comp rx=0 tx=0).stop 2023-08-09T13:52:21.844 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:52:21.836+0000 7fa7e208a640 1 -- 172.21.15.43:0/1713475899 shutdown_connections 2023-08-09T13:52:21.845 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:52:21.836+0000 7fa7e208a640 1 --2- 172.21.15.43:0/1713475899 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fa7dc0aff80 0x7fa7dc070d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:52:21.845 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:52:21.836+0000 7fa7e208a640 1 --2- 172.21.15.43:0/1713475899 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa7dc15a330 0x7fa7dc0c9780 unknown :-1 s=CLOSED pgs=1394 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:52:21.845 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:52:21.836+0000 7fa7e208a640 1 -- 172.21.15.43:0/1713475899 >> 172.21.15.43:0/1713475899 conn(0x7fa7dc0ca870 msgr2=0x7fa7dc0cac50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-09T13:52:21.845 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:52:21.836+0000 7fa7e208a640 1 -- 172.21.15.43:0/1713475899 shutdown_connections 2023-08-09T13:52:21.846 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:52:21.836+0000 7fa7e208a640 1 -- 172.21.15.43:0/1713475899 wait complete. 2023-08-09T13:52:21.846 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:52:21.838+0000 7fa7e208a640 1 Processor -- start 2023-08-09T13:52:21.846 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:52:21.838+0000 7fa7e208a640 1 -- start start 2023-08-09T13:52:21.846 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:52:21.839+0000 7fa7e208a640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa7dc0aff80 0x7fa7dc156e90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:52:21.847 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:52:21.839+0000 7fa7e208a640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fa7dc15a330 0x7fa7dc0ae230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:52:21.847 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:52:21.839+0000 7fa7e208a640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fa7dc15ea60 0x7fa7dc0ac920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:52:21.847 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:52:21.839+0000 7fa7e208a640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa7dc1573d0 con 0x7fa7dc0aff80 2023-08-09T13:52:21.847 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:52:21.839+0000 7fa7e208a640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa7dc0c5280 con 0x7fa7dc15a330 2023-08-09T13:52:21.847 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:52:21.839+0000 7fa7e208a640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa7dc0c5560 con 0x7fa7dc15ea60 2023-08-09T13:52:21.848 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:52:21.839+0000 7fa7e1088640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa7dc0aff80 0x7fa7dc156e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:52:21.848 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:52:21.840+0000 7fa7e1088640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa7dc0aff80 0x7fa7dc156e90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:57842/0 (socket says 172.21.15.43:57842) 2023-08-09T13:52:21.848 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:52:21.840+0000 7fa7e1088640 1 -- 172.21.15.43:0/341116400 learned_addr learned my addr 172.21.15.43:0/341116400 (peer_addr_for_me v2:172.21.15.43:0/0) 2023-08-09T13:52:21.848 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:52:21.840+0000 7fa7e1889640 1 --2- 172.21.15.43:0/341116400 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fa7dc15ea60 0x7fa7dc0ac920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:52:21.848 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:52:21.840+0000 7fa7e0887640 1 --2- 172.21.15.43:0/341116400 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fa7dc15a330 0x7fa7dc0ae230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:52:21.849 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:52:21.840+0000 7fa7e1088640 1 -- 172.21.15.43:0/341116400 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fa7dc15ea60 msgr2=0x7fa7dc0ac920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:52:21.849 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:52:21.840+0000 7fa7e1088640 1 --2- 172.21.15.43:0/341116400 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fa7dc15ea60 0x7fa7dc0ac920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:52:21.849 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:52:21.840+0000 7fa7e1088640 1 -- 172.21.15.43:0/341116400 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fa7dc15a330 msgr2=0x7fa7dc0ae230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:52:21.849 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:52:21.840+0000 7fa7e1088640 1 --2- 172.21.15.43:0/341116400 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fa7dc15a330 0x7fa7dc0ae230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:52:21.849 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:52:21.840+0000 7fa7e1088640 1 -- 172.21.15.43:0/341116400 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa7d80094d0 con 0x7fa7dc0aff80 2023-08-09T13:52:21.849 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:52:21.841+0000 7fa7e1088640 1 --2- 172.21.15.43:0/341116400 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa7dc0aff80 0x7fa7dc156e90 secure :-1 s=READY pgs=1395 cs=0 l=1 rev1=1 crypto rx=0x7fa7d8000c00 tx=0x7fa7d8004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-09T13:52:21.851 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:52:21.841+0000 7fa7d1ffb640 1 -- 172.21.15.43:0/341116400 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa7d8039c90 con 0x7fa7dc0aff80 2023-08-09T13:52:21.851 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:52:21.841+0000 7fa7d1ffb640 1 -- 172.21.15.43:0/341116400 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa7d8039df0 con 0x7fa7dc0aff80 2023-08-09T13:52:21.851 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:52:21.841+0000 7fa7d1ffb640 1 -- 172.21.15.43:0/341116400 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa7d8043640 con 0x7fa7dc0aff80 2023-08-09T13:52:21.851 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:52:21.841+0000 7fa7e208a640 1 -- 172.21.15.43:0/341116400 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa7dc0c5840 con 0x7fa7dc0aff80 2023-08-09T13:52:21.852 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:52:21.841+0000 7fa7e208a640 1 -- 172.21.15.43:0/341116400 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa7dc0c5e00 con 0x7fa7dc0aff80 2023-08-09T13:52:21.852 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:52:21.842+0000 7fa7e208a640 1 -- 172.21.15.43:0/341116400 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa7dc0ca690 con 0x7fa7dc0aff80 2023-08-09T13:52:21.852 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:52:21.843+0000 7fa7d1ffb640 1 -- 172.21.15.43:0/341116400 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 48) v1 ==== 88196+0+0 (secure 0 0 0) 0x7fa7d80437a0 con 0x7fa7dc0aff80 2023-08-09T13:52:21.852 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:52:21.844+0000 7fa7d1ffb640 1 --2- 172.21.15.43:0/341116400 >> [v2:172.21.15.203:6832/2041169533,v1:172.21.15.203:6833/2041169533] conn(0x7fa7c0069ef0 0x7fa7c006c3b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:52:21.852 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:52:21.844+0000 7fa7d1ffb640 1 -- 172.21.15.43:0/341116400 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(117..117 src has 1..117) v4 ==== 6770+0+0 (secure 0 0 0) 0x7fa7d803e070 con 0x7fa7dc0aff80 2023-08-09T13:52:21.852 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:52:21.845+0000 7fa7e0887640 1 --2- 172.21.15.43:0/341116400 >> [v2:172.21.15.203:6832/2041169533,v1:172.21.15.203:6833/2041169533] conn(0x7fa7c0069ef0 0x7fa7c006c3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:52:21.852 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:52:21.846+0000 7fa7e0887640 1 --2- 172.21.15.43:0/341116400 >> [v2:172.21.15.203:6832/2041169533,v1:172.21.15.203:6833/2041169533] conn(0x7fa7c0069ef0 0x7fa7c006c3b0 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7fa7dc0ad540 tx=0x7fa7cc009040 comp rx=0 tx=0).ready entity=mgr.6919 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-09T13:52:21.853 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:52:21.849+0000 7fa7d1ffb640 1 -- 172.21.15.43:0/341116400 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178066 (secure 0 0 0) 0x7fa7d8076d20 con 0x7fa7dc0aff80 2023-08-09T13:52:22.060 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:52:22.057+0000 7fa7e208a640 1 -- 172.21.15.43:0/341116400 --> [v2:172.21.15.203:6832/2041169533,v1:172.21.15.203:6833/2041169533] -- mgr_command(tid 0: {"prefix": "fs snap-schedule status", "fs": "cephfs", "path": "/bad-path", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7fa7dc15e260 con 0x7fa7c0069ef0 2023-08-09T13:52:22.076 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:52:22.073+0000 7fa7d1ffb640 1 -- 172.21.15.43:0/341116400 <== mgr.6919 v2:172.21.15.203:6832/2041169533 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+287 (secure 0 0 0) 0x7fa7cc001360 con 0x7fa7c0069ef0 2023-08-09T13:52:22.077 INFO:teuthology.orchestra.run.smithi043.stdout: 2023-08-09T13:52:22.077 INFO:teuthology.orchestra.run.smithi043.stdout:[{"fs": "cephfs", "subvol": null, "path": "/bad-path", "rel_path": "/bad-path", "schedule": "1M", "retention": {}, "start": "2023-08-09T00:00:00", "created": "2023-08-09T13:52:21", "first": null, "last": null, "last_pruned": null, "created_count": 0, "pruned_count": 0, "active": true}] 2023-08-09T13:52:22.079 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:52:22.075+0000 7fa7e208a640 1 -- 172.21.15.43:0/341116400 >> [v2:172.21.15.203:6832/2041169533,v1:172.21.15.203:6833/2041169533] conn(0x7fa7c0069ef0 msgr2=0x7fa7c006c3b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:52:22.080 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:52:22.075+0000 7fa7e208a640 1 --2- 172.21.15.43:0/341116400 >> [v2:172.21.15.203:6832/2041169533,v1:172.21.15.203:6833/2041169533] conn(0x7fa7c0069ef0 0x7fa7c006c3b0 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7fa7dc0ad540 tx=0x7fa7cc009040 comp rx=0 tx=0).stop 2023-08-09T13:52:22.080 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:52:22.075+0000 7fa7e208a640 1 -- 172.21.15.43:0/341116400 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa7dc0aff80 msgr2=0x7fa7dc156e90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:52:22.080 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:52:22.075+0000 7fa7e208a640 1 --2- 172.21.15.43:0/341116400 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa7dc0aff80 0x7fa7dc156e90 secure :-1 s=READY pgs=1395 cs=0 l=1 rev1=1 crypto rx=0x7fa7d8000c00 tx=0x7fa7d8004730 comp rx=0 tx=0).stop 2023-08-09T13:52:22.081 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:52:22.075+0000 7fa7e208a640 1 -- 172.21.15.43:0/341116400 shutdown_connections 2023-08-09T13:52:22.081 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:52:22.075+0000 7fa7e208a640 1 --2- 172.21.15.43:0/341116400 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fa7dc15ea60 0x7fa7dc0ac920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:52:22.081 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:52:22.075+0000 7fa7e208a640 1 --2- 172.21.15.43:0/341116400 >> [v2:172.21.15.203:6832/2041169533,v1:172.21.15.203:6833/2041169533] conn(0x7fa7c0069ef0 0x7fa7c006c3b0 unknown :-1 s=CLOSED pgs=23 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:52:22.081 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:52:22.075+0000 7fa7e208a640 1 --2- 172.21.15.43:0/341116400 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fa7dc15a330 0x7fa7dc0ae230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:52:22.081 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:52:22.075+0000 7fa7e208a640 1 --2- 172.21.15.43:0/341116400 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa7dc0aff80 0x7fa7dc156e90 unknown :-1 s=CLOSED pgs=1395 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:52:22.082 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:52:22.075+0000 7fa7e208a640 1 -- 172.21.15.43:0/341116400 >> 172.21.15.43:0/341116400 conn(0x7fa7dc0ca870 msgr2=0x7fa7dc0cac50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-09T13:52:22.082 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:52:22.075+0000 7fa7e208a640 1 -- 172.21.15.43:0/341116400 shutdown_connections 2023-08-09T13:52:22.082 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:52:22.076+0000 7fa7e208a640 1 -- 172.21.15.43:0/341116400 wait complete. 2023-08-09T13:52:22.094 DEBUG:tasks.cephfs.test_snap_schedules:res= [{"fs": "cephfs", "subvol": null, "path": "/bad-path", "rel_path": "/bad-path", "schedule": "1M", "retention": {}, "start": "2023-08-09T00:00:00", "created": "2023-08-09T13:52:21", "first": null, "last": null, "last_pruned": null, "created_count": 0, "pruned_count": 0, "active": true}] 2023-08-09T13:52:34.306 DEBUG:teuthology.orchestra.run.smithi043:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-08-09T13:52:34.316 DEBUG:teuthology.orchestra.run.smithi203:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-08-09T13:53:00.604 INFO:tasks.ceph.mgr.x.smithi203.stderr:2023-08-09T13:53:00.602+0000 7faf410b1640 -1 client.0 error registering admin socket command: (17) File exists 2023-08-09T13:53:00.605 INFO:tasks.ceph.mgr.x.smithi203.stderr:2023-08-09T13:53:00.602+0000 7faf410b1640 -1 client.0 error registering admin socket command: (17) File exists 2023-08-09T13:53:00.605 INFO:tasks.ceph.mgr.x.smithi203.stderr:2023-08-09T13:53:00.602+0000 7faf410b1640 -1 client.0 error registering admin socket command: (17) File exists 2023-08-09T13:53:00.605 INFO:tasks.ceph.mgr.x.smithi203.stderr:2023-08-09T13:53:00.602+0000 7faf410b1640 -1 client.0 error registering admin socket command: (17) File exists 2023-08-09T13:53:00.605 INFO:tasks.ceph.mgr.x.smithi203.stderr:2023-08-09T13:53:00.602+0000 7faf410b1640 -1 client.0 error registering admin socket command: (17) File exists 2023-08-09T13:53:04.457 DEBUG:teuthology.orchestra.run.smithi043:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-08-09T13:53:04.557 DEBUG:teuthology.orchestra.run.smithi203:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-08-09T13:53:22.096 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs snap-schedule status --fs cephfs --format json /bad-path 2023-08-09T13:53:22.335 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:22.334+0000 7fb335c1f640 1 Processor -- start 2023-08-09T13:53:22.336 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:22.334+0000 7fb335c1f640 1 -- start start 2023-08-09T13:53:22.337 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:22.335+0000 7fb335c1f640 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7fb33015c6f0 0x7fb3300d03a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:53:22.337 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:22.336+0000 7fb335c1f640 1 -- --> v1:172.21.15.43:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb3300d1890 con 0x7fb3300b80a0 2023-08-09T13:53:22.337 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:22.336+0000 7fb335c1f640 1 -- --> v1:172.21.15.203:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb3300d0c80 con 0x7fb3300b5b80 2023-08-09T13:53:22.337 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:22.336+0000 7fb335c1f640 1 -- --> v2:172.21.15.203:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb33015cad0 con 0x7fb33015c6f0 2023-08-09T13:53:22.339 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:22.336+0000 7fb32effd640 1 --1- >> v1:172.21.15.43:6789/0 conn(0x7fb3300b80a0 0x7fb3300b5470 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.43:6789/0 says I am v1:172.21.15.43:40376/0 (socket says 172.21.15.43:40376) 2023-08-09T13:53:22.339 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:22.336+0000 7fb32effd640 1 -- 172.21.15.43:0/2680568173 learned_addr learned my addr 172.21.15.43:0/2680568173 (peer_addr_for_me v1:172.21.15.43:0/0) 2023-08-09T13:53:22.339 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:22.336+0000 7fb32e7fc640 1 -- 172.21.15.43:0/2680568173 <== mon.0 v1:172.21.15.43:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 407653074 0 0) 0x7fb3300d1890 con 0x7fb3300b80a0 2023-08-09T13:53:22.339 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:22.336+0000 7fb32e7fc640 1 -- 172.21.15.43:0/2680568173 --> v1:172.21.15.43:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fb31c003520 con 0x7fb3300b80a0 2023-08-09T13:53:22.339 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:22.337+0000 7fb32f7fe640 1 --2- 172.21.15.43:0/2680568173 >> v2:172.21.15.203:3300/0 conn(0x7fb33015c6f0 0x7fb3300d03a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:53:22.339 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:22.337+0000 7fb32e7fc640 1 -- 172.21.15.43:0/2680568173 <== mon.0 v1:172.21.15.43:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 230969573 0 0) 0x7fb31c003520 con 0x7fb3300b80a0 2023-08-09T13:53:22.339 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:22.337+0000 7fb32e7fc640 1 -- 172.21.15.43:0/2680568173 >> v2:172.21.15.203:3300/0 conn(0x7fb33015c6f0 msgr2=0x7fb3300d03a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-09T13:53:22.340 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:22.337+0000 7fb32e7fc640 1 --2- 172.21.15.43:0/2680568173 >> v2:172.21.15.203:3300/0 conn(0x7fb33015c6f0 0x7fb3300d03a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:53:22.340 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:22.337+0000 7fb32e7fc640 1 -- 172.21.15.43:0/2680568173 >> v1:172.21.15.203:6789/0 conn(0x7fb3300b5b80 legacy=0x7fb3300d1d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:53:22.340 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:22.337+0000 7fb32e7fc640 1 -- 172.21.15.43:0/2680568173 --> v1:172.21.15.43:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb3300d0f60 con 0x7fb3300b80a0 2023-08-09T13:53:22.340 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:22.337+0000 7fb32e7fc640 1 -- 172.21.15.43:0/2680568173 <== mon.0 v1:172.21.15.43:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1285150740 0 0) 0x7fb318001670 con 0x7fb3300b80a0 2023-08-09T13:53:22.340 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:22.337+0000 7fb32e7fc640 1 -- 172.21.15.43:0/2680568173 >> v1:172.21.15.43:6789/0 conn(0x7fb3300b80a0 legacy=0x7fb3300b5470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:53:22.340 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:22.337+0000 7fb32e7fc640 1 --2- 172.21.15.43:0/2680568173 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fb31c003a30 0x7fb31c003e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:53:22.340 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:22.337+0000 7fb32e7fc640 1 --2- 172.21.15.43:0/2680568173 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb31c0047c0 0x7fb31c004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:53:22.341 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:22.337+0000 7fb32e7fc640 1 --2- 172.21.15.43:0/2680568173 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fb31c005380 0x7fb31c00b980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:53:22.341 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:22.337+0000 7fb32e7fc640 1 -- 172.21.15.43:0/2680568173 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb33015cad0 con 0x7fb31c0047c0 2023-08-09T13:53:22.341 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:22.337+0000 7fb32e7fc640 1 -- 172.21.15.43:0/2680568173 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb31c00c260 con 0x7fb31c003a30 2023-08-09T13:53:22.341 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:22.337+0000 7fb32e7fc640 1 -- 172.21.15.43:0/2680568173 --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb31c00c420 con 0x7fb31c005380 2023-08-09T13:53:22.341 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:22.337+0000 7fb32f7fe640 1 --2- 172.21.15.43:0/2680568173 >> v2:172.21.15.203:3300/0 conn(0x7fb33015c6f0 0x7fb3300d03a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-08-09T13:53:22.341 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:22.337+0000 7fb32effd640 1 --2- 172.21.15.43:0/2680568173 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb31c0047c0 0x7fb31c004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:53:22.345 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:22.338+0000 7fb32ffff640 1 --2- 172.21.15.43:0/2680568173 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fb31c005380 0x7fb31c00b980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:53:22.345 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:22.338+0000 7fb32effd640 1 -- 172.21.15.43:0/2680568173 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fb31c005380 msgr2=0x7fb31c00b980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-09T13:53:22.345 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:22.338+0000 7fb32effd640 1 --2- 172.21.15.43:0/2680568173 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fb31c005380 0x7fb31c00b980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:53:22.346 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:22.338+0000 7fb32effd640 1 -- 172.21.15.43:0/2680568173 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fb31c003a30 msgr2=0x7fb31c003e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-09T13:53:22.346 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:22.338+0000 7fb32effd640 1 --2- 172.21.15.43:0/2680568173 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fb31c003a30 0x7fb31c003e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:53:22.346 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:22.338+0000 7fb32effd640 1 -- 172.21.15.43:0/2680568173 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb31c003520 con 0x7fb31c0047c0 2023-08-09T13:53:22.346 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:22.338+0000 7fb32ffff640 1 --2- 172.21.15.43:0/2680568173 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fb31c005380 0x7fb31c00b980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-08-09T13:53:22.346 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:22.338+0000 7fb32ffff640 1 -- 172.21.15.43:0/2680568173 reap_dead start 2023-08-09T13:53:22.346 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:22.339+0000 7fb32effd640 1 --2- 172.21.15.43:0/2680568173 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb31c0047c0 0x7fb31c004c40 secure :-1 s=READY pgs=1397 cs=0 l=1 rev1=1 crypto rx=0x7fb31800c0e0 tx=0x7fb31c0057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d0ca4813dd5df53 server_cookie=0 in_seq=0 out_seq=0 2023-08-09T13:53:22.346 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:22.339+0000 7fb32e7fc640 1 -- 172.21.15.43:0/2680568173 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb318005ad0 con 0x7fb31c0047c0 2023-08-09T13:53:22.346 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:22.339+0000 7fb32e7fc640 1 -- 172.21.15.43:0/2680568173 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb318005c30 con 0x7fb31c0047c0 2023-08-09T13:53:22.347 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:22.339+0000 7fb32e7fc640 1 -- 172.21.15.43:0/2680568173 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb31802ca50 con 0x7fb31c0047c0 2023-08-09T13:53:22.347 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:22.340+0000 7fb335c1f640 1 -- 172.21.15.43:0/2680568173 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb31c0047c0 msgr2=0x7fb31c004c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:53:22.347 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:22.340+0000 7fb335c1f640 1 --2- 172.21.15.43:0/2680568173 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb31c0047c0 0x7fb31c004c40 secure :-1 s=READY pgs=1397 cs=0 l=1 rev1=1 crypto rx=0x7fb31800c0e0 tx=0x7fb31c0057f0 comp rx=0 tx=0).stop 2023-08-09T13:53:22.347 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:22.340+0000 7fb335c1f640 1 -- 172.21.15.43:0/2680568173 shutdown_connections 2023-08-09T13:53:22.347 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:22.340+0000 7fb335c1f640 1 --2- 172.21.15.43:0/2680568173 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb31c0047c0 0x7fb31c004c40 unknown :-1 s=CLOSED pgs=1397 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:53:22.347 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:22.340+0000 7fb335c1f640 1 -- 172.21.15.43:0/2680568173 >> 172.21.15.43:0/2680568173 conn(0x7fb3300712e0 msgr2=0x7fb3300716f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-09T13:53:22.347 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:22.340+0000 7fb335c1f640 1 -- 172.21.15.43:0/2680568173 shutdown_connections 2023-08-09T13:53:22.347 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:22.340+0000 7fb335c1f640 1 -- 172.21.15.43:0/2680568173 wait complete. 2023-08-09T13:53:22.348 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:22.342+0000 7fb335c1f640 1 Processor -- start 2023-08-09T13:53:22.348 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:22.342+0000 7fb335c1f640 1 -- start start 2023-08-09T13:53:22.348 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:22.343+0000 7fb335c1f640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb31c0047c0 0x7fb330155860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:53:22.348 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:22.343+0000 7fb335c1f640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fb330158840 0x7fb3300c9640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:53:22.348 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:22.343+0000 7fb335c1f640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fb3300ca930 0x7fb33015ceb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:53:22.348 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:22.343+0000 7fb335c1f640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb3300c9b80 con 0x7fb31c0047c0 2023-08-09T13:53:22.348 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:22.343+0000 7fb335c1f640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb3300ca150 con 0x7fb330158840 2023-08-09T13:53:22.349 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:22.343+0000 7fb335c1f640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb3300cbf00 con 0x7fb3300ca930 2023-08-09T13:53:22.349 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:22.343+0000 7fb32f7fe640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb31c0047c0 0x7fb330155860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:53:22.349 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:22.344+0000 7fb32f7fe640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb31c0047c0 0x7fb330155860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:55246/0 (socket says 172.21.15.43:55246) 2023-08-09T13:53:22.349 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:22.344+0000 7fb32f7fe640 1 -- 172.21.15.43:0/1580933044 learned_addr learned my addr 172.21.15.43:0/1580933044 (peer_addr_for_me v2:172.21.15.43:0/0) 2023-08-09T13:53:22.350 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:22.344+0000 7fb32ffff640 1 --2- 172.21.15.43:0/1580933044 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fb3300ca930 0x7fb33015ceb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:53:22.350 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:22.344+0000 7fb32f7fe640 1 -- 172.21.15.43:0/1580933044 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fb3300ca930 msgr2=0x7fb33015ceb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:53:22.351 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:22.344+0000 7fb32f7fe640 1 --2- 172.21.15.43:0/1580933044 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fb3300ca930 0x7fb33015ceb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:53:22.351 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:22.344+0000 7fb32f7fe640 1 -- 172.21.15.43:0/1580933044 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fb330158840 msgr2=0x7fb3300c9640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:53:22.351 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:22.344+0000 7fb32f7fe640 1 --2- 172.21.15.43:0/1580933044 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fb330158840 0x7fb3300c9640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:53:22.351 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:22.344+0000 7fb32f7fe640 1 -- 172.21.15.43:0/1580933044 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb3300cc1e0 con 0x7fb31c0047c0 2023-08-09T13:53:22.351 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:22.344+0000 7fb32ffff640 1 --2- 172.21.15.43:0/1580933044 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fb3300ca930 0x7fb33015ceb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-08-09T13:53:22.351 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:22.344+0000 7fb32f7fe640 1 --2- 172.21.15.43:0/1580933044 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb31c0047c0 0x7fb330155860 secure :-1 s=READY pgs=1398 cs=0 l=1 rev1=1 crypto rx=0x7fb32000df00 tx=0x7fb32000b720 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-09T13:53:22.351 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:22.345+0000 7fb327fff640 1 -- 172.21.15.43:0/1580933044 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb32000e850 con 0x7fb31c0047c0 2023-08-09T13:53:22.352 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:22.345+0000 7fb327fff640 1 -- 172.21.15.43:0/1580933044 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb32000e9d0 con 0x7fb31c0047c0 2023-08-09T13:53:22.352 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:22.345+0000 7fb327fff640 1 -- 172.21.15.43:0/1580933044 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb32000ed80 con 0x7fb31c0047c0 2023-08-09T13:53:22.352 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:22.345+0000 7fb335c1f640 1 -- 172.21.15.43:0/1580933044 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb3300ca410 con 0x7fb31c0047c0 2023-08-09T13:53:22.352 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:22.345+0000 7fb335c1f640 1 -- 172.21.15.43:0/1580933044 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb3300cafc0 con 0x7fb31c0047c0 2023-08-09T13:53:22.352 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:22.346+0000 7fb335c1f640 1 -- 172.21.15.43:0/1580933044 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb3300ca650 con 0x7fb31c0047c0 2023-08-09T13:53:22.352 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:22.347+0000 7fb327fff640 1 -- 172.21.15.43:0/1580933044 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 51) v1 ==== 88574+0+0 (secure 0 0 0) 0x7fb32001f3b0 con 0x7fb31c0047c0 2023-08-09T13:53:22.353 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:22.348+0000 7fb327fff640 1 --2- 172.21.15.43:0/1580933044 >> [v2:172.21.15.203:6832/2041169533,v1:172.21.15.203:6833/2041169533] conn(0x7fb31006a2e0 0x7fb31006c7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:53:22.353 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:22.349+0000 7fb327fff640 1 -- 172.21.15.43:0/1580933044 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(117..117 src has 1..117) v4 ==== 6770+0+0 (secure 0 0 0) 0x7fb320089950 con 0x7fb31c0047c0 2023-08-09T13:53:22.353 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:22.349+0000 7fb32effd640 1 --2- 172.21.15.43:0/1580933044 >> [v2:172.21.15.203:6832/2041169533,v1:172.21.15.203:6833/2041169533] conn(0x7fb31006a2e0 0x7fb31006c7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:53:22.353 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:22.351+0000 7fb32effd640 1 --2- 172.21.15.43:0/1580933044 >> [v2:172.21.15.203:6832/2041169533,v1:172.21.15.203:6833/2041169533] conn(0x7fb31006a2e0 0x7fb31006c7a0 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7fb318034000 tx=0x7fb318039040 comp rx=0 tx=0).ready entity=mgr.6919 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-09T13:53:22.354 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:22.353+0000 7fb327fff640 1 -- 172.21.15.43:0/1580933044 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178066 (secure 0 0 0) 0x7fb3300ca650 con 0x7fb31c0047c0 2023-08-09T13:53:22.572 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:22.570+0000 7fb335c1f640 1 -- 172.21.15.43:0/1580933044 --> [v2:172.21.15.203:6832/2041169533,v1:172.21.15.203:6833/2041169533] -- mgr_command(tid 0: {"prefix": "fs snap-schedule status", "fs": "cephfs", "path": "/bad-path", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7fb3300d1520 con 0x7fb31006a2e0 2023-08-09T13:53:22.588 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:22.587+0000 7fb327fff640 1 -- 172.21.15.43:0/1580933044 <== mgr.6919 v2:172.21.15.203:6832/2041169533 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+288 (secure 0 0 0) 0x7fb318005da0 con 0x7fb31006a2e0 2023-08-09T13:53:22.589 INFO:teuthology.orchestra.run.smithi043.stdout: 2023-08-09T13:53:22.589 INFO:teuthology.orchestra.run.smithi043.stdout:[{"fs": "cephfs", "subvol": null, "path": "/bad-path", "rel_path": "/bad-path", "schedule": "1M", "retention": {}, "start": "2023-08-09T00:00:00", "created": "2023-08-09T13:52:21", "first": null, "last": null, "last_pruned": null, "created_count": 0, "pruned_count": 0, "active": false}] 2023-08-09T13:53:22.591 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:22.590+0000 7fb335c1f640 1 -- 172.21.15.43:0/1580933044 >> [v2:172.21.15.203:6832/2041169533,v1:172.21.15.203:6833/2041169533] conn(0x7fb31006a2e0 msgr2=0x7fb31006c7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:53:22.592 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:22.590+0000 7fb335c1f640 1 --2- 172.21.15.43:0/1580933044 >> [v2:172.21.15.203:6832/2041169533,v1:172.21.15.203:6833/2041169533] conn(0x7fb31006a2e0 0x7fb31006c7a0 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7fb318034000 tx=0x7fb318039040 comp rx=0 tx=0).stop 2023-08-09T13:53:22.592 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:22.590+0000 7fb335c1f640 1 -- 172.21.15.43:0/1580933044 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb31c0047c0 msgr2=0x7fb330155860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:53:22.592 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:22.590+0000 7fb335c1f640 1 --2- 172.21.15.43:0/1580933044 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb31c0047c0 0x7fb330155860 secure :-1 s=READY pgs=1398 cs=0 l=1 rev1=1 crypto rx=0x7fb32000df00 tx=0x7fb32000b720 comp rx=0 tx=0).stop 2023-08-09T13:53:22.592 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:22.590+0000 7fb335c1f640 1 -- 172.21.15.43:0/1580933044 shutdown_connections 2023-08-09T13:53:22.592 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:22.590+0000 7fb335c1f640 1 --2- 172.21.15.43:0/1580933044 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fb3300ca930 0x7fb33015ceb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:53:22.592 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:22.590+0000 7fb335c1f640 1 --2- 172.21.15.43:0/1580933044 >> [v2:172.21.15.203:6832/2041169533,v1:172.21.15.203:6833/2041169533] conn(0x7fb31006a2e0 0x7fb31006c7a0 unknown :-1 s=CLOSED pgs=25 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:53:22.592 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:22.590+0000 7fb335c1f640 1 --2- 172.21.15.43:0/1580933044 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fb330158840 0x7fb3300c9640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:53:22.593 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:22.590+0000 7fb335c1f640 1 --2- 172.21.15.43:0/1580933044 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb31c0047c0 0x7fb330155860 unknown :-1 s=CLOSED pgs=1398 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:53:22.593 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:22.590+0000 7fb335c1f640 1 -- 172.21.15.43:0/1580933044 >> 172.21.15.43:0/1580933044 conn(0x7fb3300712e0 msgr2=0x7fb33015dbd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-09T13:53:22.593 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:22.590+0000 7fb335c1f640 1 -- 172.21.15.43:0/1580933044 shutdown_connections 2023-08-09T13:53:22.593 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:22.590+0000 7fb335c1f640 1 -- 172.21.15.43:0/1580933044 wait complete. 2023-08-09T13:53:22.611 DEBUG:tasks.cephfs.test_snap_schedules:res= [{"fs": "cephfs", "subvol": null, "path": "/bad-path", "rel_path": "/bad-path", "schedule": "1M", "retention": {}, "start": "2023-08-09T00:00:00", "created": "2023-08-09T13:52:21", "first": null, "last": null, "last_pruned": null, "created_count": 0, "pruned_count": 0, "active": false}] 2023-08-09T13:53:22.612 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs snap-schedule remove --fs cephfs /bad-path 2023-08-09T13:53:22.839 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:22.838+0000 7fd981639640 1 Processor -- start 2023-08-09T13:53:22.840 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:22.838+0000 7fd981639640 1 -- start start 2023-08-09T13:53:22.841 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:22.840+0000 7fd981639640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fd97c0c99b0 0x7fd97c15d3c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:53:22.841 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:22.840+0000 7fd981639640 1 -- --> v1:172.21.15.43:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd97c159c40 con 0x7fd97c0c86c0 2023-08-09T13:53:22.841 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:22.840+0000 7fd981639640 1 -- --> v1:172.21.15.203:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd97c159ea0 con 0x7fd97c06c510 2023-08-09T13:53:22.842 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:22.840+0000 7fd981639640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd97c157000 con 0x7fd97c0c99b0 2023-08-09T13:53:22.843 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:22.840+0000 7fd9737fe640 1 --1- >> v1:172.21.15.43:6789/0 conn(0x7fd97c0c86c0 0x7fd97c159530 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.43:6789/0 says I am v1:172.21.15.43:40386/0 (socket says 172.21.15.43:40386) 2023-08-09T13:53:22.843 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:22.840+0000 7fd9737fe640 1 -- 172.21.15.43:0/2498840540 learned_addr learned my addr 172.21.15.43:0/2498840540 (peer_addr_for_me v1:172.21.15.43:0/0) 2023-08-09T13:53:22.843 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:22.841+0000 7fd97bfff640 1 --2- 172.21.15.43:0/2498840540 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fd97c0c99b0 0x7fd97c15d3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:53:22.843 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:22.841+0000 7fd97b7fe640 1 -- 172.21.15.43:0/2498840540 <== mon.0 v1:172.21.15.43:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3515553869 0 0) 0x7fd97c159c40 con 0x7fd97c0c86c0 2023-08-09T13:53:22.844 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:22.841+0000 7fd97b7fe640 1 -- 172.21.15.43:0/2498840540 --> v1:172.21.15.43:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fd968003520 con 0x7fd97c0c86c0 2023-08-09T13:53:22.844 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:22.841+0000 7fd97b7fe640 1 -- 172.21.15.43:0/2498840540 <== mon.0 v1:172.21.15.43:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 470592631 0 0) 0x7fd968003520 con 0x7fd97c0c86c0 2023-08-09T13:53:22.844 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:22.841+0000 7fd97b7fe640 1 -- 172.21.15.43:0/2498840540 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fd97c0c99b0 msgr2=0x7fd97c15d3c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-09T13:53:22.844 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:22.841+0000 7fd97b7fe640 1 --2- 172.21.15.43:0/2498840540 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fd97c0c99b0 0x7fd97c15d3c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:53:22.844 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:22.841+0000 7fd97b7fe640 1 -- 172.21.15.43:0/2498840540 >> v1:172.21.15.203:6789/0 conn(0x7fd97c06c510 legacy=0x7fd97c06fb90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:53:22.844 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:22.841+0000 7fd97b7fe640 1 -- 172.21.15.43:0/2498840540 --> v1:172.21.15.43:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd97c15a180 con 0x7fd97c0c86c0 2023-08-09T13:53:22.844 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:22.841+0000 7fd97b7fe640 1 -- 172.21.15.43:0/2498840540 <== mon.0 v1:172.21.15.43:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1285150740 0 0) 0x7fd964001670 con 0x7fd97c0c86c0 2023-08-09T13:53:22.845 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:22.841+0000 7fd97b7fe640 1 -- 172.21.15.43:0/2498840540 >> v1:172.21.15.43:6789/0 conn(0x7fd97c0c86c0 legacy=0x7fd97c159530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:53:22.845 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:22.841+0000 7fd97b7fe640 1 --2- 172.21.15.43:0/2498840540 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fd9680039a0 0x7fd968003dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:53:22.845 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:22.841+0000 7fd97b7fe640 1 --2- 172.21.15.43:0/2498840540 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fd9680047c0 0x7fd968004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:53:22.845 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:22.841+0000 7fd97b7fe640 1 --2- 172.21.15.43:0/2498840540 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fd97c0c99b0 0x7fd968005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:53:22.845 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:22.841+0000 7fd97b7fe640 1 -- 172.21.15.43:0/2498840540 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd96800dc40 con 0x7fd9680047c0 2023-08-09T13:53:22.846 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:22.841+0000 7fd97b7fe640 1 -- 172.21.15.43:0/2498840540 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd96800dd80 con 0x7fd9680039a0 2023-08-09T13:53:22.846 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:22.841+0000 7fd97b7fe640 1 -- 172.21.15.43:0/2498840540 --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd96800df40 con 0x7fd97c0c99b0 2023-08-09T13:53:22.846 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:22.842+0000 7fd9737fe640 1 --2- 172.21.15.43:0/2498840540 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fd9680047c0 0x7fd968004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:53:22.847 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:22.842+0000 7fd9737fe640 1 -- 172.21.15.43:0/2498840540 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fd97c0c99b0 msgr2=0x7fd968005180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-09T13:53:22.847 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:22.842+0000 7fd9737fe640 1 --2- 172.21.15.43:0/2498840540 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fd97c0c99b0 0x7fd968005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:53:22.848 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:22.842+0000 7fd9737fe640 1 -- 172.21.15.43:0/2498840540 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fd9680039a0 msgr2=0x7fd968003dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-09T13:53:22.848 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:22.842+0000 7fd9737fe640 1 --2- 172.21.15.43:0/2498840540 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fd9680039a0 0x7fd968003dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:53:22.848 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:22.842+0000 7fd9737fe640 1 -- 172.21.15.43:0/2498840540 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd968003520 con 0x7fd9680047c0 2023-08-09T13:53:22.848 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:22.843+0000 7fd9737fe640 1 --2- 172.21.15.43:0/2498840540 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fd9680047c0 0x7fd968004c40 secure :-1 s=READY pgs=1400 cs=0 l=1 rev1=1 crypto rx=0x7fd97c0c8b00 tx=0x7fd9680057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c7528cbadbe37a5a server_cookie=0 in_seq=0 out_seq=0 2023-08-09T13:53:22.848 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:22.844+0000 7fd97b7fe640 1 -- 172.21.15.43:0/2498840540 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd964005ad0 con 0x7fd9680047c0 2023-08-09T13:53:22.848 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:22.844+0000 7fd97b7fe640 1 -- 172.21.15.43:0/2498840540 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd964005c50 con 0x7fd9680047c0 2023-08-09T13:53:22.849 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:22.844+0000 7fd97b7fe640 1 -- 172.21.15.43:0/2498840540 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd96402c850 con 0x7fd9680047c0 2023-08-09T13:53:22.849 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:22.844+0000 7fd981639640 1 -- 172.21.15.43:0/2498840540 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fd9680047c0 msgr2=0x7fd968004c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:53:22.849 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:22.844+0000 7fd981639640 1 --2- 172.21.15.43:0/2498840540 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fd9680047c0 0x7fd968004c40 secure :-1 s=READY pgs=1400 cs=0 l=1 rev1=1 crypto rx=0x7fd97c0c8b00 tx=0x7fd9680057f0 comp rx=0 tx=0).stop 2023-08-09T13:53:22.849 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:22.844+0000 7fd980e38640 1 -- 172.21.15.43:0/2498840540 reap_dead start 2023-08-09T13:53:22.849 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:22.844+0000 7fd981639640 1 -- 172.21.15.43:0/2498840540 shutdown_connections 2023-08-09T13:53:22.849 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:22.844+0000 7fd981639640 1 -- 172.21.15.43:0/2498840540 >> 172.21.15.43:0/2498840540 conn(0x7fd97c0ca870 msgr2=0x7fd97c0cac50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-09T13:53:22.849 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:22.844+0000 7fd981639640 1 -- 172.21.15.43:0/2498840540 shutdown_connections 2023-08-09T13:53:22.849 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:22.845+0000 7fd981639640 1 -- 172.21.15.43:0/2498840540 wait complete. 2023-08-09T13:53:22.851 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:22.846+0000 7fd981639640 1 Processor -- start 2023-08-09T13:53:22.851 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:22.846+0000 7fd981639640 1 -- start start 2023-08-09T13:53:22.851 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:22.847+0000 7fd981639640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fd9680047c0 0x7fd97c0c7950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:53:22.851 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:22.847+0000 7fd981639640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fd97c0ad850 0x7fd97c1411a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:53:22.851 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:22.847+0000 7fd981639640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fd97c0c58b0 0x7fd97c0c5d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:53:22.852 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:22.847+0000 7fd981639640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd97c0c6270 con 0x7fd9680047c0 2023-08-09T13:53:22.852 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:22.847+0000 7fd981639640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd97c0ae030 con 0x7fd97c0c58b0 2023-08-09T13:53:22.852 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:22.847+0000 7fd981639640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd97c0ae310 con 0x7fd97c0ad850 2023-08-09T13:53:22.852 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:22.848+0000 7fd97bfff640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fd9680047c0 0x7fd97c0c7950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:53:22.852 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:22.848+0000 7fd97bfff640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fd9680047c0 0x7fd97c0c7950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:55266/0 (socket says 172.21.15.43:55266) 2023-08-09T13:53:22.852 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:22.848+0000 7fd97bfff640 1 -- 172.21.15.43:0/1733188711 learned_addr learned my addr 172.21.15.43:0/1733188711 (peer_addr_for_me v2:172.21.15.43:0/0) 2023-08-09T13:53:22.852 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:22.848+0000 7fd97bfff640 1 -- 172.21.15.43:0/1733188711 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fd97c0ad850 msgr2=0x7fd97c1411a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:53:22.853 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:22.848+0000 7fd97bfff640 1 --2- 172.21.15.43:0/1733188711 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fd97c0ad850 0x7fd97c1411a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:53:22.853 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:22.848+0000 7fd97bfff640 1 -- 172.21.15.43:0/1733188711 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fd97c0c58b0 msgr2=0x7fd97c0c5d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:53:22.853 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:22.848+0000 7fd97bfff640 1 --2- 172.21.15.43:0/1733188711 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fd97c0c58b0 0x7fd97c0c5d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:53:22.853 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:22.848+0000 7fd97bfff640 1 -- 172.21.15.43:0/1733188711 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd97c0ae5f0 con 0x7fd9680047c0 2023-08-09T13:53:22.853 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:22.848+0000 7fd97bfff640 1 --2- 172.21.15.43:0/1733188711 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fd9680047c0 0x7fd97c0c7950 secure :-1 s=READY pgs=1401 cs=0 l=1 rev1=1 crypto rx=0x7fd96c00ef40 tx=0x7fd96c00b480 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-09T13:53:22.853 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:22.849+0000 7fd9797fa640 1 -- 172.21.15.43:0/1733188711 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd96c010040 con 0x7fd9680047c0 2023-08-09T13:53:22.853 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:22.849+0000 7fd9797fa640 1 -- 172.21.15.43:0/1733188711 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd96c014400 con 0x7fd9680047c0 2023-08-09T13:53:22.854 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:22.849+0000 7fd9797fa640 1 -- 172.21.15.43:0/1733188711 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd96c009c40 con 0x7fd9680047c0 2023-08-09T13:53:22.854 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:22.849+0000 7fd981639640 1 -- 172.21.15.43:0/1733188711 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd97c0ae8d0 con 0x7fd9680047c0 2023-08-09T13:53:22.854 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:22.849+0000 7fd981639640 1 -- 172.21.15.43:0/1733188711 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd97c0adc90 con 0x7fd9680047c0 2023-08-09T13:53:22.855 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:22.850+0000 7fd981639640 1 -- 172.21.15.43:0/1733188711 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd97c0ae450 con 0x7fd9680047c0 2023-08-09T13:53:22.855 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:22.851+0000 7fd9797fa640 1 -- 172.21.15.43:0/1733188711 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 51) v1 ==== 88574+0+0 (secure 0 0 0) 0x7fd96c0147b0 con 0x7fd9680047c0 2023-08-09T13:53:22.855 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:22.852+0000 7fd9797fa640 1 --2- 172.21.15.43:0/1733188711 >> [v2:172.21.15.203:6832/2041169533,v1:172.21.15.203:6833/2041169533] conn(0x7fd95c06a2e0 0x7fd95c06c7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:53:22.855 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:22.852+0000 7fd9797fa640 1 -- 172.21.15.43:0/1733188711 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(117..117 src has 1..117) v4 ==== 6770+0+0 (secure 0 0 0) 0x7fd96c089620 con 0x7fd9680047c0 2023-08-09T13:53:22.856 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:22.853+0000 7fd9737fe640 1 --2- 172.21.15.43:0/1733188711 >> [v2:172.21.15.203:6832/2041169533,v1:172.21.15.203:6833/2041169533] conn(0x7fd95c06a2e0 0x7fd95c06c7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:53:22.856 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:22.854+0000 7fd9737fe640 1 --2- 172.21.15.43:0/1733188711 >> [v2:172.21.15.203:6832/2041169533,v1:172.21.15.203:6833/2041169533] conn(0x7fd95c06a2e0 0x7fd95c06c7a0 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7fd964034000 tx=0x7fd964007000 comp rx=0 tx=0).ready entity=mgr.6919 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-09T13:53:22.858 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:22.857+0000 7fd9797fa640 1 -- 172.21.15.43:0/1733188711 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178066 (secure 0 0 0) 0x7fd97c0ae450 con 0x7fd9680047c0 2023-08-09T13:53:23.078 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:23.077+0000 7fd981639640 1 -- 172.21.15.43:0/1733188711 --> [v2:172.21.15.203:6832/2041169533,v1:172.21.15.203:6833/2041169533] -- mgr_command(tid 0: {"prefix": "fs snap-schedule remove", "fs": "cephfs", "path": "/bad-path", "target": ["mon-mgr", ""]}) v1 -- 0x7fd97c15cbc0 con 0x7fd95c06a2e0 2023-08-09T13:53:23.252 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:23.251+0000 7fd9797fa640 1 -- 172.21.15.43:0/1733188711 <== mgr.6919 v2:172.21.15.203:6832/2041169533 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+36 (secure 0 0 0) 0x7fd964000ed0 con 0x7fd95c06a2e0 2023-08-09T13:53:23.253 INFO:teuthology.orchestra.run.smithi043.stdout:Schedule removed for path /bad-path 2023-08-09T13:53:23.256 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:23.254+0000 7fd981639640 1 -- 172.21.15.43:0/1733188711 >> [v2:172.21.15.203:6832/2041169533,v1:172.21.15.203:6833/2041169533] conn(0x7fd95c06a2e0 msgr2=0x7fd95c06c7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:53:23.256 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:23.254+0000 7fd981639640 1 --2- 172.21.15.43:0/1733188711 >> [v2:172.21.15.203:6832/2041169533,v1:172.21.15.203:6833/2041169533] conn(0x7fd95c06a2e0 0x7fd95c06c7a0 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7fd964034000 tx=0x7fd964007000 comp rx=0 tx=0).stop 2023-08-09T13:53:23.256 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:23.254+0000 7fd981639640 1 -- 172.21.15.43:0/1733188711 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fd9680047c0 msgr2=0x7fd97c0c7950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:53:23.256 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:23.254+0000 7fd981639640 1 --2- 172.21.15.43:0/1733188711 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fd9680047c0 0x7fd97c0c7950 secure :-1 s=READY pgs=1401 cs=0 l=1 rev1=1 crypto rx=0x7fd96c00ef40 tx=0x7fd96c00b480 comp rx=0 tx=0).stop 2023-08-09T13:53:23.256 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:23.254+0000 7fd981639640 1 -- 172.21.15.43:0/1733188711 shutdown_connections 2023-08-09T13:53:23.257 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:23.254+0000 7fd981639640 1 --2- 172.21.15.43:0/1733188711 >> [v2:172.21.15.203:6832/2041169533,v1:172.21.15.203:6833/2041169533] conn(0x7fd95c06a2e0 0x7fd95c06c7a0 unknown :-1 s=CLOSED pgs=26 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:53:23.257 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:23.254+0000 7fd981639640 1 --2- 172.21.15.43:0/1733188711 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fd97c0c58b0 0x7fd97c0c5d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:53:23.257 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:23.254+0000 7fd981639640 1 --2- 172.21.15.43:0/1733188711 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fd97c0ad850 0x7fd97c1411a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:53:23.257 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:23.254+0000 7fd981639640 1 --2- 172.21.15.43:0/1733188711 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fd9680047c0 0x7fd97c0c7950 unknown :-1 s=CLOSED pgs=1401 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:53:23.257 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:23.254+0000 7fd981639640 1 -- 172.21.15.43:0/1733188711 >> 172.21.15.43:0/1733188711 conn(0x7fd97c0ca870 msgr2=0x7fd97c15b5c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-09T13:53:23.257 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:23.254+0000 7fd981639640 1 -- 172.21.15.43:0/1733188711 shutdown_connections 2023-08-09T13:53:23.257 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:23.254+0000 7fd981639640 1 -- 172.21.15.43:0/1733188711 wait complete. 2023-08-09T13:53:23.276 DEBUG:tasks.cephfs.test_snap_schedules:res=Schedule removed for path /bad-path 2023-08-09T13:53:23.276 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph mgr module disable snap_schedule 2023-08-09T13:53:23.507 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:23.505+0000 7f5830338640 1 Processor -- start 2023-08-09T13:53:23.507 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:23.505+0000 7f5830338640 1 -- start start 2023-08-09T13:53:23.509 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:23.507+0000 7f5830338640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f582814f940 0x7f582806c020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:53:23.509 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:23.507+0000 7f5830338640 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7f58280ccb20 0x7f582815c5c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:53:23.509 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:23.507+0000 7f5830338640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f582815cb00 0x7f5828159040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:53:23.509 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:23.507+0000 7f5830338640 1 -- --> v2:172.21.15.43:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f582815cf70 con 0x7f582814f940 2023-08-09T13:53:23.509 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:23.507+0000 7f5830338640 1 -- --> v2:172.21.15.203:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f582806c600 con 0x7f58280ccb20 2023-08-09T13:53:23.509 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:23.507+0000 7f5830338640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f582806c8e0 con 0x7f582815cb00 2023-08-09T13:53:23.510 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:23.507+0000 7f582e0d4640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f582814f940 0x7f582806c020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:53:23.510 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:23.508+0000 7f582e0d4640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f582814f940 0x7f582806c020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:55270/0 (socket says 172.21.15.43:55270) 2023-08-09T13:53:23.510 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:23.508+0000 7f582e0d4640 1 -- 172.21.15.43:0/4180193057 learned_addr learned my addr 172.21.15.43:0/4180193057 (peer_addr_for_me v2:172.21.15.43:0/0) 2023-08-09T13:53:23.511 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:23.508+0000 7f582e8d5640 1 --2- 172.21.15.43:0/4180193057 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f582815cb00 0x7f5828159040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:53:23.511 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:23.508+0000 7f582e0d4640 1 -- 172.21.15.43:0/4180193057 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f582815cb00 msgr2=0x7f5828159040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-09T13:53:23.511 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:23.508+0000 7f582d8d3640 1 --2- 172.21.15.43:0/4180193057 >> v2:172.21.15.203:3300/0 conn(0x7f58280ccb20 0x7f582815c5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:53:23.511 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:23.508+0000 7f582e0d4640 1 --2- 172.21.15.43:0/4180193057 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f582815cb00 0x7f5828159040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:53:23.511 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:23.508+0000 7f582e0d4640 1 -- 172.21.15.43:0/4180193057 >> v2:172.21.15.203:3300/0 conn(0x7f58280ccb20 msgr2=0x7f582815c5c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-09T13:53:23.511 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:23.508+0000 7f582e0d4640 1 --2- 172.21.15.43:0/4180193057 >> v2:172.21.15.203:3300/0 conn(0x7f58280ccb20 0x7f582815c5c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:53:23.511 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:23.508+0000 7f582e0d4640 1 -- 172.21.15.43:0/4180193057 --> v2:172.21.15.43:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f582806cbc0 con 0x7f582814f940 2023-08-09T13:53:23.511 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:23.508+0000 7f582e8d5640 1 --2- 172.21.15.43:0/4180193057 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f582815cb00 0x7f5828159040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-08-09T13:53:23.512 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:23.509+0000 7f582e0d4640 1 --2- 172.21.15.43:0/4180193057 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f582814f940 0x7f582806c020 secure :-1 s=READY pgs=1402 cs=0 l=1 rev1=1 crypto rx=0x7f58280ced10 tx=0x7f582402e0c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=851ef7f9d3daaeb6 server_cookie=0 in_seq=0 out_seq=0 2023-08-09T13:53:23.513 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:23.510+0000 7f582d0d2640 1 -- 172.21.15.43:0/4180193057 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f582402ee70 con 0x7f582814f940 2023-08-09T13:53:23.513 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:23.510+0000 7f582d0d2640 1 -- 172.21.15.43:0/4180193057 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5824036440 con 0x7f582814f940 2023-08-09T13:53:23.513 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:23.510+0000 7f582d0d2640 1 -- 172.21.15.43:0/4180193057 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f58240367d0 con 0x7f582814f940 2023-08-09T13:53:23.513 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:23.510+0000 7f5830338640 1 -- 172.21.15.43:0/4180193057 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f582814f940 msgr2=0x7f582806c020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:53:23.513 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:23.510+0000 7f5830338640 1 --2- 172.21.15.43:0/4180193057 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f582814f940 0x7f582806c020 secure :-1 s=READY pgs=1402 cs=0 l=1 rev1=1 crypto rx=0x7f58280ced10 tx=0x7f582402e0c0 comp rx=0 tx=0).stop 2023-08-09T13:53:23.513 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:23.511+0000 7f5830338640 1 -- 172.21.15.43:0/4180193057 shutdown_connections 2023-08-09T13:53:23.513 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:23.511+0000 7f5830338640 1 --2- 172.21.15.43:0/4180193057 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f582815cb00 0x7f5828159040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:53:23.514 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:23.511+0000 7f5830338640 1 --2- 172.21.15.43:0/4180193057 >> v2:172.21.15.203:3300/0 conn(0x7f58280ccb20 0x7f582815c5c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:53:23.514 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:23.511+0000 7f5830338640 1 --2- 172.21.15.43:0/4180193057 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f582814f940 0x7f582806c020 unknown :-1 s=CLOSED pgs=1402 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:53:23.514 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:23.511+0000 7f5830338640 1 -- 172.21.15.43:0/4180193057 >> 172.21.15.43:0/4180193057 conn(0x7f58280ce0a0 msgr2=0x7f58280ce480 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-09T13:53:23.514 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:23.511+0000 7f5830338640 1 -- 172.21.15.43:0/4180193057 shutdown_connections 2023-08-09T13:53:23.514 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:23.511+0000 7f5830338640 1 -- 172.21.15.43:0/4180193057 wait complete. 2023-08-09T13:53:23.514 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:23.512+0000 7f5830338640 1 Processor -- start 2023-08-09T13:53:23.515 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:23.513+0000 7f5830338640 1 -- start start 2023-08-09T13:53:23.516 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:23.513+0000 7f5830338640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f58280ccb20 0x7f582813ac20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:53:23.516 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:23.513+0000 7f5830338640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f582814f940 0x7f58280c5170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:53:23.516 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:23.513+0000 7f5830338640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f582815cb00 0x7f58280b3330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:53:23.516 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:23.513+0000 7f5830338640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f58280b22d0 con 0x7f582814f940 2023-08-09T13:53:23.516 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:23.513+0000 7f5830338640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f582813b160 con 0x7f582815cb00 2023-08-09T13:53:23.516 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:23.513+0000 7f5830338640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f58280b2570 con 0x7f58280ccb20 2023-08-09T13:53:23.517 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:23.514+0000 7f582d8d3640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f582814f940 0x7f58280c5170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:53:23.517 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:23.514+0000 7f582d8d3640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f582814f940 0x7f58280c5170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:55272/0 (socket says 172.21.15.43:55272) 2023-08-09T13:53:23.517 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:23.514+0000 7f582d8d3640 1 -- 172.21.15.43:0/2969301247 learned_addr learned my addr 172.21.15.43:0/2969301247 (peer_addr_for_me v2:172.21.15.43:0/0) 2023-08-09T13:53:23.517 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:23.514+0000 7f582e0d4640 1 --2- 172.21.15.43:0/2969301247 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f58280ccb20 0x7f582813ac20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:53:23.517 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:23.514+0000 7f582e8d5640 1 --2- 172.21.15.43:0/2969301247 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f582815cb00 0x7f58280b3330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:53:23.517 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:23.514+0000 7f582d8d3640 1 -- 172.21.15.43:0/2969301247 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f58280ccb20 msgr2=0x7f582813ac20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:53:23.517 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:23.514+0000 7f582d8d3640 1 --2- 172.21.15.43:0/2969301247 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f58280ccb20 0x7f582813ac20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:53:23.518 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:23.514+0000 7f582d8d3640 1 -- 172.21.15.43:0/2969301247 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f582815cb00 msgr2=0x7f58280b3330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:53:23.518 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:23.514+0000 7f582d8d3640 1 --2- 172.21.15.43:0/2969301247 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f582815cb00 0x7f58280b3330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:53:23.518 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:23.514+0000 7f582d8d3640 1 -- 172.21.15.43:0/2969301247 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f582400b770 con 0x7f582814f940 2023-08-09T13:53:23.518 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:23.515+0000 7f582d8d3640 1 --2- 172.21.15.43:0/2969301247 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f582814f940 0x7f58280c5170 secure :-1 s=READY pgs=1403 cs=0 l=1 rev1=1 crypto rx=0x7f581800c7f0 tx=0x7f581800ccb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-09T13:53:23.519 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:23.515+0000 7f581effd640 1 -- 172.21.15.43:0/2969301247 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5818004220 con 0x7f582814f940 2023-08-09T13:53:23.519 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:23.515+0000 7f581effd640 1 -- 172.21.15.43:0/2969301247 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5818004380 con 0x7f582814f940 2023-08-09T13:53:23.519 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:23.515+0000 7f581effd640 1 -- 172.21.15.43:0/2969301247 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5818007860 con 0x7f582814f940 2023-08-09T13:53:23.519 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:23.515+0000 7f5830338640 1 -- 172.21.15.43:0/2969301247 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f58280b2850 con 0x7f582814f940 2023-08-09T13:53:23.519 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:23.515+0000 7f5830338640 1 -- 172.21.15.43:0/2969301247 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f58280b2e10 con 0x7f582814f940 2023-08-09T13:53:23.519 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:23.516+0000 7f5830338640 1 -- 172.21.15.43:0/2969301247 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f58280b3000 con 0x7f582814f940 2023-08-09T13:53:23.520 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:23.517+0000 7f581effd640 1 -- 172.21.15.43:0/2969301247 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 51) v1 ==== 88574+0+0 (secure 0 0 0) 0x7f5818007bd0 con 0x7f582814f940 2023-08-09T13:53:23.520 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:23.518+0000 7f581effd640 1 --2- 172.21.15.43:0/2969301247 >> [v2:172.21.15.203:6832/2041169533,v1:172.21.15.203:6833/2041169533] conn(0x7f580c06a2e0 0x7f580c06c7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:53:23.521 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:23.518+0000 7f581effd640 1 -- 172.21.15.43:0/2969301247 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(117..117 src has 1..117) v4 ==== 6770+0+0 (secure 0 0 0) 0x7f5818088ee0 con 0x7f582814f940 2023-08-09T13:53:23.521 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:23.519+0000 7f582e0d4640 1 --2- 172.21.15.43:0/2969301247 >> [v2:172.21.15.203:6832/2041169533,v1:172.21.15.203:6833/2041169533] conn(0x7f580c06a2e0 0x7f580c06c7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:53:23.522 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:23.520+0000 7f582e0d4640 1 --2- 172.21.15.43:0/2969301247 >> [v2:172.21.15.203:6832/2041169533,v1:172.21.15.203:6833/2041169533] conn(0x7f580c06a2e0 0x7f580c06c7a0 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7f5824038000 tx=0x7f5824008040 comp rx=0 tx=0).ready entity=mgr.6919 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-09T13:53:23.524 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:23.522+0000 7f581effd640 1 -- 172.21.15.43:0/2969301247 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178066 (secure 0 0 0) 0x7f5818053d60 con 0x7f582814f940 2023-08-09T13:53:23.768 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:23.766+0000 7f5830338640 1 -- 172.21.15.43:0/2969301247 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "mgr module disable", "module": "snap_schedule"} v 0) v1 -- 0x7f58280739f0 con 0x7f582814f940 2023-08-09T13:53:24.202 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:24.200+0000 7f581effd640 1 -- 172.21.15.43:0/2969301247 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mgrmap(e 52) v1 ==== 88557+0+0 (secure 0 0 0) 0x7f5818054130 con 0x7f582814f940 2023-08-09T13:53:24.204 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:24.202+0000 7f581effd640 1 -- 172.21.15.43:0/2969301247 <== mon.0 v2:172.21.15.43:3300/0 8 ==== mon_command_ack([{"prefix": "mgr module disable", "module": "snap_schedule"}]=0 v52) v1 ==== 93+0+0 (secure 0 0 0) 0x7f58180577f0 con 0x7f582814f940 2023-08-09T13:53:24.207 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:24.206+0000 7f5830338640 1 -- 172.21.15.43:0/2969301247 >> [v2:172.21.15.203:6832/2041169533,v1:172.21.15.203:6833/2041169533] conn(0x7f580c06a2e0 msgr2=0x7f580c06c7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:53:24.208 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:24.206+0000 7f5830338640 1 --2- 172.21.15.43:0/2969301247 >> [v2:172.21.15.203:6832/2041169533,v1:172.21.15.203:6833/2041169533] conn(0x7f580c06a2e0 0x7f580c06c7a0 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7f5824038000 tx=0x7f5824008040 comp rx=0 tx=0).stop 2023-08-09T13:53:24.208 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:24.206+0000 7f5830338640 1 -- 172.21.15.43:0/2969301247 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f582814f940 msgr2=0x7f58280c5170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:53:24.208 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:24.206+0000 7f5830338640 1 --2- 172.21.15.43:0/2969301247 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f582814f940 0x7f58280c5170 secure :-1 s=READY pgs=1403 cs=0 l=1 rev1=1 crypto rx=0x7f581800c7f0 tx=0x7f581800ccb0 comp rx=0 tx=0).stop 2023-08-09T13:53:24.208 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:24.206+0000 7f5830338640 1 -- 172.21.15.43:0/2969301247 shutdown_connections 2023-08-09T13:53:24.208 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:24.206+0000 7f5830338640 1 --2- 172.21.15.43:0/2969301247 >> [v2:172.21.15.203:6832/2041169533,v1:172.21.15.203:6833/2041169533] conn(0x7f580c06a2e0 0x7f580c06c7a0 unknown :-1 s=CLOSED pgs=27 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:53:24.208 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:24.206+0000 7f5830338640 1 --2- 172.21.15.43:0/2969301247 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f582815cb00 0x7f58280b3330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:53:24.209 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:24.206+0000 7f5830338640 1 --2- 172.21.15.43:0/2969301247 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f582814f940 0x7f58280c5170 unknown :-1 s=CLOSED pgs=1403 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:53:24.209 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:24.206+0000 7f5830338640 1 --2- 172.21.15.43:0/2969301247 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f58280ccb20 0x7f582813ac20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:53:24.209 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:24.206+0000 7f5830338640 1 -- 172.21.15.43:0/2969301247 >> 172.21.15.43:0/2969301247 conn(0x7f58280ce0a0 msgr2=0x7f58280c4400 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-09T13:53:24.209 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:24.206+0000 7f5830338640 1 -- 172.21.15.43:0/2969301247 shutdown_connections 2023-08-09T13:53:24.209 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:24.206+0000 7f5830338640 1 -- 172.21.15.43:0/2969301247 wait complete. 2023-08-09T13:53:24.228 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2023-08-09T13:53:24.229 DEBUG:teuthology.orchestra.run.smithi043:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2023-08-09T13:53:24.346 DEBUG:teuthology.orchestra.run.smithi203:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2023-08-09T13:53:24.425 INFO:tasks.ceph.mgr.x.smithi203.stderr:2023-08-09T13:53:24.423+0000 7f005263d040 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2023-08-09T13:53:24.439 INFO:tasks.ceph.mgr.y.smithi043.stderr:2023-08-09T13:53:24.437+0000 7fa15623e040 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2023-08-09T13:53:24.441 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2023-08-09T13:53:24.442 DEBUG:teuthology.orchestra.run.smithi043:> set -ex 2023-08-09T13:53:24.442 DEBUG:teuthology.orchestra.run.smithi043:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-09T13:53:24.499 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-08-09T13:53:24.708 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:24.706+0000 7fa6f3457640 1 Processor -- start 2023-08-09T13:53:24.708 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:24.707+0000 7fa6f3457640 1 -- start start 2023-08-09T13:53:24.710 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:24.708+0000 7fa6f3457640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fa6ec0ce720 0x7fa6ec0ceb00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:53:24.710 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:24.708+0000 7fa6f3457640 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7fa6ec0ce080 0x7fa6ec15e5b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:53:24.710 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:24.708+0000 7fa6f3457640 1 -- --> v1:172.21.15.43:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa6ec0cd8a0 con 0x7fa6ec15eaf0 2023-08-09T13:53:24.710 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:24.709+0000 7fa6f3457640 1 -- --> v2:172.21.15.203:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa6ec0cf350 con 0x7fa6ec0ce080 2023-08-09T13:53:24.710 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:24.709+0000 7fa6f3457640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa6ec0cf040 con 0x7fa6ec0ce720 2023-08-09T13:53:24.712 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:24.709+0000 7fa6f19f4640 1 --1- >> v1:172.21.15.43:6789/0 conn(0x7fa6ec15eaf0 0x7fa6ec0cafa0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.43:6789/0 says I am v1:172.21.15.43:40394/0 (socket says 172.21.15.43:40394) 2023-08-09T13:53:24.712 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:24.709+0000 7fa6f19f4640 1 -- 172.21.15.43:0/118156563 learned_addr learned my addr 172.21.15.43:0/118156563 (peer_addr_for_me v1:172.21.15.43:0/0) 2023-08-09T13:53:24.712 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:24.709+0000 7fa6f09f2640 1 --2- 172.21.15.43:0/118156563 >> v2:172.21.15.203:3300/0 conn(0x7fa6ec0ce080 0x7fa6ec15e5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:53:24.712 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:24.709+0000 7fa6f11f3640 1 --2- 172.21.15.43:0/118156563 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fa6ec0ce720 0x7fa6ec0ceb00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:53:24.712 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:24.709+0000 7fa6e3fff640 1 -- 172.21.15.43:0/118156563 <== mon.0 v1:172.21.15.43:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 693009153 0 0) 0x7fa6ec0cd8a0 con 0x7fa6ec15eaf0 2023-08-09T13:53:24.713 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:24.709+0000 7fa6e3fff640 1 -- 172.21.15.43:0/118156563 --> v1:172.21.15.43:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fa6d8003520 con 0x7fa6ec15eaf0 2023-08-09T13:53:24.713 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:24.710+0000 7fa6e3fff640 1 -- 172.21.15.43:0/118156563 <== mon.0 v1:172.21.15.43:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1603854436 0 0) 0x7fa6d8003520 con 0x7fa6ec15eaf0 2023-08-09T13:53:24.713 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:24.710+0000 7fa6e3fff640 1 -- 172.21.15.43:0/118156563 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fa6ec0ce720 msgr2=0x7fa6ec0ceb00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-09T13:53:24.713 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:24.710+0000 7fa6e3fff640 1 --2- 172.21.15.43:0/118156563 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fa6ec0ce720 0x7fa6ec0ceb00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:53:24.713 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:24.710+0000 7fa6e3fff640 1 -- 172.21.15.43:0/118156563 >> v2:172.21.15.203:3300/0 conn(0x7fa6ec0ce080 msgr2=0x7fa6ec15e5b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-09T13:53:24.713 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:24.710+0000 7fa6e3fff640 1 --2- 172.21.15.43:0/118156563 >> v2:172.21.15.203:3300/0 conn(0x7fa6ec0ce080 0x7fa6ec15e5b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:53:24.713 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:24.710+0000 7fa6e3fff640 1 -- 172.21.15.43:0/118156563 --> v1:172.21.15.43:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa6ec071a70 con 0x7fa6ec15eaf0 2023-08-09T13:53:24.714 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:24.710+0000 7fa6e3fff640 1 -- 172.21.15.43:0/118156563 <== mon.0 v1:172.21.15.43:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1285150740 0 0) 0x7fa6e40019f0 con 0x7fa6ec15eaf0 2023-08-09T13:53:24.714 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:24.710+0000 7fa6e3fff640 1 -- 172.21.15.43:0/118156563 >> v1:172.21.15.43:6789/0 conn(0x7fa6ec15eaf0 legacy=0x7fa6ec0cafa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:53:24.714 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:24.710+0000 7fa6e3fff640 1 --2- 172.21.15.43:0/118156563 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fa6ec0ce720 0x7fa6d80039e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:53:24.714 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:24.710+0000 7fa6e3fff640 1 --2- 172.21.15.43:0/118156563 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa6d8003f60 0x7fa6d80047c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:53:24.714 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:24.710+0000 7fa6e3fff640 1 --2- 172.21.15.43:0/118156563 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fa6d8004d00 0x7fa6d8005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:53:24.714 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:24.710+0000 7fa6e3fff640 1 -- 172.21.15.43:0/118156563 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa6ec0cf350 con 0x7fa6d8003f60 2023-08-09T13:53:24.714 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:24.710+0000 7fa6e3fff640 1 -- 172.21.15.43:0/118156563 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa6ec0cf040 con 0x7fa6d8004d00 2023-08-09T13:53:24.714 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:24.710+0000 7fa6e3fff640 1 -- 172.21.15.43:0/118156563 --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa6d800ba30 con 0x7fa6ec0ce720 2023-08-09T13:53:24.715 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:24.710+0000 7fa6f09f2640 1 --2- 172.21.15.43:0/118156563 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa6d8003f60 0x7fa6d80047c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:53:24.715 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:24.711+0000 7fa6f19f4640 1 --2- 172.21.15.43:0/118156563 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fa6d8004d00 0x7fa6d8005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:53:24.715 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:24.711+0000 7fa6f11f3640 1 --2- 172.21.15.43:0/118156563 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fa6ec0ce720 0x7fa6d80039e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:53:24.716 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:24.711+0000 7fa6f09f2640 1 -- 172.21.15.43:0/118156563 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fa6ec0ce720 msgr2=0x7fa6d80039e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-09T13:53:24.716 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:24.711+0000 7fa6f09f2640 1 --2- 172.21.15.43:0/118156563 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fa6ec0ce720 0x7fa6d80039e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:53:24.716 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:24.711+0000 7fa6f09f2640 1 -- 172.21.15.43:0/118156563 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fa6d8004d00 msgr2=0x7fa6d8005180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-09T13:53:24.717 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:24.711+0000 7fa6f09f2640 1 --2- 172.21.15.43:0/118156563 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fa6d8004d00 0x7fa6d8005180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:53:24.717 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:24.711+0000 7fa6f09f2640 1 -- 172.21.15.43:0/118156563 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa6d8003520 con 0x7fa6d8003f60 2023-08-09T13:53:24.717 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:24.712+0000 7fa6f09f2640 1 --2- 172.21.15.43:0/118156563 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa6d8003f60 0x7fa6d80047c0 secure :-1 s=READY pgs=1405 cs=0 l=1 rev1=1 crypto rx=0x7fa6ec070800 tx=0x7fa6d80057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=615a1dcf414f36f0 server_cookie=0 in_seq=0 out_seq=0 2023-08-09T13:53:24.717 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:24.712+0000 7fa6e3fff640 1 -- 172.21.15.43:0/118156563 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa6dc02c680 con 0x7fa6d8003f60 2023-08-09T13:53:24.717 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:24.712+0000 7fa6e3fff640 1 -- 172.21.15.43:0/118156563 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa6dc02c7e0 con 0x7fa6d8003f60 2023-08-09T13:53:24.717 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:24.712+0000 7fa6e3fff640 1 -- 172.21.15.43:0/118156563 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa6dc02cb90 con 0x7fa6d8003f60 2023-08-09T13:53:24.717 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:24.713+0000 7fa6f3457640 1 -- 172.21.15.43:0/118156563 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa6d8003f60 msgr2=0x7fa6d80047c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:53:24.717 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:24.713+0000 7fa6f3457640 1 --2- 172.21.15.43:0/118156563 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa6d8003f60 0x7fa6d80047c0 secure :-1 s=READY pgs=1405 cs=0 l=1 rev1=1 crypto rx=0x7fa6ec070800 tx=0x7fa6d80057f0 comp rx=0 tx=0).stop 2023-08-09T13:53:24.718 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:24.713+0000 7fa6f19f4640 1 -- 172.21.15.43:0/118156563 reap_dead start 2023-08-09T13:53:24.718 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:24.713+0000 7fa6f3457640 1 -- 172.21.15.43:0/118156563 shutdown_connections 2023-08-09T13:53:24.718 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:24.713+0000 7fa6f3457640 1 -- 172.21.15.43:0/118156563 >> 172.21.15.43:0/118156563 conn(0x7fa6ec0ccd80 msgr2=0x7fa6ec0cd190 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-09T13:53:24.718 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:24.713+0000 7fa6f3457640 1 -- 172.21.15.43:0/118156563 shutdown_connections 2023-08-09T13:53:24.718 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:24.713+0000 7fa6f3457640 1 -- 172.21.15.43:0/118156563 wait complete. 2023-08-09T13:53:24.719 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:24.715+0000 7fa6f3457640 1 Processor -- start 2023-08-09T13:53:24.720 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:24.715+0000 7fa6f3457640 1 -- start start 2023-08-09T13:53:24.720 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:24.716+0000 7fa6f3457640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fa6d8003f60 0x7fa6ec150df0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:53:24.720 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:24.716+0000 7fa6f3457640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fa6ec0b0910 0x7fa6ec0b0d70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:53:24.720 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:24.716+0000 7fa6f3457640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa6ec0b12b0 0x7fa6ec0c3f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:53:24.720 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:24.716+0000 7fa6f3457640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa6ec0b1790 con 0x7fa6ec0b12b0 2023-08-09T13:53:24.720 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:24.716+0000 7fa6f3457640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa6ec0b1a70 con 0x7fa6ec0b0910 2023-08-09T13:53:24.721 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:24.716+0000 7fa6f3457640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa6ec0b1d50 con 0x7fa6d8003f60 2023-08-09T13:53:24.721 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:24.717+0000 7fa6f19f4640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa6ec0b12b0 0x7fa6ec0c3f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:53:24.721 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:24.717+0000 7fa6f19f4640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa6ec0b12b0 0x7fa6ec0c3f70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:55306/0 (socket says 172.21.15.43:55306) 2023-08-09T13:53:24.721 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:24.717+0000 7fa6f19f4640 1 -- 172.21.15.43:0/690288894 learned_addr learned my addr 172.21.15.43:0/690288894 (peer_addr_for_me v2:172.21.15.43:0/0) 2023-08-09T13:53:24.721 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:24.717+0000 7fa6f11f3640 1 --2- 172.21.15.43:0/690288894 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fa6d8003f60 0x7fa6ec150df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:53:24.721 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:24.717+0000 7fa6f09f2640 1 --2- 172.21.15.43:0/690288894 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fa6ec0b0910 0x7fa6ec0b0d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:53:24.721 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:24.717+0000 7fa6f19f4640 1 -- 172.21.15.43:0/690288894 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fa6d8003f60 msgr2=0x7fa6ec150df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:53:24.721 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:24.717+0000 7fa6f19f4640 1 --2- 172.21.15.43:0/690288894 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fa6d8003f60 0x7fa6ec150df0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:53:24.722 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:24.717+0000 7fa6f19f4640 1 -- 172.21.15.43:0/690288894 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fa6ec0b0910 msgr2=0x7fa6ec0b0d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:53:24.722 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:24.717+0000 7fa6f19f4640 1 --2- 172.21.15.43:0/690288894 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fa6ec0b0910 0x7fa6ec0b0d70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:53:24.722 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:24.717+0000 7fa6f19f4640 1 -- 172.21.15.43:0/690288894 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa6ec0b2030 con 0x7fa6ec0b12b0 2023-08-09T13:53:24.722 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:24.717+0000 7fa6f19f4640 1 --2- 172.21.15.43:0/690288894 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa6ec0b12b0 0x7fa6ec0c3f70 secure :-1 s=READY pgs=1406 cs=0 l=1 rev1=1 crypto rx=0x7fa6e4000f80 tx=0x7fa6e400e530 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-09T13:53:24.722 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:24.718+0000 7fa6e1ffb640 1 -- 172.21.15.43:0/690288894 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa6e400ebb0 con 0x7fa6ec0b12b0 2023-08-09T13:53:24.722 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:24.718+0000 7fa6e1ffb640 1 -- 172.21.15.43:0/690288894 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa6e400ed10 con 0x7fa6ec0b12b0 2023-08-09T13:53:24.722 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:24.718+0000 7fa6e1ffb640 1 -- 172.21.15.43:0/690288894 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa6e4013640 con 0x7fa6ec0b12b0 2023-08-09T13:53:24.723 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:24.718+0000 7fa6f3457640 1 -- 172.21.15.43:0/690288894 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa6ec151330 con 0x7fa6ec0b12b0 2023-08-09T13:53:24.723 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:24.718+0000 7fa6f3457640 1 -- 172.21.15.43:0/690288894 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa6ec0c4690 con 0x7fa6ec0b12b0 2023-08-09T13:53:24.723 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:24.719+0000 7fa6f3457640 1 -- 172.21.15.43:0/690288894 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa6ec0c3d50 con 0x7fa6ec0b12b0 2023-08-09T13:53:24.724 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:24.720+0000 7fa6e1ffb640 1 -- 172.21.15.43:0/690288894 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 52) v1 ==== 88557+0+0 (secure 0 0 0) 0x7fa6e40137a0 con 0x7fa6ec0b12b0 2023-08-09T13:53:24.724 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:24.721+0000 7fa6e1ffb640 1 --2- 172.21.15.43:0/690288894 >> [v2:172.21.15.203:6832/2041169533,v1:172.21.15.203:6833/2041169533] conn(0x7fa6d006a290 0x7fa6d006c750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:53:24.724 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:24.721+0000 7fa6e1ffb640 1 -- 172.21.15.43:0/690288894 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(117..117 src has 1..117) v4 ==== 6770+0+0 (secure 0 0 0) 0x7fa6e40890a0 con 0x7fa6ec0b12b0 2023-08-09T13:53:24.724 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:24.721+0000 7fa6f11f3640 1 -- 172.21.15.43:0/690288894 >> [v2:172.21.15.203:6832/2041169533,v1:172.21.15.203:6833/2041169533] conn(0x7fa6d006a290 msgr2=0x7fa6d006c750 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.203:6832/2041169533 2023-08-09T13:53:24.724 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:24.721+0000 7fa6f11f3640 1 --2- 172.21.15.43:0/690288894 >> [v2:172.21.15.203:6832/2041169533,v1:172.21.15.203:6833/2041169533] conn(0x7fa6d006a290 0x7fa6d006c750 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-08-09T13:53:24.727 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:24.725+0000 7fa6e1ffb640 1 -- 172.21.15.43:0/690288894 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178066 (secure 0 0 0) 0x7fa6e4054220 con 0x7fa6ec0b12b0 2023-08-09T13:53:24.922 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:24.921+0000 7fa6f3457640 1 -- 172.21.15.43:0/690288894 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fa6ec0b18d0 con 0x7fa6ec0b12b0 2023-08-09T13:53:24.923 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:24.922+0000 7fa6f11f3640 1 -- 172.21.15.43:0/690288894 >> [v2:172.21.15.203:6832/2041169533,v1:172.21.15.203:6833/2041169533] conn(0x7fa6d006a290 msgr2=0x7fa6d006c750 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.203:6832/2041169533 2023-08-09T13:53:24.923 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:24.922+0000 7fa6f11f3640 1 --2- 172.21.15.43:0/690288894 >> [v2:172.21.15.203:6832/2041169533,v1:172.21.15.203:6833/2041169533] conn(0x7fa6d006a290 0x7fa6d006c750 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2023-08-09T13:53:24.924 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:24.922+0000 7fa6e1ffb640 1 -- 172.21.15.43:0/690288894 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v117) v1 ==== 74+0+19014 (secure 0 0 0) 0x7fa6ec0b18d0 con 0x7fa6ec0b12b0 2023-08-09T13:53:24.924 INFO:teuthology.orchestra.run.smithi043.stdout: 2023-08-09T13:53:24.925 INFO:teuthology.orchestra.run.smithi043.stdout:{"epoch":117,"fsid":"daac87ee-1337-4250-a955-aea4e408a39b","created":"2023-08-09T13:33:41.701198+0000","modified":"2023-08-09T13:52:20.374762+0000","last_up_change":"2023-08-09T13:33:53.783453+0000","last_in_change":"2023-08-09T13:33:45.148904+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":15,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":"rbd","create_time":"2023-08-09T13:33:54.365137+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":".mgr","create_time":"2023-08-09T13:33:54.831803+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":14,"pool_name":"cephfs_metadata","create_time":"2023-08-09T13:51:57.637653+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"115","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":2.5,"score_stable":2.5,"optimal_score":1,"raw_score_acting":2.5,"raw_score_stable":2.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":15,"pool_name":"cephfs_data","create_time":"2023-08-09T13:51:58.676117+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"116","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"cbd49ecd-e896-4fbd-864e-5c37a829b165","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":114,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6800","nonce":502667335},{"type":"v1","addr":"172.21.15.43:6801","nonce":502667335}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6802","nonce":502667335},{"type":"v1","addr":"172.21.15.43:6803","nonce":502667335}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6806","nonce":502667335},{"type":"v1","addr":"172.21.15.43:6807","nonce":502667335}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6804","nonce":502667335},{"type":"v1","addr":"172.21.15.43:6805","nonce":502667335}]},"public_addr":"172.21.15.43:6801/502667335","cluster_addr":"172.21.15.43:6803/502667335","heartbeat_back_addr":"172.21.15.43:6807/502667335","heartbeat_front_addr":"172.21.15.43:6805/502667335","state":["exists","up"]},{"osd":1,"uuid":"75c7e0fe-1ec7-4fe8-a2f2-3661a65b8500","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":114,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6808","nonce":3289134610},{"type":"v1","addr":"172.21.15.43:6809","nonce":3289134610}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6810","nonce":3289134610},{"type":"v1","addr":"172.21.15.43:6812","nonce":3289134610}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6821","nonce":3289134610},{"type":"v1","addr":"172.21.15.43:6824","nonce":3289134610}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6815","nonce":3289134610},{"type":"v1","addr":"172.21.15.43:6818","nonce":3289134610}]},"public_addr":"172.21.15.43:6809/3289134610","cluster_addr":"172.21.15.43:6812/3289134610","heartbeat_back_addr":"172.21.15.43:6824/3289134610","heartbeat_front_addr":"172.21.15.43:6818/3289134610","state":["exists","up"]},{"osd":2,"uuid":"bb5c41e9-7f5f-4125-9b01-3bb8bfc2ea83","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":114,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6811","nonce":4079709736},{"type":"v1","addr":"172.21.15.43:6814","nonce":4079709736}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6817","nonce":4079709736},{"type":"v1","addr":"172.21.15.43:6820","nonce":4079709736}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6829","nonce":4079709736},{"type":"v1","addr":"172.21.15.43:6831","nonce":4079709736}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6823","nonce":4079709736},{"type":"v1","addr":"172.21.15.43:6826","nonce":4079709736}]},"public_addr":"172.21.15.43:6814/4079709736","cluster_addr":"172.21.15.43:6820/4079709736","heartbeat_back_addr":"172.21.15.43:6831/4079709736","heartbeat_front_addr":"172.21.15.43:6826/4079709736","state":["exists","up"]},{"osd":3,"uuid":"bfe4c8b4-694a-465c-87e6-2b2aab852613","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":114,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6813","nonce":1709538759},{"type":"v1","addr":"172.21.15.43:6816","nonce":1709538759}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6819","nonce":1709538759},{"type":"v1","addr":"172.21.15.43:6822","nonce":1709538759}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6828","nonce":1709538759},{"type":"v1","addr":"172.21.15.43:6830","nonce":1709538759}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6825","nonce":1709538759},{"type":"v1","addr":"172.21.15.43:6827","nonce":1709538759}]},"public_addr":"172.21.15.43:6816/1709538759","cluster_addr":"172.21.15.43:6822/1709538759","heartbeat_back_addr":"172.21.15.43:6830/1709538759","heartbeat_front_addr":"172.21.15.43:6827/1709538759","state":["exists","up"]},{"osd":4,"uuid":"fa2e8a77-aba1-4756-853e-0d174a5bdeba","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":114,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6800","nonce":4196866481},{"type":"v1","addr":"172.21.15.203:6801","nonce":4196866481}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6802","nonce":4196866481},{"type":"v1","addr":"172.21.15.203:6803","nonce":4196866481}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6806","nonce":4196866481},{"type":"v1","addr":"172.21.15.203:6807","nonce":4196866481}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6804","nonce":4196866481},{"type":"v1","addr":"172.21.15.203:6805","nonce":4196866481}]},"public_addr":"172.21.15.203:6801/4196866481","cluster_addr":"172.21.15.203:6803/4196866481","heartbeat_back_addr":"172.21.15.203:6807/4196866481","heartbeat_front_addr":"172.21.15.203:6805/4196866481","state":["exists","up"]},{"osd":5,"uuid":"81811e4b-4d24-4f51-a82c-f1d700a2a00d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":114,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6816","nonce":3059290081},{"type":"v1","addr":"172.21.15.203:6817","nonce":3059290081}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6818","nonce":3059290081},{"type":"v1","addr":"172.21.15.203:6819","nonce":3059290081}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6822","nonce":3059290081},{"type":"v1","addr":"172.21.15.203:6823","nonce":3059290081}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6820","nonce":3059290081},{"type":"v1","addr":"172.21.15.203:6821","nonce":3059290081}]},"public_addr":"172.21.15.203:6817/3059290081","cluster_addr":"172.21.15.203:6819/3059290081","heartbeat_back_addr":"172.21.15.203:6823/3059290081","heartbeat_front_addr":"172.21.15.203:6821/3059290081","state":["exists","up"]},{"osd":6,"uuid":"307bac4d-6a5b-443d-af75-56d695d7674f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":114,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6824","nonce":512155569},{"type":"v1","addr":"172.21.15.203:6825","nonce":512155569}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6826","nonce":512155569},{"type":"v1","addr":"172.21.15.203:6827","nonce":512155569}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6830","nonce":512155569},{"type":"v1","addr":"172.21.15.203:6831","nonce":512155569}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6828","nonce":512155569},{"type":"v1","addr":"172.21.15.203:6829","nonce":512155569}]},"public_addr":"172.21.15.203:6825/512155569","cluster_addr":"172.21.15.203:6827/512155569","heartbeat_back_addr":"172.21.15.203:6831/512155569","heartbeat_front_addr":"172.21.15.203:6829/512155569","state":["exists","up"]},{"osd":7,"uuid":"a95dc7ca-9f1b-4295-af70-68a22a2d7d5e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":114,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6808","nonce":3331450438},{"type":"v1","addr":"172.21.15.203:6809","nonce":3331450438}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6810","nonce":3331450438},{"type":"v1","addr":"172.21.15.203:6811","nonce":3331450438}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6814","nonce":3331450438},{"type":"v1","addr":"172.21.15.203:6815","nonce":3331450438}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6812","nonce":3331450438},{"type":"v1","addr":"172.21.15.203:6813","nonce":3331450438}]},"public_addr":"172.21.15.203:6809/3331450438","cluster_addr":"172.21.15.203:6811/3331450438","heartbeat_back_addr":"172.21.15.203:6815/3331450438","heartbeat_front_addr":"172.21.15.203:6813/3331450438","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-09T13:33:50.547652+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-09T13:33:50.589469+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-09T13:33:50.561724+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-09T13:33:50.599650+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-09T13:33:50.411855+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-09T13:33:50.219681+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-09T13:33:50.504664+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-09T13:33:50.441248+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.203:0/1488154102":"2023-08-10T13:52:20.374695+0000","172.21.15.203:0/3595044319":"2023-08-10T13:52:20.374695+0000","172.21.15.203:0/1657891023":"2023-08-10T13:52:20.374695+0000","172.21.15.203:0/2368610482":"2023-08-10T13:52:20.374695+0000","172.21.15.203:0/437192587":"2023-08-10T13:52:20.374695+0000","172.21.15.203:6833/1317108840":"2023-08-10T13:52:20.374695+0000","172.21.15.203:6832/1317108840":"2023-08-10T13:52:20.374695+0000","172.21.15.203:0/1459918882":"2023-08-10T13:52:20.374695+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":7,"snaps":[{"begin":104,"length":1},{"begin":221,"length":1}]}],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-08-09T13:53:24.927 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:24.925+0000 7fa6f3457640 1 -- 172.21.15.43:0/690288894 >> [v2:172.21.15.203:6832/2041169533,v1:172.21.15.203:6833/2041169533] conn(0x7fa6d006a290 msgr2=0x7fa6d006c750 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-08-09T13:53:24.927 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:24.925+0000 7fa6f3457640 1 --2- 172.21.15.43:0/690288894 >> [v2:172.21.15.203:6832/2041169533,v1:172.21.15.203:6833/2041169533] conn(0x7fa6d006a290 0x7fa6d006c750 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:53:24.927 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:24.925+0000 7fa6f3457640 1 -- 172.21.15.43:0/690288894 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa6ec0b12b0 msgr2=0x7fa6ec0c3f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:53:24.927 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:24.925+0000 7fa6f3457640 1 --2- 172.21.15.43:0/690288894 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa6ec0b12b0 0x7fa6ec0c3f70 secure :-1 s=READY pgs=1406 cs=0 l=1 rev1=1 crypto rx=0x7fa6e4000f80 tx=0x7fa6e400e530 comp rx=0 tx=0).stop 2023-08-09T13:53:24.927 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:24.925+0000 7fa6f3457640 1 -- 172.21.15.43:0/690288894 shutdown_connections 2023-08-09T13:53:24.928 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:24.925+0000 7fa6f3457640 1 --2- 172.21.15.43:0/690288894 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa6ec0b12b0 0x7fa6ec0c3f70 unknown :-1 s=CLOSED pgs=1406 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:53:24.928 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:24.925+0000 7fa6f3457640 1 --2- 172.21.15.43:0/690288894 >> [v2:172.21.15.203:6832/2041169533,v1:172.21.15.203:6833/2041169533] conn(0x7fa6d006a290 0x7fa6d006c750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:53:24.928 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:24.925+0000 7fa6f3457640 1 --2- 172.21.15.43:0/690288894 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fa6ec0b0910 0x7fa6ec0b0d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:53:24.928 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:24.925+0000 7fa6f3457640 1 --2- 172.21.15.43:0/690288894 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fa6d8003f60 0x7fa6ec150df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:53:24.928 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:24.925+0000 7fa6f3457640 1 -- 172.21.15.43:0/690288894 >> 172.21.15.43:0/690288894 conn(0x7fa6ec0ccd80 msgr2=0x7fa6ec0ad750 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-09T13:53:24.928 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:24.925+0000 7fa6f3457640 1 -- 172.21.15.43:0/690288894 shutdown_connections 2023-08-09T13:53:24.929 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:24.925+0000 7fa6f3457640 1 -- 172.21.15.43:0/690288894 wait complete. 2023-08-09T13:53:24.945 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': 'rbd', 'create_time': '2023-08-09T13:33:54.365137+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}, 'read_balance': {'score_acting': 2, 'score_stable': 2, 'optimal_score': 1, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': '.mgr', 'create_time': '2023-08-09T13:33:54.831803+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 14, 'pool_name': 'cephfs_metadata', 'create_time': '2023-08-09T13:51:57.637653+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '115', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 2.5, 'score_stable': 2.5, 'optimal_score': 1, 'raw_score_acting': 2.5, 'raw_score_stable': 2.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 15, 'pool_name': 'cephfs_data', 'create_time': '2023-08-09T13:51:58.676117+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '116', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-08-09T13:53:24.946 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get rbd pg_num 2023-08-09T13:53:25.140 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:25.138+0000 7ff8cffff640 1 Processor -- start 2023-08-09T13:53:25.140 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:25.138+0000 7ff8cffff640 1 -- start start 2023-08-09T13:53:25.142 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:25.140+0000 7ff8cffff640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7ff8d00b7df0 0x7ff8d00d1b30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:53:25.142 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:25.140+0000 7ff8cffff640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7ff8d00cebf0 0x7ff8d00cf040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:53:25.142 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:25.140+0000 7ff8cffff640 1 -- --> v1:172.21.15.203:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff8d00cf580 con 0x7ff8d015c2d0 2023-08-09T13:53:25.142 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:25.140+0000 7ff8cffff640 1 -- --> v2:172.21.15.43:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff8d00b8230 con 0x7ff8d00b7df0 2023-08-09T13:53:25.143 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:25.140+0000 7ff8cffff640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff8d00d16a0 con 0x7ff8d00cebf0 2023-08-09T13:53:25.143 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:25.140+0000 7ff8ce7fc640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7ff8d00b7df0 0x7ff8d00d1b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:53:25.143 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:25.140+0000 7ff8ce7fc640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7ff8d00b7df0 0x7ff8d00d1b30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:58062/0 (socket says 172.21.15.43:58062) 2023-08-09T13:53:25.143 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:25.140+0000 7ff8ce7fc640 1 -- 172.21.15.43:0/3333405298 learned_addr learned my addr 172.21.15.43:0/3333405298 (peer_addr_for_me v2:172.21.15.43:0/0) 2023-08-09T13:53:25.143 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:25.140+0000 7ff8cf7fe640 1 --2- 172.21.15.43:0/3333405298 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7ff8d00cebf0 0x7ff8d00cf040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:53:25.145 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:25.141+0000 7ff8ce7fc640 1 -- 172.21.15.43:0/3333405298 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7ff8d00cebf0 msgr2=0x7ff8d00cf040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-09T13:53:25.145 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:25.141+0000 7ff8ce7fc640 1 --2- 172.21.15.43:0/3333405298 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7ff8d00cebf0 0x7ff8d00cf040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:53:25.145 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:25.141+0000 7ff8ce7fc640 1 -- 172.21.15.43:0/3333405298 >> v1:172.21.15.203:6789/0 conn(0x7ff8d015c2d0 legacy=0x7ff8d00cd020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:53:25.145 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:25.141+0000 7ff8ce7fc640 1 -- 172.21.15.43:0/3333405298 --> v2:172.21.15.43:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff8d00d0d70 con 0x7ff8d00b7df0 2023-08-09T13:53:25.145 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:25.142+0000 7ff8ce7fc640 1 --2- 172.21.15.43:0/3333405298 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff8d00b7df0 0x7ff8d00d1b30 secure :-1 s=READY pgs=1407 cs=0 l=1 rev1=1 crypto rx=0x7ff8c4009870 tx=0x7ff8c402b980 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b197f7e799cbe2c6 server_cookie=0 in_seq=0 out_seq=0 2023-08-09T13:53:25.145 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:25.142+0000 7ff8cdffb640 1 -- 172.21.15.43:0/3333405298 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff8c4030840 con 0x7ff8d00b7df0 2023-08-09T13:53:25.146 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:25.142+0000 7ff8cdffb640 1 -- 172.21.15.43:0/3333405298 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff8c40309a0 con 0x7ff8d00b7df0 2023-08-09T13:53:25.146 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:25.142+0000 7ff8cdffb640 1 -- 172.21.15.43:0/3333405298 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff8c4030d30 con 0x7ff8d00b7df0 2023-08-09T13:53:25.146 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:25.143+0000 7ff8cffff640 1 -- 172.21.15.43:0/3333405298 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff8d00b7df0 msgr2=0x7ff8d00d1b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:53:25.146 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:25.143+0000 7ff8cffff640 1 --2- 172.21.15.43:0/3333405298 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff8d00b7df0 0x7ff8d00d1b30 secure :-1 s=READY pgs=1407 cs=0 l=1 rev1=1 crypto rx=0x7ff8c4009870 tx=0x7ff8c402b980 comp rx=0 tx=0).stop 2023-08-09T13:53:25.146 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:25.143+0000 7ff8cffff640 1 -- 172.21.15.43:0/3333405298 shutdown_connections 2023-08-09T13:53:25.146 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:25.143+0000 7ff8cffff640 1 --2- 172.21.15.43:0/3333405298 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7ff8d00cebf0 0x7ff8d00cf040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:53:25.147 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:25.143+0000 7ff8cffff640 1 --2- 172.21.15.43:0/3333405298 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff8d00b7df0 0x7ff8d00d1b30 unknown :-1 s=CLOSED pgs=1407 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:53:25.147 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:25.143+0000 7ff8cffff640 1 -- 172.21.15.43:0/3333405298 >> 172.21.15.43:0/3333405298 conn(0x7ff8d006a230 msgr2=0x7ff8d006a640 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-09T13:53:25.147 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:25.143+0000 7ff8cffff640 1 -- 172.21.15.43:0/3333405298 shutdown_connections 2023-08-09T13:53:25.147 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:25.143+0000 7ff8cffff640 1 -- 172.21.15.43:0/3333405298 wait complete. 2023-08-09T13:53:25.148 INFO:tasks.ceph.mgr.x.smithi203.stderr:2023-08-09T13:53:25.144+0000 7f005263d040 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2023-08-09T13:53:25.149 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:25.145+0000 7ff8cffff640 1 Processor -- start 2023-08-09T13:53:25.150 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:25.145+0000 7ff8cffff640 1 -- start start 2023-08-09T13:53:25.150 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:25.146+0000 7ff8cffff640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7ff8d00b7df0 0x7ff8d0156850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:53:25.150 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:25.146+0000 7ff8cffff640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7ff8d00cebf0 0x7ff8d0156040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:53:25.150 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:25.146+0000 7ff8cffff640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff8d015c2d0 0x7ff8d00a97d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:53:25.150 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:25.146+0000 7ff8cffff640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff8d00ca740 con 0x7ff8d015c2d0 2023-08-09T13:53:25.150 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:25.146+0000 7ff8cffff640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff8d00aa340 con 0x7ff8d00cebf0 2023-08-09T13:53:25.151 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:25.146+0000 7ff8cffff640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff8d00ca990 con 0x7ff8d00b7df0 2023-08-09T13:53:25.151 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:25.146+0000 7ff8cf7fe640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff8d015c2d0 0x7ff8d00a97d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:53:25.151 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:25.146+0000 7ff8cf7fe640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff8d015c2d0 0x7ff8d00a97d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:58076/0 (socket says 172.21.15.43:58076) 2023-08-09T13:53:25.151 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:25.146+0000 7ff8cf7fe640 1 -- 172.21.15.43:0/1911823532 learned_addr learned my addr 172.21.15.43:0/1911823532 (peer_addr_for_me v2:172.21.15.43:0/0) 2023-08-09T13:53:25.151 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:25.146+0000 7ff8ce7fc640 1 --2- 172.21.15.43:0/1911823532 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7ff8d00cebf0 0x7ff8d0156040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:53:25.151 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:25.146+0000 7ff8ceffd640 1 --2- 172.21.15.43:0/1911823532 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7ff8d00b7df0 0x7ff8d0156850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:53:25.151 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:25.146+0000 7ff8cf7fe640 1 -- 172.21.15.43:0/1911823532 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7ff8d00b7df0 msgr2=0x7ff8d0156850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:53:25.152 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:25.146+0000 7ff8cf7fe640 1 --2- 172.21.15.43:0/1911823532 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7ff8d00b7df0 0x7ff8d0156850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:53:25.152 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:25.146+0000 7ff8cf7fe640 1 -- 172.21.15.43:0/1911823532 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7ff8d00cebf0 msgr2=0x7ff8d0156040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:53:25.152 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:25.146+0000 7ff8cf7fe640 1 --2- 172.21.15.43:0/1911823532 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7ff8d00cebf0 0x7ff8d0156040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:53:25.152 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:25.146+0000 7ff8cf7fe640 1 -- 172.21.15.43:0/1911823532 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff8c40094d0 con 0x7ff8d015c2d0 2023-08-09T13:53:25.152 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:25.146+0000 7ff8ceffd640 1 --2- 172.21.15.43:0/1911823532 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7ff8d00b7df0 0x7ff8d0156850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-08-09T13:53:25.152 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:25.147+0000 7ff8cf7fe640 1 --2- 172.21.15.43:0/1911823532 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff8d015c2d0 0x7ff8d00a97d0 secure :-1 s=READY pgs=1408 cs=0 l=1 rev1=1 crypto rx=0x7ff8c8007e40 tx=0x7ff8c800e4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-09T13:53:25.152 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:25.147+0000 7ff8b77fe640 1 -- 172.21.15.43:0/1911823532 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff8c8014620 con 0x7ff8d015c2d0 2023-08-09T13:53:25.153 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:25.147+0000 7ff8b77fe640 1 -- 172.21.15.43:0/1911823532 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff8c800ee90 con 0x7ff8d015c2d0 2023-08-09T13:53:25.153 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:25.147+0000 7ff8b77fe640 1 -- 172.21.15.43:0/1911823532 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff8c80149e0 con 0x7ff8d015c2d0 2023-08-09T13:53:25.153 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:25.147+0000 7ff8cffff640 1 -- 172.21.15.43:0/1911823532 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff8d00cac70 con 0x7ff8d015c2d0 2023-08-09T13:53:25.153 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:25.147+0000 7ff8cffff640 1 -- 172.21.15.43:0/1911823532 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff8d01f5ea0 con 0x7ff8d015c2d0 2023-08-09T13:53:25.153 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:25.148+0000 7ff8cffff640 1 -- 172.21.15.43:0/1911823532 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff8d004e390 con 0x7ff8d015c2d0 2023-08-09T13:53:25.154 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:25.149+0000 7ff8b77fe640 1 -- 172.21.15.43:0/1911823532 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 52) v1 ==== 88557+0+0 (secure 0 0 0) 0x7ff8c8014b40 con 0x7ff8d015c2d0 2023-08-09T13:53:25.154 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:25.150+0000 7ff8b77fe640 1 --2- 172.21.15.43:0/1911823532 >> [v2:172.21.15.203:6832/2041169533,v1:172.21.15.203:6833/2041169533] conn(0x7ff8bc06a2c0 0x7ff8bc06c780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:53:25.154 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:25.150+0000 7ff8b77fe640 1 -- 172.21.15.43:0/1911823532 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(117..117 src has 1..117) v4 ==== 6770+0+0 (secure 0 0 0) 0x7ff8c80895a0 con 0x7ff8d015c2d0 2023-08-09T13:53:25.154 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:25.150+0000 7ff8ceffd640 1 -- 172.21.15.43:0/1911823532 >> [v2:172.21.15.203:6832/2041169533,v1:172.21.15.203:6833/2041169533] conn(0x7ff8bc06a2c0 msgr2=0x7ff8bc06c780 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.203:6832/2041169533 2023-08-09T13:53:25.154 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:25.150+0000 7ff8ceffd640 1 --2- 172.21.15.43:0/1911823532 >> [v2:172.21.15.203:6832/2041169533,v1:172.21.15.203:6833/2041169533] conn(0x7ff8bc06a2c0 0x7ff8bc06c780 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-08-09T13:53:25.156 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:25.154+0000 7ff8b77fe640 1 -- 172.21.15.43:0/1911823532 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178066 (secure 0 0 0) 0x7ff8c8054690 con 0x7ff8d015c2d0 2023-08-09T13:53:25.210 INFO:tasks.ceph.mgr.y.smithi043.stderr:2023-08-09T13:53:25.208+0000 7fa15623e040 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2023-08-09T13:53:25.267 INFO:tasks.ceph.mgr.x.smithi203.stderr:2023-08-09T13:53:25.264+0000 7f005263d040 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2023-08-09T13:53:25.324 INFO:tasks.ceph.mgr.x.smithi203.stderr:2023-08-09T13:53:25.321+0000 7f005263d040 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2023-08-09T13:53:25.335 INFO:tasks.ceph.mgr.y.smithi043.stderr:2023-08-09T13:53:25.334+0000 7fa15623e040 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2023-08-09T13:53:25.348 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:25.347+0000 7ff8cffff640 1 -- 172.21.15.43:0/1911823532 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"} v 0) v1 -- 0x7ff8d0065770 con 0x7ff8d015c2d0 2023-08-09T13:53:25.349 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:25.347+0000 7ff8b77fe640 1 -- 172.21.15.43:0/1911823532 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"}]=0 v117) v1 ==== 92+0+10 (secure 0 0 0) 0x7ff8c8057cb0 con 0x7ff8d015c2d0 2023-08-09T13:53:25.349 INFO:teuthology.orchestra.run.smithi043.stdout:pg_num: 8 2023-08-09T13:53:25.351 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:25.350+0000 7ff8cffff640 1 -- 172.21.15.43:0/1911823532 >> [v2:172.21.15.203:6832/2041169533,v1:172.21.15.203:6833/2041169533] conn(0x7ff8bc06a2c0 msgr2=0x7ff8bc06c780 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-08-09T13:53:25.351 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:25.350+0000 7ff8cffff640 1 --2- 172.21.15.43:0/1911823532 >> [v2:172.21.15.203:6832/2041169533,v1:172.21.15.203:6833/2041169533] conn(0x7ff8bc06a2c0 0x7ff8bc06c780 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:53:25.352 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:25.350+0000 7ff8cffff640 1 -- 172.21.15.43:0/1911823532 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff8d015c2d0 msgr2=0x7ff8d00a97d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:53:25.352 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:25.350+0000 7ff8cffff640 1 --2- 172.21.15.43:0/1911823532 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff8d015c2d0 0x7ff8d00a97d0 secure :-1 s=READY pgs=1408 cs=0 l=1 rev1=1 crypto rx=0x7ff8c8007e40 tx=0x7ff8c800e4b0 comp rx=0 tx=0).stop 2023-08-09T13:53:25.352 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:25.350+0000 7ff8cffff640 1 -- 172.21.15.43:0/1911823532 shutdown_connections 2023-08-09T13:53:25.353 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:25.350+0000 7ff8cffff640 1 --2- 172.21.15.43:0/1911823532 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff8d015c2d0 0x7ff8d00a97d0 unknown :-1 s=CLOSED pgs=1408 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:53:25.353 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:25.350+0000 7ff8cffff640 1 --2- 172.21.15.43:0/1911823532 >> [v2:172.21.15.203:6832/2041169533,v1:172.21.15.203:6833/2041169533] conn(0x7ff8bc06a2c0 0x7ff8bc06c780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:53:25.353 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:25.350+0000 7ff8cffff640 1 --2- 172.21.15.43:0/1911823532 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7ff8d00cebf0 0x7ff8d0156040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:53:25.353 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:25.350+0000 7ff8cffff640 1 --2- 172.21.15.43:0/1911823532 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7ff8d00b7df0 0x7ff8d0156850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:53:25.353 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:25.350+0000 7ff8cffff640 1 -- 172.21.15.43:0/1911823532 >> 172.21.15.43:0/1911823532 conn(0x7ff8d006a230 msgr2=0x7ff8d00b3900 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-09T13:53:25.353 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:25.351+0000 7ff8cffff640 1 -- 172.21.15.43:0/1911823532 shutdown_connections 2023-08-09T13:53:25.354 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:25.351+0000 7ff8cffff640 1 -- 172.21.15.43:0/1911823532 wait complete. 2023-08-09T13:53:25.370 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-08-09T13:53:25.373 INFO:tasks.ceph.mgr.x.smithi203.stderr:2023-08-09T13:53:25.371+0000 7f005263d040 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2023-08-09T13:53:25.396 INFO:tasks.ceph.mgr.y.smithi043.stderr:2023-08-09T13:53:25.395+0000 7fa15623e040 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2023-08-09T13:53:25.450 INFO:tasks.ceph.mgr.y.smithi043.stderr:2023-08-09T13:53:25.448+0000 7fa15623e040 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2023-08-09T13:53:25.580 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:25.578+0000 7f2eb0fe4640 1 Processor -- start 2023-08-09T13:53:25.580 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:25.579+0000 7f2eb0fe4640 1 -- start start 2023-08-09T13:53:25.581 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:25.580+0000 7f2eb0fe4640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f2eac15c550 0x7f2eac0cd6d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:53:25.582 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:25.580+0000 7f2eb0fe4640 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7f2eac0d04b0 0x7f2eac0b3ee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:53:25.582 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:25.580+0000 7f2eb0fe4640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f2eac0b4420 0x7f2eac06a760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:53:25.582 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:25.580+0000 7f2eb0fe4640 1 -- --> v2:172.21.15.43:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2eac0b4870 con 0x7f2eac15c550 2023-08-09T13:53:25.582 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:25.580+0000 7f2eb0fe4640 1 -- --> v2:172.21.15.203:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2eac0b7bd0 con 0x7f2eac0d04b0 2023-08-09T13:53:25.582 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:25.580+0000 7f2eb0fe4640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2eac0d1c00 con 0x7f2eac0b4420 2023-08-09T13:53:25.583 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:25.580+0000 7f2eaa59c640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f2eac15c550 0x7f2eac0cd6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:53:25.583 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:25.581+0000 7f2eaa59c640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f2eac15c550 0x7f2eac0cd6d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:58082/0 (socket says 172.21.15.43:58082) 2023-08-09T13:53:25.584 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:25.581+0000 7f2eaa59c640 1 -- 172.21.15.43:0/1588168648 learned_addr learned my addr 172.21.15.43:0/1588168648 (peer_addr_for_me v2:172.21.15.43:0/0) 2023-08-09T13:53:25.584 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:25.581+0000 7f2ea3fff640 1 --2- 172.21.15.43:0/1588168648 >> v2:172.21.15.203:3300/0 conn(0x7f2eac0d04b0 0x7f2eac0b3ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:53:25.584 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:25.581+0000 7f2eaad9d640 1 --2- 172.21.15.43:0/1588168648 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f2eac0b4420 0x7f2eac06a760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:53:25.584 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:25.581+0000 7f2eaa59c640 1 -- 172.21.15.43:0/1588168648 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f2eac0b4420 msgr2=0x7f2eac06a760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-09T13:53:25.584 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:25.581+0000 7f2eaa59c640 1 --2- 172.21.15.43:0/1588168648 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f2eac0b4420 0x7f2eac06a760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:53:25.584 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:25.581+0000 7f2eaa59c640 1 -- 172.21.15.43:0/1588168648 >> v2:172.21.15.203:3300/0 conn(0x7f2eac0d04b0 msgr2=0x7f2eac0b3ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-09T13:53:25.584 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:25.581+0000 7f2eaa59c640 1 --2- 172.21.15.43:0/1588168648 >> v2:172.21.15.203:3300/0 conn(0x7f2eac0d04b0 0x7f2eac0b3ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:53:25.585 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:25.581+0000 7f2eaa59c640 1 -- 172.21.15.43:0/1588168648 --> v2:172.21.15.43:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2eac15c930 con 0x7f2eac15c550 2023-08-09T13:53:25.585 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:25.582+0000 7f2eaa59c640 1 --2- 172.21.15.43:0/1588168648 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f2eac15c550 0x7f2eac0cd6d0 secure :-1 s=READY pgs=1409 cs=0 l=1 rev1=1 crypto rx=0x7f2eac0b5050 tx=0x7f2e9c02b880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1d1dfccbc307d67d server_cookie=0 in_seq=0 out_seq=0 2023-08-09T13:53:25.586 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:25.583+0000 7f2ea9d9b640 1 -- 172.21.15.43:0/1588168648 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2e9c0317e0 con 0x7f2eac15c550 2023-08-09T13:53:25.586 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:25.583+0000 7f2ea9d9b640 1 -- 172.21.15.43:0/1588168648 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2e9c031940 con 0x7f2eac15c550 2023-08-09T13:53:25.586 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:25.583+0000 7f2ea9d9b640 1 -- 172.21.15.43:0/1588168648 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2e9c031cd0 con 0x7f2eac15c550 2023-08-09T13:53:25.586 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:25.583+0000 7f2eb0fe4640 1 -- 172.21.15.43:0/1588168648 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f2eac15c550 msgr2=0x7f2eac0cd6d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:53:25.586 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:25.583+0000 7f2eb0fe4640 1 --2- 172.21.15.43:0/1588168648 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f2eac15c550 0x7f2eac0cd6d0 secure :-1 s=READY pgs=1409 cs=0 l=1 rev1=1 crypto rx=0x7f2eac0b5050 tx=0x7f2e9c02b880 comp rx=0 tx=0).stop 2023-08-09T13:53:25.587 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:25.583+0000 7f2eb0fe4640 1 -- 172.21.15.43:0/1588168648 shutdown_connections 2023-08-09T13:53:25.587 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:25.583+0000 7f2eb0fe4640 1 --2- 172.21.15.43:0/1588168648 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f2eac0b4420 0x7f2eac06a760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:53:25.587 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:25.583+0000 7f2eb0fe4640 1 --2- 172.21.15.43:0/1588168648 >> v2:172.21.15.203:3300/0 conn(0x7f2eac0d04b0 0x7f2eac0b3ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:53:25.587 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:25.583+0000 7f2eb0fe4640 1 --2- 172.21.15.43:0/1588168648 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f2eac15c550 0x7f2eac0cd6d0 unknown :-1 s=CLOSED pgs=1409 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:53:25.587 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:25.584+0000 7f2eb0fe4640 1 -- 172.21.15.43:0/1588168648 >> 172.21.15.43:0/1588168648 conn(0x7f2eac071520 msgr2=0x7f2eac0ceee0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-09T13:53:25.588 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:25.584+0000 7f2eb0fe4640 1 -- 172.21.15.43:0/1588168648 shutdown_connections 2023-08-09T13:53:25.588 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:25.584+0000 7f2eb0fe4640 1 -- 172.21.15.43:0/1588168648 wait complete. 2023-08-09T13:53:25.589 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:25.585+0000 7f2eb0fe4640 1 Processor -- start 2023-08-09T13:53:25.590 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:25.586+0000 7f2eb0fe4640 1 -- start start 2023-08-09T13:53:25.590 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:25.586+0000 7f2eb0fe4640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f2eac0b4420 0x7f2eac0cb920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:53:25.590 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:25.587+0000 7f2eb0fe4640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f2eac0d04b0 0x7f2eac0c61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:53:25.590 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:25.587+0000 7f2eb0fe4640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f2eac15c550 0x7f2eac156660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:53:25.590 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:25.587+0000 7f2eb0fe4640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2eac0caf80 con 0x7f2eac15c550 2023-08-09T13:53:25.590 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:25.587+0000 7f2eb0fe4640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2eac0cbe60 con 0x7f2eac0d04b0 2023-08-09T13:53:25.590 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:25.587+0000 7f2eb0fe4640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2eac0ca300 con 0x7f2eac0b4420 2023-08-09T13:53:25.591 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:25.587+0000 7f2eaad9d640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f2eac15c550 0x7f2eac156660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:53:25.591 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:25.587+0000 7f2eaad9d640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f2eac15c550 0x7f2eac156660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:58098/0 (socket says 172.21.15.43:58098) 2023-08-09T13:53:25.591 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:25.587+0000 7f2eaad9d640 1 -- 172.21.15.43:0/2025026381 learned_addr learned my addr 172.21.15.43:0/2025026381 (peer_addr_for_me v2:172.21.15.43:0/0) 2023-08-09T13:53:25.591 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:25.587+0000 7f2ea3fff640 1 --2- 172.21.15.43:0/2025026381 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f2eac0d04b0 0x7f2eac0c61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:53:25.591 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:25.587+0000 7f2eaad9d640 1 -- 172.21.15.43:0/2025026381 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f2eac0b4420 msgr2=0x7f2eac0cb920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:53:25.591 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:25.587+0000 7f2eaad9d640 1 --2- 172.21.15.43:0/2025026381 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f2eac0b4420 0x7f2eac0cb920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:53:25.591 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:25.587+0000 7f2eaad9d640 1 -- 172.21.15.43:0/2025026381 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f2eac0d04b0 msgr2=0x7f2eac0c61e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:53:25.592 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:25.587+0000 7f2eaad9d640 1 --2- 172.21.15.43:0/2025026381 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f2eac0d04b0 0x7f2eac0c61e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:53:25.592 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:25.587+0000 7f2eaad9d640 1 -- 172.21.15.43:0/2025026381 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2e9c0094d0 con 0x7f2eac15c550 2023-08-09T13:53:25.592 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:25.587+0000 7f2ea3fff640 1 --2- 172.21.15.43:0/2025026381 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f2eac0d04b0 0x7f2eac0c61e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-08-09T13:53:25.592 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:25.588+0000 7f2eaad9d640 1 --2- 172.21.15.43:0/2025026381 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f2eac15c550 0x7f2eac156660 secure :-1 s=READY pgs=1410 cs=0 l=1 rev1=1 crypto rx=0x7f2ea400cd70 tx=0x7f2ea400a660 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-09T13:53:25.592 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:25.588+0000 7f2ea2ffd640 1 -- 172.21.15.43:0/2025026381 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2ea4003b90 con 0x7f2eac15c550 2023-08-09T13:53:25.592 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:25.588+0000 7f2ea2ffd640 1 -- 172.21.15.43:0/2025026381 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2ea4003cf0 con 0x7f2eac15c550 2023-08-09T13:53:25.593 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:25.588+0000 7f2ea2ffd640 1 -- 172.21.15.43:0/2025026381 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2ea4004240 con 0x7f2eac15c550 2023-08-09T13:53:25.593 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:25.588+0000 7f2eb0fe4640 1 -- 172.21.15.43:0/2025026381 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2eac0ca5e0 con 0x7f2eac15c550 2023-08-09T13:53:25.593 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:25.588+0000 7f2eb0fe4640 1 -- 172.21.15.43:0/2025026381 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2eac0caba0 con 0x7f2eac15c550 2023-08-09T13:53:25.594 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:25.589+0000 7f2eb0fe4640 1 -- 172.21.15.43:0/2025026381 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2eac0739f0 con 0x7f2eac15c550 2023-08-09T13:53:25.594 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:25.590+0000 7f2ea2ffd640 1 -- 172.21.15.43:0/2025026381 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 52) v1 ==== 88557+0+0 (secure 0 0 0) 0x7f2ea40043a0 con 0x7f2eac15c550 2023-08-09T13:53:25.594 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:25.591+0000 7f2ea2ffd640 1 --2- 172.21.15.43:0/2025026381 >> [v2:172.21.15.203:6832/2041169533,v1:172.21.15.203:6833/2041169533] conn(0x7f2e8c06a290 0x7f2e8c06c750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:53:25.594 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:25.591+0000 7f2ea2ffd640 1 -- 172.21.15.43:0/2025026381 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(117..117 src has 1..117) v4 ==== 6770+0+0 (secure 0 0 0) 0x7f2ea4012070 con 0x7f2eac15c550 2023-08-09T13:53:25.594 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:25.591+0000 7f2eaa59c640 1 -- 172.21.15.43:0/2025026381 >> [v2:172.21.15.203:6832/2041169533,v1:172.21.15.203:6833/2041169533] conn(0x7f2e8c06a290 msgr2=0x7f2e8c06c750 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.203:6832/2041169533 2023-08-09T13:53:25.595 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:25.591+0000 7f2eaa59c640 1 --2- 172.21.15.43:0/2025026381 >> [v2:172.21.15.203:6832/2041169533,v1:172.21.15.203:6833/2041169533] conn(0x7f2e8c06a290 0x7f2e8c06c750 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-08-09T13:53:25.597 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:25.595+0000 7f2ea2ffd640 1 -- 172.21.15.43:0/2025026381 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178066 (secure 0 0 0) 0x7f2ea40546f0 con 0x7f2eac15c550 2023-08-09T13:53:25.789 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:25.788+0000 7f2eb0fe4640 1 -- 172.21.15.43:0/2025026381 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f2eac067be0 con 0x7f2eac15c550 2023-08-09T13:53:25.790 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:25.789+0000 7f2ea2ffd640 1 -- 172.21.15.43:0/2025026381 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v117) v1 ==== 93+0+10 (secure 0 0 0) 0x7f2ea4057d10 con 0x7f2eac15c550 2023-08-09T13:53:25.791 INFO:teuthology.orchestra.run.smithi043.stdout:pg_num: 1 2023-08-09T13:53:25.793 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:25.792+0000 7f2eaa59c640 1 -- 172.21.15.43:0/2025026381 >> [v2:172.21.15.203:6832/2041169533,v1:172.21.15.203:6833/2041169533] conn(0x7f2e8c06a290 msgr2=0x7f2e8c06c750 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.203:6832/2041169533 2023-08-09T13:53:25.793 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:25.792+0000 7f2eaa59c640 1 --2- 172.21.15.43:0/2025026381 >> [v2:172.21.15.203:6832/2041169533,v1:172.21.15.203:6833/2041169533] conn(0x7f2e8c06a290 0x7f2e8c06c750 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2023-08-09T13:53:25.794 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:25.793+0000 7f2eb0fe4640 1 -- 172.21.15.43:0/2025026381 >> [v2:172.21.15.203:6832/2041169533,v1:172.21.15.203:6833/2041169533] conn(0x7f2e8c06a290 msgr2=0x7f2e8c06c750 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-08-09T13:53:25.795 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:25.793+0000 7f2eb0fe4640 1 --2- 172.21.15.43:0/2025026381 >> [v2:172.21.15.203:6832/2041169533,v1:172.21.15.203:6833/2041169533] conn(0x7f2e8c06a290 0x7f2e8c06c750 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:53:25.795 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:25.793+0000 7f2eb0fe4640 1 -- 172.21.15.43:0/2025026381 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f2eac15c550 msgr2=0x7f2eac156660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:53:25.795 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:25.793+0000 7f2eb0fe4640 1 --2- 172.21.15.43:0/2025026381 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f2eac15c550 0x7f2eac156660 secure :-1 s=READY pgs=1410 cs=0 l=1 rev1=1 crypto rx=0x7f2ea400cd70 tx=0x7f2ea400a660 comp rx=0 tx=0).stop 2023-08-09T13:53:25.795 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:25.793+0000 7f2eb0fe4640 1 -- 172.21.15.43:0/2025026381 shutdown_connections 2023-08-09T13:53:25.795 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:25.793+0000 7f2eb0fe4640 1 --2- 172.21.15.43:0/2025026381 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f2eac15c550 0x7f2eac156660 unknown :-1 s=CLOSED pgs=1410 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:53:25.795 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:25.793+0000 7f2eb0fe4640 1 --2- 172.21.15.43:0/2025026381 >> [v2:172.21.15.203:6832/2041169533,v1:172.21.15.203:6833/2041169533] conn(0x7f2e8c06a290 0x7f2e8c06c750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:53:25.795 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:25.793+0000 7f2eb0fe4640 1 --2- 172.21.15.43:0/2025026381 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f2eac0d04b0 0x7f2eac0c61e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:53:25.796 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:25.793+0000 7f2eb0fe4640 1 --2- 172.21.15.43:0/2025026381 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f2eac0b4420 0x7f2eac0cb920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:53:25.796 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:25.793+0000 7f2eb0fe4640 1 -- 172.21.15.43:0/2025026381 >> 172.21.15.43:0/2025026381 conn(0x7f2eac071520 msgr2=0x7f2eac15ddc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-09T13:53:25.796 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:25.793+0000 7f2eb0fe4640 1 -- 172.21.15.43:0/2025026381 shutdown_connections 2023-08-09T13:53:25.796 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:25.793+0000 7f2eb0fe4640 1 -- 172.21.15.43:0/2025026381 wait complete. 2023-08-09T13:53:25.821 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-08-09T13:53:26.068 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:26.067+0000 7f10db01a640 1 Processor -- start 2023-08-09T13:53:26.069 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:26.067+0000 7f10db01a640 1 -- start start 2023-08-09T13:53:26.070 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:26.068+0000 7f10db01a640 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7f10d415c3a0 0x7f10d41608a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:53:26.070 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:26.068+0000 7f10db01a640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f10d40cd600 0x7f10d40d0580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:53:26.070 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:26.068+0000 7f10db01a640 1 -- --> v1:172.21.15.43:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f10d4160de0 con 0x7f10d40b0f00 2023-08-09T13:53:26.070 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:26.068+0000 7f10db01a640 1 -- --> v2:172.21.15.203:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f10d40b81c0 con 0x7f10d415c3a0 2023-08-09T13:53:26.070 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:26.068+0000 7f10db01a640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f10d40b7c00 con 0x7f10d40cd600 2023-08-09T13:53:26.072 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:26.069+0000 7f10d9817640 1 --1- >> v1:172.21.15.43:6789/0 conn(0x7f10d40b0f00 0x7f10d40ccef0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.43:6789/0 says I am v1:172.21.15.43:40964/0 (socket says 172.21.15.43:40964) 2023-08-09T13:53:26.072 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:26.069+0000 7f10d9817640 1 -- 172.21.15.43:0/4047204070 learned_addr learned my addr 172.21.15.43:0/4047204070 (peer_addr_for_me v1:172.21.15.43:0/0) 2023-08-09T13:53:26.072 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:26.069+0000 7f10da018640 1 --2- 172.21.15.43:0/4047204070 >> v2:172.21.15.203:3300/0 conn(0x7f10d415c3a0 0x7f10d41608a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:53:26.072 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:26.069+0000 7f10da819640 1 --2- 172.21.15.43:0/4047204070 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f10d40cd600 0x7f10d40d0580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:53:26.072 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:26.070+0000 7f10d9016640 1 -- 172.21.15.43:0/4047204070 <== mon.0 v1:172.21.15.43:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4105112362 0 0) 0x7f10d4160de0 con 0x7f10d40b0f00 2023-08-09T13:53:26.072 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:26.070+0000 7f10d9016640 1 -- 172.21.15.43:0/4047204070 --> v1:172.21.15.43:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f10c0003520 con 0x7f10d40b0f00 2023-08-09T13:53:26.072 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:26.070+0000 7f10d9016640 1 -- 172.21.15.43:0/4047204070 <== mon.0 v1:172.21.15.43:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1306456406 0 0) 0x7f10c0003520 con 0x7f10d40b0f00 2023-08-09T13:53:26.073 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:26.070+0000 7f10d9016640 1 -- 172.21.15.43:0/4047204070 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f10d40cd600 msgr2=0x7f10d40d0580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-09T13:53:26.073 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:26.070+0000 7f10d9016640 1 --2- 172.21.15.43:0/4047204070 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f10d40cd600 0x7f10d40d0580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:53:26.073 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:26.070+0000 7f10d9016640 1 -- 172.21.15.43:0/4047204070 >> v2:172.21.15.203:3300/0 conn(0x7f10d415c3a0 msgr2=0x7f10d41608a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-09T13:53:26.073 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:26.070+0000 7f10d9016640 1 --2- 172.21.15.43:0/4047204070 >> v2:172.21.15.203:3300/0 conn(0x7f10d415c3a0 0x7f10d41608a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:53:26.073 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:26.070+0000 7f10d9016640 1 -- 172.21.15.43:0/4047204070 --> v1:172.21.15.43:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f10d40b7ee0 con 0x7f10d40b0f00 2023-08-09T13:53:26.073 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:26.070+0000 7f10d9016640 1 -- 172.21.15.43:0/4047204070 <== mon.0 v1:172.21.15.43:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1285150740 0 0) 0x7f10c40017f0 con 0x7f10d40b0f00 2023-08-09T13:53:26.073 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:26.070+0000 7f10d9016640 1 -- 172.21.15.43:0/4047204070 >> v1:172.21.15.43:6789/0 conn(0x7f10d40b0f00 legacy=0x7f10d40ccef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:53:26.074 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:26.070+0000 7f10d9016640 1 --2- 172.21.15.43:0/4047204070 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f10c0003970 0x7f10c0003d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:53:26.074 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:26.070+0000 7f10d9016640 1 --2- 172.21.15.43:0/4047204070 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f10c00047c0 0x7f10c0004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:53:26.074 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:26.070+0000 7f10d9016640 1 --2- 172.21.15.43:0/4047204070 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f10d40cd600 0x7f10c0005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:53:26.074 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:26.070+0000 7f10d9016640 1 -- 172.21.15.43:0/4047204070 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f10d40b81c0 con 0x7f10c00047c0 2023-08-09T13:53:26.074 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:26.070+0000 7f10d9016640 1 -- 172.21.15.43:0/4047204070 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f10d40b7c00 con 0x7f10c0003970 2023-08-09T13:53:26.074 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:26.070+0000 7f10d9016640 1 -- 172.21.15.43:0/4047204070 --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f10c000ba60 con 0x7f10d40cd600 2023-08-09T13:53:26.075 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:26.071+0000 7f10d9817640 1 --2- 172.21.15.43:0/4047204070 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f10c00047c0 0x7f10c0004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:53:26.076 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:26.071+0000 7f10da018640 1 --2- 172.21.15.43:0/4047204070 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f10c0003970 0x7f10c0003d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:53:26.076 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:26.071+0000 7f10d9817640 1 -- 172.21.15.43:0/4047204070 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f10d40cd600 msgr2=0x7f10c0005180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-09T13:53:26.076 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:26.071+0000 7f10d9817640 1 --2- 172.21.15.43:0/4047204070 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f10d40cd600 0x7f10c0005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:53:26.076 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:26.071+0000 7f10d9817640 1 -- 172.21.15.43:0/4047204070 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f10c0003970 msgr2=0x7f10c0003d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-09T13:53:26.076 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:26.071+0000 7f10d9817640 1 --2- 172.21.15.43:0/4047204070 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f10c0003970 0x7f10c0003d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:53:26.076 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:26.071+0000 7f10d9817640 1 -- 172.21.15.43:0/4047204070 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f10c0003520 con 0x7f10c00047c0 2023-08-09T13:53:26.076 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:26.072+0000 7f10d9817640 1 --2- 172.21.15.43:0/4047204070 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f10c00047c0 0x7f10c0004c40 secure :-1 s=READY pgs=1412 cs=0 l=1 rev1=1 crypto rx=0x7f10c400c190 tx=0x7f10c00057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b24fcf51e22ba7cc server_cookie=0 in_seq=0 out_seq=0 2023-08-09T13:53:26.077 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:26.073+0000 7f10d9016640 1 -- 172.21.15.43:0/4047204070 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f10c4007270 con 0x7f10c00047c0 2023-08-09T13:53:26.077 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:26.073+0000 7f10d9016640 1 -- 172.21.15.43:0/4047204070 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f10c40073d0 con 0x7f10c00047c0 2023-08-09T13:53:26.077 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:26.073+0000 7f10d9016640 1 -- 172.21.15.43:0/4047204070 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f10c402c7c0 con 0x7f10c00047c0 2023-08-09T13:53:26.077 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:26.073+0000 7f10db01a640 1 -- 172.21.15.43:0/4047204070 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f10c00047c0 msgr2=0x7f10c0004c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:53:26.077 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:26.073+0000 7f10db01a640 1 --2- 172.21.15.43:0/4047204070 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f10c00047c0 0x7f10c0004c40 secure :-1 s=READY pgs=1412 cs=0 l=1 rev1=1 crypto rx=0x7f10c400c190 tx=0x7f10c00057f0 comp rx=0 tx=0).stop 2023-08-09T13:53:26.077 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:26.073+0000 7f10da819640 1 -- 172.21.15.43:0/4047204070 reap_dead start 2023-08-09T13:53:26.077 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:26.073+0000 7f10db01a640 1 -- 172.21.15.43:0/4047204070 shutdown_connections 2023-08-09T13:53:26.077 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:26.073+0000 7f10db01a640 1 -- 172.21.15.43:0/4047204070 >> 172.21.15.43:0/4047204070 conn(0x7f10d406a210 msgr2=0x7f10d406a620 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-09T13:53:26.078 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:26.074+0000 7f10db01a640 1 -- 172.21.15.43:0/4047204070 shutdown_connections 2023-08-09T13:53:26.078 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:26.074+0000 7f10db01a640 1 -- 172.21.15.43:0/4047204070 wait complete. 2023-08-09T13:53:26.079 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:26.076+0000 7f10db01a640 1 Processor -- start 2023-08-09T13:53:26.079 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:26.076+0000 7f10db01a640 1 -- start start 2023-08-09T13:53:26.079 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:26.077+0000 7f10db01a640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f10c00047c0 0x7f10d415d160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:53:26.080 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:26.077+0000 7f10db01a640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f10d40b4180 0x7f10d41f2320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:53:26.080 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:26.077+0000 7f10db01a640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f10d415a2f0 0x7f10d414f180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:53:26.080 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:26.077+0000 7f10db01a640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f10d415e8c0 con 0x7f10d40b4180 2023-08-09T13:53:26.080 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:26.077+0000 7f10db01a640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f10d415d790 con 0x7f10c00047c0 2023-08-09T13:53:26.080 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:26.077+0000 7f10db01a640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f10d415df20 con 0x7f10d415a2f0 2023-08-09T13:53:26.080 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:26.077+0000 7f10d9817640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f10d40b4180 0x7f10d41f2320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:53:26.080 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:26.077+0000 7f10d9817640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f10d40b4180 0x7f10d41f2320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:58116/0 (socket says 172.21.15.43:58116) 2023-08-09T13:53:26.080 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:26.077+0000 7f10d9817640 1 -- 172.21.15.43:0/599101815 learned_addr learned my addr 172.21.15.43:0/599101815 (peer_addr_for_me v2:172.21.15.43:0/0) 2023-08-09T13:53:26.081 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:26.077+0000 7f10da018640 1 --2- 172.21.15.43:0/599101815 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f10c00047c0 0x7f10d415d160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:53:26.081 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:26.077+0000 7f10da819640 1 --2- 172.21.15.43:0/599101815 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f10d415a2f0 0x7f10d414f180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:53:26.081 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:26.078+0000 7f10d9817640 1 -- 172.21.15.43:0/599101815 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f10d415a2f0 msgr2=0x7f10d414f180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:53:26.081 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:26.078+0000 7f10d9817640 1 --2- 172.21.15.43:0/599101815 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f10d415a2f0 0x7f10d414f180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:53:26.081 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:26.078+0000 7f10d9817640 1 -- 172.21.15.43:0/599101815 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f10c00047c0 msgr2=0x7f10d415d160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:53:26.081 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:26.078+0000 7f10d9817640 1 --2- 172.21.15.43:0/599101815 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f10c00047c0 0x7f10d415d160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:53:26.081 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:26.078+0000 7f10d9817640 1 -- 172.21.15.43:0/599101815 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f10d415e200 con 0x7f10d40b4180 2023-08-09T13:53:26.082 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:26.078+0000 7f10d9817640 1 --2- 172.21.15.43:0/599101815 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f10d40b4180 0x7f10d41f2320 secure :-1 s=READY pgs=1413 cs=0 l=1 rev1=1 crypto rx=0x7f10c400c190 tx=0x7f10c4001b80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-09T13:53:26.082 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:26.078+0000 7f10caffd640 1 -- 172.21.15.43:0/599101815 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f10c4039070 con 0x7f10d40b4180 2023-08-09T13:53:26.082 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:26.078+0000 7f10caffd640 1 -- 172.21.15.43:0/599101815 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f10c402dd60 con 0x7f10d40b4180 2023-08-09T13:53:26.083 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:26.078+0000 7f10db01a640 1 -- 172.21.15.43:0/599101815 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f10d415e4e0 con 0x7f10d40b4180 2023-08-09T13:53:26.083 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:26.078+0000 7f10caffd640 1 -- 172.21.15.43:0/599101815 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f10c403c040 con 0x7f10d40b4180 2023-08-09T13:53:26.083 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:26.079+0000 7f10db01a640 1 -- 172.21.15.43:0/599101815 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f10d40b45e0 con 0x7f10d40b4180 2023-08-09T13:53:26.083 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:26.080+0000 7f10db01a640 1 -- 172.21.15.43:0/599101815 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f10d415ea00 con 0x7f10d40b4180 2023-08-09T13:53:26.083 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:26.081+0000 7f10caffd640 1 -- 172.21.15.43:0/599101815 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 52) v1 ==== 88557+0+0 (secure 0 0 0) 0x7f10c4032070 con 0x7f10d40b4180 2023-08-09T13:53:26.084 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:26.081+0000 7f10caffd640 1 --2- 172.21.15.43:0/599101815 >> [v2:172.21.15.203:6832/2041169533,v1:172.21.15.203:6833/2041169533] conn(0x7f10b806a2c0 0x7f10b806c780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:53:26.084 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:26.082+0000 7f10caffd640 1 -- 172.21.15.43:0/599101815 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(117..117 src has 1..117) v4 ==== 6770+0+0 (secure 0 0 0) 0x7f10c40a9d80 con 0x7f10d40b4180 2023-08-09T13:53:26.084 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:26.082+0000 7f10da018640 1 -- 172.21.15.43:0/599101815 >> [v2:172.21.15.203:6832/2041169533,v1:172.21.15.203:6833/2041169533] conn(0x7f10b806a2c0 msgr2=0x7f10b806c780 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.203:6832/2041169533 2023-08-09T13:53:26.084 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:26.082+0000 7f10da018640 1 --2- 172.21.15.43:0/599101815 >> [v2:172.21.15.203:6832/2041169533,v1:172.21.15.203:6833/2041169533] conn(0x7f10b806a2c0 0x7f10b806c780 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-08-09T13:53:26.087 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:26.086+0000 7f10caffd640 1 -- 172.21.15.43:0/599101815 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178066 (secure 0 0 0) 0x7f10d415ea00 con 0x7f10d40b4180 2023-08-09T13:53:26.115 INFO:tasks.ceph.mgr.x.smithi203.stderr:2023-08-09T13:53:26.112+0000 7f005263d040 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2023-08-09T13:53:26.196 INFO:tasks.ceph.mgr.x.smithi203.stderr:2023-08-09T13:53:26.193+0000 7f005263d040 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2023-08-09T13:53:26.205 INFO:tasks.ceph.mgr.y.smithi043.stderr:2023-08-09T13:53:26.203+0000 7fa15623e040 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2023-08-09T13:53:26.246 INFO:tasks.ceph.mgr.x.smithi203.stderr:2023-08-09T13:53:26.244+0000 7f005263d040 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2023-08-09T13:53:26.276 INFO:tasks.ceph.mgr.y.smithi043.stderr:2023-08-09T13:53:26.274+0000 7fa15623e040 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2023-08-09T13:53:26.284 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:26.283+0000 7f10da018640 1 -- 172.21.15.43:0/599101815 >> [v2:172.21.15.203:6832/2041169533,v1:172.21.15.203:6833/2041169533] conn(0x7f10b806a2c0 msgr2=0x7f10b806c780 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.203:6832/2041169533 2023-08-09T13:53:26.284 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:26.283+0000 7f10db01a640 1 -- 172.21.15.43:0/599101815 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f10d415ea00 con 0x7f10d40b4180 2023-08-09T13:53:26.285 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:26.283+0000 7f10da018640 1 --2- 172.21.15.43:0/599101815 >> [v2:172.21.15.203:6832/2041169533,v1:172.21.15.203:6833/2041169533] conn(0x7f10b806a2c0 0x7f10b806c780 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2023-08-09T13:53:26.285 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:26.284+0000 7f10caffd640 1 -- 172.21.15.43:0/599101815 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v117) v1 ==== 104+0+11 (secure 0 0 0) 0x7f10d415ea00 con 0x7f10d40b4180 2023-08-09T13:53:26.285 INFO:teuthology.orchestra.run.smithi043.stdout:pg_num: 32 2023-08-09T13:53:26.289 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:26.287+0000 7f10db01a640 1 -- 172.21.15.43:0/599101815 >> [v2:172.21.15.203:6832/2041169533,v1:172.21.15.203:6833/2041169533] conn(0x7f10b806a2c0 msgr2=0x7f10b806c780 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-08-09T13:53:26.289 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:26.287+0000 7f10db01a640 1 --2- 172.21.15.43:0/599101815 >> [v2:172.21.15.203:6832/2041169533,v1:172.21.15.203:6833/2041169533] conn(0x7f10b806a2c0 0x7f10b806c780 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:53:26.289 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:26.287+0000 7f10db01a640 1 -- 172.21.15.43:0/599101815 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f10d40b4180 msgr2=0x7f10d41f2320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:53:26.289 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:26.287+0000 7f10db01a640 1 --2- 172.21.15.43:0/599101815 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f10d40b4180 0x7f10d41f2320 secure :-1 s=READY pgs=1413 cs=0 l=1 rev1=1 crypto rx=0x7f10c400c190 tx=0x7f10c4001b80 comp rx=0 tx=0).stop 2023-08-09T13:53:26.290 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:26.288+0000 7f10db01a640 1 -- 172.21.15.43:0/599101815 shutdown_connections 2023-08-09T13:53:26.290 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:26.288+0000 7f10db01a640 1 --2- 172.21.15.43:0/599101815 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f10d415a2f0 0x7f10d414f180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:53:26.290 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:26.288+0000 7f10db01a640 1 --2- 172.21.15.43:0/599101815 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f10d40b4180 0x7f10d41f2320 unknown :-1 s=CLOSED pgs=1413 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:53:26.290 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:26.288+0000 7f10db01a640 1 --2- 172.21.15.43:0/599101815 >> [v2:172.21.15.203:6832/2041169533,v1:172.21.15.203:6833/2041169533] conn(0x7f10b806a2c0 0x7f10b806c780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:53:26.290 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:26.288+0000 7f10db01a640 1 --2- 172.21.15.43:0/599101815 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f10c00047c0 0x7f10d415d160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:53:26.290 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:26.288+0000 7f10db01a640 1 -- 172.21.15.43:0/599101815 >> 172.21.15.43:0/599101815 conn(0x7f10d406a210 msgr2=0x7f10d40b70a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-09T13:53:26.291 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:26.288+0000 7f10db01a640 1 -- 172.21.15.43:0/599101815 shutdown_connections 2023-08-09T13:53:26.291 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:26.288+0000 7f10db01a640 1 -- 172.21.15.43:0/599101815 wait complete. 2023-08-09T13:53:26.310 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-08-09T13:53:26.335 INFO:tasks.ceph.mgr.y.smithi043.stderr:2023-08-09T13:53:26.333+0000 7fa15623e040 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2023-08-09T13:53:26.363 INFO:tasks.ceph.mgr.x.smithi203.stderr:2023-08-09T13:53:26.360+0000 7f005263d040 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2023-08-09T13:53:26.461 INFO:tasks.ceph.mgr.y.smithi043.stderr:2023-08-09T13:53:26.459+0000 7fa15623e040 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2023-08-09T13:53:26.519 INFO:tasks.ceph.mgr.x.smithi203.stderr:2023-08-09T13:53:26.516+0000 7f005263d040 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2023-08-09T13:53:26.532 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:26.530+0000 7fa919386640 1 Processor -- start 2023-08-09T13:53:26.532 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:26.530+0000 7fa919386640 1 -- start start 2023-08-09T13:53:26.533 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:26.532+0000 7fa919386640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7fa91415c550 0x7fa9140cd6d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:53:26.534 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:26.532+0000 7fa919386640 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7fa9140b45f0 0x7fa9140b59d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:53:26.534 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:26.532+0000 7fa919386640 1 -- --> v1:172.21.15.203:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa9140b7bd0 con 0x7fa9140d04b0 2023-08-09T13:53:26.534 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:26.532+0000 7fa919386640 1 -- --> v2:172.21.15.43:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa9140d1c00 con 0x7fa91415c550 2023-08-09T13:53:26.534 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:26.532+0000 7fa919386640 1 -- --> v2:172.21.15.203:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa91415c930 con 0x7fa9140b45f0 2023-08-09T13:53:26.535 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:26.532+0000 7fa912ffd640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7fa91415c550 0x7fa9140cd6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:53:26.535 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:26.532+0000 7fa912ffd640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7fa91415c550 0x7fa9140cd6d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:58126/0 (socket says 172.21.15.43:58126) 2023-08-09T13:53:26.535 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:26.532+0000 7fa912ffd640 1 -- 172.21.15.43:0/2306379188 learned_addr learned my addr 172.21.15.43:0/2306379188 (peer_addr_for_me v2:172.21.15.43:0/0) 2023-08-09T13:53:26.535 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:26.533+0000 7fa9137fe640 1 --2- 172.21.15.43:0/2306379188 >> v2:172.21.15.203:3300/0 conn(0x7fa9140b45f0 0x7fa9140b59d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:53:26.536 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:26.533+0000 7fa912ffd640 1 -- 172.21.15.43:0/2306379188 >> v2:172.21.15.203:3300/0 conn(0x7fa9140b45f0 msgr2=0x7fa9140b59d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-09T13:53:26.536 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:26.533+0000 7fa912ffd640 1 --2- 172.21.15.43:0/2306379188 >> v2:172.21.15.203:3300/0 conn(0x7fa9140b45f0 0x7fa9140b59d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:53:26.536 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:26.533+0000 7fa912ffd640 1 -- 172.21.15.43:0/2306379188 >> v1:172.21.15.203:6789/0 conn(0x7fa9140d04b0 legacy=0x7fa9140b3ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-09T13:53:26.536 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:26.533+0000 7fa912ffd640 1 -- 172.21.15.43:0/2306379188 --> v2:172.21.15.43:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa9140b54f0 con 0x7fa91415c550 2023-08-09T13:53:26.537 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:26.534+0000 7fa912ffd640 1 --2- 172.21.15.43:0/2306379188 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa91415c550 0x7fa9140cd6d0 secure :-1 s=READY pgs=1414 cs=0 l=1 rev1=1 crypto rx=0x7fa9140d01c0 tx=0x7fa90402b870 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6d3ccbfe2ff7a9b7 server_cookie=0 in_seq=0 out_seq=0 2023-08-09T13:53:26.537 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:26.534+0000 7fa9127fc640 1 -- 172.21.15.43:0/2306379188 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa9040317d0 con 0x7fa91415c550 2023-08-09T13:53:26.537 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:26.534+0000 7fa9127fc640 1 -- 172.21.15.43:0/2306379188 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa904031930 con 0x7fa91415c550 2023-08-09T13:53:26.537 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:26.534+0000 7fa9127fc640 1 -- 172.21.15.43:0/2306379188 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa904031cc0 con 0x7fa91415c550 2023-08-09T13:53:26.537 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:26.535+0000 7fa919386640 1 -- 172.21.15.43:0/2306379188 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa91415c550 msgr2=0x7fa9140cd6d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:53:26.538 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:26.535+0000 7fa919386640 1 --2- 172.21.15.43:0/2306379188 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa91415c550 0x7fa9140cd6d0 secure :-1 s=READY pgs=1414 cs=0 l=1 rev1=1 crypto rx=0x7fa9140d01c0 tx=0x7fa90402b870 comp rx=0 tx=0).stop 2023-08-09T13:53:26.538 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:26.535+0000 7fa919386640 1 -- 172.21.15.43:0/2306379188 shutdown_connections 2023-08-09T13:53:26.538 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:26.535+0000 7fa919386640 1 --2- 172.21.15.43:0/2306379188 >> v2:172.21.15.203:3300/0 conn(0x7fa9140b45f0 0x7fa9140b59d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:53:26.538 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:26.535+0000 7fa919386640 1 --2- 172.21.15.43:0/2306379188 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa91415c550 0x7fa9140cd6d0 unknown :-1 s=CLOSED pgs=1414 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:53:26.538 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:26.535+0000 7fa919386640 1 -- 172.21.15.43:0/2306379188 >> 172.21.15.43:0/2306379188 conn(0x7fa914071520 msgr2=0x7fa9140ceee0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-09T13:53:26.538 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:26.535+0000 7fa919386640 1 -- 172.21.15.43:0/2306379188 shutdown_connections 2023-08-09T13:53:26.538 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:26.535+0000 7fa919386640 1 -- 172.21.15.43:0/2306379188 wait complete. 2023-08-09T13:53:26.540 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:26.537+0000 7fa919386640 1 Processor -- start 2023-08-09T13:53:26.540 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:26.537+0000 7fa919386640 1 -- start start 2023-08-09T13:53:26.540 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:26.538+0000 7fa919386640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fa9140b45f0 0x7fa9140b3d70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:53:26.540 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:26.538+0000 7fa919386640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa9140d04b0 0x7fa9140ab9b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:53:26.540 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:26.538+0000 7fa919386640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fa91415c550 0x7fa9140abef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:53:26.540 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:26.538+0000 7fa919386640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa91414b720 con 0x7fa9140d04b0 2023-08-09T13:53:26.540 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:26.538+0000 7fa919386640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa91414c0c0 con 0x7fa9140b45f0 2023-08-09T13:53:26.541 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:26.538+0000 7fa919386640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa9140c7e10 con 0x7fa91415c550 2023-08-09T13:53:26.541 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:26.538+0000 7fa90bfff640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa9140d04b0 0x7fa9140ab9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:53:26.541 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:26.538+0000 7fa90bfff640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa9140d04b0 0x7fa9140ab9b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:58138/0 (socket says 172.21.15.43:58138) 2023-08-09T13:53:26.541 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:26.538+0000 7fa90bfff640 1 -- 172.21.15.43:0/1877297146 learned_addr learned my addr 172.21.15.43:0/1877297146 (peer_addr_for_me v2:172.21.15.43:0/0) 2023-08-09T13:53:26.542 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:26.539+0000 7fa912ffd640 1 --2- 172.21.15.43:0/1877297146 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fa9140b45f0 0x7fa9140b3d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:53:26.542 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:26.539+0000 7fa9137fe640 1 --2- 172.21.15.43:0/1877297146 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fa91415c550 0x7fa9140abef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:53:26.542 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:26.539+0000 7fa90bfff640 1 -- 172.21.15.43:0/1877297146 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fa91415c550 msgr2=0x7fa9140abef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:53:26.542 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:26.539+0000 7fa90bfff640 1 --2- 172.21.15.43:0/1877297146 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fa91415c550 0x7fa9140abef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:53:26.543 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:26.539+0000 7fa90bfff640 1 -- 172.21.15.43:0/1877297146 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fa9140b45f0 msgr2=0x7fa9140b3d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:53:26.543 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:26.539+0000 7fa90bfff640 1 --2- 172.21.15.43:0/1877297146 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fa9140b45f0 0x7fa9140b3d70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:53:26.543 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:26.539+0000 7fa90bfff640 1 -- 172.21.15.43:0/1877297146 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa9040094d0 con 0x7fa9140d04b0 2023-08-09T13:53:26.543 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:26.539+0000 7fa90bfff640 1 --2- 172.21.15.43:0/1877297146 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa9140d04b0 0x7fa9140ab9b0 secure :-1 s=READY pgs=1415 cs=0 l=1 rev1=1 crypto rx=0x7fa8fc00c440 tx=0x7fa8fc00c900 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-09T13:53:26.543 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:26.539+0000 7fa90b7fe640 1 -- 172.21.15.43:0/1877297146 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa8fc005480 con 0x7fa9140d04b0 2023-08-09T13:53:26.543 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:26.540+0000 7fa90b7fe640 1 -- 172.21.15.43:0/1877297146 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa8fc0035f0 con 0x7fa9140d04b0 2023-08-09T13:53:26.543 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:26.540+0000 7fa919386640 1 -- 172.21.15.43:0/1877297146 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa9140c7b30 con 0x7fa9140d04b0 2023-08-09T13:53:26.543 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:26.540+0000 7fa90b7fe640 1 -- 172.21.15.43:0/1877297146 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa8fc0039c0 con 0x7fa9140d04b0 2023-08-09T13:53:26.544 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:26.540+0000 7fa919386640 1 -- 172.21.15.43:0/1877297146 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa91414bd50 con 0x7fa9140d04b0 2023-08-09T13:53:26.544 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:26.541+0000 7fa919386640 1 -- 172.21.15.43:0/1877297146 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa9140739f0 con 0x7fa9140d04b0 2023-08-09T13:53:26.545 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:26.542+0000 7fa90b7fe640 1 -- 172.21.15.43:0/1877297146 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 52) v1 ==== 88557+0+0 (secure 0 0 0) 0x7fa8fc003b20 con 0x7fa9140d04b0 2023-08-09T13:53:26.545 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:26.543+0000 7fa90b7fe640 1 --2- 172.21.15.43:0/1877297146 >> [v2:172.21.15.203:6832/2041169533,v1:172.21.15.203:6833/2041169533] conn(0x7fa8f406a290 0x7fa8f406c750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:53:26.545 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:26.543+0000 7fa90b7fe640 1 -- 172.21.15.43:0/1877297146 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(117..117 src has 1..117) v4 ==== 6770+0+0 (secure 0 0 0) 0x7fa8fc088380 con 0x7fa9140d04b0 2023-08-09T13:53:26.545 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:26.543+0000 7fa912ffd640 1 -- 172.21.15.43:0/1877297146 >> [v2:172.21.15.203:6832/2041169533,v1:172.21.15.203:6833/2041169533] conn(0x7fa8f406a290 msgr2=0x7fa8f406c750 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.203:6832/2041169533 2023-08-09T13:53:26.545 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:26.543+0000 7fa912ffd640 1 --2- 172.21.15.43:0/1877297146 >> [v2:172.21.15.203:6832/2041169533,v1:172.21.15.203:6833/2041169533] conn(0x7fa8f406a290 0x7fa8f406c750 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-08-09T13:53:26.548 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:26.547+0000 7fa90b7fe640 1 -- 172.21.15.43:0/1877297146 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178066 (secure 0 0 0) 0x7fa8fc056da0 con 0x7fa9140d04b0 2023-08-09T13:53:26.627 INFO:tasks.ceph.mgr.x.smithi203.stderr:2023-08-09T13:53:26.624+0000 7f005263d040 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2023-08-09T13:53:26.644 INFO:tasks.ceph.mgr.y.smithi043.stderr:2023-08-09T13:53:26.639+0000 7fa15623e040 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2023-08-09T13:53:26.676 INFO:tasks.ceph.mgr.x.smithi203.stderr:2023-08-09T13:53:26.673+0000 7f005263d040 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2023-08-09T13:53:26.745 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:26.744+0000 7fa912ffd640 1 -- 172.21.15.43:0/1877297146 >> [v2:172.21.15.203:6832/2041169533,v1:172.21.15.203:6833/2041169533] conn(0x7fa8f406a290 msgr2=0x7fa8f406c750 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.203:6832/2041169533 2023-08-09T13:53:26.746 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:26.744+0000 7fa912ffd640 1 --2- 172.21.15.43:0/1877297146 >> [v2:172.21.15.203:6832/2041169533,v1:172.21.15.203:6833/2041169533] conn(0x7fa8f406a290 0x7fa8f406c750 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2023-08-09T13:53:26.753 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:26.751+0000 7fa919386640 1 -- 172.21.15.43:0/1877297146 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7fa9140ccfb0 con 0x7fa9140d04b0 2023-08-09T13:53:26.753 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:26.752+0000 7fa90b7fe640 1 -- 172.21.15.43:0/1877297146 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v117) v1 ==== 100+0+11 (secure 0 0 0) 0x7fa8fc017020 con 0x7fa9140d04b0 2023-08-09T13:53:26.754 INFO:teuthology.orchestra.run.smithi043.stdout:pg_num: 64 2023-08-09T13:53:26.755 INFO:tasks.ceph.mgr.y.smithi043.stderr:2023-08-09T13:53:26.754+0000 7fa15623e040 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2023-08-09T13:53:26.756 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:26.754+0000 7fa919386640 1 -- 172.21.15.43:0/1877297146 >> [v2:172.21.15.203:6832/2041169533,v1:172.21.15.203:6833/2041169533] conn(0x7fa8f406a290 msgr2=0x7fa8f406c750 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-08-09T13:53:26.756 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:26.754+0000 7fa919386640 1 --2- 172.21.15.43:0/1877297146 >> [v2:172.21.15.203:6832/2041169533,v1:172.21.15.203:6833/2041169533] conn(0x7fa8f406a290 0x7fa8f406c750 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:53:26.756 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:26.754+0000 7fa919386640 1 -- 172.21.15.43:0/1877297146 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa9140d04b0 msgr2=0x7fa9140ab9b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:53:26.757 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:26.754+0000 7fa919386640 1 --2- 172.21.15.43:0/1877297146 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa9140d04b0 0x7fa9140ab9b0 secure :-1 s=READY pgs=1415 cs=0 l=1 rev1=1 crypto rx=0x7fa8fc00c440 tx=0x7fa8fc00c900 comp rx=0 tx=0).stop 2023-08-09T13:53:26.757 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:26.755+0000 7fa919386640 1 -- 172.21.15.43:0/1877297146 shutdown_connections 2023-08-09T13:53:26.757 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:26.755+0000 7fa919386640 1 --2- 172.21.15.43:0/1877297146 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fa91415c550 0x7fa9140abef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:53:26.757 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:26.755+0000 7fa919386640 1 --2- 172.21.15.43:0/1877297146 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa9140d04b0 0x7fa9140ab9b0 unknown :-1 s=CLOSED pgs=1415 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:53:26.757 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:26.755+0000 7fa919386640 1 --2- 172.21.15.43:0/1877297146 >> [v2:172.21.15.203:6832/2041169533,v1:172.21.15.203:6833/2041169533] conn(0x7fa8f406a290 0x7fa8f406c750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:53:26.757 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:26.755+0000 7fa919386640 1 --2- 172.21.15.43:0/1877297146 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fa9140b45f0 0x7fa9140b3d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:53:26.757 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:26.755+0000 7fa919386640 1 -- 172.21.15.43:0/1877297146 >> 172.21.15.43:0/1877297146 conn(0x7fa914071520 msgr2=0x7fa9140d2440 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-09T13:53:26.757 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:26.755+0000 7fa919386640 1 -- 172.21.15.43:0/1877297146 shutdown_connections 2023-08-09T13:53:26.758 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:26.755+0000 7fa919386640 1 -- 172.21.15.43:0/1877297146 wait complete. 2023-08-09T13:53:26.773 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2023-08-09T13:53:26.808 INFO:tasks.ceph.mgr.y.smithi043.stderr:2023-08-09T13:53:26.806+0000 7fa15623e040 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2023-08-09T13:53:26.833 INFO:tasks.ceph.mgr.x.smithi203.stderr:2023-08-09T13:53:26.831+0000 7f005263d040 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2023-08-09T13:53:26.979 INFO:tasks.ceph.mgr.y.smithi043.stderr:2023-08-09T13:53:26.977+0000 7fa15623e040 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2023-08-09T13:53:27.037 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:27.035+0000 7fe033238640 1 Processor -- start 2023-08-09T13:53:27.037 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:27.036+0000 7fe033238640 1 -- start start 2023-08-09T13:53:27.038 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:27.037+0000 7fe033238640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fe02c0cd510 0x7fe02c0c96e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:53:27.039 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:27.037+0000 7fe033238640 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7fe02c15ce40 0x7fe02c0c3290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:53:27.039 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:27.037+0000 7fe033238640 1 -- --> v1:172.21.15.203:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe02c143080 con 0x7fe02c142ca0 2023-08-09T13:53:27.039 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:27.037+0000 7fe033238640 1 -- --> v2:172.21.15.203:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe02c06d400 con 0x7fe02c15ce40 2023-08-09T13:53:27.039 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:27.037+0000 7fe033238640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe02c0c9d90 con 0x7fe02c0cd510 2023-08-09T13:53:27.040 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:27.038+0000 7fe0317d5640 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7fe02c15ce40 0x7fe02c0c3290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:53:27.040 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:27.038+0000 7fe023fff640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fe02c0cd510 0x7fe02c0c96e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:53:27.041 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:27.038+0000 7fe0317d5640 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7fe02c15ce40 0x7fe02c0c3290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.203:3300/0 says I am v2:172.21.15.43:38386/0 (socket says 172.21.15.43:38386) 2023-08-09T13:53:27.041 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:27.038+0000 7fe0317d5640 1 -- 172.21.15.43:0/3846578336 learned_addr learned my addr 172.21.15.43:0/3846578336 (peer_addr_for_me v2:172.21.15.43:0/0) 2023-08-09T13:53:27.041 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:27.038+0000 7fe0317d5640 1 -- 172.21.15.43:0/3846578336 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fe02c0cd510 msgr2=0x7fe02c0c96e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-09T13:53:27.041 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:27.038+0000 7fe0317d5640 1 --2- 172.21.15.43:0/3846578336 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fe02c0cd510 0x7fe02c0c96e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:53:27.041 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:27.038+0000 7fe0317d5640 1 -- 172.21.15.43:0/3846578336 >> v1:172.21.15.203:6789/0 conn(0x7fe02c142ca0 legacy=0x7fe02c0cce00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:53:27.041 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:27.038+0000 7fe0317d5640 1 -- 172.21.15.43:0/3846578336 --> v2:172.21.15.203:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe02c0ca070 con 0x7fe02c15ce40 2023-08-09T13:53:27.042 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:27.039+0000 7fe0317d5640 1 --2- 172.21.15.43:0/3846578336 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fe02c15ce40 0x7fe02c0c3290 secure :-1 s=READY pgs=528 cs=0 l=1 rev1=1 crypto rx=0x7fe02400d150 tx=0x7fe02402e160 comp rx=0 tx=0).ready entity=mon.1 client_cookie=1fc842f8aa29293b server_cookie=0 in_seq=0 out_seq=0 2023-08-09T13:53:27.043 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:27.040+0000 7fe0237fe640 1 -- 172.21.15.43:0/3846578336 <== mon.1 v2:172.21.15.203:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe02400b0d0 con 0x7fe02c15ce40 2023-08-09T13:53:27.043 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:27.040+0000 7fe0237fe640 1 -- 172.21.15.43:0/3846578336 <== mon.1 v2:172.21.15.203:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe02400b230 con 0x7fe02c15ce40 2023-08-09T13:53:27.043 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:27.040+0000 7fe0237fe640 1 -- 172.21.15.43:0/3846578336 <== mon.1 v2:172.21.15.203:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe024009600 con 0x7fe02c15ce40 2023-08-09T13:53:27.043 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:27.040+0000 7fe033238640 1 -- 172.21.15.43:0/3846578336 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fe02c15ce40 msgr2=0x7fe02c0c3290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:53:27.043 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:27.040+0000 7fe033238640 1 --2- 172.21.15.43:0/3846578336 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fe02c15ce40 0x7fe02c0c3290 secure :-1 s=READY pgs=528 cs=0 l=1 rev1=1 crypto rx=0x7fe02400d150 tx=0x7fe02402e160 comp rx=0 tx=0).stop 2023-08-09T13:53:27.043 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:27.040+0000 7fe033238640 1 -- 172.21.15.43:0/3846578336 shutdown_connections 2023-08-09T13:53:27.044 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:27.040+0000 7fe033238640 1 --2- 172.21.15.43:0/3846578336 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fe02c15ce40 0x7fe02c0c3290 unknown :-1 s=CLOSED pgs=528 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:53:27.044 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:27.041+0000 7fe033238640 1 --2- 172.21.15.43:0/3846578336 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fe02c0cd510 0x7fe02c0c96e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:53:27.044 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:27.041+0000 7fe033238640 1 -- 172.21.15.43:0/3846578336 >> 172.21.15.43:0/3846578336 conn(0x7fe02c0ce0a0 msgr2=0x7fe02c0ce480 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-09T13:53:27.045 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:27.041+0000 7fe033238640 1 -- 172.21.15.43:0/3846578336 shutdown_connections 2023-08-09T13:53:27.045 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:27.041+0000 7fe033238640 1 -- 172.21.15.43:0/3846578336 wait complete. 2023-08-09T13:53:27.048 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:27.042+0000 7fe033238640 1 Processor -- start 2023-08-09T13:53:27.048 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:27.042+0000 7fe033238640 1 -- start start 2023-08-09T13:53:27.049 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:27.043+0000 7fe033238640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fe02c0cd510 0x7fe02c13ac70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:53:27.049 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:27.043+0000 7fe033238640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fe02c142ca0 0x7fe02c070bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:53:27.049 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:27.044+0000 7fe033238640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fe02c15ce40 0x7fe02c0c5170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:53:27.050 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:27.044+0000 7fe033238640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe02c13a850 con 0x7fe02c142ca0 2023-08-09T13:53:27.050 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:27.044+0000 7fe033238640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe02c13b1b0 con 0x7fe02c15ce40 2023-08-09T13:53:27.050 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:27.044+0000 7fe033238640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe02c0c6140 con 0x7fe02c0cd510 2023-08-09T13:53:27.051 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:27.044+0000 7fe023fff640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fe02c142ca0 0x7fe02c070bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:53:27.051 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:27.044+0000 7fe023fff640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fe02c142ca0 0x7fe02c070bd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:58154/0 (socket says 172.21.15.43:58154) 2023-08-09T13:53:27.051 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:27.044+0000 7fe023fff640 1 -- 172.21.15.43:0/2065701279 learned_addr learned my addr 172.21.15.43:0/2065701279 (peer_addr_for_me v2:172.21.15.43:0/0) 2023-08-09T13:53:27.051 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:27.044+0000 7fe0317d5640 1 --2- 172.21.15.43:0/2065701279 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fe02c15ce40 0x7fe02c0c5170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:53:27.052 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:27.044+0000 7fe030fd4640 1 --2- 172.21.15.43:0/2065701279 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fe02c0cd510 0x7fe02c13ac70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:53:27.052 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:27.044+0000 7fe023fff640 1 -- 172.21.15.43:0/2065701279 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fe02c0cd510 msgr2=0x7fe02c13ac70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:53:27.052 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:27.044+0000 7fe023fff640 1 --2- 172.21.15.43:0/2065701279 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fe02c0cd510 0x7fe02c13ac70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:53:27.053 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:27.044+0000 7fe023fff640 1 -- 172.21.15.43:0/2065701279 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fe02c15ce40 msgr2=0x7fe02c0c5170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:53:27.053 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:27.044+0000 7fe023fff640 1 --2- 172.21.15.43:0/2065701279 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fe02c15ce40 0x7fe02c0c5170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:53:27.053 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:27.044+0000 7fe023fff640 1 -- 172.21.15.43:0/2065701279 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe02400bd40 con 0x7fe02c142ca0 2023-08-09T13:53:27.053 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:27.045+0000 7fe023fff640 1 --2- 172.21.15.43:0/2065701279 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fe02c142ca0 0x7fe02c070bd0 secure :-1 s=READY pgs=1416 cs=0 l=1 rev1=1 crypto rx=0x7fe01c00e9b0 tx=0x7fe01c00ee70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-09T13:53:27.053 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:27.045+0000 7fe0217fa640 1 -- 172.21.15.43:0/2065701279 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe01c009670 con 0x7fe02c142ca0 2023-08-09T13:53:27.054 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:27.045+0000 7fe0217fa640 1 -- 172.21.15.43:0/2065701279 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe01c00c9a0 con 0x7fe02c142ca0 2023-08-09T13:53:27.054 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:27.045+0000 7fe0217fa640 1 -- 172.21.15.43:0/2065701279 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe01c00cd50 con 0x7fe02c142ca0 2023-08-09T13:53:27.054 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:27.045+0000 7fe033238640 1 -- 172.21.15.43:0/2065701279 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe02c0c5e60 con 0x7fe02c142ca0 2023-08-09T13:53:27.054 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:27.045+0000 7fe033238640 1 -- 172.21.15.43:0/2065701279 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe02c13a470 con 0x7fe02c142ca0 2023-08-09T13:53:27.055 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:27.046+0000 7fe033238640 1 -- 172.21.15.43:0/2065701279 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe02c0739f0 con 0x7fe02c142ca0 2023-08-09T13:53:27.056 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:27.047+0000 7fe0217fa640 1 -- 172.21.15.43:0/2065701279 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 52) v1 ==== 88557+0+0 (secure 0 0 0) 0x7fe01c013470 con 0x7fe02c142ca0 2023-08-09T13:53:27.056 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:27.048+0000 7fe0217fa640 1 --2- 172.21.15.43:0/2065701279 >> [v2:172.21.15.203:6832/2041169533,v1:172.21.15.203:6833/2041169533] conn(0x7fe01806a290 0x7fe01806c750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:53:27.056 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:27.048+0000 7fe0217fa640 1 -- 172.21.15.43:0/2065701279 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(117..117 src has 1..117) v4 ==== 6770+0+0 (secure 0 0 0) 0x7fe01c088ee0 con 0x7fe02c142ca0 2023-08-09T13:53:27.056 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:27.049+0000 7fe030fd4640 1 -- 172.21.15.43:0/2065701279 >> [v2:172.21.15.203:6832/2041169533,v1:172.21.15.203:6833/2041169533] conn(0x7fe01806a290 msgr2=0x7fe01806c750 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.203:6832/2041169533 2023-08-09T13:53:27.057 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:27.049+0000 7fe030fd4640 1 --2- 172.21.15.43:0/2065701279 >> [v2:172.21.15.203:6832/2041169533,v1:172.21.15.203:6833/2041169533] conn(0x7fe01806a290 0x7fe01806c750 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-08-09T13:53:27.058 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:27.056+0000 7fe0217fa640 1 -- 172.21.15.43:0/2065701279 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178066 (secure 0 0 0) 0x7fe01c054040 con 0x7fe02c142ca0 2023-08-09T13:53:27.219 INFO:tasks.ceph.mgr.x.smithi203.stderr:2023-08-09T13:53:27.216+0000 7f005263d040 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2023-08-09T13:53:27.251 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:27.250+0000 7fe030fd4640 1 -- 172.21.15.43:0/2065701279 >> [v2:172.21.15.203:6832/2041169533,v1:172.21.15.203:6833/2041169533] conn(0x7fe01806a290 msgr2=0x7fe01806c750 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.203:6832/2041169533 2023-08-09T13:53:27.252 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:27.250+0000 7fe030fd4640 1 --2- 172.21.15.43:0/2065701279 >> [v2:172.21.15.203:6832/2041169533,v1:172.21.15.203:6833/2041169533] conn(0x7fe01806a290 0x7fe01806c750 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2023-08-09T13:53:27.265 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:27.264+0000 7fe033238640 1 -- 172.21.15.43:0/2065701279 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7fe02c06df60 con 0x7fe02c142ca0 2023-08-09T13:53:27.266 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:27.265+0000 7fe0217fa640 1 -- 172.21.15.43:0/2065701279 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 8 entries v117) v1 ==== 80+0+477 (secure 0 0 0) 0x7fe01c0211f0 con 0x7fe02c142ca0 2023-08-09T13:53:27.266 INFO:teuthology.orchestra.run.smithi043.stdout:172.21.15.203:0/1488154102 2023-08-10T13:52:20.374695+0000 2023-08-09T13:53:27.267 INFO:teuthology.orchestra.run.smithi043.stdout:172.21.15.203:0/3595044319 2023-08-10T13:52:20.374695+0000 2023-08-09T13:53:27.267 INFO:teuthology.orchestra.run.smithi043.stdout:172.21.15.203:0/1657891023 2023-08-10T13:52:20.374695+0000 2023-08-09T13:53:27.267 INFO:teuthology.orchestra.run.smithi043.stdout:172.21.15.203:0/2368610482 2023-08-10T13:52:20.374695+0000 2023-08-09T13:53:27.267 INFO:teuthology.orchestra.run.smithi043.stdout:172.21.15.203:0/437192587 2023-08-10T13:52:20.374695+0000 2023-08-09T13:53:27.267 INFO:teuthology.orchestra.run.smithi043.stdout:172.21.15.203:6833/1317108840 2023-08-10T13:52:20.374695+0000 2023-08-09T13:53:27.267 INFO:teuthology.orchestra.run.smithi043.stdout:172.21.15.203:6832/1317108840 2023-08-10T13:52:20.374695+0000 2023-08-09T13:53:27.267 INFO:teuthology.orchestra.run.smithi043.stdout:172.21.15.203:0/1459918882 2023-08-10T13:52:20.374695+0000 2023-08-09T13:53:27.268 INFO:teuthology.orchestra.run.smithi043.stderr:listed 8 entries 2023-08-09T13:53:27.269 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:27.267+0000 7fe033238640 1 -- 172.21.15.43:0/2065701279 >> [v2:172.21.15.203:6832/2041169533,v1:172.21.15.203:6833/2041169533] conn(0x7fe01806a290 msgr2=0x7fe01806c750 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-08-09T13:53:27.269 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:27.267+0000 7fe033238640 1 --2- 172.21.15.43:0/2065701279 >> [v2:172.21.15.203:6832/2041169533,v1:172.21.15.203:6833/2041169533] conn(0x7fe01806a290 0x7fe01806c750 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:53:27.269 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:27.267+0000 7fe033238640 1 -- 172.21.15.43:0/2065701279 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fe02c142ca0 msgr2=0x7fe02c070bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:53:27.270 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:27.268+0000 7fe033238640 1 --2- 172.21.15.43:0/2065701279 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fe02c142ca0 0x7fe02c070bd0 secure :-1 s=READY pgs=1416 cs=0 l=1 rev1=1 crypto rx=0x7fe01c00e9b0 tx=0x7fe01c00ee70 comp rx=0 tx=0).stop 2023-08-09T13:53:27.270 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:27.268+0000 7fe033238640 1 -- 172.21.15.43:0/2065701279 shutdown_connections 2023-08-09T13:53:27.270 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:27.268+0000 7fe033238640 1 --2- 172.21.15.43:0/2065701279 >> [v2:172.21.15.203:6832/2041169533,v1:172.21.15.203:6833/2041169533] conn(0x7fe01806a290 0x7fe01806c750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:53:27.270 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:27.268+0000 7fe033238640 1 --2- 172.21.15.43:0/2065701279 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fe02c15ce40 0x7fe02c0c5170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:53:27.270 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:27.268+0000 7fe033238640 1 --2- 172.21.15.43:0/2065701279 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fe02c142ca0 0x7fe02c070bd0 unknown :-1 s=CLOSED pgs=1416 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:53:27.270 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:27.268+0000 7fe033238640 1 --2- 172.21.15.43:0/2065701279 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fe02c0cd510 0x7fe02c13ac70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:53:27.270 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:27.268+0000 7fe033238640 1 -- 172.21.15.43:0/2065701279 >> 172.21.15.43:0/2065701279 conn(0x7fe02c0ce0a0 msgr2=0x7fe02c0c8450 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-09T13:53:27.271 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:27.268+0000 7fe033238640 1 -- 172.21.15.43:0/2065701279 shutdown_connections 2023-08-09T13:53:27.271 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:27.268+0000 7fe033238640 1 -- 172.21.15.43:0/2065701279 wait complete. 2023-08-09T13:53:27.287 INFO:tasks.cephfs.fuse_mount:Running fusermount -u on ubuntu@smithi043.front.sepia.ceph.com... 2023-08-09T13:53:27.287 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-09T13:53:27.288 DEBUG:teuthology.orchestra.run.smithi043:> sudo fusermount -u /home/ubuntu/cephtest/mnt.0 2023-08-09T13:53:27.364 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi043.stderr:ceph-fuse[125126]: fuse finished with error 0 and tester_r 0 2023-08-09T13:53:27.367 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi043.front.sepia.ceph.com 2023-08-09T13:53:27.368 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-09T13:53:27.368 DEBUG:teuthology.orchestra.run.smithi043:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2023-08-09T13:53:27.403 INFO:tasks.ceph.mgr.y.smithi043.stderr:2023-08-09T13:53:27.402+0000 7fa15623e040 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2023-08-09T13:53:27.441 DEBUG:teuthology.orchestra.run.smithi203:> set -ex 2023-08-09T13:53:27.441 DEBUG:teuthology.orchestra.run.smithi203:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-09T13:53:27.450 INFO:tasks.ceph.mgr.x.smithi203.stderr:2023-08-09T13:53:27.441+0000 7f005263d040 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2023-08-09T13:53:27.495 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4053648,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497576k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=27,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248784k,nr_inodes=812196,mode=700,uid=1000,gid=1275 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497576k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2023-08-09T13:53:27.495 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi203.front.sepia.ceph.com 2023-08-09T13:53:27.496 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-09T13:53:27.496 DEBUG:teuthology.orchestra.run.smithi203:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.1) 2023-08-09T13:53:27.503 INFO:tasks.ceph.mgr.x.smithi203.stderr:2023-08-09T13:53:27.501+0000 7f005263d040 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2023-08-09T13:53:27.545 INFO:teuthology.orchestra.run.smithi203.stderr:rmdir: failed to remove '/home/ubuntu/cephtest/mnt.1': No such file or directory 2023-08-09T13:53:27.547 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-08-09T13:53:27.548 DEBUG:teuthology.orchestra.run.smithi043:> set -ex 2023-08-09T13:53:27.549 DEBUG:teuthology.orchestra.run.smithi043:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-09T13:53:27.557 INFO:tasks.ceph.mgr.x.smithi203.stderr:2023-08-09T13:53:27.555+0000 7f005263d040 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2023-08-09T13:53:27.605 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4054441,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6498844k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=27,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3249420k,nr_inodes=812355,mode=700,uid=1000,gid=1275 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6498844k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2023-08-09T13:53:27.606 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi043.front.sepia.ceph.com 2023-08-09T13:53:27.606 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-09T13:53:27.607 DEBUG:teuthology.orchestra.run.smithi043:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.2) 2023-08-09T13:53:27.609 INFO:tasks.ceph.mgr.x.smithi203.stderr:2023-08-09T13:53:27.607+0000 7f005263d040 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2023-08-09T13:53:27.642 INFO:tasks.ceph.mgr.y.smithi043.stderr:2023-08-09T13:53:27.640+0000 7fa15623e040 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2023-08-09T13:53:27.660 INFO:teuthology.orchestra.run.smithi043.stderr:rmdir: failed to remove '/home/ubuntu/cephtest/mnt.2': No such file or directory 2023-08-09T13:53:27.661 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-08-09T13:53:27.662 DEBUG:teuthology.orchestra.run.smithi203:> set -ex 2023-08-09T13:53:27.663 DEBUG:teuthology.orchestra.run.smithi203:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-09T13:53:27.702 INFO:tasks.ceph.mgr.y.smithi043.stderr:2023-08-09T13:53:27.701+0000 7fa15623e040 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2023-08-09T13:53:27.717 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4053648,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497576k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=27,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248784k,nr_inodes=812196,mode=700,uid=1000,gid=1275 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497576k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2023-08-09T13:53:27.718 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi203.front.sepia.ceph.com 2023-08-09T13:53:27.718 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-09T13:53:27.719 DEBUG:teuthology.orchestra.run.smithi203:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.3) 2023-08-09T13:53:27.755 INFO:tasks.ceph.mgr.y.smithi043.stderr:2023-08-09T13:53:27.753+0000 7fa15623e040 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2023-08-09T13:53:27.766 INFO:teuthology.orchestra.run.smithi203.stderr:rmdir: failed to remove '/home/ubuntu/cephtest/mnt.3': No such file or directory 2023-08-09T13:53:27.767 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-08-09T13:53:27.768 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-08-09T13:53:27.813 INFO:tasks.ceph.mgr.y.smithi043.stderr:2023-08-09T13:53:27.812+0000 7fa15623e040 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2023-08-09T13:53:27.908 INFO:tasks.ceph.mgr.x.smithi203.stderr:2023-08-09T13:53:27.906+0000 7f005263d040 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2023-08-09T13:53:27.965 INFO:tasks.ceph.mgr.x.smithi203.stderr:2023-08-09T13:53:27.963+0000 7f005263d040 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2023-08-09T13:53:27.985 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:27.983+0000 7fa8114c5640 1 Processor -- start 2023-08-09T13:53:27.985 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:27.984+0000 7fa8114c5640 1 -- start start 2023-08-09T13:53:27.986 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:27.985+0000 7fa8114c5640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7fa80c0b2790 0x7fa80c0d2ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:53:27.987 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:27.985+0000 7fa8114c5640 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7fa80c0b2b70 0x7fa80c0b5330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:53:27.987 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:27.985+0000 7fa8114c5640 1 -- --> v1:172.21.15.203:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa80c0ce090 con 0x7fa80c0b3050 2023-08-09T13:53:27.987 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:27.985+0000 7fa8114c5640 1 -- --> v2:172.21.15.43:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa80c0b3950 con 0x7fa80c0b2790 2023-08-09T13:53:27.987 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:27.985+0000 7fa8114c5640 1 -- --> v2:172.21.15.203:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa80c0cd7f0 con 0x7fa80c0b2b70 2023-08-09T13:53:27.988 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:27.986+0000 7fa80affd640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7fa80c0b2790 0x7fa80c0d2ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:53:27.988 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:27.986+0000 7fa80affd640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7fa80c0b2790 0x7fa80c0d2ed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:58162/0 (socket says 172.21.15.43:58162) 2023-08-09T13:53:27.988 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:27.986+0000 7fa80affd640 1 -- 172.21.15.43:0/1689098441 learned_addr learned my addr 172.21.15.43:0/1689098441 (peer_addr_for_me v2:172.21.15.43:0/0) 2023-08-09T13:53:27.988 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:27.986+0000 7fa80a7fc640 1 --2- 172.21.15.43:0/1689098441 >> v2:172.21.15.203:3300/0 conn(0x7fa80c0b2b70 0x7fa80c0b5330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:53:27.989 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:27.986+0000 7fa80affd640 1 -- 172.21.15.43:0/1689098441 >> v2:172.21.15.203:3300/0 conn(0x7fa80c0b2b70 msgr2=0x7fa80c0b5330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-09T13:53:27.989 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:27.986+0000 7fa80affd640 1 --2- 172.21.15.43:0/1689098441 >> v2:172.21.15.203:3300/0 conn(0x7fa80c0b2b70 0x7fa80c0b5330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:53:27.989 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:27.986+0000 7fa80affd640 1 -- 172.21.15.43:0/1689098441 >> v1:172.21.15.203:6789/0 conn(0x7fa80c0b3050 legacy=0x7fa80c0b3f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-09T13:53:27.989 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:27.986+0000 7fa80affd640 1 -- 172.21.15.43:0/1689098441 --> v2:172.21.15.43:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa80c0cdad0 con 0x7fa80c0b2790 2023-08-09T13:53:27.990 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:27.987+0000 7fa80affd640 1 --2- 172.21.15.43:0/1689098441 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa80c0b2790 0x7fa80c0d2ed0 secure :-1 s=READY pgs=1417 cs=0 l=1 rev1=1 crypto rx=0x7fa80c066af0 tx=0x7fa7fc02b910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=87604322e430a786 server_cookie=0 in_seq=0 out_seq=0 2023-08-09T13:53:27.990 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:27.988+0000 7fa809ffb640 1 -- 172.21.15.43:0/1689098441 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa7fc031870 con 0x7fa80c0b2790 2023-08-09T13:53:27.990 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:27.988+0000 7fa809ffb640 1 -- 172.21.15.43:0/1689098441 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa7fc0319d0 con 0x7fa80c0b2790 2023-08-09T13:53:27.990 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:27.988+0000 7fa809ffb640 1 -- 172.21.15.43:0/1689098441 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa7fc031d60 con 0x7fa80c0b2790 2023-08-09T13:53:27.991 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:27.988+0000 7fa8114c5640 1 -- 172.21.15.43:0/1689098441 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa80c0b2790 msgr2=0x7fa80c0d2ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:53:27.991 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:27.988+0000 7fa8114c5640 1 --2- 172.21.15.43:0/1689098441 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa80c0b2790 0x7fa80c0d2ed0 secure :-1 s=READY pgs=1417 cs=0 l=1 rev1=1 crypto rx=0x7fa80c066af0 tx=0x7fa7fc02b910 comp rx=0 tx=0).stop 2023-08-09T13:53:27.991 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:27.988+0000 7fa8114c5640 1 -- 172.21.15.43:0/1689098441 shutdown_connections 2023-08-09T13:53:27.991 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:27.988+0000 7fa8114c5640 1 --2- 172.21.15.43:0/1689098441 >> v2:172.21.15.203:3300/0 conn(0x7fa80c0b2b70 0x7fa80c0b5330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:53:27.991 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:27.988+0000 7fa8114c5640 1 --2- 172.21.15.43:0/1689098441 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa80c0b2790 0x7fa80c0d2ed0 unknown :-1 s=CLOSED pgs=1417 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:53:27.991 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:27.988+0000 7fa8114c5640 1 -- 172.21.15.43:0/1689098441 >> 172.21.15.43:0/1689098441 conn(0x7fa80c09b7b0 msgr2=0x7fa80c0d24e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-09T13:53:27.991 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:27.988+0000 7fa8114c5640 1 -- 172.21.15.43:0/1689098441 shutdown_connections 2023-08-09T13:53:27.992 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:27.988+0000 7fa8114c5640 1 -- 172.21.15.43:0/1689098441 wait complete. 2023-08-09T13:53:27.992 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:27.990+0000 7fa8114c5640 1 Processor -- start 2023-08-09T13:53:27.992 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:27.990+0000 7fa8114c5640 1 -- start start 2023-08-09T13:53:27.993 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:27.991+0000 7fa8114c5640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fa80c0b2790 0x7fa80c0c7ba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:53:27.993 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:27.991+0000 7fa8114c5640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa80c0b2b70 0x7fa80c15b5c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:53:27.994 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:27.991+0000 7fa8114c5640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fa80c0b3050 0x7fa80c06b810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:53:27.994 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:27.991+0000 7fa8114c5640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa80c0c80e0 con 0x7fa80c0b2b70 2023-08-09T13:53:27.994 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:27.991+0000 7fa8114c5640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa80c0a0230 con 0x7fa80c0b3050 2023-08-09T13:53:27.994 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:27.991+0000 7fa8114c5640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa80c0cb2a0 con 0x7fa80c0b2790 2023-08-09T13:53:27.994 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:27.992+0000 7fa80a7fc640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa80c0b2b70 0x7fa80c15b5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:53:27.994 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:27.992+0000 7fa80a7fc640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa80c0b2b70 0x7fa80c15b5c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:58174/0 (socket says 172.21.15.43:58174) 2023-08-09T13:53:27.994 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:27.992+0000 7fa80a7fc640 1 -- 172.21.15.43:0/900385742 learned_addr learned my addr 172.21.15.43:0/900385742 (peer_addr_for_me v2:172.21.15.43:0/0) 2023-08-09T13:53:27.995 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:27.992+0000 7fa80affd640 1 --2- 172.21.15.43:0/900385742 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fa80c0b2790 0x7fa80c0c7ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:53:27.995 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:27.992+0000 7fa80b7fe640 1 --2- 172.21.15.43:0/900385742 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fa80c0b3050 0x7fa80c06b810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:53:27.995 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:27.992+0000 7fa80a7fc640 1 -- 172.21.15.43:0/900385742 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fa80c0b2790 msgr2=0x7fa80c0c7ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:53:27.995 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:27.992+0000 7fa80a7fc640 1 --2- 172.21.15.43:0/900385742 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fa80c0b2790 0x7fa80c0c7ba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:53:27.995 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:27.992+0000 7fa80a7fc640 1 -- 172.21.15.43:0/900385742 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fa80c0b3050 msgr2=0x7fa80c06b810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:53:27.995 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:27.992+0000 7fa80a7fc640 1 --2- 172.21.15.43:0/900385742 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fa80c0b3050 0x7fa80c06b810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:53:27.995 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:27.992+0000 7fa80a7fc640 1 -- 172.21.15.43:0/900385742 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa7fc0094d0 con 0x7fa80c0b2b70 2023-08-09T13:53:27.996 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:27.992+0000 7fa80b7fe640 1 --2- 172.21.15.43:0/900385742 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fa80c0b3050 0x7fa80c06b810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-08-09T13:53:27.996 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:27.992+0000 7fa80a7fc640 1 --2- 172.21.15.43:0/900385742 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa80c0b2b70 0x7fa80c15b5c0 secure :-1 s=READY pgs=1418 cs=0 l=1 rev1=1 crypto rx=0x7fa80000d5f0 tx=0x7fa80000dab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-09T13:53:27.996 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:27.993+0000 7fa7f37fe640 1 -- 172.21.15.43:0/900385742 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa8000125f0 con 0x7fa80c0b2b70 2023-08-09T13:53:27.997 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:27.993+0000 7fa7f37fe640 1 -- 172.21.15.43:0/900385742 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa800012750 con 0x7fa80c0b2b70 2023-08-09T13:53:27.997 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:27.993+0000 7fa7f37fe640 1 -- 172.21.15.43:0/900385742 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa800012b00 con 0x7fa80c0b2b70 2023-08-09T13:53:27.997 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:27.993+0000 7fa8114c5640 1 -- 172.21.15.43:0/900385742 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa80c0cb580 con 0x7fa80c0b2b70 2023-08-09T13:53:27.997 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:27.993+0000 7fa8114c5640 1 -- 172.21.15.43:0/900385742 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa80c0cbb40 con 0x7fa80c0b2b70 2023-08-09T13:53:27.997 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:27.994+0000 7fa8114c5640 1 -- 172.21.15.43:0/900385742 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa80c0d0910 con 0x7fa80c0b2b70 2023-08-09T13:53:27.998 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:27.996+0000 7fa7f37fe640 1 -- 172.21.15.43:0/900385742 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 52) v1 ==== 88557+0+0 (secure 0 0 0) 0x7fa800012c60 con 0x7fa80c0b2b70 2023-08-09T13:53:27.998 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:27.997+0000 7fa7f37fe640 1 --2- 172.21.15.43:0/900385742 >> [v2:172.21.15.203:6832/2041169533,v1:172.21.15.203:6833/2041169533] conn(0x7fa7f806a290 0x7fa7f806c750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:53:27.998 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:27.997+0000 7fa7f37fe640 1 -- 172.21.15.43:0/900385742 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(117..117 src has 1..117) v4 ==== 6770+0+0 (secure 0 0 0) 0x7fa8000891b0 con 0x7fa80c0b2b70 2023-08-09T13:53:27.999 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:27.997+0000 7fa80affd640 1 -- 172.21.15.43:0/900385742 >> [v2:172.21.15.203:6832/2041169533,v1:172.21.15.203:6833/2041169533] conn(0x7fa7f806a290 msgr2=0x7fa7f806c750 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.203:6832/2041169533 2023-08-09T13:53:27.999 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:27.997+0000 7fa80affd640 1 --2- 172.21.15.43:0/900385742 >> [v2:172.21.15.203:6832/2041169533,v1:172.21.15.203:6833/2041169533] conn(0x7fa7f806a290 0x7fa7f806c750 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-08-09T13:53:28.003 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:28.002+0000 7fa7f37fe640 1 -- 172.21.15.43:0/900385742 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178066 (secure 0 0 0) 0x7fa80c0d0910 con 0x7fa80c0b2b70 2023-08-09T13:53:28.017 INFO:tasks.ceph.mgr.x.smithi203.stderr:2023-08-09T13:53:28.015+0000 7f005263d040 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2023-08-09T13:53:28.133 INFO:tasks.ceph.mgr.y.smithi043.stderr:2023-08-09T13:53:28.131+0000 7fa15623e040 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2023-08-09T13:53:28.176 INFO:tasks.ceph.mgr.x.smithi203.stderr:2023-08-09T13:53:28.174+0000 7f005263d040 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2023-08-09T13:53:28.194 INFO:tasks.ceph.mgr.y.smithi043.stderr:2023-08-09T13:53:28.192+0000 7fa15623e040 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2023-08-09T13:53:28.200 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:28.198+0000 7fa80affd640 1 -- 172.21.15.43:0/900385742 >> [v2:172.21.15.203:6832/2041169533,v1:172.21.15.203:6833/2041169533] conn(0x7fa7f806a290 msgr2=0x7fa7f806c750 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.203:6832/2041169533 2023-08-09T13:53:28.200 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:28.198+0000 7fa80affd640 1 --2- 172.21.15.43:0/900385742 >> [v2:172.21.15.203:6832/2041169533,v1:172.21.15.203:6833/2041169533] conn(0x7fa7f806a290 0x7fa7f806c750 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2023-08-09T13:53:28.230 INFO:tasks.ceph.mgr.x.smithi203.stderr:2023-08-09T13:53:28.228+0000 7f005263d040 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2023-08-09T13:53:28.236 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:28.235+0000 7fa8114c5640 1 -- 172.21.15.43:0/900385742 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fa80c0d0910 con 0x7fa80c0b2b70 2023-08-09T13:53:28.237 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:28.235+0000 7fa7f37fe640 1 -- 172.21.15.43:0/900385742 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 52 v52) v1 ==== 94+0+4883 (secure 0 0 0) 0x7fa80c0d0910 con 0x7fa80c0b2b70 2023-08-09T13:53:28.237 INFO:teuthology.orchestra.run.smithi043.stdout: 2023-08-09T13:53:28.237 INFO:teuthology.orchestra.run.smithi043.stdout:{"epoch":52,"default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6215,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.43:6835/2214346432","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6833","nonce":2214346432},{"type":"v1","addr":"172.21.15.43:6835","nonce":2214346432}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":47},{"gid":6823,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.203:6835/3236264045","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6834","nonce":3236264045},{"type":"v1","addr":"172.21.15.203:6835","nonce":3236264045}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":46},{"gid":6826,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.203:6837/3788289156","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6836","nonce":3788289156},{"type":"v1","addr":"172.21.15.203:6837","nonce":3788289156}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":47}],"filesystems":[{"mdsmap":{"epoch":52,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-08-09T13:51:59.655630+0000","modified":"2023-08-09T13:52:03.220527+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":8040},"failed":[],"damaged":[],"stopped":[],"info":{"gid_8040":{"gid":8040,"name":"c","rank":0,"incarnation":48,"state":"up:active","state_seq":3,"addr":"172.21.15.43:6834/2223580538","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6832","nonce":2223580538},{"type":"v1","addr":"172.21.15.43:6834","nonce":2223580538}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[15],"metadata_pool":14,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":6}]} 2023-08-09T13:53:28.238 INFO:teuthology.orchestra.run.smithi043.stderr:dumped fsmap epoch 52 2023-08-09T13:53:28.239 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:28.238+0000 7fa8114c5640 1 -- 172.21.15.43:0/900385742 >> [v2:172.21.15.203:6832/2041169533,v1:172.21.15.203:6833/2041169533] conn(0x7fa7f806a290 msgr2=0x7fa7f806c750 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-08-09T13:53:28.240 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:28.238+0000 7fa8114c5640 1 --2- 172.21.15.43:0/900385742 >> [v2:172.21.15.203:6832/2041169533,v1:172.21.15.203:6833/2041169533] conn(0x7fa7f806a290 0x7fa7f806c750 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:53:28.240 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:28.238+0000 7fa8114c5640 1 -- 172.21.15.43:0/900385742 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa80c0b2b70 msgr2=0x7fa80c15b5c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:53:28.240 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:28.238+0000 7fa8114c5640 1 --2- 172.21.15.43:0/900385742 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa80c0b2b70 0x7fa80c15b5c0 secure :-1 s=READY pgs=1418 cs=0 l=1 rev1=1 crypto rx=0x7fa80000d5f0 tx=0x7fa80000dab0 comp rx=0 tx=0).stop 2023-08-09T13:53:28.240 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:28.238+0000 7fa8114c5640 1 -- 172.21.15.43:0/900385742 shutdown_connections 2023-08-09T13:53:28.240 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:28.238+0000 7fa8114c5640 1 --2- 172.21.15.43:0/900385742 >> [v2:172.21.15.203:6832/2041169533,v1:172.21.15.203:6833/2041169533] conn(0x7fa7f806a290 0x7fa7f806c750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:53:28.241 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:28.238+0000 7fa8114c5640 1 --2- 172.21.15.43:0/900385742 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fa80c0b3050 0x7fa80c06b810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:53:28.241 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:28.238+0000 7fa8114c5640 1 --2- 172.21.15.43:0/900385742 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa80c0b2b70 0x7fa80c15b5c0 unknown :-1 s=CLOSED pgs=1418 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:53:28.241 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:28.238+0000 7fa8114c5640 1 --2- 172.21.15.43:0/900385742 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fa80c0b2790 0x7fa80c0c7ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:53:28.241 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:28.238+0000 7fa8114c5640 1 -- 172.21.15.43:0/900385742 >> 172.21.15.43:0/900385742 conn(0x7fa80c09b7b0 msgr2=0x7fa80c0b5a80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-09T13:53:28.241 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:28.238+0000 7fa8114c5640 1 -- 172.21.15.43:0/900385742 shutdown_connections 2023-08-09T13:53:28.241 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:28.238+0000 7fa8114c5640 1 -- 172.21.15.43:0/900385742 wait complete. 2023-08-09T13:53:28.256 INFO:tasks.ceph.mgr.y.smithi043.stderr:2023-08-09T13:53:28.255+0000 7fa15623e040 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2023-08-09T13:53:28.258 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-08-09T13:53:28.285 INFO:tasks.ceph.mgr.x.smithi203.stderr:2023-08-09T13:53:28.282+0000 7f005263d040 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2023-08-09T13:53:28.336 INFO:tasks.ceph.mgr.x.smithi203.stderr:2023-08-09T13:53:28.333+0000 7f005263d040 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2023-08-09T13:53:28.429 INFO:tasks.ceph.mgr.x.smithi203.stderr:2023-08-09T13:53:28.426+0000 7effd892e640 -1 client.0 error registering admin socket command: (17) File exists 2023-08-09T13:53:28.429 INFO:tasks.ceph.mgr.x.smithi203.stderr:2023-08-09T13:53:28.426+0000 7effd892e640 -1 client.0 error registering admin socket command: (17) File exists 2023-08-09T13:53:28.429 INFO:tasks.ceph.mgr.x.smithi203.stderr:2023-08-09T13:53:28.426+0000 7effd892e640 -1 client.0 error registering admin socket command: (17) File exists 2023-08-09T13:53:28.429 INFO:tasks.ceph.mgr.x.smithi203.stderr:2023-08-09T13:53:28.426+0000 7effd892e640 -1 client.0 error registering admin socket command: (17) File exists 2023-08-09T13:53:28.430 INFO:tasks.ceph.mgr.x.smithi203.stderr:2023-08-09T13:53:28.426+0000 7effd892e640 -1 client.0 error registering admin socket command: (17) File exists 2023-08-09T13:53:28.430 INFO:tasks.ceph.mgr.x.smithi203.stderr:2023-08-09T13:53:28.426+0000 7effd5127640 -1 client.0 error registering admin socket command: (17) File exists 2023-08-09T13:53:28.430 INFO:tasks.ceph.mgr.x.smithi203.stderr:2023-08-09T13:53:28.426+0000 7effd5127640 -1 client.0 error registering admin socket command: (17) File exists 2023-08-09T13:53:28.430 INFO:tasks.ceph.mgr.x.smithi203.stderr:2023-08-09T13:53:28.426+0000 7effd5127640 -1 client.0 error registering admin socket command: (17) File exists 2023-08-09T13:53:28.430 INFO:tasks.ceph.mgr.x.smithi203.stderr:2023-08-09T13:53:28.426+0000 7effd5127640 -1 client.0 error registering admin socket command: (17) File exists 2023-08-09T13:53:28.430 INFO:tasks.ceph.mgr.x.smithi203.stderr:2023-08-09T13:53:28.426+0000 7effd5127640 -1 client.0 error registering admin socket command: (17) File exists 2023-08-09T13:53:28.440 INFO:tasks.ceph.mgr.y.smithi043.stderr:2023-08-09T13:53:28.439+0000 7fa15623e040 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2023-08-09T13:53:28.494 INFO:tasks.ceph.mgr.y.smithi043.stderr:2023-08-09T13:53:28.492+0000 7fa15623e040 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2023-08-09T13:53:28.511 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:28.510+0000 7fd20f871640 1 Processor -- start 2023-08-09T13:53:28.512 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:28.510+0000 7fd20f871640 1 -- start start 2023-08-09T13:53:28.513 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:28.511+0000 7fd20f871640 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7fd2080ce720 0x7fd2080ceb00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:53:28.513 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:28.511+0000 7fd20f871640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fd2080ce080 0x7fd20815e5b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:53:28.513 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:28.511+0000 7fd20f871640 1 -- --> v1:172.21.15.203:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd2080cd8a0 con 0x7fd20815eaf0 2023-08-09T13:53:28.513 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:28.511+0000 7fd20f871640 1 -- --> v2:172.21.15.203:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd2080cf350 con 0x7fd2080ce720 2023-08-09T13:53:28.513 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:28.511+0000 7fd20f871640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd2080cf040 con 0x7fd2080ce080 2023-08-09T13:53:28.514 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:28.512+0000 7fd20d60d640 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7fd2080ce720 0x7fd2080ceb00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:53:28.514 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:28.512+0000 7fd20d60d640 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7fd2080ce720 0x7fd2080ceb00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.203:3300/0 says I am v2:172.21.15.43:38418/0 (socket says 172.21.15.43:38418) 2023-08-09T13:53:28.515 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:28.512+0000 7fd20d60d640 1 -- 172.21.15.43:0/723275895 learned_addr learned my addr 172.21.15.43:0/723275895 (peer_addr_for_me v2:172.21.15.43:0/0) 2023-08-09T13:53:28.515 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:28.512+0000 7fd20ce0c640 1 --2- 172.21.15.43:0/723275895 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fd2080ce080 0x7fd20815e5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:53:28.515 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:28.513+0000 7fd20d60d640 1 -- 172.21.15.43:0/723275895 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fd2080ce080 msgr2=0x7fd20815e5b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-09T13:53:28.515 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:28.513+0000 7fd20d60d640 1 --2- 172.21.15.43:0/723275895 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fd2080ce080 0x7fd20815e5b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:53:28.515 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:28.513+0000 7fd20d60d640 1 -- 172.21.15.43:0/723275895 >> v1:172.21.15.203:6789/0 conn(0x7fd20815eaf0 legacy=0x7fd2080cafa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:53:28.515 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:28.513+0000 7fd20d60d640 1 -- 172.21.15.43:0/723275895 --> v2:172.21.15.203:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd208071a70 con 0x7fd2080ce720 2023-08-09T13:53:28.516 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:28.514+0000 7fd20d60d640 1 --2- 172.21.15.43:0/723275895 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fd2080ce720 0x7fd2080ceb00 secure :-1 s=READY pgs=538 cs=0 l=1 rev1=1 crypto rx=0x7fd204009850 tx=0x7fd20402b8d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=63c790ad76ba493f server_cookie=0 in_seq=0 out_seq=0 2023-08-09T13:53:28.516 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:28.515+0000 7fd1fffff640 1 -- 172.21.15.43:0/723275895 <== mon.1 v2:172.21.15.203:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd204004230 con 0x7fd2080ce720 2023-08-09T13:53:28.516 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:28.515+0000 7fd1fffff640 1 -- 172.21.15.43:0/723275895 <== mon.1 v2:172.21.15.203:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd204004390 con 0x7fd2080ce720 2023-08-09T13:53:28.517 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:28.515+0000 7fd1fffff640 1 -- 172.21.15.43:0/723275895 <== mon.1 v2:172.21.15.203:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd204031830 con 0x7fd2080ce720 2023-08-09T13:53:28.517 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:28.516+0000 7fd20f871640 1 -- 172.21.15.43:0/723275895 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fd2080ce720 msgr2=0x7fd2080ceb00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:53:28.518 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:28.516+0000 7fd20f871640 1 --2- 172.21.15.43:0/723275895 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fd2080ce720 0x7fd2080ceb00 secure :-1 s=READY pgs=538 cs=0 l=1 rev1=1 crypto rx=0x7fd204009850 tx=0x7fd20402b8d0 comp rx=0 tx=0).stop 2023-08-09T13:53:28.518 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:28.516+0000 7fd20f871640 1 -- 172.21.15.43:0/723275895 shutdown_connections 2023-08-09T13:53:28.518 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:28.516+0000 7fd20f871640 1 --2- 172.21.15.43:0/723275895 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fd2080ce080 0x7fd20815e5b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:53:28.518 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:28.516+0000 7fd20f871640 1 --2- 172.21.15.43:0/723275895 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fd2080ce720 0x7fd2080ceb00 unknown :-1 s=CLOSED pgs=538 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:53:28.518 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:28.516+0000 7fd20f871640 1 -- 172.21.15.43:0/723275895 >> 172.21.15.43:0/723275895 conn(0x7fd2080ccd80 msgr2=0x7fd2080cd190 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-09T13:53:28.518 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:28.516+0000 7fd20f871640 1 -- 172.21.15.43:0/723275895 shutdown_connections 2023-08-09T13:53:28.518 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:28.516+0000 7fd20f871640 1 -- 172.21.15.43:0/723275895 wait complete. 2023-08-09T13:53:28.519 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:28.518+0000 7fd20f871640 1 Processor -- start 2023-08-09T13:53:28.519 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:28.518+0000 7fd20f871640 1 -- start start 2023-08-09T13:53:28.520 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:28.519+0000 7fd20f871640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fd2080ce080 0x7fd2080c6480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:53:28.520 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:28.519+0000 7fd20f871640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fd2080ce720 0x7fd2080b4ce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:53:28.521 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:28.519+0000 7fd20f871640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fd20815eaf0 0x7fd2081595c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:53:28.521 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:28.519+0000 7fd20f871640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd2081f0460 con 0x7fd20815eaf0 2023-08-09T13:53:28.521 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:28.519+0000 7fd20f871640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd2081efb00 con 0x7fd2080ce080 2023-08-09T13:53:28.521 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:28.519+0000 7fd20f871640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd2081efda0 con 0x7fd2080ce720 2023-08-09T13:53:28.522 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:28.519+0000 7fd20de0e640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fd20815eaf0 0x7fd2081595c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:53:28.523 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:28.519+0000 7fd20de0e640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fd20815eaf0 0x7fd2081595c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:58184/0 (socket says 172.21.15.43:58184) 2023-08-09T13:53:28.523 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:28.519+0000 7fd20de0e640 1 -- 172.21.15.43:0/260538730 learned_addr learned my addr 172.21.15.43:0/260538730 (peer_addr_for_me v2:172.21.15.43:0/0) 2023-08-09T13:53:28.523 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:28.520+0000 7fd20d60d640 1 --2- 172.21.15.43:0/260538730 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fd2080ce080 0x7fd2080c6480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:53:28.523 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:28.520+0000 7fd20ce0c640 1 --2- 172.21.15.43:0/260538730 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fd2080ce720 0x7fd2080b4ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:53:28.523 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:28.520+0000 7fd20de0e640 1 -- 172.21.15.43:0/260538730 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fd2080ce720 msgr2=0x7fd2080b4ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:53:28.523 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:28.520+0000 7fd20de0e640 1 --2- 172.21.15.43:0/260538730 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fd2080ce720 0x7fd2080b4ce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:53:28.523 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:28.520+0000 7fd20de0e640 1 -- 172.21.15.43:0/260538730 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fd2080ce080 msgr2=0x7fd2080c6480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:53:28.524 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:28.520+0000 7fd20de0e640 1 --2- 172.21.15.43:0/260538730 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fd2080ce080 0x7fd2080c6480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:53:28.524 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:28.520+0000 7fd20de0e640 1 -- 172.21.15.43:0/260538730 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd204009460 con 0x7fd20815eaf0 2023-08-09T13:53:28.524 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:28.520+0000 7fd20de0e640 1 --2- 172.21.15.43:0/260538730 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fd20815eaf0 0x7fd2081595c0 secure :-1 s=READY pgs=1420 cs=0 l=1 rev1=1 crypto rx=0x7fd20000c6b0 tx=0x7fd20000cb70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-09T13:53:28.524 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:28.521+0000 7fd1fdffb640 1 -- 172.21.15.43:0/260538730 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd200003b70 con 0x7fd20815eaf0 2023-08-09T13:53:28.524 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:28.521+0000 7fd1fdffb640 1 -- 172.21.15.43:0/260538730 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd200003cf0 con 0x7fd20815eaf0 2023-08-09T13:53:28.524 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:28.521+0000 7fd20f871640 1 -- 172.21.15.43:0/260538730 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd2081f0080 con 0x7fd20815eaf0 2023-08-09T13:53:28.524 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:28.521+0000 7fd1fdffb640 1 -- 172.21.15.43:0/260538730 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd200003ea0 con 0x7fd20815eaf0 2023-08-09T13:53:28.525 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:28.521+0000 7fd20f871640 1 -- 172.21.15.43:0/260538730 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd2080b57a0 con 0x7fd20815eaf0 2023-08-09T13:53:28.525 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:28.522+0000 7fd20f871640 1 -- 172.21.15.43:0/260538730 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd2081f05a0 con 0x7fd20815eaf0 2023-08-09T13:53:28.525 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:28.523+0000 7fd1fdffb640 1 -- 172.21.15.43:0/260538730 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 53) v1 ==== 88052+0+0 (secure 0 0 0) 0x7fd200005310 con 0x7fd20815eaf0 2023-08-09T13:53:28.526 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:28.524+0000 7fd1fdffb640 1 -- 172.21.15.43:0/260538730 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(118..118 src has 1..118) v4 ==== 7200+0+0 (secure 0 0 0) 0x7fd200088270 con 0x7fd20815eaf0 2023-08-09T13:53:28.529 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:28.528+0000 7fd1fdffb640 1 -- 172.21.15.43:0/260538730 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178066 (secure 0 0 0) 0x7fd2081f05a0 con 0x7fd20815eaf0 2023-08-09T13:53:28.553 INFO:tasks.ceph.mgr.y.smithi043.stderr:2023-08-09T13:53:28.552+0000 7fa15623e040 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2023-08-09T13:53:28.607 INFO:tasks.ceph.mgr.y.smithi043.stderr:2023-08-09T13:53:28.606+0000 7fa15623e040 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2023-08-09T13:53:28.729 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:28.727+0000 7fd20f871640 1 -- 172.21.15.43:0/260538730 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fd2081f05a0 con 0x7fd20815eaf0 2023-08-09T13:53:28.732 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:28.731+0000 7fd1fdffb640 1 -- 172.21.15.43:0/260538730 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v118) v1 ==== 74+0+19647 (secure 0 0 0) 0x7fd2081f05a0 con 0x7fd20815eaf0 2023-08-09T13:53:28.733 INFO:teuthology.orchestra.run.smithi043.stdout: 2023-08-09T13:53:28.733 INFO:teuthology.orchestra.run.smithi043.stdout:{"epoch":118,"fsid":"daac87ee-1337-4250-a955-aea4e408a39b","created":"2023-08-09T13:33:41.701198+0000","modified":"2023-08-09T13:53:28.338768+0000","last_up_change":"2023-08-09T13:33:53.783453+0000","last_in_change":"2023-08-09T13:33:45.148904+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":15,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":"rbd","create_time":"2023-08-09T13:33:54.365137+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":".mgr","create_time":"2023-08-09T13:33:54.831803+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":14,"pool_name":"cephfs_metadata","create_time":"2023-08-09T13:51:57.637653+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"115","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":2.5,"score_stable":2.5,"optimal_score":1,"raw_score_acting":2.5,"raw_score_stable":2.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":15,"pool_name":"cephfs_data","create_time":"2023-08-09T13:51:58.676117+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"116","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"cbd49ecd-e896-4fbd-864e-5c37a829b165","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":114,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6800","nonce":502667335},{"type":"v1","addr":"172.21.15.43:6801","nonce":502667335}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6802","nonce":502667335},{"type":"v1","addr":"172.21.15.43:6803","nonce":502667335}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6806","nonce":502667335},{"type":"v1","addr":"172.21.15.43:6807","nonce":502667335}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6804","nonce":502667335},{"type":"v1","addr":"172.21.15.43:6805","nonce":502667335}]},"public_addr":"172.21.15.43:6801/502667335","cluster_addr":"172.21.15.43:6803/502667335","heartbeat_back_addr":"172.21.15.43:6807/502667335","heartbeat_front_addr":"172.21.15.43:6805/502667335","state":["exists","up"]},{"osd":1,"uuid":"75c7e0fe-1ec7-4fe8-a2f2-3661a65b8500","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":114,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6808","nonce":3289134610},{"type":"v1","addr":"172.21.15.43:6809","nonce":3289134610}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6810","nonce":3289134610},{"type":"v1","addr":"172.21.15.43:6812","nonce":3289134610}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6821","nonce":3289134610},{"type":"v1","addr":"172.21.15.43:6824","nonce":3289134610}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6815","nonce":3289134610},{"type":"v1","addr":"172.21.15.43:6818","nonce":3289134610}]},"public_addr":"172.21.15.43:6809/3289134610","cluster_addr":"172.21.15.43:6812/3289134610","heartbeat_back_addr":"172.21.15.43:6824/3289134610","heartbeat_front_addr":"172.21.15.43:6818/3289134610","state":["exists","up"]},{"osd":2,"uuid":"bb5c41e9-7f5f-4125-9b01-3bb8bfc2ea83","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":114,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6811","nonce":4079709736},{"type":"v1","addr":"172.21.15.43:6814","nonce":4079709736}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6817","nonce":4079709736},{"type":"v1","addr":"172.21.15.43:6820","nonce":4079709736}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6829","nonce":4079709736},{"type":"v1","addr":"172.21.15.43:6831","nonce":4079709736}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6823","nonce":4079709736},{"type":"v1","addr":"172.21.15.43:6826","nonce":4079709736}]},"public_addr":"172.21.15.43:6814/4079709736","cluster_addr":"172.21.15.43:6820/4079709736","heartbeat_back_addr":"172.21.15.43:6831/4079709736","heartbeat_front_addr":"172.21.15.43:6826/4079709736","state":["exists","up"]},{"osd":3,"uuid":"bfe4c8b4-694a-465c-87e6-2b2aab852613","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":114,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6813","nonce":1709538759},{"type":"v1","addr":"172.21.15.43:6816","nonce":1709538759}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6819","nonce":1709538759},{"type":"v1","addr":"172.21.15.43:6822","nonce":1709538759}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6828","nonce":1709538759},{"type":"v1","addr":"172.21.15.43:6830","nonce":1709538759}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6825","nonce":1709538759},{"type":"v1","addr":"172.21.15.43:6827","nonce":1709538759}]},"public_addr":"172.21.15.43:6816/1709538759","cluster_addr":"172.21.15.43:6822/1709538759","heartbeat_back_addr":"172.21.15.43:6830/1709538759","heartbeat_front_addr":"172.21.15.43:6827/1709538759","state":["exists","up"]},{"osd":4,"uuid":"fa2e8a77-aba1-4756-853e-0d174a5bdeba","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":114,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6800","nonce":4196866481},{"type":"v1","addr":"172.21.15.203:6801","nonce":4196866481}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6802","nonce":4196866481},{"type":"v1","addr":"172.21.15.203:6803","nonce":4196866481}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6806","nonce":4196866481},{"type":"v1","addr":"172.21.15.203:6807","nonce":4196866481}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6804","nonce":4196866481},{"type":"v1","addr":"172.21.15.203:6805","nonce":4196866481}]},"public_addr":"172.21.15.203:6801/4196866481","cluster_addr":"172.21.15.203:6803/4196866481","heartbeat_back_addr":"172.21.15.203:6807/4196866481","heartbeat_front_addr":"172.21.15.203:6805/4196866481","state":["exists","up"]},{"osd":5,"uuid":"81811e4b-4d24-4f51-a82c-f1d700a2a00d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":114,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6816","nonce":3059290081},{"type":"v1","addr":"172.21.15.203:6817","nonce":3059290081}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6818","nonce":3059290081},{"type":"v1","addr":"172.21.15.203:6819","nonce":3059290081}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6822","nonce":3059290081},{"type":"v1","addr":"172.21.15.203:6823","nonce":3059290081}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6820","nonce":3059290081},{"type":"v1","addr":"172.21.15.203:6821","nonce":3059290081}]},"public_addr":"172.21.15.203:6817/3059290081","cluster_addr":"172.21.15.203:6819/3059290081","heartbeat_back_addr":"172.21.15.203:6823/3059290081","heartbeat_front_addr":"172.21.15.203:6821/3059290081","state":["exists","up"]},{"osd":6,"uuid":"307bac4d-6a5b-443d-af75-56d695d7674f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":114,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6824","nonce":512155569},{"type":"v1","addr":"172.21.15.203:6825","nonce":512155569}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6826","nonce":512155569},{"type":"v1","addr":"172.21.15.203:6827","nonce":512155569}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6830","nonce":512155569},{"type":"v1","addr":"172.21.15.203:6831","nonce":512155569}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6828","nonce":512155569},{"type":"v1","addr":"172.21.15.203:6829","nonce":512155569}]},"public_addr":"172.21.15.203:6825/512155569","cluster_addr":"172.21.15.203:6827/512155569","heartbeat_back_addr":"172.21.15.203:6831/512155569","heartbeat_front_addr":"172.21.15.203:6829/512155569","state":["exists","up"]},{"osd":7,"uuid":"a95dc7ca-9f1b-4295-af70-68a22a2d7d5e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":114,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6808","nonce":3331450438},{"type":"v1","addr":"172.21.15.203:6809","nonce":3331450438}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6810","nonce":3331450438},{"type":"v1","addr":"172.21.15.203:6811","nonce":3331450438}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6814","nonce":3331450438},{"type":"v1","addr":"172.21.15.203:6815","nonce":3331450438}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6812","nonce":3331450438},{"type":"v1","addr":"172.21.15.203:6813","nonce":3331450438}]},"public_addr":"172.21.15.203:6809/3331450438","cluster_addr":"172.21.15.203:6811/3331450438","heartbeat_back_addr":"172.21.15.203:6815/3331450438","heartbeat_front_addr":"172.21.15.203:6813/3331450438","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-09T13:33:50.547652+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-09T13:33:50.589469+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-09T13:33:50.561724+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-09T13:33:50.599650+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-09T13:33:50.411855+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-09T13:33:50.219681+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-09T13:33:50.504664+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-09T13:33:50.441248+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.203:0/3668440292":"2023-08-10T13:53:28.338643+0000","172.21.15.203:0/430012896":"2023-08-10T13:53:28.338643+0000","172.21.15.203:0/1052273067":"2023-08-10T13:53:28.338643+0000","172.21.15.203:0/2583349636":"2023-08-10T13:53:28.338643+0000","172.21.15.203:6832/2041169533":"2023-08-10T13:53:28.338643+0000","172.21.15.203:0/1488154102":"2023-08-10T13:52:20.374695+0000","172.21.15.203:0/728819840":"2023-08-10T13:53:28.338643+0000","172.21.15.203:0/3595044319":"2023-08-10T13:52:20.374695+0000","172.21.15.203:0/2916799066":"2023-08-10T13:53:28.338643+0000","172.21.15.203:0/3843164760":"2023-08-10T13:53:28.338643+0000","172.21.15.203:0/1657891023":"2023-08-10T13:52:20.374695+0000","172.21.15.203:0/329310299":"2023-08-10T13:53:28.338643+0000","172.21.15.203:0/2368610482":"2023-08-10T13:52:20.374695+0000","172.21.15.203:0/437192587":"2023-08-10T13:52:20.374695+0000","172.21.15.203:6833/1317108840":"2023-08-10T13:52:20.374695+0000","172.21.15.203:6832/1317108840":"2023-08-10T13:52:20.374695+0000","172.21.15.203:6833/2041169533":"2023-08-10T13:53:28.338643+0000","172.21.15.203:0/1459918882":"2023-08-10T13:52:20.374695+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":7,"snaps":[{"begin":104,"length":1},{"begin":221,"length":1}]}],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-08-09T13:53:28.736 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:28.733+0000 7fd20f871640 1 -- 172.21.15.43:0/260538730 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fd20815eaf0 msgr2=0x7fd2081595c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:53:28.736 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:28.733+0000 7fd20f871640 1 --2- 172.21.15.43:0/260538730 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fd20815eaf0 0x7fd2081595c0 secure :-1 s=READY pgs=1420 cs=0 l=1 rev1=1 crypto rx=0x7fd20000c6b0 tx=0x7fd20000cb70 comp rx=0 tx=0).stop 2023-08-09T13:53:28.736 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:28.734+0000 7fd20f871640 1 -- 172.21.15.43:0/260538730 shutdown_connections 2023-08-09T13:53:28.737 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:28.734+0000 7fd20f871640 1 --2- 172.21.15.43:0/260538730 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fd20815eaf0 0x7fd2081595c0 unknown :-1 s=CLOSED pgs=1420 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:53:28.737 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:28.734+0000 7fd20f871640 1 --2- 172.21.15.43:0/260538730 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fd2080ce720 0x7fd2080b4ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:53:28.737 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:28.734+0000 7fd20f871640 1 --2- 172.21.15.43:0/260538730 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fd2080ce080 0x7fd2080c6480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:53:28.737 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:28.734+0000 7fd20f871640 1 -- 172.21.15.43:0/260538730 >> 172.21.15.43:0/260538730 conn(0x7fd2080ccd80 msgr2=0x7fd2080ad780 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-09T13:53:28.737 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:28.734+0000 7fd20f871640 1 -- 172.21.15.43:0/260538730 shutdown_connections 2023-08-09T13:53:28.737 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:28.734+0000 7fd20f871640 1 -- 172.21.15.43:0/260538730 wait complete. 2023-08-09T13:53:28.753 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': 'rbd', 'create_time': '2023-08-09T13:33:54.365137+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}, 'read_balance': {'score_acting': 2, 'score_stable': 2, 'optimal_score': 1, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': '.mgr', 'create_time': '2023-08-09T13:33:54.831803+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 14, 'pool_name': 'cephfs_metadata', 'create_time': '2023-08-09T13:51:57.637653+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '115', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 2.5, 'score_stable': 2.5, 'optimal_score': 1, 'raw_score_acting': 2.5, 'raw_score_stable': 2.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 15, 'pool_name': 'cephfs_data', 'create_time': '2023-08-09T13:51:58.676117+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '116', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-08-09T13:53:28.754 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get rbd pg_num 2023-08-09T13:53:28.989 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:28.988+0000 7fa3ef59e640 1 Processor -- start 2023-08-09T13:53:28.989 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:28.988+0000 7fa3ef59e640 1 -- start start 2023-08-09T13:53:28.990 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:28.989+0000 7fa3ef59e640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7fa3f00b7df0 0x7fa3f00d1b30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:53:28.991 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:28.989+0000 7fa3ef59e640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fa3f00cebf0 0x7fa3f00cf040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:53:28.991 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:28.989+0000 7fa3ef59e640 1 -- --> v1:172.21.15.203:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa3f00cf580 con 0x7fa3f015c2d0 2023-08-09T13:53:28.991 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:28.989+0000 7fa3ef59e640 1 -- --> v2:172.21.15.43:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa3f00b8230 con 0x7fa3f00b7df0 2023-08-09T13:53:28.991 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:28.989+0000 7fa3ef59e640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa3f00d16a0 con 0x7fa3f00cebf0 2023-08-09T13:53:28.992 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:28.990+0000 7fa3edd9b640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7fa3f00b7df0 0x7fa3f00d1b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:53:28.992 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:28.990+0000 7fa3edd9b640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7fa3f00b7df0 0x7fa3f00d1b30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:58188/0 (socket says 172.21.15.43:58188) 2023-08-09T13:53:28.992 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:28.990+0000 7fa3edd9b640 1 -- 172.21.15.43:0/1513741797 learned_addr learned my addr 172.21.15.43:0/1513741797 (peer_addr_for_me v2:172.21.15.43:0/0) 2023-08-09T13:53:28.992 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:28.990+0000 7fa3eed9d640 1 --2- 172.21.15.43:0/1513741797 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fa3f00cebf0 0x7fa3f00cf040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:53:28.992 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:28.990+0000 7fa3edd9b640 1 -- 172.21.15.43:0/1513741797 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fa3f00cebf0 msgr2=0x7fa3f00cf040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-09T13:53:28.993 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:28.990+0000 7fa3edd9b640 1 --2- 172.21.15.43:0/1513741797 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fa3f00cebf0 0x7fa3f00cf040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:53:28.993 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:28.990+0000 7fa3edd9b640 1 -- 172.21.15.43:0/1513741797 >> v1:172.21.15.203:6789/0 conn(0x7fa3f015c2d0 legacy=0x7fa3f00cd020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:53:28.993 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:28.990+0000 7fa3edd9b640 1 -- 172.21.15.43:0/1513741797 --> v2:172.21.15.43:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa3f00d0d70 con 0x7fa3f00b7df0 2023-08-09T13:53:28.993 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:28.991+0000 7fa3edd9b640 1 --2- 172.21.15.43:0/1513741797 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa3f00b7df0 0x7fa3f00d1b30 secure :-1 s=READY pgs=1421 cs=0 l=1 rev1=1 crypto rx=0x7fa3e4009870 tx=0x7fa3e402b980 comp rx=0 tx=0).ready entity=mon.0 client_cookie=785e4302078b284a server_cookie=0 in_seq=0 out_seq=0 2023-08-09T13:53:28.994 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:28.992+0000 7fa3ed59a640 1 -- 172.21.15.43:0/1513741797 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa3e4030840 con 0x7fa3f00b7df0 2023-08-09T13:53:28.994 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:28.992+0000 7fa3ed59a640 1 -- 172.21.15.43:0/1513741797 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa3e40309a0 con 0x7fa3f00b7df0 2023-08-09T13:53:28.994 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:28.992+0000 7fa3ed59a640 1 -- 172.21.15.43:0/1513741797 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa3e4030d30 con 0x7fa3f00b7df0 2023-08-09T13:53:28.994 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:28.992+0000 7fa3ef59e640 1 -- 172.21.15.43:0/1513741797 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa3f00b7df0 msgr2=0x7fa3f00d1b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:53:28.994 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:28.992+0000 7fa3ef59e640 1 --2- 172.21.15.43:0/1513741797 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa3f00b7df0 0x7fa3f00d1b30 secure :-1 s=READY pgs=1421 cs=0 l=1 rev1=1 crypto rx=0x7fa3e4009870 tx=0x7fa3e402b980 comp rx=0 tx=0).stop 2023-08-09T13:53:28.995 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:28.993+0000 7fa3ef59e640 1 -- 172.21.15.43:0/1513741797 shutdown_connections 2023-08-09T13:53:28.995 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:28.993+0000 7fa3ef59e640 1 --2- 172.21.15.43:0/1513741797 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fa3f00cebf0 0x7fa3f00cf040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:53:28.995 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:28.993+0000 7fa3ef59e640 1 --2- 172.21.15.43:0/1513741797 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa3f00b7df0 0x7fa3f00d1b30 unknown :-1 s=CLOSED pgs=1421 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:53:28.995 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:28.993+0000 7fa3ef59e640 1 -- 172.21.15.43:0/1513741797 >> 172.21.15.43:0/1513741797 conn(0x7fa3f006a230 msgr2=0x7fa3f006a640 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-09T13:53:28.996 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:28.993+0000 7fa3ef59e640 1 -- 172.21.15.43:0/1513741797 shutdown_connections 2023-08-09T13:53:28.996 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:28.993+0000 7fa3ef59e640 1 -- 172.21.15.43:0/1513741797 wait complete. 2023-08-09T13:53:28.997 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:28.994+0000 7fa3ef59e640 1 Processor -- start 2023-08-09T13:53:28.997 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:28.995+0000 7fa3ef59e640 1 -- start start 2023-08-09T13:53:28.997 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:28.996+0000 7fa3ef59e640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fa3f00b7df0 0x7fa3f0157520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:53:28.997 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:28.996+0000 7fa3ef59e640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fa3f00cebf0 0x7fa3f00cb090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:53:28.997 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:28.996+0000 7fa3ef59e640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa3f015c2d0 0x7fa3f00cb5d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:53:28.998 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:28.996+0000 7fa3ef59e640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa3f0155f80 con 0x7fa3f015c2d0 2023-08-09T13:53:28.998 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:28.996+0000 7fa3ef59e640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa3f01f29c0 con 0x7fa3f00b7df0 2023-08-09T13:53:28.998 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:28.996+0000 7fa3ef59e640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa3f01f26e0 con 0x7fa3f00cebf0 2023-08-09T13:53:28.999 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:28.996+0000 7fa3eed9d640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa3f015c2d0 0x7fa3f00cb5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:53:28.999 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:28.996+0000 7fa3eed9d640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa3f015c2d0 0x7fa3f00cb5d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:58200/0 (socket says 172.21.15.43:58200) 2023-08-09T13:53:28.999 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:28.996+0000 7fa3eed9d640 1 -- 172.21.15.43:0/3669280011 learned_addr learned my addr 172.21.15.43:0/3669280011 (peer_addr_for_me v2:172.21.15.43:0/0) 2023-08-09T13:53:29.000 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:28.996+0000 7fa3ee59c640 1 --2- 172.21.15.43:0/3669280011 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fa3f00b7df0 0x7fa3f0157520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:53:29.000 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:28.996+0000 7fa3edd9b640 1 --2- 172.21.15.43:0/3669280011 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fa3f00cebf0 0x7fa3f00cb090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:53:29.000 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:28.996+0000 7fa3eed9d640 1 -- 172.21.15.43:0/3669280011 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fa3f00cebf0 msgr2=0x7fa3f00cb090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:53:29.000 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:28.996+0000 7fa3eed9d640 1 --2- 172.21.15.43:0/3669280011 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fa3f00cebf0 0x7fa3f00cb090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:53:29.000 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:28.996+0000 7fa3eed9d640 1 -- 172.21.15.43:0/3669280011 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fa3f00b7df0 msgr2=0x7fa3f0157520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:53:29.000 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:28.996+0000 7fa3eed9d640 1 --2- 172.21.15.43:0/3669280011 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fa3f00b7df0 0x7fa3f0157520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:53:29.000 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:28.997+0000 7fa3eed9d640 1 -- 172.21.15.43:0/3669280011 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa3e40094d0 con 0x7fa3f015c2d0 2023-08-09T13:53:29.000 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:28.997+0000 7fa3eed9d640 1 --2- 172.21.15.43:0/3669280011 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa3f015c2d0 0x7fa3f00cb5d0 secure :-1 s=READY pgs=1422 cs=0 l=1 rev1=1 crypto rx=0x7fa3e800d800 tx=0x7fa3e800dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-09T13:53:29.001 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:28.997+0000 7fa3deffd640 1 -- 172.21.15.43:0/3669280011 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa3e800b6c0 con 0x7fa3f015c2d0 2023-08-09T13:53:29.001 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:28.997+0000 7fa3deffd640 1 -- 172.21.15.43:0/3669280011 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa3e800b840 con 0x7fa3f015c2d0 2023-08-09T13:53:29.001 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:28.997+0000 7fa3ef59e640 1 -- 172.21.15.43:0/3669280011 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa3f015c850 con 0x7fa3f015c2d0 2023-08-09T13:53:29.001 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:28.997+0000 7fa3deffd640 1 -- 172.21.15.43:0/3669280011 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa3e800bbf0 con 0x7fa3f015c2d0 2023-08-09T13:53:29.001 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:28.997+0000 7fa3ef59e640 1 -- 172.21.15.43:0/3669280011 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa3f015ce10 con 0x7fa3f015c2d0 2023-08-09T13:53:29.002 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:28.999+0000 7fa3ef59e640 1 -- 172.21.15.43:0/3669280011 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa3f015c6b0 con 0x7fa3f015c2d0 2023-08-09T13:53:29.002 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:28.999+0000 7fa3deffd640 1 -- 172.21.15.43:0/3669280011 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 53) v1 ==== 88052+0+0 (secure 0 0 0) 0x7fa3e800bd50 con 0x7fa3f015c2d0 2023-08-09T13:53:29.002 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:29.000+0000 7fa3deffd640 1 -- 172.21.15.43:0/3669280011 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(118..118 src has 1..118) v4 ==== 7200+0+0 (secure 0 0 0) 0x7fa3e8088ab0 con 0x7fa3f015c2d0 2023-08-09T13:53:29.006 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:29.004+0000 7fa3deffd640 1 -- 172.21.15.43:0/3669280011 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178066 (secure 0 0 0) 0x7fa3f015c6b0 con 0x7fa3f015c2d0 2023-08-09T13:53:29.202 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:29.200+0000 7fa3ef59e640 1 -- 172.21.15.43:0/3669280011 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"} v 0) v1 -- 0x7fa3f015c6b0 con 0x7fa3f015c2d0 2023-08-09T13:53:29.202 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:29.201+0000 7fa3deffd640 1 -- 172.21.15.43:0/3669280011 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"}]=0 v118) v1 ==== 92+0+10 (secure 0 0 0) 0x7fa3f015c6b0 con 0x7fa3f015c2d0 2023-08-09T13:53:29.203 INFO:teuthology.orchestra.run.smithi043.stdout:pg_num: 8 2023-08-09T13:53:29.205 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:29.203+0000 7fa3ef59e640 1 -- 172.21.15.43:0/3669280011 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa3f015c2d0 msgr2=0x7fa3f00cb5d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:53:29.205 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:29.203+0000 7fa3ef59e640 1 --2- 172.21.15.43:0/3669280011 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa3f015c2d0 0x7fa3f00cb5d0 secure :-1 s=READY pgs=1422 cs=0 l=1 rev1=1 crypto rx=0x7fa3e800d800 tx=0x7fa3e800dcc0 comp rx=0 tx=0).stop 2023-08-09T13:53:29.205 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:29.204+0000 7fa3ef59e640 1 -- 172.21.15.43:0/3669280011 shutdown_connections 2023-08-09T13:53:29.206 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:29.204+0000 7fa3ef59e640 1 --2- 172.21.15.43:0/3669280011 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa3f015c2d0 0x7fa3f00cb5d0 unknown :-1 s=CLOSED pgs=1422 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:53:29.206 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:29.204+0000 7fa3ef59e640 1 --2- 172.21.15.43:0/3669280011 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fa3f00cebf0 0x7fa3f00cb090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:53:29.206 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:29.204+0000 7fa3ef59e640 1 --2- 172.21.15.43:0/3669280011 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fa3f00b7df0 0x7fa3f0157520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:53:29.206 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:29.204+0000 7fa3ef59e640 1 -- 172.21.15.43:0/3669280011 >> 172.21.15.43:0/3669280011 conn(0x7fa3f006a230 msgr2=0x7fa3f00b2790 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-09T13:53:29.206 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:29.204+0000 7fa3ef59e640 1 -- 172.21.15.43:0/3669280011 shutdown_connections 2023-08-09T13:53:29.206 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:29.204+0000 7fa3ef59e640 1 -- 172.21.15.43:0/3669280011 wait complete. 2023-08-09T13:53:29.225 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-08-09T13:53:29.459 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:29.457+0000 7f8a53d12640 1 Processor -- start 2023-08-09T13:53:29.460 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:29.457+0000 7f8a53d12640 1 -- start start 2023-08-09T13:53:29.461 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:29.459+0000 7f8a53d12640 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7f8a4c160e20 0x7f8a4c0cadd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:53:29.462 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:29.459+0000 7f8a53d12640 1 -- --> v1:172.21.15.43:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8a4c0cd0c0 con 0x7f8a4c0ce1c0 2023-08-09T13:53:29.462 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:29.459+0000 7f8a53d12640 1 -- --> v1:172.21.15.203:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8a4c161790 con 0x7f8a4c0c1620 2023-08-09T13:53:29.462 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:29.459+0000 7f8a53d12640 1 -- --> v2:172.21.15.203:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8a4c0cc290 con 0x7f8a4c160e20 2023-08-09T13:53:29.463 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:29.459+0000 7f8a51aae640 1 --1- >> v1:172.21.15.43:6789/0 conn(0x7f8a4c0ce1c0 0x7f8a4c0ce5a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.43:6789/0 says I am v1:172.21.15.43:40976/0 (socket says 172.21.15.43:40976) 2023-08-09T13:53:29.463 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:29.459+0000 7f8a51aae640 1 -- 172.21.15.43:0/1478283814 learned_addr learned my addr 172.21.15.43:0/1478283814 (peer_addr_for_me v1:172.21.15.43:0/0) 2023-08-09T13:53:29.463 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:29.460+0000 7f8a522af640 1 --2- 172.21.15.43:0/1478283814 >> v2:172.21.15.203:3300/0 conn(0x7f8a4c160e20 0x7f8a4c0cadd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:53:29.463 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:29.460+0000 7f8a50aac640 1 -- 172.21.15.43:0/1478283814 <== mon.0 v1:172.21.15.43:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3744949457 0 0) 0x7f8a4c0cd0c0 con 0x7f8a4c0ce1c0 2023-08-09T13:53:29.464 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:29.460+0000 7f8a50aac640 1 -- 172.21.15.43:0/1478283814 --> v1:172.21.15.43:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f8a38003520 con 0x7f8a4c0ce1c0 2023-08-09T13:53:29.469 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:29.460+0000 7f8a50aac640 1 -- 172.21.15.43:0/1478283814 <== mon.0 v1:172.21.15.43:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 334135476 0 0) 0x7f8a38003520 con 0x7f8a4c0ce1c0 2023-08-09T13:53:29.469 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:29.460+0000 7f8a50aac640 1 -- 172.21.15.43:0/1478283814 >> v2:172.21.15.203:3300/0 conn(0x7f8a4c160e20 msgr2=0x7f8a4c0cadd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-09T13:53:29.469 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:29.460+0000 7f8a50aac640 1 --2- 172.21.15.43:0/1478283814 >> v2:172.21.15.203:3300/0 conn(0x7f8a4c160e20 0x7f8a4c0cadd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:53:29.469 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:29.461+0000 7f8a50aac640 1 -- 172.21.15.43:0/1478283814 >> v1:172.21.15.203:6789/0 conn(0x7f8a4c0c1620 legacy=0x7f8a4c0cbb80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:53:29.469 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:29.461+0000 7f8a50aac640 1 -- 172.21.15.43:0/1478283814 --> v1:172.21.15.43:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8a4c161a70 con 0x7f8a4c0ce1c0 2023-08-09T13:53:29.470 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:29.461+0000 7f8a50aac640 1 -- 172.21.15.43:0/1478283814 <== mon.0 v1:172.21.15.43:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1285150740 0 0) 0x7f8a48001670 con 0x7f8a4c0ce1c0 2023-08-09T13:53:29.470 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:29.461+0000 7f8a50aac640 1 -- 172.21.15.43:0/1478283814 >> v1:172.21.15.43:6789/0 conn(0x7f8a4c0ce1c0 legacy=0x7f8a4c0ce5a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:53:29.470 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:29.461+0000 7f8a50aac640 1 --2- 172.21.15.43:0/1478283814 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f8a38003970 0x7f8a38003d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:53:29.470 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:29.461+0000 7f8a50aac640 1 --2- 172.21.15.43:0/1478283814 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f8a380047c0 0x7f8a38004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:53:29.470 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:29.461+0000 7f8a50aac640 1 --2- 172.21.15.43:0/1478283814 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f8a38005180 0x7f8a3800b980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:53:29.470 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:29.461+0000 7f8a50aac640 1 -- 172.21.15.43:0/1478283814 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8a4c0cc290 con 0x7f8a380047c0 2023-08-09T13:53:29.470 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:29.461+0000 7f8a50aac640 1 -- 172.21.15.43:0/1478283814 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8a3800c120 con 0x7f8a38005180 2023-08-09T13:53:29.471 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:29.461+0000 7f8a50aac640 1 -- 172.21.15.43:0/1478283814 --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8a3800c2e0 con 0x7f8a38003970 2023-08-09T13:53:29.471 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:29.461+0000 7f8a512ad640 1 --2- 172.21.15.43:0/1478283814 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f8a380047c0 0x7f8a38004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:53:29.471 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:29.461+0000 7f8a512ad640 1 -- 172.21.15.43:0/1478283814 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f8a38003970 msgr2=0x7f8a38003d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-09T13:53:29.471 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:29.461+0000 7f8a512ad640 1 --2- 172.21.15.43:0/1478283814 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f8a38003970 0x7f8a38003d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:53:29.471 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:29.461+0000 7f8a512ad640 1 -- 172.21.15.43:0/1478283814 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f8a38005180 msgr2=0x7f8a3800b980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-09T13:53:29.471 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:29.461+0000 7f8a512ad640 1 --2- 172.21.15.43:0/1478283814 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f8a38005180 0x7f8a3800b980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:53:29.471 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:29.461+0000 7f8a512ad640 1 -- 172.21.15.43:0/1478283814 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8a38003520 con 0x7f8a380047c0 2023-08-09T13:53:29.471 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:29.461+0000 7f8a522af640 1 -- 172.21.15.43:0/1478283814 reap_dead start 2023-08-09T13:53:29.472 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:29.463+0000 7f8a512ad640 1 --2- 172.21.15.43:0/1478283814 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f8a380047c0 0x7f8a38004c40 secure :-1 s=READY pgs=1424 cs=0 l=1 rev1=1 crypto rx=0x7f8a3c001720 tx=0x7f8a380057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9a15fe73cb935321 server_cookie=0 in_seq=0 out_seq=0 2023-08-09T13:53:29.472 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:29.463+0000 7f8a50aac640 1 -- 172.21.15.43:0/1478283814 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8a3c0052d0 con 0x7f8a380047c0 2023-08-09T13:53:29.472 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:29.463+0000 7f8a50aac640 1 -- 172.21.15.43:0/1478283814 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8a3c02e050 con 0x7f8a380047c0 2023-08-09T13:53:29.472 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:29.463+0000 7f8a50aac640 1 -- 172.21.15.43:0/1478283814 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8a3c003830 con 0x7f8a380047c0 2023-08-09T13:53:29.472 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:29.463+0000 7f8a53d12640 1 -- 172.21.15.43:0/1478283814 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f8a380047c0 msgr2=0x7f8a38004c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:53:29.472 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:29.463+0000 7f8a53d12640 1 --2- 172.21.15.43:0/1478283814 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f8a380047c0 0x7f8a38004c40 secure :-1 s=READY pgs=1424 cs=0 l=1 rev1=1 crypto rx=0x7f8a3c001720 tx=0x7f8a380057f0 comp rx=0 tx=0).stop 2023-08-09T13:53:29.472 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:29.464+0000 7f8a53d12640 1 -- 172.21.15.43:0/1478283814 shutdown_connections 2023-08-09T13:53:29.473 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:29.464+0000 7f8a53d12640 1 --2- 172.21.15.43:0/1478283814 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f8a380047c0 0x7f8a38004c40 unknown :-1 s=CLOSED pgs=1424 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:53:29.473 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:29.464+0000 7f8a53d12640 1 -- 172.21.15.43:0/1478283814 >> 172.21.15.43:0/1478283814 conn(0x7f8a4c0aff50 msgr2=0x7f8a4c0b0360 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-09T13:53:29.473 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:29.464+0000 7f8a53d12640 1 -- 172.21.15.43:0/1478283814 shutdown_connections 2023-08-09T13:53:29.473 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:29.464+0000 7f8a53d12640 1 -- 172.21.15.43:0/1478283814 wait complete. 2023-08-09T13:53:29.473 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:29.465+0000 7f8a53d12640 1 Processor -- start 2023-08-09T13:53:29.473 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:29.466+0000 7f8a53d12640 1 -- start start 2023-08-09T13:53:29.473 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:29.466+0000 7f8a53d12640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f8a380047c0 0x7f8a4c15d960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:53:29.473 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:29.466+0000 7f8a53d12640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f8a4c0b0c50 0x7f8a4c0c5ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:53:29.474 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:29.466+0000 7f8a53d12640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f8a4c0c6410 0x7f8a4c162900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:53:29.474 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:29.466+0000 7f8a53d12640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8a4c15dea0 con 0x7f8a4c0b0c50 2023-08-09T13:53:29.474 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:29.466+0000 7f8a53d12640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8a4c0c3940 con 0x7f8a380047c0 2023-08-09T13:53:29.474 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:29.466+0000 7f8a53d12640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8a4c0c6890 con 0x7f8a4c0c6410 2023-08-09T13:53:29.474 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:29.467+0000 7f8a512ad640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f8a4c0b0c50 0x7f8a4c0c5ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:53:29.474 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:29.467+0000 7f8a512ad640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f8a4c0b0c50 0x7f8a4c0c5ed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:58224/0 (socket says 172.21.15.43:58224) 2023-08-09T13:53:29.474 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:29.467+0000 7f8a512ad640 1 -- 172.21.15.43:0/2075616711 learned_addr learned my addr 172.21.15.43:0/2075616711 (peer_addr_for_me v2:172.21.15.43:0/0) 2023-08-09T13:53:29.474 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:29.467+0000 7f8a522af640 1 --2- 172.21.15.43:0/2075616711 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f8a4c0c6410 0x7f8a4c162900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:53:29.475 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:29.467+0000 7f8a51aae640 1 --2- 172.21.15.43:0/2075616711 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f8a380047c0 0x7f8a4c15d960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:53:29.475 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:29.467+0000 7f8a512ad640 1 -- 172.21.15.43:0/2075616711 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f8a4c0c6410 msgr2=0x7f8a4c162900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:53:29.475 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:29.467+0000 7f8a512ad640 1 --2- 172.21.15.43:0/2075616711 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f8a4c0c6410 0x7f8a4c162900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:53:29.475 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:29.467+0000 7f8a512ad640 1 -- 172.21.15.43:0/2075616711 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f8a380047c0 msgr2=0x7f8a4c15d960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:53:29.475 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:29.467+0000 7f8a512ad640 1 --2- 172.21.15.43:0/2075616711 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f8a380047c0 0x7f8a4c15d960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:53:29.475 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:29.467+0000 7f8a512ad640 1 -- 172.21.15.43:0/2075616711 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8a4c0c3010 con 0x7f8a4c0b0c50 2023-08-09T13:53:29.475 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:29.467+0000 7f8a51aae640 1 --2- 172.21.15.43:0/2075616711 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f8a380047c0 0x7f8a4c15d960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-08-09T13:53:29.476 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:29.468+0000 7f8a512ad640 1 --2- 172.21.15.43:0/2075616711 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f8a4c0b0c50 0x7f8a4c0c5ed0 secure :-1 s=READY pgs=1425 cs=0 l=1 rev1=1 crypto rx=0x7f8a3c02bbc0 tx=0x7f8a3c001120 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-09T13:53:29.477 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:29.468+0000 7f8a427fc640 1 -- 172.21.15.43:0/2075616711 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8a3c0076c0 con 0x7f8a4c0b0c50 2023-08-09T13:53:29.478 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:29.468+0000 7f8a427fc640 1 -- 172.21.15.43:0/2075616711 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8a3c007a30 con 0x7f8a4c0b0c50 2023-08-09T13:53:29.478 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:29.468+0000 7f8a53d12640 1 -- 172.21.15.43:0/2075616711 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8a4c0c32f0 con 0x7f8a4c0b0c50 2023-08-09T13:53:29.478 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:29.468+0000 7f8a427fc640 1 -- 172.21.15.43:0/2075616711 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8a3c006be0 con 0x7f8a4c0b0c50 2023-08-09T13:53:29.479 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:29.468+0000 7f8a53d12640 1 -- 172.21.15.43:0/2075616711 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8a4c0b10b0 con 0x7f8a4c0b0c50 2023-08-09T13:53:29.479 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:29.469+0000 7f8a53d12640 1 -- 172.21.15.43:0/2075616711 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8a4c0b1240 con 0x7f8a4c0b0c50 2023-08-09T13:53:29.479 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:29.470+0000 7f8a427fc640 1 -- 172.21.15.43:0/2075616711 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 54) v1 ==== 88179+0+0 (secure 0 0 0) 0x7f8a3c006d40 con 0x7f8a4c0b0c50 2023-08-09T13:53:29.479 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:29.471+0000 7f8a427fc640 1 --2- 172.21.15.43:0/2075616711 >> [v2:172.21.15.203:6832/2527059680,v1:172.21.15.203:6833/2527059680] conn(0x7f8a30069ea0 0x7f8a3006c360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:53:29.480 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:29.471+0000 7f8a427fc640 1 -- 172.21.15.43:0/2075616711 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(118..118 src has 1..118) v4 ==== 7200+0+0 (secure 0 0 0) 0x7f8a3c0aa400 con 0x7f8a4c0b0c50 2023-08-09T13:53:29.480 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:29.472+0000 7f8a51aae640 1 --2- 172.21.15.43:0/2075616711 >> [v2:172.21.15.203:6832/2527059680,v1:172.21.15.203:6833/2527059680] conn(0x7f8a30069ea0 0x7f8a3006c360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:53:29.480 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:29.473+0000 7f8a51aae640 1 --2- 172.21.15.43:0/2075616711 >> [v2:172.21.15.203:6832/2527059680,v1:172.21.15.203:6833/2527059680] conn(0x7f8a30069ea0 0x7f8a3006c360 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7f8a4c165580 tx=0x7f8a48009040 comp rx=0 tx=0).ready entity=mgr.6377 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-09T13:53:29.481 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:29.476+0000 7f8a427fc640 1 -- 172.21.15.43:0/2075616711 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178066 (secure 0 0 0) 0x7f8a3c0793a0 con 0x7f8a4c0b0c50 2023-08-09T13:53:29.662 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:29.661+0000 7f8a53d12640 1 -- 172.21.15.43:0/2075616711 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f8a4c0739f0 con 0x7f8a4c0b0c50 2023-08-09T13:53:29.663 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:29.661+0000 7f8a427fc640 1 -- 172.21.15.43:0/2075616711 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v118) v1 ==== 93+0+10 (secure 0 0 0) 0x7f8a3c078f30 con 0x7f8a4c0b0c50 2023-08-09T13:53:29.663 INFO:teuthology.orchestra.run.smithi043.stdout:pg_num: 1 2023-08-09T13:53:29.666 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:29.664+0000 7f8a53d12640 1 -- 172.21.15.43:0/2075616711 >> [v2:172.21.15.203:6832/2527059680,v1:172.21.15.203:6833/2527059680] conn(0x7f8a30069ea0 msgr2=0x7f8a3006c360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:53:29.666 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:29.664+0000 7f8a53d12640 1 --2- 172.21.15.43:0/2075616711 >> [v2:172.21.15.203:6832/2527059680,v1:172.21.15.203:6833/2527059680] conn(0x7f8a30069ea0 0x7f8a3006c360 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7f8a4c165580 tx=0x7f8a48009040 comp rx=0 tx=0).stop 2023-08-09T13:53:29.666 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:29.664+0000 7f8a53d12640 1 -- 172.21.15.43:0/2075616711 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f8a4c0b0c50 msgr2=0x7f8a4c0c5ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:53:29.666 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:29.664+0000 7f8a53d12640 1 --2- 172.21.15.43:0/2075616711 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f8a4c0b0c50 0x7f8a4c0c5ed0 secure :-1 s=READY pgs=1425 cs=0 l=1 rev1=1 crypto rx=0x7f8a3c02bbc0 tx=0x7f8a3c001120 comp rx=0 tx=0).stop 2023-08-09T13:53:29.666 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:29.664+0000 7f8a53d12640 1 -- 172.21.15.43:0/2075616711 shutdown_connections 2023-08-09T13:53:29.667 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:29.664+0000 7f8a53d12640 1 --2- 172.21.15.43:0/2075616711 >> [v2:172.21.15.203:6832/2527059680,v1:172.21.15.203:6833/2527059680] conn(0x7f8a30069ea0 0x7f8a3006c360 unknown :-1 s=CLOSED pgs=20 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:53:29.667 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:29.664+0000 7f8a53d12640 1 --2- 172.21.15.43:0/2075616711 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f8a4c0c6410 0x7f8a4c162900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:53:29.667 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:29.664+0000 7f8a53d12640 1 --2- 172.21.15.43:0/2075616711 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f8a4c0b0c50 0x7f8a4c0c5ed0 unknown :-1 s=CLOSED pgs=1425 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:53:29.667 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:29.664+0000 7f8a53d12640 1 --2- 172.21.15.43:0/2075616711 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f8a380047c0 0x7f8a4c15d960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:53:29.667 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:29.664+0000 7f8a53d12640 1 -- 172.21.15.43:0/2075616711 >> 172.21.15.43:0/2075616711 conn(0x7f8a4c0aff50 msgr2=0x7f8a4c0b0330 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-09T13:53:29.667 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:29.664+0000 7f8a53d12640 1 -- 172.21.15.43:0/2075616711 shutdown_connections 2023-08-09T13:53:29.668 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:29.664+0000 7f8a53d12640 1 -- 172.21.15.43:0/2075616711 wait complete. 2023-08-09T13:53:29.683 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-08-09T13:53:29.916 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:29.914+0000 7fd2db4c5640 1 Processor -- start 2023-08-09T13:53:29.916 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:29.914+0000 7fd2db4c5640 1 -- start start 2023-08-09T13:53:29.917 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:29.916+0000 7fd2db4c5640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7fd2d40d0a70 0x7fd2d40cd7f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:53:29.918 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:29.916+0000 7fd2db4c5640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fd2d40cdd30 0x7fd2d40b3fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:53:29.918 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:29.916+0000 7fd2db4c5640 1 -- --> v1:172.21.15.43:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd2d40d0f70 con 0x7fd2d415c680 2023-08-09T13:53:29.918 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:29.916+0000 7fd2db4c5640 1 -- --> v2:172.21.15.43:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd2d40ccd10 con 0x7fd2d40d0a70 2023-08-09T13:53:29.918 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:29.916+0000 7fd2db4c5640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd2d40ccff0 con 0x7fd2d40cdd30 2023-08-09T13:53:29.918 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:29.916+0000 7fd2d9261640 1 --1- >> v1:172.21.15.43:6789/0 conn(0x7fd2d415c680 0x7fd2d40ce760 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.43:6789/0 says I am v1:172.21.15.43:40992/0 (socket says 172.21.15.43:40992) 2023-08-09T13:53:29.918 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:29.916+0000 7fd2d9261640 1 -- 172.21.15.43:0/3091690602 learned_addr learned my addr 172.21.15.43:0/3091690602 (peer_addr_for_me v1:172.21.15.43:0/0) 2023-08-09T13:53:29.924 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:29.916+0000 7fd2d8a60640 1 --2- 172.21.15.43:0/3091690602 >> v2:172.21.15.43:3300/0 conn(0x7fd2d40d0a70 0x7fd2d40cd7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:53:29.924 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:29.917+0000 7fd2d9a62640 1 --2- 172.21.15.43:0/3091690602 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fd2d40cdd30 0x7fd2d40b3fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:53:29.925 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:29.917+0000 7fd2cbfff640 1 -- 172.21.15.43:0/3091690602 <== mon.0 v1:172.21.15.43:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2411891401 0 0) 0x7fd2d40d0f70 con 0x7fd2d415c680 2023-08-09T13:53:29.925 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:29.917+0000 7fd2cbfff640 1 -- 172.21.15.43:0/3091690602 --> v1:172.21.15.43:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fd2c0003520 con 0x7fd2d415c680 2023-08-09T13:53:29.925 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:29.917+0000 7fd2d8a60640 1 -- 172.21.15.43:0/3091690602 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fd2d40cdd30 msgr2=0x7fd2d40b3fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-09T13:53:29.925 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:29.917+0000 7fd2d8a60640 1 --2- 172.21.15.43:0/3091690602 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fd2d40cdd30 0x7fd2d40b3fa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:53:29.925 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:29.917+0000 7fd2d8a60640 1 -- 172.21.15.43:0/3091690602 >> v1:172.21.15.43:6789/0 conn(0x7fd2d415c680 legacy=0x7fd2d40ce760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:53:29.925 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:29.917+0000 7fd2d8a60640 1 -- 172.21.15.43:0/3091690602 --> v2:172.21.15.43:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd2d40cd2d0 con 0x7fd2d40d0a70 2023-08-09T13:53:29.925 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:29.917+0000 7fd2d9a62640 1 --2- 172.21.15.43:0/3091690602 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fd2d40cdd30 0x7fd2d40b3fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-08-09T13:53:29.926 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:29.918+0000 7fd2d8a60640 1 --2- 172.21.15.43:0/3091690602 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fd2d40d0a70 0x7fd2d40cd7f0 secure :-1 s=READY pgs=1427 cs=0 l=1 rev1=1 crypto rx=0x7fd2c40097a0 tx=0x7fd2c402b8b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1eaed2e1d472fc74 server_cookie=0 in_seq=0 out_seq=0 2023-08-09T13:53:29.926 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:29.919+0000 7fd2cbfff640 1 -- 172.21.15.43:0/3091690602 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd2c402d040 con 0x7fd2d40d0a70 2023-08-09T13:53:29.926 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:29.919+0000 7fd2cbfff640 1 -- 172.21.15.43:0/3091690602 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd2c4031810 con 0x7fd2d40d0a70 2023-08-09T13:53:29.926 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:29.919+0000 7fd2cbfff640 1 -- 172.21.15.43:0/3091690602 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd2c4031b80 con 0x7fd2d40d0a70 2023-08-09T13:53:29.926 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:29.919+0000 7fd2db4c5640 1 -- 172.21.15.43:0/3091690602 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fd2d40d0a70 msgr2=0x7fd2d40cd7f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:53:29.926 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:29.919+0000 7fd2db4c5640 1 --2- 172.21.15.43:0/3091690602 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fd2d40d0a70 0x7fd2d40cd7f0 secure :-1 s=READY pgs=1427 cs=0 l=1 rev1=1 crypto rx=0x7fd2c40097a0 tx=0x7fd2c402b8b0 comp rx=0 tx=0).stop 2023-08-09T13:53:29.926 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:29.920+0000 7fd2db4c5640 1 -- 172.21.15.43:0/3091690602 shutdown_connections 2023-08-09T13:53:29.927 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:29.920+0000 7fd2db4c5640 1 --2- 172.21.15.43:0/3091690602 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fd2d40cdd30 0x7fd2d40b3fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:53:29.927 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:29.920+0000 7fd2db4c5640 1 --2- 172.21.15.43:0/3091690602 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fd2d40d0a70 0x7fd2d40cd7f0 unknown :-1 s=CLOSED pgs=1427 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:53:29.927 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:29.920+0000 7fd2db4c5640 1 -- 172.21.15.43:0/3091690602 >> 172.21.15.43:0/3091690602 conn(0x7fd2d4071520 msgr2=0x7fd2d40d2520 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-09T13:53:29.927 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:29.920+0000 7fd2db4c5640 1 -- 172.21.15.43:0/3091690602 shutdown_connections 2023-08-09T13:53:29.927 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:29.920+0000 7fd2db4c5640 1 -- 172.21.15.43:0/3091690602 wait complete. 2023-08-09T13:53:29.927 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:29.922+0000 7fd2db4c5640 1 Processor -- start 2023-08-09T13:53:29.927 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:29.922+0000 7fd2db4c5640 1 -- start start 2023-08-09T13:53:29.928 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:29.923+0000 7fd2db4c5640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fd2d40d0a70 0x7fd2d415f4f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:53:29.928 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:29.923+0000 7fd2db4c5640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fd2d415c680 0x7fd2d4158b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:53:29.928 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:29.923+0000 7fd2db4c5640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fd2d40cbab0 0x7fd2d40c8260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:53:29.928 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:29.923+0000 7fd2db4c5640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd2d415ed10 con 0x7fd2d40cbab0 2023-08-09T13:53:29.928 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:29.923+0000 7fd2db4c5640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd2d415fa30 con 0x7fd2d40d0a70 2023-08-09T13:53:29.928 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:29.923+0000 7fd2db4c5640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd2d415e470 con 0x7fd2d415c680 2023-08-09T13:53:29.931 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:29.923+0000 7fd2d9a62640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fd2d40cbab0 0x7fd2d40c8260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:53:29.931 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:29.923+0000 7fd2d9a62640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fd2d40cbab0 0x7fd2d40c8260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:58256/0 (socket says 172.21.15.43:58256) 2023-08-09T13:53:29.931 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:29.923+0000 7fd2d9a62640 1 -- 172.21.15.43:0/396748199 learned_addr learned my addr 172.21.15.43:0/396748199 (peer_addr_for_me v2:172.21.15.43:0/0) 2023-08-09T13:53:29.931 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:29.923+0000 7fd2d9a62640 1 -- 172.21.15.43:0/396748199 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fd2d415c680 msgr2=0x7fd2d4158b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:53:29.932 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:29.923+0000 7fd2d9a62640 1 --2- 172.21.15.43:0/396748199 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fd2d415c680 0x7fd2d4158b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:53:29.932 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:29.924+0000 7fd2d9a62640 1 -- 172.21.15.43:0/396748199 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fd2d40d0a70 msgr2=0x7fd2d415f4f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:53:29.932 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:29.924+0000 7fd2d9261640 1 --2- 172.21.15.43:0/396748199 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fd2d40d0a70 0x7fd2d415f4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:53:29.932 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:29.924+0000 7fd2d9a62640 1 --2- 172.21.15.43:0/396748199 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fd2d40d0a70 0x7fd2d415f4f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:53:29.932 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:29.924+0000 7fd2d9a62640 1 -- 172.21.15.43:0/396748199 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd2c4009400 con 0x7fd2d40cbab0 2023-08-09T13:53:29.932 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:29.924+0000 7fd2d9261640 1 --2- 172.21.15.43:0/396748199 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fd2d40d0a70 0x7fd2d415f4f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-08-09T13:53:29.932 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:29.924+0000 7fd2d9a62640 1 --2- 172.21.15.43:0/396748199 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fd2d40cbab0 0x7fd2d40c8260 secure :-1 s=READY pgs=1428 cs=0 l=1 rev1=1 crypto rx=0x7fd2cc007ea0 tx=0x7fd2cc007f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-09T13:53:29.933 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:29.924+0000 7fd2c9ffb640 1 -- 172.21.15.43:0/396748199 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd2cc015600 con 0x7fd2d40cbab0 2023-08-09T13:53:29.933 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:29.924+0000 7fd2c9ffb640 1 -- 172.21.15.43:0/396748199 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd2cc00f040 con 0x7fd2d40cbab0 2023-08-09T13:53:29.933 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:29.924+0000 7fd2c9ffb640 1 -- 172.21.15.43:0/396748199 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd2cc0159a0 con 0x7fd2d40cbab0 2023-08-09T13:53:29.933 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:29.924+0000 7fd2db4c5640 1 -- 172.21.15.43:0/396748199 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd2d415e750 con 0x7fd2d40cbab0 2023-08-09T13:53:29.933 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:29.925+0000 7fd2db4c5640 1 -- 172.21.15.43:0/396748199 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd2d40cc070 con 0x7fd2d40cbab0 2023-08-09T13:53:29.933 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:29.926+0000 7fd2db4c5640 1 -- 172.21.15.43:0/396748199 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd2d40cc260 con 0x7fd2d40cbab0 2023-08-09T13:53:29.934 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:29.927+0000 7fd2c9ffb640 1 -- 172.21.15.43:0/396748199 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 54) v1 ==== 88179+0+0 (secure 0 0 0) 0x7fd2cc01e3b0 con 0x7fd2d40cbab0 2023-08-09T13:53:29.934 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:29.927+0000 7fd2c9ffb640 1 --2- 172.21.15.43:0/396748199 >> [v2:172.21.15.203:6832/2527059680,v1:172.21.15.203:6833/2527059680] conn(0x7fd2b8069ed0 0x7fd2b806c390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:53:29.934 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:29.928+0000 7fd2c9ffb640 1 -- 172.21.15.43:0/396748199 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(118..118 src has 1..118) v4 ==== 7200+0+0 (secure 0 0 0) 0x7fd2cc088a10 con 0x7fd2d40cbab0 2023-08-09T13:53:29.934 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:29.928+0000 7fd2d9261640 1 --2- 172.21.15.43:0/396748199 >> [v2:172.21.15.203:6832/2527059680,v1:172.21.15.203:6833/2527059680] conn(0x7fd2b8069ed0 0x7fd2b806c390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:53:29.934 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:29.930+0000 7fd2d9261640 1 --2- 172.21.15.43:0/396748199 >> [v2:172.21.15.203:6832/2527059680,v1:172.21.15.203:6833/2527059680] conn(0x7fd2b8069ed0 0x7fd2b806c390 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7fd2d0005880 tx=0x7fd2d0009040 comp rx=0 tx=0).ready entity=mgr.6377 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-09T13:53:29.934 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:29.932+0000 7fd2c9ffb640 1 -- 172.21.15.43:0/396748199 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178066 (secure 0 0 0) 0x7fd2cc053f20 con 0x7fd2d40cbab0 2023-08-09T13:53:30.139 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:30.138+0000 7fd2db4c5640 1 -- 172.21.15.43:0/396748199 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7fd2d40739f0 con 0x7fd2d40cbab0 2023-08-09T13:53:30.140 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:30.139+0000 7fd2c9ffb640 1 -- 172.21.15.43:0/396748199 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v118) v1 ==== 104+0+11 (secure 0 0 0) 0x7fd2cc057540 con 0x7fd2d40cbab0 2023-08-09T13:53:30.140 INFO:teuthology.orchestra.run.smithi043.stdout:pg_num: 32 2023-08-09T13:53:30.144 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:30.141+0000 7fd2db4c5640 1 -- 172.21.15.43:0/396748199 >> [v2:172.21.15.203:6832/2527059680,v1:172.21.15.203:6833/2527059680] conn(0x7fd2b8069ed0 msgr2=0x7fd2b806c390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:53:30.144 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:30.141+0000 7fd2db4c5640 1 --2- 172.21.15.43:0/396748199 >> [v2:172.21.15.203:6832/2527059680,v1:172.21.15.203:6833/2527059680] conn(0x7fd2b8069ed0 0x7fd2b806c390 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7fd2d0005880 tx=0x7fd2d0009040 comp rx=0 tx=0).stop 2023-08-09T13:53:30.144 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:30.141+0000 7fd2db4c5640 1 -- 172.21.15.43:0/396748199 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fd2d40cbab0 msgr2=0x7fd2d40c8260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:53:30.145 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:30.141+0000 7fd2db4c5640 1 --2- 172.21.15.43:0/396748199 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fd2d40cbab0 0x7fd2d40c8260 secure :-1 s=READY pgs=1428 cs=0 l=1 rev1=1 crypto rx=0x7fd2cc007ea0 tx=0x7fd2cc007f90 comp rx=0 tx=0).stop 2023-08-09T13:53:30.145 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:30.141+0000 7fd2db4c5640 1 -- 172.21.15.43:0/396748199 shutdown_connections 2023-08-09T13:53:30.145 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:30.141+0000 7fd2db4c5640 1 --2- 172.21.15.43:0/396748199 >> [v2:172.21.15.203:6832/2527059680,v1:172.21.15.203:6833/2527059680] conn(0x7fd2b8069ed0 0x7fd2b806c390 unknown :-1 s=CLOSED pgs=22 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:53:30.145 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:30.141+0000 7fd2db4c5640 1 --2- 172.21.15.43:0/396748199 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fd2d40cbab0 0x7fd2d40c8260 unknown :-1 s=CLOSED pgs=1428 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:53:30.146 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:30.141+0000 7fd2db4c5640 1 --2- 172.21.15.43:0/396748199 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fd2d415c680 0x7fd2d4158b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:53:30.146 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:30.141+0000 7fd2db4c5640 1 --2- 172.21.15.43:0/396748199 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fd2d40d0a70 0x7fd2d415f4f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:53:30.146 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:30.141+0000 7fd2db4c5640 1 -- 172.21.15.43:0/396748199 >> 172.21.15.43:0/396748199 conn(0x7fd2d4071520 msgr2=0x7fd2d40ce120 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-09T13:53:30.147 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:30.142+0000 7fd2db4c5640 1 -- 172.21.15.43:0/396748199 shutdown_connections 2023-08-09T13:53:30.147 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:30.142+0000 7fd2db4c5640 1 -- 172.21.15.43:0/396748199 wait complete. 2023-08-09T13:53:30.164 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-08-09T13:53:30.400 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:30.398+0000 7f2c22f12640 1 Processor -- start 2023-08-09T13:53:30.400 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:30.398+0000 7f2c22f12640 1 -- start start 2023-08-09T13:53:30.401 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:30.400+0000 7f2c22f12640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f2c1c15a0c0 0x7f2c1c0cbb80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:53:30.401 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:30.400+0000 7f2c22f12640 1 -- --> v1:172.21.15.43:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2c1c15a4a0 con 0x7f2c1c15d510 2023-08-09T13:53:30.401 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:30.400+0000 7f2c22f12640 1 -- --> v1:172.21.15.203:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2c1c0cc2d0 con 0x7f2c1c0ce1c0 2023-08-09T13:53:30.402 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:30.400+0000 7f2c22f12640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2c1c0cc0c0 con 0x7f2c1c15a0c0 2023-08-09T13:53:30.403 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:30.400+0000 7f2c1bfff640 1 --1- >> v1:172.21.15.43:6789/0 conn(0x7f2c1c15d510 0x7f2c1c0cb2b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.43:6789/0 says I am v1:172.21.15.43:40994/0 (socket says 172.21.15.43:40994) 2023-08-09T13:53:30.403 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:30.400+0000 7f2c1bfff640 1 -- 172.21.15.43:0/408459463 learned_addr learned my addr 172.21.15.43:0/408459463 (peer_addr_for_me v1:172.21.15.43:0/0) 2023-08-09T13:53:30.403 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:30.401+0000 7f2c20cae640 1 --2- 172.21.15.43:0/408459463 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f2c1c15a0c0 0x7f2c1c0cbb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:53:30.403 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:30.401+0000 7f2c1b7fe640 1 -- 172.21.15.43:0/408459463 <== mon.0 v1:172.21.15.43:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2210869261 0 0) 0x7f2c1c15a4a0 con 0x7f2c1c15d510 2023-08-09T13:53:30.403 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:30.401+0000 7f2c1b7fe640 1 -- 172.21.15.43:0/408459463 --> v1:172.21.15.43:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f2c0c003520 con 0x7f2c1c15d510 2023-08-09T13:53:30.404 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:30.401+0000 7f2c1b7fe640 1 -- 172.21.15.43:0/408459463 <== mon.0 v1:172.21.15.43:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3475190727 0 0) 0x7f2c0c003520 con 0x7f2c1c15d510 2023-08-09T13:53:30.404 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:30.401+0000 7f2c1b7fe640 1 -- 172.21.15.43:0/408459463 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f2c1c15a0c0 msgr2=0x7f2c1c0cbb80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-09T13:53:30.404 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:30.401+0000 7f2c1b7fe640 1 --2- 172.21.15.43:0/408459463 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f2c1c15a0c0 0x7f2c1c0cbb80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:53:30.404 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:30.401+0000 7f2c1b7fe640 1 -- 172.21.15.43:0/408459463 >> v1:172.21.15.203:6789/0 conn(0x7f2c1c0ce1c0 legacy=0x7f2c1c0c8600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:53:30.404 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:30.401+0000 7f2c1b7fe640 1 -- 172.21.15.43:0/408459463 --> v1:172.21.15.43:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2c1c0cc5b0 con 0x7f2c1c15d510 2023-08-09T13:53:30.404 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:30.401+0000 7f2c1b7fe640 1 -- 172.21.15.43:0/408459463 <== mon.0 v1:172.21.15.43:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1285150740 0 0) 0x7f2c08001670 con 0x7f2c1c15d510 2023-08-09T13:53:30.404 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:30.401+0000 7f2c1b7fe640 1 -- 172.21.15.43:0/408459463 >> v1:172.21.15.43:6789/0 conn(0x7f2c1c15d510 legacy=0x7f2c1c0cb2b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:53:30.404 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:30.401+0000 7f2c1b7fe640 1 --2- 172.21.15.43:0/408459463 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f2c0c0039a0 0x7f2c0c003dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:53:30.405 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:30.401+0000 7f2c1b7fe640 1 --2- 172.21.15.43:0/408459463 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f2c0c0047c0 0x7f2c0c004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:53:30.405 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:30.401+0000 7f2c1b7fe640 1 --2- 172.21.15.43:0/408459463 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f2c1c15a0c0 0x7f2c0c0051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:53:30.405 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:30.401+0000 7f2c1b7fe640 1 -- 172.21.15.43:0/408459463 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2c1c0cc0c0 con 0x7f2c0c0039a0 2023-08-09T13:53:30.405 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:30.401+0000 7f2c1b7fe640 1 -- 172.21.15.43:0/408459463 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2c0c00bab0 con 0x7f2c0c0047c0 2023-08-09T13:53:30.405 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:30.401+0000 7f2c1b7fe640 1 -- 172.21.15.43:0/408459463 --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2c0c00bc70 con 0x7f2c1c15a0c0 2023-08-09T13:53:30.405 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:30.402+0000 7f2c20cae640 1 --2- 172.21.15.43:0/408459463 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f2c0c0039a0 0x7f2c0c003dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:53:30.407 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:30.402+0000 7f2c1bfff640 1 --2- 172.21.15.43:0/408459463 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f2c0c0047c0 0x7f2c0c004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:53:30.407 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:30.402+0000 7f2c214af640 1 --2- 172.21.15.43:0/408459463 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f2c1c15a0c0 0x7f2c0c0051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:53:30.407 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:30.402+0000 7f2c20cae640 1 -- 172.21.15.43:0/408459463 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f2c1c15a0c0 msgr2=0x7f2c0c0051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-09T13:53:30.407 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:30.402+0000 7f2c20cae640 1 --2- 172.21.15.43:0/408459463 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f2c1c15a0c0 0x7f2c0c0051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:53:30.407 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:30.402+0000 7f2c20cae640 1 -- 172.21.15.43:0/408459463 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f2c0c0047c0 msgr2=0x7f2c0c004c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-09T13:53:30.407 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:30.402+0000 7f2c20cae640 1 --2- 172.21.15.43:0/408459463 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f2c0c0047c0 0x7f2c0c004c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:53:30.407 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:30.402+0000 7f2c20cae640 1 -- 172.21.15.43:0/408459463 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2c0c003520 con 0x7f2c0c0039a0 2023-08-09T13:53:30.408 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:30.403+0000 7f2c20cae640 1 --2- 172.21.15.43:0/408459463 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f2c0c0039a0 0x7f2c0c003dc0 secure :-1 s=READY pgs=1430 cs=0 l=1 rev1=1 crypto rx=0x7f2c1000ba30 tx=0x7f2c0c0057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e365774a276314c7 server_cookie=0 in_seq=0 out_seq=0 2023-08-09T13:53:30.408 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:30.404+0000 7f2c1b7fe640 1 -- 172.21.15.43:0/408459463 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2c10032640 con 0x7f2c0c0039a0 2023-08-09T13:53:30.408 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:30.404+0000 7f2c1b7fe640 1 -- 172.21.15.43:0/408459463 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2c100327a0 con 0x7f2c0c0039a0 2023-08-09T13:53:30.408 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:30.404+0000 7f2c1b7fe640 1 -- 172.21.15.43:0/408459463 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2c10032b30 con 0x7f2c0c0039a0 2023-08-09T13:53:30.408 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:30.404+0000 7f2c22f12640 1 -- 172.21.15.43:0/408459463 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f2c0c0039a0 msgr2=0x7f2c0c003dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:53:30.408 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:30.404+0000 7f2c22f12640 1 --2- 172.21.15.43:0/408459463 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f2c0c0039a0 0x7f2c0c003dc0 secure :-1 s=READY pgs=1430 cs=0 l=1 rev1=1 crypto rx=0x7f2c1000ba30 tx=0x7f2c0c0057f0 comp rx=0 tx=0).stop 2023-08-09T13:53:30.408 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:30.404+0000 7f2c214af640 1 -- 172.21.15.43:0/408459463 reap_dead start 2023-08-09T13:53:30.409 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:30.404+0000 7f2c22f12640 1 -- 172.21.15.43:0/408459463 shutdown_connections 2023-08-09T13:53:30.409 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:30.404+0000 7f2c22f12640 1 -- 172.21.15.43:0/408459463 >> 172.21.15.43:0/408459463 conn(0x7f2c1c0aff50 msgr2=0x7f2c1c0b0360 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-09T13:53:30.409 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:30.404+0000 7f2c22f12640 1 -- 172.21.15.43:0/408459463 shutdown_connections 2023-08-09T13:53:30.409 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:30.405+0000 7f2c22f12640 1 -- 172.21.15.43:0/408459463 wait complete. 2023-08-09T13:53:30.411 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:30.406+0000 7f2c22f12640 1 Processor -- start 2023-08-09T13:53:30.411 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:30.406+0000 7f2c22f12640 1 -- start start 2023-08-09T13:53:30.411 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:30.407+0000 7f2c22f12640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f2c0c0039a0 0x7f2c1c15b6e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:53:30.411 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:30.407+0000 7f2c22f12640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f2c1c148a40 0x7f2c1c15bc20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:53:30.411 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:30.407+0000 7f2c22f12640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f2c1c15da00 0x7f2c1c15deb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:53:30.411 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:30.407+0000 7f2c22f12640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2c1c14a8c0 con 0x7f2c0c0039a0 2023-08-09T13:53:30.411 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:30.407+0000 7f2c22f12640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2c1c149cb0 con 0x7f2c1c15da00 2023-08-09T13:53:30.412 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:30.407+0000 7f2c22f12640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2c1c149f90 con 0x7f2c1c148a40 2023-08-09T13:53:30.412 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:30.408+0000 7f2c20cae640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f2c0c0039a0 0x7f2c1c15b6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:53:30.412 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:30.408+0000 7f2c20cae640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f2c0c0039a0 0x7f2c1c15b6e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:58268/0 (socket says 172.21.15.43:58268) 2023-08-09T13:53:30.412 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:30.408+0000 7f2c20cae640 1 -- 172.21.15.43:0/2980964867 learned_addr learned my addr 172.21.15.43:0/2980964867 (peer_addr_for_me v2:172.21.15.43:0/0) 2023-08-09T13:53:30.412 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:30.408+0000 7f2c20cae640 1 -- 172.21.15.43:0/2980964867 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f2c1c148a40 msgr2=0x7f2c1c15bc20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:53:30.412 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:30.408+0000 7f2c20cae640 1 --2- 172.21.15.43:0/2980964867 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f2c1c148a40 0x7f2c1c15bc20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:53:30.412 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:30.408+0000 7f2c214af640 1 --2- 172.21.15.43:0/2980964867 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f2c1c15da00 0x7f2c1c15deb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:53:30.413 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:30.408+0000 7f2c20cae640 1 -- 172.21.15.43:0/2980964867 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f2c1c15da00 msgr2=0x7f2c1c15deb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:53:30.413 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:30.408+0000 7f2c20cae640 1 --2- 172.21.15.43:0/2980964867 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f2c1c15da00 0x7f2c1c15deb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:53:30.413 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:30.408+0000 7f2c20cae640 1 -- 172.21.15.43:0/2980964867 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2c1c14a270 con 0x7f2c0c0039a0 2023-08-09T13:53:30.413 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:30.409+0000 7f2c20cae640 1 --2- 172.21.15.43:0/2980964867 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f2c0c0039a0 0x7f2c1c15b6e0 secure :-1 s=READY pgs=1431 cs=0 l=1 rev1=1 crypto rx=0x7f2c1000bb50 tx=0x7f2c10004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-09T13:53:30.413 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:30.409+0000 7f2c197fa640 1 -- 172.21.15.43:0/2980964867 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2c1003ca80 con 0x7f2c0c0039a0 2023-08-09T13:53:30.413 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:30.409+0000 7f2c197fa640 1 -- 172.21.15.43:0/2980964867 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2c1003cbe0 con 0x7f2c0c0039a0 2023-08-09T13:53:30.413 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:30.409+0000 7f2c22f12640 1 -- 172.21.15.43:0/2980964867 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2c1c14a550 con 0x7f2c0c0039a0 2023-08-09T13:53:30.414 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:30.409+0000 7f2c197fa640 1 -- 172.21.15.43:0/2980964867 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2c10030410 con 0x7f2c0c0039a0 2023-08-09T13:53:30.414 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:30.409+0000 7f2c22f12640 1 -- 172.21.15.43:0/2980964867 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2c1c15f980 con 0x7f2c0c0039a0 2023-08-09T13:53:30.414 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:30.410+0000 7f2c22f12640 1 -- 172.21.15.43:0/2980964867 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2c1c0739f0 con 0x7f2c0c0039a0 2023-08-09T13:53:30.415 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:30.411+0000 7f2c197fa640 1 -- 172.21.15.43:0/2980964867 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 55) v1 ==== 88387+0+0 (secure 0 0 0) 0x7f2c10030570 con 0x7f2c0c0039a0 2023-08-09T13:53:30.415 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:30.412+0000 7f2c197fa640 1 --2- 172.21.15.43:0/2980964867 >> [v2:172.21.15.203:6832/2527059680,v1:172.21.15.203:6833/2527059680] conn(0x7f2c0006a0e0 0x7f2c0006c5a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:53:30.415 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:30.413+0000 7f2c197fa640 1 -- 172.21.15.43:0/2980964867 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(118..118 src has 1..118) v4 ==== 7200+0+0 (secure 0 0 0) 0x7f2c100ab0a0 con 0x7f2c0c0039a0 2023-08-09T13:53:30.415 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:30.413+0000 7f2c1bfff640 1 --2- 172.21.15.43:0/2980964867 >> [v2:172.21.15.203:6832/2527059680,v1:172.21.15.203:6833/2527059680] conn(0x7f2c0006a0e0 0x7f2c0006c5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:53:30.416 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:30.414+0000 7f2c1bfff640 1 --2- 172.21.15.43:0/2980964867 >> [v2:172.21.15.203:6832/2527059680,v1:172.21.15.203:6833/2527059680] conn(0x7f2c0006a0e0 0x7f2c0006c5a0 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7f2c08001050 tx=0x7f2c08001d40 comp rx=0 tx=0).ready entity=mgr.6377 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-09T13:53:30.418 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:30.417+0000 7f2c197fa640 1 -- 172.21.15.43:0/2980964867 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178066 (secure 0 0 0) 0x7f2c10079c60 con 0x7f2c0c0039a0 2023-08-09T13:53:30.615 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:30.614+0000 7f2c22f12640 1 -- 172.21.15.43:0/2980964867 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f2c1c06a3d0 con 0x7f2c0c0039a0 2023-08-09T13:53:30.616 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:30.615+0000 7f2c197fa640 1 -- 172.21.15.43:0/2980964867 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v118) v1 ==== 100+0+11 (secure 0 0 0) 0x7f2c10036070 con 0x7f2c0c0039a0 2023-08-09T13:53:30.616 INFO:teuthology.orchestra.run.smithi043.stdout:pg_num: 64 2023-08-09T13:53:30.619 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:30.617+0000 7f2c22f12640 1 -- 172.21.15.43:0/2980964867 >> [v2:172.21.15.203:6832/2527059680,v1:172.21.15.203:6833/2527059680] conn(0x7f2c0006a0e0 msgr2=0x7f2c0006c5a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:53:30.620 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:30.618+0000 7f2c22f12640 1 --2- 172.21.15.43:0/2980964867 >> [v2:172.21.15.203:6832/2527059680,v1:172.21.15.203:6833/2527059680] conn(0x7f2c0006a0e0 0x7f2c0006c5a0 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7f2c08001050 tx=0x7f2c08001d40 comp rx=0 tx=0).stop 2023-08-09T13:53:30.620 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:30.618+0000 7f2c22f12640 1 -- 172.21.15.43:0/2980964867 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f2c0c0039a0 msgr2=0x7f2c1c15b6e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:53:30.620 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:30.618+0000 7f2c22f12640 1 --2- 172.21.15.43:0/2980964867 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f2c0c0039a0 0x7f2c1c15b6e0 secure :-1 s=READY pgs=1431 cs=0 l=1 rev1=1 crypto rx=0x7f2c1000bb50 tx=0x7f2c10004730 comp rx=0 tx=0).stop 2023-08-09T13:53:30.620 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:30.618+0000 7f2c22f12640 1 -- 172.21.15.43:0/2980964867 shutdown_connections 2023-08-09T13:53:30.620 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:30.618+0000 7f2c22f12640 1 --2- 172.21.15.43:0/2980964867 >> [v2:172.21.15.203:6832/2527059680,v1:172.21.15.203:6833/2527059680] conn(0x7f2c0006a0e0 0x7f2c0006c5a0 unknown :-1 s=CLOSED pgs=23 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:53:30.620 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:30.618+0000 7f2c22f12640 1 --2- 172.21.15.43:0/2980964867 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f2c1c15da00 0x7f2c1c15deb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:53:30.621 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:30.618+0000 7f2c22f12640 1 --2- 172.21.15.43:0/2980964867 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f2c1c148a40 0x7f2c1c15bc20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:53:30.621 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:30.618+0000 7f2c22f12640 1 --2- 172.21.15.43:0/2980964867 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f2c0c0039a0 0x7f2c1c15b6e0 unknown :-1 s=CLOSED pgs=1431 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:53:30.621 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:30.618+0000 7f2c22f12640 1 -- 172.21.15.43:0/2980964867 >> 172.21.15.43:0/2980964867 conn(0x7f2c1c0aff50 msgr2=0x7f2c1c0ab930 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-09T13:53:30.621 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:30.618+0000 7f2c22f12640 1 -- 172.21.15.43:0/2980964867 shutdown_connections 2023-08-09T13:53:30.621 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:30.618+0000 7f2c22f12640 1 -- 172.21.15.43:0/2980964867 wait complete. 2023-08-09T13:53:30.638 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-08-09T13:53:30.860 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:30.859+0000 7f4c965a6640 1 Processor -- start 2023-08-09T13:53:30.861 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:30.859+0000 7f4c965a6640 1 -- start start 2023-08-09T13:53:30.862 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:30.860+0000 7f4c965a6640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f4c9015c550 0x7f4c900d1a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:53:30.862 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:30.860+0000 7f4c965a6640 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7f4c9015c930 0x7f4c900cee40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:53:30.862 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:30.860+0000 7f4c965a6640 1 -- --> v1:172.21.15.43:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4c900d0700 con 0x7f4c900d1fc0 2023-08-09T13:53:30.862 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:30.860+0000 7f4c965a6640 1 -- --> v2:172.21.15.203:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4c900d09e0 con 0x7f4c9015c930 2023-08-09T13:53:30.862 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:30.860+0000 7f4c965a6640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4c900d0cc0 con 0x7f4c9015c550 2023-08-09T13:53:30.864 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:30.861+0000 7f4c94b43640 1 --1- >> v1:172.21.15.43:6789/0 conn(0x7f4c900d1fc0 0x7f4c900b3ee0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.43:6789/0 says I am v1:172.21.15.43:40998/0 (socket says 172.21.15.43:40998) 2023-08-09T13:53:30.864 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:30.861+0000 7f4c94b43640 1 -- 172.21.15.43:0/207028815 learned_addr learned my addr 172.21.15.43:0/207028815 (peer_addr_for_me v1:172.21.15.43:0/0) 2023-08-09T13:53:30.864 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:30.861+0000 7f4c8ffff640 1 --2- 172.21.15.43:0/207028815 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f4c9015c550 0x7f4c900d1a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:53:30.864 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:30.861+0000 7f4c8f7fe640 1 --2- 172.21.15.43:0/207028815 >> v2:172.21.15.203:3300/0 conn(0x7f4c9015c930 0x7f4c900cee40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:53:30.865 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:30.861+0000 7f4c8effd640 1 -- 172.21.15.43:0/207028815 <== mon.0 v1:172.21.15.43:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3409578715 0 0) 0x7f4c900d0700 con 0x7f4c900d1fc0 2023-08-09T13:53:30.865 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:30.861+0000 7f4c8effd640 1 -- 172.21.15.43:0/207028815 --> v1:172.21.15.43:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f4c78003520 con 0x7f4c900d1fc0 2023-08-09T13:53:30.865 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:30.862+0000 7f4c8effd640 1 -- 172.21.15.43:0/207028815 <== mon.0 v1:172.21.15.43:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 4249852578 0 0) 0x7f4c78003520 con 0x7f4c900d1fc0 2023-08-09T13:53:30.865 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:30.862+0000 7f4c8effd640 1 -- 172.21.15.43:0/207028815 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f4c9015c550 msgr2=0x7f4c900d1a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-09T13:53:30.865 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:30.862+0000 7f4c8effd640 1 --2- 172.21.15.43:0/207028815 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f4c9015c550 0x7f4c900d1a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:53:30.865 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:30.862+0000 7f4c8effd640 1 -- 172.21.15.43:0/207028815 >> v2:172.21.15.203:3300/0 conn(0x7f4c9015c930 msgr2=0x7f4c900cee40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-09T13:53:30.865 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:30.862+0000 7f4c8effd640 1 --2- 172.21.15.43:0/207028815 >> v2:172.21.15.203:3300/0 conn(0x7f4c9015c930 0x7f4c900cee40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:53:30.866 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:30.862+0000 7f4c8effd640 1 -- 172.21.15.43:0/207028815 --> v1:172.21.15.43:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4c900d0fa0 con 0x7f4c900d1fc0 2023-08-09T13:53:30.866 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:30.862+0000 7f4c8effd640 1 -- 172.21.15.43:0/207028815 <== mon.0 v1:172.21.15.43:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1285150740 0 0) 0x7f4c88001670 con 0x7f4c900d1fc0 2023-08-09T13:53:30.866 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:30.862+0000 7f4c8effd640 1 -- 172.21.15.43:0/207028815 >> v1:172.21.15.43:6789/0 conn(0x7f4c900d1fc0 legacy=0x7f4c900b3ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:53:30.866 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:30.862+0000 7f4c8effd640 1 --2- 172.21.15.43:0/207028815 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f4c9015c550 0x7f4c780039e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:53:30.866 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:30.862+0000 7f4c8effd640 1 --2- 172.21.15.43:0/207028815 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f4c78003f60 0x7f4c780047c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:53:30.866 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:30.862+0000 7f4c8effd640 1 --2- 172.21.15.43:0/207028815 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f4c78004d00 0x7f4c78005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:53:30.866 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:30.862+0000 7f4c8effd640 1 -- 172.21.15.43:0/207028815 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4c900d09e0 con 0x7f4c78003f60 2023-08-09T13:53:30.867 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:30.862+0000 7f4c8effd640 1 -- 172.21.15.43:0/207028815 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4c900d0cc0 con 0x7f4c78004d00 2023-08-09T13:53:30.867 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:30.862+0000 7f4c8effd640 1 -- 172.21.15.43:0/207028815 --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4c7800ba30 con 0x7f4c9015c550 2023-08-09T13:53:30.867 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:30.862+0000 7f4c8f7fe640 1 --2- 172.21.15.43:0/207028815 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f4c78003f60 0x7f4c780047c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:53:30.867 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:30.863+0000 7f4c8f7fe640 1 -- 172.21.15.43:0/207028815 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f4c9015c550 msgr2=0x7f4c780039e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-09T13:53:30.867 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:30.863+0000 7f4c8f7fe640 1 --2- 172.21.15.43:0/207028815 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f4c9015c550 0x7f4c780039e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:53:30.867 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:30.863+0000 7f4c8f7fe640 1 -- 172.21.15.43:0/207028815 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f4c78004d00 msgr2=0x7f4c78005180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-09T13:53:30.867 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:30.863+0000 7f4c8f7fe640 1 --2- 172.21.15.43:0/207028815 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f4c78004d00 0x7f4c78005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:53:30.867 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:30.863+0000 7f4c8f7fe640 1 -- 172.21.15.43:0/207028815 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4c78003520 con 0x7f4c78003f60 2023-08-09T13:53:30.869 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:30.864+0000 7f4c8f7fe640 1 --2- 172.21.15.43:0/207028815 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f4c78003f60 0x7f4c780047c0 secure :-1 s=READY pgs=1433 cs=0 l=1 rev1=1 crypto rx=0x7f4c84009a90 tx=0x7f4c780057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9d54b677d3e0e28 server_cookie=0 in_seq=0 out_seq=0 2023-08-09T13:53:30.869 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:30.864+0000 7f4c8effd640 1 -- 172.21.15.43:0/207028815 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4c8402c6d0 con 0x7f4c78003f60 2023-08-09T13:53:30.869 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:30.864+0000 7f4c8effd640 1 -- 172.21.15.43:0/207028815 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4c8402c830 con 0x7f4c78003f60 2023-08-09T13:53:30.869 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:30.864+0000 7f4c8effd640 1 -- 172.21.15.43:0/207028815 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4c8402cbc0 con 0x7f4c78003f60 2023-08-09T13:53:30.869 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:30.865+0000 7f4c965a6640 1 -- 172.21.15.43:0/207028815 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f4c78003f60 msgr2=0x7f4c780047c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:53:30.869 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:30.865+0000 7f4c965a6640 1 --2- 172.21.15.43:0/207028815 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f4c78003f60 0x7f4c780047c0 secure :-1 s=READY pgs=1433 cs=0 l=1 rev1=1 crypto rx=0x7f4c84009a90 tx=0x7f4c780057f0 comp rx=0 tx=0).stop 2023-08-09T13:53:30.870 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:30.865+0000 7f4c94b43640 1 -- 172.21.15.43:0/207028815 reap_dead start 2023-08-09T13:53:30.870 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:30.865+0000 7f4c965a6640 1 -- 172.21.15.43:0/207028815 shutdown_connections 2023-08-09T13:53:30.870 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:30.865+0000 7f4c965a6640 1 -- 172.21.15.43:0/207028815 >> 172.21.15.43:0/207028815 conn(0x7f4c9006a4e0 msgr2=0x7f4c9006a8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-09T13:53:30.870 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:30.865+0000 7f4c965a6640 1 -- 172.21.15.43:0/207028815 shutdown_connections 2023-08-09T13:53:30.870 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:30.865+0000 7f4c965a6640 1 -- 172.21.15.43:0/207028815 wait complete. 2023-08-09T13:53:30.870 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:30.867+0000 7f4c965a6640 1 Processor -- start 2023-08-09T13:53:30.870 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:30.867+0000 7f4c965a6640 1 -- start start 2023-08-09T13:53:30.872 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:30.868+0000 7f4c965a6640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f4c78003f60 0x7f4c900c79f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:53:30.872 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:30.868+0000 7f4c965a6640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f4c9015a910 0x7f4c9015d6e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:53:30.872 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:30.868+0000 7f4c965a6640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f4c90151b10 0x7f4c9015dc20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:53:30.872 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:30.868+0000 7f4c965a6640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4c900c7f30 con 0x7f4c90151b10 2023-08-09T13:53:30.872 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:30.868+0000 7f4c965a6640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4c900b21a0 con 0x7f4c78003f60 2023-08-09T13:53:30.873 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:30.868+0000 7f4c965a6640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4c900b3000 con 0x7f4c9015a910 2023-08-09T13:53:30.873 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:30.869+0000 7f4c94b43640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f4c90151b10 0x7f4c9015dc20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:53:30.873 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:30.869+0000 7f4c94b43640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f4c90151b10 0x7f4c9015dc20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:58288/0 (socket says 172.21.15.43:58288) 2023-08-09T13:53:30.873 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:30.869+0000 7f4c94b43640 1 -- 172.21.15.43:0/4188710428 learned_addr learned my addr 172.21.15.43:0/4188710428 (peer_addr_for_me v2:172.21.15.43:0/0) 2023-08-09T13:53:30.873 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:30.869+0000 7f4c8f7fe640 1 --2- 172.21.15.43:0/4188710428 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f4c9015a910 0x7f4c9015d6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:53:30.873 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:30.869+0000 7f4c8ffff640 1 --2- 172.21.15.43:0/4188710428 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f4c78003f60 0x7f4c900c79f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:53:30.873 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:30.869+0000 7f4c94b43640 1 -- 172.21.15.43:0/4188710428 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f4c9015a910 msgr2=0x7f4c9015d6e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:53:30.874 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:30.869+0000 7f4c94b43640 1 --2- 172.21.15.43:0/4188710428 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f4c9015a910 0x7f4c9015d6e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:53:30.874 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:30.869+0000 7f4c94b43640 1 -- 172.21.15.43:0/4188710428 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f4c78003f60 msgr2=0x7f4c900c79f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:53:30.874 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:30.869+0000 7f4c94b43640 1 --2- 172.21.15.43:0/4188710428 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f4c78003f60 0x7f4c900c79f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:53:30.874 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:30.869+0000 7f4c94b43640 1 -- 172.21.15.43:0/4188710428 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4c900b23f0 con 0x7f4c90151b10 2023-08-09T13:53:30.874 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:30.869+0000 7f4c8f7fe640 1 --2- 172.21.15.43:0/4188710428 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f4c9015a910 0x7f4c9015d6e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-08-09T13:53:30.874 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:30.870+0000 7f4c94b43640 1 --2- 172.21.15.43:0/4188710428 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f4c90151b10 0x7f4c9015dc20 secure :-1 s=READY pgs=1434 cs=0 l=1 rev1=1 crypto rx=0x7f4c88000c00 tx=0x7f4c8800cb90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-09T13:53:30.874 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:30.870+0000 7f4c8cff9640 1 -- 172.21.15.43:0/4188710428 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4c88007230 con 0x7f4c90151b10 2023-08-09T13:53:30.874 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:30.870+0000 7f4c8cff9640 1 -- 172.21.15.43:0/4188710428 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4c8800ced0 con 0x7f4c90151b10 2023-08-09T13:53:30.875 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:30.870+0000 7f4c8cff9640 1 -- 172.21.15.43:0/4188710428 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4c880075d0 con 0x7f4c90151b10 2023-08-09T13:53:30.875 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:30.870+0000 7f4c965a6640 1 -- 172.21.15.43:0/4188710428 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4c900b26d0 con 0x7f4c90151b10 2023-08-09T13:53:30.875 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:30.870+0000 7f4c965a6640 1 -- 172.21.15.43:0/4188710428 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4c900b2c90 con 0x7f4c90151b10 2023-08-09T13:53:30.876 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:30.871+0000 7f4c965a6640 1 -- 172.21.15.43:0/4188710428 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4c900739f0 con 0x7f4c90151b10 2023-08-09T13:53:30.876 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:30.872+0000 7f4c8cff9640 1 -- 172.21.15.43:0/4188710428 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 55) v1 ==== 88387+0+0 (secure 0 0 0) 0x7f4c880047e0 con 0x7f4c90151b10 2023-08-09T13:53:30.876 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:30.873+0000 7f4c8cff9640 1 --2- 172.21.15.43:0/4188710428 >> [v2:172.21.15.203:6832/2527059680,v1:172.21.15.203:6833/2527059680] conn(0x7f4c7c06a110 0x7f4c7c06c5d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:53:30.876 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:30.873+0000 7f4c8cff9640 1 -- 172.21.15.43:0/4188710428 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(118..118 src has 1..118) v4 ==== 7200+0+0 (secure 0 0 0) 0x7f4c88087fc0 con 0x7f4c90151b10 2023-08-09T13:53:30.876 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:30.874+0000 7f4c8ffff640 1 --2- 172.21.15.43:0/4188710428 >> [v2:172.21.15.203:6832/2527059680,v1:172.21.15.203:6833/2527059680] conn(0x7f4c7c06a110 0x7f4c7c06c5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:53:30.877 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:30.875+0000 7f4c8ffff640 1 --2- 172.21.15.43:0/4188710428 >> [v2:172.21.15.203:6832/2527059680,v1:172.21.15.203:6833/2527059680] conn(0x7f4c7c06a110 0x7f4c7c06c5d0 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7f4c80004b30 tx=0x7f4c8000d040 comp rx=0 tx=0).ready entity=mgr.6377 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-09T13:53:30.879 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:30.878+0000 7f4c8cff9640 1 -- 172.21.15.43:0/4188710428 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178066 (secure 0 0 0) 0x7f4c88053170 con 0x7f4c90151b10 2023-08-09T13:53:31.112 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:31.111+0000 7f4c965a6640 1 -- 172.21.15.43:0/4188710428 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f4c900b3140 con 0x7f4c90151b10 2023-08-09T13:53:31.113 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:31.112+0000 7f4c8cff9640 1 -- 172.21.15.43:0/4188710428 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 52 v52) v1 ==== 94+0+4883 (secure 0 0 0) 0x7f4c8801a070 con 0x7f4c90151b10 2023-08-09T13:53:31.114 INFO:teuthology.orchestra.run.smithi043.stdout: 2023-08-09T13:53:31.114 INFO:teuthology.orchestra.run.smithi043.stdout:{"epoch":52,"default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6215,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.43:6835/2214346432","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6833","nonce":2214346432},{"type":"v1","addr":"172.21.15.43:6835","nonce":2214346432}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":47},{"gid":6823,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.203:6835/3236264045","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6834","nonce":3236264045},{"type":"v1","addr":"172.21.15.203:6835","nonce":3236264045}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":46},{"gid":6826,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.203:6837/3788289156","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6836","nonce":3788289156},{"type":"v1","addr":"172.21.15.203:6837","nonce":3788289156}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":47}],"filesystems":[{"mdsmap":{"epoch":52,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-08-09T13:51:59.655630+0000","modified":"2023-08-09T13:52:03.220527+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":8040},"failed":[],"damaged":[],"stopped":[],"info":{"gid_8040":{"gid":8040,"name":"c","rank":0,"incarnation":48,"state":"up:active","state_seq":3,"addr":"172.21.15.43:6834/2223580538","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6832","nonce":2223580538},{"type":"v1","addr":"172.21.15.43:6834","nonce":2223580538}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[15],"metadata_pool":14,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":6}]} 2023-08-09T13:53:31.115 INFO:teuthology.orchestra.run.smithi043.stderr:dumped fsmap epoch 52 2023-08-09T13:53:31.116 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:31.114+0000 7f4c965a6640 1 -- 172.21.15.43:0/4188710428 >> [v2:172.21.15.203:6832/2527059680,v1:172.21.15.203:6833/2527059680] conn(0x7f4c7c06a110 msgr2=0x7f4c7c06c5d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:53:31.116 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:31.115+0000 7f4c965a6640 1 --2- 172.21.15.43:0/4188710428 >> [v2:172.21.15.203:6832/2527059680,v1:172.21.15.203:6833/2527059680] conn(0x7f4c7c06a110 0x7f4c7c06c5d0 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7f4c80004b30 tx=0x7f4c8000d040 comp rx=0 tx=0).stop 2023-08-09T13:53:31.116 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:31.115+0000 7f4c965a6640 1 -- 172.21.15.43:0/4188710428 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f4c90151b10 msgr2=0x7f4c9015dc20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:53:31.117 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:31.115+0000 7f4c965a6640 1 --2- 172.21.15.43:0/4188710428 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f4c90151b10 0x7f4c9015dc20 secure :-1 s=READY pgs=1434 cs=0 l=1 rev1=1 crypto rx=0x7f4c88000c00 tx=0x7f4c8800cb90 comp rx=0 tx=0).stop 2023-08-09T13:53:31.117 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:31.115+0000 7f4c965a6640 1 -- 172.21.15.43:0/4188710428 shutdown_connections 2023-08-09T13:53:31.117 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:31.115+0000 7f4c965a6640 1 --2- 172.21.15.43:0/4188710428 >> [v2:172.21.15.203:6832/2527059680,v1:172.21.15.203:6833/2527059680] conn(0x7f4c7c06a110 0x7f4c7c06c5d0 unknown :-1 s=CLOSED pgs=25 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:53:31.117 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:31.115+0000 7f4c965a6640 1 --2- 172.21.15.43:0/4188710428 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f4c90151b10 0x7f4c9015dc20 unknown :-1 s=CLOSED pgs=1434 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:53:31.117 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:31.115+0000 7f4c965a6640 1 --2- 172.21.15.43:0/4188710428 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f4c9015a910 0x7f4c9015d6e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:53:31.117 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:31.115+0000 7f4c965a6640 1 --2- 172.21.15.43:0/4188710428 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f4c78003f60 0x7f4c900c79f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:53:31.117 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:31.115+0000 7f4c965a6640 1 -- 172.21.15.43:0/4188710428 >> 172.21.15.43:0/4188710428 conn(0x7f4c9006a4e0 msgr2=0x7f4c9015bab0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-09T13:53:31.118 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:31.115+0000 7f4c965a6640 1 -- 172.21.15.43:0/4188710428 shutdown_connections 2023-08-09T13:53:31.118 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:31.115+0000 7f4c965a6640 1 -- 172.21.15.43:0/4188710428 wait complete. 2023-08-09T13:53:31.134 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-08-09T13:53:31.354 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:31.353+0000 7f1309e85640 1 Processor -- start 2023-08-09T13:53:31.355 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:31.353+0000 7f1309e85640 1 -- start start 2023-08-09T13:53:31.356 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:31.355+0000 7f1309e85640 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7f13041590b0 0x7f13040cabb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:53:31.356 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:31.355+0000 7f1309e85640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f13040cb0f0 0x7f130415ce10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:53:31.356 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:31.355+0000 7f1309e85640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f1304159490 0x7f130415d350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:53:31.357 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:31.355+0000 7f1309e85640 1 -- --> v2:172.21.15.43:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f13040ca720 con 0x7f1304159490 2023-08-09T13:53:31.357 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:31.355+0000 7f1309e85640 1 -- --> v2:172.21.15.203:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1304070c90 con 0x7f13041590b0 2023-08-09T13:53:31.357 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:31.355+0000 7f1309e85640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f130415d980 con 0x7f13040cb0f0 2023-08-09T13:53:31.358 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:31.355+0000 7f1309684640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f1304159490 0x7f130415d350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:53:31.358 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:31.355+0000 7f1309684640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f1304159490 0x7f130415d350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:58292/0 (socket says 172.21.15.43:58292) 2023-08-09T13:53:31.358 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:31.355+0000 7f1309684640 1 -- 172.21.15.43:0/4036311792 learned_addr learned my addr 172.21.15.43:0/4036311792 (peer_addr_for_me v2:172.21.15.43:0/0) 2023-08-09T13:53:31.358 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:31.356+0000 7f1303fff640 1 --2- 172.21.15.43:0/4036311792 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f13040cb0f0 0x7f130415ce10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:53:31.359 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:31.356+0000 7f1308e83640 1 --2- 172.21.15.43:0/4036311792 >> v2:172.21.15.203:3300/0 conn(0x7f13041590b0 0x7f13040cabb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:53:31.359 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:31.356+0000 7f1309684640 1 -- 172.21.15.43:0/4036311792 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f13040cb0f0 msgr2=0x7f130415ce10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-09T13:53:31.359 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:31.356+0000 7f1309684640 1 --2- 172.21.15.43:0/4036311792 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f13040cb0f0 0x7f130415ce10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:53:31.359 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:31.356+0000 7f1309684640 1 -- 172.21.15.43:0/4036311792 >> v2:172.21.15.203:3300/0 conn(0x7f13041590b0 msgr2=0x7f13040cabb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-09T13:53:31.359 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:31.356+0000 7f1309684640 1 --2- 172.21.15.43:0/4036311792 >> v2:172.21.15.203:3300/0 conn(0x7f13041590b0 0x7f13040cabb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:53:31.359 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:31.356+0000 7f1309684640 1 -- 172.21.15.43:0/4036311792 --> v2:172.21.15.43:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f13040ca0d0 con 0x7f1304159490 2023-08-09T13:53:31.360 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:31.358+0000 7f1309684640 1 --2- 172.21.15.43:0/4036311792 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f1304159490 0x7f130415d350 secure :-1 s=READY pgs=1435 cs=0 l=1 rev1=1 crypto rx=0x7f13040c29a0 tx=0x7f12fc02e0e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=386b1bfa9e480314 server_cookie=0 in_seq=0 out_seq=0 2023-08-09T13:53:31.361 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:31.359+0000 7f13037fe640 1 -- 172.21.15.43:0/4036311792 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f12fc02ee90 con 0x7f1304159490 2023-08-09T13:53:31.361 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:31.359+0000 7f13037fe640 1 -- 172.21.15.43:0/4036311792 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f12fc036440 con 0x7f1304159490 2023-08-09T13:53:31.361 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:31.359+0000 7f13037fe640 1 -- 172.21.15.43:0/4036311792 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f12fc0367d0 con 0x7f1304159490 2023-08-09T13:53:31.362 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:31.360+0000 7f1309e85640 1 -- 172.21.15.43:0/4036311792 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f1304159490 msgr2=0x7f130415d350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:53:31.363 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:31.360+0000 7f1309e85640 1 --2- 172.21.15.43:0/4036311792 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f1304159490 0x7f130415d350 secure :-1 s=READY pgs=1435 cs=0 l=1 rev1=1 crypto rx=0x7f13040c29a0 tx=0x7f12fc02e0e0 comp rx=0 tx=0).stop 2023-08-09T13:53:31.363 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:31.360+0000 7f1309e85640 1 -- 172.21.15.43:0/4036311792 shutdown_connections 2023-08-09T13:53:31.363 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:31.360+0000 7f1309e85640 1 --2- 172.21.15.43:0/4036311792 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f1304159490 0x7f130415d350 unknown :-1 s=CLOSED pgs=1435 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:53:31.363 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:31.360+0000 7f1309e85640 1 --2- 172.21.15.43:0/4036311792 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f13040cb0f0 0x7f130415ce10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:53:31.364 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:31.360+0000 7f1309e85640 1 --2- 172.21.15.43:0/4036311792 >> v2:172.21.15.203:3300/0 conn(0x7f13041590b0 0x7f13040cabb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:53:31.364 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:31.360+0000 7f1309e85640 1 -- 172.21.15.43:0/4036311792 >> 172.21.15.43:0/4036311792 conn(0x7f130406a8d0 msgr2=0x7f130406acb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-09T13:53:31.364 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:31.360+0000 7f1309e85640 1 -- 172.21.15.43:0/4036311792 shutdown_connections 2023-08-09T13:53:31.365 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:31.360+0000 7f1309e85640 1 -- 172.21.15.43:0/4036311792 wait complete. 2023-08-09T13:53:31.367 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:31.362+0000 7f1309e85640 1 Processor -- start 2023-08-09T13:53:31.367 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:31.362+0000 7f1309e85640 1 -- start start 2023-08-09T13:53:31.368 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:31.363+0000 7f1309e85640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f13040cb0f0 0x7f1304153510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:53:31.368 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:31.363+0000 7f1309e85640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f13041590b0 0x7f13041598a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:53:31.368 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:31.363+0000 7f1309e85640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f1304159490 0x7f13040b36d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:53:31.368 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:31.363+0000 7f1309e85640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1304153a50 con 0x7f13040cb0f0 2023-08-09T13:53:31.368 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:31.363+0000 7f1309e85640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f130415b240 con 0x7f13041590b0 2023-08-09T13:53:31.368 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:31.363+0000 7f1309e85640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f130415a1a0 con 0x7f1304159490 2023-08-09T13:53:31.368 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:31.363+0000 7f1308e83640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f13040cb0f0 0x7f1304153510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:53:31.369 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:31.363+0000 7f1308e83640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f13040cb0f0 0x7f1304153510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:58306/0 (socket says 172.21.15.43:58306) 2023-08-09T13:53:31.369 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:31.363+0000 7f1308e83640 1 -- 172.21.15.43:0/61485739 learned_addr learned my addr 172.21.15.43:0/61485739 (peer_addr_for_me v2:172.21.15.43:0/0) 2023-08-09T13:53:31.369 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:31.364+0000 7f1303fff640 1 --2- 172.21.15.43:0/61485739 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f13041590b0 0x7f13041598a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:53:31.369 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:31.364+0000 7f1308e83640 1 -- 172.21.15.43:0/61485739 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f1304159490 msgr2=0x7f13040b36d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:53:31.369 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:31.364+0000 7f1308e83640 1 --2- 172.21.15.43:0/61485739 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f1304159490 0x7f13040b36d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:53:31.370 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:31.364+0000 7f1308e83640 1 -- 172.21.15.43:0/61485739 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f13041590b0 msgr2=0x7f13041598a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:53:31.370 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:31.364+0000 7f1308e83640 1 --2- 172.21.15.43:0/61485739 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f13041590b0 0x7f13041598a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:53:31.370 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:31.364+0000 7f1308e83640 1 -- 172.21.15.43:0/61485739 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f12fc00b770 con 0x7f13040cb0f0 2023-08-09T13:53:31.370 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:31.364+0000 7f1308e83640 1 --2- 172.21.15.43:0/61485739 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f13040cb0f0 0x7f1304153510 secure :-1 s=READY pgs=1436 cs=0 l=1 rev1=1 crypto rx=0x7f12f400efc0 tx=0x7f12f400c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-09T13:53:31.371 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:31.364+0000 7f13017fa640 1 -- 172.21.15.43:0/61485739 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f12f4014480 con 0x7f13040cb0f0 2023-08-09T13:53:31.371 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:31.364+0000 7f13017fa640 1 -- 172.21.15.43:0/61485739 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f12f400f040 con 0x7f13040cb0f0 2023-08-09T13:53:31.371 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:31.364+0000 7f13017fa640 1 -- 172.21.15.43:0/61485739 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f12f4014800 con 0x7f13040cb0f0 2023-08-09T13:53:31.371 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:31.364+0000 7f1309e85640 1 -- 172.21.15.43:0/61485739 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f130415a480 con 0x7f13040cb0f0 2023-08-09T13:53:31.371 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:31.365+0000 7f1309e85640 1 -- 172.21.15.43:0/61485739 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f130415aa40 con 0x7f13040cb0f0 2023-08-09T13:53:31.372 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:31.366+0000 7f1309e85640 1 -- 172.21.15.43:0/61485739 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f13040739f0 con 0x7f13040cb0f0 2023-08-09T13:53:31.373 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:31.367+0000 7f13017fa640 1 -- 172.21.15.43:0/61485739 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 55) v1 ==== 88387+0+0 (secure 0 0 0) 0x7f12f4014960 con 0x7f13040cb0f0 2023-08-09T13:53:31.373 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:31.368+0000 7f13017fa640 1 --2- 172.21.15.43:0/61485739 >> [v2:172.21.15.203:6832/2527059680,v1:172.21.15.203:6833/2527059680] conn(0x7f12f006a110 0x7f12f006c5d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:53:31.373 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:31.368+0000 7f13017fa640 1 -- 172.21.15.43:0/61485739 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(118..118 src has 1..118) v4 ==== 7200+0+0 (secure 0 0 0) 0x7f12f4089b40 con 0x7f13040cb0f0 2023-08-09T13:53:31.374 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:31.369+0000 7f1303fff640 1 --2- 172.21.15.43:0/61485739 >> [v2:172.21.15.203:6832/2527059680,v1:172.21.15.203:6833/2527059680] conn(0x7f12f006a110 0x7f12f006c5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:53:31.374 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:31.370+0000 7f1303fff640 1 --2- 172.21.15.43:0/61485739 >> [v2:172.21.15.203:6832/2527059680,v1:172.21.15.203:6833/2527059680] conn(0x7f12f006a110 0x7f12f006c5d0 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7f13040ae720 tx=0x7f12f8008040 comp rx=0 tx=0).ready entity=mgr.6377 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-09T13:53:31.374 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:31.372+0000 7f13017fa640 1 -- 172.21.15.43:0/61485739 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178066 (secure 0 0 0) 0x7f12f4054be0 con 0x7f13040cb0f0 2023-08-09T13:53:31.570 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:31.568+0000 7f1309e85640 1 -- 172.21.15.43:0/61485739 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f13040b1680 con 0x7f13040cb0f0 2023-08-09T13:53:31.571 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:31.570+0000 7f13017fa640 1 -- 172.21.15.43:0/61485739 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v118) v1 ==== 74+0+19647 (secure 0 0 0) 0x7f12f4058200 con 0x7f13040cb0f0 2023-08-09T13:53:31.572 INFO:teuthology.orchestra.run.smithi043.stdout: 2023-08-09T13:53:31.572 INFO:teuthology.orchestra.run.smithi043.stdout:{"epoch":118,"fsid":"daac87ee-1337-4250-a955-aea4e408a39b","created":"2023-08-09T13:33:41.701198+0000","modified":"2023-08-09T13:53:28.338768+0000","last_up_change":"2023-08-09T13:33:53.783453+0000","last_in_change":"2023-08-09T13:33:45.148904+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":15,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":"rbd","create_time":"2023-08-09T13:33:54.365137+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":".mgr","create_time":"2023-08-09T13:33:54.831803+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":14,"pool_name":"cephfs_metadata","create_time":"2023-08-09T13:51:57.637653+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"115","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":2.5,"score_stable":2.5,"optimal_score":1,"raw_score_acting":2.5,"raw_score_stable":2.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":15,"pool_name":"cephfs_data","create_time":"2023-08-09T13:51:58.676117+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"116","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"cbd49ecd-e896-4fbd-864e-5c37a829b165","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":114,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6800","nonce":502667335},{"type":"v1","addr":"172.21.15.43:6801","nonce":502667335}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6802","nonce":502667335},{"type":"v1","addr":"172.21.15.43:6803","nonce":502667335}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6806","nonce":502667335},{"type":"v1","addr":"172.21.15.43:6807","nonce":502667335}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6804","nonce":502667335},{"type":"v1","addr":"172.21.15.43:6805","nonce":502667335}]},"public_addr":"172.21.15.43:6801/502667335","cluster_addr":"172.21.15.43:6803/502667335","heartbeat_back_addr":"172.21.15.43:6807/502667335","heartbeat_front_addr":"172.21.15.43:6805/502667335","state":["exists","up"]},{"osd":1,"uuid":"75c7e0fe-1ec7-4fe8-a2f2-3661a65b8500","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":114,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6808","nonce":3289134610},{"type":"v1","addr":"172.21.15.43:6809","nonce":3289134610}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6810","nonce":3289134610},{"type":"v1","addr":"172.21.15.43:6812","nonce":3289134610}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6821","nonce":3289134610},{"type":"v1","addr":"172.21.15.43:6824","nonce":3289134610}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6815","nonce":3289134610},{"type":"v1","addr":"172.21.15.43:6818","nonce":3289134610}]},"public_addr":"172.21.15.43:6809/3289134610","cluster_addr":"172.21.15.43:6812/3289134610","heartbeat_back_addr":"172.21.15.43:6824/3289134610","heartbeat_front_addr":"172.21.15.43:6818/3289134610","state":["exists","up"]},{"osd":2,"uuid":"bb5c41e9-7f5f-4125-9b01-3bb8bfc2ea83","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":114,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6811","nonce":4079709736},{"type":"v1","addr":"172.21.15.43:6814","nonce":4079709736}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6817","nonce":4079709736},{"type":"v1","addr":"172.21.15.43:6820","nonce":4079709736}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6829","nonce":4079709736},{"type":"v1","addr":"172.21.15.43:6831","nonce":4079709736}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6823","nonce":4079709736},{"type":"v1","addr":"172.21.15.43:6826","nonce":4079709736}]},"public_addr":"172.21.15.43:6814/4079709736","cluster_addr":"172.21.15.43:6820/4079709736","heartbeat_back_addr":"172.21.15.43:6831/4079709736","heartbeat_front_addr":"172.21.15.43:6826/4079709736","state":["exists","up"]},{"osd":3,"uuid":"bfe4c8b4-694a-465c-87e6-2b2aab852613","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":114,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6813","nonce":1709538759},{"type":"v1","addr":"172.21.15.43:6816","nonce":1709538759}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6819","nonce":1709538759},{"type":"v1","addr":"172.21.15.43:6822","nonce":1709538759}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6828","nonce":1709538759},{"type":"v1","addr":"172.21.15.43:6830","nonce":1709538759}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6825","nonce":1709538759},{"type":"v1","addr":"172.21.15.43:6827","nonce":1709538759}]},"public_addr":"172.21.15.43:6816/1709538759","cluster_addr":"172.21.15.43:6822/1709538759","heartbeat_back_addr":"172.21.15.43:6830/1709538759","heartbeat_front_addr":"172.21.15.43:6827/1709538759","state":["exists","up"]},{"osd":4,"uuid":"fa2e8a77-aba1-4756-853e-0d174a5bdeba","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":114,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6800","nonce":4196866481},{"type":"v1","addr":"172.21.15.203:6801","nonce":4196866481}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6802","nonce":4196866481},{"type":"v1","addr":"172.21.15.203:6803","nonce":4196866481}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6806","nonce":4196866481},{"type":"v1","addr":"172.21.15.203:6807","nonce":4196866481}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6804","nonce":4196866481},{"type":"v1","addr":"172.21.15.203:6805","nonce":4196866481}]},"public_addr":"172.21.15.203:6801/4196866481","cluster_addr":"172.21.15.203:6803/4196866481","heartbeat_back_addr":"172.21.15.203:6807/4196866481","heartbeat_front_addr":"172.21.15.203:6805/4196866481","state":["exists","up"]},{"osd":5,"uuid":"81811e4b-4d24-4f51-a82c-f1d700a2a00d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":114,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6816","nonce":3059290081},{"type":"v1","addr":"172.21.15.203:6817","nonce":3059290081}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6818","nonce":3059290081},{"type":"v1","addr":"172.21.15.203:6819","nonce":3059290081}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6822","nonce":3059290081},{"type":"v1","addr":"172.21.15.203:6823","nonce":3059290081}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6820","nonce":3059290081},{"type":"v1","addr":"172.21.15.203:6821","nonce":3059290081}]},"public_addr":"172.21.15.203:6817/3059290081","cluster_addr":"172.21.15.203:6819/3059290081","heartbeat_back_addr":"172.21.15.203:6823/3059290081","heartbeat_front_addr":"172.21.15.203:6821/3059290081","state":["exists","up"]},{"osd":6,"uuid":"307bac4d-6a5b-443d-af75-56d695d7674f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":114,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6824","nonce":512155569},{"type":"v1","addr":"172.21.15.203:6825","nonce":512155569}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6826","nonce":512155569},{"type":"v1","addr":"172.21.15.203:6827","nonce":512155569}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6830","nonce":512155569},{"type":"v1","addr":"172.21.15.203:6831","nonce":512155569}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6828","nonce":512155569},{"type":"v1","addr":"172.21.15.203:6829","nonce":512155569}]},"public_addr":"172.21.15.203:6825/512155569","cluster_addr":"172.21.15.203:6827/512155569","heartbeat_back_addr":"172.21.15.203:6831/512155569","heartbeat_front_addr":"172.21.15.203:6829/512155569","state":["exists","up"]},{"osd":7,"uuid":"a95dc7ca-9f1b-4295-af70-68a22a2d7d5e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":114,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6808","nonce":3331450438},{"type":"v1","addr":"172.21.15.203:6809","nonce":3331450438}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6810","nonce":3331450438},{"type":"v1","addr":"172.21.15.203:6811","nonce":3331450438}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6814","nonce":3331450438},{"type":"v1","addr":"172.21.15.203:6815","nonce":3331450438}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6812","nonce":3331450438},{"type":"v1","addr":"172.21.15.203:6813","nonce":3331450438}]},"public_addr":"172.21.15.203:6809/3331450438","cluster_addr":"172.21.15.203:6811/3331450438","heartbeat_back_addr":"172.21.15.203:6815/3331450438","heartbeat_front_addr":"172.21.15.203:6813/3331450438","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-09T13:33:50.547652+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-09T13:33:50.589469+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-09T13:33:50.561724+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-09T13:33:50.599650+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-09T13:33:50.411855+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-09T13:33:50.219681+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-09T13:33:50.504664+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-09T13:33:50.441248+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.203:0/3668440292":"2023-08-10T13:53:28.338643+0000","172.21.15.203:0/430012896":"2023-08-10T13:53:28.338643+0000","172.21.15.203:0/1052273067":"2023-08-10T13:53:28.338643+0000","172.21.15.203:0/2583349636":"2023-08-10T13:53:28.338643+0000","172.21.15.203:6832/2041169533":"2023-08-10T13:53:28.338643+0000","172.21.15.203:0/1488154102":"2023-08-10T13:52:20.374695+0000","172.21.15.203:0/728819840":"2023-08-10T13:53:28.338643+0000","172.21.15.203:0/3595044319":"2023-08-10T13:52:20.374695+0000","172.21.15.203:0/2916799066":"2023-08-10T13:53:28.338643+0000","172.21.15.203:0/3843164760":"2023-08-10T13:53:28.338643+0000","172.21.15.203:0/1657891023":"2023-08-10T13:52:20.374695+0000","172.21.15.203:0/329310299":"2023-08-10T13:53:28.338643+0000","172.21.15.203:0/2368610482":"2023-08-10T13:52:20.374695+0000","172.21.15.203:0/437192587":"2023-08-10T13:52:20.374695+0000","172.21.15.203:6833/1317108840":"2023-08-10T13:52:20.374695+0000","172.21.15.203:6832/1317108840":"2023-08-10T13:52:20.374695+0000","172.21.15.203:6833/2041169533":"2023-08-10T13:53:28.338643+0000","172.21.15.203:0/1459918882":"2023-08-10T13:52:20.374695+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":7,"snaps":[{"begin":104,"length":1},{"begin":221,"length":1}]}],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-08-09T13:53:31.575 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:31.573+0000 7f1309e85640 1 -- 172.21.15.43:0/61485739 >> [v2:172.21.15.203:6832/2527059680,v1:172.21.15.203:6833/2527059680] conn(0x7f12f006a110 msgr2=0x7f12f006c5d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:53:31.575 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:31.573+0000 7f1309e85640 1 --2- 172.21.15.43:0/61485739 >> [v2:172.21.15.203:6832/2527059680,v1:172.21.15.203:6833/2527059680] conn(0x7f12f006a110 0x7f12f006c5d0 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7f13040ae720 tx=0x7f12f8008040 comp rx=0 tx=0).stop 2023-08-09T13:53:31.575 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:31.573+0000 7f1309e85640 1 -- 172.21.15.43:0/61485739 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f13040cb0f0 msgr2=0x7f1304153510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:53:31.575 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:31.573+0000 7f1309e85640 1 --2- 172.21.15.43:0/61485739 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f13040cb0f0 0x7f1304153510 secure :-1 s=READY pgs=1436 cs=0 l=1 rev1=1 crypto rx=0x7f12f400efc0 tx=0x7f12f400c490 comp rx=0 tx=0).stop 2023-08-09T13:53:31.575 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:31.573+0000 7f1309e85640 1 -- 172.21.15.43:0/61485739 shutdown_connections 2023-08-09T13:53:31.575 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:31.573+0000 7f1309e85640 1 --2- 172.21.15.43:0/61485739 >> [v2:172.21.15.203:6832/2527059680,v1:172.21.15.203:6833/2527059680] conn(0x7f12f006a110 0x7f12f006c5d0 unknown :-1 s=CLOSED pgs=26 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:53:31.576 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:31.573+0000 7f1309e85640 1 --2- 172.21.15.43:0/61485739 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f1304159490 0x7f13040b36d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:53:31.576 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:31.573+0000 7f1309e85640 1 --2- 172.21.15.43:0/61485739 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f13041590b0 0x7f13041598a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:53:31.576 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:31.573+0000 7f1309e85640 1 --2- 172.21.15.43:0/61485739 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f13040cb0f0 0x7f1304153510 unknown :-1 s=CLOSED pgs=1436 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:53:31.576 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:31.573+0000 7f1309e85640 1 -- 172.21.15.43:0/61485739 >> 172.21.15.43:0/61485739 conn(0x7f130406a8d0 msgr2=0x7f130415cd00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-09T13:53:31.576 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:31.573+0000 7f1309e85640 1 -- 172.21.15.43:0/61485739 shutdown_connections 2023-08-09T13:53:31.576 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:31.573+0000 7f1309e85640 1 -- 172.21.15.43:0/61485739 wait complete. 2023-08-09T13:53:31.595 INFO:tasks.cephfs.filesystem:Destroying file system cephfs and related pools 2023-08-09T13:53:31.595 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-08-09T13:53:31.826 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:31.825+0000 7f68e8283640 1 Processor -- start 2023-08-09T13:53:31.826 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:31.825+0000 7f68e8283640 1 -- start start 2023-08-09T13:53:31.828 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:31.827+0000 7f68e8283640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f68e00cb6b0 0x7f68e0165870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:53:31.829 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:31.827+0000 7f68e8283640 1 -- --> v1:172.21.15.43:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f68e00cd8a0 con 0x7f68e00ce080 2023-08-09T13:53:31.829 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:31.827+0000 7f68e8283640 1 -- --> v1:172.21.15.203:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f68e00cf350 con 0x7f68e00ce720 2023-08-09T13:53:31.829 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:31.827+0000 7f68e8283640 1 -- --> v2:172.21.15.43:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f68e0071a70 con 0x7f68e00cb6b0 2023-08-09T13:53:31.829 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:31.827+0000 7f68e6820640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f68e00cb6b0 0x7f68e0165870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:53:31.830 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:31.827+0000 7f68e581e640 1 --1- >> v1:172.21.15.43:6789/0 conn(0x7f68e00ce080 0x7f68e00cafa0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.43:6789/0 says I am v1:172.21.15.43:41006/0 (socket says 172.21.15.43:41006) 2023-08-09T13:53:31.830 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:31.827+0000 7f68e581e640 1 -- 172.21.15.43:0/3938014290 learned_addr learned my addr 172.21.15.43:0/3938014290 (peer_addr_for_me v1:172.21.15.43:0/0) 2023-08-09T13:53:31.832 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:31.828+0000 7f68e6820640 1 -- 172.21.15.43:0/3938014290 >> v1:172.21.15.203:6789/0 conn(0x7f68e00ce720 legacy=0x7f68e00ceb00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:53:31.832 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:31.828+0000 7f68e6820640 1 -- 172.21.15.43:0/3938014290 >> v1:172.21.15.43:6789/0 conn(0x7f68e00ce080 legacy=0x7f68e00cafa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:53:31.832 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:31.828+0000 7f68e6820640 1 -- 172.21.15.43:0/3938014290 --> v2:172.21.15.43:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f68e00af260 con 0x7f68e00cb6b0 2023-08-09T13:53:31.833 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:31.828+0000 7f68e501d640 1 -- 172.21.15.43:0/3938014290 <== mon.0 v1:172.21.15.43:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 268791049 0 0) 0x7f68e00cd8a0 con 0x7f68e00ce080 2023-08-09T13:53:31.833 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:31.829+0000 7f68e6820640 1 --2- 172.21.15.43:0/3938014290 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f68e00cb6b0 0x7f68e0165870 secure :-1 s=READY pgs=1438 cs=0 l=1 rev1=1 crypto rx=0x7f68d8009dc0 tx=0x7f68d802be90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6e43c20c8be4b9d server_cookie=0 in_seq=0 out_seq=0 2023-08-09T13:53:31.833 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:31.829+0000 7f68e501d640 1 -- 172.21.15.43:0/3938014290 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f68d802cc40 con 0x7f68e00cb6b0 2023-08-09T13:53:31.834 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:31.829+0000 7f68e501d640 1 -- 172.21.15.43:0/3938014290 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f68d802cda0 con 0x7f68e00cb6b0 2023-08-09T13:53:31.834 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:31.829+0000 7f68e501d640 1 -- 172.21.15.43:0/3938014290 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f68d8034660 con 0x7f68e00cb6b0 2023-08-09T13:53:31.834 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:31.830+0000 7f68e8283640 1 -- 172.21.15.43:0/3938014290 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f68e00cb6b0 msgr2=0x7f68e0165870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:53:31.834 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:31.830+0000 7f68e8283640 1 --2- 172.21.15.43:0/3938014290 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f68e00cb6b0 0x7f68e0165870 secure :-1 s=READY pgs=1438 cs=0 l=1 rev1=1 crypto rx=0x7f68d8009dc0 tx=0x7f68d802be90 comp rx=0 tx=0).stop 2023-08-09T13:53:31.835 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:31.830+0000 7f68e8283640 1 -- 172.21.15.43:0/3938014290 shutdown_connections 2023-08-09T13:53:31.835 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:31.830+0000 7f68e8283640 1 --2- 172.21.15.43:0/3938014290 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f68e00cb6b0 0x7f68e0165870 unknown :-1 s=CLOSED pgs=1438 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:53:31.835 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:31.830+0000 7f68e8283640 1 -- 172.21.15.43:0/3938014290 >> 172.21.15.43:0/3938014290 conn(0x7f68e00ccd80 msgr2=0x7f68e00cd190 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-09T13:53:31.835 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:31.830+0000 7f68e8283640 1 -- 172.21.15.43:0/3938014290 shutdown_connections 2023-08-09T13:53:31.836 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:31.830+0000 7f68e8283640 1 -- 172.21.15.43:0/3938014290 wait complete. 2023-08-09T13:53:31.839 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:31.832+0000 7f68e8283640 1 Processor -- start 2023-08-09T13:53:31.839 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:31.832+0000 7f68e8283640 1 -- start start 2023-08-09T13:53:31.839 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:31.833+0000 7f68e8283640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f68e00cb6b0 0x7f68e00b4020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:53:31.839 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:31.833+0000 7f68e8283640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f68e00ce080 0x7f68e0150830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:53:31.840 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:31.833+0000 7f68e8283640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f68e00ce720 0x7f68e00b24e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:53:31.840 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:31.833+0000 7f68e8283640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f68e0150d70 con 0x7f68e00ce720 2023-08-09T13:53:31.840 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:31.833+0000 7f68e8283640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f68e00b1590 con 0x7f68e00cb6b0 2023-08-09T13:53:31.840 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:31.833+0000 7f68e8283640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f68e00b12b0 con 0x7f68e00ce080 2023-08-09T13:53:31.841 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:31.833+0000 7f68e6820640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f68e00ce720 0x7f68e00b24e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:53:31.841 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:31.833+0000 7f68e6820640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f68e00ce720 0x7f68e00b24e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:58322/0 (socket says 172.21.15.43:58322) 2023-08-09T13:53:31.841 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:31.833+0000 7f68e6820640 1 -- 172.21.15.43:0/2345989465 learned_addr learned my addr 172.21.15.43:0/2345989465 (peer_addr_for_me v2:172.21.15.43:0/0) 2023-08-09T13:53:31.841 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:31.834+0000 7f68e581e640 1 --2- 172.21.15.43:0/2345989465 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f68e00ce080 0x7f68e0150830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:53:31.842 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:31.834+0000 7f68e6820640 1 -- 172.21.15.43:0/2345989465 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f68e00ce080 msgr2=0x7f68e0150830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:53:31.842 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:31.834+0000 7f68e6820640 1 --2- 172.21.15.43:0/2345989465 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f68e00ce080 0x7f68e0150830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:53:31.842 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:31.834+0000 7f68e6820640 1 -- 172.21.15.43:0/2345989465 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f68e00cb6b0 msgr2=0x7f68e00b4020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:53:31.842 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:31.834+0000 7f68e601f640 1 --2- 172.21.15.43:0/2345989465 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f68e00cb6b0 0x7f68e00b4020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:53:31.843 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:31.834+0000 7f68e6820640 1 --2- 172.21.15.43:0/2345989465 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f68e00cb6b0 0x7f68e00b4020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:53:31.843 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:31.834+0000 7f68e6820640 1 -- 172.21.15.43:0/2345989465 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f68d8009a70 con 0x7f68e00ce720 2023-08-09T13:53:31.843 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:31.834+0000 7f68e6820640 1 --2- 172.21.15.43:0/2345989465 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f68e00ce720 0x7f68e00b24e0 secure :-1 s=READY pgs=1439 cs=0 l=1 rev1=1 crypto rx=0x7f68d802c4a0 tx=0x7f68d8004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-09T13:53:31.843 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:31.834+0000 7f68deffd640 1 -- 172.21.15.43:0/2345989465 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f68d803e610 con 0x7f68e00ce720 2023-08-09T13:53:31.844 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:31.834+0000 7f68deffd640 1 -- 172.21.15.43:0/2345989465 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f68d803e770 con 0x7f68e00ce720 2023-08-09T13:53:31.844 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:31.834+0000 7f68deffd640 1 -- 172.21.15.43:0/2345989465 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f68d803eb20 con 0x7f68e00ce720 2023-08-09T13:53:31.844 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:31.835+0000 7f68e8283640 1 -- 172.21.15.43:0/2345989465 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f68e00b1a00 con 0x7f68e00ce720 2023-08-09T13:53:31.845 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:31.835+0000 7f68e8283640 1 -- 172.21.15.43:0/2345989465 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f68e00b1fc0 con 0x7f68e00ce720 2023-08-09T13:53:31.845 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:31.836+0000 7f68e8283640 1 -- 172.21.15.43:0/2345989465 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f68e00739f0 con 0x7f68e00ce720 2023-08-09T13:53:31.845 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:31.837+0000 7f68deffd640 1 -- 172.21.15.43:0/2345989465 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 55) v1 ==== 88387+0+0 (secure 0 0 0) 0x7f68d803b050 con 0x7f68e00ce720 2023-08-09T13:53:31.846 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:31.838+0000 7f68deffd640 1 --2- 172.21.15.43:0/2345989465 >> [v2:172.21.15.203:6832/2527059680,v1:172.21.15.203:6833/2527059680] conn(0x7f68cc06a0e0 0x7f68cc06c5a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:53:31.847 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:31.838+0000 7f68deffd640 1 -- 172.21.15.43:0/2345989465 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(118..118 src has 1..118) v4 ==== 7200+0+0 (secure 0 0 0) 0x7f68d80aaee0 con 0x7f68e00ce720 2023-08-09T13:53:31.847 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:31.839+0000 7f68e601f640 1 --2- 172.21.15.43:0/2345989465 >> [v2:172.21.15.203:6832/2527059680,v1:172.21.15.203:6833/2527059680] conn(0x7f68cc06a0e0 0x7f68cc06c5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:53:31.847 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:31.840+0000 7f68e601f640 1 --2- 172.21.15.43:0/2345989465 >> [v2:172.21.15.203:6832/2527059680,v1:172.21.15.203:6833/2527059680] conn(0x7f68cc06a0e0 0x7f68cc06c5a0 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7f68d0000fb0 tx=0x7f68d0009040 comp rx=0 tx=0).ready entity=mgr.6377 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-09T13:53:31.848 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:31.842+0000 7f68deffd640 1 -- 172.21.15.43:0/2345989465 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178066 (secure 0 0 0) 0x7f68d8076090 con 0x7f68e00ce720 2023-08-09T13:53:32.078 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:32.077+0000 7f68e8283640 1 -- 172.21.15.43:0/2345989465 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f68e008a410 con 0x7f68e00ce720 2023-08-09T13:53:32.080 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:32.078+0000 7f68deffd640 1 -- 172.21.15.43:0/2345989465 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 52 v52) v1 ==== 94+0+4883 (secure 0 0 0) 0x7f68d8039090 con 0x7f68e00ce720 2023-08-09T13:53:32.080 INFO:teuthology.orchestra.run.smithi043.stdout: 2023-08-09T13:53:32.080 INFO:teuthology.orchestra.run.smithi043.stdout:{"epoch":52,"default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6215,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.43:6835/2214346432","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6833","nonce":2214346432},{"type":"v1","addr":"172.21.15.43:6835","nonce":2214346432}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":47},{"gid":6823,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.203:6835/3236264045","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6834","nonce":3236264045},{"type":"v1","addr":"172.21.15.203:6835","nonce":3236264045}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":46},{"gid":6826,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.203:6837/3788289156","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6836","nonce":3788289156},{"type":"v1","addr":"172.21.15.203:6837","nonce":3788289156}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":47}],"filesystems":[{"mdsmap":{"epoch":52,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-08-09T13:51:59.655630+0000","modified":"2023-08-09T13:52:03.220527+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":8040},"failed":[],"damaged":[],"stopped":[],"info":{"gid_8040":{"gid":8040,"name":"c","rank":0,"incarnation":48,"state":"up:active","state_seq":3,"addr":"172.21.15.43:6834/2223580538","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6832","nonce":2223580538},{"type":"v1","addr":"172.21.15.43:6834","nonce":2223580538}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[15],"metadata_pool":14,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":6}]} 2023-08-09T13:53:32.081 INFO:teuthology.orchestra.run.smithi043.stderr:dumped fsmap epoch 52 2023-08-09T13:53:32.082 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:32.081+0000 7f68e8283640 1 -- 172.21.15.43:0/2345989465 >> [v2:172.21.15.203:6832/2527059680,v1:172.21.15.203:6833/2527059680] conn(0x7f68cc06a0e0 msgr2=0x7f68cc06c5a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:53:32.083 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:32.081+0000 7f68e8283640 1 --2- 172.21.15.43:0/2345989465 >> [v2:172.21.15.203:6832/2527059680,v1:172.21.15.203:6833/2527059680] conn(0x7f68cc06a0e0 0x7f68cc06c5a0 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7f68d0000fb0 tx=0x7f68d0009040 comp rx=0 tx=0).stop 2023-08-09T13:53:32.083 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:32.081+0000 7f68e8283640 1 -- 172.21.15.43:0/2345989465 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f68e00ce720 msgr2=0x7f68e00b24e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:53:32.083 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:32.081+0000 7f68e8283640 1 --2- 172.21.15.43:0/2345989465 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f68e00ce720 0x7f68e00b24e0 secure :-1 s=READY pgs=1439 cs=0 l=1 rev1=1 crypto rx=0x7f68d802c4a0 tx=0x7f68d8004a40 comp rx=0 tx=0).stop 2023-08-09T13:53:32.083 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:32.081+0000 7f68e8283640 1 -- 172.21.15.43:0/2345989465 shutdown_connections 2023-08-09T13:53:32.084 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:32.081+0000 7f68e8283640 1 --2- 172.21.15.43:0/2345989465 >> [v2:172.21.15.203:6832/2527059680,v1:172.21.15.203:6833/2527059680] conn(0x7f68cc06a0e0 0x7f68cc06c5a0 unknown :-1 s=CLOSED pgs=27 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:53:32.084 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:32.081+0000 7f68e8283640 1 --2- 172.21.15.43:0/2345989465 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f68e00ce720 0x7f68e00b24e0 unknown :-1 s=CLOSED pgs=1439 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:53:32.084 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:32.081+0000 7f68e8283640 1 --2- 172.21.15.43:0/2345989465 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f68e00ce080 0x7f68e0150830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:53:32.084 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:32.081+0000 7f68e8283640 1 --2- 172.21.15.43:0/2345989465 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f68e00cb6b0 0x7f68e00b4020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:53:32.085 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:32.081+0000 7f68e8283640 1 -- 172.21.15.43:0/2345989465 >> 172.21.15.43:0/2345989465 conn(0x7f68e00ccd80 msgr2=0x7f68e00acef0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-09T13:53:32.085 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:32.081+0000 7f68e8283640 1 -- 172.21.15.43:0/2345989465 shutdown_connections 2023-08-09T13:53:32.085 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:32.082+0000 7f68e8283640 1 -- 172.21.15.43:0/2345989465 wait complete. 2023-08-09T13:53:32.104 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-08-09T13:53:32.331 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:32.330+0000 7fc99cecc640 1 Processor -- start 2023-08-09T13:53:32.332 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:32.330+0000 7fc99cecc640 1 -- start start 2023-08-09T13:53:32.333 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:32.331+0000 7fc99cecc640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fc9980ce080 0x7fc9980cafa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:53:32.333 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:32.331+0000 7fc99cecc640 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7fc9980cb4e0 0x7fc9980cb990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:53:32.333 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:32.331+0000 7fc99cecc640 1 -- --> v1:172.21.15.43:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc9980cd8a0 con 0x7fc9980ce720 2023-08-09T13:53:32.333 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:32.331+0000 7fc99cecc640 1 -- --> v2:172.21.15.203:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc9980cbed0 con 0x7fc9980cb4e0 2023-08-09T13:53:32.333 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:32.331+0000 7fc99cecc640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc9980cf350 con 0x7fc9980ce080 2023-08-09T13:53:32.334 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:32.332+0000 7fc99659c640 1 --1- >> v1:172.21.15.43:6789/0 conn(0x7fc9980ce720 0x7fc9980ceb00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.43:6789/0 says I am v1:172.21.15.43:41022/0 (socket says 172.21.15.43:41022) 2023-08-09T13:53:32.334 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:32.332+0000 7fc99659c640 1 -- 172.21.15.43:0/2565232605 learned_addr learned my addr 172.21.15.43:0/2565232605 (peer_addr_for_me v1:172.21.15.43:0/0) 2023-08-09T13:53:32.335 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:32.332+0000 7fc995d9b640 1 --2- 172.21.15.43:0/2565232605 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fc9980ce080 0x7fc9980cafa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:53:32.335 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:32.332+0000 7fc996d9d640 1 --2- 172.21.15.43:0/2565232605 >> v2:172.21.15.203:3300/0 conn(0x7fc9980cb4e0 0x7fc9980cb990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:53:32.335 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:32.332+0000 7fc99559a640 1 -- 172.21.15.43:0/2565232605 <== mon.0 v1:172.21.15.43:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1650029244 0 0) 0x7fc9980cd8a0 con 0x7fc9980ce720 2023-08-09T13:53:32.335 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:32.332+0000 7fc99559a640 1 -- 172.21.15.43:0/2565232605 --> v1:172.21.15.43:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fc984003520 con 0x7fc9980ce720 2023-08-09T13:53:32.335 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:32.333+0000 7fc99559a640 1 -- 172.21.15.43:0/2565232605 <== mon.0 v1:172.21.15.43:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2905394689 0 0) 0x7fc984003520 con 0x7fc9980ce720 2023-08-09T13:53:32.335 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:32.333+0000 7fc99559a640 1 -- 172.21.15.43:0/2565232605 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fc9980ce080 msgr2=0x7fc9980cafa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-09T13:53:32.335 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:32.333+0000 7fc99559a640 1 --2- 172.21.15.43:0/2565232605 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fc9980ce080 0x7fc9980cafa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:53:32.336 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:32.333+0000 7fc99559a640 1 -- 172.21.15.43:0/2565232605 >> v2:172.21.15.203:3300/0 conn(0x7fc9980cb4e0 msgr2=0x7fc9980cb990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-09T13:53:32.336 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:32.333+0000 7fc99559a640 1 --2- 172.21.15.43:0/2565232605 >> v2:172.21.15.203:3300/0 conn(0x7fc9980cb4e0 0x7fc9980cb990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:53:32.336 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:32.333+0000 7fc99559a640 1 -- 172.21.15.43:0/2565232605 --> v1:172.21.15.43:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc998071a70 con 0x7fc9980ce720 2023-08-09T13:53:32.336 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:32.333+0000 7fc99559a640 1 -- 172.21.15.43:0/2565232605 <== mon.0 v1:172.21.15.43:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1285150740 0 0) 0x7fc988001b70 con 0x7fc9980ce720 2023-08-09T13:53:32.336 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:32.333+0000 7fc99559a640 1 -- 172.21.15.43:0/2565232605 >> v1:172.21.15.43:6789/0 conn(0x7fc9980ce720 legacy=0x7fc9980ceb00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:53:32.336 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:32.333+0000 7fc99559a640 1 --2- 172.21.15.43:0/2565232605 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc9840039e0 0x7fc984003e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:53:32.336 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:32.333+0000 7fc99559a640 1 --2- 172.21.15.43:0/2565232605 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fc9980ce080 0x7fc9840047c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:53:32.337 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:32.333+0000 7fc99559a640 1 --2- 172.21.15.43:0/2565232605 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fc984004d00 0x7fc984005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:53:32.337 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:32.333+0000 7fc99559a640 1 -- 172.21.15.43:0/2565232605 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc9980cbed0 con 0x7fc9840039e0 2023-08-09T13:53:32.337 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:32.333+0000 7fc99559a640 1 -- 172.21.15.43:0/2565232605 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc9980cf350 con 0x7fc984004d00 2023-08-09T13:53:32.337 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:32.333+0000 7fc99559a640 1 -- 172.21.15.43:0/2565232605 --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc98400b9d0 con 0x7fc9980ce080 2023-08-09T13:53:32.338 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:32.333+0000 7fc99659c640 1 --2- 172.21.15.43:0/2565232605 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc9840039e0 0x7fc984003e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:53:32.338 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:32.334+0000 7fc995d9b640 1 --2- 172.21.15.43:0/2565232605 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fc9980ce080 0x7fc9840047c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:53:32.339 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:32.334+0000 7fc99659c640 1 -- 172.21.15.43:0/2565232605 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fc9980ce080 msgr2=0x7fc9840047c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-09T13:53:32.339 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:32.334+0000 7fc996d9d640 1 --2- 172.21.15.43:0/2565232605 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fc984004d00 0x7fc984005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:53:32.339 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:32.334+0000 7fc99659c640 1 --2- 172.21.15.43:0/2565232605 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fc9980ce080 0x7fc9840047c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:53:32.339 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:32.334+0000 7fc99659c640 1 -- 172.21.15.43:0/2565232605 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fc984004d00 msgr2=0x7fc984005180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-09T13:53:32.339 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:32.334+0000 7fc99659c640 1 --2- 172.21.15.43:0/2565232605 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fc984004d00 0x7fc984005180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:53:32.339 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:32.334+0000 7fc99659c640 1 -- 172.21.15.43:0/2565232605 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc984003520 con 0x7fc9840039e0 2023-08-09T13:53:32.339 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:32.334+0000 7fc996d9d640 1 --2- 172.21.15.43:0/2565232605 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fc984004d00 0x7fc984005180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-08-09T13:53:32.339 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:32.335+0000 7fc99659c640 1 --2- 172.21.15.43:0/2565232605 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc9840039e0 0x7fc984003e00 secure :-1 s=READY pgs=1441 cs=0 l=1 rev1=1 crypto rx=0x7fc98800c540 tx=0x7fc9840057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2b5d4c18c6e9385c server_cookie=0 in_seq=0 out_seq=0 2023-08-09T13:53:32.340 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:32.336+0000 7fc99559a640 1 -- 172.21.15.43:0/2565232605 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc988005b20 con 0x7fc9840039e0 2023-08-09T13:53:32.340 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:32.336+0000 7fc99559a640 1 -- 172.21.15.43:0/2565232605 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc988005c80 con 0x7fc9840039e0 2023-08-09T13:53:32.340 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:32.336+0000 7fc99559a640 1 -- 172.21.15.43:0/2565232605 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc9880072b0 con 0x7fc9840039e0 2023-08-09T13:53:32.340 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:32.336+0000 7fc99cecc640 1 -- 172.21.15.43:0/2565232605 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc9840039e0 msgr2=0x7fc984003e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:53:32.341 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:32.336+0000 7fc99cecc640 1 --2- 172.21.15.43:0/2565232605 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc9840039e0 0x7fc984003e00 secure :-1 s=READY pgs=1441 cs=0 l=1 rev1=1 crypto rx=0x7fc98800c540 tx=0x7fc9840057f0 comp rx=0 tx=0).stop 2023-08-09T13:53:32.341 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:32.336+0000 7fc996d9d640 1 -- 172.21.15.43:0/2565232605 reap_dead start 2023-08-09T13:53:32.341 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:32.336+0000 7fc99cecc640 1 -- 172.21.15.43:0/2565232605 shutdown_connections 2023-08-09T13:53:32.341 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:32.336+0000 7fc99cecc640 1 -- 172.21.15.43:0/2565232605 >> 172.21.15.43:0/2565232605 conn(0x7fc9980ccd80 msgr2=0x7fc9980cd190 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-09T13:53:32.341 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:32.337+0000 7fc99cecc640 1 -- 172.21.15.43:0/2565232605 shutdown_connections 2023-08-09T13:53:32.341 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:32.337+0000 7fc99cecc640 1 -- 172.21.15.43:0/2565232605 wait complete. 2023-08-09T13:53:32.343 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:32.338+0000 7fc99cecc640 1 Processor -- start 2023-08-09T13:53:32.343 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:32.339+0000 7fc99cecc640 1 -- start start 2023-08-09T13:53:32.343 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:32.340+0000 7fc99cecc640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fc9840039e0 0x7fc998147500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:53:32.343 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:32.340+0000 7fc99cecc640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc9980b4ed0 0x7fc9980b5350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:53:32.343 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:32.340+0000 7fc99cecc640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fc998149740 0x7fc998149bc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:53:32.343 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:32.340+0000 7fc99cecc640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc9980c7340 con 0x7fc9980b4ed0 2023-08-09T13:53:32.343 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:32.340+0000 7fc99cecc640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc9980c7550 con 0x7fc998149740 2023-08-09T13:53:32.344 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:32.340+0000 7fc99cecc640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc9980c7830 con 0x7fc9840039e0 2023-08-09T13:53:32.344 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:32.340+0000 7fc995d9b640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc9980b4ed0 0x7fc9980b5350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:53:32.344 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:32.340+0000 7fc995d9b640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc9980b4ed0 0x7fc9980b5350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:58338/0 (socket says 172.21.15.43:58338) 2023-08-09T13:53:32.344 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:32.340+0000 7fc995d9b640 1 -- 172.21.15.43:0/966745993 learned_addr learned my addr 172.21.15.43:0/966745993 (peer_addr_for_me v2:172.21.15.43:0/0) 2023-08-09T13:53:32.344 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:32.340+0000 7fc995d9b640 1 -- 172.21.15.43:0/966745993 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fc9840039e0 msgr2=0x7fc998147500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:53:32.344 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:32.340+0000 7fc995d9b640 1 --2- 172.21.15.43:0/966745993 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fc9840039e0 0x7fc998147500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:53:32.344 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:32.340+0000 7fc995d9b640 1 -- 172.21.15.43:0/966745993 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fc998149740 msgr2=0x7fc998149bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:53:32.345 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:32.340+0000 7fc995d9b640 1 --2- 172.21.15.43:0/966745993 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fc998149740 0x7fc998149bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:53:32.345 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:32.340+0000 7fc995d9b640 1 -- 172.21.15.43:0/966745993 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc9980c7b10 con 0x7fc9980b4ed0 2023-08-09T13:53:32.345 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:32.341+0000 7fc995d9b640 1 --2- 172.21.15.43:0/966745993 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc9980b4ed0 0x7fc9980b5350 secure :-1 s=READY pgs=1442 cs=0 l=1 rev1=1 crypto rx=0x7fc980010890 tx=0x7fc980010d50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-09T13:53:32.345 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:32.341+0000 7fc98effd640 1 -- 172.21.15.43:0/966745993 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc98000eb60 con 0x7fc9980b4ed0 2023-08-09T13:53:32.345 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:32.341+0000 7fc98effd640 1 -- 172.21.15.43:0/966745993 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc98000ecc0 con 0x7fc9980b4ed0 2023-08-09T13:53:32.345 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:32.341+0000 7fc98effd640 1 -- 172.21.15.43:0/966745993 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc98001c400 con 0x7fc9980b4ed0 2023-08-09T13:53:32.345 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:32.341+0000 7fc99cecc640 1 -- 172.21.15.43:0/966745993 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc9980c7df0 con 0x7fc9980b4ed0 2023-08-09T13:53:32.345 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:32.341+0000 7fc99cecc640 1 -- 172.21.15.43:0/966745993 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc99814a100 con 0x7fc9980b4ed0 2023-08-09T13:53:32.346 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:32.342+0000 7fc99cecc640 1 -- 172.21.15.43:0/966745993 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc9980739f0 con 0x7fc9980b4ed0 2023-08-09T13:53:32.346 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:32.343+0000 7fc98effd640 1 -- 172.21.15.43:0/966745993 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 55) v1 ==== 88387+0+0 (secure 0 0 0) 0x7fc98001c560 con 0x7fc9980b4ed0 2023-08-09T13:53:32.346 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:32.344+0000 7fc98effd640 1 --2- 172.21.15.43:0/966745993 >> [v2:172.21.15.203:6832/2527059680,v1:172.21.15.203:6833/2527059680] conn(0x7fc97806a110 0x7fc97806c5d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:53:32.347 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:32.344+0000 7fc98effd640 1 -- 172.21.15.43:0/966745993 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(118..118 src has 1..118) v4 ==== 7200+0+0 (secure 0 0 0) 0x7fc980088f80 con 0x7fc9980b4ed0 2023-08-09T13:53:32.347 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:32.345+0000 7fc99659c640 1 --2- 172.21.15.43:0/966745993 >> [v2:172.21.15.203:6832/2527059680,v1:172.21.15.203:6833/2527059680] conn(0x7fc97806a110 0x7fc97806c5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:53:32.347 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:32.346+0000 7fc99659c640 1 --2- 172.21.15.43:0/966745993 >> [v2:172.21.15.203:6832/2527059680,v1:172.21.15.203:6833/2527059680] conn(0x7fc97806a110 0x7fc97806c5d0 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7fc988034000 tx=0x7fc98802e000 comp rx=0 tx=0).ready entity=mgr.6377 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-09T13:53:32.350 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:32.349+0000 7fc98effd640 1 -- 172.21.15.43:0/966745993 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178066 (secure 0 0 0) 0x7fc9800541e0 con 0x7fc9980b4ed0 2023-08-09T13:53:32.581 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:32.580+0000 7fc99cecc640 1 -- 172.21.15.43:0/966745993 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fc99814a380 con 0x7fc9980b4ed0 2023-08-09T13:53:32.582 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:32.581+0000 7fc98effd640 1 -- 172.21.15.43:0/966745993 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 52 v52) v1 ==== 94+0+4883 (secure 0 0 0) 0x7fc980057800 con 0x7fc9980b4ed0 2023-08-09T13:53:32.582 INFO:teuthology.orchestra.run.smithi043.stdout: 2023-08-09T13:53:32.583 INFO:teuthology.orchestra.run.smithi043.stdout:{"epoch":52,"default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6215,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.43:6835/2214346432","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6833","nonce":2214346432},{"type":"v1","addr":"172.21.15.43:6835","nonce":2214346432}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":47},{"gid":6823,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.203:6835/3236264045","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6834","nonce":3236264045},{"type":"v1","addr":"172.21.15.203:6835","nonce":3236264045}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":46},{"gid":6826,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.203:6837/3788289156","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6836","nonce":3788289156},{"type":"v1","addr":"172.21.15.203:6837","nonce":3788289156}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":47}],"filesystems":[{"mdsmap":{"epoch":52,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-08-09T13:51:59.655630+0000","modified":"2023-08-09T13:52:03.220527+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":8040},"failed":[],"damaged":[],"stopped":[],"info":{"gid_8040":{"gid":8040,"name":"c","rank":0,"incarnation":48,"state":"up:active","state_seq":3,"addr":"172.21.15.43:6834/2223580538","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6832","nonce":2223580538},{"type":"v1","addr":"172.21.15.43:6834","nonce":2223580538}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[15],"metadata_pool":14,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":6}]} 2023-08-09T13:53:32.583 INFO:teuthology.orchestra.run.smithi043.stderr:dumped fsmap epoch 52 2023-08-09T13:53:32.585 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:32.583+0000 7fc99cecc640 1 -- 172.21.15.43:0/966745993 >> [v2:172.21.15.203:6832/2527059680,v1:172.21.15.203:6833/2527059680] conn(0x7fc97806a110 msgr2=0x7fc97806c5d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:53:32.585 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:32.583+0000 7fc99cecc640 1 --2- 172.21.15.43:0/966745993 >> [v2:172.21.15.203:6832/2527059680,v1:172.21.15.203:6833/2527059680] conn(0x7fc97806a110 0x7fc97806c5d0 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7fc988034000 tx=0x7fc98802e000 comp rx=0 tx=0).stop 2023-08-09T13:53:32.585 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:32.583+0000 7fc99cecc640 1 -- 172.21.15.43:0/966745993 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc9980b4ed0 msgr2=0x7fc9980b5350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:53:32.585 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:32.583+0000 7fc99cecc640 1 --2- 172.21.15.43:0/966745993 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc9980b4ed0 0x7fc9980b5350 secure :-1 s=READY pgs=1442 cs=0 l=1 rev1=1 crypto rx=0x7fc980010890 tx=0x7fc980010d50 comp rx=0 tx=0).stop 2023-08-09T13:53:32.585 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:32.583+0000 7fc99cecc640 1 -- 172.21.15.43:0/966745993 shutdown_connections 2023-08-09T13:53:32.586 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:32.583+0000 7fc99cecc640 1 --2- 172.21.15.43:0/966745993 >> [v2:172.21.15.203:6832/2527059680,v1:172.21.15.203:6833/2527059680] conn(0x7fc97806a110 0x7fc97806c5d0 unknown :-1 s=CLOSED pgs=28 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:53:32.586 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:32.583+0000 7fc99cecc640 1 --2- 172.21.15.43:0/966745993 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fc998149740 0x7fc998149bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:53:32.586 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:32.583+0000 7fc99cecc640 1 --2- 172.21.15.43:0/966745993 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc9980b4ed0 0x7fc9980b5350 unknown :-1 s=CLOSED pgs=1442 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:53:32.586 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:32.583+0000 7fc99cecc640 1 --2- 172.21.15.43:0/966745993 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fc9840039e0 0x7fc998147500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:53:32.586 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:32.583+0000 7fc99cecc640 1 -- 172.21.15.43:0/966745993 >> 172.21.15.43:0/966745993 conn(0x7fc9980ccd80 msgr2=0x7fc9980cc010 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-09T13:53:32.586 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:32.584+0000 7fc99cecc640 1 -- 172.21.15.43:0/966745993 shutdown_connections 2023-08-09T13:53:32.586 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:32.584+0000 7fc99cecc640 1 -- 172.21.15.43:0/966745993 wait complete. 2023-08-09T13:53:32.603 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-08-09T13:53:32.828 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:32.826+0000 7f15aceb0640 1 Processor -- start 2023-08-09T13:53:32.828 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:32.827+0000 7f15aceb0640 1 -- start start 2023-08-09T13:53:32.830 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:32.828+0000 7f15aceb0640 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7f15a815f7b0 0x7f15a80b4760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:53:32.830 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:32.828+0000 7f15aceb0640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f15a80cd6b0 0x7f15a80d0e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:53:32.830 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:32.828+0000 7f15aceb0640 1 -- --> v1:172.21.15.43:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f15a80b7bf0 con 0x7f15a80b1050 2023-08-09T13:53:32.830 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:32.828+0000 7f15aceb0640 1 -- --> v2:172.21.15.203:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f15a80b7ed0 con 0x7f15a815f7b0 2023-08-09T13:53:32.830 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:32.828+0000 7f15aceb0640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f15a80d0370 con 0x7f15a80cd6b0 2023-08-09T13:53:32.830 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:32.828+0000 7f159ffff640 1 --1- >> v1:172.21.15.43:6789/0 conn(0x7f15a80b1050 0x7f15a80ccfa0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.43:6789/0 says I am v1:172.21.15.43:41038/0 (socket says 172.21.15.43:41038) 2023-08-09T13:53:32.831 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:32.828+0000 7f159ffff640 1 -- 172.21.15.43:0/2596879178 learned_addr learned my addr 172.21.15.43:0/2596879178 (peer_addr_for_me v1:172.21.15.43:0/0) 2023-08-09T13:53:32.832 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:32.829+0000 7f15a7fff640 1 --2- 172.21.15.43:0/2596879178 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f15a80cd6b0 0x7f15a80d0e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:53:32.832 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:32.829+0000 7f15a77fe640 1 --2- 172.21.15.43:0/2596879178 >> v2:172.21.15.203:3300/0 conn(0x7f15a815f7b0 0x7f15a80b4760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:53:32.833 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:32.829+0000 7f15a6ffd640 1 -- 172.21.15.43:0/2596879178 <== mon.0 v1:172.21.15.43:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3506287601 0 0) 0x7f15a80b7bf0 con 0x7f15a80b1050 2023-08-09T13:53:32.833 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:32.829+0000 7f15a6ffd640 1 -- 172.21.15.43:0/2596879178 --> v1:172.21.15.43:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f1594003520 con 0x7f15a80b1050 2023-08-09T13:53:32.833 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:32.829+0000 7f15a6ffd640 1 -- 172.21.15.43:0/2596879178 <== mon.0 v1:172.21.15.43:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2988242389 0 0) 0x7f1594003520 con 0x7f15a80b1050 2023-08-09T13:53:32.833 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:32.829+0000 7f15a6ffd640 1 -- 172.21.15.43:0/2596879178 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f15a80cd6b0 msgr2=0x7f15a80d0e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-09T13:53:32.833 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:32.829+0000 7f15a6ffd640 1 --2- 172.21.15.43:0/2596879178 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f15a80cd6b0 0x7f15a80d0e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:53:32.833 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:32.829+0000 7f15a6ffd640 1 -- 172.21.15.43:0/2596879178 >> v2:172.21.15.203:3300/0 conn(0x7f15a815f7b0 msgr2=0x7f15a80b4760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-09T13:53:32.833 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:32.829+0000 7f15a6ffd640 1 --2- 172.21.15.43:0/2596879178 >> v2:172.21.15.203:3300/0 conn(0x7f15a815f7b0 0x7f15a80b4760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:53:32.833 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:32.829+0000 7f15a6ffd640 1 -- 172.21.15.43:0/2596879178 --> v1:172.21.15.43:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f15a80d0650 con 0x7f15a80b1050 2023-08-09T13:53:32.834 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:32.829+0000 7f15a6ffd640 1 -- 172.21.15.43:0/2596879178 <== mon.0 v1:172.21.15.43:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1285150740 0 0) 0x7f1590001670 con 0x7f15a80b1050 2023-08-09T13:53:32.834 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:32.829+0000 7f15a6ffd640 1 -- 172.21.15.43:0/2596879178 >> v1:172.21.15.43:6789/0 conn(0x7f15a80b1050 legacy=0x7f15a80ccfa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:53:32.834 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:32.829+0000 7f15a6ffd640 1 --2- 172.21.15.43:0/2596879178 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f15a80cd6b0 0x7f1594003970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:53:32.834 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:32.829+0000 7f15a6ffd640 1 --2- 172.21.15.43:0/2596879178 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f1594003ef0 0x7f15940047c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:53:32.834 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:32.829+0000 7f15a6ffd640 1 --2- 172.21.15.43:0/2596879178 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f1594004d00 0x7f1594005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:53:32.834 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:32.829+0000 7f15a6ffd640 1 -- 172.21.15.43:0/2596879178 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f15a80b7ed0 con 0x7f1594004d00 2023-08-09T13:53:32.834 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:32.829+0000 7f15a6ffd640 1 -- 172.21.15.43:0/2596879178 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f15a80d0370 con 0x7f1594003ef0 2023-08-09T13:53:32.835 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:32.829+0000 7f15a6ffd640 1 -- 172.21.15.43:0/2596879178 --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f159400b980 con 0x7f15a80cd6b0 2023-08-09T13:53:32.835 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:32.830+0000 7f15a7fff640 1 --2- 172.21.15.43:0/2596879178 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f1594004d00 0x7f1594005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:53:32.835 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:32.830+0000 7f15a77fe640 1 --2- 172.21.15.43:0/2596879178 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f15a80cd6b0 0x7f1594003970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:53:32.835 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:32.830+0000 7f159ffff640 1 --2- 172.21.15.43:0/2596879178 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f1594003ef0 0x7f15940047c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:53:32.835 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:32.830+0000 7f15a7fff640 1 -- 172.21.15.43:0/2596879178 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f15a80cd6b0 msgr2=0x7f1594003970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-09T13:53:32.835 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:32.830+0000 7f15a7fff640 1 --2- 172.21.15.43:0/2596879178 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f15a80cd6b0 0x7f1594003970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:53:32.835 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:32.830+0000 7f15a7fff640 1 -- 172.21.15.43:0/2596879178 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f1594003ef0 msgr2=0x7f15940047c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-09T13:53:32.835 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:32.830+0000 7f15a7fff640 1 --2- 172.21.15.43:0/2596879178 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f1594003ef0 0x7f15940047c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:53:32.836 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:32.830+0000 7f15a7fff640 1 -- 172.21.15.43:0/2596879178 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1594003520 con 0x7f1594004d00 2023-08-09T13:53:32.836 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:32.830+0000 7f159ffff640 1 --2- 172.21.15.43:0/2596879178 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f1594003ef0 0x7f15940047c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-08-09T13:53:32.837 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:32.831+0000 7f15a7fff640 1 --2- 172.21.15.43:0/2596879178 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f1594004d00 0x7f1594005180 secure :-1 s=READY pgs=1444 cs=0 l=1 rev1=1 crypto rx=0x7f15a0009fc0 tx=0x7f15940057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ed8a9c2317fad0e9 server_cookie=0 in_seq=0 out_seq=0 2023-08-09T13:53:32.838 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:32.832+0000 7f15a6ffd640 1 -- 172.21.15.43:0/2596879178 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f15a002cc00 con 0x7f1594004d00 2023-08-09T13:53:32.838 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:32.832+0000 7f15a6ffd640 1 -- 172.21.15.43:0/2596879178 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f15a000f040 con 0x7f1594004d00 2023-08-09T13:53:32.838 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:32.832+0000 7f15a6ffd640 1 -- 172.21.15.43:0/2596879178 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f15a0034440 con 0x7f1594004d00 2023-08-09T13:53:32.838 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:32.832+0000 7f15aceb0640 1 -- 172.21.15.43:0/2596879178 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f1594004d00 msgr2=0x7f1594005180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:53:32.838 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:32.832+0000 7f15aceb0640 1 --2- 172.21.15.43:0/2596879178 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f1594004d00 0x7f1594005180 secure :-1 s=READY pgs=1444 cs=0 l=1 rev1=1 crypto rx=0x7f15a0009fc0 tx=0x7f15940057f0 comp rx=0 tx=0).stop 2023-08-09T13:53:32.838 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:32.832+0000 7f15a7fff640 1 -- 172.21.15.43:0/2596879178 reap_dead start 2023-08-09T13:53:32.838 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:32.832+0000 7f15aceb0640 1 -- 172.21.15.43:0/2596879178 shutdown_connections 2023-08-09T13:53:32.839 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:32.832+0000 7f15aceb0640 1 -- 172.21.15.43:0/2596879178 >> 172.21.15.43:0/2596879178 conn(0x7f15a806a2b0 msgr2=0x7f15a806a690 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-09T13:53:32.839 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:32.832+0000 7f15aceb0640 1 -- 172.21.15.43:0/2596879178 shutdown_connections 2023-08-09T13:53:32.839 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:32.832+0000 7f15aceb0640 1 -- 172.21.15.43:0/2596879178 wait complete. 2023-08-09T13:53:32.839 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:32.834+0000 7f15aceb0640 1 Processor -- start 2023-08-09T13:53:32.839 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:32.835+0000 7f15aceb0640 1 -- start start 2023-08-09T13:53:32.839 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:32.836+0000 7f15aceb0640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f1594004d00 0x7f15a814f740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:53:32.839 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:32.836+0000 7f15aceb0640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f15a81e56b0 0x7f15a81e5b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:53:32.839 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:32.836+0000 7f15aceb0640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f15a80d0f80 0x7f15a80d1400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:53:32.840 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:32.836+0000 7f15aceb0640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f15a81e6070 con 0x7f15a81e56b0 2023-08-09T13:53:32.840 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:32.836+0000 7f15aceb0640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f15a80d1cf0 con 0x7f15a80d0f80 2023-08-09T13:53:32.840 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:32.836+0000 7f15aceb0640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f15a80b2eb0 con 0x7f1594004d00 2023-08-09T13:53:32.840 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:32.836+0000 7f159ffff640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f15a81e56b0 0x7f15a81e5b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:53:32.840 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:32.836+0000 7f159ffff640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f15a81e56b0 0x7f15a81e5b30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:58364/0 (socket says 172.21.15.43:58364) 2023-08-09T13:53:32.840 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:32.836+0000 7f159ffff640 1 -- 172.21.15.43:0/3565216302 learned_addr learned my addr 172.21.15.43:0/3565216302 (peer_addr_for_me v2:172.21.15.43:0/0) 2023-08-09T13:53:32.840 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:32.836+0000 7f15a77fe640 1 --2- 172.21.15.43:0/3565216302 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f1594004d00 0x7f15a814f740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:53:32.841 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:32.836+0000 7f15a7fff640 1 --2- 172.21.15.43:0/3565216302 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f15a80d0f80 0x7f15a80d1400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:53:32.842 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:32.836+0000 7f159ffff640 1 -- 172.21.15.43:0/3565216302 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f1594004d00 msgr2=0x7f15a814f740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:53:32.842 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:32.836+0000 7f159ffff640 1 --2- 172.21.15.43:0/3565216302 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f1594004d00 0x7f15a814f740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:53:32.842 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:32.836+0000 7f159ffff640 1 -- 172.21.15.43:0/3565216302 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f15a80d0f80 msgr2=0x7f15a80d1400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:53:32.843 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:32.836+0000 7f159ffff640 1 --2- 172.21.15.43:0/3565216302 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f15a80d0f80 0x7f15a80d1400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:53:32.843 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:32.836+0000 7f159ffff640 1 -- 172.21.15.43:0/3565216302 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f15a80b3190 con 0x7f15a81e56b0 2023-08-09T13:53:32.843 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:32.837+0000 7f159ffff640 1 --2- 172.21.15.43:0/3565216302 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f15a81e56b0 0x7f15a81e5b30 secure :-1 s=READY pgs=1445 cs=0 l=1 rev1=1 crypto rx=0x7f159000d8b0 tx=0x7f159000dd70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-09T13:53:32.843 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:32.837+0000 7f15a4ff9640 1 -- 172.21.15.43:0/3565216302 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f15900074e0 con 0x7f15a81e56b0 2023-08-09T13:53:32.843 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:32.837+0000 7f15a4ff9640 1 -- 172.21.15.43:0/3565216302 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1590007640 con 0x7f15a81e56b0 2023-08-09T13:53:32.843 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:32.837+0000 7f15a4ff9640 1 -- 172.21.15.43:0/3565216302 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f15900049e0 con 0x7f15a81e56b0 2023-08-09T13:53:32.843 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:32.837+0000 7f15aceb0640 1 -- 172.21.15.43:0/3565216302 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f15a80b3470 con 0x7f15a81e56b0 2023-08-09T13:53:32.844 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:32.837+0000 7f15aceb0640 1 -- 172.21.15.43:0/3565216302 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f15a80b3c70 con 0x7f15a81e56b0 2023-08-09T13:53:32.844 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:32.838+0000 7f15aceb0640 1 -- 172.21.15.43:0/3565216302 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f15a80739f0 con 0x7f15a81e56b0 2023-08-09T13:53:32.844 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:32.839+0000 7f15a4ff9640 1 -- 172.21.15.43:0/3565216302 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 55) v1 ==== 88387+0+0 (secure 0 0 0) 0x7f1590004b40 con 0x7f15a81e56b0 2023-08-09T13:53:32.844 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:32.840+0000 7f15a4ff9640 1 --2- 172.21.15.43:0/3565216302 >> [v2:172.21.15.203:6832/2527059680,v1:172.21.15.203:6833/2527059680] conn(0x7f158806a0e0 0x7f158806c5a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:53:32.844 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:32.841+0000 7f15a4ff9640 1 -- 172.21.15.43:0/3565216302 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(118..118 src has 1..118) v4 ==== 7200+0+0 (secure 0 0 0) 0x7f1590088870 con 0x7f15a81e56b0 2023-08-09T13:53:32.845 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:32.841+0000 7f15a77fe640 1 --2- 172.21.15.43:0/3565216302 >> [v2:172.21.15.203:6832/2527059680,v1:172.21.15.203:6833/2527059680] conn(0x7f158806a0e0 0x7f158806c5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:53:32.845 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:32.842+0000 7f15a77fe640 1 --2- 172.21.15.43:0/3565216302 >> [v2:172.21.15.203:6832/2527059680,v1:172.21.15.203:6833/2527059680] conn(0x7f158806a0e0 0x7f158806c5a0 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7f15a8152070 tx=0x7f1598008040 comp rx=0 tx=0).ready entity=mgr.6377 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-09T13:53:32.846 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:32.845+0000 7f15a4ff9640 1 -- 172.21.15.43:0/3565216302 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178066 (secure 0 0 0) 0x7f15900575a0 con 0x7f15a81e56b0 2023-08-09T13:53:33.040 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:33.038+0000 7f15aceb0640 1 -- 172.21.15.43:0/3565216302 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f15a815c1e0 con 0x7f15a81e56b0 2023-08-09T13:53:33.043 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:33.041+0000 7f15a4ff9640 1 -- 172.21.15.43:0/3565216302 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v118) v1 ==== 74+0+19647 (secure 0 0 0) 0x7f1590019090 con 0x7f15a81e56b0 2023-08-09T13:53:33.043 INFO:teuthology.orchestra.run.smithi043.stdout: 2023-08-09T13:53:33.044 INFO:teuthology.orchestra.run.smithi043.stdout:{"epoch":118,"fsid":"daac87ee-1337-4250-a955-aea4e408a39b","created":"2023-08-09T13:33:41.701198+0000","modified":"2023-08-09T13:53:28.338768+0000","last_up_change":"2023-08-09T13:33:53.783453+0000","last_in_change":"2023-08-09T13:33:45.148904+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":15,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":"rbd","create_time":"2023-08-09T13:33:54.365137+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":".mgr","create_time":"2023-08-09T13:33:54.831803+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":14,"pool_name":"cephfs_metadata","create_time":"2023-08-09T13:51:57.637653+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"115","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":2.5,"score_stable":2.5,"optimal_score":1,"raw_score_acting":2.5,"raw_score_stable":2.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":15,"pool_name":"cephfs_data","create_time":"2023-08-09T13:51:58.676117+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"116","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"cbd49ecd-e896-4fbd-864e-5c37a829b165","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":114,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6800","nonce":502667335},{"type":"v1","addr":"172.21.15.43:6801","nonce":502667335}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6802","nonce":502667335},{"type":"v1","addr":"172.21.15.43:6803","nonce":502667335}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6806","nonce":502667335},{"type":"v1","addr":"172.21.15.43:6807","nonce":502667335}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6804","nonce":502667335},{"type":"v1","addr":"172.21.15.43:6805","nonce":502667335}]},"public_addr":"172.21.15.43:6801/502667335","cluster_addr":"172.21.15.43:6803/502667335","heartbeat_back_addr":"172.21.15.43:6807/502667335","heartbeat_front_addr":"172.21.15.43:6805/502667335","state":["exists","up"]},{"osd":1,"uuid":"75c7e0fe-1ec7-4fe8-a2f2-3661a65b8500","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":114,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6808","nonce":3289134610},{"type":"v1","addr":"172.21.15.43:6809","nonce":3289134610}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6810","nonce":3289134610},{"type":"v1","addr":"172.21.15.43:6812","nonce":3289134610}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6821","nonce":3289134610},{"type":"v1","addr":"172.21.15.43:6824","nonce":3289134610}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6815","nonce":3289134610},{"type":"v1","addr":"172.21.15.43:6818","nonce":3289134610}]},"public_addr":"172.21.15.43:6809/3289134610","cluster_addr":"172.21.15.43:6812/3289134610","heartbeat_back_addr":"172.21.15.43:6824/3289134610","heartbeat_front_addr":"172.21.15.43:6818/3289134610","state":["exists","up"]},{"osd":2,"uuid":"bb5c41e9-7f5f-4125-9b01-3bb8bfc2ea83","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":114,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6811","nonce":4079709736},{"type":"v1","addr":"172.21.15.43:6814","nonce":4079709736}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6817","nonce":4079709736},{"type":"v1","addr":"172.21.15.43:6820","nonce":4079709736}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6829","nonce":4079709736},{"type":"v1","addr":"172.21.15.43:6831","nonce":4079709736}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6823","nonce":4079709736},{"type":"v1","addr":"172.21.15.43:6826","nonce":4079709736}]},"public_addr":"172.21.15.43:6814/4079709736","cluster_addr":"172.21.15.43:6820/4079709736","heartbeat_back_addr":"172.21.15.43:6831/4079709736","heartbeat_front_addr":"172.21.15.43:6826/4079709736","state":["exists","up"]},{"osd":3,"uuid":"bfe4c8b4-694a-465c-87e6-2b2aab852613","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":114,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6813","nonce":1709538759},{"type":"v1","addr":"172.21.15.43:6816","nonce":1709538759}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6819","nonce":1709538759},{"type":"v1","addr":"172.21.15.43:6822","nonce":1709538759}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6828","nonce":1709538759},{"type":"v1","addr":"172.21.15.43:6830","nonce":1709538759}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6825","nonce":1709538759},{"type":"v1","addr":"172.21.15.43:6827","nonce":1709538759}]},"public_addr":"172.21.15.43:6816/1709538759","cluster_addr":"172.21.15.43:6822/1709538759","heartbeat_back_addr":"172.21.15.43:6830/1709538759","heartbeat_front_addr":"172.21.15.43:6827/1709538759","state":["exists","up"]},{"osd":4,"uuid":"fa2e8a77-aba1-4756-853e-0d174a5bdeba","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":114,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6800","nonce":4196866481},{"type":"v1","addr":"172.21.15.203:6801","nonce":4196866481}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6802","nonce":4196866481},{"type":"v1","addr":"172.21.15.203:6803","nonce":4196866481}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6806","nonce":4196866481},{"type":"v1","addr":"172.21.15.203:6807","nonce":4196866481}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6804","nonce":4196866481},{"type":"v1","addr":"172.21.15.203:6805","nonce":4196866481}]},"public_addr":"172.21.15.203:6801/4196866481","cluster_addr":"172.21.15.203:6803/4196866481","heartbeat_back_addr":"172.21.15.203:6807/4196866481","heartbeat_front_addr":"172.21.15.203:6805/4196866481","state":["exists","up"]},{"osd":5,"uuid":"81811e4b-4d24-4f51-a82c-f1d700a2a00d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":114,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6816","nonce":3059290081},{"type":"v1","addr":"172.21.15.203:6817","nonce":3059290081}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6818","nonce":3059290081},{"type":"v1","addr":"172.21.15.203:6819","nonce":3059290081}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6822","nonce":3059290081},{"type":"v1","addr":"172.21.15.203:6823","nonce":3059290081}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6820","nonce":3059290081},{"type":"v1","addr":"172.21.15.203:6821","nonce":3059290081}]},"public_addr":"172.21.15.203:6817/3059290081","cluster_addr":"172.21.15.203:6819/3059290081","heartbeat_back_addr":"172.21.15.203:6823/3059290081","heartbeat_front_addr":"172.21.15.203:6821/3059290081","state":["exists","up"]},{"osd":6,"uuid":"307bac4d-6a5b-443d-af75-56d695d7674f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":114,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6824","nonce":512155569},{"type":"v1","addr":"172.21.15.203:6825","nonce":512155569}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6826","nonce":512155569},{"type":"v1","addr":"172.21.15.203:6827","nonce":512155569}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6830","nonce":512155569},{"type":"v1","addr":"172.21.15.203:6831","nonce":512155569}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6828","nonce":512155569},{"type":"v1","addr":"172.21.15.203:6829","nonce":512155569}]},"public_addr":"172.21.15.203:6825/512155569","cluster_addr":"172.21.15.203:6827/512155569","heartbeat_back_addr":"172.21.15.203:6831/512155569","heartbeat_front_addr":"172.21.15.203:6829/512155569","state":["exists","up"]},{"osd":7,"uuid":"a95dc7ca-9f1b-4295-af70-68a22a2d7d5e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":114,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6808","nonce":3331450438},{"type":"v1","addr":"172.21.15.203:6809","nonce":3331450438}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6810","nonce":3331450438},{"type":"v1","addr":"172.21.15.203:6811","nonce":3331450438}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6814","nonce":3331450438},{"type":"v1","addr":"172.21.15.203:6815","nonce":3331450438}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6812","nonce":3331450438},{"type":"v1","addr":"172.21.15.203:6813","nonce":3331450438}]},"public_addr":"172.21.15.203:6809/3331450438","cluster_addr":"172.21.15.203:6811/3331450438","heartbeat_back_addr":"172.21.15.203:6815/3331450438","heartbeat_front_addr":"172.21.15.203:6813/3331450438","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-09T13:33:50.547652+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-09T13:33:50.589469+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-09T13:33:50.561724+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-09T13:33:50.599650+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-09T13:33:50.411855+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-09T13:33:50.219681+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-09T13:33:50.504664+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-09T13:33:50.441248+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.203:0/3668440292":"2023-08-10T13:53:28.338643+0000","172.21.15.203:0/430012896":"2023-08-10T13:53:28.338643+0000","172.21.15.203:0/1052273067":"2023-08-10T13:53:28.338643+0000","172.21.15.203:0/2583349636":"2023-08-10T13:53:28.338643+0000","172.21.15.203:6832/2041169533":"2023-08-10T13:53:28.338643+0000","172.21.15.203:0/1488154102":"2023-08-10T13:52:20.374695+0000","172.21.15.203:0/728819840":"2023-08-10T13:53:28.338643+0000","172.21.15.203:0/3595044319":"2023-08-10T13:52:20.374695+0000","172.21.15.203:0/2916799066":"2023-08-10T13:53:28.338643+0000","172.21.15.203:0/3843164760":"2023-08-10T13:53:28.338643+0000","172.21.15.203:0/1657891023":"2023-08-10T13:52:20.374695+0000","172.21.15.203:0/329310299":"2023-08-10T13:53:28.338643+0000","172.21.15.203:0/2368610482":"2023-08-10T13:52:20.374695+0000","172.21.15.203:0/437192587":"2023-08-10T13:52:20.374695+0000","172.21.15.203:6833/1317108840":"2023-08-10T13:52:20.374695+0000","172.21.15.203:6832/1317108840":"2023-08-10T13:52:20.374695+0000","172.21.15.203:6833/2041169533":"2023-08-10T13:53:28.338643+0000","172.21.15.203:0/1459918882":"2023-08-10T13:52:20.374695+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":7,"snaps":[{"begin":104,"length":1},{"begin":221,"length":1}]}],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-08-09T13:53:33.046 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:33.044+0000 7f15aceb0640 1 -- 172.21.15.43:0/3565216302 >> [v2:172.21.15.203:6832/2527059680,v1:172.21.15.203:6833/2527059680] conn(0x7f158806a0e0 msgr2=0x7f158806c5a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:53:33.046 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:33.044+0000 7f15aceb0640 1 --2- 172.21.15.43:0/3565216302 >> [v2:172.21.15.203:6832/2527059680,v1:172.21.15.203:6833/2527059680] conn(0x7f158806a0e0 0x7f158806c5a0 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7f15a8152070 tx=0x7f1598008040 comp rx=0 tx=0).stop 2023-08-09T13:53:33.046 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:33.044+0000 7f15aceb0640 1 -- 172.21.15.43:0/3565216302 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f15a81e56b0 msgr2=0x7f15a81e5b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:53:33.047 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:33.044+0000 7f15aceb0640 1 --2- 172.21.15.43:0/3565216302 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f15a81e56b0 0x7f15a81e5b30 secure :-1 s=READY pgs=1445 cs=0 l=1 rev1=1 crypto rx=0x7f159000d8b0 tx=0x7f159000dd70 comp rx=0 tx=0).stop 2023-08-09T13:53:33.047 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:33.045+0000 7f15aceb0640 1 -- 172.21.15.43:0/3565216302 shutdown_connections 2023-08-09T13:53:33.047 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:33.045+0000 7f15aceb0640 1 --2- 172.21.15.43:0/3565216302 >> [v2:172.21.15.203:6832/2527059680,v1:172.21.15.203:6833/2527059680] conn(0x7f158806a0e0 0x7f158806c5a0 unknown :-1 s=CLOSED pgs=29 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:53:33.047 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:33.045+0000 7f15aceb0640 1 --2- 172.21.15.43:0/3565216302 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f15a80d0f80 0x7f15a80d1400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:53:33.047 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:33.045+0000 7f15aceb0640 1 --2- 172.21.15.43:0/3565216302 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f15a81e56b0 0x7f15a81e5b30 unknown :-1 s=CLOSED pgs=1445 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:53:33.047 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:33.045+0000 7f15aceb0640 1 --2- 172.21.15.43:0/3565216302 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f1594004d00 0x7f15a814f740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:53:33.047 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:33.045+0000 7f15aceb0640 1 -- 172.21.15.43:0/3565216302 >> 172.21.15.43:0/3565216302 conn(0x7f15a806a2b0 msgr2=0x7f15a815cb50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-09T13:53:33.048 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:33.045+0000 7f15aceb0640 1 -- 172.21.15.43:0/3565216302 shutdown_connections 2023-08-09T13:53:33.048 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:33.045+0000 7f15aceb0640 1 -- 172.21.15.43:0/3565216302 wait complete. 2023-08-09T13:53:33.066 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2023-08-09T13:53:33.295 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:33.294+0000 7f1ac6037640 1 Processor -- start 2023-08-09T13:53:33.296 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:33.294+0000 7f1ac6037640 1 -- start start 2023-08-09T13:53:33.297 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:33.296+0000 7f1ac6037640 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7f1ac015c500 0x7f1ac00d0240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:53:33.297 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:33.296+0000 7f1ac6037640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f1ac00b7f90 0x7f1ac00b5310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:53:33.298 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:33.296+0000 7f1ac6037640 1 -- --> v1:172.21.15.43:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1ac015c8e0 con 0x7f1ac00b5850 2023-08-09T13:53:33.298 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:33.296+0000 7f1ac6037640 1 -- --> v2:172.21.15.203:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1ac00d2bd0 con 0x7f1ac015c500 2023-08-09T13:53:33.298 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:33.296+0000 7f1ac6037640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1ac00d0b20 con 0x7f1ac00b7f90 2023-08-09T13:53:33.299 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:33.296+0000 7f1ac5836640 1 --1- >> v1:172.21.15.43:6789/0 conn(0x7f1ac00b5850 0x7f1ac00b3d00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.43:6789/0 says I am v1:172.21.15.43:41044/0 (socket says 172.21.15.43:41044) 2023-08-09T13:53:33.299 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:33.296+0000 7f1ac5836640 1 -- 172.21.15.43:0/1286735657 learned_addr learned my addr 172.21.15.43:0/1286735657 (peer_addr_for_me v1:172.21.15.43:0/0) 2023-08-09T13:53:33.300 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:33.297+0000 7f1ac4834640 1 --2- 172.21.15.43:0/1286735657 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f1ac00b7f90 0x7f1ac00b5310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:53:33.300 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:33.297+0000 7f1ac5035640 1 --2- 172.21.15.43:0/1286735657 >> v2:172.21.15.203:3300/0 conn(0x7f1ac015c500 0x7f1ac00d0240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:53:33.300 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:33.297+0000 7f1ab7fff640 1 -- 172.21.15.43:0/1286735657 <== mon.0 v1:172.21.15.43:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 906088935 0 0) 0x7f1ac015c8e0 con 0x7f1ac00b5850 2023-08-09T13:53:33.300 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:33.297+0000 7f1ab7fff640 1 -- 172.21.15.43:0/1286735657 --> v1:172.21.15.43:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f1aac003520 con 0x7f1ac00b5850 2023-08-09T13:53:33.300 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:33.297+0000 7f1ab7fff640 1 -- 172.21.15.43:0/1286735657 <== mon.0 v1:172.21.15.43:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2336052482 0 0) 0x7f1aac003520 con 0x7f1ac00b5850 2023-08-09T13:53:33.300 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:33.297+0000 7f1ab7fff640 1 -- 172.21.15.43:0/1286735657 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f1ac00b7f90 msgr2=0x7f1ac00b5310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-09T13:53:33.300 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:33.297+0000 7f1ab7fff640 1 --2- 172.21.15.43:0/1286735657 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f1ac00b7f90 0x7f1ac00b5310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:53:33.301 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:33.297+0000 7f1ab7fff640 1 -- 172.21.15.43:0/1286735657 >> v2:172.21.15.203:3300/0 conn(0x7f1ac015c500 msgr2=0x7f1ac00d0240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-09T13:53:33.301 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:33.297+0000 7f1ab7fff640 1 --2- 172.21.15.43:0/1286735657 >> v2:172.21.15.203:3300/0 conn(0x7f1ac015c500 0x7f1ac00d0240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:53:33.301 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:33.297+0000 7f1ab7fff640 1 -- 172.21.15.43:0/1286735657 --> v1:172.21.15.43:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1ac00d0e00 con 0x7f1ac00b5850 2023-08-09T13:53:33.301 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:33.297+0000 7f1ab7fff640 1 -- 172.21.15.43:0/1286735657 <== mon.0 v1:172.21.15.43:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1285150740 0 0) 0x7f1ab80019f0 con 0x7f1ac00b5850 2023-08-09T13:53:33.301 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:33.297+0000 7f1ab7fff640 1 -- 172.21.15.43:0/1286735657 >> v1:172.21.15.43:6789/0 conn(0x7f1ac00b5850 legacy=0x7f1ac00b3d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:53:33.301 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:33.297+0000 7f1ab7fff640 1 --2- 172.21.15.43:0/1286735657 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f1ac00b7f90 0x7f1aac003970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:53:33.301 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:33.297+0000 7f1ab7fff640 1 --2- 172.21.15.43:0/1286735657 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f1aac003ef0 0x7f1aac0047c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:53:33.301 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:33.297+0000 7f1ab7fff640 1 --2- 172.21.15.43:0/1286735657 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f1aac004d00 0x7f1aac005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:53:33.302 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:33.298+0000 7f1ab7fff640 1 -- 172.21.15.43:0/1286735657 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1ac00d2bd0 con 0x7f1aac003ef0 2023-08-09T13:53:33.302 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:33.298+0000 7f1ab7fff640 1 -- 172.21.15.43:0/1286735657 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1ac00d0b20 con 0x7f1aac004d00 2023-08-09T13:53:33.302 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:33.298+0000 7f1ab7fff640 1 -- 172.21.15.43:0/1286735657 --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1aac00b9b0 con 0x7f1ac00b7f90 2023-08-09T13:53:33.302 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:33.298+0000 7f1ac4834640 1 --2- 172.21.15.43:0/1286735657 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f1aac003ef0 0x7f1aac0047c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:53:33.302 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:33.298+0000 7f1ac5035640 1 --2- 172.21.15.43:0/1286735657 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f1ac00b7f90 0x7f1aac003970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:53:33.303 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:33.298+0000 7f1ac4834640 1 -- 172.21.15.43:0/1286735657 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f1ac00b7f90 msgr2=0x7f1aac003970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-09T13:53:33.304 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:33.298+0000 7f1ac4834640 1 --2- 172.21.15.43:0/1286735657 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f1ac00b7f90 0x7f1aac003970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:53:33.304 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:33.298+0000 7f1ac4834640 1 -- 172.21.15.43:0/1286735657 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f1aac004d00 msgr2=0x7f1aac005180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-09T13:53:33.304 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:33.298+0000 7f1ac5836640 1 --2- 172.21.15.43:0/1286735657 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f1aac004d00 0x7f1aac005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:53:33.304 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:33.298+0000 7f1ac4834640 1 --2- 172.21.15.43:0/1286735657 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f1aac004d00 0x7f1aac005180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:53:33.304 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:33.298+0000 7f1ac4834640 1 -- 172.21.15.43:0/1286735657 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1aac003520 con 0x7f1aac003ef0 2023-08-09T13:53:33.304 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:33.299+0000 7f1ac5836640 1 --2- 172.21.15.43:0/1286735657 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f1aac004d00 0x7f1aac005180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-08-09T13:53:33.304 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:33.300+0000 7f1ac4834640 1 --2- 172.21.15.43:0/1286735657 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f1aac003ef0 0x7f1aac0047c0 secure :-1 s=READY pgs=1447 cs=0 l=1 rev1=1 crypto rx=0x7f1ab0009ac0 tx=0x7f1aac0057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=776811ee4543fd2b server_cookie=0 in_seq=0 out_seq=0 2023-08-09T13:53:33.305 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:33.300+0000 7f1ab7fff640 1 -- 172.21.15.43:0/1286735657 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1ab002c6b0 con 0x7f1aac003ef0 2023-08-09T13:53:33.305 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:33.300+0000 7f1ab7fff640 1 -- 172.21.15.43:0/1286735657 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1ab002c810 con 0x7f1aac003ef0 2023-08-09T13:53:33.305 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:33.300+0000 7f1ab7fff640 1 -- 172.21.15.43:0/1286735657 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1ab002cba0 con 0x7f1aac003ef0 2023-08-09T13:53:33.305 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:33.300+0000 7f1ac6037640 1 -- 172.21.15.43:0/1286735657 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f1aac003ef0 msgr2=0x7f1aac0047c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:53:33.305 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:33.300+0000 7f1ac6037640 1 --2- 172.21.15.43:0/1286735657 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f1aac003ef0 0x7f1aac0047c0 secure :-1 s=READY pgs=1447 cs=0 l=1 rev1=1 crypto rx=0x7f1ab0009ac0 tx=0x7f1aac0057f0 comp rx=0 tx=0).stop 2023-08-09T13:53:33.305 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:33.300+0000 7f1ac5836640 1 -- 172.21.15.43:0/1286735657 reap_dead start 2023-08-09T13:53:33.305 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:33.301+0000 7f1ac6037640 1 -- 172.21.15.43:0/1286735657 shutdown_connections 2023-08-09T13:53:33.305 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:33.301+0000 7f1ac6037640 1 -- 172.21.15.43:0/1286735657 >> 172.21.15.43:0/1286735657 conn(0x7f1ac00712e0 msgr2=0x7f1ac00716f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-09T13:53:33.306 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:33.301+0000 7f1ac6037640 1 -- 172.21.15.43:0/1286735657 shutdown_connections 2023-08-09T13:53:33.306 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:33.301+0000 7f1ac6037640 1 -- 172.21.15.43:0/1286735657 wait complete. 2023-08-09T13:53:33.307 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:33.303+0000 7f1ac6037640 1 Processor -- start 2023-08-09T13:53:33.307 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:33.303+0000 7f1ac6037640 1 -- start start 2023-08-09T13:53:33.307 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:33.304+0000 7f1ac6037640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f1aac003ef0 0x7f1ac00c9350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:53:33.308 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:33.304+0000 7f1ac6037640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f1ac00caa40 0x7f1ac01595d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:53:33.308 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:33.304+0000 7f1ac6037640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f1ac00cbcd0 0x7f1ac00aa5c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:53:33.308 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:33.304+0000 7f1ac6037640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1ac0159b10 con 0x7f1aac003ef0 2023-08-09T13:53:33.308 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:33.304+0000 7f1ac6037640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1ac00c99a0 con 0x7f1ac00caa40 2023-08-09T13:53:33.309 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:33.304+0000 7f1ac6037640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1ac00c9c80 con 0x7f1ac00cbcd0 2023-08-09T13:53:33.309 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:33.304+0000 7f1ac5035640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f1aac003ef0 0x7f1ac00c9350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:53:33.309 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:33.304+0000 7f1ac5035640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f1aac003ef0 0x7f1ac00c9350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:58370/0 (socket says 172.21.15.43:58370) 2023-08-09T13:53:33.309 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:33.304+0000 7f1ac5035640 1 -- 172.21.15.43:0/3050515115 learned_addr learned my addr 172.21.15.43:0/3050515115 (peer_addr_for_me v2:172.21.15.43:0/0) 2023-08-09T13:53:33.310 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:33.304+0000 7f1ac5035640 1 -- 172.21.15.43:0/3050515115 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f1ac00cbcd0 msgr2=0x7f1ac00aa5c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:53:33.310 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:33.304+0000 7f1ac5035640 1 --2- 172.21.15.43:0/3050515115 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f1ac00cbcd0 0x7f1ac00aa5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:53:33.310 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:33.304+0000 7f1ac5035640 1 -- 172.21.15.43:0/3050515115 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f1ac00caa40 msgr2=0x7f1ac01595d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:53:33.310 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:33.304+0000 7f1ac5035640 1 --2- 172.21.15.43:0/3050515115 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f1ac00caa40 0x7f1ac01595d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:53:33.311 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:33.304+0000 7f1ac5035640 1 -- 172.21.15.43:0/3050515115 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1ac00c9f60 con 0x7f1aac003ef0 2023-08-09T13:53:33.311 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:33.305+0000 7f1ac5035640 1 --2- 172.21.15.43:0/3050515115 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f1aac003ef0 0x7f1ac00c9350 secure :-1 s=READY pgs=1448 cs=0 l=1 rev1=1 crypto rx=0x7f1abc0116d0 tx=0x7f1abc011b90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-09T13:53:33.311 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:33.305+0000 7f1ab5ffb640 1 -- 172.21.15.43:0/3050515115 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1abc00f9c0 con 0x7f1aac003ef0 2023-08-09T13:53:33.312 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:33.305+0000 7f1ab5ffb640 1 -- 172.21.15.43:0/3050515115 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1abc00fb20 con 0x7f1aac003ef0 2023-08-09T13:53:33.312 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:33.305+0000 7f1ab5ffb640 1 -- 172.21.15.43:0/3050515115 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1abc00fe90 con 0x7f1aac003ef0 2023-08-09T13:53:33.312 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:33.305+0000 7f1ac6037640 1 -- 172.21.15.43:0/3050515115 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1ac00ca240 con 0x7f1aac003ef0 2023-08-09T13:53:33.313 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:33.305+0000 7f1ac6037640 1 -- 172.21.15.43:0/3050515115 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1ac00cc180 con 0x7f1aac003ef0 2023-08-09T13:53:33.314 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:33.306+0000 7f1ac6037640 1 -- 172.21.15.43:0/3050515115 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1ac00739f0 con 0x7f1aac003ef0 2023-08-09T13:53:33.314 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:33.307+0000 7f1ab5ffb640 1 -- 172.21.15.43:0/3050515115 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 56) v1 ==== 88443+0+0 (secure 0 0 0) 0x7f1abc015460 con 0x7f1aac003ef0 2023-08-09T13:53:33.314 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:33.308+0000 7f1ab5ffb640 1 --2- 172.21.15.43:0/3050515115 >> [v2:172.21.15.203:6832/2527059680,v1:172.21.15.203:6833/2527059680] conn(0x7f1aa406a1a0 0x7f1aa406c660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:53:33.314 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:33.308+0000 7f1ab5ffb640 1 -- 172.21.15.43:0/3050515115 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(118..118 src has 1..118) v4 ==== 7200+0+0 (secure 0 0 0) 0x7f1abc089170 con 0x7f1aac003ef0 2023-08-09T13:53:33.314 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:33.309+0000 7f1ac4834640 1 --2- 172.21.15.43:0/3050515115 >> [v2:172.21.15.203:6832/2527059680,v1:172.21.15.203:6833/2527059680] conn(0x7f1aa406a1a0 0x7f1aa406c660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:53:33.315 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:33.310+0000 7f1ac4834640 1 --2- 172.21.15.43:0/3050515115 >> [v2:172.21.15.203:6832/2527059680,v1:172.21.15.203:6833/2527059680] conn(0x7f1aa406a1a0 0x7f1aa406c660 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0x7f1ab0036000 tx=0x7f1ab002e000 comp rx=0 tx=0).ready entity=mgr.6377 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-09T13:53:33.315 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:33.312+0000 7f1ab5ffb640 1 -- 172.21.15.43:0/3050515115 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178066 (secure 0 0 0) 0x7f1abc057da0 con 0x7f1aac003ef0 2023-08-09T13:53:33.549 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:33.547+0000 7f1ac6037640 1 -- 172.21.15.43:0/3050515115 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) v1 -- 0x7f1ac015f3e0 con 0x7f1aac003ef0 2023-08-09T13:53:33.550 INFO:tasks.ceph.mon.a.smithi043.stderr:2023-08-09T13:53:33.549+0000 7f32867f3640 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2023-08-09T13:53:33.557 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:33.556+0000 7f1ab5ffb640 1 -- 172.21.15.43:0/3050515115 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v53) v1 ==== 161+0+0 (secure 0 0 0) 0x7f1abc057930 con 0x7f1aac003ef0 2023-08-09T13:53:33.557 INFO:teuthology.orchestra.run.smithi043.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2023-08-09T13:53:33.560 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:33.559+0000 7f1ac6037640 1 -- 172.21.15.43:0/3050515115 >> [v2:172.21.15.203:6832/2527059680,v1:172.21.15.203:6833/2527059680] conn(0x7f1aa406a1a0 msgr2=0x7f1aa406c660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:53:33.560 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:33.559+0000 7f1ac6037640 1 --2- 172.21.15.43:0/3050515115 >> [v2:172.21.15.203:6832/2527059680,v1:172.21.15.203:6833/2527059680] conn(0x7f1aa406a1a0 0x7f1aa406c660 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0x7f1ab0036000 tx=0x7f1ab002e000 comp rx=0 tx=0).stop 2023-08-09T13:53:33.561 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:33.559+0000 7f1ac6037640 1 -- 172.21.15.43:0/3050515115 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f1aac003ef0 msgr2=0x7f1ac00c9350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:53:33.561 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:33.559+0000 7f1ac6037640 1 --2- 172.21.15.43:0/3050515115 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f1aac003ef0 0x7f1ac00c9350 secure :-1 s=READY pgs=1448 cs=0 l=1 rev1=1 crypto rx=0x7f1abc0116d0 tx=0x7f1abc011b90 comp rx=0 tx=0).stop 2023-08-09T13:53:33.561 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:33.559+0000 7f1ac6037640 1 -- 172.21.15.43:0/3050515115 shutdown_connections 2023-08-09T13:53:33.561 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:33.559+0000 7f1ac6037640 1 --2- 172.21.15.43:0/3050515115 >> [v2:172.21.15.203:6832/2527059680,v1:172.21.15.203:6833/2527059680] conn(0x7f1aa406a1a0 0x7f1aa406c660 unknown :-1 s=CLOSED pgs=30 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:53:33.561 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:33.559+0000 7f1ac6037640 1 --2- 172.21.15.43:0/3050515115 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f1ac00cbcd0 0x7f1ac00aa5c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:53:33.568 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:33.559+0000 7f1ac6037640 1 --2- 172.21.15.43:0/3050515115 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f1ac00caa40 0x7f1ac01595d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:53:33.569 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:33.559+0000 7f1ac6037640 1 --2- 172.21.15.43:0/3050515115 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f1aac003ef0 0x7f1ac00c9350 unknown :-1 s=CLOSED pgs=1448 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:53:33.569 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:33.559+0000 7f1ac6037640 1 -- 172.21.15.43:0/3050515115 >> 172.21.15.43:0/3050515115 conn(0x7f1ac00712e0 msgr2=0x7f1ac015e270 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-09T13:53:33.569 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:33.559+0000 7f1ac6037640 1 -- 172.21.15.43:0/3050515115 shutdown_connections 2023-08-09T13:53:33.569 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:33.559+0000 7f1ac6037640 1 -- 172.21.15.43:0/3050515115 wait complete. 2023-08-09T13:53:33.581 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs rm cephfs --yes-i-really-mean-it 2023-08-09T13:53:33.590 INFO:tasks.ceph.mds.c.smithi043.stderr: -6296> 2023-08-09T13:52:05.668+0000 7f0151030640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2023-08-09T13:53:33.673 INFO:tasks.ceph.mds.c.smithi043.stdout:starting mds.c at 2023-08-09T13:53:33.795 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:33.793+0000 7f6542a77640 1 Processor -- start 2023-08-09T13:53:33.795 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:33.794+0000 7f6542a77640 1 -- start start 2023-08-09T13:53:33.797 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:33.795+0000 7f6542a77640 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7f653c15c550 0x7f653c0cd6d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:53:33.797 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:33.795+0000 7f6542a77640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f653c0d04b0 0x7f653c0b3ee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:53:33.797 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:33.795+0000 7f6542a77640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f653c0b4420 0x7f653c06a760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:53:33.797 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:33.795+0000 7f6542a77640 1 -- --> v2:172.21.15.43:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f653c0b4870 con 0x7f653c0d04b0 2023-08-09T13:53:33.797 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:33.795+0000 7f6542a77640 1 -- --> v2:172.21.15.203:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f653c0b7bd0 con 0x7f653c15c550 2023-08-09T13:53:33.797 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:33.795+0000 7f6542a77640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f653c0d1c00 con 0x7f653c0b4420 2023-08-09T13:53:33.798 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:33.795+0000 7f653bfff640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f653c0d04b0 0x7f653c0b3ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:53:33.798 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:33.795+0000 7f653bfff640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f653c0d04b0 0x7f653c0b3ee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:58390/0 (socket says 172.21.15.43:58390) 2023-08-09T13:53:33.798 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:33.795+0000 7f653bfff640 1 -- 172.21.15.43:0/3657839455 learned_addr learned my addr 172.21.15.43:0/3657839455 (peer_addr_for_me v2:172.21.15.43:0/0) 2023-08-09T13:53:33.799 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:33.796+0000 7f6540813640 1 --2- 172.21.15.43:0/3657839455 >> v2:172.21.15.203:3300/0 conn(0x7f653c15c550 0x7f653c0cd6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:53:33.799 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:33.796+0000 7f653bfff640 1 -- 172.21.15.43:0/3657839455 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f653c0b4420 msgr2=0x7f653c06a760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-09T13:53:33.799 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:33.796+0000 7f6541014640 1 --2- 172.21.15.43:0/3657839455 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f653c0b4420 0x7f653c06a760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:53:33.799 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:33.796+0000 7f653bfff640 1 --2- 172.21.15.43:0/3657839455 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f653c0b4420 0x7f653c06a760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:53:33.799 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:33.796+0000 7f653bfff640 1 -- 172.21.15.43:0/3657839455 >> v2:172.21.15.203:3300/0 conn(0x7f653c15c550 msgr2=0x7f653c0cd6d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-09T13:53:33.800 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:33.796+0000 7f653bfff640 1 --2- 172.21.15.43:0/3657839455 >> v2:172.21.15.203:3300/0 conn(0x7f653c15c550 0x7f653c0cd6d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:53:33.800 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:33.796+0000 7f653bfff640 1 -- 172.21.15.43:0/3657839455 --> v2:172.21.15.43:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f653c15c930 con 0x7f653c0d04b0 2023-08-09T13:53:33.800 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:33.796+0000 7f6540813640 1 --2- 172.21.15.43:0/3657839455 >> v2:172.21.15.203:3300/0 conn(0x7f653c15c550 0x7f653c0cd6d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-08-09T13:53:33.800 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:33.797+0000 7f653bfff640 1 --2- 172.21.15.43:0/3657839455 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f653c0d04b0 0x7f653c0b3ee0 secure :-1 s=READY pgs=1452 cs=0 l=1 rev1=1 crypto rx=0x7f653c0d01c0 tx=0x7f653002b870 comp rx=0 tx=0).ready entity=mon.0 client_cookie=be4c6a419c794d78 server_cookie=0 in_seq=0 out_seq=0 2023-08-09T13:53:33.800 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:33.797+0000 7f653b7fe640 1 -- 172.21.15.43:0/3657839455 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f65300317d0 con 0x7f653c0d04b0 2023-08-09T13:53:33.800 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:33.797+0000 7f653b7fe640 1 -- 172.21.15.43:0/3657839455 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6530031930 con 0x7f653c0d04b0 2023-08-09T13:53:33.800 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:33.797+0000 7f653b7fe640 1 -- 172.21.15.43:0/3657839455 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6530031cc0 con 0x7f653c0d04b0 2023-08-09T13:53:33.801 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:33.798+0000 7f6542a77640 1 -- 172.21.15.43:0/3657839455 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f653c0d04b0 msgr2=0x7f653c0b3ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:53:33.801 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:33.798+0000 7f6542a77640 1 --2- 172.21.15.43:0/3657839455 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f653c0d04b0 0x7f653c0b3ee0 secure :-1 s=READY pgs=1452 cs=0 l=1 rev1=1 crypto rx=0x7f653c0d01c0 tx=0x7f653002b870 comp rx=0 tx=0).stop 2023-08-09T13:53:33.802 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:33.798+0000 7f6542a77640 1 -- 172.21.15.43:0/3657839455 shutdown_connections 2023-08-09T13:53:33.802 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:33.798+0000 7f6542a77640 1 --2- 172.21.15.43:0/3657839455 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f653c0d04b0 0x7f653c0b3ee0 unknown :-1 s=CLOSED pgs=1452 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:53:33.802 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:33.798+0000 7f6542a77640 1 --2- 172.21.15.43:0/3657839455 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f653c0b4420 0x7f653c06a760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:53:33.802 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:33.798+0000 7f6542a77640 1 --2- 172.21.15.43:0/3657839455 >> v2:172.21.15.203:3300/0 conn(0x7f653c15c550 0x7f653c0cd6d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:53:33.802 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:33.798+0000 7f6542a77640 1 -- 172.21.15.43:0/3657839455 >> 172.21.15.43:0/3657839455 conn(0x7f653c071520 msgr2=0x7f653c0ceee0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-09T13:53:33.802 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:33.798+0000 7f6542a77640 1 -- 172.21.15.43:0/3657839455 shutdown_connections 2023-08-09T13:53:33.802 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:33.798+0000 7f6542a77640 1 -- 172.21.15.43:0/3657839455 wait complete. 2023-08-09T13:53:33.803 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:33.800+0000 7f6542a77640 1 Processor -- start 2023-08-09T13:53:33.803 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:33.800+0000 7f6542a77640 1 -- start start 2023-08-09T13:53:33.804 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:33.801+0000 7f6542a77640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f653c0b4420 0x7f653c0ab500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:53:33.804 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:33.801+0000 7f6542a77640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f653c0d04b0 0x7f653c0cbba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:53:33.805 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:33.801+0000 7f6542a77640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f653c15c550 0x7f653c15ee20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:53:33.805 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:33.801+0000 7f6542a77640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f653c1f1980 con 0x7f653c0b4420 2023-08-09T13:53:33.805 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:33.801+0000 7f6542a77640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f653c1f1c60 con 0x7f653c0d04b0 2023-08-09T13:53:33.805 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:33.801+0000 7f6542a77640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f653c1f1f40 con 0x7f653c15c550 2023-08-09T13:53:33.805 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:33.801+0000 7f6540813640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f653c0b4420 0x7f653c0ab500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:53:33.805 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:33.801+0000 7f6540813640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f653c0b4420 0x7f653c0ab500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:58396/0 (socket says 172.21.15.43:58396) 2023-08-09T13:53:33.805 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:33.801+0000 7f6540813640 1 -- 172.21.15.43:0/3433611414 learned_addr learned my addr 172.21.15.43:0/3433611414 (peer_addr_for_me v2:172.21.15.43:0/0) 2023-08-09T13:53:33.805 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:33.802+0000 7f653bfff640 1 --2- 172.21.15.43:0/3433611414 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f653c0d04b0 0x7f653c0cbba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:53:33.806 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:33.802+0000 7f6540813640 1 -- 172.21.15.43:0/3433611414 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f653c15c550 msgr2=0x7f653c15ee20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:53:33.806 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:33.802+0000 7f6541014640 1 --2- 172.21.15.43:0/3433611414 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f653c15c550 0x7f653c15ee20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:53:33.806 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:33.802+0000 7f6540813640 1 --2- 172.21.15.43:0/3433611414 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f653c15c550 0x7f653c15ee20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:53:33.806 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:33.802+0000 7f6540813640 1 -- 172.21.15.43:0/3433611414 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f653c0d04b0 msgr2=0x7f653c0cbba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:53:33.806 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:33.802+0000 7f6540813640 1 --2- 172.21.15.43:0/3433611414 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f653c0d04b0 0x7f653c0cbba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:53:33.806 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:33.802+0000 7f6540813640 1 -- 172.21.15.43:0/3433611414 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f65300094d0 con 0x7f653c0b4420 2023-08-09T13:53:33.806 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:33.802+0000 7f6540813640 1 --2- 172.21.15.43:0/3433611414 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f653c0b4420 0x7f653c0ab500 secure :-1 s=READY pgs=1453 cs=0 l=1 rev1=1 crypto rx=0x7f652c00ce00 tx=0x7f652c00a650 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-09T13:53:33.807 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:33.803+0000 7f65397fa640 1 -- 172.21.15.43:0/3433611414 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f652c00f890 con 0x7f653c0b4420 2023-08-09T13:53:33.807 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:33.803+0000 7f65397fa640 1 -- 172.21.15.43:0/3433611414 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f652c00f9f0 con 0x7f653c0b4420 2023-08-09T13:53:33.807 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:33.803+0000 7f65397fa640 1 -- 172.21.15.43:0/3433611414 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f652c00fdc0 con 0x7f653c0b4420 2023-08-09T13:53:33.807 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:33.803+0000 7f6542a77640 1 -- 172.21.15.43:0/3433611414 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f653c1f2220 con 0x7f653c0b4420 2023-08-09T13:53:33.807 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:33.803+0000 7f6542a77640 1 -- 172.21.15.43:0/3433611414 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f653c0aba40 con 0x7f653c0b4420 2023-08-09T13:53:33.808 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:33.804+0000 7f6542a77640 1 -- 172.21.15.43:0/3433611414 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f653c0739f0 con 0x7f653c0b4420 2023-08-09T13:53:33.808 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:33.805+0000 7f65397fa640 1 -- 172.21.15.43:0/3433611414 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 56) v1 ==== 88443+0+0 (secure 0 0 0) 0x7f652c00b040 con 0x7f653c0b4420 2023-08-09T13:53:33.808 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:33.806+0000 7f65397fa640 1 --2- 172.21.15.43:0/3433611414 >> [v2:172.21.15.203:6832/2527059680,v1:172.21.15.203:6833/2527059680] conn(0x7f652806a170 0x7f652806c630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:53:33.808 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:33.806+0000 7f65397fa640 1 -- 172.21.15.43:0/3433611414 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(119..119 src has 1..119) v4 ==== 7286+0+0 (secure 0 0 0) 0x7f652c0896d0 con 0x7f653c0b4420 2023-08-09T13:53:33.809 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:33.807+0000 7f653bfff640 1 --2- 172.21.15.43:0/3433611414 >> [v2:172.21.15.203:6832/2527059680,v1:172.21.15.203:6833/2527059680] conn(0x7f652806a170 0x7f652806c630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:53:33.809 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:33.808+0000 7f653bfff640 1 --2- 172.21.15.43:0/3433611414 >> [v2:172.21.15.203:6832/2527059680,v1:172.21.15.203:6833/2527059680] conn(0x7f652806a170 0x7f652806c630 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0x7f6530036000 tx=0x7f653003d040 comp rx=0 tx=0).ready entity=mgr.6377 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-09T13:53:33.812 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:33.810+0000 7f65397fa640 1 -- 172.21.15.43:0/3433611414 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178066 (secure 0 0 0) 0x7f652c054630 con 0x7f653c0b4420 2023-08-09T13:53:34.038 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:34.037+0000 7f6542a77640 1 -- 172.21.15.43:0/3433611414 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} v 0) v1 -- 0x7f653c0c8d90 con 0x7f653c0b4420 2023-08-09T13:53:34.562 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:34.560+0000 7f65397fa640 1 -- 172.21.15.43:0/3433611414 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]=0 v54) v1 ==== 104+0+0 (secure 0 0 0) 0x7f652c057c50 con 0x7f653c0b4420 2023-08-09T13:53:34.566 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:34.564+0000 7f6542a77640 1 -- 172.21.15.43:0/3433611414 >> [v2:172.21.15.203:6832/2527059680,v1:172.21.15.203:6833/2527059680] conn(0x7f652806a170 msgr2=0x7f652806c630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:53:34.566 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:34.564+0000 7f6542a77640 1 --2- 172.21.15.43:0/3433611414 >> [v2:172.21.15.203:6832/2527059680,v1:172.21.15.203:6833/2527059680] conn(0x7f652806a170 0x7f652806c630 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0x7f6530036000 tx=0x7f653003d040 comp rx=0 tx=0).stop 2023-08-09T13:53:34.566 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:34.564+0000 7f6542a77640 1 -- 172.21.15.43:0/3433611414 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f653c0b4420 msgr2=0x7f653c0ab500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:53:34.566 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:34.564+0000 7f6542a77640 1 --2- 172.21.15.43:0/3433611414 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f653c0b4420 0x7f653c0ab500 secure :-1 s=READY pgs=1453 cs=0 l=1 rev1=1 crypto rx=0x7f652c00ce00 tx=0x7f652c00a650 comp rx=0 tx=0).stop 2023-08-09T13:53:34.566 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:34.564+0000 7f6542a77640 1 -- 172.21.15.43:0/3433611414 shutdown_connections 2023-08-09T13:53:34.566 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:34.564+0000 7f6542a77640 1 --2- 172.21.15.43:0/3433611414 >> [v2:172.21.15.203:6832/2527059680,v1:172.21.15.203:6833/2527059680] conn(0x7f652806a170 0x7f652806c630 unknown :-1 s=CLOSED pgs=31 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:53:34.566 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:34.564+0000 7f6542a77640 1 --2- 172.21.15.43:0/3433611414 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f653c15c550 0x7f653c15ee20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:53:34.567 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:34.564+0000 7f6542a77640 1 --2- 172.21.15.43:0/3433611414 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f653c0d04b0 0x7f653c0cbba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:53:34.567 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:34.564+0000 7f6542a77640 1 --2- 172.21.15.43:0/3433611414 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f653c0b4420 0x7f653c0ab500 unknown :-1 s=CLOSED pgs=1453 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:53:34.567 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:34.564+0000 7f6542a77640 1 -- 172.21.15.43:0/3433611414 >> 172.21.15.43:0/3433611414 conn(0x7f653c071520 msgr2=0x7f653c15e3d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-09T13:53:34.567 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:34.565+0000 7f6542a77640 1 -- 172.21.15.43:0/3433611414 shutdown_connections 2023-08-09T13:53:34.568 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:34.565+0000 7f6542a77640 1 -- 172.21.15.43:0/3433611414 wait complete. 2023-08-09T13:53:34.586 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_metadata cephfs_metadata --yes-i-really-really-mean-it 2023-08-09T13:53:34.703 DEBUG:teuthology.orchestra.run.smithi043:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-08-09T13:53:34.709 DEBUG:teuthology.orchestra.run.smithi203:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-08-09T13:53:34.862 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:34.861+0000 7fc8107ab640 1 Processor -- start 2023-08-09T13:53:34.863 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:34.861+0000 7fc8107ab640 1 -- start start 2023-08-09T13:53:34.864 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:34.863+0000 7fc8107ab640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7fc80815da40 0x7fc8080ce4c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:53:34.865 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:34.863+0000 7fc8107ab640 1 -- --> v1:172.21.15.43:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc8080ca410 con 0x7fc808069a30 2023-08-09T13:53:34.865 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:34.863+0000 7fc8107ab640 1 -- --> v1:172.21.15.203:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc80815a050 con 0x7fc80815d5b0 2023-08-09T13:53:34.865 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:34.863+0000 7fc8107ab640 1 -- --> v2:172.21.15.43:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc808159e70 con 0x7fc80815da40 2023-08-09T13:53:34.865 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:34.863+0000 7fc80e547640 1 --1- >> v1:172.21.15.43:6789/0 conn(0x7fc808069a30 0x7fc808159760 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.43:6789/0 says I am v1:172.21.15.43:41062/0 (socket says 172.21.15.43:41062) 2023-08-09T13:53:34.865 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:34.863+0000 7fc80e547640 1 -- 172.21.15.43:0/1832654606 learned_addr learned my addr 172.21.15.43:0/1832654606 (peer_addr_for_me v1:172.21.15.43:0/0) 2023-08-09T13:53:34.865 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:34.863+0000 7fc80ed48640 1 --2- 172.21.15.43:0/1832654606 >> v2:172.21.15.43:3300/0 conn(0x7fc80815da40 0x7fc8080ce4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:53:34.866 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:34.864+0000 7fc80d545640 1 -- 172.21.15.43:0/1832654606 <== mon.0 v1:172.21.15.43:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1034019013 0 0) 0x7fc8080ca410 con 0x7fc808069a30 2023-08-09T13:53:34.866 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:34.864+0000 7fc80d545640 1 -- 172.21.15.43:0/1832654606 --> v1:172.21.15.43:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fc7f4003520 con 0x7fc808069a30 2023-08-09T13:53:34.867 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:34.864+0000 7fc80ed48640 1 -- 172.21.15.43:0/1832654606 >> v1:172.21.15.203:6789/0 conn(0x7fc80815d5b0 legacy=0x7fc808069e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-09T13:53:34.867 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:34.864+0000 7fc80ed48640 1 -- 172.21.15.43:0/1832654606 >> v1:172.21.15.43:6789/0 conn(0x7fc808069a30 legacy=0x7fc808159760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:53:34.867 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:34.864+0000 7fc80ed48640 1 -- 172.21.15.43:0/1832654606 --> v2:172.21.15.43:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc80815a330 con 0x7fc80815da40 2023-08-09T13:53:34.867 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:34.865+0000 7fc80ed48640 1 --2- 172.21.15.43:0/1832654606 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc80815da40 0x7fc8080ce4c0 secure :-1 s=READY pgs=1455 cs=0 l=1 rev1=1 crypto rx=0x7fc800009d90 tx=0x7fc80002be20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f776febb6cc364ac server_cookie=0 in_seq=0 out_seq=0 2023-08-09T13:53:34.868 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:34.865+0000 7fc80d545640 1 -- 172.21.15.43:0/1832654606 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc80002cbd0 con 0x7fc80815da40 2023-08-09T13:53:34.868 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:34.865+0000 7fc80d545640 1 -- 172.21.15.43:0/1832654606 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc80002cd30 con 0x7fc80815da40 2023-08-09T13:53:34.868 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:34.865+0000 7fc80d545640 1 -- 172.21.15.43:0/1832654606 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc80002cea0 con 0x7fc80815da40 2023-08-09T13:53:34.868 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:34.866+0000 7fc8107ab640 1 -- 172.21.15.43:0/1832654606 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc80815da40 msgr2=0x7fc8080ce4c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:53:34.868 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:34.866+0000 7fc8107ab640 1 --2- 172.21.15.43:0/1832654606 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc80815da40 0x7fc8080ce4c0 secure :-1 s=READY pgs=1455 cs=0 l=1 rev1=1 crypto rx=0x7fc800009d90 tx=0x7fc80002be20 comp rx=0 tx=0).stop 2023-08-09T13:53:34.869 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:34.866+0000 7fc8107ab640 1 -- 172.21.15.43:0/1832654606 shutdown_connections 2023-08-09T13:53:34.869 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:34.866+0000 7fc8107ab640 1 --2- 172.21.15.43:0/1832654606 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc80815da40 0x7fc8080ce4c0 unknown :-1 s=CLOSED pgs=1455 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:53:34.869 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:34.866+0000 7fc8107ab640 1 -- 172.21.15.43:0/1832654606 >> 172.21.15.43:0/1832654606 conn(0x7fc80806b990 msgr2=0x7fc80806bda0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-09T13:53:34.869 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:34.866+0000 7fc8107ab640 1 -- 172.21.15.43:0/1832654606 shutdown_connections 2023-08-09T13:53:34.869 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:34.866+0000 7fc8107ab640 1 -- 172.21.15.43:0/1832654606 wait complete. 2023-08-09T13:53:34.870 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:34.868+0000 7fc8107ab640 1 Processor -- start 2023-08-09T13:53:34.870 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:34.868+0000 7fc8107ab640 1 -- start start 2023-08-09T13:53:34.871 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:34.869+0000 7fc8107ab640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc808069a30 0x7fc8080c7f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:53:34.871 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:34.869+0000 7fc8107ab640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fc80815d5b0 0x7fc8080c0d20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:53:34.871 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:34.869+0000 7fc8107ab640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fc80815da40 0x7fc8080acda0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:53:34.871 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:34.869+0000 7fc8107ab640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc8080c9160 con 0x7fc808069a30 2023-08-09T13:53:34.871 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:34.869+0000 7fc8107ab640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc8080c1260 con 0x7fc80815da40 2023-08-09T13:53:34.871 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:34.869+0000 7fc8107ab640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc8080c8550 con 0x7fc80815d5b0 2023-08-09T13:53:34.871 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:34.869+0000 7fc80e547640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc808069a30 0x7fc8080c7f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:53:34.872 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:34.869+0000 7fc80e547640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc808069a30 0x7fc8080c7f50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:58418/0 (socket says 172.21.15.43:58418) 2023-08-09T13:53:34.872 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:34.869+0000 7fc80e547640 1 -- 172.21.15.43:0/1240813651 learned_addr learned my addr 172.21.15.43:0/1240813651 (peer_addr_for_me v2:172.21.15.43:0/0) 2023-08-09T13:53:34.873 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:34.870+0000 7fc80e547640 1 -- 172.21.15.43:0/1240813651 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fc80815d5b0 msgr2=0x7fc8080c0d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:53:34.873 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:34.870+0000 7fc80e547640 1 --2- 172.21.15.43:0/1240813651 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fc80815d5b0 0x7fc8080c0d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:53:34.873 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:34.870+0000 7fc80e547640 1 -- 172.21.15.43:0/1240813651 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fc80815da40 msgr2=0x7fc8080acda0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:53:34.873 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:34.870+0000 7fc80e547640 1 --2- 172.21.15.43:0/1240813651 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fc80815da40 0x7fc8080acda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:53:34.873 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:34.870+0000 7fc80e547640 1 -- 172.21.15.43:0/1240813651 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc800009a70 con 0x7fc808069a30 2023-08-09T13:53:34.874 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:34.870+0000 7fc80e547640 1 --2- 172.21.15.43:0/1240813651 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc808069a30 0x7fc8080c7f50 secure :-1 s=READY pgs=1456 cs=0 l=1 rev1=1 crypto rx=0x7fc80400c390 tx=0x7fc80400c850 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-09T13:53:34.874 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:34.870+0000 7fc7feffd640 1 -- 172.21.15.43:0/1240813651 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc804005ce0 con 0x7fc808069a30 2023-08-09T13:53:34.874 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:34.870+0000 7fc7feffd640 1 -- 172.21.15.43:0/1240813651 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc804005e40 con 0x7fc808069a30 2023-08-09T13:53:34.874 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:34.870+0000 7fc8107ab640 1 -- 172.21.15.43:0/1240813651 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc8080c8830 con 0x7fc808069a30 2023-08-09T13:53:34.874 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:34.870+0000 7fc7feffd640 1 -- 172.21.15.43:0/1240813651 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc8040072f0 con 0x7fc808069a30 2023-08-09T13:53:34.874 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:34.871+0000 7fc8107ab640 1 -- 172.21.15.43:0/1240813651 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc8080c8df0 con 0x7fc808069a30 2023-08-09T13:53:34.874 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:34.872+0000 7fc8107ab640 1 -- 172.21.15.43:0/1240813651 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc8080739f0 con 0x7fc808069a30 2023-08-09T13:53:34.875 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:34.873+0000 7fc7feffd640 1 -- 172.21.15.43:0/1240813651 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 56) v1 ==== 88443+0+0 (secure 0 0 0) 0x7fc804007450 con 0x7fc808069a30 2023-08-09T13:53:34.875 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:34.874+0000 7fc7feffd640 1 --2- 172.21.15.43:0/1240813651 >> [v2:172.21.15.203:6832/2527059680,v1:172.21.15.203:6833/2527059680] conn(0x7fc7ec06a170 0x7fc7ec06c630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:53:34.875 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:34.874+0000 7fc7feffd640 1 -- 172.21.15.43:0/1240813651 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(119..119 src has 1..119) v4 ==== 7286+0+0 (secure 0 0 0) 0x7fc804088350 con 0x7fc808069a30 2023-08-09T13:53:34.876 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:34.875+0000 7fc80dd46640 1 --2- 172.21.15.43:0/1240813651 >> [v2:172.21.15.203:6832/2527059680,v1:172.21.15.203:6833/2527059680] conn(0x7fc7ec06a170 0x7fc7ec06c630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:53:34.877 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:34.876+0000 7fc80dd46640 1 --2- 172.21.15.43:0/1240813651 >> [v2:172.21.15.203:6832/2527059680,v1:172.21.15.203:6833/2527059680] conn(0x7fc7ec06a170 0x7fc7ec06c630 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0x7fc80813f770 tx=0x7fc7f80079b0 comp rx=0 tx=0).ready entity=mgr.6377 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-09T13:53:34.879 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:34.878+0000 7fc7feffd640 1 -- 172.21.15.43:0/1240813651 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178066 (secure 0 0 0) 0x7fc8040533f0 con 0x7fc808069a30 2023-08-09T13:53:35.074 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:35.072+0000 7fc8107ab640 1 -- 172.21.15.43:0/1240813651 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7fc8080cbd00 con 0x7fc808069a30 2023-08-09T13:53:35.574 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:35.572+0000 7fc7feffd640 1 -- 172.21.15.43:0/1240813651 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_metadata' removed v120) v1 ==== 181+0+0 (secure 0 0 0) 0x7fc8040170d0 con 0x7fc808069a30 2023-08-09T13:53:35.576 INFO:teuthology.orchestra.run.smithi043.stderr:pool 'cephfs_metadata' removed 2023-08-09T13:53:35.580 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:35.579+0000 7fc8107ab640 1 -- 172.21.15.43:0/1240813651 >> [v2:172.21.15.203:6832/2527059680,v1:172.21.15.203:6833/2527059680] conn(0x7fc7ec06a170 msgr2=0x7fc7ec06c630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:53:35.581 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:35.579+0000 7fc8107ab640 1 --2- 172.21.15.43:0/1240813651 >> [v2:172.21.15.203:6832/2527059680,v1:172.21.15.203:6833/2527059680] conn(0x7fc7ec06a170 0x7fc7ec06c630 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0x7fc80813f770 tx=0x7fc7f80079b0 comp rx=0 tx=0).stop 2023-08-09T13:53:35.581 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:35.579+0000 7fc8107ab640 1 -- 172.21.15.43:0/1240813651 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc808069a30 msgr2=0x7fc8080c7f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:53:35.581 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:35.579+0000 7fc8107ab640 1 --2- 172.21.15.43:0/1240813651 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc808069a30 0x7fc8080c7f50 secure :-1 s=READY pgs=1456 cs=0 l=1 rev1=1 crypto rx=0x7fc80400c390 tx=0x7fc80400c850 comp rx=0 tx=0).stop 2023-08-09T13:53:35.582 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:35.581+0000 7fc8107ab640 1 -- 172.21.15.43:0/1240813651 shutdown_connections 2023-08-09T13:53:35.582 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:35.581+0000 7fc8107ab640 1 --2- 172.21.15.43:0/1240813651 >> [v2:172.21.15.203:6832/2527059680,v1:172.21.15.203:6833/2527059680] conn(0x7fc7ec06a170 0x7fc7ec06c630 unknown :-1 s=CLOSED pgs=33 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:53:35.582 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:35.581+0000 7fc8107ab640 1 --2- 172.21.15.43:0/1240813651 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fc80815da40 0x7fc8080acda0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:53:35.583 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:35.581+0000 7fc8107ab640 1 --2- 172.21.15.43:0/1240813651 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fc80815d5b0 0x7fc8080c0d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:53:35.583 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:35.581+0000 7fc8107ab640 1 --2- 172.21.15.43:0/1240813651 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc808069a30 0x7fc8080c7f50 unknown :-1 s=CLOSED pgs=1456 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:53:35.583 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:35.581+0000 7fc8107ab640 1 -- 172.21.15.43:0/1240813651 >> 172.21.15.43:0/1240813651 conn(0x7fc80806b990 msgr2=0x7fc80815aa80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-09T13:53:35.584 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:35.581+0000 7fc8107ab640 1 -- 172.21.15.43:0/1240813651 shutdown_connections 2023-08-09T13:53:35.584 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:35.582+0000 7fc8107ab640 1 -- 172.21.15.43:0/1240813651 wait complete. 2023-08-09T13:53:35.609 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_data cephfs_data --yes-i-really-really-mean-it 2023-08-09T13:53:35.893 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:35.892+0000 7fcbd426c640 1 Processor -- start 2023-08-09T13:53:35.894 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:35.892+0000 7fcbd426c640 1 -- start start 2023-08-09T13:53:35.895 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:35.893+0000 7fcbd426c640 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7fcbcc0ca370 0x7fcbcc15b2a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:53:35.895 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:35.893+0000 7fcbd426c640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7fcbcc0ce4c0 0x7fcbcc0ce950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:53:35.895 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:35.893+0000 7fcbd426c640 1 -- --> v1:172.21.15.43:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fcbcc15ae10 con 0x7fcbcc06fb70 2023-08-09T13:53:35.896 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:35.893+0000 7fcbd426c640 1 -- --> v2:172.21.15.43:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fcbcc0cee90 con 0x7fcbcc0ce4c0 2023-08-09T13:53:35.896 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:35.893+0000 7fcbd426c640 1 -- --> v2:172.21.15.203:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fcbcc154e70 con 0x7fcbcc0ca370 2023-08-09T13:53:35.896 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:35.894+0000 7fcbd2809640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7fcbcc0ce4c0 0x7fcbcc0ce950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:53:35.896 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:35.894+0000 7fcbd2008640 1 --1- >> v1:172.21.15.43:6789/0 conn(0x7fcbcc06fb70 0x7fcbcc0caab0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.43:6789/0 says I am v1:172.21.15.43:44744/0 (socket says 172.21.15.43:44744) 2023-08-09T13:53:35.896 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:35.894+0000 7fcbd2008640 1 -- 172.21.15.43:0/95132761 learned_addr learned my addr 172.21.15.43:0/95132761 (peer_addr_for_me v1:172.21.15.43:0/0) 2023-08-09T13:53:35.897 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:35.894+0000 7fcbd1807640 1 --2- 172.21.15.43:0/95132761 >> v2:172.21.15.203:3300/0 conn(0x7fcbcc0ca370 0x7fcbcc15b2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:53:35.897 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:35.894+0000 7fcbd2809640 1 -- 172.21.15.43:0/95132761 >> v2:172.21.15.203:3300/0 conn(0x7fcbcc0ca370 msgr2=0x7fcbcc15b2a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-09T13:53:35.897 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:35.894+0000 7fcbd2809640 1 --2- 172.21.15.43:0/95132761 >> v2:172.21.15.203:3300/0 conn(0x7fcbcc0ca370 0x7fcbcc15b2a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:53:35.897 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:35.894+0000 7fcbd2809640 1 -- 172.21.15.43:0/95132761 >> v1:172.21.15.43:6789/0 conn(0x7fcbcc06fb70 legacy=0x7fcbcc0caab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:53:35.898 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:35.894+0000 7fcbd2809640 1 -- 172.21.15.43:0/95132761 --> v2:172.21.15.43:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcbcc06ff50 con 0x7fcbcc0ce4c0 2023-08-09T13:53:35.898 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:35.894+0000 7fcbd1006640 1 -- 172.21.15.43:0/95132761 <== mon.0 v1:172.21.15.43:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2217802676 0 0) 0x7fcbcc15ae10 con 0x7fcbcc06fb70 2023-08-09T13:53:35.898 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:35.895+0000 7fcbd2809640 1 --2- 172.21.15.43:0/95132761 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fcbcc0ce4c0 0x7fcbcc0ce950 secure :-1 s=READY pgs=1458 cs=0 l=1 rev1=1 crypto rx=0x7fcbcc0711a0 tx=0x7fcbc402bec0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4e08bacd061ec04 server_cookie=0 in_seq=0 out_seq=0 2023-08-09T13:53:35.898 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:35.896+0000 7fcbd1006640 1 -- 172.21.15.43:0/95132761 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcbc402cc70 con 0x7fcbcc0ce4c0 2023-08-09T13:53:35.898 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:35.896+0000 7fcbd1006640 1 -- 172.21.15.43:0/95132761 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcbc402cdd0 con 0x7fcbcc0ce4c0 2023-08-09T13:53:35.898 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:35.896+0000 7fcbd1006640 1 -- 172.21.15.43:0/95132761 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcbc4034660 con 0x7fcbcc0ce4c0 2023-08-09T13:53:35.899 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:35.896+0000 7fcbd426c640 1 -- 172.21.15.43:0/95132761 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fcbcc0ce4c0 msgr2=0x7fcbcc0ce950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:53:35.899 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:35.896+0000 7fcbd426c640 1 --2- 172.21.15.43:0/95132761 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fcbcc0ce4c0 0x7fcbcc0ce950 secure :-1 s=READY pgs=1458 cs=0 l=1 rev1=1 crypto rx=0x7fcbcc0711a0 tx=0x7fcbc402bec0 comp rx=0 tx=0).stop 2023-08-09T13:53:35.899 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:35.896+0000 7fcbd426c640 1 -- 172.21.15.43:0/95132761 shutdown_connections 2023-08-09T13:53:35.899 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:35.896+0000 7fcbd426c640 1 --2- 172.21.15.43:0/95132761 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fcbcc0ce4c0 0x7fcbcc0ce950 unknown :-1 s=CLOSED pgs=1458 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:53:35.900 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:35.896+0000 7fcbd426c640 1 --2- 172.21.15.43:0/95132761 >> v2:172.21.15.203:3300/0 conn(0x7fcbcc0ca370 0x7fcbcc15b2a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:53:35.900 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:35.896+0000 7fcbd426c640 1 -- 172.21.15.43:0/95132761 >> 172.21.15.43:0/95132761 conn(0x7fcbcc06b990 msgr2=0x7fcbcc06bda0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-09T13:53:35.900 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:35.897+0000 7fcbd426c640 1 -- 172.21.15.43:0/95132761 shutdown_connections 2023-08-09T13:53:35.900 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:35.897+0000 7fcbd426c640 1 -- 172.21.15.43:0/95132761 wait complete. 2023-08-09T13:53:35.901 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:35.898+0000 7fcbd426c640 1 Processor -- start 2023-08-09T13:53:35.901 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:35.899+0000 7fcbd426c640 1 -- start start 2023-08-09T13:53:35.902 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:35.900+0000 7fcbd426c640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fcbcc06fb70 0x7fcbcc144bc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:53:35.902 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:35.900+0000 7fcbd426c640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fcbcc0ca370 0x7fcbcc0c3920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:53:35.902 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:35.900+0000 7fcbd426c640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fcbcc0ce4c0 0x7fcbcc0c3e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:53:35.902 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:35.900+0000 7fcbd426c640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcbcc1458e0 con 0x7fcbcc0ce4c0 2023-08-09T13:53:35.902 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:35.900+0000 7fcbd426c640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcbcc0c44e0 con 0x7fcbcc06fb70 2023-08-09T13:53:35.902 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:35.900+0000 7fcbd426c640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcbcc0c1d50 con 0x7fcbcc0ca370 2023-08-09T13:53:35.902 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:35.900+0000 7fcbd2809640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fcbcc0ce4c0 0x7fcbcc0c3e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:53:35.903 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:35.900+0000 7fcbd2809640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fcbcc0ce4c0 0x7fcbcc0c3e60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:53348/0 (socket says 172.21.15.43:53348) 2023-08-09T13:53:35.903 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:35.900+0000 7fcbd2809640 1 -- 172.21.15.43:0/3755264894 learned_addr learned my addr 172.21.15.43:0/3755264894 (peer_addr_for_me v2:172.21.15.43:0/0) 2023-08-09T13:53:35.904 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:35.900+0000 7fcbd2008640 1 --2- 172.21.15.43:0/3755264894 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fcbcc06fb70 0x7fcbcc144bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:53:35.904 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:35.900+0000 7fcbd1807640 1 --2- 172.21.15.43:0/3755264894 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fcbcc0ca370 0x7fcbcc0c3920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:53:35.905 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:35.900+0000 7fcbd2809640 1 -- 172.21.15.43:0/3755264894 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fcbcc0ca370 msgr2=0x7fcbcc0c3920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:53:35.905 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:35.900+0000 7fcbd2809640 1 --2- 172.21.15.43:0/3755264894 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fcbcc0ca370 0x7fcbcc0c3920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:53:35.905 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:35.901+0000 7fcbd2809640 1 -- 172.21.15.43:0/3755264894 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fcbcc06fb70 msgr2=0x7fcbcc144bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:53:35.905 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:35.901+0000 7fcbd2809640 1 --2- 172.21.15.43:0/3755264894 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fcbcc06fb70 0x7fcbcc144bc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:53:35.905 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:35.901+0000 7fcbd2809640 1 -- 172.21.15.43:0/3755264894 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcbc4009a70 con 0x7fcbcc0ce4c0 2023-08-09T13:53:35.905 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:35.901+0000 7fcbd2809640 1 --2- 172.21.15.43:0/3755264894 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fcbcc0ce4c0 0x7fcbcc0c3e60 secure :-1 s=READY pgs=1459 cs=0 l=1 rev1=1 crypto rx=0x7fcbc4009e80 tx=0x7fcbc4004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-09T13:53:35.905 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:35.901+0000 7fcbc2ffd640 1 -- 172.21.15.43:0/3755264894 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcbc4040650 con 0x7fcbcc0ce4c0 2023-08-09T13:53:35.906 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:35.901+0000 7fcbc2ffd640 1 -- 172.21.15.43:0/3755264894 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcbc40407d0 con 0x7fcbcc0ce4c0 2023-08-09T13:53:35.906 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:35.901+0000 7fcbc2ffd640 1 -- 172.21.15.43:0/3755264894 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcbc4040b80 con 0x7fcbcc0ce4c0 2023-08-09T13:53:35.906 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:35.901+0000 7fcbd426c640 1 -- 172.21.15.43:0/3755264894 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcbcc0c2030 con 0x7fcbcc0ce4c0 2023-08-09T13:53:35.906 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:35.901+0000 7fcbd426c640 1 -- 172.21.15.43:0/3755264894 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcbcc0c25f0 con 0x7fcbcc0ce4c0 2023-08-09T13:53:35.906 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:35.902+0000 7fcbd426c640 1 -- 172.21.15.43:0/3755264894 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcbcc0c27e0 con 0x7fcbcc0ce4c0 2023-08-09T13:53:35.907 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:35.903+0000 7fcbc2ffd640 1 -- 172.21.15.43:0/3755264894 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 56) v1 ==== 88443+0+0 (secure 0 0 0) 0x7fcbc4040ce0 con 0x7fcbcc0ce4c0 2023-08-09T13:53:35.907 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:35.904+0000 7fcbc2ffd640 1 --2- 172.21.15.43:0/3755264894 >> [v2:172.21.15.203:6832/2527059680,v1:172.21.15.203:6833/2527059680] conn(0x7fcbb006a170 0x7fcbb006c630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:53:35.907 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:35.904+0000 7fcbc2ffd640 1 -- 172.21.15.43:0/3755264894 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(120..120 src has 1..120) v4 ==== 6848+0+0 (secure 0 0 0) 0x7fcbc40ab860 con 0x7fcbcc0ce4c0 2023-08-09T13:53:35.907 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:35.905+0000 7fcbd2008640 1 --2- 172.21.15.43:0/3755264894 >> [v2:172.21.15.203:6832/2527059680,v1:172.21.15.203:6833/2527059680] conn(0x7fcbb006a170 0x7fcbb006c630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:53:35.908 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:35.906+0000 7fcbd2008640 1 --2- 172.21.15.43:0/3755264894 >> [v2:172.21.15.203:6832/2527059680,v1:172.21.15.203:6833/2527059680] conn(0x7fcbb006a170 0x7fcbb006c630 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0x7fcbc8007fd0 tx=0x7fcbc8009040 comp rx=0 tx=0).ready entity=mgr.6377 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-09T13:53:35.910 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:35.909+0000 7fcbc2ffd640 1 -- 172.21.15.43:0/3755264894 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178066 (secure 0 0 0) 0x7fcbc4076bd0 con 0x7fcbcc0ce4c0 2023-08-09T13:53:36.103 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:36.102+0000 7fcbd426c640 1 -- 172.21.15.43:0/3755264894 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7fcbcc0739f0 con 0x7fcbcc0ce4c0 2023-08-09T13:53:36.587 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:36.584+0000 7fcbc2ffd640 1 -- 172.21.15.43:0/3755264894 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data' removed v121) v1 ==== 169+0+0 (secure 0 0 0) 0x7fcbc407a1f0 con 0x7fcbcc0ce4c0 2023-08-09T13:53:36.588 INFO:teuthology.orchestra.run.smithi043.stderr:pool 'cephfs_data' removed 2023-08-09T13:53:36.595 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:36.592+0000 7fcbd426c640 1 -- 172.21.15.43:0/3755264894 >> [v2:172.21.15.203:6832/2527059680,v1:172.21.15.203:6833/2527059680] conn(0x7fcbb006a170 msgr2=0x7fcbb006c630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:53:36.595 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:36.592+0000 7fcbd426c640 1 --2- 172.21.15.43:0/3755264894 >> [v2:172.21.15.203:6832/2527059680,v1:172.21.15.203:6833/2527059680] conn(0x7fcbb006a170 0x7fcbb006c630 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0x7fcbc8007fd0 tx=0x7fcbc8009040 comp rx=0 tx=0).stop 2023-08-09T13:53:36.595 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:36.592+0000 7fcbd426c640 1 -- 172.21.15.43:0/3755264894 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fcbcc0ce4c0 msgr2=0x7fcbcc0c3e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:53:36.595 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:36.592+0000 7fcbd426c640 1 --2- 172.21.15.43:0/3755264894 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fcbcc0ce4c0 0x7fcbcc0c3e60 secure :-1 s=READY pgs=1459 cs=0 l=1 rev1=1 crypto rx=0x7fcbc4009e80 tx=0x7fcbc4004a40 comp rx=0 tx=0).stop 2023-08-09T13:53:36.596 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:36.593+0000 7fcbd426c640 1 -- 172.21.15.43:0/3755264894 shutdown_connections 2023-08-09T13:53:36.596 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:36.593+0000 7fcbd426c640 1 --2- 172.21.15.43:0/3755264894 >> [v2:172.21.15.203:6832/2527059680,v1:172.21.15.203:6833/2527059680] conn(0x7fcbb006a170 0x7fcbb006c630 unknown :-1 s=CLOSED pgs=34 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:53:36.596 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:36.593+0000 7fcbd426c640 1 --2- 172.21.15.43:0/3755264894 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fcbcc0ce4c0 0x7fcbcc0c3e60 unknown :-1 s=CLOSED pgs=1459 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:53:36.596 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:36.593+0000 7fcbd426c640 1 --2- 172.21.15.43:0/3755264894 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fcbcc0ca370 0x7fcbcc0c3920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:53:36.596 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:36.593+0000 7fcbd426c640 1 --2- 172.21.15.43:0/3755264894 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fcbcc06fb70 0x7fcbcc144bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:53:36.596 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:36.593+0000 7fcbd426c640 1 -- 172.21.15.43:0/3755264894 >> 172.21.15.43:0/3755264894 conn(0x7fcbcc06b990 msgr2=0x7fcbcc0b2c30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-09T13:53:36.596 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:36.594+0000 7fcbd426c640 1 -- 172.21.15.43:0/3755264894 shutdown_connections 2023-08-09T13:53:36.596 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:36.594+0000 7fcbd426c640 1 -- 172.21.15.43:0/3755264894 wait complete. 2023-08-09T13:53:36.620 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config rm mgr mgr/snap_schedule/dump_on_update 2023-08-09T13:53:36.831 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:36.830+0000 7f787eab4640 1 Processor -- start 2023-08-09T13:53:36.832 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:36.830+0000 7f787eab4640 1 -- start start 2023-08-09T13:53:36.833 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:36.831+0000 7f787eab4640 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7f787815c550 0x7f78780cd6d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:53:36.833 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:36.831+0000 7f787eab4640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f78780d04b0 0x7f78780b3ee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:53:36.833 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:36.831+0000 7f787eab4640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f78780b4420 0x7f787806a760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:53:36.833 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:36.831+0000 7f787eab4640 1 -- --> v2:172.21.15.43:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f78780b4870 con 0x7f78780b4420 2023-08-09T13:53:36.833 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:36.832+0000 7f787eab4640 1 -- --> v2:172.21.15.203:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f78780b7bd0 con 0x7f787815c550 2023-08-09T13:53:36.834 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:36.832+0000 7f787eab4640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f78780d1c00 con 0x7f78780d04b0 2023-08-09T13:53:36.834 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:36.832+0000 7f787d051640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f78780b4420 0x7f787806a760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:53:36.835 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:36.832+0000 7f787d051640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f78780b4420 0x7f787806a760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:53360/0 (socket says 172.21.15.43:53360) 2023-08-09T13:53:36.835 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:36.832+0000 7f787d051640 1 -- 172.21.15.43:0/2869852341 learned_addr learned my addr 172.21.15.43:0/2869852341 (peer_addr_for_me v2:172.21.15.43:0/0) 2023-08-09T13:53:36.835 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:36.832+0000 7f787c850640 1 --2- 172.21.15.43:0/2869852341 >> v2:172.21.15.203:3300/0 conn(0x7f787815c550 0x7f78780cd6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:53:36.835 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:36.832+0000 7f7877fff640 1 --2- 172.21.15.43:0/2869852341 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f78780d04b0 0x7f78780b3ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:53:36.835 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:36.832+0000 7f787d051640 1 -- 172.21.15.43:0/2869852341 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f78780d04b0 msgr2=0x7f78780b3ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-09T13:53:36.835 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:36.832+0000 7f787d051640 1 --2- 172.21.15.43:0/2869852341 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f78780d04b0 0x7f78780b3ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:53:36.835 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:36.832+0000 7f787d051640 1 -- 172.21.15.43:0/2869852341 >> v2:172.21.15.203:3300/0 conn(0x7f787815c550 msgr2=0x7f78780cd6d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-09T13:53:36.836 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:36.832+0000 7f787d051640 1 --2- 172.21.15.43:0/2869852341 >> v2:172.21.15.203:3300/0 conn(0x7f787815c550 0x7f78780cd6d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:53:36.836 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:36.832+0000 7f787d051640 1 -- 172.21.15.43:0/2869852341 --> v2:172.21.15.43:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f787815c930 con 0x7f78780b4420 2023-08-09T13:53:36.836 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:36.832+0000 7f7877fff640 1 --2- 172.21.15.43:0/2869852341 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f78780d04b0 0x7f78780b3ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-08-09T13:53:36.837 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:36.834+0000 7f787d051640 1 --2- 172.21.15.43:0/2869852341 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f78780b4420 0x7f787806a760 secure :-1 s=READY pgs=1460 cs=0 l=1 rev1=1 crypto rx=0x7f78780d01c0 tx=0x7f787002b910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=32147ce6f5d70671 server_cookie=0 in_seq=0 out_seq=0 2023-08-09T13:53:36.837 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:36.834+0000 7f78777fe640 1 -- 172.21.15.43:0/2869852341 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7870031870 con 0x7f78780b4420 2023-08-09T13:53:36.837 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:36.834+0000 7f78777fe640 1 -- 172.21.15.43:0/2869852341 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f78700319d0 con 0x7f78780b4420 2023-08-09T13:53:36.837 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:36.834+0000 7f78777fe640 1 -- 172.21.15.43:0/2869852341 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7870031d60 con 0x7f78780b4420 2023-08-09T13:53:36.838 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:36.834+0000 7f787eab4640 1 -- 172.21.15.43:0/2869852341 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f78780b4420 msgr2=0x7f787806a760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:53:36.838 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:36.834+0000 7f787eab4640 1 --2- 172.21.15.43:0/2869852341 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f78780b4420 0x7f787806a760 secure :-1 s=READY pgs=1460 cs=0 l=1 rev1=1 crypto rx=0x7f78780d01c0 tx=0x7f787002b910 comp rx=0 tx=0).stop 2023-08-09T13:53:36.838 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:36.835+0000 7f787eab4640 1 -- 172.21.15.43:0/2869852341 shutdown_connections 2023-08-09T13:53:36.838 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:36.835+0000 7f787eab4640 1 --2- 172.21.15.43:0/2869852341 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f78780b4420 0x7f787806a760 unknown :-1 s=CLOSED pgs=1460 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:53:36.839 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:36.835+0000 7f787eab4640 1 --2- 172.21.15.43:0/2869852341 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f78780d04b0 0x7f78780b3ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:53:36.839 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:36.835+0000 7f787eab4640 1 --2- 172.21.15.43:0/2869852341 >> v2:172.21.15.203:3300/0 conn(0x7f787815c550 0x7f78780cd6d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:53:36.839 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:36.835+0000 7f787eab4640 1 -- 172.21.15.43:0/2869852341 >> 172.21.15.43:0/2869852341 conn(0x7f7878071520 msgr2=0x7f78780ceee0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-09T13:53:36.839 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:36.835+0000 7f787eab4640 1 -- 172.21.15.43:0/2869852341 shutdown_connections 2023-08-09T13:53:36.839 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:36.835+0000 7f787eab4640 1 -- 172.21.15.43:0/2869852341 wait complete. 2023-08-09T13:53:36.841 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:36.837+0000 7f787eab4640 1 Processor -- start 2023-08-09T13:53:36.841 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:36.837+0000 7f787eab4640 1 -- start start 2023-08-09T13:53:36.841 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:36.838+0000 7f787eab4640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f78780b4420 0x7f7878158a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:53:36.841 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:36.838+0000 7f787eab4640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f78780d04b0 0x7f78780a8dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:53:36.842 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:36.838+0000 7f787eab4640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f787815c550 0x7f787815cac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:53:36.842 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:36.838+0000 7f787eab4640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7878158fa0 con 0x7f78780d04b0 2023-08-09T13:53:36.842 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:36.838+0000 7f787eab4640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f78781f3100 con 0x7f78780b4420 2023-08-09T13:53:36.842 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:36.838+0000 7f787eab4640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f78781f33e0 con 0x7f787815c550 2023-08-09T13:53:36.842 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:36.838+0000 7f7877fff640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f78780d04b0 0x7f78780a8dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:53:36.842 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:36.838+0000 7f7877fff640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f78780d04b0 0x7f78780a8dc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:53374/0 (socket says 172.21.15.43:53374) 2023-08-09T13:53:36.842 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:36.838+0000 7f7877fff640 1 -- 172.21.15.43:0/3235526428 learned_addr learned my addr 172.21.15.43:0/3235526428 (peer_addr_for_me v2:172.21.15.43:0/0) 2023-08-09T13:53:36.843 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:36.838+0000 7f787d051640 1 --2- 172.21.15.43:0/3235526428 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f787815c550 0x7f787815cac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:53:36.843 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:36.838+0000 7f787c850640 1 --2- 172.21.15.43:0/3235526428 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f78780b4420 0x7f7878158a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:53:36.843 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:36.839+0000 7f7877fff640 1 -- 172.21.15.43:0/3235526428 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f787815c550 msgr2=0x7f787815cac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:53:36.843 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:36.839+0000 7f7877fff640 1 --2- 172.21.15.43:0/3235526428 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f787815c550 0x7f787815cac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:53:36.843 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:36.839+0000 7f7877fff640 1 -- 172.21.15.43:0/3235526428 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f78780b4420 msgr2=0x7f7878158a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:53:36.843 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:36.839+0000 7f7877fff640 1 --2- 172.21.15.43:0/3235526428 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f78780b4420 0x7f7878158a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:53:36.843 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:36.839+0000 7f7877fff640 1 -- 172.21.15.43:0/3235526428 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f78700094d0 con 0x7f78780d04b0 2023-08-09T13:53:36.844 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:36.839+0000 7f7877fff640 1 --2- 172.21.15.43:0/3235526428 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f78780d04b0 0x7f78780a8dc0 secure :-1 s=READY pgs=1461 cs=0 l=1 rev1=1 crypto rx=0x7f786c00cf60 tx=0x7f786c00cf90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-09T13:53:36.844 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:36.839+0000 7f78757fa640 1 -- 172.21.15.43:0/3235526428 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f786c00d840 con 0x7f78780d04b0 2023-08-09T13:53:36.844 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:36.839+0000 7f78757fa640 1 -- 172.21.15.43:0/3235526428 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f786c00d9a0 con 0x7f78780d04b0 2023-08-09T13:53:36.844 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:36.839+0000 7f78757fa640 1 -- 172.21.15.43:0/3235526428 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f786c00dd30 con 0x7f78780d04b0 2023-08-09T13:53:36.844 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:36.839+0000 7f787eab4640 1 -- 172.21.15.43:0/3235526428 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f78781f36c0 con 0x7f78780d04b0 2023-08-09T13:53:36.844 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:36.840+0000 7f787eab4640 1 -- 172.21.15.43:0/3235526428 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f78781f3c80 con 0x7f78780d04b0 2023-08-09T13:53:36.845 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:36.841+0000 7f787eab4640 1 -- 172.21.15.43:0/3235526428 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f78780739f0 con 0x7f78780d04b0 2023-08-09T13:53:36.845 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:36.842+0000 7f78757fa640 1 -- 172.21.15.43:0/3235526428 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 56) v1 ==== 88443+0+0 (secure 0 0 0) 0x7f786c015440 con 0x7f78780d04b0 2023-08-09T13:53:36.845 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:36.843+0000 7f78757fa640 1 --2- 172.21.15.43:0/3235526428 >> [v2:172.21.15.203:6832/2527059680,v1:172.21.15.203:6833/2527059680] conn(0x7f786406a170 0x7f786406c630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:53:36.846 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:36.843+0000 7f78757fa640 1 -- 172.21.15.43:0/3235526428 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(121..121 src has 1..121) v4 ==== 6434+0+0 (secure 0 0 0) 0x7f786c0892a0 con 0x7f78780d04b0 2023-08-09T13:53:36.846 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:36.843+0000 7f787c850640 1 --2- 172.21.15.43:0/3235526428 >> [v2:172.21.15.203:6832/2527059680,v1:172.21.15.203:6833/2527059680] conn(0x7f786406a170 0x7f786406c630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:53:36.846 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:36.845+0000 7f787c850640 1 --2- 172.21.15.43:0/3235526428 >> [v2:172.21.15.203:6832/2527059680,v1:172.21.15.203:6833/2527059680] conn(0x7f786406a170 0x7f786406c630 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7f7868006fd0 tx=0x7f7868008040 comp rx=0 tx=0).ready entity=mgr.6377 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-09T13:53:36.848 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:36.847+0000 7f78757fa640 1 -- 172.21.15.43:0/3235526428 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178066 (secure 0 0 0) 0x7f786c054710 con 0x7f78780d04b0 2023-08-09T13:53:37.044 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:37.043+0000 7f787eab4640 1 -- 172.21.15.43:0/3235526428 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "config rm", "who": "mgr", "name": "mgr/snap_schedule/dump_on_update"} v 0) v1 -- 0x7f787815e6e0 con 0x7f78780d04b0 2023-08-09T13:53:37.053 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:37.052+0000 7f78757fa640 1 -- 172.21.15.43:0/3235526428 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "config rm", "who": "mgr", "name": "mgr/snap_schedule/dump_on_update"}]=0 v37) v1 ==== 115+0+0 (secure 0 0 0) 0x7f786c01a0b0 con 0x7f78780d04b0 2023-08-09T13:53:37.056 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:37.054+0000 7f787eab4640 1 -- 172.21.15.43:0/3235526428 >> [v2:172.21.15.203:6832/2527059680,v1:172.21.15.203:6833/2527059680] conn(0x7f786406a170 msgr2=0x7f786406c630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:53:37.056 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:37.054+0000 7f787eab4640 1 --2- 172.21.15.43:0/3235526428 >> [v2:172.21.15.203:6832/2527059680,v1:172.21.15.203:6833/2527059680] conn(0x7f786406a170 0x7f786406c630 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7f7868006fd0 tx=0x7f7868008040 comp rx=0 tx=0).stop 2023-08-09T13:53:37.056 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:37.054+0000 7f787eab4640 1 -- 172.21.15.43:0/3235526428 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f78780d04b0 msgr2=0x7f78780a8dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:53:37.056 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:37.054+0000 7f787eab4640 1 --2- 172.21.15.43:0/3235526428 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f78780d04b0 0x7f78780a8dc0 secure :-1 s=READY pgs=1461 cs=0 l=1 rev1=1 crypto rx=0x7f786c00cf60 tx=0x7f786c00cf90 comp rx=0 tx=0).stop 2023-08-09T13:53:37.056 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:37.055+0000 7f787eab4640 1 -- 172.21.15.43:0/3235526428 shutdown_connections 2023-08-09T13:53:37.057 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:37.055+0000 7f787eab4640 1 --2- 172.21.15.43:0/3235526428 >> [v2:172.21.15.203:6832/2527059680,v1:172.21.15.203:6833/2527059680] conn(0x7f786406a170 0x7f786406c630 unknown :-1 s=CLOSED pgs=35 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:53:37.057 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:37.055+0000 7f787eab4640 1 --2- 172.21.15.43:0/3235526428 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f787815c550 0x7f787815cac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:53:37.057 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:37.055+0000 7f787eab4640 1 --2- 172.21.15.43:0/3235526428 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f78780d04b0 0x7f78780a8dc0 unknown :-1 s=CLOSED pgs=1461 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:53:37.057 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:37.055+0000 7f787eab4640 1 --2- 172.21.15.43:0/3235526428 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f78780b4420 0x7f7878158a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:53:37.057 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:37.055+0000 7f787eab4640 1 -- 172.21.15.43:0/3235526428 >> 172.21.15.43:0/3235526428 conn(0x7f7878071520 msgr2=0x7f78780ceec0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-09T13:53:37.057 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:37.055+0000 7f787eab4640 1 -- 172.21.15.43:0/3235526428 shutdown_connections 2023-08-09T13:53:37.058 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:37.055+0000 7f787eab4640 1 -- 172.21.15.43:0/3235526428 wait complete. 2023-08-09T13:53:37.074 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config rm mgr mgr/snap_schedule/allow_m_granularity 2023-08-09T13:53:37.295 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:37.293+0000 7fd59da56640 1 Processor -- start 2023-08-09T13:53:37.295 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:37.294+0000 7fd59da56640 1 -- start start 2023-08-09T13:53:37.296 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:37.295+0000 7fd59da56640 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7fd598157ea0 0x7fd5980c97d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:53:37.297 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:37.295+0000 7fd59da56640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fd5980c90a0 0x7fd5980c9d10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:53:37.297 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:37.295+0000 7fd59da56640 1 -- --> v1:172.21.15.203:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd598158280 con 0x7fd59815a800 2023-08-09T13:53:37.297 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:37.295+0000 7fd59da56640 1 -- --> v2:172.21.15.203:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd59815b380 con 0x7fd598157ea0 2023-08-09T13:53:37.297 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:37.295+0000 7fd59da56640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd59815b840 con 0x7fd5980c90a0 2023-08-09T13:53:37.298 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:37.296+0000 7fd597fff640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fd5980c90a0 0x7fd5980c9d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:53:37.298 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:37.296+0000 7fd59ca54640 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7fd598157ea0 0x7fd5980c97d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:53:37.298 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:37.296+0000 7fd597fff640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fd5980c90a0 0x7fd5980c9d10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.203:3301/0 says I am v2:172.21.15.43:43464/0 (socket says 172.21.15.43:43464) 2023-08-09T13:53:37.298 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:37.296+0000 7fd597fff640 1 -- 172.21.15.43:0/407122968 learned_addr learned my addr 172.21.15.43:0/407122968 (peer_addr_for_me v2:172.21.15.43:0/0) 2023-08-09T13:53:37.299 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:37.297+0000 7fd597fff640 1 -- 172.21.15.43:0/407122968 >> v2:172.21.15.203:3300/0 conn(0x7fd598157ea0 msgr2=0x7fd5980c97d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-09T13:53:37.299 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:37.297+0000 7fd597fff640 1 --2- 172.21.15.43:0/407122968 >> v2:172.21.15.203:3300/0 conn(0x7fd598157ea0 0x7fd5980c97d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:53:37.299 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:37.297+0000 7fd597fff640 1 -- 172.21.15.43:0/407122968 >> v1:172.21.15.203:6789/0 conn(0x7fd59815a800 legacy=0x7fd59815ac70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:53:37.299 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:37.297+0000 7fd597fff640 1 -- 172.21.15.43:0/407122968 --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd59815b560 con 0x7fd5980c90a0 2023-08-09T13:53:37.299 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:37.297+0000 7fd59ca54640 1 --2- 172.21.15.43:0/407122968 >> v2:172.21.15.203:3300/0 conn(0x7fd598157ea0 0x7fd5980c97d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-08-09T13:53:37.300 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:37.299+0000 7fd597fff640 1 --2- 172.21.15.43:0/407122968 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fd5980c90a0 0x7fd5980c9d10 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7fd584009870 tx=0x7fd58402b8f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=84e2453d32d88e7b server_cookie=0 in_seq=0 out_seq=0 2023-08-09T13:53:37.300 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:37.299+0000 7fd5977fe640 1 -- 172.21.15.43:0/407122968 <== mon.2 v2:172.21.15.203:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd584031640 con 0x7fd5980c90a0 2023-08-09T13:53:37.301 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:37.299+0000 7fd5977fe640 1 -- 172.21.15.43:0/407122968 <== mon.2 v2:172.21.15.203:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd5840317a0 con 0x7fd5980c90a0 2023-08-09T13:53:37.301 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:37.299+0000 7fd5977fe640 1 -- 172.21.15.43:0/407122968 <== mon.2 v2:172.21.15.203:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd584031b30 con 0x7fd5980c90a0 2023-08-09T13:53:37.301 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:37.300+0000 7fd59da56640 1 -- 172.21.15.43:0/407122968 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fd5980c90a0 msgr2=0x7fd5980c9d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:53:37.301 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:37.300+0000 7fd59da56640 1 --2- 172.21.15.43:0/407122968 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fd5980c90a0 0x7fd5980c9d10 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7fd584009870 tx=0x7fd58402b8f0 comp rx=0 tx=0).stop 2023-08-09T13:53:37.302 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:37.300+0000 7fd59da56640 1 -- 172.21.15.43:0/407122968 shutdown_connections 2023-08-09T13:53:37.302 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:37.300+0000 7fd59da56640 1 --2- 172.21.15.43:0/407122968 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fd5980c90a0 0x7fd5980c9d10 unknown :-1 s=CLOSED pgs=198 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:53:37.302 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:37.300+0000 7fd59da56640 1 --2- 172.21.15.43:0/407122968 >> v2:172.21.15.203:3300/0 conn(0x7fd598157ea0 0x7fd5980c97d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:53:37.302 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:37.300+0000 7fd59da56640 1 -- 172.21.15.43:0/407122968 >> 172.21.15.43:0/407122968 conn(0x7fd5980cddd0 msgr2=0x7fd5980ce1e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-09T13:53:37.302 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:37.300+0000 7fd59da56640 1 -- 172.21.15.43:0/407122968 shutdown_connections 2023-08-09T13:53:37.302 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:37.300+0000 7fd59da56640 1 -- 172.21.15.43:0/407122968 wait complete. 2023-08-09T13:53:37.304 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:37.302+0000 7fd59da56640 1 Processor -- start 2023-08-09T13:53:37.304 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:37.303+0000 7fd59da56640 1 -- start start 2023-08-09T13:53:37.305 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:37.304+0000 7fd59da56640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fd5980c90a0 0x7fd5980c6150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:53:37.306 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:37.304+0000 7fd59da56640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fd598157ea0 0x7fd5981f23c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:53:37.306 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:37.304+0000 7fd59da56640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fd59815a800 0x7fd5981f4df0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:53:37.306 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:37.304+0000 7fd59da56640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd5980c1c70 con 0x7fd598157ea0 2023-08-09T13:53:37.306 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:37.304+0000 7fd59da56640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd5981f5330 con 0x7fd59815a800 2023-08-09T13:53:37.306 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:37.304+0000 7fd59da56640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd5981f5600 con 0x7fd5980c90a0 2023-08-09T13:53:37.307 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:37.304+0000 7fd597fff640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fd598157ea0 0x7fd5981f23c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:53:37.308 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:37.305+0000 7fd597fff640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fd598157ea0 0x7fd5981f23c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:53380/0 (socket says 172.21.15.43:53380) 2023-08-09T13:53:37.308 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:37.305+0000 7fd597fff640 1 -- 172.21.15.43:0/1354228081 learned_addr learned my addr 172.21.15.43:0/1354228081 (peer_addr_for_me v2:172.21.15.43:0/0) 2023-08-09T13:53:37.308 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:37.305+0000 7fd59ca54640 1 --2- 172.21.15.43:0/1354228081 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fd5980c90a0 0x7fd5980c6150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:53:37.308 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:37.305+0000 7fd59d255640 1 --2- 172.21.15.43:0/1354228081 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fd59815a800 0x7fd5981f4df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:53:37.308 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:37.305+0000 7fd597fff640 1 -- 172.21.15.43:0/1354228081 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fd5980c90a0 msgr2=0x7fd5980c6150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:53:37.308 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:37.305+0000 7fd597fff640 1 --2- 172.21.15.43:0/1354228081 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fd5980c90a0 0x7fd5980c6150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:53:37.309 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:37.305+0000 7fd597fff640 1 -- 172.21.15.43:0/1354228081 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fd59815a800 msgr2=0x7fd5981f4df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:53:37.309 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:37.305+0000 7fd597fff640 1 --2- 172.21.15.43:0/1354228081 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fd59815a800 0x7fd5981f4df0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:53:37.309 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:37.305+0000 7fd597fff640 1 -- 172.21.15.43:0/1354228081 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd58c009400 con 0x7fd598157ea0 2023-08-09T13:53:37.309 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:37.305+0000 7fd59ca54640 1 --2- 172.21.15.43:0/1354228081 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fd5980c90a0 0x7fd5980c6150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-08-09T13:53:37.309 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:37.305+0000 7fd59d255640 1 --2- 172.21.15.43:0/1354228081 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fd59815a800 0x7fd5981f4df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-08-09T13:53:37.309 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:37.306+0000 7fd597fff640 1 --2- 172.21.15.43:0/1354228081 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fd598157ea0 0x7fd5981f23c0 secure :-1 s=READY pgs=1462 cs=0 l=1 rev1=1 crypto rx=0x7fd584000c00 tx=0x7fd5840046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-09T13:53:37.309 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:37.306+0000 7fd5957fa640 1 -- 172.21.15.43:0/1354228081 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd58403a930 con 0x7fd598157ea0 2023-08-09T13:53:37.309 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:37.306+0000 7fd5957fa640 1 -- 172.21.15.43:0/1354228081 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd58403aa90 con 0x7fd598157ea0 2023-08-09T13:53:37.310 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:37.306+0000 7fd5957fa640 1 -- 172.21.15.43:0/1354228081 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd58403ae40 con 0x7fd598157ea0 2023-08-09T13:53:37.310 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:37.306+0000 7fd59da56640 1 -- 172.21.15.43:0/1354228081 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd5840094d0 con 0x7fd598157ea0 2023-08-09T13:53:37.310 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:37.306+0000 7fd59da56640 1 -- 172.21.15.43:0/1354228081 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd5981f5bc0 con 0x7fd598157ea0 2023-08-09T13:53:37.310 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:37.307+0000 7fd59da56640 1 -- 172.21.15.43:0/1354228081 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd59804e390 con 0x7fd598157ea0 2023-08-09T13:53:37.311 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:37.308+0000 7fd5957fa640 1 -- 172.21.15.43:0/1354228081 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 56) v1 ==== 88443+0+0 (secure 0 0 0) 0x7fd58402f050 con 0x7fd598157ea0 2023-08-09T13:53:37.311 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:37.309+0000 7fd5957fa640 1 --2- 172.21.15.43:0/1354228081 >> [v2:172.21.15.203:6832/2527059680,v1:172.21.15.203:6833/2527059680] conn(0x7fd57c06a170 0x7fd57c06c630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:53:37.311 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:37.309+0000 7fd5957fa640 1 -- 172.21.15.43:0/1354228081 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(121..121 src has 1..121) v4 ==== 6434+0+0 (secure 0 0 0) 0x7fd5840aa850 con 0x7fd598157ea0 2023-08-09T13:53:37.311 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:37.310+0000 7fd59ca54640 1 --2- 172.21.15.43:0/1354228081 >> [v2:172.21.15.203:6832/2527059680,v1:172.21.15.203:6833/2527059680] conn(0x7fd57c06a170 0x7fd57c06c630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:53:37.312 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:37.311+0000 7fd59ca54640 1 --2- 172.21.15.43:0/1354228081 >> [v2:172.21.15.203:6832/2527059680,v1:172.21.15.203:6833/2527059680] conn(0x7fd57c06a170 0x7fd57c06c630 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7fd58c009d20 tx=0x7fd58c008040 comp rx=0 tx=0).ready entity=mgr.6377 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-09T13:53:37.315 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:37.314+0000 7fd5957fa640 1 -- 172.21.15.43:0/1354228081 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178066 (secure 0 0 0) 0x7fd584075b00 con 0x7fd598157ea0 2023-08-09T13:53:37.508 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:37.507+0000 7fd59da56640 1 -- 172.21.15.43:0/1354228081 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "config rm", "who": "mgr", "name": "mgr/snap_schedule/allow_m_granularity"} v 0) v1 -- 0x7fd5980712b0 con 0x7fd598157ea0 2023-08-09T13:53:37.515 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:37.513+0000 7fd5957fa640 1 -- 172.21.15.43:0/1354228081 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "config rm", "who": "mgr", "name": "mgr/snap_schedule/allow_m_granularity"}]=0 v38) v1 ==== 120+0+0 (secure 0 0 0) 0x7fd584079120 con 0x7fd598157ea0 2023-08-09T13:53:37.517 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:37.516+0000 7fd59da56640 1 -- 172.21.15.43:0/1354228081 >> [v2:172.21.15.203:6832/2527059680,v1:172.21.15.203:6833/2527059680] conn(0x7fd57c06a170 msgr2=0x7fd57c06c630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:53:37.518 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:37.516+0000 7fd59da56640 1 --2- 172.21.15.43:0/1354228081 >> [v2:172.21.15.203:6832/2527059680,v1:172.21.15.203:6833/2527059680] conn(0x7fd57c06a170 0x7fd57c06c630 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7fd58c009d20 tx=0x7fd58c008040 comp rx=0 tx=0).stop 2023-08-09T13:53:37.518 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:37.516+0000 7fd59da56640 1 -- 172.21.15.43:0/1354228081 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fd598157ea0 msgr2=0x7fd5981f23c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:53:37.518 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:37.516+0000 7fd59da56640 1 --2- 172.21.15.43:0/1354228081 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fd598157ea0 0x7fd5981f23c0 secure :-1 s=READY pgs=1462 cs=0 l=1 rev1=1 crypto rx=0x7fd584000c00 tx=0x7fd5840046c0 comp rx=0 tx=0).stop 2023-08-09T13:53:37.519 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:37.516+0000 7fd59da56640 1 -- 172.21.15.43:0/1354228081 shutdown_connections 2023-08-09T13:53:37.519 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:37.516+0000 7fd59da56640 1 --2- 172.21.15.43:0/1354228081 >> [v2:172.21.15.203:6832/2527059680,v1:172.21.15.203:6833/2527059680] conn(0x7fd57c06a170 0x7fd57c06c630 unknown :-1 s=CLOSED pgs=36 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:53:37.519 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:37.516+0000 7fd59da56640 1 --2- 172.21.15.43:0/1354228081 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fd59815a800 0x7fd5981f4df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:53:37.519 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:37.516+0000 7fd59da56640 1 --2- 172.21.15.43:0/1354228081 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fd598157ea0 0x7fd5981f23c0 unknown :-1 s=CLOSED pgs=1462 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:53:37.519 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:37.516+0000 7fd59da56640 1 --2- 172.21.15.43:0/1354228081 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fd5980c90a0 0x7fd5980c6150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:53:37.519 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:37.516+0000 7fd59da56640 1 -- 172.21.15.43:0/1354228081 >> 172.21.15.43:0/1354228081 conn(0x7fd5980cddd0 msgr2=0x7fd59815ac70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-09T13:53:37.519 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:37.517+0000 7fd59da56640 1 -- 172.21.15.43:0/1354228081 shutdown_connections 2023-08-09T13:53:37.520 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:37.517+0000 7fd59da56640 1 -- 172.21.15.43:0/1354228081 wait complete. 2023-08-09T13:53:37.543 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config rm mon mon_allow_pool_delete 2023-08-09T13:53:37.755 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:37.754+0000 7f93cb021640 1 Processor -- start 2023-08-09T13:53:37.756 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:37.754+0000 7f93cb021640 1 -- start start 2023-08-09T13:53:37.757 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:37.756+0000 7f93cb021640 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7f93c415c620 0x7f93c40cceb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:53:37.757 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:37.756+0000 7f93cb021640 1 -- --> v1:172.21.15.43:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f93c40cd700 con 0x7f93c40b1260 2023-08-09T13:53:37.757 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:37.756+0000 7f93cb021640 1 -- --> v1:172.21.15.203:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f93c40cd9e0 con 0x7f93c40cf510 2023-08-09T13:53:37.758 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:37.756+0000 7f93cb021640 1 -- --> v2:172.21.15.203:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f93c40b1b70 con 0x7f93c415c620 2023-08-09T13:53:37.759 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:37.756+0000 7f93c3fff640 1 --1- >> v1:172.21.15.43:6789/0 conn(0x7f93c40b1260 0x7f93c40cee00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.43:6789/0 says I am v1:172.21.15.43:44750/0 (socket says 172.21.15.43:44750) 2023-08-09T13:53:37.759 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:37.756+0000 7f93c3fff640 1 -- 172.21.15.43:0/264030138 learned_addr learned my addr 172.21.15.43:0/264030138 (peer_addr_for_me v1:172.21.15.43:0/0) 2023-08-09T13:53:37.759 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:37.757+0000 7f93c8dbd640 1 --2- 172.21.15.43:0/264030138 >> v2:172.21.15.203:3300/0 conn(0x7f93c415c620 0x7f93c40cceb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:53:37.759 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:37.757+0000 7f93c37fe640 1 -- 172.21.15.43:0/264030138 <== mon.0 v1:172.21.15.43:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3668323411 0 0) 0x7f93c40cd700 con 0x7f93c40b1260 2023-08-09T13:53:37.759 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:37.757+0000 7f93c37fe640 1 -- 172.21.15.43:0/264030138 --> v1:172.21.15.43:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f93ac003520 con 0x7f93c40b1260 2023-08-09T13:53:37.759 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:37.757+0000 7f93c37fe640 1 -- 172.21.15.43:0/264030138 <== mon.0 v1:172.21.15.43:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1137416781 0 0) 0x7f93ac003520 con 0x7f93c40b1260 2023-08-09T13:53:37.759 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:37.757+0000 7f93c37fe640 1 -- 172.21.15.43:0/264030138 >> v2:172.21.15.203:3300/0 conn(0x7f93c415c620 msgr2=0x7f93c40cceb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-09T13:53:37.760 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:37.757+0000 7f93c37fe640 1 --2- 172.21.15.43:0/264030138 >> v2:172.21.15.203:3300/0 conn(0x7f93c415c620 0x7f93c40cceb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:53:37.760 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:37.757+0000 7f93c37fe640 1 -- 172.21.15.43:0/264030138 >> v1:172.21.15.203:6789/0 conn(0x7f93c40cf510 legacy=0x7f93c40d0350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:53:37.760 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:37.757+0000 7f93c37fe640 1 -- 172.21.15.43:0/264030138 --> v1:172.21.15.43:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f93c40cdcc0 con 0x7f93c40b1260 2023-08-09T13:53:37.760 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:37.757+0000 7f93c37fe640 1 -- 172.21.15.43:0/264030138 <== mon.0 v1:172.21.15.43:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1285150740 0 0) 0x7f93b8001670 con 0x7f93c40b1260 2023-08-09T13:53:37.760 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:37.757+0000 7f93c37fe640 1 -- 172.21.15.43:0/264030138 >> v1:172.21.15.43:6789/0 conn(0x7f93c40b1260 legacy=0x7f93c40cee00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:53:37.760 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:37.757+0000 7f93c37fe640 1 --2- 172.21.15.43:0/264030138 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f93ac003970 0x7f93ac003d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:53:37.760 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:37.757+0000 7f93c37fe640 1 --2- 172.21.15.43:0/264030138 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f93ac0047c0 0x7f93ac004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:53:37.761 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:37.757+0000 7f93c37fe640 1 --2- 172.21.15.43:0/264030138 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f93ac0052a0 0x7f93ac00b980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:53:37.761 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:37.757+0000 7f93c37fe640 1 -- 172.21.15.43:0/264030138 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f93ac00c140 con 0x7f93ac0052a0 2023-08-09T13:53:37.761 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:37.757+0000 7f93c37fe640 1 -- 172.21.15.43:0/264030138 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f93ac00c2b0 con 0x7f93ac003970 2023-08-09T13:53:37.761 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:37.757+0000 7f93c37fe640 1 -- 172.21.15.43:0/264030138 --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f93ac00c470 con 0x7f93ac0047c0 2023-08-09T13:53:37.762 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:37.758+0000 7f93c95be640 1 --2- 172.21.15.43:0/264030138 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f93ac0052a0 0x7f93ac00b980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:53:37.762 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:37.758+0000 7f93c3fff640 1 --2- 172.21.15.43:0/264030138 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f93ac0047c0 0x7f93ac004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:53:37.763 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:37.758+0000 7f93c8dbd640 1 --2- 172.21.15.43:0/264030138 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f93ac003970 0x7f93ac003d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:53:37.763 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:37.758+0000 7f93c95be640 1 -- 172.21.15.43:0/264030138 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f93ac0047c0 msgr2=0x7f93ac004c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-09T13:53:37.763 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:37.758+0000 7f93c95be640 1 --2- 172.21.15.43:0/264030138 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f93ac0047c0 0x7f93ac004c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:53:37.763 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:37.758+0000 7f93c95be640 1 -- 172.21.15.43:0/264030138 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f93ac003970 msgr2=0x7f93ac003d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-09T13:53:37.763 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:37.758+0000 7f93c95be640 1 --2- 172.21.15.43:0/264030138 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f93ac003970 0x7f93ac003d90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:53:37.763 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:37.758+0000 7f93c95be640 1 -- 172.21.15.43:0/264030138 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f93ac003520 con 0x7f93ac0052a0 2023-08-09T13:53:37.763 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:37.759+0000 7f93c95be640 1 -- 172.21.15.43:0/264030138 reap_dead start 2023-08-09T13:53:37.764 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:37.759+0000 7f93c95be640 1 --2- 172.21.15.43:0/264030138 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f93ac0052a0 0x7f93ac00b980 secure :-1 s=READY pgs=1464 cs=0 l=1 rev1=1 crypto rx=0x7f93bc001720 tx=0x7f93ac0057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=43a8a54a5b55f163 server_cookie=0 in_seq=0 out_seq=0 2023-08-09T13:53:37.764 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:37.759+0000 7f93c37fe640 1 -- 172.21.15.43:0/264030138 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f93bc0052d0 con 0x7f93ac0052a0 2023-08-09T13:53:37.764 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:37.760+0000 7f93c37fe640 1 -- 172.21.15.43:0/264030138 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f93bc02e050 con 0x7f93ac0052a0 2023-08-09T13:53:37.764 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:37.760+0000 7f93c37fe640 1 -- 172.21.15.43:0/264030138 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f93bc0035f0 con 0x7f93ac0052a0 2023-08-09T13:53:37.764 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:37.760+0000 7f93cb021640 1 -- 172.21.15.43:0/264030138 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f93ac0052a0 msgr2=0x7f93ac00b980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:53:37.764 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:37.760+0000 7f93cb021640 1 --2- 172.21.15.43:0/264030138 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f93ac0052a0 0x7f93ac00b980 secure :-1 s=READY pgs=1464 cs=0 l=1 rev1=1 crypto rx=0x7f93bc001720 tx=0x7f93ac0057f0 comp rx=0 tx=0).stop 2023-08-09T13:53:37.764 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:37.760+0000 7f93cb021640 1 -- 172.21.15.43:0/264030138 shutdown_connections 2023-08-09T13:53:37.765 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:37.760+0000 7f93cb021640 1 --2- 172.21.15.43:0/264030138 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f93ac0052a0 0x7f93ac00b980 unknown :-1 s=CLOSED pgs=1464 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:53:37.765 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:37.760+0000 7f93cb021640 1 -- 172.21.15.43:0/264030138 >> 172.21.15.43:0/264030138 conn(0x7f93c409b750 msgr2=0x7f93c40d2410 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-09T13:53:37.765 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:37.760+0000 7f93cb021640 1 -- 172.21.15.43:0/264030138 shutdown_connections 2023-08-09T13:53:37.765 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:37.760+0000 7f93cb021640 1 -- 172.21.15.43:0/264030138 wait complete. 2023-08-09T13:53:37.766 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:37.762+0000 7f93cb021640 1 Processor -- start 2023-08-09T13:53:37.767 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:37.763+0000 7f93cb021640 1 -- start start 2023-08-09T13:53:37.767 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:37.763+0000 7f93cb021640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f93ac0052a0 0x7f93c41554b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:53:37.767 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:37.763+0000 7f93cb021640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f93c41559f0 0x7f93c41571b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:53:37.767 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:37.763+0000 7f93cb021640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f93c40b67d0 0x7f93c41576f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:53:37.767 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:37.763+0000 7f93cb021640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f93c40b7960 con 0x7f93ac0052a0 2023-08-09T13:53:37.767 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:37.763+0000 7f93cb021640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f93c4155210 con 0x7f93c41559f0 2023-08-09T13:53:37.767 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:37.763+0000 7f93cb021640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f93c40b6ce0 con 0x7f93c40b67d0 2023-08-09T13:53:37.768 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:37.764+0000 7f93c8dbd640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f93ac0052a0 0x7f93c41554b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:53:37.768 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:37.764+0000 7f93c8dbd640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f93ac0052a0 0x7f93c41554b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:53398/0 (socket says 172.21.15.43:53398) 2023-08-09T13:53:37.768 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:37.764+0000 7f93c8dbd640 1 -- 172.21.15.43:0/2059559558 learned_addr learned my addr 172.21.15.43:0/2059559558 (peer_addr_for_me v2:172.21.15.43:0/0) 2023-08-09T13:53:37.768 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:37.764+0000 7f93c3fff640 1 --2- 172.21.15.43:0/2059559558 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f93c41559f0 0x7f93c41571b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:53:37.768 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:37.764+0000 7f93c95be640 1 --2- 172.21.15.43:0/2059559558 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f93c40b67d0 0x7f93c41576f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:53:37.768 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:37.764+0000 7f93c8dbd640 1 -- 172.21.15.43:0/2059559558 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f93c40b67d0 msgr2=0x7f93c41576f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:53:37.768 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:37.764+0000 7f93c8dbd640 1 --2- 172.21.15.43:0/2059559558 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f93c40b67d0 0x7f93c41576f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:53:37.769 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:37.764+0000 7f93c8dbd640 1 -- 172.21.15.43:0/2059559558 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f93c41559f0 msgr2=0x7f93c41571b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:53:37.769 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:37.764+0000 7f93c8dbd640 1 --2- 172.21.15.43:0/2059559558 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f93c41559f0 0x7f93c41571b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:53:37.769 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:37.764+0000 7f93c8dbd640 1 -- 172.21.15.43:0/2059559558 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f93c40b6fc0 con 0x7f93ac0052a0 2023-08-09T13:53:37.769 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:37.764+0000 7f93c3fff640 1 --2- 172.21.15.43:0/2059559558 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f93c41559f0 0x7f93c41571b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-08-09T13:53:37.769 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:37.765+0000 7f93c8dbd640 1 --2- 172.21.15.43:0/2059559558 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f93ac0052a0 0x7f93c41554b0 secure :-1 s=READY pgs=1465 cs=0 l=1 rev1=1 crypto rx=0x7f93b400f380 tx=0x7f93b400f840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-09T13:53:37.769 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:37.765+0000 7f93c17fa640 1 -- 172.21.15.43:0/2059559558 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f93b4017600 con 0x7f93ac0052a0 2023-08-09T13:53:37.769 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:37.765+0000 7f93c17fa640 1 -- 172.21.15.43:0/2059559558 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f93b4011040 con 0x7f93ac0052a0 2023-08-09T13:53:37.769 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:37.765+0000 7f93c17fa640 1 -- 172.21.15.43:0/2059559558 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f93b40179c0 con 0x7f93ac0052a0 2023-08-09T13:53:37.770 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:37.765+0000 7f93cb021640 1 -- 172.21.15.43:0/2059559558 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f93c40b72a0 con 0x7f93ac0052a0 2023-08-09T13:53:37.770 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:37.765+0000 7f93cb021640 1 -- 172.21.15.43:0/2059559558 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f93c4157c30 con 0x7f93ac0052a0 2023-08-09T13:53:37.770 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:37.766+0000 7f93cb021640 1 -- 172.21.15.43:0/2059559558 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f93c40739f0 con 0x7f93ac0052a0 2023-08-09T13:53:37.771 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:37.767+0000 7f93c17fa640 1 -- 172.21.15.43:0/2059559558 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 56) v1 ==== 88443+0+0 (secure 0 0 0) 0x7f93b4017b20 con 0x7f93ac0052a0 2023-08-09T13:53:37.771 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:37.768+0000 7f93c17fa640 1 --2- 172.21.15.43:0/2059559558 >> [v2:172.21.15.203:6832/2527059680,v1:172.21.15.203:6833/2527059680] conn(0x7f93b006a1a0 0x7f93b006c660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:53:37.771 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:37.769+0000 7f93c17fa640 1 -- 172.21.15.43:0/2059559558 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(121..121 src has 1..121) v4 ==== 6434+0+0 (secure 0 0 0) 0x7f93b4088d80 con 0x7f93ac0052a0 2023-08-09T13:53:37.771 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:37.769+0000 7f93c3fff640 1 --2- 172.21.15.43:0/2059559558 >> [v2:172.21.15.203:6832/2527059680,v1:172.21.15.203:6833/2527059680] conn(0x7f93b006a1a0 0x7f93b006c660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:53:37.771 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:37.770+0000 7f93c3fff640 1 --2- 172.21.15.43:0/2059559558 >> [v2:172.21.15.203:6832/2527059680,v1:172.21.15.203:6833/2527059680] conn(0x7f93b006a1a0 0x7f93b006c660 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7f93b8001200 tx=0x7f93b8000f70 comp rx=0 tx=0).ready entity=mgr.6377 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-09T13:53:37.774 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:37.773+0000 7f93c17fa640 1 -- 172.21.15.43:0/2059559558 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178066 (secure 0 0 0) 0x7f93b40541a0 con 0x7f93ac0052a0 2023-08-09T13:53:37.971 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:37.970+0000 7f93cb021640 1 -- 172.21.15.43:0/2059559558 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"} v 0) v1 -- 0x7f93c40cfda0 con 0x7f93ac0052a0 2023-08-09T13:53:37.979 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:37.978+0000 7f93c17fa640 1 -- 172.21.15.43:0/2059559558 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"}]=0 v39) v1 ==== 104+0+0 (secure 0 0 0) 0x7f93b40577c0 con 0x7f93ac0052a0 2023-08-09T13:53:37.982 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:37.980+0000 7f93cb021640 1 -- 172.21.15.43:0/2059559558 >> [v2:172.21.15.203:6832/2527059680,v1:172.21.15.203:6833/2527059680] conn(0x7f93b006a1a0 msgr2=0x7f93b006c660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:53:37.982 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:37.980+0000 7f93cb021640 1 --2- 172.21.15.43:0/2059559558 >> [v2:172.21.15.203:6832/2527059680,v1:172.21.15.203:6833/2527059680] conn(0x7f93b006a1a0 0x7f93b006c660 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7f93b8001200 tx=0x7f93b8000f70 comp rx=0 tx=0).stop 2023-08-09T13:53:37.982 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:37.980+0000 7f93cb021640 1 -- 172.21.15.43:0/2059559558 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f93ac0052a0 msgr2=0x7f93c41554b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:53:37.983 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:37.980+0000 7f93cb021640 1 --2- 172.21.15.43:0/2059559558 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f93ac0052a0 0x7f93c41554b0 secure :-1 s=READY pgs=1465 cs=0 l=1 rev1=1 crypto rx=0x7f93b400f380 tx=0x7f93b400f840 comp rx=0 tx=0).stop 2023-08-09T13:53:37.983 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:37.981+0000 7f93cb021640 1 -- 172.21.15.43:0/2059559558 shutdown_connections 2023-08-09T13:53:37.983 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:37.981+0000 7f93cb021640 1 --2- 172.21.15.43:0/2059559558 >> [v2:172.21.15.203:6832/2527059680,v1:172.21.15.203:6833/2527059680] conn(0x7f93b006a1a0 0x7f93b006c660 unknown :-1 s=CLOSED pgs=37 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:53:37.983 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:37.981+0000 7f93cb021640 1 --2- 172.21.15.43:0/2059559558 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f93c40b67d0 0x7f93c41576f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:53:37.983 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:37.981+0000 7f93cb021640 1 --2- 172.21.15.43:0/2059559558 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f93c41559f0 0x7f93c41571b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:53:37.983 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:37.981+0000 7f93cb021640 1 --2- 172.21.15.43:0/2059559558 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f93ac0052a0 0x7f93c41554b0 unknown :-1 s=CLOSED pgs=1465 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:53:37.983 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:37.981+0000 7f93cb021640 1 -- 172.21.15.43:0/2059559558 >> 172.21.15.43:0/2059559558 conn(0x7f93c409b750 msgr2=0x7f93c406a580 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-09T13:53:37.984 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:37.981+0000 7f93cb021640 1 -- 172.21.15.43:0/2059559558 shutdown_connections 2023-08-09T13:53:37.984 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:37.981+0000 7f93cb021640 1 -- 172.21.15.43:0/2059559558 wait complete. 2023-08-09T13:53:38.000 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Ended test tasks.cephfs.test_snap_schedules.TestSnapSchedules.test_schedule_auto_deactivation_for_non_existent_path' 2023-08-09T13:53:38.233 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:38.232+0000 7f8211acc640 1 Processor -- start 2023-08-09T13:53:38.234 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:38.232+0000 7f8211acc640 1 -- start start 2023-08-09T13:53:38.235 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:38.234+0000 7f8211acc640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f820c0c86f0 0x7f820c0c8b50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:53:38.236 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:38.234+0000 7f8211acc640 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7f820c06c8d0 0x7f820c06cd00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:53:38.236 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:38.234+0000 7f8211acc640 1 -- --> v1:172.21.15.43:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f820c0c9090 con 0x7f820c143c20 2023-08-09T13:53:38.236 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:38.234+0000 7f8211acc640 1 -- --> v2:172.21.15.43:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f820c144000 con 0x7f820c0c86f0 2023-08-09T13:53:38.236 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:38.234+0000 7f8211acc640 1 -- --> v2:172.21.15.203:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f820c06c440 con 0x7f820c06c8d0 2023-08-09T13:53:38.236 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:38.234+0000 7f820b7fe640 1 --1- >> v1:172.21.15.43:6789/0 conn(0x7f820c143c20 0x7f820c06f540 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.43:6789/0 says I am v1:172.21.15.43:44762/0 (socket says 172.21.15.43:44762) 2023-08-09T13:53:38.236 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:38.234+0000 7f820b7fe640 1 -- 172.21.15.43:0/1950873326 learned_addr learned my addr 172.21.15.43:0/1950873326 (peer_addr_for_me v1:172.21.15.43:0/0) 2023-08-09T13:53:38.237 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:38.235+0000 7f8203fff640 1 --2- 172.21.15.43:0/1950873326 >> v2:172.21.15.43:3300/0 conn(0x7f820c0c86f0 0x7f820c0c8b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:53:38.237 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:38.235+0000 7f820bfff640 1 --2- 172.21.15.43:0/1950873326 >> v2:172.21.15.203:3300/0 conn(0x7f820c06c8d0 0x7f820c06cd00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:53:38.238 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:38.235+0000 7f820affd640 1 -- 172.21.15.43:0/1950873326 <== mon.0 v1:172.21.15.43:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2249863025 0 0) 0x7f820c0c9090 con 0x7f820c143c20 2023-08-09T13:53:38.238 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:38.235+0000 7f820affd640 1 -- 172.21.15.43:0/1950873326 --> v1:172.21.15.43:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f81f8003520 con 0x7f820c143c20 2023-08-09T13:53:38.238 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:38.235+0000 7f8203fff640 1 -- 172.21.15.43:0/1950873326 >> v2:172.21.15.203:3300/0 conn(0x7f820c06c8d0 msgr2=0x7f820c06cd00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-09T13:53:38.238 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:38.235+0000 7f8203fff640 1 --2- 172.21.15.43:0/1950873326 >> v2:172.21.15.203:3300/0 conn(0x7f820c06c8d0 0x7f820c06cd00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:53:38.238 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:38.235+0000 7f8203fff640 1 -- 172.21.15.43:0/1950873326 >> v1:172.21.15.43:6789/0 conn(0x7f820c143c20 legacy=0x7f820c06f540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:53:38.238 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:38.235+0000 7f8203fff640 1 -- 172.21.15.43:0/1950873326 --> v2:172.21.15.43:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f820c06fc50 con 0x7f820c0c86f0 2023-08-09T13:53:38.238 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:38.235+0000 7f820bfff640 1 --2- 172.21.15.43:0/1950873326 >> v2:172.21.15.203:3300/0 conn(0x7f820c06c8d0 0x7f820c06cd00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-08-09T13:53:38.239 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:38.236+0000 7f8203fff640 1 --2- 172.21.15.43:0/1950873326 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f820c0c86f0 0x7f820c0c8b50 secure :-1 s=READY pgs=1467 cs=0 l=1 rev1=1 crypto rx=0x7f820c0caad0 tx=0x7f81f402b840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fc4098d8f8e70e9c server_cookie=0 in_seq=0 out_seq=0 2023-08-09T13:53:38.239 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:38.237+0000 7f820affd640 1 -- 172.21.15.43:0/1950873326 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f81f402e070 con 0x7f820c0c86f0 2023-08-09T13:53:38.240 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:38.237+0000 7f820affd640 1 -- 172.21.15.43:0/1950873326 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f81f40315a0 con 0x7f820c0c86f0 2023-08-09T13:53:38.240 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:38.237+0000 7f820affd640 1 -- 172.21.15.43:0/1950873326 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f81f4031930 con 0x7f820c0c86f0 2023-08-09T13:53:38.240 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:38.237+0000 7f8211acc640 1 -- 172.21.15.43:0/1950873326 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f820c0c86f0 msgr2=0x7f820c0c8b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:53:38.240 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:38.237+0000 7f8211acc640 1 --2- 172.21.15.43:0/1950873326 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f820c0c86f0 0x7f820c0c8b50 secure :-1 s=READY pgs=1467 cs=0 l=1 rev1=1 crypto rx=0x7f820c0caad0 tx=0x7f81f402b840 comp rx=0 tx=0).stop 2023-08-09T13:53:38.240 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:38.238+0000 7f8211acc640 1 -- 172.21.15.43:0/1950873326 shutdown_connections 2023-08-09T13:53:38.240 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:38.238+0000 7f8211acc640 1 --2- 172.21.15.43:0/1950873326 >> v2:172.21.15.203:3300/0 conn(0x7f820c06c8d0 0x7f820c06cd00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:53:38.240 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:38.238+0000 7f8211acc640 1 --2- 172.21.15.43:0/1950873326 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f820c0c86f0 0x7f820c0c8b50 unknown :-1 s=CLOSED pgs=1467 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:53:38.241 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:38.238+0000 7f8211acc640 1 -- 172.21.15.43:0/1950873326 >> 172.21.15.43:0/1950873326 conn(0x7f820c0ce0a0 msgr2=0x7f820c0ce480 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-09T13:53:38.241 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:38.238+0000 7f8211acc640 1 -- 172.21.15.43:0/1950873326 shutdown_connections 2023-08-09T13:53:38.241 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:38.238+0000 7f8211acc640 1 -- 172.21.15.43:0/1950873326 wait complete. 2023-08-09T13:53:38.241 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:38.239+0000 7f8211acc640 1 Processor -- start 2023-08-09T13:53:38.241 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:38.240+0000 7f8211acc640 1 -- start start 2023-08-09T13:53:38.242 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:38.241+0000 7f8211acc640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f820c06c8d0 0x7f820c0b1940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:53:38.242 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:38.241+0000 7f8211acc640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f820c0c86f0 0x7f820c13cbb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:53:38.242 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:38.241+0000 7f8211acc640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f820c143c20 0x7f820c0c64b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:53:38.243 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:38.241+0000 7f8211acc640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f820c13d0f0 con 0x7f820c0c86f0 2023-08-09T13:53:38.243 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:38.241+0000 7f8211acc640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f820c0c71c0 con 0x7f820c06c8d0 2023-08-09T13:53:38.243 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:38.241+0000 7f8211acc640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f820c0b1f20 con 0x7f820c143c20 2023-08-09T13:53:38.243 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:38.241+0000 7f8203fff640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f820c0c86f0 0x7f820c13cbb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:53:38.243 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:38.241+0000 7f8203fff640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f820c0c86f0 0x7f820c13cbb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:53424/0 (socket says 172.21.15.43:53424) 2023-08-09T13:53:38.243 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:38.241+0000 7f8203fff640 1 -- 172.21.15.43:0/831628731 learned_addr learned my addr 172.21.15.43:0/831628731 (peer_addr_for_me v2:172.21.15.43:0/0) 2023-08-09T13:53:38.244 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:38.241+0000 7f820b7fe640 1 --2- 172.21.15.43:0/831628731 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f820c06c8d0 0x7f820c0b1940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:53:38.244 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:38.241+0000 7f820bfff640 1 --2- 172.21.15.43:0/831628731 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f820c143c20 0x7f820c0c64b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:53:38.245 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:38.241+0000 7f8203fff640 1 -- 172.21.15.43:0/831628731 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f820c143c20 msgr2=0x7f820c0c64b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:53:38.245 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:38.241+0000 7f8203fff640 1 --2- 172.21.15.43:0/831628731 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f820c143c20 0x7f820c0c64b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:53:38.245 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:38.241+0000 7f8203fff640 1 -- 172.21.15.43:0/831628731 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f820c06c8d0 msgr2=0x7f820c0b1940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:53:38.245 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:38.242+0000 7f8203fff640 1 --2- 172.21.15.43:0/831628731 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f820c06c8d0 0x7f820c0b1940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:53:38.245 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:38.242+0000 7f8203fff640 1 -- 172.21.15.43:0/831628731 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f81f4009400 con 0x7f820c0c86f0 2023-08-09T13:53:38.245 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:38.242+0000 7f8203fff640 1 --2- 172.21.15.43:0/831628731 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f820c0c86f0 0x7f820c13cbb0 secure :-1 s=READY pgs=1468 cs=0 l=1 rev1=1 crypto rx=0x7f81f4009890 tx=0x7f81f40046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-09T13:53:38.245 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:38.242+0000 7f8208ff9640 1 -- 172.21.15.43:0/831628731 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f81f4032840 con 0x7f820c0c86f0 2023-08-09T13:53:38.246 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:38.242+0000 7f8208ff9640 1 -- 172.21.15.43:0/831628731 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f81f4031f00 con 0x7f820c0c86f0 2023-08-09T13:53:38.246 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:38.242+0000 7f8208ff9640 1 -- 172.21.15.43:0/831628731 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f81f4032be0 con 0x7f820c0c86f0 2023-08-09T13:53:38.246 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:38.242+0000 7f8211acc640 1 -- 172.21.15.43:0/831628731 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f820c0b2200 con 0x7f820c0c86f0 2023-08-09T13:53:38.246 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:38.242+0000 7f8211acc640 1 -- 172.21.15.43:0/831628731 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f820c0c69f0 con 0x7f820c0c86f0 2023-08-09T13:53:38.246 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:38.244+0000 7f8211acc640 1 -- 172.21.15.43:0/831628731 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f820c0c6ca0 con 0x7f820c0c86f0 2023-08-09T13:53:38.247 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:38.244+0000 7f8208ff9640 1 -- 172.21.15.43:0/831628731 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 56) v1 ==== 88443+0+0 (secure 0 0 0) 0x7f81f4032d40 con 0x7f820c0c86f0 2023-08-09T13:53:38.247 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:38.245+0000 7f8208ff9640 1 --2- 172.21.15.43:0/831628731 >> [v2:172.21.15.203:6832/2527059680,v1:172.21.15.203:6833/2527059680] conn(0x7f81ec06a170 0x7f81ec06c630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:53:38.247 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:38.246+0000 7f8208ff9640 1 -- 172.21.15.43:0/831628731 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(121..121 src has 1..121) v4 ==== 6434+0+0 (secure 0 0 0) 0x7f81f40aa8d0 con 0x7f820c0c86f0 2023-08-09T13:53:38.247 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:38.246+0000 7f820b7fe640 1 --2- 172.21.15.43:0/831628731 >> [v2:172.21.15.203:6832/2527059680,v1:172.21.15.203:6833/2527059680] conn(0x7f81ec06a170 0x7f81ec06c630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:53:38.248 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:38.247+0000 7f820b7fe640 1 --2- 172.21.15.43:0/831628731 >> [v2:172.21.15.203:6832/2527059680,v1:172.21.15.203:6833/2527059680] conn(0x7f81ec06a170 0x7f81ec06c630 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7f81fc0011e0 tx=0x7f81fc009040 comp rx=0 tx=0).ready entity=mgr.6377 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-09T13:53:38.251 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:38.250+0000 7f8208ff9640 1 -- 172.21.15.43:0/831628731 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178066 (secure 0 0 0) 0x7f81f4075b80 con 0x7f820c0c86f0 2023-08-09T13:53:38.515 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:38.514+0000 7f8211acc640 1 -- 172.21.15.43:0/831628731 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_snap_schedules.TestSnapSchedules.test_schedule_auto_deactivation_for_non_existent_path"]} v 0) v1 -- 0x7f820c0739f0 con 0x7f820c0c86f0 2023-08-09T13:53:38.600 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:38.599+0000 7f8208ff9640 1 -- 172.21.15.43:0/831628731 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_snap_schedules.TestSnapSchedules.test_schedule_auto_deactivation_for_non_existent_path"]}]=0 v767) v1 ==== 183+0+0 (secure 0 0 0) 0x7f81f40791a0 con 0x7f820c0c86f0 2023-08-09T13:53:38.604 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:38.602+0000 7f8211acc640 1 -- 172.21.15.43:0/831628731 >> [v2:172.21.15.203:6832/2527059680,v1:172.21.15.203:6833/2527059680] conn(0x7f81ec06a170 msgr2=0x7f81ec06c630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:53:38.604 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:38.602+0000 7f8211acc640 1 --2- 172.21.15.43:0/831628731 >> [v2:172.21.15.203:6832/2527059680,v1:172.21.15.203:6833/2527059680] conn(0x7f81ec06a170 0x7f81ec06c630 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7f81fc0011e0 tx=0x7f81fc009040 comp rx=0 tx=0).stop 2023-08-09T13:53:38.604 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:38.602+0000 7f8211acc640 1 -- 172.21.15.43:0/831628731 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f820c0c86f0 msgr2=0x7f820c13cbb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:53:38.604 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:38.602+0000 7f8211acc640 1 --2- 172.21.15.43:0/831628731 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f820c0c86f0 0x7f820c13cbb0 secure :-1 s=READY pgs=1468 cs=0 l=1 rev1=1 crypto rx=0x7f81f4009890 tx=0x7f81f40046c0 comp rx=0 tx=0).stop 2023-08-09T13:53:38.604 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:38.602+0000 7f8211acc640 1 -- 172.21.15.43:0/831628731 shutdown_connections 2023-08-09T13:53:38.604 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:38.602+0000 7f8211acc640 1 --2- 172.21.15.43:0/831628731 >> [v2:172.21.15.203:6832/2527059680,v1:172.21.15.203:6833/2527059680] conn(0x7f81ec06a170 0x7f81ec06c630 unknown :-1 s=CLOSED pgs=38 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:53:38.605 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:38.602+0000 7f8211acc640 1 --2- 172.21.15.43:0/831628731 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f820c143c20 0x7f820c0c64b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:53:38.605 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:38.602+0000 7f8211acc640 1 --2- 172.21.15.43:0/831628731 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f820c0c86f0 0x7f820c13cbb0 unknown :-1 s=CLOSED pgs=1468 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:53:38.605 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:38.602+0000 7f8211acc640 1 --2- 172.21.15.43:0/831628731 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f820c06c8d0 0x7f820c0b1940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:53:38.605 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:38.602+0000 7f8211acc640 1 -- 172.21.15.43:0/831628731 >> 172.21.15.43:0/831628731 conn(0x7f820c0ce0a0 msgr2=0x7f820c0c8af0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-09T13:53:38.605 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:38.602+0000 7f8211acc640 1 -- 172.21.15.43:0/831628731 shutdown_connections 2023-08-09T13:53:38.605 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:38.602+0000 7f8211acc640 1 -- 172.21.15.43:0/831628731 wait complete. 2023-08-09T13:53:38.625 INFO:tasks.cephfs_test_runner:test_schedule_auto_deactivation_for_non_existent_path (tasks.cephfs.test_snap_schedules.TestSnapSchedules) ... ok 2023-08-09T13:53:38.626 INFO:tasks.cephfs_test_runner:Starting test: test_snap_schedule (tasks.cephfs.test_snap_schedules.TestSnapSchedules) Test existence of a scheduled snapshot 2023-08-09T13:53:38.626 INFO:tasks.cephfs_test_runner:test_snap_schedule (tasks.cephfs.test_snap_schedules.TestSnapSchedules) 2023-08-09T13:53:38.627 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Starting test tasks.cephfs.test_snap_schedules.TestSnapSchedules.test_snap_schedule' 2023-08-09T13:53:38.862 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:38.861+0000 7f3548fab640 1 Processor -- start 2023-08-09T13:53:38.863 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:38.861+0000 7f3548fab640 1 -- start start 2023-08-09T13:53:38.865 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:38.862+0000 7f3548fab640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f354415a6a0 0x7f35440713b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:53:38.866 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:38.862+0000 7f3548fab640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f35440cbd90 0x7f35440c8fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:53:38.866 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:38.862+0000 7f3548fab640 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7f35440c94f0 0x7f354415ad20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:53:38.866 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:38.863+0000 7f3548fab640 1 -- --> v2:172.21.15.43:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f35440c9960 con 0x7f354415a6a0 2023-08-09T13:53:38.866 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:38.863+0000 7f3548fab640 1 -- --> v2:172.21.15.203:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f35440cdc70 con 0x7f35440c94f0 2023-08-09T13:53:38.867 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:38.863+0000 7f3548fab640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3544070910 con 0x7f35440cbd90 2023-08-09T13:53:38.867 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:38.863+0000 7f35437fe640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f354415a6a0 0x7f35440713b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:53:38.867 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:38.863+0000 7f35437fe640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f354415a6a0 0x7f35440713b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:53440/0 (socket says 172.21.15.43:53440) 2023-08-09T13:53:38.868 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:38.863+0000 7f35437fe640 1 -- 172.21.15.43:0/3826871595 learned_addr learned my addr 172.21.15.43:0/3826871595 (peer_addr_for_me v2:172.21.15.43:0/0) 2023-08-09T13:53:38.868 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:38.863+0000 7f3542ffd640 1 --2- 172.21.15.43:0/3826871595 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f35440cbd90 0x7f35440c8fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:53:38.868 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:38.863+0000 7f3543fff640 1 --2- 172.21.15.43:0/3826871595 >> v2:172.21.15.203:3300/0 conn(0x7f35440c94f0 0x7f354415ad20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:53:38.868 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:38.863+0000 7f35437fe640 1 -- 172.21.15.43:0/3826871595 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f35440cbd90 msgr2=0x7f35440c8fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-09T13:53:38.869 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:38.863+0000 7f35437fe640 1 --2- 172.21.15.43:0/3826871595 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f35440cbd90 0x7f35440c8fb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:53:38.869 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:38.863+0000 7f35437fe640 1 -- 172.21.15.43:0/3826871595 >> v2:172.21.15.203:3300/0 conn(0x7f35440c94f0 msgr2=0x7f354415ad20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-09T13:53:38.869 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:38.863+0000 7f35437fe640 1 --2- 172.21.15.43:0/3826871595 >> v2:172.21.15.203:3300/0 conn(0x7f35440c94f0 0x7f354415ad20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:53:38.869 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:38.863+0000 7f35437fe640 1 -- 172.21.15.43:0/3826871595 --> v2:172.21.15.43:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f35440c88c0 con 0x7f354415a6a0 2023-08-09T13:53:38.869 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:38.863+0000 7f3543fff640 1 --2- 172.21.15.43:0/3826871595 >> v2:172.21.15.203:3300/0 conn(0x7f35440c94f0 0x7f354415ad20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-08-09T13:53:38.871 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:38.865+0000 7f35437fe640 1 --2- 172.21.15.43:0/3826871595 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f354415a6a0 0x7f35440713b0 secure :-1 s=READY pgs=1469 cs=0 l=1 rev1=1 crypto rx=0x7f354415dec0 tx=0x7f353402b870 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6c58f567d74cdf72 server_cookie=0 in_seq=0 out_seq=0 2023-08-09T13:53:38.871 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:38.865+0000 7f35427fc640 1 -- 172.21.15.43:0/3826871595 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f35340317d0 con 0x7f354415a6a0 2023-08-09T13:53:38.871 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:38.865+0000 7f35427fc640 1 -- 172.21.15.43:0/3826871595 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3534031930 con 0x7f354415a6a0 2023-08-09T13:53:38.871 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:38.865+0000 7f35427fc640 1 -- 172.21.15.43:0/3826871595 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3534031cc0 con 0x7f354415a6a0 2023-08-09T13:53:38.871 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:38.866+0000 7f3548fab640 1 -- 172.21.15.43:0/3826871595 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f354415a6a0 msgr2=0x7f35440713b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:53:38.871 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:38.866+0000 7f3548fab640 1 --2- 172.21.15.43:0/3826871595 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f354415a6a0 0x7f35440713b0 secure :-1 s=READY pgs=1469 cs=0 l=1 rev1=1 crypto rx=0x7f354415dec0 tx=0x7f353402b870 comp rx=0 tx=0).stop 2023-08-09T13:53:38.872 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:38.866+0000 7f3548fab640 1 -- 172.21.15.43:0/3826871595 shutdown_connections 2023-08-09T13:53:38.872 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:38.866+0000 7f3548fab640 1 --2- 172.21.15.43:0/3826871595 >> v2:172.21.15.203:3300/0 conn(0x7f35440c94f0 0x7f354415ad20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:53:38.872 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:38.866+0000 7f3548fab640 1 --2- 172.21.15.43:0/3826871595 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f35440cbd90 0x7f35440c8fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:53:38.872 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:38.866+0000 7f3548fab640 1 --2- 172.21.15.43:0/3826871595 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f354415a6a0 0x7f35440713b0 unknown :-1 s=CLOSED pgs=1469 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:53:38.872 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:38.866+0000 7f3548fab640 1 -- 172.21.15.43:0/3826871595 >> 172.21.15.43:0/3826871595 conn(0x7f35440cde90 msgr2=0x7f35440ce2a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-09T13:53:38.872 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:38.866+0000 7f3548fab640 1 -- 172.21.15.43:0/3826871595 shutdown_connections 2023-08-09T13:53:38.872 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:38.866+0000 7f3548fab640 1 -- 172.21.15.43:0/3826871595 wait complete. 2023-08-09T13:53:38.873 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:38.868+0000 7f3548fab640 1 Processor -- start 2023-08-09T13:53:38.873 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:38.868+0000 7f3548fab640 1 -- start start 2023-08-09T13:53:38.873 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:38.869+0000 7f3548fab640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f35440c94f0 0x7f3544151710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:53:38.873 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:38.869+0000 7f3548fab640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f35440cbd90 0x7f354415bfb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:53:38.873 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:38.869+0000 7f3548fab640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f354415a6a0 0x7f354415c4f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:53:38.873 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:38.869+0000 7f3548fab640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f354409b4f0 con 0x7f354415a6a0 2023-08-09T13:53:38.873 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:38.869+0000 7f3548fab640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f354406bc60 con 0x7f35440cbd90 2023-08-09T13:53:38.874 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:38.869+0000 7f3548fab640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3544098dc0 con 0x7f35440c94f0 2023-08-09T13:53:38.874 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:38.869+0000 7f3543fff640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f354415a6a0 0x7f354415c4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:53:38.874 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:38.869+0000 7f3543fff640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f354415a6a0 0x7f354415c4f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:53450/0 (socket says 172.21.15.43:53450) 2023-08-09T13:53:38.874 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:38.869+0000 7f3543fff640 1 -- 172.21.15.43:0/2731985203 learned_addr learned my addr 172.21.15.43:0/2731985203 (peer_addr_for_me v2:172.21.15.43:0/0) 2023-08-09T13:53:38.874 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:38.870+0000 7f35437fe640 1 --2- 172.21.15.43:0/2731985203 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f35440c94f0 0x7f3544151710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:53:38.874 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:38.870+0000 7f3542ffd640 1 --2- 172.21.15.43:0/2731985203 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f35440cbd90 0x7f354415bfb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:53:38.875 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:38.870+0000 7f3543fff640 1 -- 172.21.15.43:0/2731985203 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f35440c94f0 msgr2=0x7f3544151710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:53:38.876 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:38.870+0000 7f3543fff640 1 --2- 172.21.15.43:0/2731985203 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f35440c94f0 0x7f3544151710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:53:38.876 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:38.870+0000 7f3543fff640 1 -- 172.21.15.43:0/2731985203 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f35440cbd90 msgr2=0x7f354415bfb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:53:38.876 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:38.870+0000 7f3543fff640 1 --2- 172.21.15.43:0/2731985203 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f35440cbd90 0x7f354415bfb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:53:38.876 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:38.870+0000 7f3543fff640 1 -- 172.21.15.43:0/2731985203 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f35340094d0 con 0x7f354415a6a0 2023-08-09T13:53:38.876 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:38.870+0000 7f3543fff640 1 --2- 172.21.15.43:0/2731985203 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f354415a6a0 0x7f354415c4f0 secure :-1 s=READY pgs=1470 cs=0 l=1 rev1=1 crypto rx=0x7f353c00afd0 tx=0x7f353c00ca20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-09T13:53:38.876 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:38.870+0000 7f352bfff640 1 -- 172.21.15.43:0/2731985203 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f353c004440 con 0x7f354415a6a0 2023-08-09T13:53:38.876 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:38.871+0000 7f352bfff640 1 -- 172.21.15.43:0/2731985203 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f353c003610 con 0x7f354415a6a0 2023-08-09T13:53:38.877 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:38.871+0000 7f352bfff640 1 -- 172.21.15.43:0/2731985203 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f353c0039c0 con 0x7f354415a6a0 2023-08-09T13:53:38.877 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:38.871+0000 7f3548fab640 1 -- 172.21.15.43:0/2731985203 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f35440990a0 con 0x7f354415a6a0 2023-08-09T13:53:38.877 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:38.871+0000 7f3548fab640 1 -- 172.21.15.43:0/2731985203 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3544099660 con 0x7f354415a6a0 2023-08-09T13:53:38.877 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:38.872+0000 7f3548fab640 1 -- 172.21.15.43:0/2731985203 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f354406bdd0 con 0x7f354415a6a0 2023-08-09T13:53:38.877 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:38.873+0000 7f352bfff640 1 -- 172.21.15.43:0/2731985203 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 56) v1 ==== 88443+0+0 (secure 0 0 0) 0x7f353c003b20 con 0x7f354415a6a0 2023-08-09T13:53:38.877 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:38.874+0000 7f352bfff640 1 --2- 172.21.15.43:0/2731985203 >> [v2:172.21.15.203:6832/2527059680,v1:172.21.15.203:6833/2527059680] conn(0x7f353006a170 0x7f353006c630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:53:38.877 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:38.874+0000 7f352bfff640 1 -- 172.21.15.43:0/2731985203 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(121..121 src has 1..121) v4 ==== 6434+0+0 (secure 0 0 0) 0x7f353c088ba0 con 0x7f354415a6a0 2023-08-09T13:53:38.878 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:38.874+0000 7f35437fe640 1 --2- 172.21.15.43:0/2731985203 >> [v2:172.21.15.203:6832/2527059680,v1:172.21.15.203:6833/2527059680] conn(0x7f353006a170 0x7f353006c630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:53:38.878 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:38.876+0000 7f35437fe640 1 --2- 172.21.15.43:0/2731985203 >> [v2:172.21.15.203:6832/2527059680,v1:172.21.15.203:6833/2527059680] conn(0x7f353006a170 0x7f353006c630 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7f3534036000 tx=0x7f353403d040 comp rx=0 tx=0).ready entity=mgr.6377 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-09T13:53:38.879 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:38.878+0000 7f352bfff640 1 -- 172.21.15.43:0/2731985203 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178066 (secure 0 0 0) 0x7f353c053ee0 con 0x7f354415a6a0 2023-08-09T13:53:39.152 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:39.148+0000 7f3548fab640 1 -- 172.21.15.43:0/2731985203 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_snap_schedules.TestSnapSchedules.test_snap_schedule"]} v 0) v1 -- 0x7f35440739f0 con 0x7f354415a6a0 2023-08-09T13:53:39.603 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:39.601+0000 7f352bfff640 1 -- 172.21.15.43:0/2731985203 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_snap_schedules.TestSnapSchedules.test_snap_schedule"]}]=0 v768) v1 ==== 151+0+0 (secure 0 0 0) 0x7f353c088030 con 0x7f354415a6a0 2023-08-09T13:53:39.607 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:39.605+0000 7f3548fab640 1 -- 172.21.15.43:0/2731985203 >> [v2:172.21.15.203:6832/2527059680,v1:172.21.15.203:6833/2527059680] conn(0x7f353006a170 msgr2=0x7f353006c630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:53:39.607 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:39.605+0000 7f3548fab640 1 --2- 172.21.15.43:0/2731985203 >> [v2:172.21.15.203:6832/2527059680,v1:172.21.15.203:6833/2527059680] conn(0x7f353006a170 0x7f353006c630 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7f3534036000 tx=0x7f353403d040 comp rx=0 tx=0).stop 2023-08-09T13:53:39.607 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:39.605+0000 7f3548fab640 1 -- 172.21.15.43:0/2731985203 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f354415a6a0 msgr2=0x7f354415c4f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:53:39.607 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:39.605+0000 7f3548fab640 1 --2- 172.21.15.43:0/2731985203 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f354415a6a0 0x7f354415c4f0 secure :-1 s=READY pgs=1470 cs=0 l=1 rev1=1 crypto rx=0x7f353c00afd0 tx=0x7f353c00ca20 comp rx=0 tx=0).stop 2023-08-09T13:53:39.607 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:39.605+0000 7f3548fab640 1 -- 172.21.15.43:0/2731985203 shutdown_connections 2023-08-09T13:53:39.607 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:39.605+0000 7f3548fab640 1 --2- 172.21.15.43:0/2731985203 >> [v2:172.21.15.203:6832/2527059680,v1:172.21.15.203:6833/2527059680] conn(0x7f353006a170 0x7f353006c630 unknown :-1 s=CLOSED pgs=39 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:53:39.607 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:39.605+0000 7f3548fab640 1 --2- 172.21.15.43:0/2731985203 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f354415a6a0 0x7f354415c4f0 unknown :-1 s=CLOSED pgs=1470 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:53:39.608 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:39.605+0000 7f3548fab640 1 --2- 172.21.15.43:0/2731985203 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f35440cbd90 0x7f354415bfb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:53:39.608 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:39.605+0000 7f3548fab640 1 --2- 172.21.15.43:0/2731985203 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f35440c94f0 0x7f3544151710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:53:39.608 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:39.605+0000 7f3548fab640 1 -- 172.21.15.43:0/2731985203 >> 172.21.15.43:0/2731985203 conn(0x7f35440cde90 msgr2=0x7f35440c9ae0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-09T13:53:39.608 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:39.605+0000 7f3548fab640 1 -- 172.21.15.43:0/2731985203 shutdown_connections 2023-08-09T13:53:39.608 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:39.605+0000 7f3548fab640 1 -- 172.21.15.43:0/2731985203 wait complete. 2023-08-09T13:53:39.629 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-08-09T13:53:39.848 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:39.846+0000 7ff617a06640 1 Processor -- start 2023-08-09T13:53:39.848 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:39.846+0000 7ff617a06640 1 -- start start 2023-08-09T13:53:39.849 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:39.848+0000 7ff617a06640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7ff61006cfe0 0x7ff6100c8b50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:53:39.850 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:39.848+0000 7ff617a06640 1 -- --> v1:172.21.15.43:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff6100c9090 con 0x7ff6100c86f0 2023-08-09T13:53:39.850 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:39.848+0000 7ff617a06640 1 -- --> v1:172.21.15.203:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff61006c440 con 0x7ff610143c20 2023-08-09T13:53:39.850 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:39.848+0000 7ff617a06640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff610144000 con 0x7ff61006cfe0 2023-08-09T13:53:39.851 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:39.848+0000 7ff614fa1640 1 --1- >> v1:172.21.15.43:6789/0 conn(0x7ff6100c86f0 0x7ff61006c8d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.43:6789/0 says I am v1:172.21.15.43:44772/0 (socket says 172.21.15.43:44772) 2023-08-09T13:53:39.851 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:39.849+0000 7ff614fa1640 1 -- 172.21.15.43:0/9107470 learned_addr learned my addr 172.21.15.43:0/9107470 (peer_addr_for_me v1:172.21.15.43:0/0) 2023-08-09T13:53:39.851 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:39.849+0000 7ff615fa3640 1 --2- 172.21.15.43:0/9107470 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7ff61006cfe0 0x7ff6100c8b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:53:39.852 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:39.849+0000 7ff607fff640 1 -- 172.21.15.43:0/9107470 <== mon.0 v1:172.21.15.43:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3583828631 0 0) 0x7ff6100c9090 con 0x7ff6100c86f0 2023-08-09T13:53:39.852 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:39.849+0000 7ff607fff640 1 -- 172.21.15.43:0/9107470 --> v1:172.21.15.43:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ff5fc003520 con 0x7ff6100c86f0 2023-08-09T13:53:39.852 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:39.849+0000 7ff607fff640 1 -- 172.21.15.43:0/9107470 <== mon.0 v1:172.21.15.43:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3949997238 0 0) 0x7ff5fc003520 con 0x7ff6100c86f0 2023-08-09T13:53:39.852 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:39.850+0000 7ff607fff640 1 -- 172.21.15.43:0/9107470 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7ff61006cfe0 msgr2=0x7ff6100c8b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-09T13:53:39.852 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:39.850+0000 7ff607fff640 1 --2- 172.21.15.43:0/9107470 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7ff61006cfe0 0x7ff6100c8b50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:53:39.852 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:39.850+0000 7ff607fff640 1 -- 172.21.15.43:0/9107470 >> v1:172.21.15.203:6789/0 conn(0x7ff610143c20 legacy=0x7ff61006f540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:53:39.852 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:39.850+0000 7ff607fff640 1 -- 172.21.15.43:0/9107470 --> v1:172.21.15.43:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff61006fc50 con 0x7ff6100c86f0 2023-08-09T13:53:39.853 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:39.850+0000 7ff607fff640 1 -- 172.21.15.43:0/9107470 <== mon.0 v1:172.21.15.43:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1285150740 0 0) 0x7ff600001670 con 0x7ff6100c86f0 2023-08-09T13:53:39.853 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:39.850+0000 7ff607fff640 1 -- 172.21.15.43:0/9107470 >> v1:172.21.15.43:6789/0 conn(0x7ff6100c86f0 legacy=0x7ff61006c8d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:53:39.853 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:39.850+0000 7ff607fff640 1 --2- 172.21.15.43:0/9107470 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7ff5fc003a10 0x7ff5fc003e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:53:39.853 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:39.850+0000 7ff607fff640 1 --2- 172.21.15.43:0/9107470 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7ff5fc0047c0 0x7ff5fc004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:53:39.853 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:39.850+0000 7ff607fff640 1 --2- 172.21.15.43:0/9107470 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff5fc005180 0x7ff5fc00fd90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:53:39.853 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:39.850+0000 7ff607fff640 1 -- 172.21.15.43:0/9107470 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff610144000 con 0x7ff5fc005180 2023-08-09T13:53:39.853 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:39.850+0000 7ff607fff640 1 -- 172.21.15.43:0/9107470 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff5fc010460 con 0x7ff5fc0047c0 2023-08-09T13:53:39.853 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:39.850+0000 7ff607fff640 1 -- 172.21.15.43:0/9107470 --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff5fc010620 con 0x7ff5fc003a10 2023-08-09T13:53:39.854 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:39.850+0000 7ff615fa3640 1 --2- 172.21.15.43:0/9107470 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff5fc005180 0x7ff5fc00fd90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:53:39.855 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:39.850+0000 7ff6157a2640 1 --2- 172.21.15.43:0/9107470 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7ff5fc003a10 0x7ff5fc003e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:53:39.855 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:39.850+0000 7ff614fa1640 1 --2- 172.21.15.43:0/9107470 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7ff5fc0047c0 0x7ff5fc004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:53:39.855 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:39.850+0000 7ff615fa3640 1 -- 172.21.15.43:0/9107470 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7ff5fc003a10 msgr2=0x7ff5fc003e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-09T13:53:39.855 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:39.850+0000 7ff615fa3640 1 --2- 172.21.15.43:0/9107470 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7ff5fc003a10 0x7ff5fc003e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:53:39.856 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:39.850+0000 7ff615fa3640 1 -- 172.21.15.43:0/9107470 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7ff5fc0047c0 msgr2=0x7ff5fc004c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-09T13:53:39.856 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:39.850+0000 7ff615fa3640 1 --2- 172.21.15.43:0/9107470 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7ff5fc0047c0 0x7ff5fc004c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:53:39.856 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:39.850+0000 7ff615fa3640 1 -- 172.21.15.43:0/9107470 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff61006fc50 con 0x7ff5fc005180 2023-08-09T13:53:39.856 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:39.852+0000 7ff615fa3640 1 --2- 172.21.15.43:0/9107470 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff5fc005180 0x7ff5fc00fd90 secure :-1 s=READY pgs=1472 cs=0 l=1 rev1=1 crypto rx=0x7ff61006d3c0 tx=0x7ff5fc0057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d51e3b48131665e8 server_cookie=0 in_seq=0 out_seq=0 2023-08-09T13:53:39.856 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:39.852+0000 7ff607fff640 1 -- 172.21.15.43:0/9107470 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff60802cb70 con 0x7ff5fc005180 2023-08-09T13:53:39.856 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:39.852+0000 7ff607fff640 1 -- 172.21.15.43:0/9107470 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff60800f040 con 0x7ff5fc005180 2023-08-09T13:53:39.856 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:39.852+0000 7ff607fff640 1 -- 172.21.15.43:0/9107470 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff608034440 con 0x7ff5fc005180 2023-08-09T13:53:39.857 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:39.852+0000 7ff617a06640 1 -- 172.21.15.43:0/9107470 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff5fc005180 msgr2=0x7ff5fc00fd90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:53:39.857 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:39.852+0000 7ff617a06640 1 --2- 172.21.15.43:0/9107470 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff5fc005180 0x7ff5fc00fd90 secure :-1 s=READY pgs=1472 cs=0 l=1 rev1=1 crypto rx=0x7ff61006d3c0 tx=0x7ff5fc0057f0 comp rx=0 tx=0).stop 2023-08-09T13:53:39.857 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:39.853+0000 7ff615fa3640 1 -- 172.21.15.43:0/9107470 reap_dead start 2023-08-09T13:53:39.857 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:39.853+0000 7ff617a06640 1 -- 172.21.15.43:0/9107470 shutdown_connections 2023-08-09T13:53:39.857 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:39.853+0000 7ff617a06640 1 -- 172.21.15.43:0/9107470 >> 172.21.15.43:0/9107470 conn(0x7ff6100ce0a0 msgr2=0x7ff6100ce480 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-09T13:53:39.857 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:39.853+0000 7ff617a06640 1 -- 172.21.15.43:0/9107470 shutdown_connections 2023-08-09T13:53:39.857 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:39.853+0000 7ff617a06640 1 -- 172.21.15.43:0/9107470 wait complete. 2023-08-09T13:53:39.859 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:39.855+0000 7ff617a06640 1 Processor -- start 2023-08-09T13:53:39.859 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:39.855+0000 7ff617a06640 1 -- start start 2023-08-09T13:53:39.859 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:39.856+0000 7ff617a06640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7ff5fc005180 0x7ff6100b0f40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:53:39.859 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:39.856+0000 7ff617a06640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff61013dd50 0x7ff61015aa60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:53:39.859 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:39.856+0000 7ff617a06640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7ff61015afa0 0x7ff61015b420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:53:39.860 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:39.856+0000 7ff617a06640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff6100b1480 con 0x7ff61013dd50 2023-08-09T13:53:39.860 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:39.856+0000 7ff617a06640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff6100b41d0 con 0x7ff61015afa0 2023-08-09T13:53:39.860 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:39.856+0000 7ff617a06640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff6100b3c10 con 0x7ff5fc005180 2023-08-09T13:53:39.860 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:39.856+0000 7ff614fa1640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff61013dd50 0x7ff61015aa60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:53:39.860 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:39.856+0000 7ff614fa1640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff61013dd50 0x7ff61015aa60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:53466/0 (socket says 172.21.15.43:53466) 2023-08-09T13:53:39.860 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:39.856+0000 7ff614fa1640 1 -- 172.21.15.43:0/2106385993 learned_addr learned my addr 172.21.15.43:0/2106385993 (peer_addr_for_me v2:172.21.15.43:0/0) 2023-08-09T13:53:39.860 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:39.856+0000 7ff6157a2640 1 --2- 172.21.15.43:0/2106385993 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7ff5fc005180 0x7ff6100b0f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:53:39.861 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:39.856+0000 7ff614fa1640 1 -- 172.21.15.43:0/2106385993 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7ff5fc005180 msgr2=0x7ff6100b0f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:53:39.861 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:39.856+0000 7ff614fa1640 1 --2- 172.21.15.43:0/2106385993 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7ff5fc005180 0x7ff6100b0f40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:53:39.861 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:39.856+0000 7ff614fa1640 1 -- 172.21.15.43:0/2106385993 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7ff61015afa0 msgr2=0x7ff61015b420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:53:39.861 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:39.856+0000 7ff614fa1640 1 --2- 172.21.15.43:0/2106385993 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7ff61015afa0 0x7ff61015b420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:53:39.861 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:39.856+0000 7ff614fa1640 1 -- 172.21.15.43:0/2106385993 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff6100b3ef0 con 0x7ff61013dd50 2023-08-09T13:53:39.861 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:39.857+0000 7ff614fa1640 1 --2- 172.21.15.43:0/2106385993 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff61013dd50 0x7ff61015aa60 secure :-1 s=READY pgs=1473 cs=0 l=1 rev1=1 crypto rx=0x7ff60000e840 tx=0x7ff60000ed00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-09T13:53:39.861 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:39.857+0000 7ff605ffb640 1 -- 172.21.15.43:0/2106385993 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff60000c850 con 0x7ff61013dd50 2023-08-09T13:53:39.861 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:39.857+0000 7ff605ffb640 1 -- 172.21.15.43:0/2106385993 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff60000c9b0 con 0x7ff61013dd50 2023-08-09T13:53:39.862 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:39.857+0000 7ff605ffb640 1 -- 172.21.15.43:0/2106385993 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff60000cd60 con 0x7ff61013dd50 2023-08-09T13:53:39.862 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:39.857+0000 7ff617a06640 1 -- 172.21.15.43:0/2106385993 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff6100c5830 con 0x7ff61013dd50 2023-08-09T13:53:39.862 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:39.857+0000 7ff617a06640 1 -- 172.21.15.43:0/2106385993 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff61013e1b0 con 0x7ff61013dd50 2023-08-09T13:53:39.862 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:39.858+0000 7ff617a06640 1 -- 172.21.15.43:0/2106385993 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff61013e370 con 0x7ff61013dd50 2023-08-09T13:53:39.863 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:39.859+0000 7ff605ffb640 1 -- 172.21.15.43:0/2106385993 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 56) v1 ==== 88443+0+0 (secure 0 0 0) 0x7ff60001d460 con 0x7ff61013dd50 2023-08-09T13:53:39.863 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:39.860+0000 7ff605ffb640 1 --2- 172.21.15.43:0/2106385993 >> [v2:172.21.15.203:6832/2527059680,v1:172.21.15.203:6833/2527059680] conn(0x7ff5f406a170 0x7ff5f406c630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:53:39.863 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:39.860+0000 7ff605ffb640 1 -- 172.21.15.43:0/2106385993 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(121..121 src has 1..121) v4 ==== 6434+0+0 (secure 0 0 0) 0x7ff600088930 con 0x7ff61013dd50 2023-08-09T13:53:39.863 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:39.861+0000 7ff6157a2640 1 --2- 172.21.15.43:0/2106385993 >> [v2:172.21.15.203:6832/2527059680,v1:172.21.15.203:6833/2527059680] conn(0x7ff5f406a170 0x7ff5f406c630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:53:39.863 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:39.862+0000 7ff6157a2640 1 --2- 172.21.15.43:0/2106385993 >> [v2:172.21.15.203:6832/2527059680,v1:172.21.15.203:6833/2527059680] conn(0x7ff5f406a170 0x7ff5f406c630 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7ff60c007fd0 tx=0x7ff60c009040 comp rx=0 tx=0).ready entity=mgr.6377 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-09T13:53:39.866 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:39.865+0000 7ff605ffb640 1 -- 172.21.15.43:0/2106385993 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178066 (secure 0 0 0) 0x7ff61013e370 con 0x7ff61013dd50 2023-08-09T13:53:40.061 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:40.060+0000 7ff617a06640 1 -- 172.21.15.43:0/2106385993 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7ff61013e370 con 0x7ff61013dd50 2023-08-09T13:53:40.063 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:40.062+0000 7ff605ffb640 1 -- 172.21.15.43:0/2106385993 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v121) v1 ==== 74+0+16125 (secure 0 0 0) 0x7ff61013e370 con 0x7ff61013dd50 2023-08-09T13:53:40.064 INFO:teuthology.orchestra.run.smithi043.stdout: 2023-08-09T13:53:40.064 INFO:teuthology.orchestra.run.smithi043.stdout:{"epoch":121,"fsid":"daac87ee-1337-4250-a955-aea4e408a39b","created":"2023-08-09T13:33:41.701198+0000","modified":"2023-08-09T13:53:36.567327+0000","last_up_change":"2023-08-09T13:33:53.783453+0000","last_in_change":"2023-08-09T13:33:45.148904+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":15,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":"rbd","create_time":"2023-08-09T13:33:54.365137+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":".mgr","create_time":"2023-08-09T13:33:54.831803+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"cbd49ecd-e896-4fbd-864e-5c37a829b165","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":114,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6800","nonce":502667335},{"type":"v1","addr":"172.21.15.43:6801","nonce":502667335}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6802","nonce":502667335},{"type":"v1","addr":"172.21.15.43:6803","nonce":502667335}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6806","nonce":502667335},{"type":"v1","addr":"172.21.15.43:6807","nonce":502667335}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6804","nonce":502667335},{"type":"v1","addr":"172.21.15.43:6805","nonce":502667335}]},"public_addr":"172.21.15.43:6801/502667335","cluster_addr":"172.21.15.43:6803/502667335","heartbeat_back_addr":"172.21.15.43:6807/502667335","heartbeat_front_addr":"172.21.15.43:6805/502667335","state":["exists","up"]},{"osd":1,"uuid":"75c7e0fe-1ec7-4fe8-a2f2-3661a65b8500","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":114,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6808","nonce":3289134610},{"type":"v1","addr":"172.21.15.43:6809","nonce":3289134610}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6810","nonce":3289134610},{"type":"v1","addr":"172.21.15.43:6812","nonce":3289134610}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6821","nonce":3289134610},{"type":"v1","addr":"172.21.15.43:6824","nonce":3289134610}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6815","nonce":3289134610},{"type":"v1","addr":"172.21.15.43:6818","nonce":3289134610}]},"public_addr":"172.21.15.43:6809/3289134610","cluster_addr":"172.21.15.43:6812/3289134610","heartbeat_back_addr":"172.21.15.43:6824/3289134610","heartbeat_front_addr":"172.21.15.43:6818/3289134610","state":["exists","up"]},{"osd":2,"uuid":"bb5c41e9-7f5f-4125-9b01-3bb8bfc2ea83","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":114,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6811","nonce":4079709736},{"type":"v1","addr":"172.21.15.43:6814","nonce":4079709736}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6817","nonce":4079709736},{"type":"v1","addr":"172.21.15.43:6820","nonce":4079709736}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6829","nonce":4079709736},{"type":"v1","addr":"172.21.15.43:6831","nonce":4079709736}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6823","nonce":4079709736},{"type":"v1","addr":"172.21.15.43:6826","nonce":4079709736}]},"public_addr":"172.21.15.43:6814/4079709736","cluster_addr":"172.21.15.43:6820/4079709736","heartbeat_back_addr":"172.21.15.43:6831/4079709736","heartbeat_front_addr":"172.21.15.43:6826/4079709736","state":["exists","up"]},{"osd":3,"uuid":"bfe4c8b4-694a-465c-87e6-2b2aab852613","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":114,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6813","nonce":1709538759},{"type":"v1","addr":"172.21.15.43:6816","nonce":1709538759}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6819","nonce":1709538759},{"type":"v1","addr":"172.21.15.43:6822","nonce":1709538759}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6828","nonce":1709538759},{"type":"v1","addr":"172.21.15.43:6830","nonce":1709538759}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6825","nonce":1709538759},{"type":"v1","addr":"172.21.15.43:6827","nonce":1709538759}]},"public_addr":"172.21.15.43:6816/1709538759","cluster_addr":"172.21.15.43:6822/1709538759","heartbeat_back_addr":"172.21.15.43:6830/1709538759","heartbeat_front_addr":"172.21.15.43:6827/1709538759","state":["exists","up"]},{"osd":4,"uuid":"fa2e8a77-aba1-4756-853e-0d174a5bdeba","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":114,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6800","nonce":4196866481},{"type":"v1","addr":"172.21.15.203:6801","nonce":4196866481}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6802","nonce":4196866481},{"type":"v1","addr":"172.21.15.203:6803","nonce":4196866481}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6806","nonce":4196866481},{"type":"v1","addr":"172.21.15.203:6807","nonce":4196866481}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6804","nonce":4196866481},{"type":"v1","addr":"172.21.15.203:6805","nonce":4196866481}]},"public_addr":"172.21.15.203:6801/4196866481","cluster_addr":"172.21.15.203:6803/4196866481","heartbeat_back_addr":"172.21.15.203:6807/4196866481","heartbeat_front_addr":"172.21.15.203:6805/4196866481","state":["exists","up"]},{"osd":5,"uuid":"81811e4b-4d24-4f51-a82c-f1d700a2a00d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":114,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6816","nonce":3059290081},{"type":"v1","addr":"172.21.15.203:6817","nonce":3059290081}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6818","nonce":3059290081},{"type":"v1","addr":"172.21.15.203:6819","nonce":3059290081}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6822","nonce":3059290081},{"type":"v1","addr":"172.21.15.203:6823","nonce":3059290081}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6820","nonce":3059290081},{"type":"v1","addr":"172.21.15.203:6821","nonce":3059290081}]},"public_addr":"172.21.15.203:6817/3059290081","cluster_addr":"172.21.15.203:6819/3059290081","heartbeat_back_addr":"172.21.15.203:6823/3059290081","heartbeat_front_addr":"172.21.15.203:6821/3059290081","state":["exists","up"]},{"osd":6,"uuid":"307bac4d-6a5b-443d-af75-56d695d7674f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":114,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6824","nonce":512155569},{"type":"v1","addr":"172.21.15.203:6825","nonce":512155569}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6826","nonce":512155569},{"type":"v1","addr":"172.21.15.203:6827","nonce":512155569}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6830","nonce":512155569},{"type":"v1","addr":"172.21.15.203:6831","nonce":512155569}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6828","nonce":512155569},{"type":"v1","addr":"172.21.15.203:6829","nonce":512155569}]},"public_addr":"172.21.15.203:6825/512155569","cluster_addr":"172.21.15.203:6827/512155569","heartbeat_back_addr":"172.21.15.203:6831/512155569","heartbeat_front_addr":"172.21.15.203:6829/512155569","state":["exists","up"]},{"osd":7,"uuid":"a95dc7ca-9f1b-4295-af70-68a22a2d7d5e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":114,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6808","nonce":3331450438},{"type":"v1","addr":"172.21.15.203:6809","nonce":3331450438}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6810","nonce":3331450438},{"type":"v1","addr":"172.21.15.203:6811","nonce":3331450438}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6814","nonce":3331450438},{"type":"v1","addr":"172.21.15.203:6815","nonce":3331450438}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6812","nonce":3331450438},{"type":"v1","addr":"172.21.15.203:6813","nonce":3331450438}]},"public_addr":"172.21.15.203:6809/3331450438","cluster_addr":"172.21.15.203:6811/3331450438","heartbeat_back_addr":"172.21.15.203:6815/3331450438","heartbeat_front_addr":"172.21.15.203:6813/3331450438","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-09T13:33:50.547652+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-09T13:33:50.589469+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-09T13:33:50.561724+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-09T13:33:50.599650+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-09T13:33:50.411855+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-09T13:33:50.219681+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-09T13:33:50.504664+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-09T13:33:50.441248+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.203:0/3668440292":"2023-08-10T13:53:28.338643+0000","172.21.15.43:6834/2223580538":"2023-08-10T13:53:33.549126+0000","172.21.15.203:0/430012896":"2023-08-10T13:53:28.338643+0000","172.21.15.43:6832/2223580538":"2023-08-10T13:53:33.549126+0000","172.21.15.203:0/1052273067":"2023-08-10T13:53:28.338643+0000","172.21.15.203:0/2583349636":"2023-08-10T13:53:28.338643+0000","172.21.15.203:6832/2041169533":"2023-08-10T13:53:28.338643+0000","172.21.15.203:0/1488154102":"2023-08-10T13:52:20.374695+0000","172.21.15.203:0/728819840":"2023-08-10T13:53:28.338643+0000","172.21.15.203:0/3595044319":"2023-08-10T13:52:20.374695+0000","172.21.15.203:0/2916799066":"2023-08-10T13:53:28.338643+0000","172.21.15.203:0/3843164760":"2023-08-10T13:53:28.338643+0000","172.21.15.203:0/1657891023":"2023-08-10T13:52:20.374695+0000","172.21.15.203:0/329310299":"2023-08-10T13:53:28.338643+0000","172.21.15.203:0/2368610482":"2023-08-10T13:52:20.374695+0000","172.21.15.203:0/437192587":"2023-08-10T13:52:20.374695+0000","172.21.15.203:6833/1317108840":"2023-08-10T13:52:20.374695+0000","172.21.15.203:6832/1317108840":"2023-08-10T13:52:20.374695+0000","172.21.15.203:6833/2041169533":"2023-08-10T13:53:28.338643+0000","172.21.15.203:0/1459918882":"2023-08-10T13:52:20.374695+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":7,"snaps":[{"begin":104,"length":1},{"begin":221,"length":1}]}],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-08-09T13:53:40.066 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:40.065+0000 7ff617a06640 1 -- 172.21.15.43:0/2106385993 >> [v2:172.21.15.203:6832/2527059680,v1:172.21.15.203:6833/2527059680] conn(0x7ff5f406a170 msgr2=0x7ff5f406c630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:53:40.067 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:40.065+0000 7ff617a06640 1 --2- 172.21.15.43:0/2106385993 >> [v2:172.21.15.203:6832/2527059680,v1:172.21.15.203:6833/2527059680] conn(0x7ff5f406a170 0x7ff5f406c630 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7ff60c007fd0 tx=0x7ff60c009040 comp rx=0 tx=0).stop 2023-08-09T13:53:40.067 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:40.065+0000 7ff617a06640 1 -- 172.21.15.43:0/2106385993 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff61013dd50 msgr2=0x7ff61015aa60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:53:40.067 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:40.065+0000 7ff617a06640 1 --2- 172.21.15.43:0/2106385993 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff61013dd50 0x7ff61015aa60 secure :-1 s=READY pgs=1473 cs=0 l=1 rev1=1 crypto rx=0x7ff60000e840 tx=0x7ff60000ed00 comp rx=0 tx=0).stop 2023-08-09T13:53:40.067 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:40.065+0000 7ff617a06640 1 -- 172.21.15.43:0/2106385993 shutdown_connections 2023-08-09T13:53:40.067 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:40.065+0000 7ff617a06640 1 --2- 172.21.15.43:0/2106385993 >> [v2:172.21.15.203:6832/2527059680,v1:172.21.15.203:6833/2527059680] conn(0x7ff5f406a170 0x7ff5f406c630 unknown :-1 s=CLOSED pgs=40 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:53:40.067 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:40.065+0000 7ff617a06640 1 --2- 172.21.15.43:0/2106385993 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7ff61015afa0 0x7ff61015b420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:53:40.068 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:40.065+0000 7ff617a06640 1 --2- 172.21.15.43:0/2106385993 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff61013dd50 0x7ff61015aa60 unknown :-1 s=CLOSED pgs=1473 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:53:40.068 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:40.065+0000 7ff617a06640 1 --2- 172.21.15.43:0/2106385993 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7ff5fc005180 0x7ff6100b0f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:53:40.068 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:40.065+0000 7ff617a06640 1 -- 172.21.15.43:0/2106385993 >> 172.21.15.43:0/2106385993 conn(0x7ff6100ce0a0 msgr2=0x7ff6100c2f20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-09T13:53:40.068 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:40.065+0000 7ff617a06640 1 -- 172.21.15.43:0/2106385993 shutdown_connections 2023-08-09T13:53:40.068 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:40.065+0000 7ff617a06640 1 -- 172.21.15.43:0/2106385993 wait complete. 2023-08-09T13:53:40.087 INFO:tasks.cephfs.cephfs_test_case.ceph_manager:[{'pool': 1, 'pool_name': 'rbd', 'create_time': '2023-08-09T13:33:54.365137+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}, 'read_balance': {'score_acting': 2, 'score_stable': 2, 'optimal_score': 1, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': '.mgr', 'create_time': '2023-08-09T13:33:54.831803+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-08-09T13:53:40.088 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get rbd pg_num 2023-08-09T13:53:40.320 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:40.317+0000 7f2c8848e640 1 Processor -- start 2023-08-09T13:53:40.321 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:40.318+0000 7f2c8848e640 1 -- start start 2023-08-09T13:53:40.322 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:40.319+0000 7f2c8848e640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f2c800cb2d0 0x7f2c8015b380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:53:40.322 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:40.319+0000 7f2c8848e640 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7f2c8015df60 0x7f2c800ce1c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:53:40.322 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:40.319+0000 7f2c8848e640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f2c8015e3c0 0x7f2c800ce700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:53:40.322 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:40.319+0000 7f2c8848e640 1 -- --> v2:172.21.15.43:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2c8015a970 con 0x7f2c8015e3c0 2023-08-09T13:53:40.323 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:40.319+0000 7f2c8848e640 1 -- --> v2:172.21.15.203:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2c8015aef0 con 0x7f2c8015df60 2023-08-09T13:53:40.323 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:40.319+0000 7f2c8848e640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2c8015b8c0 con 0x7f2c800cb2d0 2023-08-09T13:53:40.323 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:40.320+0000 7f2c86a2b640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f2c8015e3c0 0x7f2c800ce700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:53:40.324 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:40.320+0000 7f2c86a2b640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f2c8015e3c0 0x7f2c800ce700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:53476/0 (socket says 172.21.15.43:53476) 2023-08-09T13:53:40.324 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:40.320+0000 7f2c86a2b640 1 -- 172.21.15.43:0/1625310971 learned_addr learned my addr 172.21.15.43:0/1625310971 (peer_addr_for_me v2:172.21.15.43:0/0) 2023-08-09T13:53:40.324 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:40.320+0000 7f2c85a29640 1 --2- 172.21.15.43:0/1625310971 >> v2:172.21.15.203:3300/0 conn(0x7f2c8015df60 0x7f2c800ce1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:53:40.324 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:40.320+0000 7f2c8622a640 1 --2- 172.21.15.43:0/1625310971 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f2c800cb2d0 0x7f2c8015b380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:53:40.324 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:40.320+0000 7f2c86a2b640 1 -- 172.21.15.43:0/1625310971 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f2c800cb2d0 msgr2=0x7f2c8015b380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-09T13:53:40.324 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:40.320+0000 7f2c86a2b640 1 --2- 172.21.15.43:0/1625310971 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f2c800cb2d0 0x7f2c8015b380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:53:40.325 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:40.320+0000 7f2c86a2b640 1 -- 172.21.15.43:0/1625310971 >> v2:172.21.15.203:3300/0 conn(0x7f2c8015df60 msgr2=0x7f2c800ce1c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-09T13:53:40.325 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:40.320+0000 7f2c86a2b640 1 --2- 172.21.15.43:0/1625310971 >> v2:172.21.15.203:3300/0 conn(0x7f2c8015df60 0x7f2c800ce1c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:53:40.325 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:40.321+0000 7f2c86a2b640 1 -- 172.21.15.43:0/1625310971 --> v2:172.21.15.43:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2c8015e940 con 0x7f2c8015e3c0 2023-08-09T13:53:40.326 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:40.322+0000 7f2c86a2b640 1 --2- 172.21.15.43:0/1625310971 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f2c8015e3c0 0x7f2c800ce700 secure :-1 s=READY pgs=1474 cs=0 l=1 rev1=1 crypto rx=0x7f2c800cecf0 tx=0x7f2c7802b900 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a5df1369196e98a server_cookie=0 in_seq=0 out_seq=0 2023-08-09T13:53:40.326 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:40.322+0000 7f2c85228640 1 -- 172.21.15.43:0/1625310971 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2c78031860 con 0x7f2c8015e3c0 2023-08-09T13:53:40.326 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:40.322+0000 7f2c85228640 1 -- 172.21.15.43:0/1625310971 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2c780319c0 con 0x7f2c8015e3c0 2023-08-09T13:53:40.326 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:40.322+0000 7f2c85228640 1 -- 172.21.15.43:0/1625310971 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2c78031d50 con 0x7f2c8015e3c0 2023-08-09T13:53:40.326 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:40.323+0000 7f2c8848e640 1 -- 172.21.15.43:0/1625310971 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f2c8015e3c0 msgr2=0x7f2c800ce700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:53:40.327 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:40.323+0000 7f2c8848e640 1 --2- 172.21.15.43:0/1625310971 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f2c8015e3c0 0x7f2c800ce700 secure :-1 s=READY pgs=1474 cs=0 l=1 rev1=1 crypto rx=0x7f2c800cecf0 tx=0x7f2c7802b900 comp rx=0 tx=0).stop 2023-08-09T13:53:40.327 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:40.323+0000 7f2c8848e640 1 -- 172.21.15.43:0/1625310971 shutdown_connections 2023-08-09T13:53:40.327 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:40.323+0000 7f2c8848e640 1 --2- 172.21.15.43:0/1625310971 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f2c8015e3c0 0x7f2c800ce700 unknown :-1 s=CLOSED pgs=1474 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:53:40.327 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:40.323+0000 7f2c8848e640 1 --2- 172.21.15.43:0/1625310971 >> v2:172.21.15.203:3300/0 conn(0x7f2c8015df60 0x7f2c800ce1c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:53:40.327 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:40.323+0000 7f2c8848e640 1 --2- 172.21.15.43:0/1625310971 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f2c800cb2d0 0x7f2c8015b380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:53:40.327 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:40.323+0000 7f2c8848e640 1 -- 172.21.15.43:0/1625310971 >> 172.21.15.43:0/1625310971 conn(0x7f2c800aff50 msgr2=0x7f2c800b0360 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-09T13:53:40.328 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:40.323+0000 7f2c8848e640 1 -- 172.21.15.43:0/1625310971 shutdown_connections 2023-08-09T13:53:40.328 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:40.323+0000 7f2c8848e640 1 -- 172.21.15.43:0/1625310971 wait complete. 2023-08-09T13:53:40.329 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:40.325+0000 7f2c8848e640 1 Processor -- start 2023-08-09T13:53:40.329 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:40.325+0000 7f2c8848e640 1 -- start start 2023-08-09T13:53:40.330 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:40.326+0000 7f2c8848e640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f2c800cb2d0 0x7f2c8014c100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:53:40.330 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:40.326+0000 7f2c8848e640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f2c8015df60 0x7f2c80150690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:53:40.330 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:40.326+0000 7f2c8848e640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f2c8015e3c0 0x7f2c800b1890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:53:40.330 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:40.326+0000 7f2c8848e640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2c800c7390 con 0x7f2c8015e3c0 2023-08-09T13:53:40.331 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:40.326+0000 7f2c8848e640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2c800c6710 con 0x7f2c800cb2d0 2023-08-09T13:53:40.331 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:40.326+0000 7f2c8848e640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2c800c69f0 con 0x7f2c8015df60 2023-08-09T13:53:40.331 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:40.326+0000 7f2c86a2b640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f2c8015e3c0 0x7f2c800b1890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:53:40.331 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:40.326+0000 7f2c86a2b640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f2c8015e3c0 0x7f2c800b1890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:53490/0 (socket says 172.21.15.43:53490) 2023-08-09T13:53:40.332 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:40.326+0000 7f2c86a2b640 1 -- 172.21.15.43:0/855485905 learned_addr learned my addr 172.21.15.43:0/855485905 (peer_addr_for_me v2:172.21.15.43:0/0) 2023-08-09T13:53:40.332 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:40.326+0000 7f2c85a29640 1 --2- 172.21.15.43:0/855485905 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f2c8015df60 0x7f2c80150690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:53:40.332 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:40.327+0000 7f2c86a2b640 1 -- 172.21.15.43:0/855485905 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f2c8015df60 msgr2=0x7f2c80150690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:53:40.332 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:40.327+0000 7f2c86a2b640 1 --2- 172.21.15.43:0/855485905 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f2c8015df60 0x7f2c80150690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:53:40.333 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:40.327+0000 7f2c86a2b640 1 -- 172.21.15.43:0/855485905 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f2c800cb2d0 msgr2=0x7f2c8014c100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:53:40.333 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:40.327+0000 7f2c86a2b640 1 --2- 172.21.15.43:0/855485905 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f2c800cb2d0 0x7f2c8014c100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:53:40.333 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:40.327+0000 7f2c86a2b640 1 -- 172.21.15.43:0/855485905 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2c780094d0 con 0x7f2c8015e3c0 2023-08-09T13:53:40.333 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:40.327+0000 7f2c85a29640 1 --2- 172.21.15.43:0/855485905 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f2c8015df60 0x7f2c80150690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-08-09T13:53:40.334 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:40.327+0000 7f2c86a2b640 1 --2- 172.21.15.43:0/855485905 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f2c8015e3c0 0x7f2c800b1890 secure :-1 s=READY pgs=1475 cs=0 l=1 rev1=1 crypto rx=0x7f2c7802b940 tx=0x7f2c780046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-09T13:53:40.336 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:40.327+0000 7f2c76ffd640 1 -- 172.21.15.43:0/855485905 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2c7803ac60 con 0x7f2c8015e3c0 2023-08-09T13:53:40.336 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:40.327+0000 7f2c76ffd640 1 -- 172.21.15.43:0/855485905 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2c7803ade0 con 0x7f2c8015e3c0 2023-08-09T13:53:40.336 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:40.327+0000 7f2c76ffd640 1 -- 172.21.15.43:0/855485905 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2c78044630 con 0x7f2c8015e3c0 2023-08-09T13:53:40.336 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:40.327+0000 7f2c8848e640 1 -- 172.21.15.43:0/855485905 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2c800c6cd0 con 0x7f2c8015e3c0 2023-08-09T13:53:40.337 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:40.328+0000 7f2c8848e640 1 -- 172.21.15.43:0/855485905 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2c800c63c0 con 0x7f2c8015e3c0 2023-08-09T13:53:40.337 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:40.329+0000 7f2c8848e640 1 -- 172.21.15.43:0/855485905 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2c800c74d0 con 0x7f2c8015e3c0 2023-08-09T13:53:40.337 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:40.330+0000 7f2c76ffd640 1 -- 172.21.15.43:0/855485905 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 56) v1 ==== 88443+0+0 (secure 0 0 0) 0x7f2c7804d450 con 0x7f2c8015e3c0 2023-08-09T13:53:40.337 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:40.331+0000 7f2c76ffd640 1 --2- 172.21.15.43:0/855485905 >> [v2:172.21.15.203:6832/2527059680,v1:172.21.15.203:6833/2527059680] conn(0x7f2c6406a170 0x7f2c6406c630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:53:40.338 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:40.331+0000 7f2c76ffd640 1 -- 172.21.15.43:0/855485905 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(121..121 src has 1..121) v4 ==== 6434+0+0 (secure 0 0 0) 0x7f2c780aaff0 con 0x7f2c8015e3c0 2023-08-09T13:53:40.338 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:40.331+0000 7f2c8622a640 1 --2- 172.21.15.43:0/855485905 >> [v2:172.21.15.203:6832/2527059680,v1:172.21.15.203:6833/2527059680] conn(0x7f2c6406a170 0x7f2c6406c630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:53:40.338 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:40.332+0000 7f2c8622a640 1 --2- 172.21.15.43:0/855485905 >> [v2:172.21.15.203:6832/2527059680,v1:172.21.15.203:6833/2527059680] conn(0x7f2c6406a170 0x7f2c6406c630 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7f2c7c009540 tx=0x7f2c7c009450 comp rx=0 tx=0).ready entity=mgr.6377 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-09T13:53:40.339 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:40.335+0000 7f2c76ffd640 1 -- 172.21.15.43:0/855485905 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178066 (secure 0 0 0) 0x7f2c780dd7b0 con 0x7f2c8015e3c0 2023-08-09T13:53:40.532 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:40.529+0000 7f2c8848e640 1 -- 172.21.15.43:0/855485905 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"} v 0) v1 -- 0x7f2c800c6b30 con 0x7f2c8015e3c0 2023-08-09T13:53:40.533 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:40.530+0000 7f2c76ffd640 1 -- 172.21.15.43:0/855485905 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"}]=0 v121) v1 ==== 92+0+10 (secure 0 0 0) 0x7f2c78035020 con 0x7f2c8015e3c0 2023-08-09T13:53:40.534 INFO:teuthology.orchestra.run.smithi043.stdout:pg_num: 8 2023-08-09T13:53:40.536 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:40.533+0000 7f2c8848e640 1 -- 172.21.15.43:0/855485905 >> [v2:172.21.15.203:6832/2527059680,v1:172.21.15.203:6833/2527059680] conn(0x7f2c6406a170 msgr2=0x7f2c6406c630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:53:40.537 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:40.533+0000 7f2c8848e640 1 --2- 172.21.15.43:0/855485905 >> [v2:172.21.15.203:6832/2527059680,v1:172.21.15.203:6833/2527059680] conn(0x7f2c6406a170 0x7f2c6406c630 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7f2c7c009540 tx=0x7f2c7c009450 comp rx=0 tx=0).stop 2023-08-09T13:53:40.537 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:40.533+0000 7f2c8848e640 1 -- 172.21.15.43:0/855485905 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f2c8015e3c0 msgr2=0x7f2c800b1890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:53:40.537 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:40.533+0000 7f2c8848e640 1 --2- 172.21.15.43:0/855485905 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f2c8015e3c0 0x7f2c800b1890 secure :-1 s=READY pgs=1475 cs=0 l=1 rev1=1 crypto rx=0x7f2c7802b940 tx=0x7f2c780046c0 comp rx=0 tx=0).stop 2023-08-09T13:53:40.538 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:40.533+0000 7f2c8848e640 1 -- 172.21.15.43:0/855485905 shutdown_connections 2023-08-09T13:53:40.538 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:40.533+0000 7f2c8848e640 1 --2- 172.21.15.43:0/855485905 >> [v2:172.21.15.203:6832/2527059680,v1:172.21.15.203:6833/2527059680] conn(0x7f2c6406a170 0x7f2c6406c630 unknown :-1 s=CLOSED pgs=41 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:53:40.538 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:40.533+0000 7f2c8848e640 1 --2- 172.21.15.43:0/855485905 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f2c8015e3c0 0x7f2c800b1890 unknown :-1 s=CLOSED pgs=1475 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:53:40.539 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:40.533+0000 7f2c8848e640 1 --2- 172.21.15.43:0/855485905 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f2c8015df60 0x7f2c80150690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:53:40.539 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:40.533+0000 7f2c8848e640 1 --2- 172.21.15.43:0/855485905 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f2c800cb2d0 0x7f2c8014c100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:53:40.540 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:40.533+0000 7f2c8848e640 1 -- 172.21.15.43:0/855485905 >> 172.21.15.43:0/855485905 conn(0x7f2c800aff50 msgr2=0x7f2c8015d100 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-09T13:53:40.540 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:40.533+0000 7f2c8848e640 1 -- 172.21.15.43:0/855485905 shutdown_connections 2023-08-09T13:53:40.540 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:40.533+0000 7f2c8848e640 1 -- 172.21.15.43:0/855485905 wait complete. 2023-08-09T13:53:40.556 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-08-09T13:53:40.793 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:40.790+0000 7f468c438640 1 Processor -- start 2023-08-09T13:53:40.793 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:40.791+0000 7f468c438640 1 -- start start 2023-08-09T13:53:40.795 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:40.792+0000 7f468c438640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f4684159470 0x7f46840c9420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:53:40.795 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:40.792+0000 7f468c438640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f46840cc8b0 0x7f46840ccd40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:53:40.795 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:40.792+0000 7f468c438640 1 -- --> v1:172.21.15.203:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f46840c9d40 con 0x7f468415da80 2023-08-09T13:53:40.795 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:40.792+0000 7f468c438640 1 -- --> v2:172.21.15.43:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4684070990 con 0x7f4684159470 2023-08-09T13:53:40.795 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:40.792+0000 7f468c438640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f46840ca020 con 0x7f46840cc8b0 2023-08-09T13:53:40.797 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:40.793+0000 7f468a1d4640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f4684159470 0x7f46840c9420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:53:40.797 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:40.793+0000 7f468a1d4640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f4684159470 0x7f46840c9420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:53504/0 (socket says 172.21.15.43:53504) 2023-08-09T13:53:40.797 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:40.793+0000 7f468a1d4640 1 -- 172.21.15.43:0/2499442296 learned_addr learned my addr 172.21.15.43:0/2499442296 (peer_addr_for_me v2:172.21.15.43:0/0) 2023-08-09T13:53:40.797 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:40.793+0000 7f468a1d4640 1 -- 172.21.15.43:0/2499442296 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f46840cc8b0 msgr2=0x7f46840ccd40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-09T13:53:40.797 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:40.793+0000 7f468a9d5640 1 --2- 172.21.15.43:0/2499442296 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f46840cc8b0 0x7f46840ccd40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:53:40.798 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:40.793+0000 7f468a1d4640 1 --2- 172.21.15.43:0/2499442296 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f46840cc8b0 0x7f46840ccd40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:53:40.798 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:40.793+0000 7f468a1d4640 1 -- 172.21.15.43:0/2499442296 >> v1:172.21.15.203:6789/0 conn(0x7f468415da80 legacy=0x7f4684070db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-09T13:53:40.798 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:40.793+0000 7f468a1d4640 1 -- 172.21.15.43:0/2499442296 --> v2:172.21.15.43:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f46840705b0 con 0x7f4684159470 2023-08-09T13:53:40.798 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:40.793+0000 7f468a9d5640 1 --2- 172.21.15.43:0/2499442296 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f46840cc8b0 0x7f46840ccd40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-08-09T13:53:40.799 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:40.794+0000 7f468a1d4640 1 --2- 172.21.15.43:0/2499442296 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f4684159470 0x7f46840c9420 secure :-1 s=READY pgs=1476 cs=0 l=1 rev1=1 crypto rx=0x7f4680009e50 tx=0x7f468002bee0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c1b66b72d7b9fdc9 server_cookie=0 in_seq=0 out_seq=0 2023-08-09T13:53:40.800 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:40.795+0000 7f46891d2640 1 -- 172.21.15.43:0/2499442296 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f468002ca80 con 0x7f4684159470 2023-08-09T13:53:40.800 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:40.795+0000 7f46891d2640 1 -- 172.21.15.43:0/2499442296 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f468002cbe0 con 0x7f4684159470 2023-08-09T13:53:40.800 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:40.795+0000 7f46891d2640 1 -- 172.21.15.43:0/2499442296 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f46800343f0 con 0x7f4684159470 2023-08-09T13:53:40.800 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:40.796+0000 7f468c438640 1 -- 172.21.15.43:0/2499442296 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f4684159470 msgr2=0x7f46840c9420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:53:40.800 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:40.796+0000 7f468c438640 1 --2- 172.21.15.43:0/2499442296 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f4684159470 0x7f46840c9420 secure :-1 s=READY pgs=1476 cs=0 l=1 rev1=1 crypto rx=0x7f4680009e50 tx=0x7f468002bee0 comp rx=0 tx=0).stop 2023-08-09T13:53:40.800 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:40.796+0000 7f468c438640 1 -- 172.21.15.43:0/2499442296 shutdown_connections 2023-08-09T13:53:40.801 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:40.796+0000 7f468c438640 1 --2- 172.21.15.43:0/2499442296 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f46840cc8b0 0x7f46840ccd40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:53:40.801 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:40.796+0000 7f468c438640 1 --2- 172.21.15.43:0/2499442296 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f4684159470 0x7f46840c9420 unknown :-1 s=CLOSED pgs=1476 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:53:40.801 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:40.796+0000 7f468c438640 1 -- 172.21.15.43:0/2499442296 >> 172.21.15.43:0/2499442296 conn(0x7f46840aff50 msgr2=0x7f46840b0360 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-09T13:53:40.801 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:40.796+0000 7f468c438640 1 -- 172.21.15.43:0/2499442296 shutdown_connections 2023-08-09T13:53:40.801 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:40.796+0000 7f468c438640 1 -- 172.21.15.43:0/2499442296 wait complete. 2023-08-09T13:53:40.803 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:40.798+0000 7f468c438640 1 Processor -- start 2023-08-09T13:53:40.803 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:40.798+0000 7f468c438640 1 -- start start 2023-08-09T13:53:40.803 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:40.799+0000 7f468c438640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f46840cc8b0 0x7f4684166e00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:53:40.803 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:40.799+0000 7f468c438640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f4684159470 0x7f46840c3b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:53:40.803 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:40.799+0000 7f468c438640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f468415da80 0x7f4684149640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:53:40.804 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:40.799+0000 7f468c438640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f46840c4060 con 0x7f468415da80 2023-08-09T13:53:40.804 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:40.799+0000 7f468c438640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4684149310 con 0x7f46840cc8b0 2023-08-09T13:53:40.804 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:40.799+0000 7f468c438640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4684147f60 con 0x7f4684159470 2023-08-09T13:53:40.804 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:40.799+0000 7f468a9d5640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f468415da80 0x7f4684149640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:53:40.804 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:40.799+0000 7f468a9d5640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f468415da80 0x7f4684149640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:53512/0 (socket says 172.21.15.43:53512) 2023-08-09T13:53:40.804 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:40.799+0000 7f468a9d5640 1 -- 172.21.15.43:0/236561749 learned_addr learned my addr 172.21.15.43:0/236561749 (peer_addr_for_me v2:172.21.15.43:0/0) 2023-08-09T13:53:40.804 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:40.800+0000 7f468a1d4640 1 --2- 172.21.15.43:0/236561749 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f46840cc8b0 0x7f4684166e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:53:40.805 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:40.800+0000 7f46899d3640 1 --2- 172.21.15.43:0/236561749 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f4684159470 0x7f46840c3b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:53:40.805 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:40.800+0000 7f468a9d5640 1 -- 172.21.15.43:0/236561749 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f4684159470 msgr2=0x7f46840c3b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:53:40.805 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:40.800+0000 7f468a9d5640 1 --2- 172.21.15.43:0/236561749 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f4684159470 0x7f46840c3b20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:53:40.805 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:40.800+0000 7f468a9d5640 1 -- 172.21.15.43:0/236561749 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f46840cc8b0 msgr2=0x7f4684166e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:53:40.805 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:40.800+0000 7f468a9d5640 1 --2- 172.21.15.43:0/236561749 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f46840cc8b0 0x7f4684166e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:53:40.805 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:40.800+0000 7f468a9d5640 1 -- 172.21.15.43:0/236561749 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4680009b30 con 0x7f468415da80 2023-08-09T13:53:40.805 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:40.800+0000 7f46899d3640 1 --2- 172.21.15.43:0/236561749 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f4684159470 0x7f46840c3b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-08-09T13:53:40.805 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:40.800+0000 7f468a1d4640 1 --2- 172.21.15.43:0/236561749 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f46840cc8b0 0x7f4684166e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-08-09T13:53:40.806 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:40.800+0000 7f468a9d5640 1 --2- 172.21.15.43:0/236561749 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f468415da80 0x7f4684149640 secure :-1 s=READY pgs=1477 cs=0 l=1 rev1=1 crypto rx=0x7f467c00c7f0 tx=0x7f467c00ccb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-09T13:53:40.806 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:40.800+0000 7f467affd640 1 -- 172.21.15.43:0/236561749 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f467c007820 con 0x7f468415da80 2023-08-09T13:53:40.806 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:40.800+0000 7f467affd640 1 -- 172.21.15.43:0/236561749 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f467c007980 con 0x7f468415da80 2023-08-09T13:53:40.806 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:40.800+0000 7f467affd640 1 -- 172.21.15.43:0/236561749 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f467c007d70 con 0x7f468415da80 2023-08-09T13:53:40.806 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:40.800+0000 7f468c438640 1 -- 172.21.15.43:0/236561749 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4684148240 con 0x7f468415da80 2023-08-09T13:53:40.806 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:40.801+0000 7f468c438640 1 -- 172.21.15.43:0/236561749 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4684148800 con 0x7f468415da80 2023-08-09T13:53:40.807 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:40.802+0000 7f468c438640 1 -- 172.21.15.43:0/236561749 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4684153220 con 0x7f468415da80 2023-08-09T13:53:40.807 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:40.803+0000 7f467affd640 1 -- 172.21.15.43:0/236561749 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 56) v1 ==== 88443+0+0 (secure 0 0 0) 0x7f467c0113f0 con 0x7f468415da80 2023-08-09T13:53:40.807 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:40.804+0000 7f467affd640 1 --2- 172.21.15.43:0/236561749 >> [v2:172.21.15.203:6832/2527059680,v1:172.21.15.203:6833/2527059680] conn(0x7f466806a170 0x7f466806c630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:53:40.807 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:40.804+0000 7f467affd640 1 -- 172.21.15.43:0/236561749 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(121..121 src has 1..121) v4 ==== 6434+0+0 (secure 0 0 0) 0x7f467c088d70 con 0x7f468415da80 2023-08-09T13:53:40.808 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:40.805+0000 7f468a1d4640 1 --2- 172.21.15.43:0/236561749 >> [v2:172.21.15.203:6832/2527059680,v1:172.21.15.203:6833/2527059680] conn(0x7f466806a170 0x7f466806c630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:53:40.808 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:40.806+0000 7f468a1d4640 1 --2- 172.21.15.43:0/236561749 >> [v2:172.21.15.203:6832/2527059680,v1:172.21.15.203:6833/2527059680] conn(0x7f466806a170 0x7f466806c630 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7f4680035000 tx=0x7f4680030000 comp rx=0 tx=0).ready entity=mgr.6377 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-09T13:53:40.811 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:40.809+0000 7f467affd640 1 -- 172.21.15.43:0/236561749 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178066 (secure 0 0 0) 0x7f467c054020 con 0x7f468415da80 2023-08-09T13:53:41.001 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:40.999+0000 7f468c438640 1 -- 172.21.15.43:0/236561749 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f46841489f0 con 0x7f468415da80 2023-08-09T13:53:41.002 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:41.000+0000 7f467affd640 1 -- 172.21.15.43:0/236561749 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v121) v1 ==== 93+0+10 (secure 0 0 0) 0x7f467c057640 con 0x7f468415da80 2023-08-09T13:53:41.002 INFO:teuthology.orchestra.run.smithi043.stdout:pg_num: 1 2023-08-09T13:53:41.005 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:41.002+0000 7f468c438640 1 -- 172.21.15.43:0/236561749 >> [v2:172.21.15.203:6832/2527059680,v1:172.21.15.203:6833/2527059680] conn(0x7f466806a170 msgr2=0x7f466806c630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:53:41.005 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:41.002+0000 7f468c438640 1 --2- 172.21.15.43:0/236561749 >> [v2:172.21.15.203:6832/2527059680,v1:172.21.15.203:6833/2527059680] conn(0x7f466806a170 0x7f466806c630 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7f4680035000 tx=0x7f4680030000 comp rx=0 tx=0).stop 2023-08-09T13:53:41.006 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:41.002+0000 7f468c438640 1 -- 172.21.15.43:0/236561749 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f468415da80 msgr2=0x7f4684149640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:53:41.006 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:41.002+0000 7f468c438640 1 --2- 172.21.15.43:0/236561749 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f468415da80 0x7f4684149640 secure :-1 s=READY pgs=1477 cs=0 l=1 rev1=1 crypto rx=0x7f467c00c7f0 tx=0x7f467c00ccb0 comp rx=0 tx=0).stop 2023-08-09T13:53:41.006 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:41.003+0000 7f468c438640 1 -- 172.21.15.43:0/236561749 shutdown_connections 2023-08-09T13:53:41.006 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:41.003+0000 7f468c438640 1 --2- 172.21.15.43:0/236561749 >> [v2:172.21.15.203:6832/2527059680,v1:172.21.15.203:6833/2527059680] conn(0x7f466806a170 0x7f466806c630 unknown :-1 s=CLOSED pgs=42 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:53:41.006 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:41.003+0000 7f468c438640 1 --2- 172.21.15.43:0/236561749 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f468415da80 0x7f4684149640 unknown :-1 s=CLOSED pgs=1477 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:53:41.006 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:41.003+0000 7f468c438640 1 --2- 172.21.15.43:0/236561749 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f4684159470 0x7f46840c3b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:53:41.006 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:41.003+0000 7f468c438640 1 --2- 172.21.15.43:0/236561749 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f46840cc8b0 0x7f4684166e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:53:41.007 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:41.003+0000 7f468c438640 1 -- 172.21.15.43:0/236561749 >> 172.21.15.43:0/236561749 conn(0x7f46840aff50 msgr2=0x7f46840ad9b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-09T13:53:41.007 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:41.003+0000 7f468c438640 1 -- 172.21.15.43:0/236561749 shutdown_connections 2023-08-09T13:53:41.007 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:41.003+0000 7f468c438640 1 -- 172.21.15.43:0/236561749 wait complete. 2023-08-09T13:53:41.024 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mon mon_allow_pool_delete True 2023-08-09T13:53:41.252 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:41.250+0000 7faa7f59e640 1 Processor -- start 2023-08-09T13:53:41.252 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:41.250+0000 7faa7f59e640 1 -- start start 2023-08-09T13:53:41.254 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:41.251+0000 7faa7f59e640 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7faa800c96a0 0x7faa800cafa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:53:41.254 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:41.251+0000 7faa7f59e640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7faa8006fd40 0x7faa800cb9f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:53:41.254 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:41.252+0000 7faa7f59e640 1 -- --> v1:172.21.15.203:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7faa800b07e0 con 0x7faa801590b0 2023-08-09T13:53:41.255 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:41.252+0000 7faa7f59e640 1 -- --> v2:172.21.15.203:3300/0 -- mon_getmap magic: 0 v1 -- 0x7faa80070170 con 0x7faa800c96a0 2023-08-09T13:53:41.255 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:41.252+0000 7faa7f59e640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7faa80070420 con 0x7faa8006fd40 2023-08-09T13:53:41.256 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:41.252+0000 7faa7ed9d640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7faa8006fd40 0x7faa800cb9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:53:41.256 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:41.252+0000 7faa7ed9d640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7faa8006fd40 0x7faa800cb9f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.203:3301/0 says I am v2:172.21.15.43:43610/0 (socket says 172.21.15.43:43610) 2023-08-09T13:53:41.256 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:41.252+0000 7faa7ed9d640 1 -- 172.21.15.43:0/853219546 learned_addr learned my addr 172.21.15.43:0/853219546 (peer_addr_for_me v2:172.21.15.43:0/0) 2023-08-09T13:53:41.256 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:41.253+0000 7faa7dd9b640 1 --2- 172.21.15.43:0/853219546 >> v2:172.21.15.203:3300/0 conn(0x7faa800c96a0 0x7faa800cafa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:53:41.256 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:41.253+0000 7faa7ed9d640 1 -- 172.21.15.43:0/853219546 >> v2:172.21.15.203:3300/0 conn(0x7faa800c96a0 msgr2=0x7faa800cafa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-09T13:53:41.256 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:41.253+0000 7faa7ed9d640 1 --2- 172.21.15.43:0/853219546 >> v2:172.21.15.203:3300/0 conn(0x7faa800c96a0 0x7faa800cafa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:53:41.256 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:41.253+0000 7faa7ed9d640 1 -- 172.21.15.43:0/853219546 >> v1:172.21.15.203:6789/0 conn(0x7faa801590b0 legacy=0x7faa800c8f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:53:41.257 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:41.253+0000 7faa7ed9d640 1 -- 172.21.15.43:0/853219546 --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faa80070700 con 0x7faa8006fd40 2023-08-09T13:53:41.257 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:41.255+0000 7faa7ed9d640 1 --2- 172.21.15.43:0/853219546 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7faa8006fd40 0x7faa800cb9f0 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7faa78009870 tx=0x7faa7802b980 comp rx=0 tx=0).ready entity=mon.2 client_cookie=aef2bf6b4a221838 server_cookie=0 in_seq=0 out_seq=0 2023-08-09T13:53:41.258 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:41.255+0000 7faa7d59a640 1 -- 172.21.15.43:0/853219546 <== mon.2 v2:172.21.15.203:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faa78030840 con 0x7faa8006fd40 2023-08-09T13:53:41.258 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:41.256+0000 7faa7d59a640 1 -- 172.21.15.43:0/853219546 <== mon.2 v2:172.21.15.203:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7faa780309a0 con 0x7faa8006fd40 2023-08-09T13:53:41.258 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:41.256+0000 7faa7d59a640 1 -- 172.21.15.43:0/853219546 <== mon.2 v2:172.21.15.203:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faa78030d30 con 0x7faa8006fd40 2023-08-09T13:53:41.259 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:41.256+0000 7faa7f59e640 1 -- 172.21.15.43:0/853219546 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7faa8006fd40 msgr2=0x7faa800cb9f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:53:41.259 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:41.256+0000 7faa7f59e640 1 --2- 172.21.15.43:0/853219546 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7faa8006fd40 0x7faa800cb9f0 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7faa78009870 tx=0x7faa7802b980 comp rx=0 tx=0).stop 2023-08-09T13:53:41.259 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:41.256+0000 7faa7f59e640 1 -- 172.21.15.43:0/853219546 shutdown_connections 2023-08-09T13:53:41.259 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:41.256+0000 7faa7f59e640 1 --2- 172.21.15.43:0/853219546 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7faa8006fd40 0x7faa800cb9f0 unknown :-1 s=CLOSED pgs=199 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:53:41.259 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:41.256+0000 7faa7f59e640 1 --2- 172.21.15.43:0/853219546 >> v2:172.21.15.203:3300/0 conn(0x7faa800c96a0 0x7faa800cafa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:53:41.260 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:41.256+0000 7faa7f59e640 1 -- 172.21.15.43:0/853219546 >> 172.21.15.43:0/853219546 conn(0x7faa800af9f0 msgr2=0x7faa800afdd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-09T13:53:41.260 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:41.256+0000 7faa7f59e640 1 -- 172.21.15.43:0/853219546 shutdown_connections 2023-08-09T13:53:41.260 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:41.257+0000 7faa7f59e640 1 -- 172.21.15.43:0/853219546 wait complete. 2023-08-09T13:53:41.261 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:41.259+0000 7faa7f59e640 1 Processor -- start 2023-08-09T13:53:41.262 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:41.259+0000 7faa7f59e640 1 -- start start 2023-08-09T13:53:41.263 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:41.260+0000 7faa7f59e640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7faa8006fd40 0x7faa800c7330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:53:41.263 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:41.260+0000 7faa7f59e640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7faa800c96a0 0x7faa800b0ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:53:41.263 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:41.260+0000 7faa7f59e640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7faa801590b0 0x7faa800acaa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:53:41.264 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:41.261+0000 7faa7f59e640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faa800c7870 con 0x7faa801590b0 2023-08-09T13:53:41.264 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:41.261+0000 7faa7f59e640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faa800b3150 con 0x7faa8006fd40 2023-08-09T13:53:41.264 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:41.261+0000 7faa7f59e640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7faa800c5350 con 0x7faa800c96a0 2023-08-09T13:53:41.264 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:41.261+0000 7faa7ed9d640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7faa801590b0 0x7faa800acaa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:53:41.264 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:41.261+0000 7faa7ed9d640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7faa801590b0 0x7faa800acaa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:53522/0 (socket says 172.21.15.43:53522) 2023-08-09T13:53:41.264 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:41.261+0000 7faa7ed9d640 1 -- 172.21.15.43:0/291114583 learned_addr learned my addr 172.21.15.43:0/291114583 (peer_addr_for_me v2:172.21.15.43:0/0) 2023-08-09T13:53:41.265 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:41.261+0000 7faa7e59c640 1 --2- 172.21.15.43:0/291114583 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7faa8006fd40 0x7faa800c7330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:53:41.266 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:41.261+0000 7faa7dd9b640 1 --2- 172.21.15.43:0/291114583 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7faa800c96a0 0x7faa800b0ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:53:41.266 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:41.261+0000 7faa7ed9d640 1 -- 172.21.15.43:0/291114583 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7faa800c96a0 msgr2=0x7faa800b0ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:53:41.266 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:41.261+0000 7faa7ed9d640 1 --2- 172.21.15.43:0/291114583 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7faa800c96a0 0x7faa800b0ae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:53:41.266 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:41.261+0000 7faa7ed9d640 1 -- 172.21.15.43:0/291114583 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7faa8006fd40 msgr2=0x7faa800c7330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:53:41.266 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:41.261+0000 7faa7ed9d640 1 --2- 172.21.15.43:0/291114583 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7faa8006fd40 0x7faa800c7330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:53:41.266 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:41.261+0000 7faa7ed9d640 1 -- 172.21.15.43:0/291114583 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faa780094d0 con 0x7faa801590b0 2023-08-09T13:53:41.266 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:41.262+0000 7faa7ed9d640 1 --2- 172.21.15.43:0/291114583 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7faa801590b0 0x7faa800acaa0 secure :-1 s=READY pgs=1478 cs=0 l=1 rev1=1 crypto rx=0x7faa78009870 tx=0x7faa78004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-09T13:53:41.267 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:41.262+0000 7faa6effd640 1 -- 172.21.15.43:0/291114583 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faa78039c90 con 0x7faa801590b0 2023-08-09T13:53:41.267 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:41.262+0000 7faa6effd640 1 -- 172.21.15.43:0/291114583 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7faa78039e10 con 0x7faa801590b0 2023-08-09T13:53:41.267 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:41.262+0000 7faa6effd640 1 -- 172.21.15.43:0/291114583 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faa78043640 con 0x7faa801590b0 2023-08-09T13:53:41.267 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:41.262+0000 7faa7f59e640 1 -- 172.21.15.43:0/291114583 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7faa800b3360 con 0x7faa801590b0 2023-08-09T13:53:41.267 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:41.262+0000 7faa7f59e640 1 -- 172.21.15.43:0/291114583 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7faa800b3920 con 0x7faa801590b0 2023-08-09T13:53:41.268 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:41.264+0000 7faa7f59e640 1 -- 172.21.15.43:0/291114583 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7faa800c54c0 con 0x7faa801590b0 2023-08-09T13:53:41.268 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:41.264+0000 7faa6effd640 1 -- 172.21.15.43:0/291114583 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 56) v1 ==== 88443+0+0 (secure 0 0 0) 0x7faa780437a0 con 0x7faa801590b0 2023-08-09T13:53:41.268 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:41.265+0000 7faa6effd640 1 --2- 172.21.15.43:0/291114583 >> [v2:172.21.15.203:6832/2527059680,v1:172.21.15.203:6833/2527059680] conn(0x7faa6006a170 0x7faa6006c630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:53:41.268 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:41.265+0000 7faa6effd640 1 -- 172.21.15.43:0/291114583 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(121..121 src has 1..121) v4 ==== 6434+0+0 (secure 0 0 0) 0x7faa7803e070 con 0x7faa801590b0 2023-08-09T13:53:41.269 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:41.266+0000 7faa7e59c640 1 --2- 172.21.15.43:0/291114583 >> [v2:172.21.15.203:6832/2527059680,v1:172.21.15.203:6833/2527059680] conn(0x7faa6006a170 0x7faa6006c630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:53:41.269 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:41.267+0000 7faa7e59c640 1 --2- 172.21.15.43:0/291114583 >> [v2:172.21.15.203:6832/2527059680,v1:172.21.15.203:6833/2527059680] conn(0x7faa6006a170 0x7faa6006c630 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7faa700012e0 tx=0x7faa70009040 comp rx=0 tx=0).ready entity=mgr.6377 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-09T13:53:41.272 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:41.270+0000 7faa6effd640 1 -- 172.21.15.43:0/291114583 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178066 (secure 0 0 0) 0x7faa800c54c0 con 0x7faa801590b0 2023-08-09T13:53:41.464 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:41.461+0000 7faa7f59e640 1 -- 172.21.15.43:0/291114583 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command([{prefix=config set, name=mon_allow_pool_delete}] v 0) v1 -- 0x7faa800c54c0 con 0x7faa801590b0 2023-08-09T13:53:41.469 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:41.467+0000 7faa6effd640 1 -- 172.21.15.43:0/291114583 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_allow_pool_delete}]=0 v40)=0 v40) v1 ==== 122+0+0 (secure 0 0 0) 0x7faa800c54c0 con 0x7faa801590b0 2023-08-09T13:53:41.472 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:41.469+0000 7faa7f59e640 1 -- 172.21.15.43:0/291114583 >> [v2:172.21.15.203:6832/2527059680,v1:172.21.15.203:6833/2527059680] conn(0x7faa6006a170 msgr2=0x7faa6006c630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:53:41.472 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:41.469+0000 7faa7f59e640 1 --2- 172.21.15.43:0/291114583 >> [v2:172.21.15.203:6832/2527059680,v1:172.21.15.203:6833/2527059680] conn(0x7faa6006a170 0x7faa6006c630 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7faa700012e0 tx=0x7faa70009040 comp rx=0 tx=0).stop 2023-08-09T13:53:41.472 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:41.469+0000 7faa7f59e640 1 -- 172.21.15.43:0/291114583 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7faa801590b0 msgr2=0x7faa800acaa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:53:41.473 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:41.469+0000 7faa7f59e640 1 --2- 172.21.15.43:0/291114583 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7faa801590b0 0x7faa800acaa0 secure :-1 s=READY pgs=1478 cs=0 l=1 rev1=1 crypto rx=0x7faa78009870 tx=0x7faa78004730 comp rx=0 tx=0).stop 2023-08-09T13:53:41.473 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:41.469+0000 7faa7f59e640 1 -- 172.21.15.43:0/291114583 shutdown_connections 2023-08-09T13:53:41.473 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:41.469+0000 7faa7f59e640 1 --2- 172.21.15.43:0/291114583 >> [v2:172.21.15.203:6832/2527059680,v1:172.21.15.203:6833/2527059680] conn(0x7faa6006a170 0x7faa6006c630 unknown :-1 s=CLOSED pgs=43 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:53:41.473 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:41.469+0000 7faa7f59e640 1 --2- 172.21.15.43:0/291114583 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7faa801590b0 0x7faa800acaa0 unknown :-1 s=CLOSED pgs=1478 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:53:41.473 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:41.469+0000 7faa7f59e640 1 --2- 172.21.15.43:0/291114583 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7faa800c96a0 0x7faa800b0ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:53:41.473 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:41.469+0000 7faa7f59e640 1 --2- 172.21.15.43:0/291114583 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7faa8006fd40 0x7faa800c7330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:53:41.473 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:41.470+0000 7faa7f59e640 1 -- 172.21.15.43:0/291114583 >> 172.21.15.43:0/291114583 conn(0x7faa800af9f0 msgr2=0x7faa800ce770 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-09T13:53:41.473 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:41.470+0000 7faa7f59e640 1 -- 172.21.15.43:0/291114583 shutdown_connections 2023-08-09T13:53:41.474 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:41.470+0000 7faa7f59e640 1 -- 172.21.15.43:0/291114583 wait complete. 2023-08-09T13:53:41.492 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2023-08-09T13:53:41.492 DEBUG:teuthology.orchestra.run.smithi043:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2023-08-09T13:53:41.608 DEBUG:teuthology.orchestra.run.smithi203:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2023-08-09T13:53:41.734 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2023-08-09T13:53:41.734 DEBUG:teuthology.orchestra.run.smithi043:> set -ex 2023-08-09T13:53:41.735 DEBUG:teuthology.orchestra.run.smithi043:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-09T13:53:41.799 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4054441,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6498844k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=27,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3249420k,nr_inodes=812355,mode=700,uid=1000,gid=1275 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6498844k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2023-08-09T13:53:41.800 DEBUG:teuthology.orchestra.run.smithi203:> set -ex 2023-08-09T13:53:41.800 DEBUG:teuthology.orchestra.run.smithi203:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-09T13:53:41.860 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4053648,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497576k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=27,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248784k,nr_inodes=812196,mode=700,uid=1000,gid=1275 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497576k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2023-08-09T13:53:41.860 DEBUG:teuthology.orchestra.run.smithi043:> set -ex 2023-08-09T13:53:41.861 DEBUG:teuthology.orchestra.run.smithi043:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-09T13:53:41.923 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4054441,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6498844k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=27,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3249420k,nr_inodes=812355,mode=700,uid=1000,gid=1275 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6498844k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2023-08-09T13:53:41.923 DEBUG:teuthology.orchestra.run.smithi203:> set -ex 2023-08-09T13:53:41.923 DEBUG:teuthology.orchestra.run.smithi203:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-09T13:53:41.983 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4053648,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497576k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=27,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248784k,nr_inodes=812196,mode=700,uid=1000,gid=1275 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497576k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2023-08-09T13:53:41.984 INFO:tasks.cephfs.cephfs_test_case:[, , , ] 2023-08-09T13:53:41.984 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-08-09T13:53:42.226 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:42.223+0000 7f5e8e499640 1 Processor -- start 2023-08-09T13:53:42.226 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:42.223+0000 7f5e8e499640 1 -- start start 2023-08-09T13:53:42.227 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:42.225+0000 7f5e8e499640 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7f5e8815c630 0x7f5e880abef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:53:42.228 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:42.225+0000 7f5e8e499640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f5e880cdc60 0x7f5e8815ca10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:53:42.228 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:42.225+0000 7f5e8e499640 1 -- --> v1:172.21.15.43:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5e880cca90 con 0x7f5e880ccf20 2023-08-09T13:53:42.228 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:42.225+0000 7f5e8e499640 1 -- --> v2:172.21.15.203:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5e880cbc70 con 0x7f5e8815c630 2023-08-09T13:53:42.228 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:42.225+0000 7f5e8e499640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5e880cbe80 con 0x7f5e880cdc60 2023-08-09T13:53:42.229 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:42.225+0000 7f5e7f7fe640 1 --1- >> v1:172.21.15.43:6789/0 conn(0x7f5e880ccf20 0x7f5e880cd550 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.43:6789/0 says I am v1:172.21.15.43:44786/0 (socket says 172.21.15.43:44786) 2023-08-09T13:53:42.229 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:42.226+0000 7f5e7f7fe640 1 -- 172.21.15.43:0/1139084382 learned_addr learned my addr 172.21.15.43:0/1139084382 (peer_addr_for_me v1:172.21.15.43:0/0) 2023-08-09T13:53:42.229 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:42.226+0000 7f5e8ca36640 1 --2- 172.21.15.43:0/1139084382 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f5e880cdc60 0x7f5e8815ca10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:53:42.229 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:42.226+0000 7f5e877fe640 1 -- 172.21.15.43:0/1139084382 <== mon.0 v1:172.21.15.43:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 175040842 0 0) 0x7f5e880cca90 con 0x7f5e880ccf20 2023-08-09T13:53:42.230 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:42.226+0000 7f5e87fff640 1 --2- 172.21.15.43:0/1139084382 >> v2:172.21.15.203:3300/0 conn(0x7f5e8815c630 0x7f5e880abef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:53:42.230 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:42.226+0000 7f5e877fe640 1 -- 172.21.15.43:0/1139084382 --> v1:172.21.15.43:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f5e74003520 con 0x7f5e880ccf20 2023-08-09T13:53:42.230 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:42.226+0000 7f5e877fe640 1 -- 172.21.15.43:0/1139084382 <== mon.0 v1:172.21.15.43:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2505733562 0 0) 0x7f5e74003520 con 0x7f5e880ccf20 2023-08-09T13:53:42.230 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:42.226+0000 7f5e877fe640 1 -- 172.21.15.43:0/1139084382 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f5e880cdc60 msgr2=0x7f5e8815ca10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-09T13:53:42.230 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:42.226+0000 7f5e877fe640 1 --2- 172.21.15.43:0/1139084382 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f5e880cdc60 0x7f5e8815ca10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:53:42.230 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:42.226+0000 7f5e877fe640 1 -- 172.21.15.43:0/1139084382 >> v2:172.21.15.203:3300/0 conn(0x7f5e8815c630 msgr2=0x7f5e880abef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-09T13:53:42.230 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:42.226+0000 7f5e877fe640 1 --2- 172.21.15.43:0/1139084382 >> v2:172.21.15.203:3300/0 conn(0x7f5e8815c630 0x7f5e880abef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:53:42.231 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:42.226+0000 7f5e877fe640 1 -- 172.21.15.43:0/1139084382 --> v1:172.21.15.43:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5e880cc160 con 0x7f5e880ccf20 2023-08-09T13:53:42.231 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:42.227+0000 7f5e877fe640 1 -- 172.21.15.43:0/1139084382 <== mon.0 v1:172.21.15.43:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1285150740 0 0) 0x7f5e70001670 con 0x7f5e880ccf20 2023-08-09T13:53:42.231 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:42.227+0000 7f5e877fe640 1 -- 172.21.15.43:0/1139084382 >> v1:172.21.15.43:6789/0 conn(0x7f5e880ccf20 legacy=0x7f5e880cd550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:53:42.231 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:42.227+0000 7f5e877fe640 1 --2- 172.21.15.43:0/1139084382 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f5e740039e0 0x7f5e74003e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:53:42.231 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:42.227+0000 7f5e877fe640 1 --2- 172.21.15.43:0/1139084382 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f5e740047c0 0x7f5e74004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:53:42.231 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:42.227+0000 7f5e877fe640 1 --2- 172.21.15.43:0/1139084382 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f5e880cdc60 0x7f5e74005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:53:42.231 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:42.227+0000 7f5e877fe640 1 -- 172.21.15.43:0/1139084382 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5e880cbc70 con 0x7f5e740039e0 2023-08-09T13:53:42.231 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:42.227+0000 7f5e877fe640 1 -- 172.21.15.43:0/1139084382 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5e880cbe80 con 0x7f5e740047c0 2023-08-09T13:53:42.232 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:42.227+0000 7f5e877fe640 1 -- 172.21.15.43:0/1139084382 --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5e7400ba30 con 0x7f5e880cdc60 2023-08-09T13:53:42.232 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:42.227+0000 7f5e87fff640 1 --2- 172.21.15.43:0/1139084382 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f5e740039e0 0x7f5e74003e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:53:42.233 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:42.227+0000 7f5e7f7fe640 1 --2- 172.21.15.43:0/1139084382 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f5e740047c0 0x7f5e74004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:53:42.233 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:42.228+0000 7f5e8ca36640 1 --2- 172.21.15.43:0/1139084382 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f5e880cdc60 0x7f5e74005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:53:42.233 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:42.228+0000 7f5e87fff640 1 -- 172.21.15.43:0/1139084382 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f5e880cdc60 msgr2=0x7f5e74005180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-09T13:53:42.233 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:42.228+0000 7f5e87fff640 1 --2- 172.21.15.43:0/1139084382 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f5e880cdc60 0x7f5e74005180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:53:42.233 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:42.228+0000 7f5e87fff640 1 -- 172.21.15.43:0/1139084382 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f5e740047c0 msgr2=0x7f5e74004c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-09T13:53:42.233 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:42.228+0000 7f5e87fff640 1 --2- 172.21.15.43:0/1139084382 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f5e740047c0 0x7f5e74004c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:53:42.234 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:42.228+0000 7f5e87fff640 1 -- 172.21.15.43:0/1139084382 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5e74003520 con 0x7f5e740039e0 2023-08-09T13:53:42.234 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:42.230+0000 7f5e87fff640 1 --2- 172.21.15.43:0/1139084382 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f5e740039e0 0x7f5e74003e00 secure :-1 s=READY pgs=1480 cs=0 l=1 rev1=1 crypto rx=0x7f5e7800ba30 tx=0x7f5e740057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=46f8f27c680d1ce3 server_cookie=0 in_seq=0 out_seq=0 2023-08-09T13:53:42.234 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:42.230+0000 7f5e877fe640 1 -- 172.21.15.43:0/1139084382 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5e78032680 con 0x7f5e740039e0 2023-08-09T13:53:42.234 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:42.230+0000 7f5e877fe640 1 -- 172.21.15.43:0/1139084382 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5e780327e0 con 0x7f5e740039e0 2023-08-09T13:53:42.234 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:42.230+0000 7f5e877fe640 1 -- 172.21.15.43:0/1139084382 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5e78032b70 con 0x7f5e740039e0 2023-08-09T13:53:42.235 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:42.231+0000 7f5e8e499640 1 -- 172.21.15.43:0/1139084382 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f5e740039e0 msgr2=0x7f5e74003e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:53:42.235 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:42.231+0000 7f5e8e499640 1 --2- 172.21.15.43:0/1139084382 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f5e740039e0 0x7f5e74003e00 secure :-1 s=READY pgs=1480 cs=0 l=1 rev1=1 crypto rx=0x7f5e7800ba30 tx=0x7f5e740057f0 comp rx=0 tx=0).stop 2023-08-09T13:53:42.235 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:42.231+0000 7f5e8ca36640 1 -- 172.21.15.43:0/1139084382 reap_dead start 2023-08-09T13:53:42.235 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:42.231+0000 7f5e8e499640 1 -- 172.21.15.43:0/1139084382 shutdown_connections 2023-08-09T13:53:42.235 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:42.231+0000 7f5e8e499640 1 -- 172.21.15.43:0/1139084382 >> 172.21.15.43:0/1139084382 conn(0x7f5e880ce0a0 msgr2=0x7f5e880ce480 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-09T13:53:42.235 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:42.231+0000 7f5e8e499640 1 -- 172.21.15.43:0/1139084382 shutdown_connections 2023-08-09T13:53:42.236 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:42.231+0000 7f5e8e499640 1 -- 172.21.15.43:0/1139084382 wait complete. 2023-08-09T13:53:42.236 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:42.233+0000 7f5e8e499640 1 Processor -- start 2023-08-09T13:53:42.236 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:42.233+0000 7f5e8e499640 1 -- start start 2023-08-09T13:53:42.237 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:42.234+0000 7f5e8e499640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f5e881507a0 0x7f5e880ad980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:53:42.237 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:42.234+0000 7f5e8e499640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f5e8815b180 0x7f5e8815dd20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:53:42.237 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:42.234+0000 7f5e8e499640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f5e88150b80 0x7f5e88070490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:53:42.237 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:42.234+0000 7f5e8e499640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5e880c63e0 con 0x7f5e8815b180 2023-08-09T13:53:42.238 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:42.234+0000 7f5e8e499640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5e8815e670 con 0x7f5e88150b80 2023-08-09T13:53:42.238 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:42.234+0000 7f5e8e499640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5e8815e950 con 0x7f5e881507a0 2023-08-09T13:53:42.238 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:42.235+0000 7f5e7f7fe640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f5e8815b180 0x7f5e8815dd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:53:42.238 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:42.235+0000 7f5e7f7fe640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f5e8815b180 0x7f5e8815dd20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:53532/0 (socket says 172.21.15.43:53532) 2023-08-09T13:53:42.238 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:42.235+0000 7f5e7f7fe640 1 -- 172.21.15.43:0/1391821506 learned_addr learned my addr 172.21.15.43:0/1391821506 (peer_addr_for_me v2:172.21.15.43:0/0) 2023-08-09T13:53:42.239 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:42.235+0000 7f5e7f7fe640 1 -- 172.21.15.43:0/1391821506 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f5e881507a0 msgr2=0x7f5e880ad980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:53:42.239 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:42.235+0000 7f5e87fff640 1 --2- 172.21.15.43:0/1391821506 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f5e881507a0 0x7f5e880ad980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:53:42.239 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:42.235+0000 7f5e7f7fe640 1 --2- 172.21.15.43:0/1391821506 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f5e881507a0 0x7f5e880ad980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:53:42.239 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:42.235+0000 7f5e8ca36640 1 --2- 172.21.15.43:0/1391821506 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f5e88150b80 0x7f5e88070490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:53:42.240 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:42.235+0000 7f5e7f7fe640 1 -- 172.21.15.43:0/1391821506 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f5e88150b80 msgr2=0x7f5e88070490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:53:42.240 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:42.235+0000 7f5e7f7fe640 1 --2- 172.21.15.43:0/1391821506 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f5e88150b80 0x7f5e88070490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:53:42.240 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:42.235+0000 7f5e7f7fe640 1 -- 172.21.15.43:0/1391821506 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5e8815ec30 con 0x7f5e8815b180 2023-08-09T13:53:42.240 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:42.236+0000 7f5e7f7fe640 1 --2- 172.21.15.43:0/1391821506 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f5e8815b180 0x7f5e8815dd20 secure :-1 s=READY pgs=1481 cs=0 l=1 rev1=1 crypto rx=0x7f5e7000e780 tx=0x7f5e7000ec40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-09T13:53:42.240 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:42.236+0000 7f5e857fa640 1 -- 172.21.15.43:0/1391821506 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5e7000c6c0 con 0x7f5e8815b180 2023-08-09T13:53:42.240 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:42.236+0000 7f5e857fa640 1 -- 172.21.15.43:0/1391821506 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5e7000c820 con 0x7f5e8815b180 2023-08-09T13:53:42.240 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:42.236+0000 7f5e8e499640 1 -- 172.21.15.43:0/1391821506 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5e8815ef10 con 0x7f5e8815b180 2023-08-09T13:53:42.241 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:42.236+0000 7f5e857fa640 1 -- 172.21.15.43:0/1391821506 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5e7000cbd0 con 0x7f5e8815b180 2023-08-09T13:53:42.241 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:42.236+0000 7f5e8e499640 1 -- 172.21.15.43:0/1391821506 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5e8815e440 con 0x7f5e8815b180 2023-08-09T13:53:42.241 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:42.237+0000 7f5e8e499640 1 -- 172.21.15.43:0/1391821506 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5e880c6520 con 0x7f5e8815b180 2023-08-09T13:53:42.241 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:42.238+0000 7f5e857fa640 1 -- 172.21.15.43:0/1391821506 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 56) v1 ==== 88443+0+0 (secure 0 0 0) 0x7f5e7000cd30 con 0x7f5e8815b180 2023-08-09T13:53:42.242 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:42.239+0000 7f5e857fa640 1 --2- 172.21.15.43:0/1391821506 >> [v2:172.21.15.203:6832/2527059680,v1:172.21.15.203:6833/2527059680] conn(0x7f5e6806a170 0x7f5e6806c630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:53:42.242 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:42.239+0000 7f5e857fa640 1 -- 172.21.15.43:0/1391821506 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(121..121 src has 1..121) v4 ==== 6434+0+0 (secure 0 0 0) 0x7f5e700887f0 con 0x7f5e8815b180 2023-08-09T13:53:42.242 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:42.240+0000 7f5e87fff640 1 --2- 172.21.15.43:0/1391821506 >> [v2:172.21.15.203:6832/2527059680,v1:172.21.15.203:6833/2527059680] conn(0x7f5e6806a170 0x7f5e6806c630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:53:42.243 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:42.241+0000 7f5e87fff640 1 --2- 172.21.15.43:0/1391821506 >> [v2:172.21.15.203:6832/2527059680,v1:172.21.15.203:6833/2527059680] conn(0x7f5e6806a170 0x7f5e6806c630 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7f5e7800ba00 tx=0x7f5e78030000 comp rx=0 tx=0).ready entity=mgr.6377 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-09T13:53:42.246 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:42.244+0000 7f5e857fa640 1 -- 172.21.15.43:0/1391821506 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178066 (secure 0 0 0) 0x7f5e880c6520 con 0x7f5e8815b180 2023-08-09T13:53:42.477 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:42.475+0000 7f5e8e499640 1 -- 172.21.15.43:0/1391821506 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f5e880c6520 con 0x7f5e8815b180 2023-08-09T13:53:42.478 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:42.476+0000 7f5e857fa640 1 -- 172.21.15.43:0/1391821506 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 54 v54) v1 ==== 94+0+3705 (secure 0 0 0) 0x7f5e880c6520 con 0x7f5e8815b180 2023-08-09T13:53:42.479 INFO:teuthology.orchestra.run.smithi043.stdout: 2023-08-09T13:53:42.479 INFO:teuthology.orchestra.run.smithi043.stdout:{"epoch":54,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6215,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.43:6835/2214346432","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6833","nonce":2214346432},{"type":"v1","addr":"172.21.15.43:6835","nonce":2214346432}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":47},{"gid":6823,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.203:6835/3236264045","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6834","nonce":3236264045},{"type":"v1","addr":"172.21.15.203:6835","nonce":3236264045}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":46},{"gid":6826,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.203:6837/3788289156","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6836","nonce":3788289156},{"type":"v1","addr":"172.21.15.203:6837","nonce":3788289156}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":47},{"gid":8481,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.43:6834/2331166646","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6832","nonce":2331166646},{"type":"v1","addr":"172.21.15.43:6834","nonce":2331166646}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":54}],"filesystems":[]} 2023-08-09T13:53:42.479 INFO:teuthology.orchestra.run.smithi043.stderr:dumped fsmap epoch 54 2023-08-09T13:53:42.481 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:42.478+0000 7f5e8e499640 1 -- 172.21.15.43:0/1391821506 >> [v2:172.21.15.203:6832/2527059680,v1:172.21.15.203:6833/2527059680] conn(0x7f5e6806a170 msgr2=0x7f5e6806c630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:53:42.481 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:42.478+0000 7f5e8e499640 1 --2- 172.21.15.43:0/1391821506 >> [v2:172.21.15.203:6832/2527059680,v1:172.21.15.203:6833/2527059680] conn(0x7f5e6806a170 0x7f5e6806c630 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7f5e7800ba00 tx=0x7f5e78030000 comp rx=0 tx=0).stop 2023-08-09T13:53:42.481 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:42.478+0000 7f5e8e499640 1 -- 172.21.15.43:0/1391821506 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f5e8815b180 msgr2=0x7f5e8815dd20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:53:42.481 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:42.478+0000 7f5e8e499640 1 --2- 172.21.15.43:0/1391821506 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f5e8815b180 0x7f5e8815dd20 secure :-1 s=READY pgs=1481 cs=0 l=1 rev1=1 crypto rx=0x7f5e7000e780 tx=0x7f5e7000ec40 comp rx=0 tx=0).stop 2023-08-09T13:53:42.482 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:42.479+0000 7f5e8e499640 1 -- 172.21.15.43:0/1391821506 shutdown_connections 2023-08-09T13:53:42.482 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:42.479+0000 7f5e8e499640 1 --2- 172.21.15.43:0/1391821506 >> [v2:172.21.15.203:6832/2527059680,v1:172.21.15.203:6833/2527059680] conn(0x7f5e6806a170 0x7f5e6806c630 unknown :-1 s=CLOSED pgs=44 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:53:42.482 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:42.479+0000 7f5e8e499640 1 --2- 172.21.15.43:0/1391821506 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f5e88150b80 0x7f5e88070490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:53:42.482 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:42.479+0000 7f5e8e499640 1 --2- 172.21.15.43:0/1391821506 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f5e8815b180 0x7f5e8815dd20 unknown :-1 s=CLOSED pgs=1481 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:53:42.482 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:42.479+0000 7f5e8e499640 1 --2- 172.21.15.43:0/1391821506 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f5e881507a0 0x7f5e880ad980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:53:42.482 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:42.479+0000 7f5e8e499640 1 -- 172.21.15.43:0/1391821506 >> 172.21.15.43:0/1391821506 conn(0x7f5e880ce0a0 msgr2=0x7f5e8815cb60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-09T13:53:42.483 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:42.479+0000 7f5e8e499640 1 -- 172.21.15.43:0/1391821506 shutdown_connections 2023-08-09T13:53:42.483 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:42.479+0000 7f5e8e499640 1 -- 172.21.15.43:0/1391821506 wait complete. 2023-08-09T13:53:42.502 INFO:tasks.ceph.mds.a:Restarting daemon 2023-08-09T13:53:42.502 INFO:tasks.ceph.mds.a:Stopping old one... 2023-08-09T13:53:42.502 DEBUG:tasks.ceph.mds.a:waiting for process to exit 2023-08-09T13:53:42.502 INFO:teuthology.orchestra.run:waiting for 300 2023-08-09T13:53:42.503 INFO:tasks.ceph.mds.c:Restarting daemon 2023-08-09T13:53:42.503 INFO:tasks.ceph.mds.c:Stopping old one... 2023-08-09T13:53:42.503 DEBUG:tasks.ceph.mds.c:waiting for process to exit 2023-08-09T13:53:42.503 INFO:teuthology.orchestra.run:waiting for 300 2023-08-09T13:53:42.504 INFO:tasks.ceph.mds.b:Restarting daemon 2023-08-09T13:53:42.504 INFO:tasks.ceph.mds.b:Stopping old one... 2023-08-09T13:53:42.504 DEBUG:tasks.ceph.mds.b:waiting for process to exit 2023-08-09T13:53:42.504 INFO:teuthology.orchestra.run:waiting for 300 2023-08-09T13:53:42.505 INFO:tasks.ceph.mds.d:Restarting daemon 2023-08-09T13:53:42.505 INFO:tasks.ceph.mds.d:Stopping old one... 2023-08-09T13:53:42.505 DEBUG:tasks.ceph.mds.d:waiting for process to exit 2023-08-09T13:53:42.505 INFO:teuthology.orchestra.run:waiting for 300 2023-08-09T13:53:42.520 INFO:tasks.ceph.mds.c:Stopped 2023-08-09T13:53:42.520 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i c 2023-08-09T13:53:42.523 INFO:tasks.ceph.mds.a:Stopped 2023-08-09T13:53:42.523 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i a 2023-08-09T13:53:42.525 INFO:tasks.ceph.mds.b:Stopped 2023-08-09T13:53:42.525 DEBUG:teuthology.orchestra.run.smithi203:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i b 2023-08-09T13:53:42.538 INFO:tasks.ceph.mds.d:Stopped 2023-08-09T13:53:42.539 DEBUG:teuthology.orchestra.run.smithi203:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i d 2023-08-09T13:53:42.571 INFO:tasks.ceph.mds.c:Started 2023-08-09T13:53:42.572 DEBUG:teuthology.parallel:result is None 2023-08-09T13:53:42.579 INFO:tasks.ceph.mds.a:Started 2023-08-09T13:53:42.579 DEBUG:teuthology.parallel:result is None 2023-08-09T13:53:42.586 INFO:tasks.ceph.mds.b:Started 2023-08-09T13:53:42.586 DEBUG:teuthology.parallel:result is None 2023-08-09T13:53:42.594 INFO:tasks.ceph.mds.d:Started 2023-08-09T13:53:42.594 DEBUG:teuthology.parallel:result is None 2023-08-09T13:53:42.595 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist clear 2023-08-09T13:53:42.742 INFO:tasks.ceph.mds.c.smithi043.stdout:starting mds.c at 2023-08-09T13:53:42.746 INFO:tasks.ceph.mds.a.smithi043.stdout:starting mds.a at 2023-08-09T13:53:42.752 INFO:tasks.ceph.mds.b.smithi203.stdout:starting mds.b at 2023-08-09T13:53:42.763 INFO:tasks.ceph.mds.d.smithi203.stdout:starting mds.d at 2023-08-09T13:53:42.798 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:42.796+0000 7f5bff63f640 1 Processor -- start 2023-08-09T13:53:42.798 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:42.796+0000 7f5bff63f640 1 -- start start 2023-08-09T13:53:42.799 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:42.797+0000 7f5bff63f640 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7f5bf80d0a70 0x7f5bf80cd7f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:53:42.799 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:42.797+0000 7f5bff63f640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f5bf80cdd30 0x7f5bf80b3fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:53:42.800 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:42.797+0000 7f5bff63f640 1 -- --> v1:172.21.15.43:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5bf80d0f70 con 0x7f5bf815c680 2023-08-09T13:53:42.800 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:42.797+0000 7f5bff63f640 1 -- --> v2:172.21.15.203:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5bf80ccd10 con 0x7f5bf80d0a70 2023-08-09T13:53:42.800 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:42.797+0000 7f5bff63f640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5bf80ccff0 con 0x7f5bf80cdd30 2023-08-09T13:53:42.801 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:42.797+0000 7f5bfd3db640 1 --1- >> v1:172.21.15.43:6789/0 conn(0x7f5bf815c680 0x7f5bf80ce760 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.43:6789/0 says I am v1:172.21.15.43:44808/0 (socket says 172.21.15.43:44808) 2023-08-09T13:53:42.801 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:42.797+0000 7f5bfd3db640 1 -- 172.21.15.43:0/1125330188 learned_addr learned my addr 172.21.15.43:0/1125330188 (peer_addr_for_me v1:172.21.15.43:0/0) 2023-08-09T13:53:42.801 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:42.798+0000 7f5bfcbda640 1 --2- 172.21.15.43:0/1125330188 >> v2:172.21.15.203:3300/0 conn(0x7f5bf80d0a70 0x7f5bf80cd7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:53:42.801 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:42.798+0000 7f5bfdbdc640 1 --2- 172.21.15.43:0/1125330188 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f5bf80cdd30 0x7f5bf80b3fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:53:42.801 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:42.798+0000 7f5beffff640 1 -- 172.21.15.43:0/1125330188 <== mon.0 v1:172.21.15.43:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3299746982 0 0) 0x7f5bf80d0f70 con 0x7f5bf815c680 2023-08-09T13:53:42.802 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:42.798+0000 7f5beffff640 1 -- 172.21.15.43:0/1125330188 --> v1:172.21.15.43:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f5be4003520 con 0x7f5bf815c680 2023-08-09T13:53:42.802 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:42.798+0000 7f5bfcbda640 1 -- 172.21.15.43:0/1125330188 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f5bf80cdd30 msgr2=0x7f5bf80b3fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-09T13:53:42.802 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:42.798+0000 7f5bfcbda640 1 --2- 172.21.15.43:0/1125330188 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f5bf80cdd30 0x7f5bf80b3fa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:53:42.802 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:42.798+0000 7f5bfcbda640 1 -- 172.21.15.43:0/1125330188 >> v1:172.21.15.43:6789/0 conn(0x7f5bf815c680 legacy=0x7f5bf80ce760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:53:42.802 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:42.798+0000 7f5bfcbda640 1 -- 172.21.15.43:0/1125330188 --> v2:172.21.15.203:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5bf80cd2d0 con 0x7f5bf80d0a70 2023-08-09T13:53:42.803 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:42.800+0000 7f5bfcbda640 1 --2- 172.21.15.43:0/1125330188 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f5bf80d0a70 0x7f5bf80cd7f0 secure :-1 s=READY pgs=553 cs=0 l=1 rev1=1 crypto rx=0x7f5be80097a0 tx=0x7f5be802b8b0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=75b0c70f7af839f2 server_cookie=0 in_seq=0 out_seq=0 2023-08-09T13:53:42.803 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:42.800+0000 7f5beffff640 1 -- 172.21.15.43:0/1125330188 <== mon.1 v2:172.21.15.203:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5be802d040 con 0x7f5bf80d0a70 2023-08-09T13:53:42.803 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:42.800+0000 7f5beffff640 1 -- 172.21.15.43:0/1125330188 <== mon.1 v2:172.21.15.203:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5be8031810 con 0x7f5bf80d0a70 2023-08-09T13:53:42.803 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:42.800+0000 7f5beffff640 1 -- 172.21.15.43:0/1125330188 <== mon.1 v2:172.21.15.203:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5be8031ba0 con 0x7f5bf80d0a70 2023-08-09T13:53:42.804 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:42.801+0000 7f5bff63f640 1 -- 172.21.15.43:0/1125330188 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f5bf80d0a70 msgr2=0x7f5bf80cd7f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:53:42.804 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:42.801+0000 7f5bff63f640 1 --2- 172.21.15.43:0/1125330188 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f5bf80d0a70 0x7f5bf80cd7f0 secure :-1 s=READY pgs=553 cs=0 l=1 rev1=1 crypto rx=0x7f5be80097a0 tx=0x7f5be802b8b0 comp rx=0 tx=0).stop 2023-08-09T13:53:42.804 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:42.801+0000 7f5bff63f640 1 -- 172.21.15.43:0/1125330188 shutdown_connections 2023-08-09T13:53:42.804 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:42.801+0000 7f5bff63f640 1 --2- 172.21.15.43:0/1125330188 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f5bf80cdd30 0x7f5bf80b3fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:53:42.804 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:42.801+0000 7f5bff63f640 1 --2- 172.21.15.43:0/1125330188 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f5bf80d0a70 0x7f5bf80cd7f0 unknown :-1 s=CLOSED pgs=553 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:53:42.805 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:42.801+0000 7f5bff63f640 1 -- 172.21.15.43:0/1125330188 >> 172.21.15.43:0/1125330188 conn(0x7f5bf8071520 msgr2=0x7f5bf80d2520 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-09T13:53:42.805 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:42.801+0000 7f5bff63f640 1 -- 172.21.15.43:0/1125330188 shutdown_connections 2023-08-09T13:53:42.805 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:42.801+0000 7f5bff63f640 1 -- 172.21.15.43:0/1125330188 wait complete. 2023-08-09T13:53:42.805 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:42.803+0000 7f5bff63f640 1 Processor -- start 2023-08-09T13:53:42.805 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:42.803+0000 7f5bff63f640 1 -- start start 2023-08-09T13:53:42.807 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:42.804+0000 7f5bff63f640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f5bf80d0a70 0x7f5bf80c8cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:53:42.807 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:42.804+0000 7f5bff63f640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f5bf815c680 0x7f5bf814f3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:53:42.807 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:42.804+0000 7f5bff63f640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f5bf814f930 0x7f5bf8159ca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:53:42.807 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:42.804+0000 7f5bff63f640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5bf80cb920 con 0x7f5bf815c680 2023-08-09T13:53:42.807 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:42.804+0000 7f5bff63f640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5bf80cad10 con 0x7f5bf80d0a70 2023-08-09T13:53:42.808 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:42.804+0000 7f5bff63f640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5bf80caff0 con 0x7f5bf814f930 2023-08-09T13:53:42.808 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:42.804+0000 7f5bfcbda640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f5bf815c680 0x7f5bf814f3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:53:42.808 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:42.804+0000 7f5bfcbda640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f5bf815c680 0x7f5bf814f3f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:53576/0 (socket says 172.21.15.43:53576) 2023-08-09T13:53:42.808 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:42.804+0000 7f5bfcbda640 1 -- 172.21.15.43:0/235374192 learned_addr learned my addr 172.21.15.43:0/235374192 (peer_addr_for_me v2:172.21.15.43:0/0) 2023-08-09T13:53:42.808 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:42.804+0000 7f5bfd3db640 1 --2- 172.21.15.43:0/235374192 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f5bf80d0a70 0x7f5bf80c8cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:53:42.810 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:42.805+0000 7f5bfdbdc640 1 --2- 172.21.15.43:0/235374192 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f5bf814f930 0x7f5bf8159ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:53:42.810 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:42.805+0000 7f5bfcbda640 1 -- 172.21.15.43:0/235374192 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f5bf814f930 msgr2=0x7f5bf8159ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:53:42.810 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:42.805+0000 7f5bfcbda640 1 --2- 172.21.15.43:0/235374192 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f5bf814f930 0x7f5bf8159ca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:53:42.810 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:42.805+0000 7f5bfcbda640 1 -- 172.21.15.43:0/235374192 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f5bf80d0a70 msgr2=0x7f5bf80c8cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:53:42.810 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:42.805+0000 7f5bfcbda640 1 --2- 172.21.15.43:0/235374192 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f5bf80d0a70 0x7f5bf80c8cf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:53:42.811 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:42.805+0000 7f5bfcbda640 1 -- 172.21.15.43:0/235374192 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5be8009400 con 0x7f5bf815c680 2023-08-09T13:53:42.811 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:42.805+0000 7f5bfdbdc640 1 --2- 172.21.15.43:0/235374192 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f5bf814f930 0x7f5bf8159ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-08-09T13:53:42.812 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:42.805+0000 7f5bfcbda640 1 --2- 172.21.15.43:0/235374192 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f5bf815c680 0x7f5bf814f3f0 secure :-1 s=READY pgs=1488 cs=0 l=1 rev1=1 crypto rx=0x7f5be80098c0 tx=0x7f5be80046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-09T13:53:42.812 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:42.805+0000 7f5bedffb640 1 -- 172.21.15.43:0/235374192 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5be803ad60 con 0x7f5bf815c680 2023-08-09T13:53:42.812 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:42.805+0000 7f5bedffb640 1 -- 172.21.15.43:0/235374192 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5be8044400 con 0x7f5bf815c680 2023-08-09T13:53:42.813 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:42.805+0000 7f5bedffb640 1 -- 172.21.15.43:0/235374192 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5be80445b0 con 0x7f5bf815c680 2023-08-09T13:53:42.813 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:42.805+0000 7f5bff63f640 1 -- 172.21.15.43:0/235374192 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5bf80cb2d0 con 0x7f5bf815c680 2023-08-09T13:53:42.813 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:42.806+0000 7f5bff63f640 1 -- 172.21.15.43:0/235374192 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5bf80ca930 con 0x7f5bf815c680 2023-08-09T13:53:42.813 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:42.807+0000 7f5bff63f640 1 -- 172.21.15.43:0/235374192 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5bf80739f0 con 0x7f5bf815c680 2023-08-09T13:53:42.813 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:42.808+0000 7f5bedffb640 1 -- 172.21.15.43:0/235374192 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 56) v1 ==== 88443+0+0 (secure 0 0 0) 0x7f5be804d440 con 0x7f5bf815c680 2023-08-09T13:53:42.813 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:42.809+0000 7f5bedffb640 1 --2- 172.21.15.43:0/235374192 >> [v2:172.21.15.203:6832/2527059680,v1:172.21.15.203:6833/2527059680] conn(0x7f5bdc06a170 0x7f5bdc06c630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:53:42.813 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:42.809+0000 7f5bedffb640 1 -- 172.21.15.43:0/235374192 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(121..121 src has 1..121) v4 ==== 6434+0+0 (secure 0 0 0) 0x7f5be80aad10 con 0x7f5bf815c680 2023-08-09T13:53:42.814 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:42.809+0000 7f5bfd3db640 1 --2- 172.21.15.43:0/235374192 >> [v2:172.21.15.203:6832/2527059680,v1:172.21.15.203:6833/2527059680] conn(0x7f5bdc06a170 0x7f5bdc06c630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:53:42.814 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:42.810+0000 7f5bfd3db640 1 --2- 172.21.15.43:0/235374192 >> [v2:172.21.15.203:6832/2527059680,v1:172.21.15.203:6833/2527059680] conn(0x7f5bdc06a170 0x7f5bdc06c630 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7f5bf40011e0 tx=0x7f5bf4009040 comp rx=0 tx=0).ready entity=mgr.6377 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-09T13:53:42.815 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:42.813+0000 7f5bedffb640 1 -- 172.21.15.43:0/235374192 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178066 (secure 0 0 0) 0x7f5be80b0020 con 0x7f5bf815c680 2023-08-09T13:53:43.024 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:43.021+0000 7f5bff63f640 1 -- 172.21.15.43:0/235374192 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "osd blocklist clear"} v 0) v1 -- 0x7f5bf815a930 con 0x7f5bf815c680 2023-08-09T13:53:43.214 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:43.212+0000 7f5bedffb640 1 -- 172.21.15.43:0/235374192 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist clear"}]=0 removed all blocklist entries v123) v1 ==== 97+0+0 (secure 0 0 0) 0x7f5be80760e0 con 0x7f5bf815c680 2023-08-09T13:53:43.215 INFO:teuthology.orchestra.run.smithi043.stderr: removed all blocklist entries 2023-08-09T13:53:43.218 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:43.215+0000 7f5bff63f640 1 -- 172.21.15.43:0/235374192 >> [v2:172.21.15.203:6832/2527059680,v1:172.21.15.203:6833/2527059680] conn(0x7f5bdc06a170 msgr2=0x7f5bdc06c630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:53:43.218 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:43.215+0000 7f5bff63f640 1 --2- 172.21.15.43:0/235374192 >> [v2:172.21.15.203:6832/2527059680,v1:172.21.15.203:6833/2527059680] conn(0x7f5bdc06a170 0x7f5bdc06c630 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7f5bf40011e0 tx=0x7f5bf4009040 comp rx=0 tx=0).stop 2023-08-09T13:53:43.218 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:43.215+0000 7f5bff63f640 1 -- 172.21.15.43:0/235374192 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f5bf815c680 msgr2=0x7f5bf814f3f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:53:43.218 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:43.215+0000 7f5bff63f640 1 --2- 172.21.15.43:0/235374192 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f5bf815c680 0x7f5bf814f3f0 secure :-1 s=READY pgs=1488 cs=0 l=1 rev1=1 crypto rx=0x7f5be80098c0 tx=0x7f5be80046c0 comp rx=0 tx=0).stop 2023-08-09T13:53:43.218 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:43.215+0000 7f5bff63f640 1 -- 172.21.15.43:0/235374192 shutdown_connections 2023-08-09T13:53:43.218 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:43.215+0000 7f5bff63f640 1 --2- 172.21.15.43:0/235374192 >> [v2:172.21.15.203:6832/2527059680,v1:172.21.15.203:6833/2527059680] conn(0x7f5bdc06a170 0x7f5bdc06c630 unknown :-1 s=CLOSED pgs=45 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:53:43.219 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:43.215+0000 7f5bff63f640 1 --2- 172.21.15.43:0/235374192 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f5bf814f930 0x7f5bf8159ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:53:43.219 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:43.215+0000 7f5bff63f640 1 --2- 172.21.15.43:0/235374192 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f5bf815c680 0x7f5bf814f3f0 unknown :-1 s=CLOSED pgs=1488 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:53:43.219 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:43.215+0000 7f5bff63f640 1 --2- 172.21.15.43:0/235374192 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f5bf80d0a70 0x7f5bf80c8cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:53:43.219 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:43.215+0000 7f5bff63f640 1 -- 172.21.15.43:0/235374192 >> 172.21.15.43:0/235374192 conn(0x7f5bf8071520 msgr2=0x7f5bf81539d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-09T13:53:43.220 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:43.216+0000 7f5bff63f640 1 -- 172.21.15.43:0/235374192 shutdown_connections 2023-08-09T13:53:43.220 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:43.216+0000 7f5bff63f640 1 -- 172.21.15.43:0/235374192 wait complete. 2023-08-09T13:53:43.243 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth ls --format=json-pretty 2023-08-09T13:53:43.469 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:43.467+0000 7fe933fff640 1 Processor -- start 2023-08-09T13:53:43.470 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:43.467+0000 7fe933fff640 1 -- start start 2023-08-09T13:53:43.471 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:43.469+0000 7fe933fff640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7fe9340c8b30 0x7fe9340ad340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:53:43.472 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:43.469+0000 7fe933fff640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fe9340acdf0 0x7fe93415b2c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:53:43.472 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:43.469+0000 7fe933fff640 1 -- --> v1:172.21.15.43:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe9340cbd90 con 0x7fe9340c8750 2023-08-09T13:53:43.472 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:43.469+0000 7fe933fff640 1 -- --> v2:172.21.15.43:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe93415e810 con 0x7fe9340c8b30 2023-08-09T13:53:43.472 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:43.469+0000 7fe933fff640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe9340c9f70 con 0x7fe9340acdf0 2023-08-09T13:53:43.472 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:43.469+0000 7fe9327fc640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7fe9340c8b30 0x7fe9340ad340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:53:43.472 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:43.469+0000 7fe932ffd640 1 --1- >> v1:172.21.15.43:6789/0 conn(0x7fe9340c8750 0x7fe93406f410 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.43:6789/0 says I am v1:172.21.15.43:44810/0 (socket says 172.21.15.43:44810) 2023-08-09T13:53:43.472 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:43.469+0000 7fe932ffd640 1 -- 172.21.15.43:0/1178183594 learned_addr learned my addr 172.21.15.43:0/1178183594 (peer_addr_for_me v1:172.21.15.43:0/0) 2023-08-09T13:53:43.473 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:43.470+0000 7fe9327fc640 1 -- 172.21.15.43:0/1178183594 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fe9340acdf0 msgr2=0x7fe93415b2c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-09T13:53:43.474 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:43.470+0000 7fe9337fe640 1 --2- 172.21.15.43:0/1178183594 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fe9340acdf0 0x7fe93415b2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:53:43.474 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:43.470+0000 7fe9327fc640 1 --2- 172.21.15.43:0/1178183594 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fe9340acdf0 0x7fe93415b2c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:53:43.474 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:43.470+0000 7fe931ffb640 1 -- 172.21.15.43:0/1178183594 <== mon.0 v1:172.21.15.43:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3738325948 0 0) 0x7fe9340cbd90 con 0x7fe9340c8750 2023-08-09T13:53:43.474 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:43.470+0000 7fe9327fc640 1 -- 172.21.15.43:0/1178183594 >> v1:172.21.15.43:6789/0 conn(0x7fe9340c8750 legacy=0x7fe93406f410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:53:43.474 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:43.470+0000 7fe9327fc640 1 -- 172.21.15.43:0/1178183594 --> v2:172.21.15.43:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe9340ca2b0 con 0x7fe9340c8b30 2023-08-09T13:53:43.474 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:43.470+0000 7fe9337fe640 1 --2- 172.21.15.43:0/1178183594 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fe9340acdf0 0x7fe93415b2c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-08-09T13:53:43.475 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:43.471+0000 7fe9327fc640 1 --2- 172.21.15.43:0/1178183594 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fe9340c8b30 0x7fe9340ad340 secure :-1 s=READY pgs=1490 cs=0 l=1 rev1=1 crypto rx=0x7fe9280097a0 tx=0x7fe92802b820 comp rx=0 tx=0).ready entity=mon.0 client_cookie=27f3ee875f92b44d server_cookie=0 in_seq=0 out_seq=0 2023-08-09T13:53:43.475 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:43.471+0000 7fe931ffb640 1 -- 172.21.15.43:0/1178183594 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe928031780 con 0x7fe9340c8b30 2023-08-09T13:53:43.475 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:43.471+0000 7fe931ffb640 1 -- 172.21.15.43:0/1178183594 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe9280318e0 con 0x7fe9340c8b30 2023-08-09T13:53:43.475 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:43.471+0000 7fe931ffb640 1 -- 172.21.15.43:0/1178183594 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe928031c70 con 0x7fe9340c8b30 2023-08-09T13:53:43.476 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:43.472+0000 7fe933fff640 1 -- 172.21.15.43:0/1178183594 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fe9340c8b30 msgr2=0x7fe9340ad340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:53:43.476 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:43.472+0000 7fe933fff640 1 --2- 172.21.15.43:0/1178183594 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fe9340c8b30 0x7fe9340ad340 secure :-1 s=READY pgs=1490 cs=0 l=1 rev1=1 crypto rx=0x7fe9280097a0 tx=0x7fe92802b820 comp rx=0 tx=0).stop 2023-08-09T13:53:43.476 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:43.472+0000 7fe933fff640 1 -- 172.21.15.43:0/1178183594 shutdown_connections 2023-08-09T13:53:43.476 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:43.472+0000 7fe933fff640 1 --2- 172.21.15.43:0/1178183594 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fe9340acdf0 0x7fe93415b2c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:53:43.477 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:43.472+0000 7fe933fff640 1 --2- 172.21.15.43:0/1178183594 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fe9340c8b30 0x7fe9340ad340 unknown :-1 s=CLOSED pgs=1490 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:53:43.477 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:43.472+0000 7fe933fff640 1 -- 172.21.15.43:0/1178183594 >> 172.21.15.43:0/1178183594 conn(0x7fe9340cde90 msgr2=0x7fe9340ce2a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-09T13:53:43.477 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:43.472+0000 7fe933fff640 1 -- 172.21.15.43:0/1178183594 shutdown_connections 2023-08-09T13:53:43.477 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:43.472+0000 7fe933fff640 1 -- 172.21.15.43:0/1178183594 wait complete. 2023-08-09T13:53:43.477 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:43.474+0000 7fe933fff640 1 Processor -- start 2023-08-09T13:53:43.478 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:43.474+0000 7fe933fff640 1 -- start start 2023-08-09T13:53:43.479 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:43.475+0000 7fe933fff640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fe9340acdf0 0x7fe934159360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:53:43.479 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:43.475+0000 7fe933fff640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fe9340c8750 0x7fe9341f23a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:53:43.479 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:43.475+0000 7fe933fff640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fe9340c8b30 0x7fe934158af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:53:43.479 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:43.475+0000 7fe933fff640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe93406bb80 con 0x7fe9340c8b30 2023-08-09T13:53:43.479 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:43.475+0000 7fe933fff640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe9340c6880 con 0x7fe9340acdf0 2023-08-09T13:53:43.479 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:43.475+0000 7fe933fff640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe9340b1b00 con 0x7fe9340c8750 2023-08-09T13:53:43.479 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:43.475+0000 7fe9337fe640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fe9340c8b30 0x7fe934158af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:53:43.480 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:43.475+0000 7fe9337fe640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fe9340c8b30 0x7fe934158af0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:53596/0 (socket says 172.21.15.43:53596) 2023-08-09T13:53:43.480 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:43.475+0000 7fe9337fe640 1 -- 172.21.15.43:0/3421411748 learned_addr learned my addr 172.21.15.43:0/3421411748 (peer_addr_for_me v2:172.21.15.43:0/0) 2023-08-09T13:53:43.480 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:43.476+0000 7fe9327fc640 1 --2- 172.21.15.43:0/3421411748 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fe9340c8750 0x7fe9341f23a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:53:43.480 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:43.476+0000 7fe932ffd640 1 --2- 172.21.15.43:0/3421411748 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fe9340acdf0 0x7fe934159360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:53:43.480 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:43.476+0000 7fe9337fe640 1 -- 172.21.15.43:0/3421411748 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fe9340c8750 msgr2=0x7fe9341f23a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:53:43.480 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:43.476+0000 7fe9337fe640 1 --2- 172.21.15.43:0/3421411748 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fe9340c8750 0x7fe9341f23a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:53:43.480 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:43.476+0000 7fe9337fe640 1 -- 172.21.15.43:0/3421411748 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fe9340acdf0 msgr2=0x7fe934159360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:53:43.481 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:43.476+0000 7fe9337fe640 1 --2- 172.21.15.43:0/3421411748 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fe9340acdf0 0x7fe934159360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:53:43.481 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:43.476+0000 7fe9337fe640 1 -- 172.21.15.43:0/3421411748 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe928009400 con 0x7fe9340c8b30 2023-08-09T13:53:43.481 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:43.476+0000 7fe9327fc640 1 --2- 172.21.15.43:0/3421411748 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fe9340c8750 0x7fe9341f23a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-08-09T13:53:43.481 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:43.476+0000 7fe932ffd640 1 --2- 172.21.15.43:0/3421411748 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fe9340acdf0 0x7fe934159360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-08-09T13:53:43.481 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:43.476+0000 7fe9337fe640 1 --2- 172.21.15.43:0/3421411748 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fe9340c8b30 0x7fe934158af0 secure :-1 s=READY pgs=1491 cs=0 l=1 rev1=1 crypto rx=0x7fe92c00cd50 tx=0x7fe92c00a650 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-09T13:53:43.482 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:43.476+0000 7fe91b7fe640 1 -- 172.21.15.43:0/3421411748 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe92c00f680 con 0x7fe9340c8b30 2023-08-09T13:53:43.482 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:43.477+0000 7fe91b7fe640 1 -- 172.21.15.43:0/3421411748 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe92c00f800 con 0x7fe9340c8b30 2023-08-09T13:53:43.482 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:43.477+0000 7fe91b7fe640 1 -- 172.21.15.43:0/3421411748 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe92c00fbb0 con 0x7fe9340c8b30 2023-08-09T13:53:43.483 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:43.477+0000 7fe933fff640 1 -- 172.21.15.43:0/3421411748 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe9340b1de0 con 0x7fe9340c8b30 2023-08-09T13:53:43.483 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:43.477+0000 7fe933fff640 1 -- 172.21.15.43:0/3421411748 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe9340b23a0 con 0x7fe9340c8b30 2023-08-09T13:53:43.483 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:43.478+0000 7fe933fff640 1 -- 172.21.15.43:0/3421411748 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe9340ac540 con 0x7fe9340c8b30 2023-08-09T13:53:43.483 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:43.479+0000 7fe91b7fe640 1 -- 172.21.15.43:0/3421411748 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 56) v1 ==== 88443+0+0 (secure 0 0 0) 0x7fe92c00b040 con 0x7fe9340c8b30 2023-08-09T13:53:43.483 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:43.480+0000 7fe91b7fe640 1 --2- 172.21.15.43:0/3421411748 >> [v2:172.21.15.203:6832/2527059680,v1:172.21.15.203:6833/2527059680] conn(0x7fe92006a170 0x7fe92006c630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:53:43.483 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:43.480+0000 7fe91b7fe640 1 -- 172.21.15.43:0/3421411748 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(125..125 src has 1..125) v4 ==== 5574+0+0 (secure 0 0 0) 0x7fe92c089180 con 0x7fe9340c8b30 2023-08-09T13:53:43.484 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:43.481+0000 7fe932ffd640 1 --2- 172.21.15.43:0/3421411748 >> [v2:172.21.15.203:6832/2527059680,v1:172.21.15.203:6833/2527059680] conn(0x7fe92006a170 0x7fe92006c630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:53:43.484 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:43.482+0000 7fe932ffd640 1 --2- 172.21.15.43:0/3421411748 >> [v2:172.21.15.203:6832/2527059680,v1:172.21.15.203:6833/2527059680] conn(0x7fe92006a170 0x7fe92006c630 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7fe924000f60 tx=0x7fe924009040 comp rx=0 tx=0).ready entity=mgr.6377 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-09T13:53:43.486 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:43.484+0000 7fe91b7fe640 1 -- 172.21.15.43:0/3421411748 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178066 (secure 0 0 0) 0x7fe92c054790 con 0x7fe9340c8b30 2023-08-09T13:53:43.741 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:43.738+0000 7fe933fff640 1 -- 172.21.15.43:0/3421411748 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "auth ls", "format": "json-pretty"} v 0) v1 -- 0x7fe934065f90 con 0x7fe9340c8b30 2023-08-09T13:53:43.742 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:43.740+0000 7fe91b7fe640 1 -- 172.21.15.43:0/3421411748 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "auth ls", "format": "json-pretty"}]=0 v18) v1 ==== 80+0+5304 (secure 0 0 0) 0x7fe92c057db0 con 0x7fe9340c8b30 2023-08-09T13:53:43.742 INFO:teuthology.orchestra.run.smithi043.stdout: 2023-08-09T13:53:43.742 INFO:teuthology.orchestra.run.smithi043.stdout:{ 2023-08-09T13:53:43.743 INFO:teuthology.orchestra.run.smithi043.stdout: "auth_dump": [ 2023-08-09T13:53:43.743 INFO:teuthology.orchestra.run.smithi043.stdout: { 2023-08-09T13:53:43.743 INFO:teuthology.orchestra.run.smithi043.stdout: "entity": "mds.a", 2023-08-09T13:53:43.743 INFO:teuthology.orchestra.run.smithi043.stdout: "key": "AQCOldNkPu6GCRAAZecwctvMji0Zubb60KMlyQ==", 2023-08-09T13:53:43.743 INFO:teuthology.orchestra.run.smithi043.stdout: "caps": { 2023-08-09T13:53:43.743 INFO:teuthology.orchestra.run.smithi043.stdout: "mds": "allow", 2023-08-09T13:53:43.743 INFO:teuthology.orchestra.run.smithi043.stdout: "mgr": "allow *", 2023-08-09T13:53:43.743 INFO:teuthology.orchestra.run.smithi043.stdout: "mon": "allow *", 2023-08-09T13:53:43.744 INFO:teuthology.orchestra.run.smithi043.stdout: "osd": "allow *" 2023-08-09T13:53:43.744 INFO:teuthology.orchestra.run.smithi043.stdout: } 2023-08-09T13:53:43.744 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2023-08-09T13:53:43.744 INFO:teuthology.orchestra.run.smithi043.stdout: { 2023-08-09T13:53:43.744 INFO:teuthology.orchestra.run.smithi043.stdout: "entity": "mds.b", 2023-08-09T13:53:43.744 INFO:teuthology.orchestra.run.smithi043.stdout: "key": "AQCOldNkt2+5KxAAHcckB+S5NZHsF5z22StXmw==", 2023-08-09T13:53:43.745 INFO:teuthology.orchestra.run.smithi043.stdout: "caps": { 2023-08-09T13:53:43.745 INFO:teuthology.orchestra.run.smithi043.stdout: "mds": "allow", 2023-08-09T13:53:43.745 INFO:teuthology.orchestra.run.smithi043.stdout: "mgr": "allow *", 2023-08-09T13:53:43.745 INFO:teuthology.orchestra.run.smithi043.stdout: "mon": "allow *", 2023-08-09T13:53:43.745 INFO:teuthology.orchestra.run.smithi043.stdout: "osd": "allow *" 2023-08-09T13:53:43.746 INFO:teuthology.orchestra.run.smithi043.stdout: } 2023-08-09T13:53:43.746 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2023-08-09T13:53:43.746 INFO:teuthology.orchestra.run.smithi043.stdout: { 2023-08-09T13:53:43.746 INFO:teuthology.orchestra.run.smithi043.stdout: "entity": "mds.c", 2023-08-09T13:53:43.746 INFO:teuthology.orchestra.run.smithi043.stdout: "key": "AQCOldNk5V46GhAAQBhkvfjVSJCSHLS/YkhgCA==", 2023-08-09T13:53:43.746 INFO:teuthology.orchestra.run.smithi043.stdout: "caps": { 2023-08-09T13:53:43.746 INFO:teuthology.orchestra.run.smithi043.stdout: "mds": "allow", 2023-08-09T13:53:43.746 INFO:teuthology.orchestra.run.smithi043.stdout: "mgr": "allow *", 2023-08-09T13:53:43.747 INFO:teuthology.orchestra.run.smithi043.stdout: "mon": "allow *", 2023-08-09T13:53:43.747 INFO:teuthology.orchestra.run.smithi043.stdout: "osd": "allow *" 2023-08-09T13:53:43.747 INFO:teuthology.orchestra.run.smithi043.stdout: } 2023-08-09T13:53:43.747 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2023-08-09T13:53:43.747 INFO:teuthology.orchestra.run.smithi043.stdout: { 2023-08-09T13:53:43.747 INFO:teuthology.orchestra.run.smithi043.stdout: "entity": "mds.d", 2023-08-09T13:53:43.747 INFO:teuthology.orchestra.run.smithi043.stdout: "key": "AQCPldNkNBoTARAAMVW+rSyXQKaau6I90fUX9Q==", 2023-08-09T13:53:43.748 INFO:teuthology.orchestra.run.smithi043.stdout: "caps": { 2023-08-09T13:53:43.748 INFO:teuthology.orchestra.run.smithi043.stdout: "mds": "allow", 2023-08-09T13:53:43.748 INFO:teuthology.orchestra.run.smithi043.stdout: "mgr": "allow *", 2023-08-09T13:53:43.748 INFO:teuthology.orchestra.run.smithi043.stdout: "mon": "allow *", 2023-08-09T13:53:43.748 INFO:teuthology.orchestra.run.smithi043.stdout: "osd": "allow *" 2023-08-09T13:53:43.748 INFO:teuthology.orchestra.run.smithi043.stdout: } 2023-08-09T13:53:43.748 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2023-08-09T13:53:43.748 INFO:teuthology.orchestra.run.smithi043.stdout: { 2023-08-09T13:53:43.749 INFO:teuthology.orchestra.run.smithi043.stdout: "entity": "osd.0", 2023-08-09T13:53:43.749 INFO:teuthology.orchestra.run.smithi043.stdout: "key": "AQCRldNkltPlMhAAAt3rFvraCtI0aSqGvt6UxA==", 2023-08-09T13:53:43.749 INFO:teuthology.orchestra.run.smithi043.stdout: "caps": { 2023-08-09T13:53:43.749 INFO:teuthology.orchestra.run.smithi043.stdout: "mgr": "allow profile osd", 2023-08-09T13:53:43.749 INFO:teuthology.orchestra.run.smithi043.stdout: "mon": "allow profile osd", 2023-08-09T13:53:43.749 INFO:teuthology.orchestra.run.smithi043.stdout: "osd": "allow *" 2023-08-09T13:53:43.749 INFO:teuthology.orchestra.run.smithi043.stdout: } 2023-08-09T13:53:43.750 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2023-08-09T13:53:43.750 INFO:teuthology.orchestra.run.smithi043.stdout: { 2023-08-09T13:53:43.750 INFO:teuthology.orchestra.run.smithi043.stdout: "entity": "osd.1", 2023-08-09T13:53:43.750 INFO:teuthology.orchestra.run.smithi043.stdout: "key": "AQCVldNkzpKYEhAA+wen0kV/vUDLQztq3vhfOg==", 2023-08-09T13:53:43.750 INFO:teuthology.orchestra.run.smithi043.stdout: "caps": { 2023-08-09T13:53:43.750 INFO:teuthology.orchestra.run.smithi043.stdout: "mgr": "allow profile osd", 2023-08-09T13:53:43.750 INFO:teuthology.orchestra.run.smithi043.stdout: "mon": "allow profile osd", 2023-08-09T13:53:43.750 INFO:teuthology.orchestra.run.smithi043.stdout: "osd": "allow *" 2023-08-09T13:53:43.751 INFO:teuthology.orchestra.run.smithi043.stdout: } 2023-08-09T13:53:43.751 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2023-08-09T13:53:43.751 INFO:teuthology.orchestra.run.smithi043.stdout: { 2023-08-09T13:53:43.751 INFO:teuthology.orchestra.run.smithi043.stdout: "entity": "osd.2", 2023-08-09T13:53:43.751 INFO:teuthology.orchestra.run.smithi043.stdout: "key": "AQCYldNkYbwNHhAAQY8g9dldb1V9M2MM2SHKDg==", 2023-08-09T13:53:43.751 INFO:teuthology.orchestra.run.smithi043.stdout: "caps": { 2023-08-09T13:53:43.751 INFO:teuthology.orchestra.run.smithi043.stdout: "mgr": "allow profile osd", 2023-08-09T13:53:43.751 INFO:teuthology.orchestra.run.smithi043.stdout: "mon": "allow profile osd", 2023-08-09T13:53:43.752 INFO:teuthology.orchestra.run.smithi043.stdout: "osd": "allow *" 2023-08-09T13:53:43.752 INFO:teuthology.orchestra.run.smithi043.stdout: } 2023-08-09T13:53:43.752 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2023-08-09T13:53:43.752 INFO:teuthology.orchestra.run.smithi043.stdout: { 2023-08-09T13:53:43.752 INFO:teuthology.orchestra.run.smithi043.stdout: "entity": "osd.3", 2023-08-09T13:53:43.752 INFO:teuthology.orchestra.run.smithi043.stdout: "key": "AQCbldNkeyW6KRAAsd+kgqik2aD4U7PXwccWtA==", 2023-08-09T13:53:43.752 INFO:teuthology.orchestra.run.smithi043.stdout: "caps": { 2023-08-09T13:53:43.752 INFO:teuthology.orchestra.run.smithi043.stdout: "mgr": "allow profile osd", 2023-08-09T13:53:43.753 INFO:teuthology.orchestra.run.smithi043.stdout: "mon": "allow profile osd", 2023-08-09T13:53:43.753 INFO:teuthology.orchestra.run.smithi043.stdout: "osd": "allow *" 2023-08-09T13:53:43.753 INFO:teuthology.orchestra.run.smithi043.stdout: } 2023-08-09T13:53:43.753 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2023-08-09T13:53:43.753 INFO:teuthology.orchestra.run.smithi043.stdout: { 2023-08-09T13:53:43.753 INFO:teuthology.orchestra.run.smithi043.stdout: "entity": "osd.4", 2023-08-09T13:53:43.753 INFO:teuthology.orchestra.run.smithi043.stdout: "key": "AQCildNk80UkOBAAtX+4JGpOVcfZEHetJoeZKw==", 2023-08-09T13:53:43.753 INFO:teuthology.orchestra.run.smithi043.stdout: "caps": { 2023-08-09T13:53:43.754 INFO:teuthology.orchestra.run.smithi043.stdout: "mgr": "allow profile osd", 2023-08-09T13:53:43.754 INFO:teuthology.orchestra.run.smithi043.stdout: "mon": "allow profile osd", 2023-08-09T13:53:43.754 INFO:teuthology.orchestra.run.smithi043.stdout: "osd": "allow *" 2023-08-09T13:53:43.754 INFO:teuthology.orchestra.run.smithi043.stdout: } 2023-08-09T13:53:43.754 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2023-08-09T13:53:43.754 INFO:teuthology.orchestra.run.smithi043.stdout: { 2023-08-09T13:53:43.754 INFO:teuthology.orchestra.run.smithi043.stdout: "entity": "osd.5", 2023-08-09T13:53:43.755 INFO:teuthology.orchestra.run.smithi043.stdout: "key": "AQCmldNkCA9SBxAAZj4So6MRj2Eq1I9H9JzLDA==", 2023-08-09T13:53:43.755 INFO:teuthology.orchestra.run.smithi043.stdout: "caps": { 2023-08-09T13:53:43.755 INFO:teuthology.orchestra.run.smithi043.stdout: "mgr": "allow profile osd", 2023-08-09T13:53:43.755 INFO:teuthology.orchestra.run.smithi043.stdout: "mon": "allow profile osd", 2023-08-09T13:53:43.755 INFO:teuthology.orchestra.run.smithi043.stdout: "osd": "allow *" 2023-08-09T13:53:43.755 INFO:teuthology.orchestra.run.smithi043.stdout: } 2023-08-09T13:53:43.755 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2023-08-09T13:53:43.755 INFO:teuthology.orchestra.run.smithi043.stdout: { 2023-08-09T13:53:43.756 INFO:teuthology.orchestra.run.smithi043.stdout: "entity": "osd.6", 2023-08-09T13:53:43.756 INFO:teuthology.orchestra.run.smithi043.stdout: "key": "AQCpldNkaOwAExAAraV1tUwr9uU6GheZ+hvdqQ==", 2023-08-09T13:53:43.756 INFO:teuthology.orchestra.run.smithi043.stdout: "caps": { 2023-08-09T13:53:43.756 INFO:teuthology.orchestra.run.smithi043.stdout: "mgr": "allow profile osd", 2023-08-09T13:53:43.756 INFO:teuthology.orchestra.run.smithi043.stdout: "mon": "allow profile osd", 2023-08-09T13:53:43.756 INFO:teuthology.orchestra.run.smithi043.stdout: "osd": "allow *" 2023-08-09T13:53:43.756 INFO:teuthology.orchestra.run.smithi043.stdout: } 2023-08-09T13:53:43.757 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2023-08-09T13:53:43.757 INFO:teuthology.orchestra.run.smithi043.stdout: { 2023-08-09T13:53:43.757 INFO:teuthology.orchestra.run.smithi043.stdout: "entity": "osd.7", 2023-08-09T13:53:43.757 INFO:teuthology.orchestra.run.smithi043.stdout: "key": "AQCsldNk2/GwLBAAP42EXPivzLk44X8aMEVVCg==", 2023-08-09T13:53:43.757 INFO:teuthology.orchestra.run.smithi043.stdout: "caps": { 2023-08-09T13:53:43.757 INFO:teuthology.orchestra.run.smithi043.stdout: "mgr": "allow profile osd", 2023-08-09T13:53:43.757 INFO:teuthology.orchestra.run.smithi043.stdout: "mon": "allow profile osd", 2023-08-09T13:53:43.757 INFO:teuthology.orchestra.run.smithi043.stdout: "osd": "allow *" 2023-08-09T13:53:43.758 INFO:teuthology.orchestra.run.smithi043.stdout: } 2023-08-09T13:53:43.758 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2023-08-09T13:53:43.758 INFO:teuthology.orchestra.run.smithi043.stdout: { 2023-08-09T13:53:43.758 INFO:teuthology.orchestra.run.smithi043.stdout: "entity": "client.0", 2023-08-09T13:53:43.758 INFO:teuthology.orchestra.run.smithi043.stdout: "key": "AQCPldNkg8zrDxAAM/hnJaE6S9HU/fRwQeBV5w==", 2023-08-09T13:53:43.758 INFO:teuthology.orchestra.run.smithi043.stdout: "caps": { 2023-08-09T13:53:43.758 INFO:teuthology.orchestra.run.smithi043.stdout: "mds": "allow", 2023-08-09T13:53:43.758 INFO:teuthology.orchestra.run.smithi043.stdout: "mon": "allow r", 2023-08-09T13:53:43.759 INFO:teuthology.orchestra.run.smithi043.stdout: "osd": "allow rw tag cephfs data=cephfs" 2023-08-09T13:53:43.759 INFO:teuthology.orchestra.run.smithi043.stdout: } 2023-08-09T13:53:43.759 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2023-08-09T13:53:43.759 INFO:teuthology.orchestra.run.smithi043.stdout: { 2023-08-09T13:53:43.759 INFO:teuthology.orchestra.run.smithi043.stdout: "entity": "client.1", 2023-08-09T13:53:43.759 INFO:teuthology.orchestra.run.smithi043.stdout: "key": "AQCPldNklfFxJxAA19+L78rW6jU1uOyjey6WNA==", 2023-08-09T13:53:43.759 INFO:teuthology.orchestra.run.smithi043.stdout: "caps": { 2023-08-09T13:53:43.760 INFO:teuthology.orchestra.run.smithi043.stdout: "mds": "allow", 2023-08-09T13:53:43.760 INFO:teuthology.orchestra.run.smithi043.stdout: "mgr": "allow r", 2023-08-09T13:53:43.760 INFO:teuthology.orchestra.run.smithi043.stdout: "mon": "allow rw", 2023-08-09T13:53:43.760 INFO:teuthology.orchestra.run.smithi043.stdout: "osd": "allow rwx" 2023-08-09T13:53:43.760 INFO:teuthology.orchestra.run.smithi043.stdout: } 2023-08-09T13:53:43.760 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2023-08-09T13:53:43.761 INFO:teuthology.orchestra.run.smithi043.stdout: { 2023-08-09T13:53:43.761 INFO:teuthology.orchestra.run.smithi043.stdout: "entity": "client.2", 2023-08-09T13:53:43.761 INFO:teuthology.orchestra.run.smithi043.stdout: "key": "AQCPldNkC2lgGxAAZ8qZboOCUaQlcpkPL1ozHA==", 2023-08-09T13:53:43.761 INFO:teuthology.orchestra.run.smithi043.stdout: "caps": { 2023-08-09T13:53:43.761 INFO:teuthology.orchestra.run.smithi043.stdout: "mds": "allow", 2023-08-09T13:53:43.761 INFO:teuthology.orchestra.run.smithi043.stdout: "mgr": "allow r", 2023-08-09T13:53:43.761 INFO:teuthology.orchestra.run.smithi043.stdout: "mon": "allow rw", 2023-08-09T13:53:43.761 INFO:teuthology.orchestra.run.smithi043.stdout: "osd": "allow rwx" 2023-08-09T13:53:43.762 INFO:teuthology.orchestra.run.smithi043.stdout: } 2023-08-09T13:53:43.762 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2023-08-09T13:53:43.762 INFO:teuthology.orchestra.run.smithi043.stdout: { 2023-08-09T13:53:43.762 INFO:teuthology.orchestra.run.smithi043.stdout: "entity": "client.3", 2023-08-09T13:53:43.762 INFO:teuthology.orchestra.run.smithi043.stdout: "key": "AQCPldNkl+rXMhAA9rf3KdnSYbvngqmLYnIGGA==", 2023-08-09T13:53:43.762 INFO:teuthology.orchestra.run.smithi043.stdout: "caps": { 2023-08-09T13:53:43.762 INFO:teuthology.orchestra.run.smithi043.stdout: "mds": "allow", 2023-08-09T13:53:43.763 INFO:teuthology.orchestra.run.smithi043.stdout: "mgr": "allow r", 2023-08-09T13:53:43.763 INFO:teuthology.orchestra.run.smithi043.stdout: "mon": "allow rw", 2023-08-09T13:53:43.763 INFO:teuthology.orchestra.run.smithi043.stdout: "osd": "allow rwx" 2023-08-09T13:53:43.763 INFO:teuthology.orchestra.run.smithi043.stdout: } 2023-08-09T13:53:43.763 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2023-08-09T13:53:43.763 INFO:teuthology.orchestra.run.smithi043.stdout: { 2023-08-09T13:53:43.764 INFO:teuthology.orchestra.run.smithi043.stdout: "entity": "client.admin", 2023-08-09T13:53:43.764 INFO:teuthology.orchestra.run.smithi043.stdout: "key": "AQCMldNkUae6NRAA2FBTr0QPBdKNDoe1WK5zyg==", 2023-08-09T13:53:43.764 INFO:teuthology.orchestra.run.smithi043.stdout: "caps": { 2023-08-09T13:53:43.764 INFO:teuthology.orchestra.run.smithi043.stdout: "mds": "allow *", 2023-08-09T13:53:43.764 INFO:teuthology.orchestra.run.smithi043.stdout: "mgr": "allow *", 2023-08-09T13:53:43.764 INFO:teuthology.orchestra.run.smithi043.stdout: "mon": "allow *", 2023-08-09T13:53:43.765 INFO:teuthology.orchestra.run.smithi043.stdout: "osd": "allow *" 2023-08-09T13:53:43.765 INFO:teuthology.orchestra.run.smithi043.stdout: } 2023-08-09T13:53:43.765 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2023-08-09T13:53:43.765 INFO:teuthology.orchestra.run.smithi043.stdout: { 2023-08-09T13:53:43.765 INFO:teuthology.orchestra.run.smithi043.stdout: "entity": "mgr.x", 2023-08-09T13:53:43.765 INFO:teuthology.orchestra.run.smithi043.stdout: "key": "AQCNldNk2ufSNxAAU7msRkwLaoLZHTIsk0ck/g==", 2023-08-09T13:53:43.765 INFO:teuthology.orchestra.run.smithi043.stdout: "caps": { 2023-08-09T13:53:43.765 INFO:teuthology.orchestra.run.smithi043.stdout: "mds": "allow *", 2023-08-09T13:53:43.766 INFO:teuthology.orchestra.run.smithi043.stdout: "mon": "allow profile mgr", 2023-08-09T13:53:43.766 INFO:teuthology.orchestra.run.smithi043.stdout: "osd": "allow *" 2023-08-09T13:53:43.766 INFO:teuthology.orchestra.run.smithi043.stdout: } 2023-08-09T13:53:43.766 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2023-08-09T13:53:43.766 INFO:teuthology.orchestra.run.smithi043.stdout: { 2023-08-09T13:53:43.766 INFO:teuthology.orchestra.run.smithi043.stdout: "entity": "mgr.y", 2023-08-09T13:53:43.766 INFO:teuthology.orchestra.run.smithi043.stdout: "key": "AQCNldNk2YgzLBAAA11Q8uAMEgE/bUni+tXIKA==", 2023-08-09T13:53:43.766 INFO:teuthology.orchestra.run.smithi043.stdout: "caps": { 2023-08-09T13:53:43.767 INFO:teuthology.orchestra.run.smithi043.stdout: "mds": "allow *", 2023-08-09T13:53:43.767 INFO:teuthology.orchestra.run.smithi043.stdout: "mon": "allow profile mgr", 2023-08-09T13:53:43.767 INFO:teuthology.orchestra.run.smithi043.stdout: "osd": "allow *" 2023-08-09T13:53:43.767 INFO:teuthology.orchestra.run.smithi043.stdout: } 2023-08-09T13:53:43.767 INFO:teuthology.orchestra.run.smithi043.stdout: } 2023-08-09T13:53:43.767 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2023-08-09T13:53:43.767 INFO:teuthology.orchestra.run.smithi043.stdout:} 2023-08-09T13:53:43.769 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:43.742+0000 7fe933fff640 1 -- 172.21.15.43:0/3421411748 >> [v2:172.21.15.203:6832/2527059680,v1:172.21.15.203:6833/2527059680] conn(0x7fe92006a170 msgr2=0x7fe92006c630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:53:43.769 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:43.742+0000 7fe933fff640 1 --2- 172.21.15.43:0/3421411748 >> [v2:172.21.15.203:6832/2527059680,v1:172.21.15.203:6833/2527059680] conn(0x7fe92006a170 0x7fe92006c630 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7fe924000f60 tx=0x7fe924009040 comp rx=0 tx=0).stop 2023-08-09T13:53:43.769 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:43.742+0000 7fe933fff640 1 -- 172.21.15.43:0/3421411748 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fe9340c8b30 msgr2=0x7fe934158af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:53:43.769 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:43.742+0000 7fe933fff640 1 --2- 172.21.15.43:0/3421411748 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fe9340c8b30 0x7fe934158af0 secure :-1 s=READY pgs=1491 cs=0 l=1 rev1=1 crypto rx=0x7fe92c00cd50 tx=0x7fe92c00a650 comp rx=0 tx=0).stop 2023-08-09T13:53:43.769 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:43.742+0000 7fe933fff640 1 -- 172.21.15.43:0/3421411748 shutdown_connections 2023-08-09T13:53:43.769 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:43.742+0000 7fe933fff640 1 --2- 172.21.15.43:0/3421411748 >> [v2:172.21.15.203:6832/2527059680,v1:172.21.15.203:6833/2527059680] conn(0x7fe92006a170 0x7fe92006c630 unknown :-1 s=CLOSED pgs=47 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:53:43.770 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:43.742+0000 7fe933fff640 1 --2- 172.21.15.43:0/3421411748 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fe9340c8b30 0x7fe934158af0 unknown :-1 s=CLOSED pgs=1491 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:53:43.770 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:43.742+0000 7fe933fff640 1 --2- 172.21.15.43:0/3421411748 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fe9340c8750 0x7fe9341f23a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:53:43.770 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:43.742+0000 7fe933fff640 1 --2- 172.21.15.43:0/3421411748 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fe9340acdf0 0x7fe934159360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:53:43.770 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:43.742+0000 7fe933fff640 1 -- 172.21.15.43:0/3421411748 >> 172.21.15.43:0/3421411748 conn(0x7fe9340cde90 msgr2=0x7fe9340abe30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-09T13:53:43.770 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:43.743+0000 7fe933fff640 1 -- 172.21.15.43:0/3421411748 shutdown_connections 2023-08-09T13:53:43.770 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:43.743+0000 7fe933fff640 1 -- 172.21.15.43:0/3421411748 wait complete. 2023-08-09T13:53:43.771 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-08-09T13:53:43.992 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:43.989+0000 7fd7018cd640 1 Processor -- start 2023-08-09T13:53:43.992 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:43.989+0000 7fd7018cd640 1 -- start start 2023-08-09T13:53:43.994 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:43.991+0000 7fd7018cd640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fd6fc0c8750 0x7fd6fc0ad340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:53:43.994 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:43.991+0000 7fd7018cd640 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7fd6fc0c8b30 0x7fd6fc06f410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:53:43.994 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:43.991+0000 7fd7018cd640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7fd6fc0acdf0 0x7fd6fc070f00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:53:43.994 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:43.991+0000 7fd7018cd640 1 -- --> v2:172.21.15.43:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd6fc15e810 con 0x7fd6fc0acdf0 2023-08-09T13:53:43.995 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:43.991+0000 7fd7018cd640 1 -- --> v2:172.21.15.203:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd6fc0c9f70 con 0x7fd6fc0c8b30 2023-08-09T13:53:43.995 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:43.991+0000 7fd7018cd640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd6fc0cbd90 con 0x7fd6fc0c8750 2023-08-09T13:53:43.995 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:43.992+0000 7fd7010cc640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7fd6fc0acdf0 0x7fd6fc070f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:53:43.995 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:43.992+0000 7fd7010cc640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7fd6fc0acdf0 0x7fd6fc070f00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:53598/0 (socket says 172.21.15.43:53598) 2023-08-09T13:53:43.995 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:43.992+0000 7fd7010cc640 1 -- 172.21.15.43:0/1641554063 learned_addr learned my addr 172.21.15.43:0/1641554063 (peer_addr_for_me v2:172.21.15.43:0/0) 2023-08-09T13:53:43.996 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:43.992+0000 7fd7008cb640 1 --2- 172.21.15.43:0/1641554063 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fd6fc0c8750 0x7fd6fc0ad340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:53:43.996 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:43.992+0000 7fd7010cc640 1 -- 172.21.15.43:0/1641554063 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fd6fc0c8750 msgr2=0x7fd6fc0ad340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-09T13:53:43.997 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:43.992+0000 7fd7010cc640 1 --2- 172.21.15.43:0/1641554063 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fd6fc0c8750 0x7fd6fc0ad340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:53:43.997 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:43.992+0000 7fd6fbfff640 1 --2- 172.21.15.43:0/1641554063 >> v2:172.21.15.203:3300/0 conn(0x7fd6fc0c8b30 0x7fd6fc06f410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:53:43.997 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:43.992+0000 7fd7010cc640 1 -- 172.21.15.43:0/1641554063 >> v2:172.21.15.203:3300/0 conn(0x7fd6fc0c8b30 msgr2=0x7fd6fc06f410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-09T13:53:43.997 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:43.992+0000 7fd7010cc640 1 --2- 172.21.15.43:0/1641554063 >> v2:172.21.15.203:3300/0 conn(0x7fd6fc0c8b30 0x7fd6fc06f410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:53:43.997 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:43.992+0000 7fd7010cc640 1 -- 172.21.15.43:0/1641554063 --> v2:172.21.15.43:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd6fc0ca2b0 con 0x7fd6fc0acdf0 2023-08-09T13:53:43.997 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:43.992+0000 7fd7008cb640 1 --2- 172.21.15.43:0/1641554063 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fd6fc0c8750 0x7fd6fc0ad340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-08-09T13:53:43.997 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:43.992+0000 7fd6fbfff640 1 --2- 172.21.15.43:0/1641554063 >> v2:172.21.15.203:3300/0 conn(0x7fd6fc0c8b30 0x7fd6fc06f410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-08-09T13:53:43.998 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:43.993+0000 7fd7010cc640 1 --2- 172.21.15.43:0/1641554063 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fd6fc0acdf0 0x7fd6fc070f00 secure :-1 s=READY pgs=1492 cs=0 l=1 rev1=1 crypto rx=0x7fd6fc06de70 tx=0x7fd6f402bee0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5ce92668aa0e3b41 server_cookie=0 in_seq=0 out_seq=0 2023-08-09T13:53:43.998 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:43.994+0000 7fd6fb7fe640 1 -- 172.21.15.43:0/1641554063 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd6f402ca80 con 0x7fd6fc0acdf0 2023-08-09T13:53:43.998 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:43.994+0000 7fd6fb7fe640 1 -- 172.21.15.43:0/1641554063 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd6f402cbe0 con 0x7fd6fc0acdf0 2023-08-09T13:53:43.998 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:43.994+0000 7fd6fb7fe640 1 -- 172.21.15.43:0/1641554063 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd6f40343f0 con 0x7fd6fc0acdf0 2023-08-09T13:53:43.999 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:43.994+0000 7fd7018cd640 1 -- 172.21.15.43:0/1641554063 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fd6fc0acdf0 msgr2=0x7fd6fc070f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:53:43.999 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:43.994+0000 7fd7018cd640 1 --2- 172.21.15.43:0/1641554063 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fd6fc0acdf0 0x7fd6fc070f00 secure :-1 s=READY pgs=1492 cs=0 l=1 rev1=1 crypto rx=0x7fd6fc06de70 tx=0x7fd6f402bee0 comp rx=0 tx=0).stop 2023-08-09T13:53:43.999 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:43.995+0000 7fd7018cd640 1 -- 172.21.15.43:0/1641554063 shutdown_connections 2023-08-09T13:53:43.999 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:43.995+0000 7fd7018cd640 1 --2- 172.21.15.43:0/1641554063 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fd6fc0acdf0 0x7fd6fc070f00 unknown :-1 s=CLOSED pgs=1492 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:53:44.000 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:43.995+0000 7fd7018cd640 1 --2- 172.21.15.43:0/1641554063 >> v2:172.21.15.203:3300/0 conn(0x7fd6fc0c8b30 0x7fd6fc06f410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:53:44.000 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:43.995+0000 7fd7018cd640 1 --2- 172.21.15.43:0/1641554063 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fd6fc0c8750 0x7fd6fc0ad340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:53:44.000 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:43.995+0000 7fd7018cd640 1 -- 172.21.15.43:0/1641554063 >> 172.21.15.43:0/1641554063 conn(0x7fd6fc0cde90 msgr2=0x7fd6fc0ce2a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-09T13:53:44.000 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:43.995+0000 7fd7018cd640 1 -- 172.21.15.43:0/1641554063 shutdown_connections 2023-08-09T13:53:44.000 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:43.995+0000 7fd7018cd640 1 -- 172.21.15.43:0/1641554063 wait complete. 2023-08-09T13:53:44.000 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:43.997+0000 7fd7018cd640 1 Processor -- start 2023-08-09T13:53:44.000 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:43.997+0000 7fd7018cd640 1 -- start start 2023-08-09T13:53:44.002 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:43.998+0000 7fd7018cd640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fd6fc0acdf0 0x7fd6fc14e310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:53:44.002 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:43.998+0000 7fd7018cd640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fd6fc0c8750 0x7fd6fc098a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:53:44.002 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:43.998+0000 7fd7018cd640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fd6fc0c8b30 0x7fd6fc098f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:53:44.002 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:43.998+0000 7fd7018cd640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd6fc159730 con 0x7fd6fc0c8b30 2023-08-09T13:53:44.002 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:43.998+0000 7fd7018cd640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd6fc0c5760 con 0x7fd6fc0c8750 2023-08-09T13:53:44.003 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:43.998+0000 7fd7018cd640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd6fc14e850 con 0x7fd6fc0acdf0 2023-08-09T13:53:44.003 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:43.998+0000 7fd7010cc640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fd6fc0c8b30 0x7fd6fc098f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:53:44.003 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:43.998+0000 7fd7010cc640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fd6fc0c8b30 0x7fd6fc098f90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:53610/0 (socket says 172.21.15.43:53610) 2023-08-09T13:53:44.003 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:43.998+0000 7fd7010cc640 1 -- 172.21.15.43:0/2496385812 learned_addr learned my addr 172.21.15.43:0/2496385812 (peer_addr_for_me v2:172.21.15.43:0/0) 2023-08-09T13:53:44.003 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:43.999+0000 7fd7008cb640 1 --2- 172.21.15.43:0/2496385812 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fd6fc0acdf0 0x7fd6fc14e310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:53:44.003 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:43.999+0000 7fd6fbfff640 1 --2- 172.21.15.43:0/2496385812 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fd6fc0c8750 0x7fd6fc098a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:53:44.004 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:43.999+0000 7fd7010cc640 1 -- 172.21.15.43:0/2496385812 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fd6fc0acdf0 msgr2=0x7fd6fc14e310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:53:44.004 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:43.999+0000 7fd7010cc640 1 --2- 172.21.15.43:0/2496385812 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fd6fc0acdf0 0x7fd6fc14e310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:53:44.004 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:43.999+0000 7fd7010cc640 1 -- 172.21.15.43:0/2496385812 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fd6fc0c8750 msgr2=0x7fd6fc098a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:53:44.004 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:43.999+0000 7fd7010cc640 1 --2- 172.21.15.43:0/2496385812 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fd6fc0c8750 0x7fd6fc098a50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:53:44.004 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:43.999+0000 7fd7010cc640 1 -- 172.21.15.43:0/2496385812 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd6f4009b30 con 0x7fd6fc0c8b30 2023-08-09T13:53:44.005 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:43.999+0000 7fd7010cc640 1 --2- 172.21.15.43:0/2496385812 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fd6fc0c8b30 0x7fd6fc098f90 secure :-1 s=READY pgs=1493 cs=0 l=1 rev1=1 crypto rx=0x7fd6f4009e50 tx=0x7fd6f4004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-09T13:53:44.005 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:43.999+0000 7fd6f97fa640 1 -- 172.21.15.43:0/2496385812 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd6f4009510 con 0x7fd6fc0c8b30 2023-08-09T13:53:44.005 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:43.999+0000 7fd6f97fa640 1 -- 172.21.15.43:0/2496385812 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd6f403e450 con 0x7fd6fc0c8b30 2023-08-09T13:53:44.005 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:44.000+0000 7fd6f97fa640 1 -- 172.21.15.43:0/2496385812 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd6f403e800 con 0x7fd6fc0c8b30 2023-08-09T13:53:44.005 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:44.000+0000 7fd7018cd640 1 -- 172.21.15.43:0/2496385812 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd6fc0c5a00 con 0x7fd6fc0c8b30 2023-08-09T13:53:44.005 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:44.000+0000 7fd7018cd640 1 -- 172.21.15.43:0/2496385812 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd6fc0c5fc0 con 0x7fd6fc0c8b30 2023-08-09T13:53:44.006 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:44.001+0000 7fd7018cd640 1 -- 172.21.15.43:0/2496385812 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd6fc0c61e0 con 0x7fd6fc0c8b30 2023-08-09T13:53:44.007 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:44.002+0000 7fd6f97fa640 1 -- 172.21.15.43:0/2496385812 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 56) v1 ==== 88443+0+0 (secure 0 0 0) 0x7fd6f402cd50 con 0x7fd6fc0c8b30 2023-08-09T13:53:44.007 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:44.003+0000 7fd6f97fa640 1 --2- 172.21.15.43:0/2496385812 >> [v2:172.21.15.203:6832/2527059680,v1:172.21.15.203:6833/2527059680] conn(0x7fd6e806a1a0 0x7fd6e806c660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:53:44.007 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:44.003+0000 7fd6f97fa640 1 -- 172.21.15.43:0/2496385812 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(125..125 src has 1..125) v4 ==== 5574+0+0 (secure 0 0 0) 0x7fd6f40ab130 con 0x7fd6fc0c8b30 2023-08-09T13:53:44.007 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:44.004+0000 7fd7008cb640 1 --2- 172.21.15.43:0/2496385812 >> [v2:172.21.15.203:6832/2527059680,v1:172.21.15.203:6833/2527059680] conn(0x7fd6e806a1a0 0x7fd6e806c660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:53:44.007 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:44.005+0000 7fd7008cb640 1 --2- 172.21.15.43:0/2496385812 >> [v2:172.21.15.203:6832/2527059680,v1:172.21.15.203:6833/2527059680] conn(0x7fd6e806a1a0 0x7fd6e806c660 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7fd6f0008850 tx=0x7fd6f0008040 comp rx=0 tx=0).ready entity=mgr.6377 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-09T13:53:44.010 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:44.008+0000 7fd6f97fa640 1 -- 172.21.15.43:0/2496385812 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178066 (secure 0 0 0) 0x7fd6f4076900 con 0x7fd6fc0c8b30 2023-08-09T13:53:44.208 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:44.206+0000 7fd7018cd640 1 -- 172.21.15.43:0/2496385812 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fd6fc0739f0 con 0x7fd6fc0c8b30 2023-08-09T13:53:44.210 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:44.208+0000 7fd6f97fa640 1 -- 172.21.15.43:0/2496385812 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v125) v1 ==== 74+0+14854 (secure 0 0 0) 0x7fd6f4079f20 con 0x7fd6fc0c8b30 2023-08-09T13:53:44.211 INFO:teuthology.orchestra.run.smithi043.stdout: 2023-08-09T13:53:44.211 INFO:teuthology.orchestra.run.smithi043.stdout:{"epoch":125,"fsid":"daac87ee-1337-4250-a955-aea4e408a39b","created":"2023-08-09T13:33:41.701198+0000","modified":"2023-08-09T13:53:43.222773+0000","last_up_change":"2023-08-09T13:33:53.783453+0000","last_in_change":"2023-08-09T13:33:45.148904+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":15,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":"rbd","create_time":"2023-08-09T13:33:54.365137+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":".mgr","create_time":"2023-08-09T13:33:54.831803+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"cbd49ecd-e896-4fbd-864e-5c37a829b165","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":114,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6800","nonce":502667335},{"type":"v1","addr":"172.21.15.43:6801","nonce":502667335}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6802","nonce":502667335},{"type":"v1","addr":"172.21.15.43:6803","nonce":502667335}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6806","nonce":502667335},{"type":"v1","addr":"172.21.15.43:6807","nonce":502667335}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6804","nonce":502667335},{"type":"v1","addr":"172.21.15.43:6805","nonce":502667335}]},"public_addr":"172.21.15.43:6801/502667335","cluster_addr":"172.21.15.43:6803/502667335","heartbeat_back_addr":"172.21.15.43:6807/502667335","heartbeat_front_addr":"172.21.15.43:6805/502667335","state":["exists","up"]},{"osd":1,"uuid":"75c7e0fe-1ec7-4fe8-a2f2-3661a65b8500","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":114,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6808","nonce":3289134610},{"type":"v1","addr":"172.21.15.43:6809","nonce":3289134610}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6810","nonce":3289134610},{"type":"v1","addr":"172.21.15.43:6812","nonce":3289134610}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6821","nonce":3289134610},{"type":"v1","addr":"172.21.15.43:6824","nonce":3289134610}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6815","nonce":3289134610},{"type":"v1","addr":"172.21.15.43:6818","nonce":3289134610}]},"public_addr":"172.21.15.43:6809/3289134610","cluster_addr":"172.21.15.43:6812/3289134610","heartbeat_back_addr":"172.21.15.43:6824/3289134610","heartbeat_front_addr":"172.21.15.43:6818/3289134610","state":["exists","up"]},{"osd":2,"uuid":"bb5c41e9-7f5f-4125-9b01-3bb8bfc2ea83","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":114,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6811","nonce":4079709736},{"type":"v1","addr":"172.21.15.43:6814","nonce":4079709736}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6817","nonce":4079709736},{"type":"v1","addr":"172.21.15.43:6820","nonce":4079709736}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6829","nonce":4079709736},{"type":"v1","addr":"172.21.15.43:6831","nonce":4079709736}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6823","nonce":4079709736},{"type":"v1","addr":"172.21.15.43:6826","nonce":4079709736}]},"public_addr":"172.21.15.43:6814/4079709736","cluster_addr":"172.21.15.43:6820/4079709736","heartbeat_back_addr":"172.21.15.43:6831/4079709736","heartbeat_front_addr":"172.21.15.43:6826/4079709736","state":["exists","up"]},{"osd":3,"uuid":"bfe4c8b4-694a-465c-87e6-2b2aab852613","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":114,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6813","nonce":1709538759},{"type":"v1","addr":"172.21.15.43:6816","nonce":1709538759}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6819","nonce":1709538759},{"type":"v1","addr":"172.21.15.43:6822","nonce":1709538759}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6828","nonce":1709538759},{"type":"v1","addr":"172.21.15.43:6830","nonce":1709538759}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6825","nonce":1709538759},{"type":"v1","addr":"172.21.15.43:6827","nonce":1709538759}]},"public_addr":"172.21.15.43:6816/1709538759","cluster_addr":"172.21.15.43:6822/1709538759","heartbeat_back_addr":"172.21.15.43:6830/1709538759","heartbeat_front_addr":"172.21.15.43:6827/1709538759","state":["exists","up"]},{"osd":4,"uuid":"fa2e8a77-aba1-4756-853e-0d174a5bdeba","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":114,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6800","nonce":4196866481},{"type":"v1","addr":"172.21.15.203:6801","nonce":4196866481}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6802","nonce":4196866481},{"type":"v1","addr":"172.21.15.203:6803","nonce":4196866481}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6806","nonce":4196866481},{"type":"v1","addr":"172.21.15.203:6807","nonce":4196866481}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6804","nonce":4196866481},{"type":"v1","addr":"172.21.15.203:6805","nonce":4196866481}]},"public_addr":"172.21.15.203:6801/4196866481","cluster_addr":"172.21.15.203:6803/4196866481","heartbeat_back_addr":"172.21.15.203:6807/4196866481","heartbeat_front_addr":"172.21.15.203:6805/4196866481","state":["exists","up"]},{"osd":5,"uuid":"81811e4b-4d24-4f51-a82c-f1d700a2a00d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":114,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6816","nonce":3059290081},{"type":"v1","addr":"172.21.15.203:6817","nonce":3059290081}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6818","nonce":3059290081},{"type":"v1","addr":"172.21.15.203:6819","nonce":3059290081}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6822","nonce":3059290081},{"type":"v1","addr":"172.21.15.203:6823","nonce":3059290081}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6820","nonce":3059290081},{"type":"v1","addr":"172.21.15.203:6821","nonce":3059290081}]},"public_addr":"172.21.15.203:6817/3059290081","cluster_addr":"172.21.15.203:6819/3059290081","heartbeat_back_addr":"172.21.15.203:6823/3059290081","heartbeat_front_addr":"172.21.15.203:6821/3059290081","state":["exists","up"]},{"osd":6,"uuid":"307bac4d-6a5b-443d-af75-56d695d7674f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":114,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6824","nonce":512155569},{"type":"v1","addr":"172.21.15.203:6825","nonce":512155569}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6826","nonce":512155569},{"type":"v1","addr":"172.21.15.203:6827","nonce":512155569}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6830","nonce":512155569},{"type":"v1","addr":"172.21.15.203:6831","nonce":512155569}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6828","nonce":512155569},{"type":"v1","addr":"172.21.15.203:6829","nonce":512155569}]},"public_addr":"172.21.15.203:6825/512155569","cluster_addr":"172.21.15.203:6827/512155569","heartbeat_back_addr":"172.21.15.203:6831/512155569","heartbeat_front_addr":"172.21.15.203:6829/512155569","state":["exists","up"]},{"osd":7,"uuid":"a95dc7ca-9f1b-4295-af70-68a22a2d7d5e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":114,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6808","nonce":3331450438},{"type":"v1","addr":"172.21.15.203:6809","nonce":3331450438}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6810","nonce":3331450438},{"type":"v1","addr":"172.21.15.203:6811","nonce":3331450438}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6814","nonce":3331450438},{"type":"v1","addr":"172.21.15.203:6815","nonce":3331450438}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6812","nonce":3331450438},{"type":"v1","addr":"172.21.15.203:6813","nonce":3331450438}]},"public_addr":"172.21.15.203:6809/3331450438","cluster_addr":"172.21.15.203:6811/3331450438","heartbeat_back_addr":"172.21.15.203:6815/3331450438","heartbeat_front_addr":"172.21.15.203:6813/3331450438","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-09T13:33:50.547652+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-09T13:33:50.589469+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-09T13:33:50.561724+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-09T13:33:50.599650+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-09T13:33:50.411855+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-09T13:33:50.219681+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-09T13:33:50.504664+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-09T13:33:50.441248+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":7,"snaps":[{"begin":104,"length":1},{"begin":221,"length":1}]}],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-08-09T13:53:44.214 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:44.211+0000 7fd7018cd640 1 -- 172.21.15.43:0/2496385812 >> [v2:172.21.15.203:6832/2527059680,v1:172.21.15.203:6833/2527059680] conn(0x7fd6e806a1a0 msgr2=0x7fd6e806c660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:53:44.214 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:44.211+0000 7fd7018cd640 1 --2- 172.21.15.43:0/2496385812 >> [v2:172.21.15.203:6832/2527059680,v1:172.21.15.203:6833/2527059680] conn(0x7fd6e806a1a0 0x7fd6e806c660 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7fd6f0008850 tx=0x7fd6f0008040 comp rx=0 tx=0).stop 2023-08-09T13:53:44.215 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:44.211+0000 7fd7018cd640 1 -- 172.21.15.43:0/2496385812 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fd6fc0c8b30 msgr2=0x7fd6fc098f90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:53:44.215 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:44.211+0000 7fd7018cd640 1 --2- 172.21.15.43:0/2496385812 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fd6fc0c8b30 0x7fd6fc098f90 secure :-1 s=READY pgs=1493 cs=0 l=1 rev1=1 crypto rx=0x7fd6f4009e50 tx=0x7fd6f4004a40 comp rx=0 tx=0).stop 2023-08-09T13:53:44.215 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:44.212+0000 7fd7018cd640 1 -- 172.21.15.43:0/2496385812 shutdown_connections 2023-08-09T13:53:44.215 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:44.212+0000 7fd7018cd640 1 --2- 172.21.15.43:0/2496385812 >> [v2:172.21.15.203:6832/2527059680,v1:172.21.15.203:6833/2527059680] conn(0x7fd6e806a1a0 0x7fd6e806c660 unknown :-1 s=CLOSED pgs=48 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:53:44.215 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:44.212+0000 7fd7018cd640 1 --2- 172.21.15.43:0/2496385812 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fd6fc0c8b30 0x7fd6fc098f90 unknown :-1 s=CLOSED pgs=1493 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:53:44.216 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:44.212+0000 7fd7018cd640 1 --2- 172.21.15.43:0/2496385812 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fd6fc0c8750 0x7fd6fc098a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:53:44.216 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:44.212+0000 7fd7018cd640 1 --2- 172.21.15.43:0/2496385812 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fd6fc0acdf0 0x7fd6fc14e310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:53:44.216 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:44.212+0000 7fd7018cd640 1 -- 172.21.15.43:0/2496385812 >> 172.21.15.43:0/2496385812 conn(0x7fd6fc0cde90 msgr2=0x7fd6fc0cbf20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-09T13:53:44.216 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:44.212+0000 7fd7018cd640 1 -- 172.21.15.43:0/2496385812 shutdown_connections 2023-08-09T13:53:44.216 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:44.212+0000 7fd7018cd640 1 -- 172.21.15.43:0/2496385812 wait complete. 2023-08-09T13:53:44.238 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': 'rbd', 'create_time': '2023-08-09T13:33:54.365137+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}, 'read_balance': {'score_acting': 2, 'score_stable': 2, 'optimal_score': 1, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': '.mgr', 'create_time': '2023-08-09T13:33:54.831803+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-08-09T13:53:44.238 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get rbd pg_num 2023-08-09T13:53:44.468 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:44.465+0000 7f71af59e640 1 Processor -- start 2023-08-09T13:53:44.468 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:44.466+0000 7f71af59e640 1 -- start start 2023-08-09T13:53:44.470 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:44.467+0000 7f71af59e640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f71b00b0de0 0x7f71b00b1ee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:53:44.470 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:44.467+0000 7f71af59e640 1 -- --> v1:172.21.15.43:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f71b00b25e0 con 0x7f71b00b11c0 2023-08-09T13:53:44.470 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:44.468+0000 7f71af59e640 1 -- --> v1:172.21.15.203:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f71b00b28c0 con 0x7f71b0071280 2023-08-09T13:53:44.470 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:44.468+0000 7f71af59e640 1 -- --> v2:172.21.15.43:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f71b00b2ba0 con 0x7f71b00b0de0 2023-08-09T13:53:44.471 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:44.468+0000 7f71a5d9b640 1 --1- >> v1:172.21.15.43:6789/0 conn(0x7f71b00b11c0 0x7f71b0160650 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.43:6789/0 says I am v1:172.21.15.43:44818/0 (socket says 172.21.15.43:44818) 2023-08-09T13:53:44.471 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:44.468+0000 7f71a5d9b640 1 -- 172.21.15.43:0/1911130048 learned_addr learned my addr 172.21.15.43:0/1911130048 (peer_addr_for_me v1:172.21.15.43:0/0) 2023-08-09T13:53:44.471 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:44.468+0000 7f71ae59c640 1 --2- 172.21.15.43:0/1911130048 >> v2:172.21.15.43:3300/0 conn(0x7f71b00b0de0 0x7f71b00b1ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:53:44.471 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:44.468+0000 7f71ae59c640 1 -- 172.21.15.43:0/1911130048 >> v1:172.21.15.203:6789/0 conn(0x7f71b0071280 legacy=0x7f71b006d750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-09T13:53:44.471 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:44.468+0000 7f71ae59c640 1 -- 172.21.15.43:0/1911130048 >> v1:172.21.15.43:6789/0 conn(0x7f71b00b11c0 legacy=0x7f71b0160650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:53:44.472 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:44.468+0000 7f71add9b640 1 -- 172.21.15.43:0/1911130048 <== mon.0 v1:172.21.15.43:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2122359179 0 0) 0x7f71b00b25e0 con 0x7f71b00b11c0 2023-08-09T13:53:44.472 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:44.468+0000 7f71ae59c640 1 -- 172.21.15.43:0/1911130048 --> v2:172.21.15.43:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f71b015fe50 con 0x7f71b00b0de0 2023-08-09T13:53:44.472 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:44.470+0000 7f71ae59c640 1 --2- 172.21.15.43:0/1911130048 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f71b00b0de0 0x7f71b00b1ee0 secure :-1 s=READY pgs=1495 cs=0 l=1 rev1=1 crypto rx=0x7f71a00097a0 tx=0x7f71a002b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e87bde42968e076 server_cookie=0 in_seq=0 out_seq=0 2023-08-09T13:53:44.472 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:44.470+0000 7f71add9b640 1 -- 172.21.15.43:0/1911130048 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f71a0031830 con 0x7f71b00b0de0 2023-08-09T13:53:44.473 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:44.470+0000 7f71add9b640 1 -- 172.21.15.43:0/1911130048 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f71a0031990 con 0x7f71b00b0de0 2023-08-09T13:53:44.473 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:44.470+0000 7f71add9b640 1 -- 172.21.15.43:0/1911130048 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f71a0031d20 con 0x7f71b00b0de0 2023-08-09T13:53:44.474 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:44.471+0000 7f71af59e640 1 -- 172.21.15.43:0/1911130048 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f71b00b0de0 msgr2=0x7f71b00b1ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:53:44.474 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:44.471+0000 7f71af59e640 1 --2- 172.21.15.43:0/1911130048 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f71b00b0de0 0x7f71b00b1ee0 secure :-1 s=READY pgs=1495 cs=0 l=1 rev1=1 crypto rx=0x7f71a00097a0 tx=0x7f71a002b8d0 comp rx=0 tx=0).stop 2023-08-09T13:53:44.474 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:44.471+0000 7f71af59e640 1 -- 172.21.15.43:0/1911130048 shutdown_connections 2023-08-09T13:53:44.474 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:44.471+0000 7f71af59e640 1 --2- 172.21.15.43:0/1911130048 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f71b00b0de0 0x7f71b00b1ee0 unknown :-1 s=CLOSED pgs=1495 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:53:44.474 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:44.471+0000 7f71af59e640 1 -- 172.21.15.43:0/1911130048 >> 172.21.15.43:0/1911130048 conn(0x7f71b006cd10 msgr2=0x7f71b006a8d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-09T13:53:44.474 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:44.471+0000 7f71af59e640 1 -- 172.21.15.43:0/1911130048 shutdown_connections 2023-08-09T13:53:44.475 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:44.471+0000 7f71af59e640 1 -- 172.21.15.43:0/1911130048 wait complete. 2023-08-09T13:53:44.475 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:44.473+0000 7f71af59e640 1 Processor -- start 2023-08-09T13:53:44.476 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:44.473+0000 7f71af59e640 1 -- start start 2023-08-09T13:53:44.477 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:44.475+0000 7f71af59e640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f71b0071280 0x7f71b0157c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:53:44.477 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:44.475+0000 7f71af59e640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f71b00b0de0 0x7f71b01f1ba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:53:44.478 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:44.475+0000 7f71af59e640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f71b00b11c0 0x7f71b01f20e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:53:44.478 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:44.475+0000 7f71af59e640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f71b00cc3d0 con 0x7f71b00b0de0 2023-08-09T13:53:44.478 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:44.475+0000 7f71af59e640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f71b015af50 con 0x7f71b00b11c0 2023-08-09T13:53:44.478 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:44.475+0000 7f71af59e640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f71b015b230 con 0x7f71b0071280 2023-08-09T13:53:44.478 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:44.475+0000 7f71a5d9b640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f71b00b0de0 0x7f71b01f1ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:53:44.478 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:44.475+0000 7f71a5d9b640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f71b00b0de0 0x7f71b01f1ba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:53624/0 (socket says 172.21.15.43:53624) 2023-08-09T13:53:44.478 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:44.475+0000 7f71a5d9b640 1 -- 172.21.15.43:0/2880998536 learned_addr learned my addr 172.21.15.43:0/2880998536 (peer_addr_for_me v2:172.21.15.43:0/0) 2023-08-09T13:53:44.479 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:44.475+0000 7f71ae59c640 1 --2- 172.21.15.43:0/2880998536 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f71b0071280 0x7f71b0157c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:53:44.480 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:44.475+0000 7f71aed9d640 1 --2- 172.21.15.43:0/2880998536 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f71b00b11c0 0x7f71b01f20e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:53:44.480 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:44.476+0000 7f71a5d9b640 1 -- 172.21.15.43:0/2880998536 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f71b0071280 msgr2=0x7f71b0157c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:53:44.480 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:44.476+0000 7f71a5d9b640 1 --2- 172.21.15.43:0/2880998536 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f71b0071280 0x7f71b0157c60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:53:44.480 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:44.476+0000 7f71a5d9b640 1 -- 172.21.15.43:0/2880998536 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f71b00b11c0 msgr2=0x7f71b01f20e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:53:44.480 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:44.476+0000 7f71a5d9b640 1 --2- 172.21.15.43:0/2880998536 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f71b00b11c0 0x7f71b01f20e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:53:44.480 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:44.476+0000 7f71a5d9b640 1 -- 172.21.15.43:0/2880998536 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f71a0009400 con 0x7f71b00b0de0 2023-08-09T13:53:44.480 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:44.476+0000 7f71ae59c640 1 --2- 172.21.15.43:0/2880998536 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f71b0071280 0x7f71b0157c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-08-09T13:53:44.481 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:44.476+0000 7f71a5d9b640 1 --2- 172.21.15.43:0/2880998536 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f71b00b0de0 0x7f71b01f1ba0 secure :-1 s=READY pgs=1496 cs=0 l=1 rev1=1 crypto rx=0x7f719800c430 tx=0x7f719800c8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-09T13:53:44.481 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:44.476+0000 7f71a77fe640 1 -- 172.21.15.43:0/2880998536 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f71980074e0 con 0x7f71b00b0de0 2023-08-09T13:53:44.481 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:44.476+0000 7f71a77fe640 1 -- 172.21.15.43:0/2880998536 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7198007660 con 0x7f71b00b0de0 2023-08-09T13:53:44.481 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:44.476+0000 7f71a77fe640 1 -- 172.21.15.43:0/2880998536 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7198004a20 con 0x7f71b00b0de0 2023-08-09T13:53:44.481 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:44.476+0000 7f71af59e640 1 -- 172.21.15.43:0/2880998536 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f71b00cc620 con 0x7f71b00b0de0 2023-08-09T13:53:44.481 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:44.477+0000 7f71af59e640 1 -- 172.21.15.43:0/2880998536 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f71b015eae0 con 0x7f71b00b0de0 2023-08-09T13:53:44.482 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:44.478+0000 7f71af59e640 1 -- 172.21.15.43:0/2880998536 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f71b004e390 con 0x7f71b00b0de0 2023-08-09T13:53:44.482 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:44.479+0000 7f71a77fe640 1 -- 172.21.15.43:0/2880998536 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 56) v1 ==== 88443+0+0 (secure 0 0 0) 0x7f7198004b80 con 0x7f71b00b0de0 2023-08-09T13:53:44.482 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:44.479+0000 7f71a77fe640 1 --2- 172.21.15.43:0/2880998536 >> [v2:172.21.15.203:6832/2527059680,v1:172.21.15.203:6833/2527059680] conn(0x7f719406a1a0 0x7f719406c660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:53:44.482 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:44.480+0000 7f71a77fe640 1 -- 172.21.15.43:0/2880998536 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(125..125 src has 1..125) v4 ==== 5574+0+0 (secure 0 0 0) 0x7f7198088300 con 0x7f71b00b0de0 2023-08-09T13:53:44.483 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:44.480+0000 7f71ae59c640 1 --2- 172.21.15.43:0/2880998536 >> [v2:172.21.15.203:6832/2527059680,v1:172.21.15.203:6833/2527059680] conn(0x7f719406a1a0 0x7f719406c660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:53:44.484 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:44.481+0000 7f71ae59c640 1 --2- 172.21.15.43:0/2880998536 >> [v2:172.21.15.203:6832/2527059680,v1:172.21.15.203:6833/2527059680] conn(0x7f719406a1a0 0x7f719406c660 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7f71a0036000 tx=0x7f71a0008030 comp rx=0 tx=0).ready entity=mgr.6377 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-09T13:53:44.486 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:44.484+0000 7f71a77fe640 1 -- 172.21.15.43:0/2880998536 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178066 (secure 0 0 0) 0x7f7198053ab0 con 0x7f71b00b0de0 2023-08-09T13:53:44.675 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:44.673+0000 7f71af59e640 1 -- 172.21.15.43:0/2880998536 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"} v 0) v1 -- 0x7f71b015fbc0 con 0x7f71b00b0de0 2023-08-09T13:53:44.676 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:44.674+0000 7f71a77fe640 1 -- 172.21.15.43:0/2880998536 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"}]=0 v125) v1 ==== 92+0+10 (secure 0 0 0) 0x7f71980570d0 con 0x7f71b00b0de0 2023-08-09T13:53:44.677 INFO:teuthology.orchestra.run.smithi043.stdout:pg_num: 8 2023-08-09T13:53:44.679 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:44.676+0000 7f71af59e640 1 -- 172.21.15.43:0/2880998536 >> [v2:172.21.15.203:6832/2527059680,v1:172.21.15.203:6833/2527059680] conn(0x7f719406a1a0 msgr2=0x7f719406c660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:53:44.680 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:44.676+0000 7f71af59e640 1 --2- 172.21.15.43:0/2880998536 >> [v2:172.21.15.203:6832/2527059680,v1:172.21.15.203:6833/2527059680] conn(0x7f719406a1a0 0x7f719406c660 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7f71a0036000 tx=0x7f71a0008030 comp rx=0 tx=0).stop 2023-08-09T13:53:44.680 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:44.676+0000 7f71af59e640 1 -- 172.21.15.43:0/2880998536 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f71b00b0de0 msgr2=0x7f71b01f1ba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:53:44.680 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:44.676+0000 7f71af59e640 1 --2- 172.21.15.43:0/2880998536 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f71b00b0de0 0x7f71b01f1ba0 secure :-1 s=READY pgs=1496 cs=0 l=1 rev1=1 crypto rx=0x7f719800c430 tx=0x7f719800c8f0 comp rx=0 tx=0).stop 2023-08-09T13:53:44.680 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:44.677+0000 7f71af59e640 1 -- 172.21.15.43:0/2880998536 shutdown_connections 2023-08-09T13:53:44.680 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:44.677+0000 7f71af59e640 1 --2- 172.21.15.43:0/2880998536 >> [v2:172.21.15.203:6832/2527059680,v1:172.21.15.203:6833/2527059680] conn(0x7f719406a1a0 0x7f719406c660 unknown :-1 s=CLOSED pgs=52 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:53:44.680 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:44.677+0000 7f71af59e640 1 --2- 172.21.15.43:0/2880998536 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f71b00b11c0 0x7f71b01f20e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:53:44.680 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:44.677+0000 7f71af59e640 1 --2- 172.21.15.43:0/2880998536 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f71b00b0de0 0x7f71b01f1ba0 unknown :-1 s=CLOSED pgs=1496 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:53:44.681 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:44.677+0000 7f71af59e640 1 --2- 172.21.15.43:0/2880998536 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f71b0071280 0x7f71b0157c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:53:44.681 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:44.677+0000 7f71af59e640 1 -- 172.21.15.43:0/2880998536 >> 172.21.15.43:0/2880998536 conn(0x7f71b006cd10 msgr2=0x7f71b00cef70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-09T13:53:44.681 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:44.677+0000 7f71af59e640 1 -- 172.21.15.43:0/2880998536 shutdown_connections 2023-08-09T13:53:44.681 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:44.677+0000 7f71af59e640 1 -- 172.21.15.43:0/2880998536 wait complete. 2023-08-09T13:53:44.696 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-08-09T13:53:44.927 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:44.924+0000 7fa3fe2d7640 1 Processor -- start 2023-08-09T13:53:44.927 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:44.924+0000 7fa3fe2d7640 1 -- start start 2023-08-09T13:53:44.928 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:44.926+0000 7fa3fe2d7640 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7fa3f8157ea0 0x7fa3f80c97d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:53:44.929 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:44.926+0000 7fa3fe2d7640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7fa3f80c90a0 0x7fa3f80c9d10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:53:44.929 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:44.926+0000 7fa3fe2d7640 1 -- --> v1:172.21.15.43:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa3f8158280 con 0x7fa3f815a800 2023-08-09T13:53:44.929 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:44.926+0000 7fa3fe2d7640 1 -- --> v2:172.21.15.43:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa3f815b840 con 0x7fa3f80c90a0 2023-08-09T13:53:44.929 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:44.926+0000 7fa3fe2d7640 1 -- --> v2:172.21.15.203:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa3f80b3810 con 0x7fa3f8157ea0 2023-08-09T13:53:44.929 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:44.926+0000 7fa3fcad4640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7fa3f80c90a0 0x7fa3f80c9d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:53:44.929 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:44.926+0000 7fa3fdad6640 1 --1- >> v1:172.21.15.43:6789/0 conn(0x7fa3f815a800 0x7fa3f815ac90 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.43:6789/0 says I am v1:172.21.15.43:44828/0 (socket says 172.21.15.43:44828) 2023-08-09T13:53:44.930 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:44.926+0000 7fa3fdad6640 1 -- 172.21.15.43:0/953422407 learned_addr learned my addr 172.21.15.43:0/953422407 (peer_addr_for_me v1:172.21.15.43:0/0) 2023-08-09T13:53:44.930 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:44.927+0000 7fa3fd2d5640 1 --2- 172.21.15.43:0/953422407 >> v2:172.21.15.203:3300/0 conn(0x7fa3f8157ea0 0x7fa3f80c97d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:53:44.930 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:44.927+0000 7fa3fcad4640 1 -- 172.21.15.43:0/953422407 >> v2:172.21.15.203:3300/0 conn(0x7fa3f8157ea0 msgr2=0x7fa3f80c97d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-09T13:53:44.931 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:44.927+0000 7fa3fcad4640 1 --2- 172.21.15.43:0/953422407 >> v2:172.21.15.203:3300/0 conn(0x7fa3f8157ea0 0x7fa3f80c97d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:53:44.931 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:44.927+0000 7fa3fcad4640 1 -- 172.21.15.43:0/953422407 >> v1:172.21.15.43:6789/0 conn(0x7fa3f815a800 legacy=0x7fa3f815ac90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:53:44.931 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:44.927+0000 7fa3fcad4640 1 -- 172.21.15.43:0/953422407 --> v2:172.21.15.43:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa3f815b560 con 0x7fa3f80c90a0 2023-08-09T13:53:44.931 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:44.928+0000 7fa3fcad4640 1 --2- 172.21.15.43:0/953422407 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa3f80c90a0 0x7fa3f80c9d10 secure :-1 s=READY pgs=1498 cs=0 l=1 rev1=1 crypto rx=0x7fa3f806ad20 tx=0x7fa3e802b840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9408010462d7fc30 server_cookie=0 in_seq=0 out_seq=0 2023-08-09T13:53:44.932 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:44.928+0000 7fa3effff640 1 -- 172.21.15.43:0/953422407 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa3e802e070 con 0x7fa3f80c90a0 2023-08-09T13:53:44.932 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:44.929+0000 7fa3effff640 1 -- 172.21.15.43:0/953422407 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa3e80315a0 con 0x7fa3f80c90a0 2023-08-09T13:53:44.933 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:44.929+0000 7fa3effff640 1 -- 172.21.15.43:0/953422407 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa3e8031930 con 0x7fa3f80c90a0 2023-08-09T13:53:44.933 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:44.929+0000 7fa3fe2d7640 1 -- 172.21.15.43:0/953422407 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa3f80c90a0 msgr2=0x7fa3f80c9d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:53:44.933 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:44.929+0000 7fa3fe2d7640 1 --2- 172.21.15.43:0/953422407 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa3f80c90a0 0x7fa3f80c9d10 secure :-1 s=READY pgs=1498 cs=0 l=1 rev1=1 crypto rx=0x7fa3f806ad20 tx=0x7fa3e802b840 comp rx=0 tx=0).stop 2023-08-09T13:53:44.933 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:44.929+0000 7fa3fe2d7640 1 -- 172.21.15.43:0/953422407 shutdown_connections 2023-08-09T13:53:44.933 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:44.929+0000 7fa3fe2d7640 1 --2- 172.21.15.43:0/953422407 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa3f80c90a0 0x7fa3f80c9d10 unknown :-1 s=CLOSED pgs=1498 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:53:44.933 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:44.929+0000 7fa3fe2d7640 1 --2- 172.21.15.43:0/953422407 >> v2:172.21.15.203:3300/0 conn(0x7fa3f8157ea0 0x7fa3f80c97d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:53:44.934 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:44.929+0000 7fa3fe2d7640 1 -- 172.21.15.43:0/953422407 >> 172.21.15.43:0/953422407 conn(0x7fa3f80cddd0 msgr2=0x7fa3f80ce1e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-09T13:53:44.934 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:44.929+0000 7fa3fe2d7640 1 -- 172.21.15.43:0/953422407 shutdown_connections 2023-08-09T13:53:44.934 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:44.929+0000 7fa3fe2d7640 1 -- 172.21.15.43:0/953422407 wait complete. 2023-08-09T13:53:44.935 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:44.931+0000 7fa3fe2d7640 1 Processor -- start 2023-08-09T13:53:44.935 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:44.931+0000 7fa3fe2d7640 1 -- start start 2023-08-09T13:53:44.935 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:44.932+0000 7fa3fe2d7640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa3f80c90a0 0x7fa3f81f1560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:53:44.935 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:44.932+0000 7fa3fe2d7640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fa3f8157ea0 0x7fa3f80cc460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:53:44.936 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:44.932+0000 7fa3fe2d7640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fa3f815a800 0x7fa3f80c66a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:53:44.936 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:44.932+0000 7fa3fe2d7640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa3f80cb9a0 con 0x7fa3f80c90a0 2023-08-09T13:53:44.936 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:44.932+0000 7fa3fe2d7640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa3f80c7080 con 0x7fa3f8157ea0 2023-08-09T13:53:44.936 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:44.932+0000 7fa3fe2d7640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa3f80c7360 con 0x7fa3f815a800 2023-08-09T13:53:44.937 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:44.932+0000 7fa3fd2d5640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa3f80c90a0 0x7fa3f81f1560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:53:44.937 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:44.933+0000 7fa3fd2d5640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa3f80c90a0 0x7fa3f81f1560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:53630/0 (socket says 172.21.15.43:53630) 2023-08-09T13:53:44.937 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:44.933+0000 7fa3fd2d5640 1 -- 172.21.15.43:0/1524962096 learned_addr learned my addr 172.21.15.43:0/1524962096 (peer_addr_for_me v2:172.21.15.43:0/0) 2023-08-09T13:53:44.938 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:44.933+0000 7fa3fdad6640 1 --2- 172.21.15.43:0/1524962096 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fa3f815a800 0x7fa3f80c66a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:53:44.938 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:44.933+0000 7fa3fd2d5640 1 -- 172.21.15.43:0/1524962096 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fa3f815a800 msgr2=0x7fa3f80c66a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:53:44.938 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:44.933+0000 7fa3fd2d5640 1 --2- 172.21.15.43:0/1524962096 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fa3f815a800 0x7fa3f80c66a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:53:44.939 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:44.933+0000 7fa3fcad4640 1 --2- 172.21.15.43:0/1524962096 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fa3f8157ea0 0x7fa3f80cc460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:53:44.939 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:44.933+0000 7fa3fd2d5640 1 -- 172.21.15.43:0/1524962096 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fa3f8157ea0 msgr2=0x7fa3f80cc460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:53:44.939 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:44.933+0000 7fa3fd2d5640 1 --2- 172.21.15.43:0/1524962096 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fa3f8157ea0 0x7fa3f80cc460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:53:44.939 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:44.933+0000 7fa3fd2d5640 1 -- 172.21.15.43:0/1524962096 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa3e8009400 con 0x7fa3f80c90a0 2023-08-09T13:53:44.939 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:44.933+0000 7fa3fd2d5640 1 --2- 172.21.15.43:0/1524962096 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa3f80c90a0 0x7fa3f81f1560 secure :-1 s=READY pgs=1499 cs=0 l=1 rev1=1 crypto rx=0x7fa3f400d800 tx=0x7fa3f400dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-09T13:53:44.939 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:44.934+0000 7fa3edffb640 1 -- 172.21.15.43:0/1524962096 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa3f400b6c0 con 0x7fa3f80c90a0 2023-08-09T13:53:44.939 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:44.934+0000 7fa3edffb640 1 -- 172.21.15.43:0/1524962096 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa3f400b820 con 0x7fa3f80c90a0 2023-08-09T13:53:44.940 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:44.934+0000 7fa3edffb640 1 -- 172.21.15.43:0/1524962096 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa3f400bbb0 con 0x7fa3f80c90a0 2023-08-09T13:53:44.940 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:44.934+0000 7fa3fe2d7640 1 -- 172.21.15.43:0/1524962096 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa3f80cbc60 con 0x7fa3f80c90a0 2023-08-09T13:53:44.940 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:44.934+0000 7fa3fe2d7640 1 -- 172.21.15.43:0/1524962096 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa3f80aeee0 con 0x7fa3f80c90a0 2023-08-09T13:53:44.940 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:44.935+0000 7fa3fe2d7640 1 -- 172.21.15.43:0/1524962096 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa3f80c7520 con 0x7fa3f80c90a0 2023-08-09T13:53:44.941 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:44.936+0000 7fa3edffb640 1 -- 172.21.15.43:0/1524962096 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 56) v1 ==== 88443+0+0 (secure 0 0 0) 0x7fa3f400bd10 con 0x7fa3f80c90a0 2023-08-09T13:53:44.941 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:44.937+0000 7fa3edffb640 1 --2- 172.21.15.43:0/1524962096 >> [v2:172.21.15.203:6832/2527059680,v1:172.21.15.203:6833/2527059680] conn(0x7fa3dc06a170 0x7fa3dc06c630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:53:44.941 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:44.937+0000 7fa3edffb640 1 -- 172.21.15.43:0/1524962096 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(125..125 src has 1..125) v4 ==== 5574+0+0 (secure 0 0 0) 0x7fa3f4088b50 con 0x7fa3f80c90a0 2023-08-09T13:53:44.941 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:44.938+0000 7fa3fcad4640 1 --2- 172.21.15.43:0/1524962096 >> [v2:172.21.15.203:6832/2527059680,v1:172.21.15.203:6833/2527059680] conn(0x7fa3dc06a170 0x7fa3dc06c630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:53:44.942 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:44.939+0000 7fa3fcad4640 1 --2- 172.21.15.43:0/1524962096 >> [v2:172.21.15.203:6832/2527059680,v1:172.21.15.203:6833/2527059680] conn(0x7fa3dc06a170 0x7fa3dc06c630 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7fa3e8036000 tx=0x7fa3e803d040 comp rx=0 tx=0).ready entity=mgr.6377 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-09T13:53:44.944 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:44.941+0000 7fa3edffb640 1 -- 172.21.15.43:0/1524962096 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178066 (secure 0 0 0) 0x7fa3f4054270 con 0x7fa3f80c90a0 2023-08-09T13:53:45.138 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:45.136+0000 7fa3fe2d7640 1 -- 172.21.15.43:0/1524962096 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7fa3f804e390 con 0x7fa3f80c90a0 2023-08-09T13:53:45.139 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:45.137+0000 7fa3edffb640 1 -- 172.21.15.43:0/1524962096 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v125) v1 ==== 93+0+10 (secure 0 0 0) 0x7fa3f4057890 con 0x7fa3f80c90a0 2023-08-09T13:53:45.140 INFO:teuthology.orchestra.run.smithi043.stdout:pg_num: 1 2023-08-09T13:53:45.142 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:45.139+0000 7fa3fe2d7640 1 -- 172.21.15.43:0/1524962096 >> [v2:172.21.15.203:6832/2527059680,v1:172.21.15.203:6833/2527059680] conn(0x7fa3dc06a170 msgr2=0x7fa3dc06c630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:53:45.142 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:45.139+0000 7fa3fe2d7640 1 --2- 172.21.15.43:0/1524962096 >> [v2:172.21.15.203:6832/2527059680,v1:172.21.15.203:6833/2527059680] conn(0x7fa3dc06a170 0x7fa3dc06c630 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7fa3e8036000 tx=0x7fa3e803d040 comp rx=0 tx=0).stop 2023-08-09T13:53:45.142 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:45.139+0000 7fa3fe2d7640 1 -- 172.21.15.43:0/1524962096 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa3f80c90a0 msgr2=0x7fa3f81f1560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:53:45.143 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:45.139+0000 7fa3fe2d7640 1 --2- 172.21.15.43:0/1524962096 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa3f80c90a0 0x7fa3f81f1560 secure :-1 s=READY pgs=1499 cs=0 l=1 rev1=1 crypto rx=0x7fa3f400d800 tx=0x7fa3f400dcc0 comp rx=0 tx=0).stop 2023-08-09T13:53:45.143 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:45.140+0000 7fa3fe2d7640 1 -- 172.21.15.43:0/1524962096 shutdown_connections 2023-08-09T13:53:45.143 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:45.140+0000 7fa3fe2d7640 1 --2- 172.21.15.43:0/1524962096 >> [v2:172.21.15.203:6832/2527059680,v1:172.21.15.203:6833/2527059680] conn(0x7fa3dc06a170 0x7fa3dc06c630 unknown :-1 s=CLOSED pgs=53 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:53:45.143 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:45.140+0000 7fa3fe2d7640 1 --2- 172.21.15.43:0/1524962096 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fa3f815a800 0x7fa3f80c66a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:53:45.143 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:45.140+0000 7fa3fe2d7640 1 --2- 172.21.15.43:0/1524962096 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fa3f8157ea0 0x7fa3f80cc460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:53:45.143 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:45.140+0000 7fa3fe2d7640 1 --2- 172.21.15.43:0/1524962096 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa3f80c90a0 0x7fa3f81f1560 unknown :-1 s=CLOSED pgs=1499 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:53:45.144 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:45.140+0000 7fa3fe2d7640 1 -- 172.21.15.43:0/1524962096 >> 172.21.15.43:0/1524962096 conn(0x7fa3f80cddd0 msgr2=0x7fa3f806c1a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-09T13:53:45.144 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:45.140+0000 7fa3fe2d7640 1 -- 172.21.15.43:0/1524962096 shutdown_connections 2023-08-09T13:53:45.144 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:45.140+0000 7fa3fe2d7640 1 -- 172.21.15.43:0/1524962096 wait complete. 2023-08-09T13:53:45.162 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --format=json-pretty osd lspools 2023-08-09T13:53:45.384 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:45.382+0000 7fd7b3437640 1 Processor -- start 2023-08-09T13:53:45.385 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:45.382+0000 7fd7b3437640 1 -- start start 2023-08-09T13:53:45.386 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:45.384+0000 7fd7b3437640 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7fd7ac15f7b0 0x7fd7ac0b4760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:53:45.386 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:45.384+0000 7fd7b3437640 1 -- --> v1:172.21.15.43:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd7ac0b7bf0 con 0x7fd7ac0b1050 2023-08-09T13:53:45.387 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:45.384+0000 7fd7b3437640 1 -- --> v1:172.21.15.203:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd7ac0b7ed0 con 0x7fd7ac0cd6b0 2023-08-09T13:53:45.387 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:45.384+0000 7fd7b3437640 1 -- --> v2:172.21.15.203:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd7ac0d0370 con 0x7fd7ac15f7b0 2023-08-09T13:53:45.387 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:45.384+0000 7fd7b1c34640 1 --1- >> v1:172.21.15.43:6789/0 conn(0x7fd7ac0b1050 0x7fd7ac0ccfa0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.43:6789/0 says I am v1:172.21.15.43:53602/0 (socket says 172.21.15.43:53602) 2023-08-09T13:53:45.387 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:45.384+0000 7fd7b1c34640 1 -- 172.21.15.43:0/1135229476 learned_addr learned my addr 172.21.15.43:0/1135229476 (peer_addr_for_me v1:172.21.15.43:0/0) 2023-08-09T13:53:45.388 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:45.385+0000 7fd7b2435640 1 --2- 172.21.15.43:0/1135229476 >> v2:172.21.15.203:3300/0 conn(0x7fd7ac15f7b0 0x7fd7ac0b4760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:53:45.388 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:45.385+0000 7fd7b1433640 1 -- 172.21.15.43:0/1135229476 <== mon.0 v1:172.21.15.43:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3157130169 0 0) 0x7fd7ac0b7bf0 con 0x7fd7ac0b1050 2023-08-09T13:53:45.388 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:45.385+0000 7fd7b1433640 1 -- 172.21.15.43:0/1135229476 --> v1:172.21.15.43:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fd798003520 con 0x7fd7ac0b1050 2023-08-09T13:53:45.389 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:45.385+0000 7fd7b1433640 1 -- 172.21.15.43:0/1135229476 <== mon.0 v1:172.21.15.43:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 233910547 0 0) 0x7fd798003520 con 0x7fd7ac0b1050 2023-08-09T13:53:45.389 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:45.385+0000 7fd7b1433640 1 -- 172.21.15.43:0/1135229476 >> v2:172.21.15.203:3300/0 conn(0x7fd7ac15f7b0 msgr2=0x7fd7ac0b4760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-09T13:53:45.390 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:45.385+0000 7fd7b1433640 1 --2- 172.21.15.43:0/1135229476 >> v2:172.21.15.203:3300/0 conn(0x7fd7ac15f7b0 0x7fd7ac0b4760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:53:45.390 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:45.385+0000 7fd7b1433640 1 -- 172.21.15.43:0/1135229476 >> v1:172.21.15.203:6789/0 conn(0x7fd7ac0cd6b0 legacy=0x7fd7ac0d0e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:53:45.390 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:45.385+0000 7fd7b1433640 1 -- 172.21.15.43:0/1135229476 --> v1:172.21.15.43:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd7ac0d0650 con 0x7fd7ac0b1050 2023-08-09T13:53:45.390 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:45.385+0000 7fd7b1433640 1 -- 172.21.15.43:0/1135229476 <== mon.0 v1:172.21.15.43:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1285150740 0 0) 0x7fd79c001670 con 0x7fd7ac0b1050 2023-08-09T13:53:45.390 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:45.386+0000 7fd7b1433640 1 -- 172.21.15.43:0/1135229476 >> v1:172.21.15.43:6789/0 conn(0x7fd7ac0b1050 legacy=0x7fd7ac0ccfa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:53:45.390 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:45.386+0000 7fd7b1433640 1 --2- 172.21.15.43:0/1135229476 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fd798003970 0x7fd798003d50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:53:45.390 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:45.386+0000 7fd7b1433640 1 --2- 172.21.15.43:0/1135229476 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fd798004370 0x7fd798004840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:53:45.391 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:45.386+0000 7fd7b1433640 1 --2- 172.21.15.43:0/1135229476 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fd798005040 0x7fd79800b980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:53:45.391 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:45.386+0000 7fd7b1433640 1 -- 172.21.15.43:0/1135229476 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd7ac0d0370 con 0x7fd798004370 2023-08-09T13:53:45.391 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:45.386+0000 7fd7b1433640 1 -- 172.21.15.43:0/1135229476 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd79800bfa0 con 0x7fd798005040 2023-08-09T13:53:45.391 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:45.386+0000 7fd7b1433640 1 -- 172.21.15.43:0/1135229476 --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd79800c160 con 0x7fd798003970 2023-08-09T13:53:45.391 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:45.386+0000 7fd7b1c34640 1 --2- 172.21.15.43:0/1135229476 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fd798004370 0x7fd798004840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:53:45.391 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:45.386+0000 7fd7b1c34640 1 -- 172.21.15.43:0/1135229476 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fd798003970 msgr2=0x7fd798003d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-09T13:53:45.391 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:45.386+0000 7fd7b1c34640 1 --2- 172.21.15.43:0/1135229476 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fd798003970 0x7fd798003d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:53:45.392 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:45.386+0000 7fd7b1c34640 1 -- 172.21.15.43:0/1135229476 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fd798005040 msgr2=0x7fd79800b980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-09T13:53:45.392 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:45.386+0000 7fd7b1c34640 1 --2- 172.21.15.43:0/1135229476 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fd798005040 0x7fd79800b980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:53:45.392 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:45.386+0000 7fd7b1c34640 1 -- 172.21.15.43:0/1135229476 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd798003520 con 0x7fd798004370 2023-08-09T13:53:45.392 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:45.386+0000 7fd7b2c36640 1 -- 172.21.15.43:0/1135229476 reap_dead start 2023-08-09T13:53:45.393 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:45.387+0000 7fd7b1c34640 1 --2- 172.21.15.43:0/1135229476 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fd798004370 0x7fd798004840 secure :-1 s=READY pgs=1501 cs=0 l=1 rev1=1 crypto rx=0x7fd7ac06c000 tx=0x7fd7980057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=200571a88abcb85a server_cookie=0 in_seq=0 out_seq=0 2023-08-09T13:53:45.393 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:45.388+0000 7fd7b1433640 1 -- 172.21.15.43:0/1135229476 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd79c005ad0 con 0x7fd798004370 2023-08-09T13:53:45.393 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:45.388+0000 7fd7b1433640 1 -- 172.21.15.43:0/1135229476 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd79c005c50 con 0x7fd798004370 2023-08-09T13:53:45.393 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:45.388+0000 7fd7b1433640 1 -- 172.21.15.43:0/1135229476 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd79c02c870 con 0x7fd798004370 2023-08-09T13:53:45.394 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:45.388+0000 7fd7b3437640 1 -- 172.21.15.43:0/1135229476 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fd798004370 msgr2=0x7fd798004840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:53:45.394 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:45.388+0000 7fd7b3437640 1 --2- 172.21.15.43:0/1135229476 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fd798004370 0x7fd798004840 secure :-1 s=READY pgs=1501 cs=0 l=1 rev1=1 crypto rx=0x7fd7ac06c000 tx=0x7fd7980057f0 comp rx=0 tx=0).stop 2023-08-09T13:53:45.394 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:45.388+0000 7fd7b3437640 1 -- 172.21.15.43:0/1135229476 shutdown_connections 2023-08-09T13:53:45.394 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:45.388+0000 7fd7b3437640 1 --2- 172.21.15.43:0/1135229476 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fd798004370 0x7fd798004840 unknown :-1 s=CLOSED pgs=1501 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:53:45.394 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:45.388+0000 7fd7b3437640 1 -- 172.21.15.43:0/1135229476 >> 172.21.15.43:0/1135229476 conn(0x7fd7ac06a2b0 msgr2=0x7fd7ac06a690 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-09T13:53:45.394 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:45.388+0000 7fd7b3437640 1 -- 172.21.15.43:0/1135229476 shutdown_connections 2023-08-09T13:53:45.395 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:45.388+0000 7fd7b3437640 1 -- 172.21.15.43:0/1135229476 wait complete. 2023-08-09T13:53:45.395 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:45.390+0000 7fd7b3437640 1 Processor -- start 2023-08-09T13:53:45.395 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:45.390+0000 7fd7b3437640 1 -- start start 2023-08-09T13:53:45.396 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:45.391+0000 7fd7b3437640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fd798004370 0x7fd7ac15e690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:53:45.396 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:45.391+0000 7fd7b3437640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fd7ac1564f0 0x7fd7ac158ca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:53:45.396 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:45.391+0000 7fd7b3437640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fd7ac156970 0x7fd7ac1591e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:53:45.397 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:45.391+0000 7fd7b3437640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd7ac15d310 con 0x7fd7ac156970 2023-08-09T13:53:45.397 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:45.391+0000 7fd7b3437640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd7ac15d5f0 con 0x7fd798004370 2023-08-09T13:53:45.397 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:45.391+0000 7fd7b3437640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd7ac15d8d0 con 0x7fd7ac1564f0 2023-08-09T13:53:45.397 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:45.392+0000 7fd7b2c36640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fd7ac156970 0x7fd7ac1591e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:53:45.397 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:45.392+0000 7fd7b2c36640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fd7ac156970 0x7fd7ac1591e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:56236/0 (socket says 172.21.15.43:56236) 2023-08-09T13:53:45.397 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:45.392+0000 7fd7b2c36640 1 -- 172.21.15.43:0/2481223124 learned_addr learned my addr 172.21.15.43:0/2481223124 (peer_addr_for_me v2:172.21.15.43:0/0) 2023-08-09T13:53:45.397 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:45.392+0000 7fd7b2435640 1 --2- 172.21.15.43:0/2481223124 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fd798004370 0x7fd7ac15e690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:53:45.398 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:45.392+0000 7fd7b1c34640 1 --2- 172.21.15.43:0/2481223124 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fd7ac1564f0 0x7fd7ac158ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:53:45.398 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:45.392+0000 7fd7b2c36640 1 -- 172.21.15.43:0/2481223124 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fd7ac1564f0 msgr2=0x7fd7ac158ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:53:45.398 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:45.392+0000 7fd7b2c36640 1 --2- 172.21.15.43:0/2481223124 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fd7ac1564f0 0x7fd7ac158ca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:53:45.398 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:45.392+0000 7fd7b2c36640 1 -- 172.21.15.43:0/2481223124 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fd798004370 msgr2=0x7fd7ac15e690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:53:45.398 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:45.392+0000 7fd7b2c36640 1 --2- 172.21.15.43:0/2481223124 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fd798004370 0x7fd7ac15e690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:53:45.398 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:45.392+0000 7fd7b2c36640 1 -- 172.21.15.43:0/2481223124 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd7ac15dbb0 con 0x7fd7ac156970 2023-08-09T13:53:45.398 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:45.393+0000 7fd7b2c36640 1 --2- 172.21.15.43:0/2481223124 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fd7ac156970 0x7fd7ac1591e0 secure :-1 s=READY pgs=1502 cs=0 l=1 rev1=1 crypto rx=0x7fd7ac06af30 tx=0x7fd7a400cb90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-09T13:53:45.399 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:45.393+0000 7fd7a2ffd640 1 -- 172.21.15.43:0/2481223124 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd7a4005d40 con 0x7fd7ac156970 2023-08-09T13:53:45.399 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:45.393+0000 7fd7a2ffd640 1 -- 172.21.15.43:0/2481223124 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd7a400ced0 con 0x7fd7ac156970 2023-08-09T13:53:45.399 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:45.393+0000 7fd7a2ffd640 1 -- 172.21.15.43:0/2481223124 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd7a4005ea0 con 0x7fd7ac156970 2023-08-09T13:53:45.399 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:45.393+0000 7fd7b3437640 1 -- 172.21.15.43:0/2481223124 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd7ac15de90 con 0x7fd7ac156970 2023-08-09T13:53:45.399 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:45.393+0000 7fd7b3437640 1 -- 172.21.15.43:0/2481223124 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd7ac15ebd0 con 0x7fd7ac156970 2023-08-09T13:53:45.400 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:45.394+0000 7fd7b3437640 1 -- 172.21.15.43:0/2481223124 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd7ac15e3b0 con 0x7fd7ac156970 2023-08-09T13:53:45.400 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:45.395+0000 7fd7a2ffd640 1 -- 172.21.15.43:0/2481223124 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 56) v1 ==== 88443+0+0 (secure 0 0 0) 0x7fd7a400a850 con 0x7fd7ac156970 2023-08-09T13:53:45.400 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:45.396+0000 7fd7a2ffd640 1 --2- 172.21.15.43:0/2481223124 >> [v2:172.21.15.203:6832/2527059680,v1:172.21.15.203:6833/2527059680] conn(0x7fd79006a170 0x7fd79006c630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:53:45.400 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:45.396+0000 7fd7a2ffd640 1 -- 172.21.15.43:0/2481223124 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(125..125 src has 1..125) v4 ==== 5574+0+0 (secure 0 0 0) 0x7fd7a4087a40 con 0x7fd7ac156970 2023-08-09T13:53:45.400 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:45.397+0000 7fd7b2435640 1 --2- 172.21.15.43:0/2481223124 >> [v2:172.21.15.203:6832/2527059680,v1:172.21.15.203:6833/2527059680] conn(0x7fd79006a170 0x7fd79006c630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:53:45.401 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:45.398+0000 7fd7b2435640 1 --2- 172.21.15.43:0/2481223124 >> [v2:172.21.15.203:6832/2527059680,v1:172.21.15.203:6833/2527059680] conn(0x7fd79006a170 0x7fd79006c630 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7fd7ac06b0c0 tx=0x7fd7a8008040 comp rx=0 tx=0).ready entity=mgr.6377 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-09T13:53:45.403 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:45.401+0000 7fd7a2ffd640 1 -- 172.21.15.43:0/2481223124 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178066 (secure 0 0 0) 0x7fd7ac15e3b0 con 0x7fd7ac156970 2023-08-09T13:53:45.596 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:45.594+0000 7fd7b3437640 1 -- 172.21.15.43:0/2481223124 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) v1 -- 0x7fd7ac15e3b0 con 0x7fd7ac156970 2023-08-09T13:53:45.597 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:45.595+0000 7fd7a2ffd640 1 -- 172.21.15.43:0/2481223124 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v125) v1 ==== 84+0+126 (secure 0 0 0) 0x7fd7ac15e3b0 con 0x7fd7ac156970 2023-08-09T13:53:45.598 INFO:teuthology.orchestra.run.smithi043.stdout: 2023-08-09T13:53:45.598 INFO:teuthology.orchestra.run.smithi043.stdout:[ 2023-08-09T13:53:45.598 INFO:teuthology.orchestra.run.smithi043.stdout: { 2023-08-09T13:53:45.598 INFO:teuthology.orchestra.run.smithi043.stdout: "poolnum": 1, 2023-08-09T13:53:45.598 INFO:teuthology.orchestra.run.smithi043.stdout: "poolname": "rbd" 2023-08-09T13:53:45.598 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2023-08-09T13:53:45.599 INFO:teuthology.orchestra.run.smithi043.stdout: { 2023-08-09T13:53:45.599 INFO:teuthology.orchestra.run.smithi043.stdout: "poolnum": 2, 2023-08-09T13:53:45.599 INFO:teuthology.orchestra.run.smithi043.stdout: "poolname": ".mgr" 2023-08-09T13:53:45.599 INFO:teuthology.orchestra.run.smithi043.stdout: } 2023-08-09T13:53:45.599 INFO:teuthology.orchestra.run.smithi043.stdout:] 2023-08-09T13:53:45.600 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:45.597+0000 7fd7b3437640 1 -- 172.21.15.43:0/2481223124 >> [v2:172.21.15.203:6832/2527059680,v1:172.21.15.203:6833/2527059680] conn(0x7fd79006a170 msgr2=0x7fd79006c630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:53:45.600 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:45.597+0000 7fd7b3437640 1 --2- 172.21.15.43:0/2481223124 >> [v2:172.21.15.203:6832/2527059680,v1:172.21.15.203:6833/2527059680] conn(0x7fd79006a170 0x7fd79006c630 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7fd7ac06b0c0 tx=0x7fd7a8008040 comp rx=0 tx=0).stop 2023-08-09T13:53:45.600 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:45.597+0000 7fd7b3437640 1 -- 172.21.15.43:0/2481223124 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fd7ac156970 msgr2=0x7fd7ac1591e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:53:45.600 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:45.597+0000 7fd7b3437640 1 --2- 172.21.15.43:0/2481223124 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fd7ac156970 0x7fd7ac1591e0 secure :-1 s=READY pgs=1502 cs=0 l=1 rev1=1 crypto rx=0x7fd7ac06af30 tx=0x7fd7a400cb90 comp rx=0 tx=0).stop 2023-08-09T13:53:45.601 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:45.597+0000 7fd7b3437640 1 -- 172.21.15.43:0/2481223124 shutdown_connections 2023-08-09T13:53:45.601 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:45.597+0000 7fd7b3437640 1 --2- 172.21.15.43:0/2481223124 >> [v2:172.21.15.203:6832/2527059680,v1:172.21.15.203:6833/2527059680] conn(0x7fd79006a170 0x7fd79006c630 unknown :-1 s=CLOSED pgs=54 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:53:45.601 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:45.597+0000 7fd7b3437640 1 --2- 172.21.15.43:0/2481223124 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fd7ac156970 0x7fd7ac1591e0 unknown :-1 s=CLOSED pgs=1502 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:53:45.601 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:45.597+0000 7fd7b3437640 1 --2- 172.21.15.43:0/2481223124 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fd7ac1564f0 0x7fd7ac158ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:53:45.601 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:45.597+0000 7fd7b3437640 1 --2- 172.21.15.43:0/2481223124 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fd798004370 0x7fd7ac15e690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:53:45.602 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:45.597+0000 7fd7b3437640 1 -- 172.21.15.43:0/2481223124 >> 172.21.15.43:0/2481223124 conn(0x7fd7ac06a2b0 msgr2=0x7fd7ac0d1160 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-09T13:53:45.602 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:45.597+0000 7fd7b3437640 1 -- 172.21.15.43:0/2481223124 shutdown_connections 2023-08-09T13:53:45.602 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:45.598+0000 7fd7b3437640 1 -- 172.21.15.43:0/2481223124 wait complete. 2023-08-09T13:53:45.620 DEBUG:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2023-08-09T13:53:45.621 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_metadata --pg_num_min 64 2023-08-09T13:53:45.856 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:45.853+0000 7ff149737640 1 Processor -- start 2023-08-09T13:53:45.856 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:45.854+0000 7ff149737640 1 -- start start 2023-08-09T13:53:45.857 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:45.855+0000 7ff149737640 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7ff14415c720 0x7ff1440cd850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:53:45.858 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:45.855+0000 7ff149737640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7ff1440d0900 0x7ff1440cec60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:53:45.858 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:45.855+0000 7ff149737640 1 -- --> v1:172.21.15.203:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff1440cf5c0 con 0x7ff1440cf1a0 2023-08-09T13:53:45.858 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:45.855+0000 7ff149737640 1 -- --> v2:172.21.15.43:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff14415cb00 con 0x7ff1440d0900 2023-08-09T13:53:45.858 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:45.855+0000 7ff149737640 1 -- --> v2:172.21.15.203:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff1440d1960 con 0x7ff14415c720 2023-08-09T13:53:45.859 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:45.856+0000 7ff1437fe640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7ff1440d0900 0x7ff1440cec60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:53:45.859 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:45.856+0000 7ff1437fe640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7ff1440d0900 0x7ff1440cec60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:56252/0 (socket says 172.21.15.43:56252) 2023-08-09T13:53:45.859 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:45.856+0000 7ff1437fe640 1 -- 172.21.15.43:0/3494933471 learned_addr learned my addr 172.21.15.43:0/3494933471 (peer_addr_for_me v2:172.21.15.43:0/0) 2023-08-09T13:53:45.859 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:45.856+0000 7ff143fff640 1 --2- 172.21.15.43:0/3494933471 >> v2:172.21.15.203:3300/0 conn(0x7ff14415c720 0x7ff1440cd850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:53:45.859 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:45.856+0000 7ff1437fe640 1 -- 172.21.15.43:0/3494933471 >> v2:172.21.15.203:3300/0 conn(0x7ff14415c720 msgr2=0x7ff1440cd850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-09T13:53:45.859 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:45.856+0000 7ff1437fe640 1 --2- 172.21.15.43:0/3494933471 >> v2:172.21.15.203:3300/0 conn(0x7ff14415c720 0x7ff1440cd850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:53:45.860 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:45.856+0000 7ff1437fe640 1 -- 172.21.15.43:0/3494933471 >> v1:172.21.15.203:6789/0 conn(0x7ff1440cf1a0 legacy=0x7ff1440b3cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-09T13:53:45.860 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:45.856+0000 7ff1437fe640 1 -- 172.21.15.43:0/3494933471 --> v2:172.21.15.43:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff1440b5660 con 0x7ff1440d0900 2023-08-09T13:53:45.860 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:45.857+0000 7ff1437fe640 1 --2- 172.21.15.43:0/3494933471 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff1440d0900 0x7ff1440cec60 secure :-1 s=READY pgs=1503 cs=0 l=1 rev1=1 crypto rx=0x7ff1440b4de0 tx=0x7ff13802b910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1ddea18c8f063a0d server_cookie=0 in_seq=0 out_seq=0 2023-08-09T13:53:45.860 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:45.858+0000 7ff142ffd640 1 -- 172.21.15.43:0/3494933471 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff138031870 con 0x7ff1440d0900 2023-08-09T13:53:45.861 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:45.858+0000 7ff142ffd640 1 -- 172.21.15.43:0/3494933471 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff1380319d0 con 0x7ff1440d0900 2023-08-09T13:53:45.861 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:45.858+0000 7ff142ffd640 1 -- 172.21.15.43:0/3494933471 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff138031d60 con 0x7ff1440d0900 2023-08-09T13:53:45.861 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:45.858+0000 7ff149737640 1 -- 172.21.15.43:0/3494933471 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff1440d0900 msgr2=0x7ff1440cec60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:53:45.862 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:45.858+0000 7ff149737640 1 --2- 172.21.15.43:0/3494933471 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff1440d0900 0x7ff1440cec60 secure :-1 s=READY pgs=1503 cs=0 l=1 rev1=1 crypto rx=0x7ff1440b4de0 tx=0x7ff13802b910 comp rx=0 tx=0).stop 2023-08-09T13:53:45.862 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:45.859+0000 7ff149737640 1 -- 172.21.15.43:0/3494933471 shutdown_connections 2023-08-09T13:53:45.862 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:45.859+0000 7ff149737640 1 --2- 172.21.15.43:0/3494933471 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff1440d0900 0x7ff1440cec60 unknown :-1 s=CLOSED pgs=1503 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:53:45.862 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:45.859+0000 7ff149737640 1 --2- 172.21.15.43:0/3494933471 >> v2:172.21.15.203:3300/0 conn(0x7ff14415c720 0x7ff1440cd850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:53:45.862 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:45.859+0000 7ff149737640 1 -- 172.21.15.43:0/3494933471 >> 172.21.15.43:0/3494933471 conn(0x7ff14406a2a0 msgr2=0x7ff14406a680 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-09T13:53:45.862 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:45.859+0000 7ff149737640 1 -- 172.21.15.43:0/3494933471 shutdown_connections 2023-08-09T13:53:45.862 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:45.859+0000 7ff149737640 1 -- 172.21.15.43:0/3494933471 wait complete. 2023-08-09T13:53:45.863 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:45.860+0000 7ff149737640 1 Processor -- start 2023-08-09T13:53:45.863 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:45.861+0000 7ff149737640 1 -- start start 2023-08-09T13:53:45.864 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:45.862+0000 7ff149737640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff1440cf1a0 0x7ff1440c6700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:53:45.864 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:45.862+0000 7ff149737640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7ff1440d0900 0x7ff1441536e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:53:45.865 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:45.862+0000 7ff149737640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7ff14415c720 0x7ff1440b1fd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:53:45.865 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:45.862+0000 7ff149737640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff1440c7e30 con 0x7ff1440cf1a0 2023-08-09T13:53:45.865 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:45.862+0000 7ff149737640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff1440c6d90 con 0x7ff14415c720 2023-08-09T13:53:45.865 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:45.862+0000 7ff149737640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff1440c7070 con 0x7ff1440d0900 2023-08-09T13:53:45.865 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:45.862+0000 7ff143fff640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff1440cf1a0 0x7ff1440c6700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:53:45.865 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:45.862+0000 7ff143fff640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff1440cf1a0 0x7ff1440c6700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:56258/0 (socket says 172.21.15.43:56258) 2023-08-09T13:53:45.865 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:45.862+0000 7ff143fff640 1 -- 172.21.15.43:0/200794600 learned_addr learned my addr 172.21.15.43:0/200794600 (peer_addr_for_me v2:172.21.15.43:0/0) 2023-08-09T13:53:45.866 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:45.862+0000 7ff148f36640 1 --2- 172.21.15.43:0/200794600 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7ff14415c720 0x7ff1440b1fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:53:45.866 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:45.862+0000 7ff1437fe640 1 --2- 172.21.15.43:0/200794600 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7ff1440d0900 0x7ff1441536e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:53:45.867 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:45.862+0000 7ff143fff640 1 -- 172.21.15.43:0/200794600 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7ff1440d0900 msgr2=0x7ff1441536e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:53:45.867 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:45.862+0000 7ff143fff640 1 --2- 172.21.15.43:0/200794600 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7ff1440d0900 0x7ff1441536e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:53:45.867 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:45.862+0000 7ff143fff640 1 -- 172.21.15.43:0/200794600 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7ff14415c720 msgr2=0x7ff1440b1fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:53:45.867 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:45.862+0000 7ff143fff640 1 --2- 172.21.15.43:0/200794600 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7ff14415c720 0x7ff1440b1fd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:53:45.867 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:45.862+0000 7ff143fff640 1 -- 172.21.15.43:0/200794600 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff1380094d0 con 0x7ff1440cf1a0 2023-08-09T13:53:45.867 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:45.863+0000 7ff143fff640 1 --2- 172.21.15.43:0/200794600 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff1440cf1a0 0x7ff1440c6700 secure :-1 s=READY pgs=1504 cs=0 l=1 rev1=1 crypto rx=0x7ff13400c7f0 tx=0x7ff13400ccb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-09T13:53:45.867 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:45.863+0000 7ff140ff9640 1 -- 172.21.15.43:0/200794600 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff134004220 con 0x7ff1440cf1a0 2023-08-09T13:53:45.868 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:45.863+0000 7ff140ff9640 1 -- 172.21.15.43:0/200794600 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff134004380 con 0x7ff1440cf1a0 2023-08-09T13:53:45.868 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:45.863+0000 7ff140ff9640 1 -- 172.21.15.43:0/200794600 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff134007840 con 0x7ff1440cf1a0 2023-08-09T13:53:45.868 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:45.863+0000 7ff149737640 1 -- 172.21.15.43:0/200794600 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff1440c7350 con 0x7ff1440cf1a0 2023-08-09T13:53:45.868 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:45.863+0000 7ff149737640 1 -- 172.21.15.43:0/200794600 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff1440c7910 con 0x7ff1440cf1a0 2023-08-09T13:53:45.868 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:45.864+0000 7ff149737640 1 -- 172.21.15.43:0/200794600 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff1440c7b30 con 0x7ff1440cf1a0 2023-08-09T13:53:45.869 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:45.865+0000 7ff140ff9640 1 -- 172.21.15.43:0/200794600 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 56) v1 ==== 88443+0+0 (secure 0 0 0) 0x7ff134007bb0 con 0x7ff1440cf1a0 2023-08-09T13:53:45.869 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:45.866+0000 7ff140ff9640 1 --2- 172.21.15.43:0/200794600 >> [v2:172.21.15.203:6832/2527059680,v1:172.21.15.203:6833/2527059680] conn(0x7ff13006a170 0x7ff13006c630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:53:45.869 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:45.866+0000 7ff140ff9640 1 -- 172.21.15.43:0/200794600 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(125..125 src has 1..125) v4 ==== 5574+0+0 (secure 0 0 0) 0x7ff1340884e0 con 0x7ff1440cf1a0 2023-08-09T13:53:45.869 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:45.867+0000 7ff1437fe640 1 --2- 172.21.15.43:0/200794600 >> [v2:172.21.15.203:6832/2527059680,v1:172.21.15.203:6833/2527059680] conn(0x7ff13006a170 0x7ff13006c630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:53:45.870 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:45.868+0000 7ff1437fe640 1 --2- 172.21.15.43:0/200794600 >> [v2:172.21.15.203:6832/2527059680,v1:172.21.15.203:6833/2527059680] conn(0x7ff13006a170 0x7ff13006c630 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7ff138036000 tx=0x7ff13803c040 comp rx=0 tx=0).ready entity=mgr.6377 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-09T13:53:45.873 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:45.871+0000 7ff140ff9640 1 -- 172.21.15.43:0/200794600 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178066 (secure 0 0 0) 0x7ff134053b90 con 0x7ff1440cf1a0 2023-08-09T13:53:46.070 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:46.068+0000 7ff149737640 1 -- 172.21.15.43:0/200794600 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64} v 0) v1 -- 0x7ff14415ea70 con 0x7ff1440cf1a0 2023-08-09T13:53:46.239 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:46.236+0000 7ff140ff9640 1 -- 172.21.15.43:0/200794600 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]=0 pool 'cephfs_metadata' created v126) v1 ==== 138+0+0 (secure 0 0 0) 0x7ff134007e60 con 0x7ff1440cf1a0 2023-08-09T13:53:46.239 INFO:teuthology.orchestra.run.smithi043.stderr:pool 'cephfs_metadata' created 2023-08-09T13:53:46.243 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:46.241+0000 7ff149737640 1 -- 172.21.15.43:0/200794600 >> [v2:172.21.15.203:6832/2527059680,v1:172.21.15.203:6833/2527059680] conn(0x7ff13006a170 msgr2=0x7ff13006c630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:53:46.243 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:46.241+0000 7ff149737640 1 --2- 172.21.15.43:0/200794600 >> [v2:172.21.15.203:6832/2527059680,v1:172.21.15.203:6833/2527059680] conn(0x7ff13006a170 0x7ff13006c630 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7ff138036000 tx=0x7ff13803c040 comp rx=0 tx=0).stop 2023-08-09T13:53:46.244 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:46.241+0000 7ff149737640 1 -- 172.21.15.43:0/200794600 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff1440cf1a0 msgr2=0x7ff1440c6700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:53:46.244 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:46.241+0000 7ff149737640 1 --2- 172.21.15.43:0/200794600 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff1440cf1a0 0x7ff1440c6700 secure :-1 s=READY pgs=1504 cs=0 l=1 rev1=1 crypto rx=0x7ff13400c7f0 tx=0x7ff13400ccb0 comp rx=0 tx=0).stop 2023-08-09T13:53:46.244 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:46.241+0000 7ff149737640 1 -- 172.21.15.43:0/200794600 shutdown_connections 2023-08-09T13:53:46.244 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:46.241+0000 7ff149737640 1 --2- 172.21.15.43:0/200794600 >> [v2:172.21.15.203:6832/2527059680,v1:172.21.15.203:6833/2527059680] conn(0x7ff13006a170 0x7ff13006c630 unknown :-1 s=CLOSED pgs=55 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:53:46.245 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:46.241+0000 7ff149737640 1 --2- 172.21.15.43:0/200794600 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7ff14415c720 0x7ff1440b1fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:53:46.245 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:46.241+0000 7ff149737640 1 --2- 172.21.15.43:0/200794600 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7ff1440d0900 0x7ff1441536e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:53:46.245 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:46.241+0000 7ff149737640 1 --2- 172.21.15.43:0/200794600 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff1440cf1a0 0x7ff1440c6700 unknown :-1 s=CLOSED pgs=1504 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:53:46.245 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:46.241+0000 7ff149737640 1 -- 172.21.15.43:0/200794600 >> 172.21.15.43:0/200794600 conn(0x7ff14406a2a0 msgr2=0x7ff14406a890 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-09T13:53:46.245 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:46.242+0000 7ff149737640 1 -- 172.21.15.43:0/200794600 shutdown_connections 2023-08-09T13:53:46.246 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:46.243+0000 7ff149737640 1 -- 172.21.15.43:0/200794600 wait complete. 2023-08-09T13:53:46.271 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data 64 --pg_num_min 64 --target_size_ratio 0.9 2023-08-09T13:53:46.492 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:46.490+0000 7f427d491640 1 Processor -- start 2023-08-09T13:53:46.492 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:46.490+0000 7f427d491640 1 -- start start 2023-08-09T13:53:46.494 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:46.492+0000 7f427d491640 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7f4278157ea0 0x7f42780c97d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:53:46.494 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:46.492+0000 7f427d491640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f427815af10 0x7f42780c9d10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:53:46.495 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:46.492+0000 7f427d491640 1 -- --> v1:172.21.15.203:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4278158280 con 0x7f42780c90a0 2023-08-09T13:53:46.495 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:46.492+0000 7f427d491640 1 -- --> v2:172.21.15.43:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f427815b840 con 0x7f427815af10 2023-08-09T13:53:46.495 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:46.492+0000 7f427d491640 1 -- --> v2:172.21.15.203:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f42780b3810 con 0x7f4278157ea0 2023-08-09T13:53:46.495 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:46.492+0000 7f427cc90640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f427815af10 0x7f42780c9d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:53:46.496 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:46.492+0000 7f427cc90640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f427815af10 0x7f42780c9d10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:56274/0 (socket says 172.21.15.43:56274) 2023-08-09T13:53:46.496 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:46.492+0000 7f427cc90640 1 -- 172.21.15.43:0/1335268815 learned_addr learned my addr 172.21.15.43:0/1335268815 (peer_addr_for_me v2:172.21.15.43:0/0) 2023-08-09T13:53:46.496 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:46.493+0000 7f427cc90640 1 -- 172.21.15.43:0/1335268815 >> v2:172.21.15.203:3300/0 conn(0x7f4278157ea0 msgr2=0x7f42780c97d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-09T13:53:46.496 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:46.493+0000 7f427cc90640 1 --2- 172.21.15.43:0/1335268815 >> v2:172.21.15.203:3300/0 conn(0x7f4278157ea0 0x7f42780c97d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:53:46.496 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:46.493+0000 7f427cc90640 1 -- 172.21.15.43:0/1335268815 >> v1:172.21.15.203:6789/0 conn(0x7f42780c90a0 legacy=0x7f427815a800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-09T13:53:46.496 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:46.493+0000 7f427cc90640 1 -- 172.21.15.43:0/1335268815 --> v2:172.21.15.43:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f427815b560 con 0x7f427815af10 2023-08-09T13:53:46.497 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:46.494+0000 7f427cc90640 1 --2- 172.21.15.43:0/1335268815 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f427815af10 0x7f42780c9d10 secure :-1 s=READY pgs=1505 cs=0 l=1 rev1=1 crypto rx=0x7f427806ad20 tx=0x7f427002be50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b293a02729a304a3 server_cookie=0 in_seq=0 out_seq=0 2023-08-09T13:53:46.497 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:46.494+0000 7f4276ffd640 1 -- 172.21.15.43:0/1335268815 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f427002c9f0 con 0x7f427815af10 2023-08-09T13:53:46.497 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:46.494+0000 7f4276ffd640 1 -- 172.21.15.43:0/1335268815 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f427002cb50 con 0x7f427815af10 2023-08-09T13:53:46.497 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:46.494+0000 7f4276ffd640 1 -- 172.21.15.43:0/1335268815 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f42700343f0 con 0x7f427815af10 2023-08-09T13:53:46.498 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:46.495+0000 7f427d491640 1 -- 172.21.15.43:0/1335268815 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f427815af10 msgr2=0x7f42780c9d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:53:46.498 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:46.495+0000 7f427d491640 1 --2- 172.21.15.43:0/1335268815 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f427815af10 0x7f42780c9d10 secure :-1 s=READY pgs=1505 cs=0 l=1 rev1=1 crypto rx=0x7f427806ad20 tx=0x7f427002be50 comp rx=0 tx=0).stop 2023-08-09T13:53:46.498 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:46.495+0000 7f427d491640 1 -- 172.21.15.43:0/1335268815 shutdown_connections 2023-08-09T13:53:46.499 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:46.495+0000 7f427d491640 1 --2- 172.21.15.43:0/1335268815 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f427815af10 0x7f42780c9d10 unknown :-1 s=CLOSED pgs=1505 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:53:46.499 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:46.495+0000 7f427d491640 1 --2- 172.21.15.43:0/1335268815 >> v2:172.21.15.203:3300/0 conn(0x7f4278157ea0 0x7f42780c97d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:53:46.499 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:46.495+0000 7f427d491640 1 -- 172.21.15.43:0/1335268815 >> 172.21.15.43:0/1335268815 conn(0x7f42780cddd0 msgr2=0x7f42780ce1e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-09T13:53:46.499 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:46.495+0000 7f427d491640 1 -- 172.21.15.43:0/1335268815 shutdown_connections 2023-08-09T13:53:46.499 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:46.495+0000 7f427d491640 1 -- 172.21.15.43:0/1335268815 wait complete. 2023-08-09T13:53:46.500 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:46.497+0000 7f427d491640 1 Processor -- start 2023-08-09T13:53:46.500 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:46.497+0000 7f427d491640 1 -- start start 2023-08-09T13:53:46.501 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:46.498+0000 7f427d491640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f42780c90a0 0x7f42781f12c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:53:46.501 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:46.498+0000 7f427d491640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f4278157ea0 0x7f42780b26c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:53:46.501 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:46.498+0000 7f427d491640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f427815af10 0x7f427815c440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:53:46.501 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:46.498+0000 7f427d491640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f427814cc80 con 0x7f427815af10 2023-08-09T13:53:46.502 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:46.498+0000 7f427d491640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4278150240 con 0x7f42780c90a0 2023-08-09T13:53:46.502 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:46.498+0000 7f427d491640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f42780b2230 con 0x7f4278157ea0 2023-08-09T13:53:46.502 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:46.499+0000 7f427cc90640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f427815af10 0x7f427815c440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:53:46.502 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:46.499+0000 7f427cc90640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f427815af10 0x7f427815c440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:56286/0 (socket says 172.21.15.43:56286) 2023-08-09T13:53:46.502 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:46.499+0000 7f427cc90640 1 -- 172.21.15.43:0/1854130248 learned_addr learned my addr 172.21.15.43:0/1854130248 (peer_addr_for_me v2:172.21.15.43:0/0) 2023-08-09T13:53:46.503 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:46.499+0000 7f4277fff640 1 --2- 172.21.15.43:0/1854130248 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f42780c90a0 0x7f42781f12c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:53:46.503 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:46.499+0000 7f42777fe640 1 --2- 172.21.15.43:0/1854130248 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f4278157ea0 0x7f42780b26c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:53:46.503 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:46.499+0000 7f427cc90640 1 -- 172.21.15.43:0/1854130248 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f4278157ea0 msgr2=0x7f42780b26c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:53:46.504 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:46.499+0000 7f427cc90640 1 --2- 172.21.15.43:0/1854130248 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f4278157ea0 0x7f42780b26c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:53:46.504 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:46.499+0000 7f427cc90640 1 -- 172.21.15.43:0/1854130248 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f42780c90a0 msgr2=0x7f42781f12c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:53:46.504 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:46.499+0000 7f427cc90640 1 --2- 172.21.15.43:0/1854130248 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f42780c90a0 0x7f42781f12c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:53:46.504 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:46.499+0000 7f427cc90640 1 -- 172.21.15.43:0/1854130248 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4270009b30 con 0x7f427815af10 2023-08-09T13:53:46.504 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:46.499+0000 7f4277fff640 1 --2- 172.21.15.43:0/1854130248 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f42780c90a0 0x7f42781f12c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-08-09T13:53:46.504 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:46.499+0000 7f42777fe640 1 --2- 172.21.15.43:0/1854130248 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f4278157ea0 0x7f42780b26c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-08-09T13:53:46.505 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:46.500+0000 7f427cc90640 1 --2- 172.21.15.43:0/1854130248 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f427815af10 0x7f427815c440 secure :-1 s=READY pgs=1506 cs=0 l=1 rev1=1 crypto rx=0x7f4270009e50 tx=0x7f4270004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-09T13:53:46.505 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:46.500+0000 7f4274ff9640 1 -- 172.21.15.43:0/1854130248 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f427003e450 con 0x7f427815af10 2023-08-09T13:53:46.505 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:46.500+0000 7f4274ff9640 1 -- 172.21.15.43:0/1854130248 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f427003e5d0 con 0x7f427815af10 2023-08-09T13:53:46.505 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:46.500+0000 7f427d491640 1 -- 172.21.15.43:0/1854130248 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f42781f1800 con 0x7f427815af10 2023-08-09T13:53:46.505 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:46.500+0000 7f4274ff9640 1 -- 172.21.15.43:0/1854130248 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f427003e980 con 0x7f427815af10 2023-08-09T13:53:46.505 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:46.500+0000 7f427d491640 1 -- 172.21.15.43:0/1854130248 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f42780b1be0 con 0x7f427815af10 2023-08-09T13:53:46.506 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:46.501+0000 7f427d491640 1 -- 172.21.15.43:0/1854130248 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f42780b23c0 con 0x7f427815af10 2023-08-09T13:53:46.506 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:46.503+0000 7f4274ff9640 1 -- 172.21.15.43:0/1854130248 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 56) v1 ==== 88443+0+0 (secure 0 0 0) 0x7f427003eae0 con 0x7f427815af10 2023-08-09T13:53:46.506 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:46.504+0000 7f4274ff9640 1 --2- 172.21.15.43:0/1854130248 >> [v2:172.21.15.203:6832/2527059680,v1:172.21.15.203:6833/2527059680] conn(0x7f426406a170 0x7f426406c630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:53:46.507 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:46.504+0000 7f4274ff9640 1 -- 172.21.15.43:0/1854130248 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(126..126 src has 1..126) v4 ==== 5944+0+0 (secure 0 0 0) 0x7f42700ab8c0 con 0x7f427815af10 2023-08-09T13:53:46.507 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:46.505+0000 7f4277fff640 1 --2- 172.21.15.43:0/1854130248 >> [v2:172.21.15.203:6832/2527059680,v1:172.21.15.203:6833/2527059680] conn(0x7f426406a170 0x7f426406c630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:53:46.508 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:46.506+0000 7f4277fff640 1 --2- 172.21.15.43:0/1854130248 >> [v2:172.21.15.203:6832/2527059680,v1:172.21.15.203:6833/2527059680] conn(0x7f426406a170 0x7f426406c630 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f4268006fd0 tx=0x7f4268008040 comp rx=0 tx=0).ready entity=mgr.6377 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-09T13:53:46.510 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:46.508+0000 7f4274ff9640 1 -- 172.21.15.43:0/1854130248 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178066 (secure 0 0 0) 0x7f4270076de0 con 0x7f427815af10 2023-08-09T13:53:46.704 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:46.702+0000 7f427d491640 1 -- 172.21.15.43:0/1854130248 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9} v 0) v1 -- 0x7f4278158280 con 0x7f427815af10 2023-08-09T13:53:47.247 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:47.245+0000 7f4274ff9640 1 -- 172.21.15.43:0/1854130248 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]=0 pool 'cephfs_data' created v127) v1 ==== 170+0+0 (secure 0 0 0) 0x7f4278158280 con 0x7f427815af10 2023-08-09T13:53:47.253 INFO:teuthology.orchestra.run.smithi043.stderr:pool 'cephfs_data' created 2023-08-09T13:53:47.268 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:47.266+0000 7f427d491640 1 -- 172.21.15.43:0/1854130248 >> [v2:172.21.15.203:6832/2527059680,v1:172.21.15.203:6833/2527059680] conn(0x7f426406a170 msgr2=0x7f426406c630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:53:47.269 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:47.266+0000 7f427d491640 1 --2- 172.21.15.43:0/1854130248 >> [v2:172.21.15.203:6832/2527059680,v1:172.21.15.203:6833/2527059680] conn(0x7f426406a170 0x7f426406c630 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f4268006fd0 tx=0x7f4268008040 comp rx=0 tx=0).stop 2023-08-09T13:53:47.269 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:47.267+0000 7f427d491640 1 -- 172.21.15.43:0/1854130248 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f427815af10 msgr2=0x7f427815c440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:53:47.269 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:47.267+0000 7f427d491640 1 --2- 172.21.15.43:0/1854130248 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f427815af10 0x7f427815c440 secure :-1 s=READY pgs=1506 cs=0 l=1 rev1=1 crypto rx=0x7f4270009e50 tx=0x7f4270004a40 comp rx=0 tx=0).stop 2023-08-09T13:53:47.273 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:47.270+0000 7f427d491640 1 -- 172.21.15.43:0/1854130248 shutdown_connections 2023-08-09T13:53:47.273 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:47.270+0000 7f427d491640 1 --2- 172.21.15.43:0/1854130248 >> [v2:172.21.15.203:6832/2527059680,v1:172.21.15.203:6833/2527059680] conn(0x7f426406a170 0x7f426406c630 unknown :-1 s=CLOSED pgs=56 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:53:47.273 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:47.270+0000 7f427d491640 1 --2- 172.21.15.43:0/1854130248 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f427815af10 0x7f427815c440 unknown :-1 s=CLOSED pgs=1506 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:53:47.274 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:47.270+0000 7f427d491640 1 --2- 172.21.15.43:0/1854130248 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f4278157ea0 0x7f42780b26c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:53:47.274 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:47.270+0000 7f427d491640 1 --2- 172.21.15.43:0/1854130248 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f42780c90a0 0x7f42781f12c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:53:47.274 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:47.270+0000 7f427d491640 1 -- 172.21.15.43:0/1854130248 >> 172.21.15.43:0/1854130248 conn(0x7f42780cddd0 msgr2=0x7f42780c95c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-09T13:53:47.275 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:47.272+0000 7f427d491640 1 -- 172.21.15.43:0/1854130248 shutdown_connections 2023-08-09T13:53:47.275 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:47.272+0000 7f427d491640 1 -- 172.21.15.43:0/1854130248 wait complete. 2023-08-09T13:53:47.297 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs new cephfs cephfs_metadata cephfs_data 2023-08-09T13:53:47.495 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:47.493+0000 7f816b621640 1 Processor -- start 2023-08-09T13:53:47.496 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:47.493+0000 7f816b621640 1 -- start start 2023-08-09T13:53:47.497 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:47.495+0000 7f816b621640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f816415c550 0x7f81640cd6d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:53:47.497 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:47.495+0000 7f816b621640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f81640b45f0 0x7f81640b59d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:53:47.498 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:47.495+0000 7f816b621640 1 -- --> v1:172.21.15.43:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f81640b7bd0 con 0x7f81640d04b0 2023-08-09T13:53:47.498 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:47.495+0000 7f816b621640 1 -- --> v2:172.21.15.43:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f81640d1c00 con 0x7f81640b45f0 2023-08-09T13:53:47.498 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:47.495+0000 7f816b621640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f816415c930 con 0x7f816415c550 2023-08-09T13:53:47.498 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:47.495+0000 7f8168bbc640 1 --1- >> v1:172.21.15.43:6789/0 conn(0x7f81640d04b0 0x7f81640b3ee0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.43:6789/0 says I am v1:172.21.15.43:53608/0 (socket says 172.21.15.43:53608) 2023-08-09T13:53:47.498 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:47.495+0000 7f8168bbc640 1 -- 172.21.15.43:0/599622526 learned_addr learned my addr 172.21.15.43:0/599622526 (peer_addr_for_me v1:172.21.15.43:0/0) 2023-08-09T13:53:47.499 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:47.495+0000 7f8169bbe640 1 --2- 172.21.15.43:0/599622526 >> v2:172.21.15.43:3300/0 conn(0x7f81640b45f0 0x7f81640b59d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:53:47.499 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:47.495+0000 7f81693bd640 1 --2- 172.21.15.43:0/599622526 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f816415c550 0x7f81640cd6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:53:47.499 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:47.496+0000 7f815bfff640 1 -- 172.21.15.43:0/599622526 <== mon.0 v1:172.21.15.43:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4064818937 0 0) 0x7f81640b7bd0 con 0x7f81640d04b0 2023-08-09T13:53:47.499 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:47.496+0000 7f815bfff640 1 -- 172.21.15.43:0/599622526 --> v1:172.21.15.43:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f8150003520 con 0x7f81640d04b0 2023-08-09T13:53:47.500 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:47.496+0000 7f8169bbe640 1 -- 172.21.15.43:0/599622526 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f816415c550 msgr2=0x7f81640cd6d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-09T13:53:47.500 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:47.496+0000 7f8169bbe640 1 --2- 172.21.15.43:0/599622526 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f816415c550 0x7f81640cd6d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:53:47.500 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:47.496+0000 7f8169bbe640 1 -- 172.21.15.43:0/599622526 >> v1:172.21.15.43:6789/0 conn(0x7f81640d04b0 legacy=0x7f81640b3ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:53:47.500 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:47.496+0000 7f8169bbe640 1 -- 172.21.15.43:0/599622526 --> v2:172.21.15.43:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f81640b54f0 con 0x7f81640b45f0 2023-08-09T13:53:47.501 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:47.497+0000 7f8169bbe640 1 --2- 172.21.15.43:0/599622526 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f81640b45f0 0x7f81640b59d0 secure :-1 s=READY pgs=1508 cs=0 l=1 rev1=1 crypto rx=0x7f815c0097a0 tx=0x7f815c02b8b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b84b1ff037121740 server_cookie=0 in_seq=0 out_seq=0 2023-08-09T13:53:47.501 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:47.497+0000 7f815bfff640 1 -- 172.21.15.43:0/599622526 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f815c02d040 con 0x7f81640b45f0 2023-08-09T13:53:47.501 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:47.497+0000 7f815bfff640 1 -- 172.21.15.43:0/599622526 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f815c031810 con 0x7f81640b45f0 2023-08-09T13:53:47.501 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:47.497+0000 7f815bfff640 1 -- 172.21.15.43:0/599622526 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f815c031ba0 con 0x7f81640b45f0 2023-08-09T13:53:47.502 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:47.498+0000 7f816b621640 1 -- 172.21.15.43:0/599622526 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f81640b45f0 msgr2=0x7f81640b59d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:53:47.502 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:47.498+0000 7f816b621640 1 --2- 172.21.15.43:0/599622526 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f81640b45f0 0x7f81640b59d0 secure :-1 s=READY pgs=1508 cs=0 l=1 rev1=1 crypto rx=0x7f815c0097a0 tx=0x7f815c02b8b0 comp rx=0 tx=0).stop 2023-08-09T13:53:47.502 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:47.498+0000 7f816b621640 1 -- 172.21.15.43:0/599622526 shutdown_connections 2023-08-09T13:53:47.502 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:47.498+0000 7f816b621640 1 --2- 172.21.15.43:0/599622526 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f81640b45f0 0x7f81640b59d0 unknown :-1 s=CLOSED pgs=1508 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:53:47.502 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:47.498+0000 7f816b621640 1 --2- 172.21.15.43:0/599622526 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f816415c550 0x7f81640cd6d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:53:47.502 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:47.498+0000 7f816b621640 1 -- 172.21.15.43:0/599622526 >> 172.21.15.43:0/599622526 conn(0x7f8164071520 msgr2=0x7f81640ceee0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-09T13:53:47.502 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:47.498+0000 7f816b621640 1 -- 172.21.15.43:0/599622526 shutdown_connections 2023-08-09T13:53:47.502 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:47.498+0000 7f816b621640 1 -- 172.21.15.43:0/599622526 wait complete. 2023-08-09T13:53:47.503 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:47.500+0000 7f816b621640 1 Processor -- start 2023-08-09T13:53:47.503 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:47.500+0000 7f816b621640 1 -- start start 2023-08-09T13:53:47.504 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:47.501+0000 7f816b621640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f81640b45f0 0x7f8164159c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:53:47.505 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:47.501+0000 7f816b621640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f81640d04b0 0x7f81640caf20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:53:47.505 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:47.501+0000 7f816b621640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f816415c550 0x7f81640a9c30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:53:47.505 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:47.501+0000 7f816b621640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f81640cb930 con 0x7f81640b45f0 2023-08-09T13:53:47.505 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:47.501+0000 7f816b621640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f81640cbc10 con 0x7f81640d04b0 2023-08-09T13:53:47.505 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:47.501+0000 7f816b621640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f81640cbef0 con 0x7f816415c550 2023-08-09T13:53:47.506 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:47.501+0000 7f81693bd640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f81640b45f0 0x7f8164159c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:53:47.506 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:47.501+0000 7f81693bd640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f81640b45f0 0x7f8164159c00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:56304/0 (socket says 172.21.15.43:56304) 2023-08-09T13:53:47.506 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:47.501+0000 7f81693bd640 1 -- 172.21.15.43:0/3097571323 learned_addr learned my addr 172.21.15.43:0/3097571323 (peer_addr_for_me v2:172.21.15.43:0/0) 2023-08-09T13:53:47.507 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:47.502+0000 7f8169bbe640 1 --2- 172.21.15.43:0/3097571323 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f816415c550 0x7f81640a9c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:53:47.507 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:47.502+0000 7f81693bd640 1 -- 172.21.15.43:0/3097571323 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f816415c550 msgr2=0x7f81640a9c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:53:47.507 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:47.502+0000 7f81693bd640 1 --2- 172.21.15.43:0/3097571323 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f816415c550 0x7f81640a9c30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:53:47.507 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:47.502+0000 7f81693bd640 1 -- 172.21.15.43:0/3097571323 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f81640d04b0 msgr2=0x7f81640caf20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:53:47.508 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:47.502+0000 7f8168bbc640 1 --2- 172.21.15.43:0/3097571323 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f81640d04b0 0x7f81640caf20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:53:47.508 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:47.502+0000 7f81693bd640 1 --2- 172.21.15.43:0/3097571323 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f81640d04b0 0x7f81640caf20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:53:47.508 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:47.502+0000 7f81693bd640 1 -- 172.21.15.43:0/3097571323 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f815c009400 con 0x7f81640b45f0 2023-08-09T13:53:47.508 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:47.502+0000 7f8168bbc640 1 --2- 172.21.15.43:0/3097571323 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f81640d04b0 0x7f81640caf20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-08-09T13:53:47.509 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:47.502+0000 7f81693bd640 1 --2- 172.21.15.43:0/3097571323 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f81640b45f0 0x7f8164159c00 secure :-1 s=READY pgs=1509 cs=0 l=1 rev1=1 crypto rx=0x7f816000efc0 tx=0x7f816000c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-09T13:53:47.511 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:47.503+0000 7f8159ffb640 1 -- 172.21.15.43:0/3097571323 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8160009b80 con 0x7f81640b45f0 2023-08-09T13:53:47.511 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:47.503+0000 7f8159ffb640 1 -- 172.21.15.43:0/3097571323 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f816000f040 con 0x7f81640b45f0 2023-08-09T13:53:47.511 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:47.503+0000 7f8159ffb640 1 -- 172.21.15.43:0/3097571323 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f81600146a0 con 0x7f81640b45f0 2023-08-09T13:53:47.512 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:47.503+0000 7f816b621640 1 -- 172.21.15.43:0/3097571323 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f81640cc1d0 con 0x7f81640b45f0 2023-08-09T13:53:47.512 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:47.503+0000 7f816b621640 1 -- 172.21.15.43:0/3097571323 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f81641f1790 con 0x7f81640b45f0 2023-08-09T13:53:47.512 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:47.504+0000 7f816b621640 1 -- 172.21.15.43:0/3097571323 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f81640739f0 con 0x7f81640b45f0 2023-08-09T13:53:47.512 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:47.505+0000 7f8159ffb640 1 -- 172.21.15.43:0/3097571323 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 56) v1 ==== 88443+0+0 (secure 0 0 0) 0x7f8160014800 con 0x7f81640b45f0 2023-08-09T13:53:47.512 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:47.506+0000 7f8159ffb640 1 --2- 172.21.15.43:0/3097571323 >> [v2:172.21.15.203:6832/2527059680,v1:172.21.15.203:6833/2527059680] conn(0x7f814806a170 0x7f814806c630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:53:47.512 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:47.507+0000 7f8159ffb640 1 -- 172.21.15.43:0/3097571323 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(127..127 src has 1..127) v4 ==== 6326+0+0 (secure 0 0 0) 0x7f8160089bb0 con 0x7f81640b45f0 2023-08-09T13:53:47.513 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:47.507+0000 7f8168bbc640 1 --2- 172.21.15.43:0/3097571323 >> [v2:172.21.15.203:6832/2527059680,v1:172.21.15.203:6833/2527059680] conn(0x7f814806a170 0x7f814806c630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:53:47.513 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:47.508+0000 7f8168bbc640 1 --2- 172.21.15.43:0/3097571323 >> [v2:172.21.15.203:6832/2527059680,v1:172.21.15.203:6833/2527059680] conn(0x7f814806a170 0x7f814806c630 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f81640c5d60 tx=0x7f8154009040 comp rx=0 tx=0).ready entity=mgr.6377 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-09T13:53:47.513 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:47.510+0000 7f8159ffb640 1 -- 172.21.15.43:0/3097571323 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178066 (secure 0 0 0) 0x7f8160058a70 con 0x7f81640b45f0 2023-08-09T13:53:47.747 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:47.744+0000 7f816b621640 1 -- 172.21.15.43:0/3097571323 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"} v 0) v1 -- 0x7f81640d1270 con 0x7f81640b45f0 2023-08-09T13:53:48.285 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:48.283+0000 7f8159ffb640 1 -- 172.21.15.43:0/3097571323 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]=0 new fs with metadata pool 16 and data pool 17 v57) v1 ==== 174+0+0 (secure 0 0 0) 0x7f8160058600 con 0x7f81640b45f0 2023-08-09T13:53:48.288 INFO:teuthology.orchestra.run.smithi043.stderr:new fs with metadata pool 16 and data pool 17 2023-08-09T13:53:48.299 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:48.296+0000 7f816b621640 1 -- 172.21.15.43:0/3097571323 >> [v2:172.21.15.203:6832/2527059680,v1:172.21.15.203:6833/2527059680] conn(0x7f814806a170 msgr2=0x7f814806c630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:53:48.300 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:48.296+0000 7f816b621640 1 --2- 172.21.15.43:0/3097571323 >> [v2:172.21.15.203:6832/2527059680,v1:172.21.15.203:6833/2527059680] conn(0x7f814806a170 0x7f814806c630 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f81640c5d60 tx=0x7f8154009040 comp rx=0 tx=0).stop 2023-08-09T13:53:48.300 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:48.298+0000 7f816b621640 1 -- 172.21.15.43:0/3097571323 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f81640b45f0 msgr2=0x7f8164159c00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:53:48.301 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:48.298+0000 7f816b621640 1 --2- 172.21.15.43:0/3097571323 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f81640b45f0 0x7f8164159c00 secure :-1 s=READY pgs=1509 cs=0 l=1 rev1=1 crypto rx=0x7f816000efc0 tx=0x7f816000c490 comp rx=0 tx=0).stop 2023-08-09T13:53:48.301 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:48.298+0000 7f816b621640 1 -- 172.21.15.43:0/3097571323 shutdown_connections 2023-08-09T13:53:48.301 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:48.298+0000 7f816b621640 1 --2- 172.21.15.43:0/3097571323 >> [v2:172.21.15.203:6832/2527059680,v1:172.21.15.203:6833/2527059680] conn(0x7f814806a170 0x7f814806c630 unknown :-1 s=CLOSED pgs=57 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:53:48.301 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:48.298+0000 7f816b621640 1 --2- 172.21.15.43:0/3097571323 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f816415c550 0x7f81640a9c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:53:48.301 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:48.298+0000 7f816b621640 1 --2- 172.21.15.43:0/3097571323 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f81640d04b0 0x7f81640caf20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:53:48.301 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:48.298+0000 7f816b621640 1 --2- 172.21.15.43:0/3097571323 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f81640b45f0 0x7f8164159c00 unknown :-1 s=CLOSED pgs=1509 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:53:48.302 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:48.298+0000 7f816b621640 1 -- 172.21.15.43:0/3097571323 >> 172.21.15.43:0/3097571323 conn(0x7f8164071520 msgr2=0x7f81640c7f80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-09T13:53:48.302 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:48.299+0000 7f816b621640 1 -- 172.21.15.43:0/3097571323 shutdown_connections 2023-08-09T13:53:48.308 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:48.306+0000 7f816b621640 1 -- 172.21.15.43:0/3097571323 wait complete. 2023-08-09T13:53:48.340 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-08-09T13:53:48.530 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:48.528+0000 7f2cd0cf3640 1 Processor -- start 2023-08-09T13:53:48.531 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:48.528+0000 7f2cd0cf3640 1 -- start start 2023-08-09T13:53:48.532 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:48.529+0000 7f2cd0cf3640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f2ccc0c86f0 0x7f2ccc0c8b50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:53:48.532 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:48.529+0000 7f2cd0cf3640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f2ccc06c8d0 0x7f2ccc06cd60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:53:48.532 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:48.529+0000 7f2cd0cf3640 1 -- --> v1:172.21.15.203:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2ccc0c9090 con 0x7f2ccc143c20 2023-08-09T13:53:48.533 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:48.530+0000 7f2cd0cf3640 1 -- --> v2:172.21.15.43:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2ccc144000 con 0x7f2ccc06c8d0 2023-08-09T13:53:48.533 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:48.530+0000 7f2cd0cf3640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2ccc06c440 con 0x7f2ccc0c86f0 2023-08-09T13:53:48.534 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:48.530+0000 7f2ccad9d640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f2ccc06c8d0 0x7f2ccc06cd60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:53:48.534 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:48.530+0000 7f2ccad9d640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f2ccc06c8d0 0x7f2ccc06cd60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:56306/0 (socket says 172.21.15.43:56306) 2023-08-09T13:53:48.534 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:48.530+0000 7f2ccad9d640 1 -- 172.21.15.43:0/2603184415 learned_addr learned my addr 172.21.15.43:0/2603184415 (peer_addr_for_me v2:172.21.15.43:0/0) 2023-08-09T13:53:48.534 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:48.530+0000 7f2cc3fff640 1 --2- 172.21.15.43:0/2603184415 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f2ccc0c86f0 0x7f2ccc0c8b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:53:48.534 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:48.531+0000 7f2ccad9d640 1 -- 172.21.15.43:0/2603184415 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f2ccc0c86f0 msgr2=0x7f2ccc0c8b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-09T13:53:48.535 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:48.531+0000 7f2ccad9d640 1 --2- 172.21.15.43:0/2603184415 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f2ccc0c86f0 0x7f2ccc0c8b50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:53:48.535 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:48.531+0000 7f2ccad9d640 1 -- 172.21.15.43:0/2603184415 >> v1:172.21.15.203:6789/0 conn(0x7f2ccc143c20 legacy=0x7f2ccc06f540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:53:48.535 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:48.531+0000 7f2ccad9d640 1 -- 172.21.15.43:0/2603184415 --> v2:172.21.15.43:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2ccc06fc50 con 0x7f2ccc06c8d0 2023-08-09T13:53:48.536 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:48.532+0000 7f2ccad9d640 1 --2- 172.21.15.43:0/2603184415 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f2ccc06c8d0 0x7f2ccc06cd60 secure :-1 s=READY pgs=1510 cs=0 l=1 rev1=1 crypto rx=0x7f2cc4009e50 tx=0x7f2cc402bed0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cfad897cd5881aa6 server_cookie=0 in_seq=0 out_seq=0 2023-08-09T13:53:48.536 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:48.532+0000 7f2cc9d9b640 1 -- 172.21.15.43:0/2603184415 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2cc402ca70 con 0x7f2ccc06c8d0 2023-08-09T13:53:48.536 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:48.532+0000 7f2cc9d9b640 1 -- 172.21.15.43:0/2603184415 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2cc402cbd0 con 0x7f2ccc06c8d0 2023-08-09T13:53:48.537 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:48.532+0000 7f2cc9d9b640 1 -- 172.21.15.43:0/2603184415 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2cc40343f0 con 0x7f2ccc06c8d0 2023-08-09T13:53:48.537 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:48.533+0000 7f2cd0cf3640 1 -- 172.21.15.43:0/2603184415 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f2ccc06c8d0 msgr2=0x7f2ccc06cd60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:53:48.537 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:48.533+0000 7f2cd0cf3640 1 --2- 172.21.15.43:0/2603184415 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f2ccc06c8d0 0x7f2ccc06cd60 secure :-1 s=READY pgs=1510 cs=0 l=1 rev1=1 crypto rx=0x7f2cc4009e50 tx=0x7f2cc402bed0 comp rx=0 tx=0).stop 2023-08-09T13:53:48.537 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:48.533+0000 7f2cd0cf3640 1 -- 172.21.15.43:0/2603184415 shutdown_connections 2023-08-09T13:53:48.537 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:48.533+0000 7f2cd0cf3640 1 --2- 172.21.15.43:0/2603184415 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f2ccc06c8d0 0x7f2ccc06cd60 unknown :-1 s=CLOSED pgs=1510 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:53:48.537 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:48.533+0000 7f2cd0cf3640 1 --2- 172.21.15.43:0/2603184415 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f2ccc0c86f0 0x7f2ccc0c8b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:53:48.537 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:48.533+0000 7f2cd0cf3640 1 -- 172.21.15.43:0/2603184415 >> 172.21.15.43:0/2603184415 conn(0x7f2ccc0ce0a0 msgr2=0x7f2ccc0ce480 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-09T13:53:48.538 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:48.533+0000 7f2cd0cf3640 1 -- 172.21.15.43:0/2603184415 shutdown_connections 2023-08-09T13:53:48.538 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:48.533+0000 7f2cd0cf3640 1 -- 172.21.15.43:0/2603184415 wait complete. 2023-08-09T13:53:48.539 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:48.535+0000 7f2cd0cf3640 1 Processor -- start 2023-08-09T13:53:48.539 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:48.535+0000 7f2cd0cf3640 1 -- start start 2023-08-09T13:53:48.540 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:48.536+0000 7f2cd0cf3640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f2ccc06c8d0 0x7f2ccc0c5170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:53:48.540 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:48.536+0000 7f2cd0cf3640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f2ccc0c86f0 0x7f2ccc1f2020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:53:48.540 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:48.536+0000 7f2cd0cf3640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f2ccc143c20 0x7f2ccc132850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:53:48.540 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:48.536+0000 7f2cd0cf3640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2ccc15c750 con 0x7f2ccc06c8d0 2023-08-09T13:53:48.540 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:48.536+0000 7f2cd0cf3640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2ccc1f2560 con 0x7f2ccc143c20 2023-08-09T13:53:48.540 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:48.536+0000 7f2cd0cf3640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2ccc0c83d0 con 0x7f2ccc0c86f0 2023-08-09T13:53:48.541 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:48.536+0000 7f2cca59c640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f2ccc06c8d0 0x7f2ccc0c5170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:53:48.541 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:48.536+0000 7f2cca59c640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f2ccc06c8d0 0x7f2ccc0c5170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:56312/0 (socket says 172.21.15.43:56312) 2023-08-09T13:53:48.541 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:48.536+0000 7f2cca59c640 1 -- 172.21.15.43:0/778136674 learned_addr learned my addr 172.21.15.43:0/778136674 (peer_addr_for_me v2:172.21.15.43:0/0) 2023-08-09T13:53:48.541 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:48.536+0000 7f2cca59c640 1 -- 172.21.15.43:0/778136674 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f2ccc0c86f0 msgr2=0x7f2ccc1f2020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:53:48.541 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:48.536+0000 7f2cca59c640 1 --2- 172.21.15.43:0/778136674 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f2ccc0c86f0 0x7f2ccc1f2020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:53:48.541 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:48.536+0000 7f2cca59c640 1 -- 172.21.15.43:0/778136674 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f2ccc143c20 msgr2=0x7f2ccc132850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:53:48.542 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:48.536+0000 7f2cca59c640 1 --2- 172.21.15.43:0/778136674 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f2ccc143c20 0x7f2ccc132850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:53:48.542 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:48.536+0000 7f2cca59c640 1 -- 172.21.15.43:0/778136674 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2cc4009b30 con 0x7f2ccc06c8d0 2023-08-09T13:53:48.542 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:48.537+0000 7f2cca59c640 1 --2- 172.21.15.43:0/778136674 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f2ccc06c8d0 0x7f2ccc0c5170 secure :-1 s=READY pgs=1511 cs=0 l=1 rev1=1 crypto rx=0x7f2cbc00c480 tx=0x7f2cbc00c940 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-09T13:53:48.542 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:48.537+0000 7f2cc2ffd640 1 -- 172.21.15.43:0/778136674 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2cbc0052d0 con 0x7f2ccc06c8d0 2023-08-09T13:53:48.542 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:48.537+0000 7f2cc2ffd640 1 -- 172.21.15.43:0/778136674 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2cbc005430 con 0x7f2ccc06c8d0 2023-08-09T13:53:48.542 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:48.537+0000 7f2cc2ffd640 1 -- 172.21.15.43:0/778136674 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2cbc012870 con 0x7f2ccc06c8d0 2023-08-09T13:53:48.542 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:48.537+0000 7f2cd0cf3640 1 -- 172.21.15.43:0/778136674 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2ccc15be20 con 0x7f2ccc06c8d0 2023-08-09T13:53:48.543 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:48.537+0000 7f2cd0cf3640 1 -- 172.21.15.43:0/778136674 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2ccc15c3e0 con 0x7f2ccc06c8d0 2023-08-09T13:53:48.543 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:48.538+0000 7f2cd0cf3640 1 -- 172.21.15.43:0/778136674 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2ccc0c8b30 con 0x7f2ccc06c8d0 2023-08-09T13:53:48.543 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:48.539+0000 7f2cc2ffd640 1 -- 172.21.15.43:0/778136674 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 56) v1 ==== 88443+0+0 (secure 0 0 0) 0x7f2cbc0129d0 con 0x7f2ccc06c8d0 2023-08-09T13:53:48.544 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:48.540+0000 7f2cc2ffd640 1 --2- 172.21.15.43:0/778136674 >> [v2:172.21.15.203:6832/2527059680,v1:172.21.15.203:6833/2527059680] conn(0x7f2cac06a170 0x7f2cac06c630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:53:48.544 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:48.540+0000 7f2cc2ffd640 1 -- 172.21.15.43:0/778136674 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(128..128 src has 1..128) v4 ==== 6426+0+0 (secure 0 0 0) 0x7f2cbc088480 con 0x7f2ccc06c8d0 2023-08-09T13:53:48.544 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:48.541+0000 7f2cc3fff640 1 --2- 172.21.15.43:0/778136674 >> [v2:172.21.15.203:6832/2527059680,v1:172.21.15.203:6833/2527059680] conn(0x7f2cac06a170 0x7f2cac06c630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:53:48.544 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:48.542+0000 7f2cc3fff640 1 --2- 172.21.15.43:0/778136674 >> [v2:172.21.15.203:6832/2527059680,v1:172.21.15.203:6833/2527059680] conn(0x7f2cac06a170 0x7f2cac06c630 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f2cb4006fd0 tx=0x7f2cb4008040 comp rx=0 tx=0).ready entity=mgr.6377 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-09T13:53:48.546 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:48.544+0000 7f2cc2ffd640 1 -- 172.21.15.43:0/778136674 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178066 (secure 0 0 0) 0x7f2cbc0538e0 con 0x7f2ccc06c8d0 2023-08-09T13:53:48.734 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:48.732+0000 7f2cd0cf3640 1 -- 172.21.15.43:0/778136674 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f2ccc0739f0 con 0x7f2ccc06c8d0 2023-08-09T13:53:48.737 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:48.734+0000 7f2cc2ffd640 1 -- 172.21.15.43:0/778136674 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v128) v1 ==== 74+0+18527 (secure 0 0 0) 0x7f2cbc056f00 con 0x7f2ccc06c8d0 2023-08-09T13:53:48.737 INFO:teuthology.orchestra.run.smithi043.stdout: 2023-08-09T13:53:48.738 INFO:teuthology.orchestra.run.smithi043.stdout:{"epoch":128,"fsid":"daac87ee-1337-4250-a955-aea4e408a39b","created":"2023-08-09T13:33:41.701198+0000","modified":"2023-08-09T13:53:47.746963+0000","last_up_change":"2023-08-09T13:33:53.783453+0000","last_in_change":"2023-08-09T13:33:45.148904+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":17,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":"rbd","create_time":"2023-08-09T13:33:54.365137+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":".mgr","create_time":"2023-08-09T13:33:54.831803+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":16,"pool_name":"cephfs_metadata","create_time":"2023-08-09T13:53:46.124616+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"128","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":17,"pool_name":"cephfs_data","create_time":"2023-08-09T13:53:46.762733+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"128","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"cbd49ecd-e896-4fbd-864e-5c37a829b165","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":127,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6800","nonce":502667335},{"type":"v1","addr":"172.21.15.43:6801","nonce":502667335}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6802","nonce":502667335},{"type":"v1","addr":"172.21.15.43:6803","nonce":502667335}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6806","nonce":502667335},{"type":"v1","addr":"172.21.15.43:6807","nonce":502667335}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6804","nonce":502667335},{"type":"v1","addr":"172.21.15.43:6805","nonce":502667335}]},"public_addr":"172.21.15.43:6801/502667335","cluster_addr":"172.21.15.43:6803/502667335","heartbeat_back_addr":"172.21.15.43:6807/502667335","heartbeat_front_addr":"172.21.15.43:6805/502667335","state":["exists","up"]},{"osd":1,"uuid":"75c7e0fe-1ec7-4fe8-a2f2-3661a65b8500","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":127,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6808","nonce":3289134610},{"type":"v1","addr":"172.21.15.43:6809","nonce":3289134610}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6810","nonce":3289134610},{"type":"v1","addr":"172.21.15.43:6812","nonce":3289134610}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6821","nonce":3289134610},{"type":"v1","addr":"172.21.15.43:6824","nonce":3289134610}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6815","nonce":3289134610},{"type":"v1","addr":"172.21.15.43:6818","nonce":3289134610}]},"public_addr":"172.21.15.43:6809/3289134610","cluster_addr":"172.21.15.43:6812/3289134610","heartbeat_back_addr":"172.21.15.43:6824/3289134610","heartbeat_front_addr":"172.21.15.43:6818/3289134610","state":["exists","up"]},{"osd":2,"uuid":"bb5c41e9-7f5f-4125-9b01-3bb8bfc2ea83","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":127,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6811","nonce":4079709736},{"type":"v1","addr":"172.21.15.43:6814","nonce":4079709736}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6817","nonce":4079709736},{"type":"v1","addr":"172.21.15.43:6820","nonce":4079709736}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6829","nonce":4079709736},{"type":"v1","addr":"172.21.15.43:6831","nonce":4079709736}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6823","nonce":4079709736},{"type":"v1","addr":"172.21.15.43:6826","nonce":4079709736}]},"public_addr":"172.21.15.43:6814/4079709736","cluster_addr":"172.21.15.43:6820/4079709736","heartbeat_back_addr":"172.21.15.43:6831/4079709736","heartbeat_front_addr":"172.21.15.43:6826/4079709736","state":["exists","up"]},{"osd":3,"uuid":"bfe4c8b4-694a-465c-87e6-2b2aab852613","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":127,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6813","nonce":1709538759},{"type":"v1","addr":"172.21.15.43:6816","nonce":1709538759}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6819","nonce":1709538759},{"type":"v1","addr":"172.21.15.43:6822","nonce":1709538759}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6828","nonce":1709538759},{"type":"v1","addr":"172.21.15.43:6830","nonce":1709538759}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6825","nonce":1709538759},{"type":"v1","addr":"172.21.15.43:6827","nonce":1709538759}]},"public_addr":"172.21.15.43:6816/1709538759","cluster_addr":"172.21.15.43:6822/1709538759","heartbeat_back_addr":"172.21.15.43:6830/1709538759","heartbeat_front_addr":"172.21.15.43:6827/1709538759","state":["exists","up"]},{"osd":4,"uuid":"fa2e8a77-aba1-4756-853e-0d174a5bdeba","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":127,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6800","nonce":4196866481},{"type":"v1","addr":"172.21.15.203:6801","nonce":4196866481}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6802","nonce":4196866481},{"type":"v1","addr":"172.21.15.203:6803","nonce":4196866481}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6806","nonce":4196866481},{"type":"v1","addr":"172.21.15.203:6807","nonce":4196866481}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6804","nonce":4196866481},{"type":"v1","addr":"172.21.15.203:6805","nonce":4196866481}]},"public_addr":"172.21.15.203:6801/4196866481","cluster_addr":"172.21.15.203:6803/4196866481","heartbeat_back_addr":"172.21.15.203:6807/4196866481","heartbeat_front_addr":"172.21.15.203:6805/4196866481","state":["exists","up"]},{"osd":5,"uuid":"81811e4b-4d24-4f51-a82c-f1d700a2a00d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":127,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6816","nonce":3059290081},{"type":"v1","addr":"172.21.15.203:6817","nonce":3059290081}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6818","nonce":3059290081},{"type":"v1","addr":"172.21.15.203:6819","nonce":3059290081}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6822","nonce":3059290081},{"type":"v1","addr":"172.21.15.203:6823","nonce":3059290081}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6820","nonce":3059290081},{"type":"v1","addr":"172.21.15.203:6821","nonce":3059290081}]},"public_addr":"172.21.15.203:6817/3059290081","cluster_addr":"172.21.15.203:6819/3059290081","heartbeat_back_addr":"172.21.15.203:6823/3059290081","heartbeat_front_addr":"172.21.15.203:6821/3059290081","state":["exists","up"]},{"osd":6,"uuid":"307bac4d-6a5b-443d-af75-56d695d7674f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":127,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6824","nonce":512155569},{"type":"v1","addr":"172.21.15.203:6825","nonce":512155569}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6826","nonce":512155569},{"type":"v1","addr":"172.21.15.203:6827","nonce":512155569}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6830","nonce":512155569},{"type":"v1","addr":"172.21.15.203:6831","nonce":512155569}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6828","nonce":512155569},{"type":"v1","addr":"172.21.15.203:6829","nonce":512155569}]},"public_addr":"172.21.15.203:6825/512155569","cluster_addr":"172.21.15.203:6827/512155569","heartbeat_back_addr":"172.21.15.203:6831/512155569","heartbeat_front_addr":"172.21.15.203:6829/512155569","state":["exists","up"]},{"osd":7,"uuid":"a95dc7ca-9f1b-4295-af70-68a22a2d7d5e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":127,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6808","nonce":3331450438},{"type":"v1","addr":"172.21.15.203:6809","nonce":3331450438}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6810","nonce":3331450438},{"type":"v1","addr":"172.21.15.203:6811","nonce":3331450438}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6814","nonce":3331450438},{"type":"v1","addr":"172.21.15.203:6815","nonce":3331450438}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6812","nonce":3331450438},{"type":"v1","addr":"172.21.15.203:6813","nonce":3331450438}]},"public_addr":"172.21.15.203:6809/3331450438","cluster_addr":"172.21.15.203:6811/3331450438","heartbeat_back_addr":"172.21.15.203:6815/3331450438","heartbeat_front_addr":"172.21.15.203:6813/3331450438","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-09T13:33:50.547652+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-09T13:33:50.589469+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-09T13:33:50.561724+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-09T13:33:50.599650+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-09T13:33:50.411855+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-09T13:33:50.219681+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-09T13:33:50.504664+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-09T13:33:50.441248+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":7,"snaps":[{"begin":104,"length":1},{"begin":221,"length":1}]}],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-08-09T13:53:48.738 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:48.736+0000 7f2cd0cf3640 1 -- 172.21.15.43:0/778136674 >> [v2:172.21.15.203:6832/2527059680,v1:172.21.15.203:6833/2527059680] conn(0x7f2cac06a170 msgr2=0x7f2cac06c630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:53:48.739 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:48.736+0000 7f2cd0cf3640 1 --2- 172.21.15.43:0/778136674 >> [v2:172.21.15.203:6832/2527059680,v1:172.21.15.203:6833/2527059680] conn(0x7f2cac06a170 0x7f2cac06c630 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f2cb4006fd0 tx=0x7f2cb4008040 comp rx=0 tx=0).stop 2023-08-09T13:53:48.739 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:48.736+0000 7f2cd0cf3640 1 -- 172.21.15.43:0/778136674 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f2ccc06c8d0 msgr2=0x7f2ccc0c5170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:53:48.740 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:48.736+0000 7f2cd0cf3640 1 --2- 172.21.15.43:0/778136674 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f2ccc06c8d0 0x7f2ccc0c5170 secure :-1 s=READY pgs=1511 cs=0 l=1 rev1=1 crypto rx=0x7f2cbc00c480 tx=0x7f2cbc00c940 comp rx=0 tx=0).stop 2023-08-09T13:53:48.740 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:48.736+0000 7f2cd0cf3640 1 -- 172.21.15.43:0/778136674 shutdown_connections 2023-08-09T13:53:48.740 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:48.736+0000 7f2cd0cf3640 1 --2- 172.21.15.43:0/778136674 >> [v2:172.21.15.203:6832/2527059680,v1:172.21.15.203:6833/2527059680] conn(0x7f2cac06a170 0x7f2cac06c630 unknown :-1 s=CLOSED pgs=58 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:53:48.740 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:48.736+0000 7f2cd0cf3640 1 --2- 172.21.15.43:0/778136674 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f2ccc143c20 0x7f2ccc132850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:53:48.740 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:48.737+0000 7f2cd0cf3640 1 --2- 172.21.15.43:0/778136674 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f2ccc0c86f0 0x7f2ccc1f2020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:53:48.740 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:48.737+0000 7f2cd0cf3640 1 --2- 172.21.15.43:0/778136674 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f2ccc06c8d0 0x7f2ccc0c5170 unknown :-1 s=CLOSED pgs=1511 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:53:48.741 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:48.737+0000 7f2cd0cf3640 1 -- 172.21.15.43:0/778136674 >> 172.21.15.43:0/778136674 conn(0x7f2ccc0ce0a0 msgr2=0x7f2ccc0706d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-09T13:53:48.741 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:48.737+0000 7f2cd0cf3640 1 -- 172.21.15.43:0/778136674 shutdown_connections 2023-08-09T13:53:48.741 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:48.737+0000 7f2cd0cf3640 1 -- 172.21.15.43:0/778136674 wait complete. 2023-08-09T13:53:48.757 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-08-09T13:53:48.968 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:48.966+0000 7efdfaa78640 1 Processor -- start 2023-08-09T13:53:48.969 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:48.966+0000 7efdfaa78640 1 -- start start 2023-08-09T13:53:48.970 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:48.968+0000 7efdfaa78640 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7efdf40b1050 0x7efdf40b4760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:53:48.971 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:48.968+0000 7efdfaa78640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7efdf40cd6b0 0x7efdf40d0e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:53:48.971 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:48.968+0000 7efdfaa78640 1 -- --> v1:172.21.15.203:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7efdf40b7bf0 con 0x7efdf415f7b0 2023-08-09T13:53:48.971 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:48.968+0000 7efdfaa78640 1 -- --> v2:172.21.15.203:3300/0 -- mon_getmap magic: 0 v1 -- 0x7efdf40b7ed0 con 0x7efdf40b1050 2023-08-09T13:53:48.971 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:48.968+0000 7efdfaa78640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7efdf40d0370 con 0x7efdf40cd6b0 2023-08-09T13:53:48.972 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:48.969+0000 7efdfa277640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7efdf40cd6b0 0x7efdf40d0e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:53:48.972 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:48.969+0000 7efdfa277640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7efdf40cd6b0 0x7efdf40d0e50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.203:3301/0 says I am v2:172.21.15.43:35742/0 (socket says 172.21.15.43:35742) 2023-08-09T13:53:48.972 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:48.969+0000 7efdfa277640 1 -- 172.21.15.43:0/2944039859 learned_addr learned my addr 172.21.15.43:0/2944039859 (peer_addr_for_me v2:172.21.15.43:0/0) 2023-08-09T13:53:48.972 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:48.969+0000 7efdf9275640 1 --2- 172.21.15.43:0/2944039859 >> v2:172.21.15.203:3300/0 conn(0x7efdf40b1050 0x7efdf40b4760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:53:48.973 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:48.969+0000 7efdfa277640 1 -- 172.21.15.43:0/2944039859 >> v2:172.21.15.203:3300/0 conn(0x7efdf40b1050 msgr2=0x7efdf40b4760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-09T13:53:48.973 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:48.969+0000 7efdfa277640 1 --2- 172.21.15.43:0/2944039859 >> v2:172.21.15.203:3300/0 conn(0x7efdf40b1050 0x7efdf40b4760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:53:48.973 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:48.969+0000 7efdfa277640 1 -- 172.21.15.43:0/2944039859 >> v1:172.21.15.203:6789/0 conn(0x7efdf415f7b0 legacy=0x7efdf40ccfa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:53:48.973 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:48.969+0000 7efdfa277640 1 -- 172.21.15.43:0/2944039859 --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efdf40d0650 con 0x7efdf40cd6b0 2023-08-09T13:53:48.973 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:48.971+0000 7efdfa277640 1 --2- 172.21.15.43:0/2944039859 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7efdf40cd6b0 0x7efdf40d0e50 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7efdec009e50 tx=0x7efdec02bf60 comp rx=0 tx=0).ready entity=mon.2 client_cookie=3c66efa7a264ffd2 server_cookie=0 in_seq=0 out_seq=0 2023-08-09T13:53:48.974 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:48.972+0000 7efdf8a74640 1 -- 172.21.15.43:0/2944039859 <== mon.2 v2:172.21.15.203:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efdec02cb00 con 0x7efdf40cd6b0 2023-08-09T13:53:48.974 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:48.972+0000 7efdf8a74640 1 -- 172.21.15.43:0/2944039859 <== mon.2 v2:172.21.15.203:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efdec02cc60 con 0x7efdf40cd6b0 2023-08-09T13:53:48.974 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:48.972+0000 7efdf8a74640 1 -- 172.21.15.43:0/2944039859 <== mon.2 v2:172.21.15.203:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efdec0343f0 con 0x7efdf40cd6b0 2023-08-09T13:53:48.975 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:48.972+0000 7efdfaa78640 1 -- 172.21.15.43:0/2944039859 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7efdf40cd6b0 msgr2=0x7efdf40d0e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:53:48.975 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:48.972+0000 7efdfaa78640 1 --2- 172.21.15.43:0/2944039859 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7efdf40cd6b0 0x7efdf40d0e50 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7efdec009e50 tx=0x7efdec02bf60 comp rx=0 tx=0).stop 2023-08-09T13:53:48.975 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:48.972+0000 7efdfaa78640 1 -- 172.21.15.43:0/2944039859 shutdown_connections 2023-08-09T13:53:48.976 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:48.972+0000 7efdfaa78640 1 --2- 172.21.15.43:0/2944039859 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7efdf40cd6b0 0x7efdf40d0e50 unknown :-1 s=CLOSED pgs=204 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:53:48.976 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:48.972+0000 7efdfaa78640 1 --2- 172.21.15.43:0/2944039859 >> v2:172.21.15.203:3300/0 conn(0x7efdf40b1050 0x7efdf40b4760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:53:48.976 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:48.972+0000 7efdfaa78640 1 -- 172.21.15.43:0/2944039859 >> 172.21.15.43:0/2944039859 conn(0x7efdf406a2b0 msgr2=0x7efdf406a690 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-09T13:53:48.976 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:48.972+0000 7efdfaa78640 1 -- 172.21.15.43:0/2944039859 shutdown_connections 2023-08-09T13:53:48.976 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:48.972+0000 7efdfaa78640 1 -- 172.21.15.43:0/2944039859 wait complete. 2023-08-09T13:53:48.977 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:48.974+0000 7efdfaa78640 1 Processor -- start 2023-08-09T13:53:48.977 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:48.974+0000 7efdfaa78640 1 -- start start 2023-08-09T13:53:48.978 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:48.975+0000 7efdfaa78640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7efdf40b1050 0x7efdf40c8240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:53:48.978 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:48.975+0000 7efdfaa78640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7efdf40cd6b0 0x7efdf41ef320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:53:48.978 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:48.975+0000 7efdfaa78640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7efdf415f7b0 0x7efdf40c5e00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:53:48.978 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:48.975+0000 7efdfaa78640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efdf41f1f80 con 0x7efdf40b1050 2023-08-09T13:53:48.978 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:48.975+0000 7efdfaa78640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efdf41f5a00 con 0x7efdf415f7b0 2023-08-09T13:53:48.978 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:48.975+0000 7efdfaa78640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7efdf41f5ce0 con 0x7efdf40cd6b0 2023-08-09T13:53:48.979 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:48.975+0000 7efdf9a76640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7efdf40b1050 0x7efdf40c8240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:53:48.979 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:48.976+0000 7efdf9a76640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7efdf40b1050 0x7efdf40c8240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:56314/0 (socket says 172.21.15.43:56314) 2023-08-09T13:53:48.979 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:48.976+0000 7efdf9a76640 1 -- 172.21.15.43:0/2118420474 learned_addr learned my addr 172.21.15.43:0/2118420474 (peer_addr_for_me v2:172.21.15.43:0/0) 2023-08-09T13:53:48.980 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:48.976+0000 7efdfa277640 1 --2- 172.21.15.43:0/2118420474 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7efdf415f7b0 0x7efdf40c5e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:53:48.980 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:48.976+0000 7efdf9275640 1 --2- 172.21.15.43:0/2118420474 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7efdf40cd6b0 0x7efdf41ef320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:53:48.980 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:48.976+0000 7efdf9a76640 1 -- 172.21.15.43:0/2118420474 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7efdf40cd6b0 msgr2=0x7efdf41ef320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:53:48.981 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:48.976+0000 7efdf9a76640 1 --2- 172.21.15.43:0/2118420474 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7efdf40cd6b0 0x7efdf41ef320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:53:48.981 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:48.976+0000 7efdf9a76640 1 -- 172.21.15.43:0/2118420474 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7efdf415f7b0 msgr2=0x7efdf40c5e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:53:48.981 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:48.976+0000 7efdf9a76640 1 --2- 172.21.15.43:0/2118420474 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7efdf415f7b0 0x7efdf40c5e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:53:48.981 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:48.976+0000 7efdf9a76640 1 -- 172.21.15.43:0/2118420474 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efdec009b30 con 0x7efdf40b1050 2023-08-09T13:53:48.981 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:48.976+0000 7efdf9a76640 1 --2- 172.21.15.43:0/2118420474 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7efdf40b1050 0x7efdf40c8240 secure :-1 s=READY pgs=1512 cs=0 l=1 rev1=1 crypto rx=0x7efdf000c400 tx=0x7efdf000c8c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-09T13:53:48.981 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:48.977+0000 7efdea7fc640 1 -- 172.21.15.43:0/2118420474 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efdf00052d0 con 0x7efdf40b1050 2023-08-09T13:53:48.981 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:48.977+0000 7efdea7fc640 1 -- 172.21.15.43:0/2118420474 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efdf0005430 con 0x7efdf40b1050 2023-08-09T13:53:48.982 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:48.977+0000 7efdea7fc640 1 -- 172.21.15.43:0/2118420474 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efdf0003830 con 0x7efdf40b1050 2023-08-09T13:53:48.982 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:48.977+0000 7efdfaa78640 1 -- 172.21.15.43:0/2118420474 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7efdf41f5fc0 con 0x7efdf40b1050 2023-08-09T13:53:48.982 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:48.977+0000 7efdfaa78640 1 -- 172.21.15.43:0/2118420474 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7efdf40d2b50 con 0x7efdf40b1050 2023-08-09T13:53:48.982 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:48.978+0000 7efdfaa78640 1 -- 172.21.15.43:0/2118420474 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7efdf40739f0 con 0x7efdf40b1050 2023-08-09T13:53:48.982 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:48.979+0000 7efdea7fc640 1 -- 172.21.15.43:0/2118420474 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 56) v1 ==== 88443+0+0 (secure 0 0 0) 0x7efdf0003990 con 0x7efdf40b1050 2023-08-09T13:53:48.983 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:48.980+0000 7efdea7fc640 1 --2- 172.21.15.43:0/2118420474 >> [v2:172.21.15.203:6832/2527059680,v1:172.21.15.203:6833/2527059680] conn(0x7efdd806a170 0x7efdd806c630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:53:48.983 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:48.981+0000 7efdea7fc640 1 -- 172.21.15.43:0/2118420474 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(128..128 src has 1..128) v4 ==== 6426+0+0 (secure 0 0 0) 0x7efdf0087e10 con 0x7efdf40b1050 2023-08-09T13:53:48.983 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:48.981+0000 7efdf9275640 1 --2- 172.21.15.43:0/2118420474 >> [v2:172.21.15.203:6832/2527059680,v1:172.21.15.203:6833/2527059680] conn(0x7efdd806a170 0x7efdd806c630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:53:48.984 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:48.982+0000 7efdf9275640 1 --2- 172.21.15.43:0/2118420474 >> [v2:172.21.15.203:6832/2527059680,v1:172.21.15.203:6833/2527059680] conn(0x7efdd806a170 0x7efdd806c630 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7efdf40cfbd0 tx=0x7efde4008040 comp rx=0 tx=0).ready entity=mgr.6377 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-09T13:53:48.987 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:48.985+0000 7efdea7fc640 1 -- 172.21.15.43:0/2118420474 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178066 (secure 0 0 0) 0x7efdf008b030 con 0x7efdf40b1050 2023-08-09T13:53:49.174 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:49.171+0000 7efdfaa78640 1 -- 172.21.15.43:0/2118420474 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7efdf40d2ce0 con 0x7efdf40b1050 2023-08-09T13:53:49.177 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:49.174+0000 7efdea7fc640 1 -- 172.21.15.43:0/2118420474 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v128) v1 ==== 74+0+18527 (secure 0 0 0) 0x7efdf0053270 con 0x7efdf40b1050 2023-08-09T13:53:49.177 INFO:teuthology.orchestra.run.smithi043.stdout: 2023-08-09T13:53:49.178 INFO:teuthology.orchestra.run.smithi043.stdout:{"epoch":128,"fsid":"daac87ee-1337-4250-a955-aea4e408a39b","created":"2023-08-09T13:33:41.701198+0000","modified":"2023-08-09T13:53:47.746963+0000","last_up_change":"2023-08-09T13:33:53.783453+0000","last_in_change":"2023-08-09T13:33:45.148904+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":17,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":"rbd","create_time":"2023-08-09T13:33:54.365137+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":".mgr","create_time":"2023-08-09T13:33:54.831803+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":16,"pool_name":"cephfs_metadata","create_time":"2023-08-09T13:53:46.124616+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"128","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":17,"pool_name":"cephfs_data","create_time":"2023-08-09T13:53:46.762733+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"128","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"cbd49ecd-e896-4fbd-864e-5c37a829b165","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":127,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6800","nonce":502667335},{"type":"v1","addr":"172.21.15.43:6801","nonce":502667335}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6802","nonce":502667335},{"type":"v1","addr":"172.21.15.43:6803","nonce":502667335}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6806","nonce":502667335},{"type":"v1","addr":"172.21.15.43:6807","nonce":502667335}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6804","nonce":502667335},{"type":"v1","addr":"172.21.15.43:6805","nonce":502667335}]},"public_addr":"172.21.15.43:6801/502667335","cluster_addr":"172.21.15.43:6803/502667335","heartbeat_back_addr":"172.21.15.43:6807/502667335","heartbeat_front_addr":"172.21.15.43:6805/502667335","state":["exists","up"]},{"osd":1,"uuid":"75c7e0fe-1ec7-4fe8-a2f2-3661a65b8500","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":127,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6808","nonce":3289134610},{"type":"v1","addr":"172.21.15.43:6809","nonce":3289134610}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6810","nonce":3289134610},{"type":"v1","addr":"172.21.15.43:6812","nonce":3289134610}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6821","nonce":3289134610},{"type":"v1","addr":"172.21.15.43:6824","nonce":3289134610}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6815","nonce":3289134610},{"type":"v1","addr":"172.21.15.43:6818","nonce":3289134610}]},"public_addr":"172.21.15.43:6809/3289134610","cluster_addr":"172.21.15.43:6812/3289134610","heartbeat_back_addr":"172.21.15.43:6824/3289134610","heartbeat_front_addr":"172.21.15.43:6818/3289134610","state":["exists","up"]},{"osd":2,"uuid":"bb5c41e9-7f5f-4125-9b01-3bb8bfc2ea83","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":127,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6811","nonce":4079709736},{"type":"v1","addr":"172.21.15.43:6814","nonce":4079709736}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6817","nonce":4079709736},{"type":"v1","addr":"172.21.15.43:6820","nonce":4079709736}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6829","nonce":4079709736},{"type":"v1","addr":"172.21.15.43:6831","nonce":4079709736}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6823","nonce":4079709736},{"type":"v1","addr":"172.21.15.43:6826","nonce":4079709736}]},"public_addr":"172.21.15.43:6814/4079709736","cluster_addr":"172.21.15.43:6820/4079709736","heartbeat_back_addr":"172.21.15.43:6831/4079709736","heartbeat_front_addr":"172.21.15.43:6826/4079709736","state":["exists","up"]},{"osd":3,"uuid":"bfe4c8b4-694a-465c-87e6-2b2aab852613","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":127,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6813","nonce":1709538759},{"type":"v1","addr":"172.21.15.43:6816","nonce":1709538759}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6819","nonce":1709538759},{"type":"v1","addr":"172.21.15.43:6822","nonce":1709538759}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6828","nonce":1709538759},{"type":"v1","addr":"172.21.15.43:6830","nonce":1709538759}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6825","nonce":1709538759},{"type":"v1","addr":"172.21.15.43:6827","nonce":1709538759}]},"public_addr":"172.21.15.43:6816/1709538759","cluster_addr":"172.21.15.43:6822/1709538759","heartbeat_back_addr":"172.21.15.43:6830/1709538759","heartbeat_front_addr":"172.21.15.43:6827/1709538759","state":["exists","up"]},{"osd":4,"uuid":"fa2e8a77-aba1-4756-853e-0d174a5bdeba","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":127,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6800","nonce":4196866481},{"type":"v1","addr":"172.21.15.203:6801","nonce":4196866481}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6802","nonce":4196866481},{"type":"v1","addr":"172.21.15.203:6803","nonce":4196866481}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6806","nonce":4196866481},{"type":"v1","addr":"172.21.15.203:6807","nonce":4196866481}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6804","nonce":4196866481},{"type":"v1","addr":"172.21.15.203:6805","nonce":4196866481}]},"public_addr":"172.21.15.203:6801/4196866481","cluster_addr":"172.21.15.203:6803/4196866481","heartbeat_back_addr":"172.21.15.203:6807/4196866481","heartbeat_front_addr":"172.21.15.203:6805/4196866481","state":["exists","up"]},{"osd":5,"uuid":"81811e4b-4d24-4f51-a82c-f1d700a2a00d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":127,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6816","nonce":3059290081},{"type":"v1","addr":"172.21.15.203:6817","nonce":3059290081}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6818","nonce":3059290081},{"type":"v1","addr":"172.21.15.203:6819","nonce":3059290081}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6822","nonce":3059290081},{"type":"v1","addr":"172.21.15.203:6823","nonce":3059290081}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6820","nonce":3059290081},{"type":"v1","addr":"172.21.15.203:6821","nonce":3059290081}]},"public_addr":"172.21.15.203:6817/3059290081","cluster_addr":"172.21.15.203:6819/3059290081","heartbeat_back_addr":"172.21.15.203:6823/3059290081","heartbeat_front_addr":"172.21.15.203:6821/3059290081","state":["exists","up"]},{"osd":6,"uuid":"307bac4d-6a5b-443d-af75-56d695d7674f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":127,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6824","nonce":512155569},{"type":"v1","addr":"172.21.15.203:6825","nonce":512155569}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6826","nonce":512155569},{"type":"v1","addr":"172.21.15.203:6827","nonce":512155569}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6830","nonce":512155569},{"type":"v1","addr":"172.21.15.203:6831","nonce":512155569}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6828","nonce":512155569},{"type":"v1","addr":"172.21.15.203:6829","nonce":512155569}]},"public_addr":"172.21.15.203:6825/512155569","cluster_addr":"172.21.15.203:6827/512155569","heartbeat_back_addr":"172.21.15.203:6831/512155569","heartbeat_front_addr":"172.21.15.203:6829/512155569","state":["exists","up"]},{"osd":7,"uuid":"a95dc7ca-9f1b-4295-af70-68a22a2d7d5e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":127,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6808","nonce":3331450438},{"type":"v1","addr":"172.21.15.203:6809","nonce":3331450438}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6810","nonce":3331450438},{"type":"v1","addr":"172.21.15.203:6811","nonce":3331450438}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6814","nonce":3331450438},{"type":"v1","addr":"172.21.15.203:6815","nonce":3331450438}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6812","nonce":3331450438},{"type":"v1","addr":"172.21.15.203:6813","nonce":3331450438}]},"public_addr":"172.21.15.203:6809/3331450438","cluster_addr":"172.21.15.203:6811/3331450438","heartbeat_back_addr":"172.21.15.203:6815/3331450438","heartbeat_front_addr":"172.21.15.203:6813/3331450438","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-09T13:33:50.547652+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-09T13:33:50.589469+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-09T13:33:50.561724+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-09T13:33:50.599650+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-09T13:33:50.411855+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-09T13:33:50.219681+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-09T13:33:50.504664+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-09T13:33:50.441248+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":7,"snaps":[{"begin":104,"length":1},{"begin":221,"length":1}]}],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-08-09T13:53:49.180 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:49.177+0000 7efdfaa78640 1 -- 172.21.15.43:0/2118420474 >> [v2:172.21.15.203:6832/2527059680,v1:172.21.15.203:6833/2527059680] conn(0x7efdd806a170 msgr2=0x7efdd806c630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:53:49.181 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:49.177+0000 7efdfaa78640 1 --2- 172.21.15.43:0/2118420474 >> [v2:172.21.15.203:6832/2527059680,v1:172.21.15.203:6833/2527059680] conn(0x7efdd806a170 0x7efdd806c630 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7efdf40cfbd0 tx=0x7efde4008040 comp rx=0 tx=0).stop 2023-08-09T13:53:49.181 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:49.177+0000 7efdfaa78640 1 -- 172.21.15.43:0/2118420474 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7efdf40b1050 msgr2=0x7efdf40c8240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:53:49.181 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:49.177+0000 7efdfaa78640 1 --2- 172.21.15.43:0/2118420474 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7efdf40b1050 0x7efdf40c8240 secure :-1 s=READY pgs=1512 cs=0 l=1 rev1=1 crypto rx=0x7efdf000c400 tx=0x7efdf000c8c0 comp rx=0 tx=0).stop 2023-08-09T13:53:49.181 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:49.177+0000 7efdfaa78640 1 -- 172.21.15.43:0/2118420474 shutdown_connections 2023-08-09T13:53:49.182 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:49.177+0000 7efdfaa78640 1 --2- 172.21.15.43:0/2118420474 >> [v2:172.21.15.203:6832/2527059680,v1:172.21.15.203:6833/2527059680] conn(0x7efdd806a170 0x7efdd806c630 unknown :-1 s=CLOSED pgs=59 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:53:49.182 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:49.177+0000 7efdfaa78640 1 --2- 172.21.15.43:0/2118420474 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7efdf415f7b0 0x7efdf40c5e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:53:49.182 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:49.177+0000 7efdfaa78640 1 --2- 172.21.15.43:0/2118420474 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7efdf40cd6b0 0x7efdf41ef320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:53:49.182 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:49.177+0000 7efdfaa78640 1 --2- 172.21.15.43:0/2118420474 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7efdf40b1050 0x7efdf40c8240 unknown :-1 s=CLOSED pgs=1512 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:53:49.183 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:49.177+0000 7efdfaa78640 1 -- 172.21.15.43:0/2118420474 >> 172.21.15.43:0/2118420474 conn(0x7efdf406a2b0 msgr2=0x7efdf415cf60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-09T13:53:49.183 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:49.177+0000 7efdfaa78640 1 -- 172.21.15.43:0/2118420474 shutdown_connections 2023-08-09T13:53:49.183 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:49.178+0000 7efdfaa78640 1 -- 172.21.15.43:0/2118420474 wait complete. 2023-08-09T13:53:49.199 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs standby_count_wanted 0 2023-08-09T13:53:49.432 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:49.429+0000 7f037469b640 1 Processor -- start 2023-08-09T13:53:49.432 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:49.429+0000 7f037469b640 1 -- start start 2023-08-09T13:53:49.434 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:49.431+0000 7f037469b640 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7f036c15c630 0x7f036c0abef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:53:49.434 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:49.431+0000 7f037469b640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f036c0ccf20 0x7f036c0cd550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:53:49.434 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:49.431+0000 7f037469b640 1 -- --> v1:172.21.15.203:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f036c0cca90 con 0x7f036c0cda90 2023-08-09T13:53:49.434 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:49.431+0000 7f037469b640 1 -- --> v2:172.21.15.43:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f036c0cbc70 con 0x7f036c0ccf20 2023-08-09T13:53:49.434 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:49.431+0000 7f037469b640 1 -- --> v2:172.21.15.203:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f036c0cdea0 con 0x7f036c15c630 2023-08-09T13:53:49.435 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:49.432+0000 7f0371c36640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f036c0ccf20 0x7f036c0cd550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:53:49.435 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:49.432+0000 7f0371c36640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f036c0ccf20 0x7f036c0cd550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:56320/0 (socket says 172.21.15.43:56320) 2023-08-09T13:53:49.435 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:49.432+0000 7f0371c36640 1 -- 172.21.15.43:0/1656475024 learned_addr learned my addr 172.21.15.43:0/1656475024 (peer_addr_for_me v2:172.21.15.43:0/0) 2023-08-09T13:53:49.436 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:49.432+0000 7f0372437640 1 --2- 172.21.15.43:0/1656475024 >> v2:172.21.15.203:3300/0 conn(0x7f036c15c630 0x7f036c0abef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:53:49.436 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:49.432+0000 7f0371c36640 1 -- 172.21.15.43:0/1656475024 >> v2:172.21.15.203:3300/0 conn(0x7f036c15c630 msgr2=0x7f036c0abef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-09T13:53:49.436 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:49.432+0000 7f0371c36640 1 --2- 172.21.15.43:0/1656475024 >> v2:172.21.15.203:3300/0 conn(0x7f036c15c630 0x7f036c0abef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:53:49.436 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:49.432+0000 7f0371c36640 1 -- 172.21.15.43:0/1656475024 >> v1:172.21.15.203:6789/0 conn(0x7f036c0cda90 legacy=0x7f036c0c2f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-09T13:53:49.436 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:49.432+0000 7f0371c36640 1 -- 172.21.15.43:0/1656475024 --> v2:172.21.15.43:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f036c15da20 con 0x7f036c0ccf20 2023-08-09T13:53:49.436 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:49.432+0000 7f0372437640 1 --2- 172.21.15.43:0/1656475024 >> v2:172.21.15.203:3300/0 conn(0x7f036c15c630 0x7f036c0abef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-08-09T13:53:49.437 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:49.434+0000 7f0371c36640 1 --2- 172.21.15.43:0/1656475024 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f036c0ccf20 0x7f036c0cd550 secure :-1 s=READY pgs=1513 cs=0 l=1 rev1=1 crypto rx=0x7f036c0cbbb0 tx=0x7f035c02b910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cefbc321a9a3f0fc server_cookie=0 in_seq=0 out_seq=0 2023-08-09T13:53:49.437 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:49.434+0000 7f0371435640 1 -- 172.21.15.43:0/1656475024 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f035c031870 con 0x7f036c0ccf20 2023-08-09T13:53:49.438 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:49.434+0000 7f0371435640 1 -- 172.21.15.43:0/1656475024 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f035c0319d0 con 0x7f036c0ccf20 2023-08-09T13:53:49.438 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:49.434+0000 7f0371435640 1 -- 172.21.15.43:0/1656475024 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f035c031d60 con 0x7f036c0ccf20 2023-08-09T13:53:49.438 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:49.435+0000 7f037469b640 1 -- 172.21.15.43:0/1656475024 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f036c0ccf20 msgr2=0x7f036c0cd550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:53:49.438 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:49.435+0000 7f037469b640 1 --2- 172.21.15.43:0/1656475024 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f036c0ccf20 0x7f036c0cd550 secure :-1 s=READY pgs=1513 cs=0 l=1 rev1=1 crypto rx=0x7f036c0cbbb0 tx=0x7f035c02b910 comp rx=0 tx=0).stop 2023-08-09T13:53:49.438 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:49.435+0000 7f037469b640 1 -- 172.21.15.43:0/1656475024 shutdown_connections 2023-08-09T13:53:49.438 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:49.435+0000 7f037469b640 1 --2- 172.21.15.43:0/1656475024 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f036c0ccf20 0x7f036c0cd550 unknown :-1 s=CLOSED pgs=1513 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:53:49.438 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:49.435+0000 7f037469b640 1 --2- 172.21.15.43:0/1656475024 >> v2:172.21.15.203:3300/0 conn(0x7f036c15c630 0x7f036c0abef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:53:49.439 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:49.435+0000 7f037469b640 1 -- 172.21.15.43:0/1656475024 >> 172.21.15.43:0/1656475024 conn(0x7f036c0ce0a0 msgr2=0x7f036c0ce480 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-09T13:53:49.439 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:49.435+0000 7f037469b640 1 -- 172.21.15.43:0/1656475024 shutdown_connections 2023-08-09T13:53:49.439 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:49.435+0000 7f037469b640 1 -- 172.21.15.43:0/1656475024 wait complete. 2023-08-09T13:53:49.439 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:49.437+0000 7f037469b640 1 Processor -- start 2023-08-09T13:53:49.440 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:49.437+0000 7f037469b640 1 -- start start 2023-08-09T13:53:49.440 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:49.438+0000 7f037469b640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f036c0ccf20 0x7f036c1f2600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:53:49.440 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:49.438+0000 7f037469b640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f036c0cda90 0x7f036c1f1370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:53:49.441 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:49.438+0000 7f037469b640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f036c15c630 0x7f036c0b2870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:53:49.441 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:49.438+0000 7f037469b640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f036c1f2b40 con 0x7f036c15c630 2023-08-09T13:53:49.441 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:49.438+0000 7f037469b640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f036c1f56d0 con 0x7f036c0ccf20 2023-08-09T13:53:49.441 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:49.438+0000 7f037469b640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f036c1f59b0 con 0x7f036c0cda90 2023-08-09T13:53:49.441 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:49.438+0000 7f0372c38640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f036c15c630 0x7f036c0b2870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:53:49.441 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:49.438+0000 7f0372c38640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f036c15c630 0x7f036c0b2870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:56326/0 (socket says 172.21.15.43:56326) 2023-08-09T13:53:49.441 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:49.438+0000 7f0372c38640 1 -- 172.21.15.43:0/2041675916 learned_addr learned my addr 172.21.15.43:0/2041675916 (peer_addr_for_me v2:172.21.15.43:0/0) 2023-08-09T13:53:49.442 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:49.438+0000 7f0372437640 1 --2- 172.21.15.43:0/2041675916 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f036c0ccf20 0x7f036c1f2600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:53:49.443 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:49.439+0000 7f0371c36640 1 --2- 172.21.15.43:0/2041675916 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f036c0cda90 0x7f036c1f1370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:53:49.443 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:49.439+0000 7f0372c38640 1 -- 172.21.15.43:0/2041675916 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f036c0cda90 msgr2=0x7f036c1f1370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:53:49.443 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:49.439+0000 7f0372c38640 1 --2- 172.21.15.43:0/2041675916 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f036c0cda90 0x7f036c1f1370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:53:49.443 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:49.439+0000 7f0372c38640 1 -- 172.21.15.43:0/2041675916 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f036c0ccf20 msgr2=0x7f036c1f2600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:53:49.443 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:49.439+0000 7f0372c38640 1 --2- 172.21.15.43:0/2041675916 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f036c0ccf20 0x7f036c1f2600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:53:49.443 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:49.439+0000 7f0372c38640 1 -- 172.21.15.43:0/2041675916 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f035c0094d0 con 0x7f036c15c630 2023-08-09T13:53:49.443 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:49.439+0000 7f0371c36640 1 --2- 172.21.15.43:0/2041675916 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f036c0cda90 0x7f036c1f1370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-08-09T13:53:49.444 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:49.439+0000 7f0372c38640 1 --2- 172.21.15.43:0/2041675916 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f036c15c630 0x7f036c0b2870 secure :-1 s=READY pgs=1514 cs=0 l=1 rev1=1 crypto rx=0x7f036400c780 tx=0x7f036400cc40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-09T13:53:49.444 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:49.439+0000 7f0362ffd640 1 -- 172.21.15.43:0/2041675916 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0364004b10 con 0x7f036c15c630 2023-08-09T13:53:49.444 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:49.439+0000 7f0362ffd640 1 -- 172.21.15.43:0/2041675916 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0364004c70 con 0x7f036c15c630 2023-08-09T13:53:49.444 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:49.439+0000 7f0362ffd640 1 -- 172.21.15.43:0/2041675916 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f036400aa90 con 0x7f036c15c630 2023-08-09T13:53:49.444 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:49.439+0000 7f037469b640 1 -- 172.21.15.43:0/2041675916 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f036c1f5c90 con 0x7f036c15c630 2023-08-09T13:53:49.444 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:49.439+0000 7f037469b640 1 -- 172.21.15.43:0/2041675916 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f036c14d5b0 con 0x7f036c15c630 2023-08-09T13:53:49.445 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:49.441+0000 7f037469b640 1 -- 172.21.15.43:0/2041675916 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f036c0739f0 con 0x7f036c15c630 2023-08-09T13:53:49.445 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:49.442+0000 7f0362ffd640 1 -- 172.21.15.43:0/2041675916 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 56) v1 ==== 88443+0+0 (secure 0 0 0) 0x7f036400abf0 con 0x7f036c15c630 2023-08-09T13:53:49.445 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:49.443+0000 7f0362ffd640 1 --2- 172.21.15.43:0/2041675916 >> [v2:172.21.15.203:6832/2527059680,v1:172.21.15.203:6833/2527059680] conn(0x7f035006a170 0x7f035006c630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:53:49.445 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:49.443+0000 7f0362ffd640 1 -- 172.21.15.43:0/2041675916 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(129..129 src has 1..129) v4 ==== 6426+0+0 (secure 0 0 0) 0x7f0364088a40 con 0x7f036c15c630 2023-08-09T13:53:49.446 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:49.443+0000 7f0372437640 1 --2- 172.21.15.43:0/2041675916 >> [v2:172.21.15.203:6832/2527059680,v1:172.21.15.203:6833/2527059680] conn(0x7f035006a170 0x7f035006c630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:53:49.446 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:49.444+0000 7f0372437640 1 --2- 172.21.15.43:0/2041675916 >> [v2:172.21.15.203:6832/2527059680,v1:172.21.15.203:6833/2527059680] conn(0x7f035006a170 0x7f035006c630 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f036800a950 tx=0x7f0368008040 comp rx=0 tx=0).ready entity=mgr.6377 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-09T13:53:49.449 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:49.447+0000 7f0362ffd640 1 -- 172.21.15.43:0/2041675916 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178066 (secure 0 0 0) 0x7f036408ae60 con 0x7f036c15c630 2023-08-09T13:53:49.687 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:49.684+0000 7f037469b640 1 -- 172.21.15.43:0/2041675916 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} v 0) v1 -- 0x7f036c1f5810 con 0x7f036c15c630 2023-08-09T13:53:50.322 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:50.319+0000 7f0362ffd640 1 -- 172.21.15.43:0/2041675916 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]=0 v60) v1 ==== 118+0+0 (secure 0 0 0) 0x7f036c1f5810 con 0x7f036c15c630 2023-08-09T13:53:50.325 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:50.323+0000 7f037469b640 1 -- 172.21.15.43:0/2041675916 >> [v2:172.21.15.203:6832/2527059680,v1:172.21.15.203:6833/2527059680] conn(0x7f035006a170 msgr2=0x7f035006c630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:53:50.326 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:50.323+0000 7f037469b640 1 --2- 172.21.15.43:0/2041675916 >> [v2:172.21.15.203:6832/2527059680,v1:172.21.15.203:6833/2527059680] conn(0x7f035006a170 0x7f035006c630 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f036800a950 tx=0x7f0368008040 comp rx=0 tx=0).stop 2023-08-09T13:53:50.326 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:50.323+0000 7f037469b640 1 -- 172.21.15.43:0/2041675916 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f036c15c630 msgr2=0x7f036c0b2870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:53:50.326 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:50.323+0000 7f037469b640 1 --2- 172.21.15.43:0/2041675916 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f036c15c630 0x7f036c0b2870 secure :-1 s=READY pgs=1514 cs=0 l=1 rev1=1 crypto rx=0x7f036400c780 tx=0x7f036400cc40 comp rx=0 tx=0).stop 2023-08-09T13:53:50.327 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:50.323+0000 7f037469b640 1 -- 172.21.15.43:0/2041675916 shutdown_connections 2023-08-09T13:53:50.327 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:50.323+0000 7f037469b640 1 --2- 172.21.15.43:0/2041675916 >> [v2:172.21.15.203:6832/2527059680,v1:172.21.15.203:6833/2527059680] conn(0x7f035006a170 0x7f035006c630 unknown :-1 s=CLOSED pgs=60 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:53:50.328 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:50.323+0000 7f037469b640 1 --2- 172.21.15.43:0/2041675916 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f036c15c630 0x7f036c0b2870 unknown :-1 s=CLOSED pgs=1514 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:53:50.328 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:50.323+0000 7f037469b640 1 --2- 172.21.15.43:0/2041675916 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f036c0cda90 0x7f036c1f1370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:53:50.328 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:50.323+0000 7f037469b640 1 --2- 172.21.15.43:0/2041675916 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f036c0ccf20 0x7f036c1f2600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:53:50.329 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:50.323+0000 7f037469b640 1 -- 172.21.15.43:0/2041675916 >> 172.21.15.43:0/2041675916 conn(0x7f036c0ce0a0 msgr2=0x7f036c06b790 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-09T13:53:50.329 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:50.324+0000 7f037469b640 1 -- 172.21.15.43:0/2041675916 shutdown_connections 2023-08-09T13:53:50.330 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:50.324+0000 7f037469b640 1 -- 172.21.15.43:0/2041675916 wait complete. 2023-08-09T13:53:50.347 DEBUG:tasks.cephfs.filesystem:fs_config: {} 2023-08-09T13:53:50.348 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs allow_standby_replay false 2023-08-09T13:53:50.581 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:50.577+0000 7f4c75c0f640 1 Processor -- start 2023-08-09T13:53:50.581 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:50.577+0000 7f4c75c0f640 1 -- start start 2023-08-09T13:53:50.584 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:50.579+0000 7f4c75c0f640 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7f4c700caf90 0x7f4c700cb370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:53:50.584 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:50.579+0000 7f4c75c0f640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f4c70168cc0 0x7f4c701690c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:53:50.584 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:50.579+0000 7f4c75c0f640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f4c7006f2c0 0x7f4c7006f720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:53:50.584 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:50.579+0000 7f4c75c0f640 1 -- --> v2:172.21.15.43:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4c700c8470 con 0x7f4c7006f2c0 2023-08-09T13:53:50.585 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:50.579+0000 7f4c75c0f640 1 -- --> v2:172.21.15.203:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4c700cd3e0 con 0x7f4c700caf90 2023-08-09T13:53:50.585 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:50.579+0000 7f4c75c0f640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4c700c8680 con 0x7f4c70168cc0 2023-08-09T13:53:50.585 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:50.580+0000 7f4c7540e640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f4c7006f2c0 0x7f4c7006f720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:53:50.585 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:50.580+0000 7f4c7540e640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f4c7006f2c0 0x7f4c7006f720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:56340/0 (socket says 172.21.15.43:56340) 2023-08-09T13:53:50.586 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:50.580+0000 7f4c7540e640 1 -- 172.21.15.43:0/24342272 learned_addr learned my addr 172.21.15.43:0/24342272 (peer_addr_for_me v2:172.21.15.43:0/0) 2023-08-09T13:53:50.586 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:50.580+0000 7f4c74c0d640 1 --2- 172.21.15.43:0/24342272 >> v2:172.21.15.203:3300/0 conn(0x7f4c700caf90 0x7f4c700cb370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:53:50.586 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:50.580+0000 7f4c67fff640 1 --2- 172.21.15.43:0/24342272 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f4c70168cc0 0x7f4c701690c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:53:50.587 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:50.580+0000 7f4c7540e640 1 -- 172.21.15.43:0/24342272 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f4c70168cc0 msgr2=0x7f4c701690c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-09T13:53:50.587 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:50.580+0000 7f4c7540e640 1 --2- 172.21.15.43:0/24342272 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f4c70168cc0 0x7f4c701690c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:53:50.587 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:50.580+0000 7f4c7540e640 1 -- 172.21.15.43:0/24342272 >> v2:172.21.15.203:3300/0 conn(0x7f4c700caf90 msgr2=0x7f4c700cb370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-09T13:53:50.587 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:50.580+0000 7f4c7540e640 1 --2- 172.21.15.43:0/24342272 >> v2:172.21.15.203:3300/0 conn(0x7f4c700caf90 0x7f4c700cb370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:53:50.587 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:50.580+0000 7f4c7540e640 1 -- 172.21.15.43:0/24342272 --> v2:172.21.15.43:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4c700c8960 con 0x7f4c7006f2c0 2023-08-09T13:53:50.587 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:50.580+0000 7f4c67fff640 1 --2- 172.21.15.43:0/24342272 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f4c70168cc0 0x7f4c701690c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-08-09T13:53:50.587 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:50.580+0000 7f4c74c0d640 1 --2- 172.21.15.43:0/24342272 >> v2:172.21.15.203:3300/0 conn(0x7f4c700caf90 0x7f4c700cb370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-08-09T13:53:50.588 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:50.581+0000 7f4c7540e640 1 --2- 172.21.15.43:0/24342272 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f4c7006f2c0 0x7f4c7006f720 secure :-1 s=READY pgs=1515 cs=0 l=1 rev1=1 crypto rx=0x7f4c7006cfb0 tx=0x7f4c6802ca30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bf6e834f63267508 server_cookie=0 in_seq=0 out_seq=0 2023-08-09T13:53:50.590 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:50.582+0000 7f4c677fe640 1 -- 172.21.15.43:0/24342272 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4c68030850 con 0x7f4c7006f2c0 2023-08-09T13:53:50.590 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:50.582+0000 7f4c677fe640 1 -- 172.21.15.43:0/24342272 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4c680309b0 con 0x7f4c7006f2c0 2023-08-09T13:53:50.590 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:50.582+0000 7f4c677fe640 1 -- 172.21.15.43:0/24342272 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4c68030d40 con 0x7f4c7006f2c0 2023-08-09T13:53:50.590 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:50.582+0000 7f4c75c0f640 1 -- 172.21.15.43:0/24342272 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f4c7006f2c0 msgr2=0x7f4c7006f720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:53:50.590 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:50.582+0000 7f4c75c0f640 1 --2- 172.21.15.43:0/24342272 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f4c7006f2c0 0x7f4c7006f720 secure :-1 s=READY pgs=1515 cs=0 l=1 rev1=1 crypto rx=0x7f4c7006cfb0 tx=0x7f4c6802ca30 comp rx=0 tx=0).stop 2023-08-09T13:53:50.591 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:50.582+0000 7f4c75c0f640 1 -- 172.21.15.43:0/24342272 shutdown_connections 2023-08-09T13:53:50.591 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:50.582+0000 7f4c75c0f640 1 --2- 172.21.15.43:0/24342272 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f4c7006f2c0 0x7f4c7006f720 unknown :-1 s=CLOSED pgs=1515 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:53:50.591 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:50.582+0000 7f4c75c0f640 1 --2- 172.21.15.43:0/24342272 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f4c70168cc0 0x7f4c701690c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:53:50.591 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:50.582+0000 7f4c75c0f640 1 --2- 172.21.15.43:0/24342272 >> v2:172.21.15.203:3300/0 conn(0x7f4c700caf90 0x7f4c700cb370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:53:50.592 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:50.582+0000 7f4c75c0f640 1 -- 172.21.15.43:0/24342272 >> 172.21.15.43:0/24342272 conn(0x7f4c700af9f0 msgr2=0x7f4c700afdd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-09T13:53:50.592 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:50.583+0000 7f4c75c0f640 1 -- 172.21.15.43:0/24342272 shutdown_connections 2023-08-09T13:53:50.592 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:50.583+0000 7f4c75c0f640 1 -- 172.21.15.43:0/24342272 wait complete. 2023-08-09T13:53:50.592 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:50.584+0000 7f4c75c0f640 1 Processor -- start 2023-08-09T13:53:50.592 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:50.585+0000 7f4c75c0f640 1 -- start start 2023-08-09T13:53:50.592 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:50.586+0000 7f4c75c0f640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f4c7006f2c0 0x7f4c700c04c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:53:50.593 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:50.586+0000 7f4c75c0f640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f4c700caf90 0x7f4c700c2e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:53:50.593 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:50.586+0000 7f4c75c0f640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f4c70168cc0 0x7f4c701f27c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:53:50.593 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:50.586+0000 7f4c75c0f640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4c700c40a0 con 0x7f4c700caf90 2023-08-09T13:53:50.593 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:50.586+0000 7f4c75c0f640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4c700c3490 con 0x7f4c70168cc0 2023-08-09T13:53:50.593 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:50.586+0000 7f4c75c0f640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4c700c3770 con 0x7f4c7006f2c0 2023-08-09T13:53:50.593 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:50.586+0000 7f4c67fff640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f4c700caf90 0x7f4c700c2e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:53:50.593 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:50.586+0000 7f4c67fff640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f4c700caf90 0x7f4c700c2e20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:56346/0 (socket says 172.21.15.43:56346) 2023-08-09T13:53:50.593 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:50.586+0000 7f4c67fff640 1 -- 172.21.15.43:0/1024502683 learned_addr learned my addr 172.21.15.43:0/1024502683 (peer_addr_for_me v2:172.21.15.43:0/0) 2023-08-09T13:53:50.594 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:50.586+0000 7f4c7540e640 1 --2- 172.21.15.43:0/1024502683 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f4c70168cc0 0x7f4c701f27c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:53:50.594 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:50.586+0000 7f4c67fff640 1 -- 172.21.15.43:0/1024502683 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f4c7006f2c0 msgr2=0x7f4c700c04c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:53:50.594 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:50.586+0000 7f4c67fff640 1 --2- 172.21.15.43:0/1024502683 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f4c7006f2c0 0x7f4c700c04c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:53:50.594 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:50.586+0000 7f4c67fff640 1 -- 172.21.15.43:0/1024502683 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f4c70168cc0 msgr2=0x7f4c701f27c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:53:50.594 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:50.586+0000 7f4c67fff640 1 --2- 172.21.15.43:0/1024502683 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f4c70168cc0 0x7f4c701f27c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:53:50.594 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:50.587+0000 7f4c67fff640 1 -- 172.21.15.43:0/1024502683 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4c680094d0 con 0x7f4c700caf90 2023-08-09T13:53:50.594 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:50.587+0000 7f4c67fff640 1 --2- 172.21.15.43:0/1024502683 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f4c700caf90 0x7f4c700c2e20 secure :-1 s=READY pgs=1516 cs=0 l=1 rev1=1 crypto rx=0x7f4c6000c7f0 tx=0x7f4c6000ccb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-09T13:53:50.595 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:50.587+0000 7f4c657fa640 1 -- 172.21.15.43:0/1024502683 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4c60007820 con 0x7f4c700caf90 2023-08-09T13:53:50.595 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:50.587+0000 7f4c657fa640 1 -- 172.21.15.43:0/1024502683 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4c60007980 con 0x7f4c700caf90 2023-08-09T13:53:50.595 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:50.587+0000 7f4c657fa640 1 -- 172.21.15.43:0/1024502683 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4c60007cf0 con 0x7f4c700caf90 2023-08-09T13:53:50.595 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:50.587+0000 7f4c75c0f640 1 -- 172.21.15.43:0/1024502683 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4c700c3a50 con 0x7f4c700caf90 2023-08-09T13:53:50.595 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:50.587+0000 7f4c75c0f640 1 -- 172.21.15.43:0/1024502683 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4c701f1b60 con 0x7f4c700caf90 2023-08-09T13:53:50.596 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:50.589+0000 7f4c75c0f640 1 -- 172.21.15.43:0/1024502683 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4c7004e390 con 0x7f4c700caf90 2023-08-09T13:53:50.596 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:50.589+0000 7f4c657fa640 1 -- 172.21.15.43:0/1024502683 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 56) v1 ==== 88443+0+0 (secure 0 0 0) 0x7f4c600113f0 con 0x7f4c700caf90 2023-08-09T13:53:50.596 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:50.590+0000 7f4c657fa640 1 --2- 172.21.15.43:0/1024502683 >> [v2:172.21.15.203:6832/2527059680,v1:172.21.15.203:6833/2527059680] conn(0x7f4c5c06a170 0x7f4c5c06c630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:53:50.596 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:50.590+0000 7f4c657fa640 1 -- 172.21.15.43:0/1024502683 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(129..129 src has 1..129) v4 ==== 6426+0+0 (secure 0 0 0) 0x7f4c60088c50 con 0x7f4c700caf90 2023-08-09T13:53:50.597 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:50.591+0000 7f4c74c0d640 1 --2- 172.21.15.43:0/1024502683 >> [v2:172.21.15.203:6832/2527059680,v1:172.21.15.203:6833/2527059680] conn(0x7f4c5c06a170 0x7f4c5c06c630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:53:50.597 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:50.592+0000 7f4c74c0d640 1 --2- 172.21.15.43:0/1024502683 >> [v2:172.21.15.203:6832/2527059680,v1:172.21.15.203:6833/2527059680] conn(0x7f4c5c06a170 0x7f4c5c06c630 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f4c70168a00 tx=0x7f4c6c0083d0 comp rx=0 tx=0).ready entity=mgr.6377 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-09T13:53:50.597 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:50.595+0000 7f4c657fa640 1 -- 172.21.15.43:0/1024502683 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178066 (secure 0 0 0) 0x7f4c60053fa0 con 0x7f4c700caf90 2023-08-09T13:53:50.829 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:50.827+0000 7f4c75c0f640 1 -- 172.21.15.43:0/1024502683 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"} v 0) v1 -- 0x7f4c700c41e0 con 0x7f4c700caf90 2023-08-09T13:53:51.334 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:51.331+0000 7f4c657fa640 1 -- 172.21.15.43:0/1024502683 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"}]=0 v61) v1 ==== 122+0+0 (secure 0 0 0) 0x7f4c700c41e0 con 0x7f4c700caf90 2023-08-09T13:53:51.337 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:51.335+0000 7f4c75c0f640 1 -- 172.21.15.43:0/1024502683 >> [v2:172.21.15.203:6832/2527059680,v1:172.21.15.203:6833/2527059680] conn(0x7f4c5c06a170 msgr2=0x7f4c5c06c630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:53:51.338 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:51.335+0000 7f4c75c0f640 1 --2- 172.21.15.43:0/1024502683 >> [v2:172.21.15.203:6832/2527059680,v1:172.21.15.203:6833/2527059680] conn(0x7f4c5c06a170 0x7f4c5c06c630 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f4c70168a00 tx=0x7f4c6c0083d0 comp rx=0 tx=0).stop 2023-08-09T13:53:51.338 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:51.335+0000 7f4c75c0f640 1 -- 172.21.15.43:0/1024502683 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f4c700caf90 msgr2=0x7f4c700c2e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:53:51.338 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:51.335+0000 7f4c75c0f640 1 --2- 172.21.15.43:0/1024502683 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f4c700caf90 0x7f4c700c2e20 secure :-1 s=READY pgs=1516 cs=0 l=1 rev1=1 crypto rx=0x7f4c6000c7f0 tx=0x7f4c6000ccb0 comp rx=0 tx=0).stop 2023-08-09T13:53:51.338 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:51.335+0000 7f4c75c0f640 1 -- 172.21.15.43:0/1024502683 shutdown_connections 2023-08-09T13:53:51.338 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:51.335+0000 7f4c75c0f640 1 --2- 172.21.15.43:0/1024502683 >> [v2:172.21.15.203:6832/2527059680,v1:172.21.15.203:6833/2527059680] conn(0x7f4c5c06a170 0x7f4c5c06c630 unknown :-1 s=CLOSED pgs=61 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:53:51.338 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:51.335+0000 7f4c75c0f640 1 --2- 172.21.15.43:0/1024502683 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f4c70168cc0 0x7f4c701f27c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:53:51.338 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:51.335+0000 7f4c75c0f640 1 --2- 172.21.15.43:0/1024502683 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f4c700caf90 0x7f4c700c2e20 unknown :-1 s=CLOSED pgs=1516 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:53:51.339 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:51.335+0000 7f4c75c0f640 1 --2- 172.21.15.43:0/1024502683 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f4c7006f2c0 0x7f4c700c04c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:53:51.339 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:51.335+0000 7f4c75c0f640 1 -- 172.21.15.43:0/1024502683 >> 172.21.15.43:0/1024502683 conn(0x7f4c700af9f0 msgr2=0x7f4c700cd770 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-09T13:53:51.339 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:51.335+0000 7f4c75c0f640 1 -- 172.21.15.43:0/1024502683 shutdown_connections 2023-08-09T13:53:51.339 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:51.335+0000 7f4c75c0f640 1 -- 172.21.15.43:0/1024502683 wait complete. 2023-08-09T13:53:51.358 DEBUG:tasks.cephfs.filesystem:Not Creating any subvols for filesystem 'cephfs' 2023-08-09T13:53:51.358 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-08-09T13:53:51.593 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:51.590+0000 7fca1bfff640 1 Processor -- start 2023-08-09T13:53:51.593 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:51.590+0000 7fca1bfff640 1 -- start start 2023-08-09T13:53:51.595 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:51.592+0000 7fca1bfff640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7fca1c15f7b0 0x7fca1c0b4760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:53:51.596 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:51.592+0000 7fca1bfff640 1 -- --> v1:172.21.15.43:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fca1c0b7bf0 con 0x7fca1c0b1050 2023-08-09T13:53:51.596 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:51.592+0000 7fca1bfff640 1 -- --> v1:172.21.15.203:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fca1c0b7ed0 con 0x7fca1c0cd6b0 2023-08-09T13:53:51.596 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:51.592+0000 7fca1bfff640 1 -- --> v2:172.21.15.43:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fca1c0d0370 con 0x7fca1c15f7b0 2023-08-09T13:53:51.597 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:51.592+0000 7fca1affd640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7fca1c15f7b0 0x7fca1c0b4760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:53:51.597 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:51.592+0000 7fca1a7fc640 1 --1- >> v1:172.21.15.43:6789/0 conn(0x7fca1c0b1050 0x7fca1c0ccfa0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.43:6789/0 says I am v1:172.21.15.43:53616/0 (socket says 172.21.15.43:53616) 2023-08-09T13:53:51.597 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:51.592+0000 7fca1affd640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7fca1c15f7b0 0x7fca1c0b4760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:56356/0 (socket says 172.21.15.43:56356) 2023-08-09T13:53:51.598 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:51.592+0000 7fca1a7fc640 1 -- 172.21.15.43:0/738009732 learned_addr learned my addr 172.21.15.43:0/738009732 (peer_addr_for_me v1:172.21.15.43:0/0) 2023-08-09T13:53:51.598 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:51.593+0000 7fca1affd640 1 -- 172.21.15.43:0/738009732 >> v1:172.21.15.203:6789/0 conn(0x7fca1c0cd6b0 legacy=0x7fca1c0d0e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-09T13:53:51.598 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:51.593+0000 7fca1affd640 1 -- 172.21.15.43:0/738009732 >> v1:172.21.15.43:6789/0 conn(0x7fca1c0b1050 legacy=0x7fca1c0ccfa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:53:51.598 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:51.593+0000 7fca1affd640 1 -- 172.21.15.43:0/738009732 --> v2:172.21.15.43:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fca1c0d0650 con 0x7fca1c15f7b0 2023-08-09T13:53:51.599 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:51.594+0000 7fca1affd640 1 --2- 172.21.15.43:0/738009732 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fca1c15f7b0 0x7fca1c0b4760 secure :-1 s=READY pgs=1518 cs=0 l=1 rev1=1 crypto rx=0x7fca0c009dc0 tx=0x7fca0c02be90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e05e2d1d8c52db4f server_cookie=0 in_seq=0 out_seq=0 2023-08-09T13:53:51.600 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:51.595+0000 7fca19ffb640 1 -- 172.21.15.43:0/738009732 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fca0c02cc40 con 0x7fca1c15f7b0 2023-08-09T13:53:51.600 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:51.595+0000 7fca19ffb640 1 -- 172.21.15.43:0/738009732 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fca0c02cda0 con 0x7fca1c15f7b0 2023-08-09T13:53:51.600 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:51.595+0000 7fca19ffb640 1 -- 172.21.15.43:0/738009732 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fca0c034660 con 0x7fca1c15f7b0 2023-08-09T13:53:51.600 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:51.595+0000 7fca1bfff640 1 -- 172.21.15.43:0/738009732 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fca1c15f7b0 msgr2=0x7fca1c0b4760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:53:51.600 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:51.595+0000 7fca1bfff640 1 --2- 172.21.15.43:0/738009732 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fca1c15f7b0 0x7fca1c0b4760 secure :-1 s=READY pgs=1518 cs=0 l=1 rev1=1 crypto rx=0x7fca0c009dc0 tx=0x7fca0c02be90 comp rx=0 tx=0).stop 2023-08-09T13:53:51.600 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:51.595+0000 7fca1bfff640 1 -- 172.21.15.43:0/738009732 shutdown_connections 2023-08-09T13:53:51.600 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:51.595+0000 7fca1bfff640 1 --2- 172.21.15.43:0/738009732 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fca1c15f7b0 0x7fca1c0b4760 unknown :-1 s=CLOSED pgs=1518 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:53:51.601 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:51.595+0000 7fca1bfff640 1 -- 172.21.15.43:0/738009732 >> 172.21.15.43:0/738009732 conn(0x7fca1c06a2b0 msgr2=0x7fca1c06a690 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-09T13:53:51.601 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:51.595+0000 7fca1bfff640 1 -- 172.21.15.43:0/738009732 shutdown_connections 2023-08-09T13:53:51.601 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:51.595+0000 7fca1bfff640 1 -- 172.21.15.43:0/738009732 wait complete. 2023-08-09T13:53:51.601 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:51.597+0000 7fca1bfff640 1 Processor -- start 2023-08-09T13:53:51.601 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:51.597+0000 7fca1bfff640 1 -- start start 2023-08-09T13:53:51.602 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:51.598+0000 7fca1bfff640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fca1c0b1050 0x7fca1c0b26e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:53:51.603 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:51.598+0000 7fca1bfff640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fca1c0cd6b0 0x7fca1c14de90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:53:51.603 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:51.598+0000 7fca1bfff640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fca1c15f7b0 0x7fca1c14c810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:53:51.603 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:51.598+0000 7fca1bfff640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fca1c14e3d0 con 0x7fca1c15f7b0 2023-08-09T13:53:51.603 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:51.598+0000 7fca1bfff640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fca1c14cd50 con 0x7fca1c0cd6b0 2023-08-09T13:53:51.603 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:51.598+0000 7fca1bfff640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fca1c14e600 con 0x7fca1c0b1050 2023-08-09T13:53:51.603 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:51.599+0000 7fca1b7fe640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fca1c15f7b0 0x7fca1c14c810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:53:51.604 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:51.599+0000 7fca1b7fe640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fca1c15f7b0 0x7fca1c14c810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:56364/0 (socket says 172.21.15.43:56364) 2023-08-09T13:53:51.604 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:51.599+0000 7fca1b7fe640 1 -- 172.21.15.43:0/3808053315 learned_addr learned my addr 172.21.15.43:0/3808053315 (peer_addr_for_me v2:172.21.15.43:0/0) 2023-08-09T13:53:51.604 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:51.599+0000 7fca1a7fc640 1 --2- 172.21.15.43:0/3808053315 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fca1c0cd6b0 0x7fca1c14de90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:53:51.604 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:51.599+0000 7fca1b7fe640 1 -- 172.21.15.43:0/3808053315 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fca1c0b1050 msgr2=0x7fca1c0b26e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:53:51.604 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:51.599+0000 7fca1affd640 1 --2- 172.21.15.43:0/3808053315 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fca1c0b1050 0x7fca1c0b26e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:53:51.604 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:51.599+0000 7fca1b7fe640 1 --2- 172.21.15.43:0/3808053315 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fca1c0b1050 0x7fca1c0b26e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:53:51.604 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:51.599+0000 7fca1b7fe640 1 -- 172.21.15.43:0/3808053315 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fca1c0cd6b0 msgr2=0x7fca1c14de90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:53:51.605 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:51.599+0000 7fca1b7fe640 1 --2- 172.21.15.43:0/3808053315 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fca1c0cd6b0 0x7fca1c14de90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:53:51.605 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:51.599+0000 7fca1b7fe640 1 -- 172.21.15.43:0/3808053315 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fca0c009a70 con 0x7fca1c15f7b0 2023-08-09T13:53:51.605 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:51.599+0000 7fca1a7fc640 1 --2- 172.21.15.43:0/3808053315 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fca1c0cd6b0 0x7fca1c14de90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-08-09T13:53:51.605 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:51.599+0000 7fca1b7fe640 1 --2- 172.21.15.43:0/3808053315 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fca1c15f7b0 0x7fca1c14c810 secure :-1 s=READY pgs=1519 cs=0 l=1 rev1=1 crypto rx=0x7fca1400c410 tx=0x7fca1400c8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-09T13:53:51.605 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:51.600+0000 7fca037fe640 1 -- 172.21.15.43:0/3808053315 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fca14011790 con 0x7fca1c15f7b0 2023-08-09T13:53:51.605 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:51.600+0000 7fca037fe640 1 -- 172.21.15.43:0/3808053315 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fca140118f0 con 0x7fca1c15f7b0 2023-08-09T13:53:51.605 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:51.600+0000 7fca037fe640 1 -- 172.21.15.43:0/3808053315 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fca14011ce0 con 0x7fca1c15f7b0 2023-08-09T13:53:51.605 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:51.600+0000 7fca1bfff640 1 -- 172.21.15.43:0/3808053315 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fca1c0b1c00 con 0x7fca1c15f7b0 2023-08-09T13:53:51.606 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:51.600+0000 7fca1bfff640 1 -- 172.21.15.43:0/3808053315 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fca1c0b21c0 con 0x7fca1c15f7b0 2023-08-09T13:53:51.606 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:51.601+0000 7fca1bfff640 1 -- 172.21.15.43:0/3808053315 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fca1c0b23e0 con 0x7fca1c15f7b0 2023-08-09T13:53:51.606 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:51.602+0000 7fca037fe640 1 -- 172.21.15.43:0/3808053315 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 56) v1 ==== 88443+0+0 (secure 0 0 0) 0x7fca1401c460 con 0x7fca1c15f7b0 2023-08-09T13:53:51.607 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:51.603+0000 7fca037fe640 1 --2- 172.21.15.43:0/3808053315 >> [v2:172.21.15.203:6832/2527059680,v1:172.21.15.203:6833/2527059680] conn(0x7fca0806a170 0x7fca0806c630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:53:51.607 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:51.603+0000 7fca037fe640 1 -- 172.21.15.43:0/3808053315 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(129..129 src has 1..129) v4 ==== 6426+0+0 (secure 0 0 0) 0x7fca14088600 con 0x7fca1c15f7b0 2023-08-09T13:53:51.607 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:51.604+0000 7fca1affd640 1 --2- 172.21.15.43:0/3808053315 >> [v2:172.21.15.203:6832/2527059680,v1:172.21.15.203:6833/2527059680] conn(0x7fca0806a170 0x7fca0806c630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:53:51.607 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:51.605+0000 7fca1affd640 1 --2- 172.21.15.43:0/3808053315 >> [v2:172.21.15.203:6832/2527059680,v1:172.21.15.203:6833/2527059680] conn(0x7fca0806a170 0x7fca0806c630 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7fca0c036000 tx=0x7fca0c030000 comp rx=0 tx=0).ready entity=mgr.6377 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-09T13:53:51.609 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:51.607+0000 7fca037fe640 1 -- 172.21.15.43:0/3808053315 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178066 (secure 0 0 0) 0x7fca140539d0 con 0x7fca1c15f7b0 2023-08-09T13:53:51.842 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:51.840+0000 7fca1bfff640 1 -- 172.21.15.43:0/3808053315 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fca1c0739f0 con 0x7fca1c15f7b0 2023-08-09T13:53:51.844 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:51.841+0000 7fca037fe640 1 -- 172.21.15.43:0/3808053315 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 61 v61) v1 ==== 94+0+4880 (secure 0 0 0) 0x7fca14056ff0 con 0x7fca1c15f7b0 2023-08-09T13:53:51.844 INFO:teuthology.orchestra.run.smithi043.stdout: 2023-08-09T13:53:51.844 INFO:teuthology.orchestra.run.smithi043.stdout:{"epoch":61,"default_fscid":7,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6530,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.203:6835/1659419253","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6834","nonce":1659419253},{"type":"v1","addr":"172.21.15.203:6835","nonce":1659419253}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":56},{"gid":6536,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.203:6837/3964427549","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6836","nonce":3964427549},{"type":"v1","addr":"172.21.15.203:6837","nonce":3964427549}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":56},{"gid":8574,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.43:6833/871812392","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6832","nonce":871812392},{"type":"v1","addr":"172.21.15.43:6833","nonce":871812392}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":55}],"filesystems":[{"mdsmap":{"epoch":61,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-08-09T13:53:47.747881+0000","modified":"2023-08-09T13:53:51.317185+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":8580},"failed":[],"damaged":[],"stopped":[],"info":{"gid_8580":{"gid":8580,"name":"a","rank":0,"incarnation":57,"state":"up:active","state_seq":3,"addr":"172.21.15.43:6835/1794836673","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6834","nonce":1794836673},{"type":"v1","addr":"172.21.15.43:6835","nonce":1794836673}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[17],"metadata_pool":16,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":7}]} 2023-08-09T13:53:51.845 INFO:teuthology.orchestra.run.smithi043.stderr:dumped fsmap epoch 61 2023-08-09T13:53:51.847 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:51.844+0000 7fca1bfff640 1 -- 172.21.15.43:0/3808053315 >> [v2:172.21.15.203:6832/2527059680,v1:172.21.15.203:6833/2527059680] conn(0x7fca0806a170 msgr2=0x7fca0806c630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:53:51.848 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:51.844+0000 7fca1bfff640 1 --2- 172.21.15.43:0/3808053315 >> [v2:172.21.15.203:6832/2527059680,v1:172.21.15.203:6833/2527059680] conn(0x7fca0806a170 0x7fca0806c630 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7fca0c036000 tx=0x7fca0c030000 comp rx=0 tx=0).stop 2023-08-09T13:53:51.848 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:51.844+0000 7fca1bfff640 1 -- 172.21.15.43:0/3808053315 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fca1c15f7b0 msgr2=0x7fca1c14c810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:53:51.848 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:51.844+0000 7fca1bfff640 1 --2- 172.21.15.43:0/3808053315 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fca1c15f7b0 0x7fca1c14c810 secure :-1 s=READY pgs=1519 cs=0 l=1 rev1=1 crypto rx=0x7fca1400c410 tx=0x7fca1400c8d0 comp rx=0 tx=0).stop 2023-08-09T13:53:51.848 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:51.845+0000 7fca1bfff640 1 -- 172.21.15.43:0/3808053315 shutdown_connections 2023-08-09T13:53:51.848 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:51.845+0000 7fca1bfff640 1 --2- 172.21.15.43:0/3808053315 >> [v2:172.21.15.203:6832/2527059680,v1:172.21.15.203:6833/2527059680] conn(0x7fca0806a170 0x7fca0806c630 unknown :-1 s=CLOSED pgs=62 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:53:51.849 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:51.845+0000 7fca1bfff640 1 --2- 172.21.15.43:0/3808053315 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fca1c15f7b0 0x7fca1c14c810 unknown :-1 s=CLOSED pgs=1519 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:53:51.849 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:51.845+0000 7fca1bfff640 1 --2- 172.21.15.43:0/3808053315 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fca1c0cd6b0 0x7fca1c14de90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:53:51.849 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:51.845+0000 7fca1bfff640 1 --2- 172.21.15.43:0/3808053315 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fca1c0b1050 0x7fca1c0b26e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:53:51.849 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:51.845+0000 7fca1bfff640 1 -- 172.21.15.43:0/3808053315 >> 172.21.15.43:0/3808053315 conn(0x7fca1c06a2b0 msgr2=0x7fca1c0d2820 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-09T13:53:51.849 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:51.845+0000 7fca1bfff640 1 -- 172.21.15.43:0/3808053315 shutdown_connections 2023-08-09T13:53:51.849 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:51.845+0000 7fca1bfff640 1 -- 172.21.15.43:0/3808053315 wait complete. 2023-08-09T13:53:51.865 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-08-09T13:53:52.091 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:52.089+0000 7f5a93567640 1 Processor -- start 2023-08-09T13:53:52.092 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:52.089+0000 7f5a93567640 1 -- start start 2023-08-09T13:53:52.093 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:52.090+0000 7f5a93567640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f5a8c15c620 0x7f5a8c0cfe70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:53:52.093 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:52.090+0000 7f5a93567640 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7f5a8c0cf4f0 0x7f5a8c0d07e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:53:52.093 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:52.091+0000 7f5a93567640 1 -- --> v1:172.21.15.203:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5a8c0b2330 con 0x7f5a8c0d0d20 2023-08-09T13:53:52.093 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:52.091+0000 7f5a93567640 1 -- --> v2:172.21.15.43:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5a8c0d2ec0 con 0x7f5a8c15c620 2023-08-09T13:53:52.093 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:52.091+0000 7f5a93567640 1 -- --> v2:172.21.15.203:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5a8c0d0500 con 0x7f5a8c0cf4f0 2023-08-09T13:53:52.094 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:52.091+0000 7f5a92565640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f5a8c15c620 0x7f5a8c0cfe70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:53:52.094 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:52.091+0000 7f5a92565640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f5a8c15c620 0x7f5a8c0cfe70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:56372/0 (socket says 172.21.15.43:56372) 2023-08-09T13:53:52.094 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:52.091+0000 7f5a92565640 1 -- 172.21.15.43:0/3475509897 learned_addr learned my addr 172.21.15.43:0/3475509897 (peer_addr_for_me v2:172.21.15.43:0/0) 2023-08-09T13:53:52.095 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:52.091+0000 7f5a91d64640 1 --2- 172.21.15.43:0/3475509897 >> v2:172.21.15.203:3300/0 conn(0x7f5a8c0cf4f0 0x7f5a8c0d07e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:53:52.095 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:52.091+0000 7f5a92565640 1 -- 172.21.15.43:0/3475509897 >> v2:172.21.15.203:3300/0 conn(0x7f5a8c0cf4f0 msgr2=0x7f5a8c0d07e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-09T13:53:52.095 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:52.091+0000 7f5a92565640 1 --2- 172.21.15.43:0/3475509897 >> v2:172.21.15.203:3300/0 conn(0x7f5a8c0cf4f0 0x7f5a8c0d07e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:53:52.095 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:52.091+0000 7f5a92565640 1 -- 172.21.15.43:0/3475509897 >> v1:172.21.15.203:6789/0 conn(0x7f5a8c0d0d20 legacy=0x7f5a8c0cce40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-09T13:53:52.095 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:52.091+0000 7f5a92565640 1 -- 172.21.15.43:0/3475509897 --> v2:172.21.15.43:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5a8c0b1a00 con 0x7f5a8c15c620 2023-08-09T13:53:52.096 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:52.093+0000 7f5a92565640 1 --2- 172.21.15.43:0/3475509897 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f5a8c15c620 0x7f5a8c0cfe70 secure :-1 s=READY pgs=1520 cs=0 l=1 rev1=1 crypto rx=0x7f5a8c0d2230 tx=0x7f5a8802bea0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5151ea1fec7d6743 server_cookie=0 in_seq=0 out_seq=0 2023-08-09T13:53:52.097 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:52.093+0000 7f5a91563640 1 -- 172.21.15.43:0/3475509897 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5a8802ca40 con 0x7f5a8c15c620 2023-08-09T13:53:52.097 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:52.093+0000 7f5a91563640 1 -- 172.21.15.43:0/3475509897 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5a8802cba0 con 0x7f5a8c15c620 2023-08-09T13:53:52.097 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:52.093+0000 7f5a91563640 1 -- 172.21.15.43:0/3475509897 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5a880343f0 con 0x7f5a8c15c620 2023-08-09T13:53:52.097 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:52.093+0000 7f5a93567640 1 -- 172.21.15.43:0/3475509897 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f5a8c15c620 msgr2=0x7f5a8c0cfe70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:53:52.097 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:52.093+0000 7f5a93567640 1 --2- 172.21.15.43:0/3475509897 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f5a8c15c620 0x7f5a8c0cfe70 secure :-1 s=READY pgs=1520 cs=0 l=1 rev1=1 crypto rx=0x7f5a8c0d2230 tx=0x7f5a8802bea0 comp rx=0 tx=0).stop 2023-08-09T13:53:52.098 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:52.094+0000 7f5a93567640 1 -- 172.21.15.43:0/3475509897 shutdown_connections 2023-08-09T13:53:52.098 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:52.094+0000 7f5a93567640 1 --2- 172.21.15.43:0/3475509897 >> v2:172.21.15.203:3300/0 conn(0x7f5a8c0cf4f0 0x7f5a8c0d07e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:53:52.098 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:52.094+0000 7f5a93567640 1 --2- 172.21.15.43:0/3475509897 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f5a8c15c620 0x7f5a8c0cfe70 unknown :-1 s=CLOSED pgs=1520 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:53:52.098 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:52.094+0000 7f5a93567640 1 -- 172.21.15.43:0/3475509897 >> 172.21.15.43:0/3475509897 conn(0x7f5a8c09b650 msgr2=0x7f5a8c0d22a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-09T13:53:52.098 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:52.094+0000 7f5a93567640 1 -- 172.21.15.43:0/3475509897 shutdown_connections 2023-08-09T13:53:52.098 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:52.094+0000 7f5a93567640 1 -- 172.21.15.43:0/3475509897 wait complete. 2023-08-09T13:53:52.099 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:52.095+0000 7f5a93567640 1 Processor -- start 2023-08-09T13:53:52.099 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:52.096+0000 7f5a93567640 1 -- start start 2023-08-09T13:53:52.099 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:52.096+0000 7f5a93567640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f5a8c0cf4f0 0x7f5a8c0c8240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:53:52.100 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:52.097+0000 7f5a93567640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f5a8c0d0d20 0x7f5a8c15e9b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:53:52.100 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:52.097+0000 7f5a93567640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f5a8c15c620 0x7f5a8c15ca00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:53:52.100 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:52.097+0000 7f5a93567640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5a8c0b4ec0 con 0x7f5a8c0cf4f0 2023-08-09T13:53:52.100 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:52.097+0000 7f5a93567640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5a8c0c8780 con 0x7f5a8c0d0d20 2023-08-09T13:53:52.100 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:52.097+0000 7f5a93567640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5a8c15f390 con 0x7f5a8c15c620 2023-08-09T13:53:52.100 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:52.097+0000 7f5a92565640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f5a8c0cf4f0 0x7f5a8c0c8240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:53:52.100 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:52.097+0000 7f5a92565640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f5a8c0cf4f0 0x7f5a8c0c8240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:56380/0 (socket says 172.21.15.43:56380) 2023-08-09T13:53:52.101 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:52.097+0000 7f5a92565640 1 -- 172.21.15.43:0/4206187157 learned_addr learned my addr 172.21.15.43:0/4206187157 (peer_addr_for_me v2:172.21.15.43:0/0) 2023-08-09T13:53:52.101 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:52.097+0000 7f5a91d64640 1 --2- 172.21.15.43:0/4206187157 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f5a8c0d0d20 0x7f5a8c15e9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:53:52.102 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:52.097+0000 7f5a92565640 1 -- 172.21.15.43:0/4206187157 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f5a8c15c620 msgr2=0x7f5a8c15ca00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:53:52.102 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:52.097+0000 7f5a92565640 1 --2- 172.21.15.43:0/4206187157 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f5a8c15c620 0x7f5a8c15ca00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:53:52.102 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:52.097+0000 7f5a92565640 1 -- 172.21.15.43:0/4206187157 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f5a8c0d0d20 msgr2=0x7f5a8c15e9b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:53:52.102 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:52.097+0000 7f5a92565640 1 --2- 172.21.15.43:0/4206187157 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f5a8c0d0d20 0x7f5a8c15e9b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:53:52.102 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:52.097+0000 7f5a92565640 1 -- 172.21.15.43:0/4206187157 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5a88009b30 con 0x7f5a8c0cf4f0 2023-08-09T13:53:52.102 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:52.097+0000 7f5a91d64640 1 --2- 172.21.15.43:0/4206187157 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f5a8c0d0d20 0x7f5a8c15e9b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-08-09T13:53:52.102 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:52.098+0000 7f5a92565640 1 --2- 172.21.15.43:0/4206187157 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f5a8c0cf4f0 0x7f5a8c0c8240 secure :-1 s=READY pgs=1521 cs=0 l=1 rev1=1 crypto rx=0x7f5a88000f80 tx=0x7f5a88004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-09T13:53:52.103 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:52.098+0000 7f5a82ffd640 1 -- 172.21.15.43:0/4206187157 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5a88009510 con 0x7f5a8c0cf4f0 2023-08-09T13:53:52.103 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:52.098+0000 7f5a82ffd640 1 -- 172.21.15.43:0/4206187157 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5a8803e450 con 0x7f5a8c0cf4f0 2023-08-09T13:53:52.103 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:52.098+0000 7f5a82ffd640 1 -- 172.21.15.43:0/4206187157 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5a8803e800 con 0x7f5a8c0cf4f0 2023-08-09T13:53:52.103 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:52.098+0000 7f5a93567640 1 -- 172.21.15.43:0/4206187157 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5a8c15f670 con 0x7f5a8c0cf4f0 2023-08-09T13:53:52.103 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:52.098+0000 7f5a93567640 1 -- 172.21.15.43:0/4206187157 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5a8c0b53b0 con 0x7f5a8c0cf4f0 2023-08-09T13:53:52.104 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:52.100+0000 7f5a93567640 1 -- 172.21.15.43:0/4206187157 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5a8c0739f0 con 0x7f5a8c0cf4f0 2023-08-09T13:53:52.104 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:52.100+0000 7f5a82ffd640 1 -- 172.21.15.43:0/4206187157 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 56) v1 ==== 88443+0+0 (secure 0 0 0) 0x7f5a8803e960 con 0x7f5a8c0cf4f0 2023-08-09T13:53:52.104 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:52.102+0000 7f5a82ffd640 1 --2- 172.21.15.43:0/4206187157 >> [v2:172.21.15.203:6832/2527059680,v1:172.21.15.203:6833/2527059680] conn(0x7f5a7006a1a0 0x7f5a7006c660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:53:52.105 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:52.102+0000 7f5a82ffd640 1 -- 172.21.15.43:0/4206187157 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(129..129 src has 1..129) v4 ==== 6426+0+0 (secure 0 0 0) 0x7f5a880ab7b0 con 0x7f5a8c0cf4f0 2023-08-09T13:53:52.105 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:52.103+0000 7f5a91d64640 1 --2- 172.21.15.43:0/4206187157 >> [v2:172.21.15.203:6832/2527059680,v1:172.21.15.203:6833/2527059680] conn(0x7f5a7006a1a0 0x7f5a7006c660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:53:52.106 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:52.104+0000 7f5a91d64640 1 --2- 172.21.15.43:0/4206187157 >> [v2:172.21.15.203:6832/2527059680,v1:172.21.15.203:6833/2527059680] conn(0x7f5a7006a1a0 0x7f5a7006c660 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f5a8c0cd9a0 tx=0x7f5a7c008040 comp rx=0 tx=0).ready entity=mgr.6377 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-09T13:53:52.109 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:52.106+0000 7f5a82ffd640 1 -- 172.21.15.43:0/4206187157 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178066 (secure 0 0 0) 0x7f5a88076af0 con 0x7f5a8c0cf4f0 2023-08-09T13:53:52.303 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:52.301+0000 7f5a93567640 1 -- 172.21.15.43:0/4206187157 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f5a8c15f800 con 0x7f5a8c0cf4f0 2023-08-09T13:53:52.305 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:52.302+0000 7f5a82ffd640 1 -- 172.21.15.43:0/4206187157 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v129) v1 ==== 74+0+18514 (secure 0 0 0) 0x7f5a8807a110 con 0x7f5a8c0cf4f0 2023-08-09T13:53:52.305 INFO:teuthology.orchestra.run.smithi043.stdout: 2023-08-09T13:53:52.306 INFO:teuthology.orchestra.run.smithi043.stdout:{"epoch":129,"fsid":"daac87ee-1337-4250-a955-aea4e408a39b","created":"2023-08-09T13:33:41.701198+0000","modified":"2023-08-09T13:53:49.270221+0000","last_up_change":"2023-08-09T13:33:53.783453+0000","last_in_change":"2023-08-09T13:33:45.148904+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":17,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":"rbd","create_time":"2023-08-09T13:33:54.365137+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":".mgr","create_time":"2023-08-09T13:33:54.831803+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":16,"pool_name":"cephfs_metadata","create_time":"2023-08-09T13:53:46.124616+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"128","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":17,"pool_name":"cephfs_data","create_time":"2023-08-09T13:53:46.762733+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"129","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"cbd49ecd-e896-4fbd-864e-5c37a829b165","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":127,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6800","nonce":502667335},{"type":"v1","addr":"172.21.15.43:6801","nonce":502667335}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6802","nonce":502667335},{"type":"v1","addr":"172.21.15.43:6803","nonce":502667335}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6806","nonce":502667335},{"type":"v1","addr":"172.21.15.43:6807","nonce":502667335}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6804","nonce":502667335},{"type":"v1","addr":"172.21.15.43:6805","nonce":502667335}]},"public_addr":"172.21.15.43:6801/502667335","cluster_addr":"172.21.15.43:6803/502667335","heartbeat_back_addr":"172.21.15.43:6807/502667335","heartbeat_front_addr":"172.21.15.43:6805/502667335","state":["exists","up"]},{"osd":1,"uuid":"75c7e0fe-1ec7-4fe8-a2f2-3661a65b8500","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":127,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6808","nonce":3289134610},{"type":"v1","addr":"172.21.15.43:6809","nonce":3289134610}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6810","nonce":3289134610},{"type":"v1","addr":"172.21.15.43:6812","nonce":3289134610}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6821","nonce":3289134610},{"type":"v1","addr":"172.21.15.43:6824","nonce":3289134610}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6815","nonce":3289134610},{"type":"v1","addr":"172.21.15.43:6818","nonce":3289134610}]},"public_addr":"172.21.15.43:6809/3289134610","cluster_addr":"172.21.15.43:6812/3289134610","heartbeat_back_addr":"172.21.15.43:6824/3289134610","heartbeat_front_addr":"172.21.15.43:6818/3289134610","state":["exists","up"]},{"osd":2,"uuid":"bb5c41e9-7f5f-4125-9b01-3bb8bfc2ea83","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":127,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6811","nonce":4079709736},{"type":"v1","addr":"172.21.15.43:6814","nonce":4079709736}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6817","nonce":4079709736},{"type":"v1","addr":"172.21.15.43:6820","nonce":4079709736}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6829","nonce":4079709736},{"type":"v1","addr":"172.21.15.43:6831","nonce":4079709736}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6823","nonce":4079709736},{"type":"v1","addr":"172.21.15.43:6826","nonce":4079709736}]},"public_addr":"172.21.15.43:6814/4079709736","cluster_addr":"172.21.15.43:6820/4079709736","heartbeat_back_addr":"172.21.15.43:6831/4079709736","heartbeat_front_addr":"172.21.15.43:6826/4079709736","state":["exists","up"]},{"osd":3,"uuid":"bfe4c8b4-694a-465c-87e6-2b2aab852613","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":127,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6813","nonce":1709538759},{"type":"v1","addr":"172.21.15.43:6816","nonce":1709538759}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6819","nonce":1709538759},{"type":"v1","addr":"172.21.15.43:6822","nonce":1709538759}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6828","nonce":1709538759},{"type":"v1","addr":"172.21.15.43:6830","nonce":1709538759}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6825","nonce":1709538759},{"type":"v1","addr":"172.21.15.43:6827","nonce":1709538759}]},"public_addr":"172.21.15.43:6816/1709538759","cluster_addr":"172.21.15.43:6822/1709538759","heartbeat_back_addr":"172.21.15.43:6830/1709538759","heartbeat_front_addr":"172.21.15.43:6827/1709538759","state":["exists","up"]},{"osd":4,"uuid":"fa2e8a77-aba1-4756-853e-0d174a5bdeba","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":127,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6800","nonce":4196866481},{"type":"v1","addr":"172.21.15.203:6801","nonce":4196866481}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6802","nonce":4196866481},{"type":"v1","addr":"172.21.15.203:6803","nonce":4196866481}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6806","nonce":4196866481},{"type":"v1","addr":"172.21.15.203:6807","nonce":4196866481}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6804","nonce":4196866481},{"type":"v1","addr":"172.21.15.203:6805","nonce":4196866481}]},"public_addr":"172.21.15.203:6801/4196866481","cluster_addr":"172.21.15.203:6803/4196866481","heartbeat_back_addr":"172.21.15.203:6807/4196866481","heartbeat_front_addr":"172.21.15.203:6805/4196866481","state":["exists","up"]},{"osd":5,"uuid":"81811e4b-4d24-4f51-a82c-f1d700a2a00d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":127,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6816","nonce":3059290081},{"type":"v1","addr":"172.21.15.203:6817","nonce":3059290081}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6818","nonce":3059290081},{"type":"v1","addr":"172.21.15.203:6819","nonce":3059290081}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6822","nonce":3059290081},{"type":"v1","addr":"172.21.15.203:6823","nonce":3059290081}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6820","nonce":3059290081},{"type":"v1","addr":"172.21.15.203:6821","nonce":3059290081}]},"public_addr":"172.21.15.203:6817/3059290081","cluster_addr":"172.21.15.203:6819/3059290081","heartbeat_back_addr":"172.21.15.203:6823/3059290081","heartbeat_front_addr":"172.21.15.203:6821/3059290081","state":["exists","up"]},{"osd":6,"uuid":"307bac4d-6a5b-443d-af75-56d695d7674f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":127,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6824","nonce":512155569},{"type":"v1","addr":"172.21.15.203:6825","nonce":512155569}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6826","nonce":512155569},{"type":"v1","addr":"172.21.15.203:6827","nonce":512155569}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6830","nonce":512155569},{"type":"v1","addr":"172.21.15.203:6831","nonce":512155569}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6828","nonce":512155569},{"type":"v1","addr":"172.21.15.203:6829","nonce":512155569}]},"public_addr":"172.21.15.203:6825/512155569","cluster_addr":"172.21.15.203:6827/512155569","heartbeat_back_addr":"172.21.15.203:6831/512155569","heartbeat_front_addr":"172.21.15.203:6829/512155569","state":["exists","up"]},{"osd":7,"uuid":"a95dc7ca-9f1b-4295-af70-68a22a2d7d5e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":127,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6808","nonce":3331450438},{"type":"v1","addr":"172.21.15.203:6809","nonce":3331450438}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6810","nonce":3331450438},{"type":"v1","addr":"172.21.15.203:6811","nonce":3331450438}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6814","nonce":3331450438},{"type":"v1","addr":"172.21.15.203:6815","nonce":3331450438}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6812","nonce":3331450438},{"type":"v1","addr":"172.21.15.203:6813","nonce":3331450438}]},"public_addr":"172.21.15.203:6809/3331450438","cluster_addr":"172.21.15.203:6811/3331450438","heartbeat_back_addr":"172.21.15.203:6815/3331450438","heartbeat_front_addr":"172.21.15.203:6813/3331450438","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-09T13:33:50.547652+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-09T13:33:50.589469+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-09T13:33:50.561724+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-09T13:33:50.599650+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-09T13:33:50.411855+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-09T13:33:50.219681+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-09T13:33:50.504664+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-09T13:33:50.441248+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":7,"snaps":[{"begin":104,"length":1},{"begin":221,"length":1}]}],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-08-09T13:53:52.307 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:52.305+0000 7f5a93567640 1 -- 172.21.15.43:0/4206187157 >> [v2:172.21.15.203:6832/2527059680,v1:172.21.15.203:6833/2527059680] conn(0x7f5a7006a1a0 msgr2=0x7f5a7006c660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:53:52.307 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:52.305+0000 7f5a93567640 1 --2- 172.21.15.43:0/4206187157 >> [v2:172.21.15.203:6832/2527059680,v1:172.21.15.203:6833/2527059680] conn(0x7f5a7006a1a0 0x7f5a7006c660 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f5a8c0cd9a0 tx=0x7f5a7c008040 comp rx=0 tx=0).stop 2023-08-09T13:53:52.308 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:52.305+0000 7f5a93567640 1 -- 172.21.15.43:0/4206187157 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f5a8c0cf4f0 msgr2=0x7f5a8c0c8240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:53:52.308 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:52.305+0000 7f5a93567640 1 --2- 172.21.15.43:0/4206187157 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f5a8c0cf4f0 0x7f5a8c0c8240 secure :-1 s=READY pgs=1521 cs=0 l=1 rev1=1 crypto rx=0x7f5a88000f80 tx=0x7f5a88004a40 comp rx=0 tx=0).stop 2023-08-09T13:53:52.308 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:52.305+0000 7f5a93567640 1 -- 172.21.15.43:0/4206187157 shutdown_connections 2023-08-09T13:53:52.308 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:52.305+0000 7f5a93567640 1 --2- 172.21.15.43:0/4206187157 >> [v2:172.21.15.203:6832/2527059680,v1:172.21.15.203:6833/2527059680] conn(0x7f5a7006a1a0 0x7f5a7006c660 unknown :-1 s=CLOSED pgs=63 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:53:52.308 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:52.305+0000 7f5a93567640 1 --2- 172.21.15.43:0/4206187157 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f5a8c15c620 0x7f5a8c15ca00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:53:52.308 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:52.305+0000 7f5a93567640 1 --2- 172.21.15.43:0/4206187157 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f5a8c0d0d20 0x7f5a8c15e9b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:53:52.309 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:52.305+0000 7f5a93567640 1 --2- 172.21.15.43:0/4206187157 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f5a8c0cf4f0 0x7f5a8c0c8240 unknown :-1 s=CLOSED pgs=1521 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:53:52.309 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:52.305+0000 7f5a93567640 1 -- 172.21.15.43:0/4206187157 >> 172.21.15.43:0/4206187157 conn(0x7f5a8c09b650 msgr2=0x7f5a8c0cce00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-09T13:53:52.309 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:52.305+0000 7f5a93567640 1 -- 172.21.15.43:0/4206187157 shutdown_connections 2023-08-09T13:53:52.309 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:52.305+0000 7f5a93567640 1 -- 172.21.15.43:0/4206187157 wait complete. 2023-08-09T13:53:52.327 INFO:tasks.cephfs.filesystem.ceph_manager:waiting for clean 2023-08-09T13:53:52.327 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-08-09T13:53:52.549 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:52.547+0000 7f43fe66c640 1 Processor -- start 2023-08-09T13:53:52.550 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:52.547+0000 7f43fe66c640 1 -- start start 2023-08-09T13:53:52.551 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:52.548+0000 7f43fe66c640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f43f80ce080 0x7f43f80cafa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:53:52.551 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:52.548+0000 7f43fe66c640 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7f43f80cb4e0 0x7f43f80cb990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:53:52.551 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:52.548+0000 7f43fe66c640 1 -- --> v1:172.21.15.203:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f43f80cd8a0 con 0x7f43f80ce720 2023-08-09T13:53:52.551 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:52.548+0000 7f43fe66c640 1 -- --> v2:172.21.15.203:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f43f80cbed0 con 0x7f43f80cb4e0 2023-08-09T13:53:52.551 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:52.548+0000 7f43fe66c640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f43f80cf350 con 0x7f43f80ce080 2023-08-09T13:53:52.552 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:52.549+0000 7f43f77fe640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f43f80ce080 0x7f43f80cafa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:53:52.552 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:52.549+0000 7f43fcc09640 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7f43f80cb4e0 0x7f43f80cb990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:53:52.552 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:52.549+0000 7f43f77fe640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f43f80ce080 0x7f43f80cafa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.203:3301/0 says I am v2:172.21.15.43:35800/0 (socket says 172.21.15.43:35800) 2023-08-09T13:53:52.552 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:52.549+0000 7f43f77fe640 1 -- 172.21.15.43:0/726842217 learned_addr learned my addr 172.21.15.43:0/726842217 (peer_addr_for_me v2:172.21.15.43:0/0) 2023-08-09T13:53:52.553 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:52.549+0000 7f43f7fff640 1 --1- 172.21.15.43:0/726842217 >> v1:172.21.15.203:6789/0 conn(0x7f43f80ce720 0x7f43f80ceb00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.203:6789/0 says I am v1:172.21.15.43:55380/0 (socket says 172.21.15.43:55380) 2023-08-09T13:53:52.553 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:52.550+0000 7f43f6ffd640 1 -- 172.21.15.43:0/726842217 <== mon.1 v1:172.21.15.203:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1868789694 0 0) 0x7f43f80cd8a0 con 0x7f43f80ce720 2023-08-09T13:53:52.553 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:52.550+0000 7f43f77fe640 1 -- 172.21.15.43:0/726842217 >> v2:172.21.15.203:3300/0 conn(0x7f43f80cb4e0 msgr2=0x7f43f80cb990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-09T13:53:52.553 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:52.550+0000 7f43f77fe640 1 --2- 172.21.15.43:0/726842217 >> v2:172.21.15.203:3300/0 conn(0x7f43f80cb4e0 0x7f43f80cb990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:53:52.554 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:52.550+0000 7f43f77fe640 1 -- 172.21.15.43:0/726842217 >> v1:172.21.15.203:6789/0 conn(0x7f43f80ce720 legacy=0x7f43f80ceb00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:53:52.554 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:52.550+0000 7f43f77fe640 1 -- 172.21.15.43:0/726842217 --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f43f8071a70 con 0x7f43f80ce080 2023-08-09T13:53:52.554 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:52.550+0000 7f43fcc09640 1 --2- 172.21.15.43:0/726842217 >> v2:172.21.15.203:3300/0 conn(0x7f43f80cb4e0 0x7f43f80cb990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-08-09T13:53:52.554 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:52.551+0000 7f43f77fe640 1 --2- 172.21.15.43:0/726842217 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f43f80ce080 0x7f43f80cafa0 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7f43e0009e50 tx=0x7f43e002bee0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=5413ed5b00398a25 server_cookie=0 in_seq=0 out_seq=0 2023-08-09T13:53:52.555 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:52.552+0000 7f43f6ffd640 1 -- 172.21.15.43:0/726842217 <== mon.2 v2:172.21.15.203:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f43e002ca80 con 0x7f43f80ce080 2023-08-09T13:53:52.555 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:52.552+0000 7f43f6ffd640 1 -- 172.21.15.43:0/726842217 <== mon.2 v2:172.21.15.203:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f43e002cbe0 con 0x7f43f80ce080 2023-08-09T13:53:52.555 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:52.552+0000 7f43f6ffd640 1 -- 172.21.15.43:0/726842217 <== mon.2 v2:172.21.15.203:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f43e00343f0 con 0x7f43f80ce080 2023-08-09T13:53:52.556 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:52.553+0000 7f43fe66c640 1 -- 172.21.15.43:0/726842217 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f43f80ce080 msgr2=0x7f43f80cafa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:53:52.556 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:52.553+0000 7f43fe66c640 1 --2- 172.21.15.43:0/726842217 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f43f80ce080 0x7f43f80cafa0 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7f43e0009e50 tx=0x7f43e002bee0 comp rx=0 tx=0).stop 2023-08-09T13:53:52.556 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:52.553+0000 7f43fe66c640 1 -- 172.21.15.43:0/726842217 shutdown_connections 2023-08-09T13:53:52.556 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:52.553+0000 7f43fe66c640 1 --2- 172.21.15.43:0/726842217 >> v2:172.21.15.203:3300/0 conn(0x7f43f80cb4e0 0x7f43f80cb990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:53:52.556 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:52.553+0000 7f43fe66c640 1 --2- 172.21.15.43:0/726842217 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f43f80ce080 0x7f43f80cafa0 unknown :-1 s=CLOSED pgs=205 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:53:52.557 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:52.553+0000 7f43fe66c640 1 -- 172.21.15.43:0/726842217 >> 172.21.15.43:0/726842217 conn(0x7f43f80ccd80 msgr2=0x7f43f80cd190 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-09T13:53:52.557 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:52.553+0000 7f43fe66c640 1 -- 172.21.15.43:0/726842217 shutdown_connections 2023-08-09T13:53:52.557 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:52.553+0000 7f43fe66c640 1 -- 172.21.15.43:0/726842217 wait complete. 2023-08-09T13:53:52.558 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:52.554+0000 7f43fe66c640 1 Processor -- start 2023-08-09T13:53:52.558 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:52.555+0000 7f43fe66c640 1 -- start start 2023-08-09T13:53:52.558 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:52.555+0000 7f43fe66c640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f43f80cb4e0 0x7f43f815d1d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:53:52.558 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:52.555+0000 7f43fe66c640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f43f80ce080 0x7f43f806acc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:53:52.559 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:52.556+0000 7f43fe66c640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f43f80ce720 0x7f43f80c48a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:53:52.559 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:52.556+0000 7f43fe66c640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f43f806c1c0 con 0x7f43f80ce720 2023-08-09T13:53:52.559 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:52.556+0000 7f43fe66c640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f43f806b5b0 con 0x7f43f80cb4e0 2023-08-09T13:53:52.559 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:52.556+0000 7f43fe66c640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f43f806b890 con 0x7f43f80ce080 2023-08-09T13:53:52.562 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:52.556+0000 7f43fcc09640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f43f80ce720 0x7f43f80c48a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:53:52.562 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:52.556+0000 7f43fcc09640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f43f80ce720 0x7f43f80c48a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:56386/0 (socket says 172.21.15.43:56386) 2023-08-09T13:53:52.563 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:52.556+0000 7f43fcc09640 1 -- 172.21.15.43:0/1585633359 learned_addr learned my addr 172.21.15.43:0/1585633359 (peer_addr_for_me v2:172.21.15.43:0/0) 2023-08-09T13:53:52.563 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:52.556+0000 7f43f77fe640 1 --2- 172.21.15.43:0/1585633359 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f43f80ce080 0x7f43f806acc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:53:52.563 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:52.556+0000 7f43f7fff640 1 --2- 172.21.15.43:0/1585633359 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f43f80cb4e0 0x7f43f815d1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:53:52.564 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:52.556+0000 7f43fcc09640 1 -- 172.21.15.43:0/1585633359 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f43f80ce080 msgr2=0x7f43f806acc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:53:52.564 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:52.556+0000 7f43fcc09640 1 --2- 172.21.15.43:0/1585633359 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f43f80ce080 0x7f43f806acc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:53:52.564 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:52.556+0000 7f43fcc09640 1 -- 172.21.15.43:0/1585633359 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f43f80cb4e0 msgr2=0x7f43f815d1d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:53:52.564 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:52.556+0000 7f43fcc09640 1 --2- 172.21.15.43:0/1585633359 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f43f80cb4e0 0x7f43f815d1d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:53:52.565 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:52.556+0000 7f43fcc09640 1 -- 172.21.15.43:0/1585633359 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f43f0009400 con 0x7f43f80ce720 2023-08-09T13:53:52.565 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:52.557+0000 7f43fcc09640 1 --2- 172.21.15.43:0/1585633359 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f43f80ce720 0x7f43f80c48a0 secure :-1 s=READY pgs=1522 cs=0 l=1 rev1=1 crypto rx=0x7f43f000e880 tx=0x7f43f000ed40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-09T13:53:52.565 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:52.557+0000 7f43f4ff9640 1 -- 172.21.15.43:0/1585633359 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f43f0009660 con 0x7f43f80ce720 2023-08-09T13:53:52.566 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:52.557+0000 7f43f4ff9640 1 -- 172.21.15.43:0/1585633359 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f43f0004020 con 0x7f43f80ce720 2023-08-09T13:53:52.566 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:52.557+0000 7f43f4ff9640 1 -- 172.21.15.43:0/1585633359 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f43f00043b0 con 0x7f43f80ce720 2023-08-09T13:53:52.566 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:52.557+0000 7f43fe66c640 1 -- 172.21.15.43:0/1585633359 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f43e0009b30 con 0x7f43f80ce720 2023-08-09T13:53:52.566 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:52.557+0000 7f43fe66c640 1 -- 172.21.15.43:0/1585633359 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f43f806be50 con 0x7f43f80ce720 2023-08-09T13:53:52.567 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:52.558+0000 7f43fe66c640 1 -- 172.21.15.43:0/1585633359 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f43f806c330 con 0x7f43f80ce720 2023-08-09T13:53:52.567 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:52.559+0000 7f43f4ff9640 1 -- 172.21.15.43:0/1585633359 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 56) v1 ==== 88443+0+0 (secure 0 0 0) 0x7f43f000c940 con 0x7f43f80ce720 2023-08-09T13:53:52.568 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:52.560+0000 7f43f4ff9640 1 --2- 172.21.15.43:0/1585633359 >> [v2:172.21.15.203:6832/2527059680,v1:172.21.15.203:6833/2527059680] conn(0x7f43d806a1a0 0x7f43d806c660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:53:52.568 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:52.560+0000 7f43f4ff9640 1 -- 172.21.15.43:0/1585633359 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(129..129 src has 1..129) v4 ==== 6426+0+0 (secure 0 0 0) 0x7f43f0088ac0 con 0x7f43f80ce720 2023-08-09T13:53:52.569 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:52.561+0000 7f43f7fff640 1 --2- 172.21.15.43:0/1585633359 >> [v2:172.21.15.203:6832/2527059680,v1:172.21.15.203:6833/2527059680] conn(0x7f43d806a1a0 0x7f43d806c660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:53:52.569 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:52.562+0000 7f43f7fff640 1 --2- 172.21.15.43:0/1585633359 >> [v2:172.21.15.203:6832/2527059680,v1:172.21.15.203:6833/2527059680] conn(0x7f43d806a1a0 0x7f43d806c660 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f43e8000fb0 tx=0x7f43e8007cd0 comp rx=0 tx=0).ready entity=mgr.6377 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-09T13:53:52.569 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:52.565+0000 7f43f4ff9640 1 -- 172.21.15.43:0/1585633359 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178066 (secure 0 0 0) 0x7f43f0053f20 con 0x7f43f80ce720 2023-08-09T13:53:52.757 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:52.755+0000 7f43fe66c640 1 -- 172.21.15.43:0/1585633359 --> [v2:172.21.15.203:6832/2527059680,v1:172.21.15.203:6833/2527059680] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f43f80cdaf0 con 0x7f43d806a1a0 2023-08-09T13:53:52.789 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:52.786+0000 7f43f4ff9640 1 -- 172.21.15.43:0/1585633359 <== mgr.6377 v2:172.21.15.203:6832/2527059680 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+309767 (secure 0 0 0) 0x7f43e8005ae0 con 0x7f43d806a1a0 2023-08-09T13:53:52.794 INFO:teuthology.orchestra.run.smithi043.stdout: 2023-08-09T13:53:52.796 INFO:teuthology.orchestra.run.smithi043.stderr:dumped all 2023-08-09T13:53:52.798 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:52.791+0000 7f43fe66c640 1 -- 172.21.15.43:0/1585633359 >> [v2:172.21.15.203:6832/2527059680,v1:172.21.15.203:6833/2527059680] conn(0x7f43d806a1a0 msgr2=0x7f43d806c660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:53:52.798 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:52.791+0000 7f43fe66c640 1 --2- 172.21.15.43:0/1585633359 >> [v2:172.21.15.203:6832/2527059680,v1:172.21.15.203:6833/2527059680] conn(0x7f43d806a1a0 0x7f43d806c660 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f43e8000fb0 tx=0x7f43e8007cd0 comp rx=0 tx=0).stop 2023-08-09T13:53:52.798 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:52.791+0000 7f43fe66c640 1 -- 172.21.15.43:0/1585633359 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f43f80ce720 msgr2=0x7f43f80c48a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:53:52.799 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:52.791+0000 7f43fe66c640 1 --2- 172.21.15.43:0/1585633359 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f43f80ce720 0x7f43f80c48a0 secure :-1 s=READY pgs=1522 cs=0 l=1 rev1=1 crypto rx=0x7f43f000e880 tx=0x7f43f000ed40 comp rx=0 tx=0).stop 2023-08-09T13:53:52.799 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:52.791+0000 7f43fe66c640 1 -- 172.21.15.43:0/1585633359 shutdown_connections 2023-08-09T13:53:52.799 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:52.791+0000 7f43fe66c640 1 --2- 172.21.15.43:0/1585633359 >> [v2:172.21.15.203:6832/2527059680,v1:172.21.15.203:6833/2527059680] conn(0x7f43d806a1a0 0x7f43d806c660 unknown :-1 s=CLOSED pgs=64 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:53:52.799 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:52.791+0000 7f43fe66c640 1 --2- 172.21.15.43:0/1585633359 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f43f80ce720 0x7f43f80c48a0 unknown :-1 s=CLOSED pgs=1522 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:53:52.800 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:52.791+0000 7f43fe66c640 1 --2- 172.21.15.43:0/1585633359 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f43f80ce080 0x7f43f806acc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:53:52.800 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:52.791+0000 7f43fe66c640 1 --2- 172.21.15.43:0/1585633359 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f43f80cb4e0 0x7f43f815d1d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:53:52.800 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:52.791+0000 7f43fe66c640 1 -- 172.21.15.43:0/1585633359 >> 172.21.15.43:0/1585633359 conn(0x7f43f80ccd80 msgr2=0x7f43f815da90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-09T13:53:52.800 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:52.792+0000 7f43fe66c640 1 -- 172.21.15.43:0/1585633359 shutdown_connections 2023-08-09T13:53:52.801 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:52.792+0000 7f43fe66c640 1 -- 172.21.15.43:0/1585633359 wait complete. 2023-08-09T13:53:52.814 INFO:teuthology.orchestra.run.smithi043.stdout:{"pg_ready":true,"pg_map":{"version":26,"stamp":"2023-08-09T13:53:52.380049+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":23,"num_object_clones":0,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":0,"num_read":907,"num_read_kb":1830,"num_write":367,"num_write_kb":2430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":246,"ondisk_log_size":246,"up":210,"acting":210,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":132,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":331416,"kb_used_data":13600,"kb_used_omap":13,"kb_used_meta":317682,"kb_avail":754643304,"statfs":{"total":773094113280,"available":772754743296,"internally_reserved":0,"allocated":13926400,"data_stored":7946506,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":14248,"internal_metadata":325306456},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":37,"apply_latency_ms":37,"commit_latency_ns":37000000,"apply_latency_ns":37000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.002038"},"pg_stats":[{"pgid":"17.3f","version":"0'0","reported_seq":9,"reported_epoch":128,"state":"active+clean","last_fresh":"2023-08-09T13:53:48.321225+0000","last_change":"2023-08-09T13:53:48.317385+0000","last_active":"2023-08-09T13:53:48.321225+0000","last_peered":"2023-08-09T13:53:48.321225+0000","last_clean":"2023-08-09T13:53:48.321225+0000","last_became_active":"2023-08-09T13:53:48.317083+0000","last_became_peered":"2023-08-09T13:53:48.317083+0000","last_unstale":"2023-08-09T13:53:48.321225+0000","last_undegraded":"2023-08-09T13:53:48.321225+0000","last_fullsized":"2023-08-09T13:53:48.321225+0000","mapping_epoch":127,"log_start":"0'0","ondisk_log_start":"0'0","created":127,"last_epoch_clean":128,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:53:47.231906+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:53:47.231906+0000","last_clean_scrub_stamp":"2023-08-09T13:53:47.231906+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T15:13:30.362935+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.3e","version":"0'0","reported_seq":9,"reported_epoch":128,"state":"active+clean","last_fresh":"2023-08-09T13:53:48.311117+0000","last_change":"2023-08-09T13:53:48.308647+0000","last_active":"2023-08-09T13:53:48.311117+0000","last_peered":"2023-08-09T13:53:48.311117+0000","last_clean":"2023-08-09T13:53:48.311117+0000","last_became_active":"2023-08-09T13:53:48.308358+0000","last_became_peered":"2023-08-09T13:53:48.308358+0000","last_unstale":"2023-08-09T13:53:48.311117+0000","last_undegraded":"2023-08-09T13:53:48.311117+0000","last_fullsized":"2023-08-09T13:53:48.311117+0000","mapping_epoch":127,"log_start":"0'0","ondisk_log_start":"0'0","created":127,"last_epoch_clean":128,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:53:47.231906+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:53:47.231906+0000","last_clean_scrub_stamp":"2023-08-09T13:53:47.231906+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T20:17:37.885150+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.3d","version":"0'0","reported_seq":9,"reported_epoch":128,"state":"active+clean","last_fresh":"2023-08-09T13:53:48.325295+0000","last_change":"2023-08-09T13:53:48.318895+0000","last_active":"2023-08-09T13:53:48.325295+0000","last_peered":"2023-08-09T13:53:48.325295+0000","last_clean":"2023-08-09T13:53:48.325295+0000","last_became_active":"2023-08-09T13:53:48.318640+0000","last_became_peered":"2023-08-09T13:53:48.318640+0000","last_unstale":"2023-08-09T13:53:48.325295+0000","last_undegraded":"2023-08-09T13:53:48.325295+0000","last_fullsized":"2023-08-09T13:53:48.325295+0000","mapping_epoch":127,"log_start":"0'0","ondisk_log_start":"0'0","created":127,"last_epoch_clean":128,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:53:47.231906+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:53:47.231906+0000","last_clean_scrub_stamp":"2023-08-09T13:53:47.231906+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T23:29:56.162342+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"17.3c","version":"0'0","reported_seq":9,"reported_epoch":128,"state":"active+clean","last_fresh":"2023-08-09T13:53:48.318073+0000","last_change":"2023-08-09T13:53:48.314244+0000","last_active":"2023-08-09T13:53:48.318073+0000","last_peered":"2023-08-09T13:53:48.318073+0000","last_clean":"2023-08-09T13:53:48.318073+0000","last_became_active":"2023-08-09T13:53:48.313966+0000","last_became_peered":"2023-08-09T13:53:48.313966+0000","last_unstale":"2023-08-09T13:53:48.318073+0000","last_undegraded":"2023-08-09T13:53:48.318073+0000","last_fullsized":"2023-08-09T13:53:48.318073+0000","mapping_epoch":127,"log_start":"0'0","ondisk_log_start":"0'0","created":127,"last_epoch_clean":128,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:53:47.231906+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:53:47.231906+0000","last_clean_scrub_stamp":"2023-08-09T13:53:47.231906+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T19:03:41.276282+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.3b","version":"0'0","reported_seq":11,"reported_epoch":129,"state":"active+clean","last_fresh":"2023-08-09T13:53:49.282065+0000","last_change":"2023-08-09T13:53:48.286401+0000","last_active":"2023-08-09T13:53:49.282065+0000","last_peered":"2023-08-09T13:53:49.282065+0000","last_clean":"2023-08-09T13:53:49.282065+0000","last_became_active":"2023-08-09T13:53:48.286106+0000","last_became_peered":"2023-08-09T13:53:48.286106+0000","last_unstale":"2023-08-09T13:53:49.282065+0000","last_undegraded":"2023-08-09T13:53:49.282065+0000","last_fullsized":"2023-08-09T13:53:49.282065+0000","mapping_epoch":127,"log_start":"0'0","ondisk_log_start":"0'0","created":127,"last_epoch_clean":128,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:53:47.231906+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:53:47.231906+0000","last_clean_scrub_stamp":"2023-08-09T13:53:47.231906+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-11T00:07:14.541612+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"17.3a","version":"0'0","reported_seq":11,"reported_epoch":129,"state":"active+clean","last_fresh":"2023-08-09T13:53:49.277155+0000","last_change":"2023-08-09T13:53:48.287482+0000","last_active":"2023-08-09T13:53:49.277155+0000","last_peered":"2023-08-09T13:53:49.277155+0000","last_clean":"2023-08-09T13:53:49.277155+0000","last_became_active":"2023-08-09T13:53:48.287113+0000","last_became_peered":"2023-08-09T13:53:48.287113+0000","last_unstale":"2023-08-09T13:53:49.277155+0000","last_undegraded":"2023-08-09T13:53:49.277155+0000","last_fullsized":"2023-08-09T13:53:49.277155+0000","mapping_epoch":127,"log_start":"0'0","ondisk_log_start":"0'0","created":127,"last_epoch_clean":128,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:53:47.231906+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:53:47.231906+0000","last_clean_scrub_stamp":"2023-08-09T13:53:47.231906+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T15:27:38.211706+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"17.39","version":"0'0","reported_seq":9,"reported_epoch":128,"state":"active+clean","last_fresh":"2023-08-09T13:53:48.320308+0000","last_change":"2023-08-09T13:53:48.317266+0000","last_active":"2023-08-09T13:53:48.320308+0000","last_peered":"2023-08-09T13:53:48.320308+0000","last_clean":"2023-08-09T13:53:48.320308+0000","last_became_active":"2023-08-09T13:53:48.316563+0000","last_became_peered":"2023-08-09T13:53:48.316563+0000","last_unstale":"2023-08-09T13:53:48.320308+0000","last_undegraded":"2023-08-09T13:53:48.320308+0000","last_fullsized":"2023-08-09T13:53:48.320308+0000","mapping_epoch":127,"log_start":"0'0","ondisk_log_start":"0'0","created":127,"last_epoch_clean":128,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:53:47.231906+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:53:47.231906+0000","last_clean_scrub_stamp":"2023-08-09T13:53:47.231906+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T20:59:22.754345+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"17.38","version":"0'0","reported_seq":9,"reported_epoch":128,"state":"active+clean","last_fresh":"2023-08-09T13:53:48.309393+0000","last_change":"2023-08-09T13:53:48.306331+0000","last_active":"2023-08-09T13:53:48.309393+0000","last_peered":"2023-08-09T13:53:48.309393+0000","last_clean":"2023-08-09T13:53:48.309393+0000","last_became_active":"2023-08-09T13:53:48.305997+0000","last_became_peered":"2023-08-09T13:53:48.305997+0000","last_unstale":"2023-08-09T13:53:48.309393+0000","last_undegraded":"2023-08-09T13:53:48.309393+0000","last_fullsized":"2023-08-09T13:53:48.309393+0000","mapping_epoch":127,"log_start":"0'0","ondisk_log_start":"0'0","created":127,"last_epoch_clean":128,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:53:47.231906+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:53:47.231906+0000","last_clean_scrub_stamp":"2023-08-09T13:53:47.231906+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T17:52:01.233963+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"17.37","version":"0'0","reported_seq":9,"reported_epoch":128,"state":"active+clean","last_fresh":"2023-08-09T13:53:48.315077+0000","last_change":"2023-08-09T13:53:48.310293+0000","last_active":"2023-08-09T13:53:48.315077+0000","last_peered":"2023-08-09T13:53:48.315077+0000","last_clean":"2023-08-09T13:53:48.315077+0000","last_became_active":"2023-08-09T13:53:48.310037+0000","last_became_peered":"2023-08-09T13:53:48.310037+0000","last_unstale":"2023-08-09T13:53:48.315077+0000","last_undegraded":"2023-08-09T13:53:48.315077+0000","last_fullsized":"2023-08-09T13:53:48.315077+0000","mapping_epoch":127,"log_start":"0'0","ondisk_log_start":"0'0","created":127,"last_epoch_clean":128,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:53:47.231906+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:53:47.231906+0000","last_clean_scrub_stamp":"2023-08-09T13:53:47.231906+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T22:47:39.839446+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.36","version":"0'0","reported_seq":9,"reported_epoch":128,"state":"active+clean","last_fresh":"2023-08-09T13:53:48.324847+0000","last_change":"2023-08-09T13:53:48.320807+0000","last_active":"2023-08-09T13:53:48.324847+0000","last_peered":"2023-08-09T13:53:48.324847+0000","last_clean":"2023-08-09T13:53:48.324847+0000","last_became_active":"2023-08-09T13:53:48.320055+0000","last_became_peered":"2023-08-09T13:53:48.320055+0000","last_unstale":"2023-08-09T13:53:48.324847+0000","last_undegraded":"2023-08-09T13:53:48.324847+0000","last_fullsized":"2023-08-09T13:53:48.324847+0000","mapping_epoch":127,"log_start":"0'0","ondisk_log_start":"0'0","created":127,"last_epoch_clean":128,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:53:47.231906+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:53:47.231906+0000","last_clean_scrub_stamp":"2023-08-09T13:53:47.231906+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T20:39:00.369183+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.35","version":"0'0","reported_seq":9,"reported_epoch":128,"state":"active+clean","last_fresh":"2023-08-09T13:53:48.324801+0000","last_change":"2023-08-09T13:53:48.316614+0000","last_active":"2023-08-09T13:53:48.324801+0000","last_peered":"2023-08-09T13:53:48.324801+0000","last_clean":"2023-08-09T13:53:48.324801+0000","last_became_active":"2023-08-09T13:53:48.315558+0000","last_became_peered":"2023-08-09T13:53:48.315558+0000","last_unstale":"2023-08-09T13:53:48.324801+0000","last_undegraded":"2023-08-09T13:53:48.324801+0000","last_fullsized":"2023-08-09T13:53:48.324801+0000","mapping_epoch":127,"log_start":"0'0","ondisk_log_start":"0'0","created":127,"last_epoch_clean":128,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:53:47.231906+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:53:47.231906+0000","last_clean_scrub_stamp":"2023-08-09T13:53:47.231906+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T19:15:44.745804+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.34","version":"0'0","reported_seq":9,"reported_epoch":128,"state":"active+clean","last_fresh":"2023-08-09T13:53:48.324951+0000","last_change":"2023-08-09T13:53:48.321443+0000","last_active":"2023-08-09T13:53:48.324951+0000","last_peered":"2023-08-09T13:53:48.324951+0000","last_clean":"2023-08-09T13:53:48.324951+0000","last_became_active":"2023-08-09T13:53:48.320604+0000","last_became_peered":"2023-08-09T13:53:48.320604+0000","last_unstale":"2023-08-09T13:53:48.324951+0000","last_undegraded":"2023-08-09T13:53:48.324951+0000","last_fullsized":"2023-08-09T13:53:48.324951+0000","mapping_epoch":127,"log_start":"0'0","ondisk_log_start":"0'0","created":127,"last_epoch_clean":128,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:53:47.231906+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:53:47.231906+0000","last_clean_scrub_stamp":"2023-08-09T13:53:47.231906+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-11T01:26:12.436556+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.33","version":"0'0","reported_seq":9,"reported_epoch":128,"state":"active+clean","last_fresh":"2023-08-09T13:53:48.325763+0000","last_change":"2023-08-09T13:53:48.324709+0000","last_active":"2023-08-09T13:53:48.325763+0000","last_peered":"2023-08-09T13:53:48.325763+0000","last_clean":"2023-08-09T13:53:48.325763+0000","last_became_active":"2023-08-09T13:53:48.324422+0000","last_became_peered":"2023-08-09T13:53:48.324422+0000","last_unstale":"2023-08-09T13:53:48.325763+0000","last_undegraded":"2023-08-09T13:53:48.325763+0000","last_fullsized":"2023-08-09T13:53:48.325763+0000","mapping_epoch":127,"log_start":"0'0","ondisk_log_start":"0'0","created":127,"last_epoch_clean":128,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:53:47.231906+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:53:47.231906+0000","last_clean_scrub_stamp":"2023-08-09T13:53:47.231906+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T15:38:37.763892+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"17.32","version":"0'0","reported_seq":9,"reported_epoch":128,"state":"active+clean","last_fresh":"2023-08-09T13:53:48.317062+0000","last_change":"2023-08-09T13:53:48.314769+0000","last_active":"2023-08-09T13:53:48.317062+0000","last_peered":"2023-08-09T13:53:48.317062+0000","last_clean":"2023-08-09T13:53:48.317062+0000","last_became_active":"2023-08-09T13:53:48.314498+0000","last_became_peered":"2023-08-09T13:53:48.314498+0000","last_unstale":"2023-08-09T13:53:48.317062+0000","last_undegraded":"2023-08-09T13:53:48.317062+0000","last_fullsized":"2023-08-09T13:53:48.317062+0000","mapping_epoch":127,"log_start":"0'0","ondisk_log_start":"0'0","created":127,"last_epoch_clean":128,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:53:47.231906+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:53:47.231906+0000","last_clean_scrub_stamp":"2023-08-09T13:53:47.231906+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-11T01:26:16.686764+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"17.31","version":"0'0","reported_seq":11,"reported_epoch":129,"state":"active+clean","last_fresh":"2023-08-09T13:53:49.277638+0000","last_change":"2023-08-09T13:53:48.295587+0000","last_active":"2023-08-09T13:53:49.277638+0000","last_peered":"2023-08-09T13:53:49.277638+0000","last_clean":"2023-08-09T13:53:49.277638+0000","last_became_active":"2023-08-09T13:53:48.295280+0000","last_became_peered":"2023-08-09T13:53:48.295280+0000","last_unstale":"2023-08-09T13:53:49.277638+0000","last_undegraded":"2023-08-09T13:53:49.277638+0000","last_fullsized":"2023-08-09T13:53:49.277638+0000","mapping_epoch":127,"log_start":"0'0","ondisk_log_start":"0'0","created":127,"last_epoch_clean":128,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:53:47.231906+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:53:47.231906+0000","last_clean_scrub_stamp":"2023-08-09T13:53:47.231906+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T19:15:58.915975+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"17.30","version":"0'0","reported_seq":9,"reported_epoch":128,"state":"active+clean","last_fresh":"2023-08-09T13:53:48.320509+0000","last_change":"2023-08-09T13:53:48.315847+0000","last_active":"2023-08-09T13:53:48.320509+0000","last_peered":"2023-08-09T13:53:48.320509+0000","last_clean":"2023-08-09T13:53:48.320509+0000","last_became_active":"2023-08-09T13:53:48.315567+0000","last_became_peered":"2023-08-09T13:53:48.315567+0000","last_unstale":"2023-08-09T13:53:48.320509+0000","last_undegraded":"2023-08-09T13:53:48.320509+0000","last_fullsized":"2023-08-09T13:53:48.320509+0000","mapping_epoch":127,"log_start":"0'0","ondisk_log_start":"0'0","created":127,"last_epoch_clean":128,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:53:47.231906+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:53:47.231906+0000","last_clean_scrub_stamp":"2023-08-09T13:53:47.231906+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T22:44:56.011608+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.2f","version":"0'0","reported_seq":9,"reported_epoch":128,"state":"active+clean","last_fresh":"2023-08-09T13:53:48.325011+0000","last_change":"2023-08-09T13:53:48.318420+0000","last_active":"2023-08-09T13:53:48.325011+0000","last_peered":"2023-08-09T13:53:48.325011+0000","last_clean":"2023-08-09T13:53:48.325011+0000","last_became_active":"2023-08-09T13:53:48.318135+0000","last_became_peered":"2023-08-09T13:53:48.318135+0000","last_unstale":"2023-08-09T13:53:48.325011+0000","last_undegraded":"2023-08-09T13:53:48.325011+0000","last_fullsized":"2023-08-09T13:53:48.325011+0000","mapping_epoch":127,"log_start":"0'0","ondisk_log_start":"0'0","created":127,"last_epoch_clean":128,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:53:47.231906+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:53:47.231906+0000","last_clean_scrub_stamp":"2023-08-09T13:53:47.231906+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T22:31:35.095714+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.2e","version":"0'0","reported_seq":11,"reported_epoch":129,"state":"active+clean","last_fresh":"2023-08-09T13:53:49.281097+0000","last_change":"2023-08-09T13:53:48.314917+0000","last_active":"2023-08-09T13:53:49.281097+0000","last_peered":"2023-08-09T13:53:49.281097+0000","last_clean":"2023-08-09T13:53:49.281097+0000","last_became_active":"2023-08-09T13:53:48.314633+0000","last_became_peered":"2023-08-09T13:53:48.314633+0000","last_unstale":"2023-08-09T13:53:49.281097+0000","last_undegraded":"2023-08-09T13:53:49.281097+0000","last_fullsized":"2023-08-09T13:53:49.281097+0000","mapping_epoch":127,"log_start":"0'0","ondisk_log_start":"0'0","created":127,"last_epoch_clean":128,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:53:47.231906+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:53:47.231906+0000","last_clean_scrub_stamp":"2023-08-09T13:53:47.231906+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T18:08:49.619864+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"17.2d","version":"0'0","reported_seq":9,"reported_epoch":128,"state":"active+clean","last_fresh":"2023-08-09T13:53:48.318983+0000","last_change":"2023-08-09T13:53:48.317378+0000","last_active":"2023-08-09T13:53:48.318983+0000","last_peered":"2023-08-09T13:53:48.318983+0000","last_clean":"2023-08-09T13:53:48.318983+0000","last_became_active":"2023-08-09T13:53:48.317087+0000","last_became_peered":"2023-08-09T13:53:48.317087+0000","last_unstale":"2023-08-09T13:53:48.318983+0000","last_undegraded":"2023-08-09T13:53:48.318983+0000","last_fullsized":"2023-08-09T13:53:48.318983+0000","mapping_epoch":127,"log_start":"0'0","ondisk_log_start":"0'0","created":127,"last_epoch_clean":128,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:53:47.231906+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:53:47.231906+0000","last_clean_scrub_stamp":"2023-08-09T13:53:47.231906+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T14:47:06.672771+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"17.2c","version":"0'0","reported_seq":9,"reported_epoch":128,"state":"active+clean","last_fresh":"2023-08-09T13:53:48.324502+0000","last_change":"2023-08-09T13:53:48.315031+0000","last_active":"2023-08-09T13:53:48.324502+0000","last_peered":"2023-08-09T13:53:48.324502+0000","last_clean":"2023-08-09T13:53:48.324502+0000","last_became_active":"2023-08-09T13:53:48.314618+0000","last_became_peered":"2023-08-09T13:53:48.314618+0000","last_unstale":"2023-08-09T13:53:48.324502+0000","last_undegraded":"2023-08-09T13:53:48.324502+0000","last_fullsized":"2023-08-09T13:53:48.324502+0000","mapping_epoch":127,"log_start":"0'0","ondisk_log_start":"0'0","created":127,"last_epoch_clean":128,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:53:47.231906+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:53:47.231906+0000","last_clean_scrub_stamp":"2023-08-09T13:53:47.231906+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T19:06:46.667230+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.2b","version":"0'0","reported_seq":9,"reported_epoch":128,"state":"active+clean","last_fresh":"2023-08-09T13:53:48.311526+0000","last_change":"2023-08-09T13:53:48.307775+0000","last_active":"2023-08-09T13:53:48.311526+0000","last_peered":"2023-08-09T13:53:48.311526+0000","last_clean":"2023-08-09T13:53:48.311526+0000","last_became_active":"2023-08-09T13:53:48.307366+0000","last_became_peered":"2023-08-09T13:53:48.307366+0000","last_unstale":"2023-08-09T13:53:48.311526+0000","last_undegraded":"2023-08-09T13:53:48.311526+0000","last_fullsized":"2023-08-09T13:53:48.311526+0000","mapping_epoch":127,"log_start":"0'0","ondisk_log_start":"0'0","created":127,"last_epoch_clean":128,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:53:47.231906+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:53:47.231906+0000","last_clean_scrub_stamp":"2023-08-09T13:53:47.231906+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-11T01:26:12.436556+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.2a","version":"0'0","reported_seq":9,"reported_epoch":128,"state":"active+clean","last_fresh":"2023-08-09T13:53:48.317220+0000","last_change":"2023-08-09T13:53:48.314844+0000","last_active":"2023-08-09T13:53:48.317220+0000","last_peered":"2023-08-09T13:53:48.317220+0000","last_clean":"2023-08-09T13:53:48.317220+0000","last_became_active":"2023-08-09T13:53:48.314564+0000","last_became_peered":"2023-08-09T13:53:48.314564+0000","last_unstale":"2023-08-09T13:53:48.317220+0000","last_undegraded":"2023-08-09T13:53:48.317220+0000","last_fullsized":"2023-08-09T13:53:48.317220+0000","mapping_epoch":127,"log_start":"0'0","ondisk_log_start":"0'0","created":127,"last_epoch_clean":128,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:53:47.231906+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:53:47.231906+0000","last_clean_scrub_stamp":"2023-08-09T13:53:47.231906+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T22:45:31.755168+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"17.29","version":"0'0","reported_seq":9,"reported_epoch":128,"state":"active+clean","last_fresh":"2023-08-09T13:53:48.324099+0000","last_change":"2023-08-09T13:53:48.313683+0000","last_active":"2023-08-09T13:53:48.324099+0000","last_peered":"2023-08-09T13:53:48.324099+0000","last_clean":"2023-08-09T13:53:48.324099+0000","last_became_active":"2023-08-09T13:53:48.313375+0000","last_became_peered":"2023-08-09T13:53:48.313375+0000","last_unstale":"2023-08-09T13:53:48.324099+0000","last_undegraded":"2023-08-09T13:53:48.324099+0000","last_fullsized":"2023-08-09T13:53:48.324099+0000","mapping_epoch":127,"log_start":"0'0","ondisk_log_start":"0'0","created":127,"last_epoch_clean":128,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:53:47.231906+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:53:47.231906+0000","last_clean_scrub_stamp":"2023-08-09T13:53:47.231906+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T19:48:33.093205+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.28","version":"0'0","reported_seq":9,"reported_epoch":128,"state":"active+clean","last_fresh":"2023-08-09T13:53:48.315613+0000","last_change":"2023-08-09T13:53:48.313694+0000","last_active":"2023-08-09T13:53:48.315613+0000","last_peered":"2023-08-09T13:53:48.315613+0000","last_clean":"2023-08-09T13:53:48.315613+0000","last_became_active":"2023-08-09T13:53:48.312228+0000","last_became_peered":"2023-08-09T13:53:48.312228+0000","last_unstale":"2023-08-09T13:53:48.315613+0000","last_undegraded":"2023-08-09T13:53:48.315613+0000","last_fullsized":"2023-08-09T13:53:48.315613+0000","mapping_epoch":127,"log_start":"0'0","ondisk_log_start":"0'0","created":127,"last_epoch_clean":128,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:53:47.231906+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:53:47.231906+0000","last_clean_scrub_stamp":"2023-08-09T13:53:47.231906+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T14:21:57.770836+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"17.27","version":"0'0","reported_seq":9,"reported_epoch":128,"state":"active+clean","last_fresh":"2023-08-09T13:53:48.325030+0000","last_change":"2023-08-09T13:53:48.318790+0000","last_active":"2023-08-09T13:53:48.325030+0000","last_peered":"2023-08-09T13:53:48.325030+0000","last_clean":"2023-08-09T13:53:48.325030+0000","last_became_active":"2023-08-09T13:53:48.318492+0000","last_became_peered":"2023-08-09T13:53:48.318492+0000","last_unstale":"2023-08-09T13:53:48.325030+0000","last_undegraded":"2023-08-09T13:53:48.325030+0000","last_fullsized":"2023-08-09T13:53:48.325030+0000","mapping_epoch":127,"log_start":"0'0","ondisk_log_start":"0'0","created":127,"last_epoch_clean":128,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:53:47.231906+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:53:47.231906+0000","last_clean_scrub_stamp":"2023-08-09T13:53:47.231906+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T17:37:00.223661+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.26","version":"0'0","reported_seq":9,"reported_epoch":128,"state":"active+clean","last_fresh":"2023-08-09T13:53:48.314212+0000","last_change":"2023-08-09T13:53:48.307656+0000","last_active":"2023-08-09T13:53:48.314212+0000","last_peered":"2023-08-09T13:53:48.314212+0000","last_clean":"2023-08-09T13:53:48.314212+0000","last_became_active":"2023-08-09T13:53:48.307303+0000","last_became_peered":"2023-08-09T13:53:48.307303+0000","last_unstale":"2023-08-09T13:53:48.314212+0000","last_undegraded":"2023-08-09T13:53:48.314212+0000","last_fullsized":"2023-08-09T13:53:48.314212+0000","mapping_epoch":127,"log_start":"0'0","ondisk_log_start":"0'0","created":127,"last_epoch_clean":128,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:53:47.231906+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:53:47.231906+0000","last_clean_scrub_stamp":"2023-08-09T13:53:47.231906+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T19:06:46.667230+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.25","version":"0'0","reported_seq":11,"reported_epoch":129,"state":"active+clean","last_fresh":"2023-08-09T13:53:49.277776+0000","last_change":"2023-08-09T13:53:48.299619+0000","last_active":"2023-08-09T13:53:49.277776+0000","last_peered":"2023-08-09T13:53:49.277776+0000","last_clean":"2023-08-09T13:53:49.277776+0000","last_became_active":"2023-08-09T13:53:48.299350+0000","last_became_peered":"2023-08-09T13:53:48.299350+0000","last_unstale":"2023-08-09T13:53:49.277776+0000","last_undegraded":"2023-08-09T13:53:49.277776+0000","last_fullsized":"2023-08-09T13:53:49.277776+0000","mapping_epoch":127,"log_start":"0'0","ondisk_log_start":"0'0","created":127,"last_epoch_clean":128,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:53:47.231906+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:53:47.231906+0000","last_clean_scrub_stamp":"2023-08-09T13:53:47.231906+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T19:24:17.238820+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"17.24","version":"0'0","reported_seq":9,"reported_epoch":128,"state":"active+clean","last_fresh":"2023-08-09T13:53:48.306961+0000","last_change":"2023-08-09T13:53:48.300953+0000","last_active":"2023-08-09T13:53:48.306961+0000","last_peered":"2023-08-09T13:53:48.306961+0000","last_clean":"2023-08-09T13:53:48.306961+0000","last_became_active":"2023-08-09T13:53:48.300669+0000","last_became_peered":"2023-08-09T13:53:48.300669+0000","last_unstale":"2023-08-09T13:53:48.306961+0000","last_undegraded":"2023-08-09T13:53:48.306961+0000","last_fullsized":"2023-08-09T13:53:48.306961+0000","mapping_epoch":127,"log_start":"0'0","ondisk_log_start":"0'0","created":127,"last_epoch_clean":128,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:53:47.231906+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:53:47.231906+0000","last_clean_scrub_stamp":"2023-08-09T13:53:47.231906+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T15:29:08.206128+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"17.23","version":"0'0","reported_seq":9,"reported_epoch":128,"state":"active+clean","last_fresh":"2023-08-09T13:53:48.324224+0000","last_change":"2023-08-09T13:53:48.315018+0000","last_active":"2023-08-09T13:53:48.324224+0000","last_peered":"2023-08-09T13:53:48.324224+0000","last_clean":"2023-08-09T13:53:48.324224+0000","last_became_active":"2023-08-09T13:53:48.314730+0000","last_became_peered":"2023-08-09T13:53:48.314730+0000","last_unstale":"2023-08-09T13:53:48.324224+0000","last_undegraded":"2023-08-09T13:53:48.324224+0000","last_fullsized":"2023-08-09T13:53:48.324224+0000","mapping_epoch":127,"log_start":"0'0","ondisk_log_start":"0'0","created":127,"last_epoch_clean":128,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:53:47.231906+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:53:47.231906+0000","last_clean_scrub_stamp":"2023-08-09T13:53:47.231906+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T20:47:17.015473+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.22","version":"0'0","reported_seq":9,"reported_epoch":128,"state":"active+clean","last_fresh":"2023-08-09T13:53:48.316311+0000","last_change":"2023-08-09T13:53:48.312900+0000","last_active":"2023-08-09T13:53:48.316311+0000","last_peered":"2023-08-09T13:53:48.316311+0000","last_clean":"2023-08-09T13:53:48.316311+0000","last_became_active":"2023-08-09T13:53:48.312616+0000","last_became_peered":"2023-08-09T13:53:48.312616+0000","last_unstale":"2023-08-09T13:53:48.316311+0000","last_undegraded":"2023-08-09T13:53:48.316311+0000","last_fullsized":"2023-08-09T13:53:48.316311+0000","mapping_epoch":127,"log_start":"0'0","ondisk_log_start":"0'0","created":127,"last_epoch_clean":128,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:53:47.231906+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:53:47.231906+0000","last_clean_scrub_stamp":"2023-08-09T13:53:47.231906+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T22:33:37.323275+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"17.21","version":"0'0","reported_seq":9,"reported_epoch":128,"state":"active+clean","last_fresh":"2023-08-09T13:53:48.314244+0000","last_change":"2023-08-09T13:53:48.309422+0000","last_active":"2023-08-09T13:53:48.314244+0000","last_peered":"2023-08-09T13:53:48.314244+0000","last_clean":"2023-08-09T13:53:48.314244+0000","last_became_active":"2023-08-09T13:53:48.309069+0000","last_became_peered":"2023-08-09T13:53:48.309069+0000","last_unstale":"2023-08-09T13:53:48.314244+0000","last_undegraded":"2023-08-09T13:53:48.314244+0000","last_fullsized":"2023-08-09T13:53:48.314244+0000","mapping_epoch":127,"log_start":"0'0","ondisk_log_start":"0'0","created":127,"last_epoch_clean":128,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:53:47.231906+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:53:47.231906+0000","last_clean_scrub_stamp":"2023-08-09T13:53:47.231906+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T19:15:44.745804+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.20","version":"0'0","reported_seq":11,"reported_epoch":129,"state":"active+clean","last_fresh":"2023-08-09T13:53:49.276878+0000","last_change":"2023-08-09T13:53:48.288611+0000","last_active":"2023-08-09T13:53:49.276878+0000","last_peered":"2023-08-09T13:53:49.276878+0000","last_clean":"2023-08-09T13:53:49.276878+0000","last_became_active":"2023-08-09T13:53:48.286725+0000","last_became_peered":"2023-08-09T13:53:48.286725+0000","last_unstale":"2023-08-09T13:53:49.276878+0000","last_undegraded":"2023-08-09T13:53:49.276878+0000","last_fullsized":"2023-08-09T13:53:49.276878+0000","mapping_epoch":127,"log_start":"0'0","ondisk_log_start":"0'0","created":127,"last_epoch_clean":128,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:53:47.231906+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:53:47.231906+0000","last_clean_scrub_stamp":"2023-08-09T13:53:47.231906+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T15:01:57.136358+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"17.1e","version":"0'0","reported_seq":11,"reported_epoch":129,"state":"active+clean","last_fresh":"2023-08-09T13:53:49.277801+0000","last_change":"2023-08-09T13:53:48.285528+0000","last_active":"2023-08-09T13:53:49.277801+0000","last_peered":"2023-08-09T13:53:49.277801+0000","last_clean":"2023-08-09T13:53:49.277801+0000","last_became_active":"2023-08-09T13:53:48.285131+0000","last_became_peered":"2023-08-09T13:53:48.285131+0000","last_unstale":"2023-08-09T13:53:49.277801+0000","last_undegraded":"2023-08-09T13:53:49.277801+0000","last_fullsized":"2023-08-09T13:53:49.277801+0000","mapping_epoch":127,"log_start":"0'0","ondisk_log_start":"0'0","created":127,"last_epoch_clean":128,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:53:47.231906+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:53:47.231906+0000","last_clean_scrub_stamp":"2023-08-09T13:53:47.231906+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T17:06:45.323084+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"16.1f","version":"128'4","reported_seq":15,"reported_epoch":128,"state":"active+clean","last_fresh":"2023-08-09T13:53:48.316973+0000","last_change":"2023-08-09T13:53:47.277376+0000","last_active":"2023-08-09T13:53:48.316973+0000","last_peered":"2023-08-09T13:53:48.316973+0000","last_clean":"2023-08-09T13:53:48.316973+0000","last_became_active":"2023-08-09T13:53:47.276816+0000","last_became_peered":"2023-08-09T13:53:47.276816+0000","last_unstale":"2023-08-09T13:53:48.316973+0000","last_undegraded":"2023-08-09T13:53:48.316973+0000","last_fullsized":"2023-08-09T13:53:48.316973+0000","mapping_epoch":126,"log_start":"0'0","ondisk_log_start":"0'0","created":126,"last_epoch_clean":127,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:53:46.216803+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:53:46.216803+0000","last_clean_scrub_stamp":"2023-08-09T13:53:46.216803+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T16:42:10.354539+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"17.1f","version":"0'0","reported_seq":11,"reported_epoch":129,"state":"active+clean","last_fresh":"2023-08-09T13:53:49.281286+0000","last_change":"2023-08-09T13:53:48.315043+0000","last_active":"2023-08-09T13:53:49.281286+0000","last_peered":"2023-08-09T13:53:49.281286+0000","last_clean":"2023-08-09T13:53:49.281286+0000","last_became_active":"2023-08-09T13:53:48.314579+0000","last_became_peered":"2023-08-09T13:53:48.314579+0000","last_unstale":"2023-08-09T13:53:49.281286+0000","last_undegraded":"2023-08-09T13:53:49.281286+0000","last_fullsized":"2023-08-09T13:53:49.281286+0000","mapping_epoch":127,"log_start":"0'0","ondisk_log_start":"0'0","created":127,"last_epoch_clean":128,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:53:47.231906+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:53:47.231906+0000","last_clean_scrub_stamp":"2023-08-09T13:53:47.231906+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T20:01:49.620207+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.1e","version":"128'1","reported_seq":12,"reported_epoch":128,"state":"active+clean","last_fresh":"2023-08-09T13:53:48.320711+0000","last_change":"2023-08-09T13:53:47.278027+0000","last_active":"2023-08-09T13:53:48.320711+0000","last_peered":"2023-08-09T13:53:48.320711+0000","last_clean":"2023-08-09T13:53:48.320711+0000","last_became_active":"2023-08-09T13:53:47.277645+0000","last_became_peered":"2023-08-09T13:53:47.277645+0000","last_unstale":"2023-08-09T13:53:48.320711+0000","last_undegraded":"2023-08-09T13:53:48.320711+0000","last_fullsized":"2023-08-09T13:53:48.320711+0000","mapping_epoch":126,"log_start":"0'0","ondisk_log_start":"0'0","created":126,"last_epoch_clean":127,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:53:46.216803+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:53:46.216803+0000","last_clean_scrub_stamp":"2023-08-09T13:53:46.216803+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T14:12:39.269881+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"17.1c","version":"0'0","reported_seq":11,"reported_epoch":129,"state":"active+clean","last_fresh":"2023-08-09T13:53:49.277476+0000","last_change":"2023-08-09T13:53:48.288150+0000","last_active":"2023-08-09T13:53:49.277476+0000","last_peered":"2023-08-09T13:53:49.277476+0000","last_clean":"2023-08-09T13:53:49.277476+0000","last_became_active":"2023-08-09T13:53:48.287822+0000","last_became_peered":"2023-08-09T13:53:48.287822+0000","last_unstale":"2023-08-09T13:53:49.277476+0000","last_undegraded":"2023-08-09T13:53:49.277476+0000","last_fullsized":"2023-08-09T13:53:49.277476+0000","mapping_epoch":127,"log_start":"0'0","ondisk_log_start":"0'0","created":127,"last_epoch_clean":128,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:53:47.231906+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:53:47.231906+0000","last_clean_scrub_stamp":"2023-08-09T13:53:47.231906+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T16:47:43.779024+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"16.1d","version":"128'2","reported_seq":15,"reported_epoch":129,"state":"active+clean","last_fresh":"2023-08-09T13:53:49.281849+0000","last_change":"2023-08-09T13:53:47.265097+0000","last_active":"2023-08-09T13:53:49.281849+0000","last_peered":"2023-08-09T13:53:49.281849+0000","last_clean":"2023-08-09T13:53:49.281849+0000","last_became_active":"2023-08-09T13:53:47.264731+0000","last_became_peered":"2023-08-09T13:53:47.264731+0000","last_unstale":"2023-08-09T13:53:49.281849+0000","last_undegraded":"2023-08-09T13:53:49.281849+0000","last_fullsized":"2023-08-09T13:53:49.281849+0000","mapping_epoch":126,"log_start":"0'0","ondisk_log_start":"0'0","created":126,"last_epoch_clean":127,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:53:46.216803+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:53:46.216803+0000","last_clean_scrub_stamp":"2023-08-09T13:53:46.216803+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T16:33:44.031013+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"17.1d","version":"0'0","reported_seq":9,"reported_epoch":128,"state":"active+clean","last_fresh":"2023-08-09T13:53:48.325758+0000","last_change":"2023-08-09T13:53:48.323402+0000","last_active":"2023-08-09T13:53:48.325758+0000","last_peered":"2023-08-09T13:53:48.325758+0000","last_clean":"2023-08-09T13:53:48.325758+0000","last_became_active":"2023-08-09T13:53:48.323059+0000","last_became_peered":"2023-08-09T13:53:48.323059+0000","last_unstale":"2023-08-09T13:53:48.325758+0000","last_undegraded":"2023-08-09T13:53:48.325758+0000","last_fullsized":"2023-08-09T13:53:48.325758+0000","mapping_epoch":127,"log_start":"0'0","ondisk_log_start":"0'0","created":127,"last_epoch_clean":128,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:53:47.231906+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:53:47.231906+0000","last_clean_scrub_stamp":"2023-08-09T13:53:47.231906+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T15:29:48.199916+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"16.1c","version":"128'2","reported_seq":15,"reported_epoch":129,"state":"active+clean","last_fresh":"2023-08-09T13:53:49.282053+0000","last_change":"2023-08-09T13:53:47.262351+0000","last_active":"2023-08-09T13:53:49.282053+0000","last_peered":"2023-08-09T13:53:49.282053+0000","last_clean":"2023-08-09T13:53:49.282053+0000","last_became_active":"2023-08-09T13:53:47.261409+0000","last_became_peered":"2023-08-09T13:53:47.261409+0000","last_unstale":"2023-08-09T13:53:49.282053+0000","last_undegraded":"2023-08-09T13:53:49.282053+0000","last_fullsized":"2023-08-09T13:53:49.282053+0000","mapping_epoch":126,"log_start":"0'0","ondisk_log_start":"0'0","created":126,"last_epoch_clean":127,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:53:46.216803+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:53:46.216803+0000","last_clean_scrub_stamp":"2023-08-09T13:53:46.216803+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T14:33:48.941953+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"17.1a","version":"0'0","reported_seq":9,"reported_epoch":128,"state":"active+clean","last_fresh":"2023-08-09T13:53:48.314746+0000","last_change":"2023-08-09T13:53:48.309691+0000","last_active":"2023-08-09T13:53:48.314746+0000","last_peered":"2023-08-09T13:53:48.314746+0000","last_clean":"2023-08-09T13:53:48.314746+0000","last_became_active":"2023-08-09T13:53:48.309426+0000","last_became_peered":"2023-08-09T13:53:48.309426+0000","last_unstale":"2023-08-09T13:53:48.314746+0000","last_undegraded":"2023-08-09T13:53:48.314746+0000","last_fullsized":"2023-08-09T13:53:48.314746+0000","mapping_epoch":127,"log_start":"0'0","ondisk_log_start":"0'0","created":127,"last_epoch_clean":128,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:53:47.231906+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:53:47.231906+0000","last_clean_scrub_stamp":"2023-08-09T13:53:47.231906+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T23:03:43.024505+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.1b","version":"128'1","reported_seq":12,"reported_epoch":128,"state":"active+clean","last_fresh":"2023-08-09T13:53:48.315904+0000","last_change":"2023-08-09T13:53:47.274449+0000","last_active":"2023-08-09T13:53:48.315904+0000","last_peered":"2023-08-09T13:53:48.315904+0000","last_clean":"2023-08-09T13:53:48.315904+0000","last_became_active":"2023-08-09T13:53:47.274178+0000","last_became_peered":"2023-08-09T13:53:47.274178+0000","last_unstale":"2023-08-09T13:53:48.315904+0000","last_undegraded":"2023-08-09T13:53:48.315904+0000","last_fullsized":"2023-08-09T13:53:48.315904+0000","mapping_epoch":126,"log_start":"0'0","ondisk_log_start":"0'0","created":126,"last_epoch_clean":127,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:53:46.216803+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:53:46.216803+0000","last_clean_scrub_stamp":"2023-08-09T13:53:46.216803+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-11T00:35:49.085401+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"17.1b","version":"0'0","reported_seq":9,"reported_epoch":128,"state":"active+clean","last_fresh":"2023-08-09T13:53:48.311545+0000","last_change":"2023-08-09T13:53:48.308092+0000","last_active":"2023-08-09T13:53:48.311545+0000","last_peered":"2023-08-09T13:53:48.311545+0000","last_clean":"2023-08-09T13:53:48.311545+0000","last_became_active":"2023-08-09T13:53:48.307725+0000","last_became_peered":"2023-08-09T13:53:48.307725+0000","last_unstale":"2023-08-09T13:53:48.311545+0000","last_undegraded":"2023-08-09T13:53:48.311545+0000","last_fullsized":"2023-08-09T13:53:48.311545+0000","mapping_epoch":127,"log_start":"0'0","ondisk_log_start":"0'0","created":127,"last_epoch_clean":128,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:53:47.231906+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:53:47.231906+0000","last_clean_scrub_stamp":"2023-08-09T13:53:47.231906+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T19:03:41.276282+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"16.1a","version":"128'1","reported_seq":12,"reported_epoch":128,"state":"active+clean","last_fresh":"2023-08-09T13:53:48.314262+0000","last_change":"2023-08-09T13:53:47.268608+0000","last_active":"2023-08-09T13:53:48.314262+0000","last_peered":"2023-08-09T13:53:48.314262+0000","last_clean":"2023-08-09T13:53:48.314262+0000","last_became_active":"2023-08-09T13:53:47.268202+0000","last_became_peered":"2023-08-09T13:53:47.268202+0000","last_unstale":"2023-08-09T13:53:48.314262+0000","last_undegraded":"2023-08-09T13:53:48.314262+0000","last_fullsized":"2023-08-09T13:53:48.314262+0000","mapping_epoch":126,"log_start":"0'0","ondisk_log_start":"0'0","created":126,"last_epoch_clean":127,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:53:46.216803+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:53:46.216803+0000","last_clean_scrub_stamp":"2023-08-09T13:53:46.216803+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-11T01:51:10.913522+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"17.18","version":"0'0","reported_seq":9,"reported_epoch":128,"state":"active+clean","last_fresh":"2023-08-09T13:53:48.305357+0000","last_change":"2023-08-09T13:53:48.299714+0000","last_active":"2023-08-09T13:53:48.305357+0000","last_peered":"2023-08-09T13:53:48.305357+0000","last_clean":"2023-08-09T13:53:48.305357+0000","last_became_active":"2023-08-09T13:53:48.299390+0000","last_became_peered":"2023-08-09T13:53:48.299390+0000","last_unstale":"2023-08-09T13:53:48.305357+0000","last_undegraded":"2023-08-09T13:53:48.305357+0000","last_fullsized":"2023-08-09T13:53:48.305357+0000","mapping_epoch":127,"log_start":"0'0","ondisk_log_start":"0'0","created":127,"last_epoch_clean":128,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:53:47.231906+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:53:47.231906+0000","last_clean_scrub_stamp":"2023-08-09T13:53:47.231906+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T19:31:11.030219+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.19","version":"0'0","reported_seq":11,"reported_epoch":128,"state":"active+clean","last_fresh":"2023-08-09T13:53:48.299334+0000","last_change":"2023-08-09T13:53:47.268290+0000","last_active":"2023-08-09T13:53:48.299334+0000","last_peered":"2023-08-09T13:53:48.299334+0000","last_clean":"2023-08-09T13:53:48.299334+0000","last_became_active":"2023-08-09T13:53:47.267889+0000","last_became_peered":"2023-08-09T13:53:47.267889+0000","last_unstale":"2023-08-09T13:53:48.299334+0000","last_undegraded":"2023-08-09T13:53:48.299334+0000","last_fullsized":"2023-08-09T13:53:48.299334+0000","mapping_epoch":126,"log_start":"0'0","ondisk_log_start":"0'0","created":126,"last_epoch_clean":127,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:53:46.216803+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:53:46.216803+0000","last_clean_scrub_stamp":"2023-08-09T13:53:46.216803+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T23:46:47.146096+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.19","version":"0'0","reported_seq":9,"reported_epoch":128,"state":"active+clean","last_fresh":"2023-08-09T13:53:48.320327+0000","last_change":"2023-08-09T13:53:48.317465+0000","last_active":"2023-08-09T13:53:48.320327+0000","last_peered":"2023-08-09T13:53:48.320327+0000","last_clean":"2023-08-09T13:53:48.320327+0000","last_became_active":"2023-08-09T13:53:48.316984+0000","last_became_peered":"2023-08-09T13:53:48.316984+0000","last_unstale":"2023-08-09T13:53:48.320327+0000","last_undegraded":"2023-08-09T13:53:48.320327+0000","last_fullsized":"2023-08-09T13:53:48.320327+0000","mapping_epoch":127,"log_start":"0'0","ondisk_log_start":"0'0","created":127,"last_epoch_clean":128,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:53:47.231906+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:53:47.231906+0000","last_clean_scrub_stamp":"2023-08-09T13:53:47.231906+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T14:44:39.079325+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"16.18","version":"0'0","reported_seq":11,"reported_epoch":128,"state":"active+clean","last_fresh":"2023-08-09T13:53:48.295239+0000","last_change":"2023-08-09T13:53:47.270563+0000","last_active":"2023-08-09T13:53:48.295239+0000","last_peered":"2023-08-09T13:53:48.295239+0000","last_clean":"2023-08-09T13:53:48.295239+0000","last_became_active":"2023-08-09T13:53:47.270261+0000","last_became_peered":"2023-08-09T13:53:47.270261+0000","last_unstale":"2023-08-09T13:53:48.295239+0000","last_undegraded":"2023-08-09T13:53:48.295239+0000","last_fullsized":"2023-08-09T13:53:48.295239+0000","mapping_epoch":126,"log_start":"0'0","ondisk_log_start":"0'0","created":126,"last_epoch_clean":127,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:53:46.216803+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:53:46.216803+0000","last_clean_scrub_stamp":"2023-08-09T13:53:46.216803+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T19:48:32.078103+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.e","version":"0'0","reported_seq":11,"reported_epoch":129,"state":"active+clean","last_fresh":"2023-08-09T13:53:49.282325+0000","last_change":"2023-08-09T13:53:48.303296+0000","last_active":"2023-08-09T13:53:49.282325+0000","last_peered":"2023-08-09T13:53:49.282325+0000","last_clean":"2023-08-09T13:53:49.282325+0000","last_became_active":"2023-08-09T13:53:48.303021+0000","last_became_peered":"2023-08-09T13:53:48.303021+0000","last_unstale":"2023-08-09T13:53:49.282325+0000","last_undegraded":"2023-08-09T13:53:49.282325+0000","last_fullsized":"2023-08-09T13:53:49.282325+0000","mapping_epoch":127,"log_start":"0'0","ondisk_log_start":"0'0","created":127,"last_epoch_clean":128,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:53:47.231906+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:53:47.231906+0000","last_clean_scrub_stamp":"2023-08-09T13:53:47.231906+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T18:10:57.405290+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"16.f","version":"128'1","reported_seq":12,"reported_epoch":128,"state":"active+clean","last_fresh":"2023-08-09T13:53:48.313334+0000","last_change":"2023-08-09T13:53:47.274042+0000","last_active":"2023-08-09T13:53:48.313334+0000","last_peered":"2023-08-09T13:53:48.313334+0000","last_clean":"2023-08-09T13:53:48.313334+0000","last_became_active":"2023-08-09T13:53:47.273755+0000","last_became_peered":"2023-08-09T13:53:47.273755+0000","last_unstale":"2023-08-09T13:53:48.313334+0000","last_undegraded":"2023-08-09T13:53:48.313334+0000","last_fullsized":"2023-08-09T13:53:48.313334+0000","mapping_epoch":126,"log_start":"0'0","ondisk_log_start":"0'0","created":126,"last_epoch_clean":127,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:53:46.216803+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:53:46.216803+0000","last_clean_scrub_stamp":"2023-08-09T13:53:46.216803+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T16:58:42.872129+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.f","version":"0'0","reported_seq":11,"reported_epoch":129,"state":"active+clean","last_fresh":"2023-08-09T13:53:49.277981+0000","last_change":"2023-08-09T13:53:48.299020+0000","last_active":"2023-08-09T13:53:49.277981+0000","last_peered":"2023-08-09T13:53:49.277981+0000","last_clean":"2023-08-09T13:53:49.277981+0000","last_became_active":"2023-08-09T13:53:48.298711+0000","last_became_peered":"2023-08-09T13:53:48.298711+0000","last_unstale":"2023-08-09T13:53:49.277981+0000","last_undegraded":"2023-08-09T13:53:49.277981+0000","last_fullsized":"2023-08-09T13:53:49.277981+0000","mapping_epoch":127,"log_start":"0'0","ondisk_log_start":"0'0","created":127,"last_epoch_clean":128,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:53:47.231906+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:53:47.231906+0000","last_clean_scrub_stamp":"2023-08-09T13:53:47.231906+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T23:23:17.088727+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"16.e","version":"128'2","reported_seq":15,"reported_epoch":129,"state":"active+clean","last_fresh":"2023-08-09T13:53:49.282359+0000","last_change":"2023-08-09T13:53:47.277672+0000","last_active":"2023-08-09T13:53:49.282359+0000","last_peered":"2023-08-09T13:53:49.282359+0000","last_clean":"2023-08-09T13:53:49.282359+0000","last_became_active":"2023-08-09T13:53:47.277392+0000","last_became_peered":"2023-08-09T13:53:47.277392+0000","last_unstale":"2023-08-09T13:53:49.282359+0000","last_undegraded":"2023-08-09T13:53:49.282359+0000","last_fullsized":"2023-08-09T13:53:49.282359+0000","mapping_epoch":126,"log_start":"0'0","ondisk_log_start":"0'0","created":126,"last_epoch_clean":127,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:53:46.216803+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:53:46.216803+0000","last_clean_scrub_stamp":"2023-08-09T13:53:46.216803+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T21:37:51.370269+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"17.c","version":"0'0","reported_seq":9,"reported_epoch":128,"state":"active+clean","last_fresh":"2023-08-09T13:53:48.313921+0000","last_change":"2023-08-09T13:53:48.308240+0000","last_active":"2023-08-09T13:53:48.313921+0000","last_peered":"2023-08-09T13:53:48.313921+0000","last_clean":"2023-08-09T13:53:48.313921+0000","last_became_active":"2023-08-09T13:53:48.307919+0000","last_became_peered":"2023-08-09T13:53:48.307919+0000","last_unstale":"2023-08-09T13:53:48.313921+0000","last_undegraded":"2023-08-09T13:53:48.313921+0000","last_fullsized":"2023-08-09T13:53:48.313921+0000","mapping_epoch":127,"log_start":"0'0","ondisk_log_start":"0'0","created":127,"last_epoch_clean":128,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:53:47.231906+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:53:47.231906+0000","last_clean_scrub_stamp":"2023-08-09T13:53:47.231906+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-11T01:25:07.887861+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"16.d","version":"128'1","reported_seq":14,"reported_epoch":129,"state":"active+clean","last_fresh":"2023-08-09T13:53:49.281087+0000","last_change":"2023-08-09T13:53:47.272814+0000","last_active":"2023-08-09T13:53:49.281087+0000","last_peered":"2023-08-09T13:53:49.281087+0000","last_clean":"2023-08-09T13:53:49.281087+0000","last_became_active":"2023-08-09T13:53:47.272527+0000","last_became_peered":"2023-08-09T13:53:47.272527+0000","last_unstale":"2023-08-09T13:53:49.281087+0000","last_undegraded":"2023-08-09T13:53:49.281087+0000","last_fullsized":"2023-08-09T13:53:49.281087+0000","mapping_epoch":126,"log_start":"0'0","ondisk_log_start":"0'0","created":126,"last_epoch_clean":127,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:53:46.216803+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:53:46.216803+0000","last_clean_scrub_stamp":"2023-08-09T13:53:46.216803+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T18:26:09.496138+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"17.d","version":"0'0","reported_seq":9,"reported_epoch":128,"state":"active+clean","last_fresh":"2023-08-09T13:53:48.313968+0000","last_change":"2023-08-09T13:53:48.308420+0000","last_active":"2023-08-09T13:53:48.313968+0000","last_peered":"2023-08-09T13:53:48.313968+0000","last_clean":"2023-08-09T13:53:48.313968+0000","last_became_active":"2023-08-09T13:53:48.306769+0000","last_became_peered":"2023-08-09T13:53:48.306769+0000","last_unstale":"2023-08-09T13:53:48.313968+0000","last_undegraded":"2023-08-09T13:53:48.313968+0000","last_fullsized":"2023-08-09T13:53:48.313968+0000","mapping_epoch":127,"log_start":"0'0","ondisk_log_start":"0'0","created":127,"last_epoch_clean":128,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:53:47.231906+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:53:47.231906+0000","last_clean_scrub_stamp":"2023-08-09T13:53:47.231906+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-11T00:49:07.518548+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"16.c","version":"0'0","reported_seq":11,"reported_epoch":128,"state":"active+clean","last_fresh":"2023-08-09T13:53:48.298264+0000","last_change":"2023-08-09T13:53:47.275981+0000","last_active":"2023-08-09T13:53:48.298264+0000","last_peered":"2023-08-09T13:53:48.298264+0000","last_clean":"2023-08-09T13:53:48.298264+0000","last_became_active":"2023-08-09T13:53:47.275694+0000","last_became_peered":"2023-08-09T13:53:47.275694+0000","last_unstale":"2023-08-09T13:53:48.298264+0000","last_undegraded":"2023-08-09T13:53:48.298264+0000","last_fullsized":"2023-08-09T13:53:48.298264+0000","mapping_epoch":126,"log_start":"0'0","ondisk_log_start":"0'0","created":126,"last_epoch_clean":127,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:53:46.216803+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:53:46.216803+0000","last_clean_scrub_stamp":"2023-08-09T13:53:46.216803+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-11T01:25:39.636954+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"17.a","version":"0'0","reported_seq":9,"reported_epoch":128,"state":"active+clean","last_fresh":"2023-08-09T13:53:48.325194+0000","last_change":"2023-08-09T13:53:48.318638+0000","last_active":"2023-08-09T13:53:48.325194+0000","last_peered":"2023-08-09T13:53:48.325194+0000","last_clean":"2023-08-09T13:53:48.325194+0000","last_became_active":"2023-08-09T13:53:48.318329+0000","last_became_peered":"2023-08-09T13:53:48.318329+0000","last_unstale":"2023-08-09T13:53:48.325194+0000","last_undegraded":"2023-08-09T13:53:48.325194+0000","last_fullsized":"2023-08-09T13:53:48.325194+0000","mapping_epoch":127,"log_start":"0'0","ondisk_log_start":"0'0","created":127,"last_epoch_clean":128,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:53:47.231906+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:53:47.231906+0000","last_clean_scrub_stamp":"2023-08-09T13:53:47.231906+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-11T01:16:24.402328+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"16.b","version":"128'2","reported_seq":15,"reported_epoch":129,"state":"active+clean","last_fresh":"2023-08-09T13:53:49.277577+0000","last_change":"2023-08-09T13:53:47.262109+0000","last_active":"2023-08-09T13:53:49.277577+0000","last_peered":"2023-08-09T13:53:49.277577+0000","last_clean":"2023-08-09T13:53:49.277577+0000","last_became_active":"2023-08-09T13:53:47.261837+0000","last_became_peered":"2023-08-09T13:53:47.261837+0000","last_unstale":"2023-08-09T13:53:49.277577+0000","last_undegraded":"2023-08-09T13:53:49.277577+0000","last_fullsized":"2023-08-09T13:53:49.277577+0000","mapping_epoch":126,"log_start":"0'0","ondisk_log_start":"0'0","created":126,"last_epoch_clean":127,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:53:46.216803+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:53:46.216803+0000","last_clean_scrub_stamp":"2023-08-09T13:53:46.216803+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T17:22:46.197957+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"17.b","version":"0'0","reported_seq":9,"reported_epoch":128,"state":"active+clean","last_fresh":"2023-08-09T13:53:48.320461+0000","last_change":"2023-08-09T13:53:48.315435+0000","last_active":"2023-08-09T13:53:48.320461+0000","last_peered":"2023-08-09T13:53:48.320461+0000","last_clean":"2023-08-09T13:53:48.320461+0000","last_became_active":"2023-08-09T13:53:48.315143+0000","last_became_peered":"2023-08-09T13:53:48.315143+0000","last_unstale":"2023-08-09T13:53:48.320461+0000","last_undegraded":"2023-08-09T13:53:48.320461+0000","last_fullsized":"2023-08-09T13:53:48.320461+0000","mapping_epoch":127,"log_start":"0'0","ondisk_log_start":"0'0","created":127,"last_epoch_clean":128,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:53:47.231906+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:53:47.231906+0000","last_clean_scrub_stamp":"2023-08-09T13:53:47.231906+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T21:21:34.369855+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.a","version":"0'0","reported_seq":13,"reported_epoch":129,"state":"active+clean","last_fresh":"2023-08-09T13:53:49.281565+0000","last_change":"2023-08-09T13:53:47.265204+0000","last_active":"2023-08-09T13:53:49.281565+0000","last_peered":"2023-08-09T13:53:49.281565+0000","last_clean":"2023-08-09T13:53:49.281565+0000","last_became_active":"2023-08-09T13:53:47.264864+0000","last_became_peered":"2023-08-09T13:53:47.264864+0000","last_unstale":"2023-08-09T13:53:49.281565+0000","last_undegraded":"2023-08-09T13:53:49.281565+0000","last_fullsized":"2023-08-09T13:53:49.281565+0000","mapping_epoch":126,"log_start":"0'0","ondisk_log_start":"0'0","created":126,"last_epoch_clean":127,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:53:46.216803+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:53:46.216803+0000","last_clean_scrub_stamp":"2023-08-09T13:53:46.216803+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T15:51:16.040950+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"17.8","version":"0'0","reported_seq":9,"reported_epoch":128,"state":"active+clean","last_fresh":"2023-08-09T13:53:48.325414+0000","last_change":"2023-08-09T13:53:48.321375+0000","last_active":"2023-08-09T13:53:48.325414+0000","last_peered":"2023-08-09T13:53:48.325414+0000","last_clean":"2023-08-09T13:53:48.325414+0000","last_became_active":"2023-08-09T13:53:48.320785+0000","last_became_peered":"2023-08-09T13:53:48.320785+0000","last_unstale":"2023-08-09T13:53:48.325414+0000","last_undegraded":"2023-08-09T13:53:48.325414+0000","last_fullsized":"2023-08-09T13:53:48.325414+0000","mapping_epoch":127,"log_start":"0'0","ondisk_log_start":"0'0","created":127,"last_epoch_clean":128,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:53:47.231906+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:53:47.231906+0000","last_clean_scrub_stamp":"2023-08-09T13:53:47.231906+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T21:55:12.741547+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"16.9","version":"0'0","reported_seq":13,"reported_epoch":129,"state":"active+clean","last_fresh":"2023-08-09T13:53:49.277985+0000","last_change":"2023-08-09T13:53:47.259951+0000","last_active":"2023-08-09T13:53:49.277985+0000","last_peered":"2023-08-09T13:53:49.277985+0000","last_clean":"2023-08-09T13:53:49.277985+0000","last_became_active":"2023-08-09T13:53:47.259603+0000","last_became_peered":"2023-08-09T13:53:47.259603+0000","last_unstale":"2023-08-09T13:53:49.277985+0000","last_undegraded":"2023-08-09T13:53:49.277985+0000","last_fullsized":"2023-08-09T13:53:49.277985+0000","mapping_epoch":126,"log_start":"0'0","ondisk_log_start":"0'0","created":126,"last_epoch_clean":127,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:53:46.216803+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:53:46.216803+0000","last_clean_scrub_stamp":"2023-08-09T13:53:46.216803+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T14:01:36.656215+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"17.9","version":"0'0","reported_seq":9,"reported_epoch":128,"state":"active+clean","last_fresh":"2023-08-09T13:53:48.303798+0000","last_change":"2023-08-09T13:53:48.300390+0000","last_active":"2023-08-09T13:53:48.303798+0000","last_peered":"2023-08-09T13:53:48.303798+0000","last_clean":"2023-08-09T13:53:48.303798+0000","last_became_active":"2023-08-09T13:53:48.300109+0000","last_became_peered":"2023-08-09T13:53:48.300109+0000","last_unstale":"2023-08-09T13:53:48.303798+0000","last_undegraded":"2023-08-09T13:53:48.303798+0000","last_fullsized":"2023-08-09T13:53:48.303798+0000","mapping_epoch":127,"log_start":"0'0","ondisk_log_start":"0'0","created":127,"last_epoch_clean":128,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:53:47.231906+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:53:47.231906+0000","last_clean_scrub_stamp":"2023-08-09T13:53:47.231906+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T23:13:06.550538+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"16.8","version":"0'0","reported_seq":11,"reported_epoch":128,"state":"active+clean","last_fresh":"2023-08-09T13:53:48.295384+0000","last_change":"2023-08-09T13:53:47.275496+0000","last_active":"2023-08-09T13:53:48.295384+0000","last_peered":"2023-08-09T13:53:48.295384+0000","last_clean":"2023-08-09T13:53:48.295384+0000","last_became_active":"2023-08-09T13:53:47.274904+0000","last_became_peered":"2023-08-09T13:53:47.274904+0000","last_unstale":"2023-08-09T13:53:48.295384+0000","last_undegraded":"2023-08-09T13:53:48.295384+0000","last_fullsized":"2023-08-09T13:53:48.295384+0000","mapping_epoch":126,"log_start":"0'0","ondisk_log_start":"0'0","created":126,"last_epoch_clean":127,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:53:46.216803+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:53:46.216803+0000","last_clean_scrub_stamp":"2023-08-09T13:53:46.216803+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T22:31:34.080611+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.6","version":"0'0","reported_seq":11,"reported_epoch":129,"state":"active+clean","last_fresh":"2023-08-09T13:53:49.281107+0000","last_change":"2023-08-09T13:53:48.316445+0000","last_active":"2023-08-09T13:53:49.281107+0000","last_peered":"2023-08-09T13:53:49.281107+0000","last_clean":"2023-08-09T13:53:49.281107+0000","last_became_active":"2023-08-09T13:53:48.316150+0000","last_became_peered":"2023-08-09T13:53:48.316150+0000","last_unstale":"2023-08-09T13:53:49.281107+0000","last_undegraded":"2023-08-09T13:53:49.281107+0000","last_fullsized":"2023-08-09T13:53:49.281107+0000","mapping_epoch":127,"log_start":"0'0","ondisk_log_start":"0'0","created":127,"last_epoch_clean":128,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:53:47.231906+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:53:47.231906+0000","last_clean_scrub_stamp":"2023-08-09T13:53:47.231906+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-11T01:45:02.929600+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.7","version":"128'2","reported_seq":13,"reported_epoch":128,"state":"active+clean","last_fresh":"2023-08-09T13:53:48.315830+0000","last_change":"2023-08-09T13:53:47.276360+0000","last_active":"2023-08-09T13:53:48.315830+0000","last_peered":"2023-08-09T13:53:48.315830+0000","last_clean":"2023-08-09T13:53:48.315830+0000","last_became_active":"2023-08-09T13:53:47.275979+0000","last_became_peered":"2023-08-09T13:53:47.275979+0000","last_unstale":"2023-08-09T13:53:48.315830+0000","last_undegraded":"2023-08-09T13:53:48.315830+0000","last_fullsized":"2023-08-09T13:53:48.315830+0000","mapping_epoch":126,"log_start":"0'0","ondisk_log_start":"0'0","created":126,"last_epoch_clean":127,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:53:46.216803+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:53:46.216803+0000","last_clean_scrub_stamp":"2023-08-09T13:53:46.216803+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T18:19:49.568627+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"17.7","version":"0'0","reported_seq":9,"reported_epoch":128,"state":"active+clean","last_fresh":"2023-08-09T13:53:48.320514+0000","last_change":"2023-08-09T13:53:48.314920+0000","last_active":"2023-08-09T13:53:48.320514+0000","last_peered":"2023-08-09T13:53:48.320514+0000","last_clean":"2023-08-09T13:53:48.320514+0000","last_became_active":"2023-08-09T13:53:48.314355+0000","last_became_peered":"2023-08-09T13:53:48.314355+0000","last_unstale":"2023-08-09T13:53:48.320514+0000","last_undegraded":"2023-08-09T13:53:48.320514+0000","last_fullsized":"2023-08-09T13:53:48.320514+0000","mapping_epoch":127,"log_start":"0'0","ondisk_log_start":"0'0","created":127,"last_epoch_clean":128,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:53:47.231906+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:53:47.231906+0000","last_clean_scrub_stamp":"2023-08-09T13:53:47.231906+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T20:47:17.015473+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.6","version":"0'0","reported_seq":13,"reported_epoch":129,"state":"active+clean","last_fresh":"2023-08-09T13:53:49.281608+0000","last_change":"2023-08-09T13:53:47.273109+0000","last_active":"2023-08-09T13:53:49.281608+0000","last_peered":"2023-08-09T13:53:49.281608+0000","last_clean":"2023-08-09T13:53:49.281608+0000","last_became_active":"2023-08-09T13:53:47.272823+0000","last_became_peered":"2023-08-09T13:53:47.272823+0000","last_unstale":"2023-08-09T13:53:49.281608+0000","last_undegraded":"2023-08-09T13:53:49.281608+0000","last_fullsized":"2023-08-09T13:53:49.281608+0000","mapping_epoch":126,"log_start":"0'0","ondisk_log_start":"0'0","created":126,"last_epoch_clean":127,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:53:46.216803+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:53:46.216803+0000","last_clean_scrub_stamp":"2023-08-09T13:53:46.216803+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T15:18:35.611711+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"17.4","version":"0'0","reported_seq":9,"reported_epoch":128,"state":"active+clean","last_fresh":"2023-08-09T13:53:48.307105+0000","last_change":"2023-08-09T13:53:48.299202+0000","last_active":"2023-08-09T13:53:48.307105+0000","last_peered":"2023-08-09T13:53:48.307105+0000","last_clean":"2023-08-09T13:53:48.307105+0000","last_became_active":"2023-08-09T13:53:48.298876+0000","last_became_peered":"2023-08-09T13:53:48.298876+0000","last_unstale":"2023-08-09T13:53:48.307105+0000","last_undegraded":"2023-08-09T13:53:48.307105+0000","last_fullsized":"2023-08-09T13:53:48.307105+0000","mapping_epoch":127,"log_start":"0'0","ondisk_log_start":"0'0","created":127,"last_epoch_clean":128,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:53:47.231906+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:53:47.231906+0000","last_clean_scrub_stamp":"2023-08-09T13:53:47.231906+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T18:20:43.514302+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.5","version":"0'0","reported_seq":11,"reported_epoch":128,"state":"active+clean","last_fresh":"2023-08-09T13:53:48.297085+0000","last_change":"2023-08-09T13:53:47.270618+0000","last_active":"2023-08-09T13:53:48.297085+0000","last_peered":"2023-08-09T13:53:48.297085+0000","last_clean":"2023-08-09T13:53:48.297085+0000","last_became_active":"2023-08-09T13:53:47.270323+0000","last_became_peered":"2023-08-09T13:53:47.270323+0000","last_unstale":"2023-08-09T13:53:48.297085+0000","last_undegraded":"2023-08-09T13:53:48.297085+0000","last_fullsized":"2023-08-09T13:53:48.297085+0000","mapping_epoch":126,"log_start":"0'0","ondisk_log_start":"0'0","created":126,"last_epoch_clean":127,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:53:46.216803+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:53:46.216803+0000","last_clean_scrub_stamp":"2023-08-09T13:53:46.216803+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T14:53:29.005519+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.5","version":"0'0","reported_seq":11,"reported_epoch":129,"state":"active+clean","last_fresh":"2023-08-09T13:53:49.281314+0000","last_change":"2023-08-09T13:53:48.316215+0000","last_active":"2023-08-09T13:53:49.281314+0000","last_peered":"2023-08-09T13:53:49.281314+0000","last_clean":"2023-08-09T13:53:49.281314+0000","last_became_active":"2023-08-09T13:53:48.315918+0000","last_became_peered":"2023-08-09T13:53:48.315918+0000","last_unstale":"2023-08-09T13:53:49.281314+0000","last_undegraded":"2023-08-09T13:53:49.281314+0000","last_fullsized":"2023-08-09T13:53:49.281314+0000","mapping_epoch":127,"log_start":"0'0","ondisk_log_start":"0'0","created":127,"last_epoch_clean":128,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:53:47.231906+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:53:47.231906+0000","last_clean_scrub_stamp":"2023-08-09T13:53:47.231906+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T17:16:33.909230+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.4","version":"0'0","reported_seq":11,"reported_epoch":128,"state":"active+clean","last_fresh":"2023-08-09T13:53:48.278713+0000","last_change":"2023-08-09T13:53:47.268293+0000","last_active":"2023-08-09T13:53:48.278713+0000","last_peered":"2023-08-09T13:53:48.278713+0000","last_clean":"2023-08-09T13:53:48.278713+0000","last_became_active":"2023-08-09T13:53:47.268002+0000","last_became_peered":"2023-08-09T13:53:47.268002+0000","last_unstale":"2023-08-09T13:53:48.278713+0000","last_undegraded":"2023-08-09T13:53:48.278713+0000","last_fullsized":"2023-08-09T13:53:48.278713+0000","mapping_epoch":126,"log_start":"0'0","ondisk_log_start":"0'0","created":126,"last_epoch_clean":127,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:53:46.216803+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:53:46.216803+0000","last_clean_scrub_stamp":"2023-08-09T13:53:46.216803+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T18:45:40.148027+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"17.2","version":"0'0","reported_seq":9,"reported_epoch":128,"state":"active+clean","last_fresh":"2023-08-09T13:53:48.312284+0000","last_change":"2023-08-09T13:53:48.311397+0000","last_active":"2023-08-09T13:53:48.312284+0000","last_peered":"2023-08-09T13:53:48.312284+0000","last_clean":"2023-08-09T13:53:48.312284+0000","last_became_active":"2023-08-09T13:53:48.310984+0000","last_became_peered":"2023-08-09T13:53:48.310984+0000","last_unstale":"2023-08-09T13:53:48.312284+0000","last_undegraded":"2023-08-09T13:53:48.312284+0000","last_fullsized":"2023-08-09T13:53:48.312284+0000","mapping_epoch":127,"log_start":"0'0","ondisk_log_start":"0'0","created":127,"last_epoch_clean":128,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:53:47.231906+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:53:47.231906+0000","last_clean_scrub_stamp":"2023-08-09T13:53:47.231906+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T22:47:39.839446+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"16.3","version":"0'0","reported_seq":11,"reported_epoch":128,"state":"active+clean","last_fresh":"2023-08-09T13:53:48.298474+0000","last_change":"2023-08-09T13:53:47.270868+0000","last_active":"2023-08-09T13:53:48.298474+0000","last_peered":"2023-08-09T13:53:48.298474+0000","last_clean":"2023-08-09T13:53:48.298474+0000","last_became_active":"2023-08-09T13:53:47.270486+0000","last_became_peered":"2023-08-09T13:53:47.270486+0000","last_unstale":"2023-08-09T13:53:48.298474+0000","last_undegraded":"2023-08-09T13:53:48.298474+0000","last_fullsized":"2023-08-09T13:53:48.298474+0000","mapping_epoch":126,"log_start":"0'0","ondisk_log_start":"0'0","created":126,"last_epoch_clean":127,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:53:46.216803+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:53:46.216803+0000","last_clean_scrub_stamp":"2023-08-09T13:53:46.216803+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-11T00:46:47.133310+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"17.3","version":"0'0","reported_seq":9,"reported_epoch":128,"state":"active+clean","last_fresh":"2023-08-09T13:53:48.325042+0000","last_change":"2023-08-09T13:53:48.321816+0000","last_active":"2023-08-09T13:53:48.325042+0000","last_peered":"2023-08-09T13:53:48.325042+0000","last_clean":"2023-08-09T13:53:48.325042+0000","last_became_active":"2023-08-09T13:53:48.321517+0000","last_became_peered":"2023-08-09T13:53:48.321517+0000","last_unstale":"2023-08-09T13:53:48.325042+0000","last_undegraded":"2023-08-09T13:53:48.325042+0000","last_fullsized":"2023-08-09T13:53:48.325042+0000","mapping_epoch":127,"log_start":"0'0","ondisk_log_start":"0'0","created":127,"last_epoch_clean":128,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:53:47.231906+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:53:47.231906+0000","last_clean_scrub_stamp":"2023-08-09T13:53:47.231906+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T20:39:00.369183+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"16.2","version":"0'0","reported_seq":11,"reported_epoch":128,"state":"active+clean","last_fresh":"2023-08-09T13:53:48.298754+0000","last_change":"2023-08-09T13:53:47.265197+0000","last_active":"2023-08-09T13:53:48.298754+0000","last_peered":"2023-08-09T13:53:48.298754+0000","last_clean":"2023-08-09T13:53:48.298754+0000","last_became_active":"2023-08-09T13:53:47.264692+0000","last_became_peered":"2023-08-09T13:53:47.264692+0000","last_unstale":"2023-08-09T13:53:48.298754+0000","last_undegraded":"2023-08-09T13:53:48.298754+0000","last_fullsized":"2023-08-09T13:53:48.298754+0000","mapping_epoch":126,"log_start":"0'0","ondisk_log_start":"0'0","created":126,"last_epoch_clean":127,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:53:46.216803+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:53:46.216803+0000","last_clean_scrub_stamp":"2023-08-09T13:53:46.216803+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T15:13:29.347832+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.0","version":"0'0","reported_seq":9,"reported_epoch":128,"state":"active+clean","last_fresh":"2023-08-09T13:53:48.316400+0000","last_change":"2023-08-09T13:53:48.312958+0000","last_active":"2023-08-09T13:53:48.316400+0000","last_peered":"2023-08-09T13:53:48.316400+0000","last_clean":"2023-08-09T13:53:48.316400+0000","last_became_active":"2023-08-09T13:53:48.312643+0000","last_became_peered":"2023-08-09T13:53:48.312643+0000","last_unstale":"2023-08-09T13:53:48.316400+0000","last_undegraded":"2023-08-09T13:53:48.316400+0000","last_fullsized":"2023-08-09T13:53:48.316400+0000","mapping_epoch":127,"log_start":"0'0","ondisk_log_start":"0'0","created":127,"last_epoch_clean":128,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:53:47.231906+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:53:47.231906+0000","last_clean_scrub_stamp":"2023-08-09T13:53:47.231906+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-11T00:18:27.190529+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"16.1","version":"0'0","reported_seq":13,"reported_epoch":129,"state":"active+clean","last_fresh":"2023-08-09T13:53:49.281590+0000","last_change":"2023-08-09T13:53:47.265424+0000","last_active":"2023-08-09T13:53:49.281590+0000","last_peered":"2023-08-09T13:53:49.281590+0000","last_clean":"2023-08-09T13:53:49.281590+0000","last_became_active":"2023-08-09T13:53:47.265115+0000","last_became_peered":"2023-08-09T13:53:47.265115+0000","last_unstale":"2023-08-09T13:53:49.281590+0000","last_undegraded":"2023-08-09T13:53:49.281590+0000","last_fullsized":"2023-08-09T13:53:49.281590+0000","mapping_epoch":126,"log_start":"0'0","ondisk_log_start":"0'0","created":126,"last_epoch_clean":127,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:53:46.216803+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:53:46.216803+0000","last_clean_scrub_stamp":"2023-08-09T13:53:46.216803+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T14:19:30.281030+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"17.1","version":"0'0","reported_seq":9,"reported_epoch":128,"state":"active+clean","last_fresh":"2023-08-09T13:53:48.305307+0000","last_change":"2023-08-09T13:53:48.299324+0000","last_active":"2023-08-09T13:53:48.305307+0000","last_peered":"2023-08-09T13:53:48.305307+0000","last_clean":"2023-08-09T13:53:48.305307+0000","last_became_active":"2023-08-09T13:53:48.299002+0000","last_became_peered":"2023-08-09T13:53:48.299002+0000","last_unstale":"2023-08-09T13:53:48.305307+0000","last_undegraded":"2023-08-09T13:53:48.305307+0000","last_fullsized":"2023-08-09T13:53:48.305307+0000","mapping_epoch":127,"log_start":"0'0","ondisk_log_start":"0'0","created":127,"last_epoch_clean":128,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:53:47.231906+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:53:47.231906+0000","last_clean_scrub_stamp":"2023-08-09T13:53:47.231906+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T20:17:37.885150+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.0","version":"0'0","reported_seq":11,"reported_epoch":128,"state":"active+clean","last_fresh":"2023-08-09T13:53:48.295813+0000","last_change":"2023-08-09T13:53:47.273598+0000","last_active":"2023-08-09T13:53:48.295813+0000","last_peered":"2023-08-09T13:53:48.295813+0000","last_clean":"2023-08-09T13:53:48.295813+0000","last_became_active":"2023-08-09T13:53:47.273269+0000","last_became_peered":"2023-08-09T13:53:47.273269+0000","last_unstale":"2023-08-09T13:53:48.295813+0000","last_undegraded":"2023-08-09T13:53:48.295813+0000","last_fullsized":"2023-08-09T13:53:48.295813+0000","mapping_epoch":126,"log_start":"0'0","ondisk_log_start":"0'0","created":126,"last_epoch_clean":127,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:53:46.216803+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:53:46.216803+0000","last_clean_scrub_stamp":"2023-08-09T13:53:46.216803+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T17:36:59.208559+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.10","version":"0'0","reported_seq":9,"reported_epoch":128,"state":"active+clean","last_fresh":"2023-08-09T13:53:48.315569+0000","last_change":"2023-08-09T13:53:48.313515+0000","last_active":"2023-08-09T13:53:48.315569+0000","last_peered":"2023-08-09T13:53:48.315569+0000","last_clean":"2023-08-09T13:53:48.315569+0000","last_became_active":"2023-08-09T13:53:48.312541+0000","last_became_peered":"2023-08-09T13:53:48.312541+0000","last_unstale":"2023-08-09T13:53:48.315569+0000","last_undegraded":"2023-08-09T13:53:48.315569+0000","last_fullsized":"2023-08-09T13:53:48.315569+0000","mapping_epoch":127,"log_start":"0'0","ondisk_log_start":"0'0","created":127,"last_epoch_clean":128,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:53:47.231906+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:53:47.231906+0000","last_clean_scrub_stamp":"2023-08-09T13:53:47.231906+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T16:36:31.354568+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"16.11","version":"0'0","reported_seq":11,"reported_epoch":128,"state":"active+clean","last_fresh":"2023-08-09T13:53:48.302872+0000","last_change":"2023-08-09T13:53:47.276027+0000","last_active":"2023-08-09T13:53:48.302872+0000","last_peered":"2023-08-09T13:53:48.302872+0000","last_clean":"2023-08-09T13:53:48.302872+0000","last_became_active":"2023-08-09T13:53:47.275740+0000","last_became_peered":"2023-08-09T13:53:47.275740+0000","last_unstale":"2023-08-09T13:53:48.302872+0000","last_undegraded":"2023-08-09T13:53:48.302872+0000","last_fullsized":"2023-08-09T13:53:48.302872+0000","mapping_epoch":126,"log_start":"0'0","ondisk_log_start":"0'0","created":126,"last_epoch_clean":127,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:53:46.216803+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:53:46.216803+0000","last_clean_scrub_stamp":"2023-08-09T13:53:46.216803+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-11T01:50:52.067295+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":214,"reported_epoch":128,"state":"active+clean","last_fresh":"2023-08-09T13:53:48.294940+0000","last_change":"2023-08-09T13:33:56.401406+0000","last_active":"2023-08-09T13:53:48.294940+0000","last_peered":"2023-08-09T13:53:48.294940+0000","last_clean":"2023-08-09T13:53:48.294940+0000","last_became_active":"2023-08-09T13:33:56.400487+0000","last_became_peered":"2023-08-09T13:33:56.400487+0000","last_unstale":"2023-08-09T13:53:48.294940+0000","last_undegraded":"2023-08-09T13:53:48.294940+0000","last_fullsized":"2023-08-09T13:53:48.294940+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:33:54.845869+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:33:54.845869+0000","last_clean_scrub_stamp":"2023-08-09T13:33:54.845869+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T21:20:59.655753+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"17.11","version":"0'0","reported_seq":9,"reported_epoch":128,"state":"active+clean","last_fresh":"2023-08-09T13:53:48.308285+0000","last_change":"2023-08-09T13:53:48.306830+0000","last_active":"2023-08-09T13:53:48.308285+0000","last_peered":"2023-08-09T13:53:48.308285+0000","last_clean":"2023-08-09T13:53:48.308285+0000","last_became_active":"2023-08-09T13:53:48.306545+0000","last_became_peered":"2023-08-09T13:53:48.306545+0000","last_unstale":"2023-08-09T13:53:48.308285+0000","last_undegraded":"2023-08-09T13:53:48.308285+0000","last_fullsized":"2023-08-09T13:53:48.308285+0000","mapping_epoch":127,"log_start":"0'0","ondisk_log_start":"0'0","created":127,"last_epoch_clean":128,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:53:47.231906+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:53:47.231906+0000","last_clean_scrub_stamp":"2023-08-09T13:53:47.231906+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T19:27:21.275360+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"16.10","version":"128'8","reported_seq":21,"reported_epoch":129,"state":"active+clean","last_fresh":"2023-08-09T13:53:49.281342+0000","last_change":"2023-08-09T13:53:47.276063+0000","last_active":"2023-08-09T13:53:49.281342+0000","last_peered":"2023-08-09T13:53:49.281342+0000","last_clean":"2023-08-09T13:53:49.281342+0000","last_became_active":"2023-08-09T13:53:47.275739+0000","last_became_peered":"2023-08-09T13:53:47.275739+0000","last_unstale":"2023-08-09T13:53:49.281342+0000","last_undegraded":"2023-08-09T13:53:49.281342+0000","last_fullsized":"2023-08-09T13:53:49.281342+0000","mapping_epoch":126,"log_start":"0'0","ondisk_log_start":"0'0","created":126,"last_epoch_clean":127,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:53:46.216803+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:53:46.216803+0000","last_clean_scrub_stamp":"2023-08-09T13:53:46.216803+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T14:24:59.862405+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":218,"reported_epoch":128,"state":"active+clean","last_fresh":"2023-08-09T13:53:48.302828+0000","last_change":"2023-08-09T13:33:56.401123+0000","last_active":"2023-08-09T13:53:48.302828+0000","last_peered":"2023-08-09T13:53:48.302828+0000","last_clean":"2023-08-09T13:53:48.302828+0000","last_became_active":"2023-08-09T13:33:56.399933+0000","last_became_peered":"2023-08-09T13:33:56.399933+0000","last_unstale":"2023-08-09T13:53:48.302828+0000","last_undegraded":"2023-08-09T13:53:48.302828+0000","last_fullsized":"2023-08-09T13:53:48.302828+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:33:54.845869+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:33:54.845869+0000","last_clean_scrub_stamp":"2023-08-09T13:33:54.845869+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T21:17:47.698878+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"17.12","version":"0'0","reported_seq":9,"reported_epoch":128,"state":"active+clean","last_fresh":"2023-08-09T13:53:48.314743+0000","last_change":"2023-08-09T13:53:48.312415+0000","last_active":"2023-08-09T13:53:48.314743+0000","last_peered":"2023-08-09T13:53:48.314743+0000","last_clean":"2023-08-09T13:53:48.314743+0000","last_became_active":"2023-08-09T13:53:48.311897+0000","last_became_peered":"2023-08-09T13:53:48.311897+0000","last_unstale":"2023-08-09T13:53:48.314743+0000","last_undegraded":"2023-08-09T13:53:48.314743+0000","last_fullsized":"2023-08-09T13:53:48.314743+0000","mapping_epoch":127,"log_start":"0'0","ondisk_log_start":"0'0","created":127,"last_epoch_clean":128,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:53:47.231906+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:53:47.231906+0000","last_clean_scrub_stamp":"2023-08-09T13:53:47.231906+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T16:50:01.243839+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"16.13","version":"128'2","reported_seq":15,"reported_epoch":129,"state":"active+clean","last_fresh":"2023-08-09T13:53:49.277360+0000","last_change":"2023-08-09T13:53:47.266179+0000","last_active":"2023-08-09T13:53:49.277360+0000","last_peered":"2023-08-09T13:53:49.277360+0000","last_clean":"2023-08-09T13:53:49.277360+0000","last_became_active":"2023-08-09T13:53:47.265894+0000","last_became_peered":"2023-08-09T13:53:47.265894+0000","last_unstale":"2023-08-09T13:53:49.277360+0000","last_undegraded":"2023-08-09T13:53:49.277360+0000","last_fullsized":"2023-08-09T13:53:49.277360+0000","mapping_epoch":126,"log_start":"0'0","ondisk_log_start":"0'0","created":126,"last_epoch_clean":127,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:53:46.216803+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:53:46.216803+0000","last_clean_scrub_stamp":"2023-08-09T13:53:46.216803+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T18:22:05.646686+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.2","version":"0'0","reported_seq":214,"reported_epoch":128,"state":"active+clean","last_fresh":"2023-08-09T13:53:48.292200+0000","last_change":"2023-08-09T13:33:56.402794+0000","last_active":"2023-08-09T13:53:48.292200+0000","last_peered":"2023-08-09T13:53:48.292200+0000","last_clean":"2023-08-09T13:53:48.292200+0000","last_became_active":"2023-08-09T13:33:56.402038+0000","last_became_peered":"2023-08-09T13:33:56.402038+0000","last_unstale":"2023-08-09T13:53:48.292200+0000","last_undegraded":"2023-08-09T13:53:48.292200+0000","last_fullsized":"2023-08-09T13:53:48.292200+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:33:54.845869+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:33:54.845869+0000","last_clean_scrub_stamp":"2023-08-09T13:33:54.845869+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T14:41:45.274949+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"17.13","version":"0'0","reported_seq":9,"reported_epoch":128,"state":"active+clean","last_fresh":"2023-08-09T13:53:48.315226+0000","last_change":"2023-08-09T13:53:48.313299+0000","last_active":"2023-08-09T13:53:48.315226+0000","last_peered":"2023-08-09T13:53:48.315226+0000","last_clean":"2023-08-09T13:53:48.315226+0000","last_became_active":"2023-08-09T13:53:48.312896+0000","last_became_peered":"2023-08-09T13:53:48.312896+0000","last_unstale":"2023-08-09T13:53:48.315226+0000","last_undegraded":"2023-08-09T13:53:48.315226+0000","last_fullsized":"2023-08-09T13:53:48.315226+0000","mapping_epoch":127,"log_start":"0'0","ondisk_log_start":"0'0","created":127,"last_epoch_clean":128,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:53:47.231906+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:53:47.231906+0000","last_clean_scrub_stamp":"2023-08-09T13:53:47.231906+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-11T00:23:04.961463+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"16.12","version":"0'0","reported_seq":13,"reported_epoch":129,"state":"active+clean","last_fresh":"2023-08-09T13:53:49.280916+0000","last_change":"2023-08-09T13:53:47.274711+0000","last_active":"2023-08-09T13:53:49.280916+0000","last_peered":"2023-08-09T13:53:49.280916+0000","last_clean":"2023-08-09T13:53:49.280916+0000","last_became_active":"2023-08-09T13:53:47.274348+0000","last_became_peered":"2023-08-09T13:53:47.274348+0000","last_unstale":"2023-08-09T13:53:49.280916+0000","last_undegraded":"2023-08-09T13:53:49.280916+0000","last_fullsized":"2023-08-09T13:53:49.280916+0000","mapping_epoch":126,"log_start":"0'0","ondisk_log_start":"0'0","created":126,"last_epoch_clean":127,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:53:46.216803+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:53:46.216803+0000","last_clean_scrub_stamp":"2023-08-09T13:53:46.216803+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-11T01:24:41.202067+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"1.3","version":"0'0","reported_seq":216,"reported_epoch":129,"state":"active+clean","last_fresh":"2023-08-09T13:53:49.277683+0000","last_change":"2023-08-09T13:33:55.886749+0000","last_active":"2023-08-09T13:53:49.277683+0000","last_peered":"2023-08-09T13:53:49.277683+0000","last_clean":"2023-08-09T13:53:49.277683+0000","last_became_active":"2023-08-09T13:33:55.886116+0000","last_became_peered":"2023-08-09T13:33:55.886116+0000","last_unstale":"2023-08-09T13:53:49.277683+0000","last_undegraded":"2023-08-09T13:53:49.277683+0000","last_fullsized":"2023-08-09T13:53:49.277683+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:33:54.845869+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:33:54.845869+0000","last_clean_scrub_stamp":"2023-08-09T13:33:54.845869+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T20:49:11.516353+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"118'208","reported_seq":686,"reported_epoch":128,"state":"active+clean","last_fresh":"2023-08-09T13:53:48.294615+0000","last_change":"2023-08-09T13:33:55.886345+0000","last_active":"2023-08-09T13:53:48.294615+0000","last_peered":"2023-08-09T13:53:48.294615+0000","last_clean":"2023-08-09T13:53:48.294615+0000","last_became_active":"2023-08-09T13:33:55.885781+0000","last_became_peered":"2023-08-09T13:33:55.885781+0000","last_unstale":"2023-08-09T13:53:48.294615+0000","last_undegraded":"2023-08-09T13:53:48.294615+0000","last_fullsized":"2023-08-09T13:53:48.294615+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:33:54.845869+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:33:54.845869+0000","last_clean_scrub_stamp":"2023-08-09T13:33:54.845869+0000","objects_scrubbed":0,"log_size":208,"log_dups_size":0,"ondisk_log_size":208,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T15:52:46.028204+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":907,"num_read_kb":1830,"num_write":323,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"17.14","version":"0'0","reported_seq":9,"reported_epoch":128,"state":"active+clean","last_fresh":"2023-08-09T13:53:48.315262+0000","last_change":"2023-08-09T13:53:48.313896+0000","last_active":"2023-08-09T13:53:48.315262+0000","last_peered":"2023-08-09T13:53:48.315262+0000","last_clean":"2023-08-09T13:53:48.315262+0000","last_became_active":"2023-08-09T13:53:48.312833+0000","last_became_peered":"2023-08-09T13:53:48.312833+0000","last_unstale":"2023-08-09T13:53:48.315262+0000","last_undegraded":"2023-08-09T13:53:48.315262+0000","last_fullsized":"2023-08-09T13:53:48.315262+0000","mapping_epoch":127,"log_start":"0'0","ondisk_log_start":"0'0","created":127,"last_epoch_clean":128,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:53:47.231906+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:53:47.231906+0000","last_clean_scrub_stamp":"2023-08-09T13:53:47.231906+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T22:23:52.085612+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"16.15","version":"128'3","reported_seq":14,"reported_epoch":128,"state":"active+clean","last_fresh":"2023-08-09T13:53:48.314131+0000","last_change":"2023-08-09T13:53:47.270479+0000","last_active":"2023-08-09T13:53:48.314131+0000","last_peered":"2023-08-09T13:53:48.314131+0000","last_clean":"2023-08-09T13:53:48.314131+0000","last_became_active":"2023-08-09T13:53:47.270175+0000","last_became_peered":"2023-08-09T13:53:47.270175+0000","last_unstale":"2023-08-09T13:53:48.314131+0000","last_undegraded":"2023-08-09T13:53:48.314131+0000","last_fullsized":"2023-08-09T13:53:48.314131+0000","mapping_epoch":126,"log_start":"0'0","ondisk_log_start":"0'0","created":126,"last_epoch_clean":127,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:53:46.216803+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:53:46.216803+0000","last_clean_scrub_stamp":"2023-08-09T13:53:46.216803+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T23:46:47.146096+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":224,"reported_epoch":128,"state":"active+clean","last_fresh":"2023-08-09T13:53:48.278822+0000","last_change":"2023-08-09T13:33:55.889565+0000","last_active":"2023-08-09T13:53:48.278822+0000","last_peered":"2023-08-09T13:53:48.278822+0000","last_clean":"2023-08-09T13:53:48.278822+0000","last_became_active":"2023-08-09T13:33:55.889164+0000","last_became_peered":"2023-08-09T13:33:55.889164+0000","last_unstale":"2023-08-09T13:53:48.278822+0000","last_undegraded":"2023-08-09T13:53:48.278822+0000","last_fullsized":"2023-08-09T13:53:48.278822+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:33:54.845869+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:33:54.845869+0000","last_clean_scrub_stamp":"2023-08-09T13:33:54.845869+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T23:39:41.751394+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"17.15","version":"0'0","reported_seq":9,"reported_epoch":128,"state":"active+clean","last_fresh":"2023-08-09T13:53:48.315544+0000","last_change":"2023-08-09T13:53:48.313041+0000","last_active":"2023-08-09T13:53:48.315544+0000","last_peered":"2023-08-09T13:53:48.315544+0000","last_clean":"2023-08-09T13:53:48.315544+0000","last_became_active":"2023-08-09T13:53:48.312748+0000","last_became_peered":"2023-08-09T13:53:48.312748+0000","last_unstale":"2023-08-09T13:53:48.315544+0000","last_undegraded":"2023-08-09T13:53:48.315544+0000","last_fullsized":"2023-08-09T13:53:48.315544+0000","mapping_epoch":127,"log_start":"0'0","ondisk_log_start":"0'0","created":127,"last_epoch_clean":128,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:53:47.231906+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:53:47.231906+0000","last_clean_scrub_stamp":"2023-08-09T13:53:47.231906+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T14:52:31.823210+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"16.14","version":"128'6","reported_seq":17,"reported_epoch":128,"state":"active+clean","last_fresh":"2023-08-09T13:53:48.323049+0000","last_change":"2023-08-09T13:53:47.274888+0000","last_active":"2023-08-09T13:53:48.323049+0000","last_peered":"2023-08-09T13:53:48.323049+0000","last_clean":"2023-08-09T13:53:48.323049+0000","last_became_active":"2023-08-09T13:53:47.274017+0000","last_became_peered":"2023-08-09T13:53:47.274017+0000","last_unstale":"2023-08-09T13:53:48.323049+0000","last_undegraded":"2023-08-09T13:53:48.323049+0000","last_fullsized":"2023-08-09T13:53:48.323049+0000","mapping_epoch":126,"log_start":"0'0","ondisk_log_start":"0'0","created":126,"last_epoch_clean":127,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:53:46.216803+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:53:46.216803+0000","last_clean_scrub_stamp":"2023-08-09T13:53:46.216803+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-11T00:53:15.909254+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":220,"reported_epoch":128,"state":"active+clean","last_fresh":"2023-08-09T13:53:48.289993+0000","last_change":"2023-08-09T13:33:56.404236+0000","last_active":"2023-08-09T13:53:48.289993+0000","last_peered":"2023-08-09T13:53:48.289993+0000","last_clean":"2023-08-09T13:53:48.289993+0000","last_became_active":"2023-08-09T13:33:56.402485+0000","last_became_peered":"2023-08-09T13:33:56.402485+0000","last_unstale":"2023-08-09T13:53:48.289993+0000","last_undegraded":"2023-08-09T13:53:48.289993+0000","last_fullsized":"2023-08-09T13:53:48.289993+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:33:54.845869+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:33:54.845869+0000","last_clean_scrub_stamp":"2023-08-09T13:33:54.845869+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T22:43:50.638469+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.16","version":"0'0","reported_seq":9,"reported_epoch":128,"state":"active+clean","last_fresh":"2023-08-09T13:53:48.321631+0000","last_change":"2023-08-09T13:53:48.320119+0000","last_active":"2023-08-09T13:53:48.321631+0000","last_peered":"2023-08-09T13:53:48.321631+0000","last_clean":"2023-08-09T13:53:48.321631+0000","last_became_active":"2023-08-09T13:53:48.319858+0000","last_became_peered":"2023-08-09T13:53:48.319858+0000","last_unstale":"2023-08-09T13:53:48.321631+0000","last_undegraded":"2023-08-09T13:53:48.321631+0000","last_fullsized":"2023-08-09T13:53:48.321631+0000","mapping_epoch":127,"log_start":"0'0","ondisk_log_start":"0'0","created":127,"last_epoch_clean":128,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:53:47.231906+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:53:47.231906+0000","last_clean_scrub_stamp":"2023-08-09T13:53:47.231906+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T16:42:55.642787+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"16.17","version":"0'0","reported_seq":13,"reported_epoch":129,"state":"active+clean","last_fresh":"2023-08-09T13:53:49.282026+0000","last_change":"2023-08-09T13:53:47.265046+0000","last_active":"2023-08-09T13:53:49.282026+0000","last_peered":"2023-08-09T13:53:49.282026+0000","last_clean":"2023-08-09T13:53:49.282026+0000","last_became_active":"2023-08-09T13:53:47.264705+0000","last_became_peered":"2023-08-09T13:53:47.264705+0000","last_unstale":"2023-08-09T13:53:49.282026+0000","last_undegraded":"2023-08-09T13:53:49.282026+0000","last_fullsized":"2023-08-09T13:53:49.282026+0000","mapping_epoch":126,"log_start":"0'0","ondisk_log_start":"0'0","created":126,"last_epoch_clean":127,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:53:46.216803+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:53:46.216803+0000","last_clean_scrub_stamp":"2023-08-09T13:53:46.216803+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T19:17:46.699216+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":216,"reported_epoch":129,"state":"active+clean","last_fresh":"2023-08-09T13:53:49.281443+0000","last_change":"2023-08-09T13:33:56.401781+0000","last_active":"2023-08-09T13:53:49.281443+0000","last_peered":"2023-08-09T13:53:49.281443+0000","last_clean":"2023-08-09T13:53:49.281443+0000","last_became_active":"2023-08-09T13:33:56.400336+0000","last_became_peered":"2023-08-09T13:33:56.400336+0000","last_unstale":"2023-08-09T13:53:49.281443+0000","last_undegraded":"2023-08-09T13:53:49.281443+0000","last_fullsized":"2023-08-09T13:53:49.281443+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:33:54.845869+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:33:54.845869+0000","last_clean_scrub_stamp":"2023-08-09T13:33:54.845869+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T17:53:59.742104+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"17.17","version":"0'0","reported_seq":11,"reported_epoch":129,"state":"active+clean","last_fresh":"2023-08-09T13:53:49.281833+0000","last_change":"2023-08-09T13:53:48.320410+0000","last_active":"2023-08-09T13:53:49.281833+0000","last_peered":"2023-08-09T13:53:49.281833+0000","last_clean":"2023-08-09T13:53:49.281833+0000","last_became_active":"2023-08-09T13:53:48.319554+0000","last_became_peered":"2023-08-09T13:53:48.319554+0000","last_unstale":"2023-08-09T13:53:49.281833+0000","last_undegraded":"2023-08-09T13:53:49.281833+0000","last_fullsized":"2023-08-09T13:53:49.281833+0000","mapping_epoch":127,"log_start":"0'0","ondisk_log_start":"0'0","created":127,"last_epoch_clean":128,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:53:47.231906+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:53:47.231906+0000","last_clean_scrub_stamp":"2023-08-09T13:53:47.231906+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T17:36:31.194931+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.16","version":"0'0","reported_seq":11,"reported_epoch":128,"state":"active+clean","last_fresh":"2023-08-09T13:53:48.277632+0000","last_change":"2023-08-09T13:53:47.272505+0000","last_active":"2023-08-09T13:53:48.277632+0000","last_peered":"2023-08-09T13:53:48.277632+0000","last_clean":"2023-08-09T13:53:48.277632+0000","last_became_active":"2023-08-09T13:53:47.272230+0000","last_became_peered":"2023-08-09T13:53:47.272230+0000","last_unstale":"2023-08-09T13:53:48.277632+0000","last_undegraded":"2023-08-09T13:53:48.277632+0000","last_fullsized":"2023-08-09T13:53:48.277632+0000","mapping_epoch":126,"log_start":"0'0","ondisk_log_start":"0'0","created":126,"last_epoch_clean":127,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:53:46.216803+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:53:46.216803+0000","last_clean_scrub_stamp":"2023-08-09T13:53:46.216803+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T21:45:04.615253+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":216,"reported_epoch":129,"state":"active+clean","last_fresh":"2023-08-09T13:53:49.281515+0000","last_change":"2023-08-09T13:33:55.886389+0000","last_active":"2023-08-09T13:53:49.281515+0000","last_peered":"2023-08-09T13:53:49.281515+0000","last_clean":"2023-08-09T13:53:49.281515+0000","last_became_active":"2023-08-09T13:33:55.885638+0000","last_became_peered":"2023-08-09T13:33:55.885638+0000","last_unstale":"2023-08-09T13:53:49.281515+0000","last_undegraded":"2023-08-09T13:53:49.281515+0000","last_fullsized":"2023-08-09T13:53:49.281515+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:33:54.845869+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:33:54.845869+0000","last_clean_scrub_stamp":"2023-08-09T13:33:54.845869+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T19:20:44.485269+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]}],"pool_stats":[{"poolid":17,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":16,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":38,"ondisk_log_size":38,"up":64,"acting":64,"num_store_stats":8},{"poolid":2,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":907,"num_read_kb":1830,"num_write":323,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":208,"ondisk_log_size":208,"up":2,"acting":2,"num_store_stats":2},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":8}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834575085,"num_pgs":17,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":37356,"kb_used_data":2136,"kb_used_omap":1,"kb_used_meta":35198,"kb_avail":94334484,"statfs":{"total":96636764160,"available":96598511616,"internally_reserved":0,"allocated":2187264,"data_stored":1436443,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1781,"internal_metadata":36043019},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Aug 9 13:53:27 2023","interfaces":[{"interface":"back","average":{"1min":1.145,"5min":1.167,"15min":1.222},"min":{"1min":0.623,"5min":0.61599999999999999,"15min":0.61599999999999999},"max":{"1min":1.704,"5min":1.8919999999999999,"15min":1.8919999999999999},"last":0.91600000000000004},{"interface":"front","average":{"1min":1.3140000000000001,"5min":1.2,"15min":1.232},"min":{"1min":0.83199999999999996,"5min":0.64200000000000002,"15min":0.64200000000000002},"max":{"1min":2.3399999999999999,"5min":2.3399999999999999,"15min":2.3399999999999999},"last":0.76900000000000002}]},{"osd":1,"last update":"Wed Aug 9 13:53:27 2023","interfaces":[{"interface":"back","average":{"1min":1.266,"5min":1.2509999999999999,"15min":1.262},"min":{"1min":0.81000000000000005,"5min":0.69199999999999995,"15min":0.69199999999999995},"max":{"1min":2.1989999999999998,"5min":2.1989999999999998,"15min":2.1989999999999998},"last":1.0449999999999999},{"interface":"front","average":{"1min":1.27,"5min":1.2290000000000001,"15min":1.2669999999999999},"min":{"1min":0.96299999999999997,"5min":0.59299999999999997,"15min":0.59299999999999997},"max":{"1min":1.649,"5min":1.998,"15min":1.998},"last":1.161}]},{"osd":2,"last update":"Wed Aug 9 13:53:27 2023","interfaces":[{"interface":"back","average":{"1min":1.321,"5min":1.252,"15min":1.2509999999999999},"min":{"1min":0.90900000000000003,"5min":0.75,"15min":0.65700000000000003},"max":{"1min":1.766,"5min":2.0129999999999999,"15min":2.0129999999999999},"last":1.1240000000000001},{"interface":"front","average":{"1min":1.3180000000000001,"5min":1.274,"15min":1.357},"min":{"1min":0.88800000000000001,"5min":0.58199999999999996,"15min":0.58199999999999996},"max":{"1min":2.359,"5min":2.359,"15min":2.359},"last":1.375}]},{"osd":3,"last update":"Wed Aug 9 13:53:27 2023","interfaces":[{"interface":"back","average":{"1min":1.4039999999999999,"5min":1.3240000000000001,"15min":1.339},"min":{"1min":0.89000000000000001,"5min":0.67500000000000004,"15min":0.67500000000000004},"max":{"1min":2.4380000000000002,"5min":2.4380000000000002,"15min":2.4380000000000002},"last":0.98899999999999999},{"interface":"front","average":{"1min":1.506,"5min":1.403,"15min":1.419},"min":{"1min":1.0580000000000001,"5min":0.71499999999999997,"15min":0.71499999999999997},"max":{"1min":2.6360000000000001,"5min":2.6360000000000001,"15min":2.6360000000000001},"last":1.2310000000000001}]},{"osd":4,"last update":"Wed Aug 9 13:53:27 2023","interfaces":[{"interface":"back","average":{"1min":1.2330000000000001,"5min":1.278,"15min":1.27},"min":{"1min":0.79000000000000004,"5min":0.74099999999999999,"15min":0.59499999999999997},"max":{"1min":1.635,"5min":2.097,"15min":2.097},"last":1.5429999999999999},{"interface":"front","average":{"1min":1.4259999999999999,"5min":1.3380000000000001,"15min":1.3380000000000001},"min":{"1min":0.93500000000000005,"5min":0.60799999999999998,"15min":0.60799999999999998},"max":{"1min":2.6890000000000001,"5min":2.6890000000000001,"15min":2.6890000000000001},"last":1.4359999999999999}]},{"osd":5,"last update":"Wed Aug 9 13:53:27 2023","interfaces":[{"interface":"back","average":{"1min":1.4970000000000001,"5min":1.458,"15min":1.4219999999999999},"min":{"1min":0.98899999999999999,"5min":0.65800000000000003,"15min":0.65800000000000003},"max":{"1min":2.7509999999999999,"5min":2.7509999999999999,"15min":2.7509999999999999},"last":1.5},{"interface":"front","average":{"1min":1.498,"5min":1.4390000000000001,"15min":1.4379999999999999},"min":{"1min":1.073,"5min":0.83999999999999997,"15min":0.83999999999999997},"max":{"1min":2.8639999999999999,"5min":2.8639999999999999,"15min":2.8639999999999999},"last":1.7010000000000001}]},{"osd":6,"last update":"Wed Aug 9 13:53:27 2023","interfaces":[{"interface":"back","average":{"1min":1.6339999999999999,"5min":1.544,"15min":1.5349999999999999},"min":{"1min":1.2589999999999999,"5min":0.85499999999999998,"15min":0.85499999999999998},"max":{"1min":2.7370000000000001,"5min":2.7370000000000001,"15min":2.7370000000000001},"last":1.6419999999999999},{"interface":"front","average":{"1min":1.5169999999999999,"5min":1.4590000000000001,"15min":1.474},"min":{"1min":1.1830000000000001,"5min":0.79700000000000004,"15min":0.79700000000000004},"max":{"1min":2.7850000000000001,"5min":2.7850000000000001,"15min":2.7850000000000001},"last":1.6000000000000001}]}]},{"osd":6,"up_from":13,"seq":55834575085,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":36760,"kb_used_data":1548,"kb_used_omap":1,"kb_used_meta":35198,"kb_avail":94335080,"statfs":{"total":96636764160,"available":96599121920,"internally_reserved":0,"allocated":1585152,"data_stored":844168,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1781,"internal_metadata":36043019},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Aug 9 13:53:25 2023","interfaces":[{"interface":"back","average":{"1min":1.2569999999999999,"5min":1.125,"15min":1.1499999999999999},"min":{"1min":0.90800000000000003,"5min":0.47999999999999998,"15min":0.47999999999999998},"max":{"1min":1.6020000000000001,"5min":1.6020000000000001,"15min":1.9870000000000001},"last":1.0129999999999999},{"interface":"front","average":{"1min":1.355,"5min":1.194,"15min":1.302},"min":{"1min":0.95799999999999996,"5min":0.51800000000000002,"15min":0.51800000000000002},"max":{"1min":1.8520000000000001,"5min":1.8520000000000001,"15min":28.036000000000001},"last":1.627}]},{"osd":1,"last update":"Wed Aug 9 13:53:25 2023","interfaces":[{"interface":"back","average":{"1min":1.21,"5min":1.1659999999999999,"15min":1.22},"min":{"1min":0.76500000000000001,"5min":0.48999999999999999,"15min":0.48999999999999999},"max":{"1min":1.6639999999999999,"5min":2.2320000000000002,"15min":7.742},"last":1.2430000000000001},{"interface":"front","average":{"1min":1.335,"5min":1.2569999999999999,"15min":1.278},"min":{"1min":1.012,"5min":0.64600000000000002,"15min":0.64600000000000002},"max":{"1min":1.653,"5min":2.056,"15min":2.056},"last":0.745}]},{"osd":2,"last update":"Wed Aug 9 13:53:25 2023","interfaces":[{"interface":"back","average":{"1min":1.2709999999999999,"5min":1.2629999999999999,"15min":1.319},"min":{"1min":0.63600000000000001,"5min":0.627,"15min":0.627},"max":{"1min":1.5529999999999999,"5min":2.1899999999999999,"15min":4.0640000000000001},"last":1.147},{"interface":"front","average":{"1min":1.3400000000000001,"5min":1.2589999999999999,"15min":1.3520000000000001},"min":{"1min":0.96199999999999997,"5min":0.76800000000000002,"15min":0.70999999999999996},"max":{"1min":1.7769999999999999,"5min":2.5819999999999999,"15min":13.154999999999999},"last":1.369}]},{"osd":3,"last update":"Wed Aug 9 13:53:25 2023","interfaces":[{"interface":"back","average":{"1min":1.3720000000000001,"5min":1.3169999999999999,"15min":1.4119999999999999},"min":{"1min":0.85599999999999998,"5min":0.67200000000000004,"15min":0.67200000000000004},"max":{"1min":1.665,"5min":2.2519999999999998,"15min":13.349},"last":1.0860000000000001},{"interface":"front","average":{"1min":1.3149999999999999,"5min":1.284,"15min":1.329},"min":{"1min":0.84699999999999998,"5min":0.75700000000000001,"15min":0.75700000000000001},"max":{"1min":1.74,"5min":1.8680000000000001,"15min":7.9029999999999996},"last":1.5329999999999999}]},{"osd":4,"last update":"Wed Aug 9 13:53:25 2023","interfaces":[{"interface":"back","average":{"1min":1.2569999999999999,"5min":1.276,"15min":1.2849999999999999},"min":{"1min":0.59599999999999997,"5min":0.35099999999999998,"15min":0.35099999999999998},"max":{"1min":1.6659999999999999,"5min":1.992,"15min":5.2290000000000001},"last":1.3129999999999999},{"interface":"front","average":{"1min":1.298,"5min":1.3620000000000001,"15min":1.3839999999999999},"min":{"1min":0.83099999999999996,"5min":0.54800000000000004,"15min":0.54800000000000004},"max":{"1min":1.9179999999999999,"5min":1.9179999999999999,"15min":12.1},"last":1.1830000000000001}]},{"osd":5,"last update":"Wed Aug 9 13:53:25 2023","interfaces":[{"interface":"back","average":{"1min":1.252,"5min":1.337,"15min":1.3600000000000001},"min":{"1min":0.72599999999999998,"5min":0.51000000000000001,"15min":0.51000000000000001},"max":{"1min":1.627,"5min":2.1349999999999998,"15min":8.1899999999999995},"last":1.4239999999999999},{"interface":"front","average":{"1min":1.5569999999999999,"5min":1.552,"15min":1.5920000000000001},"min":{"1min":0.67000000000000004,"5min":0.42399999999999999,"15min":0.42399999999999999},"max":{"1min":1.7969999999999999,"5min":2.2000000000000002,"15min":8.8889999999999993},"last":1.5760000000000001}]},{"osd":7,"last update":"Wed Aug 9 13:53:25 2023","interfaces":[{"interface":"back","average":{"1min":1.6060000000000001,"5min":1.5589999999999999,"15min":1.591},"min":{"1min":0.95199999999999996,"5min":0.69299999999999995,"15min":0.62},"max":{"1min":1.9790000000000001,"5min":2.4529999999999998,"15min":9.3320000000000007},"last":1.4770000000000001},{"interface":"front","average":{"1min":1.4790000000000001,"5min":1.506,"15min":1.544},"min":{"1min":0.79600000000000004,"5min":0.59899999999999998,"15min":0.59899999999999998},"max":{"1min":1.7949999999999999,"5min":2.2909999999999999,"15min":8.0399999999999991},"last":1.667}]}]},{"osd":5,"up_from":13,"seq":55834575085,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":58912,"kb_used_data":1556,"kb_used_omap":1,"kb_used_meta":57342,"kb_avail":94312928,"statfs":{"total":96636764160,"available":96576438272,"internally_reserved":0,"allocated":1593344,"data_stored":846063,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1781,"internal_metadata":58718475},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Aug 9 13:53:23 2023","interfaces":[{"interface":"back","average":{"1min":1.2629999999999999,"5min":1.26,"15min":1.21},"min":{"1min":0.83699999999999997,"5min":0.59099999999999997,"15min":0.59099999999999997},"max":{"1min":1.7929999999999999,"5min":8.9299999999999997,"15min":8.9299999999999997},"last":1.3620000000000001},{"interface":"front","average":{"1min":1.339,"5min":1.264,"15min":1.2410000000000001},"min":{"1min":0.81599999999999995,"5min":0.57799999999999996,"15min":0.57799999999999996},"max":{"1min":2.468,"5min":8.7140000000000004,"15min":8.7140000000000004},"last":1.133}]},{"osd":1,"last update":"Wed Aug 9 13:53:23 2023","interfaces":[{"interface":"back","average":{"1min":1.234,"5min":1.2190000000000001,"15min":1.2470000000000001},"min":{"1min":0.77300000000000002,"5min":0.66600000000000004,"15min":0.64100000000000001},"max":{"1min":1.9059999999999999,"5min":8.2720000000000002,"15min":8.2720000000000002},"last":1.252},{"interface":"front","average":{"1min":1.3160000000000001,"5min":1.3600000000000001,"15min":1.319},"min":{"1min":0.92300000000000004,"5min":0.73399999999999999,"15min":0.66900000000000004},"max":{"1min":1.752,"5min":9.0020000000000007,"15min":9.0020000000000007},"last":0.93700000000000006}]},{"osd":2,"last update":"Wed Aug 9 13:53:23 2023","interfaces":[{"interface":"back","average":{"1min":1.351,"5min":1.349,"15min":1.3180000000000001},"min":{"1min":0.97299999999999998,"5min":0.753,"15min":0.61599999999999999},"max":{"1min":2.4009999999999998,"5min":8.6050000000000004,"15min":8.6050000000000004},"last":1.2290000000000001},{"interface":"front","average":{"1min":1.3560000000000001,"5min":1.3660000000000001,"15min":1.3520000000000001},"min":{"1min":0.98999999999999999,"5min":0.78300000000000003,"15min":0.73999999999999999},"max":{"1min":1.716,"5min":8.4149999999999991,"15min":8.4149999999999991},"last":1.7190000000000001}]},{"osd":3,"last update":"Wed Aug 9 13:53:23 2023","interfaces":[{"interface":"back","average":{"1min":1.341,"5min":1.4359999999999999,"15min":1.3899999999999999},"min":{"1min":0.97199999999999998,"5min":0.72599999999999998,"15min":0.59799999999999998},"max":{"1min":1.9670000000000001,"5min":9.0470000000000006,"15min":9.0470000000000006},"last":1.0700000000000001},{"interface":"front","average":{"1min":1.5660000000000001,"5min":1.4930000000000001,"15min":1.4450000000000001},"min":{"1min":1.115,"5min":0.63,"15min":0.63},"max":{"1min":2.5819999999999999,"5min":8.7669999999999995,"15min":8.7669999999999995},"last":1.518}]},{"osd":4,"last update":"Wed Aug 9 13:53:23 2023","interfaces":[{"interface":"back","average":{"1min":1.4490000000000001,"5min":1.4119999999999999,"15min":1.341},"min":{"1min":0.97099999999999997,"5min":0.54300000000000004,"15min":0.54300000000000004},"max":{"1min":2.181,"5min":8.4909999999999997,"15min":8.4909999999999997},"last":1.4219999999999999},{"interface":"front","average":{"1min":1.476,"5min":1.5009999999999999,"15min":1.4199999999999999},"min":{"1min":0.93400000000000005,"5min":0.68899999999999995,"15min":0.68899999999999995},"max":{"1min":2.1040000000000001,"5min":9.4589999999999996,"15min":9.4589999999999996},"last":1.161}]},{"osd":6,"last update":"Wed Aug 9 13:53:23 2023","interfaces":[{"interface":"back","average":{"1min":1.5,"5min":1.6299999999999999,"15min":1.4730000000000001},"min":{"1min":0.94399999999999995,"5min":0.51000000000000001,"15min":0.51000000000000001},"max":{"1min":2.6539999999999999,"5min":17.407,"15min":17.407},"last":1.3140000000000001},{"interface":"front","average":{"1min":1.5640000000000001,"5min":1.6659999999999999,"15min":1.538},"min":{"1min":0.89500000000000002,"5min":0.71199999999999997,"15min":0.71199999999999997},"max":{"1min":2.415,"5min":16.356000000000002,"15min":16.356000000000002},"last":1.6180000000000001}]},{"osd":7,"last update":"Wed Aug 9 13:53:23 2023","interfaces":[{"interface":"back","average":{"1min":1.619,"5min":1.6699999999999999,"15min":1.5940000000000001},"min":{"1min":0.96399999999999997,"5min":0.74299999999999999,"15min":0.74299999999999999},"max":{"1min":2.3740000000000001,"5min":9.9239999999999995,"15min":9.9239999999999995},"last":1.5700000000000001},{"interface":"front","average":{"1min":1.7250000000000001,"5min":1.748,"15min":1.6539999999999999},"min":{"1min":1.1819999999999999,"5min":0.65600000000000003,"15min":0.65600000000000003},"max":{"1min":2.754,"5min":10.789,"15min":10.789},"last":1.6759999999999999}]}]},{"osd":4,"up_from":13,"seq":55834575085,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32668,"kb_used_data":1548,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":94339172,"statfs":{"total":96636764160,"available":96603312128,"internally_reserved":0,"allocated":1585152,"data_stored":845459,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1781,"internal_metadata":31848715},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Aug 9 13:53:26 2023","interfaces":[{"interface":"back","average":{"1min":1.2190000000000001,"5min":1.1779999999999999,"15min":1.2010000000000001},"min":{"1min":0.76300000000000001,"5min":0.66600000000000004,"15min":0.66600000000000004},"max":{"1min":1.556,"5min":2.524,"15min":2.524},"last":0.81299999999999994},{"interface":"front","average":{"1min":1.218,"5min":1.196,"15min":1.2470000000000001},"min":{"1min":0.82599999999999996,"5min":0.62,"15min":0.62},"max":{"1min":1.6930000000000001,"5min":1.7529999999999999,"15min":1.863},"last":0.96199999999999997}]},{"osd":1,"last update":"Wed Aug 9 13:53:26 2023","interfaces":[{"interface":"back","average":{"1min":1.2529999999999999,"5min":1.25,"15min":1.294},"min":{"1min":0.97199999999999998,"5min":0.70999999999999996,"15min":0.67800000000000005},"max":{"1min":1.7430000000000001,"5min":1.827,"15min":1.8480000000000001},"last":1.1819999999999999},{"interface":"front","average":{"1min":1.2110000000000001,"5min":1.2150000000000001,"15min":1.242},"min":{"1min":0.85899999999999999,"5min":0.68899999999999995,"15min":0.68899999999999995},"max":{"1min":1.5649999999999999,"5min":1.9330000000000001,"15min":1.9330000000000001},"last":1.1379999999999999}]},{"osd":2,"last update":"Wed Aug 9 13:53:26 2023","interfaces":[{"interface":"back","average":{"1min":1.2370000000000001,"5min":1.2390000000000001,"15min":1.264},"min":{"1min":0.91800000000000004,"5min":0.69999999999999996,"15min":0.621},"max":{"1min":1.52,"5min":2.6429999999999998,"15min":2.6429999999999998},"last":1.444},{"interface":"front","average":{"1min":1.3979999999999999,"5min":1.3260000000000001,"15min":1.365},"min":{"1min":1.0429999999999999,"5min":0.76000000000000001,"15min":0.76000000000000001},"max":{"1min":1.6839999999999999,"5min":2.0910000000000002,"15min":2.0910000000000002},"last":1.079}]},{"osd":3,"last update":"Wed Aug 9 13:53:26 2023","interfaces":[{"interface":"back","average":{"1min":1.333,"5min":1.304,"15min":1.2969999999999999},"min":{"1min":1.03,"5min":0.64700000000000002,"15min":0.64700000000000002},"max":{"1min":1.5700000000000001,"5min":2.6099999999999999,"15min":2.6099999999999999},"last":1.0149999999999999},{"interface":"front","average":{"1min":1.446,"5min":1.375,"15min":1.383},"min":{"1min":1.0149999999999999,"5min":0.86299999999999999,"15min":0.86299999999999999},"max":{"1min":1.7450000000000001,"5min":2.7629999999999999,"15min":2.7629999999999999},"last":1.3180000000000001}]},{"osd":5,"last update":"Wed Aug 9 13:53:26 2023","interfaces":[{"interface":"back","average":{"1min":1.379,"5min":1.345,"15min":1.3380000000000001},"min":{"1min":0.94599999999999995,"5min":0.67300000000000004,"15min":0.63200000000000001},"max":{"1min":1.821,"5min":2.0329999999999999,"15min":2.0329999999999999},"last":1.387},{"interface":"front","average":{"1min":1.333,"5min":1.345,"15min":1.3149999999999999},"min":{"1min":1.038,"5min":0.65600000000000003,"15min":0.65600000000000003},"max":{"1min":1.7030000000000001,"5min":2.7090000000000001,"15min":2.7090000000000001},"last":1.4970000000000001}]},{"osd":6,"last update":"Wed Aug 9 13:53:26 2023","interfaces":[{"interface":"back","average":{"1min":1.47,"5min":1.407,"15min":1.341},"min":{"1min":1.099,"5min":0.84099999999999997,"15min":0.59099999999999997},"max":{"1min":1.722,"5min":2.8559999999999999,"15min":2.8559999999999999},"last":1.74},{"interface":"front","average":{"1min":1.5549999999999999,"5min":1.52,"15min":1.51},"min":{"1min":1.1759999999999999,"5min":0.52600000000000002,"15min":0.52600000000000002},"max":{"1min":1.917,"5min":2.2719999999999998,"15min":2.2719999999999998},"last":1.573}]},{"osd":7,"last update":"Wed Aug 9 13:53:26 2023","interfaces":[{"interface":"back","average":{"1min":1.5069999999999999,"5min":1.48,"15min":1.474},"min":{"1min":1.331,"5min":0.53100000000000003,"15min":0.53100000000000003},"max":{"1min":1.778,"5min":2.8050000000000002,"15min":2.8050000000000002},"last":1.776},{"interface":"front","average":{"1min":1.6180000000000001,"5min":1.5609999999999999,"15min":1.5229999999999999},"min":{"1min":1.3120000000000001,"5min":0.78700000000000003,"15min":0.74099999999999999},"max":{"1min":1.8080000000000001,"5min":3.415,"15min":3.415},"last":1.665}]}]},{"osd":3,"up_from":13,"seq":55834575086,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":36776,"kb_used_data":1564,"kb_used_omap":1,"kb_used_meta":35198,"kb_avail":94335064,"statfs":{"total":96636764160,"available":96599105536,"internally_reserved":0,"allocated":1601536,"data_stored":847141,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1781,"internal_metadata":36043019},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Aug 9 13:53:27 2023","interfaces":[{"interface":"back","average":{"1min":0.98499999999999999,"5min":0.85499999999999998,"15min":0.90200000000000002},"min":{"1min":0.56299999999999994,"5min":0.43099999999999999,"15min":0.27200000000000002},"max":{"1min":1.5660000000000001,"5min":1.589,"15min":1.8500000000000001},"last":0.63900000000000001},{"interface":"front","average":{"1min":1.129,"5min":1.022,"15min":1.101},"min":{"1min":0.73099999999999998,"5min":0.36299999999999999,"15min":0.36299999999999999},"max":{"1min":1.5580000000000001,"5min":1.6910000000000001,"15min":2.0590000000000002},"last":0.91400000000000003}]},{"osd":1,"last update":"Wed Aug 9 13:53:27 2023","interfaces":[{"interface":"back","average":{"1min":0.98899999999999999,"5min":0.94099999999999995,"15min":1.0169999999999999},"min":{"1min":0.62,"5min":0.51500000000000001,"15min":0.496},"max":{"1min":1.909,"5min":1.9410000000000001,"15min":2.5649999999999999},"last":0.98799999999999999},{"interface":"front","average":{"1min":1.1699999999999999,"5min":1.0800000000000001,"15min":1.1719999999999999},"min":{"1min":0.65000000000000002,"5min":0.48399999999999999,"15min":0.48399999999999999},"max":{"1min":1.488,"5min":1.762,"15min":2.6760000000000002},"last":0.69599999999999995}]},{"osd":2,"last update":"Wed Aug 9 13:53:27 2023","interfaces":[{"interface":"back","average":{"1min":1.1819999999999999,"5min":1.081,"15min":1.1850000000000001},"min":{"1min":0.47899999999999998,"5min":0.42999999999999999,"15min":0.42999999999999999},"max":{"1min":1.661,"5min":1.6879999999999999,"15min":1.913},"last":0.84399999999999997},{"interface":"front","average":{"1min":1.3420000000000001,"5min":1.2450000000000001,"15min":1.3149999999999999},"min":{"1min":0.60499999999999998,"5min":0.52400000000000002,"15min":0.52300000000000002},"max":{"1min":1.78,"5min":1.9399999999999999,"15min":2.0259999999999998},"last":0.74299999999999999}]},{"osd":4,"last update":"Wed Aug 9 13:53:27 2023","interfaces":[{"interface":"back","average":{"1min":1.341,"5min":1.2090000000000001,"15min":1.2709999999999999},"min":{"1min":0.78700000000000003,"5min":0.54900000000000004,"15min":0.54900000000000004},"max":{"1min":1.7350000000000001,"5min":1.742,"15min":1.8160000000000001},"last":1.0740000000000001},{"interface":"front","average":{"1min":1.4590000000000001,"5min":1.3360000000000001,"15min":1.4059999999999999},"min":{"1min":1.026,"5min":0.67600000000000005,"15min":0.628},"max":{"1min":1.819,"5min":2.782,"15min":2.782},"last":1.2589999999999999}]},{"osd":5,"last update":"Wed Aug 9 13:53:27 2023","interfaces":[{"interface":"back","average":{"1min":1.5209999999999999,"5min":1.3979999999999999,"15min":1.4790000000000001},"min":{"1min":0.92300000000000004,"5min":0.67600000000000005,"15min":0.67600000000000005},"max":{"1min":1.8320000000000001,"5min":2.0489999999999999,"15min":2.254},"last":1.1599999999999999},{"interface":"front","average":{"1min":1.49,"5min":1.4039999999999999,"15min":1.488},"min":{"1min":0.98699999999999999,"5min":0.66000000000000003,"15min":0.65900000000000003},"max":{"1min":1.774,"5min":2.3069999999999999,"15min":2.3069999999999999},"last":1.2270000000000001}]},{"osd":6,"last update":"Wed Aug 9 13:53:27 2023","interfaces":[{"interface":"back","average":{"1min":1.3839999999999999,"5min":1.377,"15min":1.452},"min":{"1min":0.84799999999999998,"5min":0.73499999999999999,"15min":0.68200000000000005},"max":{"1min":1.8640000000000001,"5min":2.8340000000000001,"15min":2.8340000000000001},"last":1.425},{"interface":"front","average":{"1min":1.5620000000000001,"5min":1.4730000000000001,"15min":1.5569999999999999},"min":{"1min":1.028,"5min":0.64100000000000001,"15min":0.64100000000000001},"max":{"1min":1.9370000000000001,"5min":2.9060000000000001,"15min":2.9060000000000001},"last":1.3440000000000001}]},{"osd":7,"last update":"Wed Aug 9 13:53:27 2023","interfaces":[{"interface":"back","average":{"1min":1.603,"5min":1.5129999999999999,"15min":1.605},"min":{"1min":1.218,"5min":0.624,"15min":0.624},"max":{"1min":2.109,"5min":2.46,"15min":2.46},"last":1.45},{"interface":"front","average":{"1min":1.464,"5min":1.462,"15min":1.552},"min":{"1min":0.71199999999999997,"5min":0.71199999999999997,"15min":0.64200000000000002},"max":{"1min":2.0230000000000001,"5min":2.7200000000000002,"15min":2.7200000000000002},"last":1.1970000000000001}]}]},{"osd":2,"up_from":13,"seq":55834575085,"num_pgs":12,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":40860,"kb_used_data":1548,"kb_used_omap":1,"kb_used_meta":39294,"kb_avail":94330980,"statfs":{"total":96636764160,"available":96594923520,"internally_reserved":0,"allocated":1585152,"data_stored":845459,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1781,"internal_metadata":40237323},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":7,"apply_latency_ms":7,"commit_latency_ns":7000000,"apply_latency_ns":7000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Aug 9 13:53:24 2023","interfaces":[{"interface":"back","average":{"1min":0.94999999999999996,"5min":0.89400000000000002,"15min":0.95099999999999996},"min":{"1min":0.40300000000000002,"5min":0.35499999999999998,"15min":0.35499999999999998},"max":{"1min":1.524,"5min":1.5760000000000001,"15min":1.752},"last":1.0780000000000001},{"interface":"front","average":{"1min":1.0760000000000001,"5min":0.97099999999999997,"15min":1},"min":{"1min":0.53800000000000003,"5min":0.38200000000000001,"15min":0.374},"max":{"1min":1.75,"5min":1.75,"15min":2.3929999999999998},"last":1.028}]},{"osd":1,"last update":"Wed Aug 9 13:53:24 2023","interfaces":[{"interface":"back","average":{"1min":1.1699999999999999,"5min":1.123,"15min":1.1779999999999999},"min":{"1min":0.499,"5min":0.39400000000000002,"15min":0.39400000000000002},"max":{"1min":1.587,"5min":1.77,"15min":1.8420000000000001},"last":1.254},{"interface":"front","average":{"1min":1.345,"5min":1.1899999999999999,"15min":1.3069999999999999},"min":{"1min":0.63400000000000001,"5min":0.435,"15min":0.435},"max":{"1min":1.643,"5min":1.8999999999999999,"15min":2.4870000000000001},"last":1.1240000000000001}]},{"osd":3,"last update":"Wed Aug 9 13:53:24 2023","interfaces":[{"interface":"back","average":{"1min":1.056,"5min":0.96899999999999997,"15min":1.038},"min":{"1min":0.57399999999999995,"5min":0.41399999999999998,"15min":0.38900000000000001},"max":{"1min":1.508,"5min":1.7130000000000001,"15min":1.7609999999999999},"last":1.1439999999999999},{"interface":"front","average":{"1min":1.2989999999999999,"5min":1.175,"15min":1.2909999999999999},"min":{"1min":0.45500000000000002,"5min":0.45500000000000002,"15min":0.45500000000000002},"max":{"1min":1.7509999999999999,"5min":1.835,"15min":8.8130000000000006},"last":1.3400000000000001}]},{"osd":4,"last update":"Wed Aug 9 13:53:24 2023","interfaces":[{"interface":"back","average":{"1min":1.448,"5min":1.377,"15min":1.444},"min":{"1min":0.622,"5min":0.622,"15min":0.622},"max":{"1min":1.7729999999999999,"5min":2.1640000000000001,"15min":2.1640000000000001},"last":1.4039999999999999},{"interface":"front","average":{"1min":1.415,"5min":1.3540000000000001,"15min":1.387},"min":{"1min":0.70399999999999996,"5min":0.64800000000000002,"15min":0.64800000000000002},"max":{"1min":1.8069999999999999,"5min":2.472,"15min":2.472},"last":1.3839999999999999}]},{"osd":5,"last update":"Wed Aug 9 13:53:24 2023","interfaces":[{"interface":"back","average":{"1min":1.5369999999999999,"5min":1.429,"15min":1.456},"min":{"1min":0.75,"5min":0.71199999999999997,"15min":0.59999999999999998},"max":{"1min":2.0600000000000001,"5min":2.4900000000000002,"15min":2.5840000000000001},"last":1.282},{"interface":"front","average":{"1min":1.5289999999999999,"5min":1.331,"15min":1.3520000000000001},"min":{"1min":0.73699999999999999,"5min":0.73699999999999999,"15min":0.56799999999999995},"max":{"1min":2.3519999999999999,"5min":2.3519999999999999,"15min":3.5659999999999998},"last":1.4450000000000001}]},{"osd":6,"last update":"Wed Aug 9 13:53:24 2023","interfaces":[{"interface":"back","average":{"1min":1.5409999999999999,"5min":1.383,"15min":1.4299999999999999},"min":{"1min":0.77600000000000002,"5min":0.77600000000000002,"15min":0.73799999999999999},"max":{"1min":2.5110000000000001,"5min":2.5110000000000001,"15min":3.5099999999999998},"last":1.3129999999999999},{"interface":"front","average":{"1min":1.6120000000000001,"5min":1.4790000000000001,"15min":1.54},"min":{"1min":0.66100000000000003,"5min":0.66100000000000003,"15min":0.66100000000000003},"max":{"1min":2.3079999999999998,"5min":2.3769999999999998,"15min":2.3769999999999998},"last":1.5489999999999999}]},{"osd":7,"last update":"Wed Aug 9 13:53:24 2023","interfaces":[{"interface":"back","average":{"1min":1.6100000000000001,"5min":1.474,"15min":1.5309999999999999},"min":{"1min":0.79200000000000004,"5min":0.78800000000000003,"15min":0.64600000000000002},"max":{"1min":2.1459999999999999,"5min":2.1970000000000001,"15min":2.5249999999999999},"last":1.492},{"interface":"front","average":{"1min":1.5740000000000001,"5min":1.4550000000000001,"15min":1.504},"min":{"1min":0.67900000000000005,"5min":0.66100000000000003,"15min":0.66100000000000003},"max":{"1min":2.4500000000000002,"5min":2.4500000000000002,"15min":3.609},"last":1.589}]}]},{"osd":1,"up_from":13,"seq":55834575085,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":47208,"kb_used_data":2136,"kb_used_omap":1,"kb_used_meta":45054,"kb_avail":94324632,"statfs":{"total":96636764160,"available":96588423168,"internally_reserved":0,"allocated":2187264,"data_stored":1435951,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1781,"internal_metadata":46135563},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Aug 9 13:53:39 2023","interfaces":[{"interface":"back","average":{"1min":0.96799999999999997,"5min":0.90800000000000003,"15min":0.92900000000000005},"min":{"1min":0.69199999999999995,"5min":0.45800000000000002,"15min":0.45800000000000002},"max":{"1min":1.5349999999999999,"5min":1.7110000000000001,"15min":2.1080000000000001},"last":1.004},{"interface":"front","average":{"1min":1.0649999999999999,"5min":1.0289999999999999,"15min":1.097},"min":{"1min":0.71399999999999997,"5min":0.40699999999999997,"15min":0.40699999999999997},"max":{"1min":1.5469999999999999,"5min":2.5070000000000001,"15min":2.5070000000000001},"last":0.94099999999999995}]},{"osd":2,"last update":"Wed Aug 9 13:53:39 2023","interfaces":[{"interface":"back","average":{"1min":1.0389999999999999,"5min":0.97299999999999998,"15min":0.99299999999999999},"min":{"1min":0.81399999999999995,"5min":0.41999999999999998,"15min":0.41099999999999998},"max":{"1min":1.476,"5min":1.798,"15min":1.907},"last":1.0700000000000001},{"interface":"front","average":{"1min":1.125,"5min":1.0880000000000001,"15min":1.169},"min":{"1min":0.60999999999999999,"5min":0.50700000000000001,"15min":0.50700000000000001},"max":{"1min":1.579,"5min":2.1280000000000001,"15min":2.145},"last":1.458}]},{"osd":3,"last update":"Wed Aug 9 13:53:39 2023","interfaces":[{"interface":"back","average":{"1min":1.2989999999999999,"5min":1.208,"15min":1.3129999999999999},"min":{"1min":0.73199999999999998,"5min":0.54600000000000004,"15min":0.54400000000000004},"max":{"1min":1.702,"5min":2.0270000000000001,"15min":2.3210000000000002},"last":1.6819999999999999},{"interface":"front","average":{"1min":1.1839999999999999,"5min":1.1659999999999999,"15min":1.24},"min":{"1min":0.92500000000000004,"5min":0.48299999999999998,"15min":0.45300000000000001},"max":{"1min":1.47,"5min":2.3959999999999999,"15min":2.7669999999999999},"last":1.4950000000000001}]},{"osd":4,"last update":"Wed Aug 9 13:53:39 2023","interfaces":[{"interface":"back","average":{"1min":1.4099999999999999,"5min":1.3540000000000001,"15min":1.4059999999999999},"min":{"1min":1.2330000000000001,"5min":0.79800000000000004,"15min":0.58099999999999996},"max":{"1min":1.6659999999999999,"5min":2.0739999999999998,"15min":2.4089999999999998},"last":1.1619999999999999},{"interface":"front","average":{"1min":1.421,"5min":1.3180000000000001,"15min":1.3440000000000001},"min":{"1min":1.0509999999999999,"5min":0.71099999999999997,"15min":0.70099999999999996},"max":{"1min":1.7589999999999999,"5min":2.0019999999999998,"15min":2.0019999999999998},"last":1.4219999999999999}]},{"osd":5,"last update":"Wed Aug 9 13:53:39 2023","interfaces":[{"interface":"back","average":{"1min":1.4530000000000001,"5min":1.3300000000000001,"15min":1.377},"min":{"1min":1.1899999999999999,"5min":0.75900000000000001,"15min":0.75900000000000001},"max":{"1min":1.8129999999999999,"5min":1.9419999999999999,"15min":2.294},"last":1.6559999999999999},{"interface":"front","average":{"1min":1.518,"5min":1.409,"15min":1.502},"min":{"1min":0.94599999999999995,"5min":0.78400000000000003,"15min":0.78400000000000003},"max":{"1min":1.7829999999999999,"5min":1.863,"15min":2.4649999999999999},"last":1.2390000000000001}]},{"osd":6,"last update":"Wed Aug 9 13:53:39 2023","interfaces":[{"interface":"back","average":{"1min":1.474,"5min":1.462,"15min":1.5109999999999999},"min":{"1min":0.96899999999999997,"5min":0.96599999999999997,"15min":0.67400000000000004},"max":{"1min":1.8720000000000001,"5min":2.23,"15min":2.5270000000000001},"last":1.7210000000000001},{"interface":"front","average":{"1min":1.554,"5min":1.4419999999999999,"15min":1.5009999999999999},"min":{"1min":1.1399999999999999,"5min":0.73599999999999999,"15min":0.73599999999999999},"max":{"1min":1.946,"5min":2.3359999999999999,"15min":2.3359999999999999},"last":1.5289999999999999}]},{"osd":7,"last update":"Wed Aug 9 13:53:39 2023","interfaces":[{"interface":"back","average":{"1min":1.6579999999999999,"5min":1.4950000000000001,"15min":1.6020000000000001},"min":{"1min":1.3009999999999999,"5min":0.72099999999999997,"15min":0.72099999999999997},"max":{"1min":2.2040000000000002,"5min":2.4340000000000002,"15min":2.5790000000000002},"last":1.5760000000000001},{"interface":"front","average":{"1min":1.5860000000000001,"5min":1.526,"15min":1.5980000000000001},"min":{"1min":1.0900000000000001,"5min":0.85099999999999998,"15min":0.85099999999999998},"max":{"1min":1.9970000000000001,"5min":2.375,"15min":2.6509999999999998},"last":1.6080000000000001}]}]},{"osd":0,"up_from":13,"seq":55834575085,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":40876,"kb_used_data":1564,"kb_used_omap":1,"kb_used_meta":39294,"kb_avail":94330964,"statfs":{"total":96636764160,"available":96594907136,"internally_reserved":0,"allocated":1601536,"data_stored":845822,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1781,"internal_metadata":40237323},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Wed Aug 9 13:53:30 2023","interfaces":[{"interface":"back","average":{"1min":0.89400000000000002,"5min":0.92700000000000005,"15min":0.92200000000000004},"min":{"1min":0.504,"5min":0.32500000000000001,"15min":0.32500000000000001},"max":{"1min":1.1639999999999999,"5min":1.6519999999999999,"15min":1.756},"last":0.82399999999999995},{"interface":"front","average":{"1min":1.1439999999999999,"5min":1.101,"15min":1.111},"min":{"1min":0.63300000000000001,"5min":0.39200000000000002,"15min":0.39200000000000002},"max":{"1min":1.7010000000000001,"5min":1.9019999999999999,"15min":1.9019999999999999},"last":1.385}]},{"osd":2,"last update":"Wed Aug 9 13:53:30 2023","interfaces":[{"interface":"back","average":{"1min":1.0760000000000001,"5min":1.0429999999999999,"15min":1.119},"min":{"1min":0.55700000000000005,"5min":0.433,"15min":0.433},"max":{"1min":1.6140000000000001,"5min":1.786,"15min":1.786},"last":1.202},{"interface":"front","average":{"1min":1.159,"5min":1.127,"15min":1.1799999999999999},"min":{"1min":0.45800000000000002,"5min":0.439,"15min":0.439},"max":{"1min":1.929,"5min":2.0129999999999999,"15min":2.0129999999999999},"last":1.1779999999999999}]},{"osd":3,"last update":"Wed Aug 9 13:53:30 2023","interfaces":[{"interface":"back","average":{"1min":1.117,"5min":1.0349999999999999,"15min":1.097},"min":{"1min":0.48299999999999998,"5min":0.48299999999999998,"15min":0.46300000000000002},"max":{"1min":1.7529999999999999,"5min":1.7529999999999999,"15min":1.881},"last":0.61199999999999999},{"interface":"front","average":{"1min":1.298,"5min":1.2410000000000001,"15min":1.304},"min":{"1min":0.69299999999999995,"5min":0.50800000000000001,"15min":0.50800000000000001},"max":{"1min":1.708,"5min":2.347,"15min":2.4649999999999999},"last":1.101}]},{"osd":4,"last update":"Wed Aug 9 13:53:30 2023","interfaces":[{"interface":"back","average":{"1min":1.3500000000000001,"5min":1.3,"15min":1.341},"min":{"1min":0.69699999999999995,"5min":0.69699999999999995,"15min":0.69699999999999995},"max":{"1min":1.8460000000000001,"5min":1.9630000000000001,"15min":2.2309999999999999},"last":1.157},{"interface":"front","average":{"1min":1.4570000000000001,"5min":1.4079999999999999,"15min":1.4490000000000001},"min":{"1min":0.83299999999999996,"5min":0.82699999999999996,"15min":0.77600000000000002},"max":{"1min":2.012,"5min":2.3540000000000001,"15min":2.5390000000000001},"last":1.27}]},{"osd":5,"last update":"Wed Aug 9 13:53:30 2023","interfaces":[{"interface":"back","average":{"1min":1.526,"5min":1.4379999999999999,"15min":1.4670000000000001},"min":{"1min":0.871,"5min":0.66300000000000003,"15min":0.66300000000000003},"max":{"1min":2.298,"5min":2.6669999999999998,"15min":2.6669999999999998},"last":1.45},{"interface":"front","average":{"1min":1.4339999999999999,"5min":1.3720000000000001,"15min":1.4430000000000001},"min":{"1min":0.97999999999999998,"5min":0.70099999999999996,"15min":0.629},"max":{"1min":1.9710000000000001,"5min":1.984,"15min":2.1499999999999999},"last":1.296}]},{"osd":6,"last update":"Wed Aug 9 13:53:30 2023","interfaces":[{"interface":"back","average":{"1min":1.5529999999999999,"5min":1.5029999999999999,"15min":1.542},"min":{"1min":0.84599999999999997,"5min":0.73599999999999999,"15min":0.73599999999999999},"max":{"1min":1.954,"5min":2.4670000000000001,"15min":2.7250000000000001},"last":1.3600000000000001},{"interface":"front","average":{"1min":1.6100000000000001,"5min":1.548,"15min":1.5680000000000001},"min":{"1min":0.73899999999999999,"5min":0.73899999999999999,"15min":0.73899999999999999},"max":{"1min":2.2109999999999999,"5min":2.3199999999999998,"15min":2.399},"last":1.554}]},{"osd":7,"last update":"Wed Aug 9 13:53:30 2023","interfaces":[{"interface":"back","average":{"1min":1.704,"5min":1.589,"15min":1.629},"min":{"1min":1.0820000000000001,"5min":0.72299999999999998,"15min":0.72299999999999998},"max":{"1min":2.234,"5min":2.3900000000000001,"15min":2.3900000000000001},"last":1.504},{"interface":"front","average":{"1min":1.5649999999999999,"5min":1.5069999999999999,"15min":1.5489999999999999},"min":{"1min":0.85099999999999998,"5min":0.79200000000000004,"15min":0.76700000000000002},"max":{"1min":2.0329999999999999,"5min":2.274,"15min":2.274},"last":1.234}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":1724,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":124,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1682,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":604,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":70,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":616,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-08-09T13:53:52.822 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-08-09T13:53:53.091 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:53.088+0000 7f3fcd234640 1 Processor -- start 2023-08-09T13:53:53.092 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:53.089+0000 7f3fcd234640 1 -- start start 2023-08-09T13:53:53.094 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:53.090+0000 7f3fcd234640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f3fc806c020 0x7f3fc815c5c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:53:53.094 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:53.090+0000 7f3fcd234640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f3fc815cb00 0x7f3fc80c8e80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:53:53.094 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:53.090+0000 7f3fcd234640 1 -- --> v1:172.21.15.43:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3fc806c600 con 0x7f3fc814f940 2023-08-09T13:53:53.094 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:53.090+0000 7f3fcd234640 1 -- --> v2:172.21.15.43:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3fc815cf90 con 0x7f3fc815cb00 2023-08-09T13:53:53.095 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:53.090+0000 7f3fcd234640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3fc806c8e0 con 0x7f3fc806c020 2023-08-09T13:53:53.095 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:53.090+0000 7f3fc759e640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f3fc815cb00 0x7f3fc80c8e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:53:53.095 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:53.090+0000 7f3fc6d9d640 1 --1- >> v1:172.21.15.43:6789/0 conn(0x7f3fc814f940 0x7f3fc80ccb20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.43:6789/0 says I am v1:172.21.15.43:53618/0 (socket says 172.21.15.43:53618) 2023-08-09T13:53:53.095 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:53.090+0000 7f3fc759e640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f3fc815cb00 0x7f3fc80c8e80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:56390/0 (socket says 172.21.15.43:56390) 2023-08-09T13:53:53.096 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:53.090+0000 7f3fc6d9d640 1 -- 172.21.15.43:0/1162840284 learned_addr learned my addr 172.21.15.43:0/1162840284 (peer_addr_for_me v1:172.21.15.43:0/0) 2023-08-09T13:53:53.096 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:53.091+0000 7f3fc759e640 1 -- 172.21.15.43:0/1162840284 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f3fc806c020 msgr2=0x7f3fc815c5c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-09T13:53:53.096 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:53.091+0000 7f3fc659c640 1 --2- 172.21.15.43:0/1162840284 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f3fc806c020 0x7f3fc815c5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:53:53.097 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:53.091+0000 7f3fc5d9b640 1 -- 172.21.15.43:0/1162840284 <== mon.0 v1:172.21.15.43:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 224668627 0 0) 0x7f3fc806c600 con 0x7f3fc814f940 2023-08-09T13:53:53.097 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:53.091+0000 7f3fc759e640 1 --2- 172.21.15.43:0/1162840284 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f3fc806c020 0x7f3fc815c5c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:53:53.097 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:53.091+0000 7f3fc759e640 1 -- 172.21.15.43:0/1162840284 >> v1:172.21.15.43:6789/0 conn(0x7f3fc814f940 legacy=0x7f3fc80ccb20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:53:53.097 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:53.091+0000 7f3fc759e640 1 -- 172.21.15.43:0/1162840284 --> v2:172.21.15.43:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3fc806cbc0 con 0x7f3fc815cb00 2023-08-09T13:53:53.098 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:53.091+0000 7f3fc659c640 1 --2- 172.21.15.43:0/1162840284 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f3fc806c020 0x7f3fc815c5c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-08-09T13:53:53.099 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:53.092+0000 7f3fc759e640 1 --2- 172.21.15.43:0/1162840284 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f3fc815cb00 0x7f3fc80c8e80 secure :-1 s=READY pgs=1524 cs=0 l=1 rev1=1 crypto rx=0x7f3fc0009dc0 tx=0x7f3fc002be90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=75ab6d48bc7bb30c server_cookie=0 in_seq=0 out_seq=0 2023-08-09T13:53:53.099 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:53.093+0000 7f3fc5d9b640 1 -- 172.21.15.43:0/1162840284 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3fc002cc40 con 0x7f3fc815cb00 2023-08-09T13:53:53.099 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:53.093+0000 7f3fc5d9b640 1 -- 172.21.15.43:0/1162840284 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3fc002cda0 con 0x7f3fc815cb00 2023-08-09T13:53:53.100 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:53.093+0000 7f3fc5d9b640 1 -- 172.21.15.43:0/1162840284 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3fc0034660 con 0x7f3fc815cb00 2023-08-09T13:53:53.100 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:53.093+0000 7f3fcd234640 1 -- 172.21.15.43:0/1162840284 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f3fc815cb00 msgr2=0x7f3fc80c8e80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:53:53.100 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:53.093+0000 7f3fcd234640 1 --2- 172.21.15.43:0/1162840284 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f3fc815cb00 0x7f3fc80c8e80 secure :-1 s=READY pgs=1524 cs=0 l=1 rev1=1 crypto rx=0x7f3fc0009dc0 tx=0x7f3fc002be90 comp rx=0 tx=0).stop 2023-08-09T13:53:53.100 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:53.093+0000 7f3fcd234640 1 -- 172.21.15.43:0/1162840284 shutdown_connections 2023-08-09T13:53:53.100 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:53.093+0000 7f3fcd234640 1 --2- 172.21.15.43:0/1162840284 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f3fc815cb00 0x7f3fc80c8e80 unknown :-1 s=CLOSED pgs=1524 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:53:53.101 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:53.093+0000 7f3fcd234640 1 --2- 172.21.15.43:0/1162840284 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f3fc806c020 0x7f3fc815c5c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:53:53.101 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:53.093+0000 7f3fcd234640 1 -- 172.21.15.43:0/1162840284 >> 172.21.15.43:0/1162840284 conn(0x7f3fc80ce0a0 msgr2=0x7f3fc80ce480 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-09T13:53:53.101 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:53.094+0000 7f3fcd234640 1 -- 172.21.15.43:0/1162840284 shutdown_connections 2023-08-09T13:53:53.101 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:53.094+0000 7f3fcd234640 1 -- 172.21.15.43:0/1162840284 wait complete. 2023-08-09T13:53:53.101 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:53.095+0000 7f3fcd234640 1 Processor -- start 2023-08-09T13:53:53.102 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:53.095+0000 7f3fcd234640 1 -- start start 2023-08-09T13:53:53.102 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:53.096+0000 7f3fcd234640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f3fc806c020 0x7f3fc8142220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:53:53.102 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:53.096+0000 7f3fcd234640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f3fc814f940 0x7f3fc815b000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:53:53.102 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:53.096+0000 7f3fcd234640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f3fc815cb00 0x7f3fc815b540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:53:53.102 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:53.096+0000 7f3fcd234640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3fc8142760 con 0x7f3fc815cb00 2023-08-09T13:53:53.102 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:53.096+0000 7f3fcd234640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3fc81488b0 con 0x7f3fc814f940 2023-08-09T13:53:53.103 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:53.096+0000 7f3fcd234640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3fc8148b90 con 0x7f3fc806c020 2023-08-09T13:53:53.103 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:53.097+0000 7f3fc759e640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f3fc815cb00 0x7f3fc815b540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:53:53.103 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:53.097+0000 7f3fc759e640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f3fc815cb00 0x7f3fc815b540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:56394/0 (socket says 172.21.15.43:56394) 2023-08-09T13:53:53.103 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:53.097+0000 7f3fc759e640 1 -- 172.21.15.43:0/2022872252 learned_addr learned my addr 172.21.15.43:0/2022872252 (peer_addr_for_me v2:172.21.15.43:0/0) 2023-08-09T13:53:53.103 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:53.097+0000 7f3fc6d9d640 1 --2- 172.21.15.43:0/2022872252 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f3fc806c020 0x7f3fc8142220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:53:53.105 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:53.097+0000 7f3fc659c640 1 --2- 172.21.15.43:0/2022872252 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f3fc814f940 0x7f3fc815b000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:53:53.105 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:53.097+0000 7f3fc759e640 1 -- 172.21.15.43:0/2022872252 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f3fc806c020 msgr2=0x7f3fc8142220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:53:53.105 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:53.097+0000 7f3fc759e640 1 --2- 172.21.15.43:0/2022872252 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f3fc806c020 0x7f3fc8142220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:53:53.105 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:53.097+0000 7f3fc759e640 1 -- 172.21.15.43:0/2022872252 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f3fc814f940 msgr2=0x7f3fc815b000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:53:53.105 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:53.097+0000 7f3fc759e640 1 --2- 172.21.15.43:0/2022872252 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f3fc814f940 0x7f3fc815b000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:53:53.105 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:53.097+0000 7f3fc759e640 1 -- 172.21.15.43:0/2022872252 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3fc0009a70 con 0x7f3fc815cb00 2023-08-09T13:53:53.106 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:53.097+0000 7f3fc659c640 1 --2- 172.21.15.43:0/2022872252 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f3fc814f940 0x7f3fc815b000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-08-09T13:53:53.106 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:53.097+0000 7f3fc6d9d640 1 --2- 172.21.15.43:0/2022872252 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f3fc806c020 0x7f3fc8142220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-08-09T13:53:53.106 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:53.098+0000 7f3fc759e640 1 --2- 172.21.15.43:0/2022872252 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f3fc815cb00 0x7f3fc815b540 secure :-1 s=READY pgs=1525 cs=0 l=1 rev1=1 crypto rx=0x7f3fc0009ee0 tx=0x7f3fc0004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-09T13:53:53.106 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:53.098+0000 7f3faf7fe640 1 -- 172.21.15.43:0/2022872252 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3fc003e610 con 0x7f3fc815cb00 2023-08-09T13:53:53.106 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:53.098+0000 7f3faf7fe640 1 -- 172.21.15.43:0/2022872252 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3fc003e770 con 0x7f3fc815cb00 2023-08-09T13:53:53.106 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:53.098+0000 7f3faf7fe640 1 -- 172.21.15.43:0/2022872252 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3fc003eb20 con 0x7f3fc815cb00 2023-08-09T13:53:53.106 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:53.098+0000 7f3fcd234640 1 -- 172.21.15.43:0/2022872252 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3fc8148e70 con 0x7f3fc815cb00 2023-08-09T13:53:53.107 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:53.098+0000 7f3fcd234640 1 -- 172.21.15.43:0/2022872252 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3fc8149430 con 0x7f3fc815cb00 2023-08-09T13:53:53.107 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:53.099+0000 7f3fcd234640 1 -- 172.21.15.43:0/2022872252 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3fc80739f0 con 0x7f3fc815cb00 2023-08-09T13:53:53.107 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:53.100+0000 7f3faf7fe640 1 -- 172.21.15.43:0/2022872252 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 56) v1 ==== 88443+0+0 (secure 0 0 0) 0x7f3fc003b050 con 0x7f3fc815cb00 2023-08-09T13:53:53.107 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:53.101+0000 7f3faf7fe640 1 --2- 172.21.15.43:0/2022872252 >> [v2:172.21.15.203:6832/2527059680,v1:172.21.15.203:6833/2527059680] conn(0x7f3fb406a170 0x7f3fb406c630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:53:53.107 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:53.102+0000 7f3faf7fe640 1 -- 172.21.15.43:0/2022872252 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(129..129 src has 1..129) v4 ==== 6426+0+0 (secure 0 0 0) 0x7f3fc00ab080 con 0x7f3fc815cb00 2023-08-09T13:53:53.107 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:53.102+0000 7f3fc6d9d640 1 --2- 172.21.15.43:0/2022872252 >> [v2:172.21.15.203:6832/2527059680,v1:172.21.15.203:6833/2527059680] conn(0x7f3fb406a170 0x7f3fb406c630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:53:53.108 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:53.103+0000 7f3fc6d9d640 1 --2- 172.21.15.43:0/2022872252 >> [v2:172.21.15.203:6832/2527059680,v1:172.21.15.203:6833/2527059680] conn(0x7f3fb406a170 0x7f3fb406c630 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f3fb8000f60 tx=0x7f3fb8007c10 comp rx=0 tx=0).ready entity=mgr.6377 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-09T13:53:53.108 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:53.105+0000 7f3faf7fe640 1 -- 172.21.15.43:0/2022872252 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178066 (secure 0 0 0) 0x7f3fc00dd7b0 con 0x7f3fc815cb00 2023-08-09T13:53:53.296 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:53.292+0000 7f3fcd234640 1 -- 172.21.15.43:0/2022872252 --> [v2:172.21.15.203:6832/2527059680,v1:172.21.15.203:6833/2527059680] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f3fc806d180 con 0x7f3fb406a170 2023-08-09T13:53:53.318 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:53.316+0000 7f3faf7fe640 1 -- 172.21.15.43:0/2022872252 <== mgr.6377 v2:172.21.15.203:6832/2527059680 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+309767 (secure 0 0 0) 0x7f3fb8007270 con 0x7f3fb406a170 2023-08-09T13:53:53.321 INFO:teuthology.orchestra.run.smithi043.stdout: 2023-08-09T13:53:53.324 INFO:teuthology.orchestra.run.smithi043.stderr:dumped all 2023-08-09T13:53:53.325 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:53.323+0000 7f3fcd234640 1 -- 172.21.15.43:0/2022872252 >> [v2:172.21.15.203:6832/2527059680,v1:172.21.15.203:6833/2527059680] conn(0x7f3fb406a170 msgr2=0x7f3fb406c630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:53:53.326 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:53.323+0000 7f3fcd234640 1 --2- 172.21.15.43:0/2022872252 >> [v2:172.21.15.203:6832/2527059680,v1:172.21.15.203:6833/2527059680] conn(0x7f3fb406a170 0x7f3fb406c630 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f3fb8000f60 tx=0x7f3fb8007c10 comp rx=0 tx=0).stop 2023-08-09T13:53:53.326 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:53.323+0000 7f3fcd234640 1 -- 172.21.15.43:0/2022872252 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f3fc815cb00 msgr2=0x7f3fc815b540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:53:53.326 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:53.323+0000 7f3fcd234640 1 --2- 172.21.15.43:0/2022872252 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f3fc815cb00 0x7f3fc815b540 secure :-1 s=READY pgs=1525 cs=0 l=1 rev1=1 crypto rx=0x7f3fc0009ee0 tx=0x7f3fc0004a40 comp rx=0 tx=0).stop 2023-08-09T13:53:53.326 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:53.323+0000 7f3fcd234640 1 -- 172.21.15.43:0/2022872252 shutdown_connections 2023-08-09T13:53:53.326 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:53.323+0000 7f3fcd234640 1 --2- 172.21.15.43:0/2022872252 >> [v2:172.21.15.203:6832/2527059680,v1:172.21.15.203:6833/2527059680] conn(0x7f3fb406a170 0x7f3fb406c630 unknown :-1 s=CLOSED pgs=65 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:53:53.326 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:53.323+0000 7f3fcd234640 1 --2- 172.21.15.43:0/2022872252 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f3fc815cb00 0x7f3fc815b540 unknown :-1 s=CLOSED pgs=1525 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:53:53.327 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:53.323+0000 7f3fcd234640 1 --2- 172.21.15.43:0/2022872252 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f3fc814f940 0x7f3fc815b000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:53:53.327 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:53.323+0000 7f3fcd234640 1 --2- 172.21.15.43:0/2022872252 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f3fc806c020 0x7f3fc8142220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:53:53.327 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:53.323+0000 7f3fcd234640 1 -- 172.21.15.43:0/2022872252 >> 172.21.15.43:0/2022872252 conn(0x7f3fc80ce0a0 msgr2=0x7f3fc8153ae0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-09T13:53:53.327 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:53.323+0000 7f3fcd234640 1 -- 172.21.15.43:0/2022872252 shutdown_connections 2023-08-09T13:53:53.327 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:53.323+0000 7f3fcd234640 1 -- 172.21.15.43:0/2022872252 wait complete. 2023-08-09T13:53:53.345 INFO:teuthology.orchestra.run.smithi043.stdout:{"pg_ready":true,"pg_map":{"version":26,"stamp":"2023-08-09T13:53:52.380049+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":23,"num_object_clones":0,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":0,"num_read":907,"num_read_kb":1830,"num_write":367,"num_write_kb":2430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":246,"ondisk_log_size":246,"up":210,"acting":210,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":132,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":331416,"kb_used_data":13600,"kb_used_omap":13,"kb_used_meta":317682,"kb_avail":754643304,"statfs":{"total":773094113280,"available":772754743296,"internally_reserved":0,"allocated":13926400,"data_stored":7946506,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":14248,"internal_metadata":325306456},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":37,"apply_latency_ms":37,"commit_latency_ns":37000000,"apply_latency_ns":37000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.002038"},"pg_stats":[{"pgid":"17.3f","version":"0'0","reported_seq":9,"reported_epoch":128,"state":"active+clean","last_fresh":"2023-08-09T13:53:48.321225+0000","last_change":"2023-08-09T13:53:48.317385+0000","last_active":"2023-08-09T13:53:48.321225+0000","last_peered":"2023-08-09T13:53:48.321225+0000","last_clean":"2023-08-09T13:53:48.321225+0000","last_became_active":"2023-08-09T13:53:48.317083+0000","last_became_peered":"2023-08-09T13:53:48.317083+0000","last_unstale":"2023-08-09T13:53:48.321225+0000","last_undegraded":"2023-08-09T13:53:48.321225+0000","last_fullsized":"2023-08-09T13:53:48.321225+0000","mapping_epoch":127,"log_start":"0'0","ondisk_log_start":"0'0","created":127,"last_epoch_clean":128,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:53:47.231906+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:53:47.231906+0000","last_clean_scrub_stamp":"2023-08-09T13:53:47.231906+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T15:13:30.362935+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.3e","version":"0'0","reported_seq":9,"reported_epoch":128,"state":"active+clean","last_fresh":"2023-08-09T13:53:48.311117+0000","last_change":"2023-08-09T13:53:48.308647+0000","last_active":"2023-08-09T13:53:48.311117+0000","last_peered":"2023-08-09T13:53:48.311117+0000","last_clean":"2023-08-09T13:53:48.311117+0000","last_became_active":"2023-08-09T13:53:48.308358+0000","last_became_peered":"2023-08-09T13:53:48.308358+0000","last_unstale":"2023-08-09T13:53:48.311117+0000","last_undegraded":"2023-08-09T13:53:48.311117+0000","last_fullsized":"2023-08-09T13:53:48.311117+0000","mapping_epoch":127,"log_start":"0'0","ondisk_log_start":"0'0","created":127,"last_epoch_clean":128,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:53:47.231906+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:53:47.231906+0000","last_clean_scrub_stamp":"2023-08-09T13:53:47.231906+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T20:17:37.885150+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.3d","version":"0'0","reported_seq":9,"reported_epoch":128,"state":"active+clean","last_fresh":"2023-08-09T13:53:48.325295+0000","last_change":"2023-08-09T13:53:48.318895+0000","last_active":"2023-08-09T13:53:48.325295+0000","last_peered":"2023-08-09T13:53:48.325295+0000","last_clean":"2023-08-09T13:53:48.325295+0000","last_became_active":"2023-08-09T13:53:48.318640+0000","last_became_peered":"2023-08-09T13:53:48.318640+0000","last_unstale":"2023-08-09T13:53:48.325295+0000","last_undegraded":"2023-08-09T13:53:48.325295+0000","last_fullsized":"2023-08-09T13:53:48.325295+0000","mapping_epoch":127,"log_start":"0'0","ondisk_log_start":"0'0","created":127,"last_epoch_clean":128,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:53:47.231906+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:53:47.231906+0000","last_clean_scrub_stamp":"2023-08-09T13:53:47.231906+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T23:29:56.162342+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"17.3c","version":"0'0","reported_seq":9,"reported_epoch":128,"state":"active+clean","last_fresh":"2023-08-09T13:53:48.318073+0000","last_change":"2023-08-09T13:53:48.314244+0000","last_active":"2023-08-09T13:53:48.318073+0000","last_peered":"2023-08-09T13:53:48.318073+0000","last_clean":"2023-08-09T13:53:48.318073+0000","last_became_active":"2023-08-09T13:53:48.313966+0000","last_became_peered":"2023-08-09T13:53:48.313966+0000","last_unstale":"2023-08-09T13:53:48.318073+0000","last_undegraded":"2023-08-09T13:53:48.318073+0000","last_fullsized":"2023-08-09T13:53:48.318073+0000","mapping_epoch":127,"log_start":"0'0","ondisk_log_start":"0'0","created":127,"last_epoch_clean":128,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:53:47.231906+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:53:47.231906+0000","last_clean_scrub_stamp":"2023-08-09T13:53:47.231906+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T19:03:41.276282+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.3b","version":"0'0","reported_seq":11,"reported_epoch":129,"state":"active+clean","last_fresh":"2023-08-09T13:53:49.282065+0000","last_change":"2023-08-09T13:53:48.286401+0000","last_active":"2023-08-09T13:53:49.282065+0000","last_peered":"2023-08-09T13:53:49.282065+0000","last_clean":"2023-08-09T13:53:49.282065+0000","last_became_active":"2023-08-09T13:53:48.286106+0000","last_became_peered":"2023-08-09T13:53:48.286106+0000","last_unstale":"2023-08-09T13:53:49.282065+0000","last_undegraded":"2023-08-09T13:53:49.282065+0000","last_fullsized":"2023-08-09T13:53:49.282065+0000","mapping_epoch":127,"log_start":"0'0","ondisk_log_start":"0'0","created":127,"last_epoch_clean":128,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:53:47.231906+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:53:47.231906+0000","last_clean_scrub_stamp":"2023-08-09T13:53:47.231906+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-11T00:07:14.541612+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"17.3a","version":"0'0","reported_seq":11,"reported_epoch":129,"state":"active+clean","last_fresh":"2023-08-09T13:53:49.277155+0000","last_change":"2023-08-09T13:53:48.287482+0000","last_active":"2023-08-09T13:53:49.277155+0000","last_peered":"2023-08-09T13:53:49.277155+0000","last_clean":"2023-08-09T13:53:49.277155+0000","last_became_active":"2023-08-09T13:53:48.287113+0000","last_became_peered":"2023-08-09T13:53:48.287113+0000","last_unstale":"2023-08-09T13:53:49.277155+0000","last_undegraded":"2023-08-09T13:53:49.277155+0000","last_fullsized":"2023-08-09T13:53:49.277155+0000","mapping_epoch":127,"log_start":"0'0","ondisk_log_start":"0'0","created":127,"last_epoch_clean":128,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:53:47.231906+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:53:47.231906+0000","last_clean_scrub_stamp":"2023-08-09T13:53:47.231906+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T15:27:38.211706+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"17.39","version":"0'0","reported_seq":9,"reported_epoch":128,"state":"active+clean","last_fresh":"2023-08-09T13:53:48.320308+0000","last_change":"2023-08-09T13:53:48.317266+0000","last_active":"2023-08-09T13:53:48.320308+0000","last_peered":"2023-08-09T13:53:48.320308+0000","last_clean":"2023-08-09T13:53:48.320308+0000","last_became_active":"2023-08-09T13:53:48.316563+0000","last_became_peered":"2023-08-09T13:53:48.316563+0000","last_unstale":"2023-08-09T13:53:48.320308+0000","last_undegraded":"2023-08-09T13:53:48.320308+0000","last_fullsized":"2023-08-09T13:53:48.320308+0000","mapping_epoch":127,"log_start":"0'0","ondisk_log_start":"0'0","created":127,"last_epoch_clean":128,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:53:47.231906+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:53:47.231906+0000","last_clean_scrub_stamp":"2023-08-09T13:53:47.231906+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T20:59:22.754345+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"17.38","version":"0'0","reported_seq":9,"reported_epoch":128,"state":"active+clean","last_fresh":"2023-08-09T13:53:48.309393+0000","last_change":"2023-08-09T13:53:48.306331+0000","last_active":"2023-08-09T13:53:48.309393+0000","last_peered":"2023-08-09T13:53:48.309393+0000","last_clean":"2023-08-09T13:53:48.309393+0000","last_became_active":"2023-08-09T13:53:48.305997+0000","last_became_peered":"2023-08-09T13:53:48.305997+0000","last_unstale":"2023-08-09T13:53:48.309393+0000","last_undegraded":"2023-08-09T13:53:48.309393+0000","last_fullsized":"2023-08-09T13:53:48.309393+0000","mapping_epoch":127,"log_start":"0'0","ondisk_log_start":"0'0","created":127,"last_epoch_clean":128,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:53:47.231906+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:53:47.231906+0000","last_clean_scrub_stamp":"2023-08-09T13:53:47.231906+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T17:52:01.233963+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"17.37","version":"0'0","reported_seq":9,"reported_epoch":128,"state":"active+clean","last_fresh":"2023-08-09T13:53:48.315077+0000","last_change":"2023-08-09T13:53:48.310293+0000","last_active":"2023-08-09T13:53:48.315077+0000","last_peered":"2023-08-09T13:53:48.315077+0000","last_clean":"2023-08-09T13:53:48.315077+0000","last_became_active":"2023-08-09T13:53:48.310037+0000","last_became_peered":"2023-08-09T13:53:48.310037+0000","last_unstale":"2023-08-09T13:53:48.315077+0000","last_undegraded":"2023-08-09T13:53:48.315077+0000","last_fullsized":"2023-08-09T13:53:48.315077+0000","mapping_epoch":127,"log_start":"0'0","ondisk_log_start":"0'0","created":127,"last_epoch_clean":128,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:53:47.231906+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:53:47.231906+0000","last_clean_scrub_stamp":"2023-08-09T13:53:47.231906+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T22:47:39.839446+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.36","version":"0'0","reported_seq":9,"reported_epoch":128,"state":"active+clean","last_fresh":"2023-08-09T13:53:48.324847+0000","last_change":"2023-08-09T13:53:48.320807+0000","last_active":"2023-08-09T13:53:48.324847+0000","last_peered":"2023-08-09T13:53:48.324847+0000","last_clean":"2023-08-09T13:53:48.324847+0000","last_became_active":"2023-08-09T13:53:48.320055+0000","last_became_peered":"2023-08-09T13:53:48.320055+0000","last_unstale":"2023-08-09T13:53:48.324847+0000","last_undegraded":"2023-08-09T13:53:48.324847+0000","last_fullsized":"2023-08-09T13:53:48.324847+0000","mapping_epoch":127,"log_start":"0'0","ondisk_log_start":"0'0","created":127,"last_epoch_clean":128,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:53:47.231906+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:53:47.231906+0000","last_clean_scrub_stamp":"2023-08-09T13:53:47.231906+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T20:39:00.369183+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.35","version":"0'0","reported_seq":9,"reported_epoch":128,"state":"active+clean","last_fresh":"2023-08-09T13:53:48.324801+0000","last_change":"2023-08-09T13:53:48.316614+0000","last_active":"2023-08-09T13:53:48.324801+0000","last_peered":"2023-08-09T13:53:48.324801+0000","last_clean":"2023-08-09T13:53:48.324801+0000","last_became_active":"2023-08-09T13:53:48.315558+0000","last_became_peered":"2023-08-09T13:53:48.315558+0000","last_unstale":"2023-08-09T13:53:48.324801+0000","last_undegraded":"2023-08-09T13:53:48.324801+0000","last_fullsized":"2023-08-09T13:53:48.324801+0000","mapping_epoch":127,"log_start":"0'0","ondisk_log_start":"0'0","created":127,"last_epoch_clean":128,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:53:47.231906+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:53:47.231906+0000","last_clean_scrub_stamp":"2023-08-09T13:53:47.231906+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T19:15:44.745804+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.34","version":"0'0","reported_seq":9,"reported_epoch":128,"state":"active+clean","last_fresh":"2023-08-09T13:53:48.324951+0000","last_change":"2023-08-09T13:53:48.321443+0000","last_active":"2023-08-09T13:53:48.324951+0000","last_peered":"2023-08-09T13:53:48.324951+0000","last_clean":"2023-08-09T13:53:48.324951+0000","last_became_active":"2023-08-09T13:53:48.320604+0000","last_became_peered":"2023-08-09T13:53:48.320604+0000","last_unstale":"2023-08-09T13:53:48.324951+0000","last_undegraded":"2023-08-09T13:53:48.324951+0000","last_fullsized":"2023-08-09T13:53:48.324951+0000","mapping_epoch":127,"log_start":"0'0","ondisk_log_start":"0'0","created":127,"last_epoch_clean":128,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:53:47.231906+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:53:47.231906+0000","last_clean_scrub_stamp":"2023-08-09T13:53:47.231906+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-11T01:26:12.436556+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.33","version":"0'0","reported_seq":9,"reported_epoch":128,"state":"active+clean","last_fresh":"2023-08-09T13:53:48.325763+0000","last_change":"2023-08-09T13:53:48.324709+0000","last_active":"2023-08-09T13:53:48.325763+0000","last_peered":"2023-08-09T13:53:48.325763+0000","last_clean":"2023-08-09T13:53:48.325763+0000","last_became_active":"2023-08-09T13:53:48.324422+0000","last_became_peered":"2023-08-09T13:53:48.324422+0000","last_unstale":"2023-08-09T13:53:48.325763+0000","last_undegraded":"2023-08-09T13:53:48.325763+0000","last_fullsized":"2023-08-09T13:53:48.325763+0000","mapping_epoch":127,"log_start":"0'0","ondisk_log_start":"0'0","created":127,"last_epoch_clean":128,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:53:47.231906+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:53:47.231906+0000","last_clean_scrub_stamp":"2023-08-09T13:53:47.231906+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T15:38:37.763892+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"17.32","version":"0'0","reported_seq":9,"reported_epoch":128,"state":"active+clean","last_fresh":"2023-08-09T13:53:48.317062+0000","last_change":"2023-08-09T13:53:48.314769+0000","last_active":"2023-08-09T13:53:48.317062+0000","last_peered":"2023-08-09T13:53:48.317062+0000","last_clean":"2023-08-09T13:53:48.317062+0000","last_became_active":"2023-08-09T13:53:48.314498+0000","last_became_peered":"2023-08-09T13:53:48.314498+0000","last_unstale":"2023-08-09T13:53:48.317062+0000","last_undegraded":"2023-08-09T13:53:48.317062+0000","last_fullsized":"2023-08-09T13:53:48.317062+0000","mapping_epoch":127,"log_start":"0'0","ondisk_log_start":"0'0","created":127,"last_epoch_clean":128,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:53:47.231906+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:53:47.231906+0000","last_clean_scrub_stamp":"2023-08-09T13:53:47.231906+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-11T01:26:16.686764+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"17.31","version":"0'0","reported_seq":11,"reported_epoch":129,"state":"active+clean","last_fresh":"2023-08-09T13:53:49.277638+0000","last_change":"2023-08-09T13:53:48.295587+0000","last_active":"2023-08-09T13:53:49.277638+0000","last_peered":"2023-08-09T13:53:49.277638+0000","last_clean":"2023-08-09T13:53:49.277638+0000","last_became_active":"2023-08-09T13:53:48.295280+0000","last_became_peered":"2023-08-09T13:53:48.295280+0000","last_unstale":"2023-08-09T13:53:49.277638+0000","last_undegraded":"2023-08-09T13:53:49.277638+0000","last_fullsized":"2023-08-09T13:53:49.277638+0000","mapping_epoch":127,"log_start":"0'0","ondisk_log_start":"0'0","created":127,"last_epoch_clean":128,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:53:47.231906+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:53:47.231906+0000","last_clean_scrub_stamp":"2023-08-09T13:53:47.231906+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T19:15:58.915975+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"17.30","version":"0'0","reported_seq":9,"reported_epoch":128,"state":"active+clean","last_fresh":"2023-08-09T13:53:48.320509+0000","last_change":"2023-08-09T13:53:48.315847+0000","last_active":"2023-08-09T13:53:48.320509+0000","last_peered":"2023-08-09T13:53:48.320509+0000","last_clean":"2023-08-09T13:53:48.320509+0000","last_became_active":"2023-08-09T13:53:48.315567+0000","last_became_peered":"2023-08-09T13:53:48.315567+0000","last_unstale":"2023-08-09T13:53:48.320509+0000","last_undegraded":"2023-08-09T13:53:48.320509+0000","last_fullsized":"2023-08-09T13:53:48.320509+0000","mapping_epoch":127,"log_start":"0'0","ondisk_log_start":"0'0","created":127,"last_epoch_clean":128,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:53:47.231906+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:53:47.231906+0000","last_clean_scrub_stamp":"2023-08-09T13:53:47.231906+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T22:44:56.011608+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.2f","version":"0'0","reported_seq":9,"reported_epoch":128,"state":"active+clean","last_fresh":"2023-08-09T13:53:48.325011+0000","last_change":"2023-08-09T13:53:48.318420+0000","last_active":"2023-08-09T13:53:48.325011+0000","last_peered":"2023-08-09T13:53:48.325011+0000","last_clean":"2023-08-09T13:53:48.325011+0000","last_became_active":"2023-08-09T13:53:48.318135+0000","last_became_peered":"2023-08-09T13:53:48.318135+0000","last_unstale":"2023-08-09T13:53:48.325011+0000","last_undegraded":"2023-08-09T13:53:48.325011+0000","last_fullsized":"2023-08-09T13:53:48.325011+0000","mapping_epoch":127,"log_start":"0'0","ondisk_log_start":"0'0","created":127,"last_epoch_clean":128,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:53:47.231906+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:53:47.231906+0000","last_clean_scrub_stamp":"2023-08-09T13:53:47.231906+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T22:31:35.095714+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.2e","version":"0'0","reported_seq":11,"reported_epoch":129,"state":"active+clean","last_fresh":"2023-08-09T13:53:49.281097+0000","last_change":"2023-08-09T13:53:48.314917+0000","last_active":"2023-08-09T13:53:49.281097+0000","last_peered":"2023-08-09T13:53:49.281097+0000","last_clean":"2023-08-09T13:53:49.281097+0000","last_became_active":"2023-08-09T13:53:48.314633+0000","last_became_peered":"2023-08-09T13:53:48.314633+0000","last_unstale":"2023-08-09T13:53:49.281097+0000","last_undegraded":"2023-08-09T13:53:49.281097+0000","last_fullsized":"2023-08-09T13:53:49.281097+0000","mapping_epoch":127,"log_start":"0'0","ondisk_log_start":"0'0","created":127,"last_epoch_clean":128,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:53:47.231906+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:53:47.231906+0000","last_clean_scrub_stamp":"2023-08-09T13:53:47.231906+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T18:08:49.619864+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"17.2d","version":"0'0","reported_seq":9,"reported_epoch":128,"state":"active+clean","last_fresh":"2023-08-09T13:53:48.318983+0000","last_change":"2023-08-09T13:53:48.317378+0000","last_active":"2023-08-09T13:53:48.318983+0000","last_peered":"2023-08-09T13:53:48.318983+0000","last_clean":"2023-08-09T13:53:48.318983+0000","last_became_active":"2023-08-09T13:53:48.317087+0000","last_became_peered":"2023-08-09T13:53:48.317087+0000","last_unstale":"2023-08-09T13:53:48.318983+0000","last_undegraded":"2023-08-09T13:53:48.318983+0000","last_fullsized":"2023-08-09T13:53:48.318983+0000","mapping_epoch":127,"log_start":"0'0","ondisk_log_start":"0'0","created":127,"last_epoch_clean":128,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:53:47.231906+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:53:47.231906+0000","last_clean_scrub_stamp":"2023-08-09T13:53:47.231906+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T14:47:06.672771+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"17.2c","version":"0'0","reported_seq":9,"reported_epoch":128,"state":"active+clean","last_fresh":"2023-08-09T13:53:48.324502+0000","last_change":"2023-08-09T13:53:48.315031+0000","last_active":"2023-08-09T13:53:48.324502+0000","last_peered":"2023-08-09T13:53:48.324502+0000","last_clean":"2023-08-09T13:53:48.324502+0000","last_became_active":"2023-08-09T13:53:48.314618+0000","last_became_peered":"2023-08-09T13:53:48.314618+0000","last_unstale":"2023-08-09T13:53:48.324502+0000","last_undegraded":"2023-08-09T13:53:48.324502+0000","last_fullsized":"2023-08-09T13:53:48.324502+0000","mapping_epoch":127,"log_start":"0'0","ondisk_log_start":"0'0","created":127,"last_epoch_clean":128,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:53:47.231906+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:53:47.231906+0000","last_clean_scrub_stamp":"2023-08-09T13:53:47.231906+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T19:06:46.667230+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.2b","version":"0'0","reported_seq":9,"reported_epoch":128,"state":"active+clean","last_fresh":"2023-08-09T13:53:48.311526+0000","last_change":"2023-08-09T13:53:48.307775+0000","last_active":"2023-08-09T13:53:48.311526+0000","last_peered":"2023-08-09T13:53:48.311526+0000","last_clean":"2023-08-09T13:53:48.311526+0000","last_became_active":"2023-08-09T13:53:48.307366+0000","last_became_peered":"2023-08-09T13:53:48.307366+0000","last_unstale":"2023-08-09T13:53:48.311526+0000","last_undegraded":"2023-08-09T13:53:48.311526+0000","last_fullsized":"2023-08-09T13:53:48.311526+0000","mapping_epoch":127,"log_start":"0'0","ondisk_log_start":"0'0","created":127,"last_epoch_clean":128,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:53:47.231906+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:53:47.231906+0000","last_clean_scrub_stamp":"2023-08-09T13:53:47.231906+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-11T01:26:12.436556+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.2a","version":"0'0","reported_seq":9,"reported_epoch":128,"state":"active+clean","last_fresh":"2023-08-09T13:53:48.317220+0000","last_change":"2023-08-09T13:53:48.314844+0000","last_active":"2023-08-09T13:53:48.317220+0000","last_peered":"2023-08-09T13:53:48.317220+0000","last_clean":"2023-08-09T13:53:48.317220+0000","last_became_active":"2023-08-09T13:53:48.314564+0000","last_became_peered":"2023-08-09T13:53:48.314564+0000","last_unstale":"2023-08-09T13:53:48.317220+0000","last_undegraded":"2023-08-09T13:53:48.317220+0000","last_fullsized":"2023-08-09T13:53:48.317220+0000","mapping_epoch":127,"log_start":"0'0","ondisk_log_start":"0'0","created":127,"last_epoch_clean":128,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:53:47.231906+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:53:47.231906+0000","last_clean_scrub_stamp":"2023-08-09T13:53:47.231906+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T22:45:31.755168+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"17.29","version":"0'0","reported_seq":9,"reported_epoch":128,"state":"active+clean","last_fresh":"2023-08-09T13:53:48.324099+0000","last_change":"2023-08-09T13:53:48.313683+0000","last_active":"2023-08-09T13:53:48.324099+0000","last_peered":"2023-08-09T13:53:48.324099+0000","last_clean":"2023-08-09T13:53:48.324099+0000","last_became_active":"2023-08-09T13:53:48.313375+0000","last_became_peered":"2023-08-09T13:53:48.313375+0000","last_unstale":"2023-08-09T13:53:48.324099+0000","last_undegraded":"2023-08-09T13:53:48.324099+0000","last_fullsized":"2023-08-09T13:53:48.324099+0000","mapping_epoch":127,"log_start":"0'0","ondisk_log_start":"0'0","created":127,"last_epoch_clean":128,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:53:47.231906+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:53:47.231906+0000","last_clean_scrub_stamp":"2023-08-09T13:53:47.231906+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T19:48:33.093205+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.28","version":"0'0","reported_seq":9,"reported_epoch":128,"state":"active+clean","last_fresh":"2023-08-09T13:53:48.315613+0000","last_change":"2023-08-09T13:53:48.313694+0000","last_active":"2023-08-09T13:53:48.315613+0000","last_peered":"2023-08-09T13:53:48.315613+0000","last_clean":"2023-08-09T13:53:48.315613+0000","last_became_active":"2023-08-09T13:53:48.312228+0000","last_became_peered":"2023-08-09T13:53:48.312228+0000","last_unstale":"2023-08-09T13:53:48.315613+0000","last_undegraded":"2023-08-09T13:53:48.315613+0000","last_fullsized":"2023-08-09T13:53:48.315613+0000","mapping_epoch":127,"log_start":"0'0","ondisk_log_start":"0'0","created":127,"last_epoch_clean":128,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:53:47.231906+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:53:47.231906+0000","last_clean_scrub_stamp":"2023-08-09T13:53:47.231906+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T14:21:57.770836+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"17.27","version":"0'0","reported_seq":9,"reported_epoch":128,"state":"active+clean","last_fresh":"2023-08-09T13:53:48.325030+0000","last_change":"2023-08-09T13:53:48.318790+0000","last_active":"2023-08-09T13:53:48.325030+0000","last_peered":"2023-08-09T13:53:48.325030+0000","last_clean":"2023-08-09T13:53:48.325030+0000","last_became_active":"2023-08-09T13:53:48.318492+0000","last_became_peered":"2023-08-09T13:53:48.318492+0000","last_unstale":"2023-08-09T13:53:48.325030+0000","last_undegraded":"2023-08-09T13:53:48.325030+0000","last_fullsized":"2023-08-09T13:53:48.325030+0000","mapping_epoch":127,"log_start":"0'0","ondisk_log_start":"0'0","created":127,"last_epoch_clean":128,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:53:47.231906+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:53:47.231906+0000","last_clean_scrub_stamp":"2023-08-09T13:53:47.231906+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T17:37:00.223661+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.26","version":"0'0","reported_seq":9,"reported_epoch":128,"state":"active+clean","last_fresh":"2023-08-09T13:53:48.314212+0000","last_change":"2023-08-09T13:53:48.307656+0000","last_active":"2023-08-09T13:53:48.314212+0000","last_peered":"2023-08-09T13:53:48.314212+0000","last_clean":"2023-08-09T13:53:48.314212+0000","last_became_active":"2023-08-09T13:53:48.307303+0000","last_became_peered":"2023-08-09T13:53:48.307303+0000","last_unstale":"2023-08-09T13:53:48.314212+0000","last_undegraded":"2023-08-09T13:53:48.314212+0000","last_fullsized":"2023-08-09T13:53:48.314212+0000","mapping_epoch":127,"log_start":"0'0","ondisk_log_start":"0'0","created":127,"last_epoch_clean":128,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:53:47.231906+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:53:47.231906+0000","last_clean_scrub_stamp":"2023-08-09T13:53:47.231906+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T19:06:46.667230+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.25","version":"0'0","reported_seq":11,"reported_epoch":129,"state":"active+clean","last_fresh":"2023-08-09T13:53:49.277776+0000","last_change":"2023-08-09T13:53:48.299619+0000","last_active":"2023-08-09T13:53:49.277776+0000","last_peered":"2023-08-09T13:53:49.277776+0000","last_clean":"2023-08-09T13:53:49.277776+0000","last_became_active":"2023-08-09T13:53:48.299350+0000","last_became_peered":"2023-08-09T13:53:48.299350+0000","last_unstale":"2023-08-09T13:53:49.277776+0000","last_undegraded":"2023-08-09T13:53:49.277776+0000","last_fullsized":"2023-08-09T13:53:49.277776+0000","mapping_epoch":127,"log_start":"0'0","ondisk_log_start":"0'0","created":127,"last_epoch_clean":128,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:53:47.231906+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:53:47.231906+0000","last_clean_scrub_stamp":"2023-08-09T13:53:47.231906+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T19:24:17.238820+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"17.24","version":"0'0","reported_seq":9,"reported_epoch":128,"state":"active+clean","last_fresh":"2023-08-09T13:53:48.306961+0000","last_change":"2023-08-09T13:53:48.300953+0000","last_active":"2023-08-09T13:53:48.306961+0000","last_peered":"2023-08-09T13:53:48.306961+0000","last_clean":"2023-08-09T13:53:48.306961+0000","last_became_active":"2023-08-09T13:53:48.300669+0000","last_became_peered":"2023-08-09T13:53:48.300669+0000","last_unstale":"2023-08-09T13:53:48.306961+0000","last_undegraded":"2023-08-09T13:53:48.306961+0000","last_fullsized":"2023-08-09T13:53:48.306961+0000","mapping_epoch":127,"log_start":"0'0","ondisk_log_start":"0'0","created":127,"last_epoch_clean":128,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:53:47.231906+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:53:47.231906+0000","last_clean_scrub_stamp":"2023-08-09T13:53:47.231906+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T15:29:08.206128+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"17.23","version":"0'0","reported_seq":9,"reported_epoch":128,"state":"active+clean","last_fresh":"2023-08-09T13:53:48.324224+0000","last_change":"2023-08-09T13:53:48.315018+0000","last_active":"2023-08-09T13:53:48.324224+0000","last_peered":"2023-08-09T13:53:48.324224+0000","last_clean":"2023-08-09T13:53:48.324224+0000","last_became_active":"2023-08-09T13:53:48.314730+0000","last_became_peered":"2023-08-09T13:53:48.314730+0000","last_unstale":"2023-08-09T13:53:48.324224+0000","last_undegraded":"2023-08-09T13:53:48.324224+0000","last_fullsized":"2023-08-09T13:53:48.324224+0000","mapping_epoch":127,"log_start":"0'0","ondisk_log_start":"0'0","created":127,"last_epoch_clean":128,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:53:47.231906+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:53:47.231906+0000","last_clean_scrub_stamp":"2023-08-09T13:53:47.231906+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T20:47:17.015473+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.22","version":"0'0","reported_seq":9,"reported_epoch":128,"state":"active+clean","last_fresh":"2023-08-09T13:53:48.316311+0000","last_change":"2023-08-09T13:53:48.312900+0000","last_active":"2023-08-09T13:53:48.316311+0000","last_peered":"2023-08-09T13:53:48.316311+0000","last_clean":"2023-08-09T13:53:48.316311+0000","last_became_active":"2023-08-09T13:53:48.312616+0000","last_became_peered":"2023-08-09T13:53:48.312616+0000","last_unstale":"2023-08-09T13:53:48.316311+0000","last_undegraded":"2023-08-09T13:53:48.316311+0000","last_fullsized":"2023-08-09T13:53:48.316311+0000","mapping_epoch":127,"log_start":"0'0","ondisk_log_start":"0'0","created":127,"last_epoch_clean":128,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:53:47.231906+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:53:47.231906+0000","last_clean_scrub_stamp":"2023-08-09T13:53:47.231906+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T22:33:37.323275+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"17.21","version":"0'0","reported_seq":9,"reported_epoch":128,"state":"active+clean","last_fresh":"2023-08-09T13:53:48.314244+0000","last_change":"2023-08-09T13:53:48.309422+0000","last_active":"2023-08-09T13:53:48.314244+0000","last_peered":"2023-08-09T13:53:48.314244+0000","last_clean":"2023-08-09T13:53:48.314244+0000","last_became_active":"2023-08-09T13:53:48.309069+0000","last_became_peered":"2023-08-09T13:53:48.309069+0000","last_unstale":"2023-08-09T13:53:48.314244+0000","last_undegraded":"2023-08-09T13:53:48.314244+0000","last_fullsized":"2023-08-09T13:53:48.314244+0000","mapping_epoch":127,"log_start":"0'0","ondisk_log_start":"0'0","created":127,"last_epoch_clean":128,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:53:47.231906+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:53:47.231906+0000","last_clean_scrub_stamp":"2023-08-09T13:53:47.231906+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T19:15:44.745804+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.20","version":"0'0","reported_seq":11,"reported_epoch":129,"state":"active+clean","last_fresh":"2023-08-09T13:53:49.276878+0000","last_change":"2023-08-09T13:53:48.288611+0000","last_active":"2023-08-09T13:53:49.276878+0000","last_peered":"2023-08-09T13:53:49.276878+0000","last_clean":"2023-08-09T13:53:49.276878+0000","last_became_active":"2023-08-09T13:53:48.286725+0000","last_became_peered":"2023-08-09T13:53:48.286725+0000","last_unstale":"2023-08-09T13:53:49.276878+0000","last_undegraded":"2023-08-09T13:53:49.276878+0000","last_fullsized":"2023-08-09T13:53:49.276878+0000","mapping_epoch":127,"log_start":"0'0","ondisk_log_start":"0'0","created":127,"last_epoch_clean":128,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:53:47.231906+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:53:47.231906+0000","last_clean_scrub_stamp":"2023-08-09T13:53:47.231906+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T15:01:57.136358+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"17.1e","version":"0'0","reported_seq":11,"reported_epoch":129,"state":"active+clean","last_fresh":"2023-08-09T13:53:49.277801+0000","last_change":"2023-08-09T13:53:48.285528+0000","last_active":"2023-08-09T13:53:49.277801+0000","last_peered":"2023-08-09T13:53:49.277801+0000","last_clean":"2023-08-09T13:53:49.277801+0000","last_became_active":"2023-08-09T13:53:48.285131+0000","last_became_peered":"2023-08-09T13:53:48.285131+0000","last_unstale":"2023-08-09T13:53:49.277801+0000","last_undegraded":"2023-08-09T13:53:49.277801+0000","last_fullsized":"2023-08-09T13:53:49.277801+0000","mapping_epoch":127,"log_start":"0'0","ondisk_log_start":"0'0","created":127,"last_epoch_clean":128,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:53:47.231906+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:53:47.231906+0000","last_clean_scrub_stamp":"2023-08-09T13:53:47.231906+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T17:06:45.323084+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"16.1f","version":"128'4","reported_seq":15,"reported_epoch":128,"state":"active+clean","last_fresh":"2023-08-09T13:53:48.316973+0000","last_change":"2023-08-09T13:53:47.277376+0000","last_active":"2023-08-09T13:53:48.316973+0000","last_peered":"2023-08-09T13:53:48.316973+0000","last_clean":"2023-08-09T13:53:48.316973+0000","last_became_active":"2023-08-09T13:53:47.276816+0000","last_became_peered":"2023-08-09T13:53:47.276816+0000","last_unstale":"2023-08-09T13:53:48.316973+0000","last_undegraded":"2023-08-09T13:53:48.316973+0000","last_fullsized":"2023-08-09T13:53:48.316973+0000","mapping_epoch":126,"log_start":"0'0","ondisk_log_start":"0'0","created":126,"last_epoch_clean":127,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:53:46.216803+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:53:46.216803+0000","last_clean_scrub_stamp":"2023-08-09T13:53:46.216803+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T16:42:10.354539+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"17.1f","version":"0'0","reported_seq":11,"reported_epoch":129,"state":"active+clean","last_fresh":"2023-08-09T13:53:49.281286+0000","last_change":"2023-08-09T13:53:48.315043+0000","last_active":"2023-08-09T13:53:49.281286+0000","last_peered":"2023-08-09T13:53:49.281286+0000","last_clean":"2023-08-09T13:53:49.281286+0000","last_became_active":"2023-08-09T13:53:48.314579+0000","last_became_peered":"2023-08-09T13:53:48.314579+0000","last_unstale":"2023-08-09T13:53:49.281286+0000","last_undegraded":"2023-08-09T13:53:49.281286+0000","last_fullsized":"2023-08-09T13:53:49.281286+0000","mapping_epoch":127,"log_start":"0'0","ondisk_log_start":"0'0","created":127,"last_epoch_clean":128,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:53:47.231906+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:53:47.231906+0000","last_clean_scrub_stamp":"2023-08-09T13:53:47.231906+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T20:01:49.620207+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.1e","version":"128'1","reported_seq":12,"reported_epoch":128,"state":"active+clean","last_fresh":"2023-08-09T13:53:48.320711+0000","last_change":"2023-08-09T13:53:47.278027+0000","last_active":"2023-08-09T13:53:48.320711+0000","last_peered":"2023-08-09T13:53:48.320711+0000","last_clean":"2023-08-09T13:53:48.320711+0000","last_became_active":"2023-08-09T13:53:47.277645+0000","last_became_peered":"2023-08-09T13:53:47.277645+0000","last_unstale":"2023-08-09T13:53:48.320711+0000","last_undegraded":"2023-08-09T13:53:48.320711+0000","last_fullsized":"2023-08-09T13:53:48.320711+0000","mapping_epoch":126,"log_start":"0'0","ondisk_log_start":"0'0","created":126,"last_epoch_clean":127,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:53:46.216803+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:53:46.216803+0000","last_clean_scrub_stamp":"2023-08-09T13:53:46.216803+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T14:12:39.269881+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"17.1c","version":"0'0","reported_seq":11,"reported_epoch":129,"state":"active+clean","last_fresh":"2023-08-09T13:53:49.277476+0000","last_change":"2023-08-09T13:53:48.288150+0000","last_active":"2023-08-09T13:53:49.277476+0000","last_peered":"2023-08-09T13:53:49.277476+0000","last_clean":"2023-08-09T13:53:49.277476+0000","last_became_active":"2023-08-09T13:53:48.287822+0000","last_became_peered":"2023-08-09T13:53:48.287822+0000","last_unstale":"2023-08-09T13:53:49.277476+0000","last_undegraded":"2023-08-09T13:53:49.277476+0000","last_fullsized":"2023-08-09T13:53:49.277476+0000","mapping_epoch":127,"log_start":"0'0","ondisk_log_start":"0'0","created":127,"last_epoch_clean":128,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:53:47.231906+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:53:47.231906+0000","last_clean_scrub_stamp":"2023-08-09T13:53:47.231906+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T16:47:43.779024+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"16.1d","version":"128'2","reported_seq":15,"reported_epoch":129,"state":"active+clean","last_fresh":"2023-08-09T13:53:49.281849+0000","last_change":"2023-08-09T13:53:47.265097+0000","last_active":"2023-08-09T13:53:49.281849+0000","last_peered":"2023-08-09T13:53:49.281849+0000","last_clean":"2023-08-09T13:53:49.281849+0000","last_became_active":"2023-08-09T13:53:47.264731+0000","last_became_peered":"2023-08-09T13:53:47.264731+0000","last_unstale":"2023-08-09T13:53:49.281849+0000","last_undegraded":"2023-08-09T13:53:49.281849+0000","last_fullsized":"2023-08-09T13:53:49.281849+0000","mapping_epoch":126,"log_start":"0'0","ondisk_log_start":"0'0","created":126,"last_epoch_clean":127,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:53:46.216803+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:53:46.216803+0000","last_clean_scrub_stamp":"2023-08-09T13:53:46.216803+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T16:33:44.031013+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"17.1d","version":"0'0","reported_seq":9,"reported_epoch":128,"state":"active+clean","last_fresh":"2023-08-09T13:53:48.325758+0000","last_change":"2023-08-09T13:53:48.323402+0000","last_active":"2023-08-09T13:53:48.325758+0000","last_peered":"2023-08-09T13:53:48.325758+0000","last_clean":"2023-08-09T13:53:48.325758+0000","last_became_active":"2023-08-09T13:53:48.323059+0000","last_became_peered":"2023-08-09T13:53:48.323059+0000","last_unstale":"2023-08-09T13:53:48.325758+0000","last_undegraded":"2023-08-09T13:53:48.325758+0000","last_fullsized":"2023-08-09T13:53:48.325758+0000","mapping_epoch":127,"log_start":"0'0","ondisk_log_start":"0'0","created":127,"last_epoch_clean":128,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:53:47.231906+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:53:47.231906+0000","last_clean_scrub_stamp":"2023-08-09T13:53:47.231906+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T15:29:48.199916+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"16.1c","version":"128'2","reported_seq":15,"reported_epoch":129,"state":"active+clean","last_fresh":"2023-08-09T13:53:49.282053+0000","last_change":"2023-08-09T13:53:47.262351+0000","last_active":"2023-08-09T13:53:49.282053+0000","last_peered":"2023-08-09T13:53:49.282053+0000","last_clean":"2023-08-09T13:53:49.282053+0000","last_became_active":"2023-08-09T13:53:47.261409+0000","last_became_peered":"2023-08-09T13:53:47.261409+0000","last_unstale":"2023-08-09T13:53:49.282053+0000","last_undegraded":"2023-08-09T13:53:49.282053+0000","last_fullsized":"2023-08-09T13:53:49.282053+0000","mapping_epoch":126,"log_start":"0'0","ondisk_log_start":"0'0","created":126,"last_epoch_clean":127,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:53:46.216803+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:53:46.216803+0000","last_clean_scrub_stamp":"2023-08-09T13:53:46.216803+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T14:33:48.941953+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"17.1a","version":"0'0","reported_seq":9,"reported_epoch":128,"state":"active+clean","last_fresh":"2023-08-09T13:53:48.314746+0000","last_change":"2023-08-09T13:53:48.309691+0000","last_active":"2023-08-09T13:53:48.314746+0000","last_peered":"2023-08-09T13:53:48.314746+0000","last_clean":"2023-08-09T13:53:48.314746+0000","last_became_active":"2023-08-09T13:53:48.309426+0000","last_became_peered":"2023-08-09T13:53:48.309426+0000","last_unstale":"2023-08-09T13:53:48.314746+0000","last_undegraded":"2023-08-09T13:53:48.314746+0000","last_fullsized":"2023-08-09T13:53:48.314746+0000","mapping_epoch":127,"log_start":"0'0","ondisk_log_start":"0'0","created":127,"last_epoch_clean":128,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:53:47.231906+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:53:47.231906+0000","last_clean_scrub_stamp":"2023-08-09T13:53:47.231906+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T23:03:43.024505+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.1b","version":"128'1","reported_seq":12,"reported_epoch":128,"state":"active+clean","last_fresh":"2023-08-09T13:53:48.315904+0000","last_change":"2023-08-09T13:53:47.274449+0000","last_active":"2023-08-09T13:53:48.315904+0000","last_peered":"2023-08-09T13:53:48.315904+0000","last_clean":"2023-08-09T13:53:48.315904+0000","last_became_active":"2023-08-09T13:53:47.274178+0000","last_became_peered":"2023-08-09T13:53:47.274178+0000","last_unstale":"2023-08-09T13:53:48.315904+0000","last_undegraded":"2023-08-09T13:53:48.315904+0000","last_fullsized":"2023-08-09T13:53:48.315904+0000","mapping_epoch":126,"log_start":"0'0","ondisk_log_start":"0'0","created":126,"last_epoch_clean":127,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:53:46.216803+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:53:46.216803+0000","last_clean_scrub_stamp":"2023-08-09T13:53:46.216803+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-11T00:35:49.085401+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"17.1b","version":"0'0","reported_seq":9,"reported_epoch":128,"state":"active+clean","last_fresh":"2023-08-09T13:53:48.311545+0000","last_change":"2023-08-09T13:53:48.308092+0000","last_active":"2023-08-09T13:53:48.311545+0000","last_peered":"2023-08-09T13:53:48.311545+0000","last_clean":"2023-08-09T13:53:48.311545+0000","last_became_active":"2023-08-09T13:53:48.307725+0000","last_became_peered":"2023-08-09T13:53:48.307725+0000","last_unstale":"2023-08-09T13:53:48.311545+0000","last_undegraded":"2023-08-09T13:53:48.311545+0000","last_fullsized":"2023-08-09T13:53:48.311545+0000","mapping_epoch":127,"log_start":"0'0","ondisk_log_start":"0'0","created":127,"last_epoch_clean":128,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:53:47.231906+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:53:47.231906+0000","last_clean_scrub_stamp":"2023-08-09T13:53:47.231906+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T19:03:41.276282+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"16.1a","version":"128'1","reported_seq":12,"reported_epoch":128,"state":"active+clean","last_fresh":"2023-08-09T13:53:48.314262+0000","last_change":"2023-08-09T13:53:47.268608+0000","last_active":"2023-08-09T13:53:48.314262+0000","last_peered":"2023-08-09T13:53:48.314262+0000","last_clean":"2023-08-09T13:53:48.314262+0000","last_became_active":"2023-08-09T13:53:47.268202+0000","last_became_peered":"2023-08-09T13:53:47.268202+0000","last_unstale":"2023-08-09T13:53:48.314262+0000","last_undegraded":"2023-08-09T13:53:48.314262+0000","last_fullsized":"2023-08-09T13:53:48.314262+0000","mapping_epoch":126,"log_start":"0'0","ondisk_log_start":"0'0","created":126,"last_epoch_clean":127,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:53:46.216803+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:53:46.216803+0000","last_clean_scrub_stamp":"2023-08-09T13:53:46.216803+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-11T01:51:10.913522+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"17.18","version":"0'0","reported_seq":9,"reported_epoch":128,"state":"active+clean","last_fresh":"2023-08-09T13:53:48.305357+0000","last_change":"2023-08-09T13:53:48.299714+0000","last_active":"2023-08-09T13:53:48.305357+0000","last_peered":"2023-08-09T13:53:48.305357+0000","last_clean":"2023-08-09T13:53:48.305357+0000","last_became_active":"2023-08-09T13:53:48.299390+0000","last_became_peered":"2023-08-09T13:53:48.299390+0000","last_unstale":"2023-08-09T13:53:48.305357+0000","last_undegraded":"2023-08-09T13:53:48.305357+0000","last_fullsized":"2023-08-09T13:53:48.305357+0000","mapping_epoch":127,"log_start":"0'0","ondisk_log_start":"0'0","created":127,"last_epoch_clean":128,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:53:47.231906+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:53:47.231906+0000","last_clean_scrub_stamp":"2023-08-09T13:53:47.231906+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T19:31:11.030219+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.19","version":"0'0","reported_seq":11,"reported_epoch":128,"state":"active+clean","last_fresh":"2023-08-09T13:53:48.299334+0000","last_change":"2023-08-09T13:53:47.268290+0000","last_active":"2023-08-09T13:53:48.299334+0000","last_peered":"2023-08-09T13:53:48.299334+0000","last_clean":"2023-08-09T13:53:48.299334+0000","last_became_active":"2023-08-09T13:53:47.267889+0000","last_became_peered":"2023-08-09T13:53:47.267889+0000","last_unstale":"2023-08-09T13:53:48.299334+0000","last_undegraded":"2023-08-09T13:53:48.299334+0000","last_fullsized":"2023-08-09T13:53:48.299334+0000","mapping_epoch":126,"log_start":"0'0","ondisk_log_start":"0'0","created":126,"last_epoch_clean":127,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:53:46.216803+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:53:46.216803+0000","last_clean_scrub_stamp":"2023-08-09T13:53:46.216803+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T23:46:47.146096+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.19","version":"0'0","reported_seq":9,"reported_epoch":128,"state":"active+clean","last_fresh":"2023-08-09T13:53:48.320327+0000","last_change":"2023-08-09T13:53:48.317465+0000","last_active":"2023-08-09T13:53:48.320327+0000","last_peered":"2023-08-09T13:53:48.320327+0000","last_clean":"2023-08-09T13:53:48.320327+0000","last_became_active":"2023-08-09T13:53:48.316984+0000","last_became_peered":"2023-08-09T13:53:48.316984+0000","last_unstale":"2023-08-09T13:53:48.320327+0000","last_undegraded":"2023-08-09T13:53:48.320327+0000","last_fullsized":"2023-08-09T13:53:48.320327+0000","mapping_epoch":127,"log_start":"0'0","ondisk_log_start":"0'0","created":127,"last_epoch_clean":128,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:53:47.231906+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:53:47.231906+0000","last_clean_scrub_stamp":"2023-08-09T13:53:47.231906+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T14:44:39.079325+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"16.18","version":"0'0","reported_seq":11,"reported_epoch":128,"state":"active+clean","last_fresh":"2023-08-09T13:53:48.295239+0000","last_change":"2023-08-09T13:53:47.270563+0000","last_active":"2023-08-09T13:53:48.295239+0000","last_peered":"2023-08-09T13:53:48.295239+0000","last_clean":"2023-08-09T13:53:48.295239+0000","last_became_active":"2023-08-09T13:53:47.270261+0000","last_became_peered":"2023-08-09T13:53:47.270261+0000","last_unstale":"2023-08-09T13:53:48.295239+0000","last_undegraded":"2023-08-09T13:53:48.295239+0000","last_fullsized":"2023-08-09T13:53:48.295239+0000","mapping_epoch":126,"log_start":"0'0","ondisk_log_start":"0'0","created":126,"last_epoch_clean":127,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:53:46.216803+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:53:46.216803+0000","last_clean_scrub_stamp":"2023-08-09T13:53:46.216803+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T19:48:32.078103+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.e","version":"0'0","reported_seq":11,"reported_epoch":129,"state":"active+clean","last_fresh":"2023-08-09T13:53:49.282325+0000","last_change":"2023-08-09T13:53:48.303296+0000","last_active":"2023-08-09T13:53:49.282325+0000","last_peered":"2023-08-09T13:53:49.282325+0000","last_clean":"2023-08-09T13:53:49.282325+0000","last_became_active":"2023-08-09T13:53:48.303021+0000","last_became_peered":"2023-08-09T13:53:48.303021+0000","last_unstale":"2023-08-09T13:53:49.282325+0000","last_undegraded":"2023-08-09T13:53:49.282325+0000","last_fullsized":"2023-08-09T13:53:49.282325+0000","mapping_epoch":127,"log_start":"0'0","ondisk_log_start":"0'0","created":127,"last_epoch_clean":128,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:53:47.231906+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:53:47.231906+0000","last_clean_scrub_stamp":"2023-08-09T13:53:47.231906+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T18:10:57.405290+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"16.f","version":"128'1","reported_seq":12,"reported_epoch":128,"state":"active+clean","last_fresh":"2023-08-09T13:53:48.313334+0000","last_change":"2023-08-09T13:53:47.274042+0000","last_active":"2023-08-09T13:53:48.313334+0000","last_peered":"2023-08-09T13:53:48.313334+0000","last_clean":"2023-08-09T13:53:48.313334+0000","last_became_active":"2023-08-09T13:53:47.273755+0000","last_became_peered":"2023-08-09T13:53:47.273755+0000","last_unstale":"2023-08-09T13:53:48.313334+0000","last_undegraded":"2023-08-09T13:53:48.313334+0000","last_fullsized":"2023-08-09T13:53:48.313334+0000","mapping_epoch":126,"log_start":"0'0","ondisk_log_start":"0'0","created":126,"last_epoch_clean":127,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:53:46.216803+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:53:46.216803+0000","last_clean_scrub_stamp":"2023-08-09T13:53:46.216803+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T16:58:42.872129+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.f","version":"0'0","reported_seq":11,"reported_epoch":129,"state":"active+clean","last_fresh":"2023-08-09T13:53:49.277981+0000","last_change":"2023-08-09T13:53:48.299020+0000","last_active":"2023-08-09T13:53:49.277981+0000","last_peered":"2023-08-09T13:53:49.277981+0000","last_clean":"2023-08-09T13:53:49.277981+0000","last_became_active":"2023-08-09T13:53:48.298711+0000","last_became_peered":"2023-08-09T13:53:48.298711+0000","last_unstale":"2023-08-09T13:53:49.277981+0000","last_undegraded":"2023-08-09T13:53:49.277981+0000","last_fullsized":"2023-08-09T13:53:49.277981+0000","mapping_epoch":127,"log_start":"0'0","ondisk_log_start":"0'0","created":127,"last_epoch_clean":128,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:53:47.231906+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:53:47.231906+0000","last_clean_scrub_stamp":"2023-08-09T13:53:47.231906+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T23:23:17.088727+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"16.e","version":"128'2","reported_seq":15,"reported_epoch":129,"state":"active+clean","last_fresh":"2023-08-09T13:53:49.282359+0000","last_change":"2023-08-09T13:53:47.277672+0000","last_active":"2023-08-09T13:53:49.282359+0000","last_peered":"2023-08-09T13:53:49.282359+0000","last_clean":"2023-08-09T13:53:49.282359+0000","last_became_active":"2023-08-09T13:53:47.277392+0000","last_became_peered":"2023-08-09T13:53:47.277392+0000","last_unstale":"2023-08-09T13:53:49.282359+0000","last_undegraded":"2023-08-09T13:53:49.282359+0000","last_fullsized":"2023-08-09T13:53:49.282359+0000","mapping_epoch":126,"log_start":"0'0","ondisk_log_start":"0'0","created":126,"last_epoch_clean":127,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:53:46.216803+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:53:46.216803+0000","last_clean_scrub_stamp":"2023-08-09T13:53:46.216803+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T21:37:51.370269+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"17.c","version":"0'0","reported_seq":9,"reported_epoch":128,"state":"active+clean","last_fresh":"2023-08-09T13:53:48.313921+0000","last_change":"2023-08-09T13:53:48.308240+0000","last_active":"2023-08-09T13:53:48.313921+0000","last_peered":"2023-08-09T13:53:48.313921+0000","last_clean":"2023-08-09T13:53:48.313921+0000","last_became_active":"2023-08-09T13:53:48.307919+0000","last_became_peered":"2023-08-09T13:53:48.307919+0000","last_unstale":"2023-08-09T13:53:48.313921+0000","last_undegraded":"2023-08-09T13:53:48.313921+0000","last_fullsized":"2023-08-09T13:53:48.313921+0000","mapping_epoch":127,"log_start":"0'0","ondisk_log_start":"0'0","created":127,"last_epoch_clean":128,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:53:47.231906+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:53:47.231906+0000","last_clean_scrub_stamp":"2023-08-09T13:53:47.231906+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-11T01:25:07.887861+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"16.d","version":"128'1","reported_seq":14,"reported_epoch":129,"state":"active+clean","last_fresh":"2023-08-09T13:53:49.281087+0000","last_change":"2023-08-09T13:53:47.272814+0000","last_active":"2023-08-09T13:53:49.281087+0000","last_peered":"2023-08-09T13:53:49.281087+0000","last_clean":"2023-08-09T13:53:49.281087+0000","last_became_active":"2023-08-09T13:53:47.272527+0000","last_became_peered":"2023-08-09T13:53:47.272527+0000","last_unstale":"2023-08-09T13:53:49.281087+0000","last_undegraded":"2023-08-09T13:53:49.281087+0000","last_fullsized":"2023-08-09T13:53:49.281087+0000","mapping_epoch":126,"log_start":"0'0","ondisk_log_start":"0'0","created":126,"last_epoch_clean":127,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:53:46.216803+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:53:46.216803+0000","last_clean_scrub_stamp":"2023-08-09T13:53:46.216803+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T18:26:09.496138+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"17.d","version":"0'0","reported_seq":9,"reported_epoch":128,"state":"active+clean","last_fresh":"2023-08-09T13:53:48.313968+0000","last_change":"2023-08-09T13:53:48.308420+0000","last_active":"2023-08-09T13:53:48.313968+0000","last_peered":"2023-08-09T13:53:48.313968+0000","last_clean":"2023-08-09T13:53:48.313968+0000","last_became_active":"2023-08-09T13:53:48.306769+0000","last_became_peered":"2023-08-09T13:53:48.306769+0000","last_unstale":"2023-08-09T13:53:48.313968+0000","last_undegraded":"2023-08-09T13:53:48.313968+0000","last_fullsized":"2023-08-09T13:53:48.313968+0000","mapping_epoch":127,"log_start":"0'0","ondisk_log_start":"0'0","created":127,"last_epoch_clean":128,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:53:47.231906+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:53:47.231906+0000","last_clean_scrub_stamp":"2023-08-09T13:53:47.231906+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-11T00:49:07.518548+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"16.c","version":"0'0","reported_seq":11,"reported_epoch":128,"state":"active+clean","last_fresh":"2023-08-09T13:53:48.298264+0000","last_change":"2023-08-09T13:53:47.275981+0000","last_active":"2023-08-09T13:53:48.298264+0000","last_peered":"2023-08-09T13:53:48.298264+0000","last_clean":"2023-08-09T13:53:48.298264+0000","last_became_active":"2023-08-09T13:53:47.275694+0000","last_became_peered":"2023-08-09T13:53:47.275694+0000","last_unstale":"2023-08-09T13:53:48.298264+0000","last_undegraded":"2023-08-09T13:53:48.298264+0000","last_fullsized":"2023-08-09T13:53:48.298264+0000","mapping_epoch":126,"log_start":"0'0","ondisk_log_start":"0'0","created":126,"last_epoch_clean":127,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:53:46.216803+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:53:46.216803+0000","last_clean_scrub_stamp":"2023-08-09T13:53:46.216803+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-11T01:25:39.636954+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"17.a","version":"0'0","reported_seq":9,"reported_epoch":128,"state":"active+clean","last_fresh":"2023-08-09T13:53:48.325194+0000","last_change":"2023-08-09T13:53:48.318638+0000","last_active":"2023-08-09T13:53:48.325194+0000","last_peered":"2023-08-09T13:53:48.325194+0000","last_clean":"2023-08-09T13:53:48.325194+0000","last_became_active":"2023-08-09T13:53:48.318329+0000","last_became_peered":"2023-08-09T13:53:48.318329+0000","last_unstale":"2023-08-09T13:53:48.325194+0000","last_undegraded":"2023-08-09T13:53:48.325194+0000","last_fullsized":"2023-08-09T13:53:48.325194+0000","mapping_epoch":127,"log_start":"0'0","ondisk_log_start":"0'0","created":127,"last_epoch_clean":128,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:53:47.231906+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:53:47.231906+0000","last_clean_scrub_stamp":"2023-08-09T13:53:47.231906+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-11T01:16:24.402328+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"16.b","version":"128'2","reported_seq":15,"reported_epoch":129,"state":"active+clean","last_fresh":"2023-08-09T13:53:49.277577+0000","last_change":"2023-08-09T13:53:47.262109+0000","last_active":"2023-08-09T13:53:49.277577+0000","last_peered":"2023-08-09T13:53:49.277577+0000","last_clean":"2023-08-09T13:53:49.277577+0000","last_became_active":"2023-08-09T13:53:47.261837+0000","last_became_peered":"2023-08-09T13:53:47.261837+0000","last_unstale":"2023-08-09T13:53:49.277577+0000","last_undegraded":"2023-08-09T13:53:49.277577+0000","last_fullsized":"2023-08-09T13:53:49.277577+0000","mapping_epoch":126,"log_start":"0'0","ondisk_log_start":"0'0","created":126,"last_epoch_clean":127,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:53:46.216803+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:53:46.216803+0000","last_clean_scrub_stamp":"2023-08-09T13:53:46.216803+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T17:22:46.197957+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"17.b","version":"0'0","reported_seq":9,"reported_epoch":128,"state":"active+clean","last_fresh":"2023-08-09T13:53:48.320461+0000","last_change":"2023-08-09T13:53:48.315435+0000","last_active":"2023-08-09T13:53:48.320461+0000","last_peered":"2023-08-09T13:53:48.320461+0000","last_clean":"2023-08-09T13:53:48.320461+0000","last_became_active":"2023-08-09T13:53:48.315143+0000","last_became_peered":"2023-08-09T13:53:48.315143+0000","last_unstale":"2023-08-09T13:53:48.320461+0000","last_undegraded":"2023-08-09T13:53:48.320461+0000","last_fullsized":"2023-08-09T13:53:48.320461+0000","mapping_epoch":127,"log_start":"0'0","ondisk_log_start":"0'0","created":127,"last_epoch_clean":128,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:53:47.231906+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:53:47.231906+0000","last_clean_scrub_stamp":"2023-08-09T13:53:47.231906+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T21:21:34.369855+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.a","version":"0'0","reported_seq":13,"reported_epoch":129,"state":"active+clean","last_fresh":"2023-08-09T13:53:49.281565+0000","last_change":"2023-08-09T13:53:47.265204+0000","last_active":"2023-08-09T13:53:49.281565+0000","last_peered":"2023-08-09T13:53:49.281565+0000","last_clean":"2023-08-09T13:53:49.281565+0000","last_became_active":"2023-08-09T13:53:47.264864+0000","last_became_peered":"2023-08-09T13:53:47.264864+0000","last_unstale":"2023-08-09T13:53:49.281565+0000","last_undegraded":"2023-08-09T13:53:49.281565+0000","last_fullsized":"2023-08-09T13:53:49.281565+0000","mapping_epoch":126,"log_start":"0'0","ondisk_log_start":"0'0","created":126,"last_epoch_clean":127,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:53:46.216803+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:53:46.216803+0000","last_clean_scrub_stamp":"2023-08-09T13:53:46.216803+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T15:51:16.040950+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"17.8","version":"0'0","reported_seq":9,"reported_epoch":128,"state":"active+clean","last_fresh":"2023-08-09T13:53:48.325414+0000","last_change":"2023-08-09T13:53:48.321375+0000","last_active":"2023-08-09T13:53:48.325414+0000","last_peered":"2023-08-09T13:53:48.325414+0000","last_clean":"2023-08-09T13:53:48.325414+0000","last_became_active":"2023-08-09T13:53:48.320785+0000","last_became_peered":"2023-08-09T13:53:48.320785+0000","last_unstale":"2023-08-09T13:53:48.325414+0000","last_undegraded":"2023-08-09T13:53:48.325414+0000","last_fullsized":"2023-08-09T13:53:48.325414+0000","mapping_epoch":127,"log_start":"0'0","ondisk_log_start":"0'0","created":127,"last_epoch_clean":128,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:53:47.231906+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:53:47.231906+0000","last_clean_scrub_stamp":"2023-08-09T13:53:47.231906+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T21:55:12.741547+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"16.9","version":"0'0","reported_seq":13,"reported_epoch":129,"state":"active+clean","last_fresh":"2023-08-09T13:53:49.277985+0000","last_change":"2023-08-09T13:53:47.259951+0000","last_active":"2023-08-09T13:53:49.277985+0000","last_peered":"2023-08-09T13:53:49.277985+0000","last_clean":"2023-08-09T13:53:49.277985+0000","last_became_active":"2023-08-09T13:53:47.259603+0000","last_became_peered":"2023-08-09T13:53:47.259603+0000","last_unstale":"2023-08-09T13:53:49.277985+0000","last_undegraded":"2023-08-09T13:53:49.277985+0000","last_fullsized":"2023-08-09T13:53:49.277985+0000","mapping_epoch":126,"log_start":"0'0","ondisk_log_start":"0'0","created":126,"last_epoch_clean":127,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:53:46.216803+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:53:46.216803+0000","last_clean_scrub_stamp":"2023-08-09T13:53:46.216803+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T14:01:36.656215+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"17.9","version":"0'0","reported_seq":9,"reported_epoch":128,"state":"active+clean","last_fresh":"2023-08-09T13:53:48.303798+0000","last_change":"2023-08-09T13:53:48.300390+0000","last_active":"2023-08-09T13:53:48.303798+0000","last_peered":"2023-08-09T13:53:48.303798+0000","last_clean":"2023-08-09T13:53:48.303798+0000","last_became_active":"2023-08-09T13:53:48.300109+0000","last_became_peered":"2023-08-09T13:53:48.300109+0000","last_unstale":"2023-08-09T13:53:48.303798+0000","last_undegraded":"2023-08-09T13:53:48.303798+0000","last_fullsized":"2023-08-09T13:53:48.303798+0000","mapping_epoch":127,"log_start":"0'0","ondisk_log_start":"0'0","created":127,"last_epoch_clean":128,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:53:47.231906+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:53:47.231906+0000","last_clean_scrub_stamp":"2023-08-09T13:53:47.231906+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T23:13:06.550538+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"16.8","version":"0'0","reported_seq":11,"reported_epoch":128,"state":"active+clean","last_fresh":"2023-08-09T13:53:48.295384+0000","last_change":"2023-08-09T13:53:47.275496+0000","last_active":"2023-08-09T13:53:48.295384+0000","last_peered":"2023-08-09T13:53:48.295384+0000","last_clean":"2023-08-09T13:53:48.295384+0000","last_became_active":"2023-08-09T13:53:47.274904+0000","last_became_peered":"2023-08-09T13:53:47.274904+0000","last_unstale":"2023-08-09T13:53:48.295384+0000","last_undegraded":"2023-08-09T13:53:48.295384+0000","last_fullsized":"2023-08-09T13:53:48.295384+0000","mapping_epoch":126,"log_start":"0'0","ondisk_log_start":"0'0","created":126,"last_epoch_clean":127,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:53:46.216803+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:53:46.216803+0000","last_clean_scrub_stamp":"2023-08-09T13:53:46.216803+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T22:31:34.080611+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.6","version":"0'0","reported_seq":11,"reported_epoch":129,"state":"active+clean","last_fresh":"2023-08-09T13:53:49.281107+0000","last_change":"2023-08-09T13:53:48.316445+0000","last_active":"2023-08-09T13:53:49.281107+0000","last_peered":"2023-08-09T13:53:49.281107+0000","last_clean":"2023-08-09T13:53:49.281107+0000","last_became_active":"2023-08-09T13:53:48.316150+0000","last_became_peered":"2023-08-09T13:53:48.316150+0000","last_unstale":"2023-08-09T13:53:49.281107+0000","last_undegraded":"2023-08-09T13:53:49.281107+0000","last_fullsized":"2023-08-09T13:53:49.281107+0000","mapping_epoch":127,"log_start":"0'0","ondisk_log_start":"0'0","created":127,"last_epoch_clean":128,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:53:47.231906+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:53:47.231906+0000","last_clean_scrub_stamp":"2023-08-09T13:53:47.231906+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-11T01:45:02.929600+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.7","version":"128'2","reported_seq":13,"reported_epoch":128,"state":"active+clean","last_fresh":"2023-08-09T13:53:48.315830+0000","last_change":"2023-08-09T13:53:47.276360+0000","last_active":"2023-08-09T13:53:48.315830+0000","last_peered":"2023-08-09T13:53:48.315830+0000","last_clean":"2023-08-09T13:53:48.315830+0000","last_became_active":"2023-08-09T13:53:47.275979+0000","last_became_peered":"2023-08-09T13:53:47.275979+0000","last_unstale":"2023-08-09T13:53:48.315830+0000","last_undegraded":"2023-08-09T13:53:48.315830+0000","last_fullsized":"2023-08-09T13:53:48.315830+0000","mapping_epoch":126,"log_start":"0'0","ondisk_log_start":"0'0","created":126,"last_epoch_clean":127,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:53:46.216803+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:53:46.216803+0000","last_clean_scrub_stamp":"2023-08-09T13:53:46.216803+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T18:19:49.568627+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"17.7","version":"0'0","reported_seq":9,"reported_epoch":128,"state":"active+clean","last_fresh":"2023-08-09T13:53:48.320514+0000","last_change":"2023-08-09T13:53:48.314920+0000","last_active":"2023-08-09T13:53:48.320514+0000","last_peered":"2023-08-09T13:53:48.320514+0000","last_clean":"2023-08-09T13:53:48.320514+0000","last_became_active":"2023-08-09T13:53:48.314355+0000","last_became_peered":"2023-08-09T13:53:48.314355+0000","last_unstale":"2023-08-09T13:53:48.320514+0000","last_undegraded":"2023-08-09T13:53:48.320514+0000","last_fullsized":"2023-08-09T13:53:48.320514+0000","mapping_epoch":127,"log_start":"0'0","ondisk_log_start":"0'0","created":127,"last_epoch_clean":128,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:53:47.231906+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:53:47.231906+0000","last_clean_scrub_stamp":"2023-08-09T13:53:47.231906+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T20:47:17.015473+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.6","version":"0'0","reported_seq":13,"reported_epoch":129,"state":"active+clean","last_fresh":"2023-08-09T13:53:49.281608+0000","last_change":"2023-08-09T13:53:47.273109+0000","last_active":"2023-08-09T13:53:49.281608+0000","last_peered":"2023-08-09T13:53:49.281608+0000","last_clean":"2023-08-09T13:53:49.281608+0000","last_became_active":"2023-08-09T13:53:47.272823+0000","last_became_peered":"2023-08-09T13:53:47.272823+0000","last_unstale":"2023-08-09T13:53:49.281608+0000","last_undegraded":"2023-08-09T13:53:49.281608+0000","last_fullsized":"2023-08-09T13:53:49.281608+0000","mapping_epoch":126,"log_start":"0'0","ondisk_log_start":"0'0","created":126,"last_epoch_clean":127,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:53:46.216803+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:53:46.216803+0000","last_clean_scrub_stamp":"2023-08-09T13:53:46.216803+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T15:18:35.611711+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"17.4","version":"0'0","reported_seq":9,"reported_epoch":128,"state":"active+clean","last_fresh":"2023-08-09T13:53:48.307105+0000","last_change":"2023-08-09T13:53:48.299202+0000","last_active":"2023-08-09T13:53:48.307105+0000","last_peered":"2023-08-09T13:53:48.307105+0000","last_clean":"2023-08-09T13:53:48.307105+0000","last_became_active":"2023-08-09T13:53:48.298876+0000","last_became_peered":"2023-08-09T13:53:48.298876+0000","last_unstale":"2023-08-09T13:53:48.307105+0000","last_undegraded":"2023-08-09T13:53:48.307105+0000","last_fullsized":"2023-08-09T13:53:48.307105+0000","mapping_epoch":127,"log_start":"0'0","ondisk_log_start":"0'0","created":127,"last_epoch_clean":128,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:53:47.231906+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:53:47.231906+0000","last_clean_scrub_stamp":"2023-08-09T13:53:47.231906+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T18:20:43.514302+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.5","version":"0'0","reported_seq":11,"reported_epoch":128,"state":"active+clean","last_fresh":"2023-08-09T13:53:48.297085+0000","last_change":"2023-08-09T13:53:47.270618+0000","last_active":"2023-08-09T13:53:48.297085+0000","last_peered":"2023-08-09T13:53:48.297085+0000","last_clean":"2023-08-09T13:53:48.297085+0000","last_became_active":"2023-08-09T13:53:47.270323+0000","last_became_peered":"2023-08-09T13:53:47.270323+0000","last_unstale":"2023-08-09T13:53:48.297085+0000","last_undegraded":"2023-08-09T13:53:48.297085+0000","last_fullsized":"2023-08-09T13:53:48.297085+0000","mapping_epoch":126,"log_start":"0'0","ondisk_log_start":"0'0","created":126,"last_epoch_clean":127,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:53:46.216803+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:53:46.216803+0000","last_clean_scrub_stamp":"2023-08-09T13:53:46.216803+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T14:53:29.005519+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.5","version":"0'0","reported_seq":11,"reported_epoch":129,"state":"active+clean","last_fresh":"2023-08-09T13:53:49.281314+0000","last_change":"2023-08-09T13:53:48.316215+0000","last_active":"2023-08-09T13:53:49.281314+0000","last_peered":"2023-08-09T13:53:49.281314+0000","last_clean":"2023-08-09T13:53:49.281314+0000","last_became_active":"2023-08-09T13:53:48.315918+0000","last_became_peered":"2023-08-09T13:53:48.315918+0000","last_unstale":"2023-08-09T13:53:49.281314+0000","last_undegraded":"2023-08-09T13:53:49.281314+0000","last_fullsized":"2023-08-09T13:53:49.281314+0000","mapping_epoch":127,"log_start":"0'0","ondisk_log_start":"0'0","created":127,"last_epoch_clean":128,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:53:47.231906+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:53:47.231906+0000","last_clean_scrub_stamp":"2023-08-09T13:53:47.231906+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T17:16:33.909230+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.4","version":"0'0","reported_seq":11,"reported_epoch":128,"state":"active+clean","last_fresh":"2023-08-09T13:53:48.278713+0000","last_change":"2023-08-09T13:53:47.268293+0000","last_active":"2023-08-09T13:53:48.278713+0000","last_peered":"2023-08-09T13:53:48.278713+0000","last_clean":"2023-08-09T13:53:48.278713+0000","last_became_active":"2023-08-09T13:53:47.268002+0000","last_became_peered":"2023-08-09T13:53:47.268002+0000","last_unstale":"2023-08-09T13:53:48.278713+0000","last_undegraded":"2023-08-09T13:53:48.278713+0000","last_fullsized":"2023-08-09T13:53:48.278713+0000","mapping_epoch":126,"log_start":"0'0","ondisk_log_start":"0'0","created":126,"last_epoch_clean":127,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:53:46.216803+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:53:46.216803+0000","last_clean_scrub_stamp":"2023-08-09T13:53:46.216803+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T18:45:40.148027+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"17.2","version":"0'0","reported_seq":9,"reported_epoch":128,"state":"active+clean","last_fresh":"2023-08-09T13:53:48.312284+0000","last_change":"2023-08-09T13:53:48.311397+0000","last_active":"2023-08-09T13:53:48.312284+0000","last_peered":"2023-08-09T13:53:48.312284+0000","last_clean":"2023-08-09T13:53:48.312284+0000","last_became_active":"2023-08-09T13:53:48.310984+0000","last_became_peered":"2023-08-09T13:53:48.310984+0000","last_unstale":"2023-08-09T13:53:48.312284+0000","last_undegraded":"2023-08-09T13:53:48.312284+0000","last_fullsized":"2023-08-09T13:53:48.312284+0000","mapping_epoch":127,"log_start":"0'0","ondisk_log_start":"0'0","created":127,"last_epoch_clean":128,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:53:47.231906+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:53:47.231906+0000","last_clean_scrub_stamp":"2023-08-09T13:53:47.231906+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T22:47:39.839446+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"16.3","version":"0'0","reported_seq":11,"reported_epoch":128,"state":"active+clean","last_fresh":"2023-08-09T13:53:48.298474+0000","last_change":"2023-08-09T13:53:47.270868+0000","last_active":"2023-08-09T13:53:48.298474+0000","last_peered":"2023-08-09T13:53:48.298474+0000","last_clean":"2023-08-09T13:53:48.298474+0000","last_became_active":"2023-08-09T13:53:47.270486+0000","last_became_peered":"2023-08-09T13:53:47.270486+0000","last_unstale":"2023-08-09T13:53:48.298474+0000","last_undegraded":"2023-08-09T13:53:48.298474+0000","last_fullsized":"2023-08-09T13:53:48.298474+0000","mapping_epoch":126,"log_start":"0'0","ondisk_log_start":"0'0","created":126,"last_epoch_clean":127,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:53:46.216803+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:53:46.216803+0000","last_clean_scrub_stamp":"2023-08-09T13:53:46.216803+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-11T00:46:47.133310+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"17.3","version":"0'0","reported_seq":9,"reported_epoch":128,"state":"active+clean","last_fresh":"2023-08-09T13:53:48.325042+0000","last_change":"2023-08-09T13:53:48.321816+0000","last_active":"2023-08-09T13:53:48.325042+0000","last_peered":"2023-08-09T13:53:48.325042+0000","last_clean":"2023-08-09T13:53:48.325042+0000","last_became_active":"2023-08-09T13:53:48.321517+0000","last_became_peered":"2023-08-09T13:53:48.321517+0000","last_unstale":"2023-08-09T13:53:48.325042+0000","last_undegraded":"2023-08-09T13:53:48.325042+0000","last_fullsized":"2023-08-09T13:53:48.325042+0000","mapping_epoch":127,"log_start":"0'0","ondisk_log_start":"0'0","created":127,"last_epoch_clean":128,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:53:47.231906+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:53:47.231906+0000","last_clean_scrub_stamp":"2023-08-09T13:53:47.231906+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T20:39:00.369183+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"16.2","version":"0'0","reported_seq":11,"reported_epoch":128,"state":"active+clean","last_fresh":"2023-08-09T13:53:48.298754+0000","last_change":"2023-08-09T13:53:47.265197+0000","last_active":"2023-08-09T13:53:48.298754+0000","last_peered":"2023-08-09T13:53:48.298754+0000","last_clean":"2023-08-09T13:53:48.298754+0000","last_became_active":"2023-08-09T13:53:47.264692+0000","last_became_peered":"2023-08-09T13:53:47.264692+0000","last_unstale":"2023-08-09T13:53:48.298754+0000","last_undegraded":"2023-08-09T13:53:48.298754+0000","last_fullsized":"2023-08-09T13:53:48.298754+0000","mapping_epoch":126,"log_start":"0'0","ondisk_log_start":"0'0","created":126,"last_epoch_clean":127,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:53:46.216803+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:53:46.216803+0000","last_clean_scrub_stamp":"2023-08-09T13:53:46.216803+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T15:13:29.347832+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.0","version":"0'0","reported_seq":9,"reported_epoch":128,"state":"active+clean","last_fresh":"2023-08-09T13:53:48.316400+0000","last_change":"2023-08-09T13:53:48.312958+0000","last_active":"2023-08-09T13:53:48.316400+0000","last_peered":"2023-08-09T13:53:48.316400+0000","last_clean":"2023-08-09T13:53:48.316400+0000","last_became_active":"2023-08-09T13:53:48.312643+0000","last_became_peered":"2023-08-09T13:53:48.312643+0000","last_unstale":"2023-08-09T13:53:48.316400+0000","last_undegraded":"2023-08-09T13:53:48.316400+0000","last_fullsized":"2023-08-09T13:53:48.316400+0000","mapping_epoch":127,"log_start":"0'0","ondisk_log_start":"0'0","created":127,"last_epoch_clean":128,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:53:47.231906+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:53:47.231906+0000","last_clean_scrub_stamp":"2023-08-09T13:53:47.231906+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-11T00:18:27.190529+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"16.1","version":"0'0","reported_seq":13,"reported_epoch":129,"state":"active+clean","last_fresh":"2023-08-09T13:53:49.281590+0000","last_change":"2023-08-09T13:53:47.265424+0000","last_active":"2023-08-09T13:53:49.281590+0000","last_peered":"2023-08-09T13:53:49.281590+0000","last_clean":"2023-08-09T13:53:49.281590+0000","last_became_active":"2023-08-09T13:53:47.265115+0000","last_became_peered":"2023-08-09T13:53:47.265115+0000","last_unstale":"2023-08-09T13:53:49.281590+0000","last_undegraded":"2023-08-09T13:53:49.281590+0000","last_fullsized":"2023-08-09T13:53:49.281590+0000","mapping_epoch":126,"log_start":"0'0","ondisk_log_start":"0'0","created":126,"last_epoch_clean":127,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:53:46.216803+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:53:46.216803+0000","last_clean_scrub_stamp":"2023-08-09T13:53:46.216803+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T14:19:30.281030+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"17.1","version":"0'0","reported_seq":9,"reported_epoch":128,"state":"active+clean","last_fresh":"2023-08-09T13:53:48.305307+0000","last_change":"2023-08-09T13:53:48.299324+0000","last_active":"2023-08-09T13:53:48.305307+0000","last_peered":"2023-08-09T13:53:48.305307+0000","last_clean":"2023-08-09T13:53:48.305307+0000","last_became_active":"2023-08-09T13:53:48.299002+0000","last_became_peered":"2023-08-09T13:53:48.299002+0000","last_unstale":"2023-08-09T13:53:48.305307+0000","last_undegraded":"2023-08-09T13:53:48.305307+0000","last_fullsized":"2023-08-09T13:53:48.305307+0000","mapping_epoch":127,"log_start":"0'0","ondisk_log_start":"0'0","created":127,"last_epoch_clean":128,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:53:47.231906+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:53:47.231906+0000","last_clean_scrub_stamp":"2023-08-09T13:53:47.231906+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T20:17:37.885150+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.0","version":"0'0","reported_seq":11,"reported_epoch":128,"state":"active+clean","last_fresh":"2023-08-09T13:53:48.295813+0000","last_change":"2023-08-09T13:53:47.273598+0000","last_active":"2023-08-09T13:53:48.295813+0000","last_peered":"2023-08-09T13:53:48.295813+0000","last_clean":"2023-08-09T13:53:48.295813+0000","last_became_active":"2023-08-09T13:53:47.273269+0000","last_became_peered":"2023-08-09T13:53:47.273269+0000","last_unstale":"2023-08-09T13:53:48.295813+0000","last_undegraded":"2023-08-09T13:53:48.295813+0000","last_fullsized":"2023-08-09T13:53:48.295813+0000","mapping_epoch":126,"log_start":"0'0","ondisk_log_start":"0'0","created":126,"last_epoch_clean":127,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:53:46.216803+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:53:46.216803+0000","last_clean_scrub_stamp":"2023-08-09T13:53:46.216803+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T17:36:59.208559+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.10","version":"0'0","reported_seq":9,"reported_epoch":128,"state":"active+clean","last_fresh":"2023-08-09T13:53:48.315569+0000","last_change":"2023-08-09T13:53:48.313515+0000","last_active":"2023-08-09T13:53:48.315569+0000","last_peered":"2023-08-09T13:53:48.315569+0000","last_clean":"2023-08-09T13:53:48.315569+0000","last_became_active":"2023-08-09T13:53:48.312541+0000","last_became_peered":"2023-08-09T13:53:48.312541+0000","last_unstale":"2023-08-09T13:53:48.315569+0000","last_undegraded":"2023-08-09T13:53:48.315569+0000","last_fullsized":"2023-08-09T13:53:48.315569+0000","mapping_epoch":127,"log_start":"0'0","ondisk_log_start":"0'0","created":127,"last_epoch_clean":128,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:53:47.231906+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:53:47.231906+0000","last_clean_scrub_stamp":"2023-08-09T13:53:47.231906+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T16:36:31.354568+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"16.11","version":"0'0","reported_seq":11,"reported_epoch":128,"state":"active+clean","last_fresh":"2023-08-09T13:53:48.302872+0000","last_change":"2023-08-09T13:53:47.276027+0000","last_active":"2023-08-09T13:53:48.302872+0000","last_peered":"2023-08-09T13:53:48.302872+0000","last_clean":"2023-08-09T13:53:48.302872+0000","last_became_active":"2023-08-09T13:53:47.275740+0000","last_became_peered":"2023-08-09T13:53:47.275740+0000","last_unstale":"2023-08-09T13:53:48.302872+0000","last_undegraded":"2023-08-09T13:53:48.302872+0000","last_fullsized":"2023-08-09T13:53:48.302872+0000","mapping_epoch":126,"log_start":"0'0","ondisk_log_start":"0'0","created":126,"last_epoch_clean":127,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:53:46.216803+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:53:46.216803+0000","last_clean_scrub_stamp":"2023-08-09T13:53:46.216803+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-11T01:50:52.067295+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":214,"reported_epoch":128,"state":"active+clean","last_fresh":"2023-08-09T13:53:48.294940+0000","last_change":"2023-08-09T13:33:56.401406+0000","last_active":"2023-08-09T13:53:48.294940+0000","last_peered":"2023-08-09T13:53:48.294940+0000","last_clean":"2023-08-09T13:53:48.294940+0000","last_became_active":"2023-08-09T13:33:56.400487+0000","last_became_peered":"2023-08-09T13:33:56.400487+0000","last_unstale":"2023-08-09T13:53:48.294940+0000","last_undegraded":"2023-08-09T13:53:48.294940+0000","last_fullsized":"2023-08-09T13:53:48.294940+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:33:54.845869+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:33:54.845869+0000","last_clean_scrub_stamp":"2023-08-09T13:33:54.845869+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T21:20:59.655753+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"17.11","version":"0'0","reported_seq":9,"reported_epoch":128,"state":"active+clean","last_fresh":"2023-08-09T13:53:48.308285+0000","last_change":"2023-08-09T13:53:48.306830+0000","last_active":"2023-08-09T13:53:48.308285+0000","last_peered":"2023-08-09T13:53:48.308285+0000","last_clean":"2023-08-09T13:53:48.308285+0000","last_became_active":"2023-08-09T13:53:48.306545+0000","last_became_peered":"2023-08-09T13:53:48.306545+0000","last_unstale":"2023-08-09T13:53:48.308285+0000","last_undegraded":"2023-08-09T13:53:48.308285+0000","last_fullsized":"2023-08-09T13:53:48.308285+0000","mapping_epoch":127,"log_start":"0'0","ondisk_log_start":"0'0","created":127,"last_epoch_clean":128,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:53:47.231906+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:53:47.231906+0000","last_clean_scrub_stamp":"2023-08-09T13:53:47.231906+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T19:27:21.275360+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"16.10","version":"128'8","reported_seq":21,"reported_epoch":129,"state":"active+clean","last_fresh":"2023-08-09T13:53:49.281342+0000","last_change":"2023-08-09T13:53:47.276063+0000","last_active":"2023-08-09T13:53:49.281342+0000","last_peered":"2023-08-09T13:53:49.281342+0000","last_clean":"2023-08-09T13:53:49.281342+0000","last_became_active":"2023-08-09T13:53:47.275739+0000","last_became_peered":"2023-08-09T13:53:47.275739+0000","last_unstale":"2023-08-09T13:53:49.281342+0000","last_undegraded":"2023-08-09T13:53:49.281342+0000","last_fullsized":"2023-08-09T13:53:49.281342+0000","mapping_epoch":126,"log_start":"0'0","ondisk_log_start":"0'0","created":126,"last_epoch_clean":127,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:53:46.216803+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:53:46.216803+0000","last_clean_scrub_stamp":"2023-08-09T13:53:46.216803+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T14:24:59.862405+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":218,"reported_epoch":128,"state":"active+clean","last_fresh":"2023-08-09T13:53:48.302828+0000","last_change":"2023-08-09T13:33:56.401123+0000","last_active":"2023-08-09T13:53:48.302828+0000","last_peered":"2023-08-09T13:53:48.302828+0000","last_clean":"2023-08-09T13:53:48.302828+0000","last_became_active":"2023-08-09T13:33:56.399933+0000","last_became_peered":"2023-08-09T13:33:56.399933+0000","last_unstale":"2023-08-09T13:53:48.302828+0000","last_undegraded":"2023-08-09T13:53:48.302828+0000","last_fullsized":"2023-08-09T13:53:48.302828+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:33:54.845869+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:33:54.845869+0000","last_clean_scrub_stamp":"2023-08-09T13:33:54.845869+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T21:17:47.698878+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"17.12","version":"0'0","reported_seq":9,"reported_epoch":128,"state":"active+clean","last_fresh":"2023-08-09T13:53:48.314743+0000","last_change":"2023-08-09T13:53:48.312415+0000","last_active":"2023-08-09T13:53:48.314743+0000","last_peered":"2023-08-09T13:53:48.314743+0000","last_clean":"2023-08-09T13:53:48.314743+0000","last_became_active":"2023-08-09T13:53:48.311897+0000","last_became_peered":"2023-08-09T13:53:48.311897+0000","last_unstale":"2023-08-09T13:53:48.314743+0000","last_undegraded":"2023-08-09T13:53:48.314743+0000","last_fullsized":"2023-08-09T13:53:48.314743+0000","mapping_epoch":127,"log_start":"0'0","ondisk_log_start":"0'0","created":127,"last_epoch_clean":128,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:53:47.231906+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:53:47.231906+0000","last_clean_scrub_stamp":"2023-08-09T13:53:47.231906+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T16:50:01.243839+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"16.13","version":"128'2","reported_seq":15,"reported_epoch":129,"state":"active+clean","last_fresh":"2023-08-09T13:53:49.277360+0000","last_change":"2023-08-09T13:53:47.266179+0000","last_active":"2023-08-09T13:53:49.277360+0000","last_peered":"2023-08-09T13:53:49.277360+0000","last_clean":"2023-08-09T13:53:49.277360+0000","last_became_active":"2023-08-09T13:53:47.265894+0000","last_became_peered":"2023-08-09T13:53:47.265894+0000","last_unstale":"2023-08-09T13:53:49.277360+0000","last_undegraded":"2023-08-09T13:53:49.277360+0000","last_fullsized":"2023-08-09T13:53:49.277360+0000","mapping_epoch":126,"log_start":"0'0","ondisk_log_start":"0'0","created":126,"last_epoch_clean":127,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:53:46.216803+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:53:46.216803+0000","last_clean_scrub_stamp":"2023-08-09T13:53:46.216803+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T18:22:05.646686+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.2","version":"0'0","reported_seq":214,"reported_epoch":128,"state":"active+clean","last_fresh":"2023-08-09T13:53:48.292200+0000","last_change":"2023-08-09T13:33:56.402794+0000","last_active":"2023-08-09T13:53:48.292200+0000","last_peered":"2023-08-09T13:53:48.292200+0000","last_clean":"2023-08-09T13:53:48.292200+0000","last_became_active":"2023-08-09T13:33:56.402038+0000","last_became_peered":"2023-08-09T13:33:56.402038+0000","last_unstale":"2023-08-09T13:53:48.292200+0000","last_undegraded":"2023-08-09T13:53:48.292200+0000","last_fullsized":"2023-08-09T13:53:48.292200+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:33:54.845869+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:33:54.845869+0000","last_clean_scrub_stamp":"2023-08-09T13:33:54.845869+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T14:41:45.274949+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"17.13","version":"0'0","reported_seq":9,"reported_epoch":128,"state":"active+clean","last_fresh":"2023-08-09T13:53:48.315226+0000","last_change":"2023-08-09T13:53:48.313299+0000","last_active":"2023-08-09T13:53:48.315226+0000","last_peered":"2023-08-09T13:53:48.315226+0000","last_clean":"2023-08-09T13:53:48.315226+0000","last_became_active":"2023-08-09T13:53:48.312896+0000","last_became_peered":"2023-08-09T13:53:48.312896+0000","last_unstale":"2023-08-09T13:53:48.315226+0000","last_undegraded":"2023-08-09T13:53:48.315226+0000","last_fullsized":"2023-08-09T13:53:48.315226+0000","mapping_epoch":127,"log_start":"0'0","ondisk_log_start":"0'0","created":127,"last_epoch_clean":128,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:53:47.231906+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:53:47.231906+0000","last_clean_scrub_stamp":"2023-08-09T13:53:47.231906+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-11T00:23:04.961463+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"16.12","version":"0'0","reported_seq":13,"reported_epoch":129,"state":"active+clean","last_fresh":"2023-08-09T13:53:49.280916+0000","last_change":"2023-08-09T13:53:47.274711+0000","last_active":"2023-08-09T13:53:49.280916+0000","last_peered":"2023-08-09T13:53:49.280916+0000","last_clean":"2023-08-09T13:53:49.280916+0000","last_became_active":"2023-08-09T13:53:47.274348+0000","last_became_peered":"2023-08-09T13:53:47.274348+0000","last_unstale":"2023-08-09T13:53:49.280916+0000","last_undegraded":"2023-08-09T13:53:49.280916+0000","last_fullsized":"2023-08-09T13:53:49.280916+0000","mapping_epoch":126,"log_start":"0'0","ondisk_log_start":"0'0","created":126,"last_epoch_clean":127,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:53:46.216803+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:53:46.216803+0000","last_clean_scrub_stamp":"2023-08-09T13:53:46.216803+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-11T01:24:41.202067+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"1.3","version":"0'0","reported_seq":216,"reported_epoch":129,"state":"active+clean","last_fresh":"2023-08-09T13:53:49.277683+0000","last_change":"2023-08-09T13:33:55.886749+0000","last_active":"2023-08-09T13:53:49.277683+0000","last_peered":"2023-08-09T13:53:49.277683+0000","last_clean":"2023-08-09T13:53:49.277683+0000","last_became_active":"2023-08-09T13:33:55.886116+0000","last_became_peered":"2023-08-09T13:33:55.886116+0000","last_unstale":"2023-08-09T13:53:49.277683+0000","last_undegraded":"2023-08-09T13:53:49.277683+0000","last_fullsized":"2023-08-09T13:53:49.277683+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:33:54.845869+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:33:54.845869+0000","last_clean_scrub_stamp":"2023-08-09T13:33:54.845869+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T20:49:11.516353+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"118'208","reported_seq":686,"reported_epoch":128,"state":"active+clean","last_fresh":"2023-08-09T13:53:48.294615+0000","last_change":"2023-08-09T13:33:55.886345+0000","last_active":"2023-08-09T13:53:48.294615+0000","last_peered":"2023-08-09T13:53:48.294615+0000","last_clean":"2023-08-09T13:53:48.294615+0000","last_became_active":"2023-08-09T13:33:55.885781+0000","last_became_peered":"2023-08-09T13:33:55.885781+0000","last_unstale":"2023-08-09T13:53:48.294615+0000","last_undegraded":"2023-08-09T13:53:48.294615+0000","last_fullsized":"2023-08-09T13:53:48.294615+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:33:54.845869+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:33:54.845869+0000","last_clean_scrub_stamp":"2023-08-09T13:33:54.845869+0000","objects_scrubbed":0,"log_size":208,"log_dups_size":0,"ondisk_log_size":208,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T15:52:46.028204+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":907,"num_read_kb":1830,"num_write":323,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"17.14","version":"0'0","reported_seq":9,"reported_epoch":128,"state":"active+clean","last_fresh":"2023-08-09T13:53:48.315262+0000","last_change":"2023-08-09T13:53:48.313896+0000","last_active":"2023-08-09T13:53:48.315262+0000","last_peered":"2023-08-09T13:53:48.315262+0000","last_clean":"2023-08-09T13:53:48.315262+0000","last_became_active":"2023-08-09T13:53:48.312833+0000","last_became_peered":"2023-08-09T13:53:48.312833+0000","last_unstale":"2023-08-09T13:53:48.315262+0000","last_undegraded":"2023-08-09T13:53:48.315262+0000","last_fullsized":"2023-08-09T13:53:48.315262+0000","mapping_epoch":127,"log_start":"0'0","ondisk_log_start":"0'0","created":127,"last_epoch_clean":128,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:53:47.231906+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:53:47.231906+0000","last_clean_scrub_stamp":"2023-08-09T13:53:47.231906+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T22:23:52.085612+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"16.15","version":"128'3","reported_seq":14,"reported_epoch":128,"state":"active+clean","last_fresh":"2023-08-09T13:53:48.314131+0000","last_change":"2023-08-09T13:53:47.270479+0000","last_active":"2023-08-09T13:53:48.314131+0000","last_peered":"2023-08-09T13:53:48.314131+0000","last_clean":"2023-08-09T13:53:48.314131+0000","last_became_active":"2023-08-09T13:53:47.270175+0000","last_became_peered":"2023-08-09T13:53:47.270175+0000","last_unstale":"2023-08-09T13:53:48.314131+0000","last_undegraded":"2023-08-09T13:53:48.314131+0000","last_fullsized":"2023-08-09T13:53:48.314131+0000","mapping_epoch":126,"log_start":"0'0","ondisk_log_start":"0'0","created":126,"last_epoch_clean":127,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:53:46.216803+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:53:46.216803+0000","last_clean_scrub_stamp":"2023-08-09T13:53:46.216803+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T23:46:47.146096+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":224,"reported_epoch":128,"state":"active+clean","last_fresh":"2023-08-09T13:53:48.278822+0000","last_change":"2023-08-09T13:33:55.889565+0000","last_active":"2023-08-09T13:53:48.278822+0000","last_peered":"2023-08-09T13:53:48.278822+0000","last_clean":"2023-08-09T13:53:48.278822+0000","last_became_active":"2023-08-09T13:33:55.889164+0000","last_became_peered":"2023-08-09T13:33:55.889164+0000","last_unstale":"2023-08-09T13:53:48.278822+0000","last_undegraded":"2023-08-09T13:53:48.278822+0000","last_fullsized":"2023-08-09T13:53:48.278822+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:33:54.845869+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:33:54.845869+0000","last_clean_scrub_stamp":"2023-08-09T13:33:54.845869+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T23:39:41.751394+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"17.15","version":"0'0","reported_seq":9,"reported_epoch":128,"state":"active+clean","last_fresh":"2023-08-09T13:53:48.315544+0000","last_change":"2023-08-09T13:53:48.313041+0000","last_active":"2023-08-09T13:53:48.315544+0000","last_peered":"2023-08-09T13:53:48.315544+0000","last_clean":"2023-08-09T13:53:48.315544+0000","last_became_active":"2023-08-09T13:53:48.312748+0000","last_became_peered":"2023-08-09T13:53:48.312748+0000","last_unstale":"2023-08-09T13:53:48.315544+0000","last_undegraded":"2023-08-09T13:53:48.315544+0000","last_fullsized":"2023-08-09T13:53:48.315544+0000","mapping_epoch":127,"log_start":"0'0","ondisk_log_start":"0'0","created":127,"last_epoch_clean":128,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:53:47.231906+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:53:47.231906+0000","last_clean_scrub_stamp":"2023-08-09T13:53:47.231906+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T14:52:31.823210+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"16.14","version":"128'6","reported_seq":17,"reported_epoch":128,"state":"active+clean","last_fresh":"2023-08-09T13:53:48.323049+0000","last_change":"2023-08-09T13:53:47.274888+0000","last_active":"2023-08-09T13:53:48.323049+0000","last_peered":"2023-08-09T13:53:48.323049+0000","last_clean":"2023-08-09T13:53:48.323049+0000","last_became_active":"2023-08-09T13:53:47.274017+0000","last_became_peered":"2023-08-09T13:53:47.274017+0000","last_unstale":"2023-08-09T13:53:48.323049+0000","last_undegraded":"2023-08-09T13:53:48.323049+0000","last_fullsized":"2023-08-09T13:53:48.323049+0000","mapping_epoch":126,"log_start":"0'0","ondisk_log_start":"0'0","created":126,"last_epoch_clean":127,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:53:46.216803+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:53:46.216803+0000","last_clean_scrub_stamp":"2023-08-09T13:53:46.216803+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-11T00:53:15.909254+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":220,"reported_epoch":128,"state":"active+clean","last_fresh":"2023-08-09T13:53:48.289993+0000","last_change":"2023-08-09T13:33:56.404236+0000","last_active":"2023-08-09T13:53:48.289993+0000","last_peered":"2023-08-09T13:53:48.289993+0000","last_clean":"2023-08-09T13:53:48.289993+0000","last_became_active":"2023-08-09T13:33:56.402485+0000","last_became_peered":"2023-08-09T13:33:56.402485+0000","last_unstale":"2023-08-09T13:53:48.289993+0000","last_undegraded":"2023-08-09T13:53:48.289993+0000","last_fullsized":"2023-08-09T13:53:48.289993+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:33:54.845869+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:33:54.845869+0000","last_clean_scrub_stamp":"2023-08-09T13:33:54.845869+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T22:43:50.638469+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.16","version":"0'0","reported_seq":9,"reported_epoch":128,"state":"active+clean","last_fresh":"2023-08-09T13:53:48.321631+0000","last_change":"2023-08-09T13:53:48.320119+0000","last_active":"2023-08-09T13:53:48.321631+0000","last_peered":"2023-08-09T13:53:48.321631+0000","last_clean":"2023-08-09T13:53:48.321631+0000","last_became_active":"2023-08-09T13:53:48.319858+0000","last_became_peered":"2023-08-09T13:53:48.319858+0000","last_unstale":"2023-08-09T13:53:48.321631+0000","last_undegraded":"2023-08-09T13:53:48.321631+0000","last_fullsized":"2023-08-09T13:53:48.321631+0000","mapping_epoch":127,"log_start":"0'0","ondisk_log_start":"0'0","created":127,"last_epoch_clean":128,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:53:47.231906+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:53:47.231906+0000","last_clean_scrub_stamp":"2023-08-09T13:53:47.231906+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T16:42:55.642787+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"16.17","version":"0'0","reported_seq":13,"reported_epoch":129,"state":"active+clean","last_fresh":"2023-08-09T13:53:49.282026+0000","last_change":"2023-08-09T13:53:47.265046+0000","last_active":"2023-08-09T13:53:49.282026+0000","last_peered":"2023-08-09T13:53:49.282026+0000","last_clean":"2023-08-09T13:53:49.282026+0000","last_became_active":"2023-08-09T13:53:47.264705+0000","last_became_peered":"2023-08-09T13:53:47.264705+0000","last_unstale":"2023-08-09T13:53:49.282026+0000","last_undegraded":"2023-08-09T13:53:49.282026+0000","last_fullsized":"2023-08-09T13:53:49.282026+0000","mapping_epoch":126,"log_start":"0'0","ondisk_log_start":"0'0","created":126,"last_epoch_clean":127,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:53:46.216803+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:53:46.216803+0000","last_clean_scrub_stamp":"2023-08-09T13:53:46.216803+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T19:17:46.699216+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":216,"reported_epoch":129,"state":"active+clean","last_fresh":"2023-08-09T13:53:49.281443+0000","last_change":"2023-08-09T13:33:56.401781+0000","last_active":"2023-08-09T13:53:49.281443+0000","last_peered":"2023-08-09T13:53:49.281443+0000","last_clean":"2023-08-09T13:53:49.281443+0000","last_became_active":"2023-08-09T13:33:56.400336+0000","last_became_peered":"2023-08-09T13:33:56.400336+0000","last_unstale":"2023-08-09T13:53:49.281443+0000","last_undegraded":"2023-08-09T13:53:49.281443+0000","last_fullsized":"2023-08-09T13:53:49.281443+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:33:54.845869+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:33:54.845869+0000","last_clean_scrub_stamp":"2023-08-09T13:33:54.845869+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T17:53:59.742104+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"17.17","version":"0'0","reported_seq":11,"reported_epoch":129,"state":"active+clean","last_fresh":"2023-08-09T13:53:49.281833+0000","last_change":"2023-08-09T13:53:48.320410+0000","last_active":"2023-08-09T13:53:49.281833+0000","last_peered":"2023-08-09T13:53:49.281833+0000","last_clean":"2023-08-09T13:53:49.281833+0000","last_became_active":"2023-08-09T13:53:48.319554+0000","last_became_peered":"2023-08-09T13:53:48.319554+0000","last_unstale":"2023-08-09T13:53:49.281833+0000","last_undegraded":"2023-08-09T13:53:49.281833+0000","last_fullsized":"2023-08-09T13:53:49.281833+0000","mapping_epoch":127,"log_start":"0'0","ondisk_log_start":"0'0","created":127,"last_epoch_clean":128,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:53:47.231906+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:53:47.231906+0000","last_clean_scrub_stamp":"2023-08-09T13:53:47.231906+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T17:36:31.194931+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.16","version":"0'0","reported_seq":11,"reported_epoch":128,"state":"active+clean","last_fresh":"2023-08-09T13:53:48.277632+0000","last_change":"2023-08-09T13:53:47.272505+0000","last_active":"2023-08-09T13:53:48.277632+0000","last_peered":"2023-08-09T13:53:48.277632+0000","last_clean":"2023-08-09T13:53:48.277632+0000","last_became_active":"2023-08-09T13:53:47.272230+0000","last_became_peered":"2023-08-09T13:53:47.272230+0000","last_unstale":"2023-08-09T13:53:48.277632+0000","last_undegraded":"2023-08-09T13:53:48.277632+0000","last_fullsized":"2023-08-09T13:53:48.277632+0000","mapping_epoch":126,"log_start":"0'0","ondisk_log_start":"0'0","created":126,"last_epoch_clean":127,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:53:46.216803+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:53:46.216803+0000","last_clean_scrub_stamp":"2023-08-09T13:53:46.216803+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T21:45:04.615253+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":216,"reported_epoch":129,"state":"active+clean","last_fresh":"2023-08-09T13:53:49.281515+0000","last_change":"2023-08-09T13:33:55.886389+0000","last_active":"2023-08-09T13:53:49.281515+0000","last_peered":"2023-08-09T13:53:49.281515+0000","last_clean":"2023-08-09T13:53:49.281515+0000","last_became_active":"2023-08-09T13:33:55.885638+0000","last_became_peered":"2023-08-09T13:33:55.885638+0000","last_unstale":"2023-08-09T13:53:49.281515+0000","last_undegraded":"2023-08-09T13:53:49.281515+0000","last_fullsized":"2023-08-09T13:53:49.281515+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:33:54.845869+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:33:54.845869+0000","last_clean_scrub_stamp":"2023-08-09T13:33:54.845869+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T19:20:44.485269+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]}],"pool_stats":[{"poolid":17,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":16,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":38,"ondisk_log_size":38,"up":64,"acting":64,"num_store_stats":8},{"poolid":2,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":907,"num_read_kb":1830,"num_write":323,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":208,"ondisk_log_size":208,"up":2,"acting":2,"num_store_stats":2},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":8}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834575085,"num_pgs":17,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":37356,"kb_used_data":2136,"kb_used_omap":1,"kb_used_meta":35198,"kb_avail":94334484,"statfs":{"total":96636764160,"available":96598511616,"internally_reserved":0,"allocated":2187264,"data_stored":1436443,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1781,"internal_metadata":36043019},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Aug 9 13:53:27 2023","interfaces":[{"interface":"back","average":{"1min":1.145,"5min":1.167,"15min":1.222},"min":{"1min":0.623,"5min":0.61599999999999999,"15min":0.61599999999999999},"max":{"1min":1.704,"5min":1.8919999999999999,"15min":1.8919999999999999},"last":0.91600000000000004},{"interface":"front","average":{"1min":1.3140000000000001,"5min":1.2,"15min":1.232},"min":{"1min":0.83199999999999996,"5min":0.64200000000000002,"15min":0.64200000000000002},"max":{"1min":2.3399999999999999,"5min":2.3399999999999999,"15min":2.3399999999999999},"last":0.76900000000000002}]},{"osd":1,"last update":"Wed Aug 9 13:53:27 2023","interfaces":[{"interface":"back","average":{"1min":1.266,"5min":1.2509999999999999,"15min":1.262},"min":{"1min":0.81000000000000005,"5min":0.69199999999999995,"15min":0.69199999999999995},"max":{"1min":2.1989999999999998,"5min":2.1989999999999998,"15min":2.1989999999999998},"last":1.0449999999999999},{"interface":"front","average":{"1min":1.27,"5min":1.2290000000000001,"15min":1.2669999999999999},"min":{"1min":0.96299999999999997,"5min":0.59299999999999997,"15min":0.59299999999999997},"max":{"1min":1.649,"5min":1.998,"15min":1.998},"last":1.161}]},{"osd":2,"last update":"Wed Aug 9 13:53:27 2023","interfaces":[{"interface":"back","average":{"1min":1.321,"5min":1.252,"15min":1.2509999999999999},"min":{"1min":0.90900000000000003,"5min":0.75,"15min":0.65700000000000003},"max":{"1min":1.766,"5min":2.0129999999999999,"15min":2.0129999999999999},"last":1.1240000000000001},{"interface":"front","average":{"1min":1.3180000000000001,"5min":1.274,"15min":1.357},"min":{"1min":0.88800000000000001,"5min":0.58199999999999996,"15min":0.58199999999999996},"max":{"1min":2.359,"5min":2.359,"15min":2.359},"last":1.375}]},{"osd":3,"last update":"Wed Aug 9 13:53:27 2023","interfaces":[{"interface":"back","average":{"1min":1.4039999999999999,"5min":1.3240000000000001,"15min":1.339},"min":{"1min":0.89000000000000001,"5min":0.67500000000000004,"15min":0.67500000000000004},"max":{"1min":2.4380000000000002,"5min":2.4380000000000002,"15min":2.4380000000000002},"last":0.98899999999999999},{"interface":"front","average":{"1min":1.506,"5min":1.403,"15min":1.419},"min":{"1min":1.0580000000000001,"5min":0.71499999999999997,"15min":0.71499999999999997},"max":{"1min":2.6360000000000001,"5min":2.6360000000000001,"15min":2.6360000000000001},"last":1.2310000000000001}]},{"osd":4,"last update":"Wed Aug 9 13:53:27 2023","interfaces":[{"interface":"back","average":{"1min":1.2330000000000001,"5min":1.278,"15min":1.27},"min":{"1min":0.79000000000000004,"5min":0.74099999999999999,"15min":0.59499999999999997},"max":{"1min":1.635,"5min":2.097,"15min":2.097},"last":1.5429999999999999},{"interface":"front","average":{"1min":1.4259999999999999,"5min":1.3380000000000001,"15min":1.3380000000000001},"min":{"1min":0.93500000000000005,"5min":0.60799999999999998,"15min":0.60799999999999998},"max":{"1min":2.6890000000000001,"5min":2.6890000000000001,"15min":2.6890000000000001},"last":1.4359999999999999}]},{"osd":5,"last update":"Wed Aug 9 13:53:27 2023","interfaces":[{"interface":"back","average":{"1min":1.4970000000000001,"5min":1.458,"15min":1.4219999999999999},"min":{"1min":0.98899999999999999,"5min":0.65800000000000003,"15min":0.65800000000000003},"max":{"1min":2.7509999999999999,"5min":2.7509999999999999,"15min":2.7509999999999999},"last":1.5},{"interface":"front","average":{"1min":1.498,"5min":1.4390000000000001,"15min":1.4379999999999999},"min":{"1min":1.073,"5min":0.83999999999999997,"15min":0.83999999999999997},"max":{"1min":2.8639999999999999,"5min":2.8639999999999999,"15min":2.8639999999999999},"last":1.7010000000000001}]},{"osd":6,"last update":"Wed Aug 9 13:53:27 2023","interfaces":[{"interface":"back","average":{"1min":1.6339999999999999,"5min":1.544,"15min":1.5349999999999999},"min":{"1min":1.2589999999999999,"5min":0.85499999999999998,"15min":0.85499999999999998},"max":{"1min":2.7370000000000001,"5min":2.7370000000000001,"15min":2.7370000000000001},"last":1.6419999999999999},{"interface":"front","average":{"1min":1.5169999999999999,"5min":1.4590000000000001,"15min":1.474},"min":{"1min":1.1830000000000001,"5min":0.79700000000000004,"15min":0.79700000000000004},"max":{"1min":2.7850000000000001,"5min":2.7850000000000001,"15min":2.7850000000000001},"last":1.6000000000000001}]}]},{"osd":6,"up_from":13,"seq":55834575085,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":36760,"kb_used_data":1548,"kb_used_omap":1,"kb_used_meta":35198,"kb_avail":94335080,"statfs":{"total":96636764160,"available":96599121920,"internally_reserved":0,"allocated":1585152,"data_stored":844168,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1781,"internal_metadata":36043019},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Aug 9 13:53:25 2023","interfaces":[{"interface":"back","average":{"1min":1.2569999999999999,"5min":1.125,"15min":1.1499999999999999},"min":{"1min":0.90800000000000003,"5min":0.47999999999999998,"15min":0.47999999999999998},"max":{"1min":1.6020000000000001,"5min":1.6020000000000001,"15min":1.9870000000000001},"last":1.0129999999999999},{"interface":"front","average":{"1min":1.355,"5min":1.194,"15min":1.302},"min":{"1min":0.95799999999999996,"5min":0.51800000000000002,"15min":0.51800000000000002},"max":{"1min":1.8520000000000001,"5min":1.8520000000000001,"15min":28.036000000000001},"last":1.627}]},{"osd":1,"last update":"Wed Aug 9 13:53:25 2023","interfaces":[{"interface":"back","average":{"1min":1.21,"5min":1.1659999999999999,"15min":1.22},"min":{"1min":0.76500000000000001,"5min":0.48999999999999999,"15min":0.48999999999999999},"max":{"1min":1.6639999999999999,"5min":2.2320000000000002,"15min":7.742},"last":1.2430000000000001},{"interface":"front","average":{"1min":1.335,"5min":1.2569999999999999,"15min":1.278},"min":{"1min":1.012,"5min":0.64600000000000002,"15min":0.64600000000000002},"max":{"1min":1.653,"5min":2.056,"15min":2.056},"last":0.745}]},{"osd":2,"last update":"Wed Aug 9 13:53:25 2023","interfaces":[{"interface":"back","average":{"1min":1.2709999999999999,"5min":1.2629999999999999,"15min":1.319},"min":{"1min":0.63600000000000001,"5min":0.627,"15min":0.627},"max":{"1min":1.5529999999999999,"5min":2.1899999999999999,"15min":4.0640000000000001},"last":1.147},{"interface":"front","average":{"1min":1.3400000000000001,"5min":1.2589999999999999,"15min":1.3520000000000001},"min":{"1min":0.96199999999999997,"5min":0.76800000000000002,"15min":0.70999999999999996},"max":{"1min":1.7769999999999999,"5min":2.5819999999999999,"15min":13.154999999999999},"last":1.369}]},{"osd":3,"last update":"Wed Aug 9 13:53:25 2023","interfaces":[{"interface":"back","average":{"1min":1.3720000000000001,"5min":1.3169999999999999,"15min":1.4119999999999999},"min":{"1min":0.85599999999999998,"5min":0.67200000000000004,"15min":0.67200000000000004},"max":{"1min":1.665,"5min":2.2519999999999998,"15min":13.349},"last":1.0860000000000001},{"interface":"front","average":{"1min":1.3149999999999999,"5min":1.284,"15min":1.329},"min":{"1min":0.84699999999999998,"5min":0.75700000000000001,"15min":0.75700000000000001},"max":{"1min":1.74,"5min":1.8680000000000001,"15min":7.9029999999999996},"last":1.5329999999999999}]},{"osd":4,"last update":"Wed Aug 9 13:53:25 2023","interfaces":[{"interface":"back","average":{"1min":1.2569999999999999,"5min":1.276,"15min":1.2849999999999999},"min":{"1min":0.59599999999999997,"5min":0.35099999999999998,"15min":0.35099999999999998},"max":{"1min":1.6659999999999999,"5min":1.992,"15min":5.2290000000000001},"last":1.3129999999999999},{"interface":"front","average":{"1min":1.298,"5min":1.3620000000000001,"15min":1.3839999999999999},"min":{"1min":0.83099999999999996,"5min":0.54800000000000004,"15min":0.54800000000000004},"max":{"1min":1.9179999999999999,"5min":1.9179999999999999,"15min":12.1},"last":1.1830000000000001}]},{"osd":5,"last update":"Wed Aug 9 13:53:25 2023","interfaces":[{"interface":"back","average":{"1min":1.252,"5min":1.337,"15min":1.3600000000000001},"min":{"1min":0.72599999999999998,"5min":0.51000000000000001,"15min":0.51000000000000001},"max":{"1min":1.627,"5min":2.1349999999999998,"15min":8.1899999999999995},"last":1.4239999999999999},{"interface":"front","average":{"1min":1.5569999999999999,"5min":1.552,"15min":1.5920000000000001},"min":{"1min":0.67000000000000004,"5min":0.42399999999999999,"15min":0.42399999999999999},"max":{"1min":1.7969999999999999,"5min":2.2000000000000002,"15min":8.8889999999999993},"last":1.5760000000000001}]},{"osd":7,"last update":"Wed Aug 9 13:53:25 2023","interfaces":[{"interface":"back","average":{"1min":1.6060000000000001,"5min":1.5589999999999999,"15min":1.591},"min":{"1min":0.95199999999999996,"5min":0.69299999999999995,"15min":0.62},"max":{"1min":1.9790000000000001,"5min":2.4529999999999998,"15min":9.3320000000000007},"last":1.4770000000000001},{"interface":"front","average":{"1min":1.4790000000000001,"5min":1.506,"15min":1.544},"min":{"1min":0.79600000000000004,"5min":0.59899999999999998,"15min":0.59899999999999998},"max":{"1min":1.7949999999999999,"5min":2.2909999999999999,"15min":8.0399999999999991},"last":1.667}]}]},{"osd":5,"up_from":13,"seq":55834575085,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":58912,"kb_used_data":1556,"kb_used_omap":1,"kb_used_meta":57342,"kb_avail":94312928,"statfs":{"total":96636764160,"available":96576438272,"internally_reserved":0,"allocated":1593344,"data_stored":846063,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1781,"internal_metadata":58718475},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Aug 9 13:53:23 2023","interfaces":[{"interface":"back","average":{"1min":1.2629999999999999,"5min":1.26,"15min":1.21},"min":{"1min":0.83699999999999997,"5min":0.59099999999999997,"15min":0.59099999999999997},"max":{"1min":1.7929999999999999,"5min":8.9299999999999997,"15min":8.9299999999999997},"last":1.3620000000000001},{"interface":"front","average":{"1min":1.339,"5min":1.264,"15min":1.2410000000000001},"min":{"1min":0.81599999999999995,"5min":0.57799999999999996,"15min":0.57799999999999996},"max":{"1min":2.468,"5min":8.7140000000000004,"15min":8.7140000000000004},"last":1.133}]},{"osd":1,"last update":"Wed Aug 9 13:53:23 2023","interfaces":[{"interface":"back","average":{"1min":1.234,"5min":1.2190000000000001,"15min":1.2470000000000001},"min":{"1min":0.77300000000000002,"5min":0.66600000000000004,"15min":0.64100000000000001},"max":{"1min":1.9059999999999999,"5min":8.2720000000000002,"15min":8.2720000000000002},"last":1.252},{"interface":"front","average":{"1min":1.3160000000000001,"5min":1.3600000000000001,"15min":1.319},"min":{"1min":0.92300000000000004,"5min":0.73399999999999999,"15min":0.66900000000000004},"max":{"1min":1.752,"5min":9.0020000000000007,"15min":9.0020000000000007},"last":0.93700000000000006}]},{"osd":2,"last update":"Wed Aug 9 13:53:23 2023","interfaces":[{"interface":"back","average":{"1min":1.351,"5min":1.349,"15min":1.3180000000000001},"min":{"1min":0.97299999999999998,"5min":0.753,"15min":0.61599999999999999},"max":{"1min":2.4009999999999998,"5min":8.6050000000000004,"15min":8.6050000000000004},"last":1.2290000000000001},{"interface":"front","average":{"1min":1.3560000000000001,"5min":1.3660000000000001,"15min":1.3520000000000001},"min":{"1min":0.98999999999999999,"5min":0.78300000000000003,"15min":0.73999999999999999},"max":{"1min":1.716,"5min":8.4149999999999991,"15min":8.4149999999999991},"last":1.7190000000000001}]},{"osd":3,"last update":"Wed Aug 9 13:53:23 2023","interfaces":[{"interface":"back","average":{"1min":1.341,"5min":1.4359999999999999,"15min":1.3899999999999999},"min":{"1min":0.97199999999999998,"5min":0.72599999999999998,"15min":0.59799999999999998},"max":{"1min":1.9670000000000001,"5min":9.0470000000000006,"15min":9.0470000000000006},"last":1.0700000000000001},{"interface":"front","average":{"1min":1.5660000000000001,"5min":1.4930000000000001,"15min":1.4450000000000001},"min":{"1min":1.115,"5min":0.63,"15min":0.63},"max":{"1min":2.5819999999999999,"5min":8.7669999999999995,"15min":8.7669999999999995},"last":1.518}]},{"osd":4,"last update":"Wed Aug 9 13:53:23 2023","interfaces":[{"interface":"back","average":{"1min":1.4490000000000001,"5min":1.4119999999999999,"15min":1.341},"min":{"1min":0.97099999999999997,"5min":0.54300000000000004,"15min":0.54300000000000004},"max":{"1min":2.181,"5min":8.4909999999999997,"15min":8.4909999999999997},"last":1.4219999999999999},{"interface":"front","average":{"1min":1.476,"5min":1.5009999999999999,"15min":1.4199999999999999},"min":{"1min":0.93400000000000005,"5min":0.68899999999999995,"15min":0.68899999999999995},"max":{"1min":2.1040000000000001,"5min":9.4589999999999996,"15min":9.4589999999999996},"last":1.161}]},{"osd":6,"last update":"Wed Aug 9 13:53:23 2023","interfaces":[{"interface":"back","average":{"1min":1.5,"5min":1.6299999999999999,"15min":1.4730000000000001},"min":{"1min":0.94399999999999995,"5min":0.51000000000000001,"15min":0.51000000000000001},"max":{"1min":2.6539999999999999,"5min":17.407,"15min":17.407},"last":1.3140000000000001},{"interface":"front","average":{"1min":1.5640000000000001,"5min":1.6659999999999999,"15min":1.538},"min":{"1min":0.89500000000000002,"5min":0.71199999999999997,"15min":0.71199999999999997},"max":{"1min":2.415,"5min":16.356000000000002,"15min":16.356000000000002},"last":1.6180000000000001}]},{"osd":7,"last update":"Wed Aug 9 13:53:23 2023","interfaces":[{"interface":"back","average":{"1min":1.619,"5min":1.6699999999999999,"15min":1.5940000000000001},"min":{"1min":0.96399999999999997,"5min":0.74299999999999999,"15min":0.74299999999999999},"max":{"1min":2.3740000000000001,"5min":9.9239999999999995,"15min":9.9239999999999995},"last":1.5700000000000001},{"interface":"front","average":{"1min":1.7250000000000001,"5min":1.748,"15min":1.6539999999999999},"min":{"1min":1.1819999999999999,"5min":0.65600000000000003,"15min":0.65600000000000003},"max":{"1min":2.754,"5min":10.789,"15min":10.789},"last":1.6759999999999999}]}]},{"osd":4,"up_from":13,"seq":55834575085,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32668,"kb_used_data":1548,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":94339172,"statfs":{"total":96636764160,"available":96603312128,"internally_reserved":0,"allocated":1585152,"data_stored":845459,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1781,"internal_metadata":31848715},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Aug 9 13:53:26 2023","interfaces":[{"interface":"back","average":{"1min":1.2190000000000001,"5min":1.1779999999999999,"15min":1.2010000000000001},"min":{"1min":0.76300000000000001,"5min":0.66600000000000004,"15min":0.66600000000000004},"max":{"1min":1.556,"5min":2.524,"15min":2.524},"last":0.81299999999999994},{"interface":"front","average":{"1min":1.218,"5min":1.196,"15min":1.2470000000000001},"min":{"1min":0.82599999999999996,"5min":0.62,"15min":0.62},"max":{"1min":1.6930000000000001,"5min":1.7529999999999999,"15min":1.863},"last":0.96199999999999997}]},{"osd":1,"last update":"Wed Aug 9 13:53:26 2023","interfaces":[{"interface":"back","average":{"1min":1.2529999999999999,"5min":1.25,"15min":1.294},"min":{"1min":0.97199999999999998,"5min":0.70999999999999996,"15min":0.67800000000000005},"max":{"1min":1.7430000000000001,"5min":1.827,"15min":1.8480000000000001},"last":1.1819999999999999},{"interface":"front","average":{"1min":1.2110000000000001,"5min":1.2150000000000001,"15min":1.242},"min":{"1min":0.85899999999999999,"5min":0.68899999999999995,"15min":0.68899999999999995},"max":{"1min":1.5649999999999999,"5min":1.9330000000000001,"15min":1.9330000000000001},"last":1.1379999999999999}]},{"osd":2,"last update":"Wed Aug 9 13:53:26 2023","interfaces":[{"interface":"back","average":{"1min":1.2370000000000001,"5min":1.2390000000000001,"15min":1.264},"min":{"1min":0.91800000000000004,"5min":0.69999999999999996,"15min":0.621},"max":{"1min":1.52,"5min":2.6429999999999998,"15min":2.6429999999999998},"last":1.444},{"interface":"front","average":{"1min":1.3979999999999999,"5min":1.3260000000000001,"15min":1.365},"min":{"1min":1.0429999999999999,"5min":0.76000000000000001,"15min":0.76000000000000001},"max":{"1min":1.6839999999999999,"5min":2.0910000000000002,"15min":2.0910000000000002},"last":1.079}]},{"osd":3,"last update":"Wed Aug 9 13:53:26 2023","interfaces":[{"interface":"back","average":{"1min":1.333,"5min":1.304,"15min":1.2969999999999999},"min":{"1min":1.03,"5min":0.64700000000000002,"15min":0.64700000000000002},"max":{"1min":1.5700000000000001,"5min":2.6099999999999999,"15min":2.6099999999999999},"last":1.0149999999999999},{"interface":"front","average":{"1min":1.446,"5min":1.375,"15min":1.383},"min":{"1min":1.0149999999999999,"5min":0.86299999999999999,"15min":0.86299999999999999},"max":{"1min":1.7450000000000001,"5min":2.7629999999999999,"15min":2.7629999999999999},"last":1.3180000000000001}]},{"osd":5,"last update":"Wed Aug 9 13:53:26 2023","interfaces":[{"interface":"back","average":{"1min":1.379,"5min":1.345,"15min":1.3380000000000001},"min":{"1min":0.94599999999999995,"5min":0.67300000000000004,"15min":0.63200000000000001},"max":{"1min":1.821,"5min":2.0329999999999999,"15min":2.0329999999999999},"last":1.387},{"interface":"front","average":{"1min":1.333,"5min":1.345,"15min":1.3149999999999999},"min":{"1min":1.038,"5min":0.65600000000000003,"15min":0.65600000000000003},"max":{"1min":1.7030000000000001,"5min":2.7090000000000001,"15min":2.7090000000000001},"last":1.4970000000000001}]},{"osd":6,"last update":"Wed Aug 9 13:53:26 2023","interfaces":[{"interface":"back","average":{"1min":1.47,"5min":1.407,"15min":1.341},"min":{"1min":1.099,"5min":0.84099999999999997,"15min":0.59099999999999997},"max":{"1min":1.722,"5min":2.8559999999999999,"15min":2.8559999999999999},"last":1.74},{"interface":"front","average":{"1min":1.5549999999999999,"5min":1.52,"15min":1.51},"min":{"1min":1.1759999999999999,"5min":0.52600000000000002,"15min":0.52600000000000002},"max":{"1min":1.917,"5min":2.2719999999999998,"15min":2.2719999999999998},"last":1.573}]},{"osd":7,"last update":"Wed Aug 9 13:53:26 2023","interfaces":[{"interface":"back","average":{"1min":1.5069999999999999,"5min":1.48,"15min":1.474},"min":{"1min":1.331,"5min":0.53100000000000003,"15min":0.53100000000000003},"max":{"1min":1.778,"5min":2.8050000000000002,"15min":2.8050000000000002},"last":1.776},{"interface":"front","average":{"1min":1.6180000000000001,"5min":1.5609999999999999,"15min":1.5229999999999999},"min":{"1min":1.3120000000000001,"5min":0.78700000000000003,"15min":0.74099999999999999},"max":{"1min":1.8080000000000001,"5min":3.415,"15min":3.415},"last":1.665}]}]},{"osd":3,"up_from":13,"seq":55834575086,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":36776,"kb_used_data":1564,"kb_used_omap":1,"kb_used_meta":35198,"kb_avail":94335064,"statfs":{"total":96636764160,"available":96599105536,"internally_reserved":0,"allocated":1601536,"data_stored":847141,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1781,"internal_metadata":36043019},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Aug 9 13:53:27 2023","interfaces":[{"interface":"back","average":{"1min":0.98499999999999999,"5min":0.85499999999999998,"15min":0.90200000000000002},"min":{"1min":0.56299999999999994,"5min":0.43099999999999999,"15min":0.27200000000000002},"max":{"1min":1.5660000000000001,"5min":1.589,"15min":1.8500000000000001},"last":0.63900000000000001},{"interface":"front","average":{"1min":1.129,"5min":1.022,"15min":1.101},"min":{"1min":0.73099999999999998,"5min":0.36299999999999999,"15min":0.36299999999999999},"max":{"1min":1.5580000000000001,"5min":1.6910000000000001,"15min":2.0590000000000002},"last":0.91400000000000003}]},{"osd":1,"last update":"Wed Aug 9 13:53:27 2023","interfaces":[{"interface":"back","average":{"1min":0.98899999999999999,"5min":0.94099999999999995,"15min":1.0169999999999999},"min":{"1min":0.62,"5min":0.51500000000000001,"15min":0.496},"max":{"1min":1.909,"5min":1.9410000000000001,"15min":2.5649999999999999},"last":0.98799999999999999},{"interface":"front","average":{"1min":1.1699999999999999,"5min":1.0800000000000001,"15min":1.1719999999999999},"min":{"1min":0.65000000000000002,"5min":0.48399999999999999,"15min":0.48399999999999999},"max":{"1min":1.488,"5min":1.762,"15min":2.6760000000000002},"last":0.69599999999999995}]},{"osd":2,"last update":"Wed Aug 9 13:53:27 2023","interfaces":[{"interface":"back","average":{"1min":1.1819999999999999,"5min":1.081,"15min":1.1850000000000001},"min":{"1min":0.47899999999999998,"5min":0.42999999999999999,"15min":0.42999999999999999},"max":{"1min":1.661,"5min":1.6879999999999999,"15min":1.913},"last":0.84399999999999997},{"interface":"front","average":{"1min":1.3420000000000001,"5min":1.2450000000000001,"15min":1.3149999999999999},"min":{"1min":0.60499999999999998,"5min":0.52400000000000002,"15min":0.52300000000000002},"max":{"1min":1.78,"5min":1.9399999999999999,"15min":2.0259999999999998},"last":0.74299999999999999}]},{"osd":4,"last update":"Wed Aug 9 13:53:27 2023","interfaces":[{"interface":"back","average":{"1min":1.341,"5min":1.2090000000000001,"15min":1.2709999999999999},"min":{"1min":0.78700000000000003,"5min":0.54900000000000004,"15min":0.54900000000000004},"max":{"1min":1.7350000000000001,"5min":1.742,"15min":1.8160000000000001},"last":1.0740000000000001},{"interface":"front","average":{"1min":1.4590000000000001,"5min":1.3360000000000001,"15min":1.4059999999999999},"min":{"1min":1.026,"5min":0.67600000000000005,"15min":0.628},"max":{"1min":1.819,"5min":2.782,"15min":2.782},"last":1.2589999999999999}]},{"osd":5,"last update":"Wed Aug 9 13:53:27 2023","interfaces":[{"interface":"back","average":{"1min":1.5209999999999999,"5min":1.3979999999999999,"15min":1.4790000000000001},"min":{"1min":0.92300000000000004,"5min":0.67600000000000005,"15min":0.67600000000000005},"max":{"1min":1.8320000000000001,"5min":2.0489999999999999,"15min":2.254},"last":1.1599999999999999},{"interface":"front","average":{"1min":1.49,"5min":1.4039999999999999,"15min":1.488},"min":{"1min":0.98699999999999999,"5min":0.66000000000000003,"15min":0.65900000000000003},"max":{"1min":1.774,"5min":2.3069999999999999,"15min":2.3069999999999999},"last":1.2270000000000001}]},{"osd":6,"last update":"Wed Aug 9 13:53:27 2023","interfaces":[{"interface":"back","average":{"1min":1.3839999999999999,"5min":1.377,"15min":1.452},"min":{"1min":0.84799999999999998,"5min":0.73499999999999999,"15min":0.68200000000000005},"max":{"1min":1.8640000000000001,"5min":2.8340000000000001,"15min":2.8340000000000001},"last":1.425},{"interface":"front","average":{"1min":1.5620000000000001,"5min":1.4730000000000001,"15min":1.5569999999999999},"min":{"1min":1.028,"5min":0.64100000000000001,"15min":0.64100000000000001},"max":{"1min":1.9370000000000001,"5min":2.9060000000000001,"15min":2.9060000000000001},"last":1.3440000000000001}]},{"osd":7,"last update":"Wed Aug 9 13:53:27 2023","interfaces":[{"interface":"back","average":{"1min":1.603,"5min":1.5129999999999999,"15min":1.605},"min":{"1min":1.218,"5min":0.624,"15min":0.624},"max":{"1min":2.109,"5min":2.46,"15min":2.46},"last":1.45},{"interface":"front","average":{"1min":1.464,"5min":1.462,"15min":1.552},"min":{"1min":0.71199999999999997,"5min":0.71199999999999997,"15min":0.64200000000000002},"max":{"1min":2.0230000000000001,"5min":2.7200000000000002,"15min":2.7200000000000002},"last":1.1970000000000001}]}]},{"osd":2,"up_from":13,"seq":55834575085,"num_pgs":12,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":40860,"kb_used_data":1548,"kb_used_omap":1,"kb_used_meta":39294,"kb_avail":94330980,"statfs":{"total":96636764160,"available":96594923520,"internally_reserved":0,"allocated":1585152,"data_stored":845459,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1781,"internal_metadata":40237323},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":7,"apply_latency_ms":7,"commit_latency_ns":7000000,"apply_latency_ns":7000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Aug 9 13:53:24 2023","interfaces":[{"interface":"back","average":{"1min":0.94999999999999996,"5min":0.89400000000000002,"15min":0.95099999999999996},"min":{"1min":0.40300000000000002,"5min":0.35499999999999998,"15min":0.35499999999999998},"max":{"1min":1.524,"5min":1.5760000000000001,"15min":1.752},"last":1.0780000000000001},{"interface":"front","average":{"1min":1.0760000000000001,"5min":0.97099999999999997,"15min":1},"min":{"1min":0.53800000000000003,"5min":0.38200000000000001,"15min":0.374},"max":{"1min":1.75,"5min":1.75,"15min":2.3929999999999998},"last":1.028}]},{"osd":1,"last update":"Wed Aug 9 13:53:24 2023","interfaces":[{"interface":"back","average":{"1min":1.1699999999999999,"5min":1.123,"15min":1.1779999999999999},"min":{"1min":0.499,"5min":0.39400000000000002,"15min":0.39400000000000002},"max":{"1min":1.587,"5min":1.77,"15min":1.8420000000000001},"last":1.254},{"interface":"front","average":{"1min":1.345,"5min":1.1899999999999999,"15min":1.3069999999999999},"min":{"1min":0.63400000000000001,"5min":0.435,"15min":0.435},"max":{"1min":1.643,"5min":1.8999999999999999,"15min":2.4870000000000001},"last":1.1240000000000001}]},{"osd":3,"last update":"Wed Aug 9 13:53:24 2023","interfaces":[{"interface":"back","average":{"1min":1.056,"5min":0.96899999999999997,"15min":1.038},"min":{"1min":0.57399999999999995,"5min":0.41399999999999998,"15min":0.38900000000000001},"max":{"1min":1.508,"5min":1.7130000000000001,"15min":1.7609999999999999},"last":1.1439999999999999},{"interface":"front","average":{"1min":1.2989999999999999,"5min":1.175,"15min":1.2909999999999999},"min":{"1min":0.45500000000000002,"5min":0.45500000000000002,"15min":0.45500000000000002},"max":{"1min":1.7509999999999999,"5min":1.835,"15min":8.8130000000000006},"last":1.3400000000000001}]},{"osd":4,"last update":"Wed Aug 9 13:53:24 2023","interfaces":[{"interface":"back","average":{"1min":1.448,"5min":1.377,"15min":1.444},"min":{"1min":0.622,"5min":0.622,"15min":0.622},"max":{"1min":1.7729999999999999,"5min":2.1640000000000001,"15min":2.1640000000000001},"last":1.4039999999999999},{"interface":"front","average":{"1min":1.415,"5min":1.3540000000000001,"15min":1.387},"min":{"1min":0.70399999999999996,"5min":0.64800000000000002,"15min":0.64800000000000002},"max":{"1min":1.8069999999999999,"5min":2.472,"15min":2.472},"last":1.3839999999999999}]},{"osd":5,"last update":"Wed Aug 9 13:53:24 2023","interfaces":[{"interface":"back","average":{"1min":1.5369999999999999,"5min":1.429,"15min":1.456},"min":{"1min":0.75,"5min":0.71199999999999997,"15min":0.59999999999999998},"max":{"1min":2.0600000000000001,"5min":2.4900000000000002,"15min":2.5840000000000001},"last":1.282},{"interface":"front","average":{"1min":1.5289999999999999,"5min":1.331,"15min":1.3520000000000001},"min":{"1min":0.73699999999999999,"5min":0.73699999999999999,"15min":0.56799999999999995},"max":{"1min":2.3519999999999999,"5min":2.3519999999999999,"15min":3.5659999999999998},"last":1.4450000000000001}]},{"osd":6,"last update":"Wed Aug 9 13:53:24 2023","interfaces":[{"interface":"back","average":{"1min":1.5409999999999999,"5min":1.383,"15min":1.4299999999999999},"min":{"1min":0.77600000000000002,"5min":0.77600000000000002,"15min":0.73799999999999999},"max":{"1min":2.5110000000000001,"5min":2.5110000000000001,"15min":3.5099999999999998},"last":1.3129999999999999},{"interface":"front","average":{"1min":1.6120000000000001,"5min":1.4790000000000001,"15min":1.54},"min":{"1min":0.66100000000000003,"5min":0.66100000000000003,"15min":0.66100000000000003},"max":{"1min":2.3079999999999998,"5min":2.3769999999999998,"15min":2.3769999999999998},"last":1.5489999999999999}]},{"osd":7,"last update":"Wed Aug 9 13:53:24 2023","interfaces":[{"interface":"back","average":{"1min":1.6100000000000001,"5min":1.474,"15min":1.5309999999999999},"min":{"1min":0.79200000000000004,"5min":0.78800000000000003,"15min":0.64600000000000002},"max":{"1min":2.1459999999999999,"5min":2.1970000000000001,"15min":2.5249999999999999},"last":1.492},{"interface":"front","average":{"1min":1.5740000000000001,"5min":1.4550000000000001,"15min":1.504},"min":{"1min":0.67900000000000005,"5min":0.66100000000000003,"15min":0.66100000000000003},"max":{"1min":2.4500000000000002,"5min":2.4500000000000002,"15min":3.609},"last":1.589}]}]},{"osd":1,"up_from":13,"seq":55834575085,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":47208,"kb_used_data":2136,"kb_used_omap":1,"kb_used_meta":45054,"kb_avail":94324632,"statfs":{"total":96636764160,"available":96588423168,"internally_reserved":0,"allocated":2187264,"data_stored":1435951,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1781,"internal_metadata":46135563},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Aug 9 13:53:39 2023","interfaces":[{"interface":"back","average":{"1min":0.96799999999999997,"5min":0.90800000000000003,"15min":0.92900000000000005},"min":{"1min":0.69199999999999995,"5min":0.45800000000000002,"15min":0.45800000000000002},"max":{"1min":1.5349999999999999,"5min":1.7110000000000001,"15min":2.1080000000000001},"last":1.004},{"interface":"front","average":{"1min":1.0649999999999999,"5min":1.0289999999999999,"15min":1.097},"min":{"1min":0.71399999999999997,"5min":0.40699999999999997,"15min":0.40699999999999997},"max":{"1min":1.5469999999999999,"5min":2.5070000000000001,"15min":2.5070000000000001},"last":0.94099999999999995}]},{"osd":2,"last update":"Wed Aug 9 13:53:39 2023","interfaces":[{"interface":"back","average":{"1min":1.0389999999999999,"5min":0.97299999999999998,"15min":0.99299999999999999},"min":{"1min":0.81399999999999995,"5min":0.41999999999999998,"15min":0.41099999999999998},"max":{"1min":1.476,"5min":1.798,"15min":1.907},"last":1.0700000000000001},{"interface":"front","average":{"1min":1.125,"5min":1.0880000000000001,"15min":1.169},"min":{"1min":0.60999999999999999,"5min":0.50700000000000001,"15min":0.50700000000000001},"max":{"1min":1.579,"5min":2.1280000000000001,"15min":2.145},"last":1.458}]},{"osd":3,"last update":"Wed Aug 9 13:53:39 2023","interfaces":[{"interface":"back","average":{"1min":1.2989999999999999,"5min":1.208,"15min":1.3129999999999999},"min":{"1min":0.73199999999999998,"5min":0.54600000000000004,"15min":0.54400000000000004},"max":{"1min":1.702,"5min":2.0270000000000001,"15min":2.3210000000000002},"last":1.6819999999999999},{"interface":"front","average":{"1min":1.1839999999999999,"5min":1.1659999999999999,"15min":1.24},"min":{"1min":0.92500000000000004,"5min":0.48299999999999998,"15min":0.45300000000000001},"max":{"1min":1.47,"5min":2.3959999999999999,"15min":2.7669999999999999},"last":1.4950000000000001}]},{"osd":4,"last update":"Wed Aug 9 13:53:39 2023","interfaces":[{"interface":"back","average":{"1min":1.4099999999999999,"5min":1.3540000000000001,"15min":1.4059999999999999},"min":{"1min":1.2330000000000001,"5min":0.79800000000000004,"15min":0.58099999999999996},"max":{"1min":1.6659999999999999,"5min":2.0739999999999998,"15min":2.4089999999999998},"last":1.1619999999999999},{"interface":"front","average":{"1min":1.421,"5min":1.3180000000000001,"15min":1.3440000000000001},"min":{"1min":1.0509999999999999,"5min":0.71099999999999997,"15min":0.70099999999999996},"max":{"1min":1.7589999999999999,"5min":2.0019999999999998,"15min":2.0019999999999998},"last":1.4219999999999999}]},{"osd":5,"last update":"Wed Aug 9 13:53:39 2023","interfaces":[{"interface":"back","average":{"1min":1.4530000000000001,"5min":1.3300000000000001,"15min":1.377},"min":{"1min":1.1899999999999999,"5min":0.75900000000000001,"15min":0.75900000000000001},"max":{"1min":1.8129999999999999,"5min":1.9419999999999999,"15min":2.294},"last":1.6559999999999999},{"interface":"front","average":{"1min":1.518,"5min":1.409,"15min":1.502},"min":{"1min":0.94599999999999995,"5min":0.78400000000000003,"15min":0.78400000000000003},"max":{"1min":1.7829999999999999,"5min":1.863,"15min":2.4649999999999999},"last":1.2390000000000001}]},{"osd":6,"last update":"Wed Aug 9 13:53:39 2023","interfaces":[{"interface":"back","average":{"1min":1.474,"5min":1.462,"15min":1.5109999999999999},"min":{"1min":0.96899999999999997,"5min":0.96599999999999997,"15min":0.67400000000000004},"max":{"1min":1.8720000000000001,"5min":2.23,"15min":2.5270000000000001},"last":1.7210000000000001},{"interface":"front","average":{"1min":1.554,"5min":1.4419999999999999,"15min":1.5009999999999999},"min":{"1min":1.1399999999999999,"5min":0.73599999999999999,"15min":0.73599999999999999},"max":{"1min":1.946,"5min":2.3359999999999999,"15min":2.3359999999999999},"last":1.5289999999999999}]},{"osd":7,"last update":"Wed Aug 9 13:53:39 2023","interfaces":[{"interface":"back","average":{"1min":1.6579999999999999,"5min":1.4950000000000001,"15min":1.6020000000000001},"min":{"1min":1.3009999999999999,"5min":0.72099999999999997,"15min":0.72099999999999997},"max":{"1min":2.2040000000000002,"5min":2.4340000000000002,"15min":2.5790000000000002},"last":1.5760000000000001},{"interface":"front","average":{"1min":1.5860000000000001,"5min":1.526,"15min":1.5980000000000001},"min":{"1min":1.0900000000000001,"5min":0.85099999999999998,"15min":0.85099999999999998},"max":{"1min":1.9970000000000001,"5min":2.375,"15min":2.6509999999999998},"last":1.6080000000000001}]}]},{"osd":0,"up_from":13,"seq":55834575085,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":40876,"kb_used_data":1564,"kb_used_omap":1,"kb_used_meta":39294,"kb_avail":94330964,"statfs":{"total":96636764160,"available":96594907136,"internally_reserved":0,"allocated":1601536,"data_stored":845822,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1781,"internal_metadata":40237323},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Wed Aug 9 13:53:30 2023","interfaces":[{"interface":"back","average":{"1min":0.89400000000000002,"5min":0.92700000000000005,"15min":0.92200000000000004},"min":{"1min":0.504,"5min":0.32500000000000001,"15min":0.32500000000000001},"max":{"1min":1.1639999999999999,"5min":1.6519999999999999,"15min":1.756},"last":0.82399999999999995},{"interface":"front","average":{"1min":1.1439999999999999,"5min":1.101,"15min":1.111},"min":{"1min":0.63300000000000001,"5min":0.39200000000000002,"15min":0.39200000000000002},"max":{"1min":1.7010000000000001,"5min":1.9019999999999999,"15min":1.9019999999999999},"last":1.385}]},{"osd":2,"last update":"Wed Aug 9 13:53:30 2023","interfaces":[{"interface":"back","average":{"1min":1.0760000000000001,"5min":1.0429999999999999,"15min":1.119},"min":{"1min":0.55700000000000005,"5min":0.433,"15min":0.433},"max":{"1min":1.6140000000000001,"5min":1.786,"15min":1.786},"last":1.202},{"interface":"front","average":{"1min":1.159,"5min":1.127,"15min":1.1799999999999999},"min":{"1min":0.45800000000000002,"5min":0.439,"15min":0.439},"max":{"1min":1.929,"5min":2.0129999999999999,"15min":2.0129999999999999},"last":1.1779999999999999}]},{"osd":3,"last update":"Wed Aug 9 13:53:30 2023","interfaces":[{"interface":"back","average":{"1min":1.117,"5min":1.0349999999999999,"15min":1.097},"min":{"1min":0.48299999999999998,"5min":0.48299999999999998,"15min":0.46300000000000002},"max":{"1min":1.7529999999999999,"5min":1.7529999999999999,"15min":1.881},"last":0.61199999999999999},{"interface":"front","average":{"1min":1.298,"5min":1.2410000000000001,"15min":1.304},"min":{"1min":0.69299999999999995,"5min":0.50800000000000001,"15min":0.50800000000000001},"max":{"1min":1.708,"5min":2.347,"15min":2.4649999999999999},"last":1.101}]},{"osd":4,"last update":"Wed Aug 9 13:53:30 2023","interfaces":[{"interface":"back","average":{"1min":1.3500000000000001,"5min":1.3,"15min":1.341},"min":{"1min":0.69699999999999995,"5min":0.69699999999999995,"15min":0.69699999999999995},"max":{"1min":1.8460000000000001,"5min":1.9630000000000001,"15min":2.2309999999999999},"last":1.157},{"interface":"front","average":{"1min":1.4570000000000001,"5min":1.4079999999999999,"15min":1.4490000000000001},"min":{"1min":0.83299999999999996,"5min":0.82699999999999996,"15min":0.77600000000000002},"max":{"1min":2.012,"5min":2.3540000000000001,"15min":2.5390000000000001},"last":1.27}]},{"osd":5,"last update":"Wed Aug 9 13:53:30 2023","interfaces":[{"interface":"back","average":{"1min":1.526,"5min":1.4379999999999999,"15min":1.4670000000000001},"min":{"1min":0.871,"5min":0.66300000000000003,"15min":0.66300000000000003},"max":{"1min":2.298,"5min":2.6669999999999998,"15min":2.6669999999999998},"last":1.45},{"interface":"front","average":{"1min":1.4339999999999999,"5min":1.3720000000000001,"15min":1.4430000000000001},"min":{"1min":0.97999999999999998,"5min":0.70099999999999996,"15min":0.629},"max":{"1min":1.9710000000000001,"5min":1.984,"15min":2.1499999999999999},"last":1.296}]},{"osd":6,"last update":"Wed Aug 9 13:53:30 2023","interfaces":[{"interface":"back","average":{"1min":1.5529999999999999,"5min":1.5029999999999999,"15min":1.542},"min":{"1min":0.84599999999999997,"5min":0.73599999999999999,"15min":0.73599999999999999},"max":{"1min":1.954,"5min":2.4670000000000001,"15min":2.7250000000000001},"last":1.3600000000000001},{"interface":"front","average":{"1min":1.6100000000000001,"5min":1.548,"15min":1.5680000000000001},"min":{"1min":0.73899999999999999,"5min":0.73899999999999999,"15min":0.73899999999999999},"max":{"1min":2.2109999999999999,"5min":2.3199999999999998,"15min":2.399},"last":1.554}]},{"osd":7,"last update":"Wed Aug 9 13:53:30 2023","interfaces":[{"interface":"back","average":{"1min":1.704,"5min":1.589,"15min":1.629},"min":{"1min":1.0820000000000001,"5min":0.72299999999999998,"15min":0.72299999999999998},"max":{"1min":2.234,"5min":2.3900000000000001,"15min":2.3900000000000001},"last":1.504},{"interface":"front","average":{"1min":1.5649999999999999,"5min":1.5069999999999999,"15min":1.5489999999999999},"min":{"1min":0.85099999999999998,"5min":0.79200000000000004,"15min":0.76700000000000002},"max":{"1min":2.0329999999999999,"5min":2.274,"15min":2.274},"last":1.234}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":1724,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":124,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1682,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":604,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":70,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":616,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-08-09T13:53:53.356 INFO:tasks.cephfs.filesystem.ceph_manager:clean! 2023-08-09T13:53:53.357 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth caps client.0 mon 'allow r' osd 'allow rw tag cephfs data=cephfs' mds allow 2023-08-09T13:53:53.628 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:53.625+0000 7f6813538640 1 Processor -- start 2023-08-09T13:53:53.628 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:53.626+0000 7f6813538640 1 -- start start 2023-08-09T13:53:53.630 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:53.627+0000 7f6813538640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f680c159470 0x7f680c0c9420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:53:53.630 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:53.627+0000 7f6813538640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f680c15da80 0x7f680c070db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:53:53.630 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:53.627+0000 7f6813538640 1 -- --> v1:172.21.15.43:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f680c0c9d40 con 0x7f680c0712f0 2023-08-09T13:53:53.630 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:53.627+0000 7f6813538640 1 -- --> v2:172.21.15.43:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f680c070990 con 0x7f680c159470 2023-08-09T13:53:53.630 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:53.627+0000 7f6813538640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f680c0ca020 con 0x7f680c15da80 2023-08-09T13:53:53.631 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:53.628+0000 7f68112d4640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f680c159470 0x7f680c0c9420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:53:53.631 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:53.628+0000 7f68112d4640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f680c159470 0x7f680c0c9420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:56402/0 (socket says 172.21.15.43:56402) 2023-08-09T13:53:53.631 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:53.628+0000 7f6811ad5640 1 --1- >> v1:172.21.15.43:6789/0 conn(0x7f680c0712f0 0x7f680c0cc8b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.43:6789/0 says I am v1:172.21.15.43:53622/0 (socket says 172.21.15.43:53622) 2023-08-09T13:53:53.631 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:53.628+0000 7f68112d4640 1 -- 172.21.15.43:0/1844183314 learned_addr learned my addr 172.21.15.43:0/1844183314 (peer_addr_for_me v2:172.21.15.43:0/0) 2023-08-09T13:53:53.632 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:53.628+0000 7f6810ad3640 1 --2- 172.21.15.43:0/1844183314 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f680c15da80 0x7f680c070db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:53:53.632 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:53.628+0000 7f68112d4640 1 -- 172.21.15.43:0/1844183314 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f680c15da80 msgr2=0x7f680c070db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-09T13:53:53.632 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:53.628+0000 7f68112d4640 1 --2- 172.21.15.43:0/1844183314 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f680c15da80 0x7f680c070db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:53:53.632 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:53.628+0000 7f68112d4640 1 -- 172.21.15.43:0/1844183314 >> v1:172.21.15.43:6789/0 conn(0x7f680c0712f0 legacy=0x7f680c0cc8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:53:53.632 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:53.628+0000 7f68112d4640 1 -- 172.21.15.43:0/1844183314 --> v2:172.21.15.43:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f680c0705b0 con 0x7f680c159470 2023-08-09T13:53:53.632 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:53.628+0000 7f6810ad3640 1 --2- 172.21.15.43:0/1844183314 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f680c15da80 0x7f680c070db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-08-09T13:53:53.633 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:53.629+0000 7f68112d4640 1 --2- 172.21.15.43:0/1844183314 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f680c159470 0x7f680c0c9420 secure :-1 s=READY pgs=1527 cs=0 l=1 rev1=1 crypto rx=0x7f68080098a0 tx=0x7f680802b980 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fbbb7f0f363b7d55 server_cookie=0 in_seq=0 out_seq=0 2023-08-09T13:53:53.633 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:53.630+0000 7f6803fff640 1 -- 172.21.15.43:0/1844183314 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6808030840 con 0x7f680c159470 2023-08-09T13:53:53.634 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:53.630+0000 7f6803fff640 1 -- 172.21.15.43:0/1844183314 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f68080309a0 con 0x7f680c159470 2023-08-09T13:53:53.634 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:53.630+0000 7f6803fff640 1 -- 172.21.15.43:0/1844183314 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6808030d30 con 0x7f680c159470 2023-08-09T13:53:53.634 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:53.630+0000 7f6813538640 1 -- 172.21.15.43:0/1844183314 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f680c159470 msgr2=0x7f680c0c9420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:53:53.634 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:53.630+0000 7f6813538640 1 --2- 172.21.15.43:0/1844183314 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f680c159470 0x7f680c0c9420 secure :-1 s=READY pgs=1527 cs=0 l=1 rev1=1 crypto rx=0x7f68080098a0 tx=0x7f680802b980 comp rx=0 tx=0).stop 2023-08-09T13:53:53.634 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:53.631+0000 7f6813538640 1 -- 172.21.15.43:0/1844183314 shutdown_connections 2023-08-09T13:53:53.634 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:53.631+0000 7f6813538640 1 --2- 172.21.15.43:0/1844183314 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f680c15da80 0x7f680c070db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:53:53.634 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:53.631+0000 7f6813538640 1 --2- 172.21.15.43:0/1844183314 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f680c159470 0x7f680c0c9420 unknown :-1 s=CLOSED pgs=1527 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:53:53.635 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:53.631+0000 7f6813538640 1 -- 172.21.15.43:0/1844183314 >> 172.21.15.43:0/1844183314 conn(0x7f680c0aff50 msgr2=0x7f680c0b0360 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-09T13:53:53.635 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:53.631+0000 7f6813538640 1 -- 172.21.15.43:0/1844183314 shutdown_connections 2023-08-09T13:53:53.635 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:53.631+0000 7f6813538640 1 -- 172.21.15.43:0/1844183314 wait complete. 2023-08-09T13:53:53.635 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:53.633+0000 7f6813538640 1 Processor -- start 2023-08-09T13:53:53.635 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:53.633+0000 7f6813538640 1 -- start start 2023-08-09T13:53:53.636 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:53.634+0000 7f6813538640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f680c0712f0 0x7f680c164630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:53:53.636 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:53.634+0000 7f6813538640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f680c159470 0x7f680c1511f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:53:53.637 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:53.634+0000 7f6813538640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f680c15da80 0x7f680c151730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:53:53.637 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:53.634+0000 7f6813538640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f680c151c70 con 0x7f680c0712f0 2023-08-09T13:53:53.637 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:53.634+0000 7f6813538640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f680c164eb0 con 0x7f680c15da80 2023-08-09T13:53:53.637 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:53.634+0000 7f6813538640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f680c165190 con 0x7f680c159470 2023-08-09T13:53:53.637 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:53.634+0000 7f68112d4640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f680c0712f0 0x7f680c164630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:53:53.637 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:53.634+0000 7f68112d4640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f680c0712f0 0x7f680c164630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:56410/0 (socket says 172.21.15.43:56410) 2023-08-09T13:53:53.637 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:53.634+0000 7f68112d4640 1 -- 172.21.15.43:0/1252411650 learned_addr learned my addr 172.21.15.43:0/1252411650 (peer_addr_for_me v2:172.21.15.43:0/0) 2023-08-09T13:53:53.638 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:53.635+0000 7f6810ad3640 1 --2- 172.21.15.43:0/1252411650 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f680c159470 0x7f680c1511f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:53:53.639 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:53.635+0000 7f68112d4640 1 -- 172.21.15.43:0/1252411650 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f680c159470 msgr2=0x7f680c1511f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:53:53.639 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:53.635+0000 7f68112d4640 1 --2- 172.21.15.43:0/1252411650 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f680c159470 0x7f680c1511f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:53:53.639 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:53.635+0000 7f68112d4640 1 -- 172.21.15.43:0/1252411650 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f680c15da80 msgr2=0x7f680c151730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:53:53.639 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:53.635+0000 7f68112d4640 1 --2- 172.21.15.43:0/1252411650 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f680c15da80 0x7f680c151730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:53:53.639 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:53.635+0000 7f68112d4640 1 -- 172.21.15.43:0/1252411650 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f68080094d0 con 0x7f680c0712f0 2023-08-09T13:53:53.639 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:53.635+0000 7f6810ad3640 1 --2- 172.21.15.43:0/1252411650 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f680c159470 0x7f680c1511f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-08-09T13:53:53.639 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:53.635+0000 7f68112d4640 1 --2- 172.21.15.43:0/1252411650 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f680c0712f0 0x7f680c164630 secure :-1 s=READY pgs=1528 cs=0 l=1 rev1=1 crypto rx=0x7f68080098a0 tx=0x7f6808004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-09T13:53:53.640 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:53.635+0000 7f6801ffb640 1 -- 172.21.15.43:0/1252411650 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6808039d60 con 0x7f680c0712f0 2023-08-09T13:53:53.640 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:53.635+0000 7f6801ffb640 1 -- 172.21.15.43:0/1252411650 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6808043400 con 0x7f680c0712f0 2023-08-09T13:53:53.640 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:53.635+0000 7f6801ffb640 1 -- 172.21.15.43:0/1252411650 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f68080437b0 con 0x7f680c0712f0 2023-08-09T13:53:53.640 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:53.635+0000 7f6813538640 1 -- 172.21.15.43:0/1252411650 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f680c165470 con 0x7f680c0712f0 2023-08-09T13:53:53.640 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:53.636+0000 7f6813538640 1 -- 172.21.15.43:0/1252411650 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f680c165a30 con 0x7f680c0712f0 2023-08-09T13:53:53.641 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:53.637+0000 7f6813538640 1 -- 172.21.15.43:0/1252411650 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f680c071110 con 0x7f680c0712f0 2023-08-09T13:53:53.641 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:53.638+0000 7f6801ffb640 1 -- 172.21.15.43:0/1252411650 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 56) v1 ==== 88443+0+0 (secure 0 0 0) 0x7f6808043910 con 0x7f680c0712f0 2023-08-09T13:53:53.641 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:53.639+0000 7f6801ffb640 1 --2- 172.21.15.43:0/1252411650 >> [v2:172.21.15.203:6832/2527059680,v1:172.21.15.203:6833/2527059680] conn(0x7f67f006a170 0x7f67f006c630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:53:53.641 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:53.639+0000 7f6801ffb640 1 -- 172.21.15.43:0/1252411650 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(129..129 src has 1..129) v4 ==== 6426+0+0 (secure 0 0 0) 0x7f680803e070 con 0x7f680c0712f0 2023-08-09T13:53:53.642 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:53.639+0000 7f6810ad3640 1 --2- 172.21.15.43:0/1252411650 >> [v2:172.21.15.203:6832/2527059680,v1:172.21.15.203:6833/2527059680] conn(0x7f67f006a170 0x7f67f006c630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:53:53.643 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:53.641+0000 7f6810ad3640 1 --2- 172.21.15.43:0/1252411650 >> [v2:172.21.15.203:6832/2527059680,v1:172.21.15.203:6833/2527059680] conn(0x7f67f006a170 0x7f67f006c630 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f67fc006fd0 tx=0x7f67fc008040 comp rx=0 tx=0).ready entity=mgr.6377 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-09T13:53:53.645 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:53.643+0000 7f6801ffb640 1 -- 172.21.15.43:0/1252411650 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178066 (secure 0 0 0) 0x7f6808076f10 con 0x7f680c0712f0 2023-08-09T13:53:53.909 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:53.907+0000 7f6813538640 1 -- 172.21.15.43:0/1252411650 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]} v 0) v1 -- 0x7f680c0739f0 con 0x7f680c0712f0 2023-08-09T13:53:53.915 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:53.913+0000 7f6801ffb640 1 -- 172.21.15.43:0/1252411650 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]}]=0 updated caps for client.0 v19) v1 ==== 190+0+147 (secure 0 0 0) 0x7f68080340e0 con 0x7f680c0712f0 2023-08-09T13:53:53.916 INFO:teuthology.orchestra.run.smithi043.stdout:[client.0] 2023-08-09T13:53:53.916 INFO:teuthology.orchestra.run.smithi043.stdout: key = AQCPldNkg8zrDxAAM/hnJaE6S9HU/fRwQeBV5w== 2023-08-09T13:53:53.916 INFO:teuthology.orchestra.run.smithi043.stdout: caps mds = "allow" 2023-08-09T13:53:53.916 INFO:teuthology.orchestra.run.smithi043.stdout: caps mon = "allow r" 2023-08-09T13:53:53.916 INFO:teuthology.orchestra.run.smithi043.stdout: caps osd = "allow rw tag cephfs data=cephfs" 2023-08-09T13:53:53.917 INFO:teuthology.orchestra.run.smithi043.stderr:updated caps for client.0 2023-08-09T13:53:53.918 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:53.915+0000 7f6813538640 1 -- 172.21.15.43:0/1252411650 >> [v2:172.21.15.203:6832/2527059680,v1:172.21.15.203:6833/2527059680] conn(0x7f67f006a170 msgr2=0x7f67f006c630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:53:53.918 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:53.915+0000 7f6813538640 1 --2- 172.21.15.43:0/1252411650 >> [v2:172.21.15.203:6832/2527059680,v1:172.21.15.203:6833/2527059680] conn(0x7f67f006a170 0x7f67f006c630 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f67fc006fd0 tx=0x7f67fc008040 comp rx=0 tx=0).stop 2023-08-09T13:53:53.918 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:53.915+0000 7f6813538640 1 -- 172.21.15.43:0/1252411650 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f680c0712f0 msgr2=0x7f680c164630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:53:53.918 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:53.915+0000 7f6813538640 1 --2- 172.21.15.43:0/1252411650 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f680c0712f0 0x7f680c164630 secure :-1 s=READY pgs=1528 cs=0 l=1 rev1=1 crypto rx=0x7f68080098a0 tx=0x7f6808004730 comp rx=0 tx=0).stop 2023-08-09T13:53:53.919 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:53.915+0000 7f6813538640 1 -- 172.21.15.43:0/1252411650 shutdown_connections 2023-08-09T13:53:53.919 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:53.915+0000 7f6813538640 1 --2- 172.21.15.43:0/1252411650 >> [v2:172.21.15.203:6832/2527059680,v1:172.21.15.203:6833/2527059680] conn(0x7f67f006a170 0x7f67f006c630 unknown :-1 s=CLOSED pgs=66 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:53:53.919 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:53.915+0000 7f6813538640 1 --2- 172.21.15.43:0/1252411650 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f680c15da80 0x7f680c151730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:53:53.919 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:53.915+0000 7f6813538640 1 --2- 172.21.15.43:0/1252411650 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f680c159470 0x7f680c1511f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:53:53.919 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:53.915+0000 7f6813538640 1 --2- 172.21.15.43:0/1252411650 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f680c0712f0 0x7f680c164630 unknown :-1 s=CLOSED pgs=1528 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:53:53.919 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:53.915+0000 7f6813538640 1 -- 172.21.15.43:0/1252411650 >> 172.21.15.43:0/1252411650 conn(0x7f680c0aff50 msgr2=0x7f680c0ce190 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-09T13:53:53.920 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:53.916+0000 7f6813538640 1 -- 172.21.15.43:0/1252411650 shutdown_connections 2023-08-09T13:53:53.920 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:53.916+0000 7f6813538640 1 -- 172.21.15.43:0/1252411650 wait complete. 2023-08-09T13:53:53.937 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-08-09T13:53:54.160 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:54.158+0000 7f1bf4967640 1 Processor -- start 2023-08-09T13:53:54.161 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:54.158+0000 7f1bf4967640 1 -- start start 2023-08-09T13:53:54.162 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:54.160+0000 7f1bf4967640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f1bf015a6a0 0x7f1bf00713b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:53:54.162 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:54.160+0000 7f1bf4967640 1 -- --> v1:172.21.15.43:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1bf00cdc70 con 0x7f1bf00cbd90 2023-08-09T13:53:54.163 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:54.160+0000 7f1bf4967640 1 -- --> v1:172.21.15.203:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1bf0070910 con 0x7f1bf00c96c0 2023-08-09T13:53:54.163 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:54.160+0000 7f1bf4967640 1 -- --> v2:172.21.15.43:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1bf00c88c0 con 0x7f1bf015a6a0 2023-08-09T13:53:54.163 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:54.160+0000 7f1bef7fe640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f1bf015a6a0 0x7f1bf00713b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:53:54.164 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:54.160+0000 7f1bef7fe640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f1bf015a6a0 0x7f1bf00713b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:56420/0 (socket says 172.21.15.43:56420) 2023-08-09T13:53:54.164 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:54.160+0000 7f1bef7fe640 1 -- 172.21.15.43:0/2267645782 learned_addr learned my addr 172.21.15.43:0/2267645782 (peer_addr_for_me v2:172.21.15.43:0/0) 2023-08-09T13:53:54.164 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:54.161+0000 7f1bef7fe640 1 -- 172.21.15.43:0/2267645782 >> v1:172.21.15.203:6789/0 conn(0x7f1bf00c96c0 legacy=0x7f1bf015ad20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-09T13:53:54.164 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:54.161+0000 7f1bef7fe640 1 -- 172.21.15.43:0/2267645782 >> v1:172.21.15.43:6789/0 conn(0x7f1bf00cbd90 legacy=0x7f1bf00c8fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:53:54.164 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:54.161+0000 7f1bee7fc640 1 -- 172.21.15.43:0/2267645782 <== mon.0 v1:172.21.15.43:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3793991400 0 0) 0x7f1bf00cdc70 con 0x7f1bf00cbd90 2023-08-09T13:53:54.164 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:54.161+0000 7f1bef7fe640 1 -- 172.21.15.43:0/2267645782 --> v2:172.21.15.43:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1bf0070bb0 con 0x7f1bf015a6a0 2023-08-09T13:53:54.165 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:54.162+0000 7f1bef7fe640 1 --2- 172.21.15.43:0/2267645782 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f1bf015a6a0 0x7f1bf00713b0 secure :-1 s=READY pgs=1530 cs=0 l=1 rev1=1 crypto rx=0x7f1be0009870 tx=0x7f1be002b910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2135d4a2a55f43d6 server_cookie=0 in_seq=0 out_seq=0 2023-08-09T13:53:54.165 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:54.162+0000 7f1bee7fc640 1 -- 172.21.15.43:0/2267645782 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1be0031870 con 0x7f1bf015a6a0 2023-08-09T13:53:54.166 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:54.162+0000 7f1bee7fc640 1 -- 172.21.15.43:0/2267645782 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1be00319d0 con 0x7f1bf015a6a0 2023-08-09T13:53:54.166 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:54.162+0000 7f1bee7fc640 1 -- 172.21.15.43:0/2267645782 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1be0031d60 con 0x7f1bf015a6a0 2023-08-09T13:53:54.166 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:54.163+0000 7f1bf4967640 1 -- 172.21.15.43:0/2267645782 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f1bf015a6a0 msgr2=0x7f1bf00713b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:53:54.166 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:54.163+0000 7f1bf4967640 1 --2- 172.21.15.43:0/2267645782 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f1bf015a6a0 0x7f1bf00713b0 secure :-1 s=READY pgs=1530 cs=0 l=1 rev1=1 crypto rx=0x7f1be0009870 tx=0x7f1be002b910 comp rx=0 tx=0).stop 2023-08-09T13:53:54.166 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:54.163+0000 7f1bf4967640 1 -- 172.21.15.43:0/2267645782 shutdown_connections 2023-08-09T13:53:54.166 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:54.163+0000 7f1bf4967640 1 --2- 172.21.15.43:0/2267645782 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f1bf015a6a0 0x7f1bf00713b0 unknown :-1 s=CLOSED pgs=1530 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:53:54.166 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:54.163+0000 7f1bf4967640 1 -- 172.21.15.43:0/2267645782 >> 172.21.15.43:0/2267645782 conn(0x7f1bf00cde90 msgr2=0x7f1bf00ce2a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-09T13:53:54.167 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:54.163+0000 7f1bf4967640 1 -- 172.21.15.43:0/2267645782 shutdown_connections 2023-08-09T13:53:54.167 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:54.163+0000 7f1bf4967640 1 -- 172.21.15.43:0/2267645782 wait complete. 2023-08-09T13:53:54.167 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:54.165+0000 7f1bf4967640 1 Processor -- start 2023-08-09T13:53:54.167 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:54.165+0000 7f1bf4967640 1 -- start start 2023-08-09T13:53:54.168 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:54.166+0000 7f1bf4967640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f1bf00c96c0 0x7f1bf0155860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:53:54.169 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:54.166+0000 7f1bf4967640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f1bf00cbd90 0x7f1bf009afc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:53:54.169 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:54.166+0000 7f1bf4967640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f1bf015a6a0 0x7f1bf00cc600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:53:54.169 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:54.166+0000 7f1bf4967640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1bf00b24e0 con 0x7f1bf00c96c0 2023-08-09T13:53:54.169 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:54.166+0000 7f1bf4967640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1bf00b27c0 con 0x7f1bf00cbd90 2023-08-09T13:53:54.169 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:54.166+0000 7f1bf4967640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1bf00b2aa0 con 0x7f1bf015a6a0 2023-08-09T13:53:54.169 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:54.166+0000 7f1bef7fe640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f1bf00c96c0 0x7f1bf0155860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:53:54.169 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:54.166+0000 7f1bef7fe640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f1bf00c96c0 0x7f1bf0155860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:56432/0 (socket says 172.21.15.43:56432) 2023-08-09T13:53:54.170 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:54.166+0000 7f1bef7fe640 1 -- 172.21.15.43:0/419351935 learned_addr learned my addr 172.21.15.43:0/419351935 (peer_addr_for_me v2:172.21.15.43:0/0) 2023-08-09T13:53:54.170 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:54.167+0000 7f1bef7fe640 1 -- 172.21.15.43:0/419351935 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f1bf015a6a0 msgr2=0x7f1bf00cc600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:53:54.170 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:54.167+0000 7f1bef7fe640 1 --2- 172.21.15.43:0/419351935 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f1bf015a6a0 0x7f1bf00cc600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:53:54.171 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:54.167+0000 7f1bef7fe640 1 -- 172.21.15.43:0/419351935 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f1bf00cbd90 msgr2=0x7f1bf009afc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:53:54.171 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:54.167+0000 7f1bef7fe640 1 --2- 172.21.15.43:0/419351935 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f1bf00cbd90 0x7f1bf009afc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:53:54.171 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:54.167+0000 7f1bef7fe640 1 -- 172.21.15.43:0/419351935 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1be00094d0 con 0x7f1bf00c96c0 2023-08-09T13:53:54.171 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:54.167+0000 7f1bef7fe640 1 --2- 172.21.15.43:0/419351935 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f1bf00c96c0 0x7f1bf0155860 secure :-1 s=READY pgs=1531 cs=0 l=1 rev1=1 crypto rx=0x7f1be0009870 tx=0x7f1be00046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-09T13:53:54.171 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:54.167+0000 7f1bd7fff640 1 -- 172.21.15.43:0/419351935 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1be003ad30 con 0x7f1bf00c96c0 2023-08-09T13:53:54.171 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:54.167+0000 7f1bd7fff640 1 -- 172.21.15.43:0/419351935 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1be003ae90 con 0x7f1bf00c96c0 2023-08-09T13:53:54.171 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:54.167+0000 7f1bd7fff640 1 -- 172.21.15.43:0/419351935 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1be0044630 con 0x7f1bf00c96c0 2023-08-09T13:53:54.172 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:54.167+0000 7f1bf4967640 1 -- 172.21.15.43:0/419351935 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1bf00b2d80 con 0x7f1bf00c96c0 2023-08-09T13:53:54.172 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:54.168+0000 7f1bf4967640 1 -- 172.21.15.43:0/419351935 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1bf00b1800 con 0x7f1bf00c96c0 2023-08-09T13:53:54.172 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:54.169+0000 7f1bf4967640 1 -- 172.21.15.43:0/419351935 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1bf00b1a50 con 0x7f1bf00c96c0 2023-08-09T13:53:54.172 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:54.169+0000 7f1bd7fff640 1 -- 172.21.15.43:0/419351935 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 56) v1 ==== 88443+0+0 (secure 0 0 0) 0x7f1be004d450 con 0x7f1bf00c96c0 2023-08-09T13:53:54.173 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:54.170+0000 7f1bd7fff640 1 --2- 172.21.15.43:0/419351935 >> [v2:172.21.15.203:6832/2527059680,v1:172.21.15.203:6833/2527059680] conn(0x7f1bdc06a170 0x7f1bdc06c630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:53:54.173 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:54.170+0000 7f1bd7fff640 1 -- 172.21.15.43:0/419351935 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(129..129 src has 1..129) v4 ==== 6426+0+0 (secure 0 0 0) 0x7f1be002d040 con 0x7f1bf00c96c0 2023-08-09T13:53:54.173 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:54.171+0000 7f1beeffd640 1 --2- 172.21.15.43:0/419351935 >> [v2:172.21.15.203:6832/2527059680,v1:172.21.15.203:6833/2527059680] conn(0x7f1bdc06a170 0x7f1bdc06c630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:53:54.174 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:54.172+0000 7f1beeffd640 1 --2- 172.21.15.43:0/419351935 >> [v2:172.21.15.203:6832/2527059680,v1:172.21.15.203:6833/2527059680] conn(0x7f1bdc06a170 0x7f1bdc06c630 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f1bf015dd10 tx=0x7f1be4007910 comp rx=0 tx=0).ready entity=mgr.6377 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-09T13:53:54.177 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:54.175+0000 7f1bd7fff640 1 -- 172.21.15.43:0/419351935 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178066 (secure 0 0 0) 0x7f1be0076450 con 0x7f1bf00c96c0 2023-08-09T13:53:54.224 INFO:tasks.ceph.mds.a.smithi043.stderr:2023-08-09T13:53:54.221+0000 7fb54aad3640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2023-08-09T13:53:54.400 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:54.397+0000 7f1bf4967640 1 -- 172.21.15.43:0/419351935 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f1bf00b2620 con 0x7f1bf00c96c0 2023-08-09T13:53:54.401 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:54.399+0000 7f1bd7fff640 1 -- 172.21.15.43:0/419351935 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 61 v61) v1 ==== 94+0+4880 (secure 0 0 0) 0x7f1bf00b2620 con 0x7f1bf00c96c0 2023-08-09T13:53:54.401 INFO:teuthology.orchestra.run.smithi043.stdout: 2023-08-09T13:53:54.401 INFO:teuthology.orchestra.run.smithi043.stdout:{"epoch":61,"default_fscid":7,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6530,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.203:6835/1659419253","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6834","nonce":1659419253},{"type":"v1","addr":"172.21.15.203:6835","nonce":1659419253}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":56},{"gid":6536,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.203:6837/3964427549","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6836","nonce":3964427549},{"type":"v1","addr":"172.21.15.203:6837","nonce":3964427549}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":56},{"gid":8574,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.43:6833/871812392","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6832","nonce":871812392},{"type":"v1","addr":"172.21.15.43:6833","nonce":871812392}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":55}],"filesystems":[{"mdsmap":{"epoch":61,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-08-09T13:53:47.747881+0000","modified":"2023-08-09T13:53:51.317185+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":8580},"failed":[],"damaged":[],"stopped":[],"info":{"gid_8580":{"gid":8580,"name":"a","rank":0,"incarnation":57,"state":"up:active","state_seq":3,"addr":"172.21.15.43:6835/1794836673","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6834","nonce":1794836673},{"type":"v1","addr":"172.21.15.43:6835","nonce":1794836673}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[17],"metadata_pool":16,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":7}]} 2023-08-09T13:53:54.402 INFO:teuthology.orchestra.run.smithi043.stderr:dumped fsmap epoch 61 2023-08-09T13:53:54.403 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:54.401+0000 7f1bf4967640 1 -- 172.21.15.43:0/419351935 >> [v2:172.21.15.203:6832/2527059680,v1:172.21.15.203:6833/2527059680] conn(0x7f1bdc06a170 msgr2=0x7f1bdc06c630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:53:54.404 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:54.401+0000 7f1bf4967640 1 --2- 172.21.15.43:0/419351935 >> [v2:172.21.15.203:6832/2527059680,v1:172.21.15.203:6833/2527059680] conn(0x7f1bdc06a170 0x7f1bdc06c630 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f1bf015dd10 tx=0x7f1be4007910 comp rx=0 tx=0).stop 2023-08-09T13:53:54.404 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:54.401+0000 7f1bf4967640 1 -- 172.21.15.43:0/419351935 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f1bf00c96c0 msgr2=0x7f1bf0155860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:53:54.404 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:54.401+0000 7f1bf4967640 1 --2- 172.21.15.43:0/419351935 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f1bf00c96c0 0x7f1bf0155860 secure :-1 s=READY pgs=1531 cs=0 l=1 rev1=1 crypto rx=0x7f1be0009870 tx=0x7f1be00046c0 comp rx=0 tx=0).stop 2023-08-09T13:53:54.405 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:54.401+0000 7f1bf4967640 1 -- 172.21.15.43:0/419351935 shutdown_connections 2023-08-09T13:53:54.405 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:54.401+0000 7f1bf4967640 1 --2- 172.21.15.43:0/419351935 >> [v2:172.21.15.203:6832/2527059680,v1:172.21.15.203:6833/2527059680] conn(0x7f1bdc06a170 0x7f1bdc06c630 unknown :-1 s=CLOSED pgs=67 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:53:54.405 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:54.401+0000 7f1bf4967640 1 --2- 172.21.15.43:0/419351935 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f1bf015a6a0 0x7f1bf00cc600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:53:54.405 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:54.401+0000 7f1bf4967640 1 --2- 172.21.15.43:0/419351935 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f1bf00cbd90 0x7f1bf009afc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:53:54.405 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:54.401+0000 7f1bf4967640 1 --2- 172.21.15.43:0/419351935 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f1bf00c96c0 0x7f1bf0155860 unknown :-1 s=CLOSED pgs=1531 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:53:54.405 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:54.401+0000 7f1bf4967640 1 -- 172.21.15.43:0/419351935 >> 172.21.15.43:0/419351935 conn(0x7f1bf00cde90 msgr2=0x7f1bf015abf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-09T13:53:54.405 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:54.402+0000 7f1bf4967640 1 -- 172.21.15.43:0/419351935 shutdown_connections 2023-08-09T13:53:54.406 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:54.402+0000 7f1bf4967640 1 -- 172.21.15.43:0/419351935 wait complete. 2023-08-09T13:53:54.823 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 61, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2023-08-09T13:53:47.747881+0000', 'modified': '2023-08-09T13:53:51.317185+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 8580}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_8580': {'gid': 8580, 'name': 'a', 'rank': 0, 'incarnation': 57, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.43:6835/1794836673', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.43:6834', 'nonce': 1794836673}, {'type': 'v1', 'addr': '172.21.15.43:6835', 'nonce': 1794836673}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138322906710015, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}}, 'data_pools': [17], 'metadata_pool': 16, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0} 2023-08-09T13:53:54.824 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2023-08-09T13:53:54.824 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.a status 2023-08-09T13:53:55.066 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:55.063+0000 7f0507fff640 1 Processor -- start 2023-08-09T13:53:55.066 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:55.063+0000 7f0507fff640 1 -- start start 2023-08-09T13:53:55.067 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:55.065+0000 7f0507fff640 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7f050815c8e0 0x7f050806f3d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:53:55.067 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:55.065+0000 7f0507fff640 1 -- --> v1:172.21.15.43:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0508070730 con 0x7f05080abbb0 2023-08-09T13:53:55.067 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:55.065+0000 7f0507fff640 1 -- --> v1:172.21.15.203:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0508071620 con 0x7f05080c8670 2023-08-09T13:53:55.068 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:55.065+0000 7f0507fff640 1 -- --> v2:172.21.15.203:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0508156e60 con 0x7f050815c8e0 2023-08-09T13:53:55.069 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:55.065+0000 7f05077fe640 1 --1- >> v1:172.21.15.43:6789/0 conn(0x7f05080abbb0 0x7f05080ac000 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.43:6789/0 says I am v1:172.21.15.43:48516/0 (socket says 172.21.15.43:48516) 2023-08-09T13:53:55.069 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:55.065+0000 7f05077fe640 1 -- 172.21.15.43:0/2273844098 learned_addr learned my addr 172.21.15.43:0/2273844098 (peer_addr_for_me v1:172.21.15.43:0/0) 2023-08-09T13:53:55.069 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:55.066+0000 7f05067fc640 1 -- 172.21.15.43:0/2273844098 <== mon.0 v1:172.21.15.43:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2002989780 0 0) 0x7f0508070730 con 0x7f05080abbb0 2023-08-09T13:53:55.069 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:55.066+0000 7f04fffff640 1 --2- 172.21.15.43:0/2273844098 >> v2:172.21.15.203:3300/0 conn(0x7f050815c8e0 0x7f050806f3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:53:55.069 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:55.066+0000 7f05067fc640 1 -- 172.21.15.43:0/2273844098 --> v1:172.21.15.43:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f04f4003520 con 0x7f05080abbb0 2023-08-09T13:53:55.070 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:55.066+0000 7f05067fc640 1 -- 172.21.15.43:0/2273844098 <== mon.0 v1:172.21.15.43:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 660253255 0 0) 0x7f04f4003520 con 0x7f05080abbb0 2023-08-09T13:53:55.070 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:55.066+0000 7f05067fc640 1 -- 172.21.15.43:0/2273844098 >> v2:172.21.15.203:3300/0 conn(0x7f050815c8e0 msgr2=0x7f050806f3d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-09T13:53:55.070 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:55.066+0000 7f05067fc640 1 --2- 172.21.15.43:0/2273844098 >> v2:172.21.15.203:3300/0 conn(0x7f050815c8e0 0x7f050806f3d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:53:55.070 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:55.066+0000 7f05067fc640 1 -- 172.21.15.43:0/2273844098 >> v1:172.21.15.203:6789/0 conn(0x7f05080c8670 legacy=0x7f05080c8a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:53:55.070 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:55.066+0000 7f05067fc640 1 -- 172.21.15.43:0/2273844098 --> v1:172.21.15.43:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f050806fb20 con 0x7f05080abbb0 2023-08-09T13:53:55.070 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:55.066+0000 7f05067fc640 1 -- 172.21.15.43:0/2273844098 <== mon.0 v1:172.21.15.43:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1285150740 0 0) 0x7f05000019f0 con 0x7f05080abbb0 2023-08-09T13:53:55.070 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:55.066+0000 7f05067fc640 1 -- 172.21.15.43:0/2273844098 >> v1:172.21.15.43:6789/0 conn(0x7f05080abbb0 legacy=0x7f05080ac000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:53:55.071 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:55.066+0000 7f05067fc640 1 --2- 172.21.15.43:0/2273844098 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f04f4003970 0x7f04f4003d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:53:55.071 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:55.066+0000 7f05067fc640 1 --2- 172.21.15.43:0/2273844098 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f04f40047c0 0x7f04f4004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:53:55.071 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:55.066+0000 7f05067fc640 1 --2- 172.21.15.43:0/2273844098 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f04f4005380 0x7f04f400b980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:53:55.071 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:55.066+0000 7f05067fc640 1 -- 172.21.15.43:0/2273844098 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0508156e60 con 0x7f04f4005380 2023-08-09T13:53:55.071 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:55.066+0000 7f05067fc640 1 -- 172.21.15.43:0/2273844098 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f04f400c210 con 0x7f04f4003970 2023-08-09T13:53:55.071 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:55.066+0000 7f05067fc640 1 -- 172.21.15.43:0/2273844098 --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f04f400c3d0 con 0x7f04f40047c0 2023-08-09T13:53:55.071 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:55.066+0000 7f05077fe640 1 --2- 172.21.15.43:0/2273844098 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f04f4005380 0x7f04f400b980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:53:55.071 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:55.067+0000 7f0506ffd640 1 --2- 172.21.15.43:0/2273844098 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f04f4003970 0x7f04f4003d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:53:55.073 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:55.067+0000 7f04fffff640 1 --2- 172.21.15.43:0/2273844098 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f04f40047c0 0x7f04f4004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:53:55.073 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:55.067+0000 7f05077fe640 1 -- 172.21.15.43:0/2273844098 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f04f40047c0 msgr2=0x7f04f4004c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-09T13:53:55.073 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:55.067+0000 7f05077fe640 1 --2- 172.21.15.43:0/2273844098 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f04f40047c0 0x7f04f4004c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:53:55.073 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:55.067+0000 7f05077fe640 1 -- 172.21.15.43:0/2273844098 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f04f4003970 msgr2=0x7f04f4003d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-09T13:53:55.073 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:55.067+0000 7f05077fe640 1 --2- 172.21.15.43:0/2273844098 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f04f4003970 0x7f04f4003d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:53:55.073 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:55.067+0000 7f05077fe640 1 -- 172.21.15.43:0/2273844098 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f04f4003520 con 0x7f04f4005380 2023-08-09T13:53:55.074 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:55.067+0000 7f04fffff640 1 --2- 172.21.15.43:0/2273844098 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f04f40047c0 0x7f04f4004c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-08-09T13:53:55.074 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:55.068+0000 7f05077fe640 1 -- 172.21.15.43:0/2273844098 reap_dead start 2023-08-09T13:53:55.074 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:55.068+0000 7f05077fe640 1 --2- 172.21.15.43:0/2273844098 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f04f4005380 0x7f04f400b980 secure :-1 s=READY pgs=1533 cs=0 l=1 rev1=1 crypto rx=0x7f050000a830 tx=0x7f04f40057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=30ac58d64348687 server_cookie=0 in_seq=0 out_seq=0 2023-08-09T13:53:55.074 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:55.068+0000 7f05067fc640 1 -- 172.21.15.43:0/2273844098 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f05000072d0 con 0x7f04f4005380 2023-08-09T13:53:55.074 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:55.068+0000 7f05067fc640 1 -- 172.21.15.43:0/2273844098 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f050002f070 con 0x7f04f4005380 2023-08-09T13:53:55.074 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:55.068+0000 7f05067fc640 1 -- 172.21.15.43:0/2273844098 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f05000076b0 con 0x7f04f4005380 2023-08-09T13:53:55.074 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:55.069+0000 7f0507fff640 1 -- 172.21.15.43:0/2273844098 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f04f4005380 msgr2=0x7f04f400b980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:53:55.075 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:55.069+0000 7f0507fff640 1 --2- 172.21.15.43:0/2273844098 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f04f4005380 0x7f04f400b980 secure :-1 s=READY pgs=1533 cs=0 l=1 rev1=1 crypto rx=0x7f050000a830 tx=0x7f04f40057f0 comp rx=0 tx=0).stop 2023-08-09T13:53:55.075 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:55.069+0000 7f0507fff640 1 -- 172.21.15.43:0/2273844098 shutdown_connections 2023-08-09T13:53:55.075 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:55.069+0000 7f0507fff640 1 --2- 172.21.15.43:0/2273844098 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f04f4005380 0x7f04f400b980 unknown :-1 s=CLOSED pgs=1533 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:53:55.075 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:55.069+0000 7f0507fff640 1 -- 172.21.15.43:0/2273844098 >> 172.21.15.43:0/2273844098 conn(0x7f05080af950 msgr2=0x7f05080afd60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-09T13:53:55.075 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:55.069+0000 7f0507fff640 1 -- 172.21.15.43:0/2273844098 shutdown_connections 2023-08-09T13:53:55.075 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:55.069+0000 7f0507fff640 1 -- 172.21.15.43:0/2273844098 wait complete. 2023-08-09T13:53:55.077 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:55.071+0000 7f0507fff640 1 Processor -- start 2023-08-09T13:53:55.077 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:55.071+0000 7f0507fff640 1 -- start start 2023-08-09T13:53:55.077 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:55.072+0000 7f0507fff640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f04f4005380 0x7f05081f5860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:53:55.077 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:55.072+0000 7f0507fff640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f05080c3800 0x7f05080c1990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:53:55.077 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:55.072+0000 7f0507fff640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f05080c1ed0 0x7f05080adca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:53:55.077 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:55.072+0000 7f0507fff640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f05080c2350 con 0x7f05080c3800 2023-08-09T13:53:55.078 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:55.072+0000 7f0507fff640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f05081f5da0 con 0x7f04f4005380 2023-08-09T13:53:55.078 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:55.072+0000 7f0507fff640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f05080c4060 con 0x7f05080c1ed0 2023-08-09T13:53:55.078 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:55.072+0000 7f04fffff640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f05080c3800 0x7f05080c1990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:53:55.078 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:55.072+0000 7f04fffff640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f05080c3800 0x7f05080c1990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:34298/0 (socket says 172.21.15.43:34298) 2023-08-09T13:53:55.078 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:55.072+0000 7f04fffff640 1 -- 172.21.15.43:0/3582121083 learned_addr learned my addr 172.21.15.43:0/3582121083 (peer_addr_for_me v2:172.21.15.43:0/0) 2023-08-09T13:53:55.078 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:55.073+0000 7f0506ffd640 1 --2- 172.21.15.43:0/3582121083 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f04f4005380 0x7f05081f5860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:53:55.078 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:55.073+0000 7f04fffff640 1 -- 172.21.15.43:0/3582121083 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f05080c1ed0 msgr2=0x7f05080adca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:53:55.079 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:55.073+0000 7f04fffff640 1 --2- 172.21.15.43:0/3582121083 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f05080c1ed0 0x7f05080adca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:53:55.079 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:55.073+0000 7f04fffff640 1 -- 172.21.15.43:0/3582121083 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f04f4005380 msgr2=0x7f05081f5860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:53:55.079 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:55.073+0000 7f04fffff640 1 --2- 172.21.15.43:0/3582121083 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f04f4005380 0x7f05081f5860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:53:55.079 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:55.073+0000 7f04fffff640 1 -- 172.21.15.43:0/3582121083 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f05080c4340 con 0x7f05080c3800 2023-08-09T13:53:55.079 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:55.073+0000 7f0506ffd640 1 --2- 172.21.15.43:0/3582121083 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f04f4005380 0x7f05081f5860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-08-09T13:53:55.079 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:55.073+0000 7f04fffff640 1 --2- 172.21.15.43:0/3582121083 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f05080c3800 0x7f05080c1990 secure :-1 s=READY pgs=1534 cs=0 l=1 rev1=1 crypto rx=0x7f04f000e7e0 tx=0x7f04f000eca0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-09T13:53:55.079 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:55.073+0000 7f04ff7fe640 1 -- 172.21.15.43:0/3582121083 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f04f000c6d0 con 0x7f05080c3800 2023-08-09T13:53:55.079 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:55.073+0000 7f04ff7fe640 1 -- 172.21.15.43:0/3582121083 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f04f000c830 con 0x7f05080c3800 2023-08-09T13:53:55.080 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:55.073+0000 7f04ff7fe640 1 -- 172.21.15.43:0/3582121083 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f04f000cba0 con 0x7f05080c3800 2023-08-09T13:53:55.080 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:55.073+0000 7f0507fff640 1 -- 172.21.15.43:0/3582121083 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f05080c4620 con 0x7f05080c3800 2023-08-09T13:53:55.080 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:55.074+0000 7f0507fff640 1 -- 172.21.15.43:0/3582121083 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f05080ad780 con 0x7f05080c3800 2023-08-09T13:53:55.081 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:55.076+0000 7f04ff7fe640 1 -- 172.21.15.43:0/3582121083 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 56) v1 ==== 88443+0+0 (secure 0 0 0) 0x7f04f000cd00 con 0x7f05080c3800 2023-08-09T13:53:55.081 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:55.077+0000 7f04ff7fe640 1 --2- 172.21.15.43:0/3582121083 >> [v2:172.21.15.203:6832/2527059680,v1:172.21.15.203:6833/2527059680] conn(0x7f04e806a170 0x7f04e806c630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:53:55.081 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:55.077+0000 7f04ff7fe640 1 -- 172.21.15.43:0/3582121083 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(129..129 src has 1..129) v4 ==== 6426+0+0 (secure 0 0 0) 0x7f04f0088770 con 0x7f05080c3800 2023-08-09T13:53:55.081 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:55.078+0000 7f0506ffd640 1 --2- 172.21.15.43:0/3582121083 >> [v2:172.21.15.203:6832/2527059680,v1:172.21.15.203:6833/2527059680] conn(0x7f04e806a170 0x7f04e806c630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:53:55.081 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:55.078+0000 7f050fdf9740 1 Processor -- start 2023-08-09T13:53:55.081 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:55.078+0000 7f050fdf9740 1 -- start start 2023-08-09T13:53:55.082 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:55.079+0000 7f050fdf9740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5585ddd19e30 0x5585ddd1a210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:53:55.083 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:55.079+0000 7f050fdf9740 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5585ddce9db0 0x5585ddd170a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:53:55.083 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:55.079+0000 7f050fdf9740 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x5585ddced9b0 0x5585ddd175e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:53:55.083 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:55.079+0000 7f050fdf9740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x5585ddc5b8f0 con 0x5585ddd19e30 2023-08-09T13:53:55.083 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:55.079+0000 7f050fdf9740 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x5585ddc58fe0 con 0x5585ddce9db0 2023-08-09T13:53:55.083 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:55.079+0000 7f050fdf9740 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x5585ddc548d0 con 0x5585ddced9b0 2023-08-09T13:53:55.083 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:55.079+0000 7f0506ffd640 1 --2- 172.21.15.43:0/3582121083 >> [v2:172.21.15.203:6832/2527059680,v1:172.21.15.203:6833/2527059680] conn(0x7f04e806a170 0x7f04e806c630 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f04f8001630 tx=0x7f04f8009040 comp rx=0 tx=0).ready entity=mgr.6377 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-09T13:53:55.084 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:55.079+0000 7f0506ffd640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5585ddd19e30 0x5585ddd1a210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:53:55.084 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:55.079+0000 7f0506ffd640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5585ddd19e30 0x5585ddd1a210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:34308/0 (socket says 172.21.15.43:34308) 2023-08-09T13:53:55.084 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:55.079+0000 7f0506ffd640 1 -- 172.21.15.43:0/3745053112 learned_addr learned my addr 172.21.15.43:0/3745053112 (peer_addr_for_me v2:172.21.15.43:0/0) 2023-08-09T13:53:55.084 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:55.079+0000 7f04fffff640 1 --2- 172.21.15.43:0/3745053112 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5585ddce9db0 0x5585ddd170a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:53:55.084 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:55.080+0000 7f05077fe640 1 --2- 172.21.15.43:0/3745053112 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x5585ddced9b0 0x5585ddd175e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:53:55.084 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:55.080+0000 7f0506ffd640 1 -- 172.21.15.43:0/3745053112 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x5585ddced9b0 msgr2=0x5585ddd175e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-09T13:53:55.084 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:55.080+0000 7f0506ffd640 1 --2- 172.21.15.43:0/3745053112 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x5585ddced9b0 0x5585ddd175e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:53:55.085 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:55.080+0000 7f0506ffd640 1 -- 172.21.15.43:0/3745053112 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5585ddce9db0 msgr2=0x5585ddd170a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-09T13:53:55.085 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:55.080+0000 7f0506ffd640 1 --2- 172.21.15.43:0/3745053112 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5585ddce9db0 0x5585ddd170a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:53:55.085 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:55.080+0000 7f0506ffd640 1 -- 172.21.15.43:0/3745053112 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5585ddc0dac0 con 0x5585ddd19e30 2023-08-09T13:53:55.085 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:55.080+0000 7f0506ffd640 1 --2- 172.21.15.43:0/3745053112 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5585ddd19e30 0x5585ddd1a210 secure :-1 s=READY pgs=1535 cs=0 l=1 rev1=1 crypto rx=0x7f04f800f7a0 tx=0x7f04f8012000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6d354bf8d74a193e server_cookie=0 in_seq=0 out_seq=0 2023-08-09T13:53:55.085 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:55.080+0000 7f04fcff9640 1 -- 172.21.15.43:0/3745053112 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f04f8007240 con 0x5585ddd19e30 2023-08-09T13:53:55.085 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:55.080+0000 7f04fcff9640 1 -- 172.21.15.43:0/3745053112 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f04f800d050 con 0x5585ddd19e30 2023-08-09T13:53:55.085 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:55.080+0000 7f04fcff9640 1 -- 172.21.15.43:0/3745053112 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f04f80075a0 con 0x5585ddd19e30 2023-08-09T13:53:55.088 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:55.081+0000 7f050fdf9740 1 -- 172.21.15.43:0/3745053112 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5585ddd19e30 msgr2=0x5585ddd1a210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:53:55.088 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:55.081+0000 7f050fdf9740 1 --2- 172.21.15.43:0/3745053112 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5585ddd19e30 0x5585ddd1a210 secure :-1 s=READY pgs=1535 cs=0 l=1 rev1=1 crypto rx=0x7f04f800f7a0 tx=0x7f04f8012000 comp rx=0 tx=0).stop 2023-08-09T13:53:55.088 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:55.081+0000 7f050fdf9740 1 -- 172.21.15.43:0/3745053112 shutdown_connections 2023-08-09T13:53:55.088 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:55.081+0000 7f050fdf9740 1 --2- 172.21.15.43:0/3745053112 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x5585ddced9b0 0x5585ddd175e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:53:55.088 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:55.081+0000 7f050fdf9740 1 --2- 172.21.15.43:0/3745053112 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5585ddce9db0 0x5585ddd170a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:53:55.088 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:55.081+0000 7f050fdf9740 1 --2- 172.21.15.43:0/3745053112 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5585ddd19e30 0x5585ddd1a210 unknown :-1 s=CLOSED pgs=1535 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:53:55.089 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:55.081+0000 7f050fdf9740 1 -- 172.21.15.43:0/3745053112 >> 172.21.15.43:0/3745053112 conn(0x5585ddcec0c0 msgr2=0x5585ddd34c40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-09T13:53:55.089 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:55.081+0000 7f050fdf9740 1 -- 172.21.15.43:0/3745053112 shutdown_connections 2023-08-09T13:53:55.089 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:55.081+0000 7f050fdf9740 1 -- 172.21.15.43:0/3745053112 wait complete. 2023-08-09T13:53:55.089 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:55.082+0000 7f050fdf9740 1 -- start start 2023-08-09T13:53:55.089 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:55.082+0000 7f050fdf9740 1 Processor -- start 2023-08-09T13:53:55.089 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:55.083+0000 7f050fdf9740 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5585ddced9b0 0x5585ddd1eac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:53:55.089 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:55.083+0000 7f050fdf9740 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x5585ddd19e30 0x5585ddd12df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:53:55.090 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:55.083+0000 7f050fdf9740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5585ddd13330 0x5585ddd34c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:53:55.090 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:55.083+0000 7f050fdf9740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x5585ddbf4d40 con 0x5585ddd13330 2023-08-09T13:53:55.090 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:55.083+0000 7f050fdf9740 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x5585ddbf49e0 con 0x5585ddced9b0 2023-08-09T13:53:55.090 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:55.083+0000 7f050fdf9740 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x5585dd9baf20 con 0x5585ddd19e30 2023-08-09T13:53:55.090 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:55.083+0000 7f050fdf9740 10 client.0 _ll_register_callbacks cb 0x5585ddc0d160 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-08-09T13:53:55.090 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:55.084+0000 7f05077fe640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5585ddd13330 0x5585ddd34c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:53:55.090 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:55.084+0000 7f05077fe640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5585ddd13330 0x5585ddd34c90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:34322/0 (socket says 172.21.15.43:34322) 2023-08-09T13:53:55.090 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:55.084+0000 7f05077fe640 1 -- 172.21.15.43:0/2760438372 learned_addr learned my addr 172.21.15.43:0/2760438372 (peer_addr_for_me v2:172.21.15.43:0/0) 2023-08-09T13:53:55.091 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:55.084+0000 7f0506ffd640 1 --2- 172.21.15.43:0/2760438372 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5585ddced9b0 0x5585ddd1eac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:53:55.091 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:55.084+0000 7f04fffff640 1 --2- 172.21.15.43:0/2760438372 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x5585ddd19e30 0x5585ddd12df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:53:55.091 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:55.084+0000 7f05077fe640 1 -- 172.21.15.43:0/2760438372 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x5585ddd19e30 msgr2=0x5585ddd12df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-09T13:53:55.091 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:55.084+0000 7f05077fe640 1 --2- 172.21.15.43:0/2760438372 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x5585ddd19e30 0x5585ddd12df0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:53:55.091 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:55.084+0000 7f05077fe640 1 -- 172.21.15.43:0/2760438372 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5585ddced9b0 msgr2=0x5585ddd1eac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-09T13:53:55.091 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:55.084+0000 7f05077fe640 1 --2- 172.21.15.43:0/2760438372 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5585ddced9b0 0x5585ddd1eac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:53:55.091 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:55.084+0000 7f05077fe640 1 -- 172.21.15.43:0/2760438372 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0508071620 con 0x5585ddd13330 2023-08-09T13:53:55.092 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:55.084+0000 7f0506ffd640 1 --2- 172.21.15.43:0/2760438372 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5585ddced9b0 0x5585ddd1eac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-08-09T13:53:55.092 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:55.084+0000 7f05077fe640 1 --2- 172.21.15.43:0/2760438372 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5585ddd13330 0x5585ddd34c90 secure :-1 s=READY pgs=1536 cs=0 l=1 rev1=1 crypto rx=0x7f050000a920 tx=0x7f0500001d80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a8a05a724ca5351c server_cookie=0 in_seq=0 out_seq=0 2023-08-09T13:53:55.092 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:55.084+0000 7f04e5ffb640 10 client.0 ms_handle_connect on v2:172.21.15.43:3300/0 2023-08-09T13:53:55.092 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:55.085+0000 7f04e5ffb640 1 -- 172.21.15.43:0/2760438372 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0500001730 con 0x5585ddd13330 2023-08-09T13:53:55.092 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:55.085+0000 7f050fdf9740 1 -- 172.21.15.43:0/2760438372 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5585ddc2c500 con 0x5585ddd13330 2023-08-09T13:53:55.092 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:55.085+0000 7f04e5ffb640 1 -- 172.21.15.43:0/2760438372 <== mon.0 v2:172.21.15.43:3300/0 2 ==== osd_map(129..129 src has 1..129) v4 ==== 6426+0+0 (secure 0 0 0) 0x7f0500001890 con 0x5585ddd13330 2023-08-09T13:53:55.093 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:55.085+0000 7f04e5ffb640 1 -- 172.21.15.43:0/2760438372 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_get_version_reply(handle=1 version=61) v2 ==== 24+0+0 (secure 0 0 0) 0x7f0500001a50 con 0x5585ddd13330 2023-08-09T13:53:55.093 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:55.085+0000 7f050fdf9740 10 client.8730 fetch_fsmap learned FSMap version 61 2023-08-09T13:53:55.093 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:55.085+0000 7f050fdf9740 1 -- 172.21.15.43:0/2760438372 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,fsmap=61,monmap=2+}) v3 -- 0x5585ddc52f50 con 0x5585ddd13330 2023-08-09T13:53:55.093 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:55.085+0000 7f04e5ffb640 1 -- 172.21.15.43:0/2760438372 <== mon.0 v2:172.21.15.43:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x5585ddc2c500 con 0x5585ddd13330 2023-08-09T13:53:55.093 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:55.085+0000 7f04e5ffb640 1 -- 172.21.15.43:0/2760438372 <== mon.0 v2:172.21.15.43:3300/0 5 ==== fsmap(e 61) v1 ==== 3294+0+0 (secure 0 0 0) 0x7f0500034430 con 0x5585ddd13330 2023-08-09T13:53:55.093 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:55.085+0000 7f050fdf9740 10 client.8730 fetch_fsmap finished waiting for FSMap version 61 2023-08-09T13:53:55.093 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:55.085+0000 7f050fdf9740 10 client.8730 resolve_mds: resolved name 'a' to daemon mds.a 2023-08-09T13:53:55.093 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:55.085+0000 7f050fdf9740 20 client.8730 populate_metadata read hostname 'smithi043' 2023-08-09T13:53:55.094 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:55.085+0000 7f050fdf9740 1 --2- 172.21.15.43:0/2760438372 >> [v2:172.21.15.43:6834/1794836673,v1:172.21.15.43:6835/1794836673] conn(0x5585ddd443c0 0x5585ddd447a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:53:55.094 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:55.085+0000 7f050fdf9740 4 client.8730 mds_command: new command op to 8580 tid=0[{"prefix": "get_command_descriptions"}] 2023-08-09T13:53:55.094 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:55.085+0000 7f050fdf9740 1 -- 172.21.15.43:0/2760438372 --> [v2:172.21.15.43:6834/1794836673,v1:172.21.15.43:6835/1794836673] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5585ddc29bc0 con 0x5585ddd443c0 2023-08-09T13:53:55.095 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:55.086+0000 7f0506ffd640 1 --2- 172.21.15.43:0/2760438372 >> [v2:172.21.15.43:6834/1794836673,v1:172.21.15.43:6835/1794836673] conn(0x5585ddd443c0 0x5585ddd447a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:53:55.095 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:55.086+0000 7f0506ffd640 1 --2- 172.21.15.43:0/2760438372 >> [v2:172.21.15.43:6834/1794836673,v1:172.21.15.43:6835/1794836673] conn(0x5585ddd443c0 0x5585ddd447a0 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d48790d14e4e7b8d server_cookie=53b37d860b1f3730 in_seq=0 out_seq=0 2023-08-09T13:53:55.095 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:55.086+0000 7f04e5ffb640 10 client.8730 ms_handle_connect on v2:172.21.15.43:6834/1794836673 2023-08-09T13:53:55.095 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:55.087+0000 7f04e5ffb640 1 -- 172.21.15.43:0/2760438372 <== mds.0 v2:172.21.15.43:6834/1794836673 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18017 (crc 0 0 0) 0x5585ddc29bc0 con 0x5585ddd443c0 2023-08-09T13:53:55.096 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:55.087+0000 7f04e5ffb640 10 client.8730 handle_command_reply: tid=0 2023-08-09T13:53:55.096 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:55.087+0000 7f050fdf9740 1 client.8730 shutdown 2023-08-09T13:53:55.096 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:55.088+0000 7f050fdf9740 1 -- 172.21.15.43:0/2760438372 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5585ddd13330 msgr2=0x5585ddd34c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:53:55.096 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:55.088+0000 7f050fdf9740 1 --2- 172.21.15.43:0/2760438372 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5585ddd13330 0x5585ddd34c90 secure :-1 s=READY pgs=1536 cs=0 l=1 rev1=1 crypto rx=0x7f050000a920 tx=0x7f0500001d80 comp rx=0 tx=0).stop 2023-08-09T13:53:55.096 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:55.088+0000 7f050fdf9740 1 -- 172.21.15.43:0/2760438372 shutdown_connections 2023-08-09T13:53:55.096 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:55.088+0000 7f050fdf9740 1 --2- 172.21.15.43:0/2760438372 >> [v2:172.21.15.43:6834/1794836673,v1:172.21.15.43:6835/1794836673] conn(0x5585ddd443c0 0x5585ddd447a0 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:53:55.097 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:55.088+0000 7f050fdf9740 1 --2- 172.21.15.43:0/2760438372 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5585ddd13330 0x5585ddd34c90 unknown :-1 s=CLOSED pgs=1536 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:53:55.097 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:55.088+0000 7f050fdf9740 1 --2- 172.21.15.43:0/2760438372 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x5585ddd19e30 0x5585ddd12df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:53:55.097 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:55.088+0000 7f050fdf9740 1 --2- 172.21.15.43:0/2760438372 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5585ddced9b0 0x5585ddd1eac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:53:55.097 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:55.088+0000 7f04e5ffb640 0 client.8730 ms_handle_reset on v2:172.21.15.43:6834/1794836673 2023-08-09T13:53:55.097 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:55.088+0000 7f050fdf9740 1 -- 172.21.15.43:0/2760438372 >> 172.21.15.43:0/2760438372 conn(0x5585ddce9db0 msgr2=0x5585ddcee370 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-09T13:53:55.097 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:55.088+0000 7f050fdf9740 1 -- 172.21.15.43:0/2760438372 shutdown_connections 2023-08-09T13:53:55.097 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:55.088+0000 7f050fdf9740 1 -- 172.21.15.43:0/2760438372 wait complete. 2023-08-09T13:53:55.097 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:55.088+0000 7f050fdf9740 20 client.8730 trim_cache size 0 max 16384 2023-08-09T13:53:55.107 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:55.105+0000 7f050fdf9740 1 Processor -- start 2023-08-09T13:53:55.108 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:55.105+0000 7f050fdf9740 1 -- start start 2023-08-09T13:53:55.109 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:55.106+0000 7f050fdf9740 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5585ddced9b0 0x5585ddd38150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:53:55.109 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:55.106+0000 7f050fdf9740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5585ddd13330 0x5585ddd375f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:53:55.109 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:55.106+0000 7f050fdf9740 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x5585ddd19e30 0x5585ddd03670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:53:55.109 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:55.106+0000 7f050fdf9740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x5585ddd1f230 con 0x5585ddd13330 2023-08-09T13:53:55.109 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:55.106+0000 7f050fdf9740 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x5585ddd1a5f0 con 0x5585ddced9b0 2023-08-09T13:53:55.109 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:55.106+0000 7f050fdf9740 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x5585ddd59290 con 0x5585ddd19e30 2023-08-09T13:53:55.109 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:55.106+0000 7f04fffff640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5585ddd13330 0x5585ddd375f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:53:55.110 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:55.106+0000 7f04fffff640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5585ddd13330 0x5585ddd375f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:34328/0 (socket says 172.21.15.43:34328) 2023-08-09T13:53:55.110 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:55.106+0000 7f04fffff640 1 -- 172.21.15.43:0/3022216283 learned_addr learned my addr 172.21.15.43:0/3022216283 (peer_addr_for_me v2:172.21.15.43:0/0) 2023-08-09T13:53:55.110 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:55.107+0000 7f0506ffd640 1 --2- 172.21.15.43:0/3022216283 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5585ddced9b0 0x5585ddd38150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:53:55.111 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:55.107+0000 7f05077fe640 1 --2- 172.21.15.43:0/3022216283 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x5585ddd19e30 0x5585ddd03670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:53:55.111 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:55.107+0000 7f04fffff640 1 -- 172.21.15.43:0/3022216283 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x5585ddd19e30 msgr2=0x5585ddd03670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-09T13:53:55.111 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:55.107+0000 7f04fffff640 1 --2- 172.21.15.43:0/3022216283 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x5585ddd19e30 0x5585ddd03670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:53:55.111 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:55.107+0000 7f04fffff640 1 -- 172.21.15.43:0/3022216283 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5585ddced9b0 msgr2=0x5585ddd38150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-09T13:53:55.111 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:55.107+0000 7f04fffff640 1 --2- 172.21.15.43:0/3022216283 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5585ddced9b0 0x5585ddd38150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:53:55.111 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:55.107+0000 7f04fffff640 1 -- 172.21.15.43:0/3022216283 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5585ddc86300 con 0x5585ddd13330 2023-08-09T13:53:55.111 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:55.107+0000 7f05077fe640 1 --2- 172.21.15.43:0/3022216283 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x5585ddd19e30 0x5585ddd03670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-08-09T13:53:55.112 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:55.108+0000 7f04fffff640 1 --2- 172.21.15.43:0/3022216283 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5585ddd13330 0x5585ddd375f0 secure :-1 s=READY pgs=1537 cs=0 l=1 rev1=1 crypto rx=0x7f04f0006a90 tx=0x7f04f0088040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=da0fbd1e165674d6 server_cookie=0 in_seq=0 out_seq=0 2023-08-09T13:53:55.112 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:55.108+0000 7f04fd7fa640 1 -- 172.21.15.43:0/3022216283 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f04f005f490 con 0x5585ddd13330 2023-08-09T13:53:55.112 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:55.108+0000 7f04fd7fa640 1 -- 172.21.15.43:0/3022216283 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f04f0053e90 con 0x5585ddd13330 2023-08-09T13:53:55.112 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:55.108+0000 7f04fd7fa640 1 -- 172.21.15.43:0/3022216283 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f04f0086b20 con 0x5585ddd13330 2023-08-09T13:53:55.113 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:55.109+0000 7f050fdf9740 1 -- 172.21.15.43:0/3022216283 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5585ddd13330 msgr2=0x5585ddd375f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:53:55.113 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:55.109+0000 7f050fdf9740 1 --2- 172.21.15.43:0/3022216283 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5585ddd13330 0x5585ddd375f0 secure :-1 s=READY pgs=1537 cs=0 l=1 rev1=1 crypto rx=0x7f04f0006a90 tx=0x7f04f0088040 comp rx=0 tx=0).stop 2023-08-09T13:53:55.113 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:55.109+0000 7f050fdf9740 1 -- 172.21.15.43:0/3022216283 shutdown_connections 2023-08-09T13:53:55.113 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:55.109+0000 7f050fdf9740 1 --2- 172.21.15.43:0/3022216283 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x5585ddd19e30 0x5585ddd03670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:53:55.114 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:55.109+0000 7f050fdf9740 1 --2- 172.21.15.43:0/3022216283 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5585ddd13330 0x5585ddd375f0 unknown :-1 s=CLOSED pgs=1537 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:53:55.114 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:55.109+0000 7f050fdf9740 1 --2- 172.21.15.43:0/3022216283 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5585ddced9b0 0x5585ddd38150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:53:55.114 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:55.109+0000 7f050fdf9740 1 -- 172.21.15.43:0/3022216283 >> 172.21.15.43:0/3022216283 conn(0x5585ddce9db0 msgr2=0x5585ddce5650 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-09T13:53:55.114 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:55.109+0000 7f050fdf9740 1 -- 172.21.15.43:0/3022216283 shutdown_connections 2023-08-09T13:53:55.114 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:55.109+0000 7f050fdf9740 1 -- 172.21.15.43:0/3022216283 wait complete. 2023-08-09T13:53:55.114 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:55.110+0000 7f050fdf9740 1 -- start start 2023-08-09T13:53:55.114 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:55.110+0000 7f050fdf9740 1 Processor -- start 2023-08-09T13:53:55.117 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:55.111+0000 7f050fdf9740 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x5585ddced9b0 0x5585ddd83720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:53:55.117 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:55.111+0000 7f050fdf9740 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5585ddd19e30 0x5585ddd03670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:53:55.117 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:55.111+0000 7f050fdf9740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5585ddd17070 0x5585ddd03bb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:53:55.118 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:55.111+0000 7f050fdf9740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x5585ddd02270 con 0x5585ddd17070 2023-08-09T13:53:55.118 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:55.111+0000 7f050fdf9740 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x5585ddd1ecd0 con 0x5585ddd19e30 2023-08-09T13:53:55.118 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:55.111+0000 7f050fdf9740 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x5585ddd467b0 con 0x5585ddced9b0 2023-08-09T13:53:55.118 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:55.111+0000 7f050fdf9740 10 client.0 _ll_register_callbacks cb 0x7f04f40036b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-08-09T13:53:55.118 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:55.111+0000 7f05077fe640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5585ddd17070 0x5585ddd03bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:53:55.118 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:55.111+0000 7f05077fe640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5585ddd17070 0x5585ddd03bb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:34334/0 (socket says 172.21.15.43:34334) 2023-08-09T13:53:55.118 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:55.111+0000 7f05077fe640 1 -- 172.21.15.43:0/2608007535 learned_addr learned my addr 172.21.15.43:0/2608007535 (peer_addr_for_me v2:172.21.15.43:0/0) 2023-08-09T13:53:55.119 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:55.112+0000 7f05077fe640 1 -- 172.21.15.43:0/2608007535 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x5585ddced9b0 msgr2=0x5585ddd83720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-09T13:53:55.119 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:55.112+0000 7f05077fe640 1 --2- 172.21.15.43:0/2608007535 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x5585ddced9b0 0x5585ddd83720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:53:55.119 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:55.112+0000 7f05077fe640 1 -- 172.21.15.43:0/2608007535 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5585ddd19e30 msgr2=0x5585ddd03670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-09T13:53:55.119 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:55.112+0000 7f05077fe640 1 --2- 172.21.15.43:0/2608007535 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5585ddd19e30 0x5585ddd03670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:53:55.119 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:55.112+0000 7f05077fe640 1 -- 172.21.15.43:0/2608007535 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f05080ad780 con 0x5585ddd17070 2023-08-09T13:53:55.119 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:55.112+0000 7f05077fe640 1 --2- 172.21.15.43:0/2608007535 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5585ddd17070 0x5585ddd03bb0 secure :-1 s=READY pgs=1538 cs=0 l=1 rev1=1 crypto rx=0x5585ddd1ae90 tx=0x7f0500030000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=35ff43b53b702da7 server_cookie=0 in_seq=0 out_seq=0 2023-08-09T13:53:55.119 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:55.112+0000 7f04e5ffb640 10 client.0 ms_handle_connect on v2:172.21.15.43:3300/0 2023-08-09T13:53:55.119 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:55.112+0000 7f04e5ffb640 1 -- 172.21.15.43:0/2608007535 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f050005a470 con 0x5585ddd17070 2023-08-09T13:53:55.120 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:55.112+0000 7f04e5ffb640 1 -- 172.21.15.43:0/2608007535 <== mon.0 v2:172.21.15.43:3300/0 2 ==== osd_map(129..129 src has 1..129) v4 ==== 6426+0+0 (secure 0 0 0) 0x7f050005a5d0 con 0x5585ddd17070 2023-08-09T13:53:55.120 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:55.112+0000 7f050fdf9740 1 -- 172.21.15.43:0/2608007535 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5585ddc7c800 con 0x5585ddd17070 2023-08-09T13:53:55.120 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:55.113+0000 7f04e5ffb640 1 -- 172.21.15.43:0/2608007535 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_get_version_reply(handle=1 version=61) v2 ==== 24+0+0 (secure 0 0 0) 0x7f050005a790 con 0x5585ddd17070 2023-08-09T13:53:55.120 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:55.113+0000 7f050fdf9740 10 client.8736 fetch_fsmap learned FSMap version 61 2023-08-09T13:53:55.120 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:55.113+0000 7f050fdf9740 1 -- 172.21.15.43:0/2608007535 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,fsmap=61,monmap=2+}) v3 -- 0x5585ddbf3a10 con 0x5585ddd17070 2023-08-09T13:53:55.120 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:55.113+0000 7f04e5ffb640 1 -- 172.21.15.43:0/2608007535 <== mon.0 v2:172.21.15.43:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x5585ddc7c800 con 0x5585ddd17070 2023-08-09T13:53:55.120 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:55.113+0000 7f04e5ffb640 1 -- 172.21.15.43:0/2608007535 <== mon.0 v2:172.21.15.43:3300/0 5 ==== fsmap(e 61) v1 ==== 3294+0+0 (secure 0 0 0) 0x7f050005a8f0 con 0x5585ddd17070 2023-08-09T13:53:55.121 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:55.113+0000 7f050fdf9740 10 client.8736 fetch_fsmap finished waiting for FSMap version 61 2023-08-09T13:53:55.121 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:55.113+0000 7f050fdf9740 10 client.8736 resolve_mds: resolved name 'a' to daemon mds.a 2023-08-09T13:53:55.121 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:55.113+0000 7f050fdf9740 20 client.8736 populate_metadata read hostname 'smithi043' 2023-08-09T13:53:55.121 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:55.113+0000 7f050fdf9740 1 --2- 172.21.15.43:0/2608007535 >> [v2:172.21.15.43:6834/1794836673,v1:172.21.15.43:6835/1794836673] conn(0x5585ddd500b0 0x5585ddd9da90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:53:55.121 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:55.113+0000 7f050fdf9740 4 client.8736 mds_command: new command op to 8580 tid=0[{"prefix": "status"}] 2023-08-09T13:53:55.121 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:55.113+0000 7f050fdf9740 1 -- 172.21.15.43:0/2608007535 --> [v2:172.21.15.43:6834/1794836673,v1:172.21.15.43:6835/1794836673] -- command(tid 0: {"prefix": "status"}) v1 -- 0x5585ddc4d8b0 con 0x5585ddd500b0 2023-08-09T13:53:55.121 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:55.113+0000 7f0506ffd640 1 --2- 172.21.15.43:0/2608007535 >> [v2:172.21.15.43:6834/1794836673,v1:172.21.15.43:6835/1794836673] conn(0x5585ddd500b0 0x5585ddd9da90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:53:55.121 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:55.114+0000 7f0506ffd640 1 --2- 172.21.15.43:0/2608007535 >> [v2:172.21.15.43:6834/1794836673,v1:172.21.15.43:6835/1794836673] conn(0x5585ddd500b0 0x5585ddd9da90 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d1c11fbdf514ce91 server_cookie=c6f3820c9df359ad in_seq=0 out_seq=0 2023-08-09T13:53:55.122 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:55.114+0000 7f04e5ffb640 10 client.8736 ms_handle_connect on v2:172.21.15.43:6834/1794836673 2023-08-09T13:53:55.122 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:55.114+0000 7f04e5ffb640 1 -- 172.21.15.43:0/2608007535 <== mds.0 v2:172.21.15.43:6834/1794836673 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+333 (crc 0 0 0) 0x5585ddc4d8b0 con 0x5585ddd500b0 2023-08-09T13:53:55.122 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:55.114+0000 7f04e5ffb640 10 client.8736 handle_command_reply: tid=0 2023-08-09T13:53:55.122 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:55.114+0000 7f050fdf9740 1 client.8736 shutdown 2023-08-09T13:53:55.122 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:55.114+0000 7f050fdf9740 1 -- 172.21.15.43:0/2608007535 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5585ddd17070 msgr2=0x5585ddd03bb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:53:55.122 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:55.114+0000 7f050fdf9740 1 --2- 172.21.15.43:0/2608007535 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5585ddd17070 0x5585ddd03bb0 secure :-1 s=READY pgs=1538 cs=0 l=1 rev1=1 crypto rx=0x5585ddd1ae90 tx=0x7f0500030000 comp rx=0 tx=0).stop 2023-08-09T13:53:55.122 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:55.114+0000 7f050fdf9740 1 -- 172.21.15.43:0/2608007535 shutdown_connections 2023-08-09T13:53:55.123 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:55.114+0000 7f050fdf9740 1 --2- 172.21.15.43:0/2608007535 >> [v2:172.21.15.43:6834/1794836673,v1:172.21.15.43:6835/1794836673] conn(0x5585ddd500b0 0x5585ddd9da90 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:53:55.123 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:55.114+0000 7f050fdf9740 1 --2- 172.21.15.43:0/2608007535 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5585ddd17070 0x5585ddd03bb0 unknown :-1 s=CLOSED pgs=1538 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:53:55.123 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:55.114+0000 7f050fdf9740 1 --2- 172.21.15.43:0/2608007535 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5585ddd19e30 0x5585ddd03670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:53:55.123 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:55.114+0000 7f050fdf9740 1 --2- 172.21.15.43:0/2608007535 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x5585ddced9b0 0x5585ddd83720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:53:55.123 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:55.114+0000 7f04e5ffb640 0 client.8736 ms_handle_reset on v2:172.21.15.43:6834/1794836673 2023-08-09T13:53:55.123 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:55.114+0000 7f050fdf9740 1 -- 172.21.15.43:0/2608007535 >> 172.21.15.43:0/2608007535 conn(0x5585ddce9db0 msgr2=0x5585ddd38150 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-09T13:53:55.123 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:55.115+0000 7f050fdf9740 1 -- 172.21.15.43:0/2608007535 shutdown_connections 2023-08-09T13:53:55.124 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:55.115+0000 7f050fdf9740 1 -- 172.21.15.43:0/2608007535 wait complete. 2023-08-09T13:53:55.124 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:55.115+0000 7f050fdf9740 20 client.8736 trim_cache size 0 max 16384 2023-08-09T13:53:55.124 INFO:teuthology.orchestra.run.smithi043.stdout:{ 2023-08-09T13:53:55.124 INFO:teuthology.orchestra.run.smithi043.stdout: "cluster_fsid": "daac87ee-1337-4250-a955-aea4e408a39b", 2023-08-09T13:53:55.124 INFO:teuthology.orchestra.run.smithi043.stdout: "whoami": 0, 2023-08-09T13:53:55.125 INFO:teuthology.orchestra.run.smithi043.stdout: "id": 8580, 2023-08-09T13:53:55.125 INFO:teuthology.orchestra.run.smithi043.stdout: "want_state": "up:active", 2023-08-09T13:53:55.125 INFO:teuthology.orchestra.run.smithi043.stdout: "state": "up:active", 2023-08-09T13:53:55.125 INFO:teuthology.orchestra.run.smithi043.stdout: "fs_name": "cephfs", 2023-08-09T13:53:55.125 INFO:teuthology.orchestra.run.smithi043.stdout: "rank_uptime": 6.8431909769999999, 2023-08-09T13:53:55.125 INFO:teuthology.orchestra.run.smithi043.stdout: "mdsmap_epoch": 61, 2023-08-09T13:53:55.126 INFO:teuthology.orchestra.run.smithi043.stdout: "osdmap_epoch": 129, 2023-08-09T13:53:55.126 INFO:teuthology.orchestra.run.smithi043.stdout: "osdmap_epoch_barrier": 129, 2023-08-09T13:53:55.126 INFO:teuthology.orchestra.run.smithi043.stdout: "uptime": 12.364746737000001 2023-08-09T13:53:55.126 INFO:teuthology.orchestra.run.smithi043.stdout:} 2023-08-09T13:53:55.127 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:55.115+0000 7f0507fff640 1 -- 172.21.15.43:0/3582121083 >> [v2:172.21.15.203:6832/2527059680,v1:172.21.15.203:6833/2527059680] conn(0x7f04e806a170 msgr2=0x7f04e806c630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:53:55.127 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:55.115+0000 7f0507fff640 1 --2- 172.21.15.43:0/3582121083 >> [v2:172.21.15.203:6832/2527059680,v1:172.21.15.203:6833/2527059680] conn(0x7f04e806a170 0x7f04e806c630 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f04f8001630 tx=0x7f04f8009040 comp rx=0 tx=0).stop 2023-08-09T13:53:55.127 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:55.115+0000 7f0507fff640 1 -- 172.21.15.43:0/3582121083 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f05080c3800 msgr2=0x7f05080c1990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:53:55.127 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:55.115+0000 7f0507fff640 1 --2- 172.21.15.43:0/3582121083 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f05080c3800 0x7f05080c1990 secure :-1 s=READY pgs=1534 cs=0 l=1 rev1=1 crypto rx=0x7f04f000e7e0 tx=0x7f04f000eca0 comp rx=0 tx=0).stop 2023-08-09T13:53:55.127 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:55.116+0000 7f0507fff640 1 -- 172.21.15.43:0/3582121083 shutdown_connections 2023-08-09T13:53:55.127 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:55.116+0000 7f0507fff640 1 --2- 172.21.15.43:0/3582121083 >> [v2:172.21.15.203:6832/2527059680,v1:172.21.15.203:6833/2527059680] conn(0x7f04e806a170 0x7f04e806c630 unknown :-1 s=CLOSED pgs=68 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:53:55.128 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:55.116+0000 7f0507fff640 1 --2- 172.21.15.43:0/3582121083 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f05080c1ed0 0x7f05080adca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:53:55.128 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:55.116+0000 7f0507fff640 1 --2- 172.21.15.43:0/3582121083 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f05080c3800 0x7f05080c1990 unknown :-1 s=CLOSED pgs=1534 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:53:55.128 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:55.116+0000 7f0507fff640 1 --2- 172.21.15.43:0/3582121083 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f04f4005380 0x7f05081f5860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:53:55.128 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:55.116+0000 7f0507fff640 1 -- 172.21.15.43:0/3582121083 >> 172.21.15.43:0/3582121083 conn(0x7f05080af950 msgr2=0x7f0508154b80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-09T13:53:55.128 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:55.116+0000 7f0507fff640 1 -- 172.21.15.43:0/3582121083 shutdown_connections 2023-08-09T13:53:55.128 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:55.116+0000 7f0507fff640 1 -- 172.21.15.43:0/3582121083 wait complete. 2023-08-09T13:53:55.135 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2023-08-09T13:53:55.363 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:55.361+0000 7fe878be1640 1 Processor -- start 2023-08-09T13:53:55.363 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:55.361+0000 7fe878be1640 1 -- start start 2023-08-09T13:53:55.364 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:55.362+0000 7fe878be1640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7fe8740b2790 0x7fe8740d2ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:53:55.364 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:55.362+0000 7fe878be1640 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7fe8740b3050 0x7fe8740b3f60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:53:55.365 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:55.362+0000 7fe878be1640 1 -- --> v1:172.21.15.43:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe8740ce090 con 0x7fe8740b2b70 2023-08-09T13:53:55.365 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:55.362+0000 7fe878be1640 1 -- --> v2:172.21.15.43:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe8740b3950 con 0x7fe8740b2790 2023-08-09T13:53:55.365 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:55.362+0000 7fe878be1640 1 -- --> v2:172.21.15.203:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe8740cd7f0 con 0x7fe8740b3050 2023-08-09T13:53:55.366 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:55.362+0000 7fe87259c640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7fe8740b2790 0x7fe8740d2ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:53:55.366 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:55.362+0000 7fe86bfff640 1 --1- >> v1:172.21.15.43:6789/0 conn(0x7fe8740b2b70 0x7fe8740b5330 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.43:6789/0 says I am v1:172.21.15.43:48528/0 (socket says 172.21.15.43:48528) 2023-08-09T13:53:55.366 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:55.362+0000 7fe86bfff640 1 -- 172.21.15.43:0/498445824 learned_addr learned my addr 172.21.15.43:0/498445824 (peer_addr_for_me v1:172.21.15.43:0/0) 2023-08-09T13:53:55.366 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:55.363+0000 7fe872d9d640 1 --2- 172.21.15.43:0/498445824 >> v2:172.21.15.203:3300/0 conn(0x7fe8740b3050 0x7fe8740b3f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:53:55.366 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:55.363+0000 7fe871d9b640 1 -- 172.21.15.43:0/498445824 <== mon.0 v1:172.21.15.43:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3982899208 0 0) 0x7fe8740ce090 con 0x7fe8740b2b70 2023-08-09T13:53:55.366 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:55.363+0000 7fe871d9b640 1 -- 172.21.15.43:0/498445824 --> v1:172.21.15.43:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fe860003520 con 0x7fe8740b2b70 2023-08-09T13:53:55.367 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:55.363+0000 7fe87259c640 1 -- 172.21.15.43:0/498445824 >> v2:172.21.15.203:3300/0 conn(0x7fe8740b3050 msgr2=0x7fe8740b3f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-09T13:53:55.367 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:55.363+0000 7fe87259c640 1 --2- 172.21.15.43:0/498445824 >> v2:172.21.15.203:3300/0 conn(0x7fe8740b3050 0x7fe8740b3f60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:53:55.367 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:55.363+0000 7fe87259c640 1 -- 172.21.15.43:0/498445824 >> v1:172.21.15.43:6789/0 conn(0x7fe8740b2b70 legacy=0x7fe8740b5330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:53:55.367 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:55.363+0000 7fe87259c640 1 -- 172.21.15.43:0/498445824 --> v2:172.21.15.43:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe8740cdad0 con 0x7fe8740b2790 2023-08-09T13:53:55.367 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:55.365+0000 7fe87259c640 1 --2- 172.21.15.43:0/498445824 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fe8740b2790 0x7fe8740d2ed0 secure :-1 s=READY pgs=1540 cs=0 l=1 rev1=1 crypto rx=0x7fe87408a270 tx=0x7fe86402b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b97287ebb6abd9a2 server_cookie=0 in_seq=0 out_seq=0 2023-08-09T13:53:55.368 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:55.365+0000 7fe871d9b640 1 -- 172.21.15.43:0/498445824 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe864031830 con 0x7fe8740b2790 2023-08-09T13:53:55.368 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:55.365+0000 7fe871d9b640 1 -- 172.21.15.43:0/498445824 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe864031990 con 0x7fe8740b2790 2023-08-09T13:53:55.368 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:55.365+0000 7fe871d9b640 1 -- 172.21.15.43:0/498445824 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe864031d00 con 0x7fe8740b2790 2023-08-09T13:53:55.369 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:55.366+0000 7fe878be1640 1 -- 172.21.15.43:0/498445824 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fe8740b2790 msgr2=0x7fe8740d2ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:53:55.369 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:55.366+0000 7fe878be1640 1 --2- 172.21.15.43:0/498445824 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fe8740b2790 0x7fe8740d2ed0 secure :-1 s=READY pgs=1540 cs=0 l=1 rev1=1 crypto rx=0x7fe87408a270 tx=0x7fe86402b8d0 comp rx=0 tx=0).stop 2023-08-09T13:53:55.369 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:55.366+0000 7fe878be1640 1 -- 172.21.15.43:0/498445824 shutdown_connections 2023-08-09T13:53:55.369 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:55.366+0000 7fe878be1640 1 --2- 172.21.15.43:0/498445824 >> v2:172.21.15.203:3300/0 conn(0x7fe8740b3050 0x7fe8740b3f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:53:55.369 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:55.366+0000 7fe878be1640 1 --2- 172.21.15.43:0/498445824 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fe8740b2790 0x7fe8740d2ed0 unknown :-1 s=CLOSED pgs=1540 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:53:55.369 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:55.366+0000 7fe878be1640 1 -- 172.21.15.43:0/498445824 >> 172.21.15.43:0/498445824 conn(0x7fe87409b7b0 msgr2=0x7fe8740d24e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-09T13:53:55.370 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:55.366+0000 7fe878be1640 1 -- 172.21.15.43:0/498445824 shutdown_connections 2023-08-09T13:53:55.370 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:55.366+0000 7fe878be1640 1 -- 172.21.15.43:0/498445824 wait complete. 2023-08-09T13:53:55.370 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:55.368+0000 7fe878be1640 1 Processor -- start 2023-08-09T13:53:55.370 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:55.368+0000 7fe878be1640 1 -- start start 2023-08-09T13:53:55.371 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:55.369+0000 7fe878be1640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fe8740b2790 0x7fe8741f2a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:53:55.371 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:55.369+0000 7fe878be1640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fe8740b2b70 0x7fe8740ca840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:53:55.371 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:55.369+0000 7fe878be1640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fe8740b3050 0x7fe87415b1c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:53:55.371 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:55.369+0000 7fe878be1640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe8740cb020 con 0x7fe8740b2790 2023-08-09T13:53:55.372 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:55.369+0000 7fe878be1640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe8741f63f0 con 0x7fe8740b3050 2023-08-09T13:53:55.372 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:55.369+0000 7fe878be1640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe8740cad80 con 0x7fe8740b2b70 2023-08-09T13:53:55.373 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:55.369+0000 7fe87259c640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fe8740b2790 0x7fe8741f2a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:53:55.373 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:55.369+0000 7fe87259c640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fe8740b2790 0x7fe8741f2a60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:34346/0 (socket says 172.21.15.43:34346) 2023-08-09T13:53:55.373 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:55.369+0000 7fe87259c640 1 -- 172.21.15.43:0/1850732158 learned_addr learned my addr 172.21.15.43:0/1850732158 (peer_addr_for_me v2:172.21.15.43:0/0) 2023-08-09T13:53:55.373 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:55.369+0000 7fe872d9d640 1 --2- 172.21.15.43:0/1850732158 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fe8740b3050 0x7fe87415b1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:53:55.374 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:55.369+0000 7fe86bfff640 1 --2- 172.21.15.43:0/1850732158 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fe8740b2b70 0x7fe8740ca840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:53:55.374 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:55.370+0000 7fe87259c640 1 -- 172.21.15.43:0/1850732158 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fe8740b2b70 msgr2=0x7fe8740ca840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:53:55.374 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:55.370+0000 7fe87259c640 1 --2- 172.21.15.43:0/1850732158 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fe8740b2b70 0x7fe8740ca840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:53:55.374 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:55.370+0000 7fe87259c640 1 -- 172.21.15.43:0/1850732158 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fe8740b3050 msgr2=0x7fe87415b1c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:53:55.374 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:55.370+0000 7fe87259c640 1 --2- 172.21.15.43:0/1850732158 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fe8740b3050 0x7fe87415b1c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:53:55.374 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:55.370+0000 7fe87259c640 1 -- 172.21.15.43:0/1850732158 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe864009400 con 0x7fe8740b2790 2023-08-09T13:53:55.374 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:55.370+0000 7fe87259c640 1 --2- 172.21.15.43:0/1850732158 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fe8740b2790 0x7fe8741f2a60 secure :-1 s=READY pgs=1541 cs=0 l=1 rev1=1 crypto rx=0x7fe864000c00 tx=0x7fe8640046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-09T13:53:55.374 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:55.370+0000 7fe86affd640 1 -- 172.21.15.43:0/1850732158 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe86403a830 con 0x7fe8740b2790 2023-08-09T13:53:55.375 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:55.370+0000 7fe86affd640 1 -- 172.21.15.43:0/1850732158 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe86403a990 con 0x7fe8740b2790 2023-08-09T13:53:55.375 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:55.370+0000 7fe86affd640 1 -- 172.21.15.43:0/1850732158 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe86403ad20 con 0x7fe8740b2790 2023-08-09T13:53:55.375 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:55.371+0000 7fe878be1640 1 -- 172.21.15.43:0/1850732158 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe8741f5630 con 0x7fe8740b2790 2023-08-09T13:53:55.375 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:55.371+0000 7fe878be1640 1 -- 172.21.15.43:0/1850732158 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe8741f5bf0 con 0x7fe8740b2790 2023-08-09T13:53:55.375 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:55.372+0000 7fe878be1640 1 -- 172.21.15.43:0/1850732158 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe8740739f0 con 0x7fe8740b2790 2023-08-09T13:53:55.376 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:55.373+0000 7fe86affd640 1 -- 172.21.15.43:0/1850732158 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 56) v1 ==== 88443+0+0 (secure 0 0 0) 0x7fe86404d440 con 0x7fe8740b2790 2023-08-09T13:53:55.376 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:55.374+0000 7fe86affd640 1 --2- 172.21.15.43:0/1850732158 >> [v2:172.21.15.203:6832/2527059680,v1:172.21.15.203:6833/2527059680] conn(0x7fe85406a170 0x7fe85406c630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:53:55.377 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:55.374+0000 7fe86affd640 1 -- 172.21.15.43:0/1850732158 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(129..129 src has 1..129) v4 ==== 6426+0+0 (secure 0 0 0) 0x7fe8640aaa30 con 0x7fe8740b2790 2023-08-09T13:53:55.377 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:55.375+0000 7fe86bfff640 1 --2- 172.21.15.43:0/1850732158 >> [v2:172.21.15.203:6832/2527059680,v1:172.21.15.203:6833/2527059680] conn(0x7fe85406a170 0x7fe85406c630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:53:55.378 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:55.376+0000 7fe86bfff640 1 --2- 172.21.15.43:0/1850732158 >> [v2:172.21.15.203:6832/2527059680,v1:172.21.15.203:6833/2527059680] conn(0x7fe85406a170 0x7fe85406c630 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7fe8740cc5b0 tx=0x7fe85c009040 comp rx=0 tx=0).ready entity=mgr.6377 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-09T13:53:55.381 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:55.379+0000 7fe86affd640 1 -- 172.21.15.43:0/1850732158 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178066 (secure 0 0 0) 0x7fe864075e20 con 0x7fe8740b2790 2023-08-09T13:53:55.616 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:55.613+0000 7fe878be1640 1 -- 172.21.15.43:0/1850732158 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7fe8740b7490 con 0x7fe8740b2790 2023-08-09T13:53:55.617 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:55.614+0000 7fe86affd640 1 -- 172.21.15.43:0/1850732158 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v61) v1 ==== 53+0+73 (secure 0 0 0) 0x7fe864079440 con 0x7fe8740b2790 2023-08-09T13:53:55.618 INFO:teuthology.orchestra.run.smithi043.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2023-08-09T13:53:55.619 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:55.617+0000 7fe878be1640 1 -- 172.21.15.43:0/1850732158 >> [v2:172.21.15.203:6832/2527059680,v1:172.21.15.203:6833/2527059680] conn(0x7fe85406a170 msgr2=0x7fe85406c630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:53:55.620 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:55.617+0000 7fe878be1640 1 --2- 172.21.15.43:0/1850732158 >> [v2:172.21.15.203:6832/2527059680,v1:172.21.15.203:6833/2527059680] conn(0x7fe85406a170 0x7fe85406c630 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7fe8740cc5b0 tx=0x7fe85c009040 comp rx=0 tx=0).stop 2023-08-09T13:53:55.620 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:55.617+0000 7fe878be1640 1 -- 172.21.15.43:0/1850732158 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fe8740b2790 msgr2=0x7fe8741f2a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:53:55.621 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:55.617+0000 7fe878be1640 1 --2- 172.21.15.43:0/1850732158 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fe8740b2790 0x7fe8741f2a60 secure :-1 s=READY pgs=1541 cs=0 l=1 rev1=1 crypto rx=0x7fe864000c00 tx=0x7fe8640046c0 comp rx=0 tx=0).stop 2023-08-09T13:53:55.621 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:55.617+0000 7fe878be1640 1 -- 172.21.15.43:0/1850732158 shutdown_connections 2023-08-09T13:53:55.621 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:55.617+0000 7fe878be1640 1 --2- 172.21.15.43:0/1850732158 >> [v2:172.21.15.203:6832/2527059680,v1:172.21.15.203:6833/2527059680] conn(0x7fe85406a170 0x7fe85406c630 unknown :-1 s=CLOSED pgs=69 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:53:55.621 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:55.617+0000 7fe878be1640 1 --2- 172.21.15.43:0/1850732158 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fe8740b3050 0x7fe87415b1c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:53:55.622 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:55.617+0000 7fe878be1640 1 --2- 172.21.15.43:0/1850732158 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fe8740b2b70 0x7fe8740ca840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:53:55.622 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:55.617+0000 7fe878be1640 1 --2- 172.21.15.43:0/1850732158 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fe8740b2790 0x7fe8741f2a60 unknown :-1 s=CLOSED pgs=1541 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:53:55.622 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:55.617+0000 7fe878be1640 1 -- 172.21.15.43:0/1850732158 >> 172.21.15.43:0/1850732158 conn(0x7fe87409b7b0 msgr2=0x7fe8740b79b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-09T13:53:55.623 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:55.617+0000 7fe878be1640 1 -- 172.21.15.43:0/1850732158 shutdown_connections 2023-08-09T13:53:55.623 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:55.617+0000 7fe878be1640 1 -- 172.21.15.43:0/1850732158 wait complete. 2023-08-09T13:53:55.637 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2023-08-09T13:53:55.637 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2023-08-09T13:53:55.637 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi043.front.sepia.ceph.com 2023-08-09T13:53:55.638 INFO:tasks.cephfs.mount:self.client.name = client.0 2023-08-09T13:53:55.638 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2023-08-09T13:53:55.638 INFO:tasks.cephfs.mount:self.cephfs_name = None 2023-08-09T13:53:55.638 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2023-08-09T13:53:55.639 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2023-08-09T13:53:55.639 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2023-08-09T13:53:55.639 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-09T13:53:55.640 DEBUG:teuthology.orchestra.run.smithi043:> ip addr 2023-08-09T13:53:55.696 INFO:teuthology.orchestra.run.smithi043.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2023-08-09T13:53:55.697 INFO:teuthology.orchestra.run.smithi043.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2023-08-09T13:53:55.697 INFO:teuthology.orchestra.run.smithi043.stdout: inet 127.0.0.1/8 scope host lo 2023-08-09T13:53:55.697 INFO:teuthology.orchestra.run.smithi043.stdout: valid_lft forever preferred_lft forever 2023-08-09T13:53:55.697 INFO:teuthology.orchestra.run.smithi043.stdout: inet6 ::1/128 scope host 2023-08-09T13:53:55.697 INFO:teuthology.orchestra.run.smithi043.stdout: valid_lft forever preferred_lft forever 2023-08-09T13:53:55.697 INFO:teuthology.orchestra.run.smithi043.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-08-09T13:53:55.697 INFO:teuthology.orchestra.run.smithi043.stdout: link/ether 0c:c4:7a:6b:fd:9c brd ff:ff:ff:ff:ff:ff 2023-08-09T13:53:55.698 INFO:teuthology.orchestra.run.smithi043.stdout: altname enp7s0f0 2023-08-09T13:53:55.698 INFO:teuthology.orchestra.run.smithi043.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-08-09T13:53:55.698 INFO:teuthology.orchestra.run.smithi043.stdout: link/ether 0c:c4:7a:6b:fd:9d brd ff:ff:ff:ff:ff:ff 2023-08-09T13:53:55.698 INFO:teuthology.orchestra.run.smithi043.stdout: altname enp7s0f1 2023-08-09T13:53:55.698 INFO:teuthology.orchestra.run.smithi043.stdout:4: ens1f0: mtu 1500 qdisc mq state UP group default qlen 1000 2023-08-09T13:53:55.698 INFO:teuthology.orchestra.run.smithi043.stdout: link/ether 0c:c4:7a:bd:13:58 brd ff:ff:ff:ff:ff:ff 2023-08-09T13:53:55.698 INFO:teuthology.orchestra.run.smithi043.stdout: altname enp1s0f0 2023-08-09T13:53:55.698 INFO:teuthology.orchestra.run.smithi043.stdout: inet 172.21.15.43/20 brd 172.21.15.255 scope global dynamic noprefixroute ens1f0 2023-08-09T13:53:55.699 INFO:teuthology.orchestra.run.smithi043.stdout: valid_lft 41068sec preferred_lft 41068sec 2023-08-09T13:53:55.699 INFO:teuthology.orchestra.run.smithi043.stdout: inet6 fe80::ec4:7aff:febd:1358/64 scope link 2023-08-09T13:53:55.699 INFO:teuthology.orchestra.run.smithi043.stdout: valid_lft forever preferred_lft forever 2023-08-09T13:53:55.699 INFO:teuthology.orchestra.run.smithi043.stdout:5: ens1f1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-08-09T13:53:55.699 INFO:teuthology.orchestra.run.smithi043.stdout: link/ether 0c:c4:7a:bd:13:59 brd ff:ff:ff:ff:ff:ff 2023-08-09T13:53:55.699 INFO:teuthology.orchestra.run.smithi043.stdout: altname enp1s0f1 2023-08-09T13:53:55.699 INFO:teuthology.orchestra.run.smithi043.stdout:8: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2023-08-09T13:53:55.700 INFO:teuthology.orchestra.run.smithi043.stdout: link/ether 56:43:50:19:f4:0c brd ff:ff:ff:ff:ff:ff 2023-08-09T13:53:55.700 INFO:teuthology.orchestra.run.smithi043.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2023-08-09T13:53:55.700 INFO:teuthology.orchestra.run.smithi043.stdout: valid_lft forever preferred_lft forever 2023-08-09T13:53:55.700 INFO:teuthology.orchestra.run.smithi043.stdout: inet6 fe80::e494:65ff:fe92:a182/64 scope link 2023-08-09T13:53:55.700 INFO:teuthology.orchestra.run.smithi043.stdout: valid_lft forever preferred_lft forever 2023-08-09T13:53:55.700 INFO:teuthology.orchestra.run.smithi043.stdout:9: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2023-08-09T13:53:55.700 INFO:teuthology.orchestra.run.smithi043.stdout: link/ether 7e:93:26:fa:07:8f brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.0 2023-08-09T13:53:55.700 INFO:teuthology.orchestra.run.smithi043.stdout: inet6 fe80::7c93:26ff:fefa:78f/64 scope link 2023-08-09T13:53:55.701 INFO:teuthology.orchestra.run.smithi043.stdout: valid_lft forever preferred_lft forever 2023-08-09T13:53:55.701 INFO:teuthology.orchestra.run.smithi043.stdout:10: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2023-08-09T13:53:55.701 INFO:teuthology.orchestra.run.smithi043.stdout: link/ether 56:43:50:19:f4:0c brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.2 2023-08-09T13:53:55.701 INFO:teuthology.orchestra.run.smithi043.stdout: inet6 fe80::5443:50ff:fe19:f40c/64 scope link 2023-08-09T13:53:55.701 INFO:teuthology.orchestra.run.smithi043.stdout: valid_lft forever preferred_lft forever 2023-08-09T13:53:55.702 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-09T13:53:55.702 DEBUG:teuthology.orchestra.run.smithi043:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2023-08-09T13:53:55.802 INFO:teuthology.orchestra.run.smithi043.stdout:1 2023-08-09T13:53:55.806 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-09T13:53:55.806 DEBUG:teuthology.orchestra.run.smithi043:> route 2023-08-09T13:53:55.862 INFO:teuthology.orchestra.run.smithi043.stdout:Kernel IP routing table 2023-08-09T13:53:55.863 INFO:teuthology.orchestra.run.smithi043.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2023-08-09T13:53:55.863 INFO:teuthology.orchestra.run.smithi043.stdout:default _gateway 0.0.0.0 UG 100 0 0 ens1f0 2023-08-09T13:53:55.863 INFO:teuthology.orchestra.run.smithi043.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 ens1f0 2023-08-09T13:53:55.863 INFO:teuthology.orchestra.run.smithi043.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2023-08-09T13:53:55.864 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-09T13:53:55.865 DEBUG:teuthology.orchestra.run.smithi043:> (cd / && exec bash -c ' 2023-08-09T13:53:55.865 DEBUG:teuthology.orchestra.run.smithi043:> set -e 2023-08-09T13:53:55.865 DEBUG:teuthology.orchestra.run.smithi043:> sudo iptables -A FORWARD -o ens1f0 -i ceph-brx -j ACCEPT 2023-08-09T13:53:55.865 DEBUG:teuthology.orchestra.run.smithi043:> sudo iptables -A FORWARD -i ens1f0 -o ceph-brx -j ACCEPT 2023-08-09T13:53:55.866 DEBUG:teuthology.orchestra.run.smithi043:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o ens1f0 -j MASQUERADE 2023-08-09T13:53:55.866 DEBUG:teuthology.orchestra.run.smithi043:> ') 2023-08-09T13:53:56.068 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-09T13:53:56.069 DEBUG:teuthology.orchestra.run.smithi043:> ip netns list 2023-08-09T13:53:56.126 INFO:teuthology.orchestra.run.smithi043.stdout:ceph-ns--home-ubuntu-cephtest-mnt.2 (id: 1) 2023-08-09T13:53:56.126 INFO:teuthology.orchestra.run.smithi043.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2023-08-09T13:53:56.127 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2023-08-09T13:53:56.127 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-09T13:53:56.127 DEBUG:teuthology.orchestra.run.smithi043:> sudo ip link set brx.0 up 2023-08-09T13:53:56.233 INFO:tasks.cephfs.fuse_mount:Client client.0 config is {} 2023-08-09T13:53:56.234 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-08-09T13:53:56.234 DEBUG:teuthology.orchestra.run.smithi043:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2023-08-09T13:53:56.296 INFO:teuthology.orchestra.run.smithi043.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2023-08-09T13:53:56.297 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-08-09T13:53:56.297 DEBUG:teuthology.orchestra.run.smithi043:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2023-08-09T13:53:56.363 DEBUG:teuthology.orchestra.run.smithi043:> sudo modprobe fuse 2023-08-09T13:53:56.469 INFO:teuthology.orchestra.run:Running command with timeout 30 2023-08-09T13:53:56.470 DEBUG:teuthology.orchestra.run.smithi043:> sudo mount -t fusectl /sys/fs/fuse/connections /sys/fs/fuse/connections 2023-08-09T13:53:56.569 INFO:teuthology.orchestra.run.smithi043.stderr:mount: /sys/fs/fuse/connections: /sys/fs/fuse/connections already mounted or mount point busy. 2023-08-09T13:53:56.573 DEBUG:teuthology.orchestra.run:got remote process result: 32 2023-08-09T13:53:56.573 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-09T13:53:56.574 DEBUG:teuthology.orchestra.run.smithi043:> ls /sys/fs/fuse/connections 2023-08-09T13:53:56.634 INFO:tasks.cephfs.fuse_mount:Pre-mount connections: [] 2023-08-09T13:53:56.634 DEBUG:teuthology.orchestra.run.smithi043:> (cd /home/ubuntu/cephtest && exec sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-fuse -f --admin-socket '/var/run/ceph/$cluster-$name.$pid.asok' /home/ubuntu/cephtest/mnt.0 --id 0) 2023-08-09T13:53:56.641 DEBUG:teuthology.orchestra.run.smithi043:> sudo modprobe fuse 2023-08-09T13:53:56.733 INFO:teuthology.orchestra.run:Running command with timeout 30 2023-08-09T13:53:56.733 DEBUG:teuthology.orchestra.run.smithi043:> sudo mount -t fusectl /sys/fs/fuse/connections /sys/fs/fuse/connections 2023-08-09T13:53:56.812 INFO:teuthology.orchestra.run.smithi043.stderr:mount: /sys/fs/fuse/connections: /sys/fs/fuse/connections already mounted or mount point busy. 2023-08-09T13:53:56.815 DEBUG:teuthology.orchestra.run:got remote process result: 32 2023-08-09T13:53:56.816 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-09T13:53:56.817 DEBUG:teuthology.orchestra.run.smithi043:> ls /sys/fs/fuse/connections 2023-08-09T13:53:56.845 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi043.stderr:2023-08-09T13:53:56.843+0000 7f6f898b9480 -1 init, newargv = 0x55b8037b7bd0 newargc=15 2023-08-09T13:53:56.846 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi043.stderr:ceph-fuse[128482]: starting ceph client 2023-08-09T13:53:56.853 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi043.stderr:ceph-fuse[128482]: starting fuse 2023-08-09T13:53:56.862 INFO:teuthology.orchestra.run.smithi043.stdout:40 2023-08-09T13:53:56.863 INFO:tasks.cephfs.fuse_mount:Post-mount connections: [40] 2023-08-09T13:53:56.863 DEBUG:teuthology.orchestra.run.smithi043:> sudo stdin-killer -- python3 -c ' 2023-08-09T13:53:56.864 DEBUG:teuthology.orchestra.run.smithi043:> import glob 2023-08-09T13:53:56.864 DEBUG:teuthology.orchestra.run.smithi043:> import re 2023-08-09T13:53:56.864 DEBUG:teuthology.orchestra.run.smithi043:> import os 2023-08-09T13:53:56.864 DEBUG:teuthology.orchestra.run.smithi043:> import subprocess 2023-08-09T13:53:56.865 DEBUG:teuthology.orchestra.run.smithi043:> 2023-08-09T13:53:56.865 DEBUG:teuthology.orchestra.run.smithi043:> def _find_admin_socket(client_name): 2023-08-09T13:53:56.865 DEBUG:teuthology.orchestra.run.smithi043:> asok_path = "/var/run/ceph/ceph-client.0.*.asok" 2023-08-09T13:53:56.866 DEBUG:teuthology.orchestra.run.smithi043:> files = glob.glob(asok_path) 2023-08-09T13:53:56.866 DEBUG:teuthology.orchestra.run.smithi043:> mountpoint = "/home/ubuntu/cephtest/mnt.0" 2023-08-09T13:53:56.867 DEBUG:teuthology.orchestra.run.smithi043:> 2023-08-09T13:53:56.867 DEBUG:teuthology.orchestra.run.smithi043:> # Given a non-glob path, it better be there 2023-08-09T13:53:56.867 DEBUG:teuthology.orchestra.run.smithi043:> if "*" not in asok_path: 2023-08-09T13:53:56.867 DEBUG:teuthology.orchestra.run.smithi043:> assert(len(files) == 1) 2023-08-09T13:53:56.868 DEBUG:teuthology.orchestra.run.smithi043:> return files[0] 2023-08-09T13:53:56.868 DEBUG:teuthology.orchestra.run.smithi043:> 2023-08-09T13:53:56.869 DEBUG:teuthology.orchestra.run.smithi043:> for f in files: 2023-08-09T13:53:56.869 DEBUG:teuthology.orchestra.run.smithi043:> pid = re.match(".*\.(\d+)\.asok$", f).group(1) 2023-08-09T13:53:56.869 DEBUG:teuthology.orchestra.run.smithi043:> if os.path.exists("/proc/{0}".format(pid)): 2023-08-09T13:53:56.870 DEBUG:teuthology.orchestra.run.smithi043:> with open("/proc/{0}/cmdline".format(pid), '"'"'r'"'"') as proc_f: 2023-08-09T13:53:56.870 DEBUG:teuthology.orchestra.run.smithi043:> contents = proc_f.read() 2023-08-09T13:53:56.870 DEBUG:teuthology.orchestra.run.smithi043:> if mountpoint in contents: 2023-08-09T13:53:56.871 DEBUG:teuthology.orchestra.run.smithi043:> return f 2023-08-09T13:53:56.871 DEBUG:teuthology.orchestra.run.smithi043:> raise RuntimeError("Client socket {0} not found".format(client_name)) 2023-08-09T13:53:56.871 DEBUG:teuthology.orchestra.run.smithi043:> 2023-08-09T13:53:56.872 DEBUG:teuthology.orchestra.run.smithi043:> print(_find_admin_socket("client.0")) 2023-08-09T13:53:56.872 DEBUG:teuthology.orchestra.run.smithi043:> ' 2023-08-09T13:53:57.085 INFO:teuthology.orchestra.run.smithi043.stdout:/var/run/ceph/ceph-client.0.128482.asok 2023-08-09T13:53:57.089 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:57 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2023-08-09T13:53:57.103 INFO:tasks.cephfs.fuse_mount:Found client admin socket at /var/run/ceph/ceph-client.0.128482.asok 2023-08-09T13:53:57.103 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-09T13:53:57.103 DEBUG:teuthology.orchestra.run.smithi043:> sudo ceph --admin-daemon /var/run/ceph/ceph-client.0.128482.asok status 2023-08-09T13:53:57.366 INFO:teuthology.orchestra.run.smithi043.stdout:{ 2023-08-09T13:53:57.367 INFO:teuthology.orchestra.run.smithi043.stdout: "metadata": { 2023-08-09T13:53:57.367 INFO:teuthology.orchestra.run.smithi043.stdout: "ceph_sha1": "668a86f232c9454b2ce5928d1432a29a72964d85", 2023-08-09T13:53:57.367 INFO:teuthology.orchestra.run.smithi043.stdout: "ceph_version": "ceph version 18.0.0-5389-g668a86f2 (668a86f232c9454b2ce5928d1432a29a72964d85) reef (dev)", 2023-08-09T13:53:57.367 INFO:teuthology.orchestra.run.smithi043.stdout: "entity_id": "0", 2023-08-09T13:53:57.367 INFO:teuthology.orchestra.run.smithi043.stdout: "hostname": "smithi043", 2023-08-09T13:53:57.368 INFO:teuthology.orchestra.run.smithi043.stdout: "mount_point": "/home/ubuntu/cephtest/mnt.0", 2023-08-09T13:53:57.368 INFO:teuthology.orchestra.run.smithi043.stdout: "pid": "128482", 2023-08-09T13:53:57.368 INFO:teuthology.orchestra.run.smithi043.stdout: "root": "/" 2023-08-09T13:53:57.368 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2023-08-09T13:53:57.368 INFO:teuthology.orchestra.run.smithi043.stdout: "dentry_count": 0, 2023-08-09T13:53:57.368 INFO:teuthology.orchestra.run.smithi043.stdout: "dentry_pinned_count": 0, 2023-08-09T13:53:57.368 INFO:teuthology.orchestra.run.smithi043.stdout: "id": 8751, 2023-08-09T13:53:57.369 INFO:teuthology.orchestra.run.smithi043.stdout: "inst": { 2023-08-09T13:53:57.369 INFO:teuthology.orchestra.run.smithi043.stdout: "name": { 2023-08-09T13:53:57.369 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "client", 2023-08-09T13:53:57.369 INFO:teuthology.orchestra.run.smithi043.stdout: "num": 8751 2023-08-09T13:53:57.369 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2023-08-09T13:53:57.369 INFO:teuthology.orchestra.run.smithi043.stdout: "addr": { 2023-08-09T13:53:57.369 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "v1", 2023-08-09T13:53:57.369 INFO:teuthology.orchestra.run.smithi043.stdout: "addr": "192.168.0.1:0", 2023-08-09T13:53:57.370 INFO:teuthology.orchestra.run.smithi043.stdout: "nonce": 3306202650 2023-08-09T13:53:57.370 INFO:teuthology.orchestra.run.smithi043.stdout: } 2023-08-09T13:53:57.370 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2023-08-09T13:53:57.370 INFO:teuthology.orchestra.run.smithi043.stdout: "addr": { 2023-08-09T13:53:57.370 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "v1", 2023-08-09T13:53:57.370 INFO:teuthology.orchestra.run.smithi043.stdout: "addr": "192.168.0.1:0", 2023-08-09T13:53:57.370 INFO:teuthology.orchestra.run.smithi043.stdout: "nonce": 3306202650 2023-08-09T13:53:57.370 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2023-08-09T13:53:57.371 INFO:teuthology.orchestra.run.smithi043.stdout: "inst_str": "client.8751 192.168.0.1:0/3306202650", 2023-08-09T13:53:57.371 INFO:teuthology.orchestra.run.smithi043.stdout: "addr_str": "192.168.0.1:0/3306202650", 2023-08-09T13:53:57.371 INFO:teuthology.orchestra.run.smithi043.stdout: "inode_count": 1, 2023-08-09T13:53:57.371 INFO:teuthology.orchestra.run.smithi043.stdout: "mds_epoch": 61, 2023-08-09T13:53:57.371 INFO:teuthology.orchestra.run.smithi043.stdout: "osd_epoch": 129, 2023-08-09T13:53:57.371 INFO:teuthology.orchestra.run.smithi043.stdout: "osd_epoch_barrier": 0, 2023-08-09T13:53:57.371 INFO:teuthology.orchestra.run.smithi043.stdout: "blocklisted": false, 2023-08-09T13:53:57.371 INFO:teuthology.orchestra.run.smithi043.stdout: "fs_name": "cephfs" 2023-08-09T13:53:57.372 INFO:teuthology.orchestra.run.smithi043.stdout:} 2023-08-09T13:53:57.382 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-09T13:53:57.383 DEBUG:teuthology.orchestra.run.smithi043:> stat --file-system '--printf=%T 2023-08-09T13:53:57.383 DEBUG:teuthology.orchestra.run.smithi043:> ' -- /home/ubuntu/cephtest/mnt.0 2023-08-09T13:53:57.439 INFO:teuthology.orchestra.run.smithi043.stdout:fuseblk 2023-08-09T13:53:57.439 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.0 2023-08-09T13:53:57.440 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-09T13:53:57.440 DEBUG:teuthology.orchestra.run.smithi043:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2023-08-09T13:53:57.544 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs volume ls 2023-08-09T13:53:57.778 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:57.776+0000 7fc6ae19b640 1 Processor -- start 2023-08-09T13:53:57.779 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:57.776+0000 7fc6ae19b640 1 -- start start 2023-08-09T13:53:57.780 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:57.777+0000 7fc6ae19b640 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7fc6a80b1050 0x7fc6a80b4760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:53:57.780 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:57.777+0000 7fc6ae19b640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fc6a80cd6b0 0x7fc6a80d0e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:53:57.780 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:57.778+0000 7fc6ae19b640 1 -- --> v1:172.21.15.43:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc6a80b7bf0 con 0x7fc6a815f7b0 2023-08-09T13:53:57.780 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:57.778+0000 7fc6ae19b640 1 -- --> v2:172.21.15.203:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc6a80b7ed0 con 0x7fc6a80b1050 2023-08-09T13:53:57.781 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:57.778+0000 7fc6ae19b640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc6a80d0370 con 0x7fc6a80cd6b0 2023-08-09T13:53:57.781 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:57.778+0000 7fc6ad199640 1 --1- >> v1:172.21.15.43:6789/0 conn(0x7fc6a815f7b0 0x7fc6a80ccfa0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.43:6789/0 says I am v1:172.21.15.43:48540/0 (socket says 172.21.15.43:48540) 2023-08-09T13:53:57.781 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:57.778+0000 7fc6ad199640 1 -- 172.21.15.43:0/614646892 learned_addr learned my addr 172.21.15.43:0/614646892 (peer_addr_for_me v1:172.21.15.43:0/0) 2023-08-09T13:53:57.782 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:57.778+0000 7fc6ad99a640 1 --2- 172.21.15.43:0/614646892 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fc6a80cd6b0 0x7fc6a80d0e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:53:57.783 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:57.778+0000 7fc6ac998640 1 --2- 172.21.15.43:0/614646892 >> v2:172.21.15.203:3300/0 conn(0x7fc6a80b1050 0x7fc6a80b4760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:53:57.783 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:57.778+0000 7fc69ffff640 1 -- 172.21.15.43:0/614646892 <== mon.0 v1:172.21.15.43:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3169765953 0 0) 0x7fc6a80b7bf0 con 0x7fc6a815f7b0 2023-08-09T13:53:57.783 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:57.778+0000 7fc69ffff640 1 -- 172.21.15.43:0/614646892 --> v1:172.21.15.43:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fc694003520 con 0x7fc6a815f7b0 2023-08-09T13:53:57.783 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:57.779+0000 7fc69ffff640 1 -- 172.21.15.43:0/614646892 <== mon.0 v1:172.21.15.43:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2638778673 0 0) 0x7fc694003520 con 0x7fc6a815f7b0 2023-08-09T13:53:57.783 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:57.779+0000 7fc69ffff640 1 -- 172.21.15.43:0/614646892 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fc6a80cd6b0 msgr2=0x7fc6a80d0e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-09T13:53:57.783 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:57.779+0000 7fc69ffff640 1 --2- 172.21.15.43:0/614646892 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fc6a80cd6b0 0x7fc6a80d0e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:53:57.783 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:57.779+0000 7fc69ffff640 1 -- 172.21.15.43:0/614646892 >> v2:172.21.15.203:3300/0 conn(0x7fc6a80b1050 msgr2=0x7fc6a80b4760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-09T13:53:57.783 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:57.779+0000 7fc69ffff640 1 --2- 172.21.15.43:0/614646892 >> v2:172.21.15.203:3300/0 conn(0x7fc6a80b1050 0x7fc6a80b4760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:53:57.784 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:57.779+0000 7fc69ffff640 1 -- 172.21.15.43:0/614646892 --> v1:172.21.15.43:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc6a80d0650 con 0x7fc6a815f7b0 2023-08-09T13:53:57.784 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:57.779+0000 7fc69ffff640 1 -- 172.21.15.43:0/614646892 <== mon.0 v1:172.21.15.43:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1285150740 0 0) 0x7fc6a4001670 con 0x7fc6a815f7b0 2023-08-09T13:53:57.784 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:57.779+0000 7fc69ffff640 1 -- 172.21.15.43:0/614646892 >> v1:172.21.15.43:6789/0 conn(0x7fc6a815f7b0 legacy=0x7fc6a80ccfa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:53:57.784 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:57.779+0000 7fc69ffff640 1 --2- 172.21.15.43:0/614646892 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fc694003970 0x7fc694003d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:53:57.784 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:57.779+0000 7fc69ffff640 1 --2- 172.21.15.43:0/614646892 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc6940047c0 0x7fc694004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:53:57.784 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:57.779+0000 7fc69ffff640 1 --2- 172.21.15.43:0/614646892 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fc6a80cd6b0 0x7fc694005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:53:57.784 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:57.779+0000 7fc69ffff640 1 -- 172.21.15.43:0/614646892 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc6a80b7ed0 con 0x7fc6940047c0 2023-08-09T13:53:57.785 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:57.779+0000 7fc69ffff640 1 -- 172.21.15.43:0/614646892 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc6a80d0370 con 0x7fc694003970 2023-08-09T13:53:57.785 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:57.779+0000 7fc69ffff640 1 -- 172.21.15.43:0/614646892 --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc69400b980 con 0x7fc6a80cd6b0 2023-08-09T13:53:57.785 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:57.779+0000 7fc6ac998640 1 --2- 172.21.15.43:0/614646892 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc6940047c0 0x7fc694004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:53:57.785 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:57.780+0000 7fc6ad99a640 1 --2- 172.21.15.43:0/614646892 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fc6a80cd6b0 0x7fc694005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:53:57.785 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:57.780+0000 7fc6ac998640 1 -- 172.21.15.43:0/614646892 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fc6a80cd6b0 msgr2=0x7fc694005180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-09T13:53:57.785 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:57.780+0000 7fc6ac998640 1 --2- 172.21.15.43:0/614646892 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fc6a80cd6b0 0x7fc694005180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:53:57.785 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:57.780+0000 7fc6ad199640 1 --2- 172.21.15.43:0/614646892 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fc694003970 0x7fc694003d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:53:57.785 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:57.780+0000 7fc6ac998640 1 -- 172.21.15.43:0/614646892 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fc694003970 msgr2=0x7fc694003d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-09T13:53:57.786 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:57.780+0000 7fc6ac998640 1 --2- 172.21.15.43:0/614646892 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fc694003970 0x7fc694003d90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:53:57.786 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:57.780+0000 7fc6ac998640 1 -- 172.21.15.43:0/614646892 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc694003520 con 0x7fc6940047c0 2023-08-09T13:53:57.787 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:57.781+0000 7fc6ac998640 1 --2- 172.21.15.43:0/614646892 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc6940047c0 0x7fc694004c40 secure :-1 s=READY pgs=1546 cs=0 l=1 rev1=1 crypto rx=0x7fc698009a90 tx=0x7fc6940057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8d5bdd26debb3ed1 server_cookie=0 in_seq=0 out_seq=0 2023-08-09T13:53:57.787 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:57.781+0000 7fc69ffff640 1 -- 172.21.15.43:0/614646892 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc69802c6b0 con 0x7fc6940047c0 2023-08-09T13:53:57.787 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:57.781+0000 7fc69ffff640 1 -- 172.21.15.43:0/614646892 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc69802c810 con 0x7fc6940047c0 2023-08-09T13:53:57.787 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:57.781+0000 7fc69ffff640 1 -- 172.21.15.43:0/614646892 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc69802cbc0 con 0x7fc6940047c0 2023-08-09T13:53:57.787 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:57.782+0000 7fc6ae19b640 1 -- 172.21.15.43:0/614646892 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc6940047c0 msgr2=0x7fc694004c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:53:57.788 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:57.782+0000 7fc6ae19b640 1 --2- 172.21.15.43:0/614646892 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc6940047c0 0x7fc694004c40 secure :-1 s=READY pgs=1546 cs=0 l=1 rev1=1 crypto rx=0x7fc698009a90 tx=0x7fc6940057f0 comp rx=0 tx=0).stop 2023-08-09T13:53:57.788 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:57.782+0000 7fc6ad99a640 1 -- 172.21.15.43:0/614646892 reap_dead start 2023-08-09T13:53:57.788 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:57.782+0000 7fc6ae19b640 1 -- 172.21.15.43:0/614646892 shutdown_connections 2023-08-09T13:53:57.788 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:57.782+0000 7fc6ae19b640 1 -- 172.21.15.43:0/614646892 >> 172.21.15.43:0/614646892 conn(0x7fc6a806a2b0 msgr2=0x7fc6a806a690 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-09T13:53:57.788 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:57.782+0000 7fc6ae19b640 1 -- 172.21.15.43:0/614646892 shutdown_connections 2023-08-09T13:53:57.788 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:57.782+0000 7fc6ae19b640 1 -- 172.21.15.43:0/614646892 wait complete. 2023-08-09T13:53:57.788 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:57.784+0000 7fc6ae19b640 1 Processor -- start 2023-08-09T13:53:57.788 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:57.784+0000 7fc6ae19b640 1 -- start start 2023-08-09T13:53:57.790 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:57.785+0000 7fc6ae19b640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc6940047c0 0x7fc6a8158ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:53:57.790 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:57.785+0000 7fc6ae19b640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fc6a8159420 0x7fc6a815b9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:53:57.790 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:57.785+0000 7fc6ae19b640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fc6a815bf30 0x7fc6a81f0f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:53:57.790 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:57.785+0000 7fc6ae19b640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc6a815c3b0 con 0x7fc6940047c0 2023-08-09T13:53:57.790 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:57.785+0000 7fc6ae19b640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc6a8157350 con 0x7fc6a815bf30 2023-08-09T13:53:57.790 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:57.785+0000 7fc6ae19b640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc6a8157630 con 0x7fc6a8159420 2023-08-09T13:53:57.791 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:57.785+0000 7fc6ad199640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc6940047c0 0x7fc6a8158ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:53:57.791 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:57.786+0000 7fc6ad199640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc6940047c0 0x7fc6a8158ee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:34362/0 (socket says 172.21.15.43:34362) 2023-08-09T13:53:57.791 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:57.786+0000 7fc6ad199640 1 -- 172.21.15.43:0/2812736651 learned_addr learned my addr 172.21.15.43:0/2812736651 (peer_addr_for_me v2:172.21.15.43:0/0) 2023-08-09T13:53:57.791 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:57.786+0000 7fc6ac998640 1 --2- 172.21.15.43:0/2812736651 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fc6a8159420 0x7fc6a815b9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:53:57.791 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:57.786+0000 7fc6ad199640 1 -- 172.21.15.43:0/2812736651 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fc6a8159420 msgr2=0x7fc6a815b9f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:53:57.791 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:57.786+0000 7fc6ad199640 1 --2- 172.21.15.43:0/2812736651 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fc6a8159420 0x7fc6a815b9f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:53:57.791 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:57.786+0000 7fc6ad199640 1 -- 172.21.15.43:0/2812736651 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fc6a815bf30 msgr2=0x7fc6a81f0f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:53:57.792 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:57.786+0000 7fc6ad199640 1 --2- 172.21.15.43:0/2812736651 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fc6a815bf30 0x7fc6a81f0f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:53:57.792 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:57.786+0000 7fc6ad199640 1 -- 172.21.15.43:0/2812736651 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc6a8157910 con 0x7fc6940047c0 2023-08-09T13:53:57.792 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:57.786+0000 7fc6ad199640 1 --2- 172.21.15.43:0/2812736651 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc6940047c0 0x7fc6a8158ee0 secure :-1 s=READY pgs=1547 cs=0 l=1 rev1=1 crypto rx=0x7fc6a400d7e0 tx=0x7fc6a400dca0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-09T13:53:57.792 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:57.787+0000 7fc69dffb640 1 -- 172.21.15.43:0/2812736651 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc6a40074e0 con 0x7fc6940047c0 2023-08-09T13:53:57.792 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:57.787+0000 7fc69dffb640 1 -- 172.21.15.43:0/2812736651 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc6a4007640 con 0x7fc6940047c0 2023-08-09T13:53:57.792 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:57.787+0000 7fc69dffb640 1 -- 172.21.15.43:0/2812736651 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc6a4004a00 con 0x7fc6940047c0 2023-08-09T13:53:57.792 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:57.787+0000 7fc6ae19b640 1 -- 172.21.15.43:0/2812736651 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc6a8157bf0 con 0x7fc6940047c0 2023-08-09T13:53:57.792 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:57.787+0000 7fc6ae19b640 1 -- 172.21.15.43:0/2812736651 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc6a815a0f0 con 0x7fc6940047c0 2023-08-09T13:53:57.793 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:57.788+0000 7fc6ae19b640 1 -- 172.21.15.43:0/2812736651 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc6a80739f0 con 0x7fc6940047c0 2023-08-09T13:53:57.793 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:57.789+0000 7fc69dffb640 1 -- 172.21.15.43:0/2812736651 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 56) v1 ==== 88443+0+0 (secure 0 0 0) 0x7fc6a4004b60 con 0x7fc6940047c0 2023-08-09T13:53:57.794 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:57.790+0000 7fc69dffb640 1 --2- 172.21.15.43:0/2812736651 >> [v2:172.21.15.203:6832/2527059680,v1:172.21.15.203:6833/2527059680] conn(0x7fc68c06a170 0x7fc68c06c630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:53:57.794 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:57.790+0000 7fc69dffb640 1 -- 172.21.15.43:0/2812736651 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(129..129 src has 1..129) v4 ==== 6426+0+0 (secure 0 0 0) 0x7fc6a4088360 con 0x7fc6940047c0 2023-08-09T13:53:57.794 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:57.791+0000 7fc6ac998640 1 --2- 172.21.15.43:0/2812736651 >> [v2:172.21.15.203:6832/2527059680,v1:172.21.15.203:6833/2527059680] conn(0x7fc68c06a170 0x7fc68c06c630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:53:57.794 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:57.792+0000 7fc6ac998640 1 --2- 172.21.15.43:0/2812736651 >> [v2:172.21.15.203:6832/2527059680,v1:172.21.15.203:6833/2527059680] conn(0x7fc68c06a170 0x7fc68c06c630 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7fc698036000 tx=0x7fc698030000 comp rx=0 tx=0).ready entity=mgr.6377 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-09T13:53:57.797 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:57.794+0000 7fc69dffb640 1 -- 172.21.15.43:0/2812736651 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178066 (secure 0 0 0) 0x7fc6a4053620 con 0x7fc6940047c0 2023-08-09T13:53:58.014 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:58.011+0000 7fc6ae19b640 1 -- 172.21.15.43:0/2812736651 --> [v2:172.21.15.203:6832/2527059680,v1:172.21.15.203:6833/2527059680] -- mgr_command(tid 0: {"prefix": "fs volume ls", "target": ["mon-mgr", ""]}) v1 -- 0x7fc6a80b81b0 con 0x7fc68c06a170 2023-08-09T13:53:58.017 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:58.015+0000 7fc69dffb640 1 -- 172.21.15.43:0/2812736651 <== mgr.6377 v2:172.21.15.203:6832/2527059680 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+41 (secure 0 0 0) 0x7fc69802ca00 con 0x7fc68c06a170 2023-08-09T13:53:58.018 INFO:teuthology.orchestra.run.smithi043.stdout:[ 2023-08-09T13:53:58.018 INFO:teuthology.orchestra.run.smithi043.stdout: { 2023-08-09T13:53:58.018 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "cephfs" 2023-08-09T13:53:58.018 INFO:teuthology.orchestra.run.smithi043.stdout: } 2023-08-09T13:53:58.018 INFO:teuthology.orchestra.run.smithi043.stdout:] 2023-08-09T13:53:58.020 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:58.018+0000 7fc6ae19b640 1 -- 172.21.15.43:0/2812736651 >> [v2:172.21.15.203:6832/2527059680,v1:172.21.15.203:6833/2527059680] conn(0x7fc68c06a170 msgr2=0x7fc68c06c630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:53:58.020 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:58.018+0000 7fc6ae19b640 1 --2- 172.21.15.43:0/2812736651 >> [v2:172.21.15.203:6832/2527059680,v1:172.21.15.203:6833/2527059680] conn(0x7fc68c06a170 0x7fc68c06c630 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7fc698036000 tx=0x7fc698030000 comp rx=0 tx=0).stop 2023-08-09T13:53:58.020 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:58.018+0000 7fc6ae19b640 1 -- 172.21.15.43:0/2812736651 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc6940047c0 msgr2=0x7fc6a8158ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:53:58.020 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:58.018+0000 7fc6ae19b640 1 --2- 172.21.15.43:0/2812736651 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc6940047c0 0x7fc6a8158ee0 secure :-1 s=READY pgs=1547 cs=0 l=1 rev1=1 crypto rx=0x7fc6a400d7e0 tx=0x7fc6a400dca0 comp rx=0 tx=0).stop 2023-08-09T13:53:58.021 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:58.018+0000 7fc6ae19b640 1 -- 172.21.15.43:0/2812736651 shutdown_connections 2023-08-09T13:53:58.021 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:58.018+0000 7fc6ae19b640 1 --2- 172.21.15.43:0/2812736651 >> [v2:172.21.15.203:6832/2527059680,v1:172.21.15.203:6833/2527059680] conn(0x7fc68c06a170 0x7fc68c06c630 unknown :-1 s=CLOSED pgs=70 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:53:58.021 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:58.018+0000 7fc6ae19b640 1 --2- 172.21.15.43:0/2812736651 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fc6a815bf30 0x7fc6a81f0f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:53:58.021 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:58.018+0000 7fc6ae19b640 1 --2- 172.21.15.43:0/2812736651 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fc6a8159420 0x7fc6a815b9f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:53:58.021 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:58.018+0000 7fc6ae19b640 1 --2- 172.21.15.43:0/2812736651 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc6940047c0 0x7fc6a8158ee0 unknown :-1 s=CLOSED pgs=1547 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:53:58.021 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:58.018+0000 7fc6ae19b640 1 -- 172.21.15.43:0/2812736651 >> 172.21.15.43:0/2812736651 conn(0x7fc6a806a2b0 msgr2=0x7fc6a815cd80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-09T13:53:58.022 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:58.018+0000 7fc6ae19b640 1 -- 172.21.15.43:0/2812736651 shutdown_connections 2023-08-09T13:53:58.022 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:58.018+0000 7fc6ae19b640 1 -- 172.21.15.43:0/2812736651 wait complete. 2023-08-09T13:53:58.040 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph mgr module enable snap_schedule 2023-08-09T13:53:58.266 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:58.263+0000 7f66622bf640 1 Processor -- start 2023-08-09T13:53:58.266 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:58.263+0000 7f66622bf640 1 -- start start 2023-08-09T13:53:58.267 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:58.265+0000 7f66622bf640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f665c0c86f0 0x7f665c0c8b50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:53:58.267 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:58.265+0000 7f66622bf640 1 -- --> v1:172.21.15.43:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f665c0c9090 con 0x7f665c06c8d0 2023-08-09T13:53:58.268 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:58.265+0000 7f66622bf640 1 -- --> v1:172.21.15.203:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f665c06c440 con 0x7f665c143c20 2023-08-09T13:53:58.268 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:58.265+0000 7f66622bf640 1 -- --> v2:172.21.15.43:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f665c144000 con 0x7f665c0c86f0 2023-08-09T13:53:58.268 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:58.265+0000 7f665b7fe640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f665c0c86f0 0x7f665c0c8b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:53:58.268 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:58.265+0000 7f666085c640 1 --1- >> v1:172.21.15.43:6789/0 conn(0x7f665c06c8d0 0x7f665c06cd00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.43:6789/0 says I am v1:172.21.15.43:48552/0 (socket says 172.21.15.43:48552) 2023-08-09T13:53:58.268 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:58.265+0000 7f665b7fe640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f665c0c86f0 0x7f665c0c8b50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:34376/0 (socket says 172.21.15.43:34376) 2023-08-09T13:53:58.268 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:58.265+0000 7f666085c640 1 -- 172.21.15.43:0/536263163 learned_addr learned my addr 172.21.15.43:0/536263163 (peer_addr_for_me v1:172.21.15.43:0/0) 2023-08-09T13:53:58.269 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:58.266+0000 7f665b7fe640 1 -- 172.21.15.43:0/536263163 >> v1:172.21.15.203:6789/0 conn(0x7f665c143c20 legacy=0x7f665c06f540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-09T13:53:58.269 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:58.266+0000 7f665b7fe640 1 -- 172.21.15.43:0/536263163 >> v1:172.21.15.43:6789/0 conn(0x7f665c06c8d0 legacy=0x7f665c06cd00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:53:58.269 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:58.266+0000 7f665b7fe640 1 -- 172.21.15.43:0/536263163 --> v2:172.21.15.43:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f665c06fc50 con 0x7f665c0c86f0 2023-08-09T13:53:58.270 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:58.266+0000 7f665affd640 1 -- 172.21.15.43:0/536263163 <== mon.0 v1:172.21.15.43:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3162567463 0 0) 0x7f665c0c9090 con 0x7f665c06c8d0 2023-08-09T13:53:58.270 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:58.267+0000 7f665b7fe640 1 --2- 172.21.15.43:0/536263163 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f665c0c86f0 0x7f665c0c8b50 secure :-1 s=READY pgs=1549 cs=0 l=1 rev1=1 crypto rx=0x7f66500097a0 tx=0x7f665002b7a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2e0caaedcb2268b server_cookie=0 in_seq=0 out_seq=0 2023-08-09T13:53:58.270 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:58.267+0000 7f665affd640 1 -- 172.21.15.43:0/536263163 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f665002f050 con 0x7f665c0c86f0 2023-08-09T13:53:58.270 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:58.267+0000 7f665affd640 1 -- 172.21.15.43:0/536263163 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6650031700 con 0x7f665c0c86f0 2023-08-09T13:53:58.270 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:58.267+0000 7f665affd640 1 -- 172.21.15.43:0/536263163 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6650031a90 con 0x7f665c0c86f0 2023-08-09T13:53:58.271 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:58.268+0000 7f66622bf640 1 -- 172.21.15.43:0/536263163 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f665c0c86f0 msgr2=0x7f665c0c8b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:53:58.271 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:58.268+0000 7f66622bf640 1 --2- 172.21.15.43:0/536263163 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f665c0c86f0 0x7f665c0c8b50 secure :-1 s=READY pgs=1549 cs=0 l=1 rev1=1 crypto rx=0x7f66500097a0 tx=0x7f665002b7a0 comp rx=0 tx=0).stop 2023-08-09T13:53:58.271 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:58.268+0000 7f66622bf640 1 -- 172.21.15.43:0/536263163 shutdown_connections 2023-08-09T13:53:58.271 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:58.268+0000 7f66622bf640 1 --2- 172.21.15.43:0/536263163 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f665c0c86f0 0x7f665c0c8b50 unknown :-1 s=CLOSED pgs=1549 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:53:58.271 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:58.268+0000 7f66622bf640 1 -- 172.21.15.43:0/536263163 >> 172.21.15.43:0/536263163 conn(0x7f665c0ce0a0 msgr2=0x7f665c0ce480 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-09T13:53:58.272 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:58.268+0000 7f66622bf640 1 -- 172.21.15.43:0/536263163 shutdown_connections 2023-08-09T13:53:58.272 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:58.268+0000 7f66622bf640 1 -- 172.21.15.43:0/536263163 wait complete. 2023-08-09T13:53:58.272 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:58.270+0000 7f66622bf640 1 Processor -- start 2023-08-09T13:53:58.272 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:58.270+0000 7f66622bf640 1 -- start start 2023-08-09T13:53:58.273 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:58.271+0000 7f66622bf640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f665c06c8d0 0x7f665c1f2050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:53:58.273 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:58.271+0000 7f66622bf640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f665c0c86f0 0x7f665c15c130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:53:58.274 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:58.271+0000 7f66622bf640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f665c143c20 0x7f665c0b3450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:53:58.274 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:58.271+0000 7f66622bf640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f665c1f2590 con 0x7f665c06c8d0 2023-08-09T13:53:58.274 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:58.271+0000 7f66622bf640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f665c0abb80 con 0x7f665c143c20 2023-08-09T13:53:58.274 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:58.271+0000 7f66622bf640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f665c0abe60 con 0x7f665c0c86f0 2023-08-09T13:53:58.274 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:58.271+0000 7f665bfff640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f665c06c8d0 0x7f665c1f2050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:53:58.274 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:58.271+0000 7f665bfff640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f665c06c8d0 0x7f665c1f2050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:34378/0 (socket says 172.21.15.43:34378) 2023-08-09T13:53:58.274 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:58.271+0000 7f665bfff640 1 -- 172.21.15.43:0/3229577900 learned_addr learned my addr 172.21.15.43:0/3229577900 (peer_addr_for_me v2:172.21.15.43:0/0) 2023-08-09T13:53:58.275 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:58.272+0000 7f665bfff640 1 -- 172.21.15.43:0/3229577900 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f665c0c86f0 msgr2=0x7f665c15c130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:53:58.275 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:58.272+0000 7f665bfff640 1 --2- 172.21.15.43:0/3229577900 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f665c0c86f0 0x7f665c15c130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:53:58.276 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:58.272+0000 7f665bfff640 1 -- 172.21.15.43:0/3229577900 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f665c143c20 msgr2=0x7f665c0b3450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:53:58.276 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:58.272+0000 7f665bfff640 1 --2- 172.21.15.43:0/3229577900 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f665c143c20 0x7f665c0b3450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:53:58.276 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:58.272+0000 7f665bfff640 1 -- 172.21.15.43:0/3229577900 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6650009400 con 0x7f665c06c8d0 2023-08-09T13:53:58.276 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:58.272+0000 7f665bfff640 1 --2- 172.21.15.43:0/3229577900 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f665c06c8d0 0x7f665c1f2050 secure :-1 s=READY pgs=1550 cs=0 l=1 rev1=1 crypto rx=0x7f664c00c430 tx=0x7f664c00c8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-09T13:53:58.276 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:58.272+0000 7f6658ff9640 1 -- 172.21.15.43:0/3229577900 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f664c011790 con 0x7f665c06c8d0 2023-08-09T13:53:58.276 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:58.272+0000 7f6658ff9640 1 -- 172.21.15.43:0/3229577900 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f664c0118f0 con 0x7f665c06c8d0 2023-08-09T13:53:58.276 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:58.272+0000 7f6658ff9640 1 -- 172.21.15.43:0/3229577900 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f664c011ce0 con 0x7f665c06c8d0 2023-08-09T13:53:58.277 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:58.272+0000 7f66622bf640 1 -- 172.21.15.43:0/3229577900 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f665c0ac140 con 0x7f665c06c8d0 2023-08-09T13:53:58.277 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:58.272+0000 7f66622bf640 1 -- 172.21.15.43:0/3229577900 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f665c0ac700 con 0x7f665c06c8d0 2023-08-09T13:53:58.277 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:58.274+0000 7f66622bf640 1 -- 172.21.15.43:0/3229577900 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f665c0739f0 con 0x7f665c06c8d0 2023-08-09T13:53:58.277 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:58.275+0000 7f6658ff9640 1 -- 172.21.15.43:0/3229577900 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 56) v1 ==== 88443+0+0 (secure 0 0 0) 0x7f664c01c460 con 0x7f665c06c8d0 2023-08-09T13:53:58.278 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:58.276+0000 7f6658ff9640 1 --2- 172.21.15.43:0/3229577900 >> [v2:172.21.15.203:6832/2527059680,v1:172.21.15.203:6833/2527059680] conn(0x7f664806a170 0x7f664806c630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:53:58.278 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:58.276+0000 7f6658ff9640 1 -- 172.21.15.43:0/3229577900 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(129..129 src has 1..129) v4 ==== 6426+0+0 (secure 0 0 0) 0x7f664c088730 con 0x7f665c06c8d0 2023-08-09T13:53:58.278 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:58.276+0000 7f665b7fe640 1 --2- 172.21.15.43:0/3229577900 >> [v2:172.21.15.203:6832/2527059680,v1:172.21.15.203:6833/2527059680] conn(0x7f664806a170 0x7f664806c630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:53:58.280 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:58.277+0000 7f665b7fe640 1 --2- 172.21.15.43:0/3229577900 >> [v2:172.21.15.203:6832/2527059680,v1:172.21.15.203:6833/2527059680] conn(0x7f664806a170 0x7f664806c630 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f6650035000 tx=0x7f6650030000 comp rx=0 tx=0).ready entity=mgr.6377 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-09T13:53:58.282 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:58.280+0000 7f6658ff9640 1 -- 172.21.15.43:0/3229577900 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178066 (secure 0 0 0) 0x7f664c053c30 con 0x7f665c06c8d0 2023-08-09T13:53:58.529 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:58.526+0000 7f66622bf640 1 -- 172.21.15.43:0/3229577900 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "mgr module enable", "module": "snap_schedule"} v 0) v1 -- 0x7f665c0b4290 con 0x7f665c06c8d0 2023-08-09T13:53:59.214 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:59.211+0000 7f6658ff9640 1 -- 172.21.15.43:0/3229577900 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mgrmap(e 57) v1 ==== 88460+0+0 (secure 0 0 0) 0x7f664c054000 con 0x7f665c06c8d0 2023-08-09T13:53:59.217 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:59.215+0000 7f6658ff9640 1 -- 172.21.15.43:0/3229577900 <== mon.0 v2:172.21.15.43:3300/0 8 ==== mon_command_ack([{"prefix": "mgr module enable", "module": "snap_schedule"}]=0 v57) v1 ==== 92+0+0 (secure 0 0 0) 0x7f664c0576c0 con 0x7f665c06c8d0 2023-08-09T13:53:59.221 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:59.219+0000 7f66622bf640 1 -- 172.21.15.43:0/3229577900 >> [v2:172.21.15.203:6832/2527059680,v1:172.21.15.203:6833/2527059680] conn(0x7f664806a170 msgr2=0x7f664806c630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:53:59.221 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:59.219+0000 7f66622bf640 1 --2- 172.21.15.43:0/3229577900 >> [v2:172.21.15.203:6832/2527059680,v1:172.21.15.203:6833/2527059680] conn(0x7f664806a170 0x7f664806c630 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f6650035000 tx=0x7f6650030000 comp rx=0 tx=0).stop 2023-08-09T13:53:59.222 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:59.219+0000 7f66622bf640 1 -- 172.21.15.43:0/3229577900 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f665c06c8d0 msgr2=0x7f665c1f2050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:53:59.222 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:59.219+0000 7f66622bf640 1 --2- 172.21.15.43:0/3229577900 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f665c06c8d0 0x7f665c1f2050 secure :-1 s=READY pgs=1550 cs=0 l=1 rev1=1 crypto rx=0x7f664c00c430 tx=0x7f664c00c8f0 comp rx=0 tx=0).stop 2023-08-09T13:53:59.222 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:59.219+0000 7f66622bf640 1 -- 172.21.15.43:0/3229577900 shutdown_connections 2023-08-09T13:53:59.222 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:59.219+0000 7f66622bf640 1 --2- 172.21.15.43:0/3229577900 >> [v2:172.21.15.203:6832/2527059680,v1:172.21.15.203:6833/2527059680] conn(0x7f664806a170 0x7f664806c630 unknown :-1 s=CLOSED pgs=71 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:53:59.223 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:59.219+0000 7f66622bf640 1 --2- 172.21.15.43:0/3229577900 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f665c143c20 0x7f665c0b3450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:53:59.223 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:59.219+0000 7f66622bf640 1 --2- 172.21.15.43:0/3229577900 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f665c0c86f0 0x7f665c15c130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:53:59.223 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:59.219+0000 7f66622bf640 1 --2- 172.21.15.43:0/3229577900 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f665c06c8d0 0x7f665c1f2050 unknown :-1 s=CLOSED pgs=1550 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:53:59.223 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:59.219+0000 7f66622bf640 1 -- 172.21.15.43:0/3229577900 >> 172.21.15.43:0/3229577900 conn(0x7f665c0ce0a0 msgr2=0x7f665c06fe40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-09T13:53:59.223 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:59.219+0000 7f66622bf640 1 -- 172.21.15.43:0/3229577900 shutdown_connections 2023-08-09T13:53:59.223 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:59.220+0000 7f66622bf640 1 -- 172.21.15.43:0/3229577900 wait complete. 2023-08-09T13:53:59.243 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mgr mgr/snap_schedule/allow_m_granularity True 2023-08-09T13:53:59.430 INFO:tasks.ceph.mgr.x.smithi203.stderr:2023-08-09T13:53:59.429+0000 7f0993c41040 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2023-08-09T13:53:59.452 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:59.450+0000 7fafb7a99640 1 Processor -- start 2023-08-09T13:53:59.453 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:59.450+0000 7fafb7a99640 1 -- start start 2023-08-09T13:53:59.454 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:59.451+0000 7fafb7a99640 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7fafb015c550 0x7fafb00cd6d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:53:59.454 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:59.451+0000 7fafb7a99640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fafb00d04b0 0x7fafb00b3ee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:53:59.454 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:59.451+0000 7fafb7a99640 1 -- --> v1:172.21.15.43:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fafb00b4870 con 0x7fafb00b4420 2023-08-09T13:53:59.454 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:59.451+0000 7fafb7a99640 1 -- --> v2:172.21.15.203:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fafb00b7bd0 con 0x7fafb015c550 2023-08-09T13:53:59.455 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:59.451+0000 7fafb7a99640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fafb00d1c00 con 0x7fafb00d04b0 2023-08-09T13:53:59.456 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:59.452+0000 7fafb6036640 1 --1- >> v1:172.21.15.43:6789/0 conn(0x7fafb00b4420 0x7fafb006a760 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.43:6789/0 says I am v1:172.21.15.43:48572/0 (socket says 172.21.15.43:48572) 2023-08-09T13:53:59.456 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:59.452+0000 7fafb6036640 1 -- 172.21.15.43:0/2448247016 learned_addr learned my addr 172.21.15.43:0/2448247016 (peer_addr_for_me v1:172.21.15.43:0/0) 2023-08-09T13:53:59.456 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:59.452+0000 7fafb5034640 1 --2- 172.21.15.43:0/2448247016 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fafb00d04b0 0x7fafb00b3ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:53:59.456 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:59.452+0000 7fafb5835640 1 --2- 172.21.15.43:0/2448247016 >> v2:172.21.15.203:3300/0 conn(0x7fafb015c550 0x7fafb00cd6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:53:59.456 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:59.452+0000 7fafb4833640 1 -- 172.21.15.43:0/2448247016 <== mon.0 v1:172.21.15.43:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2943479005 0 0) 0x7fafa8001490 con 0x7fafb00b4420 2023-08-09T13:53:59.456 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:59.452+0000 7fafb4833640 1 -- 172.21.15.43:0/2448247016 --> v1:172.21.15.43:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7faf9c003520 con 0x7fafb00b4420 2023-08-09T13:53:59.457 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:59.453+0000 7fafb5034640 1 -- 172.21.15.43:0/2448247016 >> v2:172.21.15.203:3300/0 conn(0x7fafb015c550 msgr2=0x7fafb00cd6d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-09T13:53:59.457 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:59.453+0000 7fafb5034640 1 --2- 172.21.15.43:0/2448247016 >> v2:172.21.15.203:3300/0 conn(0x7fafb015c550 0x7fafb00cd6d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:53:59.457 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:59.453+0000 7fafb5034640 1 -- 172.21.15.43:0/2448247016 >> v1:172.21.15.43:6789/0 conn(0x7fafb00b4420 legacy=0x7fafb006a760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:53:59.457 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:59.453+0000 7fafb5034640 1 -- 172.21.15.43:0/2448247016 --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fafb015c930 con 0x7fafb00d04b0 2023-08-09T13:53:59.458 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:59.455+0000 7fafb5034640 1 --2- 172.21.15.43:0/2448247016 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fafb00d04b0 0x7fafb00b3ee0 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7fafa00097a0 tx=0x7fafa002b8b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=569fed1fe0934ab4 server_cookie=0 in_seq=0 out_seq=0 2023-08-09T13:53:59.458 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:59.455+0000 7fafb4833640 1 -- 172.21.15.43:0/2448247016 <== mon.2 v2:172.21.15.203:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fafa002d040 con 0x7fafb00d04b0 2023-08-09T13:53:59.458 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:59.455+0000 7fafb4833640 1 -- 172.21.15.43:0/2448247016 <== mon.2 v2:172.21.15.203:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fafa0031810 con 0x7fafb00d04b0 2023-08-09T13:53:59.458 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:59.455+0000 7fafb4833640 1 -- 172.21.15.43:0/2448247016 <== mon.2 v2:172.21.15.203:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fafa0031ba0 con 0x7fafb00d04b0 2023-08-09T13:53:59.459 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:59.456+0000 7fafb7a99640 1 -- 172.21.15.43:0/2448247016 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fafb00d04b0 msgr2=0x7fafb00b3ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:53:59.459 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:59.456+0000 7fafb7a99640 1 --2- 172.21.15.43:0/2448247016 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fafb00d04b0 0x7fafb00b3ee0 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7fafa00097a0 tx=0x7fafa002b8b0 comp rx=0 tx=0).stop 2023-08-09T13:53:59.459 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:59.456+0000 7fafb7a99640 1 -- 172.21.15.43:0/2448247016 shutdown_connections 2023-08-09T13:53:59.459 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:59.456+0000 7fafb7a99640 1 --2- 172.21.15.43:0/2448247016 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fafb00d04b0 0x7fafb00b3ee0 unknown :-1 s=CLOSED pgs=207 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:53:59.459 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:59.456+0000 7fafb7a99640 1 --2- 172.21.15.43:0/2448247016 >> v2:172.21.15.203:3300/0 conn(0x7fafb015c550 0x7fafb00cd6d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:53:59.459 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:59.456+0000 7fafb7a99640 1 -- 172.21.15.43:0/2448247016 >> 172.21.15.43:0/2448247016 conn(0x7fafb0071520 msgr2=0x7fafb00ceee0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-09T13:53:59.459 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:59.456+0000 7fafb7a99640 1 -- 172.21.15.43:0/2448247016 shutdown_connections 2023-08-09T13:53:59.460 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:59.456+0000 7fafb7a99640 1 -- 172.21.15.43:0/2448247016 wait complete. 2023-08-09T13:53:59.460 INFO:tasks.ceph.mgr.y.smithi043.stderr:2023-08-09T13:53:59.457+0000 7f122d838040 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2023-08-09T13:53:59.461 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:59.458+0000 7fafb7a99640 1 Processor -- start 2023-08-09T13:53:59.461 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:59.458+0000 7fafb7a99640 1 -- start start 2023-08-09T13:53:59.462 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:59.459+0000 7fafb7a99640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fafb00b4420 0x7fafb014b3b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:53:59.463 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:59.459+0000 7fafb7a99640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fafb00d04b0 0x7fafb0149320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:53:59.463 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:59.459+0000 7fafb7a99640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fafb015c550 0x7fafb01f2160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:53:59.463 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:59.459+0000 7fafb7a99640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fafb00c96d0 con 0x7fafb00d04b0 2023-08-09T13:53:59.463 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:59.459+0000 7fafb7a99640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fafb00a9110 con 0x7fafb00b4420 2023-08-09T13:53:59.463 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:59.459+0000 7fafb7a99640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fafb014b990 con 0x7fafb015c550 2023-08-09T13:53:59.464 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:59.459+0000 7fafb5034640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fafb00d04b0 0x7fafb0149320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:53:59.464 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:59.460+0000 7fafb5034640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fafb00d04b0 0x7fafb0149320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:34392/0 (socket says 172.21.15.43:34392) 2023-08-09T13:53:59.464 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:59.460+0000 7fafb5034640 1 -- 172.21.15.43:0/1630442948 learned_addr learned my addr 172.21.15.43:0/1630442948 (peer_addr_for_me v2:172.21.15.43:0/0) 2023-08-09T13:53:59.464 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:59.460+0000 7fafb6036640 1 --2- 172.21.15.43:0/1630442948 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fafb015c550 0x7fafb01f2160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:53:59.464 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:59.460+0000 7fafb5835640 1 --2- 172.21.15.43:0/1630442948 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fafb00b4420 0x7fafb014b3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:53:59.465 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:59.460+0000 7fafb5034640 1 -- 172.21.15.43:0/1630442948 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fafb015c550 msgr2=0x7fafb01f2160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:53:59.465 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:59.460+0000 7fafb5034640 1 --2- 172.21.15.43:0/1630442948 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fafb015c550 0x7fafb01f2160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:53:59.465 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:59.460+0000 7fafb5034640 1 -- 172.21.15.43:0/1630442948 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fafb00b4420 msgr2=0x7fafb014b3b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:53:59.465 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:59.460+0000 7fafb5034640 1 --2- 172.21.15.43:0/1630442948 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fafb00b4420 0x7fafb014b3b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:53:59.465 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:59.460+0000 7fafb5034640 1 -- 172.21.15.43:0/1630442948 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fafa0009400 con 0x7fafb00d04b0 2023-08-09T13:53:59.465 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:59.460+0000 7fafb5835640 1 --2- 172.21.15.43:0/1630442948 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fafb00b4420 0x7fafb014b3b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-08-09T13:53:59.466 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:59.461+0000 7fafb5034640 1 --2- 172.21.15.43:0/1630442948 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fafb00d04b0 0x7fafb0149320 secure :-1 s=READY pgs=1555 cs=0 l=1 rev1=1 crypto rx=0x7fafa00098c0 tx=0x7fafa00046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-09T13:53:59.466 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:59.461+0000 7fafa67fc640 1 -- 172.21.15.43:0/1630442948 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fafa003ad60 con 0x7fafb00d04b0 2023-08-09T13:53:59.466 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:59.461+0000 7fafa67fc640 1 -- 172.21.15.43:0/1630442948 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fafa0044400 con 0x7fafb00d04b0 2023-08-09T13:53:59.467 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:59.461+0000 7fafb7a99640 1 -- 172.21.15.43:0/1630442948 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fafb014bc70 con 0x7fafb00d04b0 2023-08-09T13:53:59.467 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:59.461+0000 7fafa67fc640 1 -- 172.21.15.43:0/1630442948 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fafa00445b0 con 0x7fafb00d04b0 2023-08-09T13:53:59.467 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:59.461+0000 7fafb7a99640 1 -- 172.21.15.43:0/1630442948 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fafb014c230 con 0x7fafb00d04b0 2023-08-09T13:53:59.467 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:59.463+0000 7fafb7a99640 1 -- 172.21.15.43:0/1630442948 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fafb00c9810 con 0x7fafb00d04b0 2023-08-09T13:53:59.469 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:59.464+0000 7fafa67fc640 1 -- 172.21.15.43:0/1630442948 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 57) v1 ==== 88460+0+0 (secure 0 0 0) 0x7fafa004d440 con 0x7fafb00d04b0 2023-08-09T13:53:59.469 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:59.466+0000 7fafa67fc640 1 --2- 172.21.15.43:0/1630442948 >> [v2:172.21.15.203:6832/2527059680,v1:172.21.15.203:6833/2527059680] conn(0x7faf9406a1c0 0x7faf9406c680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:53:59.469 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:59.466+0000 7fafa67fc640 1 -- 172.21.15.43:0/1630442948 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(129..129 src has 1..129) v4 ==== 6426+0+0 (secure 0 0 0) 0x7fafa00aaaa0 con 0x7fafb00d04b0 2023-08-09T13:53:59.469 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:59.466+0000 7fafb5835640 1 -- 172.21.15.43:0/1630442948 >> [v2:172.21.15.203:6832/2527059680,v1:172.21.15.203:6833/2527059680] conn(0x7faf9406a1c0 msgr2=0x7faf9406c680 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.203:6832/2527059680 2023-08-09T13:53:59.470 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:59.466+0000 7fafb5835640 1 --2- 172.21.15.43:0/1630442948 >> [v2:172.21.15.203:6832/2527059680,v1:172.21.15.203:6833/2527059680] conn(0x7faf9406a1c0 0x7faf9406c680 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-08-09T13:53:59.473 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:59.471+0000 7fafa67fc640 1 -- 172.21.15.43:0/1630442948 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178066 (secure 0 0 0) 0x7fafb00c9810 con 0x7fafb00d04b0 2023-08-09T13:53:59.665 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:59.662+0000 7fafb7a99640 1 -- 172.21.15.43:0/1630442948 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command([{prefix=config set, name=mgr/snap_schedule/allow_m_granularity}] v 0) v1 -- 0x7fafb00c9810 con 0x7fafb00d04b0 2023-08-09T13:53:59.669 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:59.667+0000 7fafb5835640 1 -- 172.21.15.43:0/1630442948 >> [v2:172.21.15.203:6832/2527059680,v1:172.21.15.203:6833/2527059680] conn(0x7faf9406a1c0 msgr2=0x7faf9406c680 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.203:6832/2527059680 2023-08-09T13:53:59.670 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:59.667+0000 7fafb5835640 1 --2- 172.21.15.43:0/1630442948 >> [v2:172.21.15.203:6832/2527059680,v1:172.21.15.203:6833/2527059680] conn(0x7faf9406a1c0 0x7faf9406c680 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2023-08-09T13:53:59.671 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:59.669+0000 7fafa67fc640 1 -- 172.21.15.43:0/1630442948 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mgr/snap_schedule/allow_m_granularity}]=0 v41)=0 v41) v1 ==== 138+0+0 (secure 0 0 0) 0x7fafb00c9810 con 0x7fafb00d04b0 2023-08-09T13:53:59.674 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:59.672+0000 7fafb7a99640 1 -- 172.21.15.43:0/1630442948 >> [v2:172.21.15.203:6832/2527059680,v1:172.21.15.203:6833/2527059680] conn(0x7faf9406a1c0 msgr2=0x7faf9406c680 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-08-09T13:53:59.675 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:59.672+0000 7fafb7a99640 1 --2- 172.21.15.43:0/1630442948 >> [v2:172.21.15.203:6832/2527059680,v1:172.21.15.203:6833/2527059680] conn(0x7faf9406a1c0 0x7faf9406c680 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:53:59.675 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:59.672+0000 7fafb7a99640 1 -- 172.21.15.43:0/1630442948 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fafb00d04b0 msgr2=0x7fafb0149320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:53:59.675 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:59.672+0000 7fafb7a99640 1 --2- 172.21.15.43:0/1630442948 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fafb00d04b0 0x7fafb0149320 secure :-1 s=READY pgs=1555 cs=0 l=1 rev1=1 crypto rx=0x7fafa00098c0 tx=0x7fafa00046c0 comp rx=0 tx=0).stop 2023-08-09T13:53:59.675 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:59.672+0000 7fafb7a99640 1 -- 172.21.15.43:0/1630442948 shutdown_connections 2023-08-09T13:53:59.675 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:59.672+0000 7fafb7a99640 1 --2- 172.21.15.43:0/1630442948 >> [v2:172.21.15.203:6832/2527059680,v1:172.21.15.203:6833/2527059680] conn(0x7faf9406a1c0 0x7faf9406c680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:53:59.675 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:59.672+0000 7fafb7a99640 1 --2- 172.21.15.43:0/1630442948 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fafb015c550 0x7fafb01f2160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:53:59.675 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:59.672+0000 7fafb7a99640 1 --2- 172.21.15.43:0/1630442948 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fafb00d04b0 0x7fafb0149320 unknown :-1 s=CLOSED pgs=1555 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:53:59.675 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:59.672+0000 7fafb7a99640 1 --2- 172.21.15.43:0/1630442948 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fafb00b4420 0x7fafb014b3b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:53:59.676 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:59.672+0000 7fafb7a99640 1 -- 172.21.15.43:0/1630442948 >> 172.21.15.43:0/1630442948 conn(0x7fafb0071520 msgr2=0x7fafb015ecb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-09T13:53:59.676 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:59.672+0000 7fafb7a99640 1 -- 172.21.15.43:0/1630442948 shutdown_connections 2023-08-09T13:53:59.676 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:59.672+0000 7fafb7a99640 1 -- 172.21.15.43:0/1630442948 wait complete. 2023-08-09T13:53:59.693 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mgr mgr/snap_schedule/dump_on_update True 2023-08-09T13:53:59.906 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:59.902+0000 7f52a2986640 1 Processor -- start 2023-08-09T13:53:59.907 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:59.902+0000 7f52a2986640 1 -- start start 2023-08-09T13:53:59.907 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:59.904+0000 7f52a2986640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f529c070cf0 0x7f529c15ce50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:53:59.907 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:59.904+0000 7f52a2986640 1 -- --> v1:172.21.15.43:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f529c06ea70 con 0x7f529c15d390 2023-08-09T13:53:59.907 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:59.904+0000 7f52a2986640 1 -- --> v1:172.21.15.203:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f529c06dcb0 con 0x7f529c15ca70 2023-08-09T13:53:59.907 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:59.904+0000 7f52a2986640 1 -- --> v2:172.21.15.43:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f529c06df90 con 0x7f529c070cf0 2023-08-09T13:53:59.908 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:59.904+0000 7f52a0f23640 1 --1- >> v1:172.21.15.43:6789/0 conn(0x7f529c15d390 0x7f529c0697c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.43:6789/0 says I am v1:172.21.15.43:48580/0 (socket says 172.21.15.43:48580) 2023-08-09T13:53:59.908 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:59.904+0000 7f52a0f23640 1 -- 172.21.15.43:0/1541792389 learned_addr learned my addr 172.21.15.43:0/1541792389 (peer_addr_for_me v1:172.21.15.43:0/0) 2023-08-09T13:53:59.908 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:59.904+0000 7f529b7fe640 1 --2- 172.21.15.43:0/1541792389 >> v2:172.21.15.43:3300/0 conn(0x7f529c070cf0 0x7f529c15ce50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:53:59.908 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:59.905+0000 7f529affd640 1 -- 172.21.15.43:0/1541792389 <== mon.0 v1:172.21.15.43:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3561115072 0 0) 0x7f529c06ea70 con 0x7f529c15d390 2023-08-09T13:53:59.908 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:59.905+0000 7f529b7fe640 1 -- 172.21.15.43:0/1541792389 >> v1:172.21.15.203:6789/0 conn(0x7f529c15ca70 legacy=0x7f529c0705e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:53:59.909 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:59.905+0000 7f529b7fe640 1 -- 172.21.15.43:0/1541792389 >> v1:172.21.15.43:6789/0 conn(0x7f529c15d390 legacy=0x7f529c0697c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:53:59.909 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:59.905+0000 7f529b7fe640 1 -- 172.21.15.43:0/1541792389 --> v2:172.21.15.43:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f529c06e270 con 0x7f529c070cf0 2023-08-09T13:53:59.909 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:59.906+0000 7f529b7fe640 1 --2- 172.21.15.43:0/1541792389 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f529c070cf0 0x7f529c15ce50 secure :-1 s=READY pgs=1557 cs=0 l=1 rev1=1 crypto rx=0x7f52900097a0 tx=0x7f529002b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a0196164442f0451 server_cookie=0 in_seq=0 out_seq=0 2023-08-09T13:53:59.910 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:59.906+0000 7f529affd640 1 -- 172.21.15.43:0/1541792389 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5290031830 con 0x7f529c070cf0 2023-08-09T13:53:59.910 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:59.906+0000 7f529affd640 1 -- 172.21.15.43:0/1541792389 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5290031990 con 0x7f529c070cf0 2023-08-09T13:53:59.910 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:59.906+0000 7f529affd640 1 -- 172.21.15.43:0/1541792389 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5290031d20 con 0x7f529c070cf0 2023-08-09T13:53:59.910 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:59.907+0000 7f52a2986640 1 -- 172.21.15.43:0/1541792389 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f529c070cf0 msgr2=0x7f529c15ce50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:53:59.910 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:59.907+0000 7f52a2986640 1 --2- 172.21.15.43:0/1541792389 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f529c070cf0 0x7f529c15ce50 secure :-1 s=READY pgs=1557 cs=0 l=1 rev1=1 crypto rx=0x7f52900097a0 tx=0x7f529002b8d0 comp rx=0 tx=0).stop 2023-08-09T13:53:59.910 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:59.907+0000 7f52a2986640 1 -- 172.21.15.43:0/1541792389 shutdown_connections 2023-08-09T13:53:59.911 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:59.907+0000 7f52a2986640 1 --2- 172.21.15.43:0/1541792389 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f529c070cf0 0x7f529c15ce50 unknown :-1 s=CLOSED pgs=1557 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:53:59.911 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:59.907+0000 7f52a2986640 1 -- 172.21.15.43:0/1541792389 >> 172.21.15.43:0/1541792389 conn(0x7f529c0cf0b0 msgr2=0x7f529c0cf4c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-09T13:53:59.911 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:59.907+0000 7f52a2986640 1 -- 172.21.15.43:0/1541792389 shutdown_connections 2023-08-09T13:53:59.911 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:59.907+0000 7f52a2986640 1 -- 172.21.15.43:0/1541792389 wait complete. 2023-08-09T13:53:59.911 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:59.909+0000 7f52a2986640 1 Processor -- start 2023-08-09T13:53:59.912 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:59.909+0000 7f52a2986640 1 -- start start 2023-08-09T13:53:59.913 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:59.910+0000 7f52a2986640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f529c070cf0 0x7f529c15b5a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:53:59.913 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:59.910+0000 7f52a2986640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f529c15ca70 0x7f529c0c3bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:53:59.913 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:59.910+0000 7f52a2986640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f529c15d390 0x7f529c0c4130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:53:59.913 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:59.910+0000 7f52a2986640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f529c0c58e0 con 0x7f529c15d390 2023-08-09T13:53:59.913 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:59.910+0000 7f52a2986640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f529c0c4840 con 0x7f529c070cf0 2023-08-09T13:53:59.913 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:59.910+0000 7f52a2986640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f529c0c4b20 con 0x7f529c15ca70 2023-08-09T13:53:59.914 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:59.910+0000 7f52a0f23640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f529c15d390 0x7f529c0c4130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:53:59.914 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:59.910+0000 7f52a0f23640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f529c15d390 0x7f529c0c4130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:34422/0 (socket says 172.21.15.43:34422) 2023-08-09T13:53:59.914 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:59.910+0000 7f52a0f23640 1 -- 172.21.15.43:0/1911687691 learned_addr learned my addr 172.21.15.43:0/1911687691 (peer_addr_for_me v2:172.21.15.43:0/0) 2023-08-09T13:53:59.914 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:59.911+0000 7f529b7fe640 1 --2- 172.21.15.43:0/1911687691 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f529c15ca70 0x7f529c0c3bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:53:59.915 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:59.911+0000 7f529bfff640 1 --2- 172.21.15.43:0/1911687691 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f529c070cf0 0x7f529c15b5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:53:59.915 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:59.911+0000 7f52a0f23640 1 -- 172.21.15.43:0/1911687691 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f529c15ca70 msgr2=0x7f529c0c3bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:53:59.915 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:59.911+0000 7f52a0f23640 1 --2- 172.21.15.43:0/1911687691 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f529c15ca70 0x7f529c0c3bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:53:59.916 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:59.911+0000 7f52a0f23640 1 -- 172.21.15.43:0/1911687691 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f529c070cf0 msgr2=0x7f529c15b5a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:53:59.916 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:59.911+0000 7f52a0f23640 1 --2- 172.21.15.43:0/1911687691 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f529c070cf0 0x7f529c15b5a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:53:59.916 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:59.911+0000 7f52a0f23640 1 -- 172.21.15.43:0/1911687691 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5290009400 con 0x7f529c15d390 2023-08-09T13:53:59.916 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:59.911+0000 7f52a0f23640 1 --2- 172.21.15.43:0/1911687691 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f529c15d390 0x7f529c0c4130 secure :-1 s=READY pgs=1558 cs=0 l=1 rev1=1 crypto rx=0x7f529400c700 tx=0x7f529400cbc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-09T13:53:59.916 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:59.911+0000 7f5298ff9640 1 -- 172.21.15.43:0/1911687691 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f529400ce10 con 0x7f529c15d390 2023-08-09T13:53:59.916 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:59.911+0000 7f5298ff9640 1 -- 172.21.15.43:0/1911687691 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f529400a640 con 0x7f529c15d390 2023-08-09T13:53:59.917 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:59.911+0000 7f5298ff9640 1 -- 172.21.15.43:0/1911687691 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f529400aa70 con 0x7f529c15d390 2023-08-09T13:53:59.917 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:59.912+0000 7f52a2986640 1 -- 172.21.15.43:0/1911687691 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f529c0c4e00 con 0x7f529c15d390 2023-08-09T13:53:59.917 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:59.912+0000 7f52a2986640 1 -- 172.21.15.43:0/1911687691 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f529c0c53c0 con 0x7f529c15d390 2023-08-09T13:53:59.917 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:59.913+0000 7f52a2986640 1 -- 172.21.15.43:0/1911687691 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f529c0c55b0 con 0x7f529c15d390 2023-08-09T13:53:59.917 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:59.914+0000 7f5298ff9640 1 -- 172.21.15.43:0/1911687691 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 57) v1 ==== 88460+0+0 (secure 0 0 0) 0x7f529400abd0 con 0x7f529c15d390 2023-08-09T13:53:59.918 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:59.915+0000 7f5298ff9640 1 --2- 172.21.15.43:0/1911687691 >> [v2:172.21.15.203:6832/2527059680,v1:172.21.15.203:6833/2527059680] conn(0x7f528806a1f0 0x7f528806c6b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:53:59.918 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:59.915+0000 7f5298ff9640 1 -- 172.21.15.43:0/1911687691 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(129..129 src has 1..129) v4 ==== 6426+0+0 (secure 0 0 0) 0x7f5294087ca0 con 0x7f529c15d390 2023-08-09T13:53:59.918 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:59.915+0000 7f529bfff640 1 -- 172.21.15.43:0/1911687691 >> [v2:172.21.15.203:6832/2527059680,v1:172.21.15.203:6833/2527059680] conn(0x7f528806a1f0 msgr2=0x7f528806c6b0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.203:6832/2527059680 2023-08-09T13:53:59.918 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:59.915+0000 7f529bfff640 1 --2- 172.21.15.43:0/1911687691 >> [v2:172.21.15.203:6832/2527059680,v1:172.21.15.203:6833/2527059680] conn(0x7f528806a1f0 0x7f528806c6b0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-08-09T13:53:59.926 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:53:59.923+0000 7f5298ff9640 1 -- 172.21.15.43:0/1911687691 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178066 (secure 0 0 0) 0x7f5294053100 con 0x7f529c15d390 2023-08-09T13:54:00.112 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:54:00.110+0000 7f52a2986640 1 -- 172.21.15.43:0/1911687691 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command([{prefix=config set, name=mgr/snap_schedule/dump_on_update}] v 0) v1 -- 0x7f529c0739f0 con 0x7f529c15d390 2023-08-09T13:54:00.118 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:54:00.116+0000 7f529bfff640 1 -- 172.21.15.43:0/1911687691 >> [v2:172.21.15.203:6832/2527059680,v1:172.21.15.203:6833/2527059680] conn(0x7f528806a1f0 msgr2=0x7f528806c6b0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.203:6832/2527059680 2023-08-09T13:54:00.118 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:54:00.116+0000 7f529bfff640 1 --2- 172.21.15.43:0/1911687691 >> [v2:172.21.15.203:6832/2527059680,v1:172.21.15.203:6833/2527059680] conn(0x7f528806a1f0 0x7f528806c6b0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2023-08-09T13:54:00.119 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:54:00.117+0000 7f5298ff9640 1 -- 172.21.15.43:0/1911687691 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mgr/snap_schedule/dump_on_update}]=0 v42)=0 v42) v1 ==== 133+0+0 (secure 0 0 0) 0x7f52940180c0 con 0x7f529c15d390 2023-08-09T13:54:00.123 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:54:00.120+0000 7f52a2986640 1 -- 172.21.15.43:0/1911687691 >> [v2:172.21.15.203:6832/2527059680,v1:172.21.15.203:6833/2527059680] conn(0x7f528806a1f0 msgr2=0x7f528806c6b0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-08-09T13:54:00.123 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:54:00.120+0000 7f52a2986640 1 --2- 172.21.15.43:0/1911687691 >> [v2:172.21.15.203:6832/2527059680,v1:172.21.15.203:6833/2527059680] conn(0x7f528806a1f0 0x7f528806c6b0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:54:00.123 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:54:00.120+0000 7f52a2986640 1 -- 172.21.15.43:0/1911687691 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f529c15d390 msgr2=0x7f529c0c4130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:54:00.124 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:54:00.120+0000 7f52a2986640 1 --2- 172.21.15.43:0/1911687691 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f529c15d390 0x7f529c0c4130 secure :-1 s=READY pgs=1558 cs=0 l=1 rev1=1 crypto rx=0x7f529400c700 tx=0x7f529400cbc0 comp rx=0 tx=0).stop 2023-08-09T13:54:00.124 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:54:00.121+0000 7f52a2986640 1 -- 172.21.15.43:0/1911687691 shutdown_connections 2023-08-09T13:54:00.124 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:54:00.121+0000 7f52a2986640 1 --2- 172.21.15.43:0/1911687691 >> [v2:172.21.15.203:6832/2527059680,v1:172.21.15.203:6833/2527059680] conn(0x7f528806a1f0 0x7f528806c6b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:54:00.124 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:54:00.121+0000 7f52a2986640 1 --2- 172.21.15.43:0/1911687691 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f529c15d390 0x7f529c0c4130 unknown :-1 s=CLOSED pgs=1558 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:54:00.124 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:54:00.121+0000 7f52a2986640 1 --2- 172.21.15.43:0/1911687691 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f529c15ca70 0x7f529c0c3bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:54:00.124 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:54:00.121+0000 7f52a2986640 1 --2- 172.21.15.43:0/1911687691 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f529c070cf0 0x7f529c15b5a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:54:00.124 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:54:00.121+0000 7f52a2986640 1 -- 172.21.15.43:0/1911687691 >> 172.21.15.43:0/1911687691 conn(0x7f529c0cf0b0 msgr2=0x7f529c0ac270 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-09T13:54:00.125 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:54:00.121+0000 7f52a2986640 1 -- 172.21.15.43:0/1911687691 shutdown_connections 2023-08-09T13:54:00.125 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:54:00.121+0000 7f52a2986640 1 -- 172.21.15.43:0/1911687691 wait complete. 2023-08-09T13:54:00.146 INFO:tasks.ceph.mgr.x.smithi203.stderr:2023-08-09T13:54:00.145+0000 7f0993c41040 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2023-08-09T13:54:00.148 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-09T13:54:00.149 DEBUG:teuthology.orchestra.run.smithi043:> (cd /home/ubuntu/cephtest/mnt.0 && exec mkdir -p snap_test_dir1) 2023-08-09T13:54:00.211 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs snap-schedule add --fs cephfs snap_test_dir1 1M 2023-08-09T13:54:00.229 INFO:tasks.ceph.mgr.y.smithi043.stderr:2023-08-09T13:54:00.226+0000 7f122d838040 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2023-08-09T13:54:00.266 INFO:tasks.ceph.mgr.x.smithi203.stderr:2023-08-09T13:54:00.264+0000 7f0993c41040 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2023-08-09T13:54:00.322 INFO:tasks.ceph.mgr.x.smithi203.stderr:2023-08-09T13:54:00.320+0000 7f0993c41040 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2023-08-09T13:54:00.357 INFO:tasks.ceph.mgr.y.smithi043.stderr:2023-08-09T13:54:00.354+0000 7f122d838040 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2023-08-09T13:54:00.371 INFO:tasks.ceph.mgr.x.smithi203.stderr:2023-08-09T13:54:00.370+0000 7f0993c41040 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2023-08-09T13:54:00.416 INFO:tasks.ceph.mgr.y.smithi043.stderr:2023-08-09T13:54:00.414+0000 7f122d838040 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2023-08-09T13:54:00.432 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:54:00.430+0000 7ffb252ec640 1 Processor -- start 2023-08-09T13:54:00.433 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:54:00.430+0000 7ffb252ec640 1 -- start start 2023-08-09T13:54:00.434 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:54:00.431+0000 7ffb252ec640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7ffb2015a530 0x7ffb2006eaa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:54:00.434 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:54:00.432+0000 7ffb252ec640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7ffb20069ed0 0x7ffb20163f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:54:00.435 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:54:00.432+0000 7ffb252ec640 1 -- --> v1:172.21.15.203:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ffb200c9730 con 0x7ffb2006d290 2023-08-09T13:54:00.435 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:54:00.432+0000 7ffb252ec640 1 -- --> v2:172.21.15.43:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ffb200ca590 con 0x7ffb20069ed0 2023-08-09T13:54:00.435 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:54:00.432+0000 7ffb252ec640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ffb200719c0 con 0x7ffb2015a530 2023-08-09T13:54:00.435 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:54:00.432+0000 7ffb1f7fe640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7ffb20069ed0 0x7ffb20163f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:54:00.435 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:54:00.432+0000 7ffb1f7fe640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7ffb20069ed0 0x7ffb20163f40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:34428/0 (socket says 172.21.15.43:34428) 2023-08-09T13:54:00.435 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:54:00.432+0000 7ffb1f7fe640 1 -- 172.21.15.43:0/1810438619 learned_addr learned my addr 172.21.15.43:0/1810438619 (peer_addr_for_me v2:172.21.15.43:0/0) 2023-08-09T13:54:00.435 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:54:00.432+0000 7ffb1effd640 1 --2- 172.21.15.43:0/1810438619 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7ffb2015a530 0x7ffb2006eaa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:54:00.436 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:54:00.433+0000 7ffb1f7fe640 1 -- 172.21.15.43:0/1810438619 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7ffb2015a530 msgr2=0x7ffb2006eaa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-09T13:54:00.436 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:54:00.433+0000 7ffb1f7fe640 1 --2- 172.21.15.43:0/1810438619 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7ffb2015a530 0x7ffb2006eaa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:54:00.436 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:54:00.433+0000 7ffb1f7fe640 1 -- 172.21.15.43:0/1810438619 >> v1:172.21.15.203:6789/0 conn(0x7ffb2006d290 legacy=0x7ffb200697c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:54:00.437 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:54:00.433+0000 7ffb1f7fe640 1 -- 172.21.15.43:0/1810438619 --> v2:172.21.15.43:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ffb200c9980 con 0x7ffb20069ed0 2023-08-09T13:54:00.437 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:54:00.434+0000 7ffb1f7fe640 1 --2- 172.21.15.43:0/1810438619 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ffb20069ed0 0x7ffb20163f40 secure :-1 s=READY pgs=1559 cs=0 l=1 rev1=1 crypto rx=0x7ffb18009e50 tx=0x7ffb1802bf60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1f7fc438fc584570 server_cookie=0 in_seq=0 out_seq=0 2023-08-09T13:54:00.437 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:54:00.435+0000 7ffb1dffb640 1 -- 172.21.15.43:0/1810438619 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffb1802cb00 con 0x7ffb20069ed0 2023-08-09T13:54:00.437 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:54:00.435+0000 7ffb1dffb640 1 -- 172.21.15.43:0/1810438619 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ffb1802cc60 con 0x7ffb20069ed0 2023-08-09T13:54:00.438 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:54:00.435+0000 7ffb1dffb640 1 -- 172.21.15.43:0/1810438619 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffb180343f0 con 0x7ffb20069ed0 2023-08-09T13:54:00.438 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:54:00.435+0000 7ffb252ec640 1 -- 172.21.15.43:0/1810438619 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ffb20069ed0 msgr2=0x7ffb20163f40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:54:00.439 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:54:00.435+0000 7ffb252ec640 1 --2- 172.21.15.43:0/1810438619 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ffb20069ed0 0x7ffb20163f40 secure :-1 s=READY pgs=1559 cs=0 l=1 rev1=1 crypto rx=0x7ffb18009e50 tx=0x7ffb1802bf60 comp rx=0 tx=0).stop 2023-08-09T13:54:00.439 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:54:00.435+0000 7ffb252ec640 1 -- 172.21.15.43:0/1810438619 shutdown_connections 2023-08-09T13:54:00.439 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:54:00.435+0000 7ffb252ec640 1 --2- 172.21.15.43:0/1810438619 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ffb20069ed0 0x7ffb20163f40 unknown :-1 s=CLOSED pgs=1559 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:54:00.439 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:54:00.435+0000 7ffb252ec640 1 --2- 172.21.15.43:0/1810438619 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7ffb2015a530 0x7ffb2006eaa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:54:00.439 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:54:00.435+0000 7ffb252ec640 1 -- 172.21.15.43:0/1810438619 >> 172.21.15.43:0/1810438619 conn(0x7ffb200cbb60 msgr2=0x7ffb200cbf70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-09T13:54:00.439 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:54:00.436+0000 7ffb252ec640 1 -- 172.21.15.43:0/1810438619 shutdown_connections 2023-08-09T13:54:00.439 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:54:00.436+0000 7ffb252ec640 1 -- 172.21.15.43:0/1810438619 wait complete. 2023-08-09T13:54:00.440 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:54:00.437+0000 7ffb252ec640 1 Processor -- start 2023-08-09T13:54:00.440 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:54:00.437+0000 7ffb252ec640 1 -- start start 2023-08-09T13:54:00.441 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:54:00.438+0000 7ffb252ec640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7ffb20069ed0 0x7ffb20152870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:54:00.441 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:54:00.438+0000 7ffb252ec640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ffb2006d290 0x7ffb200afcd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:54:00.441 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:54:00.438+0000 7ffb252ec640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7ffb2015a530 0x7ffb200c44b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:54:00.442 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:54:00.438+0000 7ffb252ec640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffb20152620 con 0x7ffb2006d290 2023-08-09T13:54:00.442 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:54:00.438+0000 7ffb252ec640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffb200b0360 con 0x7ffb20069ed0 2023-08-09T13:54:00.442 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:54:00.438+0000 7ffb252ec640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ffb200c3730 con 0x7ffb2015a530 2023-08-09T13:54:00.442 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:54:00.439+0000 7ffb1e7fc640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ffb2006d290 0x7ffb200afcd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:54:00.442 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:54:00.439+0000 7ffb1e7fc640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ffb2006d290 0x7ffb200afcd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:34432/0 (socket says 172.21.15.43:34432) 2023-08-09T13:54:00.442 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:54:00.439+0000 7ffb1e7fc640 1 -- 172.21.15.43:0/1625666684 learned_addr learned my addr 172.21.15.43:0/1625666684 (peer_addr_for_me v2:172.21.15.43:0/0) 2023-08-09T13:54:00.442 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:54:00.439+0000 7ffb1effd640 1 --2- 172.21.15.43:0/1625666684 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7ffb20069ed0 0x7ffb20152870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:54:00.443 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:54:00.439+0000 7ffb1e7fc640 1 -- 172.21.15.43:0/1625666684 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7ffb2015a530 msgr2=0x7ffb200c44b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:54:00.444 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:54:00.439+0000 7ffb1e7fc640 1 --2- 172.21.15.43:0/1625666684 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7ffb2015a530 0x7ffb200c44b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:54:00.444 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:54:00.439+0000 7ffb1e7fc640 1 -- 172.21.15.43:0/1625666684 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7ffb20069ed0 msgr2=0x7ffb20152870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:54:00.444 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:54:00.439+0000 7ffb1e7fc640 1 --2- 172.21.15.43:0/1625666684 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7ffb20069ed0 0x7ffb20152870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:54:00.444 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:54:00.439+0000 7ffb1e7fc640 1 -- 172.21.15.43:0/1625666684 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ffb18009b30 con 0x7ffb2006d290 2023-08-09T13:54:00.444 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:54:00.439+0000 7ffb1e7fc640 1 --2- 172.21.15.43:0/1625666684 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ffb2006d290 0x7ffb200afcd0 secure :-1 s=READY pgs=1560 cs=0 l=1 rev1=1 crypto rx=0x7ffb0800c400 tx=0x7ffb0800c8c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-09T13:54:00.444 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:54:00.440+0000 7ffb177fe640 1 -- 172.21.15.43:0/1625666684 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffb080052d0 con 0x7ffb2006d290 2023-08-09T13:54:00.444 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:54:00.440+0000 7ffb177fe640 1 -- 172.21.15.43:0/1625666684 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ffb08005430 con 0x7ffb2006d290 2023-08-09T13:54:00.445 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:54:00.440+0000 7ffb177fe640 1 -- 172.21.15.43:0/1625666684 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffb08003830 con 0x7ffb2006d290 2023-08-09T13:54:00.445 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:54:00.440+0000 7ffb252ec640 1 -- 172.21.15.43:0/1625666684 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ffb200c39d0 con 0x7ffb2006d290 2023-08-09T13:54:00.445 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:54:00.440+0000 7ffb252ec640 1 -- 172.21.15.43:0/1625666684 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ffb200c3f90 con 0x7ffb2006d290 2023-08-09T13:54:00.445 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:54:00.441+0000 7ffb252ec640 1 -- 172.21.15.43:0/1625666684 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ffb200c41b0 con 0x7ffb2006d290 2023-08-09T13:54:00.446 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:54:00.442+0000 7ffb177fe640 1 -- 172.21.15.43:0/1625666684 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 57) v1 ==== 88460+0+0 (secure 0 0 0) 0x7ffb08003990 con 0x7ffb2006d290 2023-08-09T13:54:00.446 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:54:00.443+0000 7ffb177fe640 1 --2- 172.21.15.43:0/1625666684 >> [v2:172.21.15.203:6832/2527059680,v1:172.21.15.203:6833/2527059680] conn(0x7ffb0006a1f0 0x7ffb0006c6b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:54:00.446 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:54:00.443+0000 7ffb177fe640 1 -- 172.21.15.43:0/1625666684 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(129..129 src has 1..129) v4 ==== 6426+0+0 (secure 0 0 0) 0x7ffb08087e90 con 0x7ffb2006d290 2023-08-09T13:54:00.446 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:54:00.443+0000 7ffb1effd640 1 -- 172.21.15.43:0/1625666684 >> [v2:172.21.15.203:6832/2527059680,v1:172.21.15.203:6833/2527059680] conn(0x7ffb0006a1f0 msgr2=0x7ffb0006c6b0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.203:6832/2527059680 2023-08-09T13:54:00.446 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:54:00.443+0000 7ffb1effd640 1 --2- 172.21.15.43:0/1625666684 >> [v2:172.21.15.203:6832/2527059680,v1:172.21.15.203:6833/2527059680] conn(0x7ffb0006a1f0 0x7ffb0006c6b0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-08-09T13:54:00.453 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:54:00.451+0000 7ffb177fe640 1 -- 172.21.15.43:0/1625666684 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178066 (secure 0 0 0) 0x7ffb080532f0 con 0x7ffb2006d290 2023-08-09T13:54:00.475 INFO:tasks.ceph.mgr.y.smithi043.stderr:2023-08-09T13:54:00.472+0000 7f122d838040 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2023-08-09T13:54:00.646 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:54:00.644+0000 7ffb1effd640 1 -- 172.21.15.43:0/1625666684 >> [v2:172.21.15.203:6832/2527059680,v1:172.21.15.203:6833/2527059680] conn(0x7ffb0006a1f0 msgr2=0x7ffb0006c6b0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.203:6832/2527059680 2023-08-09T13:54:00.647 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:54:00.644+0000 7ffb1effd640 1 --2- 172.21.15.43:0/1625666684 >> [v2:172.21.15.203:6832/2527059680,v1:172.21.15.203:6833/2527059680] conn(0x7ffb0006a1f0 0x7ffb0006c6b0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2023-08-09T13:54:00.670 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:54:00.667+0000 7ffb252ec640 1 -- 172.21.15.43:0/1625666684 --> [v2:172.21.15.203:6832/2527059680,v1:172.21.15.203:6833/2527059680] -- mgr_command(tid 0: {"prefix": "fs snap-schedule add", "fs": "cephfs", "path": "snap_test_dir1", "snap_schedule": "1M", "target": ["mon-mgr", ""]}) v1 -- 0x7ffb200c9f40 con 0x7ffb0006a1f0 2023-08-09T13:54:01.047 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:54:01.045+0000 7ffb1effd640 1 -- 172.21.15.43:0/1625666684 >> [v2:172.21.15.203:6832/2527059680,v1:172.21.15.203:6833/2527059680] conn(0x7ffb0006a1f0 msgr2=0x7ffb0006c6b0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.203:6832/2527059680 2023-08-09T13:54:01.048 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:54:01.045+0000 7ffb1effd640 1 --2- 172.21.15.43:0/1625666684 >> [v2:172.21.15.203:6832/2527059680,v1:172.21.15.203:6833/2527059680] conn(0x7ffb0006a1f0 0x7ffb0006c6b0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2023-08-09T13:54:01.057 INFO:tasks.ceph.mgr.x.smithi203.stderr:2023-08-09T13:54:01.055+0000 7f0993c41040 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2023-08-09T13:54:01.118 INFO:tasks.ceph.mgr.x.smithi203.stderr:2023-08-09T13:54:01.115+0000 7f0993c41040 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2023-08-09T13:54:01.167 INFO:tasks.ceph.mgr.x.smithi203.stderr:2023-08-09T13:54:01.165+0000 7f0993c41040 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2023-08-09T13:54:01.226 INFO:tasks.ceph.mgr.y.smithi043.stderr:2023-08-09T13:54:01.223+0000 7f122d838040 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2023-08-09T13:54:01.285 INFO:tasks.ceph.mgr.x.smithi203.stderr:2023-08-09T13:54:01.282+0000 7f0993c41040 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2023-08-09T13:54:01.290 INFO:tasks.ceph.mgr.y.smithi043.stderr:2023-08-09T13:54:01.287+0000 7f122d838040 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2023-08-09T13:54:01.342 INFO:tasks.ceph.mgr.y.smithi043.stderr:2023-08-09T13:54:01.340+0000 7f122d838040 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2023-08-09T13:54:01.442 INFO:tasks.ceph.mgr.x.smithi203.stderr:2023-08-09T13:54:01.439+0000 7f0993c41040 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2023-08-09T13:54:01.466 INFO:tasks.ceph.mgr.y.smithi043.stderr:2023-08-09T13:54:01.464+0000 7f122d838040 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2023-08-09T13:54:01.548 INFO:tasks.ceph.mgr.x.smithi203.stderr:2023-08-09T13:54:01.546+0000 7f0993c41040 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2023-08-09T13:54:01.597 INFO:tasks.ceph.mgr.x.smithi203.stderr:2023-08-09T13:54:01.595+0000 7f0993c41040 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2023-08-09T13:54:01.630 INFO:tasks.ceph.mgr.y.smithi043.stderr:2023-08-09T13:54:01.628+0000 7f122d838040 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2023-08-09T13:54:01.743 INFO:tasks.ceph.mgr.y.smithi043.stderr:2023-08-09T13:54:01.741+0000 7f122d838040 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2023-08-09T13:54:01.756 INFO:tasks.ceph.mgr.x.smithi203.stderr:2023-08-09T13:54:01.754+0000 7f0993c41040 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2023-08-09T13:54:01.795 INFO:tasks.ceph.mgr.y.smithi043.stderr:2023-08-09T13:54:01.792+0000 7f122d838040 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2023-08-09T13:54:01.849 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:54:01.846+0000 7ffb1effd640 1 -- 172.21.15.43:0/1625666684 >> [v2:172.21.15.203:6832/2527059680,v1:172.21.15.203:6833/2527059680] conn(0x7ffb0006a1f0 msgr2=0x7ffb0006c6b0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.203:6832/2527059680 2023-08-09T13:54:01.849 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:54:01.846+0000 7ffb1effd640 1 --2- 172.21.15.43:0/1625666684 >> [v2:172.21.15.203:6832/2527059680,v1:172.21.15.203:6833/2527059680] conn(0x7ffb0006a1f0 0x7ffb0006c6b0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2023-08-09T13:54:01.963 INFO:tasks.ceph.mgr.y.smithi043.stderr:2023-08-09T13:54:01.960+0000 7f122d838040 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2023-08-09T13:54:02.137 INFO:tasks.ceph.mgr.x.smithi203.stderr:2023-08-09T13:54:02.135+0000 7f0993c41040 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2023-08-09T13:54:02.360 INFO:tasks.ceph.mgr.x.smithi203.stderr:2023-08-09T13:54:02.358+0000 7f0993c41040 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2023-08-09T13:54:02.369 INFO:tasks.ceph.mgr.y.smithi043.stderr:2023-08-09T13:54:02.366+0000 7f122d838040 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2023-08-09T13:54:02.418 INFO:tasks.ceph.mgr.x.smithi203.stderr:2023-08-09T13:54:02.415+0000 7f0993c41040 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2023-08-09T13:54:02.468 INFO:tasks.ceph.mgr.x.smithi203.stderr:2023-08-09T13:54:02.466+0000 7f0993c41040 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2023-08-09T13:54:02.520 INFO:tasks.ceph.mgr.x.smithi203.stderr:2023-08-09T13:54:02.517+0000 7f0993c41040 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2023-08-09T13:54:02.599 INFO:tasks.ceph.mgr.y.smithi043.stderr:2023-08-09T13:54:02.596+0000 7f122d838040 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2023-08-09T13:54:02.659 INFO:tasks.ceph.mgr.y.smithi043.stderr:2023-08-09T13:54:02.656+0000 7f122d838040 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2023-08-09T13:54:02.711 INFO:tasks.ceph.mgr.y.smithi043.stderr:2023-08-09T13:54:02.708+0000 7f122d838040 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2023-08-09T13:54:02.766 INFO:tasks.ceph.mgr.y.smithi043.stderr:2023-08-09T13:54:02.763+0000 7f122d838040 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2023-08-09T13:54:02.815 INFO:tasks.ceph.mgr.x.smithi203.stderr:2023-08-09T13:54:02.812+0000 7f0993c41040 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2023-08-09T13:54:02.872 INFO:tasks.ceph.mgr.x.smithi203.stderr:2023-08-09T13:54:02.870+0000 7f0993c41040 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2023-08-09T13:54:02.924 INFO:tasks.ceph.mgr.x.smithi203.stderr:2023-08-09T13:54:02.922+0000 7f0993c41040 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2023-08-09T13:54:03.077 INFO:tasks.ceph.mgr.y.smithi043.stderr:2023-08-09T13:54:03.075+0000 7f122d838040 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2023-08-09T13:54:03.084 INFO:tasks.ceph.mgr.x.smithi203.stderr:2023-08-09T13:54:03.081+0000 7f0993c41040 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2023-08-09T13:54:03.133 INFO:tasks.ceph.mgr.x.smithi203.stderr:2023-08-09T13:54:03.130+0000 7f0993c41040 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2023-08-09T13:54:03.138 INFO:tasks.ceph.mgr.y.smithi043.stderr:2023-08-09T13:54:03.136+0000 7f122d838040 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2023-08-09T13:54:03.185 INFO:tasks.ceph.mgr.x.smithi203.stderr:2023-08-09T13:54:03.183+0000 7f0993c41040 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2023-08-09T13:54:03.193 INFO:tasks.ceph.mgr.y.smithi043.stderr:2023-08-09T13:54:03.190+0000 7f122d838040 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2023-08-09T13:54:03.235 INFO:tasks.ceph.mgr.x.smithi203.stderr:2023-08-09T13:54:03.233+0000 7f0993c41040 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2023-08-09T13:54:03.264 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:54:03.262+0000 7ffb177fe640 1 -- 172.21.15.43:0/1625666684 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mgrmap(e 58) v1 ==== 88069+0+0 (secure 0 0 0) 0x7ffb080536c0 con 0x7ffb2006d290 2023-08-09T13:54:03.265 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:54:03.262+0000 7ffb177fe640 1 -- 172.21.15.43:0/1625666684 >> [v2:172.21.15.203:6832/2527059680,v1:172.21.15.203:6833/2527059680] conn(0x7ffb0006a1f0 msgr2=0x7ffb0006c6b0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-08-09T13:54:03.265 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:54:03.262+0000 7ffb177fe640 1 --2- 172.21.15.43:0/1625666684 >> [v2:172.21.15.203:6832/2527059680,v1:172.21.15.203:6833/2527059680] conn(0x7ffb0006a1f0 0x7ffb0006c6b0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:54:03.378 INFO:tasks.ceph.mgr.y.smithi043.stderr:2023-08-09T13:54:03.375+0000 7f122d838040 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2023-08-09T13:54:03.403 INFO:tasks.ceph.mgr.x.smithi203.stderr:2023-08-09T13:54:03.401+0000 7f090ffaf640 -1 client.0 error registering admin socket command: (17) File exists 2023-08-09T13:54:03.404 INFO:tasks.ceph.mgr.x.smithi203.stderr:2023-08-09T13:54:03.401+0000 7f090ffaf640 -1 client.0 error registering admin socket command: (17) File exists 2023-08-09T13:54:03.404 INFO:tasks.ceph.mgr.x.smithi203.stderr:2023-08-09T13:54:03.401+0000 7f090ffaf640 -1 client.0 error registering admin socket command: (17) File exists 2023-08-09T13:54:03.404 INFO:tasks.ceph.mgr.x.smithi203.stderr:2023-08-09T13:54:03.401+0000 7f090ffaf640 -1 client.0 error registering admin socket command: (17) File exists 2023-08-09T13:54:03.404 INFO:tasks.ceph.mgr.x.smithi203.stderr:2023-08-09T13:54:03.401+0000 7f090ffaf640 -1 client.0 error registering admin socket command: (17) File exists 2023-08-09T13:54:03.404 INFO:tasks.ceph.mgr.x.smithi203.stderr:2023-08-09T13:54:03.401+0000 7f0912fb5640 -1 client.0 error registering admin socket command: (17) File exists 2023-08-09T13:54:03.404 INFO:tasks.ceph.mgr.x.smithi203.stderr:2023-08-09T13:54:03.401+0000 7f0912fb5640 -1 client.0 error registering admin socket command: (17) File exists 2023-08-09T13:54:03.404 INFO:tasks.ceph.mgr.x.smithi203.stderr:2023-08-09T13:54:03.401+0000 7f0912fb5640 -1 client.0 error registering admin socket command: (17) File exists 2023-08-09T13:54:03.405 INFO:tasks.ceph.mgr.x.smithi203.stderr:2023-08-09T13:54:03.401+0000 7f0912fb5640 -1 client.0 error registering admin socket command: (17) File exists 2023-08-09T13:54:03.405 INFO:tasks.ceph.mgr.x.smithi203.stderr:2023-08-09T13:54:03.401+0000 7f0912fb5640 -1 client.0 error registering admin socket command: (17) File exists 2023-08-09T13:54:03.432 INFO:tasks.ceph.mgr.y.smithi043.stderr:2023-08-09T13:54:03.429+0000 7f122d838040 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2023-08-09T13:54:03.487 INFO:tasks.ceph.mgr.y.smithi043.stderr:2023-08-09T13:54:03.485+0000 7f122d838040 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2023-08-09T13:54:03.540 INFO:tasks.ceph.mgr.y.smithi043.stderr:2023-08-09T13:54:03.537+0000 7f122d838040 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2023-08-09T13:54:04.272 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:54:04.270+0000 7ffb177fe640 1 -- 172.21.15.43:0/1625666684 <== mon.0 v2:172.21.15.43:3300/0 8 ==== mgrmap(e 59) v1 ==== 88196+0+0 (secure 0 0 0) 0x7ffb08086040 con 0x7ffb2006d290 2023-08-09T13:54:04.274 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:54:04.271+0000 7ffb177fe640 1 --2- 172.21.15.43:0/1625666684 >> [v2:172.21.15.203:6832/1796635339,v1:172.21.15.203:6833/1796635339] conn(0x7ffb0001c9f0 0x7ffb00074cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:54:04.274 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:54:04.271+0000 7ffb177fe640 1 -- 172.21.15.43:0/1625666684 --> [v2:172.21.15.203:6832/1796635339,v1:172.21.15.203:6833/1796635339] -- mgr_command(tid 0: {"prefix": "fs snap-schedule add", "fs": "cephfs", "path": "snap_test_dir1", "snap_schedule": "1M", "target": ["mon-mgr", ""]}) v1 -- 0x7ffb0000f810 con 0x7ffb0001c9f0 2023-08-09T13:54:04.281 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:54:04.279+0000 7ffb1effd640 1 --2- 172.21.15.43:0/1625666684 >> [v2:172.21.15.203:6832/1796635339,v1:172.21.15.203:6833/1796635339] conn(0x7ffb0001c9f0 0x7ffb00074cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:54:04.290 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:54:04.288+0000 7ffb1effd640 1 --2- 172.21.15.43:0/1625666684 >> [v2:172.21.15.203:6832/1796635339,v1:172.21.15.203:6833/1796635339] conn(0x7ffb0001c9f0 0x7ffb00074cf0 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7ffb10006fd0 tx=0x7ffb10008040 comp rx=0 tx=0).ready entity=mgr.7288 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-09T13:54:04.426 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:54:04.423+0000 7ffb177fe640 1 -- 172.21.15.43:0/1625666684 <== mgr.7288 v2:172.21.15.203:6832/1796635339 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+37 (secure 0 0 0) 0x7ffb10004040 con 0x7ffb0001c9f0 2023-08-09T13:54:04.426 INFO:teuthology.orchestra.run.smithi043.stdout:Schedule set for path snap_test_dir1 2023-08-09T13:54:04.430 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:54:04.427+0000 7ffb252ec640 1 -- 172.21.15.43:0/1625666684 >> [v2:172.21.15.203:6832/1796635339,v1:172.21.15.203:6833/1796635339] conn(0x7ffb0001c9f0 msgr2=0x7ffb00074cf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:54:04.430 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:54:04.427+0000 7ffb252ec640 1 --2- 172.21.15.43:0/1625666684 >> [v2:172.21.15.203:6832/1796635339,v1:172.21.15.203:6833/1796635339] conn(0x7ffb0001c9f0 0x7ffb00074cf0 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7ffb10006fd0 tx=0x7ffb10008040 comp rx=0 tx=0).stop 2023-08-09T13:54:04.430 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:54:04.427+0000 7ffb252ec640 1 -- 172.21.15.43:0/1625666684 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ffb2006d290 msgr2=0x7ffb200afcd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:54:04.431 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:54:04.427+0000 7ffb252ec640 1 --2- 172.21.15.43:0/1625666684 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ffb2006d290 0x7ffb200afcd0 secure :-1 s=READY pgs=1560 cs=0 l=1 rev1=1 crypto rx=0x7ffb0800c400 tx=0x7ffb0800c8c0 comp rx=0 tx=0).stop 2023-08-09T13:54:04.431 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:54:04.427+0000 7ffb1f7fe640 1 -- 172.21.15.43:0/1625666684 reap_dead start 2023-08-09T13:54:04.431 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:54:04.427+0000 7ffb252ec640 1 -- 172.21.15.43:0/1625666684 shutdown_connections 2023-08-09T13:54:04.431 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:54:04.427+0000 7ffb252ec640 1 -- 172.21.15.43:0/1625666684 >> 172.21.15.43:0/1625666684 conn(0x7ffb200cbb60 msgr2=0x7ffb20159530 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-09T13:54:04.432 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:54:04.427+0000 7ffb252ec640 1 -- 172.21.15.43:0/1625666684 shutdown_connections 2023-08-09T13:54:04.432 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:54:04.427+0000 7ffb252ec640 1 -- 172.21.15.43:0/1625666684 wait complete. 2023-08-09T13:54:04.452 DEBUG:tasks.cephfs.test_snap_schedules:res=Schedule set for path snap_test_dir1 2023-08-09T13:54:04.452 DEBUG:tasks.cephfs.test_snap_schedules:expecting snap snap_test_dir1/.snap/scheduled-2023-08-09-13_55_00 in ~56s... 2023-08-09T13:54:04.452 DEBUG:tasks.cephfs.test_snap_schedules:expected_schedule: ['1M'], expected_retention: [] 2023-08-09T13:54:04.453 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs snap-schedule list --fs cephfs --format json snap_test_dir1 2023-08-09T13:54:04.738 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:54:04.735+0000 7f0a510ad640 1 Processor -- start 2023-08-09T13:54:04.738 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:54:04.736+0000 7f0a510ad640 1 -- start start 2023-08-09T13:54:04.741 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:54:04.737+0000 7f0a510ad640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f0a4c160fe0 0x7f0a4c0b3ec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:54:04.741 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:54:04.737+0000 7f0a510ad640 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7f0a4c0b4400 0x7f0a4c0d0520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:54:04.741 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:54:04.737+0000 7f0a510ad640 1 -- --> v1:172.21.15.43:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0a4c0ccd30 con 0x7f0a4c15a510 2023-08-09T13:54:04.742 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:54:04.737+0000 7f0a510ad640 1 -- --> v2:172.21.15.43:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0a4c15a8f0 con 0x7f0a4c160fe0 2023-08-09T13:54:04.742 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:54:04.737+0000 7f0a510ad640 1 -- --> v2:172.21.15.203:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0a4c0b4870 con 0x7f0a4c0b4400 2023-08-09T13:54:04.742 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:54:04.738+0000 7f0a4ad9d640 1 --1- >> v1:172.21.15.43:6789/0 conn(0x7f0a4c15a510 0x7f0a4c15d5e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.43:6789/0 says I am v1:172.21.15.43:48592/0 (socket says 172.21.15.43:48592) 2023-08-09T13:54:04.742 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:54:04.738+0000 7f0a4ad9d640 1 -- 172.21.15.43:0/1263904517 learned_addr learned my addr 172.21.15.43:0/1263904517 (peer_addr_for_me v1:172.21.15.43:0/0) 2023-08-09T13:54:04.743 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:54:04.738+0000 7f0a4a59c640 1 --2- 172.21.15.43:0/1263904517 >> v2:172.21.15.43:3300/0 conn(0x7f0a4c160fe0 0x7f0a4c0b3ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:54:04.743 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:54:04.738+0000 7f0a4b59e640 1 --2- 172.21.15.43:0/1263904517 >> v2:172.21.15.203:3300/0 conn(0x7f0a4c0b4400 0x7f0a4c0d0520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:54:04.743 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:54:04.738+0000 7f0a49d9b640 1 -- 172.21.15.43:0/1263904517 <== mon.0 v1:172.21.15.43:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1285204725 0 0) 0x7f0a4c0ccd30 con 0x7f0a4c15a510 2023-08-09T13:54:04.744 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:54:04.738+0000 7f0a49d9b640 1 -- 172.21.15.43:0/1263904517 --> v1:172.21.15.43:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f0a34003520 con 0x7f0a4c15a510 2023-08-09T13:54:04.744 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:54:04.738+0000 7f0a4a59c640 1 -- 172.21.15.43:0/1263904517 >> v2:172.21.15.203:3300/0 conn(0x7f0a4c0b4400 msgr2=0x7f0a4c0d0520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-09T13:54:04.744 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:54:04.738+0000 7f0a4a59c640 1 --2- 172.21.15.43:0/1263904517 >> v2:172.21.15.203:3300/0 conn(0x7f0a4c0b4400 0x7f0a4c0d0520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:54:04.744 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:54:04.738+0000 7f0a4a59c640 1 -- 172.21.15.43:0/1263904517 >> v1:172.21.15.43:6789/0 conn(0x7f0a4c15a510 legacy=0x7f0a4c15d5e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:54:04.745 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:54:04.738+0000 7f0a4a59c640 1 -- 172.21.15.43:0/1263904517 --> v2:172.21.15.43:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0a4c15dcf0 con 0x7f0a4c160fe0 2023-08-09T13:54:04.746 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:54:04.740+0000 7f0a4a59c640 1 --2- 172.21.15.43:0/1263904517 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f0a4c160fe0 0x7f0a4c0b3ec0 secure :-1 s=READY pgs=1562 cs=0 l=1 rev1=1 crypto rx=0x7f0a4c0c94a0 tx=0x7f0a4002b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=47e822a892d78e90 server_cookie=0 in_seq=0 out_seq=0 2023-08-09T13:54:04.747 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:54:04.740+0000 7f0a49d9b640 1 -- 172.21.15.43:0/1263904517 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0a40031830 con 0x7f0a4c160fe0 2023-08-09T13:54:04.747 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:54:04.740+0000 7f0a49d9b640 1 -- 172.21.15.43:0/1263904517 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0a40031990 con 0x7f0a4c160fe0 2023-08-09T13:54:04.747 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:54:04.740+0000 7f0a49d9b640 1 -- 172.21.15.43:0/1263904517 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0a40031d20 con 0x7f0a4c160fe0 2023-08-09T13:54:04.747 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:54:04.740+0000 7f0a510ad640 1 -- 172.21.15.43:0/1263904517 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f0a4c160fe0 msgr2=0x7f0a4c0b3ec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:54:04.747 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:54:04.740+0000 7f0a510ad640 1 --2- 172.21.15.43:0/1263904517 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f0a4c160fe0 0x7f0a4c0b3ec0 secure :-1 s=READY pgs=1562 cs=0 l=1 rev1=1 crypto rx=0x7f0a4c0c94a0 tx=0x7f0a4002b8d0 comp rx=0 tx=0).stop 2023-08-09T13:54:04.748 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:54:04.741+0000 7f0a510ad640 1 -- 172.21.15.43:0/1263904517 shutdown_connections 2023-08-09T13:54:04.748 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:54:04.741+0000 7f0a510ad640 1 --2- 172.21.15.43:0/1263904517 >> v2:172.21.15.203:3300/0 conn(0x7f0a4c0b4400 0x7f0a4c0d0520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:54:04.748 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:54:04.741+0000 7f0a510ad640 1 --2- 172.21.15.43:0/1263904517 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f0a4c160fe0 0x7f0a4c0b3ec0 unknown :-1 s=CLOSED pgs=1562 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:54:04.748 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:54:04.741+0000 7f0a510ad640 1 -- 172.21.15.43:0/1263904517 >> 172.21.15.43:0/1263904517 conn(0x7f0a4c0ceec0 msgr2=0x7f0a4c0cf2d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-09T13:54:04.748 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:54:04.741+0000 7f0a510ad640 1 -- 172.21.15.43:0/1263904517 shutdown_connections 2023-08-09T13:54:04.748 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:54:04.741+0000 7f0a510ad640 1 -- 172.21.15.43:0/1263904517 wait complete. 2023-08-09T13:54:04.748 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:54:04.743+0000 7f0a510ad640 1 Processor -- start 2023-08-09T13:54:04.749 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:54:04.743+0000 7f0a510ad640 1 -- start start 2023-08-09T13:54:04.749 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:54:04.744+0000 7f0a510ad640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f0a4c0b4400 0x7f0a4c0c7b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:54:04.749 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:54:04.744+0000 7f0a510ad640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f0a4c15a510 0x7f0a4c1567f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:54:04.749 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:54:04.744+0000 7f0a510ad640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f0a4c160fe0 0x7f0a4c15f200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:54:04.749 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:54:04.744+0000 7f0a510ad640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0a4c0c9c50 con 0x7f0a4c160fe0 2023-08-09T13:54:04.749 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:54:04.744+0000 7f0a510ad640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0a4c0c8430 con 0x7f0a4c0b4400 2023-08-09T13:54:04.750 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:54:04.744+0000 7f0a510ad640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0a4c0c8710 con 0x7f0a4c15a510 2023-08-09T13:54:04.750 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:54:04.744+0000 7f0a4b59e640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f0a4c160fe0 0x7f0a4c15f200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:54:04.750 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:54:04.744+0000 7f0a4b59e640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f0a4c160fe0 0x7f0a4c15f200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:34448/0 (socket says 172.21.15.43:34448) 2023-08-09T13:54:04.750 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:54:04.744+0000 7f0a4b59e640 1 -- 172.21.15.43:0/29025917 learned_addr learned my addr 172.21.15.43:0/29025917 (peer_addr_for_me v2:172.21.15.43:0/0) 2023-08-09T13:54:04.751 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:54:04.745+0000 7f0a4b59e640 1 -- 172.21.15.43:0/29025917 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f0a4c15a510 msgr2=0x7f0a4c1567f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:54:04.751 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:54:04.745+0000 7f0a4b59e640 1 --2- 172.21.15.43:0/29025917 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f0a4c15a510 0x7f0a4c1567f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:54:04.752 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:54:04.745+0000 7f0a4b59e640 1 -- 172.21.15.43:0/29025917 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f0a4c0b4400 msgr2=0x7f0a4c0c7b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:54:04.752 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:54:04.745+0000 7f0a4b59e640 1 --2- 172.21.15.43:0/29025917 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f0a4c0b4400 0x7f0a4c0c7b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:54:04.752 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:54:04.745+0000 7f0a4b59e640 1 -- 172.21.15.43:0/29025917 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0a40009400 con 0x7f0a4c160fe0 2023-08-09T13:54:04.752 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:54:04.745+0000 7f0a4b59e640 1 --2- 172.21.15.43:0/29025917 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f0a4c160fe0 0x7f0a4c15f200 secure :-1 s=READY pgs=1563 cs=0 l=1 rev1=1 crypto rx=0x7f0a44007e60 tx=0x7f0a4400e4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-09T13:54:04.752 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:54:04.745+0000 7f0a337fe640 1 -- 172.21.15.43:0/29025917 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0a44014420 con 0x7f0a4c160fe0 2023-08-09T13:54:04.752 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:54:04.745+0000 7f0a337fe640 1 -- 172.21.15.43:0/29025917 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0a440145a0 con 0x7f0a4c160fe0 2023-08-09T13:54:04.753 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:54:04.745+0000 7f0a337fe640 1 -- 172.21.15.43:0/29025917 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0a44014950 con 0x7f0a4c160fe0 2023-08-09T13:54:04.753 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:54:04.745+0000 7f0a510ad640 1 -- 172.21.15.43:0/29025917 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0a4c0c89f0 con 0x7f0a4c160fe0 2023-08-09T13:54:04.753 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:54:04.745+0000 7f0a510ad640 1 -- 172.21.15.43:0/29025917 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0a4c0c8090 con 0x7f0a4c160fe0 2023-08-09T13:54:04.753 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:54:04.746+0000 7f0a510ad640 1 -- 172.21.15.43:0/29025917 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0a4c0739f0 con 0x7f0a4c160fe0 2023-08-09T13:54:04.753 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:54:04.749+0000 7f0a337fe640 1 -- 172.21.15.43:0/29025917 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 59) v1 ==== 88196+0+0 (secure 0 0 0) 0x7f0a4403ab80 con 0x7f0a4c160fe0 2023-08-09T13:54:04.754 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:54:04.749+0000 7f0a337fe640 1 --2- 172.21.15.43:0/29025917 >> [v2:172.21.15.203:6832/1796635339,v1:172.21.15.203:6833/1796635339] conn(0x7f0a38069ef0 0x7f0a3806c3b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:54:04.754 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:54:04.749+0000 7f0a337fe640 1 -- 172.21.15.43:0/29025917 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(130..130 src has 1..130) v4 ==== 6770+0+0 (secure 0 0 0) 0x7f0a44089420 con 0x7f0a4c160fe0 2023-08-09T13:54:04.754 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:54:04.750+0000 7f0a4ad9d640 1 --2- 172.21.15.43:0/29025917 >> [v2:172.21.15.203:6832/1796635339,v1:172.21.15.203:6833/1796635339] conn(0x7f0a38069ef0 0x7f0a3806c3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:54:04.754 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:54:04.751+0000 7f0a4ad9d640 1 --2- 172.21.15.43:0/29025917 >> [v2:172.21.15.203:6832/1796635339,v1:172.21.15.203:6833/1796635339] conn(0x7f0a38069ef0 0x7f0a3806c3b0 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7f0a3c001610 tx=0x7f0a3c007930 comp rx=0 tx=0).ready entity=mgr.7288 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-09T13:54:04.755 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:54:04.753+0000 7f0a337fe640 1 -- 172.21.15.43:0/29025917 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178066 (secure 0 0 0) 0x7f0a440585c0 con 0x7f0a4c160fe0 2023-08-09T13:54:04.846 DEBUG:teuthology.orchestra.run.smithi043:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-08-09T13:54:04.854 DEBUG:teuthology.orchestra.run.smithi203:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-08-09T13:54:04.976 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:54:04.973+0000 7f0a510ad640 1 -- 172.21.15.43:0/29025917 --> [v2:172.21.15.203:6832/1796635339,v1:172.21.15.203:6833/1796635339] -- mgr_command(tid 0: {"prefix": "fs snap-schedule list", "fs": "cephfs", "path": "snap_test_dir1", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f0a4c0c8220 con 0x7f0a38069ef0 2023-08-09T13:54:04.992 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:54:04.990+0000 7f0a337fe640 1 -- 172.21.15.43:0/29025917 <== mgr.7288 v2:172.21.15.203:6832/1796635339 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+66 (secure 0 0 0) 0x7f0a4c0c8220 con 0x7f0a38069ef0 2023-08-09T13:54:04.993 INFO:teuthology.orchestra.run.smithi043.stdout: 2023-08-09T13:54:04.993 INFO:teuthology.orchestra.run.smithi043.stdout:{"path": "snap_test_dir1", "schedule": ["1M"], "retention": [{}]} 2023-08-09T13:54:04.995 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:54:04.992+0000 7f0a510ad640 1 -- 172.21.15.43:0/29025917 >> [v2:172.21.15.203:6832/1796635339,v1:172.21.15.203:6833/1796635339] conn(0x7f0a38069ef0 msgr2=0x7f0a3806c3b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:54:04.995 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:54:04.992+0000 7f0a510ad640 1 --2- 172.21.15.43:0/29025917 >> [v2:172.21.15.203:6832/1796635339,v1:172.21.15.203:6833/1796635339] conn(0x7f0a38069ef0 0x7f0a3806c3b0 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7f0a3c001610 tx=0x7f0a3c007930 comp rx=0 tx=0).stop 2023-08-09T13:54:04.995 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:54:04.992+0000 7f0a510ad640 1 -- 172.21.15.43:0/29025917 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f0a4c160fe0 msgr2=0x7f0a4c15f200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:54:04.996 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:54:04.992+0000 7f0a510ad640 1 --2- 172.21.15.43:0/29025917 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f0a4c160fe0 0x7f0a4c15f200 secure :-1 s=READY pgs=1563 cs=0 l=1 rev1=1 crypto rx=0x7f0a44007e60 tx=0x7f0a4400e4b0 comp rx=0 tx=0).stop 2023-08-09T13:54:04.996 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:54:04.993+0000 7f0a510ad640 1 -- 172.21.15.43:0/29025917 shutdown_connections 2023-08-09T13:54:04.996 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:54:04.993+0000 7f0a510ad640 1 --2- 172.21.15.43:0/29025917 >> [v2:172.21.15.203:6832/1796635339,v1:172.21.15.203:6833/1796635339] conn(0x7f0a38069ef0 0x7f0a3806c3b0 unknown :-1 s=CLOSED pgs=23 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:54:04.996 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:54:04.993+0000 7f0a510ad640 1 --2- 172.21.15.43:0/29025917 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f0a4c160fe0 0x7f0a4c15f200 unknown :-1 s=CLOSED pgs=1563 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:54:04.996 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:54:04.993+0000 7f0a510ad640 1 --2- 172.21.15.43:0/29025917 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f0a4c15a510 0x7f0a4c1567f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:54:04.996 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:54:04.993+0000 7f0a510ad640 1 --2- 172.21.15.43:0/29025917 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f0a4c0b4400 0x7f0a4c0c7b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:54:04.996 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:54:04.993+0000 7f0a510ad640 1 -- 172.21.15.43:0/29025917 >> 172.21.15.43:0/29025917 conn(0x7f0a4c0ceec0 msgr2=0x7f0a4c06aa80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-09T13:54:04.997 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:54:04.993+0000 7f0a510ad640 1 -- 172.21.15.43:0/29025917 shutdown_connections 2023-08-09T13:54:04.997 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:54:04.993+0000 7f0a510ad640 1 -- 172.21.15.43:0/29025917 wait complete. 2023-08-09T13:54:05.015 DEBUG:tasks.cephfs.test_snap_schedules:res= {"path": "snap_test_dir1", "schedule": ["1M"], "retention": [{}]} 2023-08-09T13:54:05.015 DEBUG:tasks.cephfs.test_snap_schedules:json_res: {'path': 'snap_test_dir1', 'schedule': ['1M'], 'retention': [{}]} 2023-08-09T13:54:05.016 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-09T13:54:05.016 DEBUG:teuthology.orchestra.run.smithi043:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-08-09T13:54:05.126 INFO:tasks.cephfs.test_snap_schedules:snapshots: set() 2023-08-09T13:54:05.127 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-08-09T13:54:05.127 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-08-09T13:54:06.129 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-09T13:54:06.129 DEBUG:teuthology.orchestra.run.smithi043:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-08-09T13:54:06.204 INFO:tasks.cephfs.test_snap_schedules:snapshots: set() 2023-08-09T13:54:06.204 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-08-09T13:54:06.204 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-08-09T13:54:07.206 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-09T13:54:07.206 DEBUG:teuthology.orchestra.run.smithi043:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-08-09T13:54:07.285 INFO:tasks.cephfs.test_snap_schedules:snapshots: set() 2023-08-09T13:54:07.286 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-08-09T13:54:07.286 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-08-09T13:54:08.287 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-09T13:54:08.288 DEBUG:teuthology.orchestra.run.smithi043:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-08-09T13:54:08.362 INFO:tasks.cephfs.test_snap_schedules:snapshots: set() 2023-08-09T13:54:08.363 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-08-09T13:54:08.363 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-08-09T13:54:09.364 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-09T13:54:09.365 DEBUG:teuthology.orchestra.run.smithi043:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-08-09T13:54:09.439 INFO:tasks.cephfs.test_snap_schedules:snapshots: set() 2023-08-09T13:54:09.440 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-08-09T13:54:09.440 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-08-09T13:54:10.441 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-09T13:54:10.442 DEBUG:teuthology.orchestra.run.smithi043:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-08-09T13:54:10.519 INFO:tasks.cephfs.test_snap_schedules:snapshots: set() 2023-08-09T13:54:10.520 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-08-09T13:54:10.520 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-08-09T13:54:11.521 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-09T13:54:11.522 DEBUG:teuthology.orchestra.run.smithi043:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-08-09T13:54:11.599 INFO:tasks.cephfs.test_snap_schedules:snapshots: set() 2023-08-09T13:54:11.599 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-08-09T13:54:11.599 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-08-09T13:54:12.600 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-09T13:54:12.601 DEBUG:teuthology.orchestra.run.smithi043:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-08-09T13:54:12.676 INFO:tasks.cephfs.test_snap_schedules:snapshots: set() 2023-08-09T13:54:12.676 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-08-09T13:54:12.677 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-08-09T13:54:13.678 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-09T13:54:13.679 DEBUG:teuthology.orchestra.run.smithi043:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-08-09T13:54:13.758 INFO:tasks.cephfs.test_snap_schedules:snapshots: set() 2023-08-09T13:54:13.759 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-08-09T13:54:13.759 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-08-09T13:54:14.760 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-09T13:54:14.761 DEBUG:teuthology.orchestra.run.smithi043:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-08-09T13:54:14.836 INFO:tasks.cephfs.test_snap_schedules:snapshots: set() 2023-08-09T13:54:14.837 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-08-09T13:54:14.837 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-08-09T13:54:15.838 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-09T13:54:15.839 DEBUG:teuthology.orchestra.run.smithi043:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-08-09T13:54:15.914 INFO:tasks.cephfs.test_snap_schedules:snapshots: set() 2023-08-09T13:54:15.915 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-08-09T13:54:15.915 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-08-09T13:54:16.916 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-09T13:54:16.916 DEBUG:teuthology.orchestra.run.smithi043:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-08-09T13:54:16.991 INFO:tasks.cephfs.test_snap_schedules:snapshots: set() 2023-08-09T13:54:16.992 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-08-09T13:54:16.992 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-08-09T13:54:17.993 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-09T13:54:17.994 DEBUG:teuthology.orchestra.run.smithi043:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-08-09T13:54:18.069 INFO:tasks.cephfs.test_snap_schedules:snapshots: set() 2023-08-09T13:54:18.070 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-08-09T13:54:18.070 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-08-09T13:54:19.071 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-09T13:54:19.071 DEBUG:teuthology.orchestra.run.smithi043:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-08-09T13:54:19.148 INFO:tasks.cephfs.test_snap_schedules:snapshots: set() 2023-08-09T13:54:19.148 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-08-09T13:54:19.149 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-08-09T13:54:20.150 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-09T13:54:20.151 DEBUG:teuthology.orchestra.run.smithi043:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-08-09T13:54:20.226 INFO:tasks.cephfs.test_snap_schedules:snapshots: set() 2023-08-09T13:54:20.227 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-08-09T13:54:20.227 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-08-09T13:54:21.228 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-09T13:54:21.228 DEBUG:teuthology.orchestra.run.smithi043:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-08-09T13:54:21.307 INFO:tasks.cephfs.test_snap_schedules:snapshots: set() 2023-08-09T13:54:21.307 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-08-09T13:54:21.307 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-08-09T13:54:22.308 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-09T13:54:22.309 DEBUG:teuthology.orchestra.run.smithi043:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-08-09T13:54:22.385 INFO:tasks.cephfs.test_snap_schedules:snapshots: set() 2023-08-09T13:54:22.386 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-08-09T13:54:22.386 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-08-09T13:54:23.387 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-09T13:54:23.387 DEBUG:teuthology.orchestra.run.smithi043:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-08-09T13:54:23.459 INFO:tasks.cephfs.test_snap_schedules:snapshots: set() 2023-08-09T13:54:23.460 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-08-09T13:54:23.460 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-08-09T13:54:24.462 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-09T13:54:24.462 DEBUG:teuthology.orchestra.run.smithi043:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-08-09T13:54:24.538 INFO:tasks.cephfs.test_snap_schedules:snapshots: set() 2023-08-09T13:54:24.538 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-08-09T13:54:24.538 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-08-09T13:54:25.539 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-09T13:54:25.540 DEBUG:teuthology.orchestra.run.smithi043:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-08-09T13:54:25.614 INFO:tasks.cephfs.test_snap_schedules:snapshots: set() 2023-08-09T13:54:25.615 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-08-09T13:54:25.615 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-08-09T13:54:26.616 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-09T13:54:26.617 DEBUG:teuthology.orchestra.run.smithi043:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-08-09T13:54:26.693 INFO:tasks.cephfs.test_snap_schedules:snapshots: set() 2023-08-09T13:54:26.694 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-08-09T13:54:26.694 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-08-09T13:54:27.695 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-09T13:54:27.695 DEBUG:teuthology.orchestra.run.smithi043:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-08-09T13:54:27.772 INFO:tasks.cephfs.test_snap_schedules:snapshots: set() 2023-08-09T13:54:27.773 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-08-09T13:54:27.773 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-08-09T13:54:28.774 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-09T13:54:28.775 DEBUG:teuthology.orchestra.run.smithi043:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-08-09T13:54:28.854 INFO:tasks.cephfs.test_snap_schedules:snapshots: set() 2023-08-09T13:54:28.854 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-08-09T13:54:28.855 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-08-09T13:54:29.856 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-09T13:54:29.856 DEBUG:teuthology.orchestra.run.smithi043:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-08-09T13:54:29.930 INFO:tasks.cephfs.test_snap_schedules:snapshots: set() 2023-08-09T13:54:29.931 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-08-09T13:54:29.931 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-08-09T13:54:30.932 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-09T13:54:30.932 DEBUG:teuthology.orchestra.run.smithi043:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-08-09T13:54:31.009 INFO:tasks.cephfs.test_snap_schedules:snapshots: set() 2023-08-09T13:54:31.010 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-08-09T13:54:31.010 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-08-09T13:54:32.011 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-09T13:54:32.012 DEBUG:teuthology.orchestra.run.smithi043:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-08-09T13:54:32.090 INFO:tasks.cephfs.test_snap_schedules:snapshots: set() 2023-08-09T13:54:32.090 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-08-09T13:54:32.091 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-08-09T13:54:33.091 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-09T13:54:33.092 DEBUG:teuthology.orchestra.run.smithi043:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-08-09T13:54:33.170 INFO:tasks.cephfs.test_snap_schedules:snapshots: set() 2023-08-09T13:54:33.170 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-08-09T13:54:33.170 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-08-09T13:54:34.171 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-09T13:54:34.171 DEBUG:teuthology.orchestra.run.smithi043:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-08-09T13:54:34.245 INFO:tasks.cephfs.test_snap_schedules:snapshots: set() 2023-08-09T13:54:34.245 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-08-09T13:54:34.246 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-08-09T13:54:35.027 DEBUG:teuthology.orchestra.run.smithi043:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-08-09T13:54:35.036 DEBUG:teuthology.orchestra.run.smithi203:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-08-09T13:54:35.247 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-09T13:54:35.247 DEBUG:teuthology.orchestra.run.smithi043:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-08-09T13:54:35.324 INFO:tasks.cephfs.test_snap_schedules:snapshots: set() 2023-08-09T13:54:35.324 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-08-09T13:54:35.324 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-08-09T13:54:36.325 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-09T13:54:36.326 DEBUG:teuthology.orchestra.run.smithi043:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-08-09T13:54:36.400 INFO:tasks.cephfs.test_snap_schedules:snapshots: set() 2023-08-09T13:54:36.400 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-08-09T13:54:36.401 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-08-09T13:54:37.401 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-09T13:54:37.402 DEBUG:teuthology.orchestra.run.smithi043:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-08-09T13:54:37.478 INFO:tasks.cephfs.test_snap_schedules:snapshots: set() 2023-08-09T13:54:37.478 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-08-09T13:54:37.479 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-08-09T13:54:38.479 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-09T13:54:38.480 DEBUG:teuthology.orchestra.run.smithi043:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-08-09T13:54:38.560 INFO:tasks.cephfs.test_snap_schedules:snapshots: set() 2023-08-09T13:54:38.560 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-08-09T13:54:38.561 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-08-09T13:54:39.561 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-09T13:54:39.562 DEBUG:teuthology.orchestra.run.smithi043:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-08-09T13:54:39.637 INFO:tasks.cephfs.test_snap_schedules:snapshots: set() 2023-08-09T13:54:39.637 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-08-09T13:54:39.638 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-08-09T13:54:40.639 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-09T13:54:40.639 DEBUG:teuthology.orchestra.run.smithi043:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-08-09T13:54:40.716 INFO:tasks.cephfs.test_snap_schedules:snapshots: set() 2023-08-09T13:54:40.716 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-08-09T13:54:40.716 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-08-09T13:54:41.718 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-09T13:54:41.718 DEBUG:teuthology.orchestra.run.smithi043:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-08-09T13:54:41.796 INFO:tasks.cephfs.test_snap_schedules:snapshots: set() 2023-08-09T13:54:41.796 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-08-09T13:54:41.796 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-08-09T13:54:42.797 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-09T13:54:42.797 DEBUG:teuthology.orchestra.run.smithi043:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-08-09T13:54:42.870 INFO:tasks.cephfs.test_snap_schedules:snapshots: set() 2023-08-09T13:54:42.870 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-08-09T13:54:42.870 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-08-09T13:54:43.871 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-09T13:54:43.872 DEBUG:teuthology.orchestra.run.smithi043:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-08-09T13:54:43.948 INFO:tasks.cephfs.test_snap_schedules:snapshots: set() 2023-08-09T13:54:43.948 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-08-09T13:54:43.948 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-08-09T13:54:44.949 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-09T13:54:44.950 DEBUG:teuthology.orchestra.run.smithi043:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-08-09T13:54:45.021 INFO:tasks.cephfs.test_snap_schedules:snapshots: set() 2023-08-09T13:54:45.021 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-08-09T13:54:45.021 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-08-09T13:54:46.022 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-09T13:54:46.023 DEBUG:teuthology.orchestra.run.smithi043:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-08-09T13:54:46.098 INFO:tasks.cephfs.test_snap_schedules:snapshots: set() 2023-08-09T13:54:46.099 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-08-09T13:54:46.099 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-08-09T13:54:47.100 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-09T13:54:47.101 DEBUG:teuthology.orchestra.run.smithi043:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-08-09T13:54:47.177 INFO:tasks.cephfs.test_snap_schedules:snapshots: set() 2023-08-09T13:54:47.178 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-08-09T13:54:47.178 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-08-09T13:54:48.179 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-09T13:54:48.180 DEBUG:teuthology.orchestra.run.smithi043:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-08-09T13:54:48.255 INFO:tasks.cephfs.test_snap_schedules:snapshots: set() 2023-08-09T13:54:48.256 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-08-09T13:54:48.256 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-08-09T13:54:49.257 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-09T13:54:49.258 DEBUG:teuthology.orchestra.run.smithi043:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-08-09T13:54:49.334 INFO:tasks.cephfs.test_snap_schedules:snapshots: set() 2023-08-09T13:54:49.334 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-08-09T13:54:49.335 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-08-09T13:54:50.336 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-09T13:54:50.337 DEBUG:teuthology.orchestra.run.smithi043:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-08-09T13:54:50.414 INFO:tasks.cephfs.test_snap_schedules:snapshots: set() 2023-08-09T13:54:50.414 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-08-09T13:54:50.414 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-08-09T13:54:51.415 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-09T13:54:51.416 DEBUG:teuthology.orchestra.run.smithi043:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-08-09T13:54:51.491 INFO:tasks.cephfs.test_snap_schedules:snapshots: set() 2023-08-09T13:54:51.491 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-08-09T13:54:51.492 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-08-09T13:54:52.493 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-09T13:54:52.493 DEBUG:teuthology.orchestra.run.smithi043:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-08-09T13:54:52.569 INFO:tasks.cephfs.test_snap_schedules:snapshots: set() 2023-08-09T13:54:52.569 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-08-09T13:54:52.569 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-08-09T13:54:53.571 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-09T13:54:53.572 DEBUG:teuthology.orchestra.run.smithi043:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-08-09T13:54:53.648 INFO:tasks.cephfs.test_snap_schedules:snapshots: set() 2023-08-09T13:54:53.649 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-08-09T13:54:53.649 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-08-09T13:54:54.650 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-09T13:54:54.650 DEBUG:teuthology.orchestra.run.smithi043:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-08-09T13:54:54.726 INFO:tasks.cephfs.test_snap_schedules:snapshots: set() 2023-08-09T13:54:54.726 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-08-09T13:54:54.726 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-08-09T13:54:55.728 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-09T13:54:55.728 DEBUG:teuthology.orchestra.run.smithi043:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-08-09T13:54:55.804 INFO:tasks.cephfs.test_snap_schedules:snapshots: set() 2023-08-09T13:54:55.804 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-08-09T13:54:55.804 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-08-09T13:54:56.806 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-09T13:54:56.806 DEBUG:teuthology.orchestra.run.smithi043:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-08-09T13:54:56.880 INFO:tasks.cephfs.test_snap_schedules:snapshots: set() 2023-08-09T13:54:56.881 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-08-09T13:54:56.881 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-08-09T13:54:57.882 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-09T13:54:57.882 DEBUG:teuthology.orchestra.run.smithi043:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-08-09T13:54:57.959 INFO:tasks.cephfs.test_snap_schedules:snapshots: set() 2023-08-09T13:54:57.959 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-08-09T13:54:57.960 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-08-09T13:54:58.960 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-09T13:54:58.961 DEBUG:teuthology.orchestra.run.smithi043:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-08-09T13:54:59.036 INFO:tasks.cephfs.test_snap_schedules:snapshots: set() 2023-08-09T13:54:59.037 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-08-09T13:54:59.037 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-08-09T13:55:00.038 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-09T13:55:00.039 DEBUG:teuthology.orchestra.run.smithi043:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-08-09T13:55:00.112 INFO:tasks.cephfs.test_snap_schedules:snapshots: set() 2023-08-09T13:55:00.112 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-08-09T13:55:00.112 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-08-09T13:55:00.447 INFO:tasks.ceph.mgr.x.smithi203.stderr:2023-08-09T13:55:00.444+0000 7f0902f95640 -1 client.0 error registering admin socket command: (17) File exists 2023-08-09T13:55:00.447 INFO:tasks.ceph.mgr.x.smithi203.stderr:2023-08-09T13:55:00.444+0000 7f0902f95640 -1 client.0 error registering admin socket command: (17) File exists 2023-08-09T13:55:00.447 INFO:tasks.ceph.mgr.x.smithi203.stderr:2023-08-09T13:55:00.444+0000 7f0902f95640 -1 client.0 error registering admin socket command: (17) File exists 2023-08-09T13:55:00.447 INFO:tasks.ceph.mgr.x.smithi203.stderr:2023-08-09T13:55:00.444+0000 7f0902f95640 -1 client.0 error registering admin socket command: (17) File exists 2023-08-09T13:55:00.447 INFO:tasks.ceph.mgr.x.smithi203.stderr:2023-08-09T13:55:00.444+0000 7f0902f95640 -1 client.0 error registering admin socket command: (17) File exists 2023-08-09T13:55:01.114 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-09T13:55:01.114 DEBUG:teuthology.orchestra.run.smithi043:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-08-09T13:55:01.187 INFO:teuthology.orchestra.run.smithi043.stdout:scheduled-2023-08-09-13_55_00_UTC 2023-08-09T13:55:01.189 INFO:tasks.cephfs.test_snap_schedules:snapshots: {'scheduled-2023-08-09-13_55_00_UTC'} 2023-08-09T13:55:01.189 INFO:tasks.cephfs.test_snap_schedules:added: {'scheduled-2023-08-09-13_55_00_UTC'} 2023-08-09T13:55:01.189 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-08-09T13:55:01.189 DEBUG:tasks.cephfs.test_snap_schedules:snapshots added=['scheduled-2023-08-09-13_55_00_UTC'] 2023-08-09T13:55:01.190 DEBUG:tasks.cephfs.test_snap_schedules:exec=1691589244.4526298, now = 1691589301.1902084, timo = 56 2023-08-09T13:55:02.191 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs snap-schedule remove --fs cephfs snap_test_dir1 2023-08-09T13:55:02.428 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:02.427+0000 7fec7759e640 1 Processor -- start 2023-08-09T13:55:02.451 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:02.427+0000 7fec7759e640 1 -- start start 2023-08-09T13:55:02.454 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:02.428+0000 7fec7759e640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fec780cd3c0 0x7fec780cec70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:55:02.455 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:02.428+0000 7fec7759e640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7fec780cf1b0 0x7fec780b5880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:55:02.455 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:02.428+0000 7fec7759e640 1 -- --> v1:172.21.15.203:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fec780cf620 con 0x7fec7815c400 2023-08-09T13:55:02.455 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:02.428+0000 7fec7759e640 1 -- --> v2:172.21.15.43:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fec7815c7e0 con 0x7fec780cf1b0 2023-08-09T13:55:02.455 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:02.428+0000 7fec7759e640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fec780d13e0 con 0x7fec780cd3c0 2023-08-09T13:55:02.455 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:02.429+0000 7fec76d9d640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7fec780cf1b0 0x7fec780b5880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:55:02.455 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:02.429+0000 7fec76d9d640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7fec780cf1b0 0x7fec780b5880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:55244/0 (socket says 172.21.15.43:55244) 2023-08-09T13:55:02.455 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:02.429+0000 7fec76d9d640 1 -- 172.21.15.43:0/3541977428 learned_addr learned my addr 172.21.15.43:0/3541977428 (peer_addr_for_me v2:172.21.15.43:0/0) 2023-08-09T13:55:02.456 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:02.429+0000 7fec76d9d640 1 -- 172.21.15.43:0/3541977428 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fec780cd3c0 msgr2=0x7fec780cec70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-09T13:55:02.456 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:02.429+0000 7fec76d9d640 1 --2- 172.21.15.43:0/3541977428 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fec780cd3c0 0x7fec780cec70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:55:02.456 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:02.429+0000 7fec76d9d640 1 -- 172.21.15.43:0/3541977428 >> v1:172.21.15.203:6789/0 conn(0x7fec7815c400 legacy=0x7fec780d0cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-09T13:55:02.456 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:02.429+0000 7fec76d9d640 1 -- 172.21.15.43:0/3541977428 --> v2:172.21.15.43:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fec780d1690 con 0x7fec780cf1b0 2023-08-09T13:55:02.456 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:02.430+0000 7fec76d9d640 1 --2- 172.21.15.43:0/3541977428 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fec780cf1b0 0x7fec780b5880 secure :-1 s=READY pgs=1564 cs=0 l=1 rev1=1 crypto rx=0x7fec70009e50 tx=0x7fec7002bf60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e2d426b7212690b4 server_cookie=0 in_seq=0 out_seq=0 2023-08-09T13:55:02.456 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:02.431+0000 7fec7559a640 1 -- 172.21.15.43:0/3541977428 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fec7002cb00 con 0x7fec780cf1b0 2023-08-09T13:55:02.457 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:02.431+0000 7fec7559a640 1 -- 172.21.15.43:0/3541977428 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fec7002cc60 con 0x7fec780cf1b0 2023-08-09T13:55:02.457 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:02.431+0000 7fec7559a640 1 -- 172.21.15.43:0/3541977428 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fec700343f0 con 0x7fec780cf1b0 2023-08-09T13:55:02.457 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:02.431+0000 7fec7759e640 1 -- 172.21.15.43:0/3541977428 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fec780cf1b0 msgr2=0x7fec780b5880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:55:02.457 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:02.431+0000 7fec7759e640 1 --2- 172.21.15.43:0/3541977428 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fec780cf1b0 0x7fec780b5880 secure :-1 s=READY pgs=1564 cs=0 l=1 rev1=1 crypto rx=0x7fec70009e50 tx=0x7fec7002bf60 comp rx=0 tx=0).stop 2023-08-09T13:55:02.457 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:02.432+0000 7fec7759e640 1 -- 172.21.15.43:0/3541977428 shutdown_connections 2023-08-09T13:55:02.457 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:02.432+0000 7fec7759e640 1 --2- 172.21.15.43:0/3541977428 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fec780cf1b0 0x7fec780b5880 unknown :-1 s=CLOSED pgs=1564 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:55:02.457 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:02.432+0000 7fec7759e640 1 --2- 172.21.15.43:0/3541977428 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fec780cd3c0 0x7fec780cec70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:55:02.457 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:02.432+0000 7fec7759e640 1 -- 172.21.15.43:0/3541977428 >> 172.21.15.43:0/3541977428 conn(0x7fec7806a320 msgr2=0x7fec7806a730 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-09T13:55:02.458 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:02.432+0000 7fec7759e640 1 -- 172.21.15.43:0/3541977428 shutdown_connections 2023-08-09T13:55:02.458 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:02.432+0000 7fec7759e640 1 -- 172.21.15.43:0/3541977428 wait complete. 2023-08-09T13:55:02.458 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:02.433+0000 7fec7759e640 1 Processor -- start 2023-08-09T13:55:02.458 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:02.434+0000 7fec7759e640 1 -- start start 2023-08-09T13:55:02.458 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:02.435+0000 7fec7759e640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fec780cd3c0 0x7fec781f21b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:55:02.458 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:02.435+0000 7fec7759e640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fec780cf1b0 0x7fec781f2a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:55:02.458 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:02.435+0000 7fec7759e640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fec7815c400 0x7fec781556d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:55:02.459 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:02.435+0000 7fec7759e640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fec7815e440 con 0x7fec7815c400 2023-08-09T13:55:02.459 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:02.435+0000 7fec7759e640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fec7815daa0 con 0x7fec780cd3c0 2023-08-09T13:55:02.459 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:02.435+0000 7fec7759e640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fec7815dd80 con 0x7fec780cf1b0 2023-08-09T13:55:02.459 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:02.435+0000 7fec76d9d640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fec7815c400 0x7fec781556d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:55:02.459 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:02.435+0000 7fec76d9d640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fec7815c400 0x7fec781556d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:55256/0 (socket says 172.21.15.43:55256) 2023-08-09T13:55:02.459 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:02.435+0000 7fec76d9d640 1 -- 172.21.15.43:0/2438299935 learned_addr learned my addr 172.21.15.43:0/2438299935 (peer_addr_for_me v2:172.21.15.43:0/0) 2023-08-09T13:55:02.459 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:02.435+0000 7fec7659c640 1 --2- 172.21.15.43:0/2438299935 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fec780cd3c0 0x7fec781f21b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:55:02.459 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:02.435+0000 7fec75d9b640 1 --2- 172.21.15.43:0/2438299935 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fec780cf1b0 0x7fec781f2a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:55:02.460 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:02.435+0000 7fec76d9d640 1 -- 172.21.15.43:0/2438299935 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fec780cf1b0 msgr2=0x7fec781f2a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:55:02.460 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:02.435+0000 7fec76d9d640 1 --2- 172.21.15.43:0/2438299935 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fec780cf1b0 0x7fec781f2a00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:55:02.460 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:02.435+0000 7fec76d9d640 1 -- 172.21.15.43:0/2438299935 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fec780cd3c0 msgr2=0x7fec781f21b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:55:02.460 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:02.435+0000 7fec76d9d640 1 --2- 172.21.15.43:0/2438299935 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fec780cd3c0 0x7fec781f21b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:55:02.460 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:02.435+0000 7fec76d9d640 1 -- 172.21.15.43:0/2438299935 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fec70009b30 con 0x7fec7815c400 2023-08-09T13:55:02.460 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:02.436+0000 7fec76d9d640 1 --2- 172.21.15.43:0/2438299935 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fec7815c400 0x7fec781556d0 secure :-1 s=READY pgs=1565 cs=0 l=1 rev1=1 crypto rx=0x7fec70036040 tx=0x7fec70004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-09T13:55:02.460 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:02.436+0000 7fec66ffd640 1 -- 172.21.15.43:0/2438299935 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fec7003e450 con 0x7fec7815c400 2023-08-09T13:55:02.461 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:02.436+0000 7fec66ffd640 1 -- 172.21.15.43:0/2438299935 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fec7003e5d0 con 0x7fec7815c400 2023-08-09T13:55:02.461 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:02.436+0000 7fec66ffd640 1 -- 172.21.15.43:0/2438299935 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fec7003e980 con 0x7fec7815c400 2023-08-09T13:55:02.461 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:02.436+0000 7fec7759e640 1 -- 172.21.15.43:0/2438299935 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fec7815e060 con 0x7fec7815c400 2023-08-09T13:55:02.461 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:02.436+0000 7fec7759e640 1 -- 172.21.15.43:0/2438299935 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fec781f2f40 con 0x7fec7815c400 2023-08-09T13:55:02.461 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:02.438+0000 7fec7759e640 1 -- 172.21.15.43:0/2438299935 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fec7815e580 con 0x7fec7815c400 2023-08-09T13:55:02.461 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:02.438+0000 7fec66ffd640 1 -- 172.21.15.43:0/2438299935 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 62) v1 ==== 88574+0+0 (secure 0 0 0) 0x7fec7003eae0 con 0x7fec7815c400 2023-08-09T13:55:02.461 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:02.439+0000 7fec66ffd640 1 --2- 172.21.15.43:0/2438299935 >> [v2:172.21.15.203:6832/1796635339,v1:172.21.15.203:6833/1796635339] conn(0x7fec5806a2e0 0x7fec5806c7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:55:02.461 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:02.439+0000 7fec66ffd640 1 -- 172.21.15.43:0/2438299935 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(130..130 src has 1..130) v4 ==== 6770+0+0 (secure 0 0 0) 0x7fec700aba60 con 0x7fec7815c400 2023-08-09T13:55:02.462 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:02.440+0000 7fec7659c640 1 --2- 172.21.15.43:0/2438299935 >> [v2:172.21.15.203:6832/1796635339,v1:172.21.15.203:6833/1796635339] conn(0x7fec5806a2e0 0x7fec5806c7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:55:02.462 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:02.441+0000 7fec7659c640 1 --2- 172.21.15.43:0/2438299935 >> [v2:172.21.15.203:6832/1796635339,v1:172.21.15.203:6833/1796635339] conn(0x7fec5806a2e0 0x7fec5806c7a0 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7fec6800a750 tx=0x7fec68008040 comp rx=0 tx=0).ready entity=mgr.7288 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-09T13:55:02.462 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:02.444+0000 7fec66ffd640 1 -- 172.21.15.43:0/2438299935 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178066 (secure 0 0 0) 0x7fec7815e580 con 0x7fec7815c400 2023-08-09T13:55:02.666 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:02.665+0000 7fec7759e640 1 -- 172.21.15.43:0/2438299935 --> [v2:172.21.15.203:6832/1796635339,v1:172.21.15.203:6833/1796635339] -- mgr_command(tid 0: {"prefix": "fs snap-schedule remove", "fs": "cephfs", "path": "snap_test_dir1", "target": ["mon-mgr", ""]}) v1 -- 0x7fec780d1c50 con 0x7fec5806a2e0 2023-08-09T13:55:02.822 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:02.821+0000 7fec66ffd640 1 -- 172.21.15.43:0/2438299935 <== mgr.7288 v2:172.21.15.203:6832/1796635339 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+41 (secure 0 0 0) 0x7fec6800a380 con 0x7fec5806a2e0 2023-08-09T13:55:02.823 INFO:teuthology.orchestra.run.smithi043.stdout:Schedule removed for path snap_test_dir1 2023-08-09T13:55:02.825 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:02.824+0000 7fec7759e640 1 -- 172.21.15.43:0/2438299935 >> [v2:172.21.15.203:6832/1796635339,v1:172.21.15.203:6833/1796635339] conn(0x7fec5806a2e0 msgr2=0x7fec5806c7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:55:02.826 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:02.824+0000 7fec7759e640 1 --2- 172.21.15.43:0/2438299935 >> [v2:172.21.15.203:6832/1796635339,v1:172.21.15.203:6833/1796635339] conn(0x7fec5806a2e0 0x7fec5806c7a0 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7fec6800a750 tx=0x7fec68008040 comp rx=0 tx=0).stop 2023-08-09T13:55:02.826 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:02.824+0000 7fec7759e640 1 -- 172.21.15.43:0/2438299935 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fec7815c400 msgr2=0x7fec781556d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:55:02.826 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:02.824+0000 7fec7759e640 1 --2- 172.21.15.43:0/2438299935 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fec7815c400 0x7fec781556d0 secure :-1 s=READY pgs=1565 cs=0 l=1 rev1=1 crypto rx=0x7fec70036040 tx=0x7fec70004a40 comp rx=0 tx=0).stop 2023-08-09T13:55:02.826 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:02.824+0000 7fec7759e640 1 -- 172.21.15.43:0/2438299935 shutdown_connections 2023-08-09T13:55:02.826 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:02.824+0000 7fec7759e640 1 --2- 172.21.15.43:0/2438299935 >> [v2:172.21.15.203:6832/1796635339,v1:172.21.15.203:6833/1796635339] conn(0x7fec5806a2e0 0x7fec5806c7a0 unknown :-1 s=CLOSED pgs=25 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:55:02.826 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:02.824+0000 7fec7759e640 1 --2- 172.21.15.43:0/2438299935 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fec7815c400 0x7fec781556d0 unknown :-1 s=CLOSED pgs=1565 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:55:02.827 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:02.824+0000 7fec7759e640 1 --2- 172.21.15.43:0/2438299935 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fec780cf1b0 0x7fec781f2a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:55:02.827 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:02.824+0000 7fec7759e640 1 --2- 172.21.15.43:0/2438299935 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fec780cd3c0 0x7fec781f21b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:55:02.827 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:02.824+0000 7fec7759e640 1 -- 172.21.15.43:0/2438299935 >> 172.21.15.43:0/2438299935 conn(0x7fec7806a320 msgr2=0x7fec7806a700 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-09T13:55:02.827 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:02.824+0000 7fec7759e640 1 -- 172.21.15.43:0/2438299935 shutdown_connections 2023-08-09T13:55:02.827 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:02.824+0000 7fec7759e640 1 -- 172.21.15.43:0/2438299935 wait complete. 2023-08-09T13:55:02.847 DEBUG:tasks.cephfs.test_snap_schedules:res=Schedule removed for path snap_test_dir1 2023-08-09T13:55:02.848 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-09T13:55:02.848 DEBUG:teuthology.orchestra.run.smithi043:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-08-09T13:55:02.904 INFO:teuthology.orchestra.run.smithi043.stdout:scheduled-2023-08-09-13_55_00_UTC 2023-08-09T13:55:02.905 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/.snap/scheduled-2023-08-09-13_55_00_UTC 2023-08-09T13:55:02.905 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-09T13:55:02.905 DEBUG:teuthology.orchestra.run.smithi043:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/.snap/scheduled-2023-08-09-13_55_00_UTC) 2023-08-09T13:55:02.960 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-09T13:55:02.961 DEBUG:teuthology.orchestra.run.smithi043:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1) 2023-08-09T13:55:03.009 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph mgr module disable snap_schedule 2023-08-09T13:55:03.205 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:03.203+0000 7f6b71cf3640 1 Processor -- start 2023-08-09T13:55:03.205 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:03.203+0000 7f6b71cf3640 1 -- start start 2023-08-09T13:55:03.206 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:03.205+0000 7f6b71cf3640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f6b6c0b3fa0 0x7f6b6c0b43f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:55:03.206 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:03.205+0000 7f6b71cf3640 1 -- --> v1:172.21.15.43:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6b6c0b4930 con 0x7f6b6c15c680 2023-08-09T13:55:03.206 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:03.205+0000 7f6b71cf3640 1 -- --> v1:172.21.15.203:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6b6c0cdf00 con 0x7f6b6c0d0a70 2023-08-09T13:55:03.206 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:03.205+0000 7f6b71cf3640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6b6c0d0f70 con 0x7f6b6c0b3fa0 2023-08-09T13:55:03.207 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:03.205+0000 7f6b6b7fe640 1 --1- >> v1:172.21.15.43:6789/0 conn(0x7f6b6c15c680 0x7f6b6c0ce760 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.43:6789/0 says I am v1:172.21.15.43:35002/0 (socket says 172.21.15.43:35002) 2023-08-09T13:55:03.207 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:03.205+0000 7f6b6b7fe640 1 -- 172.21.15.43:0/29072714 learned_addr learned my addr 172.21.15.43:0/29072714 (peer_addr_for_me v1:172.21.15.43:0/0) 2023-08-09T13:55:03.208 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:03.205+0000 7f6b6a7fc640 1 -- 172.21.15.43:0/29072714 <== mon.0 v1:172.21.15.43:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3099910 0 0) 0x7f6b5c001110 con 0x7f6b6c15c680 2023-08-09T13:55:03.208 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:03.205+0000 7f6b6a7fc640 1 -- 172.21.15.43:0/29072714 --> v1:172.21.15.43:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f6b54003520 con 0x7f6b6c15c680 2023-08-09T13:55:03.209 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:03.206+0000 7f6b6bfff640 1 --2- 172.21.15.43:0/29072714 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f6b6c0b3fa0 0x7f6b6c0b43f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:55:03.209 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:03.206+0000 7f6b6a7fc640 1 -- 172.21.15.43:0/29072714 <== mon.0 v1:172.21.15.43:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2392512524 0 0) 0x7f6b54003520 con 0x7f6b6c15c680 2023-08-09T13:55:03.209 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:03.206+0000 7f6b6a7fc640 1 -- 172.21.15.43:0/29072714 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f6b6c0b3fa0 msgr2=0x7f6b6c0b43f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-09T13:55:03.209 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:03.206+0000 7f6b6a7fc640 1 --2- 172.21.15.43:0/29072714 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f6b6c0b3fa0 0x7f6b6c0b43f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:55:03.209 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:03.206+0000 7f6b6a7fc640 1 -- 172.21.15.43:0/29072714 >> v1:172.21.15.203:6789/0 conn(0x7f6b6c0d0a70 legacy=0x7f6b6c0cd7f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:55:03.209 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:03.206+0000 7f6b6a7fc640 1 -- 172.21.15.43:0/29072714 --> v1:172.21.15.43:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6b6c0ccd10 con 0x7f6b6c15c680 2023-08-09T13:55:03.210 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:03.206+0000 7f6b6a7fc640 1 -- 172.21.15.43:0/29072714 <== mon.0 v1:172.21.15.43:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1285150740 0 0) 0x7f6b5c001800 con 0x7f6b6c15c680 2023-08-09T13:55:03.210 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:03.206+0000 7f6b6a7fc640 1 -- 172.21.15.43:0/29072714 >> v1:172.21.15.43:6789/0 conn(0x7f6b6c15c680 legacy=0x7f6b6c0ce760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:55:03.210 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:03.206+0000 7f6b6a7fc640 1 --2- 172.21.15.43:0/29072714 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f6b54003a10 0x7f6b54003e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:55:03.210 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:03.206+0000 7f6b6a7fc640 1 --2- 172.21.15.43:0/29072714 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f6b540047c0 0x7f6b54004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:55:03.210 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:03.206+0000 7f6b6a7fc640 1 --2- 172.21.15.43:0/29072714 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f6b540052a0 0x7f6b5400b980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:55:03.210 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:03.206+0000 7f6b6a7fc640 1 -- 172.21.15.43:0/29072714 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6b6c0d0f70 con 0x7f6b540047c0 2023-08-09T13:55:03.210 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:03.206+0000 7f6b6a7fc640 1 -- 172.21.15.43:0/29072714 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6b5400c180 con 0x7f6b540052a0 2023-08-09T13:55:03.210 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:03.206+0000 7f6b6a7fc640 1 -- 172.21.15.43:0/29072714 --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6b5400c340 con 0x7f6b54003a10 2023-08-09T13:55:03.211 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:03.206+0000 7f6b6bfff640 1 --2- 172.21.15.43:0/29072714 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f6b6c0b3fa0 0x7f6b6c0b43f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-08-09T13:55:03.211 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:03.206+0000 7f6b6affd640 1 --2- 172.21.15.43:0/29072714 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f6b540047c0 0x7f6b54004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:55:03.211 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:03.207+0000 7f6b6b7fe640 1 --2- 172.21.15.43:0/29072714 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f6b54003a10 0x7f6b54003e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:55:03.211 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:03.207+0000 7f6b6affd640 1 -- 172.21.15.43:0/29072714 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f6b54003a10 msgr2=0x7f6b54003e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-09T13:55:03.211 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:03.207+0000 7f6b6affd640 1 --2- 172.21.15.43:0/29072714 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f6b54003a10 0x7f6b54003e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:55:03.211 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:03.207+0000 7f6b6affd640 1 -- 172.21.15.43:0/29072714 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f6b540052a0 msgr2=0x7f6b5400b980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-09T13:55:03.211 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:03.207+0000 7f6b6affd640 1 --2- 172.21.15.43:0/29072714 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f6b540052a0 0x7f6b5400b980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:55:03.212 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:03.207+0000 7f6b6affd640 1 -- 172.21.15.43:0/29072714 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6b54003520 con 0x7f6b540047c0 2023-08-09T13:55:03.213 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:03.208+0000 7f6b6affd640 1 --2- 172.21.15.43:0/29072714 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f6b540047c0 0x7f6b54004c40 secure :-1 s=READY pgs=1567 cs=0 l=1 rev1=1 crypto rx=0x7f6b60001720 tx=0x7f6b540057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ba15aecbe2b9b709 server_cookie=0 in_seq=0 out_seq=0 2023-08-09T13:55:03.213 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:03.208+0000 7f6b6a7fc640 1 -- 172.21.15.43:0/29072714 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6b600052d0 con 0x7f6b540047c0 2023-08-09T13:55:03.213 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:03.208+0000 7f6b6a7fc640 1 -- 172.21.15.43:0/29072714 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6b6002e050 con 0x7f6b540047c0 2023-08-09T13:55:03.213 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:03.208+0000 7f6b6a7fc640 1 -- 172.21.15.43:0/29072714 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6b60003830 con 0x7f6b540047c0 2023-08-09T13:55:03.213 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:03.209+0000 7f6b71cf3640 1 -- 172.21.15.43:0/29072714 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f6b540047c0 msgr2=0x7f6b54004c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:55:03.213 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:03.209+0000 7f6b71cf3640 1 --2- 172.21.15.43:0/29072714 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f6b540047c0 0x7f6b54004c40 secure :-1 s=READY pgs=1567 cs=0 l=1 rev1=1 crypto rx=0x7f6b60001720 tx=0x7f6b540057f0 comp rx=0 tx=0).stop 2023-08-09T13:55:03.213 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:03.209+0000 7f6b6bfff640 1 -- 172.21.15.43:0/29072714 reap_dead start 2023-08-09T13:55:03.214 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:03.209+0000 7f6b71cf3640 1 -- 172.21.15.43:0/29072714 shutdown_connections 2023-08-09T13:55:03.214 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:03.209+0000 7f6b71cf3640 1 -- 172.21.15.43:0/29072714 >> 172.21.15.43:0/29072714 conn(0x7f6b6c071520 msgr2=0x7f6b6c0d2520 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-09T13:55:03.214 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:03.209+0000 7f6b71cf3640 1 -- 172.21.15.43:0/29072714 shutdown_connections 2023-08-09T13:55:03.214 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:03.209+0000 7f6b71cf3640 1 -- 172.21.15.43:0/29072714 wait complete. 2023-08-09T13:55:03.214 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:03.211+0000 7f6b71cf3640 1 Processor -- start 2023-08-09T13:55:03.214 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:03.211+0000 7f6b71cf3640 1 -- start start 2023-08-09T13:55:03.216 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:03.212+0000 7f6b71cf3640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f6b540047c0 0x7f6b6c1458c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:55:03.217 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:03.212+0000 7f6b71cf3640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f6b6c0cbc10 0x7f6b6c0cc090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:55:03.217 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:03.212+0000 7f6b71cf3640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f6b6c15e510 0x7f6b6c15e990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:55:03.217 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:03.212+0000 7f6b71cf3640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6b6c15eed0 con 0x7f6b6c0cbc10 2023-08-09T13:55:03.217 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:03.212+0000 7f6b71cf3640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6b6c0caff0 con 0x7f6b6c15e510 2023-08-09T13:55:03.217 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:03.212+0000 7f6b71cf3640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6b6c145e00 con 0x7f6b540047c0 2023-08-09T13:55:03.217 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:03.212+0000 7f6b6affd640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f6b6c0cbc10 0x7f6b6c0cc090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:55:03.218 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:03.212+0000 7f6b6affd640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f6b6c0cbc10 0x7f6b6c0cc090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:55272/0 (socket says 172.21.15.43:55272) 2023-08-09T13:55:03.218 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:03.212+0000 7f6b6affd640 1 -- 172.21.15.43:0/1971053726 learned_addr learned my addr 172.21.15.43:0/1971053726 (peer_addr_for_me v2:172.21.15.43:0/0) 2023-08-09T13:55:03.218 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:03.213+0000 7f6b6b7fe640 1 --2- 172.21.15.43:0/1971053726 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f6b540047c0 0x7f6b6c1458c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:55:03.218 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:03.213+0000 7f6b6bfff640 1 --2- 172.21.15.43:0/1971053726 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f6b6c15e510 0x7f6b6c15e990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:55:03.218 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:03.213+0000 7f6b6affd640 1 -- 172.21.15.43:0/1971053726 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f6b540047c0 msgr2=0x7f6b6c1458c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:55:03.218 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:03.213+0000 7f6b6affd640 1 --2- 172.21.15.43:0/1971053726 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f6b540047c0 0x7f6b6c1458c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:55:03.218 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:03.213+0000 7f6b6affd640 1 -- 172.21.15.43:0/1971053726 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f6b6c15e510 msgr2=0x7f6b6c15e990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:55:03.219 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:03.213+0000 7f6b6affd640 1 --2- 172.21.15.43:0/1971053726 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f6b6c15e510 0x7f6b6c15e990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:55:03.219 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:03.213+0000 7f6b6affd640 1 -- 172.21.15.43:0/1971053726 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6b6c0ca3e0 con 0x7f6b6c0cbc10 2023-08-09T13:55:03.219 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:03.213+0000 7f6b6bfff640 1 --2- 172.21.15.43:0/1971053726 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f6b6c15e510 0x7f6b6c15e990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-08-09T13:55:03.219 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:03.213+0000 7f6b6affd640 1 --2- 172.21.15.43:0/1971053726 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f6b6c0cbc10 0x7f6b6c0cc090 secure :-1 s=READY pgs=1568 cs=0 l=1 rev1=1 crypto rx=0x7f6b60008fd0 tx=0x7f6b60001120 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-09T13:55:03.219 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:03.213+0000 7f6b53fff640 1 -- 172.21.15.43:0/1971053726 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6b600076c0 con 0x7f6b6c0cbc10 2023-08-09T13:55:03.219 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:03.214+0000 7f6b53fff640 1 -- 172.21.15.43:0/1971053726 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6b60007a30 con 0x7f6b6c0cbc10 2023-08-09T13:55:03.219 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:03.214+0000 7f6b53fff640 1 -- 172.21.15.43:0/1971053726 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6b60006be0 con 0x7f6b6c0cbc10 2023-08-09T13:55:03.219 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:03.214+0000 7f6b71cf3640 1 -- 172.21.15.43:0/1971053726 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6b6c0ca6c0 con 0x7f6b6c0cbc10 2023-08-09T13:55:03.220 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:03.214+0000 7f6b71cf3640 1 -- 172.21.15.43:0/1971053726 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6b6c0cac80 con 0x7f6b6c0cbc10 2023-08-09T13:55:03.220 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:03.215+0000 7f6b71cf3640 1 -- 172.21.15.43:0/1971053726 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6b6c0cae40 con 0x7f6b6c0cbc10 2023-08-09T13:55:03.221 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:03.217+0000 7f6b53fff640 1 -- 172.21.15.43:0/1971053726 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 62) v1 ==== 88574+0+0 (secure 0 0 0) 0x7f6b60006d40 con 0x7f6b6c0cbc10 2023-08-09T13:55:03.221 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:03.218+0000 7f6b53fff640 1 --2- 172.21.15.43:0/1971053726 >> [v2:172.21.15.203:6832/1796635339,v1:172.21.15.203:6833/1796635339] conn(0x7f6b5806a2e0 0x7f6b5806c7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:55:03.221 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:03.218+0000 7f6b53fff640 1 -- 172.21.15.43:0/1971053726 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(130..130 src has 1..130) v4 ==== 6770+0+0 (secure 0 0 0) 0x7f6b600aae00 con 0x7f6b6c0cbc10 2023-08-09T13:55:03.221 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:03.218+0000 7f6b6b7fe640 1 --2- 172.21.15.43:0/1971053726 >> [v2:172.21.15.203:6832/1796635339,v1:172.21.15.203:6833/1796635339] conn(0x7f6b5806a2e0 0x7f6b5806c7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:55:03.221 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:03.219+0000 7f6b6b7fe640 1 --2- 172.21.15.43:0/1971053726 >> [v2:172.21.15.203:6832/1796635339,v1:172.21.15.203:6833/1796635339] conn(0x7f6b5806a2e0 0x7f6b5806c7a0 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7f6b5c0012b0 tx=0x7f6b5c009040 comp rx=0 tx=0).ready entity=mgr.7288 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-09T13:55:03.223 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:03.222+0000 7f6b53fff640 1 -- 172.21.15.43:0/1971053726 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178066 (secure 0 0 0) 0x7f6b60075e80 con 0x7f6b6c0cbc10 2023-08-09T13:55:03.478 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:03.476+0000 7f6b71cf3640 1 -- 172.21.15.43:0/1971053726 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "mgr module disable", "module": "snap_schedule"} v 0) v1 -- 0x7f6b6c1f1e60 con 0x7f6b6c0cbc10 2023-08-09T13:55:04.361 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:04.359+0000 7f6b53fff640 1 -- 172.21.15.43:0/1971053726 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mgrmap(e 63) v1 ==== 88557+0+0 (secure 0 0 0) 0x7f6b600af070 con 0x7f6b6c0cbc10 2023-08-09T13:55:04.364 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:04.363+0000 7f6b53fff640 1 -- 172.21.15.43:0/1971053726 <== mon.0 v2:172.21.15.43:3300/0 8 ==== mon_command_ack([{"prefix": "mgr module disable", "module": "snap_schedule"}]=0 v63) v1 ==== 93+0+0 (secure 0 0 0) 0x7f6b600ade30 con 0x7f6b6c0cbc10 2023-08-09T13:55:04.368 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:04.367+0000 7f6b71cf3640 1 -- 172.21.15.43:0/1971053726 >> [v2:172.21.15.203:6832/1796635339,v1:172.21.15.203:6833/1796635339] conn(0x7f6b5806a2e0 msgr2=0x7f6b5806c7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:55:04.369 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:04.367+0000 7f6b71cf3640 1 --2- 172.21.15.43:0/1971053726 >> [v2:172.21.15.203:6832/1796635339,v1:172.21.15.203:6833/1796635339] conn(0x7f6b5806a2e0 0x7f6b5806c7a0 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7f6b5c0012b0 tx=0x7f6b5c009040 comp rx=0 tx=0).stop 2023-08-09T13:55:04.369 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:04.367+0000 7f6b71cf3640 1 -- 172.21.15.43:0/1971053726 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f6b6c0cbc10 msgr2=0x7f6b6c0cc090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:55:04.369 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:04.367+0000 7f6b71cf3640 1 --2- 172.21.15.43:0/1971053726 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f6b6c0cbc10 0x7f6b6c0cc090 secure :-1 s=READY pgs=1568 cs=0 l=1 rev1=1 crypto rx=0x7f6b60008fd0 tx=0x7f6b60001120 comp rx=0 tx=0).stop 2023-08-09T13:55:04.369 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:04.367+0000 7f6b71cf3640 1 -- 172.21.15.43:0/1971053726 shutdown_connections 2023-08-09T13:55:04.369 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:04.367+0000 7f6b71cf3640 1 --2- 172.21.15.43:0/1971053726 >> [v2:172.21.15.203:6832/1796635339,v1:172.21.15.203:6833/1796635339] conn(0x7f6b5806a2e0 0x7f6b5806c7a0 unknown :-1 s=CLOSED pgs=26 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:55:04.369 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:04.367+0000 7f6b71cf3640 1 --2- 172.21.15.43:0/1971053726 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f6b6c15e510 0x7f6b6c15e990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:55:04.370 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:04.367+0000 7f6b71cf3640 1 --2- 172.21.15.43:0/1971053726 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f6b6c0cbc10 0x7f6b6c0cc090 unknown :-1 s=CLOSED pgs=1568 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:55:04.370 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:04.367+0000 7f6b71cf3640 1 --2- 172.21.15.43:0/1971053726 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f6b540047c0 0x7f6b6c1458c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:55:04.370 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:04.367+0000 7f6b71cf3640 1 -- 172.21.15.43:0/1971053726 >> 172.21.15.43:0/1971053726 conn(0x7f6b6c071520 msgr2=0x7f6b6c0b6cf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-09T13:55:04.370 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:04.367+0000 7f6b71cf3640 1 -- 172.21.15.43:0/1971053726 shutdown_connections 2023-08-09T13:55:04.370 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:04.367+0000 7f6b71cf3640 1 -- 172.21.15.43:0/1971053726 wait complete. 2023-08-09T13:55:04.397 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2023-08-09T13:55:04.397 DEBUG:teuthology.orchestra.run.smithi043:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2023-08-09T13:55:04.502 DEBUG:teuthology.orchestra.run.smithi203:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2023-08-09T13:55:04.597 INFO:tasks.ceph.mgr.x.smithi203.stderr:2023-08-09T13:55:04.594+0000 7f8eeb831040 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2023-08-09T13:55:04.602 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2023-08-09T13:55:04.603 DEBUG:teuthology.orchestra.run.smithi043:> set -ex 2023-08-09T13:55:04.603 DEBUG:teuthology.orchestra.run.smithi043:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-09T13:55:04.612 INFO:tasks.ceph.mgr.y.smithi043.stderr:2023-08-09T13:55:04.608+0000 7f106ca0f040 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2023-08-09T13:55:04.660 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-08-09T13:55:04.873 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:04.871+0000 7fcedbfff640 1 Processor -- start 2023-08-09T13:55:04.873 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:04.872+0000 7fcedbfff640 1 -- start start 2023-08-09T13:55:04.875 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:04.873+0000 7fcedbfff640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fcedc15a2c0 0x7fcedc0b1e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:55:04.875 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:04.873+0000 7fcedbfff640 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7fcedc070b00 0x7fcedc0b23b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:55:04.875 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:04.873+0000 7fcedbfff640 1 -- --> v1:172.21.15.203:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fcedc0714c0 con 0x7fcedc06c530 2023-08-09T13:55:04.875 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:04.873+0000 7fcedbfff640 1 -- --> v2:172.21.15.203:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fcedc0c9b50 con 0x7fcedc070b00 2023-08-09T13:55:04.875 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:04.873+0000 7fcedbfff640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcedc0b1480 con 0x7fcedc15a2c0 2023-08-09T13:55:04.876 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:04.874+0000 7fcedaffd640 1 --1- >> v1:172.21.15.203:6789/0 conn(0x7fcedc06c530 0x7fcedc06c910 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.203:6789/0 says I am v1:172.21.15.43:36030/0 (socket says 172.21.15.43:36030) 2023-08-09T13:55:04.876 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:04.874+0000 7fcedaffd640 1 -- 172.21.15.43:0/163804919 learned_addr learned my addr 172.21.15.43:0/163804919 (peer_addr_for_me v1:172.21.15.43:0/0) 2023-08-09T13:55:04.876 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:04.874+0000 7fcedb7fe640 1 --2- 172.21.15.43:0/163804919 >> v2:172.21.15.203:3300/0 conn(0x7fcedc070b00 0x7fcedc0b23b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:55:04.877 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:04.874+0000 7fced3fff640 1 --2- 172.21.15.43:0/163804919 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fcedc15a2c0 0x7fcedc0b1e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:55:04.877 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:04.874+0000 7fceda7fc640 1 -- 172.21.15.43:0/163804919 <== mon.1 v1:172.21.15.203:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3051380365 0 0) 0x7fcedc0714c0 con 0x7fcedc06c530 2023-08-09T13:55:04.877 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:04.874+0000 7fceda7fc640 1 -- 172.21.15.43:0/163804919 --> v1:172.21.15.203:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fcec8003520 con 0x7fcedc06c530 2023-08-09T13:55:04.877 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:04.874+0000 7fcedb7fe640 1 -- 172.21.15.43:0/163804919 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fcedc15a2c0 msgr2=0x7fcedc0b1e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-09T13:55:04.877 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:04.874+0000 7fcedb7fe640 1 --2- 172.21.15.43:0/163804919 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fcedc15a2c0 0x7fcedc0b1e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:55:04.877 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:04.874+0000 7fcedb7fe640 1 -- 172.21.15.43:0/163804919 >> v1:172.21.15.203:6789/0 conn(0x7fcedc06c530 legacy=0x7fcedc06c910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:55:04.877 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:04.874+0000 7fcedb7fe640 1 -- 172.21.15.43:0/163804919 --> v2:172.21.15.203:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcedc0711e0 con 0x7fcedc070b00 2023-08-09T13:55:04.878 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:04.876+0000 7fcedb7fe640 1 --2- 172.21.15.43:0/163804919 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fcedc070b00 0x7fcedc0b23b0 secure :-1 s=READY pgs=590 cs=0 l=1 rev1=1 crypto rx=0x7fced4009d90 tx=0x7fced402bea0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=c605e299c0ab687b server_cookie=0 in_seq=0 out_seq=0 2023-08-09T13:55:04.878 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:04.876+0000 7fceda7fc640 1 -- 172.21.15.43:0/163804919 <== mon.1 v2:172.21.15.203:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fced402cc50 con 0x7fcedc070b00 2023-08-09T13:55:04.878 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:04.876+0000 7fceda7fc640 1 -- 172.21.15.43:0/163804919 <== mon.1 v2:172.21.15.203:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fced402cdb0 con 0x7fcedc070b00 2023-08-09T13:55:04.878 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:04.876+0000 7fceda7fc640 1 -- 172.21.15.43:0/163804919 <== mon.1 v2:172.21.15.203:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fced4034640 con 0x7fcedc070b00 2023-08-09T13:55:04.879 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:04.877+0000 7fcedbfff640 1 -- 172.21.15.43:0/163804919 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fcedc070b00 msgr2=0x7fcedc0b23b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:55:04.879 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:04.877+0000 7fcedbfff640 1 --2- 172.21.15.43:0/163804919 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fcedc070b00 0x7fcedc0b23b0 secure :-1 s=READY pgs=590 cs=0 l=1 rev1=1 crypto rx=0x7fced4009d90 tx=0x7fced402bea0 comp rx=0 tx=0).stop 2023-08-09T13:55:04.879 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:04.877+0000 7fcedbfff640 1 -- 172.21.15.43:0/163804919 shutdown_connections 2023-08-09T13:55:04.880 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:04.877+0000 7fcedbfff640 1 --2- 172.21.15.43:0/163804919 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fcedc070b00 0x7fcedc0b23b0 unknown :-1 s=CLOSED pgs=590 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:55:04.880 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:04.877+0000 7fcedbfff640 1 --2- 172.21.15.43:0/163804919 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fcedc15a2c0 0x7fcedc0b1e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:55:04.880 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:04.877+0000 7fcedbfff640 1 -- 172.21.15.43:0/163804919 >> 172.21.15.43:0/163804919 conn(0x7fcedc06a8d0 msgr2=0x7fcedc06acb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-09T13:55:04.880 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:04.877+0000 7fcedbfff640 1 -- 172.21.15.43:0/163804919 shutdown_connections 2023-08-09T13:55:04.880 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:04.877+0000 7fcedbfff640 1 -- 172.21.15.43:0/163804919 wait complete. 2023-08-09T13:55:04.881 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:04.880+0000 7fcedbfff640 1 Processor -- start 2023-08-09T13:55:04.881 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:04.880+0000 7fcedbfff640 1 -- start start 2023-08-09T13:55:04.882 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:04.881+0000 7fcedbfff640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fcedc06c530 0x7fcedc14cff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:55:04.883 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:04.881+0000 7fcedbfff640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fcedc070b00 0x7fcedc0ce280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:55:04.883 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:04.881+0000 7fcedbfff640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fcedc0ce7c0 0x7fcedc0af5f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:55:04.883 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:04.881+0000 7fcedbfff640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcedc1e3fd0 con 0x7fcedc070b00 2023-08-09T13:55:04.883 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:04.881+0000 7fcedbfff640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcedc14d530 con 0x7fcedc06c530 2023-08-09T13:55:04.883 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:04.881+0000 7fcedbfff640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcedc1e2f30 con 0x7fcedc0ce7c0 2023-08-09T13:55:04.884 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:04.881+0000 7fced3fff640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fcedc070b00 0x7fcedc0ce280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:55:04.885 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:04.882+0000 7fcedb7fe640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fcedc0ce7c0 0x7fcedc0af5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:55:04.885 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:04.882+0000 7fced3fff640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fcedc070b00 0x7fcedc0ce280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:55292/0 (socket says 172.21.15.43:55292) 2023-08-09T13:55:04.885 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:04.882+0000 7fced3fff640 1 -- 172.21.15.43:0/2189685592 learned_addr learned my addr 172.21.15.43:0/2189685592 (peer_addr_for_me v2:172.21.15.43:0/0) 2023-08-09T13:55:04.885 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:04.882+0000 7fcedaffd640 1 --2- 172.21.15.43:0/2189685592 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fcedc06c530 0x7fcedc14cff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:55:04.885 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:04.882+0000 7fced3fff640 1 -- 172.21.15.43:0/2189685592 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fcedc0ce7c0 msgr2=0x7fcedc0af5f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:55:04.885 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:04.882+0000 7fced3fff640 1 --2- 172.21.15.43:0/2189685592 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fcedc0ce7c0 0x7fcedc0af5f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:55:04.885 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:04.882+0000 7fced3fff640 1 -- 172.21.15.43:0/2189685592 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fcedc06c530 msgr2=0x7fcedc14cff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:55:04.886 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:04.882+0000 7fced3fff640 1 --2- 172.21.15.43:0/2189685592 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fcedc06c530 0x7fcedc14cff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:55:04.886 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:04.882+0000 7fced3fff640 1 -- 172.21.15.43:0/2189685592 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fced4009a70 con 0x7fcedc070b00 2023-08-09T13:55:04.886 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:04.883+0000 7fced3fff640 1 --2- 172.21.15.43:0/2189685592 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fcedc070b00 0x7fcedc0ce280 secure :-1 s=READY pgs=1570 cs=0 l=1 rev1=1 crypto rx=0x7fcec400e9b0 tx=0x7fcec400ee70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-09T13:55:04.886 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:04.883+0000 7fced37fe640 1 -- 172.21.15.43:0/2189685592 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcec4009670 con 0x7fcedc070b00 2023-08-09T13:55:04.886 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:04.883+0000 7fced37fe640 1 -- 172.21.15.43:0/2189685592 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcec400c9a0 con 0x7fcedc070b00 2023-08-09T13:55:04.886 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:04.883+0000 7fcedbfff640 1 -- 172.21.15.43:0/2189685592 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcedc1e3210 con 0x7fcedc070b00 2023-08-09T13:55:04.886 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:04.883+0000 7fced37fe640 1 -- 172.21.15.43:0/2189685592 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcec400cd50 con 0x7fcedc070b00 2023-08-09T13:55:04.886 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:04.883+0000 7fcedbfff640 1 -- 172.21.15.43:0/2189685592 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcedc1e37d0 con 0x7fcedc070b00 2023-08-09T13:55:04.887 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:04.884+0000 7fcedbfff640 1 -- 172.21.15.43:0/2189685592 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcedc0739f0 con 0x7fcedc070b00 2023-08-09T13:55:04.887 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:04.886+0000 7fced37fe640 1 -- 172.21.15.43:0/2189685592 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 63) v1 ==== 88557+0+0 (secure 0 0 0) 0x7fcec4013470 con 0x7fcedc070b00 2023-08-09T13:55:04.888 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:04.886+0000 7fced37fe640 1 --2- 172.21.15.43:0/2189685592 >> [v2:172.21.15.203:6832/1796635339,v1:172.21.15.203:6833/1796635339] conn(0x7fcebc06a290 0x7fcebc06c750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:55:04.888 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:04.887+0000 7fced37fe640 1 -- 172.21.15.43:0/2189685592 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(132..132 src has 1..132) v4 ==== 6828+0+0 (secure 0 0 0) 0x7fcec40896a0 con 0x7fcedc070b00 2023-08-09T13:55:04.888 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:04.887+0000 7fcedaffd640 1 -- 172.21.15.43:0/2189685592 >> [v2:172.21.15.203:6832/1796635339,v1:172.21.15.203:6833/1796635339] conn(0x7fcebc06a290 msgr2=0x7fcebc06c750 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.203:6832/1796635339 2023-08-09T13:55:04.888 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:04.887+0000 7fcedaffd640 1 --2- 172.21.15.43:0/2189685592 >> [v2:172.21.15.203:6832/1796635339,v1:172.21.15.203:6833/1796635339] conn(0x7fcebc06a290 0x7fcebc06c750 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-08-09T13:55:04.891 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:04.890+0000 7fced37fe640 1 -- 172.21.15.43:0/2189685592 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178066 (secure 0 0 0) 0x7fcec40547c0 con 0x7fcedc070b00 2023-08-09T13:55:05.088 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:05.086+0000 7fcedbfff640 1 -- 172.21.15.43:0/2189685592 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fcedc0b0820 con 0x7fcedc070b00 2023-08-09T13:55:05.089 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:05.088+0000 7fcedaffd640 1 -- 172.21.15.43:0/2189685592 >> [v2:172.21.15.203:6832/1796635339,v1:172.21.15.203:6833/1796635339] conn(0x7fcebc06a290 msgr2=0x7fcebc06c750 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.203:6832/1796635339 2023-08-09T13:55:05.089 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:05.088+0000 7fcedaffd640 1 --2- 172.21.15.43:0/2189685592 >> [v2:172.21.15.203:6832/1796635339,v1:172.21.15.203:6833/1796635339] conn(0x7fcebc06a290 0x7fcebc06c750 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2023-08-09T13:55:05.091 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:05.090+0000 7fced37fe640 1 -- 172.21.15.43:0/2189685592 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v132) v1 ==== 74+0+19116 (secure 0 0 0) 0x7fcec4017090 con 0x7fcedc070b00 2023-08-09T13:55:05.091 INFO:teuthology.orchestra.run.smithi043.stdout: 2023-08-09T13:55:05.092 INFO:teuthology.orchestra.run.smithi043.stdout:{"epoch":132,"fsid":"daac87ee-1337-4250-a955-aea4e408a39b","created":"2023-08-09T13:33:41.701198+0000","modified":"2023-08-09T13:55:04.346235+0000","last_up_change":"2023-08-09T13:33:53.783453+0000","last_in_change":"2023-08-09T13:33:45.148904+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":17,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":"rbd","create_time":"2023-08-09T13:33:54.365137+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":".mgr","create_time":"2023-08-09T13:33:54.831803+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":16,"pool_name":"cephfs_metadata","create_time":"2023-08-09T13:53:46.124616+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"128","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":17,"pool_name":"cephfs_data","create_time":"2023-08-09T13:53:46.762733+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"129","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"cbd49ecd-e896-4fbd-864e-5c37a829b165","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":127,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6800","nonce":502667335},{"type":"v1","addr":"172.21.15.43:6801","nonce":502667335}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6802","nonce":502667335},{"type":"v1","addr":"172.21.15.43:6803","nonce":502667335}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6806","nonce":502667335},{"type":"v1","addr":"172.21.15.43:6807","nonce":502667335}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6804","nonce":502667335},{"type":"v1","addr":"172.21.15.43:6805","nonce":502667335}]},"public_addr":"172.21.15.43:6801/502667335","cluster_addr":"172.21.15.43:6803/502667335","heartbeat_back_addr":"172.21.15.43:6807/502667335","heartbeat_front_addr":"172.21.15.43:6805/502667335","state":["exists","up"]},{"osd":1,"uuid":"75c7e0fe-1ec7-4fe8-a2f2-3661a65b8500","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":131,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6808","nonce":3289134610},{"type":"v1","addr":"172.21.15.43:6809","nonce":3289134610}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6810","nonce":3289134610},{"type":"v1","addr":"172.21.15.43:6812","nonce":3289134610}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6821","nonce":3289134610},{"type":"v1","addr":"172.21.15.43:6824","nonce":3289134610}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6815","nonce":3289134610},{"type":"v1","addr":"172.21.15.43:6818","nonce":3289134610}]},"public_addr":"172.21.15.43:6809/3289134610","cluster_addr":"172.21.15.43:6812/3289134610","heartbeat_back_addr":"172.21.15.43:6824/3289134610","heartbeat_front_addr":"172.21.15.43:6818/3289134610","state":["exists","up"]},{"osd":2,"uuid":"bb5c41e9-7f5f-4125-9b01-3bb8bfc2ea83","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":127,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6811","nonce":4079709736},{"type":"v1","addr":"172.21.15.43:6814","nonce":4079709736}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6817","nonce":4079709736},{"type":"v1","addr":"172.21.15.43:6820","nonce":4079709736}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6829","nonce":4079709736},{"type":"v1","addr":"172.21.15.43:6831","nonce":4079709736}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6823","nonce":4079709736},{"type":"v1","addr":"172.21.15.43:6826","nonce":4079709736}]},"public_addr":"172.21.15.43:6814/4079709736","cluster_addr":"172.21.15.43:6820/4079709736","heartbeat_back_addr":"172.21.15.43:6831/4079709736","heartbeat_front_addr":"172.21.15.43:6826/4079709736","state":["exists","up"]},{"osd":3,"uuid":"bfe4c8b4-694a-465c-87e6-2b2aab852613","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":127,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6813","nonce":1709538759},{"type":"v1","addr":"172.21.15.43:6816","nonce":1709538759}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6819","nonce":1709538759},{"type":"v1","addr":"172.21.15.43:6822","nonce":1709538759}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6828","nonce":1709538759},{"type":"v1","addr":"172.21.15.43:6830","nonce":1709538759}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6825","nonce":1709538759},{"type":"v1","addr":"172.21.15.43:6827","nonce":1709538759}]},"public_addr":"172.21.15.43:6816/1709538759","cluster_addr":"172.21.15.43:6822/1709538759","heartbeat_back_addr":"172.21.15.43:6830/1709538759","heartbeat_front_addr":"172.21.15.43:6827/1709538759","state":["exists","up"]},{"osd":4,"uuid":"fa2e8a77-aba1-4756-853e-0d174a5bdeba","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":127,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6800","nonce":4196866481},{"type":"v1","addr":"172.21.15.203:6801","nonce":4196866481}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6802","nonce":4196866481},{"type":"v1","addr":"172.21.15.203:6803","nonce":4196866481}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6806","nonce":4196866481},{"type":"v1","addr":"172.21.15.203:6807","nonce":4196866481}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6804","nonce":4196866481},{"type":"v1","addr":"172.21.15.203:6805","nonce":4196866481}]},"public_addr":"172.21.15.203:6801/4196866481","cluster_addr":"172.21.15.203:6803/4196866481","heartbeat_back_addr":"172.21.15.203:6807/4196866481","heartbeat_front_addr":"172.21.15.203:6805/4196866481","state":["exists","up"]},{"osd":5,"uuid":"81811e4b-4d24-4f51-a82c-f1d700a2a00d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":127,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6816","nonce":3059290081},{"type":"v1","addr":"172.21.15.203:6817","nonce":3059290081}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6818","nonce":3059290081},{"type":"v1","addr":"172.21.15.203:6819","nonce":3059290081}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6822","nonce":3059290081},{"type":"v1","addr":"172.21.15.203:6823","nonce":3059290081}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6820","nonce":3059290081},{"type":"v1","addr":"172.21.15.203:6821","nonce":3059290081}]},"public_addr":"172.21.15.203:6817/3059290081","cluster_addr":"172.21.15.203:6819/3059290081","heartbeat_back_addr":"172.21.15.203:6823/3059290081","heartbeat_front_addr":"172.21.15.203:6821/3059290081","state":["exists","up"]},{"osd":6,"uuid":"307bac4d-6a5b-443d-af75-56d695d7674f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":131,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6824","nonce":512155569},{"type":"v1","addr":"172.21.15.203:6825","nonce":512155569}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6826","nonce":512155569},{"type":"v1","addr":"172.21.15.203:6827","nonce":512155569}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6830","nonce":512155569},{"type":"v1","addr":"172.21.15.203:6831","nonce":512155569}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6828","nonce":512155569},{"type":"v1","addr":"172.21.15.203:6829","nonce":512155569}]},"public_addr":"172.21.15.203:6825/512155569","cluster_addr":"172.21.15.203:6827/512155569","heartbeat_back_addr":"172.21.15.203:6831/512155569","heartbeat_front_addr":"172.21.15.203:6829/512155569","state":["exists","up"]},{"osd":7,"uuid":"a95dc7ca-9f1b-4295-af70-68a22a2d7d5e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":127,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6808","nonce":3331450438},{"type":"v1","addr":"172.21.15.203:6809","nonce":3331450438}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6810","nonce":3331450438},{"type":"v1","addr":"172.21.15.203:6811","nonce":3331450438}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6814","nonce":3331450438},{"type":"v1","addr":"172.21.15.203:6815","nonce":3331450438}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6812","nonce":3331450438},{"type":"v1","addr":"172.21.15.203:6813","nonce":3331450438}]},"public_addr":"172.21.15.203:6809/3331450438","cluster_addr":"172.21.15.203:6811/3331450438","heartbeat_back_addr":"172.21.15.203:6815/3331450438","heartbeat_front_addr":"172.21.15.203:6813/3331450438","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-09T13:33:50.547652+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-09T13:33:50.589469+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-09T13:33:50.561724+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-09T13:33:50.599650+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-09T13:33:50.411855+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-09T13:33:50.219681+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-09T13:33:50.504664+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-09T13:33:50.441248+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"17.5","mappings":[{"from":0,"to":5}]},{"pgid":"17.f","mappings":[{"from":0,"to":2}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.203:6833/2527059680":"2023-08-10T13:54:03.238790+0000","172.21.15.203:6832/2527059680":"2023-08-10T13:54:03.238790+0000","172.21.15.203:0/431304609":"2023-08-10T13:54:03.238790+0000","172.21.15.203:0/826469783":"2023-08-10T13:54:03.238790+0000","172.21.15.203:0/1456953448":"2023-08-10T13:54:03.238790+0000","172.21.15.203:0/2004944696":"2023-08-10T13:54:03.238790+0000","172.21.15.203:0/916024371":"2023-08-10T13:54:03.238790+0000","172.21.15.203:0/4209557762":"2023-08-10T13:54:03.238790+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":7,"snaps":[{"begin":104,"length":1},{"begin":221,"length":1}]}],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-08-09T13:55:05.093 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:05.092+0000 7fcedbfff640 1 -- 172.21.15.43:0/2189685592 >> [v2:172.21.15.203:6832/1796635339,v1:172.21.15.203:6833/1796635339] conn(0x7fcebc06a290 msgr2=0x7fcebc06c750 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-08-09T13:55:05.093 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:05.092+0000 7fcedbfff640 1 --2- 172.21.15.43:0/2189685592 >> [v2:172.21.15.203:6832/1796635339,v1:172.21.15.203:6833/1796635339] conn(0x7fcebc06a290 0x7fcebc06c750 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:55:05.093 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:05.092+0000 7fcedbfff640 1 -- 172.21.15.43:0/2189685592 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fcedc070b00 msgr2=0x7fcedc0ce280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:55:05.094 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:05.092+0000 7fcedbfff640 1 --2- 172.21.15.43:0/2189685592 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fcedc070b00 0x7fcedc0ce280 secure :-1 s=READY pgs=1570 cs=0 l=1 rev1=1 crypto rx=0x7fcec400e9b0 tx=0x7fcec400ee70 comp rx=0 tx=0).stop 2023-08-09T13:55:05.094 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:05.092+0000 7fcedbfff640 1 -- 172.21.15.43:0/2189685592 shutdown_connections 2023-08-09T13:55:05.094 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:05.092+0000 7fcedbfff640 1 --2- 172.21.15.43:0/2189685592 >> [v2:172.21.15.203:6832/1796635339,v1:172.21.15.203:6833/1796635339] conn(0x7fcebc06a290 0x7fcebc06c750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:55:05.094 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:05.092+0000 7fcedbfff640 1 --2- 172.21.15.43:0/2189685592 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fcedc0ce7c0 0x7fcedc0af5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:55:05.095 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:05.092+0000 7fcedbfff640 1 --2- 172.21.15.43:0/2189685592 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fcedc070b00 0x7fcedc0ce280 unknown :-1 s=CLOSED pgs=1570 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:55:05.095 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:05.092+0000 7fcedbfff640 1 --2- 172.21.15.43:0/2189685592 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fcedc06c530 0x7fcedc14cff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:55:05.095 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:05.092+0000 7fcedbfff640 1 -- 172.21.15.43:0/2189685592 >> 172.21.15.43:0/2189685592 conn(0x7fcedc06a8d0 msgr2=0x7fcedc151500 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-09T13:55:05.095 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:05.092+0000 7fcedbfff640 1 -- 172.21.15.43:0/2189685592 shutdown_connections 2023-08-09T13:55:05.095 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:05.092+0000 7fcedbfff640 1 -- 172.21.15.43:0/2189685592 wait complete. 2023-08-09T13:55:05.113 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': 'rbd', 'create_time': '2023-08-09T13:33:54.365137+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}, 'read_balance': {'score_acting': 2, 'score_stable': 2, 'optimal_score': 1, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': '.mgr', 'create_time': '2023-08-09T13:33:54.831803+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 16, 'pool_name': 'cephfs_metadata', 'create_time': '2023-08-09T13:53:46.124616+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '128', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.75, 'score_stable': 1.75, 'optimal_score': 1, 'raw_score_acting': 1.75, 'raw_score_stable': 1.75, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 17, 'pool_name': 'cephfs_data', 'create_time': '2023-08-09T13:53:46.762733+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '129', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.75, 'score_stable': 1.75, 'optimal_score': 1, 'raw_score_acting': 1.75, 'raw_score_stable': 1.75, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-08-09T13:55:05.114 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get rbd pg_num 2023-08-09T13:55:05.198 DEBUG:teuthology.orchestra.run.smithi043:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-08-09T13:55:05.204 DEBUG:teuthology.orchestra.run.smithi203:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-08-09T13:55:05.324 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:05.322+0000 7fc339dd5640 1 Processor -- start 2023-08-09T13:55:05.324 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:05.323+0000 7fc339dd5640 1 -- start start 2023-08-09T13:55:05.326 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:05.324+0000 7fc339dd5640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fc33415c550 0x7fc3340cd6d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:55:05.326 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:05.324+0000 7fc339dd5640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7fc3340d04b0 0x7fc3340b3ee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:55:05.326 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:05.324+0000 7fc339dd5640 1 -- --> v1:172.21.15.43:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc3340b4870 con 0x7fc3340b4420 2023-08-09T13:55:05.326 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:05.324+0000 7fc339dd5640 1 -- --> v2:172.21.15.43:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc3340b7bd0 con 0x7fc3340d04b0 2023-08-09T13:55:05.326 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:05.324+0000 7fc339dd5640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc3340d1c00 con 0x7fc33415c550 2023-08-09T13:55:05.326 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:05.324+0000 7fc332ffd640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7fc3340d04b0 0x7fc3340b3ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:55:05.327 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:05.324+0000 7fc332ffd640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7fc3340d04b0 0x7fc3340b3ee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:56362/0 (socket says 172.21.15.43:56362) 2023-08-09T13:55:05.327 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:05.324+0000 7fc332ffd640 1 -- 172.21.15.43:0/3256750275 learned_addr learned my addr 172.21.15.43:0/3256750275 (peer_addr_for_me v2:172.21.15.43:0/0) 2023-08-09T13:55:05.327 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:05.324+0000 7fc333fff640 1 --1- 172.21.15.43:0/3256750275 >> v1:172.21.15.43:6789/0 conn(0x7fc3340b4420 0x7fc33406a760 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.43:6789/0 says I am v1:172.21.15.43:42134/0 (socket says 172.21.15.43:42134) 2023-08-09T13:55:05.327 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:05.325+0000 7fc3337fe640 1 --2- 172.21.15.43:0/3256750275 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fc33415c550 0x7fc3340cd6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:55:05.328 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:05.325+0000 7fc332ffd640 1 -- 172.21.15.43:0/3256750275 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fc33415c550 msgr2=0x7fc3340cd6d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-09T13:55:05.328 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:05.325+0000 7fc332ffd640 1 --2- 172.21.15.43:0/3256750275 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fc33415c550 0x7fc3340cd6d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:55:05.328 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:05.325+0000 7fc332ffd640 1 -- 172.21.15.43:0/3256750275 >> v1:172.21.15.43:6789/0 conn(0x7fc3340b4420 legacy=0x7fc33406a760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:55:05.328 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:05.325+0000 7fc3327fc640 1 -- 172.21.15.43:0/3256750275 <== mon.0 v1:172.21.15.43:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1169848573 0 0) 0x7fc32c001490 con 0x7fc3340b4420 2023-08-09T13:55:05.329 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:05.325+0000 7fc332ffd640 1 -- 172.21.15.43:0/3256750275 --> v2:172.21.15.43:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc33415c930 con 0x7fc3340d04b0 2023-08-09T13:55:05.329 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:05.326+0000 7fc332ffd640 1 --2- 172.21.15.43:0/3256750275 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc3340d04b0 0x7fc3340b3ee0 secure :-1 s=READY pgs=1572 cs=0 l=1 rev1=1 crypto rx=0x7fc328009870 tx=0x7fc32802b980 comp rx=0 tx=0).ready entity=mon.0 client_cookie=47649c6d1f22d1b server_cookie=0 in_seq=0 out_seq=0 2023-08-09T13:55:05.329 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:05.327+0000 7fc3327fc640 1 -- 172.21.15.43:0/3256750275 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc328030840 con 0x7fc3340d04b0 2023-08-09T13:55:05.329 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:05.327+0000 7fc3327fc640 1 -- 172.21.15.43:0/3256750275 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc3280309a0 con 0x7fc3340d04b0 2023-08-09T13:55:05.329 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:05.327+0000 7fc3327fc640 1 -- 172.21.15.43:0/3256750275 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc328030d30 con 0x7fc3340d04b0 2023-08-09T13:55:05.330 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:05.327+0000 7fc339dd5640 1 -- 172.21.15.43:0/3256750275 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc3340d04b0 msgr2=0x7fc3340b3ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:55:05.330 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:05.327+0000 7fc339dd5640 1 --2- 172.21.15.43:0/3256750275 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc3340d04b0 0x7fc3340b3ee0 secure :-1 s=READY pgs=1572 cs=0 l=1 rev1=1 crypto rx=0x7fc328009870 tx=0x7fc32802b980 comp rx=0 tx=0).stop 2023-08-09T13:55:05.330 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:05.327+0000 7fc339dd5640 1 -- 172.21.15.43:0/3256750275 shutdown_connections 2023-08-09T13:55:05.330 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:05.327+0000 7fc339dd5640 1 --2- 172.21.15.43:0/3256750275 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc3340d04b0 0x7fc3340b3ee0 unknown :-1 s=CLOSED pgs=1572 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:55:05.330 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:05.327+0000 7fc339dd5640 1 --2- 172.21.15.43:0/3256750275 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fc33415c550 0x7fc3340cd6d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:55:05.331 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:05.327+0000 7fc339dd5640 1 -- 172.21.15.43:0/3256750275 >> 172.21.15.43:0/3256750275 conn(0x7fc334071520 msgr2=0x7fc3340ceee0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-09T13:55:05.331 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:05.328+0000 7fc339dd5640 1 -- 172.21.15.43:0/3256750275 shutdown_connections 2023-08-09T13:55:05.331 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:05.328+0000 7fc339dd5640 1 -- 172.21.15.43:0/3256750275 wait complete. 2023-08-09T13:55:05.332 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:05.329+0000 7fc339dd5640 1 Processor -- start 2023-08-09T13:55:05.332 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:05.329+0000 7fc339dd5640 1 -- start start 2023-08-09T13:55:05.332 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:05.330+0000 7fc339dd5640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc3340b4420 0x7fc3341589d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:55:05.332 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:05.330+0000 7fc339dd5640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fc3340d04b0 0x7fc3340c70f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:55:05.333 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:05.330+0000 7fc339dd5640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fc33415c550 0x7fc3340c7630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:55:05.333 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:05.330+0000 7fc339dd5640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc3340aa880 con 0x7fc3340b4420 2023-08-09T13:55:05.333 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:05.330+0000 7fc339dd5640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc3340caeb0 con 0x7fc33415c550 2023-08-09T13:55:05.333 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:05.330+0000 7fc339dd5640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc3340cb190 con 0x7fc3340d04b0 2023-08-09T13:55:05.333 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:05.331+0000 7fc3337fe640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc3340b4420 0x7fc3341589d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:55:05.333 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:05.331+0000 7fc3337fe640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc3340b4420 0x7fc3341589d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:56364/0 (socket says 172.21.15.43:56364) 2023-08-09T13:55:05.333 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:05.331+0000 7fc3337fe640 1 -- 172.21.15.43:0/3400158323 learned_addr learned my addr 172.21.15.43:0/3400158323 (peer_addr_for_me v2:172.21.15.43:0/0) 2023-08-09T13:55:05.334 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:05.331+0000 7fc333fff640 1 --2- 172.21.15.43:0/3400158323 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fc33415c550 0x7fc3340c7630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:55:05.334 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:05.331+0000 7fc332ffd640 1 --2- 172.21.15.43:0/3400158323 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fc3340d04b0 0x7fc3340c70f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:55:05.335 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:05.331+0000 7fc3337fe640 1 -- 172.21.15.43:0/3400158323 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fc3340d04b0 msgr2=0x7fc3340c70f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:55:05.335 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:05.331+0000 7fc3337fe640 1 --2- 172.21.15.43:0/3400158323 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fc3340d04b0 0x7fc3340c70f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:55:05.335 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:05.331+0000 7fc3337fe640 1 -- 172.21.15.43:0/3400158323 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fc33415c550 msgr2=0x7fc3340c7630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:55:05.335 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:05.331+0000 7fc3337fe640 1 --2- 172.21.15.43:0/3400158323 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fc33415c550 0x7fc3340c7630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:55:05.335 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:05.331+0000 7fc3337fe640 1 -- 172.21.15.43:0/3400158323 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc3280094d0 con 0x7fc3340b4420 2023-08-09T13:55:05.335 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:05.331+0000 7fc3337fe640 1 --2- 172.21.15.43:0/3400158323 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc3340b4420 0x7fc3341589d0 secure :-1 s=READY pgs=1573 cs=0 l=1 rev1=1 crypto rx=0x7fc32400e870 tx=0x7fc32400ed30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-09T13:55:05.336 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:05.332+0000 7fc31bfff640 1 -- 172.21.15.43:0/3400158323 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc3240094f0 con 0x7fc3340b4420 2023-08-09T13:55:05.336 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:05.332+0000 7fc31bfff640 1 -- 172.21.15.43:0/3400158323 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc32400c940 con 0x7fc3340b4420 2023-08-09T13:55:05.336 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:05.332+0000 7fc31bfff640 1 -- 172.21.15.43:0/3400158323 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc32400ccf0 con 0x7fc3340b4420 2023-08-09T13:55:05.336 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:05.332+0000 7fc339dd5640 1 -- 172.21.15.43:0/3400158323 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc3340cb470 con 0x7fc3340b4420 2023-08-09T13:55:05.336 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:05.332+0000 7fc339dd5640 1 -- 172.21.15.43:0/3400158323 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc3340cba30 con 0x7fc3340b4420 2023-08-09T13:55:05.337 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:05.333+0000 7fc339dd5640 1 -- 172.21.15.43:0/3400158323 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc3340739f0 con 0x7fc3340b4420 2023-08-09T13:55:05.337 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:05.334+0000 7fc31bfff640 1 -- 172.21.15.43:0/3400158323 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 63) v1 ==== 88557+0+0 (secure 0 0 0) 0x7fc324013470 con 0x7fc3340b4420 2023-08-09T13:55:05.337 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:05.335+0000 7fc31bfff640 1 --2- 172.21.15.43:0/3400158323 >> [v2:172.21.15.203:6832/1796635339,v1:172.21.15.203:6833/1796635339] conn(0x7fc32006a290 0x7fc32006c750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:55:05.338 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:05.335+0000 7fc31bfff640 1 -- 172.21.15.43:0/3400158323 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(132..132 src has 1..132) v4 ==== 6828+0+0 (secure 0 0 0) 0x7fc324088d00 con 0x7fc3340b4420 2023-08-09T13:55:05.338 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:05.335+0000 7fc332ffd640 1 -- 172.21.15.43:0/3400158323 >> [v2:172.21.15.203:6832/1796635339,v1:172.21.15.203:6833/1796635339] conn(0x7fc32006a290 msgr2=0x7fc32006c750 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.203:6832/1796635339 2023-08-09T13:55:05.338 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:05.335+0000 7fc332ffd640 1 --2- 172.21.15.43:0/3400158323 >> [v2:172.21.15.203:6832/1796635339,v1:172.21.15.203:6833/1796635339] conn(0x7fc32006a290 0x7fc32006c750 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-08-09T13:55:05.345 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:05.343+0000 7fc31bfff640 1 -- 172.21.15.43:0/3400158323 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178066 (secure 0 0 0) 0x7fc324053e30 con 0x7fc3340b4420 2023-08-09T13:55:05.353 INFO:tasks.ceph.mgr.x.smithi203.stderr:2023-08-09T13:55:05.350+0000 7f8eeb831040 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2023-08-09T13:55:05.384 INFO:tasks.ceph.mgr.y.smithi043.stderr:2023-08-09T13:55:05.383+0000 7f106ca0f040 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2023-08-09T13:55:05.480 INFO:tasks.ceph.mgr.x.smithi203.stderr:2023-08-09T13:55:05.477+0000 7f8eeb831040 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2023-08-09T13:55:05.509 INFO:tasks.ceph.mgr.y.smithi043.stderr:2023-08-09T13:55:05.507+0000 7f106ca0f040 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2023-08-09T13:55:05.537 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:05.536+0000 7fc332ffd640 1 -- 172.21.15.43:0/3400158323 >> [v2:172.21.15.203:6832/1796635339,v1:172.21.15.203:6833/1796635339] conn(0x7fc32006a290 msgr2=0x7fc32006c750 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.203:6832/1796635339 2023-08-09T13:55:05.538 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:05.536+0000 7fc332ffd640 1 --2- 172.21.15.43:0/3400158323 >> [v2:172.21.15.203:6832/1796635339,v1:172.21.15.203:6833/1796635339] conn(0x7fc32006a290 0x7fc32006c750 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2023-08-09T13:55:05.539 INFO:tasks.ceph.mgr.x.smithi203.stderr:2023-08-09T13:55:05.537+0000 7f8eeb831040 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2023-08-09T13:55:05.541 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:05.540+0000 7fc339dd5640 1 -- 172.21.15.43:0/3400158323 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"} v 0) v1 -- 0x7fc3340b3ce0 con 0x7fc3340b4420 2023-08-09T13:55:05.542 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:05.541+0000 7fc31bfff640 1 -- 172.21.15.43:0/3400158323 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"}]=0 v132) v1 ==== 92+0+10 (secure 0 0 0) 0x7fc324057450 con 0x7fc3340b4420 2023-08-09T13:55:05.542 INFO:teuthology.orchestra.run.smithi043.stdout:pg_num: 8 2023-08-09T13:55:05.544 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:05.543+0000 7fc339dd5640 1 -- 172.21.15.43:0/3400158323 >> [v2:172.21.15.203:6832/1796635339,v1:172.21.15.203:6833/1796635339] conn(0x7fc32006a290 msgr2=0x7fc32006c750 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-08-09T13:55:05.545 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:05.543+0000 7fc339dd5640 1 --2- 172.21.15.43:0/3400158323 >> [v2:172.21.15.203:6832/1796635339,v1:172.21.15.203:6833/1796635339] conn(0x7fc32006a290 0x7fc32006c750 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:55:05.545 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:05.543+0000 7fc339dd5640 1 -- 172.21.15.43:0/3400158323 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc3340b4420 msgr2=0x7fc3341589d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:55:05.545 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:05.543+0000 7fc339dd5640 1 --2- 172.21.15.43:0/3400158323 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc3340b4420 0x7fc3341589d0 secure :-1 s=READY pgs=1573 cs=0 l=1 rev1=1 crypto rx=0x7fc32400e870 tx=0x7fc32400ed30 comp rx=0 tx=0).stop 2023-08-09T13:55:05.545 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:05.543+0000 7fc339dd5640 1 -- 172.21.15.43:0/3400158323 shutdown_connections 2023-08-09T13:55:05.545 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:05.543+0000 7fc339dd5640 1 --2- 172.21.15.43:0/3400158323 >> [v2:172.21.15.203:6832/1796635339,v1:172.21.15.203:6833/1796635339] conn(0x7fc32006a290 0x7fc32006c750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:55:05.545 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:05.543+0000 7fc339dd5640 1 --2- 172.21.15.43:0/3400158323 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fc33415c550 0x7fc3340c7630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:55:05.546 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:05.543+0000 7fc339dd5640 1 --2- 172.21.15.43:0/3400158323 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fc3340d04b0 0x7fc3340c70f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:55:05.546 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:05.543+0000 7fc339dd5640 1 --2- 172.21.15.43:0/3400158323 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc3340b4420 0x7fc3341589d0 unknown :-1 s=CLOSED pgs=1573 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:55:05.546 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:05.543+0000 7fc339dd5640 1 -- 172.21.15.43:0/3400158323 >> 172.21.15.43:0/3400158323 conn(0x7fc334071520 msgr2=0x7fc3340b58d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-09T13:55:05.546 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:05.543+0000 7fc339dd5640 1 -- 172.21.15.43:0/3400158323 shutdown_connections 2023-08-09T13:55:05.546 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:05.543+0000 7fc339dd5640 1 -- 172.21.15.43:0/3400158323 wait complete. 2023-08-09T13:55:05.561 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-08-09T13:55:05.569 INFO:tasks.ceph.mgr.y.smithi043.stderr:2023-08-09T13:55:05.568+0000 7f106ca0f040 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2023-08-09T13:55:05.592 INFO:tasks.ceph.mgr.x.smithi203.stderr:2023-08-09T13:55:05.589+0000 7f8eeb831040 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2023-08-09T13:55:05.622 INFO:tasks.ceph.mgr.y.smithi043.stderr:2023-08-09T13:55:05.620+0000 7f106ca0f040 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2023-08-09T13:55:05.781 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:05.780+0000 7fe46bf84640 1 Processor -- start 2023-08-09T13:55:05.781 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:05.780+0000 7fe46bf84640 1 -- start start 2023-08-09T13:55:05.783 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:05.782+0000 7fe46bf84640 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7fe4640cee90 0x7fe4640cdd90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:55:05.784 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:05.782+0000 7fe46bf84640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7fe4640ccbb0 0x7fe4640cd060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:55:05.784 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:05.782+0000 7fe46bf84640 1 -- --> v1:172.21.15.203:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe4640cb240 con 0x7fe4640afb20 2023-08-09T13:55:05.784 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:05.782+0000 7fe46bf84640 1 -- --> v2:172.21.15.43:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe4640cd5a0 con 0x7fe4640ccbb0 2023-08-09T13:55:05.784 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:05.782+0000 7fe46bf84640 1 -- --> v2:172.21.15.203:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe4640b1440 con 0x7fe4640cee90 2023-08-09T13:55:05.784 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:05.782+0000 7fe46951f640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7fe4640ccbb0 0x7fe4640cd060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:55:05.785 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:05.782+0000 7fe46951f640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7fe4640ccbb0 0x7fe4640cd060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:56374/0 (socket says 172.21.15.43:56374) 2023-08-09T13:55:05.785 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:05.782+0000 7fe46951f640 1 -- 172.21.15.43:0/2066141657 learned_addr learned my addr 172.21.15.43:0/2066141657 (peer_addr_for_me v2:172.21.15.43:0/0) 2023-08-09T13:55:05.786 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:05.783+0000 7fe469d20640 1 --2- 172.21.15.43:0/2066141657 >> v2:172.21.15.203:3300/0 conn(0x7fe4640cee90 0x7fe4640cdd90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:55:05.786 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:05.783+0000 7fe46951f640 1 -- 172.21.15.43:0/2066141657 >> v2:172.21.15.203:3300/0 conn(0x7fe4640cee90 msgr2=0x7fe4640cdd90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-09T13:55:05.786 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:05.783+0000 7fe46951f640 1 --2- 172.21.15.43:0/2066141657 >> v2:172.21.15.203:3300/0 conn(0x7fe4640cee90 0x7fe4640cdd90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:55:05.786 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:05.783+0000 7fe46951f640 1 -- 172.21.15.43:0/2066141657 >> v1:172.21.15.203:6789/0 conn(0x7fe4640afb20 legacy=0x7fe4640aff30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-09T13:55:05.786 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:05.783+0000 7fe46951f640 1 -- 172.21.15.43:0/2066141657 --> v2:172.21.15.43:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe4640cf270 con 0x7fe4640ccbb0 2023-08-09T13:55:05.786 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:05.784+0000 7fe46951f640 1 --2- 172.21.15.43:0/2066141657 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fe4640ccbb0 0x7fe4640cd060 secure :-1 s=READY pgs=1574 cs=0 l=1 rev1=1 crypto rx=0x7fe4640b1580 tx=0x7fe45402b870 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b8f3c88ab619e416 server_cookie=0 in_seq=0 out_seq=0 2023-08-09T13:55:05.787 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:05.784+0000 7fe468d1e640 1 -- 172.21.15.43:0/2066141657 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe4540317d0 con 0x7fe4640ccbb0 2023-08-09T13:55:05.787 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:05.784+0000 7fe468d1e640 1 -- 172.21.15.43:0/2066141657 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe454031930 con 0x7fe4640ccbb0 2023-08-09T13:55:05.787 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:05.784+0000 7fe468d1e640 1 -- 172.21.15.43:0/2066141657 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe454031cc0 con 0x7fe4640ccbb0 2023-08-09T13:55:05.788 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:05.785+0000 7fe46bf84640 1 -- 172.21.15.43:0/2066141657 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fe4640ccbb0 msgr2=0x7fe4640cd060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:55:05.789 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:05.785+0000 7fe46bf84640 1 --2- 172.21.15.43:0/2066141657 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fe4640ccbb0 0x7fe4640cd060 secure :-1 s=READY pgs=1574 cs=0 l=1 rev1=1 crypto rx=0x7fe4640b1580 tx=0x7fe45402b870 comp rx=0 tx=0).stop 2023-08-09T13:55:05.789 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:05.785+0000 7fe46bf84640 1 -- 172.21.15.43:0/2066141657 shutdown_connections 2023-08-09T13:55:05.789 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:05.785+0000 7fe46bf84640 1 --2- 172.21.15.43:0/2066141657 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fe4640ccbb0 0x7fe4640cd060 unknown :-1 s=CLOSED pgs=1574 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:55:05.789 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:05.785+0000 7fe46bf84640 1 --2- 172.21.15.43:0/2066141657 >> v2:172.21.15.203:3300/0 conn(0x7fe4640cee90 0x7fe4640cdd90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:55:05.790 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:05.785+0000 7fe46bf84640 1 -- 172.21.15.43:0/2066141657 >> 172.21.15.43:0/2066141657 conn(0x7fe464089fe0 msgr2=0x7fe4640d01c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-09T13:55:05.790 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:05.785+0000 7fe46bf84640 1 -- 172.21.15.43:0/2066141657 shutdown_connections 2023-08-09T13:55:05.790 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:05.785+0000 7fe46bf84640 1 -- 172.21.15.43:0/2066141657 wait complete. 2023-08-09T13:55:05.791 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:05.787+0000 7fe46bf84640 1 Processor -- start 2023-08-09T13:55:05.791 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:05.787+0000 7fe46bf84640 1 -- start start 2023-08-09T13:55:05.794 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:05.788+0000 7fe46bf84640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fe4640afb20 0x7fe4641539b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:55:05.795 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:05.788+0000 7fe46bf84640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fe4640ccbb0 0x7fe4640cbdb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:55:05.795 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:05.788+0000 7fe46bf84640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fe4640cee90 0x7fe4640cc2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:55:05.795 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:05.788+0000 7fe46bf84640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe4640b2f10 con 0x7fe4640afb20 2023-08-09T13:55:05.796 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:05.788+0000 7fe46bf84640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe4640b31f0 con 0x7fe4640cee90 2023-08-09T13:55:05.796 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:05.788+0000 7fe46bf84640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe4640b34d0 con 0x7fe4640ccbb0 2023-08-09T13:55:05.796 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:05.788+0000 7fe469d20640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fe4640afb20 0x7fe4641539b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:55:05.796 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:05.788+0000 7fe469d20640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fe4640afb20 0x7fe4641539b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:56386/0 (socket says 172.21.15.43:56386) 2023-08-09T13:55:05.797 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:05.788+0000 7fe469d20640 1 -- 172.21.15.43:0/223306092 learned_addr learned my addr 172.21.15.43:0/223306092 (peer_addr_for_me v2:172.21.15.43:0/0) 2023-08-09T13:55:05.797 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:05.788+0000 7fe46951f640 1 --2- 172.21.15.43:0/223306092 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fe4640ccbb0 0x7fe4640cbdb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:55:05.797 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:05.789+0000 7fe469d20640 1 -- 172.21.15.43:0/223306092 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fe4640ccbb0 msgr2=0x7fe4640cbdb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:55:05.798 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:05.789+0000 7fe469d20640 1 --2- 172.21.15.43:0/223306092 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fe4640ccbb0 0x7fe4640cbdb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:55:05.798 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:05.789+0000 7fe469d20640 1 -- 172.21.15.43:0/223306092 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fe4640cee90 msgr2=0x7fe4640cc2f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:55:05.798 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:05.789+0000 7fe469d20640 1 --2- 172.21.15.43:0/223306092 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fe4640cee90 0x7fe4640cc2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:55:05.798 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:05.789+0000 7fe469d20640 1 -- 172.21.15.43:0/223306092 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe4540094d0 con 0x7fe4640afb20 2023-08-09T13:55:05.799 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:05.789+0000 7fe46951f640 1 --2- 172.21.15.43:0/223306092 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fe4640ccbb0 0x7fe4640cbdb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-08-09T13:55:05.799 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:05.789+0000 7fe469d20640 1 --2- 172.21.15.43:0/223306092 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fe4640afb20 0x7fe4641539b0 secure :-1 s=READY pgs=1575 cs=0 l=1 rev1=1 crypto rx=0x7fe46000d7d0 tx=0x7fe46000dc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-09T13:55:05.799 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:05.789+0000 7fe45a7fc640 1 -- 172.21.15.43:0/223306092 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe4600096c0 con 0x7fe4640afb20 2023-08-09T13:55:05.800 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:05.789+0000 7fe45a7fc640 1 -- 172.21.15.43:0/223306092 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe460009820 con 0x7fe4640afb20 2023-08-09T13:55:05.800 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:05.789+0000 7fe45a7fc640 1 -- 172.21.15.43:0/223306092 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe460009bd0 con 0x7fe4640afb20 2023-08-09T13:55:05.800 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:05.789+0000 7fe46bf84640 1 -- 172.21.15.43:0/223306092 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe4640b37b0 con 0x7fe4640afb20 2023-08-09T13:55:05.800 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:05.789+0000 7fe46bf84640 1 -- 172.21.15.43:0/223306092 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe464153ef0 con 0x7fe4640afb20 2023-08-09T13:55:05.801 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:05.790+0000 7fe46bf84640 1 -- 172.21.15.43:0/223306092 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe46415dfc0 con 0x7fe4640afb20 2023-08-09T13:55:05.801 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:05.791+0000 7fe45a7fc640 1 -- 172.21.15.43:0/223306092 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 63) v1 ==== 88557+0+0 (secure 0 0 0) 0x7fe460009d30 con 0x7fe4640afb20 2023-08-09T13:55:05.801 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:05.792+0000 7fe45a7fc640 1 --2- 172.21.15.43:0/223306092 >> [v2:172.21.15.203:6832/1796635339,v1:172.21.15.203:6833/1796635339] conn(0x7fe44806a290 0x7fe44806c750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:55:05.802 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:05.792+0000 7fe45a7fc640 1 -- 172.21.15.43:0/223306092 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(132..132 src has 1..132) v4 ==== 6828+0+0 (secure 0 0 0) 0x7fe460089460 con 0x7fe4640afb20 2023-08-09T13:55:05.802 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:05.792+0000 7fe46951f640 1 -- 172.21.15.43:0/223306092 >> [v2:172.21.15.203:6832/1796635339,v1:172.21.15.203:6833/1796635339] conn(0x7fe44806a290 msgr2=0x7fe44806c750 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.203:6832/1796635339 2023-08-09T13:55:05.802 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:05.792+0000 7fe46951f640 1 --2- 172.21.15.43:0/223306092 >> [v2:172.21.15.203:6832/1796635339,v1:172.21.15.203:6833/1796635339] conn(0x7fe44806a290 0x7fe44806c750 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-08-09T13:55:05.803 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:05.800+0000 7fe45a7fc640 1 -- 172.21.15.43:0/223306092 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178066 (secure 0 0 0) 0x7fe46415dfc0 con 0x7fe4640afb20 2023-08-09T13:55:05.993 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:05.991+0000 7fe46bf84640 1 -- 172.21.15.43:0/223306092 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7fe46415dfc0 con 0x7fe4640afb20 2023-08-09T13:55:05.994 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:05.992+0000 7fe45a7fc640 1 -- 172.21.15.43:0/223306092 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v132) v1 ==== 93+0+10 (secure 0 0 0) 0x7fe46415dfc0 con 0x7fe4640afb20 2023-08-09T13:55:05.994 INFO:teuthology.orchestra.run.smithi043.stdout:pg_num: 1 2023-08-09T13:55:05.994 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:05.993+0000 7fe46951f640 1 -- 172.21.15.43:0/223306092 >> [v2:172.21.15.203:6832/1796635339,v1:172.21.15.203:6833/1796635339] conn(0x7fe44806a290 msgr2=0x7fe44806c750 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.203:6832/1796635339 2023-08-09T13:55:05.995 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:05.993+0000 7fe46951f640 1 --2- 172.21.15.43:0/223306092 >> [v2:172.21.15.203:6832/1796635339,v1:172.21.15.203:6833/1796635339] conn(0x7fe44806a290 0x7fe44806c750 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2023-08-09T13:55:05.996 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:05.994+0000 7fe46bf84640 1 -- 172.21.15.43:0/223306092 >> [v2:172.21.15.203:6832/1796635339,v1:172.21.15.203:6833/1796635339] conn(0x7fe44806a290 msgr2=0x7fe44806c750 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-08-09T13:55:05.996 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:05.994+0000 7fe46bf84640 1 --2- 172.21.15.43:0/223306092 >> [v2:172.21.15.203:6832/1796635339,v1:172.21.15.203:6833/1796635339] conn(0x7fe44806a290 0x7fe44806c750 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:55:05.996 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:05.994+0000 7fe46bf84640 1 -- 172.21.15.43:0/223306092 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fe4640afb20 msgr2=0x7fe4641539b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:55:05.997 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:05.994+0000 7fe46bf84640 1 --2- 172.21.15.43:0/223306092 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fe4640afb20 0x7fe4641539b0 secure :-1 s=READY pgs=1575 cs=0 l=1 rev1=1 crypto rx=0x7fe46000d7d0 tx=0x7fe46000dc90 comp rx=0 tx=0).stop 2023-08-09T13:55:05.997 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:05.995+0000 7fe46bf84640 1 -- 172.21.15.43:0/223306092 shutdown_connections 2023-08-09T13:55:05.997 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:05.995+0000 7fe46bf84640 1 --2- 172.21.15.43:0/223306092 >> [v2:172.21.15.203:6832/1796635339,v1:172.21.15.203:6833/1796635339] conn(0x7fe44806a290 0x7fe44806c750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:55:05.997 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:05.995+0000 7fe46bf84640 1 --2- 172.21.15.43:0/223306092 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fe4640cee90 0x7fe4640cc2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:55:05.997 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:05.995+0000 7fe46bf84640 1 --2- 172.21.15.43:0/223306092 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fe4640ccbb0 0x7fe4640cbdb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:55:05.997 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:05.995+0000 7fe46bf84640 1 --2- 172.21.15.43:0/223306092 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fe4640afb20 0x7fe4641539b0 unknown :-1 s=CLOSED pgs=1575 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:55:05.997 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:05.995+0000 7fe46bf84640 1 -- 172.21.15.43:0/223306092 >> 172.21.15.43:0/223306092 conn(0x7fe464089fe0 msgr2=0x7fe46406e2b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-09T13:55:05.998 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:05.995+0000 7fe46bf84640 1 -- 172.21.15.43:0/223306092 shutdown_connections 2023-08-09T13:55:05.998 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:05.995+0000 7fe46bf84640 1 -- 172.21.15.43:0/223306092 wait complete. 2023-08-09T13:55:06.017 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-08-09T13:55:06.215 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:06.213+0000 7ff6ecd64640 1 Processor -- start 2023-08-09T13:55:06.215 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:06.214+0000 7ff6ecd64640 1 -- start start 2023-08-09T13:55:06.216 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:06.215+0000 7ff6ecd64640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7ff6e80ce720 0x7ff6e80ceb00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:55:06.216 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:06.215+0000 7ff6ecd64640 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7ff6e80ce080 0x7ff6e815e5b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:55:06.217 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:06.215+0000 7ff6ecd64640 1 -- --> v1:172.21.15.43:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff6e80cd8a0 con 0x7ff6e815eaf0 2023-08-09T13:55:06.217 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:06.215+0000 7ff6ecd64640 1 -- --> v2:172.21.15.203:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff6e80cf350 con 0x7ff6e80ce080 2023-08-09T13:55:06.217 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:06.215+0000 7ff6ecd64640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff6e80cf040 con 0x7ff6e80ce720 2023-08-09T13:55:06.218 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:06.215+0000 7ff6e6d9d640 1 --1- >> v1:172.21.15.43:6789/0 conn(0x7ff6e815eaf0 0x7ff6e80cafa0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.43:6789/0 says I am v1:172.21.15.43:42148/0 (socket says 172.21.15.43:42148) 2023-08-09T13:55:06.219 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:06.215+0000 7ff6e6d9d640 1 -- 172.21.15.43:0/2342911568 learned_addr learned my addr 172.21.15.43:0/2342911568 (peer_addr_for_me v1:172.21.15.43:0/0) 2023-08-09T13:55:06.219 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:06.216+0000 7ff6e659c640 1 --2- 172.21.15.43:0/2342911568 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7ff6e80ce720 0x7ff6e80ceb00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:55:06.219 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:06.216+0000 7ff6ddd9b640 1 --2- 172.21.15.43:0/2342911568 >> v2:172.21.15.203:3300/0 conn(0x7ff6e80ce080 0x7ff6e815e5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:55:06.220 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:06.216+0000 7ff6e5d9b640 1 -- 172.21.15.43:0/2342911568 <== mon.0 v1:172.21.15.43:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3706912377 0 0) 0x7ff6e80cd8a0 con 0x7ff6e815eaf0 2023-08-09T13:55:06.220 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:06.216+0000 7ff6e5d9b640 1 -- 172.21.15.43:0/2342911568 --> v1:172.21.15.43:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ff6d4003520 con 0x7ff6e815eaf0 2023-08-09T13:55:06.220 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:06.216+0000 7ff6e659c640 1 -- 172.21.15.43:0/2342911568 >> v2:172.21.15.203:3300/0 conn(0x7ff6e80ce080 msgr2=0x7ff6e815e5b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-09T13:55:06.220 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:06.216+0000 7ff6e659c640 1 --2- 172.21.15.43:0/2342911568 >> v2:172.21.15.203:3300/0 conn(0x7ff6e80ce080 0x7ff6e815e5b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:55:06.221 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:06.216+0000 7ff6e659c640 1 -- 172.21.15.43:0/2342911568 >> v1:172.21.15.43:6789/0 conn(0x7ff6e815eaf0 legacy=0x7ff6e80cafa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:55:06.221 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:06.216+0000 7ff6e659c640 1 -- 172.21.15.43:0/2342911568 --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff6e8071a70 con 0x7ff6e80ce720 2023-08-09T13:55:06.226 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:06.217+0000 7ff6e659c640 1 --2- 172.21.15.43:0/2342911568 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7ff6e80ce720 0x7ff6e80ceb00 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7ff6d8009790 tx=0x7ff6d802b820 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b4ef244facb835e9 server_cookie=0 in_seq=0 out_seq=0 2023-08-09T13:55:06.226 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:06.218+0000 7ff6e5d9b640 1 -- 172.21.15.43:0/2342911568 <== mon.2 v2:172.21.15.203:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff6d8031780 con 0x7ff6e80ce720 2023-08-09T13:55:06.226 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:06.218+0000 7ff6e5d9b640 1 -- 172.21.15.43:0/2342911568 <== mon.2 v2:172.21.15.203:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff6d80318e0 con 0x7ff6e80ce720 2023-08-09T13:55:06.226 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:06.218+0000 7ff6e5d9b640 1 -- 172.21.15.43:0/2342911568 <== mon.2 v2:172.21.15.203:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff6d8031c50 con 0x7ff6e80ce720 2023-08-09T13:55:06.227 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:06.218+0000 7ff6ecd64640 1 -- 172.21.15.43:0/2342911568 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7ff6e80ce720 msgr2=0x7ff6e80ceb00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:55:06.227 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:06.218+0000 7ff6ecd64640 1 --2- 172.21.15.43:0/2342911568 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7ff6e80ce720 0x7ff6e80ceb00 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7ff6d8009790 tx=0x7ff6d802b820 comp rx=0 tx=0).stop 2023-08-09T13:55:06.227 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:06.218+0000 7ff6ecd64640 1 -- 172.21.15.43:0/2342911568 shutdown_connections 2023-08-09T13:55:06.227 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:06.219+0000 7ff6ecd64640 1 --2- 172.21.15.43:0/2342911568 >> v2:172.21.15.203:3300/0 conn(0x7ff6e80ce080 0x7ff6e815e5b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:55:06.227 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:06.219+0000 7ff6ecd64640 1 --2- 172.21.15.43:0/2342911568 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7ff6e80ce720 0x7ff6e80ceb00 unknown :-1 s=CLOSED pgs=217 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:55:06.228 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:06.219+0000 7ff6ecd64640 1 -- 172.21.15.43:0/2342911568 >> 172.21.15.43:0/2342911568 conn(0x7ff6e80ccd80 msgr2=0x7ff6e80cd190 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-09T13:55:06.228 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:06.219+0000 7ff6ecd64640 1 -- 172.21.15.43:0/2342911568 shutdown_connections 2023-08-09T13:55:06.228 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:06.219+0000 7ff6ecd64640 1 -- 172.21.15.43:0/2342911568 wait complete. 2023-08-09T13:55:06.229 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:06.220+0000 7ff6ecd64640 1 Processor -- start 2023-08-09T13:55:06.229 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:06.220+0000 7ff6ecd64640 1 -- start start 2023-08-09T13:55:06.229 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:06.221+0000 7ff6ecd64640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7ff6e80ce080 0x7ff6e80b2e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:55:06.229 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:06.221+0000 7ff6ecd64640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff6e80ce720 0x7ff6e80b49b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:55:06.229 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:06.221+0000 7ff6ecd64640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7ff6e815eaf0 0x7ff6e80b4ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:55:06.230 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:06.221+0000 7ff6ecd64640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff6e80b2840 con 0x7ff6e80ce720 2023-08-09T13:55:06.230 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:06.221+0000 7ff6ecd64640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff6e806c200 con 0x7ff6e815eaf0 2023-08-09T13:55:06.230 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:06.221+0000 7ff6ecd64640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff6e806b680 con 0x7ff6e80ce080 2023-08-09T13:55:06.230 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:06.222+0000 7ff6ddd9b640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff6e80ce720 0x7ff6e80b49b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:55:06.230 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:06.222+0000 7ff6ddd9b640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff6e80ce720 0x7ff6e80b49b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:56388/0 (socket says 172.21.15.43:56388) 2023-08-09T13:55:06.230 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:06.222+0000 7ff6ddd9b640 1 -- 172.21.15.43:0/4083336984 learned_addr learned my addr 172.21.15.43:0/4083336984 (peer_addr_for_me v2:172.21.15.43:0/0) 2023-08-09T13:55:06.230 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:06.222+0000 7ff6e659c640 1 --2- 172.21.15.43:0/4083336984 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7ff6e80ce080 0x7ff6e80b2e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:55:06.230 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:06.222+0000 7ff6e6d9d640 1 --2- 172.21.15.43:0/4083336984 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7ff6e815eaf0 0x7ff6e80b4ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:55:06.231 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:06.222+0000 7ff6ddd9b640 1 -- 172.21.15.43:0/4083336984 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7ff6e80ce080 msgr2=0x7ff6e80b2e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:55:06.231 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:06.222+0000 7ff6ddd9b640 1 --2- 172.21.15.43:0/4083336984 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7ff6e80ce080 0x7ff6e80b2e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:55:06.231 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:06.222+0000 7ff6ddd9b640 1 -- 172.21.15.43:0/4083336984 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7ff6e815eaf0 msgr2=0x7ff6e80b4ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:55:06.231 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:06.222+0000 7ff6ddd9b640 1 --2- 172.21.15.43:0/4083336984 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7ff6e815eaf0 0x7ff6e80b4ef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:55:06.231 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:06.222+0000 7ff6ddd9b640 1 -- 172.21.15.43:0/4083336984 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff6d80093a0 con 0x7ff6e80ce720 2023-08-09T13:55:06.231 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:06.222+0000 7ff6e6d9d640 1 --2- 172.21.15.43:0/4083336984 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7ff6e815eaf0 0x7ff6e80b4ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-08-09T13:55:06.231 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:06.223+0000 7ff6ddd9b640 1 --2- 172.21.15.43:0/4083336984 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff6e80ce720 0x7ff6e80b49b0 secure :-1 s=READY pgs=1577 cs=0 l=1 rev1=1 crypto rx=0x7ff6d000e870 tx=0x7ff6d000ed30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-09T13:55:06.231 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:06.223+0000 7ff6df7fe640 1 -- 172.21.15.43:0/4083336984 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff6d000ee60 con 0x7ff6e80ce720 2023-08-09T13:55:06.232 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:06.223+0000 7ff6df7fe640 1 -- 172.21.15.43:0/4083336984 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff6d000c940 con 0x7ff6e80ce720 2023-08-09T13:55:06.232 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:06.223+0000 7ff6ecd64640 1 -- 172.21.15.43:0/4083336984 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff6e806b960 con 0x7ff6e80ce720 2023-08-09T13:55:06.232 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:06.223+0000 7ff6df7fe640 1 -- 172.21.15.43:0/4083336984 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff6d000cd10 con 0x7ff6e80ce720 2023-08-09T13:55:06.232 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:06.223+0000 7ff6ecd64640 1 -- 172.21.15.43:0/4083336984 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff6e806bf20 con 0x7ff6e80ce720 2023-08-09T13:55:06.232 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:06.224+0000 7ff6ecd64640 1 -- 172.21.15.43:0/4083336984 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff6e80739f0 con 0x7ff6e80ce720 2023-08-09T13:55:06.233 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:06.225+0000 7ff6df7fe640 1 -- 172.21.15.43:0/4083336984 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 63) v1 ==== 88557+0+0 (secure 0 0 0) 0x7ff6d0013470 con 0x7ff6e80ce720 2023-08-09T13:55:06.233 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:06.227+0000 7ff6df7fe640 1 --2- 172.21.15.43:0/4083336984 >> [v2:172.21.15.203:6832/1796635339,v1:172.21.15.203:6833/1796635339] conn(0x7ff6c806a290 0x7ff6c806c750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:55:06.233 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:06.227+0000 7ff6df7fe640 1 -- 172.21.15.43:0/4083336984 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(132..132 src has 1..132) v4 ==== 6828+0+0 (secure 0 0 0) 0x7ff6d0088d90 con 0x7ff6e80ce720 2023-08-09T13:55:06.233 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:06.227+0000 7ff6e659c640 1 -- 172.21.15.43:0/4083336984 >> [v2:172.21.15.203:6832/1796635339,v1:172.21.15.203:6833/1796635339] conn(0x7ff6c806a290 msgr2=0x7ff6c806c750 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.203:6832/1796635339 2023-08-09T13:55:06.234 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:06.227+0000 7ff6e659c640 1 --2- 172.21.15.43:0/4083336984 >> [v2:172.21.15.203:6832/1796635339,v1:172.21.15.203:6833/1796635339] conn(0x7ff6c806a290 0x7ff6c806c750 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-08-09T13:55:06.234 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:06.231+0000 7ff6df7fe640 1 -- 172.21.15.43:0/4083336984 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178066 (secure 0 0 0) 0x7ff6d0053e10 con 0x7ff6e80ce720 2023-08-09T13:55:06.314 INFO:tasks.ceph.mgr.x.smithi203.stderr:2023-08-09T13:55:06.312+0000 7f8eeb831040 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2023-08-09T13:55:06.361 INFO:tasks.ceph.mgr.y.smithi043.stderr:2023-08-09T13:55:06.359+0000 7f106ca0f040 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2023-08-09T13:55:06.377 INFO:tasks.ceph.mgr.x.smithi203.stderr:2023-08-09T13:55:06.374+0000 7f8eeb831040 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2023-08-09T13:55:06.419 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:06.418+0000 7ff6ecd64640 1 -- 172.21.15.43:0/4083336984 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7ff6e80c7a70 con 0x7ff6e80ce720 2023-08-09T13:55:06.420 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:06.419+0000 7ff6df7fe640 1 -- 172.21.15.43:0/4083336984 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v132) v1 ==== 104+0+11 (secure 0 0 0) 0x7ff6d0057430 con 0x7ff6e80ce720 2023-08-09T13:55:06.420 INFO:teuthology.orchestra.run.smithi043.stdout:pg_num: 32 2023-08-09T13:55:06.422 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:06.421+0000 7ff6ecd64640 1 -- 172.21.15.43:0/4083336984 >> [v2:172.21.15.203:6832/1796635339,v1:172.21.15.203:6833/1796635339] conn(0x7ff6c806a290 msgr2=0x7ff6c806c750 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-08-09T13:55:06.423 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:06.421+0000 7ff6ecd64640 1 --2- 172.21.15.43:0/4083336984 >> [v2:172.21.15.203:6832/1796635339,v1:172.21.15.203:6833/1796635339] conn(0x7ff6c806a290 0x7ff6c806c750 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:55:06.423 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:06.421+0000 7ff6ecd64640 1 -- 172.21.15.43:0/4083336984 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff6e80ce720 msgr2=0x7ff6e80b49b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:55:06.423 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:06.421+0000 7ff6ecd64640 1 --2- 172.21.15.43:0/4083336984 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff6e80ce720 0x7ff6e80b49b0 secure :-1 s=READY pgs=1577 cs=0 l=1 rev1=1 crypto rx=0x7ff6d000e870 tx=0x7ff6d000ed30 comp rx=0 tx=0).stop 2023-08-09T13:55:06.423 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:06.421+0000 7ff6ecd64640 1 -- 172.21.15.43:0/4083336984 shutdown_connections 2023-08-09T13:55:06.423 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:06.421+0000 7ff6ecd64640 1 --2- 172.21.15.43:0/4083336984 >> [v2:172.21.15.203:6832/1796635339,v1:172.21.15.203:6833/1796635339] conn(0x7ff6c806a290 0x7ff6c806c750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:55:06.423 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:06.421+0000 7ff6ecd64640 1 --2- 172.21.15.43:0/4083336984 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7ff6e815eaf0 0x7ff6e80b4ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:55:06.423 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:06.421+0000 7ff6ecd64640 1 --2- 172.21.15.43:0/4083336984 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff6e80ce720 0x7ff6e80b49b0 unknown :-1 s=CLOSED pgs=1577 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:55:06.424 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:06.421+0000 7ff6ecd64640 1 --2- 172.21.15.43:0/4083336984 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7ff6e80ce080 0x7ff6e80b2e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:55:06.424 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:06.421+0000 7ff6ecd64640 1 -- 172.21.15.43:0/4083336984 >> 172.21.15.43:0/4083336984 conn(0x7ff6e80ccd80 msgr2=0x7ff6e806d2a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-09T13:55:06.424 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:06.421+0000 7ff6ecd64640 1 -- 172.21.15.43:0/4083336984 shutdown_connections 2023-08-09T13:55:06.424 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:06.421+0000 7ff6ecd64640 1 -- 172.21.15.43:0/4083336984 wait complete. 2023-08-09T13:55:06.426 INFO:tasks.ceph.mgr.y.smithi043.stderr:2023-08-09T13:55:06.425+0000 7f106ca0f040 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2023-08-09T13:55:06.426 INFO:tasks.ceph.mgr.x.smithi203.stderr:2023-08-09T13:55:06.424+0000 7f8eeb831040 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2023-08-09T13:55:06.440 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-08-09T13:55:06.482 INFO:tasks.ceph.mgr.y.smithi043.stderr:2023-08-09T13:55:06.480+0000 7f106ca0f040 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2023-08-09T13:55:06.543 INFO:tasks.ceph.mgr.x.smithi203.stderr:2023-08-09T13:55:06.541+0000 7f8eeb831040 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2023-08-09T13:55:06.608 INFO:tasks.ceph.mgr.y.smithi043.stderr:2023-08-09T13:55:06.606+0000 7f106ca0f040 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2023-08-09T13:55:06.647 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:06.645+0000 7f6516dee640 1 Processor -- start 2023-08-09T13:55:06.647 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:06.645+0000 7f6516dee640 1 -- start start 2023-08-09T13:55:06.649 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:06.646+0000 7f6516dee640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f65100b0de0 0x7f651006b280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:55:06.649 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:06.646+0000 7f6516dee640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f65100ca810 0x7f65100caca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:55:06.649 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:06.646+0000 7f6516dee640 1 -- --> v1:172.21.15.43:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f65100cefd0 con 0x7f651015c940 2023-08-09T13:55:06.649 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:06.646+0000 7f6516dee640 1 -- --> v2:172.21.15.43:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f65100b1660 con 0x7f65100b0de0 2023-08-09T13:55:06.649 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:06.646+0000 7f6516dee640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f65100cf580 con 0x7f65100ca810 2023-08-09T13:55:06.650 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:06.647+0000 7f6514b8a640 1 --1- >> v1:172.21.15.43:6789/0 conn(0x7f651015c940 0x7f651006ab70 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.43:6789/0 says I am v1:172.21.15.43:42158/0 (socket says 172.21.15.43:42158) 2023-08-09T13:55:06.650 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:06.647+0000 7f6514b8a640 1 -- 172.21.15.43:0/3123302945 learned_addr learned my addr 172.21.15.43:0/3123302945 (peer_addr_for_me v1:172.21.15.43:0/0) 2023-08-09T13:55:06.650 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:06.647+0000 7f650ffff640 1 --2- 172.21.15.43:0/3123302945 >> v2:172.21.15.43:3300/0 conn(0x7f65100b0de0 0x7f651006b280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:55:06.651 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:06.647+0000 7f651538b640 1 --2- 172.21.15.43:0/3123302945 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f65100ca810 0x7f65100caca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:55:06.651 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:06.647+0000 7f650f7fe640 1 -- 172.21.15.43:0/3123302945 <== mon.0 v1:172.21.15.43:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3843926526 0 0) 0x7f6504001290 con 0x7f651015c940 2023-08-09T13:55:06.651 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:06.647+0000 7f650f7fe640 1 -- 172.21.15.43:0/3123302945 --> v1:172.21.15.43:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f6500003520 con 0x7f651015c940 2023-08-09T13:55:06.651 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:06.647+0000 7f650ffff640 1 -- 172.21.15.43:0/3123302945 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f65100ca810 msgr2=0x7f65100caca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-09T13:55:06.651 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:06.647+0000 7f650ffff640 1 --2- 172.21.15.43:0/3123302945 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f65100ca810 0x7f65100caca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:55:06.651 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:06.647+0000 7f650ffff640 1 -- 172.21.15.43:0/3123302945 >> v1:172.21.15.43:6789/0 conn(0x7f651015c940 legacy=0x7f651006ab70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:55:06.652 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:06.647+0000 7f650ffff640 1 -- 172.21.15.43:0/3123302945 --> v2:172.21.15.43:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f651015cd20 con 0x7f65100b0de0 2023-08-09T13:55:06.652 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:06.647+0000 7f651538b640 1 --2- 172.21.15.43:0/3123302945 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f65100ca810 0x7f65100caca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-08-09T13:55:06.652 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:06.649+0000 7f650ffff640 1 --2- 172.21.15.43:0/3123302945 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f65100b0de0 0x7f651006b280 secure :-1 s=READY pgs=1579 cs=0 l=1 rev1=1 crypto rx=0x7f64fc0097a0 tx=0x7f64fc02b8b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b57871db97170348 server_cookie=0 in_seq=0 out_seq=0 2023-08-09T13:55:06.653 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:06.649+0000 7f650f7fe640 1 -- 172.21.15.43:0/3123302945 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f64fc02d040 con 0x7f65100b0de0 2023-08-09T13:55:06.653 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:06.649+0000 7f650f7fe640 1 -- 172.21.15.43:0/3123302945 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f64fc031810 con 0x7f65100b0de0 2023-08-09T13:55:06.653 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:06.649+0000 7f650f7fe640 1 -- 172.21.15.43:0/3123302945 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f64fc031ba0 con 0x7f65100b0de0 2023-08-09T13:55:06.653 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:06.649+0000 7f6516dee640 1 -- 172.21.15.43:0/3123302945 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f65100b0de0 msgr2=0x7f651006b280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:55:06.653 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:06.649+0000 7f6516dee640 1 --2- 172.21.15.43:0/3123302945 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f65100b0de0 0x7f651006b280 secure :-1 s=READY pgs=1579 cs=0 l=1 rev1=1 crypto rx=0x7f64fc0097a0 tx=0x7f64fc02b8b0 comp rx=0 tx=0).stop 2023-08-09T13:55:06.653 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:06.650+0000 7f6516dee640 1 -- 172.21.15.43:0/3123302945 shutdown_connections 2023-08-09T13:55:06.654 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:06.650+0000 7f6516dee640 1 --2- 172.21.15.43:0/3123302945 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f65100ca810 0x7f65100caca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:55:06.654 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:06.650+0000 7f6516dee640 1 --2- 172.21.15.43:0/3123302945 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f65100b0de0 0x7f651006b280 unknown :-1 s=CLOSED pgs=1579 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:55:06.654 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:06.650+0000 7f6516dee640 1 -- 172.21.15.43:0/3123302945 >> 172.21.15.43:0/3123302945 conn(0x7f6510089fe0 msgr2=0x7f65100d01c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-09T13:55:06.654 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:06.650+0000 7f6516dee640 1 -- 172.21.15.43:0/3123302945 shutdown_connections 2023-08-09T13:55:06.654 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:06.650+0000 7f6516dee640 1 -- 172.21.15.43:0/3123302945 wait complete. 2023-08-09T13:55:06.655 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:06.651+0000 7f6516dee640 1 Processor -- start 2023-08-09T13:55:06.655 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:06.651+0000 7f6516dee640 1 -- start start 2023-08-09T13:55:06.655 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:06.652+0000 7f6516dee640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f65100b0de0 0x7f65100c3cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:55:06.655 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:06.652+0000 7f6516dee640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f65100ca810 0x7f651014b580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:55:06.655 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:06.652+0000 7f6516dee640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f651015c940 0x7f6510156e90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:55:06.656 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:06.652+0000 7f6516dee640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f65100c4ee0 con 0x7f65100b0de0 2023-08-09T13:55:06.656 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:06.652+0000 7f6516dee640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f65100c42d0 con 0x7f65100ca810 2023-08-09T13:55:06.656 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:06.652+0000 7f6516dee640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f65100c45b0 con 0x7f651015c940 2023-08-09T13:55:06.656 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:06.653+0000 7f6514b8a640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f65100b0de0 0x7f65100c3cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:55:06.656 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:06.653+0000 7f6514b8a640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f65100b0de0 0x7f65100c3cf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:56414/0 (socket says 172.21.15.43:56414) 2023-08-09T13:55:06.656 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:06.653+0000 7f6514b8a640 1 -- 172.21.15.43:0/1627282799 learned_addr learned my addr 172.21.15.43:0/1627282799 (peer_addr_for_me v2:172.21.15.43:0/0) 2023-08-09T13:55:06.657 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:06.653+0000 7f651538b640 1 --2- 172.21.15.43:0/1627282799 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f651015c940 0x7f6510156e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:55:06.658 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:06.653+0000 7f650ffff640 1 --2- 172.21.15.43:0/1627282799 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f65100ca810 0x7f651014b580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:55:06.658 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:06.653+0000 7f6514b8a640 1 -- 172.21.15.43:0/1627282799 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f651015c940 msgr2=0x7f6510156e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:55:06.658 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:06.653+0000 7f6514b8a640 1 --2- 172.21.15.43:0/1627282799 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f651015c940 0x7f6510156e90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:55:06.658 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:06.653+0000 7f6514b8a640 1 -- 172.21.15.43:0/1627282799 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f65100ca810 msgr2=0x7f651014b580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:55:06.658 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:06.653+0000 7f6514b8a640 1 --2- 172.21.15.43:0/1627282799 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f65100ca810 0x7f651014b580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:55:06.658 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:06.653+0000 7f6514b8a640 1 -- 172.21.15.43:0/1627282799 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f64fc009400 con 0x7f65100b0de0 2023-08-09T13:55:06.658 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:06.654+0000 7f6514b8a640 1 --2- 172.21.15.43:0/1627282799 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f65100b0de0 0x7f65100c3cf0 secure :-1 s=READY pgs=1580 cs=0 l=1 rev1=1 crypto rx=0x7f650400c400 tx=0x7f650400c8c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-09T13:55:06.658 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:06.654+0000 7f650d7fa640 1 -- 172.21.15.43:0/1627282799 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f65040074e0 con 0x7f65100b0de0 2023-08-09T13:55:06.659 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:06.654+0000 7f650d7fa640 1 -- 172.21.15.43:0/1627282799 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6504007640 con 0x7f65100b0de0 2023-08-09T13:55:06.659 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:06.654+0000 7f6516dee640 1 -- 172.21.15.43:0/1627282799 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f65100c4890 con 0x7f65100b0de0 2023-08-09T13:55:06.659 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:06.654+0000 7f650d7fa640 1 -- 172.21.15.43:0/1627282799 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f65040059c0 con 0x7f65100b0de0 2023-08-09T13:55:06.659 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:06.654+0000 7f6516dee640 1 -- 172.21.15.43:0/1627282799 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f651014bac0 con 0x7f65100b0de0 2023-08-09T13:55:06.660 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:06.656+0000 7f6516dee640 1 -- 172.21.15.43:0/1627282799 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f65100b1ab0 con 0x7f65100b0de0 2023-08-09T13:55:06.660 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:06.656+0000 7f650d7fa640 1 -- 172.21.15.43:0/1627282799 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 63) v1 ==== 88557+0+0 (secure 0 0 0) 0x7f6504005b20 con 0x7f65100b0de0 2023-08-09T13:55:06.660 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:06.657+0000 7f650d7fa640 1 --2- 172.21.15.43:0/1627282799 >> [v2:172.21.15.203:6832/1796635339,v1:172.21.15.203:6833/1796635339] conn(0x7f64f406a290 0x7f64f406c750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:55:06.660 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:06.658+0000 7f650d7fa640 1 -- 172.21.15.43:0/1627282799 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(132..132 src has 1..132) v4 ==== 6828+0+0 (secure 0 0 0) 0x7f65040882d0 con 0x7f65100b0de0 2023-08-09T13:55:06.661 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:06.658+0000 7f650ffff640 1 -- 172.21.15.43:0/1627282799 >> [v2:172.21.15.203:6832/1796635339,v1:172.21.15.203:6833/1796635339] conn(0x7f64f406a290 msgr2=0x7f64f406c750 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.203:6832/1796635339 2023-08-09T13:55:06.661 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:06.658+0000 7f650ffff640 1 --2- 172.21.15.43:0/1627282799 >> [v2:172.21.15.203:6832/1796635339,v1:172.21.15.203:6833/1796635339] conn(0x7f64f406a290 0x7f64f406c750 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-08-09T13:55:06.664 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:06.662+0000 7f650d7fa640 1 -- 172.21.15.43:0/1627282799 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178066 (secure 0 0 0) 0x7f6504056cc0 con 0x7f65100b0de0 2023-08-09T13:55:06.700 INFO:tasks.ceph.mgr.x.smithi203.stderr:2023-08-09T13:55:06.697+0000 7f8eeb831040 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2023-08-09T13:55:06.808 INFO:tasks.ceph.mgr.x.smithi203.stderr:2023-08-09T13:55:06.806+0000 7f8eeb831040 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2023-08-09T13:55:06.856 INFO:tasks.ceph.mgr.y.smithi043.stderr:2023-08-09T13:55:06.853+0000 7f106ca0f040 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2023-08-09T13:55:06.857 INFO:tasks.ceph.mgr.x.smithi203.stderr:2023-08-09T13:55:06.855+0000 7f8eeb831040 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2023-08-09T13:55:06.861 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:06.859+0000 7f650ffff640 1 -- 172.21.15.43:0/1627282799 >> [v2:172.21.15.203:6832/1796635339,v1:172.21.15.203:6833/1796635339] conn(0x7f64f406a290 msgr2=0x7f64f406c750 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.203:6832/1796635339 2023-08-09T13:55:06.862 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:06.859+0000 7f650ffff640 1 --2- 172.21.15.43:0/1627282799 >> [v2:172.21.15.203:6832/1796635339,v1:172.21.15.203:6833/1796635339] conn(0x7f64f406a290 0x7f64f406c750 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2023-08-09T13:55:06.944 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:06.941+0000 7f6516dee640 1 -- 172.21.15.43:0/1627282799 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f65100cefd0 con 0x7f65100b0de0 2023-08-09T13:55:06.944 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:06.942+0000 7f650d7fa640 1 -- 172.21.15.43:0/1627282799 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v132) v1 ==== 100+0+11 (secure 0 0 0) 0x7f65100cefd0 con 0x7f65100b0de0 2023-08-09T13:55:06.945 INFO:teuthology.orchestra.run.smithi043.stdout:pg_num: 64 2023-08-09T13:55:06.948 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:06.945+0000 7f6516dee640 1 -- 172.21.15.43:0/1627282799 >> [v2:172.21.15.203:6832/1796635339,v1:172.21.15.203:6833/1796635339] conn(0x7f64f406a290 msgr2=0x7f64f406c750 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-08-09T13:55:06.948 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:06.945+0000 7f6516dee640 1 --2- 172.21.15.43:0/1627282799 >> [v2:172.21.15.203:6832/1796635339,v1:172.21.15.203:6833/1796635339] conn(0x7f64f406a290 0x7f64f406c750 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:55:06.948 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:06.945+0000 7f6516dee640 1 -- 172.21.15.43:0/1627282799 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f65100b0de0 msgr2=0x7f65100c3cf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:55:06.948 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:06.945+0000 7f6516dee640 1 --2- 172.21.15.43:0/1627282799 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f65100b0de0 0x7f65100c3cf0 secure :-1 s=READY pgs=1580 cs=0 l=1 rev1=1 crypto rx=0x7f650400c400 tx=0x7f650400c8c0 comp rx=0 tx=0).stop 2023-08-09T13:55:06.948 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:06.945+0000 7f6516dee640 1 -- 172.21.15.43:0/1627282799 shutdown_connections 2023-08-09T13:55:06.949 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:06.945+0000 7f6516dee640 1 --2- 172.21.15.43:0/1627282799 >> [v2:172.21.15.203:6832/1796635339,v1:172.21.15.203:6833/1796635339] conn(0x7f64f406a290 0x7f64f406c750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:55:06.949 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:06.945+0000 7f6516dee640 1 --2- 172.21.15.43:0/1627282799 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f651015c940 0x7f6510156e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:55:06.949 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:06.945+0000 7f6516dee640 1 --2- 172.21.15.43:0/1627282799 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f65100ca810 0x7f651014b580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:55:06.949 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:06.945+0000 7f6516dee640 1 --2- 172.21.15.43:0/1627282799 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f65100b0de0 0x7f65100c3cf0 unknown :-1 s=CLOSED pgs=1580 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:55:06.949 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:06.945+0000 7f6516dee640 1 -- 172.21.15.43:0/1627282799 >> 172.21.15.43:0/1627282799 conn(0x7f6510089fe0 msgr2=0x7f65100c9f20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-09T13:55:06.949 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:06.945+0000 7f6516dee640 1 -- 172.21.15.43:0/1627282799 shutdown_connections 2023-08-09T13:55:06.949 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:06.946+0000 7f6516dee640 1 -- 172.21.15.43:0/1627282799 wait complete. 2023-08-09T13:55:06.966 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2023-08-09T13:55:06.971 INFO:tasks.ceph.mgr.y.smithi043.stderr:2023-08-09T13:55:06.969+0000 7f106ca0f040 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2023-08-09T13:55:07.017 INFO:tasks.ceph.mgr.x.smithi203.stderr:2023-08-09T13:55:07.014+0000 7f8eeb831040 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2023-08-09T13:55:07.024 INFO:tasks.ceph.mgr.y.smithi043.stderr:2023-08-09T13:55:07.022+0000 7f106ca0f040 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2023-08-09T13:55:07.184 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:07.182+0000 7f74108f3640 1 Processor -- start 2023-08-09T13:55:07.185 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:07.182+0000 7f74108f3640 1 -- start start 2023-08-09T13:55:07.186 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:07.184+0000 7f74108f3640 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7f740c0ccdd0 0x7f740c0ca580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:55:07.186 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:07.184+0000 7f74108f3640 1 -- --> v1:172.21.15.43:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f740c0cd300 con 0x7f740c070df0 2023-08-09T13:55:07.186 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:07.184+0000 7f74108f3640 1 -- --> v1:172.21.15.203:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f740c0714c0 con 0x7f740c15c970 2023-08-09T13:55:07.187 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:07.184+0000 7f74108f3640 1 -- --> v2:172.21.15.203:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f740c0caac0 con 0x7f740c0ccdd0 2023-08-09T13:55:07.187 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:07.184+0000 7f7401d9b640 1 --1- >> v1:172.21.15.43:6789/0 conn(0x7f740c070df0 0x7f740c06e0e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.43:6789/0 says I am v1:172.21.15.43:42166/0 (socket says 172.21.15.43:42166) 2023-08-09T13:55:07.187 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:07.184+0000 7f7401d9b640 1 -- 172.21.15.43:0/1793096948 learned_addr learned my addr 172.21.15.43:0/1793096948 (peer_addr_for_me v1:172.21.15.43:0/0) 2023-08-09T13:55:07.188 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:07.184+0000 7f740ad9d640 1 --2- 172.21.15.43:0/1793096948 >> v2:172.21.15.203:3300/0 conn(0x7f740c0ccdd0 0x7f740c0ca580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:55:07.188 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:07.185+0000 7f7409d9b640 1 -- 172.21.15.43:0/1793096948 <== mon.0 v1:172.21.15.43:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1475707152 0 0) 0x7f740c0cd300 con 0x7f740c070df0 2023-08-09T13:55:07.189 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:07.185+0000 7f7409d9b640 1 -- 172.21.15.43:0/1793096948 --> v1:172.21.15.43:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f73ec003520 con 0x7f740c070df0 2023-08-09T13:55:07.189 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:07.185+0000 7f7409d9b640 1 -- 172.21.15.43:0/1793096948 <== mon.0 v1:172.21.15.43:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3104746945 0 0) 0x7f73ec003520 con 0x7f740c070df0 2023-08-09T13:55:07.189 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:07.185+0000 7f7409d9b640 1 -- 172.21.15.43:0/1793096948 >> v2:172.21.15.203:3300/0 conn(0x7f740c0ccdd0 msgr2=0x7f740c0ca580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-09T13:55:07.189 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:07.185+0000 7f7409d9b640 1 --2- 172.21.15.43:0/1793096948 >> v2:172.21.15.203:3300/0 conn(0x7f740c0ccdd0 0x7f740c0ca580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:55:07.189 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:07.185+0000 7f7409d9b640 1 -- 172.21.15.43:0/1793096948 >> v1:172.21.15.203:6789/0 conn(0x7f740c15c970 legacy=0x7f740c0706e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:55:07.189 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:07.185+0000 7f7409d9b640 1 -- 172.21.15.43:0/1793096948 --> v1:172.21.15.43:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f740c0717a0 con 0x7f740c070df0 2023-08-09T13:55:07.189 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:07.185+0000 7f7409d9b640 1 -- 172.21.15.43:0/1793096948 <== mon.0 v1:172.21.15.43:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1285150740 0 0) 0x7f73f4001670 con 0x7f740c070df0 2023-08-09T13:55:07.190 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:07.185+0000 7f7409d9b640 1 -- 172.21.15.43:0/1793096948 >> v1:172.21.15.43:6789/0 conn(0x7f740c070df0 legacy=0x7f740c06e0e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:55:07.190 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:07.185+0000 7f7409d9b640 1 --2- 172.21.15.43:0/1793096948 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f73ec0039e0 0x7f73ec003e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:55:07.190 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:07.185+0000 7f7409d9b640 1 --2- 172.21.15.43:0/1793096948 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f73ec0047c0 0x7f73ec004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:55:07.190 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:07.185+0000 7f7409d9b640 1 --2- 172.21.15.43:0/1793096948 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f73ec005180 0x7f73ec00db90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:55:07.190 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:07.185+0000 7f7409d9b640 1 -- 172.21.15.43:0/1793096948 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f740c0caac0 con 0x7f73ec005180 2023-08-09T13:55:07.190 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:07.185+0000 7f7409d9b640 1 -- 172.21.15.43:0/1793096948 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f73ec00e2c0 con 0x7f73ec0039e0 2023-08-09T13:55:07.190 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:07.185+0000 7f7409d9b640 1 -- 172.21.15.43:0/1793096948 --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f73ec00e480 con 0x7f73ec0047c0 2023-08-09T13:55:07.190 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:07.185+0000 7f740ad9d640 1 --2- 172.21.15.43:0/1793096948 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f73ec005180 0x7f73ec00db90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:55:07.191 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:07.186+0000 7f740a59c640 1 --2- 172.21.15.43:0/1793096948 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f73ec0039e0 0x7f73ec003e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:55:07.191 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:07.186+0000 7f7401d9b640 1 --2- 172.21.15.43:0/1793096948 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f73ec0047c0 0x7f73ec004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:55:07.191 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:07.186+0000 7f740ad9d640 1 -- 172.21.15.43:0/1793096948 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f73ec0047c0 msgr2=0x7f73ec004c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-09T13:55:07.191 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:07.186+0000 7f740ad9d640 1 --2- 172.21.15.43:0/1793096948 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f73ec0047c0 0x7f73ec004c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:55:07.191 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:07.186+0000 7f740ad9d640 1 -- 172.21.15.43:0/1793096948 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f73ec0039e0 msgr2=0x7f73ec003e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-09T13:55:07.191 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:07.186+0000 7f740ad9d640 1 --2- 172.21.15.43:0/1793096948 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f73ec0039e0 0x7f73ec003e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:55:07.191 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:07.186+0000 7f740ad9d640 1 -- 172.21.15.43:0/1793096948 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f73ec003520 con 0x7f73ec005180 2023-08-09T13:55:07.192 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:07.187+0000 7f740ad9d640 1 -- 172.21.15.43:0/1793096948 reap_dead start 2023-08-09T13:55:07.193 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:07.187+0000 7f740ad9d640 1 --2- 172.21.15.43:0/1793096948 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f73ec005180 0x7f73ec00db90 secure :-1 s=READY pgs=1582 cs=0 l=1 rev1=1 crypto rx=0x7f7404009fc0 tx=0x7f73ec0057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2c309cefa65c9c5b server_cookie=0 in_seq=0 out_seq=0 2023-08-09T13:55:07.193 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:07.187+0000 7f7409d9b640 1 -- 172.21.15.43:0/1793096948 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f740402cc60 con 0x7f73ec005180 2023-08-09T13:55:07.193 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:07.187+0000 7f7409d9b640 1 -- 172.21.15.43:0/1793096948 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f740400f040 con 0x7f73ec005180 2023-08-09T13:55:07.193 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:07.187+0000 7f7409d9b640 1 -- 172.21.15.43:0/1793096948 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7404003990 con 0x7f73ec005180 2023-08-09T13:55:07.193 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:07.188+0000 7f74108f3640 1 -- 172.21.15.43:0/1793096948 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f73ec005180 msgr2=0x7f73ec00db90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:55:07.193 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:07.188+0000 7f74108f3640 1 --2- 172.21.15.43:0/1793096948 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f73ec005180 0x7f73ec00db90 secure :-1 s=READY pgs=1582 cs=0 l=1 rev1=1 crypto rx=0x7f7404009fc0 tx=0x7f73ec0057f0 comp rx=0 tx=0).stop 2023-08-09T13:55:07.193 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:07.188+0000 7f74108f3640 1 -- 172.21.15.43:0/1793096948 shutdown_connections 2023-08-09T13:55:07.194 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:07.188+0000 7f74108f3640 1 --2- 172.21.15.43:0/1793096948 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f73ec005180 0x7f73ec00db90 unknown :-1 s=CLOSED pgs=1582 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:55:07.194 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:07.188+0000 7f74108f3640 1 -- 172.21.15.43:0/1793096948 >> 172.21.15.43:0/1793096948 conn(0x7f740c0cbc80 msgr2=0x7f740c0cc090 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-09T13:55:07.194 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:07.188+0000 7f74108f3640 1 -- 172.21.15.43:0/1793096948 shutdown_connections 2023-08-09T13:55:07.194 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:07.188+0000 7f74108f3640 1 -- 172.21.15.43:0/1793096948 wait complete. 2023-08-09T13:55:07.194 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:07.190+0000 7f74108f3640 1 Processor -- start 2023-08-09T13:55:07.194 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:07.190+0000 7f74108f3640 1 -- start start 2023-08-09T13:55:07.196 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:07.191+0000 7f74108f3640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f73ec005180 0x7f740c0ac7b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:55:07.196 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:07.191+0000 7f74108f3640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f740c0afe20 0x7f740c0b02a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:55:07.196 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:07.191+0000 7f74108f3640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f740c0b31d0 0x7f740c0b3650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:55:07.196 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:07.191+0000 7f74108f3640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f740c0b07e0 con 0x7f740c0afe20 2023-08-09T13:55:07.196 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:07.191+0000 7f74108f3640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f740c0b4bb0 con 0x7f740c0b31d0 2023-08-09T13:55:07.196 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:07.191+0000 7f74108f3640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f740c0accf0 con 0x7f73ec005180 2023-08-09T13:55:07.197 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:07.191+0000 7f7401d9b640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f740c0afe20 0x7f740c0b02a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:55:07.197 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:07.191+0000 7f7401d9b640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f740c0afe20 0x7f740c0b02a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:56434/0 (socket says 172.21.15.43:56434) 2023-08-09T13:55:07.197 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:07.191+0000 7f7401d9b640 1 -- 172.21.15.43:0/3836245267 learned_addr learned my addr 172.21.15.43:0/3836245267 (peer_addr_for_me v2:172.21.15.43:0/0) 2023-08-09T13:55:07.197 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:07.191+0000 7f740a59c640 1 --2- 172.21.15.43:0/3836245267 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f73ec005180 0x7f740c0ac7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:55:07.197 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:07.192+0000 7f7401d9b640 1 -- 172.21.15.43:0/3836245267 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f73ec005180 msgr2=0x7f740c0ac7b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:55:07.197 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:07.192+0000 7f7401d9b640 1 --2- 172.21.15.43:0/3836245267 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f73ec005180 0x7f740c0ac7b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:55:07.197 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:07.192+0000 7f740ad9d640 1 --2- 172.21.15.43:0/3836245267 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f740c0b31d0 0x7f740c0b3650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:55:07.198 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:07.192+0000 7f7401d9b640 1 -- 172.21.15.43:0/3836245267 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f740c0b31d0 msgr2=0x7f740c0b3650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:55:07.198 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:07.192+0000 7f7401d9b640 1 --2- 172.21.15.43:0/3836245267 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f740c0b31d0 0x7f740c0b3650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:55:07.198 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:07.192+0000 7f7401d9b640 1 -- 172.21.15.43:0/3836245267 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f740c0b3fa0 con 0x7f740c0afe20 2023-08-09T13:55:07.198 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:07.192+0000 7f740a59c640 1 --2- 172.21.15.43:0/3836245267 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f73ec005180 0x7f740c0ac7b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-08-09T13:55:07.198 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:07.192+0000 7f7401d9b640 1 --2- 172.21.15.43:0/3836245267 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f740c0afe20 0x7f740c0b02a0 secure :-1 s=READY pgs=1583 cs=0 l=1 rev1=1 crypto rx=0x7f73f400e760 tx=0x7f73f400ec20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-09T13:55:07.198 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:07.192+0000 7f74037fe640 1 -- 172.21.15.43:0/3836245267 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f73f400c6c0 con 0x7f740c0afe20 2023-08-09T13:55:07.198 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:07.192+0000 7f74037fe640 1 -- 172.21.15.43:0/3836245267 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f73f400c820 con 0x7f740c0afe20 2023-08-09T13:55:07.198 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:07.193+0000 7f74037fe640 1 -- 172.21.15.43:0/3836245267 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f73f400cc10 con 0x7f740c0afe20 2023-08-09T13:55:07.199 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:07.193+0000 7f74108f3640 1 -- 172.21.15.43:0/3836245267 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f740c0b4280 con 0x7f740c0afe20 2023-08-09T13:55:07.199 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:07.193+0000 7f74108f3640 1 -- 172.21.15.43:0/3836245267 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f740c0b4840 con 0x7f740c0afe20 2023-08-09T13:55:07.199 INFO:tasks.ceph.mgr.y.smithi043.stderr:2023-08-09T13:55:07.194+0000 7f106ca0f040 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2023-08-09T13:55:07.200 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:07.194+0000 7f74108f3640 1 -- 172.21.15.43:0/3836245267 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f740c0b4a00 con 0x7f740c0afe20 2023-08-09T13:55:07.200 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:07.195+0000 7f74037fe640 1 -- 172.21.15.43:0/3836245267 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 63) v1 ==== 88557+0+0 (secure 0 0 0) 0x7f73f401d460 con 0x7f740c0afe20 2023-08-09T13:55:07.200 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:07.196+0000 7f74037fe640 1 --2- 172.21.15.43:0/3836245267 >> [v2:172.21.15.203:6832/1796635339,v1:172.21.15.203:6833/1796635339] conn(0x7f73f006a290 0x7f73f006c750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:55:07.200 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:07.196+0000 7f74037fe640 1 -- 172.21.15.43:0/3836245267 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(132..132 src has 1..132) v4 ==== 6828+0+0 (secure 0 0 0) 0x7f73f4088ad0 con 0x7f740c0afe20 2023-08-09T13:55:07.200 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:07.196+0000 7f740a59c640 1 -- 172.21.15.43:0/3836245267 >> [v2:172.21.15.203:6832/1796635339,v1:172.21.15.203:6833/1796635339] conn(0x7f73f006a290 msgr2=0x7f73f006c750 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.203:6832/1796635339 2023-08-09T13:55:07.200 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:07.196+0000 7f740a59c640 1 --2- 172.21.15.43:0/3836245267 >> [v2:172.21.15.203:6832/1796635339,v1:172.21.15.203:6833/1796635339] conn(0x7f73f006a290 0x7f73f006c750 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-08-09T13:55:07.202 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:07.200+0000 7f74037fe640 1 -- 172.21.15.43:0/3836245267 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178066 (secure 0 0 0) 0x7f73f4053c20 con 0x7f740c0afe20 2023-08-09T13:55:07.399 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:07.397+0000 7f740a59c640 1 -- 172.21.15.43:0/3836245267 >> [v2:172.21.15.203:6832/1796635339,v1:172.21.15.203:6833/1796635339] conn(0x7f73f006a290 msgr2=0x7f73f006c750 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.203:6832/1796635339 2023-08-09T13:55:07.400 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:07.397+0000 7f740a59c640 1 --2- 172.21.15.43:0/3836245267 >> [v2:172.21.15.203:6832/1796635339,v1:172.21.15.203:6833/1796635339] conn(0x7f73f006a290 0x7f73f006c750 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2023-08-09T13:55:07.404 INFO:tasks.ceph.mgr.x.smithi203.stderr:2023-08-09T13:55:07.402+0000 7f8eeb831040 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2023-08-09T13:55:07.414 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:07.412+0000 7f74108f3640 1 -- 172.21.15.43:0/3836245267 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7f740c0739f0 con 0x7f740c0afe20 2023-08-09T13:55:07.415 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:07.413+0000 7f74037fe640 1 -- 172.21.15.43:0/3836245267 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 8 entries v132) v1 ==== 80+0+475 (secure 0 0 0) 0x7f73f4057240 con 0x7f740c0afe20 2023-08-09T13:55:07.416 INFO:teuthology.orchestra.run.smithi043.stdout:172.21.15.203:6833/2527059680 2023-08-10T13:54:03.238790+0000 2023-08-09T13:55:07.416 INFO:teuthology.orchestra.run.smithi043.stdout:172.21.15.203:6832/2527059680 2023-08-10T13:54:03.238790+0000 2023-08-09T13:55:07.416 INFO:teuthology.orchestra.run.smithi043.stdout:172.21.15.203:0/431304609 2023-08-10T13:54:03.238790+0000 2023-08-09T13:55:07.416 INFO:teuthology.orchestra.run.smithi043.stdout:172.21.15.203:0/826469783 2023-08-10T13:54:03.238790+0000 2023-08-09T13:55:07.416 INFO:teuthology.orchestra.run.smithi043.stdout:172.21.15.203:0/1456953448 2023-08-10T13:54:03.238790+0000 2023-08-09T13:55:07.416 INFO:teuthology.orchestra.run.smithi043.stdout:172.21.15.203:0/2004944696 2023-08-10T13:54:03.238790+0000 2023-08-09T13:55:07.417 INFO:teuthology.orchestra.run.smithi043.stdout:172.21.15.203:0/916024371 2023-08-10T13:54:03.238790+0000 2023-08-09T13:55:07.417 INFO:teuthology.orchestra.run.smithi043.stdout:172.21.15.203:0/4209557762 2023-08-10T13:54:03.238790+0000 2023-08-09T13:55:07.417 INFO:teuthology.orchestra.run.smithi043.stderr:listed 8 entries 2023-08-09T13:55:07.418 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:07.415+0000 7f74108f3640 1 -- 172.21.15.43:0/3836245267 >> [v2:172.21.15.203:6832/1796635339,v1:172.21.15.203:6833/1796635339] conn(0x7f73f006a290 msgr2=0x7f73f006c750 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-08-09T13:55:07.418 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:07.415+0000 7f74108f3640 1 --2- 172.21.15.43:0/3836245267 >> [v2:172.21.15.203:6832/1796635339,v1:172.21.15.203:6833/1796635339] conn(0x7f73f006a290 0x7f73f006c750 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:55:07.418 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:07.415+0000 7f74108f3640 1 -- 172.21.15.43:0/3836245267 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f740c0afe20 msgr2=0x7f740c0b02a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:55:07.419 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:07.415+0000 7f74108f3640 1 --2- 172.21.15.43:0/3836245267 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f740c0afe20 0x7f740c0b02a0 secure :-1 s=READY pgs=1583 cs=0 l=1 rev1=1 crypto rx=0x7f73f400e760 tx=0x7f73f400ec20 comp rx=0 tx=0).stop 2023-08-09T13:55:07.419 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:07.415+0000 7f74108f3640 1 -- 172.21.15.43:0/3836245267 shutdown_connections 2023-08-09T13:55:07.419 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:07.415+0000 7f74108f3640 1 --2- 172.21.15.43:0/3836245267 >> [v2:172.21.15.203:6832/1796635339,v1:172.21.15.203:6833/1796635339] conn(0x7f73f006a290 0x7f73f006c750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:55:07.419 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:07.415+0000 7f74108f3640 1 --2- 172.21.15.43:0/3836245267 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f740c0b31d0 0x7f740c0b3650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:55:07.419 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:07.415+0000 7f74108f3640 1 --2- 172.21.15.43:0/3836245267 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f740c0afe20 0x7f740c0b02a0 unknown :-1 s=CLOSED pgs=1583 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:55:07.419 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:07.415+0000 7f74108f3640 1 --2- 172.21.15.43:0/3836245267 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f73ec005180 0x7f740c0ac7b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:55:07.419 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:07.415+0000 7f74108f3640 1 -- 172.21.15.43:0/3836245267 >> 172.21.15.43:0/3836245267 conn(0x7f740c0cbc80 msgr2=0x7f740c06cfe0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-09T13:55:07.419 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:07.416+0000 7f74108f3640 1 -- 172.21.15.43:0/3836245267 shutdown_connections 2023-08-09T13:55:07.420 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:07.416+0000 7f74108f3640 1 -- 172.21.15.43:0/3836245267 wait complete. 2023-08-09T13:55:07.436 INFO:tasks.cephfs.fuse_mount:Running fusermount -u on ubuntu@smithi043.front.sepia.ceph.com... 2023-08-09T13:55:07.436 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-09T13:55:07.437 DEBUG:teuthology.orchestra.run.smithi043:> sudo fusermount -u /home/ubuntu/cephtest/mnt.0 2023-08-09T13:55:07.515 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi043.stderr:ceph-fuse[128482]: fuse finished with error 0 and tester_r 0 2023-08-09T13:55:07.518 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi043.front.sepia.ceph.com 2023-08-09T13:55:07.519 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-09T13:55:07.519 DEBUG:teuthology.orchestra.run.smithi043:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2023-08-09T13:55:07.611 INFO:tasks.ceph.mgr.y.smithi043.stderr:2023-08-09T13:55:07.608+0000 7f106ca0f040 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2023-08-09T13:55:07.625 INFO:tasks.ceph.mgr.x.smithi203.stderr:2023-08-09T13:55:07.623+0000 7f8eeb831040 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2023-08-09T13:55:07.682 INFO:tasks.ceph.mgr.x.smithi203.stderr:2023-08-09T13:55:07.679+0000 7f8eeb831040 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2023-08-09T13:55:07.696 DEBUG:teuthology.orchestra.run.smithi203:> set -ex 2023-08-09T13:55:07.696 DEBUG:teuthology.orchestra.run.smithi203:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-09T13:55:07.733 INFO:tasks.ceph.mgr.x.smithi203.stderr:2023-08-09T13:55:07.731+0000 7f8eeb831040 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2023-08-09T13:55:07.752 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4053648,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497576k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=27,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248784k,nr_inodes=812196,mode=700,uid=1000,gid=1275 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497576k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2023-08-09T13:55:07.753 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi203.front.sepia.ceph.com 2023-08-09T13:55:07.753 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-09T13:55:07.754 DEBUG:teuthology.orchestra.run.smithi203:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.1) 2023-08-09T13:55:07.788 INFO:tasks.ceph.mgr.x.smithi203.stderr:2023-08-09T13:55:07.786+0000 7f8eeb831040 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2023-08-09T13:55:07.839 INFO:teuthology.orchestra.run.smithi203.stderr:rmdir: failed to remove '/home/ubuntu/cephtest/mnt.1': No such file or directory 2023-08-09T13:55:07.840 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-08-09T13:55:07.841 DEBUG:teuthology.orchestra.run.smithi043:> set -ex 2023-08-09T13:55:07.841 DEBUG:teuthology.orchestra.run.smithi043:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-09T13:55:07.850 INFO:tasks.ceph.mgr.y.smithi043.stderr:2023-08-09T13:55:07.842+0000 7f106ca0f040 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2023-08-09T13:55:07.898 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4054441,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6498844k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=27,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3249420k,nr_inodes=812355,mode=700,uid=1000,gid=1275 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6498844k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2023-08-09T13:55:07.898 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi043.front.sepia.ceph.com 2023-08-09T13:55:07.899 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-09T13:55:07.899 DEBUG:teuthology.orchestra.run.smithi043:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.2) 2023-08-09T13:55:07.905 INFO:tasks.ceph.mgr.y.smithi043.stderr:2023-08-09T13:55:07.903+0000 7f106ca0f040 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2023-08-09T13:55:07.948 INFO:teuthology.orchestra.run.smithi043.stderr:rmdir: failed to remove '/home/ubuntu/cephtest/mnt.2': No such file or directory 2023-08-09T13:55:07.949 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-08-09T13:55:07.950 DEBUG:teuthology.orchestra.run.smithi203:> set -ex 2023-08-09T13:55:07.950 DEBUG:teuthology.orchestra.run.smithi203:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-09T13:55:07.959 INFO:tasks.ceph.mgr.y.smithi043.stderr:2023-08-09T13:55:07.957+0000 7f106ca0f040 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2023-08-09T13:55:08.005 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4053648,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497576k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=27,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248784k,nr_inodes=812196,mode=700,uid=1000,gid=1275 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497576k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2023-08-09T13:55:08.005 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi203.front.sepia.ceph.com 2023-08-09T13:55:08.006 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-09T13:55:08.006 DEBUG:teuthology.orchestra.run.smithi203:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.3) 2023-08-09T13:55:08.014 INFO:tasks.ceph.mgr.y.smithi043.stderr:2023-08-09T13:55:08.011+0000 7f106ca0f040 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2023-08-09T13:55:08.054 INFO:teuthology.orchestra.run.smithi203.stderr:rmdir: failed to remove '/home/ubuntu/cephtest/mnt.3': No such file or directory 2023-08-09T13:55:08.055 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-08-09T13:55:08.056 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-08-09T13:55:08.092 INFO:tasks.ceph.mgr.x.smithi203.stderr:2023-08-09T13:55:08.089+0000 7f8eeb831040 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2023-08-09T13:55:08.149 INFO:tasks.ceph.mgr.x.smithi203.stderr:2023-08-09T13:55:08.147+0000 7f8eeb831040 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2023-08-09T13:55:08.201 INFO:tasks.ceph.mgr.x.smithi203.stderr:2023-08-09T13:55:08.198+0000 7f8eeb831040 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2023-08-09T13:55:08.271 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:08.269+0000 7f0ceb1e6640 1 Processor -- start 2023-08-09T13:55:08.271 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:08.269+0000 7f0ceb1e6640 1 -- start start 2023-08-09T13:55:08.273 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:08.270+0000 7f0ceb1e6640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f0ce40d0400 0x7f0ce40cacc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:55:08.273 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:08.270+0000 7f0ceb1e6640 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7f0ce40cb200 0x7f0ce40b06c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:55:08.273 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:08.270+0000 7f0ceb1e6640 1 -- --> v1:172.21.15.203:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0ce40cb9f0 con 0x7f0ce415a350 2023-08-09T13:55:08.273 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:08.270+0000 7f0ceb1e6640 1 -- --> v2:172.21.15.203:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0ce40cbfb0 con 0x7f0ce40cb200 2023-08-09T13:55:08.273 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:08.270+0000 7f0ceb1e6640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0ce40cbcd0 con 0x7f0ce40d0400 2023-08-09T13:55:08.274 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:08.271+0000 7f0ce99e3640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f0ce40d0400 0x7f0ce40cacc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:55:08.274 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:08.271+0000 7f0cea9e5640 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7f0ce40cb200 0x7f0ce40b06c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:55:08.274 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:08.271+0000 7f0ce99e3640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f0ce40d0400 0x7f0ce40cacc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.203:3301/0 says I am v2:172.21.15.43:47022/0 (socket says 172.21.15.43:47022) 2023-08-09T13:55:08.275 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:08.271+0000 7f0ce99e3640 1 -- 172.21.15.43:0/3254154097 learned_addr learned my addr 172.21.15.43:0/3254154097 (peer_addr_for_me v2:172.21.15.43:0/0) 2023-08-09T13:55:08.275 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:08.272+0000 7f0cea9e5640 1 -- 172.21.15.43:0/3254154097 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f0ce40d0400 msgr2=0x7f0ce40cacc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-09T13:55:08.275 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:08.272+0000 7f0cea9e5640 1 --2- 172.21.15.43:0/3254154097 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f0ce40d0400 0x7f0ce40cacc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:55:08.275 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:08.272+0000 7f0cea9e5640 1 -- 172.21.15.43:0/3254154097 >> v1:172.21.15.203:6789/0 conn(0x7f0ce415a350 legacy=0x7f0ce40cfcf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:55:08.275 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:08.272+0000 7f0cea9e5640 1 -- 172.21.15.43:0/3254154097 --> v2:172.21.15.203:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0ce40afec0 con 0x7f0ce40cb200 2023-08-09T13:55:08.276 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:08.274+0000 7f0cea9e5640 1 --2- 172.21.15.43:0/3254154097 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f0ce40cb200 0x7f0ce40b06c0 secure :-1 s=READY pgs=594 cs=0 l=1 rev1=1 crypto rx=0x7f0cdc009d90 tx=0x7f0cdc02be10 comp rx=0 tx=0).ready entity=mon.1 client_cookie=fc741d9664b86618 server_cookie=0 in_seq=0 out_seq=0 2023-08-09T13:55:08.277 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:08.274+0000 7f0ce91e2640 1 -- 172.21.15.43:0/3254154097 <== mon.1 v2:172.21.15.203:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0cdc02cbc0 con 0x7f0ce40cb200 2023-08-09T13:55:08.277 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:08.274+0000 7f0ce91e2640 1 -- 172.21.15.43:0/3254154097 <== mon.1 v2:172.21.15.203:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0cdc02cd20 con 0x7f0ce40cb200 2023-08-09T13:55:08.277 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:08.274+0000 7f0ce91e2640 1 -- 172.21.15.43:0/3254154097 <== mon.1 v2:172.21.15.203:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0cdc034660 con 0x7f0ce40cb200 2023-08-09T13:55:08.278 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:08.275+0000 7f0ceb1e6640 1 -- 172.21.15.43:0/3254154097 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f0ce40cb200 msgr2=0x7f0ce40b06c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:55:08.278 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:08.275+0000 7f0ceb1e6640 1 --2- 172.21.15.43:0/3254154097 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f0ce40cb200 0x7f0ce40b06c0 secure :-1 s=READY pgs=594 cs=0 l=1 rev1=1 crypto rx=0x7f0cdc009d90 tx=0x7f0cdc02be10 comp rx=0 tx=0).stop 2023-08-09T13:55:08.278 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:08.275+0000 7f0ceb1e6640 1 -- 172.21.15.43:0/3254154097 shutdown_connections 2023-08-09T13:55:08.278 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:08.275+0000 7f0ceb1e6640 1 --2- 172.21.15.43:0/3254154097 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f0ce40cb200 0x7f0ce40b06c0 unknown :-1 s=CLOSED pgs=594 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:55:08.279 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:08.275+0000 7f0ceb1e6640 1 --2- 172.21.15.43:0/3254154097 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f0ce40d0400 0x7f0ce40cacc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:55:08.279 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:08.275+0000 7f0ceb1e6640 1 -- 172.21.15.43:0/3254154097 >> 172.21.15.43:0/3254154097 conn(0x7f0ce4089fe0 msgr2=0x7f0ce4099450 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-09T13:55:08.279 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:08.275+0000 7f0ceb1e6640 1 -- 172.21.15.43:0/3254154097 shutdown_connections 2023-08-09T13:55:08.279 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:08.275+0000 7f0ceb1e6640 1 -- 172.21.15.43:0/3254154097 wait complete. 2023-08-09T13:55:08.279 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:08.277+0000 7f0ceb1e6640 1 Processor -- start 2023-08-09T13:55:08.280 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:08.277+0000 7f0ceb1e6640 1 -- start start 2023-08-09T13:55:08.281 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:08.279+0000 7f0ceb1e6640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f0ce40cb200 0x7f0ce40c9fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:55:08.282 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:08.279+0000 7f0ceb1e6640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f0ce40d0400 0x7f0ce406e190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:55:08.282 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:08.279+0000 7f0ceb1e6640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f0ce415a350 0x7f0ce4150370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:55:08.282 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:08.279+0000 7f0ceb1e6640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0ce4157030 con 0x7f0ce415a350 2023-08-09T13:55:08.282 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:08.279+0000 7f0ceb1e6640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0ce40c8fd0 con 0x7f0ce40cb200 2023-08-09T13:55:08.282 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:08.279+0000 7f0ceb1e6640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0ce4157280 con 0x7f0ce40d0400 2023-08-09T13:55:08.283 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:08.279+0000 7f0cea9e5640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f0ce415a350 0x7f0ce4150370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:55:08.283 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:08.279+0000 7f0cea9e5640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f0ce415a350 0x7f0ce4150370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:56438/0 (socket says 172.21.15.43:56438) 2023-08-09T13:55:08.283 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:08.279+0000 7f0cea9e5640 1 -- 172.21.15.43:0/4109319746 learned_addr learned my addr 172.21.15.43:0/4109319746 (peer_addr_for_me v2:172.21.15.43:0/0) 2023-08-09T13:55:08.283 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:08.279+0000 7f0cea1e4640 1 --2- 172.21.15.43:0/4109319746 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f0ce40cb200 0x7f0ce40c9fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:55:08.284 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:08.279+0000 7f0ce99e3640 1 --2- 172.21.15.43:0/4109319746 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f0ce40d0400 0x7f0ce406e190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:55:08.285 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:08.280+0000 7f0cea9e5640 1 -- 172.21.15.43:0/4109319746 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f0ce40d0400 msgr2=0x7f0ce406e190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:55:08.285 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:08.280+0000 7f0cea9e5640 1 --2- 172.21.15.43:0/4109319746 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f0ce40d0400 0x7f0ce406e190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:55:08.285 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:08.280+0000 7f0cea9e5640 1 -- 172.21.15.43:0/4109319746 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f0ce40cb200 msgr2=0x7f0ce40c9fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:55:08.285 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:08.280+0000 7f0cea9e5640 1 --2- 172.21.15.43:0/4109319746 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f0ce40cb200 0x7f0ce40c9fc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:55:08.285 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:08.280+0000 7f0cea9e5640 1 -- 172.21.15.43:0/4109319746 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0cdc009a70 con 0x7f0ce415a350 2023-08-09T13:55:08.285 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:08.280+0000 7f0cea1e4640 1 --2- 172.21.15.43:0/4109319746 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f0ce40cb200 0x7f0ce40c9fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-08-09T13:55:08.286 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:08.280+0000 7f0cea9e5640 1 --2- 172.21.15.43:0/4109319746 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f0ce415a350 0x7f0ce4150370 secure :-1 s=READY pgs=1584 cs=0 l=1 rev1=1 crypto rx=0x7f0cdc009d90 tx=0x7f0cdc004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-09T13:55:08.286 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:08.280+0000 7f0cdaffd640 1 -- 172.21.15.43:0/4109319746 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0cdc040650 con 0x7f0ce415a350 2023-08-09T13:55:08.286 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:08.280+0000 7f0cdaffd640 1 -- 172.21.15.43:0/4109319746 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0cdc02ce90 con 0x7f0ce415a350 2023-08-09T13:55:08.287 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:08.280+0000 7f0cdaffd640 1 -- 172.21.15.43:0/4109319746 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0cdc0409f0 con 0x7f0ce415a350 2023-08-09T13:55:08.287 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:08.280+0000 7f0ceb1e6640 1 -- 172.21.15.43:0/4109319746 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0ce40c9220 con 0x7f0ce415a350 2023-08-09T13:55:08.287 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:08.281+0000 7f0ceb1e6640 1 -- 172.21.15.43:0/4109319746 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0ce40c97e0 con 0x7f0ce415a350 2023-08-09T13:55:08.287 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:08.282+0000 7f0ceb1e6640 1 -- 172.21.15.43:0/4109319746 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0ce40739f0 con 0x7f0ce415a350 2023-08-09T13:55:08.288 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:08.284+0000 7f0cdaffd640 1 -- 172.21.15.43:0/4109319746 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 63) v1 ==== 88557+0+0 (secure 0 0 0) 0x7f0cdc040b50 con 0x7f0ce415a350 2023-08-09T13:55:08.289 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:08.285+0000 7f0cdaffd640 1 --2- 172.21.15.43:0/4109319746 >> [v2:172.21.15.203:6832/1796635339,v1:172.21.15.203:6833/1796635339] conn(0x7f0cc806a290 0x7f0cc806c750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:55:08.289 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:08.285+0000 7f0cdaffd640 1 -- 172.21.15.43:0/4109319746 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(132..132 src has 1..132) v4 ==== 6828+0+0 (secure 0 0 0) 0x7f0cdc0ab900 con 0x7f0ce415a350 2023-08-09T13:55:08.289 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:08.285+0000 7f0cea1e4640 1 -- 172.21.15.43:0/4109319746 >> [v2:172.21.15.203:6832/1796635339,v1:172.21.15.203:6833/1796635339] conn(0x7f0cc806a290 msgr2=0x7f0cc806c750 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.203:6832/1796635339 2023-08-09T13:55:08.289 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:08.285+0000 7f0cea1e4640 1 --2- 172.21.15.43:0/4109319746 >> [v2:172.21.15.203:6832/1796635339,v1:172.21.15.203:6833/1796635339] conn(0x7f0cc806a290 0x7f0cc806c750 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-08-09T13:55:08.292 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:08.290+0000 7f0cdaffd640 1 -- 172.21.15.43:0/4109319746 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178066 (secure 0 0 0) 0x7f0cdc076980 con 0x7f0ce415a350 2023-08-09T13:55:08.334 INFO:tasks.ceph.mgr.y.smithi043.stderr:2023-08-09T13:55:08.332+0000 7f106ca0f040 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2023-08-09T13:55:08.367 INFO:tasks.ceph.mgr.x.smithi203.stderr:2023-08-09T13:55:08.364+0000 7f8eeb831040 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2023-08-09T13:55:08.404 INFO:tasks.ceph.mgr.y.smithi043.stderr:2023-08-09T13:55:08.402+0000 7f106ca0f040 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2023-08-09T13:55:08.419 INFO:tasks.ceph.mgr.x.smithi203.stderr:2023-08-09T13:55:08.416+0000 7f8eeb831040 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2023-08-09T13:55:08.459 INFO:tasks.ceph.mgr.y.smithi043.stderr:2023-08-09T13:55:08.457+0000 7f106ca0f040 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2023-08-09T13:55:08.472 INFO:tasks.ceph.mgr.x.smithi203.stderr:2023-08-09T13:55:08.469+0000 7f8eeb831040 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2023-08-09T13:55:08.488 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:08.486+0000 7f0cea1e4640 1 -- 172.21.15.43:0/4109319746 >> [v2:172.21.15.203:6832/1796635339,v1:172.21.15.203:6833/1796635339] conn(0x7f0cc806a290 msgr2=0x7f0cc806c750 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.203:6832/1796635339 2023-08-09T13:55:08.489 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:08.486+0000 7f0cea1e4640 1 --2- 172.21.15.43:0/4109319746 >> [v2:172.21.15.203:6832/1796635339,v1:172.21.15.203:6833/1796635339] conn(0x7f0cc806a290 0x7f0cc806c750 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2023-08-09T13:55:08.526 INFO:tasks.ceph.mgr.x.smithi203.stderr:2023-08-09T13:55:08.523+0000 7f8eeb831040 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2023-08-09T13:55:08.541 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:08.539+0000 7f0ceb1e6640 1 -- 172.21.15.43:0/4109319746 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f0ce40afca0 con 0x7f0ce415a350 2023-08-09T13:55:08.546 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:08.543+0000 7f0cdaffd640 1 -- 172.21.15.43:0/4109319746 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mgrmap(e 64) v1 ==== 88052+0+0 (secure 0 0 0) 0x7f0cdc0a9d50 con 0x7f0ce415a350 2023-08-09T13:55:08.547 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:08.544+0000 7f0cdaffd640 1 -- 172.21.15.43:0/4109319746 >> [v2:172.21.15.203:6832/1796635339,v1:172.21.15.203:6833/1796635339] conn(0x7f0cc806a290 msgr2=0x7f0cc806c750 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-08-09T13:55:08.547 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:08.544+0000 7f0cdaffd640 1 --2- 172.21.15.43:0/4109319746 >> [v2:172.21.15.203:6832/1796635339,v1:172.21.15.203:6833/1796635339] conn(0x7f0cc806a290 0x7f0cc806c750 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:55:08.550 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:08.547+0000 7f0cdaffd640 1 -- 172.21.15.43:0/4109319746 <== mon.0 v2:172.21.15.43:3300/0 8 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 61 v61) v1 ==== 94+0+4880 (secure 0 0 0) 0x7f0cdc08dc00 con 0x7f0ce415a350 2023-08-09T13:55:08.550 INFO:teuthology.orchestra.run.smithi043.stdout: 2023-08-09T13:55:08.551 INFO:teuthology.orchestra.run.smithi043.stdout:{"epoch":61,"default_fscid":7,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6530,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.203:6835/1659419253","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6834","nonce":1659419253},{"type":"v1","addr":"172.21.15.203:6835","nonce":1659419253}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":56},{"gid":6536,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.203:6837/3964427549","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6836","nonce":3964427549},{"type":"v1","addr":"172.21.15.203:6837","nonce":3964427549}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":56},{"gid":8574,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.43:6833/871812392","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6832","nonce":871812392},{"type":"v1","addr":"172.21.15.43:6833","nonce":871812392}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":55}],"filesystems":[{"mdsmap":{"epoch":61,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-08-09T13:53:47.747881+0000","modified":"2023-08-09T13:53:51.317185+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":8580},"failed":[],"damaged":[],"stopped":[],"info":{"gid_8580":{"gid":8580,"name":"a","rank":0,"incarnation":57,"state":"up:active","state_seq":3,"addr":"172.21.15.43:6835/1794836673","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6834","nonce":1794836673},{"type":"v1","addr":"172.21.15.43:6835","nonce":1794836673}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[17],"metadata_pool":16,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":7}]} 2023-08-09T13:55:08.551 INFO:teuthology.orchestra.run.smithi043.stderr:dumped fsmap epoch 61 2023-08-09T13:55:08.553 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:08.550+0000 7f0ceb1e6640 1 -- 172.21.15.43:0/4109319746 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f0ce415a350 msgr2=0x7f0ce4150370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:55:08.554 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:08.550+0000 7f0ceb1e6640 1 --2- 172.21.15.43:0/4109319746 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f0ce415a350 0x7f0ce4150370 secure :-1 s=READY pgs=1584 cs=0 l=1 rev1=1 crypto rx=0x7f0cdc009d90 tx=0x7f0cdc004a40 comp rx=0 tx=0).stop 2023-08-09T13:55:08.554 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:08.550+0000 7f0ceb1e6640 1 -- 172.21.15.43:0/4109319746 shutdown_connections 2023-08-09T13:55:08.554 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:08.550+0000 7f0ceb1e6640 1 --2- 172.21.15.43:0/4109319746 >> [v2:172.21.15.203:6832/1796635339,v1:172.21.15.203:6833/1796635339] conn(0x7f0cc806a290 0x7f0cc806c750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:55:08.554 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:08.550+0000 7f0ceb1e6640 1 --2- 172.21.15.43:0/4109319746 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f0ce415a350 0x7f0ce4150370 unknown :-1 s=CLOSED pgs=1584 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:55:08.555 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:08.550+0000 7f0ceb1e6640 1 --2- 172.21.15.43:0/4109319746 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f0ce40d0400 0x7f0ce406e190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:55:08.555 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:08.550+0000 7f0ceb1e6640 1 --2- 172.21.15.43:0/4109319746 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f0ce40cb200 0x7f0ce40c9fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:55:08.555 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:08.550+0000 7f0ceb1e6640 1 -- 172.21.15.43:0/4109319746 >> 172.21.15.43:0/4109319746 conn(0x7f0ce4089fe0 msgr2=0x7f0ce406d710 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-09T13:55:08.555 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:08.550+0000 7f0ceb1e6640 1 -- 172.21.15.43:0/4109319746 shutdown_connections 2023-08-09T13:55:08.555 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:08.551+0000 7f0ceb1e6640 1 -- 172.21.15.43:0/4109319746 wait complete. 2023-08-09T13:55:08.579 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-08-09T13:55:08.622 INFO:tasks.ceph.mgr.x.smithi203.stderr:2023-08-09T13:55:08.619+0000 7f8e70cde640 -1 client.0 error registering admin socket command: (17) File exists 2023-08-09T13:55:08.622 INFO:tasks.ceph.mgr.x.smithi203.stderr:2023-08-09T13:55:08.620+0000 7f8e70cde640 -1 client.0 error registering admin socket command: (17) File exists 2023-08-09T13:55:08.622 INFO:tasks.ceph.mgr.x.smithi203.stderr:2023-08-09T13:55:08.620+0000 7f8e70cde640 -1 client.0 error registering admin socket command: (17) File exists 2023-08-09T13:55:08.623 INFO:tasks.ceph.mgr.x.smithi203.stderr:2023-08-09T13:55:08.620+0000 7f8e70cde640 -1 client.0 error registering admin socket command: (17) File exists 2023-08-09T13:55:08.623 INFO:tasks.ceph.mgr.x.smithi203.stderr:2023-08-09T13:55:08.620+0000 7f8e70cde640 -1 client.0 error registering admin socket command: (17) File exists 2023-08-09T13:55:08.623 INFO:tasks.ceph.mgr.x.smithi203.stderr:2023-08-09T13:55:08.620+0000 7f8e6c4d5640 -1 client.0 error registering admin socket command: (17) File exists 2023-08-09T13:55:08.623 INFO:tasks.ceph.mgr.x.smithi203.stderr:2023-08-09T13:55:08.620+0000 7f8e6c4d5640 -1 client.0 error registering admin socket command: (17) File exists 2023-08-09T13:55:08.623 INFO:tasks.ceph.mgr.x.smithi203.stderr:2023-08-09T13:55:08.620+0000 7f8e6c4d5640 -1 client.0 error registering admin socket command: (17) File exists 2023-08-09T13:55:08.623 INFO:tasks.ceph.mgr.x.smithi203.stderr:2023-08-09T13:55:08.620+0000 7f8e6c4d5640 -1 client.0 error registering admin socket command: (17) File exists 2023-08-09T13:55:08.623 INFO:tasks.ceph.mgr.x.smithi203.stderr:2023-08-09T13:55:08.620+0000 7f8e6c4d5640 -1 client.0 error registering admin socket command: (17) File exists 2023-08-09T13:55:08.650 INFO:tasks.ceph.mgr.y.smithi043.stderr:2023-08-09T13:55:08.648+0000 7f106ca0f040 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2023-08-09T13:55:08.703 INFO:tasks.ceph.mgr.y.smithi043.stderr:2023-08-09T13:55:08.700+0000 7f106ca0f040 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2023-08-09T13:55:08.761 INFO:tasks.ceph.mgr.y.smithi043.stderr:2023-08-09T13:55:08.759+0000 7f106ca0f040 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2023-08-09T13:55:08.770 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:08.768+0000 7f5cabfff640 1 Processor -- start 2023-08-09T13:55:08.770 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:08.768+0000 7f5cabfff640 1 -- start start 2023-08-09T13:55:08.772 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:08.769+0000 7f5cabfff640 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7f5cac0b1050 0x7f5cac0b4760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:55:08.772 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:08.769+0000 7f5cabfff640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f5cac0cd6b0 0x7f5cac0d0e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:55:08.772 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:08.769+0000 7f5cabfff640 1 -- --> v1:172.21.15.43:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5cac0b7bf0 con 0x7f5cac15f7b0 2023-08-09T13:55:08.772 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:08.769+0000 7f5cabfff640 1 -- --> v2:172.21.15.203:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5cac0b7ed0 con 0x7f5cac0b1050 2023-08-09T13:55:08.772 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:08.769+0000 7f5cabfff640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5cac0d0370 con 0x7f5cac0cd6b0 2023-08-09T13:55:08.773 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:08.770+0000 7f5caaffd640 1 --1- >> v1:172.21.15.43:6789/0 conn(0x7f5cac15f7b0 0x7f5cac0ccfa0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.43:6789/0 says I am v1:172.21.15.43:42174/0 (socket says 172.21.15.43:42174) 2023-08-09T13:55:08.773 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:08.770+0000 7f5caaffd640 1 -- 172.21.15.43:0/1791158018 learned_addr learned my addr 172.21.15.43:0/1791158018 (peer_addr_for_me v1:172.21.15.43:0/0) 2023-08-09T13:55:08.774 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:08.770+0000 7f5ca3fff640 1 --2- 172.21.15.43:0/1791158018 >> v2:172.21.15.203:3300/0 conn(0x7f5cac0b1050 0x7f5cac0b4760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:55:08.775 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:08.770+0000 7f5cab7fe640 1 --2- 172.21.15.43:0/1791158018 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f5cac0cd6b0 0x7f5cac0d0e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:55:08.775 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:08.770+0000 7f5caa7fc640 1 -- 172.21.15.43:0/1791158018 <== mon.0 v1:172.21.15.43:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 440031149 0 0) 0x7f5cac0b7bf0 con 0x7f5cac15f7b0 2023-08-09T13:55:08.775 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:08.770+0000 7f5caa7fc640 1 -- 172.21.15.43:0/1791158018 --> v1:172.21.15.43:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f5c98003520 con 0x7f5cac15f7b0 2023-08-09T13:55:08.775 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:08.771+0000 7f5caa7fc640 1 -- 172.21.15.43:0/1791158018 <== mon.0 v1:172.21.15.43:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 42106662 0 0) 0x7f5c98003520 con 0x7f5cac15f7b0 2023-08-09T13:55:08.775 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:08.771+0000 7f5caa7fc640 1 -- 172.21.15.43:0/1791158018 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f5cac0cd6b0 msgr2=0x7f5cac0d0e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-09T13:55:08.775 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:08.771+0000 7f5caa7fc640 1 --2- 172.21.15.43:0/1791158018 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f5cac0cd6b0 0x7f5cac0d0e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:55:08.775 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:08.771+0000 7f5caa7fc640 1 -- 172.21.15.43:0/1791158018 >> v2:172.21.15.203:3300/0 conn(0x7f5cac0b1050 msgr2=0x7f5cac0b4760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-09T13:55:08.775 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:08.771+0000 7f5caa7fc640 1 --2- 172.21.15.43:0/1791158018 >> v2:172.21.15.203:3300/0 conn(0x7f5cac0b1050 0x7f5cac0b4760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:55:08.776 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:08.771+0000 7f5caa7fc640 1 -- 172.21.15.43:0/1791158018 --> v1:172.21.15.43:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5cac0d0650 con 0x7f5cac15f7b0 2023-08-09T13:55:08.776 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:08.771+0000 7f5caa7fc640 1 -- 172.21.15.43:0/1791158018 <== mon.0 v1:172.21.15.43:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1285150740 0 0) 0x7f5c9c001670 con 0x7f5cac15f7b0 2023-08-09T13:55:08.776 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:08.771+0000 7f5caa7fc640 1 -- 172.21.15.43:0/1791158018 >> v1:172.21.15.43:6789/0 conn(0x7f5cac15f7b0 legacy=0x7f5cac0ccfa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:55:08.776 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:08.771+0000 7f5caa7fc640 1 --2- 172.21.15.43:0/1791158018 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f5cac0cd6b0 0x7f5c98003970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:55:08.776 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:08.771+0000 7f5caa7fc640 1 --2- 172.21.15.43:0/1791158018 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f5c98003ef0 0x7f5c980047c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:55:08.776 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:08.771+0000 7f5caa7fc640 1 --2- 172.21.15.43:0/1791158018 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f5c98004d00 0x7f5c98005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:55:08.776 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:08.771+0000 7f5caa7fc640 1 -- 172.21.15.43:0/1791158018 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5cac0b7ed0 con 0x7f5c98003ef0 2023-08-09T13:55:08.777 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:08.771+0000 7f5caa7fc640 1 -- 172.21.15.43:0/1791158018 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5cac0d0370 con 0x7f5c98004d00 2023-08-09T13:55:08.777 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:08.771+0000 7f5caa7fc640 1 -- 172.21.15.43:0/1791158018 --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5c9800b980 con 0x7f5cac0cd6b0 2023-08-09T13:55:08.777 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:08.771+0000 7f5ca3fff640 1 --2- 172.21.15.43:0/1791158018 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f5c98003ef0 0x7f5c980047c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:55:08.777 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:08.771+0000 7f5cab7fe640 1 --2- 172.21.15.43:0/1791158018 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f5c98004d00 0x7f5c98005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:55:08.777 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:08.771+0000 7f5ca3fff640 1 -- 172.21.15.43:0/1791158018 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f5cac0cd6b0 msgr2=0x7f5c98003970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-09T13:55:08.777 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:08.771+0000 7f5ca3fff640 1 --2- 172.21.15.43:0/1791158018 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f5cac0cd6b0 0x7f5c98003970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:55:08.777 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:08.772+0000 7f5ca3fff640 1 -- 172.21.15.43:0/1791158018 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f5c98004d00 msgr2=0x7f5c98005180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-09T13:55:08.777 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:08.772+0000 7f5ca3fff640 1 --2- 172.21.15.43:0/1791158018 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f5c98004d00 0x7f5c98005180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:55:08.778 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:08.772+0000 7f5ca3fff640 1 -- 172.21.15.43:0/1791158018 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5c98003520 con 0x7f5c98003ef0 2023-08-09T13:55:08.779 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:08.773+0000 7f5ca3fff640 1 --2- 172.21.15.43:0/1791158018 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f5c98003ef0 0x7f5c980047c0 secure :-1 s=READY pgs=1587 cs=0 l=1 rev1=1 crypto rx=0x7f5c94009a90 tx=0x7f5c980057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1a8465216a05bd09 server_cookie=0 in_seq=0 out_seq=0 2023-08-09T13:55:08.779 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:08.773+0000 7f5caa7fc640 1 -- 172.21.15.43:0/1791158018 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5c9402c6b0 con 0x7f5c98003ef0 2023-08-09T13:55:08.779 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:08.773+0000 7f5caa7fc640 1 -- 172.21.15.43:0/1791158018 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5c9402c810 con 0x7f5c98003ef0 2023-08-09T13:55:08.779 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:08.773+0000 7f5caa7fc640 1 -- 172.21.15.43:0/1791158018 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5c9402cbc0 con 0x7f5c98003ef0 2023-08-09T13:55:08.780 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:08.774+0000 7f5cabfff640 1 -- 172.21.15.43:0/1791158018 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f5c98003ef0 msgr2=0x7f5c980047c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:55:08.780 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:08.774+0000 7f5cabfff640 1 --2- 172.21.15.43:0/1791158018 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f5c98003ef0 0x7f5c980047c0 secure :-1 s=READY pgs=1587 cs=0 l=1 rev1=1 crypto rx=0x7f5c94009a90 tx=0x7f5c980057f0 comp rx=0 tx=0).stop 2023-08-09T13:55:08.780 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:08.774+0000 7f5cab7fe640 1 -- 172.21.15.43:0/1791158018 reap_dead start 2023-08-09T13:55:08.780 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:08.774+0000 7f5cabfff640 1 -- 172.21.15.43:0/1791158018 shutdown_connections 2023-08-09T13:55:08.780 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:08.774+0000 7f5cabfff640 1 -- 172.21.15.43:0/1791158018 >> 172.21.15.43:0/1791158018 conn(0x7f5cac06a2b0 msgr2=0x7f5cac06a690 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-09T13:55:08.780 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:08.774+0000 7f5cabfff640 1 -- 172.21.15.43:0/1791158018 shutdown_connections 2023-08-09T13:55:08.780 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:08.774+0000 7f5cabfff640 1 -- 172.21.15.43:0/1791158018 wait complete. 2023-08-09T13:55:08.781 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:08.776+0000 7f5cabfff640 1 Processor -- start 2023-08-09T13:55:08.781 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:08.776+0000 7f5cabfff640 1 -- start start 2023-08-09T13:55:08.781 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:08.777+0000 7f5cabfff640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f5c98003ef0 0x7f5cac158740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:55:08.781 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:08.777+0000 7f5cabfff640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f5cac15ce90 0x7f5cac0b6010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:55:08.781 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:08.777+0000 7f5cabfff640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f5cac15d310 0x7f5cac0b6550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:55:08.781 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:08.777+0000 7f5cabfff640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5cac0ca590 con 0x7f5cac15ce90 2023-08-09T13:55:08.781 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:08.777+0000 7f5cabfff640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5cac0ca870 con 0x7f5c98003ef0 2023-08-09T13:55:08.781 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:08.777+0000 7f5cabfff640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5cac0cab50 con 0x7f5cac15d310 2023-08-09T13:55:08.783 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:08.777+0000 7f5ca3fff640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f5cac15ce90 0x7f5cac0b6010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:55:08.783 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:08.777+0000 7f5ca3fff640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f5cac15ce90 0x7f5cac0b6010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:56456/0 (socket says 172.21.15.43:56456) 2023-08-09T13:55:08.783 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:08.777+0000 7f5ca3fff640 1 -- 172.21.15.43:0/2059495433 learned_addr learned my addr 172.21.15.43:0/2059495433 (peer_addr_for_me v2:172.21.15.43:0/0) 2023-08-09T13:55:08.783 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:08.777+0000 7f5caaffd640 1 --2- 172.21.15.43:0/2059495433 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f5c98003ef0 0x7f5cac158740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:55:08.783 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:08.777+0000 7f5cab7fe640 1 --2- 172.21.15.43:0/2059495433 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f5cac15d310 0x7f5cac0b6550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:55:08.783 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:08.777+0000 7f5ca3fff640 1 -- 172.21.15.43:0/2059495433 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f5cac15d310 msgr2=0x7f5cac0b6550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:55:08.784 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:08.777+0000 7f5ca3fff640 1 --2- 172.21.15.43:0/2059495433 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f5cac15d310 0x7f5cac0b6550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:55:08.784 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:08.778+0000 7f5ca3fff640 1 -- 172.21.15.43:0/2059495433 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f5c98003ef0 msgr2=0x7f5cac158740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:55:08.784 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:08.778+0000 7f5ca3fff640 1 --2- 172.21.15.43:0/2059495433 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f5c98003ef0 0x7f5cac158740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:55:08.784 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:08.778+0000 7f5ca3fff640 1 -- 172.21.15.43:0/2059495433 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5cac0cae30 con 0x7f5cac15ce90 2023-08-09T13:55:08.784 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:08.778+0000 7f5ca3fff640 1 --2- 172.21.15.43:0/2059495433 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f5cac15ce90 0x7f5cac0b6010 secure :-1 s=READY pgs=1588 cs=0 l=1 rev1=1 crypto rx=0x7f5c94037040 tx=0x7f5c94004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-09T13:55:08.784 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:08.778+0000 7f5ca37fe640 1 -- 172.21.15.43:0/2059495433 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5c94034c80 con 0x7f5cac15ce90 2023-08-09T13:55:08.784 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:08.778+0000 7f5ca37fe640 1 -- 172.21.15.43:0/2059495433 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5c94034de0 con 0x7f5cac15ce90 2023-08-09T13:55:08.784 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:08.778+0000 7f5cabfff640 1 -- 172.21.15.43:0/2059495433 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5cac0cb110 con 0x7f5cac15ce90 2023-08-09T13:55:08.785 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:08.778+0000 7f5ca37fe640 1 -- 172.21.15.43:0/2059495433 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5c9403e680 con 0x7f5cac15ce90 2023-08-09T13:55:08.785 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:08.778+0000 7f5cabfff640 1 -- 172.21.15.43:0/2059495433 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5cac0b0be0 con 0x7f5cac15ce90 2023-08-09T13:55:08.785 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:08.780+0000 7f5cabfff640 1 -- 172.21.15.43:0/2059495433 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5cac0b3840 con 0x7f5cac15ce90 2023-08-09T13:55:08.785 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:08.781+0000 7f5ca37fe640 1 -- 172.21.15.43:0/2059495433 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 64) v1 ==== 88052+0+0 (secure 0 0 0) 0x7f5c9403e7e0 con 0x7f5cac15ce90 2023-08-09T13:55:08.786 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:08.782+0000 7f5ca37fe640 1 -- 172.21.15.43:0/2059495433 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(134..134 src has 1..134) v4 ==== 7286+0+0 (secure 0 0 0) 0x7f5c940aaa40 con 0x7f5cac15ce90 2023-08-09T13:55:08.788 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:08.786+0000 7f5ca37fe640 1 -- 172.21.15.43:0/2059495433 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178066 (secure 0 0 0) 0x7f5c94076040 con 0x7f5cac15ce90 2023-08-09T13:55:08.816 INFO:tasks.ceph.mgr.y.smithi043.stderr:2023-08-09T13:55:08.813+0000 7f106ca0f040 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2023-08-09T13:55:08.991 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:08.989+0000 7f5cabfff640 1 -- 172.21.15.43:0/2059495433 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f5cac0739f0 con 0x7f5cac15ce90 2023-08-09T13:55:08.994 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:08.992+0000 7f5ca37fe640 1 -- 172.21.15.43:0/2059495433 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v134) v1 ==== 74+0+19819 (secure 0 0 0) 0x7f5c940a2e60 con 0x7f5cac15ce90 2023-08-09T13:55:08.995 INFO:teuthology.orchestra.run.smithi043.stdout: 2023-08-09T13:55:08.995 INFO:teuthology.orchestra.run.smithi043.stdout:{"epoch":134,"fsid":"daac87ee-1337-4250-a955-aea4e408a39b","created":"2023-08-09T13:33:41.701198+0000","modified":"2023-08-09T13:55:08.528629+0000","last_up_change":"2023-08-09T13:33:53.783453+0000","last_in_change":"2023-08-09T13:33:45.148904+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":17,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":"rbd","create_time":"2023-08-09T13:33:54.365137+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":".mgr","create_time":"2023-08-09T13:33:54.831803+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":16,"pool_name":"cephfs_metadata","create_time":"2023-08-09T13:53:46.124616+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"128","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":17,"pool_name":"cephfs_data","create_time":"2023-08-09T13:53:46.762733+0000","flags":8193,"flags_names":"hashpspool,selfmanaged_snaps","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"133","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":3,"snap_epoch":133,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"cbd49ecd-e896-4fbd-864e-5c37a829b165","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":127,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6800","nonce":502667335},{"type":"v1","addr":"172.21.15.43:6801","nonce":502667335}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6802","nonce":502667335},{"type":"v1","addr":"172.21.15.43:6803","nonce":502667335}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6806","nonce":502667335},{"type":"v1","addr":"172.21.15.43:6807","nonce":502667335}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6804","nonce":502667335},{"type":"v1","addr":"172.21.15.43:6805","nonce":502667335}]},"public_addr":"172.21.15.43:6801/502667335","cluster_addr":"172.21.15.43:6803/502667335","heartbeat_back_addr":"172.21.15.43:6807/502667335","heartbeat_front_addr":"172.21.15.43:6805/502667335","state":["exists","up"]},{"osd":1,"uuid":"75c7e0fe-1ec7-4fe8-a2f2-3661a65b8500","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":131,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6808","nonce":3289134610},{"type":"v1","addr":"172.21.15.43:6809","nonce":3289134610}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6810","nonce":3289134610},{"type":"v1","addr":"172.21.15.43:6812","nonce":3289134610}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6821","nonce":3289134610},{"type":"v1","addr":"172.21.15.43:6824","nonce":3289134610}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6815","nonce":3289134610},{"type":"v1","addr":"172.21.15.43:6818","nonce":3289134610}]},"public_addr":"172.21.15.43:6809/3289134610","cluster_addr":"172.21.15.43:6812/3289134610","heartbeat_back_addr":"172.21.15.43:6824/3289134610","heartbeat_front_addr":"172.21.15.43:6818/3289134610","state":["exists","up"]},{"osd":2,"uuid":"bb5c41e9-7f5f-4125-9b01-3bb8bfc2ea83","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":127,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6811","nonce":4079709736},{"type":"v1","addr":"172.21.15.43:6814","nonce":4079709736}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6817","nonce":4079709736},{"type":"v1","addr":"172.21.15.43:6820","nonce":4079709736}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6829","nonce":4079709736},{"type":"v1","addr":"172.21.15.43:6831","nonce":4079709736}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6823","nonce":4079709736},{"type":"v1","addr":"172.21.15.43:6826","nonce":4079709736}]},"public_addr":"172.21.15.43:6814/4079709736","cluster_addr":"172.21.15.43:6820/4079709736","heartbeat_back_addr":"172.21.15.43:6831/4079709736","heartbeat_front_addr":"172.21.15.43:6826/4079709736","state":["exists","up"]},{"osd":3,"uuid":"bfe4c8b4-694a-465c-87e6-2b2aab852613","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":127,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6813","nonce":1709538759},{"type":"v1","addr":"172.21.15.43:6816","nonce":1709538759}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6819","nonce":1709538759},{"type":"v1","addr":"172.21.15.43:6822","nonce":1709538759}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6828","nonce":1709538759},{"type":"v1","addr":"172.21.15.43:6830","nonce":1709538759}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6825","nonce":1709538759},{"type":"v1","addr":"172.21.15.43:6827","nonce":1709538759}]},"public_addr":"172.21.15.43:6816/1709538759","cluster_addr":"172.21.15.43:6822/1709538759","heartbeat_back_addr":"172.21.15.43:6830/1709538759","heartbeat_front_addr":"172.21.15.43:6827/1709538759","state":["exists","up"]},{"osd":4,"uuid":"fa2e8a77-aba1-4756-853e-0d174a5bdeba","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":127,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6800","nonce":4196866481},{"type":"v1","addr":"172.21.15.203:6801","nonce":4196866481}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6802","nonce":4196866481},{"type":"v1","addr":"172.21.15.203:6803","nonce":4196866481}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6806","nonce":4196866481},{"type":"v1","addr":"172.21.15.203:6807","nonce":4196866481}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6804","nonce":4196866481},{"type":"v1","addr":"172.21.15.203:6805","nonce":4196866481}]},"public_addr":"172.21.15.203:6801/4196866481","cluster_addr":"172.21.15.203:6803/4196866481","heartbeat_back_addr":"172.21.15.203:6807/4196866481","heartbeat_front_addr":"172.21.15.203:6805/4196866481","state":["exists","up"]},{"osd":5,"uuid":"81811e4b-4d24-4f51-a82c-f1d700a2a00d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":127,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6816","nonce":3059290081},{"type":"v1","addr":"172.21.15.203:6817","nonce":3059290081}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6818","nonce":3059290081},{"type":"v1","addr":"172.21.15.203:6819","nonce":3059290081}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6822","nonce":3059290081},{"type":"v1","addr":"172.21.15.203:6823","nonce":3059290081}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6820","nonce":3059290081},{"type":"v1","addr":"172.21.15.203:6821","nonce":3059290081}]},"public_addr":"172.21.15.203:6817/3059290081","cluster_addr":"172.21.15.203:6819/3059290081","heartbeat_back_addr":"172.21.15.203:6823/3059290081","heartbeat_front_addr":"172.21.15.203:6821/3059290081","state":["exists","up"]},{"osd":6,"uuid":"307bac4d-6a5b-443d-af75-56d695d7674f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":131,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6824","nonce":512155569},{"type":"v1","addr":"172.21.15.203:6825","nonce":512155569}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6826","nonce":512155569},{"type":"v1","addr":"172.21.15.203:6827","nonce":512155569}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6830","nonce":512155569},{"type":"v1","addr":"172.21.15.203:6831","nonce":512155569}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6828","nonce":512155569},{"type":"v1","addr":"172.21.15.203:6829","nonce":512155569}]},"public_addr":"172.21.15.203:6825/512155569","cluster_addr":"172.21.15.203:6827/512155569","heartbeat_back_addr":"172.21.15.203:6831/512155569","heartbeat_front_addr":"172.21.15.203:6829/512155569","state":["exists","up"]},{"osd":7,"uuid":"a95dc7ca-9f1b-4295-af70-68a22a2d7d5e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":127,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6808","nonce":3331450438},{"type":"v1","addr":"172.21.15.203:6809","nonce":3331450438}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6810","nonce":3331450438},{"type":"v1","addr":"172.21.15.203:6811","nonce":3331450438}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6814","nonce":3331450438},{"type":"v1","addr":"172.21.15.203:6815","nonce":3331450438}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6812","nonce":3331450438},{"type":"v1","addr":"172.21.15.203:6813","nonce":3331450438}]},"public_addr":"172.21.15.203:6809/3331450438","cluster_addr":"172.21.15.203:6811/3331450438","heartbeat_back_addr":"172.21.15.203:6815/3331450438","heartbeat_front_addr":"172.21.15.203:6813/3331450438","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-09T13:33:50.547652+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-09T13:33:50.589469+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-09T13:33:50.561724+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-09T13:33:50.599650+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-09T13:33:50.411855+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-09T13:33:50.219681+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-09T13:33:50.504664+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-09T13:33:50.441248+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"17.5","mappings":[{"from":0,"to":5}]},{"pgid":"17.f","mappings":[{"from":0,"to":2}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.203:0/1151006972":"2023-08-10T13:55:08.528494+0000","172.21.15.203:0/274636031":"2023-08-10T13:55:08.528494+0000","172.21.15.203:0/3526078924":"2023-08-10T13:55:08.528494+0000","172.21.15.203:6833/1796635339":"2023-08-10T13:55:08.528494+0000","172.21.15.203:6832/1796635339":"2023-08-10T13:55:08.528494+0000","172.21.15.203:0/1153043111":"2023-08-10T13:55:08.528494+0000","172.21.15.203:0/3657978600":"2023-08-10T13:55:08.528494+0000","172.21.15.203:0/3093730384":"2023-08-10T13:55:08.528494+0000","172.21.15.203:6833/2527059680":"2023-08-10T13:54:03.238790+0000","172.21.15.203:6832/2527059680":"2023-08-10T13:54:03.238790+0000","172.21.15.203:0/431304609":"2023-08-10T13:54:03.238790+0000","172.21.15.203:0/1911913635":"2023-08-10T13:55:08.528494+0000","172.21.15.203:0/826469783":"2023-08-10T13:54:03.238790+0000","172.21.15.203:0/1225835978":"2023-08-10T13:55:08.528494+0000","172.21.15.203:0/1456953448":"2023-08-10T13:54:03.238790+0000","172.21.15.203:0/2004944696":"2023-08-10T13:54:03.238790+0000","172.21.15.203:0/916024371":"2023-08-10T13:54:03.238790+0000","172.21.15.203:0/4209557762":"2023-08-10T13:54:03.238790+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":7,"snaps":[{"begin":104,"length":1},{"begin":221,"length":1}]},{"pool":17,"snaps":[{"begin":2,"length":2}]}],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-08-09T13:55:08.997 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:08.995+0000 7f5cabfff640 1 -- 172.21.15.43:0/2059495433 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f5cac15ce90 msgr2=0x7f5cac0b6010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:55:08.997 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:08.995+0000 7f5cabfff640 1 --2- 172.21.15.43:0/2059495433 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f5cac15ce90 0x7f5cac0b6010 secure :-1 s=READY pgs=1588 cs=0 l=1 rev1=1 crypto rx=0x7f5c94037040 tx=0x7f5c94004730 comp rx=0 tx=0).stop 2023-08-09T13:55:08.998 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:08.995+0000 7f5cabfff640 1 -- 172.21.15.43:0/2059495433 shutdown_connections 2023-08-09T13:55:08.998 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:08.995+0000 7f5cabfff640 1 --2- 172.21.15.43:0/2059495433 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f5cac15d310 0x7f5cac0b6550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:55:08.998 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:08.995+0000 7f5cabfff640 1 --2- 172.21.15.43:0/2059495433 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f5cac15ce90 0x7f5cac0b6010 unknown :-1 s=CLOSED pgs=1588 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:55:08.998 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:08.995+0000 7f5cabfff640 1 --2- 172.21.15.43:0/2059495433 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f5c98003ef0 0x7f5cac158740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:55:08.998 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:08.995+0000 7f5cabfff640 1 -- 172.21.15.43:0/2059495433 >> 172.21.15.43:0/2059495433 conn(0x7f5cac06a2b0 msgr2=0x7f5cac15dc90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-09T13:55:08.999 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:08.995+0000 7f5cabfff640 1 -- 172.21.15.43:0/2059495433 shutdown_connections 2023-08-09T13:55:08.999 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:08.995+0000 7f5cabfff640 1 -- 172.21.15.43:0/2059495433 wait complete. 2023-08-09T13:55:09.016 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': 'rbd', 'create_time': '2023-08-09T13:33:54.365137+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}, 'read_balance': {'score_acting': 2, 'score_stable': 2, 'optimal_score': 1, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': '.mgr', 'create_time': '2023-08-09T13:33:54.831803+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 16, 'pool_name': 'cephfs_metadata', 'create_time': '2023-08-09T13:53:46.124616+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '128', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.75, 'score_stable': 1.75, 'optimal_score': 1, 'raw_score_acting': 1.75, 'raw_score_stable': 1.75, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 17, 'pool_name': 'cephfs_data', 'create_time': '2023-08-09T13:53:46.762733+0000', 'flags': 8193, 'flags_names': 'hashpspool,selfmanaged_snaps', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '133', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 3, 'snap_epoch': 133, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.75, 'score_stable': 1.75, 'optimal_score': 1, 'raw_score_acting': 1.75, 'raw_score_stable': 1.75, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-08-09T13:55:09.017 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get rbd pg_num 2023-08-09T13:55:09.240 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:09.238+0000 7f69f5c21640 1 Processor -- start 2023-08-09T13:55:09.241 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:09.238+0000 7f69f5c21640 1 -- start start 2023-08-09T13:55:09.242 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:09.240+0000 7f69f5c21640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f69f00b7df0 0x7f69f00d1b30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:55:09.242 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:09.240+0000 7f69f5c21640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f69f00cebf0 0x7f69f00cf040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:55:09.242 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:09.240+0000 7f69f5c21640 1 -- --> v1:172.21.15.43:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f69f00cf580 con 0x7f69f015c2d0 2023-08-09T13:55:09.243 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:09.240+0000 7f69f5c21640 1 -- --> v2:172.21.15.43:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f69f00b8230 con 0x7f69f00b7df0 2023-08-09T13:55:09.243 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:09.240+0000 7f69f5c21640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f69f00d16a0 con 0x7f69f00cebf0 2023-08-09T13:55:09.243 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:09.240+0000 7f69e7fff640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f69f00b7df0 0x7f69f00d1b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:55:09.244 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:09.240+0000 7f69e7fff640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f69f00b7df0 0x7f69f00d1b30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:56458/0 (socket says 172.21.15.43:56458) 2023-08-09T13:55:09.244 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:09.240+0000 7f69e7fff640 1 -- 172.21.15.43:0/2719261110 learned_addr learned my addr 172.21.15.43:0/2719261110 (peer_addr_for_me v2:172.21.15.43:0/0) 2023-08-09T13:55:09.244 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:09.240+0000 7f69f4c1f640 1 --1- 172.21.15.43:0/2719261110 >> v1:172.21.15.43:6789/0 conn(0x7f69f015c2d0 0x7f69f00cd020 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.43:6789/0 says I am v1:172.21.15.43:42190/0 (socket says 172.21.15.43:42190) 2023-08-09T13:55:09.244 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:09.241+0000 7f69f5420640 1 --2- 172.21.15.43:0/2719261110 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f69f00cebf0 0x7f69f00cf040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:55:09.244 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:09.241+0000 7f69e7fff640 1 -- 172.21.15.43:0/2719261110 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f69f00cebf0 msgr2=0x7f69f00cf040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-09T13:55:09.244 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:09.241+0000 7f69e7fff640 1 --2- 172.21.15.43:0/2719261110 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f69f00cebf0 0x7f69f00cf040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:55:09.244 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:09.241+0000 7f69e7fff640 1 -- 172.21.15.43:0/2719261110 >> v1:172.21.15.43:6789/0 conn(0x7f69f015c2d0 legacy=0x7f69f00cd020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:55:09.245 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:09.241+0000 7f69e7fff640 1 -- 172.21.15.43:0/2719261110 --> v2:172.21.15.43:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f69f00d0d70 con 0x7f69f00b7df0 2023-08-09T13:55:09.245 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:09.241+0000 7f69f5420640 1 --2- 172.21.15.43:0/2719261110 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f69f00cebf0 0x7f69f00cf040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-08-09T13:55:09.246 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:09.242+0000 7f69e7fff640 1 --2- 172.21.15.43:0/2719261110 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f69f00b7df0 0x7f69f00d1b30 secure :-1 s=READY pgs=1590 cs=0 l=1 rev1=1 crypto rx=0x7f69e0009870 tx=0x7f69e002b980 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cbdd132e0ebb76e6 server_cookie=0 in_seq=0 out_seq=0 2023-08-09T13:55:09.246 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:09.242+0000 7f69e77fe640 1 -- 172.21.15.43:0/2719261110 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f69e0030840 con 0x7f69f00b7df0 2023-08-09T13:55:09.246 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:09.242+0000 7f69e77fe640 1 -- 172.21.15.43:0/2719261110 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f69e00309a0 con 0x7f69f00b7df0 2023-08-09T13:55:09.246 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:09.242+0000 7f69e77fe640 1 -- 172.21.15.43:0/2719261110 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f69e0030d30 con 0x7f69f00b7df0 2023-08-09T13:55:09.246 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:09.243+0000 7f69f5c21640 1 -- 172.21.15.43:0/2719261110 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f69f00b7df0 msgr2=0x7f69f00d1b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:55:09.246 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:09.243+0000 7f69f5c21640 1 --2- 172.21.15.43:0/2719261110 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f69f00b7df0 0x7f69f00d1b30 secure :-1 s=READY pgs=1590 cs=0 l=1 rev1=1 crypto rx=0x7f69e0009870 tx=0x7f69e002b980 comp rx=0 tx=0).stop 2023-08-09T13:55:09.246 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:09.243+0000 7f69f5c21640 1 -- 172.21.15.43:0/2719261110 shutdown_connections 2023-08-09T13:55:09.247 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:09.243+0000 7f69f5c21640 1 --2- 172.21.15.43:0/2719261110 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f69f00cebf0 0x7f69f00cf040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:55:09.247 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:09.243+0000 7f69f5c21640 1 --2- 172.21.15.43:0/2719261110 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f69f00b7df0 0x7f69f00d1b30 unknown :-1 s=CLOSED pgs=1590 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:55:09.247 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:09.243+0000 7f69f5c21640 1 -- 172.21.15.43:0/2719261110 >> 172.21.15.43:0/2719261110 conn(0x7f69f006a230 msgr2=0x7f69f006a640 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-09T13:55:09.247 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:09.243+0000 7f69f5c21640 1 -- 172.21.15.43:0/2719261110 shutdown_connections 2023-08-09T13:55:09.247 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:09.243+0000 7f69f5c21640 1 -- 172.21.15.43:0/2719261110 wait complete. 2023-08-09T13:55:09.248 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:09.245+0000 7f69f5c21640 1 Processor -- start 2023-08-09T13:55:09.248 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:09.245+0000 7f69f5c21640 1 -- start start 2023-08-09T13:55:09.249 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:09.246+0000 7f69f5c21640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f69f00b7df0 0x7f69f00cb550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:55:09.249 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:09.246+0000 7f69f5c21640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f69f00cebf0 0x7f69f00a9130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:55:09.249 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:09.246+0000 7f69f5c21640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f69f015c2d0 0x7f69f015dca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:55:09.249 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:09.246+0000 7f69f5c21640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f69f00cb0c0 con 0x7f69f015c2d0 2023-08-09T13:55:09.249 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:09.246+0000 7f69f5c21640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f69f015f110 con 0x7f69f00cebf0 2023-08-09T13:55:09.249 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:09.246+0000 7f69f5c21640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f69f015f3f0 con 0x7f69f00b7df0 2023-08-09T13:55:09.250 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:09.246+0000 7f69f5420640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f69f015c2d0 0x7f69f015dca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:55:09.250 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:09.246+0000 7f69f5420640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f69f015c2d0 0x7f69f015dca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:56474/0 (socket says 172.21.15.43:56474) 2023-08-09T13:55:09.250 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:09.246+0000 7f69f5420640 1 -- 172.21.15.43:0/1760708651 learned_addr learned my addr 172.21.15.43:0/1760708651 (peer_addr_for_me v2:172.21.15.43:0/0) 2023-08-09T13:55:09.250 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:09.246+0000 7f69e7fff640 1 --2- 172.21.15.43:0/1760708651 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f69f00cebf0 0x7f69f00a9130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:55:09.250 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:09.246+0000 7f69f5420640 1 -- 172.21.15.43:0/1760708651 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f69f00b7df0 msgr2=0x7f69f00cb550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:55:09.250 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:09.246+0000 7f69f5420640 1 --2- 172.21.15.43:0/1760708651 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f69f00b7df0 0x7f69f00cb550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:55:09.250 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:09.246+0000 7f69f5420640 1 -- 172.21.15.43:0/1760708651 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f69f00cebf0 msgr2=0x7f69f00a9130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:55:09.250 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:09.246+0000 7f69f5420640 1 --2- 172.21.15.43:0/1760708651 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f69f00cebf0 0x7f69f00a9130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:55:09.251 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:09.247+0000 7f69f5420640 1 -- 172.21.15.43:0/1760708651 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f69e00094d0 con 0x7f69f015c2d0 2023-08-09T13:55:09.251 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:09.247+0000 7f69f5420640 1 --2- 172.21.15.43:0/1760708651 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f69f015c2d0 0x7f69f015dca0 secure :-1 s=READY pgs=1591 cs=0 l=1 rev1=1 crypto rx=0x7f69e800cef0 tx=0x7f69e800a710 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-09T13:55:09.252 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:09.247+0000 7f69e57fa640 1 -- 172.21.15.43:0/1760708651 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f69e800d840 con 0x7f69f015c2d0 2023-08-09T13:55:09.252 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:09.247+0000 7f69e57fa640 1 -- 172.21.15.43:0/1760708651 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f69e800d9c0 con 0x7f69f015c2d0 2023-08-09T13:55:09.252 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:09.247+0000 7f69e57fa640 1 -- 172.21.15.43:0/1760708651 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f69e800dd70 con 0x7f69f015c2d0 2023-08-09T13:55:09.252 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:09.247+0000 7f69f5c21640 1 -- 172.21.15.43:0/1760708651 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f69f00ca790 con 0x7f69f015c2d0 2023-08-09T13:55:09.252 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:09.248+0000 7f69f5c21640 1 -- 172.21.15.43:0/1760708651 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f69f00cad50 con 0x7f69f015c2d0 2023-08-09T13:55:09.252 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:09.249+0000 7f69f5c21640 1 -- 172.21.15.43:0/1760708651 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f69f004e390 con 0x7f69f015c2d0 2023-08-09T13:55:09.252 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:09.249+0000 7f69e57fa640 1 -- 172.21.15.43:0/1760708651 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 64) v1 ==== 88052+0+0 (secure 0 0 0) 0x7f69e801e3b0 con 0x7f69f015c2d0 2023-08-09T13:55:09.253 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:09.250+0000 7f69e57fa640 1 -- 172.21.15.43:0/1760708651 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(134..134 src has 1..134) v4 ==== 7286+0+0 (secure 0 0 0) 0x7f69e80890e0 con 0x7f69f015c2d0 2023-08-09T13:55:09.257 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:09.255+0000 7f69e57fa640 1 -- 172.21.15.43:0/1760708651 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178066 (secure 0 0 0) 0x7f69e8054720 con 0x7f69f015c2d0 2023-08-09T13:55:09.458 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:09.455+0000 7f69f5c21640 1 -- 172.21.15.43:0/1760708651 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"} v 0) v1 -- 0x7f69f0065770 con 0x7f69f015c2d0 2023-08-09T13:55:09.459 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:09.457+0000 7f69e57fa640 1 -- 172.21.15.43:0/1760708651 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"}]=0 v134) v1 ==== 92+0+10 (secure 0 0 0) 0x7f69e801a0a0 con 0x7f69f015c2d0 2023-08-09T13:55:09.459 INFO:teuthology.orchestra.run.smithi043.stdout:pg_num: 8 2023-08-09T13:55:09.462 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:09.459+0000 7f69f5c21640 1 -- 172.21.15.43:0/1760708651 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f69f015c2d0 msgr2=0x7f69f015dca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:55:09.462 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:09.459+0000 7f69f5c21640 1 --2- 172.21.15.43:0/1760708651 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f69f015c2d0 0x7f69f015dca0 secure :-1 s=READY pgs=1591 cs=0 l=1 rev1=1 crypto rx=0x7f69e800cef0 tx=0x7f69e800a710 comp rx=0 tx=0).stop 2023-08-09T13:55:09.462 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:09.459+0000 7f69f5c21640 1 -- 172.21.15.43:0/1760708651 shutdown_connections 2023-08-09T13:55:09.463 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:09.459+0000 7f69f5c21640 1 --2- 172.21.15.43:0/1760708651 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f69f015c2d0 0x7f69f015dca0 unknown :-1 s=CLOSED pgs=1591 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:55:09.463 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:09.459+0000 7f69f5c21640 1 --2- 172.21.15.43:0/1760708651 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f69f00cebf0 0x7f69f00a9130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:55:09.463 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:09.459+0000 7f69f5c21640 1 --2- 172.21.15.43:0/1760708651 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f69f00b7df0 0x7f69f00cb550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:55:09.463 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:09.459+0000 7f69f5c21640 1 -- 172.21.15.43:0/1760708651 >> 172.21.15.43:0/1760708651 conn(0x7f69f006a230 msgr2=0x7f69f00b5230 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-09T13:55:09.463 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:09.460+0000 7f69f5c21640 1 -- 172.21.15.43:0/1760708651 shutdown_connections 2023-08-09T13:55:09.464 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:09.460+0000 7f69f5c21640 1 -- 172.21.15.43:0/1760708651 wait complete. 2023-08-09T13:55:09.481 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-08-09T13:55:09.719 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:09.716+0000 7fab0f7c9640 1 Processor -- start 2023-08-09T13:55:09.720 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:09.717+0000 7fab0f7c9640 1 -- start start 2023-08-09T13:55:09.721 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:09.718+0000 7fab0f7c9640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7fab080b45f0 0x7fab080cd6d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:55:09.722 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:09.718+0000 7fab0f7c9640 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7fab080d04b0 0x7fab080b59d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:55:09.722 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:09.718+0000 7fab0f7c9640 1 -- --> v1:172.21.15.203:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fab080b7bd0 con 0x7fab0815c550 2023-08-09T13:55:09.722 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:09.718+0000 7fab0f7c9640 1 -- --> v2:172.21.15.43:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fab080d1c00 con 0x7fab080b45f0 2023-08-09T13:55:09.722 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:09.718+0000 7fab0f7c9640 1 -- --> v2:172.21.15.203:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fab0815c930 con 0x7fab080d04b0 2023-08-09T13:55:09.722 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:09.719+0000 7fab0cd64640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7fab080b45f0 0x7fab080cd6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:55:09.723 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:09.719+0000 7fab0cd64640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7fab080b45f0 0x7fab080cd6d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:56490/0 (socket says 172.21.15.43:56490) 2023-08-09T13:55:09.723 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:09.719+0000 7fab0cd64640 1 -- 172.21.15.43:0/2340000087 learned_addr learned my addr 172.21.15.43:0/2340000087 (peer_addr_for_me v2:172.21.15.43:0/0) 2023-08-09T13:55:09.723 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:09.719+0000 7fab0dd66640 1 --2- 172.21.15.43:0/2340000087 >> v2:172.21.15.203:3300/0 conn(0x7fab080d04b0 0x7fab080b59d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:55:09.725 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:09.719+0000 7fab0cd64640 1 -- 172.21.15.43:0/2340000087 >> v2:172.21.15.203:3300/0 conn(0x7fab080d04b0 msgr2=0x7fab080b59d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-09T13:55:09.725 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:09.719+0000 7fab0cd64640 1 --2- 172.21.15.43:0/2340000087 >> v2:172.21.15.203:3300/0 conn(0x7fab080d04b0 0x7fab080b59d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:55:09.725 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:09.719+0000 7fab0cd64640 1 -- 172.21.15.43:0/2340000087 >> v1:172.21.15.203:6789/0 conn(0x7fab0815c550 legacy=0x7fab080b3ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-09T13:55:09.725 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:09.719+0000 7fab0cd64640 1 -- 172.21.15.43:0/2340000087 --> v2:172.21.15.43:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fab080b54f0 con 0x7fab080b45f0 2023-08-09T13:55:09.725 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:09.720+0000 7fab0cd64640 1 --2- 172.21.15.43:0/2340000087 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fab080b45f0 0x7fab080cd6d0 secure :-1 s=READY pgs=1592 cs=0 l=1 rev1=1 crypto rx=0x7fab080ce7f0 tx=0x7faaf802b910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e60a2240ae78a63e server_cookie=0 in_seq=0 out_seq=0 2023-08-09T13:55:09.726 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:09.721+0000 7faafffff640 1 -- 172.21.15.43:0/2340000087 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faaf8031870 con 0x7fab080b45f0 2023-08-09T13:55:09.726 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:09.721+0000 7faafffff640 1 -- 172.21.15.43:0/2340000087 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7faaf80319d0 con 0x7fab080b45f0 2023-08-09T13:55:09.726 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:09.721+0000 7faafffff640 1 -- 172.21.15.43:0/2340000087 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faaf8031d60 con 0x7fab080b45f0 2023-08-09T13:55:09.726 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:09.721+0000 7fab0f7c9640 1 -- 172.21.15.43:0/2340000087 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fab080b45f0 msgr2=0x7fab080cd6d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:55:09.726 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:09.721+0000 7fab0f7c9640 1 --2- 172.21.15.43:0/2340000087 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fab080b45f0 0x7fab080cd6d0 secure :-1 s=READY pgs=1592 cs=0 l=1 rev1=1 crypto rx=0x7fab080ce7f0 tx=0x7faaf802b910 comp rx=0 tx=0).stop 2023-08-09T13:55:09.726 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:09.722+0000 7fab0f7c9640 1 -- 172.21.15.43:0/2340000087 shutdown_connections 2023-08-09T13:55:09.727 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:09.722+0000 7fab0f7c9640 1 --2- 172.21.15.43:0/2340000087 >> v2:172.21.15.203:3300/0 conn(0x7fab080d04b0 0x7fab080b59d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:55:09.727 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:09.722+0000 7fab0f7c9640 1 --2- 172.21.15.43:0/2340000087 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fab080b45f0 0x7fab080cd6d0 unknown :-1 s=CLOSED pgs=1592 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:55:09.727 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:09.722+0000 7fab0f7c9640 1 -- 172.21.15.43:0/2340000087 >> 172.21.15.43:0/2340000087 conn(0x7fab08071520 msgr2=0x7fab080ceee0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-09T13:55:09.727 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:09.722+0000 7fab0f7c9640 1 -- 172.21.15.43:0/2340000087 shutdown_connections 2023-08-09T13:55:09.727 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:09.722+0000 7fab0f7c9640 1 -- 172.21.15.43:0/2340000087 wait complete. 2023-08-09T13:55:09.729 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:09.723+0000 7fab0f7c9640 1 Processor -- start 2023-08-09T13:55:09.729 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:09.724+0000 7fab0f7c9640 1 -- start start 2023-08-09T13:55:09.729 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:09.724+0000 7fab0f7c9640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fab080b45f0 0x7fab080b36c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:55:09.729 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:09.724+0000 7fab0f7c9640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fab080d04b0 0x7fab080cbb20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:55:09.729 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:09.725+0000 7fab0f7c9640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fab0815c550 0x7fab081f51f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:55:09.729 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:09.725+0000 7fab0f7c9640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fab080caf50 con 0x7fab080b45f0 2023-08-09T13:55:09.729 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:09.725+0000 7fab0f7c9640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fab080ca3d0 con 0x7fab080d04b0 2023-08-09T13:55:09.730 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:09.725+0000 7fab0f7c9640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fab080ca6b0 con 0x7fab0815c550 2023-08-09T13:55:09.730 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:09.725+0000 7fab0d565640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fab080b45f0 0x7fab080b36c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:55:09.730 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:09.725+0000 7fab0d565640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fab080b45f0 0x7fab080b36c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:56506/0 (socket says 172.21.15.43:56506) 2023-08-09T13:55:09.730 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:09.725+0000 7fab0d565640 1 -- 172.21.15.43:0/730060586 learned_addr learned my addr 172.21.15.43:0/730060586 (peer_addr_for_me v2:172.21.15.43:0/0) 2023-08-09T13:55:09.730 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:09.725+0000 7fab0dd66640 1 --2- 172.21.15.43:0/730060586 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fab0815c550 0x7fab081f51f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:55:09.730 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:09.725+0000 7fab0cd64640 1 --2- 172.21.15.43:0/730060586 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fab080d04b0 0x7fab080cbb20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:55:09.730 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:09.725+0000 7fab0d565640 1 -- 172.21.15.43:0/730060586 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fab0815c550 msgr2=0x7fab081f51f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:55:09.730 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:09.725+0000 7fab0d565640 1 --2- 172.21.15.43:0/730060586 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fab0815c550 0x7fab081f51f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:55:09.731 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:09.726+0000 7fab0d565640 1 -- 172.21.15.43:0/730060586 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fab080d04b0 msgr2=0x7fab080cbb20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:55:09.731 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:09.726+0000 7fab0d565640 1 --2- 172.21.15.43:0/730060586 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fab080d04b0 0x7fab080cbb20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:55:09.731 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:09.726+0000 7fab0d565640 1 -- 172.21.15.43:0/730060586 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faaf80094d0 con 0x7fab080b45f0 2023-08-09T13:55:09.731 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:09.726+0000 7fab0d565640 1 --2- 172.21.15.43:0/730060586 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fab080b45f0 0x7fab080b36c0 secure :-1 s=READY pgs=1593 cs=0 l=1 rev1=1 crypto rx=0x7fab0400b3e0 tx=0x7fab0400b8a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-09T13:55:09.731 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:09.726+0000 7faafdffb640 1 -- 172.21.15.43:0/730060586 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fab04011830 con 0x7fab080b45f0 2023-08-09T13:55:09.731 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:09.726+0000 7faafdffb640 1 -- 172.21.15.43:0/730060586 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fab040119b0 con 0x7fab080b45f0 2023-08-09T13:55:09.731 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:09.726+0000 7faafdffb640 1 -- 172.21.15.43:0/730060586 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fab04011d60 con 0x7fab080b45f0 2023-08-09T13:55:09.731 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:09.726+0000 7fab0f7c9640 1 -- 172.21.15.43:0/730060586 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fab080ca990 con 0x7fab080b45f0 2023-08-09T13:55:09.732 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:09.726+0000 7fab0f7c9640 1 -- 172.21.15.43:0/730060586 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fab080b3c00 con 0x7fab080b45f0 2023-08-09T13:55:09.733 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:09.727+0000 7fab0f7c9640 1 -- 172.21.15.43:0/730060586 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fab080739f0 con 0x7fab080b45f0 2023-08-09T13:55:09.733 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:09.728+0000 7faafdffb640 1 -- 172.21.15.43:0/730060586 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 65) v1 ==== 88179+0+0 (secure 0 0 0) 0x7fab0401a400 con 0x7fab080b45f0 2023-08-09T13:55:09.733 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:09.729+0000 7faafdffb640 1 --2- 172.21.15.43:0/730060586 >> [v2:172.21.15.203:6832/3490266242,v1:172.21.15.203:6833/3490266242] conn(0x7faaec069ea0 0x7faaec06c360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:55:09.733 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:09.729+0000 7faafdffb640 1 -- 172.21.15.43:0/730060586 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(134..134 src has 1..134) v4 ==== 7286+0+0 (secure 0 0 0) 0x7fab040888e0 con 0x7fab080b45f0 2023-08-09T13:55:09.733 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:09.730+0000 7fab0cd64640 1 --2- 172.21.15.43:0/730060586 >> [v2:172.21.15.203:6832/3490266242,v1:172.21.15.203:6833/3490266242] conn(0x7faaec069ea0 0x7faaec06c360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:55:09.734 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:09.731+0000 7fab0cd64640 1 --2- 172.21.15.43:0/730060586 >> [v2:172.21.15.203:6832/3490266242,v1:172.21.15.203:6833/3490266242] conn(0x7faaec069ea0 0x7faaec06c360 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7faaf8036000 tx=0x7faaf803c040 comp rx=0 tx=0).ready entity=mgr.6659 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-09T13:55:09.736 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:09.734+0000 7faafdffb640 1 -- 172.21.15.43:0/730060586 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178066 (secure 0 0 0) 0x7fab04053d70 con 0x7fab080b45f0 2023-08-09T13:55:09.929 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:09.926+0000 7fab0f7c9640 1 -- 172.21.15.43:0/730060586 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7fab0815fa80 con 0x7fab080b45f0 2023-08-09T13:55:09.930 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:09.927+0000 7faafdffb640 1 -- 172.21.15.43:0/730060586 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v134) v1 ==== 93+0+10 (secure 0 0 0) 0x7fab04057390 con 0x7fab080b45f0 2023-08-09T13:55:09.930 INFO:teuthology.orchestra.run.smithi043.stdout:pg_num: 1 2023-08-09T13:55:09.932 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:09.930+0000 7fab0f7c9640 1 -- 172.21.15.43:0/730060586 >> [v2:172.21.15.203:6832/3490266242,v1:172.21.15.203:6833/3490266242] conn(0x7faaec069ea0 msgr2=0x7faaec06c360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:55:09.933 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:09.930+0000 7fab0f7c9640 1 --2- 172.21.15.43:0/730060586 >> [v2:172.21.15.203:6832/3490266242,v1:172.21.15.203:6833/3490266242] conn(0x7faaec069ea0 0x7faaec06c360 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7faaf8036000 tx=0x7faaf803c040 comp rx=0 tx=0).stop 2023-08-09T13:55:09.933 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:09.930+0000 7fab0f7c9640 1 -- 172.21.15.43:0/730060586 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fab080b45f0 msgr2=0x7fab080b36c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:55:09.933 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:09.930+0000 7fab0f7c9640 1 --2- 172.21.15.43:0/730060586 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fab080b45f0 0x7fab080b36c0 secure :-1 s=READY pgs=1593 cs=0 l=1 rev1=1 crypto rx=0x7fab0400b3e0 tx=0x7fab0400b8a0 comp rx=0 tx=0).stop 2023-08-09T13:55:09.933 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:09.930+0000 7fab0f7c9640 1 -- 172.21.15.43:0/730060586 shutdown_connections 2023-08-09T13:55:09.933 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:09.930+0000 7fab0f7c9640 1 --2- 172.21.15.43:0/730060586 >> [v2:172.21.15.203:6832/3490266242,v1:172.21.15.203:6833/3490266242] conn(0x7faaec069ea0 0x7faaec06c360 unknown :-1 s=CLOSED pgs=20 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:55:09.933 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:09.930+0000 7fab0f7c9640 1 --2- 172.21.15.43:0/730060586 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fab0815c550 0x7fab081f51f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:55:09.933 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:09.930+0000 7fab0f7c9640 1 --2- 172.21.15.43:0/730060586 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fab080d04b0 0x7fab080cbb20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:55:09.934 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:09.930+0000 7fab0f7c9640 1 --2- 172.21.15.43:0/730060586 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fab080b45f0 0x7fab080b36c0 unknown :-1 s=CLOSED pgs=1593 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:55:09.934 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:09.930+0000 7fab0f7c9640 1 -- 172.21.15.43:0/730060586 >> 172.21.15.43:0/730060586 conn(0x7fab08071520 msgr2=0x7fab0815cb30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-09T13:55:09.934 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:09.930+0000 7fab0f7c9640 1 -- 172.21.15.43:0/730060586 shutdown_connections 2023-08-09T13:55:09.934 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:09.931+0000 7fab0f7c9640 1 -- 172.21.15.43:0/730060586 wait complete. 2023-08-09T13:55:09.951 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-08-09T13:55:10.181 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:10.178+0000 7f9d07124640 1 Processor -- start 2023-08-09T13:55:10.181 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:10.178+0000 7f9d07124640 1 -- start start 2023-08-09T13:55:10.182 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:10.180+0000 7f9d07124640 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7f9d000b3fa0 0x7f9d000b43b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:55:10.182 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:10.180+0000 7f9d07124640 1 -- --> v1:172.21.15.43:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9d000b48f0 con 0x7f9d000d0a70 2023-08-09T13:55:10.182 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:10.180+0000 7f9d07124640 1 -- --> v1:172.21.15.203:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9d000cdf00 con 0x7f9d0015c680 2023-08-09T13:55:10.183 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:10.180+0000 7f9d07124640 1 -- --> v2:172.21.15.203:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9d000d0f70 con 0x7f9d000b3fa0 2023-08-09T13:55:10.183 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:10.180+0000 7f9cfffff640 1 --1- >> v1:172.21.15.43:6789/0 conn(0x7f9d000d0a70 0x7f9d000cd7f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.43:6789/0 says I am v1:172.21.15.43:42194/0 (socket says 172.21.15.43:42194) 2023-08-09T13:55:10.183 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:10.180+0000 7f9cfffff640 1 -- 172.21.15.43:0/2316477048 learned_addr learned my addr 172.21.15.43:0/2316477048 (peer_addr_for_me v1:172.21.15.43:0/0) 2023-08-09T13:55:10.184 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:10.180+0000 7f9d056c1640 1 --2- 172.21.15.43:0/2316477048 >> v2:172.21.15.203:3300/0 conn(0x7f9d000b3fa0 0x7f9d000b43b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:55:10.184 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:10.180+0000 7f9cff7fe640 1 -- 172.21.15.43:0/2316477048 <== mon.0 v1:172.21.15.43:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4266101410 0 0) 0x7f9d000b48f0 con 0x7f9d000d0a70 2023-08-09T13:55:10.185 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:10.180+0000 7f9cff7fe640 1 -- 172.21.15.43:0/2316477048 --> v1:172.21.15.43:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f9cf0003520 con 0x7f9d000d0a70 2023-08-09T13:55:10.185 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:10.181+0000 7f9cff7fe640 1 -- 172.21.15.43:0/2316477048 <== mon.0 v1:172.21.15.43:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1709860381 0 0) 0x7f9cf0003520 con 0x7f9d000d0a70 2023-08-09T13:55:10.185 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:10.181+0000 7f9cff7fe640 1 -- 172.21.15.43:0/2316477048 >> v2:172.21.15.203:3300/0 conn(0x7f9d000b3fa0 msgr2=0x7f9d000b43b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-09T13:55:10.185 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:10.181+0000 7f9cff7fe640 1 --2- 172.21.15.43:0/2316477048 >> v2:172.21.15.203:3300/0 conn(0x7f9d000b3fa0 0x7f9d000b43b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:55:10.185 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:10.181+0000 7f9cff7fe640 1 -- 172.21.15.43:0/2316477048 >> v1:172.21.15.203:6789/0 conn(0x7f9d0015c680 legacy=0x7f9d000ce760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-09T13:55:10.185 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:10.181+0000 7f9cff7fe640 1 -- 172.21.15.43:0/2316477048 --> v1:172.21.15.43:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9d000ccd10 con 0x7f9d000d0a70 2023-08-09T13:55:10.185 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:10.181+0000 7f9cff7fe640 1 -- 172.21.15.43:0/2316477048 <== mon.0 v1:172.21.15.43:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1285150740 0 0) 0x7f9cec001670 con 0x7f9d000d0a70 2023-08-09T13:55:10.186 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:10.181+0000 7f9cff7fe640 1 -- 172.21.15.43:0/2316477048 >> v1:172.21.15.43:6789/0 conn(0x7f9d000d0a70 legacy=0x7f9d000cd7f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:55:10.186 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:10.181+0000 7f9cff7fe640 1 --2- 172.21.15.43:0/2316477048 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f9cf00039e0 0x7f9cf0003de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:55:10.186 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:10.181+0000 7f9cff7fe640 1 --2- 172.21.15.43:0/2316477048 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f9cf00047c0 0x7f9cf0004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:55:10.186 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:10.181+0000 7f9cff7fe640 1 --2- 172.21.15.43:0/2316477048 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f9cf0005180 0x7f9cf000b980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:55:10.186 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:10.181+0000 7f9cff7fe640 1 -- 172.21.15.43:0/2316477048 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9d000d0f70 con 0x7f9cf0005180 2023-08-09T13:55:10.186 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:10.181+0000 7f9cff7fe640 1 -- 172.21.15.43:0/2316477048 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9cf000c080 con 0x7f9cf00039e0 2023-08-09T13:55:10.186 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:10.181+0000 7f9cff7fe640 1 -- 172.21.15.43:0/2316477048 --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9cf000c240 con 0x7f9cf00047c0 2023-08-09T13:55:10.186 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:10.181+0000 7f9d056c1640 1 --2- 172.21.15.43:0/2316477048 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f9cf0005180 0x7f9cf000b980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:55:10.187 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:10.182+0000 7f9d04ec0640 1 --2- 172.21.15.43:0/2316477048 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f9cf00039e0 0x7f9cf0003de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:55:10.187 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:10.182+0000 7f9cfffff640 1 --2- 172.21.15.43:0/2316477048 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f9cf00047c0 0x7f9cf0004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:55:10.187 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:10.182+0000 7f9d056c1640 1 -- 172.21.15.43:0/2316477048 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f9cf00047c0 msgr2=0x7f9cf0004c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-09T13:55:10.187 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:10.182+0000 7f9d056c1640 1 --2- 172.21.15.43:0/2316477048 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f9cf00047c0 0x7f9cf0004c40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:55:10.187 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:10.182+0000 7f9d056c1640 1 -- 172.21.15.43:0/2316477048 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f9cf00039e0 msgr2=0x7f9cf0003de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-09T13:55:10.187 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:10.182+0000 7f9d056c1640 1 --2- 172.21.15.43:0/2316477048 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f9cf00039e0 0x7f9cf0003de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:55:10.187 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:10.182+0000 7f9d056c1640 1 -- 172.21.15.43:0/2316477048 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9cf0003520 con 0x7f9cf0005180 2023-08-09T13:55:10.188 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:10.183+0000 7f9d056c1640 1 -- 172.21.15.43:0/2316477048 reap_dead start 2023-08-09T13:55:10.188 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:10.184+0000 7f9d056c1640 1 --2- 172.21.15.43:0/2316477048 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f9cf0005180 0x7f9cf000b980 secure :-1 s=READY pgs=1595 cs=0 l=1 rev1=1 crypto rx=0x7f9cf8009fc0 tx=0x7f9cf00057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ae5444700363d862 server_cookie=0 in_seq=0 out_seq=0 2023-08-09T13:55:10.189 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:10.184+0000 7f9cff7fe640 1 -- 172.21.15.43:0/2316477048 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9cf802cc60 con 0x7f9cf0005180 2023-08-09T13:55:10.189 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:10.184+0000 7f9cff7fe640 1 -- 172.21.15.43:0/2316477048 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9cf800f040 con 0x7f9cf0005180 2023-08-09T13:55:10.189 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:10.184+0000 7f9cff7fe640 1 -- 172.21.15.43:0/2316477048 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9cf8003990 con 0x7f9cf0005180 2023-08-09T13:55:10.189 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:10.185+0000 7f9d07124640 1 -- 172.21.15.43:0/2316477048 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f9cf0005180 msgr2=0x7f9cf000b980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:55:10.189 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:10.185+0000 7f9d07124640 1 --2- 172.21.15.43:0/2316477048 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f9cf0005180 0x7f9cf000b980 secure :-1 s=READY pgs=1595 cs=0 l=1 rev1=1 crypto rx=0x7f9cf8009fc0 tx=0x7f9cf00057f0 comp rx=0 tx=0).stop 2023-08-09T13:55:10.189 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:10.185+0000 7f9d07124640 1 -- 172.21.15.43:0/2316477048 shutdown_connections 2023-08-09T13:55:10.189 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:10.185+0000 7f9d07124640 1 --2- 172.21.15.43:0/2316477048 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f9cf0005180 0x7f9cf000b980 unknown :-1 s=CLOSED pgs=1595 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:55:10.190 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:10.185+0000 7f9d07124640 1 -- 172.21.15.43:0/2316477048 >> 172.21.15.43:0/2316477048 conn(0x7f9d00071520 msgr2=0x7f9d000d2520 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-09T13:55:10.190 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:10.185+0000 7f9d07124640 1 -- 172.21.15.43:0/2316477048 shutdown_connections 2023-08-09T13:55:10.190 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:10.185+0000 7f9d07124640 1 -- 172.21.15.43:0/2316477048 wait complete. 2023-08-09T13:55:10.190 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:10.187+0000 7f9d07124640 1 Processor -- start 2023-08-09T13:55:10.191 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:10.187+0000 7f9d07124640 1 -- start start 2023-08-09T13:55:10.191 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:10.188+0000 7f9d07124640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f9cf0005180 0x7f9d000cc000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:55:10.191 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:10.188+0000 7f9d07124640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f9d000cb200 0x7f9d00150a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:55:10.191 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:10.188+0000 7f9d07124640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f9d0015ca60 0x7f9d0015cee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:55:10.191 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:10.188+0000 7f9d07124640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9d00150f70 con 0x7f9d0015ca60 2023-08-09T13:55:10.191 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:10.188+0000 7f9d07124640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9d001544f0 con 0x7f9cf0005180 2023-08-09T13:55:10.191 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:10.188+0000 7f9d07124640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9d001547d0 con 0x7f9d000cb200 2023-08-09T13:55:10.193 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:10.189+0000 7f9d056c1640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f9d0015ca60 0x7f9d0015cee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:55:10.193 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:10.189+0000 7f9d056c1640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f9d0015ca60 0x7f9d0015cee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:56536/0 (socket says 172.21.15.43:56536) 2023-08-09T13:55:10.193 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:10.189+0000 7f9d056c1640 1 -- 172.21.15.43:0/3506765510 learned_addr learned my addr 172.21.15.43:0/3506765510 (peer_addr_for_me v2:172.21.15.43:0/0) 2023-08-09T13:55:10.193 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:10.189+0000 7f9d04ec0640 1 --2- 172.21.15.43:0/3506765510 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f9cf0005180 0x7f9d000cc000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:55:10.193 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:10.189+0000 7f9cfffff640 1 --2- 172.21.15.43:0/3506765510 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f9d000cb200 0x7f9d00150a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:55:10.193 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:10.189+0000 7f9d056c1640 1 -- 172.21.15.43:0/3506765510 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f9d000cb200 msgr2=0x7f9d00150a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:55:10.193 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:10.189+0000 7f9d056c1640 1 --2- 172.21.15.43:0/3506765510 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f9d000cb200 0x7f9d00150a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:55:10.194 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:10.189+0000 7f9d056c1640 1 -- 172.21.15.43:0/3506765510 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f9cf0005180 msgr2=0x7f9d000cc000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:55:10.194 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:10.189+0000 7f9d056c1640 1 --2- 172.21.15.43:0/3506765510 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f9cf0005180 0x7f9d000cc000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:55:10.194 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:10.189+0000 7f9d056c1640 1 -- 172.21.15.43:0/3506765510 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9d00154ab0 con 0x7f9d0015ca60 2023-08-09T13:55:10.194 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:10.189+0000 7f9cfffff640 1 --2- 172.21.15.43:0/3506765510 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f9d000cb200 0x7f9d00150a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-08-09T13:55:10.194 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:10.189+0000 7f9d04ec0640 1 --2- 172.21.15.43:0/3506765510 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f9cf0005180 0x7f9d000cc000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-08-09T13:55:10.194 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:10.189+0000 7f9d056c1640 1 --2- 172.21.15.43:0/3506765510 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f9d0015ca60 0x7f9d0015cee0 secure :-1 s=READY pgs=1596 cs=0 l=1 rev1=1 crypto rx=0x7f9cf802c6d0 tx=0x7f9cf8009750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-09T13:55:10.194 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:10.190+0000 7f9cfd7fa640 1 -- 172.21.15.43:0/3506765510 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9cf8035e40 con 0x7f9d0015ca60 2023-08-09T13:55:10.195 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:10.190+0000 7f9cfd7fa640 1 -- 172.21.15.43:0/3506765510 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9cf8003990 con 0x7f9d0015ca60 2023-08-09T13:55:10.195 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:10.190+0000 7f9cfd7fa640 1 -- 172.21.15.43:0/3506765510 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9cf8003d80 con 0x7f9d0015ca60 2023-08-09T13:55:10.195 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:10.190+0000 7f9d07124640 1 -- 172.21.15.43:0/3506765510 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9d00154d90 con 0x7f9d0015ca60 2023-08-09T13:55:10.195 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:10.190+0000 7f9d07124640 1 -- 172.21.15.43:0/3506765510 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9d000cf470 con 0x7f9d0015ca60 2023-08-09T13:55:10.195 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:10.192+0000 7f9d07124640 1 -- 172.21.15.43:0/3506765510 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9d000739f0 con 0x7f9d0015ca60 2023-08-09T13:55:10.196 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:10.192+0000 7f9cfd7fa640 1 -- 172.21.15.43:0/3506765510 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 65) v1 ==== 88179+0+0 (secure 0 0 0) 0x7f9cf8004030 con 0x7f9d0015ca60 2023-08-09T13:55:10.196 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:10.193+0000 7f9cfd7fa640 1 --2- 172.21.15.43:0/3506765510 >> [v2:172.21.15.203:6832/3490266242,v1:172.21.15.203:6833/3490266242] conn(0x7f9ce4069ea0 0x7f9ce406c360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:55:10.196 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:10.193+0000 7f9cfd7fa640 1 -- 172.21.15.43:0/3506765510 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(134..134 src has 1..134) v4 ==== 7286+0+0 (secure 0 0 0) 0x7f9cf80ab4c0 con 0x7f9d0015ca60 2023-08-09T13:55:10.196 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:10.194+0000 7f9d04ec0640 1 --2- 172.21.15.43:0/3506765510 >> [v2:172.21.15.203:6832/3490266242,v1:172.21.15.203:6833/3490266242] conn(0x7f9ce4069ea0 0x7f9ce406c360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:55:10.197 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:10.195+0000 7f9d04ec0640 1 --2- 172.21.15.43:0/3506765510 >> [v2:172.21.15.203:6832/3490266242,v1:172.21.15.203:6833/3490266242] conn(0x7f9ce4069ea0 0x7f9ce406c360 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7f9cf4004b60 tx=0x7f9cf4009040 comp rx=0 tx=0).ready entity=mgr.6659 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-09T13:55:10.199 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:10.197+0000 7f9cfd7fa640 1 -- 172.21.15.43:0/3506765510 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178066 (secure 0 0 0) 0x7f9cf8076b40 con 0x7f9d0015ca60 2023-08-09T13:55:10.401 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:10.398+0000 7f9d07124640 1 -- 172.21.15.43:0/3506765510 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f9d0015fe60 con 0x7f9d0015ca60 2023-08-09T13:55:10.401 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:10.399+0000 7f9cfd7fa640 1 -- 172.21.15.43:0/3506765510 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v134) v1 ==== 104+0+11 (secure 0 0 0) 0x7f9cf807a160 con 0x7f9d0015ca60 2023-08-09T13:55:10.402 INFO:teuthology.orchestra.run.smithi043.stdout:pg_num: 32 2023-08-09T13:55:10.404 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:10.401+0000 7f9d07124640 1 -- 172.21.15.43:0/3506765510 >> [v2:172.21.15.203:6832/3490266242,v1:172.21.15.203:6833/3490266242] conn(0x7f9ce4069ea0 msgr2=0x7f9ce406c360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:55:10.404 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:10.401+0000 7f9d07124640 1 --2- 172.21.15.43:0/3506765510 >> [v2:172.21.15.203:6832/3490266242,v1:172.21.15.203:6833/3490266242] conn(0x7f9ce4069ea0 0x7f9ce406c360 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7f9cf4004b60 tx=0x7f9cf4009040 comp rx=0 tx=0).stop 2023-08-09T13:55:10.404 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:10.401+0000 7f9d07124640 1 -- 172.21.15.43:0/3506765510 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f9d0015ca60 msgr2=0x7f9d0015cee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:55:10.405 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:10.401+0000 7f9d07124640 1 --2- 172.21.15.43:0/3506765510 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f9d0015ca60 0x7f9d0015cee0 secure :-1 s=READY pgs=1596 cs=0 l=1 rev1=1 crypto rx=0x7f9cf802c6d0 tx=0x7f9cf8009750 comp rx=0 tx=0).stop 2023-08-09T13:55:10.405 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:10.402+0000 7f9d07124640 1 -- 172.21.15.43:0/3506765510 shutdown_connections 2023-08-09T13:55:10.405 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:10.402+0000 7f9d07124640 1 --2- 172.21.15.43:0/3506765510 >> [v2:172.21.15.203:6832/3490266242,v1:172.21.15.203:6833/3490266242] conn(0x7f9ce4069ea0 0x7f9ce406c360 unknown :-1 s=CLOSED pgs=22 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:55:10.405 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:10.402+0000 7f9d07124640 1 --2- 172.21.15.43:0/3506765510 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f9d0015ca60 0x7f9d0015cee0 unknown :-1 s=CLOSED pgs=1596 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:55:10.405 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:10.402+0000 7f9d07124640 1 --2- 172.21.15.43:0/3506765510 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f9d000cb200 0x7f9d00150a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:55:10.405 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:10.402+0000 7f9d07124640 1 --2- 172.21.15.43:0/3506765510 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f9cf0005180 0x7f9d000cc000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:55:10.405 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:10.402+0000 7f9d07124640 1 -- 172.21.15.43:0/3506765510 >> 172.21.15.43:0/3506765510 conn(0x7f9d00071520 msgr2=0x7f9d000d2500 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-09T13:55:10.406 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:10.402+0000 7f9d07124640 1 -- 172.21.15.43:0/3506765510 shutdown_connections 2023-08-09T13:55:10.406 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:10.402+0000 7f9d07124640 1 -- 172.21.15.43:0/3506765510 wait complete. 2023-08-09T13:55:10.425 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-08-09T13:55:10.663 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:10.660+0000 7fa142a5d640 1 Processor -- start 2023-08-09T13:55:10.663 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:10.661+0000 7fa142a5d640 1 -- start start 2023-08-09T13:55:10.665 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:10.662+0000 7fa142a5d640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7fa13c0c9ee0 0x7fa13c15a800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:55:10.665 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:10.662+0000 7fa142a5d640 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7fa13c0c90a0 0x7fa13c15ad40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:55:10.665 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:10.662+0000 7fa142a5d640 1 -- --> v1:172.21.15.43:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa13c158280 con 0x7fa13c157ea0 2023-08-09T13:55:10.665 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:10.662+0000 7fa142a5d640 1 -- --> v2:172.21.15.43:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa13c15b840 con 0x7fa13c0c9ee0 2023-08-09T13:55:10.665 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:10.662+0000 7fa142a5d640 1 -- --> v2:172.21.15.203:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa13c0b3810 con 0x7fa13c0c90a0 2023-08-09T13:55:10.666 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:10.663+0000 7fa141a5b640 1 --1- >> v1:172.21.15.43:6789/0 conn(0x7fa13c157ea0 0x7fa13c0c97d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.43:6789/0 says I am v1:172.21.15.43:42200/0 (socket says 172.21.15.43:42200) 2023-08-09T13:55:10.666 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:10.663+0000 7fa141a5b640 1 -- 172.21.15.43:0/2878075119 learned_addr learned my addr 172.21.15.43:0/2878075119 (peer_addr_for_me v1:172.21.15.43:0/0) 2023-08-09T13:55:10.666 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:10.663+0000 7fa14225c640 1 --2- 172.21.15.43:0/2878075119 >> v2:172.21.15.203:3300/0 conn(0x7fa13c0c90a0 0x7fa13c15ad40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:55:10.666 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:10.663+0000 7fa14125a640 1 --2- 172.21.15.43:0/2878075119 >> v2:172.21.15.43:3300/0 conn(0x7fa13c0c9ee0 0x7fa13c15a800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:55:10.667 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:10.663+0000 7fa140a59640 1 -- 172.21.15.43:0/2878075119 <== mon.0 v1:172.21.15.43:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3057856633 0 0) 0x7fa138001110 con 0x7fa13c157ea0 2023-08-09T13:55:10.667 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:10.663+0000 7fa140a59640 1 -- 172.21.15.43:0/2878075119 --> v1:172.21.15.43:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fa128003520 con 0x7fa13c157ea0 2023-08-09T13:55:10.667 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:10.663+0000 7fa14125a640 1 -- 172.21.15.43:0/2878075119 >> v2:172.21.15.203:3300/0 conn(0x7fa13c0c90a0 msgr2=0x7fa13c15ad40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-09T13:55:10.667 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:10.663+0000 7fa14125a640 1 --2- 172.21.15.43:0/2878075119 >> v2:172.21.15.203:3300/0 conn(0x7fa13c0c90a0 0x7fa13c15ad40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:55:10.667 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:10.663+0000 7fa14125a640 1 -- 172.21.15.43:0/2878075119 >> v1:172.21.15.43:6789/0 conn(0x7fa13c157ea0 legacy=0x7fa13c0c97d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:55:10.667 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:10.663+0000 7fa14125a640 1 -- 172.21.15.43:0/2878075119 --> v2:172.21.15.43:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa13c15b560 con 0x7fa13c0c9ee0 2023-08-09T13:55:10.668 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:10.665+0000 7fa14125a640 1 --2- 172.21.15.43:0/2878075119 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa13c0c9ee0 0x7fa13c15a800 secure :-1 s=READY pgs=1600 cs=0 l=1 rev1=1 crypto rx=0x7fa13c070190 tx=0x7fa12c02b840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=687635152f56ca77 server_cookie=0 in_seq=0 out_seq=0 2023-08-09T13:55:10.668 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:10.665+0000 7fa140a59640 1 -- 172.21.15.43:0/2878075119 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa12c02e070 con 0x7fa13c0c9ee0 2023-08-09T13:55:10.668 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:10.665+0000 7fa140a59640 1 -- 172.21.15.43:0/2878075119 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa12c0315a0 con 0x7fa13c0c9ee0 2023-08-09T13:55:10.668 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:10.665+0000 7fa140a59640 1 -- 172.21.15.43:0/2878075119 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa12c031930 con 0x7fa13c0c9ee0 2023-08-09T13:55:10.669 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:10.666+0000 7fa142a5d640 1 -- 172.21.15.43:0/2878075119 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa13c0c9ee0 msgr2=0x7fa13c15a800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:55:10.669 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:10.666+0000 7fa142a5d640 1 --2- 172.21.15.43:0/2878075119 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa13c0c9ee0 0x7fa13c15a800 secure :-1 s=READY pgs=1600 cs=0 l=1 rev1=1 crypto rx=0x7fa13c070190 tx=0x7fa12c02b840 comp rx=0 tx=0).stop 2023-08-09T13:55:10.669 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:10.666+0000 7fa142a5d640 1 -- 172.21.15.43:0/2878075119 shutdown_connections 2023-08-09T13:55:10.669 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:10.666+0000 7fa142a5d640 1 --2- 172.21.15.43:0/2878075119 >> v2:172.21.15.203:3300/0 conn(0x7fa13c0c90a0 0x7fa13c15ad40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:55:10.670 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:10.666+0000 7fa142a5d640 1 --2- 172.21.15.43:0/2878075119 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa13c0c9ee0 0x7fa13c15a800 unknown :-1 s=CLOSED pgs=1600 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:55:10.670 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:10.666+0000 7fa142a5d640 1 -- 172.21.15.43:0/2878075119 >> 172.21.15.43:0/2878075119 conn(0x7fa13c0cddd0 msgr2=0x7fa13c0ce1e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-09T13:55:10.670 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:10.666+0000 7fa142a5d640 1 -- 172.21.15.43:0/2878075119 shutdown_connections 2023-08-09T13:55:10.670 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:10.666+0000 7fa142a5d640 1 -- 172.21.15.43:0/2878075119 wait complete. 2023-08-09T13:55:10.670 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:10.668+0000 7fa142a5d640 1 Processor -- start 2023-08-09T13:55:10.671 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:10.668+0000 7fa142a5d640 1 -- start start 2023-08-09T13:55:10.672 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:10.669+0000 7fa142a5d640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fa13c0c90a0 0x7fa13c0c65c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:55:10.672 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:10.669+0000 7fa142a5d640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa13c0c9ee0 0x7fa13c0c3110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:55:10.672 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:10.669+0000 7fa142a5d640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fa13c157ea0 0x7fa13c157350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:55:10.672 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:10.669+0000 7fa142a5d640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa13c0c4500 con 0x7fa13c0c9ee0 2023-08-09T13:55:10.672 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:10.669+0000 7fa142a5d640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa13c0c3650 con 0x7fa13c0c90a0 2023-08-09T13:55:10.672 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:10.669+0000 7fa142a5d640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa13c0c38f0 con 0x7fa13c157ea0 2023-08-09T13:55:10.673 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:10.669+0000 7fa14125a640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa13c0c9ee0 0x7fa13c0c3110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:55:10.673 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:10.670+0000 7fa14125a640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa13c0c9ee0 0x7fa13c0c3110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:56546/0 (socket says 172.21.15.43:56546) 2023-08-09T13:55:10.673 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:10.670+0000 7fa14125a640 1 -- 172.21.15.43:0/605257647 learned_addr learned my addr 172.21.15.43:0/605257647 (peer_addr_for_me v2:172.21.15.43:0/0) 2023-08-09T13:55:10.674 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:10.670+0000 7fa141a5b640 1 --2- 172.21.15.43:0/605257647 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fa13c0c90a0 0x7fa13c0c65c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:55:10.674 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:10.670+0000 7fa14225c640 1 --2- 172.21.15.43:0/605257647 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fa13c157ea0 0x7fa13c157350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:55:10.674 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:10.670+0000 7fa14125a640 1 -- 172.21.15.43:0/605257647 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fa13c157ea0 msgr2=0x7fa13c157350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:55:10.674 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:10.670+0000 7fa14125a640 1 --2- 172.21.15.43:0/605257647 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fa13c157ea0 0x7fa13c157350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:55:10.674 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:10.670+0000 7fa14125a640 1 -- 172.21.15.43:0/605257647 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fa13c0c90a0 msgr2=0x7fa13c0c65c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:55:10.675 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:10.670+0000 7fa14125a640 1 --2- 172.21.15.43:0/605257647 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fa13c0c90a0 0x7fa13c0c65c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:55:10.675 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:10.670+0000 7fa14125a640 1 -- 172.21.15.43:0/605257647 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa12c009400 con 0x7fa13c0c9ee0 2023-08-09T13:55:10.675 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:10.670+0000 7fa14225c640 1 --2- 172.21.15.43:0/605257647 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fa13c157ea0 0x7fa13c157350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-08-09T13:55:10.675 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:10.670+0000 7fa14125a640 1 --2- 172.21.15.43:0/605257647 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa13c0c9ee0 0x7fa13c0c3110 secure :-1 s=READY pgs=1601 cs=0 l=1 rev1=1 crypto rx=0x7fa12c034040 tx=0x7fa12c0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-09T13:55:10.675 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:10.671+0000 7fa1327fc640 1 -- 172.21.15.43:0/605257647 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa12c032630 con 0x7fa13c0c9ee0 2023-08-09T13:55:10.675 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:10.671+0000 7fa1327fc640 1 -- 172.21.15.43:0/605257647 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa12c032790 con 0x7fa13c0c9ee0 2023-08-09T13:55:10.675 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:10.671+0000 7fa1327fc640 1 -- 172.21.15.43:0/605257647 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa12c032b40 con 0x7fa13c0c9ee0 2023-08-09T13:55:10.675 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:10.671+0000 7fa142a5d640 1 -- 172.21.15.43:0/605257647 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa13c0c3bd0 con 0x7fa13c0c9ee0 2023-08-09T13:55:10.676 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:10.671+0000 7fa142a5d640 1 -- 172.21.15.43:0/605257647 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa13c0c4190 con 0x7fa13c0c9ee0 2023-08-09T13:55:10.676 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:10.672+0000 7fa142a5d640 1 -- 172.21.15.43:0/605257647 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa13c0c46b0 con 0x7fa13c0c9ee0 2023-08-09T13:55:10.676 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:10.674+0000 7fa1327fc640 1 -- 172.21.15.43:0/605257647 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 65) v1 ==== 88179+0+0 (secure 0 0 0) 0x7fa12c04e430 con 0x7fa13c0c9ee0 2023-08-09T13:55:10.677 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:10.675+0000 7fa1327fc640 1 --2- 172.21.15.43:0/605257647 >> [v2:172.21.15.203:6832/3490266242,v1:172.21.15.203:6833/3490266242] conn(0x7fa120069ea0 0x7fa12006c360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:55:10.677 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:10.675+0000 7fa1327fc640 1 -- 172.21.15.43:0/605257647 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(134..134 src has 1..134) v4 ==== 7286+0+0 (secure 0 0 0) 0x7fa12c0aa680 con 0x7fa13c0c9ee0 2023-08-09T13:55:10.677 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:10.675+0000 7fa141a5b640 1 --2- 172.21.15.43:0/605257647 >> [v2:172.21.15.203:6832/3490266242,v1:172.21.15.203:6833/3490266242] conn(0x7fa120069ea0 0x7fa12006c360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:55:10.678 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:10.676+0000 7fa141a5b640 1 --2- 172.21.15.43:0/605257647 >> [v2:172.21.15.203:6832/3490266242,v1:172.21.15.203:6833/3490266242] conn(0x7fa120069ea0 0x7fa12006c360 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7fa138001270 tx=0x7fa138007bb0 comp rx=0 tx=0).ready entity=mgr.6659 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-09T13:55:10.682 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:10.679+0000 7fa1327fc640 1 -- 172.21.15.43:0/605257647 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178066 (secure 0 0 0) 0x7fa13c0c46b0 con 0x7fa13c0c9ee0 2023-08-09T13:55:10.877 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:10.874+0000 7fa142a5d640 1 -- 172.21.15.43:0/605257647 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7fa13c04e390 con 0x7fa13c0c9ee0 2023-08-09T13:55:10.877 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:10.875+0000 7fa1327fc640 1 -- 172.21.15.43:0/605257647 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v134) v1 ==== 100+0+11 (secure 0 0 0) 0x7fa12c075be0 con 0x7fa13c0c9ee0 2023-08-09T13:55:10.878 INFO:teuthology.orchestra.run.smithi043.stdout:pg_num: 64 2023-08-09T13:55:10.880 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:10.877+0000 7fa142a5d640 1 -- 172.21.15.43:0/605257647 >> [v2:172.21.15.203:6832/3490266242,v1:172.21.15.203:6833/3490266242] conn(0x7fa120069ea0 msgr2=0x7fa12006c360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:55:10.881 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:10.877+0000 7fa142a5d640 1 --2- 172.21.15.43:0/605257647 >> [v2:172.21.15.203:6832/3490266242,v1:172.21.15.203:6833/3490266242] conn(0x7fa120069ea0 0x7fa12006c360 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7fa138001270 tx=0x7fa138007bb0 comp rx=0 tx=0).stop 2023-08-09T13:55:10.881 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:10.878+0000 7fa142a5d640 1 -- 172.21.15.43:0/605257647 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa13c0c9ee0 msgr2=0x7fa13c0c3110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:55:10.881 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:10.878+0000 7fa142a5d640 1 --2- 172.21.15.43:0/605257647 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa13c0c9ee0 0x7fa13c0c3110 secure :-1 s=READY pgs=1601 cs=0 l=1 rev1=1 crypto rx=0x7fa12c034040 tx=0x7fa12c0046c0 comp rx=0 tx=0).stop 2023-08-09T13:55:10.881 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:10.878+0000 7fa142a5d640 1 -- 172.21.15.43:0/605257647 shutdown_connections 2023-08-09T13:55:10.881 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:10.878+0000 7fa142a5d640 1 --2- 172.21.15.43:0/605257647 >> [v2:172.21.15.203:6832/3490266242,v1:172.21.15.203:6833/3490266242] conn(0x7fa120069ea0 0x7fa12006c360 unknown :-1 s=CLOSED pgs=24 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:55:10.881 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:10.878+0000 7fa142a5d640 1 --2- 172.21.15.43:0/605257647 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fa13c157ea0 0x7fa13c157350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:55:10.881 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:10.878+0000 7fa142a5d640 1 --2- 172.21.15.43:0/605257647 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa13c0c9ee0 0x7fa13c0c3110 unknown :-1 s=CLOSED pgs=1601 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:55:10.882 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:10.878+0000 7fa142a5d640 1 --2- 172.21.15.43:0/605257647 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fa13c0c90a0 0x7fa13c0c65c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:55:10.882 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:10.878+0000 7fa142a5d640 1 -- 172.21.15.43:0/605257647 >> 172.21.15.43:0/605257647 conn(0x7fa13c0cddd0 msgr2=0x7fa13c06c480 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-09T13:55:10.882 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:10.878+0000 7fa142a5d640 1 -- 172.21.15.43:0/605257647 shutdown_connections 2023-08-09T13:55:10.882 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:10.878+0000 7fa142a5d640 1 -- 172.21.15.43:0/605257647 wait complete. 2023-08-09T13:55:10.899 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-08-09T13:55:11.129 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:11.127+0000 7fc2a7234640 1 Processor -- start 2023-08-09T13:55:11.130 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:11.127+0000 7fc2a7234640 1 -- start start 2023-08-09T13:55:11.131 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:11.129+0000 7fc2a7234640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fc2a00b2b70 0x7fc2a00d2ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:55:11.131 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:11.129+0000 7fc2a7234640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7fc2a00b3050 0x7fc2a00b3f60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:55:11.131 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:11.129+0000 7fc2a7234640 1 -- --> v1:172.21.15.203:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc2a00ce090 con 0x7fc2a00b2790 2023-08-09T13:55:11.131 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:11.129+0000 7fc2a7234640 1 -- --> v2:172.21.15.43:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc2a00b3950 con 0x7fc2a00b3050 2023-08-09T13:55:11.132 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:11.129+0000 7fc2a7234640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc2a00cd7f0 con 0x7fc2a00b2b70 2023-08-09T13:55:11.132 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:11.129+0000 7fc2a57d1640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7fc2a00b3050 0x7fc2a00b3f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:55:11.133 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:11.129+0000 7fc2a57d1640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7fc2a00b3050 0x7fc2a00b3f60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:56558/0 (socket says 172.21.15.43:56558) 2023-08-09T13:55:11.133 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:11.129+0000 7fc2a57d1640 1 -- 172.21.15.43:0/3730346461 learned_addr learned my addr 172.21.15.43:0/3730346461 (peer_addr_for_me v2:172.21.15.43:0/0) 2023-08-09T13:55:11.133 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:11.129+0000 7fc29ffff640 1 --2- 172.21.15.43:0/3730346461 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fc2a00b2b70 0x7fc2a00d2ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:55:11.133 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:11.130+0000 7fc2a57d1640 1 -- 172.21.15.43:0/3730346461 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fc2a00b2b70 msgr2=0x7fc2a00d2ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-09T13:55:11.133 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:11.130+0000 7fc2a57d1640 1 --2- 172.21.15.43:0/3730346461 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fc2a00b2b70 0x7fc2a00d2ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:55:11.133 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:11.130+0000 7fc2a57d1640 1 -- 172.21.15.43:0/3730346461 >> v1:172.21.15.203:6789/0 conn(0x7fc2a00b2790 legacy=0x7fc2a00b5330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-09T13:55:11.133 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:11.130+0000 7fc2a57d1640 1 -- 172.21.15.43:0/3730346461 --> v2:172.21.15.43:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc2a00cdad0 con 0x7fc2a00b3050 2023-08-09T13:55:11.134 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:11.131+0000 7fc2a57d1640 1 --2- 172.21.15.43:0/3730346461 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc2a00b3050 0x7fc2a00b3f60 secure :-1 s=READY pgs=1602 cs=0 l=1 rev1=1 crypto rx=0x7fc298009870 tx=0x7fc29802b8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=125650ac783bebd3 server_cookie=0 in_seq=0 out_seq=0 2023-08-09T13:55:11.135 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:11.131+0000 7fc29f7fe640 1 -- 172.21.15.43:0/3730346461 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc298031640 con 0x7fc2a00b3050 2023-08-09T13:55:11.135 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:11.131+0000 7fc29f7fe640 1 -- 172.21.15.43:0/3730346461 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc2980317a0 con 0x7fc2a00b3050 2023-08-09T13:55:11.135 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:11.131+0000 7fc29f7fe640 1 -- 172.21.15.43:0/3730346461 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc298031b30 con 0x7fc2a00b3050 2023-08-09T13:55:11.135 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:11.132+0000 7fc2a7234640 1 -- 172.21.15.43:0/3730346461 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc2a00b3050 msgr2=0x7fc2a00b3f60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:55:11.136 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:11.132+0000 7fc2a7234640 1 --2- 172.21.15.43:0/3730346461 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc2a00b3050 0x7fc2a00b3f60 secure :-1 s=READY pgs=1602 cs=0 l=1 rev1=1 crypto rx=0x7fc298009870 tx=0x7fc29802b8f0 comp rx=0 tx=0).stop 2023-08-09T13:55:11.136 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:11.132+0000 7fc2a7234640 1 -- 172.21.15.43:0/3730346461 shutdown_connections 2023-08-09T13:55:11.136 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:11.132+0000 7fc2a7234640 1 --2- 172.21.15.43:0/3730346461 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc2a00b3050 0x7fc2a00b3f60 unknown :-1 s=CLOSED pgs=1602 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:55:11.136 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:11.132+0000 7fc2a7234640 1 --2- 172.21.15.43:0/3730346461 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fc2a00b2b70 0x7fc2a00d2ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:55:11.136 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:11.132+0000 7fc2a7234640 1 -- 172.21.15.43:0/3730346461 >> 172.21.15.43:0/3730346461 conn(0x7fc2a009b7b0 msgr2=0x7fc2a00d24e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-09T13:55:11.137 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:11.132+0000 7fc2a7234640 1 -- 172.21.15.43:0/3730346461 shutdown_connections 2023-08-09T13:55:11.137 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:11.132+0000 7fc2a7234640 1 -- 172.21.15.43:0/3730346461 wait complete. 2023-08-09T13:55:11.140 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:11.134+0000 7fc2a7234640 1 Processor -- start 2023-08-09T13:55:11.140 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:11.134+0000 7fc2a7234640 1 -- start start 2023-08-09T13:55:11.141 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:11.135+0000 7fc2a7234640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc2a00b2790 0x7fc2a0154870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:55:11.141 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:11.135+0000 7fc2a7234640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fc2a00b2b70 0x7fc2a01555f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:55:11.141 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:11.135+0000 7fc2a7234640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fc2a00b3050 0x7fc2a00cb780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:55:11.142 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:11.135+0000 7fc2a7234640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc2a0154db0 con 0x7fc2a00b2790 2023-08-09T13:55:11.142 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:11.135+0000 7fc2a7234640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc2a01f2100 con 0x7fc2a00b3050 2023-08-09T13:55:11.142 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:11.135+0000 7fc2a7234640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc2a00b1a20 con 0x7fc2a00b2b70 2023-08-09T13:55:11.142 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:11.135+0000 7fc2a4fd0640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc2a00b2790 0x7fc2a0154870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:55:11.143 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:11.135+0000 7fc2a4fd0640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc2a00b2790 0x7fc2a0154870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:56570/0 (socket says 172.21.15.43:56570) 2023-08-09T13:55:11.143 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:11.135+0000 7fc2a4fd0640 1 -- 172.21.15.43:0/539766607 learned_addr learned my addr 172.21.15.43:0/539766607 (peer_addr_for_me v2:172.21.15.43:0/0) 2023-08-09T13:55:11.143 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:11.135+0000 7fc2a57d1640 1 --2- 172.21.15.43:0/539766607 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fc2a00b3050 0x7fc2a00cb780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:55:11.143 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:11.135+0000 7fc2a4fd0640 1 -- 172.21.15.43:0/539766607 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fc2a00b2b70 msgr2=0x7fc2a01555f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:55:11.144 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:11.135+0000 7fc2a4fd0640 1 --2- 172.21.15.43:0/539766607 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fc2a00b2b70 0x7fc2a01555f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:55:11.144 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:11.135+0000 7fc2a4fd0640 1 -- 172.21.15.43:0/539766607 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fc2a00b3050 msgr2=0x7fc2a00cb780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:55:11.144 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:11.135+0000 7fc2a4fd0640 1 --2- 172.21.15.43:0/539766607 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fc2a00b3050 0x7fc2a00cb780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:55:11.144 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:11.136+0000 7fc2a4fd0640 1 -- 172.21.15.43:0/539766607 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc2980094d0 con 0x7fc2a00b2790 2023-08-09T13:55:11.145 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:11.136+0000 7fc2a4fd0640 1 --2- 172.21.15.43:0/539766607 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc2a00b2790 0x7fc2a0154870 secure :-1 s=READY pgs=1603 cs=0 l=1 rev1=1 crypto rx=0x7fc29000ba50 tx=0x7fc29000bf10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-09T13:55:11.145 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:11.136+0000 7fc29d7fa640 1 -- 172.21.15.43:0/539766607 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc29000cbe0 con 0x7fc2a00b2790 2023-08-09T13:55:11.145 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:11.136+0000 7fc29d7fa640 1 -- 172.21.15.43:0/539766607 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc29000cd80 con 0x7fc2a00b2790 2023-08-09T13:55:11.145 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:11.136+0000 7fc29d7fa640 1 -- 172.21.15.43:0/539766607 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc290014680 con 0x7fc2a00b2790 2023-08-09T13:55:11.145 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:11.136+0000 7fc2a7234640 1 -- 172.21.15.43:0/539766607 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc2a0155b30 con 0x7fc2a00b2790 2023-08-09T13:55:11.146 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:11.136+0000 7fc2a7234640 1 -- 172.21.15.43:0/539766607 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc2a01f2340 con 0x7fc2a00b2790 2023-08-09T13:55:11.146 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:11.138+0000 7fc2a7234640 1 -- 172.21.15.43:0/539766607 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc2a00739f0 con 0x7fc2a00b2790 2023-08-09T13:55:11.147 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:11.138+0000 7fc29d7fa640 1 -- 172.21.15.43:0/539766607 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 65) v1 ==== 88179+0+0 (secure 0 0 0) 0x7fc2900147e0 con 0x7fc2a00b2790 2023-08-09T13:55:11.147 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:11.139+0000 7fc29d7fa640 1 --2- 172.21.15.43:0/539766607 >> [v2:172.21.15.203:6832/3490266242,v1:172.21.15.203:6833/3490266242] conn(0x7fc28c069ea0 0x7fc28c06c360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:55:11.147 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:11.140+0000 7fc29d7fa640 1 -- 172.21.15.43:0/539766607 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(134..134 src has 1..134) v4 ==== 7286+0+0 (secure 0 0 0) 0x7fc290088640 con 0x7fc2a00b2790 2023-08-09T13:55:11.147 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:11.140+0000 7fc29ffff640 1 --2- 172.21.15.43:0/539766607 >> [v2:172.21.15.203:6832/3490266242,v1:172.21.15.203:6833/3490266242] conn(0x7fc28c069ea0 0x7fc28c06c360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:55:11.147 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:11.141+0000 7fc29ffff640 1 --2- 172.21.15.43:0/539766607 >> [v2:172.21.15.203:6832/3490266242,v1:172.21.15.203:6833/3490266242] conn(0x7fc28c069ea0 0x7fc28c06c360 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7fc294006fd0 tx=0x7fc294008040 comp rx=0 tx=0).ready entity=mgr.6659 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-09T13:55:11.147 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:11.144+0000 7fc29d7fa640 1 -- 172.21.15.43:0/539766607 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178066 (secure 0 0 0) 0x7fc290053b80 con 0x7fc2a00b2790 2023-08-09T13:55:11.382 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:11.380+0000 7fc2a7234640 1 -- 172.21.15.43:0/539766607 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fc2a00b4b00 con 0x7fc2a00b2790 2023-08-09T13:55:11.385 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:11.383+0000 7fc29d7fa640 1 -- 172.21.15.43:0/539766607 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 61 v61) v1 ==== 94+0+4880 (secure 0 0 0) 0x7fc2900571a0 con 0x7fc2a00b2790 2023-08-09T13:55:11.385 INFO:teuthology.orchestra.run.smithi043.stdout: 2023-08-09T13:55:11.386 INFO:teuthology.orchestra.run.smithi043.stdout:{"epoch":61,"default_fscid":7,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6530,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.203:6835/1659419253","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6834","nonce":1659419253},{"type":"v1","addr":"172.21.15.203:6835","nonce":1659419253}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":56},{"gid":6536,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.203:6837/3964427549","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6836","nonce":3964427549},{"type":"v1","addr":"172.21.15.203:6837","nonce":3964427549}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":56},{"gid":8574,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.43:6833/871812392","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6832","nonce":871812392},{"type":"v1","addr":"172.21.15.43:6833","nonce":871812392}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":55}],"filesystems":[{"mdsmap":{"epoch":61,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-08-09T13:53:47.747881+0000","modified":"2023-08-09T13:53:51.317185+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":8580},"failed":[],"damaged":[],"stopped":[],"info":{"gid_8580":{"gid":8580,"name":"a","rank":0,"incarnation":57,"state":"up:active","state_seq":3,"addr":"172.21.15.43:6835/1794836673","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6834","nonce":1794836673},{"type":"v1","addr":"172.21.15.43:6835","nonce":1794836673}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[17],"metadata_pool":16,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":7}]} 2023-08-09T13:55:11.386 INFO:teuthology.orchestra.run.smithi043.stderr:dumped fsmap epoch 61 2023-08-09T13:55:11.388 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:11.385+0000 7fc2a7234640 1 -- 172.21.15.43:0/539766607 >> [v2:172.21.15.203:6832/3490266242,v1:172.21.15.203:6833/3490266242] conn(0x7fc28c069ea0 msgr2=0x7fc28c06c360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:55:11.388 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:11.385+0000 7fc2a7234640 1 --2- 172.21.15.43:0/539766607 >> [v2:172.21.15.203:6832/3490266242,v1:172.21.15.203:6833/3490266242] conn(0x7fc28c069ea0 0x7fc28c06c360 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7fc294006fd0 tx=0x7fc294008040 comp rx=0 tx=0).stop 2023-08-09T13:55:11.388 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:11.385+0000 7fc2a7234640 1 -- 172.21.15.43:0/539766607 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc2a00b2790 msgr2=0x7fc2a0154870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:55:11.388 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:11.385+0000 7fc2a7234640 1 --2- 172.21.15.43:0/539766607 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc2a00b2790 0x7fc2a0154870 secure :-1 s=READY pgs=1603 cs=0 l=1 rev1=1 crypto rx=0x7fc29000ba50 tx=0x7fc29000bf10 comp rx=0 tx=0).stop 2023-08-09T13:55:11.389 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:11.386+0000 7fc2a7234640 1 -- 172.21.15.43:0/539766607 shutdown_connections 2023-08-09T13:55:11.389 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:11.386+0000 7fc2a7234640 1 --2- 172.21.15.43:0/539766607 >> [v2:172.21.15.203:6832/3490266242,v1:172.21.15.203:6833/3490266242] conn(0x7fc28c069ea0 0x7fc28c06c360 unknown :-1 s=CLOSED pgs=25 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:55:11.389 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:11.386+0000 7fc2a7234640 1 --2- 172.21.15.43:0/539766607 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fc2a00b3050 0x7fc2a00cb780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:55:11.389 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:11.386+0000 7fc2a7234640 1 --2- 172.21.15.43:0/539766607 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fc2a00b2b70 0x7fc2a01555f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:55:11.389 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:11.386+0000 7fc2a7234640 1 --2- 172.21.15.43:0/539766607 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc2a00b2790 0x7fc2a0154870 unknown :-1 s=CLOSED pgs=1603 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:55:11.389 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:11.386+0000 7fc2a7234640 1 -- 172.21.15.43:0/539766607 >> 172.21.15.43:0/539766607 conn(0x7fc2a009b7b0 msgr2=0x7fc2a006af90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-09T13:55:11.389 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:11.386+0000 7fc2a7234640 1 -- 172.21.15.43:0/539766607 shutdown_connections 2023-08-09T13:55:11.390 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:11.386+0000 7fc2a7234640 1 -- 172.21.15.43:0/539766607 wait complete. 2023-08-09T13:55:11.411 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-08-09T13:55:11.630 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:11.628+0000 7f97b9691640 1 Processor -- start 2023-08-09T13:55:11.631 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:11.628+0000 7f97b9691640 1 -- start start 2023-08-09T13:55:11.632 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:11.630+0000 7f97b9691640 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7f97b415a240 0x7f97b40d0c70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:55:11.632 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:11.630+0000 7f97b9691640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f97b40cb770 0x7f97b40ce8a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:55:11.633 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:11.630+0000 7f97b9691640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f97b40cede0 0x7f97b406d6a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:55:11.633 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:11.630+0000 7f97b9691640 1 -- --> v2:172.21.15.43:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f97b40cbce0 con 0x7f97b40cb770 2023-08-09T13:55:11.633 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:11.630+0000 7f97b9691640 1 -- --> v2:172.21.15.203:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f97b40cbfc0 con 0x7f97b415a240 2023-08-09T13:55:11.633 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:11.630+0000 7f97b9691640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f97b40cddc0 con 0x7f97b40cede0 2023-08-09T13:55:11.634 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:11.630+0000 7f97b37fe640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f97b40cb770 0x7f97b40ce8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:55:11.634 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:11.630+0000 7f97b37fe640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f97b40cb770 0x7f97b40ce8a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:56572/0 (socket says 172.21.15.43:56572) 2023-08-09T13:55:11.634 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:11.630+0000 7f97b37fe640 1 -- 172.21.15.43:0/3176486975 learned_addr learned my addr 172.21.15.43:0/3176486975 (peer_addr_for_me v2:172.21.15.43:0/0) 2023-08-09T13:55:11.634 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:11.631+0000 7f97b3fff640 1 --2- 172.21.15.43:0/3176486975 >> v2:172.21.15.203:3300/0 conn(0x7f97b415a240 0x7f97b40d0c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:55:11.634 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:11.631+0000 7f97b8e90640 1 --2- 172.21.15.43:0/3176486975 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f97b40cede0 0x7f97b406d6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:55:11.635 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:11.631+0000 7f97b37fe640 1 -- 172.21.15.43:0/3176486975 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f97b40cede0 msgr2=0x7f97b406d6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-09T13:55:11.635 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:11.631+0000 7f97b37fe640 1 --2- 172.21.15.43:0/3176486975 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f97b40cede0 0x7f97b406d6a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:55:11.635 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:11.631+0000 7f97b37fe640 1 -- 172.21.15.43:0/3176486975 >> v2:172.21.15.203:3300/0 conn(0x7f97b415a240 msgr2=0x7f97b40d0c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-09T13:55:11.635 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:11.631+0000 7f97b37fe640 1 --2- 172.21.15.43:0/3176486975 >> v2:172.21.15.203:3300/0 conn(0x7f97b415a240 0x7f97b40d0c70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:55:11.635 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:11.631+0000 7f97b37fe640 1 -- 172.21.15.43:0/3176486975 --> v2:172.21.15.43:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f97b40ce0a0 con 0x7f97b40cb770 2023-08-09T13:55:11.636 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:11.632+0000 7f97b37fe640 1 --2- 172.21.15.43:0/3176486975 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f97b40cb770 0x7f97b40ce8a0 secure :-1 s=READY pgs=1604 cs=0 l=1 rev1=1 crypto rx=0x7f97b406fef0 tx=0x7f97a802b920 comp rx=0 tx=0).ready entity=mon.0 client_cookie=efaabc0f2599d14f server_cookie=0 in_seq=0 out_seq=0 2023-08-09T13:55:11.636 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:11.632+0000 7f97b2ffd640 1 -- 172.21.15.43:0/3176486975 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f97a8031880 con 0x7f97b40cb770 2023-08-09T13:55:11.636 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:11.632+0000 7f97b2ffd640 1 -- 172.21.15.43:0/3176486975 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f97a80319e0 con 0x7f97b40cb770 2023-08-09T13:55:11.636 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:11.632+0000 7f97b2ffd640 1 -- 172.21.15.43:0/3176486975 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f97a8031d70 con 0x7f97b40cb770 2023-08-09T13:55:11.637 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:11.633+0000 7f97b9691640 1 -- 172.21.15.43:0/3176486975 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f97b40cb770 msgr2=0x7f97b40ce8a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:55:11.637 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:11.633+0000 7f97b9691640 1 --2- 172.21.15.43:0/3176486975 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f97b40cb770 0x7f97b40ce8a0 secure :-1 s=READY pgs=1604 cs=0 l=1 rev1=1 crypto rx=0x7f97b406fef0 tx=0x7f97a802b920 comp rx=0 tx=0).stop 2023-08-09T13:55:11.637 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:11.633+0000 7f97b9691640 1 -- 172.21.15.43:0/3176486975 shutdown_connections 2023-08-09T13:55:11.637 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:11.633+0000 7f97b9691640 1 --2- 172.21.15.43:0/3176486975 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f97b40cb770 0x7f97b40ce8a0 unknown :-1 s=CLOSED pgs=1604 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:55:11.637 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:11.633+0000 7f97b9691640 1 --2- 172.21.15.43:0/3176486975 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f97b40cede0 0x7f97b406d6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:55:11.637 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:11.633+0000 7f97b9691640 1 --2- 172.21.15.43:0/3176486975 >> v2:172.21.15.203:3300/0 conn(0x7f97b415a240 0x7f97b40d0c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:55:11.637 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:11.633+0000 7f97b9691640 1 -- 172.21.15.43:0/3176486975 >> 172.21.15.43:0/3176486975 conn(0x7f97b40d0150 msgr2=0x7f97b40d0560 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-09T13:55:11.637 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:11.633+0000 7f97b9691640 1 -- 172.21.15.43:0/3176486975 shutdown_connections 2023-08-09T13:55:11.638 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:11.633+0000 7f97b9691640 1 -- 172.21.15.43:0/3176486975 wait complete. 2023-08-09T13:55:11.638 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:11.636+0000 7f97b9691640 1 Processor -- start 2023-08-09T13:55:11.638 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:11.636+0000 7f97b9691640 1 -- start start 2023-08-09T13:55:11.639 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:11.637+0000 7f97b9691640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f97b40cb770 0x7f97b4150350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:55:11.640 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:11.637+0000 7f97b9691640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f97b415a240 0x7f97b40c9b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:55:11.640 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:11.637+0000 7f97b9691640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f97b40c4e10 0x7f97b414fac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:55:11.640 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:11.637+0000 7f97b9691640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f97b40c8c10 con 0x7f97b415a240 2023-08-09T13:55:11.640 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:11.637+0000 7f97b9691640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f97b40ca0a0 con 0x7f97b40c4e10 2023-08-09T13:55:11.640 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:11.637+0000 7f97b9691640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f97b40c8930 con 0x7f97b40cb770 2023-08-09T13:55:11.641 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:11.637+0000 7f97b37fe640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f97b415a240 0x7f97b40c9b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:55:11.642 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:11.638+0000 7f97b37fe640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f97b415a240 0x7f97b40c9b60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:56584/0 (socket says 172.21.15.43:56584) 2023-08-09T13:55:11.642 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:11.638+0000 7f97b37fe640 1 -- 172.21.15.43:0/1893098994 learned_addr learned my addr 172.21.15.43:0/1893098994 (peer_addr_for_me v2:172.21.15.43:0/0) 2023-08-09T13:55:11.642 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:11.638+0000 7f97b8e90640 1 --2- 172.21.15.43:0/1893098994 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f97b40c4e10 0x7f97b414fac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:55:11.642 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:11.638+0000 7f97b3fff640 1 --2- 172.21.15.43:0/1893098994 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f97b40cb770 0x7f97b4150350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:55:11.642 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:11.638+0000 7f97b37fe640 1 -- 172.21.15.43:0/1893098994 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f97b40cb770 msgr2=0x7f97b4150350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:55:11.642 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:11.638+0000 7f97b37fe640 1 --2- 172.21.15.43:0/1893098994 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f97b40cb770 0x7f97b4150350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:55:11.642 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:11.638+0000 7f97b37fe640 1 -- 172.21.15.43:0/1893098994 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f97b40c4e10 msgr2=0x7f97b414fac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:55:11.643 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:11.638+0000 7f97b37fe640 1 --2- 172.21.15.43:0/1893098994 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f97b40c4e10 0x7f97b414fac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:55:11.643 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:11.638+0000 7f97b37fe640 1 -- 172.21.15.43:0/1893098994 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f97a80094d0 con 0x7f97b415a240 2023-08-09T13:55:11.643 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:11.639+0000 7f97b37fe640 1 --2- 172.21.15.43:0/1893098994 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f97b415a240 0x7f97b40c9b60 secure :-1 s=READY pgs=1605 cs=0 l=1 rev1=1 crypto rx=0x7f97a802b960 tx=0x7f97a80046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-09T13:55:11.643 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:11.639+0000 7f97b0ff9640 1 -- 172.21.15.43:0/1893098994 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f97a803ad30 con 0x7f97b415a240 2023-08-09T13:55:11.643 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:11.639+0000 7f97b0ff9640 1 -- 172.21.15.43:0/1893098994 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f97a80443f0 con 0x7f97b415a240 2023-08-09T13:55:11.643 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:11.639+0000 7f97b0ff9640 1 -- 172.21.15.43:0/1893098994 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f97a80447a0 con 0x7f97b415a240 2023-08-09T13:55:11.643 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:11.639+0000 7f97b9691640 1 -- 172.21.15.43:0/1893098994 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f97b40c9080 con 0x7f97b415a240 2023-08-09T13:55:11.643 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:11.639+0000 7f97b9691640 1 -- 172.21.15.43:0/1893098994 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f97b40c9640 con 0x7f97b415a240 2023-08-09T13:55:11.644 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:11.640+0000 7f97b9691640 1 -- 172.21.15.43:0/1893098994 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f97b40c8ee0 con 0x7f97b415a240 2023-08-09T13:55:11.644 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:11.641+0000 7f97b0ff9640 1 -- 172.21.15.43:0/1893098994 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 66) v1 ==== 88387+0+0 (secure 0 0 0) 0x7f97a804d450 con 0x7f97b415a240 2023-08-09T13:55:11.645 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:11.642+0000 7f97b0ff9640 1 --2- 172.21.15.43:0/1893098994 >> [v2:172.21.15.203:6832/3490266242,v1:172.21.15.203:6833/3490266242] conn(0x7f97a006a0e0 0x7f97a006c5a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:55:11.645 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:11.643+0000 7f97b0ff9640 1 -- 172.21.15.43:0/1893098994 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(134..134 src has 1..134) v4 ==== 7286+0+0 (secure 0 0 0) 0x7f97a80ab260 con 0x7f97b415a240 2023-08-09T13:55:11.645 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:11.643+0000 7f97b3fff640 1 --2- 172.21.15.43:0/1893098994 >> [v2:172.21.15.203:6832/3490266242,v1:172.21.15.203:6833/3490266242] conn(0x7f97a006a0e0 0x7f97a006c5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:55:11.647 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:11.644+0000 7f97b3fff640 1 --2- 172.21.15.43:0/1893098994 >> [v2:172.21.15.203:6832/3490266242,v1:172.21.15.203:6833/3490266242] conn(0x7f97a006a0e0 0x7f97a006c5a0 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7f97b41524c0 tx=0x7f97a4008040 comp rx=0 tx=0).ready entity=mgr.6659 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-09T13:55:11.650 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:11.648+0000 7f97b0ff9640 1 -- 172.21.15.43:0/1893098994 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178066 (secure 0 0 0) 0x7f97b40c8ee0 con 0x7f97b415a240 2023-08-09T13:55:11.863 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:11.860+0000 7f97b9691640 1 -- 172.21.15.43:0/1893098994 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f97b40c8ee0 con 0x7f97b415a240 2023-08-09T13:55:11.865 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:11.863+0000 7f97b0ff9640 1 -- 172.21.15.43:0/1893098994 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v134) v1 ==== 74+0+19819 (secure 0 0 0) 0x7f97b40c8ee0 con 0x7f97b415a240 2023-08-09T13:55:11.866 INFO:teuthology.orchestra.run.smithi043.stdout: 2023-08-09T13:55:11.866 INFO:teuthology.orchestra.run.smithi043.stdout:{"epoch":134,"fsid":"daac87ee-1337-4250-a955-aea4e408a39b","created":"2023-08-09T13:33:41.701198+0000","modified":"2023-08-09T13:55:08.528629+0000","last_up_change":"2023-08-09T13:33:53.783453+0000","last_in_change":"2023-08-09T13:33:45.148904+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":17,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":"rbd","create_time":"2023-08-09T13:33:54.365137+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":".mgr","create_time":"2023-08-09T13:33:54.831803+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":16,"pool_name":"cephfs_metadata","create_time":"2023-08-09T13:53:46.124616+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"128","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":17,"pool_name":"cephfs_data","create_time":"2023-08-09T13:53:46.762733+0000","flags":8193,"flags_names":"hashpspool,selfmanaged_snaps","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"133","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":3,"snap_epoch":133,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"cbd49ecd-e896-4fbd-864e-5c37a829b165","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":127,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6800","nonce":502667335},{"type":"v1","addr":"172.21.15.43:6801","nonce":502667335}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6802","nonce":502667335},{"type":"v1","addr":"172.21.15.43:6803","nonce":502667335}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6806","nonce":502667335},{"type":"v1","addr":"172.21.15.43:6807","nonce":502667335}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6804","nonce":502667335},{"type":"v1","addr":"172.21.15.43:6805","nonce":502667335}]},"public_addr":"172.21.15.43:6801/502667335","cluster_addr":"172.21.15.43:6803/502667335","heartbeat_back_addr":"172.21.15.43:6807/502667335","heartbeat_front_addr":"172.21.15.43:6805/502667335","state":["exists","up"]},{"osd":1,"uuid":"75c7e0fe-1ec7-4fe8-a2f2-3661a65b8500","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":131,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6808","nonce":3289134610},{"type":"v1","addr":"172.21.15.43:6809","nonce":3289134610}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6810","nonce":3289134610},{"type":"v1","addr":"172.21.15.43:6812","nonce":3289134610}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6821","nonce":3289134610},{"type":"v1","addr":"172.21.15.43:6824","nonce":3289134610}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6815","nonce":3289134610},{"type":"v1","addr":"172.21.15.43:6818","nonce":3289134610}]},"public_addr":"172.21.15.43:6809/3289134610","cluster_addr":"172.21.15.43:6812/3289134610","heartbeat_back_addr":"172.21.15.43:6824/3289134610","heartbeat_front_addr":"172.21.15.43:6818/3289134610","state":["exists","up"]},{"osd":2,"uuid":"bb5c41e9-7f5f-4125-9b01-3bb8bfc2ea83","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":127,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6811","nonce":4079709736},{"type":"v1","addr":"172.21.15.43:6814","nonce":4079709736}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6817","nonce":4079709736},{"type":"v1","addr":"172.21.15.43:6820","nonce":4079709736}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6829","nonce":4079709736},{"type":"v1","addr":"172.21.15.43:6831","nonce":4079709736}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6823","nonce":4079709736},{"type":"v1","addr":"172.21.15.43:6826","nonce":4079709736}]},"public_addr":"172.21.15.43:6814/4079709736","cluster_addr":"172.21.15.43:6820/4079709736","heartbeat_back_addr":"172.21.15.43:6831/4079709736","heartbeat_front_addr":"172.21.15.43:6826/4079709736","state":["exists","up"]},{"osd":3,"uuid":"bfe4c8b4-694a-465c-87e6-2b2aab852613","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":127,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6813","nonce":1709538759},{"type":"v1","addr":"172.21.15.43:6816","nonce":1709538759}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6819","nonce":1709538759},{"type":"v1","addr":"172.21.15.43:6822","nonce":1709538759}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6828","nonce":1709538759},{"type":"v1","addr":"172.21.15.43:6830","nonce":1709538759}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6825","nonce":1709538759},{"type":"v1","addr":"172.21.15.43:6827","nonce":1709538759}]},"public_addr":"172.21.15.43:6816/1709538759","cluster_addr":"172.21.15.43:6822/1709538759","heartbeat_back_addr":"172.21.15.43:6830/1709538759","heartbeat_front_addr":"172.21.15.43:6827/1709538759","state":["exists","up"]},{"osd":4,"uuid":"fa2e8a77-aba1-4756-853e-0d174a5bdeba","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":127,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6800","nonce":4196866481},{"type":"v1","addr":"172.21.15.203:6801","nonce":4196866481}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6802","nonce":4196866481},{"type":"v1","addr":"172.21.15.203:6803","nonce":4196866481}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6806","nonce":4196866481},{"type":"v1","addr":"172.21.15.203:6807","nonce":4196866481}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6804","nonce":4196866481},{"type":"v1","addr":"172.21.15.203:6805","nonce":4196866481}]},"public_addr":"172.21.15.203:6801/4196866481","cluster_addr":"172.21.15.203:6803/4196866481","heartbeat_back_addr":"172.21.15.203:6807/4196866481","heartbeat_front_addr":"172.21.15.203:6805/4196866481","state":["exists","up"]},{"osd":5,"uuid":"81811e4b-4d24-4f51-a82c-f1d700a2a00d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":127,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6816","nonce":3059290081},{"type":"v1","addr":"172.21.15.203:6817","nonce":3059290081}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6818","nonce":3059290081},{"type":"v1","addr":"172.21.15.203:6819","nonce":3059290081}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6822","nonce":3059290081},{"type":"v1","addr":"172.21.15.203:6823","nonce":3059290081}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6820","nonce":3059290081},{"type":"v1","addr":"172.21.15.203:6821","nonce":3059290081}]},"public_addr":"172.21.15.203:6817/3059290081","cluster_addr":"172.21.15.203:6819/3059290081","heartbeat_back_addr":"172.21.15.203:6823/3059290081","heartbeat_front_addr":"172.21.15.203:6821/3059290081","state":["exists","up"]},{"osd":6,"uuid":"307bac4d-6a5b-443d-af75-56d695d7674f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":131,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6824","nonce":512155569},{"type":"v1","addr":"172.21.15.203:6825","nonce":512155569}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6826","nonce":512155569},{"type":"v1","addr":"172.21.15.203:6827","nonce":512155569}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6830","nonce":512155569},{"type":"v1","addr":"172.21.15.203:6831","nonce":512155569}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6828","nonce":512155569},{"type":"v1","addr":"172.21.15.203:6829","nonce":512155569}]},"public_addr":"172.21.15.203:6825/512155569","cluster_addr":"172.21.15.203:6827/512155569","heartbeat_back_addr":"172.21.15.203:6831/512155569","heartbeat_front_addr":"172.21.15.203:6829/512155569","state":["exists","up"]},{"osd":7,"uuid":"a95dc7ca-9f1b-4295-af70-68a22a2d7d5e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":127,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6808","nonce":3331450438},{"type":"v1","addr":"172.21.15.203:6809","nonce":3331450438}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6810","nonce":3331450438},{"type":"v1","addr":"172.21.15.203:6811","nonce":3331450438}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6814","nonce":3331450438},{"type":"v1","addr":"172.21.15.203:6815","nonce":3331450438}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6812","nonce":3331450438},{"type":"v1","addr":"172.21.15.203:6813","nonce":3331450438}]},"public_addr":"172.21.15.203:6809/3331450438","cluster_addr":"172.21.15.203:6811/3331450438","heartbeat_back_addr":"172.21.15.203:6815/3331450438","heartbeat_front_addr":"172.21.15.203:6813/3331450438","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-09T13:33:50.547652+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-09T13:33:50.589469+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-09T13:33:50.561724+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-09T13:33:50.599650+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-09T13:33:50.411855+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-09T13:33:50.219681+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-09T13:33:50.504664+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-09T13:33:50.441248+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"17.5","mappings":[{"from":0,"to":5}]},{"pgid":"17.f","mappings":[{"from":0,"to":2}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.203:0/1151006972":"2023-08-10T13:55:08.528494+0000","172.21.15.203:0/274636031":"2023-08-10T13:55:08.528494+0000","172.21.15.203:0/3526078924":"2023-08-10T13:55:08.528494+0000","172.21.15.203:6833/1796635339":"2023-08-10T13:55:08.528494+0000","172.21.15.203:6832/1796635339":"2023-08-10T13:55:08.528494+0000","172.21.15.203:0/1153043111":"2023-08-10T13:55:08.528494+0000","172.21.15.203:0/3657978600":"2023-08-10T13:55:08.528494+0000","172.21.15.203:0/3093730384":"2023-08-10T13:55:08.528494+0000","172.21.15.203:6833/2527059680":"2023-08-10T13:54:03.238790+0000","172.21.15.203:6832/2527059680":"2023-08-10T13:54:03.238790+0000","172.21.15.203:0/431304609":"2023-08-10T13:54:03.238790+0000","172.21.15.203:0/1911913635":"2023-08-10T13:55:08.528494+0000","172.21.15.203:0/826469783":"2023-08-10T13:54:03.238790+0000","172.21.15.203:0/1225835978":"2023-08-10T13:55:08.528494+0000","172.21.15.203:0/1456953448":"2023-08-10T13:54:03.238790+0000","172.21.15.203:0/2004944696":"2023-08-10T13:54:03.238790+0000","172.21.15.203:0/916024371":"2023-08-10T13:54:03.238790+0000","172.21.15.203:0/4209557762":"2023-08-10T13:54:03.238790+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":7,"snaps":[{"begin":104,"length":1},{"begin":221,"length":1}]},{"pool":17,"snaps":[{"begin":2,"length":2}]}],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-08-09T13:55:11.868 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:11.866+0000 7f97b9691640 1 -- 172.21.15.43:0/1893098994 >> [v2:172.21.15.203:6832/3490266242,v1:172.21.15.203:6833/3490266242] conn(0x7f97a006a0e0 msgr2=0x7f97a006c5a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:55:11.869 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:11.866+0000 7f97b9691640 1 --2- 172.21.15.43:0/1893098994 >> [v2:172.21.15.203:6832/3490266242,v1:172.21.15.203:6833/3490266242] conn(0x7f97a006a0e0 0x7f97a006c5a0 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7f97b41524c0 tx=0x7f97a4008040 comp rx=0 tx=0).stop 2023-08-09T13:55:11.869 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:11.866+0000 7f97b9691640 1 -- 172.21.15.43:0/1893098994 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f97b415a240 msgr2=0x7f97b40c9b60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:55:11.869 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:11.866+0000 7f97b9691640 1 --2- 172.21.15.43:0/1893098994 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f97b415a240 0x7f97b40c9b60 secure :-1 s=READY pgs=1605 cs=0 l=1 rev1=1 crypto rx=0x7f97a802b960 tx=0x7f97a80046c0 comp rx=0 tx=0).stop 2023-08-09T13:55:11.869 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:11.866+0000 7f97b9691640 1 -- 172.21.15.43:0/1893098994 shutdown_connections 2023-08-09T13:55:11.869 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:11.866+0000 7f97b9691640 1 --2- 172.21.15.43:0/1893098994 >> [v2:172.21.15.203:6832/3490266242,v1:172.21.15.203:6833/3490266242] conn(0x7f97a006a0e0 0x7f97a006c5a0 unknown :-1 s=CLOSED pgs=26 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:55:11.869 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:11.866+0000 7f97b9691640 1 --2- 172.21.15.43:0/1893098994 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f97b40c4e10 0x7f97b414fac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:55:11.870 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:11.866+0000 7f97b9691640 1 --2- 172.21.15.43:0/1893098994 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f97b415a240 0x7f97b40c9b60 unknown :-1 s=CLOSED pgs=1605 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:55:11.870 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:11.866+0000 7f97b9691640 1 --2- 172.21.15.43:0/1893098994 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f97b40cb770 0x7f97b4150350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:55:11.870 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:11.866+0000 7f97b9691640 1 -- 172.21.15.43:0/1893098994 >> 172.21.15.43:0/1893098994 conn(0x7f97b40d0150 msgr2=0x7f97b40c6ab0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-09T13:55:11.870 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:11.866+0000 7f97b9691640 1 -- 172.21.15.43:0/1893098994 shutdown_connections 2023-08-09T13:55:11.870 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:11.866+0000 7f97b9691640 1 -- 172.21.15.43:0/1893098994 wait complete. 2023-08-09T13:55:11.888 INFO:tasks.cephfs.filesystem:Destroying file system cephfs and related pools 2023-08-09T13:55:11.889 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-08-09T13:55:12.117 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:12.115+0000 7fed69f86640 1 Processor -- start 2023-08-09T13:55:12.117 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:12.115+0000 7fed69f86640 1 -- start start 2023-08-09T13:55:12.119 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:12.116+0000 7fed69f86640 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7fed6415f7b0 0x7fed640b4760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:55:12.119 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:12.116+0000 7fed69f86640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7fed640cd6b0 0x7fed640d0e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:55:12.119 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:12.116+0000 7fed69f86640 1 -- --> v1:172.21.15.43:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fed640b7bf0 con 0x7fed640b1050 2023-08-09T13:55:12.119 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:12.116+0000 7fed69f86640 1 -- --> v2:172.21.15.43:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fed640b7ed0 con 0x7fed640cd6b0 2023-08-09T13:55:12.119 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:12.116+0000 7fed69f86640 1 -- --> v2:172.21.15.203:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fed640d0370 con 0x7fed6415f7b0 2023-08-09T13:55:12.120 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:12.117+0000 7fed63fff640 1 --1- >> v1:172.21.15.43:6789/0 conn(0x7fed640b1050 0x7fed640ccfa0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.43:6789/0 says I am v1:172.21.15.43:42204/0 (socket says 172.21.15.43:42204) 2023-08-09T13:55:12.120 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:12.117+0000 7fed63fff640 1 -- 172.21.15.43:0/2730661128 learned_addr learned my addr 172.21.15.43:0/2730661128 (peer_addr_for_me v1:172.21.15.43:0/0) 2023-08-09T13:55:12.121 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:12.117+0000 7fed69785640 1 --2- 172.21.15.43:0/2730661128 >> v2:172.21.15.43:3300/0 conn(0x7fed640cd6b0 0x7fed640d0e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:55:12.121 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:12.117+0000 7fed68f84640 1 --2- 172.21.15.43:0/2730661128 >> v2:172.21.15.203:3300/0 conn(0x7fed6415f7b0 0x7fed640b4760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:55:12.121 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:12.117+0000 7fed637fe640 1 -- 172.21.15.43:0/2730661128 <== mon.0 v1:172.21.15.43:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1887742686 0 0) 0x7fed640b7bf0 con 0x7fed640b1050 2023-08-09T13:55:12.121 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:12.117+0000 7fed637fe640 1 -- 172.21.15.43:0/2730661128 --> v1:172.21.15.43:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fed50003520 con 0x7fed640b1050 2023-08-09T13:55:12.121 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:12.118+0000 7fed69785640 1 -- 172.21.15.43:0/2730661128 >> v2:172.21.15.203:3300/0 conn(0x7fed6415f7b0 msgr2=0x7fed640b4760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-09T13:55:12.122 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:12.118+0000 7fed69785640 1 --2- 172.21.15.43:0/2730661128 >> v2:172.21.15.203:3300/0 conn(0x7fed6415f7b0 0x7fed640b4760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:55:12.122 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:12.118+0000 7fed69785640 1 -- 172.21.15.43:0/2730661128 >> v1:172.21.15.43:6789/0 conn(0x7fed640b1050 legacy=0x7fed640ccfa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:55:12.122 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:12.118+0000 7fed69785640 1 -- 172.21.15.43:0/2730661128 --> v2:172.21.15.43:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fed640d0650 con 0x7fed640cd6b0 2023-08-09T13:55:12.123 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:12.119+0000 7fed69785640 1 --2- 172.21.15.43:0/2730661128 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fed640cd6b0 0x7fed640d0e50 secure :-1 s=READY pgs=1607 cs=0 l=1 rev1=1 crypto rx=0x7fed5c0097a0 tx=0x7fed5c02b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=da157f078a3afedb server_cookie=0 in_seq=0 out_seq=0 2023-08-09T13:55:12.123 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:12.119+0000 7fed637fe640 1 -- 172.21.15.43:0/2730661128 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fed5c031830 con 0x7fed640cd6b0 2023-08-09T13:55:12.123 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:12.119+0000 7fed637fe640 1 -- 172.21.15.43:0/2730661128 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fed5c031990 con 0x7fed640cd6b0 2023-08-09T13:55:12.123 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:12.119+0000 7fed637fe640 1 -- 172.21.15.43:0/2730661128 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fed5c031d00 con 0x7fed640cd6b0 2023-08-09T13:55:12.123 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:12.120+0000 7fed69f86640 1 -- 172.21.15.43:0/2730661128 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fed640cd6b0 msgr2=0x7fed640d0e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:55:12.124 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:12.120+0000 7fed69f86640 1 --2- 172.21.15.43:0/2730661128 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fed640cd6b0 0x7fed640d0e50 secure :-1 s=READY pgs=1607 cs=0 l=1 rev1=1 crypto rx=0x7fed5c0097a0 tx=0x7fed5c02b8d0 comp rx=0 tx=0).stop 2023-08-09T13:55:12.124 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:12.120+0000 7fed69f86640 1 -- 172.21.15.43:0/2730661128 shutdown_connections 2023-08-09T13:55:12.124 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:12.120+0000 7fed69f86640 1 --2- 172.21.15.43:0/2730661128 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fed640cd6b0 0x7fed640d0e50 unknown :-1 s=CLOSED pgs=1607 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:55:12.124 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:12.120+0000 7fed69f86640 1 --2- 172.21.15.43:0/2730661128 >> v2:172.21.15.203:3300/0 conn(0x7fed6415f7b0 0x7fed640b4760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:55:12.124 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:12.120+0000 7fed69f86640 1 -- 172.21.15.43:0/2730661128 >> 172.21.15.43:0/2730661128 conn(0x7fed6406a2b0 msgr2=0x7fed6406a690 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-09T13:55:12.124 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:12.120+0000 7fed69f86640 1 -- 172.21.15.43:0/2730661128 shutdown_connections 2023-08-09T13:55:12.124 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:12.120+0000 7fed69f86640 1 -- 172.21.15.43:0/2730661128 wait complete. 2023-08-09T13:55:12.125 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:12.122+0000 7fed69f86640 1 Processor -- start 2023-08-09T13:55:12.126 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:12.122+0000 7fed69f86640 1 -- start start 2023-08-09T13:55:12.126 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:12.123+0000 7fed69f86640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fed640b1050 0x7fed641e5610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:55:12.126 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:12.123+0000 7fed69f86640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fed640cd6b0 0x7fed641f51b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:55:12.126 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:12.123+0000 7fed69f86640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fed6415f7b0 0x7fed64148460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:55:12.126 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:12.123+0000 7fed69f86640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fed641f56f0 con 0x7fed640cd6b0 2023-08-09T13:55:12.126 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:12.123+0000 7fed69f86640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fed641490f0 con 0x7fed6415f7b0 2023-08-09T13:55:12.127 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:12.123+0000 7fed69f86640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fed640b1d70 con 0x7fed640b1050 2023-08-09T13:55:12.127 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:12.123+0000 7fed63fff640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fed640cd6b0 0x7fed641f51b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:55:12.127 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:12.123+0000 7fed63fff640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fed640cd6b0 0x7fed641f51b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:56600/0 (socket says 172.21.15.43:56600) 2023-08-09T13:55:12.127 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:12.123+0000 7fed63fff640 1 -- 172.21.15.43:0/249611312 learned_addr learned my addr 172.21.15.43:0/249611312 (peer_addr_for_me v2:172.21.15.43:0/0) 2023-08-09T13:55:12.128 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:12.124+0000 7fed63fff640 1 -- 172.21.15.43:0/249611312 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fed640b1050 msgr2=0x7fed641e5610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:55:12.128 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:12.124+0000 7fed63fff640 1 --2- 172.21.15.43:0/249611312 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fed640b1050 0x7fed641e5610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:55:12.128 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:12.124+0000 7fed63fff640 1 -- 172.21.15.43:0/249611312 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fed6415f7b0 msgr2=0x7fed64148460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:55:12.129 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:12.124+0000 7fed63fff640 1 --2- 172.21.15.43:0/249611312 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fed6415f7b0 0x7fed64148460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:55:12.129 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:12.124+0000 7fed63fff640 1 -- 172.21.15.43:0/249611312 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fed5c009400 con 0x7fed640cd6b0 2023-08-09T13:55:12.129 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:12.124+0000 7fed63fff640 1 --2- 172.21.15.43:0/249611312 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fed640cd6b0 0x7fed641f51b0 secure :-1 s=READY pgs=1608 cs=0 l=1 rev1=1 crypto rx=0x7fed4c00c480 tx=0x7fed4c00c940 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-09T13:55:12.129 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:12.124+0000 7fed617fa640 1 -- 172.21.15.43:0/249611312 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fed4c005ce0 con 0x7fed640cd6b0 2023-08-09T13:55:12.129 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:12.124+0000 7fed617fa640 1 -- 172.21.15.43:0/249611312 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fed4c005e40 con 0x7fed640cd6b0 2023-08-09T13:55:12.129 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:12.124+0000 7fed69f86640 1 -- 172.21.15.43:0/249611312 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fed641f5900 con 0x7fed640cd6b0 2023-08-09T13:55:12.129 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:12.124+0000 7fed617fa640 1 -- 172.21.15.43:0/249611312 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fed4c0072b0 con 0x7fed640cd6b0 2023-08-09T13:55:12.129 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:12.124+0000 7fed69f86640 1 -- 172.21.15.43:0/249611312 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fed641f5ec0 con 0x7fed640cd6b0 2023-08-09T13:55:12.130 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:12.126+0000 7fed69f86640 1 -- 172.21.15.43:0/249611312 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fed640739f0 con 0x7fed640cd6b0 2023-08-09T13:55:12.130 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:12.126+0000 7fed617fa640 1 -- 172.21.15.43:0/249611312 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 66) v1 ==== 88387+0+0 (secure 0 0 0) 0x7fed4c0074d0 con 0x7fed640cd6b0 2023-08-09T13:55:12.130 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:12.128+0000 7fed617fa640 1 --2- 172.21.15.43:0/249611312 >> [v2:172.21.15.203:6832/3490266242,v1:172.21.15.203:6833/3490266242] conn(0x7fed4406a0e0 0x7fed4406c5a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:55:12.131 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:12.128+0000 7fed617fa640 1 -- 172.21.15.43:0/249611312 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(134..134 src has 1..134) v4 ==== 7286+0+0 (secure 0 0 0) 0x7fed4c088110 con 0x7fed640cd6b0 2023-08-09T13:55:12.131 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:12.128+0000 7fed68f84640 1 --2- 172.21.15.43:0/249611312 >> [v2:172.21.15.203:6832/3490266242,v1:172.21.15.203:6833/3490266242] conn(0x7fed4406a0e0 0x7fed4406c5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:55:12.132 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:12.129+0000 7fed68f84640 1 --2- 172.21.15.43:0/249611312 >> [v2:172.21.15.203:6832/3490266242,v1:172.21.15.203:6833/3490266242] conn(0x7fed4406a0e0 0x7fed4406c5a0 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7fed540070b0 tx=0x7fed54007040 comp rx=0 tx=0).ready entity=mgr.6659 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-09T13:55:12.134 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:12.132+0000 7fed617fa640 1 -- 172.21.15.43:0/249611312 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178066 (secure 0 0 0) 0x7fed4c053390 con 0x7fed640cd6b0 2023-08-09T13:55:12.383 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:12.380+0000 7fed69f86640 1 -- 172.21.15.43:0/249611312 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fed640c8c30 con 0x7fed640cd6b0 2023-08-09T13:55:12.387 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:12.384+0000 7fed617fa640 1 -- 172.21.15.43:0/249611312 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 61 v61) v1 ==== 94+0+4880 (secure 0 0 0) 0x7fed4c017020 con 0x7fed640cd6b0 2023-08-09T13:55:12.387 INFO:teuthology.orchestra.run.smithi043.stdout: 2023-08-09T13:55:12.387 INFO:teuthology.orchestra.run.smithi043.stdout:{"epoch":61,"default_fscid":7,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6530,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.203:6835/1659419253","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6834","nonce":1659419253},{"type":"v1","addr":"172.21.15.203:6835","nonce":1659419253}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":56},{"gid":6536,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.203:6837/3964427549","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6836","nonce":3964427549},{"type":"v1","addr":"172.21.15.203:6837","nonce":3964427549}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":56},{"gid":8574,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.43:6833/871812392","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6832","nonce":871812392},{"type":"v1","addr":"172.21.15.43:6833","nonce":871812392}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":55}],"filesystems":[{"mdsmap":{"epoch":61,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-08-09T13:53:47.747881+0000","modified":"2023-08-09T13:53:51.317185+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":8580},"failed":[],"damaged":[],"stopped":[],"info":{"gid_8580":{"gid":8580,"name":"a","rank":0,"incarnation":57,"state":"up:active","state_seq":3,"addr":"172.21.15.43:6835/1794836673","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6834","nonce":1794836673},{"type":"v1","addr":"172.21.15.43:6835","nonce":1794836673}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[17],"metadata_pool":16,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":7}]} 2023-08-09T13:55:12.388 INFO:teuthology.orchestra.run.smithi043.stderr:dumped fsmap epoch 61 2023-08-09T13:55:12.390 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:12.387+0000 7fed69f86640 1 -- 172.21.15.43:0/249611312 >> [v2:172.21.15.203:6832/3490266242,v1:172.21.15.203:6833/3490266242] conn(0x7fed4406a0e0 msgr2=0x7fed4406c5a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:55:12.390 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:12.387+0000 7fed69f86640 1 --2- 172.21.15.43:0/249611312 >> [v2:172.21.15.203:6832/3490266242,v1:172.21.15.203:6833/3490266242] conn(0x7fed4406a0e0 0x7fed4406c5a0 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7fed540070b0 tx=0x7fed54007040 comp rx=0 tx=0).stop 2023-08-09T13:55:12.390 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:12.387+0000 7fed69f86640 1 -- 172.21.15.43:0/249611312 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fed640cd6b0 msgr2=0x7fed641f51b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:55:12.390 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:12.387+0000 7fed69f86640 1 --2- 172.21.15.43:0/249611312 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fed640cd6b0 0x7fed641f51b0 secure :-1 s=READY pgs=1608 cs=0 l=1 rev1=1 crypto rx=0x7fed4c00c480 tx=0x7fed4c00c940 comp rx=0 tx=0).stop 2023-08-09T13:55:12.391 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:12.388+0000 7fed69f86640 1 -- 172.21.15.43:0/249611312 shutdown_connections 2023-08-09T13:55:12.391 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:12.388+0000 7fed69f86640 1 --2- 172.21.15.43:0/249611312 >> [v2:172.21.15.203:6832/3490266242,v1:172.21.15.203:6833/3490266242] conn(0x7fed4406a0e0 0x7fed4406c5a0 unknown :-1 s=CLOSED pgs=27 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:55:12.391 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:12.388+0000 7fed69f86640 1 --2- 172.21.15.43:0/249611312 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fed6415f7b0 0x7fed64148460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:55:12.391 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:12.388+0000 7fed69f86640 1 --2- 172.21.15.43:0/249611312 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fed640cd6b0 0x7fed641f51b0 unknown :-1 s=CLOSED pgs=1608 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:55:12.391 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:12.388+0000 7fed69f86640 1 --2- 172.21.15.43:0/249611312 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fed640b1050 0x7fed641e5610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:55:12.391 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:12.388+0000 7fed69f86640 1 -- 172.21.15.43:0/249611312 >> 172.21.15.43:0/249611312 conn(0x7fed6406a2b0 msgr2=0x7fed6415ab10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-09T13:55:12.391 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:12.388+0000 7fed69f86640 1 -- 172.21.15.43:0/249611312 shutdown_connections 2023-08-09T13:55:12.392 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:12.388+0000 7fed69f86640 1 -- 172.21.15.43:0/249611312 wait complete. 2023-08-09T13:55:12.410 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-08-09T13:55:12.635 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:12.632+0000 7f6b60245640 1 Processor -- start 2023-08-09T13:55:12.635 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:12.632+0000 7f6b60245640 1 -- start start 2023-08-09T13:55:12.637 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:12.634+0000 7f6b60245640 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7f6b5806cfe0 0x7f6b580c8b50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:55:12.637 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:12.634+0000 7f6b60245640 1 -- --> v1:172.21.15.43:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6b580c9090 con 0x7f6b58143c20 2023-08-09T13:55:12.637 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:12.634+0000 7f6b60245640 1 -- --> v1:172.21.15.203:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6b5806c440 con 0x7f6b580c86f0 2023-08-09T13:55:12.637 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:12.634+0000 7f6b60245640 1 -- --> v2:172.21.15.203:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6b58144000 con 0x7f6b5806cfe0 2023-08-09T13:55:12.637 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:12.635+0000 7f6b5dfe1640 1 --1- >> v1:172.21.15.43:6789/0 conn(0x7f6b58143c20 0x7f6b5806f540 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.43:6789/0 says I am v1:172.21.15.43:42220/0 (socket says 172.21.15.43:42220) 2023-08-09T13:55:12.637 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:12.635+0000 7f6b5dfe1640 1 -- 172.21.15.43:0/528568922 learned_addr learned my addr 172.21.15.43:0/528568922 (peer_addr_for_me v1:172.21.15.43:0/0) 2023-08-09T13:55:12.639 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:12.635+0000 7f6b5e7e2640 1 --2- 172.21.15.43:0/528568922 >> v2:172.21.15.203:3300/0 conn(0x7f6b5806cfe0 0x7f6b580c8b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:55:12.639 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:12.635+0000 7f6b5cfdf640 1 -- 172.21.15.43:0/528568922 <== mon.0 v1:172.21.15.43:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 847451853 0 0) 0x7f6b580c9090 con 0x7f6b58143c20 2023-08-09T13:55:12.639 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:12.635+0000 7f6b5cfdf640 1 -- 172.21.15.43:0/528568922 --> v1:172.21.15.43:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f6b44003520 con 0x7f6b58143c20 2023-08-09T13:55:12.639 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:12.636+0000 7f6b5cfdf640 1 -- 172.21.15.43:0/528568922 <== mon.0 v1:172.21.15.43:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2232605834 0 0) 0x7f6b44003520 con 0x7f6b58143c20 2023-08-09T13:55:12.639 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:12.636+0000 7f6b5cfdf640 1 -- 172.21.15.43:0/528568922 >> v2:172.21.15.203:3300/0 conn(0x7f6b5806cfe0 msgr2=0x7f6b580c8b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-09T13:55:12.639 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:12.636+0000 7f6b5cfdf640 1 --2- 172.21.15.43:0/528568922 >> v2:172.21.15.203:3300/0 conn(0x7f6b5806cfe0 0x7f6b580c8b50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:55:12.640 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:12.636+0000 7f6b5cfdf640 1 -- 172.21.15.43:0/528568922 >> v1:172.21.15.203:6789/0 conn(0x7f6b580c86f0 legacy=0x7f6b5806c8d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:55:12.640 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:12.636+0000 7f6b5cfdf640 1 -- 172.21.15.43:0/528568922 --> v1:172.21.15.43:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6b5806fc50 con 0x7f6b58143c20 2023-08-09T13:55:12.640 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:12.636+0000 7f6b5cfdf640 1 -- 172.21.15.43:0/528568922 <== mon.0 v1:172.21.15.43:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1285150740 0 0) 0x7f6b54001670 con 0x7f6b58143c20 2023-08-09T13:55:12.640 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:12.636+0000 7f6b5cfdf640 1 -- 172.21.15.43:0/528568922 >> v1:172.21.15.43:6789/0 conn(0x7f6b58143c20 legacy=0x7f6b5806f540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:55:12.640 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:12.636+0000 7f6b5cfdf640 1 --2- 172.21.15.43:0/528568922 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f6b440039e0 0x7f6b44003e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:55:12.640 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:12.636+0000 7f6b5cfdf640 1 --2- 172.21.15.43:0/528568922 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f6b440047c0 0x7f6b44004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:55:12.640 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:12.636+0000 7f6b5cfdf640 1 --2- 172.21.15.43:0/528568922 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f6b44005280 0x7f6b4400b980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:55:12.641 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:12.636+0000 7f6b5cfdf640 1 -- 172.21.15.43:0/528568922 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6b58144000 con 0x7f6b44005280 2023-08-09T13:55:12.641 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:12.636+0000 7f6b5cfdf640 1 -- 172.21.15.43:0/528568922 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6b4400c1c0 con 0x7f6b440039e0 2023-08-09T13:55:12.641 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:12.636+0000 7f6b5cfdf640 1 -- 172.21.15.43:0/528568922 --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6b4400c380 con 0x7f6b440047c0 2023-08-09T13:55:12.641 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:12.636+0000 7f6b5e7e2640 1 --2- 172.21.15.43:0/528568922 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f6b44005280 0x7f6b4400b980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:55:12.641 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:12.636+0000 7f6b5dfe1640 1 --2- 172.21.15.43:0/528568922 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f6b440039e0 0x7f6b44003e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:55:12.641 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:12.636+0000 7f6b5e7e2640 1 -- 172.21.15.43:0/528568922 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f6b440047c0 msgr2=0x7f6b44004c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-09T13:55:12.641 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:12.636+0000 7f6b5e7e2640 1 --2- 172.21.15.43:0/528568922 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f6b440047c0 0x7f6b44004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:55:12.641 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:12.636+0000 7f6b5e7e2640 1 -- 172.21.15.43:0/528568922 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f6b440039e0 msgr2=0x7f6b44003e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-09T13:55:12.642 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:12.636+0000 7f6b5e7e2640 1 --2- 172.21.15.43:0/528568922 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f6b440039e0 0x7f6b44003e00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:55:12.642 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:12.636+0000 7f6b5e7e2640 1 -- 172.21.15.43:0/528568922 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6b44003520 con 0x7f6b44005280 2023-08-09T13:55:12.643 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:12.637+0000 7f6b5e7e2640 1 -- 172.21.15.43:0/528568922 reap_dead start 2023-08-09T13:55:12.643 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:12.638+0000 7f6b5e7e2640 1 --2- 172.21.15.43:0/528568922 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f6b44005280 0x7f6b4400b980 secure :-1 s=READY pgs=1610 cs=0 l=1 rev1=1 crypto rx=0x7f6b50009fc0 tx=0x7f6b440057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3a1e0776aa80cc50 server_cookie=0 in_seq=0 out_seq=0 2023-08-09T13:55:12.643 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:12.638+0000 7f6b5cfdf640 1 -- 172.21.15.43:0/528568922 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6b5002cc30 con 0x7f6b44005280 2023-08-09T13:55:12.643 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:12.638+0000 7f6b5cfdf640 1 -- 172.21.15.43:0/528568922 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6b5000f040 con 0x7f6b44005280 2023-08-09T13:55:12.643 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:12.638+0000 7f6b5cfdf640 1 -- 172.21.15.43:0/528568922 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6b50003990 con 0x7f6b44005280 2023-08-09T13:55:12.643 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:12.638+0000 7f6b60245640 1 -- 172.21.15.43:0/528568922 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f6b44005280 msgr2=0x7f6b4400b980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:55:12.644 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:12.638+0000 7f6b60245640 1 --2- 172.21.15.43:0/528568922 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f6b44005280 0x7f6b4400b980 secure :-1 s=READY pgs=1610 cs=0 l=1 rev1=1 crypto rx=0x7f6b50009fc0 tx=0x7f6b440057f0 comp rx=0 tx=0).stop 2023-08-09T13:55:12.644 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:12.639+0000 7f6b60245640 1 -- 172.21.15.43:0/528568922 shutdown_connections 2023-08-09T13:55:12.644 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:12.639+0000 7f6b60245640 1 --2- 172.21.15.43:0/528568922 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f6b44005280 0x7f6b4400b980 unknown :-1 s=CLOSED pgs=1610 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:55:12.644 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:12.639+0000 7f6b60245640 1 -- 172.21.15.43:0/528568922 >> 172.21.15.43:0/528568922 conn(0x7f6b580ce0a0 msgr2=0x7f6b580ce480 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-09T13:55:12.644 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:12.639+0000 7f6b60245640 1 -- 172.21.15.43:0/528568922 shutdown_connections 2023-08-09T13:55:12.644 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:12.639+0000 7f6b60245640 1 -- 172.21.15.43:0/528568922 wait complete. 2023-08-09T13:55:12.644 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:12.640+0000 7f6b60245640 1 Processor -- start 2023-08-09T13:55:12.645 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:12.641+0000 7f6b60245640 1 -- start start 2023-08-09T13:55:12.646 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:12.642+0000 7f6b60245640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f6b44005280 0x7f6b58159df0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:55:12.646 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:12.642+0000 7f6b60245640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f6b580c6d50 0x7f6b580c3d70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:55:12.646 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:12.642+0000 7f6b60245640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f6b580c42b0 0x7f6b58139e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:55:12.646 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:12.642+0000 7f6b60245640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6b580c6930 con 0x7f6b44005280 2023-08-09T13:55:12.647 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:12.642+0000 7f6b60245640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6b5815a330 con 0x7f6b580c42b0 2023-08-09T13:55:12.647 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:12.642+0000 7f6b60245640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6b580c5cb0 con 0x7f6b580c6d50 2023-08-09T13:55:12.647 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:12.642+0000 7f6b5dfe1640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f6b44005280 0x7f6b58159df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:55:12.647 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:12.642+0000 7f6b5dfe1640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f6b44005280 0x7f6b58159df0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:56620/0 (socket says 172.21.15.43:56620) 2023-08-09T13:55:12.647 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:12.642+0000 7f6b5dfe1640 1 -- 172.21.15.43:0/1786845509 learned_addr learned my addr 172.21.15.43:0/1786845509 (peer_addr_for_me v2:172.21.15.43:0/0) 2023-08-09T13:55:12.647 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:12.642+0000 7f6b5e7e2640 1 --2- 172.21.15.43:0/1786845509 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f6b580c42b0 0x7f6b58139e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:55:12.647 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:12.642+0000 7f6b5dfe1640 1 -- 172.21.15.43:0/1786845509 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f6b580c6d50 msgr2=0x7f6b580c3d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:55:12.648 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:12.642+0000 7f6b5dfe1640 1 --2- 172.21.15.43:0/1786845509 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f6b580c6d50 0x7f6b580c3d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:55:12.648 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:12.642+0000 7f6b5dfe1640 1 -- 172.21.15.43:0/1786845509 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f6b580c42b0 msgr2=0x7f6b58139e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:55:12.648 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:12.642+0000 7f6b5dfe1640 1 --2- 172.21.15.43:0/1786845509 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f6b580c42b0 0x7f6b58139e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:55:12.648 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:12.642+0000 7f6b5dfe1640 1 -- 172.21.15.43:0/1786845509 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6b580c5f90 con 0x7f6b44005280 2023-08-09T13:55:12.648 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:12.643+0000 7f6b5dfe1640 1 --2- 172.21.15.43:0/1786845509 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f6b44005280 0x7f6b58159df0 secure :-1 s=READY pgs=1611 cs=0 l=1 rev1=1 crypto rx=0x7f6b5400d7a0 tx=0x7f6b5400dc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-09T13:55:12.648 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:12.643+0000 7f6b4e7fc640 1 -- 172.21.15.43:0/1786845509 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6b540074e0 con 0x7f6b44005280 2023-08-09T13:55:12.648 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:12.643+0000 7f6b4e7fc640 1 -- 172.21.15.43:0/1786845509 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6b54007640 con 0x7f6b44005280 2023-08-09T13:55:12.648 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:12.643+0000 7f6b4e7fc640 1 -- 172.21.15.43:0/1786845509 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6b54004a00 con 0x7f6b44005280 2023-08-09T13:55:12.649 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:12.643+0000 7f6b60245640 1 -- 172.21.15.43:0/1786845509 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6b580c6270 con 0x7f6b44005280 2023-08-09T13:55:12.649 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:12.643+0000 7f6b60245640 1 -- 172.21.15.43:0/1786845509 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6b580aedc0 con 0x7f6b44005280 2023-08-09T13:55:12.649 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:12.644+0000 7f6b60245640 1 -- 172.21.15.43:0/1786845509 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6b580739f0 con 0x7f6b44005280 2023-08-09T13:55:12.650 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:12.645+0000 7f6b4e7fc640 1 -- 172.21.15.43:0/1786845509 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 66) v1 ==== 88387+0+0 (secure 0 0 0) 0x7f6b54004b60 con 0x7f6b44005280 2023-08-09T13:55:12.650 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:12.646+0000 7f6b4e7fc640 1 --2- 172.21.15.43:0/1786845509 >> [v2:172.21.15.203:6832/3490266242,v1:172.21.15.203:6833/3490266242] conn(0x7f6b3c06a0e0 0x7f6b3c06c5a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:55:12.650 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:12.647+0000 7f6b4e7fc640 1 -- 172.21.15.43:0/1786845509 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(135..135 src has 1..135) v4 ==== 7286+0+0 (secure 0 0 0) 0x7f6b54088410 con 0x7f6b44005280 2023-08-09T13:55:12.650 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:12.647+0000 7f6b5d7e0640 1 --2- 172.21.15.43:0/1786845509 >> [v2:172.21.15.203:6832/3490266242,v1:172.21.15.203:6833/3490266242] conn(0x7f6b3c06a0e0 0x7f6b3c06c5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:55:12.651 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:12.648+0000 7f6b5d7e0640 1 --2- 172.21.15.43:0/1786845509 >> [v2:172.21.15.203:6832/3490266242,v1:172.21.15.203:6833/3490266242] conn(0x7f6b3c06a0e0 0x7f6b3c06c5a0 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7f6b581f57e0 tx=0x7f6b480079b0 comp rx=0 tx=0).ready entity=mgr.6659 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-09T13:55:12.656 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:12.652+0000 7f6b4e7fc640 1 -- 172.21.15.43:0/1786845509 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178066 (secure 0 0 0) 0x7f6b540535c0 con 0x7f6b44005280 2023-08-09T13:55:12.892 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:12.890+0000 7f6b60245640 1 -- 172.21.15.43:0/1786845509 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f6b580cec90 con 0x7f6b44005280 2023-08-09T13:55:12.893 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:12.891+0000 7f6b4e7fc640 1 -- 172.21.15.43:0/1786845509 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 61 v61) v1 ==== 94+0+4880 (secure 0 0 0) 0x7f6b54019020 con 0x7f6b44005280 2023-08-09T13:55:12.894 INFO:teuthology.orchestra.run.smithi043.stdout: 2023-08-09T13:55:12.894 INFO:teuthology.orchestra.run.smithi043.stdout:{"epoch":61,"default_fscid":7,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6530,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.203:6835/1659419253","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6834","nonce":1659419253},{"type":"v1","addr":"172.21.15.203:6835","nonce":1659419253}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":56},{"gid":6536,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.203:6837/3964427549","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6836","nonce":3964427549},{"type":"v1","addr":"172.21.15.203:6837","nonce":3964427549}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":56},{"gid":8574,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.43:6833/871812392","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6832","nonce":871812392},{"type":"v1","addr":"172.21.15.43:6833","nonce":871812392}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":55}],"filesystems":[{"mdsmap":{"epoch":61,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-08-09T13:53:47.747881+0000","modified":"2023-08-09T13:53:51.317185+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":8580},"failed":[],"damaged":[],"stopped":[],"info":{"gid_8580":{"gid":8580,"name":"a","rank":0,"incarnation":57,"state":"up:active","state_seq":3,"addr":"172.21.15.43:6835/1794836673","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6834","nonce":1794836673},{"type":"v1","addr":"172.21.15.43:6835","nonce":1794836673}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[17],"metadata_pool":16,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":7}]} 2023-08-09T13:55:12.894 INFO:teuthology.orchestra.run.smithi043.stderr:dumped fsmap epoch 61 2023-08-09T13:55:12.896 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:12.893+0000 7f6b60245640 1 -- 172.21.15.43:0/1786845509 >> [v2:172.21.15.203:6832/3490266242,v1:172.21.15.203:6833/3490266242] conn(0x7f6b3c06a0e0 msgr2=0x7f6b3c06c5a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:55:12.896 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:12.893+0000 7f6b60245640 1 --2- 172.21.15.43:0/1786845509 >> [v2:172.21.15.203:6832/3490266242,v1:172.21.15.203:6833/3490266242] conn(0x7f6b3c06a0e0 0x7f6b3c06c5a0 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7f6b581f57e0 tx=0x7f6b480079b0 comp rx=0 tx=0).stop 2023-08-09T13:55:12.897 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:12.894+0000 7f6b60245640 1 -- 172.21.15.43:0/1786845509 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f6b44005280 msgr2=0x7f6b58159df0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:55:12.897 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:12.894+0000 7f6b60245640 1 --2- 172.21.15.43:0/1786845509 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f6b44005280 0x7f6b58159df0 secure :-1 s=READY pgs=1611 cs=0 l=1 rev1=1 crypto rx=0x7f6b5400d7a0 tx=0x7f6b5400dc60 comp rx=0 tx=0).stop 2023-08-09T13:55:12.897 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:12.894+0000 7f6b60245640 1 -- 172.21.15.43:0/1786845509 shutdown_connections 2023-08-09T13:55:12.897 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:12.894+0000 7f6b60245640 1 --2- 172.21.15.43:0/1786845509 >> [v2:172.21.15.203:6832/3490266242,v1:172.21.15.203:6833/3490266242] conn(0x7f6b3c06a0e0 0x7f6b3c06c5a0 unknown :-1 s=CLOSED pgs=28 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:55:12.897 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:12.894+0000 7f6b60245640 1 --2- 172.21.15.43:0/1786845509 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f6b580c42b0 0x7f6b58139e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:55:12.897 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:12.894+0000 7f6b60245640 1 --2- 172.21.15.43:0/1786845509 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f6b580c6d50 0x7f6b580c3d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:55:12.897 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:12.894+0000 7f6b60245640 1 --2- 172.21.15.43:0/1786845509 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f6b44005280 0x7f6b58159df0 unknown :-1 s=CLOSED pgs=1611 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:55:12.897 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:12.894+0000 7f6b60245640 1 -- 172.21.15.43:0/1786845509 >> 172.21.15.43:0/1786845509 conn(0x7f6b580ce0a0 msgr2=0x7f6b5815baf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-09T13:55:12.898 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:12.894+0000 7f6b60245640 1 -- 172.21.15.43:0/1786845509 shutdown_connections 2023-08-09T13:55:12.898 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:12.894+0000 7f6b60245640 1 -- 172.21.15.43:0/1786845509 wait complete. 2023-08-09T13:55:12.916 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-08-09T13:55:13.144 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:13.142+0000 7f1c9625d640 1 Processor -- start 2023-08-09T13:55:13.145 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:13.142+0000 7f1c9625d640 1 -- start start 2023-08-09T13:55:13.146 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:13.144+0000 7f1c9625d640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f1c900cd6b0 0x7f1c900b4760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:55:13.146 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:13.144+0000 7f1c9625d640 1 -- --> v1:172.21.15.43:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1c900b7bf0 con 0x7f1c9015f7b0 2023-08-09T13:55:13.146 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:13.144+0000 7f1c9625d640 1 -- --> v1:172.21.15.203:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1c900b7ed0 con 0x7f1c900b1050 2023-08-09T13:55:13.147 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:13.144+0000 7f1c9625d640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1c900d0370 con 0x7f1c900cd6b0 2023-08-09T13:55:13.147 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:13.144+0000 7f1c9525b640 1 --1- >> v1:172.21.15.43:6789/0 conn(0x7f1c9015f7b0 0x7f1c900ccfa0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.43:6789/0 says I am v1:172.21.15.43:42228/0 (socket says 172.21.15.43:42228) 2023-08-09T13:55:13.147 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:13.144+0000 7f1c9525b640 1 -- 172.21.15.43:0/4036225855 learned_addr learned my addr 172.21.15.43:0/4036225855 (peer_addr_for_me v1:172.21.15.43:0/0) 2023-08-09T13:55:13.148 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:13.144+0000 7f1c95a5c640 1 --2- 172.21.15.43:0/4036225855 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f1c900cd6b0 0x7f1c900b4760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:55:13.149 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:13.144+0000 7f1c87fff640 1 -- 172.21.15.43:0/4036225855 <== mon.0 v1:172.21.15.43:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2871875333 0 0) 0x7f1c900b7bf0 con 0x7f1c9015f7b0 2023-08-09T13:55:13.149 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:13.144+0000 7f1c87fff640 1 -- 172.21.15.43:0/4036225855 --> v1:172.21.15.43:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f1c7c003520 con 0x7f1c9015f7b0 2023-08-09T13:55:13.149 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:13.145+0000 7f1c87fff640 1 -- 172.21.15.43:0/4036225855 <== mon.0 v1:172.21.15.43:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 4272877775 0 0) 0x7f1c7c003520 con 0x7f1c9015f7b0 2023-08-09T13:55:13.149 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:13.145+0000 7f1c87fff640 1 -- 172.21.15.43:0/4036225855 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f1c900cd6b0 msgr2=0x7f1c900b4760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-09T13:55:13.149 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:13.145+0000 7f1c87fff640 1 --2- 172.21.15.43:0/4036225855 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f1c900cd6b0 0x7f1c900b4760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:55:13.149 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:13.145+0000 7f1c87fff640 1 -- 172.21.15.43:0/4036225855 >> v1:172.21.15.203:6789/0 conn(0x7f1c900b1050 legacy=0x7f1c900d0e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:55:13.149 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:13.145+0000 7f1c87fff640 1 -- 172.21.15.43:0/4036225855 --> v1:172.21.15.43:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1c900d0650 con 0x7f1c9015f7b0 2023-08-09T13:55:13.150 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:13.145+0000 7f1c87fff640 1 -- 172.21.15.43:0/4036225855 <== mon.0 v1:172.21.15.43:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1285150740 0 0) 0x7f1c8c001670 con 0x7f1c9015f7b0 2023-08-09T13:55:13.150 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:13.145+0000 7f1c87fff640 1 -- 172.21.15.43:0/4036225855 >> v1:172.21.15.43:6789/0 conn(0x7f1c9015f7b0 legacy=0x7f1c900ccfa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:55:13.150 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:13.145+0000 7f1c87fff640 1 --2- 172.21.15.43:0/4036225855 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f1c900cd6b0 0x7f1c7c0039a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:55:13.150 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:13.145+0000 7f1c87fff640 1 --2- 172.21.15.43:0/4036225855 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f1c7c003f20 0x7f1c7c0047c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:55:13.150 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:13.145+0000 7f1c87fff640 1 --2- 172.21.15.43:0/4036225855 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f1c7c004d00 0x7f1c7c005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:55:13.150 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:13.145+0000 7f1c87fff640 1 -- 172.21.15.43:0/4036225855 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1c900d0370 con 0x7f1c7c003f20 2023-08-09T13:55:13.150 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:13.145+0000 7f1c87fff640 1 -- 172.21.15.43:0/4036225855 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1c7c00ba60 con 0x7f1c7c004d00 2023-08-09T13:55:13.151 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:13.145+0000 7f1c87fff640 1 -- 172.21.15.43:0/4036225855 --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1c7c00bc20 con 0x7f1c900cd6b0 2023-08-09T13:55:13.151 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:13.145+0000 7f1c94a5a640 1 --2- 172.21.15.43:0/4036225855 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f1c7c003f20 0x7f1c7c0047c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:55:13.151 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:13.146+0000 7f1c9525b640 1 --2- 172.21.15.43:0/4036225855 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f1c900cd6b0 0x7f1c7c0039a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:55:13.151 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:13.146+0000 7f1c95a5c640 1 --2- 172.21.15.43:0/4036225855 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f1c7c004d00 0x7f1c7c005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:55:13.151 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:13.146+0000 7f1c94a5a640 1 -- 172.21.15.43:0/4036225855 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f1c900cd6b0 msgr2=0x7f1c7c0039a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-09T13:55:13.151 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:13.146+0000 7f1c94a5a640 1 --2- 172.21.15.43:0/4036225855 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f1c900cd6b0 0x7f1c7c0039a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:55:13.151 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:13.146+0000 7f1c94a5a640 1 -- 172.21.15.43:0/4036225855 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f1c7c004d00 msgr2=0x7f1c7c005180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-09T13:55:13.152 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:13.146+0000 7f1c94a5a640 1 --2- 172.21.15.43:0/4036225855 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f1c7c004d00 0x7f1c7c005180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:55:13.152 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:13.146+0000 7f1c94a5a640 1 -- 172.21.15.43:0/4036225855 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1c7c003520 con 0x7f1c7c003f20 2023-08-09T13:55:13.152 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:13.146+0000 7f1c95a5c640 1 --2- 172.21.15.43:0/4036225855 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f1c7c004d00 0x7f1c7c005180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-08-09T13:55:13.153 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:13.147+0000 7f1c94a5a640 1 --2- 172.21.15.43:0/4036225855 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f1c7c003f20 0x7f1c7c0047c0 secure :-1 s=READY pgs=1613 cs=0 l=1 rev1=1 crypto rx=0x7f1c80001720 tx=0x7f1c7c0057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fc15727457c887e3 server_cookie=0 in_seq=0 out_seq=0 2023-08-09T13:55:13.153 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:13.147+0000 7f1c87fff640 1 -- 172.21.15.43:0/4036225855 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1c800052d0 con 0x7f1c7c003f20 2023-08-09T13:55:13.153 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:13.147+0000 7f1c87fff640 1 -- 172.21.15.43:0/4036225855 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1c8002e050 con 0x7f1c7c003f20 2023-08-09T13:55:13.153 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:13.147+0000 7f1c87fff640 1 -- 172.21.15.43:0/4036225855 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1c80003830 con 0x7f1c7c003f20 2023-08-09T13:55:13.153 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:13.148+0000 7f1c9625d640 1 -- 172.21.15.43:0/4036225855 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f1c7c003f20 msgr2=0x7f1c7c0047c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:55:13.154 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:13.148+0000 7f1c9625d640 1 --2- 172.21.15.43:0/4036225855 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f1c7c003f20 0x7f1c7c0047c0 secure :-1 s=READY pgs=1613 cs=0 l=1 rev1=1 crypto rx=0x7f1c80001720 tx=0x7f1c7c0057f0 comp rx=0 tx=0).stop 2023-08-09T13:55:13.154 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:13.148+0000 7f1c95a5c640 1 -- 172.21.15.43:0/4036225855 reap_dead start 2023-08-09T13:55:13.154 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:13.148+0000 7f1c9625d640 1 -- 172.21.15.43:0/4036225855 shutdown_connections 2023-08-09T13:55:13.154 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:13.148+0000 7f1c9625d640 1 -- 172.21.15.43:0/4036225855 >> 172.21.15.43:0/4036225855 conn(0x7f1c9006a2b0 msgr2=0x7f1c9006a690 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-09T13:55:13.154 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:13.148+0000 7f1c9625d640 1 -- 172.21.15.43:0/4036225855 shutdown_connections 2023-08-09T13:55:13.154 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:13.148+0000 7f1c9625d640 1 -- 172.21.15.43:0/4036225855 wait complete. 2023-08-09T13:55:13.154 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:13.150+0000 7f1c9625d640 1 Processor -- start 2023-08-09T13:55:13.154 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:13.150+0000 7f1c9625d640 1 -- start start 2023-08-09T13:55:13.156 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:13.151+0000 7f1c9625d640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f1c7c003f20 0x7f1c901572b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:55:13.156 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:13.151+0000 7f1c9625d640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f1c9014b830 0x7f1c9014aa10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:55:13.156 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:13.151+0000 7f1c9625d640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f1c900cc8b0 0x7f1c900ccd60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:55:13.156 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:13.151+0000 7f1c9625d640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1c900cd2a0 con 0x7f1c900cc8b0 2023-08-09T13:55:13.157 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:13.151+0000 7f1c9625d640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1c90159a10 con 0x7f1c9014b830 2023-08-09T13:55:13.157 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:13.151+0000 7f1c9625d640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1c901577f0 con 0x7f1c7c003f20 2023-08-09T13:55:13.157 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:13.152+0000 7f1c95a5c640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f1c900cc8b0 0x7f1c900ccd60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:55:13.157 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:13.152+0000 7f1c95a5c640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f1c900cc8b0 0x7f1c900ccd60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:56644/0 (socket says 172.21.15.43:56644) 2023-08-09T13:55:13.157 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:13.152+0000 7f1c95a5c640 1 -- 172.21.15.43:0/1137121187 learned_addr learned my addr 172.21.15.43:0/1137121187 (peer_addr_for_me v2:172.21.15.43:0/0) 2023-08-09T13:55:13.157 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:13.152+0000 7f1c9525b640 1 --2- 172.21.15.43:0/1137121187 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f1c7c003f20 0x7f1c901572b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:55:13.157 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:13.152+0000 7f1c94a5a640 1 --2- 172.21.15.43:0/1137121187 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f1c9014b830 0x7f1c9014aa10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:55:13.157 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:13.152+0000 7f1c95a5c640 1 -- 172.21.15.43:0/1137121187 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f1c7c003f20 msgr2=0x7f1c901572b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:55:13.158 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:13.152+0000 7f1c95a5c640 1 --2- 172.21.15.43:0/1137121187 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f1c7c003f20 0x7f1c901572b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:55:13.158 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:13.152+0000 7f1c95a5c640 1 -- 172.21.15.43:0/1137121187 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f1c9014b830 msgr2=0x7f1c9014aa10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:55:13.158 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:13.152+0000 7f1c95a5c640 1 --2- 172.21.15.43:0/1137121187 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f1c9014b830 0x7f1c9014aa10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:55:13.158 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:13.152+0000 7f1c95a5c640 1 -- 172.21.15.43:0/1137121187 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1c90157a90 con 0x7f1c900cc8b0 2023-08-09T13:55:13.158 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:13.152+0000 7f1c94a5a640 1 --2- 172.21.15.43:0/1137121187 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f1c9014b830 0x7f1c9014aa10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-08-09T13:55:13.158 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:13.152+0000 7f1c95a5c640 1 --2- 172.21.15.43:0/1137121187 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f1c900cc8b0 0x7f1c900ccd60 secure :-1 s=READY pgs=1614 cs=0 l=1 rev1=1 crypto rx=0x7f1c9006af30 tx=0x7f1c8800f4a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-09T13:55:13.158 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:13.153+0000 7f1c85ffb640 1 -- 172.21.15.43:0/1137121187 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1c88017400 con 0x7f1c900cc8b0 2023-08-09T13:55:13.159 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:13.153+0000 7f1c85ffb640 1 -- 172.21.15.43:0/1137121187 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1c88011040 con 0x7f1c900cc8b0 2023-08-09T13:55:13.159 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:13.153+0000 7f1c85ffb640 1 -- 172.21.15.43:0/1137121187 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1c880177c0 con 0x7f1c900cc8b0 2023-08-09T13:55:13.159 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:13.153+0000 7f1c9625d640 1 -- 172.21.15.43:0/1137121187 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1c90157d70 con 0x7f1c900cc8b0 2023-08-09T13:55:13.159 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:13.153+0000 7f1c9625d640 1 -- 172.21.15.43:0/1137121187 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1c90158330 con 0x7f1c900cc8b0 2023-08-09T13:55:13.160 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:13.154+0000 7f1c9625d640 1 -- 172.21.15.43:0/1137121187 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1c900739f0 con 0x7f1c900cc8b0 2023-08-09T13:55:13.160 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:13.155+0000 7f1c85ffb640 1 -- 172.21.15.43:0/1137121187 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 66) v1 ==== 88387+0+0 (secure 0 0 0) 0x7f1c880203b0 con 0x7f1c900cc8b0 2023-08-09T13:55:13.160 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:13.156+0000 7f1c85ffb640 1 --2- 172.21.15.43:0/1137121187 >> [v2:172.21.15.203:6832/3490266242,v1:172.21.15.203:6833/3490266242] conn(0x7f1c7406a110 0x7f1c7406c5d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:55:13.160 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:13.156+0000 7f1c85ffb640 1 -- 172.21.15.43:0/1137121187 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(135..135 src has 1..135) v4 ==== 7286+0+0 (secure 0 0 0) 0x7f1c88088f70 con 0x7f1c900cc8b0 2023-08-09T13:55:13.160 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:13.157+0000 7f1c9525b640 1 --2- 172.21.15.43:0/1137121187 >> [v2:172.21.15.203:6832/3490266242,v1:172.21.15.203:6833/3490266242] conn(0x7f1c7406a110 0x7f1c7406c5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:55:13.161 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:13.158+0000 7f1c9525b640 1 --2- 172.21.15.43:0/1137121187 >> [v2:172.21.15.203:6832/3490266242,v1:172.21.15.203:6833/3490266242] conn(0x7f1c7406a110 0x7f1c7406c5d0 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7f1c8c001c90 tx=0x7f1c8c009040 comp rx=0 tx=0).ready entity=mgr.6659 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-09T13:55:13.163 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:13.161+0000 7f1c85ffb640 1 -- 172.21.15.43:0/1137121187 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178066 (secure 0 0 0) 0x7f1c88054020 con 0x7f1c900cc8b0 2023-08-09T13:55:13.357 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:13.354+0000 7f1c9625d640 1 -- 172.21.15.43:0/1137121187 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f1c900d28d0 con 0x7f1c900cc8b0 2023-08-09T13:55:13.359 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:13.357+0000 7f1c85ffb640 1 -- 172.21.15.43:0/1137121187 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v135) v1 ==== 74+0+19818 (secure 0 0 0) 0x7f1c88057640 con 0x7f1c900cc8b0 2023-08-09T13:55:13.359 INFO:teuthology.orchestra.run.smithi043.stdout: 2023-08-09T13:55:13.360 INFO:teuthology.orchestra.run.smithi043.stdout:{"epoch":135,"fsid":"daac87ee-1337-4250-a955-aea4e408a39b","created":"2023-08-09T13:33:41.701198+0000","modified":"2023-08-09T13:55:12.209406+0000","last_up_change":"2023-08-09T13:33:53.783453+0000","last_in_change":"2023-08-09T13:33:45.148904+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":17,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":"rbd","create_time":"2023-08-09T13:33:54.365137+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":".mgr","create_time":"2023-08-09T13:33:54.831803+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":16,"pool_name":"cephfs_metadata","create_time":"2023-08-09T13:53:46.124616+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"128","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":17,"pool_name":"cephfs_data","create_time":"2023-08-09T13:53:46.762733+0000","flags":8193,"flags_names":"hashpspool,selfmanaged_snaps","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"133","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":3,"snap_epoch":133,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"cbd49ecd-e896-4fbd-864e-5c37a829b165","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":127,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6800","nonce":502667335},{"type":"v1","addr":"172.21.15.43:6801","nonce":502667335}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6802","nonce":502667335},{"type":"v1","addr":"172.21.15.43:6803","nonce":502667335}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6806","nonce":502667335},{"type":"v1","addr":"172.21.15.43:6807","nonce":502667335}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6804","nonce":502667335},{"type":"v1","addr":"172.21.15.43:6805","nonce":502667335}]},"public_addr":"172.21.15.43:6801/502667335","cluster_addr":"172.21.15.43:6803/502667335","heartbeat_back_addr":"172.21.15.43:6807/502667335","heartbeat_front_addr":"172.21.15.43:6805/502667335","state":["exists","up"]},{"osd":1,"uuid":"75c7e0fe-1ec7-4fe8-a2f2-3661a65b8500","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":131,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6808","nonce":3289134610},{"type":"v1","addr":"172.21.15.43:6809","nonce":3289134610}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6810","nonce":3289134610},{"type":"v1","addr":"172.21.15.43:6812","nonce":3289134610}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6821","nonce":3289134610},{"type":"v1","addr":"172.21.15.43:6824","nonce":3289134610}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6815","nonce":3289134610},{"type":"v1","addr":"172.21.15.43:6818","nonce":3289134610}]},"public_addr":"172.21.15.43:6809/3289134610","cluster_addr":"172.21.15.43:6812/3289134610","heartbeat_back_addr":"172.21.15.43:6824/3289134610","heartbeat_front_addr":"172.21.15.43:6818/3289134610","state":["exists","up"]},{"osd":2,"uuid":"bb5c41e9-7f5f-4125-9b01-3bb8bfc2ea83","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":127,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6811","nonce":4079709736},{"type":"v1","addr":"172.21.15.43:6814","nonce":4079709736}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6817","nonce":4079709736},{"type":"v1","addr":"172.21.15.43:6820","nonce":4079709736}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6829","nonce":4079709736},{"type":"v1","addr":"172.21.15.43:6831","nonce":4079709736}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6823","nonce":4079709736},{"type":"v1","addr":"172.21.15.43:6826","nonce":4079709736}]},"public_addr":"172.21.15.43:6814/4079709736","cluster_addr":"172.21.15.43:6820/4079709736","heartbeat_back_addr":"172.21.15.43:6831/4079709736","heartbeat_front_addr":"172.21.15.43:6826/4079709736","state":["exists","up"]},{"osd":3,"uuid":"bfe4c8b4-694a-465c-87e6-2b2aab852613","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":127,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6813","nonce":1709538759},{"type":"v1","addr":"172.21.15.43:6816","nonce":1709538759}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6819","nonce":1709538759},{"type":"v1","addr":"172.21.15.43:6822","nonce":1709538759}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6828","nonce":1709538759},{"type":"v1","addr":"172.21.15.43:6830","nonce":1709538759}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6825","nonce":1709538759},{"type":"v1","addr":"172.21.15.43:6827","nonce":1709538759}]},"public_addr":"172.21.15.43:6816/1709538759","cluster_addr":"172.21.15.43:6822/1709538759","heartbeat_back_addr":"172.21.15.43:6830/1709538759","heartbeat_front_addr":"172.21.15.43:6827/1709538759","state":["exists","up"]},{"osd":4,"uuid":"fa2e8a77-aba1-4756-853e-0d174a5bdeba","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":127,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6800","nonce":4196866481},{"type":"v1","addr":"172.21.15.203:6801","nonce":4196866481}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6802","nonce":4196866481},{"type":"v1","addr":"172.21.15.203:6803","nonce":4196866481}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6806","nonce":4196866481},{"type":"v1","addr":"172.21.15.203:6807","nonce":4196866481}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6804","nonce":4196866481},{"type":"v1","addr":"172.21.15.203:6805","nonce":4196866481}]},"public_addr":"172.21.15.203:6801/4196866481","cluster_addr":"172.21.15.203:6803/4196866481","heartbeat_back_addr":"172.21.15.203:6807/4196866481","heartbeat_front_addr":"172.21.15.203:6805/4196866481","state":["exists","up"]},{"osd":5,"uuid":"81811e4b-4d24-4f51-a82c-f1d700a2a00d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":127,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6816","nonce":3059290081},{"type":"v1","addr":"172.21.15.203:6817","nonce":3059290081}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6818","nonce":3059290081},{"type":"v1","addr":"172.21.15.203:6819","nonce":3059290081}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6822","nonce":3059290081},{"type":"v1","addr":"172.21.15.203:6823","nonce":3059290081}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6820","nonce":3059290081},{"type":"v1","addr":"172.21.15.203:6821","nonce":3059290081}]},"public_addr":"172.21.15.203:6817/3059290081","cluster_addr":"172.21.15.203:6819/3059290081","heartbeat_back_addr":"172.21.15.203:6823/3059290081","heartbeat_front_addr":"172.21.15.203:6821/3059290081","state":["exists","up"]},{"osd":6,"uuid":"307bac4d-6a5b-443d-af75-56d695d7674f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":131,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6824","nonce":512155569},{"type":"v1","addr":"172.21.15.203:6825","nonce":512155569}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6826","nonce":512155569},{"type":"v1","addr":"172.21.15.203:6827","nonce":512155569}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6830","nonce":512155569},{"type":"v1","addr":"172.21.15.203:6831","nonce":512155569}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6828","nonce":512155569},{"type":"v1","addr":"172.21.15.203:6829","nonce":512155569}]},"public_addr":"172.21.15.203:6825/512155569","cluster_addr":"172.21.15.203:6827/512155569","heartbeat_back_addr":"172.21.15.203:6831/512155569","heartbeat_front_addr":"172.21.15.203:6829/512155569","state":["exists","up"]},{"osd":7,"uuid":"a95dc7ca-9f1b-4295-af70-68a22a2d7d5e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":127,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6808","nonce":3331450438},{"type":"v1","addr":"172.21.15.203:6809","nonce":3331450438}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6810","nonce":3331450438},{"type":"v1","addr":"172.21.15.203:6811","nonce":3331450438}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6814","nonce":3331450438},{"type":"v1","addr":"172.21.15.203:6815","nonce":3331450438}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6812","nonce":3331450438},{"type":"v1","addr":"172.21.15.203:6813","nonce":3331450438}]},"public_addr":"172.21.15.203:6809/3331450438","cluster_addr":"172.21.15.203:6811/3331450438","heartbeat_back_addr":"172.21.15.203:6815/3331450438","heartbeat_front_addr":"172.21.15.203:6813/3331450438","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-09T13:33:50.547652+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-09T13:33:50.589469+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-09T13:33:50.561724+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-09T13:33:50.599650+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-09T13:33:50.411855+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-09T13:33:50.219681+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-09T13:33:50.504664+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-09T13:33:50.441248+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"17.5","mappings":[{"from":0,"to":5}]},{"pgid":"17.f","mappings":[{"from":0,"to":2}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.203:0/1151006972":"2023-08-10T13:55:08.528494+0000","172.21.15.203:0/274636031":"2023-08-10T13:55:08.528494+0000","172.21.15.203:0/3526078924":"2023-08-10T13:55:08.528494+0000","172.21.15.203:6833/1796635339":"2023-08-10T13:55:08.528494+0000","172.21.15.203:6832/1796635339":"2023-08-10T13:55:08.528494+0000","172.21.15.203:0/1153043111":"2023-08-10T13:55:08.528494+0000","172.21.15.203:0/3657978600":"2023-08-10T13:55:08.528494+0000","172.21.15.203:0/3093730384":"2023-08-10T13:55:08.528494+0000","172.21.15.203:6833/2527059680":"2023-08-10T13:54:03.238790+0000","172.21.15.203:6832/2527059680":"2023-08-10T13:54:03.238790+0000","172.21.15.203:0/431304609":"2023-08-10T13:54:03.238790+0000","172.21.15.203:0/1911913635":"2023-08-10T13:55:08.528494+0000","172.21.15.203:0/826469783":"2023-08-10T13:54:03.238790+0000","172.21.15.203:0/1225835978":"2023-08-10T13:55:08.528494+0000","172.21.15.203:0/1456953448":"2023-08-10T13:54:03.238790+0000","172.21.15.203:0/2004944696":"2023-08-10T13:54:03.238790+0000","172.21.15.203:0/916024371":"2023-08-10T13:54:03.238790+0000","172.21.15.203:0/4209557762":"2023-08-10T13:54:03.238790+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":7,"snaps":[{"begin":104,"length":1},{"begin":221,"length":1}]}],"new_removed_snaps":[],"new_purged_snaps":[{"pool":17,"snaps":[{"begin":2,"length":2}]}],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-08-09T13:55:13.362 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:13.359+0000 7f1c9625d640 1 -- 172.21.15.43:0/1137121187 >> [v2:172.21.15.203:6832/3490266242,v1:172.21.15.203:6833/3490266242] conn(0x7f1c7406a110 msgr2=0x7f1c7406c5d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:55:13.362 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:13.359+0000 7f1c9625d640 1 --2- 172.21.15.43:0/1137121187 >> [v2:172.21.15.203:6832/3490266242,v1:172.21.15.203:6833/3490266242] conn(0x7f1c7406a110 0x7f1c7406c5d0 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7f1c8c001c90 tx=0x7f1c8c009040 comp rx=0 tx=0).stop 2023-08-09T13:55:13.362 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:13.359+0000 7f1c9625d640 1 -- 172.21.15.43:0/1137121187 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f1c900cc8b0 msgr2=0x7f1c900ccd60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:55:13.363 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:13.359+0000 7f1c9625d640 1 --2- 172.21.15.43:0/1137121187 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f1c900cc8b0 0x7f1c900ccd60 secure :-1 s=READY pgs=1614 cs=0 l=1 rev1=1 crypto rx=0x7f1c9006af30 tx=0x7f1c8800f4a0 comp rx=0 tx=0).stop 2023-08-09T13:55:13.363 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:13.359+0000 7f1c9625d640 1 -- 172.21.15.43:0/1137121187 shutdown_connections 2023-08-09T13:55:13.363 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:13.359+0000 7f1c9625d640 1 --2- 172.21.15.43:0/1137121187 >> [v2:172.21.15.203:6832/3490266242,v1:172.21.15.203:6833/3490266242] conn(0x7f1c7406a110 0x7f1c7406c5d0 unknown :-1 s=CLOSED pgs=29 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:55:13.363 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:13.359+0000 7f1c9625d640 1 --2- 172.21.15.43:0/1137121187 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f1c900cc8b0 0x7f1c900ccd60 unknown :-1 s=CLOSED pgs=1614 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:55:13.363 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:13.359+0000 7f1c9625d640 1 --2- 172.21.15.43:0/1137121187 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f1c9014b830 0x7f1c9014aa10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:55:13.363 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:13.359+0000 7f1c9625d640 1 --2- 172.21.15.43:0/1137121187 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f1c7c003f20 0x7f1c901572b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:55:13.364 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:13.359+0000 7f1c9625d640 1 -- 172.21.15.43:0/1137121187 >> 172.21.15.43:0/1137121187 conn(0x7f1c9006a2b0 msgr2=0x7f1c900b4550 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-09T13:55:13.364 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:13.360+0000 7f1c9625d640 1 -- 172.21.15.43:0/1137121187 shutdown_connections 2023-08-09T13:55:13.364 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:13.360+0000 7f1c9625d640 1 -- 172.21.15.43:0/1137121187 wait complete. 2023-08-09T13:55:13.381 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2023-08-09T13:55:13.595 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:13.592+0000 7f3e62d01640 1 Processor -- start 2023-08-09T13:55:13.595 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:13.592+0000 7f3e62d01640 1 -- start start 2023-08-09T13:55:13.596 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:13.594+0000 7f3e62d01640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f3e5c142ca0 0x7f3e5c0c96e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:55:13.596 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:13.594+0000 7f3e62d01640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f3e5c0cd510 0x7f3e5c0c3290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:55:13.596 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:13.594+0000 7f3e62d01640 1 -- --> v1:172.21.15.203:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3e5c143080 con 0x7f3e5c15ce40 2023-08-09T13:55:13.597 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:13.594+0000 7f3e62d01640 1 -- --> v2:172.21.15.43:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3e5c06d400 con 0x7f3e5c142ca0 2023-08-09T13:55:13.597 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:13.594+0000 7f3e62d01640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3e5c0c9d90 con 0x7f3e5c0cd510 2023-08-09T13:55:13.597 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:13.594+0000 7f3e60a9d640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f3e5c142ca0 0x7f3e5c0c96e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:55:13.597 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:13.594+0000 7f3e60a9d640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f3e5c142ca0 0x7f3e5c0c96e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:56652/0 (socket says 172.21.15.43:56652) 2023-08-09T13:55:13.598 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:13.594+0000 7f3e60a9d640 1 -- 172.21.15.43:0/3936754755 learned_addr learned my addr 172.21.15.43:0/3936754755 (peer_addr_for_me v2:172.21.15.43:0/0) 2023-08-09T13:55:13.598 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:13.594+0000 7f3e60a9d640 1 -- 172.21.15.43:0/3936754755 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f3e5c0cd510 msgr2=0x7f3e5c0c3290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-09T13:55:13.598 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:13.594+0000 7f3e60a9d640 1 --2- 172.21.15.43:0/3936754755 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f3e5c0cd510 0x7f3e5c0c3290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:55:13.598 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:13.594+0000 7f3e60a9d640 1 -- 172.21.15.43:0/3936754755 >> v1:172.21.15.203:6789/0 conn(0x7f3e5c15ce40 legacy=0x7f3e5c0cce00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-09T13:55:13.598 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:13.594+0000 7f3e60a9d640 1 -- 172.21.15.43:0/3936754755 --> v2:172.21.15.43:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3e5c0ca070 con 0x7f3e5c142ca0 2023-08-09T13:55:13.599 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:13.596+0000 7f3e60a9d640 1 --2- 172.21.15.43:0/3936754755 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f3e5c142ca0 0x7f3e5c0c96e0 secure :-1 s=READY pgs=1615 cs=0 l=1 rev1=1 crypto rx=0x7f3e5000bb10 tx=0x7f3e5002e0c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6d337f2e6af94733 server_cookie=0 in_seq=0 out_seq=0 2023-08-09T13:55:13.599 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:13.596+0000 7f3e5b7fe640 1 -- 172.21.15.43:0/3936754755 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3e5002ee70 con 0x7f3e5c142ca0 2023-08-09T13:55:13.599 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:13.596+0000 7f3e5b7fe640 1 -- 172.21.15.43:0/3936754755 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3e50036440 con 0x7f3e5c142ca0 2023-08-09T13:55:13.600 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:13.596+0000 7f3e5b7fe640 1 -- 172.21.15.43:0/3936754755 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3e500367d0 con 0x7f3e5c142ca0 2023-08-09T13:55:13.600 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:13.596+0000 7f3e62d01640 1 -- 172.21.15.43:0/3936754755 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f3e5c142ca0 msgr2=0x7f3e5c0c96e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:55:13.600 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:13.596+0000 7f3e62d01640 1 --2- 172.21.15.43:0/3936754755 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f3e5c142ca0 0x7f3e5c0c96e0 secure :-1 s=READY pgs=1615 cs=0 l=1 rev1=1 crypto rx=0x7f3e5000bb10 tx=0x7f3e5002e0c0 comp rx=0 tx=0).stop 2023-08-09T13:55:13.600 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:13.597+0000 7f3e62d01640 1 -- 172.21.15.43:0/3936754755 shutdown_connections 2023-08-09T13:55:13.600 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:13.597+0000 7f3e62d01640 1 --2- 172.21.15.43:0/3936754755 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f3e5c0cd510 0x7f3e5c0c3290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:55:13.600 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:13.597+0000 7f3e62d01640 1 --2- 172.21.15.43:0/3936754755 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f3e5c142ca0 0x7f3e5c0c96e0 unknown :-1 s=CLOSED pgs=1615 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:55:13.600 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:13.597+0000 7f3e62d01640 1 -- 172.21.15.43:0/3936754755 >> 172.21.15.43:0/3936754755 conn(0x7f3e5c0ce0a0 msgr2=0x7f3e5c0ce480 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-09T13:55:13.601 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:13.597+0000 7f3e62d01640 1 -- 172.21.15.43:0/3936754755 shutdown_connections 2023-08-09T13:55:13.601 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:13.597+0000 7f3e62d01640 1 -- 172.21.15.43:0/3936754755 wait complete. 2023-08-09T13:55:13.601 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:13.599+0000 7f3e62d01640 1 Processor -- start 2023-08-09T13:55:13.601 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:13.599+0000 7f3e62d01640 1 -- start start 2023-08-09T13:55:13.602 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:13.600+0000 7f3e62d01640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f3e5c0cd510 0x7f3e5c1481a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:55:13.602 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:13.600+0000 7f3e62d01640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f3e5c142ca0 0x7f3e5c15c520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:55:13.602 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:13.600+0000 7f3e62d01640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f3e5c15ce40 0x7f3e5c13aa90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:55:13.602 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:13.600+0000 7f3e62d01640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3e5c15c100 con 0x7f3e5c0cd510 2023-08-09T13:55:13.602 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:13.600+0000 7f3e62d01640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3e5c1486e0 con 0x7f3e5c142ca0 2023-08-09T13:55:13.603 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:13.600+0000 7f3e62d01640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3e5c15b480 con 0x7f3e5c15ce40 2023-08-09T13:55:13.604 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:13.600+0000 7f3e60a9d640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f3e5c0cd510 0x7f3e5c1481a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:55:13.604 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:13.600+0000 7f3e60a9d640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f3e5c0cd510 0x7f3e5c1481a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:56656/0 (socket says 172.21.15.43:56656) 2023-08-09T13:55:13.604 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:13.600+0000 7f3e60a9d640 1 -- 172.21.15.43:0/3742947073 learned_addr learned my addr 172.21.15.43:0/3742947073 (peer_addr_for_me v2:172.21.15.43:0/0) 2023-08-09T13:55:13.604 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:13.600+0000 7f3e60a9d640 1 -- 172.21.15.43:0/3742947073 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f3e5c15ce40 msgr2=0x7f3e5c13aa90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:55:13.604 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:13.600+0000 7f3e60a9d640 1 --2- 172.21.15.43:0/3742947073 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f3e5c15ce40 0x7f3e5c13aa90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:55:13.604 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:13.600+0000 7f3e60a9d640 1 -- 172.21.15.43:0/3742947073 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f3e5c142ca0 msgr2=0x7f3e5c15c520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:55:13.605 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:13.600+0000 7f3e5bfff640 1 --2- 172.21.15.43:0/3742947073 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f3e5c142ca0 0x7f3e5c15c520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:55:13.605 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:13.600+0000 7f3e60a9d640 1 --2- 172.21.15.43:0/3742947073 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f3e5c142ca0 0x7f3e5c15c520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:55:13.605 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:13.600+0000 7f3e60a9d640 1 -- 172.21.15.43:0/3742947073 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3e5000b770 con 0x7f3e5c0cd510 2023-08-09T13:55:13.605 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:13.601+0000 7f3e5bfff640 1 --2- 172.21.15.43:0/3742947073 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f3e5c142ca0 0x7f3e5c15c520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-08-09T13:55:13.605 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:13.601+0000 7f3e60a9d640 1 --2- 172.21.15.43:0/3742947073 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f3e5c0cd510 0x7f3e5c1481a0 secure :-1 s=READY pgs=1616 cs=0 l=1 rev1=1 crypto rx=0x7f3e5000b2c0 tx=0x7f3e50004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-09T13:55:13.605 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:13.601+0000 7f3e597fa640 1 -- 172.21.15.43:0/3742947073 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3e50034950 con 0x7f3e5c0cd510 2023-08-09T13:55:13.605 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:13.601+0000 7f3e597fa640 1 -- 172.21.15.43:0/3742947073 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3e50034ad0 con 0x7f3e5c0cd510 2023-08-09T13:55:13.605 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:13.601+0000 7f3e597fa640 1 -- 172.21.15.43:0/3742947073 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3e50034e80 con 0x7f3e5c0cd510 2023-08-09T13:55:13.606 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:13.601+0000 7f3e62d01640 1 -- 172.21.15.43:0/3742947073 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3e5c15b760 con 0x7f3e5c0cd510 2023-08-09T13:55:13.606 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:13.601+0000 7f3e62d01640 1 -- 172.21.15.43:0/3742947073 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3e5c15bd20 con 0x7f3e5c0cd510 2023-08-09T13:55:13.606 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:13.602+0000 7f3e62d01640 1 -- 172.21.15.43:0/3742947073 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3e5c15c240 con 0x7f3e5c0cd510 2023-08-09T13:55:13.606 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:13.603+0000 7f3e597fa640 1 -- 172.21.15.43:0/3742947073 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 67) v1 ==== 88443+0+0 (secure 0 0 0) 0x7f3e50041480 con 0x7f3e5c0cd510 2023-08-09T13:55:13.607 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:13.604+0000 7f3e597fa640 1 --2- 172.21.15.43:0/3742947073 >> [v2:172.21.15.203:6832/3490266242,v1:172.21.15.203:6833/3490266242] conn(0x7f3e4006a170 0x7f3e4006c630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:55:13.607 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:13.604+0000 7f3e597fa640 1 -- 172.21.15.43:0/3742947073 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(135..135 src has 1..135) v4 ==== 7286+0+0 (secure 0 0 0) 0x7f3e500abdf0 con 0x7f3e5c0cd510 2023-08-09T13:55:13.607 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:13.605+0000 7f3e5bfff640 1 --2- 172.21.15.43:0/3742947073 >> [v2:172.21.15.203:6832/3490266242,v1:172.21.15.203:6833/3490266242] conn(0x7f3e4006a170 0x7f3e4006c630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:55:13.608 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:13.606+0000 7f3e5bfff640 1 --2- 172.21.15.43:0/3742947073 >> [v2:172.21.15.203:6832/3490266242,v1:172.21.15.203:6833/3490266242] conn(0x7f3e4006a170 0x7f3e4006c630 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0x7f3e5c0c7970 tx=0x7f3e48008040 comp rx=0 tx=0).ready entity=mgr.6659 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-09T13:55:13.611 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:13.609+0000 7f3e597fa640 1 -- 172.21.15.43:0/3742947073 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178066 (secure 0 0 0) 0x7f3e50077010 con 0x7f3e5c0cd510 2023-08-09T13:55:13.847 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:13.844+0000 7f3e62d01640 1 -- 172.21.15.43:0/3742947073 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) v1 -- 0x7f3e5c15c240 con 0x7f3e5c0cd510 2023-08-09T13:55:13.848 INFO:tasks.ceph.mon.a.smithi043.stderr:2023-08-09T13:55:13.846+0000 7f32867f3640 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2023-08-09T13:55:13.858 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:13.855+0000 7f3e597fa640 1 -- 172.21.15.43:0/3742947073 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v62) v1 ==== 161+0+0 (secure 0 0 0) 0x7f3e5003b090 con 0x7f3e5c0cd510 2023-08-09T13:55:13.858 INFO:teuthology.orchestra.run.smithi043.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2023-08-09T13:55:13.862 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:13.860+0000 7f3e62d01640 1 -- 172.21.15.43:0/3742947073 >> [v2:172.21.15.203:6832/3490266242,v1:172.21.15.203:6833/3490266242] conn(0x7f3e4006a170 msgr2=0x7f3e4006c630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:55:13.862 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:13.860+0000 7f3e62d01640 1 --2- 172.21.15.43:0/3742947073 >> [v2:172.21.15.203:6832/3490266242,v1:172.21.15.203:6833/3490266242] conn(0x7f3e4006a170 0x7f3e4006c630 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0x7f3e5c0c7970 tx=0x7f3e48008040 comp rx=0 tx=0).stop 2023-08-09T13:55:13.862 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:13.860+0000 7f3e62d01640 1 -- 172.21.15.43:0/3742947073 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f3e5c0cd510 msgr2=0x7f3e5c1481a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:55:13.863 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:13.860+0000 7f3e62d01640 1 --2- 172.21.15.43:0/3742947073 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f3e5c0cd510 0x7f3e5c1481a0 secure :-1 s=READY pgs=1616 cs=0 l=1 rev1=1 crypto rx=0x7f3e5000b2c0 tx=0x7f3e50004730 comp rx=0 tx=0).stop 2023-08-09T13:55:13.863 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:13.860+0000 7f3e62d01640 1 -- 172.21.15.43:0/3742947073 shutdown_connections 2023-08-09T13:55:13.863 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:13.860+0000 7f3e62d01640 1 --2- 172.21.15.43:0/3742947073 >> [v2:172.21.15.203:6832/3490266242,v1:172.21.15.203:6833/3490266242] conn(0x7f3e4006a170 0x7f3e4006c630 unknown :-1 s=CLOSED pgs=30 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:55:13.863 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:13.860+0000 7f3e62d01640 1 --2- 172.21.15.43:0/3742947073 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f3e5c15ce40 0x7f3e5c13aa90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:55:13.863 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:13.860+0000 7f3e62d01640 1 --2- 172.21.15.43:0/3742947073 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f3e5c142ca0 0x7f3e5c15c520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:55:13.864 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:13.860+0000 7f3e62d01640 1 --2- 172.21.15.43:0/3742947073 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f3e5c0cd510 0x7f3e5c1481a0 unknown :-1 s=CLOSED pgs=1616 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:55:13.864 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:13.860+0000 7f3e62d01640 1 -- 172.21.15.43:0/3742947073 >> 172.21.15.43:0/3742947073 conn(0x7f3e5c0ce0a0 msgr2=0x7f3e5c0c84b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-09T13:55:13.864 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:13.861+0000 7f3e62d01640 1 -- 172.21.15.43:0/3742947073 shutdown_connections 2023-08-09T13:55:13.864 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:13.861+0000 7f3e62d01640 1 -- 172.21.15.43:0/3742947073 wait complete. 2023-08-09T13:55:13.883 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs rm cephfs --yes-i-really-mean-it 2023-08-09T13:55:13.955 INFO:tasks.ceph.mds.a.smithi043.stdout:starting mds.a at 2023-08-09T13:55:14.092 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:14.089+0000 7fb950cc5640 1 Processor -- start 2023-08-09T13:55:14.092 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:14.089+0000 7fb950cc5640 1 -- start start 2023-08-09T13:55:14.093 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:14.090+0000 7fb950cc5640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fb94c0b45f0 0x7fb94c0cd6d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:55:14.093 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:14.090+0000 7fb950cc5640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7fb94c0d04b0 0x7fb94c0b59d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:55:14.093 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:14.091+0000 7fb950cc5640 1 -- --> v1:172.21.15.43:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb94c0b7bd0 con 0x7fb94c15c550 2023-08-09T13:55:14.094 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:14.091+0000 7fb950cc5640 1 -- --> v2:172.21.15.43:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb94c0d1c00 con 0x7fb94c0d04b0 2023-08-09T13:55:14.094 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:14.091+0000 7fb950cc5640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb94c15c930 con 0x7fb94c0b45f0 2023-08-09T13:55:14.094 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:14.091+0000 7fb94a59c640 1 --1- >> v1:172.21.15.43:6789/0 conn(0x7fb94c15c550 0x7fb94c0b3ee0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.43:6789/0 says I am v1:172.21.15.43:42240/0 (socket says 172.21.15.43:42240) 2023-08-09T13:55:14.094 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:14.091+0000 7fb94a59c640 1 -- 172.21.15.43:0/3857901180 learned_addr learned my addr 172.21.15.43:0/3857901180 (peer_addr_for_me v1:172.21.15.43:0/0) 2023-08-09T13:55:14.094 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:14.091+0000 7fb94ad9d640 1 --2- 172.21.15.43:0/3857901180 >> v2:172.21.15.43:3300/0 conn(0x7fb94c0d04b0 0x7fb94c0b59d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:55:14.096 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:14.091+0000 7fb941d9b640 1 --2- 172.21.15.43:0/3857901180 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fb94c0b45f0 0x7fb94c0cd6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:55:14.096 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:14.091+0000 7fb949d9b640 1 -- 172.21.15.43:0/3857901180 <== mon.0 v1:172.21.15.43:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 627764404 0 0) 0x7fb94c0b7bd0 con 0x7fb94c15c550 2023-08-09T13:55:14.096 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:14.092+0000 7fb949d9b640 1 -- 172.21.15.43:0/3857901180 --> v1:172.21.15.43:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fb938003520 con 0x7fb94c15c550 2023-08-09T13:55:14.097 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:14.092+0000 7fb94ad9d640 1 -- 172.21.15.43:0/3857901180 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fb94c0b45f0 msgr2=0x7fb94c0cd6d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-09T13:55:14.097 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:14.092+0000 7fb94ad9d640 1 --2- 172.21.15.43:0/3857901180 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fb94c0b45f0 0x7fb94c0cd6d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:55:14.097 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:14.092+0000 7fb94ad9d640 1 -- 172.21.15.43:0/3857901180 >> v1:172.21.15.43:6789/0 conn(0x7fb94c15c550 legacy=0x7fb94c0b3ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:55:14.098 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:14.092+0000 7fb94ad9d640 1 -- 172.21.15.43:0/3857901180 --> v2:172.21.15.43:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb94c0b54f0 con 0x7fb94c0d04b0 2023-08-09T13:55:14.098 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:14.092+0000 7fb941d9b640 1 --2- 172.21.15.43:0/3857901180 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fb94c0b45f0 0x7fb94c0cd6d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-08-09T13:55:14.098 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:14.093+0000 7fb94ad9d640 1 --2- 172.21.15.43:0/3857901180 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb94c0d04b0 0x7fb94c0b59d0 secure :-1 s=READY pgs=1621 cs=0 l=1 rev1=1 crypto rx=0x7fb9440097a0 tx=0x7fb94402b8b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e4a3aae757c7586f server_cookie=0 in_seq=0 out_seq=0 2023-08-09T13:55:14.098 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:14.093+0000 7fb949d9b640 1 -- 172.21.15.43:0/3857901180 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb94402d040 con 0x7fb94c0d04b0 2023-08-09T13:55:14.098 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:14.093+0000 7fb949d9b640 1 -- 172.21.15.43:0/3857901180 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb944031810 con 0x7fb94c0d04b0 2023-08-09T13:55:14.099 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:14.093+0000 7fb949d9b640 1 -- 172.21.15.43:0/3857901180 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb944031ba0 con 0x7fb94c0d04b0 2023-08-09T13:55:14.100 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:14.094+0000 7fb950cc5640 1 -- 172.21.15.43:0/3857901180 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb94c0d04b0 msgr2=0x7fb94c0b59d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:55:14.100 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:14.094+0000 7fb950cc5640 1 --2- 172.21.15.43:0/3857901180 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb94c0d04b0 0x7fb94c0b59d0 secure :-1 s=READY pgs=1621 cs=0 l=1 rev1=1 crypto rx=0x7fb9440097a0 tx=0x7fb94402b8b0 comp rx=0 tx=0).stop 2023-08-09T13:55:14.100 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:14.094+0000 7fb950cc5640 1 -- 172.21.15.43:0/3857901180 shutdown_connections 2023-08-09T13:55:14.100 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:14.094+0000 7fb950cc5640 1 --2- 172.21.15.43:0/3857901180 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb94c0d04b0 0x7fb94c0b59d0 unknown :-1 s=CLOSED pgs=1621 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:55:14.100 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:14.094+0000 7fb950cc5640 1 --2- 172.21.15.43:0/3857901180 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fb94c0b45f0 0x7fb94c0cd6d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:55:14.101 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:14.094+0000 7fb950cc5640 1 -- 172.21.15.43:0/3857901180 >> 172.21.15.43:0/3857901180 conn(0x7fb94c071520 msgr2=0x7fb94c0ceee0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-09T13:55:14.101 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:14.094+0000 7fb950cc5640 1 -- 172.21.15.43:0/3857901180 shutdown_connections 2023-08-09T13:55:14.101 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:14.094+0000 7fb950cc5640 1 -- 172.21.15.43:0/3857901180 wait complete. 2023-08-09T13:55:14.101 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:14.096+0000 7fb950cc5640 1 Processor -- start 2023-08-09T13:55:14.101 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:14.096+0000 7fb950cc5640 1 -- start start 2023-08-09T13:55:14.101 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:14.097+0000 7fb950cc5640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fb94c0b45f0 0x7fb94c0d29e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:55:14.101 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:14.097+0000 7fb950cc5640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb94c0d04b0 0x7fb94c1f4c90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:55:14.101 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:14.097+0000 7fb950cc5640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fb94c15c550 0x7fb94c1f51d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:55:14.102 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:14.097+0000 7fb950cc5640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb94c1f5860 con 0x7fb94c0d04b0 2023-08-09T13:55:14.102 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:14.097+0000 7fb950cc5640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb94c1f5ad0 con 0x7fb94c0b45f0 2023-08-09T13:55:14.102 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:14.097+0000 7fb950cc5640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb94c0a8dc0 con 0x7fb94c15c550 2023-08-09T13:55:14.102 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:14.098+0000 7fb941d9b640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb94c0d04b0 0x7fb94c1f4c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:55:14.102 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:14.098+0000 7fb941d9b640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb94c0d04b0 0x7fb94c1f4c90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:56684/0 (socket says 172.21.15.43:56684) 2023-08-09T13:55:14.102 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:14.098+0000 7fb941d9b640 1 -- 172.21.15.43:0/189145194 learned_addr learned my addr 172.21.15.43:0/189145194 (peer_addr_for_me v2:172.21.15.43:0/0) 2023-08-09T13:55:14.103 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:14.098+0000 7fb94a59c640 1 --2- 172.21.15.43:0/189145194 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fb94c0b45f0 0x7fb94c0d29e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:55:14.104 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:14.098+0000 7fb94ad9d640 1 --2- 172.21.15.43:0/189145194 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fb94c15c550 0x7fb94c1f51d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:55:14.104 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:14.098+0000 7fb941d9b640 1 -- 172.21.15.43:0/189145194 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fb94c15c550 msgr2=0x7fb94c1f51d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:55:14.104 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:14.098+0000 7fb941d9b640 1 --2- 172.21.15.43:0/189145194 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fb94c15c550 0x7fb94c1f51d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:55:14.104 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:14.098+0000 7fb941d9b640 1 -- 172.21.15.43:0/189145194 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fb94c0b45f0 msgr2=0x7fb94c0d29e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:55:14.104 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:14.098+0000 7fb941d9b640 1 --2- 172.21.15.43:0/189145194 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fb94c0b45f0 0x7fb94c0d29e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:55:14.104 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:14.098+0000 7fb941d9b640 1 -- 172.21.15.43:0/189145194 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb944009400 con 0x7fb94c0d04b0 2023-08-09T13:55:14.105 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:14.098+0000 7fb941d9b640 1 --2- 172.21.15.43:0/189145194 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb94c0d04b0 0x7fb94c1f4c90 secure :-1 s=READY pgs=1622 cs=0 l=1 rev1=1 crypto rx=0x7fb934009fc0 tx=0x7fb93400d560 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-09T13:55:14.105 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:14.099+0000 7fb9437fe640 1 -- 172.21.15.43:0/189145194 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb934003b90 con 0x7fb94c0d04b0 2023-08-09T13:55:14.105 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:14.099+0000 7fb9437fe640 1 -- 172.21.15.43:0/189145194 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb93400f040 con 0x7fb94c0d04b0 2023-08-09T13:55:14.105 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:14.099+0000 7fb9437fe640 1 -- 172.21.15.43:0/189145194 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb934004030 con 0x7fb94c0d04b0 2023-08-09T13:55:14.105 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:14.099+0000 7fb950cc5640 1 -- 172.21.15.43:0/189145194 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb94c0a90a0 con 0x7fb94c0d04b0 2023-08-09T13:55:14.105 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:14.099+0000 7fb950cc5640 1 -- 172.21.15.43:0/189145194 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb94c159f70 con 0x7fb94c0d04b0 2023-08-09T13:55:14.105 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:14.100+0000 7fb950cc5640 1 -- 172.21.15.43:0/189145194 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb94c0739f0 con 0x7fb94c0d04b0 2023-08-09T13:55:14.105 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:14.101+0000 7fb9437fe640 1 -- 172.21.15.43:0/189145194 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 67) v1 ==== 88443+0+0 (secure 0 0 0) 0x7fb934004190 con 0x7fb94c0d04b0 2023-08-09T13:55:14.106 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:14.102+0000 7fb9437fe640 1 --2- 172.21.15.43:0/189145194 >> [v2:172.21.15.203:6832/3490266242,v1:172.21.15.203:6833/3490266242] conn(0x7fb92c06a170 0x7fb92c06c630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:55:14.106 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:14.102+0000 7fb9437fe640 1 -- 172.21.15.43:0/189145194 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(136..136 src has 1..136) v4 ==== 7344+0+0 (secure 0 0 0) 0x7fb934089010 con 0x7fb94c0d04b0 2023-08-09T13:55:14.107 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:14.103+0000 7fb94a59c640 1 --2- 172.21.15.43:0/189145194 >> [v2:172.21.15.203:6832/3490266242,v1:172.21.15.203:6833/3490266242] conn(0x7fb92c06a170 0x7fb92c06c630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:55:14.107 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:14.104+0000 7fb94a59c640 1 --2- 172.21.15.43:0/189145194 >> [v2:172.21.15.203:6832/3490266242,v1:172.21.15.203:6833/3490266242] conn(0x7fb92c06a170 0x7fb92c06c630 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0x7fb93c005880 tx=0x7fb93c009040 comp rx=0 tx=0).ready entity=mgr.6659 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-09T13:55:14.109 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:14.106+0000 7fb9437fe640 1 -- 172.21.15.43:0/189145194 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178066 (secure 0 0 0) 0x7fb934054090 con 0x7fb94c0d04b0 2023-08-09T13:55:14.338 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:14.336+0000 7fb950cc5640 1 -- 172.21.15.43:0/189145194 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} v 0) v1 -- 0x7fb94c0c8640 con 0x7fb94c0d04b0 2023-08-09T13:55:14.860 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:14.858+0000 7fb9437fe640 1 -- 172.21.15.43:0/189145194 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]=0 v63) v1 ==== 104+0+0 (secure 0 0 0) 0x7fb9340576b0 con 0x7fb94c0d04b0 2023-08-09T13:55:14.864 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:14.861+0000 7fb950cc5640 1 -- 172.21.15.43:0/189145194 >> [v2:172.21.15.203:6832/3490266242,v1:172.21.15.203:6833/3490266242] conn(0x7fb92c06a170 msgr2=0x7fb92c06c630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:55:14.864 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:14.861+0000 7fb950cc5640 1 --2- 172.21.15.43:0/189145194 >> [v2:172.21.15.203:6832/3490266242,v1:172.21.15.203:6833/3490266242] conn(0x7fb92c06a170 0x7fb92c06c630 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0x7fb93c005880 tx=0x7fb93c009040 comp rx=0 tx=0).stop 2023-08-09T13:55:14.865 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:14.861+0000 7fb950cc5640 1 -- 172.21.15.43:0/189145194 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb94c0d04b0 msgr2=0x7fb94c1f4c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:55:14.865 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:14.861+0000 7fb950cc5640 1 --2- 172.21.15.43:0/189145194 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb94c0d04b0 0x7fb94c1f4c90 secure :-1 s=READY pgs=1622 cs=0 l=1 rev1=1 crypto rx=0x7fb934009fc0 tx=0x7fb93400d560 comp rx=0 tx=0).stop 2023-08-09T13:55:14.865 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:14.861+0000 7fb950cc5640 1 -- 172.21.15.43:0/189145194 shutdown_connections 2023-08-09T13:55:14.866 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:14.861+0000 7fb950cc5640 1 --2- 172.21.15.43:0/189145194 >> [v2:172.21.15.203:6832/3490266242,v1:172.21.15.203:6833/3490266242] conn(0x7fb92c06a170 0x7fb92c06c630 unknown :-1 s=CLOSED pgs=31 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:55:14.866 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:14.861+0000 7fb950cc5640 1 --2- 172.21.15.43:0/189145194 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fb94c15c550 0x7fb94c1f51d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:55:14.866 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:14.861+0000 7fb950cc5640 1 --2- 172.21.15.43:0/189145194 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb94c0d04b0 0x7fb94c1f4c90 unknown :-1 s=CLOSED pgs=1622 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:55:14.866 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:14.861+0000 7fb950cc5640 1 --2- 172.21.15.43:0/189145194 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fb94c0b45f0 0x7fb94c0d29e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:55:14.867 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:14.861+0000 7fb950cc5640 1 -- 172.21.15.43:0/189145194 >> 172.21.15.43:0/189145194 conn(0x7fb94c071520 msgr2=0x7fb94c067560 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-09T13:55:14.867 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:14.862+0000 7fb950cc5640 1 -- 172.21.15.43:0/189145194 shutdown_connections 2023-08-09T13:55:14.867 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:14.862+0000 7fb950cc5640 1 -- 172.21.15.43:0/189145194 wait complete. 2023-08-09T13:55:14.888 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_metadata cephfs_metadata --yes-i-really-really-mean-it 2023-08-09T13:55:15.126 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:15.123+0000 7f24869ab640 1 Processor -- start 2023-08-09T13:55:15.126 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:15.123+0000 7f24869ab640 1 -- start start 2023-08-09T13:55:15.128 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:15.125+0000 7f24869ab640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f24800ca370 0x7f248015b2a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:55:15.128 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:15.125+0000 7f24869ab640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f24800ce4c0 0x7f24800ce950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:55:15.129 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:15.125+0000 7f24869ab640 1 -- --> v1:172.21.15.43:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f248015ae10 con 0x7f248006fb70 2023-08-09T13:55:15.129 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:15.125+0000 7f24869ab640 1 -- --> v2:172.21.15.43:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f24800cee90 con 0x7f24800ce4c0 2023-08-09T13:55:15.129 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:15.125+0000 7f24869ab640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2480154e70 con 0x7f24800ca370 2023-08-09T13:55:15.129 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:15.125+0000 7f247ffff640 1 --1- >> v1:172.21.15.43:6789/0 conn(0x7f248006fb70 0x7f24800caab0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.43:6789/0 says I am v1:172.21.15.43:60884/0 (socket says 172.21.15.43:60884) 2023-08-09T13:55:15.130 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:15.125+0000 7f247ffff640 1 -- 172.21.15.43:0/1643053033 learned_addr learned my addr 172.21.15.43:0/1643053033 (peer_addr_for_me v1:172.21.15.43:0/0) 2023-08-09T13:55:15.130 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:15.125+0000 7f2484f48640 1 --2- 172.21.15.43:0/1643053033 >> v2:172.21.15.43:3300/0 conn(0x7f24800ce4c0 0x7f24800ce950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:55:15.133 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:15.126+0000 7f24777fe640 1 --2- 172.21.15.43:0/1643053033 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f24800ca370 0x7f248015b2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:55:15.133 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:15.126+0000 7f247f7fe640 1 -- 172.21.15.43:0/1643053033 <== mon.0 v1:172.21.15.43:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1649399729 0 0) 0x7f248015ae10 con 0x7f248006fb70 2023-08-09T13:55:15.133 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:15.126+0000 7f247f7fe640 1 -- 172.21.15.43:0/1643053033 --> v1:172.21.15.43:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f2460003520 con 0x7f248006fb70 2023-08-09T13:55:15.134 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:15.126+0000 7f2484f48640 1 -- 172.21.15.43:0/1643053033 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f24800ca370 msgr2=0x7f248015b2a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-09T13:55:15.134 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:15.126+0000 7f2484f48640 1 --2- 172.21.15.43:0/1643053033 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f24800ca370 0x7f248015b2a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:55:15.134 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:15.126+0000 7f2484f48640 1 -- 172.21.15.43:0/1643053033 >> v1:172.21.15.43:6789/0 conn(0x7f248006fb70 legacy=0x7f24800caab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:55:15.135 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:15.126+0000 7f2484f48640 1 -- 172.21.15.43:0/1643053033 --> v2:172.21.15.43:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f248006ff50 con 0x7f24800ce4c0 2023-08-09T13:55:15.135 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:15.127+0000 7f2484f48640 1 --2- 172.21.15.43:0/1643053033 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f24800ce4c0 0x7f24800ce950 secure :-1 s=READY pgs=1624 cs=0 l=1 rev1=1 crypto rx=0x7f2478009d90 tx=0x7f247802bea0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2dbca2643fa0cf1e server_cookie=0 in_seq=0 out_seq=0 2023-08-09T13:55:15.135 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:15.127+0000 7f247f7fe640 1 -- 172.21.15.43:0/1643053033 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f247802cc50 con 0x7f24800ce4c0 2023-08-09T13:55:15.135 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:15.128+0000 7f247f7fe640 1 -- 172.21.15.43:0/1643053033 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f247802cdb0 con 0x7f24800ce4c0 2023-08-09T13:55:15.136 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:15.128+0000 7f247f7fe640 1 -- 172.21.15.43:0/1643053033 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2478034640 con 0x7f24800ce4c0 2023-08-09T13:55:15.136 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:15.128+0000 7f24869ab640 1 -- 172.21.15.43:0/1643053033 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f24800ce4c0 msgr2=0x7f24800ce950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:55:15.136 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:15.128+0000 7f24869ab640 1 --2- 172.21.15.43:0/1643053033 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f24800ce4c0 0x7f24800ce950 secure :-1 s=READY pgs=1624 cs=0 l=1 rev1=1 crypto rx=0x7f2478009d90 tx=0x7f247802bea0 comp rx=0 tx=0).stop 2023-08-09T13:55:15.137 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:15.128+0000 7f24869ab640 1 -- 172.21.15.43:0/1643053033 shutdown_connections 2023-08-09T13:55:15.137 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:15.128+0000 7f24869ab640 1 --2- 172.21.15.43:0/1643053033 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f24800ce4c0 0x7f24800ce950 unknown :-1 s=CLOSED pgs=1624 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:55:15.137 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:15.128+0000 7f24869ab640 1 --2- 172.21.15.43:0/1643053033 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f24800ca370 0x7f248015b2a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:55:15.137 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:15.128+0000 7f24869ab640 1 -- 172.21.15.43:0/1643053033 >> 172.21.15.43:0/1643053033 conn(0x7f248006b990 msgr2=0x7f248006bda0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-09T13:55:15.138 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:15.128+0000 7f24869ab640 1 -- 172.21.15.43:0/1643053033 shutdown_connections 2023-08-09T13:55:15.138 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:15.128+0000 7f24869ab640 1 -- 172.21.15.43:0/1643053033 wait complete. 2023-08-09T13:55:15.138 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:15.130+0000 7f24869ab640 1 Processor -- start 2023-08-09T13:55:15.138 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:15.130+0000 7f24869ab640 1 -- start start 2023-08-09T13:55:15.141 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:15.131+0000 7f24869ab640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f248006fb70 0x7f24800c3da0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:55:15.142 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:15.131+0000 7f24869ab640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f24800ca370 0x7f24800cd7c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:55:15.142 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:15.131+0000 7f24869ab640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f24800ce4c0 0x7f2480153f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:55:15.142 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:15.131+0000 7f24869ab640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f24800cdd00 con 0x7f248006fb70 2023-08-09T13:55:15.143 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:15.131+0000 7f24869ab640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f24800c6a00 con 0x7f24800ca370 2023-08-09T13:55:15.143 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:15.131+0000 7f24869ab640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2480154440 con 0x7f24800ce4c0 2023-08-09T13:55:15.143 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:15.132+0000 7f247ffff640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f248006fb70 0x7f24800c3da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:55:15.144 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:15.132+0000 7f247ffff640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f248006fb70 0x7f24800c3da0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:36216/0 (socket says 172.21.15.43:36216) 2023-08-09T13:55:15.144 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:15.132+0000 7f247ffff640 1 -- 172.21.15.43:0/3495859081 learned_addr learned my addr 172.21.15.43:0/3495859081 (peer_addr_for_me v2:172.21.15.43:0/0) 2023-08-09T13:55:15.144 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:15.132+0000 7f24777fe640 1 --2- 172.21.15.43:0/3495859081 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f24800ca370 0x7f24800cd7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:55:15.144 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:15.132+0000 7f247ffff640 1 -- 172.21.15.43:0/3495859081 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f24800ce4c0 msgr2=0x7f2480153f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:55:15.145 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:15.132+0000 7f2484f48640 1 --2- 172.21.15.43:0/3495859081 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f24800ce4c0 0x7f2480153f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:55:15.145 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:15.132+0000 7f247ffff640 1 --2- 172.21.15.43:0/3495859081 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f24800ce4c0 0x7f2480153f00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:55:15.145 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:15.132+0000 7f247ffff640 1 -- 172.21.15.43:0/3495859081 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f24800ca370 msgr2=0x7f24800cd7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:55:15.146 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:15.132+0000 7f247ffff640 1 --2- 172.21.15.43:0/3495859081 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f24800ca370 0x7f24800cd7c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:55:15.146 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:15.132+0000 7f247ffff640 1 -- 172.21.15.43:0/3495859081 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2478009a70 con 0x7f248006fb70 2023-08-09T13:55:15.146 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:15.132+0000 7f24777fe640 1 --2- 172.21.15.43:0/3495859081 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f24800ca370 0x7f24800cd7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-08-09T13:55:15.146 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:15.132+0000 7f2484f48640 1 --2- 172.21.15.43:0/3495859081 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f24800ce4c0 0x7f2480153f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-08-09T13:55:15.147 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:15.133+0000 7f247ffff640 1 --2- 172.21.15.43:0/3495859081 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f248006fb70 0x7f24800c3da0 secure :-1 s=READY pgs=1625 cs=0 l=1 rev1=1 crypto rx=0x7f247000c500 tx=0x7f247000c9c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-09T13:55:15.147 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:15.133+0000 7f247d7fa640 1 -- 172.21.15.43:0/3495859081 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2470005ce0 con 0x7f248006fb70 2023-08-09T13:55:15.147 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:15.133+0000 7f247d7fa640 1 -- 172.21.15.43:0/3495859081 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2470005e40 con 0x7f248006fb70 2023-08-09T13:55:15.148 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:15.133+0000 7f247d7fa640 1 -- 172.21.15.43:0/3495859081 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2470007330 con 0x7f248006fb70 2023-08-09T13:55:15.148 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:15.133+0000 7f24869ab640 1 -- 172.21.15.43:0/3495859081 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f24800c4480 con 0x7f248006fb70 2023-08-09T13:55:15.148 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:15.133+0000 7f24869ab640 1 -- 172.21.15.43:0/3495859081 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f24800c4a40 con 0x7f248006fb70 2023-08-09T13:55:15.148 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:15.134+0000 7f24869ab640 1 -- 172.21.15.43:0/3495859081 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f24800739f0 con 0x7f248006fb70 2023-08-09T13:55:15.149 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:15.135+0000 7f247d7fa640 1 -- 172.21.15.43:0/3495859081 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 67) v1 ==== 88443+0+0 (secure 0 0 0) 0x7f2470007490 con 0x7f248006fb70 2023-08-09T13:55:15.149 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:15.136+0000 7f247d7fa640 1 --2- 172.21.15.43:0/3495859081 >> [v2:172.21.15.203:6832/3490266242,v1:172.21.15.203:6833/3490266242] conn(0x7f246406a170 0x7f246406c630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:55:15.149 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:15.136+0000 7f247d7fa640 1 -- 172.21.15.43:0/3495859081 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(136..136 src has 1..136) v4 ==== 7344+0+0 (secure 0 0 0) 0x7f2470088270 con 0x7f248006fb70 2023-08-09T13:55:15.150 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:15.137+0000 7f24777fe640 1 --2- 172.21.15.43:0/3495859081 >> [v2:172.21.15.203:6832/3490266242,v1:172.21.15.203:6833/3490266242] conn(0x7f246406a170 0x7f246406c630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:55:15.150 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:15.138+0000 7f24777fe640 1 --2- 172.21.15.43:0/3495859081 >> [v2:172.21.15.203:6832/3490266242,v1:172.21.15.203:6833/3490266242] conn(0x7f246406a170 0x7f246406c630 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0x7f24800cce30 tx=0x7f2468008040 comp rx=0 tx=0).ready entity=mgr.6659 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-09T13:55:15.151 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:15.140+0000 7f247d7fa640 1 -- 172.21.15.43:0/3495859081 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178066 (secure 0 0 0) 0x7f2470053420 con 0x7f248006fb70 2023-08-09T13:55:15.339 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:15.337+0000 7f24869ab640 1 -- 172.21.15.43:0/3495859081 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7f248006a360 con 0x7f248006fb70 2023-08-09T13:55:15.869 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:15.866+0000 7f247d7fa640 1 -- 172.21.15.43:0/3495859081 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_metadata' removed v137) v1 ==== 181+0+0 (secure 0 0 0) 0x7f2470056a40 con 0x7f248006fb70 2023-08-09T13:55:15.870 INFO:teuthology.orchestra.run.smithi043.stderr:pool 'cephfs_metadata' removed 2023-08-09T13:55:15.876 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:15.873+0000 7f24869ab640 1 -- 172.21.15.43:0/3495859081 >> [v2:172.21.15.203:6832/3490266242,v1:172.21.15.203:6833/3490266242] conn(0x7f246406a170 msgr2=0x7f246406c630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:55:15.876 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:15.873+0000 7f24869ab640 1 --2- 172.21.15.43:0/3495859081 >> [v2:172.21.15.203:6832/3490266242,v1:172.21.15.203:6833/3490266242] conn(0x7f246406a170 0x7f246406c630 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0x7f24800cce30 tx=0x7f2468008040 comp rx=0 tx=0).stop 2023-08-09T13:55:15.877 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:15.873+0000 7f24869ab640 1 -- 172.21.15.43:0/3495859081 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f248006fb70 msgr2=0x7f24800c3da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:55:15.877 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:15.873+0000 7f24869ab640 1 --2- 172.21.15.43:0/3495859081 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f248006fb70 0x7f24800c3da0 secure :-1 s=READY pgs=1625 cs=0 l=1 rev1=1 crypto rx=0x7f247000c500 tx=0x7f247000c9c0 comp rx=0 tx=0).stop 2023-08-09T13:55:15.877 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:15.874+0000 7f24869ab640 1 -- 172.21.15.43:0/3495859081 shutdown_connections 2023-08-09T13:55:15.877 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:15.874+0000 7f24869ab640 1 --2- 172.21.15.43:0/3495859081 >> [v2:172.21.15.203:6832/3490266242,v1:172.21.15.203:6833/3490266242] conn(0x7f246406a170 0x7f246406c630 unknown :-1 s=CLOSED pgs=33 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:55:15.877 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:15.874+0000 7f24869ab640 1 --2- 172.21.15.43:0/3495859081 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f24800ce4c0 0x7f2480153f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:55:15.877 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:15.874+0000 7f24869ab640 1 --2- 172.21.15.43:0/3495859081 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f24800ca370 0x7f24800cd7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:55:15.877 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:15.874+0000 7f24869ab640 1 --2- 172.21.15.43:0/3495859081 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f248006fb70 0x7f24800c3da0 unknown :-1 s=CLOSED pgs=1625 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:55:15.878 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:15.874+0000 7f24869ab640 1 -- 172.21.15.43:0/3495859081 >> 172.21.15.43:0/3495859081 conn(0x7f248006b990 msgr2=0x7f24800caab0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-09T13:55:15.878 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:15.874+0000 7f24869ab640 1 -- 172.21.15.43:0/3495859081 shutdown_connections 2023-08-09T13:55:15.878 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:15.874+0000 7f24869ab640 1 -- 172.21.15.43:0/3495859081 wait complete. 2023-08-09T13:55:15.900 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_data cephfs_data --yes-i-really-really-mean-it 2023-08-09T13:55:16.134 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:16.131+0000 7f0af9f70640 1 Processor -- start 2023-08-09T13:55:16.134 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:16.131+0000 7f0af9f70640 1 -- start start 2023-08-09T13:55:16.135 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:16.133+0000 7f0af9f70640 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7f0af415c720 0x7f0af40cd850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:55:16.136 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:16.133+0000 7f0af9f70640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f0af40cf370 0x7f0af40b5de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:55:16.136 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:16.133+0000 7f0af9f70640 1 -- --> v1:172.21.15.203:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0af40b5660 con 0x7f0af40d0900 2023-08-09T13:55:16.136 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:16.133+0000 7f0af9f70640 1 -- --> v2:172.21.15.43:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0af415cb00 con 0x7f0af40cf370 2023-08-09T13:55:16.136 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:16.133+0000 7f0af9f70640 1 -- --> v2:172.21.15.203:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0af40d1960 con 0x7f0af415c720 2023-08-09T13:55:16.137 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:16.133+0000 7f0af976f640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f0af40cf370 0x7f0af40b5de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:55:16.137 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:16.133+0000 7f0af976f640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f0af40cf370 0x7f0af40b5de0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:36224/0 (socket says 172.21.15.43:36224) 2023-08-09T13:55:16.137 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:16.133+0000 7f0af976f640 1 -- 172.21.15.43:0/2763256749 learned_addr learned my addr 172.21.15.43:0/2763256749 (peer_addr_for_me v2:172.21.15.43:0/0) 2023-08-09T13:55:16.137 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:16.134+0000 7f0af8f6e640 1 --2- 172.21.15.43:0/2763256749 >> v2:172.21.15.203:3300/0 conn(0x7f0af415c720 0x7f0af40cd850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:55:16.137 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:16.134+0000 7f0af976f640 1 -- 172.21.15.43:0/2763256749 >> v2:172.21.15.203:3300/0 conn(0x7f0af415c720 msgr2=0x7f0af40cd850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-09T13:55:16.138 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:16.134+0000 7f0af976f640 1 --2- 172.21.15.43:0/2763256749 >> v2:172.21.15.203:3300/0 conn(0x7f0af415c720 0x7f0af40cd850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:55:16.138 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:16.134+0000 7f0af976f640 1 -- 172.21.15.43:0/2763256749 >> v1:172.21.15.203:6789/0 conn(0x7f0af40d0900 legacy=0x7f0af40cec60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-09T13:55:16.138 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:16.134+0000 7f0af976f640 1 -- 172.21.15.43:0/2763256749 --> v2:172.21.15.43:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0af40b5900 con 0x7f0af40cf370 2023-08-09T13:55:16.139 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:16.135+0000 7f0af976f640 1 --2- 172.21.15.43:0/2763256749 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f0af40cf370 0x7f0af40b5de0 secure :-1 s=READY pgs=1626 cs=0 l=1 rev1=1 crypto rx=0x7f0af40b4de0 tx=0x7f0aec02b910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3f27a31209e85eee server_cookie=0 in_seq=0 out_seq=0 2023-08-09T13:55:16.139 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:16.135+0000 7f0aeb7fe640 1 -- 172.21.15.43:0/2763256749 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0aec031870 con 0x7f0af40cf370 2023-08-09T13:55:16.139 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:16.136+0000 7f0aeb7fe640 1 -- 172.21.15.43:0/2763256749 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0aec0319d0 con 0x7f0af40cf370 2023-08-09T13:55:16.139 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:16.136+0000 7f0aeb7fe640 1 -- 172.21.15.43:0/2763256749 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0aec031d60 con 0x7f0af40cf370 2023-08-09T13:55:16.139 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:16.136+0000 7f0af9f70640 1 -- 172.21.15.43:0/2763256749 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f0af40cf370 msgr2=0x7f0af40b5de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:55:16.140 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:16.136+0000 7f0af9f70640 1 --2- 172.21.15.43:0/2763256749 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f0af40cf370 0x7f0af40b5de0 secure :-1 s=READY pgs=1626 cs=0 l=1 rev1=1 crypto rx=0x7f0af40b4de0 tx=0x7f0aec02b910 comp rx=0 tx=0).stop 2023-08-09T13:55:16.140 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:16.136+0000 7f0af9f70640 1 -- 172.21.15.43:0/2763256749 shutdown_connections 2023-08-09T13:55:16.140 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:16.136+0000 7f0af9f70640 1 --2- 172.21.15.43:0/2763256749 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f0af40cf370 0x7f0af40b5de0 unknown :-1 s=CLOSED pgs=1626 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:55:16.140 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:16.136+0000 7f0af9f70640 1 --2- 172.21.15.43:0/2763256749 >> v2:172.21.15.203:3300/0 conn(0x7f0af415c720 0x7f0af40cd850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:55:16.140 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:16.136+0000 7f0af9f70640 1 -- 172.21.15.43:0/2763256749 >> 172.21.15.43:0/2763256749 conn(0x7f0af406a2a0 msgr2=0x7f0af406a680 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-09T13:55:16.140 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:16.136+0000 7f0af9f70640 1 -- 172.21.15.43:0/2763256749 shutdown_connections 2023-08-09T13:55:16.140 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:16.136+0000 7f0af9f70640 1 -- 172.21.15.43:0/2763256749 wait complete. 2023-08-09T13:55:16.141 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:16.138+0000 7f0af9f70640 1 Processor -- start 2023-08-09T13:55:16.141 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:16.138+0000 7f0af9f70640 1 -- start start 2023-08-09T13:55:16.142 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:16.139+0000 7f0af9f70640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f0af40cf370 0x7f0af41f5b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:55:16.142 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:16.139+0000 7f0af9f70640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f0af40d0900 0x7f0af415ed50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:55:16.143 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:16.139+0000 7f0af9f70640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f0af415c720 0x7f0af4149c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:55:16.143 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:16.139+0000 7f0af9f70640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0af40b22a0 con 0x7f0af40d0900 2023-08-09T13:55:16.143 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:16.139+0000 7f0af9f70640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0af415f880 con 0x7f0af40cf370 2023-08-09T13:55:16.143 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:16.139+0000 7f0af9f70640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0af415f5a0 con 0x7f0af415c720 2023-08-09T13:55:16.143 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:16.139+0000 7f0aebfff640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f0af40d0900 0x7f0af415ed50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:55:16.143 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:16.139+0000 7f0aebfff640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f0af40d0900 0x7f0af415ed50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:36232/0 (socket says 172.21.15.43:36232) 2023-08-09T13:55:16.143 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:16.139+0000 7f0aebfff640 1 -- 172.21.15.43:0/3965345328 learned_addr learned my addr 172.21.15.43:0/3965345328 (peer_addr_for_me v2:172.21.15.43:0/0) 2023-08-09T13:55:16.144 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:16.140+0000 7f0af8f6e640 1 --2- 172.21.15.43:0/3965345328 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f0af40cf370 0x7f0af41f5b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:55:16.144 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:16.140+0000 7f0af976f640 1 --2- 172.21.15.43:0/3965345328 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f0af415c720 0x7f0af4149c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:55:16.144 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:16.140+0000 7f0aebfff640 1 -- 172.21.15.43:0/3965345328 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f0af415c720 msgr2=0x7f0af4149c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:55:16.144 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:16.140+0000 7f0aebfff640 1 --2- 172.21.15.43:0/3965345328 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f0af415c720 0x7f0af4149c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:55:16.144 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:16.140+0000 7f0aebfff640 1 -- 172.21.15.43:0/3965345328 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f0af40cf370 msgr2=0x7f0af41f5b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:55:16.144 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:16.140+0000 7f0aebfff640 1 --2- 172.21.15.43:0/3965345328 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f0af40cf370 0x7f0af41f5b20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:55:16.144 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:16.140+0000 7f0aebfff640 1 -- 172.21.15.43:0/3965345328 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0aec0094d0 con 0x7f0af40d0900 2023-08-09T13:55:16.145 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:16.140+0000 7f0af976f640 1 --2- 172.21.15.43:0/3965345328 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f0af415c720 0x7f0af4149c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-08-09T13:55:16.145 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:16.140+0000 7f0aebfff640 1 --2- 172.21.15.43:0/3965345328 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f0af40d0900 0x7f0af415ed50 secure :-1 s=READY pgs=1627 cs=0 l=1 rev1=1 crypto rx=0x7f0ae400c430 tx=0x7f0ae400c8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-09T13:55:16.146 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:16.141+0000 7f0ae97fa640 1 -- 172.21.15.43:0/3965345328 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0ae4011790 con 0x7f0af40d0900 2023-08-09T13:55:16.146 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:16.141+0000 7f0ae97fa640 1 -- 172.21.15.43:0/3965345328 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0ae40118f0 con 0x7f0af40d0900 2023-08-09T13:55:16.146 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:16.141+0000 7f0ae97fa640 1 -- 172.21.15.43:0/3965345328 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0ae4011ce0 con 0x7f0af40d0900 2023-08-09T13:55:16.146 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:16.141+0000 7f0af9f70640 1 -- 172.21.15.43:0/3965345328 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0af40b1c50 con 0x7f0af40d0900 2023-08-09T13:55:16.146 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:16.141+0000 7f0af9f70640 1 -- 172.21.15.43:0/3965345328 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0af415f290 con 0x7f0af40d0900 2023-08-09T13:55:16.146 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:16.142+0000 7f0af9f70640 1 -- 172.21.15.43:0/3965345328 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0af40b81d0 con 0x7f0af40d0900 2023-08-09T13:55:16.146 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:16.143+0000 7f0ae97fa640 1 -- 172.21.15.43:0/3965345328 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 67) v1 ==== 88443+0+0 (secure 0 0 0) 0x7f0ae401c460 con 0x7f0af40d0900 2023-08-09T13:55:16.147 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:16.144+0000 7f0ae97fa640 1 --2- 172.21.15.43:0/3965345328 >> [v2:172.21.15.203:6832/3490266242,v1:172.21.15.203:6833/3490266242] conn(0x7f0ae006a170 0x7f0ae006c630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:55:16.147 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:16.144+0000 7f0ae97fa640 1 -- 172.21.15.43:0/3965345328 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(137..137 src has 1..137) v4 ==== 6906+0+0 (secure 0 0 0) 0x7f0ae4088a30 con 0x7f0af40d0900 2023-08-09T13:55:16.147 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:16.145+0000 7f0af8f6e640 1 --2- 172.21.15.43:0/3965345328 >> [v2:172.21.15.203:6832/3490266242,v1:172.21.15.203:6833/3490266242] conn(0x7f0ae006a170 0x7f0ae006c630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:55:16.148 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:16.146+0000 7f0af8f6e640 1 --2- 172.21.15.43:0/3965345328 >> [v2:172.21.15.203:6832/3490266242,v1:172.21.15.203:6833/3490266242] conn(0x7f0ae006a170 0x7f0ae006c630 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0x7f0af00070b0 tx=0x7f0af0007040 comp rx=0 tx=0).ready entity=mgr.6659 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-09T13:55:16.151 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:16.148+0000 7f0ae97fa640 1 -- 172.21.15.43:0/3965345328 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178066 (secure 0 0 0) 0x7f0ae4053cb0 con 0x7f0af40d0900 2023-08-09T13:55:16.348 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:16.346+0000 7f0af9f70640 1 -- 172.21.15.43:0/3965345328 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7f0af40739f0 con 0x7f0af40d0900 2023-08-09T13:55:16.871 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:16.868+0000 7f0ae97fa640 1 -- 172.21.15.43:0/3965345328 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data' removed v138) v1 ==== 169+0+0 (secure 0 0 0) 0x7f0ae40572d0 con 0x7f0af40d0900 2023-08-09T13:55:16.876 INFO:teuthology.orchestra.run.smithi043.stderr:pool 'cephfs_data' removed 2023-08-09T13:55:16.883 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:16.881+0000 7f0af9f70640 1 -- 172.21.15.43:0/3965345328 >> [v2:172.21.15.203:6832/3490266242,v1:172.21.15.203:6833/3490266242] conn(0x7f0ae006a170 msgr2=0x7f0ae006c630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:55:16.884 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:16.881+0000 7f0af9f70640 1 --2- 172.21.15.43:0/3965345328 >> [v2:172.21.15.203:6832/3490266242,v1:172.21.15.203:6833/3490266242] conn(0x7f0ae006a170 0x7f0ae006c630 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0x7f0af00070b0 tx=0x7f0af0007040 comp rx=0 tx=0).stop 2023-08-09T13:55:16.884 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:16.881+0000 7f0af9f70640 1 -- 172.21.15.43:0/3965345328 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f0af40d0900 msgr2=0x7f0af415ed50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:55:16.884 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:16.881+0000 7f0af9f70640 1 --2- 172.21.15.43:0/3965345328 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f0af40d0900 0x7f0af415ed50 secure :-1 s=READY pgs=1627 cs=0 l=1 rev1=1 crypto rx=0x7f0ae400c430 tx=0x7f0ae400c8f0 comp rx=0 tx=0).stop 2023-08-09T13:55:16.885 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:16.882+0000 7f0af9f70640 1 -- 172.21.15.43:0/3965345328 shutdown_connections 2023-08-09T13:55:16.885 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:16.882+0000 7f0af9f70640 1 --2- 172.21.15.43:0/3965345328 >> [v2:172.21.15.203:6832/3490266242,v1:172.21.15.203:6833/3490266242] conn(0x7f0ae006a170 0x7f0ae006c630 unknown :-1 s=CLOSED pgs=34 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:55:16.885 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:16.882+0000 7f0af9f70640 1 --2- 172.21.15.43:0/3965345328 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f0af415c720 0x7f0af4149c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:55:16.885 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:16.882+0000 7f0af9f70640 1 --2- 172.21.15.43:0/3965345328 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f0af40d0900 0x7f0af415ed50 unknown :-1 s=CLOSED pgs=1627 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:55:16.885 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:16.882+0000 7f0af9f70640 1 --2- 172.21.15.43:0/3965345328 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f0af40cf370 0x7f0af41f5b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:55:16.885 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:16.882+0000 7f0af9f70640 1 -- 172.21.15.43:0/3965345328 >> 172.21.15.43:0/3965345328 conn(0x7f0af406a2a0 msgr2=0x7f0af415b9c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-09T13:55:16.886 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:16.884+0000 7f0af9f70640 1 -- 172.21.15.43:0/3965345328 shutdown_connections 2023-08-09T13:55:16.886 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:16.884+0000 7f0af9f70640 1 -- 172.21.15.43:0/3965345328 wait complete. 2023-08-09T13:55:16.911 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config rm mgr mgr/snap_schedule/dump_on_update 2023-08-09T13:55:17.107 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:17.105+0000 7feb6dd5d640 1 Processor -- start 2023-08-09T13:55:17.108 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:17.105+0000 7feb6dd5d640 1 -- start start 2023-08-09T13:55:17.108 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:17.106+0000 7feb6dd5d640 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7feb6815df60 0x7feb680ce1c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:55:17.109 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:17.106+0000 7feb6dd5d640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7feb6815e3d0 0x7feb680ce700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:55:17.109 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:17.106+0000 7feb6dd5d640 1 -- --> v1:172.21.15.43:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7feb6815aef0 con 0x7feb680cb2d0 2023-08-09T13:55:17.109 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:17.106+0000 7feb6dd5d640 1 -- --> v2:172.21.15.43:3300/0 -- mon_getmap magic: 0 v1 -- 0x7feb6815a970 con 0x7feb6815e3d0 2023-08-09T13:55:17.109 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:17.106+0000 7feb6dd5d640 1 -- --> v2:172.21.15.203:3300/0 -- mon_getmap magic: 0 v1 -- 0x7feb6815e940 con 0x7feb6815df60 2023-08-09T13:55:17.109 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:17.106+0000 7feb677fe640 1 --1- >> v1:172.21.15.43:6789/0 conn(0x7feb680cb2d0 0x7feb6815b380 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.43:6789/0 says I am v1:172.21.15.43:60890/0 (socket says 172.21.15.43:60890) 2023-08-09T13:55:17.109 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:17.106+0000 7feb677fe640 1 -- 172.21.15.43:0/853477523 learned_addr learned my addr 172.21.15.43:0/853477523 (peer_addr_for_me v1:172.21.15.43:0/0) 2023-08-09T13:55:17.110 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:17.106+0000 7feb67fff640 1 --2- 172.21.15.43:0/853477523 >> v2:172.21.15.43:3300/0 conn(0x7feb6815e3d0 0x7feb680ce700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:55:17.110 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:17.107+0000 7feb5effd640 1 --2- 172.21.15.43:0/853477523 >> v2:172.21.15.203:3300/0 conn(0x7feb6815df60 0x7feb680ce1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:55:17.110 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:17.107+0000 7feb66ffd640 1 -- 172.21.15.43:0/853477523 <== mon.0 v1:172.21.15.43:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2855409817 0 0) 0x7feb6815aef0 con 0x7feb680cb2d0 2023-08-09T13:55:17.111 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:17.107+0000 7feb66ffd640 1 -- 172.21.15.43:0/853477523 --> v1:172.21.15.43:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7feb54003520 con 0x7feb680cb2d0 2023-08-09T13:55:17.111 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:17.107+0000 7feb67fff640 1 -- 172.21.15.43:0/853477523 >> v2:172.21.15.203:3300/0 conn(0x7feb6815df60 msgr2=0x7feb680ce1c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-09T13:55:17.111 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:17.107+0000 7feb67fff640 1 --2- 172.21.15.43:0/853477523 >> v2:172.21.15.203:3300/0 conn(0x7feb6815df60 0x7feb680ce1c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:55:17.111 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:17.107+0000 7feb67fff640 1 -- 172.21.15.43:0/853477523 >> v1:172.21.15.43:6789/0 conn(0x7feb680cb2d0 legacy=0x7feb6815b380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:55:17.111 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:17.107+0000 7feb67fff640 1 -- 172.21.15.43:0/853477523 --> v2:172.21.15.43:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7feb6815ec20 con 0x7feb6815e3d0 2023-08-09T13:55:17.112 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:17.108+0000 7feb67fff640 1 --2- 172.21.15.43:0/853477523 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7feb6815e3d0 0x7feb680ce700 secure :-1 s=READY pgs=1629 cs=0 l=1 rev1=1 crypto rx=0x7feb680cf0b0 tx=0x7feb6002be30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=73c62f0fe1635bd server_cookie=0 in_seq=0 out_seq=0 2023-08-09T13:55:17.112 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:17.109+0000 7feb66ffd640 1 -- 172.21.15.43:0/853477523 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feb6002cbe0 con 0x7feb6815e3d0 2023-08-09T13:55:17.112 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:17.109+0000 7feb66ffd640 1 -- 172.21.15.43:0/853477523 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7feb6002cd40 con 0x7feb6815e3d0 2023-08-09T13:55:17.113 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:17.109+0000 7feb66ffd640 1 -- 172.21.15.43:0/853477523 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feb60034640 con 0x7feb6815e3d0 2023-08-09T13:55:17.113 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:17.109+0000 7feb6dd5d640 1 -- 172.21.15.43:0/853477523 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7feb6815e3d0 msgr2=0x7feb680ce700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:55:17.113 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:17.109+0000 7feb6dd5d640 1 --2- 172.21.15.43:0/853477523 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7feb6815e3d0 0x7feb680ce700 secure :-1 s=READY pgs=1629 cs=0 l=1 rev1=1 crypto rx=0x7feb680cf0b0 tx=0x7feb6002be30 comp rx=0 tx=0).stop 2023-08-09T13:55:17.113 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:17.109+0000 7feb6dd5d640 1 -- 172.21.15.43:0/853477523 shutdown_connections 2023-08-09T13:55:17.113 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:17.109+0000 7feb6dd5d640 1 --2- 172.21.15.43:0/853477523 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7feb6815e3d0 0x7feb680ce700 unknown :-1 s=CLOSED pgs=1629 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:55:17.113 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:17.109+0000 7feb6dd5d640 1 --2- 172.21.15.43:0/853477523 >> v2:172.21.15.203:3300/0 conn(0x7feb6815df60 0x7feb680ce1c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:55:17.113 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:17.109+0000 7feb6dd5d640 1 -- 172.21.15.43:0/853477523 >> 172.21.15.43:0/853477523 conn(0x7feb680aff50 msgr2=0x7feb680b0360 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-09T13:55:17.114 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:17.109+0000 7feb6dd5d640 1 -- 172.21.15.43:0/853477523 shutdown_connections 2023-08-09T13:55:17.114 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:17.109+0000 7feb6dd5d640 1 -- 172.21.15.43:0/853477523 wait complete. 2023-08-09T13:55:17.114 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:17.111+0000 7feb6dd5d640 1 Processor -- start 2023-08-09T13:55:17.114 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:17.111+0000 7feb6dd5d640 1 -- start start 2023-08-09T13:55:17.115 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:17.112+0000 7feb6dd5d640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7feb680cb2d0 0x7feb681f26e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:55:17.115 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:17.112+0000 7feb6dd5d640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7feb6815df60 0x7feb681f2c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:55:17.115 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:17.113+0000 7feb6dd5d640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7feb6815e3d0 0x7feb68153730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:55:17.116 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:17.113+0000 7feb6dd5d640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feb680c5b80 con 0x7feb680cb2d0 2023-08-09T13:55:17.116 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:17.113+0000 7feb6dd5d640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feb680b2cc0 con 0x7feb6815df60 2023-08-09T13:55:17.116 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:17.113+0000 7feb6dd5d640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7feb680c5220 con 0x7feb6815e3d0 2023-08-09T13:55:17.116 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:17.113+0000 7feb677fe640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7feb680cb2d0 0x7feb681f26e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:55:17.116 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:17.113+0000 7feb677fe640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7feb680cb2d0 0x7feb681f26e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:36258/0 (socket says 172.21.15.43:36258) 2023-08-09T13:55:17.116 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:17.113+0000 7feb677fe640 1 -- 172.21.15.43:0/2521488062 learned_addr learned my addr 172.21.15.43:0/2521488062 (peer_addr_for_me v2:172.21.15.43:0/0) 2023-08-09T13:55:17.117 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:17.113+0000 7feb5effd640 1 --2- 172.21.15.43:0/2521488062 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7feb6815df60 0x7feb681f2c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:55:17.117 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:17.113+0000 7feb67fff640 1 --2- 172.21.15.43:0/2521488062 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7feb6815e3d0 0x7feb68153730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:55:17.118 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:17.113+0000 7feb677fe640 1 -- 172.21.15.43:0/2521488062 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7feb6815e3d0 msgr2=0x7feb68153730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:55:17.118 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:17.113+0000 7feb677fe640 1 --2- 172.21.15.43:0/2521488062 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7feb6815e3d0 0x7feb68153730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:55:17.118 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:17.113+0000 7feb677fe640 1 -- 172.21.15.43:0/2521488062 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7feb6815df60 msgr2=0x7feb681f2c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:55:17.118 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:17.113+0000 7feb677fe640 1 --2- 172.21.15.43:0/2521488062 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7feb6815df60 0x7feb681f2c70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:55:17.118 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:17.113+0000 7feb677fe640 1 -- 172.21.15.43:0/2521488062 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7feb60009a70 con 0x7feb680cb2d0 2023-08-09T13:55:17.118 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:17.113+0000 7feb67fff640 1 --2- 172.21.15.43:0/2521488062 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7feb6815e3d0 0x7feb68153730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-08-09T13:55:17.118 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:17.114+0000 7feb677fe640 1 --2- 172.21.15.43:0/2521488062 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7feb680cb2d0 0x7feb681f26e0 secure :-1 s=READY pgs=1630 cs=0 l=1 rev1=1 crypto rx=0x7feb5800c460 tx=0x7feb5800c920 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-09T13:55:17.119 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:17.114+0000 7feb64ff9640 1 -- 172.21.15.43:0/2521488062 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feb58005ce0 con 0x7feb680cb2d0 2023-08-09T13:55:17.119 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:17.114+0000 7feb64ff9640 1 -- 172.21.15.43:0/2521488062 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7feb58005e40 con 0x7feb680cb2d0 2023-08-09T13:55:17.119 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:17.114+0000 7feb64ff9640 1 -- 172.21.15.43:0/2521488062 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feb58007290 con 0x7feb680cb2d0 2023-08-09T13:55:17.119 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:17.114+0000 7feb6dd5d640 1 -- 172.21.15.43:0/2521488062 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7feb680c54c0 con 0x7feb680cb2d0 2023-08-09T13:55:17.119 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:17.114+0000 7feb6dd5d640 1 -- 172.21.15.43:0/2521488062 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7feb680b2470 con 0x7feb680cb2d0 2023-08-09T13:55:17.120 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:17.115+0000 7feb6dd5d640 1 -- 172.21.15.43:0/2521488062 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7feb680739f0 con 0x7feb680cb2d0 2023-08-09T13:55:17.120 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:17.116+0000 7feb64ff9640 1 -- 172.21.15.43:0/2521488062 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 67) v1 ==== 88443+0+0 (secure 0 0 0) 0x7feb580073f0 con 0x7feb680cb2d0 2023-08-09T13:55:17.120 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:17.118+0000 7feb64ff9640 1 --2- 172.21.15.43:0/2521488062 >> [v2:172.21.15.203:6832/3490266242,v1:172.21.15.203:6833/3490266242] conn(0x7feb4806a1a0 0x7feb4806c660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:55:17.120 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:17.118+0000 7feb64ff9640 1 -- 172.21.15.43:0/2521488062 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(138..138 src has 1..138) v4 ==== 6434+0+0 (secure 0 0 0) 0x7feb58087e60 con 0x7feb680cb2d0 2023-08-09T13:55:17.121 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:17.119+0000 7feb5effd640 1 --2- 172.21.15.43:0/2521488062 >> [v2:172.21.15.203:6832/3490266242,v1:172.21.15.203:6833/3490266242] conn(0x7feb4806a1a0 0x7feb4806c660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:55:17.122 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:17.120+0000 7feb5effd640 1 --2- 172.21.15.43:0/2521488062 >> [v2:172.21.15.203:6832/3490266242,v1:172.21.15.203:6833/3490266242] conn(0x7feb4806a1a0 0x7feb4806c660 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7feb681f2c20 tx=0x7feb50008040 comp rx=0 tx=0).ready entity=mgr.6659 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-09T13:55:17.125 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:17.122+0000 7feb64ff9640 1 -- 172.21.15.43:0/2521488062 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178066 (secure 0 0 0) 0x7feb58053220 con 0x7feb680cb2d0 2023-08-09T13:55:17.320 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:17.318+0000 7feb6dd5d640 1 -- 172.21.15.43:0/2521488062 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "config rm", "who": "mgr", "name": "mgr/snap_schedule/dump_on_update"} v 0) v1 -- 0x7feb680cbb80 con 0x7feb680cb2d0 2023-08-09T13:55:17.326 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:17.324+0000 7feb64ff9640 1 -- 172.21.15.43:0/2521488062 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "config rm", "who": "mgr", "name": "mgr/snap_schedule/dump_on_update"}]=0 v43) v1 ==== 115+0+0 (secure 0 0 0) 0x7feb58017020 con 0x7feb680cb2d0 2023-08-09T13:55:17.329 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:17.326+0000 7feb6dd5d640 1 -- 172.21.15.43:0/2521488062 >> [v2:172.21.15.203:6832/3490266242,v1:172.21.15.203:6833/3490266242] conn(0x7feb4806a1a0 msgr2=0x7feb4806c660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:55:17.329 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:17.326+0000 7feb6dd5d640 1 --2- 172.21.15.43:0/2521488062 >> [v2:172.21.15.203:6832/3490266242,v1:172.21.15.203:6833/3490266242] conn(0x7feb4806a1a0 0x7feb4806c660 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7feb681f2c20 tx=0x7feb50008040 comp rx=0 tx=0).stop 2023-08-09T13:55:17.329 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:17.326+0000 7feb6dd5d640 1 -- 172.21.15.43:0/2521488062 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7feb680cb2d0 msgr2=0x7feb681f26e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:55:17.329 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:17.326+0000 7feb6dd5d640 1 --2- 172.21.15.43:0/2521488062 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7feb680cb2d0 0x7feb681f26e0 secure :-1 s=READY pgs=1630 cs=0 l=1 rev1=1 crypto rx=0x7feb5800c460 tx=0x7feb5800c920 comp rx=0 tx=0).stop 2023-08-09T13:55:17.329 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:17.327+0000 7feb6dd5d640 1 -- 172.21.15.43:0/2521488062 shutdown_connections 2023-08-09T13:55:17.330 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:17.327+0000 7feb6dd5d640 1 --2- 172.21.15.43:0/2521488062 >> [v2:172.21.15.203:6832/3490266242,v1:172.21.15.203:6833/3490266242] conn(0x7feb4806a1a0 0x7feb4806c660 unknown :-1 s=CLOSED pgs=35 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:55:17.330 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:17.327+0000 7feb6dd5d640 1 --2- 172.21.15.43:0/2521488062 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7feb6815e3d0 0x7feb68153730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:55:17.330 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:17.327+0000 7feb6dd5d640 1 --2- 172.21.15.43:0/2521488062 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7feb6815df60 0x7feb681f2c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:55:17.330 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:17.327+0000 7feb6dd5d640 1 --2- 172.21.15.43:0/2521488062 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7feb680cb2d0 0x7feb681f26e0 unknown :-1 s=CLOSED pgs=1630 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:55:17.331 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:17.327+0000 7feb6dd5d640 1 -- 172.21.15.43:0/2521488062 >> 172.21.15.43:0/2521488062 conn(0x7feb680aff50 msgr2=0x7feb68154540 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-09T13:55:17.331 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:17.327+0000 7feb6dd5d640 1 -- 172.21.15.43:0/2521488062 shutdown_connections 2023-08-09T13:55:17.331 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:17.327+0000 7feb6dd5d640 1 -- 172.21.15.43:0/2521488062 wait complete. 2023-08-09T13:55:17.348 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config rm mgr mgr/snap_schedule/allow_m_granularity 2023-08-09T13:55:17.583 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:17.581+0000 7f86230e0640 1 Processor -- start 2023-08-09T13:55:17.583 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:17.581+0000 7f86230e0640 1 -- start start 2023-08-09T13:55:17.585 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:17.582+0000 7f86230e0640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f861c0b7df0 0x7f861c0d1b30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:55:17.585 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:17.582+0000 7f86230e0640 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7f861c0cebf0 0x7f861c0cf060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:55:17.585 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:17.582+0000 7f86230e0640 1 -- --> v1:172.21.15.43:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f861c0d16a0 con 0x7f861c15c2d0 2023-08-09T13:55:17.585 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:17.582+0000 7f86230e0640 1 -- --> v2:172.21.15.203:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f861c0b8230 con 0x7f861c0cebf0 2023-08-09T13:55:17.585 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:17.582+0000 7f86230e0640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f861c0cf5a0 con 0x7f861c0b7df0 2023-08-09T13:55:17.586 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:17.583+0000 7f86220de640 1 --1- >> v1:172.21.15.43:6789/0 conn(0x7f861c15c2d0 0x7f861c0cd020 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.43:6789/0 says I am v1:172.21.15.43:60904/0 (socket says 172.21.15.43:60904) 2023-08-09T13:55:17.586 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:17.583+0000 7f86220de640 1 -- 172.21.15.43:0/119218333 learned_addr learned my addr 172.21.15.43:0/119218333 (peer_addr_for_me v1:172.21.15.43:0/0) 2023-08-09T13:55:17.587 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:17.583+0000 7f86228df640 1 --2- 172.21.15.43:0/119218333 >> v2:172.21.15.203:3300/0 conn(0x7f861c0cebf0 0x7f861c0cf060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:55:17.588 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:17.583+0000 7f86218dd640 1 --2- 172.21.15.43:0/119218333 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f861c0b7df0 0x7f861c0d1b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:55:17.588 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:17.583+0000 7f86210dc640 1 -- 172.21.15.43:0/119218333 <== mon.0 v1:172.21.15.43:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 751557504 0 0) 0x7f861c0d16a0 con 0x7f861c15c2d0 2023-08-09T13:55:17.588 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:17.583+0000 7f86210dc640 1 -- 172.21.15.43:0/119218333 --> v1:172.21.15.43:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f8604003520 con 0x7f861c15c2d0 2023-08-09T13:55:17.588 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:17.584+0000 7f86210dc640 1 -- 172.21.15.43:0/119218333 <== mon.0 v1:172.21.15.43:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1738127410 0 0) 0x7f8604003520 con 0x7f861c15c2d0 2023-08-09T13:55:17.588 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:17.584+0000 7f86210dc640 1 -- 172.21.15.43:0/119218333 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f861c0b7df0 msgr2=0x7f861c0d1b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-09T13:55:17.588 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:17.584+0000 7f86210dc640 1 --2- 172.21.15.43:0/119218333 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f861c0b7df0 0x7f861c0d1b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:55:17.589 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:17.584+0000 7f86210dc640 1 -- 172.21.15.43:0/119218333 >> v2:172.21.15.203:3300/0 conn(0x7f861c0cebf0 msgr2=0x7f861c0cf060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-09T13:55:17.589 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:17.584+0000 7f86210dc640 1 --2- 172.21.15.43:0/119218333 >> v2:172.21.15.203:3300/0 conn(0x7f861c0cebf0 0x7f861c0cf060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:55:17.589 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:17.584+0000 7f86210dc640 1 -- 172.21.15.43:0/119218333 --> v1:172.21.15.43:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f861c0d0d70 con 0x7f861c15c2d0 2023-08-09T13:55:17.589 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:17.584+0000 7f86210dc640 1 -- 172.21.15.43:0/119218333 <== mon.0 v1:172.21.15.43:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1285150740 0 0) 0x7f860c0019f0 con 0x7f861c15c2d0 2023-08-09T13:55:17.589 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:17.584+0000 7f86210dc640 1 -- 172.21.15.43:0/119218333 >> v1:172.21.15.43:6789/0 conn(0x7f861c15c2d0 legacy=0x7f861c0cd020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:55:17.589 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:17.584+0000 7f86210dc640 1 --2- 172.21.15.43:0/119218333 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f8604003970 0x7f8604003d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:55:17.589 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:17.584+0000 7f86210dc640 1 --2- 172.21.15.43:0/119218333 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f86040047c0 0x7f8604004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:55:17.589 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:17.584+0000 7f86210dc640 1 --2- 172.21.15.43:0/119218333 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f861c0b7df0 0x7f8604005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:55:17.590 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:17.584+0000 7f86210dc640 1 -- 172.21.15.43:0/119218333 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f861c0cf5a0 con 0x7f86040047c0 2023-08-09T13:55:17.590 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:17.584+0000 7f86210dc640 1 -- 172.21.15.43:0/119218333 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f860400b980 con 0x7f8604003970 2023-08-09T13:55:17.590 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:17.584+0000 7f86210dc640 1 -- 172.21.15.43:0/119218333 --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f860400bac0 con 0x7f861c0b7df0 2023-08-09T13:55:17.590 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:17.584+0000 7f86218dd640 1 --2- 172.21.15.43:0/119218333 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f86040047c0 0x7f8604004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:55:17.590 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:17.585+0000 7f86220de640 1 --2- 172.21.15.43:0/119218333 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f8604003970 0x7f8604003d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:55:17.590 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:17.585+0000 7f86228df640 1 --2- 172.21.15.43:0/119218333 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f861c0b7df0 0x7f8604005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:55:17.590 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:17.585+0000 7f86218dd640 1 -- 172.21.15.43:0/119218333 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f861c0b7df0 msgr2=0x7f8604005180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-09T13:55:17.590 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:17.585+0000 7f86218dd640 1 --2- 172.21.15.43:0/119218333 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f861c0b7df0 0x7f8604005180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:55:17.591 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:17.585+0000 7f86218dd640 1 -- 172.21.15.43:0/119218333 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f8604003970 msgr2=0x7f8604003d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-09T13:55:17.591 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:17.585+0000 7f86218dd640 1 --2- 172.21.15.43:0/119218333 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f8604003970 0x7f8604003d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:55:17.591 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:17.585+0000 7f86218dd640 1 -- 172.21.15.43:0/119218333 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8604003520 con 0x7f86040047c0 2023-08-09T13:55:17.591 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:17.585+0000 7f86220de640 1 --2- 172.21.15.43:0/119218333 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f8604003970 0x7f8604003d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-08-09T13:55:17.592 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:17.586+0000 7f86218dd640 1 --2- 172.21.15.43:0/119218333 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f86040047c0 0x7f8604004c40 secure :-1 s=READY pgs=1632 cs=0 l=1 rev1=1 crypto rx=0x7f8610009ac0 tx=0x7f86040057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d8386558dc2d4501 server_cookie=0 in_seq=0 out_seq=0 2023-08-09T13:55:17.593 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:17.587+0000 7f86210dc640 1 -- 172.21.15.43:0/119218333 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f861002c6b0 con 0x7f86040047c0 2023-08-09T13:55:17.593 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:17.587+0000 7f86210dc640 1 -- 172.21.15.43:0/119218333 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f861002c810 con 0x7f86040047c0 2023-08-09T13:55:17.593 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:17.587+0000 7f86210dc640 1 -- 172.21.15.43:0/119218333 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f861002cba0 con 0x7f86040047c0 2023-08-09T13:55:17.593 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:17.587+0000 7f86230e0640 1 -- 172.21.15.43:0/119218333 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f86040047c0 msgr2=0x7f8604004c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:55:17.593 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:17.587+0000 7f86230e0640 1 --2- 172.21.15.43:0/119218333 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f86040047c0 0x7f8604004c40 secure :-1 s=READY pgs=1632 cs=0 l=1 rev1=1 crypto rx=0x7f8610009ac0 tx=0x7f86040057f0 comp rx=0 tx=0).stop 2023-08-09T13:55:17.593 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:17.587+0000 7f86228df640 1 -- 172.21.15.43:0/119218333 reap_dead start 2023-08-09T13:55:17.593 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:17.587+0000 7f86230e0640 1 -- 172.21.15.43:0/119218333 shutdown_connections 2023-08-09T13:55:17.593 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:17.587+0000 7f86230e0640 1 -- 172.21.15.43:0/119218333 >> 172.21.15.43:0/119218333 conn(0x7f861c06a230 msgr2=0x7f861c06a640 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-09T13:55:17.594 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:17.588+0000 7f86230e0640 1 -- 172.21.15.43:0/119218333 shutdown_connections 2023-08-09T13:55:17.594 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:17.588+0000 7f86230e0640 1 -- 172.21.15.43:0/119218333 wait complete. 2023-08-09T13:55:17.594 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:17.589+0000 7f86230e0640 1 Processor -- start 2023-08-09T13:55:17.594 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:17.590+0000 7f86230e0640 1 -- start start 2023-08-09T13:55:17.595 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:17.591+0000 7f86230e0640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f86040047c0 0x7f861c1f4ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:55:17.595 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:17.591+0000 7f86230e0640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f861c158a80 0x7f861c1f5010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:55:17.596 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:17.591+0000 7f86230e0640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f861c1f5550 0x7f861c0c9d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:55:17.596 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:17.591+0000 7f86230e0640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f861c0cbd30 con 0x7f86040047c0 2023-08-09T13:55:17.596 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:17.591+0000 7f86230e0640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f861c0cc540 con 0x7f861c1f5550 2023-08-09T13:55:17.596 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:17.591+0000 7f86230e0640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f861c0cbf80 con 0x7f861c158a80 2023-08-09T13:55:17.596 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:17.591+0000 7f86220de640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f86040047c0 0x7f861c1f4ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:55:17.596 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:17.591+0000 7f86220de640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f86040047c0 0x7f861c1f4ad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:36276/0 (socket says 172.21.15.43:36276) 2023-08-09T13:55:17.596 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:17.591+0000 7f86220de640 1 -- 172.21.15.43:0/993324418 learned_addr learned my addr 172.21.15.43:0/993324418 (peer_addr_for_me v2:172.21.15.43:0/0) 2023-08-09T13:55:17.597 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:17.591+0000 7f86218dd640 1 --2- 172.21.15.43:0/993324418 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f861c158a80 0x7f861c1f5010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:55:17.597 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:17.591+0000 7f86220de640 1 -- 172.21.15.43:0/993324418 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f861c158a80 msgr2=0x7f861c1f5010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:55:17.597 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:17.591+0000 7f86228df640 1 --2- 172.21.15.43:0/993324418 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f861c1f5550 0x7f861c0c9d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:55:17.597 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:17.591+0000 7f86220de640 1 --2- 172.21.15.43:0/993324418 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f861c158a80 0x7f861c1f5010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:55:17.597 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:17.591+0000 7f86220de640 1 -- 172.21.15.43:0/993324418 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f861c1f5550 msgr2=0x7f861c0c9d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:55:17.597 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:17.591+0000 7f86220de640 1 --2- 172.21.15.43:0/993324418 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f861c1f5550 0x7f861c0c9d00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:55:17.597 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:17.591+0000 7f86220de640 1 -- 172.21.15.43:0/993324418 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f861c0cc260 con 0x7f86040047c0 2023-08-09T13:55:17.598 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:17.592+0000 7f86220de640 1 --2- 172.21.15.43:0/993324418 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f86040047c0 0x7f861c1f4ad0 secure :-1 s=READY pgs=1633 cs=0 l=1 rev1=1 crypto rx=0x7f860c007b30 tx=0x7f860c00e530 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-09T13:55:17.598 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:17.592+0000 7f861affd640 1 -- 172.21.15.43:0/993324418 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f860c013420 con 0x7f86040047c0 2023-08-09T13:55:17.598 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:17.592+0000 7f861affd640 1 -- 172.21.15.43:0/993324418 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f860c013580 con 0x7f86040047c0 2023-08-09T13:55:17.598 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:17.592+0000 7f861affd640 1 -- 172.21.15.43:0/993324418 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f860c0138f0 con 0x7f86040047c0 2023-08-09T13:55:17.598 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:17.592+0000 7f86230e0640 1 -- 172.21.15.43:0/993324418 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f861c0c9500 con 0x7f86040047c0 2023-08-09T13:55:17.598 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:17.592+0000 7f86230e0640 1 -- 172.21.15.43:0/993324418 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f861c0ca240 con 0x7f86040047c0 2023-08-09T13:55:17.599 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:17.593+0000 7f86230e0640 1 -- 172.21.15.43:0/993324418 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f861c04e390 con 0x7f86040047c0 2023-08-09T13:55:17.599 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:17.594+0000 7f861affd640 1 -- 172.21.15.43:0/993324418 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 67) v1 ==== 88443+0+0 (secure 0 0 0) 0x7f860c013a50 con 0x7f86040047c0 2023-08-09T13:55:17.599 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:17.595+0000 7f861affd640 1 --2- 172.21.15.43:0/993324418 >> [v2:172.21.15.203:6832/3490266242,v1:172.21.15.203:6833/3490266242] conn(0x7f860806a170 0x7f860806c630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:55:17.599 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:17.595+0000 7f861affd640 1 -- 172.21.15.43:0/993324418 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(138..138 src has 1..138) v4 ==== 6434+0+0 (secure 0 0 0) 0x7f860c089140 con 0x7f86040047c0 2023-08-09T13:55:17.599 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:17.596+0000 7f86218dd640 1 --2- 172.21.15.43:0/993324418 >> [v2:172.21.15.203:6832/3490266242,v1:172.21.15.203:6833/3490266242] conn(0x7f860806a170 0x7f860806c630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:55:17.600 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:17.597+0000 7f86218dd640 1 --2- 172.21.15.43:0/993324418 >> [v2:172.21.15.203:6832/3490266242,v1:172.21.15.203:6833/3490266242] conn(0x7f860806a170 0x7f860806c630 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7f8610036000 tx=0x7f8610030000 comp rx=0 tx=0).ready entity=mgr.6659 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-09T13:55:17.602 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:17.600+0000 7f861affd640 1 -- 172.21.15.43:0/993324418 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178066 (secure 0 0 0) 0x7f860c054590 con 0x7f86040047c0 2023-08-09T13:55:17.801 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:17.798+0000 7f86230e0640 1 -- 172.21.15.43:0/993324418 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "config rm", "who": "mgr", "name": "mgr/snap_schedule/allow_m_granularity"} v 0) v1 -- 0x7f861c15bc60 con 0x7f86040047c0 2023-08-09T13:55:17.806 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:17.804+0000 7f861affd640 1 -- 172.21.15.43:0/993324418 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "config rm", "who": "mgr", "name": "mgr/snap_schedule/allow_m_granularity"}]=0 v44) v1 ==== 120+0+0 (secure 0 0 0) 0x7f860c057bb0 con 0x7f86040047c0 2023-08-09T13:55:17.809 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:17.807+0000 7f86230e0640 1 -- 172.21.15.43:0/993324418 >> [v2:172.21.15.203:6832/3490266242,v1:172.21.15.203:6833/3490266242] conn(0x7f860806a170 msgr2=0x7f860806c630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:55:17.809 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:17.807+0000 7f86230e0640 1 --2- 172.21.15.43:0/993324418 >> [v2:172.21.15.203:6832/3490266242,v1:172.21.15.203:6833/3490266242] conn(0x7f860806a170 0x7f860806c630 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7f8610036000 tx=0x7f8610030000 comp rx=0 tx=0).stop 2023-08-09T13:55:17.809 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:17.807+0000 7f86230e0640 1 -- 172.21.15.43:0/993324418 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f86040047c0 msgr2=0x7f861c1f4ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:55:17.810 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:17.807+0000 7f86230e0640 1 --2- 172.21.15.43:0/993324418 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f86040047c0 0x7f861c1f4ad0 secure :-1 s=READY pgs=1633 cs=0 l=1 rev1=1 crypto rx=0x7f860c007b30 tx=0x7f860c00e530 comp rx=0 tx=0).stop 2023-08-09T13:55:17.810 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:17.807+0000 7f86230e0640 1 -- 172.21.15.43:0/993324418 shutdown_connections 2023-08-09T13:55:17.810 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:17.807+0000 7f86230e0640 1 --2- 172.21.15.43:0/993324418 >> [v2:172.21.15.203:6832/3490266242,v1:172.21.15.203:6833/3490266242] conn(0x7f860806a170 0x7f860806c630 unknown :-1 s=CLOSED pgs=36 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:55:17.810 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:17.807+0000 7f86230e0640 1 --2- 172.21.15.43:0/993324418 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f861c1f5550 0x7f861c0c9d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:55:17.810 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:17.807+0000 7f86230e0640 1 --2- 172.21.15.43:0/993324418 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f861c158a80 0x7f861c1f5010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:55:17.810 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:17.807+0000 7f86230e0640 1 --2- 172.21.15.43:0/993324418 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f86040047c0 0x7f861c1f4ad0 unknown :-1 s=CLOSED pgs=1633 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:55:17.811 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:17.807+0000 7f86230e0640 1 -- 172.21.15.43:0/993324418 >> 172.21.15.43:0/993324418 conn(0x7f861c06a230 msgr2=0x7f861c0b71d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-09T13:55:17.811 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:17.807+0000 7f86230e0640 1 -- 172.21.15.43:0/993324418 shutdown_connections 2023-08-09T13:55:17.811 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:17.807+0000 7f86230e0640 1 -- 172.21.15.43:0/993324418 wait complete. 2023-08-09T13:55:17.828 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config rm mon mon_allow_pool_delete 2023-08-09T13:55:18.038 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:18.036+0000 7f3d40838640 1 Processor -- start 2023-08-09T13:55:18.038 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:18.036+0000 7f3d40838640 1 -- start start 2023-08-09T13:55:18.040 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:18.037+0000 7f3d40838640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f3d3c0cb2d0 0x7f3d3c15b380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:55:18.040 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:18.037+0000 7f3d40838640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f3d3c15df60 0x7f3d3c0ce1c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:55:18.040 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:18.038+0000 7f3d40838640 1 -- --> v1:172.21.15.43:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3d3c15aef0 con 0x7f3d3c15e3c0 2023-08-09T13:55:18.040 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:18.038+0000 7f3d40838640 1 -- --> v2:172.21.15.43:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3d3c15a970 con 0x7f3d3c15df60 2023-08-09T13:55:18.041 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:18.038+0000 7f3d40838640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3d3c15b8c0 con 0x7f3d3c0cb2d0 2023-08-09T13:55:18.041 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:18.038+0000 7f3d3ad9d640 1 --1- >> v1:172.21.15.43:6789/0 conn(0x7f3d3c15e3c0 0x7f3d3c0c9f90 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.43:6789/0 says I am v1:172.21.15.43:60916/0 (socket says 172.21.15.43:60916) 2023-08-09T13:55:18.042 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:18.038+0000 7f3d3ad9d640 1 -- 172.21.15.43:0/2909579015 learned_addr learned my addr 172.21.15.43:0/2909579015 (peer_addr_for_me v1:172.21.15.43:0/0) 2023-08-09T13:55:18.042 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:18.038+0000 7f3d33fff640 1 --2- 172.21.15.43:0/2909579015 >> v2:172.21.15.43:3300/0 conn(0x7f3d3c15df60 0x7f3d3c0ce1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:55:18.042 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:18.038+0000 7f3d3a59c640 1 --2- 172.21.15.43:0/2909579015 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f3d3c0cb2d0 0x7f3d3c15b380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:55:18.042 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:18.038+0000 7f3d33fff640 1 -- 172.21.15.43:0/2909579015 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f3d3c0cb2d0 msgr2=0x7f3d3c15b380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-09T13:55:18.042 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:18.038+0000 7f3d33fff640 1 --2- 172.21.15.43:0/2909579015 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f3d3c0cb2d0 0x7f3d3c15b380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:55:18.042 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:18.039+0000 7f3d33fff640 1 -- 172.21.15.43:0/2909579015 >> v1:172.21.15.43:6789/0 conn(0x7f3d3c15e3c0 legacy=0x7f3d3c0c9f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:55:18.043 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:18.039+0000 7f3d33fff640 1 -- 172.21.15.43:0/2909579015 --> v2:172.21.15.43:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3d3c15e940 con 0x7f3d3c15df60 2023-08-09T13:55:18.043 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:18.039+0000 7f3d39d9b640 1 -- 172.21.15.43:0/2909579015 <== mon.0 v1:172.21.15.43:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3888315332 0 0) 0x7f3d3c15aef0 con 0x7f3d3c15e3c0 2023-08-09T13:55:18.044 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:18.040+0000 7f3d33fff640 1 --2- 172.21.15.43:0/2909579015 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f3d3c15df60 0x7f3d3c0ce1c0 secure :-1 s=READY pgs=1635 cs=0 l=1 rev1=1 crypto rx=0x7f3d240097a0 tx=0x7f3d2402b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e4a5f36b038bda9a server_cookie=0 in_seq=0 out_seq=0 2023-08-09T13:55:18.044 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:18.040+0000 7f3d39d9b640 1 -- 172.21.15.43:0/2909579015 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3d24031830 con 0x7f3d3c15df60 2023-08-09T13:55:18.044 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:18.040+0000 7f3d39d9b640 1 -- 172.21.15.43:0/2909579015 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3d24031990 con 0x7f3d3c15df60 2023-08-09T13:55:18.044 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:18.040+0000 7f3d39d9b640 1 -- 172.21.15.43:0/2909579015 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3d24031d20 con 0x7f3d3c15df60 2023-08-09T13:55:18.045 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:18.041+0000 7f3d40838640 1 -- 172.21.15.43:0/2909579015 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f3d3c15df60 msgr2=0x7f3d3c0ce1c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:55:18.045 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:18.041+0000 7f3d40838640 1 --2- 172.21.15.43:0/2909579015 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f3d3c15df60 0x7f3d3c0ce1c0 secure :-1 s=READY pgs=1635 cs=0 l=1 rev1=1 crypto rx=0x7f3d240097a0 tx=0x7f3d2402b8d0 comp rx=0 tx=0).stop 2023-08-09T13:55:18.045 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:18.041+0000 7f3d40838640 1 -- 172.21.15.43:0/2909579015 shutdown_connections 2023-08-09T13:55:18.045 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:18.041+0000 7f3d40838640 1 --2- 172.21.15.43:0/2909579015 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f3d3c15df60 0x7f3d3c0ce1c0 unknown :-1 s=CLOSED pgs=1635 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:55:18.045 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:18.041+0000 7f3d40838640 1 --2- 172.21.15.43:0/2909579015 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f3d3c0cb2d0 0x7f3d3c15b380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:55:18.045 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:18.041+0000 7f3d40838640 1 -- 172.21.15.43:0/2909579015 >> 172.21.15.43:0/2909579015 conn(0x7f3d3c0aff50 msgr2=0x7f3d3c0b0360 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-09T13:55:18.046 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:18.041+0000 7f3d40838640 1 -- 172.21.15.43:0/2909579015 shutdown_connections 2023-08-09T13:55:18.046 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:18.041+0000 7f3d40838640 1 -- 172.21.15.43:0/2909579015 wait complete. 2023-08-09T13:55:18.047 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:18.043+0000 7f3d40838640 1 Processor -- start 2023-08-09T13:55:18.047 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:18.043+0000 7f3d40838640 1 -- start start 2023-08-09T13:55:18.047 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:18.044+0000 7f3d40838640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f3d3c0cb2d0 0x7f3d3c149500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:55:18.048 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:18.044+0000 7f3d40838640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f3d3c15df60 0x7f3d3c0cbfa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:55:18.048 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:18.044+0000 7f3d40838640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f3d3c15e3c0 0x7f3d3c159420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:55:18.048 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:18.044+0000 7f3d40838640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3d3c149a40 con 0x7f3d3c15e3c0 2023-08-09T13:55:18.048 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:18.044+0000 7f3d40838640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3d3c0cd650 con 0x7f3d3c0cb2d0 2023-08-09T13:55:18.048 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:18.044+0000 7f3d40838640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3d3c0cca40 con 0x7f3d3c15df60 2023-08-09T13:55:18.048 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:18.044+0000 7f3d3ad9d640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f3d3c15e3c0 0x7f3d3c159420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:55:18.049 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:18.044+0000 7f3d3ad9d640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f3d3c15e3c0 0x7f3d3c159420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:36284/0 (socket says 172.21.15.43:36284) 2023-08-09T13:55:18.049 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:18.044+0000 7f3d3ad9d640 1 -- 172.21.15.43:0/549044486 learned_addr learned my addr 172.21.15.43:0/549044486 (peer_addr_for_me v2:172.21.15.43:0/0) 2023-08-09T13:55:18.049 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:18.045+0000 7f3d33fff640 1 --2- 172.21.15.43:0/549044486 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f3d3c15df60 0x7f3d3c0cbfa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:55:18.049 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:18.045+0000 7f3d3a59c640 1 --2- 172.21.15.43:0/549044486 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f3d3c0cb2d0 0x7f3d3c149500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:55:18.049 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:18.045+0000 7f3d3ad9d640 1 -- 172.21.15.43:0/549044486 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f3d3c15df60 msgr2=0x7f3d3c0cbfa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:55:18.050 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:18.045+0000 7f3d3ad9d640 1 --2- 172.21.15.43:0/549044486 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f3d3c15df60 0x7f3d3c0cbfa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:55:18.050 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:18.045+0000 7f3d3ad9d640 1 -- 172.21.15.43:0/549044486 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f3d3c0cb2d0 msgr2=0x7f3d3c149500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:55:18.050 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:18.045+0000 7f3d3ad9d640 1 --2- 172.21.15.43:0/549044486 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f3d3c0cb2d0 0x7f3d3c149500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:55:18.050 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:18.045+0000 7f3d3ad9d640 1 -- 172.21.15.43:0/549044486 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3d24009400 con 0x7f3d3c15e3c0 2023-08-09T13:55:18.050 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:18.045+0000 7f3d3ad9d640 1 --2- 172.21.15.43:0/549044486 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f3d3c15e3c0 0x7f3d3c159420 secure :-1 s=READY pgs=1636 cs=0 l=1 rev1=1 crypto rx=0x7f3d3400c780 tx=0x7f3d3400cc40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-09T13:55:18.051 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:18.045+0000 7f3d32ffd640 1 -- 172.21.15.43:0/549044486 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3d3400ce90 con 0x7f3d3c15e3c0 2023-08-09T13:55:18.051 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:18.045+0000 7f3d32ffd640 1 -- 172.21.15.43:0/549044486 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3d3400a640 con 0x7f3d3c15e3c0 2023-08-09T13:55:18.052 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:18.045+0000 7f3d32ffd640 1 -- 172.21.15.43:0/549044486 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3d3400a9d0 con 0x7f3d3c15e3c0 2023-08-09T13:55:18.052 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:18.045+0000 7f3d40838640 1 -- 172.21.15.43:0/549044486 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3d3c0ccd20 con 0x7f3d3c15e3c0 2023-08-09T13:55:18.052 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:18.046+0000 7f3d40838640 1 -- 172.21.15.43:0/549044486 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3d3c0cd2e0 con 0x7f3d3c15e3c0 2023-08-09T13:55:18.052 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:18.047+0000 7f3d40838640 1 -- 172.21.15.43:0/549044486 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3d3c0739f0 con 0x7f3d3c15e3c0 2023-08-09T13:55:18.052 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:18.048+0000 7f3d32ffd640 1 -- 172.21.15.43:0/549044486 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 67) v1 ==== 88443+0+0 (secure 0 0 0) 0x7f3d3400ab30 con 0x7f3d3c15e3c0 2023-08-09T13:55:18.053 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:18.048+0000 7f3d32ffd640 1 --2- 172.21.15.43:0/549044486 >> [v2:172.21.15.203:6832/3490266242,v1:172.21.15.203:6833/3490266242] conn(0x7f3d1c06a170 0x7f3d1c06c630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:55:18.053 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:18.049+0000 7f3d32ffd640 1 -- 172.21.15.43:0/549044486 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(138..138 src has 1..138) v4 ==== 6434+0+0 (secure 0 0 0) 0x7f3d34087f20 con 0x7f3d3c15e3c0 2023-08-09T13:55:18.053 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:18.049+0000 7f3d3a59c640 1 --2- 172.21.15.43:0/549044486 >> [v2:172.21.15.203:6832/3490266242,v1:172.21.15.203:6833/3490266242] conn(0x7f3d1c06a170 0x7f3d1c06c630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:55:18.054 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:18.050+0000 7f3d3a59c640 1 --2- 172.21.15.43:0/549044486 >> [v2:172.21.15.203:6832/3490266242,v1:172.21.15.203:6833/3490266242] conn(0x7f3d1c06a170 0x7f3d1c06c630 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7f3d2c006fd0 tx=0x7f3d2c008040 comp rx=0 tx=0).ready entity=mgr.6659 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-09T13:55:18.055 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:18.053+0000 7f3d32ffd640 1 -- 172.21.15.43:0/549044486 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178066 (secure 0 0 0) 0x7f3d34053250 con 0x7f3d3c15e3c0 2023-08-09T13:55:18.251 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:18.249+0000 7f3d40838640 1 -- 172.21.15.43:0/549044486 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"} v 0) v1 -- 0x7f3d3c0cd790 con 0x7f3d3c15e3c0 2023-08-09T13:55:18.260 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:18.257+0000 7f3d32ffd640 1 -- 172.21.15.43:0/549044486 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"}]=0 v45) v1 ==== 104+0+0 (secure 0 0 0) 0x7f3d34018070 con 0x7f3d3c15e3c0 2023-08-09T13:55:18.263 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:18.261+0000 7f3d40838640 1 -- 172.21.15.43:0/549044486 >> [v2:172.21.15.203:6832/3490266242,v1:172.21.15.203:6833/3490266242] conn(0x7f3d1c06a170 msgr2=0x7f3d1c06c630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:55:18.263 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:18.261+0000 7f3d40838640 1 --2- 172.21.15.43:0/549044486 >> [v2:172.21.15.203:6832/3490266242,v1:172.21.15.203:6833/3490266242] conn(0x7f3d1c06a170 0x7f3d1c06c630 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7f3d2c006fd0 tx=0x7f3d2c008040 comp rx=0 tx=0).stop 2023-08-09T13:55:18.263 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:18.261+0000 7f3d40838640 1 -- 172.21.15.43:0/549044486 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f3d3c15e3c0 msgr2=0x7f3d3c159420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:55:18.263 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:18.261+0000 7f3d40838640 1 --2- 172.21.15.43:0/549044486 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f3d3c15e3c0 0x7f3d3c159420 secure :-1 s=READY pgs=1636 cs=0 l=1 rev1=1 crypto rx=0x7f3d3400c780 tx=0x7f3d3400cc40 comp rx=0 tx=0).stop 2023-08-09T13:55:18.264 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:18.261+0000 7f3d40838640 1 -- 172.21.15.43:0/549044486 shutdown_connections 2023-08-09T13:55:18.264 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:18.261+0000 7f3d40838640 1 --2- 172.21.15.43:0/549044486 >> [v2:172.21.15.203:6832/3490266242,v1:172.21.15.203:6833/3490266242] conn(0x7f3d1c06a170 0x7f3d1c06c630 unknown :-1 s=CLOSED pgs=37 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:55:18.264 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:18.261+0000 7f3d40838640 1 --2- 172.21.15.43:0/549044486 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f3d3c15e3c0 0x7f3d3c159420 unknown :-1 s=CLOSED pgs=1636 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:55:18.264 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:18.261+0000 7f3d40838640 1 --2- 172.21.15.43:0/549044486 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f3d3c15df60 0x7f3d3c0cbfa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:55:18.264 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:18.261+0000 7f3d40838640 1 --2- 172.21.15.43:0/549044486 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f3d3c0cb2d0 0x7f3d3c149500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:55:18.265 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:18.261+0000 7f3d40838640 1 -- 172.21.15.43:0/549044486 >> 172.21.15.43:0/549044486 conn(0x7f3d3c0aff50 msgr2=0x7f3d3c070390 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-09T13:55:18.265 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:18.261+0000 7f3d40838640 1 -- 172.21.15.43:0/549044486 shutdown_connections 2023-08-09T13:55:18.265 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:18.261+0000 7f3d40838640 1 -- 172.21.15.43:0/549044486 wait complete. 2023-08-09T13:55:18.282 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Ended test tasks.cephfs.test_snap_schedules.TestSnapSchedules.test_snap_schedule' 2023-08-09T13:55:18.510 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:18.508+0000 7fcde3456640 1 Processor -- start 2023-08-09T13:55:18.511 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:18.508+0000 7fcde3456640 1 -- start start 2023-08-09T13:55:18.512 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:18.510+0000 7fcde3456640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fcddc0c8b30 0x7fcddc0ad340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:55:18.512 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:18.510+0000 7fcde3456640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7fcddc0acdf0 0x7fcddc15b2c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:55:18.512 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:18.510+0000 7fcde3456640 1 -- --> v1:172.21.15.43:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fcddc0cbd90 con 0x7fcddc0c8750 2023-08-09T13:55:18.512 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:18.510+0000 7fcde3456640 1 -- --> v2:172.21.15.43:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fcddc15e810 con 0x7fcddc0acdf0 2023-08-09T13:55:18.512 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:18.510+0000 7fcde3456640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcddc0c9f70 con 0x7fcddc0c8b30 2023-08-09T13:55:18.513 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:18.510+0000 7fcde2c55640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7fcddc0acdf0 0x7fcddc15b2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:55:18.513 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:18.510+0000 7fcde2454640 1 --1- >> v1:172.21.15.43:6789/0 conn(0x7fcddc0c8750 0x7fcddc06f410 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.43:6789/0 says I am v1:172.21.15.43:60924/0 (socket says 172.21.15.43:60924) 2023-08-09T13:55:18.514 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:18.510+0000 7fcde2454640 1 -- 172.21.15.43:0/2715871483 learned_addr learned my addr 172.21.15.43:0/2715871483 (peer_addr_for_me v1:172.21.15.43:0/0) 2023-08-09T13:55:18.514 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:18.510+0000 7fcde1c53640 1 --2- 172.21.15.43:0/2715871483 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fcddc0c8b30 0x7fcddc0ad340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:55:18.514 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:18.511+0000 7fcde1452640 1 -- 172.21.15.43:0/2715871483 <== mon.0 v1:172.21.15.43:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 607455444 0 0) 0x7fcddc0cbd90 con 0x7fcddc0c8750 2023-08-09T13:55:18.514 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:18.511+0000 7fcde1452640 1 -- 172.21.15.43:0/2715871483 --> v1:172.21.15.43:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fcdc8003520 con 0x7fcddc0c8750 2023-08-09T13:55:18.514 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:18.511+0000 7fcde2c55640 1 -- 172.21.15.43:0/2715871483 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fcddc0c8b30 msgr2=0x7fcddc0ad340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-09T13:55:18.514 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:18.511+0000 7fcde2c55640 1 --2- 172.21.15.43:0/2715871483 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fcddc0c8b30 0x7fcddc0ad340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:55:18.514 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:18.511+0000 7fcde2c55640 1 -- 172.21.15.43:0/2715871483 >> v1:172.21.15.43:6789/0 conn(0x7fcddc0c8750 legacy=0x7fcddc06f410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:55:18.515 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:18.511+0000 7fcde2c55640 1 -- 172.21.15.43:0/2715871483 --> v2:172.21.15.43:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcddc0ca2b0 con 0x7fcddc0acdf0 2023-08-09T13:55:18.515 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:18.512+0000 7fcde2c55640 1 --2- 172.21.15.43:0/2715871483 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fcddc0acdf0 0x7fcddc15b2c0 secure :-1 s=READY pgs=1638 cs=0 l=1 rev1=1 crypto rx=0x7fcdd4009d90 tx=0x7fcdd402be10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7c615761c3dc3028 server_cookie=0 in_seq=0 out_seq=0 2023-08-09T13:55:18.516 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:18.512+0000 7fcde1452640 1 -- 172.21.15.43:0/2715871483 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcdd402cbc0 con 0x7fcddc0acdf0 2023-08-09T13:55:18.516 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:18.512+0000 7fcde1452640 1 -- 172.21.15.43:0/2715871483 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcdd402cd20 con 0x7fcddc0acdf0 2023-08-09T13:55:18.516 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:18.512+0000 7fcde1452640 1 -- 172.21.15.43:0/2715871483 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcdd4034640 con 0x7fcddc0acdf0 2023-08-09T13:55:18.516 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:18.513+0000 7fcde3456640 1 -- 172.21.15.43:0/2715871483 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fcddc0acdf0 msgr2=0x7fcddc15b2c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:55:18.516 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:18.513+0000 7fcde3456640 1 --2- 172.21.15.43:0/2715871483 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fcddc0acdf0 0x7fcddc15b2c0 secure :-1 s=READY pgs=1638 cs=0 l=1 rev1=1 crypto rx=0x7fcdd4009d90 tx=0x7fcdd402be10 comp rx=0 tx=0).stop 2023-08-09T13:55:18.516 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:18.513+0000 7fcde3456640 1 -- 172.21.15.43:0/2715871483 shutdown_connections 2023-08-09T13:55:18.516 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:18.513+0000 7fcde3456640 1 --2- 172.21.15.43:0/2715871483 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fcddc0acdf0 0x7fcddc15b2c0 unknown :-1 s=CLOSED pgs=1638 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:55:18.517 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:18.513+0000 7fcde3456640 1 --2- 172.21.15.43:0/2715871483 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fcddc0c8b30 0x7fcddc0ad340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:55:18.517 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:18.513+0000 7fcde3456640 1 -- 172.21.15.43:0/2715871483 >> 172.21.15.43:0/2715871483 conn(0x7fcddc0cde90 msgr2=0x7fcddc0ce2a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-09T13:55:18.517 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:18.513+0000 7fcde3456640 1 -- 172.21.15.43:0/2715871483 shutdown_connections 2023-08-09T13:55:18.517 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:18.513+0000 7fcde3456640 1 -- 172.21.15.43:0/2715871483 wait complete. 2023-08-09T13:55:18.517 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:18.515+0000 7fcde3456640 1 Processor -- start 2023-08-09T13:55:18.518 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:18.515+0000 7fcde3456640 1 -- start start 2023-08-09T13:55:18.519 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:18.516+0000 7fcde3456640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fcddc0acdf0 0x7fcddc1526a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:55:18.519 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:18.516+0000 7fcde3456640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fcddc0c8750 0x7fcddc1e3ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:55:18.519 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:18.516+0000 7fcde3456640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fcddc0c8b30 0x7fcddc1e5db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:55:18.519 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:18.516+0000 7fcde3456640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcddc09bb00 con 0x7fcddc0c8750 2023-08-09T13:55:18.519 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:18.516+0000 7fcde3456640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcddc09b820 con 0x7fcddc0c8b30 2023-08-09T13:55:18.519 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:18.516+0000 7fcde3456640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcddc1e4220 con 0x7fcddc0acdf0 2023-08-09T13:55:18.519 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:18.516+0000 7fcde1c53640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fcddc0c8750 0x7fcddc1e3ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:55:18.520 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:18.516+0000 7fcde1c53640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fcddc0c8750 0x7fcddc1e3ac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:36302/0 (socket says 172.21.15.43:36302) 2023-08-09T13:55:18.520 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:18.516+0000 7fcde1c53640 1 -- 172.21.15.43:0/1949217505 learned_addr learned my addr 172.21.15.43:0/1949217505 (peer_addr_for_me v2:172.21.15.43:0/0) 2023-08-09T13:55:18.520 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:18.516+0000 7fcde2454640 1 --2- 172.21.15.43:0/1949217505 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fcddc0acdf0 0x7fcddc1526a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:55:18.520 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:18.516+0000 7fcde2c55640 1 --2- 172.21.15.43:0/1949217505 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fcddc0c8b30 0x7fcddc1e5db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:55:18.520 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:18.516+0000 7fcde1c53640 1 -- 172.21.15.43:0/1949217505 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fcddc0acdf0 msgr2=0x7fcddc1526a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:55:18.520 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:18.516+0000 7fcde1c53640 1 --2- 172.21.15.43:0/1949217505 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fcddc0acdf0 0x7fcddc1526a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:55:18.520 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:18.516+0000 7fcde1c53640 1 -- 172.21.15.43:0/1949217505 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fcddc0c8b30 msgr2=0x7fcddc1e5db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:55:18.521 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:18.516+0000 7fcde1c53640 1 --2- 172.21.15.43:0/1949217505 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fcddc0c8b30 0x7fcddc1e5db0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:55:18.521 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:18.516+0000 7fcde1c53640 1 -- 172.21.15.43:0/1949217505 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcdd4009a70 con 0x7fcddc0c8750 2023-08-09T13:55:18.521 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:18.517+0000 7fcde1c53640 1 --2- 172.21.15.43:0/1949217505 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fcddc0c8750 0x7fcddc1e3ac0 secure :-1 s=READY pgs=1639 cs=0 l=1 rev1=1 crypto rx=0x7fcdcc00d800 tx=0x7fcdcc00dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-09T13:55:18.522 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:18.517+0000 7fcdd2ffd640 1 -- 172.21.15.43:0/1949217505 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcdcc00b6c0 con 0x7fcddc0c8750 2023-08-09T13:55:18.522 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:18.517+0000 7fcdd2ffd640 1 -- 172.21.15.43:0/1949217505 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcdcc00b820 con 0x7fcddc0c8750 2023-08-09T13:55:18.522 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:18.517+0000 7fcde3456640 1 -- 172.21.15.43:0/1949217505 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcddc1e4500 con 0x7fcddc0c8750 2023-08-09T13:55:18.522 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:18.517+0000 7fcdd2ffd640 1 -- 172.21.15.43:0/1949217505 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcdcc00bbb0 con 0x7fcddc0c8750 2023-08-09T13:55:18.522 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:18.517+0000 7fcde3456640 1 -- 172.21.15.43:0/1949217505 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcddc0c4f60 con 0x7fcddc0c8750 2023-08-09T13:55:18.522 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:18.519+0000 7fcde3456640 1 -- 172.21.15.43:0/1949217505 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcddc0c9230 con 0x7fcddc0c8750 2023-08-09T13:55:18.523 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:18.519+0000 7fcdd2ffd640 1 -- 172.21.15.43:0/1949217505 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 67) v1 ==== 88443+0+0 (secure 0 0 0) 0x7fcdcc00bd10 con 0x7fcddc0c8750 2023-08-09T13:55:18.523 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:18.520+0000 7fcdd2ffd640 1 --2- 172.21.15.43:0/1949217505 >> [v2:172.21.15.203:6832/3490266242,v1:172.21.15.203:6833/3490266242] conn(0x7fcdc006a1a0 0x7fcdc006c660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:55:18.523 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:18.521+0000 7fcdd2ffd640 1 -- 172.21.15.43:0/1949217505 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(138..138 src has 1..138) v4 ==== 6434+0+0 (secure 0 0 0) 0x7fcdcc088d20 con 0x7fcddc0c8750 2023-08-09T13:55:18.523 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:18.521+0000 7fcde2454640 1 --2- 172.21.15.43:0/1949217505 >> [v2:172.21.15.203:6832/3490266242,v1:172.21.15.203:6833/3490266242] conn(0x7fcdc006a1a0 0x7fcdc006c660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:55:18.524 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:18.522+0000 7fcde2454640 1 --2- 172.21.15.43:0/1949217505 >> [v2:172.21.15.203:6832/3490266242,v1:172.21.15.203:6833/3490266242] conn(0x7fcdc006a1a0 0x7fcdc006c660 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7fcdd8000f60 tx=0x7fcdd8009040 comp rx=0 tx=0).ready entity=mgr.6659 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-09T13:55:18.527 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:18.525+0000 7fcdd2ffd640 1 -- 172.21.15.43:0/1949217505 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178066 (secure 0 0 0) 0x7fcdcc054210 con 0x7fcddc0c8750 2023-08-09T13:55:18.793 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:18.790+0000 7fcde3456640 1 -- 172.21.15.43:0/1949217505 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_snap_schedules.TestSnapSchedules.test_snap_schedule"]} v 0) v1 -- 0x7fcddc0739f0 con 0x7fcddc0c8750 2023-08-09T13:55:18.874 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:18.872+0000 7fcdd2ffd640 1 -- 172.21.15.43:0/1949217505 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_snap_schedules.TestSnapSchedules.test_snap_schedule"]}]=0 v836) v1 ==== 148+0+0 (secure 0 0 0) 0x7fcdcc057830 con 0x7fcddc0c8750 2023-08-09T13:55:18.877 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:18.875+0000 7fcde3456640 1 -- 172.21.15.43:0/1949217505 >> [v2:172.21.15.203:6832/3490266242,v1:172.21.15.203:6833/3490266242] conn(0x7fcdc006a1a0 msgr2=0x7fcdc006c660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:55:18.877 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:18.875+0000 7fcde3456640 1 --2- 172.21.15.43:0/1949217505 >> [v2:172.21.15.203:6832/3490266242,v1:172.21.15.203:6833/3490266242] conn(0x7fcdc006a1a0 0x7fcdc006c660 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7fcdd8000f60 tx=0x7fcdd8009040 comp rx=0 tx=0).stop 2023-08-09T13:55:18.878 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:18.875+0000 7fcde3456640 1 -- 172.21.15.43:0/1949217505 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fcddc0c8750 msgr2=0x7fcddc1e3ac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:55:18.878 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:18.875+0000 7fcde3456640 1 --2- 172.21.15.43:0/1949217505 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fcddc0c8750 0x7fcddc1e3ac0 secure :-1 s=READY pgs=1639 cs=0 l=1 rev1=1 crypto rx=0x7fcdcc00d800 tx=0x7fcdcc00dcc0 comp rx=0 tx=0).stop 2023-08-09T13:55:18.878 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:18.875+0000 7fcde3456640 1 -- 172.21.15.43:0/1949217505 shutdown_connections 2023-08-09T13:55:18.878 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:18.875+0000 7fcde3456640 1 --2- 172.21.15.43:0/1949217505 >> [v2:172.21.15.203:6832/3490266242,v1:172.21.15.203:6833/3490266242] conn(0x7fcdc006a1a0 0x7fcdc006c660 unknown :-1 s=CLOSED pgs=38 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:55:18.879 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:18.875+0000 7fcde3456640 1 --2- 172.21.15.43:0/1949217505 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fcddc0c8b30 0x7fcddc1e5db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:55:18.879 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:18.875+0000 7fcde3456640 1 --2- 172.21.15.43:0/1949217505 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fcddc0c8750 0x7fcddc1e3ac0 unknown :-1 s=CLOSED pgs=1639 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:55:18.879 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:18.875+0000 7fcde3456640 1 --2- 172.21.15.43:0/1949217505 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fcddc0acdf0 0x7fcddc1526a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:55:18.879 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:18.875+0000 7fcde3456640 1 -- 172.21.15.43:0/1949217505 >> 172.21.15.43:0/1949217505 conn(0x7fcddc0cde90 msgr2=0x7fcddc158780 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-09T13:55:18.879 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:18.875+0000 7fcde3456640 1 -- 172.21.15.43:0/1949217505 shutdown_connections 2023-08-09T13:55:18.879 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:18.876+0000 7fcde3456640 1 -- 172.21.15.43:0/1949217505 wait complete. 2023-08-09T13:55:18.898 INFO:tasks.cephfs_test_runner:Test existence of a scheduled snapshot ... ok 2023-08-09T13:55:18.898 INFO:tasks.cephfs_test_runner:Starting test: test_snap_schedule_for_number_of_snaps_retention (tasks.cephfs.test_snap_schedules.TestSnapSchedules) 2023-08-09T13:55:18.898 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Starting test tasks.cephfs.test_snap_schedules.TestSnapSchedules.test_snap_schedule_for_number_of_snaps_retention' 2023-08-09T13:55:19.127 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:19.125+0000 7fe76dae5640 1 Processor -- start 2023-08-09T13:55:19.128 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:19.125+0000 7fe76dae5640 1 -- start start 2023-08-09T13:55:19.129 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:19.127+0000 7fe76dae5640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fe7681586b0 0x7fe7680b02a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:55:19.129 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:19.127+0000 7fe76dae5640 1 -- --> v1:172.21.15.43:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe7680caa20 con 0x7fe7680b26d0 2023-08-09T13:55:19.130 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:19.127+0000 7fe76dae5640 1 -- --> v1:172.21.15.203:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe7680c9e10 con 0x7fe7680cd090 2023-08-09T13:55:19.130 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:19.127+0000 7fe76dae5640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe7680ca0f0 con 0x7fe7681586b0 2023-08-09T13:55:19.130 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:19.127+0000 7fe76cae3640 1 --1- >> v1:172.21.15.43:6789/0 conn(0x7fe7680b26d0 0x7fe7680b2ab0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.43:6789/0 says I am v1:172.21.15.43:60934/0 (socket says 172.21.15.43:60934) 2023-08-09T13:55:19.130 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:19.127+0000 7fe76cae3640 1 -- 172.21.15.43:0/781960502 learned_addr learned my addr 172.21.15.43:0/781960502 (peer_addr_for_me v1:172.21.15.43:0/0) 2023-08-09T13:55:19.131 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:19.128+0000 7fe7677fe640 1 -- 172.21.15.43:0/781960502 <== mon.0 v1:172.21.15.43:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 82559801 0 0) 0x7fe7680caa20 con 0x7fe7680b26d0 2023-08-09T13:55:19.132 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:19.128+0000 7fe7677fe640 1 -- 172.21.15.43:0/781960502 --> v1:172.21.15.43:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fe758003520 con 0x7fe7680b26d0 2023-08-09T13:55:19.132 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:19.128+0000 7fe767fff640 1 --2- 172.21.15.43:0/781960502 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fe7681586b0 0x7fe7680b02a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:55:19.132 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:19.128+0000 7fe7677fe640 1 -- 172.21.15.43:0/781960502 <== mon.0 v1:172.21.15.43:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3772540493 0 0) 0x7fe758003520 con 0x7fe7680b26d0 2023-08-09T13:55:19.132 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:19.128+0000 7fe7677fe640 1 -- 172.21.15.43:0/781960502 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fe7681586b0 msgr2=0x7fe7680b02a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-09T13:55:19.132 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:19.128+0000 7fe7677fe640 1 --2- 172.21.15.43:0/781960502 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fe7681586b0 0x7fe7680b02a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:55:19.132 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:19.128+0000 7fe7677fe640 1 -- 172.21.15.43:0/781960502 >> v1:172.21.15.203:6789/0 conn(0x7fe7680cd090 legacy=0x7fe7680cd540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:55:19.132 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:19.128+0000 7fe7677fe640 1 -- 172.21.15.43:0/781960502 --> v1:172.21.15.43:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe7680ca3d0 con 0x7fe7680b26d0 2023-08-09T13:55:19.133 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:19.128+0000 7fe7677fe640 1 -- 172.21.15.43:0/781960502 <== mon.0 v1:172.21.15.43:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1285150740 0 0) 0x7fe75c001670 con 0x7fe7680b26d0 2023-08-09T13:55:19.133 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:19.128+0000 7fe7677fe640 1 -- 172.21.15.43:0/781960502 >> v1:172.21.15.43:6789/0 conn(0x7fe7680b26d0 legacy=0x7fe7680b2ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:55:19.133 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:19.128+0000 7fe7677fe640 1 --2- 172.21.15.43:0/781960502 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fe7580039a0 0x7fe758003d80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:55:19.133 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:19.128+0000 7fe7677fe640 1 --2- 172.21.15.43:0/781960502 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fe758004300 0x7fe7580047e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:55:19.133 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:19.128+0000 7fe7677fe640 1 --2- 172.21.15.43:0/781960502 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fe758004fe0 0x7fe758005950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:55:19.133 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:19.128+0000 7fe7677fe640 1 -- 172.21.15.43:0/781960502 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe7680ca0f0 con 0x7fe758004300 2023-08-09T13:55:19.133 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:19.128+0000 7fe7677fe640 1 -- 172.21.15.43:0/781960502 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe758005e90 con 0x7fe758004fe0 2023-08-09T13:55:19.133 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:19.128+0000 7fe7677fe640 1 -- 172.21.15.43:0/781960502 --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe758006050 con 0x7fe7580039a0 2023-08-09T13:55:19.134 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:19.129+0000 7fe767fff640 1 --2- 172.21.15.43:0/781960502 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fe758004300 0x7fe7580047e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:55:19.134 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:19.129+0000 7fe767fff640 1 -- 172.21.15.43:0/781960502 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fe7580039a0 msgr2=0x7fe758003d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-09T13:55:19.134 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:19.129+0000 7fe767fff640 1 --2- 172.21.15.43:0/781960502 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fe7580039a0 0x7fe758003d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:55:19.134 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:19.129+0000 7fe767fff640 1 -- 172.21.15.43:0/781960502 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fe758004fe0 msgr2=0x7fe758005950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-09T13:55:19.134 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:19.129+0000 7fe767fff640 1 --2- 172.21.15.43:0/781960502 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fe758004fe0 0x7fe758005950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:55:19.134 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:19.129+0000 7fe767fff640 1 -- 172.21.15.43:0/781960502 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe758003520 con 0x7fe758004300 2023-08-09T13:55:19.135 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:19.130+0000 7fe767fff640 1 --2- 172.21.15.43:0/781960502 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fe758004300 0x7fe7580047e0 secure :-1 s=READY pgs=1641 cs=0 l=1 rev1=1 crypto rx=0x7fe754009ac0 tx=0x7fe7580057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7f53a12c99a8aa9 server_cookie=0 in_seq=0 out_seq=0 2023-08-09T13:55:19.136 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:19.130+0000 7fe7677fe640 1 -- 172.21.15.43:0/781960502 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe75402c710 con 0x7fe758004300 2023-08-09T13:55:19.136 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:19.131+0000 7fe7677fe640 1 -- 172.21.15.43:0/781960502 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe75402c870 con 0x7fe758004300 2023-08-09T13:55:19.136 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:19.131+0000 7fe7677fe640 1 -- 172.21.15.43:0/781960502 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe75402cc00 con 0x7fe758004300 2023-08-09T13:55:19.136 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:19.131+0000 7fe76dae5640 1 -- 172.21.15.43:0/781960502 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fe758004300 msgr2=0x7fe7580047e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:55:19.136 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:19.131+0000 7fe76dae5640 1 --2- 172.21.15.43:0/781960502 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fe758004300 0x7fe7580047e0 secure :-1 s=READY pgs=1641 cs=0 l=1 rev1=1 crypto rx=0x7fe754009ac0 tx=0x7fe7580057f0 comp rx=0 tx=0).stop 2023-08-09T13:55:19.136 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:19.131+0000 7fe76d2e4640 1 -- 172.21.15.43:0/781960502 reap_dead start 2023-08-09T13:55:19.136 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:19.131+0000 7fe76dae5640 1 -- 172.21.15.43:0/781960502 shutdown_connections 2023-08-09T13:55:19.137 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:19.131+0000 7fe76dae5640 1 -- 172.21.15.43:0/781960502 >> 172.21.15.43:0/781960502 conn(0x7fe7680cad20 msgr2=0x7fe7680cb130 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-09T13:55:19.137 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:19.131+0000 7fe76dae5640 1 -- 172.21.15.43:0/781960502 shutdown_connections 2023-08-09T13:55:19.137 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:19.131+0000 7fe76dae5640 1 -- 172.21.15.43:0/781960502 wait complete. 2023-08-09T13:55:19.137 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:19.133+0000 7fe76dae5640 1 Processor -- start 2023-08-09T13:55:19.137 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:19.133+0000 7fe76dae5640 1 -- start start 2023-08-09T13:55:19.138 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:19.134+0000 7fe76dae5640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fe758004300 0x7fe7680aafb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:55:19.139 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:19.134+0000 7fe76dae5640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fe768146200 0x7fe7680c7b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:55:19.139 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:19.134+0000 7fe76dae5640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fe7680c8050 0x7fe768145490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:55:19.139 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:19.134+0000 7fe76dae5640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe7680ac960 con 0x7fe758004300 2023-08-09T13:55:19.139 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:19.134+0000 7fe76dae5640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe7680ab8c0 con 0x7fe7680c8050 2023-08-09T13:55:19.139 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:19.134+0000 7fe76dae5640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe7680abba0 con 0x7fe768146200 2023-08-09T13:55:19.139 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:19.135+0000 7fe76cae3640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fe758004300 0x7fe7680aafb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:55:19.139 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:19.135+0000 7fe76cae3640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fe758004300 0x7fe7680aafb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:36322/0 (socket says 172.21.15.43:36322) 2023-08-09T13:55:19.140 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:19.135+0000 7fe76cae3640 1 -- 172.21.15.43:0/3749165825 learned_addr learned my addr 172.21.15.43:0/3749165825 (peer_addr_for_me v2:172.21.15.43:0/0) 2023-08-09T13:55:19.140 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:19.135+0000 7fe767fff640 1 --2- 172.21.15.43:0/3749165825 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fe768146200 0x7fe7680c7b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:55:19.140 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:19.135+0000 7fe76d2e4640 1 --2- 172.21.15.43:0/3749165825 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fe7680c8050 0x7fe768145490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:55:19.140 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:19.135+0000 7fe76cae3640 1 -- 172.21.15.43:0/3749165825 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fe768146200 msgr2=0x7fe7680c7b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:55:19.140 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:19.135+0000 7fe76cae3640 1 --2- 172.21.15.43:0/3749165825 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fe768146200 0x7fe7680c7b10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:55:19.140 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:19.135+0000 7fe76cae3640 1 -- 172.21.15.43:0/3749165825 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fe7680c8050 msgr2=0x7fe768145490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:55:19.140 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:19.135+0000 7fe76cae3640 1 --2- 172.21.15.43:0/3749165825 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fe7680c8050 0x7fe768145490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:55:19.141 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:19.135+0000 7fe76cae3640 1 -- 172.21.15.43:0/3749165825 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe7680abe80 con 0x7fe758004300 2023-08-09T13:55:19.141 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:19.135+0000 7fe76d2e4640 1 --2- 172.21.15.43:0/3749165825 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fe7680c8050 0x7fe768145490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-08-09T13:55:19.141 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:19.135+0000 7fe76cae3640 1 --2- 172.21.15.43:0/3749165825 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fe758004300 0x7fe7680aafb0 secure :-1 s=READY pgs=1642 cs=0 l=1 rev1=1 crypto rx=0x7fe75c00c730 tx=0x7fe75c00cbf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-09T13:55:19.141 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:19.136+0000 7fe7657fa640 1 -- 172.21.15.43:0/3749165825 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe75c0074e0 con 0x7fe758004300 2023-08-09T13:55:19.141 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:19.136+0000 7fe7657fa640 1 -- 172.21.15.43:0/3749165825 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe75c007640 con 0x7fe758004300 2023-08-09T13:55:19.141 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:19.136+0000 7fe7657fa640 1 -- 172.21.15.43:0/3749165825 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe75c004a00 con 0x7fe758004300 2023-08-09T13:55:19.141 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:19.136+0000 7fe76dae5640 1 -- 172.21.15.43:0/3749165825 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe7680ac160 con 0x7fe758004300 2023-08-09T13:55:19.141 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:19.136+0000 7fe76dae5640 1 -- 172.21.15.43:0/3749165825 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe7680ab4f0 con 0x7fe758004300 2023-08-09T13:55:19.142 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:19.137+0000 7fe76dae5640 1 -- 172.21.15.43:0/3749165825 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe7680ac680 con 0x7fe758004300 2023-08-09T13:55:19.142 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:19.138+0000 7fe7657fa640 1 -- 172.21.15.43:0/3749165825 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 67) v1 ==== 88443+0+0 (secure 0 0 0) 0x7fe75c004b60 con 0x7fe758004300 2023-08-09T13:55:19.142 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:19.139+0000 7fe7657fa640 1 --2- 172.21.15.43:0/3749165825 >> [v2:172.21.15.203:6832/3490266242,v1:172.21.15.203:6833/3490266242] conn(0x7fe74c06a170 0x7fe74c06c630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:55:19.143 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:19.139+0000 7fe7657fa640 1 -- 172.21.15.43:0/3749165825 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(138..138 src has 1..138) v4 ==== 6434+0+0 (secure 0 0 0) 0x7fe75c087ce0 con 0x7fe758004300 2023-08-09T13:55:19.143 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:19.140+0000 7fe767fff640 1 --2- 172.21.15.43:0/3749165825 >> [v2:172.21.15.203:6832/3490266242,v1:172.21.15.203:6833/3490266242] conn(0x7fe74c06a170 0x7fe74c06c630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:55:19.143 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:19.141+0000 7fe767fff640 1 --2- 172.21.15.43:0/3749165825 >> [v2:172.21.15.203:6832/3490266242,v1:172.21.15.203:6833/3490266242] conn(0x7fe74c06a170 0x7fe74c06c630 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7fe754036000 tx=0x7fe754030000 comp rx=0 tx=0).ready entity=mgr.6659 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-09T13:55:19.146 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:19.144+0000 7fe7657fa640 1 -- 172.21.15.43:0/3749165825 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178066 (secure 0 0 0) 0x7fe7680ac680 con 0x7fe758004300 2023-08-09T13:55:19.415 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:19.412+0000 7fe76dae5640 1 -- 172.21.15.43:0/3749165825 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_snap_schedules.TestSnapSchedules.test_snap_schedule_for_number_of_snaps_retention"]} v 0) v1 -- 0x7fe7680ac680 con 0x7fe758004300 2023-08-09T13:55:19.878 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:19.875+0000 7fe7657fa640 1 -- 172.21.15.43:0/3749165825 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_snap_schedules.TestSnapSchedules.test_snap_schedule_for_number_of_snaps_retention"]}]=0 v837) v1 ==== 181+0+0 (secure 0 0 0) 0x7fe75c018020 con 0x7fe758004300 2023-08-09T13:55:19.881 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:19.879+0000 7fe76dae5640 1 -- 172.21.15.43:0/3749165825 >> [v2:172.21.15.203:6832/3490266242,v1:172.21.15.203:6833/3490266242] conn(0x7fe74c06a170 msgr2=0x7fe74c06c630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:55:19.882 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:19.879+0000 7fe76dae5640 1 --2- 172.21.15.43:0/3749165825 >> [v2:172.21.15.203:6832/3490266242,v1:172.21.15.203:6833/3490266242] conn(0x7fe74c06a170 0x7fe74c06c630 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7fe754036000 tx=0x7fe754030000 comp rx=0 tx=0).stop 2023-08-09T13:55:19.882 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:19.879+0000 7fe76dae5640 1 -- 172.21.15.43:0/3749165825 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fe758004300 msgr2=0x7fe7680aafb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:55:19.882 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:19.879+0000 7fe76dae5640 1 --2- 172.21.15.43:0/3749165825 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fe758004300 0x7fe7680aafb0 secure :-1 s=READY pgs=1642 cs=0 l=1 rev1=1 crypto rx=0x7fe75c00c730 tx=0x7fe75c00cbf0 comp rx=0 tx=0).stop 2023-08-09T13:55:19.882 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:19.879+0000 7fe76dae5640 1 -- 172.21.15.43:0/3749165825 shutdown_connections 2023-08-09T13:55:19.882 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:19.879+0000 7fe76dae5640 1 --2- 172.21.15.43:0/3749165825 >> [v2:172.21.15.203:6832/3490266242,v1:172.21.15.203:6833/3490266242] conn(0x7fe74c06a170 0x7fe74c06c630 unknown :-1 s=CLOSED pgs=39 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:55:19.882 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:19.879+0000 7fe76dae5640 1 --2- 172.21.15.43:0/3749165825 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fe7680c8050 0x7fe768145490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:55:19.883 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:19.879+0000 7fe76dae5640 1 --2- 172.21.15.43:0/3749165825 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fe768146200 0x7fe7680c7b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:55:19.883 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:19.879+0000 7fe76dae5640 1 --2- 172.21.15.43:0/3749165825 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fe758004300 0x7fe7680aafb0 unknown :-1 s=CLOSED pgs=1642 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:55:19.883 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:19.879+0000 7fe76dae5640 1 -- 172.21.15.43:0/3749165825 >> 172.21.15.43:0/3749165825 conn(0x7fe7680cad20 msgr2=0x7fe7680b10f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-09T13:55:19.883 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:19.879+0000 7fe76dae5640 1 -- 172.21.15.43:0/3749165825 shutdown_connections 2023-08-09T13:55:19.883 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:19.879+0000 7fe76dae5640 1 -- 172.21.15.43:0/3749165825 wait complete. 2023-08-09T13:55:19.905 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-08-09T13:55:20.142 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:20.140+0000 7f7a26ead640 1 Processor -- start 2023-08-09T13:55:20.142 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:20.140+0000 7f7a26ead640 1 -- start start 2023-08-09T13:55:20.144 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:20.142+0000 7f7a26ead640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f7a200b1050 0x7f7a200b4760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:55:20.144 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:20.142+0000 7f7a26ead640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f7a200cd6b0 0x7f7a200d0e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:55:20.144 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:20.142+0000 7f7a26ead640 1 -- --> v1:172.21.15.203:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7a200b7bf0 con 0x7f7a2015f7b0 2023-08-09T13:55:20.144 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:20.142+0000 7f7a26ead640 1 -- --> v2:172.21.15.43:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7a200b7ed0 con 0x7f7a200cd6b0 2023-08-09T13:55:20.145 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:20.142+0000 7f7a26ead640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7a200d0370 con 0x7f7a200b1050 2023-08-09T13:55:20.145 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:20.142+0000 7f7a266ac640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f7a200cd6b0 0x7f7a200d0e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:55:20.145 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:20.142+0000 7f7a266ac640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f7a200cd6b0 0x7f7a200d0e50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:36326/0 (socket says 172.21.15.43:36326) 2023-08-09T13:55:20.146 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:20.142+0000 7f7a266ac640 1 -- 172.21.15.43:0/1660370886 learned_addr learned my addr 172.21.15.43:0/1660370886 (peer_addr_for_me v2:172.21.15.43:0/0) 2023-08-09T13:55:20.146 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:20.142+0000 7f7a256aa640 1 --2- 172.21.15.43:0/1660370886 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f7a200b1050 0x7f7a200b4760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:55:20.146 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:20.143+0000 7f7a266ac640 1 -- 172.21.15.43:0/1660370886 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f7a200b1050 msgr2=0x7f7a200b4760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-09T13:55:20.146 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:20.143+0000 7f7a266ac640 1 --2- 172.21.15.43:0/1660370886 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f7a200b1050 0x7f7a200b4760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:55:20.146 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:20.143+0000 7f7a266ac640 1 -- 172.21.15.43:0/1660370886 >> v1:172.21.15.203:6789/0 conn(0x7f7a2015f7b0 legacy=0x7f7a200ccfa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-09T13:55:20.146 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:20.143+0000 7f7a266ac640 1 -- 172.21.15.43:0/1660370886 --> v2:172.21.15.43:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7a200d0650 con 0x7f7a200cd6b0 2023-08-09T13:55:20.147 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:20.144+0000 7f7a266ac640 1 --2- 172.21.15.43:0/1660370886 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f7a200cd6b0 0x7f7a200d0e50 secure :-1 s=READY pgs=1643 cs=0 l=1 rev1=1 crypto rx=0x7f7a18009870 tx=0x7f7a1802b8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=39b79bd9a3d45985 server_cookie=0 in_seq=0 out_seq=0 2023-08-09T13:55:20.147 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:20.144+0000 7f7a24ea9640 1 -- 172.21.15.43:0/1660370886 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7a18031640 con 0x7f7a200cd6b0 2023-08-09T13:55:20.147 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:20.144+0000 7f7a24ea9640 1 -- 172.21.15.43:0/1660370886 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7a180317a0 con 0x7f7a200cd6b0 2023-08-09T13:55:20.147 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:20.144+0000 7f7a24ea9640 1 -- 172.21.15.43:0/1660370886 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7a18031b30 con 0x7f7a200cd6b0 2023-08-09T13:55:20.148 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:20.145+0000 7f7a26ead640 1 -- 172.21.15.43:0/1660370886 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f7a200cd6b0 msgr2=0x7f7a200d0e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:55:20.148 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:20.145+0000 7f7a26ead640 1 --2- 172.21.15.43:0/1660370886 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f7a200cd6b0 0x7f7a200d0e50 secure :-1 s=READY pgs=1643 cs=0 l=1 rev1=1 crypto rx=0x7f7a18009870 tx=0x7f7a1802b8f0 comp rx=0 tx=0).stop 2023-08-09T13:55:20.148 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:20.145+0000 7f7a26ead640 1 -- 172.21.15.43:0/1660370886 shutdown_connections 2023-08-09T13:55:20.148 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:20.145+0000 7f7a26ead640 1 --2- 172.21.15.43:0/1660370886 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f7a200cd6b0 0x7f7a200d0e50 unknown :-1 s=CLOSED pgs=1643 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:55:20.149 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:20.145+0000 7f7a26ead640 1 --2- 172.21.15.43:0/1660370886 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f7a200b1050 0x7f7a200b4760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:55:20.149 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:20.145+0000 7f7a26ead640 1 -- 172.21.15.43:0/1660370886 >> 172.21.15.43:0/1660370886 conn(0x7f7a2006a2b0 msgr2=0x7f7a2006a690 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-09T13:55:20.149 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:20.145+0000 7f7a26ead640 1 -- 172.21.15.43:0/1660370886 shutdown_connections 2023-08-09T13:55:20.149 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:20.145+0000 7f7a26ead640 1 -- 172.21.15.43:0/1660370886 wait complete. 2023-08-09T13:55:20.150 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:20.148+0000 7f7a26ead640 1 Processor -- start 2023-08-09T13:55:20.150 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:20.148+0000 7f7a26ead640 1 -- start start 2023-08-09T13:55:20.152 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:20.149+0000 7f7a26ead640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f7a200b1050 0x7f7a2014bb20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:55:20.152 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:20.149+0000 7f7a26ead640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f7a200cd6b0 0x7f7a200b2bb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:55:20.152 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:20.149+0000 7f7a26ead640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f7a2015f7b0 0x7f7a200b30f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:55:20.152 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:20.149+0000 7f7a26ead640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7a2014c060 con 0x7f7a200cd6b0 2023-08-09T13:55:20.152 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:20.149+0000 7f7a26ead640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7a20159140 con 0x7f7a2015f7b0 2023-08-09T13:55:20.152 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:20.149+0000 7f7a26ead640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7a20159420 con 0x7f7a200b1050 2023-08-09T13:55:20.154 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:20.150+0000 7f7a256aa640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f7a200cd6b0 0x7f7a200b2bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:55:20.154 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:20.150+0000 7f7a256aa640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f7a200cd6b0 0x7f7a200b2bb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:36342/0 (socket says 172.21.15.43:36342) 2023-08-09T13:55:20.154 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:20.150+0000 7f7a256aa640 1 -- 172.21.15.43:0/3796906023 learned_addr learned my addr 172.21.15.43:0/3796906023 (peer_addr_for_me v2:172.21.15.43:0/0) 2023-08-09T13:55:20.154 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:20.150+0000 7f7a266ac640 1 --2- 172.21.15.43:0/3796906023 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f7a2015f7b0 0x7f7a200b30f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:55:20.154 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:20.150+0000 7f7a25eab640 1 --2- 172.21.15.43:0/3796906023 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f7a200b1050 0x7f7a2014bb20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:55:20.154 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:20.150+0000 7f7a256aa640 1 -- 172.21.15.43:0/3796906023 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f7a200b1050 msgr2=0x7f7a2014bb20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:55:20.154 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:20.150+0000 7f7a256aa640 1 --2- 172.21.15.43:0/3796906023 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f7a200b1050 0x7f7a2014bb20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:55:20.155 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:20.150+0000 7f7a256aa640 1 -- 172.21.15.43:0/3796906023 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f7a2015f7b0 msgr2=0x7f7a200b30f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:55:20.155 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:20.150+0000 7f7a256aa640 1 --2- 172.21.15.43:0/3796906023 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f7a2015f7b0 0x7f7a200b30f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:55:20.155 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:20.150+0000 7f7a256aa640 1 -- 172.21.15.43:0/3796906023 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7a180094d0 con 0x7f7a200cd6b0 2023-08-09T13:55:20.155 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:20.150+0000 7f7a266ac640 1 --2- 172.21.15.43:0/3796906023 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f7a2015f7b0 0x7f7a200b30f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-08-09T13:55:20.155 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:20.151+0000 7f7a256aa640 1 --2- 172.21.15.43:0/3796906023 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f7a200cd6b0 0x7f7a200b2bb0 secure :-1 s=READY pgs=1644 cs=0 l=1 rev1=1 crypto rx=0x7f7a1000d7d0 tx=0x7f7a1000dc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-09T13:55:20.155 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:20.151+0000 7f7a167fc640 1 -- 172.21.15.43:0/3796906023 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7a1000b6c0 con 0x7f7a200cd6b0 2023-08-09T13:55:20.155 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:20.151+0000 7f7a167fc640 1 -- 172.21.15.43:0/3796906023 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7a1000b820 con 0x7f7a200cd6b0 2023-08-09T13:55:20.155 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:20.151+0000 7f7a167fc640 1 -- 172.21.15.43:0/3796906023 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7a1000bbb0 con 0x7f7a200cd6b0 2023-08-09T13:55:20.156 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:20.151+0000 7f7a26ead640 1 -- 172.21.15.43:0/3796906023 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7a20159700 con 0x7f7a200cd6b0 2023-08-09T13:55:20.156 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:20.152+0000 7f7a26ead640 1 -- 172.21.15.43:0/3796906023 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7a201583f0 con 0x7f7a200cd6b0 2023-08-09T13:55:20.156 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:20.153+0000 7f7a26ead640 1 -- 172.21.15.43:0/3796906023 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7a20159940 con 0x7f7a200cd6b0 2023-08-09T13:55:20.156 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:20.154+0000 7f7a167fc640 1 -- 172.21.15.43:0/3796906023 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 67) v1 ==== 88443+0+0 (secure 0 0 0) 0x7f7a1000bd10 con 0x7f7a200cd6b0 2023-08-09T13:55:20.157 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:20.155+0000 7f7a167fc640 1 --2- 172.21.15.43:0/3796906023 >> [v2:172.21.15.203:6832/3490266242,v1:172.21.15.203:6833/3490266242] conn(0x7f7a0406a170 0x7f7a0406c630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:55:20.157 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:20.155+0000 7f7a167fc640 1 -- 172.21.15.43:0/3796906023 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(138..138 src has 1..138) v4 ==== 6434+0+0 (secure 0 0 0) 0x7f7a10088dc0 con 0x7f7a200cd6b0 2023-08-09T13:55:20.158 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:20.156+0000 7f7a25eab640 1 --2- 172.21.15.43:0/3796906023 >> [v2:172.21.15.203:6832/3490266242,v1:172.21.15.203:6833/3490266242] conn(0x7f7a0406a170 0x7f7a0406c630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:55:20.159 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:20.157+0000 7f7a25eab640 1 --2- 172.21.15.43:0/3796906023 >> [v2:172.21.15.203:6832/3490266242,v1:172.21.15.203:6833/3490266242] conn(0x7f7a0406a170 0x7f7a0406c630 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7f7a1c009f20 tx=0x7f7a1c0094d0 comp rx=0 tx=0).ready entity=mgr.6659 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-09T13:55:20.161 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:20.159+0000 7f7a167fc640 1 -- 172.21.15.43:0/3796906023 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178066 (secure 0 0 0) 0x7f7a20159940 con 0x7f7a200cd6b0 2023-08-09T13:55:20.359 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:20.357+0000 7f7a26ead640 1 -- 172.21.15.43:0/3796906023 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f7a20159940 con 0x7f7a200cd6b0 2023-08-09T13:55:20.362 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:20.359+0000 7f7a167fc640 1 -- 172.21.15.43:0/3796906023 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v138) v1 ==== 74+0+16125 (secure 0 0 0) 0x7f7a20159940 con 0x7f7a200cd6b0 2023-08-09T13:55:20.362 INFO:teuthology.orchestra.run.smithi043.stdout: 2023-08-09T13:55:20.363 INFO:teuthology.orchestra.run.smithi043.stdout:{"epoch":138,"fsid":"daac87ee-1337-4250-a955-aea4e408a39b","created":"2023-08-09T13:33:41.701198+0000","modified":"2023-08-09T13:55:16.858544+0000","last_up_change":"2023-08-09T13:33:53.783453+0000","last_in_change":"2023-08-09T13:33:45.148904+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":17,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":"rbd","create_time":"2023-08-09T13:33:54.365137+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":".mgr","create_time":"2023-08-09T13:33:54.831803+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"cbd49ecd-e896-4fbd-864e-5c37a829b165","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":127,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6800","nonce":502667335},{"type":"v1","addr":"172.21.15.43:6801","nonce":502667335}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6802","nonce":502667335},{"type":"v1","addr":"172.21.15.43:6803","nonce":502667335}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6806","nonce":502667335},{"type":"v1","addr":"172.21.15.43:6807","nonce":502667335}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6804","nonce":502667335},{"type":"v1","addr":"172.21.15.43:6805","nonce":502667335}]},"public_addr":"172.21.15.43:6801/502667335","cluster_addr":"172.21.15.43:6803/502667335","heartbeat_back_addr":"172.21.15.43:6807/502667335","heartbeat_front_addr":"172.21.15.43:6805/502667335","state":["exists","up"]},{"osd":1,"uuid":"75c7e0fe-1ec7-4fe8-a2f2-3661a65b8500","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":131,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6808","nonce":3289134610},{"type":"v1","addr":"172.21.15.43:6809","nonce":3289134610}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6810","nonce":3289134610},{"type":"v1","addr":"172.21.15.43:6812","nonce":3289134610}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6821","nonce":3289134610},{"type":"v1","addr":"172.21.15.43:6824","nonce":3289134610}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6815","nonce":3289134610},{"type":"v1","addr":"172.21.15.43:6818","nonce":3289134610}]},"public_addr":"172.21.15.43:6809/3289134610","cluster_addr":"172.21.15.43:6812/3289134610","heartbeat_back_addr":"172.21.15.43:6824/3289134610","heartbeat_front_addr":"172.21.15.43:6818/3289134610","state":["exists","up"]},{"osd":2,"uuid":"bb5c41e9-7f5f-4125-9b01-3bb8bfc2ea83","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":127,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6811","nonce":4079709736},{"type":"v1","addr":"172.21.15.43:6814","nonce":4079709736}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6817","nonce":4079709736},{"type":"v1","addr":"172.21.15.43:6820","nonce":4079709736}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6829","nonce":4079709736},{"type":"v1","addr":"172.21.15.43:6831","nonce":4079709736}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6823","nonce":4079709736},{"type":"v1","addr":"172.21.15.43:6826","nonce":4079709736}]},"public_addr":"172.21.15.43:6814/4079709736","cluster_addr":"172.21.15.43:6820/4079709736","heartbeat_back_addr":"172.21.15.43:6831/4079709736","heartbeat_front_addr":"172.21.15.43:6826/4079709736","state":["exists","up"]},{"osd":3,"uuid":"bfe4c8b4-694a-465c-87e6-2b2aab852613","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":127,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6813","nonce":1709538759},{"type":"v1","addr":"172.21.15.43:6816","nonce":1709538759}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6819","nonce":1709538759},{"type":"v1","addr":"172.21.15.43:6822","nonce":1709538759}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6828","nonce":1709538759},{"type":"v1","addr":"172.21.15.43:6830","nonce":1709538759}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6825","nonce":1709538759},{"type":"v1","addr":"172.21.15.43:6827","nonce":1709538759}]},"public_addr":"172.21.15.43:6816/1709538759","cluster_addr":"172.21.15.43:6822/1709538759","heartbeat_back_addr":"172.21.15.43:6830/1709538759","heartbeat_front_addr":"172.21.15.43:6827/1709538759","state":["exists","up"]},{"osd":4,"uuid":"fa2e8a77-aba1-4756-853e-0d174a5bdeba","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":127,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6800","nonce":4196866481},{"type":"v1","addr":"172.21.15.203:6801","nonce":4196866481}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6802","nonce":4196866481},{"type":"v1","addr":"172.21.15.203:6803","nonce":4196866481}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6806","nonce":4196866481},{"type":"v1","addr":"172.21.15.203:6807","nonce":4196866481}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6804","nonce":4196866481},{"type":"v1","addr":"172.21.15.203:6805","nonce":4196866481}]},"public_addr":"172.21.15.203:6801/4196866481","cluster_addr":"172.21.15.203:6803/4196866481","heartbeat_back_addr":"172.21.15.203:6807/4196866481","heartbeat_front_addr":"172.21.15.203:6805/4196866481","state":["exists","up"]},{"osd":5,"uuid":"81811e4b-4d24-4f51-a82c-f1d700a2a00d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":127,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6816","nonce":3059290081},{"type":"v1","addr":"172.21.15.203:6817","nonce":3059290081}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6818","nonce":3059290081},{"type":"v1","addr":"172.21.15.203:6819","nonce":3059290081}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6822","nonce":3059290081},{"type":"v1","addr":"172.21.15.203:6823","nonce":3059290081}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6820","nonce":3059290081},{"type":"v1","addr":"172.21.15.203:6821","nonce":3059290081}]},"public_addr":"172.21.15.203:6817/3059290081","cluster_addr":"172.21.15.203:6819/3059290081","heartbeat_back_addr":"172.21.15.203:6823/3059290081","heartbeat_front_addr":"172.21.15.203:6821/3059290081","state":["exists","up"]},{"osd":6,"uuid":"307bac4d-6a5b-443d-af75-56d695d7674f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":131,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6824","nonce":512155569},{"type":"v1","addr":"172.21.15.203:6825","nonce":512155569}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6826","nonce":512155569},{"type":"v1","addr":"172.21.15.203:6827","nonce":512155569}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6830","nonce":512155569},{"type":"v1","addr":"172.21.15.203:6831","nonce":512155569}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6828","nonce":512155569},{"type":"v1","addr":"172.21.15.203:6829","nonce":512155569}]},"public_addr":"172.21.15.203:6825/512155569","cluster_addr":"172.21.15.203:6827/512155569","heartbeat_back_addr":"172.21.15.203:6831/512155569","heartbeat_front_addr":"172.21.15.203:6829/512155569","state":["exists","up"]},{"osd":7,"uuid":"a95dc7ca-9f1b-4295-af70-68a22a2d7d5e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":127,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6808","nonce":3331450438},{"type":"v1","addr":"172.21.15.203:6809","nonce":3331450438}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6810","nonce":3331450438},{"type":"v1","addr":"172.21.15.203:6811","nonce":3331450438}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6814","nonce":3331450438},{"type":"v1","addr":"172.21.15.203:6815","nonce":3331450438}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6812","nonce":3331450438},{"type":"v1","addr":"172.21.15.203:6813","nonce":3331450438}]},"public_addr":"172.21.15.203:6809/3331450438","cluster_addr":"172.21.15.203:6811/3331450438","heartbeat_back_addr":"172.21.15.203:6815/3331450438","heartbeat_front_addr":"172.21.15.203:6813/3331450438","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-09T13:33:50.547652+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-09T13:33:50.589469+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-09T13:33:50.561724+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-09T13:33:50.599650+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-09T13:33:50.411855+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-09T13:33:50.219681+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-09T13:33:50.504664+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-09T13:33:50.441248+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.43:6835/1794836673":"2023-08-10T13:55:13.847203+0000","172.21.15.43:6834/1794836673":"2023-08-10T13:55:13.847203+0000","172.21.15.203:0/1151006972":"2023-08-10T13:55:08.528494+0000","172.21.15.203:0/274636031":"2023-08-10T13:55:08.528494+0000","172.21.15.203:0/3526078924":"2023-08-10T13:55:08.528494+0000","172.21.15.203:6833/1796635339":"2023-08-10T13:55:08.528494+0000","172.21.15.203:6832/1796635339":"2023-08-10T13:55:08.528494+0000","172.21.15.203:0/1153043111":"2023-08-10T13:55:08.528494+0000","172.21.15.203:0/3657978600":"2023-08-10T13:55:08.528494+0000","172.21.15.203:0/3093730384":"2023-08-10T13:55:08.528494+0000","172.21.15.203:6833/2527059680":"2023-08-10T13:54:03.238790+0000","172.21.15.203:6832/2527059680":"2023-08-10T13:54:03.238790+0000","172.21.15.203:0/431304609":"2023-08-10T13:54:03.238790+0000","172.21.15.203:0/1911913635":"2023-08-10T13:55:08.528494+0000","172.21.15.203:0/826469783":"2023-08-10T13:54:03.238790+0000","172.21.15.203:0/1225835978":"2023-08-10T13:55:08.528494+0000","172.21.15.203:0/1456953448":"2023-08-10T13:54:03.238790+0000","172.21.15.203:0/2004944696":"2023-08-10T13:54:03.238790+0000","172.21.15.203:0/916024371":"2023-08-10T13:54:03.238790+0000","172.21.15.203:0/4209557762":"2023-08-10T13:54:03.238790+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":7,"snaps":[{"begin":104,"length":1},{"begin":221,"length":1}]}],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-08-09T13:55:20.365 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:20.362+0000 7f7a26ead640 1 -- 172.21.15.43:0/3796906023 >> [v2:172.21.15.203:6832/3490266242,v1:172.21.15.203:6833/3490266242] conn(0x7f7a0406a170 msgr2=0x7f7a0406c630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:55:20.365 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:20.362+0000 7f7a26ead640 1 --2- 172.21.15.43:0/3796906023 >> [v2:172.21.15.203:6832/3490266242,v1:172.21.15.203:6833/3490266242] conn(0x7f7a0406a170 0x7f7a0406c630 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7f7a1c009f20 tx=0x7f7a1c0094d0 comp rx=0 tx=0).stop 2023-08-09T13:55:20.365 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:20.362+0000 7f7a26ead640 1 -- 172.21.15.43:0/3796906023 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f7a200cd6b0 msgr2=0x7f7a200b2bb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:55:20.366 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:20.362+0000 7f7a26ead640 1 --2- 172.21.15.43:0/3796906023 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f7a200cd6b0 0x7f7a200b2bb0 secure :-1 s=READY pgs=1644 cs=0 l=1 rev1=1 crypto rx=0x7f7a1000d7d0 tx=0x7f7a1000dc90 comp rx=0 tx=0).stop 2023-08-09T13:55:20.366 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:20.362+0000 7f7a26ead640 1 -- 172.21.15.43:0/3796906023 shutdown_connections 2023-08-09T13:55:20.366 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:20.362+0000 7f7a26ead640 1 --2- 172.21.15.43:0/3796906023 >> [v2:172.21.15.203:6832/3490266242,v1:172.21.15.203:6833/3490266242] conn(0x7f7a0406a170 0x7f7a0406c630 unknown :-1 s=CLOSED pgs=40 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:55:20.366 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:20.362+0000 7f7a26ead640 1 --2- 172.21.15.43:0/3796906023 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f7a2015f7b0 0x7f7a200b30f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:55:20.366 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:20.362+0000 7f7a26ead640 1 --2- 172.21.15.43:0/3796906023 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f7a200cd6b0 0x7f7a200b2bb0 unknown :-1 s=CLOSED pgs=1644 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:55:20.366 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:20.362+0000 7f7a26ead640 1 --2- 172.21.15.43:0/3796906023 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f7a200b1050 0x7f7a2014bb20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:55:20.366 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:20.362+0000 7f7a26ead640 1 -- 172.21.15.43:0/3796906023 >> 172.21.15.43:0/3796906023 conn(0x7f7a2006a2b0 msgr2=0x7f7a2015a8c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-09T13:55:20.366 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:20.362+0000 7f7a26ead640 1 -- 172.21.15.43:0/3796906023 shutdown_connections 2023-08-09T13:55:20.367 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:20.363+0000 7f7a26ead640 1 -- 172.21.15.43:0/3796906023 wait complete. 2023-08-09T13:55:20.383 INFO:tasks.cephfs.cephfs_test_case.ceph_manager:[{'pool': 1, 'pool_name': 'rbd', 'create_time': '2023-08-09T13:33:54.365137+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}, 'read_balance': {'score_acting': 2, 'score_stable': 2, 'optimal_score': 1, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': '.mgr', 'create_time': '2023-08-09T13:33:54.831803+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-08-09T13:55:20.384 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get rbd pg_num 2023-08-09T13:55:20.617 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:20.615+0000 7f1ac1f62640 1 Processor -- start 2023-08-09T13:55:20.618 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:20.615+0000 7f1ac1f62640 1 -- start start 2023-08-09T13:55:20.620 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:20.616+0000 7f1ac1f62640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f1abc157ea0 0x7f1abc0c97d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:55:20.620 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:20.616+0000 7f1ac1f62640 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7f1abc0c90a0 0x7f1abc0c9d10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:55:20.621 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:20.617+0000 7f1ac1f62640 1 -- --> v1:172.21.15.203:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1abc158280 con 0x7f1abc15a800 2023-08-09T13:55:20.621 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:20.617+0000 7f1ac1f62640 1 -- --> v2:172.21.15.43:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1abc15b840 con 0x7f1abc157ea0 2023-08-09T13:55:20.621 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:20.617+0000 7f1ac1f62640 1 -- --> v2:172.21.15.203:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1abc0b3810 con 0x7f1abc0c90a0 2023-08-09T13:55:20.622 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:20.617+0000 7f1ac0f60640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f1abc157ea0 0x7f1abc0c97d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:55:20.622 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:20.617+0000 7f1ac0f60640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f1abc157ea0 0x7f1abc0c97d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:36350/0 (socket says 172.21.15.43:36350) 2023-08-09T13:55:20.622 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:20.617+0000 7f1ac0f60640 1 -- 172.21.15.43:0/3483024568 learned_addr learned my addr 172.21.15.43:0/3483024568 (peer_addr_for_me v2:172.21.15.43:0/0) 2023-08-09T13:55:20.622 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:20.617+0000 7f1abbfff640 1 --2- 172.21.15.43:0/3483024568 >> v2:172.21.15.203:3300/0 conn(0x7f1abc0c90a0 0x7f1abc0c9d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:55:20.623 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:20.617+0000 7f1ac0f60640 1 -- 172.21.15.43:0/3483024568 >> v2:172.21.15.203:3300/0 conn(0x7f1abc0c90a0 msgr2=0x7f1abc0c9d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-09T13:55:20.623 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:20.617+0000 7f1ac0f60640 1 --2- 172.21.15.43:0/3483024568 >> v2:172.21.15.203:3300/0 conn(0x7f1abc0c90a0 0x7f1abc0c9d10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:55:20.623 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:20.617+0000 7f1ac0f60640 1 -- 172.21.15.43:0/3483024568 >> v1:172.21.15.203:6789/0 conn(0x7f1abc15a800 legacy=0x7f1abc15ac90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-09T13:55:20.623 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:20.617+0000 7f1ac0f60640 1 -- 172.21.15.43:0/3483024568 --> v2:172.21.15.43:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1abc15b560 con 0x7f1abc157ea0 2023-08-09T13:55:20.624 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:20.617+0000 7f1abbfff640 1 --2- 172.21.15.43:0/3483024568 >> v2:172.21.15.203:3300/0 conn(0x7f1abc0c90a0 0x7f1abc0c9d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-08-09T13:55:20.625 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:20.619+0000 7f1ac0f60640 1 --2- 172.21.15.43:0/3483024568 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f1abc157ea0 0x7f1abc0c97d0 secure :-1 s=READY pgs=1645 cs=0 l=1 rev1=1 crypto rx=0x7f1abc0ceea0 tx=0x7f1ab002b880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f726c7c5c973e0e3 server_cookie=0 in_seq=0 out_seq=0 2023-08-09T13:55:20.626 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:20.619+0000 7f1abb7fe640 1 -- 172.21.15.43:0/3483024568 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1ab00317e0 con 0x7f1abc157ea0 2023-08-09T13:55:20.626 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:20.619+0000 7f1abb7fe640 1 -- 172.21.15.43:0/3483024568 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1ab0031940 con 0x7f1abc157ea0 2023-08-09T13:55:20.626 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:20.619+0000 7f1abb7fe640 1 -- 172.21.15.43:0/3483024568 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1ab0031cd0 con 0x7f1abc157ea0 2023-08-09T13:55:20.626 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:20.620+0000 7f1ac1f62640 1 -- 172.21.15.43:0/3483024568 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f1abc157ea0 msgr2=0x7f1abc0c97d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:55:20.626 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:20.620+0000 7f1ac1f62640 1 --2- 172.21.15.43:0/3483024568 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f1abc157ea0 0x7f1abc0c97d0 secure :-1 s=READY pgs=1645 cs=0 l=1 rev1=1 crypto rx=0x7f1abc0ceea0 tx=0x7f1ab002b880 comp rx=0 tx=0).stop 2023-08-09T13:55:20.626 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:20.620+0000 7f1ac1f62640 1 -- 172.21.15.43:0/3483024568 shutdown_connections 2023-08-09T13:55:20.627 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:20.620+0000 7f1ac1f62640 1 --2- 172.21.15.43:0/3483024568 >> v2:172.21.15.203:3300/0 conn(0x7f1abc0c90a0 0x7f1abc0c9d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:55:20.627 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:20.620+0000 7f1ac1f62640 1 --2- 172.21.15.43:0/3483024568 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f1abc157ea0 0x7f1abc0c97d0 unknown :-1 s=CLOSED pgs=1645 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:55:20.627 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:20.620+0000 7f1ac1f62640 1 -- 172.21.15.43:0/3483024568 >> 172.21.15.43:0/3483024568 conn(0x7f1abc0cddd0 msgr2=0x7f1abc0ce1e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-09T13:55:20.627 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:20.620+0000 7f1ac1f62640 1 -- 172.21.15.43:0/3483024568 shutdown_connections 2023-08-09T13:55:20.627 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:20.620+0000 7f1ac1f62640 1 -- 172.21.15.43:0/3483024568 wait complete. 2023-08-09T13:55:20.627 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:20.622+0000 7f1ac1f62640 1 Processor -- start 2023-08-09T13:55:20.627 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:20.622+0000 7f1ac1f62640 1 -- start start 2023-08-09T13:55:20.628 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:20.623+0000 7f1ac1f62640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f1abc0c90a0 0x7f1abc1f43e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:55:20.628 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:20.623+0000 7f1ac1f62640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f1abc157ea0 0x7f1abc1f1580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:55:20.628 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:20.623+0000 7f1ac1f62640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f1abc15a800 0x7f1abc1f1ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:55:20.628 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:20.623+0000 7f1ac1f62640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1abc1f4920 con 0x7f1abc15a800 2023-08-09T13:55:20.628 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:20.623+0000 7f1ac1f62640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1abc0cc8d0 con 0x7f1abc0c90a0 2023-08-09T13:55:20.628 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:20.623+0000 7f1ac1f62640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1abc0ccbb0 con 0x7f1abc157ea0 2023-08-09T13:55:20.630 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:20.624+0000 7f1ac1761640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f1abc15a800 0x7f1abc1f1ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:55:20.630 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:20.624+0000 7f1ac1761640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f1abc15a800 0x7f1abc1f1ac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:36360/0 (socket says 172.21.15.43:36360) 2023-08-09T13:55:20.630 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:20.624+0000 7f1ac1761640 1 -- 172.21.15.43:0/1251503197 learned_addr learned my addr 172.21.15.43:0/1251503197 (peer_addr_for_me v2:172.21.15.43:0/0) 2023-08-09T13:55:20.630 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:20.624+0000 7f1ac0f60640 1 --2- 172.21.15.43:0/1251503197 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f1abc0c90a0 0x7f1abc1f43e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:55:20.631 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:20.624+0000 7f1abbfff640 1 --2- 172.21.15.43:0/1251503197 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f1abc157ea0 0x7f1abc1f1580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:55:20.631 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:20.624+0000 7f1ac1761640 1 -- 172.21.15.43:0/1251503197 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f1abc157ea0 msgr2=0x7f1abc1f1580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:55:20.631 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:20.624+0000 7f1ac1761640 1 --2- 172.21.15.43:0/1251503197 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f1abc157ea0 0x7f1abc1f1580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:55:20.631 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:20.624+0000 7f1ac1761640 1 -- 172.21.15.43:0/1251503197 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f1abc0c90a0 msgr2=0x7f1abc1f43e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:55:20.631 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:20.624+0000 7f1ac1761640 1 --2- 172.21.15.43:0/1251503197 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f1abc0c90a0 0x7f1abc1f43e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:55:20.631 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:20.624+0000 7f1ac1761640 1 -- 172.21.15.43:0/1251503197 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1ab00094d0 con 0x7f1abc15a800 2023-08-09T13:55:20.631 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:20.624+0000 7f1ac0f60640 1 --2- 172.21.15.43:0/1251503197 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f1abc0c90a0 0x7f1abc1f43e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-08-09T13:55:20.631 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:20.624+0000 7f1ac1761640 1 --2- 172.21.15.43:0/1251503197 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f1abc15a800 0x7f1abc1f1ac0 secure :-1 s=READY pgs=1646 cs=0 l=1 rev1=1 crypto rx=0x7f1ab400b990 tx=0x7f1ab400be50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-09T13:55:20.632 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:20.625+0000 7f1ab97fa640 1 -- 172.21.15.43:0/1251503197 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1ab400cba0 con 0x7f1abc15a800 2023-08-09T13:55:20.632 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:20.625+0000 7f1ab97fa640 1 -- 172.21.15.43:0/1251503197 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1ab400cd00 con 0x7f1abc15a800 2023-08-09T13:55:20.632 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:20.625+0000 7f1ab97fa640 1 -- 172.21.15.43:0/1251503197 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1ab4014680 con 0x7f1abc15a800 2023-08-09T13:55:20.632 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:20.625+0000 7f1ac1f62640 1 -- 172.21.15.43:0/1251503197 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1abc0cce90 con 0x7f1abc15a800 2023-08-09T13:55:20.632 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:20.625+0000 7f1ac1f62640 1 -- 172.21.15.43:0/1251503197 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1abc0cd450 con 0x7f1abc15a800 2023-08-09T13:55:20.632 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:20.626+0000 7f1ac1f62640 1 -- 172.21.15.43:0/1251503197 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1abc1f2700 con 0x7f1abc15a800 2023-08-09T13:55:20.633 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:20.627+0000 7f1ab97fa640 1 -- 172.21.15.43:0/1251503197 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 67) v1 ==== 88443+0+0 (secure 0 0 0) 0x7f1ab40147e0 con 0x7f1abc15a800 2023-08-09T13:55:20.633 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:20.628+0000 7f1ab97fa640 1 --2- 172.21.15.43:0/1251503197 >> [v2:172.21.15.203:6832/3490266242,v1:172.21.15.203:6833/3490266242] conn(0x7f1aa006a170 0x7f1aa006c630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:55:20.633 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:20.628+0000 7f1ab97fa640 1 -- 172.21.15.43:0/1251503197 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(138..138 src has 1..138) v4 ==== 6434+0+0 (secure 0 0 0) 0x7f1ab40888c0 con 0x7f1abc15a800 2023-08-09T13:55:20.634 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:20.629+0000 7f1ac0f60640 1 --2- 172.21.15.43:0/1251503197 >> [v2:172.21.15.203:6832/3490266242,v1:172.21.15.203:6833/3490266242] conn(0x7f1aa006a170 0x7f1aa006c630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:55:20.634 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:20.630+0000 7f1ac0f60640 1 --2- 172.21.15.43:0/1251503197 >> [v2:172.21.15.203:6832/3490266242,v1:172.21.15.203:6833/3490266242] conn(0x7f1aa006a170 0x7f1aa006c630 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7f1ab0036000 tx=0x7f1ab003d040 comp rx=0 tx=0).ready entity=mgr.6659 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-09T13:55:20.634 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:20.632+0000 7f1ab97fa640 1 -- 172.21.15.43:0/1251503197 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178066 (secure 0 0 0) 0x7f1ab4053c80 con 0x7f1abc15a800 2023-08-09T13:55:20.827 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:20.824+0000 7f1ac1f62640 1 -- 172.21.15.43:0/1251503197 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"} v 0) v1 -- 0x7f1abc04e390 con 0x7f1abc15a800 2023-08-09T13:55:20.827 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:20.825+0000 7f1ab97fa640 1 -- 172.21.15.43:0/1251503197 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"}]=0 v138) v1 ==== 92+0+10 (secure 0 0 0) 0x7f1ab4019020 con 0x7f1abc15a800 2023-08-09T13:55:20.828 INFO:teuthology.orchestra.run.smithi043.stdout:pg_num: 8 2023-08-09T13:55:20.831 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:20.828+0000 7f1ac1f62640 1 -- 172.21.15.43:0/1251503197 >> [v2:172.21.15.203:6832/3490266242,v1:172.21.15.203:6833/3490266242] conn(0x7f1aa006a170 msgr2=0x7f1aa006c630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:55:20.831 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:20.828+0000 7f1ac1f62640 1 --2- 172.21.15.43:0/1251503197 >> [v2:172.21.15.203:6832/3490266242,v1:172.21.15.203:6833/3490266242] conn(0x7f1aa006a170 0x7f1aa006c630 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7f1ab0036000 tx=0x7f1ab003d040 comp rx=0 tx=0).stop 2023-08-09T13:55:20.831 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:20.828+0000 7f1ac1f62640 1 -- 172.21.15.43:0/1251503197 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f1abc15a800 msgr2=0x7f1abc1f1ac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:55:20.831 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:20.828+0000 7f1ac1f62640 1 --2- 172.21.15.43:0/1251503197 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f1abc15a800 0x7f1abc1f1ac0 secure :-1 s=READY pgs=1646 cs=0 l=1 rev1=1 crypto rx=0x7f1ab400b990 tx=0x7f1ab400be50 comp rx=0 tx=0).stop 2023-08-09T13:55:20.831 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:20.828+0000 7f1ac1f62640 1 -- 172.21.15.43:0/1251503197 shutdown_connections 2023-08-09T13:55:20.832 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:20.828+0000 7f1ac1f62640 1 --2- 172.21.15.43:0/1251503197 >> [v2:172.21.15.203:6832/3490266242,v1:172.21.15.203:6833/3490266242] conn(0x7f1aa006a170 0x7f1aa006c630 unknown :-1 s=CLOSED pgs=41 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:55:20.832 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:20.828+0000 7f1ac1f62640 1 --2- 172.21.15.43:0/1251503197 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f1abc15a800 0x7f1abc1f1ac0 unknown :-1 s=CLOSED pgs=1646 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:55:20.832 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:20.828+0000 7f1ac1f62640 1 --2- 172.21.15.43:0/1251503197 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f1abc157ea0 0x7f1abc1f1580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:55:20.832 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:20.828+0000 7f1ac1f62640 1 --2- 172.21.15.43:0/1251503197 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f1abc0c90a0 0x7f1abc1f43e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:55:20.832 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:20.828+0000 7f1ac1f62640 1 -- 172.21.15.43:0/1251503197 >> 172.21.15.43:0/1251503197 conn(0x7f1abc0cddd0 msgr2=0x7f1abc06b3e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-09T13:55:20.832 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:20.828+0000 7f1ac1f62640 1 -- 172.21.15.43:0/1251503197 shutdown_connections 2023-08-09T13:55:20.832 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:20.829+0000 7f1ac1f62640 1 -- 172.21.15.43:0/1251503197 wait complete. 2023-08-09T13:55:20.850 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-08-09T13:55:21.090 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:21.087+0000 7f7ff6153640 1 Processor -- start 2023-08-09T13:55:21.090 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:21.087+0000 7f7ff6153640 1 -- start start 2023-08-09T13:55:21.091 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:21.089+0000 7f7ff6153640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f7ff0157ea0 0x7f7ff00c97d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:55:21.092 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:21.089+0000 7f7ff6153640 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7f7ff00c90a0 0x7f7ff00c9d10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:55:21.092 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:21.089+0000 7f7ff6153640 1 -- --> v1:172.21.15.43:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7ff0158280 con 0x7f7ff015a800 2023-08-09T13:55:21.092 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:21.089+0000 7f7ff6153640 1 -- --> v2:172.21.15.203:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7ff015b840 con 0x7f7ff00c90a0 2023-08-09T13:55:21.092 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:21.089+0000 7f7ff6153640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7ff015b350 con 0x7f7ff0157ea0 2023-08-09T13:55:21.094 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:21.089+0000 7f7ff5952640 1 --1- >> v1:172.21.15.43:6789/0 conn(0x7f7ff015a800 0x7f7ff015ac40 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.43:6789/0 says I am v1:172.21.15.43:60946/0 (socket says 172.21.15.43:60946) 2023-08-09T13:55:21.094 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:21.089+0000 7f7ff5952640 1 -- 172.21.15.43:0/2129940785 learned_addr learned my addr 172.21.15.43:0/2129940785 (peer_addr_for_me v1:172.21.15.43:0/0) 2023-08-09T13:55:21.095 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:21.090+0000 7f7ff5151640 1 --2- 172.21.15.43:0/2129940785 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f7ff0157ea0 0x7f7ff00c97d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:55:21.095 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:21.090+0000 7f7ff4950640 1 --2- 172.21.15.43:0/2129940785 >> v2:172.21.15.203:3300/0 conn(0x7f7ff00c90a0 0x7f7ff00c9d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:55:21.095 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:21.090+0000 7f7fe7fff640 1 -- 172.21.15.43:0/2129940785 <== mon.0 v1:172.21.15.43:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 451614487 0 0) 0x7f7fe8001490 con 0x7f7ff015a800 2023-08-09T13:55:21.095 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:21.090+0000 7f7fe7fff640 1 -- 172.21.15.43:0/2129940785 --> v1:172.21.15.43:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f7fdc003520 con 0x7f7ff015a800 2023-08-09T13:55:21.096 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:21.090+0000 7f7fe7fff640 1 -- 172.21.15.43:0/2129940785 <== mon.0 v1:172.21.15.43:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1314499176 0 0) 0x7f7fdc003520 con 0x7f7ff015a800 2023-08-09T13:55:21.096 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:21.090+0000 7f7fe7fff640 1 -- 172.21.15.43:0/2129940785 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f7ff0157ea0 msgr2=0x7f7ff00c97d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-09T13:55:21.096 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:21.090+0000 7f7fe7fff640 1 --2- 172.21.15.43:0/2129940785 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f7ff0157ea0 0x7f7ff00c97d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:55:21.096 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:21.090+0000 7f7fe7fff640 1 -- 172.21.15.43:0/2129940785 >> v2:172.21.15.203:3300/0 conn(0x7f7ff00c90a0 msgr2=0x7f7ff00c9d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-09T13:55:21.097 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:21.090+0000 7f7fe7fff640 1 --2- 172.21.15.43:0/2129940785 >> v2:172.21.15.203:3300/0 conn(0x7f7ff00c90a0 0x7f7ff00c9d10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:55:21.097 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:21.090+0000 7f7fe7fff640 1 -- 172.21.15.43:0/2129940785 --> v1:172.21.15.43:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7ff015b560 con 0x7f7ff015a800 2023-08-09T13:55:21.097 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:21.090+0000 7f7fe7fff640 1 -- 172.21.15.43:0/2129940785 <== mon.0 v1:172.21.15.43:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1285150740 0 0) 0x7f7fe8001b80 con 0x7f7ff015a800 2023-08-09T13:55:21.097 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:21.090+0000 7f7fe7fff640 1 -- 172.21.15.43:0/2129940785 >> v1:172.21.15.43:6789/0 conn(0x7f7ff015a800 legacy=0x7f7ff015ac40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:55:21.097 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:21.090+0000 7f7fe7fff640 1 --2- 172.21.15.43:0/2129940785 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f7fdc003970 0x7f7fdc003d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:55:21.098 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:21.090+0000 7f7fe7fff640 1 --2- 172.21.15.43:0/2129940785 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f7ff0157ea0 0x7f7fdc0047c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:55:21.098 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:21.090+0000 7f7fe7fff640 1 --2- 172.21.15.43:0/2129940785 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f7fdc004d00 0x7f7fdc005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:55:21.098 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:21.090+0000 7f7fe7fff640 1 -- 172.21.15.43:0/2129940785 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7ff015b840 con 0x7f7fdc004d00 2023-08-09T13:55:21.098 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:21.090+0000 7f7fe7fff640 1 -- 172.21.15.43:0/2129940785 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7ff015b350 con 0x7f7fdc003970 2023-08-09T13:55:21.099 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:21.090+0000 7f7fe7fff640 1 -- 172.21.15.43:0/2129940785 --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7fdc00b980 con 0x7f7ff0157ea0 2023-08-09T13:55:21.099 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:21.091+0000 7f7ff5952640 1 --2- 172.21.15.43:0/2129940785 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f7fdc004d00 0x7f7fdc005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:55:21.099 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:21.091+0000 7f7ff5151640 1 --2- 172.21.15.43:0/2129940785 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f7fdc003970 0x7f7fdc003d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:55:21.099 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:21.091+0000 7f7ff5952640 1 -- 172.21.15.43:0/2129940785 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f7ff0157ea0 msgr2=0x7f7fdc0047c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-09T13:55:21.099 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:21.091+0000 7f7ff5952640 1 --2- 172.21.15.43:0/2129940785 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f7ff0157ea0 0x7f7fdc0047c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:55:21.100 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:21.091+0000 7f7ff5952640 1 -- 172.21.15.43:0/2129940785 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f7fdc003970 msgr2=0x7f7fdc003d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-09T13:55:21.100 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:21.091+0000 7f7ff5952640 1 --2- 172.21.15.43:0/2129940785 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f7fdc003970 0x7f7fdc003d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:55:21.100 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:21.091+0000 7f7ff5952640 1 -- 172.21.15.43:0/2129940785 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7fdc003520 con 0x7f7fdc004d00 2023-08-09T13:55:21.102 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:21.092+0000 7f7ff5952640 1 --2- 172.21.15.43:0/2129940785 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f7fdc004d00 0x7f7fdc005180 secure :-1 s=READY pgs=1648 cs=0 l=1 rev1=1 crypto rx=0x7f7fe800a860 tx=0x7f7fdc0057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=60a25031fdf66fd2 server_cookie=0 in_seq=0 out_seq=0 2023-08-09T13:55:21.103 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:21.093+0000 7f7fe7fff640 1 -- 172.21.15.43:0/2129940785 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7fe8034630 con 0x7f7fdc004d00 2023-08-09T13:55:21.103 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:21.093+0000 7f7fe7fff640 1 -- 172.21.15.43:0/2129940785 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7fe802f070 con 0x7f7fdc004d00 2023-08-09T13:55:21.103 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:21.093+0000 7f7fe7fff640 1 -- 172.21.15.43:0/2129940785 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7fe80349f0 con 0x7f7fdc004d00 2023-08-09T13:55:21.103 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:21.093+0000 7f7ff6153640 1 -- 172.21.15.43:0/2129940785 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f7fdc004d00 msgr2=0x7f7fdc005180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:55:21.103 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:21.093+0000 7f7ff6153640 1 --2- 172.21.15.43:0/2129940785 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f7fdc004d00 0x7f7fdc005180 secure :-1 s=READY pgs=1648 cs=0 l=1 rev1=1 crypto rx=0x7f7fe800a860 tx=0x7f7fdc0057f0 comp rx=0 tx=0).stop 2023-08-09T13:55:21.103 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:21.093+0000 7f7ff5952640 1 -- 172.21.15.43:0/2129940785 reap_dead start 2023-08-09T13:55:21.104 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:21.093+0000 7f7ff6153640 1 -- 172.21.15.43:0/2129940785 shutdown_connections 2023-08-09T13:55:21.104 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:21.093+0000 7f7ff6153640 1 -- 172.21.15.43:0/2129940785 >> 172.21.15.43:0/2129940785 conn(0x7f7ff00cddd0 msgr2=0x7f7ff00ce1e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-09T13:55:21.104 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:21.093+0000 7f7ff6153640 1 -- 172.21.15.43:0/2129940785 shutdown_connections 2023-08-09T13:55:21.104 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:21.094+0000 7f7ff6153640 1 -- 172.21.15.43:0/2129940785 wait complete. 2023-08-09T13:55:21.104 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:21.095+0000 7f7ff6153640 1 Processor -- start 2023-08-09T13:55:21.104 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:21.095+0000 7f7ff6153640 1 -- start start 2023-08-09T13:55:21.104 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:21.096+0000 7f7ff6153640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f7fdc004d00 0x7f7ff0146fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:55:21.105 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:21.096+0000 7f7ff6153640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f7ff014a0d0 0x7f7ff0070be0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:55:21.105 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:21.096+0000 7f7ff6153640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f7ff014a550 0x7f7ff0071120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:55:21.105 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:21.096+0000 7f7ff6153640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7ff0147500 con 0x7f7ff014a550 2023-08-09T13:55:21.105 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:21.096+0000 7f7ff6153640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7ff014ac60 con 0x7f7ff014a0d0 2023-08-09T13:55:21.105 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:21.096+0000 7f7ff6153640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7ff00c49a0 con 0x7f7fdc004d00 2023-08-09T13:55:21.105 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:21.097+0000 7f7ff5952640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f7ff014a550 0x7f7ff0071120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:55:21.106 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:21.097+0000 7f7ff5952640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f7ff014a550 0x7f7ff0071120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:36376/0 (socket says 172.21.15.43:36376) 2023-08-09T13:55:21.106 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:21.097+0000 7f7ff5952640 1 -- 172.21.15.43:0/4046822377 learned_addr learned my addr 172.21.15.43:0/4046822377 (peer_addr_for_me v2:172.21.15.43:0/0) 2023-08-09T13:55:21.106 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:21.097+0000 7f7ff5151640 1 --2- 172.21.15.43:0/4046822377 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f7fdc004d00 0x7f7ff0146fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:55:21.106 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:21.097+0000 7f7ff4950640 1 --2- 172.21.15.43:0/4046822377 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f7ff014a0d0 0x7f7ff0070be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:55:21.106 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:21.097+0000 7f7ff5952640 1 -- 172.21.15.43:0/4046822377 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f7fdc004d00 msgr2=0x7f7ff0146fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:55:21.106 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:21.097+0000 7f7ff5952640 1 --2- 172.21.15.43:0/4046822377 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f7fdc004d00 0x7f7ff0146fc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:55:21.107 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:21.097+0000 7f7ff5952640 1 -- 172.21.15.43:0/4046822377 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f7ff014a0d0 msgr2=0x7f7ff0070be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:55:21.107 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:21.097+0000 7f7ff5952640 1 --2- 172.21.15.43:0/4046822377 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f7ff014a0d0 0x7f7ff0070be0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:55:21.107 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:21.097+0000 7f7ff5952640 1 -- 172.21.15.43:0/4046822377 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7ff00c46c0 con 0x7f7ff014a550 2023-08-09T13:55:21.107 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:21.097+0000 7f7ff4950640 1 --2- 172.21.15.43:0/4046822377 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f7ff014a0d0 0x7f7ff0070be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-08-09T13:55:21.107 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:21.097+0000 7f7ff5952640 1 --2- 172.21.15.43:0/4046822377 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f7ff014a550 0x7f7ff0071120 secure :-1 s=READY pgs=1649 cs=0 l=1 rev1=1 crypto rx=0x7f7fe802cb80 tx=0x7f7fe8001f10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-09T13:55:21.107 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:21.098+0000 7f7fe5ffb640 1 -- 172.21.15.43:0/4046822377 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7fe8042070 con 0x7f7ff014a550 2023-08-09T13:55:21.107 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:21.098+0000 7f7fe5ffb640 1 -- 172.21.15.43:0/4046822377 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7fe8001490 con 0x7f7ff014a550 2023-08-09T13:55:21.107 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:21.098+0000 7f7fe5ffb640 1 -- 172.21.15.43:0/4046822377 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7fe8034630 con 0x7f7ff014a550 2023-08-09T13:55:21.108 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:21.098+0000 7f7ff6153640 1 -- 172.21.15.43:0/4046822377 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7ff0147910 con 0x7f7ff014a550 2023-08-09T13:55:21.108 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:21.098+0000 7f7ff6153640 1 -- 172.21.15.43:0/4046822377 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7ff0147ed0 con 0x7f7ff014a550 2023-08-09T13:55:21.108 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:21.099+0000 7f7ff6153640 1 -- 172.21.15.43:0/4046822377 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7ff0147770 con 0x7f7ff014a550 2023-08-09T13:55:21.108 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:21.100+0000 7f7fe5ffb640 1 -- 172.21.15.43:0/4046822377 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 67) v1 ==== 88443+0+0 (secure 0 0 0) 0x7f7fe8031030 con 0x7f7ff014a550 2023-08-09T13:55:21.109 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:21.101+0000 7f7fe5ffb640 1 --2- 172.21.15.43:0/4046822377 >> [v2:172.21.15.203:6832/3490266242,v1:172.21.15.203:6833/3490266242] conn(0x7f7fd406a170 0x7f7fd406c630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:55:21.109 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:21.101+0000 7f7fe5ffb640 1 -- 172.21.15.43:0/4046822377 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(138..138 src has 1..138) v4 ==== 6434+0+0 (secure 0 0 0) 0x7f7fe80aa020 con 0x7f7ff014a550 2023-08-09T13:55:21.109 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:21.102+0000 7f7ff5151640 1 --2- 172.21.15.43:0/4046822377 >> [v2:172.21.15.203:6832/3490266242,v1:172.21.15.203:6833/3490266242] conn(0x7f7fd406a170 0x7f7fd406c630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:55:21.109 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:21.103+0000 7f7ff5151640 1 --2- 172.21.15.43:0/4046822377 >> [v2:172.21.15.203:6832/3490266242,v1:172.21.15.203:6833/3490266242] conn(0x7f7fd406a170 0x7f7fd406c630 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7f7fec009a30 tx=0x7f7fec008040 comp rx=0 tx=0).ready entity=mgr.6659 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-09T13:55:21.109 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:21.105+0000 7f7fe5ffb640 1 -- 172.21.15.43:0/4046822377 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178066 (secure 0 0 0) 0x7f7ff0147770 con 0x7f7ff014a550 2023-08-09T13:55:21.306 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:21.303+0000 7f7ff6153640 1 -- 172.21.15.43:0/4046822377 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f7ff0158280 con 0x7f7ff014a550 2023-08-09T13:55:21.307 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:21.304+0000 7f7fe5ffb640 1 -- 172.21.15.43:0/4046822377 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v138) v1 ==== 93+0+10 (secure 0 0 0) 0x7f7ff0158280 con 0x7f7ff014a550 2023-08-09T13:55:21.307 INFO:teuthology.orchestra.run.smithi043.stdout:pg_num: 1 2023-08-09T13:55:21.310 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:21.307+0000 7f7ff6153640 1 -- 172.21.15.43:0/4046822377 >> [v2:172.21.15.203:6832/3490266242,v1:172.21.15.203:6833/3490266242] conn(0x7f7fd406a170 msgr2=0x7f7fd406c630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:55:21.311 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:21.307+0000 7f7ff6153640 1 --2- 172.21.15.43:0/4046822377 >> [v2:172.21.15.203:6832/3490266242,v1:172.21.15.203:6833/3490266242] conn(0x7f7fd406a170 0x7f7fd406c630 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7f7fec009a30 tx=0x7f7fec008040 comp rx=0 tx=0).stop 2023-08-09T13:55:21.311 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:21.307+0000 7f7ff6153640 1 -- 172.21.15.43:0/4046822377 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f7ff014a550 msgr2=0x7f7ff0071120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:55:21.311 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:21.307+0000 7f7ff6153640 1 --2- 172.21.15.43:0/4046822377 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f7ff014a550 0x7f7ff0071120 secure :-1 s=READY pgs=1649 cs=0 l=1 rev1=1 crypto rx=0x7f7fe802cb80 tx=0x7f7fe8001f10 comp rx=0 tx=0).stop 2023-08-09T13:55:21.311 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:21.307+0000 7f7ff6153640 1 -- 172.21.15.43:0/4046822377 shutdown_connections 2023-08-09T13:55:21.311 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:21.307+0000 7f7ff6153640 1 --2- 172.21.15.43:0/4046822377 >> [v2:172.21.15.203:6832/3490266242,v1:172.21.15.203:6833/3490266242] conn(0x7f7fd406a170 0x7f7fd406c630 unknown :-1 s=CLOSED pgs=42 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:55:21.312 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:21.307+0000 7f7ff6153640 1 --2- 172.21.15.43:0/4046822377 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f7ff014a550 0x7f7ff0071120 unknown :-1 s=CLOSED pgs=1649 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:55:21.312 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:21.307+0000 7f7ff6153640 1 --2- 172.21.15.43:0/4046822377 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f7ff014a0d0 0x7f7ff0070be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:55:21.312 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:21.307+0000 7f7ff6153640 1 --2- 172.21.15.43:0/4046822377 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f7fdc004d00 0x7f7ff0146fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:55:21.313 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:21.307+0000 7f7ff6153640 1 -- 172.21.15.43:0/4046822377 >> 172.21.15.43:0/4046822377 conn(0x7f7ff00cddd0 msgr2=0x7f7ff00c9a90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-09T13:55:21.314 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:21.308+0000 7f7ff6153640 1 -- 172.21.15.43:0/4046822377 shutdown_connections 2023-08-09T13:55:21.314 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:21.308+0000 7f7ff6153640 1 -- 172.21.15.43:0/4046822377 wait complete. 2023-08-09T13:55:21.331 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mon mon_allow_pool_delete True 2023-08-09T13:55:21.562 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:21.560+0000 7fa4b09b1640 1 Processor -- start 2023-08-09T13:55:21.563 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:21.560+0000 7fa4b09b1640 1 -- start start 2023-08-09T13:55:21.564 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:21.561+0000 7fa4b09b1640 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7fa4ac0d0a70 0x7fa4ac0cd7f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:55:21.564 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:21.561+0000 7fa4b09b1640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7fa4ac0cdd30 0x7fa4ac0b3fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:55:21.565 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:21.561+0000 7fa4b09b1640 1 -- --> v1:172.21.15.203:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa4ac0d0f70 con 0x7fa4ac15c680 2023-08-09T13:55:21.565 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:21.561+0000 7fa4b09b1640 1 -- --> v2:172.21.15.43:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa4ac0ccd10 con 0x7fa4ac0cdd30 2023-08-09T13:55:21.565 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:21.561+0000 7fa4b09b1640 1 -- --> v2:172.21.15.203:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa4ac0ccff0 con 0x7fa4ac0d0a70 2023-08-09T13:55:21.565 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:21.561+0000 7fa4aad9d640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7fa4ac0cdd30 0x7fa4ac0b3fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:55:21.566 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:21.562+0000 7fa4aad9d640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7fa4ac0cdd30 0x7fa4ac0b3fa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:36378/0 (socket says 172.21.15.43:36378) 2023-08-09T13:55:21.566 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:21.562+0000 7fa4aad9d640 1 -- 172.21.15.43:0/2759016045 learned_addr learned my addr 172.21.15.43:0/2759016045 (peer_addr_for_me v2:172.21.15.43:0/0) 2023-08-09T13:55:21.566 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:21.562+0000 7fa4a1d9b640 1 --2- 172.21.15.43:0/2759016045 >> v2:172.21.15.203:3300/0 conn(0x7fa4ac0d0a70 0x7fa4ac0cd7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:55:21.566 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:21.562+0000 7fa4aad9d640 1 -- 172.21.15.43:0/2759016045 >> v2:172.21.15.203:3300/0 conn(0x7fa4ac0d0a70 msgr2=0x7fa4ac0cd7f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-09T13:55:21.566 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:21.562+0000 7fa4aad9d640 1 --2- 172.21.15.43:0/2759016045 >> v2:172.21.15.203:3300/0 conn(0x7fa4ac0d0a70 0x7fa4ac0cd7f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:55:21.566 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:21.562+0000 7fa4aad9d640 1 -- 172.21.15.43:0/2759016045 >> v1:172.21.15.203:6789/0 conn(0x7fa4ac15c680 legacy=0x7fa4ac0ce760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-09T13:55:21.566 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:21.562+0000 7fa4aad9d640 1 -- 172.21.15.43:0/2759016045 --> v2:172.21.15.43:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa4ac0cd2d0 con 0x7fa4ac0cdd30 2023-08-09T13:55:21.567 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:21.563+0000 7fa4aad9d640 1 --2- 172.21.15.43:0/2759016045 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa4ac0cdd30 0x7fa4ac0b3fa0 secure :-1 s=READY pgs=1650 cs=0 l=1 rev1=1 crypto rx=0x7fa4ac0b5400 tx=0x7fa4a402bee0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=541dbe72090ebfd5 server_cookie=0 in_seq=0 out_seq=0 2023-08-09T13:55:21.568 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:21.563+0000 7fa4a9d9b640 1 -- 172.21.15.43:0/2759016045 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa4a402ca80 con 0x7fa4ac0cdd30 2023-08-09T13:55:21.568 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:21.563+0000 7fa4a9d9b640 1 -- 172.21.15.43:0/2759016045 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa4a402cbe0 con 0x7fa4ac0cdd30 2023-08-09T13:55:21.568 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:21.563+0000 7fa4a9d9b640 1 -- 172.21.15.43:0/2759016045 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa4a40343f0 con 0x7fa4ac0cdd30 2023-08-09T13:55:21.568 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:21.564+0000 7fa4b09b1640 1 -- 172.21.15.43:0/2759016045 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa4ac0cdd30 msgr2=0x7fa4ac0b3fa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:55:21.568 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:21.564+0000 7fa4b09b1640 1 --2- 172.21.15.43:0/2759016045 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa4ac0cdd30 0x7fa4ac0b3fa0 secure :-1 s=READY pgs=1650 cs=0 l=1 rev1=1 crypto rx=0x7fa4ac0b5400 tx=0x7fa4a402bee0 comp rx=0 tx=0).stop 2023-08-09T13:55:21.568 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:21.564+0000 7fa4b09b1640 1 -- 172.21.15.43:0/2759016045 shutdown_connections 2023-08-09T13:55:21.569 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:21.564+0000 7fa4b09b1640 1 --2- 172.21.15.43:0/2759016045 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa4ac0cdd30 0x7fa4ac0b3fa0 unknown :-1 s=CLOSED pgs=1650 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:55:21.569 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:21.564+0000 7fa4b09b1640 1 --2- 172.21.15.43:0/2759016045 >> v2:172.21.15.203:3300/0 conn(0x7fa4ac0d0a70 0x7fa4ac0cd7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:55:21.569 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:21.564+0000 7fa4b09b1640 1 -- 172.21.15.43:0/2759016045 >> 172.21.15.43:0/2759016045 conn(0x7fa4ac071520 msgr2=0x7fa4ac0d2520 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-09T13:55:21.569 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:21.564+0000 7fa4b09b1640 1 -- 172.21.15.43:0/2759016045 shutdown_connections 2023-08-09T13:55:21.569 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:21.564+0000 7fa4b09b1640 1 -- 172.21.15.43:0/2759016045 wait complete. 2023-08-09T13:55:21.570 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:21.566+0000 7fa4b09b1640 1 Processor -- start 2023-08-09T13:55:21.571 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:21.566+0000 7fa4b09b1640 1 -- start start 2023-08-09T13:55:21.571 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:21.567+0000 7fa4b09b1640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa4ac0d0a70 0x7fa4ac0c8e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:55:21.571 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:21.567+0000 7fa4b09b1640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fa4ac15c680 0x7fa4ac155050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:55:21.571 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:21.567+0000 7fa4b09b1640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fa4ac0cbc80 0x7fa4ac15ea00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:55:21.571 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:21.567+0000 7fa4b09b1640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa4ac0ca2f0 con 0x7fa4ac0d0a70 2023-08-09T13:55:21.571 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:21.567+0000 7fa4b09b1640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa4ac0c9720 con 0x7fa4ac0cbc80 2023-08-09T13:55:21.572 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:21.567+0000 7fa4b09b1640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa4ac0c93a0 con 0x7fa4ac15c680 2023-08-09T13:55:21.572 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:21.567+0000 7fa4aa59c640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa4ac0d0a70 0x7fa4ac0c8e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:55:21.572 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:21.568+0000 7fa4aa59c640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa4ac0d0a70 0x7fa4ac0c8e60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:36382/0 (socket says 172.21.15.43:36382) 2023-08-09T13:55:21.572 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:21.568+0000 7fa4aa59c640 1 -- 172.21.15.43:0/765954441 learned_addr learned my addr 172.21.15.43:0/765954441 (peer_addr_for_me v2:172.21.15.43:0/0) 2023-08-09T13:55:21.572 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:21.568+0000 7fa4aad9d640 1 --2- 172.21.15.43:0/765954441 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fa4ac0cbc80 0x7fa4ac15ea00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:55:21.573 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:21.568+0000 7fa4a1d9b640 1 --2- 172.21.15.43:0/765954441 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fa4ac15c680 0x7fa4ac155050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:55:21.573 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:21.568+0000 7fa4aa59c640 1 -- 172.21.15.43:0/765954441 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fa4ac15c680 msgr2=0x7fa4ac155050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:55:21.573 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:21.568+0000 7fa4aa59c640 1 --2- 172.21.15.43:0/765954441 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fa4ac15c680 0x7fa4ac155050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:55:21.573 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:21.568+0000 7fa4aa59c640 1 -- 172.21.15.43:0/765954441 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fa4ac0cbc80 msgr2=0x7fa4ac15ea00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:55:21.573 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:21.568+0000 7fa4aa59c640 1 --2- 172.21.15.43:0/765954441 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fa4ac0cbc80 0x7fa4ac15ea00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:55:21.573 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:21.568+0000 7fa4aa59c640 1 -- 172.21.15.43:0/765954441 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa4a4009b30 con 0x7fa4ac0d0a70 2023-08-09T13:55:21.573 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:21.568+0000 7fa4aad9d640 1 --2- 172.21.15.43:0/765954441 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fa4ac0cbc80 0x7fa4ac15ea00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-08-09T13:55:21.574 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:21.568+0000 7fa4aa59c640 1 --2- 172.21.15.43:0/765954441 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa4ac0d0a70 0x7fa4ac0c8e60 secure :-1 s=READY pgs=1651 cs=0 l=1 rev1=1 crypto rx=0x7fa49c00b3e0 tx=0x7fa49c00b8a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-09T13:55:21.575 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:21.569+0000 7fa4a37fe640 1 -- 172.21.15.43:0/765954441 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa49c011830 con 0x7fa4ac0d0a70 2023-08-09T13:55:21.576 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:21.569+0000 7fa4a37fe640 1 -- 172.21.15.43:0/765954441 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa49c0119b0 con 0x7fa4ac0d0a70 2023-08-09T13:55:21.576 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:21.569+0000 7fa4b09b1640 1 -- 172.21.15.43:0/765954441 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa4ac0c99c0 con 0x7fa4ac0d0a70 2023-08-09T13:55:21.576 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:21.569+0000 7fa4a37fe640 1 -- 172.21.15.43:0/765954441 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa49c011d60 con 0x7fa4ac0d0a70 2023-08-09T13:55:21.577 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:21.569+0000 7fa4b09b1640 1 -- 172.21.15.43:0/765954441 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa4ac0c9f80 con 0x7fa4ac0d0a70 2023-08-09T13:55:21.577 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:21.570+0000 7fa4b09b1640 1 -- 172.21.15.43:0/765954441 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa4ac0ca140 con 0x7fa4ac0d0a70 2023-08-09T13:55:21.577 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:21.571+0000 7fa4a37fe640 1 -- 172.21.15.43:0/765954441 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 67) v1 ==== 88443+0+0 (secure 0 0 0) 0x7fa49c01a400 con 0x7fa4ac0d0a70 2023-08-09T13:55:21.577 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:21.572+0000 7fa4a37fe640 1 --2- 172.21.15.43:0/765954441 >> [v2:172.21.15.203:6832/3490266242,v1:172.21.15.203:6833/3490266242] conn(0x7fa48c06a170 0x7fa48c06c630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:55:21.578 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:21.572+0000 7fa4a37fe640 1 -- 172.21.15.43:0/765954441 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(138..138 src has 1..138) v4 ==== 6434+0+0 (secure 0 0 0) 0x7fa49c088a00 con 0x7fa4ac0d0a70 2023-08-09T13:55:21.578 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:21.573+0000 7fa4a1d9b640 1 --2- 172.21.15.43:0/765954441 >> [v2:172.21.15.203:6832/3490266242,v1:172.21.15.203:6833/3490266242] conn(0x7fa48c06a170 0x7fa48c06c630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:55:21.579 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:21.574+0000 7fa4a1d9b640 1 --2- 172.21.15.43:0/765954441 >> [v2:172.21.15.203:6832/3490266242,v1:172.21.15.203:6833/3490266242] conn(0x7fa48c06a170 0x7fa48c06c630 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7fa4ac15e9b0 tx=0x7fa494008040 comp rx=0 tx=0).ready entity=mgr.6659 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-09T13:55:21.579 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:21.576+0000 7fa4a37fe640 1 -- 172.21.15.43:0/765954441 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178066 (secure 0 0 0) 0x7fa49c053e50 con 0x7fa4ac0d0a70 2023-08-09T13:55:21.772 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:21.770+0000 7fa4b09b1640 1 -- 172.21.15.43:0/765954441 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command([{prefix=config set, name=mon_allow_pool_delete}] v 0) v1 -- 0x7fa4ac0739f0 con 0x7fa4ac0d0a70 2023-08-09T13:55:21.778 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:21.775+0000 7fa4a37fe640 1 -- 172.21.15.43:0/765954441 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_allow_pool_delete}]=0 v46)=0 v46) v1 ==== 122+0+0 (secure 0 0 0) 0x7fa49c057470 con 0x7fa4ac0d0a70 2023-08-09T13:55:21.781 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:21.779+0000 7fa4b09b1640 1 -- 172.21.15.43:0/765954441 >> [v2:172.21.15.203:6832/3490266242,v1:172.21.15.203:6833/3490266242] conn(0x7fa48c06a170 msgr2=0x7fa48c06c630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:55:21.781 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:21.779+0000 7fa4b09b1640 1 --2- 172.21.15.43:0/765954441 >> [v2:172.21.15.203:6832/3490266242,v1:172.21.15.203:6833/3490266242] conn(0x7fa48c06a170 0x7fa48c06c630 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7fa4ac15e9b0 tx=0x7fa494008040 comp rx=0 tx=0).stop 2023-08-09T13:55:21.782 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:21.779+0000 7fa4b09b1640 1 -- 172.21.15.43:0/765954441 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa4ac0d0a70 msgr2=0x7fa4ac0c8e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:55:21.782 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:21.779+0000 7fa4b09b1640 1 --2- 172.21.15.43:0/765954441 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa4ac0d0a70 0x7fa4ac0c8e60 secure :-1 s=READY pgs=1651 cs=0 l=1 rev1=1 crypto rx=0x7fa49c00b3e0 tx=0x7fa49c00b8a0 comp rx=0 tx=0).stop 2023-08-09T13:55:21.782 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:21.779+0000 7fa4b09b1640 1 -- 172.21.15.43:0/765954441 shutdown_connections 2023-08-09T13:55:21.782 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:21.779+0000 7fa4b09b1640 1 --2- 172.21.15.43:0/765954441 >> [v2:172.21.15.203:6832/3490266242,v1:172.21.15.203:6833/3490266242] conn(0x7fa48c06a170 0x7fa48c06c630 unknown :-1 s=CLOSED pgs=43 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:55:21.783 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:21.779+0000 7fa4b09b1640 1 --2- 172.21.15.43:0/765954441 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fa4ac0cbc80 0x7fa4ac15ea00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:55:21.783 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:21.779+0000 7fa4b09b1640 1 --2- 172.21.15.43:0/765954441 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fa4ac15c680 0x7fa4ac155050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:55:21.783 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:21.779+0000 7fa4b09b1640 1 --2- 172.21.15.43:0/765954441 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa4ac0d0a70 0x7fa4ac0c8e60 unknown :-1 s=CLOSED pgs=1651 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:55:21.783 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:21.779+0000 7fa4b09b1640 1 -- 172.21.15.43:0/765954441 >> 172.21.15.43:0/765954441 conn(0x7fa4ac071520 msgr2=0x7fa4ac0cf310 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-09T13:55:21.783 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:21.779+0000 7fa4b09b1640 1 -- 172.21.15.43:0/765954441 shutdown_connections 2023-08-09T13:55:21.783 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:21.779+0000 7fa4b09b1640 1 -- 172.21.15.43:0/765954441 wait complete. 2023-08-09T13:55:21.800 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2023-08-09T13:55:21.801 DEBUG:teuthology.orchestra.run.smithi043:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2023-08-09T13:55:21.919 DEBUG:teuthology.orchestra.run.smithi203:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2023-08-09T13:55:22.042 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2023-08-09T13:55:22.043 DEBUG:teuthology.orchestra.run.smithi043:> set -ex 2023-08-09T13:55:22.043 DEBUG:teuthology.orchestra.run.smithi043:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-09T13:55:22.105 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4054441,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6498844k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=27,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3249420k,nr_inodes=812355,mode=700,uid=1000,gid=1275 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6498844k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2023-08-09T13:55:22.105 DEBUG:teuthology.orchestra.run.smithi203:> set -ex 2023-08-09T13:55:22.106 DEBUG:teuthology.orchestra.run.smithi203:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-09T13:55:22.151 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4053648,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497576k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=27,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248784k,nr_inodes=812196,mode=700,uid=1000,gid=1275 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497576k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2023-08-09T13:55:22.151 DEBUG:teuthology.orchestra.run.smithi043:> set -ex 2023-08-09T13:55:22.152 DEBUG:teuthology.orchestra.run.smithi043:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-09T13:55:22.214 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4054441,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6498844k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=27,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3249420k,nr_inodes=812355,mode=700,uid=1000,gid=1275 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6498844k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2023-08-09T13:55:22.214 DEBUG:teuthology.orchestra.run.smithi203:> set -ex 2023-08-09T13:55:22.215 DEBUG:teuthology.orchestra.run.smithi203:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-09T13:55:22.259 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4053648,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497576k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=27,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248784k,nr_inodes=812196,mode=700,uid=1000,gid=1275 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497576k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2023-08-09T13:55:22.260 INFO:tasks.cephfs.cephfs_test_case:[, , , ] 2023-08-09T13:55:22.260 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-08-09T13:55:22.499 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:22.496+0000 7f659e06a640 1 Processor -- start 2023-08-09T13:55:22.499 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:22.497+0000 7f659e06a640 1 -- start start 2023-08-09T13:55:22.501 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:22.498+0000 7f659e06a640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f6598160380 0x7f65980ca0c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:55:22.501 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:22.498+0000 7f659e06a640 1 -- --> v1:172.21.15.43:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f65980c8680 con 0x7f659815c780 2023-08-09T13:55:22.502 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:22.498+0000 7f659e06a640 1 -- --> v1:172.21.15.203:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f65980c8960 con 0x7f6598160cb0 2023-08-09T13:55:22.502 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:22.498+0000 7f659e06a640 1 -- --> v2:172.21.15.43:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f65980aaf00 con 0x7f6598160380 2023-08-09T13:55:22.502 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:22.498+0000 7f659d068640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f6598160380 0x7f65980ca0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:55:22.503 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:22.498+0000 7f659d068640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f6598160380 0x7f65980ca0c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:36392/0 (socket says 172.21.15.43:36392) 2023-08-09T13:55:22.503 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:22.498+0000 7f659d068640 1 -- 172.21.15.43:0/901009821 learned_addr learned my addr 172.21.15.43:0/901009821 (peer_addr_for_me v2:172.21.15.43:0/0) 2023-08-09T13:55:22.505 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:22.499+0000 7f659d068640 1 -- 172.21.15.43:0/901009821 >> v1:172.21.15.203:6789/0 conn(0x7f6598160cb0 legacy=0x7f659815c070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-09T13:55:22.505 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:22.499+0000 7f659d068640 1 -- 172.21.15.43:0/901009821 >> v1:172.21.15.43:6789/0 conn(0x7f659815c780 legacy=0x7f65980ab700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:55:22.505 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:22.499+0000 7f659d068640 1 -- 172.21.15.43:0/901009821 --> v2:172.21.15.43:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f65980ab1e0 con 0x7f6598160380 2023-08-09T13:55:22.505 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:22.500+0000 7f659d068640 1 --2- 172.21.15.43:0/901009821 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f6598160380 0x7f65980ca0c0 secure :-1 s=READY pgs=1653 cs=0 l=1 rev1=1 crypto rx=0x7f6594009870 tx=0x7f659402ba30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=edebe8d87e5bde44 server_cookie=0 in_seq=0 out_seq=0 2023-08-09T13:55:22.505 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:22.500+0000 7f658ffff640 1 -- 172.21.15.43:0/901009821 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f659402f850 con 0x7f6598160380 2023-08-09T13:55:22.505 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:22.500+0000 7f658ffff640 1 -- 172.21.15.43:0/901009821 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f659402f9b0 con 0x7f6598160380 2023-08-09T13:55:22.506 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:22.500+0000 7f658ffff640 1 -- 172.21.15.43:0/901009821 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f659402fd40 con 0x7f6598160380 2023-08-09T13:55:22.506 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:22.501+0000 7f659e06a640 1 -- 172.21.15.43:0/901009821 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f6598160380 msgr2=0x7f65980ca0c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:55:22.506 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:22.501+0000 7f659e06a640 1 --2- 172.21.15.43:0/901009821 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f6598160380 0x7f65980ca0c0 secure :-1 s=READY pgs=1653 cs=0 l=1 rev1=1 crypto rx=0x7f6594009870 tx=0x7f659402ba30 comp rx=0 tx=0).stop 2023-08-09T13:55:22.506 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:22.501+0000 7f659e06a640 1 -- 172.21.15.43:0/901009821 shutdown_connections 2023-08-09T13:55:22.506 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:22.501+0000 7f659e06a640 1 --2- 172.21.15.43:0/901009821 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f6598160380 0x7f65980ca0c0 unknown :-1 s=CLOSED pgs=1653 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:55:22.506 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:22.501+0000 7f659e06a640 1 -- 172.21.15.43:0/901009821 >> 172.21.15.43:0/901009821 conn(0x7f65980cde90 msgr2=0x7f65980ce2a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-09T13:55:22.506 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:22.501+0000 7f659e06a640 1 -- 172.21.15.43:0/901009821 shutdown_connections 2023-08-09T13:55:22.507 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:22.501+0000 7f659e06a640 1 -- 172.21.15.43:0/901009821 wait complete. 2023-08-09T13:55:22.508 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:22.503+0000 7f659e06a640 1 Processor -- start 2023-08-09T13:55:22.508 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:22.503+0000 7f659e06a640 1 -- start start 2023-08-09T13:55:22.508 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:22.504+0000 7f659e06a640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f659815c780 0x7f65981593f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:55:22.509 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:22.504+0000 7f659e06a640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f6598160380 0x7f6598153250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:55:22.509 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:22.504+0000 7f659e06a640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f6598160cb0 0x7f659815e770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:55:22.509 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:22.504+0000 7f659e06a640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f65981f3460 con 0x7f6598160cb0 2023-08-09T13:55:22.509 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:22.504+0000 7f659e06a640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f659815ecb0 con 0x7f6598160380 2023-08-09T13:55:22.509 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:22.504+0000 7f659e06a640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f65981f2590 con 0x7f659815c780 2023-08-09T13:55:22.509 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:22.504+0000 7f659d869640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f6598160cb0 0x7f659815e770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:55:22.509 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:22.504+0000 7f659d869640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f6598160cb0 0x7f659815e770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:36406/0 (socket says 172.21.15.43:36406) 2023-08-09T13:55:22.510 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:22.504+0000 7f659d869640 1 -- 172.21.15.43:0/903012495 learned_addr learned my addr 172.21.15.43:0/903012495 (peer_addr_for_me v2:172.21.15.43:0/0) 2023-08-09T13:55:22.510 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:22.505+0000 7f659c867640 1 --2- 172.21.15.43:0/903012495 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f6598160380 0x7f6598153250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:55:22.510 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:22.505+0000 7f659d068640 1 --2- 172.21.15.43:0/903012495 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f659815c780 0x7f65981593f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:55:22.510 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:22.505+0000 7f659d869640 1 -- 172.21.15.43:0/903012495 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f659815c780 msgr2=0x7f65981593f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:55:22.510 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:22.505+0000 7f659d869640 1 --2- 172.21.15.43:0/903012495 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f659815c780 0x7f65981593f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:55:22.510 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:22.505+0000 7f659d869640 1 -- 172.21.15.43:0/903012495 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f6598160380 msgr2=0x7f6598153250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:55:22.510 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:22.505+0000 7f659d869640 1 --2- 172.21.15.43:0/903012495 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f6598160380 0x7f6598153250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:55:22.511 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:22.505+0000 7f659d869640 1 -- 172.21.15.43:0/903012495 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f65940094d0 con 0x7f6598160cb0 2023-08-09T13:55:22.511 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:22.505+0000 7f659d869640 1 --2- 172.21.15.43:0/903012495 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f6598160cb0 0x7f659815e770 secure :-1 s=READY pgs=1654 cs=0 l=1 rev1=1 crypto rx=0x7f659000c730 tx=0x7f659000cbf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-09T13:55:22.511 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:22.506+0000 7f658dffb640 1 -- 172.21.15.43:0/903012495 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6590003b90 con 0x7f6598160cb0 2023-08-09T13:55:22.511 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:22.506+0000 7f658dffb640 1 -- 172.21.15.43:0/903012495 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6590003cf0 con 0x7f6598160cb0 2023-08-09T13:55:22.511 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:22.506+0000 7f658dffb640 1 -- 172.21.15.43:0/903012495 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f659000aa90 con 0x7f6598160cb0 2023-08-09T13:55:22.511 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:22.506+0000 7f659e06a640 1 -- 172.21.15.43:0/903012495 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f65980ca780 con 0x7f6598160cb0 2023-08-09T13:55:22.511 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:22.506+0000 7f659e06a640 1 -- 172.21.15.43:0/903012495 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f65981f2ac0 con 0x7f6598160cb0 2023-08-09T13:55:22.512 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:22.507+0000 7f659e06a640 1 -- 172.21.15.43:0/903012495 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f65980739f0 con 0x7f6598160cb0 2023-08-09T13:55:22.512 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:22.508+0000 7f658dffb640 1 -- 172.21.15.43:0/903012495 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 67) v1 ==== 88443+0+0 (secure 0 0 0) 0x7f659000abf0 con 0x7f6598160cb0 2023-08-09T13:55:22.512 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:22.509+0000 7f658dffb640 1 --2- 172.21.15.43:0/903012495 >> [v2:172.21.15.203:6832/3490266242,v1:172.21.15.203:6833/3490266242] conn(0x7f657c06a1a0 0x7f657c06c660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:55:22.513 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:22.509+0000 7f658dffb640 1 -- 172.21.15.43:0/903012495 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(138..138 src has 1..138) v4 ==== 6434+0+0 (secure 0 0 0) 0x7f6590088650 con 0x7f6598160cb0 2023-08-09T13:55:22.513 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:22.510+0000 7f659d068640 1 --2- 172.21.15.43:0/903012495 >> [v2:172.21.15.203:6832/3490266242,v1:172.21.15.203:6833/3490266242] conn(0x7f657c06a1a0 0x7f657c06c660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:55:22.513 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:22.511+0000 7f659d068640 1 --2- 172.21.15.43:0/903012495 >> [v2:172.21.15.203:6832/3490266242,v1:172.21.15.203:6833/3490266242] conn(0x7f657c06a1a0 0x7f657c06c660 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7f6594036000 tx=0x7f6594008030 comp rx=0 tx=0).ready entity=mgr.6659 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-09T13:55:22.515 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:22.513+0000 7f658dffb640 1 -- 172.21.15.43:0/903012495 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178066 (secure 0 0 0) 0x7f6590053aa0 con 0x7f6598160cb0 2023-08-09T13:55:22.749 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:22.746+0000 7f659e06a640 1 -- 172.21.15.43:0/903012495 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f659815ee50 con 0x7f6598160cb0 2023-08-09T13:55:22.750 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:22.747+0000 7f658dffb640 1 -- 172.21.15.43:0/903012495 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 63 v63) v1 ==== 94+0+3702 (secure 0 0 0) 0x7f65900570c0 con 0x7f6598160cb0 2023-08-09T13:55:22.750 INFO:teuthology.orchestra.run.smithi043.stdout: 2023-08-09T13:55:22.750 INFO:teuthology.orchestra.run.smithi043.stdout:{"epoch":63,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6530,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.203:6835/1659419253","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6834","nonce":1659419253},{"type":"v1","addr":"172.21.15.203:6835","nonce":1659419253}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":56},{"gid":6536,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.203:6837/3964427549","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6836","nonce":3964427549},{"type":"v1","addr":"172.21.15.203:6837","nonce":3964427549}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":56},{"gid":8574,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.43:6833/871812392","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6832","nonce":871812392},{"type":"v1","addr":"172.21.15.43:6833","nonce":871812392}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":55},{"gid":9012,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.43:6835/1311569975","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6834","nonce":1311569975},{"type":"v1","addr":"172.21.15.43:6835","nonce":1311569975}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":63}],"filesystems":[]} 2023-08-09T13:55:22.751 INFO:teuthology.orchestra.run.smithi043.stderr:dumped fsmap epoch 63 2023-08-09T13:55:22.753 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:22.750+0000 7f659e06a640 1 -- 172.21.15.43:0/903012495 >> [v2:172.21.15.203:6832/3490266242,v1:172.21.15.203:6833/3490266242] conn(0x7f657c06a1a0 msgr2=0x7f657c06c660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:55:22.753 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:22.750+0000 7f659e06a640 1 --2- 172.21.15.43:0/903012495 >> [v2:172.21.15.203:6832/3490266242,v1:172.21.15.203:6833/3490266242] conn(0x7f657c06a1a0 0x7f657c06c660 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7f6594036000 tx=0x7f6594008030 comp rx=0 tx=0).stop 2023-08-09T13:55:22.753 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:22.750+0000 7f659e06a640 1 -- 172.21.15.43:0/903012495 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f6598160cb0 msgr2=0x7f659815e770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:55:22.753 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:22.750+0000 7f659e06a640 1 --2- 172.21.15.43:0/903012495 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f6598160cb0 0x7f659815e770 secure :-1 s=READY pgs=1654 cs=0 l=1 rev1=1 crypto rx=0x7f659000c730 tx=0x7f659000cbf0 comp rx=0 tx=0).stop 2023-08-09T13:55:22.753 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:22.750+0000 7f659e06a640 1 -- 172.21.15.43:0/903012495 shutdown_connections 2023-08-09T13:55:22.753 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:22.750+0000 7f659e06a640 1 --2- 172.21.15.43:0/903012495 >> [v2:172.21.15.203:6832/3490266242,v1:172.21.15.203:6833/3490266242] conn(0x7f657c06a1a0 0x7f657c06c660 unknown :-1 s=CLOSED pgs=44 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:55:22.754 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:22.750+0000 7f659e06a640 1 --2- 172.21.15.43:0/903012495 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f6598160cb0 0x7f659815e770 unknown :-1 s=CLOSED pgs=1654 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:55:22.754 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:22.750+0000 7f659e06a640 1 --2- 172.21.15.43:0/903012495 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f6598160380 0x7f6598153250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:55:22.754 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:22.750+0000 7f659e06a640 1 --2- 172.21.15.43:0/903012495 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f659815c780 0x7f65981593f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:55:22.754 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:22.750+0000 7f659e06a640 1 -- 172.21.15.43:0/903012495 >> 172.21.15.43:0/903012495 conn(0x7f65980cde90 msgr2=0x7f65980adc40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-09T13:55:22.754 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:22.750+0000 7f659e06a640 1 -- 172.21.15.43:0/903012495 shutdown_connections 2023-08-09T13:55:22.754 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:22.750+0000 7f659e06a640 1 -- 172.21.15.43:0/903012495 wait complete. 2023-08-09T13:55:22.775 INFO:tasks.ceph.mds.a:Restarting daemon 2023-08-09T13:55:22.775 INFO:tasks.ceph.mds.a:Stopping old one... 2023-08-09T13:55:22.775 DEBUG:tasks.ceph.mds.a:waiting for process to exit 2023-08-09T13:55:22.775 INFO:teuthology.orchestra.run:waiting for 300 2023-08-09T13:55:22.776 INFO:tasks.ceph.mds.c:Restarting daemon 2023-08-09T13:55:22.776 INFO:tasks.ceph.mds.c:Stopping old one... 2023-08-09T13:55:22.776 DEBUG:tasks.ceph.mds.c:waiting for process to exit 2023-08-09T13:55:22.776 INFO:teuthology.orchestra.run:waiting for 300 2023-08-09T13:55:22.777 INFO:tasks.ceph.mds.b:Restarting daemon 2023-08-09T13:55:22.777 INFO:tasks.ceph.mds.b:Stopping old one... 2023-08-09T13:55:22.777 DEBUG:tasks.ceph.mds.b:waiting for process to exit 2023-08-09T13:55:22.777 INFO:teuthology.orchestra.run:waiting for 300 2023-08-09T13:55:22.778 INFO:tasks.ceph.mds.d:Restarting daemon 2023-08-09T13:55:22.778 INFO:tasks.ceph.mds.d:Stopping old one... 2023-08-09T13:55:22.778 DEBUG:tasks.ceph.mds.d:waiting for process to exit 2023-08-09T13:55:22.778 INFO:teuthology.orchestra.run:waiting for 300 2023-08-09T13:55:22.798 INFO:tasks.ceph.mds.d:Stopped 2023-08-09T13:55:22.799 DEBUG:teuthology.orchestra.run.smithi203:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i d 2023-08-09T13:55:22.800 INFO:tasks.ceph.mds.a:Stopped 2023-08-09T13:55:22.801 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i a 2023-08-09T13:55:22.803 INFO:tasks.ceph.mds.c:Stopped 2023-08-09T13:55:22.803 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i c 2023-08-09T13:55:22.808 INFO:tasks.ceph.mds.b:Stopped 2023-08-09T13:55:22.808 DEBUG:teuthology.orchestra.run.smithi203:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i b 2023-08-09T13:55:22.851 INFO:tasks.ceph.mds.a:Started 2023-08-09T13:55:22.852 DEBUG:teuthology.parallel:result is None 2023-08-09T13:55:22.856 INFO:tasks.ceph.mds.d:Started 2023-08-09T13:55:22.857 DEBUG:teuthology.parallel:result is None 2023-08-09T13:55:22.859 INFO:tasks.ceph.mds.c:Started 2023-08-09T13:55:22.859 DEBUG:teuthology.parallel:result is None 2023-08-09T13:55:22.863 INFO:tasks.ceph.mds.b:Started 2023-08-09T13:55:22.864 DEBUG:teuthology.parallel:result is None 2023-08-09T13:55:22.864 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist clear 2023-08-09T13:55:23.021 INFO:tasks.ceph.mds.a.smithi043.stdout:starting mds.a at 2023-08-09T13:55:23.021 INFO:tasks.ceph.mds.c.smithi043.stdout:starting mds.c at 2023-08-09T13:55:23.024 INFO:tasks.ceph.mds.d.smithi203.stdout:starting mds.d at 2023-08-09T13:55:23.027 INFO:tasks.ceph.mds.b.smithi203.stdout:starting mds.b at 2023-08-09T13:55:23.065 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:23.063+0000 7fd9b9dc9640 1 Processor -- start 2023-08-09T13:55:23.065 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:23.063+0000 7fd9b9dc9640 1 -- start start 2023-08-09T13:55:23.066 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:23.064+0000 7fd9b9dc9640 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7fd9b415c680 0x7fd9b40ce760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:55:23.066 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:23.064+0000 7fd9b9dc9640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fd9b40d0a70 0x7fd9b40cd7f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:55:23.066 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:23.064+0000 7fd9b9dc9640 1 -- --> v1:172.21.15.43:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd9b40d0f70 con 0x7fd9b40cdd30 2023-08-09T13:55:23.067 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:23.064+0000 7fd9b9dc9640 1 -- --> v2:172.21.15.203:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd9b40ccd10 con 0x7fd9b415c680 2023-08-09T13:55:23.067 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:23.064+0000 7fd9b9dc9640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd9b40ccff0 con 0x7fd9b40d0a70 2023-08-09T13:55:23.069 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:23.064+0000 7fd9b3fff640 1 --1- >> v1:172.21.15.43:6789/0 conn(0x7fd9b40cdd30 0x7fd9b40b3fa0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.43:6789/0 says I am v1:172.21.15.43:60978/0 (socket says 172.21.15.43:60978) 2023-08-09T13:55:23.069 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:23.064+0000 7fd9b3fff640 1 -- 172.21.15.43:0/1288964232 learned_addr learned my addr 172.21.15.43:0/1288964232 (peer_addr_for_me v1:172.21.15.43:0/0) 2023-08-09T13:55:23.069 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:23.065+0000 7fd9b2ffd640 1 --2- 172.21.15.43:0/1288964232 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fd9b40d0a70 0x7fd9b40cd7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:55:23.069 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:23.065+0000 7fd9b37fe640 1 --2- 172.21.15.43:0/1288964232 >> v2:172.21.15.203:3300/0 conn(0x7fd9b415c680 0x7fd9b40ce760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:55:23.070 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:23.065+0000 7fd9b27fc640 1 -- 172.21.15.43:0/1288964232 <== mon.0 v1:172.21.15.43:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3516918630 0 0) 0x7fd9b40d0f70 con 0x7fd9b40cdd30 2023-08-09T13:55:23.070 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:23.065+0000 7fd9b27fc640 1 -- 172.21.15.43:0/1288964232 --> v1:172.21.15.43:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fd9a0003520 con 0x7fd9b40cdd30 2023-08-09T13:55:23.070 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:23.065+0000 7fd9b2ffd640 1 -- 172.21.15.43:0/1288964232 >> v2:172.21.15.203:3300/0 conn(0x7fd9b415c680 msgr2=0x7fd9b40ce760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-09T13:55:23.070 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:23.065+0000 7fd9b2ffd640 1 --2- 172.21.15.43:0/1288964232 >> v2:172.21.15.203:3300/0 conn(0x7fd9b415c680 0x7fd9b40ce760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:55:23.070 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:23.065+0000 7fd9b2ffd640 1 -- 172.21.15.43:0/1288964232 >> v1:172.21.15.43:6789/0 conn(0x7fd9b40cdd30 legacy=0x7fd9b40b3fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:55:23.070 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:23.065+0000 7fd9b27fc640 1 -- 172.21.15.43:0/1288964232 <== mon.0 v1:172.21.15.43:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 67959109 0 0) 0x7fd9a0003520 con 0x7fd9b40cdd30 2023-08-09T13:55:23.070 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:23.065+0000 7fd9b2ffd640 1 -- 172.21.15.43:0/1288964232 --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd9b40cd2d0 con 0x7fd9b40d0a70 2023-08-09T13:55:23.070 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:23.066+0000 7fd9b2ffd640 1 --2- 172.21.15.43:0/1288964232 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fd9b40d0a70 0x7fd9b40cd7f0 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7fd99c0097a0 tx=0x7fd99c02b790 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b4050dce1f892d64 server_cookie=0 in_seq=0 out_seq=0 2023-08-09T13:55:23.071 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:23.067+0000 7fd9b27fc640 1 -- 172.21.15.43:0/1288964232 <== mon.2 v2:172.21.15.203:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd9ac001670 con 0x7fd9b40d0a70 2023-08-09T13:55:23.072 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:23.067+0000 7fd9b27fc640 1 -- 172.21.15.43:0/1288964232 <== mon.2 v2:172.21.15.203:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd99c02e050 con 0x7fd9b40d0a70 2023-08-09T13:55:23.072 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:23.067+0000 7fd9b27fc640 1 -- 172.21.15.43:0/1288964232 <== mon.2 v2:172.21.15.203:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd99c030730 con 0x7fd9b40d0a70 2023-08-09T13:55:23.072 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:23.067+0000 7fd9b9dc9640 1 -- 172.21.15.43:0/1288964232 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fd9b40d0a70 msgr2=0x7fd9b40cd7f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:55:23.072 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:23.067+0000 7fd9b9dc9640 1 --2- 172.21.15.43:0/1288964232 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fd9b40d0a70 0x7fd9b40cd7f0 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7fd99c0097a0 tx=0x7fd99c02b790 comp rx=0 tx=0).stop 2023-08-09T13:55:23.072 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:23.067+0000 7fd9b9dc9640 1 -- 172.21.15.43:0/1288964232 shutdown_connections 2023-08-09T13:55:23.072 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:23.067+0000 7fd9b9dc9640 1 --2- 172.21.15.43:0/1288964232 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fd9b40d0a70 0x7fd9b40cd7f0 unknown :-1 s=CLOSED pgs=226 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:55:23.072 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:23.067+0000 7fd9b9dc9640 1 --2- 172.21.15.43:0/1288964232 >> v2:172.21.15.203:3300/0 conn(0x7fd9b415c680 0x7fd9b40ce760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:55:23.073 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:23.067+0000 7fd9b9dc9640 1 -- 172.21.15.43:0/1288964232 >> 172.21.15.43:0/1288964232 conn(0x7fd9b4071520 msgr2=0x7fd9b40d2520 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-09T13:55:23.073 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:23.068+0000 7fd9b9dc9640 1 -- 172.21.15.43:0/1288964232 shutdown_connections 2023-08-09T13:55:23.073 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:23.068+0000 7fd9b9dc9640 1 -- 172.21.15.43:0/1288964232 wait complete. 2023-08-09T13:55:23.073 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:23.069+0000 7fd9b9dc9640 1 Processor -- start 2023-08-09T13:55:23.074 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:23.069+0000 7fd9b9dc9640 1 -- start start 2023-08-09T13:55:23.074 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:23.070+0000 7fd9b9dc9640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fd9b40d0a70 0x7fd9b41f4e90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:55:23.074 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:23.070+0000 7fd9b9dc9640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fd9b415c680 0x7fd9b40b6c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:55:23.075 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:23.071+0000 7fd9b9dc9640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fd9b40b7180 0x7fd9b40c95f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:55:23.075 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:23.071+0000 7fd9b9dc9640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd9b40b7630 con 0x7fd9b40d0a70 2023-08-09T13:55:23.075 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:23.071+0000 7fd9b9dc9640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd9b4146660 con 0x7fd9b40b7180 2023-08-09T13:55:23.075 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:23.071+0000 7fd9b9dc9640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd9b40ca320 con 0x7fd9b415c680 2023-08-09T13:55:23.075 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:23.071+0000 7fd9b37fe640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fd9b40d0a70 0x7fd9b41f4e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:55:23.075 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:23.071+0000 7fd9b37fe640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fd9b40d0a70 0x7fd9b41f4e90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:36452/0 (socket says 172.21.15.43:36452) 2023-08-09T13:55:23.075 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:23.071+0000 7fd9b37fe640 1 -- 172.21.15.43:0/3235345208 learned_addr learned my addr 172.21.15.43:0/3235345208 (peer_addr_for_me v2:172.21.15.43:0/0) 2023-08-09T13:55:23.076 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:23.071+0000 7fd9b3fff640 1 --2- 172.21.15.43:0/3235345208 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fd9b40b7180 0x7fd9b40c95f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:55:23.076 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:23.071+0000 7fd9b2ffd640 1 --2- 172.21.15.43:0/3235345208 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fd9b415c680 0x7fd9b40b6c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:55:23.076 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:23.071+0000 7fd9b37fe640 1 -- 172.21.15.43:0/3235345208 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fd9b415c680 msgr2=0x7fd9b40b6c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:55:23.076 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:23.071+0000 7fd9b37fe640 1 --2- 172.21.15.43:0/3235345208 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fd9b415c680 0x7fd9b40b6c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:55:23.076 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:23.071+0000 7fd9b37fe640 1 -- 172.21.15.43:0/3235345208 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fd9b40b7180 msgr2=0x7fd9b40c95f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:55:23.076 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:23.071+0000 7fd9b37fe640 1 --2- 172.21.15.43:0/3235345208 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fd9b40b7180 0x7fd9b40c95f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:55:23.076 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:23.071+0000 7fd9b37fe640 1 -- 172.21.15.43:0/3235345208 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd99c009400 con 0x7fd9b40d0a70 2023-08-09T13:55:23.077 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:23.072+0000 7fd9b37fe640 1 --2- 172.21.15.43:0/3235345208 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fd9b40d0a70 0x7fd9b41f4e90 secure :-1 s=READY pgs=1662 cs=0 l=1 rev1=1 crypto rx=0x7fd9a400efc0 tx=0x7fd9a400c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-09T13:55:23.077 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:23.072+0000 7fd9abfff640 1 -- 172.21.15.43:0/3235345208 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd9a4009b80 con 0x7fd9b40d0a70 2023-08-09T13:55:23.077 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:23.072+0000 7fd9abfff640 1 -- 172.21.15.43:0/3235345208 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd9a400f040 con 0x7fd9b40d0a70 2023-08-09T13:55:23.077 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:23.072+0000 7fd9abfff640 1 -- 172.21.15.43:0/3235345208 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd9a40146a0 con 0x7fd9b40d0a70 2023-08-09T13:55:23.077 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:23.072+0000 7fd9b9dc9640 1 -- 172.21.15.43:0/3235345208 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd9b41468b0 con 0x7fd9b40d0a70 2023-08-09T13:55:23.079 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:23.072+0000 7fd9b9dc9640 1 -- 172.21.15.43:0/3235345208 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd9b4146e70 con 0x7fd9b40d0a70 2023-08-09T13:55:23.079 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:23.074+0000 7fd9b9dc9640 1 -- 172.21.15.43:0/3235345208 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd9b40739f0 con 0x7fd9b40d0a70 2023-08-09T13:55:23.080 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:23.074+0000 7fd9abfff640 1 -- 172.21.15.43:0/3235345208 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 67) v1 ==== 88443+0+0 (secure 0 0 0) 0x7fd9a4014800 con 0x7fd9b40d0a70 2023-08-09T13:55:23.080 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:23.076+0000 7fd9abfff640 1 --2- 172.21.15.43:0/3235345208 >> [v2:172.21.15.203:6832/3490266242,v1:172.21.15.203:6833/3490266242] conn(0x7fd99406a170 0x7fd99406c630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:55:23.080 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:23.076+0000 7fd9abfff640 1 -- 172.21.15.43:0/3235345208 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(138..138 src has 1..138) v4 ==== 6434+0+0 (secure 0 0 0) 0x7fd9a4089440 con 0x7fd9b40d0a70 2023-08-09T13:55:23.080 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:23.077+0000 7fd9b2ffd640 1 --2- 172.21.15.43:0/3235345208 >> [v2:172.21.15.203:6832/3490266242,v1:172.21.15.203:6833/3490266242] conn(0x7fd99406a170 0x7fd99406c630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:55:23.080 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:23.078+0000 7fd9b2ffd640 1 --2- 172.21.15.43:0/3235345208 >> [v2:172.21.15.203:6832/3490266242,v1:172.21.15.203:6833/3490266242] conn(0x7fd99406a170 0x7fd99406c630 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7fd99c034000 tx=0x7fd99c02f000 comp rx=0 tx=0).ready entity=mgr.6659 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-09T13:55:23.083 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:23.080+0000 7fd9abfff640 1 -- 172.21.15.43:0/3235345208 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178066 (secure 0 0 0) 0x7fd9a40582c0 con 0x7fd9b40d0a70 2023-08-09T13:55:23.299 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:23.296+0000 7fd9b9dc9640 1 -- 172.21.15.43:0/3235345208 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "osd blocklist clear"} v 0) v1 -- 0x7fd9b415fe60 con 0x7fd9b40d0a70 2023-08-09T13:55:24.278 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:24.275+0000 7fd9abfff640 1 -- 172.21.15.43:0/3235345208 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist clear"}]=0 removed all blocklist entries v143) v1 ==== 97+0+0 (secure 0 0 0) 0x7fd9a4057e50 con 0x7fd9b40d0a70 2023-08-09T13:55:24.278 INFO:teuthology.orchestra.run.smithi043.stderr: removed all blocklist entries 2023-08-09T13:55:24.282 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:24.279+0000 7fd9b9dc9640 1 -- 172.21.15.43:0/3235345208 >> [v2:172.21.15.203:6832/3490266242,v1:172.21.15.203:6833/3490266242] conn(0x7fd99406a170 msgr2=0x7fd99406c630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:55:24.282 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:24.279+0000 7fd9b9dc9640 1 --2- 172.21.15.43:0/3235345208 >> [v2:172.21.15.203:6832/3490266242,v1:172.21.15.203:6833/3490266242] conn(0x7fd99406a170 0x7fd99406c630 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7fd99c034000 tx=0x7fd99c02f000 comp rx=0 tx=0).stop 2023-08-09T13:55:24.282 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:24.279+0000 7fd9b9dc9640 1 -- 172.21.15.43:0/3235345208 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fd9b40d0a70 msgr2=0x7fd9b41f4e90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:55:24.282 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:24.279+0000 7fd9b9dc9640 1 --2- 172.21.15.43:0/3235345208 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fd9b40d0a70 0x7fd9b41f4e90 secure :-1 s=READY pgs=1662 cs=0 l=1 rev1=1 crypto rx=0x7fd9a400efc0 tx=0x7fd9a400c490 comp rx=0 tx=0).stop 2023-08-09T13:55:24.282 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:24.279+0000 7fd9b9dc9640 1 -- 172.21.15.43:0/3235345208 shutdown_connections 2023-08-09T13:55:24.283 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:24.279+0000 7fd9b9dc9640 1 --2- 172.21.15.43:0/3235345208 >> [v2:172.21.15.203:6832/3490266242,v1:172.21.15.203:6833/3490266242] conn(0x7fd99406a170 0x7fd99406c630 unknown :-1 s=CLOSED pgs=45 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:55:24.283 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:24.279+0000 7fd9b9dc9640 1 --2- 172.21.15.43:0/3235345208 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fd9b40b7180 0x7fd9b40c95f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:55:24.283 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:24.279+0000 7fd9b9dc9640 1 --2- 172.21.15.43:0/3235345208 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fd9b415c680 0x7fd9b40b6c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:55:24.283 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:24.279+0000 7fd9b9dc9640 1 --2- 172.21.15.43:0/3235345208 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fd9b40d0a70 0x7fd9b41f4e90 unknown :-1 s=CLOSED pgs=1662 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:55:24.283 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:24.279+0000 7fd9b9dc9640 1 -- 172.21.15.43:0/3235345208 >> 172.21.15.43:0/3235345208 conn(0x7fd9b4071520 msgr2=0x7fd9b415a2c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-09T13:55:24.283 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:24.280+0000 7fd9b9dc9640 1 -- 172.21.15.43:0/3235345208 shutdown_connections 2023-08-09T13:55:24.284 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:24.280+0000 7fd9b9dc9640 1 -- 172.21.15.43:0/3235345208 wait complete. 2023-08-09T13:55:24.303 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth ls --format=json-pretty 2023-08-09T13:55:24.535 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:24.532+0000 7f5828986640 1 Processor -- start 2023-08-09T13:55:24.536 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:24.533+0000 7f5828986640 1 -- start start 2023-08-09T13:55:24.538 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:24.534+0000 7f5828986640 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7f582415f7b0 0x7f58240b4760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:55:24.538 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:24.534+0000 7f5828986640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f58240cd6b0 0x7f58240d0e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:55:24.538 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:24.534+0000 7f5828986640 1 -- --> v1:172.21.15.43:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f58240b7bf0 con 0x7f58240b1050 2023-08-09T13:55:24.539 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:24.534+0000 7f5828986640 1 -- --> v2:172.21.15.43:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f58240b7ed0 con 0x7f58240cd6b0 2023-08-09T13:55:24.539 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:24.534+0000 7f5828986640 1 -- --> v2:172.21.15.203:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f58240d0370 con 0x7f582415f7b0 2023-08-09T13:55:24.539 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:24.534+0000 7f5823fff640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f58240cd6b0 0x7f58240d0e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:55:24.539 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:24.535+0000 7f581bfff640 1 --1- >> v1:172.21.15.43:6789/0 conn(0x7f58240b1050 0x7f58240ccfa0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.43:6789/0 says I am v1:172.21.15.43:60990/0 (socket says 172.21.15.43:60990) 2023-08-09T13:55:24.540 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:24.535+0000 7f5823fff640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f58240cd6b0 0x7f58240d0e50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:36456/0 (socket says 172.21.15.43:36456) 2023-08-09T13:55:24.540 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:24.535+0000 7f581bfff640 1 -- 172.21.15.43:0/2363257066 learned_addr learned my addr 172.21.15.43:0/2363257066 (peer_addr_for_me v1:172.21.15.43:0/0) 2023-08-09T13:55:24.540 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:24.535+0000 7f58237fe640 1 --2- 172.21.15.43:0/2363257066 >> v2:172.21.15.203:3300/0 conn(0x7f582415f7b0 0x7f58240b4760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:55:24.541 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:24.535+0000 7f5823fff640 1 -- 172.21.15.43:0/2363257066 >> v2:172.21.15.203:3300/0 conn(0x7f582415f7b0 msgr2=0x7f58240b4760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-09T13:55:24.541 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:24.535+0000 7f5823fff640 1 --2- 172.21.15.43:0/2363257066 >> v2:172.21.15.203:3300/0 conn(0x7f582415f7b0 0x7f58240b4760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:55:24.541 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:24.535+0000 7f5823fff640 1 -- 172.21.15.43:0/2363257066 >> v1:172.21.15.43:6789/0 conn(0x7f58240b1050 legacy=0x7f58240ccfa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:55:24.541 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:24.535+0000 7f5823fff640 1 -- 172.21.15.43:0/2363257066 --> v2:172.21.15.43:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f58240d0650 con 0x7f58240cd6b0 2023-08-09T13:55:24.546 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:24.536+0000 7f5823fff640 1 --2- 172.21.15.43:0/2363257066 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f58240cd6b0 0x7f58240d0e50 secure :-1 s=READY pgs=1664 cs=0 l=1 rev1=1 crypto rx=0x7f581c009d90 tx=0x7f581c02beb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=31f825b5ed07bf2a server_cookie=0 in_seq=0 out_seq=0 2023-08-09T13:55:24.546 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:24.537+0000 7f5822ffd640 1 -- 172.21.15.43:0/2363257066 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f581c02cc60 con 0x7f58240cd6b0 2023-08-09T13:55:24.547 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:24.537+0000 7f5822ffd640 1 -- 172.21.15.43:0/2363257066 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f581c02cdc0 con 0x7f58240cd6b0 2023-08-09T13:55:24.547 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:24.537+0000 7f5822ffd640 1 -- 172.21.15.43:0/2363257066 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f581c034660 con 0x7f58240cd6b0 2023-08-09T13:55:24.547 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:24.537+0000 7f5828986640 1 -- 172.21.15.43:0/2363257066 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f58240cd6b0 msgr2=0x7f58240d0e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:55:24.548 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:24.537+0000 7f5828986640 1 --2- 172.21.15.43:0/2363257066 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f58240cd6b0 0x7f58240d0e50 secure :-1 s=READY pgs=1664 cs=0 l=1 rev1=1 crypto rx=0x7f581c009d90 tx=0x7f581c02beb0 comp rx=0 tx=0).stop 2023-08-09T13:55:24.548 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:24.537+0000 7f5828986640 1 -- 172.21.15.43:0/2363257066 shutdown_connections 2023-08-09T13:55:24.548 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:24.537+0000 7f5828986640 1 --2- 172.21.15.43:0/2363257066 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f58240cd6b0 0x7f58240d0e50 unknown :-1 s=CLOSED pgs=1664 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:55:24.549 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:24.537+0000 7f5828986640 1 --2- 172.21.15.43:0/2363257066 >> v2:172.21.15.203:3300/0 conn(0x7f582415f7b0 0x7f58240b4760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:55:24.549 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:24.537+0000 7f5828986640 1 -- 172.21.15.43:0/2363257066 >> 172.21.15.43:0/2363257066 conn(0x7f582406a2b0 msgr2=0x7f582406a690 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-09T13:55:24.549 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:24.538+0000 7f5828986640 1 -- 172.21.15.43:0/2363257066 shutdown_connections 2023-08-09T13:55:24.549 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:24.538+0000 7f5828986640 1 -- 172.21.15.43:0/2363257066 wait complete. 2023-08-09T13:55:24.550 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:24.539+0000 7f5828986640 1 Processor -- start 2023-08-09T13:55:24.550 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:24.540+0000 7f5828986640 1 -- start start 2023-08-09T13:55:24.550 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:24.541+0000 7f5828986640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f58240b1050 0x7f58240c8020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:55:24.551 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:24.541+0000 7f5828986640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f58240cd6b0 0x7f58240c7110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:55:24.551 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:24.541+0000 7f5828986640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f582415f7b0 0x7f58240d0d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:55:24.551 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:24.541+0000 7f5828986640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f582414dfd0 con 0x7f58240b1050 2023-08-09T13:55:24.551 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:24.541+0000 7f5828986640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f58240c69a0 con 0x7f58240cd6b0 2023-08-09T13:55:24.552 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:24.541+0000 7f5828986640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f58240c6c80 con 0x7f582415f7b0 2023-08-09T13:55:24.552 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:24.541+0000 7f58237fe640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f58240b1050 0x7f58240c8020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:55:24.552 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:24.541+0000 7f58237fe640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f58240b1050 0x7f58240c8020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:36460/0 (socket says 172.21.15.43:36460) 2023-08-09T13:55:24.552 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:24.541+0000 7f58237fe640 1 -- 172.21.15.43:0/967525290 learned_addr learned my addr 172.21.15.43:0/967525290 (peer_addr_for_me v2:172.21.15.43:0/0) 2023-08-09T13:55:24.553 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:24.541+0000 7f581bfff640 1 --2- 172.21.15.43:0/967525290 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f58240cd6b0 0x7f58240c7110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:55:24.553 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:24.541+0000 7f58237fe640 1 -- 172.21.15.43:0/967525290 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f582415f7b0 msgr2=0x7f58240d0d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:55:24.553 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:24.541+0000 7f58237fe640 1 --2- 172.21.15.43:0/967525290 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f582415f7b0 0x7f58240d0d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:55:24.554 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:24.541+0000 7f58237fe640 1 -- 172.21.15.43:0/967525290 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f58240cd6b0 msgr2=0x7f58240c7110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:55:24.554 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:24.541+0000 7f58237fe640 1 --2- 172.21.15.43:0/967525290 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f58240cd6b0 0x7f58240c7110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:55:24.554 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:24.541+0000 7f58237fe640 1 -- 172.21.15.43:0/967525290 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f581c009a70 con 0x7f58240b1050 2023-08-09T13:55:24.554 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:24.542+0000 7f58237fe640 1 --2- 172.21.15.43:0/967525290 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f58240b1050 0x7f58240c8020 secure :-1 s=READY pgs=1665 cs=0 l=1 rev1=1 crypto rx=0x7f581400d5f0 tx=0x7f581400dab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-09T13:55:24.555 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:24.542+0000 7f5820ff9640 1 -- 172.21.15.43:0/967525290 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f58140125f0 con 0x7f58240b1050 2023-08-09T13:55:24.555 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:24.542+0000 7f5820ff9640 1 -- 172.21.15.43:0/967525290 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5814012750 con 0x7f58240b1050 2023-08-09T13:55:24.555 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:24.542+0000 7f5820ff9640 1 -- 172.21.15.43:0/967525290 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5814012b00 con 0x7f58240b1050 2023-08-09T13:55:24.555 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:24.542+0000 7f5828986640 1 -- 172.21.15.43:0/967525290 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5824153a40 con 0x7f58240b1050 2023-08-09T13:55:24.556 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:24.542+0000 7f5828986640 1 -- 172.21.15.43:0/967525290 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5824150f80 con 0x7f58240b1050 2023-08-09T13:55:24.556 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:24.544+0000 7f5828986640 1 -- 172.21.15.43:0/967525290 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f58240739f0 con 0x7f58240b1050 2023-08-09T13:55:24.558 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:24.544+0000 7f5820ff9640 1 -- 172.21.15.43:0/967525290 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 67) v1 ==== 88443+0+0 (secure 0 0 0) 0x7f5814012c60 con 0x7f58240b1050 2023-08-09T13:55:24.558 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:24.545+0000 7f5820ff9640 1 --2- 172.21.15.43:0/967525290 >> [v2:172.21.15.203:6832/3490266242,v1:172.21.15.203:6833/3490266242] conn(0x7f580406a1a0 0x7f580406c660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:55:24.558 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:24.546+0000 7f5820ff9640 1 -- 172.21.15.43:0/967525290 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(143..143 src has 1..143) v4 ==== 5574+0+0 (secure 0 0 0) 0x7f5814088a40 con 0x7f58240b1050 2023-08-09T13:55:24.558 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:24.546+0000 7f581bfff640 1 --2- 172.21.15.43:0/967525290 >> [v2:172.21.15.203:6832/3490266242,v1:172.21.15.203:6833/3490266242] conn(0x7f580406a1a0 0x7f580406c660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:55:24.559 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:24.547+0000 7f581bfff640 1 --2- 172.21.15.43:0/967525290 >> [v2:172.21.15.203:6832/3490266242,v1:172.21.15.203:6833/3490266242] conn(0x7f580406a1a0 0x7f580406c660 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7f58240b69e0 tx=0x7f580c009040 comp rx=0 tx=0).ready entity=mgr.6659 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-09T13:55:24.559 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:24.550+0000 7f5820ff9640 1 -- 172.21.15.43:0/967525290 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178066 (secure 0 0 0) 0x7f5814054160 con 0x7f58240b1050 2023-08-09T13:55:24.822 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:24.820+0000 7f5828986640 1 -- 172.21.15.43:0/967525290 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "auth ls", "format": "json-pretty"} v 0) v1 -- 0x7f582415ca40 con 0x7f58240b1050 2023-08-09T13:55:24.823 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:24.821+0000 7f5820ff9640 1 -- 172.21.15.43:0/967525290 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "auth ls", "format": "json-pretty"}]=0 v19) v1 ==== 80+0+5304 (secure 0 0 0) 0x7f5814057780 con 0x7f58240b1050 2023-08-09T13:55:24.824 INFO:teuthology.orchestra.run.smithi043.stdout: 2023-08-09T13:55:24.824 INFO:teuthology.orchestra.run.smithi043.stdout:{ 2023-08-09T13:55:24.824 INFO:teuthology.orchestra.run.smithi043.stdout: "auth_dump": [ 2023-08-09T13:55:24.824 INFO:teuthology.orchestra.run.smithi043.stdout: { 2023-08-09T13:55:24.824 INFO:teuthology.orchestra.run.smithi043.stdout: "entity": "mds.a", 2023-08-09T13:55:24.824 INFO:teuthology.orchestra.run.smithi043.stdout: "key": "AQCOldNkPu6GCRAAZecwctvMji0Zubb60KMlyQ==", 2023-08-09T13:55:24.825 INFO:teuthology.orchestra.run.smithi043.stdout: "caps": { 2023-08-09T13:55:24.825 INFO:teuthology.orchestra.run.smithi043.stdout: "mds": "allow", 2023-08-09T13:55:24.825 INFO:teuthology.orchestra.run.smithi043.stdout: "mgr": "allow *", 2023-08-09T13:55:24.825 INFO:teuthology.orchestra.run.smithi043.stdout: "mon": "allow *", 2023-08-09T13:55:24.825 INFO:teuthology.orchestra.run.smithi043.stdout: "osd": "allow *" 2023-08-09T13:55:24.825 INFO:teuthology.orchestra.run.smithi043.stdout: } 2023-08-09T13:55:24.825 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2023-08-09T13:55:24.825 INFO:teuthology.orchestra.run.smithi043.stdout: { 2023-08-09T13:55:24.826 INFO:teuthology.orchestra.run.smithi043.stdout: "entity": "mds.b", 2023-08-09T13:55:24.826 INFO:teuthology.orchestra.run.smithi043.stdout: "key": "AQCOldNkt2+5KxAAHcckB+S5NZHsF5z22StXmw==", 2023-08-09T13:55:24.826 INFO:teuthology.orchestra.run.smithi043.stdout: "caps": { 2023-08-09T13:55:24.826 INFO:teuthology.orchestra.run.smithi043.stdout: "mds": "allow", 2023-08-09T13:55:24.826 INFO:teuthology.orchestra.run.smithi043.stdout: "mgr": "allow *", 2023-08-09T13:55:24.826 INFO:teuthology.orchestra.run.smithi043.stdout: "mon": "allow *", 2023-08-09T13:55:24.826 INFO:teuthology.orchestra.run.smithi043.stdout: "osd": "allow *" 2023-08-09T13:55:24.827 INFO:teuthology.orchestra.run.smithi043.stdout: } 2023-08-09T13:55:24.827 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2023-08-09T13:55:24.827 INFO:teuthology.orchestra.run.smithi043.stdout: { 2023-08-09T13:55:24.827 INFO:teuthology.orchestra.run.smithi043.stdout: "entity": "mds.c", 2023-08-09T13:55:24.827 INFO:teuthology.orchestra.run.smithi043.stdout: "key": "AQCOldNk5V46GhAAQBhkvfjVSJCSHLS/YkhgCA==", 2023-08-09T13:55:24.827 INFO:teuthology.orchestra.run.smithi043.stdout: "caps": { 2023-08-09T13:55:24.827 INFO:teuthology.orchestra.run.smithi043.stdout: "mds": "allow", 2023-08-09T13:55:24.828 INFO:teuthology.orchestra.run.smithi043.stdout: "mgr": "allow *", 2023-08-09T13:55:24.828 INFO:teuthology.orchestra.run.smithi043.stdout: "mon": "allow *", 2023-08-09T13:55:24.828 INFO:teuthology.orchestra.run.smithi043.stdout: "osd": "allow *" 2023-08-09T13:55:24.828 INFO:teuthology.orchestra.run.smithi043.stdout: } 2023-08-09T13:55:24.828 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2023-08-09T13:55:24.828 INFO:teuthology.orchestra.run.smithi043.stdout: { 2023-08-09T13:55:24.828 INFO:teuthology.orchestra.run.smithi043.stdout: "entity": "mds.d", 2023-08-09T13:55:24.828 INFO:teuthology.orchestra.run.smithi043.stdout: "key": "AQCPldNkNBoTARAAMVW+rSyXQKaau6I90fUX9Q==", 2023-08-09T13:55:24.829 INFO:teuthology.orchestra.run.smithi043.stdout: "caps": { 2023-08-09T13:55:24.829 INFO:teuthology.orchestra.run.smithi043.stdout: "mds": "allow", 2023-08-09T13:55:24.829 INFO:teuthology.orchestra.run.smithi043.stdout: "mgr": "allow *", 2023-08-09T13:55:24.829 INFO:teuthology.orchestra.run.smithi043.stdout: "mon": "allow *", 2023-08-09T13:55:24.829 INFO:teuthology.orchestra.run.smithi043.stdout: "osd": "allow *" 2023-08-09T13:55:24.829 INFO:teuthology.orchestra.run.smithi043.stdout: } 2023-08-09T13:55:24.829 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2023-08-09T13:55:24.829 INFO:teuthology.orchestra.run.smithi043.stdout: { 2023-08-09T13:55:24.830 INFO:teuthology.orchestra.run.smithi043.stdout: "entity": "osd.0", 2023-08-09T13:55:24.830 INFO:teuthology.orchestra.run.smithi043.stdout: "key": "AQCRldNkltPlMhAAAt3rFvraCtI0aSqGvt6UxA==", 2023-08-09T13:55:24.830 INFO:teuthology.orchestra.run.smithi043.stdout: "caps": { 2023-08-09T13:55:24.830 INFO:teuthology.orchestra.run.smithi043.stdout: "mgr": "allow profile osd", 2023-08-09T13:55:24.830 INFO:teuthology.orchestra.run.smithi043.stdout: "mon": "allow profile osd", 2023-08-09T13:55:24.830 INFO:teuthology.orchestra.run.smithi043.stdout: "osd": "allow *" 2023-08-09T13:55:24.830 INFO:teuthology.orchestra.run.smithi043.stdout: } 2023-08-09T13:55:24.830 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2023-08-09T13:55:24.831 INFO:teuthology.orchestra.run.smithi043.stdout: { 2023-08-09T13:55:24.831 INFO:teuthology.orchestra.run.smithi043.stdout: "entity": "osd.1", 2023-08-09T13:55:24.831 INFO:teuthology.orchestra.run.smithi043.stdout: "key": "AQCVldNkzpKYEhAA+wen0kV/vUDLQztq3vhfOg==", 2023-08-09T13:55:24.831 INFO:teuthology.orchestra.run.smithi043.stdout: "caps": { 2023-08-09T13:55:24.831 INFO:teuthology.orchestra.run.smithi043.stdout: "mgr": "allow profile osd", 2023-08-09T13:55:24.831 INFO:teuthology.orchestra.run.smithi043.stdout: "mon": "allow profile osd", 2023-08-09T13:55:24.831 INFO:teuthology.orchestra.run.smithi043.stdout: "osd": "allow *" 2023-08-09T13:55:24.832 INFO:teuthology.orchestra.run.smithi043.stdout: } 2023-08-09T13:55:24.832 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2023-08-09T13:55:24.832 INFO:teuthology.orchestra.run.smithi043.stdout: { 2023-08-09T13:55:24.832 INFO:teuthology.orchestra.run.smithi043.stdout: "entity": "osd.2", 2023-08-09T13:55:24.832 INFO:teuthology.orchestra.run.smithi043.stdout: "key": "AQCYldNkYbwNHhAAQY8g9dldb1V9M2MM2SHKDg==", 2023-08-09T13:55:24.832 INFO:teuthology.orchestra.run.smithi043.stdout: "caps": { 2023-08-09T13:55:24.832 INFO:teuthology.orchestra.run.smithi043.stdout: "mgr": "allow profile osd", 2023-08-09T13:55:24.832 INFO:teuthology.orchestra.run.smithi043.stdout: "mon": "allow profile osd", 2023-08-09T13:55:24.833 INFO:teuthology.orchestra.run.smithi043.stdout: "osd": "allow *" 2023-08-09T13:55:24.833 INFO:teuthology.orchestra.run.smithi043.stdout: } 2023-08-09T13:55:24.833 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2023-08-09T13:55:24.833 INFO:teuthology.orchestra.run.smithi043.stdout: { 2023-08-09T13:55:24.833 INFO:teuthology.orchestra.run.smithi043.stdout: "entity": "osd.3", 2023-08-09T13:55:24.833 INFO:teuthology.orchestra.run.smithi043.stdout: "key": "AQCbldNkeyW6KRAAsd+kgqik2aD4U7PXwccWtA==", 2023-08-09T13:55:24.833 INFO:teuthology.orchestra.run.smithi043.stdout: "caps": { 2023-08-09T13:55:24.834 INFO:teuthology.orchestra.run.smithi043.stdout: "mgr": "allow profile osd", 2023-08-09T13:55:24.834 INFO:teuthology.orchestra.run.smithi043.stdout: "mon": "allow profile osd", 2023-08-09T13:55:24.834 INFO:teuthology.orchestra.run.smithi043.stdout: "osd": "allow *" 2023-08-09T13:55:24.834 INFO:teuthology.orchestra.run.smithi043.stdout: } 2023-08-09T13:55:24.834 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2023-08-09T13:55:24.834 INFO:teuthology.orchestra.run.smithi043.stdout: { 2023-08-09T13:55:24.834 INFO:teuthology.orchestra.run.smithi043.stdout: "entity": "osd.4", 2023-08-09T13:55:24.835 INFO:teuthology.orchestra.run.smithi043.stdout: "key": "AQCildNk80UkOBAAtX+4JGpOVcfZEHetJoeZKw==", 2023-08-09T13:55:24.835 INFO:teuthology.orchestra.run.smithi043.stdout: "caps": { 2023-08-09T13:55:24.835 INFO:teuthology.orchestra.run.smithi043.stdout: "mgr": "allow profile osd", 2023-08-09T13:55:24.835 INFO:teuthology.orchestra.run.smithi043.stdout: "mon": "allow profile osd", 2023-08-09T13:55:24.835 INFO:teuthology.orchestra.run.smithi043.stdout: "osd": "allow *" 2023-08-09T13:55:24.835 INFO:teuthology.orchestra.run.smithi043.stdout: } 2023-08-09T13:55:24.835 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2023-08-09T13:55:24.836 INFO:teuthology.orchestra.run.smithi043.stdout: { 2023-08-09T13:55:24.836 INFO:teuthology.orchestra.run.smithi043.stdout: "entity": "osd.5", 2023-08-09T13:55:24.836 INFO:teuthology.orchestra.run.smithi043.stdout: "key": "AQCmldNkCA9SBxAAZj4So6MRj2Eq1I9H9JzLDA==", 2023-08-09T13:55:24.836 INFO:teuthology.orchestra.run.smithi043.stdout: "caps": { 2023-08-09T13:55:24.836 INFO:teuthology.orchestra.run.smithi043.stdout: "mgr": "allow profile osd", 2023-08-09T13:55:24.837 INFO:teuthology.orchestra.run.smithi043.stdout: "mon": "allow profile osd", 2023-08-09T13:55:24.837 INFO:teuthology.orchestra.run.smithi043.stdout: "osd": "allow *" 2023-08-09T13:55:24.837 INFO:teuthology.orchestra.run.smithi043.stdout: } 2023-08-09T13:55:24.837 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2023-08-09T13:55:24.837 INFO:teuthology.orchestra.run.smithi043.stdout: { 2023-08-09T13:55:24.837 INFO:teuthology.orchestra.run.smithi043.stdout: "entity": "osd.6", 2023-08-09T13:55:24.837 INFO:teuthology.orchestra.run.smithi043.stdout: "key": "AQCpldNkaOwAExAAraV1tUwr9uU6GheZ+hvdqQ==", 2023-08-09T13:55:24.838 INFO:teuthology.orchestra.run.smithi043.stdout: "caps": { 2023-08-09T13:55:24.838 INFO:teuthology.orchestra.run.smithi043.stdout: "mgr": "allow profile osd", 2023-08-09T13:55:24.838 INFO:teuthology.orchestra.run.smithi043.stdout: "mon": "allow profile osd", 2023-08-09T13:55:24.838 INFO:teuthology.orchestra.run.smithi043.stdout: "osd": "allow *" 2023-08-09T13:55:24.838 INFO:teuthology.orchestra.run.smithi043.stdout: } 2023-08-09T13:55:24.838 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2023-08-09T13:55:24.838 INFO:teuthology.orchestra.run.smithi043.stdout: { 2023-08-09T13:55:24.838 INFO:teuthology.orchestra.run.smithi043.stdout: "entity": "osd.7", 2023-08-09T13:55:24.839 INFO:teuthology.orchestra.run.smithi043.stdout: "key": "AQCsldNk2/GwLBAAP42EXPivzLk44X8aMEVVCg==", 2023-08-09T13:55:24.839 INFO:teuthology.orchestra.run.smithi043.stdout: "caps": { 2023-08-09T13:55:24.839 INFO:teuthology.orchestra.run.smithi043.stdout: "mgr": "allow profile osd", 2023-08-09T13:55:24.839 INFO:teuthology.orchestra.run.smithi043.stdout: "mon": "allow profile osd", 2023-08-09T13:55:24.839 INFO:teuthology.orchestra.run.smithi043.stdout: "osd": "allow *" 2023-08-09T13:55:24.839 INFO:teuthology.orchestra.run.smithi043.stdout: } 2023-08-09T13:55:24.839 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2023-08-09T13:55:24.839 INFO:teuthology.orchestra.run.smithi043.stdout: { 2023-08-09T13:55:24.840 INFO:teuthology.orchestra.run.smithi043.stdout: "entity": "client.0", 2023-08-09T13:55:24.840 INFO:teuthology.orchestra.run.smithi043.stdout: "key": "AQCPldNkg8zrDxAAM/hnJaE6S9HU/fRwQeBV5w==", 2023-08-09T13:55:24.840 INFO:teuthology.orchestra.run.smithi043.stdout: "caps": { 2023-08-09T13:55:24.840 INFO:teuthology.orchestra.run.smithi043.stdout: "mds": "allow", 2023-08-09T13:55:24.840 INFO:teuthology.orchestra.run.smithi043.stdout: "mon": "allow r", 2023-08-09T13:55:24.840 INFO:teuthology.orchestra.run.smithi043.stdout: "osd": "allow rw tag cephfs data=cephfs" 2023-08-09T13:55:24.840 INFO:teuthology.orchestra.run.smithi043.stdout: } 2023-08-09T13:55:24.841 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2023-08-09T13:55:24.841 INFO:teuthology.orchestra.run.smithi043.stdout: { 2023-08-09T13:55:24.841 INFO:teuthology.orchestra.run.smithi043.stdout: "entity": "client.1", 2023-08-09T13:55:24.841 INFO:teuthology.orchestra.run.smithi043.stdout: "key": "AQCPldNklfFxJxAA19+L78rW6jU1uOyjey6WNA==", 2023-08-09T13:55:24.841 INFO:teuthology.orchestra.run.smithi043.stdout: "caps": { 2023-08-09T13:55:24.841 INFO:teuthology.orchestra.run.smithi043.stdout: "mds": "allow", 2023-08-09T13:55:24.841 INFO:teuthology.orchestra.run.smithi043.stdout: "mgr": "allow r", 2023-08-09T13:55:24.841 INFO:teuthology.orchestra.run.smithi043.stdout: "mon": "allow rw", 2023-08-09T13:55:24.842 INFO:teuthology.orchestra.run.smithi043.stdout: "osd": "allow rwx" 2023-08-09T13:55:24.842 INFO:teuthology.orchestra.run.smithi043.stdout: } 2023-08-09T13:55:24.842 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2023-08-09T13:55:24.842 INFO:teuthology.orchestra.run.smithi043.stdout: { 2023-08-09T13:55:24.842 INFO:teuthology.orchestra.run.smithi043.stdout: "entity": "client.2", 2023-08-09T13:55:24.842 INFO:teuthology.orchestra.run.smithi043.stdout: "key": "AQCPldNkC2lgGxAAZ8qZboOCUaQlcpkPL1ozHA==", 2023-08-09T13:55:24.842 INFO:teuthology.orchestra.run.smithi043.stdout: "caps": { 2023-08-09T13:55:24.843 INFO:teuthology.orchestra.run.smithi043.stdout: "mds": "allow", 2023-08-09T13:55:24.843 INFO:teuthology.orchestra.run.smithi043.stdout: "mgr": "allow r", 2023-08-09T13:55:24.843 INFO:teuthology.orchestra.run.smithi043.stdout: "mon": "allow rw", 2023-08-09T13:55:24.843 INFO:teuthology.orchestra.run.smithi043.stdout: "osd": "allow rwx" 2023-08-09T13:55:24.843 INFO:teuthology.orchestra.run.smithi043.stdout: } 2023-08-09T13:55:24.843 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2023-08-09T13:55:24.843 INFO:teuthology.orchestra.run.smithi043.stdout: { 2023-08-09T13:55:24.844 INFO:teuthology.orchestra.run.smithi043.stdout: "entity": "client.3", 2023-08-09T13:55:24.844 INFO:teuthology.orchestra.run.smithi043.stdout: "key": "AQCPldNkl+rXMhAA9rf3KdnSYbvngqmLYnIGGA==", 2023-08-09T13:55:24.844 INFO:teuthology.orchestra.run.smithi043.stdout: "caps": { 2023-08-09T13:55:24.844 INFO:teuthology.orchestra.run.smithi043.stdout: "mds": "allow", 2023-08-09T13:55:24.844 INFO:teuthology.orchestra.run.smithi043.stdout: "mgr": "allow r", 2023-08-09T13:55:24.844 INFO:teuthology.orchestra.run.smithi043.stdout: "mon": "allow rw", 2023-08-09T13:55:24.844 INFO:teuthology.orchestra.run.smithi043.stdout: "osd": "allow rwx" 2023-08-09T13:55:24.845 INFO:teuthology.orchestra.run.smithi043.stdout: } 2023-08-09T13:55:24.845 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2023-08-09T13:55:24.845 INFO:teuthology.orchestra.run.smithi043.stdout: { 2023-08-09T13:55:24.845 INFO:teuthology.orchestra.run.smithi043.stdout: "entity": "client.admin", 2023-08-09T13:55:24.845 INFO:teuthology.orchestra.run.smithi043.stdout: "key": "AQCMldNkUae6NRAA2FBTr0QPBdKNDoe1WK5zyg==", 2023-08-09T13:55:24.845 INFO:teuthology.orchestra.run.smithi043.stdout: "caps": { 2023-08-09T13:55:24.845 INFO:teuthology.orchestra.run.smithi043.stdout: "mds": "allow *", 2023-08-09T13:55:24.845 INFO:teuthology.orchestra.run.smithi043.stdout: "mgr": "allow *", 2023-08-09T13:55:24.846 INFO:teuthology.orchestra.run.smithi043.stdout: "mon": "allow *", 2023-08-09T13:55:24.846 INFO:teuthology.orchestra.run.smithi043.stdout: "osd": "allow *" 2023-08-09T13:55:24.846 INFO:teuthology.orchestra.run.smithi043.stdout: } 2023-08-09T13:55:24.846 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2023-08-09T13:55:24.846 INFO:teuthology.orchestra.run.smithi043.stdout: { 2023-08-09T13:55:24.846 INFO:teuthology.orchestra.run.smithi043.stdout: "entity": "mgr.x", 2023-08-09T13:55:24.846 INFO:teuthology.orchestra.run.smithi043.stdout: "key": "AQCNldNk2ufSNxAAU7msRkwLaoLZHTIsk0ck/g==", 2023-08-09T13:55:24.846 INFO:teuthology.orchestra.run.smithi043.stdout: "caps": { 2023-08-09T13:55:24.847 INFO:teuthology.orchestra.run.smithi043.stdout: "mds": "allow *", 2023-08-09T13:55:24.847 INFO:teuthology.orchestra.run.smithi043.stdout: "mon": "allow profile mgr", 2023-08-09T13:55:24.847 INFO:teuthology.orchestra.run.smithi043.stdout: "osd": "allow *" 2023-08-09T13:55:24.847 INFO:teuthology.orchestra.run.smithi043.stdout: } 2023-08-09T13:55:24.847 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2023-08-09T13:55:24.847 INFO:teuthology.orchestra.run.smithi043.stdout: { 2023-08-09T13:55:24.847 INFO:teuthology.orchestra.run.smithi043.stdout: "entity": "mgr.y", 2023-08-09T13:55:24.848 INFO:teuthology.orchestra.run.smithi043.stdout: "key": "AQCNldNk2YgzLBAAA11Q8uAMEgE/bUni+tXIKA==", 2023-08-09T13:55:24.848 INFO:teuthology.orchestra.run.smithi043.stdout: "caps": { 2023-08-09T13:55:24.848 INFO:teuthology.orchestra.run.smithi043.stdout: "mds": "allow *", 2023-08-09T13:55:24.848 INFO:teuthology.orchestra.run.smithi043.stdout: "mon": "allow profile mgr", 2023-08-09T13:55:24.848 INFO:teuthology.orchestra.run.smithi043.stdout: "osd": "allow *" 2023-08-09T13:55:24.848 INFO:teuthology.orchestra.run.smithi043.stdout: } 2023-08-09T13:55:24.848 INFO:teuthology.orchestra.run.smithi043.stdout: } 2023-08-09T13:55:24.848 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2023-08-09T13:55:24.849 INFO:teuthology.orchestra.run.smithi043.stdout:} 2023-08-09T13:55:24.850 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:24.824+0000 7f5828986640 1 -- 172.21.15.43:0/967525290 >> [v2:172.21.15.203:6832/3490266242,v1:172.21.15.203:6833/3490266242] conn(0x7f580406a1a0 msgr2=0x7f580406c660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:55:24.850 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:24.824+0000 7f5828986640 1 --2- 172.21.15.43:0/967525290 >> [v2:172.21.15.203:6832/3490266242,v1:172.21.15.203:6833/3490266242] conn(0x7f580406a1a0 0x7f580406c660 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7f58240b69e0 tx=0x7f580c009040 comp rx=0 tx=0).stop 2023-08-09T13:55:24.850 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:24.824+0000 7f5828986640 1 -- 172.21.15.43:0/967525290 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f58240b1050 msgr2=0x7f58240c8020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:55:24.850 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:24.824+0000 7f5828986640 1 --2- 172.21.15.43:0/967525290 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f58240b1050 0x7f58240c8020 secure :-1 s=READY pgs=1665 cs=0 l=1 rev1=1 crypto rx=0x7f581400d5f0 tx=0x7f581400dab0 comp rx=0 tx=0).stop 2023-08-09T13:55:24.850 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:24.824+0000 7f5828986640 1 -- 172.21.15.43:0/967525290 shutdown_connections 2023-08-09T13:55:24.851 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:24.824+0000 7f5828986640 1 --2- 172.21.15.43:0/967525290 >> [v2:172.21.15.203:6832/3490266242,v1:172.21.15.203:6833/3490266242] conn(0x7f580406a1a0 0x7f580406c660 unknown :-1 s=CLOSED pgs=50 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:55:24.851 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:24.824+0000 7f5828986640 1 --2- 172.21.15.43:0/967525290 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f582415f7b0 0x7f58240d0d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:55:24.851 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:24.824+0000 7f5828986640 1 --2- 172.21.15.43:0/967525290 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f58240cd6b0 0x7f58240c7110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:55:24.851 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:24.824+0000 7f5828986640 1 --2- 172.21.15.43:0/967525290 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f58240b1050 0x7f58240c8020 unknown :-1 s=CLOSED pgs=1665 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:55:24.851 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:24.824+0000 7f5828986640 1 -- 172.21.15.43:0/967525290 >> 172.21.15.43:0/967525290 conn(0x7f582406a2b0 msgr2=0x7f58240b5400 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-09T13:55:24.851 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:24.824+0000 7f5828986640 1 -- 172.21.15.43:0/967525290 shutdown_connections 2023-08-09T13:55:24.851 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:24.824+0000 7f5828986640 1 -- 172.21.15.43:0/967525290 wait complete. 2023-08-09T13:55:24.852 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-08-09T13:55:25.077 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:25.074+0000 7f19de283640 1 Processor -- start 2023-08-09T13:55:25.077 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:25.075+0000 7f19de283640 1 -- start start 2023-08-09T13:55:25.079 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:25.076+0000 7f19de283640 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7f19d8143c20 0x7f19d806f540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:55:25.079 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:25.076+0000 7f19de283640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f19d80c86f0 0x7f19d80c8b50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:55:25.079 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:25.076+0000 7f19de283640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f19d806c8d0 0x7f19d806cd60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:55:25.079 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:25.076+0000 7f19de283640 1 -- --> v2:172.21.15.43:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f19d8144000 con 0x7f19d806c8d0 2023-08-09T13:55:25.079 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:25.076+0000 7f19de283640 1 -- --> v2:172.21.15.203:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f19d80c9090 con 0x7f19d8143c20 2023-08-09T13:55:25.080 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:25.076+0000 7f19de283640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f19d806c440 con 0x7f19d80c86f0 2023-08-09T13:55:25.081 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:25.077+0000 7f19dc820640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f19d806c8d0 0x7f19d806cd60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:55:25.081 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:25.077+0000 7f19dc820640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f19d806c8d0 0x7f19d806cd60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:52416/0 (socket says 172.21.15.43:52416) 2023-08-09T13:55:25.081 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:25.077+0000 7f19dc820640 1 -- 172.21.15.43:0/1089077722 learned_addr learned my addr 172.21.15.43:0/1089077722 (peer_addr_for_me v2:172.21.15.43:0/0) 2023-08-09T13:55:25.081 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:25.077+0000 7f19d7fff640 1 --2- 172.21.15.43:0/1089077722 >> v2:172.21.15.203:3300/0 conn(0x7f19d8143c20 0x7f19d806f540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:55:25.081 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:25.077+0000 7f19dc820640 1 -- 172.21.15.43:0/1089077722 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f19d80c86f0 msgr2=0x7f19d80c8b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-09T13:55:25.081 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:25.077+0000 7f19dc820640 1 --2- 172.21.15.43:0/1089077722 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f19d80c86f0 0x7f19d80c8b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:55:25.081 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:25.077+0000 7f19dc820640 1 -- 172.21.15.43:0/1089077722 >> v2:172.21.15.203:3300/0 conn(0x7f19d8143c20 msgr2=0x7f19d806f540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-09T13:55:25.082 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:25.077+0000 7f19dc820640 1 --2- 172.21.15.43:0/1089077722 >> v2:172.21.15.203:3300/0 conn(0x7f19d8143c20 0x7f19d806f540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:55:25.082 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:25.077+0000 7f19dc820640 1 -- 172.21.15.43:0/1089077722 --> v2:172.21.15.43:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f19d806d2a0 con 0x7f19d806c8d0 2023-08-09T13:55:25.082 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:25.077+0000 7f19d7fff640 1 --2- 172.21.15.43:0/1089077722 >> v2:172.21.15.203:3300/0 conn(0x7f19d8143c20 0x7f19d806f540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-08-09T13:55:25.082 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:25.079+0000 7f19dc820640 1 --2- 172.21.15.43:0/1089077722 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f19d806c8d0 0x7f19d806cd60 secure :-1 s=READY pgs=1666 cs=0 l=1 rev1=1 crypto rx=0x7f19d806e0f0 tx=0x7f19d002be50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4f5dfeb38a82d265 server_cookie=0 in_seq=0 out_seq=0 2023-08-09T13:55:25.083 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:25.079+0000 7f19d6ffd640 1 -- 172.21.15.43:0/1089077722 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f19d002c9f0 con 0x7f19d806c8d0 2023-08-09T13:55:25.083 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:25.079+0000 7f19d6ffd640 1 -- 172.21.15.43:0/1089077722 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f19d002cb50 con 0x7f19d806c8d0 2023-08-09T13:55:25.083 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:25.079+0000 7f19d6ffd640 1 -- 172.21.15.43:0/1089077722 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f19d00343f0 con 0x7f19d806c8d0 2023-08-09T13:55:25.083 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:25.079+0000 7f19de283640 1 -- 172.21.15.43:0/1089077722 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f19d806c8d0 msgr2=0x7f19d806cd60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:55:25.083 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:25.079+0000 7f19de283640 1 --2- 172.21.15.43:0/1089077722 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f19d806c8d0 0x7f19d806cd60 secure :-1 s=READY pgs=1666 cs=0 l=1 rev1=1 crypto rx=0x7f19d806e0f0 tx=0x7f19d002be50 comp rx=0 tx=0).stop 2023-08-09T13:55:25.084 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:25.080+0000 7f19de283640 1 -- 172.21.15.43:0/1089077722 shutdown_connections 2023-08-09T13:55:25.084 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:25.080+0000 7f19de283640 1 --2- 172.21.15.43:0/1089077722 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f19d806c8d0 0x7f19d806cd60 unknown :-1 s=CLOSED pgs=1666 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:55:25.084 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:25.080+0000 7f19de283640 1 --2- 172.21.15.43:0/1089077722 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f19d80c86f0 0x7f19d80c8b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:55:25.084 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:25.080+0000 7f19de283640 1 --2- 172.21.15.43:0/1089077722 >> v2:172.21.15.203:3300/0 conn(0x7f19d8143c20 0x7f19d806f540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:55:25.084 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:25.080+0000 7f19de283640 1 -- 172.21.15.43:0/1089077722 >> 172.21.15.43:0/1089077722 conn(0x7f19d80ce0a0 msgr2=0x7f19d80ce480 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-09T13:55:25.084 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:25.080+0000 7f19de283640 1 -- 172.21.15.43:0/1089077722 shutdown_connections 2023-08-09T13:55:25.084 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:25.080+0000 7f19de283640 1 -- 172.21.15.43:0/1089077722 wait complete. 2023-08-09T13:55:25.086 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:25.081+0000 7f19de283640 1 Processor -- start 2023-08-09T13:55:25.086 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:25.082+0000 7f19de283640 1 -- start start 2023-08-09T13:55:25.086 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:25.083+0000 7f19de283640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f19d806c8d0 0x7f19d80c7ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:55:25.086 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:25.083+0000 7f19de283640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f19d80c86f0 0x7f19d80c5d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:55:25.087 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:25.083+0000 7f19de283640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f19d8143c20 0x7f19d8130310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:55:25.087 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:25.083+0000 7f19de283640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f19d815c160 con 0x7f19d8143c20 2023-08-09T13:55:25.087 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:25.083+0000 7f19de283640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f19d80c8000 con 0x7f19d806c8d0 2023-08-09T13:55:25.087 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:25.083+0000 7f19de283640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f19d815b550 con 0x7f19d80c86f0 2023-08-09T13:55:25.087 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:25.083+0000 7f19dc820640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f19d8143c20 0x7f19d8130310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:55:25.087 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:25.083+0000 7f19dc820640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f19d8143c20 0x7f19d8130310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:52422/0 (socket says 172.21.15.43:52422) 2023-08-09T13:55:25.087 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:25.083+0000 7f19dc820640 1 -- 172.21.15.43:0/2320302420 learned_addr learned my addr 172.21.15.43:0/2320302420 (peer_addr_for_me v2:172.21.15.43:0/0) 2023-08-09T13:55:25.088 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:25.083+0000 7f19d7fff640 1 --2- 172.21.15.43:0/2320302420 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f19d806c8d0 0x7f19d80c7ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:55:25.088 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:25.083+0000 7f19d77fe640 1 --2- 172.21.15.43:0/2320302420 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f19d80c86f0 0x7f19d80c5d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:55:25.088 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:25.084+0000 7f19dc820640 1 -- 172.21.15.43:0/2320302420 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f19d80c86f0 msgr2=0x7f19d80c5d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:55:25.088 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:25.084+0000 7f19dc820640 1 --2- 172.21.15.43:0/2320302420 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f19d80c86f0 0x7f19d80c5d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:55:25.088 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:25.084+0000 7f19dc820640 1 -- 172.21.15.43:0/2320302420 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f19d806c8d0 msgr2=0x7f19d80c7ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:55:25.088 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:25.084+0000 7f19dc820640 1 --2- 172.21.15.43:0/2320302420 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f19d806c8d0 0x7f19d80c7ac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:55:25.088 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:25.084+0000 7f19dc820640 1 -- 172.21.15.43:0/2320302420 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f19d0009b30 con 0x7f19d8143c20 2023-08-09T13:55:25.089 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:25.084+0000 7f19d77fe640 1 --2- 172.21.15.43:0/2320302420 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f19d80c86f0 0x7f19d80c5d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-08-09T13:55:25.090 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:25.084+0000 7f19dc820640 1 --2- 172.21.15.43:0/2320302420 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f19d8143c20 0x7f19d8130310 secure :-1 s=READY pgs=1667 cs=0 l=1 rev1=1 crypto rx=0x7f19d0009e50 tx=0x7f19d0004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-09T13:55:25.090 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:25.084+0000 7f19d4ff9640 1 -- 172.21.15.43:0/2320302420 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f19d0009510 con 0x7f19d8143c20 2023-08-09T13:55:25.090 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:25.084+0000 7f19d4ff9640 1 -- 172.21.15.43:0/2320302420 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f19d003e450 con 0x7f19d8143c20 2023-08-09T13:55:25.090 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:25.084+0000 7f19d4ff9640 1 -- 172.21.15.43:0/2320302420 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f19d003e800 con 0x7f19d8143c20 2023-08-09T13:55:25.090 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:25.084+0000 7f19de283640 1 -- 172.21.15.43:0/2320302420 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f19d815b830 con 0x7f19d8143c20 2023-08-09T13:55:25.090 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:25.084+0000 7f19de283640 1 -- 172.21.15.43:0/2320302420 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f19d815bdf0 con 0x7f19d8143c20 2023-08-09T13:55:25.090 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:25.086+0000 7f19de283640 1 -- 172.21.15.43:0/2320302420 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f19d815c310 con 0x7f19d8143c20 2023-08-09T13:55:25.091 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:25.086+0000 7f19d4ff9640 1 -- 172.21.15.43:0/2320302420 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 67) v1 ==== 88443+0+0 (secure 0 0 0) 0x7f19d003e960 con 0x7f19d8143c20 2023-08-09T13:55:25.091 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:25.087+0000 7f19d4ff9640 1 --2- 172.21.15.43:0/2320302420 >> [v2:172.21.15.203:6832/3490266242,v1:172.21.15.203:6833/3490266242] conn(0x7f19c406a170 0x7f19c406c630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:55:25.091 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:25.087+0000 7f19d4ff9640 1 -- 172.21.15.43:0/2320302420 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(143..143 src has 1..143) v4 ==== 5574+0+0 (secure 0 0 0) 0x7f19d00ab370 con 0x7f19d8143c20 2023-08-09T13:55:25.091 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:25.088+0000 7f19d7fff640 1 --2- 172.21.15.43:0/2320302420 >> [v2:172.21.15.203:6832/3490266242,v1:172.21.15.203:6833/3490266242] conn(0x7f19c406a170 0x7f19c406c630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:55:25.091 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:25.089+0000 7f19d7fff640 1 --2- 172.21.15.43:0/2320302420 >> [v2:172.21.15.203:6832/3490266242,v1:172.21.15.203:6833/3490266242] conn(0x7f19c406a170 0x7f19c406c630 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7f19c800a990 tx=0x7f19c8008040 comp rx=0 tx=0).ready entity=mgr.6659 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-09T13:55:25.094 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:25.092+0000 7f19d4ff9640 1 -- 172.21.15.43:0/2320302420 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178066 (secure 0 0 0) 0x7f19d815c310 con 0x7f19d8143c20 2023-08-09T13:55:25.297 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:25.294+0000 7f19de283640 1 -- 172.21.15.43:0/2320302420 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f19d80739f0 con 0x7f19d8143c20 2023-08-09T13:55:25.298 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:25.296+0000 7f19d4ff9640 1 -- 172.21.15.43:0/2320302420 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v143) v1 ==== 74+0+14854 (secure 0 0 0) 0x7f19d0076a90 con 0x7f19d8143c20 2023-08-09T13:55:25.299 INFO:teuthology.orchestra.run.smithi043.stdout: 2023-08-09T13:55:25.299 INFO:teuthology.orchestra.run.smithi043.stdout:{"epoch":143,"fsid":"daac87ee-1337-4250-a955-aea4e408a39b","created":"2023-08-09T13:33:41.701198+0000","modified":"2023-08-09T13:55:24.269281+0000","last_up_change":"2023-08-09T13:33:53.783453+0000","last_in_change":"2023-08-09T13:33:45.148904+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":17,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":"rbd","create_time":"2023-08-09T13:33:54.365137+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":".mgr","create_time":"2023-08-09T13:33:54.831803+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"cbd49ecd-e896-4fbd-864e-5c37a829b165","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":127,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6800","nonce":502667335},{"type":"v1","addr":"172.21.15.43:6801","nonce":502667335}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6802","nonce":502667335},{"type":"v1","addr":"172.21.15.43:6803","nonce":502667335}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6806","nonce":502667335},{"type":"v1","addr":"172.21.15.43:6807","nonce":502667335}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6804","nonce":502667335},{"type":"v1","addr":"172.21.15.43:6805","nonce":502667335}]},"public_addr":"172.21.15.43:6801/502667335","cluster_addr":"172.21.15.43:6803/502667335","heartbeat_back_addr":"172.21.15.43:6807/502667335","heartbeat_front_addr":"172.21.15.43:6805/502667335","state":["exists","up"]},{"osd":1,"uuid":"75c7e0fe-1ec7-4fe8-a2f2-3661a65b8500","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":131,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6808","nonce":3289134610},{"type":"v1","addr":"172.21.15.43:6809","nonce":3289134610}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6810","nonce":3289134610},{"type":"v1","addr":"172.21.15.43:6812","nonce":3289134610}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6821","nonce":3289134610},{"type":"v1","addr":"172.21.15.43:6824","nonce":3289134610}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6815","nonce":3289134610},{"type":"v1","addr":"172.21.15.43:6818","nonce":3289134610}]},"public_addr":"172.21.15.43:6809/3289134610","cluster_addr":"172.21.15.43:6812/3289134610","heartbeat_back_addr":"172.21.15.43:6824/3289134610","heartbeat_front_addr":"172.21.15.43:6818/3289134610","state":["exists","up"]},{"osd":2,"uuid":"bb5c41e9-7f5f-4125-9b01-3bb8bfc2ea83","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":127,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6811","nonce":4079709736},{"type":"v1","addr":"172.21.15.43:6814","nonce":4079709736}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6817","nonce":4079709736},{"type":"v1","addr":"172.21.15.43:6820","nonce":4079709736}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6829","nonce":4079709736},{"type":"v1","addr":"172.21.15.43:6831","nonce":4079709736}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6823","nonce":4079709736},{"type":"v1","addr":"172.21.15.43:6826","nonce":4079709736}]},"public_addr":"172.21.15.43:6814/4079709736","cluster_addr":"172.21.15.43:6820/4079709736","heartbeat_back_addr":"172.21.15.43:6831/4079709736","heartbeat_front_addr":"172.21.15.43:6826/4079709736","state":["exists","up"]},{"osd":3,"uuid":"bfe4c8b4-694a-465c-87e6-2b2aab852613","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":127,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6813","nonce":1709538759},{"type":"v1","addr":"172.21.15.43:6816","nonce":1709538759}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6819","nonce":1709538759},{"type":"v1","addr":"172.21.15.43:6822","nonce":1709538759}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6828","nonce":1709538759},{"type":"v1","addr":"172.21.15.43:6830","nonce":1709538759}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6825","nonce":1709538759},{"type":"v1","addr":"172.21.15.43:6827","nonce":1709538759}]},"public_addr":"172.21.15.43:6816/1709538759","cluster_addr":"172.21.15.43:6822/1709538759","heartbeat_back_addr":"172.21.15.43:6830/1709538759","heartbeat_front_addr":"172.21.15.43:6827/1709538759","state":["exists","up"]},{"osd":4,"uuid":"fa2e8a77-aba1-4756-853e-0d174a5bdeba","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":127,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6800","nonce":4196866481},{"type":"v1","addr":"172.21.15.203:6801","nonce":4196866481}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6802","nonce":4196866481},{"type":"v1","addr":"172.21.15.203:6803","nonce":4196866481}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6806","nonce":4196866481},{"type":"v1","addr":"172.21.15.203:6807","nonce":4196866481}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6804","nonce":4196866481},{"type":"v1","addr":"172.21.15.203:6805","nonce":4196866481}]},"public_addr":"172.21.15.203:6801/4196866481","cluster_addr":"172.21.15.203:6803/4196866481","heartbeat_back_addr":"172.21.15.203:6807/4196866481","heartbeat_front_addr":"172.21.15.203:6805/4196866481","state":["exists","up"]},{"osd":5,"uuid":"81811e4b-4d24-4f51-a82c-f1d700a2a00d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":127,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6816","nonce":3059290081},{"type":"v1","addr":"172.21.15.203:6817","nonce":3059290081}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6818","nonce":3059290081},{"type":"v1","addr":"172.21.15.203:6819","nonce":3059290081}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6822","nonce":3059290081},{"type":"v1","addr":"172.21.15.203:6823","nonce":3059290081}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6820","nonce":3059290081},{"type":"v1","addr":"172.21.15.203:6821","nonce":3059290081}]},"public_addr":"172.21.15.203:6817/3059290081","cluster_addr":"172.21.15.203:6819/3059290081","heartbeat_back_addr":"172.21.15.203:6823/3059290081","heartbeat_front_addr":"172.21.15.203:6821/3059290081","state":["exists","up"]},{"osd":6,"uuid":"307bac4d-6a5b-443d-af75-56d695d7674f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":131,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6824","nonce":512155569},{"type":"v1","addr":"172.21.15.203:6825","nonce":512155569}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6826","nonce":512155569},{"type":"v1","addr":"172.21.15.203:6827","nonce":512155569}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6830","nonce":512155569},{"type":"v1","addr":"172.21.15.203:6831","nonce":512155569}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6828","nonce":512155569},{"type":"v1","addr":"172.21.15.203:6829","nonce":512155569}]},"public_addr":"172.21.15.203:6825/512155569","cluster_addr":"172.21.15.203:6827/512155569","heartbeat_back_addr":"172.21.15.203:6831/512155569","heartbeat_front_addr":"172.21.15.203:6829/512155569","state":["exists","up"]},{"osd":7,"uuid":"a95dc7ca-9f1b-4295-af70-68a22a2d7d5e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":127,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6808","nonce":3331450438},{"type":"v1","addr":"172.21.15.203:6809","nonce":3331450438}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6810","nonce":3331450438},{"type":"v1","addr":"172.21.15.203:6811","nonce":3331450438}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6814","nonce":3331450438},{"type":"v1","addr":"172.21.15.203:6815","nonce":3331450438}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6812","nonce":3331450438},{"type":"v1","addr":"172.21.15.203:6813","nonce":3331450438}]},"public_addr":"172.21.15.203:6809/3331450438","cluster_addr":"172.21.15.203:6811/3331450438","heartbeat_back_addr":"172.21.15.203:6815/3331450438","heartbeat_front_addr":"172.21.15.203:6813/3331450438","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-09T13:33:50.547652+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-09T13:33:50.589469+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-09T13:33:50.561724+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-09T13:33:50.599650+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-09T13:33:50.411855+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-09T13:33:50.219681+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-09T13:33:50.504664+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-09T13:33:50.441248+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":7,"snaps":[{"begin":104,"length":1},{"begin":221,"length":1}]}],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-08-09T13:55:25.301 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:25.299+0000 7f19de283640 1 -- 172.21.15.43:0/2320302420 >> [v2:172.21.15.203:6832/3490266242,v1:172.21.15.203:6833/3490266242] conn(0x7f19c406a170 msgr2=0x7f19c406c630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:55:25.301 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:25.299+0000 7f19de283640 1 --2- 172.21.15.43:0/2320302420 >> [v2:172.21.15.203:6832/3490266242,v1:172.21.15.203:6833/3490266242] conn(0x7f19c406a170 0x7f19c406c630 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7f19c800a990 tx=0x7f19c8008040 comp rx=0 tx=0).stop 2023-08-09T13:55:25.302 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:25.299+0000 7f19de283640 1 -- 172.21.15.43:0/2320302420 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f19d8143c20 msgr2=0x7f19d8130310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:55:25.302 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:25.299+0000 7f19de283640 1 --2- 172.21.15.43:0/2320302420 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f19d8143c20 0x7f19d8130310 secure :-1 s=READY pgs=1667 cs=0 l=1 rev1=1 crypto rx=0x7f19d0009e50 tx=0x7f19d0004a40 comp rx=0 tx=0).stop 2023-08-09T13:55:25.302 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:25.299+0000 7f19de283640 1 -- 172.21.15.43:0/2320302420 shutdown_connections 2023-08-09T13:55:25.302 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:25.299+0000 7f19de283640 1 --2- 172.21.15.43:0/2320302420 >> [v2:172.21.15.203:6832/3490266242,v1:172.21.15.203:6833/3490266242] conn(0x7f19c406a170 0x7f19c406c630 unknown :-1 s=CLOSED pgs=51 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:55:25.302 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:25.299+0000 7f19de283640 1 --2- 172.21.15.43:0/2320302420 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f19d8143c20 0x7f19d8130310 unknown :-1 s=CLOSED pgs=1667 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:55:25.302 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:25.299+0000 7f19de283640 1 --2- 172.21.15.43:0/2320302420 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f19d80c86f0 0x7f19d80c5d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:55:25.302 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:25.299+0000 7f19de283640 1 --2- 172.21.15.43:0/2320302420 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f19d806c8d0 0x7f19d80c7ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:55:25.303 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:25.299+0000 7f19de283640 1 -- 172.21.15.43:0/2320302420 >> 172.21.15.43:0/2320302420 conn(0x7f19d80ce0a0 msgr2=0x7f19d80c9220 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-09T13:55:25.303 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:25.299+0000 7f19de283640 1 -- 172.21.15.43:0/2320302420 shutdown_connections 2023-08-09T13:55:25.303 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:25.299+0000 7f19de283640 1 -- 172.21.15.43:0/2320302420 wait complete. 2023-08-09T13:55:25.321 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': 'rbd', 'create_time': '2023-08-09T13:33:54.365137+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}, 'read_balance': {'score_acting': 2, 'score_stable': 2, 'optimal_score': 1, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': '.mgr', 'create_time': '2023-08-09T13:33:54.831803+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-08-09T13:55:25.322 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get rbd pg_num 2023-08-09T13:55:25.554 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:25.552+0000 7fa1c1c0f640 1 Processor -- start 2023-08-09T13:55:25.555 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:25.552+0000 7fa1c1c0f640 1 -- start start 2023-08-09T13:55:25.556 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:25.554+0000 7fa1c1c0f640 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7fa1bc15f970 0x7fa1bc0cf1d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:55:25.556 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:25.554+0000 7fa1c1c0f640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7fa1bc0d1ec0 0x7fa1bc0b5670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:55:25.556 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:25.554+0000 7fa1c1c0f640 1 -- --> v1:172.21.15.43:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa1bc0d24a0 con 0x7fa1bc0cf710 2023-08-09T13:55:25.557 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:25.554+0000 7fa1c1c0f640 1 -- --> v2:172.21.15.43:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa1bc0d1280 con 0x7fa1bc0d1ec0 2023-08-09T13:55:25.557 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:25.554+0000 7fa1c1c0f640 1 -- --> v2:172.21.15.203:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa1bc0cfba0 con 0x7fa1bc15f970 2023-08-09T13:55:25.558 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:25.554+0000 7fa1b3fff640 1 --1- >> v1:172.21.15.43:6789/0 conn(0x7fa1bc0cf710 0x7fa1bc0d17b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.43:6789/0 says I am v1:172.21.15.43:56562/0 (socket says 172.21.15.43:56562) 2023-08-09T13:55:25.558 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:25.554+0000 7fa1b3fff640 1 -- 172.21.15.43:0/2561609865 learned_addr learned my addr 172.21.15.43:0/2561609865 (peer_addr_for_me v1:172.21.15.43:0/0) 2023-08-09T13:55:25.558 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:25.554+0000 7fa1c140e640 1 --2- 172.21.15.43:0/2561609865 >> v2:172.21.15.43:3300/0 conn(0x7fa1bc0d1ec0 0x7fa1bc0b5670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:55:25.558 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:25.555+0000 7fa1c0c0d640 1 --2- 172.21.15.43:0/2561609865 >> v2:172.21.15.203:3300/0 conn(0x7fa1bc15f970 0x7fa1bc0cf1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:55:25.558 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:25.555+0000 7fa1b37fe640 1 -- 172.21.15.43:0/2561609865 <== mon.0 v1:172.21.15.43:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3825281947 0 0) 0x7fa1bc0d24a0 con 0x7fa1bc0cf710 2023-08-09T13:55:25.558 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:25.555+0000 7fa1b37fe640 1 -- 172.21.15.43:0/2561609865 --> v1:172.21.15.43:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fa1a4003520 con 0x7fa1bc0cf710 2023-08-09T13:55:25.558 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:25.555+0000 7fa1c140e640 1 -- 172.21.15.43:0/2561609865 >> v2:172.21.15.203:3300/0 conn(0x7fa1bc15f970 msgr2=0x7fa1bc0cf1d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-09T13:55:25.559 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:25.555+0000 7fa1c140e640 1 --2- 172.21.15.43:0/2561609865 >> v2:172.21.15.203:3300/0 conn(0x7fa1bc15f970 0x7fa1bc0cf1d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:55:25.559 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:25.555+0000 7fa1c140e640 1 -- 172.21.15.43:0/2561609865 >> v1:172.21.15.43:6789/0 conn(0x7fa1bc0cf710 legacy=0x7fa1bc0d17b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:55:25.559 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:25.555+0000 7fa1c140e640 1 -- 172.21.15.43:0/2561609865 --> v2:172.21.15.43:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa1bc0d2780 con 0x7fa1bc0d1ec0 2023-08-09T13:55:25.559 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:25.555+0000 7fa1c0c0d640 1 --2- 172.21.15.43:0/2561609865 >> v2:172.21.15.203:3300/0 conn(0x7fa1bc15f970 0x7fa1bc0cf1d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-08-09T13:55:25.560 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:25.556+0000 7fa1c140e640 1 --2- 172.21.15.43:0/2561609865 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa1bc0d1ec0 0x7fa1bc0b5670 secure :-1 s=READY pgs=1669 cs=0 l=1 rev1=1 crypto rx=0x7fa1bc0b4940 tx=0x7fa1b402be40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=df07a491e16fbd77 server_cookie=0 in_seq=0 out_seq=0 2023-08-09T13:55:25.560 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:25.556+0000 7fa1b37fe640 1 -- 172.21.15.43:0/2561609865 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa1b402cbf0 con 0x7fa1bc0d1ec0 2023-08-09T13:55:25.560 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:25.557+0000 7fa1b37fe640 1 -- 172.21.15.43:0/2561609865 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa1b402cd50 con 0x7fa1bc0d1ec0 2023-08-09T13:55:25.560 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:25.557+0000 7fa1b37fe640 1 -- 172.21.15.43:0/2561609865 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa1b4034640 con 0x7fa1bc0d1ec0 2023-08-09T13:55:25.560 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:25.557+0000 7fa1c1c0f640 1 -- 172.21.15.43:0/2561609865 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa1bc0d1ec0 msgr2=0x7fa1bc0b5670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:55:25.561 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:25.557+0000 7fa1c1c0f640 1 --2- 172.21.15.43:0/2561609865 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa1bc0d1ec0 0x7fa1bc0b5670 secure :-1 s=READY pgs=1669 cs=0 l=1 rev1=1 crypto rx=0x7fa1bc0b4940 tx=0x7fa1b402be40 comp rx=0 tx=0).stop 2023-08-09T13:55:25.561 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:25.557+0000 7fa1c1c0f640 1 -- 172.21.15.43:0/2561609865 shutdown_connections 2023-08-09T13:55:25.561 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:25.557+0000 7fa1c1c0f640 1 --2- 172.21.15.43:0/2561609865 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa1bc0d1ec0 0x7fa1bc0b5670 unknown :-1 s=CLOSED pgs=1669 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:55:25.561 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:25.557+0000 7fa1c1c0f640 1 --2- 172.21.15.43:0/2561609865 >> v2:172.21.15.203:3300/0 conn(0x7fa1bc15f970 0x7fa1bc0cf1d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:55:25.561 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:25.557+0000 7fa1c1c0f640 1 -- 172.21.15.43:0/2561609865 >> 172.21.15.43:0/2561609865 conn(0x7fa1bc0712e0 msgr2=0x7fa1bc0716f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-09T13:55:25.561 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:25.557+0000 7fa1c1c0f640 1 -- 172.21.15.43:0/2561609865 shutdown_connections 2023-08-09T13:55:25.561 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:25.557+0000 7fa1c1c0f640 1 -- 172.21.15.43:0/2561609865 wait complete. 2023-08-09T13:55:25.562 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:25.559+0000 7fa1c1c0f640 1 Processor -- start 2023-08-09T13:55:25.562 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:25.559+0000 7fa1c1c0f640 1 -- start start 2023-08-09T13:55:25.563 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:25.560+0000 7fa1c1c0f640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fa1bc0cf710 0x7fa1bc15e8c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:55:25.563 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:25.560+0000 7fa1c1c0f640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa1bc0d1ec0 0x7fa1bc1503d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:55:25.563 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:25.560+0000 7fa1c1c0f640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fa1bc15f970 0x7fa1bc150910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:55:25.563 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:25.560+0000 7fa1c1c0f640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa1bc0c69d0 con 0x7fa1bc0d1ec0 2023-08-09T13:55:25.563 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:25.560+0000 7fa1c1c0f640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa1bc0c6cb0 con 0x7fa1bc15f970 2023-08-09T13:55:25.564 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:25.560+0000 7fa1c1c0f640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa1bc0c6f90 con 0x7fa1bc0cf710 2023-08-09T13:55:25.564 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:25.561+0000 7fa1b3fff640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa1bc0d1ec0 0x7fa1bc1503d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:55:25.564 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:25.561+0000 7fa1b3fff640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa1bc0d1ec0 0x7fa1bc1503d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:52448/0 (socket says 172.21.15.43:52448) 2023-08-09T13:55:25.564 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:25.561+0000 7fa1b3fff640 1 -- 172.21.15.43:0/3970767647 learned_addr learned my addr 172.21.15.43:0/3970767647 (peer_addr_for_me v2:172.21.15.43:0/0) 2023-08-09T13:55:25.565 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:25.561+0000 7fa1c0c0d640 1 --2- 172.21.15.43:0/3970767647 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fa1bc0cf710 0x7fa1bc15e8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:55:25.565 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:25.561+0000 7fa1c140e640 1 --2- 172.21.15.43:0/3970767647 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fa1bc15f970 0x7fa1bc150910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:55:25.565 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:25.561+0000 7fa1b3fff640 1 -- 172.21.15.43:0/3970767647 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fa1bc0cf710 msgr2=0x7fa1bc15e8c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:55:25.565 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:25.561+0000 7fa1b3fff640 1 --2- 172.21.15.43:0/3970767647 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fa1bc0cf710 0x7fa1bc15e8c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:55:25.566 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:25.561+0000 7fa1b3fff640 1 -- 172.21.15.43:0/3970767647 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fa1bc15f970 msgr2=0x7fa1bc150910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:55:25.566 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:25.561+0000 7fa1b3fff640 1 --2- 172.21.15.43:0/3970767647 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fa1bc15f970 0x7fa1bc150910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:55:25.566 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:25.561+0000 7fa1b3fff640 1 -- 172.21.15.43:0/3970767647 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa1b4009a70 con 0x7fa1bc0d1ec0 2023-08-09T13:55:25.566 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:25.562+0000 7fa1b3fff640 1 --2- 172.21.15.43:0/3970767647 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa1bc0d1ec0 0x7fa1bc1503d0 secure :-1 s=READY pgs=1670 cs=0 l=1 rev1=1 crypto rx=0x7fa1ac00c430 tx=0x7fa1ac00c8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-09T13:55:25.566 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:25.562+0000 7fa1b17fa640 1 -- 172.21.15.43:0/3970767647 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa1ac005ce0 con 0x7fa1bc0d1ec0 2023-08-09T13:55:25.566 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:25.562+0000 7fa1b17fa640 1 -- 172.21.15.43:0/3970767647 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa1ac005e40 con 0x7fa1bc0d1ec0 2023-08-09T13:55:25.566 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:25.562+0000 7fa1c1c0f640 1 -- 172.21.15.43:0/3970767647 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa1bc0c7270 con 0x7fa1bc0d1ec0 2023-08-09T13:55:25.566 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:25.562+0000 7fa1b17fa640 1 -- 172.21.15.43:0/3970767647 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa1ac0072f0 con 0x7fa1bc0d1ec0 2023-08-09T13:55:25.567 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:25.562+0000 7fa1c1c0f640 1 -- 172.21.15.43:0/3970767647 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa1bc15ee00 con 0x7fa1bc0d1ec0 2023-08-09T13:55:25.567 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:25.563+0000 7fa1c1c0f640 1 -- 172.21.15.43:0/3970767647 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa1bc0c7790 con 0x7fa1bc0d1ec0 2023-08-09T13:55:25.567 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:25.564+0000 7fa1b17fa640 1 -- 172.21.15.43:0/3970767647 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 67) v1 ==== 88443+0+0 (secure 0 0 0) 0x7fa1ac007450 con 0x7fa1bc0d1ec0 2023-08-09T13:55:25.568 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:25.565+0000 7fa1b17fa640 1 --2- 172.21.15.43:0/3970767647 >> [v2:172.21.15.203:6832/3490266242,v1:172.21.15.203:6833/3490266242] conn(0x7fa1a806a1a0 0x7fa1a806c660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:55:25.568 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:25.565+0000 7fa1b17fa640 1 -- 172.21.15.43:0/3970767647 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(143..143 src has 1..143) v4 ==== 5574+0+0 (secure 0 0 0) 0x7fa1ac087bc0 con 0x7fa1bc0d1ec0 2023-08-09T13:55:25.568 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:25.566+0000 7fa1c0c0d640 1 --2- 172.21.15.43:0/3970767647 >> [v2:172.21.15.203:6832/3490266242,v1:172.21.15.203:6833/3490266242] conn(0x7fa1a806a1a0 0x7fa1a806c660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:55:25.569 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:25.567+0000 7fa1c0c0d640 1 --2- 172.21.15.43:0/3970767647 >> [v2:172.21.15.203:6832/3490266242,v1:172.21.15.203:6833/3490266242] conn(0x7fa1a806a1a0 0x7fa1a806c660 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7fa1bc0b21a0 tx=0x7fa1b8008040 comp rx=0 tx=0).ready entity=mgr.6659 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-09T13:55:25.572 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:25.569+0000 7fa1b17fa640 1 -- 172.21.15.43:0/3970767647 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178066 (secure 0 0 0) 0x7fa1bc0c7790 con 0x7fa1bc0d1ec0 2023-08-09T13:55:25.766 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:25.763+0000 7fa1c1c0f640 1 -- 172.21.15.43:0/3970767647 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"} v 0) v1 -- 0x7fa1bc0c7790 con 0x7fa1bc0d1ec0 2023-08-09T13:55:25.767 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:25.764+0000 7fa1b17fa640 1 -- 172.21.15.43:0/3970767647 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"}]=0 v143) v1 ==== 92+0+10 (secure 0 0 0) 0x7fa1bc0c7790 con 0x7fa1bc0d1ec0 2023-08-09T13:55:25.767 INFO:teuthology.orchestra.run.smithi043.stdout:pg_num: 8 2023-08-09T13:55:25.769 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:25.767+0000 7fa1c1c0f640 1 -- 172.21.15.43:0/3970767647 >> [v2:172.21.15.203:6832/3490266242,v1:172.21.15.203:6833/3490266242] conn(0x7fa1a806a1a0 msgr2=0x7fa1a806c660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:55:25.769 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:25.767+0000 7fa1c1c0f640 1 --2- 172.21.15.43:0/3970767647 >> [v2:172.21.15.203:6832/3490266242,v1:172.21.15.203:6833/3490266242] conn(0x7fa1a806a1a0 0x7fa1a806c660 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7fa1bc0b21a0 tx=0x7fa1b8008040 comp rx=0 tx=0).stop 2023-08-09T13:55:25.769 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:25.767+0000 7fa1c1c0f640 1 -- 172.21.15.43:0/3970767647 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa1bc0d1ec0 msgr2=0x7fa1bc1503d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:55:25.769 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:25.767+0000 7fa1c1c0f640 1 --2- 172.21.15.43:0/3970767647 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa1bc0d1ec0 0x7fa1bc1503d0 secure :-1 s=READY pgs=1670 cs=0 l=1 rev1=1 crypto rx=0x7fa1ac00c430 tx=0x7fa1ac00c8f0 comp rx=0 tx=0).stop 2023-08-09T13:55:25.770 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:25.767+0000 7fa1c1c0f640 1 -- 172.21.15.43:0/3970767647 shutdown_connections 2023-08-09T13:55:25.770 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:25.767+0000 7fa1c1c0f640 1 --2- 172.21.15.43:0/3970767647 >> [v2:172.21.15.203:6832/3490266242,v1:172.21.15.203:6833/3490266242] conn(0x7fa1a806a1a0 0x7fa1a806c660 unknown :-1 s=CLOSED pgs=52 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:55:25.770 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:25.767+0000 7fa1c1c0f640 1 --2- 172.21.15.43:0/3970767647 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fa1bc15f970 0x7fa1bc150910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:55:25.771 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:25.767+0000 7fa1c1c0f640 1 --2- 172.21.15.43:0/3970767647 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa1bc0d1ec0 0x7fa1bc1503d0 unknown :-1 s=CLOSED pgs=1670 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:55:25.771 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:25.767+0000 7fa1c1c0f640 1 --2- 172.21.15.43:0/3970767647 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fa1bc0cf710 0x7fa1bc15e8c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:55:25.771 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:25.767+0000 7fa1c1c0f640 1 -- 172.21.15.43:0/3970767647 >> 172.21.15.43:0/3970767647 conn(0x7fa1bc0712e0 msgr2=0x7fa1bc0716c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-09T13:55:25.771 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:25.767+0000 7fa1c1c0f640 1 -- 172.21.15.43:0/3970767647 shutdown_connections 2023-08-09T13:55:25.771 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:25.767+0000 7fa1c1c0f640 1 -- 172.21.15.43:0/3970767647 wait complete. 2023-08-09T13:55:25.788 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-08-09T13:55:26.017 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:26.015+0000 7fa2f1dfe640 1 Processor -- start 2023-08-09T13:55:26.017 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:26.015+0000 7fa2f1dfe640 1 -- start start 2023-08-09T13:55:26.019 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:26.016+0000 7fa2f1dfe640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7fa2ec157ea0 0x7fa2ec0c97d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:55:26.019 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:26.017+0000 7fa2f1dfe640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fa2ec0c90a0 0x7fa2ec0c9d10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:55:26.019 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:26.017+0000 7fa2f1dfe640 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7fa2ec15a800 0x7fa2ec15ac70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:55:26.019 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:26.017+0000 7fa2f1dfe640 1 -- --> v2:172.21.15.43:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa2ec15b840 con 0x7fa2ec157ea0 2023-08-09T13:55:26.020 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:26.017+0000 7fa2f1dfe640 1 -- --> v2:172.21.15.203:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa2ec0b3810 con 0x7fa2ec15a800 2023-08-09T13:55:26.020 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:26.017+0000 7fa2f1dfe640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa2ec158280 con 0x7fa2ec0c90a0 2023-08-09T13:55:26.021 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:26.017+0000 7fa2f0dfc640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7fa2ec157ea0 0x7fa2ec0c97d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:55:26.021 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:26.017+0000 7fa2f0dfc640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7fa2ec157ea0 0x7fa2ec0c97d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:52452/0 (socket says 172.21.15.43:52452) 2023-08-09T13:55:26.021 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:26.017+0000 7fa2f0dfc640 1 -- 172.21.15.43:0/4178290686 learned_addr learned my addr 172.21.15.43:0/4178290686 (peer_addr_for_me v2:172.21.15.43:0/0) 2023-08-09T13:55:26.021 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:26.017+0000 7fa2ebfff640 1 --2- 172.21.15.43:0/4178290686 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fa2ec0c90a0 0x7fa2ec0c9d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:55:26.021 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:26.017+0000 7fa2f15fd640 1 --2- 172.21.15.43:0/4178290686 >> v2:172.21.15.203:3300/0 conn(0x7fa2ec15a800 0x7fa2ec15ac70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:55:26.021 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:26.018+0000 7fa2f0dfc640 1 -- 172.21.15.43:0/4178290686 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fa2ec0c90a0 msgr2=0x7fa2ec0c9d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-09T13:55:26.021 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:26.018+0000 7fa2f0dfc640 1 --2- 172.21.15.43:0/4178290686 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fa2ec0c90a0 0x7fa2ec0c9d10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:55:26.022 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:26.018+0000 7fa2f0dfc640 1 -- 172.21.15.43:0/4178290686 >> v2:172.21.15.203:3300/0 conn(0x7fa2ec15a800 msgr2=0x7fa2ec15ac70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-09T13:55:26.022 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:26.018+0000 7fa2f0dfc640 1 --2- 172.21.15.43:0/4178290686 >> v2:172.21.15.203:3300/0 conn(0x7fa2ec15a800 0x7fa2ec15ac70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:55:26.022 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:26.018+0000 7fa2f0dfc640 1 -- 172.21.15.43:0/4178290686 --> v2:172.21.15.43:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa2ec15b560 con 0x7fa2ec157ea0 2023-08-09T13:55:26.022 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:26.018+0000 7fa2f15fd640 1 --2- 172.21.15.43:0/4178290686 >> v2:172.21.15.203:3300/0 conn(0x7fa2ec15a800 0x7fa2ec15ac70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-08-09T13:55:26.023 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:26.019+0000 7fa2f0dfc640 1 --2- 172.21.15.43:0/4178290686 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa2ec157ea0 0x7fa2ec0c97d0 secure :-1 s=READY pgs=1671 cs=0 l=1 rev1=1 crypto rx=0x7fa2ec0b09c0 tx=0x7fa2e002b860 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9667b34b0abb9a80 server_cookie=0 in_seq=0 out_seq=0 2023-08-09T13:55:26.023 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:26.019+0000 7fa2eb7fe640 1 -- 172.21.15.43:0/4178290686 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa2e00317c0 con 0x7fa2ec157ea0 2023-08-09T13:55:26.023 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:26.019+0000 7fa2eb7fe640 1 -- 172.21.15.43:0/4178290686 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa2e0031920 con 0x7fa2ec157ea0 2023-08-09T13:55:26.024 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:26.019+0000 7fa2eb7fe640 1 -- 172.21.15.43:0/4178290686 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa2e0031cb0 con 0x7fa2ec157ea0 2023-08-09T13:55:26.024 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:26.020+0000 7fa2f1dfe640 1 -- 172.21.15.43:0/4178290686 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa2ec157ea0 msgr2=0x7fa2ec0c97d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:55:26.024 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:26.020+0000 7fa2f1dfe640 1 --2- 172.21.15.43:0/4178290686 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa2ec157ea0 0x7fa2ec0c97d0 secure :-1 s=READY pgs=1671 cs=0 l=1 rev1=1 crypto rx=0x7fa2ec0b09c0 tx=0x7fa2e002b860 comp rx=0 tx=0).stop 2023-08-09T13:55:26.024 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:26.020+0000 7fa2f1dfe640 1 -- 172.21.15.43:0/4178290686 shutdown_connections 2023-08-09T13:55:26.024 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:26.020+0000 7fa2f1dfe640 1 --2- 172.21.15.43:0/4178290686 >> v2:172.21.15.203:3300/0 conn(0x7fa2ec15a800 0x7fa2ec15ac70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:55:26.024 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:26.020+0000 7fa2f1dfe640 1 --2- 172.21.15.43:0/4178290686 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fa2ec0c90a0 0x7fa2ec0c9d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:55:26.024 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:26.020+0000 7fa2f1dfe640 1 --2- 172.21.15.43:0/4178290686 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa2ec157ea0 0x7fa2ec0c97d0 unknown :-1 s=CLOSED pgs=1671 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:55:26.025 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:26.020+0000 7fa2f1dfe640 1 -- 172.21.15.43:0/4178290686 >> 172.21.15.43:0/4178290686 conn(0x7fa2ec0cddd0 msgr2=0x7fa2ec0ce1e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-09T13:55:26.025 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:26.020+0000 7fa2f1dfe640 1 -- 172.21.15.43:0/4178290686 shutdown_connections 2023-08-09T13:55:26.025 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:26.020+0000 7fa2f1dfe640 1 -- 172.21.15.43:0/4178290686 wait complete. 2023-08-09T13:55:26.026 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:26.022+0000 7fa2f1dfe640 1 Processor -- start 2023-08-09T13:55:26.026 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:26.022+0000 7fa2f1dfe640 1 -- start start 2023-08-09T13:55:26.026 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:26.023+0000 7fa2f1dfe640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa2ec0c90a0 0x7fa2ec0cd000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:55:26.026 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:26.023+0000 7fa2f1dfe640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fa2ec157ea0 0x7fa2ec1555e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:55:26.026 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:26.023+0000 7fa2f1dfe640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fa2ec15a800 0x7fa2ec14c580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:55:26.026 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:26.023+0000 7fa2f1dfe640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa2ec0c64f0 con 0x7fa2ec0c90a0 2023-08-09T13:55:26.026 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:26.023+0000 7fa2f1dfe640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa2ec154ac0 con 0x7fa2ec157ea0 2023-08-09T13:55:26.027 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:26.023+0000 7fa2f1dfe640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa2ec153f40 con 0x7fa2ec15a800 2023-08-09T13:55:26.027 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:26.023+0000 7fa2f0dfc640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa2ec0c90a0 0x7fa2ec0cd000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:55:26.027 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:26.024+0000 7fa2f0dfc640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa2ec0c90a0 0x7fa2ec0cd000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:52458/0 (socket says 172.21.15.43:52458) 2023-08-09T13:55:26.027 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:26.024+0000 7fa2f0dfc640 1 -- 172.21.15.43:0/1568305503 learned_addr learned my addr 172.21.15.43:0/1568305503 (peer_addr_for_me v2:172.21.15.43:0/0) 2023-08-09T13:55:26.028 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:26.024+0000 7fa2f15fd640 1 --2- 172.21.15.43:0/1568305503 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fa2ec15a800 0x7fa2ec14c580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:55:26.028 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:26.024+0000 7fa2ebfff640 1 --2- 172.21.15.43:0/1568305503 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fa2ec157ea0 0x7fa2ec1555e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:55:26.028 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:26.024+0000 7fa2f0dfc640 1 -- 172.21.15.43:0/1568305503 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fa2ec15a800 msgr2=0x7fa2ec14c580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:55:26.028 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:26.024+0000 7fa2f0dfc640 1 --2- 172.21.15.43:0/1568305503 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fa2ec15a800 0x7fa2ec14c580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:55:26.029 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:26.024+0000 7fa2f0dfc640 1 -- 172.21.15.43:0/1568305503 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fa2ec157ea0 msgr2=0x7fa2ec1555e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:55:26.029 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:26.024+0000 7fa2f0dfc640 1 --2- 172.21.15.43:0/1568305503 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fa2ec157ea0 0x7fa2ec1555e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:55:26.029 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:26.024+0000 7fa2f0dfc640 1 -- 172.21.15.43:0/1568305503 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa2e00094d0 con 0x7fa2ec0c90a0 2023-08-09T13:55:26.029 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:26.024+0000 7fa2f0dfc640 1 --2- 172.21.15.43:0/1568305503 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa2ec0c90a0 0x7fa2ec0cd000 secure :-1 s=READY pgs=1672 cs=0 l=1 rev1=1 crypto rx=0x7fa2e002b8a0 tx=0x7fa2e00046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-09T13:55:26.029 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:26.025+0000 7fa2e97fa640 1 -- 172.21.15.43:0/1568305503 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa2e0032860 con 0x7fa2ec0c90a0 2023-08-09T13:55:26.029 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:26.025+0000 7fa2e97fa640 1 -- 172.21.15.43:0/1568305503 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa2e00329e0 con 0x7fa2ec0c90a0 2023-08-09T13:55:26.029 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:26.025+0000 7fa2e97fa640 1 -- 172.21.15.43:0/1568305503 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa2e0032d90 con 0x7fa2ec0c90a0 2023-08-09T13:55:26.030 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:26.025+0000 7fa2f1dfe640 1 -- 172.21.15.43:0/1568305503 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa2ec154220 con 0x7fa2ec0c90a0 2023-08-09T13:55:26.030 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:26.025+0000 7fa2f1dfe640 1 -- 172.21.15.43:0/1568305503 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa2ec1547e0 con 0x7fa2ec0c90a0 2023-08-09T13:55:26.030 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:26.026+0000 7fa2f1dfe640 1 -- 172.21.15.43:0/1568305503 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa2ec0c6660 con 0x7fa2ec0c90a0 2023-08-09T13:55:26.030 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:26.027+0000 7fa2e97fa640 1 -- 172.21.15.43:0/1568305503 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 67) v1 ==== 88443+0+0 (secure 0 0 0) 0x7fa2e004e450 con 0x7fa2ec0c90a0 2023-08-09T13:55:26.031 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:26.028+0000 7fa2e97fa640 1 --2- 172.21.15.43:0/1568305503 >> [v2:172.21.15.203:6832/3490266242,v1:172.21.15.203:6833/3490266242] conn(0x7fa2d006a170 0x7fa2d006c630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:55:26.031 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:26.028+0000 7fa2e97fa640 1 -- 172.21.15.43:0/1568305503 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(143..143 src has 1..143) v4 ==== 5574+0+0 (secure 0 0 0) 0x7fa2e00aa780 con 0x7fa2ec0c90a0 2023-08-09T13:55:26.031 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:26.029+0000 7fa2ebfff640 1 --2- 172.21.15.43:0/1568305503 >> [v2:172.21.15.203:6832/3490266242,v1:172.21.15.203:6833/3490266242] conn(0x7fa2d006a170 0x7fa2d006c630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:55:26.032 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:26.030+0000 7fa2ebfff640 1 --2- 172.21.15.43:0/1568305503 >> [v2:172.21.15.203:6832/3490266242,v1:172.21.15.203:6833/3490266242] conn(0x7fa2d006a170 0x7fa2d006c630 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7fa2ec1f0930 tx=0x7fa2d8008040 comp rx=0 tx=0).ready entity=mgr.6659 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-09T13:55:26.034 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:26.032+0000 7fa2e97fa640 1 -- 172.21.15.43:0/1568305503 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178066 (secure 0 0 0) 0x7fa2e0075ea0 con 0x7fa2ec0c90a0 2023-08-09T13:55:26.230 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:26.227+0000 7fa2f1dfe640 1 -- 172.21.15.43:0/1568305503 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7fa2ec04e390 con 0x7fa2ec0c90a0 2023-08-09T13:55:26.230 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:26.228+0000 7fa2e97fa640 1 -- 172.21.15.43:0/1568305503 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v143) v1 ==== 93+0+10 (secure 0 0 0) 0x7fa2e00794c0 con 0x7fa2ec0c90a0 2023-08-09T13:55:26.230 INFO:teuthology.orchestra.run.smithi043.stdout:pg_num: 1 2023-08-09T13:55:26.233 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:26.231+0000 7fa2f1dfe640 1 -- 172.21.15.43:0/1568305503 >> [v2:172.21.15.203:6832/3490266242,v1:172.21.15.203:6833/3490266242] conn(0x7fa2d006a170 msgr2=0x7fa2d006c630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:55:26.233 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:26.231+0000 7fa2f1dfe640 1 --2- 172.21.15.43:0/1568305503 >> [v2:172.21.15.203:6832/3490266242,v1:172.21.15.203:6833/3490266242] conn(0x7fa2d006a170 0x7fa2d006c630 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7fa2ec1f0930 tx=0x7fa2d8008040 comp rx=0 tx=0).stop 2023-08-09T13:55:26.234 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:26.231+0000 7fa2f1dfe640 1 -- 172.21.15.43:0/1568305503 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa2ec0c90a0 msgr2=0x7fa2ec0cd000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:55:26.234 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:26.231+0000 7fa2f1dfe640 1 --2- 172.21.15.43:0/1568305503 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa2ec0c90a0 0x7fa2ec0cd000 secure :-1 s=READY pgs=1672 cs=0 l=1 rev1=1 crypto rx=0x7fa2e002b8a0 tx=0x7fa2e00046c0 comp rx=0 tx=0).stop 2023-08-09T13:55:26.234 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:26.231+0000 7fa2f1dfe640 1 -- 172.21.15.43:0/1568305503 shutdown_connections 2023-08-09T13:55:26.234 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:26.231+0000 7fa2f1dfe640 1 --2- 172.21.15.43:0/1568305503 >> [v2:172.21.15.203:6832/3490266242,v1:172.21.15.203:6833/3490266242] conn(0x7fa2d006a170 0x7fa2d006c630 unknown :-1 s=CLOSED pgs=53 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:55:26.234 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:26.231+0000 7fa2f1dfe640 1 --2- 172.21.15.43:0/1568305503 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fa2ec15a800 0x7fa2ec14c580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:55:26.234 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:26.231+0000 7fa2f1dfe640 1 --2- 172.21.15.43:0/1568305503 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fa2ec157ea0 0x7fa2ec1555e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:55:26.235 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:26.231+0000 7fa2f1dfe640 1 --2- 172.21.15.43:0/1568305503 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa2ec0c90a0 0x7fa2ec0cd000 unknown :-1 s=CLOSED pgs=1672 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:55:26.235 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:26.231+0000 7fa2f1dfe640 1 -- 172.21.15.43:0/1568305503 >> 172.21.15.43:0/1568305503 conn(0x7fa2ec0cddd0 msgr2=0x7fa2ec0c0cf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-09T13:55:26.235 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:26.231+0000 7fa2f1dfe640 1 -- 172.21.15.43:0/1568305503 shutdown_connections 2023-08-09T13:55:26.235 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:26.231+0000 7fa2f1dfe640 1 -- 172.21.15.43:0/1568305503 wait complete. 2023-08-09T13:55:26.253 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --format=json-pretty osd lspools 2023-08-09T13:55:26.480 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:26.478+0000 7fb5b7fff640 1 Processor -- start 2023-08-09T13:55:26.481 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:26.478+0000 7fb5b7fff640 1 -- start start 2023-08-09T13:55:26.482 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:26.479+0000 7fb5b7fff640 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7fb5b80b1050 0x7fb5b80b4760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:55:26.482 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:26.479+0000 7fb5b7fff640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7fb5b80cd6b0 0x7fb5b80d0e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:55:26.482 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:26.479+0000 7fb5b7fff640 1 -- --> v1:172.21.15.43:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb5b80b7bf0 con 0x7fb5b815f7b0 2023-08-09T13:55:26.482 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:26.479+0000 7fb5b7fff640 1 -- --> v2:172.21.15.43:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb5b80b7ed0 con 0x7fb5b80cd6b0 2023-08-09T13:55:26.482 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:26.479+0000 7fb5b7fff640 1 -- --> v2:172.21.15.203:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb5b80d0370 con 0x7fb5b80b1050 2023-08-09T13:55:26.482 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:26.479+0000 7fb5b77fe640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7fb5b80cd6b0 0x7fb5b80d0e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:55:26.482 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:26.479+0000 7fb5b77fe640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7fb5b80cd6b0 0x7fb5b80d0e50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:52468/0 (socket says 172.21.15.43:52468) 2023-08-09T13:55:26.483 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:26.479+0000 7fb5b77fe640 1 -- 172.21.15.43:0/1984962419 learned_addr learned my addr 172.21.15.43:0/1984962419 (peer_addr_for_me v2:172.21.15.43:0/0) 2023-08-09T13:55:26.483 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:26.479+0000 7fb5b6ffd640 1 --1- 172.21.15.43:0/1984962419 >> v1:172.21.15.43:6789/0 conn(0x7fb5b815f7b0 0x7fb5b80ccfa0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.43:6789/0 says I am v1:172.21.15.43:56576/0 (socket says 172.21.15.43:56576) 2023-08-09T13:55:26.483 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:26.480+0000 7fb5b77fe640 1 -- 172.21.15.43:0/1984962419 >> v2:172.21.15.203:3300/0 conn(0x7fb5b80b1050 msgr2=0x7fb5b80b4760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-09T13:55:26.484 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:26.480+0000 7fb5b77fe640 1 --2- 172.21.15.43:0/1984962419 >> v2:172.21.15.203:3300/0 conn(0x7fb5b80b1050 0x7fb5b80b4760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:55:26.484 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:26.480+0000 7fb5b77fe640 1 -- 172.21.15.43:0/1984962419 >> v1:172.21.15.43:6789/0 conn(0x7fb5b815f7b0 legacy=0x7fb5b80ccfa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:55:26.484 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:26.480+0000 7fb5b77fe640 1 -- 172.21.15.43:0/1984962419 --> v2:172.21.15.43:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb5b80d0650 con 0x7fb5b80cd6b0 2023-08-09T13:55:26.484 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:26.481+0000 7fb5b77fe640 1 --2- 172.21.15.43:0/1984962419 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb5b80cd6b0 0x7fb5b80d0e50 secure :-1 s=READY pgs=1674 cs=0 l=1 rev1=1 crypto rx=0x7fb5b80cfaf0 tx=0x7fb5b002b9b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=72ab512ad010354c server_cookie=0 in_seq=0 out_seq=0 2023-08-09T13:55:26.484 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:26.481+0000 7fb5b5ffb640 1 -- 172.21.15.43:0/1984962419 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb5b0030870 con 0x7fb5b80cd6b0 2023-08-09T13:55:26.484 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:26.481+0000 7fb5b5ffb640 1 -- 172.21.15.43:0/1984962419 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb5b00309d0 con 0x7fb5b80cd6b0 2023-08-09T13:55:26.485 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:26.481+0000 7fb5b5ffb640 1 -- 172.21.15.43:0/1984962419 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb5b0030d60 con 0x7fb5b80cd6b0 2023-08-09T13:55:26.485 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:26.482+0000 7fb5b7fff640 1 -- 172.21.15.43:0/1984962419 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb5b80cd6b0 msgr2=0x7fb5b80d0e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:55:26.485 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:26.482+0000 7fb5b7fff640 1 --2- 172.21.15.43:0/1984962419 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb5b80cd6b0 0x7fb5b80d0e50 secure :-1 s=READY pgs=1674 cs=0 l=1 rev1=1 crypto rx=0x7fb5b80cfaf0 tx=0x7fb5b002b9b0 comp rx=0 tx=0).stop 2023-08-09T13:55:26.486 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:26.482+0000 7fb5b7fff640 1 -- 172.21.15.43:0/1984962419 shutdown_connections 2023-08-09T13:55:26.486 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:26.482+0000 7fb5b7fff640 1 --2- 172.21.15.43:0/1984962419 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb5b80cd6b0 0x7fb5b80d0e50 unknown :-1 s=CLOSED pgs=1674 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:55:26.486 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:26.482+0000 7fb5b7fff640 1 --2- 172.21.15.43:0/1984962419 >> v2:172.21.15.203:3300/0 conn(0x7fb5b80b1050 0x7fb5b80b4760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:55:26.486 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:26.482+0000 7fb5b7fff640 1 -- 172.21.15.43:0/1984962419 >> 172.21.15.43:0/1984962419 conn(0x7fb5b806a2b0 msgr2=0x7fb5b806a690 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-09T13:55:26.486 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:26.482+0000 7fb5b7fff640 1 -- 172.21.15.43:0/1984962419 shutdown_connections 2023-08-09T13:55:26.486 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:26.482+0000 7fb5b7fff640 1 -- 172.21.15.43:0/1984962419 wait complete. 2023-08-09T13:55:26.487 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:26.484+0000 7fb5b7fff640 1 Processor -- start 2023-08-09T13:55:26.487 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:26.484+0000 7fb5b7fff640 1 -- start start 2023-08-09T13:55:26.488 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:26.485+0000 7fb5b7fff640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fb5b80b1050 0x7fb5b80c6810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:55:26.488 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:26.485+0000 7fb5b7fff640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fb5b80cd6b0 0x7fb5b80c87b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:55:26.488 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:26.485+0000 7fb5b7fff640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb5b815f7b0 0x7fb5b8155710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:55:26.488 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:26.485+0000 7fb5b7fff640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb5b80c8cf0 con 0x7fb5b815f7b0 2023-08-09T13:55:26.488 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:26.485+0000 7fb5b7fff640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb5b80c6ea0 con 0x7fb5b80b1050 2023-08-09T13:55:26.488 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:26.485+0000 7fb5b7fff640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb5b80c7180 con 0x7fb5b80cd6b0 2023-08-09T13:55:26.489 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:26.485+0000 7fb5b77fe640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb5b815f7b0 0x7fb5b8155710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:55:26.489 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:26.485+0000 7fb5b77fe640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb5b815f7b0 0x7fb5b8155710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:52482/0 (socket says 172.21.15.43:52482) 2023-08-09T13:55:26.489 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:26.485+0000 7fb5b77fe640 1 -- 172.21.15.43:0/3819581980 learned_addr learned my addr 172.21.15.43:0/3819581980 (peer_addr_for_me v2:172.21.15.43:0/0) 2023-08-09T13:55:26.490 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:26.486+0000 7fb5b6ffd640 1 --2- 172.21.15.43:0/3819581980 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fb5b80b1050 0x7fb5b80c6810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:55:26.490 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:26.486+0000 7fb5b77fe640 1 -- 172.21.15.43:0/3819581980 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fb5b80cd6b0 msgr2=0x7fb5b80c87b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:55:26.490 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:26.486+0000 7fb5b67fc640 1 --2- 172.21.15.43:0/3819581980 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fb5b80cd6b0 0x7fb5b80c87b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:55:26.490 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:26.486+0000 7fb5b77fe640 1 --2- 172.21.15.43:0/3819581980 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fb5b80cd6b0 0x7fb5b80c87b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:55:26.491 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:26.486+0000 7fb5b77fe640 1 -- 172.21.15.43:0/3819581980 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fb5b80b1050 msgr2=0x7fb5b80c6810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:55:26.491 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:26.486+0000 7fb5b77fe640 1 --2- 172.21.15.43:0/3819581980 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fb5b80b1050 0x7fb5b80c6810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:55:26.491 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:26.486+0000 7fb5b77fe640 1 -- 172.21.15.43:0/3819581980 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb5b00094d0 con 0x7fb5b815f7b0 2023-08-09T13:55:26.491 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:26.486+0000 7fb5b6ffd640 1 --2- 172.21.15.43:0/3819581980 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fb5b80b1050 0x7fb5b80c6810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-08-09T13:55:26.491 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:26.486+0000 7fb5b67fc640 1 --2- 172.21.15.43:0/3819581980 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fb5b80cd6b0 0x7fb5b80c87b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-08-09T13:55:26.491 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:26.486+0000 7fb5b77fe640 1 --2- 172.21.15.43:0/3819581980 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb5b815f7b0 0x7fb5b8155710 secure :-1 s=READY pgs=1675 cs=0 l=1 rev1=1 crypto rx=0x7fb5b002b9f0 tx=0x7fb5b0004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-09T13:55:26.491 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:26.487+0000 7fb59f7fe640 1 -- 172.21.15.43:0/3819581980 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb5b0043400 con 0x7fb5b815f7b0 2023-08-09T13:55:26.492 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:26.487+0000 7fb59f7fe640 1 -- 172.21.15.43:0/3819581980 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb5b0043580 con 0x7fb5b815f7b0 2023-08-09T13:55:26.492 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:26.487+0000 7fb5b7fff640 1 -- 172.21.15.43:0/3819581980 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb5b80c7460 con 0x7fb5b815f7b0 2023-08-09T13:55:26.492 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:26.487+0000 7fb59f7fe640 1 -- 172.21.15.43:0/3819581980 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb5b0043930 con 0x7fb5b815f7b0 2023-08-09T13:55:26.492 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:26.487+0000 7fb5b7fff640 1 -- 172.21.15.43:0/3819581980 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb5b80c9f00 con 0x7fb5b815f7b0 2023-08-09T13:55:26.493 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:26.488+0000 7fb5b7fff640 1 -- 172.21.15.43:0/3819581980 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb5b80ca160 con 0x7fb5b815f7b0 2023-08-09T13:55:26.493 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:26.489+0000 7fb59f7fe640 1 -- 172.21.15.43:0/3819581980 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 67) v1 ==== 88443+0+0 (secure 0 0 0) 0x7fb5b0043a90 con 0x7fb5b815f7b0 2023-08-09T13:55:26.493 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:26.490+0000 7fb59f7fe640 1 --2- 172.21.15.43:0/3819581980 >> [v2:172.21.15.203:6832/3490266242,v1:172.21.15.203:6833/3490266242] conn(0x7fb5a406a1a0 0x7fb5a406c660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:55:26.493 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:26.490+0000 7fb59f7fe640 1 -- 172.21.15.43:0/3819581980 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(143..143 src has 1..143) v4 ==== 5574+0+0 (secure 0 0 0) 0x7fb5b003e070 con 0x7fb5b815f7b0 2023-08-09T13:55:26.493 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:26.491+0000 7fb5b6ffd640 1 --2- 172.21.15.43:0/3819581980 >> [v2:172.21.15.203:6832/3490266242,v1:172.21.15.203:6833/3490266242] conn(0x7fb5a406a1a0 0x7fb5a406c660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:55:26.494 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:26.492+0000 7fb5b6ffd640 1 --2- 172.21.15.43:0/3819581980 >> [v2:172.21.15.203:6832/3490266242,v1:172.21.15.203:6833/3490266242] conn(0x7fb5a406a1a0 0x7fb5a406c660 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7fb5a8007fd0 tx=0x7fb5a8009040 comp rx=0 tx=0).ready entity=mgr.6659 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-09T13:55:26.497 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:26.494+0000 7fb59f7fe640 1 -- 172.21.15.43:0/3819581980 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178066 (secure 0 0 0) 0x7fb5b80ca160 con 0x7fb5b815f7b0 2023-08-09T13:55:26.686 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:26.684+0000 7fb5b7fff640 1 -- 172.21.15.43:0/3819581980 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) v1 -- 0x7fb5b80ca160 con 0x7fb5b815f7b0 2023-08-09T13:55:26.687 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:26.685+0000 7fb59f7fe640 1 -- 172.21.15.43:0/3819581980 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v143) v1 ==== 84+0+126 (secure 0 0 0) 0x7fb5b0034090 con 0x7fb5b815f7b0 2023-08-09T13:55:26.688 INFO:teuthology.orchestra.run.smithi043.stdout: 2023-08-09T13:55:26.688 INFO:teuthology.orchestra.run.smithi043.stdout:[ 2023-08-09T13:55:26.689 INFO:teuthology.orchestra.run.smithi043.stdout: { 2023-08-09T13:55:26.689 INFO:teuthology.orchestra.run.smithi043.stdout: "poolnum": 1, 2023-08-09T13:55:26.689 INFO:teuthology.orchestra.run.smithi043.stdout: "poolname": "rbd" 2023-08-09T13:55:26.689 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2023-08-09T13:55:26.690 INFO:teuthology.orchestra.run.smithi043.stdout: { 2023-08-09T13:55:26.690 INFO:teuthology.orchestra.run.smithi043.stdout: "poolnum": 2, 2023-08-09T13:55:26.690 INFO:teuthology.orchestra.run.smithi043.stdout: "poolname": ".mgr" 2023-08-09T13:55:26.691 INFO:teuthology.orchestra.run.smithi043.stdout: } 2023-08-09T13:55:26.691 INFO:teuthology.orchestra.run.smithi043.stdout:] 2023-08-09T13:55:26.693 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:26.687+0000 7fb5b7fff640 1 -- 172.21.15.43:0/3819581980 >> [v2:172.21.15.203:6832/3490266242,v1:172.21.15.203:6833/3490266242] conn(0x7fb5a406a1a0 msgr2=0x7fb5a406c660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:55:26.693 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:26.687+0000 7fb5b7fff640 1 --2- 172.21.15.43:0/3819581980 >> [v2:172.21.15.203:6832/3490266242,v1:172.21.15.203:6833/3490266242] conn(0x7fb5a406a1a0 0x7fb5a406c660 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7fb5a8007fd0 tx=0x7fb5a8009040 comp rx=0 tx=0).stop 2023-08-09T13:55:26.694 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:26.687+0000 7fb5b7fff640 1 -- 172.21.15.43:0/3819581980 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb5b815f7b0 msgr2=0x7fb5b8155710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:55:26.694 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:26.687+0000 7fb5b7fff640 1 --2- 172.21.15.43:0/3819581980 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb5b815f7b0 0x7fb5b8155710 secure :-1 s=READY pgs=1675 cs=0 l=1 rev1=1 crypto rx=0x7fb5b002b9f0 tx=0x7fb5b0004730 comp rx=0 tx=0).stop 2023-08-09T13:55:26.694 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:26.688+0000 7fb5b7fff640 1 -- 172.21.15.43:0/3819581980 shutdown_connections 2023-08-09T13:55:26.694 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:26.688+0000 7fb5b7fff640 1 --2- 172.21.15.43:0/3819581980 >> [v2:172.21.15.203:6832/3490266242,v1:172.21.15.203:6833/3490266242] conn(0x7fb5a406a1a0 0x7fb5a406c660 unknown :-1 s=CLOSED pgs=54 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:55:26.695 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:26.688+0000 7fb5b7fff640 1 --2- 172.21.15.43:0/3819581980 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb5b815f7b0 0x7fb5b8155710 unknown :-1 s=CLOSED pgs=1675 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:55:26.695 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:26.688+0000 7fb5b7fff640 1 --2- 172.21.15.43:0/3819581980 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fb5b80cd6b0 0x7fb5b80c87b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:55:26.695 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:26.688+0000 7fb5b7fff640 1 --2- 172.21.15.43:0/3819581980 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fb5b80b1050 0x7fb5b80c6810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:55:26.695 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:26.688+0000 7fb5b7fff640 1 -- 172.21.15.43:0/3819581980 >> 172.21.15.43:0/3819581980 conn(0x7fb5b806a2b0 msgr2=0x7fb5b815ab80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-09T13:55:26.696 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:26.688+0000 7fb5b7fff640 1 -- 172.21.15.43:0/3819581980 shutdown_connections 2023-08-09T13:55:26.696 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:26.688+0000 7fb5b7fff640 1 -- 172.21.15.43:0/3819581980 wait complete. 2023-08-09T13:55:26.709 DEBUG:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2023-08-09T13:55:26.710 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_metadata --pg_num_min 64 2023-08-09T13:55:26.942 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:26.939+0000 7f8b935d6640 1 Processor -- start 2023-08-09T13:55:26.943 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:26.939+0000 7f8b935d6640 1 -- start start 2023-08-09T13:55:26.945 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:26.941+0000 7f8b935d6640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f8b8c0ca370 0x7f8b8c15b2a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:55:26.945 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:26.941+0000 7f8b935d6640 1 -- --> v1:172.21.15.43:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8b8c15ae10 con 0x7f8b8c0ce4c0 2023-08-09T13:55:26.945 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:26.941+0000 7f8b935d6640 1 -- --> v1:172.21.15.203:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8b8c154e70 con 0x7f8b8c06fb70 2023-08-09T13:55:26.945 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:26.941+0000 7f8b935d6640 1 -- --> v2:172.21.15.43:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8b8c06ff50 con 0x7f8b8c0ca370 2023-08-09T13:55:26.946 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:26.941+0000 7f8b90b71640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f8b8c0ca370 0x7f8b8c15b2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:55:26.946 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:26.941+0000 7f8b90b71640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f8b8c0ca370 0x7f8b8c15b2a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:52488/0 (socket says 172.21.15.43:52488) 2023-08-09T13:55:26.946 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:26.941+0000 7f8b90b71640 1 -- 172.21.15.43:0/704183500 learned_addr learned my addr 172.21.15.43:0/704183500 (peer_addr_for_me v2:172.21.15.43:0/0) 2023-08-09T13:55:26.946 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:26.941+0000 7f8b91b73640 1 --1- 172.21.15.43:0/704183500 >> v1:172.21.15.43:6789/0 conn(0x7f8b8c0ce4c0 0x7f8b8c0c86f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.43:6789/0 says I am v1:172.21.15.43:56592/0 (socket says 172.21.15.43:56592) 2023-08-09T13:55:26.947 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:26.942+0000 7f8b90b71640 1 -- 172.21.15.43:0/704183500 >> v1:172.21.15.203:6789/0 conn(0x7f8b8c06fb70 legacy=0x7f8b8c0caab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:55:26.947 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:26.942+0000 7f8b90b71640 1 -- 172.21.15.43:0/704183500 >> v1:172.21.15.43:6789/0 conn(0x7f8b8c0ce4c0 legacy=0x7f8b8c0c86f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:55:26.947 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:26.942+0000 7f8b90b71640 1 -- 172.21.15.43:0/704183500 --> v2:172.21.15.43:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8b8c0cb320 con 0x7f8b8c0ca370 2023-08-09T13:55:26.947 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:26.942+0000 7f8b83fff640 1 -- 172.21.15.43:0/704183500 <== mon.0 v1:172.21.15.43:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 209366457 0 0) 0x7f8b8c15ae10 con 0x7f8b8c0ce4c0 2023-08-09T13:55:26.951 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:26.943+0000 7f8b90b71640 1 --2- 172.21.15.43:0/704183500 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f8b8c0ca370 0x7f8b8c15b2a0 secure :-1 s=READY pgs=1677 cs=0 l=1 rev1=1 crypto rx=0x7f8b7c009870 tx=0x7f8b7c02ba30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e7def4cf1dbc70d0 server_cookie=0 in_seq=0 out_seq=0 2023-08-09T13:55:26.952 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:26.943+0000 7f8b83fff640 1 -- 172.21.15.43:0/704183500 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8b7c02f850 con 0x7f8b8c0ca370 2023-08-09T13:55:26.952 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:26.943+0000 7f8b83fff640 1 -- 172.21.15.43:0/704183500 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8b7c02f9b0 con 0x7f8b8c0ca370 2023-08-09T13:55:26.952 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:26.943+0000 7f8b83fff640 1 -- 172.21.15.43:0/704183500 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8b7c02fd40 con 0x7f8b8c0ca370 2023-08-09T13:55:26.953 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:26.944+0000 7f8b935d6640 1 -- 172.21.15.43:0/704183500 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f8b8c0ca370 msgr2=0x7f8b8c15b2a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:55:26.953 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:26.944+0000 7f8b935d6640 1 --2- 172.21.15.43:0/704183500 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f8b8c0ca370 0x7f8b8c15b2a0 secure :-1 s=READY pgs=1677 cs=0 l=1 rev1=1 crypto rx=0x7f8b7c009870 tx=0x7f8b7c02ba30 comp rx=0 tx=0).stop 2023-08-09T13:55:26.953 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:26.944+0000 7f8b935d6640 1 -- 172.21.15.43:0/704183500 shutdown_connections 2023-08-09T13:55:26.953 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:26.944+0000 7f8b935d6640 1 --2- 172.21.15.43:0/704183500 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f8b8c0ca370 0x7f8b8c15b2a0 unknown :-1 s=CLOSED pgs=1677 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:55:26.954 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:26.944+0000 7f8b935d6640 1 -- 172.21.15.43:0/704183500 >> 172.21.15.43:0/704183500 conn(0x7f8b8c06b990 msgr2=0x7f8b8c06bda0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-09T13:55:26.954 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:26.944+0000 7f8b935d6640 1 -- 172.21.15.43:0/704183500 shutdown_connections 2023-08-09T13:55:26.954 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:26.944+0000 7f8b935d6640 1 -- 172.21.15.43:0/704183500 wait complete. 2023-08-09T13:55:26.954 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:26.946+0000 7f8b935d6640 1 Processor -- start 2023-08-09T13:55:26.955 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:26.946+0000 7f8b935d6640 1 -- start start 2023-08-09T13:55:26.955 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:26.947+0000 7f8b935d6640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f8b8c06fb70 0x7f8b8c153c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:55:26.955 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:26.947+0000 7f8b935d6640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f8b8c0ca370 0x7f8b8c0b3370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:55:26.955 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:26.947+0000 7f8b935d6640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f8b8c0ce4c0 0x7f8b8c0b38b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:55:26.956 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:26.947+0000 7f8b935d6640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8b8c0c5170 con 0x7f8b8c0ca370 2023-08-09T13:55:26.956 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:26.947+0000 7f8b935d6640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8b8c0c5cf0 con 0x7f8b8c06fb70 2023-08-09T13:55:26.956 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:26.947+0000 7f8b935d6640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8b8c0c53c0 con 0x7f8b8c0ce4c0 2023-08-09T13:55:26.956 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:26.947+0000 7f8b90b71640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f8b8c0ca370 0x7f8b8c0b3370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:55:26.957 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:26.947+0000 7f8b90b71640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f8b8c0ca370 0x7f8b8c0b3370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:52490/0 (socket says 172.21.15.43:52490) 2023-08-09T13:55:26.957 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:26.947+0000 7f8b90b71640 1 -- 172.21.15.43:0/2341331926 learned_addr learned my addr 172.21.15.43:0/2341331926 (peer_addr_for_me v2:172.21.15.43:0/0) 2023-08-09T13:55:26.957 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:26.948+0000 7f8b91b73640 1 --2- 172.21.15.43:0/2341331926 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f8b8c0ce4c0 0x7f8b8c0b38b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:55:26.957 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:26.948+0000 7f8b91372640 1 --2- 172.21.15.43:0/2341331926 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f8b8c06fb70 0x7f8b8c153c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:55:26.958 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:26.948+0000 7f8b90b71640 1 -- 172.21.15.43:0/2341331926 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f8b8c0ce4c0 msgr2=0x7f8b8c0b38b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:55:26.958 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:26.948+0000 7f8b90b71640 1 --2- 172.21.15.43:0/2341331926 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f8b8c0ce4c0 0x7f8b8c0b38b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:55:26.958 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:26.948+0000 7f8b90b71640 1 -- 172.21.15.43:0/2341331926 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f8b8c06fb70 msgr2=0x7f8b8c153c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:55:26.958 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:26.948+0000 7f8b90b71640 1 --2- 172.21.15.43:0/2341331926 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f8b8c06fb70 0x7f8b8c153c70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:55:26.959 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:26.948+0000 7f8b90b71640 1 -- 172.21.15.43:0/2341331926 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8b7c0094d0 con 0x7f8b8c0ca370 2023-08-09T13:55:26.959 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:26.948+0000 7f8b90b71640 1 --2- 172.21.15.43:0/2341331926 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f8b8c0ca370 0x7f8b8c0b3370 secure :-1 s=READY pgs=1678 cs=0 l=1 rev1=1 crypto rx=0x7f8b7c000c00 tx=0x7f8b7c004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-09T13:55:26.959 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:26.949+0000 7f8b81ffb640 1 -- 172.21.15.43:0/2341331926 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8b7c02f800 con 0x7f8b8c0ca370 2023-08-09T13:55:26.960 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:26.949+0000 7f8b81ffb640 1 -- 172.21.15.43:0/2341331926 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8b7c041400 con 0x7f8b8c0ca370 2023-08-09T13:55:26.960 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:26.949+0000 7f8b935d6640 1 -- 172.21.15.43:0/2341331926 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8b8c0c56a0 con 0x7f8b8c0ca370 2023-08-09T13:55:26.960 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:26.949+0000 7f8b81ffb640 1 -- 172.21.15.43:0/2341331926 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8b7c0417b0 con 0x7f8b8c0ca370 2023-08-09T13:55:26.960 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:26.949+0000 7f8b935d6640 1 -- 172.21.15.43:0/2341331926 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8b8c1541b0 con 0x7f8b8c0ca370 2023-08-09T13:55:26.962 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:26.950+0000 7f8b935d6640 1 -- 172.21.15.43:0/2341331926 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8b8c0739f0 con 0x7f8b8c0ca370 2023-08-09T13:55:26.962 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:26.951+0000 7f8b81ffb640 1 -- 172.21.15.43:0/2341331926 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 67) v1 ==== 88443+0+0 (secure 0 0 0) 0x7f8b7c041910 con 0x7f8b8c0ca370 2023-08-09T13:55:26.963 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:26.952+0000 7f8b81ffb640 1 --2- 172.21.15.43:0/2341331926 >> [v2:172.21.15.203:6832/3490266242,v1:172.21.15.203:6833/3490266242] conn(0x7f8b7006a170 0x7f8b7006c630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:55:26.963 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:26.952+0000 7f8b81ffb640 1 -- 172.21.15.43:0/2341331926 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(143..143 src has 1..143) v4 ==== 5574+0+0 (secure 0 0 0) 0x7f8b7c03c070 con 0x7f8b8c0ca370 2023-08-09T13:55:26.963 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:26.953+0000 7f8b91372640 1 --2- 172.21.15.43:0/2341331926 >> [v2:172.21.15.203:6832/3490266242,v1:172.21.15.203:6833/3490266242] conn(0x7f8b7006a170 0x7f8b7006c630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:55:26.963 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:26.954+0000 7f8b91372640 1 --2- 172.21.15.43:0/2341331926 >> [v2:172.21.15.203:6832/3490266242,v1:172.21.15.203:6833/3490266242] conn(0x7f8b7006a170 0x7f8b7006c630 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f8b88000f60 tx=0x7f8b88009040 comp rx=0 tx=0).ready entity=mgr.6659 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-09T13:55:26.964 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:26.956+0000 7f8b81ffb640 1 -- 172.21.15.43:0/2341331926 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178066 (secure 0 0 0) 0x7f8b7c076cd0 con 0x7f8b8c0ca370 2023-08-09T13:55:27.150 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:27.148+0000 7f8b935d6640 1 -- 172.21.15.43:0/2341331926 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64} v 0) v1 -- 0x7f8b8c06a360 con 0x7f8b8c0ca370 2023-08-09T13:55:27.287 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:27.284+0000 7f8b81ffb640 1 -- 172.21.15.43:0/2341331926 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]=0 pool 'cephfs_metadata' created v144) v1 ==== 138+0+0 (secure 0 0 0) 0x7f8b7c07a2f0 con 0x7f8b8c0ca370 2023-08-09T13:55:27.288 INFO:teuthology.orchestra.run.smithi043.stderr:pool 'cephfs_metadata' created 2023-08-09T13:55:27.293 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:27.290+0000 7f8b935d6640 1 -- 172.21.15.43:0/2341331926 >> [v2:172.21.15.203:6832/3490266242,v1:172.21.15.203:6833/3490266242] conn(0x7f8b7006a170 msgr2=0x7f8b7006c630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:55:27.293 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:27.290+0000 7f8b935d6640 1 --2- 172.21.15.43:0/2341331926 >> [v2:172.21.15.203:6832/3490266242,v1:172.21.15.203:6833/3490266242] conn(0x7f8b7006a170 0x7f8b7006c630 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f8b88000f60 tx=0x7f8b88009040 comp rx=0 tx=0).stop 2023-08-09T13:55:27.293 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:27.290+0000 7f8b935d6640 1 -- 172.21.15.43:0/2341331926 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f8b8c0ca370 msgr2=0x7f8b8c0b3370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:55:27.293 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:27.290+0000 7f8b935d6640 1 --2- 172.21.15.43:0/2341331926 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f8b8c0ca370 0x7f8b8c0b3370 secure :-1 s=READY pgs=1678 cs=0 l=1 rev1=1 crypto rx=0x7f8b7c000c00 tx=0x7f8b7c004730 comp rx=0 tx=0).stop 2023-08-09T13:55:27.293 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:27.290+0000 7f8b935d6640 1 -- 172.21.15.43:0/2341331926 shutdown_connections 2023-08-09T13:55:27.293 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:27.290+0000 7f8b935d6640 1 --2- 172.21.15.43:0/2341331926 >> [v2:172.21.15.203:6832/3490266242,v1:172.21.15.203:6833/3490266242] conn(0x7f8b7006a170 0x7f8b7006c630 unknown :-1 s=CLOSED pgs=55 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:55:27.294 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:27.290+0000 7f8b935d6640 1 --2- 172.21.15.43:0/2341331926 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f8b8c0ce4c0 0x7f8b8c0b38b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:55:27.294 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:27.290+0000 7f8b935d6640 1 --2- 172.21.15.43:0/2341331926 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f8b8c0ca370 0x7f8b8c0b3370 unknown :-1 s=CLOSED pgs=1678 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:55:27.294 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:27.290+0000 7f8b935d6640 1 --2- 172.21.15.43:0/2341331926 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f8b8c06fb70 0x7f8b8c153c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:55:27.294 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:27.290+0000 7f8b935d6640 1 -- 172.21.15.43:0/2341331926 >> 172.21.15.43:0/2341331926 conn(0x7f8b8c06b990 msgr2=0x7f8b8c06bd70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-09T13:55:27.294 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:27.290+0000 7f8b935d6640 1 -- 172.21.15.43:0/2341331926 shutdown_connections 2023-08-09T13:55:27.295 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:27.291+0000 7f8b935d6640 1 -- 172.21.15.43:0/2341331926 wait complete. 2023-08-09T13:55:27.319 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data 64 --pg_num_min 64 --target_size_ratio 0.9 2023-08-09T13:55:27.550 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:27.547+0000 7f3b0cde1640 1 Processor -- start 2023-08-09T13:55:27.550 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:27.547+0000 7f3b0cde1640 1 -- start start 2023-08-09T13:55:27.552 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:27.549+0000 7f3b0cde1640 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7f3b08143310 0x7f3b0806fce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:55:27.552 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:27.549+0000 7f3b0cde1640 1 -- --> v1:172.21.15.43:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3b0806b580 con 0x7f3b080cde00 2023-08-09T13:55:27.552 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:27.549+0000 7f3b0cde1640 1 -- --> v1:172.21.15.203:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3b0806a970 con 0x7f3b0815c830 2023-08-09T13:55:27.552 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:27.549+0000 7f3b0cde1640 1 -- --> v2:172.21.15.203:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3b0806ac50 con 0x7f3b08143310 2023-08-09T13:55:27.553 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:27.550+0000 7f3b0659c640 1 --1- >> v1:172.21.15.43:6789/0 conn(0x7f3b080cde00 0x7f3b080ce1e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.43:6789/0 says I am v1:172.21.15.43:56606/0 (socket says 172.21.15.43:56606) 2023-08-09T13:55:27.554 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:27.550+0000 7f3b0659c640 1 -- 172.21.15.43:0/1726887982 learned_addr learned my addr 172.21.15.43:0/1726887982 (peer_addr_for_me v1:172.21.15.43:0/0) 2023-08-09T13:55:27.554 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:27.550+0000 7f3b06d9d640 1 --2- 172.21.15.43:0/1726887982 >> v2:172.21.15.203:3300/0 conn(0x7f3b08143310 0x7f3b0806fce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:55:27.554 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:27.550+0000 7f3b0559a640 1 -- 172.21.15.43:0/1726887982 <== mon.0 v1:172.21.15.43:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1922328736 0 0) 0x7f3b0806b580 con 0x7f3b080cde00 2023-08-09T13:55:27.554 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:27.550+0000 7f3b0559a640 1 -- 172.21.15.43:0/1726887982 --> v1:172.21.15.43:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f3af0003520 con 0x7f3b080cde00 2023-08-09T13:55:27.554 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:27.551+0000 7f3b0559a640 1 -- 172.21.15.43:0/1726887982 <== mon.0 v1:172.21.15.43:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1193952385 0 0) 0x7f3af0003520 con 0x7f3b080cde00 2023-08-09T13:55:27.554 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:27.551+0000 7f3b0559a640 1 -- 172.21.15.43:0/1726887982 >> v2:172.21.15.203:3300/0 conn(0x7f3b08143310 msgr2=0x7f3b0806fce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-09T13:55:27.554 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:27.551+0000 7f3b0559a640 1 --2- 172.21.15.43:0/1726887982 >> v2:172.21.15.203:3300/0 conn(0x7f3b08143310 0x7f3b0806fce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:55:27.555 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:27.551+0000 7f3b0559a640 1 -- 172.21.15.43:0/1726887982 >> v1:172.21.15.203:6789/0 conn(0x7f3b0815c830 legacy=0x7f3b0806ba10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:55:27.555 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:27.551+0000 7f3b0559a640 1 -- 172.21.15.43:0/1726887982 --> v1:172.21.15.43:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3b0806af30 con 0x7f3b080cde00 2023-08-09T13:55:27.555 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:27.551+0000 7f3b0559a640 1 -- 172.21.15.43:0/1726887982 <== mon.0 v1:172.21.15.43:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1285150740 0 0) 0x7f3af8001670 con 0x7f3b080cde00 2023-08-09T13:55:27.555 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:27.551+0000 7f3b0559a640 1 -- 172.21.15.43:0/1726887982 >> v1:172.21.15.43:6789/0 conn(0x7f3b080cde00 legacy=0x7f3b080ce1e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:55:27.555 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:27.551+0000 7f3b0559a640 1 --2- 172.21.15.43:0/1726887982 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f3af0003970 0x7f3af0003d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:55:27.555 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:27.551+0000 7f3b0559a640 1 --2- 172.21.15.43:0/1726887982 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f3af00047c0 0x7f3af0004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:55:27.555 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:27.551+0000 7f3b0559a640 1 --2- 172.21.15.43:0/1726887982 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f3af0005180 0x7f3af000b980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:55:27.556 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:27.551+0000 7f3b0559a640 1 -- 172.21.15.43:0/1726887982 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3b0806ac50 con 0x7f3af0003970 2023-08-09T13:55:27.556 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:27.551+0000 7f3b0559a640 1 -- 172.21.15.43:0/1726887982 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3af000c020 con 0x7f3af00047c0 2023-08-09T13:55:27.556 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:27.551+0000 7f3b0559a640 1 -- 172.21.15.43:0/1726887982 --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3af000c1e0 con 0x7f3af0005180 2023-08-09T13:55:27.556 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:27.551+0000 7f3b0659c640 1 --2- 172.21.15.43:0/1726887982 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f3af0003970 0x7f3af0003d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:55:27.557 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:27.552+0000 7f3b05d9b640 1 --2- 172.21.15.43:0/1726887982 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f3af00047c0 0x7f3af0004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:55:27.557 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:27.552+0000 7f3b06d9d640 1 --2- 172.21.15.43:0/1726887982 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f3af0005180 0x7f3af000b980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:55:27.558 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:27.552+0000 7f3b0659c640 1 -- 172.21.15.43:0/1726887982 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f3af0005180 msgr2=0x7f3af000b980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-09T13:55:27.558 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:27.552+0000 7f3b0659c640 1 --2- 172.21.15.43:0/1726887982 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f3af0005180 0x7f3af000b980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:55:27.558 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:27.552+0000 7f3b0659c640 1 -- 172.21.15.43:0/1726887982 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f3af00047c0 msgr2=0x7f3af0004c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-09T13:55:27.558 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:27.552+0000 7f3b0659c640 1 --2- 172.21.15.43:0/1726887982 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f3af00047c0 0x7f3af0004c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:55:27.558 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:27.552+0000 7f3b0659c640 1 -- 172.21.15.43:0/1726887982 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3af0003520 con 0x7f3af0003970 2023-08-09T13:55:27.558 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:27.552+0000 7f3b06d9d640 1 -- 172.21.15.43:0/1726887982 reap_dead start 2023-08-09T13:55:27.558 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:27.553+0000 7f3b0659c640 1 --2- 172.21.15.43:0/1726887982 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f3af0003970 0x7f3af0003d90 secure :-1 s=READY pgs=1680 cs=0 l=1 rev1=1 crypto rx=0x7f3af800c240 tx=0x7f3af00057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f534f2a0797a118d server_cookie=0 in_seq=0 out_seq=0 2023-08-09T13:55:27.558 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:27.553+0000 7f3b0559a640 1 -- 172.21.15.43:0/1726887982 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3af8005ad0 con 0x7f3af0003970 2023-08-09T13:55:27.559 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:27.553+0000 7f3b0559a640 1 -- 172.21.15.43:0/1726887982 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3af8005c30 con 0x7f3af0003970 2023-08-09T13:55:27.559 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:27.553+0000 7f3b0559a640 1 -- 172.21.15.43:0/1726887982 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3af802ca00 con 0x7f3af0003970 2023-08-09T13:55:27.559 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:27.554+0000 7f3b0cde1640 1 -- 172.21.15.43:0/1726887982 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f3af0003970 msgr2=0x7f3af0003d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:55:27.559 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:27.554+0000 7f3b0cde1640 1 --2- 172.21.15.43:0/1726887982 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f3af0003970 0x7f3af0003d90 secure :-1 s=READY pgs=1680 cs=0 l=1 rev1=1 crypto rx=0x7f3af800c240 tx=0x7f3af00057f0 comp rx=0 tx=0).stop 2023-08-09T13:55:27.559 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:27.554+0000 7f3b0cde1640 1 -- 172.21.15.43:0/1726887982 shutdown_connections 2023-08-09T13:55:27.559 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:27.554+0000 7f3b0cde1640 1 --2- 172.21.15.43:0/1726887982 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f3af0003970 0x7f3af0003d90 unknown :-1 s=CLOSED pgs=1680 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:55:27.559 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:27.554+0000 7f3b0cde1640 1 -- 172.21.15.43:0/1726887982 >> 172.21.15.43:0/1726887982 conn(0x7f3b0806f150 msgr2=0x7f3b0806f560 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-09T13:55:27.560 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:27.554+0000 7f3b0cde1640 1 -- 172.21.15.43:0/1726887982 shutdown_connections 2023-08-09T13:55:27.560 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:27.554+0000 7f3b0cde1640 1 -- 172.21.15.43:0/1726887982 wait complete. 2023-08-09T13:55:27.561 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:27.556+0000 7f3b0cde1640 1 Processor -- start 2023-08-09T13:55:27.561 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:27.556+0000 7f3b0cde1640 1 -- start start 2023-08-09T13:55:27.561 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:27.557+0000 7f3b0cde1640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f3af0003970 0x7f3b0815a060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:55:27.562 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:27.557+0000 7f3b0cde1640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f3b080b5c60 0x7f3b081f1d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:55:27.562 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:27.557+0000 7f3b0cde1640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f3b081f2280 0x7f3b081f2710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:55:27.562 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:27.557+0000 7f3b0cde1640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3b081f2c50 con 0x7f3b080b5c60 2023-08-09T13:55:27.562 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:27.557+0000 7f3b0cde1640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3b0815af90 con 0x7f3b081f2280 2023-08-09T13:55:27.562 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:27.557+0000 7f3b0cde1640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3b0815a9d0 con 0x7f3af0003970 2023-08-09T13:55:27.562 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:27.557+0000 7f3b05d9b640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f3b080b5c60 0x7f3b081f1d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:55:27.562 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:27.557+0000 7f3b05d9b640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f3b080b5c60 0x7f3b081f1d40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:52496/0 (socket says 172.21.15.43:52496) 2023-08-09T13:55:27.563 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:27.557+0000 7f3b05d9b640 1 -- 172.21.15.43:0/3746507470 learned_addr learned my addr 172.21.15.43:0/3746507470 (peer_addr_for_me v2:172.21.15.43:0/0) 2023-08-09T13:55:27.563 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:27.557+0000 7f3b0659c640 1 --2- 172.21.15.43:0/3746507470 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f3af0003970 0x7f3b0815a060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:55:27.563 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:27.558+0000 7f3b06d9d640 1 --2- 172.21.15.43:0/3746507470 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f3b081f2280 0x7f3b081f2710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:55:27.563 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:27.558+0000 7f3b05d9b640 1 -- 172.21.15.43:0/3746507470 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f3af0003970 msgr2=0x7f3b0815a060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:55:27.563 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:27.558+0000 7f3b05d9b640 1 --2- 172.21.15.43:0/3746507470 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f3af0003970 0x7f3b0815a060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:55:27.563 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:27.558+0000 7f3b05d9b640 1 -- 172.21.15.43:0/3746507470 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f3b081f2280 msgr2=0x7f3b081f2710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:55:27.563 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:27.558+0000 7f3b05d9b640 1 --2- 172.21.15.43:0/3746507470 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f3b081f2280 0x7f3b081f2710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:55:27.563 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:27.558+0000 7f3b05d9b640 1 -- 172.21.15.43:0/3746507470 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3b0815acb0 con 0x7f3b080b5c60 2023-08-09T13:55:27.564 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:27.558+0000 7f3b05d9b640 1 --2- 172.21.15.43:0/3746507470 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f3b080b5c60 0x7f3b081f1d40 secure :-1 s=READY pgs=1681 cs=0 l=1 rev1=1 crypto rx=0x7f3afc00e410 tx=0x7f3afc00e8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-09T13:55:27.564 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:27.558+0000 7f3af6ffd640 1 -- 172.21.15.43:0/3746507470 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3afc0052e0 con 0x7f3b080b5c60 2023-08-09T13:55:27.564 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:27.558+0000 7f3af6ffd640 1 -- 172.21.15.43:0/3746507470 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3afc005440 con 0x7f3b080b5c60 2023-08-09T13:55:27.564 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:27.558+0000 7f3af6ffd640 1 -- 172.21.15.43:0/3746507470 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3afc003810 con 0x7f3b080b5c60 2023-08-09T13:55:27.564 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:27.558+0000 7f3b0cde1640 1 -- 172.21.15.43:0/3746507470 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3b080b3270 con 0x7f3b080b5c60 2023-08-09T13:55:27.564 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:27.558+0000 7f3b0cde1640 1 -- 172.21.15.43:0/3746507470 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3b0815a5a0 con 0x7f3b080b5c60 2023-08-09T13:55:27.565 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:27.559+0000 7f3b0cde1640 1 -- 172.21.15.43:0/3746507470 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3b0815a730 con 0x7f3b080b5c60 2023-08-09T13:55:27.565 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:27.561+0000 7f3af6ffd640 1 -- 172.21.15.43:0/3746507470 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 67) v1 ==== 88443+0+0 (secure 0 0 0) 0x7f3afc003970 con 0x7f3b080b5c60 2023-08-09T13:55:27.565 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:27.562+0000 7f3af6ffd640 1 --2- 172.21.15.43:0/3746507470 >> [v2:172.21.15.203:6832/3490266242,v1:172.21.15.203:6833/3490266242] conn(0x7f3ae806a1a0 0x7f3ae806c660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:55:27.565 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:27.562+0000 7f3af6ffd640 1 -- 172.21.15.43:0/3746507470 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(144..144 src has 1..144) v4 ==== 5944+0+0 (secure 0 0 0) 0x7f3afc0886b0 con 0x7f3b080b5c60 2023-08-09T13:55:27.566 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:27.562+0000 7f3b0659c640 1 --2- 172.21.15.43:0/3746507470 >> [v2:172.21.15.203:6832/3490266242,v1:172.21.15.203:6833/3490266242] conn(0x7f3ae806a1a0 0x7f3ae806c660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:55:27.566 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:27.564+0000 7f3b0659c640 1 --2- 172.21.15.43:0/3746507470 >> [v2:172.21.15.203:6832/3490266242,v1:172.21.15.203:6833/3490266242] conn(0x7f3ae806a1a0 0x7f3ae806c660 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f3af8034000 tx=0x7f3af8009030 comp rx=0 tx=0).ready entity=mgr.6659 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-09T13:55:27.568 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:27.566+0000 7f3af6ffd640 1 -- 172.21.15.43:0/3746507470 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178066 (secure 0 0 0) 0x7f3afc053e20 con 0x7f3b080b5c60 2023-08-09T13:55:27.767 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:27.765+0000 7f3b0cde1640 1 -- 172.21.15.43:0/3746507470 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9} v 0) v1 -- 0x7f3b080739f0 con 0x7f3b080b5c60 2023-08-09T13:55:28.307 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:28.303+0000 7f3af6ffd640 1 -- 172.21.15.43:0/3746507470 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]=0 pool 'cephfs_data' created v145) v1 ==== 170+0+0 (secure 0 0 0) 0x7f3afc057440 con 0x7f3b080b5c60 2023-08-09T13:55:28.308 INFO:teuthology.orchestra.run.smithi043.stderr:pool 'cephfs_data' created 2023-08-09T13:55:28.314 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:28.312+0000 7f3b0cde1640 1 -- 172.21.15.43:0/3746507470 >> [v2:172.21.15.203:6832/3490266242,v1:172.21.15.203:6833/3490266242] conn(0x7f3ae806a1a0 msgr2=0x7f3ae806c660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:55:28.315 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:28.312+0000 7f3b0cde1640 1 --2- 172.21.15.43:0/3746507470 >> [v2:172.21.15.203:6832/3490266242,v1:172.21.15.203:6833/3490266242] conn(0x7f3ae806a1a0 0x7f3ae806c660 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f3af8034000 tx=0x7f3af8009030 comp rx=0 tx=0).stop 2023-08-09T13:55:28.315 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:28.312+0000 7f3b0cde1640 1 -- 172.21.15.43:0/3746507470 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f3b080b5c60 msgr2=0x7f3b081f1d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:55:28.315 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:28.312+0000 7f3b0cde1640 1 --2- 172.21.15.43:0/3746507470 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f3b080b5c60 0x7f3b081f1d40 secure :-1 s=READY pgs=1681 cs=0 l=1 rev1=1 crypto rx=0x7f3afc00e410 tx=0x7f3afc00e8d0 comp rx=0 tx=0).stop 2023-08-09T13:55:28.316 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:28.312+0000 7f3b0cde1640 1 -- 172.21.15.43:0/3746507470 shutdown_connections 2023-08-09T13:55:28.316 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:28.312+0000 7f3b0cde1640 1 --2- 172.21.15.43:0/3746507470 >> [v2:172.21.15.203:6832/3490266242,v1:172.21.15.203:6833/3490266242] conn(0x7f3ae806a1a0 0x7f3ae806c660 unknown :-1 s=CLOSED pgs=56 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:55:28.316 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:28.312+0000 7f3b0cde1640 1 --2- 172.21.15.43:0/3746507470 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f3b081f2280 0x7f3b081f2710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:55:28.316 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:28.312+0000 7f3b0cde1640 1 --2- 172.21.15.43:0/3746507470 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f3b080b5c60 0x7f3b081f1d40 unknown :-1 s=CLOSED pgs=1681 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:55:28.316 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:28.312+0000 7f3b0cde1640 1 --2- 172.21.15.43:0/3746507470 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f3af0003970 0x7f3b0815a060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:55:28.316 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:28.312+0000 7f3b0cde1640 1 -- 172.21.15.43:0/3746507470 >> 172.21.15.43:0/3746507470 conn(0x7f3b0806f150 msgr2=0x7f3b08160560 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-09T13:55:28.317 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:28.313+0000 7f3b0cde1640 1 -- 172.21.15.43:0/3746507470 shutdown_connections 2023-08-09T13:55:28.317 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:28.314+0000 7f3b0cde1640 1 -- 172.21.15.43:0/3746507470 wait complete. 2023-08-09T13:55:28.346 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs new cephfs cephfs_metadata cephfs_data 2023-08-09T13:55:28.541 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:28.539+0000 7f54851e8640 1 Processor -- start 2023-08-09T13:55:28.542 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:28.539+0000 7f54851e8640 1 -- start start 2023-08-09T13:55:28.543 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:28.540+0000 7f54851e8640 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7f54800cb2d0 0x7f548015b380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:55:28.543 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:28.541+0000 7f54851e8640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f548015df60 0x7f54800ce1c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:55:28.543 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:28.541+0000 7f54851e8640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f548015e3c0 0x7f54800ce700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:55:28.543 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:28.541+0000 7f54851e8640 1 -- --> v2:172.21.15.43:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f548015a970 con 0x7f548015e3c0 2023-08-09T13:55:28.544 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:28.541+0000 7f54851e8640 1 -- --> v2:172.21.15.203:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f548015aef0 con 0x7f54800cb2d0 2023-08-09T13:55:28.544 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:28.541+0000 7f54851e8640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f548015b8c0 con 0x7f548015df60 2023-08-09T13:55:28.544 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:28.541+0000 7f547f59e640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f548015e3c0 0x7f54800ce700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:55:28.545 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:28.541+0000 7f547f59e640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f548015e3c0 0x7f54800ce700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:52510/0 (socket says 172.21.15.43:52510) 2023-08-09T13:55:28.545 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:28.541+0000 7f547f59e640 1 -- 172.21.15.43:0/2793567417 learned_addr learned my addr 172.21.15.43:0/2793567417 (peer_addr_for_me v2:172.21.15.43:0/0) 2023-08-09T13:55:28.545 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:28.541+0000 7f547e59c640 1 --2- 172.21.15.43:0/2793567417 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f548015df60 0x7f54800ce1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:55:28.545 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:28.541+0000 7f547f59e640 1 -- 172.21.15.43:0/2793567417 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f548015df60 msgr2=0x7f54800ce1c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-09T13:55:28.545 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:28.541+0000 7f547f59e640 1 --2- 172.21.15.43:0/2793567417 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f548015df60 0x7f54800ce1c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:55:28.545 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:28.541+0000 7f547ed9d640 1 --2- 172.21.15.43:0/2793567417 >> v2:172.21.15.203:3300/0 conn(0x7f54800cb2d0 0x7f548015b380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:55:28.545 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:28.541+0000 7f547f59e640 1 -- 172.21.15.43:0/2793567417 >> v2:172.21.15.203:3300/0 conn(0x7f54800cb2d0 msgr2=0x7f548015b380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-09T13:55:28.546 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:28.542+0000 7f547f59e640 1 --2- 172.21.15.43:0/2793567417 >> v2:172.21.15.203:3300/0 conn(0x7f54800cb2d0 0x7f548015b380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:55:28.546 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:28.542+0000 7f547f59e640 1 -- 172.21.15.43:0/2793567417 --> v2:172.21.15.43:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f548015e940 con 0x7f548015e3c0 2023-08-09T13:55:28.547 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:28.543+0000 7f547f59e640 1 --2- 172.21.15.43:0/2793567417 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f548015e3c0 0x7f54800ce700 secure :-1 s=READY pgs=1682 cs=0 l=1 rev1=1 crypto rx=0x7f54800cede0 tx=0x7f547802bee0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=263a88fd8105fc4a server_cookie=0 in_seq=0 out_seq=0 2023-08-09T13:55:28.547 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:28.543+0000 7f547dd9b640 1 -- 172.21.15.43:0/2793567417 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f547802ca80 con 0x7f548015e3c0 2023-08-09T13:55:28.547 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:28.543+0000 7f547dd9b640 1 -- 172.21.15.43:0/2793567417 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f547802cbe0 con 0x7f548015e3c0 2023-08-09T13:55:28.547 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:28.543+0000 7f547dd9b640 1 -- 172.21.15.43:0/2793567417 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f54780343f0 con 0x7f548015e3c0 2023-08-09T13:55:28.547 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:28.544+0000 7f54851e8640 1 -- 172.21.15.43:0/2793567417 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f548015e3c0 msgr2=0x7f54800ce700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:55:28.547 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:28.544+0000 7f54851e8640 1 --2- 172.21.15.43:0/2793567417 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f548015e3c0 0x7f54800ce700 secure :-1 s=READY pgs=1682 cs=0 l=1 rev1=1 crypto rx=0x7f54800cede0 tx=0x7f547802bee0 comp rx=0 tx=0).stop 2023-08-09T13:55:28.547 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:28.544+0000 7f54851e8640 1 -- 172.21.15.43:0/2793567417 shutdown_connections 2023-08-09T13:55:28.548 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:28.544+0000 7f54851e8640 1 --2- 172.21.15.43:0/2793567417 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f548015e3c0 0x7f54800ce700 unknown :-1 s=CLOSED pgs=1682 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:55:28.548 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:28.544+0000 7f54851e8640 1 --2- 172.21.15.43:0/2793567417 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f548015df60 0x7f54800ce1c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:55:28.548 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:28.544+0000 7f54851e8640 1 --2- 172.21.15.43:0/2793567417 >> v2:172.21.15.203:3300/0 conn(0x7f54800cb2d0 0x7f548015b380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:55:28.548 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:28.544+0000 7f54851e8640 1 -- 172.21.15.43:0/2793567417 >> 172.21.15.43:0/2793567417 conn(0x7f54800aff50 msgr2=0x7f54800b0360 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-09T13:55:28.548 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:28.544+0000 7f54851e8640 1 -- 172.21.15.43:0/2793567417 shutdown_connections 2023-08-09T13:55:28.549 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:28.544+0000 7f54851e8640 1 -- 172.21.15.43:0/2793567417 wait complete. 2023-08-09T13:55:28.549 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:28.546+0000 7f54851e8640 1 Processor -- start 2023-08-09T13:55:28.549 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:28.547+0000 7f54851e8640 1 -- start start 2023-08-09T13:55:28.550 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:28.548+0000 7f54851e8640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f54800cb2d0 0x7f54800c44c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:55:28.550 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:28.548+0000 7f54851e8640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f548015df60 0x7f548014b210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:55:28.550 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:28.548+0000 7f54851e8640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f548015e3c0 0x7f548014b750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:55:28.551 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:28.548+0000 7f54851e8640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f54800c4a00 con 0x7f548015df60 2023-08-09T13:55:28.551 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:28.548+0000 7f54851e8640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f54800c5380 con 0x7f54800cb2d0 2023-08-09T13:55:28.551 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:28.548+0000 7f54851e8640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f54801485a0 con 0x7f548015e3c0 2023-08-09T13:55:28.552 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:28.548+0000 7f547e59c640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f548015df60 0x7f548014b210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:55:28.552 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:28.548+0000 7f547e59c640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f548015df60 0x7f548014b210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:52526/0 (socket says 172.21.15.43:52526) 2023-08-09T13:55:28.552 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:28.548+0000 7f547e59c640 1 -- 172.21.15.43:0/2466750881 learned_addr learned my addr 172.21.15.43:0/2466750881 (peer_addr_for_me v2:172.21.15.43:0/0) 2023-08-09T13:55:28.552 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:28.549+0000 7f547f59e640 1 --2- 172.21.15.43:0/2466750881 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f548015e3c0 0x7f548014b750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:55:28.553 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:28.549+0000 7f547ed9d640 1 --2- 172.21.15.43:0/2466750881 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f54800cb2d0 0x7f54800c44c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:55:28.553 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:28.549+0000 7f547e59c640 1 -- 172.21.15.43:0/2466750881 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f548015e3c0 msgr2=0x7f548014b750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:55:28.553 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:28.549+0000 7f547e59c640 1 --2- 172.21.15.43:0/2466750881 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f548015e3c0 0x7f548014b750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:55:28.553 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:28.549+0000 7f547e59c640 1 -- 172.21.15.43:0/2466750881 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f54800cb2d0 msgr2=0x7f54800c44c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:55:28.553 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:28.549+0000 7f547e59c640 1 --2- 172.21.15.43:0/2466750881 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f54800cb2d0 0x7f54800c44c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:55:28.553 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:28.549+0000 7f547e59c640 1 -- 172.21.15.43:0/2466750881 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5478009b30 con 0x7f548015df60 2023-08-09T13:55:28.553 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:28.549+0000 7f547e59c640 1 --2- 172.21.15.43:0/2466750881 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f548015df60 0x7f548014b210 secure :-1 s=READY pgs=1683 cs=0 l=1 rev1=1 crypto rx=0x7f547400d800 tx=0x7f547400dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-09T13:55:28.554 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:28.549+0000 7f54677fe640 1 -- 172.21.15.43:0/2466750881 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f547400b6c0 con 0x7f548015df60 2023-08-09T13:55:28.554 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:28.549+0000 7f54677fe640 1 -- 172.21.15.43:0/2466750881 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f547400b820 con 0x7f548015df60 2023-08-09T13:55:28.554 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:28.549+0000 7f54677fe640 1 -- 172.21.15.43:0/2466750881 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f547400bbb0 con 0x7f548015df60 2023-08-09T13:55:28.554 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:28.549+0000 7f54851e8640 1 -- 172.21.15.43:0/2466750881 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f548014c090 con 0x7f548015df60 2023-08-09T13:55:28.554 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:28.550+0000 7f54851e8640 1 -- 172.21.15.43:0/2466750881 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f548014cb20 con 0x7f548015df60 2023-08-09T13:55:28.554 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:28.551+0000 7f54851e8640 1 -- 172.21.15.43:0/2466750881 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f54800739f0 con 0x7f548015df60 2023-08-09T13:55:28.555 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:28.552+0000 7f54677fe640 1 -- 172.21.15.43:0/2466750881 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 67) v1 ==== 88443+0+0 (secure 0 0 0) 0x7f547400bd10 con 0x7f548015df60 2023-08-09T13:55:28.555 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:28.553+0000 7f54677fe640 1 --2- 172.21.15.43:0/2466750881 >> [v2:172.21.15.203:6832/3490266242,v1:172.21.15.203:6833/3490266242] conn(0x7f546c06a170 0x7f546c06c630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:55:28.555 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:28.553+0000 7f54677fe640 1 -- 172.21.15.43:0/2466750881 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(145..145 src has 1..145) v4 ==== 6326+0+0 (secure 0 0 0) 0x7f5474088e40 con 0x7f548015df60 2023-08-09T13:55:28.556 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:28.554+0000 7f547ed9d640 1 --2- 172.21.15.43:0/2466750881 >> [v2:172.21.15.203:6832/3490266242,v1:172.21.15.203:6833/3490266242] conn(0x7f546c06a170 0x7f546c06c630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:55:28.557 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:28.555+0000 7f547ed9d640 1 --2- 172.21.15.43:0/2466750881 >> [v2:172.21.15.203:6832/3490266242,v1:172.21.15.203:6833/3490266242] conn(0x7f546c06a170 0x7f546c06c630 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f547000a990 tx=0x7f5470008040 comp rx=0 tx=0).ready entity=mgr.6659 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-09T13:55:28.560 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:28.558+0000 7f54677fe640 1 -- 172.21.15.43:0/2466750881 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178066 (secure 0 0 0) 0x7f5474054270 con 0x7f548015df60 2023-08-09T13:55:28.793 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:28.791+0000 7f54851e8640 1 -- 172.21.15.43:0/2466750881 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"} v 0) v1 -- 0x7f548006f5a0 con 0x7f548015df60 2023-08-09T13:55:29.350 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:29.348+0000 7f54677fe640 1 -- 172.21.15.43:0/2466750881 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]=0 new fs with metadata pool 18 and data pool 19 v66) v1 ==== 174+0+0 (secure 0 0 0) 0x7f5474057890 con 0x7f548015df60 2023-08-09T13:55:29.351 INFO:teuthology.orchestra.run.smithi043.stderr:new fs with metadata pool 18 and data pool 19 2023-08-09T13:55:29.357 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:29.355+0000 7f54851e8640 1 -- 172.21.15.43:0/2466750881 >> [v2:172.21.15.203:6832/3490266242,v1:172.21.15.203:6833/3490266242] conn(0x7f546c06a170 msgr2=0x7f546c06c630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:55:29.357 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:29.355+0000 7f54851e8640 1 --2- 172.21.15.43:0/2466750881 >> [v2:172.21.15.203:6832/3490266242,v1:172.21.15.203:6833/3490266242] conn(0x7f546c06a170 0x7f546c06c630 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f547000a990 tx=0x7f5470008040 comp rx=0 tx=0).stop 2023-08-09T13:55:29.357 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:29.355+0000 7f54851e8640 1 -- 172.21.15.43:0/2466750881 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f548015df60 msgr2=0x7f548014b210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:55:29.357 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:29.355+0000 7f54851e8640 1 --2- 172.21.15.43:0/2466750881 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f548015df60 0x7f548014b210 secure :-1 s=READY pgs=1683 cs=0 l=1 rev1=1 crypto rx=0x7f547400d800 tx=0x7f547400dcc0 comp rx=0 tx=0).stop 2023-08-09T13:55:29.358 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:29.355+0000 7f54851e8640 1 -- 172.21.15.43:0/2466750881 shutdown_connections 2023-08-09T13:55:29.358 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:29.355+0000 7f54851e8640 1 --2- 172.21.15.43:0/2466750881 >> [v2:172.21.15.203:6832/3490266242,v1:172.21.15.203:6833/3490266242] conn(0x7f546c06a170 0x7f546c06c630 secure :-1 s=CLOSED pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f547000a990 tx=0x7f5470008040 comp rx=0 tx=0).stop 2023-08-09T13:55:29.358 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:29.355+0000 7f54851e8640 1 --2- 172.21.15.43:0/2466750881 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f548015e3c0 0x7f548014b750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:55:29.358 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:29.355+0000 7f54851e8640 1 --2- 172.21.15.43:0/2466750881 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f548015df60 0x7f548014b210 unknown :-1 s=CLOSED pgs=1683 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:55:29.359 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:29.355+0000 7f54851e8640 1 --2- 172.21.15.43:0/2466750881 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f54800cb2d0 0x7f54800c44c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:55:29.359 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:29.355+0000 7f54851e8640 1 -- 172.21.15.43:0/2466750881 >> 172.21.15.43:0/2466750881 conn(0x7f54800aff50 msgr2=0x7f54800b0330 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-09T13:55:29.363 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:29.361+0000 7f54851e8640 1 -- 172.21.15.43:0/2466750881 shutdown_connections 2023-08-09T13:55:29.365 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:29.363+0000 7f54851e8640 1 -- 172.21.15.43:0/2466750881 wait complete. 2023-08-09T13:55:29.399 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-08-09T13:55:29.590 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:29.588+0000 7f4df0984640 1 Processor -- start 2023-08-09T13:55:29.590 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:29.588+0000 7f4df0984640 1 -- start start 2023-08-09T13:55:29.592 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:29.589+0000 7f4df0984640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f4dec14a5c0 0x7f4dec160f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:55:29.592 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:29.589+0000 7f4df0984640 1 -- --> v1:172.21.15.43:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4dec0c35e0 con 0x7f4dec0cc380 2023-08-09T13:55:29.592 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:29.589+0000 7f4df0984640 1 -- --> v1:172.21.15.203:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4dec0c38a0 con 0x7f4dec0c3130 2023-08-09T13:55:29.592 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:29.589+0000 7f4df0984640 1 -- --> v2:172.21.15.43:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4dec0c4420 con 0x7f4dec14a5c0 2023-08-09T13:55:29.593 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:29.590+0000 7f4dea59c640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f4dec14a5c0 0x7f4dec160f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:55:29.593 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:29.590+0000 7f4dea59c640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f4dec14a5c0 0x7f4dec160f20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:52528/0 (socket says 172.21.15.43:52528) 2023-08-09T13:55:29.593 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:29.590+0000 7f4dead9d640 1 --1- >> v1:172.21.15.43:6789/0 conn(0x7f4dec0cc380 0x7f4dec06bb50 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.43:6789/0 says I am v1:172.21.15.43:56622/0 (socket says 172.21.15.43:56622) 2023-08-09T13:55:29.594 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:29.590+0000 7f4dea59c640 1 -- 172.21.15.43:0/341077472 learned_addr learned my addr 172.21.15.43:0/341077472 (peer_addr_for_me v2:172.21.15.43:0/0) 2023-08-09T13:55:29.594 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:29.590+0000 7f4dea59c640 1 -- 172.21.15.43:0/341077472 >> v1:172.21.15.203:6789/0 conn(0x7f4dec0c3130 legacy=0x7f4dec0cbc70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-09T13:55:29.594 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:29.590+0000 7f4dea59c640 1 -- 172.21.15.43:0/341077472 >> v1:172.21.15.43:6789/0 conn(0x7f4dec0cc380 legacy=0x7f4dec06bb50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:55:29.594 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:29.590+0000 7f4dea59c640 1 -- 172.21.15.43:0/341077472 --> v2:172.21.15.43:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4dec0c3b80 con 0x7f4dec14a5c0 2023-08-09T13:55:29.594 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:29.590+0000 7f4de959a640 1 -- 172.21.15.43:0/341077472 <== mon.0 v1:172.21.15.43:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3568656039 0 0) 0x7f4dec0c35e0 con 0x7f4dec0cc380 2023-08-09T13:55:29.595 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:29.592+0000 7f4dea59c640 1 --2- 172.21.15.43:0/341077472 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f4dec14a5c0 0x7f4dec160f20 secure :-1 s=READY pgs=1685 cs=0 l=1 rev1=1 crypto rx=0x7f4ddc0098a0 tx=0x7f4ddc02b8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=17fb1d1f6e7de3db server_cookie=0 in_seq=0 out_seq=0 2023-08-09T13:55:29.595 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:29.592+0000 7f4de959a640 1 -- 172.21.15.43:0/341077472 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4ddc031640 con 0x7f4dec14a5c0 2023-08-09T13:55:29.595 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:29.592+0000 7f4de959a640 1 -- 172.21.15.43:0/341077472 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4ddc0317a0 con 0x7f4dec14a5c0 2023-08-09T13:55:29.596 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:29.592+0000 7f4de959a640 1 -- 172.21.15.43:0/341077472 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4ddc031b30 con 0x7f4dec14a5c0 2023-08-09T13:55:29.596 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:29.592+0000 7f4df0984640 1 -- 172.21.15.43:0/341077472 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f4dec14a5c0 msgr2=0x7f4dec160f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:55:29.596 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:29.592+0000 7f4df0984640 1 --2- 172.21.15.43:0/341077472 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f4dec14a5c0 0x7f4dec160f20 secure :-1 s=READY pgs=1685 cs=0 l=1 rev1=1 crypto rx=0x7f4ddc0098a0 tx=0x7f4ddc02b8f0 comp rx=0 tx=0).stop 2023-08-09T13:55:29.596 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:29.593+0000 7f4df0984640 1 -- 172.21.15.43:0/341077472 shutdown_connections 2023-08-09T13:55:29.596 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:29.593+0000 7f4df0984640 1 --2- 172.21.15.43:0/341077472 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f4dec14a5c0 0x7f4dec160f20 unknown :-1 s=CLOSED pgs=1685 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:55:29.596 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:29.593+0000 7f4df0984640 1 -- 172.21.15.43:0/341077472 >> 172.21.15.43:0/341077472 conn(0x7f4dec0ce0a0 msgr2=0x7f4dec0ce480 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-09T13:55:29.596 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:29.593+0000 7f4df0984640 1 -- 172.21.15.43:0/341077472 shutdown_connections 2023-08-09T13:55:29.597 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:29.593+0000 7f4df0984640 1 -- 172.21.15.43:0/341077472 wait complete. 2023-08-09T13:55:29.597 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:29.594+0000 7f4df0984640 1 Processor -- start 2023-08-09T13:55:29.597 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:29.595+0000 7f4df0984640 1 -- start start 2023-08-09T13:55:29.598 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:29.596+0000 7f4df0984640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f4dec0c3130 0x7f4dec160700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:55:29.598 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:29.596+0000 7f4df0984640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f4dec0cc380 0x7f4dec0b30a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:55:29.599 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:29.596+0000 7f4df0984640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f4dec14a5c0 0x7f4dec0caa80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:55:29.599 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:29.596+0000 7f4df0984640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4dec147000 con 0x7f4dec0cc380 2023-08-09T13:55:29.599 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:29.596+0000 7f4df0984640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4dec160c40 con 0x7f4dec14a5c0 2023-08-09T13:55:29.599 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:29.596+0000 7f4df0984640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4dec0b3a10 con 0x7f4dec0c3130 2023-08-09T13:55:29.599 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:29.596+0000 7f4de9d9b640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f4dec0cc380 0x7f4dec0b30a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:55:29.599 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:29.596+0000 7f4de9d9b640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f4dec0cc380 0x7f4dec0b30a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:52540/0 (socket says 172.21.15.43:52540) 2023-08-09T13:55:29.600 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:29.596+0000 7f4de9d9b640 1 -- 172.21.15.43:0/1234786808 learned_addr learned my addr 172.21.15.43:0/1234786808 (peer_addr_for_me v2:172.21.15.43:0/0) 2023-08-09T13:55:29.602 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:29.596+0000 7f4dea59c640 1 --2- 172.21.15.43:0/1234786808 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f4dec0c3130 0x7f4dec160700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:55:29.602 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:29.597+0000 7f4dead9d640 1 --2- 172.21.15.43:0/1234786808 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f4dec14a5c0 0x7f4dec0caa80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:55:29.603 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:29.597+0000 7f4de9d9b640 1 -- 172.21.15.43:0/1234786808 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f4dec0c3130 msgr2=0x7f4dec160700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:55:29.603 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:29.597+0000 7f4de9d9b640 1 --2- 172.21.15.43:0/1234786808 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f4dec0c3130 0x7f4dec160700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:55:29.603 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:29.597+0000 7f4de9d9b640 1 -- 172.21.15.43:0/1234786808 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f4dec14a5c0 msgr2=0x7f4dec0caa80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:55:29.604 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:29.597+0000 7f4de9d9b640 1 --2- 172.21.15.43:0/1234786808 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f4dec14a5c0 0x7f4dec0caa80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:55:29.604 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:29.597+0000 7f4de9d9b640 1 -- 172.21.15.43:0/1234786808 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4ddc0094d0 con 0x7f4dec0cc380 2023-08-09T13:55:29.604 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:29.597+0000 7f4de9d9b640 1 --2- 172.21.15.43:0/1234786808 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f4dec0cc380 0x7f4dec0b30a0 secure :-1 s=READY pgs=1686 cs=0 l=1 rev1=1 crypto rx=0x7f4dd400c430 tx=0x7f4dd400c8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-09T13:55:29.605 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:29.597+0000 7f4de2ffd640 1 -- 172.21.15.43:0/1234786808 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4dd4011790 con 0x7f4dec0cc380 2023-08-09T13:55:29.605 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:29.597+0000 7f4de2ffd640 1 -- 172.21.15.43:0/1234786808 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4dd40118f0 con 0x7f4dec0cc380 2023-08-09T13:55:29.605 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:29.597+0000 7f4df0984640 1 -- 172.21.15.43:0/1234786808 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4dec146940 con 0x7f4dec0cc380 2023-08-09T13:55:29.606 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:29.598+0000 7f4de2ffd640 1 -- 172.21.15.43:0/1234786808 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4dd4011ce0 con 0x7f4dec0cc380 2023-08-09T13:55:29.606 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:29.598+0000 7f4df0984640 1 -- 172.21.15.43:0/1234786808 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4dec0b3cf0 con 0x7f4dec0cc380 2023-08-09T13:55:29.606 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:29.599+0000 7f4df0984640 1 -- 172.21.15.43:0/1234786808 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4dec1467a0 con 0x7f4dec0cc380 2023-08-09T13:55:29.607 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:29.600+0000 7f4de2ffd640 1 -- 172.21.15.43:0/1234786808 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 67) v1 ==== 88443+0+0 (secure 0 0 0) 0x7f4dd401c460 con 0x7f4dec0cc380 2023-08-09T13:55:29.608 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:29.601+0000 7f4de2ffd640 1 --2- 172.21.15.43:0/1234786808 >> [v2:172.21.15.203:6832/3490266242,v1:172.21.15.203:6833/3490266242] conn(0x7f4dcc06a1a0 0x7f4dcc06c660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:55:29.609 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:29.601+0000 7f4de2ffd640 1 -- 172.21.15.43:0/1234786808 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(146..146 src has 1..146) v4 ==== 6426+0+0 (secure 0 0 0) 0x7f4dd40887d0 con 0x7f4dec0cc380 2023-08-09T13:55:29.609 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:29.601+0000 7f4dea59c640 1 --2- 172.21.15.43:0/1234786808 >> [v2:172.21.15.203:6832/3490266242,v1:172.21.15.203:6833/3490266242] conn(0x7f4dcc06a1a0 0x7f4dcc06c660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:55:29.609 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:29.602+0000 7f4dea59c640 1 --2- 172.21.15.43:0/1234786808 >> [v2:172.21.15.203:6832/3490266242,v1:172.21.15.203:6833/3490266242] conn(0x7f4dcc06a1a0 0x7f4dcc06c660 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f4ddc036000 tx=0x7f4ddc03c040 comp rx=0 tx=0).ready entity=mgr.6659 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-09T13:55:29.609 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:29.605+0000 7f4de2ffd640 1 -- 172.21.15.43:0/1234786808 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178066 (secure 0 0 0) 0x7f4dec1467a0 con 0x7f4dec0cc380 2023-08-09T13:55:29.803 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:29.800+0000 7f4df0984640 1 -- 172.21.15.43:0/1234786808 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f4dec147140 con 0x7f4dec0cc380 2023-08-09T13:55:29.806 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:29.803+0000 7f4de2ffd640 1 -- 172.21.15.43:0/1234786808 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v146) v1 ==== 74+0+18579 (secure 0 0 0) 0x7f4dec147140 con 0x7f4dec0cc380 2023-08-09T13:55:29.807 INFO:teuthology.orchestra.run.smithi043.stdout: 2023-08-09T13:55:29.807 INFO:teuthology.orchestra.run.smithi043.stdout:{"epoch":146,"fsid":"daac87ee-1337-4250-a955-aea4e408a39b","created":"2023-08-09T13:33:41.701198+0000","modified":"2023-08-09T13:55:28.793575+0000","last_up_change":"2023-08-09T13:33:53.783453+0000","last_in_change":"2023-08-09T13:33:45.148904+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":19,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":"rbd","create_time":"2023-08-09T13:33:54.365137+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":".mgr","create_time":"2023-08-09T13:33:54.831803+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":18,"pool_name":"cephfs_metadata","create_time":"2023-08-09T13:55:27.207507+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"146","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":19,"pool_name":"cephfs_data","create_time":"2023-08-09T13:55:27.825578+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"146","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"cbd49ecd-e896-4fbd-864e-5c37a829b165","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":145,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6800","nonce":502667335},{"type":"v1","addr":"172.21.15.43:6801","nonce":502667335}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6802","nonce":502667335},{"type":"v1","addr":"172.21.15.43:6803","nonce":502667335}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6806","nonce":502667335},{"type":"v1","addr":"172.21.15.43:6807","nonce":502667335}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6804","nonce":502667335},{"type":"v1","addr":"172.21.15.43:6805","nonce":502667335}]},"public_addr":"172.21.15.43:6801/502667335","cluster_addr":"172.21.15.43:6803/502667335","heartbeat_back_addr":"172.21.15.43:6807/502667335","heartbeat_front_addr":"172.21.15.43:6805/502667335","state":["exists","up"]},{"osd":1,"uuid":"75c7e0fe-1ec7-4fe8-a2f2-3661a65b8500","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":145,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6808","nonce":3289134610},{"type":"v1","addr":"172.21.15.43:6809","nonce":3289134610}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6810","nonce":3289134610},{"type":"v1","addr":"172.21.15.43:6812","nonce":3289134610}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6821","nonce":3289134610},{"type":"v1","addr":"172.21.15.43:6824","nonce":3289134610}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6815","nonce":3289134610},{"type":"v1","addr":"172.21.15.43:6818","nonce":3289134610}]},"public_addr":"172.21.15.43:6809/3289134610","cluster_addr":"172.21.15.43:6812/3289134610","heartbeat_back_addr":"172.21.15.43:6824/3289134610","heartbeat_front_addr":"172.21.15.43:6818/3289134610","state":["exists","up"]},{"osd":2,"uuid":"bb5c41e9-7f5f-4125-9b01-3bb8bfc2ea83","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":145,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6811","nonce":4079709736},{"type":"v1","addr":"172.21.15.43:6814","nonce":4079709736}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6817","nonce":4079709736},{"type":"v1","addr":"172.21.15.43:6820","nonce":4079709736}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6829","nonce":4079709736},{"type":"v1","addr":"172.21.15.43:6831","nonce":4079709736}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6823","nonce":4079709736},{"type":"v1","addr":"172.21.15.43:6826","nonce":4079709736}]},"public_addr":"172.21.15.43:6814/4079709736","cluster_addr":"172.21.15.43:6820/4079709736","heartbeat_back_addr":"172.21.15.43:6831/4079709736","heartbeat_front_addr":"172.21.15.43:6826/4079709736","state":["exists","up"]},{"osd":3,"uuid":"bfe4c8b4-694a-465c-87e6-2b2aab852613","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":145,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6813","nonce":1709538759},{"type":"v1","addr":"172.21.15.43:6816","nonce":1709538759}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6819","nonce":1709538759},{"type":"v1","addr":"172.21.15.43:6822","nonce":1709538759}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6828","nonce":1709538759},{"type":"v1","addr":"172.21.15.43:6830","nonce":1709538759}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6825","nonce":1709538759},{"type":"v1","addr":"172.21.15.43:6827","nonce":1709538759}]},"public_addr":"172.21.15.43:6816/1709538759","cluster_addr":"172.21.15.43:6822/1709538759","heartbeat_back_addr":"172.21.15.43:6830/1709538759","heartbeat_front_addr":"172.21.15.43:6827/1709538759","state":["exists","up"]},{"osd":4,"uuid":"fa2e8a77-aba1-4756-853e-0d174a5bdeba","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":145,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6800","nonce":4196866481},{"type":"v1","addr":"172.21.15.203:6801","nonce":4196866481}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6802","nonce":4196866481},{"type":"v1","addr":"172.21.15.203:6803","nonce":4196866481}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6806","nonce":4196866481},{"type":"v1","addr":"172.21.15.203:6807","nonce":4196866481}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6804","nonce":4196866481},{"type":"v1","addr":"172.21.15.203:6805","nonce":4196866481}]},"public_addr":"172.21.15.203:6801/4196866481","cluster_addr":"172.21.15.203:6803/4196866481","heartbeat_back_addr":"172.21.15.203:6807/4196866481","heartbeat_front_addr":"172.21.15.203:6805/4196866481","state":["exists","up"]},{"osd":5,"uuid":"81811e4b-4d24-4f51-a82c-f1d700a2a00d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":145,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6816","nonce":3059290081},{"type":"v1","addr":"172.21.15.203:6817","nonce":3059290081}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6818","nonce":3059290081},{"type":"v1","addr":"172.21.15.203:6819","nonce":3059290081}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6822","nonce":3059290081},{"type":"v1","addr":"172.21.15.203:6823","nonce":3059290081}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6820","nonce":3059290081},{"type":"v1","addr":"172.21.15.203:6821","nonce":3059290081}]},"public_addr":"172.21.15.203:6817/3059290081","cluster_addr":"172.21.15.203:6819/3059290081","heartbeat_back_addr":"172.21.15.203:6823/3059290081","heartbeat_front_addr":"172.21.15.203:6821/3059290081","state":["exists","up"]},{"osd":6,"uuid":"307bac4d-6a5b-443d-af75-56d695d7674f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":145,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6824","nonce":512155569},{"type":"v1","addr":"172.21.15.203:6825","nonce":512155569}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6826","nonce":512155569},{"type":"v1","addr":"172.21.15.203:6827","nonce":512155569}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6830","nonce":512155569},{"type":"v1","addr":"172.21.15.203:6831","nonce":512155569}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6828","nonce":512155569},{"type":"v1","addr":"172.21.15.203:6829","nonce":512155569}]},"public_addr":"172.21.15.203:6825/512155569","cluster_addr":"172.21.15.203:6827/512155569","heartbeat_back_addr":"172.21.15.203:6831/512155569","heartbeat_front_addr":"172.21.15.203:6829/512155569","state":["exists","up"]},{"osd":7,"uuid":"a95dc7ca-9f1b-4295-af70-68a22a2d7d5e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":145,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6808","nonce":3331450438},{"type":"v1","addr":"172.21.15.203:6809","nonce":3331450438}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6810","nonce":3331450438},{"type":"v1","addr":"172.21.15.203:6811","nonce":3331450438}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6814","nonce":3331450438},{"type":"v1","addr":"172.21.15.203:6815","nonce":3331450438}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6812","nonce":3331450438},{"type":"v1","addr":"172.21.15.203:6813","nonce":3331450438}]},"public_addr":"172.21.15.203:6809/3331450438","cluster_addr":"172.21.15.203:6811/3331450438","heartbeat_back_addr":"172.21.15.203:6815/3331450438","heartbeat_front_addr":"172.21.15.203:6813/3331450438","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-09T13:33:50.547652+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-09T13:33:50.589469+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-09T13:33:50.561724+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-09T13:33:50.599650+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-09T13:33:50.411855+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-09T13:33:50.219681+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-09T13:33:50.504664+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-09T13:33:50.441248+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":7,"snaps":[{"begin":104,"length":1},{"begin":221,"length":1}]}],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-08-09T13:55:29.808 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:29.806+0000 7f4df0984640 1 -- 172.21.15.43:0/1234786808 >> [v2:172.21.15.203:6832/3490266242,v1:172.21.15.203:6833/3490266242] conn(0x7f4dcc06a1a0 msgr2=0x7f4dcc06c660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:55:29.808 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:29.806+0000 7f4df0984640 1 --2- 172.21.15.43:0/1234786808 >> [v2:172.21.15.203:6832/3490266242,v1:172.21.15.203:6833/3490266242] conn(0x7f4dcc06a1a0 0x7f4dcc06c660 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f4ddc036000 tx=0x7f4ddc03c040 comp rx=0 tx=0).stop 2023-08-09T13:55:29.808 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:29.806+0000 7f4df0984640 1 -- 172.21.15.43:0/1234786808 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f4dec0cc380 msgr2=0x7f4dec0b30a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:55:29.809 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:29.806+0000 7f4df0984640 1 --2- 172.21.15.43:0/1234786808 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f4dec0cc380 0x7f4dec0b30a0 secure :-1 s=READY pgs=1686 cs=0 l=1 rev1=1 crypto rx=0x7f4dd400c430 tx=0x7f4dd400c8f0 comp rx=0 tx=0).stop 2023-08-09T13:55:29.809 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:29.806+0000 7f4df0984640 1 -- 172.21.15.43:0/1234786808 shutdown_connections 2023-08-09T13:55:29.809 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:29.806+0000 7f4df0984640 1 --2- 172.21.15.43:0/1234786808 >> [v2:172.21.15.203:6832/3490266242,v1:172.21.15.203:6833/3490266242] conn(0x7f4dcc06a1a0 0x7f4dcc06c660 unknown :-1 s=CLOSED pgs=58 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:55:29.809 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:29.806+0000 7f4df0984640 1 --2- 172.21.15.43:0/1234786808 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f4dec14a5c0 0x7f4dec0caa80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:55:29.810 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:29.806+0000 7f4df0984640 1 --2- 172.21.15.43:0/1234786808 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f4dec0cc380 0x7f4dec0b30a0 unknown :-1 s=CLOSED pgs=1686 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:55:29.810 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:29.806+0000 7f4df0984640 1 --2- 172.21.15.43:0/1234786808 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f4dec0c3130 0x7f4dec160700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:55:29.810 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:29.806+0000 7f4df0984640 1 -- 172.21.15.43:0/1234786808 >> 172.21.15.43:0/1234786808 conn(0x7f4dec0ce0a0 msgr2=0x7f4dec0ce480 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-09T13:55:29.810 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:29.806+0000 7f4df0984640 1 -- 172.21.15.43:0/1234786808 shutdown_connections 2023-08-09T13:55:29.810 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:29.806+0000 7f4df0984640 1 -- 172.21.15.43:0/1234786808 wait complete. 2023-08-09T13:55:29.830 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-08-09T13:55:30.065 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:30.063+0000 7f8cebfff640 1 Processor -- start 2023-08-09T13:55:30.065 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:30.063+0000 7f8cebfff640 1 -- start start 2023-08-09T13:55:30.067 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:30.065+0000 7f8cebfff640 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7f8cec15c320 0x7f8cec06b2c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:55:30.067 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:30.065+0000 7f8cebfff640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f8cec15c700 0x7f8cec06b800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:55:30.067 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:30.065+0000 7f8cebfff640 1 -- --> v1:172.21.15.43:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8cec0b14b0 con 0x7f8cec0b0980 2023-08-09T13:55:30.067 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:30.065+0000 7f8cebfff640 1 -- --> v2:172.21.15.43:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8cec0cae50 con 0x7f8cec15c700 2023-08-09T13:55:30.068 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:30.065+0000 7f8cebfff640 1 -- --> v2:172.21.15.203:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8cec0ca870 con 0x7f8cec15c320 2023-08-09T13:55:30.068 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:30.065+0000 7f8cea7fc640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f8cec15c700 0x7f8cec06b800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:55:30.069 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:30.065+0000 7f8ceb7fe640 1 --1- >> v1:172.21.15.43:6789/0 conn(0x7f8cec0b0980 0x7f8cec0b0da0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.43:6789/0 says I am v1:172.21.15.43:56626/0 (socket says 172.21.15.43:56626) 2023-08-09T13:55:30.069 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:30.065+0000 7f8ceb7fe640 1 -- 172.21.15.43:0/1765469068 learned_addr learned my addr 172.21.15.43:0/1765469068 (peer_addr_for_me v1:172.21.15.43:0/0) 2023-08-09T13:55:30.069 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:30.065+0000 7f8ceaffd640 1 --2- 172.21.15.43:0/1765469068 >> v2:172.21.15.203:3300/0 conn(0x7f8cec15c320 0x7f8cec06b2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:55:30.069 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:30.066+0000 7f8ce9ffb640 1 -- 172.21.15.43:0/1765469068 <== mon.0 v1:172.21.15.43:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1542519373 0 0) 0x7f8cec0b14b0 con 0x7f8cec0b0980 2023-08-09T13:55:30.069 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:30.066+0000 7f8cea7fc640 1 -- 172.21.15.43:0/1765469068 >> v2:172.21.15.203:3300/0 conn(0x7f8cec15c320 msgr2=0x7f8cec06b2c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-09T13:55:30.069 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:30.066+0000 7f8cea7fc640 1 --2- 172.21.15.43:0/1765469068 >> v2:172.21.15.203:3300/0 conn(0x7f8cec15c320 0x7f8cec06b2c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:55:30.069 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:30.066+0000 7f8cea7fc640 1 -- 172.21.15.43:0/1765469068 >> v1:172.21.15.43:6789/0 conn(0x7f8cec0b0980 legacy=0x7f8cec0b0da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:55:30.069 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:30.066+0000 7f8cea7fc640 1 -- 172.21.15.43:0/1765469068 --> v2:172.21.15.43:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8cec0b16e0 con 0x7f8cec15c700 2023-08-09T13:55:30.070 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:30.067+0000 7f8cea7fc640 1 --2- 172.21.15.43:0/1765469068 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f8cec15c700 0x7f8cec06b800 secure :-1 s=READY pgs=1688 cs=0 l=1 rev1=1 crypto rx=0x7f8cec08a970 tx=0x7f8ce002b960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6deb2c8f1dcd1cd server_cookie=0 in_seq=0 out_seq=0 2023-08-09T13:55:30.071 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:30.067+0000 7f8ce9ffb640 1 -- 172.21.15.43:0/1765469068 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8ce0030610 con 0x7f8cec15c700 2023-08-09T13:55:30.071 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:30.067+0000 7f8ce9ffb640 1 -- 172.21.15.43:0/1765469068 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8ce0030770 con 0x7f8cec15c700 2023-08-09T13:55:30.071 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:30.067+0000 7f8ce9ffb640 1 -- 172.21.15.43:0/1765469068 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8ce0030b00 con 0x7f8cec15c700 2023-08-09T13:55:30.071 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:30.068+0000 7f8cebfff640 1 -- 172.21.15.43:0/1765469068 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f8cec15c700 msgr2=0x7f8cec06b800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:55:30.071 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:30.068+0000 7f8cebfff640 1 --2- 172.21.15.43:0/1765469068 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f8cec15c700 0x7f8cec06b800 secure :-1 s=READY pgs=1688 cs=0 l=1 rev1=1 crypto rx=0x7f8cec08a970 tx=0x7f8ce002b960 comp rx=0 tx=0).stop 2023-08-09T13:55:30.071 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:30.068+0000 7f8cebfff640 1 -- 172.21.15.43:0/1765469068 shutdown_connections 2023-08-09T13:55:30.071 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:30.068+0000 7f8cebfff640 1 --2- 172.21.15.43:0/1765469068 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f8cec15c700 0x7f8cec06b800 unknown :-1 s=CLOSED pgs=1688 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:55:30.072 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:30.068+0000 7f8cebfff640 1 --2- 172.21.15.43:0/1765469068 >> v2:172.21.15.203:3300/0 conn(0x7f8cec15c320 0x7f8cec06b2c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:55:30.072 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:30.068+0000 7f8cebfff640 1 -- 172.21.15.43:0/1765469068 >> 172.21.15.43:0/1765469068 conn(0x7f8cec089fe0 msgr2=0x7f8cec099450 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-09T13:55:30.072 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:30.068+0000 7f8cebfff640 1 -- 172.21.15.43:0/1765469068 shutdown_connections 2023-08-09T13:55:30.072 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:30.068+0000 7f8cebfff640 1 -- 172.21.15.43:0/1765469068 wait complete. 2023-08-09T13:55:30.072 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:30.070+0000 7f8cebfff640 1 Processor -- start 2023-08-09T13:55:30.073 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:30.070+0000 7f8cebfff640 1 -- start start 2023-08-09T13:55:30.074 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:30.071+0000 7f8cebfff640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f8cec0b0980 0x7f8cec1f16c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:55:30.074 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:30.071+0000 7f8cebfff640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f8cec15c320 0x7f8cec0c7250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:55:30.074 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:30.071+0000 7f8cebfff640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f8cec15c700 0x7f8cec157c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:55:30.074 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:30.071+0000 7f8cebfff640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8cec150f60 con 0x7f8cec0b0980 2023-08-09T13:55:30.075 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:30.071+0000 7f8cebfff640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8cec1582e0 con 0x7f8cec15c320 2023-08-09T13:55:30.075 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:30.071+0000 7f8cebfff640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8cec1585c0 con 0x7f8cec15c700 2023-08-09T13:55:30.075 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:30.071+0000 7f8ceaffd640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f8cec0b0980 0x7f8cec1f16c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:55:30.075 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:30.071+0000 7f8ceaffd640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f8cec0b0980 0x7f8cec1f16c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:52558/0 (socket says 172.21.15.43:52558) 2023-08-09T13:55:30.075 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:30.071+0000 7f8ceaffd640 1 -- 172.21.15.43:0/222240009 learned_addr learned my addr 172.21.15.43:0/222240009 (peer_addr_for_me v2:172.21.15.43:0/0) 2023-08-09T13:55:30.075 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:30.072+0000 7f8ceaffd640 1 -- 172.21.15.43:0/222240009 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f8cec15c700 msgr2=0x7f8cec157c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:55:30.075 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:30.072+0000 7f8ceaffd640 1 --2- 172.21.15.43:0/222240009 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f8cec15c700 0x7f8cec157c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:55:30.075 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:30.072+0000 7f8ceaffd640 1 -- 172.21.15.43:0/222240009 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f8cec15c320 msgr2=0x7f8cec0c7250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:55:30.076 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:30.072+0000 7f8ceaffd640 1 --2- 172.21.15.43:0/222240009 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f8cec15c320 0x7f8cec0c7250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:55:30.076 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:30.072+0000 7f8ceaffd640 1 -- 172.21.15.43:0/222240009 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8ce0009400 con 0x7f8cec0b0980 2023-08-09T13:55:30.077 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:30.072+0000 7f8ceaffd640 1 --2- 172.21.15.43:0/222240009 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f8cec0b0980 0x7f8cec1f16c0 secure :-1 s=READY pgs=1689 cs=0 l=1 rev1=1 crypto rx=0x7f8cdc007ef0 tx=0x7f8cdc00e4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-09T13:55:30.077 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:30.072+0000 7f8cd3fff640 1 -- 172.21.15.43:0/222240009 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8cdc014620 con 0x7f8cec0b0980 2023-08-09T13:55:30.077 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:30.072+0000 7f8cd3fff640 1 -- 172.21.15.43:0/222240009 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8cdc014780 con 0x7f8cec0b0980 2023-08-09T13:55:30.077 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:30.072+0000 7f8cd3fff640 1 -- 172.21.15.43:0/222240009 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8cdc014b30 con 0x7f8cec0b0980 2023-08-09T13:55:30.077 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:30.072+0000 7f8cebfff640 1 -- 172.21.15.43:0/222240009 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8cec1588a0 con 0x7f8cec0b0980 2023-08-09T13:55:30.077 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:30.072+0000 7f8cebfff640 1 -- 172.21.15.43:0/222240009 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8cec15e670 con 0x7f8cec0b0980 2023-08-09T13:55:30.078 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:30.074+0000 7f8cebfff640 1 -- 172.21.15.43:0/222240009 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8cec0739f0 con 0x7f8cec0b0980 2023-08-09T13:55:30.078 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:30.075+0000 7f8cd3fff640 1 -- 172.21.15.43:0/222240009 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 67) v1 ==== 88443+0+0 (secure 0 0 0) 0x7f8cdc014c90 con 0x7f8cec0b0980 2023-08-09T13:55:30.078 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:30.076+0000 7f8cd3fff640 1 --2- 172.21.15.43:0/222240009 >> [v2:172.21.15.203:6832/3490266242,v1:172.21.15.203:6833/3490266242] conn(0x7f8cd806a170 0x7f8cd806c630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:55:30.078 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:30.076+0000 7f8cd3fff640 1 -- 172.21.15.43:0/222240009 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(146..146 src has 1..146) v4 ==== 6426+0+0 (secure 0 0 0) 0x7f8cdc089390 con 0x7f8cec0b0980 2023-08-09T13:55:30.079 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:30.077+0000 7f8cea7fc640 1 --2- 172.21.15.43:0/222240009 >> [v2:172.21.15.203:6832/3490266242,v1:172.21.15.203:6833/3490266242] conn(0x7f8cd806a170 0x7f8cd806c630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:55:30.080 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:30.078+0000 7f8cea7fc640 1 --2- 172.21.15.43:0/222240009 >> [v2:172.21.15.203:6832/3490266242,v1:172.21.15.203:6833/3490266242] conn(0x7f8cd806a170 0x7f8cd806c630 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f8ce0036000 tx=0x7f8ce0030000 comp rx=0 tx=0).ready entity=mgr.6659 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-09T13:55:30.082 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:30.080+0000 7f8cd3fff640 1 -- 172.21.15.43:0/222240009 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178066 (secure 0 0 0) 0x7f8cdc054760 con 0x7f8cec0b0980 2023-08-09T13:55:30.282 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:30.280+0000 7f8cebfff640 1 -- 172.21.15.43:0/222240009 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f8cec15e800 con 0x7f8cec0b0980 2023-08-09T13:55:30.284 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:30.282+0000 7f8cd3fff640 1 -- 172.21.15.43:0/222240009 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v146) v1 ==== 74+0+18579 (secure 0 0 0) 0x7f8cdc057d80 con 0x7f8cec0b0980 2023-08-09T13:55:30.285 INFO:teuthology.orchestra.run.smithi043.stdout: 2023-08-09T13:55:30.285 INFO:teuthology.orchestra.run.smithi043.stdout:{"epoch":146,"fsid":"daac87ee-1337-4250-a955-aea4e408a39b","created":"2023-08-09T13:33:41.701198+0000","modified":"2023-08-09T13:55:28.793575+0000","last_up_change":"2023-08-09T13:33:53.783453+0000","last_in_change":"2023-08-09T13:33:45.148904+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":19,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":"rbd","create_time":"2023-08-09T13:33:54.365137+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":".mgr","create_time":"2023-08-09T13:33:54.831803+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":18,"pool_name":"cephfs_metadata","create_time":"2023-08-09T13:55:27.207507+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"146","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":19,"pool_name":"cephfs_data","create_time":"2023-08-09T13:55:27.825578+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"146","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"cbd49ecd-e896-4fbd-864e-5c37a829b165","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":145,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6800","nonce":502667335},{"type":"v1","addr":"172.21.15.43:6801","nonce":502667335}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6802","nonce":502667335},{"type":"v1","addr":"172.21.15.43:6803","nonce":502667335}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6806","nonce":502667335},{"type":"v1","addr":"172.21.15.43:6807","nonce":502667335}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6804","nonce":502667335},{"type":"v1","addr":"172.21.15.43:6805","nonce":502667335}]},"public_addr":"172.21.15.43:6801/502667335","cluster_addr":"172.21.15.43:6803/502667335","heartbeat_back_addr":"172.21.15.43:6807/502667335","heartbeat_front_addr":"172.21.15.43:6805/502667335","state":["exists","up"]},{"osd":1,"uuid":"75c7e0fe-1ec7-4fe8-a2f2-3661a65b8500","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":145,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6808","nonce":3289134610},{"type":"v1","addr":"172.21.15.43:6809","nonce":3289134610}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6810","nonce":3289134610},{"type":"v1","addr":"172.21.15.43:6812","nonce":3289134610}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6821","nonce":3289134610},{"type":"v1","addr":"172.21.15.43:6824","nonce":3289134610}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6815","nonce":3289134610},{"type":"v1","addr":"172.21.15.43:6818","nonce":3289134610}]},"public_addr":"172.21.15.43:6809/3289134610","cluster_addr":"172.21.15.43:6812/3289134610","heartbeat_back_addr":"172.21.15.43:6824/3289134610","heartbeat_front_addr":"172.21.15.43:6818/3289134610","state":["exists","up"]},{"osd":2,"uuid":"bb5c41e9-7f5f-4125-9b01-3bb8bfc2ea83","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":145,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6811","nonce":4079709736},{"type":"v1","addr":"172.21.15.43:6814","nonce":4079709736}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6817","nonce":4079709736},{"type":"v1","addr":"172.21.15.43:6820","nonce":4079709736}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6829","nonce":4079709736},{"type":"v1","addr":"172.21.15.43:6831","nonce":4079709736}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6823","nonce":4079709736},{"type":"v1","addr":"172.21.15.43:6826","nonce":4079709736}]},"public_addr":"172.21.15.43:6814/4079709736","cluster_addr":"172.21.15.43:6820/4079709736","heartbeat_back_addr":"172.21.15.43:6831/4079709736","heartbeat_front_addr":"172.21.15.43:6826/4079709736","state":["exists","up"]},{"osd":3,"uuid":"bfe4c8b4-694a-465c-87e6-2b2aab852613","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":145,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6813","nonce":1709538759},{"type":"v1","addr":"172.21.15.43:6816","nonce":1709538759}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6819","nonce":1709538759},{"type":"v1","addr":"172.21.15.43:6822","nonce":1709538759}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6828","nonce":1709538759},{"type":"v1","addr":"172.21.15.43:6830","nonce":1709538759}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6825","nonce":1709538759},{"type":"v1","addr":"172.21.15.43:6827","nonce":1709538759}]},"public_addr":"172.21.15.43:6816/1709538759","cluster_addr":"172.21.15.43:6822/1709538759","heartbeat_back_addr":"172.21.15.43:6830/1709538759","heartbeat_front_addr":"172.21.15.43:6827/1709538759","state":["exists","up"]},{"osd":4,"uuid":"fa2e8a77-aba1-4756-853e-0d174a5bdeba","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":145,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6800","nonce":4196866481},{"type":"v1","addr":"172.21.15.203:6801","nonce":4196866481}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6802","nonce":4196866481},{"type":"v1","addr":"172.21.15.203:6803","nonce":4196866481}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6806","nonce":4196866481},{"type":"v1","addr":"172.21.15.203:6807","nonce":4196866481}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6804","nonce":4196866481},{"type":"v1","addr":"172.21.15.203:6805","nonce":4196866481}]},"public_addr":"172.21.15.203:6801/4196866481","cluster_addr":"172.21.15.203:6803/4196866481","heartbeat_back_addr":"172.21.15.203:6807/4196866481","heartbeat_front_addr":"172.21.15.203:6805/4196866481","state":["exists","up"]},{"osd":5,"uuid":"81811e4b-4d24-4f51-a82c-f1d700a2a00d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":145,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6816","nonce":3059290081},{"type":"v1","addr":"172.21.15.203:6817","nonce":3059290081}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6818","nonce":3059290081},{"type":"v1","addr":"172.21.15.203:6819","nonce":3059290081}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6822","nonce":3059290081},{"type":"v1","addr":"172.21.15.203:6823","nonce":3059290081}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6820","nonce":3059290081},{"type":"v1","addr":"172.21.15.203:6821","nonce":3059290081}]},"public_addr":"172.21.15.203:6817/3059290081","cluster_addr":"172.21.15.203:6819/3059290081","heartbeat_back_addr":"172.21.15.203:6823/3059290081","heartbeat_front_addr":"172.21.15.203:6821/3059290081","state":["exists","up"]},{"osd":6,"uuid":"307bac4d-6a5b-443d-af75-56d695d7674f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":145,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6824","nonce":512155569},{"type":"v1","addr":"172.21.15.203:6825","nonce":512155569}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6826","nonce":512155569},{"type":"v1","addr":"172.21.15.203:6827","nonce":512155569}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6830","nonce":512155569},{"type":"v1","addr":"172.21.15.203:6831","nonce":512155569}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6828","nonce":512155569},{"type":"v1","addr":"172.21.15.203:6829","nonce":512155569}]},"public_addr":"172.21.15.203:6825/512155569","cluster_addr":"172.21.15.203:6827/512155569","heartbeat_back_addr":"172.21.15.203:6831/512155569","heartbeat_front_addr":"172.21.15.203:6829/512155569","state":["exists","up"]},{"osd":7,"uuid":"a95dc7ca-9f1b-4295-af70-68a22a2d7d5e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":145,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6808","nonce":3331450438},{"type":"v1","addr":"172.21.15.203:6809","nonce":3331450438}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6810","nonce":3331450438},{"type":"v1","addr":"172.21.15.203:6811","nonce":3331450438}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6814","nonce":3331450438},{"type":"v1","addr":"172.21.15.203:6815","nonce":3331450438}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6812","nonce":3331450438},{"type":"v1","addr":"172.21.15.203:6813","nonce":3331450438}]},"public_addr":"172.21.15.203:6809/3331450438","cluster_addr":"172.21.15.203:6811/3331450438","heartbeat_back_addr":"172.21.15.203:6815/3331450438","heartbeat_front_addr":"172.21.15.203:6813/3331450438","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-09T13:33:50.547652+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-09T13:33:50.589469+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-09T13:33:50.561724+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-09T13:33:50.599650+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-09T13:33:50.411855+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-09T13:33:50.219681+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-09T13:33:50.504664+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-09T13:33:50.441248+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":7,"snaps":[{"begin":104,"length":1},{"begin":221,"length":1}]}],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-08-09T13:55:30.288 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:30.285+0000 7f8cebfff640 1 -- 172.21.15.43:0/222240009 >> [v2:172.21.15.203:6832/3490266242,v1:172.21.15.203:6833/3490266242] conn(0x7f8cd806a170 msgr2=0x7f8cd806c630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:55:30.288 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:30.285+0000 7f8cebfff640 1 --2- 172.21.15.43:0/222240009 >> [v2:172.21.15.203:6832/3490266242,v1:172.21.15.203:6833/3490266242] conn(0x7f8cd806a170 0x7f8cd806c630 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f8ce0036000 tx=0x7f8ce0030000 comp rx=0 tx=0).stop 2023-08-09T13:55:30.288 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:30.285+0000 7f8cebfff640 1 -- 172.21.15.43:0/222240009 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f8cec0b0980 msgr2=0x7f8cec1f16c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:55:30.289 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:30.285+0000 7f8cebfff640 1 --2- 172.21.15.43:0/222240009 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f8cec0b0980 0x7f8cec1f16c0 secure :-1 s=READY pgs=1689 cs=0 l=1 rev1=1 crypto rx=0x7f8cdc007ef0 tx=0x7f8cdc00e4b0 comp rx=0 tx=0).stop 2023-08-09T13:55:30.289 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:30.286+0000 7f8cebfff640 1 -- 172.21.15.43:0/222240009 shutdown_connections 2023-08-09T13:55:30.289 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:30.286+0000 7f8cebfff640 1 --2- 172.21.15.43:0/222240009 >> [v2:172.21.15.203:6832/3490266242,v1:172.21.15.203:6833/3490266242] conn(0x7f8cd806a170 0x7f8cd806c630 unknown :-1 s=CLOSED pgs=59 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:55:30.289 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:30.286+0000 7f8cebfff640 1 --2- 172.21.15.43:0/222240009 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f8cec15c700 0x7f8cec157c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:55:30.289 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:30.286+0000 7f8cebfff640 1 --2- 172.21.15.43:0/222240009 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f8cec15c320 0x7f8cec0c7250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:55:30.289 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:30.286+0000 7f8cebfff640 1 --2- 172.21.15.43:0/222240009 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f8cec0b0980 0x7f8cec1f16c0 unknown :-1 s=CLOSED pgs=1689 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:55:30.289 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:30.286+0000 7f8cebfff640 1 -- 172.21.15.43:0/222240009 >> 172.21.15.43:0/222240009 conn(0x7f8cec089fe0 msgr2=0x7f8cec15b6d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-09T13:55:30.289 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:30.286+0000 7f8cebfff640 1 -- 172.21.15.43:0/222240009 shutdown_connections 2023-08-09T13:55:30.290 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:30.286+0000 7f8cebfff640 1 -- 172.21.15.43:0/222240009 wait complete. 2023-08-09T13:55:30.306 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs standby_count_wanted 0 2023-08-09T13:55:30.520 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:30.518+0000 7f8d4868e640 1 Processor -- start 2023-08-09T13:55:30.521 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:30.518+0000 7f8d4868e640 1 -- start start 2023-08-09T13:55:30.522 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:30.519+0000 7f8d4868e640 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7f8d400d0a70 0x7f8d400cd7f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:55:30.522 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:30.519+0000 7f8d4868e640 1 -- --> v1:172.21.15.43:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8d400d0f70 con 0x7f8d400cdd30 2023-08-09T13:55:30.522 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:30.519+0000 7f8d4868e640 1 -- --> v1:172.21.15.203:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8d400ccd10 con 0x7f8d4015c680 2023-08-09T13:55:30.522 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:30.519+0000 7f8d4868e640 1 -- --> v2:172.21.15.203:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8d400ccff0 con 0x7f8d400d0a70 2023-08-09T13:55:30.523 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:30.520+0000 7f8d46c2b640 1 --1- >> v1:172.21.15.43:6789/0 conn(0x7f8d400cdd30 0x7f8d400b3fa0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.43:6789/0 says I am v1:172.21.15.43:56636/0 (socket says 172.21.15.43:56636) 2023-08-09T13:55:30.523 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:30.520+0000 7f8d46c2b640 1 -- 172.21.15.43:0/2226405749 learned_addr learned my addr 172.21.15.43:0/2226405749 (peer_addr_for_me v1:172.21.15.43:0/0) 2023-08-09T13:55:30.523 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:30.520+0000 7f8d45c29640 1 --2- 172.21.15.43:0/2226405749 >> v2:172.21.15.203:3300/0 conn(0x7f8d400d0a70 0x7f8d400cd7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:55:30.523 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:30.520+0000 7f8d45428640 1 -- 172.21.15.43:0/2226405749 <== mon.0 v1:172.21.15.43:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2028262496 0 0) 0x7f8d400d0f70 con 0x7f8d400cdd30 2023-08-09T13:55:30.523 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:30.520+0000 7f8d45428640 1 -- 172.21.15.43:0/2226405749 --> v1:172.21.15.43:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f8d2c003520 con 0x7f8d400cdd30 2023-08-09T13:55:30.524 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:30.521+0000 7f8d45428640 1 -- 172.21.15.43:0/2226405749 <== mon.0 v1:172.21.15.43:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 803374147 0 0) 0x7f8d2c003520 con 0x7f8d400cdd30 2023-08-09T13:55:30.524 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:30.521+0000 7f8d45428640 1 -- 172.21.15.43:0/2226405749 >> v2:172.21.15.203:3300/0 conn(0x7f8d400d0a70 msgr2=0x7f8d400cd7f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-09T13:55:30.524 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:30.521+0000 7f8d45428640 1 --2- 172.21.15.43:0/2226405749 >> v2:172.21.15.203:3300/0 conn(0x7f8d400d0a70 0x7f8d400cd7f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:55:30.525 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:30.521+0000 7f8d45428640 1 -- 172.21.15.43:0/2226405749 >> v1:172.21.15.203:6789/0 conn(0x7f8d4015c680 legacy=0x7f8d400ce760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:55:30.525 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:30.521+0000 7f8d45428640 1 -- 172.21.15.43:0/2226405749 --> v1:172.21.15.43:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8d400cd2d0 con 0x7f8d400cdd30 2023-08-09T13:55:30.525 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:30.521+0000 7f8d45428640 1 -- 172.21.15.43:0/2226405749 <== mon.0 v1:172.21.15.43:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1285150740 0 0) 0x7f8d38001670 con 0x7f8d400cdd30 2023-08-09T13:55:30.525 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:30.521+0000 7f8d45428640 1 -- 172.21.15.43:0/2226405749 >> v1:172.21.15.43:6789/0 conn(0x7f8d400cdd30 legacy=0x7f8d400b3fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:55:30.525 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:30.521+0000 7f8d45428640 1 --2- 172.21.15.43:0/2226405749 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f8d2c0039e0 0x7f8d2c003e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:55:30.525 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:30.521+0000 7f8d45428640 1 --2- 172.21.15.43:0/2226405749 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f8d2c0047c0 0x7f8d2c004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:55:30.525 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:30.521+0000 7f8d45428640 1 --2- 172.21.15.43:0/2226405749 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f8d2c005180 0x7f8d2c00b980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:55:30.526 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:30.521+0000 7f8d45428640 1 -- 172.21.15.43:0/2226405749 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8d400ccff0 con 0x7f8d2c0047c0 2023-08-09T13:55:30.526 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:30.521+0000 7f8d45428640 1 -- 172.21.15.43:0/2226405749 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8d2c00c0b0 con 0x7f8d2c0039e0 2023-08-09T13:55:30.526 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:30.521+0000 7f8d45428640 1 -- 172.21.15.43:0/2226405749 --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8d2c00c270 con 0x7f8d2c005180 2023-08-09T13:55:30.526 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:30.521+0000 7f8d45c29640 1 --2- 172.21.15.43:0/2226405749 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f8d2c0047c0 0x7f8d2c004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:55:30.526 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:30.522+0000 7f8d45c29640 1 -- 172.21.15.43:0/2226405749 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f8d2c005180 msgr2=0x7f8d2c00b980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-09T13:55:30.526 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:30.522+0000 7f8d45c29640 1 --2- 172.21.15.43:0/2226405749 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f8d2c005180 0x7f8d2c00b980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:55:30.526 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:30.522+0000 7f8d45c29640 1 -- 172.21.15.43:0/2226405749 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f8d2c0039e0 msgr2=0x7f8d2c003e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-09T13:55:30.526 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:30.522+0000 7f8d45c29640 1 --2- 172.21.15.43:0/2226405749 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f8d2c0039e0 0x7f8d2c003e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:55:30.527 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:30.522+0000 7f8d45c29640 1 -- 172.21.15.43:0/2226405749 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8d2c003520 con 0x7f8d2c0047c0 2023-08-09T13:55:30.527 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:30.522+0000 7f8d46c2b640 1 -- 172.21.15.43:0/2226405749 reap_dead start 2023-08-09T13:55:30.528 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:30.523+0000 7f8d45c29640 1 --2- 172.21.15.43:0/2226405749 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f8d2c0047c0 0x7f8d2c004c40 secure :-1 s=READY pgs=1691 cs=0 l=1 rev1=1 crypto rx=0x7f8d30009a90 tx=0x7f8d2c0057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=40bf1f8d04658ec5 server_cookie=0 in_seq=0 out_seq=0 2023-08-09T13:55:30.528 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:30.523+0000 7f8d45428640 1 -- 172.21.15.43:0/2226405749 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8d3002c680 con 0x7f8d2c0047c0 2023-08-09T13:55:30.528 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:30.523+0000 7f8d45428640 1 -- 172.21.15.43:0/2226405749 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8d3002c7e0 con 0x7f8d2c0047c0 2023-08-09T13:55:30.528 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:30.523+0000 7f8d45428640 1 -- 172.21.15.43:0/2226405749 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8d3002cb70 con 0x7f8d2c0047c0 2023-08-09T13:55:30.528 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:30.524+0000 7f8d4868e640 1 -- 172.21.15.43:0/2226405749 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f8d2c0047c0 msgr2=0x7f8d2c004c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:55:30.528 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:30.524+0000 7f8d4868e640 1 --2- 172.21.15.43:0/2226405749 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f8d2c0047c0 0x7f8d2c004c40 secure :-1 s=READY pgs=1691 cs=0 l=1 rev1=1 crypto rx=0x7f8d30009a90 tx=0x7f8d2c0057f0 comp rx=0 tx=0).stop 2023-08-09T13:55:30.528 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:30.524+0000 7f8d4868e640 1 -- 172.21.15.43:0/2226405749 shutdown_connections 2023-08-09T13:55:30.529 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:30.524+0000 7f8d4868e640 1 --2- 172.21.15.43:0/2226405749 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f8d2c0047c0 0x7f8d2c004c40 unknown :-1 s=CLOSED pgs=1691 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:55:30.529 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:30.524+0000 7f8d4868e640 1 -- 172.21.15.43:0/2226405749 >> 172.21.15.43:0/2226405749 conn(0x7f8d40071520 msgr2=0x7f8d400d2520 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-09T13:55:30.529 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:30.524+0000 7f8d4868e640 1 -- 172.21.15.43:0/2226405749 shutdown_connections 2023-08-09T13:55:30.529 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:30.524+0000 7f8d4868e640 1 -- 172.21.15.43:0/2226405749 wait complete. 2023-08-09T13:55:30.530 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:30.526+0000 7f8d4868e640 1 Processor -- start 2023-08-09T13:55:30.530 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:30.526+0000 7f8d4868e640 1 -- start start 2023-08-09T13:55:30.530 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:30.527+0000 7f8d4868e640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f8d2c0047c0 0x7f8d400cbff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:55:30.531 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:30.527+0000 7f8d4868e640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f8d400ca000 0x7f8d40155ec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:55:30.531 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:30.527+0000 7f8d4868e640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f8d401f57f0 0x7f8d401f5c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:55:30.531 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:30.527+0000 7f8d4868e640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8d4014e240 con 0x7f8d2c0047c0 2023-08-09T13:55:30.531 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:30.527+0000 7f8d4868e640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8d40156400 con 0x7f8d400ca000 2023-08-09T13:55:30.531 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:30.527+0000 7f8d4868e640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8d401f61b0 con 0x7f8d401f57f0 2023-08-09T13:55:30.531 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:30.527+0000 7f8d4642a640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f8d2c0047c0 0x7f8d400cbff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:55:30.531 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:30.527+0000 7f8d4642a640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f8d2c0047c0 0x7f8d400cbff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:52582/0 (socket says 172.21.15.43:52582) 2023-08-09T13:55:30.532 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:30.527+0000 7f8d4642a640 1 -- 172.21.15.43:0/1570980422 learned_addr learned my addr 172.21.15.43:0/1570980422 (peer_addr_for_me v2:172.21.15.43:0/0) 2023-08-09T13:55:30.532 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:30.528+0000 7f8d45c29640 1 --2- 172.21.15.43:0/1570980422 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f8d400ca000 0x7f8d40155ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:55:30.532 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:30.528+0000 7f8d4642a640 1 -- 172.21.15.43:0/1570980422 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f8d401f57f0 msgr2=0x7f8d401f5c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:55:30.532 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:30.528+0000 7f8d4642a640 1 --2- 172.21.15.43:0/1570980422 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f8d401f57f0 0x7f8d401f5c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:55:30.532 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:30.528+0000 7f8d4642a640 1 -- 172.21.15.43:0/1570980422 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f8d400ca000 msgr2=0x7f8d40155ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:55:30.532 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:30.528+0000 7f8d4642a640 1 --2- 172.21.15.43:0/1570980422 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f8d400ca000 0x7f8d40155ec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:55:30.532 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:30.528+0000 7f8d4642a640 1 -- 172.21.15.43:0/1570980422 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8d4015dd80 con 0x7f8d2c0047c0 2023-08-09T13:55:30.532 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:30.528+0000 7f8d4642a640 1 --2- 172.21.15.43:0/1570980422 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f8d2c0047c0 0x7f8d400cbff0 secure :-1 s=READY pgs=1692 cs=0 l=1 rev1=1 crypto rx=0x7f8d3c00c710 tx=0x7f8d3c00cbd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-09T13:55:30.533 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:30.528+0000 7f8d36ffd640 1 -- 172.21.15.43:0/1570980422 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8d3c006d50 con 0x7f8d2c0047c0 2023-08-09T13:55:30.533 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:30.528+0000 7f8d36ffd640 1 -- 172.21.15.43:0/1570980422 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8d3c00ce20 con 0x7f8d2c0047c0 2023-08-09T13:55:30.534 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:30.528+0000 7f8d36ffd640 1 -- 172.21.15.43:0/1570980422 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8d3c003bb0 con 0x7f8d2c0047c0 2023-08-09T13:55:30.534 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:30.528+0000 7f8d4868e640 1 -- 172.21.15.43:0/1570980422 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8d400cafa0 con 0x7f8d2c0047c0 2023-08-09T13:55:30.534 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:30.529+0000 7f8d4868e640 1 -- 172.21.15.43:0/1570980422 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8d400ca700 con 0x7f8d2c0047c0 2023-08-09T13:55:30.534 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:30.530+0000 7f8d4868e640 1 -- 172.21.15.43:0/1570980422 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8d400cd510 con 0x7f8d2c0047c0 2023-08-09T13:55:30.534 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:30.531+0000 7f8d36ffd640 1 -- 172.21.15.43:0/1570980422 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 67) v1 ==== 88443+0+0 (secure 0 0 0) 0x7f8d3c003d10 con 0x7f8d2c0047c0 2023-08-09T13:55:30.535 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:30.532+0000 7f8d36ffd640 1 --2- 172.21.15.43:0/1570980422 >> [v2:172.21.15.203:6832/3490266242,v1:172.21.15.203:6833/3490266242] conn(0x7f8d2406a170 0x7f8d2406c630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:55:30.535 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:30.532+0000 7f8d36ffd640 1 -- 172.21.15.43:0/1570980422 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(147..147 src has 1..147) v4 ==== 6426+0+0 (secure 0 0 0) 0x7f8d3c088370 con 0x7f8d2c0047c0 2023-08-09T13:55:30.535 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:30.532+0000 7f8d45c29640 1 --2- 172.21.15.43:0/1570980422 >> [v2:172.21.15.203:6832/3490266242,v1:172.21.15.203:6833/3490266242] conn(0x7f8d2406a170 0x7f8d2406c630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:55:30.536 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:30.534+0000 7f8d45c29640 1 --2- 172.21.15.43:0/1570980422 >> [v2:172.21.15.203:6832/3490266242,v1:172.21.15.203:6833/3490266242] conn(0x7f8d2406a170 0x7f8d2406c630 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f8d30036000 tx=0x7f8d30030000 comp rx=0 tx=0).ready entity=mgr.6659 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-09T13:55:30.538 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:30.536+0000 7f8d36ffd640 1 -- 172.21.15.43:0/1570980422 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178066 (secure 0 0 0) 0x7f8d3c053870 con 0x7f8d2c0047c0 2023-08-09T13:55:30.772 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:30.770+0000 7f8d4868e640 1 -- 172.21.15.43:0/1570980422 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} v 0) v1 -- 0x7f8d400739f0 con 0x7f8d2c0047c0 2023-08-09T13:55:31.393 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:31.390+0000 7f8d36ffd640 1 -- 172.21.15.43:0/1570980422 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]=0 v69) v1 ==== 118+0+0 (secure 0 0 0) 0x7f8d3c056e90 con 0x7f8d2c0047c0 2023-08-09T13:55:31.397 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:31.394+0000 7f8d4868e640 1 -- 172.21.15.43:0/1570980422 >> [v2:172.21.15.203:6832/3490266242,v1:172.21.15.203:6833/3490266242] conn(0x7f8d2406a170 msgr2=0x7f8d2406c630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:55:31.397 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:31.394+0000 7f8d4868e640 1 --2- 172.21.15.43:0/1570980422 >> [v2:172.21.15.203:6832/3490266242,v1:172.21.15.203:6833/3490266242] conn(0x7f8d2406a170 0x7f8d2406c630 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f8d30036000 tx=0x7f8d30030000 comp rx=0 tx=0).stop 2023-08-09T13:55:31.397 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:31.394+0000 7f8d4868e640 1 -- 172.21.15.43:0/1570980422 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f8d2c0047c0 msgr2=0x7f8d400cbff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:55:31.397 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:31.394+0000 7f8d4868e640 1 --2- 172.21.15.43:0/1570980422 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f8d2c0047c0 0x7f8d400cbff0 secure :-1 s=READY pgs=1692 cs=0 l=1 rev1=1 crypto rx=0x7f8d3c00c710 tx=0x7f8d3c00cbd0 comp rx=0 tx=0).stop 2023-08-09T13:55:31.397 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:31.394+0000 7f8d4868e640 1 -- 172.21.15.43:0/1570980422 shutdown_connections 2023-08-09T13:55:31.397 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:31.394+0000 7f8d4868e640 1 --2- 172.21.15.43:0/1570980422 >> [v2:172.21.15.203:6832/3490266242,v1:172.21.15.203:6833/3490266242] conn(0x7f8d2406a170 0x7f8d2406c630 unknown :-1 s=CLOSED pgs=60 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:55:31.398 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:31.394+0000 7f8d4868e640 1 --2- 172.21.15.43:0/1570980422 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f8d401f57f0 0x7f8d401f5c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:55:31.398 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:31.394+0000 7f8d4868e640 1 --2- 172.21.15.43:0/1570980422 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f8d400ca000 0x7f8d40155ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:55:31.398 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:31.394+0000 7f8d4868e640 1 --2- 172.21.15.43:0/1570980422 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f8d2c0047c0 0x7f8d400cbff0 unknown :-1 s=CLOSED pgs=1692 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:55:31.398 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:31.394+0000 7f8d4868e640 1 -- 172.21.15.43:0/1570980422 >> 172.21.15.43:0/1570980422 conn(0x7f8d40071520 msgr2=0x7f8d40156ab0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-09T13:55:31.398 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:31.394+0000 7f8d4868e640 1 -- 172.21.15.43:0/1570980422 shutdown_connections 2023-08-09T13:55:31.398 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:31.395+0000 7f8d4868e640 1 -- 172.21.15.43:0/1570980422 wait complete. 2023-08-09T13:55:31.418 DEBUG:tasks.cephfs.filesystem:fs_config: {} 2023-08-09T13:55:31.418 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs allow_standby_replay false 2023-08-09T13:55:31.649 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:31.646+0000 7fe0216c5640 1 Processor -- start 2023-08-09T13:55:31.650 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:31.647+0000 7fe0216c5640 1 -- start start 2023-08-09T13:55:31.651 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:31.648+0000 7fe0216c5640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fe01c0ccf20 0x7fe01c0cd550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:55:31.651 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:31.648+0000 7fe0216c5640 1 -- --> v1:172.21.15.43:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe01c0cca90 con 0x7fe01c15c630 2023-08-09T13:55:31.651 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:31.648+0000 7fe0216c5640 1 -- --> v1:172.21.15.203:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe01c0cbe80 con 0x7fe01c0cda90 2023-08-09T13:55:31.651 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:31.649+0000 7fe0216c5640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe01c0ac600 con 0x7fe01c0ccf20 2023-08-09T13:55:31.652 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:31.649+0000 7fe01affd640 1 --1- >> v1:172.21.15.43:6789/0 conn(0x7fe01c15c630 0x7fe01c0abef0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.43:6789/0 says I am v1:172.21.15.43:56646/0 (socket says 172.21.15.43:56646) 2023-08-09T13:55:31.653 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:31.649+0000 7fe01affd640 1 -- 172.21.15.43:0/1467427078 learned_addr learned my addr 172.21.15.43:0/1467427078 (peer_addr_for_me v1:172.21.15.43:0/0) 2023-08-09T13:55:31.653 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:31.649+0000 7fe01a7fc640 1 --2- 172.21.15.43:0/1467427078 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fe01c0ccf20 0x7fe01c0cd550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:55:31.653 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:31.649+0000 7fe019ffb640 1 -- 172.21.15.43:0/1467427078 <== mon.0 v1:172.21.15.43:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1607806756 0 0) 0x7fe01c0cca90 con 0x7fe01c15c630 2023-08-09T13:55:31.653 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:31.649+0000 7fe019ffb640 1 -- 172.21.15.43:0/1467427078 --> v1:172.21.15.43:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fe004003520 con 0x7fe01c15c630 2023-08-09T13:55:31.653 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:31.650+0000 7fe019ffb640 1 -- 172.21.15.43:0/1467427078 <== mon.0 v1:172.21.15.43:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 257243331 0 0) 0x7fe004003520 con 0x7fe01c15c630 2023-08-09T13:55:31.653 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:31.650+0000 7fe019ffb640 1 -- 172.21.15.43:0/1467427078 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fe01c0ccf20 msgr2=0x7fe01c0cd550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-09T13:55:31.653 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:31.650+0000 7fe019ffb640 1 --2- 172.21.15.43:0/1467427078 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fe01c0ccf20 0x7fe01c0cd550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:55:31.654 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:31.650+0000 7fe019ffb640 1 -- 172.21.15.43:0/1467427078 >> v1:172.21.15.203:6789/0 conn(0x7fe01c0cda90 legacy=0x7fe01c15ca10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:55:31.654 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:31.650+0000 7fe019ffb640 1 -- 172.21.15.43:0/1467427078 --> v1:172.21.15.43:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe01c0cc160 con 0x7fe01c15c630 2023-08-09T13:55:31.654 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:31.650+0000 7fe019ffb640 1 -- 172.21.15.43:0/1467427078 <== mon.0 v1:172.21.15.43:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1285150740 0 0) 0x7fe00c001670 con 0x7fe01c15c630 2023-08-09T13:55:31.654 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:31.650+0000 7fe019ffb640 1 -- 172.21.15.43:0/1467427078 >> v1:172.21.15.43:6789/0 conn(0x7fe01c15c630 legacy=0x7fe01c0abef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:55:31.654 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:31.650+0000 7fe019ffb640 1 --2- 172.21.15.43:0/1467427078 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fe004003a10 0x7fe004003e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:55:31.654 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:31.650+0000 7fe019ffb640 1 --2- 172.21.15.43:0/1467427078 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fe0040047c0 0x7fe004004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:55:31.654 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:31.650+0000 7fe019ffb640 1 --2- 172.21.15.43:0/1467427078 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fe01c0ccf20 0x7fe004005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:55:31.654 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:31.650+0000 7fe019ffb640 1 -- 172.21.15.43:0/1467427078 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe01c0ac600 con 0x7fe004003a10 2023-08-09T13:55:31.655 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:31.650+0000 7fe019ffb640 1 -- 172.21.15.43:0/1467427078 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe00400ba80 con 0x7fe0040047c0 2023-08-09T13:55:31.655 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:31.650+0000 7fe019ffb640 1 -- 172.21.15.43:0/1467427078 --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe00400bc40 con 0x7fe01c0ccf20 2023-08-09T13:55:31.655 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:31.650+0000 7fe01affd640 1 --2- 172.21.15.43:0/1467427078 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fe004003a10 0x7fe004003e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:55:31.658 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:31.651+0000 7fe01b7fe640 1 --2- 172.21.15.43:0/1467427078 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fe01c0ccf20 0x7fe004005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:55:31.658 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:31.651+0000 7fe01a7fc640 1 --2- 172.21.15.43:0/1467427078 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fe0040047c0 0x7fe004004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:55:31.658 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:31.651+0000 7fe01affd640 1 -- 172.21.15.43:0/1467427078 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fe01c0ccf20 msgr2=0x7fe004005180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-09T13:55:31.658 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:31.651+0000 7fe01affd640 1 --2- 172.21.15.43:0/1467427078 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fe01c0ccf20 0x7fe004005180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:55:31.659 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:31.651+0000 7fe01affd640 1 -- 172.21.15.43:0/1467427078 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fe0040047c0 msgr2=0x7fe004004c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-09T13:55:31.659 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:31.651+0000 7fe01affd640 1 --2- 172.21.15.43:0/1467427078 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fe0040047c0 0x7fe004004c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:55:31.659 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:31.651+0000 7fe01affd640 1 -- 172.21.15.43:0/1467427078 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe004003520 con 0x7fe004003a10 2023-08-09T13:55:31.659 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:31.651+0000 7fe01a7fc640 1 --2- 172.21.15.43:0/1467427078 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fe0040047c0 0x7fe004004c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-08-09T13:55:31.659 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:31.652+0000 7fe01affd640 1 --2- 172.21.15.43:0/1467427078 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fe004003a10 0x7fe004003e30 secure :-1 s=READY pgs=1694 cs=0 l=1 rev1=1 crypto rx=0x7fe00c00c240 tx=0x7fe0040057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a04f28ca58c3da04 server_cookie=0 in_seq=0 out_seq=0 2023-08-09T13:55:31.659 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:31.653+0000 7fe019ffb640 1 -- 172.21.15.43:0/1467427078 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe00c005ad0 con 0x7fe004003a10 2023-08-09T13:55:31.659 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:31.653+0000 7fe019ffb640 1 -- 172.21.15.43:0/1467427078 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe00c005c50 con 0x7fe004003a10 2023-08-09T13:55:31.659 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:31.653+0000 7fe019ffb640 1 -- 172.21.15.43:0/1467427078 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe00c02c8b0 con 0x7fe004003a10 2023-08-09T13:55:31.660 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:31.653+0000 7fe0216c5640 1 -- 172.21.15.43:0/1467427078 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fe004003a10 msgr2=0x7fe004003e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:55:31.660 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:31.653+0000 7fe0216c5640 1 --2- 172.21.15.43:0/1467427078 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fe004003a10 0x7fe004003e30 secure :-1 s=READY pgs=1694 cs=0 l=1 rev1=1 crypto rx=0x7fe00c00c240 tx=0x7fe0040057f0 comp rx=0 tx=0).stop 2023-08-09T13:55:31.660 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:31.653+0000 7fe01b7fe640 1 -- 172.21.15.43:0/1467427078 reap_dead start 2023-08-09T13:55:31.660 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:31.653+0000 7fe0216c5640 1 -- 172.21.15.43:0/1467427078 shutdown_connections 2023-08-09T13:55:31.660 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:31.653+0000 7fe0216c5640 1 -- 172.21.15.43:0/1467427078 >> 172.21.15.43:0/1467427078 conn(0x7fe01c0ce0a0 msgr2=0x7fe01c0ce480 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-09T13:55:31.660 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:31.654+0000 7fe0216c5640 1 -- 172.21.15.43:0/1467427078 shutdown_connections 2023-08-09T13:55:31.660 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:31.654+0000 7fe0216c5640 1 -- 172.21.15.43:0/1467427078 wait complete. 2023-08-09T13:55:31.661 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:31.655+0000 7fe0216c5640 1 Processor -- start 2023-08-09T13:55:31.661 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:31.655+0000 7fe0216c5640 1 -- start start 2023-08-09T13:55:31.662 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:31.656+0000 7fe0216c5640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fe004003a10 0x7fe01c153db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:55:31.662 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:31.656+0000 7fe0216c5640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fe01c1f50c0 0x7fe01c158620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:55:31.663 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:31.656+0000 7fe0216c5640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fe01c15cbf0 0x7fe01c15d070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:55:31.663 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:31.656+0000 7fe0216c5640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe01c0c52a0 con 0x7fe01c1f50c0 2023-08-09T13:55:31.663 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:31.656+0000 7fe0216c5640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe01c158b60 con 0x7fe004003a10 2023-08-09T13:55:31.663 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:31.656+0000 7fe0216c5640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe01c1f5610 con 0x7fe01c15cbf0 2023-08-09T13:55:31.663 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:31.657+0000 7fe01a7fc640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fe01c1f50c0 0x7fe01c158620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:55:31.663 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:31.657+0000 7fe01a7fc640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fe01c1f50c0 0x7fe01c158620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:52594/0 (socket says 172.21.15.43:52594) 2023-08-09T13:55:31.663 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:31.657+0000 7fe01a7fc640 1 -- 172.21.15.43:0/2822439552 learned_addr learned my addr 172.21.15.43:0/2822439552 (peer_addr_for_me v2:172.21.15.43:0/0) 2023-08-09T13:55:31.664 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:31.657+0000 7fe01affd640 1 --2- 172.21.15.43:0/2822439552 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fe004003a10 0x7fe01c153db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:55:31.664 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:31.657+0000 7fe01b7fe640 1 --2- 172.21.15.43:0/2822439552 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fe01c15cbf0 0x7fe01c15d070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:55:31.664 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:31.657+0000 7fe01a7fc640 1 -- 172.21.15.43:0/2822439552 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fe01c15cbf0 msgr2=0x7fe01c15d070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:55:31.664 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:31.657+0000 7fe01a7fc640 1 --2- 172.21.15.43:0/2822439552 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fe01c15cbf0 0x7fe01c15d070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:55:31.664 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:31.657+0000 7fe01a7fc640 1 -- 172.21.15.43:0/2822439552 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fe004003a10 msgr2=0x7fe01c153db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:55:31.664 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:31.657+0000 7fe01a7fc640 1 --2- 172.21.15.43:0/2822439552 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fe004003a10 0x7fe01c153db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:55:31.664 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:31.657+0000 7fe01a7fc640 1 -- 172.21.15.43:0/2822439552 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe01c1f58f0 con 0x7fe01c1f50c0 2023-08-09T13:55:31.664 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:31.658+0000 7fe01a7fc640 1 --2- 172.21.15.43:0/2822439552 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fe01c1f50c0 0x7fe01c158620 secure :-1 s=READY pgs=1695 cs=0 l=1 rev1=1 crypto rx=0x7fe0100108b0 tx=0x7fe010010d70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-09T13:55:31.665 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:31.658+0000 7fe0037fe640 1 -- 172.21.15.43:0/2822439552 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe01000eb00 con 0x7fe01c1f50c0 2023-08-09T13:55:31.665 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:31.658+0000 7fe0037fe640 1 -- 172.21.15.43:0/2822439552 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe01000ec60 con 0x7fe01c1f50c0 2023-08-09T13:55:31.665 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:31.658+0000 7fe0037fe640 1 -- 172.21.15.43:0/2822439552 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe01001c400 con 0x7fe01c1f50c0 2023-08-09T13:55:31.665 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:31.658+0000 7fe0216c5640 1 -- 172.21.15.43:0/2822439552 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe01c1f5bd0 con 0x7fe01c1f50c0 2023-08-09T13:55:31.665 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:31.658+0000 7fe0216c5640 1 -- 172.21.15.43:0/2822439552 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe01c0c3fd0 con 0x7fe01c1f50c0 2023-08-09T13:55:31.665 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:31.660+0000 7fe0216c5640 1 -- 172.21.15.43:0/2822439552 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe01c0c53e0 con 0x7fe01c1f50c0 2023-08-09T13:55:31.666 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:31.661+0000 7fe0037fe640 1 -- 172.21.15.43:0/2822439552 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 67) v1 ==== 88443+0+0 (secure 0 0 0) 0x7fe01001c560 con 0x7fe01c1f50c0 2023-08-09T13:55:31.666 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:31.662+0000 7fe0037fe640 1 --2- 172.21.15.43:0/2822439552 >> [v2:172.21.15.203:6832/3490266242,v1:172.21.15.203:6833/3490266242] conn(0x7fe00806a1a0 0x7fe00806c660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:55:31.666 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:31.662+0000 7fe0037fe640 1 -- 172.21.15.43:0/2822439552 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(147..147 src has 1..147) v4 ==== 6426+0+0 (secure 0 0 0) 0x7fe010088de0 con 0x7fe01c1f50c0 2023-08-09T13:55:31.666 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:31.662+0000 7fe01affd640 1 --2- 172.21.15.43:0/2822439552 >> [v2:172.21.15.203:6832/3490266242,v1:172.21.15.203:6833/3490266242] conn(0x7fe00806a1a0 0x7fe00806c660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:55:31.667 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:31.663+0000 7fe01affd640 1 --2- 172.21.15.43:0/2822439552 >> [v2:172.21.15.203:6832/3490266242,v1:172.21.15.203:6833/3490266242] conn(0x7fe00806a1a0 0x7fe00806c660 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7fe00c034000 tx=0x7fe00c009030 comp rx=0 tx=0).ready entity=mgr.6659 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-09T13:55:31.668 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:31.666+0000 7fe0037fe640 1 -- 172.21.15.43:0/2822439552 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178066 (secure 0 0 0) 0x7fe01c0c53e0 con 0x7fe01c1f50c0 2023-08-09T13:55:31.899 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:31.897+0000 7fe0216c5640 1 -- 172.21.15.43:0/2822439552 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"} v 0) v1 -- 0x7fe01c0c53e0 con 0x7fe01c1f50c0 2023-08-09T13:55:32.395 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:32.392+0000 7fe0037fe640 1 -- 172.21.15.43:0/2822439552 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"}]=0 v70) v1 ==== 122+0+0 (secure 0 0 0) 0x7fe01c0c53e0 con 0x7fe01c1f50c0 2023-08-09T13:55:32.398 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:32.396+0000 7fe0216c5640 1 -- 172.21.15.43:0/2822439552 >> [v2:172.21.15.203:6832/3490266242,v1:172.21.15.203:6833/3490266242] conn(0x7fe00806a1a0 msgr2=0x7fe00806c660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:55:32.399 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:32.396+0000 7fe0216c5640 1 --2- 172.21.15.43:0/2822439552 >> [v2:172.21.15.203:6832/3490266242,v1:172.21.15.203:6833/3490266242] conn(0x7fe00806a1a0 0x7fe00806c660 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7fe00c034000 tx=0x7fe00c009030 comp rx=0 tx=0).stop 2023-08-09T13:55:32.399 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:32.396+0000 7fe0216c5640 1 -- 172.21.15.43:0/2822439552 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fe01c1f50c0 msgr2=0x7fe01c158620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:55:32.399 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:32.396+0000 7fe0216c5640 1 --2- 172.21.15.43:0/2822439552 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fe01c1f50c0 0x7fe01c158620 secure :-1 s=READY pgs=1695 cs=0 l=1 rev1=1 crypto rx=0x7fe0100108b0 tx=0x7fe010010d70 comp rx=0 tx=0).stop 2023-08-09T13:55:32.399 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:32.396+0000 7fe0216c5640 1 -- 172.21.15.43:0/2822439552 shutdown_connections 2023-08-09T13:55:32.399 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:32.396+0000 7fe0216c5640 1 --2- 172.21.15.43:0/2822439552 >> [v2:172.21.15.203:6832/3490266242,v1:172.21.15.203:6833/3490266242] conn(0x7fe00806a1a0 0x7fe00806c660 unknown :-1 s=CLOSED pgs=61 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:55:32.400 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:32.396+0000 7fe0216c5640 1 --2- 172.21.15.43:0/2822439552 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fe01c15cbf0 0x7fe01c15d070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:55:32.400 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:32.396+0000 7fe0216c5640 1 --2- 172.21.15.43:0/2822439552 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fe01c1f50c0 0x7fe01c158620 unknown :-1 s=CLOSED pgs=1695 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:55:32.400 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:32.396+0000 7fe0216c5640 1 --2- 172.21.15.43:0/2822439552 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fe004003a10 0x7fe01c153db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:55:32.400 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:32.396+0000 7fe0216c5640 1 -- 172.21.15.43:0/2822439552 >> 172.21.15.43:0/2822439552 conn(0x7fe01c0ce0a0 msgr2=0x7fe01c15bc20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-09T13:55:32.401 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:32.396+0000 7fe0216c5640 1 -- 172.21.15.43:0/2822439552 shutdown_connections 2023-08-09T13:55:32.401 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:32.396+0000 7fe0216c5640 1 -- 172.21.15.43:0/2822439552 wait complete. 2023-08-09T13:55:32.419 DEBUG:tasks.cephfs.filesystem:Not Creating any subvols for filesystem 'cephfs' 2023-08-09T13:55:32.419 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-08-09T13:55:32.655 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:32.652+0000 7f074d5cb640 1 Processor -- start 2023-08-09T13:55:32.655 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:32.653+0000 7f074d5cb640 1 -- start start 2023-08-09T13:55:32.656 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:32.654+0000 7f074d5cb640 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7f07480c8750 0x7f07480ad340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:55:32.657 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:32.654+0000 7f074d5cb640 1 -- --> v1:172.21.15.43:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f07480cbd90 con 0x7f07480c8b30 2023-08-09T13:55:32.657 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:32.654+0000 7f074d5cb640 1 -- --> v1:172.21.15.203:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f07480ca2b0 con 0x7f07480acdf0 2023-08-09T13:55:32.657 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:32.654+0000 7f074d5cb640 1 -- --> v2:172.21.15.203:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f074815e810 con 0x7f07480c8750 2023-08-09T13:55:32.657 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:32.655+0000 7f073ffff640 1 --1- >> v1:172.21.15.43:6789/0 conn(0x7f07480c8b30 0x7f074806f410 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.43:6789/0 says I am v1:172.21.15.43:56654/0 (socket says 172.21.15.43:56654) 2023-08-09T13:55:32.658 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:32.655+0000 7f073ffff640 1 -- 172.21.15.43:0/3865614774 learned_addr learned my addr 172.21.15.43:0/3865614774 (peer_addr_for_me v1:172.21.15.43:0/0) 2023-08-09T13:55:32.659 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:32.655+0000 7f0747fff640 1 --2- 172.21.15.43:0/3865614774 >> v2:172.21.15.203:3300/0 conn(0x7f07480c8750 0x7f07480ad340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:55:32.659 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:32.655+0000 7f07477fe640 1 -- 172.21.15.43:0/3865614774 <== mon.0 v1:172.21.15.43:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 467262473 0 0) 0x7f07480cbd90 con 0x7f07480c8b30 2023-08-09T13:55:32.659 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:32.655+0000 7f07477fe640 1 -- 172.21.15.43:0/3865614774 --> v1:172.21.15.43:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f0734003520 con 0x7f07480c8b30 2023-08-09T13:55:32.659 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:32.655+0000 7f07477fe640 1 -- 172.21.15.43:0/3865614774 <== mon.0 v1:172.21.15.43:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 636235266 0 0) 0x7f0734003520 con 0x7f07480c8b30 2023-08-09T13:55:32.660 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:32.655+0000 7f07477fe640 1 -- 172.21.15.43:0/3865614774 >> v2:172.21.15.203:3300/0 conn(0x7f07480c8750 msgr2=0x7f07480ad340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-09T13:55:32.660 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:32.655+0000 7f07477fe640 1 --2- 172.21.15.43:0/3865614774 >> v2:172.21.15.203:3300/0 conn(0x7f07480c8750 0x7f07480ad340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:55:32.660 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:32.656+0000 7f07477fe640 1 -- 172.21.15.43:0/3865614774 >> v1:172.21.15.203:6789/0 conn(0x7f07480acdf0 legacy=0x7f0748070f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:55:32.660 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:32.656+0000 7f07477fe640 1 -- 172.21.15.43:0/3865614774 --> v1:172.21.15.43:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f07480c9a20 con 0x7f07480c8b30 2023-08-09T13:55:32.660 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:32.656+0000 7f07477fe640 1 -- 172.21.15.43:0/3865614774 <== mon.0 v1:172.21.15.43:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1285150740 0 0) 0x7f0730001670 con 0x7f07480c8b30 2023-08-09T13:55:32.660 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:32.656+0000 7f07477fe640 1 -- 172.21.15.43:0/3865614774 >> v1:172.21.15.43:6789/0 conn(0x7f07480c8b30 legacy=0x7f074806f410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:55:32.660 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:32.656+0000 7f07477fe640 1 --2- 172.21.15.43:0/3865614774 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f0734003970 0x7f0734003d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:55:32.661 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:32.656+0000 7f07477fe640 1 --2- 172.21.15.43:0/3865614774 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f07340047c0 0x7f0734004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:55:32.661 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:32.656+0000 7f07477fe640 1 --2- 172.21.15.43:0/3865614774 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f0734005180 0x7f073400b980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:55:32.661 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:32.656+0000 7f07477fe640 1 -- 172.21.15.43:0/3865614774 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f074815e810 con 0x7f0734003970 2023-08-09T13:55:32.661 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:32.656+0000 7f07477fe640 1 -- 172.21.15.43:0/3865614774 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f073400c120 con 0x7f07340047c0 2023-08-09T13:55:32.661 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:32.656+0000 7f07477fe640 1 -- 172.21.15.43:0/3865614774 --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f073400c2e0 con 0x7f0734005180 2023-08-09T13:55:32.661 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:32.656+0000 7f0747fff640 1 --2- 172.21.15.43:0/3865614774 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f0734003970 0x7f0734003d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:55:32.661 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:32.656+0000 7f073ffff640 1 --2- 172.21.15.43:0/3865614774 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f07340047c0 0x7f0734004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:55:32.662 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:32.656+0000 7f0747fff640 1 -- 172.21.15.43:0/3865614774 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f0734005180 msgr2=0x7f073400b980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-09T13:55:32.662 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:32.656+0000 7f0747fff640 1 --2- 172.21.15.43:0/3865614774 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f0734005180 0x7f073400b980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:55:32.662 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:32.656+0000 7f0747fff640 1 -- 172.21.15.43:0/3865614774 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f07340047c0 msgr2=0x7f0734004c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-09T13:55:32.662 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:32.656+0000 7f0747fff640 1 --2- 172.21.15.43:0/3865614774 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f07340047c0 0x7f0734004c40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:55:32.662 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:32.656+0000 7f0747fff640 1 -- 172.21.15.43:0/3865614774 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0734003520 con 0x7f0734003970 2023-08-09T13:55:32.662 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:32.657+0000 7f074cdca640 1 -- 172.21.15.43:0/3865614774 reap_dead start 2023-08-09T13:55:32.663 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:32.658+0000 7f0747fff640 1 --2- 172.21.15.43:0/3865614774 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f0734003970 0x7f0734003d90 secure :-1 s=READY pgs=1697 cs=0 l=1 rev1=1 crypto rx=0x7f0738009ac0 tx=0x7f07340057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=61436000179b6c03 server_cookie=0 in_seq=0 out_seq=0 2023-08-09T13:55:32.663 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:32.658+0000 7f07477fe640 1 -- 172.21.15.43:0/3865614774 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f073802c6d0 con 0x7f0734003970 2023-08-09T13:55:32.663 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:32.658+0000 7f07477fe640 1 -- 172.21.15.43:0/3865614774 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f073802c830 con 0x7f0734003970 2023-08-09T13:55:32.664 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:32.658+0000 7f07477fe640 1 -- 172.21.15.43:0/3865614774 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f073802cbc0 con 0x7f0734003970 2023-08-09T13:55:32.664 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:32.658+0000 7f074d5cb640 1 -- 172.21.15.43:0/3865614774 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f0734003970 msgr2=0x7f0734003d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:55:32.664 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:32.658+0000 7f074d5cb640 1 --2- 172.21.15.43:0/3865614774 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f0734003970 0x7f0734003d90 secure :-1 s=READY pgs=1697 cs=0 l=1 rev1=1 crypto rx=0x7f0738009ac0 tx=0x7f07340057f0 comp rx=0 tx=0).stop 2023-08-09T13:55:32.664 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:32.658+0000 7f074d5cb640 1 -- 172.21.15.43:0/3865614774 shutdown_connections 2023-08-09T13:55:32.664 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:32.658+0000 7f074d5cb640 1 --2- 172.21.15.43:0/3865614774 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f0734003970 0x7f0734003d90 unknown :-1 s=CLOSED pgs=1697 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:55:32.664 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:32.658+0000 7f074d5cb640 1 -- 172.21.15.43:0/3865614774 >> 172.21.15.43:0/3865614774 conn(0x7f07480cde90 msgr2=0x7f07480ce2a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-09T13:55:32.664 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:32.659+0000 7f074d5cb640 1 -- 172.21.15.43:0/3865614774 shutdown_connections 2023-08-09T13:55:32.665 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:32.659+0000 7f074d5cb640 1 -- 172.21.15.43:0/3865614774 wait complete. 2023-08-09T13:55:32.665 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:32.660+0000 7f074d5cb640 1 Processor -- start 2023-08-09T13:55:32.665 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:32.661+0000 7f074d5cb640 1 -- start start 2023-08-09T13:55:32.666 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:32.661+0000 7f074d5cb640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f0734003970 0x7f0748158f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:55:32.666 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:32.661+0000 7f074d5cb640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f07480c06d0 0x7f07480c0b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:55:32.666 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:32.662+0000 7f074d5cb640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f074806bba0 0x7f074806c020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:55:32.667 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:32.662+0000 7f074d5cb640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f07480c9f70 con 0x7f07480c06d0 2023-08-09T13:55:32.667 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:32.662+0000 7f074d5cb640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f07480b3ae0 con 0x7f0734003970 2023-08-09T13:55:32.667 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:32.662+0000 7f074d5cb640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f07480c1090 con 0x7f074806bba0 2023-08-09T13:55:32.667 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:32.662+0000 7f073ffff640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f07480c06d0 0x7f07480c0b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:55:32.667 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:32.662+0000 7f073ffff640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f07480c06d0 0x7f07480c0b50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:52616/0 (socket says 172.21.15.43:52616) 2023-08-09T13:55:32.667 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:32.662+0000 7f073ffff640 1 -- 172.21.15.43:0/1075703124 learned_addr learned my addr 172.21.15.43:0/1075703124 (peer_addr_for_me v2:172.21.15.43:0/0) 2023-08-09T13:55:32.667 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:32.662+0000 7f0747fff640 1 --2- 172.21.15.43:0/1075703124 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f0734003970 0x7f0748158f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:55:32.668 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:32.662+0000 7f073ffff640 1 -- 172.21.15.43:0/1075703124 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f074806bba0 msgr2=0x7f074806c020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:55:32.668 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:32.662+0000 7f073ffff640 1 --2- 172.21.15.43:0/1075703124 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f074806bba0 0x7f074806c020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:55:32.668 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:32.662+0000 7f073ffff640 1 -- 172.21.15.43:0/1075703124 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f0734003970 msgr2=0x7f0748158f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:55:32.668 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:32.662+0000 7f073ffff640 1 --2- 172.21.15.43:0/1075703124 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f0734003970 0x7f0748158f90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:55:32.668 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:32.662+0000 7f073ffff640 1 -- 172.21.15.43:0/1075703124 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f07480c6870 con 0x7f07480c06d0 2023-08-09T13:55:32.668 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:32.663+0000 7f073ffff640 1 --2- 172.21.15.43:0/1075703124 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f07480c06d0 0x7f07480c0b50 secure :-1 s=READY pgs=1698 cs=0 l=1 rev1=1 crypto rx=0x7f073000d8e0 tx=0x7f073000dda0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-09T13:55:32.668 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:32.663+0000 7f07457fa640 1 -- 172.21.15.43:0/1075703124 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f07300074e0 con 0x7f07480c06d0 2023-08-09T13:55:32.668 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:32.663+0000 7f07457fa640 1 -- 172.21.15.43:0/1075703124 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0730007640 con 0x7f07480c06d0 2023-08-09T13:55:32.669 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:32.663+0000 7f074d5cb640 1 -- 172.21.15.43:0/1075703124 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f074806c850 con 0x7f07480c06d0 2023-08-09T13:55:32.669 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:32.663+0000 7f07457fa640 1 -- 172.21.15.43:0/1075703124 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f07300049e0 con 0x7f07480c06d0 2023-08-09T13:55:32.669 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:32.663+0000 7f074d5cb640 1 -- 172.21.15.43:0/1075703124 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f07480c6220 con 0x7f07480c06d0 2023-08-09T13:55:32.669 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:32.664+0000 7f074d5cb640 1 -- 172.21.15.43:0/1075703124 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f07480c6a00 con 0x7f07480c06d0 2023-08-09T13:55:32.670 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:32.665+0000 7f07457fa640 1 -- 172.21.15.43:0/1075703124 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 67) v1 ==== 88443+0+0 (secure 0 0 0) 0x7f0730004b40 con 0x7f07480c06d0 2023-08-09T13:55:32.670 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:32.666+0000 7f07457fa640 1 --2- 172.21.15.43:0/1075703124 >> [v2:172.21.15.203:6832/3490266242,v1:172.21.15.203:6833/3490266242] conn(0x7f072806a170 0x7f072806c630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:55:32.670 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:32.666+0000 7f07457fa640 1 -- 172.21.15.43:0/1075703124 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(147..147 src has 1..147) v4 ==== 6426+0+0 (secure 0 0 0) 0x7f0730088460 con 0x7f07480c06d0 2023-08-09T13:55:32.670 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:32.667+0000 7f0747fff640 1 --2- 172.21.15.43:0/1075703124 >> [v2:172.21.15.203:6832/3490266242,v1:172.21.15.203:6833/3490266242] conn(0x7f072806a170 0x7f072806c630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:55:32.670 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:32.668+0000 7f0747fff640 1 --2- 172.21.15.43:0/1075703124 >> [v2:172.21.15.203:6832/3490266242,v1:172.21.15.203:6833/3490266242] conn(0x7f072806a170 0x7f072806c630 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f0738036000 tx=0x7f0738030000 comp rx=0 tx=0).ready entity=mgr.6659 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-09T13:55:32.673 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:32.671+0000 7f07457fa640 1 -- 172.21.15.43:0/1075703124 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178066 (secure 0 0 0) 0x7f07300571b0 con 0x7f07480c06d0 2023-08-09T13:55:32.905 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:32.902+0000 7f074d5cb640 1 -- 172.21.15.43:0/1075703124 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f07480c9230 con 0x7f07480c06d0 2023-08-09T13:55:32.906 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:32.904+0000 7f07457fa640 1 -- 172.21.15.43:0/1075703124 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 70 v70) v1 ==== 94+0+4883 (secure 0 0 0) 0x7f0730056d40 con 0x7f07480c06d0 2023-08-09T13:55:32.906 INFO:teuthology.orchestra.run.smithi043.stdout: 2023-08-09T13:55:32.907 INFO:teuthology.orchestra.run.smithi043.stdout:{"epoch":70,"default_fscid":8,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6812,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.203:6835/2100579784","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6834","nonce":2100579784},{"type":"v1","addr":"172.21.15.203:6835","nonce":2100579784}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":65},{"gid":6815,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.203:6837/1689128061","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6836","nonce":1689128061},{"type":"v1","addr":"172.21.15.203:6837","nonce":1689128061}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":65},{"gid":7543,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.43:6835/1126540259","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6833","nonce":1126540259},{"type":"v1","addr":"172.21.15.43:6835","nonce":1126540259}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":65}],"filesystems":[{"mdsmap":{"epoch":70,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-08-09T13:55:28.794042+0000","modified":"2023-08-09T13:55:32.388404+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":9126},"failed":[],"damaged":[],"stopped":[],"info":{"gid_9126":{"gid":9126,"name":"a","rank":0,"incarnation":66,"state":"up:active","state_seq":3,"addr":"172.21.15.43:6834/2023142010","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6832","nonce":2023142010},{"type":"v1","addr":"172.21.15.43:6834","nonce":2023142010}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[19],"metadata_pool":18,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":8}]} 2023-08-09T13:55:32.907 INFO:teuthology.orchestra.run.smithi043.stderr:dumped fsmap epoch 70 2023-08-09T13:55:32.909 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:32.907+0000 7f074d5cb640 1 -- 172.21.15.43:0/1075703124 >> [v2:172.21.15.203:6832/3490266242,v1:172.21.15.203:6833/3490266242] conn(0x7f072806a170 msgr2=0x7f072806c630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:55:32.909 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:32.907+0000 7f074d5cb640 1 --2- 172.21.15.43:0/1075703124 >> [v2:172.21.15.203:6832/3490266242,v1:172.21.15.203:6833/3490266242] conn(0x7f072806a170 0x7f072806c630 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f0738036000 tx=0x7f0738030000 comp rx=0 tx=0).stop 2023-08-09T13:55:32.909 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:32.907+0000 7f074d5cb640 1 -- 172.21.15.43:0/1075703124 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f07480c06d0 msgr2=0x7f07480c0b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:55:32.909 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:32.907+0000 7f074d5cb640 1 --2- 172.21.15.43:0/1075703124 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f07480c06d0 0x7f07480c0b50 secure :-1 s=READY pgs=1698 cs=0 l=1 rev1=1 crypto rx=0x7f073000d8e0 tx=0x7f073000dda0 comp rx=0 tx=0).stop 2023-08-09T13:55:32.910 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:32.907+0000 7f074d5cb640 1 -- 172.21.15.43:0/1075703124 shutdown_connections 2023-08-09T13:55:32.910 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:32.907+0000 7f074d5cb640 1 --2- 172.21.15.43:0/1075703124 >> [v2:172.21.15.203:6832/3490266242,v1:172.21.15.203:6833/3490266242] conn(0x7f072806a170 0x7f072806c630 unknown :-1 s=CLOSED pgs=62 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:55:32.910 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:32.907+0000 7f074d5cb640 1 --2- 172.21.15.43:0/1075703124 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f074806bba0 0x7f074806c020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:55:32.910 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:32.907+0000 7f074d5cb640 1 --2- 172.21.15.43:0/1075703124 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f07480c06d0 0x7f07480c0b50 unknown :-1 s=CLOSED pgs=1698 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:55:32.910 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:32.907+0000 7f074d5cb640 1 --2- 172.21.15.43:0/1075703124 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f0734003970 0x7f0748158f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:55:32.910 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:32.907+0000 7f074d5cb640 1 -- 172.21.15.43:0/1075703124 >> 172.21.15.43:0/1075703124 conn(0x7f07480cde90 msgr2=0x7f07480ac1b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-09T13:55:32.911 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:32.907+0000 7f074d5cb640 1 -- 172.21.15.43:0/1075703124 shutdown_connections 2023-08-09T13:55:32.911 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:32.907+0000 7f074d5cb640 1 -- 172.21.15.43:0/1075703124 wait complete. 2023-08-09T13:55:32.928 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-08-09T13:55:33.147 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:33.144+0000 7f08fb569640 1 Processor -- start 2023-08-09T13:55:33.147 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:33.145+0000 7f08fb569640 1 -- start start 2023-08-09T13:55:33.149 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:33.146+0000 7f08fb569640 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7f08f415c620 0x7f08f40cfe70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:55:33.149 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:33.146+0000 7f08fb569640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f08f40cf4f0 0x7f08f40d07e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:55:33.149 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:33.146+0000 7f08fb569640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f08f40d0d20 0x7f08f40cce40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:55:33.149 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:33.146+0000 7f08fb569640 1 -- --> v2:172.21.15.43:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f08f40b2330 con 0x7f08f40cf4f0 2023-08-09T13:55:33.149 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:33.146+0000 7f08fb569640 1 -- --> v2:172.21.15.203:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f08f40d2ec0 con 0x7f08f415c620 2023-08-09T13:55:33.150 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:33.146+0000 7f08fb569640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f08f40d1170 con 0x7f08f40d0d20 2023-08-09T13:55:33.150 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:33.147+0000 7f08f9d66640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f08f40cf4f0 0x7f08f40d07e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:55:33.150 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:33.147+0000 7f08f9d66640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f08f40cf4f0 0x7f08f40d07e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:52626/0 (socket says 172.21.15.43:52626) 2023-08-09T13:55:33.150 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:33.147+0000 7f08f9d66640 1 -- 172.21.15.43:0/3543920568 learned_addr learned my addr 172.21.15.43:0/3543920568 (peer_addr_for_me v2:172.21.15.43:0/0) 2023-08-09T13:55:33.151 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:33.147+0000 7f08fa567640 1 --2- 172.21.15.43:0/3543920568 >> v2:172.21.15.203:3300/0 conn(0x7f08f415c620 0x7f08f40cfe70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:55:33.151 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:33.147+0000 7f08fad68640 1 --2- 172.21.15.43:0/3543920568 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f08f40d0d20 0x7f08f40cce40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:55:33.151 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:33.147+0000 7f08f9d66640 1 -- 172.21.15.43:0/3543920568 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f08f40d0d20 msgr2=0x7f08f40cce40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-09T13:55:33.151 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:33.147+0000 7f08f9d66640 1 --2- 172.21.15.43:0/3543920568 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f08f40d0d20 0x7f08f40cce40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:55:33.152 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:33.147+0000 7f08f9d66640 1 -- 172.21.15.43:0/3543920568 >> v2:172.21.15.203:3300/0 conn(0x7f08f415c620 msgr2=0x7f08f40cfe70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-09T13:55:33.152 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:33.147+0000 7f08f9d66640 1 --2- 172.21.15.43:0/3543920568 >> v2:172.21.15.203:3300/0 conn(0x7f08f415c620 0x7f08f40cfe70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:55:33.152 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:33.147+0000 7f08f9d66640 1 -- 172.21.15.43:0/3543920568 --> v2:172.21.15.43:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f08f40d0500 con 0x7f08f40cf4f0 2023-08-09T13:55:33.152 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:33.147+0000 7f08fa567640 1 --2- 172.21.15.43:0/3543920568 >> v2:172.21.15.203:3300/0 conn(0x7f08f415c620 0x7f08f40cfe70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-08-09T13:55:33.152 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:33.147+0000 7f08fad68640 1 --2- 172.21.15.43:0/3543920568 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f08f40d0d20 0x7f08f40cce40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-08-09T13:55:33.152 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:33.148+0000 7f08f9d66640 1 --2- 172.21.15.43:0/3543920568 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f08f40cf4f0 0x7f08f40d07e0 secure :-1 s=READY pgs=1699 cs=0 l=1 rev1=1 crypto rx=0x7f08f408b2f0 tx=0x7f08e402b890 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ddd894e074cd5e71 server_cookie=0 in_seq=0 out_seq=0 2023-08-09T13:55:33.152 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:33.149+0000 7f08f9565640 1 -- 172.21.15.43:0/3543920568 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f08e40317f0 con 0x7f08f40cf4f0 2023-08-09T13:55:33.152 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:33.149+0000 7f08f9565640 1 -- 172.21.15.43:0/3543920568 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f08e4031950 con 0x7f08f40cf4f0 2023-08-09T13:55:33.153 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:33.149+0000 7f08f9565640 1 -- 172.21.15.43:0/3543920568 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f08e4031ce0 con 0x7f08f40cf4f0 2023-08-09T13:55:33.153 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:33.149+0000 7f08fb569640 1 -- 172.21.15.43:0/3543920568 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f08f40cf4f0 msgr2=0x7f08f40d07e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:55:33.153 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:33.149+0000 7f08fb569640 1 --2- 172.21.15.43:0/3543920568 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f08f40cf4f0 0x7f08f40d07e0 secure :-1 s=READY pgs=1699 cs=0 l=1 rev1=1 crypto rx=0x7f08f408b2f0 tx=0x7f08e402b890 comp rx=0 tx=0).stop 2023-08-09T13:55:33.154 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:33.150+0000 7f08fb569640 1 -- 172.21.15.43:0/3543920568 shutdown_connections 2023-08-09T13:55:33.154 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:33.150+0000 7f08fb569640 1 --2- 172.21.15.43:0/3543920568 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f08f40cf4f0 0x7f08f40d07e0 unknown :-1 s=CLOSED pgs=1699 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:55:33.154 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:33.150+0000 7f08fb569640 1 --2- 172.21.15.43:0/3543920568 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f08f40d0d20 0x7f08f40cce40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:55:33.154 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:33.150+0000 7f08fb569640 1 --2- 172.21.15.43:0/3543920568 >> v2:172.21.15.203:3300/0 conn(0x7f08f415c620 0x7f08f40cfe70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:55:33.154 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:33.150+0000 7f08fb569640 1 -- 172.21.15.43:0/3543920568 >> 172.21.15.43:0/3543920568 conn(0x7f08f409b650 msgr2=0x7f08f40d22a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-09T13:55:33.154 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:33.150+0000 7f08fb569640 1 -- 172.21.15.43:0/3543920568 shutdown_connections 2023-08-09T13:55:33.154 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:33.150+0000 7f08fb569640 1 -- 172.21.15.43:0/3543920568 wait complete. 2023-08-09T13:55:33.155 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:33.152+0000 7f08fb569640 1 Processor -- start 2023-08-09T13:55:33.155 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:33.152+0000 7f08fb569640 1 -- start start 2023-08-09T13:55:33.156 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:33.154+0000 7f08fb569640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f08f40cf4f0 0x7f08f4159ce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:55:33.156 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:33.154+0000 7f08fb569640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f08f40d0d20 0x7f08f40c9370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:55:33.157 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:33.154+0000 7f08fb569640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f08f415c620 0x7f08f40b4ec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:55:33.157 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:33.154+0000 7f08fb569640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f08f40c7410 con 0x7f08f415c620 2023-08-09T13:55:33.157 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:33.154+0000 7f08fb569640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f08f40c98b0 con 0x7f08f40d0d20 2023-08-09T13:55:33.157 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:33.154+0000 7f08fb569640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f08f40b0a20 con 0x7f08f40cf4f0 2023-08-09T13:55:33.157 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:33.154+0000 7f08fad68640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f08f415c620 0x7f08f40b4ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:55:33.157 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:33.154+0000 7f08fad68640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f08f415c620 0x7f08f40b4ec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:52628/0 (socket says 172.21.15.43:52628) 2023-08-09T13:55:33.157 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:33.154+0000 7f08fad68640 1 -- 172.21.15.43:0/227707393 learned_addr learned my addr 172.21.15.43:0/227707393 (peer_addr_for_me v2:172.21.15.43:0/0) 2023-08-09T13:55:33.158 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:33.154+0000 7f08fa567640 1 --2- 172.21.15.43:0/227707393 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f08f40cf4f0 0x7f08f4159ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:55:33.159 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:33.154+0000 7f08f9d66640 1 --2- 172.21.15.43:0/227707393 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f08f40d0d20 0x7f08f40c9370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:55:33.159 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:33.155+0000 7f08fad68640 1 -- 172.21.15.43:0/227707393 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f08f40cf4f0 msgr2=0x7f08f4159ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:55:33.159 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:33.155+0000 7f08fad68640 1 --2- 172.21.15.43:0/227707393 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f08f40cf4f0 0x7f08f4159ce0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:55:33.159 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:33.155+0000 7f08fad68640 1 -- 172.21.15.43:0/227707393 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f08f40d0d20 msgr2=0x7f08f40c9370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:55:33.159 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:33.155+0000 7f08fad68640 1 --2- 172.21.15.43:0/227707393 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f08f40d0d20 0x7f08f40c9370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:55:33.159 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:33.155+0000 7f08fad68640 1 -- 172.21.15.43:0/227707393 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f08e40094d0 con 0x7f08f415c620 2023-08-09T13:55:33.159 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:33.155+0000 7f08f9d66640 1 --2- 172.21.15.43:0/227707393 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f08f40d0d20 0x7f08f40c9370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-08-09T13:55:33.160 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:33.155+0000 7f08fad68640 1 --2- 172.21.15.43:0/227707393 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f08f415c620 0x7f08f40b4ec0 secure :-1 s=READY pgs=1700 cs=0 l=1 rev1=1 crypto rx=0x7f08ec00cd50 tx=0x7f08ec00a650 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-09T13:55:33.160 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:33.155+0000 7f08eaffd640 1 -- 172.21.15.43:0/227707393 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f08ec00f680 con 0x7f08f415c620 2023-08-09T13:55:33.160 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:33.155+0000 7f08eaffd640 1 -- 172.21.15.43:0/227707393 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f08ec00f800 con 0x7f08f415c620 2023-08-09T13:55:33.160 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:33.155+0000 7f08eaffd640 1 -- 172.21.15.43:0/227707393 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f08ec00fbb0 con 0x7f08f415c620 2023-08-09T13:55:33.160 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:33.155+0000 7f08fb569640 1 -- 172.21.15.43:0/227707393 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f08f40c9ae0 con 0x7f08f415c620 2023-08-09T13:55:33.160 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:33.155+0000 7f08fb569640 1 -- 172.21.15.43:0/227707393 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f08f40c6d50 con 0x7f08f415c620 2023-08-09T13:55:33.161 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:33.157+0000 7f08fb569640 1 -- 172.21.15.43:0/227707393 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f08f40ccad0 con 0x7f08f415c620 2023-08-09T13:55:33.161 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:33.158+0000 7f08eaffd640 1 -- 172.21.15.43:0/227707393 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 67) v1 ==== 88443+0+0 (secure 0 0 0) 0x7f08ec00b040 con 0x7f08f415c620 2023-08-09T13:55:33.161 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:33.158+0000 7f08eaffd640 1 --2- 172.21.15.43:0/227707393 >> [v2:172.21.15.203:6832/3490266242,v1:172.21.15.203:6833/3490266242] conn(0x7f08d806a170 0x7f08d806c630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:55:33.161 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:33.159+0000 7f08eaffd640 1 -- 172.21.15.43:0/227707393 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(147..147 src has 1..147) v4 ==== 6426+0+0 (secure 0 0 0) 0x7f08ec089330 con 0x7f08f415c620 2023-08-09T13:55:33.162 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:33.159+0000 7f08fa567640 1 --2- 172.21.15.43:0/227707393 >> [v2:172.21.15.203:6832/3490266242,v1:172.21.15.203:6833/3490266242] conn(0x7f08d806a170 0x7f08d806c630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:55:33.163 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:33.161+0000 7f08fa567640 1 --2- 172.21.15.43:0/227707393 >> [v2:172.21.15.203:6832/3490266242,v1:172.21.15.203:6833/3490266242] conn(0x7f08d806a170 0x7f08d806c630 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f08f000b800 tx=0x7f08f0008040 comp rx=0 tx=0).ready entity=mgr.6659 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-09T13:55:33.165 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:33.163+0000 7f08eaffd640 1 -- 172.21.15.43:0/227707393 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178066 (secure 0 0 0) 0x7f08ec054720 con 0x7f08f415c620 2023-08-09T13:55:33.238 INFO:tasks.ceph.mds.a.smithi043.stderr:2023-08-09T13:55:33.236+0000 7f2f02137640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2023-08-09T13:55:33.360 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:33.358+0000 7f08fb569640 1 -- 172.21.15.43:0/227707393 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f08f40c7550 con 0x7f08f415c620 2023-08-09T13:55:33.362 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:33.360+0000 7f08eaffd640 1 -- 172.21.15.43:0/227707393 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v147) v1 ==== 74+0+18566 (secure 0 0 0) 0x7f08f40c7550 con 0x7f08f415c620 2023-08-09T13:55:33.363 INFO:teuthology.orchestra.run.smithi043.stdout: 2023-08-09T13:55:33.363 INFO:teuthology.orchestra.run.smithi043.stdout:{"epoch":147,"fsid":"daac87ee-1337-4250-a955-aea4e408a39b","created":"2023-08-09T13:33:41.701198+0000","modified":"2023-08-09T13:55:30.330576+0000","last_up_change":"2023-08-09T13:33:53.783453+0000","last_in_change":"2023-08-09T13:33:45.148904+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":19,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":"rbd","create_time":"2023-08-09T13:33:54.365137+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":".mgr","create_time":"2023-08-09T13:33:54.831803+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":18,"pool_name":"cephfs_metadata","create_time":"2023-08-09T13:55:27.207507+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"146","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":19,"pool_name":"cephfs_data","create_time":"2023-08-09T13:55:27.825578+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"147","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"cbd49ecd-e896-4fbd-864e-5c37a829b165","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":145,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6800","nonce":502667335},{"type":"v1","addr":"172.21.15.43:6801","nonce":502667335}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6802","nonce":502667335},{"type":"v1","addr":"172.21.15.43:6803","nonce":502667335}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6806","nonce":502667335},{"type":"v1","addr":"172.21.15.43:6807","nonce":502667335}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6804","nonce":502667335},{"type":"v1","addr":"172.21.15.43:6805","nonce":502667335}]},"public_addr":"172.21.15.43:6801/502667335","cluster_addr":"172.21.15.43:6803/502667335","heartbeat_back_addr":"172.21.15.43:6807/502667335","heartbeat_front_addr":"172.21.15.43:6805/502667335","state":["exists","up"]},{"osd":1,"uuid":"75c7e0fe-1ec7-4fe8-a2f2-3661a65b8500","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":145,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6808","nonce":3289134610},{"type":"v1","addr":"172.21.15.43:6809","nonce":3289134610}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6810","nonce":3289134610},{"type":"v1","addr":"172.21.15.43:6812","nonce":3289134610}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6821","nonce":3289134610},{"type":"v1","addr":"172.21.15.43:6824","nonce":3289134610}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6815","nonce":3289134610},{"type":"v1","addr":"172.21.15.43:6818","nonce":3289134610}]},"public_addr":"172.21.15.43:6809/3289134610","cluster_addr":"172.21.15.43:6812/3289134610","heartbeat_back_addr":"172.21.15.43:6824/3289134610","heartbeat_front_addr":"172.21.15.43:6818/3289134610","state":["exists","up"]},{"osd":2,"uuid":"bb5c41e9-7f5f-4125-9b01-3bb8bfc2ea83","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":145,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6811","nonce":4079709736},{"type":"v1","addr":"172.21.15.43:6814","nonce":4079709736}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6817","nonce":4079709736},{"type":"v1","addr":"172.21.15.43:6820","nonce":4079709736}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6829","nonce":4079709736},{"type":"v1","addr":"172.21.15.43:6831","nonce":4079709736}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6823","nonce":4079709736},{"type":"v1","addr":"172.21.15.43:6826","nonce":4079709736}]},"public_addr":"172.21.15.43:6814/4079709736","cluster_addr":"172.21.15.43:6820/4079709736","heartbeat_back_addr":"172.21.15.43:6831/4079709736","heartbeat_front_addr":"172.21.15.43:6826/4079709736","state":["exists","up"]},{"osd":3,"uuid":"bfe4c8b4-694a-465c-87e6-2b2aab852613","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":145,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6813","nonce":1709538759},{"type":"v1","addr":"172.21.15.43:6816","nonce":1709538759}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6819","nonce":1709538759},{"type":"v1","addr":"172.21.15.43:6822","nonce":1709538759}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6828","nonce":1709538759},{"type":"v1","addr":"172.21.15.43:6830","nonce":1709538759}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6825","nonce":1709538759},{"type":"v1","addr":"172.21.15.43:6827","nonce":1709538759}]},"public_addr":"172.21.15.43:6816/1709538759","cluster_addr":"172.21.15.43:6822/1709538759","heartbeat_back_addr":"172.21.15.43:6830/1709538759","heartbeat_front_addr":"172.21.15.43:6827/1709538759","state":["exists","up"]},{"osd":4,"uuid":"fa2e8a77-aba1-4756-853e-0d174a5bdeba","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":145,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6800","nonce":4196866481},{"type":"v1","addr":"172.21.15.203:6801","nonce":4196866481}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6802","nonce":4196866481},{"type":"v1","addr":"172.21.15.203:6803","nonce":4196866481}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6806","nonce":4196866481},{"type":"v1","addr":"172.21.15.203:6807","nonce":4196866481}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6804","nonce":4196866481},{"type":"v1","addr":"172.21.15.203:6805","nonce":4196866481}]},"public_addr":"172.21.15.203:6801/4196866481","cluster_addr":"172.21.15.203:6803/4196866481","heartbeat_back_addr":"172.21.15.203:6807/4196866481","heartbeat_front_addr":"172.21.15.203:6805/4196866481","state":["exists","up"]},{"osd":5,"uuid":"81811e4b-4d24-4f51-a82c-f1d700a2a00d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":145,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6816","nonce":3059290081},{"type":"v1","addr":"172.21.15.203:6817","nonce":3059290081}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6818","nonce":3059290081},{"type":"v1","addr":"172.21.15.203:6819","nonce":3059290081}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6822","nonce":3059290081},{"type":"v1","addr":"172.21.15.203:6823","nonce":3059290081}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6820","nonce":3059290081},{"type":"v1","addr":"172.21.15.203:6821","nonce":3059290081}]},"public_addr":"172.21.15.203:6817/3059290081","cluster_addr":"172.21.15.203:6819/3059290081","heartbeat_back_addr":"172.21.15.203:6823/3059290081","heartbeat_front_addr":"172.21.15.203:6821/3059290081","state":["exists","up"]},{"osd":6,"uuid":"307bac4d-6a5b-443d-af75-56d695d7674f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":145,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6824","nonce":512155569},{"type":"v1","addr":"172.21.15.203:6825","nonce":512155569}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6826","nonce":512155569},{"type":"v1","addr":"172.21.15.203:6827","nonce":512155569}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6830","nonce":512155569},{"type":"v1","addr":"172.21.15.203:6831","nonce":512155569}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6828","nonce":512155569},{"type":"v1","addr":"172.21.15.203:6829","nonce":512155569}]},"public_addr":"172.21.15.203:6825/512155569","cluster_addr":"172.21.15.203:6827/512155569","heartbeat_back_addr":"172.21.15.203:6831/512155569","heartbeat_front_addr":"172.21.15.203:6829/512155569","state":["exists","up"]},{"osd":7,"uuid":"a95dc7ca-9f1b-4295-af70-68a22a2d7d5e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":145,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6808","nonce":3331450438},{"type":"v1","addr":"172.21.15.203:6809","nonce":3331450438}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6810","nonce":3331450438},{"type":"v1","addr":"172.21.15.203:6811","nonce":3331450438}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6814","nonce":3331450438},{"type":"v1","addr":"172.21.15.203:6815","nonce":3331450438}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6812","nonce":3331450438},{"type":"v1","addr":"172.21.15.203:6813","nonce":3331450438}]},"public_addr":"172.21.15.203:6809/3331450438","cluster_addr":"172.21.15.203:6811/3331450438","heartbeat_back_addr":"172.21.15.203:6815/3331450438","heartbeat_front_addr":"172.21.15.203:6813/3331450438","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-09T13:33:50.547652+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-09T13:33:50.589469+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-09T13:33:50.561724+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-09T13:33:50.599650+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-09T13:33:50.411855+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-09T13:33:50.219681+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-09T13:33:50.504664+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-09T13:33:50.441248+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":7,"snaps":[{"begin":104,"length":1},{"begin":221,"length":1}]}],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-08-09T13:55:33.365 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:33.363+0000 7f08fb569640 1 -- 172.21.15.43:0/227707393 >> [v2:172.21.15.203:6832/3490266242,v1:172.21.15.203:6833/3490266242] conn(0x7f08d806a170 msgr2=0x7f08d806c630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:55:33.366 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:33.363+0000 7f08fb569640 1 --2- 172.21.15.43:0/227707393 >> [v2:172.21.15.203:6832/3490266242,v1:172.21.15.203:6833/3490266242] conn(0x7f08d806a170 0x7f08d806c630 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f08f000b800 tx=0x7f08f0008040 comp rx=0 tx=0).stop 2023-08-09T13:55:33.366 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:33.363+0000 7f08fb569640 1 -- 172.21.15.43:0/227707393 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f08f415c620 msgr2=0x7f08f40b4ec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:55:33.366 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:33.363+0000 7f08fb569640 1 --2- 172.21.15.43:0/227707393 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f08f415c620 0x7f08f40b4ec0 secure :-1 s=READY pgs=1700 cs=0 l=1 rev1=1 crypto rx=0x7f08ec00cd50 tx=0x7f08ec00a650 comp rx=0 tx=0).stop 2023-08-09T13:55:33.366 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:33.363+0000 7f08fb569640 1 -- 172.21.15.43:0/227707393 shutdown_connections 2023-08-09T13:55:33.366 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:33.363+0000 7f08fb569640 1 --2- 172.21.15.43:0/227707393 >> [v2:172.21.15.203:6832/3490266242,v1:172.21.15.203:6833/3490266242] conn(0x7f08d806a170 0x7f08d806c630 unknown :-1 s=CLOSED pgs=63 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:55:33.366 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:33.363+0000 7f08fb569640 1 --2- 172.21.15.43:0/227707393 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f08f415c620 0x7f08f40b4ec0 unknown :-1 s=CLOSED pgs=1700 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:55:33.366 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:33.363+0000 7f08fb569640 1 --2- 172.21.15.43:0/227707393 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f08f40d0d20 0x7f08f40c9370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:55:33.366 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:33.363+0000 7f08fb569640 1 --2- 172.21.15.43:0/227707393 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f08f40cf4f0 0x7f08f4159ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:55:33.367 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:33.363+0000 7f08fb569640 1 -- 172.21.15.43:0/227707393 >> 172.21.15.43:0/227707393 conn(0x7f08f409b650 msgr2=0x7f08f40ce8d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-09T13:55:33.367 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:33.363+0000 7f08fb569640 1 -- 172.21.15.43:0/227707393 shutdown_connections 2023-08-09T13:55:33.367 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:33.363+0000 7f08fb569640 1 -- 172.21.15.43:0/227707393 wait complete. 2023-08-09T13:55:33.385 INFO:tasks.cephfs.filesystem.ceph_manager:waiting for clean 2023-08-09T13:55:33.386 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-08-09T13:55:33.602 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:33.599+0000 7f44fa625640 1 Processor -- start 2023-08-09T13:55:33.602 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:33.599+0000 7f44fa625640 1 -- start start 2023-08-09T13:55:33.603 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:33.601+0000 7f44fa625640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f44f415f7b0 0x7f44f40b4760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:55:33.604 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:33.601+0000 7f44fa625640 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7f44f40b1050 0x7f44f40ccfa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:55:33.604 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:33.601+0000 7f44fa625640 1 -- --> v1:172.21.15.203:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f44f40cd930 con 0x7f44f40cd4e0 2023-08-09T13:55:33.604 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:33.601+0000 7f44fa625640 1 -- --> v2:172.21.15.203:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f44f40b7bf0 con 0x7f44f40b1050 2023-08-09T13:55:33.604 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:33.601+0000 7f44fa625640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f44f40b7ed0 con 0x7f44f415f7b0 2023-08-09T13:55:33.605 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:33.602+0000 7f44f9623640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f44f415f7b0 0x7f44f40b4760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:55:33.605 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:33.602+0000 7f44f8e22640 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7f44f40b1050 0x7f44f40ccfa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:55:33.605 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:33.602+0000 7f44f9623640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f44f415f7b0 0x7f44f40b4760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.203:3301/0 says I am v2:172.21.15.43:44150/0 (socket says 172.21.15.43:44150) 2023-08-09T13:55:33.605 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:33.602+0000 7f44f9623640 1 -- 172.21.15.43:0/3599797014 learned_addr learned my addr 172.21.15.43:0/3599797014 (peer_addr_for_me v2:172.21.15.43:0/0) 2023-08-09T13:55:33.605 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:33.603+0000 7f44f8e22640 1 -- 172.21.15.43:0/3599797014 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f44f415f7b0 msgr2=0x7f44f40b4760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-09T13:55:33.606 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:33.603+0000 7f44f8e22640 1 --2- 172.21.15.43:0/3599797014 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f44f415f7b0 0x7f44f40b4760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:55:33.606 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:33.603+0000 7f44f8e22640 1 -- 172.21.15.43:0/3599797014 >> v1:172.21.15.203:6789/0 conn(0x7f44f40cd4e0 legacy=0x7f44f40ceae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:55:33.606 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:33.603+0000 7f44f8e22640 1 -- 172.21.15.43:0/3599797014 --> v2:172.21.15.203:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f44f40d0370 con 0x7f44f40b1050 2023-08-09T13:55:33.606 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:33.604+0000 7f44f8e22640 1 --2- 172.21.15.43:0/3599797014 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f44f40b1050 0x7f44f40ccfa0 secure :-1 s=READY pgs=622 cs=0 l=1 rev1=1 crypto rx=0x7f44e40097a0 tx=0x7f44e402b8b0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=cb00b8510c8a1b22 server_cookie=0 in_seq=0 out_seq=0 2023-08-09T13:55:33.607 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:33.604+0000 7f44ebfff640 1 -- 172.21.15.43:0/3599797014 <== mon.1 v2:172.21.15.203:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f44e402d040 con 0x7f44f40b1050 2023-08-09T13:55:33.607 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:33.604+0000 7f44ebfff640 1 -- 172.21.15.43:0/3599797014 <== mon.1 v2:172.21.15.203:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f44e4031810 con 0x7f44f40b1050 2023-08-09T13:55:33.607 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:33.604+0000 7f44ebfff640 1 -- 172.21.15.43:0/3599797014 <== mon.1 v2:172.21.15.203:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f44e4031ba0 con 0x7f44f40b1050 2023-08-09T13:55:33.608 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:33.605+0000 7f44fa625640 1 -- 172.21.15.43:0/3599797014 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f44f40b1050 msgr2=0x7f44f40ccfa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:55:33.608 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:33.605+0000 7f44fa625640 1 --2- 172.21.15.43:0/3599797014 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f44f40b1050 0x7f44f40ccfa0 secure :-1 s=READY pgs=622 cs=0 l=1 rev1=1 crypto rx=0x7f44e40097a0 tx=0x7f44e402b8b0 comp rx=0 tx=0).stop 2023-08-09T13:55:33.608 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:33.605+0000 7f44fa625640 1 -- 172.21.15.43:0/3599797014 shutdown_connections 2023-08-09T13:55:33.608 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:33.605+0000 7f44fa625640 1 --2- 172.21.15.43:0/3599797014 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f44f40b1050 0x7f44f40ccfa0 unknown :-1 s=CLOSED pgs=622 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:55:33.608 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:33.605+0000 7f44fa625640 1 --2- 172.21.15.43:0/3599797014 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f44f415f7b0 0x7f44f40b4760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:55:33.608 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:33.605+0000 7f44fa625640 1 -- 172.21.15.43:0/3599797014 >> 172.21.15.43:0/3599797014 conn(0x7f44f406a2b0 msgr2=0x7f44f406a690 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-09T13:55:33.609 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:33.605+0000 7f44fa625640 1 -- 172.21.15.43:0/3599797014 shutdown_connections 2023-08-09T13:55:33.609 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:33.605+0000 7f44fa625640 1 -- 172.21.15.43:0/3599797014 wait complete. 2023-08-09T13:55:33.609 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:33.607+0000 7f44fa625640 1 Processor -- start 2023-08-09T13:55:33.609 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:33.607+0000 7f44fa625640 1 -- start start 2023-08-09T13:55:33.610 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:33.608+0000 7f44fa625640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f44f40b1050 0x7f44f40d2aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:55:33.611 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:33.608+0000 7f44fa625640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f44f40cd4e0 0x7f44f41f5100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:55:33.611 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:33.608+0000 7f44fa625640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f44f415f7b0 0x7f44f41f1330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:55:33.611 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:33.608+0000 7f44fa625640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f44f40d1230 con 0x7f44f40b1050 2023-08-09T13:55:33.611 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:33.608+0000 7f44fa625640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f44f40d1510 con 0x7f44f40cd4e0 2023-08-09T13:55:33.611 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:33.608+0000 7f44fa625640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f44f40d17f0 con 0x7f44f415f7b0 2023-08-09T13:55:33.611 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:33.608+0000 7f44f9623640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f44f40b1050 0x7f44f40d2aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:55:33.612 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:33.608+0000 7f44f9623640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f44f40b1050 0x7f44f40d2aa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:52634/0 (socket says 172.21.15.43:52634) 2023-08-09T13:55:33.612 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:33.608+0000 7f44f9623640 1 -- 172.21.15.43:0/46630530 learned_addr learned my addr 172.21.15.43:0/46630530 (peer_addr_for_me v2:172.21.15.43:0/0) 2023-08-09T13:55:33.613 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:33.609+0000 7f44f8e22640 1 --2- 172.21.15.43:0/46630530 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f44f40cd4e0 0x7f44f41f5100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:55:33.613 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:33.609+0000 7f44f9623640 1 -- 172.21.15.43:0/46630530 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f44f415f7b0 msgr2=0x7f44f41f1330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:55:33.613 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:33.609+0000 7f44f9623640 1 --2- 172.21.15.43:0/46630530 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f44f415f7b0 0x7f44f41f1330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:55:33.613 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:33.609+0000 7f44f9623640 1 -- 172.21.15.43:0/46630530 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f44f40cd4e0 msgr2=0x7f44f41f5100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:55:33.613 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:33.609+0000 7f44f9623640 1 --2- 172.21.15.43:0/46630530 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f44f40cd4e0 0x7f44f41f5100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:55:33.613 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:33.609+0000 7f44f9623640 1 -- 172.21.15.43:0/46630530 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f44e4009400 con 0x7f44f40b1050 2023-08-09T13:55:33.614 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:33.609+0000 7f44f8e22640 1 --2- 172.21.15.43:0/46630530 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f44f40cd4e0 0x7f44f41f5100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-08-09T13:55:33.614 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:33.609+0000 7f44f9623640 1 --2- 172.21.15.43:0/46630530 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f44f40b1050 0x7f44f40d2aa0 secure :-1 s=READY pgs=1701 cs=0 l=1 rev1=1 crypto rx=0x7f44f000e840 tx=0x7f44f000ed00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-09T13:55:33.614 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:33.610+0000 7f44e9ffb640 1 -- 172.21.15.43:0/46630530 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f44f000c8e0 con 0x7f44f40b1050 2023-08-09T13:55:33.614 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:33.610+0000 7f44e9ffb640 1 -- 172.21.15.43:0/46630530 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f44f000ca40 con 0x7f44f40b1050 2023-08-09T13:55:33.614 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:33.610+0000 7f44e9ffb640 1 -- 172.21.15.43:0/46630530 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f44f000cdf0 con 0x7f44f40b1050 2023-08-09T13:55:33.614 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:33.610+0000 7f44fa625640 1 -- 172.21.15.43:0/46630530 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f44f40d1ad0 con 0x7f44f40b1050 2023-08-09T13:55:33.614 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:33.610+0000 7f44fa625640 1 -- 172.21.15.43:0/46630530 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f44f41f5640 con 0x7f44f40b1050 2023-08-09T13:55:33.615 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:33.611+0000 7f44fa625640 1 -- 172.21.15.43:0/46630530 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f44f40739f0 con 0x7f44f40b1050 2023-08-09T13:55:33.615 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:33.612+0000 7f44e9ffb640 1 -- 172.21.15.43:0/46630530 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 67) v1 ==== 88443+0+0 (secure 0 0 0) 0x7f44f0013470 con 0x7f44f40b1050 2023-08-09T13:55:33.615 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:33.613+0000 7f44e9ffb640 1 --2- 172.21.15.43:0/46630530 >> [v2:172.21.15.203:6832/3490266242,v1:172.21.15.203:6833/3490266242] conn(0x7f44d806a170 0x7f44d806c630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:55:33.616 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:33.613+0000 7f44e9ffb640 1 -- 172.21.15.43:0/46630530 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(147..147 src has 1..147) v4 ==== 6426+0+0 (secure 0 0 0) 0x7f44f0088a50 con 0x7f44f40b1050 2023-08-09T13:55:33.616 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:33.614+0000 7f44f8e22640 1 --2- 172.21.15.43:0/46630530 >> [v2:172.21.15.203:6832/3490266242,v1:172.21.15.203:6833/3490266242] conn(0x7f44d806a170 0x7f44d806c630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:55:33.617 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:33.615+0000 7f44f8e22640 1 --2- 172.21.15.43:0/46630530 >> [v2:172.21.15.203:6832/3490266242,v1:172.21.15.203:6833/3490266242] conn(0x7f44d806a170 0x7f44d806c630 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f44e4036000 tx=0x7f44e403c040 comp rx=0 tx=0).ready entity=mgr.6659 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-09T13:55:33.620 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:33.617+0000 7f44e9ffb640 1 -- 172.21.15.43:0/46630530 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178066 (secure 0 0 0) 0x7f44f0053e20 con 0x7f44f40b1050 2023-08-09T13:55:33.809 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:33.806+0000 7f44fa625640 1 -- 172.21.15.43:0/46630530 --> [v2:172.21.15.203:6832/3490266242,v1:172.21.15.203:6833/3490266242] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f44f40d1680 con 0x7f44d806a170 2023-08-09T13:55:33.840 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:33.838+0000 7f44e9ffb640 1 -- 172.21.15.43:0/46630530 <== mgr.6659 v2:172.21.15.203:6832/3490266242 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+309661 (secure 0 0 0) 0x7f44f40d1680 con 0x7f44d806a170 2023-08-09T13:55:33.844 INFO:teuthology.orchestra.run.smithi043.stdout: 2023-08-09T13:55:33.845 INFO:teuthology.orchestra.run.smithi043.stderr:dumped all 2023-08-09T13:55:33.847 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:33.843+0000 7f44fa625640 1 -- 172.21.15.43:0/46630530 >> [v2:172.21.15.203:6832/3490266242,v1:172.21.15.203:6833/3490266242] conn(0x7f44d806a170 msgr2=0x7f44d806c630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:55:33.847 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:33.843+0000 7f44fa625640 1 --2- 172.21.15.43:0/46630530 >> [v2:172.21.15.203:6832/3490266242,v1:172.21.15.203:6833/3490266242] conn(0x7f44d806a170 0x7f44d806c630 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f44e4036000 tx=0x7f44e403c040 comp rx=0 tx=0).stop 2023-08-09T13:55:33.847 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:33.843+0000 7f44fa625640 1 -- 172.21.15.43:0/46630530 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f44f40b1050 msgr2=0x7f44f40d2aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:55:33.847 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:33.843+0000 7f44fa625640 1 --2- 172.21.15.43:0/46630530 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f44f40b1050 0x7f44f40d2aa0 secure :-1 s=READY pgs=1701 cs=0 l=1 rev1=1 crypto rx=0x7f44f000e840 tx=0x7f44f000ed00 comp rx=0 tx=0).stop 2023-08-09T13:55:33.847 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:33.843+0000 7f44fa625640 1 -- 172.21.15.43:0/46630530 shutdown_connections 2023-08-09T13:55:33.847 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:33.843+0000 7f44fa625640 1 --2- 172.21.15.43:0/46630530 >> [v2:172.21.15.203:6832/3490266242,v1:172.21.15.203:6833/3490266242] conn(0x7f44d806a170 0x7f44d806c630 unknown :-1 s=CLOSED pgs=64 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:55:33.848 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:33.843+0000 7f44fa625640 1 --2- 172.21.15.43:0/46630530 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f44f415f7b0 0x7f44f41f1330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:55:33.848 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:33.843+0000 7f44fa625640 1 --2- 172.21.15.43:0/46630530 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f44f40cd4e0 0x7f44f41f5100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:55:33.848 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:33.843+0000 7f44fa625640 1 --2- 172.21.15.43:0/46630530 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f44f40b1050 0x7f44f40d2aa0 unknown :-1 s=CLOSED pgs=1701 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:55:33.848 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:33.843+0000 7f44fa625640 1 -- 172.21.15.43:0/46630530 >> 172.21.15.43:0/46630530 conn(0x7f44f406a2b0 msgr2=0x7f44f40b6ca0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-09T13:55:33.848 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:33.843+0000 7f44fa625640 1 -- 172.21.15.43:0/46630530 shutdown_connections 2023-08-09T13:55:33.848 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:33.843+0000 7f44fa625640 1 -- 172.21.15.43:0/46630530 wait complete. 2023-08-09T13:55:33.865 INFO:teuthology.orchestra.run.smithi043.stdout:{"pg_ready":true,"pg_map":{"version":28,"stamp":"2023-08-09T13:55:32.575069+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":23,"num_object_clones":0,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":0,"num_read":1099,"num_read_kb":2242,"num_write":407,"num_write_kb":2430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":268,"ondisk_log_size":268,"up":210,"acting":210,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":79,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":333052,"kb_used_data":15232,"kb_used_omap":13,"kb_used_meta":317682,"kb_avail":754641668,"statfs":{"total":773094113280,"available":772753068032,"internally_reserved":0,"allocated":15597568,"data_stored":8906086,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":14248,"internal_metadata":325306456},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":34,"apply_latency_ms":34,"commit_latency_ns":34000000,"apply_latency_ns":34000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"5.286280"},"pg_stats":[{"pgid":"19.3f","version":"0'0","reported_seq":9,"reported_epoch":146,"state":"active+clean","last_fresh":"2023-08-09T13:55:29.389106+0000","last_change":"2023-08-09T13:55:29.386485+0000","last_active":"2023-08-09T13:55:29.389106+0000","last_peered":"2023-08-09T13:55:29.389106+0000","last_clean":"2023-08-09T13:55:29.389106+0000","last_became_active":"2023-08-09T13:55:29.386195+0000","last_became_peered":"2023-08-09T13:55:29.386195+0000","last_unstale":"2023-08-09T13:55:29.389106+0000","last_undegraded":"2023-08-09T13:55:29.389106+0000","last_fullsized":"2023-08-09T13:55:29.389106+0000","mapping_epoch":145,"log_start":"0'0","ondisk_log_start":"0'0","created":145,"last_epoch_clean":146,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:55:28.281790+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:55:28.281790+0000","last_clean_scrub_stamp":"2023-08-09T13:55:28.281790+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T18:35:28.179248+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"19.3e","version":"0'0","reported_seq":9,"reported_epoch":146,"state":"active+clean","last_fresh":"2023-08-09T13:55:29.391257+0000","last_change":"2023-08-09T13:55:29.390587+0000","last_active":"2023-08-09T13:55:29.391257+0000","last_peered":"2023-08-09T13:55:29.391257+0000","last_clean":"2023-08-09T13:55:29.391257+0000","last_became_active":"2023-08-09T13:55:29.390279+0000","last_became_peered":"2023-08-09T13:55:29.390279+0000","last_unstale":"2023-08-09T13:55:29.391257+0000","last_undegraded":"2023-08-09T13:55:29.391257+0000","last_fullsized":"2023-08-09T13:55:29.391257+0000","mapping_epoch":145,"log_start":"0'0","ondisk_log_start":"0'0","created":145,"last_epoch_clean":146,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:55:28.281790+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:55:28.281790+0000","last_clean_scrub_stamp":"2023-08-09T13:55:28.281790+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T19:28:17.384228+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"19.3d","version":"0'0","reported_seq":9,"reported_epoch":146,"state":"active+clean","last_fresh":"2023-08-09T13:55:29.387997+0000","last_change":"2023-08-09T13:55:29.378897+0000","last_active":"2023-08-09T13:55:29.387997+0000","last_peered":"2023-08-09T13:55:29.387997+0000","last_clean":"2023-08-09T13:55:29.387997+0000","last_became_active":"2023-08-09T13:55:29.378542+0000","last_became_peered":"2023-08-09T13:55:29.378542+0000","last_unstale":"2023-08-09T13:55:29.387997+0000","last_undegraded":"2023-08-09T13:55:29.387997+0000","last_fullsized":"2023-08-09T13:55:29.387997+0000","mapping_epoch":145,"log_start":"0'0","ondisk_log_start":"0'0","created":145,"last_epoch_clean":146,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:55:28.281790+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:55:28.281790+0000","last_clean_scrub_stamp":"2023-08-09T13:55:28.281790+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T19:35:21.339885+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"19.3c","version":"0'0","reported_seq":9,"reported_epoch":146,"state":"active+clean","last_fresh":"2023-08-09T13:55:29.374630+0000","last_change":"2023-08-09T13:55:29.371565+0000","last_active":"2023-08-09T13:55:29.374630+0000","last_peered":"2023-08-09T13:55:29.374630+0000","last_clean":"2023-08-09T13:55:29.374630+0000","last_became_active":"2023-08-09T13:55:29.371289+0000","last_became_peered":"2023-08-09T13:55:29.371289+0000","last_unstale":"2023-08-09T13:55:29.374630+0000","last_undegraded":"2023-08-09T13:55:29.374630+0000","last_fullsized":"2023-08-09T13:55:29.374630+0000","mapping_epoch":145,"log_start":"0'0","ondisk_log_start":"0'0","created":145,"last_epoch_clean":146,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:55:28.281790+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:55:28.281790+0000","last_clean_scrub_stamp":"2023-08-09T13:55:28.281790+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T20:03:28.279961+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"19.3b","version":"0'0","reported_seq":9,"reported_epoch":146,"state":"active+clean","last_fresh":"2023-08-09T13:55:29.374346+0000","last_change":"2023-08-09T13:55:29.367110+0000","last_active":"2023-08-09T13:55:29.374346+0000","last_peered":"2023-08-09T13:55:29.374346+0000","last_clean":"2023-08-09T13:55:29.374346+0000","last_became_active":"2023-08-09T13:55:29.366425+0000","last_became_peered":"2023-08-09T13:55:29.366425+0000","last_unstale":"2023-08-09T13:55:29.374346+0000","last_undegraded":"2023-08-09T13:55:29.374346+0000","last_fullsized":"2023-08-09T13:55:29.374346+0000","mapping_epoch":145,"log_start":"0'0","ondisk_log_start":"0'0","created":145,"last_epoch_clean":146,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:55:28.281790+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:55:28.281790+0000","last_clean_scrub_stamp":"2023-08-09T13:55:28.281790+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T20:18:12.293111+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"19.3a","version":"0'0","reported_seq":9,"reported_epoch":146,"state":"active+clean","last_fresh":"2023-08-09T13:55:29.387444+0000","last_change":"2023-08-09T13:55:29.383887+0000","last_active":"2023-08-09T13:55:29.387444+0000","last_peered":"2023-08-09T13:55:29.387444+0000","last_clean":"2023-08-09T13:55:29.387444+0000","last_became_active":"2023-08-09T13:55:29.383450+0000","last_became_peered":"2023-08-09T13:55:29.383450+0000","last_unstale":"2023-08-09T13:55:29.387444+0000","last_undegraded":"2023-08-09T13:55:29.387444+0000","last_fullsized":"2023-08-09T13:55:29.387444+0000","mapping_epoch":145,"log_start":"0'0","ondisk_log_start":"0'0","created":145,"last_epoch_clean":146,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:55:28.281790+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:55:28.281790+0000","last_clean_scrub_stamp":"2023-08-09T13:55:28.281790+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T16:00:03.485826+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"19.39","version":"0'0","reported_seq":9,"reported_epoch":146,"state":"active+clean","last_fresh":"2023-08-09T13:55:29.387384+0000","last_change":"2023-08-09T13:55:29.384060+0000","last_active":"2023-08-09T13:55:29.387384+0000","last_peered":"2023-08-09T13:55:29.387384+0000","last_clean":"2023-08-09T13:55:29.387384+0000","last_became_active":"2023-08-09T13:55:29.383747+0000","last_became_peered":"2023-08-09T13:55:29.383747+0000","last_unstale":"2023-08-09T13:55:29.387384+0000","last_undegraded":"2023-08-09T13:55:29.387384+0000","last_fullsized":"2023-08-09T13:55:29.387384+0000","mapping_epoch":145,"log_start":"0'0","ondisk_log_start":"0'0","created":145,"last_epoch_clean":146,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:55:28.281790+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:55:28.281790+0000","last_clean_scrub_stamp":"2023-08-09T13:55:28.281790+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T19:03:21.579607+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"19.38","version":"0'0","reported_seq":9,"reported_epoch":146,"state":"active+clean","last_fresh":"2023-08-09T13:55:29.387159+0000","last_change":"2023-08-09T13:55:29.380287+0000","last_active":"2023-08-09T13:55:29.387159+0000","last_peered":"2023-08-09T13:55:29.387159+0000","last_clean":"2023-08-09T13:55:29.387159+0000","last_became_active":"2023-08-09T13:55:29.379307+0000","last_became_peered":"2023-08-09T13:55:29.379307+0000","last_unstale":"2023-08-09T13:55:29.387159+0000","last_undegraded":"2023-08-09T13:55:29.387159+0000","last_fullsized":"2023-08-09T13:55:29.387159+0000","mapping_epoch":145,"log_start":"0'0","ondisk_log_start":"0'0","created":145,"last_epoch_clean":146,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:55:28.281790+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:55:28.281790+0000","last_clean_scrub_stamp":"2023-08-09T13:55:28.281790+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T23:51:16.157673+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"19.37","version":"0'0","reported_seq":9,"reported_epoch":146,"state":"active+clean","last_fresh":"2023-08-09T13:55:29.391218+0000","last_change":"2023-08-09T13:55:29.388275+0000","last_active":"2023-08-09T13:55:29.391218+0000","last_peered":"2023-08-09T13:55:29.391218+0000","last_clean":"2023-08-09T13:55:29.391218+0000","last_became_active":"2023-08-09T13:55:29.387985+0000","last_became_peered":"2023-08-09T13:55:29.387985+0000","last_unstale":"2023-08-09T13:55:29.391218+0000","last_undegraded":"2023-08-09T13:55:29.391218+0000","last_fullsized":"2023-08-09T13:55:29.391218+0000","mapping_epoch":145,"log_start":"0'0","ondisk_log_start":"0'0","created":145,"last_epoch_clean":146,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:55:28.281790+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:55:28.281790+0000","last_clean_scrub_stamp":"2023-08-09T13:55:28.281790+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-11T01:22:01.234257+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"19.36","version":"0'0","reported_seq":9,"reported_epoch":146,"state":"active+clean","last_fresh":"2023-08-09T13:55:29.391242+0000","last_change":"2023-08-09T13:55:29.390500+0000","last_active":"2023-08-09T13:55:29.391242+0000","last_peered":"2023-08-09T13:55:29.391242+0000","last_clean":"2023-08-09T13:55:29.391242+0000","last_became_active":"2023-08-09T13:55:29.390188+0000","last_became_peered":"2023-08-09T13:55:29.390188+0000","last_unstale":"2023-08-09T13:55:29.391242+0000","last_undegraded":"2023-08-09T13:55:29.391242+0000","last_fullsized":"2023-08-09T13:55:29.391242+0000","mapping_epoch":145,"log_start":"0'0","ondisk_log_start":"0'0","created":145,"last_epoch_clean":146,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:55:28.281790+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:55:28.281790+0000","last_clean_scrub_stamp":"2023-08-09T13:55:28.281790+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T22:01:14.561371+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"19.35","version":"0'0","reported_seq":9,"reported_epoch":146,"state":"active+clean","last_fresh":"2023-08-09T13:55:29.365979+0000","last_change":"2023-08-09T13:55:29.362462+0000","last_active":"2023-08-09T13:55:29.365979+0000","last_peered":"2023-08-09T13:55:29.365979+0000","last_clean":"2023-08-09T13:55:29.365979+0000","last_became_active":"2023-08-09T13:55:29.362150+0000","last_became_peered":"2023-08-09T13:55:29.362150+0000","last_unstale":"2023-08-09T13:55:29.365979+0000","last_undegraded":"2023-08-09T13:55:29.365979+0000","last_fullsized":"2023-08-09T13:55:29.365979+0000","mapping_epoch":145,"log_start":"0'0","ondisk_log_start":"0'0","created":145,"last_epoch_clean":146,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:55:28.281790+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:55:28.281790+0000","last_clean_scrub_stamp":"2023-08-09T13:55:28.281790+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-11T00:49:19.550086+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"19.34","version":"0'0","reported_seq":9,"reported_epoch":146,"state":"active+clean","last_fresh":"2023-08-09T13:55:29.382541+0000","last_change":"2023-08-09T13:55:29.377575+0000","last_active":"2023-08-09T13:55:29.382541+0000","last_peered":"2023-08-09T13:55:29.382541+0000","last_clean":"2023-08-09T13:55:29.382541+0000","last_became_active":"2023-08-09T13:55:29.377268+0000","last_became_peered":"2023-08-09T13:55:29.377268+0000","last_unstale":"2023-08-09T13:55:29.382541+0000","last_undegraded":"2023-08-09T13:55:29.382541+0000","last_fullsized":"2023-08-09T13:55:29.382541+0000","mapping_epoch":145,"log_start":"0'0","ondisk_log_start":"0'0","created":145,"last_epoch_clean":146,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:55:28.281790+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:55:28.281790+0000","last_clean_scrub_stamp":"2023-08-09T13:55:28.281790+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T16:24:58.970272+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"19.33","version":"0'0","reported_seq":9,"reported_epoch":146,"state":"active+clean","last_fresh":"2023-08-09T13:55:29.390050+0000","last_change":"2023-08-09T13:55:29.387036+0000","last_active":"2023-08-09T13:55:29.390050+0000","last_peered":"2023-08-09T13:55:29.390050+0000","last_clean":"2023-08-09T13:55:29.390050+0000","last_became_active":"2023-08-09T13:55:29.386717+0000","last_became_peered":"2023-08-09T13:55:29.386717+0000","last_unstale":"2023-08-09T13:55:29.390050+0000","last_undegraded":"2023-08-09T13:55:29.390050+0000","last_fullsized":"2023-08-09T13:55:29.390050+0000","mapping_epoch":145,"log_start":"0'0","ondisk_log_start":"0'0","created":145,"last_epoch_clean":146,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:55:28.281790+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:55:28.281790+0000","last_clean_scrub_stamp":"2023-08-09T13:55:28.281790+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T14:35:39.842197+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"19.32","version":"0'0","reported_seq":9,"reported_epoch":146,"state":"active+clean","last_fresh":"2023-08-09T13:55:29.373597+0000","last_change":"2023-08-09T13:55:29.370486+0000","last_active":"2023-08-09T13:55:29.373597+0000","last_peered":"2023-08-09T13:55:29.373597+0000","last_clean":"2023-08-09T13:55:29.373597+0000","last_became_active":"2023-08-09T13:55:29.370203+0000","last_became_peered":"2023-08-09T13:55:29.370203+0000","last_unstale":"2023-08-09T13:55:29.373597+0000","last_undegraded":"2023-08-09T13:55:29.373597+0000","last_fullsized":"2023-08-09T13:55:29.373597+0000","mapping_epoch":145,"log_start":"0'0","ondisk_log_start":"0'0","created":145,"last_epoch_clean":146,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:55:28.281790+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:55:28.281790+0000","last_clean_scrub_stamp":"2023-08-09T13:55:28.281790+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T14:04:05.932473+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"19.31","version":"0'0","reported_seq":9,"reported_epoch":146,"state":"active+clean","last_fresh":"2023-08-09T13:55:29.384296+0000","last_change":"2023-08-09T13:55:29.380328+0000","last_active":"2023-08-09T13:55:29.384296+0000","last_peered":"2023-08-09T13:55:29.384296+0000","last_clean":"2023-08-09T13:55:29.384296+0000","last_became_active":"2023-08-09T13:55:29.379188+0000","last_became_peered":"2023-08-09T13:55:29.379188+0000","last_unstale":"2023-08-09T13:55:29.384296+0000","last_undegraded":"2023-08-09T13:55:29.384296+0000","last_fullsized":"2023-08-09T13:55:29.384296+0000","mapping_epoch":145,"log_start":"0'0","ondisk_log_start":"0'0","created":145,"last_epoch_clean":146,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:55:28.281790+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:55:28.281790+0000","last_clean_scrub_stamp":"2023-08-09T13:55:28.281790+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T17:13:59.904533+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"19.30","version":"0'0","reported_seq":9,"reported_epoch":146,"state":"active+clean","last_fresh":"2023-08-09T13:55:29.361848+0000","last_change":"2023-08-09T13:55:29.358473+0000","last_active":"2023-08-09T13:55:29.361848+0000","last_peered":"2023-08-09T13:55:29.361848+0000","last_clean":"2023-08-09T13:55:29.361848+0000","last_became_active":"2023-08-09T13:55:29.358161+0000","last_became_peered":"2023-08-09T13:55:29.358161+0000","last_unstale":"2023-08-09T13:55:29.361848+0000","last_undegraded":"2023-08-09T13:55:29.361848+0000","last_fullsized":"2023-08-09T13:55:29.361848+0000","mapping_epoch":145,"log_start":"0'0","ondisk_log_start":"0'0","created":145,"last_epoch_clean":146,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:55:28.281790+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:55:28.281790+0000","last_clean_scrub_stamp":"2023-08-09T13:55:28.281790+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T21:37:44.655459+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"19.2f","version":"0'0","reported_seq":9,"reported_epoch":146,"state":"active+clean","last_fresh":"2023-08-09T13:55:29.385874+0000","last_change":"2023-08-09T13:55:29.380515+0000","last_active":"2023-08-09T13:55:29.385874+0000","last_peered":"2023-08-09T13:55:29.385874+0000","last_clean":"2023-08-09T13:55:29.385874+0000","last_became_active":"2023-08-09T13:55:29.380238+0000","last_became_peered":"2023-08-09T13:55:29.380238+0000","last_unstale":"2023-08-09T13:55:29.385874+0000","last_undegraded":"2023-08-09T13:55:29.385874+0000","last_fullsized":"2023-08-09T13:55:29.385874+0000","mapping_epoch":145,"log_start":"0'0","ondisk_log_start":"0'0","created":145,"last_epoch_clean":146,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:55:28.281790+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:55:28.281790+0000","last_clean_scrub_stamp":"2023-08-09T13:55:28.281790+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T17:10:44.094430+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"19.2e","version":"0'0","reported_seq":9,"reported_epoch":146,"state":"active+clean","last_fresh":"2023-08-09T13:55:29.389740+0000","last_change":"2023-08-09T13:55:29.388791+0000","last_active":"2023-08-09T13:55:29.389740+0000","last_peered":"2023-08-09T13:55:29.389740+0000","last_clean":"2023-08-09T13:55:29.389740+0000","last_became_active":"2023-08-09T13:55:29.388541+0000","last_became_peered":"2023-08-09T13:55:29.388541+0000","last_unstale":"2023-08-09T13:55:29.389740+0000","last_undegraded":"2023-08-09T13:55:29.389740+0000","last_fullsized":"2023-08-09T13:55:29.389740+0000","mapping_epoch":145,"log_start":"0'0","ondisk_log_start":"0'0","created":145,"last_epoch_clean":146,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:55:28.281790+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:55:28.281790+0000","last_clean_scrub_stamp":"2023-08-09T13:55:28.281790+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T15:57:57.305300+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"19.2d","version":"0'0","reported_seq":9,"reported_epoch":146,"state":"active+clean","last_fresh":"2023-08-09T13:55:29.388225+0000","last_change":"2023-08-09T13:55:29.385790+0000","last_active":"2023-08-09T13:55:29.388225+0000","last_peered":"2023-08-09T13:55:29.388225+0000","last_clean":"2023-08-09T13:55:29.388225+0000","last_became_active":"2023-08-09T13:55:29.385427+0000","last_became_peered":"2023-08-09T13:55:29.385427+0000","last_unstale":"2023-08-09T13:55:29.388225+0000","last_undegraded":"2023-08-09T13:55:29.388225+0000","last_fullsized":"2023-08-09T13:55:29.388225+0000","mapping_epoch":145,"log_start":"0'0","ondisk_log_start":"0'0","created":145,"last_epoch_clean":146,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:55:28.281790+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:55:28.281790+0000","last_clean_scrub_stamp":"2023-08-09T13:55:28.281790+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T23:24:53.963659+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"19.2c","version":"0'0","reported_seq":9,"reported_epoch":146,"state":"active+clean","last_fresh":"2023-08-09T13:55:29.372590+0000","last_change":"2023-08-09T13:55:29.369236+0000","last_active":"2023-08-09T13:55:29.372590+0000","last_peered":"2023-08-09T13:55:29.372590+0000","last_clean":"2023-08-09T13:55:29.372590+0000","last_became_active":"2023-08-09T13:55:29.368288+0000","last_became_peered":"2023-08-09T13:55:29.368288+0000","last_unstale":"2023-08-09T13:55:29.372590+0000","last_undegraded":"2023-08-09T13:55:29.372590+0000","last_fullsized":"2023-08-09T13:55:29.372590+0000","mapping_epoch":145,"log_start":"0'0","ondisk_log_start":"0'0","created":145,"last_epoch_clean":146,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:55:28.281790+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:55:28.281790+0000","last_clean_scrub_stamp":"2023-08-09T13:55:28.281790+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T17:12:58.100966+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"19.2b","version":"0'0","reported_seq":9,"reported_epoch":146,"state":"active+clean","last_fresh":"2023-08-09T13:55:29.374364+0000","last_change":"2023-08-09T13:55:29.370194+0000","last_active":"2023-08-09T13:55:29.374364+0000","last_peered":"2023-08-09T13:55:29.374364+0000","last_clean":"2023-08-09T13:55:29.374364+0000","last_became_active":"2023-08-09T13:55:29.366801+0000","last_became_peered":"2023-08-09T13:55:29.366801+0000","last_unstale":"2023-08-09T13:55:29.374364+0000","last_undegraded":"2023-08-09T13:55:29.374364+0000","last_fullsized":"2023-08-09T13:55:29.374364+0000","mapping_epoch":145,"log_start":"0'0","ondisk_log_start":"0'0","created":145,"last_epoch_clean":146,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:55:28.281790+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:55:28.281790+0000","last_clean_scrub_stamp":"2023-08-09T13:55:28.281790+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-11T01:25:32.168578+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"19.2a","version":"0'0","reported_seq":9,"reported_epoch":146,"state":"active+clean","last_fresh":"2023-08-09T13:55:29.372802+0000","last_change":"2023-08-09T13:55:29.367927+0000","last_active":"2023-08-09T13:55:29.372802+0000","last_peered":"2023-08-09T13:55:29.372802+0000","last_clean":"2023-08-09T13:55:29.372802+0000","last_became_active":"2023-08-09T13:55:29.367661+0000","last_became_peered":"2023-08-09T13:55:29.367661+0000","last_unstale":"2023-08-09T13:55:29.372802+0000","last_undegraded":"2023-08-09T13:55:29.372802+0000","last_fullsized":"2023-08-09T13:55:29.372802+0000","mapping_epoch":145,"log_start":"0'0","ondisk_log_start":"0'0","created":145,"last_epoch_clean":146,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:55:28.281790+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:55:28.281790+0000","last_clean_scrub_stamp":"2023-08-09T13:55:28.281790+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T21:07:25.695017+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"19.29","version":"0'0","reported_seq":9,"reported_epoch":146,"state":"active+clean","last_fresh":"2023-08-09T13:55:29.391562+0000","last_change":"2023-08-09T13:55:29.389415+0000","last_active":"2023-08-09T13:55:29.391562+0000","last_peered":"2023-08-09T13:55:29.391562+0000","last_clean":"2023-08-09T13:55:29.391562+0000","last_became_active":"2023-08-09T13:55:29.388766+0000","last_became_peered":"2023-08-09T13:55:29.388766+0000","last_unstale":"2023-08-09T13:55:29.391562+0000","last_undegraded":"2023-08-09T13:55:29.391562+0000","last_fullsized":"2023-08-09T13:55:29.391562+0000","mapping_epoch":145,"log_start":"0'0","ondisk_log_start":"0'0","created":145,"last_epoch_clean":146,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:55:28.281790+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:55:28.281790+0000","last_clean_scrub_stamp":"2023-08-09T13:55:28.281790+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-11T00:12:19.333562+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"19.28","version":"0'0","reported_seq":9,"reported_epoch":146,"state":"active+clean","last_fresh":"2023-08-09T13:55:29.364335+0000","last_change":"2023-08-09T13:55:29.359649+0000","last_active":"2023-08-09T13:55:29.364335+0000","last_peered":"2023-08-09T13:55:29.364335+0000","last_clean":"2023-08-09T13:55:29.364335+0000","last_became_active":"2023-08-09T13:55:29.359319+0000","last_became_peered":"2023-08-09T13:55:29.359319+0000","last_unstale":"2023-08-09T13:55:29.364335+0000","last_undegraded":"2023-08-09T13:55:29.364335+0000","last_fullsized":"2023-08-09T13:55:29.364335+0000","mapping_epoch":145,"log_start":"0'0","ondisk_log_start":"0'0","created":145,"last_epoch_clean":146,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:55:28.281790+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:55:28.281790+0000","last_clean_scrub_stamp":"2023-08-09T13:55:28.281790+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T18:18:16.953218+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"19.27","version":"0'0","reported_seq":9,"reported_epoch":146,"state":"active+clean","last_fresh":"2023-08-09T13:55:29.380340+0000","last_change":"2023-08-09T13:55:29.379554+0000","last_active":"2023-08-09T13:55:29.380340+0000","last_peered":"2023-08-09T13:55:29.380340+0000","last_clean":"2023-08-09T13:55:29.380340+0000","last_became_active":"2023-08-09T13:55:29.379266+0000","last_became_peered":"2023-08-09T13:55:29.379266+0000","last_unstale":"2023-08-09T13:55:29.380340+0000","last_undegraded":"2023-08-09T13:55:29.380340+0000","last_fullsized":"2023-08-09T13:55:29.380340+0000","mapping_epoch":145,"log_start":"0'0","ondisk_log_start":"0'0","created":145,"last_epoch_clean":146,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:55:28.281790+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:55:28.281790+0000","last_clean_scrub_stamp":"2023-08-09T13:55:28.281790+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-11T01:16:10.186553+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"19.26","version":"0'0","reported_seq":9,"reported_epoch":146,"state":"active+clean","last_fresh":"2023-08-09T13:55:29.361872+0000","last_change":"2023-08-09T13:55:29.358663+0000","last_active":"2023-08-09T13:55:29.361872+0000","last_peered":"2023-08-09T13:55:29.361872+0000","last_clean":"2023-08-09T13:55:29.361872+0000","last_became_active":"2023-08-09T13:55:29.358306+0000","last_became_peered":"2023-08-09T13:55:29.358306+0000","last_unstale":"2023-08-09T13:55:29.361872+0000","last_undegraded":"2023-08-09T13:55:29.361872+0000","last_fullsized":"2023-08-09T13:55:29.361872+0000","mapping_epoch":145,"log_start":"0'0","ondisk_log_start":"0'0","created":145,"last_epoch_clean":146,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:55:28.281790+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:55:28.281790+0000","last_clean_scrub_stamp":"2023-08-09T13:55:28.281790+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T21:08:27.586494+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"19.25","version":"0'0","reported_seq":9,"reported_epoch":146,"state":"active+clean","last_fresh":"2023-08-09T13:55:29.390087+0000","last_change":"2023-08-09T13:55:29.387223+0000","last_active":"2023-08-09T13:55:29.390087+0000","last_peered":"2023-08-09T13:55:29.390087+0000","last_clean":"2023-08-09T13:55:29.390087+0000","last_became_active":"2023-08-09T13:55:29.386948+0000","last_became_peered":"2023-08-09T13:55:29.386948+0000","last_unstale":"2023-08-09T13:55:29.390087+0000","last_undegraded":"2023-08-09T13:55:29.390087+0000","last_fullsized":"2023-08-09T13:55:29.390087+0000","mapping_epoch":145,"log_start":"0'0","ondisk_log_start":"0'0","created":145,"last_epoch_clean":146,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:55:28.281790+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:55:28.281790+0000","last_clean_scrub_stamp":"2023-08-09T13:55:28.281790+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-11T01:52:05.886750+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"19.24","version":"0'0","reported_seq":9,"reported_epoch":146,"state":"active+clean","last_fresh":"2023-08-09T13:55:29.373086+0000","last_change":"2023-08-09T13:55:29.367568+0000","last_active":"2023-08-09T13:55:29.373086+0000","last_peered":"2023-08-09T13:55:29.373086+0000","last_clean":"2023-08-09T13:55:29.373086+0000","last_became_active":"2023-08-09T13:55:29.366511+0000","last_became_peered":"2023-08-09T13:55:29.366511+0000","last_unstale":"2023-08-09T13:55:29.373086+0000","last_undegraded":"2023-08-09T13:55:29.373086+0000","last_fullsized":"2023-08-09T13:55:29.373086+0000","mapping_epoch":145,"log_start":"0'0","ondisk_log_start":"0'0","created":145,"last_epoch_clean":146,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:55:28.281790+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:55:28.281790+0000","last_clean_scrub_stamp":"2023-08-09T13:55:28.281790+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T19:17:25.795688+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"19.23","version":"0'0","reported_seq":9,"reported_epoch":146,"state":"active+clean","last_fresh":"2023-08-09T13:55:29.384436+0000","last_change":"2023-08-09T13:55:29.382018+0000","last_active":"2023-08-09T13:55:29.384436+0000","last_peered":"2023-08-09T13:55:29.384436+0000","last_clean":"2023-08-09T13:55:29.384436+0000","last_became_active":"2023-08-09T13:55:29.381757+0000","last_became_peered":"2023-08-09T13:55:29.381757+0000","last_unstale":"2023-08-09T13:55:29.384436+0000","last_undegraded":"2023-08-09T13:55:29.384436+0000","last_fullsized":"2023-08-09T13:55:29.384436+0000","mapping_epoch":145,"log_start":"0'0","ondisk_log_start":"0'0","created":145,"last_epoch_clean":146,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:55:28.281790+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:55:28.281790+0000","last_clean_scrub_stamp":"2023-08-09T13:55:28.281790+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T19:34:27.895516+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"19.22","version":"0'0","reported_seq":9,"reported_epoch":146,"state":"active+clean","last_fresh":"2023-08-09T13:55:29.382771+0000","last_change":"2023-08-09T13:55:29.377418+0000","last_active":"2023-08-09T13:55:29.382771+0000","last_peered":"2023-08-09T13:55:29.382771+0000","last_clean":"2023-08-09T13:55:29.382771+0000","last_became_active":"2023-08-09T13:55:29.377096+0000","last_became_peered":"2023-08-09T13:55:29.377096+0000","last_unstale":"2023-08-09T13:55:29.382771+0000","last_undegraded":"2023-08-09T13:55:29.382771+0000","last_fullsized":"2023-08-09T13:55:29.382771+0000","mapping_epoch":145,"log_start":"0'0","ondisk_log_start":"0'0","created":145,"last_epoch_clean":146,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:55:28.281790+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:55:28.281790+0000","last_clean_scrub_stamp":"2023-08-09T13:55:28.281790+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T19:37:22.045044+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"19.21","version":"0'0","reported_seq":9,"reported_epoch":146,"state":"active+clean","last_fresh":"2023-08-09T13:55:29.381074+0000","last_change":"2023-08-09T13:55:29.379944+0000","last_active":"2023-08-09T13:55:29.381074+0000","last_peered":"2023-08-09T13:55:29.381074+0000","last_clean":"2023-08-09T13:55:29.381074+0000","last_became_active":"2023-08-09T13:55:29.379602+0000","last_became_peered":"2023-08-09T13:55:29.379602+0000","last_unstale":"2023-08-09T13:55:29.381074+0000","last_undegraded":"2023-08-09T13:55:29.381074+0000","last_fullsized":"2023-08-09T13:55:29.381074+0000","mapping_epoch":145,"log_start":"0'0","ondisk_log_start":"0'0","created":145,"last_epoch_clean":146,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:55:28.281790+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:55:28.281790+0000","last_clean_scrub_stamp":"2023-08-09T13:55:28.281790+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T21:02:16.712303+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"19.20","version":"0'0","reported_seq":9,"reported_epoch":146,"state":"active+clean","last_fresh":"2023-08-09T13:55:29.389771+0000","last_change":"2023-08-09T13:55:29.384552+0000","last_active":"2023-08-09T13:55:29.389771+0000","last_peered":"2023-08-09T13:55:29.389771+0000","last_clean":"2023-08-09T13:55:29.389771+0000","last_became_active":"2023-08-09T13:55:29.383361+0000","last_became_peered":"2023-08-09T13:55:29.383361+0000","last_unstale":"2023-08-09T13:55:29.389771+0000","last_undegraded":"2023-08-09T13:55:29.389771+0000","last_fullsized":"2023-08-09T13:55:29.389771+0000","mapping_epoch":145,"log_start":"0'0","ondisk_log_start":"0'0","created":145,"last_epoch_clean":146,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:55:28.281790+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:55:28.281790+0000","last_clean_scrub_stamp":"2023-08-09T13:55:28.281790+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-11T00:40:08.873250+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"19.1e","version":"0'0","reported_seq":9,"reported_epoch":146,"state":"active+clean","last_fresh":"2023-08-09T13:55:29.373187+0000","last_change":"2023-08-09T13:55:29.370200+0000","last_active":"2023-08-09T13:55:29.373187+0000","last_peered":"2023-08-09T13:55:29.373187+0000","last_clean":"2023-08-09T13:55:29.373187+0000","last_became_active":"2023-08-09T13:55:29.369913+0000","last_became_peered":"2023-08-09T13:55:29.369913+0000","last_unstale":"2023-08-09T13:55:29.373187+0000","last_undegraded":"2023-08-09T13:55:29.373187+0000","last_fullsized":"2023-08-09T13:55:29.373187+0000","mapping_epoch":145,"log_start":"0'0","ondisk_log_start":"0'0","created":145,"last_epoch_clean":146,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:55:28.281790+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:55:28.281790+0000","last_clean_scrub_stamp":"2023-08-09T13:55:28.281790+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T20:10:16.052483+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"18.1f","version":"146'4","reported_seq":15,"reported_epoch":146,"state":"active+clean","last_fresh":"2023-08-09T13:55:29.386444+0000","last_change":"2023-08-09T13:55:28.330435+0000","last_active":"2023-08-09T13:55:29.386444+0000","last_peered":"2023-08-09T13:55:29.386444+0000","last_clean":"2023-08-09T13:55:29.386444+0000","last_became_active":"2023-08-09T13:55:28.329974+0000","last_became_peered":"2023-08-09T13:55:28.329974+0000","last_unstale":"2023-08-09T13:55:29.386444+0000","last_undegraded":"2023-08-09T13:55:29.386444+0000","last_fullsized":"2023-08-09T13:55:29.386444+0000","mapping_epoch":144,"log_start":"0'0","ondisk_log_start":"0'0","created":144,"last_epoch_clean":145,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:55:27.277343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:55:27.277343+0000","last_clean_scrub_stamp":"2023-08-09T13:55:27.277343+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T23:38:51.687655+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"19.1f","version":"0'0","reported_seq":9,"reported_epoch":146,"state":"active+clean","last_fresh":"2023-08-09T13:55:29.382345+0000","last_change":"2023-08-09T13:55:29.376050+0000","last_active":"2023-08-09T13:55:29.382345+0000","last_peered":"2023-08-09T13:55:29.382345+0000","last_clean":"2023-08-09T13:55:29.382345+0000","last_became_active":"2023-08-09T13:55:29.375762+0000","last_became_peered":"2023-08-09T13:55:29.375762+0000","last_unstale":"2023-08-09T13:55:29.382345+0000","last_undegraded":"2023-08-09T13:55:29.382345+0000","last_fullsized":"2023-08-09T13:55:29.382345+0000","mapping_epoch":145,"log_start":"0'0","ondisk_log_start":"0'0","created":145,"last_epoch_clean":146,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:55:28.281790+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:55:28.281790+0000","last_clean_scrub_stamp":"2023-08-09T13:55:28.281790+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T17:52:18.127836+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"18.1e","version":"146'1","reported_seq":12,"reported_epoch":146,"state":"active+clean","last_fresh":"2023-08-09T13:55:29.386221+0000","last_change":"2023-08-09T13:55:28.314187+0000","last_active":"2023-08-09T13:55:29.386221+0000","last_peered":"2023-08-09T13:55:29.386221+0000","last_clean":"2023-08-09T13:55:29.386221+0000","last_became_active":"2023-08-09T13:55:28.313832+0000","last_became_peered":"2023-08-09T13:55:28.313832+0000","last_unstale":"2023-08-09T13:55:29.386221+0000","last_undegraded":"2023-08-09T13:55:29.386221+0000","last_fullsized":"2023-08-09T13:55:29.386221+0000","mapping_epoch":144,"log_start":"0'0","ondisk_log_start":"0'0","created":144,"last_epoch_clean":145,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:55:27.277343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:55:27.277343+0000","last_clean_scrub_stamp":"2023-08-09T13:55:27.277343+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T22:49:19.884883+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"19.1c","version":"0'0","reported_seq":9,"reported_epoch":146,"state":"active+clean","last_fresh":"2023-08-09T13:55:29.389833+0000","last_change":"2023-08-09T13:55:29.386097+0000","last_active":"2023-08-09T13:55:29.389833+0000","last_peered":"2023-08-09T13:55:29.389833+0000","last_clean":"2023-08-09T13:55:29.389833+0000","last_became_active":"2023-08-09T13:55:29.385801+0000","last_became_peered":"2023-08-09T13:55:29.385801+0000","last_unstale":"2023-08-09T13:55:29.389833+0000","last_undegraded":"2023-08-09T13:55:29.389833+0000","last_fullsized":"2023-08-09T13:55:29.389833+0000","mapping_epoch":145,"log_start":"0'0","ondisk_log_start":"0'0","created":145,"last_epoch_clean":146,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:55:28.281790+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:55:28.281790+0000","last_clean_scrub_stamp":"2023-08-09T13:55:28.281790+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T14:50:08.755167+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"18.1d","version":"146'2","reported_seq":13,"reported_epoch":146,"state":"active+clean","last_fresh":"2023-08-09T13:55:29.365637+0000","last_change":"2023-08-09T13:55:28.323885+0000","last_active":"2023-08-09T13:55:29.365637+0000","last_peered":"2023-08-09T13:55:29.365637+0000","last_clean":"2023-08-09T13:55:29.365637+0000","last_became_active":"2023-08-09T13:55:28.323620+0000","last_became_peered":"2023-08-09T13:55:28.323620+0000","last_unstale":"2023-08-09T13:55:29.365637+0000","last_undegraded":"2023-08-09T13:55:29.365637+0000","last_fullsized":"2023-08-09T13:55:29.365637+0000","mapping_epoch":144,"log_start":"0'0","ondisk_log_start":"0'0","created":144,"last_epoch_clean":145,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:55:27.277343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:55:27.277343+0000","last_clean_scrub_stamp":"2023-08-09T13:55:27.277343+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T23:58:37.536745+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"19.1d","version":"0'0","reported_seq":9,"reported_epoch":146,"state":"active+clean","last_fresh":"2023-08-09T13:55:29.374137+0000","last_change":"2023-08-09T13:55:29.369649+0000","last_active":"2023-08-09T13:55:29.374137+0000","last_peered":"2023-08-09T13:55:29.374137+0000","last_clean":"2023-08-09T13:55:29.374137+0000","last_became_active":"2023-08-09T13:55:29.367832+0000","last_became_peered":"2023-08-09T13:55:29.367832+0000","last_unstale":"2023-08-09T13:55:29.374137+0000","last_undegraded":"2023-08-09T13:55:29.374137+0000","last_fullsized":"2023-08-09T13:55:29.374137+0000","mapping_epoch":145,"log_start":"0'0","ondisk_log_start":"0'0","created":145,"last_epoch_clean":146,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:55:28.281790+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:55:28.281790+0000","last_clean_scrub_stamp":"2023-08-09T13:55:28.281790+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T22:16:04.137516+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"18.1c","version":"146'2","reported_seq":13,"reported_epoch":146,"state":"active+clean","last_fresh":"2023-08-09T13:55:29.373744+0000","last_change":"2023-08-09T13:55:28.316189+0000","last_active":"2023-08-09T13:55:29.373744+0000","last_peered":"2023-08-09T13:55:29.373744+0000","last_clean":"2023-08-09T13:55:29.373744+0000","last_became_active":"2023-08-09T13:55:28.315766+0000","last_became_peered":"2023-08-09T13:55:28.315766+0000","last_unstale":"2023-08-09T13:55:29.373744+0000","last_undegraded":"2023-08-09T13:55:29.373744+0000","last_fullsized":"2023-08-09T13:55:29.373744+0000","mapping_epoch":144,"log_start":"0'0","ondisk_log_start":"0'0","created":144,"last_epoch_clean":145,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:55:27.277343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:55:27.277343+0000","last_clean_scrub_stamp":"2023-08-09T13:55:27.277343+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-11T01:38:08.974687+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"19.1a","version":"0'0","reported_seq":9,"reported_epoch":146,"state":"active+clean","last_fresh":"2023-08-09T13:55:29.389247+0000","last_change":"2023-08-09T13:55:29.387307+0000","last_active":"2023-08-09T13:55:29.389247+0000","last_peered":"2023-08-09T13:55:29.389247+0000","last_clean":"2023-08-09T13:55:29.389247+0000","last_became_active":"2023-08-09T13:55:29.385513+0000","last_became_peered":"2023-08-09T13:55:29.385513+0000","last_unstale":"2023-08-09T13:55:29.389247+0000","last_undegraded":"2023-08-09T13:55:29.389247+0000","last_fullsized":"2023-08-09T13:55:29.389247+0000","mapping_epoch":145,"log_start":"0'0","ondisk_log_start":"0'0","created":145,"last_epoch_clean":146,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:55:28.281790+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:55:28.281790+0000","last_clean_scrub_stamp":"2023-08-09T13:55:28.281790+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T22:04:16.087605+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"18.1b","version":"146'1","reported_seq":12,"reported_epoch":146,"state":"active+clean","last_fresh":"2023-08-09T13:55:29.373764+0000","last_change":"2023-08-09T13:55:28.321515+0000","last_active":"2023-08-09T13:55:29.373764+0000","last_peered":"2023-08-09T13:55:29.373764+0000","last_clean":"2023-08-09T13:55:29.373764+0000","last_became_active":"2023-08-09T13:55:28.319623+0000","last_became_peered":"2023-08-09T13:55:28.319623+0000","last_unstale":"2023-08-09T13:55:29.373764+0000","last_undegraded":"2023-08-09T13:55:29.373764+0000","last_fullsized":"2023-08-09T13:55:29.373764+0000","mapping_epoch":144,"log_start":"0'0","ondisk_log_start":"0'0","created":144,"last_epoch_clean":145,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:55:27.277343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:55:27.277343+0000","last_clean_scrub_stamp":"2023-08-09T13:55:27.277343+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T18:14:21.871364+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"19.1b","version":"0'0","reported_seq":9,"reported_epoch":146,"state":"active+clean","last_fresh":"2023-08-09T13:55:29.387426+0000","last_change":"2023-08-09T13:55:29.383710+0000","last_active":"2023-08-09T13:55:29.387426+0000","last_peered":"2023-08-09T13:55:29.387426+0000","last_clean":"2023-08-09T13:55:29.387426+0000","last_became_active":"2023-08-09T13:55:29.383372+0000","last_became_peered":"2023-08-09T13:55:29.383372+0000","last_unstale":"2023-08-09T13:55:29.387426+0000","last_undegraded":"2023-08-09T13:55:29.387426+0000","last_fullsized":"2023-08-09T13:55:29.387426+0000","mapping_epoch":145,"log_start":"0'0","ondisk_log_start":"0'0","created":145,"last_epoch_clean":146,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:55:28.281790+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:55:28.281790+0000","last_clean_scrub_stamp":"2023-08-09T13:55:28.281790+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-11T00:02:15.023221+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"18.1a","version":"146'1","reported_seq":12,"reported_epoch":146,"state":"active+clean","last_fresh":"2023-08-09T13:55:29.370556+0000","last_change":"2023-08-09T13:55:28.323112+0000","last_active":"2023-08-09T13:55:29.370556+0000","last_peered":"2023-08-09T13:55:29.370556+0000","last_clean":"2023-08-09T13:55:29.370556+0000","last_became_active":"2023-08-09T13:55:28.322814+0000","last_became_peered":"2023-08-09T13:55:28.322814+0000","last_unstale":"2023-08-09T13:55:29.370556+0000","last_undegraded":"2023-08-09T13:55:29.370556+0000","last_fullsized":"2023-08-09T13:55:29.370556+0000","mapping_epoch":144,"log_start":"0'0","ondisk_log_start":"0'0","created":144,"last_epoch_clean":145,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:55:27.277343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:55:27.277343+0000","last_clean_scrub_stamp":"2023-08-09T13:55:27.277343+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T14:11:35.933204+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"19.18","version":"0'0","reported_seq":9,"reported_epoch":146,"state":"active+clean","last_fresh":"2023-08-09T13:55:29.387140+0000","last_change":"2023-08-09T13:55:29.380096+0000","last_active":"2023-08-09T13:55:29.387140+0000","last_peered":"2023-08-09T13:55:29.387140+0000","last_clean":"2023-08-09T13:55:29.387140+0000","last_became_active":"2023-08-09T13:55:29.378924+0000","last_became_peered":"2023-08-09T13:55:29.378924+0000","last_unstale":"2023-08-09T13:55:29.387140+0000","last_undegraded":"2023-08-09T13:55:29.387140+0000","last_fullsized":"2023-08-09T13:55:29.387140+0000","mapping_epoch":145,"log_start":"0'0","ondisk_log_start":"0'0","created":145,"last_epoch_clean":146,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:55:28.281790+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:55:28.281790+0000","last_clean_scrub_stamp":"2023-08-09T13:55:28.281790+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T18:31:58.455662+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"18.19","version":"0'0","reported_seq":11,"reported_epoch":146,"state":"active+clean","last_fresh":"2023-08-09T13:55:29.364645+0000","last_change":"2023-08-09T13:55:28.333674+0000","last_active":"2023-08-09T13:55:29.364645+0000","last_peered":"2023-08-09T13:55:29.364645+0000","last_clean":"2023-08-09T13:55:29.364645+0000","last_became_active":"2023-08-09T13:55:28.333331+0000","last_became_peered":"2023-08-09T13:55:28.333331+0000","last_unstale":"2023-08-09T13:55:29.364645+0000","last_undegraded":"2023-08-09T13:55:29.364645+0000","last_fullsized":"2023-08-09T13:55:29.364645+0000","mapping_epoch":144,"log_start":"0'0","ondisk_log_start":"0'0","created":144,"last_epoch_clean":145,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:55:27.277343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:55:27.277343+0000","last_clean_scrub_stamp":"2023-08-09T13:55:27.277343+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T19:05:36.216831+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"19.19","version":"0'0","reported_seq":9,"reported_epoch":146,"state":"active+clean","last_fresh":"2023-08-09T13:55:29.365807+0000","last_change":"2023-08-09T13:55:29.362843+0000","last_active":"2023-08-09T13:55:29.365807+0000","last_peered":"2023-08-09T13:55:29.365807+0000","last_clean":"2023-08-09T13:55:29.365807+0000","last_became_active":"2023-08-09T13:55:29.362566+0000","last_became_peered":"2023-08-09T13:55:29.362566+0000","last_unstale":"2023-08-09T13:55:29.365807+0000","last_undegraded":"2023-08-09T13:55:29.365807+0000","last_fullsized":"2023-08-09T13:55:29.365807+0000","mapping_epoch":145,"log_start":"0'0","ondisk_log_start":"0'0","created":145,"last_epoch_clean":146,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:55:28.281790+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:55:28.281790+0000","last_clean_scrub_stamp":"2023-08-09T13:55:28.281790+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-11T01:09:55.545777+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"18.18","version":"0'0","reported_seq":11,"reported_epoch":146,"state":"active+clean","last_fresh":"2023-08-09T13:55:29.353641+0000","last_change":"2023-08-09T13:55:28.324609+0000","last_active":"2023-08-09T13:55:29.353641+0000","last_peered":"2023-08-09T13:55:29.353641+0000","last_clean":"2023-08-09T13:55:29.353641+0000","last_became_active":"2023-08-09T13:55:28.319370+0000","last_became_peered":"2023-08-09T13:55:28.319370+0000","last_unstale":"2023-08-09T13:55:29.353641+0000","last_undegraded":"2023-08-09T13:55:29.353641+0000","last_fullsized":"2023-08-09T13:55:29.353641+0000","mapping_epoch":144,"log_start":"0'0","ondisk_log_start":"0'0","created":144,"last_epoch_clean":145,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:55:27.277343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:55:27.277343+0000","last_clean_scrub_stamp":"2023-08-09T13:55:27.277343+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T21:43:02.662378+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"19.e","version":"0'0","reported_seq":9,"reported_epoch":146,"state":"active+clean","last_fresh":"2023-08-09T13:55:29.382294+0000","last_change":"2023-08-09T13:55:29.377981+0000","last_active":"2023-08-09T13:55:29.382294+0000","last_peered":"2023-08-09T13:55:29.382294+0000","last_clean":"2023-08-09T13:55:29.382294+0000","last_became_active":"2023-08-09T13:55:29.377680+0000","last_became_peered":"2023-08-09T13:55:29.377680+0000","last_unstale":"2023-08-09T13:55:29.382294+0000","last_undegraded":"2023-08-09T13:55:29.382294+0000","last_fullsized":"2023-08-09T13:55:29.382294+0000","mapping_epoch":145,"log_start":"0'0","ondisk_log_start":"0'0","created":145,"last_epoch_clean":146,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:55:28.281790+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:55:28.281790+0000","last_clean_scrub_stamp":"2023-08-09T13:55:28.281790+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-11T00:35:01.430681+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"18.f","version":"146'1","reported_seq":12,"reported_epoch":146,"state":"active+clean","last_fresh":"2023-08-09T13:55:29.390727+0000","last_change":"2023-08-09T13:55:28.329080+0000","last_active":"2023-08-09T13:55:29.390727+0000","last_peered":"2023-08-09T13:55:29.390727+0000","last_clean":"2023-08-09T13:55:29.390727+0000","last_became_active":"2023-08-09T13:55:28.328777+0000","last_became_peered":"2023-08-09T13:55:28.328777+0000","last_unstale":"2023-08-09T13:55:29.390727+0000","last_undegraded":"2023-08-09T13:55:29.390727+0000","last_fullsized":"2023-08-09T13:55:29.390727+0000","mapping_epoch":144,"log_start":"0'0","ondisk_log_start":"0'0","created":144,"last_epoch_clean":145,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:55:27.277343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:55:27.277343+0000","last_clean_scrub_stamp":"2023-08-09T13:55:27.277343+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T22:33:15.141151+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"19.f","version":"0'0","reported_seq":9,"reported_epoch":146,"state":"active+clean","last_fresh":"2023-08-09T13:55:29.387430+0000","last_change":"2023-08-09T13:55:29.370021+0000","last_active":"2023-08-09T13:55:29.387430+0000","last_peered":"2023-08-09T13:55:29.387430+0000","last_clean":"2023-08-09T13:55:29.387430+0000","last_became_active":"2023-08-09T13:55:29.368833+0000","last_became_peered":"2023-08-09T13:55:29.368833+0000","last_unstale":"2023-08-09T13:55:29.387430+0000","last_undegraded":"2023-08-09T13:55:29.387430+0000","last_fullsized":"2023-08-09T13:55:29.387430+0000","mapping_epoch":145,"log_start":"0'0","ondisk_log_start":"0'0","created":145,"last_epoch_clean":146,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:55:28.281790+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:55:28.281790+0000","last_clean_scrub_stamp":"2023-08-09T13:55:28.281790+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T19:08:27.717114+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"18.e","version":"146'2","reported_seq":13,"reported_epoch":146,"state":"active+clean","last_fresh":"2023-08-09T13:55:29.376770+0000","last_change":"2023-08-09T13:55:28.323302+0000","last_active":"2023-08-09T13:55:29.376770+0000","last_peered":"2023-08-09T13:55:29.376770+0000","last_clean":"2023-08-09T13:55:29.376770+0000","last_became_active":"2023-08-09T13:55:28.323032+0000","last_became_peered":"2023-08-09T13:55:28.323032+0000","last_unstale":"2023-08-09T13:55:29.376770+0000","last_undegraded":"2023-08-09T13:55:29.376770+0000","last_fullsized":"2023-08-09T13:55:29.376770+0000","mapping_epoch":144,"log_start":"0'0","ondisk_log_start":"0'0","created":144,"last_epoch_clean":145,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:55:27.277343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:55:27.277343+0000","last_clean_scrub_stamp":"2023-08-09T13:55:27.277343+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T23:31:38.128489+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"19.c","version":"0'0","reported_seq":9,"reported_epoch":146,"state":"active+clean","last_fresh":"2023-08-09T13:55:29.363728+0000","last_change":"2023-08-09T13:55:29.361604+0000","last_active":"2023-08-09T13:55:29.363728+0000","last_peered":"2023-08-09T13:55:29.363728+0000","last_clean":"2023-08-09T13:55:29.363728+0000","last_became_active":"2023-08-09T13:55:29.361295+0000","last_became_peered":"2023-08-09T13:55:29.361295+0000","last_unstale":"2023-08-09T13:55:29.363728+0000","last_undegraded":"2023-08-09T13:55:29.363728+0000","last_fullsized":"2023-08-09T13:55:29.363728+0000","mapping_epoch":145,"log_start":"0'0","ondisk_log_start":"0'0","created":145,"last_epoch_clean":146,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:55:28.281790+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:55:28.281790+0000","last_clean_scrub_stamp":"2023-08-09T13:55:28.281790+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-11T00:36:30.884672+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"18.d","version":"146'1","reported_seq":12,"reported_epoch":146,"state":"active+clean","last_fresh":"2023-08-09T13:55:29.384282+0000","last_change":"2023-08-09T13:55:28.321976+0000","last_active":"2023-08-09T13:55:29.384282+0000","last_peered":"2023-08-09T13:55:29.384282+0000","last_clean":"2023-08-09T13:55:29.384282+0000","last_became_active":"2023-08-09T13:55:28.321604+0000","last_became_peered":"2023-08-09T13:55:28.321604+0000","last_unstale":"2023-08-09T13:55:29.384282+0000","last_undegraded":"2023-08-09T13:55:29.384282+0000","last_fullsized":"2023-08-09T13:55:29.384282+0000","mapping_epoch":144,"log_start":"0'0","ondisk_log_start":"0'0","created":144,"last_epoch_clean":145,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:55:27.277343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:55:27.277343+0000","last_clean_scrub_stamp":"2023-08-09T13:55:27.277343+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T22:52:07.842471+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"19.d","version":"0'0","reported_seq":9,"reported_epoch":146,"state":"active+clean","last_fresh":"2023-08-09T13:55:29.367390+0000","last_change":"2023-08-09T13:55:29.365501+0000","last_active":"2023-08-09T13:55:29.367390+0000","last_peered":"2023-08-09T13:55:29.367390+0000","last_clean":"2023-08-09T13:55:29.367390+0000","last_became_active":"2023-08-09T13:55:29.365217+0000","last_became_peered":"2023-08-09T13:55:29.365217+0000","last_unstale":"2023-08-09T13:55:29.367390+0000","last_undegraded":"2023-08-09T13:55:29.367390+0000","last_fullsized":"2023-08-09T13:55:29.367390+0000","mapping_epoch":145,"log_start":"0'0","ondisk_log_start":"0'0","created":145,"last_epoch_clean":146,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:55:28.281790+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:55:28.281790+0000","last_clean_scrub_stamp":"2023-08-09T13:55:28.281790+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T20:01:41.336446+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"18.c","version":"0'0","reported_seq":11,"reported_epoch":146,"state":"active+clean","last_fresh":"2023-08-09T13:55:29.362642+0000","last_change":"2023-08-09T13:55:28.315212+0000","last_active":"2023-08-09T13:55:29.362642+0000","last_peered":"2023-08-09T13:55:29.362642+0000","last_clean":"2023-08-09T13:55:29.362642+0000","last_became_active":"2023-08-09T13:55:28.314816+0000","last_became_peered":"2023-08-09T13:55:28.314816+0000","last_unstale":"2023-08-09T13:55:29.362642+0000","last_undegraded":"2023-08-09T13:55:29.362642+0000","last_fullsized":"2023-08-09T13:55:29.362642+0000","mapping_epoch":144,"log_start":"0'0","ondisk_log_start":"0'0","created":144,"last_epoch_clean":145,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:55:27.277343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:55:27.277343+0000","last_clean_scrub_stamp":"2023-08-09T13:55:27.277343+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T17:38:40.269099+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"19.a","version":"0'0","reported_seq":9,"reported_epoch":146,"state":"active+clean","last_fresh":"2023-08-09T13:55:29.389981+0000","last_change":"2023-08-09T13:55:29.386663+0000","last_active":"2023-08-09T13:55:29.389981+0000","last_peered":"2023-08-09T13:55:29.389981+0000","last_clean":"2023-08-09T13:55:29.389981+0000","last_became_active":"2023-08-09T13:55:29.386280+0000","last_became_peered":"2023-08-09T13:55:29.386280+0000","last_unstale":"2023-08-09T13:55:29.389981+0000","last_undegraded":"2023-08-09T13:55:29.389981+0000","last_fullsized":"2023-08-09T13:55:29.389981+0000","mapping_epoch":145,"log_start":"0'0","ondisk_log_start":"0'0","created":145,"last_epoch_clean":146,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:55:28.281790+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:55:28.281790+0000","last_clean_scrub_stamp":"2023-08-09T13:55:28.281790+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T19:07:50.742521+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"18.b","version":"146'2","reported_seq":13,"reported_epoch":146,"state":"active+clean","last_fresh":"2023-08-09T13:55:29.389375+0000","last_change":"2023-08-09T13:55:28.316450+0000","last_active":"2023-08-09T13:55:29.389375+0000","last_peered":"2023-08-09T13:55:29.389375+0000","last_clean":"2023-08-09T13:55:29.389375+0000","last_became_active":"2023-08-09T13:55:28.316125+0000","last_became_peered":"2023-08-09T13:55:28.316125+0000","last_unstale":"2023-08-09T13:55:29.389375+0000","last_undegraded":"2023-08-09T13:55:29.389375+0000","last_fullsized":"2023-08-09T13:55:29.389375+0000","mapping_epoch":144,"log_start":"0'0","ondisk_log_start":"0'0","created":144,"last_epoch_clean":145,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:55:27.277343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:55:27.277343+0000","last_clean_scrub_stamp":"2023-08-09T13:55:27.277343+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T19:05:21.321719+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"19.b","version":"0'0","reported_seq":9,"reported_epoch":146,"state":"active+clean","last_fresh":"2023-08-09T13:55:29.385510+0000","last_change":"2023-08-09T13:55:29.380286+0000","last_active":"2023-08-09T13:55:29.385510+0000","last_peered":"2023-08-09T13:55:29.385510+0000","last_clean":"2023-08-09T13:55:29.385510+0000","last_became_active":"2023-08-09T13:55:29.379950+0000","last_became_peered":"2023-08-09T13:55:29.379950+0000","last_unstale":"2023-08-09T13:55:29.385510+0000","last_undegraded":"2023-08-09T13:55:29.385510+0000","last_fullsized":"2023-08-09T13:55:29.385510+0000","mapping_epoch":145,"log_start":"0'0","ondisk_log_start":"0'0","created":145,"last_epoch_clean":146,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:55:28.281790+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:55:28.281790+0000","last_clean_scrub_stamp":"2023-08-09T13:55:28.281790+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T14:06:52.572647+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"18.a","version":"0'0","reported_seq":11,"reported_epoch":146,"state":"active+clean","last_fresh":"2023-08-09T13:55:29.343279+0000","last_change":"2023-08-09T13:55:28.322913+0000","last_active":"2023-08-09T13:55:29.343279+0000","last_peered":"2023-08-09T13:55:29.343279+0000","last_clean":"2023-08-09T13:55:29.343279+0000","last_became_active":"2023-08-09T13:55:28.322548+0000","last_became_peered":"2023-08-09T13:55:28.322548+0000","last_unstale":"2023-08-09T13:55:29.343279+0000","last_undegraded":"2023-08-09T13:55:29.343279+0000","last_fullsized":"2023-08-09T13:55:29.343279+0000","mapping_epoch":144,"log_start":"0'0","ondisk_log_start":"0'0","created":144,"last_epoch_clean":145,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:55:27.277343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:55:27.277343+0000","last_clean_scrub_stamp":"2023-08-09T13:55:27.277343+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-11T00:11:46.016919+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"19.8","version":"0'0","reported_seq":9,"reported_epoch":146,"state":"active+clean","last_fresh":"2023-08-09T13:55:29.365998+0000","last_change":"2023-08-09T13:55:29.365069+0000","last_active":"2023-08-09T13:55:29.365998+0000","last_peered":"2023-08-09T13:55:29.365998+0000","last_clean":"2023-08-09T13:55:29.365998+0000","last_became_active":"2023-08-09T13:55:29.364763+0000","last_became_peered":"2023-08-09T13:55:29.364763+0000","last_unstale":"2023-08-09T13:55:29.365998+0000","last_undegraded":"2023-08-09T13:55:29.365998+0000","last_fullsized":"2023-08-09T13:55:29.365998+0000","mapping_epoch":145,"log_start":"0'0","ondisk_log_start":"0'0","created":145,"last_epoch_clean":146,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:55:28.281790+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:55:28.281790+0000","last_clean_scrub_stamp":"2023-08-09T13:55:28.281790+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T15:43:38.940271+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"18.9","version":"0'0","reported_seq":11,"reported_epoch":146,"state":"active+clean","last_fresh":"2023-08-09T13:55:29.343582+0000","last_change":"2023-08-09T13:55:28.322323+0000","last_active":"2023-08-09T13:55:29.343582+0000","last_peered":"2023-08-09T13:55:29.343582+0000","last_clean":"2023-08-09T13:55:29.343582+0000","last_became_active":"2023-08-09T13:55:28.321651+0000","last_became_peered":"2023-08-09T13:55:28.321651+0000","last_unstale":"2023-08-09T13:55:29.343582+0000","last_undegraded":"2023-08-09T13:55:29.343582+0000","last_fullsized":"2023-08-09T13:55:29.343582+0000","mapping_epoch":144,"log_start":"0'0","ondisk_log_start":"0'0","created":144,"last_epoch_clean":145,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:55:27.277343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:55:27.277343+0000","last_clean_scrub_stamp":"2023-08-09T13:55:27.277343+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T19:19:29.231021+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"19.9","version":"0'0","reported_seq":9,"reported_epoch":146,"state":"active+clean","last_fresh":"2023-08-09T13:55:29.380375+0000","last_change":"2023-08-09T13:55:29.378637+0000","last_active":"2023-08-09T13:55:29.380375+0000","last_peered":"2023-08-09T13:55:29.380375+0000","last_clean":"2023-08-09T13:55:29.380375+0000","last_became_active":"2023-08-09T13:55:29.377377+0000","last_became_peered":"2023-08-09T13:55:29.377377+0000","last_unstale":"2023-08-09T13:55:29.380375+0000","last_undegraded":"2023-08-09T13:55:29.380375+0000","last_fullsized":"2023-08-09T13:55:29.380375+0000","mapping_epoch":145,"log_start":"0'0","ondisk_log_start":"0'0","created":145,"last_epoch_clean":146,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:55:28.281790+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:55:28.281790+0000","last_clean_scrub_stamp":"2023-08-09T13:55:28.281790+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T20:36:38.323524+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"18.8","version":"0'0","reported_seq":11,"reported_epoch":146,"state":"active+clean","last_fresh":"2023-08-09T13:55:29.335915+0000","last_change":"2023-08-09T13:55:28.325598+0000","last_active":"2023-08-09T13:55:29.335915+0000","last_peered":"2023-08-09T13:55:29.335915+0000","last_clean":"2023-08-09T13:55:29.335915+0000","last_became_active":"2023-08-09T13:55:28.325300+0000","last_became_peered":"2023-08-09T13:55:28.325300+0000","last_unstale":"2023-08-09T13:55:29.335915+0000","last_undegraded":"2023-08-09T13:55:29.335915+0000","last_fullsized":"2023-08-09T13:55:29.335915+0000","mapping_epoch":144,"log_start":"0'0","ondisk_log_start":"0'0","created":144,"last_epoch_clean":145,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:55:27.277343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:55:27.277343+0000","last_clean_scrub_stamp":"2023-08-09T13:55:27.277343+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T23:29:41.577663+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"19.6","version":"0'0","reported_seq":9,"reported_epoch":146,"state":"active+clean","last_fresh":"2023-08-09T13:55:29.393336+0000","last_change":"2023-08-09T13:55:29.389917+0000","last_active":"2023-08-09T13:55:29.393336+0000","last_peered":"2023-08-09T13:55:29.393336+0000","last_clean":"2023-08-09T13:55:29.393336+0000","last_became_active":"2023-08-09T13:55:29.387910+0000","last_became_peered":"2023-08-09T13:55:29.387910+0000","last_unstale":"2023-08-09T13:55:29.393336+0000","last_undegraded":"2023-08-09T13:55:29.393336+0000","last_fullsized":"2023-08-09T13:55:29.393336+0000","mapping_epoch":145,"log_start":"0'0","ondisk_log_start":"0'0","created":145,"last_epoch_clean":146,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:55:28.281790+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:55:28.281790+0000","last_clean_scrub_stamp":"2023-08-09T13:55:28.281790+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T22:42:15.840116+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"18.7","version":"146'2","reported_seq":13,"reported_epoch":146,"state":"active+clean","last_fresh":"2023-08-09T13:55:29.385963+0000","last_change":"2023-08-09T13:55:28.323788+0000","last_active":"2023-08-09T13:55:29.385963+0000","last_peered":"2023-08-09T13:55:29.385963+0000","last_clean":"2023-08-09T13:55:29.385963+0000","last_became_active":"2023-08-09T13:55:28.323507+0000","last_became_peered":"2023-08-09T13:55:28.323507+0000","last_unstale":"2023-08-09T13:55:29.385963+0000","last_undegraded":"2023-08-09T13:55:29.385963+0000","last_fullsized":"2023-08-09T13:55:29.385963+0000","mapping_epoch":144,"log_start":"0'0","ondisk_log_start":"0'0","created":144,"last_epoch_clean":145,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:55:27.277343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:55:27.277343+0000","last_clean_scrub_stamp":"2023-08-09T13:55:27.277343+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T19:29:58.936799+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"19.7","version":"0'0","reported_seq":9,"reported_epoch":146,"state":"active+clean","last_fresh":"2023-08-09T13:55:29.382851+0000","last_change":"2023-08-09T13:55:29.377753+0000","last_active":"2023-08-09T13:55:29.382851+0000","last_peered":"2023-08-09T13:55:29.382851+0000","last_clean":"2023-08-09T13:55:29.382851+0000","last_became_active":"2023-08-09T13:55:29.377285+0000","last_became_peered":"2023-08-09T13:55:29.377285+0000","last_unstale":"2023-08-09T13:55:29.382851+0000","last_undegraded":"2023-08-09T13:55:29.382851+0000","last_fullsized":"2023-08-09T13:55:29.382851+0000","mapping_epoch":145,"log_start":"0'0","ondisk_log_start":"0'0","created":145,"last_epoch_clean":146,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:55:28.281790+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:55:28.281790+0000","last_clean_scrub_stamp":"2023-08-09T13:55:28.281790+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T20:48:01.956488+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"18.6","version":"0'0","reported_seq":11,"reported_epoch":146,"state":"active+clean","last_fresh":"2023-08-09T13:55:29.362948+0000","last_change":"2023-08-09T13:55:28.328542+0000","last_active":"2023-08-09T13:55:29.362948+0000","last_peered":"2023-08-09T13:55:29.362948+0000","last_clean":"2023-08-09T13:55:29.362948+0000","last_became_active":"2023-08-09T13:55:28.328244+0000","last_became_peered":"2023-08-09T13:55:28.328244+0000","last_unstale":"2023-08-09T13:55:29.362948+0000","last_undegraded":"2023-08-09T13:55:29.362948+0000","last_fullsized":"2023-08-09T13:55:29.362948+0000","mapping_epoch":144,"log_start":"0'0","ondisk_log_start":"0'0","created":144,"last_epoch_clean":145,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:55:27.277343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:55:27.277343+0000","last_clean_scrub_stamp":"2023-08-09T13:55:27.277343+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T14:46:50.318176+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"19.4","version":"0'0","reported_seq":9,"reported_epoch":146,"state":"active+clean","last_fresh":"2023-08-09T13:55:29.379950+0000","last_change":"2023-08-09T13:55:29.378985+0000","last_active":"2023-08-09T13:55:29.379950+0000","last_peered":"2023-08-09T13:55:29.379950+0000","last_clean":"2023-08-09T13:55:29.379950+0000","last_became_active":"2023-08-09T13:55:29.378675+0000","last_became_peered":"2023-08-09T13:55:29.378675+0000","last_unstale":"2023-08-09T13:55:29.379950+0000","last_undegraded":"2023-08-09T13:55:29.379950+0000","last_fullsized":"2023-08-09T13:55:29.379950+0000","mapping_epoch":145,"log_start":"0'0","ondisk_log_start":"0'0","created":145,"last_epoch_clean":146,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:55:28.281790+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:55:28.281790+0000","last_clean_scrub_stamp":"2023-08-09T13:55:28.281790+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T16:13:09.951708+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"18.5","version":"0'0","reported_seq":11,"reported_epoch":146,"state":"active+clean","last_fresh":"2023-08-09T13:55:29.357421+0000","last_change":"2023-08-09T13:55:28.327073+0000","last_active":"2023-08-09T13:55:29.357421+0000","last_peered":"2023-08-09T13:55:29.357421+0000","last_clean":"2023-08-09T13:55:29.357421+0000","last_became_active":"2023-08-09T13:55:28.326813+0000","last_became_peered":"2023-08-09T13:55:28.326813+0000","last_unstale":"2023-08-09T13:55:29.357421+0000","last_undegraded":"2023-08-09T13:55:29.357421+0000","last_fullsized":"2023-08-09T13:55:29.357421+0000","mapping_epoch":144,"log_start":"0'0","ondisk_log_start":"0'0","created":144,"last_epoch_clean":145,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:55:27.277343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:55:27.277343+0000","last_clean_scrub_stamp":"2023-08-09T13:55:27.277343+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T16:20:54.438480+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"19.5","version":"0'0","reported_seq":9,"reported_epoch":146,"state":"active+clean","last_fresh":"2023-08-09T13:55:29.387417+0000","last_change":"2023-08-09T13:55:29.385183+0000","last_active":"2023-08-09T13:55:29.387417+0000","last_peered":"2023-08-09T13:55:29.387417+0000","last_clean":"2023-08-09T13:55:29.387417+0000","last_became_active":"2023-08-09T13:55:29.384576+0000","last_became_peered":"2023-08-09T13:55:29.384576+0000","last_unstale":"2023-08-09T13:55:29.387417+0000","last_undegraded":"2023-08-09T13:55:29.387417+0000","last_fullsized":"2023-08-09T13:55:29.387417+0000","mapping_epoch":145,"log_start":"0'0","ondisk_log_start":"0'0","created":145,"last_epoch_clean":146,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:55:28.281790+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:55:28.281790+0000","last_clean_scrub_stamp":"2023-08-09T13:55:28.281790+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T16:02:59.215551+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"18.4","version":"0'0","reported_seq":11,"reported_epoch":146,"state":"active+clean","last_fresh":"2023-08-09T13:55:29.335446+0000","last_change":"2023-08-09T13:55:28.317485+0000","last_active":"2023-08-09T13:55:29.335446+0000","last_peered":"2023-08-09T13:55:29.335446+0000","last_clean":"2023-08-09T13:55:29.335446+0000","last_became_active":"2023-08-09T13:55:28.317110+0000","last_became_peered":"2023-08-09T13:55:28.317110+0000","last_unstale":"2023-08-09T13:55:29.335446+0000","last_undegraded":"2023-08-09T13:55:29.335446+0000","last_fullsized":"2023-08-09T13:55:29.335446+0000","mapping_epoch":144,"log_start":"0'0","ondisk_log_start":"0'0","created":144,"last_epoch_clean":145,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:55:27.277343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:55:27.277343+0000","last_clean_scrub_stamp":"2023-08-09T13:55:27.277343+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T20:06:03.354893+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"19.2","version":"0'0","reported_seq":9,"reported_epoch":146,"state":"active+clean","last_fresh":"2023-08-09T13:55:29.389893+0000","last_change":"2023-08-09T13:55:29.386265+0000","last_active":"2023-08-09T13:55:29.389893+0000","last_peered":"2023-08-09T13:55:29.389893+0000","last_clean":"2023-08-09T13:55:29.389893+0000","last_became_active":"2023-08-09T13:55:29.385908+0000","last_became_peered":"2023-08-09T13:55:29.385908+0000","last_unstale":"2023-08-09T13:55:29.389893+0000","last_undegraded":"2023-08-09T13:55:29.389893+0000","last_fullsized":"2023-08-09T13:55:29.389893+0000","mapping_epoch":145,"log_start":"0'0","ondisk_log_start":"0'0","created":145,"last_epoch_clean":146,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:55:28.281790+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:55:28.281790+0000","last_clean_scrub_stamp":"2023-08-09T13:55:28.281790+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T22:31:20.809531+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"18.3","version":"0'0","reported_seq":11,"reported_epoch":146,"state":"active+clean","last_fresh":"2023-08-09T13:55:29.353043+0000","last_change":"2023-08-09T13:55:28.328336+0000","last_active":"2023-08-09T13:55:29.353043+0000","last_peered":"2023-08-09T13:55:29.353043+0000","last_clean":"2023-08-09T13:55:29.353043+0000","last_became_active":"2023-08-09T13:55:28.328036+0000","last_became_peered":"2023-08-09T13:55:28.328036+0000","last_unstale":"2023-08-09T13:55:29.353043+0000","last_undegraded":"2023-08-09T13:55:29.353043+0000","last_fullsized":"2023-08-09T13:55:29.353043+0000","mapping_epoch":144,"log_start":"0'0","ondisk_log_start":"0'0","created":144,"last_epoch_clean":145,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:55:27.277343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:55:27.277343+0000","last_clean_scrub_stamp":"2023-08-09T13:55:27.277343+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T17:11:15.222036+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"19.3","version":"0'0","reported_seq":9,"reported_epoch":146,"state":"active+clean","last_fresh":"2023-08-09T13:55:29.366142+0000","last_change":"2023-08-09T13:55:29.363281+0000","last_active":"2023-08-09T13:55:29.366142+0000","last_peered":"2023-08-09T13:55:29.366142+0000","last_clean":"2023-08-09T13:55:29.366142+0000","last_became_active":"2023-08-09T13:55:29.362967+0000","last_became_peered":"2023-08-09T13:55:29.362967+0000","last_unstale":"2023-08-09T13:55:29.366142+0000","last_undegraded":"2023-08-09T13:55:29.366142+0000","last_fullsized":"2023-08-09T13:55:29.366142+0000","mapping_epoch":145,"log_start":"0'0","ondisk_log_start":"0'0","created":145,"last_epoch_clean":146,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:55:28.281790+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:55:28.281790+0000","last_clean_scrub_stamp":"2023-08-09T13:55:28.281790+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T20:19:18.935035+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"18.2","version":"0'0","reported_seq":11,"reported_epoch":146,"state":"active+clean","last_fresh":"2023-08-09T13:55:29.357584+0000","last_change":"2023-08-09T13:55:28.325478+0000","last_active":"2023-08-09T13:55:29.357584+0000","last_peered":"2023-08-09T13:55:29.357584+0000","last_clean":"2023-08-09T13:55:29.357584+0000","last_became_active":"2023-08-09T13:55:28.325182+0000","last_became_peered":"2023-08-09T13:55:28.325182+0000","last_unstale":"2023-08-09T13:55:29.357584+0000","last_undegraded":"2023-08-09T13:55:29.357584+0000","last_fullsized":"2023-08-09T13:55:29.357584+0000","mapping_epoch":144,"log_start":"0'0","ondisk_log_start":"0'0","created":144,"last_epoch_clean":145,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:55:27.277343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:55:27.277343+0000","last_clean_scrub_stamp":"2023-08-09T13:55:27.277343+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-11T00:35:07.214932+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"19.0","version":"0'0","reported_seq":9,"reported_epoch":146,"state":"active+clean","last_fresh":"2023-08-09T13:55:29.383677+0000","last_change":"2023-08-09T13:55:29.380961+0000","last_active":"2023-08-09T13:55:29.383677+0000","last_peered":"2023-08-09T13:55:29.383677+0000","last_clean":"2023-08-09T13:55:29.383677+0000","last_became_active":"2023-08-09T13:55:29.380639+0000","last_became_peered":"2023-08-09T13:55:29.380639+0000","last_unstale":"2023-08-09T13:55:29.383677+0000","last_undegraded":"2023-08-09T13:55:29.383677+0000","last_fullsized":"2023-08-09T13:55:29.383677+0000","mapping_epoch":145,"log_start":"0'0","ondisk_log_start":"0'0","created":145,"last_epoch_clean":146,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:55:28.281790+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:55:28.281790+0000","last_clean_scrub_stamp":"2023-08-09T13:55:28.281790+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T18:00:24.971826+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"18.1","version":"0'0","reported_seq":11,"reported_epoch":146,"state":"active+clean","last_fresh":"2023-08-09T13:55:29.343750+0000","last_change":"2023-08-09T13:55:28.322516+0000","last_active":"2023-08-09T13:55:29.343750+0000","last_peered":"2023-08-09T13:55:29.343750+0000","last_clean":"2023-08-09T13:55:29.343750+0000","last_became_active":"2023-08-09T13:55:28.322036+0000","last_became_peered":"2023-08-09T13:55:28.322036+0000","last_unstale":"2023-08-09T13:55:29.343750+0000","last_undegraded":"2023-08-09T13:55:29.343750+0000","last_fullsized":"2023-08-09T13:55:29.343750+0000","mapping_epoch":144,"log_start":"0'0","ondisk_log_start":"0'0","created":144,"last_epoch_clean":145,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:55:27.277343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:55:27.277343+0000","last_clean_scrub_stamp":"2023-08-09T13:55:27.277343+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-11T01:20:23.987524+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"19.1","version":"0'0","reported_seq":9,"reported_epoch":146,"state":"active+clean","last_fresh":"2023-08-09T13:55:29.370909+0000","last_change":"2023-08-09T13:55:29.368598+0000","last_active":"2023-08-09T13:55:29.370909+0000","last_peered":"2023-08-09T13:55:29.370909+0000","last_clean":"2023-08-09T13:55:29.370909+0000","last_became_active":"2023-08-09T13:55:29.368284+0000","last_became_peered":"2023-08-09T13:55:29.368284+0000","last_unstale":"2023-08-09T13:55:29.370909+0000","last_undegraded":"2023-08-09T13:55:29.370909+0000","last_fullsized":"2023-08-09T13:55:29.370909+0000","mapping_epoch":145,"log_start":"0'0","ondisk_log_start":"0'0","created":145,"last_epoch_clean":146,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:55:28.281790+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:55:28.281790+0000","last_clean_scrub_stamp":"2023-08-09T13:55:28.281790+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T22:11:08.463964+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"18.0","version":"0'0","reported_seq":11,"reported_epoch":146,"state":"active+clean","last_fresh":"2023-08-09T13:55:29.336303+0000","last_change":"2023-08-09T13:55:28.321218+0000","last_active":"2023-08-09T13:55:29.336303+0000","last_peered":"2023-08-09T13:55:29.336303+0000","last_clean":"2023-08-09T13:55:29.336303+0000","last_became_active":"2023-08-09T13:55:28.320885+0000","last_became_peered":"2023-08-09T13:55:28.320885+0000","last_unstale":"2023-08-09T13:55:29.336303+0000","last_undegraded":"2023-08-09T13:55:29.336303+0000","last_fullsized":"2023-08-09T13:55:29.336303+0000","mapping_epoch":144,"log_start":"0'0","ondisk_log_start":"0'0","created":144,"last_epoch_clean":145,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:55:27.277343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:55:27.277343+0000","last_clean_scrub_stamp":"2023-08-09T13:55:27.277343+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-11T01:17:28.831020+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"19.12","version":"0'0","reported_seq":9,"reported_epoch":146,"state":"active+clean","last_fresh":"2023-08-09T13:55:29.380776+0000","last_change":"2023-08-09T13:55:29.378718+0000","last_active":"2023-08-09T13:55:29.380776+0000","last_peered":"2023-08-09T13:55:29.380776+0000","last_clean":"2023-08-09T13:55:29.380776+0000","last_became_active":"2023-08-09T13:55:29.378360+0000","last_became_peered":"2023-08-09T13:55:29.378360+0000","last_unstale":"2023-08-09T13:55:29.380776+0000","last_undegraded":"2023-08-09T13:55:29.380776+0000","last_fullsized":"2023-08-09T13:55:29.380776+0000","mapping_epoch":145,"log_start":"0'0","ondisk_log_start":"0'0","created":145,"last_epoch_clean":146,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:55:28.281790+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:55:28.281790+0000","last_clean_scrub_stamp":"2023-08-09T13:55:28.281790+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T22:03:35.883538+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"18.13","version":"146'2","reported_seq":13,"reported_epoch":146,"state":"active+clean","last_fresh":"2023-08-09T13:55:29.383182+0000","last_change":"2023-08-09T13:55:28.330858+0000","last_active":"2023-08-09T13:55:29.383182+0000","last_peered":"2023-08-09T13:55:29.383182+0000","last_clean":"2023-08-09T13:55:29.383182+0000","last_became_active":"2023-08-09T13:55:28.330570+0000","last_became_peered":"2023-08-09T13:55:28.330570+0000","last_unstale":"2023-08-09T13:55:29.383182+0000","last_undegraded":"2023-08-09T13:55:29.383182+0000","last_fullsized":"2023-08-09T13:55:29.383182+0000","mapping_epoch":144,"log_start":"0'0","ondisk_log_start":"0'0","created":144,"last_epoch_clean":145,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:55:27.277343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:55:27.277343+0000","last_clean_scrub_stamp":"2023-08-09T13:55:27.277343+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-11T00:17:08.270713+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":249,"reported_epoch":146,"state":"active+clean","last_fresh":"2023-08-09T13:55:29.355802+0000","last_change":"2023-08-09T13:33:56.401406+0000","last_active":"2023-08-09T13:55:29.355802+0000","last_peered":"2023-08-09T13:55:29.355802+0000","last_clean":"2023-08-09T13:55:29.355802+0000","last_became_active":"2023-08-09T13:33:56.400487+0000","last_became_peered":"2023-08-09T13:33:56.400487+0000","last_unstale":"2023-08-09T13:55:29.355802+0000","last_undegraded":"2023-08-09T13:55:29.355802+0000","last_fullsized":"2023-08-09T13:55:29.355802+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:33:54.845869+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:33:54.845869+0000","last_clean_scrub_stamp":"2023-08-09T13:33:54.845869+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T23:33:46.735697+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"19.13","version":"0'0","reported_seq":9,"reported_epoch":146,"state":"active+clean","last_fresh":"2023-08-09T13:55:29.358546+0000","last_change":"2023-08-09T13:55:29.347419+0000","last_active":"2023-08-09T13:55:29.358546+0000","last_peered":"2023-08-09T13:55:29.358546+0000","last_clean":"2023-08-09T13:55:29.358546+0000","last_became_active":"2023-08-09T13:55:29.347094+0000","last_became_peered":"2023-08-09T13:55:29.347094+0000","last_unstale":"2023-08-09T13:55:29.358546+0000","last_undegraded":"2023-08-09T13:55:29.358546+0000","last_fullsized":"2023-08-09T13:55:29.358546+0000","mapping_epoch":145,"log_start":"0'0","ondisk_log_start":"0'0","created":145,"last_epoch_clean":146,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:55:28.281790+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:55:28.281790+0000","last_clean_scrub_stamp":"2023-08-09T13:55:28.281790+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-11T00:19:35.426972+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"18.12","version":"0'0","reported_seq":11,"reported_epoch":146,"state":"active+clean","last_fresh":"2023-08-09T13:55:29.341204+0000","last_change":"2023-08-09T13:55:28.320668+0000","last_active":"2023-08-09T13:55:29.341204+0000","last_peered":"2023-08-09T13:55:29.341204+0000","last_clean":"2023-08-09T13:55:29.341204+0000","last_became_active":"2023-08-09T13:55:28.320258+0000","last_became_peered":"2023-08-09T13:55:28.320258+0000","last_unstale":"2023-08-09T13:55:29.341204+0000","last_undegraded":"2023-08-09T13:55:29.341204+0000","last_fullsized":"2023-08-09T13:55:29.341204+0000","mapping_epoch":144,"log_start":"0'0","ondisk_log_start":"0'0","created":144,"last_epoch_clean":145,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:55:27.277343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:55:27.277343+0000","last_clean_scrub_stamp":"2023-08-09T13:55:27.277343+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T19:58:41.585493+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":251,"reported_epoch":146,"state":"active+clean","last_fresh":"2023-08-09T13:55:29.352896+0000","last_change":"2023-08-09T13:33:56.401123+0000","last_active":"2023-08-09T13:55:29.352896+0000","last_peered":"2023-08-09T13:55:29.352896+0000","last_clean":"2023-08-09T13:55:29.352896+0000","last_became_active":"2023-08-09T13:33:56.399933+0000","last_became_peered":"2023-08-09T13:33:56.399933+0000","last_unstale":"2023-08-09T13:55:29.352896+0000","last_undegraded":"2023-08-09T13:55:29.352896+0000","last_fullsized":"2023-08-09T13:55:29.352896+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:33:54.845869+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:33:54.845869+0000","last_clean_scrub_stamp":"2023-08-09T13:33:54.845869+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T22:43:50.638469+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"19.10","version":"0'0","reported_seq":9,"reported_epoch":146,"state":"active+clean","last_fresh":"2023-08-09T13:55:29.382441+0000","last_change":"2023-08-09T13:55:29.380953+0000","last_active":"2023-08-09T13:55:29.382441+0000","last_peered":"2023-08-09T13:55:29.382441+0000","last_clean":"2023-08-09T13:55:29.382441+0000","last_became_active":"2023-08-09T13:55:29.380675+0000","last_became_peered":"2023-08-09T13:55:29.380675+0000","last_unstale":"2023-08-09T13:55:29.382441+0000","last_undegraded":"2023-08-09T13:55:29.382441+0000","last_fullsized":"2023-08-09T13:55:29.382441+0000","mapping_epoch":145,"log_start":"0'0","ondisk_log_start":"0'0","created":145,"last_epoch_clean":146,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:55:28.281790+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:55:28.281790+0000","last_clean_scrub_stamp":"2023-08-09T13:55:28.281790+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T22:11:23.849215+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"18.11","version":"0'0","reported_seq":11,"reported_epoch":146,"state":"active+clean","last_fresh":"2023-08-09T13:55:29.354962+0000","last_change":"2023-08-09T13:55:28.328548+0000","last_active":"2023-08-09T13:55:29.354962+0000","last_peered":"2023-08-09T13:55:29.354962+0000","last_clean":"2023-08-09T13:55:29.354962+0000","last_became_active":"2023-08-09T13:55:28.328239+0000","last_became_peered":"2023-08-09T13:55:28.328239+0000","last_unstale":"2023-08-09T13:55:29.354962+0000","last_undegraded":"2023-08-09T13:55:29.354962+0000","last_fullsized":"2023-08-09T13:55:29.354962+0000","mapping_epoch":144,"log_start":"0'0","ondisk_log_start":"0'0","created":144,"last_epoch_clean":145,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:55:27.277343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:55:27.277343+0000","last_clean_scrub_stamp":"2023-08-09T13:55:27.277343+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T17:22:59.355534+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.2","version":"0'0","reported_seq":249,"reported_epoch":146,"state":"active+clean","last_fresh":"2023-08-09T13:55:29.357771+0000","last_change":"2023-08-09T13:33:56.402794+0000","last_active":"2023-08-09T13:55:29.357771+0000","last_peered":"2023-08-09T13:55:29.357771+0000","last_clean":"2023-08-09T13:55:29.357771+0000","last_became_active":"2023-08-09T13:33:56.402038+0000","last_became_peered":"2023-08-09T13:33:56.402038+0000","last_unstale":"2023-08-09T13:55:29.357771+0000","last_undegraded":"2023-08-09T13:55:29.357771+0000","last_fullsized":"2023-08-09T13:55:29.357771+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:33:54.845869+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:33:54.845869+0000","last_clean_scrub_stamp":"2023-08-09T13:33:54.845869+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T14:14:28.831260+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"19.11","version":"0'0","reported_seq":9,"reported_epoch":146,"state":"active+clean","last_fresh":"2023-08-09T13:55:29.392037+0000","last_change":"2023-08-09T13:55:29.389616+0000","last_active":"2023-08-09T13:55:29.392037+0000","last_peered":"2023-08-09T13:55:29.392037+0000","last_clean":"2023-08-09T13:55:29.392037+0000","last_became_active":"2023-08-09T13:55:29.389114+0000","last_became_peered":"2023-08-09T13:55:29.389114+0000","last_unstale":"2023-08-09T13:55:29.392037+0000","last_undegraded":"2023-08-09T13:55:29.392037+0000","last_fullsized":"2023-08-09T13:55:29.392037+0000","mapping_epoch":145,"log_start":"0'0","ondisk_log_start":"0'0","created":145,"last_epoch_clean":146,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:55:28.281790+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:55:28.281790+0000","last_clean_scrub_stamp":"2023-08-09T13:55:28.281790+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T18:10:48.285444+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"18.10","version":"146'8","reported_seq":19,"reported_epoch":146,"state":"active+clean","last_fresh":"2023-08-09T13:55:29.391972+0000","last_change":"2023-08-09T13:55:28.322058+0000","last_active":"2023-08-09T13:55:29.391972+0000","last_peered":"2023-08-09T13:55:29.391972+0000","last_clean":"2023-08-09T13:55:29.391972+0000","last_became_active":"2023-08-09T13:55:28.321769+0000","last_became_peered":"2023-08-09T13:55:28.321769+0000","last_unstale":"2023-08-09T13:55:29.391972+0000","last_undegraded":"2023-08-09T13:55:29.391972+0000","last_fullsized":"2023-08-09T13:55:29.391972+0000","mapping_epoch":144,"log_start":"0'0","ondisk_log_start":"0'0","created":144,"last_epoch_clean":145,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:55:27.277343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:55:27.277343+0000","last_clean_scrub_stamp":"2023-08-09T13:55:27.277343+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T15:34:41.903487+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"1.3","version":"0'0","reported_seq":251,"reported_epoch":146,"state":"active+clean","last_fresh":"2023-08-09T13:55:29.353256+0000","last_change":"2023-08-09T13:33:55.886749+0000","last_active":"2023-08-09T13:55:29.353256+0000","last_peered":"2023-08-09T13:55:29.353256+0000","last_clean":"2023-08-09T13:55:29.353256+0000","last_became_active":"2023-08-09T13:33:55.886116+0000","last_became_peered":"2023-08-09T13:33:55.886116+0000","last_unstale":"2023-08-09T13:55:29.353256+0000","last_undegraded":"2023-08-09T13:55:29.353256+0000","last_fullsized":"2023-08-09T13:55:29.353256+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:33:54.845869+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:33:54.845869+0000","last_clean_scrub_stamp":"2023-08-09T13:33:54.845869+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T22:34:38.570994+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"134'230","reported_seq":799,"reported_epoch":146,"state":"active+clean","last_fresh":"2023-08-09T13:55:29.355455+0000","last_change":"2023-08-09T13:33:55.886345+0000","last_active":"2023-08-09T13:55:29.355455+0000","last_peered":"2023-08-09T13:55:29.355455+0000","last_clean":"2023-08-09T13:55:29.355455+0000","last_became_active":"2023-08-09T13:33:55.885781+0000","last_became_peered":"2023-08-09T13:33:55.885781+0000","last_unstale":"2023-08-09T13:55:29.355455+0000","last_undegraded":"2023-08-09T13:55:29.355455+0000","last_fullsized":"2023-08-09T13:55:29.355455+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:33:54.845869+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:33:54.845869+0000","last_clean_scrub_stamp":"2023-08-09T13:33:54.845869+0000","objects_scrubbed":0,"log_size":230,"log_dups_size":0,"ondisk_log_size":230,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T20:59:57.120396+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1099,"num_read_kb":2242,"num_write":363,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"19.16","version":"0'0","reported_seq":9,"reported_epoch":146,"state":"active+clean","last_fresh":"2023-08-09T13:55:29.363725+0000","last_change":"2023-08-09T13:55:29.357951+0000","last_active":"2023-08-09T13:55:29.363725+0000","last_peered":"2023-08-09T13:55:29.363725+0000","last_clean":"2023-08-09T13:55:29.363725+0000","last_became_active":"2023-08-09T13:55:29.357461+0000","last_became_peered":"2023-08-09T13:55:29.357461+0000","last_unstale":"2023-08-09T13:55:29.363725+0000","last_undegraded":"2023-08-09T13:55:29.363725+0000","last_fullsized":"2023-08-09T13:55:29.363725+0000","mapping_epoch":145,"log_start":"0'0","ondisk_log_start":"0'0","created":145,"last_epoch_clean":146,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:55:28.281790+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:55:28.281790+0000","last_clean_scrub_stamp":"2023-08-09T13:55:28.281790+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T16:37:22.262338+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"18.17","version":"0'0","reported_seq":11,"reported_epoch":146,"state":"active+clean","last_fresh":"2023-08-09T13:55:29.361433+0000","last_change":"2023-08-09T13:55:28.319541+0000","last_active":"2023-08-09T13:55:29.361433+0000","last_peered":"2023-08-09T13:55:29.361433+0000","last_clean":"2023-08-09T13:55:29.361433+0000","last_became_active":"2023-08-09T13:55:28.319094+0000","last_became_peered":"2023-08-09T13:55:28.319094+0000","last_unstale":"2023-08-09T13:55:29.361433+0000","last_undegraded":"2023-08-09T13:55:29.361433+0000","last_fullsized":"2023-08-09T13:55:29.361433+0000","mapping_epoch":144,"log_start":"0'0","ondisk_log_start":"0'0","created":144,"last_epoch_clean":145,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:55:27.277343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:55:27.277343+0000","last_clean_scrub_stamp":"2023-08-09T13:55:27.277343+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T19:50:13.138643+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":257,"reported_epoch":146,"state":"active+clean","last_fresh":"2023-08-09T13:55:29.344850+0000","last_change":"2023-08-09T13:33:55.889565+0000","last_active":"2023-08-09T13:55:29.344850+0000","last_peered":"2023-08-09T13:55:29.344850+0000","last_clean":"2023-08-09T13:55:29.344850+0000","last_became_active":"2023-08-09T13:33:55.889164+0000","last_became_peered":"2023-08-09T13:33:55.889164+0000","last_unstale":"2023-08-09T13:55:29.344850+0000","last_undegraded":"2023-08-09T13:55:29.344850+0000","last_fullsized":"2023-08-09T13:55:29.344850+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:33:54.845869+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:33:54.845869+0000","last_clean_scrub_stamp":"2023-08-09T13:33:54.845869+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T23:48:46.682375+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"19.17","version":"0'0","reported_seq":9,"reported_epoch":146,"state":"active+clean","last_fresh":"2023-08-09T13:55:29.387450+0000","last_change":"2023-08-09T13:55:29.369149+0000","last_active":"2023-08-09T13:55:29.387450+0000","last_peered":"2023-08-09T13:55:29.387450+0000","last_clean":"2023-08-09T13:55:29.387450+0000","last_became_active":"2023-08-09T13:55:29.367877+0000","last_became_peered":"2023-08-09T13:55:29.367877+0000","last_unstale":"2023-08-09T13:55:29.387450+0000","last_undegraded":"2023-08-09T13:55:29.387450+0000","last_fullsized":"2023-08-09T13:55:29.387450+0000","mapping_epoch":145,"log_start":"0'0","ondisk_log_start":"0'0","created":145,"last_epoch_clean":146,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:55:28.281790+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:55:28.281790+0000","last_clean_scrub_stamp":"2023-08-09T13:55:28.281790+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T20:40:41.419067+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"18.16","version":"0'0","reported_seq":11,"reported_epoch":146,"state":"active+clean","last_fresh":"2023-08-09T13:55:29.345931+0000","last_change":"2023-08-09T13:55:28.322495+0000","last_active":"2023-08-09T13:55:29.345931+0000","last_peered":"2023-08-09T13:55:29.345931+0000","last_clean":"2023-08-09T13:55:29.345931+0000","last_became_active":"2023-08-09T13:55:28.322189+0000","last_became_peered":"2023-08-09T13:55:28.322189+0000","last_unstale":"2023-08-09T13:55:29.345931+0000","last_undegraded":"2023-08-09T13:55:29.345931+0000","last_fullsized":"2023-08-09T13:55:29.345931+0000","mapping_epoch":144,"log_start":"0'0","ondisk_log_start":"0'0","created":144,"last_epoch_clean":145,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:55:27.277343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:55:27.277343+0000","last_clean_scrub_stamp":"2023-08-09T13:55:27.277343+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T19:10:18.092027+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":251,"reported_epoch":146,"state":"active+clean","last_fresh":"2023-08-09T13:55:29.368694+0000","last_change":"2023-08-09T13:33:56.404236+0000","last_active":"2023-08-09T13:55:29.368694+0000","last_peered":"2023-08-09T13:55:29.368694+0000","last_clean":"2023-08-09T13:55:29.368694+0000","last_became_active":"2023-08-09T13:33:56.402485+0000","last_became_peered":"2023-08-09T13:33:56.402485+0000","last_unstale":"2023-08-09T13:55:29.368694+0000","last_undegraded":"2023-08-09T13:55:29.368694+0000","last_fullsized":"2023-08-09T13:55:29.368694+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:33:54.845869+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:33:54.845869+0000","last_clean_scrub_stamp":"2023-08-09T13:33:54.845869+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-11T00:14:49.582247+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"19.14","version":"0'0","reported_seq":9,"reported_epoch":146,"state":"active+clean","last_fresh":"2023-08-09T13:55:29.374857+0000","last_change":"2023-08-09T13:55:29.373999+0000","last_active":"2023-08-09T13:55:29.374857+0000","last_peered":"2023-08-09T13:55:29.374857+0000","last_clean":"2023-08-09T13:55:29.374857+0000","last_became_active":"2023-08-09T13:55:29.373714+0000","last_became_peered":"2023-08-09T13:55:29.373714+0000","last_unstale":"2023-08-09T13:55:29.374857+0000","last_undegraded":"2023-08-09T13:55:29.374857+0000","last_fullsized":"2023-08-09T13:55:29.374857+0000","mapping_epoch":145,"log_start":"0'0","ondisk_log_start":"0'0","created":145,"last_epoch_clean":146,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:55:28.281790+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:55:28.281790+0000","last_clean_scrub_stamp":"2023-08-09T13:55:28.281790+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T22:39:16.014980+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"18.15","version":"146'3","reported_seq":14,"reported_epoch":146,"state":"active+clean","last_fresh":"2023-08-09T13:55:29.387562+0000","last_change":"2023-08-09T13:55:28.322247+0000","last_active":"2023-08-09T13:55:29.387562+0000","last_peered":"2023-08-09T13:55:29.387562+0000","last_clean":"2023-08-09T13:55:29.387562+0000","last_became_active":"2023-08-09T13:55:28.321893+0000","last_became_peered":"2023-08-09T13:55:28.321893+0000","last_unstale":"2023-08-09T13:55:29.387562+0000","last_undegraded":"2023-08-09T13:55:29.387562+0000","last_fullsized":"2023-08-09T13:55:29.387562+0000","mapping_epoch":144,"log_start":"0'0","ondisk_log_start":"0'0","created":144,"last_epoch_clean":145,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:55:27.277343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:55:27.277343+0000","last_clean_scrub_stamp":"2023-08-09T13:55:27.277343+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T20:48:57.060910+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":244,"reported_epoch":146,"state":"active+clean","last_fresh":"2023-08-09T13:55:29.372009+0000","last_change":"2023-08-09T13:33:56.401781+0000","last_active":"2023-08-09T13:55:29.372009+0000","last_peered":"2023-08-09T13:55:29.372009+0000","last_clean":"2023-08-09T13:55:29.372009+0000","last_became_active":"2023-08-09T13:33:56.400336+0000","last_became_peered":"2023-08-09T13:33:56.400336+0000","last_unstale":"2023-08-09T13:55:29.372009+0000","last_undegraded":"2023-08-09T13:55:29.372009+0000","last_fullsized":"2023-08-09T13:55:29.372009+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:33:54.845869+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:33:54.845869+0000","last_clean_scrub_stamp":"2023-08-09T13:33:54.845869+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T23:47:41.691639+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"19.15","version":"0'0","reported_seq":9,"reported_epoch":146,"state":"active+clean","last_fresh":"2023-08-09T13:55:29.385460+0000","last_change":"2023-08-09T13:55:29.381047+0000","last_active":"2023-08-09T13:55:29.385460+0000","last_peered":"2023-08-09T13:55:29.385460+0000","last_clean":"2023-08-09T13:55:29.385460+0000","last_became_active":"2023-08-09T13:55:29.380540+0000","last_became_peered":"2023-08-09T13:55:29.380540+0000","last_unstale":"2023-08-09T13:55:29.385460+0000","last_undegraded":"2023-08-09T13:55:29.385460+0000","last_fullsized":"2023-08-09T13:55:29.385460+0000","mapping_epoch":145,"log_start":"0'0","ondisk_log_start":"0'0","created":145,"last_epoch_clean":146,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:55:28.281790+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:55:28.281790+0000","last_clean_scrub_stamp":"2023-08-09T13:55:28.281790+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T22:40:35.323784+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"18.14","version":"146'6","reported_seq":17,"reported_epoch":146,"state":"active+clean","last_fresh":"2023-08-09T13:55:29.391101+0000","last_change":"2023-08-09T13:55:28.328980+0000","last_active":"2023-08-09T13:55:29.391101+0000","last_peered":"2023-08-09T13:55:29.391101+0000","last_clean":"2023-08-09T13:55:29.391101+0000","last_became_active":"2023-08-09T13:55:28.328700+0000","last_became_peered":"2023-08-09T13:55:28.328700+0000","last_unstale":"2023-08-09T13:55:29.391101+0000","last_undegraded":"2023-08-09T13:55:29.391101+0000","last_fullsized":"2023-08-09T13:55:29.391101+0000","mapping_epoch":144,"log_start":"0'0","ondisk_log_start":"0'0","created":144,"last_epoch_clean":145,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:55:27.277343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:55:27.277343+0000","last_clean_scrub_stamp":"2023-08-09T13:55:27.277343+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T17:15:19.603732+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":244,"reported_epoch":146,"state":"active+clean","last_fresh":"2023-08-09T13:55:29.371695+0000","last_change":"2023-08-09T13:33:55.886389+0000","last_active":"2023-08-09T13:55:29.371695+0000","last_peered":"2023-08-09T13:55:29.371695+0000","last_clean":"2023-08-09T13:55:29.371695+0000","last_became_active":"2023-08-09T13:33:55.885638+0000","last_became_peered":"2023-08-09T13:33:55.885638+0000","last_unstale":"2023-08-09T13:55:29.371695+0000","last_undegraded":"2023-08-09T13:55:29.371695+0000","last_fullsized":"2023-08-09T13:55:29.371695+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:33:54.845869+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:33:54.845869+0000","last_clean_scrub_stamp":"2023-08-09T13:33:54.845869+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T22:03:00.007141+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]}],"pool_stats":[{"poolid":19,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":18,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":38,"ondisk_log_size":38,"up":64,"acting":64,"num_store_stats":8},{"poolid":2,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1099,"num_read_kb":2242,"num_write":363,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":230,"ondisk_log_size":230,"up":2,"acting":2,"num_store_stats":2},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":8}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834575104,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":37552,"kb_used_data":2340,"kb_used_omap":1,"kb_used_meta":35198,"kb_avail":94334288,"statfs":{"total":96636764160,"available":96598310912,"internally_reserved":0,"allocated":2396160,"data_stored":1556377,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1781,"internal_metadata":36043019},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Aug 9 13:55:29 2023","interfaces":[{"interface":"back","average":{"1min":1.2070000000000001,"5min":1.1160000000000001,"15min":1.198},"min":{"1min":0.73099999999999998,"5min":0.60499999999999998,"15min":0.60499999999999998},"max":{"1min":1.71,"5min":1.869,"15min":1.8919999999999999},"last":0.75700000000000001},{"interface":"front","average":{"1min":1.1200000000000001,"5min":1.171,"15min":1.214},"min":{"1min":0.745,"5min":0.64200000000000002,"15min":0.64200000000000002},"max":{"1min":1.6870000000000001,"5min":2.3399999999999999,"15min":2.3399999999999999},"last":0.745}]},{"osd":1,"last update":"Wed Aug 9 13:55:29 2023","interfaces":[{"interface":"back","average":{"1min":1.276,"5min":1.258,"15min":1.2629999999999999},"min":{"1min":0.53100000000000003,"5min":0.53100000000000003,"15min":0.53100000000000003},"max":{"1min":1.6639999999999999,"5min":2.1989999999999998,"15min":2.1989999999999998},"last":0.53100000000000003},{"interface":"front","average":{"1min":1.226,"5min":1.212,"15min":1.2549999999999999},"min":{"1min":0.68200000000000005,"5min":0.68200000000000005,"15min":0.59299999999999997},"max":{"1min":1.8460000000000001,"5min":1.998,"15min":1.998},"last":0.68200000000000005}]},{"osd":2,"last update":"Wed Aug 9 13:55:29 2023","interfaces":[{"interface":"back","average":{"1min":1.1639999999999999,"5min":1.2250000000000001,"15min":1.248},"min":{"1min":0.76300000000000001,"5min":0.68300000000000005,"15min":0.65700000000000003},"max":{"1min":1.5620000000000001,"5min":2.0089999999999999,"15min":2.0129999999999999},"last":0.77400000000000002},{"interface":"front","average":{"1min":1.2190000000000001,"5min":1.2410000000000001,"15min":1.3300000000000001},"min":{"1min":0.70499999999999996,"5min":0.58199999999999996,"15min":0.58199999999999996},"max":{"1min":1.6619999999999999,"5min":2.359,"15min":2.359},"last":0.70499999999999996}]},{"osd":3,"last update":"Wed Aug 9 13:55:29 2023","interfaces":[{"interface":"back","average":{"1min":1.2,"5min":1.284,"15min":1.3260000000000001},"min":{"1min":0.73199999999999998,"5min":0.63,"15min":0.63},"max":{"1min":1.694,"5min":2.4380000000000002,"15min":2.4380000000000002},"last":0.73199999999999998},{"interface":"front","average":{"1min":1.3460000000000001,"5min":1.3779999999999999,"15min":1.415},"min":{"1min":0.78700000000000003,"5min":0.65700000000000003,"15min":0.65700000000000003},"max":{"1min":1.7889999999999999,"5min":2.6360000000000001,"15min":2.6360000000000001},"last":0.78700000000000003}]},{"osd":4,"last update":"Wed Aug 9 13:55:29 2023","interfaces":[{"interface":"back","average":{"1min":1.1719999999999999,"5min":1.234,"15min":1.2609999999999999},"min":{"1min":0.57699999999999996,"5min":0.57699999999999996,"15min":0.57699999999999996},"max":{"1min":1.647,"5min":2.097,"15min":2.097},"last":0.57699999999999996},{"interface":"front","average":{"1min":1.256,"5min":1.335,"15min":1.337},"min":{"1min":0.57499999999999996,"5min":0.52300000000000002,"15min":0.52300000000000002},"max":{"1min":1.641,"5min":2.6890000000000001,"15min":2.6890000000000001},"last":0.60399999999999998}]},{"osd":5,"last update":"Wed Aug 9 13:55:29 2023","interfaces":[{"interface":"back","average":{"1min":1.3029999999999999,"5min":1.427,"15min":1.425},"min":{"1min":0.64200000000000002,"5min":0.501,"15min":0.501},"max":{"1min":1.7589999999999999,"5min":2.7509999999999999,"15min":2.7509999999999999},"last":0.64200000000000002},{"interface":"front","average":{"1min":1.3899999999999999,"5min":1.429,"15min":1.4399999999999999},"min":{"1min":0.629,"5min":0.629,"15min":0.629},"max":{"1min":1.95,"5min":2.8639999999999999,"15min":2.8639999999999999},"last":0.629}]},{"osd":6,"last update":"Wed Aug 9 13:55:29 2023","interfaces":[{"interface":"back","average":{"1min":1.4610000000000001,"5min":1.528,"15min":1.532},"min":{"1min":0.71899999999999997,"5min":0.71899999999999997,"15min":0.71899999999999997},"max":{"1min":1.7889999999999999,"5min":2.7370000000000001,"15min":2.7370000000000001},"last":0.71899999999999997},{"interface":"front","average":{"1min":1.4379999999999999,"5min":1.4630000000000001,"15min":1.4730000000000001},"min":{"1min":0.624,"5min":0.57099999999999995,"15min":0.57099999999999995},"max":{"1min":1.8959999999999999,"5min":2.7850000000000001,"15min":2.7850000000000001},"last":0.66700000000000004}]}]},{"osd":6,"up_from":13,"seq":55834575104,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":36972,"kb_used_data":1756,"kb_used_omap":1,"kb_used_meta":35198,"kb_avail":94334868,"statfs":{"total":96636764160,"available":96598904832,"internally_reserved":0,"allocated":1798144,"data_stored":964648,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1781,"internal_metadata":36043019},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Aug 9 13:55:28 2023","interfaces":[{"interface":"back","average":{"1min":1.242,"5min":1.141,"15min":1.1519999999999999},"min":{"1min":0.79700000000000004,"5min":0.47999999999999998,"15min":0.47999999999999998},"max":{"1min":2.2149999999999999,"5min":2.2149999999999999,"15min":2.2149999999999999},"last":1.468},{"interface":"front","average":{"1min":1.1779999999999999,"5min":1.159,"15min":1.3009999999999999},"min":{"1min":0.69499999999999995,"5min":0.51800000000000002,"15min":0.51800000000000002},"max":{"1min":1.6579999999999999,"5min":1.8520000000000001,"15min":28.036000000000001},"last":1.0109999999999999}]},{"osd":1,"last update":"Wed Aug 9 13:55:28 2023","interfaces":[{"interface":"back","average":{"1min":1.2250000000000001,"5min":1.1639999999999999,"15min":1.224},"min":{"1min":0.71699999999999997,"5min":0.48999999999999999,"15min":0.48999999999999999},"max":{"1min":1.9019999999999999,"5min":2.2320000000000002,"15min":7.742},"last":1.208},{"interface":"front","average":{"1min":1.272,"5min":1.2250000000000001,"15min":1.258},"min":{"1min":0.79500000000000004,"5min":0.63,"15min":0.63},"max":{"1min":2.1120000000000001,"5min":2.1120000000000001,"15min":2.1120000000000001},"last":1.339}]},{"osd":2,"last update":"Wed Aug 9 13:55:28 2023","interfaces":[{"interface":"back","average":{"1min":1.3480000000000001,"5min":1.2869999999999999,"15min":1.3200000000000001},"min":{"1min":0.88200000000000001,"5min":0.627,"15min":0.627},"max":{"1min":1.988,"5min":2.1899999999999999,"15min":4.0640000000000001},"last":1.121},{"interface":"front","average":{"1min":1.2190000000000001,"5min":1.2569999999999999,"15min":1.339},"min":{"1min":0.77300000000000002,"5min":0.77300000000000002,"15min":0.70999999999999996},"max":{"1min":1.637,"5min":2.5819999999999999,"15min":13.154999999999999},"last":1.5}]},{"osd":3,"last update":"Wed Aug 9 13:55:28 2023","interfaces":[{"interface":"back","average":{"1min":1.3069999999999999,"5min":1.3120000000000001,"15min":1.4139999999999999},"min":{"1min":0.89100000000000001,"5min":0.67200000000000004,"15min":0.67200000000000004},"max":{"1min":2.1749999999999998,"5min":2.1749999999999998,"15min":13.349},"last":1.4019999999999999},{"interface":"front","average":{"1min":1.4179999999999999,"5min":1.3029999999999999,"15min":1.3300000000000001},"min":{"1min":0.95699999999999996,"5min":0.75700000000000001,"15min":0.75700000000000001},"max":{"1min":1.984,"5min":1.984,"15min":7.9029999999999996},"last":1.53}]},{"osd":4,"last update":"Wed Aug 9 13:55:28 2023","interfaces":[{"interface":"back","average":{"1min":1.3300000000000001,"5min":1.294,"15min":1.2869999999999999},"min":{"1min":0.91000000000000003,"5min":0.35099999999999998,"15min":0.35099999999999998},"max":{"1min":1.7470000000000001,"5min":1.833,"15min":5.2290000000000001},"last":0.96299999999999997},{"interface":"front","average":{"1min":1.397,"5min":1.3520000000000001,"15min":1.3839999999999999},"min":{"1min":1.056,"5min":0.54800000000000004,"15min":0.54800000000000004},"max":{"1min":2.0299999999999998,"5min":2.0299999999999998,"15min":12.1},"last":1.1539999999999999}]},{"osd":5,"last update":"Wed Aug 9 13:55:28 2023","interfaces":[{"interface":"back","average":{"1min":1.351,"5min":1.3160000000000001,"15min":1.3660000000000001},"min":{"1min":0.85399999999999998,"5min":0.51000000000000001,"15min":0.51000000000000001},"max":{"1min":1.7310000000000001,"5min":2.1349999999999998,"15min":8.1899999999999995},"last":1.4410000000000001},{"interface":"front","average":{"1min":1.6020000000000001,"5min":1.5649999999999999,"15min":1.593},"min":{"1min":1.1859999999999999,"5min":0.42399999999999999,"15min":0.42399999999999999},"max":{"1min":2.29,"5min":2.29,"15min":8.8889999999999993},"last":1.5820000000000001}]},{"osd":7,"last update":"Wed Aug 9 13:55:28 2023","interfaces":[{"interface":"back","average":{"1min":1.5700000000000001,"5min":1.5700000000000001,"15min":1.5940000000000001},"min":{"1min":1.3140000000000001,"5min":0.69299999999999995,"15min":0.62},"max":{"1min":2.0499999999999998,"5min":2.4529999999999998,"15min":9.3320000000000007},"last":1.6519999999999999},{"interface":"front","average":{"1min":1.603,"5min":1.5189999999999999,"15min":1.552},"min":{"1min":1.3520000000000001,"5min":0.59899999999999998,"15min":0.59899999999999998},"max":{"1min":2.117,"5min":2.2909999999999999,"15min":8.0399999999999991},"last":1.611}]}]},{"osd":5,"up_from":13,"seq":55834575104,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":59120,"kb_used_data":1760,"kb_used_omap":1,"kb_used_meta":57342,"kb_avail":94312720,"statfs":{"total":96636764160,"available":96576225280,"internally_reserved":0,"allocated":1802240,"data_stored":966009,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1781,"internal_metadata":58718475},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Aug 9 13:55:26 2023","interfaces":[{"interface":"back","average":{"1min":1.0329999999999999,"5min":1.21,"15min":1.1930000000000001},"min":{"1min":0.70499999999999996,"5min":0.59099999999999997,"15min":0.59099999999999997},"max":{"1min":1.6120000000000001,"5min":8.9299999999999997,"15min":8.9299999999999997},"last":0.89600000000000002},{"interface":"front","average":{"1min":1.121,"5min":1.25,"15min":1.22},"min":{"1min":0.71399999999999997,"5min":0.56000000000000005,"15min":0.56000000000000005},"max":{"1min":1.6379999999999999,"5min":8.7140000000000004,"15min":8.7140000000000004},"last":1.0840000000000001}]},{"osd":1,"last update":"Wed Aug 9 13:55:26 2023","interfaces":[{"interface":"back","average":{"1min":1.1699999999999999,"5min":1.2310000000000001,"15min":1.236},"min":{"1min":0.79500000000000004,"5min":0.66600000000000004,"15min":0.66600000000000004},"max":{"1min":1.5640000000000001,"5min":8.2720000000000002,"15min":8.2720000000000002},"last":1.1890000000000001},{"interface":"front","average":{"1min":1.26,"5min":1.3420000000000001,"15min":1.323},"min":{"1min":0.91900000000000004,"5min":0.66300000000000003,"15min":0.66300000000000003},"max":{"1min":1.554,"5min":9.0020000000000007,"15min":9.0020000000000007},"last":1.1419999999999999}]},{"osd":2,"last update":"Wed Aug 9 13:55:26 2023","interfaces":[{"interface":"back","average":{"1min":1.163,"5min":1.325,"15min":1.3029999999999999},"min":{"1min":0.72099999999999997,"5min":0.72099999999999997,"15min":0.68999999999999995},"max":{"1min":1.7270000000000001,"5min":8.6050000000000004,"15min":8.6050000000000004},"last":0.85699999999999998},{"interface":"front","average":{"1min":1.2230000000000001,"5min":1.345,"15min":1.335},"min":{"1min":0.81399999999999995,"5min":0.78300000000000003,"15min":0.78300000000000003},"max":{"1min":1.5980000000000001,"5min":8.4149999999999991,"15min":8.4149999999999991},"last":0.95899999999999996}]},{"osd":3,"last update":"Wed Aug 9 13:55:26 2023","interfaces":[{"interface":"back","average":{"1min":1.1930000000000001,"5min":1.373,"15min":1.365},"min":{"1min":0.83699999999999997,"5min":0.66200000000000003,"15min":0.66200000000000003},"max":{"1min":1.7829999999999999,"5min":9.0470000000000006,"15min":9.0470000000000006},"last":1.028},{"interface":"front","average":{"1min":1.379,"5min":1.4770000000000001,"15min":1.4490000000000001},"min":{"1min":1.04,"5min":0.63,"15min":0.63},"max":{"1min":1.6259999999999999,"5min":8.7669999999999995,"15min":8.7669999999999995},"last":1.2609999999999999}]},{"osd":4,"last update":"Wed Aug 9 13:55:26 2023","interfaces":[{"interface":"back","average":{"1min":1.266,"5min":1.4199999999999999,"15min":1.3400000000000001},"min":{"1min":1.0129999999999999,"5min":0.54300000000000004,"15min":0.54300000000000004},"max":{"1min":1.4990000000000001,"5min":8.4909999999999997,"15min":8.4909999999999997},"last":1.319},{"interface":"front","average":{"1min":1.4810000000000001,"5min":1.5349999999999999,"15min":1.4239999999999999},"min":{"1min":1.2390000000000001,"5min":0.68899999999999995,"15min":0.68899999999999995},"max":{"1min":1.8979999999999999,"5min":9.4589999999999996,"15min":9.4589999999999996},"last":1.2390000000000001}]},{"osd":6,"last update":"Wed Aug 9 13:55:26 2023","interfaces":[{"interface":"back","average":{"1min":1.347,"5min":1.6259999999999999,"15min":1.47},"min":{"1min":0.98199999999999998,"5min":0.51000000000000001,"15min":0.51000000000000001},"max":{"1min":1.641,"5min":17.407,"15min":17.407},"last":1.45},{"interface":"front","average":{"1min":1.4139999999999999,"5min":1.6599999999999999,"15min":1.5309999999999999},"min":{"1min":1.05,"5min":0.71199999999999997,"15min":0.71199999999999997},"max":{"1min":1.6619999999999999,"5min":16.356000000000002,"15min":16.356000000000002},"last":1.413}]},{"osd":7,"last update":"Wed Aug 9 13:55:26 2023","interfaces":[{"interface":"back","average":{"1min":1.5629999999999999,"5min":1.681,"15min":1.593},"min":{"1min":1.333,"5min":0.74299999999999999,"15min":0.74299999999999999},"max":{"1min":2,"5min":9.9239999999999995,"15min":9.9239999999999995},"last":1.375},{"interface":"front","average":{"1min":1.5780000000000001,"5min":1.742,"15min":1.651},"min":{"1min":1.302,"5min":0.65600000000000003,"15min":0.65600000000000003},"max":{"1min":1.8280000000000001,"5min":10.789,"15min":10.789},"last":1.518}]}]},{"osd":4,"up_from":13,"seq":55834575104,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32880,"kb_used_data":1760,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":94338960,"statfs":{"total":96636764160,"available":96603095040,"internally_reserved":0,"allocated":1802240,"data_stored":966009,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1781,"internal_metadata":31848715},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":7,"apply_latency_ms":7,"commit_latency_ns":7000000,"apply_latency_ns":7000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Aug 9 13:54:30 2023","interfaces":[{"interface":"back","average":{"1min":1.163,"5min":1.165,"15min":1.198},"min":{"1min":0.54500000000000004,"5min":0.54500000000000004,"15min":0.54500000000000004},"max":{"1min":1.73,"5min":1.768,"15min":2.524},"last":1.5},{"interface":"front","average":{"1min":1.226,"5min":1.1890000000000001,"15min":1.2450000000000001},"min":{"1min":0.60399999999999998,"5min":0.60399999999999998,"15min":0.60399999999999998},"max":{"1min":2.0779999999999998,"5min":2.0779999999999998,"15min":2.0779999999999998},"last":1.008}]},{"osd":1,"last update":"Wed Aug 9 13:54:30 2023","interfaces":[{"interface":"back","average":{"1min":1.361,"5min":1.258,"15min":1.2949999999999999},"min":{"1min":0.82399999999999995,"5min":0.70999999999999996,"15min":0.67800000000000005},"max":{"1min":1.887,"5min":1.887,"15min":1.887},"last":1.3049999999999999},{"interface":"front","average":{"1min":1.1699999999999999,"5min":1.1910000000000001,"15min":1.24},"min":{"1min":0.74399999999999999,"5min":0.68899999999999995,"15min":0.68899999999999995},"max":{"1min":1.5509999999999999,"5min":1.9330000000000001,"15min":1.9330000000000001},"last":1.4079999999999999}]},{"osd":2,"last update":"Wed Aug 9 13:54:30 2023","interfaces":[{"interface":"back","average":{"1min":1.3140000000000001,"5min":1.2370000000000001,"15min":1.2649999999999999},"min":{"1min":0.79000000000000004,"5min":0.69999999999999996,"15min":0.621},"max":{"1min":1.7050000000000001,"5min":1.7050000000000001,"15min":2.6429999999999998},"last":1.081},{"interface":"front","average":{"1min":1.3480000000000001,"5min":1.3280000000000001,"15min":1.3640000000000001},"min":{"1min":0.77300000000000002,"5min":0.76000000000000001,"15min":0.76000000000000001},"max":{"1min":1.6870000000000001,"5min":2.0910000000000002,"15min":2.0910000000000002},"last":1.1539999999999999}]},{"osd":3,"last update":"Wed Aug 9 13:54:30 2023","interfaces":[{"interface":"back","average":{"1min":1.2030000000000001,"5min":1.2569999999999999,"15min":1.292},"min":{"1min":0.80000000000000004,"5min":0.64700000000000002,"15min":0.64700000000000002},"max":{"1min":1.5589999999999999,"5min":1.921,"15min":2.6099999999999999},"last":1.127},{"interface":"front","average":{"1min":1.379,"5min":1.3680000000000001,"15min":1.3839999999999999},"min":{"1min":1.147,"5min":0.86299999999999999,"15min":0.86299999999999999},"max":{"1min":1.9419999999999999,"5min":2.3119999999999998,"15min":2.7629999999999999},"last":1.202}]},{"osd":5,"last update":"Wed Aug 9 13:54:30 2023","interfaces":[{"interface":"back","average":{"1min":1.296,"5min":1.337,"15min":1.3360000000000001},"min":{"1min":0.58699999999999997,"5min":0.58699999999999997,"15min":0.58699999999999997},"max":{"1min":1.97,"5min":2.0329999999999999,"15min":2.0329999999999999},"last":1.4390000000000001},{"interface":"front","average":{"1min":1.256,"5min":1.3120000000000001,"15min":1.3160000000000001},"min":{"1min":0.67500000000000004,"5min":0.65600000000000003,"15min":0.65600000000000003},"max":{"1min":1.6850000000000001,"5min":2.5880000000000001,"15min":2.7090000000000001},"last":1.252}]},{"osd":6,"last update":"Wed Aug 9 13:54:30 2023","interfaces":[{"interface":"back","average":{"1min":1.3049999999999999,"5min":1.4019999999999999,"15min":1.3440000000000001},"min":{"1min":0.752,"5min":0.752,"15min":0.59099999999999997},"max":{"1min":1.952,"5min":2.5430000000000001,"15min":2.8559999999999999},"last":1.373},{"interface":"front","average":{"1min":1.4970000000000001,"5min":1.5069999999999999,"15min":1.512},"min":{"1min":0.65100000000000002,"5min":0.52600000000000002,"15min":0.52600000000000002},"max":{"1min":2.2050000000000001,"5min":2.2050000000000001,"15min":2.2719999999999998},"last":1.5309999999999999}]},{"osd":7,"last update":"Wed Aug 9 13:54:30 2023","interfaces":[{"interface":"back","average":{"1min":1.3600000000000001,"5min":1.4470000000000001,"15min":1.472},"min":{"1min":0.72399999999999998,"5min":0.53100000000000003,"15min":0.53100000000000003},"max":{"1min":1.8109999999999999,"5min":2.4279999999999999,"15min":2.8050000000000002},"last":1.333},{"interface":"front","average":{"1min":1.518,"5min":1.5469999999999999,"15min":1.5309999999999999},"min":{"1min":0.70099999999999996,"5min":0.70099999999999996,"15min":0.70099999999999996},"max":{"1min":2.105,"5min":2.6429999999999998,"15min":3.415},"last":1.2869999999999999}]}]},{"osd":3,"up_from":13,"seq":55834575105,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":36976,"kb_used_data":1760,"kb_used_omap":1,"kb_used_meta":35198,"kb_avail":94334864,"statfs":{"total":96636764160,"available":96598900736,"internally_reserved":0,"allocated":1802240,"data_stored":966009,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1781,"internal_metadata":36043019},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Aug 9 13:54:28 2023","interfaces":[{"interface":"back","average":{"1min":0.91100000000000003,"5min":0.86399999999999999,"15min":0.90200000000000002},"min":{"1min":0.60699999999999998,"5min":0.43099999999999999,"15min":0.27200000000000002},"max":{"1min":1.4219999999999999,"5min":1.589,"15min":1.8500000000000001},"last":0.72499999999999998},{"interface":"front","average":{"1min":0.95599999999999996,"5min":0.98099999999999998,"15min":1.093},"min":{"1min":0.57799999999999996,"5min":0.36299999999999999,"15min":0.36299999999999999},"max":{"1min":1.5880000000000001,"5min":1.6910000000000001,"15min":2.0590000000000002},"last":0.92300000000000004}]},{"osd":1,"last update":"Wed Aug 9 13:54:28 2023","interfaces":[{"interface":"back","average":{"1min":0.97499999999999998,"5min":0.95299999999999996,"15min":1.0149999999999999},"min":{"1min":0.64200000000000002,"5min":0.51500000000000001,"15min":0.496},"max":{"1min":1.573,"5min":1.9410000000000001,"15min":2.5649999999999999},"last":0.77200000000000002},{"interface":"front","average":{"1min":1.0609999999999999,"5min":1.0740000000000001,"15min":1.1639999999999999},"min":{"1min":0.48699999999999999,"5min":0.48399999999999999,"15min":0.48399999999999999},"max":{"1min":1.637,"5min":1.762,"15min":2.6760000000000002},"last":1.0109999999999999}]},{"osd":2,"last update":"Wed Aug 9 13:54:28 2023","interfaces":[{"interface":"back","average":{"1min":1.1419999999999999,"5min":1.099,"15min":1.1739999999999999},"min":{"1min":0.64100000000000001,"5min":0.42999999999999999,"15min":0.42999999999999999},"max":{"1min":1.7629999999999999,"5min":1.7629999999999999,"15min":1.913},"last":0.64400000000000002},{"interface":"front","average":{"1min":1.246,"5min":1.2230000000000001,"15min":1.3029999999999999},"min":{"1min":0.69899999999999995,"5min":0.52400000000000002,"15min":0.52300000000000002},"max":{"1min":1.8120000000000001,"5min":1.9399999999999999,"15min":2.0259999999999998},"last":1.163}]},{"osd":4,"last update":"Wed Aug 9 13:54:28 2023","interfaces":[{"interface":"back","average":{"1min":1.321,"5min":1.2350000000000001,"15min":1.274},"min":{"1min":0.995,"5min":0.54900000000000004,"15min":0.54900000000000004},"max":{"1min":1.778,"5min":1.778,"15min":1.8160000000000001},"last":1.1160000000000001},{"interface":"front","average":{"1min":1.3640000000000001,"5min":1.3560000000000001,"15min":1.403},"min":{"1min":0.78200000000000003,"5min":0.67600000000000005,"15min":0.628},"max":{"1min":1.74,"5min":2.782,"15min":2.782},"last":1.0640000000000001}]},{"osd":5,"last update":"Wed Aug 9 13:54:28 2023","interfaces":[{"interface":"back","average":{"1min":1.3600000000000001,"5min":1.397,"15min":1.4670000000000001},"min":{"1min":0.73299999999999998,"5min":0.67600000000000005,"15min":0.67600000000000005},"max":{"1min":1.8100000000000001,"5min":2.0489999999999999,"15min":2.254},"last":1.2689999999999999},{"interface":"front","average":{"1min":1.4490000000000001,"5min":1.409,"15min":1.4850000000000001},"min":{"1min":0.88200000000000001,"5min":0.66000000000000003,"15min":0.65900000000000003},"max":{"1min":1.9419999999999999,"5min":2.3069999999999999,"15min":2.3069999999999999},"last":1.298}]},{"osd":6,"last update":"Wed Aug 9 13:54:28 2023","interfaces":[{"interface":"back","average":{"1min":1.3049999999999999,"5min":1.355,"15min":1.4379999999999999},"min":{"1min":0.84899999999999998,"5min":0.78200000000000003,"15min":0.68200000000000005},"max":{"1min":1.752,"5min":2.8340000000000001,"15min":2.8340000000000001},"last":1.236},{"interface":"front","average":{"1min":1.5009999999999999,"5min":1.4790000000000001,"15min":1.548},"min":{"1min":0.85699999999999998,"5min":0.64100000000000001,"15min":0.64100000000000001},"max":{"1min":2.3919999999999999,"5min":2.9060000000000001,"15min":2.9060000000000001},"last":1.202}]},{"osd":7,"last update":"Wed Aug 9 13:54:28 2023","interfaces":[{"interface":"back","average":{"1min":1.536,"5min":1.504,"15min":1.5980000000000001},"min":{"1min":0.83599999999999997,"5min":0.624,"15min":0.624},"max":{"1min":1.8879999999999999,"5min":2.46,"15min":2.46},"last":1.403},{"interface":"front","average":{"1min":1.492,"5min":1.454,"15min":1.542},"min":{"1min":0.92300000000000004,"5min":0.71199999999999997,"15min":0.64200000000000002},"max":{"1min":2.3159999999999998,"5min":2.7200000000000002,"15min":2.7200000000000002},"last":1.349}]}]},{"osd":2,"up_from":13,"seq":55834575104,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":41072,"kb_used_data":1760,"kb_used_omap":1,"kb_used_meta":39294,"kb_avail":94330768,"statfs":{"total":96636764160,"available":96594706432,"internally_reserved":0,"allocated":1802240,"data_stored":966009,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1781,"internal_metadata":40237323},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Aug 9 13:54:24 2023","interfaces":[{"interface":"back","average":{"1min":0.88500000000000001,"5min":0.89700000000000002,"15min":0.94799999999999995},"min":{"1min":0.57299999999999995,"5min":0.35499999999999998,"15min":0.35499999999999998},"max":{"1min":1.5860000000000001,"5min":1.5860000000000001,"15min":1.752},"last":1.123},{"interface":"front","average":{"1min":1.0289999999999999,"5min":0.97199999999999998,"15min":1},"min":{"1min":0.66900000000000004,"5min":0.38200000000000001,"15min":0.374},"max":{"1min":1.4810000000000001,"5min":1.75,"15min":2.3929999999999998},"last":0.99299999999999999}]},{"osd":1,"last update":"Wed Aug 9 13:54:24 2023","interfaces":[{"interface":"back","average":{"1min":1.1970000000000001,"5min":1.1379999999999999,"15min":1.1830000000000001},"min":{"1min":0.64200000000000002,"5min":0.39400000000000002,"15min":0.39400000000000002},"max":{"1min":1.8109999999999999,"5min":1.8109999999999999,"15min":1.8420000000000001},"last":0.75800000000000001},{"interface":"front","average":{"1min":1.2929999999999999,"5min":1.1919999999999999,"15min":1.304},"min":{"1min":0.66800000000000004,"5min":0.435,"15min":0.435},"max":{"1min":1.883,"5min":1.8999999999999999,"15min":2.4870000000000001},"last":0.90900000000000003}]},{"osd":3,"last update":"Wed Aug 9 13:54:24 2023","interfaces":[{"interface":"back","average":{"1min":0.96399999999999997,"5min":0.95799999999999996,"15min":1.0289999999999999},"min":{"1min":0.624,"5min":0.41399999999999998,"15min":0.38900000000000001},"max":{"1min":1.3089999999999999,"5min":1.7130000000000001,"15min":1.7609999999999999},"last":0.57499999999999996},{"interface":"front","average":{"1min":1.256,"5min":1.177,"15min":1.2889999999999999},"min":{"1min":0.81899999999999995,"5min":0.45500000000000002,"15min":0.45500000000000002},"max":{"1min":1.669,"5min":1.835,"15min":8.8130000000000006},"last":1.05}]},{"osd":4,"last update":"Wed Aug 9 13:54:24 2023","interfaces":[{"interface":"back","average":{"1min":1.425,"5min":1.3799999999999999,"15min":1.4430000000000001},"min":{"1min":0.98399999999999999,"5min":0.622,"15min":0.622},"max":{"1min":1.706,"5min":2.1640000000000001,"15min":2.1640000000000001},"last":1.4299999999999999},{"interface":"front","average":{"1min":1.341,"5min":1.353,"15min":1.3819999999999999},"min":{"1min":0.96599999999999997,"5min":0.64800000000000002,"15min":0.64800000000000002},"max":{"1min":1.613,"5min":2.472,"15min":2.472},"last":1.0840000000000001}]},{"osd":5,"last update":"Wed Aug 9 13:54:24 2023","interfaces":[{"interface":"back","average":{"1min":1.4510000000000001,"5min":1.4259999999999999,"15min":1.4530000000000001},"min":{"1min":1.0660000000000001,"5min":0.71199999999999997,"15min":0.59999999999999998},"max":{"1min":1.9259999999999999,"5min":2.4900000000000002,"15min":2.5840000000000001},"last":1.3240000000000001},{"interface":"front","average":{"1min":1.391,"5min":1.335,"15min":1.3560000000000001},"min":{"1min":1.083,"5min":0.73699999999999999,"15min":0.56799999999999995},"max":{"1min":1.7450000000000001,"5min":2.3519999999999999,"15min":3.5659999999999998},"last":1.147}]},{"osd":6,"last update":"Wed Aug 9 13:54:24 2023","interfaces":[{"interface":"back","average":{"1min":1.4039999999999999,"5min":1.377,"15min":1.4279999999999999},"min":{"1min":1.0469999999999999,"5min":0.77600000000000002,"15min":0.73799999999999999},"max":{"1min":1.8340000000000001,"5min":2.5110000000000001,"15min":3.5099999999999998},"last":0.94799999999999995},{"interface":"front","average":{"1min":1.603,"5min":1.496,"15min":1.5429999999999999},"min":{"1min":1.323,"5min":0.66100000000000003,"15min":0.66100000000000003},"max":{"1min":1.962,"5min":2.3769999999999998,"15min":2.3769999999999998},"last":1.353}]},{"osd":7,"last update":"Wed Aug 9 13:54:24 2023","interfaces":[{"interface":"back","average":{"1min":1.526,"5min":1.482,"15min":1.5309999999999999},"min":{"1min":0.91600000000000004,"5min":0.78800000000000003,"15min":0.64600000000000002},"max":{"1min":1.8280000000000001,"5min":2.1970000000000001,"15min":2.5249999999999999},"last":1.1719999999999999},{"interface":"front","average":{"1min":1.5349999999999999,"5min":1.456,"15min":1.5089999999999999},"min":{"1min":1.214,"5min":0.66100000000000003,"15min":0.66100000000000003},"max":{"1min":1.9279999999999999,"5min":2.4500000000000002,"15min":3.609},"last":1.012}]}]},{"osd":1,"up_from":13,"seq":55834575104,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":47412,"kb_used_data":2340,"kb_used_omap":1,"kb_used_meta":45054,"kb_avail":94324428,"statfs":{"total":96636764160,"available":96588214272,"internally_reserved":0,"allocated":2396160,"data_stored":1556377,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1781,"internal_metadata":46135563},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Aug 9 13:54:40 2023","interfaces":[{"interface":"back","average":{"1min":1.018,"5min":0.91500000000000004,"15min":0.93500000000000005},"min":{"1min":0.81399999999999995,"5min":0.45800000000000002,"15min":0.45800000000000002},"max":{"1min":1.262,"5min":1.7110000000000001,"15min":2.1080000000000001},"last":0.94699999999999995},{"interface":"front","average":{"1min":1.0920000000000001,"5min":1.0129999999999999,"15min":1.0860000000000001},"min":{"1min":0.77200000000000002,"5min":0.40699999999999997,"15min":0.40699999999999997},"max":{"1min":1.5109999999999999,"5min":2.5070000000000001,"15min":2.5070000000000001},"last":1.304}]},{"osd":2,"last update":"Wed Aug 9 13:54:40 2023","interfaces":[{"interface":"back","average":{"1min":1.056,"5min":0.97899999999999998,"15min":0.995},"min":{"1min":0.81799999999999995,"5min":0.41999999999999998,"15min":0.41099999999999998},"max":{"1min":1.734,"5min":1.798,"15min":1.907},"last":0.89900000000000002},{"interface":"front","average":{"1min":1.2150000000000001,"5min":1.089,"15min":1.1659999999999999},"min":{"1min":0.82099999999999995,"5min":0.50700000000000001,"15min":0.50700000000000001},"max":{"1min":1.645,"5min":1.645,"15min":2.145},"last":1.212}]},{"osd":3,"last update":"Wed Aug 9 13:54:40 2023","interfaces":[{"interface":"back","average":{"1min":1.391,"5min":1.2190000000000001,"15min":1.3120000000000001},"min":{"1min":0.89900000000000002,"5min":0.54600000000000004,"15min":0.54400000000000004},"max":{"1min":1.748,"5min":1.994,"15min":2.3210000000000002},"last":1.6830000000000001},{"interface":"front","average":{"1min":1.262,"5min":1.1639999999999999,"15min":1.2410000000000001},"min":{"1min":0.873,"5min":0.48299999999999998,"15min":0.45300000000000001},"max":{"1min":1.6830000000000001,"5min":2.3959999999999999,"15min":2.7669999999999999},"last":0.80100000000000005}]},{"osd":4,"last update":"Wed Aug 9 13:54:40 2023","interfaces":[{"interface":"back","average":{"1min":1.4359999999999999,"5min":1.353,"15min":1.409},"min":{"1min":1.0149999999999999,"5min":0.79800000000000004,"15min":0.58099999999999996},"max":{"1min":1.8460000000000001,"5min":2.0739999999999998,"15min":2.4089999999999998},"last":1.085},{"interface":"front","average":{"1min":1.3580000000000001,"5min":1.3160000000000001,"15min":1.3460000000000001},"min":{"1min":1.0740000000000001,"5min":0.71099999999999997,"15min":0.70099999999999996},"max":{"1min":1.77,"5min":2.0019999999999998,"15min":2.0019999999999998},"last":1.5820000000000001}]},{"osd":5,"last update":"Wed Aug 9 13:54:40 2023","interfaces":[{"interface":"back","average":{"1min":1.448,"5min":1.357,"15min":1.3799999999999999},"min":{"1min":1.075,"5min":0.75900000000000001,"15min":0.75900000000000001},"max":{"1min":1.8899999999999999,"5min":1.9419999999999999,"15min":2.294},"last":1.4970000000000001},{"interface":"front","average":{"1min":1.5620000000000001,"5min":1.4299999999999999,"15min":1.506},"min":{"1min":1.2390000000000001,"5min":0.78400000000000003,"15min":0.78400000000000003},"max":{"1min":1.756,"5min":1.863,"15min":2.4649999999999999},"last":1.798}]},{"osd":6,"last update":"Wed Aug 9 13:54:40 2023","interfaces":[{"interface":"back","average":{"1min":1.611,"5min":1.4770000000000001,"15min":1.514},"min":{"1min":1.3069999999999999,"5min":0.96599999999999997,"15min":0.67400000000000004},"max":{"1min":1.9199999999999999,"5min":2.23,"15min":2.5270000000000001},"last":1.53},{"interface":"front","average":{"1min":1.516,"5min":1.4550000000000001,"15min":1.496},"min":{"1min":1.131,"5min":0.73599999999999999,"15min":0.73599999999999999},"max":{"1min":1.748,"5min":2.3359999999999999,"15min":2.3359999999999999},"last":1.6379999999999999}]},{"osd":7,"last update":"Wed Aug 9 13:54:40 2023","interfaces":[{"interface":"back","average":{"1min":1.6799999999999999,"5min":1.5189999999999999,"15min":1.601},"min":{"1min":1.389,"5min":0.72099999999999997,"15min":0.72099999999999997},"max":{"1min":1.8939999999999999,"5min":2.4340000000000002,"15min":2.5790000000000002},"last":1.925},{"interface":"front","average":{"1min":1.651,"5min":1.544,"15min":1.597},"min":{"1min":1.4490000000000001,"5min":0.85099999999999998,"15min":0.85099999999999998},"max":{"1min":2.0110000000000001,"5min":2.375,"15min":2.6509999999999998},"last":1.7410000000000001}]}]},{"osd":0,"up_from":13,"seq":55834575104,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":41068,"kb_used_data":1756,"kb_used_omap":1,"kb_used_meta":39294,"kb_avail":94330772,"statfs":{"total":96636764160,"available":96594710528,"internally_reserved":0,"allocated":1798144,"data_stored":964648,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1781,"internal_metadata":40237323},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Wed Aug 9 13:54:31 2023","interfaces":[{"interface":"back","average":{"1min":0.88300000000000001,"5min":0.91500000000000004,"15min":0.92100000000000004},"min":{"1min":0.31900000000000001,"5min":0.31900000000000001,"15min":0.31900000000000001},"max":{"1min":2.2410000000000001,"5min":2.2410000000000001,"15min":2.2410000000000001},"last":0.80200000000000005},{"interface":"front","average":{"1min":0.93000000000000005,"5min":1.032,"15min":1.0920000000000001},"min":{"1min":0.34499999999999997,"5min":0.34499999999999997,"15min":0.34499999999999997},"max":{"1min":1.6479999999999999,"5min":1.9019999999999999,"15min":1.9019999999999999},"last":1.177}]},{"osd":2,"last update":"Wed Aug 9 13:54:31 2023","interfaces":[{"interface":"back","average":{"1min":1.016,"5min":1.0269999999999999,"15min":1.1100000000000001},"min":{"1min":0.55600000000000005,"5min":0.433,"15min":0.433},"max":{"1min":1.4470000000000001,"5min":1.786,"15min":1.786},"last":1.8799999999999999},{"interface":"front","average":{"1min":1.0269999999999999,"5min":1.099,"15min":1.165},"min":{"1min":0.47599999999999998,"5min":0.439,"15min":0.439},"max":{"1min":1.6000000000000001,"5min":2.0129999999999999,"15min":2.0129999999999999},"last":1.629}]},{"osd":3,"last update":"Wed Aug 9 13:54:31 2023","interfaces":[{"interface":"back","average":{"1min":0.94199999999999995,"5min":1.008,"15min":1.0840000000000001},"min":{"1min":0.44700000000000001,"5min":0.44700000000000001,"15min":0.44700000000000001},"max":{"1min":1.4379999999999999,"5min":1.7529999999999999,"15min":1.881},"last":0.876},{"interface":"front","average":{"1min":1.143,"5min":1.212,"15min":1.2909999999999999},"min":{"1min":0.52400000000000002,"5min":0.50800000000000001,"15min":0.50800000000000001},"max":{"1min":1.577,"5min":2.347,"15min":2.4649999999999999},"last":1.3999999999999999}]},{"osd":4,"last update":"Wed Aug 9 13:54:31 2023","interfaces":[{"interface":"back","average":{"1min":1.28,"5min":1.2909999999999999,"15min":1.333},"min":{"1min":0.76100000000000001,"5min":0.69699999999999995,"15min":0.69699999999999995},"max":{"1min":1.6870000000000001,"5min":1.9630000000000001,"15min":2.2309999999999999},"last":1.677},{"interface":"front","average":{"1min":1.2909999999999999,"5min":1.379,"15min":1.4359999999999999},"min":{"1min":0.871,"5min":0.82699999999999996,"15min":0.77600000000000002},"max":{"1min":1.655,"5min":2.3540000000000001,"15min":2.5390000000000001},"last":2.0150000000000001}]},{"osd":5,"last update":"Wed Aug 9 13:54:31 2023","interfaces":[{"interface":"back","average":{"1min":1.345,"5min":1.4059999999999999,"15min":1.454},"min":{"1min":0.98399999999999999,"5min":0.66300000000000003,"15min":0.66300000000000003},"max":{"1min":1.772,"5min":2.6669999999999998,"15min":2.6669999999999998},"last":1.75},{"interface":"front","average":{"1min":1.3460000000000001,"5min":1.3540000000000001,"15min":1.431},"min":{"1min":0.89900000000000002,"5min":0.70099999999999996,"15min":0.629},"max":{"1min":1.7210000000000001,"5min":1.9710000000000001,"15min":2.1499999999999999},"last":1.272}]},{"osd":6,"last update":"Wed Aug 9 13:54:31 2023","interfaces":[{"interface":"back","average":{"1min":1.371,"5min":1.474,"15min":1.5289999999999999},"min":{"1min":0.83899999999999997,"5min":0.73599999999999999,"15min":0.73599999999999999},"max":{"1min":1.7110000000000001,"5min":2.4670000000000001,"15min":2.7250000000000001},"last":1.7749999999999999},{"interface":"front","average":{"1min":1.448,"5min":1.5269999999999999,"15min":1.5600000000000001},"min":{"1min":1.0169999999999999,"5min":0.73899999999999999,"15min":0.73899999999999999},"max":{"1min":1.881,"5min":2.3199999999999998,"15min":2.399},"last":1.8120000000000001}]},{"osd":7,"last update":"Wed Aug 9 13:54:31 2023","interfaces":[{"interface":"back","average":{"1min":1.419,"5min":1.544,"15min":1.6100000000000001},"min":{"1min":0.90400000000000003,"5min":0.72299999999999998,"15min":0.72299999999999998},"max":{"1min":1.8799999999999999,"5min":2.3900000000000001,"15min":2.3900000000000001},"last":1.5389999999999999},{"interface":"front","average":{"1min":1.3140000000000001,"5min":1.4510000000000001,"15min":1.532},"min":{"1min":0.86599999999999999,"5min":0.79200000000000004,"15min":0.76700000000000002},"max":{"1min":1.905,"5min":2.274,"15min":2.274},"last":1.591}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":18,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":18,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":18,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":616,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":18,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":160,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":18,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":70,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":18,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":112,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":18,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":18,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":2250,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":19,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":19,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":19,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":19,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":19,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":19,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":19,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":19,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-08-09T13:55:33.872 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-08-09T13:55:34.141 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:34.138+0000 7f560e3fc640 1 Processor -- start 2023-08-09T13:55:34.141 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:34.138+0000 7f560e3fc640 1 -- start start 2023-08-09T13:55:34.142 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:34.140+0000 7f560e3fc640 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7f56080c0e60 0x7f5608160e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:55:34.142 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:34.140+0000 7f560e3fc640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f56080c9620 0x7f56080c9a50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:55:34.143 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:34.140+0000 7f560e3fc640 1 -- --> v1:172.21.15.203:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f56081614b0 con 0x7f5608149e10 2023-08-09T13:55:34.143 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:34.140+0000 7f560e3fc640 1 -- --> v2:172.21.15.203:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5608164540 con 0x7f56080c0e60 2023-08-09T13:55:34.143 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:34.140+0000 7f560e3fc640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f560814a2a0 con 0x7f56080c9620 2023-08-09T13:55:34.143 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:34.141+0000 7f560d3fa640 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7f56080c0e60 0x7f5608160e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:55:34.144 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:34.141+0000 7f560dbfb640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f56080c9620 0x7f56080c9a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:55:34.144 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:34.141+0000 7f560dbfb640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f56080c9620 0x7f56080c9a50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.203:3301/0 says I am v2:172.21.15.43:44164/0 (socket says 172.21.15.43:44164) 2023-08-09T13:55:34.144 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:34.141+0000 7f560d3fa640 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7f56080c0e60 0x7f5608160e70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.203:3300/0 says I am v2:172.21.15.43:43684/0 (socket says 172.21.15.43:43684) 2023-08-09T13:55:34.144 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:34.141+0000 7f560dbfb640 1 -- 172.21.15.43:0/101519528 learned_addr learned my addr 172.21.15.43:0/101519528 (peer_addr_for_me v2:172.21.15.43:0/0) 2023-08-09T13:55:34.144 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:34.141+0000 7f560cbf9640 1 --1- 172.21.15.43:0/101519528 >> v1:172.21.15.203:6789/0 conn(0x7f5608149e10 0x7f56080c1240 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.203:6789/0 says I am v1:172.21.15.43:34286/0 (socket says 172.21.15.43:34286) 2023-08-09T13:55:34.145 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:34.142+0000 7f560d3fa640 1 -- 172.21.15.43:0/101519528 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f56080c9620 msgr2=0x7f56080c9a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-09T13:55:34.145 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:34.142+0000 7f560d3fa640 1 --2- 172.21.15.43:0/101519528 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f56080c9620 0x7f56080c9a50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:55:34.145 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:34.142+0000 7f560d3fa640 1 -- 172.21.15.43:0/101519528 >> v1:172.21.15.203:6789/0 conn(0x7f5608149e10 legacy=0x7f56080c1240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:55:34.145 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:34.142+0000 7f560d3fa640 1 -- 172.21.15.43:0/101519528 --> v2:172.21.15.203:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5608161790 con 0x7f56080c0e60 2023-08-09T13:55:34.145 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:34.143+0000 7f560d3fa640 1 --2- 172.21.15.43:0/101519528 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f56080c0e60 0x7f5608160e70 secure :-1 s=READY pgs=624 cs=0 l=1 rev1=1 crypto rx=0x7f56040097d0 tx=0x7f560402b8a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=833a3b0e3b0f8e32 server_cookie=0 in_seq=0 out_seq=0 2023-08-09T13:55:34.146 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:34.144+0000 7f55fffff640 1 -- 172.21.15.43:0/101519528 <== mon.1 v2:172.21.15.203:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5604031800 con 0x7f56080c0e60 2023-08-09T13:55:34.146 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:34.144+0000 7f55fffff640 1 -- 172.21.15.43:0/101519528 <== mon.1 v2:172.21.15.203:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5604031960 con 0x7f56080c0e60 2023-08-09T13:55:34.146 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:34.144+0000 7f55fffff640 1 -- 172.21.15.43:0/101519528 <== mon.1 v2:172.21.15.203:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5604031cf0 con 0x7f56080c0e60 2023-08-09T13:55:34.147 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:34.144+0000 7f560e3fc640 1 -- 172.21.15.43:0/101519528 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f56080c0e60 msgr2=0x7f5608160e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:55:34.147 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:34.144+0000 7f560e3fc640 1 --2- 172.21.15.43:0/101519528 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f56080c0e60 0x7f5608160e70 secure :-1 s=READY pgs=624 cs=0 l=1 rev1=1 crypto rx=0x7f56040097d0 tx=0x7f560402b8a0 comp rx=0 tx=0).stop 2023-08-09T13:55:34.147 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:34.145+0000 7f560e3fc640 1 -- 172.21.15.43:0/101519528 shutdown_connections 2023-08-09T13:55:34.147 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:34.145+0000 7f560e3fc640 1 --2- 172.21.15.43:0/101519528 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f56080c9620 0x7f56080c9a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:55:34.148 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:34.145+0000 7f560e3fc640 1 --2- 172.21.15.43:0/101519528 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f56080c0e60 0x7f5608160e70 unknown :-1 s=CLOSED pgs=624 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:55:34.148 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:34.145+0000 7f560e3fc640 1 -- 172.21.15.43:0/101519528 >> 172.21.15.43:0/101519528 conn(0x7f56080cde90 msgr2=0x7f56080ce2a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-09T13:55:34.148 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:34.145+0000 7f560e3fc640 1 -- 172.21.15.43:0/101519528 shutdown_connections 2023-08-09T13:55:34.148 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:34.145+0000 7f560e3fc640 1 -- 172.21.15.43:0/101519528 wait complete. 2023-08-09T13:55:34.149 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:34.147+0000 7f560e3fc640 1 Processor -- start 2023-08-09T13:55:34.150 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:34.148+0000 7f560e3fc640 1 -- start start 2023-08-09T13:55:34.151 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:34.149+0000 7f560e3fc640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f56080c0e60 0x7f5608149350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:55:34.151 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:34.149+0000 7f560e3fc640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f56080c9620 0x7f56080c1240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:55:34.152 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:34.149+0000 7f560e3fc640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f5608149e10 0x7f560806ff10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:55:34.152 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:34.149+0000 7f560e3fc640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f56080c4b40 con 0x7f56080c0e60 2023-08-09T13:55:34.152 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:34.149+0000 7f560e3fc640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f56080c5000 con 0x7f5608149e10 2023-08-09T13:55:34.152 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:34.149+0000 7f560e3fc640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5608149890 con 0x7f56080c9620 2023-08-09T13:55:34.153 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:34.149+0000 7f560d3fa640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f56080c0e60 0x7f5608149350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:55:34.153 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:34.149+0000 7f560d3fa640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f56080c0e60 0x7f5608149350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:52642/0 (socket says 172.21.15.43:52642) 2023-08-09T13:55:34.154 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:34.149+0000 7f560d3fa640 1 -- 172.21.15.43:0/1462527053 learned_addr learned my addr 172.21.15.43:0/1462527053 (peer_addr_for_me v2:172.21.15.43:0/0) 2023-08-09T13:55:34.154 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:34.150+0000 7f560dbfb640 1 --2- 172.21.15.43:0/1462527053 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f5608149e10 0x7f560806ff10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:55:34.154 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:34.150+0000 7f560cbf9640 1 --2- 172.21.15.43:0/1462527053 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f56080c9620 0x7f56080c1240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:55:34.154 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:34.150+0000 7f560d3fa640 1 -- 172.21.15.43:0/1462527053 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f56080c9620 msgr2=0x7f56080c1240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:55:34.154 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:34.150+0000 7f560d3fa640 1 --2- 172.21.15.43:0/1462527053 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f56080c9620 0x7f56080c1240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:55:34.154 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:34.150+0000 7f560d3fa640 1 -- 172.21.15.43:0/1462527053 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f5608149e10 msgr2=0x7f560806ff10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:55:34.154 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:34.150+0000 7f560d3fa640 1 --2- 172.21.15.43:0/1462527053 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f5608149e10 0x7f560806ff10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:55:34.154 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:34.150+0000 7f560d3fa640 1 -- 172.21.15.43:0/1462527053 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5604009400 con 0x7f56080c0e60 2023-08-09T13:55:34.155 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:34.150+0000 7f560cbf9640 1 --2- 172.21.15.43:0/1462527053 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f56080c9620 0x7f56080c1240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-08-09T13:55:34.155 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:34.150+0000 7f560d3fa640 1 --2- 172.21.15.43:0/1462527053 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f56080c0e60 0x7f5608149350 secure :-1 s=READY pgs=1702 cs=0 l=1 rev1=1 crypto rx=0x7f56040098f0 tx=0x7f56040046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-09T13:55:34.155 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:34.151+0000 7f55fdffb640 1 -- 172.21.15.43:0/1462527053 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f560403ac60 con 0x7f56080c0e60 2023-08-09T13:55:34.155 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:34.151+0000 7f55fdffb640 1 -- 172.21.15.43:0/1462527053 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f560403adc0 con 0x7f56080c0e60 2023-08-09T13:55:34.155 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:34.151+0000 7f560e3fc640 1 -- 172.21.15.43:0/1462527053 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f56080c5270 con 0x7f56080c0e60 2023-08-09T13:55:34.155 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:34.151+0000 7f55fdffb640 1 -- 172.21.15.43:0/1462527053 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5604044630 con 0x7f56080c0e60 2023-08-09T13:55:34.155 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:34.151+0000 7f560e3fc640 1 -- 172.21.15.43:0/1462527053 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f56080c5830 con 0x7f56080c0e60 2023-08-09T13:55:34.156 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:34.152+0000 7f560e3fc640 1 -- 172.21.15.43:0/1462527053 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f56080739f0 con 0x7f56080c0e60 2023-08-09T13:55:34.156 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:34.153+0000 7f55fdffb640 1 -- 172.21.15.43:0/1462527053 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 67) v1 ==== 88443+0+0 (secure 0 0 0) 0x7f560404d450 con 0x7f56080c0e60 2023-08-09T13:55:34.156 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:34.154+0000 7f55fdffb640 1 --2- 172.21.15.43:0/1462527053 >> [v2:172.21.15.203:6832/3490266242,v1:172.21.15.203:6833/3490266242] conn(0x7f55ec06a170 0x7f55ec06c630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:55:34.157 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:34.154+0000 7f55fdffb640 1 -- 172.21.15.43:0/1462527053 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(147..147 src has 1..147) v4 ==== 6426+0+0 (secure 0 0 0) 0x7f560402e070 con 0x7f56080c0e60 2023-08-09T13:55:34.157 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:34.155+0000 7f560cbf9640 1 --2- 172.21.15.43:0/1462527053 >> [v2:172.21.15.203:6832/3490266242,v1:172.21.15.203:6833/3490266242] conn(0x7f55ec06a170 0x7f55ec06c630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:55:34.158 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:34.156+0000 7f560cbf9640 1 --2- 172.21.15.43:0/1462527053 >> [v2:172.21.15.203:6832/3490266242,v1:172.21.15.203:6833/3490266242] conn(0x7f55ec06a170 0x7f55ec06c630 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f56080c4240 tx=0x7f55f8009040 comp rx=0 tx=0).ready entity=mgr.6659 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-09T13:55:34.161 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:34.158+0000 7f55fdffb640 1 -- 172.21.15.43:0/1462527053 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178066 (secure 0 0 0) 0x7f5604076070 con 0x7f56080c0e60 2023-08-09T13:55:34.350 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:34.348+0000 7f560e3fc640 1 -- 172.21.15.43:0/1462527053 --> [v2:172.21.15.203:6832/3490266242,v1:172.21.15.203:6833/3490266242] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f5608161d50 con 0x7f55ec06a170 2023-08-09T13:55:34.375 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:34.373+0000 7f55fdffb640 1 -- 172.21.15.43:0/1462527053 <== mgr.6659 v2:172.21.15.203:6832/3490266242 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+309661 (secure 0 0 0) 0x7f55f8001cc0 con 0x7f55ec06a170 2023-08-09T13:55:34.378 INFO:teuthology.orchestra.run.smithi043.stdout: 2023-08-09T13:55:34.380 INFO:teuthology.orchestra.run.smithi043.stderr:dumped all 2023-08-09T13:55:34.381 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:34.378+0000 7f560e3fc640 1 -- 172.21.15.43:0/1462527053 >> [v2:172.21.15.203:6832/3490266242,v1:172.21.15.203:6833/3490266242] conn(0x7f55ec06a170 msgr2=0x7f55ec06c630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:55:34.381 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:34.378+0000 7f560e3fc640 1 --2- 172.21.15.43:0/1462527053 >> [v2:172.21.15.203:6832/3490266242,v1:172.21.15.203:6833/3490266242] conn(0x7f55ec06a170 0x7f55ec06c630 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f56080c4240 tx=0x7f55f8009040 comp rx=0 tx=0).stop 2023-08-09T13:55:34.381 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:34.378+0000 7f560e3fc640 1 -- 172.21.15.43:0/1462527053 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f56080c0e60 msgr2=0x7f5608149350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:55:34.382 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:34.378+0000 7f560e3fc640 1 --2- 172.21.15.43:0/1462527053 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f56080c0e60 0x7f5608149350 secure :-1 s=READY pgs=1702 cs=0 l=1 rev1=1 crypto rx=0x7f56040098f0 tx=0x7f56040046c0 comp rx=0 tx=0).stop 2023-08-09T13:55:34.382 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:34.378+0000 7f560e3fc640 1 -- 172.21.15.43:0/1462527053 shutdown_connections 2023-08-09T13:55:34.382 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:34.378+0000 7f560e3fc640 1 --2- 172.21.15.43:0/1462527053 >> [v2:172.21.15.203:6832/3490266242,v1:172.21.15.203:6833/3490266242] conn(0x7f55ec06a170 0x7f55ec06c630 unknown :-1 s=CLOSED pgs=65 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:55:34.382 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:34.378+0000 7f560e3fc640 1 --2- 172.21.15.43:0/1462527053 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f5608149e10 0x7f560806ff10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:55:34.382 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:34.378+0000 7f560e3fc640 1 --2- 172.21.15.43:0/1462527053 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f56080c9620 0x7f56080c1240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:55:34.382 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:34.378+0000 7f560e3fc640 1 --2- 172.21.15.43:0/1462527053 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f56080c0e60 0x7f5608149350 unknown :-1 s=CLOSED pgs=1702 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:55:34.382 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:34.378+0000 7f560e3fc640 1 -- 172.21.15.43:0/1462527053 >> 172.21.15.43:0/1462527053 conn(0x7f56080cde90 msgr2=0x7f56080aee90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-09T13:55:34.383 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:34.378+0000 7f560e3fc640 1 -- 172.21.15.43:0/1462527053 shutdown_connections 2023-08-09T13:55:34.383 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:34.379+0000 7f560e3fc640 1 -- 172.21.15.43:0/1462527053 wait complete. 2023-08-09T13:55:34.401 INFO:teuthology.orchestra.run.smithi043.stdout:{"pg_ready":true,"pg_map":{"version":28,"stamp":"2023-08-09T13:55:32.575069+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":23,"num_object_clones":0,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":0,"num_read":1099,"num_read_kb":2242,"num_write":407,"num_write_kb":2430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":268,"ondisk_log_size":268,"up":210,"acting":210,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":79,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":333052,"kb_used_data":15232,"kb_used_omap":13,"kb_used_meta":317682,"kb_avail":754641668,"statfs":{"total":773094113280,"available":772753068032,"internally_reserved":0,"allocated":15597568,"data_stored":8906086,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":14248,"internal_metadata":325306456},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":34,"apply_latency_ms":34,"commit_latency_ns":34000000,"apply_latency_ns":34000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"5.286280"},"pg_stats":[{"pgid":"19.3f","version":"0'0","reported_seq":9,"reported_epoch":146,"state":"active+clean","last_fresh":"2023-08-09T13:55:29.389106+0000","last_change":"2023-08-09T13:55:29.386485+0000","last_active":"2023-08-09T13:55:29.389106+0000","last_peered":"2023-08-09T13:55:29.389106+0000","last_clean":"2023-08-09T13:55:29.389106+0000","last_became_active":"2023-08-09T13:55:29.386195+0000","last_became_peered":"2023-08-09T13:55:29.386195+0000","last_unstale":"2023-08-09T13:55:29.389106+0000","last_undegraded":"2023-08-09T13:55:29.389106+0000","last_fullsized":"2023-08-09T13:55:29.389106+0000","mapping_epoch":145,"log_start":"0'0","ondisk_log_start":"0'0","created":145,"last_epoch_clean":146,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:55:28.281790+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:55:28.281790+0000","last_clean_scrub_stamp":"2023-08-09T13:55:28.281790+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T18:35:28.179248+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"19.3e","version":"0'0","reported_seq":9,"reported_epoch":146,"state":"active+clean","last_fresh":"2023-08-09T13:55:29.391257+0000","last_change":"2023-08-09T13:55:29.390587+0000","last_active":"2023-08-09T13:55:29.391257+0000","last_peered":"2023-08-09T13:55:29.391257+0000","last_clean":"2023-08-09T13:55:29.391257+0000","last_became_active":"2023-08-09T13:55:29.390279+0000","last_became_peered":"2023-08-09T13:55:29.390279+0000","last_unstale":"2023-08-09T13:55:29.391257+0000","last_undegraded":"2023-08-09T13:55:29.391257+0000","last_fullsized":"2023-08-09T13:55:29.391257+0000","mapping_epoch":145,"log_start":"0'0","ondisk_log_start":"0'0","created":145,"last_epoch_clean":146,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:55:28.281790+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:55:28.281790+0000","last_clean_scrub_stamp":"2023-08-09T13:55:28.281790+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T19:28:17.384228+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"19.3d","version":"0'0","reported_seq":9,"reported_epoch":146,"state":"active+clean","last_fresh":"2023-08-09T13:55:29.387997+0000","last_change":"2023-08-09T13:55:29.378897+0000","last_active":"2023-08-09T13:55:29.387997+0000","last_peered":"2023-08-09T13:55:29.387997+0000","last_clean":"2023-08-09T13:55:29.387997+0000","last_became_active":"2023-08-09T13:55:29.378542+0000","last_became_peered":"2023-08-09T13:55:29.378542+0000","last_unstale":"2023-08-09T13:55:29.387997+0000","last_undegraded":"2023-08-09T13:55:29.387997+0000","last_fullsized":"2023-08-09T13:55:29.387997+0000","mapping_epoch":145,"log_start":"0'0","ondisk_log_start":"0'0","created":145,"last_epoch_clean":146,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:55:28.281790+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:55:28.281790+0000","last_clean_scrub_stamp":"2023-08-09T13:55:28.281790+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T19:35:21.339885+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"19.3c","version":"0'0","reported_seq":9,"reported_epoch":146,"state":"active+clean","last_fresh":"2023-08-09T13:55:29.374630+0000","last_change":"2023-08-09T13:55:29.371565+0000","last_active":"2023-08-09T13:55:29.374630+0000","last_peered":"2023-08-09T13:55:29.374630+0000","last_clean":"2023-08-09T13:55:29.374630+0000","last_became_active":"2023-08-09T13:55:29.371289+0000","last_became_peered":"2023-08-09T13:55:29.371289+0000","last_unstale":"2023-08-09T13:55:29.374630+0000","last_undegraded":"2023-08-09T13:55:29.374630+0000","last_fullsized":"2023-08-09T13:55:29.374630+0000","mapping_epoch":145,"log_start":"0'0","ondisk_log_start":"0'0","created":145,"last_epoch_clean":146,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:55:28.281790+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:55:28.281790+0000","last_clean_scrub_stamp":"2023-08-09T13:55:28.281790+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T20:03:28.279961+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"19.3b","version":"0'0","reported_seq":9,"reported_epoch":146,"state":"active+clean","last_fresh":"2023-08-09T13:55:29.374346+0000","last_change":"2023-08-09T13:55:29.367110+0000","last_active":"2023-08-09T13:55:29.374346+0000","last_peered":"2023-08-09T13:55:29.374346+0000","last_clean":"2023-08-09T13:55:29.374346+0000","last_became_active":"2023-08-09T13:55:29.366425+0000","last_became_peered":"2023-08-09T13:55:29.366425+0000","last_unstale":"2023-08-09T13:55:29.374346+0000","last_undegraded":"2023-08-09T13:55:29.374346+0000","last_fullsized":"2023-08-09T13:55:29.374346+0000","mapping_epoch":145,"log_start":"0'0","ondisk_log_start":"0'0","created":145,"last_epoch_clean":146,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:55:28.281790+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:55:28.281790+0000","last_clean_scrub_stamp":"2023-08-09T13:55:28.281790+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T20:18:12.293111+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"19.3a","version":"0'0","reported_seq":9,"reported_epoch":146,"state":"active+clean","last_fresh":"2023-08-09T13:55:29.387444+0000","last_change":"2023-08-09T13:55:29.383887+0000","last_active":"2023-08-09T13:55:29.387444+0000","last_peered":"2023-08-09T13:55:29.387444+0000","last_clean":"2023-08-09T13:55:29.387444+0000","last_became_active":"2023-08-09T13:55:29.383450+0000","last_became_peered":"2023-08-09T13:55:29.383450+0000","last_unstale":"2023-08-09T13:55:29.387444+0000","last_undegraded":"2023-08-09T13:55:29.387444+0000","last_fullsized":"2023-08-09T13:55:29.387444+0000","mapping_epoch":145,"log_start":"0'0","ondisk_log_start":"0'0","created":145,"last_epoch_clean":146,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:55:28.281790+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:55:28.281790+0000","last_clean_scrub_stamp":"2023-08-09T13:55:28.281790+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T16:00:03.485826+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"19.39","version":"0'0","reported_seq":9,"reported_epoch":146,"state":"active+clean","last_fresh":"2023-08-09T13:55:29.387384+0000","last_change":"2023-08-09T13:55:29.384060+0000","last_active":"2023-08-09T13:55:29.387384+0000","last_peered":"2023-08-09T13:55:29.387384+0000","last_clean":"2023-08-09T13:55:29.387384+0000","last_became_active":"2023-08-09T13:55:29.383747+0000","last_became_peered":"2023-08-09T13:55:29.383747+0000","last_unstale":"2023-08-09T13:55:29.387384+0000","last_undegraded":"2023-08-09T13:55:29.387384+0000","last_fullsized":"2023-08-09T13:55:29.387384+0000","mapping_epoch":145,"log_start":"0'0","ondisk_log_start":"0'0","created":145,"last_epoch_clean":146,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:55:28.281790+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:55:28.281790+0000","last_clean_scrub_stamp":"2023-08-09T13:55:28.281790+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T19:03:21.579607+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"19.38","version":"0'0","reported_seq":9,"reported_epoch":146,"state":"active+clean","last_fresh":"2023-08-09T13:55:29.387159+0000","last_change":"2023-08-09T13:55:29.380287+0000","last_active":"2023-08-09T13:55:29.387159+0000","last_peered":"2023-08-09T13:55:29.387159+0000","last_clean":"2023-08-09T13:55:29.387159+0000","last_became_active":"2023-08-09T13:55:29.379307+0000","last_became_peered":"2023-08-09T13:55:29.379307+0000","last_unstale":"2023-08-09T13:55:29.387159+0000","last_undegraded":"2023-08-09T13:55:29.387159+0000","last_fullsized":"2023-08-09T13:55:29.387159+0000","mapping_epoch":145,"log_start":"0'0","ondisk_log_start":"0'0","created":145,"last_epoch_clean":146,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:55:28.281790+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:55:28.281790+0000","last_clean_scrub_stamp":"2023-08-09T13:55:28.281790+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T23:51:16.157673+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"19.37","version":"0'0","reported_seq":9,"reported_epoch":146,"state":"active+clean","last_fresh":"2023-08-09T13:55:29.391218+0000","last_change":"2023-08-09T13:55:29.388275+0000","last_active":"2023-08-09T13:55:29.391218+0000","last_peered":"2023-08-09T13:55:29.391218+0000","last_clean":"2023-08-09T13:55:29.391218+0000","last_became_active":"2023-08-09T13:55:29.387985+0000","last_became_peered":"2023-08-09T13:55:29.387985+0000","last_unstale":"2023-08-09T13:55:29.391218+0000","last_undegraded":"2023-08-09T13:55:29.391218+0000","last_fullsized":"2023-08-09T13:55:29.391218+0000","mapping_epoch":145,"log_start":"0'0","ondisk_log_start":"0'0","created":145,"last_epoch_clean":146,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:55:28.281790+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:55:28.281790+0000","last_clean_scrub_stamp":"2023-08-09T13:55:28.281790+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-11T01:22:01.234257+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"19.36","version":"0'0","reported_seq":9,"reported_epoch":146,"state":"active+clean","last_fresh":"2023-08-09T13:55:29.391242+0000","last_change":"2023-08-09T13:55:29.390500+0000","last_active":"2023-08-09T13:55:29.391242+0000","last_peered":"2023-08-09T13:55:29.391242+0000","last_clean":"2023-08-09T13:55:29.391242+0000","last_became_active":"2023-08-09T13:55:29.390188+0000","last_became_peered":"2023-08-09T13:55:29.390188+0000","last_unstale":"2023-08-09T13:55:29.391242+0000","last_undegraded":"2023-08-09T13:55:29.391242+0000","last_fullsized":"2023-08-09T13:55:29.391242+0000","mapping_epoch":145,"log_start":"0'0","ondisk_log_start":"0'0","created":145,"last_epoch_clean":146,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:55:28.281790+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:55:28.281790+0000","last_clean_scrub_stamp":"2023-08-09T13:55:28.281790+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T22:01:14.561371+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"19.35","version":"0'0","reported_seq":9,"reported_epoch":146,"state":"active+clean","last_fresh":"2023-08-09T13:55:29.365979+0000","last_change":"2023-08-09T13:55:29.362462+0000","last_active":"2023-08-09T13:55:29.365979+0000","last_peered":"2023-08-09T13:55:29.365979+0000","last_clean":"2023-08-09T13:55:29.365979+0000","last_became_active":"2023-08-09T13:55:29.362150+0000","last_became_peered":"2023-08-09T13:55:29.362150+0000","last_unstale":"2023-08-09T13:55:29.365979+0000","last_undegraded":"2023-08-09T13:55:29.365979+0000","last_fullsized":"2023-08-09T13:55:29.365979+0000","mapping_epoch":145,"log_start":"0'0","ondisk_log_start":"0'0","created":145,"last_epoch_clean":146,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:55:28.281790+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:55:28.281790+0000","last_clean_scrub_stamp":"2023-08-09T13:55:28.281790+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-11T00:49:19.550086+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"19.34","version":"0'0","reported_seq":9,"reported_epoch":146,"state":"active+clean","last_fresh":"2023-08-09T13:55:29.382541+0000","last_change":"2023-08-09T13:55:29.377575+0000","last_active":"2023-08-09T13:55:29.382541+0000","last_peered":"2023-08-09T13:55:29.382541+0000","last_clean":"2023-08-09T13:55:29.382541+0000","last_became_active":"2023-08-09T13:55:29.377268+0000","last_became_peered":"2023-08-09T13:55:29.377268+0000","last_unstale":"2023-08-09T13:55:29.382541+0000","last_undegraded":"2023-08-09T13:55:29.382541+0000","last_fullsized":"2023-08-09T13:55:29.382541+0000","mapping_epoch":145,"log_start":"0'0","ondisk_log_start":"0'0","created":145,"last_epoch_clean":146,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:55:28.281790+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:55:28.281790+0000","last_clean_scrub_stamp":"2023-08-09T13:55:28.281790+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T16:24:58.970272+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"19.33","version":"0'0","reported_seq":9,"reported_epoch":146,"state":"active+clean","last_fresh":"2023-08-09T13:55:29.390050+0000","last_change":"2023-08-09T13:55:29.387036+0000","last_active":"2023-08-09T13:55:29.390050+0000","last_peered":"2023-08-09T13:55:29.390050+0000","last_clean":"2023-08-09T13:55:29.390050+0000","last_became_active":"2023-08-09T13:55:29.386717+0000","last_became_peered":"2023-08-09T13:55:29.386717+0000","last_unstale":"2023-08-09T13:55:29.390050+0000","last_undegraded":"2023-08-09T13:55:29.390050+0000","last_fullsized":"2023-08-09T13:55:29.390050+0000","mapping_epoch":145,"log_start":"0'0","ondisk_log_start":"0'0","created":145,"last_epoch_clean":146,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:55:28.281790+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:55:28.281790+0000","last_clean_scrub_stamp":"2023-08-09T13:55:28.281790+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T14:35:39.842197+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"19.32","version":"0'0","reported_seq":9,"reported_epoch":146,"state":"active+clean","last_fresh":"2023-08-09T13:55:29.373597+0000","last_change":"2023-08-09T13:55:29.370486+0000","last_active":"2023-08-09T13:55:29.373597+0000","last_peered":"2023-08-09T13:55:29.373597+0000","last_clean":"2023-08-09T13:55:29.373597+0000","last_became_active":"2023-08-09T13:55:29.370203+0000","last_became_peered":"2023-08-09T13:55:29.370203+0000","last_unstale":"2023-08-09T13:55:29.373597+0000","last_undegraded":"2023-08-09T13:55:29.373597+0000","last_fullsized":"2023-08-09T13:55:29.373597+0000","mapping_epoch":145,"log_start":"0'0","ondisk_log_start":"0'0","created":145,"last_epoch_clean":146,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:55:28.281790+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:55:28.281790+0000","last_clean_scrub_stamp":"2023-08-09T13:55:28.281790+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T14:04:05.932473+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"19.31","version":"0'0","reported_seq":9,"reported_epoch":146,"state":"active+clean","last_fresh":"2023-08-09T13:55:29.384296+0000","last_change":"2023-08-09T13:55:29.380328+0000","last_active":"2023-08-09T13:55:29.384296+0000","last_peered":"2023-08-09T13:55:29.384296+0000","last_clean":"2023-08-09T13:55:29.384296+0000","last_became_active":"2023-08-09T13:55:29.379188+0000","last_became_peered":"2023-08-09T13:55:29.379188+0000","last_unstale":"2023-08-09T13:55:29.384296+0000","last_undegraded":"2023-08-09T13:55:29.384296+0000","last_fullsized":"2023-08-09T13:55:29.384296+0000","mapping_epoch":145,"log_start":"0'0","ondisk_log_start":"0'0","created":145,"last_epoch_clean":146,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:55:28.281790+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:55:28.281790+0000","last_clean_scrub_stamp":"2023-08-09T13:55:28.281790+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T17:13:59.904533+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"19.30","version":"0'0","reported_seq":9,"reported_epoch":146,"state":"active+clean","last_fresh":"2023-08-09T13:55:29.361848+0000","last_change":"2023-08-09T13:55:29.358473+0000","last_active":"2023-08-09T13:55:29.361848+0000","last_peered":"2023-08-09T13:55:29.361848+0000","last_clean":"2023-08-09T13:55:29.361848+0000","last_became_active":"2023-08-09T13:55:29.358161+0000","last_became_peered":"2023-08-09T13:55:29.358161+0000","last_unstale":"2023-08-09T13:55:29.361848+0000","last_undegraded":"2023-08-09T13:55:29.361848+0000","last_fullsized":"2023-08-09T13:55:29.361848+0000","mapping_epoch":145,"log_start":"0'0","ondisk_log_start":"0'0","created":145,"last_epoch_clean":146,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:55:28.281790+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:55:28.281790+0000","last_clean_scrub_stamp":"2023-08-09T13:55:28.281790+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T21:37:44.655459+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"19.2f","version":"0'0","reported_seq":9,"reported_epoch":146,"state":"active+clean","last_fresh":"2023-08-09T13:55:29.385874+0000","last_change":"2023-08-09T13:55:29.380515+0000","last_active":"2023-08-09T13:55:29.385874+0000","last_peered":"2023-08-09T13:55:29.385874+0000","last_clean":"2023-08-09T13:55:29.385874+0000","last_became_active":"2023-08-09T13:55:29.380238+0000","last_became_peered":"2023-08-09T13:55:29.380238+0000","last_unstale":"2023-08-09T13:55:29.385874+0000","last_undegraded":"2023-08-09T13:55:29.385874+0000","last_fullsized":"2023-08-09T13:55:29.385874+0000","mapping_epoch":145,"log_start":"0'0","ondisk_log_start":"0'0","created":145,"last_epoch_clean":146,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:55:28.281790+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:55:28.281790+0000","last_clean_scrub_stamp":"2023-08-09T13:55:28.281790+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T17:10:44.094430+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"19.2e","version":"0'0","reported_seq":9,"reported_epoch":146,"state":"active+clean","last_fresh":"2023-08-09T13:55:29.389740+0000","last_change":"2023-08-09T13:55:29.388791+0000","last_active":"2023-08-09T13:55:29.389740+0000","last_peered":"2023-08-09T13:55:29.389740+0000","last_clean":"2023-08-09T13:55:29.389740+0000","last_became_active":"2023-08-09T13:55:29.388541+0000","last_became_peered":"2023-08-09T13:55:29.388541+0000","last_unstale":"2023-08-09T13:55:29.389740+0000","last_undegraded":"2023-08-09T13:55:29.389740+0000","last_fullsized":"2023-08-09T13:55:29.389740+0000","mapping_epoch":145,"log_start":"0'0","ondisk_log_start":"0'0","created":145,"last_epoch_clean":146,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:55:28.281790+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:55:28.281790+0000","last_clean_scrub_stamp":"2023-08-09T13:55:28.281790+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T15:57:57.305300+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"19.2d","version":"0'0","reported_seq":9,"reported_epoch":146,"state":"active+clean","last_fresh":"2023-08-09T13:55:29.388225+0000","last_change":"2023-08-09T13:55:29.385790+0000","last_active":"2023-08-09T13:55:29.388225+0000","last_peered":"2023-08-09T13:55:29.388225+0000","last_clean":"2023-08-09T13:55:29.388225+0000","last_became_active":"2023-08-09T13:55:29.385427+0000","last_became_peered":"2023-08-09T13:55:29.385427+0000","last_unstale":"2023-08-09T13:55:29.388225+0000","last_undegraded":"2023-08-09T13:55:29.388225+0000","last_fullsized":"2023-08-09T13:55:29.388225+0000","mapping_epoch":145,"log_start":"0'0","ondisk_log_start":"0'0","created":145,"last_epoch_clean":146,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:55:28.281790+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:55:28.281790+0000","last_clean_scrub_stamp":"2023-08-09T13:55:28.281790+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T23:24:53.963659+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"19.2c","version":"0'0","reported_seq":9,"reported_epoch":146,"state":"active+clean","last_fresh":"2023-08-09T13:55:29.372590+0000","last_change":"2023-08-09T13:55:29.369236+0000","last_active":"2023-08-09T13:55:29.372590+0000","last_peered":"2023-08-09T13:55:29.372590+0000","last_clean":"2023-08-09T13:55:29.372590+0000","last_became_active":"2023-08-09T13:55:29.368288+0000","last_became_peered":"2023-08-09T13:55:29.368288+0000","last_unstale":"2023-08-09T13:55:29.372590+0000","last_undegraded":"2023-08-09T13:55:29.372590+0000","last_fullsized":"2023-08-09T13:55:29.372590+0000","mapping_epoch":145,"log_start":"0'0","ondisk_log_start":"0'0","created":145,"last_epoch_clean":146,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:55:28.281790+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:55:28.281790+0000","last_clean_scrub_stamp":"2023-08-09T13:55:28.281790+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T17:12:58.100966+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"19.2b","version":"0'0","reported_seq":9,"reported_epoch":146,"state":"active+clean","last_fresh":"2023-08-09T13:55:29.374364+0000","last_change":"2023-08-09T13:55:29.370194+0000","last_active":"2023-08-09T13:55:29.374364+0000","last_peered":"2023-08-09T13:55:29.374364+0000","last_clean":"2023-08-09T13:55:29.374364+0000","last_became_active":"2023-08-09T13:55:29.366801+0000","last_became_peered":"2023-08-09T13:55:29.366801+0000","last_unstale":"2023-08-09T13:55:29.374364+0000","last_undegraded":"2023-08-09T13:55:29.374364+0000","last_fullsized":"2023-08-09T13:55:29.374364+0000","mapping_epoch":145,"log_start":"0'0","ondisk_log_start":"0'0","created":145,"last_epoch_clean":146,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:55:28.281790+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:55:28.281790+0000","last_clean_scrub_stamp":"2023-08-09T13:55:28.281790+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-11T01:25:32.168578+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"19.2a","version":"0'0","reported_seq":9,"reported_epoch":146,"state":"active+clean","last_fresh":"2023-08-09T13:55:29.372802+0000","last_change":"2023-08-09T13:55:29.367927+0000","last_active":"2023-08-09T13:55:29.372802+0000","last_peered":"2023-08-09T13:55:29.372802+0000","last_clean":"2023-08-09T13:55:29.372802+0000","last_became_active":"2023-08-09T13:55:29.367661+0000","last_became_peered":"2023-08-09T13:55:29.367661+0000","last_unstale":"2023-08-09T13:55:29.372802+0000","last_undegraded":"2023-08-09T13:55:29.372802+0000","last_fullsized":"2023-08-09T13:55:29.372802+0000","mapping_epoch":145,"log_start":"0'0","ondisk_log_start":"0'0","created":145,"last_epoch_clean":146,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:55:28.281790+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:55:28.281790+0000","last_clean_scrub_stamp":"2023-08-09T13:55:28.281790+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T21:07:25.695017+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"19.29","version":"0'0","reported_seq":9,"reported_epoch":146,"state":"active+clean","last_fresh":"2023-08-09T13:55:29.391562+0000","last_change":"2023-08-09T13:55:29.389415+0000","last_active":"2023-08-09T13:55:29.391562+0000","last_peered":"2023-08-09T13:55:29.391562+0000","last_clean":"2023-08-09T13:55:29.391562+0000","last_became_active":"2023-08-09T13:55:29.388766+0000","last_became_peered":"2023-08-09T13:55:29.388766+0000","last_unstale":"2023-08-09T13:55:29.391562+0000","last_undegraded":"2023-08-09T13:55:29.391562+0000","last_fullsized":"2023-08-09T13:55:29.391562+0000","mapping_epoch":145,"log_start":"0'0","ondisk_log_start":"0'0","created":145,"last_epoch_clean":146,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:55:28.281790+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:55:28.281790+0000","last_clean_scrub_stamp":"2023-08-09T13:55:28.281790+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-11T00:12:19.333562+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"19.28","version":"0'0","reported_seq":9,"reported_epoch":146,"state":"active+clean","last_fresh":"2023-08-09T13:55:29.364335+0000","last_change":"2023-08-09T13:55:29.359649+0000","last_active":"2023-08-09T13:55:29.364335+0000","last_peered":"2023-08-09T13:55:29.364335+0000","last_clean":"2023-08-09T13:55:29.364335+0000","last_became_active":"2023-08-09T13:55:29.359319+0000","last_became_peered":"2023-08-09T13:55:29.359319+0000","last_unstale":"2023-08-09T13:55:29.364335+0000","last_undegraded":"2023-08-09T13:55:29.364335+0000","last_fullsized":"2023-08-09T13:55:29.364335+0000","mapping_epoch":145,"log_start":"0'0","ondisk_log_start":"0'0","created":145,"last_epoch_clean":146,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:55:28.281790+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:55:28.281790+0000","last_clean_scrub_stamp":"2023-08-09T13:55:28.281790+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T18:18:16.953218+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"19.27","version":"0'0","reported_seq":9,"reported_epoch":146,"state":"active+clean","last_fresh":"2023-08-09T13:55:29.380340+0000","last_change":"2023-08-09T13:55:29.379554+0000","last_active":"2023-08-09T13:55:29.380340+0000","last_peered":"2023-08-09T13:55:29.380340+0000","last_clean":"2023-08-09T13:55:29.380340+0000","last_became_active":"2023-08-09T13:55:29.379266+0000","last_became_peered":"2023-08-09T13:55:29.379266+0000","last_unstale":"2023-08-09T13:55:29.380340+0000","last_undegraded":"2023-08-09T13:55:29.380340+0000","last_fullsized":"2023-08-09T13:55:29.380340+0000","mapping_epoch":145,"log_start":"0'0","ondisk_log_start":"0'0","created":145,"last_epoch_clean":146,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:55:28.281790+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:55:28.281790+0000","last_clean_scrub_stamp":"2023-08-09T13:55:28.281790+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-11T01:16:10.186553+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"19.26","version":"0'0","reported_seq":9,"reported_epoch":146,"state":"active+clean","last_fresh":"2023-08-09T13:55:29.361872+0000","last_change":"2023-08-09T13:55:29.358663+0000","last_active":"2023-08-09T13:55:29.361872+0000","last_peered":"2023-08-09T13:55:29.361872+0000","last_clean":"2023-08-09T13:55:29.361872+0000","last_became_active":"2023-08-09T13:55:29.358306+0000","last_became_peered":"2023-08-09T13:55:29.358306+0000","last_unstale":"2023-08-09T13:55:29.361872+0000","last_undegraded":"2023-08-09T13:55:29.361872+0000","last_fullsized":"2023-08-09T13:55:29.361872+0000","mapping_epoch":145,"log_start":"0'0","ondisk_log_start":"0'0","created":145,"last_epoch_clean":146,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:55:28.281790+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:55:28.281790+0000","last_clean_scrub_stamp":"2023-08-09T13:55:28.281790+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T21:08:27.586494+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"19.25","version":"0'0","reported_seq":9,"reported_epoch":146,"state":"active+clean","last_fresh":"2023-08-09T13:55:29.390087+0000","last_change":"2023-08-09T13:55:29.387223+0000","last_active":"2023-08-09T13:55:29.390087+0000","last_peered":"2023-08-09T13:55:29.390087+0000","last_clean":"2023-08-09T13:55:29.390087+0000","last_became_active":"2023-08-09T13:55:29.386948+0000","last_became_peered":"2023-08-09T13:55:29.386948+0000","last_unstale":"2023-08-09T13:55:29.390087+0000","last_undegraded":"2023-08-09T13:55:29.390087+0000","last_fullsized":"2023-08-09T13:55:29.390087+0000","mapping_epoch":145,"log_start":"0'0","ondisk_log_start":"0'0","created":145,"last_epoch_clean":146,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:55:28.281790+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:55:28.281790+0000","last_clean_scrub_stamp":"2023-08-09T13:55:28.281790+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-11T01:52:05.886750+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"19.24","version":"0'0","reported_seq":9,"reported_epoch":146,"state":"active+clean","last_fresh":"2023-08-09T13:55:29.373086+0000","last_change":"2023-08-09T13:55:29.367568+0000","last_active":"2023-08-09T13:55:29.373086+0000","last_peered":"2023-08-09T13:55:29.373086+0000","last_clean":"2023-08-09T13:55:29.373086+0000","last_became_active":"2023-08-09T13:55:29.366511+0000","last_became_peered":"2023-08-09T13:55:29.366511+0000","last_unstale":"2023-08-09T13:55:29.373086+0000","last_undegraded":"2023-08-09T13:55:29.373086+0000","last_fullsized":"2023-08-09T13:55:29.373086+0000","mapping_epoch":145,"log_start":"0'0","ondisk_log_start":"0'0","created":145,"last_epoch_clean":146,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:55:28.281790+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:55:28.281790+0000","last_clean_scrub_stamp":"2023-08-09T13:55:28.281790+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T19:17:25.795688+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"19.23","version":"0'0","reported_seq":9,"reported_epoch":146,"state":"active+clean","last_fresh":"2023-08-09T13:55:29.384436+0000","last_change":"2023-08-09T13:55:29.382018+0000","last_active":"2023-08-09T13:55:29.384436+0000","last_peered":"2023-08-09T13:55:29.384436+0000","last_clean":"2023-08-09T13:55:29.384436+0000","last_became_active":"2023-08-09T13:55:29.381757+0000","last_became_peered":"2023-08-09T13:55:29.381757+0000","last_unstale":"2023-08-09T13:55:29.384436+0000","last_undegraded":"2023-08-09T13:55:29.384436+0000","last_fullsized":"2023-08-09T13:55:29.384436+0000","mapping_epoch":145,"log_start":"0'0","ondisk_log_start":"0'0","created":145,"last_epoch_clean":146,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:55:28.281790+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:55:28.281790+0000","last_clean_scrub_stamp":"2023-08-09T13:55:28.281790+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T19:34:27.895516+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"19.22","version":"0'0","reported_seq":9,"reported_epoch":146,"state":"active+clean","last_fresh":"2023-08-09T13:55:29.382771+0000","last_change":"2023-08-09T13:55:29.377418+0000","last_active":"2023-08-09T13:55:29.382771+0000","last_peered":"2023-08-09T13:55:29.382771+0000","last_clean":"2023-08-09T13:55:29.382771+0000","last_became_active":"2023-08-09T13:55:29.377096+0000","last_became_peered":"2023-08-09T13:55:29.377096+0000","last_unstale":"2023-08-09T13:55:29.382771+0000","last_undegraded":"2023-08-09T13:55:29.382771+0000","last_fullsized":"2023-08-09T13:55:29.382771+0000","mapping_epoch":145,"log_start":"0'0","ondisk_log_start":"0'0","created":145,"last_epoch_clean":146,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:55:28.281790+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:55:28.281790+0000","last_clean_scrub_stamp":"2023-08-09T13:55:28.281790+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T19:37:22.045044+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"19.21","version":"0'0","reported_seq":9,"reported_epoch":146,"state":"active+clean","last_fresh":"2023-08-09T13:55:29.381074+0000","last_change":"2023-08-09T13:55:29.379944+0000","last_active":"2023-08-09T13:55:29.381074+0000","last_peered":"2023-08-09T13:55:29.381074+0000","last_clean":"2023-08-09T13:55:29.381074+0000","last_became_active":"2023-08-09T13:55:29.379602+0000","last_became_peered":"2023-08-09T13:55:29.379602+0000","last_unstale":"2023-08-09T13:55:29.381074+0000","last_undegraded":"2023-08-09T13:55:29.381074+0000","last_fullsized":"2023-08-09T13:55:29.381074+0000","mapping_epoch":145,"log_start":"0'0","ondisk_log_start":"0'0","created":145,"last_epoch_clean":146,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:55:28.281790+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:55:28.281790+0000","last_clean_scrub_stamp":"2023-08-09T13:55:28.281790+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T21:02:16.712303+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"19.20","version":"0'0","reported_seq":9,"reported_epoch":146,"state":"active+clean","last_fresh":"2023-08-09T13:55:29.389771+0000","last_change":"2023-08-09T13:55:29.384552+0000","last_active":"2023-08-09T13:55:29.389771+0000","last_peered":"2023-08-09T13:55:29.389771+0000","last_clean":"2023-08-09T13:55:29.389771+0000","last_became_active":"2023-08-09T13:55:29.383361+0000","last_became_peered":"2023-08-09T13:55:29.383361+0000","last_unstale":"2023-08-09T13:55:29.389771+0000","last_undegraded":"2023-08-09T13:55:29.389771+0000","last_fullsized":"2023-08-09T13:55:29.389771+0000","mapping_epoch":145,"log_start":"0'0","ondisk_log_start":"0'0","created":145,"last_epoch_clean":146,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:55:28.281790+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:55:28.281790+0000","last_clean_scrub_stamp":"2023-08-09T13:55:28.281790+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-11T00:40:08.873250+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"19.1e","version":"0'0","reported_seq":9,"reported_epoch":146,"state":"active+clean","last_fresh":"2023-08-09T13:55:29.373187+0000","last_change":"2023-08-09T13:55:29.370200+0000","last_active":"2023-08-09T13:55:29.373187+0000","last_peered":"2023-08-09T13:55:29.373187+0000","last_clean":"2023-08-09T13:55:29.373187+0000","last_became_active":"2023-08-09T13:55:29.369913+0000","last_became_peered":"2023-08-09T13:55:29.369913+0000","last_unstale":"2023-08-09T13:55:29.373187+0000","last_undegraded":"2023-08-09T13:55:29.373187+0000","last_fullsized":"2023-08-09T13:55:29.373187+0000","mapping_epoch":145,"log_start":"0'0","ondisk_log_start":"0'0","created":145,"last_epoch_clean":146,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:55:28.281790+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:55:28.281790+0000","last_clean_scrub_stamp":"2023-08-09T13:55:28.281790+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T20:10:16.052483+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"18.1f","version":"146'4","reported_seq":15,"reported_epoch":146,"state":"active+clean","last_fresh":"2023-08-09T13:55:29.386444+0000","last_change":"2023-08-09T13:55:28.330435+0000","last_active":"2023-08-09T13:55:29.386444+0000","last_peered":"2023-08-09T13:55:29.386444+0000","last_clean":"2023-08-09T13:55:29.386444+0000","last_became_active":"2023-08-09T13:55:28.329974+0000","last_became_peered":"2023-08-09T13:55:28.329974+0000","last_unstale":"2023-08-09T13:55:29.386444+0000","last_undegraded":"2023-08-09T13:55:29.386444+0000","last_fullsized":"2023-08-09T13:55:29.386444+0000","mapping_epoch":144,"log_start":"0'0","ondisk_log_start":"0'0","created":144,"last_epoch_clean":145,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:55:27.277343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:55:27.277343+0000","last_clean_scrub_stamp":"2023-08-09T13:55:27.277343+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T23:38:51.687655+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"19.1f","version":"0'0","reported_seq":9,"reported_epoch":146,"state":"active+clean","last_fresh":"2023-08-09T13:55:29.382345+0000","last_change":"2023-08-09T13:55:29.376050+0000","last_active":"2023-08-09T13:55:29.382345+0000","last_peered":"2023-08-09T13:55:29.382345+0000","last_clean":"2023-08-09T13:55:29.382345+0000","last_became_active":"2023-08-09T13:55:29.375762+0000","last_became_peered":"2023-08-09T13:55:29.375762+0000","last_unstale":"2023-08-09T13:55:29.382345+0000","last_undegraded":"2023-08-09T13:55:29.382345+0000","last_fullsized":"2023-08-09T13:55:29.382345+0000","mapping_epoch":145,"log_start":"0'0","ondisk_log_start":"0'0","created":145,"last_epoch_clean":146,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:55:28.281790+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:55:28.281790+0000","last_clean_scrub_stamp":"2023-08-09T13:55:28.281790+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T17:52:18.127836+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"18.1e","version":"146'1","reported_seq":12,"reported_epoch":146,"state":"active+clean","last_fresh":"2023-08-09T13:55:29.386221+0000","last_change":"2023-08-09T13:55:28.314187+0000","last_active":"2023-08-09T13:55:29.386221+0000","last_peered":"2023-08-09T13:55:29.386221+0000","last_clean":"2023-08-09T13:55:29.386221+0000","last_became_active":"2023-08-09T13:55:28.313832+0000","last_became_peered":"2023-08-09T13:55:28.313832+0000","last_unstale":"2023-08-09T13:55:29.386221+0000","last_undegraded":"2023-08-09T13:55:29.386221+0000","last_fullsized":"2023-08-09T13:55:29.386221+0000","mapping_epoch":144,"log_start":"0'0","ondisk_log_start":"0'0","created":144,"last_epoch_clean":145,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:55:27.277343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:55:27.277343+0000","last_clean_scrub_stamp":"2023-08-09T13:55:27.277343+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T22:49:19.884883+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"19.1c","version":"0'0","reported_seq":9,"reported_epoch":146,"state":"active+clean","last_fresh":"2023-08-09T13:55:29.389833+0000","last_change":"2023-08-09T13:55:29.386097+0000","last_active":"2023-08-09T13:55:29.389833+0000","last_peered":"2023-08-09T13:55:29.389833+0000","last_clean":"2023-08-09T13:55:29.389833+0000","last_became_active":"2023-08-09T13:55:29.385801+0000","last_became_peered":"2023-08-09T13:55:29.385801+0000","last_unstale":"2023-08-09T13:55:29.389833+0000","last_undegraded":"2023-08-09T13:55:29.389833+0000","last_fullsized":"2023-08-09T13:55:29.389833+0000","mapping_epoch":145,"log_start":"0'0","ondisk_log_start":"0'0","created":145,"last_epoch_clean":146,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:55:28.281790+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:55:28.281790+0000","last_clean_scrub_stamp":"2023-08-09T13:55:28.281790+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T14:50:08.755167+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"18.1d","version":"146'2","reported_seq":13,"reported_epoch":146,"state":"active+clean","last_fresh":"2023-08-09T13:55:29.365637+0000","last_change":"2023-08-09T13:55:28.323885+0000","last_active":"2023-08-09T13:55:29.365637+0000","last_peered":"2023-08-09T13:55:29.365637+0000","last_clean":"2023-08-09T13:55:29.365637+0000","last_became_active":"2023-08-09T13:55:28.323620+0000","last_became_peered":"2023-08-09T13:55:28.323620+0000","last_unstale":"2023-08-09T13:55:29.365637+0000","last_undegraded":"2023-08-09T13:55:29.365637+0000","last_fullsized":"2023-08-09T13:55:29.365637+0000","mapping_epoch":144,"log_start":"0'0","ondisk_log_start":"0'0","created":144,"last_epoch_clean":145,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:55:27.277343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:55:27.277343+0000","last_clean_scrub_stamp":"2023-08-09T13:55:27.277343+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T23:58:37.536745+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"19.1d","version":"0'0","reported_seq":9,"reported_epoch":146,"state":"active+clean","last_fresh":"2023-08-09T13:55:29.374137+0000","last_change":"2023-08-09T13:55:29.369649+0000","last_active":"2023-08-09T13:55:29.374137+0000","last_peered":"2023-08-09T13:55:29.374137+0000","last_clean":"2023-08-09T13:55:29.374137+0000","last_became_active":"2023-08-09T13:55:29.367832+0000","last_became_peered":"2023-08-09T13:55:29.367832+0000","last_unstale":"2023-08-09T13:55:29.374137+0000","last_undegraded":"2023-08-09T13:55:29.374137+0000","last_fullsized":"2023-08-09T13:55:29.374137+0000","mapping_epoch":145,"log_start":"0'0","ondisk_log_start":"0'0","created":145,"last_epoch_clean":146,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:55:28.281790+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:55:28.281790+0000","last_clean_scrub_stamp":"2023-08-09T13:55:28.281790+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T22:16:04.137516+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"18.1c","version":"146'2","reported_seq":13,"reported_epoch":146,"state":"active+clean","last_fresh":"2023-08-09T13:55:29.373744+0000","last_change":"2023-08-09T13:55:28.316189+0000","last_active":"2023-08-09T13:55:29.373744+0000","last_peered":"2023-08-09T13:55:29.373744+0000","last_clean":"2023-08-09T13:55:29.373744+0000","last_became_active":"2023-08-09T13:55:28.315766+0000","last_became_peered":"2023-08-09T13:55:28.315766+0000","last_unstale":"2023-08-09T13:55:29.373744+0000","last_undegraded":"2023-08-09T13:55:29.373744+0000","last_fullsized":"2023-08-09T13:55:29.373744+0000","mapping_epoch":144,"log_start":"0'0","ondisk_log_start":"0'0","created":144,"last_epoch_clean":145,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:55:27.277343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:55:27.277343+0000","last_clean_scrub_stamp":"2023-08-09T13:55:27.277343+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-11T01:38:08.974687+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"19.1a","version":"0'0","reported_seq":9,"reported_epoch":146,"state":"active+clean","last_fresh":"2023-08-09T13:55:29.389247+0000","last_change":"2023-08-09T13:55:29.387307+0000","last_active":"2023-08-09T13:55:29.389247+0000","last_peered":"2023-08-09T13:55:29.389247+0000","last_clean":"2023-08-09T13:55:29.389247+0000","last_became_active":"2023-08-09T13:55:29.385513+0000","last_became_peered":"2023-08-09T13:55:29.385513+0000","last_unstale":"2023-08-09T13:55:29.389247+0000","last_undegraded":"2023-08-09T13:55:29.389247+0000","last_fullsized":"2023-08-09T13:55:29.389247+0000","mapping_epoch":145,"log_start":"0'0","ondisk_log_start":"0'0","created":145,"last_epoch_clean":146,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:55:28.281790+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:55:28.281790+0000","last_clean_scrub_stamp":"2023-08-09T13:55:28.281790+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T22:04:16.087605+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"18.1b","version":"146'1","reported_seq":12,"reported_epoch":146,"state":"active+clean","last_fresh":"2023-08-09T13:55:29.373764+0000","last_change":"2023-08-09T13:55:28.321515+0000","last_active":"2023-08-09T13:55:29.373764+0000","last_peered":"2023-08-09T13:55:29.373764+0000","last_clean":"2023-08-09T13:55:29.373764+0000","last_became_active":"2023-08-09T13:55:28.319623+0000","last_became_peered":"2023-08-09T13:55:28.319623+0000","last_unstale":"2023-08-09T13:55:29.373764+0000","last_undegraded":"2023-08-09T13:55:29.373764+0000","last_fullsized":"2023-08-09T13:55:29.373764+0000","mapping_epoch":144,"log_start":"0'0","ondisk_log_start":"0'0","created":144,"last_epoch_clean":145,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:55:27.277343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:55:27.277343+0000","last_clean_scrub_stamp":"2023-08-09T13:55:27.277343+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T18:14:21.871364+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"19.1b","version":"0'0","reported_seq":9,"reported_epoch":146,"state":"active+clean","last_fresh":"2023-08-09T13:55:29.387426+0000","last_change":"2023-08-09T13:55:29.383710+0000","last_active":"2023-08-09T13:55:29.387426+0000","last_peered":"2023-08-09T13:55:29.387426+0000","last_clean":"2023-08-09T13:55:29.387426+0000","last_became_active":"2023-08-09T13:55:29.383372+0000","last_became_peered":"2023-08-09T13:55:29.383372+0000","last_unstale":"2023-08-09T13:55:29.387426+0000","last_undegraded":"2023-08-09T13:55:29.387426+0000","last_fullsized":"2023-08-09T13:55:29.387426+0000","mapping_epoch":145,"log_start":"0'0","ondisk_log_start":"0'0","created":145,"last_epoch_clean":146,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:55:28.281790+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:55:28.281790+0000","last_clean_scrub_stamp":"2023-08-09T13:55:28.281790+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-11T00:02:15.023221+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"18.1a","version":"146'1","reported_seq":12,"reported_epoch":146,"state":"active+clean","last_fresh":"2023-08-09T13:55:29.370556+0000","last_change":"2023-08-09T13:55:28.323112+0000","last_active":"2023-08-09T13:55:29.370556+0000","last_peered":"2023-08-09T13:55:29.370556+0000","last_clean":"2023-08-09T13:55:29.370556+0000","last_became_active":"2023-08-09T13:55:28.322814+0000","last_became_peered":"2023-08-09T13:55:28.322814+0000","last_unstale":"2023-08-09T13:55:29.370556+0000","last_undegraded":"2023-08-09T13:55:29.370556+0000","last_fullsized":"2023-08-09T13:55:29.370556+0000","mapping_epoch":144,"log_start":"0'0","ondisk_log_start":"0'0","created":144,"last_epoch_clean":145,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:55:27.277343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:55:27.277343+0000","last_clean_scrub_stamp":"2023-08-09T13:55:27.277343+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T14:11:35.933204+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"19.18","version":"0'0","reported_seq":9,"reported_epoch":146,"state":"active+clean","last_fresh":"2023-08-09T13:55:29.387140+0000","last_change":"2023-08-09T13:55:29.380096+0000","last_active":"2023-08-09T13:55:29.387140+0000","last_peered":"2023-08-09T13:55:29.387140+0000","last_clean":"2023-08-09T13:55:29.387140+0000","last_became_active":"2023-08-09T13:55:29.378924+0000","last_became_peered":"2023-08-09T13:55:29.378924+0000","last_unstale":"2023-08-09T13:55:29.387140+0000","last_undegraded":"2023-08-09T13:55:29.387140+0000","last_fullsized":"2023-08-09T13:55:29.387140+0000","mapping_epoch":145,"log_start":"0'0","ondisk_log_start":"0'0","created":145,"last_epoch_clean":146,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:55:28.281790+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:55:28.281790+0000","last_clean_scrub_stamp":"2023-08-09T13:55:28.281790+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T18:31:58.455662+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"18.19","version":"0'0","reported_seq":11,"reported_epoch":146,"state":"active+clean","last_fresh":"2023-08-09T13:55:29.364645+0000","last_change":"2023-08-09T13:55:28.333674+0000","last_active":"2023-08-09T13:55:29.364645+0000","last_peered":"2023-08-09T13:55:29.364645+0000","last_clean":"2023-08-09T13:55:29.364645+0000","last_became_active":"2023-08-09T13:55:28.333331+0000","last_became_peered":"2023-08-09T13:55:28.333331+0000","last_unstale":"2023-08-09T13:55:29.364645+0000","last_undegraded":"2023-08-09T13:55:29.364645+0000","last_fullsized":"2023-08-09T13:55:29.364645+0000","mapping_epoch":144,"log_start":"0'0","ondisk_log_start":"0'0","created":144,"last_epoch_clean":145,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:55:27.277343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:55:27.277343+0000","last_clean_scrub_stamp":"2023-08-09T13:55:27.277343+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T19:05:36.216831+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"19.19","version":"0'0","reported_seq":9,"reported_epoch":146,"state":"active+clean","last_fresh":"2023-08-09T13:55:29.365807+0000","last_change":"2023-08-09T13:55:29.362843+0000","last_active":"2023-08-09T13:55:29.365807+0000","last_peered":"2023-08-09T13:55:29.365807+0000","last_clean":"2023-08-09T13:55:29.365807+0000","last_became_active":"2023-08-09T13:55:29.362566+0000","last_became_peered":"2023-08-09T13:55:29.362566+0000","last_unstale":"2023-08-09T13:55:29.365807+0000","last_undegraded":"2023-08-09T13:55:29.365807+0000","last_fullsized":"2023-08-09T13:55:29.365807+0000","mapping_epoch":145,"log_start":"0'0","ondisk_log_start":"0'0","created":145,"last_epoch_clean":146,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:55:28.281790+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:55:28.281790+0000","last_clean_scrub_stamp":"2023-08-09T13:55:28.281790+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-11T01:09:55.545777+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"18.18","version":"0'0","reported_seq":11,"reported_epoch":146,"state":"active+clean","last_fresh":"2023-08-09T13:55:29.353641+0000","last_change":"2023-08-09T13:55:28.324609+0000","last_active":"2023-08-09T13:55:29.353641+0000","last_peered":"2023-08-09T13:55:29.353641+0000","last_clean":"2023-08-09T13:55:29.353641+0000","last_became_active":"2023-08-09T13:55:28.319370+0000","last_became_peered":"2023-08-09T13:55:28.319370+0000","last_unstale":"2023-08-09T13:55:29.353641+0000","last_undegraded":"2023-08-09T13:55:29.353641+0000","last_fullsized":"2023-08-09T13:55:29.353641+0000","mapping_epoch":144,"log_start":"0'0","ondisk_log_start":"0'0","created":144,"last_epoch_clean":145,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:55:27.277343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:55:27.277343+0000","last_clean_scrub_stamp":"2023-08-09T13:55:27.277343+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T21:43:02.662378+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"19.e","version":"0'0","reported_seq":9,"reported_epoch":146,"state":"active+clean","last_fresh":"2023-08-09T13:55:29.382294+0000","last_change":"2023-08-09T13:55:29.377981+0000","last_active":"2023-08-09T13:55:29.382294+0000","last_peered":"2023-08-09T13:55:29.382294+0000","last_clean":"2023-08-09T13:55:29.382294+0000","last_became_active":"2023-08-09T13:55:29.377680+0000","last_became_peered":"2023-08-09T13:55:29.377680+0000","last_unstale":"2023-08-09T13:55:29.382294+0000","last_undegraded":"2023-08-09T13:55:29.382294+0000","last_fullsized":"2023-08-09T13:55:29.382294+0000","mapping_epoch":145,"log_start":"0'0","ondisk_log_start":"0'0","created":145,"last_epoch_clean":146,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:55:28.281790+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:55:28.281790+0000","last_clean_scrub_stamp":"2023-08-09T13:55:28.281790+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-11T00:35:01.430681+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"18.f","version":"146'1","reported_seq":12,"reported_epoch":146,"state":"active+clean","last_fresh":"2023-08-09T13:55:29.390727+0000","last_change":"2023-08-09T13:55:28.329080+0000","last_active":"2023-08-09T13:55:29.390727+0000","last_peered":"2023-08-09T13:55:29.390727+0000","last_clean":"2023-08-09T13:55:29.390727+0000","last_became_active":"2023-08-09T13:55:28.328777+0000","last_became_peered":"2023-08-09T13:55:28.328777+0000","last_unstale":"2023-08-09T13:55:29.390727+0000","last_undegraded":"2023-08-09T13:55:29.390727+0000","last_fullsized":"2023-08-09T13:55:29.390727+0000","mapping_epoch":144,"log_start":"0'0","ondisk_log_start":"0'0","created":144,"last_epoch_clean":145,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:55:27.277343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:55:27.277343+0000","last_clean_scrub_stamp":"2023-08-09T13:55:27.277343+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T22:33:15.141151+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"19.f","version":"0'0","reported_seq":9,"reported_epoch":146,"state":"active+clean","last_fresh":"2023-08-09T13:55:29.387430+0000","last_change":"2023-08-09T13:55:29.370021+0000","last_active":"2023-08-09T13:55:29.387430+0000","last_peered":"2023-08-09T13:55:29.387430+0000","last_clean":"2023-08-09T13:55:29.387430+0000","last_became_active":"2023-08-09T13:55:29.368833+0000","last_became_peered":"2023-08-09T13:55:29.368833+0000","last_unstale":"2023-08-09T13:55:29.387430+0000","last_undegraded":"2023-08-09T13:55:29.387430+0000","last_fullsized":"2023-08-09T13:55:29.387430+0000","mapping_epoch":145,"log_start":"0'0","ondisk_log_start":"0'0","created":145,"last_epoch_clean":146,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:55:28.281790+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:55:28.281790+0000","last_clean_scrub_stamp":"2023-08-09T13:55:28.281790+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T19:08:27.717114+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"18.e","version":"146'2","reported_seq":13,"reported_epoch":146,"state":"active+clean","last_fresh":"2023-08-09T13:55:29.376770+0000","last_change":"2023-08-09T13:55:28.323302+0000","last_active":"2023-08-09T13:55:29.376770+0000","last_peered":"2023-08-09T13:55:29.376770+0000","last_clean":"2023-08-09T13:55:29.376770+0000","last_became_active":"2023-08-09T13:55:28.323032+0000","last_became_peered":"2023-08-09T13:55:28.323032+0000","last_unstale":"2023-08-09T13:55:29.376770+0000","last_undegraded":"2023-08-09T13:55:29.376770+0000","last_fullsized":"2023-08-09T13:55:29.376770+0000","mapping_epoch":144,"log_start":"0'0","ondisk_log_start":"0'0","created":144,"last_epoch_clean":145,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:55:27.277343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:55:27.277343+0000","last_clean_scrub_stamp":"2023-08-09T13:55:27.277343+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T23:31:38.128489+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"19.c","version":"0'0","reported_seq":9,"reported_epoch":146,"state":"active+clean","last_fresh":"2023-08-09T13:55:29.363728+0000","last_change":"2023-08-09T13:55:29.361604+0000","last_active":"2023-08-09T13:55:29.363728+0000","last_peered":"2023-08-09T13:55:29.363728+0000","last_clean":"2023-08-09T13:55:29.363728+0000","last_became_active":"2023-08-09T13:55:29.361295+0000","last_became_peered":"2023-08-09T13:55:29.361295+0000","last_unstale":"2023-08-09T13:55:29.363728+0000","last_undegraded":"2023-08-09T13:55:29.363728+0000","last_fullsized":"2023-08-09T13:55:29.363728+0000","mapping_epoch":145,"log_start":"0'0","ondisk_log_start":"0'0","created":145,"last_epoch_clean":146,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:55:28.281790+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:55:28.281790+0000","last_clean_scrub_stamp":"2023-08-09T13:55:28.281790+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-11T00:36:30.884672+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"18.d","version":"146'1","reported_seq":12,"reported_epoch":146,"state":"active+clean","last_fresh":"2023-08-09T13:55:29.384282+0000","last_change":"2023-08-09T13:55:28.321976+0000","last_active":"2023-08-09T13:55:29.384282+0000","last_peered":"2023-08-09T13:55:29.384282+0000","last_clean":"2023-08-09T13:55:29.384282+0000","last_became_active":"2023-08-09T13:55:28.321604+0000","last_became_peered":"2023-08-09T13:55:28.321604+0000","last_unstale":"2023-08-09T13:55:29.384282+0000","last_undegraded":"2023-08-09T13:55:29.384282+0000","last_fullsized":"2023-08-09T13:55:29.384282+0000","mapping_epoch":144,"log_start":"0'0","ondisk_log_start":"0'0","created":144,"last_epoch_clean":145,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:55:27.277343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:55:27.277343+0000","last_clean_scrub_stamp":"2023-08-09T13:55:27.277343+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T22:52:07.842471+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"19.d","version":"0'0","reported_seq":9,"reported_epoch":146,"state":"active+clean","last_fresh":"2023-08-09T13:55:29.367390+0000","last_change":"2023-08-09T13:55:29.365501+0000","last_active":"2023-08-09T13:55:29.367390+0000","last_peered":"2023-08-09T13:55:29.367390+0000","last_clean":"2023-08-09T13:55:29.367390+0000","last_became_active":"2023-08-09T13:55:29.365217+0000","last_became_peered":"2023-08-09T13:55:29.365217+0000","last_unstale":"2023-08-09T13:55:29.367390+0000","last_undegraded":"2023-08-09T13:55:29.367390+0000","last_fullsized":"2023-08-09T13:55:29.367390+0000","mapping_epoch":145,"log_start":"0'0","ondisk_log_start":"0'0","created":145,"last_epoch_clean":146,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:55:28.281790+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:55:28.281790+0000","last_clean_scrub_stamp":"2023-08-09T13:55:28.281790+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T20:01:41.336446+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"18.c","version":"0'0","reported_seq":11,"reported_epoch":146,"state":"active+clean","last_fresh":"2023-08-09T13:55:29.362642+0000","last_change":"2023-08-09T13:55:28.315212+0000","last_active":"2023-08-09T13:55:29.362642+0000","last_peered":"2023-08-09T13:55:29.362642+0000","last_clean":"2023-08-09T13:55:29.362642+0000","last_became_active":"2023-08-09T13:55:28.314816+0000","last_became_peered":"2023-08-09T13:55:28.314816+0000","last_unstale":"2023-08-09T13:55:29.362642+0000","last_undegraded":"2023-08-09T13:55:29.362642+0000","last_fullsized":"2023-08-09T13:55:29.362642+0000","mapping_epoch":144,"log_start":"0'0","ondisk_log_start":"0'0","created":144,"last_epoch_clean":145,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:55:27.277343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:55:27.277343+0000","last_clean_scrub_stamp":"2023-08-09T13:55:27.277343+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T17:38:40.269099+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"19.a","version":"0'0","reported_seq":9,"reported_epoch":146,"state":"active+clean","last_fresh":"2023-08-09T13:55:29.389981+0000","last_change":"2023-08-09T13:55:29.386663+0000","last_active":"2023-08-09T13:55:29.389981+0000","last_peered":"2023-08-09T13:55:29.389981+0000","last_clean":"2023-08-09T13:55:29.389981+0000","last_became_active":"2023-08-09T13:55:29.386280+0000","last_became_peered":"2023-08-09T13:55:29.386280+0000","last_unstale":"2023-08-09T13:55:29.389981+0000","last_undegraded":"2023-08-09T13:55:29.389981+0000","last_fullsized":"2023-08-09T13:55:29.389981+0000","mapping_epoch":145,"log_start":"0'0","ondisk_log_start":"0'0","created":145,"last_epoch_clean":146,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:55:28.281790+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:55:28.281790+0000","last_clean_scrub_stamp":"2023-08-09T13:55:28.281790+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T19:07:50.742521+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"18.b","version":"146'2","reported_seq":13,"reported_epoch":146,"state":"active+clean","last_fresh":"2023-08-09T13:55:29.389375+0000","last_change":"2023-08-09T13:55:28.316450+0000","last_active":"2023-08-09T13:55:29.389375+0000","last_peered":"2023-08-09T13:55:29.389375+0000","last_clean":"2023-08-09T13:55:29.389375+0000","last_became_active":"2023-08-09T13:55:28.316125+0000","last_became_peered":"2023-08-09T13:55:28.316125+0000","last_unstale":"2023-08-09T13:55:29.389375+0000","last_undegraded":"2023-08-09T13:55:29.389375+0000","last_fullsized":"2023-08-09T13:55:29.389375+0000","mapping_epoch":144,"log_start":"0'0","ondisk_log_start":"0'0","created":144,"last_epoch_clean":145,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:55:27.277343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:55:27.277343+0000","last_clean_scrub_stamp":"2023-08-09T13:55:27.277343+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T19:05:21.321719+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"19.b","version":"0'0","reported_seq":9,"reported_epoch":146,"state":"active+clean","last_fresh":"2023-08-09T13:55:29.385510+0000","last_change":"2023-08-09T13:55:29.380286+0000","last_active":"2023-08-09T13:55:29.385510+0000","last_peered":"2023-08-09T13:55:29.385510+0000","last_clean":"2023-08-09T13:55:29.385510+0000","last_became_active":"2023-08-09T13:55:29.379950+0000","last_became_peered":"2023-08-09T13:55:29.379950+0000","last_unstale":"2023-08-09T13:55:29.385510+0000","last_undegraded":"2023-08-09T13:55:29.385510+0000","last_fullsized":"2023-08-09T13:55:29.385510+0000","mapping_epoch":145,"log_start":"0'0","ondisk_log_start":"0'0","created":145,"last_epoch_clean":146,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:55:28.281790+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:55:28.281790+0000","last_clean_scrub_stamp":"2023-08-09T13:55:28.281790+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T14:06:52.572647+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"18.a","version":"0'0","reported_seq":11,"reported_epoch":146,"state":"active+clean","last_fresh":"2023-08-09T13:55:29.343279+0000","last_change":"2023-08-09T13:55:28.322913+0000","last_active":"2023-08-09T13:55:29.343279+0000","last_peered":"2023-08-09T13:55:29.343279+0000","last_clean":"2023-08-09T13:55:29.343279+0000","last_became_active":"2023-08-09T13:55:28.322548+0000","last_became_peered":"2023-08-09T13:55:28.322548+0000","last_unstale":"2023-08-09T13:55:29.343279+0000","last_undegraded":"2023-08-09T13:55:29.343279+0000","last_fullsized":"2023-08-09T13:55:29.343279+0000","mapping_epoch":144,"log_start":"0'0","ondisk_log_start":"0'0","created":144,"last_epoch_clean":145,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:55:27.277343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:55:27.277343+0000","last_clean_scrub_stamp":"2023-08-09T13:55:27.277343+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-11T00:11:46.016919+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"19.8","version":"0'0","reported_seq":9,"reported_epoch":146,"state":"active+clean","last_fresh":"2023-08-09T13:55:29.365998+0000","last_change":"2023-08-09T13:55:29.365069+0000","last_active":"2023-08-09T13:55:29.365998+0000","last_peered":"2023-08-09T13:55:29.365998+0000","last_clean":"2023-08-09T13:55:29.365998+0000","last_became_active":"2023-08-09T13:55:29.364763+0000","last_became_peered":"2023-08-09T13:55:29.364763+0000","last_unstale":"2023-08-09T13:55:29.365998+0000","last_undegraded":"2023-08-09T13:55:29.365998+0000","last_fullsized":"2023-08-09T13:55:29.365998+0000","mapping_epoch":145,"log_start":"0'0","ondisk_log_start":"0'0","created":145,"last_epoch_clean":146,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:55:28.281790+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:55:28.281790+0000","last_clean_scrub_stamp":"2023-08-09T13:55:28.281790+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T15:43:38.940271+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"18.9","version":"0'0","reported_seq":11,"reported_epoch":146,"state":"active+clean","last_fresh":"2023-08-09T13:55:29.343582+0000","last_change":"2023-08-09T13:55:28.322323+0000","last_active":"2023-08-09T13:55:29.343582+0000","last_peered":"2023-08-09T13:55:29.343582+0000","last_clean":"2023-08-09T13:55:29.343582+0000","last_became_active":"2023-08-09T13:55:28.321651+0000","last_became_peered":"2023-08-09T13:55:28.321651+0000","last_unstale":"2023-08-09T13:55:29.343582+0000","last_undegraded":"2023-08-09T13:55:29.343582+0000","last_fullsized":"2023-08-09T13:55:29.343582+0000","mapping_epoch":144,"log_start":"0'0","ondisk_log_start":"0'0","created":144,"last_epoch_clean":145,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:55:27.277343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:55:27.277343+0000","last_clean_scrub_stamp":"2023-08-09T13:55:27.277343+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T19:19:29.231021+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"19.9","version":"0'0","reported_seq":9,"reported_epoch":146,"state":"active+clean","last_fresh":"2023-08-09T13:55:29.380375+0000","last_change":"2023-08-09T13:55:29.378637+0000","last_active":"2023-08-09T13:55:29.380375+0000","last_peered":"2023-08-09T13:55:29.380375+0000","last_clean":"2023-08-09T13:55:29.380375+0000","last_became_active":"2023-08-09T13:55:29.377377+0000","last_became_peered":"2023-08-09T13:55:29.377377+0000","last_unstale":"2023-08-09T13:55:29.380375+0000","last_undegraded":"2023-08-09T13:55:29.380375+0000","last_fullsized":"2023-08-09T13:55:29.380375+0000","mapping_epoch":145,"log_start":"0'0","ondisk_log_start":"0'0","created":145,"last_epoch_clean":146,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:55:28.281790+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:55:28.281790+0000","last_clean_scrub_stamp":"2023-08-09T13:55:28.281790+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T20:36:38.323524+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"18.8","version":"0'0","reported_seq":11,"reported_epoch":146,"state":"active+clean","last_fresh":"2023-08-09T13:55:29.335915+0000","last_change":"2023-08-09T13:55:28.325598+0000","last_active":"2023-08-09T13:55:29.335915+0000","last_peered":"2023-08-09T13:55:29.335915+0000","last_clean":"2023-08-09T13:55:29.335915+0000","last_became_active":"2023-08-09T13:55:28.325300+0000","last_became_peered":"2023-08-09T13:55:28.325300+0000","last_unstale":"2023-08-09T13:55:29.335915+0000","last_undegraded":"2023-08-09T13:55:29.335915+0000","last_fullsized":"2023-08-09T13:55:29.335915+0000","mapping_epoch":144,"log_start":"0'0","ondisk_log_start":"0'0","created":144,"last_epoch_clean":145,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:55:27.277343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:55:27.277343+0000","last_clean_scrub_stamp":"2023-08-09T13:55:27.277343+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T23:29:41.577663+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"19.6","version":"0'0","reported_seq":9,"reported_epoch":146,"state":"active+clean","last_fresh":"2023-08-09T13:55:29.393336+0000","last_change":"2023-08-09T13:55:29.389917+0000","last_active":"2023-08-09T13:55:29.393336+0000","last_peered":"2023-08-09T13:55:29.393336+0000","last_clean":"2023-08-09T13:55:29.393336+0000","last_became_active":"2023-08-09T13:55:29.387910+0000","last_became_peered":"2023-08-09T13:55:29.387910+0000","last_unstale":"2023-08-09T13:55:29.393336+0000","last_undegraded":"2023-08-09T13:55:29.393336+0000","last_fullsized":"2023-08-09T13:55:29.393336+0000","mapping_epoch":145,"log_start":"0'0","ondisk_log_start":"0'0","created":145,"last_epoch_clean":146,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:55:28.281790+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:55:28.281790+0000","last_clean_scrub_stamp":"2023-08-09T13:55:28.281790+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T22:42:15.840116+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"18.7","version":"146'2","reported_seq":13,"reported_epoch":146,"state":"active+clean","last_fresh":"2023-08-09T13:55:29.385963+0000","last_change":"2023-08-09T13:55:28.323788+0000","last_active":"2023-08-09T13:55:29.385963+0000","last_peered":"2023-08-09T13:55:29.385963+0000","last_clean":"2023-08-09T13:55:29.385963+0000","last_became_active":"2023-08-09T13:55:28.323507+0000","last_became_peered":"2023-08-09T13:55:28.323507+0000","last_unstale":"2023-08-09T13:55:29.385963+0000","last_undegraded":"2023-08-09T13:55:29.385963+0000","last_fullsized":"2023-08-09T13:55:29.385963+0000","mapping_epoch":144,"log_start":"0'0","ondisk_log_start":"0'0","created":144,"last_epoch_clean":145,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:55:27.277343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:55:27.277343+0000","last_clean_scrub_stamp":"2023-08-09T13:55:27.277343+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T19:29:58.936799+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"19.7","version":"0'0","reported_seq":9,"reported_epoch":146,"state":"active+clean","last_fresh":"2023-08-09T13:55:29.382851+0000","last_change":"2023-08-09T13:55:29.377753+0000","last_active":"2023-08-09T13:55:29.382851+0000","last_peered":"2023-08-09T13:55:29.382851+0000","last_clean":"2023-08-09T13:55:29.382851+0000","last_became_active":"2023-08-09T13:55:29.377285+0000","last_became_peered":"2023-08-09T13:55:29.377285+0000","last_unstale":"2023-08-09T13:55:29.382851+0000","last_undegraded":"2023-08-09T13:55:29.382851+0000","last_fullsized":"2023-08-09T13:55:29.382851+0000","mapping_epoch":145,"log_start":"0'0","ondisk_log_start":"0'0","created":145,"last_epoch_clean":146,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:55:28.281790+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:55:28.281790+0000","last_clean_scrub_stamp":"2023-08-09T13:55:28.281790+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T20:48:01.956488+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"18.6","version":"0'0","reported_seq":11,"reported_epoch":146,"state":"active+clean","last_fresh":"2023-08-09T13:55:29.362948+0000","last_change":"2023-08-09T13:55:28.328542+0000","last_active":"2023-08-09T13:55:29.362948+0000","last_peered":"2023-08-09T13:55:29.362948+0000","last_clean":"2023-08-09T13:55:29.362948+0000","last_became_active":"2023-08-09T13:55:28.328244+0000","last_became_peered":"2023-08-09T13:55:28.328244+0000","last_unstale":"2023-08-09T13:55:29.362948+0000","last_undegraded":"2023-08-09T13:55:29.362948+0000","last_fullsized":"2023-08-09T13:55:29.362948+0000","mapping_epoch":144,"log_start":"0'0","ondisk_log_start":"0'0","created":144,"last_epoch_clean":145,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:55:27.277343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:55:27.277343+0000","last_clean_scrub_stamp":"2023-08-09T13:55:27.277343+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T14:46:50.318176+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"19.4","version":"0'0","reported_seq":9,"reported_epoch":146,"state":"active+clean","last_fresh":"2023-08-09T13:55:29.379950+0000","last_change":"2023-08-09T13:55:29.378985+0000","last_active":"2023-08-09T13:55:29.379950+0000","last_peered":"2023-08-09T13:55:29.379950+0000","last_clean":"2023-08-09T13:55:29.379950+0000","last_became_active":"2023-08-09T13:55:29.378675+0000","last_became_peered":"2023-08-09T13:55:29.378675+0000","last_unstale":"2023-08-09T13:55:29.379950+0000","last_undegraded":"2023-08-09T13:55:29.379950+0000","last_fullsized":"2023-08-09T13:55:29.379950+0000","mapping_epoch":145,"log_start":"0'0","ondisk_log_start":"0'0","created":145,"last_epoch_clean":146,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:55:28.281790+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:55:28.281790+0000","last_clean_scrub_stamp":"2023-08-09T13:55:28.281790+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T16:13:09.951708+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"18.5","version":"0'0","reported_seq":11,"reported_epoch":146,"state":"active+clean","last_fresh":"2023-08-09T13:55:29.357421+0000","last_change":"2023-08-09T13:55:28.327073+0000","last_active":"2023-08-09T13:55:29.357421+0000","last_peered":"2023-08-09T13:55:29.357421+0000","last_clean":"2023-08-09T13:55:29.357421+0000","last_became_active":"2023-08-09T13:55:28.326813+0000","last_became_peered":"2023-08-09T13:55:28.326813+0000","last_unstale":"2023-08-09T13:55:29.357421+0000","last_undegraded":"2023-08-09T13:55:29.357421+0000","last_fullsized":"2023-08-09T13:55:29.357421+0000","mapping_epoch":144,"log_start":"0'0","ondisk_log_start":"0'0","created":144,"last_epoch_clean":145,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:55:27.277343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:55:27.277343+0000","last_clean_scrub_stamp":"2023-08-09T13:55:27.277343+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T16:20:54.438480+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"19.5","version":"0'0","reported_seq":9,"reported_epoch":146,"state":"active+clean","last_fresh":"2023-08-09T13:55:29.387417+0000","last_change":"2023-08-09T13:55:29.385183+0000","last_active":"2023-08-09T13:55:29.387417+0000","last_peered":"2023-08-09T13:55:29.387417+0000","last_clean":"2023-08-09T13:55:29.387417+0000","last_became_active":"2023-08-09T13:55:29.384576+0000","last_became_peered":"2023-08-09T13:55:29.384576+0000","last_unstale":"2023-08-09T13:55:29.387417+0000","last_undegraded":"2023-08-09T13:55:29.387417+0000","last_fullsized":"2023-08-09T13:55:29.387417+0000","mapping_epoch":145,"log_start":"0'0","ondisk_log_start":"0'0","created":145,"last_epoch_clean":146,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:55:28.281790+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:55:28.281790+0000","last_clean_scrub_stamp":"2023-08-09T13:55:28.281790+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T16:02:59.215551+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"18.4","version":"0'0","reported_seq":11,"reported_epoch":146,"state":"active+clean","last_fresh":"2023-08-09T13:55:29.335446+0000","last_change":"2023-08-09T13:55:28.317485+0000","last_active":"2023-08-09T13:55:29.335446+0000","last_peered":"2023-08-09T13:55:29.335446+0000","last_clean":"2023-08-09T13:55:29.335446+0000","last_became_active":"2023-08-09T13:55:28.317110+0000","last_became_peered":"2023-08-09T13:55:28.317110+0000","last_unstale":"2023-08-09T13:55:29.335446+0000","last_undegraded":"2023-08-09T13:55:29.335446+0000","last_fullsized":"2023-08-09T13:55:29.335446+0000","mapping_epoch":144,"log_start":"0'0","ondisk_log_start":"0'0","created":144,"last_epoch_clean":145,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:55:27.277343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:55:27.277343+0000","last_clean_scrub_stamp":"2023-08-09T13:55:27.277343+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T20:06:03.354893+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"19.2","version":"0'0","reported_seq":9,"reported_epoch":146,"state":"active+clean","last_fresh":"2023-08-09T13:55:29.389893+0000","last_change":"2023-08-09T13:55:29.386265+0000","last_active":"2023-08-09T13:55:29.389893+0000","last_peered":"2023-08-09T13:55:29.389893+0000","last_clean":"2023-08-09T13:55:29.389893+0000","last_became_active":"2023-08-09T13:55:29.385908+0000","last_became_peered":"2023-08-09T13:55:29.385908+0000","last_unstale":"2023-08-09T13:55:29.389893+0000","last_undegraded":"2023-08-09T13:55:29.389893+0000","last_fullsized":"2023-08-09T13:55:29.389893+0000","mapping_epoch":145,"log_start":"0'0","ondisk_log_start":"0'0","created":145,"last_epoch_clean":146,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:55:28.281790+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:55:28.281790+0000","last_clean_scrub_stamp":"2023-08-09T13:55:28.281790+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T22:31:20.809531+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"18.3","version":"0'0","reported_seq":11,"reported_epoch":146,"state":"active+clean","last_fresh":"2023-08-09T13:55:29.353043+0000","last_change":"2023-08-09T13:55:28.328336+0000","last_active":"2023-08-09T13:55:29.353043+0000","last_peered":"2023-08-09T13:55:29.353043+0000","last_clean":"2023-08-09T13:55:29.353043+0000","last_became_active":"2023-08-09T13:55:28.328036+0000","last_became_peered":"2023-08-09T13:55:28.328036+0000","last_unstale":"2023-08-09T13:55:29.353043+0000","last_undegraded":"2023-08-09T13:55:29.353043+0000","last_fullsized":"2023-08-09T13:55:29.353043+0000","mapping_epoch":144,"log_start":"0'0","ondisk_log_start":"0'0","created":144,"last_epoch_clean":145,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:55:27.277343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:55:27.277343+0000","last_clean_scrub_stamp":"2023-08-09T13:55:27.277343+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T17:11:15.222036+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"19.3","version":"0'0","reported_seq":9,"reported_epoch":146,"state":"active+clean","last_fresh":"2023-08-09T13:55:29.366142+0000","last_change":"2023-08-09T13:55:29.363281+0000","last_active":"2023-08-09T13:55:29.366142+0000","last_peered":"2023-08-09T13:55:29.366142+0000","last_clean":"2023-08-09T13:55:29.366142+0000","last_became_active":"2023-08-09T13:55:29.362967+0000","last_became_peered":"2023-08-09T13:55:29.362967+0000","last_unstale":"2023-08-09T13:55:29.366142+0000","last_undegraded":"2023-08-09T13:55:29.366142+0000","last_fullsized":"2023-08-09T13:55:29.366142+0000","mapping_epoch":145,"log_start":"0'0","ondisk_log_start":"0'0","created":145,"last_epoch_clean":146,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:55:28.281790+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:55:28.281790+0000","last_clean_scrub_stamp":"2023-08-09T13:55:28.281790+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T20:19:18.935035+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"18.2","version":"0'0","reported_seq":11,"reported_epoch":146,"state":"active+clean","last_fresh":"2023-08-09T13:55:29.357584+0000","last_change":"2023-08-09T13:55:28.325478+0000","last_active":"2023-08-09T13:55:29.357584+0000","last_peered":"2023-08-09T13:55:29.357584+0000","last_clean":"2023-08-09T13:55:29.357584+0000","last_became_active":"2023-08-09T13:55:28.325182+0000","last_became_peered":"2023-08-09T13:55:28.325182+0000","last_unstale":"2023-08-09T13:55:29.357584+0000","last_undegraded":"2023-08-09T13:55:29.357584+0000","last_fullsized":"2023-08-09T13:55:29.357584+0000","mapping_epoch":144,"log_start":"0'0","ondisk_log_start":"0'0","created":144,"last_epoch_clean":145,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:55:27.277343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:55:27.277343+0000","last_clean_scrub_stamp":"2023-08-09T13:55:27.277343+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-11T00:35:07.214932+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"19.0","version":"0'0","reported_seq":9,"reported_epoch":146,"state":"active+clean","last_fresh":"2023-08-09T13:55:29.383677+0000","last_change":"2023-08-09T13:55:29.380961+0000","last_active":"2023-08-09T13:55:29.383677+0000","last_peered":"2023-08-09T13:55:29.383677+0000","last_clean":"2023-08-09T13:55:29.383677+0000","last_became_active":"2023-08-09T13:55:29.380639+0000","last_became_peered":"2023-08-09T13:55:29.380639+0000","last_unstale":"2023-08-09T13:55:29.383677+0000","last_undegraded":"2023-08-09T13:55:29.383677+0000","last_fullsized":"2023-08-09T13:55:29.383677+0000","mapping_epoch":145,"log_start":"0'0","ondisk_log_start":"0'0","created":145,"last_epoch_clean":146,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:55:28.281790+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:55:28.281790+0000","last_clean_scrub_stamp":"2023-08-09T13:55:28.281790+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T18:00:24.971826+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"18.1","version":"0'0","reported_seq":11,"reported_epoch":146,"state":"active+clean","last_fresh":"2023-08-09T13:55:29.343750+0000","last_change":"2023-08-09T13:55:28.322516+0000","last_active":"2023-08-09T13:55:29.343750+0000","last_peered":"2023-08-09T13:55:29.343750+0000","last_clean":"2023-08-09T13:55:29.343750+0000","last_became_active":"2023-08-09T13:55:28.322036+0000","last_became_peered":"2023-08-09T13:55:28.322036+0000","last_unstale":"2023-08-09T13:55:29.343750+0000","last_undegraded":"2023-08-09T13:55:29.343750+0000","last_fullsized":"2023-08-09T13:55:29.343750+0000","mapping_epoch":144,"log_start":"0'0","ondisk_log_start":"0'0","created":144,"last_epoch_clean":145,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:55:27.277343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:55:27.277343+0000","last_clean_scrub_stamp":"2023-08-09T13:55:27.277343+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-11T01:20:23.987524+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"19.1","version":"0'0","reported_seq":9,"reported_epoch":146,"state":"active+clean","last_fresh":"2023-08-09T13:55:29.370909+0000","last_change":"2023-08-09T13:55:29.368598+0000","last_active":"2023-08-09T13:55:29.370909+0000","last_peered":"2023-08-09T13:55:29.370909+0000","last_clean":"2023-08-09T13:55:29.370909+0000","last_became_active":"2023-08-09T13:55:29.368284+0000","last_became_peered":"2023-08-09T13:55:29.368284+0000","last_unstale":"2023-08-09T13:55:29.370909+0000","last_undegraded":"2023-08-09T13:55:29.370909+0000","last_fullsized":"2023-08-09T13:55:29.370909+0000","mapping_epoch":145,"log_start":"0'0","ondisk_log_start":"0'0","created":145,"last_epoch_clean":146,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:55:28.281790+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:55:28.281790+0000","last_clean_scrub_stamp":"2023-08-09T13:55:28.281790+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T22:11:08.463964+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"18.0","version":"0'0","reported_seq":11,"reported_epoch":146,"state":"active+clean","last_fresh":"2023-08-09T13:55:29.336303+0000","last_change":"2023-08-09T13:55:28.321218+0000","last_active":"2023-08-09T13:55:29.336303+0000","last_peered":"2023-08-09T13:55:29.336303+0000","last_clean":"2023-08-09T13:55:29.336303+0000","last_became_active":"2023-08-09T13:55:28.320885+0000","last_became_peered":"2023-08-09T13:55:28.320885+0000","last_unstale":"2023-08-09T13:55:29.336303+0000","last_undegraded":"2023-08-09T13:55:29.336303+0000","last_fullsized":"2023-08-09T13:55:29.336303+0000","mapping_epoch":144,"log_start":"0'0","ondisk_log_start":"0'0","created":144,"last_epoch_clean":145,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:55:27.277343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:55:27.277343+0000","last_clean_scrub_stamp":"2023-08-09T13:55:27.277343+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-11T01:17:28.831020+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"19.12","version":"0'0","reported_seq":9,"reported_epoch":146,"state":"active+clean","last_fresh":"2023-08-09T13:55:29.380776+0000","last_change":"2023-08-09T13:55:29.378718+0000","last_active":"2023-08-09T13:55:29.380776+0000","last_peered":"2023-08-09T13:55:29.380776+0000","last_clean":"2023-08-09T13:55:29.380776+0000","last_became_active":"2023-08-09T13:55:29.378360+0000","last_became_peered":"2023-08-09T13:55:29.378360+0000","last_unstale":"2023-08-09T13:55:29.380776+0000","last_undegraded":"2023-08-09T13:55:29.380776+0000","last_fullsized":"2023-08-09T13:55:29.380776+0000","mapping_epoch":145,"log_start":"0'0","ondisk_log_start":"0'0","created":145,"last_epoch_clean":146,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:55:28.281790+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:55:28.281790+0000","last_clean_scrub_stamp":"2023-08-09T13:55:28.281790+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T22:03:35.883538+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"18.13","version":"146'2","reported_seq":13,"reported_epoch":146,"state":"active+clean","last_fresh":"2023-08-09T13:55:29.383182+0000","last_change":"2023-08-09T13:55:28.330858+0000","last_active":"2023-08-09T13:55:29.383182+0000","last_peered":"2023-08-09T13:55:29.383182+0000","last_clean":"2023-08-09T13:55:29.383182+0000","last_became_active":"2023-08-09T13:55:28.330570+0000","last_became_peered":"2023-08-09T13:55:28.330570+0000","last_unstale":"2023-08-09T13:55:29.383182+0000","last_undegraded":"2023-08-09T13:55:29.383182+0000","last_fullsized":"2023-08-09T13:55:29.383182+0000","mapping_epoch":144,"log_start":"0'0","ondisk_log_start":"0'0","created":144,"last_epoch_clean":145,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:55:27.277343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:55:27.277343+0000","last_clean_scrub_stamp":"2023-08-09T13:55:27.277343+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-11T00:17:08.270713+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":249,"reported_epoch":146,"state":"active+clean","last_fresh":"2023-08-09T13:55:29.355802+0000","last_change":"2023-08-09T13:33:56.401406+0000","last_active":"2023-08-09T13:55:29.355802+0000","last_peered":"2023-08-09T13:55:29.355802+0000","last_clean":"2023-08-09T13:55:29.355802+0000","last_became_active":"2023-08-09T13:33:56.400487+0000","last_became_peered":"2023-08-09T13:33:56.400487+0000","last_unstale":"2023-08-09T13:55:29.355802+0000","last_undegraded":"2023-08-09T13:55:29.355802+0000","last_fullsized":"2023-08-09T13:55:29.355802+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:33:54.845869+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:33:54.845869+0000","last_clean_scrub_stamp":"2023-08-09T13:33:54.845869+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T23:33:46.735697+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"19.13","version":"0'0","reported_seq":9,"reported_epoch":146,"state":"active+clean","last_fresh":"2023-08-09T13:55:29.358546+0000","last_change":"2023-08-09T13:55:29.347419+0000","last_active":"2023-08-09T13:55:29.358546+0000","last_peered":"2023-08-09T13:55:29.358546+0000","last_clean":"2023-08-09T13:55:29.358546+0000","last_became_active":"2023-08-09T13:55:29.347094+0000","last_became_peered":"2023-08-09T13:55:29.347094+0000","last_unstale":"2023-08-09T13:55:29.358546+0000","last_undegraded":"2023-08-09T13:55:29.358546+0000","last_fullsized":"2023-08-09T13:55:29.358546+0000","mapping_epoch":145,"log_start":"0'0","ondisk_log_start":"0'0","created":145,"last_epoch_clean":146,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:55:28.281790+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:55:28.281790+0000","last_clean_scrub_stamp":"2023-08-09T13:55:28.281790+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-11T00:19:35.426972+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"18.12","version":"0'0","reported_seq":11,"reported_epoch":146,"state":"active+clean","last_fresh":"2023-08-09T13:55:29.341204+0000","last_change":"2023-08-09T13:55:28.320668+0000","last_active":"2023-08-09T13:55:29.341204+0000","last_peered":"2023-08-09T13:55:29.341204+0000","last_clean":"2023-08-09T13:55:29.341204+0000","last_became_active":"2023-08-09T13:55:28.320258+0000","last_became_peered":"2023-08-09T13:55:28.320258+0000","last_unstale":"2023-08-09T13:55:29.341204+0000","last_undegraded":"2023-08-09T13:55:29.341204+0000","last_fullsized":"2023-08-09T13:55:29.341204+0000","mapping_epoch":144,"log_start":"0'0","ondisk_log_start":"0'0","created":144,"last_epoch_clean":145,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:55:27.277343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:55:27.277343+0000","last_clean_scrub_stamp":"2023-08-09T13:55:27.277343+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T19:58:41.585493+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":251,"reported_epoch":146,"state":"active+clean","last_fresh":"2023-08-09T13:55:29.352896+0000","last_change":"2023-08-09T13:33:56.401123+0000","last_active":"2023-08-09T13:55:29.352896+0000","last_peered":"2023-08-09T13:55:29.352896+0000","last_clean":"2023-08-09T13:55:29.352896+0000","last_became_active":"2023-08-09T13:33:56.399933+0000","last_became_peered":"2023-08-09T13:33:56.399933+0000","last_unstale":"2023-08-09T13:55:29.352896+0000","last_undegraded":"2023-08-09T13:55:29.352896+0000","last_fullsized":"2023-08-09T13:55:29.352896+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:33:54.845869+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:33:54.845869+0000","last_clean_scrub_stamp":"2023-08-09T13:33:54.845869+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T22:43:50.638469+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"19.10","version":"0'0","reported_seq":9,"reported_epoch":146,"state":"active+clean","last_fresh":"2023-08-09T13:55:29.382441+0000","last_change":"2023-08-09T13:55:29.380953+0000","last_active":"2023-08-09T13:55:29.382441+0000","last_peered":"2023-08-09T13:55:29.382441+0000","last_clean":"2023-08-09T13:55:29.382441+0000","last_became_active":"2023-08-09T13:55:29.380675+0000","last_became_peered":"2023-08-09T13:55:29.380675+0000","last_unstale":"2023-08-09T13:55:29.382441+0000","last_undegraded":"2023-08-09T13:55:29.382441+0000","last_fullsized":"2023-08-09T13:55:29.382441+0000","mapping_epoch":145,"log_start":"0'0","ondisk_log_start":"0'0","created":145,"last_epoch_clean":146,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:55:28.281790+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:55:28.281790+0000","last_clean_scrub_stamp":"2023-08-09T13:55:28.281790+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T22:11:23.849215+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"18.11","version":"0'0","reported_seq":11,"reported_epoch":146,"state":"active+clean","last_fresh":"2023-08-09T13:55:29.354962+0000","last_change":"2023-08-09T13:55:28.328548+0000","last_active":"2023-08-09T13:55:29.354962+0000","last_peered":"2023-08-09T13:55:29.354962+0000","last_clean":"2023-08-09T13:55:29.354962+0000","last_became_active":"2023-08-09T13:55:28.328239+0000","last_became_peered":"2023-08-09T13:55:28.328239+0000","last_unstale":"2023-08-09T13:55:29.354962+0000","last_undegraded":"2023-08-09T13:55:29.354962+0000","last_fullsized":"2023-08-09T13:55:29.354962+0000","mapping_epoch":144,"log_start":"0'0","ondisk_log_start":"0'0","created":144,"last_epoch_clean":145,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:55:27.277343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:55:27.277343+0000","last_clean_scrub_stamp":"2023-08-09T13:55:27.277343+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T17:22:59.355534+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.2","version":"0'0","reported_seq":249,"reported_epoch":146,"state":"active+clean","last_fresh":"2023-08-09T13:55:29.357771+0000","last_change":"2023-08-09T13:33:56.402794+0000","last_active":"2023-08-09T13:55:29.357771+0000","last_peered":"2023-08-09T13:55:29.357771+0000","last_clean":"2023-08-09T13:55:29.357771+0000","last_became_active":"2023-08-09T13:33:56.402038+0000","last_became_peered":"2023-08-09T13:33:56.402038+0000","last_unstale":"2023-08-09T13:55:29.357771+0000","last_undegraded":"2023-08-09T13:55:29.357771+0000","last_fullsized":"2023-08-09T13:55:29.357771+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:33:54.845869+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:33:54.845869+0000","last_clean_scrub_stamp":"2023-08-09T13:33:54.845869+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T14:14:28.831260+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"19.11","version":"0'0","reported_seq":9,"reported_epoch":146,"state":"active+clean","last_fresh":"2023-08-09T13:55:29.392037+0000","last_change":"2023-08-09T13:55:29.389616+0000","last_active":"2023-08-09T13:55:29.392037+0000","last_peered":"2023-08-09T13:55:29.392037+0000","last_clean":"2023-08-09T13:55:29.392037+0000","last_became_active":"2023-08-09T13:55:29.389114+0000","last_became_peered":"2023-08-09T13:55:29.389114+0000","last_unstale":"2023-08-09T13:55:29.392037+0000","last_undegraded":"2023-08-09T13:55:29.392037+0000","last_fullsized":"2023-08-09T13:55:29.392037+0000","mapping_epoch":145,"log_start":"0'0","ondisk_log_start":"0'0","created":145,"last_epoch_clean":146,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:55:28.281790+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:55:28.281790+0000","last_clean_scrub_stamp":"2023-08-09T13:55:28.281790+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T18:10:48.285444+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"18.10","version":"146'8","reported_seq":19,"reported_epoch":146,"state":"active+clean","last_fresh":"2023-08-09T13:55:29.391972+0000","last_change":"2023-08-09T13:55:28.322058+0000","last_active":"2023-08-09T13:55:29.391972+0000","last_peered":"2023-08-09T13:55:29.391972+0000","last_clean":"2023-08-09T13:55:29.391972+0000","last_became_active":"2023-08-09T13:55:28.321769+0000","last_became_peered":"2023-08-09T13:55:28.321769+0000","last_unstale":"2023-08-09T13:55:29.391972+0000","last_undegraded":"2023-08-09T13:55:29.391972+0000","last_fullsized":"2023-08-09T13:55:29.391972+0000","mapping_epoch":144,"log_start":"0'0","ondisk_log_start":"0'0","created":144,"last_epoch_clean":145,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:55:27.277343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:55:27.277343+0000","last_clean_scrub_stamp":"2023-08-09T13:55:27.277343+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T15:34:41.903487+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"1.3","version":"0'0","reported_seq":251,"reported_epoch":146,"state":"active+clean","last_fresh":"2023-08-09T13:55:29.353256+0000","last_change":"2023-08-09T13:33:55.886749+0000","last_active":"2023-08-09T13:55:29.353256+0000","last_peered":"2023-08-09T13:55:29.353256+0000","last_clean":"2023-08-09T13:55:29.353256+0000","last_became_active":"2023-08-09T13:33:55.886116+0000","last_became_peered":"2023-08-09T13:33:55.886116+0000","last_unstale":"2023-08-09T13:55:29.353256+0000","last_undegraded":"2023-08-09T13:55:29.353256+0000","last_fullsized":"2023-08-09T13:55:29.353256+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:33:54.845869+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:33:54.845869+0000","last_clean_scrub_stamp":"2023-08-09T13:33:54.845869+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T22:34:38.570994+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"134'230","reported_seq":799,"reported_epoch":146,"state":"active+clean","last_fresh":"2023-08-09T13:55:29.355455+0000","last_change":"2023-08-09T13:33:55.886345+0000","last_active":"2023-08-09T13:55:29.355455+0000","last_peered":"2023-08-09T13:55:29.355455+0000","last_clean":"2023-08-09T13:55:29.355455+0000","last_became_active":"2023-08-09T13:33:55.885781+0000","last_became_peered":"2023-08-09T13:33:55.885781+0000","last_unstale":"2023-08-09T13:55:29.355455+0000","last_undegraded":"2023-08-09T13:55:29.355455+0000","last_fullsized":"2023-08-09T13:55:29.355455+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:33:54.845869+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:33:54.845869+0000","last_clean_scrub_stamp":"2023-08-09T13:33:54.845869+0000","objects_scrubbed":0,"log_size":230,"log_dups_size":0,"ondisk_log_size":230,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T20:59:57.120396+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1099,"num_read_kb":2242,"num_write":363,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"19.16","version":"0'0","reported_seq":9,"reported_epoch":146,"state":"active+clean","last_fresh":"2023-08-09T13:55:29.363725+0000","last_change":"2023-08-09T13:55:29.357951+0000","last_active":"2023-08-09T13:55:29.363725+0000","last_peered":"2023-08-09T13:55:29.363725+0000","last_clean":"2023-08-09T13:55:29.363725+0000","last_became_active":"2023-08-09T13:55:29.357461+0000","last_became_peered":"2023-08-09T13:55:29.357461+0000","last_unstale":"2023-08-09T13:55:29.363725+0000","last_undegraded":"2023-08-09T13:55:29.363725+0000","last_fullsized":"2023-08-09T13:55:29.363725+0000","mapping_epoch":145,"log_start":"0'0","ondisk_log_start":"0'0","created":145,"last_epoch_clean":146,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:55:28.281790+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:55:28.281790+0000","last_clean_scrub_stamp":"2023-08-09T13:55:28.281790+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T16:37:22.262338+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"18.17","version":"0'0","reported_seq":11,"reported_epoch":146,"state":"active+clean","last_fresh":"2023-08-09T13:55:29.361433+0000","last_change":"2023-08-09T13:55:28.319541+0000","last_active":"2023-08-09T13:55:29.361433+0000","last_peered":"2023-08-09T13:55:29.361433+0000","last_clean":"2023-08-09T13:55:29.361433+0000","last_became_active":"2023-08-09T13:55:28.319094+0000","last_became_peered":"2023-08-09T13:55:28.319094+0000","last_unstale":"2023-08-09T13:55:29.361433+0000","last_undegraded":"2023-08-09T13:55:29.361433+0000","last_fullsized":"2023-08-09T13:55:29.361433+0000","mapping_epoch":144,"log_start":"0'0","ondisk_log_start":"0'0","created":144,"last_epoch_clean":145,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:55:27.277343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:55:27.277343+0000","last_clean_scrub_stamp":"2023-08-09T13:55:27.277343+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T19:50:13.138643+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":257,"reported_epoch":146,"state":"active+clean","last_fresh":"2023-08-09T13:55:29.344850+0000","last_change":"2023-08-09T13:33:55.889565+0000","last_active":"2023-08-09T13:55:29.344850+0000","last_peered":"2023-08-09T13:55:29.344850+0000","last_clean":"2023-08-09T13:55:29.344850+0000","last_became_active":"2023-08-09T13:33:55.889164+0000","last_became_peered":"2023-08-09T13:33:55.889164+0000","last_unstale":"2023-08-09T13:55:29.344850+0000","last_undegraded":"2023-08-09T13:55:29.344850+0000","last_fullsized":"2023-08-09T13:55:29.344850+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:33:54.845869+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:33:54.845869+0000","last_clean_scrub_stamp":"2023-08-09T13:33:54.845869+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T23:48:46.682375+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"19.17","version":"0'0","reported_seq":9,"reported_epoch":146,"state":"active+clean","last_fresh":"2023-08-09T13:55:29.387450+0000","last_change":"2023-08-09T13:55:29.369149+0000","last_active":"2023-08-09T13:55:29.387450+0000","last_peered":"2023-08-09T13:55:29.387450+0000","last_clean":"2023-08-09T13:55:29.387450+0000","last_became_active":"2023-08-09T13:55:29.367877+0000","last_became_peered":"2023-08-09T13:55:29.367877+0000","last_unstale":"2023-08-09T13:55:29.387450+0000","last_undegraded":"2023-08-09T13:55:29.387450+0000","last_fullsized":"2023-08-09T13:55:29.387450+0000","mapping_epoch":145,"log_start":"0'0","ondisk_log_start":"0'0","created":145,"last_epoch_clean":146,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:55:28.281790+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:55:28.281790+0000","last_clean_scrub_stamp":"2023-08-09T13:55:28.281790+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T20:40:41.419067+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"18.16","version":"0'0","reported_seq":11,"reported_epoch":146,"state":"active+clean","last_fresh":"2023-08-09T13:55:29.345931+0000","last_change":"2023-08-09T13:55:28.322495+0000","last_active":"2023-08-09T13:55:29.345931+0000","last_peered":"2023-08-09T13:55:29.345931+0000","last_clean":"2023-08-09T13:55:29.345931+0000","last_became_active":"2023-08-09T13:55:28.322189+0000","last_became_peered":"2023-08-09T13:55:28.322189+0000","last_unstale":"2023-08-09T13:55:29.345931+0000","last_undegraded":"2023-08-09T13:55:29.345931+0000","last_fullsized":"2023-08-09T13:55:29.345931+0000","mapping_epoch":144,"log_start":"0'0","ondisk_log_start":"0'0","created":144,"last_epoch_clean":145,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:55:27.277343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:55:27.277343+0000","last_clean_scrub_stamp":"2023-08-09T13:55:27.277343+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T19:10:18.092027+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":251,"reported_epoch":146,"state":"active+clean","last_fresh":"2023-08-09T13:55:29.368694+0000","last_change":"2023-08-09T13:33:56.404236+0000","last_active":"2023-08-09T13:55:29.368694+0000","last_peered":"2023-08-09T13:55:29.368694+0000","last_clean":"2023-08-09T13:55:29.368694+0000","last_became_active":"2023-08-09T13:33:56.402485+0000","last_became_peered":"2023-08-09T13:33:56.402485+0000","last_unstale":"2023-08-09T13:55:29.368694+0000","last_undegraded":"2023-08-09T13:55:29.368694+0000","last_fullsized":"2023-08-09T13:55:29.368694+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:33:54.845869+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:33:54.845869+0000","last_clean_scrub_stamp":"2023-08-09T13:33:54.845869+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-11T00:14:49.582247+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"19.14","version":"0'0","reported_seq":9,"reported_epoch":146,"state":"active+clean","last_fresh":"2023-08-09T13:55:29.374857+0000","last_change":"2023-08-09T13:55:29.373999+0000","last_active":"2023-08-09T13:55:29.374857+0000","last_peered":"2023-08-09T13:55:29.374857+0000","last_clean":"2023-08-09T13:55:29.374857+0000","last_became_active":"2023-08-09T13:55:29.373714+0000","last_became_peered":"2023-08-09T13:55:29.373714+0000","last_unstale":"2023-08-09T13:55:29.374857+0000","last_undegraded":"2023-08-09T13:55:29.374857+0000","last_fullsized":"2023-08-09T13:55:29.374857+0000","mapping_epoch":145,"log_start":"0'0","ondisk_log_start":"0'0","created":145,"last_epoch_clean":146,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:55:28.281790+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:55:28.281790+0000","last_clean_scrub_stamp":"2023-08-09T13:55:28.281790+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T22:39:16.014980+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"18.15","version":"146'3","reported_seq":14,"reported_epoch":146,"state":"active+clean","last_fresh":"2023-08-09T13:55:29.387562+0000","last_change":"2023-08-09T13:55:28.322247+0000","last_active":"2023-08-09T13:55:29.387562+0000","last_peered":"2023-08-09T13:55:29.387562+0000","last_clean":"2023-08-09T13:55:29.387562+0000","last_became_active":"2023-08-09T13:55:28.321893+0000","last_became_peered":"2023-08-09T13:55:28.321893+0000","last_unstale":"2023-08-09T13:55:29.387562+0000","last_undegraded":"2023-08-09T13:55:29.387562+0000","last_fullsized":"2023-08-09T13:55:29.387562+0000","mapping_epoch":144,"log_start":"0'0","ondisk_log_start":"0'0","created":144,"last_epoch_clean":145,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:55:27.277343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:55:27.277343+0000","last_clean_scrub_stamp":"2023-08-09T13:55:27.277343+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T20:48:57.060910+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":244,"reported_epoch":146,"state":"active+clean","last_fresh":"2023-08-09T13:55:29.372009+0000","last_change":"2023-08-09T13:33:56.401781+0000","last_active":"2023-08-09T13:55:29.372009+0000","last_peered":"2023-08-09T13:55:29.372009+0000","last_clean":"2023-08-09T13:55:29.372009+0000","last_became_active":"2023-08-09T13:33:56.400336+0000","last_became_peered":"2023-08-09T13:33:56.400336+0000","last_unstale":"2023-08-09T13:55:29.372009+0000","last_undegraded":"2023-08-09T13:55:29.372009+0000","last_fullsized":"2023-08-09T13:55:29.372009+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:33:54.845869+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:33:54.845869+0000","last_clean_scrub_stamp":"2023-08-09T13:33:54.845869+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T23:47:41.691639+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"19.15","version":"0'0","reported_seq":9,"reported_epoch":146,"state":"active+clean","last_fresh":"2023-08-09T13:55:29.385460+0000","last_change":"2023-08-09T13:55:29.381047+0000","last_active":"2023-08-09T13:55:29.385460+0000","last_peered":"2023-08-09T13:55:29.385460+0000","last_clean":"2023-08-09T13:55:29.385460+0000","last_became_active":"2023-08-09T13:55:29.380540+0000","last_became_peered":"2023-08-09T13:55:29.380540+0000","last_unstale":"2023-08-09T13:55:29.385460+0000","last_undegraded":"2023-08-09T13:55:29.385460+0000","last_fullsized":"2023-08-09T13:55:29.385460+0000","mapping_epoch":145,"log_start":"0'0","ondisk_log_start":"0'0","created":145,"last_epoch_clean":146,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:55:28.281790+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:55:28.281790+0000","last_clean_scrub_stamp":"2023-08-09T13:55:28.281790+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T22:40:35.323784+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"18.14","version":"146'6","reported_seq":17,"reported_epoch":146,"state":"active+clean","last_fresh":"2023-08-09T13:55:29.391101+0000","last_change":"2023-08-09T13:55:28.328980+0000","last_active":"2023-08-09T13:55:29.391101+0000","last_peered":"2023-08-09T13:55:29.391101+0000","last_clean":"2023-08-09T13:55:29.391101+0000","last_became_active":"2023-08-09T13:55:28.328700+0000","last_became_peered":"2023-08-09T13:55:28.328700+0000","last_unstale":"2023-08-09T13:55:29.391101+0000","last_undegraded":"2023-08-09T13:55:29.391101+0000","last_fullsized":"2023-08-09T13:55:29.391101+0000","mapping_epoch":144,"log_start":"0'0","ondisk_log_start":"0'0","created":144,"last_epoch_clean":145,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:55:27.277343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:55:27.277343+0000","last_clean_scrub_stamp":"2023-08-09T13:55:27.277343+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T17:15:19.603732+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":244,"reported_epoch":146,"state":"active+clean","last_fresh":"2023-08-09T13:55:29.371695+0000","last_change":"2023-08-09T13:33:55.886389+0000","last_active":"2023-08-09T13:55:29.371695+0000","last_peered":"2023-08-09T13:55:29.371695+0000","last_clean":"2023-08-09T13:55:29.371695+0000","last_became_active":"2023-08-09T13:33:55.885638+0000","last_became_peered":"2023-08-09T13:33:55.885638+0000","last_unstale":"2023-08-09T13:55:29.371695+0000","last_undegraded":"2023-08-09T13:55:29.371695+0000","last_fullsized":"2023-08-09T13:55:29.371695+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:33:54.845869+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:33:54.845869+0000","last_clean_scrub_stamp":"2023-08-09T13:33:54.845869+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T22:03:00.007141+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]}],"pool_stats":[{"poolid":19,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":18,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":38,"ondisk_log_size":38,"up":64,"acting":64,"num_store_stats":8},{"poolid":2,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1099,"num_read_kb":2242,"num_write":363,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":230,"ondisk_log_size":230,"up":2,"acting":2,"num_store_stats":2},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":8}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834575104,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":37552,"kb_used_data":2340,"kb_used_omap":1,"kb_used_meta":35198,"kb_avail":94334288,"statfs":{"total":96636764160,"available":96598310912,"internally_reserved":0,"allocated":2396160,"data_stored":1556377,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1781,"internal_metadata":36043019},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Aug 9 13:55:29 2023","interfaces":[{"interface":"back","average":{"1min":1.2070000000000001,"5min":1.1160000000000001,"15min":1.198},"min":{"1min":0.73099999999999998,"5min":0.60499999999999998,"15min":0.60499999999999998},"max":{"1min":1.71,"5min":1.869,"15min":1.8919999999999999},"last":0.75700000000000001},{"interface":"front","average":{"1min":1.1200000000000001,"5min":1.171,"15min":1.214},"min":{"1min":0.745,"5min":0.64200000000000002,"15min":0.64200000000000002},"max":{"1min":1.6870000000000001,"5min":2.3399999999999999,"15min":2.3399999999999999},"last":0.745}]},{"osd":1,"last update":"Wed Aug 9 13:55:29 2023","interfaces":[{"interface":"back","average":{"1min":1.276,"5min":1.258,"15min":1.2629999999999999},"min":{"1min":0.53100000000000003,"5min":0.53100000000000003,"15min":0.53100000000000003},"max":{"1min":1.6639999999999999,"5min":2.1989999999999998,"15min":2.1989999999999998},"last":0.53100000000000003},{"interface":"front","average":{"1min":1.226,"5min":1.212,"15min":1.2549999999999999},"min":{"1min":0.68200000000000005,"5min":0.68200000000000005,"15min":0.59299999999999997},"max":{"1min":1.8460000000000001,"5min":1.998,"15min":1.998},"last":0.68200000000000005}]},{"osd":2,"last update":"Wed Aug 9 13:55:29 2023","interfaces":[{"interface":"back","average":{"1min":1.1639999999999999,"5min":1.2250000000000001,"15min":1.248},"min":{"1min":0.76300000000000001,"5min":0.68300000000000005,"15min":0.65700000000000003},"max":{"1min":1.5620000000000001,"5min":2.0089999999999999,"15min":2.0129999999999999},"last":0.77400000000000002},{"interface":"front","average":{"1min":1.2190000000000001,"5min":1.2410000000000001,"15min":1.3300000000000001},"min":{"1min":0.70499999999999996,"5min":0.58199999999999996,"15min":0.58199999999999996},"max":{"1min":1.6619999999999999,"5min":2.359,"15min":2.359},"last":0.70499999999999996}]},{"osd":3,"last update":"Wed Aug 9 13:55:29 2023","interfaces":[{"interface":"back","average":{"1min":1.2,"5min":1.284,"15min":1.3260000000000001},"min":{"1min":0.73199999999999998,"5min":0.63,"15min":0.63},"max":{"1min":1.694,"5min":2.4380000000000002,"15min":2.4380000000000002},"last":0.73199999999999998},{"interface":"front","average":{"1min":1.3460000000000001,"5min":1.3779999999999999,"15min":1.415},"min":{"1min":0.78700000000000003,"5min":0.65700000000000003,"15min":0.65700000000000003},"max":{"1min":1.7889999999999999,"5min":2.6360000000000001,"15min":2.6360000000000001},"last":0.78700000000000003}]},{"osd":4,"last update":"Wed Aug 9 13:55:29 2023","interfaces":[{"interface":"back","average":{"1min":1.1719999999999999,"5min":1.234,"15min":1.2609999999999999},"min":{"1min":0.57699999999999996,"5min":0.57699999999999996,"15min":0.57699999999999996},"max":{"1min":1.647,"5min":2.097,"15min":2.097},"last":0.57699999999999996},{"interface":"front","average":{"1min":1.256,"5min":1.335,"15min":1.337},"min":{"1min":0.57499999999999996,"5min":0.52300000000000002,"15min":0.52300000000000002},"max":{"1min":1.641,"5min":2.6890000000000001,"15min":2.6890000000000001},"last":0.60399999999999998}]},{"osd":5,"last update":"Wed Aug 9 13:55:29 2023","interfaces":[{"interface":"back","average":{"1min":1.3029999999999999,"5min":1.427,"15min":1.425},"min":{"1min":0.64200000000000002,"5min":0.501,"15min":0.501},"max":{"1min":1.7589999999999999,"5min":2.7509999999999999,"15min":2.7509999999999999},"last":0.64200000000000002},{"interface":"front","average":{"1min":1.3899999999999999,"5min":1.429,"15min":1.4399999999999999},"min":{"1min":0.629,"5min":0.629,"15min":0.629},"max":{"1min":1.95,"5min":2.8639999999999999,"15min":2.8639999999999999},"last":0.629}]},{"osd":6,"last update":"Wed Aug 9 13:55:29 2023","interfaces":[{"interface":"back","average":{"1min":1.4610000000000001,"5min":1.528,"15min":1.532},"min":{"1min":0.71899999999999997,"5min":0.71899999999999997,"15min":0.71899999999999997},"max":{"1min":1.7889999999999999,"5min":2.7370000000000001,"15min":2.7370000000000001},"last":0.71899999999999997},{"interface":"front","average":{"1min":1.4379999999999999,"5min":1.4630000000000001,"15min":1.4730000000000001},"min":{"1min":0.624,"5min":0.57099999999999995,"15min":0.57099999999999995},"max":{"1min":1.8959999999999999,"5min":2.7850000000000001,"15min":2.7850000000000001},"last":0.66700000000000004}]}]},{"osd":6,"up_from":13,"seq":55834575104,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":36972,"kb_used_data":1756,"kb_used_omap":1,"kb_used_meta":35198,"kb_avail":94334868,"statfs":{"total":96636764160,"available":96598904832,"internally_reserved":0,"allocated":1798144,"data_stored":964648,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1781,"internal_metadata":36043019},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Aug 9 13:55:28 2023","interfaces":[{"interface":"back","average":{"1min":1.242,"5min":1.141,"15min":1.1519999999999999},"min":{"1min":0.79700000000000004,"5min":0.47999999999999998,"15min":0.47999999999999998},"max":{"1min":2.2149999999999999,"5min":2.2149999999999999,"15min":2.2149999999999999},"last":1.468},{"interface":"front","average":{"1min":1.1779999999999999,"5min":1.159,"15min":1.3009999999999999},"min":{"1min":0.69499999999999995,"5min":0.51800000000000002,"15min":0.51800000000000002},"max":{"1min":1.6579999999999999,"5min":1.8520000000000001,"15min":28.036000000000001},"last":1.0109999999999999}]},{"osd":1,"last update":"Wed Aug 9 13:55:28 2023","interfaces":[{"interface":"back","average":{"1min":1.2250000000000001,"5min":1.1639999999999999,"15min":1.224},"min":{"1min":0.71699999999999997,"5min":0.48999999999999999,"15min":0.48999999999999999},"max":{"1min":1.9019999999999999,"5min":2.2320000000000002,"15min":7.742},"last":1.208},{"interface":"front","average":{"1min":1.272,"5min":1.2250000000000001,"15min":1.258},"min":{"1min":0.79500000000000004,"5min":0.63,"15min":0.63},"max":{"1min":2.1120000000000001,"5min":2.1120000000000001,"15min":2.1120000000000001},"last":1.339}]},{"osd":2,"last update":"Wed Aug 9 13:55:28 2023","interfaces":[{"interface":"back","average":{"1min":1.3480000000000001,"5min":1.2869999999999999,"15min":1.3200000000000001},"min":{"1min":0.88200000000000001,"5min":0.627,"15min":0.627},"max":{"1min":1.988,"5min":2.1899999999999999,"15min":4.0640000000000001},"last":1.121},{"interface":"front","average":{"1min":1.2190000000000001,"5min":1.2569999999999999,"15min":1.339},"min":{"1min":0.77300000000000002,"5min":0.77300000000000002,"15min":0.70999999999999996},"max":{"1min":1.637,"5min":2.5819999999999999,"15min":13.154999999999999},"last":1.5}]},{"osd":3,"last update":"Wed Aug 9 13:55:28 2023","interfaces":[{"interface":"back","average":{"1min":1.3069999999999999,"5min":1.3120000000000001,"15min":1.4139999999999999},"min":{"1min":0.89100000000000001,"5min":0.67200000000000004,"15min":0.67200000000000004},"max":{"1min":2.1749999999999998,"5min":2.1749999999999998,"15min":13.349},"last":1.4019999999999999},{"interface":"front","average":{"1min":1.4179999999999999,"5min":1.3029999999999999,"15min":1.3300000000000001},"min":{"1min":0.95699999999999996,"5min":0.75700000000000001,"15min":0.75700000000000001},"max":{"1min":1.984,"5min":1.984,"15min":7.9029999999999996},"last":1.53}]},{"osd":4,"last update":"Wed Aug 9 13:55:28 2023","interfaces":[{"interface":"back","average":{"1min":1.3300000000000001,"5min":1.294,"15min":1.2869999999999999},"min":{"1min":0.91000000000000003,"5min":0.35099999999999998,"15min":0.35099999999999998},"max":{"1min":1.7470000000000001,"5min":1.833,"15min":5.2290000000000001},"last":0.96299999999999997},{"interface":"front","average":{"1min":1.397,"5min":1.3520000000000001,"15min":1.3839999999999999},"min":{"1min":1.056,"5min":0.54800000000000004,"15min":0.54800000000000004},"max":{"1min":2.0299999999999998,"5min":2.0299999999999998,"15min":12.1},"last":1.1539999999999999}]},{"osd":5,"last update":"Wed Aug 9 13:55:28 2023","interfaces":[{"interface":"back","average":{"1min":1.351,"5min":1.3160000000000001,"15min":1.3660000000000001},"min":{"1min":0.85399999999999998,"5min":0.51000000000000001,"15min":0.51000000000000001},"max":{"1min":1.7310000000000001,"5min":2.1349999999999998,"15min":8.1899999999999995},"last":1.4410000000000001},{"interface":"front","average":{"1min":1.6020000000000001,"5min":1.5649999999999999,"15min":1.593},"min":{"1min":1.1859999999999999,"5min":0.42399999999999999,"15min":0.42399999999999999},"max":{"1min":2.29,"5min":2.29,"15min":8.8889999999999993},"last":1.5820000000000001}]},{"osd":7,"last update":"Wed Aug 9 13:55:28 2023","interfaces":[{"interface":"back","average":{"1min":1.5700000000000001,"5min":1.5700000000000001,"15min":1.5940000000000001},"min":{"1min":1.3140000000000001,"5min":0.69299999999999995,"15min":0.62},"max":{"1min":2.0499999999999998,"5min":2.4529999999999998,"15min":9.3320000000000007},"last":1.6519999999999999},{"interface":"front","average":{"1min":1.603,"5min":1.5189999999999999,"15min":1.552},"min":{"1min":1.3520000000000001,"5min":0.59899999999999998,"15min":0.59899999999999998},"max":{"1min":2.117,"5min":2.2909999999999999,"15min":8.0399999999999991},"last":1.611}]}]},{"osd":5,"up_from":13,"seq":55834575104,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":59120,"kb_used_data":1760,"kb_used_omap":1,"kb_used_meta":57342,"kb_avail":94312720,"statfs":{"total":96636764160,"available":96576225280,"internally_reserved":0,"allocated":1802240,"data_stored":966009,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1781,"internal_metadata":58718475},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Aug 9 13:55:26 2023","interfaces":[{"interface":"back","average":{"1min":1.0329999999999999,"5min":1.21,"15min":1.1930000000000001},"min":{"1min":0.70499999999999996,"5min":0.59099999999999997,"15min":0.59099999999999997},"max":{"1min":1.6120000000000001,"5min":8.9299999999999997,"15min":8.9299999999999997},"last":0.89600000000000002},{"interface":"front","average":{"1min":1.121,"5min":1.25,"15min":1.22},"min":{"1min":0.71399999999999997,"5min":0.56000000000000005,"15min":0.56000000000000005},"max":{"1min":1.6379999999999999,"5min":8.7140000000000004,"15min":8.7140000000000004},"last":1.0840000000000001}]},{"osd":1,"last update":"Wed Aug 9 13:55:26 2023","interfaces":[{"interface":"back","average":{"1min":1.1699999999999999,"5min":1.2310000000000001,"15min":1.236},"min":{"1min":0.79500000000000004,"5min":0.66600000000000004,"15min":0.66600000000000004},"max":{"1min":1.5640000000000001,"5min":8.2720000000000002,"15min":8.2720000000000002},"last":1.1890000000000001},{"interface":"front","average":{"1min":1.26,"5min":1.3420000000000001,"15min":1.323},"min":{"1min":0.91900000000000004,"5min":0.66300000000000003,"15min":0.66300000000000003},"max":{"1min":1.554,"5min":9.0020000000000007,"15min":9.0020000000000007},"last":1.1419999999999999}]},{"osd":2,"last update":"Wed Aug 9 13:55:26 2023","interfaces":[{"interface":"back","average":{"1min":1.163,"5min":1.325,"15min":1.3029999999999999},"min":{"1min":0.72099999999999997,"5min":0.72099999999999997,"15min":0.68999999999999995},"max":{"1min":1.7270000000000001,"5min":8.6050000000000004,"15min":8.6050000000000004},"last":0.85699999999999998},{"interface":"front","average":{"1min":1.2230000000000001,"5min":1.345,"15min":1.335},"min":{"1min":0.81399999999999995,"5min":0.78300000000000003,"15min":0.78300000000000003},"max":{"1min":1.5980000000000001,"5min":8.4149999999999991,"15min":8.4149999999999991},"last":0.95899999999999996}]},{"osd":3,"last update":"Wed Aug 9 13:55:26 2023","interfaces":[{"interface":"back","average":{"1min":1.1930000000000001,"5min":1.373,"15min":1.365},"min":{"1min":0.83699999999999997,"5min":0.66200000000000003,"15min":0.66200000000000003},"max":{"1min":1.7829999999999999,"5min":9.0470000000000006,"15min":9.0470000000000006},"last":1.028},{"interface":"front","average":{"1min":1.379,"5min":1.4770000000000001,"15min":1.4490000000000001},"min":{"1min":1.04,"5min":0.63,"15min":0.63},"max":{"1min":1.6259999999999999,"5min":8.7669999999999995,"15min":8.7669999999999995},"last":1.2609999999999999}]},{"osd":4,"last update":"Wed Aug 9 13:55:26 2023","interfaces":[{"interface":"back","average":{"1min":1.266,"5min":1.4199999999999999,"15min":1.3400000000000001},"min":{"1min":1.0129999999999999,"5min":0.54300000000000004,"15min":0.54300000000000004},"max":{"1min":1.4990000000000001,"5min":8.4909999999999997,"15min":8.4909999999999997},"last":1.319},{"interface":"front","average":{"1min":1.4810000000000001,"5min":1.5349999999999999,"15min":1.4239999999999999},"min":{"1min":1.2390000000000001,"5min":0.68899999999999995,"15min":0.68899999999999995},"max":{"1min":1.8979999999999999,"5min":9.4589999999999996,"15min":9.4589999999999996},"last":1.2390000000000001}]},{"osd":6,"last update":"Wed Aug 9 13:55:26 2023","interfaces":[{"interface":"back","average":{"1min":1.347,"5min":1.6259999999999999,"15min":1.47},"min":{"1min":0.98199999999999998,"5min":0.51000000000000001,"15min":0.51000000000000001},"max":{"1min":1.641,"5min":17.407,"15min":17.407},"last":1.45},{"interface":"front","average":{"1min":1.4139999999999999,"5min":1.6599999999999999,"15min":1.5309999999999999},"min":{"1min":1.05,"5min":0.71199999999999997,"15min":0.71199999999999997},"max":{"1min":1.6619999999999999,"5min":16.356000000000002,"15min":16.356000000000002},"last":1.413}]},{"osd":7,"last update":"Wed Aug 9 13:55:26 2023","interfaces":[{"interface":"back","average":{"1min":1.5629999999999999,"5min":1.681,"15min":1.593},"min":{"1min":1.333,"5min":0.74299999999999999,"15min":0.74299999999999999},"max":{"1min":2,"5min":9.9239999999999995,"15min":9.9239999999999995},"last":1.375},{"interface":"front","average":{"1min":1.5780000000000001,"5min":1.742,"15min":1.651},"min":{"1min":1.302,"5min":0.65600000000000003,"15min":0.65600000000000003},"max":{"1min":1.8280000000000001,"5min":10.789,"15min":10.789},"last":1.518}]}]},{"osd":4,"up_from":13,"seq":55834575104,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32880,"kb_used_data":1760,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":94338960,"statfs":{"total":96636764160,"available":96603095040,"internally_reserved":0,"allocated":1802240,"data_stored":966009,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1781,"internal_metadata":31848715},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":7,"apply_latency_ms":7,"commit_latency_ns":7000000,"apply_latency_ns":7000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Aug 9 13:54:30 2023","interfaces":[{"interface":"back","average":{"1min":1.163,"5min":1.165,"15min":1.198},"min":{"1min":0.54500000000000004,"5min":0.54500000000000004,"15min":0.54500000000000004},"max":{"1min":1.73,"5min":1.768,"15min":2.524},"last":1.5},{"interface":"front","average":{"1min":1.226,"5min":1.1890000000000001,"15min":1.2450000000000001},"min":{"1min":0.60399999999999998,"5min":0.60399999999999998,"15min":0.60399999999999998},"max":{"1min":2.0779999999999998,"5min":2.0779999999999998,"15min":2.0779999999999998},"last":1.008}]},{"osd":1,"last update":"Wed Aug 9 13:54:30 2023","interfaces":[{"interface":"back","average":{"1min":1.361,"5min":1.258,"15min":1.2949999999999999},"min":{"1min":0.82399999999999995,"5min":0.70999999999999996,"15min":0.67800000000000005},"max":{"1min":1.887,"5min":1.887,"15min":1.887},"last":1.3049999999999999},{"interface":"front","average":{"1min":1.1699999999999999,"5min":1.1910000000000001,"15min":1.24},"min":{"1min":0.74399999999999999,"5min":0.68899999999999995,"15min":0.68899999999999995},"max":{"1min":1.5509999999999999,"5min":1.9330000000000001,"15min":1.9330000000000001},"last":1.4079999999999999}]},{"osd":2,"last update":"Wed Aug 9 13:54:30 2023","interfaces":[{"interface":"back","average":{"1min":1.3140000000000001,"5min":1.2370000000000001,"15min":1.2649999999999999},"min":{"1min":0.79000000000000004,"5min":0.69999999999999996,"15min":0.621},"max":{"1min":1.7050000000000001,"5min":1.7050000000000001,"15min":2.6429999999999998},"last":1.081},{"interface":"front","average":{"1min":1.3480000000000001,"5min":1.3280000000000001,"15min":1.3640000000000001},"min":{"1min":0.77300000000000002,"5min":0.76000000000000001,"15min":0.76000000000000001},"max":{"1min":1.6870000000000001,"5min":2.0910000000000002,"15min":2.0910000000000002},"last":1.1539999999999999}]},{"osd":3,"last update":"Wed Aug 9 13:54:30 2023","interfaces":[{"interface":"back","average":{"1min":1.2030000000000001,"5min":1.2569999999999999,"15min":1.292},"min":{"1min":0.80000000000000004,"5min":0.64700000000000002,"15min":0.64700000000000002},"max":{"1min":1.5589999999999999,"5min":1.921,"15min":2.6099999999999999},"last":1.127},{"interface":"front","average":{"1min":1.379,"5min":1.3680000000000001,"15min":1.3839999999999999},"min":{"1min":1.147,"5min":0.86299999999999999,"15min":0.86299999999999999},"max":{"1min":1.9419999999999999,"5min":2.3119999999999998,"15min":2.7629999999999999},"last":1.202}]},{"osd":5,"last update":"Wed Aug 9 13:54:30 2023","interfaces":[{"interface":"back","average":{"1min":1.296,"5min":1.337,"15min":1.3360000000000001},"min":{"1min":0.58699999999999997,"5min":0.58699999999999997,"15min":0.58699999999999997},"max":{"1min":1.97,"5min":2.0329999999999999,"15min":2.0329999999999999},"last":1.4390000000000001},{"interface":"front","average":{"1min":1.256,"5min":1.3120000000000001,"15min":1.3160000000000001},"min":{"1min":0.67500000000000004,"5min":0.65600000000000003,"15min":0.65600000000000003},"max":{"1min":1.6850000000000001,"5min":2.5880000000000001,"15min":2.7090000000000001},"last":1.252}]},{"osd":6,"last update":"Wed Aug 9 13:54:30 2023","interfaces":[{"interface":"back","average":{"1min":1.3049999999999999,"5min":1.4019999999999999,"15min":1.3440000000000001},"min":{"1min":0.752,"5min":0.752,"15min":0.59099999999999997},"max":{"1min":1.952,"5min":2.5430000000000001,"15min":2.8559999999999999},"last":1.373},{"interface":"front","average":{"1min":1.4970000000000001,"5min":1.5069999999999999,"15min":1.512},"min":{"1min":0.65100000000000002,"5min":0.52600000000000002,"15min":0.52600000000000002},"max":{"1min":2.2050000000000001,"5min":2.2050000000000001,"15min":2.2719999999999998},"last":1.5309999999999999}]},{"osd":7,"last update":"Wed Aug 9 13:54:30 2023","interfaces":[{"interface":"back","average":{"1min":1.3600000000000001,"5min":1.4470000000000001,"15min":1.472},"min":{"1min":0.72399999999999998,"5min":0.53100000000000003,"15min":0.53100000000000003},"max":{"1min":1.8109999999999999,"5min":2.4279999999999999,"15min":2.8050000000000002},"last":1.333},{"interface":"front","average":{"1min":1.518,"5min":1.5469999999999999,"15min":1.5309999999999999},"min":{"1min":0.70099999999999996,"5min":0.70099999999999996,"15min":0.70099999999999996},"max":{"1min":2.105,"5min":2.6429999999999998,"15min":3.415},"last":1.2869999999999999}]}]},{"osd":3,"up_from":13,"seq":55834575105,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":36976,"kb_used_data":1760,"kb_used_omap":1,"kb_used_meta":35198,"kb_avail":94334864,"statfs":{"total":96636764160,"available":96598900736,"internally_reserved":0,"allocated":1802240,"data_stored":966009,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1781,"internal_metadata":36043019},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Aug 9 13:54:28 2023","interfaces":[{"interface":"back","average":{"1min":0.91100000000000003,"5min":0.86399999999999999,"15min":0.90200000000000002},"min":{"1min":0.60699999999999998,"5min":0.43099999999999999,"15min":0.27200000000000002},"max":{"1min":1.4219999999999999,"5min":1.589,"15min":1.8500000000000001},"last":0.72499999999999998},{"interface":"front","average":{"1min":0.95599999999999996,"5min":0.98099999999999998,"15min":1.093},"min":{"1min":0.57799999999999996,"5min":0.36299999999999999,"15min":0.36299999999999999},"max":{"1min":1.5880000000000001,"5min":1.6910000000000001,"15min":2.0590000000000002},"last":0.92300000000000004}]},{"osd":1,"last update":"Wed Aug 9 13:54:28 2023","interfaces":[{"interface":"back","average":{"1min":0.97499999999999998,"5min":0.95299999999999996,"15min":1.0149999999999999},"min":{"1min":0.64200000000000002,"5min":0.51500000000000001,"15min":0.496},"max":{"1min":1.573,"5min":1.9410000000000001,"15min":2.5649999999999999},"last":0.77200000000000002},{"interface":"front","average":{"1min":1.0609999999999999,"5min":1.0740000000000001,"15min":1.1639999999999999},"min":{"1min":0.48699999999999999,"5min":0.48399999999999999,"15min":0.48399999999999999},"max":{"1min":1.637,"5min":1.762,"15min":2.6760000000000002},"last":1.0109999999999999}]},{"osd":2,"last update":"Wed Aug 9 13:54:28 2023","interfaces":[{"interface":"back","average":{"1min":1.1419999999999999,"5min":1.099,"15min":1.1739999999999999},"min":{"1min":0.64100000000000001,"5min":0.42999999999999999,"15min":0.42999999999999999},"max":{"1min":1.7629999999999999,"5min":1.7629999999999999,"15min":1.913},"last":0.64400000000000002},{"interface":"front","average":{"1min":1.246,"5min":1.2230000000000001,"15min":1.3029999999999999},"min":{"1min":0.69899999999999995,"5min":0.52400000000000002,"15min":0.52300000000000002},"max":{"1min":1.8120000000000001,"5min":1.9399999999999999,"15min":2.0259999999999998},"last":1.163}]},{"osd":4,"last update":"Wed Aug 9 13:54:28 2023","interfaces":[{"interface":"back","average":{"1min":1.321,"5min":1.2350000000000001,"15min":1.274},"min":{"1min":0.995,"5min":0.54900000000000004,"15min":0.54900000000000004},"max":{"1min":1.778,"5min":1.778,"15min":1.8160000000000001},"last":1.1160000000000001},{"interface":"front","average":{"1min":1.3640000000000001,"5min":1.3560000000000001,"15min":1.403},"min":{"1min":0.78200000000000003,"5min":0.67600000000000005,"15min":0.628},"max":{"1min":1.74,"5min":2.782,"15min":2.782},"last":1.0640000000000001}]},{"osd":5,"last update":"Wed Aug 9 13:54:28 2023","interfaces":[{"interface":"back","average":{"1min":1.3600000000000001,"5min":1.397,"15min":1.4670000000000001},"min":{"1min":0.73299999999999998,"5min":0.67600000000000005,"15min":0.67600000000000005},"max":{"1min":1.8100000000000001,"5min":2.0489999999999999,"15min":2.254},"last":1.2689999999999999},{"interface":"front","average":{"1min":1.4490000000000001,"5min":1.409,"15min":1.4850000000000001},"min":{"1min":0.88200000000000001,"5min":0.66000000000000003,"15min":0.65900000000000003},"max":{"1min":1.9419999999999999,"5min":2.3069999999999999,"15min":2.3069999999999999},"last":1.298}]},{"osd":6,"last update":"Wed Aug 9 13:54:28 2023","interfaces":[{"interface":"back","average":{"1min":1.3049999999999999,"5min":1.355,"15min":1.4379999999999999},"min":{"1min":0.84899999999999998,"5min":0.78200000000000003,"15min":0.68200000000000005},"max":{"1min":1.752,"5min":2.8340000000000001,"15min":2.8340000000000001},"last":1.236},{"interface":"front","average":{"1min":1.5009999999999999,"5min":1.4790000000000001,"15min":1.548},"min":{"1min":0.85699999999999998,"5min":0.64100000000000001,"15min":0.64100000000000001},"max":{"1min":2.3919999999999999,"5min":2.9060000000000001,"15min":2.9060000000000001},"last":1.202}]},{"osd":7,"last update":"Wed Aug 9 13:54:28 2023","interfaces":[{"interface":"back","average":{"1min":1.536,"5min":1.504,"15min":1.5980000000000001},"min":{"1min":0.83599999999999997,"5min":0.624,"15min":0.624},"max":{"1min":1.8879999999999999,"5min":2.46,"15min":2.46},"last":1.403},{"interface":"front","average":{"1min":1.492,"5min":1.454,"15min":1.542},"min":{"1min":0.92300000000000004,"5min":0.71199999999999997,"15min":0.64200000000000002},"max":{"1min":2.3159999999999998,"5min":2.7200000000000002,"15min":2.7200000000000002},"last":1.349}]}]},{"osd":2,"up_from":13,"seq":55834575104,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":41072,"kb_used_data":1760,"kb_used_omap":1,"kb_used_meta":39294,"kb_avail":94330768,"statfs":{"total":96636764160,"available":96594706432,"internally_reserved":0,"allocated":1802240,"data_stored":966009,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1781,"internal_metadata":40237323},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Aug 9 13:54:24 2023","interfaces":[{"interface":"back","average":{"1min":0.88500000000000001,"5min":0.89700000000000002,"15min":0.94799999999999995},"min":{"1min":0.57299999999999995,"5min":0.35499999999999998,"15min":0.35499999999999998},"max":{"1min":1.5860000000000001,"5min":1.5860000000000001,"15min":1.752},"last":1.123},{"interface":"front","average":{"1min":1.0289999999999999,"5min":0.97199999999999998,"15min":1},"min":{"1min":0.66900000000000004,"5min":0.38200000000000001,"15min":0.374},"max":{"1min":1.4810000000000001,"5min":1.75,"15min":2.3929999999999998},"last":0.99299999999999999}]},{"osd":1,"last update":"Wed Aug 9 13:54:24 2023","interfaces":[{"interface":"back","average":{"1min":1.1970000000000001,"5min":1.1379999999999999,"15min":1.1830000000000001},"min":{"1min":0.64200000000000002,"5min":0.39400000000000002,"15min":0.39400000000000002},"max":{"1min":1.8109999999999999,"5min":1.8109999999999999,"15min":1.8420000000000001},"last":0.75800000000000001},{"interface":"front","average":{"1min":1.2929999999999999,"5min":1.1919999999999999,"15min":1.304},"min":{"1min":0.66800000000000004,"5min":0.435,"15min":0.435},"max":{"1min":1.883,"5min":1.8999999999999999,"15min":2.4870000000000001},"last":0.90900000000000003}]},{"osd":3,"last update":"Wed Aug 9 13:54:24 2023","interfaces":[{"interface":"back","average":{"1min":0.96399999999999997,"5min":0.95799999999999996,"15min":1.0289999999999999},"min":{"1min":0.624,"5min":0.41399999999999998,"15min":0.38900000000000001},"max":{"1min":1.3089999999999999,"5min":1.7130000000000001,"15min":1.7609999999999999},"last":0.57499999999999996},{"interface":"front","average":{"1min":1.256,"5min":1.177,"15min":1.2889999999999999},"min":{"1min":0.81899999999999995,"5min":0.45500000000000002,"15min":0.45500000000000002},"max":{"1min":1.669,"5min":1.835,"15min":8.8130000000000006},"last":1.05}]},{"osd":4,"last update":"Wed Aug 9 13:54:24 2023","interfaces":[{"interface":"back","average":{"1min":1.425,"5min":1.3799999999999999,"15min":1.4430000000000001},"min":{"1min":0.98399999999999999,"5min":0.622,"15min":0.622},"max":{"1min":1.706,"5min":2.1640000000000001,"15min":2.1640000000000001},"last":1.4299999999999999},{"interface":"front","average":{"1min":1.341,"5min":1.353,"15min":1.3819999999999999},"min":{"1min":0.96599999999999997,"5min":0.64800000000000002,"15min":0.64800000000000002},"max":{"1min":1.613,"5min":2.472,"15min":2.472},"last":1.0840000000000001}]},{"osd":5,"last update":"Wed Aug 9 13:54:24 2023","interfaces":[{"interface":"back","average":{"1min":1.4510000000000001,"5min":1.4259999999999999,"15min":1.4530000000000001},"min":{"1min":1.0660000000000001,"5min":0.71199999999999997,"15min":0.59999999999999998},"max":{"1min":1.9259999999999999,"5min":2.4900000000000002,"15min":2.5840000000000001},"last":1.3240000000000001},{"interface":"front","average":{"1min":1.391,"5min":1.335,"15min":1.3560000000000001},"min":{"1min":1.083,"5min":0.73699999999999999,"15min":0.56799999999999995},"max":{"1min":1.7450000000000001,"5min":2.3519999999999999,"15min":3.5659999999999998},"last":1.147}]},{"osd":6,"last update":"Wed Aug 9 13:54:24 2023","interfaces":[{"interface":"back","average":{"1min":1.4039999999999999,"5min":1.377,"15min":1.4279999999999999},"min":{"1min":1.0469999999999999,"5min":0.77600000000000002,"15min":0.73799999999999999},"max":{"1min":1.8340000000000001,"5min":2.5110000000000001,"15min":3.5099999999999998},"last":0.94799999999999995},{"interface":"front","average":{"1min":1.603,"5min":1.496,"15min":1.5429999999999999},"min":{"1min":1.323,"5min":0.66100000000000003,"15min":0.66100000000000003},"max":{"1min":1.962,"5min":2.3769999999999998,"15min":2.3769999999999998},"last":1.353}]},{"osd":7,"last update":"Wed Aug 9 13:54:24 2023","interfaces":[{"interface":"back","average":{"1min":1.526,"5min":1.482,"15min":1.5309999999999999},"min":{"1min":0.91600000000000004,"5min":0.78800000000000003,"15min":0.64600000000000002},"max":{"1min":1.8280000000000001,"5min":2.1970000000000001,"15min":2.5249999999999999},"last":1.1719999999999999},{"interface":"front","average":{"1min":1.5349999999999999,"5min":1.456,"15min":1.5089999999999999},"min":{"1min":1.214,"5min":0.66100000000000003,"15min":0.66100000000000003},"max":{"1min":1.9279999999999999,"5min":2.4500000000000002,"15min":3.609},"last":1.012}]}]},{"osd":1,"up_from":13,"seq":55834575104,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":47412,"kb_used_data":2340,"kb_used_omap":1,"kb_used_meta":45054,"kb_avail":94324428,"statfs":{"total":96636764160,"available":96588214272,"internally_reserved":0,"allocated":2396160,"data_stored":1556377,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1781,"internal_metadata":46135563},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Aug 9 13:54:40 2023","interfaces":[{"interface":"back","average":{"1min":1.018,"5min":0.91500000000000004,"15min":0.93500000000000005},"min":{"1min":0.81399999999999995,"5min":0.45800000000000002,"15min":0.45800000000000002},"max":{"1min":1.262,"5min":1.7110000000000001,"15min":2.1080000000000001},"last":0.94699999999999995},{"interface":"front","average":{"1min":1.0920000000000001,"5min":1.0129999999999999,"15min":1.0860000000000001},"min":{"1min":0.77200000000000002,"5min":0.40699999999999997,"15min":0.40699999999999997},"max":{"1min":1.5109999999999999,"5min":2.5070000000000001,"15min":2.5070000000000001},"last":1.304}]},{"osd":2,"last update":"Wed Aug 9 13:54:40 2023","interfaces":[{"interface":"back","average":{"1min":1.056,"5min":0.97899999999999998,"15min":0.995},"min":{"1min":0.81799999999999995,"5min":0.41999999999999998,"15min":0.41099999999999998},"max":{"1min":1.734,"5min":1.798,"15min":1.907},"last":0.89900000000000002},{"interface":"front","average":{"1min":1.2150000000000001,"5min":1.089,"15min":1.1659999999999999},"min":{"1min":0.82099999999999995,"5min":0.50700000000000001,"15min":0.50700000000000001},"max":{"1min":1.645,"5min":1.645,"15min":2.145},"last":1.212}]},{"osd":3,"last update":"Wed Aug 9 13:54:40 2023","interfaces":[{"interface":"back","average":{"1min":1.391,"5min":1.2190000000000001,"15min":1.3120000000000001},"min":{"1min":0.89900000000000002,"5min":0.54600000000000004,"15min":0.54400000000000004},"max":{"1min":1.748,"5min":1.994,"15min":2.3210000000000002},"last":1.6830000000000001},{"interface":"front","average":{"1min":1.262,"5min":1.1639999999999999,"15min":1.2410000000000001},"min":{"1min":0.873,"5min":0.48299999999999998,"15min":0.45300000000000001},"max":{"1min":1.6830000000000001,"5min":2.3959999999999999,"15min":2.7669999999999999},"last":0.80100000000000005}]},{"osd":4,"last update":"Wed Aug 9 13:54:40 2023","interfaces":[{"interface":"back","average":{"1min":1.4359999999999999,"5min":1.353,"15min":1.409},"min":{"1min":1.0149999999999999,"5min":0.79800000000000004,"15min":0.58099999999999996},"max":{"1min":1.8460000000000001,"5min":2.0739999999999998,"15min":2.4089999999999998},"last":1.085},{"interface":"front","average":{"1min":1.3580000000000001,"5min":1.3160000000000001,"15min":1.3460000000000001},"min":{"1min":1.0740000000000001,"5min":0.71099999999999997,"15min":0.70099999999999996},"max":{"1min":1.77,"5min":2.0019999999999998,"15min":2.0019999999999998},"last":1.5820000000000001}]},{"osd":5,"last update":"Wed Aug 9 13:54:40 2023","interfaces":[{"interface":"back","average":{"1min":1.448,"5min":1.357,"15min":1.3799999999999999},"min":{"1min":1.075,"5min":0.75900000000000001,"15min":0.75900000000000001},"max":{"1min":1.8899999999999999,"5min":1.9419999999999999,"15min":2.294},"last":1.4970000000000001},{"interface":"front","average":{"1min":1.5620000000000001,"5min":1.4299999999999999,"15min":1.506},"min":{"1min":1.2390000000000001,"5min":0.78400000000000003,"15min":0.78400000000000003},"max":{"1min":1.756,"5min":1.863,"15min":2.4649999999999999},"last":1.798}]},{"osd":6,"last update":"Wed Aug 9 13:54:40 2023","interfaces":[{"interface":"back","average":{"1min":1.611,"5min":1.4770000000000001,"15min":1.514},"min":{"1min":1.3069999999999999,"5min":0.96599999999999997,"15min":0.67400000000000004},"max":{"1min":1.9199999999999999,"5min":2.23,"15min":2.5270000000000001},"last":1.53},{"interface":"front","average":{"1min":1.516,"5min":1.4550000000000001,"15min":1.496},"min":{"1min":1.131,"5min":0.73599999999999999,"15min":0.73599999999999999},"max":{"1min":1.748,"5min":2.3359999999999999,"15min":2.3359999999999999},"last":1.6379999999999999}]},{"osd":7,"last update":"Wed Aug 9 13:54:40 2023","interfaces":[{"interface":"back","average":{"1min":1.6799999999999999,"5min":1.5189999999999999,"15min":1.601},"min":{"1min":1.389,"5min":0.72099999999999997,"15min":0.72099999999999997},"max":{"1min":1.8939999999999999,"5min":2.4340000000000002,"15min":2.5790000000000002},"last":1.925},{"interface":"front","average":{"1min":1.651,"5min":1.544,"15min":1.597},"min":{"1min":1.4490000000000001,"5min":0.85099999999999998,"15min":0.85099999999999998},"max":{"1min":2.0110000000000001,"5min":2.375,"15min":2.6509999999999998},"last":1.7410000000000001}]}]},{"osd":0,"up_from":13,"seq":55834575104,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":41068,"kb_used_data":1756,"kb_used_omap":1,"kb_used_meta":39294,"kb_avail":94330772,"statfs":{"total":96636764160,"available":96594710528,"internally_reserved":0,"allocated":1798144,"data_stored":964648,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1781,"internal_metadata":40237323},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Wed Aug 9 13:54:31 2023","interfaces":[{"interface":"back","average":{"1min":0.88300000000000001,"5min":0.91500000000000004,"15min":0.92100000000000004},"min":{"1min":0.31900000000000001,"5min":0.31900000000000001,"15min":0.31900000000000001},"max":{"1min":2.2410000000000001,"5min":2.2410000000000001,"15min":2.2410000000000001},"last":0.80200000000000005},{"interface":"front","average":{"1min":0.93000000000000005,"5min":1.032,"15min":1.0920000000000001},"min":{"1min":0.34499999999999997,"5min":0.34499999999999997,"15min":0.34499999999999997},"max":{"1min":1.6479999999999999,"5min":1.9019999999999999,"15min":1.9019999999999999},"last":1.177}]},{"osd":2,"last update":"Wed Aug 9 13:54:31 2023","interfaces":[{"interface":"back","average":{"1min":1.016,"5min":1.0269999999999999,"15min":1.1100000000000001},"min":{"1min":0.55600000000000005,"5min":0.433,"15min":0.433},"max":{"1min":1.4470000000000001,"5min":1.786,"15min":1.786},"last":1.8799999999999999},{"interface":"front","average":{"1min":1.0269999999999999,"5min":1.099,"15min":1.165},"min":{"1min":0.47599999999999998,"5min":0.439,"15min":0.439},"max":{"1min":1.6000000000000001,"5min":2.0129999999999999,"15min":2.0129999999999999},"last":1.629}]},{"osd":3,"last update":"Wed Aug 9 13:54:31 2023","interfaces":[{"interface":"back","average":{"1min":0.94199999999999995,"5min":1.008,"15min":1.0840000000000001},"min":{"1min":0.44700000000000001,"5min":0.44700000000000001,"15min":0.44700000000000001},"max":{"1min":1.4379999999999999,"5min":1.7529999999999999,"15min":1.881},"last":0.876},{"interface":"front","average":{"1min":1.143,"5min":1.212,"15min":1.2909999999999999},"min":{"1min":0.52400000000000002,"5min":0.50800000000000001,"15min":0.50800000000000001},"max":{"1min":1.577,"5min":2.347,"15min":2.4649999999999999},"last":1.3999999999999999}]},{"osd":4,"last update":"Wed Aug 9 13:54:31 2023","interfaces":[{"interface":"back","average":{"1min":1.28,"5min":1.2909999999999999,"15min":1.333},"min":{"1min":0.76100000000000001,"5min":0.69699999999999995,"15min":0.69699999999999995},"max":{"1min":1.6870000000000001,"5min":1.9630000000000001,"15min":2.2309999999999999},"last":1.677},{"interface":"front","average":{"1min":1.2909999999999999,"5min":1.379,"15min":1.4359999999999999},"min":{"1min":0.871,"5min":0.82699999999999996,"15min":0.77600000000000002},"max":{"1min":1.655,"5min":2.3540000000000001,"15min":2.5390000000000001},"last":2.0150000000000001}]},{"osd":5,"last update":"Wed Aug 9 13:54:31 2023","interfaces":[{"interface":"back","average":{"1min":1.345,"5min":1.4059999999999999,"15min":1.454},"min":{"1min":0.98399999999999999,"5min":0.66300000000000003,"15min":0.66300000000000003},"max":{"1min":1.772,"5min":2.6669999999999998,"15min":2.6669999999999998},"last":1.75},{"interface":"front","average":{"1min":1.3460000000000001,"5min":1.3540000000000001,"15min":1.431},"min":{"1min":0.89900000000000002,"5min":0.70099999999999996,"15min":0.629},"max":{"1min":1.7210000000000001,"5min":1.9710000000000001,"15min":2.1499999999999999},"last":1.272}]},{"osd":6,"last update":"Wed Aug 9 13:54:31 2023","interfaces":[{"interface":"back","average":{"1min":1.371,"5min":1.474,"15min":1.5289999999999999},"min":{"1min":0.83899999999999997,"5min":0.73599999999999999,"15min":0.73599999999999999},"max":{"1min":1.7110000000000001,"5min":2.4670000000000001,"15min":2.7250000000000001},"last":1.7749999999999999},{"interface":"front","average":{"1min":1.448,"5min":1.5269999999999999,"15min":1.5600000000000001},"min":{"1min":1.0169999999999999,"5min":0.73899999999999999,"15min":0.73899999999999999},"max":{"1min":1.881,"5min":2.3199999999999998,"15min":2.399},"last":1.8120000000000001}]},{"osd":7,"last update":"Wed Aug 9 13:54:31 2023","interfaces":[{"interface":"back","average":{"1min":1.419,"5min":1.544,"15min":1.6100000000000001},"min":{"1min":0.90400000000000003,"5min":0.72299999999999998,"15min":0.72299999999999998},"max":{"1min":1.8799999999999999,"5min":2.3900000000000001,"15min":2.3900000000000001},"last":1.5389999999999999},{"interface":"front","average":{"1min":1.3140000000000001,"5min":1.4510000000000001,"15min":1.532},"min":{"1min":0.86599999999999999,"5min":0.79200000000000004,"15min":0.76700000000000002},"max":{"1min":1.905,"5min":2.274,"15min":2.274},"last":1.591}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":18,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":18,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":18,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":616,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":18,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":160,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":18,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":70,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":18,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":112,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":18,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":18,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":2250,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":19,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":19,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":19,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":19,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":19,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":19,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":19,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":19,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-08-09T13:55:34.409 INFO:tasks.cephfs.filesystem.ceph_manager:clean! 2023-08-09T13:55:34.409 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth caps client.0 mon 'allow r' osd 'allow rw tag cephfs data=cephfs' mds allow 2023-08-09T13:55:34.692 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:34.690+0000 7fe7a7fff640 1 Processor -- start 2023-08-09T13:55:34.693 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:34.690+0000 7fe7a7fff640 1 -- start start 2023-08-09T13:55:34.694 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:34.691+0000 7fe7a7fff640 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7fe7a80b7df0 0x7fe7a80d1b30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:55:34.694 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:34.692+0000 7fe7a7fff640 1 -- --> v1:172.21.15.43:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe7a80d16a0 con 0x7fe7a815c2d0 2023-08-09T13:55:34.694 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:34.692+0000 7fe7a7fff640 1 -- --> v1:172.21.15.203:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe7a80d0d70 con 0x7fe7a80cebf0 2023-08-09T13:55:34.694 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:34.692+0000 7fe7a7fff640 1 -- --> v2:172.21.15.203:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe7a80b8230 con 0x7fe7a80b7df0 2023-08-09T13:55:34.695 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:34.692+0000 7fe7a6ffd640 1 --1- >> v1:172.21.15.43:6789/0 conn(0x7fe7a815c2d0 0x7fe7a80cd020 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.43:6789/0 says I am v1:172.21.15.43:56662/0 (socket says 172.21.15.43:56662) 2023-08-09T13:55:34.695 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:34.692+0000 7fe7a6ffd640 1 -- 172.21.15.43:0/1789736811 learned_addr learned my addr 172.21.15.43:0/1789736811 (peer_addr_for_me v1:172.21.15.43:0/0) 2023-08-09T13:55:34.695 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:34.692+0000 7fe7a67fc640 1 --2- 172.21.15.43:0/1789736811 >> v2:172.21.15.203:3300/0 conn(0x7fe7a80b7df0 0x7fe7a80d1b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:55:34.695 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:34.693+0000 7fe7a5ffb640 1 -- 172.21.15.43:0/1789736811 <== mon.0 v1:172.21.15.43:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1510785898 0 0) 0x7fe7a80d16a0 con 0x7fe7a815c2d0 2023-08-09T13:55:34.696 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:34.693+0000 7fe7a5ffb640 1 -- 172.21.15.43:0/1789736811 --> v1:172.21.15.43:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fe794003520 con 0x7fe7a815c2d0 2023-08-09T13:55:34.697 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:34.693+0000 7fe7a5ffb640 1 -- 172.21.15.43:0/1789736811 <== mon.0 v1:172.21.15.43:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3110641675 0 0) 0x7fe794003520 con 0x7fe7a815c2d0 2023-08-09T13:55:34.697 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:34.693+0000 7fe7a5ffb640 1 -- 172.21.15.43:0/1789736811 >> v2:172.21.15.203:3300/0 conn(0x7fe7a80b7df0 msgr2=0x7fe7a80d1b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-09T13:55:34.697 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:34.693+0000 7fe7a5ffb640 1 --2- 172.21.15.43:0/1789736811 >> v2:172.21.15.203:3300/0 conn(0x7fe7a80b7df0 0x7fe7a80d1b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:55:34.698 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:34.693+0000 7fe7a5ffb640 1 -- 172.21.15.43:0/1789736811 >> v1:172.21.15.203:6789/0 conn(0x7fe7a80cebf0 legacy=0x7fe7a80b5270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:55:34.698 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:34.693+0000 7fe7a5ffb640 1 -- 172.21.15.43:0/1789736811 --> v1:172.21.15.43:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe7a80d1050 con 0x7fe7a815c2d0 2023-08-09T13:55:34.698 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:34.693+0000 7fe7a5ffb640 1 -- 172.21.15.43:0/1789736811 <== mon.0 v1:172.21.15.43:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1285150740 0 0) 0x7fe798001670 con 0x7fe7a815c2d0 2023-08-09T13:55:34.698 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:34.693+0000 7fe7a5ffb640 1 -- 172.21.15.43:0/1789736811 >> v1:172.21.15.43:6789/0 conn(0x7fe7a815c2d0 legacy=0x7fe7a80cd020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:55:34.698 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:34.693+0000 7fe7a5ffb640 1 --2- 172.21.15.43:0/1789736811 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fe794003970 0x7fe794003d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:55:34.698 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:34.693+0000 7fe7a5ffb640 1 --2- 172.21.15.43:0/1789736811 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fe7940047c0 0x7fe794004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:55:34.698 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:34.693+0000 7fe7a5ffb640 1 --2- 172.21.15.43:0/1789736811 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fe794005180 0x7fe79400b980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:55:34.698 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:34.693+0000 7fe7a5ffb640 1 -- 172.21.15.43:0/1789736811 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe79400bff0 con 0x7fe7940047c0 2023-08-09T13:55:34.699 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:34.693+0000 7fe7a5ffb640 1 -- 172.21.15.43:0/1789736811 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe79400c160 con 0x7fe794003970 2023-08-09T13:55:34.699 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:34.693+0000 7fe7a5ffb640 1 -- 172.21.15.43:0/1789736811 --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe79400c320 con 0x7fe794005180 2023-08-09T13:55:34.699 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:34.694+0000 7fe7a67fc640 1 --2- 172.21.15.43:0/1789736811 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fe7940047c0 0x7fe794004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:55:34.699 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:34.694+0000 7fe7a77fe640 1 --2- 172.21.15.43:0/1789736811 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fe794005180 0x7fe79400b980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:55:34.699 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:34.694+0000 7fe7a67fc640 1 -- 172.21.15.43:0/1789736811 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fe794005180 msgr2=0x7fe79400b980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-09T13:55:34.699 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:34.694+0000 7fe7a67fc640 1 --2- 172.21.15.43:0/1789736811 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fe794005180 0x7fe79400b980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:55:34.700 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:34.694+0000 7fe7a67fc640 1 -- 172.21.15.43:0/1789736811 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fe794003970 msgr2=0x7fe794003d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-09T13:55:34.700 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:34.694+0000 7fe7a6ffd640 1 --2- 172.21.15.43:0/1789736811 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fe794003970 0x7fe794003d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:55:34.700 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:34.694+0000 7fe7a67fc640 1 --2- 172.21.15.43:0/1789736811 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fe794003970 0x7fe794003d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:55:34.700 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:34.694+0000 7fe7a77fe640 1 -- 172.21.15.43:0/1789736811 reap_dead start 2023-08-09T13:55:34.700 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:34.694+0000 7fe7a67fc640 1 -- 172.21.15.43:0/1789736811 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe794003520 con 0x7fe7940047c0 2023-08-09T13:55:34.700 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:34.694+0000 7fe7a6ffd640 1 --2- 172.21.15.43:0/1789736811 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fe794003970 0x7fe794003d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-08-09T13:55:34.702 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:34.695+0000 7fe7a67fc640 1 --2- 172.21.15.43:0/1789736811 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fe7940047c0 0x7fe794004c40 secure :-1 s=READY pgs=1704 cs=0 l=1 rev1=1 crypto rx=0x7fe790009fc0 tx=0x7fe7940057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=de1a41cfa347ea70 server_cookie=0 in_seq=0 out_seq=0 2023-08-09T13:55:34.702 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:34.696+0000 7fe7a5ffb640 1 -- 172.21.15.43:0/1789736811 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe79002cc20 con 0x7fe7940047c0 2023-08-09T13:55:34.702 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:34.696+0000 7fe7a5ffb640 1 -- 172.21.15.43:0/1789736811 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe79000f040 con 0x7fe7940047c0 2023-08-09T13:55:34.702 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:34.696+0000 7fe7a5ffb640 1 -- 172.21.15.43:0/1789736811 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe790034440 con 0x7fe7940047c0 2023-08-09T13:55:34.702 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:34.696+0000 7fe7a7fff640 1 -- 172.21.15.43:0/1789736811 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fe7940047c0 msgr2=0x7fe794004c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:55:34.702 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:34.696+0000 7fe7a7fff640 1 --2- 172.21.15.43:0/1789736811 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fe7940047c0 0x7fe794004c40 secure :-1 s=READY pgs=1704 cs=0 l=1 rev1=1 crypto rx=0x7fe790009fc0 tx=0x7fe7940057f0 comp rx=0 tx=0).stop 2023-08-09T13:55:34.703 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:34.696+0000 7fe7a7fff640 1 -- 172.21.15.43:0/1789736811 shutdown_connections 2023-08-09T13:55:34.703 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:34.696+0000 7fe7a7fff640 1 --2- 172.21.15.43:0/1789736811 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fe7940047c0 0x7fe794004c40 unknown :-1 s=CLOSED pgs=1704 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:55:34.703 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:34.696+0000 7fe7a7fff640 1 -- 172.21.15.43:0/1789736811 >> 172.21.15.43:0/1789736811 conn(0x7fe7a806a230 msgr2=0x7fe7a806a640 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-09T13:55:34.703 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:34.697+0000 7fe7a7fff640 1 -- 172.21.15.43:0/1789736811 shutdown_connections 2023-08-09T13:55:34.703 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:34.697+0000 7fe7a7fff640 1 -- 172.21.15.43:0/1789736811 wait complete. 2023-08-09T13:55:34.703 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:34.698+0000 7fe7a7fff640 1 Processor -- start 2023-08-09T13:55:34.703 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:34.699+0000 7fe7a7fff640 1 -- start start 2023-08-09T13:55:34.703 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:34.699+0000 7fe7a7fff640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fe7940047c0 0x7fe7a80b4220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:55:34.704 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:34.699+0000 7fe7a7fff640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fe7a8158220 0x7fe7a80c7890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:55:34.704 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:34.699+0000 7fe7a7fff640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fe7a80b5c80 0x7fe7a80b6100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:55:34.704 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:34.699+0000 7fe7a7fff640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe7a80b6640 con 0x7fe7a8158220 2023-08-09T13:55:34.704 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:34.700+0000 7fe7a7fff640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe7a80c7dd0 con 0x7fe7a80b5c80 2023-08-09T13:55:34.704 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:34.700+0000 7fe7a7fff640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe7a81587e0 con 0x7fe7940047c0 2023-08-09T13:55:34.705 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:34.700+0000 7fe7a67fc640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fe7a8158220 0x7fe7a80c7890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:55:34.706 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:34.700+0000 7fe7a67fc640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fe7a8158220 0x7fe7a80c7890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:52648/0 (socket says 172.21.15.43:52648) 2023-08-09T13:55:34.706 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:34.700+0000 7fe7a67fc640 1 -- 172.21.15.43:0/828684862 learned_addr learned my addr 172.21.15.43:0/828684862 (peer_addr_for_me v2:172.21.15.43:0/0) 2023-08-09T13:55:34.706 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:34.700+0000 7fe7a6ffd640 1 --2- 172.21.15.43:0/828684862 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fe7940047c0 0x7fe7a80b4220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:55:34.706 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:34.700+0000 7fe7a67fc640 1 -- 172.21.15.43:0/828684862 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fe7940047c0 msgr2=0x7fe7a80b4220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:55:34.706 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:34.700+0000 7fe7a67fc640 1 --2- 172.21.15.43:0/828684862 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fe7940047c0 0x7fe7a80b4220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:55:34.706 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:34.700+0000 7fe7a67fc640 1 -- 172.21.15.43:0/828684862 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fe7a80b5c80 msgr2=0x7fe7a80b6100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:55:34.706 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:34.700+0000 7fe7a77fe640 1 --2- 172.21.15.43:0/828684862 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fe7a80b5c80 0x7fe7a80b6100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:55:34.707 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:34.700+0000 7fe7a67fc640 1 --2- 172.21.15.43:0/828684862 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fe7a80b5c80 0x7fe7a80b6100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:55:34.707 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:34.700+0000 7fe7a67fc640 1 -- 172.21.15.43:0/828684862 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe7a8158ac0 con 0x7fe7a8158220 2023-08-09T13:55:34.707 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:34.701+0000 7fe7a67fc640 1 --2- 172.21.15.43:0/828684862 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fe7a8158220 0x7fe7a80c7890 secure :-1 s=READY pgs=1705 cs=0 l=1 rev1=1 crypto rx=0x7fe79000e230 tx=0x7fe790009750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-09T13:55:34.707 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:34.701+0000 7fe79ffff640 1 -- 172.21.15.43:0/828684862 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe790034c50 con 0x7fe7a8158220 2023-08-09T13:55:34.707 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:34.701+0000 7fe79ffff640 1 -- 172.21.15.43:0/828684862 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe790034dd0 con 0x7fe7a8158220 2023-08-09T13:55:34.707 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:34.701+0000 7fe79ffff640 1 -- 172.21.15.43:0/828684862 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe790003bd0 con 0x7fe7a8158220 2023-08-09T13:55:34.708 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:34.701+0000 7fe7a7fff640 1 -- 172.21.15.43:0/828684862 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe7a80c9350 con 0x7fe7a8158220 2023-08-09T13:55:34.708 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:34.701+0000 7fe7a7fff640 1 -- 172.21.15.43:0/828684862 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe7a80c9910 con 0x7fe7a8158220 2023-08-09T13:55:34.708 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:34.703+0000 7fe7a7fff640 1 -- 172.21.15.43:0/828684862 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe7a80cc3f0 con 0x7fe7a8158220 2023-08-09T13:55:34.709 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:34.704+0000 7fe79ffff640 1 -- 172.21.15.43:0/828684862 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 67) v1 ==== 88443+0+0 (secure 0 0 0) 0x7fe790040450 con 0x7fe7a8158220 2023-08-09T13:55:34.709 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:34.704+0000 7fe79ffff640 1 --2- 172.21.15.43:0/828684862 >> [v2:172.21.15.203:6832/3490266242,v1:172.21.15.203:6833/3490266242] conn(0x7fe78806a170 0x7fe78806c630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:55:34.709 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:34.705+0000 7fe79ffff640 1 -- 172.21.15.43:0/828684862 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(147..147 src has 1..147) v4 ==== 6426+0+0 (secure 0 0 0) 0x7fe7900ab1d0 con 0x7fe7a8158220 2023-08-09T13:55:34.709 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:34.705+0000 7fe7a6ffd640 1 --2- 172.21.15.43:0/828684862 >> [v2:172.21.15.203:6832/3490266242,v1:172.21.15.203:6833/3490266242] conn(0x7fe78806a170 0x7fe78806c630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:55:34.709 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:34.706+0000 7fe7a6ffd640 1 --2- 172.21.15.43:0/828684862 >> [v2:172.21.15.203:6832/3490266242,v1:172.21.15.203:6833/3490266242] conn(0x7fe78806a170 0x7fe78806c630 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7fe798001200 tx=0x7fe798001110 comp rx=0 tx=0).ready entity=mgr.6659 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-09T13:55:34.711 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:34.709+0000 7fe79ffff640 1 -- 172.21.15.43:0/828684862 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178066 (secure 0 0 0) 0x7fe7900765b0 con 0x7fe7a8158220 2023-08-09T13:55:34.979 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:34.976+0000 7fe7a7fff640 1 -- 172.21.15.43:0/828684862 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]} v 0) v1 -- 0x7fe7a80c9aa0 con 0x7fe7a8158220 2023-08-09T13:55:34.987 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:34.984+0000 7fe79ffff640 1 -- 172.21.15.43:0/828684862 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]}]=0 updated caps for client.0 v20) v1 ==== 190+0+147 (secure 0 0 0) 0x7fe790079bd0 con 0x7fe7a8158220 2023-08-09T13:55:34.987 INFO:teuthology.orchestra.run.smithi043.stdout:[client.0] 2023-08-09T13:55:34.987 INFO:teuthology.orchestra.run.smithi043.stdout: key = AQCPldNkg8zrDxAAM/hnJaE6S9HU/fRwQeBV5w== 2023-08-09T13:55:34.988 INFO:teuthology.orchestra.run.smithi043.stdout: caps mds = "allow" 2023-08-09T13:55:34.988 INFO:teuthology.orchestra.run.smithi043.stdout: caps mon = "allow r" 2023-08-09T13:55:34.988 INFO:teuthology.orchestra.run.smithi043.stdout: caps osd = "allow rw tag cephfs data=cephfs" 2023-08-09T13:55:34.988 INFO:teuthology.orchestra.run.smithi043.stderr:updated caps for client.0 2023-08-09T13:55:34.989 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:34.987+0000 7fe7a7fff640 1 -- 172.21.15.43:0/828684862 >> [v2:172.21.15.203:6832/3490266242,v1:172.21.15.203:6833/3490266242] conn(0x7fe78806a170 msgr2=0x7fe78806c630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:55:34.989 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:34.987+0000 7fe7a7fff640 1 --2- 172.21.15.43:0/828684862 >> [v2:172.21.15.203:6832/3490266242,v1:172.21.15.203:6833/3490266242] conn(0x7fe78806a170 0x7fe78806c630 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7fe798001200 tx=0x7fe798001110 comp rx=0 tx=0).stop 2023-08-09T13:55:34.990 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:34.987+0000 7fe7a7fff640 1 -- 172.21.15.43:0/828684862 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fe7a8158220 msgr2=0x7fe7a80c7890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:55:34.990 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:34.987+0000 7fe7a7fff640 1 --2- 172.21.15.43:0/828684862 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fe7a8158220 0x7fe7a80c7890 secure :-1 s=READY pgs=1705 cs=0 l=1 rev1=1 crypto rx=0x7fe79000e230 tx=0x7fe790009750 comp rx=0 tx=0).stop 2023-08-09T13:55:34.990 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:34.987+0000 7fe7a7fff640 1 -- 172.21.15.43:0/828684862 shutdown_connections 2023-08-09T13:55:34.991 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:34.987+0000 7fe7a7fff640 1 --2- 172.21.15.43:0/828684862 >> [v2:172.21.15.203:6832/3490266242,v1:172.21.15.203:6833/3490266242] conn(0x7fe78806a170 0x7fe78806c630 unknown :-1 s=CLOSED pgs=66 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:55:34.991 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:34.987+0000 7fe7a7fff640 1 --2- 172.21.15.43:0/828684862 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fe7a80b5c80 0x7fe7a80b6100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:55:34.991 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:34.987+0000 7fe7a7fff640 1 --2- 172.21.15.43:0/828684862 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fe7a8158220 0x7fe7a80c7890 unknown :-1 s=CLOSED pgs=1705 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:55:34.991 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:34.987+0000 7fe7a7fff640 1 --2- 172.21.15.43:0/828684862 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fe7940047c0 0x7fe7a80b4220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:55:34.991 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:34.987+0000 7fe7a7fff640 1 -- 172.21.15.43:0/828684862 >> 172.21.15.43:0/828684862 conn(0x7fe7a806a230 msgr2=0x7fe7a80cfd60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-09T13:55:34.991 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:34.988+0000 7fe7a7fff640 1 -- 172.21.15.43:0/828684862 shutdown_connections 2023-08-09T13:55:34.991 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:34.988+0000 7fe7a7fff640 1 -- 172.21.15.43:0/828684862 wait complete. 2023-08-09T13:55:35.010 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-08-09T13:55:35.236 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:35.234+0000 7fe0aa0f5640 1 Processor -- start 2023-08-09T13:55:35.237 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:35.234+0000 7fe0aa0f5640 1 -- start start 2023-08-09T13:55:35.238 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:35.235+0000 7fe0aa0f5640 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7fe0a40b1050 0x7fe0a40b4760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:55:35.238 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:35.236+0000 7fe0aa0f5640 1 -- --> v1:172.21.15.43:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe0a40b7bf0 con 0x7fe0a415f7b0 2023-08-09T13:55:35.238 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:35.236+0000 7fe0aa0f5640 1 -- --> v1:172.21.15.203:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe0a40b7ed0 con 0x7fe0a40cd6b0 2023-08-09T13:55:35.239 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:35.236+0000 7fe0aa0f5640 1 -- --> v2:172.21.15.203:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe0a40d0370 con 0x7fe0a40b1050 2023-08-09T13:55:35.239 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:35.236+0000 7fe0a90f3640 1 --1- >> v1:172.21.15.43:6789/0 conn(0x7fe0a415f7b0 0x7fe0a40ccfa0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.43:6789/0 says I am v1:172.21.15.43:56842/0 (socket says 172.21.15.43:56842) 2023-08-09T13:55:35.239 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:35.236+0000 7fe0a90f3640 1 -- 172.21.15.43:0/579194984 learned_addr learned my addr 172.21.15.43:0/579194984 (peer_addr_for_me v1:172.21.15.43:0/0) 2023-08-09T13:55:35.241 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:35.236+0000 7fe0a88f2640 1 --2- 172.21.15.43:0/579194984 >> v2:172.21.15.203:3300/0 conn(0x7fe0a40b1050 0x7fe0a40b4760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:55:35.241 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:35.236+0000 7fe09bfff640 1 -- 172.21.15.43:0/579194984 <== mon.0 v1:172.21.15.43:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1511747821 0 0) 0x7fe0a40b7bf0 con 0x7fe0a415f7b0 2023-08-09T13:55:35.241 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:35.236+0000 7fe09bfff640 1 -- 172.21.15.43:0/579194984 --> v1:172.21.15.43:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fe090003520 con 0x7fe0a415f7b0 2023-08-09T13:55:35.242 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:35.237+0000 7fe09bfff640 1 -- 172.21.15.43:0/579194984 <== mon.0 v1:172.21.15.43:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 669028335 0 0) 0x7fe090003520 con 0x7fe0a415f7b0 2023-08-09T13:55:35.242 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:35.237+0000 7fe09bfff640 1 -- 172.21.15.43:0/579194984 >> v2:172.21.15.203:3300/0 conn(0x7fe0a40b1050 msgr2=0x7fe0a40b4760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-09T13:55:35.242 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:35.237+0000 7fe09bfff640 1 --2- 172.21.15.43:0/579194984 >> v2:172.21.15.203:3300/0 conn(0x7fe0a40b1050 0x7fe0a40b4760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:55:35.242 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:35.237+0000 7fe09bfff640 1 -- 172.21.15.43:0/579194984 >> v1:172.21.15.203:6789/0 conn(0x7fe0a40cd6b0 legacy=0x7fe0a40d0e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:55:35.242 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:35.237+0000 7fe09bfff640 1 -- 172.21.15.43:0/579194984 --> v1:172.21.15.43:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe0a40d0650 con 0x7fe0a415f7b0 2023-08-09T13:55:35.242 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:35.237+0000 7fe09bfff640 1 -- 172.21.15.43:0/579194984 <== mon.0 v1:172.21.15.43:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1285150740 0 0) 0x7fe0a0001670 con 0x7fe0a415f7b0 2023-08-09T13:55:35.242 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:35.237+0000 7fe09bfff640 1 -- 172.21.15.43:0/579194984 >> v1:172.21.15.43:6789/0 conn(0x7fe0a415f7b0 legacy=0x7fe0a40ccfa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:55:35.243 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:35.237+0000 7fe09bfff640 1 --2- 172.21.15.43:0/579194984 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fe090003970 0x7fe090003d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:55:35.243 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:35.237+0000 7fe09bfff640 1 --2- 172.21.15.43:0/579194984 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fe0900047c0 0x7fe090004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:55:35.243 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:35.237+0000 7fe09bfff640 1 --2- 172.21.15.43:0/579194984 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fe090005180 0x7fe09000b980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:55:35.243 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:35.237+0000 7fe09bfff640 1 -- 172.21.15.43:0/579194984 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe0a40d0370 con 0x7fe090005180 2023-08-09T13:55:35.243 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:35.237+0000 7fe09bfff640 1 -- 172.21.15.43:0/579194984 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe09000c020 con 0x7fe0900047c0 2023-08-09T13:55:35.243 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:35.237+0000 7fe09bfff640 1 -- 172.21.15.43:0/579194984 --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe09000c1e0 con 0x7fe090003970 2023-08-09T13:55:35.243 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:35.237+0000 7fe0a98f4640 1 --2- 172.21.15.43:0/579194984 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fe090005180 0x7fe09000b980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:55:35.244 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:35.238+0000 7fe0a88f2640 1 --2- 172.21.15.43:0/579194984 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fe0900047c0 0x7fe090004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:55:35.244 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:35.238+0000 7fe0a90f3640 1 --2- 172.21.15.43:0/579194984 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fe090003970 0x7fe090003d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:55:35.244 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:35.238+0000 7fe0a98f4640 1 -- 172.21.15.43:0/579194984 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fe090003970 msgr2=0x7fe090003d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-09T13:55:35.244 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:35.238+0000 7fe0a98f4640 1 --2- 172.21.15.43:0/579194984 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fe090003970 0x7fe090003d90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:55:35.244 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:35.238+0000 7fe0a98f4640 1 -- 172.21.15.43:0/579194984 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fe0900047c0 msgr2=0x7fe090004c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-09T13:55:35.244 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:35.238+0000 7fe0a98f4640 1 --2- 172.21.15.43:0/579194984 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fe0900047c0 0x7fe090004c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:55:35.245 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:35.238+0000 7fe0a98f4640 1 -- 172.21.15.43:0/579194984 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe090003520 con 0x7fe090005180 2023-08-09T13:55:35.245 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:35.239+0000 7fe0a98f4640 1 -- 172.21.15.43:0/579194984 reap_dead start 2023-08-09T13:55:35.245 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:35.239+0000 7fe0a98f4640 1 --2- 172.21.15.43:0/579194984 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fe090005180 0x7fe09000b980 secure :-1 s=READY pgs=1707 cs=0 l=1 rev1=1 crypto rx=0x7fe09c001e20 tx=0x7fe0900057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3feab255931a53b9 server_cookie=0 in_seq=0 out_seq=0 2023-08-09T13:55:35.247 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:35.239+0000 7fe09bfff640 1 -- 172.21.15.43:0/579194984 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe09c003b70 con 0x7fe090005180 2023-08-09T13:55:35.248 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:35.239+0000 7fe09bfff640 1 -- 172.21.15.43:0/579194984 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe09c003cd0 con 0x7fe090005180 2023-08-09T13:55:35.248 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:35.239+0000 7fe09bfff640 1 -- 172.21.15.43:0/579194984 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe09c005310 con 0x7fe090005180 2023-08-09T13:55:35.248 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:35.240+0000 7fe0aa0f5640 1 -- 172.21.15.43:0/579194984 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fe090005180 msgr2=0x7fe09000b980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:55:35.248 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:35.240+0000 7fe0aa0f5640 1 --2- 172.21.15.43:0/579194984 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fe090005180 0x7fe09000b980 secure :-1 s=READY pgs=1707 cs=0 l=1 rev1=1 crypto rx=0x7fe09c001e20 tx=0x7fe0900057f0 comp rx=0 tx=0).stop 2023-08-09T13:55:35.248 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:35.240+0000 7fe0aa0f5640 1 -- 172.21.15.43:0/579194984 shutdown_connections 2023-08-09T13:55:35.248 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:35.240+0000 7fe0aa0f5640 1 --2- 172.21.15.43:0/579194984 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fe090005180 0x7fe09000b980 unknown :-1 s=CLOSED pgs=1707 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:55:35.249 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:35.240+0000 7fe0aa0f5640 1 -- 172.21.15.43:0/579194984 >> 172.21.15.43:0/579194984 conn(0x7fe0a406a2b0 msgr2=0x7fe0a406a690 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-09T13:55:35.249 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:35.240+0000 7fe0aa0f5640 1 -- 172.21.15.43:0/579194984 shutdown_connections 2023-08-09T13:55:35.249 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:35.240+0000 7fe0aa0f5640 1 -- 172.21.15.43:0/579194984 wait complete. 2023-08-09T13:55:35.249 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:35.242+0000 7fe0aa0f5640 1 Processor -- start 2023-08-09T13:55:35.249 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:35.242+0000 7fe0aa0f5640 1 -- start start 2023-08-09T13:55:35.249 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:35.243+0000 7fe0aa0f5640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fe090005180 0x7fe0a4159df0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:55:35.249 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:35.243+0000 7fe0aa0f5640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fe0a40c93a0 0x7fe0a41f5050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:55:35.250 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:35.243+0000 7fe0aa0f5640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fe0a41f5590 0x7fe0a4157440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:55:35.250 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:35.243+0000 7fe0aa0f5640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe0a4154030 con 0x7fe0a40c93a0 2023-08-09T13:55:35.250 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:35.243+0000 7fe0aa0f5640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe0a41f5a10 con 0x7fe0a41f5590 2023-08-09T13:55:35.250 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:35.243+0000 7fe0aa0f5640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe0a40b3220 con 0x7fe090005180 2023-08-09T13:55:35.250 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:35.243+0000 7fe0a88f2640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fe0a40c93a0 0x7fe0a41f5050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:55:35.250 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:35.243+0000 7fe0a88f2640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fe0a40c93a0 0x7fe0a41f5050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:53246/0 (socket says 172.21.15.43:53246) 2023-08-09T13:55:35.250 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:35.243+0000 7fe0a88f2640 1 -- 172.21.15.43:0/3599607651 learned_addr learned my addr 172.21.15.43:0/3599607651 (peer_addr_for_me v2:172.21.15.43:0/0) 2023-08-09T13:55:35.251 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:35.243+0000 7fe0a98f4640 1 --2- 172.21.15.43:0/3599607651 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fe0a41f5590 0x7fe0a4157440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:55:35.251 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:35.243+0000 7fe0a88f2640 1 -- 172.21.15.43:0/3599607651 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fe090005180 msgr2=0x7fe0a4159df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:55:35.251 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:35.243+0000 7fe0a90f3640 1 --2- 172.21.15.43:0/3599607651 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fe090005180 0x7fe0a4159df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:55:35.251 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:35.244+0000 7fe0a88f2640 1 --2- 172.21.15.43:0/3599607651 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fe090005180 0x7fe0a4159df0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:55:35.251 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:35.244+0000 7fe0a88f2640 1 -- 172.21.15.43:0/3599607651 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fe0a41f5590 msgr2=0x7fe0a4157440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:55:35.251 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:35.244+0000 7fe0a88f2640 1 --2- 172.21.15.43:0/3599607651 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fe0a41f5590 0x7fe0a4157440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:55:35.251 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:35.244+0000 7fe0a88f2640 1 -- 172.21.15.43:0/3599607651 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe0a40c9820 con 0x7fe0a40c93a0 2023-08-09T13:55:35.251 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:35.244+0000 7fe0a98f4640 1 --2- 172.21.15.43:0/3599607651 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fe0a41f5590 0x7fe0a4157440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-08-09T13:55:35.252 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:35.244+0000 7fe0a88f2640 1 --2- 172.21.15.43:0/3599607651 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fe0a40c93a0 0x7fe0a41f5050 secure :-1 s=READY pgs=1708 cs=0 l=1 rev1=1 crypto rx=0x7fe094011560 tx=0x7fe094009f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-09T13:55:35.252 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:35.244+0000 7fe099ffb640 1 -- 172.21.15.43:0/3599607651 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe094015930 con 0x7fe0a40c93a0 2023-08-09T13:55:35.252 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:35.244+0000 7fe099ffb640 1 -- 172.21.15.43:0/3599607651 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe094015a90 con 0x7fe0a40c93a0 2023-08-09T13:55:35.252 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:35.244+0000 7fe099ffb640 1 -- 172.21.15.43:0/3599607651 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe094015e40 con 0x7fe0a40c93a0 2023-08-09T13:55:35.252 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:35.244+0000 7fe0aa0f5640 1 -- 172.21.15.43:0/3599607651 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe0a40b2610 con 0x7fe0a40c93a0 2023-08-09T13:55:35.252 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:35.245+0000 7fe0aa0f5640 1 -- 172.21.15.43:0/3599607651 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe0a40b2bd0 con 0x7fe0a40c93a0 2023-08-09T13:55:35.253 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:35.246+0000 7fe0aa0f5640 1 -- 172.21.15.43:0/3599607651 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe0a40739f0 con 0x7fe0a40c93a0 2023-08-09T13:55:35.253 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:35.247+0000 7fe099ffb640 1 -- 172.21.15.43:0/3599607651 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 67) v1 ==== 88443+0+0 (secure 0 0 0) 0x7fe09401d400 con 0x7fe0a40c93a0 2023-08-09T13:55:35.253 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:35.248+0000 7fe099ffb640 1 --2- 172.21.15.43:0/3599607651 >> [v2:172.21.15.203:6832/3490266242,v1:172.21.15.203:6833/3490266242] conn(0x7fe08806a170 0x7fe08806c630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:55:35.254 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:35.248+0000 7fe099ffb640 1 -- 172.21.15.43:0/3599607651 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(147..147 src has 1..147) v4 ==== 6426+0+0 (secure 0 0 0) 0x7fe094089010 con 0x7fe0a40c93a0 2023-08-09T13:55:35.254 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:35.249+0000 7fe0a90f3640 1 --2- 172.21.15.43:0/3599607651 >> [v2:172.21.15.203:6832/3490266242,v1:172.21.15.203:6833/3490266242] conn(0x7fe08806a170 0x7fe08806c630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:55:35.254 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:35.250+0000 7fe0a90f3640 1 --2- 172.21.15.43:0/3599607651 >> [v2:172.21.15.203:6832/3490266242,v1:172.21.15.203:6833/3490266242] conn(0x7fe08806a170 0x7fe08806c630 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7fe0a0001eb0 tx=0x7fe0a0009040 comp rx=0 tx=0).ready entity=mgr.6659 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-09T13:55:35.255 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:35.253+0000 7fe099ffb640 1 -- 172.21.15.43:0/3599607651 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178066 (secure 0 0 0) 0x7fe094054510 con 0x7fe0a40c93a0 2023-08-09T13:55:35.379 DEBUG:teuthology.orchestra.run.smithi043:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-08-09T13:55:35.388 DEBUG:teuthology.orchestra.run.smithi203:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-08-09T13:55:35.489 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:35.487+0000 7fe0aa0f5640 1 -- 172.21.15.43:0/3599607651 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fe0a415c910 con 0x7fe0a40c93a0 2023-08-09T13:55:35.490 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:35.488+0000 7fe099ffb640 1 -- 172.21.15.43:0/3599607651 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 70 v70) v1 ==== 94+0+4883 (secure 0 0 0) 0x7fe09405b330 con 0x7fe0a40c93a0 2023-08-09T13:55:35.491 INFO:teuthology.orchestra.run.smithi043.stdout: 2023-08-09T13:55:35.491 INFO:teuthology.orchestra.run.smithi043.stdout:{"epoch":70,"default_fscid":8,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6812,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.203:6835/2100579784","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6834","nonce":2100579784},{"type":"v1","addr":"172.21.15.203:6835","nonce":2100579784}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":65},{"gid":6815,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.203:6837/1689128061","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6836","nonce":1689128061},{"type":"v1","addr":"172.21.15.203:6837","nonce":1689128061}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":65},{"gid":7543,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.43:6835/1126540259","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6833","nonce":1126540259},{"type":"v1","addr":"172.21.15.43:6835","nonce":1126540259}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":65}],"filesystems":[{"mdsmap":{"epoch":70,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-08-09T13:55:28.794042+0000","modified":"2023-08-09T13:55:32.388404+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":9126},"failed":[],"damaged":[],"stopped":[],"info":{"gid_9126":{"gid":9126,"name":"a","rank":0,"incarnation":66,"state":"up:active","state_seq":3,"addr":"172.21.15.43:6834/2023142010","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6832","nonce":2023142010},{"type":"v1","addr":"172.21.15.43:6834","nonce":2023142010}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[19],"metadata_pool":18,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":8}]} 2023-08-09T13:55:35.491 INFO:teuthology.orchestra.run.smithi043.stderr:dumped fsmap epoch 70 2023-08-09T13:55:35.493 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:35.491+0000 7fe0aa0f5640 1 -- 172.21.15.43:0/3599607651 >> [v2:172.21.15.203:6832/3490266242,v1:172.21.15.203:6833/3490266242] conn(0x7fe08806a170 msgr2=0x7fe08806c630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:55:35.493 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:35.491+0000 7fe0aa0f5640 1 --2- 172.21.15.43:0/3599607651 >> [v2:172.21.15.203:6832/3490266242,v1:172.21.15.203:6833/3490266242] conn(0x7fe08806a170 0x7fe08806c630 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7fe0a0001eb0 tx=0x7fe0a0009040 comp rx=0 tx=0).stop 2023-08-09T13:55:35.494 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:35.491+0000 7fe0aa0f5640 1 -- 172.21.15.43:0/3599607651 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fe0a40c93a0 msgr2=0x7fe0a41f5050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:55:35.494 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:35.491+0000 7fe0aa0f5640 1 --2- 172.21.15.43:0/3599607651 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fe0a40c93a0 0x7fe0a41f5050 secure :-1 s=READY pgs=1708 cs=0 l=1 rev1=1 crypto rx=0x7fe094011560 tx=0x7fe094009f90 comp rx=0 tx=0).stop 2023-08-09T13:55:35.494 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:35.491+0000 7fe0aa0f5640 1 -- 172.21.15.43:0/3599607651 shutdown_connections 2023-08-09T13:55:35.494 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:35.491+0000 7fe0aa0f5640 1 --2- 172.21.15.43:0/3599607651 >> [v2:172.21.15.203:6832/3490266242,v1:172.21.15.203:6833/3490266242] conn(0x7fe08806a170 0x7fe08806c630 unknown :-1 s=CLOSED pgs=67 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:55:35.494 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:35.491+0000 7fe0aa0f5640 1 --2- 172.21.15.43:0/3599607651 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fe0a41f5590 0x7fe0a4157440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:55:35.494 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:35.491+0000 7fe0aa0f5640 1 --2- 172.21.15.43:0/3599607651 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fe0a40c93a0 0x7fe0a41f5050 unknown :-1 s=CLOSED pgs=1708 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:55:35.494 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:35.491+0000 7fe0aa0f5640 1 --2- 172.21.15.43:0/3599607651 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fe090005180 0x7fe0a4159df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:55:35.495 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:35.491+0000 7fe0aa0f5640 1 -- 172.21.15.43:0/3599607651 >> 172.21.15.43:0/3599607651 conn(0x7fe0a406a2b0 msgr2=0x7fe0a40b4710 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-09T13:55:35.495 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:35.491+0000 7fe0aa0f5640 1 -- 172.21.15.43:0/3599607651 shutdown_connections 2023-08-09T13:55:35.495 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:35.491+0000 7fe0aa0f5640 1 -- 172.21.15.43:0/3599607651 wait complete. 2023-08-09T13:55:35.917 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 70, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2023-08-09T13:55:28.794042+0000', 'modified': '2023-08-09T13:55:32.388404+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 9126}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_9126': {'gid': 9126, 'name': 'a', 'rank': 0, 'incarnation': 66, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.43:6834/2023142010', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.43:6832', 'nonce': 2023142010}, {'type': 'v1', 'addr': '172.21.15.43:6834', 'nonce': 2023142010}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138322906710015, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}}, 'data_pools': [19], 'metadata_pool': 18, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0} 2023-08-09T13:55:35.917 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2023-08-09T13:55:35.918 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.a status 2023-08-09T13:55:36.162 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:36.159+0000 7fcb61f42640 1 Processor -- start 2023-08-09T13:55:36.162 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:36.160+0000 7fcb61f42640 1 -- start start 2023-08-09T13:55:36.163 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:36.161+0000 7fcb61f42640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fcb5c15c680 0x7fcb5c0ce760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:55:36.164 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:36.161+0000 7fcb61f42640 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7fcb5c0b3fa0 0x7fcb5c0b43d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:55:36.164 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:36.161+0000 7fcb61f42640 1 -- --> v1:172.21.15.203:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fcb5c0b4910 con 0x7fcb5c0d0a70 2023-08-09T13:55:36.164 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:36.161+0000 7fcb61f42640 1 -- --> v2:172.21.15.203:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fcb5c0cdf00 con 0x7fcb5c0b3fa0 2023-08-09T13:55:36.164 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:36.161+0000 7fcb61f42640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcb5c0d0f70 con 0x7fcb5c15c680 2023-08-09T13:55:36.164 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:36.162+0000 7fcb5bfff640 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7fcb5c0b3fa0 0x7fcb5c0b43d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:55:36.165 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:36.162+0000 7fcb5b7fe640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fcb5c15c680 0x7fcb5c0ce760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:55:36.165 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:36.162+0000 7fcb5affd640 1 --1- >> v1:172.21.15.203:6789/0 conn(0x7fcb5c0d0a70 0x7fcb5c0cd7f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.203:6789/0 says I am v1:172.21.15.43:36372/0 (socket says 172.21.15.43:36372) 2023-08-09T13:55:36.165 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:36.162+0000 7fcb5affd640 1 -- 172.21.15.43:0/891161072 learned_addr learned my addr 172.21.15.43:0/891161072 (peer_addr_for_me v1:172.21.15.43:0/0) 2023-08-09T13:55:36.166 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:36.163+0000 7fcb5bfff640 1 -- 172.21.15.43:0/891161072 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fcb5c15c680 msgr2=0x7fcb5c0ce760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-09T13:55:36.166 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:36.163+0000 7fcb5bfff640 1 --2- 172.21.15.43:0/891161072 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fcb5c15c680 0x7fcb5c0ce760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:55:36.166 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:36.163+0000 7fcb5bfff640 1 -- 172.21.15.43:0/891161072 >> v1:172.21.15.203:6789/0 conn(0x7fcb5c0d0a70 legacy=0x7fcb5c0cd7f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:55:36.166 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:36.163+0000 7fcb5a7fc640 1 -- 172.21.15.43:0/891161072 <== mon.1 v1:172.21.15.203:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1044710834 0 0) 0x7fcb5c0b4910 con 0x7fcb5c0d0a70 2023-08-09T13:55:36.166 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:36.163+0000 7fcb5bfff640 1 -- 172.21.15.43:0/891161072 --> v2:172.21.15.203:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcb5c0ccd10 con 0x7fcb5c0b3fa0 2023-08-09T13:55:36.166 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:36.163+0000 7fcb5b7fe640 1 --2- 172.21.15.43:0/891161072 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fcb5c15c680 0x7fcb5c0ce760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-08-09T13:55:36.167 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:36.165+0000 7fcb5bfff640 1 --2- 172.21.15.43:0/891161072 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fcb5c0b3fa0 0x7fcb5c0b43d0 secure :-1 s=READY pgs=628 cs=0 l=1 rev1=1 crypto rx=0x7fcb54009d90 tx=0x7fcb5402bea0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=57c26eb45cdbbe4b server_cookie=0 in_seq=0 out_seq=0 2023-08-09T13:55:36.168 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:36.165+0000 7fcb5a7fc640 1 -- 172.21.15.43:0/891161072 <== mon.1 v2:172.21.15.203:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcb5402cc50 con 0x7fcb5c0b3fa0 2023-08-09T13:55:36.168 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:36.166+0000 7fcb5a7fc640 1 -- 172.21.15.43:0/891161072 <== mon.1 v2:172.21.15.203:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcb5402cdb0 con 0x7fcb5c0b3fa0 2023-08-09T13:55:36.168 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:36.166+0000 7fcb5a7fc640 1 -- 172.21.15.43:0/891161072 <== mon.1 v2:172.21.15.203:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcb54034660 con 0x7fcb5c0b3fa0 2023-08-09T13:55:36.169 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:36.166+0000 7fcb61f42640 1 -- 172.21.15.43:0/891161072 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fcb5c0b3fa0 msgr2=0x7fcb5c0b43d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:55:36.169 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:36.166+0000 7fcb61f42640 1 --2- 172.21.15.43:0/891161072 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fcb5c0b3fa0 0x7fcb5c0b43d0 secure :-1 s=READY pgs=628 cs=0 l=1 rev1=1 crypto rx=0x7fcb54009d90 tx=0x7fcb5402bea0 comp rx=0 tx=0).stop 2023-08-09T13:55:36.169 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:36.166+0000 7fcb61f42640 1 -- 172.21.15.43:0/891161072 shutdown_connections 2023-08-09T13:55:36.169 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:36.166+0000 7fcb61f42640 1 --2- 172.21.15.43:0/891161072 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fcb5c0b3fa0 0x7fcb5c0b43d0 unknown :-1 s=CLOSED pgs=628 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:55:36.170 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:36.166+0000 7fcb61f42640 1 --2- 172.21.15.43:0/891161072 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fcb5c15c680 0x7fcb5c0ce760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:55:36.170 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:36.166+0000 7fcb61f42640 1 -- 172.21.15.43:0/891161072 >> 172.21.15.43:0/891161072 conn(0x7fcb5c071520 msgr2=0x7fcb5c0d2520 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-09T13:55:36.170 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:36.166+0000 7fcb61f42640 1 -- 172.21.15.43:0/891161072 shutdown_connections 2023-08-09T13:55:36.170 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:36.166+0000 7fcb61f42640 1 -- 172.21.15.43:0/891161072 wait complete. 2023-08-09T13:55:36.170 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:36.168+0000 7fcb61f42640 1 Processor -- start 2023-08-09T13:55:36.171 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:36.168+0000 7fcb61f42640 1 -- start start 2023-08-09T13:55:36.171 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:36.169+0000 7fcb61f42640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fcb5c0b3fa0 0x7fcb5c0c8180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:55:36.172 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:36.169+0000 7fcb61f42640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fcb5c0d0a70 0x7fcb5c157800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:55:36.172 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:36.169+0000 7fcb61f42640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fcb5c15c680 0x7fcb5c157d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:55:36.172 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:36.169+0000 7fcb61f42640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcb5c0c86c0 con 0x7fcb5c15c680 2023-08-09T13:55:36.172 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:36.169+0000 7fcb61f42640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcb5c1550b0 con 0x7fcb5c0d0a70 2023-08-09T13:55:36.172 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:36.169+0000 7fcb61f42640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcb5c155390 con 0x7fcb5c0b3fa0 2023-08-09T13:55:36.173 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:36.170+0000 7fcb5bfff640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fcb5c15c680 0x7fcb5c157d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:55:36.174 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:36.170+0000 7fcb5bfff640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fcb5c15c680 0x7fcb5c157d40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:53260/0 (socket says 172.21.15.43:53260) 2023-08-09T13:55:36.174 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:36.170+0000 7fcb5bfff640 1 -- 172.21.15.43:0/768276650 learned_addr learned my addr 172.21.15.43:0/768276650 (peer_addr_for_me v2:172.21.15.43:0/0) 2023-08-09T13:55:36.174 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:36.170+0000 7fcb5b7fe640 1 --2- 172.21.15.43:0/768276650 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fcb5c0b3fa0 0x7fcb5c0c8180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:55:36.174 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:36.170+0000 7fcb5affd640 1 --2- 172.21.15.43:0/768276650 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fcb5c0d0a70 0x7fcb5c157800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:55:36.174 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:36.170+0000 7fcb5bfff640 1 -- 172.21.15.43:0/768276650 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fcb5c0b3fa0 msgr2=0x7fcb5c0c8180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:55:36.174 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:36.170+0000 7fcb5bfff640 1 --2- 172.21.15.43:0/768276650 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fcb5c0b3fa0 0x7fcb5c0c8180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:55:36.174 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:36.170+0000 7fcb5bfff640 1 -- 172.21.15.43:0/768276650 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fcb5c0d0a70 msgr2=0x7fcb5c157800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:55:36.174 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:36.170+0000 7fcb5bfff640 1 --2- 172.21.15.43:0/768276650 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fcb5c0d0a70 0x7fcb5c157800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:55:36.175 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:36.170+0000 7fcb5bfff640 1 -- 172.21.15.43:0/768276650 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcb4c009400 con 0x7fcb5c15c680 2023-08-09T13:55:36.175 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:36.170+0000 7fcb5bfff640 1 --2- 172.21.15.43:0/768276650 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fcb5c15c680 0x7fcb5c157d40 secure :-1 s=READY pgs=1709 cs=0 l=1 rev1=1 crypto rx=0x7fcb54009d90 tx=0x7fcb54004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-09T13:55:36.175 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:36.171+0000 7fcb43fff640 1 -- 172.21.15.43:0/768276650 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcb54040650 con 0x7fcb5c15c680 2023-08-09T13:55:36.175 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:36.171+0000 7fcb43fff640 1 -- 172.21.15.43:0/768276650 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcb540407b0 con 0x7fcb5c15c680 2023-08-09T13:55:36.175 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:36.171+0000 7fcb61f42640 1 -- 172.21.15.43:0/768276650 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcb54009a70 con 0x7fcb5c15c680 2023-08-09T13:55:36.175 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:36.171+0000 7fcb43fff640 1 -- 172.21.15.43:0/768276650 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcb54040b60 con 0x7fcb5c15c680 2023-08-09T13:55:36.176 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:36.171+0000 7fcb61f42640 1 -- 172.21.15.43:0/768276650 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcb5c144540 con 0x7fcb5c15c680 2023-08-09T13:55:36.176 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:36.173+0000 7fcb43fff640 1 -- 172.21.15.43:0/768276650 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 67) v1 ==== 88443+0+0 (secure 0 0 0) 0x7fcb54040cc0 con 0x7fcb5c15c680 2023-08-09T13:55:36.176 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:36.174+0000 7fcb43fff640 1 --2- 172.21.15.43:0/768276650 >> [v2:172.21.15.203:6832/3490266242,v1:172.21.15.203:6833/3490266242] conn(0x7fcb4806a170 0x7fcb4806c630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:55:36.177 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:36.174+0000 7fcb43fff640 1 -- 172.21.15.43:0/768276650 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(147..147 src has 1..147) v4 ==== 6426+0+0 (secure 0 0 0) 0x7fcb540ab700 con 0x7fcb5c15c680 2023-08-09T13:55:36.177 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:36.175+0000 7fcb5b7fe640 1 --2- 172.21.15.43:0/768276650 >> [v2:172.21.15.203:6832/3490266242,v1:172.21.15.203:6833/3490266242] conn(0x7fcb4806a170 0x7fcb4806c630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:55:36.177 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:36.175+0000 7fcb64b85740 1 Processor -- start 2023-08-09T13:55:36.178 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:36.175+0000 7fcb64b85740 1 -- start start 2023-08-09T13:55:36.180 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:36.176+0000 7fcb5b7fe640 1 --2- 172.21.15.43:0/768276650 >> [v2:172.21.15.203:6832/3490266242,v1:172.21.15.203:6833/3490266242] conn(0x7fcb4806a170 0x7fcb4806c630 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7fcb4c009bc0 tx=0x7fcb4c008040 comp rx=0 tx=0).ready entity=mgr.6659 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-09T13:55:36.180 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:36.176+0000 7fcb64b85740 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x55d4388dbc50 0x55d4388dc030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:55:36.180 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:36.176+0000 7fcb64b85740 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55d4388abbd0 0x55d4388d8ec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:55:36.180 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:36.177+0000 7fcb64b85740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55d4388af7d0 0x55d4388d9400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:55:36.180 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:36.177+0000 7fcb64b85740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d43881d8f0 con 0x55d4388af7d0 2023-08-09T13:55:36.181 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:36.177+0000 7fcb64b85740 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d43881afe0 con 0x55d4388abbd0 2023-08-09T13:55:36.181 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:36.177+0000 7fcb64b85740 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x55d4388168d0 con 0x55d4388dbc50 2023-08-09T13:55:36.181 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:36.177+0000 7fcb5bfff640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55d4388af7d0 0x55d4388d9400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:55:36.181 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:36.177+0000 7fcb5bfff640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55d4388af7d0 0x55d4388d9400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:53264/0 (socket says 172.21.15.43:53264) 2023-08-09T13:55:36.181 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:36.177+0000 7fcb5bfff640 1 -- 172.21.15.43:0/2110658474 learned_addr learned my addr 172.21.15.43:0/2110658474 (peer_addr_for_me v2:172.21.15.43:0/0) 2023-08-09T13:55:36.181 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:36.177+0000 7fcb5b7fe640 1 --2- 172.21.15.43:0/2110658474 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x55d4388dbc50 0x55d4388dc030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:55:36.181 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:36.177+0000 7fcb5affd640 1 --2- 172.21.15.43:0/2110658474 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55d4388abbd0 0x55d4388d8ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:55:36.182 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:36.177+0000 7fcb5bfff640 1 -- 172.21.15.43:0/2110658474 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x55d4388dbc50 msgr2=0x55d4388dc030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-09T13:55:36.182 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:36.177+0000 7fcb5bfff640 1 --2- 172.21.15.43:0/2110658474 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x55d4388dbc50 0x55d4388dc030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:55:36.182 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:36.177+0000 7fcb5bfff640 1 -- 172.21.15.43:0/2110658474 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55d4388abbd0 msgr2=0x55d4388d8ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-09T13:55:36.182 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:36.177+0000 7fcb5bfff640 1 --2- 172.21.15.43:0/2110658474 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55d4388abbd0 0x55d4388d8ec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:55:36.182 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:36.177+0000 7fcb5bfff640 1 -- 172.21.15.43:0/2110658474 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55d4387cfac0 con 0x55d4388af7d0 2023-08-09T13:55:36.182 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:36.178+0000 7fcb5bfff640 1 --2- 172.21.15.43:0/2110658474 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55d4388af7d0 0x55d4388d9400 secure :-1 s=READY pgs=1710 cs=0 l=1 rev1=1 crypto rx=0x7fcb54081f70 tx=0x7fcb54009450 comp rx=0 tx=0).ready entity=mon.0 client_cookie=210504c71ce5dbb1 server_cookie=0 in_seq=0 out_seq=0 2023-08-09T13:55:36.184 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:36.178+0000 7fcb417fa640 1 -- 172.21.15.43:0/2110658474 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcb540824a0 con 0x55d4388af7d0 2023-08-09T13:55:36.184 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:36.178+0000 7fcb417fa640 1 -- 172.21.15.43:0/2110658474 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcb54050de0 con 0x55d4388af7d0 2023-08-09T13:55:36.184 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:36.178+0000 7fcb417fa640 1 -- 172.21.15.43:0/2110658474 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcb54076ea0 con 0x55d4388af7d0 2023-08-09T13:55:36.185 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:36.178+0000 7fcb64b85740 1 -- 172.21.15.43:0/2110658474 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55d4388af7d0 msgr2=0x55d4388d9400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:55:36.185 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:36.178+0000 7fcb64b85740 1 --2- 172.21.15.43:0/2110658474 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55d4388af7d0 0x55d4388d9400 secure :-1 s=READY pgs=1710 cs=0 l=1 rev1=1 crypto rx=0x7fcb54081f70 tx=0x7fcb54009450 comp rx=0 tx=0).stop 2023-08-09T13:55:36.185 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:36.179+0000 7fcb64b85740 1 -- 172.21.15.43:0/2110658474 shutdown_connections 2023-08-09T13:55:36.185 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:36.179+0000 7fcb64b85740 1 --2- 172.21.15.43:0/2110658474 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55d4388af7d0 0x55d4388d9400 unknown :-1 s=CLOSED pgs=1710 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:55:36.185 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:36.179+0000 7fcb64b85740 1 --2- 172.21.15.43:0/2110658474 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55d4388abbd0 0x55d4388d8ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:55:36.185 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:36.179+0000 7fcb64b85740 1 --2- 172.21.15.43:0/2110658474 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x55d4388dbc50 0x55d4388dc030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:55:36.185 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:36.179+0000 7fcb64b85740 1 -- 172.21.15.43:0/2110658474 >> 172.21.15.43:0/2110658474 conn(0x55d4388adee0 msgr2=0x55d4388f6a60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-09T13:55:36.186 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:36.179+0000 7fcb64b85740 1 -- 172.21.15.43:0/2110658474 shutdown_connections 2023-08-09T13:55:36.186 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:36.179+0000 7fcb64b85740 1 -- 172.21.15.43:0/2110658474 wait complete. 2023-08-09T13:55:36.186 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:36.179+0000 7fcb64b85740 1 -- start start 2023-08-09T13:55:36.186 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:36.180+0000 7fcb64b85740 1 Processor -- start 2023-08-09T13:55:36.186 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:36.181+0000 7fcb64b85740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55d4388af7d0 0x55d4388e0390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:55:36.186 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:36.181+0000 7fcb64b85740 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x55d4388dbc50 0x55d4388dad30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:55:36.186 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:36.181+0000 7fcb64b85740 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55d4388c8170 0x55d4388db270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:55:36.187 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:36.181+0000 7fcb64b85740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d4387b6d40 con 0x55d4388af7d0 2023-08-09T13:55:36.187 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:36.181+0000 7fcb64b85740 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d4387b69e0 con 0x55d4388c8170 2023-08-09T13:55:36.187 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:36.181+0000 7fcb64b85740 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x55d43857cf20 con 0x55d4388dbc50 2023-08-09T13:55:36.187 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:36.181+0000 7fcb64b85740 10 client.0 _ll_register_callbacks cb 0x55d4387cf160 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-08-09T13:55:36.187 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:36.181+0000 7fcb5b7fe640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55d4388af7d0 0x55d4388e0390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:55:36.187 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:36.181+0000 7fcb5b7fe640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55d4388af7d0 0x55d4388e0390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:53266/0 (socket says 172.21.15.43:53266) 2023-08-09T13:55:36.187 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:36.181+0000 7fcb5b7fe640 1 -- 172.21.15.43:0/2902736822 learned_addr learned my addr 172.21.15.43:0/2902736822 (peer_addr_for_me v2:172.21.15.43:0/0) 2023-08-09T13:55:36.188 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:36.181+0000 7fcb5bfff640 1 --2- 172.21.15.43:0/2902736822 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55d4388c8170 0x55d4388db270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:55:36.188 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:36.181+0000 7fcb5affd640 1 --2- 172.21.15.43:0/2902736822 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x55d4388dbc50 0x55d4388dad30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:55:36.188 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:36.181+0000 7fcb5b7fe640 1 -- 172.21.15.43:0/2902736822 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x55d4388dbc50 msgr2=0x55d4388dad30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-09T13:55:36.188 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:36.181+0000 7fcb5b7fe640 1 --2- 172.21.15.43:0/2902736822 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x55d4388dbc50 0x55d4388dad30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:55:36.188 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:36.181+0000 7fcb5b7fe640 1 -- 172.21.15.43:0/2902736822 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55d4388c8170 msgr2=0x55d4388db270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-09T13:55:36.188 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:36.181+0000 7fcb5b7fe640 1 --2- 172.21.15.43:0/2902736822 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55d4388c8170 0x55d4388db270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:55:36.188 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:36.181+0000 7fcb5b7fe640 1 -- 172.21.15.43:0/2902736822 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcb5c144540 con 0x55d4388af7d0 2023-08-09T13:55:36.188 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:36.181+0000 7fcb5bfff640 1 --2- 172.21.15.43:0/2902736822 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55d4388c8170 0x55d4388db270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-08-09T13:55:36.189 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:36.182+0000 7fcb5b7fe640 1 --2- 172.21.15.43:0/2902736822 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55d4388af7d0 0x55d4388e0390 secure :-1 s=READY pgs=1711 cs=0 l=1 rev1=1 crypto rx=0x7fcb4c015890 tx=0x7fcb4c0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c24376e7ad5c2d2c server_cookie=0 in_seq=0 out_seq=0 2023-08-09T13:55:36.189 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:36.182+0000 7fcb3a7fc640 10 client.0 ms_handle_connect on v2:172.21.15.43:3300/0 2023-08-09T13:55:36.191 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:36.182+0000 7fcb3a7fc640 1 -- 172.21.15.43:0/2902736822 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcb4c015e10 con 0x55d4388af7d0 2023-08-09T13:55:36.191 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:36.182+0000 7fcb64b85740 1 -- 172.21.15.43:0/2902736822 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55d4387ee500 con 0x55d4388af7d0 2023-08-09T13:55:36.192 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:36.182+0000 7fcb3a7fc640 1 -- 172.21.15.43:0/2902736822 <== mon.0 v2:172.21.15.43:3300/0 2 ==== osd_map(147..147 src has 1..147) v4 ==== 6426+0+0 (secure 0 0 0) 0x7fcb4c014b70 con 0x55d4388af7d0 2023-08-09T13:55:36.192 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:36.183+0000 7fcb3a7fc640 1 -- 172.21.15.43:0/2902736822 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_get_version_reply(handle=1 version=70) v2 ==== 24+0+0 (secure 0 0 0) 0x7fcb4c014d30 con 0x55d4388af7d0 2023-08-09T13:55:36.192 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:36.183+0000 7fcb64b85740 10 client.9270 fetch_fsmap learned FSMap version 70 2023-08-09T13:55:36.192 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:36.183+0000 7fcb64b85740 1 -- 172.21.15.43:0/2902736822 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,fsmap=70,monmap=2+}) v3 -- 0x55d438814f50 con 0x55d4388af7d0 2023-08-09T13:55:36.192 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:36.183+0000 7fcb3a7fc640 1 -- 172.21.15.43:0/2902736822 <== mon.0 v2:172.21.15.43:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x55d4387ee500 con 0x55d4388af7d0 2023-08-09T13:55:36.192 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:36.183+0000 7fcb3a7fc640 1 -- 172.21.15.43:0/2902736822 <== mon.0 v2:172.21.15.43:3300/0 5 ==== fsmap(e 70) v1 ==== 3294+0+0 (secure 0 0 0) 0x7fcb4c02b440 con 0x55d4388af7d0 2023-08-09T13:55:36.192 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:36.183+0000 7fcb64b85740 10 client.9270 fetch_fsmap finished waiting for FSMap version 70 2023-08-09T13:55:36.193 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:36.183+0000 7fcb64b85740 10 client.9270 resolve_mds: resolved name 'a' to daemon mds.a 2023-08-09T13:55:36.193 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:36.183+0000 7fcb64b85740 20 client.9270 populate_metadata read hostname 'smithi043' 2023-08-09T13:55:36.193 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:36.183+0000 7fcb64b85740 1 --2- 172.21.15.43:0/2902736822 >> [v2:172.21.15.43:6832/2023142010,v1:172.21.15.43:6834/2023142010] conn(0x55d4388dc590 0x55d4388dc990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:55:36.193 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:36.183+0000 7fcb64b85740 4 client.9270 mds_command: new command op to 9126 tid=0[{"prefix": "get_command_descriptions"}] 2023-08-09T13:55:36.193 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:36.183+0000 7fcb64b85740 1 -- 172.21.15.43:0/2902736822 --> [v2:172.21.15.43:6832/2023142010,v1:172.21.15.43:6834/2023142010] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55d4387ebbc0 con 0x55d4388dc590 2023-08-09T13:55:36.193 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:36.183+0000 7fcb5affd640 1 --2- 172.21.15.43:0/2902736822 >> [v2:172.21.15.43:6832/2023142010,v1:172.21.15.43:6834/2023142010] conn(0x55d4388dc590 0x55d4388dc990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:55:36.193 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:36.184+0000 7fcb5affd640 1 --2- 172.21.15.43:0/2902736822 >> [v2:172.21.15.43:6832/2023142010,v1:172.21.15.43:6834/2023142010] conn(0x55d4388dc590 0x55d4388dc990 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=eed03a7c96226496 server_cookie=f3a0f8c1cbd73bd in_seq=0 out_seq=0 2023-08-09T13:55:36.193 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:36.184+0000 7fcb3a7fc640 10 client.9270 ms_handle_connect on v2:172.21.15.43:6832/2023142010 2023-08-09T13:55:36.194 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:36.185+0000 7fcb3a7fc640 1 -- 172.21.15.43:0/2902736822 <== mds.0 v2:172.21.15.43:6832/2023142010 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18017 (crc 0 0 0) 0x55d4387ebbc0 con 0x55d4388dc590 2023-08-09T13:55:36.194 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:36.185+0000 7fcb3a7fc640 10 client.9270 handle_command_reply: tid=0 2023-08-09T13:55:36.194 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:36.185+0000 7fcb64b85740 1 client.9270 shutdown 2023-08-09T13:55:36.194 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:36.185+0000 7fcb64b85740 1 -- 172.21.15.43:0/2902736822 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55d4388af7d0 msgr2=0x55d4388e0390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:55:36.194 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:36.185+0000 7fcb64b85740 1 --2- 172.21.15.43:0/2902736822 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55d4388af7d0 0x55d4388e0390 secure :-1 s=READY pgs=1711 cs=0 l=1 rev1=1 crypto rx=0x7fcb4c015890 tx=0x7fcb4c0046c0 comp rx=0 tx=0).stop 2023-08-09T13:55:36.194 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:36.185+0000 7fcb64b85740 1 -- 172.21.15.43:0/2902736822 shutdown_connections 2023-08-09T13:55:36.194 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:36.185+0000 7fcb64b85740 1 --2- 172.21.15.43:0/2902736822 >> [v2:172.21.15.43:6832/2023142010,v1:172.21.15.43:6834/2023142010] conn(0x55d4388dc590 0x55d4388dc990 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:55:36.195 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:36.185+0000 7fcb64b85740 1 --2- 172.21.15.43:0/2902736822 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55d4388c8170 0x55d4388db270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:55:36.195 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:36.185+0000 7fcb64b85740 1 --2- 172.21.15.43:0/2902736822 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x55d4388dbc50 0x55d4388dad30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:55:36.195 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:36.185+0000 7fcb64b85740 1 --2- 172.21.15.43:0/2902736822 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55d4388af7d0 0x55d4388e0390 unknown :-1 s=CLOSED pgs=1711 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:55:36.195 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:36.185+0000 7fcb64b85740 1 -- 172.21.15.43:0/2902736822 >> 172.21.15.43:0/2902736822 conn(0x55d4388abbd0 msgr2=0x55d4388df9d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-09T13:55:36.195 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:36.185+0000 7fcb3a7fc640 0 client.9270 ms_handle_reset on v2:172.21.15.43:6832/2023142010 2023-08-09T13:55:36.195 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:36.185+0000 7fcb64b85740 1 -- 172.21.15.43:0/2902736822 shutdown_connections 2023-08-09T13:55:36.195 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:36.185+0000 7fcb64b85740 1 -- 172.21.15.43:0/2902736822 wait complete. 2023-08-09T13:55:36.195 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:36.186+0000 7fcb64b85740 20 client.9270 trim_cache size 0 max 16384 2023-08-09T13:55:36.205 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:36.203+0000 7fcb64b85740 1 Processor -- start 2023-08-09T13:55:36.205 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:36.203+0000 7fcb64b85740 1 -- start start 2023-08-09T13:55:36.206 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:36.204+0000 7fcb64b85740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55d4388af7d0 0x55d4388cb570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:55:36.206 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:36.204+0000 7fcb64b85740 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55d4388c8170 0x55d4388aace0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:55:36.206 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:36.204+0000 7fcb64b85740 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x55d4388dbc50 0x55d4388fa3f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:55:36.206 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:36.204+0000 7fcb64b85740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d4388ad1c0 con 0x55d4388af7d0 2023-08-09T13:55:36.206 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:36.204+0000 7fcb64b85740 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d4388e0900 con 0x55d4388c8170 2023-08-09T13:55:36.207 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:36.204+0000 7fcb64b85740 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x55d4388c8020 con 0x55d4388dbc50 2023-08-09T13:55:36.207 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:36.204+0000 7fcb5b7fe640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55d4388af7d0 0x55d4388cb570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:55:36.208 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:36.204+0000 7fcb5b7fe640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55d4388af7d0 0x55d4388cb570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:53270/0 (socket says 172.21.15.43:53270) 2023-08-09T13:55:36.208 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:36.204+0000 7fcb5b7fe640 1 -- 172.21.15.43:0/2412668191 learned_addr learned my addr 172.21.15.43:0/2412668191 (peer_addr_for_me v2:172.21.15.43:0/0) 2023-08-09T13:55:36.208 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:36.204+0000 7fcb5b7fe640 1 -- 172.21.15.43:0/2412668191 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x55d4388dbc50 msgr2=0x55d4388fa3f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-09T13:55:36.208 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:36.204+0000 7fcb5b7fe640 1 --2- 172.21.15.43:0/2412668191 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x55d4388dbc50 0x55d4388fa3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:55:36.208 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:36.204+0000 7fcb5b7fe640 1 -- 172.21.15.43:0/2412668191 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55d4388c8170 msgr2=0x55d4388aace0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-09T13:55:36.208 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:36.204+0000 7fcb5b7fe640 1 --2- 172.21.15.43:0/2412668191 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55d4388c8170 0x55d4388aace0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:55:36.208 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:36.204+0000 7fcb5b7fe640 1 -- 172.21.15.43:0/2412668191 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55d4387b5a10 con 0x55d4388af7d0 2023-08-09T13:55:36.209 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:36.205+0000 7fcb5b7fe640 1 --2- 172.21.15.43:0/2412668191 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55d4388af7d0 0x55d4388cb570 secure :-1 s=READY pgs=1712 cs=0 l=1 rev1=1 crypto rx=0x7fcb4c0096a0 tx=0x7fcb4c022000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=50bc1ca8990bdb9e server_cookie=0 in_seq=0 out_seq=0 2023-08-09T13:55:36.209 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:36.205+0000 7fcb41ffb640 1 -- 172.21.15.43:0/2412668191 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcb4c0147b0 con 0x55d4388af7d0 2023-08-09T13:55:36.209 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:36.205+0000 7fcb41ffb640 1 -- 172.21.15.43:0/2412668191 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcb4c009810 con 0x55d4388af7d0 2023-08-09T13:55:36.209 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:36.205+0000 7fcb41ffb640 1 -- 172.21.15.43:0/2412668191 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcb4c015e10 con 0x55d4388af7d0 2023-08-09T13:55:36.210 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:36.206+0000 7fcb64b85740 1 -- 172.21.15.43:0/2412668191 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55d4388af7d0 msgr2=0x55d4388cb570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:55:36.210 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:36.206+0000 7fcb64b85740 1 --2- 172.21.15.43:0/2412668191 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55d4388af7d0 0x55d4388cb570 secure :-1 s=READY pgs=1712 cs=0 l=1 rev1=1 crypto rx=0x7fcb4c0096a0 tx=0x7fcb4c022000 comp rx=0 tx=0).stop 2023-08-09T13:55:36.210 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:36.206+0000 7fcb64b85740 1 -- 172.21.15.43:0/2412668191 shutdown_connections 2023-08-09T13:55:36.211 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:36.206+0000 7fcb64b85740 1 --2- 172.21.15.43:0/2412668191 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x55d4388dbc50 0x55d4388fa3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:55:36.211 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:36.206+0000 7fcb64b85740 1 --2- 172.21.15.43:0/2412668191 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55d4388c8170 0x55d4388aace0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:55:36.211 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:36.206+0000 7fcb64b85740 1 --2- 172.21.15.43:0/2412668191 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55d4388af7d0 0x55d4388cb570 unknown :-1 s=CLOSED pgs=1712 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:55:36.211 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:36.206+0000 7fcb64b85740 1 -- 172.21.15.43:0/2412668191 >> 172.21.15.43:0/2412668191 conn(0x55d4388abbd0 msgr2=0x55d4388e2c80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-09T13:55:36.211 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:36.206+0000 7fcb64b85740 1 -- 172.21.15.43:0/2412668191 shutdown_connections 2023-08-09T13:55:36.211 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:36.206+0000 7fcb64b85740 1 -- 172.21.15.43:0/2412668191 wait complete. 2023-08-09T13:55:36.211 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:36.207+0000 7fcb64b85740 1 -- start start 2023-08-09T13:55:36.212 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:36.207+0000 7fcb64b85740 1 Processor -- start 2023-08-09T13:55:36.212 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:36.208+0000 7fcb64b85740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55d4388af7d0 0x55d4388d86c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:55:36.212 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:36.208+0000 7fcb64b85740 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55d4388c8170 0x55d4388d8c00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:55:36.212 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:36.208+0000 7fcb64b85740 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x55d4388dbc50 0x55d43895f890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:55:36.212 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:36.208+0000 7fcb64b85740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d4388cb2b0 con 0x55d4388af7d0 2023-08-09T13:55:36.212 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:36.208+0000 7fcb64b85740 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d4388d9e80 con 0x55d4388c8170 2023-08-09T13:55:36.212 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:36.208+0000 7fcb64b85740 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x55d4388dca70 con 0x55d4388dbc50 2023-08-09T13:55:36.212 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:36.208+0000 7fcb64b85740 10 client.0 _ll_register_callbacks cb 0x7fcb4c011ac0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-08-09T13:55:36.213 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:36.208+0000 7fcb5b7fe640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55d4388af7d0 0x55d4388d86c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:55:36.213 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:36.208+0000 7fcb5b7fe640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55d4388af7d0 0x55d4388d86c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:53276/0 (socket says 172.21.15.43:53276) 2023-08-09T13:55:36.213 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:36.208+0000 7fcb5b7fe640 1 -- 172.21.15.43:0/497761918 learned_addr learned my addr 172.21.15.43:0/497761918 (peer_addr_for_me v2:172.21.15.43:0/0) 2023-08-09T13:55:36.215 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:36.208+0000 7fcb5b7fe640 1 -- 172.21.15.43:0/497761918 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x55d4388dbc50 msgr2=0x55d43895f890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-09T13:55:36.215 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:36.208+0000 7fcb5b7fe640 1 --2- 172.21.15.43:0/497761918 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x55d4388dbc50 0x55d43895f890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:55:36.215 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:36.208+0000 7fcb5b7fe640 1 -- 172.21.15.43:0/497761918 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55d4388c8170 msgr2=0x55d4388d8c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-09T13:55:36.215 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:36.208+0000 7fcb5b7fe640 1 --2- 172.21.15.43:0/497761918 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55d4388c8170 0x55d4388d8c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:55:36.215 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:36.208+0000 7fcb5b7fe640 1 -- 172.21.15.43:0/497761918 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55d438814f50 con 0x55d4388af7d0 2023-08-09T13:55:36.216 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:36.209+0000 7fcb5b7fe640 1 --2- 172.21.15.43:0/497761918 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55d4388af7d0 0x55d4388d86c0 secure :-1 s=READY pgs=1713 cs=0 l=1 rev1=1 crypto rx=0x7fcb4c02e010 tx=0x7fcb4c016000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e70c0f41f54eee36 server_cookie=0 in_seq=0 out_seq=0 2023-08-09T13:55:36.216 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:36.209+0000 7fcb3a7fc640 10 client.0 ms_handle_connect on v2:172.21.15.43:3300/0 2023-08-09T13:55:36.216 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:36.209+0000 7fcb3a7fc640 1 -- 172.21.15.43:0/497761918 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcb4c015e10 con 0x55d4388af7d0 2023-08-09T13:55:36.216 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:36.209+0000 7fcb3a7fc640 1 -- 172.21.15.43:0/497761918 <== mon.0 v2:172.21.15.43:3300/0 2 ==== osd_map(147..147 src has 1..147) v4 ==== 6426+0+0 (secure 0 0 0) 0x7fcb4c02bb10 con 0x55d4388af7d0 2023-08-09T13:55:36.216 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:36.209+0000 7fcb64b85740 1 -- 172.21.15.43:0/497761918 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55d43883e800 con 0x55d4388af7d0 2023-08-09T13:55:36.216 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:36.209+0000 7fcb3a7fc640 1 -- 172.21.15.43:0/497761918 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_get_version_reply(handle=1 version=70) v2 ==== 24+0+0 (secure 0 0 0) 0x7fcb4c02bcd0 con 0x55d4388af7d0 2023-08-09T13:55:36.216 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:36.209+0000 7fcb64b85740 10 client.9276 fetch_fsmap learned FSMap version 70 2023-08-09T13:55:36.216 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:36.209+0000 7fcb64b85740 1 -- 172.21.15.43:0/497761918 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,fsmap=70,monmap=2+}) v3 -- 0x55d4387dade0 con 0x55d4388af7d0 2023-08-09T13:55:36.217 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:36.210+0000 7fcb3a7fc640 1 -- 172.21.15.43:0/497761918 <== mon.0 v2:172.21.15.43:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x55d43883e800 con 0x55d4388af7d0 2023-08-09T13:55:36.217 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:36.210+0000 7fcb3a7fc640 1 -- 172.21.15.43:0/497761918 <== mon.0 v2:172.21.15.43:3300/0 5 ==== fsmap(e 70) v1 ==== 3294+0+0 (secure 0 0 0) 0x7fcb4c047d20 con 0x55d4388af7d0 2023-08-09T13:55:36.217 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:36.210+0000 7fcb64b85740 10 client.9276 fetch_fsmap finished waiting for FSMap version 70 2023-08-09T13:55:36.217 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:36.210+0000 7fcb64b85740 10 client.9276 resolve_mds: resolved name 'a' to daemon mds.a 2023-08-09T13:55:36.217 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:36.210+0000 7fcb64b85740 20 client.9276 populate_metadata read hostname 'smithi043' 2023-08-09T13:55:36.217 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:36.210+0000 7fcb64b85740 1 --2- 172.21.15.43:0/497761918 >> [v2:172.21.15.43:6832/2023142010,v1:172.21.15.43:6834/2023142010] conn(0x55d4388db3a0 0x55d43895fdd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:55:36.217 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:36.210+0000 7fcb64b85740 4 client.9276 mds_command: new command op to 9126 tid=0[{"prefix": "status"}] 2023-08-09T13:55:36.218 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:36.210+0000 7fcb64b85740 1 -- 172.21.15.43:0/497761918 --> [v2:172.21.15.43:6832/2023142010,v1:172.21.15.43:6834/2023142010] -- command(tid 0: {"prefix": "status"}) v1 -- 0x55d43880f8b0 con 0x55d4388db3a0 2023-08-09T13:55:36.218 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:36.210+0000 7fcb5affd640 1 --2- 172.21.15.43:0/497761918 >> [v2:172.21.15.43:6832/2023142010,v1:172.21.15.43:6834/2023142010] conn(0x55d4388db3a0 0x55d43895fdd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:55:36.218 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:36.211+0000 7fcb5affd640 1 --2- 172.21.15.43:0/497761918 >> [v2:172.21.15.43:6832/2023142010,v1:172.21.15.43:6834/2023142010] conn(0x55d4388db3a0 0x55d43895fdd0 crc :-1 s=READY pgs=14 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=dd31cc3e6bec78f2 server_cookie=39a795396bd11fc in_seq=0 out_seq=0 2023-08-09T13:55:36.218 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:36.211+0000 7fcb3a7fc640 10 client.9276 ms_handle_connect on v2:172.21.15.43:6832/2023142010 2023-08-09T13:55:36.218 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:36.211+0000 7fcb3a7fc640 1 -- 172.21.15.43:0/497761918 <== mds.0 v2:172.21.15.43:6832/2023142010 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+327 (crc 0 0 0) 0x55d43880f8b0 con 0x55d4388db3a0 2023-08-09T13:55:36.218 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:36.211+0000 7fcb3a7fc640 10 client.9276 handle_command_reply: tid=0 2023-08-09T13:55:36.218 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:36.211+0000 7fcb64b85740 1 client.9276 shutdown 2023-08-09T13:55:36.218 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:36.211+0000 7fcb64b85740 1 -- 172.21.15.43:0/497761918 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55d4388af7d0 msgr2=0x55d4388d86c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:55:36.219 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:36.211+0000 7fcb64b85740 1 --2- 172.21.15.43:0/497761918 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55d4388af7d0 0x55d4388d86c0 secure :-1 s=READY pgs=1713 cs=0 l=1 rev1=1 crypto rx=0x7fcb4c02e010 tx=0x7fcb4c016000 comp rx=0 tx=0).stop 2023-08-09T13:55:36.219 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:36.211+0000 7fcb64b85740 1 -- 172.21.15.43:0/497761918 shutdown_connections 2023-08-09T13:55:36.219 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:36.211+0000 7fcb64b85740 1 --2- 172.21.15.43:0/497761918 >> [v2:172.21.15.43:6832/2023142010,v1:172.21.15.43:6834/2023142010] conn(0x55d4388db3a0 0x55d43895fdd0 crc :-1 s=READY pgs=14 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:55:36.219 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:36.211+0000 7fcb64b85740 1 --2- 172.21.15.43:0/497761918 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x55d4388dbc50 0x55d43895f890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:55:36.219 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:36.211+0000 7fcb64b85740 1 --2- 172.21.15.43:0/497761918 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55d4388c8170 0x55d4388d8c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:55:36.219 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:36.211+0000 7fcb64b85740 1 --2- 172.21.15.43:0/497761918 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55d4388af7d0 0x55d4388d86c0 unknown :-1 s=CLOSED pgs=1713 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:55:36.219 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:36.211+0000 7fcb3a7fc640 0 client.9276 ms_handle_reset on v2:172.21.15.43:6832/2023142010 2023-08-09T13:55:36.219 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:36.211+0000 7fcb64b85740 1 -- 172.21.15.43:0/497761918 >> 172.21.15.43:0/497761918 conn(0x55d4388abbd0 msgr2=0x55d4388c7350 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-09T13:55:36.220 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:36.211+0000 7fcb64b85740 1 -- 172.21.15.43:0/497761918 shutdown_connections 2023-08-09T13:55:36.220 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:36.211+0000 7fcb64b85740 1 -- 172.21.15.43:0/497761918 wait complete. 2023-08-09T13:55:36.220 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:36.212+0000 7fcb64b85740 20 client.9276 trim_cache size 0 max 16384 2023-08-09T13:55:36.220 INFO:teuthology.orchestra.run.smithi043.stdout:{ 2023-08-09T13:55:36.221 INFO:teuthology.orchestra.run.smithi043.stdout: "cluster_fsid": "daac87ee-1337-4250-a955-aea4e408a39b", 2023-08-09T13:55:36.221 INFO:teuthology.orchestra.run.smithi043.stdout: "whoami": 0, 2023-08-09T13:55:36.221 INFO:teuthology.orchestra.run.smithi043.stdout: "id": 9126, 2023-08-09T13:55:36.221 INFO:teuthology.orchestra.run.smithi043.stdout: "want_state": "up:active", 2023-08-09T13:55:36.221 INFO:teuthology.orchestra.run.smithi043.stdout: "state": "up:active", 2023-08-09T13:55:36.221 INFO:teuthology.orchestra.run.smithi043.stdout: "fs_name": "cephfs", 2023-08-09T13:55:36.221 INFO:teuthology.orchestra.run.smithi043.stdout: "rank_uptime": 6.8798417269999996, 2023-08-09T13:55:36.222 INFO:teuthology.orchestra.run.smithi043.stdout: "mdsmap_epoch": 70, 2023-08-09T13:55:36.222 INFO:teuthology.orchestra.run.smithi043.stdout: "osdmap_epoch": 147, 2023-08-09T13:55:36.222 INFO:teuthology.orchestra.run.smithi043.stdout: "osdmap_epoch_barrier": 147, 2023-08-09T13:55:36.222 INFO:teuthology.orchestra.run.smithi043.stdout: "uptime": 13.185988134 2023-08-09T13:55:36.222 INFO:teuthology.orchestra.run.smithi043.stdout:} 2023-08-09T13:55:36.223 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:36.212+0000 7fcb61f42640 1 -- 172.21.15.43:0/768276650 >> [v2:172.21.15.203:6832/3490266242,v1:172.21.15.203:6833/3490266242] conn(0x7fcb4806a170 msgr2=0x7fcb4806c630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:55:36.223 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:36.212+0000 7fcb61f42640 1 --2- 172.21.15.43:0/768276650 >> [v2:172.21.15.203:6832/3490266242,v1:172.21.15.203:6833/3490266242] conn(0x7fcb4806a170 0x7fcb4806c630 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7fcb4c009bc0 tx=0x7fcb4c008040 comp rx=0 tx=0).stop 2023-08-09T13:55:36.223 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:36.212+0000 7fcb61f42640 1 -- 172.21.15.43:0/768276650 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fcb5c15c680 msgr2=0x7fcb5c157d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:55:36.223 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:36.212+0000 7fcb61f42640 1 --2- 172.21.15.43:0/768276650 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fcb5c15c680 0x7fcb5c157d40 secure :-1 s=READY pgs=1709 cs=0 l=1 rev1=1 crypto rx=0x7fcb54009d90 tx=0x7fcb54004a40 comp rx=0 tx=0).stop 2023-08-09T13:55:36.223 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:36.212+0000 7fcb61f42640 1 -- 172.21.15.43:0/768276650 shutdown_connections 2023-08-09T13:55:36.224 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:36.212+0000 7fcb61f42640 1 --2- 172.21.15.43:0/768276650 >> [v2:172.21.15.203:6832/3490266242,v1:172.21.15.203:6833/3490266242] conn(0x7fcb4806a170 0x7fcb4806c630 unknown :-1 s=CLOSED pgs=68 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:55:36.224 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:36.212+0000 7fcb61f42640 1 --2- 172.21.15.43:0/768276650 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fcb5c15c680 0x7fcb5c157d40 unknown :-1 s=CLOSED pgs=1709 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:55:36.224 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:36.212+0000 7fcb61f42640 1 --2- 172.21.15.43:0/768276650 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fcb5c0d0a70 0x7fcb5c157800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:55:36.224 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:36.212+0000 7fcb61f42640 1 --2- 172.21.15.43:0/768276650 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fcb5c0b3fa0 0x7fcb5c0c8180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:55:36.224 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:36.212+0000 7fcb61f42640 1 -- 172.21.15.43:0/768276650 >> 172.21.15.43:0/768276650 conn(0x7fcb5c071520 msgr2=0x7fcb5c0b7660 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-09T13:55:36.224 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:36.212+0000 7fcb61f42640 1 -- 172.21.15.43:0/768276650 shutdown_connections 2023-08-09T13:55:36.224 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:36.212+0000 7fcb61f42640 1 -- 172.21.15.43:0/768276650 wait complete. 2023-08-09T13:55:36.233 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2023-08-09T13:55:36.445 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:36.443+0000 7fdee761f640 1 Processor -- start 2023-08-09T13:55:36.445 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:36.443+0000 7fdee761f640 1 -- start start 2023-08-09T13:55:36.447 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:36.444+0000 7fdee761f640 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7fdee015c930 0x7fdee00cee40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:55:36.447 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:36.445+0000 7fdee761f640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fdee00cf380 0x7fdee00b5460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:55:36.447 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:36.445+0000 7fdee761f640 1 -- --> v1:172.21.15.203:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fdee00d0700 con 0x7fdee015c550 2023-08-09T13:55:36.447 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:36.445+0000 7fdee761f640 1 -- --> v2:172.21.15.203:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fdee00cf830 con 0x7fdee015c930 2023-08-09T13:55:36.448 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:36.445+0000 7fdee761f640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdee00d09e0 con 0x7fdee00cf380 2023-08-09T13:55:36.449 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:36.445+0000 7fdee4bba640 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7fdee015c930 0x7fdee00cee40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:55:36.449 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:36.445+0000 7fdee4bba640 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7fdee015c930 0x7fdee00cee40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.203:3300/0 says I am v2:172.21.15.43:36200/0 (socket says 172.21.15.43:36200) 2023-08-09T13:55:36.449 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:36.445+0000 7fdee4bba640 1 -- 172.21.15.43:0/1181518135 learned_addr learned my addr 172.21.15.43:0/1181518135 (peer_addr_for_me v2:172.21.15.43:0/0) 2023-08-09T13:55:36.449 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:36.445+0000 7fdee53bb640 1 --1- 172.21.15.43:0/1181518135 >> v1:172.21.15.203:6789/0 conn(0x7fdee015c550 0x7fdee00d1a80 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.203:6789/0 says I am v1:172.21.15.43:36384/0 (socket says 172.21.15.43:36384) 2023-08-09T13:55:36.449 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:36.446+0000 7fdee5bbc640 1 --2- 172.21.15.43:0/1181518135 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fdee00cf380 0x7fdee00b5460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:55:36.449 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:36.446+0000 7fded7fff640 1 -- 172.21.15.43:0/1181518135 <== mon.1 v1:172.21.15.203:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1756970027 0 0) 0x7fdee00d0700 con 0x7fdee015c550 2023-08-09T13:55:36.449 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:36.446+0000 7fded7fff640 1 -- 172.21.15.43:0/1181518135 --> v1:172.21.15.203:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fdecc003520 con 0x7fdee015c550 2023-08-09T13:55:36.450 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:36.446+0000 7fdee4bba640 1 -- 172.21.15.43:0/1181518135 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fdee00cf380 msgr2=0x7fdee00b5460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-09T13:55:36.450 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:36.446+0000 7fdee4bba640 1 --2- 172.21.15.43:0/1181518135 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fdee00cf380 0x7fdee00b5460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:55:36.450 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:36.446+0000 7fdee4bba640 1 -- 172.21.15.43:0/1181518135 >> v1:172.21.15.203:6789/0 conn(0x7fdee015c550 legacy=0x7fdee00d1a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:55:36.450 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:36.446+0000 7fdee4bba640 1 -- 172.21.15.43:0/1181518135 --> v2:172.21.15.203:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdee00d0cc0 con 0x7fdee015c930 2023-08-09T13:55:36.451 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:36.448+0000 7fdee4bba640 1 --2- 172.21.15.43:0/1181518135 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fdee015c930 0x7fdee00cee40 secure :-1 s=READY pgs=630 cs=0 l=1 rev1=1 crypto rx=0x7fded0009870 tx=0x7fded002ba20 comp rx=0 tx=0).ready entity=mon.1 client_cookie=53185f2e7dc1aac6 server_cookie=0 in_seq=0 out_seq=0 2023-08-09T13:55:36.451 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:36.448+0000 7fded7fff640 1 -- 172.21.15.43:0/1181518135 <== mon.1 v2:172.21.15.203:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fded002f840 con 0x7fdee015c930 2023-08-09T13:55:36.451 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:36.448+0000 7fded7fff640 1 -- 172.21.15.43:0/1181518135 <== mon.1 v2:172.21.15.203:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fded002f9a0 con 0x7fdee015c930 2023-08-09T13:55:36.451 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:36.448+0000 7fded7fff640 1 -- 172.21.15.43:0/1181518135 <== mon.1 v2:172.21.15.203:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fded002fd10 con 0x7fdee015c930 2023-08-09T13:55:36.452 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:36.449+0000 7fdee761f640 1 -- 172.21.15.43:0/1181518135 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fdee015c930 msgr2=0x7fdee00cee40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:55:36.452 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:36.449+0000 7fdee761f640 1 --2- 172.21.15.43:0/1181518135 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fdee015c930 0x7fdee00cee40 secure :-1 s=READY pgs=630 cs=0 l=1 rev1=1 crypto rx=0x7fded0009870 tx=0x7fded002ba20 comp rx=0 tx=0).stop 2023-08-09T13:55:36.452 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:36.449+0000 7fdee761f640 1 -- 172.21.15.43:0/1181518135 shutdown_connections 2023-08-09T13:55:36.452 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:36.449+0000 7fdee761f640 1 --2- 172.21.15.43:0/1181518135 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fdee00cf380 0x7fdee00b5460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:55:36.452 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:36.449+0000 7fdee761f640 1 --2- 172.21.15.43:0/1181518135 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fdee015c930 0x7fdee00cee40 unknown :-1 s=CLOSED pgs=630 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:55:36.452 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:36.449+0000 7fdee761f640 1 -- 172.21.15.43:0/1181518135 >> 172.21.15.43:0/1181518135 conn(0x7fdee006a4e0 msgr2=0x7fdee006a8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-09T13:55:36.453 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:36.449+0000 7fdee761f640 1 -- 172.21.15.43:0/1181518135 shutdown_connections 2023-08-09T13:55:36.453 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:36.449+0000 7fdee761f640 1 -- 172.21.15.43:0/1181518135 wait complete. 2023-08-09T13:55:36.453 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:36.451+0000 7fdee761f640 1 Processor -- start 2023-08-09T13:55:36.453 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:36.451+0000 7fdee761f640 1 -- start start 2023-08-09T13:55:36.455 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:36.452+0000 7fdee761f640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fdee00cf380 0x7fdee01f5540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:55:36.455 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:36.452+0000 7fdee761f640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fdee015c550 0x7fdee00c4cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:55:36.455 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:36.452+0000 7fdee761f640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fdee015c930 0x7fdee00caf60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:55:36.455 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:36.452+0000 7fdee761f640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdee00cb4a0 con 0x7fdee015c550 2023-08-09T13:55:36.455 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:36.452+0000 7fdee761f640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdee00cb700 con 0x7fdee00cf380 2023-08-09T13:55:36.455 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:36.452+0000 7fdee761f640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdee00cb9e0 con 0x7fdee015c930 2023-08-09T13:55:36.455 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:36.452+0000 7fdee4bba640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fdee015c550 0x7fdee00c4cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:55:36.456 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:36.452+0000 7fdee4bba640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fdee015c550 0x7fdee00c4cc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:53284/0 (socket says 172.21.15.43:53284) 2023-08-09T13:55:36.456 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:36.452+0000 7fdee4bba640 1 -- 172.21.15.43:0/1737601603 learned_addr learned my addr 172.21.15.43:0/1737601603 (peer_addr_for_me v2:172.21.15.43:0/0) 2023-08-09T13:55:36.457 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:36.453+0000 7fdee53bb640 1 --2- 172.21.15.43:0/1737601603 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fdee00cf380 0x7fdee01f5540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:55:36.457 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:36.453+0000 7fdee5bbc640 1 --2- 172.21.15.43:0/1737601603 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fdee015c930 0x7fdee00caf60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:55:36.457 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:36.453+0000 7fdee4bba640 1 -- 172.21.15.43:0/1737601603 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fdee015c930 msgr2=0x7fdee00caf60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:55:36.457 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:36.453+0000 7fdee4bba640 1 --2- 172.21.15.43:0/1737601603 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fdee015c930 0x7fdee00caf60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:55:36.457 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:36.453+0000 7fdee4bba640 1 -- 172.21.15.43:0/1737601603 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fdee00cf380 msgr2=0x7fdee01f5540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:55:36.457 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:36.453+0000 7fdee4bba640 1 --2- 172.21.15.43:0/1737601603 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fdee00cf380 0x7fdee01f5540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:55:36.458 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:36.453+0000 7fdee4bba640 1 -- 172.21.15.43:0/1737601603 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fded00094d0 con 0x7fdee015c550 2023-08-09T13:55:36.458 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:36.453+0000 7fdee4bba640 1 --2- 172.21.15.43:0/1737601603 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fdee015c550 0x7fdee00c4cc0 secure :-1 s=READY pgs=1714 cs=0 l=1 rev1=1 crypto rx=0x7fded002ba60 tx=0x7fded00046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-09T13:55:36.458 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:36.454+0000 7fded5ffb640 1 -- 172.21.15.43:0/1737601603 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fded003b040 con 0x7fdee015c550 2023-08-09T13:55:36.458 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:36.454+0000 7fded5ffb640 1 -- 172.21.15.43:0/1737601603 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fded0037c60 con 0x7fdee015c550 2023-08-09T13:55:36.458 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:36.454+0000 7fded5ffb640 1 -- 172.21.15.43:0/1737601603 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fded0042450 con 0x7fdee015c550 2023-08-09T13:55:36.458 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:36.454+0000 7fdee761f640 1 -- 172.21.15.43:0/1737601603 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdee00cbcc0 con 0x7fdee015c550 2023-08-09T13:55:36.458 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:36.454+0000 7fdee761f640 1 -- 172.21.15.43:0/1737601603 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdee00b1cd0 con 0x7fdee015c550 2023-08-09T13:55:36.459 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:36.455+0000 7fdee761f640 1 -- 172.21.15.43:0/1737601603 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdee00c6850 con 0x7fdee015c550 2023-08-09T13:55:36.459 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:36.456+0000 7fded5ffb640 1 -- 172.21.15.43:0/1737601603 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 67) v1 ==== 88443+0+0 (secure 0 0 0) 0x7fded003e070 con 0x7fdee015c550 2023-08-09T13:55:36.460 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:36.457+0000 7fded5ffb640 1 --2- 172.21.15.43:0/1737601603 >> [v2:172.21.15.203:6832/3490266242,v1:172.21.15.203:6833/3490266242] conn(0x7fdec406a170 0x7fdec406c630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:55:36.460 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:36.457+0000 7fded5ffb640 1 -- 172.21.15.43:0/1737601603 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(147..147 src has 1..147) v4 ==== 6426+0+0 (secure 0 0 0) 0x7fded00ab830 con 0x7fdee015c550 2023-08-09T13:55:36.460 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:36.458+0000 7fdee53bb640 1 --2- 172.21.15.43:0/1737601603 >> [v2:172.21.15.203:6832/3490266242,v1:172.21.15.203:6833/3490266242] conn(0x7fdec406a170 0x7fdec406c630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:55:36.461 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:36.459+0000 7fdee53bb640 1 --2- 172.21.15.43:0/1737601603 >> [v2:172.21.15.203:6832/3490266242,v1:172.21.15.203:6833/3490266242] conn(0x7fdec406a170 0x7fdec406c630 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7fdedc0011e0 tx=0x7fdedc007b70 comp rx=0 tx=0).ready entity=mgr.6659 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-09T13:55:36.463 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:36.461+0000 7fded5ffb640 1 -- 172.21.15.43:0/1737601603 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178066 (secure 0 0 0) 0x7fded0076ca0 con 0x7fdee015c550 2023-08-09T13:55:36.694 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:36.692+0000 7fdee761f640 1 -- 172.21.15.43:0/1737601603 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7fdee00739f0 con 0x7fdee015c550 2023-08-09T13:55:36.695 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:36.693+0000 7fded5ffb640 1 -- 172.21.15.43:0/1737601603 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v70) v1 ==== 53+0+73 (secure 0 0 0) 0x7fded007a2c0 con 0x7fdee015c550 2023-08-09T13:55:36.695 INFO:teuthology.orchestra.run.smithi043.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2023-08-09T13:55:36.698 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:36.695+0000 7fdee761f640 1 -- 172.21.15.43:0/1737601603 >> [v2:172.21.15.203:6832/3490266242,v1:172.21.15.203:6833/3490266242] conn(0x7fdec406a170 msgr2=0x7fdec406c630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:55:36.698 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:36.695+0000 7fdee761f640 1 --2- 172.21.15.43:0/1737601603 >> [v2:172.21.15.203:6832/3490266242,v1:172.21.15.203:6833/3490266242] conn(0x7fdec406a170 0x7fdec406c630 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7fdedc0011e0 tx=0x7fdedc007b70 comp rx=0 tx=0).stop 2023-08-09T13:55:36.698 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:36.695+0000 7fdee761f640 1 -- 172.21.15.43:0/1737601603 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fdee015c550 msgr2=0x7fdee00c4cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:55:36.698 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:36.695+0000 7fdee761f640 1 --2- 172.21.15.43:0/1737601603 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fdee015c550 0x7fdee00c4cc0 secure :-1 s=READY pgs=1714 cs=0 l=1 rev1=1 crypto rx=0x7fded002ba60 tx=0x7fded00046c0 comp rx=0 tx=0).stop 2023-08-09T13:55:36.699 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:36.695+0000 7fdee761f640 1 -- 172.21.15.43:0/1737601603 shutdown_connections 2023-08-09T13:55:36.699 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:36.696+0000 7fdee761f640 1 --2- 172.21.15.43:0/1737601603 >> [v2:172.21.15.203:6832/3490266242,v1:172.21.15.203:6833/3490266242] conn(0x7fdec406a170 0x7fdec406c630 unknown :-1 s=CLOSED pgs=69 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:55:36.699 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:36.696+0000 7fdee761f640 1 --2- 172.21.15.43:0/1737601603 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fdee015c930 0x7fdee00caf60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:55:36.699 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:36.696+0000 7fdee761f640 1 --2- 172.21.15.43:0/1737601603 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fdee015c550 0x7fdee00c4cc0 unknown :-1 s=CLOSED pgs=1714 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:55:36.699 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:36.696+0000 7fdee761f640 1 --2- 172.21.15.43:0/1737601603 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fdee00cf380 0x7fdee01f5540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:55:36.700 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:36.696+0000 7fdee761f640 1 -- 172.21.15.43:0/1737601603 >> 172.21.15.43:0/1737601603 conn(0x7fdee006a4e0 msgr2=0x7fdee006a9a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-09T13:55:36.700 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:36.696+0000 7fdee761f640 1 -- 172.21.15.43:0/1737601603 shutdown_connections 2023-08-09T13:55:36.700 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:36.696+0000 7fdee761f640 1 -- 172.21.15.43:0/1737601603 wait complete. 2023-08-09T13:55:36.718 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2023-08-09T13:55:36.718 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2023-08-09T13:55:36.718 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi043.front.sepia.ceph.com 2023-08-09T13:55:36.719 INFO:tasks.cephfs.mount:self.client.name = client.0 2023-08-09T13:55:36.719 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2023-08-09T13:55:36.719 INFO:tasks.cephfs.mount:self.cephfs_name = None 2023-08-09T13:55:36.719 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2023-08-09T13:55:36.720 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2023-08-09T13:55:36.720 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2023-08-09T13:55:36.720 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-09T13:55:36.721 DEBUG:teuthology.orchestra.run.smithi043:> ip addr 2023-08-09T13:55:36.776 INFO:teuthology.orchestra.run.smithi043.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2023-08-09T13:55:36.776 INFO:teuthology.orchestra.run.smithi043.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2023-08-09T13:55:36.776 INFO:teuthology.orchestra.run.smithi043.stdout: inet 127.0.0.1/8 scope host lo 2023-08-09T13:55:36.776 INFO:teuthology.orchestra.run.smithi043.stdout: valid_lft forever preferred_lft forever 2023-08-09T13:55:36.777 INFO:teuthology.orchestra.run.smithi043.stdout: inet6 ::1/128 scope host 2023-08-09T13:55:36.777 INFO:teuthology.orchestra.run.smithi043.stdout: valid_lft forever preferred_lft forever 2023-08-09T13:55:36.777 INFO:teuthology.orchestra.run.smithi043.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-08-09T13:55:36.777 INFO:teuthology.orchestra.run.smithi043.stdout: link/ether 0c:c4:7a:6b:fd:9c brd ff:ff:ff:ff:ff:ff 2023-08-09T13:55:36.777 INFO:teuthology.orchestra.run.smithi043.stdout: altname enp7s0f0 2023-08-09T13:55:36.777 INFO:teuthology.orchestra.run.smithi043.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-08-09T13:55:36.777 INFO:teuthology.orchestra.run.smithi043.stdout: link/ether 0c:c4:7a:6b:fd:9d brd ff:ff:ff:ff:ff:ff 2023-08-09T13:55:36.778 INFO:teuthology.orchestra.run.smithi043.stdout: altname enp7s0f1 2023-08-09T13:55:36.778 INFO:teuthology.orchestra.run.smithi043.stdout:4: ens1f0: mtu 1500 qdisc mq state UP group default qlen 1000 2023-08-09T13:55:36.778 INFO:teuthology.orchestra.run.smithi043.stdout: link/ether 0c:c4:7a:bd:13:58 brd ff:ff:ff:ff:ff:ff 2023-08-09T13:55:36.778 INFO:teuthology.orchestra.run.smithi043.stdout: altname enp1s0f0 2023-08-09T13:55:36.778 INFO:teuthology.orchestra.run.smithi043.stdout: inet 172.21.15.43/20 brd 172.21.15.255 scope global dynamic noprefixroute ens1f0 2023-08-09T13:55:36.778 INFO:teuthology.orchestra.run.smithi043.stdout: valid_lft 40967sec preferred_lft 40967sec 2023-08-09T13:55:36.778 INFO:teuthology.orchestra.run.smithi043.stdout: inet6 fe80::ec4:7aff:febd:1358/64 scope link 2023-08-09T13:55:36.778 INFO:teuthology.orchestra.run.smithi043.stdout: valid_lft forever preferred_lft forever 2023-08-09T13:55:36.779 INFO:teuthology.orchestra.run.smithi043.stdout:5: ens1f1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-08-09T13:55:36.779 INFO:teuthology.orchestra.run.smithi043.stdout: link/ether 0c:c4:7a:bd:13:59 brd ff:ff:ff:ff:ff:ff 2023-08-09T13:55:36.779 INFO:teuthology.orchestra.run.smithi043.stdout: altname enp1s0f1 2023-08-09T13:55:36.779 INFO:teuthology.orchestra.run.smithi043.stdout:8: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2023-08-09T13:55:36.780 INFO:teuthology.orchestra.run.smithi043.stdout: link/ether 56:43:50:19:f4:0c brd ff:ff:ff:ff:ff:ff 2023-08-09T13:55:36.780 INFO:teuthology.orchestra.run.smithi043.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2023-08-09T13:55:36.780 INFO:teuthology.orchestra.run.smithi043.stdout: valid_lft forever preferred_lft forever 2023-08-09T13:55:36.780 INFO:teuthology.orchestra.run.smithi043.stdout: inet6 fe80::e494:65ff:fe92:a182/64 scope link 2023-08-09T13:55:36.780 INFO:teuthology.orchestra.run.smithi043.stdout: valid_lft forever preferred_lft forever 2023-08-09T13:55:36.781 INFO:teuthology.orchestra.run.smithi043.stdout:9: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2023-08-09T13:55:36.781 INFO:teuthology.orchestra.run.smithi043.stdout: link/ether 7e:93:26:fa:07:8f brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.0 2023-08-09T13:55:36.781 INFO:teuthology.orchestra.run.smithi043.stdout: inet6 fe80::7c93:26ff:fefa:78f/64 scope link 2023-08-09T13:55:36.781 INFO:teuthology.orchestra.run.smithi043.stdout: valid_lft forever preferred_lft forever 2023-08-09T13:55:36.781 INFO:teuthology.orchestra.run.smithi043.stdout:10: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2023-08-09T13:55:36.781 INFO:teuthology.orchestra.run.smithi043.stdout: link/ether 56:43:50:19:f4:0c brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.2 2023-08-09T13:55:36.781 INFO:teuthology.orchestra.run.smithi043.stdout: inet6 fe80::5443:50ff:fe19:f40c/64 scope link 2023-08-09T13:55:36.781 INFO:teuthology.orchestra.run.smithi043.stdout: valid_lft forever preferred_lft forever 2023-08-09T13:55:36.783 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-09T13:55:36.783 DEBUG:teuthology.orchestra.run.smithi043:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2023-08-09T13:55:36.884 INFO:teuthology.orchestra.run.smithi043.stdout:1 2023-08-09T13:55:36.889 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-09T13:55:36.889 DEBUG:teuthology.orchestra.run.smithi043:> route 2023-08-09T13:55:36.948 INFO:teuthology.orchestra.run.smithi043.stdout:Kernel IP routing table 2023-08-09T13:55:36.948 INFO:teuthology.orchestra.run.smithi043.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2023-08-09T13:55:36.948 INFO:teuthology.orchestra.run.smithi043.stdout:default _gateway 0.0.0.0 UG 100 0 0 ens1f0 2023-08-09T13:55:36.949 INFO:teuthology.orchestra.run.smithi043.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 ens1f0 2023-08-09T13:55:36.949 INFO:teuthology.orchestra.run.smithi043.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2023-08-09T13:55:36.950 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-09T13:55:36.950 DEBUG:teuthology.orchestra.run.smithi043:> (cd / && exec bash -c ' 2023-08-09T13:55:36.950 DEBUG:teuthology.orchestra.run.smithi043:> set -e 2023-08-09T13:55:36.951 DEBUG:teuthology.orchestra.run.smithi043:> sudo iptables -A FORWARD -o ens1f0 -i ceph-brx -j ACCEPT 2023-08-09T13:55:36.951 DEBUG:teuthology.orchestra.run.smithi043:> sudo iptables -A FORWARD -i ens1f0 -o ceph-brx -j ACCEPT 2023-08-09T13:55:36.951 DEBUG:teuthology.orchestra.run.smithi043:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o ens1f0 -j MASQUERADE 2023-08-09T13:55:36.952 DEBUG:teuthology.orchestra.run.smithi043:> ') 2023-08-09T13:55:37.149 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-09T13:55:37.149 DEBUG:teuthology.orchestra.run.smithi043:> ip netns list 2023-08-09T13:55:37.201 INFO:teuthology.orchestra.run.smithi043.stdout:ceph-ns--home-ubuntu-cephtest-mnt.2 (id: 1) 2023-08-09T13:55:37.202 INFO:teuthology.orchestra.run.smithi043.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2023-08-09T13:55:37.203 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2023-08-09T13:55:37.203 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-09T13:55:37.204 DEBUG:teuthology.orchestra.run.smithi043:> sudo ip link set brx.0 up 2023-08-09T13:55:37.309 INFO:tasks.cephfs.fuse_mount:Client client.0 config is {} 2023-08-09T13:55:37.310 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-08-09T13:55:37.310 DEBUG:teuthology.orchestra.run.smithi043:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2023-08-09T13:55:37.372 INFO:teuthology.orchestra.run.smithi043.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2023-08-09T13:55:37.372 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-08-09T13:55:37.373 DEBUG:teuthology.orchestra.run.smithi043:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2023-08-09T13:55:37.433 DEBUG:teuthology.orchestra.run.smithi043:> sudo modprobe fuse 2023-08-09T13:55:37.531 INFO:teuthology.orchestra.run:Running command with timeout 30 2023-08-09T13:55:37.532 DEBUG:teuthology.orchestra.run.smithi043:> sudo mount -t fusectl /sys/fs/fuse/connections /sys/fs/fuse/connections 2023-08-09T13:55:37.636 INFO:teuthology.orchestra.run.smithi043.stderr:mount: /sys/fs/fuse/connections: /sys/fs/fuse/connections already mounted or mount point busy. 2023-08-09T13:55:37.641 DEBUG:teuthology.orchestra.run:got remote process result: 32 2023-08-09T13:55:37.641 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-09T13:55:37.642 DEBUG:teuthology.orchestra.run.smithi043:> ls /sys/fs/fuse/connections 2023-08-09T13:55:37.699 INFO:tasks.cephfs.fuse_mount:Pre-mount connections: [] 2023-08-09T13:55:37.700 DEBUG:teuthology.orchestra.run.smithi043:> (cd /home/ubuntu/cephtest && exec sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-fuse -f --admin-socket '/var/run/ceph/$cluster-$name.$pid.asok' /home/ubuntu/cephtest/mnt.0 --id 0) 2023-08-09T13:55:37.706 DEBUG:teuthology.orchestra.run.smithi043:> sudo modprobe fuse 2023-08-09T13:55:37.794 INFO:teuthology.orchestra.run:Running command with timeout 30 2023-08-09T13:55:37.794 DEBUG:teuthology.orchestra.run.smithi043:> sudo mount -t fusectl /sys/fs/fuse/connections /sys/fs/fuse/connections 2023-08-09T13:55:37.876 INFO:teuthology.orchestra.run.smithi043.stderr:mount: /sys/fs/fuse/connections: /sys/fs/fuse/connections already mounted or mount point busy. 2023-08-09T13:55:37.879 DEBUG:teuthology.orchestra.run:got remote process result: 32 2023-08-09T13:55:37.880 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-09T13:55:37.880 DEBUG:teuthology.orchestra.run.smithi043:> ls /sys/fs/fuse/connections 2023-08-09T13:55:37.909 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi043.stderr:2023-08-09T13:55:37.907+0000 7efdcb453480 -1 init, newargv = 0x55969205d590 newargc=15 2023-08-09T13:55:37.909 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi043.stderr:ceph-fuse[133240]: starting ceph client 2023-08-09T13:55:37.919 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi043.stderr:ceph-fuse[133240]: starting fuse 2023-08-09T13:55:37.928 INFO:teuthology.orchestra.run.smithi043.stdout:40 2023-08-09T13:55:37.929 INFO:tasks.cephfs.fuse_mount:Post-mount connections: [40] 2023-08-09T13:55:37.929 DEBUG:teuthology.orchestra.run.smithi043:> sudo stdin-killer -- python3 -c ' 2023-08-09T13:55:37.930 DEBUG:teuthology.orchestra.run.smithi043:> import glob 2023-08-09T13:55:37.930 DEBUG:teuthology.orchestra.run.smithi043:> import re 2023-08-09T13:55:37.930 DEBUG:teuthology.orchestra.run.smithi043:> import os 2023-08-09T13:55:37.930 DEBUG:teuthology.orchestra.run.smithi043:> import subprocess 2023-08-09T13:55:37.931 DEBUG:teuthology.orchestra.run.smithi043:> 2023-08-09T13:55:37.931 DEBUG:teuthology.orchestra.run.smithi043:> def _find_admin_socket(client_name): 2023-08-09T13:55:37.931 DEBUG:teuthology.orchestra.run.smithi043:> asok_path = "/var/run/ceph/ceph-client.0.*.asok" 2023-08-09T13:55:37.932 DEBUG:teuthology.orchestra.run.smithi043:> files = glob.glob(asok_path) 2023-08-09T13:55:37.932 DEBUG:teuthology.orchestra.run.smithi043:> mountpoint = "/home/ubuntu/cephtest/mnt.0" 2023-08-09T13:55:37.932 DEBUG:teuthology.orchestra.run.smithi043:> 2023-08-09T13:55:37.933 DEBUG:teuthology.orchestra.run.smithi043:> # Given a non-glob path, it better be there 2023-08-09T13:55:37.933 DEBUG:teuthology.orchestra.run.smithi043:> if "*" not in asok_path: 2023-08-09T13:55:37.933 DEBUG:teuthology.orchestra.run.smithi043:> assert(len(files) == 1) 2023-08-09T13:55:37.934 DEBUG:teuthology.orchestra.run.smithi043:> return files[0] 2023-08-09T13:55:37.934 DEBUG:teuthology.orchestra.run.smithi043:> 2023-08-09T13:55:37.934 DEBUG:teuthology.orchestra.run.smithi043:> for f in files: 2023-08-09T13:55:37.935 DEBUG:teuthology.orchestra.run.smithi043:> pid = re.match(".*\.(\d+)\.asok$", f).group(1) 2023-08-09T13:55:37.935 DEBUG:teuthology.orchestra.run.smithi043:> if os.path.exists("/proc/{0}".format(pid)): 2023-08-09T13:55:37.935 DEBUG:teuthology.orchestra.run.smithi043:> with open("/proc/{0}/cmdline".format(pid), '"'"'r'"'"') as proc_f: 2023-08-09T13:55:37.935 DEBUG:teuthology.orchestra.run.smithi043:> contents = proc_f.read() 2023-08-09T13:55:37.936 DEBUG:teuthology.orchestra.run.smithi043:> if mountpoint in contents: 2023-08-09T13:55:37.936 DEBUG:teuthology.orchestra.run.smithi043:> return f 2023-08-09T13:55:37.936 DEBUG:teuthology.orchestra.run.smithi043:> raise RuntimeError("Client socket {0} not found".format(client_name)) 2023-08-09T13:55:37.937 DEBUG:teuthology.orchestra.run.smithi043:> 2023-08-09T13:55:37.937 DEBUG:teuthology.orchestra.run.smithi043:> print(_find_admin_socket("client.0")) 2023-08-09T13:55:37.937 DEBUG:teuthology.orchestra.run.smithi043:> ' 2023-08-09T13:55:38.149 INFO:teuthology.orchestra.run.smithi043.stdout:/var/run/ceph/ceph-client.0.133240.asok 2023-08-09T13:55:38.153 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:38 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2023-08-09T13:55:38.166 INFO:tasks.cephfs.fuse_mount:Found client admin socket at /var/run/ceph/ceph-client.0.133240.asok 2023-08-09T13:55:38.167 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-09T13:55:38.167 DEBUG:teuthology.orchestra.run.smithi043:> sudo ceph --admin-daemon /var/run/ceph/ceph-client.0.133240.asok status 2023-08-09T13:55:38.416 INFO:teuthology.orchestra.run.smithi043.stdout:{ 2023-08-09T13:55:38.416 INFO:teuthology.orchestra.run.smithi043.stdout: "metadata": { 2023-08-09T13:55:38.417 INFO:teuthology.orchestra.run.smithi043.stdout: "ceph_sha1": "668a86f232c9454b2ce5928d1432a29a72964d85", 2023-08-09T13:55:38.417 INFO:teuthology.orchestra.run.smithi043.stdout: "ceph_version": "ceph version 18.0.0-5389-g668a86f2 (668a86f232c9454b2ce5928d1432a29a72964d85) reef (dev)", 2023-08-09T13:55:38.417 INFO:teuthology.orchestra.run.smithi043.stdout: "entity_id": "0", 2023-08-09T13:55:38.417 INFO:teuthology.orchestra.run.smithi043.stdout: "hostname": "smithi043", 2023-08-09T13:55:38.417 INFO:teuthology.orchestra.run.smithi043.stdout: "mount_point": "/home/ubuntu/cephtest/mnt.0", 2023-08-09T13:55:38.417 INFO:teuthology.orchestra.run.smithi043.stdout: "pid": "133240", 2023-08-09T13:55:38.417 INFO:teuthology.orchestra.run.smithi043.stdout: "root": "/" 2023-08-09T13:55:38.418 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2023-08-09T13:55:38.418 INFO:teuthology.orchestra.run.smithi043.stdout: "dentry_count": 0, 2023-08-09T13:55:38.418 INFO:teuthology.orchestra.run.smithi043.stdout: "dentry_pinned_count": 0, 2023-08-09T13:55:38.418 INFO:teuthology.orchestra.run.smithi043.stdout: "id": 9285, 2023-08-09T13:55:38.418 INFO:teuthology.orchestra.run.smithi043.stdout: "inst": { 2023-08-09T13:55:38.418 INFO:teuthology.orchestra.run.smithi043.stdout: "name": { 2023-08-09T13:55:38.418 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "client", 2023-08-09T13:55:38.419 INFO:teuthology.orchestra.run.smithi043.stdout: "num": 9285 2023-08-09T13:55:38.419 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2023-08-09T13:55:38.419 INFO:teuthology.orchestra.run.smithi043.stdout: "addr": { 2023-08-09T13:55:38.419 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "v1", 2023-08-09T13:55:38.419 INFO:teuthology.orchestra.run.smithi043.stdout: "addr": "192.168.0.1:0", 2023-08-09T13:55:38.419 INFO:teuthology.orchestra.run.smithi043.stdout: "nonce": 281762577 2023-08-09T13:55:38.419 INFO:teuthology.orchestra.run.smithi043.stdout: } 2023-08-09T13:55:38.420 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2023-08-09T13:55:38.420 INFO:teuthology.orchestra.run.smithi043.stdout: "addr": { 2023-08-09T13:55:38.420 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "v1", 2023-08-09T13:55:38.420 INFO:teuthology.orchestra.run.smithi043.stdout: "addr": "192.168.0.1:0", 2023-08-09T13:55:38.420 INFO:teuthology.orchestra.run.smithi043.stdout: "nonce": 281762577 2023-08-09T13:55:38.420 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2023-08-09T13:55:38.420 INFO:teuthology.orchestra.run.smithi043.stdout: "inst_str": "client.9285 192.168.0.1:0/281762577", 2023-08-09T13:55:38.421 INFO:teuthology.orchestra.run.smithi043.stdout: "addr_str": "192.168.0.1:0/281762577", 2023-08-09T13:55:38.421 INFO:teuthology.orchestra.run.smithi043.stdout: "inode_count": 1, 2023-08-09T13:55:38.421 INFO:teuthology.orchestra.run.smithi043.stdout: "mds_epoch": 70, 2023-08-09T13:55:38.421 INFO:teuthology.orchestra.run.smithi043.stdout: "osd_epoch": 147, 2023-08-09T13:55:38.421 INFO:teuthology.orchestra.run.smithi043.stdout: "osd_epoch_barrier": 0, 2023-08-09T13:55:38.421 INFO:teuthology.orchestra.run.smithi043.stdout: "blocklisted": false, 2023-08-09T13:55:38.421 INFO:teuthology.orchestra.run.smithi043.stdout: "fs_name": "cephfs" 2023-08-09T13:55:38.422 INFO:teuthology.orchestra.run.smithi043.stdout:} 2023-08-09T13:55:38.432 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-09T13:55:38.432 DEBUG:teuthology.orchestra.run.smithi043:> stat --file-system '--printf=%T 2023-08-09T13:55:38.433 DEBUG:teuthology.orchestra.run.smithi043:> ' -- /home/ubuntu/cephtest/mnt.0 2023-08-09T13:55:38.488 INFO:teuthology.orchestra.run.smithi043.stdout:fuseblk 2023-08-09T13:55:38.488 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.0 2023-08-09T13:55:38.489 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-09T13:55:38.489 DEBUG:teuthology.orchestra.run.smithi043:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2023-08-09T13:55:38.595 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs volume ls 2023-08-09T13:55:38.835 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:38.833+0000 7f9037247640 1 Processor -- start 2023-08-09T13:55:38.836 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:38.833+0000 7f9037247640 1 -- start start 2023-08-09T13:55:38.837 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:38.835+0000 7f9037247640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f90301546f0 0x7f903015e640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:55:38.837 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:38.835+0000 7f9037247640 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7f903015b460 0x7f903006b3d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:55:38.838 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:38.835+0000 7f9037247640 1 -- --> v1:172.21.15.43:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f903006d770 con 0x7f9030070ab0 2023-08-09T13:55:38.838 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:38.835+0000 7f9037247640 1 -- --> v2:172.21.15.203:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f903006cf30 con 0x7f903015b460 2023-08-09T13:55:38.838 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:38.835+0000 7f9037247640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f903006d1b0 con 0x7f90301546f0 2023-08-09T13:55:38.839 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:38.835+0000 7f9036a46640 1 --1- >> v1:172.21.15.43:6789/0 conn(0x7f9030070ab0 0x7f90300c84b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.43:6789/0 says I am v1:172.21.15.43:56848/0 (socket says 172.21.15.43:56848) 2023-08-09T13:55:38.839 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:38.835+0000 7f9036a46640 1 -- 172.21.15.43:0/2953138082 learned_addr learned my addr 172.21.15.43:0/2953138082 (peer_addr_for_me v1:172.21.15.43:0/0) 2023-08-09T13:55:38.839 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:38.836+0000 7f9035a44640 1 --2- 172.21.15.43:0/2953138082 >> v2:172.21.15.203:3300/0 conn(0x7f903015b460 0x7f903006b3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:55:38.840 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:38.836+0000 7f9036245640 1 --2- 172.21.15.43:0/2953138082 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f90301546f0 0x7f903015e640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:55:38.840 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:38.836+0000 7f9035243640 1 -- 172.21.15.43:0/2953138082 <== mon.0 v1:172.21.15.43:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2634270730 0 0) 0x7f9028001490 con 0x7f9030070ab0 2023-08-09T13:55:38.840 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:38.836+0000 7f9035243640 1 -- 172.21.15.43:0/2953138082 --> v1:172.21.15.43:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f901c003520 con 0x7f9030070ab0 2023-08-09T13:55:38.840 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:38.836+0000 7f9035243640 1 -- 172.21.15.43:0/2953138082 <== mon.0 v1:172.21.15.43:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 4026489487 0 0) 0x7f901c003520 con 0x7f9030070ab0 2023-08-09T13:55:38.840 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:38.836+0000 7f9035243640 1 -- 172.21.15.43:0/2953138082 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f90301546f0 msgr2=0x7f903015e640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-09T13:55:38.840 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:38.836+0000 7f9035243640 1 --2- 172.21.15.43:0/2953138082 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f90301546f0 0x7f903015e640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:55:38.840 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:38.836+0000 7f9035243640 1 -- 172.21.15.43:0/2953138082 >> v2:172.21.15.203:3300/0 conn(0x7f903015b460 msgr2=0x7f903006b3d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-09T13:55:38.840 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:38.836+0000 7f9035243640 1 --2- 172.21.15.43:0/2953138082 >> v2:172.21.15.203:3300/0 conn(0x7f903015b460 0x7f903006b3d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:55:38.841 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:38.836+0000 7f9035243640 1 -- 172.21.15.43:0/2953138082 --> v1:172.21.15.43:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f903006d490 con 0x7f9030070ab0 2023-08-09T13:55:38.841 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:38.836+0000 7f9035243640 1 -- 172.21.15.43:0/2953138082 <== mon.0 v1:172.21.15.43:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1285150740 0 0) 0x7f9028001b80 con 0x7f9030070ab0 2023-08-09T13:55:38.841 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:38.836+0000 7f9035243640 1 -- 172.21.15.43:0/2953138082 >> v1:172.21.15.43:6789/0 conn(0x7f9030070ab0 legacy=0x7f90300c84b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:55:38.841 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:38.837+0000 7f9035243640 1 --2- 172.21.15.43:0/2953138082 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f901c003970 0x7f901c003d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:55:38.841 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:38.837+0000 7f9035243640 1 --2- 172.21.15.43:0/2953138082 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f90301546f0 0x7f901c0047c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:55:38.841 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:38.837+0000 7f9035243640 1 --2- 172.21.15.43:0/2953138082 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f901c004d00 0x7f901c005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:55:38.841 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:38.837+0000 7f9035243640 1 -- 172.21.15.43:0/2953138082 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f903006cf30 con 0x7f901c003970 2023-08-09T13:55:38.842 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:38.837+0000 7f9035243640 1 -- 172.21.15.43:0/2953138082 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f903006d1b0 con 0x7f901c004d00 2023-08-09T13:55:38.842 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:38.837+0000 7f9035243640 1 -- 172.21.15.43:0/2953138082 --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f901c00b980 con 0x7f90301546f0 2023-08-09T13:55:38.842 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:38.837+0000 7f9036245640 1 --2- 172.21.15.43:0/2953138082 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f901c003970 0x7f901c003d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:55:38.843 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:38.837+0000 7f9036245640 1 -- 172.21.15.43:0/2953138082 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f90301546f0 msgr2=0x7f901c0047c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-09T13:55:38.843 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:38.837+0000 7f9036245640 1 --2- 172.21.15.43:0/2953138082 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f90301546f0 0x7f901c0047c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:55:38.843 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:38.837+0000 7f9036245640 1 -- 172.21.15.43:0/2953138082 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f901c004d00 msgr2=0x7f901c005180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-09T13:55:38.843 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:38.837+0000 7f9036245640 1 --2- 172.21.15.43:0/2953138082 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f901c004d00 0x7f901c005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:55:38.843 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:38.837+0000 7f9036245640 1 -- 172.21.15.43:0/2953138082 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f901c003520 con 0x7f901c003970 2023-08-09T13:55:38.844 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:38.838+0000 7f9036245640 1 --2- 172.21.15.43:0/2953138082 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f901c003970 0x7f901c003d90 secure :-1 s=READY pgs=1718 cs=0 l=1 rev1=1 crypto rx=0x7f902c009ac0 tx=0x7f901c0057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=99206741d83707b1 server_cookie=0 in_seq=0 out_seq=0 2023-08-09T13:55:38.844 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:38.839+0000 7f9035243640 1 -- 172.21.15.43:0/2953138082 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f902c02c6f0 con 0x7f901c003970 2023-08-09T13:55:38.844 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:38.839+0000 7f9035243640 1 -- 172.21.15.43:0/2953138082 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f902c02c850 con 0x7f901c003970 2023-08-09T13:55:38.844 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:38.839+0000 7f9035243640 1 -- 172.21.15.43:0/2953138082 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f902c02cbe0 con 0x7f901c003970 2023-08-09T13:55:38.844 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:38.839+0000 7f9037247640 1 -- 172.21.15.43:0/2953138082 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f901c003970 msgr2=0x7f901c003d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:55:38.844 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:38.839+0000 7f9037247640 1 --2- 172.21.15.43:0/2953138082 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f901c003970 0x7f901c003d90 secure :-1 s=READY pgs=1718 cs=0 l=1 rev1=1 crypto rx=0x7f902c009ac0 tx=0x7f901c0057f0 comp rx=0 tx=0).stop 2023-08-09T13:55:38.844 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:38.839+0000 7f9036a46640 1 -- 172.21.15.43:0/2953138082 reap_dead start 2023-08-09T13:55:38.844 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:38.840+0000 7f9037247640 1 -- 172.21.15.43:0/2953138082 shutdown_connections 2023-08-09T13:55:38.845 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:38.840+0000 7f9037247640 1 -- 172.21.15.43:0/2953138082 >> 172.21.15.43:0/2953138082 conn(0x7f90300af950 msgr2=0x7f90300afd60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-09T13:55:38.845 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:38.840+0000 7f9037247640 1 -- 172.21.15.43:0/2953138082 shutdown_connections 2023-08-09T13:55:38.845 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:38.840+0000 7f9037247640 1 -- 172.21.15.43:0/2953138082 wait complete. 2023-08-09T13:55:38.845 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:38.842+0000 7f9037247640 1 Processor -- start 2023-08-09T13:55:38.846 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:38.843+0000 7f9037247640 1 -- start start 2023-08-09T13:55:38.846 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:38.844+0000 7f9037247640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f90300c7620 0x7f9030153660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:55:38.847 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:38.844+0000 7f9037247640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f90300c8f70 0x7f90300c93f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:55:38.847 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:38.844+0000 7f9037247640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f90300c3a20 0x7f90300c3ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:55:38.847 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:38.844+0000 7f9037247640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f90300c43e0 con 0x7f90300c7620 2023-08-09T13:55:38.847 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:38.844+0000 7f9037247640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f90301531d0 con 0x7f90300c3a20 2023-08-09T13:55:38.847 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:38.844+0000 7f9037247640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f90300c9a30 con 0x7f90300c8f70 2023-08-09T13:55:38.847 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:38.844+0000 7f9036245640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f90300c7620 0x7f9030153660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:55:38.847 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:38.844+0000 7f9036245640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f90300c7620 0x7f9030153660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:53302/0 (socket says 172.21.15.43:53302) 2023-08-09T13:55:38.848 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:38.844+0000 7f9036245640 1 -- 172.21.15.43:0/3011632200 learned_addr learned my addr 172.21.15.43:0/3011632200 (peer_addr_for_me v2:172.21.15.43:0/0) 2023-08-09T13:55:38.849 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:38.845+0000 7f9035a44640 1 --2- 172.21.15.43:0/3011632200 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f90300c8f70 0x7f90300c93f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:55:38.849 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:38.845+0000 7f9036245640 1 -- 172.21.15.43:0/3011632200 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f90300c8f70 msgr2=0x7f90300c93f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:55:38.849 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:38.845+0000 7f9036245640 1 --2- 172.21.15.43:0/3011632200 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f90300c8f70 0x7f90300c93f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:55:38.849 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:38.845+0000 7f9036245640 1 -- 172.21.15.43:0/3011632200 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f90300c3a20 msgr2=0x7f90300c3ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:55:38.849 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:38.845+0000 7f9036245640 1 --2- 172.21.15.43:0/3011632200 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f90300c3a20 0x7f90300c3ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:55:38.849 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:38.845+0000 7f9036245640 1 -- 172.21.15.43:0/3011632200 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f90300c9d10 con 0x7f90300c7620 2023-08-09T13:55:38.849 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:38.845+0000 7f9035a44640 1 --2- 172.21.15.43:0/3011632200 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f90300c8f70 0x7f90300c93f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-08-09T13:55:38.850 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:38.845+0000 7f9036245640 1 --2- 172.21.15.43:0/3011632200 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f90300c7620 0x7f9030153660 secure :-1 s=READY pgs=1719 cs=0 l=1 rev1=1 crypto rx=0x7f902c037040 tx=0x7f902c004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-09T13:55:38.850 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:38.845+0000 7f9026ffd640 1 -- 172.21.15.43:0/3011632200 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f902c003810 con 0x7f90300c7620 2023-08-09T13:55:38.850 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:38.845+0000 7f9026ffd640 1 -- 172.21.15.43:0/3011632200 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f902c003970 con 0x7f90300c7620 2023-08-09T13:55:38.850 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:38.845+0000 7f9026ffd640 1 -- 172.21.15.43:0/3011632200 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f902c003d00 con 0x7f90300c7620 2023-08-09T13:55:38.850 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:38.845+0000 7f9037247640 1 -- 172.21.15.43:0/3011632200 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9030152b80 con 0x7f90300c7620 2023-08-09T13:55:38.850 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:38.846+0000 7f9037247640 1 -- 172.21.15.43:0/3011632200 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f90300c7a00 con 0x7f90300c7620 2023-08-09T13:55:38.851 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:38.847+0000 7f9037247640 1 -- 172.21.15.43:0/3011632200 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9030065f90 con 0x7f90300c7620 2023-08-09T13:55:38.851 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:38.848+0000 7f9026ffd640 1 -- 172.21.15.43:0/3011632200 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 67) v1 ==== 88443+0+0 (secure 0 0 0) 0x7f902c03e440 con 0x7f90300c7620 2023-08-09T13:55:38.851 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:38.849+0000 7f9026ffd640 1 --2- 172.21.15.43:0/3011632200 >> [v2:172.21.15.203:6832/3490266242,v1:172.21.15.203:6833/3490266242] conn(0x7f901406e5b0 0x7f9014070a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:55:38.852 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:38.849+0000 7f9026ffd640 1 -- 172.21.15.43:0/3011632200 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(147..147 src has 1..147) v4 ==== 6426+0+0 (secure 0 0 0) 0x7f902c0aa2a0 con 0x7f90300c7620 2023-08-09T13:55:38.852 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:38.850+0000 7f9035a44640 1 --2- 172.21.15.43:0/3011632200 >> [v2:172.21.15.203:6832/3490266242,v1:172.21.15.203:6833/3490266242] conn(0x7f901406e5b0 0x7f9014070a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:55:38.853 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:38.851+0000 7f9035a44640 1 --2- 172.21.15.43:0/3011632200 >> [v2:172.21.15.203:6832/3490266242,v1:172.21.15.203:6833/3490266242] conn(0x7f901406e5b0 0x7f9014070a70 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f90300c4750 tx=0x7f9020008040 comp rx=0 tx=0).ready entity=mgr.6659 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-09T13:55:38.856 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:38.854+0000 7f9026ffd640 1 -- 172.21.15.43:0/3011632200 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178066 (secure 0 0 0) 0x7f902c075670 con 0x7f90300c7620 2023-08-09T13:55:39.072 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:39.070+0000 7f9037247640 1 -- 172.21.15.43:0/3011632200 --> [v2:172.21.15.203:6832/3490266242,v1:172.21.15.203:6833/3490266242] -- mgr_command(tid 0: {"prefix": "fs volume ls", "target": ["mon-mgr", ""]}) v1 -- 0x7f903015e120 con 0x7f901406e5b0 2023-08-09T13:55:39.076 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:39.074+0000 7f9026ffd640 1 -- 172.21.15.43:0/3011632200 <== mgr.6659 v2:172.21.15.203:6832/3490266242 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+41 (secure 0 0 0) 0x7f90200107f0 con 0x7f901406e5b0 2023-08-09T13:55:39.076 INFO:teuthology.orchestra.run.smithi043.stdout:[ 2023-08-09T13:55:39.077 INFO:teuthology.orchestra.run.smithi043.stdout: { 2023-08-09T13:55:39.077 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "cephfs" 2023-08-09T13:55:39.077 INFO:teuthology.orchestra.run.smithi043.stdout: } 2023-08-09T13:55:39.077 INFO:teuthology.orchestra.run.smithi043.stdout:] 2023-08-09T13:55:39.079 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:39.076+0000 7f9037247640 1 -- 172.21.15.43:0/3011632200 >> [v2:172.21.15.203:6832/3490266242,v1:172.21.15.203:6833/3490266242] conn(0x7f901406e5b0 msgr2=0x7f9014070a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:55:39.079 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:39.076+0000 7f9037247640 1 --2- 172.21.15.43:0/3011632200 >> [v2:172.21.15.203:6832/3490266242,v1:172.21.15.203:6833/3490266242] conn(0x7f901406e5b0 0x7f9014070a70 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f90300c4750 tx=0x7f9020008040 comp rx=0 tx=0).stop 2023-08-09T13:55:39.079 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:39.076+0000 7f9037247640 1 -- 172.21.15.43:0/3011632200 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f90300c7620 msgr2=0x7f9030153660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:55:39.079 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:39.076+0000 7f9037247640 1 --2- 172.21.15.43:0/3011632200 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f90300c7620 0x7f9030153660 secure :-1 s=READY pgs=1719 cs=0 l=1 rev1=1 crypto rx=0x7f902c037040 tx=0x7f902c004730 comp rx=0 tx=0).stop 2023-08-09T13:55:39.079 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:39.076+0000 7f9037247640 1 -- 172.21.15.43:0/3011632200 shutdown_connections 2023-08-09T13:55:39.080 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:39.076+0000 7f9037247640 1 --2- 172.21.15.43:0/3011632200 >> [v2:172.21.15.203:6832/3490266242,v1:172.21.15.203:6833/3490266242] conn(0x7f901406e5b0 0x7f9014070a70 unknown :-1 s=CLOSED pgs=70 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:55:39.080 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:39.076+0000 7f9037247640 1 --2- 172.21.15.43:0/3011632200 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f90300c3a20 0x7f90300c3ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:55:39.080 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:39.076+0000 7f9037247640 1 --2- 172.21.15.43:0/3011632200 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f90300c8f70 0x7f90300c93f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:55:39.080 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:39.076+0000 7f9037247640 1 --2- 172.21.15.43:0/3011632200 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f90300c7620 0x7f9030153660 unknown :-1 s=CLOSED pgs=1719 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:55:39.080 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:39.076+0000 7f9037247640 1 -- 172.21.15.43:0/3011632200 >> 172.21.15.43:0/3011632200 conn(0x7f90300af950 msgr2=0x7f9030150210 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-09T13:55:39.080 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:39.076+0000 7f9037247640 1 -- 172.21.15.43:0/3011632200 shutdown_connections 2023-08-09T13:55:39.081 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:39.076+0000 7f9037247640 1 -- 172.21.15.43:0/3011632200 wait complete. 2023-08-09T13:55:39.098 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph mgr module enable snap_schedule 2023-08-09T13:55:39.319 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:39.317+0000 7f7d7363f640 1 Processor -- start 2023-08-09T13:55:39.320 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:39.317+0000 7f7d7363f640 1 -- start start 2023-08-09T13:55:39.321 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:39.319+0000 7f7d7363f640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f7d6c143c20 0x7f7d6c06f540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:55:39.321 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:39.319+0000 7f7d7363f640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f7d6c0c86f0 0x7f7d6c0c8b50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:55:39.322 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:39.319+0000 7f7d7363f640 1 -- --> v1:172.21.15.43:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7d6c0c9090 con 0x7f7d6c06c8d0 2023-08-09T13:55:39.322 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:39.319+0000 7f7d7363f640 1 -- --> v2:172.21.15.43:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7d6c144000 con 0x7f7d6c0c86f0 2023-08-09T13:55:39.322 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:39.319+0000 7f7d7363f640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7d6c06c440 con 0x7f7d6c143c20 2023-08-09T13:55:39.323 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:39.319+0000 7f7d70bda640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f7d6c0c86f0 0x7f7d6c0c8b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:55:39.323 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:39.319+0000 7f7d71bdc640 1 --1- >> v1:172.21.15.43:6789/0 conn(0x7f7d6c06c8d0 0x7f7d6c06cd60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.43:6789/0 says I am v1:172.21.15.43:56856/0 (socket says 172.21.15.43:56856) 2023-08-09T13:55:39.323 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:39.319+0000 7f7d70bda640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f7d6c0c86f0 0x7f7d6c0c8b50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:53316/0 (socket says 172.21.15.43:53316) 2023-08-09T13:55:39.323 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:39.319+0000 7f7d71bdc640 1 -- 172.21.15.43:0/1572855590 learned_addr learned my addr 172.21.15.43:0/1572855590 (peer_addr_for_me v1:172.21.15.43:0/0) 2023-08-09T13:55:39.323 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:39.320+0000 7f7d713db640 1 --2- 172.21.15.43:0/1572855590 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f7d6c143c20 0x7f7d6c06f540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:55:39.323 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:39.320+0000 7f7d70bda640 1 -- 172.21.15.43:0/1572855590 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f7d6c143c20 msgr2=0x7f7d6c06f540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-09T13:55:39.323 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:39.320+0000 7f7d70bda640 1 --2- 172.21.15.43:0/1572855590 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f7d6c143c20 0x7f7d6c06f540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:55:39.324 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:39.320+0000 7f7d70bda640 1 -- 172.21.15.43:0/1572855590 >> v1:172.21.15.43:6789/0 conn(0x7f7d6c06c8d0 legacy=0x7f7d6c06cd60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:55:39.324 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:39.320+0000 7f7d70bda640 1 -- 172.21.15.43:0/1572855590 --> v2:172.21.15.43:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7d6c06b830 con 0x7f7d6c0c86f0 2023-08-09T13:55:39.324 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:39.320+0000 7f7d713db640 1 --2- 172.21.15.43:0/1572855590 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f7d6c143c20 0x7f7d6c06f540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-08-09T13:55:39.325 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:39.321+0000 7f7d70bda640 1 --2- 172.21.15.43:0/1572855590 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f7d6c0c86f0 0x7f7d6c0c8b50 secure :-1 s=READY pgs=1721 cs=0 l=1 rev1=1 crypto rx=0x7f7d5c0097a0 tx=0x7f7d5c02b820 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f89804ada65a356e server_cookie=0 in_seq=0 out_seq=0 2023-08-09T13:55:39.325 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:39.321+0000 7f7d63fff640 1 -- 172.21.15.43:0/1572855590 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7d5c031780 con 0x7f7d6c0c86f0 2023-08-09T13:55:39.325 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:39.321+0000 7f7d63fff640 1 -- 172.21.15.43:0/1572855590 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7d5c0318e0 con 0x7f7d6c0c86f0 2023-08-09T13:55:39.325 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:39.321+0000 7f7d63fff640 1 -- 172.21.15.43:0/1572855590 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7d5c031c70 con 0x7f7d6c0c86f0 2023-08-09T13:55:39.326 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:39.322+0000 7f7d7363f640 1 -- 172.21.15.43:0/1572855590 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f7d6c0c86f0 msgr2=0x7f7d6c0c8b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:55:39.326 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:39.322+0000 7f7d7363f640 1 --2- 172.21.15.43:0/1572855590 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f7d6c0c86f0 0x7f7d6c0c8b50 secure :-1 s=READY pgs=1721 cs=0 l=1 rev1=1 crypto rx=0x7f7d5c0097a0 tx=0x7f7d5c02b820 comp rx=0 tx=0).stop 2023-08-09T13:55:39.326 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:39.322+0000 7f7d7363f640 1 -- 172.21.15.43:0/1572855590 shutdown_connections 2023-08-09T13:55:39.326 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:39.322+0000 7f7d7363f640 1 --2- 172.21.15.43:0/1572855590 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f7d6c0c86f0 0x7f7d6c0c8b50 unknown :-1 s=CLOSED pgs=1721 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:55:39.326 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:39.322+0000 7f7d7363f640 1 --2- 172.21.15.43:0/1572855590 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f7d6c143c20 0x7f7d6c06f540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:55:39.326 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:39.322+0000 7f7d7363f640 1 -- 172.21.15.43:0/1572855590 >> 172.21.15.43:0/1572855590 conn(0x7f7d6c0ce0a0 msgr2=0x7f7d6c0ce480 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-09T13:55:39.326 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:39.322+0000 7f7d7363f640 1 -- 172.21.15.43:0/1572855590 shutdown_connections 2023-08-09T13:55:39.327 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:39.322+0000 7f7d7363f640 1 -- 172.21.15.43:0/1572855590 wait complete. 2023-08-09T13:55:39.328 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:39.324+0000 7f7d7363f640 1 Processor -- start 2023-08-09T13:55:39.328 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:39.324+0000 7f7d7363f640 1 -- start start 2023-08-09T13:55:39.328 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:39.325+0000 7f7d7363f640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f7d6c06c8d0 0x7f7d6c142cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:55:39.328 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:39.325+0000 7f7d7363f640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f7d6c0c86f0 0x7f7d6c15d1e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:55:39.328 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:39.325+0000 7f7d7363f640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f7d6c143c20 0x7f7d6c13fa50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:55:39.328 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:39.325+0000 7f7d7363f640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7d6c15cd50 con 0x7f7d6c06c8d0 2023-08-09T13:55:39.329 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:39.325+0000 7f7d7363f640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7d6c15c140 con 0x7f7d6c143c20 2023-08-09T13:55:39.329 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:39.325+0000 7f7d7363f640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7d6c15c420 con 0x7f7d6c0c86f0 2023-08-09T13:55:39.329 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:39.325+0000 7f7d713db640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f7d6c06c8d0 0x7f7d6c142cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:55:39.329 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:39.325+0000 7f7d713db640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f7d6c06c8d0 0x7f7d6c142cf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:53332/0 (socket says 172.21.15.43:53332) 2023-08-09T13:55:39.329 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:39.325+0000 7f7d713db640 1 -- 172.21.15.43:0/3396015869 learned_addr learned my addr 172.21.15.43:0/3396015869 (peer_addr_for_me v2:172.21.15.43:0/0) 2023-08-09T13:55:39.330 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:39.326+0000 7f7d71bdc640 1 --2- 172.21.15.43:0/3396015869 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f7d6c143c20 0x7f7d6c13fa50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:55:39.330 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:39.326+0000 7f7d70bda640 1 --2- 172.21.15.43:0/3396015869 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f7d6c0c86f0 0x7f7d6c15d1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:55:39.330 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:39.326+0000 7f7d713db640 1 -- 172.21.15.43:0/3396015869 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f7d6c0c86f0 msgr2=0x7f7d6c15d1e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:55:39.331 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:39.326+0000 7f7d713db640 1 --2- 172.21.15.43:0/3396015869 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f7d6c0c86f0 0x7f7d6c15d1e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:55:39.331 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:39.326+0000 7f7d713db640 1 -- 172.21.15.43:0/3396015869 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f7d6c143c20 msgr2=0x7f7d6c13fa50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:55:39.331 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:39.326+0000 7f7d713db640 1 --2- 172.21.15.43:0/3396015869 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f7d6c143c20 0x7f7d6c13fa50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:55:39.331 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:39.326+0000 7f7d713db640 1 -- 172.21.15.43:0/3396015869 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7d5c009400 con 0x7f7d6c06c8d0 2023-08-09T13:55:39.331 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:39.326+0000 7f7d71bdc640 1 --2- 172.21.15.43:0/3396015869 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f7d6c143c20 0x7f7d6c13fa50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-08-09T13:55:39.331 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:39.326+0000 7f7d713db640 1 --2- 172.21.15.43:0/3396015869 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f7d6c06c8d0 0x7f7d6c142cf0 secure :-1 s=READY pgs=1722 cs=0 l=1 rev1=1 crypto rx=0x7f7d6800c7f0 tx=0x7f7d6800cbf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-09T13:55:39.331 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:39.326+0000 7f7d61ffb640 1 -- 172.21.15.43:0/3396015869 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7d68007a30 con 0x7f7d6c06c8d0 2023-08-09T13:55:39.331 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:39.327+0000 7f7d61ffb640 1 -- 172.21.15.43:0/3396015869 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7d68007b90 con 0x7f7d6c06c8d0 2023-08-09T13:55:39.332 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:39.327+0000 7f7d61ffb640 1 -- 172.21.15.43:0/3396015869 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7d6800ce40 con 0x7f7d6c06c8d0 2023-08-09T13:55:39.332 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:39.327+0000 7f7d7363f640 1 -- 172.21.15.43:0/3396015869 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7d6c15c700 con 0x7f7d6c06c8d0 2023-08-09T13:55:39.332 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:39.327+0000 7f7d7363f640 1 -- 172.21.15.43:0/3396015869 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7d6c13ff90 con 0x7f7d6c06c8d0 2023-08-09T13:55:39.332 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:39.328+0000 7f7d7363f640 1 -- 172.21.15.43:0/3396015869 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7d6c0739f0 con 0x7f7d6c06c8d0 2023-08-09T13:55:39.333 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:39.329+0000 7f7d61ffb640 1 -- 172.21.15.43:0/3396015869 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 67) v1 ==== 88443+0+0 (secure 0 0 0) 0x7f7d680113f0 con 0x7f7d6c06c8d0 2023-08-09T13:55:39.333 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:39.330+0000 7f7d61ffb640 1 --2- 172.21.15.43:0/3396015869 >> [v2:172.21.15.203:6832/3490266242,v1:172.21.15.203:6833/3490266242] conn(0x7f7d5006a170 0x7f7d5006c630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:55:39.333 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:39.331+0000 7f7d61ffb640 1 -- 172.21.15.43:0/3396015869 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(147..147 src has 1..147) v4 ==== 6426+0+0 (secure 0 0 0) 0x7f7d68088b70 con 0x7f7d6c06c8d0 2023-08-09T13:55:39.333 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:39.331+0000 7f7d70bda640 1 --2- 172.21.15.43:0/3396015869 >> [v2:172.21.15.203:6832/3490266242,v1:172.21.15.203:6833/3490266242] conn(0x7f7d5006a170 0x7f7d5006c630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:55:39.335 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:39.333+0000 7f7d70bda640 1 --2- 172.21.15.43:0/3396015869 >> [v2:172.21.15.203:6832/3490266242,v1:172.21.15.203:6833/3490266242] conn(0x7f7d5006a170 0x7f7d5006c630 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f7d5c036000 tx=0x7f7d5c03d040 comp rx=0 tx=0).ready entity=mgr.6659 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-09T13:55:39.337 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:39.335+0000 7f7d61ffb640 1 -- 172.21.15.43:0/3396015869 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178066 (secure 0 0 0) 0x7f7d68053e30 con 0x7f7d6c06c8d0 2023-08-09T13:55:39.584 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:39.582+0000 7f7d7363f640 1 -- 172.21.15.43:0/3396015869 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "mgr module enable", "module": "snap_schedule"} v 0) v1 -- 0x7f7d6c0cec90 con 0x7f7d6c06c8d0 2023-08-09T13:55:40.467 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:40.465+0000 7f7d61ffb640 1 -- 172.21.15.43:0/3396015869 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mgrmap(e 68) v1 ==== 88460+0+0 (secure 0 0 0) 0x7f7d68054200 con 0x7f7d6c06c8d0 2023-08-09T13:55:40.471 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:40.469+0000 7f7d61ffb640 1 -- 172.21.15.43:0/3396015869 <== mon.0 v2:172.21.15.43:3300/0 8 ==== mon_command_ack([{"prefix": "mgr module enable", "module": "snap_schedule"}]=0 v68) v1 ==== 92+0+0 (secure 0 0 0) 0x7f7d68053e30 con 0x7f7d6c06c8d0 2023-08-09T13:55:40.475 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:40.473+0000 7f7d7363f640 1 -- 172.21.15.43:0/3396015869 >> [v2:172.21.15.203:6832/3490266242,v1:172.21.15.203:6833/3490266242] conn(0x7f7d5006a170 msgr2=0x7f7d5006c630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:55:40.475 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:40.473+0000 7f7d7363f640 1 --2- 172.21.15.43:0/3396015869 >> [v2:172.21.15.203:6832/3490266242,v1:172.21.15.203:6833/3490266242] conn(0x7f7d5006a170 0x7f7d5006c630 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f7d5c036000 tx=0x7f7d5c03d040 comp rx=0 tx=0).stop 2023-08-09T13:55:40.476 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:40.473+0000 7f7d7363f640 1 -- 172.21.15.43:0/3396015869 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f7d6c06c8d0 msgr2=0x7f7d6c142cf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:55:40.476 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:40.473+0000 7f7d7363f640 1 --2- 172.21.15.43:0/3396015869 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f7d6c06c8d0 0x7f7d6c142cf0 secure :-1 s=READY pgs=1722 cs=0 l=1 rev1=1 crypto rx=0x7f7d6800c7f0 tx=0x7f7d6800cbf0 comp rx=0 tx=0).stop 2023-08-09T13:55:40.476 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:40.473+0000 7f7d7363f640 1 -- 172.21.15.43:0/3396015869 shutdown_connections 2023-08-09T13:55:40.476 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:40.473+0000 7f7d7363f640 1 --2- 172.21.15.43:0/3396015869 >> [v2:172.21.15.203:6832/3490266242,v1:172.21.15.203:6833/3490266242] conn(0x7f7d5006a170 0x7f7d5006c630 unknown :-1 s=CLOSED pgs=71 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:55:40.476 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:40.473+0000 7f7d7363f640 1 --2- 172.21.15.43:0/3396015869 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f7d6c143c20 0x7f7d6c13fa50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:55:40.476 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:40.473+0000 7f7d7363f640 1 --2- 172.21.15.43:0/3396015869 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f7d6c0c86f0 0x7f7d6c15d1e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:55:40.476 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:40.473+0000 7f7d7363f640 1 --2- 172.21.15.43:0/3396015869 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f7d6c06c8d0 0x7f7d6c142cf0 unknown :-1 s=CLOSED pgs=1722 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:55:40.477 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:40.473+0000 7f7d7363f640 1 -- 172.21.15.43:0/3396015869 >> 172.21.15.43:0/3396015869 conn(0x7f7d6c0ce0a0 msgr2=0x7f7d6c0c9220 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-09T13:55:40.477 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:40.473+0000 7f7d7363f640 1 -- 172.21.15.43:0/3396015869 shutdown_connections 2023-08-09T13:55:40.477 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:40.474+0000 7f7d7363f640 1 -- 172.21.15.43:0/3396015869 wait complete. 2023-08-09T13:55:40.499 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mgr mgr/snap_schedule/allow_m_granularity True 2023-08-09T13:55:40.687 INFO:tasks.ceph.mgr.x.smithi203.stderr:2023-08-09T13:55:40.685+0000 7f21a2445040 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2023-08-09T13:55:40.694 INFO:tasks.ceph.mgr.y.smithi043.stderr:2023-08-09T13:55:40.691+0000 7fb823a2b040 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2023-08-09T13:55:40.709 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:40.707+0000 7f5945eb8640 1 Processor -- start 2023-08-09T13:55:40.710 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:40.707+0000 7f5945eb8640 1 -- start start 2023-08-09T13:55:40.711 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:40.709+0000 7f5945eb8640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f59400b45f0 0x7f59400cd6d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:55:40.711 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:40.709+0000 7f5945eb8640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f59400d04b0 0x7f59400b59d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:55:40.712 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:40.709+0000 7f5945eb8640 1 -- --> v1:172.21.15.43:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f59400b7bd0 con 0x7f594015c550 2023-08-09T13:55:40.712 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:40.709+0000 7f5945eb8640 1 -- --> v2:172.21.15.43:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f59400d1c00 con 0x7f59400b45f0 2023-08-09T13:55:40.712 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:40.709+0000 7f5945eb8640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f594015c930 con 0x7f59400d04b0 2023-08-09T13:55:40.713 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:40.709+0000 7f593f7fe640 1 --1- >> v1:172.21.15.43:6789/0 conn(0x7f594015c550 0x7f59400b3ee0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.43:6789/0 says I am v1:172.21.15.43:56882/0 (socket says 172.21.15.43:56882) 2023-08-09T13:55:40.713 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:40.709+0000 7f593f7fe640 1 -- 172.21.15.43:0/3459556772 learned_addr learned my addr 172.21.15.43:0/3459556772 (peer_addr_for_me v1:172.21.15.43:0/0) 2023-08-09T13:55:40.713 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:40.709+0000 7f5936ffd640 1 --2- 172.21.15.43:0/3459556772 >> v2:172.21.15.43:3300/0 conn(0x7f59400b45f0 0x7f59400cd6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:55:40.713 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:40.710+0000 7f593ffff640 1 --2- 172.21.15.43:0/3459556772 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f59400d04b0 0x7f59400b59d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:55:40.713 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:40.710+0000 7f5936ffd640 1 -- 172.21.15.43:0/3459556772 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f59400d04b0 msgr2=0x7f59400b59d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-09T13:55:40.713 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:40.710+0000 7f5936ffd640 1 --2- 172.21.15.43:0/3459556772 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f59400d04b0 0x7f59400b59d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:55:40.714 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:40.710+0000 7f5936ffd640 1 -- 172.21.15.43:0/3459556772 >> v1:172.21.15.43:6789/0 conn(0x7f594015c550 legacy=0x7f59400b3ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:55:40.714 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:40.710+0000 7f5936ffd640 1 -- 172.21.15.43:0/3459556772 --> v2:172.21.15.43:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f59400b54f0 con 0x7f59400b45f0 2023-08-09T13:55:40.714 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:40.712+0000 7f5936ffd640 1 --2- 172.21.15.43:0/3459556772 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f59400b45f0 0x7f59400cd6d0 secure :-1 s=READY pgs=1727 cs=0 l=1 rev1=1 crypto rx=0x7f59280097a0 tx=0x7f592802b8b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=77445f417683d004 server_cookie=0 in_seq=0 out_seq=0 2023-08-09T13:55:40.714 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:40.712+0000 7f593effd640 1 -- 172.21.15.43:0/3459556772 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f592802d040 con 0x7f59400b45f0 2023-08-09T13:55:40.715 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:40.712+0000 7f593effd640 1 -- 172.21.15.43:0/3459556772 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5928031810 con 0x7f59400b45f0 2023-08-09T13:55:40.715 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:40.712+0000 7f593effd640 1 -- 172.21.15.43:0/3459556772 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5928031ba0 con 0x7f59400b45f0 2023-08-09T13:55:40.715 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:40.713+0000 7f5945eb8640 1 -- 172.21.15.43:0/3459556772 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f59400b45f0 msgr2=0x7f59400cd6d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:55:40.716 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:40.713+0000 7f5945eb8640 1 --2- 172.21.15.43:0/3459556772 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f59400b45f0 0x7f59400cd6d0 secure :-1 s=READY pgs=1727 cs=0 l=1 rev1=1 crypto rx=0x7f59280097a0 tx=0x7f592802b8b0 comp rx=0 tx=0).stop 2023-08-09T13:55:40.716 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:40.713+0000 7f5945eb8640 1 -- 172.21.15.43:0/3459556772 shutdown_connections 2023-08-09T13:55:40.716 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:40.713+0000 7f5945eb8640 1 --2- 172.21.15.43:0/3459556772 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f59400d04b0 0x7f59400b59d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:55:40.716 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:40.713+0000 7f5945eb8640 1 --2- 172.21.15.43:0/3459556772 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f59400b45f0 0x7f59400cd6d0 unknown :-1 s=CLOSED pgs=1727 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:55:40.716 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:40.713+0000 7f5945eb8640 1 -- 172.21.15.43:0/3459556772 >> 172.21.15.43:0/3459556772 conn(0x7f5940071520 msgr2=0x7f59400ceee0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-09T13:55:40.716 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:40.713+0000 7f5945eb8640 1 -- 172.21.15.43:0/3459556772 shutdown_connections 2023-08-09T13:55:40.717 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:40.713+0000 7f5945eb8640 1 -- 172.21.15.43:0/3459556772 wait complete. 2023-08-09T13:55:40.717 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:40.715+0000 7f5945eb8640 1 Processor -- start 2023-08-09T13:55:40.718 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:40.715+0000 7f5945eb8640 1 -- start start 2023-08-09T13:55:40.718 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:40.716+0000 7f5945eb8640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f59400b45f0 0x7f594015b9e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:55:40.719 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:40.716+0000 7f5945eb8640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f59400d04b0 0x7f59400cbef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:55:40.719 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:40.716+0000 7f5945eb8640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f594015c550 0x7f594015a480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:55:40.719 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:40.716+0000 7f5945eb8640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f59401e5a10 con 0x7f59400b45f0 2023-08-09T13:55:40.719 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:40.716+0000 7f5945eb8640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f59401e7ea0 con 0x7f59400d04b0 2023-08-09T13:55:40.719 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:40.716+0000 7f5945eb8640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f59401e8180 con 0x7f594015c550 2023-08-09T13:55:40.720 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:40.717+0000 7f593f7fe640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f59400b45f0 0x7f594015b9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:55:40.721 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:40.717+0000 7f593f7fe640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f59400b45f0 0x7f594015b9e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:53378/0 (socket says 172.21.15.43:53378) 2023-08-09T13:55:40.721 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:40.717+0000 7f593f7fe640 1 -- 172.21.15.43:0/1053770391 learned_addr learned my addr 172.21.15.43:0/1053770391 (peer_addr_for_me v2:172.21.15.43:0/0) 2023-08-09T13:55:40.721 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:40.717+0000 7f593ffff640 1 --2- 172.21.15.43:0/1053770391 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f594015c550 0x7f594015a480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:55:40.721 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:40.717+0000 7f5936ffd640 1 --2- 172.21.15.43:0/1053770391 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f59400d04b0 0x7f59400cbef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:55:40.721 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:40.717+0000 7f593f7fe640 1 -- 172.21.15.43:0/1053770391 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f594015c550 msgr2=0x7f594015a480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:55:40.721 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:40.717+0000 7f593f7fe640 1 --2- 172.21.15.43:0/1053770391 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f594015c550 0x7f594015a480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:55:40.721 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:40.717+0000 7f593f7fe640 1 -- 172.21.15.43:0/1053770391 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f59400d04b0 msgr2=0x7f59400cbef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:55:40.722 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:40.717+0000 7f593f7fe640 1 --2- 172.21.15.43:0/1053770391 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f59400d04b0 0x7f59400cbef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:55:40.722 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:40.717+0000 7f593f7fe640 1 -- 172.21.15.43:0/1053770391 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5928009400 con 0x7f59400b45f0 2023-08-09T13:55:40.722 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:40.718+0000 7f593f7fe640 1 --2- 172.21.15.43:0/1053770391 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f59400b45f0 0x7f594015b9e0 secure :-1 s=READY pgs=1728 cs=0 l=1 rev1=1 crypto rx=0x7f593000cbc0 tx=0x7f593000a060 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-09T13:55:40.722 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:40.718+0000 7f593cff9640 1 -- 172.21.15.43:0/1053770391 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f593000aca0 con 0x7f59400b45f0 2023-08-09T13:55:40.722 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:40.718+0000 7f593cff9640 1 -- 172.21.15.43:0/1053770391 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f593000ae00 con 0x7f59400b45f0 2023-08-09T13:55:40.722 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:40.718+0000 7f5945eb8640 1 -- 172.21.15.43:0/1053770391 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f59401e8460 con 0x7f59400b45f0 2023-08-09T13:55:40.722 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:40.718+0000 7f593cff9640 1 -- 172.21.15.43:0/1053770391 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5930012600 con 0x7f59400b45f0 2023-08-09T13:55:40.722 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:40.718+0000 7f5945eb8640 1 -- 172.21.15.43:0/1053770391 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f59400a9110 con 0x7f59400b45f0 2023-08-09T13:55:40.723 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:40.720+0000 7f5945eb8640 1 -- 172.21.15.43:0/1053770391 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f59400739f0 con 0x7f59400b45f0 2023-08-09T13:55:40.723 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:40.720+0000 7f593cff9640 1 -- 172.21.15.43:0/1053770391 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 68) v1 ==== 88460+0+0 (secure 0 0 0) 0x7f5930007070 con 0x7f59400b45f0 2023-08-09T13:55:40.724 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:40.721+0000 7f593cff9640 1 --2- 172.21.15.43:0/1053770391 >> [v2:172.21.15.203:6832/3490266242,v1:172.21.15.203:6833/3490266242] conn(0x7f592006a1c0 0x7f592006c680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:55:40.724 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:40.721+0000 7f593cff9640 1 -- 172.21.15.43:0/1053770391 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(147..147 src has 1..147) v4 ==== 6426+0+0 (secure 0 0 0) 0x7f5930088740 con 0x7f59400b45f0 2023-08-09T13:55:40.724 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:40.721+0000 7f5936ffd640 1 -- 172.21.15.43:0/1053770391 >> [v2:172.21.15.203:6832/3490266242,v1:172.21.15.203:6833/3490266242] conn(0x7f592006a1c0 msgr2=0x7f592006c680 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.203:6832/3490266242 2023-08-09T13:55:40.724 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:40.721+0000 7f5936ffd640 1 --2- 172.21.15.43:0/1053770391 >> [v2:172.21.15.203:6832/3490266242,v1:172.21.15.203:6833/3490266242] conn(0x7f592006a1c0 0x7f592006c680 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-08-09T13:55:40.727 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:40.725+0000 7f593cff9640 1 -- 172.21.15.43:0/1053770391 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178066 (secure 0 0 0) 0x7f5930053c60 con 0x7f59400b45f0 2023-08-09T13:55:40.925 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:40.922+0000 7f5936ffd640 1 -- 172.21.15.43:0/1053770391 >> [v2:172.21.15.203:6832/3490266242,v1:172.21.15.203:6833/3490266242] conn(0x7f592006a1c0 msgr2=0x7f592006c680 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.203:6832/3490266242 2023-08-09T13:55:40.925 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:40.922+0000 7f5936ffd640 1 --2- 172.21.15.43:0/1053770391 >> [v2:172.21.15.203:6832/3490266242,v1:172.21.15.203:6833/3490266242] conn(0x7f592006a1c0 0x7f592006c680 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2023-08-09T13:55:40.938 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:40.935+0000 7f5945eb8640 1 -- 172.21.15.43:0/1053770391 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command([{prefix=config set, name=mgr/snap_schedule/allow_m_granularity}] v 0) v1 -- 0x7f59400d1300 con 0x7f59400b45f0 2023-08-09T13:55:40.945 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:40.943+0000 7f593cff9640 1 -- 172.21.15.43:0/1053770391 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mgr/snap_schedule/allow_m_granularity}]=0 v47)=0 v47) v1 ==== 138+0+0 (secure 0 0 0) 0x7f5930057280 con 0x7f59400b45f0 2023-08-09T13:55:40.948 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:40.946+0000 7f5945eb8640 1 -- 172.21.15.43:0/1053770391 >> [v2:172.21.15.203:6832/3490266242,v1:172.21.15.203:6833/3490266242] conn(0x7f592006a1c0 msgr2=0x7f592006c680 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-08-09T13:55:40.948 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:40.946+0000 7f5945eb8640 1 --2- 172.21.15.43:0/1053770391 >> [v2:172.21.15.203:6832/3490266242,v1:172.21.15.203:6833/3490266242] conn(0x7f592006a1c0 0x7f592006c680 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:55:40.949 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:40.946+0000 7f5945eb8640 1 -- 172.21.15.43:0/1053770391 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f59400b45f0 msgr2=0x7f594015b9e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:55:40.949 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:40.946+0000 7f5945eb8640 1 --2- 172.21.15.43:0/1053770391 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f59400b45f0 0x7f594015b9e0 secure :-1 s=READY pgs=1728 cs=0 l=1 rev1=1 crypto rx=0x7f593000cbc0 tx=0x7f593000a060 comp rx=0 tx=0).stop 2023-08-09T13:55:40.949 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:40.946+0000 7f5945eb8640 1 -- 172.21.15.43:0/1053770391 shutdown_connections 2023-08-09T13:55:40.949 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:40.946+0000 7f5945eb8640 1 --2- 172.21.15.43:0/1053770391 >> [v2:172.21.15.203:6832/3490266242,v1:172.21.15.203:6833/3490266242] conn(0x7f592006a1c0 0x7f592006c680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:55:40.949 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:40.946+0000 7f5945eb8640 1 --2- 172.21.15.43:0/1053770391 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f594015c550 0x7f594015a480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:55:40.949 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:40.946+0000 7f5945eb8640 1 --2- 172.21.15.43:0/1053770391 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f59400d04b0 0x7f59400cbef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:55:40.949 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:40.946+0000 7f5945eb8640 1 --2- 172.21.15.43:0/1053770391 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f59400b45f0 0x7f594015b9e0 unknown :-1 s=CLOSED pgs=1728 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:55:40.950 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:40.946+0000 7f5945eb8640 1 -- 172.21.15.43:0/1053770391 >> 172.21.15.43:0/1053770391 conn(0x7f5940071520 msgr2=0x7f594015e240 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-09T13:55:40.950 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:40.946+0000 7f5945eb8640 1 -- 172.21.15.43:0/1053770391 shutdown_connections 2023-08-09T13:55:40.950 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:40.946+0000 7f5945eb8640 1 -- 172.21.15.43:0/1053770391 wait complete. 2023-08-09T13:55:40.968 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mgr mgr/snap_schedule/dump_on_update True 2023-08-09T13:55:41.187 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:41.184+0000 7f7db5912640 1 Processor -- start 2023-08-09T13:55:41.187 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:41.184+0000 7f7db5912640 1 -- start start 2023-08-09T13:55:41.188 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:41.186+0000 7f7db5912640 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7f7db00cee90 0x7f7db00cdd90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:55:41.188 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:41.186+0000 7f7db5912640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f7db00ccbb0 0x7f7db00cd030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:55:41.188 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:41.186+0000 7f7db5912640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f7db00afb20 0x7f7db00aff70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:55:41.188 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:41.186+0000 7f7db5912640 1 -- --> v2:172.21.15.43:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7db00b1440 con 0x7f7db00afb20 2023-08-09T13:55:41.189 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:41.186+0000 7f7db5912640 1 -- --> v2:172.21.15.203:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7db00cd570 con 0x7f7db00cee90 2023-08-09T13:55:41.189 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:41.186+0000 7f7db5912640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7db015dfe0 con 0x7f7db00ccbb0 2023-08-09T13:55:41.189 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:41.186+0000 7f7daf7fe640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f7db00afb20 0x7f7db00aff70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:55:41.190 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:41.186+0000 7f7daf7fe640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f7db00afb20 0x7f7db00aff70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:53384/0 (socket says 172.21.15.43:53384) 2023-08-09T13:55:41.190 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:41.186+0000 7f7daf7fe640 1 -- 172.21.15.43:0/1495311132 learned_addr learned my addr 172.21.15.43:0/1495311132 (peer_addr_for_me v2:172.21.15.43:0/0) 2023-08-09T13:55:41.190 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:41.186+0000 7f7da7fff640 1 --2- 172.21.15.43:0/1495311132 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f7db00ccbb0 0x7f7db00cd030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:55:41.190 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:41.186+0000 7f7daeffd640 1 --2- 172.21.15.43:0/1495311132 >> v2:172.21.15.203:3300/0 conn(0x7f7db00cee90 0x7f7db00cdd90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:55:41.190 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:41.186+0000 7f7daf7fe640 1 -- 172.21.15.43:0/1495311132 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f7db00ccbb0 msgr2=0x7f7db00cd030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-09T13:55:41.190 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:41.186+0000 7f7daf7fe640 1 --2- 172.21.15.43:0/1495311132 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f7db00ccbb0 0x7f7db00cd030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:55:41.190 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:41.187+0000 7f7daf7fe640 1 -- 172.21.15.43:0/1495311132 >> v2:172.21.15.203:3300/0 conn(0x7f7db00cee90 msgr2=0x7f7db00cdd90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-09T13:55:41.191 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:41.187+0000 7f7daf7fe640 1 --2- 172.21.15.43:0/1495311132 >> v2:172.21.15.203:3300/0 conn(0x7f7db00cee90 0x7f7db00cdd90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:55:41.191 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:41.187+0000 7f7daf7fe640 1 -- 172.21.15.43:0/1495311132 --> v2:172.21.15.43:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7db00cb240 con 0x7f7db00afb20 2023-08-09T13:55:41.192 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:41.188+0000 7f7daf7fe640 1 --2- 172.21.15.43:0/1495311132 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f7db00afb20 0x7f7db00aff70 secure :-1 s=READY pgs=1729 cs=0 l=1 rev1=1 crypto rx=0x7f7db00b2db0 tx=0x7f7da802bee0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c9fa04cb52bca358 server_cookie=0 in_seq=0 out_seq=0 2023-08-09T13:55:41.192 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:41.188+0000 7f7dae7fc640 1 -- 172.21.15.43:0/1495311132 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7da802ca80 con 0x7f7db00afb20 2023-08-09T13:55:41.192 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:41.189+0000 7f7dae7fc640 1 -- 172.21.15.43:0/1495311132 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7da802cbe0 con 0x7f7db00afb20 2023-08-09T13:55:41.192 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:41.189+0000 7f7dae7fc640 1 -- 172.21.15.43:0/1495311132 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7da80343f0 con 0x7f7db00afb20 2023-08-09T13:55:41.192 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:41.189+0000 7f7db5912640 1 -- 172.21.15.43:0/1495311132 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f7db00afb20 msgr2=0x7f7db00aff70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:55:41.192 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:41.189+0000 7f7db5912640 1 --2- 172.21.15.43:0/1495311132 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f7db00afb20 0x7f7db00aff70 secure :-1 s=READY pgs=1729 cs=0 l=1 rev1=1 crypto rx=0x7f7db00b2db0 tx=0x7f7da802bee0 comp rx=0 tx=0).stop 2023-08-09T13:55:41.193 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:41.189+0000 7f7db5912640 1 -- 172.21.15.43:0/1495311132 shutdown_connections 2023-08-09T13:55:41.193 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:41.189+0000 7f7db5912640 1 --2- 172.21.15.43:0/1495311132 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f7db00afb20 0x7f7db00aff70 unknown :-1 s=CLOSED pgs=1729 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:55:41.193 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:41.189+0000 7f7db5912640 1 --2- 172.21.15.43:0/1495311132 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f7db00ccbb0 0x7f7db00cd030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:55:41.193 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:41.189+0000 7f7db5912640 1 --2- 172.21.15.43:0/1495311132 >> v2:172.21.15.203:3300/0 conn(0x7f7db00cee90 0x7f7db00cdd90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:55:41.193 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:41.189+0000 7f7db5912640 1 -- 172.21.15.43:0/1495311132 >> 172.21.15.43:0/1495311132 conn(0x7f7db0089fe0 msgr2=0x7f7db00d01c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-09T13:55:41.193 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:41.189+0000 7f7db5912640 1 -- 172.21.15.43:0/1495311132 shutdown_connections 2023-08-09T13:55:41.193 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:41.189+0000 7f7db5912640 1 -- 172.21.15.43:0/1495311132 wait complete. 2023-08-09T13:55:41.194 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:41.191+0000 7f7db5912640 1 Processor -- start 2023-08-09T13:55:41.194 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:41.191+0000 7f7db5912640 1 -- start start 2023-08-09T13:55:41.195 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:41.192+0000 7f7db5912640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f7db00afb20 0x7f7db00c9c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:55:41.195 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:41.192+0000 7f7db5912640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f7db00ccbb0 0x7f7db00b2d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:55:41.195 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:41.192+0000 7f7db5912640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f7db00cee90 0x7f7db00c7bc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:55:41.196 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:41.192+0000 7f7db5912640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7db00c3350 con 0x7f7db00ccbb0 2023-08-09T13:55:41.196 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:41.192+0000 7f7db5912640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7db00c3070 con 0x7f7db00cee90 2023-08-09T13:55:41.196 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:41.192+0000 7f7db5912640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7db006b480 con 0x7f7db00afb20 2023-08-09T13:55:41.196 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:41.192+0000 7f7da7fff640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f7db00ccbb0 0x7f7db00b2d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:55:41.196 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:41.193+0000 7f7da7fff640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f7db00ccbb0 0x7f7db00b2d90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:53390/0 (socket says 172.21.15.43:53390) 2023-08-09T13:55:41.196 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:41.193+0000 7f7da7fff640 1 -- 172.21.15.43:0/210894382 learned_addr learned my addr 172.21.15.43:0/210894382 (peer_addr_for_me v2:172.21.15.43:0/0) 2023-08-09T13:55:41.196 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:41.193+0000 7f7daeffd640 1 --2- 172.21.15.43:0/210894382 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f7db00afb20 0x7f7db00c9c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:55:41.197 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:41.193+0000 7f7daf7fe640 1 --2- 172.21.15.43:0/210894382 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f7db00cee90 0x7f7db00c7bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:55:41.197 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:41.193+0000 7f7da7fff640 1 -- 172.21.15.43:0/210894382 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f7db00afb20 msgr2=0x7f7db00c9c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:55:41.197 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:41.193+0000 7f7da7fff640 1 --2- 172.21.15.43:0/210894382 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f7db00afb20 0x7f7db00c9c70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:55:41.197 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:41.193+0000 7f7da7fff640 1 -- 172.21.15.43:0/210894382 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f7db00cee90 msgr2=0x7f7db00c7bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:55:41.197 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:41.193+0000 7f7da7fff640 1 --2- 172.21.15.43:0/210894382 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f7db00cee90 0x7f7db00c7bc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:55:41.197 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:41.193+0000 7f7da7fff640 1 -- 172.21.15.43:0/210894382 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7da8009b30 con 0x7f7db00ccbb0 2023-08-09T13:55:41.198 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:41.193+0000 7f7da7fff640 1 --2- 172.21.15.43:0/210894382 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f7db00ccbb0 0x7f7db00b2d90 secure :-1 s=READY pgs=1730 cs=0 l=1 rev1=1 crypto rx=0x7f7d9800d800 tx=0x7f7d9800dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-09T13:55:41.198 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:41.194+0000 7f7dacff9640 1 -- 172.21.15.43:0/210894382 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7d980096a0 con 0x7f7db00ccbb0 2023-08-09T13:55:41.198 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:41.194+0000 7f7dacff9640 1 -- 172.21.15.43:0/210894382 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7d98009800 con 0x7f7db00ccbb0 2023-08-09T13:55:41.199 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:41.194+0000 7f7dacff9640 1 -- 172.21.15.43:0/210894382 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7d98009b90 con 0x7f7db00ccbb0 2023-08-09T13:55:41.199 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:41.194+0000 7f7db5912640 1 -- 172.21.15.43:0/210894382 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7db006b760 con 0x7f7db00ccbb0 2023-08-09T13:55:41.199 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:41.194+0000 7f7db5912640 1 -- 172.21.15.43:0/210894382 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7db00c2c90 con 0x7f7db00ccbb0 2023-08-09T13:55:41.199 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:41.195+0000 7f7db5912640 1 -- 172.21.15.43:0/210894382 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7db006b2e0 con 0x7f7db00ccbb0 2023-08-09T13:55:41.199 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:41.196+0000 7f7dacff9640 1 -- 172.21.15.43:0/210894382 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 68) v1 ==== 88460+0+0 (secure 0 0 0) 0x7f7d98009cf0 con 0x7f7db00ccbb0 2023-08-09T13:55:41.200 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:41.197+0000 7f7dacff9640 1 --2- 172.21.15.43:0/210894382 >> [v2:172.21.15.203:6832/3490266242,v1:172.21.15.203:6833/3490266242] conn(0x7f7d9006a1c0 0x7f7d9006c680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:55:41.200 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:41.197+0000 7f7dacff9640 1 -- 172.21.15.43:0/210894382 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(147..147 src has 1..147) v4 ==== 6426+0+0 (secure 0 0 0) 0x7f7d98089000 con 0x7f7db00ccbb0 2023-08-09T13:55:41.200 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:41.197+0000 7f7daeffd640 1 -- 172.21.15.43:0/210894382 >> [v2:172.21.15.203:6832/3490266242,v1:172.21.15.203:6833/3490266242] conn(0x7f7d9006a1c0 msgr2=0x7f7d9006c680 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.203:6832/3490266242 2023-08-09T13:55:41.200 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:41.197+0000 7f7daeffd640 1 --2- 172.21.15.43:0/210894382 >> [v2:172.21.15.203:6832/3490266242,v1:172.21.15.203:6833/3490266242] conn(0x7f7d9006a1c0 0x7f7d9006c680 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-08-09T13:55:41.207 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:41.205+0000 7f7dacff9640 1 -- 172.21.15.43:0/210894382 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178066 (secure 0 0 0) 0x7f7db006b2e0 con 0x7f7db00ccbb0 2023-08-09T13:55:41.391 INFO:tasks.ceph.mgr.x.smithi203.stderr:2023-08-09T13:55:41.388+0000 7f21a2445040 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2023-08-09T13:55:41.400 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:41.397+0000 7f7db5912640 1 -- 172.21.15.43:0/210894382 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command([{prefix=config set, name=mgr/snap_schedule/dump_on_update}] v 0) v1 -- 0x7f7db006b2e0 con 0x7f7db00ccbb0 2023-08-09T13:55:41.400 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:41.398+0000 7f7daeffd640 1 -- 172.21.15.43:0/210894382 >> [v2:172.21.15.203:6832/3490266242,v1:172.21.15.203:6833/3490266242] conn(0x7f7d9006a1c0 msgr2=0x7f7d9006c680 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.203:6832/3490266242 2023-08-09T13:55:41.401 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:41.398+0000 7f7daeffd640 1 --2- 172.21.15.43:0/210894382 >> [v2:172.21.15.203:6832/3490266242,v1:172.21.15.203:6833/3490266242] conn(0x7f7d9006a1c0 0x7f7d9006c680 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2023-08-09T13:55:41.406 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:41.403+0000 7f7dacff9640 1 -- 172.21.15.43:0/210894382 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mgr/snap_schedule/dump_on_update}]=0 v48)=0 v48) v1 ==== 133+0+0 (secure 0 0 0) 0x7f7db006b2e0 con 0x7f7db00ccbb0 2023-08-09T13:55:41.409 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:41.406+0000 7f7db5912640 1 -- 172.21.15.43:0/210894382 >> [v2:172.21.15.203:6832/3490266242,v1:172.21.15.203:6833/3490266242] conn(0x7f7d9006a1c0 msgr2=0x7f7d9006c680 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-08-09T13:55:41.409 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:41.406+0000 7f7db5912640 1 --2- 172.21.15.43:0/210894382 >> [v2:172.21.15.203:6832/3490266242,v1:172.21.15.203:6833/3490266242] conn(0x7f7d9006a1c0 0x7f7d9006c680 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:55:41.409 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:41.406+0000 7f7db5912640 1 -- 172.21.15.43:0/210894382 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f7db00ccbb0 msgr2=0x7f7db00b2d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:55:41.409 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:41.406+0000 7f7db5912640 1 --2- 172.21.15.43:0/210894382 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f7db00ccbb0 0x7f7db00b2d90 secure :-1 s=READY pgs=1730 cs=0 l=1 rev1=1 crypto rx=0x7f7d9800d800 tx=0x7f7d9800dcc0 comp rx=0 tx=0).stop 2023-08-09T13:55:41.409 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:41.406+0000 7f7db5912640 1 -- 172.21.15.43:0/210894382 shutdown_connections 2023-08-09T13:55:41.410 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:41.406+0000 7f7db5912640 1 --2- 172.21.15.43:0/210894382 >> [v2:172.21.15.203:6832/3490266242,v1:172.21.15.203:6833/3490266242] conn(0x7f7d9006a1c0 0x7f7d9006c680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:55:41.410 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:41.406+0000 7f7db5912640 1 --2- 172.21.15.43:0/210894382 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f7db00cee90 0x7f7db00c7bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:55:41.410 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:41.406+0000 7f7db5912640 1 --2- 172.21.15.43:0/210894382 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f7db00ccbb0 0x7f7db00b2d90 unknown :-1 s=CLOSED pgs=1730 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:55:41.410 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:41.406+0000 7f7db5912640 1 --2- 172.21.15.43:0/210894382 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f7db00afb20 0x7f7db00c9c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:55:41.410 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:41.406+0000 7f7db5912640 1 -- 172.21.15.43:0/210894382 >> 172.21.15.43:0/210894382 conn(0x7f7db0089fe0 msgr2=0x7f7db015d590 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-09T13:55:41.410 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:41.407+0000 7f7db5912640 1 -- 172.21.15.43:0/210894382 shutdown_connections 2023-08-09T13:55:41.410 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:41.407+0000 7f7db5912640 1 -- 172.21.15.43:0/210894382 wait complete. 2023-08-09T13:55:41.428 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-09T13:55:41.428 DEBUG:teuthology.orchestra.run.smithi043:> (cd /home/ubuntu/cephtest/mnt.0 && exec mkdir -p snap_test_dir1) 2023-08-09T13:55:41.474 INFO:tasks.ceph.mgr.y.smithi043.stderr:2023-08-09T13:55:41.472+0000 7fb823a2b040 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2023-08-09T13:55:41.495 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs snap-schedule add --fs cephfs /snap_test_dir1 1M 2023-08-09T13:55:41.513 INFO:tasks.ceph.mgr.x.smithi203.stderr:2023-08-09T13:55:41.510+0000 7f21a2445040 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2023-08-09T13:55:41.570 INFO:tasks.ceph.mgr.x.smithi203.stderr:2023-08-09T13:55:41.567+0000 7f21a2445040 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2023-08-09T13:55:41.603 INFO:tasks.ceph.mgr.y.smithi043.stderr:2023-08-09T13:55:41.601+0000 7fb823a2b040 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2023-08-09T13:55:41.619 INFO:tasks.ceph.mgr.x.smithi203.stderr:2023-08-09T13:55:41.616+0000 7f21a2445040 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2023-08-09T13:55:41.663 INFO:tasks.ceph.mgr.y.smithi043.stderr:2023-08-09T13:55:41.661+0000 7fb823a2b040 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2023-08-09T13:55:41.717 INFO:tasks.ceph.mgr.y.smithi043.stderr:2023-08-09T13:55:41.714+0000 7fb823a2b040 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2023-08-09T13:55:41.761 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:41.759+0000 7f24abfff640 1 Processor -- start 2023-08-09T13:55:41.762 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:41.759+0000 7f24abfff640 1 -- start start 2023-08-09T13:55:41.763 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:41.760+0000 7f24abfff640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f24ac159170 0x7f24ac0c9670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:55:41.763 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:41.761+0000 7f24abfff640 1 -- --> v1:172.21.15.43:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f24ac0701c0 con 0x7f24ac06cd40 2023-08-09T13:55:41.763 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:41.761+0000 7f24abfff640 1 -- --> v1:172.21.15.203:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f24ac159550 con 0x7f24ac070650 2023-08-09T13:55:41.763 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:41.761+0000 7f24abfff640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f24ac0cd2e0 con 0x7f24ac159170 2023-08-09T13:55:41.765 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:41.761+0000 7f24aa7fc640 1 --1- >> v1:172.21.15.43:6789/0 conn(0x7f24ac06cd40 0x7f24ac06d160 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.43:6789/0 says I am v1:172.21.15.43:56888/0 (socket says 172.21.15.43:56888) 2023-08-09T13:55:41.765 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:41.761+0000 7f24aa7fc640 1 -- 172.21.15.43:0/823888501 learned_addr learned my addr 172.21.15.43:0/823888501 (peer_addr_for_me v1:172.21.15.43:0/0) 2023-08-09T13:55:41.765 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:41.761+0000 7f24aaffd640 1 --2- 172.21.15.43:0/823888501 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f24ac159170 0x7f24ac0c9670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:55:41.765 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:41.762+0000 7f24a9ffb640 1 -- 172.21.15.43:0/823888501 <== mon.0 v1:172.21.15.43:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4141296654 0 0) 0x7f24ac0701c0 con 0x7f24ac06cd40 2023-08-09T13:55:41.765 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:41.762+0000 7f24a9ffb640 1 -- 172.21.15.43:0/823888501 --> v1:172.21.15.43:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f2494005730 con 0x7f24ac06cd40 2023-08-09T13:55:41.765 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:41.762+0000 7f24a9ffb640 1 -- 172.21.15.43:0/823888501 <== mon.0 v1:172.21.15.43:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3650531838 0 0) 0x7f2494005730 con 0x7f24ac06cd40 2023-08-09T13:55:41.765 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:41.762+0000 7f24a9ffb640 1 -- 172.21.15.43:0/823888501 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f24ac159170 msgr2=0x7f24ac0c9670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-09T13:55:41.766 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:41.762+0000 7f24a9ffb640 1 --2- 172.21.15.43:0/823888501 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f24ac159170 0x7f24ac0c9670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:55:41.766 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:41.762+0000 7f24a9ffb640 1 -- 172.21.15.43:0/823888501 >> v1:172.21.15.203:6789/0 conn(0x7f24ac070650 legacy=0x7f24ac070ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:55:41.766 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:41.762+0000 7f24a9ffb640 1 -- 172.21.15.43:0/823888501 --> v1:172.21.15.43:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f24ac0cd5c0 con 0x7f24ac06cd40 2023-08-09T13:55:41.766 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:41.762+0000 7f24a9ffb640 1 -- 172.21.15.43:0/823888501 <== mon.0 v1:172.21.15.43:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1285150740 0 0) 0x7f24a0001670 con 0x7f24ac06cd40 2023-08-09T13:55:41.766 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:41.762+0000 7f24a9ffb640 1 -- 172.21.15.43:0/823888501 >> v1:172.21.15.43:6789/0 conn(0x7f24ac06cd40 legacy=0x7f24ac06d160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:55:41.766 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:41.762+0000 7f24a9ffb640 1 --2- 172.21.15.43:0/823888501 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f24ac159170 0x7f2494005bb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:55:41.766 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:41.762+0000 7f24a9ffb640 1 --2- 172.21.15.43:0/823888501 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f2494006130 0x7f24940069d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:55:41.767 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:41.762+0000 7f24a9ffb640 1 --2- 172.21.15.43:0/823888501 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f2494006f10 0x7f2494007390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:55:41.767 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:41.762+0000 7f24a9ffb640 1 -- 172.21.15.43:0/823888501 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f24ac0cd2e0 con 0x7f2494006130 2023-08-09T13:55:41.767 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:41.762+0000 7f24a9ffb640 1 -- 172.21.15.43:0/823888501 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f249400dd20 con 0x7f2494006f10 2023-08-09T13:55:41.767 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:41.762+0000 7f24a9ffb640 1 -- 172.21.15.43:0/823888501 --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f249400dee0 con 0x7f24ac159170 2023-08-09T13:55:41.767 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:41.762+0000 7f24aa7fc640 1 --2- 172.21.15.43:0/823888501 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f2494006130 0x7f24940069d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:55:41.768 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:41.763+0000 7f24ab7fe640 1 --2- 172.21.15.43:0/823888501 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f2494006f10 0x7f2494007390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:55:41.768 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:41.763+0000 7f24aaffd640 1 --2- 172.21.15.43:0/823888501 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f24ac159170 0x7f2494005bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:55:41.768 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:41.763+0000 7f24aa7fc640 1 -- 172.21.15.43:0/823888501 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f24ac159170 msgr2=0x7f2494005bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-09T13:55:41.769 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:41.763+0000 7f24aa7fc640 1 --2- 172.21.15.43:0/823888501 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f24ac159170 0x7f2494005bb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:55:41.769 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:41.763+0000 7f24aa7fc640 1 -- 172.21.15.43:0/823888501 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f2494006f10 msgr2=0x7f2494007390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-09T13:55:41.769 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:41.763+0000 7f24aa7fc640 1 --2- 172.21.15.43:0/823888501 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f2494006f10 0x7f2494007390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:55:41.769 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:41.763+0000 7f24aa7fc640 1 -- 172.21.15.43:0/823888501 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2494005730 con 0x7f2494006130 2023-08-09T13:55:41.769 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:41.764+0000 7f24aa7fc640 1 --2- 172.21.15.43:0/823888501 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f2494006130 0x7f24940069d0 secure :-1 s=READY pgs=1732 cs=0 l=1 rev1=1 crypto rx=0x7f24a000c190 tx=0x7f2494007a00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3b9c7b634e6055b5 server_cookie=0 in_seq=0 out_seq=0 2023-08-09T13:55:41.769 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:41.764+0000 7f24a9ffb640 1 -- 172.21.15.43:0/823888501 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f24a0005ad0 con 0x7f2494006130 2023-08-09T13:55:41.769 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:41.764+0000 7f24a9ffb640 1 -- 172.21.15.43:0/823888501 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f24a0005c30 con 0x7f2494006130 2023-08-09T13:55:41.770 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:41.764+0000 7f24a9ffb640 1 -- 172.21.15.43:0/823888501 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f24a002c8a0 con 0x7f2494006130 2023-08-09T13:55:41.770 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:41.765+0000 7f24abfff640 1 -- 172.21.15.43:0/823888501 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f2494006130 msgr2=0x7f24940069d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:55:41.770 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:41.765+0000 7f24abfff640 1 --2- 172.21.15.43:0/823888501 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f2494006130 0x7f24940069d0 secure :-1 s=READY pgs=1732 cs=0 l=1 rev1=1 crypto rx=0x7f24a000c190 tx=0x7f2494007a00 comp rx=0 tx=0).stop 2023-08-09T13:55:41.770 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:41.765+0000 7f24ab7fe640 1 -- 172.21.15.43:0/823888501 reap_dead start 2023-08-09T13:55:41.770 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:41.765+0000 7f24abfff640 1 -- 172.21.15.43:0/823888501 shutdown_connections 2023-08-09T13:55:41.770 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:41.765+0000 7f24abfff640 1 -- 172.21.15.43:0/823888501 >> 172.21.15.43:0/823888501 conn(0x7f24ac0cde00 msgr2=0x7f24ac0ce210 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-09T13:55:41.771 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:41.765+0000 7f24abfff640 1 -- 172.21.15.43:0/823888501 shutdown_connections 2023-08-09T13:55:41.771 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:41.765+0000 7f24abfff640 1 -- 172.21.15.43:0/823888501 wait complete. 2023-08-09T13:55:41.774 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:41.767+0000 7f24abfff640 1 Processor -- start 2023-08-09T13:55:41.774 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:41.767+0000 7f24abfff640 1 -- start start 2023-08-09T13:55:41.774 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:41.768+0000 7f24abfff640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f2494006130 0x7f24ac1f1a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:55:41.775 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:41.768+0000 7f24abfff640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f24ac14d6b0 0x7f24ac15b330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:55:41.775 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:41.768+0000 7f24abfff640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f24ac1f1fc0 0x7f24ac15b870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:55:41.775 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:41.768+0000 7f24abfff640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f24ac1f2ae0 con 0x7f24ac1f1fc0 2023-08-09T13:55:41.775 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:41.768+0000 7f24abfff640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f24ac1f2520 con 0x7f24ac14d6b0 2023-08-09T13:55:41.775 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:41.768+0000 7f24abfff640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f24ac1f2800 con 0x7f2494006130 2023-08-09T13:55:41.775 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:41.768+0000 7f24ab7fe640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f24ac1f1fc0 0x7f24ac15b870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:55:41.775 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:41.768+0000 7f24ab7fe640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f24ac1f1fc0 0x7f24ac15b870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:53410/0 (socket says 172.21.15.43:53410) 2023-08-09T13:55:41.776 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:41.768+0000 7f24ab7fe640 1 -- 172.21.15.43:0/1492146241 learned_addr learned my addr 172.21.15.43:0/1492146241 (peer_addr_for_me v2:172.21.15.43:0/0) 2023-08-09T13:55:41.776 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:41.768+0000 7f24aa7fc640 1 --2- 172.21.15.43:0/1492146241 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f24ac14d6b0 0x7f24ac15b330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:55:41.776 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:41.769+0000 7f24aaffd640 1 --2- 172.21.15.43:0/1492146241 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f2494006130 0x7f24ac1f1a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:55:41.776 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:41.769+0000 7f24ab7fe640 1 -- 172.21.15.43:0/1492146241 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f2494006130 msgr2=0x7f24ac1f1a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:55:41.776 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:41.769+0000 7f24ab7fe640 1 --2- 172.21.15.43:0/1492146241 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f2494006130 0x7f24ac1f1a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:55:41.776 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:41.769+0000 7f24ab7fe640 1 -- 172.21.15.43:0/1492146241 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f24ac14d6b0 msgr2=0x7f24ac15b330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:55:41.777 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:41.769+0000 7f24ab7fe640 1 --2- 172.21.15.43:0/1492146241 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f24ac14d6b0 0x7f24ac15b330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:55:41.777 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:41.769+0000 7f24ab7fe640 1 -- 172.21.15.43:0/1492146241 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f24ac0c1ad0 con 0x7f24ac1f1fc0 2023-08-09T13:55:41.777 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:41.769+0000 7f24ab7fe640 1 --2- 172.21.15.43:0/1492146241 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f24ac1f1fc0 0x7f24ac15b870 secure :-1 s=READY pgs=1733 cs=0 l=1 rev1=1 crypto rx=0x7f24a4000c00 tx=0x7f24a400e8a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-09T13:55:41.777 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:41.769+0000 7f24937fe640 1 -- 172.21.15.43:0/1492146241 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f24a40052e0 con 0x7f24ac1f1fc0 2023-08-09T13:55:41.777 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:41.770+0000 7f24937fe640 1 -- 172.21.15.43:0/1492146241 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f24a4005440 con 0x7f24ac1f1fc0 2023-08-09T13:55:41.777 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:41.770+0000 7f24937fe640 1 -- 172.21.15.43:0/1492146241 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f24a4003830 con 0x7f24ac1f1fc0 2023-08-09T13:55:41.777 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:41.770+0000 7f24abfff640 1 -- 172.21.15.43:0/1492146241 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f24ac0c1db0 con 0x7f24ac1f1fc0 2023-08-09T13:55:41.777 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:41.770+0000 7f24abfff640 1 -- 172.21.15.43:0/1492146241 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f24ac15aa20 con 0x7f24ac1f1fc0 2023-08-09T13:55:41.778 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:41.771+0000 7f24abfff640 1 -- 172.21.15.43:0/1492146241 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f24ac0c1930 con 0x7f24ac1f1fc0 2023-08-09T13:55:41.778 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:41.772+0000 7f24937fe640 1 -- 172.21.15.43:0/1492146241 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 68) v1 ==== 88460+0+0 (secure 0 0 0) 0x7f24a4003990 con 0x7f24ac1f1fc0 2023-08-09T13:55:41.778 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:41.774+0000 7f24937fe640 1 --2- 172.21.15.43:0/1492146241 >> [v2:172.21.15.203:6832/3490266242,v1:172.21.15.203:6833/3490266242] conn(0x7f249806a1c0 0x7f249806c680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:55:41.779 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:41.774+0000 7f24937fe640 1 -- 172.21.15.43:0/1492146241 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(147..147 src has 1..147) v4 ==== 6426+0+0 (secure 0 0 0) 0x7f24a4087e50 con 0x7f24ac1f1fc0 2023-08-09T13:55:41.779 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:41.774+0000 7f24aaffd640 1 -- 172.21.15.43:0/1492146241 >> [v2:172.21.15.203:6832/3490266242,v1:172.21.15.203:6833/3490266242] conn(0x7f249806a1c0 msgr2=0x7f249806c680 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.203:6832/3490266242 2023-08-09T13:55:41.779 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:41.774+0000 7f24aaffd640 1 --2- 172.21.15.43:0/1492146241 >> [v2:172.21.15.203:6832/3490266242,v1:172.21.15.203:6833/3490266242] conn(0x7f249806a1c0 0x7f249806c680 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-08-09T13:55:41.780 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:41.778+0000 7f24937fe640 1 -- 172.21.15.43:0/1492146241 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178066 (secure 0 0 0) 0x7f24ac0c1930 con 0x7f24ac1f1fc0 2023-08-09T13:55:41.977 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:41.975+0000 7f24aaffd640 1 -- 172.21.15.43:0/1492146241 >> [v2:172.21.15.203:6832/3490266242,v1:172.21.15.203:6833/3490266242] conn(0x7f249806a1c0 msgr2=0x7f249806c680 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.203:6832/3490266242 2023-08-09T13:55:41.978 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:41.975+0000 7f24aaffd640 1 --2- 172.21.15.43:0/1492146241 >> [v2:172.21.15.203:6832/3490266242,v1:172.21.15.203:6833/3490266242] conn(0x7f249806a1c0 0x7f249806c680 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2023-08-09T13:55:41.992 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:41.989+0000 7f24abfff640 1 -- 172.21.15.43:0/1492146241 --> [v2:172.21.15.203:6832/3490266242,v1:172.21.15.203:6833/3490266242] -- mgr_command(tid 0: {"prefix": "fs snap-schedule add", "fs": "cephfs", "path": "/snap_test_dir1", "snap_schedule": "1M", "target": ["mon-mgr", ""]}) v1 -- 0x7f24ac0b0c50 con 0x7f249806a1c0 2023-08-09T13:55:42.318 INFO:tasks.ceph.mgr.x.smithi203.stderr:2023-08-09T13:55:42.315+0000 7f21a2445040 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2023-08-09T13:55:42.378 INFO:tasks.ceph.mgr.x.smithi203.stderr:2023-08-09T13:55:42.376+0000 7f21a2445040 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2023-08-09T13:55:42.379 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:42.376+0000 7f24aaffd640 1 -- 172.21.15.43:0/1492146241 >> [v2:172.21.15.203:6832/3490266242,v1:172.21.15.203:6833/3490266242] conn(0x7f249806a1c0 msgr2=0x7f249806c680 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.203:6832/3490266242 2023-08-09T13:55:42.379 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:42.376+0000 7f24aaffd640 1 --2- 172.21.15.43:0/1492146241 >> [v2:172.21.15.203:6832/3490266242,v1:172.21.15.203:6833/3490266242] conn(0x7f249806a1c0 0x7f249806c680 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2023-08-09T13:55:42.428 INFO:tasks.ceph.mgr.x.smithi203.stderr:2023-08-09T13:55:42.425+0000 7f21a2445040 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2023-08-09T13:55:42.451 INFO:tasks.ceph.mgr.y.smithi043.stderr:2023-08-09T13:55:42.448+0000 7fb823a2b040 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2023-08-09T13:55:42.515 INFO:tasks.ceph.mgr.y.smithi043.stderr:2023-08-09T13:55:42.512+0000 7fb823a2b040 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2023-08-09T13:55:42.544 INFO:tasks.ceph.mgr.x.smithi203.stderr:2023-08-09T13:55:42.542+0000 7f21a2445040 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2023-08-09T13:55:42.566 INFO:tasks.ceph.mgr.y.smithi043.stderr:2023-08-09T13:55:42.563+0000 7fb823a2b040 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2023-08-09T13:55:42.688 INFO:tasks.ceph.mgr.y.smithi043.stderr:2023-08-09T13:55:42.685+0000 7fb823a2b040 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2023-08-09T13:55:42.701 INFO:tasks.ceph.mgr.x.smithi203.stderr:2023-08-09T13:55:42.698+0000 7f21a2445040 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2023-08-09T13:55:42.807 INFO:tasks.ceph.mgr.x.smithi203.stderr:2023-08-09T13:55:42.805+0000 7f21a2445040 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2023-08-09T13:55:42.851 INFO:tasks.ceph.mgr.y.smithi043.stderr:2023-08-09T13:55:42.849+0000 7fb823a2b040 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2023-08-09T13:55:42.856 INFO:tasks.ceph.mgr.x.smithi203.stderr:2023-08-09T13:55:42.853+0000 7f21a2445040 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2023-08-09T13:55:42.964 INFO:tasks.ceph.mgr.y.smithi043.stderr:2023-08-09T13:55:42.961+0000 7fb823a2b040 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2023-08-09T13:55:43.013 INFO:tasks.ceph.mgr.x.smithi203.stderr:2023-08-09T13:55:43.011+0000 7f21a2445040 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2023-08-09T13:55:43.015 INFO:tasks.ceph.mgr.y.smithi043.stderr:2023-08-09T13:55:43.012+0000 7fb823a2b040 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2023-08-09T13:55:43.180 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:43.177+0000 7f24aaffd640 1 -- 172.21.15.43:0/1492146241 >> [v2:172.21.15.203:6832/3490266242,v1:172.21.15.203:6833/3490266242] conn(0x7f249806a1c0 msgr2=0x7f249806c680 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.203:6832/3490266242 2023-08-09T13:55:43.180 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:43.177+0000 7f24aaffd640 1 --2- 172.21.15.43:0/1492146241 >> [v2:172.21.15.203:6832/3490266242,v1:172.21.15.203:6833/3490266242] conn(0x7f249806a1c0 0x7f249806c680 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2023-08-09T13:55:43.183 INFO:tasks.ceph.mgr.y.smithi043.stderr:2023-08-09T13:55:43.181+0000 7fb823a2b040 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2023-08-09T13:55:43.394 INFO:tasks.ceph.mgr.x.smithi203.stderr:2023-08-09T13:55:43.391+0000 7f21a2445040 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2023-08-09T13:55:43.584 INFO:tasks.ceph.mgr.y.smithi043.stderr:2023-08-09T13:55:43.581+0000 7fb823a2b040 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2023-08-09T13:55:43.615 INFO:tasks.ceph.mgr.x.smithi203.stderr:2023-08-09T13:55:43.612+0000 7f21a2445040 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2023-08-09T13:55:43.672 INFO:tasks.ceph.mgr.x.smithi203.stderr:2023-08-09T13:55:43.670+0000 7f21a2445040 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2023-08-09T13:55:43.722 INFO:tasks.ceph.mgr.x.smithi203.stderr:2023-08-09T13:55:43.720+0000 7f21a2445040 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2023-08-09T13:55:43.774 INFO:tasks.ceph.mgr.x.smithi203.stderr:2023-08-09T13:55:43.772+0000 7f21a2445040 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2023-08-09T13:55:43.814 INFO:tasks.ceph.mgr.y.smithi043.stderr:2023-08-09T13:55:43.812+0000 7fb823a2b040 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2023-08-09T13:55:43.874 INFO:tasks.ceph.mgr.y.smithi043.stderr:2023-08-09T13:55:43.872+0000 7fb823a2b040 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2023-08-09T13:55:43.927 INFO:tasks.ceph.mgr.y.smithi043.stderr:2023-08-09T13:55:43.924+0000 7fb823a2b040 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2023-08-09T13:55:43.982 INFO:tasks.ceph.mgr.y.smithi043.stderr:2023-08-09T13:55:43.979+0000 7fb823a2b040 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2023-08-09T13:55:44.068 INFO:tasks.ceph.mgr.x.smithi203.stderr:2023-08-09T13:55:44.065+0000 7f21a2445040 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2023-08-09T13:55:44.124 INFO:tasks.ceph.mgr.x.smithi203.stderr:2023-08-09T13:55:44.122+0000 7f21a2445040 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2023-08-09T13:55:44.176 INFO:tasks.ceph.mgr.x.smithi203.stderr:2023-08-09T13:55:44.174+0000 7f21a2445040 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2023-08-09T13:55:44.290 INFO:tasks.ceph.mgr.y.smithi043.stderr:2023-08-09T13:55:44.288+0000 7fb823a2b040 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2023-08-09T13:55:44.339 INFO:tasks.ceph.mgr.x.smithi203.stderr:2023-08-09T13:55:44.337+0000 7f21a2445040 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2023-08-09T13:55:44.353 INFO:tasks.ceph.mgr.y.smithi043.stderr:2023-08-09T13:55:44.351+0000 7fb823a2b040 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2023-08-09T13:55:44.390 INFO:tasks.ceph.mgr.x.smithi203.stderr:2023-08-09T13:55:44.387+0000 7f21a2445040 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2023-08-09T13:55:44.408 INFO:tasks.ceph.mgr.y.smithi043.stderr:2023-08-09T13:55:44.406+0000 7fb823a2b040 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2023-08-09T13:55:44.444 INFO:tasks.ceph.mgr.x.smithi203.stderr:2023-08-09T13:55:44.442+0000 7f21a2445040 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2023-08-09T13:55:44.494 INFO:tasks.ceph.mgr.x.smithi203.stderr:2023-08-09T13:55:44.492+0000 7f21a2445040 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2023-08-09T13:55:44.516 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:44.513+0000 7f24937fe640 1 -- 172.21.15.43:0/1492146241 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mgrmap(e 69) v1 ==== 88069+0+0 (secure 0 0 0) 0x7f24a40537d0 con 0x7f24ac1f1fc0 2023-08-09T13:55:44.517 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:44.514+0000 7f24937fe640 1 -- 172.21.15.43:0/1492146241 >> [v2:172.21.15.203:6832/3490266242,v1:172.21.15.203:6833/3490266242] conn(0x7f249806a1c0 msgr2=0x7f249806c680 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-08-09T13:55:44.517 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:44.514+0000 7f24937fe640 1 --2- 172.21.15.43:0/1492146241 >> [v2:172.21.15.203:6832/3490266242,v1:172.21.15.203:6833/3490266242] conn(0x7f249806a1c0 0x7f249806c680 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:55:44.588 INFO:tasks.ceph.mgr.y.smithi043.stderr:2023-08-09T13:55:44.585+0000 7fb823a2b040 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2023-08-09T13:55:44.641 INFO:tasks.ceph.mgr.y.smithi043.stderr:2023-08-09T13:55:44.638+0000 7fb823a2b040 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2023-08-09T13:55:44.658 INFO:tasks.ceph.mgr.x.smithi203.stderr:2023-08-09T13:55:44.656+0000 7f2121f7b640 -1 client.0 error registering admin socket command: (17) File exists 2023-08-09T13:55:44.658 INFO:tasks.ceph.mgr.x.smithi203.stderr:2023-08-09T13:55:44.656+0000 7f2121f7b640 -1 client.0 error registering admin socket command: (17) File exists 2023-08-09T13:55:44.659 INFO:tasks.ceph.mgr.x.smithi203.stderr:2023-08-09T13:55:44.656+0000 7f2121f7b640 -1 client.0 error registering admin socket command: (17) File exists 2023-08-09T13:55:44.659 INFO:tasks.ceph.mgr.x.smithi203.stderr:2023-08-09T13:55:44.656+0000 7f2121f7b640 -1 client.0 error registering admin socket command: (17) File exists 2023-08-09T13:55:44.659 INFO:tasks.ceph.mgr.x.smithi203.stderr:2023-08-09T13:55:44.656+0000 7f2121f7b640 -1 client.0 error registering admin socket command: (17) File exists 2023-08-09T13:55:44.659 INFO:tasks.ceph.mgr.x.smithi203.stderr:2023-08-09T13:55:44.657+0000 7f211ef75640 -1 client.0 error registering admin socket command: (17) File exists 2023-08-09T13:55:44.659 INFO:tasks.ceph.mgr.x.smithi203.stderr:2023-08-09T13:55:44.657+0000 7f211ef75640 -1 client.0 error registering admin socket command: (17) File exists 2023-08-09T13:55:44.660 INFO:tasks.ceph.mgr.x.smithi203.stderr:2023-08-09T13:55:44.657+0000 7f211ef75640 -1 client.0 error registering admin socket command: (17) File exists 2023-08-09T13:55:44.660 INFO:tasks.ceph.mgr.x.smithi203.stderr:2023-08-09T13:55:44.657+0000 7f211ef75640 -1 client.0 error registering admin socket command: (17) File exists 2023-08-09T13:55:44.660 INFO:tasks.ceph.mgr.x.smithi203.stderr:2023-08-09T13:55:44.657+0000 7f211ef75640 -1 client.0 error registering admin socket command: (17) File exists 2023-08-09T13:55:44.697 INFO:tasks.ceph.mgr.y.smithi043.stderr:2023-08-09T13:55:44.694+0000 7fb823a2b040 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2023-08-09T13:55:44.749 INFO:tasks.ceph.mgr.y.smithi043.stderr:2023-08-09T13:55:44.747+0000 7fb823a2b040 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2023-08-09T13:55:45.528 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:45.525+0000 7f24937fe640 1 -- 172.21.15.43:0/1492146241 <== mon.0 v2:172.21.15.43:3300/0 8 ==== mgrmap(e 70) v1 ==== 88196+0+0 (secure 0 0 0) 0x7f24a4086070 con 0x7f24ac1f1fc0 2023-08-09T13:55:45.529 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:45.526+0000 7f24937fe640 1 --2- 172.21.15.43:0/1492146241 >> [v2:172.21.15.203:6832/372694067,v1:172.21.15.203:6833/372694067] conn(0x7f2498072cb0 0x7f24980750a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:55:45.529 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:45.527+0000 7f24937fe640 1 -- 172.21.15.43:0/1492146241 --> [v2:172.21.15.203:6832/372694067,v1:172.21.15.203:6833/372694067] -- mgr_command(tid 0: {"prefix": "fs snap-schedule add", "fs": "cephfs", "path": "/snap_test_dir1", "snap_schedule": "1M", "target": ["mon-mgr", ""]}) v1 -- 0x7f2498036270 con 0x7f2498072cb0 2023-08-09T13:55:45.536 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:45.534+0000 7f24aaffd640 1 --2- 172.21.15.43:0/1492146241 >> [v2:172.21.15.203:6832/372694067,v1:172.21.15.203:6833/372694067] conn(0x7f2498072cb0 0x7f24980750a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:55:45.549 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:45.547+0000 7f24aaffd640 1 --2- 172.21.15.43:0/1492146241 >> [v2:172.21.15.203:6832/372694067,v1:172.21.15.203:6833/372694067] conn(0x7f2498072cb0 0x7f24980750a0 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7f24ac0c4930 tx=0x7f249c00b040 comp rx=0 tx=0).ready entity=mgr.6869 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-09T13:55:45.691 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:45.689+0000 7f24937fe640 1 -- 172.21.15.43:0/1492146241 <== mgr.6869 v2:172.21.15.203:6832/372694067 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+38 (secure 0 0 0) 0x7f249c011040 con 0x7f2498072cb0 2023-08-09T13:55:45.691 INFO:teuthology.orchestra.run.smithi043.stdout:Schedule set for path /snap_test_dir1 2023-08-09T13:55:45.694 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:45.691+0000 7f24abfff640 1 -- 172.21.15.43:0/1492146241 >> [v2:172.21.15.203:6832/372694067,v1:172.21.15.203:6833/372694067] conn(0x7f2498072cb0 msgr2=0x7f24980750a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:55:45.694 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:45.691+0000 7f24abfff640 1 --2- 172.21.15.43:0/1492146241 >> [v2:172.21.15.203:6832/372694067,v1:172.21.15.203:6833/372694067] conn(0x7f2498072cb0 0x7f24980750a0 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7f24ac0c4930 tx=0x7f249c00b040 comp rx=0 tx=0).stop 2023-08-09T13:55:45.694 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:45.691+0000 7f24abfff640 1 -- 172.21.15.43:0/1492146241 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f24ac1f1fc0 msgr2=0x7f24ac15b870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:55:45.695 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:45.691+0000 7f24abfff640 1 --2- 172.21.15.43:0/1492146241 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f24ac1f1fc0 0x7f24ac15b870 secure :-1 s=READY pgs=1733 cs=0 l=1 rev1=1 crypto rx=0x7f24a4000c00 tx=0x7f24a400e8a0 comp rx=0 tx=0).stop 2023-08-09T13:55:45.695 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:45.691+0000 7f24ab7fe640 1 -- 172.21.15.43:0/1492146241 reap_dead start 2023-08-09T13:55:45.695 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:45.692+0000 7f24abfff640 1 -- 172.21.15.43:0/1492146241 shutdown_connections 2023-08-09T13:55:45.695 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:45.692+0000 7f24abfff640 1 -- 172.21.15.43:0/1492146241 >> 172.21.15.43:0/1492146241 conn(0x7f24ac0cde00 msgr2=0x7f24ac14f130 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-09T13:55:45.695 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:45.692+0000 7f24abfff640 1 -- 172.21.15.43:0/1492146241 shutdown_connections 2023-08-09T13:55:45.695 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:45.692+0000 7f24abfff640 1 -- 172.21.15.43:0/1492146241 wait complete. 2023-08-09T13:55:45.716 DEBUG:tasks.cephfs.test_snap_schedules:res=Schedule set for path /snap_test_dir1 2023-08-09T13:55:45.717 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs snap-schedule retention add --fs cephfs /snap_test_dir1 55n 2023-08-09T13:55:45.953 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:45.950+0000 7f3d7cdab640 1 Processor -- start 2023-08-09T13:55:45.953 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:45.950+0000 7f3d7cdab640 1 -- start start 2023-08-09T13:55:45.955 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:45.952+0000 7f3d7cdab640 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7f3d7815df60 0x7f3d780ce1c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:55:45.955 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:45.952+0000 7f3d7cdab640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f3d7815e3d0 0x7f3d780ce700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:55:45.955 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:45.952+0000 7f3d7cdab640 1 -- --> v1:172.21.15.203:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3d7815aef0 con 0x7f3d780cb2d0 2023-08-09T13:55:45.955 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:45.952+0000 7f3d7cdab640 1 -- --> v2:172.21.15.203:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3d7815a970 con 0x7f3d7815df60 2023-08-09T13:55:45.955 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:45.952+0000 7f3d7cdab640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3d7815e940 con 0x7f3d7815e3d0 2023-08-09T13:55:45.956 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:45.953+0000 7f3d76d9d640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f3d7815e3d0 0x7f3d780ce700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:55:45.956 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:45.953+0000 7f3d75d9b640 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7f3d7815df60 0x7f3d780ce1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:55:45.956 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:45.953+0000 7f3d7659c640 1 --1- >> v1:172.21.15.203:6789/0 conn(0x7f3d780cb2d0 0x7f3d7815b380 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.203:6789/0 says I am v1:172.21.15.43:59630/0 (socket says 172.21.15.43:59630) 2023-08-09T13:55:45.957 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:45.953+0000 7f3d7659c640 1 -- 172.21.15.43:0/629232576 learned_addr learned my addr 172.21.15.43:0/629232576 (peer_addr_for_me v1:172.21.15.43:0/0) 2023-08-09T13:55:45.957 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:45.954+0000 7f3d75d9b640 1 -- 172.21.15.43:0/629232576 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f3d7815e3d0 msgr2=0x7f3d780ce700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-09T13:55:45.957 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:45.954+0000 7f3d75d9b640 1 --2- 172.21.15.43:0/629232576 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f3d7815e3d0 0x7f3d780ce700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:55:45.957 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:45.954+0000 7f3d75d9b640 1 -- 172.21.15.43:0/629232576 >> v1:172.21.15.203:6789/0 conn(0x7f3d780cb2d0 legacy=0x7f3d7815b380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:55:45.957 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:45.954+0000 7f3d75d9b640 1 -- 172.21.15.43:0/629232576 --> v2:172.21.15.203:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3d7815ec20 con 0x7f3d7815df60 2023-08-09T13:55:45.957 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:45.955+0000 7f3d75d9b640 1 --2- 172.21.15.43:0/629232576 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f3d7815df60 0x7f3d780ce1c0 secure :-1 s=READY pgs=640 cs=0 l=1 rev1=1 crypto rx=0x7f3d6c0097a0 tx=0x7f3d6c02b820 comp rx=0 tx=0).ready entity=mon.1 client_cookie=94a8f1c06c8dc72d server_cookie=0 in_seq=0 out_seq=0 2023-08-09T13:55:45.958 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:45.956+0000 7f3d7559a640 1 -- 172.21.15.43:0/629232576 <== mon.1 v2:172.21.15.203:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3d6c031780 con 0x7f3d7815df60 2023-08-09T13:55:45.958 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:45.956+0000 7f3d7559a640 1 -- 172.21.15.43:0/629232576 <== mon.1 v2:172.21.15.203:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3d6c0318e0 con 0x7f3d7815df60 2023-08-09T13:55:45.958 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:45.956+0000 7f3d7559a640 1 -- 172.21.15.43:0/629232576 <== mon.1 v2:172.21.15.203:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3d6c031c70 con 0x7f3d7815df60 2023-08-09T13:55:45.959 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:45.956+0000 7f3d7cdab640 1 -- 172.21.15.43:0/629232576 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f3d7815df60 msgr2=0x7f3d780ce1c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:55:45.959 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:45.956+0000 7f3d7cdab640 1 --2- 172.21.15.43:0/629232576 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f3d7815df60 0x7f3d780ce1c0 secure :-1 s=READY pgs=640 cs=0 l=1 rev1=1 crypto rx=0x7f3d6c0097a0 tx=0x7f3d6c02b820 comp rx=0 tx=0).stop 2023-08-09T13:55:45.959 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:45.956+0000 7f3d7cdab640 1 -- 172.21.15.43:0/629232576 shutdown_connections 2023-08-09T13:55:45.960 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:45.956+0000 7f3d7cdab640 1 --2- 172.21.15.43:0/629232576 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f3d7815e3d0 0x7f3d780ce700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:55:45.960 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:45.957+0000 7f3d7cdab640 1 --2- 172.21.15.43:0/629232576 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f3d7815df60 0x7f3d780ce1c0 unknown :-1 s=CLOSED pgs=640 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:55:45.960 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:45.957+0000 7f3d7cdab640 1 -- 172.21.15.43:0/629232576 >> 172.21.15.43:0/629232576 conn(0x7f3d780aff50 msgr2=0x7f3d780b0360 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-09T13:55:45.960 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:45.957+0000 7f3d7cdab640 1 -- 172.21.15.43:0/629232576 shutdown_connections 2023-08-09T13:55:45.960 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:45.957+0000 7f3d7cdab640 1 -- 172.21.15.43:0/629232576 wait complete. 2023-08-09T13:55:45.961 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:45.958+0000 7f3d7cdab640 1 Processor -- start 2023-08-09T13:55:45.961 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:45.959+0000 7f3d7cdab640 1 -- start start 2023-08-09T13:55:45.962 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:45.959+0000 7f3d7cdab640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f3d780cb2d0 0x7f3d781f1840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:55:45.962 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:45.959+0000 7f3d7cdab640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f3d7815df60 0x7f3d780c7120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:55:45.962 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:45.959+0000 7f3d7cdab640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f3d7815e3d0 0x7f3d7815ccb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:55:45.962 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:45.959+0000 7f3d7cdab640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3d78150730 con 0x7f3d7815e3d0 2023-08-09T13:55:45.962 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:45.959+0000 7f3d7cdab640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3d780b0df0 con 0x7f3d7815df60 2023-08-09T13:55:45.962 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:45.959+0000 7f3d7cdab640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3d7814fe90 con 0x7f3d780cb2d0 2023-08-09T13:55:45.964 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:45.960+0000 7f3d76d9d640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f3d7815e3d0 0x7f3d7815ccb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:55:45.964 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:45.960+0000 7f3d76d9d640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f3d7815e3d0 0x7f3d7815ccb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:53828/0 (socket says 172.21.15.43:53828) 2023-08-09T13:55:45.964 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:45.960+0000 7f3d76d9d640 1 -- 172.21.15.43:0/3386852974 learned_addr learned my addr 172.21.15.43:0/3386852974 (peer_addr_for_me v2:172.21.15.43:0/0) 2023-08-09T13:55:45.964 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:45.960+0000 7f3d75d9b640 1 --2- 172.21.15.43:0/3386852974 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f3d7815df60 0x7f3d780c7120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:55:45.964 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:45.960+0000 7f3d7659c640 1 --2- 172.21.15.43:0/3386852974 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f3d780cb2d0 0x7f3d781f1840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:55:45.964 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:45.960+0000 7f3d76d9d640 1 -- 172.21.15.43:0/3386852974 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f3d780cb2d0 msgr2=0x7f3d781f1840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:55:45.965 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:45.960+0000 7f3d76d9d640 1 --2- 172.21.15.43:0/3386852974 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f3d780cb2d0 0x7f3d781f1840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:55:45.965 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:45.960+0000 7f3d76d9d640 1 -- 172.21.15.43:0/3386852974 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f3d7815df60 msgr2=0x7f3d780c7120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:55:45.965 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:45.960+0000 7f3d76d9d640 1 --2- 172.21.15.43:0/3386852974 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f3d7815df60 0x7f3d780c7120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:55:45.965 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:45.960+0000 7f3d76d9d640 1 -- 172.21.15.43:0/3386852974 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3d6c009400 con 0x7f3d7815e3d0 2023-08-09T13:55:45.965 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:45.960+0000 7f3d7659c640 1 --2- 172.21.15.43:0/3386852974 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f3d780cb2d0 0x7f3d781f1840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-08-09T13:55:45.965 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:45.961+0000 7f3d76d9d640 1 --2- 172.21.15.43:0/3386852974 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f3d7815e3d0 0x7f3d7815ccb0 secure :-1 s=READY pgs=1734 cs=0 l=1 rev1=1 crypto rx=0x7f3d7000efc0 tx=0x7f3d7000c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-09T13:55:45.965 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:45.961+0000 7f3d66ffd640 1 -- 172.21.15.43:0/3386852974 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3d70009ba0 con 0x7f3d7815e3d0 2023-08-09T13:55:45.965 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:45.961+0000 7f3d66ffd640 1 -- 172.21.15.43:0/3386852974 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3d7000f040 con 0x7f3d7815e3d0 2023-08-09T13:55:45.966 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:45.961+0000 7f3d66ffd640 1 -- 172.21.15.43:0/3386852974 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3d700146c0 con 0x7f3d7815e3d0 2023-08-09T13:55:45.966 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:45.961+0000 7f3d7cdab640 1 -- 172.21.15.43:0/3386852974 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3d78150170 con 0x7f3d7815e3d0 2023-08-09T13:55:45.966 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:45.961+0000 7f3d7cdab640 1 -- 172.21.15.43:0/3386852974 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3d780c7660 con 0x7f3d7815e3d0 2023-08-09T13:55:45.967 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:45.962+0000 7f3d7cdab640 1 -- 172.21.15.43:0/3386852974 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3d780b0c50 con 0x7f3d7815e3d0 2023-08-09T13:55:45.967 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:45.963+0000 7f3d66ffd640 1 -- 172.21.15.43:0/3386852974 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 70) v1 ==== 88196+0+0 (secure 0 0 0) 0x7f3d70014820 con 0x7f3d7815e3d0 2023-08-09T13:55:45.967 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:45.964+0000 7f3d66ffd640 1 --2- 172.21.15.43:0/3386852974 >> [v2:172.21.15.203:6832/372694067,v1:172.21.15.203:6833/372694067] conn(0x7f3d58069ef0 0x7f3d5806c3b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:55:45.968 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:45.964+0000 7f3d66ffd640 1 -- 172.21.15.43:0/3386852974 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(148..148 src has 1..148) v4 ==== 6770+0+0 (secure 0 0 0) 0x7f3d700893c0 con 0x7f3d7815e3d0 2023-08-09T13:55:45.968 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:45.965+0000 7f3d7659c640 1 --2- 172.21.15.43:0/3386852974 >> [v2:172.21.15.203:6832/372694067,v1:172.21.15.203:6833/372694067] conn(0x7f3d58069ef0 0x7f3d5806c3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:55:45.968 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:45.966+0000 7f3d7659c640 1 --2- 172.21.15.43:0/3386852974 >> [v2:172.21.15.203:6832/372694067,v1:172.21.15.203:6833/372694067] conn(0x7f3d58069ef0 0x7f3d5806c3b0 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7f3d68005880 tx=0x7f3d68009040 comp rx=0 tx=0).ready entity=mgr.6869 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-09T13:55:45.971 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:45.969+0000 7f3d66ffd640 1 -- 172.21.15.43:0/3386852974 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178066 (secure 0 0 0) 0x7f3d780b0c50 con 0x7f3d7815e3d0 2023-08-09T13:55:46.187 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:46.185+0000 7f3d7cdab640 1 -- 172.21.15.43:0/3386852974 --> [v2:172.21.15.203:6832/372694067,v1:172.21.15.203:6833/372694067] -- mgr_command(tid 0: {"prefix": "fs snap-schedule retention add", "fs": "cephfs", "path": "/snap_test_dir1", "retention_spec_or_period": "55n", "target": ["mon-mgr", ""]}) v1 -- 0x7f3d7815ab80 con 0x7f3d58069ef0 2023-08-09T13:55:46.329 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:46.327+0000 7f3d66ffd640 1 -- 172.21.15.43:0/3386852974 <== mgr.6869 v2:172.21.15.203:6832/372694067 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+40 (secure 0 0 0) 0x7f3d680013d0 con 0x7f3d58069ef0 2023-08-09T13:55:46.329 INFO:teuthology.orchestra.run.smithi043.stdout:Retention added to path /snap_test_dir1 2023-08-09T13:55:46.331 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:46.329+0000 7f3d7cdab640 1 -- 172.21.15.43:0/3386852974 >> [v2:172.21.15.203:6832/372694067,v1:172.21.15.203:6833/372694067] conn(0x7f3d58069ef0 msgr2=0x7f3d5806c3b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:55:46.332 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:46.329+0000 7f3d7cdab640 1 --2- 172.21.15.43:0/3386852974 >> [v2:172.21.15.203:6832/372694067,v1:172.21.15.203:6833/372694067] conn(0x7f3d58069ef0 0x7f3d5806c3b0 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7f3d68005880 tx=0x7f3d68009040 comp rx=0 tx=0).stop 2023-08-09T13:55:46.332 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:46.329+0000 7f3d7cdab640 1 -- 172.21.15.43:0/3386852974 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f3d7815e3d0 msgr2=0x7f3d7815ccb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:55:46.332 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:46.329+0000 7f3d7cdab640 1 --2- 172.21.15.43:0/3386852974 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f3d7815e3d0 0x7f3d7815ccb0 secure :-1 s=READY pgs=1734 cs=0 l=1 rev1=1 crypto rx=0x7f3d7000efc0 tx=0x7f3d7000c490 comp rx=0 tx=0).stop 2023-08-09T13:55:46.333 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:46.329+0000 7f3d7cdab640 1 -- 172.21.15.43:0/3386852974 shutdown_connections 2023-08-09T13:55:46.333 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:46.329+0000 7f3d7cdab640 1 --2- 172.21.15.43:0/3386852974 >> [v2:172.21.15.203:6832/372694067,v1:172.21.15.203:6833/372694067] conn(0x7f3d58069ef0 0x7f3d5806c3b0 unknown :-1 s=CLOSED pgs=23 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:55:46.333 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:46.329+0000 7f3d7cdab640 1 --2- 172.21.15.43:0/3386852974 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f3d7815e3d0 0x7f3d7815ccb0 unknown :-1 s=CLOSED pgs=1734 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:55:46.333 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:46.329+0000 7f3d7cdab640 1 --2- 172.21.15.43:0/3386852974 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f3d7815df60 0x7f3d780c7120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:55:46.333 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:46.329+0000 7f3d7cdab640 1 --2- 172.21.15.43:0/3386852974 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f3d780cb2d0 0x7f3d781f1840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:55:46.333 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:46.329+0000 7f3d7cdab640 1 -- 172.21.15.43:0/3386852974 >> 172.21.15.43:0/3386852974 conn(0x7f3d780aff50 msgr2=0x7f3d780ca350 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-09T13:55:46.334 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:46.330+0000 7f3d7cdab640 1 -- 172.21.15.43:0/3386852974 shutdown_connections 2023-08-09T13:55:46.334 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:46.330+0000 7f3d7cdab640 1 -- 172.21.15.43:0/3386852974 wait complete. 2023-08-09T13:55:46.352 DEBUG:tasks.cephfs.test_snap_schedules:res=Retention added to path /snap_test_dir1 2023-08-09T13:55:46.352 DEBUG:tasks.cephfs.test_snap_schedules:expected_schedule: ['1M'], expected_retention: [] 2023-08-09T13:55:46.353 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs snap-schedule list --fs cephfs --format json /snap_test_dir1 2023-08-09T13:55:46.589 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:46.586+0000 7f671a9c8640 1 Processor -- start 2023-08-09T13:55:46.589 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:46.586+0000 7f671a9c8640 1 -- start start 2023-08-09T13:55:46.591 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:46.588+0000 7f671a9c8640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f671415a510 0x7f6714160fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:55:46.591 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:46.588+0000 7f671a9c8640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f671415d5e0 0x7f67140b3ec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:55:46.591 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:46.588+0000 7f671a9c8640 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7f671415da60 0x7f67140b4400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:55:46.592 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:46.588+0000 7f671a9c8640 1 -- --> v2:172.21.15.43:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f671415a8f0 con 0x7f671415a510 2023-08-09T13:55:46.592 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:46.588+0000 7f671a9c8640 1 -- --> v2:172.21.15.203:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f67140ccd30 con 0x7f671415da60 2023-08-09T13:55:46.592 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:46.588+0000 7f671a9c8640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f67140cdd40 con 0x7f671415d5e0 2023-08-09T13:55:46.592 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:46.588+0000 7f6713fff640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f671415a510 0x7f6714160fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:55:46.592 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:46.589+0000 7f6713fff640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f671415a510 0x7f6714160fe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:53832/0 (socket says 172.21.15.43:53832) 2023-08-09T13:55:46.592 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:46.589+0000 7f6713fff640 1 -- 172.21.15.43:0/3717699246 learned_addr learned my addr 172.21.15.43:0/3717699246 (peer_addr_for_me v2:172.21.15.43:0/0) 2023-08-09T13:55:46.593 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:46.589+0000 7f6718f65640 1 --2- 172.21.15.43:0/3717699246 >> v2:172.21.15.203:3300/0 conn(0x7f671415da60 0x7f67140b4400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:55:46.594 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:46.589+0000 7f670b7fe640 1 --2- 172.21.15.43:0/3717699246 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f671415d5e0 0x7f67140b3ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:55:46.594 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:46.589+0000 7f6713fff640 1 -- 172.21.15.43:0/3717699246 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f671415d5e0 msgr2=0x7f67140b3ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-09T13:55:46.594 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:46.589+0000 7f6713fff640 1 --2- 172.21.15.43:0/3717699246 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f671415d5e0 0x7f67140b3ec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:55:46.594 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:46.589+0000 7f6713fff640 1 -- 172.21.15.43:0/3717699246 >> v2:172.21.15.203:3300/0 conn(0x7f671415da60 msgr2=0x7f67140b4400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-09T13:55:46.594 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:46.589+0000 7f6713fff640 1 --2- 172.21.15.43:0/3717699246 >> v2:172.21.15.203:3300/0 conn(0x7f671415da60 0x7f67140b4400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:55:46.594 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:46.589+0000 7f6713fff640 1 -- 172.21.15.43:0/3717699246 --> v2:172.21.15.43:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f67140ccf80 con 0x7f671415a510 2023-08-09T13:55:46.594 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:46.589+0000 7f6718f65640 1 --2- 172.21.15.43:0/3717699246 >> v2:172.21.15.203:3300/0 conn(0x7f671415da60 0x7f67140b4400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-08-09T13:55:46.595 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:46.590+0000 7f6713fff640 1 --2- 172.21.15.43:0/3717699246 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f671415a510 0x7f6714160fe0 secure :-1 s=READY pgs=1735 cs=0 l=1 rev1=1 crypto rx=0x7f67140d1c30 tx=0x7f670402b900 comp rx=0 tx=0).ready entity=mon.0 client_cookie=17259a766ee9a326 server_cookie=0 in_seq=0 out_seq=0 2023-08-09T13:55:46.595 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:46.591+0000 7f67137fe640 1 -- 172.21.15.43:0/3717699246 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6704031860 con 0x7f671415a510 2023-08-09T13:55:46.595 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:46.591+0000 7f67137fe640 1 -- 172.21.15.43:0/3717699246 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f67040319c0 con 0x7f671415a510 2023-08-09T13:55:46.595 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:46.591+0000 7f67137fe640 1 -- 172.21.15.43:0/3717699246 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6704031d50 con 0x7f671415a510 2023-08-09T13:55:46.596 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:46.592+0000 7f671a9c8640 1 -- 172.21.15.43:0/3717699246 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f671415a510 msgr2=0x7f6714160fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:55:46.596 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:46.592+0000 7f671a9c8640 1 --2- 172.21.15.43:0/3717699246 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f671415a510 0x7f6714160fe0 secure :-1 s=READY pgs=1735 cs=0 l=1 rev1=1 crypto rx=0x7f67140d1c30 tx=0x7f670402b900 comp rx=0 tx=0).stop 2023-08-09T13:55:46.596 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:46.592+0000 7f671a9c8640 1 -- 172.21.15.43:0/3717699246 shutdown_connections 2023-08-09T13:55:46.596 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:46.592+0000 7f671a9c8640 1 --2- 172.21.15.43:0/3717699246 >> v2:172.21.15.203:3300/0 conn(0x7f671415da60 0x7f67140b4400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:55:46.596 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:46.592+0000 7f671a9c8640 1 --2- 172.21.15.43:0/3717699246 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f671415d5e0 0x7f67140b3ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:55:46.596 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:46.592+0000 7f671a9c8640 1 --2- 172.21.15.43:0/3717699246 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f671415a510 0x7f6714160fe0 unknown :-1 s=CLOSED pgs=1735 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:55:46.597 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:46.592+0000 7f671a9c8640 1 -- 172.21.15.43:0/3717699246 >> 172.21.15.43:0/3717699246 conn(0x7f67140ceec0 msgr2=0x7f67140cf2d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-09T13:55:46.597 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:46.592+0000 7f671a9c8640 1 -- 172.21.15.43:0/3717699246 shutdown_connections 2023-08-09T13:55:46.597 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:46.592+0000 7f671a9c8640 1 -- 172.21.15.43:0/3717699246 wait complete. 2023-08-09T13:55:46.598 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:46.594+0000 7f671a9c8640 1 Processor -- start 2023-08-09T13:55:46.598 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:46.594+0000 7f671a9c8640 1 -- start start 2023-08-09T13:55:46.598 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:46.595+0000 7f671a9c8640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f671415a510 0x7f671415e220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:55:46.599 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:46.595+0000 7f671a9c8640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f671415d5e0 0x7f67140c7760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:55:46.599 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:46.595+0000 7f671a9c8640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f671415da60 0x7f67140c7ca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:55:46.599 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:46.595+0000 7f671a9c8640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f67140c61d0 con 0x7f671415a510 2023-08-09T13:55:46.599 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:46.595+0000 7f671a9c8640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f67140c82d0 con 0x7f671415da60 2023-08-09T13:55:46.599 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:46.595+0000 7f671a9c8640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f67140c85b0 con 0x7f671415d5e0 2023-08-09T13:55:46.599 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:46.595+0000 7f6713fff640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f671415a510 0x7f671415e220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:55:46.599 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:46.595+0000 7f6713fff640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f671415a510 0x7f671415e220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:53836/0 (socket says 172.21.15.43:53836) 2023-08-09T13:55:46.599 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:46.595+0000 7f6713fff640 1 -- 172.21.15.43:0/1188029869 learned_addr learned my addr 172.21.15.43:0/1188029869 (peer_addr_for_me v2:172.21.15.43:0/0) 2023-08-09T13:55:46.600 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:46.596+0000 7f670b7fe640 1 --2- 172.21.15.43:0/1188029869 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f671415d5e0 0x7f67140c7760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:55:46.600 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:46.596+0000 7f6713fff640 1 -- 172.21.15.43:0/1188029869 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f671415d5e0 msgr2=0x7f67140c7760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:55:46.600 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:46.596+0000 7f6713fff640 1 --2- 172.21.15.43:0/1188029869 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f671415d5e0 0x7f67140c7760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:55:46.600 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:46.596+0000 7f6713fff640 1 -- 172.21.15.43:0/1188029869 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f671415da60 msgr2=0x7f67140c7ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:55:46.600 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:46.596+0000 7f6713fff640 1 --2- 172.21.15.43:0/1188029869 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f671415da60 0x7f67140c7ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:55:46.600 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:46.596+0000 7f6713fff640 1 -- 172.21.15.43:0/1188029869 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f67040094d0 con 0x7f671415a510 2023-08-09T13:55:46.600 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:46.596+0000 7f6713fff640 1 --2- 172.21.15.43:0/1188029869 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f671415a510 0x7f671415e220 secure :-1 s=READY pgs=1736 cs=0 l=1 rev1=1 crypto rx=0x7f670402b940 tx=0x7f67040046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-09T13:55:46.601 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:46.596+0000 7f67117fa640 1 -- 172.21.15.43:0/1188029869 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f670403ad30 con 0x7f671415a510 2023-08-09T13:55:46.601 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:46.596+0000 7f67117fa640 1 -- 172.21.15.43:0/1188029869 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f67040443f0 con 0x7f671415a510 2023-08-09T13:55:46.601 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:46.596+0000 7f67117fa640 1 -- 172.21.15.43:0/1188029869 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f67040447a0 con 0x7f671415a510 2023-08-09T13:55:46.602 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:46.596+0000 7f671a9c8640 1 -- 172.21.15.43:0/1188029869 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f67140c8890 con 0x7f671415a510 2023-08-09T13:55:46.602 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:46.597+0000 7f671a9c8640 1 -- 172.21.15.43:0/1188029869 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f67140c9510 con 0x7f671415a510 2023-08-09T13:55:46.602 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:46.597+0000 7f671a9c8640 1 -- 172.21.15.43:0/1188029869 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f67140c9700 con 0x7f671415a510 2023-08-09T13:55:46.602 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:46.599+0000 7f67117fa640 1 -- 172.21.15.43:0/1188029869 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 71) v1 ==== 88461+0+0 (secure 0 0 0) 0x7f670404d450 con 0x7f671415a510 2023-08-09T13:55:46.603 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:46.599+0000 7f67117fa640 1 --2- 172.21.15.43:0/1188029869 >> [v2:172.21.15.203:6832/372694067,v1:172.21.15.203:6833/372694067] conn(0x7f66f406a1c0 0x7f66f406c680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T13:55:46.603 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:46.600+0000 7f67117fa640 1 -- 172.21.15.43:0/1188029869 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(148..148 src has 1..148) v4 ==== 6770+0+0 (secure 0 0 0) 0x7f67040ab170 con 0x7f671415a510 2023-08-09T13:55:46.603 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:46.600+0000 7f670b7fe640 1 --2- 172.21.15.43:0/1188029869 >> [v2:172.21.15.203:6832/372694067,v1:172.21.15.203:6833/372694067] conn(0x7f66f406a1c0 0x7f66f406c680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T13:55:46.604 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:46.602+0000 7f670b7fe640 1 --2- 172.21.15.43:0/1188029869 >> [v2:172.21.15.203:6832/372694067,v1:172.21.15.203:6833/372694067] conn(0x7f66f406a1c0 0x7f66f406c680 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7f66fc006fd0 tx=0x7f66fc008040 comp rx=0 tx=0).ready entity=mgr.6869 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-09T13:55:46.606 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:46.604+0000 7f67117fa640 1 -- 172.21.15.43:0/1188029869 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178066 (secure 0 0 0) 0x7f67040762d0 con 0x7f671415a510 2023-08-09T13:55:46.824 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:46.821+0000 7f671a9c8640 1 -- 172.21.15.43:0/1188029869 --> [v2:172.21.15.203:6832/372694067,v1:172.21.15.203:6833/372694067] -- mgr_command(tid 0: {"prefix": "fs snap-schedule list", "fs": "cephfs", "path": "/snap_test_dir1", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f67140cd540 con 0x7f66f406a1c0 2023-08-09T13:55:46.841 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:46.838+0000 7f67117fa640 1 -- 172.21.15.43:0/1188029869 <== mgr.6869 v2:172.21.15.203:6832/372694067 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+74 (secure 0 0 0) 0x7f66fc015440 con 0x7f66f406a1c0 2023-08-09T13:55:46.841 INFO:teuthology.orchestra.run.smithi043.stdout: 2023-08-09T13:55:46.841 INFO:teuthology.orchestra.run.smithi043.stdout:{"path": "/snap_test_dir1", "schedule": ["1M"], "retention": [{"n": 55}]} 2023-08-09T13:55:46.843 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:46.841+0000 7f671a9c8640 1 -- 172.21.15.43:0/1188029869 >> [v2:172.21.15.203:6832/372694067,v1:172.21.15.203:6833/372694067] conn(0x7f66f406a1c0 msgr2=0x7f66f406c680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:55:46.843 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:46.841+0000 7f671a9c8640 1 --2- 172.21.15.43:0/1188029869 >> [v2:172.21.15.203:6832/372694067,v1:172.21.15.203:6833/372694067] conn(0x7f66f406a1c0 0x7f66f406c680 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7f66fc006fd0 tx=0x7f66fc008040 comp rx=0 tx=0).stop 2023-08-09T13:55:46.844 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:46.841+0000 7f671a9c8640 1 -- 172.21.15.43:0/1188029869 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f671415a510 msgr2=0x7f671415e220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T13:55:46.844 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:46.841+0000 7f671a9c8640 1 --2- 172.21.15.43:0/1188029869 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f671415a510 0x7f671415e220 secure :-1 s=READY pgs=1736 cs=0 l=1 rev1=1 crypto rx=0x7f670402b940 tx=0x7f67040046c0 comp rx=0 tx=0).stop 2023-08-09T13:55:46.844 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:46.841+0000 7f671a9c8640 1 -- 172.21.15.43:0/1188029869 shutdown_connections 2023-08-09T13:55:46.844 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:46.841+0000 7f671a9c8640 1 --2- 172.21.15.43:0/1188029869 >> [v2:172.21.15.203:6832/372694067,v1:172.21.15.203:6833/372694067] conn(0x7f66f406a1c0 0x7f66f406c680 unknown :-1 s=CLOSED pgs=24 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:55:46.844 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:46.841+0000 7f671a9c8640 1 --2- 172.21.15.43:0/1188029869 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f671415da60 0x7f67140c7ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:55:46.844 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:46.841+0000 7f671a9c8640 1 --2- 172.21.15.43:0/1188029869 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f671415d5e0 0x7f67140c7760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:55:46.844 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:46.841+0000 7f671a9c8640 1 --2- 172.21.15.43:0/1188029869 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f671415a510 0x7f671415e220 unknown :-1 s=CLOSED pgs=1736 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T13:55:46.845 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:46.841+0000 7f671a9c8640 1 -- 172.21.15.43:0/1188029869 >> 172.21.15.43:0/1188029869 conn(0x7f67140ceec0 msgr2=0x7f6714160240 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-09T13:55:46.845 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:46.841+0000 7f671a9c8640 1 -- 172.21.15.43:0/1188029869 shutdown_connections 2023-08-09T13:55:46.845 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T13:55:46.841+0000 7f671a9c8640 1 -- 172.21.15.43:0/1188029869 wait complete. 2023-08-09T13:55:46.864 DEBUG:tasks.cephfs.test_snap_schedules:res= {"path": "/snap_test_dir1", "schedule": ["1M"], "retention": [{"n": 55}]} 2023-08-09T13:55:46.864 DEBUG:tasks.cephfs.test_snap_schedules:json_res: {'path': '/snap_test_dir1', 'schedule': ['1M'], 'retention': [{'n': 55}]} 2023-08-09T13:56:00.351 INFO:tasks.ceph.mgr.x.smithi203.stderr:2023-08-09T13:56:00.349+0000 7f2110f59640 -1 client.0 error registering admin socket command: (17) File exists 2023-08-09T13:56:00.351 INFO:tasks.ceph.mgr.x.smithi203.stderr:2023-08-09T13:56:00.349+0000 7f2110f59640 -1 client.0 error registering admin socket command: (17) File exists 2023-08-09T13:56:00.351 INFO:tasks.ceph.mgr.x.smithi203.stderr:2023-08-09T13:56:00.349+0000 7f2110f59640 -1 client.0 error registering admin socket command: (17) File exists 2023-08-09T13:56:00.352 INFO:tasks.ceph.mgr.x.smithi203.stderr:2023-08-09T13:56:00.349+0000 7f2110f59640 -1 client.0 error registering admin socket command: (17) File exists 2023-08-09T13:56:00.352 INFO:tasks.ceph.mgr.x.smithi203.stderr:2023-08-09T13:56:00.349+0000 7f2110f59640 -1 client.0 error registering admin socket command: (17) File exists 2023-08-09T13:56:05.538 DEBUG:teuthology.orchestra.run.smithi043:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-08-09T13:56:05.547 DEBUG:teuthology.orchestra.run.smithi203:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-08-09T13:56:35.692 DEBUG:teuthology.orchestra.run.smithi043:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-08-09T13:56:35.701 DEBUG:teuthology.orchestra.run.smithi203:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-08-09T13:57:05.856 DEBUG:teuthology.orchestra.run.smithi043:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-08-09T13:57:05.865 DEBUG:teuthology.orchestra.run.smithi203:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-08-09T13:57:36.001 DEBUG:teuthology.orchestra.run.smithi043:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-08-09T13:57:36.011 DEBUG:teuthology.orchestra.run.smithi203:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-08-09T13:58:06.141 DEBUG:teuthology.orchestra.run.smithi043:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-08-09T13:58:06.150 DEBUG:teuthology.orchestra.run.smithi203:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-08-09T13:58:36.298 DEBUG:teuthology.orchestra.run.smithi043:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-08-09T13:58:36.307 DEBUG:teuthology.orchestra.run.smithi203:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-08-09T13:59:06.446 DEBUG:teuthology.orchestra.run.smithi043:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-08-09T13:59:06.455 DEBUG:teuthology.orchestra.run.smithi203:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-08-09T13:59:36.608 DEBUG:teuthology.orchestra.run.smithi043:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-08-09T13:59:36.618 DEBUG:teuthology.orchestra.run.smithi203:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-08-09T14:00:06.785 DEBUG:teuthology.orchestra.run.smithi043:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-08-09T14:00:06.795 DEBUG:teuthology.orchestra.run.smithi203:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-08-09T14:00:36.961 DEBUG:teuthology.orchestra.run.smithi043:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-08-09T14:00:36.971 DEBUG:teuthology.orchestra.run.smithi203:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-08-09T14:01:07.123 DEBUG:teuthology.orchestra.run.smithi043:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-08-09T14:01:07.133 DEBUG:teuthology.orchestra.run.smithi203:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-08-09T14:01:37.272 DEBUG:teuthology.orchestra.run.smithi043:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-08-09T14:01:37.282 DEBUG:teuthology.orchestra.run.smithi203:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-08-09T14:02:07.431 DEBUG:teuthology.orchestra.run.smithi043:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-08-09T14:02:07.440 DEBUG:teuthology.orchestra.run.smithi203:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-08-09T14:02:37.585 DEBUG:teuthology.orchestra.run.smithi043:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-08-09T14:02:37.594 DEBUG:teuthology.orchestra.run.smithi203:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-08-09T14:03:07.726 DEBUG:teuthology.orchestra.run.smithi043:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-08-09T14:03:07.736 DEBUG:teuthology.orchestra.run.smithi203:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-08-09T14:03:37.880 DEBUG:teuthology.orchestra.run.smithi043:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-08-09T14:03:37.889 DEBUG:teuthology.orchestra.run.smithi203:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-08-09T14:04:08.058 DEBUG:teuthology.orchestra.run.smithi043:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-08-09T14:04:08.068 DEBUG:teuthology.orchestra.run.smithi203:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-08-09T14:04:38.264 DEBUG:teuthology.orchestra.run.smithi043:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-08-09T14:04:38.273 DEBUG:teuthology.orchestra.run.smithi203:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-08-09T14:05:08.422 DEBUG:teuthology.orchestra.run.smithi043:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-08-09T14:05:08.431 DEBUG:teuthology.orchestra.run.smithi203:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-08-09T14:05:38.577 DEBUG:teuthology.orchestra.run.smithi043:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-08-09T14:05:38.587 DEBUG:teuthology.orchestra.run.smithi203:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-08-09T14:06:08.742 DEBUG:teuthology.orchestra.run.smithi043:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-08-09T14:06:08.751 DEBUG:teuthology.orchestra.run.smithi203:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-08-09T14:06:38.892 DEBUG:teuthology.orchestra.run.smithi043:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-08-09T14:06:38.902 DEBUG:teuthology.orchestra.run.smithi203:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-08-09T14:07:09.049 DEBUG:teuthology.orchestra.run.smithi043:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-08-09T14:07:09.058 DEBUG:teuthology.orchestra.run.smithi203:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-08-09T14:07:39.229 DEBUG:teuthology.orchestra.run.smithi043:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-08-09T14:07:39.238 DEBUG:teuthology.orchestra.run.smithi203:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-08-09T14:08:09.379 DEBUG:teuthology.orchestra.run.smithi043:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-08-09T14:08:09.388 DEBUG:teuthology.orchestra.run.smithi203:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-08-09T14:08:39.549 DEBUG:teuthology.orchestra.run.smithi043:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-08-09T14:08:39.559 DEBUG:teuthology.orchestra.run.smithi203:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-08-09T14:09:09.799 DEBUG:teuthology.orchestra.run.smithi043:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-08-09T14:09:09.808 DEBUG:teuthology.orchestra.run.smithi203:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-08-09T14:09:39.982 DEBUG:teuthology.orchestra.run.smithi043:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-08-09T14:09:39.991 DEBUG:teuthology.orchestra.run.smithi203:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-08-09T14:10:10.135 DEBUG:teuthology.orchestra.run.smithi043:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-08-09T14:10:10.144 DEBUG:teuthology.orchestra.run.smithi203:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-08-09T14:10:40.300 DEBUG:teuthology.orchestra.run.smithi043:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-08-09T14:10:40.309 DEBUG:teuthology.orchestra.run.smithi203:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-08-09T14:11:10.468 DEBUG:teuthology.orchestra.run.smithi043:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-08-09T14:11:10.477 DEBUG:teuthology.orchestra.run.smithi203:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-08-09T14:11:40.675 DEBUG:teuthology.orchestra.run.smithi043:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-08-09T14:11:40.685 DEBUG:teuthology.orchestra.run.smithi203:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-08-09T14:12:10.853 DEBUG:teuthology.orchestra.run.smithi043:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-08-09T14:12:10.862 DEBUG:teuthology.orchestra.run.smithi203:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-08-09T14:12:41.008 DEBUG:teuthology.orchestra.run.smithi043:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-08-09T14:12:41.028 DEBUG:teuthology.orchestra.run.smithi203:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-08-09T14:13:11.163 DEBUG:teuthology.orchestra.run.smithi043:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-08-09T14:13:11.172 DEBUG:teuthology.orchestra.run.smithi203:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-08-09T14:13:41.354 DEBUG:teuthology.orchestra.run.smithi043:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-08-09T14:13:41.364 DEBUG:teuthology.orchestra.run.smithi203:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-08-09T14:14:11.494 DEBUG:teuthology.orchestra.run.smithi043:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-08-09T14:14:11.504 DEBUG:teuthology.orchestra.run.smithi203:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-08-09T14:14:41.650 DEBUG:teuthology.orchestra.run.smithi043:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-08-09T14:14:41.660 DEBUG:teuthology.orchestra.run.smithi203:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-08-09T14:15:11.800 DEBUG:teuthology.orchestra.run.smithi043:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-08-09T14:15:11.810 DEBUG:teuthology.orchestra.run.smithi203:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-08-09T14:15:41.953 DEBUG:teuthology.orchestra.run.smithi043:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-08-09T14:15:41.963 DEBUG:teuthology.orchestra.run.smithi203:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-08-09T14:16:12.123 DEBUG:teuthology.orchestra.run.smithi043:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-08-09T14:16:12.132 DEBUG:teuthology.orchestra.run.smithi203:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-08-09T14:16:42.280 DEBUG:teuthology.orchestra.run.smithi043:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-08-09T14:16:42.290 DEBUG:teuthology.orchestra.run.smithi203:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-08-09T14:17:12.448 DEBUG:teuthology.orchestra.run.smithi043:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-08-09T14:17:12.458 DEBUG:teuthology.orchestra.run.smithi203:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-08-09T14:17:42.612 DEBUG:teuthology.orchestra.run.smithi043:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-08-09T14:17:42.621 DEBUG:teuthology.orchestra.run.smithi203:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-08-09T14:18:12.760 DEBUG:teuthology.orchestra.run.smithi043:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-08-09T14:18:12.770 DEBUG:teuthology.orchestra.run.smithi203:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-08-09T14:18:42.951 DEBUG:teuthology.orchestra.run.smithi043:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-08-09T14:18:42.960 DEBUG:teuthology.orchestra.run.smithi203:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-08-09T14:19:13.107 DEBUG:teuthology.orchestra.run.smithi043:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-08-09T14:19:13.116 DEBUG:teuthology.orchestra.run.smithi203:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-08-09T14:19:43.254 DEBUG:teuthology.orchestra.run.smithi043:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-08-09T14:19:43.264 DEBUG:teuthology.orchestra.run.smithi203:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-08-09T14:20:13.405 DEBUG:teuthology.orchestra.run.smithi043:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-08-09T14:20:13.414 DEBUG:teuthology.orchestra.run.smithi203:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-08-09T14:20:43.570 DEBUG:teuthology.orchestra.run.smithi043:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-08-09T14:20:43.579 DEBUG:teuthology.orchestra.run.smithi203:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-08-09T14:21:13.751 DEBUG:teuthology.orchestra.run.smithi043:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-08-09T14:21:13.760 DEBUG:teuthology.orchestra.run.smithi203:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-08-09T14:21:43.914 DEBUG:teuthology.orchestra.run.smithi043:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-08-09T14:21:43.924 DEBUG:teuthology.orchestra.run.smithi203:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-08-09T14:22:14.173 DEBUG:teuthology.orchestra.run.smithi043:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-08-09T14:22:14.183 DEBUG:teuthology.orchestra.run.smithi203:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-08-09T14:22:44.329 DEBUG:teuthology.orchestra.run.smithi043:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-08-09T14:22:44.338 DEBUG:teuthology.orchestra.run.smithi203:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-08-09T14:23:14.487 DEBUG:teuthology.orchestra.run.smithi043:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-08-09T14:23:14.497 DEBUG:teuthology.orchestra.run.smithi203:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-08-09T14:23:44.709 DEBUG:teuthology.orchestra.run.smithi043:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-08-09T14:23:44.720 DEBUG:teuthology.orchestra.run.smithi203:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-08-09T14:24:14.861 DEBUG:teuthology.orchestra.run.smithi043:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-08-09T14:24:14.871 DEBUG:teuthology.orchestra.run.smithi203:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-08-09T14:24:45.064 DEBUG:teuthology.orchestra.run.smithi043:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-08-09T14:24:45.073 DEBUG:teuthology.orchestra.run.smithi203:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-08-09T14:25:15.304 DEBUG:teuthology.orchestra.run.smithi043:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-08-09T14:25:15.315 DEBUG:teuthology.orchestra.run.smithi203:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-08-09T14:25:45.518 DEBUG:teuthology.orchestra.run.smithi043:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-08-09T14:25:45.528 DEBUG:teuthology.orchestra.run.smithi203:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-08-09T14:26:15.741 DEBUG:teuthology.orchestra.run.smithi043:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-08-09T14:26:15.751 DEBUG:teuthology.orchestra.run.smithi203:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-08-09T14:26:45.892 DEBUG:teuthology.orchestra.run.smithi043:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-08-09T14:26:45.902 DEBUG:teuthology.orchestra.run.smithi203:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-08-09T14:27:16.103 DEBUG:teuthology.orchestra.run.smithi043:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-08-09T14:27:16.113 DEBUG:teuthology.orchestra.run.smithi203:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-08-09T14:27:46.256 DEBUG:teuthology.orchestra.run.smithi043:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-08-09T14:27:46.266 DEBUG:teuthology.orchestra.run.smithi203:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-08-09T14:28:16.510 DEBUG:teuthology.orchestra.run.smithi043:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-08-09T14:28:16.521 DEBUG:teuthology.orchestra.run.smithi203:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-08-09T14:28:46.661 DEBUG:teuthology.orchestra.run.smithi043:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-08-09T14:28:46.670 DEBUG:teuthology.orchestra.run.smithi203:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-08-09T14:29:16.857 DEBUG:teuthology.orchestra.run.smithi043:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-08-09T14:29:16.867 DEBUG:teuthology.orchestra.run.smithi203:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-08-09T14:29:47.015 DEBUG:teuthology.orchestra.run.smithi043:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-08-09T14:29:47.025 DEBUG:teuthology.orchestra.run.smithi203:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-08-09T14:30:17.179 DEBUG:teuthology.orchestra.run.smithi043:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-08-09T14:30:17.189 DEBUG:teuthology.orchestra.run.smithi203:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-08-09T14:30:47.328 DEBUG:teuthology.orchestra.run.smithi043:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-08-09T14:30:47.339 DEBUG:teuthology.orchestra.run.smithi203:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-08-09T14:31:17.493 DEBUG:teuthology.orchestra.run.smithi043:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-08-09T14:31:17.502 DEBUG:teuthology.orchestra.run.smithi203:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-08-09T14:31:47.700 DEBUG:teuthology.orchestra.run.smithi043:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-08-09T14:31:47.710 DEBUG:teuthology.orchestra.run.smithi203:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-08-09T14:32:17.881 DEBUG:teuthology.orchestra.run.smithi043:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-08-09T14:32:17.890 DEBUG:teuthology.orchestra.run.smithi203:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-08-09T14:32:48.029 DEBUG:teuthology.orchestra.run.smithi043:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-08-09T14:32:48.038 DEBUG:teuthology.orchestra.run.smithi203:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-08-09T14:33:18.193 DEBUG:teuthology.orchestra.run.smithi043:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-08-09T14:33:18.202 DEBUG:teuthology.orchestra.run.smithi203:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-08-09T14:33:48.359 DEBUG:teuthology.orchestra.run.smithi043:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-08-09T14:33:48.368 DEBUG:teuthology.orchestra.run.smithi203:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-08-09T14:34:18.564 DEBUG:teuthology.orchestra.run.smithi043:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-08-09T14:34:18.575 DEBUG:teuthology.orchestra.run.smithi203:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-08-09T14:34:48.721 DEBUG:teuthology.orchestra.run.smithi043:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-08-09T14:34:48.730 DEBUG:teuthology.orchestra.run.smithi203:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-08-09T14:35:18.910 DEBUG:teuthology.orchestra.run.smithi043:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-08-09T14:35:18.920 DEBUG:teuthology.orchestra.run.smithi203:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-08-09T14:35:49.067 DEBUG:teuthology.orchestra.run.smithi043:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-08-09T14:35:49.077 DEBUG:teuthology.orchestra.run.smithi203:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-08-09T14:36:19.298 DEBUG:teuthology.orchestra.run.smithi043:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-08-09T14:36:19.308 DEBUG:teuthology.orchestra.run.smithi203:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-08-09T14:36:49.465 DEBUG:teuthology.orchestra.run.smithi043:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-08-09T14:36:49.475 DEBUG:teuthology.orchestra.run.smithi203:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-08-09T14:37:19.679 DEBUG:teuthology.orchestra.run.smithi043:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-08-09T14:37:19.688 DEBUG:teuthology.orchestra.run.smithi203:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-08-09T14:37:49.843 DEBUG:teuthology.orchestra.run.smithi043:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-08-09T14:37:49.852 DEBUG:teuthology.orchestra.run.smithi203:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-08-09T14:38:20.041 DEBUG:teuthology.orchestra.run.smithi043:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-08-09T14:38:20.052 DEBUG:teuthology.orchestra.run.smithi203:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-08-09T14:38:50.206 DEBUG:teuthology.orchestra.run.smithi043:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-08-09T14:38:50.216 DEBUG:teuthology.orchestra.run.smithi203:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-08-09T14:39:20.485 DEBUG:teuthology.orchestra.run.smithi043:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-08-09T14:39:20.495 DEBUG:teuthology.orchestra.run.smithi203:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-08-09T14:39:50.645 DEBUG:teuthology.orchestra.run.smithi043:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-08-09T14:39:50.655 DEBUG:teuthology.orchestra.run.smithi203:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-08-09T14:40:20.800 DEBUG:teuthology.orchestra.run.smithi043:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-08-09T14:40:20.810 DEBUG:teuthology.orchestra.run.smithi203:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-08-09T14:40:51.015 DEBUG:teuthology.orchestra.run.smithi043:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-08-09T14:40:51.025 DEBUG:teuthology.orchestra.run.smithi203:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-08-09T14:41:21.181 DEBUG:teuthology.orchestra.run.smithi043:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-08-09T14:41:21.191 DEBUG:teuthology.orchestra.run.smithi203:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-08-09T14:41:51.328 DEBUG:teuthology.orchestra.run.smithi043:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-08-09T14:41:51.337 DEBUG:teuthology.orchestra.run.smithi203:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-08-09T14:42:21.476 DEBUG:teuthology.orchestra.run.smithi043:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-08-09T14:42:21.486 DEBUG:teuthology.orchestra.run.smithi203:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-08-09T14:42:51.692 DEBUG:teuthology.orchestra.run.smithi043:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-08-09T14:42:51.702 DEBUG:teuthology.orchestra.run.smithi203:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-08-09T14:43:21.841 DEBUG:teuthology.orchestra.run.smithi043:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-08-09T14:43:21.850 DEBUG:teuthology.orchestra.run.smithi203:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-08-09T14:43:52.054 DEBUG:teuthology.orchestra.run.smithi043:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-08-09T14:43:52.063 DEBUG:teuthology.orchestra.run.smithi203:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-08-09T14:44:22.204 DEBUG:teuthology.orchestra.run.smithi043:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-08-09T14:44:22.213 DEBUG:teuthology.orchestra.run.smithi203:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-08-09T14:44:52.433 DEBUG:teuthology.orchestra.run.smithi043:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-08-09T14:44:52.443 DEBUG:teuthology.orchestra.run.smithi203:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-08-09T14:45:22.584 DEBUG:teuthology.orchestra.run.smithi043:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-08-09T14:45:22.593 DEBUG:teuthology.orchestra.run.smithi203:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-08-09T14:45:52.748 DEBUG:teuthology.orchestra.run.smithi043:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-08-09T14:45:52.758 DEBUG:teuthology.orchestra.run.smithi203:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-08-09T14:46:22.913 DEBUG:teuthology.orchestra.run.smithi043:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-08-09T14:46:22.922 DEBUG:teuthology.orchestra.run.smithi203:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-08-09T14:46:53.093 DEBUG:teuthology.orchestra.run.smithi043:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-08-09T14:46:53.102 DEBUG:teuthology.orchestra.run.smithi203:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-08-09T14:47:23.269 DEBUG:teuthology.orchestra.run.smithi043:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-08-09T14:47:23.279 DEBUG:teuthology.orchestra.run.smithi203:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-08-09T14:47:53.466 DEBUG:teuthology.orchestra.run.smithi043:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-08-09T14:47:53.475 DEBUG:teuthology.orchestra.run.smithi203:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-08-09T14:48:23.621 DEBUG:teuthology.orchestra.run.smithi043:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-08-09T14:48:23.631 DEBUG:teuthology.orchestra.run.smithi203:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-08-09T14:48:53.871 DEBUG:teuthology.orchestra.run.smithi043:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-08-09T14:48:53.880 DEBUG:teuthology.orchestra.run.smithi203:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-08-09T14:49:24.043 DEBUG:teuthology.orchestra.run.smithi043:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-08-09T14:49:24.052 DEBUG:teuthology.orchestra.run.smithi203:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-08-09T14:49:54.191 DEBUG:teuthology.orchestra.run.smithi043:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-08-09T14:49:54.201 DEBUG:teuthology.orchestra.run.smithi203:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-08-09T14:50:24.339 DEBUG:teuthology.orchestra.run.smithi043:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-08-09T14:50:24.349 DEBUG:teuthology.orchestra.run.smithi203:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-08-09T14:50:54.497 DEBUG:teuthology.orchestra.run.smithi043:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-08-09T14:50:54.506 DEBUG:teuthology.orchestra.run.smithi203:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-08-09T14:51:15.865 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-09T14:51:15.866 DEBUG:teuthology.orchestra.run.smithi043:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-08-09T14:51:15.947 INFO:teuthology.orchestra.run.smithi043.stdout:scheduled-2023-08-09-13_57_00_UTC 2023-08-09T14:51:15.947 INFO:teuthology.orchestra.run.smithi043.stdout:scheduled-2023-08-09-13_58_00_UTC 2023-08-09T14:51:15.948 INFO:teuthology.orchestra.run.smithi043.stdout:scheduled-2023-08-09-13_59_00_UTC 2023-08-09T14:51:15.948 INFO:teuthology.orchestra.run.smithi043.stdout:scheduled-2023-08-09-14_00_00_UTC 2023-08-09T14:51:15.948 INFO:teuthology.orchestra.run.smithi043.stdout:scheduled-2023-08-09-14_01_00_UTC 2023-08-09T14:51:15.949 INFO:teuthology.orchestra.run.smithi043.stdout:scheduled-2023-08-09-14_02_00_UTC 2023-08-09T14:51:15.949 INFO:teuthology.orchestra.run.smithi043.stdout:scheduled-2023-08-09-14_03_00_UTC 2023-08-09T14:51:15.949 INFO:teuthology.orchestra.run.smithi043.stdout:scheduled-2023-08-09-14_04_00_UTC 2023-08-09T14:51:15.949 INFO:teuthology.orchestra.run.smithi043.stdout:scheduled-2023-08-09-14_05_00_UTC 2023-08-09T14:51:15.949 INFO:teuthology.orchestra.run.smithi043.stdout:scheduled-2023-08-09-14_06_00_UTC 2023-08-09T14:51:15.950 INFO:teuthology.orchestra.run.smithi043.stdout:scheduled-2023-08-09-14_07_00_UTC 2023-08-09T14:51:15.950 INFO:teuthology.orchestra.run.smithi043.stdout:scheduled-2023-08-09-14_08_00_UTC 2023-08-09T14:51:15.950 INFO:teuthology.orchestra.run.smithi043.stdout:scheduled-2023-08-09-14_09_00_UTC 2023-08-09T14:51:15.950 INFO:teuthology.orchestra.run.smithi043.stdout:scheduled-2023-08-09-14_10_00_UTC 2023-08-09T14:51:15.950 INFO:teuthology.orchestra.run.smithi043.stdout:scheduled-2023-08-09-14_11_00_UTC 2023-08-09T14:51:15.950 INFO:teuthology.orchestra.run.smithi043.stdout:scheduled-2023-08-09-14_12_00_UTC 2023-08-09T14:51:15.950 INFO:teuthology.orchestra.run.smithi043.stdout:scheduled-2023-08-09-14_13_01_UTC 2023-08-09T14:51:15.950 INFO:teuthology.orchestra.run.smithi043.stdout:scheduled-2023-08-09-14_14_00_UTC 2023-08-09T14:51:15.951 INFO:teuthology.orchestra.run.smithi043.stdout:scheduled-2023-08-09-14_15_00_UTC 2023-08-09T14:51:15.951 INFO:teuthology.orchestra.run.smithi043.stdout:scheduled-2023-08-09-14_16_00_UTC 2023-08-09T14:51:15.951 INFO:teuthology.orchestra.run.smithi043.stdout:scheduled-2023-08-09-14_17_00_UTC 2023-08-09T14:51:15.951 INFO:teuthology.orchestra.run.smithi043.stdout:scheduled-2023-08-09-14_18_00_UTC 2023-08-09T14:51:15.951 INFO:teuthology.orchestra.run.smithi043.stdout:scheduled-2023-08-09-14_19_00_UTC 2023-08-09T14:51:15.951 INFO:teuthology.orchestra.run.smithi043.stdout:scheduled-2023-08-09-14_20_00_UTC 2023-08-09T14:51:15.951 INFO:teuthology.orchestra.run.smithi043.stdout:scheduled-2023-08-09-14_21_00_UTC 2023-08-09T14:51:15.951 INFO:teuthology.orchestra.run.smithi043.stdout:scheduled-2023-08-09-14_22_00_UTC 2023-08-09T14:51:15.952 INFO:teuthology.orchestra.run.smithi043.stdout:scheduled-2023-08-09-14_23_00_UTC 2023-08-09T14:51:15.952 INFO:teuthology.orchestra.run.smithi043.stdout:scheduled-2023-08-09-14_24_00_UTC 2023-08-09T14:51:15.952 INFO:teuthology.orchestra.run.smithi043.stdout:scheduled-2023-08-09-14_25_00_UTC 2023-08-09T14:51:15.952 INFO:teuthology.orchestra.run.smithi043.stdout:scheduled-2023-08-09-14_26_00_UTC 2023-08-09T14:51:15.952 INFO:teuthology.orchestra.run.smithi043.stdout:scheduled-2023-08-09-14_27_00_UTC 2023-08-09T14:51:15.952 INFO:teuthology.orchestra.run.smithi043.stdout:scheduled-2023-08-09-14_28_00_UTC 2023-08-09T14:51:15.952 INFO:teuthology.orchestra.run.smithi043.stdout:scheduled-2023-08-09-14_29_00_UTC 2023-08-09T14:51:15.953 INFO:teuthology.orchestra.run.smithi043.stdout:scheduled-2023-08-09-14_30_00_UTC 2023-08-09T14:51:15.953 INFO:teuthology.orchestra.run.smithi043.stdout:scheduled-2023-08-09-14_31_00_UTC 2023-08-09T14:51:15.953 INFO:teuthology.orchestra.run.smithi043.stdout:scheduled-2023-08-09-14_32_00_UTC 2023-08-09T14:51:15.953 INFO:teuthology.orchestra.run.smithi043.stdout:scheduled-2023-08-09-14_33_00_UTC 2023-08-09T14:51:15.953 INFO:teuthology.orchestra.run.smithi043.stdout:scheduled-2023-08-09-14_34_00_UTC 2023-08-09T14:51:15.953 INFO:teuthology.orchestra.run.smithi043.stdout:scheduled-2023-08-09-14_35_00_UTC 2023-08-09T14:51:15.953 INFO:teuthology.orchestra.run.smithi043.stdout:scheduled-2023-08-09-14_36_00_UTC 2023-08-09T14:51:15.954 INFO:teuthology.orchestra.run.smithi043.stdout:scheduled-2023-08-09-14_37_00_UTC 2023-08-09T14:51:15.954 INFO:teuthology.orchestra.run.smithi043.stdout:scheduled-2023-08-09-14_38_00_UTC 2023-08-09T14:51:15.954 INFO:teuthology.orchestra.run.smithi043.stdout:scheduled-2023-08-09-14_39_00_UTC 2023-08-09T14:51:15.954 INFO:teuthology.orchestra.run.smithi043.stdout:scheduled-2023-08-09-14_40_00_UTC 2023-08-09T14:51:15.954 INFO:teuthology.orchestra.run.smithi043.stdout:scheduled-2023-08-09-14_41_00_UTC 2023-08-09T14:51:15.954 INFO:teuthology.orchestra.run.smithi043.stdout:scheduled-2023-08-09-14_42_00_UTC 2023-08-09T14:51:15.954 INFO:teuthology.orchestra.run.smithi043.stdout:scheduled-2023-08-09-14_43_00_UTC 2023-08-09T14:51:15.954 INFO:teuthology.orchestra.run.smithi043.stdout:scheduled-2023-08-09-14_44_00_UTC 2023-08-09T14:51:15.955 INFO:teuthology.orchestra.run.smithi043.stdout:scheduled-2023-08-09-14_45_00_UTC 2023-08-09T14:51:15.955 INFO:teuthology.orchestra.run.smithi043.stdout:scheduled-2023-08-09-14_46_00_UTC 2023-08-09T14:51:15.955 INFO:teuthology.orchestra.run.smithi043.stdout:scheduled-2023-08-09-14_47_00_UTC 2023-08-09T14:51:15.955 INFO:teuthology.orchestra.run.smithi043.stdout:scheduled-2023-08-09-14_48_00_UTC 2023-08-09T14:51:15.955 INFO:teuthology.orchestra.run.smithi043.stdout:scheduled-2023-08-09-14_49_00_UTC 2023-08-09T14:51:15.955 INFO:teuthology.orchestra.run.smithi043.stdout:scheduled-2023-08-09-14_50_00_UTC 2023-08-09T14:51:15.955 INFO:teuthology.orchestra.run.smithi043.stdout:scheduled-2023-08-09-14_51_00_UTC 2023-08-09T14:51:15.956 DEBUG:tasks.cephfs.test_snap_schedules:snapshots: ['scheduled-2023-08-09-13_57_00_UTC', 'scheduled-2023-08-09-13_58_00_UTC', 'scheduled-2023-08-09-13_59_00_UTC', 'scheduled-2023-08-09-14_00_00_UTC', 'scheduled-2023-08-09-14_01_00_UTC', 'scheduled-2023-08-09-14_02_00_UTC', 'scheduled-2023-08-09-14_03_00_UTC', 'scheduled-2023-08-09-14_04_00_UTC', 'scheduled-2023-08-09-14_05_00_UTC', 'scheduled-2023-08-09-14_06_00_UTC', 'scheduled-2023-08-09-14_07_00_UTC', 'scheduled-2023-08-09-14_08_00_UTC', 'scheduled-2023-08-09-14_09_00_UTC', 'scheduled-2023-08-09-14_10_00_UTC', 'scheduled-2023-08-09-14_11_00_UTC', 'scheduled-2023-08-09-14_12_00_UTC', 'scheduled-2023-08-09-14_13_01_UTC', 'scheduled-2023-08-09-14_14_00_UTC', 'scheduled-2023-08-09-14_15_00_UTC', 'scheduled-2023-08-09-14_16_00_UTC', 'scheduled-2023-08-09-14_17_00_UTC', 'scheduled-2023-08-09-14_18_00_UTC', 'scheduled-2023-08-09-14_19_00_UTC', 'scheduled-2023-08-09-14_20_00_UTC', 'scheduled-2023-08-09-14_21_00_UTC', 'scheduled-2023-08-09-14_22_00_UTC', 'scheduled-2023-08-09-14_23_00_UTC', 'scheduled-2023-08-09-14_24_00_UTC', 'scheduled-2023-08-09-14_25_00_UTC', 'scheduled-2023-08-09-14_26_00_UTC', 'scheduled-2023-08-09-14_27_00_UTC', 'scheduled-2023-08-09-14_28_00_UTC', 'scheduled-2023-08-09-14_29_00_UTC', 'scheduled-2023-08-09-14_30_00_UTC', 'scheduled-2023-08-09-14_31_00_UTC', 'scheduled-2023-08-09-14_32_00_UTC', 'scheduled-2023-08-09-14_33_00_UTC', 'scheduled-2023-08-09-14_34_00_UTC', 'scheduled-2023-08-09-14_35_00_UTC', 'scheduled-2023-08-09-14_36_00_UTC', 'scheduled-2023-08-09-14_37_00_UTC', 'scheduled-2023-08-09-14_38_00_UTC', 'scheduled-2023-08-09-14_39_00_UTC', 'scheduled-2023-08-09-14_40_00_UTC', 'scheduled-2023-08-09-14_41_00_UTC', 'scheduled-2023-08-09-14_42_00_UTC', 'scheduled-2023-08-09-14_43_00_UTC', 'scheduled-2023-08-09-14_44_00_UTC', 'scheduled-2023-08-09-14_45_00_UTC', 'scheduled-2023-08-09-14_46_00_UTC', 'scheduled-2023-08-09-14_47_00_UTC', 'scheduled-2023-08-09-14_48_00_UTC', 'scheduled-2023-08-09-14_49_00_UTC', 'scheduled-2023-08-09-14_50_00_UTC', 'scheduled-2023-08-09-14_51_00_UTC'] 2023-08-09T14:51:15.957 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs snap-schedule status --fs cephfs --format json /snap_test_dir1 2023-08-09T14:51:16.205 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:16.204+0000 7fef095fd640 1 Processor -- start 2023-08-09T14:51:16.205 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:16.204+0000 7fef095fd640 1 -- start start 2023-08-09T14:51:16.206 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:16.205+0000 7fef095fd640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7fef0406fb70 0x7fef040caab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T14:51:16.206 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:16.206+0000 7fef095fd640 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7fef040ce4c0 0x7fef040ce950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T14:51:16.206 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:16.206+0000 7fef095fd640 1 -- --> v1:172.21.15.203:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fef0415b9b0 con 0x7fef040ca370 2023-08-09T14:51:16.206 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:16.206+0000 7fef095fd640 1 -- --> v2:172.21.15.43:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fef040cee90 con 0x7fef0406fb70 2023-08-09T14:51:16.207 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:16.206+0000 7fef095fd640 1 -- --> v2:172.21.15.203:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fef0415ae10 con 0x7fef040ce4c0 2023-08-09T14:51:16.207 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:16.206+0000 7fef02ffd640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7fef0406fb70 0x7fef040caab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T14:51:16.207 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:16.206+0000 7fef02ffd640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7fef0406fb70 0x7fef040caab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:39024/0 (socket says 172.21.15.43:39024) 2023-08-09T14:51:16.208 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:16.206+0000 7fef02ffd640 1 -- 172.21.15.43:0/3376096597 learned_addr learned my addr 172.21.15.43:0/3376096597 (peer_addr_for_me v2:172.21.15.43:0/0) 2023-08-09T14:51:16.208 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:16.206+0000 7fef037fe640 1 --2- 172.21.15.43:0/3376096597 >> v2:172.21.15.203:3300/0 conn(0x7fef040ce4c0 0x7fef040ce950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T14:51:16.208 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:16.207+0000 7fef02ffd640 1 -- 172.21.15.43:0/3376096597 >> v2:172.21.15.203:3300/0 conn(0x7fef040ce4c0 msgr2=0x7fef040ce950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-09T14:51:16.208 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:16.207+0000 7fef02ffd640 1 --2- 172.21.15.43:0/3376096597 >> v2:172.21.15.203:3300/0 conn(0x7fef040ce4c0 0x7fef040ce950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T14:51:16.208 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:16.207+0000 7fef02ffd640 1 -- 172.21.15.43:0/3376096597 >> v1:172.21.15.203:6789/0 conn(0x7fef040ca370 legacy=0x7fef0415b2a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-09T14:51:16.208 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:16.207+0000 7fef02ffd640 1 -- 172.21.15.43:0/3376096597 --> v2:172.21.15.43:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fef04154e70 con 0x7fef0406fb70 2023-08-09T14:51:16.211 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:16.208+0000 7fef02ffd640 1 --2- 172.21.15.43:0/3376096597 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fef0406fb70 0x7fef040caab0 secure :-1 s=READY pgs=1739 cs=0 l=1 rev1=1 crypto rx=0x7fef04069e70 tx=0x7feef402b910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=968ff2d7aee3206c server_cookie=0 in_seq=0 out_seq=0 2023-08-09T14:51:16.211 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:16.209+0000 7fef01ffb640 1 -- 172.21.15.43:0/3376096597 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feef4031870 con 0x7fef0406fb70 2023-08-09T14:51:16.212 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:16.209+0000 7fef01ffb640 1 -- 172.21.15.43:0/3376096597 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7feef40319d0 con 0x7fef0406fb70 2023-08-09T14:51:16.212 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:16.209+0000 7fef01ffb640 1 -- 172.21.15.43:0/3376096597 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feef4031d60 con 0x7fef0406fb70 2023-08-09T14:51:16.213 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:16.209+0000 7fef095fd640 1 -- 172.21.15.43:0/3376096597 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fef0406fb70 msgr2=0x7fef040caab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T14:51:16.213 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:16.209+0000 7fef095fd640 1 --2- 172.21.15.43:0/3376096597 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fef0406fb70 0x7fef040caab0 secure :-1 s=READY pgs=1739 cs=0 l=1 rev1=1 crypto rx=0x7fef04069e70 tx=0x7feef402b910 comp rx=0 tx=0).stop 2023-08-09T14:51:16.213 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:16.209+0000 7fef095fd640 1 -- 172.21.15.43:0/3376096597 shutdown_connections 2023-08-09T14:51:16.213 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:16.209+0000 7fef095fd640 1 --2- 172.21.15.43:0/3376096597 >> v2:172.21.15.203:3300/0 conn(0x7fef040ce4c0 0x7fef040ce950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T14:51:16.213 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:16.209+0000 7fef095fd640 1 --2- 172.21.15.43:0/3376096597 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fef0406fb70 0x7fef040caab0 unknown :-1 s=CLOSED pgs=1739 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T14:51:16.214 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:16.209+0000 7fef095fd640 1 -- 172.21.15.43:0/3376096597 >> 172.21.15.43:0/3376096597 conn(0x7fef0406b990 msgr2=0x7fef0406bda0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-09T14:51:16.214 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:16.209+0000 7fef095fd640 1 -- 172.21.15.43:0/3376096597 shutdown_connections 2023-08-09T14:51:16.214 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:16.210+0000 7fef095fd640 1 -- 172.21.15.43:0/3376096597 wait complete. 2023-08-09T14:51:16.214 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:16.211+0000 7fef095fd640 1 Processor -- start 2023-08-09T14:51:16.214 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:16.212+0000 7fef095fd640 1 -- start start 2023-08-09T14:51:16.214 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:16.213+0000 7fef095fd640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fef0406fb70 0x7fef0414ce90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T14:51:16.214 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:16.213+0000 7fef095fd640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fef040ca370 0x7fef040cc260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T14:51:16.215 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:16.213+0000 7fef095fd640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fef040ce4c0 0x7fef040c25e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T14:51:16.215 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:16.213+0000 7fef095fd640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fef040c2f50 con 0x7fef0406fb70 2023-08-09T14:51:16.215 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:16.213+0000 7fef095fd640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fef0414c0d0 con 0x7fef040ca370 2023-08-09T14:51:16.215 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:16.213+0000 7fef095fd640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fef0414c3b0 con 0x7fef040ce4c0 2023-08-09T14:51:16.216 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:16.213+0000 7fef02ffd640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fef0406fb70 0x7fef0414ce90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T14:51:16.216 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:16.213+0000 7fef02ffd640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fef0406fb70 0x7fef0414ce90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:39040/0 (socket says 172.21.15.43:39040) 2023-08-09T14:51:16.216 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:16.213+0000 7fef02ffd640 1 -- 172.21.15.43:0/3540133235 learned_addr learned my addr 172.21.15.43:0/3540133235 (peer_addr_for_me v2:172.21.15.43:0/0) 2023-08-09T14:51:16.217 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:16.213+0000 7fef027fc640 1 --2- 172.21.15.43:0/3540133235 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fef040ca370 0x7fef040cc260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T14:51:16.217 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:16.213+0000 7fef02ffd640 1 -- 172.21.15.43:0/3540133235 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fef040ce4c0 msgr2=0x7fef040c25e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T14:51:16.217 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:16.213+0000 7fef02ffd640 1 --2- 172.21.15.43:0/3540133235 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fef040ce4c0 0x7fef040c25e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T14:51:16.217 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:16.213+0000 7fef02ffd640 1 -- 172.21.15.43:0/3540133235 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fef040ca370 msgr2=0x7fef040cc260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T14:51:16.217 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:16.213+0000 7fef02ffd640 1 --2- 172.21.15.43:0/3540133235 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fef040ca370 0x7fef040cc260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T14:51:16.217 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:16.213+0000 7fef02ffd640 1 -- 172.21.15.43:0/3540133235 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7feef40094d0 con 0x7fef0406fb70 2023-08-09T14:51:16.217 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:16.214+0000 7fef02ffd640 1 --2- 172.21.15.43:0/3540133235 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fef0406fb70 0x7fef0414ce90 secure :-1 s=READY pgs=1740 cs=0 l=1 rev1=1 crypto rx=0x7feef4000c00 tx=0x7feef40046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-09T14:51:16.217 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:16.214+0000 7feeeb7fe640 1 -- 172.21.15.43:0/3540133235 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feef403abe0 con 0x7fef0406fb70 2023-08-09T14:51:16.218 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:16.214+0000 7feeeb7fe640 1 -- 172.21.15.43:0/3540133235 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7feef403ad40 con 0x7fef0406fb70 2023-08-09T14:51:16.218 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:16.214+0000 7feeeb7fe640 1 -- 172.21.15.43:0/3540133235 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feef4044610 con 0x7fef0406fb70 2023-08-09T14:51:16.218 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:16.214+0000 7fef095fd640 1 -- 172.21.15.43:0/3540133235 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fef0414c690 con 0x7fef0406fb70 2023-08-09T14:51:16.218 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:16.214+0000 7fef095fd640 1 -- 172.21.15.43:0/3540133235 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fef040cc7a0 con 0x7fef0406fb70 2023-08-09T14:51:16.218 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:16.215+0000 7fef095fd640 1 -- 172.21.15.43:0/3540133235 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fef040cc9f0 con 0x7fef0406fb70 2023-08-09T14:51:16.219 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:16.216+0000 7feeeb7fe640 1 -- 172.21.15.43:0/3540133235 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 74) v1 ==== 88472+0+0 (secure 0 0 0) 0x7feef404d450 con 0x7fef0406fb70 2023-08-09T14:51:16.219 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:16.217+0000 7feeeb7fe640 1 --2- 172.21.15.43:0/3540133235 >> [v2:172.21.15.203:6832/372694067,v1:172.21.15.203:6833/372694067] conn(0x7feef006a1c0 0x7feef006c680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T14:51:16.219 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:16.217+0000 7feeeb7fe640 1 -- 172.21.15.43:0/3540133235 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(152..152 src has 1..152) v4 ==== 6872+0+0 (secure 0 0 0) 0x7feef402d040 con 0x7fef0406fb70 2023-08-09T14:51:16.219 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:16.218+0000 7fef027fc640 1 --2- 172.21.15.43:0/3540133235 >> [v2:172.21.15.203:6832/372694067,v1:172.21.15.203:6833/372694067] conn(0x7feef006a1c0 0x7feef006c680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T14:51:16.220 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:16.219+0000 7fef027fc640 1 --2- 172.21.15.43:0/3540133235 >> [v2:172.21.15.203:6832/372694067,v1:172.21.15.203:6833/372694067] conn(0x7feef006a1c0 0x7feef006c680 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7fef040ccc80 tx=0x7feef8009040 comp rx=0 tx=0).ready entity=mgr.6869 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-09T14:51:16.223 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:16.223+0000 7feeeb7fe640 1 -- 172.21.15.43:0/3540133235 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178066 (secure 0 0 0) 0x7feef4076310 con 0x7fef0406fb70 2023-08-09T14:51:16.439 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:16.439+0000 7fef095fd640 1 -- 172.21.15.43:0/3540133235 --> [v2:172.21.15.203:6832/372694067,v1:172.21.15.203:6833/372694067] -- mgr_command(tid 0: {"prefix": "fs snap-schedule status", "fs": "cephfs", "path": "/snap_test_dir1", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7fef040cb320 con 0x7feef006a1c0 2023-08-09T14:51:16.458 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:16.458+0000 7feeeb7fe640 1 -- 172.21.15.43:0/3540133235 <== mgr.6869 v2:172.21.15.203:6832/372694067 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+358 (secure 0 0 0) 0x7feef8003980 con 0x7feef006a1c0 2023-08-09T14:51:16.459 INFO:teuthology.orchestra.run.smithi043.stdout: 2023-08-09T14:51:16.459 INFO:teuthology.orchestra.run.smithi043.stdout:[{"fs": "cephfs", "subvol": null, "path": "/snap_test_dir1", "rel_path": "/snap_test_dir1", "schedule": "1M", "retention": {"n": 55}, "start": "2023-08-09T00:00:00", "created": "2023-08-09T13:55:45", "first": "2023-08-09T13:56:00", "last": "2023-08-09T14:51:00", "last_pruned": "2023-08-09T14:51:00", "created_count": 56, "pruned_count": 1, "active": true}] 2023-08-09T14:51:16.461 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:16.460+0000 7fef095fd640 1 -- 172.21.15.43:0/3540133235 >> [v2:172.21.15.203:6832/372694067,v1:172.21.15.203:6833/372694067] conn(0x7feef006a1c0 msgr2=0x7feef006c680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T14:51:16.461 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:16.460+0000 7fef095fd640 1 --2- 172.21.15.43:0/3540133235 >> [v2:172.21.15.203:6832/372694067,v1:172.21.15.203:6833/372694067] conn(0x7feef006a1c0 0x7feef006c680 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7fef040ccc80 tx=0x7feef8009040 comp rx=0 tx=0).stop 2023-08-09T14:51:16.461 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:16.460+0000 7fef095fd640 1 -- 172.21.15.43:0/3540133235 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fef0406fb70 msgr2=0x7fef0414ce90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T14:51:16.461 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:16.460+0000 7fef095fd640 1 --2- 172.21.15.43:0/3540133235 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fef0406fb70 0x7fef0414ce90 secure :-1 s=READY pgs=1740 cs=0 l=1 rev1=1 crypto rx=0x7feef4000c00 tx=0x7feef40046c0 comp rx=0 tx=0).stop 2023-08-09T14:51:16.461 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:16.460+0000 7fef095fd640 1 -- 172.21.15.43:0/3540133235 shutdown_connections 2023-08-09T14:51:16.462 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:16.460+0000 7fef095fd640 1 --2- 172.21.15.43:0/3540133235 >> [v2:172.21.15.203:6832/372694067,v1:172.21.15.203:6833/372694067] conn(0x7feef006a1c0 0x7feef006c680 unknown :-1 s=CLOSED pgs=68 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T14:51:16.462 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:16.460+0000 7fef095fd640 1 --2- 172.21.15.43:0/3540133235 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fef040ce4c0 0x7fef040c25e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T14:51:16.462 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:16.460+0000 7fef095fd640 1 --2- 172.21.15.43:0/3540133235 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fef040ca370 0x7fef040cc260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T14:51:16.462 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:16.460+0000 7fef095fd640 1 --2- 172.21.15.43:0/3540133235 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fef0406fb70 0x7fef0414ce90 unknown :-1 s=CLOSED pgs=1740 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T14:51:16.462 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:16.460+0000 7fef095fd640 1 -- 172.21.15.43:0/3540133235 >> 172.21.15.43:0/3540133235 conn(0x7fef0406b990 msgr2=0x7fef04154490 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-09T14:51:16.462 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:16.461+0000 7fef095fd640 1 -- 172.21.15.43:0/3540133235 shutdown_connections 2023-08-09T14:51:16.462 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:16.461+0000 7fef095fd640 1 -- 172.21.15.43:0/3540133235 wait complete. 2023-08-09T14:51:16.482 DEBUG:tasks.cephfs.test_snap_schedules:res= [{"fs": "cephfs", "subvol": null, "path": "/snap_test_dir1", "rel_path": "/snap_test_dir1", "schedule": "1M", "retention": {"n": 55}, "start": "2023-08-09T00:00:00", "created": "2023-08-09T13:55:45", "first": "2023-08-09T13:56:00", "last": "2023-08-09T14:51:00", "last_pruned": "2023-08-09T14:51:00", "created_count": 56, "pruned_count": 1, "active": true}] 2023-08-09T14:51:16.483 DEBUG:tasks.cephfs.test_snap_schedules:json_res: {'fs': 'cephfs', 'subvol': None, 'path': '/snap_test_dir1', 'rel_path': '/snap_test_dir1', 'schedule': '1M', 'retention': {'n': 55}, 'start': '2023-08-09T00:00:00', 'created': '2023-08-09T13:55:45', 'first': '2023-08-09T13:56:00', 'last': '2023-08-09T14:51:00', 'last_pruned': '2023-08-09T14:51:00', 'created_count': 56, 'pruned_count': 1, 'active': True} 2023-08-09T14:51:16.483 DEBUG:tasks.cephfs.test_snap_schedules:fs_count: 55 2023-08-09T14:51:16.483 DEBUG:tasks.cephfs.test_snap_schedules:db_count: 56 2023-08-09T14:51:16.483 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs snap-schedule remove --fs cephfs /snap_test_dir1 2023-08-09T14:51:16.726 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:16.726+0000 7efd0dfc6640 1 Processor -- start 2023-08-09T14:51:16.726 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:16.726+0000 7efd0dfc6640 1 -- start start 2023-08-09T14:51:16.727 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:16.727+0000 7efd0dfc6640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7efd0815c720 0x7efd080cd850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T14:51:16.728 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:16.727+0000 7efd0dfc6640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7efd080d0900 0x7efd080cec60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T14:51:16.728 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:16.727+0000 7efd0dfc6640 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7efd080cf1a0 0x7efd080b3cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T14:51:16.728 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:16.727+0000 7efd0dfc6640 1 -- --> v2:172.21.15.43:3300/0 -- mon_getmap magic: 0 v1 -- 0x7efd0815cb00 con 0x7efd0815c720 2023-08-09T14:51:16.728 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:16.727+0000 7efd0dfc6640 1 -- --> v2:172.21.15.203:3300/0 -- mon_getmap magic: 0 v1 -- 0x7efd080cf630 con 0x7efd080cf1a0 2023-08-09T14:51:16.728 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:16.727+0000 7efd0dfc6640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7efd080d1960 con 0x7efd080d0900 2023-08-09T14:51:16.729 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:16.728+0000 7efd0cfc4640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7efd0815c720 0x7efd080cd850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T14:51:16.729 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:16.728+0000 7efd0cfc4640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7efd0815c720 0x7efd080cd850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:39042/0 (socket says 172.21.15.43:39042) 2023-08-09T14:51:16.729 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:16.728+0000 7efd0cfc4640 1 -- 172.21.15.43:0/2333318067 learned_addr learned my addr 172.21.15.43:0/2333318067 (peer_addr_for_me v2:172.21.15.43:0/0) 2023-08-09T14:51:16.730 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:16.728+0000 7efd0d7c5640 1 --2- 172.21.15.43:0/2333318067 >> v2:172.21.15.203:3300/0 conn(0x7efd080cf1a0 0x7efd080b3cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T14:51:16.730 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:16.728+0000 7efcfffff640 1 --2- 172.21.15.43:0/2333318067 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7efd080d0900 0x7efd080cec60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T14:51:16.730 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:16.728+0000 7efd0cfc4640 1 -- 172.21.15.43:0/2333318067 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7efd080d0900 msgr2=0x7efd080cec60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-09T14:51:16.730 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:16.728+0000 7efd0cfc4640 1 --2- 172.21.15.43:0/2333318067 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7efd080d0900 0x7efd080cec60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T14:51:16.730 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:16.728+0000 7efd0cfc4640 1 -- 172.21.15.43:0/2333318067 >> v2:172.21.15.203:3300/0 conn(0x7efd080cf1a0 msgr2=0x7efd080b3cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-09T14:51:16.730 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:16.728+0000 7efd0cfc4640 1 --2- 172.21.15.43:0/2333318067 >> v2:172.21.15.203:3300/0 conn(0x7efd080cf1a0 0x7efd080b3cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T14:51:16.730 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:16.728+0000 7efd0cfc4640 1 -- 172.21.15.43:0/2333318067 --> v2:172.21.15.43:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efd080b5660 con 0x7efd0815c720 2023-08-09T14:51:16.731 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:16.729+0000 7efd0cfc4640 1 --2- 172.21.15.43:0/2333318067 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7efd0815c720 0x7efd080cd850 secure :-1 s=READY pgs=1741 cs=0 l=1 rev1=1 crypto rx=0x7efd080b8380 tx=0x7efd0402b8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b029f975e6cc4000 server_cookie=0 in_seq=0 out_seq=0 2023-08-09T14:51:16.732 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:16.730+0000 7efcff7fe640 1 -- 172.21.15.43:0/2333318067 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efd04031640 con 0x7efd0815c720 2023-08-09T14:51:16.732 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:16.730+0000 7efcff7fe640 1 -- 172.21.15.43:0/2333318067 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efd040317a0 con 0x7efd0815c720 2023-08-09T14:51:16.732 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:16.730+0000 7efcff7fe640 1 -- 172.21.15.43:0/2333318067 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efd04031b30 con 0x7efd0815c720 2023-08-09T14:51:16.732 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:16.730+0000 7efd0dfc6640 1 -- 172.21.15.43:0/2333318067 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7efd0815c720 msgr2=0x7efd080cd850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T14:51:16.732 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:16.730+0000 7efd0dfc6640 1 --2- 172.21.15.43:0/2333318067 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7efd0815c720 0x7efd080cd850 secure :-1 s=READY pgs=1741 cs=0 l=1 rev1=1 crypto rx=0x7efd080b8380 tx=0x7efd0402b8f0 comp rx=0 tx=0).stop 2023-08-09T14:51:16.732 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:16.730+0000 7efd0dfc6640 1 -- 172.21.15.43:0/2333318067 shutdown_connections 2023-08-09T14:51:16.732 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:16.730+0000 7efd0dfc6640 1 --2- 172.21.15.43:0/2333318067 >> v2:172.21.15.203:3300/0 conn(0x7efd080cf1a0 0x7efd080b3cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T14:51:16.733 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:16.730+0000 7efd0dfc6640 1 --2- 172.21.15.43:0/2333318067 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7efd080d0900 0x7efd080cec60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T14:51:16.733 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:16.730+0000 7efd0dfc6640 1 --2- 172.21.15.43:0/2333318067 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7efd0815c720 0x7efd080cd850 unknown :-1 s=CLOSED pgs=1741 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T14:51:16.733 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:16.730+0000 7efd0dfc6640 1 -- 172.21.15.43:0/2333318067 >> 172.21.15.43:0/2333318067 conn(0x7efd0806a2a0 msgr2=0x7efd0806a680 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-09T14:51:16.733 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:16.730+0000 7efd0dfc6640 1 -- 172.21.15.43:0/2333318067 shutdown_connections 2023-08-09T14:51:16.733 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:16.730+0000 7efd0dfc6640 1 -- 172.21.15.43:0/2333318067 wait complete. 2023-08-09T14:51:16.734 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:16.732+0000 7efd0dfc6640 1 Processor -- start 2023-08-09T14:51:16.734 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:16.732+0000 7efd0dfc6640 1 -- start start 2023-08-09T14:51:16.734 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:16.733+0000 7efd0dfc6640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7efd080cf1a0 0x7efd0815eda0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T14:51:16.734 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:16.733+0000 7efd0dfc6640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7efd080d0900 0x7efd0815f2e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T14:51:16.735 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:16.733+0000 7efd0dfc6640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7efd0815c720 0x7efd0815ce70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T14:51:16.735 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:16.733+0000 7efd0dfc6640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efd080b2500 con 0x7efd080d0900 2023-08-09T14:51:16.735 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:16.733+0000 7efd0dfc6640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efd0815e760 con 0x7efd080cf1a0 2023-08-09T14:51:16.735 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:16.733+0000 7efd0dfc6640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7efd0815dbe0 con 0x7efd0815c720 2023-08-09T14:51:16.735 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:16.734+0000 7efcfffff640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7efd080d0900 0x7efd0815f2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T14:51:16.735 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:16.734+0000 7efcfffff640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7efd080d0900 0x7efd0815f2e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:39050/0 (socket says 172.21.15.43:39050) 2023-08-09T14:51:16.735 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:16.734+0000 7efcfffff640 1 -- 172.21.15.43:0/2765564847 learned_addr learned my addr 172.21.15.43:0/2765564847 (peer_addr_for_me v2:172.21.15.43:0/0) 2023-08-09T14:51:16.736 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:16.734+0000 7efd0cfc4640 1 --2- 172.21.15.43:0/2765564847 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7efd080cf1a0 0x7efd0815eda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T14:51:16.737 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:16.734+0000 7efd0d7c5640 1 --2- 172.21.15.43:0/2765564847 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7efd0815c720 0x7efd0815ce70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T14:51:16.737 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:16.734+0000 7efcfffff640 1 -- 172.21.15.43:0/2765564847 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7efd0815c720 msgr2=0x7efd0815ce70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T14:51:16.737 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:16.734+0000 7efcfffff640 1 --2- 172.21.15.43:0/2765564847 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7efd0815c720 0x7efd0815ce70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T14:51:16.737 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:16.734+0000 7efcfffff640 1 -- 172.21.15.43:0/2765564847 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7efd080cf1a0 msgr2=0x7efd0815eda0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T14:51:16.737 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:16.734+0000 7efcfffff640 1 --2- 172.21.15.43:0/2765564847 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7efd080cf1a0 0x7efd0815eda0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T14:51:16.737 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:16.734+0000 7efcfffff640 1 -- 172.21.15.43:0/2765564847 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efd040094d0 con 0x7efd080d0900 2023-08-09T14:51:16.737 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:16.735+0000 7efcfffff640 1 --2- 172.21.15.43:0/2765564847 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7efd080d0900 0x7efd0815f2e0 secure :-1 s=READY pgs=1742 cs=0 l=1 rev1=1 crypto rx=0x7efcf800d800 tx=0x7efcf800dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-09T14:51:16.738 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:16.735+0000 7efcfd7fa640 1 -- 172.21.15.43:0/2765564847 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efcf800b6c0 con 0x7efd080d0900 2023-08-09T14:51:16.738 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:16.735+0000 7efcfd7fa640 1 -- 172.21.15.43:0/2765564847 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efcf800b820 con 0x7efd080d0900 2023-08-09T14:51:16.738 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:16.735+0000 7efcfd7fa640 1 -- 172.21.15.43:0/2765564847 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efcf800bbb0 con 0x7efd080d0900 2023-08-09T14:51:16.738 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:16.735+0000 7efd0dfc6640 1 -- 172.21.15.43:0/2765564847 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7efd0815dec0 con 0x7efd080d0900 2023-08-09T14:51:16.738 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:16.735+0000 7efd0dfc6640 1 -- 172.21.15.43:0/2765564847 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7efd0815e480 con 0x7efd080d0900 2023-08-09T14:51:16.738 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:16.736+0000 7efd0dfc6640 1 -- 172.21.15.43:0/2765564847 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7efd080739f0 con 0x7efd080d0900 2023-08-09T14:51:16.739 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:16.737+0000 7efcfd7fa640 1 -- 172.21.15.43:0/2765564847 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 74) v1 ==== 88472+0+0 (secure 0 0 0) 0x7efcf800bd10 con 0x7efd080d0900 2023-08-09T14:51:16.739 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:16.738+0000 7efcfd7fa640 1 --2- 172.21.15.43:0/2765564847 >> [v2:172.21.15.203:6832/372694067,v1:172.21.15.203:6833/372694067] conn(0x7efcf406a1c0 0x7efcf406c680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T14:51:16.739 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:16.738+0000 7efcfd7fa640 1 -- 172.21.15.43:0/2765564847 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(152..152 src has 1..152) v4 ==== 6872+0+0 (secure 0 0 0) 0x7efcf8089060 con 0x7efd080d0900 2023-08-09T14:51:16.739 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:16.739+0000 7efd0cfc4640 1 --2- 172.21.15.43:0/2765564847 >> [v2:172.21.15.203:6832/372694067,v1:172.21.15.203:6833/372694067] conn(0x7efcf406a1c0 0x7efcf406c680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T14:51:16.740 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:16.740+0000 7efd0cfc4640 1 --2- 172.21.15.43:0/2765564847 >> [v2:172.21.15.203:6832/372694067,v1:172.21.15.203:6833/372694067] conn(0x7efcf406a1c0 0x7efcf406c680 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7efd04036000 tx=0x7efd0403c040 comp rx=0 tx=0).ready entity=mgr.6869 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-09T14:51:16.743 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:16.742+0000 7efcfd7fa640 1 -- 172.21.15.43:0/2765564847 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178066 (secure 0 0 0) 0x7efcf80bb7b0 con 0x7efd080d0900 2023-08-09T14:51:16.965 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:16.965+0000 7efd0dfc6640 1 -- 172.21.15.43:0/2765564847 --> [v2:172.21.15.203:6832/372694067,v1:172.21.15.203:6833/372694067] -- mgr_command(tid 0: {"prefix": "fs snap-schedule remove", "fs": "cephfs", "path": "/snap_test_dir1", "target": ["mon-mgr", ""]}) v1 -- 0x7efd080d10c0 con 0x7efcf406a1c0 2023-08-09T14:51:17.139 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:17.138+0000 7efcfd7fa640 1 -- 172.21.15.43:0/2765564847 <== mgr.6869 v2:172.21.15.203:6832/372694067 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+42 (secure 0 0 0) 0x7efd04031990 con 0x7efcf406a1c0 2023-08-09T14:51:17.139 INFO:teuthology.orchestra.run.smithi043.stdout:Schedule removed for path /snap_test_dir1 2023-08-09T14:51:17.143 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:17.141+0000 7efd0dfc6640 1 -- 172.21.15.43:0/2765564847 >> [v2:172.21.15.203:6832/372694067,v1:172.21.15.203:6833/372694067] conn(0x7efcf406a1c0 msgr2=0x7efcf406c680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T14:51:17.143 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:17.141+0000 7efd0dfc6640 1 --2- 172.21.15.43:0/2765564847 >> [v2:172.21.15.203:6832/372694067,v1:172.21.15.203:6833/372694067] conn(0x7efcf406a1c0 0x7efcf406c680 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7efd04036000 tx=0x7efd0403c040 comp rx=0 tx=0).stop 2023-08-09T14:51:17.143 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:17.142+0000 7efd0dfc6640 1 -- 172.21.15.43:0/2765564847 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7efd080d0900 msgr2=0x7efd0815f2e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T14:51:17.144 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:17.142+0000 7efd0dfc6640 1 --2- 172.21.15.43:0/2765564847 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7efd080d0900 0x7efd0815f2e0 secure :-1 s=READY pgs=1742 cs=0 l=1 rev1=1 crypto rx=0x7efcf800d800 tx=0x7efcf800dcc0 comp rx=0 tx=0).stop 2023-08-09T14:51:17.144 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:17.142+0000 7efd0dfc6640 1 -- 172.21.15.43:0/2765564847 shutdown_connections 2023-08-09T14:51:17.144 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:17.142+0000 7efd0dfc6640 1 --2- 172.21.15.43:0/2765564847 >> [v2:172.21.15.203:6832/372694067,v1:172.21.15.203:6833/372694067] conn(0x7efcf406a1c0 0x7efcf406c680 unknown :-1 s=CLOSED pgs=69 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T14:51:17.145 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:17.142+0000 7efd0dfc6640 1 --2- 172.21.15.43:0/2765564847 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7efd0815c720 0x7efd0815ce70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T14:51:17.145 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:17.142+0000 7efd0dfc6640 1 --2- 172.21.15.43:0/2765564847 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7efd080d0900 0x7efd0815f2e0 unknown :-1 s=CLOSED pgs=1742 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T14:51:17.145 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:17.142+0000 7efd0dfc6640 1 --2- 172.21.15.43:0/2765564847 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7efd080cf1a0 0x7efd0815eda0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T14:51:17.145 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:17.142+0000 7efd0dfc6640 1 -- 172.21.15.43:0/2765564847 >> 172.21.15.43:0/2765564847 conn(0x7efd0806a2a0 msgr2=0x7efd080cbb40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-09T14:51:17.145 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:17.142+0000 7efd0dfc6640 1 -- 172.21.15.43:0/2765564847 shutdown_connections 2023-08-09T14:51:17.146 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:17.142+0000 7efd0dfc6640 1 -- 172.21.15.43:0/2765564847 wait complete. 2023-08-09T14:51:17.163 DEBUG:tasks.cephfs.test_snap_schedules:res=Schedule removed for path /snap_test_dir1 2023-08-09T14:51:17.163 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-09T14:51:17.163 DEBUG:teuthology.orchestra.run.smithi043:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-08-09T14:51:17.232 INFO:teuthology.orchestra.run.smithi043.stdout:scheduled-2023-08-09-13_57_00_UTC 2023-08-09T14:51:17.233 INFO:teuthology.orchestra.run.smithi043.stdout:scheduled-2023-08-09-13_58_00_UTC 2023-08-09T14:51:17.233 INFO:teuthology.orchestra.run.smithi043.stdout:scheduled-2023-08-09-13_59_00_UTC 2023-08-09T14:51:17.233 INFO:teuthology.orchestra.run.smithi043.stdout:scheduled-2023-08-09-14_00_00_UTC 2023-08-09T14:51:17.233 INFO:teuthology.orchestra.run.smithi043.stdout:scheduled-2023-08-09-14_01_00_UTC 2023-08-09T14:51:17.233 INFO:teuthology.orchestra.run.smithi043.stdout:scheduled-2023-08-09-14_02_00_UTC 2023-08-09T14:51:17.233 INFO:teuthology.orchestra.run.smithi043.stdout:scheduled-2023-08-09-14_03_00_UTC 2023-08-09T14:51:17.233 INFO:teuthology.orchestra.run.smithi043.stdout:scheduled-2023-08-09-14_04_00_UTC 2023-08-09T14:51:17.234 INFO:teuthology.orchestra.run.smithi043.stdout:scheduled-2023-08-09-14_05_00_UTC 2023-08-09T14:51:17.234 INFO:teuthology.orchestra.run.smithi043.stdout:scheduled-2023-08-09-14_06_00_UTC 2023-08-09T14:51:17.234 INFO:teuthology.orchestra.run.smithi043.stdout:scheduled-2023-08-09-14_07_00_UTC 2023-08-09T14:51:17.234 INFO:teuthology.orchestra.run.smithi043.stdout:scheduled-2023-08-09-14_08_00_UTC 2023-08-09T14:51:17.234 INFO:teuthology.orchestra.run.smithi043.stdout:scheduled-2023-08-09-14_09_00_UTC 2023-08-09T14:51:17.234 INFO:teuthology.orchestra.run.smithi043.stdout:scheduled-2023-08-09-14_10_00_UTC 2023-08-09T14:51:17.235 INFO:teuthology.orchestra.run.smithi043.stdout:scheduled-2023-08-09-14_11_00_UTC 2023-08-09T14:51:17.235 INFO:teuthology.orchestra.run.smithi043.stdout:scheduled-2023-08-09-14_12_00_UTC 2023-08-09T14:51:17.235 INFO:teuthology.orchestra.run.smithi043.stdout:scheduled-2023-08-09-14_13_01_UTC 2023-08-09T14:51:17.235 INFO:teuthology.orchestra.run.smithi043.stdout:scheduled-2023-08-09-14_14_00_UTC 2023-08-09T14:51:17.235 INFO:teuthology.orchestra.run.smithi043.stdout:scheduled-2023-08-09-14_15_00_UTC 2023-08-09T14:51:17.235 INFO:teuthology.orchestra.run.smithi043.stdout:scheduled-2023-08-09-14_16_00_UTC 2023-08-09T14:51:17.235 INFO:teuthology.orchestra.run.smithi043.stdout:scheduled-2023-08-09-14_17_00_UTC 2023-08-09T14:51:17.236 INFO:teuthology.orchestra.run.smithi043.stdout:scheduled-2023-08-09-14_18_00_UTC 2023-08-09T14:51:17.236 INFO:teuthology.orchestra.run.smithi043.stdout:scheduled-2023-08-09-14_19_00_UTC 2023-08-09T14:51:17.236 INFO:teuthology.orchestra.run.smithi043.stdout:scheduled-2023-08-09-14_20_00_UTC 2023-08-09T14:51:17.236 INFO:teuthology.orchestra.run.smithi043.stdout:scheduled-2023-08-09-14_21_00_UTC 2023-08-09T14:51:17.236 INFO:teuthology.orchestra.run.smithi043.stdout:scheduled-2023-08-09-14_22_00_UTC 2023-08-09T14:51:17.236 INFO:teuthology.orchestra.run.smithi043.stdout:scheduled-2023-08-09-14_23_00_UTC 2023-08-09T14:51:17.236 INFO:teuthology.orchestra.run.smithi043.stdout:scheduled-2023-08-09-14_24_00_UTC 2023-08-09T14:51:17.237 INFO:teuthology.orchestra.run.smithi043.stdout:scheduled-2023-08-09-14_25_00_UTC 2023-08-09T14:51:17.237 INFO:teuthology.orchestra.run.smithi043.stdout:scheduled-2023-08-09-14_26_00_UTC 2023-08-09T14:51:17.237 INFO:teuthology.orchestra.run.smithi043.stdout:scheduled-2023-08-09-14_27_00_UTC 2023-08-09T14:51:17.237 INFO:teuthology.orchestra.run.smithi043.stdout:scheduled-2023-08-09-14_28_00_UTC 2023-08-09T14:51:17.237 INFO:teuthology.orchestra.run.smithi043.stdout:scheduled-2023-08-09-14_29_00_UTC 2023-08-09T14:51:17.237 INFO:teuthology.orchestra.run.smithi043.stdout:scheduled-2023-08-09-14_30_00_UTC 2023-08-09T14:51:17.237 INFO:teuthology.orchestra.run.smithi043.stdout:scheduled-2023-08-09-14_31_00_UTC 2023-08-09T14:51:17.238 INFO:teuthology.orchestra.run.smithi043.stdout:scheduled-2023-08-09-14_32_00_UTC 2023-08-09T14:51:17.238 INFO:teuthology.orchestra.run.smithi043.stdout:scheduled-2023-08-09-14_33_00_UTC 2023-08-09T14:51:17.238 INFO:teuthology.orchestra.run.smithi043.stdout:scheduled-2023-08-09-14_34_00_UTC 2023-08-09T14:51:17.238 INFO:teuthology.orchestra.run.smithi043.stdout:scheduled-2023-08-09-14_35_00_UTC 2023-08-09T14:51:17.238 INFO:teuthology.orchestra.run.smithi043.stdout:scheduled-2023-08-09-14_36_00_UTC 2023-08-09T14:51:17.238 INFO:teuthology.orchestra.run.smithi043.stdout:scheduled-2023-08-09-14_37_00_UTC 2023-08-09T14:51:17.238 INFO:teuthology.orchestra.run.smithi043.stdout:scheduled-2023-08-09-14_38_00_UTC 2023-08-09T14:51:17.238 INFO:teuthology.orchestra.run.smithi043.stdout:scheduled-2023-08-09-14_39_00_UTC 2023-08-09T14:51:17.239 INFO:teuthology.orchestra.run.smithi043.stdout:scheduled-2023-08-09-14_40_00_UTC 2023-08-09T14:51:17.239 INFO:teuthology.orchestra.run.smithi043.stdout:scheduled-2023-08-09-14_41_00_UTC 2023-08-09T14:51:17.239 INFO:teuthology.orchestra.run.smithi043.stdout:scheduled-2023-08-09-14_42_00_UTC 2023-08-09T14:51:17.239 INFO:teuthology.orchestra.run.smithi043.stdout:scheduled-2023-08-09-14_43_00_UTC 2023-08-09T14:51:17.239 INFO:teuthology.orchestra.run.smithi043.stdout:scheduled-2023-08-09-14_44_00_UTC 2023-08-09T14:51:17.239 INFO:teuthology.orchestra.run.smithi043.stdout:scheduled-2023-08-09-14_45_00_UTC 2023-08-09T14:51:17.239 INFO:teuthology.orchestra.run.smithi043.stdout:scheduled-2023-08-09-14_46_00_UTC 2023-08-09T14:51:17.240 INFO:teuthology.orchestra.run.smithi043.stdout:scheduled-2023-08-09-14_47_00_UTC 2023-08-09T14:51:17.240 INFO:teuthology.orchestra.run.smithi043.stdout:scheduled-2023-08-09-14_48_00_UTC 2023-08-09T14:51:17.240 INFO:teuthology.orchestra.run.smithi043.stdout:scheduled-2023-08-09-14_49_00_UTC 2023-08-09T14:51:17.240 INFO:teuthology.orchestra.run.smithi043.stdout:scheduled-2023-08-09-14_50_00_UTC 2023-08-09T14:51:17.240 INFO:teuthology.orchestra.run.smithi043.stdout:scheduled-2023-08-09-14_51_00_UTC 2023-08-09T14:51:17.241 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/.snap/scheduled-2023-08-09-13_57_00_UTC 2023-08-09T14:51:17.242 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-09T14:51:17.242 DEBUG:teuthology.orchestra.run.smithi043:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/.snap/scheduled-2023-08-09-13_57_00_UTC) 2023-08-09T14:51:17.317 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/.snap/scheduled-2023-08-09-13_58_00_UTC 2023-08-09T14:51:17.318 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-09T14:51:17.319 DEBUG:teuthology.orchestra.run.smithi043:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/.snap/scheduled-2023-08-09-13_58_00_UTC) 2023-08-09T14:51:17.398 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/.snap/scheduled-2023-08-09-13_59_00_UTC 2023-08-09T14:51:17.398 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-09T14:51:17.399 DEBUG:teuthology.orchestra.run.smithi043:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/.snap/scheduled-2023-08-09-13_59_00_UTC) 2023-08-09T14:51:17.478 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/.snap/scheduled-2023-08-09-14_00_00_UTC 2023-08-09T14:51:17.479 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-09T14:51:17.479 DEBUG:teuthology.orchestra.run.smithi043:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/.snap/scheduled-2023-08-09-14_00_00_UTC) 2023-08-09T14:51:17.560 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/.snap/scheduled-2023-08-09-14_01_00_UTC 2023-08-09T14:51:17.561 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-09T14:51:17.561 DEBUG:teuthology.orchestra.run.smithi043:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/.snap/scheduled-2023-08-09-14_01_00_UTC) 2023-08-09T14:51:17.638 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/.snap/scheduled-2023-08-09-14_02_00_UTC 2023-08-09T14:51:17.639 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-09T14:51:17.639 DEBUG:teuthology.orchestra.run.smithi043:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/.snap/scheduled-2023-08-09-14_02_00_UTC) 2023-08-09T14:51:17.721 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/.snap/scheduled-2023-08-09-14_03_00_UTC 2023-08-09T14:51:17.721 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-09T14:51:17.722 DEBUG:teuthology.orchestra.run.smithi043:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/.snap/scheduled-2023-08-09-14_03_00_UTC) 2023-08-09T14:51:17.799 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/.snap/scheduled-2023-08-09-14_04_00_UTC 2023-08-09T14:51:17.799 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-09T14:51:17.800 DEBUG:teuthology.orchestra.run.smithi043:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/.snap/scheduled-2023-08-09-14_04_00_UTC) 2023-08-09T14:51:17.879 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/.snap/scheduled-2023-08-09-14_05_00_UTC 2023-08-09T14:51:17.879 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-09T14:51:17.880 DEBUG:teuthology.orchestra.run.smithi043:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/.snap/scheduled-2023-08-09-14_05_00_UTC) 2023-08-09T14:51:17.962 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/.snap/scheduled-2023-08-09-14_06_00_UTC 2023-08-09T14:51:17.962 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-09T14:51:17.963 DEBUG:teuthology.orchestra.run.smithi043:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/.snap/scheduled-2023-08-09-14_06_00_UTC) 2023-08-09T14:51:18.038 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/.snap/scheduled-2023-08-09-14_07_00_UTC 2023-08-09T14:51:18.038 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-09T14:51:18.039 DEBUG:teuthology.orchestra.run.smithi043:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/.snap/scheduled-2023-08-09-14_07_00_UTC) 2023-08-09T14:51:18.113 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/.snap/scheduled-2023-08-09-14_08_00_UTC 2023-08-09T14:51:18.113 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-09T14:51:18.113 DEBUG:teuthology.orchestra.run.smithi043:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/.snap/scheduled-2023-08-09-14_08_00_UTC) 2023-08-09T14:51:18.193 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/.snap/scheduled-2023-08-09-14_09_00_UTC 2023-08-09T14:51:18.193 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-09T14:51:18.194 DEBUG:teuthology.orchestra.run.smithi043:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/.snap/scheduled-2023-08-09-14_09_00_UTC) 2023-08-09T14:51:18.278 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/.snap/scheduled-2023-08-09-14_10_00_UTC 2023-08-09T14:51:18.278 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-09T14:51:18.279 DEBUG:teuthology.orchestra.run.smithi043:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/.snap/scheduled-2023-08-09-14_10_00_UTC) 2023-08-09T14:51:18.358 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/.snap/scheduled-2023-08-09-14_11_00_UTC 2023-08-09T14:51:18.359 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-09T14:51:18.359 DEBUG:teuthology.orchestra.run.smithi043:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/.snap/scheduled-2023-08-09-14_11_00_UTC) 2023-08-09T14:51:18.437 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/.snap/scheduled-2023-08-09-14_12_00_UTC 2023-08-09T14:51:18.437 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-09T14:51:18.438 DEBUG:teuthology.orchestra.run.smithi043:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/.snap/scheduled-2023-08-09-14_12_00_UTC) 2023-08-09T14:51:18.519 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/.snap/scheduled-2023-08-09-14_13_01_UTC 2023-08-09T14:51:18.520 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-09T14:51:18.520 DEBUG:teuthology.orchestra.run.smithi043:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/.snap/scheduled-2023-08-09-14_13_01_UTC) 2023-08-09T14:51:18.600 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/.snap/scheduled-2023-08-09-14_14_00_UTC 2023-08-09T14:51:18.600 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-09T14:51:18.601 DEBUG:teuthology.orchestra.run.smithi043:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/.snap/scheduled-2023-08-09-14_14_00_UTC) 2023-08-09T14:51:18.675 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/.snap/scheduled-2023-08-09-14_15_00_UTC 2023-08-09T14:51:18.675 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-09T14:51:18.675 DEBUG:teuthology.orchestra.run.smithi043:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/.snap/scheduled-2023-08-09-14_15_00_UTC) 2023-08-09T14:51:18.758 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/.snap/scheduled-2023-08-09-14_16_00_UTC 2023-08-09T14:51:18.758 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-09T14:51:18.759 DEBUG:teuthology.orchestra.run.smithi043:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/.snap/scheduled-2023-08-09-14_16_00_UTC) 2023-08-09T14:51:18.835 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/.snap/scheduled-2023-08-09-14_17_00_UTC 2023-08-09T14:51:18.836 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-09T14:51:18.837 DEBUG:teuthology.orchestra.run.smithi043:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/.snap/scheduled-2023-08-09-14_17_00_UTC) 2023-08-09T14:51:18.918 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/.snap/scheduled-2023-08-09-14_18_00_UTC 2023-08-09T14:51:18.918 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-09T14:51:18.919 DEBUG:teuthology.orchestra.run.smithi043:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/.snap/scheduled-2023-08-09-14_18_00_UTC) 2023-08-09T14:51:19.030 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/.snap/scheduled-2023-08-09-14_19_00_UTC 2023-08-09T14:51:19.031 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-09T14:51:19.031 DEBUG:teuthology.orchestra.run.smithi043:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/.snap/scheduled-2023-08-09-14_19_00_UTC) 2023-08-09T14:51:19.086 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/.snap/scheduled-2023-08-09-14_20_00_UTC 2023-08-09T14:51:19.087 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-09T14:51:19.087 DEBUG:teuthology.orchestra.run.smithi043:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/.snap/scheduled-2023-08-09-14_20_00_UTC) 2023-08-09T14:51:19.144 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/.snap/scheduled-2023-08-09-14_21_00_UTC 2023-08-09T14:51:19.145 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-09T14:51:19.145 DEBUG:teuthology.orchestra.run.smithi043:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/.snap/scheduled-2023-08-09-14_21_00_UTC) 2023-08-09T14:51:19.208 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/.snap/scheduled-2023-08-09-14_22_00_UTC 2023-08-09T14:51:19.209 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-09T14:51:19.209 DEBUG:teuthology.orchestra.run.smithi043:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/.snap/scheduled-2023-08-09-14_22_00_UTC) 2023-08-09T14:51:19.295 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/.snap/scheduled-2023-08-09-14_23_00_UTC 2023-08-09T14:51:19.296 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-09T14:51:19.296 DEBUG:teuthology.orchestra.run.smithi043:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/.snap/scheduled-2023-08-09-14_23_00_UTC) 2023-08-09T14:51:19.365 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/.snap/scheduled-2023-08-09-14_24_00_UTC 2023-08-09T14:51:19.365 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-09T14:51:19.366 DEBUG:teuthology.orchestra.run.smithi043:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/.snap/scheduled-2023-08-09-14_24_00_UTC) 2023-08-09T14:51:19.448 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/.snap/scheduled-2023-08-09-14_25_00_UTC 2023-08-09T14:51:19.448 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-09T14:51:19.449 DEBUG:teuthology.orchestra.run.smithi043:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/.snap/scheduled-2023-08-09-14_25_00_UTC) 2023-08-09T14:51:19.526 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/.snap/scheduled-2023-08-09-14_26_00_UTC 2023-08-09T14:51:19.526 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-09T14:51:19.527 DEBUG:teuthology.orchestra.run.smithi043:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/.snap/scheduled-2023-08-09-14_26_00_UTC) 2023-08-09T14:51:19.602 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/.snap/scheduled-2023-08-09-14_27_00_UTC 2023-08-09T14:51:19.603 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-09T14:51:19.603 DEBUG:teuthology.orchestra.run.smithi043:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/.snap/scheduled-2023-08-09-14_27_00_UTC) 2023-08-09T14:51:19.691 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/.snap/scheduled-2023-08-09-14_28_00_UTC 2023-08-09T14:51:19.692 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-09T14:51:19.692 DEBUG:teuthology.orchestra.run.smithi043:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/.snap/scheduled-2023-08-09-14_28_00_UTC) 2023-08-09T14:51:19.762 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/.snap/scheduled-2023-08-09-14_29_00_UTC 2023-08-09T14:51:19.763 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-09T14:51:19.763 DEBUG:teuthology.orchestra.run.smithi043:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/.snap/scheduled-2023-08-09-14_29_00_UTC) 2023-08-09T14:51:19.827 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/.snap/scheduled-2023-08-09-14_30_00_UTC 2023-08-09T14:51:19.827 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-09T14:51:19.827 DEBUG:teuthology.orchestra.run.smithi043:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/.snap/scheduled-2023-08-09-14_30_00_UTC) 2023-08-09T14:51:19.899 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/.snap/scheduled-2023-08-09-14_31_00_UTC 2023-08-09T14:51:19.899 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-09T14:51:19.900 DEBUG:teuthology.orchestra.run.smithi043:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/.snap/scheduled-2023-08-09-14_31_00_UTC) 2023-08-09T14:51:20.031 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/.snap/scheduled-2023-08-09-14_32_00_UTC 2023-08-09T14:51:20.031 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-09T14:51:20.031 DEBUG:teuthology.orchestra.run.smithi043:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/.snap/scheduled-2023-08-09-14_32_00_UTC) 2023-08-09T14:51:20.109 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/.snap/scheduled-2023-08-09-14_33_00_UTC 2023-08-09T14:51:20.109 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-09T14:51:20.110 DEBUG:teuthology.orchestra.run.smithi043:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/.snap/scheduled-2023-08-09-14_33_00_UTC) 2023-08-09T14:51:20.169 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/.snap/scheduled-2023-08-09-14_34_00_UTC 2023-08-09T14:51:20.169 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-09T14:51:20.170 DEBUG:teuthology.orchestra.run.smithi043:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/.snap/scheduled-2023-08-09-14_34_00_UTC) 2023-08-09T14:51:20.232 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/.snap/scheduled-2023-08-09-14_35_00_UTC 2023-08-09T14:51:20.233 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-09T14:51:20.233 DEBUG:teuthology.orchestra.run.smithi043:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/.snap/scheduled-2023-08-09-14_35_00_UTC) 2023-08-09T14:51:20.316 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/.snap/scheduled-2023-08-09-14_36_00_UTC 2023-08-09T14:51:20.316 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-09T14:51:20.317 DEBUG:teuthology.orchestra.run.smithi043:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/.snap/scheduled-2023-08-09-14_36_00_UTC) 2023-08-09T14:51:20.390 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/.snap/scheduled-2023-08-09-14_37_00_UTC 2023-08-09T14:51:20.391 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-09T14:51:20.391 DEBUG:teuthology.orchestra.run.smithi043:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/.snap/scheduled-2023-08-09-14_37_00_UTC) 2023-08-09T14:51:20.452 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/.snap/scheduled-2023-08-09-14_38_00_UTC 2023-08-09T14:51:20.452 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-09T14:51:20.453 DEBUG:teuthology.orchestra.run.smithi043:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/.snap/scheduled-2023-08-09-14_38_00_UTC) 2023-08-09T14:51:20.522 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/.snap/scheduled-2023-08-09-14_39_00_UTC 2023-08-09T14:51:20.523 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-09T14:51:20.523 DEBUG:teuthology.orchestra.run.smithi043:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/.snap/scheduled-2023-08-09-14_39_00_UTC) 2023-08-09T14:51:20.597 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/.snap/scheduled-2023-08-09-14_40_00_UTC 2023-08-09T14:51:20.597 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-09T14:51:20.598 DEBUG:teuthology.orchestra.run.smithi043:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/.snap/scheduled-2023-08-09-14_40_00_UTC) 2023-08-09T14:51:20.671 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/.snap/scheduled-2023-08-09-14_41_00_UTC 2023-08-09T14:51:20.672 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-09T14:51:20.672 DEBUG:teuthology.orchestra.run.smithi043:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/.snap/scheduled-2023-08-09-14_41_00_UTC) 2023-08-09T14:51:20.743 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/.snap/scheduled-2023-08-09-14_42_00_UTC 2023-08-09T14:51:20.744 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-09T14:51:20.744 DEBUG:teuthology.orchestra.run.smithi043:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/.snap/scheduled-2023-08-09-14_42_00_UTC) 2023-08-09T14:51:20.820 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/.snap/scheduled-2023-08-09-14_43_00_UTC 2023-08-09T14:51:20.820 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-09T14:51:20.821 DEBUG:teuthology.orchestra.run.smithi043:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/.snap/scheduled-2023-08-09-14_43_00_UTC) 2023-08-09T14:51:20.896 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/.snap/scheduled-2023-08-09-14_44_00_UTC 2023-08-09T14:51:20.897 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-09T14:51:20.897 DEBUG:teuthology.orchestra.run.smithi043:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/.snap/scheduled-2023-08-09-14_44_00_UTC) 2023-08-09T14:51:20.990 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/.snap/scheduled-2023-08-09-14_45_00_UTC 2023-08-09T14:51:20.990 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-09T14:51:20.991 DEBUG:teuthology.orchestra.run.smithi043:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/.snap/scheduled-2023-08-09-14_45_00_UTC) 2023-08-09T14:51:21.067 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/.snap/scheduled-2023-08-09-14_46_00_UTC 2023-08-09T14:51:21.068 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-09T14:51:21.068 DEBUG:teuthology.orchestra.run.smithi043:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/.snap/scheduled-2023-08-09-14_46_00_UTC) 2023-08-09T14:51:21.118 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/.snap/scheduled-2023-08-09-14_47_00_UTC 2023-08-09T14:51:21.119 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-09T14:51:21.119 DEBUG:teuthology.orchestra.run.smithi043:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/.snap/scheduled-2023-08-09-14_47_00_UTC) 2023-08-09T14:51:21.172 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/.snap/scheduled-2023-08-09-14_48_00_UTC 2023-08-09T14:51:21.172 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-09T14:51:21.173 DEBUG:teuthology.orchestra.run.smithi043:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/.snap/scheduled-2023-08-09-14_48_00_UTC) 2023-08-09T14:51:21.231 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/.snap/scheduled-2023-08-09-14_49_00_UTC 2023-08-09T14:51:21.231 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-09T14:51:21.232 DEBUG:teuthology.orchestra.run.smithi043:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/.snap/scheduled-2023-08-09-14_49_00_UTC) 2023-08-09T14:51:21.295 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/.snap/scheduled-2023-08-09-14_50_00_UTC 2023-08-09T14:51:21.296 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-09T14:51:21.297 DEBUG:teuthology.orchestra.run.smithi043:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/.snap/scheduled-2023-08-09-14_50_00_UTC) 2023-08-09T14:51:21.369 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/.snap/scheduled-2023-08-09-14_51_00_UTC 2023-08-09T14:51:21.369 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-09T14:51:21.370 DEBUG:teuthology.orchestra.run.smithi043:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/.snap/scheduled-2023-08-09-14_51_00_UTC) 2023-08-09T14:51:21.425 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-09T14:51:21.425 DEBUG:teuthology.orchestra.run.smithi043:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1) 2023-08-09T14:51:21.480 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph mgr module disable snap_schedule 2023-08-09T14:51:21.726 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:21.724+0000 7f49b9283640 1 Processor -- start 2023-08-09T14:51:21.726 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:21.724+0000 7f49b9283640 1 -- start start 2023-08-09T14:51:21.727 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:21.726+0000 7f49b9283640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f49b415c680 0x7f49b40ce760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T14:51:21.727 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:21.726+0000 7f49b9283640 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7f49b40b3fa0 0x7f49b40b43d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T14:51:21.728 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:21.726+0000 7f49b9283640 1 -- --> v1:172.21.15.43:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f49b40b4910 con 0x7f49b40d0a70 2023-08-09T14:51:21.728 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:21.726+0000 7f49b9283640 1 -- --> v2:172.21.15.203:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f49b40cdf00 con 0x7f49b40b3fa0 2023-08-09T14:51:21.728 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:21.726+0000 7f49b9283640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f49b40d0f70 con 0x7f49b415c680 2023-08-09T14:51:21.729 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:21.726+0000 7f49b27fc640 1 --1- >> v1:172.21.15.43:6789/0 conn(0x7f49b40d0a70 0x7f49b40cd7f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.43:6789/0 says I am v1:172.21.15.43:50798/0 (socket says 172.21.15.43:50798) 2023-08-09T14:51:21.729 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:21.726+0000 7f49b27fc640 1 -- 172.21.15.43:0/3564774613 learned_addr learned my addr 172.21.15.43:0/3564774613 (peer_addr_for_me v1:172.21.15.43:0/0) 2023-08-09T14:51:21.729 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:21.727+0000 7f49b37fe640 1 --2- 172.21.15.43:0/3564774613 >> v2:172.21.15.203:3300/0 conn(0x7f49b40b3fa0 0x7f49b40b43d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T14:51:21.729 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:21.727+0000 7f49b2ffd640 1 --2- 172.21.15.43:0/3564774613 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f49b415c680 0x7f49b40ce760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T14:51:21.729 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:21.727+0000 7f49b1ffb640 1 -- 172.21.15.43:0/3564774613 <== mon.0 v1:172.21.15.43:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3040365702 0 0) 0x7f49b40b4910 con 0x7f49b40d0a70 2023-08-09T14:51:21.730 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:21.727+0000 7f49b1ffb640 1 -- 172.21.15.43:0/3564774613 --> v1:172.21.15.43:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f499c003520 con 0x7f49b40d0a70 2023-08-09T14:51:21.730 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:21.727+0000 7f49b2ffd640 1 -- 172.21.15.43:0/3564774613 >> v2:172.21.15.203:3300/0 conn(0x7f49b40b3fa0 msgr2=0x7f49b40b43d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-09T14:51:21.730 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:21.727+0000 7f49b2ffd640 1 --2- 172.21.15.43:0/3564774613 >> v2:172.21.15.203:3300/0 conn(0x7f49b40b3fa0 0x7f49b40b43d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T14:51:21.730 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:21.727+0000 7f49b2ffd640 1 -- 172.21.15.43:0/3564774613 >> v1:172.21.15.43:6789/0 conn(0x7f49b40d0a70 legacy=0x7f49b40cd7f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T14:51:21.730 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:21.727+0000 7f49b2ffd640 1 -- 172.21.15.43:0/3564774613 --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f49b40ccd10 con 0x7f49b415c680 2023-08-09T14:51:21.731 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:21.729+0000 7f49b2ffd640 1 --2- 172.21.15.43:0/3564774613 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f49b415c680 0x7f49b40ce760 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7f49a40097a0 tx=0x7f49a402b8b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a00842fbfcacc08a server_cookie=0 in_seq=0 out_seq=0 2023-08-09T14:51:21.731 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:21.730+0000 7f49b1ffb640 1 -- 172.21.15.43:0/3564774613 <== mon.2 v2:172.21.15.203:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f49a402d040 con 0x7f49b415c680 2023-08-09T14:51:21.731 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:21.730+0000 7f49b1ffb640 1 -- 172.21.15.43:0/3564774613 <== mon.2 v2:172.21.15.203:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f49a4031810 con 0x7f49b415c680 2023-08-09T14:51:21.732 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:21.730+0000 7f49b1ffb640 1 -- 172.21.15.43:0/3564774613 <== mon.2 v2:172.21.15.203:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f49a4031ba0 con 0x7f49b415c680 2023-08-09T14:51:21.732 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:21.731+0000 7f49b9283640 1 -- 172.21.15.43:0/3564774613 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f49b415c680 msgr2=0x7f49b40ce760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T14:51:21.733 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:21.731+0000 7f49b9283640 1 --2- 172.21.15.43:0/3564774613 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f49b415c680 0x7f49b40ce760 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7f49a40097a0 tx=0x7f49a402b8b0 comp rx=0 tx=0).stop 2023-08-09T14:51:21.733 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:21.731+0000 7f49b9283640 1 -- 172.21.15.43:0/3564774613 shutdown_connections 2023-08-09T14:51:21.733 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:21.731+0000 7f49b9283640 1 --2- 172.21.15.43:0/3564774613 >> v2:172.21.15.203:3300/0 conn(0x7f49b40b3fa0 0x7f49b40b43d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T14:51:21.733 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:21.731+0000 7f49b9283640 1 --2- 172.21.15.43:0/3564774613 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f49b415c680 0x7f49b40ce760 unknown :-1 s=CLOSED pgs=238 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T14:51:21.733 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:21.731+0000 7f49b9283640 1 -- 172.21.15.43:0/3564774613 >> 172.21.15.43:0/3564774613 conn(0x7f49b4071520 msgr2=0x7f49b40d2520 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-09T14:51:21.733 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:21.731+0000 7f49b9283640 1 -- 172.21.15.43:0/3564774613 shutdown_connections 2023-08-09T14:51:21.733 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:21.731+0000 7f49b9283640 1 -- 172.21.15.43:0/3564774613 wait complete. 2023-08-09T14:51:21.734 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:21.733+0000 7f49b9283640 1 Processor -- start 2023-08-09T14:51:21.735 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:21.733+0000 7f49b9283640 1 -- start start 2023-08-09T14:51:21.737 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:21.735+0000 7f49b9283640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f49b40b3fa0 0x7f49b40c7f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T14:51:21.737 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:21.735+0000 7f49b9283640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f49b40d0a70 0x7f49b40b6c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T14:51:21.737 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:21.735+0000 7f49b9283640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f49b415c680 0x7f49b4154180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T14:51:21.737 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:21.735+0000 7f49b9283640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f49b41446e0 con 0x7f49b40d0a70 2023-08-09T14:51:21.737 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:21.735+0000 7f49b9283640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f49b415e3f0 con 0x7f49b415c680 2023-08-09T14:51:21.737 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:21.735+0000 7f49b9283640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f49b415d7e0 con 0x7f49b40b3fa0 2023-08-09T14:51:21.738 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:21.735+0000 7f49b27fc640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f49b40d0a70 0x7f49b40b6c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T14:51:21.738 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:21.735+0000 7f49b27fc640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f49b40d0a70 0x7f49b40b6c70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:39054/0 (socket says 172.21.15.43:39054) 2023-08-09T14:51:21.738 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:21.735+0000 7f49b27fc640 1 -- 172.21.15.43:0/3379143035 learned_addr learned my addr 172.21.15.43:0/3379143035 (peer_addr_for_me v2:172.21.15.43:0/0) 2023-08-09T14:51:21.738 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:21.735+0000 7f49b2ffd640 1 --2- 172.21.15.43:0/3379143035 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f49b40b3fa0 0x7f49b40c7f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T14:51:21.738 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:21.735+0000 7f49b37fe640 1 --2- 172.21.15.43:0/3379143035 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f49b415c680 0x7f49b4154180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T14:51:21.739 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:21.736+0000 7f49b27fc640 1 -- 172.21.15.43:0/3379143035 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f49b40b3fa0 msgr2=0x7f49b40c7f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T14:51:21.739 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:21.736+0000 7f49b27fc640 1 --2- 172.21.15.43:0/3379143035 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f49b40b3fa0 0x7f49b40c7f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T14:51:21.740 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:21.736+0000 7f49b27fc640 1 -- 172.21.15.43:0/3379143035 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f49b415c680 msgr2=0x7f49b4154180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T14:51:21.740 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:21.736+0000 7f49b27fc640 1 --2- 172.21.15.43:0/3379143035 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f49b415c680 0x7f49b4154180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T14:51:21.740 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:21.736+0000 7f49b27fc640 1 -- 172.21.15.43:0/3379143035 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f49a4009400 con 0x7f49b40d0a70 2023-08-09T14:51:21.740 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:21.737+0000 7f49b27fc640 1 --2- 172.21.15.43:0/3379143035 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f49b40d0a70 0x7f49b40b6c70 secure :-1 s=READY pgs=1744 cs=0 l=1 rev1=1 crypto rx=0x7f49a800c500 tx=0x7f49a800c9c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-09T14:51:21.740 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:21.737+0000 7f499b7fe640 1 -- 172.21.15.43:0/3379143035 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f49a8005ce0 con 0x7f49b40d0a70 2023-08-09T14:51:21.740 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:21.737+0000 7f499b7fe640 1 -- 172.21.15.43:0/3379143035 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f49a8005e40 con 0x7f49b40d0a70 2023-08-09T14:51:21.740 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:21.737+0000 7f49b9283640 1 -- 172.21.15.43:0/3379143035 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f49b415dac0 con 0x7f49b40d0a70 2023-08-09T14:51:21.741 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:21.737+0000 7f499b7fe640 1 -- 172.21.15.43:0/3379143035 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f49a8007330 con 0x7f49b40d0a70 2023-08-09T14:51:21.741 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:21.737+0000 7f49b9283640 1 -- 172.21.15.43:0/3379143035 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f49b415e080 con 0x7f49b40d0a70 2023-08-09T14:51:21.741 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:21.739+0000 7f49b9283640 1 -- 172.21.15.43:0/3379143035 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f49b40739f0 con 0x7f49b40d0a70 2023-08-09T14:51:21.742 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:21.740+0000 7f499b7fe640 1 -- 172.21.15.43:0/3379143035 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 74) v1 ==== 88472+0+0 (secure 0 0 0) 0x7f49a8007550 con 0x7f49b40d0a70 2023-08-09T14:51:21.743 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:21.741+0000 7f499b7fe640 1 --2- 172.21.15.43:0/3379143035 >> [v2:172.21.15.203:6832/372694067,v1:172.21.15.203:6833/372694067] conn(0x7f49a006a1c0 0x7f49a006c680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T14:51:21.743 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:21.742+0000 7f499b7fe640 1 -- 172.21.15.43:0/3379143035 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(155..155 src has 1..155) v4 ==== 6916+0+0 (secure 0 0 0) 0x7f49a80880c0 con 0x7f49b40d0a70 2023-08-09T14:51:21.743 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:21.742+0000 7f49b2ffd640 1 --2- 172.21.15.43:0/3379143035 >> [v2:172.21.15.203:6832/372694067,v1:172.21.15.203:6833/372694067] conn(0x7f49a006a1c0 0x7f49a006c680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T14:51:21.745 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:21.743+0000 7f49b2ffd640 1 --2- 172.21.15.43:0/3379143035 >> [v2:172.21.15.203:6832/372694067,v1:172.21.15.203:6833/372694067] conn(0x7f49a006a1c0 0x7f49a006c680 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f49a4036000 tx=0x7f49a403c040 comp rx=0 tx=0).ready entity=mgr.6869 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-09T14:51:21.750 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:21.749+0000 7f499b7fe640 1 -- 172.21.15.43:0/3379143035 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178066 (secure 0 0 0) 0x7f49a80534a0 con 0x7f49b40d0a70 2023-08-09T14:51:22.078 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:22.076+0000 7f49b9283640 1 -- 172.21.15.43:0/3379143035 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "mgr module disable", "module": "snap_schedule"} v 0) v1 -- 0x7f49b415e530 con 0x7f49b40d0a70 2023-08-09T14:51:23.022 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:23.020+0000 7f499b7fe640 1 -- 172.21.15.43:0/3379143035 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mgrmap(e 75) v1 ==== 88455+0+0 (secure 0 0 0) 0x7f49a8053870 con 0x7f49b40d0a70 2023-08-09T14:51:23.027 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:23.026+0000 7f499b7fe640 1 -- 172.21.15.43:0/3379143035 <== mon.0 v2:172.21.15.43:3300/0 8 ==== mon_command_ack([{"prefix": "mgr module disable", "module": "snap_schedule"}]=0 v75) v1 ==== 93+0+0 (secure 0 0 0) 0x7f49a80534a0 con 0x7f49b40d0a70 2023-08-09T14:51:23.038 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:23.037+0000 7f49b9283640 1 -- 172.21.15.43:0/3379143035 >> [v2:172.21.15.203:6832/372694067,v1:172.21.15.203:6833/372694067] conn(0x7f49a006a1c0 msgr2=0x7f49a006c680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T14:51:23.039 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:23.037+0000 7f49b9283640 1 --2- 172.21.15.43:0/3379143035 >> [v2:172.21.15.203:6832/372694067,v1:172.21.15.203:6833/372694067] conn(0x7f49a006a1c0 0x7f49a006c680 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f49a4036000 tx=0x7f49a403c040 comp rx=0 tx=0).stop 2023-08-09T14:51:23.039 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:23.038+0000 7f49b9283640 1 -- 172.21.15.43:0/3379143035 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f49b40d0a70 msgr2=0x7f49b40b6c70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T14:51:23.040 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:23.038+0000 7f49b9283640 1 --2- 172.21.15.43:0/3379143035 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f49b40d0a70 0x7f49b40b6c70 secure :-1 s=READY pgs=1744 cs=0 l=1 rev1=1 crypto rx=0x7f49a800c500 tx=0x7f49a800c9c0 comp rx=0 tx=0).stop 2023-08-09T14:51:23.041 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:23.039+0000 7f49b9283640 1 -- 172.21.15.43:0/3379143035 shutdown_connections 2023-08-09T14:51:23.041 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:23.039+0000 7f49b9283640 1 --2- 172.21.15.43:0/3379143035 >> [v2:172.21.15.203:6832/372694067,v1:172.21.15.203:6833/372694067] conn(0x7f49a006a1c0 0x7f49a006c680 unknown :-1 s=CLOSED pgs=70 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T14:51:23.041 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:23.039+0000 7f49b9283640 1 --2- 172.21.15.43:0/3379143035 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f49b415c680 0x7f49b4154180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T14:51:23.041 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:23.039+0000 7f49b9283640 1 --2- 172.21.15.43:0/3379143035 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f49b40d0a70 0x7f49b40b6c70 unknown :-1 s=CLOSED pgs=1744 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T14:51:23.041 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:23.039+0000 7f49b9283640 1 --2- 172.21.15.43:0/3379143035 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f49b40b3fa0 0x7f49b40c7f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T14:51:23.042 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:23.039+0000 7f49b9283640 1 -- 172.21.15.43:0/3379143035 >> 172.21.15.43:0/3379143035 conn(0x7f49b4071520 msgr2=0x7f49b40ce2b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-09T14:51:23.042 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:23.039+0000 7f49b9283640 1 -- 172.21.15.43:0/3379143035 shutdown_connections 2023-08-09T14:51:23.042 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:23.040+0000 7f49b9283640 1 -- 172.21.15.43:0/3379143035 wait complete. 2023-08-09T14:51:23.069 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2023-08-09T14:51:23.070 DEBUG:teuthology.orchestra.run.smithi043:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2023-08-09T14:51:23.167 DEBUG:teuthology.orchestra.run.smithi203:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2023-08-09T14:51:23.280 INFO:tasks.ceph.mgr.y.smithi043.stderr:2023-08-09T14:51:23.278+0000 7f6a9a356040 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2023-08-09T14:51:23.281 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2023-08-09T14:51:23.282 DEBUG:teuthology.orchestra.run.smithi043:> set -ex 2023-08-09T14:51:23.283 DEBUG:teuthology.orchestra.run.smithi043:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-09T14:51:23.284 INFO:tasks.ceph.mgr.x.smithi203.stderr:2023-08-09T14:51:23.283+0000 7fe0afe0f040 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2023-08-09T14:51:23.325 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-08-09T14:51:23.579 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:23.577+0000 7fe6d1fe1640 1 Processor -- start 2023-08-09T14:51:23.579 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:23.578+0000 7fe6d1fe1640 1 -- start start 2023-08-09T14:51:23.581 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:23.579+0000 7fe6d1fe1640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7fe6cc0b2b70 0x7fe6cc0d2ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T14:51:23.582 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:23.579+0000 7fe6d1fe1640 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7fe6cc0b3050 0x7fe6cc0b3f60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T14:51:23.582 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:23.579+0000 7fe6d1fe1640 1 -- --> v1:172.21.15.203:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe6cc0ce090 con 0x7fe6cc0b2790 2023-08-09T14:51:23.582 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:23.579+0000 7fe6d1fe1640 1 -- --> v2:172.21.15.43:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe6cc0b3950 con 0x7fe6cc0b2b70 2023-08-09T14:51:23.582 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:23.579+0000 7fe6d1fe1640 1 -- --> v2:172.21.15.203:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe6cc0cd7f0 con 0x7fe6cc0b3050 2023-08-09T14:51:23.582 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:23.579+0000 7fe6caffd640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7fe6cc0b2b70 0x7fe6cc0d2ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T14:51:23.583 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:23.579+0000 7fe6caffd640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7fe6cc0b2b70 0x7fe6cc0d2ed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:39072/0 (socket says 172.21.15.43:39072) 2023-08-09T14:51:23.583 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:23.579+0000 7fe6caffd640 1 -- 172.21.15.43:0/3352377022 learned_addr learned my addr 172.21.15.43:0/3352377022 (peer_addr_for_me v2:172.21.15.43:0/0) 2023-08-09T14:51:23.583 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:23.580+0000 7fe6cbfff640 1 --2- 172.21.15.43:0/3352377022 >> v2:172.21.15.203:3300/0 conn(0x7fe6cc0b3050 0x7fe6cc0b3f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T14:51:23.583 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:23.580+0000 7fe6caffd640 1 -- 172.21.15.43:0/3352377022 >> v2:172.21.15.203:3300/0 conn(0x7fe6cc0b3050 msgr2=0x7fe6cc0b3f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-09T14:51:23.583 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:23.580+0000 7fe6caffd640 1 --2- 172.21.15.43:0/3352377022 >> v2:172.21.15.203:3300/0 conn(0x7fe6cc0b3050 0x7fe6cc0b3f60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T14:51:23.583 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:23.580+0000 7fe6caffd640 1 -- 172.21.15.43:0/3352377022 >> v1:172.21.15.203:6789/0 conn(0x7fe6cc0b2790 legacy=0x7fe6cc0b5330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-09T14:51:23.584 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:23.580+0000 7fe6caffd640 1 -- 172.21.15.43:0/3352377022 --> v2:172.21.15.43:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe6cc0cdad0 con 0x7fe6cc0b2b70 2023-08-09T14:51:23.584 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:23.581+0000 7fe6caffd640 1 --2- 172.21.15.43:0/3352377022 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fe6cc0b2b70 0x7fe6cc0d2ed0 secure :-1 s=READY pgs=1747 cs=0 l=1 rev1=1 crypto rx=0x7fe6cc08a270 tx=0x7fe6c002b910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=624ec305b891dd2c server_cookie=0 in_seq=0 out_seq=0 2023-08-09T14:51:23.585 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:23.581+0000 7fe6ca7fc640 1 -- 172.21.15.43:0/3352377022 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe6c0031870 con 0x7fe6cc0b2b70 2023-08-09T14:51:23.585 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:23.581+0000 7fe6ca7fc640 1 -- 172.21.15.43:0/3352377022 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe6c00319d0 con 0x7fe6cc0b2b70 2023-08-09T14:51:23.585 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:23.581+0000 7fe6ca7fc640 1 -- 172.21.15.43:0/3352377022 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe6c0031d60 con 0x7fe6cc0b2b70 2023-08-09T14:51:23.585 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:23.582+0000 7fe6d1fe1640 1 -- 172.21.15.43:0/3352377022 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fe6cc0b2b70 msgr2=0x7fe6cc0d2ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T14:51:23.585 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:23.582+0000 7fe6d1fe1640 1 --2- 172.21.15.43:0/3352377022 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fe6cc0b2b70 0x7fe6cc0d2ed0 secure :-1 s=READY pgs=1747 cs=0 l=1 rev1=1 crypto rx=0x7fe6cc08a270 tx=0x7fe6c002b910 comp rx=0 tx=0).stop 2023-08-09T14:51:23.585 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:23.582+0000 7fe6d1fe1640 1 -- 172.21.15.43:0/3352377022 shutdown_connections 2023-08-09T14:51:23.586 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:23.582+0000 7fe6d1fe1640 1 --2- 172.21.15.43:0/3352377022 >> v2:172.21.15.203:3300/0 conn(0x7fe6cc0b3050 0x7fe6cc0b3f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T14:51:23.586 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:23.582+0000 7fe6d1fe1640 1 --2- 172.21.15.43:0/3352377022 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fe6cc0b2b70 0x7fe6cc0d2ed0 unknown :-1 s=CLOSED pgs=1747 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T14:51:23.586 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:23.582+0000 7fe6d1fe1640 1 -- 172.21.15.43:0/3352377022 >> 172.21.15.43:0/3352377022 conn(0x7fe6cc09b7b0 msgr2=0x7fe6cc0d24e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-09T14:51:23.586 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:23.582+0000 7fe6d1fe1640 1 -- 172.21.15.43:0/3352377022 shutdown_connections 2023-08-09T14:51:23.586 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:23.582+0000 7fe6d1fe1640 1 -- 172.21.15.43:0/3352377022 wait complete. 2023-08-09T14:51:23.588 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:23.584+0000 7fe6d1fe1640 1 Processor -- start 2023-08-09T14:51:23.588 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:23.584+0000 7fe6d1fe1640 1 -- start start 2023-08-09T14:51:23.588 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:23.585+0000 7fe6d1fe1640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fe6cc0b2790 0x7fe6cc15b850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T14:51:23.588 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:23.585+0000 7fe6d1fe1640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fe6cc0b2b70 0x7fe6cc0b1960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T14:51:23.589 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:23.585+0000 7fe6d1fe1640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fe6cc0b3050 0x7fe6cc1596c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T14:51:23.589 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:23.585+0000 7fe6d1fe1640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe6cc156cb0 con 0x7fe6cc0b3050 2023-08-09T14:51:23.589 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:23.585+0000 7fe6d1fe1640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe6cc156450 con 0x7fe6cc0b2790 2023-08-09T14:51:23.589 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:23.585+0000 7fe6d1fe1640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe6cc156730 con 0x7fe6cc0b2b70 2023-08-09T14:51:23.589 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:23.585+0000 7fe6cbfff640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fe6cc0b3050 0x7fe6cc1596c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T14:51:23.589 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:23.585+0000 7fe6cbfff640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fe6cc0b3050 0x7fe6cc1596c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:39082/0 (socket says 172.21.15.43:39082) 2023-08-09T14:51:23.589 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:23.585+0000 7fe6cbfff640 1 -- 172.21.15.43:0/3732006264 learned_addr learned my addr 172.21.15.43:0/3732006264 (peer_addr_for_me v2:172.21.15.43:0/0) 2023-08-09T14:51:23.590 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:23.586+0000 7fe6cb7fe640 1 --2- 172.21.15.43:0/3732006264 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fe6cc0b2790 0x7fe6cc15b850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T14:51:23.590 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:23.586+0000 7fe6caffd640 1 --2- 172.21.15.43:0/3732006264 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fe6cc0b2b70 0x7fe6cc0b1960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T14:51:23.590 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:23.586+0000 7fe6cbfff640 1 -- 172.21.15.43:0/3732006264 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fe6cc0b2b70 msgr2=0x7fe6cc0b1960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T14:51:23.590 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:23.586+0000 7fe6cbfff640 1 --2- 172.21.15.43:0/3732006264 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fe6cc0b2b70 0x7fe6cc0b1960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T14:51:23.591 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:23.586+0000 7fe6cbfff640 1 -- 172.21.15.43:0/3732006264 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fe6cc0b2790 msgr2=0x7fe6cc15b850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T14:51:23.591 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:23.586+0000 7fe6cbfff640 1 --2- 172.21.15.43:0/3732006264 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fe6cc0b2790 0x7fe6cc15b850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T14:51:23.591 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:23.586+0000 7fe6cbfff640 1 -- 172.21.15.43:0/3732006264 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe6c00094d0 con 0x7fe6cc0b3050 2023-08-09T14:51:23.591 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:23.586+0000 7fe6cb7fe640 1 --2- 172.21.15.43:0/3732006264 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fe6cc0b2790 0x7fe6cc15b850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-08-09T14:51:23.591 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:23.586+0000 7fe6cbfff640 1 --2- 172.21.15.43:0/3732006264 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fe6cc0b3050 0x7fe6cc1596c0 secure :-1 s=READY pgs=1748 cs=0 l=1 rev1=1 crypto rx=0x7fe6c400cc30 tx=0x7fe6c400a5c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-09T14:51:23.591 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:23.586+0000 7fe6b3fff640 1 -- 172.21.15.43:0/3732006264 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe6c400ce40 con 0x7fe6cc0b3050 2023-08-09T14:51:23.591 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:23.586+0000 7fe6b3fff640 1 -- 172.21.15.43:0/3732006264 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe6c4003990 con 0x7fe6cc0b3050 2023-08-09T14:51:23.592 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:23.586+0000 7fe6b3fff640 1 -- 172.21.15.43:0/3732006264 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe6c4003d20 con 0x7fe6cc0b3050 2023-08-09T14:51:23.592 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:23.586+0000 7fe6d1fe1640 1 -- 172.21.15.43:0/3732006264 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe6cc156f00 con 0x7fe6cc0b3050 2023-08-09T14:51:23.592 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:23.587+0000 7fe6d1fe1640 1 -- 172.21.15.43:0/3732006264 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe6cc15bd90 con 0x7fe6cc0b3050 2023-08-09T14:51:23.592 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:23.588+0000 7fe6d1fe1640 1 -- 172.21.15.43:0/3732006264 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe6cc157420 con 0x7fe6cc0b3050 2023-08-09T14:51:23.594 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:23.592+0000 7fe6b3fff640 1 -- 172.21.15.43:0/3732006264 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 75) v1 ==== 88455+0+0 (secure 0 0 0) 0x7fe6c4003e80 con 0x7fe6cc0b3050 2023-08-09T14:51:23.596 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:23.594+0000 7fe6b3fff640 1 --2- 172.21.15.43:0/3732006264 >> [v2:172.21.15.203:6832/372694067,v1:172.21.15.203:6833/372694067] conn(0x7fe6b806a1f0 0x7fe6b806c6b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T14:51:23.596 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:23.594+0000 7fe6b3fff640 1 -- 172.21.15.43:0/3732006264 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(158..158 src has 1..158) v4 ==== 6916+0+0 (secure 0 0 0) 0x7fe6c40890e0 con 0x7fe6cc0b3050 2023-08-09T14:51:23.596 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:23.595+0000 7fe6cb7fe640 1 -- 172.21.15.43:0/3732006264 >> [v2:172.21.15.203:6832/372694067,v1:172.21.15.203:6833/372694067] conn(0x7fe6b806a1f0 msgr2=0x7fe6b806c6b0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.203:6832/372694067 2023-08-09T14:51:23.596 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:23.595+0000 7fe6cb7fe640 1 --2- 172.21.15.43:0/3732006264 >> [v2:172.21.15.203:6832/372694067,v1:172.21.15.203:6833/372694067] conn(0x7fe6b806a1f0 0x7fe6b806c6b0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-08-09T14:51:23.599 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:23.598+0000 7fe6b3fff640 1 -- 172.21.15.43:0/3732006264 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178066 (secure 0 0 0) 0x7fe6c40542c0 con 0x7fe6cc0b3050 2023-08-09T14:51:23.798 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:23.796+0000 7fe6cb7fe640 1 -- 172.21.15.43:0/3732006264 >> [v2:172.21.15.203:6832/372694067,v1:172.21.15.203:6833/372694067] conn(0x7fe6b806a1f0 msgr2=0x7fe6b806c6b0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.203:6832/372694067 2023-08-09T14:51:23.798 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:23.796+0000 7fe6cb7fe640 1 --2- 172.21.15.43:0/3732006264 >> [v2:172.21.15.203:6832/372694067,v1:172.21.15.203:6833/372694067] conn(0x7fe6b806a1f0 0x7fe6b806c6b0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2023-08-09T14:51:23.800 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:23.798+0000 7fe6d1fe1640 1 -- 172.21.15.43:0/3732006264 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fe6cc157420 con 0x7fe6cc0b3050 2023-08-09T14:51:23.801 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:23.800+0000 7fe6b3fff640 1 -- 172.21.15.43:0/3732006264 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v158) v1 ==== 74+0+19283 (secure 0 0 0) 0x7fe6cc157420 con 0x7fe6cc0b3050 2023-08-09T14:51:23.802 INFO:teuthology.orchestra.run.smithi043.stdout: 2023-08-09T14:51:23.802 INFO:teuthology.orchestra.run.smithi043.stdout:{"epoch":158,"fsid":"daac87ee-1337-4250-a955-aea4e408a39b","created":"2023-08-09T13:33:41.701198+0000","modified":"2023-08-09T14:51:23.585885+0000","last_up_change":"2023-08-09T13:33:53.783453+0000","last_in_change":"2023-08-09T13:33:45.148904+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":19,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":"rbd","create_time":"2023-08-09T13:33:54.365137+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":".mgr","create_time":"2023-08-09T13:33:54.831803+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":18,"pool_name":"cephfs_metadata","create_time":"2023-08-09T13:55:27.207507+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"146","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":19,"pool_name":"cephfs_data","create_time":"2023-08-09T13:55:27.825578+0000","flags":8193,"flags_names":"hashpspool,selfmanaged_snaps","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"157","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":112,"snap_epoch":157,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.25,"score_stable":1.25,"optimal_score":1,"raw_score_acting":1.25,"raw_score_stable":1.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"cbd49ecd-e896-4fbd-864e-5c37a829b165","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":149,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6800","nonce":502667335},{"type":"v1","addr":"172.21.15.43:6801","nonce":502667335}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6802","nonce":502667335},{"type":"v1","addr":"172.21.15.43:6803","nonce":502667335}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6806","nonce":502667335},{"type":"v1","addr":"172.21.15.43:6807","nonce":502667335}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6804","nonce":502667335},{"type":"v1","addr":"172.21.15.43:6805","nonce":502667335}]},"public_addr":"172.21.15.43:6801/502667335","cluster_addr":"172.21.15.43:6803/502667335","heartbeat_back_addr":"172.21.15.43:6807/502667335","heartbeat_front_addr":"172.21.15.43:6805/502667335","state":["exists","up"]},{"osd":1,"uuid":"75c7e0fe-1ec7-4fe8-a2f2-3661a65b8500","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":145,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6808","nonce":3289134610},{"type":"v1","addr":"172.21.15.43:6809","nonce":3289134610}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6810","nonce":3289134610},{"type":"v1","addr":"172.21.15.43:6812","nonce":3289134610}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6821","nonce":3289134610},{"type":"v1","addr":"172.21.15.43:6824","nonce":3289134610}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6815","nonce":3289134610},{"type":"v1","addr":"172.21.15.43:6818","nonce":3289134610}]},"public_addr":"172.21.15.43:6809/3289134610","cluster_addr":"172.21.15.43:6812/3289134610","heartbeat_back_addr":"172.21.15.43:6824/3289134610","heartbeat_front_addr":"172.21.15.43:6818/3289134610","state":["exists","up"]},{"osd":2,"uuid":"bb5c41e9-7f5f-4125-9b01-3bb8bfc2ea83","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":145,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6811","nonce":4079709736},{"type":"v1","addr":"172.21.15.43:6814","nonce":4079709736}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6817","nonce":4079709736},{"type":"v1","addr":"172.21.15.43:6820","nonce":4079709736}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6829","nonce":4079709736},{"type":"v1","addr":"172.21.15.43:6831","nonce":4079709736}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6823","nonce":4079709736},{"type":"v1","addr":"172.21.15.43:6826","nonce":4079709736}]},"public_addr":"172.21.15.43:6814/4079709736","cluster_addr":"172.21.15.43:6820/4079709736","heartbeat_back_addr":"172.21.15.43:6831/4079709736","heartbeat_front_addr":"172.21.15.43:6826/4079709736","state":["exists","up"]},{"osd":3,"uuid":"bfe4c8b4-694a-465c-87e6-2b2aab852613","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":149,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6813","nonce":1709538759},{"type":"v1","addr":"172.21.15.43:6816","nonce":1709538759}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6819","nonce":1709538759},{"type":"v1","addr":"172.21.15.43:6822","nonce":1709538759}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6828","nonce":1709538759},{"type":"v1","addr":"172.21.15.43:6830","nonce":1709538759}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6825","nonce":1709538759},{"type":"v1","addr":"172.21.15.43:6827","nonce":1709538759}]},"public_addr":"172.21.15.43:6816/1709538759","cluster_addr":"172.21.15.43:6822/1709538759","heartbeat_back_addr":"172.21.15.43:6830/1709538759","heartbeat_front_addr":"172.21.15.43:6827/1709538759","state":["exists","up"]},{"osd":4,"uuid":"fa2e8a77-aba1-4756-853e-0d174a5bdeba","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":145,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6800","nonce":4196866481},{"type":"v1","addr":"172.21.15.203:6801","nonce":4196866481}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6802","nonce":4196866481},{"type":"v1","addr":"172.21.15.203:6803","nonce":4196866481}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6806","nonce":4196866481},{"type":"v1","addr":"172.21.15.203:6807","nonce":4196866481}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6804","nonce":4196866481},{"type":"v1","addr":"172.21.15.203:6805","nonce":4196866481}]},"public_addr":"172.21.15.203:6801/4196866481","cluster_addr":"172.21.15.203:6803/4196866481","heartbeat_back_addr":"172.21.15.203:6807/4196866481","heartbeat_front_addr":"172.21.15.203:6805/4196866481","state":["exists","up"]},{"osd":5,"uuid":"81811e4b-4d24-4f51-a82c-f1d700a2a00d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":145,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6816","nonce":3059290081},{"type":"v1","addr":"172.21.15.203:6817","nonce":3059290081}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6818","nonce":3059290081},{"type":"v1","addr":"172.21.15.203:6819","nonce":3059290081}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6822","nonce":3059290081},{"type":"v1","addr":"172.21.15.203:6823","nonce":3059290081}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6820","nonce":3059290081},{"type":"v1","addr":"172.21.15.203:6821","nonce":3059290081}]},"public_addr":"172.21.15.203:6817/3059290081","cluster_addr":"172.21.15.203:6819/3059290081","heartbeat_back_addr":"172.21.15.203:6823/3059290081","heartbeat_front_addr":"172.21.15.203:6821/3059290081","state":["exists","up"]},{"osd":6,"uuid":"307bac4d-6a5b-443d-af75-56d695d7674f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":145,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6824","nonce":512155569},{"type":"v1","addr":"172.21.15.203:6825","nonce":512155569}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6826","nonce":512155569},{"type":"v1","addr":"172.21.15.203:6827","nonce":512155569}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6830","nonce":512155569},{"type":"v1","addr":"172.21.15.203:6831","nonce":512155569}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6828","nonce":512155569},{"type":"v1","addr":"172.21.15.203:6829","nonce":512155569}]},"public_addr":"172.21.15.203:6825/512155569","cluster_addr":"172.21.15.203:6827/512155569","heartbeat_back_addr":"172.21.15.203:6831/512155569","heartbeat_front_addr":"172.21.15.203:6829/512155569","state":["exists","up"]},{"osd":7,"uuid":"a95dc7ca-9f1b-4295-af70-68a22a2d7d5e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":145,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6808","nonce":3331450438},{"type":"v1","addr":"172.21.15.203:6809","nonce":3331450438}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6810","nonce":3331450438},{"type":"v1","addr":"172.21.15.203:6811","nonce":3331450438}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6814","nonce":3331450438},{"type":"v1","addr":"172.21.15.203:6815","nonce":3331450438}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6812","nonce":3331450438},{"type":"v1","addr":"172.21.15.203:6813","nonce":3331450438}]},"public_addr":"172.21.15.203:6809/3331450438","cluster_addr":"172.21.15.203:6811/3331450438","heartbeat_back_addr":"172.21.15.203:6815/3331450438","heartbeat_front_addr":"172.21.15.203:6813/3331450438","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-09T13:33:50.547652+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-09T13:33:50.589469+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-09T13:33:50.561724+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-09T13:33:50.599650+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-09T13:33:50.411855+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-09T13:33:50.219681+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-09T13:33:50.504664+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-09T13:33:50.441248+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"19.25","mappings":[{"from":2,"to":0}]},{"pgid":"19.26","mappings":[{"from":2,"to":0}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.203:0/2901879784":"2023-08-10T13:55:44.498027+0000","172.21.15.203:0/1501229236":"2023-08-10T13:55:44.498027+0000","172.21.15.203:0/3615051942":"2023-08-10T13:55:44.498027+0000","172.21.15.203:6833/3490266242":"2023-08-10T13:55:44.498027+0000","172.21.15.203:6832/3490266242":"2023-08-10T13:55:44.498027+0000","172.21.15.203:0/3892021074":"2023-08-10T13:55:44.498027+0000","172.21.15.203:0/750810033":"2023-08-10T13:55:44.498027+0000","172.21.15.203:0/1164133940":"2023-08-10T13:55:44.498027+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":7,"snaps":[{"begin":104,"length":1},{"begin":221,"length":1}]},{"pool":19,"snaps":[{"begin":23,"length":34},{"begin":79,"length":34}]}],"new_removed_snaps":[],"new_purged_snaps":[{"pool":19,"snaps":[{"begin":3,"length":20},{"begin":59,"length":20}]}],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-08-09T14:51:23.804 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:23.802+0000 7fe6d1fe1640 1 -- 172.21.15.43:0/3732006264 >> [v2:172.21.15.203:6832/372694067,v1:172.21.15.203:6833/372694067] conn(0x7fe6b806a1f0 msgr2=0x7fe6b806c6b0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-08-09T14:51:23.804 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:23.802+0000 7fe6d1fe1640 1 --2- 172.21.15.43:0/3732006264 >> [v2:172.21.15.203:6832/372694067,v1:172.21.15.203:6833/372694067] conn(0x7fe6b806a1f0 0x7fe6b806c6b0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T14:51:23.804 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:23.802+0000 7fe6d1fe1640 1 -- 172.21.15.43:0/3732006264 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fe6cc0b3050 msgr2=0x7fe6cc1596c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T14:51:23.805 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:23.802+0000 7fe6d1fe1640 1 --2- 172.21.15.43:0/3732006264 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fe6cc0b3050 0x7fe6cc1596c0 secure :-1 s=READY pgs=1748 cs=0 l=1 rev1=1 crypto rx=0x7fe6c400cc30 tx=0x7fe6c400a5c0 comp rx=0 tx=0).stop 2023-08-09T14:51:23.805 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:23.802+0000 7fe6d1fe1640 1 -- 172.21.15.43:0/3732006264 shutdown_connections 2023-08-09T14:51:23.805 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:23.802+0000 7fe6d1fe1640 1 --2- 172.21.15.43:0/3732006264 >> [v2:172.21.15.203:6832/372694067,v1:172.21.15.203:6833/372694067] conn(0x7fe6b806a1f0 0x7fe6b806c6b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T14:51:23.805 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:23.802+0000 7fe6d1fe1640 1 --2- 172.21.15.43:0/3732006264 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fe6cc0b3050 0x7fe6cc1596c0 unknown :-1 s=CLOSED pgs=1748 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T14:51:23.805 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:23.802+0000 7fe6d1fe1640 1 --2- 172.21.15.43:0/3732006264 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fe6cc0b2b70 0x7fe6cc0b1960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T14:51:23.805 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:23.803+0000 7fe6d1fe1640 1 --2- 172.21.15.43:0/3732006264 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fe6cc0b2790 0x7fe6cc15b850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T14:51:23.805 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:23.803+0000 7fe6d1fe1640 1 -- 172.21.15.43:0/3732006264 >> 172.21.15.43:0/3732006264 conn(0x7fe6cc09b7b0 msgr2=0x7fe6cc1609a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-09T14:51:23.805 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:23.803+0000 7fe6d1fe1640 1 -- 172.21.15.43:0/3732006264 shutdown_connections 2023-08-09T14:51:23.806 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:23.803+0000 7fe6d1fe1640 1 -- 172.21.15.43:0/3732006264 wait complete. 2023-08-09T14:51:23.823 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': 'rbd', 'create_time': '2023-08-09T13:33:54.365137+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}, 'read_balance': {'score_acting': 2, 'score_stable': 2, 'optimal_score': 1, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': '.mgr', 'create_time': '2023-08-09T13:33:54.831803+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 18, 'pool_name': 'cephfs_metadata', 'create_time': '2023-08-09T13:55:27.207507+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '146', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 19, 'pool_name': 'cephfs_data', 'create_time': '2023-08-09T13:55:27.825578+0000', 'flags': 8193, 'flags_names': 'hashpspool,selfmanaged_snaps', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '157', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 112, 'snap_epoch': 157, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.25, 'score_stable': 1.25, 'optimal_score': 1, 'raw_score_acting': 1.25, 'raw_score_stable': 1.25, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-08-09T14:51:23.823 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get rbd pg_num 2023-08-09T14:51:24.036 INFO:tasks.ceph.mgr.x.smithi203.stderr:2023-08-09T14:51:24.036+0000 7fe0afe0f040 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2023-08-09T14:51:24.040 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:24.039+0000 7fd65a712640 1 Processor -- start 2023-08-09T14:51:24.041 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:24.039+0000 7fd65a712640 1 -- start start 2023-08-09T14:51:24.042 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:24.041+0000 7fd65a712640 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7fd6540b45f0 0x7fd6540cd6d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T14:51:24.042 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:24.041+0000 7fd65a712640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7fd6540d04b0 0x7fd6540b59d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T14:51:24.043 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:24.041+0000 7fd65a712640 1 -- --> v1:172.21.15.203:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd6540b7bd0 con 0x7fd65415c550 2023-08-09T14:51:24.043 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:24.041+0000 7fd65a712640 1 -- --> v2:172.21.15.43:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd6540d1c00 con 0x7fd6540d04b0 2023-08-09T14:51:24.043 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:24.041+0000 7fd65a712640 1 -- --> v2:172.21.15.203:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd65415c930 con 0x7fd6540b45f0 2023-08-09T14:51:24.043 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:24.041+0000 7fd658caf640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7fd6540d04b0 0x7fd6540b59d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T14:51:24.044 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:24.041+0000 7fd658caf640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7fd6540d04b0 0x7fd6540b59d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:39090/0 (socket says 172.21.15.43:39090) 2023-08-09T14:51:24.044 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:24.041+0000 7fd658caf640 1 -- 172.21.15.43:0/3590101036 learned_addr learned my addr 172.21.15.43:0/3590101036 (peer_addr_for_me v2:172.21.15.43:0/0) 2023-08-09T14:51:24.044 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:24.041+0000 7fd6537fe640 1 --2- 172.21.15.43:0/3590101036 >> v2:172.21.15.203:3300/0 conn(0x7fd6540b45f0 0x7fd6540cd6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T14:51:24.044 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:24.042+0000 7fd658caf640 1 -- 172.21.15.43:0/3590101036 >> v2:172.21.15.203:3300/0 conn(0x7fd6540b45f0 msgr2=0x7fd6540cd6d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-09T14:51:24.044 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:24.042+0000 7fd658caf640 1 --2- 172.21.15.43:0/3590101036 >> v2:172.21.15.203:3300/0 conn(0x7fd6540b45f0 0x7fd6540cd6d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T14:51:24.044 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:24.042+0000 7fd658caf640 1 -- 172.21.15.43:0/3590101036 >> v1:172.21.15.203:6789/0 conn(0x7fd65415c550 legacy=0x7fd6540b3ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-09T14:51:24.045 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:24.042+0000 7fd658caf640 1 -- 172.21.15.43:0/3590101036 --> v2:172.21.15.43:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd6540b54f0 con 0x7fd6540d04b0 2023-08-09T14:51:24.045 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:24.043+0000 7fd658caf640 1 --2- 172.21.15.43:0/3590101036 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fd6540d04b0 0x7fd6540b59d0 secure :-1 s=READY pgs=1749 cs=0 l=1 rev1=1 crypto rx=0x7fd6540b5050 tx=0x7fd64c02bee0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ecdfc728fb9d51f6 server_cookie=0 in_seq=0 out_seq=0 2023-08-09T14:51:24.045 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:24.044+0000 7fd652ffd640 1 -- 172.21.15.43:0/3590101036 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd64c02ca80 con 0x7fd6540d04b0 2023-08-09T14:51:24.045 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:24.044+0000 7fd652ffd640 1 -- 172.21.15.43:0/3590101036 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd64c02cbe0 con 0x7fd6540d04b0 2023-08-09T14:51:24.046 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:24.044+0000 7fd652ffd640 1 -- 172.21.15.43:0/3590101036 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd64c0343f0 con 0x7fd6540d04b0 2023-08-09T14:51:24.046 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:24.045+0000 7fd65a712640 1 -- 172.21.15.43:0/3590101036 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fd6540d04b0 msgr2=0x7fd6540b59d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T14:51:24.046 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:24.045+0000 7fd65a712640 1 --2- 172.21.15.43:0/3590101036 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fd6540d04b0 0x7fd6540b59d0 secure :-1 s=READY pgs=1749 cs=0 l=1 rev1=1 crypto rx=0x7fd6540b5050 tx=0x7fd64c02bee0 comp rx=0 tx=0).stop 2023-08-09T14:51:24.046 INFO:tasks.ceph.mgr.y.smithi043.stderr:2023-08-09T14:51:24.045+0000 7f6a9a356040 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2023-08-09T14:51:24.047 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:24.045+0000 7fd65a712640 1 -- 172.21.15.43:0/3590101036 shutdown_connections 2023-08-09T14:51:24.047 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:24.045+0000 7fd65a712640 1 --2- 172.21.15.43:0/3590101036 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fd6540d04b0 0x7fd6540b59d0 unknown :-1 s=CLOSED pgs=1749 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T14:51:24.047 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:24.045+0000 7fd65a712640 1 --2- 172.21.15.43:0/3590101036 >> v2:172.21.15.203:3300/0 conn(0x7fd6540b45f0 0x7fd6540cd6d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T14:51:24.048 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:24.045+0000 7fd65a712640 1 -- 172.21.15.43:0/3590101036 >> 172.21.15.43:0/3590101036 conn(0x7fd654071520 msgr2=0x7fd6540ceee0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-09T14:51:24.048 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:24.045+0000 7fd65a712640 1 -- 172.21.15.43:0/3590101036 shutdown_connections 2023-08-09T14:51:24.048 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:24.045+0000 7fd65a712640 1 -- 172.21.15.43:0/3590101036 wait complete. 2023-08-09T14:51:24.049 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:24.047+0000 7fd65a712640 1 Processor -- start 2023-08-09T14:51:24.049 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:24.048+0000 7fd65a712640 1 -- start start 2023-08-09T14:51:24.051 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:24.049+0000 7fd65a712640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fd6540b45f0 0x7fd65415ac10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T14:51:24.051 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:24.049+0000 7fd65a712640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fd6540d04b0 0x7fd6540aaac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T14:51:24.051 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:24.049+0000 7fd65a712640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fd65415c550 0x7fd6540cb540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T14:51:24.051 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:24.049+0000 7fd65a712640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd65415b150 con 0x7fd6540d04b0 2023-08-09T14:51:24.052 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:24.049+0000 7fd65a712640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd6541e6be0 con 0x7fd6540b45f0 2023-08-09T14:51:24.052 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:24.049+0000 7fd65a712640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd6540a9d00 con 0x7fd65415c550 2023-08-09T14:51:24.052 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:24.050+0000 7fd6537fe640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fd6540d04b0 0x7fd6540aaac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T14:51:24.052 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:24.050+0000 7fd6537fe640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fd6540d04b0 0x7fd6540aaac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:39102/0 (socket says 172.21.15.43:39102) 2023-08-09T14:51:24.052 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:24.050+0000 7fd6537fe640 1 -- 172.21.15.43:0/2219407410 learned_addr learned my addr 172.21.15.43:0/2219407410 (peer_addr_for_me v2:172.21.15.43:0/0) 2023-08-09T14:51:24.052 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:24.050+0000 7fd653fff640 1 --2- 172.21.15.43:0/2219407410 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fd6540b45f0 0x7fd65415ac10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T14:51:24.053 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:24.050+0000 7fd658caf640 1 --2- 172.21.15.43:0/2219407410 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fd65415c550 0x7fd6540cb540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T14:51:24.054 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:24.050+0000 7fd6537fe640 1 -- 172.21.15.43:0/2219407410 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fd65415c550 msgr2=0x7fd6540cb540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T14:51:24.054 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:24.050+0000 7fd6537fe640 1 --2- 172.21.15.43:0/2219407410 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fd65415c550 0x7fd6540cb540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T14:51:24.054 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:24.050+0000 7fd6537fe640 1 -- 172.21.15.43:0/2219407410 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fd6540b45f0 msgr2=0x7fd65415ac10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T14:51:24.054 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:24.050+0000 7fd6537fe640 1 --2- 172.21.15.43:0/2219407410 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fd6540b45f0 0x7fd65415ac10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T14:51:24.054 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:24.050+0000 7fd6537fe640 1 -- 172.21.15.43:0/2219407410 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd64c009b30 con 0x7fd6540d04b0 2023-08-09T14:51:24.054 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:24.051+0000 7fd6537fe640 1 --2- 172.21.15.43:0/2219407410 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fd6540d04b0 0x7fd6540aaac0 secure :-1 s=READY pgs=1750 cs=0 l=1 rev1=1 crypto rx=0x7fd64800e9e0 tx=0x7fd64800eea0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-09T14:51:24.055 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:24.051+0000 7fd650ff9640 1 -- 172.21.15.43:0/2219407410 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd648009670 con 0x7fd6540d04b0 2023-08-09T14:51:24.055 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:24.051+0000 7fd650ff9640 1 -- 172.21.15.43:0/2219407410 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd64800c9d0 con 0x7fd6540d04b0 2023-08-09T14:51:24.055 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:24.051+0000 7fd650ff9640 1 -- 172.21.15.43:0/2219407410 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd64800cd80 con 0x7fd6540d04b0 2023-08-09T14:51:24.055 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:24.051+0000 7fd65a712640 1 -- 172.21.15.43:0/2219407410 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd6540a9fe0 con 0x7fd6540d04b0 2023-08-09T14:51:24.055 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:24.051+0000 7fd65a712640 1 -- 172.21.15.43:0/2219407410 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd6540aa2c0 con 0x7fd6540d04b0 2023-08-09T14:51:24.055 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:24.052+0000 7fd65a712640 1 -- 172.21.15.43:0/2219407410 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd6540739f0 con 0x7fd6540d04b0 2023-08-09T14:51:24.056 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:24.053+0000 7fd650ff9640 1 -- 172.21.15.43:0/2219407410 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 75) v1 ==== 88455+0+0 (secure 0 0 0) 0x7fd648013470 con 0x7fd6540d04b0 2023-08-09T14:51:24.056 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:24.055+0000 7fd650ff9640 1 --2- 172.21.15.43:0/2219407410 >> [v2:172.21.15.203:6832/372694067,v1:172.21.15.203:6833/372694067] conn(0x7fd64006a170 0x7fd64006c630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T14:51:24.056 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:24.055+0000 7fd650ff9640 1 -- 172.21.15.43:0/2219407410 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(158..158 src has 1..158) v4 ==== 6916+0+0 (secure 0 0 0) 0x7fd648088c70 con 0x7fd6540d04b0 2023-08-09T14:51:24.057 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:24.055+0000 7fd653fff640 1 -- 172.21.15.43:0/2219407410 >> [v2:172.21.15.203:6832/372694067,v1:172.21.15.203:6833/372694067] conn(0x7fd64006a170 msgr2=0x7fd64006c630 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.203:6832/372694067 2023-08-09T14:51:24.057 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:24.055+0000 7fd653fff640 1 --2- 172.21.15.43:0/2219407410 >> [v2:172.21.15.203:6832/372694067,v1:172.21.15.203:6833/372694067] conn(0x7fd64006a170 0x7fd64006c630 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-08-09T14:51:24.061 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:24.059+0000 7fd650ff9640 1 -- 172.21.15.43:0/2219407410 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178066 (secure 0 0 0) 0x7fd648053f00 con 0x7fd6540d04b0 2023-08-09T14:51:24.156 INFO:tasks.ceph.mgr.x.smithi203.stderr:2023-08-09T14:51:24.155+0000 7fe0afe0f040 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2023-08-09T14:51:24.175 INFO:tasks.ceph.mgr.y.smithi043.stderr:2023-08-09T14:51:24.173+0000 7f6a9a356040 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2023-08-09T14:51:24.212 INFO:tasks.ceph.mgr.x.smithi203.stderr:2023-08-09T14:51:24.211+0000 7fe0afe0f040 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2023-08-09T14:51:24.234 INFO:tasks.ceph.mgr.y.smithi043.stderr:2023-08-09T14:51:24.232+0000 7f6a9a356040 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2023-08-09T14:51:24.251 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:24.250+0000 7fd65a712640 1 -- 172.21.15.43:0/2219407410 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"} v 0) v1 -- 0x7fd6540ccff0 con 0x7fd6540d04b0 2023-08-09T14:51:24.252 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:24.250+0000 7fd650ff9640 1 -- 172.21.15.43:0/2219407410 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"}]=0 v158) v1 ==== 92+0+10 (secure 0 0 0) 0x7fd648057520 con 0x7fd6540d04b0 2023-08-09T14:51:24.252 INFO:teuthology.orchestra.run.smithi043.stdout:pg_num: 8 2023-08-09T14:51:24.255 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:24.253+0000 7fd65a712640 1 -- 172.21.15.43:0/2219407410 >> [v2:172.21.15.203:6832/372694067,v1:172.21.15.203:6833/372694067] conn(0x7fd64006a170 msgr2=0x7fd64006c630 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-08-09T14:51:24.255 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:24.253+0000 7fd65a712640 1 --2- 172.21.15.43:0/2219407410 >> [v2:172.21.15.203:6832/372694067,v1:172.21.15.203:6833/372694067] conn(0x7fd64006a170 0x7fd64006c630 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T14:51:24.255 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:24.253+0000 7fd65a712640 1 -- 172.21.15.43:0/2219407410 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fd6540d04b0 msgr2=0x7fd6540aaac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T14:51:24.255 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:24.253+0000 7fd65a712640 1 --2- 172.21.15.43:0/2219407410 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fd6540d04b0 0x7fd6540aaac0 secure :-1 s=READY pgs=1750 cs=0 l=1 rev1=1 crypto rx=0x7fd64800e9e0 tx=0x7fd64800eea0 comp rx=0 tx=0).stop 2023-08-09T14:51:24.255 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:24.253+0000 7fd65a712640 1 -- 172.21.15.43:0/2219407410 shutdown_connections 2023-08-09T14:51:24.255 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:24.253+0000 7fd65a712640 1 --2- 172.21.15.43:0/2219407410 >> [v2:172.21.15.203:6832/372694067,v1:172.21.15.203:6833/372694067] conn(0x7fd64006a170 0x7fd64006c630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T14:51:24.256 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:24.253+0000 7fd65a712640 1 --2- 172.21.15.43:0/2219407410 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fd65415c550 0x7fd6540cb540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T14:51:24.256 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:24.253+0000 7fd65a712640 1 --2- 172.21.15.43:0/2219407410 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fd6540d04b0 0x7fd6540aaac0 unknown :-1 s=CLOSED pgs=1750 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T14:51:24.256 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:24.253+0000 7fd65a712640 1 --2- 172.21.15.43:0/2219407410 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fd6540b45f0 0x7fd65415ac10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T14:51:24.256 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:24.253+0000 7fd65a712640 1 -- 172.21.15.43:0/2219407410 >> 172.21.15.43:0/2219407410 conn(0x7fd654071520 msgr2=0x7fd65415f270 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-09T14:51:24.256 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:24.253+0000 7fd65a712640 1 -- 172.21.15.43:0/2219407410 shutdown_connections 2023-08-09T14:51:24.256 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:24.253+0000 7fd65a712640 1 -- 172.21.15.43:0/2219407410 wait complete. 2023-08-09T14:51:24.261 INFO:tasks.ceph.mgr.x.smithi203.stderr:2023-08-09T14:51:24.260+0000 7fe0afe0f040 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2023-08-09T14:51:24.276 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-08-09T14:51:24.288 INFO:tasks.ceph.mgr.y.smithi043.stderr:2023-08-09T14:51:24.286+0000 7f6a9a356040 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2023-08-09T14:51:24.496 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:24.495+0000 7fe06d226640 1 Processor -- start 2023-08-09T14:51:24.497 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:24.495+0000 7fe06d226640 1 -- start start 2023-08-09T14:51:24.498 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:24.497+0000 7fe06d226640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fe0680b4610 0x7fe0680cd6f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T14:51:24.498 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:24.497+0000 7fe06d226640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7fe0680d04f0 0x7fe0680b5b60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T14:51:24.498 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:24.497+0000 7fe06d226640 1 -- --> v1:172.21.15.203:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe0680b56b0 con 0x7fe06815c600 2023-08-09T14:51:24.499 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:24.497+0000 7fe06d226640 1 -- --> v2:172.21.15.43:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe06815c9e0 con 0x7fe0680d04f0 2023-08-09T14:51:24.499 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:24.497+0000 7fe06d226640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe0680d0980 con 0x7fe0680b4610 2023-08-09T14:51:24.499 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:24.497+0000 7fe06759e640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7fe0680d04f0 0x7fe0680b5b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T14:51:24.499 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:24.497+0000 7fe06759e640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7fe0680d04f0 0x7fe0680b5b60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:39104/0 (socket says 172.21.15.43:39104) 2023-08-09T14:51:24.499 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:24.497+0000 7fe06759e640 1 -- 172.21.15.43:0/1165516321 learned_addr learned my addr 172.21.15.43:0/1165516321 (peer_addr_for_me v2:172.21.15.43:0/0) 2023-08-09T14:51:24.500 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:24.497+0000 7fe06659c640 1 --2- 172.21.15.43:0/1165516321 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fe0680b4610 0x7fe0680cd6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T14:51:24.500 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:24.497+0000 7fe06759e640 1 -- 172.21.15.43:0/1165516321 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fe0680b4610 msgr2=0x7fe0680cd6f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-09T14:51:24.500 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:24.497+0000 7fe06759e640 1 --2- 172.21.15.43:0/1165516321 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fe0680b4610 0x7fe0680cd6f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T14:51:24.501 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:24.497+0000 7fe06759e640 1 -- 172.21.15.43:0/1165516321 >> v1:172.21.15.203:6789/0 conn(0x7fe06815c600 legacy=0x7fe0680b3f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-09T14:51:24.501 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:24.497+0000 7fe06759e640 1 -- 172.21.15.43:0/1165516321 --> v2:172.21.15.43:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe0680d0c40 con 0x7fe0680d04f0 2023-08-09T14:51:24.501 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:24.498+0000 7fe06759e640 1 --2- 172.21.15.43:0/1165516321 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fe0680d04f0 0x7fe0680b5b60 secure :-1 s=READY pgs=1751 cs=0 l=1 rev1=1 crypto rx=0x7fe060009e50 tx=0x7fe06002bf60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=41ecaab329c43af6 server_cookie=0 in_seq=0 out_seq=0 2023-08-09T14:51:24.501 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:24.499+0000 7fe065d9b640 1 -- 172.21.15.43:0/1165516321 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe06002cb00 con 0x7fe0680d04f0 2023-08-09T14:51:24.501 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:24.499+0000 7fe065d9b640 1 -- 172.21.15.43:0/1165516321 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe06002cc60 con 0x7fe0680d04f0 2023-08-09T14:51:24.501 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:24.499+0000 7fe065d9b640 1 -- 172.21.15.43:0/1165516321 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe0600343f0 con 0x7fe0680d04f0 2023-08-09T14:51:24.502 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:24.499+0000 7fe06d226640 1 -- 172.21.15.43:0/1165516321 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fe0680d04f0 msgr2=0x7fe0680b5b60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T14:51:24.502 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:24.499+0000 7fe06d226640 1 --2- 172.21.15.43:0/1165516321 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fe0680d04f0 0x7fe0680b5b60 secure :-1 s=READY pgs=1751 cs=0 l=1 rev1=1 crypto rx=0x7fe060009e50 tx=0x7fe06002bf60 comp rx=0 tx=0).stop 2023-08-09T14:51:24.502 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:24.500+0000 7fe06d226640 1 -- 172.21.15.43:0/1165516321 shutdown_connections 2023-08-09T14:51:24.502 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:24.500+0000 7fe06d226640 1 --2- 172.21.15.43:0/1165516321 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fe0680d04f0 0x7fe0680b5b60 unknown :-1 s=CLOSED pgs=1751 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T14:51:24.503 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:24.500+0000 7fe06d226640 1 --2- 172.21.15.43:0/1165516321 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fe0680b4610 0x7fe0680cd6f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T14:51:24.503 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:24.500+0000 7fe06d226640 1 -- 172.21.15.43:0/1165516321 >> 172.21.15.43:0/1165516321 conn(0x7fe068071520 msgr2=0x7fe0680cef00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-09T14:51:24.503 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:24.500+0000 7fe06d226640 1 -- 172.21.15.43:0/1165516321 shutdown_connections 2023-08-09T14:51:24.503 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:24.500+0000 7fe06d226640 1 -- 172.21.15.43:0/1165516321 wait complete. 2023-08-09T14:51:24.503 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:24.501+0000 7fe06d226640 1 Processor -- start 2023-08-09T14:51:24.504 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:24.502+0000 7fe06d226640 1 -- start start 2023-08-09T14:51:24.505 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:24.503+0000 7fe06d226640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fe0680b4610 0x7fe068158ca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T14:51:24.505 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:24.503+0000 7fe06d226640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fe0680d04f0 0x7fe0680c8c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T14:51:24.505 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:24.503+0000 7fe06d226640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fe06815c600 0x7fe0680c7110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T14:51:24.505 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:24.503+0000 7fe06d226640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe0680c87e0 con 0x7fe0680b4610 2023-08-09T14:51:24.506 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:24.503+0000 7fe06d226640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe0681591e0 con 0x7fe06815c600 2023-08-09T14:51:24.506 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:24.503+0000 7fe06d226640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe0680c91b0 con 0x7fe0680d04f0 2023-08-09T14:51:24.506 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:24.503+0000 7fe066d9d640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fe0680b4610 0x7fe068158ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T14:51:24.506 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:24.503+0000 7fe066d9d640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fe0680b4610 0x7fe068158ca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:39112/0 (socket says 172.21.15.43:39112) 2023-08-09T14:51:24.506 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:24.503+0000 7fe066d9d640 1 -- 172.21.15.43:0/2231715922 learned_addr learned my addr 172.21.15.43:0/2231715922 (peer_addr_for_me v2:172.21.15.43:0/0) 2023-08-09T14:51:24.506 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:24.503+0000 7fe06659c640 1 --2- 172.21.15.43:0/2231715922 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fe0680d04f0 0x7fe0680c8c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T14:51:24.506 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:24.503+0000 7fe06759e640 1 --2- 172.21.15.43:0/2231715922 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fe06815c600 0x7fe0680c7110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T14:51:24.507 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:24.503+0000 7fe066d9d640 1 -- 172.21.15.43:0/2231715922 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fe0680d04f0 msgr2=0x7fe0680c8c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T14:51:24.507 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:24.503+0000 7fe066d9d640 1 --2- 172.21.15.43:0/2231715922 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fe0680d04f0 0x7fe0680c8c70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T14:51:24.507 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:24.503+0000 7fe066d9d640 1 -- 172.21.15.43:0/2231715922 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fe06815c600 msgr2=0x7fe0680c7110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T14:51:24.507 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:24.503+0000 7fe066d9d640 1 --2- 172.21.15.43:0/2231715922 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fe06815c600 0x7fe0680c7110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T14:51:24.507 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:24.503+0000 7fe066d9d640 1 -- 172.21.15.43:0/2231715922 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe060009b30 con 0x7fe0680b4610 2023-08-09T14:51:24.507 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:24.503+0000 7fe06659c640 1 --2- 172.21.15.43:0/2231715922 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fe0680d04f0 0x7fe0680c8c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-08-09T14:51:24.507 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:24.504+0000 7fe066d9d640 1 --2- 172.21.15.43:0/2231715922 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fe0680b4610 0x7fe068158ca0 secure :-1 s=READY pgs=1752 cs=0 l=1 rev1=1 crypto rx=0x7fe05000c430 tx=0x7fe05000c8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-09T14:51:24.509 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:24.504+0000 7fe05f7fe640 1 -- 172.21.15.43:0/2231715922 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe050011790 con 0x7fe0680b4610 2023-08-09T14:51:24.509 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:24.504+0000 7fe05f7fe640 1 -- 172.21.15.43:0/2231715922 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe0500118f0 con 0x7fe0680b4610 2023-08-09T14:51:24.509 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:24.504+0000 7fe05f7fe640 1 -- 172.21.15.43:0/2231715922 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe050011ce0 con 0x7fe0680b4610 2023-08-09T14:51:24.509 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:24.504+0000 7fe06d226640 1 -- 172.21.15.43:0/2231715922 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe0680c7bd0 con 0x7fe0680b4610 2023-08-09T14:51:24.509 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:24.504+0000 7fe06d226640 1 -- 172.21.15.43:0/2231715922 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe0680c8190 con 0x7fe0680b4610 2023-08-09T14:51:24.509 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:24.505+0000 7fe06d226640 1 -- 172.21.15.43:0/2231715922 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe06804e3e0 con 0x7fe0680b4610 2023-08-09T14:51:24.509 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:24.506+0000 7fe05f7fe640 1 -- 172.21.15.43:0/2231715922 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 75) v1 ==== 88455+0+0 (secure 0 0 0) 0x7fe05001c460 con 0x7fe0680b4610 2023-08-09T14:51:24.510 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:24.507+0000 7fe05f7fe640 1 --2- 172.21.15.43:0/2231715922 >> [v2:172.21.15.203:6832/372694067,v1:172.21.15.203:6833/372694067] conn(0x7fe04806a170 0x7fe04806c630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T14:51:24.510 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:24.507+0000 7fe05f7fe640 1 -- 172.21.15.43:0/2231715922 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(158..158 src has 1..158) v4 ==== 6916+0+0 (secure 0 0 0) 0x7fe050088aa0 con 0x7fe0680b4610 2023-08-09T14:51:24.510 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:24.507+0000 7fe06659c640 1 -- 172.21.15.43:0/2231715922 >> [v2:172.21.15.203:6832/372694067,v1:172.21.15.203:6833/372694067] conn(0x7fe04806a170 msgr2=0x7fe04806c630 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.203:6832/372694067 2023-08-09T14:51:24.510 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:24.507+0000 7fe06659c640 1 --2- 172.21.15.43:0/2231715922 >> [v2:172.21.15.203:6832/372694067,v1:172.21.15.203:6833/372694067] conn(0x7fe04806a170 0x7fe04806c630 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-08-09T14:51:24.517 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:24.515+0000 7fe05f7fe640 1 -- 172.21.15.43:0/2231715922 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178066 (secure 0 0 0) 0x7fe050053c80 con 0x7fe0680b4610 2023-08-09T14:51:24.694 DEBUG:teuthology.orchestra.run.smithi043:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-08-09T14:51:24.727 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:24.708+0000 7fe06659c640 1 -- 172.21.15.43:0/2231715922 >> [v2:172.21.15.203:6832/372694067,v1:172.21.15.203:6833/372694067] conn(0x7fe04806a170 msgr2=0x7fe04806c630 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.203:6832/372694067 2023-08-09T14:51:24.728 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:24.708+0000 7fe06659c640 1 --2- 172.21.15.43:0/2231715922 >> [v2:172.21.15.203:6832/372694067,v1:172.21.15.203:6833/372694067] conn(0x7fe04806a170 0x7fe04806c630 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2023-08-09T14:51:24.735 DEBUG:teuthology.orchestra.run.smithi203:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-08-09T14:51:24.743 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:24.741+0000 7fe06d226640 1 -- 172.21.15.43:0/2231715922 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7fe0680739f0 con 0x7fe0680b4610 2023-08-09T14:51:24.743 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:24.741+0000 7fe05f7fe640 1 -- 172.21.15.43:0/2231715922 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v159) v1 ==== 93+0+10 (secure 0 0 0) 0x7fe0500572a0 con 0x7fe0680b4610 2023-08-09T14:51:24.744 INFO:teuthology.orchestra.run.smithi043.stdout:pg_num: 1 2023-08-09T14:51:24.746 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:24.744+0000 7fe06d226640 1 -- 172.21.15.43:0/2231715922 >> [v2:172.21.15.203:6832/372694067,v1:172.21.15.203:6833/372694067] conn(0x7fe04806a170 msgr2=0x7fe04806c630 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-08-09T14:51:24.746 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:24.744+0000 7fe06d226640 1 --2- 172.21.15.43:0/2231715922 >> [v2:172.21.15.203:6832/372694067,v1:172.21.15.203:6833/372694067] conn(0x7fe04806a170 0x7fe04806c630 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T14:51:24.746 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:24.744+0000 7fe06d226640 1 -- 172.21.15.43:0/2231715922 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fe0680b4610 msgr2=0x7fe068158ca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T14:51:24.746 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:24.744+0000 7fe06d226640 1 --2- 172.21.15.43:0/2231715922 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fe0680b4610 0x7fe068158ca0 secure :-1 s=READY pgs=1752 cs=0 l=1 rev1=1 crypto rx=0x7fe05000c430 tx=0x7fe05000c8f0 comp rx=0 tx=0).stop 2023-08-09T14:51:24.747 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:24.744+0000 7fe06d226640 1 -- 172.21.15.43:0/2231715922 shutdown_connections 2023-08-09T14:51:24.747 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:24.744+0000 7fe06d226640 1 --2- 172.21.15.43:0/2231715922 >> [v2:172.21.15.203:6832/372694067,v1:172.21.15.203:6833/372694067] conn(0x7fe04806a170 0x7fe04806c630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T14:51:24.747 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:24.744+0000 7fe06d226640 1 --2- 172.21.15.43:0/2231715922 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fe06815c600 0x7fe0680c7110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T14:51:24.747 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:24.744+0000 7fe06d226640 1 --2- 172.21.15.43:0/2231715922 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fe0680d04f0 0x7fe0680c8c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T14:51:24.748 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:24.744+0000 7fe06d226640 1 --2- 172.21.15.43:0/2231715922 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fe0680b4610 0x7fe068158ca0 unknown :-1 s=CLOSED pgs=1752 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T14:51:24.748 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:24.744+0000 7fe06d226640 1 -- 172.21.15.43:0/2231715922 >> 172.21.15.43:0/2231715922 conn(0x7fe068071520 msgr2=0x7fe0680ceed0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-09T14:51:24.748 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:24.744+0000 7fe06d226640 1 -- 172.21.15.43:0/2231715922 shutdown_connections 2023-08-09T14:51:24.748 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:24.744+0000 7fe06d226640 1 -- 172.21.15.43:0/2231715922 wait complete. 2023-08-09T14:51:24.764 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-08-09T14:51:24.957 INFO:tasks.ceph.mgr.x.smithi203.stderr:2023-08-09T14:51:24.956+0000 7fe0afe0f040 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2023-08-09T14:51:25.009 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:25.008+0000 7fbb6780d640 1 Processor -- start 2023-08-09T14:51:25.010 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:25.008+0000 7fbb6780d640 1 -- start start 2023-08-09T14:51:25.011 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:25.009+0000 7fbb6780d640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7fbb6015c680 0x7fbb600ce760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T14:51:25.011 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:25.009+0000 7fbb6780d640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fbb600b3fa0 0x7fbb600b43f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T14:51:25.011 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:25.009+0000 7fbb6780d640 1 -- --> v1:172.21.15.43:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbb600b4930 con 0x7fbb600d0a70 2023-08-09T14:51:25.011 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:25.009+0000 7fbb6780d640 1 -- --> v2:172.21.15.43:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbb600cdf00 con 0x7fbb6015c680 2023-08-09T14:51:25.011 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:25.009+0000 7fbb6780d640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbb600d0f70 con 0x7fbb600b3fa0 2023-08-09T14:51:25.012 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:25.009+0000 7fbb655a9640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7fbb6015c680 0x7fbb600ce760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T14:51:25.012 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:25.010+0000 7fbb655a9640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7fbb6015c680 0x7fbb600ce760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:39734/0 (socket says 172.21.15.43:39734) 2023-08-09T14:51:25.012 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:25.010+0000 7fbb64da8640 1 --1- >> v1:172.21.15.43:6789/0 conn(0x7fbb600d0a70 0x7fbb600cd7f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.43:6789/0 says I am v1:172.21.15.43:54848/0 (socket says 172.21.15.43:54848) 2023-08-09T14:51:25.012 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:25.010+0000 7fbb655a9640 1 -- 172.21.15.43:0/20085536 learned_addr learned my addr 172.21.15.43:0/20085536 (peer_addr_for_me v2:172.21.15.43:0/0) 2023-08-09T14:51:25.013 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:25.010+0000 7fbb65daa640 1 --2- 172.21.15.43:0/20085536 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fbb600b3fa0 0x7fbb600b43f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T14:51:25.013 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:25.010+0000 7fbb655a9640 1 -- 172.21.15.43:0/20085536 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fbb600b3fa0 msgr2=0x7fbb600b43f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-09T14:51:25.013 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:25.010+0000 7fbb655a9640 1 --2- 172.21.15.43:0/20085536 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fbb600b3fa0 0x7fbb600b43f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T14:51:25.013 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:25.010+0000 7fbb655a9640 1 -- 172.21.15.43:0/20085536 >> v1:172.21.15.43:6789/0 conn(0x7fbb600d0a70 legacy=0x7fbb600cd7f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T14:51:25.014 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:25.010+0000 7fbb655a9640 1 -- 172.21.15.43:0/20085536 --> v2:172.21.15.43:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbb600ccd10 con 0x7fbb6015c680 2023-08-09T14:51:25.014 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:25.010+0000 7fbb65daa640 1 --2- 172.21.15.43:0/20085536 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fbb600b3fa0 0x7fbb600b43f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-08-09T14:51:25.014 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:25.011+0000 7fbb655a9640 1 --2- 172.21.15.43:0/20085536 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fbb6015c680 0x7fbb600ce760 secure :-1 s=READY pgs=1754 cs=0 l=1 rev1=1 crypto rx=0x7fbb5c009870 tx=0x7fbb5c02b980 comp rx=0 tx=0).ready entity=mon.0 client_cookie=221955dcd2434d83 server_cookie=0 in_seq=0 out_seq=0 2023-08-09T14:51:25.014 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:25.012+0000 7fbb57fff640 1 -- 172.21.15.43:0/20085536 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbb5c030840 con 0x7fbb6015c680 2023-08-09T14:51:25.014 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:25.012+0000 7fbb57fff640 1 -- 172.21.15.43:0/20085536 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbb5c0309a0 con 0x7fbb6015c680 2023-08-09T14:51:25.014 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:25.012+0000 7fbb57fff640 1 -- 172.21.15.43:0/20085536 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbb5c030d30 con 0x7fbb6015c680 2023-08-09T14:51:25.015 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:25.012+0000 7fbb6780d640 1 -- 172.21.15.43:0/20085536 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fbb6015c680 msgr2=0x7fbb600ce760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T14:51:25.015 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:25.012+0000 7fbb6780d640 1 --2- 172.21.15.43:0/20085536 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fbb6015c680 0x7fbb600ce760 secure :-1 s=READY pgs=1754 cs=0 l=1 rev1=1 crypto rx=0x7fbb5c009870 tx=0x7fbb5c02b980 comp rx=0 tx=0).stop 2023-08-09T14:51:25.015 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:25.012+0000 7fbb6780d640 1 -- 172.21.15.43:0/20085536 shutdown_connections 2023-08-09T14:51:25.015 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:25.012+0000 7fbb6780d640 1 --2- 172.21.15.43:0/20085536 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fbb600b3fa0 0x7fbb600b43f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T14:51:25.016 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:25.012+0000 7fbb6780d640 1 --2- 172.21.15.43:0/20085536 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fbb6015c680 0x7fbb600ce760 unknown :-1 s=CLOSED pgs=1754 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T14:51:25.016 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:25.012+0000 7fbb6780d640 1 -- 172.21.15.43:0/20085536 >> 172.21.15.43:0/20085536 conn(0x7fbb60071520 msgr2=0x7fbb600d2520 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-09T14:51:25.016 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:25.012+0000 7fbb6780d640 1 -- 172.21.15.43:0/20085536 shutdown_connections 2023-08-09T14:51:25.016 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:25.012+0000 7fbb6780d640 1 -- 172.21.15.43:0/20085536 wait complete. 2023-08-09T14:51:25.017 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:25.014+0000 7fbb6780d640 1 Processor -- start 2023-08-09T14:51:25.017 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:25.014+0000 7fbb6780d640 1 -- start start 2023-08-09T14:51:25.022 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:25.015+0000 7fbb6780d640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fbb600b3fa0 0x7fbb6015ec30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T14:51:25.023 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:25.015+0000 7fbb6780d640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fbb600d0a70 0x7fbb6015f590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T14:51:25.023 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:25.015+0000 7fbb6780d640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fbb6015c680 0x7fbb60152340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T14:51:25.023 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:25.015+0000 7fbb6780d640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbb6015f210 con 0x7fbb6015c680 2023-08-09T14:51:25.024 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:25.015+0000 7fbb6780d640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbb60154c50 con 0x7fbb600b3fa0 2023-08-09T14:51:25.024 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:25.015+0000 7fbb6780d640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbb60152f30 con 0x7fbb600d0a70 2023-08-09T14:51:25.024 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:25.016+0000 7fbb65daa640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fbb6015c680 0x7fbb60152340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T14:51:25.024 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:25.016+0000 7fbb65daa640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fbb6015c680 0x7fbb60152340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:39738/0 (socket says 172.21.15.43:39738) 2023-08-09T14:51:25.025 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:25.016+0000 7fbb65daa640 1 -- 172.21.15.43:0/167226172 learned_addr learned my addr 172.21.15.43:0/167226172 (peer_addr_for_me v2:172.21.15.43:0/0) 2023-08-09T14:51:25.025 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:25.016+0000 7fbb655a9640 1 --2- 172.21.15.43:0/167226172 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fbb600b3fa0 0x7fbb6015ec30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T14:51:25.025 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:25.016+0000 7fbb64da8640 1 --2- 172.21.15.43:0/167226172 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fbb600d0a70 0x7fbb6015f590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T14:51:25.026 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:25.016+0000 7fbb65daa640 1 -- 172.21.15.43:0/167226172 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fbb600d0a70 msgr2=0x7fbb6015f590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T14:51:25.026 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:25.016+0000 7fbb65daa640 1 --2- 172.21.15.43:0/167226172 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fbb600d0a70 0x7fbb6015f590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T14:51:25.026 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:25.016+0000 7fbb65daa640 1 -- 172.21.15.43:0/167226172 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fbb600b3fa0 msgr2=0x7fbb6015ec30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T14:51:25.027 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:25.016+0000 7fbb65daa640 1 --2- 172.21.15.43:0/167226172 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fbb600b3fa0 0x7fbb6015ec30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T14:51:25.027 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:25.016+0000 7fbb65daa640 1 -- 172.21.15.43:0/167226172 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbb5c0094d0 con 0x7fbb6015c680 2023-08-09T14:51:25.027 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:25.017+0000 7fbb65daa640 1 --2- 172.21.15.43:0/167226172 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fbb6015c680 0x7fbb60152340 secure :-1 s=READY pgs=1755 cs=0 l=1 rev1=1 crypto rx=0x7fbb5800cef0 tx=0x7fbb5800a710 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-09T14:51:25.027 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:25.017+0000 7fbb55ffb640 1 -- 172.21.15.43:0/167226172 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbb5800d840 con 0x7fbb6015c680 2023-08-09T14:51:25.027 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:25.017+0000 7fbb55ffb640 1 -- 172.21.15.43:0/167226172 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbb5800d9c0 con 0x7fbb6015c680 2023-08-09T14:51:25.028 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:25.017+0000 7fbb55ffb640 1 -- 172.21.15.43:0/167226172 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbb5800dd70 con 0x7fbb6015c680 2023-08-09T14:51:25.028 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:25.017+0000 7fbb6780d640 1 -- 172.21.15.43:0/167226172 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbb60154320 con 0x7fbb6015c680 2023-08-09T14:51:25.028 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:25.017+0000 7fbb6780d640 1 -- 172.21.15.43:0/167226172 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbb601548e0 con 0x7fbb6015c680 2023-08-09T14:51:25.028 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:25.019+0000 7fbb6780d640 1 -- 172.21.15.43:0/167226172 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbb60154de0 con 0x7fbb6015c680 2023-08-09T14:51:25.028 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:25.021+0000 7fbb55ffb640 1 -- 172.21.15.43:0/167226172 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 75) v1 ==== 88455+0+0 (secure 0 0 0) 0x7fbb5801e3b0 con 0x7fbb6015c680 2023-08-09T14:51:25.029 INFO:tasks.ceph.mgr.x.smithi203.stderr:2023-08-09T14:51:25.023+0000 7fe0afe0f040 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2023-08-09T14:51:25.030 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:25.022+0000 7fbb55ffb640 1 --2- 172.21.15.43:0/167226172 >> [v2:172.21.15.203:6832/372694067,v1:172.21.15.203:6833/372694067] conn(0x7fbb4406a170 0x7fbb4406c630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T14:51:25.030 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:25.022+0000 7fbb55ffb640 1 -- 172.21.15.43:0/167226172 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(159..159 src has 1..159) v4 ==== 6916+0+0 (secure 0 0 0) 0x7fbb58089490 con 0x7fbb6015c680 2023-08-09T14:51:25.030 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:25.022+0000 7fbb655a9640 1 -- 172.21.15.43:0/167226172 >> [v2:172.21.15.203:6832/372694067,v1:172.21.15.203:6833/372694067] conn(0x7fbb4406a170 msgr2=0x7fbb4406c630 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.203:6832/372694067 2023-08-09T14:51:25.030 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:25.022+0000 7fbb655a9640 1 --2- 172.21.15.43:0/167226172 >> [v2:172.21.15.203:6832/372694067,v1:172.21.15.203:6833/372694067] conn(0x7fbb4406a170 0x7fbb4406c630 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-08-09T14:51:25.030 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:25.026+0000 7fbb55ffb640 1 -- 172.21.15.43:0/167226172 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178066 (secure 0 0 0) 0x7fbb58054890 con 0x7fbb6015c680 2023-08-09T14:51:25.069 INFO:tasks.ceph.mgr.y.smithi043.stderr:2023-08-09T14:51:25.068+0000 7f6a9a356040 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2023-08-09T14:51:25.074 INFO:tasks.ceph.mgr.x.smithi203.stderr:2023-08-09T14:51:25.073+0000 7fe0afe0f040 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2023-08-09T14:51:25.136 INFO:tasks.ceph.mgr.y.smithi043.stderr:2023-08-09T14:51:25.134+0000 7f6a9a356040 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2023-08-09T14:51:25.189 INFO:tasks.ceph.mgr.y.smithi043.stderr:2023-08-09T14:51:25.187+0000 7f6a9a356040 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2023-08-09T14:51:25.192 INFO:tasks.ceph.mgr.x.smithi203.stderr:2023-08-09T14:51:25.192+0000 7fe0afe0f040 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2023-08-09T14:51:25.212 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:25.210+0000 7fbb6780d640 1 -- 172.21.15.43:0/167226172 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7fbb600739f0 con 0x7fbb6015c680 2023-08-09T14:51:25.213 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:25.211+0000 7fbb55ffb640 1 -- 172.21.15.43:0/167226172 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v159) v1 ==== 104+0+11 (secure 0 0 0) 0x7fbb58057eb0 con 0x7fbb6015c680 2023-08-09T14:51:25.213 INFO:teuthology.orchestra.run.smithi043.stdout:pg_num: 32 2023-08-09T14:51:25.215 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:25.214+0000 7fbb6780d640 1 -- 172.21.15.43:0/167226172 >> [v2:172.21.15.203:6832/372694067,v1:172.21.15.203:6833/372694067] conn(0x7fbb4406a170 msgr2=0x7fbb4406c630 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-08-09T14:51:25.216 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:25.214+0000 7fbb6780d640 1 --2- 172.21.15.43:0/167226172 >> [v2:172.21.15.203:6832/372694067,v1:172.21.15.203:6833/372694067] conn(0x7fbb4406a170 0x7fbb4406c630 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T14:51:25.216 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:25.214+0000 7fbb6780d640 1 -- 172.21.15.43:0/167226172 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fbb6015c680 msgr2=0x7fbb60152340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T14:51:25.216 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:25.214+0000 7fbb6780d640 1 --2- 172.21.15.43:0/167226172 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fbb6015c680 0x7fbb60152340 secure :-1 s=READY pgs=1755 cs=0 l=1 rev1=1 crypto rx=0x7fbb5800cef0 tx=0x7fbb5800a710 comp rx=0 tx=0).stop 2023-08-09T14:51:25.216 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:25.214+0000 7fbb6780d640 1 -- 172.21.15.43:0/167226172 shutdown_connections 2023-08-09T14:51:25.216 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:25.214+0000 7fbb6780d640 1 --2- 172.21.15.43:0/167226172 >> [v2:172.21.15.203:6832/372694067,v1:172.21.15.203:6833/372694067] conn(0x7fbb4406a170 0x7fbb4406c630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T14:51:25.216 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:25.214+0000 7fbb6780d640 1 --2- 172.21.15.43:0/167226172 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fbb6015c680 0x7fbb60152340 unknown :-1 s=CLOSED pgs=1755 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T14:51:25.216 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:25.214+0000 7fbb6780d640 1 --2- 172.21.15.43:0/167226172 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fbb600d0a70 0x7fbb6015f590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T14:51:25.217 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:25.214+0000 7fbb6780d640 1 --2- 172.21.15.43:0/167226172 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fbb600b3fa0 0x7fbb6015ec30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T14:51:25.217 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:25.214+0000 7fbb6780d640 1 -- 172.21.15.43:0/167226172 >> 172.21.15.43:0/167226172 conn(0x7fbb60071520 msgr2=0x7fbb6015af90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-09T14:51:25.217 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:25.214+0000 7fbb6780d640 1 -- 172.21.15.43:0/167226172 shutdown_connections 2023-08-09T14:51:25.217 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:25.214+0000 7fbb6780d640 1 -- 172.21.15.43:0/167226172 wait complete. 2023-08-09T14:51:25.233 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-08-09T14:51:25.318 INFO:tasks.ceph.mgr.y.smithi043.stderr:2023-08-09T14:51:25.317+0000 7f6a9a356040 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2023-08-09T14:51:25.347 INFO:tasks.ceph.mgr.x.smithi203.stderr:2023-08-09T14:51:25.347+0000 7fe0afe0f040 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2023-08-09T14:51:25.439 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:25.437+0000 7f626b49f640 1 Processor -- start 2023-08-09T14:51:25.439 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:25.437+0000 7f626b49f640 1 -- start start 2023-08-09T14:51:25.440 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:25.439+0000 7f626b49f640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f626415a1b0 0x7f62640b1ad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T14:51:25.441 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:25.439+0000 7f626b49f640 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7f626415e9c0 0x7f62640b2010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T14:51:25.441 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:25.439+0000 7f626b49f640 1 -- --> v1:172.21.15.43:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f62640cc6c0 con 0x7f62640cd630 2023-08-09T14:51:25.441 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:25.439+0000 7f626b49f640 1 -- --> v2:172.21.15.203:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f626415dee0 con 0x7f626415e9c0 2023-08-09T14:51:25.441 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:25.439+0000 7f626b49f640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f626415e1c0 con 0x7f626415a1b0 2023-08-09T14:51:25.441 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:25.439+0000 7f6269c9c640 1 --1- >> v1:172.21.15.43:6789/0 conn(0x7f62640cd630 0x7f62640cdc20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.43:6789/0 says I am v1:172.21.15.43:54854/0 (socket says 172.21.15.43:54854) 2023-08-09T14:51:25.441 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:25.439+0000 7f6269c9c640 1 -- 172.21.15.43:0/1941900579 learned_addr learned my addr 172.21.15.43:0/1941900579 (peer_addr_for_me v1:172.21.15.43:0/0) 2023-08-09T14:51:25.443 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:25.440+0000 7f626a49d640 1 --2- 172.21.15.43:0/1941900579 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f626415a1b0 0x7f62640b1ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T14:51:25.443 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:25.440+0000 7f626ac9e640 1 --2- 172.21.15.43:0/1941900579 >> v2:172.21.15.203:3300/0 conn(0x7f626415e9c0 0x7f62640b2010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T14:51:25.443 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:25.440+0000 7f626949b640 1 -- 172.21.15.43:0/1941900579 <== mon.0 v1:172.21.15.43:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1047005773 0 0) 0x7f62640cc6c0 con 0x7f62640cd630 2023-08-09T14:51:25.444 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:25.440+0000 7f626949b640 1 -- 172.21.15.43:0/1941900579 --> v1:172.21.15.43:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f6250003520 con 0x7f62640cd630 2023-08-09T14:51:25.444 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:25.440+0000 7f626949b640 1 -- 172.21.15.43:0/1941900579 <== mon.0 v1:172.21.15.43:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3602323160 0 0) 0x7f6250003520 con 0x7f62640cd630 2023-08-09T14:51:25.444 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:25.440+0000 7f626949b640 1 -- 172.21.15.43:0/1941900579 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f626415a1b0 msgr2=0x7f62640b1ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-09T14:51:25.444 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:25.440+0000 7f626949b640 1 --2- 172.21.15.43:0/1941900579 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f626415a1b0 0x7f62640b1ad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T14:51:25.444 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:25.440+0000 7f626949b640 1 -- 172.21.15.43:0/1941900579 >> v2:172.21.15.203:3300/0 conn(0x7f626415e9c0 msgr2=0x7f62640b2010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-09T14:51:25.444 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:25.440+0000 7f626949b640 1 --2- 172.21.15.43:0/1941900579 >> v2:172.21.15.203:3300/0 conn(0x7f626415e9c0 0x7f62640b2010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T14:51:25.444 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:25.440+0000 7f626949b640 1 -- 172.21.15.43:0/1941900579 --> v1:172.21.15.43:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f626415e4a0 con 0x7f62640cd630 2023-08-09T14:51:25.445 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:25.440+0000 7f626949b640 1 -- 172.21.15.43:0/1941900579 <== mon.0 v1:172.21.15.43:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1285150740 0 0) 0x7f6254001670 con 0x7f62640cd630 2023-08-09T14:51:25.445 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:25.440+0000 7f626949b640 1 -- 172.21.15.43:0/1941900579 >> v1:172.21.15.43:6789/0 conn(0x7f62640cd630 legacy=0x7f62640cdc20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T14:51:25.445 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:25.440+0000 7f626949b640 1 --2- 172.21.15.43:0/1941900579 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f626415a1b0 0x7f6250003970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T14:51:25.445 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:25.440+0000 7f626949b640 1 --2- 172.21.15.43:0/1941900579 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f6250003ef0 0x7f62500047c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T14:51:25.445 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:25.440+0000 7f626949b640 1 --2- 172.21.15.43:0/1941900579 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f6250004d00 0x7f6250005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T14:51:25.445 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:25.441+0000 7f626949b640 1 -- 172.21.15.43:0/1941900579 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f626415dee0 con 0x7f6250004d00 2023-08-09T14:51:25.445 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:25.441+0000 7f626949b640 1 -- 172.21.15.43:0/1941900579 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f626415e1c0 con 0x7f6250003ef0 2023-08-09T14:51:25.446 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:25.441+0000 7f626949b640 1 -- 172.21.15.43:0/1941900579 --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f625000b980 con 0x7f626415a1b0 2023-08-09T14:51:25.446 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:25.441+0000 7f626ac9e640 1 --2- 172.21.15.43:0/1941900579 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f6250004d00 0x7f6250005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T14:51:25.446 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:25.441+0000 7f6269c9c640 1 --2- 172.21.15.43:0/1941900579 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f6250003ef0 0x7f62500047c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T14:51:25.446 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:25.441+0000 7f626a49d640 1 --2- 172.21.15.43:0/1941900579 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f626415a1b0 0x7f6250003970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T14:51:25.446 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:25.441+0000 7f626ac9e640 1 -- 172.21.15.43:0/1941900579 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f626415a1b0 msgr2=0x7f6250003970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-09T14:51:25.446 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:25.441+0000 7f626ac9e640 1 --2- 172.21.15.43:0/1941900579 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f626415a1b0 0x7f6250003970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T14:51:25.446 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:25.441+0000 7f626ac9e640 1 -- 172.21.15.43:0/1941900579 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f6250003ef0 msgr2=0x7f62500047c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-09T14:51:25.447 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:25.441+0000 7f626ac9e640 1 --2- 172.21.15.43:0/1941900579 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f6250003ef0 0x7f62500047c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T14:51:25.447 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:25.441+0000 7f626ac9e640 1 -- 172.21.15.43:0/1941900579 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6250003520 con 0x7f6250004d00 2023-08-09T14:51:25.448 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:25.443+0000 7f626ac9e640 1 --2- 172.21.15.43:0/1941900579 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f6250004d00 0x7f6250005180 secure :-1 s=READY pgs=1757 cs=0 l=1 rev1=1 crypto rx=0x7f625c009f90 tx=0x7f62500057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=15ee5c02cffc7a04 server_cookie=0 in_seq=0 out_seq=0 2023-08-09T14:51:25.448 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:25.443+0000 7f626949b640 1 -- 172.21.15.43:0/1941900579 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f625c02cc00 con 0x7f6250004d00 2023-08-09T14:51:25.448 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:25.444+0000 7f626949b640 1 -- 172.21.15.43:0/1941900579 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f625c00f040 con 0x7f6250004d00 2023-08-09T14:51:25.448 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:25.444+0000 7f626949b640 1 -- 172.21.15.43:0/1941900579 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f625c034440 con 0x7f6250004d00 2023-08-09T14:51:25.449 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:25.444+0000 7f626b49f640 1 -- 172.21.15.43:0/1941900579 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f6250004d00 msgr2=0x7f6250005180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T14:51:25.449 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:25.444+0000 7f626b49f640 1 --2- 172.21.15.43:0/1941900579 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f6250004d00 0x7f6250005180 secure :-1 s=READY pgs=1757 cs=0 l=1 rev1=1 crypto rx=0x7f625c009f90 tx=0x7f62500057f0 comp rx=0 tx=0).stop 2023-08-09T14:51:25.449 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:25.444+0000 7f626ac9e640 1 -- 172.21.15.43:0/1941900579 reap_dead start 2023-08-09T14:51:25.449 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:25.444+0000 7f626b49f640 1 -- 172.21.15.43:0/1941900579 shutdown_connections 2023-08-09T14:51:25.450 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:25.444+0000 7f626b49f640 1 -- 172.21.15.43:0/1941900579 >> 172.21.15.43:0/1941900579 conn(0x7f62640ccad0 msgr2=0x7f62640cceb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-09T14:51:25.450 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:25.444+0000 7f626b49f640 1 -- 172.21.15.43:0/1941900579 shutdown_connections 2023-08-09T14:51:25.450 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:25.445+0000 7f626b49f640 1 -- 172.21.15.43:0/1941900579 wait complete. 2023-08-09T14:51:25.450 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:25.446+0000 7f626b49f640 1 Processor -- start 2023-08-09T14:51:25.450 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:25.446+0000 7f626b49f640 1 -- start start 2023-08-09T14:51:25.453 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:25.447+0000 7f626b49f640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f6250004d00 0x7f62640c7df0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T14:51:25.454 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:25.447+0000 7f626b49f640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f62640c7120 0x7f62641565c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T14:51:25.454 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:25.447+0000 7f626b49f640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f626415be20 0x7f626415c2d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T14:51:25.454 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:25.447+0000 7f626b49f640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f626415c810 con 0x7f62640c7120 2023-08-09T14:51:25.454 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:25.447+0000 7f626b49f640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f62640c8330 con 0x7f6250004d00 2023-08-09T14:51:25.455 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:25.447+0000 7f626b49f640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6264156e10 con 0x7f626415be20 2023-08-09T14:51:25.455 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:25.448+0000 7f6269c9c640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f62640c7120 0x7f62641565c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T14:51:25.455 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:25.448+0000 7f6269c9c640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f62640c7120 0x7f62641565c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:39760/0 (socket says 172.21.15.43:39760) 2023-08-09T14:51:25.455 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:25.448+0000 7f6269c9c640 1 -- 172.21.15.43:0/760978086 learned_addr learned my addr 172.21.15.43:0/760978086 (peer_addr_for_me v2:172.21.15.43:0/0) 2023-08-09T14:51:25.455 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:25.448+0000 7f626a49d640 1 --2- 172.21.15.43:0/760978086 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f6250004d00 0x7f62640c7df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T14:51:25.456 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:25.448+0000 7f626ac9e640 1 --2- 172.21.15.43:0/760978086 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f626415be20 0x7f626415c2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T14:51:25.456 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:25.448+0000 7f6269c9c640 1 -- 172.21.15.43:0/760978086 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f626415be20 msgr2=0x7f626415c2d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T14:51:25.456 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:25.448+0000 7f6269c9c640 1 --2- 172.21.15.43:0/760978086 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f626415be20 0x7f626415c2d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T14:51:25.456 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:25.448+0000 7f6269c9c640 1 -- 172.21.15.43:0/760978086 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f6250004d00 msgr2=0x7f62640c7df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T14:51:25.457 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:25.448+0000 7f6269c9c640 1 --2- 172.21.15.43:0/760978086 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f6250004d00 0x7f62640c7df0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T14:51:25.457 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:25.448+0000 7f6269c9c640 1 -- 172.21.15.43:0/760978086 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f62641570f0 con 0x7f62640c7120 2023-08-09T14:51:25.457 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:25.448+0000 7f626a49d640 1 --2- 172.21.15.43:0/760978086 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f6250004d00 0x7f62640c7df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-08-09T14:51:25.457 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:25.448+0000 7f6269c9c640 1 --2- 172.21.15.43:0/760978086 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f62640c7120 0x7f62641565c0 secure :-1 s=READY pgs=1758 cs=0 l=1 rev1=1 crypto rx=0x7f6254007310 tx=0x7f62540077d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-09T14:51:25.457 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:25.449+0000 7f625affd640 1 -- 172.21.15.43:0/760978086 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f625400fa20 con 0x7f62640c7120 2023-08-09T14:51:25.458 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:25.449+0000 7f625affd640 1 -- 172.21.15.43:0/760978086 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f625400fb80 con 0x7f62640c7120 2023-08-09T14:51:25.458 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:25.449+0000 7f626b49f640 1 -- 172.21.15.43:0/760978086 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f62641573d0 con 0x7f62640c7120 2023-08-09T14:51:25.458 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:25.449+0000 7f625affd640 1 -- 172.21.15.43:0/760978086 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f62540133f0 con 0x7f62640c7120 2023-08-09T14:51:25.458 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:25.449+0000 7f626b49f640 1 -- 172.21.15.43:0/760978086 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f62640c88a0 con 0x7f62640c7120 2023-08-09T14:51:25.459 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:25.450+0000 7f626b49f640 1 -- 172.21.15.43:0/760978086 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f62640c8a60 con 0x7f62640c7120 2023-08-09T14:51:25.459 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:25.451+0000 7f625affd640 1 -- 172.21.15.43:0/760978086 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 75) v1 ==== 88455+0+0 (secure 0 0 0) 0x7f6254013550 con 0x7f62640c7120 2023-08-09T14:51:25.459 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:25.452+0000 7f625affd640 1 --2- 172.21.15.43:0/760978086 >> [v2:172.21.15.203:6832/372694067,v1:172.21.15.203:6833/372694067] conn(0x7f624806a170 0x7f624806c630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T14:51:25.459 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:25.452+0000 7f625affd640 1 -- 172.21.15.43:0/760978086 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(159..159 src has 1..159) v4 ==== 6916+0+0 (secure 0 0 0) 0x7f6254088e20 con 0x7f62640c7120 2023-08-09T14:51:25.459 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:25.452+0000 7f626a49d640 1 -- 172.21.15.43:0/760978086 >> [v2:172.21.15.203:6832/372694067,v1:172.21.15.203:6833/372694067] conn(0x7f624806a170 msgr2=0x7f624806c630 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.203:6832/372694067 2023-08-09T14:51:25.460 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:25.452+0000 7f626a49d640 1 --2- 172.21.15.43:0/760978086 >> [v2:172.21.15.203:6832/372694067,v1:172.21.15.203:6833/372694067] conn(0x7f624806a170 0x7f624806c630 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-08-09T14:51:25.460 INFO:tasks.ceph.mgr.x.smithi203.stderr:2023-08-09T14:51:25.455+0000 7fe0afe0f040 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2023-08-09T14:51:25.461 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:25.456+0000 7f625affd640 1 -- 172.21.15.43:0/760978086 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178066 (secure 0 0 0) 0x7f6254054000 con 0x7f62640c7120 2023-08-09T14:51:25.489 INFO:tasks.ceph.mgr.y.smithi043.stderr:2023-08-09T14:51:25.487+0000 7f6a9a356040 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2023-08-09T14:51:25.505 INFO:tasks.ceph.mgr.x.smithi203.stderr:2023-08-09T14:51:25.504+0000 7fe0afe0f040 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2023-08-09T14:51:25.602 INFO:tasks.ceph.mgr.y.smithi043.stderr:2023-08-09T14:51:25.601+0000 7f6a9a356040 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2023-08-09T14:51:25.646 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:25.645+0000 7f626b49f640 1 -- 172.21.15.43:0/760978086 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f62640c8700 con 0x7f62640c7120 2023-08-09T14:51:25.647 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:25.646+0000 7f625affd640 1 -- 172.21.15.43:0/760978086 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v159) v1 ==== 100+0+11 (secure 0 0 0) 0x7f62640c8700 con 0x7f62640c7120 2023-08-09T14:51:25.647 INFO:teuthology.orchestra.run.smithi043.stdout:pg_num: 64 2023-08-09T14:51:25.650 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:25.648+0000 7f626b49f640 1 -- 172.21.15.43:0/760978086 >> [v2:172.21.15.203:6832/372694067,v1:172.21.15.203:6833/372694067] conn(0x7f624806a170 msgr2=0x7f624806c630 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-08-09T14:51:25.650 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:25.648+0000 7f626b49f640 1 --2- 172.21.15.43:0/760978086 >> [v2:172.21.15.203:6832/372694067,v1:172.21.15.203:6833/372694067] conn(0x7f624806a170 0x7f624806c630 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T14:51:25.650 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:25.648+0000 7f626b49f640 1 -- 172.21.15.43:0/760978086 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f62640c7120 msgr2=0x7f62641565c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T14:51:25.650 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:25.648+0000 7f626b49f640 1 --2- 172.21.15.43:0/760978086 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f62640c7120 0x7f62641565c0 secure :-1 s=READY pgs=1758 cs=0 l=1 rev1=1 crypto rx=0x7f6254007310 tx=0x7f62540077d0 comp rx=0 tx=0).stop 2023-08-09T14:51:25.650 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:25.648+0000 7f626b49f640 1 -- 172.21.15.43:0/760978086 shutdown_connections 2023-08-09T14:51:25.651 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:25.648+0000 7f626b49f640 1 --2- 172.21.15.43:0/760978086 >> [v2:172.21.15.203:6832/372694067,v1:172.21.15.203:6833/372694067] conn(0x7f624806a170 0x7f624806c630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T14:51:25.651 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:25.648+0000 7f626b49f640 1 --2- 172.21.15.43:0/760978086 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f626415be20 0x7f626415c2d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T14:51:25.651 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:25.648+0000 7f626b49f640 1 --2- 172.21.15.43:0/760978086 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f62640c7120 0x7f62641565c0 unknown :-1 s=CLOSED pgs=1758 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T14:51:25.651 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:25.648+0000 7f626b49f640 1 --2- 172.21.15.43:0/760978086 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f6250004d00 0x7f62640c7df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T14:51:25.651 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:25.648+0000 7f626b49f640 1 -- 172.21.15.43:0/760978086 >> 172.21.15.43:0/760978086 conn(0x7f62640ccad0 msgr2=0x7f62640b1790 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-09T14:51:25.651 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:25.648+0000 7f626b49f640 1 -- 172.21.15.43:0/760978086 shutdown_connections 2023-08-09T14:51:25.651 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:25.649+0000 7f626b49f640 1 -- 172.21.15.43:0/760978086 wait complete. 2023-08-09T14:51:25.655 INFO:tasks.ceph.mgr.y.smithi043.stderr:2023-08-09T14:51:25.654+0000 7f6a9a356040 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2023-08-09T14:51:25.666 INFO:tasks.ceph.mgr.x.smithi203.stderr:2023-08-09T14:51:25.665+0000 7fe0afe0f040 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2023-08-09T14:51:25.668 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2023-08-09T14:51:25.827 INFO:tasks.ceph.mgr.y.smithi043.stderr:2023-08-09T14:51:25.825+0000 7f6a9a356040 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2023-08-09T14:51:25.876 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:25.875+0000 7f0c1ac2c640 1 Processor -- start 2023-08-09T14:51:25.877 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:25.875+0000 7f0c1ac2c640 1 -- start start 2023-08-09T14:51:25.878 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:25.877+0000 7f0c1ac2c640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f0c140704c0 0x7f0c141670f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T14:51:25.879 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:25.877+0000 7f0c1ac2c640 1 -- --> v1:172.21.15.43:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0c14071620 con 0x7f0c14069ed0 2023-08-09T14:51:25.879 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:25.877+0000 7f0c1ac2c640 1 -- --> v1:172.21.15.203:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0c14071ab0 con 0x7f0c14142b40 2023-08-09T14:51:25.879 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:25.877+0000 7f0c1ac2c640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0c14142f20 con 0x7f0c140704c0 2023-08-09T14:51:25.879 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:25.877+0000 7f0c19429640 1 --1- >> v1:172.21.15.43:6789/0 conn(0x7f0c14069ed0 0x7f0c1406c960 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.43:6789/0 says I am v1:172.21.15.43:54860/0 (socket says 172.21.15.43:54860) 2023-08-09T14:51:25.879 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:25.877+0000 7f0c19429640 1 -- 172.21.15.43:0/2436044618 learned_addr learned my addr 172.21.15.43:0/2436044618 (peer_addr_for_me v1:172.21.15.43:0/0) 2023-08-09T14:51:25.881 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:25.877+0000 7f0c1a42b640 1 --2- 172.21.15.43:0/2436044618 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f0c140704c0 0x7f0c141670f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T14:51:25.881 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:25.878+0000 7f0c18c28640 1 -- 172.21.15.43:0/2436044618 <== mon.0 v1:172.21.15.43:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2692030129 0 0) 0x7f0c14071620 con 0x7f0c14069ed0 2023-08-09T14:51:25.881 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:25.878+0000 7f0c18c28640 1 -- 172.21.15.43:0/2436044618 --> v1:172.21.15.43:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f0c00003520 con 0x7f0c14069ed0 2023-08-09T14:51:25.882 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:25.878+0000 7f0c18c28640 1 -- 172.21.15.43:0/2436044618 <== mon.1 v1:172.21.15.203:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4158963615 0 0) 0x7f0c14071ab0 con 0x7f0c14142b40 2023-08-09T14:51:25.882 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:25.878+0000 7f0c18c28640 1 -- 172.21.15.43:0/2436044618 --> v1:172.21.15.203:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f0c14071620 con 0x7f0c14142b40 2023-08-09T14:51:25.882 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:25.878+0000 7f0c18c28640 1 -- 172.21.15.43:0/2436044618 <== mon.0 v1:172.21.15.43:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3333046479 0 0) 0x7f0c00003520 con 0x7f0c14069ed0 2023-08-09T14:51:25.882 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:25.878+0000 7f0c18c28640 1 -- 172.21.15.43:0/2436044618 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f0c140704c0 msgr2=0x7f0c141670f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-09T14:51:25.882 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:25.878+0000 7f0c18c28640 1 --2- 172.21.15.43:0/2436044618 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f0c140704c0 0x7f0c141670f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T14:51:25.882 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:25.878+0000 7f0c18c28640 1 -- 172.21.15.43:0/2436044618 >> v1:172.21.15.203:6789/0 conn(0x7f0c14142b40 legacy=0x7f0c140697c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T14:51:25.882 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:25.878+0000 7f0c18c28640 1 -- 172.21.15.43:0/2436044618 --> v1:172.21.15.43:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0c14070a10 con 0x7f0c14069ed0 2023-08-09T14:51:25.883 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:25.878+0000 7f0c18c28640 1 -- 172.21.15.43:0/2436044618 <== mon.0 v1:172.21.15.43:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1285150740 0 0) 0x7f0c04001670 con 0x7f0c14069ed0 2023-08-09T14:51:25.883 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:25.878+0000 7f0c18c28640 1 -- 172.21.15.43:0/2436044618 >> v1:172.21.15.43:6789/0 conn(0x7f0c14069ed0 legacy=0x7f0c1406c960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T14:51:25.883 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:25.878+0000 7f0c18c28640 1 --2- 172.21.15.43:0/2436044618 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f0c00003b30 0x7f0c00003f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T14:51:25.883 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:25.878+0000 7f0c18c28640 1 --2- 172.21.15.43:0/2436044618 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f0c000047c0 0x7f0c00004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T14:51:25.883 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:25.878+0000 7f0c18c28640 1 --2- 172.21.15.43:0/2436044618 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f0c00005180 0x7f0c0000b980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T14:51:25.883 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:25.878+0000 7f0c18c28640 1 -- 172.21.15.43:0/2436044618 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0c14142f20 con 0x7f0c00005180 2023-08-09T14:51:25.883 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:25.878+0000 7f0c18c28640 1 -- 172.21.15.43:0/2436044618 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0c0000bff0 con 0x7f0c000047c0 2023-08-09T14:51:25.884 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:25.878+0000 7f0c18c28640 1 -- 172.21.15.43:0/2436044618 --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0c0000c1b0 con 0x7f0c00003b30 2023-08-09T14:51:25.884 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:25.879+0000 7f0c1a42b640 1 --2- 172.21.15.43:0/2436044618 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f0c00005180 0x7f0c0000b980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T14:51:25.884 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:25.879+0000 7f0c19429640 1 --2- 172.21.15.43:0/2436044618 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f0c000047c0 0x7f0c00004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T14:51:25.884 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:25.879+0000 7f0c19c2a640 1 --2- 172.21.15.43:0/2436044618 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f0c00003b30 0x7f0c00003f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T14:51:25.884 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:25.879+0000 7f0c1a42b640 1 -- 172.21.15.43:0/2436044618 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f0c00003b30 msgr2=0x7f0c00003f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-09T14:51:25.884 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:25.879+0000 7f0c1a42b640 1 --2- 172.21.15.43:0/2436044618 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f0c00003b30 0x7f0c00003f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T14:51:25.884 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:25.879+0000 7f0c1a42b640 1 -- 172.21.15.43:0/2436044618 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f0c000047c0 msgr2=0x7f0c00004c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-09T14:51:25.885 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:25.879+0000 7f0c1a42b640 1 --2- 172.21.15.43:0/2436044618 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f0c000047c0 0x7f0c00004c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T14:51:25.885 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:25.879+0000 7f0c1a42b640 1 -- 172.21.15.43:0/2436044618 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0c00003520 con 0x7f0c00005180 2023-08-09T14:51:25.885 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:25.879+0000 7f0c19429640 1 --2- 172.21.15.43:0/2436044618 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f0c000047c0 0x7f0c00004c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-08-09T14:51:25.886 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:25.880+0000 7f0c1a42b640 1 --2- 172.21.15.43:0/2436044618 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f0c00005180 0x7f0c0000b980 secure :-1 s=READY pgs=1760 cs=0 l=1 rev1=1 crypto rx=0x7f0c140708a0 tx=0x7f0c000057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d2afb7da51e35d43 server_cookie=0 in_seq=0 out_seq=0 2023-08-09T14:51:25.886 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:25.881+0000 7f0c18c28640 1 -- 172.21.15.43:0/2436044618 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0c0c02cb70 con 0x7f0c00005180 2023-08-09T14:51:25.886 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:25.881+0000 7f0c18c28640 1 -- 172.21.15.43:0/2436044618 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0c0c00f040 con 0x7f0c00005180 2023-08-09T14:51:25.887 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:25.881+0000 7f0c18c28640 1 -- 172.21.15.43:0/2436044618 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0c0c034440 con 0x7f0c00005180 2023-08-09T14:51:25.887 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:25.881+0000 7f0c1ac2c640 1 -- 172.21.15.43:0/2436044618 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f0c00005180 msgr2=0x7f0c0000b980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T14:51:25.887 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:25.881+0000 7f0c1ac2c640 1 --2- 172.21.15.43:0/2436044618 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f0c00005180 0x7f0c0000b980 secure :-1 s=READY pgs=1760 cs=0 l=1 rev1=1 crypto rx=0x7f0c140708a0 tx=0x7f0c000057f0 comp rx=0 tx=0).stop 2023-08-09T14:51:25.887 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:25.881+0000 7f0c1a42b640 1 -- 172.21.15.43:0/2436044618 reap_dead start 2023-08-09T14:51:25.887 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:25.882+0000 7f0c1ac2c640 1 -- 172.21.15.43:0/2436044618 shutdown_connections 2023-08-09T14:51:25.887 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:25.882+0000 7f0c1ac2c640 1 -- 172.21.15.43:0/2436044618 >> 172.21.15.43:0/2436044618 conn(0x7f0c140b0c50 msgr2=0x7f0c140b1060 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-09T14:51:25.888 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:25.882+0000 7f0c1ac2c640 1 -- 172.21.15.43:0/2436044618 shutdown_connections 2023-08-09T14:51:25.888 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:25.882+0000 7f0c1ac2c640 1 -- 172.21.15.43:0/2436044618 wait complete. 2023-08-09T14:51:25.888 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:25.884+0000 7f0c1ac2c640 1 Processor -- start 2023-08-09T14:51:25.888 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:25.885+0000 7f0c1ac2c640 1 -- start start 2023-08-09T14:51:25.889 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:25.886+0000 7f0c1ac2c640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f0c00005180 0x7f0c1413faa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T14:51:25.890 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:25.886+0000 7f0c1ac2c640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f0c140c7d70 0x7f0c141f18e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T14:51:25.890 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:25.886+0000 7f0c1ac2c640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f0c140af360 0x7f0c140af7e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T14:51:25.890 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:25.886+0000 7f0c1ac2c640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0c140afd20 con 0x7f0c00005180 2023-08-09T14:51:25.890 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:25.886+0000 7f0c1ac2c640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0c141e4070 con 0x7f0c140c7d70 2023-08-09T14:51:25.890 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:25.886+0000 7f0c1ac2c640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0c141f2810 con 0x7f0c140af360 2023-08-09T14:51:25.890 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:25.886+0000 7f0c19c2a640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f0c00005180 0x7f0c1413faa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T14:51:25.891 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:25.886+0000 7f0c19c2a640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f0c00005180 0x7f0c1413faa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:39774/0 (socket says 172.21.15.43:39774) 2023-08-09T14:51:25.891 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:25.887+0000 7f0c19c2a640 1 -- 172.21.15.43:0/1904586344 learned_addr learned my addr 172.21.15.43:0/1904586344 (peer_addr_for_me v2:172.21.15.43:0/0) 2023-08-09T14:51:25.891 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:25.887+0000 7f0c19429640 1 --2- 172.21.15.43:0/1904586344 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f0c140c7d70 0x7f0c141f18e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T14:51:25.891 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:25.887+0000 7f0c1a42b640 1 --2- 172.21.15.43:0/1904586344 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f0c140af360 0x7f0c140af7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T14:51:25.891 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:25.887+0000 7f0c19c2a640 1 -- 172.21.15.43:0/1904586344 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f0c140af360 msgr2=0x7f0c140af7e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T14:51:25.891 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:25.887+0000 7f0c19c2a640 1 --2- 172.21.15.43:0/1904586344 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f0c140af360 0x7f0c140af7e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T14:51:25.892 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:25.887+0000 7f0c19c2a640 1 -- 172.21.15.43:0/1904586344 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f0c140c7d70 msgr2=0x7f0c141f18e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T14:51:25.892 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:25.887+0000 7f0c19c2a640 1 --2- 172.21.15.43:0/1904586344 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f0c140c7d70 0x7f0c141f18e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T14:51:25.892 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:25.887+0000 7f0c19c2a640 1 -- 172.21.15.43:0/1904586344 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0c141f2250 con 0x7f0c00005180 2023-08-09T14:51:25.892 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:25.887+0000 7f0c19c2a640 1 --2- 172.21.15.43:0/1904586344 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f0c00005180 0x7f0c1413faa0 secure :-1 s=READY pgs=1761 cs=0 l=1 rev1=1 crypto rx=0x7f0c1000f770 tx=0x7f0c1000fc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-09T14:51:25.892 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:25.888+0000 7f0c0a7fc640 1 -- 172.21.15.43:0/1904586344 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0c100014a0 con 0x7f0c00005180 2023-08-09T14:51:25.892 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:25.888+0000 7f0c0a7fc640 1 -- 172.21.15.43:0/1904586344 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0c1000fe80 con 0x7f0c00005180 2023-08-09T14:51:25.892 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:25.888+0000 7f0c0a7fc640 1 -- 172.21.15.43:0/1904586344 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0c1000d730 con 0x7f0c00005180 2023-08-09T14:51:25.892 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:25.888+0000 7f0c1ac2c640 1 -- 172.21.15.43:0/1904586344 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0c141f2530 con 0x7f0c00005180 2023-08-09T14:51:25.893 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:25.888+0000 7f0c1ac2c640 1 -- 172.21.15.43:0/1904586344 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0c141f1e20 con 0x7f0c00005180 2023-08-09T14:51:25.893 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:25.889+0000 7f0c1ac2c640 1 -- 172.21.15.43:0/1904586344 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0c141462c0 con 0x7f0c00005180 2023-08-09T14:51:25.894 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:25.890+0000 7f0c0a7fc640 1 -- 172.21.15.43:0/1904586344 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 75) v1 ==== 88455+0+0 (secure 0 0 0) 0x7f0c1000d890 con 0x7f0c00005180 2023-08-09T14:51:25.894 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:25.891+0000 7f0c0a7fc640 1 --2- 172.21.15.43:0/1904586344 >> [v2:172.21.15.203:6832/372694067,v1:172.21.15.203:6833/372694067] conn(0x7f0bf806a1a0 0x7f0bf806c660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T14:51:25.894 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:25.891+0000 7f0c0a7fc640 1 -- 172.21.15.43:0/1904586344 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(159..159 src has 1..159) v4 ==== 6916+0+0 (secure 0 0 0) 0x7f0c10088ac0 con 0x7f0c00005180 2023-08-09T14:51:25.894 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:25.891+0000 7f0c19429640 1 -- 172.21.15.43:0/1904586344 >> [v2:172.21.15.203:6832/372694067,v1:172.21.15.203:6833/372694067] conn(0x7f0bf806a1a0 msgr2=0x7f0bf806c660 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.203:6832/372694067 2023-08-09T14:51:25.894 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:25.891+0000 7f0c19429640 1 --2- 172.21.15.43:0/1904586344 >> [v2:172.21.15.203:6832/372694067,v1:172.21.15.203:6833/372694067] conn(0x7f0bf806a1a0 0x7f0bf806c660 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-08-09T14:51:25.897 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:25.895+0000 7f0c0a7fc640 1 -- 172.21.15.43:0/1904586344 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178066 (secure 0 0 0) 0x7f0c10053d30 con 0x7f0c00005180 2023-08-09T14:51:26.051 INFO:tasks.ceph.mgr.x.smithi203.stderr:2023-08-09T14:51:26.051+0000 7fe0afe0f040 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2023-08-09T14:51:26.094 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:26.092+0000 7f0c19429640 1 -- 172.21.15.43:0/1904586344 >> [v2:172.21.15.203:6832/372694067,v1:172.21.15.203:6833/372694067] conn(0x7f0bf806a1a0 msgr2=0x7f0bf806c660 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.203:6832/372694067 2023-08-09T14:51:26.096 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:26.092+0000 7f0c19429640 1 --2- 172.21.15.43:0/1904586344 >> [v2:172.21.15.203:6832/372694067,v1:172.21.15.203:6833/372694067] conn(0x7f0bf806a1a0 0x7f0bf806c660 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2023-08-09T14:51:26.102 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:26.100+0000 7f0c1ac2c640 1 -- 172.21.15.43:0/1904586344 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7f0c141f1fe0 con 0x7f0c00005180 2023-08-09T14:51:26.102 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:26.101+0000 7f0c0a7fc640 1 -- 172.21.15.43:0/1904586344 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 8 entries v159) v1 ==== 80+0+477 (secure 0 0 0) 0x7f0c10057350 con 0x7f0c00005180 2023-08-09T14:51:26.103 INFO:teuthology.orchestra.run.smithi043.stdout:172.21.15.203:0/2901879784 2023-08-10T13:55:44.498027+0000 2023-08-09T14:51:26.103 INFO:teuthology.orchestra.run.smithi043.stdout:172.21.15.203:0/1501229236 2023-08-10T13:55:44.498027+0000 2023-08-09T14:51:26.103 INFO:teuthology.orchestra.run.smithi043.stdout:172.21.15.203:0/3615051942 2023-08-10T13:55:44.498027+0000 2023-08-09T14:51:26.103 INFO:teuthology.orchestra.run.smithi043.stdout:172.21.15.203:6833/3490266242 2023-08-10T13:55:44.498027+0000 2023-08-09T14:51:26.103 INFO:teuthology.orchestra.run.smithi043.stdout:172.21.15.203:6832/3490266242 2023-08-10T13:55:44.498027+0000 2023-08-09T14:51:26.104 INFO:teuthology.orchestra.run.smithi043.stdout:172.21.15.203:0/3892021074 2023-08-10T13:55:44.498027+0000 2023-08-09T14:51:26.104 INFO:teuthology.orchestra.run.smithi043.stdout:172.21.15.203:0/750810033 2023-08-10T13:55:44.498027+0000 2023-08-09T14:51:26.104 INFO:teuthology.orchestra.run.smithi043.stdout:172.21.15.203:0/1164133940 2023-08-10T13:55:44.498027+0000 2023-08-09T14:51:26.104 INFO:teuthology.orchestra.run.smithi043.stderr:listed 8 entries 2023-08-09T14:51:26.105 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:26.103+0000 7f0c1ac2c640 1 -- 172.21.15.43:0/1904586344 >> [v2:172.21.15.203:6832/372694067,v1:172.21.15.203:6833/372694067] conn(0x7f0bf806a1a0 msgr2=0x7f0bf806c660 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-08-09T14:51:26.105 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:26.103+0000 7f0c1ac2c640 1 --2- 172.21.15.43:0/1904586344 >> [v2:172.21.15.203:6832/372694067,v1:172.21.15.203:6833/372694067] conn(0x7f0bf806a1a0 0x7f0bf806c660 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T14:51:26.105 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:26.103+0000 7f0c1ac2c640 1 -- 172.21.15.43:0/1904586344 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f0c00005180 msgr2=0x7f0c1413faa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T14:51:26.105 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:26.103+0000 7f0c1ac2c640 1 --2- 172.21.15.43:0/1904586344 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f0c00005180 0x7f0c1413faa0 secure :-1 s=READY pgs=1761 cs=0 l=1 rev1=1 crypto rx=0x7f0c1000f770 tx=0x7f0c1000fc30 comp rx=0 tx=0).stop 2023-08-09T14:51:26.106 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:26.103+0000 7f0c1ac2c640 1 -- 172.21.15.43:0/1904586344 shutdown_connections 2023-08-09T14:51:26.106 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:26.103+0000 7f0c1ac2c640 1 --2- 172.21.15.43:0/1904586344 >> [v2:172.21.15.203:6832/372694067,v1:172.21.15.203:6833/372694067] conn(0x7f0bf806a1a0 0x7f0bf806c660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T14:51:26.106 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:26.103+0000 7f0c1ac2c640 1 --2- 172.21.15.43:0/1904586344 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f0c140af360 0x7f0c140af7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T14:51:26.106 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:26.103+0000 7f0c1ac2c640 1 --2- 172.21.15.43:0/1904586344 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f0c140c7d70 0x7f0c141f18e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T14:51:26.106 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:26.103+0000 7f0c1ac2c640 1 --2- 172.21.15.43:0/1904586344 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f0c00005180 0x7f0c1413faa0 unknown :-1 s=CLOSED pgs=1761 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T14:51:26.106 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:26.103+0000 7f0c1ac2c640 1 -- 172.21.15.43:0/1904586344 >> 172.21.15.43:0/1904586344 conn(0x7f0c140b0c50 msgr2=0x7f0c141419b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-09T14:51:26.106 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:26.104+0000 7f0c1ac2c640 1 -- 172.21.15.43:0/1904586344 shutdown_connections 2023-08-09T14:51:26.107 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:26.104+0000 7f0c1ac2c640 1 -- 172.21.15.43:0/1904586344 wait complete. 2023-08-09T14:51:26.123 INFO:tasks.cephfs.fuse_mount:Running fusermount -u on ubuntu@smithi043.front.sepia.ceph.com... 2023-08-09T14:51:26.123 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-09T14:51:26.124 DEBUG:teuthology.orchestra.run.smithi043:> sudo fusermount -u /home/ubuntu/cephtest/mnt.0 2023-08-09T14:51:26.209 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi043.stderr:ceph-fuse[133240]: fuse finished with error 0 and tester_r 0 2023-08-09T14:51:26.215 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi043.front.sepia.ceph.com 2023-08-09T14:51:26.216 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-09T14:51:26.216 DEBUG:teuthology.orchestra.run.smithi043:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2023-08-09T14:51:26.249 INFO:tasks.ceph.mgr.y.smithi043.stderr:2023-08-09T14:51:26.247+0000 7f6a9a356040 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2023-08-09T14:51:26.273 INFO:tasks.ceph.mgr.x.smithi203.stderr:2023-08-09T14:51:26.273+0000 7fe0afe0f040 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2023-08-09T14:51:26.330 INFO:tasks.ceph.mgr.x.smithi203.stderr:2023-08-09T14:51:26.330+0000 7fe0afe0f040 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2023-08-09T14:51:26.380 INFO:tasks.ceph.mgr.x.smithi203.stderr:2023-08-09T14:51:26.380+0000 7fe0afe0f040 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2023-08-09T14:51:26.390 DEBUG:teuthology.orchestra.run.smithi203:> set -ex 2023-08-09T14:51:26.391 DEBUG:teuthology.orchestra.run.smithi203:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-09T14:51:26.434 INFO:tasks.ceph.mgr.x.smithi203.stderr:2023-08-09T14:51:26.434+0000 7fe0afe0f040 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2023-08-09T14:51:26.449 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4053648,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497576k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=27,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248784k,nr_inodes=812196,mode=700,uid=1000,gid=1275 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497576k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2023-08-09T14:51:26.449 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi203.front.sepia.ceph.com 2023-08-09T14:51:26.450 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-09T14:51:26.450 DEBUG:teuthology.orchestra.run.smithi203:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.1) 2023-08-09T14:51:26.480 INFO:tasks.ceph.mgr.y.smithi043.stderr:2023-08-09T14:51:26.479+0000 7f6a9a356040 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2023-08-09T14:51:26.541 INFO:tasks.ceph.mgr.y.smithi043.stderr:2023-08-09T14:51:26.540+0000 7f6a9a356040 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2023-08-09T14:51:26.544 INFO:teuthology.orchestra.run.smithi203.stderr:rmdir: failed to remove '/home/ubuntu/cephtest/mnt.1': No such file or directory 2023-08-09T14:51:26.545 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-08-09T14:51:26.546 DEBUG:teuthology.orchestra.run.smithi043:> set -ex 2023-08-09T14:51:26.546 DEBUG:teuthology.orchestra.run.smithi043:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-09T14:51:26.594 INFO:tasks.ceph.mgr.y.smithi043.stderr:2023-08-09T14:51:26.593+0000 7f6a9a356040 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2023-08-09T14:51:26.603 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4054441,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6498844k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=27,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3249420k,nr_inodes=812355,mode=700,uid=1000,gid=1275 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6498844k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2023-08-09T14:51:26.603 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi043.front.sepia.ceph.com 2023-08-09T14:51:26.604 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-09T14:51:26.604 DEBUG:teuthology.orchestra.run.smithi043:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.2) 2023-08-09T14:51:26.652 INFO:tasks.ceph.mgr.y.smithi043.stderr:2023-08-09T14:51:26.648+0000 7f6a9a356040 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2023-08-09T14:51:26.701 INFO:teuthology.orchestra.run.smithi043.stderr:rmdir: failed to remove '/home/ubuntu/cephtest/mnt.2': No such file or directory 2023-08-09T14:51:26.702 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-08-09T14:51:26.703 DEBUG:teuthology.orchestra.run.smithi203:> set -ex 2023-08-09T14:51:26.703 DEBUG:teuthology.orchestra.run.smithi203:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-09T14:51:26.744 INFO:tasks.ceph.mgr.x.smithi203.stderr:2023-08-09T14:51:26.744+0000 7fe0afe0f040 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2023-08-09T14:51:26.759 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4053648,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497576k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=27,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248784k,nr_inodes=812196,mode=700,uid=1000,gid=1275 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497576k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2023-08-09T14:51:26.759 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi203.front.sepia.ceph.com 2023-08-09T14:51:26.759 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-09T14:51:26.760 DEBUG:teuthology.orchestra.run.smithi203:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.3) 2023-08-09T14:51:26.808 INFO:tasks.ceph.mgr.x.smithi203.stderr:2023-08-09T14:51:26.804+0000 7fe0afe0f040 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2023-08-09T14:51:26.853 INFO:teuthology.orchestra.run.smithi203.stderr:rmdir: failed to remove '/home/ubuntu/cephtest/mnt.3': No such file or directory 2023-08-09T14:51:26.854 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-08-09T14:51:26.855 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-08-09T14:51:26.859 INFO:tasks.ceph.mgr.x.smithi203.stderr:2023-08-09T14:51:26.859+0000 7fe0afe0f040 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2023-08-09T14:51:26.960 INFO:tasks.ceph.mgr.y.smithi043.stderr:2023-08-09T14:51:26.959+0000 7f6a9a356040 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2023-08-09T14:51:27.022 INFO:tasks.ceph.mgr.x.smithi203.stderr:2023-08-09T14:51:27.021+0000 7fe0afe0f040 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2023-08-09T14:51:27.022 INFO:tasks.ceph.mgr.y.smithi043.stderr:2023-08-09T14:51:27.021+0000 7f6a9a356040 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2023-08-09T14:51:27.071 INFO:tasks.ceph.mgr.x.smithi203.stderr:2023-08-09T14:51:27.070+0000 7fe0afe0f040 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2023-08-09T14:51:27.078 INFO:tasks.ceph.mgr.y.smithi043.stderr:2023-08-09T14:51:27.077+0000 7f6a9a356040 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2023-08-09T14:51:27.079 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:27.078+0000 7f751051c640 1 Processor -- start 2023-08-09T14:51:27.080 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:27.078+0000 7f751051c640 1 -- start start 2023-08-09T14:51:27.081 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:27.080+0000 7f751051c640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f750815c550 0x7f75080d1a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T14:51:27.081 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:27.080+0000 7f751051c640 1 -- --> v1:172.21.15.43:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f75080d0700 con 0x7f75080cf550 2023-08-09T14:51:27.081 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:27.080+0000 7f751051c640 1 -- --> v1:172.21.15.203:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f75080d09e0 con 0x7f750815c930 2023-08-09T14:51:27.081 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:27.080+0000 7f751051c640 1 -- --> v2:172.21.15.43:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f75080d0cc0 con 0x7f750815c550 2023-08-09T14:51:27.082 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:27.080+0000 7f750e2b8640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f750815c550 0x7f75080d1a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T14:51:27.082 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:27.080+0000 7f750e2b8640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f750815c550 0x7f75080d1a80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:39778/0 (socket says 172.21.15.43:39778) 2023-08-09T14:51:27.083 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:27.080+0000 7f750eab9640 1 --1- >> v1:172.21.15.43:6789/0 conn(0x7f75080cf550 0x7f75080b5460 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.43:6789/0 says I am v1:172.21.15.43:54876/0 (socket says 172.21.15.43:54876) 2023-08-09T14:51:27.083 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:27.080+0000 7f750e2b8640 1 -- 172.21.15.43:0/2447666415 learned_addr learned my addr 172.21.15.43:0/2447666415 (peer_addr_for_me v2:172.21.15.43:0/0) 2023-08-09T14:51:27.083 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:27.081+0000 7f750e2b8640 1 -- 172.21.15.43:0/2447666415 >> v1:172.21.15.203:6789/0 conn(0x7f750815c930 legacy=0x7f75080cee40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-09T14:51:27.083 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:27.081+0000 7f750e2b8640 1 -- 172.21.15.43:0/2447666415 >> v1:172.21.15.43:6789/0 conn(0x7f75080cf550 legacy=0x7f75080b5460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T14:51:27.083 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:27.081+0000 7f750d2b6640 1 -- 172.21.15.43:0/2447666415 <== mon.0 v1:172.21.15.43:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 506769241 0 0) 0x7f75080d0700 con 0x7f75080cf550 2023-08-09T14:51:27.083 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:27.081+0000 7f750e2b8640 1 -- 172.21.15.43:0/2447666415 --> v2:172.21.15.43:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f75080d0fa0 con 0x7f750815c550 2023-08-09T14:51:27.084 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:27.082+0000 7f750e2b8640 1 --2- 172.21.15.43:0/2447666415 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f750815c550 0x7f75080d1a80 secure :-1 s=READY pgs=1763 cs=0 l=1 rev1=1 crypto rx=0x7f7504009870 tx=0x7f750402ba20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b3b203a389547d46 server_cookie=0 in_seq=0 out_seq=0 2023-08-09T14:51:27.084 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:27.083+0000 7f750d2b6640 1 -- 172.21.15.43:0/2447666415 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f750402f840 con 0x7f750815c550 2023-08-09T14:51:27.084 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:27.083+0000 7f750d2b6640 1 -- 172.21.15.43:0/2447666415 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f750402f9a0 con 0x7f750815c550 2023-08-09T14:51:27.084 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:27.083+0000 7f750d2b6640 1 -- 172.21.15.43:0/2447666415 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f750402fd30 con 0x7f750815c550 2023-08-09T14:51:27.085 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:27.083+0000 7f751051c640 1 -- 172.21.15.43:0/2447666415 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f750815c550 msgr2=0x7f75080d1a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T14:51:27.085 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:27.083+0000 7f751051c640 1 --2- 172.21.15.43:0/2447666415 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f750815c550 0x7f75080d1a80 secure :-1 s=READY pgs=1763 cs=0 l=1 rev1=1 crypto rx=0x7f7504009870 tx=0x7f750402ba20 comp rx=0 tx=0).stop 2023-08-09T14:51:27.085 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:27.083+0000 7f751051c640 1 -- 172.21.15.43:0/2447666415 shutdown_connections 2023-08-09T14:51:27.086 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:27.084+0000 7f751051c640 1 --2- 172.21.15.43:0/2447666415 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f750815c550 0x7f75080d1a80 unknown :-1 s=CLOSED pgs=1763 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T14:51:27.086 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:27.084+0000 7f751051c640 1 -- 172.21.15.43:0/2447666415 >> 172.21.15.43:0/2447666415 conn(0x7f750806a4e0 msgr2=0x7f750806a8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-09T14:51:27.087 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:27.084+0000 7f751051c640 1 -- 172.21.15.43:0/2447666415 shutdown_connections 2023-08-09T14:51:27.087 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:27.084+0000 7f751051c640 1 -- 172.21.15.43:0/2447666415 wait complete. 2023-08-09T14:51:27.087 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:27.085+0000 7f751051c640 1 Processor -- start 2023-08-09T14:51:27.087 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:27.086+0000 7f751051c640 1 -- start start 2023-08-09T14:51:27.089 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:27.087+0000 7f751051c640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f75080cf550 0x7f75080cc9e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T14:51:27.089 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:27.087+0000 7f751051c640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f750815c550 0x7f75080c8440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T14:51:27.089 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:27.087+0000 7f751051c640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f750815c930 0x7f75080c8980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T14:51:27.089 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:27.087+0000 7f751051c640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f75080b3950 con 0x7f750815c550 2023-08-09T14:51:27.089 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:27.087+0000 7f751051c640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f75080c8ec0 con 0x7f75080cf550 2023-08-09T14:51:27.090 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:27.087+0000 7f751051c640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f75080b3d70 con 0x7f750815c930 2023-08-09T14:51:27.090 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:27.087+0000 7f750dab7640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f750815c550 0x7f75080c8440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T14:51:27.090 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:27.087+0000 7f750dab7640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f750815c550 0x7f75080c8440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:39780/0 (socket says 172.21.15.43:39780) 2023-08-09T14:51:27.090 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:27.087+0000 7f750dab7640 1 -- 172.21.15.43:0/3531141656 learned_addr learned my addr 172.21.15.43:0/3531141656 (peer_addr_for_me v2:172.21.15.43:0/0) 2023-08-09T14:51:27.090 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:27.087+0000 7f750e2b8640 1 --2- 172.21.15.43:0/3531141656 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f75080cf550 0x7f75080cc9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T14:51:27.090 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:27.087+0000 7f750eab9640 1 --2- 172.21.15.43:0/3531141656 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f750815c930 0x7f75080c8980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T14:51:27.090 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:27.087+0000 7f750dab7640 1 -- 172.21.15.43:0/3531141656 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f750815c930 msgr2=0x7f75080c8980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T14:51:27.091 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:27.087+0000 7f750dab7640 1 --2- 172.21.15.43:0/3531141656 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f750815c930 0x7f75080c8980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T14:51:27.091 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:27.087+0000 7f750dab7640 1 -- 172.21.15.43:0/3531141656 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f75080cf550 msgr2=0x7f75080cc9e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T14:51:27.091 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:27.088+0000 7f750dab7640 1 --2- 172.21.15.43:0/3531141656 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f75080cf550 0x7f75080cc9e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T14:51:27.091 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:27.088+0000 7f750dab7640 1 -- 172.21.15.43:0/3531141656 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f75040094d0 con 0x7f750815c550 2023-08-09T14:51:27.091 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:27.088+0000 7f750eab9640 1 --2- 172.21.15.43:0/3531141656 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f750815c930 0x7f75080c8980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-08-09T14:51:27.091 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:27.088+0000 7f750e2b8640 1 --2- 172.21.15.43:0/3531141656 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f75080cf550 0x7f75080cc9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-08-09T14:51:27.092 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:27.088+0000 7f750dab7640 1 --2- 172.21.15.43:0/3531141656 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f750815c550 0x7f75080c8440 secure :-1 s=READY pgs=1764 cs=0 l=1 rev1=1 crypto rx=0x7f74f800b3f0 tx=0x7f74f800b8b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-09T14:51:27.092 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:27.088+0000 7f74feffd640 1 -- 172.21.15.43:0/3531141656 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f74f8011830 con 0x7f750815c550 2023-08-09T14:51:27.093 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:27.088+0000 7f74feffd640 1 -- 172.21.15.43:0/3531141656 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f74f8011990 con 0x7f750815c550 2023-08-09T14:51:27.093 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:27.088+0000 7f74feffd640 1 -- 172.21.15.43:0/3531141656 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f74f8011d40 con 0x7f750815c550 2023-08-09T14:51:27.093 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:27.088+0000 7f751051c640 1 -- 172.21.15.43:0/3531141656 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f75080b2fb0 con 0x7f750815c550 2023-08-09T14:51:27.093 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:27.088+0000 7f751051c640 1 -- 172.21.15.43:0/3531141656 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f75080b3570 con 0x7f750815c550 2023-08-09T14:51:27.093 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:27.090+0000 7f751051c640 1 -- 172.21.15.43:0/3531141656 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f75080b3a90 con 0x7f750815c550 2023-08-09T14:51:27.093 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:27.091+0000 7f74feffd640 1 -- 172.21.15.43:0/3531141656 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 75) v1 ==== 88455+0+0 (secure 0 0 0) 0x7f74f801a400 con 0x7f750815c550 2023-08-09T14:51:27.094 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:27.092+0000 7f74feffd640 1 --2- 172.21.15.43:0/3531141656 >> [v2:172.21.15.203:6832/372694067,v1:172.21.15.203:6833/372694067] conn(0x7f74ec06a170 0x7f74ec06c630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T14:51:27.094 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:27.092+0000 7f74feffd640 1 -- 172.21.15.43:0/3531141656 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(159..159 src has 1..159) v4 ==== 6916+0+0 (secure 0 0 0) 0x7f74f8088950 con 0x7f750815c550 2023-08-09T14:51:27.094 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:27.092+0000 7f750e2b8640 1 -- 172.21.15.43:0/3531141656 >> [v2:172.21.15.203:6832/372694067,v1:172.21.15.203:6833/372694067] conn(0x7f74ec06a170 msgr2=0x7f74ec06c630 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.203:6832/372694067 2023-08-09T14:51:27.094 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:27.092+0000 7f750e2b8640 1 --2- 172.21.15.43:0/3531141656 >> [v2:172.21.15.203:6832/372694067,v1:172.21.15.203:6833/372694067] conn(0x7f74ec06a170 0x7f74ec06c630 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-08-09T14:51:27.097 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:27.096+0000 7f74feffd640 1 -- 172.21.15.43:0/3531141656 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178066 (secure 0 0 0) 0x7f75080b3a90 con 0x7f750815c550 2023-08-09T14:51:27.125 INFO:tasks.ceph.mgr.x.smithi203.stderr:2023-08-09T14:51:27.125+0000 7fe0afe0f040 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2023-08-09T14:51:27.176 INFO:tasks.ceph.mgr.x.smithi203.stderr:2023-08-09T14:51:27.176+0000 7fe0afe0f040 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2023-08-09T14:51:27.195 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:27.193+0000 7f74feffd640 1 -- 172.21.15.43:0/3531141656 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mgrmap(e 76) v1 ==== 88052+0+0 (secure 0 0 0) 0x7f74f8053f90 con 0x7f750815c550 2023-08-09T14:51:27.196 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:27.194+0000 7f74feffd640 1 -- 172.21.15.43:0/3531141656 >> [v2:172.21.15.203:6832/372694067,v1:172.21.15.203:6833/372694067] conn(0x7f74ec06a170 msgr2=0x7f74ec06c630 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-08-09T14:51:27.196 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:27.194+0000 7f74feffd640 1 --2- 172.21.15.43:0/3531141656 >> [v2:172.21.15.203:6832/372694067,v1:172.21.15.203:6833/372694067] conn(0x7f74ec06a170 0x7f74ec06c630 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T14:51:27.262 INFO:tasks.ceph.mgr.y.smithi043.stderr:2023-08-09T14:51:27.260+0000 7f6a9a356040 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2023-08-09T14:51:27.271 INFO:tasks.ceph.mgr.x.smithi203.stderr:2023-08-09T14:51:27.270+0000 7fe035eff640 -1 client.0 error registering admin socket command: (17) File exists 2023-08-09T14:51:27.271 INFO:tasks.ceph.mgr.x.smithi203.stderr:2023-08-09T14:51:27.271+0000 7fe035eff640 -1 client.0 error registering admin socket command: (17) File exists 2023-08-09T14:51:27.271 INFO:tasks.ceph.mgr.x.smithi203.stderr:2023-08-09T14:51:27.271+0000 7fe035eff640 -1 client.0 error registering admin socket command: (17) File exists 2023-08-09T14:51:27.272 INFO:tasks.ceph.mgr.x.smithi203.stderr:2023-08-09T14:51:27.271+0000 7fe035eff640 -1 client.0 error registering admin socket command: (17) File exists 2023-08-09T14:51:27.272 INFO:tasks.ceph.mgr.x.smithi203.stderr:2023-08-09T14:51:27.271+0000 7fe035eff640 -1 client.0 error registering admin socket command: (17) File exists 2023-08-09T14:51:27.272 INFO:tasks.ceph.mgr.x.smithi203.stderr:2023-08-09T14:51:27.271+0000 7fe032ef9640 -1 client.0 error registering admin socket command: (17) File exists 2023-08-09T14:51:27.272 INFO:tasks.ceph.mgr.x.smithi203.stderr:2023-08-09T14:51:27.271+0000 7fe032ef9640 -1 client.0 error registering admin socket command: (17) File exists 2023-08-09T14:51:27.273 INFO:tasks.ceph.mgr.x.smithi203.stderr:2023-08-09T14:51:27.271+0000 7fe032ef9640 -1 client.0 error registering admin socket command: (17) File exists 2023-08-09T14:51:27.273 INFO:tasks.ceph.mgr.x.smithi203.stderr:2023-08-09T14:51:27.271+0000 7fe032ef9640 -1 client.0 error registering admin socket command: (17) File exists 2023-08-09T14:51:27.273 INFO:tasks.ceph.mgr.x.smithi203.stderr:2023-08-09T14:51:27.271+0000 7fe032ef9640 -1 client.0 error registering admin socket command: (17) File exists 2023-08-09T14:51:27.319 INFO:tasks.ceph.mgr.y.smithi043.stderr:2023-08-09T14:51:27.317+0000 7f6a9a356040 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2023-08-09T14:51:27.338 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:27.337+0000 7f751051c640 1 -- 172.21.15.43:0/3531141656 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f75080b3a90 con 0x7f750815c550 2023-08-09T14:51:27.339 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:27.338+0000 7f74feffd640 1 -- 172.21.15.43:0/3531141656 <== mon.0 v2:172.21.15.43:3300/0 8 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 70 v70) v1 ==== 94+0+4883 (secure 0 0 0) 0x7f75080b3a90 con 0x7f750815c550 2023-08-09T14:51:27.340 INFO:teuthology.orchestra.run.smithi043.stdout: 2023-08-09T14:51:27.340 INFO:teuthology.orchestra.run.smithi043.stdout:{"epoch":70,"default_fscid":8,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6812,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.203:6835/2100579784","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6834","nonce":2100579784},{"type":"v1","addr":"172.21.15.203:6835","nonce":2100579784}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":65},{"gid":6815,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.203:6837/1689128061","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6836","nonce":1689128061},{"type":"v1","addr":"172.21.15.203:6837","nonce":1689128061}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":65},{"gid":7543,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.43:6835/1126540259","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6833","nonce":1126540259},{"type":"v1","addr":"172.21.15.43:6835","nonce":1126540259}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":65}],"filesystems":[{"mdsmap":{"epoch":70,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-08-09T13:55:28.794042+0000","modified":"2023-08-09T13:55:32.388404+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":9126},"failed":[],"damaged":[],"stopped":[],"info":{"gid_9126":{"gid":9126,"name":"a","rank":0,"incarnation":66,"state":"up:active","state_seq":3,"addr":"172.21.15.43:6834/2023142010","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6832","nonce":2023142010},{"type":"v1","addr":"172.21.15.43:6834","nonce":2023142010}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[19],"metadata_pool":18,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":8}]} 2023-08-09T14:51:27.340 INFO:teuthology.orchestra.run.smithi043.stderr:dumped fsmap epoch 70 2023-08-09T14:51:27.342 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:27.340+0000 7f751051c640 1 -- 172.21.15.43:0/3531141656 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f750815c550 msgr2=0x7f75080c8440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T14:51:27.342 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:27.340+0000 7f751051c640 1 --2- 172.21.15.43:0/3531141656 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f750815c550 0x7f75080c8440 secure :-1 s=READY pgs=1764 cs=0 l=1 rev1=1 crypto rx=0x7f74f800b3f0 tx=0x7f74f800b8b0 comp rx=0 tx=0).stop 2023-08-09T14:51:27.343 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:27.340+0000 7f751051c640 1 -- 172.21.15.43:0/3531141656 shutdown_connections 2023-08-09T14:51:27.343 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:27.340+0000 7f751051c640 1 --2- 172.21.15.43:0/3531141656 >> [v2:172.21.15.203:6832/372694067,v1:172.21.15.203:6833/372694067] conn(0x7f74ec06a170 0x7f74ec06c630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T14:51:27.343 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:27.340+0000 7f751051c640 1 --2- 172.21.15.43:0/3531141656 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f750815c930 0x7f75080c8980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T14:51:27.343 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:27.340+0000 7f751051c640 1 --2- 172.21.15.43:0/3531141656 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f750815c550 0x7f75080c8440 unknown :-1 s=CLOSED pgs=1764 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T14:51:27.343 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:27.340+0000 7f751051c640 1 --2- 172.21.15.43:0/3531141656 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f75080cf550 0x7f75080cc9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T14:51:27.344 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:27.340+0000 7f751051c640 1 -- 172.21.15.43:0/3531141656 >> 172.21.15.43:0/3531141656 conn(0x7f750806a4e0 msgr2=0x7f750815dae0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-09T14:51:27.344 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:27.341+0000 7f751051c640 1 -- 172.21.15.43:0/3531141656 shutdown_connections 2023-08-09T14:51:27.344 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:27.341+0000 7f751051c640 1 -- 172.21.15.43:0/3531141656 wait complete. 2023-08-09T14:51:27.359 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-08-09T14:51:27.376 INFO:tasks.ceph.mgr.y.smithi043.stderr:2023-08-09T14:51:27.375+0000 7f6a9a356040 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2023-08-09T14:51:27.430 INFO:tasks.ceph.mgr.y.smithi043.stderr:2023-08-09T14:51:27.428+0000 7f6a9a356040 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2023-08-09T14:51:27.565 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:27.563+0000 7fbfe9a2d640 1 Processor -- start 2023-08-09T14:51:27.565 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:27.563+0000 7fbfe9a2d640 1 -- start start 2023-08-09T14:51:27.566 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:27.565+0000 7fbfe9a2d640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7fbfe4143c20 0x7fbfe406f540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T14:51:27.567 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:27.565+0000 7fbfe9a2d640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fbfe406cfe0 0x7fbfe40c8b50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T14:51:27.567 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:27.565+0000 7fbfe9a2d640 1 -- --> v1:172.21.15.203:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbfe40c9090 con 0x7fbfe40c86f0 2023-08-09T14:51:27.567 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:27.565+0000 7fbfe9a2d640 1 -- --> v2:172.21.15.43:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbfe4144000 con 0x7fbfe4143c20 2023-08-09T14:51:27.567 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:27.565+0000 7fbfe9a2d640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbfe406c440 con 0x7fbfe406cfe0 2023-08-09T14:51:27.568 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:27.566+0000 7fbfe2ffd640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7fbfe4143c20 0x7fbfe406f540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T14:51:27.568 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:27.566+0000 7fbfe2ffd640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7fbfe4143c20 0x7fbfe406f540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:39788/0 (socket says 172.21.15.43:39788) 2023-08-09T14:51:27.568 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:27.566+0000 7fbfe2ffd640 1 -- 172.21.15.43:0/1020687151 learned_addr learned my addr 172.21.15.43:0/1020687151 (peer_addr_for_me v2:172.21.15.43:0/0) 2023-08-09T14:51:27.569 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:27.566+0000 7fbfe37fe640 1 --2- 172.21.15.43:0/1020687151 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fbfe406cfe0 0x7fbfe40c8b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T14:51:27.569 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:27.566+0000 7fbfe2ffd640 1 -- 172.21.15.43:0/1020687151 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fbfe406cfe0 msgr2=0x7fbfe40c8b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-09T14:51:27.570 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:27.566+0000 7fbfe2ffd640 1 --2- 172.21.15.43:0/1020687151 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fbfe406cfe0 0x7fbfe40c8b50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T14:51:27.570 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:27.566+0000 7fbfe2ffd640 1 -- 172.21.15.43:0/1020687151 >> v1:172.21.15.203:6789/0 conn(0x7fbfe40c86f0 legacy=0x7fbfe406c8d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-09T14:51:27.570 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:27.566+0000 7fbfe2ffd640 1 -- 172.21.15.43:0/1020687151 --> v2:172.21.15.43:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbfe406b830 con 0x7fbfe4143c20 2023-08-09T14:51:27.571 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:27.567+0000 7fbfe2ffd640 1 --2- 172.21.15.43:0/1020687151 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fbfe4143c20 0x7fbfe406f540 secure :-1 s=READY pgs=1766 cs=0 l=1 rev1=1 crypto rx=0x7fbfd4009870 tx=0x7fbfd402b860 comp rx=0 tx=0).ready entity=mon.0 client_cookie=375311a18009793b server_cookie=0 in_seq=0 out_seq=0 2023-08-09T14:51:27.572 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:27.568+0000 7fbfe27fc640 1 -- 172.21.15.43:0/1020687151 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbfd40317c0 con 0x7fbfe4143c20 2023-08-09T14:51:27.572 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:27.568+0000 7fbfe27fc640 1 -- 172.21.15.43:0/1020687151 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbfd4031920 con 0x7fbfe4143c20 2023-08-09T14:51:27.572 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:27.568+0000 7fbfe27fc640 1 -- 172.21.15.43:0/1020687151 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbfd4031cb0 con 0x7fbfe4143c20 2023-08-09T14:51:27.572 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:27.568+0000 7fbfe9a2d640 1 -- 172.21.15.43:0/1020687151 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fbfe4143c20 msgr2=0x7fbfe406f540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T14:51:27.572 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:27.568+0000 7fbfe9a2d640 1 --2- 172.21.15.43:0/1020687151 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fbfe4143c20 0x7fbfe406f540 secure :-1 s=READY pgs=1766 cs=0 l=1 rev1=1 crypto rx=0x7fbfd4009870 tx=0x7fbfd402b860 comp rx=0 tx=0).stop 2023-08-09T14:51:27.572 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:27.568+0000 7fbfe9a2d640 1 -- 172.21.15.43:0/1020687151 shutdown_connections 2023-08-09T14:51:27.573 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:27.568+0000 7fbfe9a2d640 1 --2- 172.21.15.43:0/1020687151 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fbfe406cfe0 0x7fbfe40c8b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T14:51:27.573 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:27.568+0000 7fbfe9a2d640 1 --2- 172.21.15.43:0/1020687151 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fbfe4143c20 0x7fbfe406f540 unknown :-1 s=CLOSED pgs=1766 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T14:51:27.573 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:27.568+0000 7fbfe9a2d640 1 -- 172.21.15.43:0/1020687151 >> 172.21.15.43:0/1020687151 conn(0x7fbfe40ce0a0 msgr2=0x7fbfe40ce480 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-09T14:51:27.573 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:27.569+0000 7fbfe9a2d640 1 -- 172.21.15.43:0/1020687151 shutdown_connections 2023-08-09T14:51:27.573 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:27.569+0000 7fbfe9a2d640 1 -- 172.21.15.43:0/1020687151 wait complete. 2023-08-09T14:51:27.573 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:27.570+0000 7fbfe9a2d640 1 Processor -- start 2023-08-09T14:51:27.576 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:27.570+0000 7fbfe9a2d640 1 -- start start 2023-08-09T14:51:27.576 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:27.571+0000 7fbfe9a2d640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fbfe406cfe0 0x7fbfe4132880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T14:51:27.576 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:27.571+0000 7fbfe9a2d640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fbfe40c86f0 0x7fbfe40c4000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T14:51:27.576 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:27.571+0000 7fbfe9a2d640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fbfe4143c20 0x7fbfe40ac520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T14:51:27.576 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:27.572+0000 7fbfe9a2d640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbfe40c6690 con 0x7fbfe40c86f0 2023-08-09T14:51:27.577 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:27.572+0000 7fbfe9a2d640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbfe40c6340 con 0x7fbfe406cfe0 2023-08-09T14:51:27.577 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:27.572+0000 7fbfe9a2d640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbfe40c4a50 con 0x7fbfe4143c20 2023-08-09T14:51:27.577 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:27.572+0000 7fbfdbfff640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fbfe40c86f0 0x7fbfe40c4000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T14:51:27.577 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:27.572+0000 7fbfdbfff640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fbfe40c86f0 0x7fbfe40c4000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:39794/0 (socket says 172.21.15.43:39794) 2023-08-09T14:51:27.577 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:27.572+0000 7fbfdbfff640 1 -- 172.21.15.43:0/172818517 learned_addr learned my addr 172.21.15.43:0/172818517 (peer_addr_for_me v2:172.21.15.43:0/0) 2023-08-09T14:51:27.577 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:27.572+0000 7fbfe2ffd640 1 --2- 172.21.15.43:0/172818517 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fbfe406cfe0 0x7fbfe4132880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T14:51:27.577 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:27.572+0000 7fbfe37fe640 1 --2- 172.21.15.43:0/172818517 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fbfe4143c20 0x7fbfe40ac520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T14:51:27.578 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:27.572+0000 7fbfdbfff640 1 -- 172.21.15.43:0/172818517 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fbfe4143c20 msgr2=0x7fbfe40ac520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T14:51:27.578 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:27.572+0000 7fbfdbfff640 1 --2- 172.21.15.43:0/172818517 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fbfe4143c20 0x7fbfe40ac520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T14:51:27.578 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:27.572+0000 7fbfdbfff640 1 -- 172.21.15.43:0/172818517 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fbfe406cfe0 msgr2=0x7fbfe4132880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T14:51:27.578 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:27.572+0000 7fbfdbfff640 1 --2- 172.21.15.43:0/172818517 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fbfe406cfe0 0x7fbfe4132880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T14:51:27.578 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:27.572+0000 7fbfdbfff640 1 -- 172.21.15.43:0/172818517 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbfd40094d0 con 0x7fbfe40c86f0 2023-08-09T14:51:27.578 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:27.573+0000 7fbfdbfff640 1 --2- 172.21.15.43:0/172818517 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fbfe40c86f0 0x7fbfe40c4000 secure :-1 s=READY pgs=1767 cs=0 l=1 rev1=1 crypto rx=0x7fbfcc00c430 tx=0x7fbfcc00c8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-09T14:51:27.578 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:27.573+0000 7fbfe0ff9640 1 -- 172.21.15.43:0/172818517 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbfcc011790 con 0x7fbfe40c86f0 2023-08-09T14:51:27.579 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:27.573+0000 7fbfe0ff9640 1 -- 172.21.15.43:0/172818517 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbfcc0118f0 con 0x7fbfe40c86f0 2023-08-09T14:51:27.579 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:27.573+0000 7fbfe0ff9640 1 -- 172.21.15.43:0/172818517 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbfcc011ce0 con 0x7fbfe40c86f0 2023-08-09T14:51:27.579 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:27.573+0000 7fbfe9a2d640 1 -- 172.21.15.43:0/172818517 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbfe40c4d30 con 0x7fbfe40c86f0 2023-08-09T14:51:27.579 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:27.573+0000 7fbfe9a2d640 1 -- 172.21.15.43:0/172818517 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbfe40aba50 con 0x7fbfe40c86f0 2023-08-09T14:51:27.579 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:27.574+0000 7fbfe9a2d640 1 -- 172.21.15.43:0/172818517 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbfe40c6840 con 0x7fbfe40c86f0 2023-08-09T14:51:27.580 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:27.575+0000 7fbfe0ff9640 1 -- 172.21.15.43:0/172818517 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 76) v1 ==== 88052+0+0 (secure 0 0 0) 0x7fbfcc01c460 con 0x7fbfe40c86f0 2023-08-09T14:51:27.580 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:27.577+0000 7fbfe0ff9640 1 -- 172.21.15.43:0/172818517 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(160..160 src has 1..160) v4 ==== 7216+0+0 (secure 0 0 0) 0x7fbfcc088470 con 0x7fbfe40c86f0 2023-08-09T14:51:27.582 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:27.581+0000 7fbfe0ff9640 1 -- 172.21.15.43:0/172818517 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178066 (secure 0 0 0) 0x7fbfe40c6840 con 0x7fbfe40c86f0 2023-08-09T14:51:27.778 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:27.777+0000 7fbfe9a2d640 1 -- 172.21.15.43:0/172818517 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fbfe40c4f20 con 0x7fbfe40c86f0 2023-08-09T14:51:27.782 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:27.780+0000 7fbfe0ff9640 1 -- 172.21.15.43:0/172818517 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v160) v1 ==== 74+0+19721 (secure 0 0 0) 0x7fbfcc053a60 con 0x7fbfe40c86f0 2023-08-09T14:51:27.782 INFO:teuthology.orchestra.run.smithi043.stdout: 2023-08-09T14:51:27.783 INFO:teuthology.orchestra.run.smithi043.stdout:{"epoch":160,"fsid":"daac87ee-1337-4250-a955-aea4e408a39b","created":"2023-08-09T13:33:41.701198+0000","modified":"2023-08-09T14:51:27.179740+0000","last_up_change":"2023-08-09T13:33:53.783453+0000","last_in_change":"2023-08-09T13:33:45.148904+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":19,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":"rbd","create_time":"2023-08-09T13:33:54.365137+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":".mgr","create_time":"2023-08-09T13:33:54.831803+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":18,"pool_name":"cephfs_metadata","create_time":"2023-08-09T13:55:27.207507+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"146","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":19,"pool_name":"cephfs_data","create_time":"2023-08-09T13:55:27.825578+0000","flags":8193,"flags_names":"hashpspool,selfmanaged_snaps","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"159","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":113,"snap_epoch":159,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.25,"score_stable":1.25,"optimal_score":1,"raw_score_acting":1.25,"raw_score_stable":1.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"cbd49ecd-e896-4fbd-864e-5c37a829b165","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":149,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6800","nonce":502667335},{"type":"v1","addr":"172.21.15.43:6801","nonce":502667335}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6802","nonce":502667335},{"type":"v1","addr":"172.21.15.43:6803","nonce":502667335}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6806","nonce":502667335},{"type":"v1","addr":"172.21.15.43:6807","nonce":502667335}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6804","nonce":502667335},{"type":"v1","addr":"172.21.15.43:6805","nonce":502667335}]},"public_addr":"172.21.15.43:6801/502667335","cluster_addr":"172.21.15.43:6803/502667335","heartbeat_back_addr":"172.21.15.43:6807/502667335","heartbeat_front_addr":"172.21.15.43:6805/502667335","state":["exists","up"]},{"osd":1,"uuid":"75c7e0fe-1ec7-4fe8-a2f2-3661a65b8500","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":145,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6808","nonce":3289134610},{"type":"v1","addr":"172.21.15.43:6809","nonce":3289134610}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6810","nonce":3289134610},{"type":"v1","addr":"172.21.15.43:6812","nonce":3289134610}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6821","nonce":3289134610},{"type":"v1","addr":"172.21.15.43:6824","nonce":3289134610}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6815","nonce":3289134610},{"type":"v1","addr":"172.21.15.43:6818","nonce":3289134610}]},"public_addr":"172.21.15.43:6809/3289134610","cluster_addr":"172.21.15.43:6812/3289134610","heartbeat_back_addr":"172.21.15.43:6824/3289134610","heartbeat_front_addr":"172.21.15.43:6818/3289134610","state":["exists","up"]},{"osd":2,"uuid":"bb5c41e9-7f5f-4125-9b01-3bb8bfc2ea83","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":145,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6811","nonce":4079709736},{"type":"v1","addr":"172.21.15.43:6814","nonce":4079709736}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6817","nonce":4079709736},{"type":"v1","addr":"172.21.15.43:6820","nonce":4079709736}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6829","nonce":4079709736},{"type":"v1","addr":"172.21.15.43:6831","nonce":4079709736}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6823","nonce":4079709736},{"type":"v1","addr":"172.21.15.43:6826","nonce":4079709736}]},"public_addr":"172.21.15.43:6814/4079709736","cluster_addr":"172.21.15.43:6820/4079709736","heartbeat_back_addr":"172.21.15.43:6831/4079709736","heartbeat_front_addr":"172.21.15.43:6826/4079709736","state":["exists","up"]},{"osd":3,"uuid":"bfe4c8b4-694a-465c-87e6-2b2aab852613","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":149,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6813","nonce":1709538759},{"type":"v1","addr":"172.21.15.43:6816","nonce":1709538759}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6819","nonce":1709538759},{"type":"v1","addr":"172.21.15.43:6822","nonce":1709538759}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6828","nonce":1709538759},{"type":"v1","addr":"172.21.15.43:6830","nonce":1709538759}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6825","nonce":1709538759},{"type":"v1","addr":"172.21.15.43:6827","nonce":1709538759}]},"public_addr":"172.21.15.43:6816/1709538759","cluster_addr":"172.21.15.43:6822/1709538759","heartbeat_back_addr":"172.21.15.43:6830/1709538759","heartbeat_front_addr":"172.21.15.43:6827/1709538759","state":["exists","up"]},{"osd":4,"uuid":"fa2e8a77-aba1-4756-853e-0d174a5bdeba","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":145,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6800","nonce":4196866481},{"type":"v1","addr":"172.21.15.203:6801","nonce":4196866481}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6802","nonce":4196866481},{"type":"v1","addr":"172.21.15.203:6803","nonce":4196866481}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6806","nonce":4196866481},{"type":"v1","addr":"172.21.15.203:6807","nonce":4196866481}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6804","nonce":4196866481},{"type":"v1","addr":"172.21.15.203:6805","nonce":4196866481}]},"public_addr":"172.21.15.203:6801/4196866481","cluster_addr":"172.21.15.203:6803/4196866481","heartbeat_back_addr":"172.21.15.203:6807/4196866481","heartbeat_front_addr":"172.21.15.203:6805/4196866481","state":["exists","up"]},{"osd":5,"uuid":"81811e4b-4d24-4f51-a82c-f1d700a2a00d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":145,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6816","nonce":3059290081},{"type":"v1","addr":"172.21.15.203:6817","nonce":3059290081}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6818","nonce":3059290081},{"type":"v1","addr":"172.21.15.203:6819","nonce":3059290081}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6822","nonce":3059290081},{"type":"v1","addr":"172.21.15.203:6823","nonce":3059290081}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6820","nonce":3059290081},{"type":"v1","addr":"172.21.15.203:6821","nonce":3059290081}]},"public_addr":"172.21.15.203:6817/3059290081","cluster_addr":"172.21.15.203:6819/3059290081","heartbeat_back_addr":"172.21.15.203:6823/3059290081","heartbeat_front_addr":"172.21.15.203:6821/3059290081","state":["exists","up"]},{"osd":6,"uuid":"307bac4d-6a5b-443d-af75-56d695d7674f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":145,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6824","nonce":512155569},{"type":"v1","addr":"172.21.15.203:6825","nonce":512155569}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6826","nonce":512155569},{"type":"v1","addr":"172.21.15.203:6827","nonce":512155569}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6830","nonce":512155569},{"type":"v1","addr":"172.21.15.203:6831","nonce":512155569}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6828","nonce":512155569},{"type":"v1","addr":"172.21.15.203:6829","nonce":512155569}]},"public_addr":"172.21.15.203:6825/512155569","cluster_addr":"172.21.15.203:6827/512155569","heartbeat_back_addr":"172.21.15.203:6831/512155569","heartbeat_front_addr":"172.21.15.203:6829/512155569","state":["exists","up"]},{"osd":7,"uuid":"a95dc7ca-9f1b-4295-af70-68a22a2d7d5e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":145,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6808","nonce":3331450438},{"type":"v1","addr":"172.21.15.203:6809","nonce":3331450438}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6810","nonce":3331450438},{"type":"v1","addr":"172.21.15.203:6811","nonce":3331450438}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6814","nonce":3331450438},{"type":"v1","addr":"172.21.15.203:6815","nonce":3331450438}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6812","nonce":3331450438},{"type":"v1","addr":"172.21.15.203:6813","nonce":3331450438}]},"public_addr":"172.21.15.203:6809/3331450438","cluster_addr":"172.21.15.203:6811/3331450438","heartbeat_back_addr":"172.21.15.203:6815/3331450438","heartbeat_front_addr":"172.21.15.203:6813/3331450438","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-09T13:33:50.547652+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-09T13:33:50.589469+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-09T13:33:50.561724+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-09T13:33:50.599650+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-09T13:33:50.411855+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-09T13:33:50.219681+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-09T13:33:50.504664+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-09T13:33:50.441248+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"19.25","mappings":[{"from":2,"to":0}]},{"pgid":"19.26","mappings":[{"from":2,"to":0}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.203:0/1340171653":"2023-08-10T14:51:27.179649+0000","172.21.15.203:0/2575410559":"2023-08-10T14:51:27.179649+0000","172.21.15.203:0/1196185412":"2023-08-10T14:51:27.179649+0000","172.21.15.203:6833/372694067":"2023-08-10T14:51:27.179649+0000","172.21.15.203:6832/372694067":"2023-08-10T14:51:27.179649+0000","172.21.15.203:0/2726918685":"2023-08-10T14:51:27.179649+0000","172.21.15.203:0/3273594633":"2023-08-10T14:51:27.179649+0000","172.21.15.203:0/2477376154":"2023-08-10T14:51:27.179649+0000","172.21.15.203:0/2901879784":"2023-08-10T13:55:44.498027+0000","172.21.15.203:0/1501229236":"2023-08-10T13:55:44.498027+0000","172.21.15.203:0/3615051942":"2023-08-10T13:55:44.498027+0000","172.21.15.203:6833/3490266242":"2023-08-10T13:55:44.498027+0000","172.21.15.203:6832/3490266242":"2023-08-10T13:55:44.498027+0000","172.21.15.203:0/3892021074":"2023-08-10T13:55:44.498027+0000","172.21.15.203:0/750810033":"2023-08-10T13:55:44.498027+0000","172.21.15.203:0/1164133940":"2023-08-10T13:55:44.498027+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":7,"snaps":[{"begin":104,"length":1},{"begin":221,"length":1}]},{"pool":19,"snaps":[{"begin":23,"length":35},{"begin":79,"length":35}]}],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-08-09T14:51:27.784 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:27.782+0000 7fbfe9a2d640 1 -- 172.21.15.43:0/172818517 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fbfe40c86f0 msgr2=0x7fbfe40c4000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T14:51:27.784 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:27.782+0000 7fbfe9a2d640 1 --2- 172.21.15.43:0/172818517 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fbfe40c86f0 0x7fbfe40c4000 secure :-1 s=READY pgs=1767 cs=0 l=1 rev1=1 crypto rx=0x7fbfcc00c430 tx=0x7fbfcc00c8f0 comp rx=0 tx=0).stop 2023-08-09T14:51:27.784 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:27.783+0000 7fbfe9a2d640 1 -- 172.21.15.43:0/172818517 shutdown_connections 2023-08-09T14:51:27.785 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:27.783+0000 7fbfe9a2d640 1 --2- 172.21.15.43:0/172818517 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fbfe4143c20 0x7fbfe40ac520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T14:51:27.785 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:27.783+0000 7fbfe9a2d640 1 --2- 172.21.15.43:0/172818517 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fbfe40c86f0 0x7fbfe40c4000 unknown :-1 s=CLOSED pgs=1767 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T14:51:27.785 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:27.783+0000 7fbfe9a2d640 1 --2- 172.21.15.43:0/172818517 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fbfe406cfe0 0x7fbfe4132880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T14:51:27.785 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:27.783+0000 7fbfe9a2d640 1 -- 172.21.15.43:0/172818517 >> 172.21.15.43:0/172818517 conn(0x7fbfe40ce0a0 msgr2=0x7fbfe415abd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-09T14:51:27.785 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:27.783+0000 7fbfe9a2d640 1 -- 172.21.15.43:0/172818517 shutdown_connections 2023-08-09T14:51:27.785 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:27.783+0000 7fbfe9a2d640 1 -- 172.21.15.43:0/172818517 wait complete. 2023-08-09T14:51:27.803 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': 'rbd', 'create_time': '2023-08-09T13:33:54.365137+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}, 'read_balance': {'score_acting': 2, 'score_stable': 2, 'optimal_score': 1, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': '.mgr', 'create_time': '2023-08-09T13:33:54.831803+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 18, 'pool_name': 'cephfs_metadata', 'create_time': '2023-08-09T13:55:27.207507+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '146', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 19, 'pool_name': 'cephfs_data', 'create_time': '2023-08-09T13:55:27.825578+0000', 'flags': 8193, 'flags_names': 'hashpspool,selfmanaged_snaps', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '159', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 113, 'snap_epoch': 159, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.25, 'score_stable': 1.25, 'optimal_score': 1, 'raw_score_acting': 1.25, 'raw_score_stable': 1.25, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-08-09T14:51:27.804 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get rbd pg_num 2023-08-09T14:51:28.083 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:28.081+0000 7faa7ecfa640 1 Processor -- start 2023-08-09T14:51:28.083 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:28.081+0000 7faa7ecfa640 1 -- start start 2023-08-09T14:51:28.084 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:28.083+0000 7faa7ecfa640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7faa780cb2d0 0x7faa7815b380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T14:51:28.084 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:28.083+0000 7faa7ecfa640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7faa7815df60 0x7faa780ce1c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T14:51:28.084 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:28.083+0000 7faa7ecfa640 1 -- --> v1:172.21.15.203:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7faa7815aef0 con 0x7faa7815e3c0 2023-08-09T14:51:28.085 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:28.083+0000 7faa7ecfa640 1 -- --> v2:172.21.15.43:3300/0 -- mon_getmap magic: 0 v1 -- 0x7faa7815a970 con 0x7faa7815df60 2023-08-09T14:51:28.085 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:28.083+0000 7faa7ecfa640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7faa7815b8c0 con 0x7faa780cb2d0 2023-08-09T14:51:28.085 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:28.083+0000 7faa77fff640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7faa7815df60 0x7faa780ce1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T14:51:28.086 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:28.083+0000 7faa77fff640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7faa7815df60 0x7faa780ce1c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:39800/0 (socket says 172.21.15.43:39800) 2023-08-09T14:51:28.086 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:28.083+0000 7faa77fff640 1 -- 172.21.15.43:0/754984814 learned_addr learned my addr 172.21.15.43:0/754984814 (peer_addr_for_me v2:172.21.15.43:0/0) 2023-08-09T14:51:28.086 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:28.083+0000 7faa7ca96640 1 --2- 172.21.15.43:0/754984814 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7faa780cb2d0 0x7faa7815b380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T14:51:28.086 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:28.084+0000 7faa77fff640 1 -- 172.21.15.43:0/754984814 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7faa780cb2d0 msgr2=0x7faa7815b380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-09T14:51:28.086 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:28.084+0000 7faa77fff640 1 --2- 172.21.15.43:0/754984814 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7faa780cb2d0 0x7faa7815b380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T14:51:28.087 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:28.084+0000 7faa77fff640 1 -- 172.21.15.43:0/754984814 >> v1:172.21.15.203:6789/0 conn(0x7faa7815e3c0 legacy=0x7faa780c9f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-09T14:51:28.087 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:28.084+0000 7faa77fff640 1 -- 172.21.15.43:0/754984814 --> v2:172.21.15.43:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faa7815e940 con 0x7faa7815df60 2023-08-09T14:51:28.088 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:28.085+0000 7faa77fff640 1 --2- 172.21.15.43:0/754984814 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7faa7815df60 0x7faa780ce1c0 secure :-1 s=READY pgs=1768 cs=0 l=1 rev1=1 crypto rx=0x7faa64009870 tx=0x7faa6402b980 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fcc062998214e244 server_cookie=0 in_seq=0 out_seq=0 2023-08-09T14:51:28.088 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:28.085+0000 7faa777fe640 1 -- 172.21.15.43:0/754984814 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faa64030840 con 0x7faa7815df60 2023-08-09T14:51:28.088 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:28.085+0000 7faa777fe640 1 -- 172.21.15.43:0/754984814 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7faa640309a0 con 0x7faa7815df60 2023-08-09T14:51:28.088 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:28.085+0000 7faa777fe640 1 -- 172.21.15.43:0/754984814 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faa64030d30 con 0x7faa7815df60 2023-08-09T14:51:28.088 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:28.086+0000 7faa7ecfa640 1 -- 172.21.15.43:0/754984814 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7faa7815df60 msgr2=0x7faa780ce1c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T14:51:28.088 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:28.086+0000 7faa7ecfa640 1 --2- 172.21.15.43:0/754984814 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7faa7815df60 0x7faa780ce1c0 secure :-1 s=READY pgs=1768 cs=0 l=1 rev1=1 crypto rx=0x7faa64009870 tx=0x7faa6402b980 comp rx=0 tx=0).stop 2023-08-09T14:51:28.089 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:28.086+0000 7faa7ecfa640 1 -- 172.21.15.43:0/754984814 shutdown_connections 2023-08-09T14:51:28.089 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:28.086+0000 7faa7ecfa640 1 --2- 172.21.15.43:0/754984814 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7faa7815df60 0x7faa780ce1c0 unknown :-1 s=CLOSED pgs=1768 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T14:51:28.089 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:28.086+0000 7faa7ecfa640 1 --2- 172.21.15.43:0/754984814 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7faa780cb2d0 0x7faa7815b380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T14:51:28.089 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:28.086+0000 7faa7ecfa640 1 -- 172.21.15.43:0/754984814 >> 172.21.15.43:0/754984814 conn(0x7faa780aff50 msgr2=0x7faa780b0360 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-09T14:51:28.089 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:28.086+0000 7faa7ecfa640 1 -- 172.21.15.43:0/754984814 shutdown_connections 2023-08-09T14:51:28.089 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:28.086+0000 7faa7ecfa640 1 -- 172.21.15.43:0/754984814 wait complete. 2023-08-09T14:51:28.090 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:28.088+0000 7faa7ecfa640 1 Processor -- start 2023-08-09T14:51:28.090 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:28.088+0000 7faa7ecfa640 1 -- start start 2023-08-09T14:51:28.091 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:28.089+0000 7faa7ecfa640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7faa780cb2d0 0x7faa78140760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T14:51:28.091 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:28.089+0000 7faa7ecfa640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7faa7815df60 0x7faa781f56b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T14:51:28.091 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:28.089+0000 7faa7ecfa640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7faa7815e3c0 0x7faa780ccd00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T14:51:28.091 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:28.089+0000 7faa7ecfa640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faa780c7070 con 0x7faa7815e3c0 2023-08-09T14:51:28.091 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:28.089+0000 7faa7ecfa640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faa78140ca0 con 0x7faa780cb2d0 2023-08-09T14:51:28.091 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:28.089+0000 7faa7ecfa640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7faa780c5340 con 0x7faa7815df60 2023-08-09T14:51:28.092 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:28.089+0000 7faa7d297640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7faa7815e3c0 0x7faa780ccd00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T14:51:28.092 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:28.089+0000 7faa7d297640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7faa7815e3c0 0x7faa780ccd00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:39812/0 (socket says 172.21.15.43:39812) 2023-08-09T14:51:28.092 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:28.089+0000 7faa7d297640 1 -- 172.21.15.43:0/2793333773 learned_addr learned my addr 172.21.15.43:0/2793333773 (peer_addr_for_me v2:172.21.15.43:0/0) 2023-08-09T14:51:28.093 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:28.090+0000 7faa7ca96640 1 --2- 172.21.15.43:0/2793333773 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7faa780cb2d0 0x7faa78140760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T14:51:28.093 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:28.090+0000 7faa77fff640 1 --2- 172.21.15.43:0/2793333773 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7faa7815df60 0x7faa781f56b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T14:51:28.093 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:28.090+0000 7faa7d297640 1 -- 172.21.15.43:0/2793333773 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7faa7815df60 msgr2=0x7faa781f56b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T14:51:28.093 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:28.090+0000 7faa7d297640 1 --2- 172.21.15.43:0/2793333773 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7faa7815df60 0x7faa781f56b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T14:51:28.094 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:28.090+0000 7faa7d297640 1 -- 172.21.15.43:0/2793333773 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7faa780cb2d0 msgr2=0x7faa78140760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T14:51:28.094 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:28.090+0000 7faa7d297640 1 --2- 172.21.15.43:0/2793333773 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7faa780cb2d0 0x7faa78140760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T14:51:28.094 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:28.090+0000 7faa7d297640 1 -- 172.21.15.43:0/2793333773 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faa640094d0 con 0x7faa7815e3c0 2023-08-09T14:51:28.094 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:28.090+0000 7faa77fff640 1 --2- 172.21.15.43:0/2793333773 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7faa7815df60 0x7faa781f56b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-08-09T14:51:28.094 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:28.090+0000 7faa7d297640 1 --2- 172.21.15.43:0/2793333773 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7faa7815e3c0 0x7faa780ccd00 secure :-1 s=READY pgs=1769 cs=0 l=1 rev1=1 crypto rx=0x7faa7000c7d0 tx=0x7faa7000cc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-09T14:51:28.094 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:28.090+0000 7faa757fa640 1 -- 172.21.15.43:0/2793333773 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faa70003b70 con 0x7faa7815e3c0 2023-08-09T14:51:28.094 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:28.091+0000 7faa757fa640 1 -- 172.21.15.43:0/2793333773 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7faa70003cf0 con 0x7faa7815e3c0 2023-08-09T14:51:28.095 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:28.091+0000 7faa757fa640 1 -- 172.21.15.43:0/2793333773 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faa7000aa90 con 0x7faa7815e3c0 2023-08-09T14:51:28.095 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:28.091+0000 7faa7ecfa640 1 -- 172.21.15.43:0/2793333773 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7faa781f5bf0 con 0x7faa7815e3c0 2023-08-09T14:51:28.095 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:28.091+0000 7faa7ecfa640 1 -- 172.21.15.43:0/2793333773 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7faa78142f90 con 0x7faa7815e3c0 2023-08-09T14:51:28.095 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:28.092+0000 7faa7ecfa640 1 -- 172.21.15.43:0/2793333773 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7faa780c71e0 con 0x7faa7815e3c0 2023-08-09T14:51:28.095 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:28.093+0000 7faa757fa640 1 -- 172.21.15.43:0/2793333773 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 76) v1 ==== 88052+0+0 (secure 0 0 0) 0x7faa7000abf0 con 0x7faa7815e3c0 2023-08-09T14:51:28.096 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:28.094+0000 7faa757fa640 1 -- 172.21.15.43:0/2793333773 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(160..160 src has 1..160) v4 ==== 7216+0+0 (secure 0 0 0) 0x7faa700883d0 con 0x7faa7815e3c0 2023-08-09T14:51:28.099 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:28.098+0000 7faa757fa640 1 -- 172.21.15.43:0/2793333773 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178066 (secure 0 0 0) 0x7faa700539c0 con 0x7faa7815e3c0 2023-08-09T14:51:28.209 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:28.208+0000 7faa757fa640 1 -- 172.21.15.43:0/2793333773 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mgrmap(e 77) v1 ==== 88179+0+0 (secure 0 0 0) 0x7faa70010dc0 con 0x7faa7815e3c0 2023-08-09T14:51:28.210 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:28.208+0000 7faa757fa640 1 --2- 172.21.15.43:0/2793333773 >> [v2:172.21.15.203:6832/2566816332,v1:172.21.15.203:6833/2566816332] conn(0x7faa5c0705c0 0x7faa5c0729b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T14:51:28.211 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:28.210+0000 7faa7ca96640 1 --2- 172.21.15.43:0/2793333773 >> [v2:172.21.15.203:6832/2566816332,v1:172.21.15.203:6833/2566816332] conn(0x7faa5c0705c0 0x7faa5c0729b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T14:51:28.220 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:28.218+0000 7faa7ca96640 1 --2- 172.21.15.43:0/2793333773 >> [v2:172.21.15.203:6832/2566816332,v1:172.21.15.203:6833/2566816332] conn(0x7faa5c0705c0 0x7faa5c0729b0 secure :-1 s=READY pgs=5 cs=0 l=1 rev1=1 crypto rx=0x7faa6c006fd0 tx=0x7faa6c008040 comp rx=0 tx=0).ready entity=mgr.7765 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-09T14:51:28.300 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:28.299+0000 7faa7ecfa640 1 -- 172.21.15.43:0/2793333773 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"} v 0) v1 -- 0x7faa780739f0 con 0x7faa7815e3c0 2023-08-09T14:51:28.301 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:28.300+0000 7faa757fa640 1 -- 172.21.15.43:0/2793333773 <== mon.0 v2:172.21.15.43:3300/0 8 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"}]=0 v160) v1 ==== 92+0+10 (secure 0 0 0) 0x7faa70003e60 con 0x7faa7815e3c0 2023-08-09T14:51:28.301 INFO:teuthology.orchestra.run.smithi043.stdout:pg_num: 8 2023-08-09T14:51:28.304 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:28.303+0000 7faa7ecfa640 1 -- 172.21.15.43:0/2793333773 >> [v2:172.21.15.203:6832/2566816332,v1:172.21.15.203:6833/2566816332] conn(0x7faa5c0705c0 msgr2=0x7faa5c0729b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T14:51:28.305 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:28.303+0000 7faa7ecfa640 1 --2- 172.21.15.43:0/2793333773 >> [v2:172.21.15.203:6832/2566816332,v1:172.21.15.203:6833/2566816332] conn(0x7faa5c0705c0 0x7faa5c0729b0 secure :-1 s=READY pgs=5 cs=0 l=1 rev1=1 crypto rx=0x7faa6c006fd0 tx=0x7faa6c008040 comp rx=0 tx=0).stop 2023-08-09T14:51:28.305 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:28.303+0000 7faa7ecfa640 1 -- 172.21.15.43:0/2793333773 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7faa7815e3c0 msgr2=0x7faa780ccd00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T14:51:28.305 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:28.303+0000 7faa7ecfa640 1 --2- 172.21.15.43:0/2793333773 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7faa7815e3c0 0x7faa780ccd00 secure :-1 s=READY pgs=1769 cs=0 l=1 rev1=1 crypto rx=0x7faa7000c7d0 tx=0x7faa7000cc90 comp rx=0 tx=0).stop 2023-08-09T14:51:28.305 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:28.303+0000 7faa7ecfa640 1 -- 172.21.15.43:0/2793333773 shutdown_connections 2023-08-09T14:51:28.305 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:28.303+0000 7faa7ecfa640 1 --2- 172.21.15.43:0/2793333773 >> [v2:172.21.15.203:6832/2566816332,v1:172.21.15.203:6833/2566816332] conn(0x7faa5c0705c0 0x7faa5c0729b0 unknown :-1 s=CLOSED pgs=5 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T14:51:28.305 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:28.303+0000 7faa7ecfa640 1 --2- 172.21.15.43:0/2793333773 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7faa7815e3c0 0x7faa780ccd00 unknown :-1 s=CLOSED pgs=1769 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T14:51:28.305 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:28.303+0000 7faa7ecfa640 1 --2- 172.21.15.43:0/2793333773 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7faa7815df60 0x7faa781f56b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T14:51:28.306 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:28.303+0000 7faa7ecfa640 1 --2- 172.21.15.43:0/2793333773 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7faa780cb2d0 0x7faa78140760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T14:51:28.306 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:28.303+0000 7faa7ecfa640 1 -- 172.21.15.43:0/2793333773 >> 172.21.15.43:0/2793333773 conn(0x7faa780aff50 msgr2=0x7faa780ce190 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-09T14:51:28.306 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:28.303+0000 7faa7ecfa640 1 -- 172.21.15.43:0/2793333773 shutdown_connections 2023-08-09T14:51:28.306 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:28.303+0000 7faa7ecfa640 1 -- 172.21.15.43:0/2793333773 wait complete. 2023-08-09T14:51:28.324 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-08-09T14:51:28.561 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:28.560+0000 7fe385386640 1 Processor -- start 2023-08-09T14:51:28.562 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:28.560+0000 7fe385386640 1 -- start start 2023-08-09T14:51:28.563 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:28.562+0000 7fe385386640 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7fe3800c8900 0x7fe3800cb9a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T14:51:28.564 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:28.562+0000 7fe385386640 1 -- --> v1:172.21.15.43:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe380070250 con 0x7fe380159400 2023-08-09T14:51:28.564 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:28.562+0000 7fe385386640 1 -- --> v1:172.21.15.203:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe3800c86c0 con 0x7fe3800c9560 2023-08-09T14:51:28.564 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:28.562+0000 7fe385386640 1 -- --> v2:172.21.15.203:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe38006fc60 con 0x7fe3800c8900 2023-08-09T14:51:28.564 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:28.562+0000 7fe37ffff640 1 --1- >> v1:172.21.15.43:6789/0 conn(0x7fe380159400 0x7fe3800c8e50 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.43:6789/0 says I am v1:172.21.15.43:54882/0 (socket says 172.21.15.43:54882) 2023-08-09T14:51:28.564 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:28.562+0000 7fe37ffff640 1 -- 172.21.15.43:0/2910303283 learned_addr learned my addr 172.21.15.43:0/2910303283 (peer_addr_for_me v1:172.21.15.43:0/0) 2023-08-09T14:51:28.565 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:28.563+0000 7fe384b85640 1 --2- 172.21.15.43:0/2910303283 >> v2:172.21.15.203:3300/0 conn(0x7fe3800c8900 0x7fe3800cb9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T14:51:28.565 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:28.563+0000 7fe37f7fe640 1 -- 172.21.15.43:0/2910303283 <== mon.0 v1:172.21.15.43:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1255888525 0 0) 0x7fe380070250 con 0x7fe380159400 2023-08-09T14:51:28.565 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:28.563+0000 7fe37f7fe640 1 -- 172.21.15.43:0/2910303283 --> v1:172.21.15.43:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fe36c003520 con 0x7fe380159400 2023-08-09T14:51:28.567 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:28.563+0000 7fe37f7fe640 1 -- 172.21.15.43:0/2910303283 <== mon.0 v1:172.21.15.43:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2112948170 0 0) 0x7fe36c003520 con 0x7fe380159400 2023-08-09T14:51:28.567 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:28.563+0000 7fe37f7fe640 1 -- 172.21.15.43:0/2910303283 >> v2:172.21.15.203:3300/0 conn(0x7fe3800c8900 msgr2=0x7fe3800cb9a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-09T14:51:28.567 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:28.563+0000 7fe37f7fe640 1 --2- 172.21.15.43:0/2910303283 >> v2:172.21.15.203:3300/0 conn(0x7fe3800c8900 0x7fe3800cb9a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T14:51:28.567 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:28.563+0000 7fe37f7fe640 1 -- 172.21.15.43:0/2910303283 >> v1:172.21.15.203:6789/0 conn(0x7fe3800c9560 legacy=0x7fe3800706e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T14:51:28.568 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:28.563+0000 7fe37f7fe640 1 -- 172.21.15.43:0/2910303283 --> v1:172.21.15.43:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe3800af290 con 0x7fe380159400 2023-08-09T14:51:28.568 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:28.563+0000 7fe37f7fe640 1 -- 172.21.15.43:0/2910303283 <== mon.0 v1:172.21.15.43:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1285150740 0 0) 0x7fe3700019f0 con 0x7fe380159400 2023-08-09T14:51:28.568 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:28.563+0000 7fe37f7fe640 1 -- 172.21.15.43:0/2910303283 >> v1:172.21.15.43:6789/0 conn(0x7fe380159400 legacy=0x7fe3800c8e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T14:51:28.568 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:28.563+0000 7fe37f7fe640 1 --2- 172.21.15.43:0/2910303283 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fe36c003970 0x7fe36c003d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T14:51:28.568 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:28.563+0000 7fe37f7fe640 1 --2- 172.21.15.43:0/2910303283 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fe36c0047c0 0x7fe36c004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T14:51:28.568 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:28.563+0000 7fe37f7fe640 1 --2- 172.21.15.43:0/2910303283 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fe36c005180 0x7fe36c00db90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T14:51:28.568 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:28.563+0000 7fe37f7fe640 1 -- 172.21.15.43:0/2910303283 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe38006fc60 con 0x7fe36c0047c0 2023-08-09T14:51:28.569 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:28.563+0000 7fe37f7fe640 1 -- 172.21.15.43:0/2910303283 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe36c00e330 con 0x7fe36c003970 2023-08-09T14:51:28.569 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:28.563+0000 7fe37f7fe640 1 -- 172.21.15.43:0/2910303283 --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe36c00e4f0 con 0x7fe36c005180 2023-08-09T14:51:28.569 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:28.564+0000 7fe3777fe640 1 --2- 172.21.15.43:0/2910303283 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fe36c0047c0 0x7fe36c004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T14:51:28.569 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:28.564+0000 7fe384b85640 1 --2- 172.21.15.43:0/2910303283 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fe36c005180 0x7fe36c00db90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T14:51:28.569 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:28.564+0000 7fe3777fe640 1 -- 172.21.15.43:0/2910303283 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fe36c005180 msgr2=0x7fe36c00db90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-09T14:51:28.569 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:28.564+0000 7fe37ffff640 1 --2- 172.21.15.43:0/2910303283 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fe36c003970 0x7fe36c003d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T14:51:28.569 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:28.564+0000 7fe3777fe640 1 --2- 172.21.15.43:0/2910303283 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fe36c005180 0x7fe36c00db90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T14:51:28.569 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:28.564+0000 7fe3777fe640 1 -- 172.21.15.43:0/2910303283 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fe36c003970 msgr2=0x7fe36c003d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-09T14:51:28.570 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:28.564+0000 7fe3777fe640 1 --2- 172.21.15.43:0/2910303283 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fe36c003970 0x7fe36c003d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T14:51:28.570 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:28.564+0000 7fe3777fe640 1 -- 172.21.15.43:0/2910303283 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe36c003520 con 0x7fe36c0047c0 2023-08-09T14:51:28.570 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:28.564+0000 7fe384b85640 1 --2- 172.21.15.43:0/2910303283 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fe36c005180 0x7fe36c00db90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-08-09T14:51:28.570 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:28.564+0000 7fe37ffff640 1 --2- 172.21.15.43:0/2910303283 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fe36c003970 0x7fe36c003d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-08-09T14:51:28.570 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:28.564+0000 7fe384b85640 1 -- 172.21.15.43:0/2910303283 reap_dead start 2023-08-09T14:51:28.570 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:28.565+0000 7fe3777fe640 1 --2- 172.21.15.43:0/2910303283 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fe36c0047c0 0x7fe36c004c40 secure :-1 s=READY pgs=1771 cs=0 l=1 rev1=1 crypto rx=0x7fe368001750 tx=0x7fe36c0057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=17561f68d0f828f1 server_cookie=0 in_seq=0 out_seq=0 2023-08-09T14:51:28.570 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:28.566+0000 7fe37f7fe640 1 -- 172.21.15.43:0/2910303283 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe3680052d0 con 0x7fe36c0047c0 2023-08-09T14:51:28.570 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:28.566+0000 7fe37f7fe640 1 -- 172.21.15.43:0/2910303283 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe36802e050 con 0x7fe36c0047c0 2023-08-09T14:51:28.571 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:28.566+0000 7fe37f7fe640 1 -- 172.21.15.43:0/2910303283 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe368003830 con 0x7fe36c0047c0 2023-08-09T14:51:28.572 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:28.566+0000 7fe385386640 1 -- 172.21.15.43:0/2910303283 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fe36c0047c0 msgr2=0x7fe36c004c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T14:51:28.573 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:28.566+0000 7fe385386640 1 --2- 172.21.15.43:0/2910303283 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fe36c0047c0 0x7fe36c004c40 secure :-1 s=READY pgs=1771 cs=0 l=1 rev1=1 crypto rx=0x7fe368001750 tx=0x7fe36c0057f0 comp rx=0 tx=0).stop 2023-08-09T14:51:28.573 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:28.566+0000 7fe385386640 1 -- 172.21.15.43:0/2910303283 shutdown_connections 2023-08-09T14:51:28.573 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:28.566+0000 7fe385386640 1 --2- 172.21.15.43:0/2910303283 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fe36c0047c0 0x7fe36c004c40 unknown :-1 s=CLOSED pgs=1771 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T14:51:28.573 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:28.566+0000 7fe385386640 1 -- 172.21.15.43:0/2910303283 >> 172.21.15.43:0/2910303283 conn(0x7fe3800cddd0 msgr2=0x7fe3800ce1e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-09T14:51:28.573 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:28.566+0000 7fe385386640 1 -- 172.21.15.43:0/2910303283 shutdown_connections 2023-08-09T14:51:28.573 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:28.566+0000 7fe385386640 1 -- 172.21.15.43:0/2910303283 wait complete. 2023-08-09T14:51:28.574 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:28.568+0000 7fe385386640 1 Processor -- start 2023-08-09T14:51:28.574 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:28.568+0000 7fe385386640 1 -- start start 2023-08-09T14:51:28.574 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:28.569+0000 7fe385386640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fe36c0047c0 0x7fe3800c5820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T14:51:28.574 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:28.569+0000 7fe385386640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fe3801569c0 0x7fe380070950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T14:51:28.574 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:28.569+0000 7fe385386640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fe38015a250 0x7fe38014b020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T14:51:28.574 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:28.569+0000 7fe385386640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe3800c5390 con 0x7fe38015a250 2023-08-09T14:51:28.574 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:28.569+0000 7fe385386640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe38015abc0 con 0x7fe36c0047c0 2023-08-09T14:51:28.574 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:28.569+0000 7fe385386640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe38015aea0 con 0x7fe3801569c0 2023-08-09T14:51:28.575 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:28.570+0000 7fe384b85640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fe38015a250 0x7fe38014b020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T14:51:28.575 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:28.570+0000 7fe384b85640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fe38015a250 0x7fe38014b020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:39830/0 (socket says 172.21.15.43:39830) 2023-08-09T14:51:28.575 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:28.570+0000 7fe384b85640 1 -- 172.21.15.43:0/4120748576 learned_addr learned my addr 172.21.15.43:0/4120748576 (peer_addr_for_me v2:172.21.15.43:0/0) 2023-08-09T14:51:28.575 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:28.570+0000 7fe3777fe640 1 --2- 172.21.15.43:0/4120748576 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fe3801569c0 0x7fe380070950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T14:51:28.575 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:28.570+0000 7fe384b85640 1 -- 172.21.15.43:0/4120748576 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fe3801569c0 msgr2=0x7fe380070950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T14:51:28.575 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:28.570+0000 7fe384b85640 1 --2- 172.21.15.43:0/4120748576 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fe3801569c0 0x7fe380070950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T14:51:28.575 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:28.570+0000 7fe384b85640 1 -- 172.21.15.43:0/4120748576 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fe36c0047c0 msgr2=0x7fe3800c5820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T14:51:28.576 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:28.570+0000 7fe37ffff640 1 --2- 172.21.15.43:0/4120748576 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fe36c0047c0 0x7fe3800c5820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T14:51:28.576 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:28.570+0000 7fe384b85640 1 --2- 172.21.15.43:0/4120748576 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fe36c0047c0 0x7fe3800c5820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T14:51:28.576 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:28.570+0000 7fe384b85640 1 -- 172.21.15.43:0/4120748576 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe3800c5020 con 0x7fe38015a250 2023-08-09T14:51:28.576 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:28.570+0000 7fe384b85640 1 --2- 172.21.15.43:0/4120748576 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fe38015a250 0x7fe38014b020 secure :-1 s=READY pgs=1772 cs=0 l=1 rev1=1 crypto rx=0x7fe38006d620 tx=0x7fe378010c20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-09T14:51:28.576 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:28.571+0000 7fe37d7fa640 1 -- 172.21.15.43:0/4120748576 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe37800e840 con 0x7fe38015a250 2023-08-09T14:51:28.576 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:28.571+0000 7fe37d7fa640 1 -- 172.21.15.43:0/4120748576 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe37800e9a0 con 0x7fe38015a250 2023-08-09T14:51:28.576 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:28.571+0000 7fe385386640 1 -- 172.21.15.43:0/4120748576 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe3800c5d60 con 0x7fe38015a250 2023-08-09T14:51:28.576 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:28.571+0000 7fe37d7fa640 1 -- 172.21.15.43:0/4120748576 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe37800ed30 con 0x7fe38015a250 2023-08-09T14:51:28.577 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:28.571+0000 7fe385386640 1 -- 172.21.15.43:0/4120748576 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe380071070 con 0x7fe38015a250 2023-08-09T14:51:28.577 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:28.572+0000 7fe385386640 1 -- 172.21.15.43:0/4120748576 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe38015b0c0 con 0x7fe38015a250 2023-08-09T14:51:28.578 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:28.573+0000 7fe37d7fa640 1 -- 172.21.15.43:0/4120748576 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 77) v1 ==== 88179+0+0 (secure 0 0 0) 0x7fe37801c400 con 0x7fe38015a250 2023-08-09T14:51:28.578 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:28.575+0000 7fe37d7fa640 1 --2- 172.21.15.43:0/4120748576 >> [v2:172.21.15.203:6832/2566816332,v1:172.21.15.203:6833/2566816332] conn(0x7fe360069ea0 0x7fe36006c360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T14:51:28.578 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:28.575+0000 7fe37d7fa640 1 -- 172.21.15.43:0/4120748576 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(160..160 src has 1..160) v4 ==== 7216+0+0 (secure 0 0 0) 0x7fe378088490 con 0x7fe38015a250 2023-08-09T14:51:28.578 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:28.575+0000 7fe37ffff640 1 --2- 172.21.15.43:0/4120748576 >> [v2:172.21.15.203:6832/2566816332,v1:172.21.15.203:6833/2566816332] conn(0x7fe360069ea0 0x7fe36006c360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T14:51:28.578 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:28.576+0000 7fe37ffff640 1 --2- 172.21.15.43:0/4120748576 >> [v2:172.21.15.203:6832/2566816332,v1:172.21.15.203:6833/2566816332] conn(0x7fe360069ea0 0x7fe36006c360 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7fe370001580 tx=0x7fe370009040 comp rx=0 tx=0).ready entity=mgr.7765 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-09T14:51:28.580 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:28.579+0000 7fe37d7fa640 1 -- 172.21.15.43:0/4120748576 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178066 (secure 0 0 0) 0x7fe3780574d0 con 0x7fe38015a250 2023-08-09T14:51:28.776 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:28.774+0000 7fe385386640 1 -- 172.21.15.43:0/4120748576 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7fe380071070 con 0x7fe38015a250 2023-08-09T14:51:28.776 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:28.775+0000 7fe37d7fa640 1 -- 172.21.15.43:0/4120748576 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v160) v1 ==== 93+0+10 (secure 0 0 0) 0x7fe380071070 con 0x7fe38015a250 2023-08-09T14:51:28.777 INFO:teuthology.orchestra.run.smithi043.stdout:pg_num: 1 2023-08-09T14:51:28.779 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:28.778+0000 7fe385386640 1 -- 172.21.15.43:0/4120748576 >> [v2:172.21.15.203:6832/2566816332,v1:172.21.15.203:6833/2566816332] conn(0x7fe360069ea0 msgr2=0x7fe36006c360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T14:51:28.780 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:28.778+0000 7fe385386640 1 --2- 172.21.15.43:0/4120748576 >> [v2:172.21.15.203:6832/2566816332,v1:172.21.15.203:6833/2566816332] conn(0x7fe360069ea0 0x7fe36006c360 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7fe370001580 tx=0x7fe370009040 comp rx=0 tx=0).stop 2023-08-09T14:51:28.780 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:28.778+0000 7fe385386640 1 -- 172.21.15.43:0/4120748576 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fe38015a250 msgr2=0x7fe38014b020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T14:51:28.780 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:28.778+0000 7fe385386640 1 --2- 172.21.15.43:0/4120748576 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fe38015a250 0x7fe38014b020 secure :-1 s=READY pgs=1772 cs=0 l=1 rev1=1 crypto rx=0x7fe38006d620 tx=0x7fe378010c20 comp rx=0 tx=0).stop 2023-08-09T14:51:28.780 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:28.778+0000 7fe385386640 1 -- 172.21.15.43:0/4120748576 shutdown_connections 2023-08-09T14:51:28.780 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:28.778+0000 7fe385386640 1 --2- 172.21.15.43:0/4120748576 >> [v2:172.21.15.203:6832/2566816332,v1:172.21.15.203:6833/2566816332] conn(0x7fe360069ea0 0x7fe36006c360 unknown :-1 s=CLOSED pgs=22 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T14:51:28.781 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:28.778+0000 7fe385386640 1 --2- 172.21.15.43:0/4120748576 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fe38015a250 0x7fe38014b020 unknown :-1 s=CLOSED pgs=1772 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T14:51:28.781 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:28.778+0000 7fe385386640 1 --2- 172.21.15.43:0/4120748576 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fe3801569c0 0x7fe380070950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T14:51:28.781 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:28.778+0000 7fe385386640 1 --2- 172.21.15.43:0/4120748576 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fe36c0047c0 0x7fe3800c5820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T14:51:28.781 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:28.778+0000 7fe385386640 1 -- 172.21.15.43:0/4120748576 >> 172.21.15.43:0/4120748576 conn(0x7fe3800cddd0 msgr2=0x7fe3800b2a00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-09T14:51:28.781 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:28.778+0000 7fe385386640 1 -- 172.21.15.43:0/4120748576 shutdown_connections 2023-08-09T14:51:28.781 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:28.778+0000 7fe385386640 1 -- 172.21.15.43:0/4120748576 wait complete. 2023-08-09T14:51:28.799 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-08-09T14:51:29.029 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:29.027+0000 7efe15bc8640 1 Processor -- start 2023-08-09T14:51:29.029 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:29.028+0000 7efe15bc8640 1 -- start start 2023-08-09T14:51:29.031 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:29.029+0000 7efe15bc8640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7efe10143c20 0x7efe1006f540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T14:51:29.031 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:29.029+0000 7efe15bc8640 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7efe1006cfe0 0x7efe100c8b50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T14:51:29.031 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:29.030+0000 7efe15bc8640 1 -- --> v1:172.21.15.43:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7efe100c9090 con 0x7efe100c86f0 2023-08-09T14:51:29.031 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:29.030+0000 7efe15bc8640 1 -- --> v2:172.21.15.203:3300/0 -- mon_getmap magic: 0 v1 -- 0x7efe10144000 con 0x7efe1006cfe0 2023-08-09T14:51:29.031 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:29.030+0000 7efe15bc8640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7efe1006c440 con 0x7efe10143c20 2023-08-09T14:51:29.033 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:29.030+0000 7efe0effd640 1 --1- >> v1:172.21.15.43:6789/0 conn(0x7efe100c86f0 0x7efe1006c8d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.43:6789/0 says I am v1:172.21.15.43:54898/0 (socket says 172.21.15.43:54898) 2023-08-09T14:51:29.033 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:29.030+0000 7efe0effd640 1 -- 172.21.15.43:0/1403865720 learned_addr learned my addr 172.21.15.43:0/1403865720 (peer_addr_for_me v1:172.21.15.43:0/0) 2023-08-09T14:51:29.033 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:29.030+0000 7efe0e7fc640 1 -- 172.21.15.43:0/1403865720 <== mon.0 v1:172.21.15.43:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3265475231 0 0) 0x7efe100c9090 con 0x7efe100c86f0 2023-08-09T14:51:29.033 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:29.030+0000 7efe0e7fc640 1 -- 172.21.15.43:0/1403865720 --> v1:172.21.15.43:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7efdfc003520 con 0x7efe100c86f0 2023-08-09T14:51:29.034 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:29.031+0000 7efe0f7fe640 1 --2- 172.21.15.43:0/1403865720 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7efe10143c20 0x7efe1006f540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T14:51:29.034 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:29.031+0000 7efe0ffff640 1 --2- 172.21.15.43:0/1403865720 >> v2:172.21.15.203:3300/0 conn(0x7efe1006cfe0 0x7efe100c8b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T14:51:29.034 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:29.031+0000 7efe0e7fc640 1 -- 172.21.15.43:0/1403865720 <== mon.0 v1:172.21.15.43:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3514818773 0 0) 0x7efdfc003520 con 0x7efe100c86f0 2023-08-09T14:51:29.034 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:29.031+0000 7efe0e7fc640 1 -- 172.21.15.43:0/1403865720 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7efe10143c20 msgr2=0x7efe1006f540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-09T14:51:29.034 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:29.031+0000 7efe0e7fc640 1 --2- 172.21.15.43:0/1403865720 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7efe10143c20 0x7efe1006f540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T14:51:29.034 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:29.031+0000 7efe0e7fc640 1 -- 172.21.15.43:0/1403865720 >> v2:172.21.15.203:3300/0 conn(0x7efe1006cfe0 msgr2=0x7efe100c8b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-09T14:51:29.034 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:29.031+0000 7efe0e7fc640 1 --2- 172.21.15.43:0/1403865720 >> v2:172.21.15.203:3300/0 conn(0x7efe1006cfe0 0x7efe100c8b50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T14:51:29.035 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:29.031+0000 7efe0e7fc640 1 -- 172.21.15.43:0/1403865720 --> v1:172.21.15.43:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efe1006b830 con 0x7efe100c86f0 2023-08-09T14:51:29.035 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:29.031+0000 7efe0e7fc640 1 -- 172.21.15.43:0/1403865720 <== mon.0 v1:172.21.15.43:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1285150740 0 0) 0x7efdf8001670 con 0x7efe100c86f0 2023-08-09T14:51:29.035 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:29.031+0000 7efe0e7fc640 1 -- 172.21.15.43:0/1403865720 >> v1:172.21.15.43:6789/0 conn(0x7efe100c86f0 legacy=0x7efe1006c8d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T14:51:29.035 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:29.031+0000 7efe0e7fc640 1 --2- 172.21.15.43:0/1403865720 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7efdfc0039e0 0x7efdfc003e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T14:51:29.035 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:29.031+0000 7efe0e7fc640 1 --2- 172.21.15.43:0/1403865720 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7efdfc0047c0 0x7efdfc004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T14:51:29.035 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:29.031+0000 7efe0e7fc640 1 --2- 172.21.15.43:0/1403865720 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7efe10143c20 0x7efdfc0051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T14:51:29.035 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:29.031+0000 7efe0e7fc640 1 -- 172.21.15.43:0/1403865720 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efe10144000 con 0x7efdfc0047c0 2023-08-09T14:51:29.035 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:29.031+0000 7efe0e7fc640 1 -- 172.21.15.43:0/1403865720 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efe1006c440 con 0x7efdfc0039e0 2023-08-09T14:51:29.036 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:29.031+0000 7efe0e7fc640 1 -- 172.21.15.43:0/1403865720 --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7efdfc00bb30 con 0x7efe10143c20 2023-08-09T14:51:29.036 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:29.031+0000 7efe0effd640 1 --2- 172.21.15.43:0/1403865720 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7efdfc0047c0 0x7efdfc004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T14:51:29.036 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:29.032+0000 7efe0ffff640 1 --2- 172.21.15.43:0/1403865720 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7efe10143c20 0x7efdfc0051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T14:51:29.036 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:29.032+0000 7efe0effd640 1 -- 172.21.15.43:0/1403865720 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7efe10143c20 msgr2=0x7efdfc0051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-09T14:51:29.036 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:29.032+0000 7efe0effd640 1 --2- 172.21.15.43:0/1403865720 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7efe10143c20 0x7efdfc0051c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T14:51:29.036 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:29.032+0000 7efe0effd640 1 -- 172.21.15.43:0/1403865720 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7efdfc0039e0 msgr2=0x7efdfc003e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-09T14:51:29.036 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:29.032+0000 7efe0effd640 1 --2- 172.21.15.43:0/1403865720 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7efdfc0039e0 0x7efdfc003e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T14:51:29.036 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:29.032+0000 7efe0effd640 1 -- 172.21.15.43:0/1403865720 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efdfc003520 con 0x7efdfc0047c0 2023-08-09T14:51:29.038 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:29.033+0000 7efe0effd640 1 --2- 172.21.15.43:0/1403865720 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7efdfc0047c0 0x7efdfc004c40 secure :-1 s=READY pgs=1774 cs=0 l=1 rev1=1 crypto rx=0x7efdf800c240 tx=0x7efdfc0057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3794f813cc02ae18 server_cookie=0 in_seq=0 out_seq=0 2023-08-09T14:51:29.038 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:29.033+0000 7efe0e7fc640 1 -- 172.21.15.43:0/1403865720 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efdf802cb00 con 0x7efdfc0047c0 2023-08-09T14:51:29.038 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:29.033+0000 7efe0e7fc640 1 -- 172.21.15.43:0/1403865720 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efdf802cc60 con 0x7efdfc0047c0 2023-08-09T14:51:29.038 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:29.033+0000 7efe0e7fc640 1 -- 172.21.15.43:0/1403865720 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efdf8005af0 con 0x7efdfc0047c0 2023-08-09T14:51:29.038 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:29.034+0000 7efe15bc8640 1 -- 172.21.15.43:0/1403865720 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7efdfc0047c0 msgr2=0x7efdfc004c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T14:51:29.038 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:29.034+0000 7efe15bc8640 1 --2- 172.21.15.43:0/1403865720 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7efdfc0047c0 0x7efdfc004c40 secure :-1 s=READY pgs=1774 cs=0 l=1 rev1=1 crypto rx=0x7efdf800c240 tx=0x7efdfc0057f0 comp rx=0 tx=0).stop 2023-08-09T14:51:29.038 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:29.034+0000 7efe0ffff640 1 -- 172.21.15.43:0/1403865720 reap_dead start 2023-08-09T14:51:29.039 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:29.034+0000 7efe15bc8640 1 -- 172.21.15.43:0/1403865720 shutdown_connections 2023-08-09T14:51:29.039 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:29.034+0000 7efe15bc8640 1 -- 172.21.15.43:0/1403865720 >> 172.21.15.43:0/1403865720 conn(0x7efe100ce0a0 msgr2=0x7efe100ce480 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-09T14:51:29.039 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:29.034+0000 7efe15bc8640 1 -- 172.21.15.43:0/1403865720 shutdown_connections 2023-08-09T14:51:29.039 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:29.034+0000 7efe15bc8640 1 -- 172.21.15.43:0/1403865720 wait complete. 2023-08-09T14:51:29.039 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:29.036+0000 7efe15bc8640 1 Processor -- start 2023-08-09T14:51:29.039 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:29.036+0000 7efe15bc8640 1 -- start start 2023-08-09T14:51:29.041 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:29.037+0000 7efe15bc8640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7efdfc0047c0 0x7efe10150cd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T14:51:29.041 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:29.037+0000 7efe15bc8640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7efe100b2c00 0x7efe100c6470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T14:51:29.041 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:29.037+0000 7efe15bc8640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7efe1006f4d0 0x7efe1006f950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T14:51:29.041 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:29.037+0000 7efe15bc8640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efe1006fe90 con 0x7efe1006f4d0 2023-08-09T14:51:29.041 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:29.037+0000 7efe15bc8640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efe1013c3e0 con 0x7efe100b2c00 2023-08-09T14:51:29.041 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:29.037+0000 7efe15bc8640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7efe101efe90 con 0x7efdfc0047c0 2023-08-09T14:51:29.042 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:29.037+0000 7efe0ffff640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7efe1006f4d0 0x7efe1006f950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T14:51:29.043 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:29.037+0000 7efe0ffff640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7efe1006f4d0 0x7efe1006f950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:39852/0 (socket says 172.21.15.43:39852) 2023-08-09T14:51:29.043 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:29.037+0000 7efe0ffff640 1 -- 172.21.15.43:0/2936691960 learned_addr learned my addr 172.21.15.43:0/2936691960 (peer_addr_for_me v2:172.21.15.43:0/0) 2023-08-09T14:51:29.043 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:29.038+0000 7efe0f7fe640 1 --2- 172.21.15.43:0/2936691960 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7efdfc0047c0 0x7efe10150cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T14:51:29.043 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:29.038+0000 7efe0effd640 1 --2- 172.21.15.43:0/2936691960 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7efe100b2c00 0x7efe100c6470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T14:51:29.043 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:29.038+0000 7efe0ffff640 1 -- 172.21.15.43:0/2936691960 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7efdfc0047c0 msgr2=0x7efe10150cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T14:51:29.043 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:29.038+0000 7efe0ffff640 1 --2- 172.21.15.43:0/2936691960 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7efdfc0047c0 0x7efe10150cd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T14:51:29.043 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:29.038+0000 7efe0ffff640 1 -- 172.21.15.43:0/2936691960 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7efe100b2c00 msgr2=0x7efe100c6470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T14:51:29.044 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:29.038+0000 7efe0ffff640 1 --2- 172.21.15.43:0/2936691960 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7efe100b2c00 0x7efe100c6470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T14:51:29.044 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:29.038+0000 7efe0ffff640 1 -- 172.21.15.43:0/2936691960 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efe100b3180 con 0x7efe1006f4d0 2023-08-09T14:51:29.044 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:29.038+0000 7efe0ffff640 1 --2- 172.21.15.43:0/2936691960 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7efe1006f4d0 0x7efe1006f950 secure :-1 s=READY pgs=1775 cs=0 l=1 rev1=1 crypto rx=0x7efe100caad0 tx=0x7efe0800e8b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-09T14:51:29.044 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:29.038+0000 7efe07fff640 1 -- 172.21.15.43:0/2936691960 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efe0800c070 con 0x7efe1006f4d0 2023-08-09T14:51:29.044 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:29.038+0000 7efe07fff640 1 -- 172.21.15.43:0/2936691960 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efe08004020 con 0x7efe1006f4d0 2023-08-09T14:51:29.044 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:29.038+0000 7efe07fff640 1 -- 172.21.15.43:0/2936691960 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efe080043d0 con 0x7efe1006f4d0 2023-08-09T14:51:29.044 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:29.038+0000 7efe15bc8640 1 -- 172.21.15.43:0/2936691960 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7efe100b3460 con 0x7efe1006f4d0 2023-08-09T14:51:29.044 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:29.039+0000 7efe15bc8640 1 -- 172.21.15.43:0/2936691960 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7efe100b3a20 con 0x7efe1006f4d0 2023-08-09T14:51:29.045 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:29.040+0000 7efe15bc8640 1 -- 172.21.15.43:0/2936691960 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7efe100739f0 con 0x7efe1006f4d0 2023-08-09T14:51:29.045 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:29.041+0000 7efe07fff640 1 -- 172.21.15.43:0/2936691960 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 77) v1 ==== 88179+0+0 (secure 0 0 0) 0x7efe08003610 con 0x7efe1006f4d0 2023-08-09T14:51:29.046 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:29.042+0000 7efe07fff640 1 --2- 172.21.15.43:0/2936691960 >> [v2:172.21.15.203:6832/2566816332,v1:172.21.15.203:6833/2566816332] conn(0x7efdf0069ea0 0x7efdf006c360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T14:51:29.046 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:29.042+0000 7efe07fff640 1 -- 172.21.15.43:0/2936691960 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(160..160 src has 1..160) v4 ==== 7216+0+0 (secure 0 0 0) 0x7efe08087d90 con 0x7efe1006f4d0 2023-08-09T14:51:29.046 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:29.042+0000 7efe0f7fe640 1 --2- 172.21.15.43:0/2936691960 >> [v2:172.21.15.203:6832/2566816332,v1:172.21.15.203:6833/2566816332] conn(0x7efdf0069ea0 0x7efdf006c360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T14:51:29.046 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:29.044+0000 7efe0f7fe640 1 --2- 172.21.15.43:0/2936691960 >> [v2:172.21.15.203:6832/2566816332,v1:172.21.15.203:6833/2566816332] conn(0x7efdf0069ea0 0x7efdf006c360 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7efe000070b0 tx=0x7efe0000b040 comp rx=0 tx=0).ready entity=mgr.7765 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-09T14:51:29.048 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:29.046+0000 7efe07fff640 1 -- 172.21.15.43:0/2936691960 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178066 (secure 0 0 0) 0x7efe08053290 con 0x7efe1006f4d0 2023-08-09T14:51:29.243 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:29.242+0000 7efe15bc8640 1 -- 172.21.15.43:0/2936691960 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7efe1006c0a0 con 0x7efe1006f4d0 2023-08-09T14:51:29.244 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:29.243+0000 7efe07fff640 1 -- 172.21.15.43:0/2936691960 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v160) v1 ==== 104+0+11 (secure 0 0 0) 0x7efe080568b0 con 0x7efe1006f4d0 2023-08-09T14:51:29.245 INFO:teuthology.orchestra.run.smithi043.stdout:pg_num: 32 2023-08-09T14:51:29.247 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:29.245+0000 7efe15bc8640 1 -- 172.21.15.43:0/2936691960 >> [v2:172.21.15.203:6832/2566816332,v1:172.21.15.203:6833/2566816332] conn(0x7efdf0069ea0 msgr2=0x7efdf006c360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T14:51:29.247 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:29.245+0000 7efe15bc8640 1 --2- 172.21.15.43:0/2936691960 >> [v2:172.21.15.203:6832/2566816332,v1:172.21.15.203:6833/2566816332] conn(0x7efdf0069ea0 0x7efdf006c360 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7efe000070b0 tx=0x7efe0000b040 comp rx=0 tx=0).stop 2023-08-09T14:51:29.247 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:29.245+0000 7efe15bc8640 1 -- 172.21.15.43:0/2936691960 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7efe1006f4d0 msgr2=0x7efe1006f950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T14:51:29.247 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:29.245+0000 7efe15bc8640 1 --2- 172.21.15.43:0/2936691960 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7efe1006f4d0 0x7efe1006f950 secure :-1 s=READY pgs=1775 cs=0 l=1 rev1=1 crypto rx=0x7efe100caad0 tx=0x7efe0800e8b0 comp rx=0 tx=0).stop 2023-08-09T14:51:29.248 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:29.246+0000 7efe15bc8640 1 -- 172.21.15.43:0/2936691960 shutdown_connections 2023-08-09T14:51:29.248 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:29.246+0000 7efe15bc8640 1 --2- 172.21.15.43:0/2936691960 >> [v2:172.21.15.203:6832/2566816332,v1:172.21.15.203:6833/2566816332] conn(0x7efdf0069ea0 0x7efdf006c360 unknown :-1 s=CLOSED pgs=23 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T14:51:29.248 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:29.246+0000 7efe15bc8640 1 --2- 172.21.15.43:0/2936691960 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7efe1006f4d0 0x7efe1006f950 unknown :-1 s=CLOSED pgs=1775 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T14:51:29.248 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:29.246+0000 7efe15bc8640 1 --2- 172.21.15.43:0/2936691960 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7efe100b2c00 0x7efe100c6470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T14:51:29.248 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:29.246+0000 7efe15bc8640 1 --2- 172.21.15.43:0/2936691960 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7efdfc0047c0 0x7efe10150cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T14:51:29.249 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:29.246+0000 7efe15bc8640 1 -- 172.21.15.43:0/2936691960 >> 172.21.15.43:0/2936691960 conn(0x7efe100ce0a0 msgr2=0x7efe100adf80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-09T14:51:29.249 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:29.246+0000 7efe15bc8640 1 -- 172.21.15.43:0/2936691960 shutdown_connections 2023-08-09T14:51:29.249 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:29.246+0000 7efe15bc8640 1 -- 172.21.15.43:0/2936691960 wait complete. 2023-08-09T14:51:29.269 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-08-09T14:51:29.508 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:29.506+0000 7f295e81f640 1 Processor -- start 2023-08-09T14:51:29.508 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:29.506+0000 7f295e81f640 1 -- start start 2023-08-09T14:51:29.509 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:29.508+0000 7f295e81f640 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7f295815c550 0x7f29580cd6d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T14:51:29.510 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:29.508+0000 7f295e81f640 1 -- --> v1:172.21.15.43:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f29580b7bd0 con 0x7f29580b45f0 2023-08-09T14:51:29.510 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:29.508+0000 7f295e81f640 1 -- --> v1:172.21.15.203:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f29580d1c00 con 0x7f29580d04b0 2023-08-09T14:51:29.510 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:29.508+0000 7f295e81f640 1 -- --> v2:172.21.15.203:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f295815c930 con 0x7f295815c550 2023-08-09T14:51:29.511 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:29.508+0000 7f295cdbc640 1 --1- >> v1:172.21.15.43:6789/0 conn(0x7f29580b45f0 0x7f29580b59d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.43:6789/0 says I am v1:172.21.15.43:54906/0 (socket says 172.21.15.43:54906) 2023-08-09T14:51:29.511 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:29.509+0000 7f295cdbc640 1 -- 172.21.15.43:0/564273687 learned_addr learned my addr 172.21.15.43:0/564273687 (peer_addr_for_me v1:172.21.15.43:0/0) 2023-08-09T14:51:29.512 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:29.509+0000 7f2957fff640 1 --2- 172.21.15.43:0/564273687 >> v2:172.21.15.203:3300/0 conn(0x7f295815c550 0x7f29580cd6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T14:51:29.512 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:29.509+0000 7f29577fe640 1 -- 172.21.15.43:0/564273687 <== mon.0 v1:172.21.15.43:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1300982891 0 0) 0x7f29580b7bd0 con 0x7f29580b45f0 2023-08-09T14:51:29.512 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:29.509+0000 7f29577fe640 1 -- 172.21.15.43:0/564273687 --> v1:172.21.15.43:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f2944003520 con 0x7f29580b45f0 2023-08-09T14:51:29.512 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:29.509+0000 7f29577fe640 1 -- 172.21.15.43:0/564273687 <== mon.0 v1:172.21.15.43:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 549494031 0 0) 0x7f2944003520 con 0x7f29580b45f0 2023-08-09T14:51:29.512 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:29.509+0000 7f29577fe640 1 -- 172.21.15.43:0/564273687 >> v2:172.21.15.203:3300/0 conn(0x7f295815c550 msgr2=0x7f29580cd6d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-09T14:51:29.512 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:29.509+0000 7f29577fe640 1 --2- 172.21.15.43:0/564273687 >> v2:172.21.15.203:3300/0 conn(0x7f295815c550 0x7f29580cd6d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T14:51:29.512 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:29.509+0000 7f29577fe640 1 -- 172.21.15.43:0/564273687 >> v1:172.21.15.203:6789/0 conn(0x7f29580d04b0 legacy=0x7f29580b3ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T14:51:29.513 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:29.510+0000 7f29577fe640 1 -- 172.21.15.43:0/564273687 --> v1:172.21.15.43:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f29580b54f0 con 0x7f29580b45f0 2023-08-09T14:51:29.513 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:29.510+0000 7f29577fe640 1 -- 172.21.15.43:0/564273687 <== mon.0 v1:172.21.15.43:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1285150740 0 0) 0x7f2950001a10 con 0x7f29580b45f0 2023-08-09T14:51:29.513 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:29.510+0000 7f29577fe640 1 -- 172.21.15.43:0/564273687 >> v1:172.21.15.43:6789/0 conn(0x7f29580b45f0 legacy=0x7f29580b59d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T14:51:29.513 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:29.510+0000 7f29577fe640 1 --2- 172.21.15.43:0/564273687 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f2944003970 0x7f2944003d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T14:51:29.513 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:29.510+0000 7f29577fe640 1 --2- 172.21.15.43:0/564273687 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f29440047c0 0x7f2944004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T14:51:29.513 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:29.510+0000 7f29577fe640 1 --2- 172.21.15.43:0/564273687 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f2944005180 0x7f294400b980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T14:51:29.513 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:29.510+0000 7f29577fe640 1 -- 172.21.15.43:0/564273687 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f295815c930 con 0x7f29440047c0 2023-08-09T14:51:29.514 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:29.510+0000 7f29577fe640 1 -- 172.21.15.43:0/564273687 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f294400c120 con 0x7f2944005180 2023-08-09T14:51:29.514 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:29.510+0000 7f29577fe640 1 -- 172.21.15.43:0/564273687 --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f294400c2e0 con 0x7f2944003970 2023-08-09T14:51:29.514 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:29.510+0000 7f294ffff640 1 --2- 172.21.15.43:0/564273687 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f29440047c0 0x7f2944004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T14:51:29.515 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:29.510+0000 7f2957fff640 1 --2- 172.21.15.43:0/564273687 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f2944003970 0x7f2944003d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T14:51:29.515 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:29.510+0000 7f294ffff640 1 -- 172.21.15.43:0/564273687 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f2944003970 msgr2=0x7f2944003d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-09T14:51:29.515 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:29.510+0000 7f294ffff640 1 --2- 172.21.15.43:0/564273687 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f2944003970 0x7f2944003d90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T14:51:29.516 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:29.510+0000 7f294ffff640 1 -- 172.21.15.43:0/564273687 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f2944005180 msgr2=0x7f294400b980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-09T14:51:29.516 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:29.510+0000 7f294ffff640 1 --2- 172.21.15.43:0/564273687 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f2944005180 0x7f294400b980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T14:51:29.516 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:29.510+0000 7f294ffff640 1 -- 172.21.15.43:0/564273687 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2944003520 con 0x7f29440047c0 2023-08-09T14:51:29.516 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:29.510+0000 7f295cdbc640 1 -- 172.21.15.43:0/564273687 reap_dead start 2023-08-09T14:51:29.516 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:29.512+0000 7f294ffff640 1 --2- 172.21.15.43:0/564273687 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f29440047c0 0x7f2944004c40 secure :-1 s=READY pgs=1778 cs=0 l=1 rev1=1 crypto rx=0x7f2940001750 tx=0x7f29440057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e2704573e801e84a server_cookie=0 in_seq=0 out_seq=0 2023-08-09T14:51:29.516 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:29.512+0000 7f29577fe640 1 -- 172.21.15.43:0/564273687 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f29400052d0 con 0x7f29440047c0 2023-08-09T14:51:29.517 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:29.512+0000 7f29577fe640 1 -- 172.21.15.43:0/564273687 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f294002e050 con 0x7f29440047c0 2023-08-09T14:51:29.517 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:29.512+0000 7f29577fe640 1 -- 172.21.15.43:0/564273687 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2940003830 con 0x7f29440047c0 2023-08-09T14:51:29.517 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:29.512+0000 7f295e81f640 1 -- 172.21.15.43:0/564273687 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f29440047c0 msgr2=0x7f2944004c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T14:51:29.517 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:29.512+0000 7f295e81f640 1 --2- 172.21.15.43:0/564273687 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f29440047c0 0x7f2944004c40 secure :-1 s=READY pgs=1778 cs=0 l=1 rev1=1 crypto rx=0x7f2940001750 tx=0x7f29440057f0 comp rx=0 tx=0).stop 2023-08-09T14:51:29.517 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:29.513+0000 7f295e81f640 1 -- 172.21.15.43:0/564273687 shutdown_connections 2023-08-09T14:51:29.517 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:29.513+0000 7f295e81f640 1 --2- 172.21.15.43:0/564273687 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f29440047c0 0x7f2944004c40 unknown :-1 s=CLOSED pgs=1778 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T14:51:29.517 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:29.513+0000 7f295e81f640 1 -- 172.21.15.43:0/564273687 >> 172.21.15.43:0/564273687 conn(0x7f2958071520 msgr2=0x7f29580ceee0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-09T14:51:29.518 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:29.513+0000 7f295e81f640 1 -- 172.21.15.43:0/564273687 shutdown_connections 2023-08-09T14:51:29.518 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:29.513+0000 7f295e81f640 1 -- 172.21.15.43:0/564273687 wait complete. 2023-08-09T14:51:29.519 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:29.514+0000 7f295e81f640 1 Processor -- start 2023-08-09T14:51:29.519 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:29.515+0000 7f295e81f640 1 -- start start 2023-08-09T14:51:29.520 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:29.516+0000 7f295e81f640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f29440047c0 0x7f2958151c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T14:51:29.520 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:29.516+0000 7f295e81f640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f29580ca1f0 0x7f29580ca670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T14:51:29.520 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:29.516+0000 7f295e81f640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f2958153be0 0x7f2958154060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T14:51:29.520 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:29.516+0000 7f295e81f640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f29581545a0 con 0x7f29440047c0 2023-08-09T14:51:29.520 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:29.516+0000 7f295e81f640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2958158e80 con 0x7f29580ca1f0 2023-08-09T14:51:29.520 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:29.516+0000 7f295e81f640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2958152150 con 0x7f2958153be0 2023-08-09T14:51:29.520 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:29.516+0000 7f2957fff640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f29440047c0 0x7f2958151c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T14:51:29.521 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:29.516+0000 7f2957fff640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f29440047c0 0x7f2958151c10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:39868/0 (socket says 172.21.15.43:39868) 2023-08-09T14:51:29.521 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:29.516+0000 7f2957fff640 1 -- 172.21.15.43:0/2778387976 learned_addr learned my addr 172.21.15.43:0/2778387976 (peer_addr_for_me v2:172.21.15.43:0/0) 2023-08-09T14:51:29.521 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:29.517+0000 7f2957fff640 1 -- 172.21.15.43:0/2778387976 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f2958153be0 msgr2=0x7f2958154060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T14:51:29.521 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:29.517+0000 7f2957fff640 1 --2- 172.21.15.43:0/2778387976 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f2958153be0 0x7f2958154060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T14:51:29.521 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:29.517+0000 7f2957fff640 1 -- 172.21.15.43:0/2778387976 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f29580ca1f0 msgr2=0x7f29580ca670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T14:51:29.521 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:29.517+0000 7f2957fff640 1 --2- 172.21.15.43:0/2778387976 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f29580ca1f0 0x7f29580ca670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T14:51:29.521 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:29.517+0000 7f2957fff640 1 -- 172.21.15.43:0/2778387976 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2958158270 con 0x7f29440047c0 2023-08-09T14:51:29.522 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:29.517+0000 7f2957fff640 1 --2- 172.21.15.43:0/2778387976 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f29440047c0 0x7f2958151c10 secure :-1 s=READY pgs=1779 cs=0 l=1 rev1=1 crypto rx=0x7f29480108e0 tx=0x7f2948010da0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-09T14:51:29.522 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:29.517+0000 7f29557fa640 1 -- 172.21.15.43:0/2778387976 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f294800eb00 con 0x7f29440047c0 2023-08-09T14:51:29.522 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:29.517+0000 7f29557fa640 1 -- 172.21.15.43:0/2778387976 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f294800ec60 con 0x7f29440047c0 2023-08-09T14:51:29.522 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:29.517+0000 7f295e81f640 1 -- 172.21.15.43:0/2778387976 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2958158550 con 0x7f29440047c0 2023-08-09T14:51:29.522 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:29.517+0000 7f29557fa640 1 -- 172.21.15.43:0/2778387976 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f294801c400 con 0x7f29440047c0 2023-08-09T14:51:29.522 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:29.517+0000 7f295e81f640 1 -- 172.21.15.43:0/2778387976 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2958158b10 con 0x7f29440047c0 2023-08-09T14:51:29.523 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:29.518+0000 7f295e81f640 1 -- 172.21.15.43:0/2778387976 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f29580739f0 con 0x7f29440047c0 2023-08-09T14:51:29.523 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:29.520+0000 7f29557fa640 1 -- 172.21.15.43:0/2778387976 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 77) v1 ==== 88179+0+0 (secure 0 0 0) 0x7f294801c560 con 0x7f29440047c0 2023-08-09T14:51:29.523 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:29.520+0000 7f29557fa640 1 --2- 172.21.15.43:0/2778387976 >> [v2:172.21.15.203:6832/2566816332,v1:172.21.15.203:6833/2566816332] conn(0x7f2938069ea0 0x7f293806c360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T14:51:29.523 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:29.521+0000 7f29557fa640 1 -- 172.21.15.43:0/2778387976 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(160..160 src has 1..160) v4 ==== 7216+0+0 (secure 0 0 0) 0x7f2948088df0 con 0x7f29440047c0 2023-08-09T14:51:29.524 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:29.521+0000 7f294ffff640 1 --2- 172.21.15.43:0/2778387976 >> [v2:172.21.15.203:6832/2566816332,v1:172.21.15.203:6833/2566816332] conn(0x7f2938069ea0 0x7f293806c360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T14:51:29.524 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:29.523+0000 7f294ffff640 1 --2- 172.21.15.43:0/2778387976 >> [v2:172.21.15.203:6832/2566816332,v1:172.21.15.203:6833/2566816332] conn(0x7f2938069ea0 0x7f293806c360 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7f2940033000 tx=0x7f294002f000 comp rx=0 tx=0).ready entity=mgr.7765 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-09T14:51:29.526 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:29.525+0000 7f29557fa640 1 -- 172.21.15.43:0/2778387976 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178066 (secure 0 0 0) 0x7f2948057e30 con 0x7f29440047c0 2023-08-09T14:51:29.608 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:29.606+0000 7f29557fa640 1 -- 172.21.15.43:0/2778387976 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mgrmap(e 78) v1 ==== 88387+0+0 (secure 0 0 0) 0x7f2948054770 con 0x7f29440047c0 2023-08-09T14:51:29.729 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:29.727+0000 7f295e81f640 1 -- 172.21.15.43:0/2778387976 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f29580cf1c0 con 0x7f29440047c0 2023-08-09T14:51:29.730 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:29.728+0000 7f29557fa640 1 -- 172.21.15.43:0/2778387976 <== mon.0 v2:172.21.15.43:3300/0 8 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v160) v1 ==== 100+0+11 (secure 0 0 0) 0x7f2948057ba0 con 0x7f29440047c0 2023-08-09T14:51:29.730 INFO:teuthology.orchestra.run.smithi043.stdout:pg_num: 64 2023-08-09T14:51:29.732 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:29.731+0000 7f295e81f640 1 -- 172.21.15.43:0/2778387976 >> [v2:172.21.15.203:6832/2566816332,v1:172.21.15.203:6833/2566816332] conn(0x7f2938069ea0 msgr2=0x7f293806c360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T14:51:29.733 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:29.731+0000 7f295e81f640 1 --2- 172.21.15.43:0/2778387976 >> [v2:172.21.15.203:6832/2566816332,v1:172.21.15.203:6833/2566816332] conn(0x7f2938069ea0 0x7f293806c360 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7f2940033000 tx=0x7f294002f000 comp rx=0 tx=0).stop 2023-08-09T14:51:29.733 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:29.731+0000 7f295e81f640 1 -- 172.21.15.43:0/2778387976 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f29440047c0 msgr2=0x7f2958151c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T14:51:29.733 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:29.731+0000 7f295e81f640 1 --2- 172.21.15.43:0/2778387976 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f29440047c0 0x7f2958151c10 secure :-1 s=READY pgs=1779 cs=0 l=1 rev1=1 crypto rx=0x7f29480108e0 tx=0x7f2948010da0 comp rx=0 tx=0).stop 2023-08-09T14:51:29.733 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:29.731+0000 7f295e81f640 1 -- 172.21.15.43:0/2778387976 shutdown_connections 2023-08-09T14:51:29.733 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:29.731+0000 7f295e81f640 1 --2- 172.21.15.43:0/2778387976 >> [v2:172.21.15.203:6832/2566816332,v1:172.21.15.203:6833/2566816332] conn(0x7f2938069ea0 0x7f293806c360 unknown :-1 s=CLOSED pgs=25 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T14:51:29.734 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:29.731+0000 7f295e81f640 1 --2- 172.21.15.43:0/2778387976 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f2958153be0 0x7f2958154060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T14:51:29.734 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:29.731+0000 7f295e81f640 1 --2- 172.21.15.43:0/2778387976 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f29580ca1f0 0x7f29580ca670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T14:51:29.734 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:29.731+0000 7f295e81f640 1 --2- 172.21.15.43:0/2778387976 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f29440047c0 0x7f2958151c10 unknown :-1 s=CLOSED pgs=1779 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T14:51:29.734 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:29.731+0000 7f295e81f640 1 -- 172.21.15.43:0/2778387976 >> 172.21.15.43:0/2778387976 conn(0x7f2958071520 msgr2=0x7f295815cad0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-09T14:51:29.734 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:29.731+0000 7f295e81f640 1 -- 172.21.15.43:0/2778387976 shutdown_connections 2023-08-09T14:51:29.734 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:29.731+0000 7f295e81f640 1 -- 172.21.15.43:0/2778387976 wait complete. 2023-08-09T14:51:29.753 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-08-09T14:51:29.971 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:29.969+0000 7f45281d6640 1 Processor -- start 2023-08-09T14:51:29.971 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:29.969+0000 7f45281d6640 1 -- start start 2023-08-09T14:51:29.973 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:29.971+0000 7f45281d6640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f45200cdc60 0x7f4520167500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T14:51:29.973 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:29.971+0000 7f45281d6640 1 -- --> v1:172.21.15.43:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f45200cca90 con 0x7f452015c630 2023-08-09T14:51:29.973 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:29.971+0000 7f45281d6640 1 -- --> v1:172.21.15.203:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f452015da20 con 0x7f45200ccf20 2023-08-09T14:51:29.974 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:29.971+0000 7f45281d6640 1 -- --> v2:172.21.15.43:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f45200ac600 con 0x7f45200cdc60 2023-08-09T14:51:29.974 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:29.971+0000 7f4526773640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f45200cdc60 0x7f4520167500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T14:51:29.974 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:29.971+0000 7f4525f72640 1 --1- >> v1:172.21.15.43:6789/0 conn(0x7f452015c630 0x7f45200abef0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.43:6789/0 says I am v1:172.21.15.43:54916/0 (socket says 172.21.15.43:54916) 2023-08-09T14:51:29.974 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:29.971+0000 7f4525f72640 1 -- 172.21.15.43:0/605849989 learned_addr learned my addr 172.21.15.43:0/605849989 (peer_addr_for_me v1:172.21.15.43:0/0) 2023-08-09T14:51:29.976 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:29.972+0000 7f4526773640 1 -- 172.21.15.43:0/605849989 >> v1:172.21.15.203:6789/0 conn(0x7f45200ccf20 legacy=0x7f45200cd550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T14:51:29.976 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:29.972+0000 7f4526773640 1 -- 172.21.15.43:0/605849989 >> v1:172.21.15.43:6789/0 conn(0x7f452015c630 legacy=0x7f45200abef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T14:51:29.976 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:29.972+0000 7f4526773640 1 -- 172.21.15.43:0/605849989 --> v2:172.21.15.43:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f45200cbe80 con 0x7f45200cdc60 2023-08-09T14:51:29.977 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:29.972+0000 7f4524f70640 1 -- 172.21.15.43:0/605849989 <== mon.0 v1:172.21.15.43:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4287387618 0 0) 0x7f45200cca90 con 0x7f452015c630 2023-08-09T14:51:29.977 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:29.973+0000 7f4526773640 1 --2- 172.21.15.43:0/605849989 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f45200cdc60 0x7f4520167500 secure :-1 s=READY pgs=1781 cs=0 l=1 rev1=1 crypto rx=0x7f45180097a0 tx=0x7f451802b960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=351567474c0a2aeb server_cookie=0 in_seq=0 out_seq=0 2023-08-09T14:51:29.977 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:29.974+0000 7f4524f70640 1 -- 172.21.15.43:0/605849989 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4518030610 con 0x7f45200cdc60 2023-08-09T14:51:29.977 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:29.974+0000 7f4524f70640 1 -- 172.21.15.43:0/605849989 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4518030770 con 0x7f45200cdc60 2023-08-09T14:51:29.978 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:29.974+0000 7f4524f70640 1 -- 172.21.15.43:0/605849989 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4518030b00 con 0x7f45200cdc60 2023-08-09T14:51:29.978 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:29.974+0000 7f45281d6640 1 -- 172.21.15.43:0/605849989 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f45200cdc60 msgr2=0x7f4520167500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T14:51:29.978 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:29.974+0000 7f45281d6640 1 --2- 172.21.15.43:0/605849989 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f45200cdc60 0x7f4520167500 secure :-1 s=READY pgs=1781 cs=0 l=1 rev1=1 crypto rx=0x7f45180097a0 tx=0x7f451802b960 comp rx=0 tx=0).stop 2023-08-09T14:51:29.978 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:29.974+0000 7f45281d6640 1 -- 172.21.15.43:0/605849989 shutdown_connections 2023-08-09T14:51:29.978 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:29.974+0000 7f45281d6640 1 --2- 172.21.15.43:0/605849989 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f45200cdc60 0x7f4520167500 unknown :-1 s=CLOSED pgs=1781 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T14:51:29.979 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:29.974+0000 7f45281d6640 1 -- 172.21.15.43:0/605849989 >> 172.21.15.43:0/605849989 conn(0x7f45200ce0a0 msgr2=0x7f45200ce480 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-09T14:51:29.979 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:29.975+0000 7f45281d6640 1 -- 172.21.15.43:0/605849989 shutdown_connections 2023-08-09T14:51:29.979 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:29.975+0000 7f45281d6640 1 -- 172.21.15.43:0/605849989 wait complete. 2023-08-09T14:51:29.982 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:29.977+0000 7f45281d6640 1 Processor -- start 2023-08-09T14:51:29.982 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:29.977+0000 7f45281d6640 1 -- start start 2023-08-09T14:51:29.982 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:29.978+0000 7f45281d6640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f45200ccf20 0x7f4520070b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T14:51:29.982 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:29.978+0000 7f45281d6640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f45200cdc60 0x7f452015dd20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T14:51:29.983 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:29.978+0000 7f45281d6640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f452015c630 0x7f452015e260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T14:51:29.983 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:29.978+0000 7f45281d6640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4520158400 con 0x7f45200ccf20 2023-08-09T14:51:29.983 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:29.978+0000 7f45281d6640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f452015e890 con 0x7f45200cdc60 2023-08-09T14:51:29.983 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:29.978+0000 7f45281d6640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f452015eb70 con 0x7f452015c630 2023-08-09T14:51:29.984 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:29.979+0000 7f4525f72640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f45200ccf20 0x7f4520070b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T14:51:29.984 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:29.979+0000 7f4525f72640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f45200ccf20 0x7f4520070b60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:39890/0 (socket says 172.21.15.43:39890) 2023-08-09T14:51:29.984 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:29.979+0000 7f4525f72640 1 -- 172.21.15.43:0/1206301334 learned_addr learned my addr 172.21.15.43:0/1206301334 (peer_addr_for_me v2:172.21.15.43:0/0) 2023-08-09T14:51:29.985 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:29.979+0000 7f4525771640 1 --2- 172.21.15.43:0/1206301334 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f45200cdc60 0x7f452015dd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T14:51:29.985 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:29.979+0000 7f4525f72640 1 -- 172.21.15.43:0/1206301334 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f452015c630 msgr2=0x7f452015e260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T14:51:29.985 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:29.979+0000 7f4525f72640 1 --2- 172.21.15.43:0/1206301334 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f452015c630 0x7f452015e260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T14:51:29.985 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:29.979+0000 7f4525f72640 1 -- 172.21.15.43:0/1206301334 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f45200cdc60 msgr2=0x7f452015dd20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T14:51:29.986 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:29.979+0000 7f4525f72640 1 --2- 172.21.15.43:0/1206301334 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f45200cdc60 0x7f452015dd20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T14:51:29.986 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:29.979+0000 7f4525f72640 1 -- 172.21.15.43:0/1206301334 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4518009400 con 0x7f45200ccf20 2023-08-09T14:51:29.986 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:29.980+0000 7f4525f72640 1 --2- 172.21.15.43:0/1206301334 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f45200ccf20 0x7f4520070b60 secure :-1 s=READY pgs=1782 cs=0 l=1 rev1=1 crypto rx=0x7f451c00c8c0 tx=0x7f451c00cd80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-09T14:51:29.986 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:29.980+0000 7f45167fc640 1 -- 172.21.15.43:0/1206301334 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f451c00a640 con 0x7f45200ccf20 2023-08-09T14:51:29.987 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:29.980+0000 7f45167fc640 1 -- 172.21.15.43:0/1206301334 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f451c00a7a0 con 0x7f45200ccf20 2023-08-09T14:51:29.987 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:29.980+0000 7f45281d6640 1 -- 172.21.15.43:0/1206301334 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f452015ee50 con 0x7f45200ccf20 2023-08-09T14:51:29.987 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:29.980+0000 7f45167fc640 1 -- 172.21.15.43:0/1206301334 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f451c00ab90 con 0x7f45200ccf20 2023-08-09T14:51:29.988 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:29.980+0000 7f45281d6640 1 -- 172.21.15.43:0/1206301334 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f452015f410 con 0x7f45200ccf20 2023-08-09T14:51:29.989 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:29.981+0000 7f45281d6640 1 -- 172.21.15.43:0/1206301334 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f452015f630 con 0x7f45200ccf20 2023-08-09T14:51:29.990 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:29.982+0000 7f45167fc640 1 -- 172.21.15.43:0/1206301334 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 78) v1 ==== 88387+0+0 (secure 0 0 0) 0x7f451c00acf0 con 0x7f45200ccf20 2023-08-09T14:51:29.990 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:29.983+0000 7f45167fc640 1 --2- 172.21.15.43:0/1206301334 >> [v2:172.21.15.203:6832/2566816332,v1:172.21.15.203:6833/2566816332] conn(0x7f450406a0e0 0x7f450406c5a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T14:51:29.990 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:29.983+0000 7f45167fc640 1 -- 172.21.15.43:0/1206301334 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(160..160 src has 1..160) v4 ==== 7216+0+0 (secure 0 0 0) 0x7f451c088170 con 0x7f45200ccf20 2023-08-09T14:51:29.991 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:29.984+0000 7f4525771640 1 --2- 172.21.15.43:0/1206301334 >> [v2:172.21.15.203:6832/2566816332,v1:172.21.15.203:6833/2566816332] conn(0x7f450406a0e0 0x7f450406c5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T14:51:29.991 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:29.985+0000 7f4525771640 1 --2- 172.21.15.43:0/1206301334 >> [v2:172.21.15.203:6832/2566816332,v1:172.21.15.203:6833/2566816332] conn(0x7f450406a0e0 0x7f450406c5a0 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7f45200c4990 tx=0x7f4510009040 comp rx=0 tx=0).ready entity=mgr.7765 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-09T14:51:29.991 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:29.987+0000 7f45167fc640 1 -- 172.21.15.43:0/1206301334 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178066 (secure 0 0 0) 0x7f451c053310 con 0x7f45200ccf20 2023-08-09T14:51:30.213 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:30.212+0000 7f45281d6640 1 -- 172.21.15.43:0/1206301334 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f45200739f0 con 0x7f45200ccf20 2023-08-09T14:51:30.214 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:30.212+0000 7f45167fc640 1 -- 172.21.15.43:0/1206301334 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 70 v70) v1 ==== 94+0+4883 (secure 0 0 0) 0x7f451c018070 con 0x7f45200ccf20 2023-08-09T14:51:30.214 INFO:teuthology.orchestra.run.smithi043.stdout: 2023-08-09T14:51:30.215 INFO:teuthology.orchestra.run.smithi043.stdout:{"epoch":70,"default_fscid":8,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6812,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.203:6835/2100579784","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6834","nonce":2100579784},{"type":"v1","addr":"172.21.15.203:6835","nonce":2100579784}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":65},{"gid":6815,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.203:6837/1689128061","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6836","nonce":1689128061},{"type":"v1","addr":"172.21.15.203:6837","nonce":1689128061}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":65},{"gid":7543,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.43:6835/1126540259","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6833","nonce":1126540259},{"type":"v1","addr":"172.21.15.43:6835","nonce":1126540259}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":65}],"filesystems":[{"mdsmap":{"epoch":70,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-08-09T13:55:28.794042+0000","modified":"2023-08-09T13:55:32.388404+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":9126},"failed":[],"damaged":[],"stopped":[],"info":{"gid_9126":{"gid":9126,"name":"a","rank":0,"incarnation":66,"state":"up:active","state_seq":3,"addr":"172.21.15.43:6834/2023142010","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6832","nonce":2023142010},{"type":"v1","addr":"172.21.15.43:6834","nonce":2023142010}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[19],"metadata_pool":18,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":8}]} 2023-08-09T14:51:30.215 INFO:teuthology.orchestra.run.smithi043.stderr:dumped fsmap epoch 70 2023-08-09T14:51:30.217 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:30.215+0000 7f45281d6640 1 -- 172.21.15.43:0/1206301334 >> [v2:172.21.15.203:6832/2566816332,v1:172.21.15.203:6833/2566816332] conn(0x7f450406a0e0 msgr2=0x7f450406c5a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T14:51:30.217 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:30.215+0000 7f45281d6640 1 --2- 172.21.15.43:0/1206301334 >> [v2:172.21.15.203:6832/2566816332,v1:172.21.15.203:6833/2566816332] conn(0x7f450406a0e0 0x7f450406c5a0 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7f45200c4990 tx=0x7f4510009040 comp rx=0 tx=0).stop 2023-08-09T14:51:30.217 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:30.215+0000 7f45281d6640 1 -- 172.21.15.43:0/1206301334 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f45200ccf20 msgr2=0x7f4520070b60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T14:51:30.217 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:30.215+0000 7f45281d6640 1 --2- 172.21.15.43:0/1206301334 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f45200ccf20 0x7f4520070b60 secure :-1 s=READY pgs=1782 cs=0 l=1 rev1=1 crypto rx=0x7f451c00c8c0 tx=0x7f451c00cd80 comp rx=0 tx=0).stop 2023-08-09T14:51:30.217 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:30.215+0000 7f45281d6640 1 -- 172.21.15.43:0/1206301334 shutdown_connections 2023-08-09T14:51:30.218 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:30.215+0000 7f45281d6640 1 --2- 172.21.15.43:0/1206301334 >> [v2:172.21.15.203:6832/2566816332,v1:172.21.15.203:6833/2566816332] conn(0x7f450406a0e0 0x7f450406c5a0 unknown :-1 s=CLOSED pgs=26 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T14:51:30.218 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:30.215+0000 7f45281d6640 1 --2- 172.21.15.43:0/1206301334 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f452015c630 0x7f452015e260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T14:51:30.218 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:30.215+0000 7f45281d6640 1 --2- 172.21.15.43:0/1206301334 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f45200cdc60 0x7f452015dd20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T14:51:30.218 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:30.215+0000 7f45281d6640 1 --2- 172.21.15.43:0/1206301334 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f45200ccf20 0x7f4520070b60 unknown :-1 s=CLOSED pgs=1782 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T14:51:30.218 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:30.215+0000 7f45281d6640 1 -- 172.21.15.43:0/1206301334 >> 172.21.15.43:0/1206301334 conn(0x7f45200ce0a0 msgr2=0x7f452006b790 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-09T14:51:30.218 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:30.215+0000 7f45281d6640 1 -- 172.21.15.43:0/1206301334 shutdown_connections 2023-08-09T14:51:30.218 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:30.215+0000 7f45281d6640 1 -- 172.21.15.43:0/1206301334 wait complete. 2023-08-09T14:51:30.234 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-08-09T14:51:30.445 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:30.444+0000 7fa0c9147640 1 Processor -- start 2023-08-09T14:51:30.446 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:30.444+0000 7fa0c9147640 1 -- start start 2023-08-09T14:51:30.447 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:30.446+0000 7fa0c9147640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fa0c40b2790 0x7fa0c40d2ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T14:51:30.447 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:30.446+0000 7fa0c9147640 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7fa0c40b2b70 0x7fa0c40b5330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T14:51:30.448 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:30.446+0000 7fa0c9147640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7fa0c40b3050 0x7fa0c40b3f60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T14:51:30.448 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:30.446+0000 7fa0c9147640 1 -- --> v2:172.21.15.43:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa0c40b3950 con 0x7fa0c40b3050 2023-08-09T14:51:30.448 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:30.446+0000 7fa0c9147640 1 -- --> v2:172.21.15.203:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa0c40ce090 con 0x7fa0c40b2b70 2023-08-09T14:51:30.448 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:30.446+0000 7fa0c9147640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa0c40cd7f0 con 0x7fa0c40b2790 2023-08-09T14:51:30.449 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:30.446+0000 7fa0c359e640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7fa0c40b3050 0x7fa0c40b3f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T14:51:30.449 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:30.446+0000 7fa0c359e640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7fa0c40b3050 0x7fa0c40b3f60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:39892/0 (socket says 172.21.15.43:39892) 2023-08-09T14:51:30.449 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:30.446+0000 7fa0c359e640 1 -- 172.21.15.43:0/808113167 learned_addr learned my addr 172.21.15.43:0/808113167 (peer_addr_for_me v2:172.21.15.43:0/0) 2023-08-09T14:51:30.449 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:30.447+0000 7fa0c259c640 1 --2- 172.21.15.43:0/808113167 >> v2:172.21.15.203:3300/0 conn(0x7fa0c40b2b70 0x7fa0c40b5330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T14:51:30.449 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:30.447+0000 7fa0c2d9d640 1 --2- 172.21.15.43:0/808113167 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fa0c40b2790 0x7fa0c40d2ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T14:51:30.450 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:30.447+0000 7fa0c359e640 1 -- 172.21.15.43:0/808113167 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fa0c40b2790 msgr2=0x7fa0c40d2ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-09T14:51:30.450 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:30.447+0000 7fa0c359e640 1 --2- 172.21.15.43:0/808113167 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fa0c40b2790 0x7fa0c40d2ed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T14:51:30.450 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:30.447+0000 7fa0c359e640 1 -- 172.21.15.43:0/808113167 >> v2:172.21.15.203:3300/0 conn(0x7fa0c40b2b70 msgr2=0x7fa0c40b5330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-09T14:51:30.450 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:30.447+0000 7fa0c359e640 1 --2- 172.21.15.43:0/808113167 >> v2:172.21.15.203:3300/0 conn(0x7fa0c40b2b70 0x7fa0c40b5330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T14:51:30.450 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:30.447+0000 7fa0c359e640 1 -- 172.21.15.43:0/808113167 --> v2:172.21.15.43:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa0c40cdad0 con 0x7fa0c40b3050 2023-08-09T14:51:30.450 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:30.447+0000 7fa0c259c640 1 --2- 172.21.15.43:0/808113167 >> v2:172.21.15.203:3300/0 conn(0x7fa0c40b2b70 0x7fa0c40b5330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-08-09T14:51:30.451 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:30.449+0000 7fa0c359e640 1 --2- 172.21.15.43:0/808113167 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa0c40b3050 0x7fa0c40b3f60 secure :-1 s=READY pgs=1783 cs=0 l=1 rev1=1 crypto rx=0x7fa0c409bee0 tx=0x7fa0bc02bed0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2a5ec281bced2cb1 server_cookie=0 in_seq=0 out_seq=0 2023-08-09T14:51:30.451 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:30.449+0000 7fa0c1d9b640 1 -- 172.21.15.43:0/808113167 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa0bc02ca70 con 0x7fa0c40b3050 2023-08-09T14:51:30.451 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:30.449+0000 7fa0c1d9b640 1 -- 172.21.15.43:0/808113167 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa0bc02cbd0 con 0x7fa0c40b3050 2023-08-09T14:51:30.451 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:30.449+0000 7fa0c1d9b640 1 -- 172.21.15.43:0/808113167 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa0bc0343f0 con 0x7fa0c40b3050 2023-08-09T14:51:30.451 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:30.450+0000 7fa0c9147640 1 -- 172.21.15.43:0/808113167 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa0c40b3050 msgr2=0x7fa0c40b3f60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T14:51:30.452 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:30.450+0000 7fa0c9147640 1 --2- 172.21.15.43:0/808113167 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa0c40b3050 0x7fa0c40b3f60 secure :-1 s=READY pgs=1783 cs=0 l=1 rev1=1 crypto rx=0x7fa0c409bee0 tx=0x7fa0bc02bed0 comp rx=0 tx=0).stop 2023-08-09T14:51:30.452 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:30.450+0000 7fa0c9147640 1 -- 172.21.15.43:0/808113167 shutdown_connections 2023-08-09T14:51:30.452 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:30.450+0000 7fa0c9147640 1 --2- 172.21.15.43:0/808113167 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa0c40b3050 0x7fa0c40b3f60 unknown :-1 s=CLOSED pgs=1783 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T14:51:30.453 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:30.450+0000 7fa0c9147640 1 --2- 172.21.15.43:0/808113167 >> v2:172.21.15.203:3300/0 conn(0x7fa0c40b2b70 0x7fa0c40b5330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T14:51:30.453 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:30.450+0000 7fa0c9147640 1 --2- 172.21.15.43:0/808113167 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fa0c40b2790 0x7fa0c40d2ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T14:51:30.453 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:30.450+0000 7fa0c9147640 1 -- 172.21.15.43:0/808113167 >> 172.21.15.43:0/808113167 conn(0x7fa0c409b7b0 msgr2=0x7fa0c40d24e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-09T14:51:30.453 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:30.450+0000 7fa0c9147640 1 -- 172.21.15.43:0/808113167 shutdown_connections 2023-08-09T14:51:30.453 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:30.450+0000 7fa0c9147640 1 -- 172.21.15.43:0/808113167 wait complete. 2023-08-09T14:51:30.454 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:30.452+0000 7fa0c9147640 1 Processor -- start 2023-08-09T14:51:30.454 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:30.452+0000 7fa0c9147640 1 -- start start 2023-08-09T14:51:30.454 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:30.453+0000 7fa0c9147640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fa0c40b2790 0x7fa0c40c6010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T14:51:30.455 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:30.453+0000 7fa0c9147640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fa0c40b2b70 0x7fa0c414f9e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T14:51:30.455 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:30.453+0000 7fa0c9147640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa0c40b3050 0x7fa0c406a5f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T14:51:30.455 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:30.453+0000 7fa0c9147640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa0c414ff20 con 0x7fa0c40b3050 2023-08-09T14:51:30.455 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:30.453+0000 7fa0c9147640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa0c40c66a0 con 0x7fa0c40b2b70 2023-08-09T14:51:30.455 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:30.453+0000 7fa0c9147640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa0c40c6980 con 0x7fa0c40b2790 2023-08-09T14:51:30.455 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:30.453+0000 7fa0c359e640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa0c40b3050 0x7fa0c406a5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T14:51:30.455 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:30.453+0000 7fa0c359e640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa0c40b3050 0x7fa0c406a5f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:39908/0 (socket says 172.21.15.43:39908) 2023-08-09T14:51:30.456 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:30.453+0000 7fa0c359e640 1 -- 172.21.15.43:0/1405029249 learned_addr learned my addr 172.21.15.43:0/1405029249 (peer_addr_for_me v2:172.21.15.43:0/0) 2023-08-09T14:51:30.456 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:30.453+0000 7fa0c359e640 1 -- 172.21.15.43:0/1405029249 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fa0c40b2790 msgr2=0x7fa0c40c6010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T14:51:30.457 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:30.453+0000 7fa0c359e640 1 --2- 172.21.15.43:0/1405029249 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fa0c40b2790 0x7fa0c40c6010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T14:51:30.457 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:30.453+0000 7fa0c359e640 1 -- 172.21.15.43:0/1405029249 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fa0c40b2b70 msgr2=0x7fa0c414f9e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T14:51:30.457 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:30.453+0000 7fa0c359e640 1 --2- 172.21.15.43:0/1405029249 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fa0c40b2b70 0x7fa0c414f9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T14:51:30.457 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:30.453+0000 7fa0c359e640 1 -- 172.21.15.43:0/1405029249 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa0bc009b30 con 0x7fa0c40b3050 2023-08-09T14:51:30.457 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:30.454+0000 7fa0c359e640 1 --2- 172.21.15.43:0/1405029249 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa0c40b3050 0x7fa0c406a5f0 secure :-1 s=READY pgs=1784 cs=0 l=1 rev1=1 crypto rx=0x7fa0bc009e20 tx=0x7fa0bc004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-09T14:51:30.457 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:30.454+0000 7fa0ab7fe640 1 -- 172.21.15.43:0/1405029249 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa0bc040670 con 0x7fa0c40b3050 2023-08-09T14:51:30.457 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:30.454+0000 7fa0ab7fe640 1 -- 172.21.15.43:0/1405029249 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa0bc0407f0 con 0x7fa0c40b3050 2023-08-09T14:51:30.457 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:30.454+0000 7fa0ab7fe640 1 -- 172.21.15.43:0/1405029249 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa0bc040ba0 con 0x7fa0c40b3050 2023-08-09T14:51:30.458 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:30.454+0000 7fa0c9147640 1 -- 172.21.15.43:0/1405029249 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa0c40c6c60 con 0x7fa0c40b3050 2023-08-09T14:51:30.458 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:30.454+0000 7fa0c9147640 1 -- 172.21.15.43:0/1405029249 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa0c40c7970 con 0x7fa0c40b3050 2023-08-09T14:51:30.458 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:30.455+0000 7fa0c9147640 1 -- 172.21.15.43:0/1405029249 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa0c40739f0 con 0x7fa0c40b3050 2023-08-09T14:51:30.458 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:30.456+0000 7fa0ab7fe640 1 -- 172.21.15.43:0/1405029249 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 78) v1 ==== 88387+0+0 (secure 0 0 0) 0x7fa0bc040d00 con 0x7fa0c40b3050 2023-08-09T14:51:30.459 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:30.457+0000 7fa0ab7fe640 1 --2- 172.21.15.43:0/1405029249 >> [v2:172.21.15.203:6832/2566816332,v1:172.21.15.203:6833/2566816332] conn(0x7fa0b006a0e0 0x7fa0b006c5a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T14:51:30.459 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:30.457+0000 7fa0ab7fe640 1 -- 172.21.15.43:0/1405029249 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(160..160 src has 1..160) v4 ==== 7216+0+0 (secure 0 0 0) 0x7fa0bc0abca0 con 0x7fa0c40b3050 2023-08-09T14:51:30.459 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:30.458+0000 7fa0c2d9d640 1 --2- 172.21.15.43:0/1405029249 >> [v2:172.21.15.203:6832/2566816332,v1:172.21.15.203:6833/2566816332] conn(0x7fa0b006a0e0 0x7fa0b006c5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T14:51:30.460 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:30.459+0000 7fa0c2d9d640 1 --2- 172.21.15.43:0/1405029249 >> [v2:172.21.15.203:6832/2566816332,v1:172.21.15.203:6833/2566816332] conn(0x7fa0b006a0e0 0x7fa0b006c5a0 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7fa0c40b77b0 tx=0x7fa0b4008040 comp rx=0 tx=0).ready entity=mgr.7765 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-09T14:51:30.463 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:30.462+0000 7fa0ab7fe640 1 -- 172.21.15.43:0/1405029249 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178066 (secure 0 0 0) 0x7fa0bc076f00 con 0x7fa0c40b3050 2023-08-09T14:51:30.655 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:30.653+0000 7fa0c9147640 1 -- 172.21.15.43:0/1405029249 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fa0c40b4b00 con 0x7fa0c40b3050 2023-08-09T14:51:30.658 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:30.657+0000 7fa0ab7fe640 1 -- 172.21.15.43:0/1405029249 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v160) v1 ==== 74+0+19721 (secure 0 0 0) 0x7fa0bc009510 con 0x7fa0c40b3050 2023-08-09T14:51:30.659 INFO:teuthology.orchestra.run.smithi043.stdout: 2023-08-09T14:51:30.659 INFO:teuthology.orchestra.run.smithi043.stdout:{"epoch":160,"fsid":"daac87ee-1337-4250-a955-aea4e408a39b","created":"2023-08-09T13:33:41.701198+0000","modified":"2023-08-09T14:51:27.179740+0000","last_up_change":"2023-08-09T13:33:53.783453+0000","last_in_change":"2023-08-09T13:33:45.148904+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":19,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":"rbd","create_time":"2023-08-09T13:33:54.365137+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":".mgr","create_time":"2023-08-09T13:33:54.831803+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":18,"pool_name":"cephfs_metadata","create_time":"2023-08-09T13:55:27.207507+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"146","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":19,"pool_name":"cephfs_data","create_time":"2023-08-09T13:55:27.825578+0000","flags":8193,"flags_names":"hashpspool,selfmanaged_snaps","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"159","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":113,"snap_epoch":159,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.25,"score_stable":1.25,"optimal_score":1,"raw_score_acting":1.25,"raw_score_stable":1.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"cbd49ecd-e896-4fbd-864e-5c37a829b165","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":149,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6800","nonce":502667335},{"type":"v1","addr":"172.21.15.43:6801","nonce":502667335}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6802","nonce":502667335},{"type":"v1","addr":"172.21.15.43:6803","nonce":502667335}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6806","nonce":502667335},{"type":"v1","addr":"172.21.15.43:6807","nonce":502667335}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6804","nonce":502667335},{"type":"v1","addr":"172.21.15.43:6805","nonce":502667335}]},"public_addr":"172.21.15.43:6801/502667335","cluster_addr":"172.21.15.43:6803/502667335","heartbeat_back_addr":"172.21.15.43:6807/502667335","heartbeat_front_addr":"172.21.15.43:6805/502667335","state":["exists","up"]},{"osd":1,"uuid":"75c7e0fe-1ec7-4fe8-a2f2-3661a65b8500","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":145,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6808","nonce":3289134610},{"type":"v1","addr":"172.21.15.43:6809","nonce":3289134610}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6810","nonce":3289134610},{"type":"v1","addr":"172.21.15.43:6812","nonce":3289134610}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6821","nonce":3289134610},{"type":"v1","addr":"172.21.15.43:6824","nonce":3289134610}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6815","nonce":3289134610},{"type":"v1","addr":"172.21.15.43:6818","nonce":3289134610}]},"public_addr":"172.21.15.43:6809/3289134610","cluster_addr":"172.21.15.43:6812/3289134610","heartbeat_back_addr":"172.21.15.43:6824/3289134610","heartbeat_front_addr":"172.21.15.43:6818/3289134610","state":["exists","up"]},{"osd":2,"uuid":"bb5c41e9-7f5f-4125-9b01-3bb8bfc2ea83","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":145,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6811","nonce":4079709736},{"type":"v1","addr":"172.21.15.43:6814","nonce":4079709736}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6817","nonce":4079709736},{"type":"v1","addr":"172.21.15.43:6820","nonce":4079709736}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6829","nonce":4079709736},{"type":"v1","addr":"172.21.15.43:6831","nonce":4079709736}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6823","nonce":4079709736},{"type":"v1","addr":"172.21.15.43:6826","nonce":4079709736}]},"public_addr":"172.21.15.43:6814/4079709736","cluster_addr":"172.21.15.43:6820/4079709736","heartbeat_back_addr":"172.21.15.43:6831/4079709736","heartbeat_front_addr":"172.21.15.43:6826/4079709736","state":["exists","up"]},{"osd":3,"uuid":"bfe4c8b4-694a-465c-87e6-2b2aab852613","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":149,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6813","nonce":1709538759},{"type":"v1","addr":"172.21.15.43:6816","nonce":1709538759}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6819","nonce":1709538759},{"type":"v1","addr":"172.21.15.43:6822","nonce":1709538759}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6828","nonce":1709538759},{"type":"v1","addr":"172.21.15.43:6830","nonce":1709538759}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6825","nonce":1709538759},{"type":"v1","addr":"172.21.15.43:6827","nonce":1709538759}]},"public_addr":"172.21.15.43:6816/1709538759","cluster_addr":"172.21.15.43:6822/1709538759","heartbeat_back_addr":"172.21.15.43:6830/1709538759","heartbeat_front_addr":"172.21.15.43:6827/1709538759","state":["exists","up"]},{"osd":4,"uuid":"fa2e8a77-aba1-4756-853e-0d174a5bdeba","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":145,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6800","nonce":4196866481},{"type":"v1","addr":"172.21.15.203:6801","nonce":4196866481}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6802","nonce":4196866481},{"type":"v1","addr":"172.21.15.203:6803","nonce":4196866481}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6806","nonce":4196866481},{"type":"v1","addr":"172.21.15.203:6807","nonce":4196866481}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6804","nonce":4196866481},{"type":"v1","addr":"172.21.15.203:6805","nonce":4196866481}]},"public_addr":"172.21.15.203:6801/4196866481","cluster_addr":"172.21.15.203:6803/4196866481","heartbeat_back_addr":"172.21.15.203:6807/4196866481","heartbeat_front_addr":"172.21.15.203:6805/4196866481","state":["exists","up"]},{"osd":5,"uuid":"81811e4b-4d24-4f51-a82c-f1d700a2a00d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":145,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6816","nonce":3059290081},{"type":"v1","addr":"172.21.15.203:6817","nonce":3059290081}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6818","nonce":3059290081},{"type":"v1","addr":"172.21.15.203:6819","nonce":3059290081}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6822","nonce":3059290081},{"type":"v1","addr":"172.21.15.203:6823","nonce":3059290081}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6820","nonce":3059290081},{"type":"v1","addr":"172.21.15.203:6821","nonce":3059290081}]},"public_addr":"172.21.15.203:6817/3059290081","cluster_addr":"172.21.15.203:6819/3059290081","heartbeat_back_addr":"172.21.15.203:6823/3059290081","heartbeat_front_addr":"172.21.15.203:6821/3059290081","state":["exists","up"]},{"osd":6,"uuid":"307bac4d-6a5b-443d-af75-56d695d7674f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":145,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6824","nonce":512155569},{"type":"v1","addr":"172.21.15.203:6825","nonce":512155569}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6826","nonce":512155569},{"type":"v1","addr":"172.21.15.203:6827","nonce":512155569}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6830","nonce":512155569},{"type":"v1","addr":"172.21.15.203:6831","nonce":512155569}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6828","nonce":512155569},{"type":"v1","addr":"172.21.15.203:6829","nonce":512155569}]},"public_addr":"172.21.15.203:6825/512155569","cluster_addr":"172.21.15.203:6827/512155569","heartbeat_back_addr":"172.21.15.203:6831/512155569","heartbeat_front_addr":"172.21.15.203:6829/512155569","state":["exists","up"]},{"osd":7,"uuid":"a95dc7ca-9f1b-4295-af70-68a22a2d7d5e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":145,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6808","nonce":3331450438},{"type":"v1","addr":"172.21.15.203:6809","nonce":3331450438}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6810","nonce":3331450438},{"type":"v1","addr":"172.21.15.203:6811","nonce":3331450438}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6814","nonce":3331450438},{"type":"v1","addr":"172.21.15.203:6815","nonce":3331450438}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6812","nonce":3331450438},{"type":"v1","addr":"172.21.15.203:6813","nonce":3331450438}]},"public_addr":"172.21.15.203:6809/3331450438","cluster_addr":"172.21.15.203:6811/3331450438","heartbeat_back_addr":"172.21.15.203:6815/3331450438","heartbeat_front_addr":"172.21.15.203:6813/3331450438","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-09T13:33:50.547652+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-09T13:33:50.589469+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-09T13:33:50.561724+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-09T13:33:50.599650+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-09T13:33:50.411855+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-09T13:33:50.219681+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-09T13:33:50.504664+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-09T13:33:50.441248+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"19.25","mappings":[{"from":2,"to":0}]},{"pgid":"19.26","mappings":[{"from":2,"to":0}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.203:0/1340171653":"2023-08-10T14:51:27.179649+0000","172.21.15.203:0/2575410559":"2023-08-10T14:51:27.179649+0000","172.21.15.203:0/1196185412":"2023-08-10T14:51:27.179649+0000","172.21.15.203:6833/372694067":"2023-08-10T14:51:27.179649+0000","172.21.15.203:6832/372694067":"2023-08-10T14:51:27.179649+0000","172.21.15.203:0/2726918685":"2023-08-10T14:51:27.179649+0000","172.21.15.203:0/3273594633":"2023-08-10T14:51:27.179649+0000","172.21.15.203:0/2477376154":"2023-08-10T14:51:27.179649+0000","172.21.15.203:0/2901879784":"2023-08-10T13:55:44.498027+0000","172.21.15.203:0/1501229236":"2023-08-10T13:55:44.498027+0000","172.21.15.203:0/3615051942":"2023-08-10T13:55:44.498027+0000","172.21.15.203:6833/3490266242":"2023-08-10T13:55:44.498027+0000","172.21.15.203:6832/3490266242":"2023-08-10T13:55:44.498027+0000","172.21.15.203:0/3892021074":"2023-08-10T13:55:44.498027+0000","172.21.15.203:0/750810033":"2023-08-10T13:55:44.498027+0000","172.21.15.203:0/1164133940":"2023-08-10T13:55:44.498027+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":7,"snaps":[{"begin":104,"length":1},{"begin":221,"length":1}]},{"pool":19,"snaps":[{"begin":23,"length":35},{"begin":79,"length":35}]}],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-08-09T14:51:30.661 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:30.659+0000 7fa0c9147640 1 -- 172.21.15.43:0/1405029249 >> [v2:172.21.15.203:6832/2566816332,v1:172.21.15.203:6833/2566816332] conn(0x7fa0b006a0e0 msgr2=0x7fa0b006c5a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T14:51:30.661 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:30.659+0000 7fa0c9147640 1 --2- 172.21.15.43:0/1405029249 >> [v2:172.21.15.203:6832/2566816332,v1:172.21.15.203:6833/2566816332] conn(0x7fa0b006a0e0 0x7fa0b006c5a0 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7fa0c40b77b0 tx=0x7fa0b4008040 comp rx=0 tx=0).stop 2023-08-09T14:51:30.661 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:30.659+0000 7fa0c9147640 1 -- 172.21.15.43:0/1405029249 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa0c40b3050 msgr2=0x7fa0c406a5f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T14:51:30.662 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:30.659+0000 7fa0c9147640 1 --2- 172.21.15.43:0/1405029249 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa0c40b3050 0x7fa0c406a5f0 secure :-1 s=READY pgs=1784 cs=0 l=1 rev1=1 crypto rx=0x7fa0bc009e20 tx=0x7fa0bc004a40 comp rx=0 tx=0).stop 2023-08-09T14:51:30.662 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:30.659+0000 7fa0c9147640 1 -- 172.21.15.43:0/1405029249 shutdown_connections 2023-08-09T14:51:30.662 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:30.659+0000 7fa0c9147640 1 --2- 172.21.15.43:0/1405029249 >> [v2:172.21.15.203:6832/2566816332,v1:172.21.15.203:6833/2566816332] conn(0x7fa0b006a0e0 0x7fa0b006c5a0 unknown :-1 s=CLOSED pgs=27 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T14:51:30.662 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:30.659+0000 7fa0c9147640 1 --2- 172.21.15.43:0/1405029249 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa0c40b3050 0x7fa0c406a5f0 unknown :-1 s=CLOSED pgs=1784 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T14:51:30.662 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:30.659+0000 7fa0c9147640 1 --2- 172.21.15.43:0/1405029249 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fa0c40b2b70 0x7fa0c414f9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T14:51:30.662 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:30.659+0000 7fa0c9147640 1 --2- 172.21.15.43:0/1405029249 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fa0c40b2790 0x7fa0c40c6010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T14:51:30.662 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:30.659+0000 7fa0c9147640 1 -- 172.21.15.43:0/1405029249 >> 172.21.15.43:0/1405029249 conn(0x7fa0c409b7b0 msgr2=0x7fa0c415b990 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-09T14:51:30.663 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:30.659+0000 7fa0c9147640 1 -- 172.21.15.43:0/1405029249 shutdown_connections 2023-08-09T14:51:30.663 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:30.659+0000 7fa0c9147640 1 -- 172.21.15.43:0/1405029249 wait complete. 2023-08-09T14:51:30.680 INFO:tasks.cephfs.filesystem:Destroying file system cephfs and related pools 2023-08-09T14:51:30.681 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-08-09T14:51:30.900 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:30.899+0000 7f23f00da640 1 Processor -- start 2023-08-09T14:51:30.901 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:30.899+0000 7f23f00da640 1 -- start start 2023-08-09T14:51:30.902 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:30.901+0000 7f23f00da640 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7f23e8143c20 0x7f23e806f540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T14:51:30.902 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:30.901+0000 7f23f00da640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f23e806cfe0 0x7f23e80c8b50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T14:51:30.902 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:30.901+0000 7f23f00da640 1 -- --> v1:172.21.15.43:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f23e80c9090 con 0x7f23e80c86f0 2023-08-09T14:51:30.903 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:30.901+0000 7f23f00da640 1 -- --> v2:172.21.15.43:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f23e8144000 con 0x7f23e806cfe0 2023-08-09T14:51:30.903 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:30.901+0000 7f23f00da640 1 -- --> v2:172.21.15.203:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f23e806c440 con 0x7f23e8143c20 2023-08-09T14:51:30.904 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:30.901+0000 7f23ee677640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f23e806cfe0 0x7f23e80c8b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T14:51:30.904 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:30.901+0000 7f23ee677640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f23e806cfe0 0x7f23e80c8b50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:39912/0 (socket says 172.21.15.43:39912) 2023-08-09T14:51:30.904 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:30.901+0000 7f23ee677640 1 -- 172.21.15.43:0/438910270 learned_addr learned my addr 172.21.15.43:0/438910270 (peer_addr_for_me v2:172.21.15.43:0/0) 2023-08-09T14:51:30.904 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:30.902+0000 7f23ede76640 1 --2- 172.21.15.43:0/438910270 >> v2:172.21.15.203:3300/0 conn(0x7f23e8143c20 0x7f23e806f540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T14:51:30.904 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:30.902+0000 7f23ee677640 1 -- 172.21.15.43:0/438910270 >> v2:172.21.15.203:3300/0 conn(0x7f23e8143c20 msgr2=0x7f23e806f540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-09T14:51:30.904 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:30.902+0000 7f23ee677640 1 --2- 172.21.15.43:0/438910270 >> v2:172.21.15.203:3300/0 conn(0x7f23e8143c20 0x7f23e806f540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T14:51:30.905 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:30.902+0000 7f23ee677640 1 -- 172.21.15.43:0/438910270 >> v1:172.21.15.43:6789/0 conn(0x7f23e80c86f0 legacy=0x7f23e806c8d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T14:51:30.905 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:30.902+0000 7f23ee677640 1 -- 172.21.15.43:0/438910270 --> v2:172.21.15.43:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f23e806b830 con 0x7f23e806cfe0 2023-08-09T14:51:30.905 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:30.902+0000 7f23ece74640 1 -- 172.21.15.43:0/438910270 <== mon.0 v1:172.21.15.43:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 252422203 0 0) 0x7f23e80c9090 con 0x7f23e80c86f0 2023-08-09T14:51:30.906 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:30.903+0000 7f23ee677640 1 --2- 172.21.15.43:0/438910270 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f23e806cfe0 0x7f23e80c8b50 secure :-1 s=READY pgs=1786 cs=0 l=1 rev1=1 crypto rx=0x7f23e806e0f0 tx=0x7f23e002be50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c89b4062eed6a3e1 server_cookie=0 in_seq=0 out_seq=0 2023-08-09T14:51:30.906 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:30.903+0000 7f23ece74640 1 -- 172.21.15.43:0/438910270 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f23e002c9f0 con 0x7f23e806cfe0 2023-08-09T14:51:30.907 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:30.903+0000 7f23ece74640 1 -- 172.21.15.43:0/438910270 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f23e002cb50 con 0x7f23e806cfe0 2023-08-09T14:51:30.907 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:30.903+0000 7f23ece74640 1 -- 172.21.15.43:0/438910270 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f23e00343f0 con 0x7f23e806cfe0 2023-08-09T14:51:30.907 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:30.904+0000 7f23f00da640 1 -- 172.21.15.43:0/438910270 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f23e806cfe0 msgr2=0x7f23e80c8b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T14:51:30.907 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:30.904+0000 7f23f00da640 1 --2- 172.21.15.43:0/438910270 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f23e806cfe0 0x7f23e80c8b50 secure :-1 s=READY pgs=1786 cs=0 l=1 rev1=1 crypto rx=0x7f23e806e0f0 tx=0x7f23e002be50 comp rx=0 tx=0).stop 2023-08-09T14:51:30.907 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:30.904+0000 7f23f00da640 1 -- 172.21.15.43:0/438910270 shutdown_connections 2023-08-09T14:51:30.907 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:30.904+0000 7f23f00da640 1 --2- 172.21.15.43:0/438910270 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f23e806cfe0 0x7f23e80c8b50 unknown :-1 s=CLOSED pgs=1786 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T14:51:30.907 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:30.904+0000 7f23f00da640 1 --2- 172.21.15.43:0/438910270 >> v2:172.21.15.203:3300/0 conn(0x7f23e8143c20 0x7f23e806f540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T14:51:30.908 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:30.904+0000 7f23f00da640 1 -- 172.21.15.43:0/438910270 >> 172.21.15.43:0/438910270 conn(0x7f23e80ce0a0 msgr2=0x7f23e80ce480 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-09T14:51:30.908 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:30.904+0000 7f23f00da640 1 -- 172.21.15.43:0/438910270 shutdown_connections 2023-08-09T14:51:30.908 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:30.904+0000 7f23f00da640 1 -- 172.21.15.43:0/438910270 wait complete. 2023-08-09T14:51:30.909 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:30.906+0000 7f23f00da640 1 Processor -- start 2023-08-09T14:51:30.909 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:30.906+0000 7f23f00da640 1 -- start start 2023-08-09T14:51:30.909 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:30.907+0000 7f23f00da640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f23e806cfe0 0x7f23e8158f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T14:51:30.910 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:30.907+0000 7f23f00da640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f23e80c86f0 0x7f23e815a9d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T14:51:30.910 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:30.907+0000 7f23f00da640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f23e8143c20 0x7f23e815c4c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T14:51:30.910 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:30.907+0000 7f23f00da640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f23e80b3c10 con 0x7f23e80c86f0 2023-08-09T14:51:30.910 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:30.907+0000 7f23f00da640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f23e815d850 con 0x7f23e8143c20 2023-08-09T14:51:30.910 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:30.907+0000 7f23f00da640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f23e81594b0 con 0x7f23e806cfe0 2023-08-09T14:51:30.910 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:30.907+0000 7f23ed675640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f23e80c86f0 0x7f23e815a9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T14:51:30.910 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:30.907+0000 7f23ed675640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f23e80c86f0 0x7f23e815a9d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:39916/0 (socket says 172.21.15.43:39916) 2023-08-09T14:51:30.911 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:30.907+0000 7f23ed675640 1 -- 172.21.15.43:0/906250578 learned_addr learned my addr 172.21.15.43:0/906250578 (peer_addr_for_me v2:172.21.15.43:0/0) 2023-08-09T14:51:30.911 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:30.908+0000 7f23ee677640 1 --2- 172.21.15.43:0/906250578 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f23e8143c20 0x7f23e815c4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T14:51:30.911 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:30.908+0000 7f23ed675640 1 -- 172.21.15.43:0/906250578 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f23e806cfe0 msgr2=0x7f23e8158f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T14:51:30.911 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:30.908+0000 7f23ed675640 1 --2- 172.21.15.43:0/906250578 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f23e806cfe0 0x7f23e8158f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T14:51:30.911 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:30.908+0000 7f23ed675640 1 -- 172.21.15.43:0/906250578 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f23e8143c20 msgr2=0x7f23e815c4c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T14:51:30.911 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:30.908+0000 7f23ed675640 1 --2- 172.21.15.43:0/906250578 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f23e8143c20 0x7f23e815c4c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T14:51:30.911 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:30.908+0000 7f23ed675640 1 -- 172.21.15.43:0/906250578 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f23e0009b30 con 0x7f23e80c86f0 2023-08-09T14:51:30.912 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:30.908+0000 7f23ee677640 1 --2- 172.21.15.43:0/906250578 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f23e8143c20 0x7f23e815c4c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-08-09T14:51:30.912 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:30.908+0000 7f23ed675640 1 --2- 172.21.15.43:0/906250578 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f23e80c86f0 0x7f23e815a9d0 secure :-1 s=READY pgs=1787 cs=0 l=1 rev1=1 crypto rx=0x7f23d800c410 tx=0x7f23d800c8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-09T14:51:30.913 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:30.908+0000 7f23de7fc640 1 -- 172.21.15.43:0/906250578 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f23d80035f0 con 0x7f23e80c86f0 2023-08-09T14:51:30.913 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:30.908+0000 7f23de7fc640 1 -- 172.21.15.43:0/906250578 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f23d8003770 con 0x7f23e80c86f0 2023-08-09T14:51:30.913 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:30.908+0000 7f23de7fc640 1 -- 172.21.15.43:0/906250578 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f23d8003b20 con 0x7f23e80c86f0 2023-08-09T14:51:30.913 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:30.908+0000 7f23f00da640 1 -- 172.21.15.43:0/906250578 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f23e80b3930 con 0x7f23e80c86f0 2023-08-09T14:51:30.913 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:30.909+0000 7f23f00da640 1 -- 172.21.15.43:0/906250578 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f23e815d570 con 0x7f23e80c86f0 2023-08-09T14:51:30.914 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:30.910+0000 7f23f00da640 1 -- 172.21.15.43:0/906250578 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f23e80739f0 con 0x7f23e80c86f0 2023-08-09T14:51:30.914 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:30.911+0000 7f23de7fc640 1 -- 172.21.15.43:0/906250578 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 78) v1 ==== 88387+0+0 (secure 0 0 0) 0x7f23d8003c80 con 0x7f23e80c86f0 2023-08-09T14:51:30.914 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:30.911+0000 7f23de7fc640 1 --2- 172.21.15.43:0/906250578 >> [v2:172.21.15.203:6832/2566816332,v1:172.21.15.203:6833/2566816332] conn(0x7f23cc06a0e0 0x7f23cc06c5a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T14:51:30.914 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:30.912+0000 7f23de7fc640 1 -- 172.21.15.43:0/906250578 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(160..160 src has 1..160) v4 ==== 7216+0+0 (secure 0 0 0) 0x7f23d80880c0 con 0x7f23e80c86f0 2023-08-09T14:51:30.915 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:30.912+0000 7f23ede76640 1 --2- 172.21.15.43:0/906250578 >> [v2:172.21.15.203:6832/2566816332,v1:172.21.15.203:6833/2566816332] conn(0x7f23cc06a0e0 0x7f23cc06c5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T14:51:30.915 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:30.913+0000 7f23ede76640 1 --2- 172.21.15.43:0/906250578 >> [v2:172.21.15.203:6832/2566816332,v1:172.21.15.203:6833/2566816332] conn(0x7f23cc06a0e0 0x7f23cc06c5a0 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7f23e4006fd0 tx=0x7f23e4008040 comp rx=0 tx=0).ready entity=mgr.7765 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-09T14:51:30.917 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:30.916+0000 7f23de7fc640 1 -- 172.21.15.43:0/906250578 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178066 (secure 0 0 0) 0x7f23d80532f0 con 0x7f23e80c86f0 2023-08-09T14:51:31.150 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:31.149+0000 7f23f00da640 1 -- 172.21.15.43:0/906250578 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f23e813ca30 con 0x7f23e80c86f0 2023-08-09T14:51:31.151 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:31.150+0000 7f23de7fc640 1 -- 172.21.15.43:0/906250578 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 70 v70) v1 ==== 94+0+4883 (secure 0 0 0) 0x7f23d8017090 con 0x7f23e80c86f0 2023-08-09T14:51:31.151 INFO:teuthology.orchestra.run.smithi043.stdout: 2023-08-09T14:51:31.152 INFO:teuthology.orchestra.run.smithi043.stdout:{"epoch":70,"default_fscid":8,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6812,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.203:6835/2100579784","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6834","nonce":2100579784},{"type":"v1","addr":"172.21.15.203:6835","nonce":2100579784}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":65},{"gid":6815,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.203:6837/1689128061","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6836","nonce":1689128061},{"type":"v1","addr":"172.21.15.203:6837","nonce":1689128061}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":65},{"gid":7543,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.43:6835/1126540259","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6833","nonce":1126540259},{"type":"v1","addr":"172.21.15.43:6835","nonce":1126540259}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":65}],"filesystems":[{"mdsmap":{"epoch":70,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-08-09T13:55:28.794042+0000","modified":"2023-08-09T13:55:32.388404+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":9126},"failed":[],"damaged":[],"stopped":[],"info":{"gid_9126":{"gid":9126,"name":"a","rank":0,"incarnation":66,"state":"up:active","state_seq":3,"addr":"172.21.15.43:6834/2023142010","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6832","nonce":2023142010},{"type":"v1","addr":"172.21.15.43:6834","nonce":2023142010}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[19],"metadata_pool":18,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":8}]} 2023-08-09T14:51:31.152 INFO:teuthology.orchestra.run.smithi043.stderr:dumped fsmap epoch 70 2023-08-09T14:51:31.154 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:31.152+0000 7f23f00da640 1 -- 172.21.15.43:0/906250578 >> [v2:172.21.15.203:6832/2566816332,v1:172.21.15.203:6833/2566816332] conn(0x7f23cc06a0e0 msgr2=0x7f23cc06c5a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T14:51:31.154 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:31.152+0000 7f23f00da640 1 --2- 172.21.15.43:0/906250578 >> [v2:172.21.15.203:6832/2566816332,v1:172.21.15.203:6833/2566816332] conn(0x7f23cc06a0e0 0x7f23cc06c5a0 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7f23e4006fd0 tx=0x7f23e4008040 comp rx=0 tx=0).stop 2023-08-09T14:51:31.154 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:31.152+0000 7f23f00da640 1 -- 172.21.15.43:0/906250578 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f23e80c86f0 msgr2=0x7f23e815a9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T14:51:31.154 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:31.152+0000 7f23f00da640 1 --2- 172.21.15.43:0/906250578 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f23e80c86f0 0x7f23e815a9d0 secure :-1 s=READY pgs=1787 cs=0 l=1 rev1=1 crypto rx=0x7f23d800c410 tx=0x7f23d800c8d0 comp rx=0 tx=0).stop 2023-08-09T14:51:31.154 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:31.152+0000 7f23f00da640 1 -- 172.21.15.43:0/906250578 shutdown_connections 2023-08-09T14:51:31.154 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:31.152+0000 7f23f00da640 1 --2- 172.21.15.43:0/906250578 >> [v2:172.21.15.203:6832/2566816332,v1:172.21.15.203:6833/2566816332] conn(0x7f23cc06a0e0 0x7f23cc06c5a0 unknown :-1 s=CLOSED pgs=28 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T14:51:31.155 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:31.152+0000 7f23f00da640 1 --2- 172.21.15.43:0/906250578 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f23e8143c20 0x7f23e815c4c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T14:51:31.155 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:31.152+0000 7f23f00da640 1 --2- 172.21.15.43:0/906250578 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f23e80c86f0 0x7f23e815a9d0 unknown :-1 s=CLOSED pgs=1787 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T14:51:31.155 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:31.152+0000 7f23f00da640 1 --2- 172.21.15.43:0/906250578 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f23e806cfe0 0x7f23e8158f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T14:51:31.155 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:31.152+0000 7f23f00da640 1 -- 172.21.15.43:0/906250578 >> 172.21.15.43:0/906250578 conn(0x7f23e80ce0a0 msgr2=0x7f23e80adc00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-09T14:51:31.155 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:31.152+0000 7f23f00da640 1 -- 172.21.15.43:0/906250578 shutdown_connections 2023-08-09T14:51:31.155 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:31.152+0000 7f23f00da640 1 -- 172.21.15.43:0/906250578 wait complete. 2023-08-09T14:51:31.174 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-08-09T14:51:31.396 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:31.394+0000 7fad88b86640 1 Processor -- start 2023-08-09T14:51:31.396 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:31.394+0000 7fad88b86640 1 -- start start 2023-08-09T14:51:31.397 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:31.396+0000 7fad88b86640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fad840713b0 0x7fad840c8fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T14:51:31.398 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:31.396+0000 7fad88b86640 1 -- --> v1:172.21.15.43:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fad840cdc70 con 0x7fad8415a6a0 2023-08-09T14:51:31.398 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:31.396+0000 7fad88b86640 1 -- --> v1:172.21.15.203:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fad84070910 con 0x7fad840c94f0 2023-08-09T14:51:31.398 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:31.396+0000 7fad88b86640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fad840c9960 con 0x7fad840713b0 2023-08-09T14:51:31.398 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:31.396+0000 7fad837fe640 1 --1- >> v1:172.21.15.43:6789/0 conn(0x7fad8415a6a0 0x7fad840cbd90 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.43:6789/0 says I am v1:172.21.15.43:54936/0 (socket says 172.21.15.43:54936) 2023-08-09T14:51:31.398 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:31.396+0000 7fad837fe640 1 -- 172.21.15.43:0/2568569365 learned_addr learned my addr 172.21.15.43:0/2568569365 (peer_addr_for_me v1:172.21.15.43:0/0) 2023-08-09T14:51:31.400 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:31.397+0000 7fad7bfff640 1 --2- 172.21.15.43:0/2568569365 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fad840713b0 0x7fad840c8fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T14:51:31.400 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:31.397+0000 7fad82ffd640 1 -- 172.21.15.43:0/2568569365 <== mon.0 v1:172.21.15.43:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2167440363 0 0) 0x7fad840cdc70 con 0x7fad8415a6a0 2023-08-09T14:51:31.400 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:31.397+0000 7fad82ffd640 1 -- 172.21.15.43:0/2568569365 --> v1:172.21.15.43:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fad70003520 con 0x7fad8415a6a0 2023-08-09T14:51:31.400 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:31.397+0000 7fad82ffd640 1 -- 172.21.15.43:0/2568569365 <== mon.0 v1:172.21.15.43:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 44226126 0 0) 0x7fad70003520 con 0x7fad8415a6a0 2023-08-09T14:51:31.400 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:31.397+0000 7fad82ffd640 1 -- 172.21.15.43:0/2568569365 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fad840713b0 msgr2=0x7fad840c8fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-09T14:51:31.400 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:31.397+0000 7fad82ffd640 1 --2- 172.21.15.43:0/2568569365 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fad840713b0 0x7fad840c8fb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T14:51:31.401 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:31.397+0000 7fad82ffd640 1 -- 172.21.15.43:0/2568569365 >> v1:172.21.15.203:6789/0 conn(0x7fad840c94f0 legacy=0x7fad8415ad20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T14:51:31.401 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:31.397+0000 7fad82ffd640 1 -- 172.21.15.43:0/2568569365 --> v1:172.21.15.43:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fad840c88c0 con 0x7fad8415a6a0 2023-08-09T14:51:31.401 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:31.397+0000 7fad82ffd640 1 -- 172.21.15.43:0/2568569365 <== mon.0 v1:172.21.15.43:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1285150740 0 0) 0x7fad74001670 con 0x7fad8415a6a0 2023-08-09T14:51:31.401 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:31.397+0000 7fad82ffd640 1 -- 172.21.15.43:0/2568569365 >> v1:172.21.15.43:6789/0 conn(0x7fad8415a6a0 legacy=0x7fad840cbd90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T14:51:31.401 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:31.397+0000 7fad82ffd640 1 --2- 172.21.15.43:0/2568569365 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fad700039a0 0x7fad70003dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T14:51:31.401 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:31.397+0000 7fad82ffd640 1 --2- 172.21.15.43:0/2568569365 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fad700047c0 0x7fad70004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T14:51:31.401 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:31.397+0000 7fad82ffd640 1 --2- 172.21.15.43:0/2568569365 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fad840713b0 0x7fad700051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T14:51:31.402 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:31.397+0000 7fad82ffd640 1 -- 172.21.15.43:0/2568569365 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fad840c9960 con 0x7fad700047c0 2023-08-09T14:51:31.402 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:31.397+0000 7fad82ffd640 1 -- 172.21.15.43:0/2568569365 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fad7000bae0 con 0x7fad700039a0 2023-08-09T14:51:31.402 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:31.397+0000 7fad82ffd640 1 -- 172.21.15.43:0/2568569365 --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fad7000bca0 con 0x7fad840713b0 2023-08-09T14:51:31.402 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:31.398+0000 7fad7bfff640 1 --2- 172.21.15.43:0/2568569365 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fad700047c0 0x7fad70004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T14:51:31.402 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:31.398+0000 7fad837fe640 1 --2- 172.21.15.43:0/2568569365 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fad700039a0 0x7fad70003dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T14:51:31.402 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:31.398+0000 7fad7bfff640 1 -- 172.21.15.43:0/2568569365 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fad840713b0 msgr2=0x7fad700051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-09T14:51:31.402 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:31.398+0000 7fad7bfff640 1 --2- 172.21.15.43:0/2568569365 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fad840713b0 0x7fad700051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T14:51:31.402 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:31.398+0000 7fad7bfff640 1 -- 172.21.15.43:0/2568569365 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fad700039a0 msgr2=0x7fad70003dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-09T14:51:31.403 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:31.398+0000 7fad7bfff640 1 --2- 172.21.15.43:0/2568569365 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fad700039a0 0x7fad70003dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T14:51:31.403 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:31.398+0000 7fad7bfff640 1 -- 172.21.15.43:0/2568569365 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fad70003520 con 0x7fad700047c0 2023-08-09T14:51:31.403 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:31.398+0000 7fad837fe640 1 --2- 172.21.15.43:0/2568569365 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fad700039a0 0x7fad70003dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-08-09T14:51:31.404 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:31.400+0000 7fad7bfff640 1 --2- 172.21.15.43:0/2568569365 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fad700047c0 0x7fad70004c40 secure :-1 s=READY pgs=1789 cs=0 l=1 rev1=1 crypto rx=0x7fad6c00bd60 tx=0x7fad700057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4b4b812b66edece8 server_cookie=0 in_seq=0 out_seq=0 2023-08-09T14:51:31.404 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:31.400+0000 7fad82ffd640 1 -- 172.21.15.43:0/2568569365 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fad6c02f710 con 0x7fad700047c0 2023-08-09T14:51:31.405 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:31.400+0000 7fad82ffd640 1 -- 172.21.15.43:0/2568569365 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fad6c02f870 con 0x7fad700047c0 2023-08-09T14:51:31.405 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:31.400+0000 7fad82ffd640 1 -- 172.21.15.43:0/2568569365 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fad6c02fc00 con 0x7fad700047c0 2023-08-09T14:51:31.405 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:31.400+0000 7fad88b86640 1 -- 172.21.15.43:0/2568569365 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fad700047c0 msgr2=0x7fad70004c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T14:51:31.405 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:31.400+0000 7fad88b86640 1 --2- 172.21.15.43:0/2568569365 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fad700047c0 0x7fad70004c40 secure :-1 s=READY pgs=1789 cs=0 l=1 rev1=1 crypto rx=0x7fad6c00bd60 tx=0x7fad700057f0 comp rx=0 tx=0).stop 2023-08-09T14:51:31.405 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:31.400+0000 7fad83fff640 1 -- 172.21.15.43:0/2568569365 reap_dead start 2023-08-09T14:51:31.405 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:31.401+0000 7fad88b86640 1 -- 172.21.15.43:0/2568569365 shutdown_connections 2023-08-09T14:51:31.405 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:31.401+0000 7fad88b86640 1 -- 172.21.15.43:0/2568569365 >> 172.21.15.43:0/2568569365 conn(0x7fad840cde90 msgr2=0x7fad840ce2a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-09T14:51:31.405 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:31.401+0000 7fad88b86640 1 -- 172.21.15.43:0/2568569365 shutdown_connections 2023-08-09T14:51:31.406 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:31.401+0000 7fad88b86640 1 -- 172.21.15.43:0/2568569365 wait complete. 2023-08-09T14:51:31.406 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:31.402+0000 7fad88b86640 1 Processor -- start 2023-08-09T14:51:31.406 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:31.403+0000 7fad88b86640 1 -- start start 2023-08-09T14:51:31.407 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:31.404+0000 7fad88b86640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fad700047c0 0x7fad840b3100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T14:51:31.407 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:31.404+0000 7fad88b86640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fad8409a4b0 0x7fad8409a910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T14:51:31.408 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:31.404+0000 7fad88b86640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fad841f4fd0 0x7fad841f5450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T14:51:31.408 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:31.404+0000 7fad88b86640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fad841f5990 con 0x7fad841f4fd0 2023-08-09T14:51:31.408 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:31.404+0000 7fad88b86640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fad8409af70 con 0x7fad700047c0 2023-08-09T14:51:31.408 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:31.404+0000 7fad88b86640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fad8409b250 con 0x7fad8409a4b0 2023-08-09T14:51:31.408 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:31.404+0000 7fad83fff640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fad841f4fd0 0x7fad841f5450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T14:51:31.408 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:31.404+0000 7fad83fff640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fad841f4fd0 0x7fad841f5450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:39934/0 (socket says 172.21.15.43:39934) 2023-08-09T14:51:31.408 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:31.404+0000 7fad83fff640 1 -- 172.21.15.43:0/91573845 learned_addr learned my addr 172.21.15.43:0/91573845 (peer_addr_for_me v2:172.21.15.43:0/0) 2023-08-09T14:51:31.409 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:31.404+0000 7fad83fff640 1 -- 172.21.15.43:0/91573845 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fad8409a4b0 msgr2=0x7fad8409a910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T14:51:31.409 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:31.404+0000 7fad83fff640 1 --2- 172.21.15.43:0/91573845 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fad8409a4b0 0x7fad8409a910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T14:51:31.409 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:31.404+0000 7fad83fff640 1 -- 172.21.15.43:0/91573845 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fad700047c0 msgr2=0x7fad840b3100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T14:51:31.409 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:31.404+0000 7fad83fff640 1 --2- 172.21.15.43:0/91573845 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fad700047c0 0x7fad840b3100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T14:51:31.409 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:31.405+0000 7fad83fff640 1 -- 172.21.15.43:0/91573845 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fad8409b530 con 0x7fad841f4fd0 2023-08-09T14:51:31.409 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:31.405+0000 7fad83fff640 1 --2- 172.21.15.43:0/91573845 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fad841f4fd0 0x7fad841f5450 secure :-1 s=READY pgs=1790 cs=0 l=1 rev1=1 crypto rx=0x7fad840cb9a0 tx=0x7fad7c00cb90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-09T14:51:31.409 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:31.405+0000 7fad80ff9640 1 -- 172.21.15.43:0/91573845 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fad7c005d40 con 0x7fad841f4fd0 2023-08-09T14:51:31.409 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:31.405+0000 7fad80ff9640 1 -- 172.21.15.43:0/91573845 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fad7c00ced0 con 0x7fad841f4fd0 2023-08-09T14:51:31.410 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:31.405+0000 7fad80ff9640 1 -- 172.21.15.43:0/91573845 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fad7c005ea0 con 0x7fad841f4fd0 2023-08-09T14:51:31.410 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:31.405+0000 7fad88b86640 1 -- 172.21.15.43:0/91573845 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fad840c6120 con 0x7fad841f4fd0 2023-08-09T14:51:31.410 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:31.405+0000 7fad88b86640 1 -- 172.21.15.43:0/91573845 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fad840b3640 con 0x7fad841f4fd0 2023-08-09T14:51:31.410 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:31.406+0000 7fad88b86640 1 -- 172.21.15.43:0/91573845 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fad840739f0 con 0x7fad841f4fd0 2023-08-09T14:51:31.411 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:31.407+0000 7fad80ff9640 1 -- 172.21.15.43:0/91573845 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 79) v1 ==== 88443+0+0 (secure 0 0 0) 0x7fad7c00a850 con 0x7fad841f4fd0 2023-08-09T14:51:31.411 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:31.408+0000 7fad80ff9640 1 --2- 172.21.15.43:0/91573845 >> [v2:172.21.15.203:6832/2566816332,v1:172.21.15.203:6833/2566816332] conn(0x7fad6406a1a0 0x7fad6406c660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T14:51:31.411 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:31.408+0000 7fad80ff9640 1 -- 172.21.15.43:0/91573845 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(160..160 src has 1..160) v4 ==== 7216+0+0 (secure 0 0 0) 0x7fad7c087d40 con 0x7fad841f4fd0 2023-08-09T14:51:31.411 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:31.409+0000 7fad837fe640 1 --2- 172.21.15.43:0/91573845 >> [v2:172.21.15.203:6832/2566816332,v1:172.21.15.203:6833/2566816332] conn(0x7fad6406a1a0 0x7fad6406c660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T14:51:31.411 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:31.410+0000 7fad837fe640 1 --2- 172.21.15.43:0/91573845 >> [v2:172.21.15.203:6832/2566816332,v1:172.21.15.203:6833/2566816332] conn(0x7fad6406a1a0 0x7fad6406c660 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7fad740017a0 tx=0x7fad74009040 comp rx=0 tx=0).ready entity=mgr.7765 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-09T14:51:31.414 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:31.413+0000 7fad80ff9640 1 -- 172.21.15.43:0/91573845 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178066 (secure 0 0 0) 0x7fad7c052e20 con 0x7fad841f4fd0 2023-08-09T14:51:31.653 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:31.652+0000 7fad88b86640 1 -- 172.21.15.43:0/91573845 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fad840cec10 con 0x7fad841f4fd0 2023-08-09T14:51:31.654 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:31.653+0000 7fad80ff9640 1 -- 172.21.15.43:0/91573845 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 70 v70) v1 ==== 94+0+4883 (secure 0 0 0) 0x7fad7c056440 con 0x7fad841f4fd0 2023-08-09T14:51:31.655 INFO:teuthology.orchestra.run.smithi043.stdout: 2023-08-09T14:51:31.655 INFO:teuthology.orchestra.run.smithi043.stdout:{"epoch":70,"default_fscid":8,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6812,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.203:6835/2100579784","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6834","nonce":2100579784},{"type":"v1","addr":"172.21.15.203:6835","nonce":2100579784}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":65},{"gid":6815,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.203:6837/1689128061","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6836","nonce":1689128061},{"type":"v1","addr":"172.21.15.203:6837","nonce":1689128061}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":65},{"gid":7543,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.43:6835/1126540259","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6833","nonce":1126540259},{"type":"v1","addr":"172.21.15.43:6835","nonce":1126540259}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":65}],"filesystems":[{"mdsmap":{"epoch":70,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-08-09T13:55:28.794042+0000","modified":"2023-08-09T13:55:32.388404+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":9126},"failed":[],"damaged":[],"stopped":[],"info":{"gid_9126":{"gid":9126,"name":"a","rank":0,"incarnation":66,"state":"up:active","state_seq":3,"addr":"172.21.15.43:6834/2023142010","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6832","nonce":2023142010},{"type":"v1","addr":"172.21.15.43:6834","nonce":2023142010}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[19],"metadata_pool":18,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":8}]} 2023-08-09T14:51:31.655 INFO:teuthology.orchestra.run.smithi043.stderr:dumped fsmap epoch 70 2023-08-09T14:51:31.657 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:31.656+0000 7fad88b86640 1 -- 172.21.15.43:0/91573845 >> [v2:172.21.15.203:6832/2566816332,v1:172.21.15.203:6833/2566816332] conn(0x7fad6406a1a0 msgr2=0x7fad6406c660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T14:51:31.657 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:31.656+0000 7fad88b86640 1 --2- 172.21.15.43:0/91573845 >> [v2:172.21.15.203:6832/2566816332,v1:172.21.15.203:6833/2566816332] conn(0x7fad6406a1a0 0x7fad6406c660 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7fad740017a0 tx=0x7fad74009040 comp rx=0 tx=0).stop 2023-08-09T14:51:31.657 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:31.656+0000 7fad88b86640 1 -- 172.21.15.43:0/91573845 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fad841f4fd0 msgr2=0x7fad841f5450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T14:51:31.657 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:31.656+0000 7fad88b86640 1 --2- 172.21.15.43:0/91573845 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fad841f4fd0 0x7fad841f5450 secure :-1 s=READY pgs=1790 cs=0 l=1 rev1=1 crypto rx=0x7fad840cb9a0 tx=0x7fad7c00cb90 comp rx=0 tx=0).stop 2023-08-09T14:51:31.658 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:31.656+0000 7fad88b86640 1 -- 172.21.15.43:0/91573845 shutdown_connections 2023-08-09T14:51:31.658 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:31.656+0000 7fad88b86640 1 --2- 172.21.15.43:0/91573845 >> [v2:172.21.15.203:6832/2566816332,v1:172.21.15.203:6833/2566816332] conn(0x7fad6406a1a0 0x7fad6406c660 unknown :-1 s=CLOSED pgs=29 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T14:51:31.658 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:31.656+0000 7fad88b86640 1 --2- 172.21.15.43:0/91573845 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fad841f4fd0 0x7fad841f5450 unknown :-1 s=CLOSED pgs=1790 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T14:51:31.658 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:31.656+0000 7fad88b86640 1 --2- 172.21.15.43:0/91573845 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fad8409a4b0 0x7fad8409a910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T14:51:31.659 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:31.656+0000 7fad88b86640 1 --2- 172.21.15.43:0/91573845 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fad700047c0 0x7fad840b3100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T14:51:31.659 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:31.656+0000 7fad88b86640 1 -- 172.21.15.43:0/91573845 >> 172.21.15.43:0/91573845 conn(0x7fad840cde90 msgr2=0x7fad840cbd90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-09T14:51:31.659 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:31.656+0000 7fad88b86640 1 -- 172.21.15.43:0/91573845 shutdown_connections 2023-08-09T14:51:31.659 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:31.656+0000 7fad88b86640 1 -- 172.21.15.43:0/91573845 wait complete. 2023-08-09T14:51:31.677 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-08-09T14:51:31.899 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:31.898+0000 7f7f85c3f640 1 Processor -- start 2023-08-09T14:51:31.900 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:31.898+0000 7f7f85c3f640 1 -- start start 2023-08-09T14:51:31.901 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:31.900+0000 7f7f85c3f640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f7f8014a5c0 0x7f7f80160f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T14:51:31.901 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:31.900+0000 7f7f85c3f640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f7f800c3130 0x7f7f800cbc70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T14:51:31.902 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:31.900+0000 7f7f85c3f640 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7f7f800cc1b0 0x7f7f80164510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T14:51:31.902 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:31.900+0000 7f7f85c3f640 1 -- --> v2:172.21.15.43:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7f800c4420 con 0x7f7f8014a5c0 2023-08-09T14:51:31.902 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:31.900+0000 7f7f85c3f640 1 -- --> v2:172.21.15.203:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7f800cc640 con 0x7f7f800cc1b0 2023-08-09T14:51:31.902 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:31.900+0000 7f7f85c3f640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7f800c38a0 con 0x7f7f800c3130 2023-08-09T14:51:31.903 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:31.900+0000 7f7f7f7fe640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f7f8014a5c0 0x7f7f80160f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T14:51:31.903 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:31.900+0000 7f7f7f7fe640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f7f8014a5c0 0x7f7f80160f20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:39938/0 (socket says 172.21.15.43:39938) 2023-08-09T14:51:31.903 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:31.900+0000 7f7f7f7fe640 1 -- 172.21.15.43:0/417271451 learned_addr learned my addr 172.21.15.43:0/417271451 (peer_addr_for_me v2:172.21.15.43:0/0) 2023-08-09T14:51:31.903 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:31.901+0000 7f7f7ffff640 1 --2- 172.21.15.43:0/417271451 >> v2:172.21.15.203:3300/0 conn(0x7f7f800cc1b0 0x7f7f80164510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T14:51:31.903 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:31.901+0000 7f7f7effd640 1 --2- 172.21.15.43:0/417271451 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f7f800c3130 0x7f7f800cbc70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T14:51:31.904 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:31.901+0000 7f7f7f7fe640 1 -- 172.21.15.43:0/417271451 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f7f800c3130 msgr2=0x7f7f800cbc70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-09T14:51:31.904 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:31.901+0000 7f7f7f7fe640 1 --2- 172.21.15.43:0/417271451 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f7f800c3130 0x7f7f800cbc70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T14:51:31.904 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:31.901+0000 7f7f7f7fe640 1 -- 172.21.15.43:0/417271451 >> v2:172.21.15.203:3300/0 conn(0x7f7f800cc1b0 msgr2=0x7f7f80164510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-09T14:51:31.904 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:31.901+0000 7f7f7f7fe640 1 --2- 172.21.15.43:0/417271451 >> v2:172.21.15.203:3300/0 conn(0x7f7f800cc1b0 0x7f7f80164510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T14:51:31.904 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:31.901+0000 7f7f7f7fe640 1 -- 172.21.15.43:0/417271451 --> v2:172.21.15.43:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7f800c3b80 con 0x7f7f8014a5c0 2023-08-09T14:51:31.905 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:31.902+0000 7f7f7f7fe640 1 --2- 172.21.15.43:0/417271451 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f7f8014a5c0 0x7f7f80160f20 secure :-1 s=READY pgs=1791 cs=0 l=1 rev1=1 crypto rx=0x7f7f8006f4d0 tx=0x7f7f7002b870 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ea5c23b0e4e9a6b2 server_cookie=0 in_seq=0 out_seq=0 2023-08-09T14:51:31.905 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:31.903+0000 7f7f7e7fc640 1 -- 172.21.15.43:0/417271451 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7f700317d0 con 0x7f7f8014a5c0 2023-08-09T14:51:31.906 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:31.903+0000 7f7f7e7fc640 1 -- 172.21.15.43:0/417271451 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7f70031930 con 0x7f7f8014a5c0 2023-08-09T14:51:31.906 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:31.903+0000 7f7f7e7fc640 1 -- 172.21.15.43:0/417271451 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7f70031cc0 con 0x7f7f8014a5c0 2023-08-09T14:51:31.906 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:31.903+0000 7f7f85c3f640 1 -- 172.21.15.43:0/417271451 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f7f8014a5c0 msgr2=0x7f7f80160f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T14:51:31.906 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:31.903+0000 7f7f85c3f640 1 --2- 172.21.15.43:0/417271451 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f7f8014a5c0 0x7f7f80160f20 secure :-1 s=READY pgs=1791 cs=0 l=1 rev1=1 crypto rx=0x7f7f8006f4d0 tx=0x7f7f7002b870 comp rx=0 tx=0).stop 2023-08-09T14:51:31.906 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:31.903+0000 7f7f85c3f640 1 -- 172.21.15.43:0/417271451 shutdown_connections 2023-08-09T14:51:31.907 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:31.903+0000 7f7f85c3f640 1 --2- 172.21.15.43:0/417271451 >> v2:172.21.15.203:3300/0 conn(0x7f7f800cc1b0 0x7f7f80164510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T14:51:31.907 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:31.903+0000 7f7f85c3f640 1 --2- 172.21.15.43:0/417271451 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f7f800c3130 0x7f7f800cbc70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T14:51:31.907 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:31.903+0000 7f7f85c3f640 1 --2- 172.21.15.43:0/417271451 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f7f8014a5c0 0x7f7f80160f20 unknown :-1 s=CLOSED pgs=1791 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T14:51:31.907 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:31.903+0000 7f7f85c3f640 1 -- 172.21.15.43:0/417271451 >> 172.21.15.43:0/417271451 conn(0x7f7f800ce0a0 msgr2=0x7f7f800ce480 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-09T14:51:31.907 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:31.904+0000 7f7f85c3f640 1 -- 172.21.15.43:0/417271451 shutdown_connections 2023-08-09T14:51:31.907 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:31.904+0000 7f7f85c3f640 1 -- 172.21.15.43:0/417271451 wait complete. 2023-08-09T14:51:31.909 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:31.905+0000 7f7f85c3f640 1 Processor -- start 2023-08-09T14:51:31.910 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:31.906+0000 7f7f85c3f640 1 -- start start 2023-08-09T14:51:31.910 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:31.907+0000 7f7f85c3f640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f7f800c3130 0x7f7f800c5860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T14:51:31.910 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:31.907+0000 7f7f85c3f640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f7f800cc1b0 0x7f7f8006ba50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T14:51:31.910 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:31.907+0000 7f7f85c3f640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f7f8014a5c0 0x7f7f8006bf90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T14:51:31.910 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:31.907+0000 7f7f85c3f640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7f800c8540 con 0x7f7f8014a5c0 2023-08-09T14:51:31.910 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:31.907+0000 7f7f85c3f640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7f801f9990 con 0x7f7f800cc1b0 2023-08-09T14:51:31.910 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:31.907+0000 7f7f85c3f640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7f801f9be0 con 0x7f7f800c3130 2023-08-09T14:51:31.910 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:31.907+0000 7f7f7ffff640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f7f8014a5c0 0x7f7f8006bf90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T14:51:31.911 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:31.907+0000 7f7f7ffff640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f7f8014a5c0 0x7f7f8006bf90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:39940/0 (socket says 172.21.15.43:39940) 2023-08-09T14:51:31.911 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:31.907+0000 7f7f7ffff640 1 -- 172.21.15.43:0/3574343466 learned_addr learned my addr 172.21.15.43:0/3574343466 (peer_addr_for_me v2:172.21.15.43:0/0) 2023-08-09T14:51:31.911 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:31.907+0000 7f7f7ffff640 1 -- 172.21.15.43:0/3574343466 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f7f800c3130 msgr2=0x7f7f800c5860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T14:51:31.911 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:31.907+0000 7f7f7ffff640 1 --2- 172.21.15.43:0/3574343466 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f7f800c3130 0x7f7f800c5860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T14:51:31.911 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:31.907+0000 7f7f7ffff640 1 -- 172.21.15.43:0/3574343466 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f7f800cc1b0 msgr2=0x7f7f8006ba50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T14:51:31.911 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:31.907+0000 7f7f7ffff640 1 --2- 172.21.15.43:0/3574343466 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f7f800cc1b0 0x7f7f8006ba50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T14:51:31.911 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:31.907+0000 7f7f7ffff640 1 -- 172.21.15.43:0/3574343466 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7f700094d0 con 0x7f7f8014a5c0 2023-08-09T14:51:31.912 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:31.908+0000 7f7f7ffff640 1 --2- 172.21.15.43:0/3574343466 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f7f8014a5c0 0x7f7f8006bf90 secure :-1 s=READY pgs=1792 cs=0 l=1 rev1=1 crypto rx=0x7f7f78007cf0 tx=0x7f7f7800e4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-09T14:51:31.912 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:31.908+0000 7f7f67fff640 1 -- 172.21.15.43:0/3574343466 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7f78014420 con 0x7f7f8014a5c0 2023-08-09T14:51:31.912 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:31.908+0000 7f7f67fff640 1 -- 172.21.15.43:0/3574343466 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7f78014580 con 0x7f7f8014a5c0 2023-08-09T14:51:31.912 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:31.908+0000 7f7f67fff640 1 -- 172.21.15.43:0/3574343466 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7f78014930 con 0x7f7f8014a5c0 2023-08-09T14:51:31.912 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:31.908+0000 7f7f85c3f640 1 -- 172.21.15.43:0/3574343466 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7f801f9ec0 con 0x7f7f8014a5c0 2023-08-09T14:51:31.912 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:31.908+0000 7f7f85c3f640 1 -- 172.21.15.43:0/3574343466 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7f801fa480 con 0x7f7f8014a5c0 2023-08-09T14:51:31.913 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:31.909+0000 7f7f85c3f640 1 -- 172.21.15.43:0/3574343466 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7f800739f0 con 0x7f7f8014a5c0 2023-08-09T14:51:31.913 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:31.910+0000 7f7f67fff640 1 -- 172.21.15.43:0/3574343466 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 79) v1 ==== 88443+0+0 (secure 0 0 0) 0x7f7f78014a90 con 0x7f7f8014a5c0 2023-08-09T14:51:31.913 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:31.911+0000 7f7f67fff640 1 --2- 172.21.15.43:0/3574343466 >> [v2:172.21.15.203:6832/2566816332,v1:172.21.15.203:6833/2566816332] conn(0x7f7f6c06a170 0x7f7f6c06c630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T14:51:31.913 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:31.911+0000 7f7f67fff640 1 -- 172.21.15.43:0/3574343466 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(160..160 src has 1..160) v4 ==== 7216+0+0 (secure 0 0 0) 0x7f7f780894a0 con 0x7f7f8014a5c0 2023-08-09T14:51:31.914 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:31.912+0000 7f7f7f7fe640 1 --2- 172.21.15.43:0/3574343466 >> [v2:172.21.15.203:6832/2566816332,v1:172.21.15.203:6833/2566816332] conn(0x7f7f6c06a170 0x7f7f6c06c630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T14:51:31.914 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:31.913+0000 7f7f7f7fe640 1 --2- 172.21.15.43:0/3574343466 >> [v2:172.21.15.203:6832/2566816332,v1:172.21.15.203:6833/2566816332] conn(0x7f7f6c06a170 0x7f7f6c06c630 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0x7f7f70036000 tx=0x7f7f70030000 comp rx=0 tx=0).ready entity=mgr.7765 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-09T14:51:31.918 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:31.917+0000 7f7f67fff640 1 -- 172.21.15.43:0/3574343466 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178066 (secure 0 0 0) 0x7f7f78054580 con 0x7f7f8014a5c0 2023-08-09T14:51:32.113 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:32.112+0000 7f7f85c3f640 1 -- 172.21.15.43:0/3574343466 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f7f80146c70 con 0x7f7f8014a5c0 2023-08-09T14:51:32.115 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:32.114+0000 7f7f67fff640 1 -- 172.21.15.43:0/3574343466 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v160) v1 ==== 74+0+19721 (secure 0 0 0) 0x7f7f78057ba0 con 0x7f7f8014a5c0 2023-08-09T14:51:32.116 INFO:teuthology.orchestra.run.smithi043.stdout: 2023-08-09T14:51:32.116 INFO:teuthology.orchestra.run.smithi043.stdout:{"epoch":160,"fsid":"daac87ee-1337-4250-a955-aea4e408a39b","created":"2023-08-09T13:33:41.701198+0000","modified":"2023-08-09T14:51:27.179740+0000","last_up_change":"2023-08-09T13:33:53.783453+0000","last_in_change":"2023-08-09T13:33:45.148904+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":19,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":"rbd","create_time":"2023-08-09T13:33:54.365137+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":".mgr","create_time":"2023-08-09T13:33:54.831803+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":18,"pool_name":"cephfs_metadata","create_time":"2023-08-09T13:55:27.207507+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"146","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":19,"pool_name":"cephfs_data","create_time":"2023-08-09T13:55:27.825578+0000","flags":8193,"flags_names":"hashpspool,selfmanaged_snaps","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"159","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":113,"snap_epoch":159,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.25,"score_stable":1.25,"optimal_score":1,"raw_score_acting":1.25,"raw_score_stable":1.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"cbd49ecd-e896-4fbd-864e-5c37a829b165","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":149,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6800","nonce":502667335},{"type":"v1","addr":"172.21.15.43:6801","nonce":502667335}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6802","nonce":502667335},{"type":"v1","addr":"172.21.15.43:6803","nonce":502667335}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6806","nonce":502667335},{"type":"v1","addr":"172.21.15.43:6807","nonce":502667335}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6804","nonce":502667335},{"type":"v1","addr":"172.21.15.43:6805","nonce":502667335}]},"public_addr":"172.21.15.43:6801/502667335","cluster_addr":"172.21.15.43:6803/502667335","heartbeat_back_addr":"172.21.15.43:6807/502667335","heartbeat_front_addr":"172.21.15.43:6805/502667335","state":["exists","up"]},{"osd":1,"uuid":"75c7e0fe-1ec7-4fe8-a2f2-3661a65b8500","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":145,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6808","nonce":3289134610},{"type":"v1","addr":"172.21.15.43:6809","nonce":3289134610}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6810","nonce":3289134610},{"type":"v1","addr":"172.21.15.43:6812","nonce":3289134610}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6821","nonce":3289134610},{"type":"v1","addr":"172.21.15.43:6824","nonce":3289134610}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6815","nonce":3289134610},{"type":"v1","addr":"172.21.15.43:6818","nonce":3289134610}]},"public_addr":"172.21.15.43:6809/3289134610","cluster_addr":"172.21.15.43:6812/3289134610","heartbeat_back_addr":"172.21.15.43:6824/3289134610","heartbeat_front_addr":"172.21.15.43:6818/3289134610","state":["exists","up"]},{"osd":2,"uuid":"bb5c41e9-7f5f-4125-9b01-3bb8bfc2ea83","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":145,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6811","nonce":4079709736},{"type":"v1","addr":"172.21.15.43:6814","nonce":4079709736}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6817","nonce":4079709736},{"type":"v1","addr":"172.21.15.43:6820","nonce":4079709736}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6829","nonce":4079709736},{"type":"v1","addr":"172.21.15.43:6831","nonce":4079709736}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6823","nonce":4079709736},{"type":"v1","addr":"172.21.15.43:6826","nonce":4079709736}]},"public_addr":"172.21.15.43:6814/4079709736","cluster_addr":"172.21.15.43:6820/4079709736","heartbeat_back_addr":"172.21.15.43:6831/4079709736","heartbeat_front_addr":"172.21.15.43:6826/4079709736","state":["exists","up"]},{"osd":3,"uuid":"bfe4c8b4-694a-465c-87e6-2b2aab852613","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":149,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6813","nonce":1709538759},{"type":"v1","addr":"172.21.15.43:6816","nonce":1709538759}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6819","nonce":1709538759},{"type":"v1","addr":"172.21.15.43:6822","nonce":1709538759}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6828","nonce":1709538759},{"type":"v1","addr":"172.21.15.43:6830","nonce":1709538759}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6825","nonce":1709538759},{"type":"v1","addr":"172.21.15.43:6827","nonce":1709538759}]},"public_addr":"172.21.15.43:6816/1709538759","cluster_addr":"172.21.15.43:6822/1709538759","heartbeat_back_addr":"172.21.15.43:6830/1709538759","heartbeat_front_addr":"172.21.15.43:6827/1709538759","state":["exists","up"]},{"osd":4,"uuid":"fa2e8a77-aba1-4756-853e-0d174a5bdeba","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":145,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6800","nonce":4196866481},{"type":"v1","addr":"172.21.15.203:6801","nonce":4196866481}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6802","nonce":4196866481},{"type":"v1","addr":"172.21.15.203:6803","nonce":4196866481}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6806","nonce":4196866481},{"type":"v1","addr":"172.21.15.203:6807","nonce":4196866481}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6804","nonce":4196866481},{"type":"v1","addr":"172.21.15.203:6805","nonce":4196866481}]},"public_addr":"172.21.15.203:6801/4196866481","cluster_addr":"172.21.15.203:6803/4196866481","heartbeat_back_addr":"172.21.15.203:6807/4196866481","heartbeat_front_addr":"172.21.15.203:6805/4196866481","state":["exists","up"]},{"osd":5,"uuid":"81811e4b-4d24-4f51-a82c-f1d700a2a00d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":145,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6816","nonce":3059290081},{"type":"v1","addr":"172.21.15.203:6817","nonce":3059290081}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6818","nonce":3059290081},{"type":"v1","addr":"172.21.15.203:6819","nonce":3059290081}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6822","nonce":3059290081},{"type":"v1","addr":"172.21.15.203:6823","nonce":3059290081}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6820","nonce":3059290081},{"type":"v1","addr":"172.21.15.203:6821","nonce":3059290081}]},"public_addr":"172.21.15.203:6817/3059290081","cluster_addr":"172.21.15.203:6819/3059290081","heartbeat_back_addr":"172.21.15.203:6823/3059290081","heartbeat_front_addr":"172.21.15.203:6821/3059290081","state":["exists","up"]},{"osd":6,"uuid":"307bac4d-6a5b-443d-af75-56d695d7674f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":145,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6824","nonce":512155569},{"type":"v1","addr":"172.21.15.203:6825","nonce":512155569}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6826","nonce":512155569},{"type":"v1","addr":"172.21.15.203:6827","nonce":512155569}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6830","nonce":512155569},{"type":"v1","addr":"172.21.15.203:6831","nonce":512155569}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6828","nonce":512155569},{"type":"v1","addr":"172.21.15.203:6829","nonce":512155569}]},"public_addr":"172.21.15.203:6825/512155569","cluster_addr":"172.21.15.203:6827/512155569","heartbeat_back_addr":"172.21.15.203:6831/512155569","heartbeat_front_addr":"172.21.15.203:6829/512155569","state":["exists","up"]},{"osd":7,"uuid":"a95dc7ca-9f1b-4295-af70-68a22a2d7d5e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":145,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6808","nonce":3331450438},{"type":"v1","addr":"172.21.15.203:6809","nonce":3331450438}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6810","nonce":3331450438},{"type":"v1","addr":"172.21.15.203:6811","nonce":3331450438}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6814","nonce":3331450438},{"type":"v1","addr":"172.21.15.203:6815","nonce":3331450438}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6812","nonce":3331450438},{"type":"v1","addr":"172.21.15.203:6813","nonce":3331450438}]},"public_addr":"172.21.15.203:6809/3331450438","cluster_addr":"172.21.15.203:6811/3331450438","heartbeat_back_addr":"172.21.15.203:6815/3331450438","heartbeat_front_addr":"172.21.15.203:6813/3331450438","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-09T13:33:50.547652+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-09T13:33:50.589469+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-09T13:33:50.561724+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-09T13:33:50.599650+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-09T13:33:50.411855+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-09T13:33:50.219681+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-09T13:33:50.504664+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-09T13:33:50.441248+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"19.25","mappings":[{"from":2,"to":0}]},{"pgid":"19.26","mappings":[{"from":2,"to":0}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.203:0/1340171653":"2023-08-10T14:51:27.179649+0000","172.21.15.203:0/2575410559":"2023-08-10T14:51:27.179649+0000","172.21.15.203:0/1196185412":"2023-08-10T14:51:27.179649+0000","172.21.15.203:6833/372694067":"2023-08-10T14:51:27.179649+0000","172.21.15.203:6832/372694067":"2023-08-10T14:51:27.179649+0000","172.21.15.203:0/2726918685":"2023-08-10T14:51:27.179649+0000","172.21.15.203:0/3273594633":"2023-08-10T14:51:27.179649+0000","172.21.15.203:0/2477376154":"2023-08-10T14:51:27.179649+0000","172.21.15.203:0/2901879784":"2023-08-10T13:55:44.498027+0000","172.21.15.203:0/1501229236":"2023-08-10T13:55:44.498027+0000","172.21.15.203:0/3615051942":"2023-08-10T13:55:44.498027+0000","172.21.15.203:6833/3490266242":"2023-08-10T13:55:44.498027+0000","172.21.15.203:6832/3490266242":"2023-08-10T13:55:44.498027+0000","172.21.15.203:0/3892021074":"2023-08-10T13:55:44.498027+0000","172.21.15.203:0/750810033":"2023-08-10T13:55:44.498027+0000","172.21.15.203:0/1164133940":"2023-08-10T13:55:44.498027+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":7,"snaps":[{"begin":104,"length":1},{"begin":221,"length":1}]},{"pool":19,"snaps":[{"begin":23,"length":35},{"begin":79,"length":35}]}],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-08-09T14:51:32.118 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:32.116+0000 7f7f85c3f640 1 -- 172.21.15.43:0/3574343466 >> [v2:172.21.15.203:6832/2566816332,v1:172.21.15.203:6833/2566816332] conn(0x7f7f6c06a170 msgr2=0x7f7f6c06c630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T14:51:32.118 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:32.116+0000 7f7f85c3f640 1 --2- 172.21.15.43:0/3574343466 >> [v2:172.21.15.203:6832/2566816332,v1:172.21.15.203:6833/2566816332] conn(0x7f7f6c06a170 0x7f7f6c06c630 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0x7f7f70036000 tx=0x7f7f70030000 comp rx=0 tx=0).stop 2023-08-09T14:51:32.118 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:32.116+0000 7f7f85c3f640 1 -- 172.21.15.43:0/3574343466 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f7f8014a5c0 msgr2=0x7f7f8006bf90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T14:51:32.119 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:32.116+0000 7f7f85c3f640 1 --2- 172.21.15.43:0/3574343466 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f7f8014a5c0 0x7f7f8006bf90 secure :-1 s=READY pgs=1792 cs=0 l=1 rev1=1 crypto rx=0x7f7f78007cf0 tx=0x7f7f7800e4b0 comp rx=0 tx=0).stop 2023-08-09T14:51:32.119 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:32.117+0000 7f7f85c3f640 1 -- 172.21.15.43:0/3574343466 shutdown_connections 2023-08-09T14:51:32.119 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:32.117+0000 7f7f85c3f640 1 --2- 172.21.15.43:0/3574343466 >> [v2:172.21.15.203:6832/2566816332,v1:172.21.15.203:6833/2566816332] conn(0x7f7f6c06a170 0x7f7f6c06c630 unknown :-1 s=CLOSED pgs=30 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T14:51:32.119 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:32.117+0000 7f7f85c3f640 1 --2- 172.21.15.43:0/3574343466 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f7f8014a5c0 0x7f7f8006bf90 unknown :-1 s=CLOSED pgs=1792 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T14:51:32.119 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:32.117+0000 7f7f85c3f640 1 --2- 172.21.15.43:0/3574343466 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f7f800cc1b0 0x7f7f8006ba50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T14:51:32.119 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:32.117+0000 7f7f85c3f640 1 --2- 172.21.15.43:0/3574343466 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f7f800c3130 0x7f7f800c5860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T14:51:32.119 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:32.117+0000 7f7f85c3f640 1 -- 172.21.15.43:0/3574343466 >> 172.21.15.43:0/3574343466 conn(0x7f7f800ce0a0 msgr2=0x7f7f800c4a20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-09T14:51:32.120 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:32.117+0000 7f7f85c3f640 1 -- 172.21.15.43:0/3574343466 shutdown_connections 2023-08-09T14:51:32.120 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:32.117+0000 7f7f85c3f640 1 -- 172.21.15.43:0/3574343466 wait complete. 2023-08-09T14:51:32.139 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2023-08-09T14:51:32.375 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:32.374+0000 7f8b031c4640 1 Processor -- start 2023-08-09T14:51:32.376 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:32.374+0000 7f8b031c4640 1 -- start start 2023-08-09T14:51:32.377 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:32.375+0000 7f8b031c4640 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7f8afc0ce080 0x7f8afc0cafa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T14:51:32.377 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:32.375+0000 7f8b031c4640 1 -- --> v1:172.21.15.43:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8afc0cd8a0 con 0x7f8afc0ce720 2023-08-09T14:51:32.377 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:32.376+0000 7f8b031c4640 1 -- --> v1:172.21.15.203:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8afc0cf350 con 0x7f8afc0cb4e0 2023-08-09T14:51:32.377 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:32.376+0000 7f8b031c4640 1 -- --> v2:172.21.15.203:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8afc071a70 con 0x7f8afc0ce080 2023-08-09T14:51:32.378 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:32.376+0000 7f8b00f60640 1 --1- >> v1:172.21.15.43:6789/0 conn(0x7f8afc0ce720 0x7f8afc0ceb00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.43:6789/0 says I am v1:172.21.15.43:54940/0 (socket says 172.21.15.43:54940) 2023-08-09T14:51:32.379 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:32.376+0000 7f8b00f60640 1 -- 172.21.15.43:0/3949449976 learned_addr learned my addr 172.21.15.43:0/3949449976 (peer_addr_for_me v1:172.21.15.43:0/0) 2023-08-09T14:51:32.379 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:32.376+0000 7f8af3fff640 1 --2- 172.21.15.43:0/3949449976 >> v2:172.21.15.203:3300/0 conn(0x7f8afc0ce080 0x7f8afc0cafa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T14:51:32.379 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:32.376+0000 7f8af37fe640 1 -- 172.21.15.43:0/3949449976 <== mon.0 v1:172.21.15.43:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3834907112 0 0) 0x7f8afc0cd8a0 con 0x7f8afc0ce720 2023-08-09T14:51:32.379 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:32.376+0000 7f8af37fe640 1 -- 172.21.15.43:0/3949449976 --> v1:172.21.15.43:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f8ae4003520 con 0x7f8afc0ce720 2023-08-09T14:51:32.379 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:32.377+0000 7f8af37fe640 1 -- 172.21.15.43:0/3949449976 <== mon.0 v1:172.21.15.43:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2281462382 0 0) 0x7f8ae4003520 con 0x7f8afc0ce720 2023-08-09T14:51:32.379 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:32.377+0000 7f8af37fe640 1 -- 172.21.15.43:0/3949449976 >> v2:172.21.15.203:3300/0 conn(0x7f8afc0ce080 msgr2=0x7f8afc0cafa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-09T14:51:32.379 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:32.377+0000 7f8af37fe640 1 --2- 172.21.15.43:0/3949449976 >> v2:172.21.15.203:3300/0 conn(0x7f8afc0ce080 0x7f8afc0cafa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T14:51:32.380 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:32.377+0000 7f8af37fe640 1 -- 172.21.15.43:0/3949449976 >> v1:172.21.15.203:6789/0 conn(0x7f8afc0cb4e0 legacy=0x7f8afc06d2a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T14:51:32.380 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:32.377+0000 7f8af37fe640 1 -- 172.21.15.43:0/3949449976 --> v1:172.21.15.43:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8afc0af260 con 0x7f8afc0ce720 2023-08-09T14:51:32.380 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:32.377+0000 7f8af37fe640 1 -- 172.21.15.43:0/3949449976 <== mon.0 v1:172.21.15.43:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1285150740 0 0) 0x7f8af8005410 con 0x7f8afc0ce720 2023-08-09T14:51:32.380 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:32.377+0000 7f8af37fe640 1 -- 172.21.15.43:0/3949449976 >> v1:172.21.15.43:6789/0 conn(0x7f8afc0ce720 legacy=0x7f8afc0ceb00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T14:51:32.380 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:32.377+0000 7f8af37fe640 1 --2- 172.21.15.43:0/3949449976 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f8ae40039e0 0x7f8ae4003e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T14:51:32.380 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:32.377+0000 7f8af37fe640 1 --2- 172.21.15.43:0/3949449976 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f8ae40047c0 0x7f8ae4004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T14:51:32.381 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:32.377+0000 7f8af37fe640 1 --2- 172.21.15.43:0/3949449976 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f8ae40051f0 0x7f8ae400b980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T14:51:32.381 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:32.377+0000 7f8af37fe640 1 -- 172.21.15.43:0/3949449976 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8afc071a70 con 0x7f8ae40047c0 2023-08-09T14:51:32.381 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:32.377+0000 7f8af37fe640 1 -- 172.21.15.43:0/3949449976 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8ae400c150 con 0x7f8ae40039e0 2023-08-09T14:51:32.381 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:32.377+0000 7f8af37fe640 1 -- 172.21.15.43:0/3949449976 --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8ae400c310 con 0x7f8ae40051f0 2023-08-09T14:51:32.382 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:32.377+0000 7f8af3fff640 1 --2- 172.21.15.43:0/3949449976 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f8ae40047c0 0x7f8ae4004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T14:51:32.382 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:32.378+0000 7f8b00f60640 1 --2- 172.21.15.43:0/3949449976 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f8ae40039e0 0x7f8ae4003e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T14:51:32.383 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:32.378+0000 7f8af3fff640 1 -- 172.21.15.43:0/3949449976 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f8ae40051f0 msgr2=0x7f8ae400b980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-09T14:51:32.383 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:32.378+0000 7f8b01761640 1 --2- 172.21.15.43:0/3949449976 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f8ae40051f0 0x7f8ae400b980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T14:51:32.383 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:32.378+0000 7f8af3fff640 1 --2- 172.21.15.43:0/3949449976 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f8ae40051f0 0x7f8ae400b980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T14:51:32.383 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:32.378+0000 7f8af3fff640 1 -- 172.21.15.43:0/3949449976 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f8ae40039e0 msgr2=0x7f8ae4003e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-09T14:51:32.383 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:32.378+0000 7f8af3fff640 1 --2- 172.21.15.43:0/3949449976 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f8ae40039e0 0x7f8ae4003e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T14:51:32.383 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:32.378+0000 7f8af3fff640 1 -- 172.21.15.43:0/3949449976 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8ae4003520 con 0x7f8ae40047c0 2023-08-09T14:51:32.383 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:32.378+0000 7f8b01761640 1 -- 172.21.15.43:0/3949449976 reap_dead start 2023-08-09T14:51:32.384 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:32.379+0000 7f8af3fff640 1 --2- 172.21.15.43:0/3949449976 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f8ae40047c0 0x7f8ae4004c40 secure :-1 s=READY pgs=1794 cs=0 l=1 rev1=1 crypto rx=0x7f8aec009a90 tx=0x7f8ae40057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bae7703d7558b72f server_cookie=0 in_seq=0 out_seq=0 2023-08-09T14:51:32.384 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:32.379+0000 7f8af37fe640 1 -- 172.21.15.43:0/3949449976 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8aec02c680 con 0x7f8ae40047c0 2023-08-09T14:51:32.384 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:32.379+0000 7f8af37fe640 1 -- 172.21.15.43:0/3949449976 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8aec02c7e0 con 0x7f8ae40047c0 2023-08-09T14:51:32.384 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:32.379+0000 7f8af37fe640 1 -- 172.21.15.43:0/3949449976 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8aec02cb70 con 0x7f8ae40047c0 2023-08-09T14:51:32.384 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:32.380+0000 7f8b031c4640 1 -- 172.21.15.43:0/3949449976 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f8ae40047c0 msgr2=0x7f8ae4004c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T14:51:32.384 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:32.380+0000 7f8b031c4640 1 --2- 172.21.15.43:0/3949449976 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f8ae40047c0 0x7f8ae4004c40 secure :-1 s=READY pgs=1794 cs=0 l=1 rev1=1 crypto rx=0x7f8aec009a90 tx=0x7f8ae40057f0 comp rx=0 tx=0).stop 2023-08-09T14:51:32.385 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:32.380+0000 7f8b031c4640 1 -- 172.21.15.43:0/3949449976 shutdown_connections 2023-08-09T14:51:32.385 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:32.380+0000 7f8b031c4640 1 --2- 172.21.15.43:0/3949449976 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f8ae40047c0 0x7f8ae4004c40 unknown :-1 s=CLOSED pgs=1794 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T14:51:32.385 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:32.380+0000 7f8b031c4640 1 -- 172.21.15.43:0/3949449976 >> 172.21.15.43:0/3949449976 conn(0x7f8afc0ccd80 msgr2=0x7f8afc0cd190 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-09T14:51:32.385 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:32.380+0000 7f8b031c4640 1 -- 172.21.15.43:0/3949449976 shutdown_connections 2023-08-09T14:51:32.385 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:32.380+0000 7f8b031c4640 1 -- 172.21.15.43:0/3949449976 wait complete. 2023-08-09T14:51:32.388 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:32.382+0000 7f8b031c4640 1 Processor -- start 2023-08-09T14:51:32.388 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:32.382+0000 7f8b031c4640 1 -- start start 2023-08-09T14:51:32.389 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:32.383+0000 7f8b031c4640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f8ae40047c0 0x7f8afc0c3000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T14:51:32.389 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:32.383+0000 7f8b031c4640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f8afc0b2450 0x7f8afc151ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T14:51:32.389 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:32.383+0000 7f8b031c4640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f8afc0b4ee0 0x7f8afc0b2f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T14:51:32.389 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:32.383+0000 7f8b031c4640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8afc06a990 con 0x7f8afc0b4ee0 2023-08-09T14:51:32.390 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:32.383+0000 7f8b031c4640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8afc0b5470 con 0x7f8ae40047c0 2023-08-09T14:51:32.390 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:32.383+0000 7f8b031c4640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8afc0b5750 con 0x7f8afc0b2450 2023-08-09T14:51:32.390 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:32.384+0000 7f8b01761640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f8afc0b4ee0 0x7f8afc0b2f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T14:51:32.391 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:32.384+0000 7f8b01761640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f8afc0b4ee0 0x7f8afc0b2f20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:39960/0 (socket says 172.21.15.43:39960) 2023-08-09T14:51:32.391 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:32.384+0000 7f8b01761640 1 -- 172.21.15.43:0/637776240 learned_addr learned my addr 172.21.15.43:0/637776240 (peer_addr_for_me v2:172.21.15.43:0/0) 2023-08-09T14:51:32.391 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:32.384+0000 7f8af3fff640 1 --2- 172.21.15.43:0/637776240 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f8afc0b2450 0x7f8afc151ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T14:51:32.391 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:32.384+0000 7f8b00f60640 1 --2- 172.21.15.43:0/637776240 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f8ae40047c0 0x7f8afc0c3000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T14:51:32.392 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:32.384+0000 7f8b01761640 1 -- 172.21.15.43:0/637776240 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f8afc0b2450 msgr2=0x7f8afc151ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T14:51:32.392 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:32.384+0000 7f8b01761640 1 --2- 172.21.15.43:0/637776240 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f8afc0b2450 0x7f8afc151ad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T14:51:32.392 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:32.384+0000 7f8b01761640 1 -- 172.21.15.43:0/637776240 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f8ae40047c0 msgr2=0x7f8afc0c3000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T14:51:32.393 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:32.384+0000 7f8b01761640 1 --2- 172.21.15.43:0/637776240 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f8ae40047c0 0x7f8afc0c3000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T14:51:32.393 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:32.384+0000 7f8b01761640 1 -- 172.21.15.43:0/637776240 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8afc0b5a30 con 0x7f8afc0b4ee0 2023-08-09T14:51:32.393 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:32.384+0000 7f8af3fff640 1 --2- 172.21.15.43:0/637776240 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f8afc0b2450 0x7f8afc151ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-08-09T14:51:32.393 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:32.384+0000 7f8b00f60640 1 --2- 172.21.15.43:0/637776240 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f8ae40047c0 0x7f8afc0c3000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-08-09T14:51:32.393 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:32.384+0000 7f8b01761640 1 --2- 172.21.15.43:0/637776240 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f8afc0b4ee0 0x7f8afc0b2f20 secure :-1 s=READY pgs=1795 cs=0 l=1 rev1=1 crypto rx=0x7f8afc0afd30 tx=0x7f8af400d890 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-09T14:51:32.393 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:32.385+0000 7f8af17fa640 1 -- 172.21.15.43:0/637776240 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8af400b070 con 0x7f8afc0b4ee0 2023-08-09T14:51:32.393 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:32.385+0000 7f8af17fa640 1 -- 172.21.15.43:0/637776240 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8af40054e0 con 0x7f8afc0b4ee0 2023-08-09T14:51:32.393 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:32.385+0000 7f8af17fa640 1 -- 172.21.15.43:0/637776240 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8af4003810 con 0x7f8afc0b4ee0 2023-08-09T14:51:32.394 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:32.385+0000 7f8b031c4640 1 -- 172.21.15.43:0/637776240 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8afc0b5d10 con 0x7f8afc0b4ee0 2023-08-09T14:51:32.394 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:32.385+0000 7f8b031c4640 1 -- 172.21.15.43:0/637776240 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8afc0b3640 con 0x7f8afc0b4ee0 2023-08-09T14:51:32.394 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:32.386+0000 7f8b031c4640 1 -- 172.21.15.43:0/637776240 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8afc152010 con 0x7f8afc0b4ee0 2023-08-09T14:51:32.395 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:32.387+0000 7f8af17fa640 1 -- 172.21.15.43:0/637776240 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 79) v1 ==== 88443+0+0 (secure 0 0 0) 0x7f8af4003970 con 0x7f8afc0b4ee0 2023-08-09T14:51:32.395 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:32.388+0000 7f8af17fa640 1 --2- 172.21.15.43:0/637776240 >> [v2:172.21.15.203:6832/2566816332,v1:172.21.15.203:6833/2566816332] conn(0x7f8ae806a170 0x7f8ae806c630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T14:51:32.395 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:32.388+0000 7f8af17fa640 1 -- 172.21.15.43:0/637776240 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(160..160 src has 1..160) v4 ==== 7216+0+0 (secure 0 0 0) 0x7f8af40883a0 con 0x7f8afc0b4ee0 2023-08-09T14:51:32.395 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:32.389+0000 7f8b00f60640 1 --2- 172.21.15.43:0/637776240 >> [v2:172.21.15.203:6832/2566816332,v1:172.21.15.203:6833/2566816332] conn(0x7f8ae806a170 0x7f8ae806c630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T14:51:32.395 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:32.390+0000 7f8b00f60640 1 --2- 172.21.15.43:0/637776240 >> [v2:172.21.15.203:6832/2566816332,v1:172.21.15.203:6833/2566816332] conn(0x7f8ae806a170 0x7f8ae806c630 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0x7f8afc1f5730 tx=0x7f8af8009040 comp rx=0 tx=0).ready entity=mgr.7765 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-09T14:51:32.396 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:32.393+0000 7f8af17fa640 1 -- 172.21.15.43:0/637776240 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178066 (secure 0 0 0) 0x7f8af4053370 con 0x7f8afc0b4ee0 2023-08-09T14:51:32.630 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:32.628+0000 7f8b031c4640 1 -- 172.21.15.43:0/637776240 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) v1 -- 0x7f8afc0739f0 con 0x7f8afc0b4ee0 2023-08-09T14:51:32.631 INFO:tasks.ceph.mon.a.smithi043.stderr:2023-08-09T14:51:32.630+0000 7f32867f3640 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2023-08-09T14:51:32.638 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:32.637+0000 7f8af17fa640 1 -- 172.21.15.43:0/637776240 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v71) v1 ==== 161+0+0 (secure 0 0 0) 0x7f8af4056990 con 0x7f8afc0b4ee0 2023-08-09T14:51:32.639 INFO:teuthology.orchestra.run.smithi043.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2023-08-09T14:51:32.643 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:32.641+0000 7f8b031c4640 1 -- 172.21.15.43:0/637776240 >> [v2:172.21.15.203:6832/2566816332,v1:172.21.15.203:6833/2566816332] conn(0x7f8ae806a170 msgr2=0x7f8ae806c630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T14:51:32.643 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:32.641+0000 7f8b031c4640 1 --2- 172.21.15.43:0/637776240 >> [v2:172.21.15.203:6832/2566816332,v1:172.21.15.203:6833/2566816332] conn(0x7f8ae806a170 0x7f8ae806c630 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0x7f8afc1f5730 tx=0x7f8af8009040 comp rx=0 tx=0).stop 2023-08-09T14:51:32.644 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:32.641+0000 7f8b031c4640 1 -- 172.21.15.43:0/637776240 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f8afc0b4ee0 msgr2=0x7f8afc0b2f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T14:51:32.644 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:32.641+0000 7f8b031c4640 1 --2- 172.21.15.43:0/637776240 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f8afc0b4ee0 0x7f8afc0b2f20 secure :-1 s=READY pgs=1795 cs=0 l=1 rev1=1 crypto rx=0x7f8afc0afd30 tx=0x7f8af400d890 comp rx=0 tx=0).stop 2023-08-09T14:51:32.644 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:32.641+0000 7f8b031c4640 1 -- 172.21.15.43:0/637776240 shutdown_connections 2023-08-09T14:51:32.645 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:32.641+0000 7f8b031c4640 1 --2- 172.21.15.43:0/637776240 >> [v2:172.21.15.203:6832/2566816332,v1:172.21.15.203:6833/2566816332] conn(0x7f8ae806a170 0x7f8ae806c630 unknown :-1 s=CLOSED pgs=31 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T14:51:32.645 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:32.641+0000 7f8b031c4640 1 --2- 172.21.15.43:0/637776240 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f8afc0b4ee0 0x7f8afc0b2f20 unknown :-1 s=CLOSED pgs=1795 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T14:51:32.647 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:32.641+0000 7f8b031c4640 1 --2- 172.21.15.43:0/637776240 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f8afc0b2450 0x7f8afc151ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T14:51:32.647 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:32.641+0000 7f8b031c4640 1 --2- 172.21.15.43:0/637776240 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f8ae40047c0 0x7f8afc0c3000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T14:51:32.647 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:32.641+0000 7f8b031c4640 1 -- 172.21.15.43:0/637776240 >> 172.21.15.43:0/637776240 conn(0x7f8afc0ccd80 msgr2=0x7f8afc1538f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-09T14:51:32.648 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:32.641+0000 7f8b031c4640 1 -- 172.21.15.43:0/637776240 shutdown_connections 2023-08-09T14:51:32.648 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:32.641+0000 7f8b031c4640 1 -- 172.21.15.43:0/637776240 wait complete. 2023-08-09T14:51:32.666 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs rm cephfs --yes-i-really-mean-it 2023-08-09T14:51:32.739 INFO:tasks.ceph.mds.a.smithi043.stdout:starting mds.a at 2023-08-09T14:51:32.877 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:32.875+0000 7fb3714d9640 1 Processor -- start 2023-08-09T14:51:32.877 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:32.875+0000 7fb3714d9640 1 -- start start 2023-08-09T14:51:32.878 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:32.876+0000 7fb3714d9640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7fb36c0ce8d0 0x7fb36c0c9f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T14:51:32.878 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:32.876+0000 7fb3714d9640 1 -- --> v1:172.21.15.43:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb36c15aef0 con 0x7fb36c0cb2d0 2023-08-09T14:51:32.878 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:32.876+0000 7fb3714d9640 1 -- --> v1:172.21.15.203:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb36c15e940 con 0x7fb36c15df60 2023-08-09T14:51:32.878 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:32.876+0000 7fb3714d9640 1 -- --> v2:172.21.15.43:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb36c15a970 con 0x7fb36c0ce8d0 2023-08-09T14:51:32.879 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:32.877+0000 7fb36affd640 1 --1- >> v1:172.21.15.43:6789/0 conn(0x7fb36c0cb2d0 0x7fb36c15b380 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.43:6789/0 says I am v1:172.21.15.43:54946/0 (socket says 172.21.15.43:54946) 2023-08-09T14:51:32.879 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:32.877+0000 7fb36affd640 1 -- 172.21.15.43:0/2907548186 learned_addr learned my addr 172.21.15.43:0/2907548186 (peer_addr_for_me v1:172.21.15.43:0/0) 2023-08-09T14:51:32.879 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:32.877+0000 7fb36b7fe640 1 --2- 172.21.15.43:0/2907548186 >> v2:172.21.15.43:3300/0 conn(0x7fb36c0ce8d0 0x7fb36c0c9f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T14:51:32.883 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:32.877+0000 7fb36b7fe640 1 -- 172.21.15.43:0/2907548186 >> v1:172.21.15.203:6789/0 conn(0x7fb36c15df60 legacy=0x7fb36c0ce1c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-09T14:51:32.883 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:32.877+0000 7fb36b7fe640 1 -- 172.21.15.43:0/2907548186 >> v1:172.21.15.43:6789/0 conn(0x7fb36c0cb2d0 legacy=0x7fb36c15b380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T14:51:32.883 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:32.877+0000 7fb369ffb640 1 -- 172.21.15.43:0/2907548186 <== mon.0 v1:172.21.15.43:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2753960263 0 0) 0x7fb36c15aef0 con 0x7fb36c0cb2d0 2023-08-09T14:51:32.883 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:32.877+0000 7fb36b7fe640 1 -- 172.21.15.43:0/2907548186 --> v2:172.21.15.43:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb36c15ec20 con 0x7fb36c0ce8d0 2023-08-09T14:51:32.884 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:32.878+0000 7fb36b7fe640 1 --2- 172.21.15.43:0/2907548186 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb36c0ce8d0 0x7fb36c0c9f90 secure :-1 s=READY pgs=1798 cs=0 l=1 rev1=1 crypto rx=0x7fb364009d90 tx=0x7fb36402beb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ad8da48a2626ed1a server_cookie=0 in_seq=0 out_seq=0 2023-08-09T14:51:32.884 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:32.879+0000 7fb369ffb640 1 -- 172.21.15.43:0/2907548186 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb36402cc60 con 0x7fb36c0ce8d0 2023-08-09T14:51:32.884 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:32.879+0000 7fb369ffb640 1 -- 172.21.15.43:0/2907548186 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb36402cdc0 con 0x7fb36c0ce8d0 2023-08-09T14:51:32.885 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:32.879+0000 7fb369ffb640 1 -- 172.21.15.43:0/2907548186 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb364034660 con 0x7fb36c0ce8d0 2023-08-09T14:51:32.885 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:32.879+0000 7fb3714d9640 1 -- 172.21.15.43:0/2907548186 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb36c0ce8d0 msgr2=0x7fb36c0c9f90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T14:51:32.885 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:32.879+0000 7fb3714d9640 1 --2- 172.21.15.43:0/2907548186 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb36c0ce8d0 0x7fb36c0c9f90 secure :-1 s=READY pgs=1798 cs=0 l=1 rev1=1 crypto rx=0x7fb364009d90 tx=0x7fb36402beb0 comp rx=0 tx=0).stop 2023-08-09T14:51:32.886 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:32.879+0000 7fb3714d9640 1 -- 172.21.15.43:0/2907548186 shutdown_connections 2023-08-09T14:51:32.886 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:32.879+0000 7fb3714d9640 1 --2- 172.21.15.43:0/2907548186 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb36c0ce8d0 0x7fb36c0c9f90 unknown :-1 s=CLOSED pgs=1798 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T14:51:32.886 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:32.879+0000 7fb3714d9640 1 -- 172.21.15.43:0/2907548186 >> 172.21.15.43:0/2907548186 conn(0x7fb36c0aff50 msgr2=0x7fb36c0b0360 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-09T14:51:32.886 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:32.879+0000 7fb3714d9640 1 -- 172.21.15.43:0/2907548186 shutdown_connections 2023-08-09T14:51:32.887 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:32.879+0000 7fb3714d9640 1 -- 172.21.15.43:0/2907548186 wait complete. 2023-08-09T14:51:32.887 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:32.881+0000 7fb3714d9640 1 Processor -- start 2023-08-09T14:51:32.887 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:32.881+0000 7fb3714d9640 1 -- start start 2023-08-09T14:51:32.891 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:32.882+0000 7fb3714d9640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb36c0cb2d0 0x7fb36c0b1660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T14:51:32.891 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:32.882+0000 7fb3714d9640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fb36c0ce8d0 0x7fb36c14ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T14:51:32.891 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:32.882+0000 7fb3714d9640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fb36c15df60 0x7fb36c159c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T14:51:32.891 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:32.882+0000 7fb3714d9640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb36c0b0c50 con 0x7fb36c0cb2d0 2023-08-09T14:51:32.891 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:32.882+0000 7fb3714d9640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb36c0b11d0 con 0x7fb36c15df60 2023-08-09T14:51:32.891 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:32.882+0000 7fb3714d9640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb36c0cd6e0 con 0x7fb36c0ce8d0 2023-08-09T14:51:32.892 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:32.883+0000 7fb36affd640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb36c0cb2d0 0x7fb36c0b1660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T14:51:32.892 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:32.883+0000 7fb36affd640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb36c0cb2d0 0x7fb36c0b1660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:39980/0 (socket says 172.21.15.43:39980) 2023-08-09T14:51:32.892 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:32.883+0000 7fb36affd640 1 -- 172.21.15.43:0/3928554873 learned_addr learned my addr 172.21.15.43:0/3928554873 (peer_addr_for_me v2:172.21.15.43:0/0) 2023-08-09T14:51:32.892 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:32.883+0000 7fb36a7fc640 1 --2- 172.21.15.43:0/3928554873 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fb36c0ce8d0 0x7fb36c14ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T14:51:32.892 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:32.883+0000 7fb36affd640 1 -- 172.21.15.43:0/3928554873 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fb36c0ce8d0 msgr2=0x7fb36c14ede0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T14:51:32.892 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:32.883+0000 7fb36affd640 1 --2- 172.21.15.43:0/3928554873 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fb36c0ce8d0 0x7fb36c14ede0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T14:51:32.892 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:32.883+0000 7fb36b7fe640 1 --2- 172.21.15.43:0/3928554873 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fb36c15df60 0x7fb36c159c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T14:51:32.893 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:32.883+0000 7fb36affd640 1 -- 172.21.15.43:0/3928554873 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fb36c15df60 msgr2=0x7fb36c159c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T14:51:32.893 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:32.883+0000 7fb36affd640 1 --2- 172.21.15.43:0/3928554873 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fb36c15df60 0x7fb36c159c80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T14:51:32.893 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:32.883+0000 7fb36affd640 1 -- 172.21.15.43:0/3928554873 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb364009a70 con 0x7fb36c0cb2d0 2023-08-09T14:51:32.893 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:32.883+0000 7fb36a7fc640 1 --2- 172.21.15.43:0/3928554873 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fb36c0ce8d0 0x7fb36c14ede0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-08-09T14:51:32.893 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:32.884+0000 7fb36affd640 1 --2- 172.21.15.43:0/3928554873 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb36c0cb2d0 0x7fb36c0b1660 secure :-1 s=READY pgs=1799 cs=0 l=1 rev1=1 crypto rx=0x7fb35c00c500 tx=0x7fb35c00c9c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-09T14:51:32.893 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:32.884+0000 7fb3537fe640 1 -- 172.21.15.43:0/3928554873 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb35c007560 con 0x7fb36c0cb2d0 2023-08-09T14:51:32.893 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:32.884+0000 7fb3537fe640 1 -- 172.21.15.43:0/3928554873 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb35c0047e0 con 0x7fb36c0cb2d0 2023-08-09T14:51:32.894 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:32.884+0000 7fb3537fe640 1 -- 172.21.15.43:0/3928554873 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb35c004b90 con 0x7fb36c0cb2d0 2023-08-09T14:51:32.894 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:32.884+0000 7fb3714d9640 1 -- 172.21.15.43:0/3928554873 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb36c0cd9c0 con 0x7fb36c0cb2d0 2023-08-09T14:51:32.894 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:32.884+0000 7fb3714d9640 1 -- 172.21.15.43:0/3928554873 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb36c0cd3d0 con 0x7fb36c0cb2d0 2023-08-09T14:51:32.894 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:32.885+0000 7fb3714d9640 1 -- 172.21.15.43:0/3928554873 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb36c0739f0 con 0x7fb36c0cb2d0 2023-08-09T14:51:32.894 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:32.886+0000 7fb3537fe640 1 -- 172.21.15.43:0/3928554873 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 79) v1 ==== 88443+0+0 (secure 0 0 0) 0x7fb35c004cf0 con 0x7fb36c0cb2d0 2023-08-09T14:51:32.894 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:32.887+0000 7fb3537fe640 1 --2- 172.21.15.43:0/3928554873 >> [v2:172.21.15.203:6832/2566816332,v1:172.21.15.203:6833/2566816332] conn(0x7fb35806a170 0x7fb35806c630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T14:51:32.894 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:32.888+0000 7fb3537fe640 1 -- 172.21.15.43:0/3928554873 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(161..161 src has 1..161) v4 ==== 7302+0+0 (secure 0 0 0) 0x7fb35c087df0 con 0x7fb36c0cb2d0 2023-08-09T14:51:32.895 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:32.888+0000 7fb36a7fc640 1 --2- 172.21.15.43:0/3928554873 >> [v2:172.21.15.203:6832/2566816332,v1:172.21.15.203:6833/2566816332] conn(0x7fb35806a170 0x7fb35806c630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T14:51:32.895 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:32.889+0000 7fb36a7fc640 1 --2- 172.21.15.43:0/3928554873 >> [v2:172.21.15.203:6832/2566816332,v1:172.21.15.203:6833/2566816332] conn(0x7fb35806a170 0x7fb35806c630 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0x7fb36c0c7e20 tx=0x7fb360008040 comp rx=0 tx=0).ready entity=mgr.7765 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-09T14:51:32.895 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:32.891+0000 7fb3537fe640 1 -- 172.21.15.43:0/3928554873 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178066 (secure 0 0 0) 0x7fb35c052ee0 con 0x7fb36c0cb2d0 2023-08-09T14:51:33.126 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:33.125+0000 7fb3714d9640 1 -- 172.21.15.43:0/3928554873 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} v 0) v1 -- 0x7fb36c0cbe30 con 0x7fb36c0cb2d0 2023-08-09T14:51:33.642 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:33.641+0000 7fb3537fe640 1 -- 172.21.15.43:0/3928554873 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]=0 v72) v1 ==== 104+0+0 (secure 0 0 0) 0x7fb35c056500 con 0x7fb36c0cb2d0 2023-08-09T14:51:33.646 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:33.644+0000 7fb3714d9640 1 -- 172.21.15.43:0/3928554873 >> [v2:172.21.15.203:6832/2566816332,v1:172.21.15.203:6833/2566816332] conn(0x7fb35806a170 msgr2=0x7fb35806c630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T14:51:33.647 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:33.644+0000 7fb3714d9640 1 --2- 172.21.15.43:0/3928554873 >> [v2:172.21.15.203:6832/2566816332,v1:172.21.15.203:6833/2566816332] conn(0x7fb35806a170 0x7fb35806c630 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0x7fb36c0c7e20 tx=0x7fb360008040 comp rx=0 tx=0).stop 2023-08-09T14:51:33.647 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:33.644+0000 7fb3714d9640 1 -- 172.21.15.43:0/3928554873 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb36c0cb2d0 msgr2=0x7fb36c0b1660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T14:51:33.647 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:33.644+0000 7fb3714d9640 1 --2- 172.21.15.43:0/3928554873 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb36c0cb2d0 0x7fb36c0b1660 secure :-1 s=READY pgs=1799 cs=0 l=1 rev1=1 crypto rx=0x7fb35c00c500 tx=0x7fb35c00c9c0 comp rx=0 tx=0).stop 2023-08-09T14:51:33.647 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:33.645+0000 7fb3714d9640 1 -- 172.21.15.43:0/3928554873 shutdown_connections 2023-08-09T14:51:33.647 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:33.645+0000 7fb3714d9640 1 --2- 172.21.15.43:0/3928554873 >> [v2:172.21.15.203:6832/2566816332,v1:172.21.15.203:6833/2566816332] conn(0x7fb35806a170 0x7fb35806c630 unknown :-1 s=CLOSED pgs=32 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T14:51:33.647 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:33.645+0000 7fb3714d9640 1 --2- 172.21.15.43:0/3928554873 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fb36c15df60 0x7fb36c159c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T14:51:33.647 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:33.645+0000 7fb3714d9640 1 --2- 172.21.15.43:0/3928554873 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fb36c0ce8d0 0x7fb36c14ede0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T14:51:33.648 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:33.645+0000 7fb3714d9640 1 --2- 172.21.15.43:0/3928554873 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb36c0cb2d0 0x7fb36c0b1660 unknown :-1 s=CLOSED pgs=1799 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T14:51:33.648 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:33.645+0000 7fb3714d9640 1 -- 172.21.15.43:0/3928554873 >> 172.21.15.43:0/3928554873 conn(0x7fb36c0aff50 msgr2=0x7fb36c0ce190 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-09T14:51:33.648 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:33.645+0000 7fb3714d9640 1 -- 172.21.15.43:0/3928554873 shutdown_connections 2023-08-09T14:51:33.649 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:33.646+0000 7fb3714d9640 1 -- 172.21.15.43:0/3928554873 wait complete. 2023-08-09T14:51:33.669 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_metadata cephfs_metadata --yes-i-really-really-mean-it 2023-08-09T14:51:33.895 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:33.894+0000 7f2ab204d640 1 Processor -- start 2023-08-09T14:51:33.896 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:33.894+0000 7f2ab204d640 1 -- start start 2023-08-09T14:51:33.897 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:33.895+0000 7f2ab204d640 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7f2aac069a30 0x7f2aac159760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T14:51:33.897 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:33.895+0000 7f2ab204d640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f2aac15da40 0x7f2aac0ce4c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T14:51:33.897 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:33.895+0000 7f2ab204d640 1 -- --> v1:172.21.15.43:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2aac0ca410 con 0x7f2aac15d5b0 2023-08-09T14:51:33.897 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:33.895+0000 7f2ab204d640 1 -- --> v2:172.21.15.203:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2aac15ce80 con 0x7f2aac069a30 2023-08-09T14:51:33.897 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:33.896+0000 7f2ab204d640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2aac15a050 con 0x7f2aac15da40 2023-08-09T14:51:33.899 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:33.896+0000 7f2aa2ffd640 1 --1- >> v1:172.21.15.43:6789/0 conn(0x7f2aac15d5b0 0x7f2aac069e10 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.43:6789/0 says I am v1:172.21.15.43:54960/0 (socket says 172.21.15.43:54960) 2023-08-09T14:51:33.899 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:33.896+0000 7f2aa2ffd640 1 -- 172.21.15.43:0/2789073838 learned_addr learned my addr 172.21.15.43:0/2789073838 (peer_addr_for_me v1:172.21.15.43:0/0) 2023-08-09T14:51:33.899 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:33.896+0000 7f2aab7fe640 1 --2- 172.21.15.43:0/2789073838 >> v2:172.21.15.203:3300/0 conn(0x7f2aac069a30 0x7f2aac159760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T14:51:33.899 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:33.896+0000 7f2aabfff640 1 --2- 172.21.15.43:0/2789073838 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f2aac15da40 0x7f2aac0ce4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T14:51:33.900 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:33.896+0000 7f2aaaffd640 1 -- 172.21.15.43:0/2789073838 <== mon.0 v1:172.21.15.43:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3010084928 0 0) 0x7f2aac0ca410 con 0x7f2aac15d5b0 2023-08-09T14:51:33.900 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:33.896+0000 7f2aaaffd640 1 -- 172.21.15.43:0/2789073838 --> v1:172.21.15.43:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f2a98003520 con 0x7f2aac15d5b0 2023-08-09T14:51:33.900 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:33.897+0000 7f2aaaffd640 1 -- 172.21.15.43:0/2789073838 <== mon.0 v1:172.21.15.43:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 4228663426 0 0) 0x7f2a98003520 con 0x7f2aac15d5b0 2023-08-09T14:51:33.900 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:33.897+0000 7f2aaaffd640 1 -- 172.21.15.43:0/2789073838 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f2aac15da40 msgr2=0x7f2aac0ce4c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-09T14:51:33.900 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:33.897+0000 7f2aaaffd640 1 --2- 172.21.15.43:0/2789073838 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f2aac15da40 0x7f2aac0ce4c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T14:51:33.901 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:33.897+0000 7f2aaaffd640 1 -- 172.21.15.43:0/2789073838 >> v2:172.21.15.203:3300/0 conn(0x7f2aac069a30 msgr2=0x7f2aac159760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-09T14:51:33.901 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:33.897+0000 7f2aaaffd640 1 --2- 172.21.15.43:0/2789073838 >> v2:172.21.15.203:3300/0 conn(0x7f2aac069a30 0x7f2aac159760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T14:51:33.901 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:33.897+0000 7f2aaaffd640 1 -- 172.21.15.43:0/2789073838 --> v1:172.21.15.43:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2aac15a330 con 0x7f2aac15d5b0 2023-08-09T14:51:33.901 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:33.897+0000 7f2aaaffd640 1 -- 172.21.15.43:0/2789073838 <== mon.0 v1:172.21.15.43:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1285150740 0 0) 0x7f2a94001670 con 0x7f2aac15d5b0 2023-08-09T14:51:33.901 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:33.897+0000 7f2aabfff640 1 --2- 172.21.15.43:0/2789073838 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f2aac15da40 0x7f2aac0ce4c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-08-09T14:51:33.901 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:33.897+0000 7f2aaaffd640 1 -- 172.21.15.43:0/2789073838 >> v1:172.21.15.43:6789/0 conn(0x7f2aac15d5b0 legacy=0x7f2aac069e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T14:51:33.901 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:33.897+0000 7f2aaaffd640 1 --2- 172.21.15.43:0/2789073838 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f2a980039e0 0x7f2a98003e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T14:51:33.901 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:33.897+0000 7f2aaaffd640 1 --2- 172.21.15.43:0/2789073838 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f2aac15da40 0x7f2a980047c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T14:51:33.902 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:33.897+0000 7f2aaaffd640 1 --2- 172.21.15.43:0/2789073838 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f2a98004d00 0x7f2a98005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T14:51:33.902 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:33.897+0000 7f2aaaffd640 1 -- 172.21.15.43:0/2789073838 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2aac15ce80 con 0x7f2a980039e0 2023-08-09T14:51:33.902 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:33.897+0000 7f2aaaffd640 1 -- 172.21.15.43:0/2789073838 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2aac15a050 con 0x7f2a98004d00 2023-08-09T14:51:33.902 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:33.897+0000 7f2aaaffd640 1 -- 172.21.15.43:0/2789073838 --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2a9800dc60 con 0x7f2aac15da40 2023-08-09T14:51:33.902 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:33.897+0000 7f2aab7fe640 1 --2- 172.21.15.43:0/2789073838 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f2a980039e0 0x7f2a98003e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T14:51:33.902 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:33.898+0000 7f2aa2ffd640 1 --2- 172.21.15.43:0/2789073838 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f2aac15da40 0x7f2a980047c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T14:51:33.902 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:33.898+0000 7f2aab7fe640 1 -- 172.21.15.43:0/2789073838 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f2aac15da40 msgr2=0x7f2a980047c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-09T14:51:33.903 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:33.898+0000 7f2aab7fe640 1 --2- 172.21.15.43:0/2789073838 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f2aac15da40 0x7f2a980047c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T14:51:33.903 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:33.898+0000 7f2aab7fe640 1 -- 172.21.15.43:0/2789073838 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f2a98004d00 msgr2=0x7f2a98005180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-09T14:51:33.903 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:33.898+0000 7f2aab7fe640 1 --2- 172.21.15.43:0/2789073838 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f2a98004d00 0x7f2a98005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T14:51:33.903 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:33.898+0000 7f2aab7fe640 1 -- 172.21.15.43:0/2789073838 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2a98003520 con 0x7f2a980039e0 2023-08-09T14:51:33.904 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:33.899+0000 7f2aab7fe640 1 --2- 172.21.15.43:0/2789073838 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f2a980039e0 0x7f2a98003e00 secure :-1 s=READY pgs=1801 cs=0 l=1 rev1=1 crypto rx=0x7f2a9c009ac0 tx=0x7f2a980057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=588e855f8773cd8a server_cookie=0 in_seq=0 out_seq=0 2023-08-09T14:51:33.905 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:33.899+0000 7f2aaaffd640 1 -- 172.21.15.43:0/2789073838 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2a9c02c630 con 0x7f2a980039e0 2023-08-09T14:51:33.905 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:33.899+0000 7f2aaaffd640 1 -- 172.21.15.43:0/2789073838 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2a9c02c790 con 0x7f2a980039e0 2023-08-09T14:51:33.905 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:33.899+0000 7f2aaaffd640 1 -- 172.21.15.43:0/2789073838 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2a9c02cb20 con 0x7f2a980039e0 2023-08-09T14:51:33.905 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:33.900+0000 7f2ab204d640 1 -- 172.21.15.43:0/2789073838 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f2a980039e0 msgr2=0x7f2a98003e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T14:51:33.905 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:33.900+0000 7f2ab204d640 1 --2- 172.21.15.43:0/2789073838 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f2a980039e0 0x7f2a98003e00 secure :-1 s=READY pgs=1801 cs=0 l=1 rev1=1 crypto rx=0x7f2a9c009ac0 tx=0x7f2a980057f0 comp rx=0 tx=0).stop 2023-08-09T14:51:33.905 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:33.900+0000 7f2aabfff640 1 -- 172.21.15.43:0/2789073838 reap_dead start 2023-08-09T14:51:33.906 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:33.900+0000 7f2ab204d640 1 -- 172.21.15.43:0/2789073838 shutdown_connections 2023-08-09T14:51:33.906 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:33.900+0000 7f2ab204d640 1 -- 172.21.15.43:0/2789073838 >> 172.21.15.43:0/2789073838 conn(0x7f2aac06b990 msgr2=0x7f2aac06bda0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-09T14:51:33.906 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:33.900+0000 7f2ab204d640 1 -- 172.21.15.43:0/2789073838 shutdown_connections 2023-08-09T14:51:33.906 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:33.900+0000 7f2ab204d640 1 -- 172.21.15.43:0/2789073838 wait complete. 2023-08-09T14:51:33.906 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:33.902+0000 7f2ab204d640 1 Processor -- start 2023-08-09T14:51:33.906 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:33.902+0000 7f2ab204d640 1 -- start start 2023-08-09T14:51:33.906 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:33.903+0000 7f2ab204d640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f2a980039e0 0x7f2aac0c5960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T14:51:33.906 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:33.903+0000 7f2ab204d640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f2aac0c8f20 0x7f2aac15baf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T14:51:33.907 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:33.903+0000 7f2ab204d640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f2aac14f610 0x7f2aac0c3be0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T14:51:33.907 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:33.903+0000 7f2ab204d640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2aac15c030 con 0x7f2a980039e0 2023-08-09T14:51:33.907 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:33.903+0000 7f2ab204d640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2aac0c8a90 con 0x7f2aac0c8f20 2023-08-09T14:51:33.907 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:33.903+0000 7f2ab204d640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2aac14fac0 con 0x7f2aac14f610 2023-08-09T14:51:33.907 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:33.903+0000 7f2aab7fe640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f2a980039e0 0x7f2aac0c5960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T14:51:33.907 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:33.903+0000 7f2aab7fe640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f2a980039e0 0x7f2aac0c5960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:40000/0 (socket says 172.21.15.43:40000) 2023-08-09T14:51:33.907 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:33.903+0000 7f2aab7fe640 1 -- 172.21.15.43:0/1168835667 learned_addr learned my addr 172.21.15.43:0/1168835667 (peer_addr_for_me v2:172.21.15.43:0/0) 2023-08-09T14:51:33.908 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:33.904+0000 7f2aab7fe640 1 -- 172.21.15.43:0/1168835667 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f2aac14f610 msgr2=0x7f2aac0c3be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T14:51:33.909 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:33.904+0000 7f2aabfff640 1 --2- 172.21.15.43:0/1168835667 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f2aac14f610 0x7f2aac0c3be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T14:51:33.909 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:33.904+0000 7f2aab7fe640 1 --2- 172.21.15.43:0/1168835667 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f2aac14f610 0x7f2aac0c3be0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T14:51:33.909 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:33.904+0000 7f2aab7fe640 1 -- 172.21.15.43:0/1168835667 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f2aac0c8f20 msgr2=0x7f2aac15baf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T14:51:33.909 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:33.904+0000 7f2aab7fe640 1 --2- 172.21.15.43:0/1168835667 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f2aac0c8f20 0x7f2aac15baf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T14:51:33.909 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:33.904+0000 7f2aab7fe640 1 -- 172.21.15.43:0/1168835667 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2aac0c5ff0 con 0x7f2a980039e0 2023-08-09T14:51:33.909 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:33.904+0000 7f2aab7fe640 1 --2- 172.21.15.43:0/1168835667 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f2a980039e0 0x7f2aac0c5960 secure :-1 s=READY pgs=1802 cs=0 l=1 rev1=1 crypto rx=0x7f2a9c02c1d0 tx=0x7f2a9c004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-09T14:51:33.909 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:33.904+0000 7f2aa8ff9640 1 -- 172.21.15.43:0/1168835667 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2a9c003810 con 0x7f2a980039e0 2023-08-09T14:51:33.910 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:33.904+0000 7f2aa8ff9640 1 -- 172.21.15.43:0/1168835667 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2a9c003970 con 0x7f2a980039e0 2023-08-09T14:51:33.910 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:33.904+0000 7f2aa8ff9640 1 -- 172.21.15.43:0/1168835667 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2a9c003d20 con 0x7f2a980039e0 2023-08-09T14:51:33.910 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:33.904+0000 7f2ab204d640 1 -- 172.21.15.43:0/1168835667 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2aac0c8160 con 0x7f2a980039e0 2023-08-09T14:51:33.910 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:33.905+0000 7f2ab204d640 1 -- 172.21.15.43:0/1168835667 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2aac0c8720 con 0x7f2a980039e0 2023-08-09T14:51:33.910 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:33.906+0000 7f2ab204d640 1 -- 172.21.15.43:0/1168835667 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2aac0739f0 con 0x7f2a980039e0 2023-08-09T14:51:33.910 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:33.907+0000 7f2aa8ff9640 1 -- 172.21.15.43:0/1168835667 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 79) v1 ==== 88443+0+0 (secure 0 0 0) 0x7f2a9c040450 con 0x7f2a980039e0 2023-08-09T14:51:33.911 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:33.908+0000 7f2aa8ff9640 1 --2- 172.21.15.43:0/1168835667 >> [v2:172.21.15.203:6832/2566816332,v1:172.21.15.203:6833/2566816332] conn(0x7f2a8c06a1a0 0x7f2a8c06c660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T14:51:33.911 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:33.908+0000 7f2aa8ff9640 1 -- 172.21.15.43:0/1168835667 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(162..162 src has 1..162) v4 ==== 7302+0+0 (secure 0 0 0) 0x7f2a9c0aac60 con 0x7f2a980039e0 2023-08-09T14:51:33.911 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:33.909+0000 7f2aa2ffd640 1 --2- 172.21.15.43:0/1168835667 >> [v2:172.21.15.203:6832/2566816332,v1:172.21.15.203:6833/2566816332] conn(0x7f2a8c06a1a0 0x7f2a8c06c660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T14:51:33.912 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:33.910+0000 7f2aa2ffd640 1 --2- 172.21.15.43:0/1168835667 >> [v2:172.21.15.203:6832/2566816332,v1:172.21.15.203:6833/2566816332] conn(0x7f2a8c06a1a0 0x7f2a8c06c660 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0x7f2aac1f57a0 tx=0x7f2a94001d70 comp rx=0 tx=0).ready entity=mgr.7765 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-09T14:51:33.913 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:33.912+0000 7f2aa8ff9640 1 -- 172.21.15.43:0/1168835667 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178066 (secure 0 0 0) 0x7f2a9c075c30 con 0x7f2a980039e0 2023-08-09T14:51:34.111 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:34.109+0000 7f2ab204d640 1 -- 172.21.15.43:0/1168835667 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7f2aac0cbd00 con 0x7f2a980039e0 2023-08-09T14:51:34.657 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:34.655+0000 7f2aa8ff9640 1 -- 172.21.15.43:0/1168835667 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_metadata' removed v163) v1 ==== 181+0+0 (secure 0 0 0) 0x7f2a9c0440d0 con 0x7f2a980039e0 2023-08-09T14:51:34.658 INFO:teuthology.orchestra.run.smithi043.stderr:pool 'cephfs_metadata' removed 2023-08-09T14:51:34.664 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:34.661+0000 7f2ab204d640 1 -- 172.21.15.43:0/1168835667 >> [v2:172.21.15.203:6832/2566816332,v1:172.21.15.203:6833/2566816332] conn(0x7f2a8c06a1a0 msgr2=0x7f2a8c06c660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T14:51:34.664 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:34.661+0000 7f2ab204d640 1 --2- 172.21.15.43:0/1168835667 >> [v2:172.21.15.203:6832/2566816332,v1:172.21.15.203:6833/2566816332] conn(0x7f2a8c06a1a0 0x7f2a8c06c660 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0x7f2aac1f57a0 tx=0x7f2a94001d70 comp rx=0 tx=0).stop 2023-08-09T14:51:34.664 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:34.662+0000 7f2ab204d640 1 -- 172.21.15.43:0/1168835667 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f2a980039e0 msgr2=0x7f2aac0c5960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T14:51:34.664 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:34.662+0000 7f2ab204d640 1 --2- 172.21.15.43:0/1168835667 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f2a980039e0 0x7f2aac0c5960 secure :-1 s=READY pgs=1802 cs=0 l=1 rev1=1 crypto rx=0x7f2a9c02c1d0 tx=0x7f2a9c004730 comp rx=0 tx=0).stop 2023-08-09T14:51:34.665 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:34.662+0000 7f2ab204d640 1 -- 172.21.15.43:0/1168835667 shutdown_connections 2023-08-09T14:51:34.665 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:34.663+0000 7f2ab204d640 1 --2- 172.21.15.43:0/1168835667 >> [v2:172.21.15.203:6832/2566816332,v1:172.21.15.203:6833/2566816332] conn(0x7f2a8c06a1a0 0x7f2a8c06c660 unknown :-1 s=CLOSED pgs=34 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T14:51:34.665 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:34.663+0000 7f2ab204d640 1 --2- 172.21.15.43:0/1168835667 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f2aac14f610 0x7f2aac0c3be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T14:51:34.665 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:34.663+0000 7f2ab204d640 1 --2- 172.21.15.43:0/1168835667 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f2aac0c8f20 0x7f2aac15baf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T14:51:34.665 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:34.663+0000 7f2ab204d640 1 --2- 172.21.15.43:0/1168835667 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f2a980039e0 0x7f2aac0c5960 unknown :-1 s=CLOSED pgs=1802 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T14:51:34.665 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:34.663+0000 7f2ab204d640 1 -- 172.21.15.43:0/1168835667 >> 172.21.15.43:0/1168835667 conn(0x7f2aac06b990 msgr2=0x7f2aac15c200 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-09T14:51:34.666 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:34.664+0000 7f2ab204d640 1 -- 172.21.15.43:0/1168835667 shutdown_connections 2023-08-09T14:51:34.666 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:34.664+0000 7f2ab204d640 1 -- 172.21.15.43:0/1168835667 wait complete. 2023-08-09T14:51:34.691 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_data cephfs_data --yes-i-really-really-mean-it 2023-08-09T14:51:34.915 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:34.914+0000 7fc87aad7640 1 Processor -- start 2023-08-09T14:51:34.916 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:34.914+0000 7fc87aad7640 1 -- start start 2023-08-09T14:51:34.917 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:34.915+0000 7fc87aad7640 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7fc87415a330 0x7fc8740c9780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T14:51:34.917 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:34.916+0000 7fc87aad7640 1 -- --> v1:172.21.15.43:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc87415dd30 con 0x7fc8740aff80 2023-08-09T14:51:34.917 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:34.916+0000 7fc87aad7640 1 -- --> v1:172.21.15.203:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc874071490 con 0x7fc87415ea60 2023-08-09T14:51:34.917 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:34.916+0000 7fc87aad7640 1 -- --> v2:172.21.15.203:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc8740c9cc0 con 0x7fc87415a330 2023-08-09T14:51:34.919 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:34.916+0000 7fc87a2d6640 1 --1- >> v1:172.21.15.43:6789/0 conn(0x7fc8740aff80 0x7fc87406b400 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.43:6789/0 says I am v1:172.21.15.43:54970/0 (socket says 172.21.15.43:54970) 2023-08-09T14:51:34.919 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:34.916+0000 7fc87a2d6640 1 -- 172.21.15.43:0/1090836383 learned_addr learned my addr 172.21.15.43:0/1090836383 (peer_addr_for_me v1:172.21.15.43:0/0) 2023-08-09T14:51:34.919 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:34.916+0000 7fc879ad5640 1 --2- 172.21.15.43:0/1090836383 >> v2:172.21.15.203:3300/0 conn(0x7fc87415a330 0x7fc8740c9780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T14:51:34.919 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:34.917+0000 7fc878ad3640 1 -- 172.21.15.43:0/1090836383 <== mon.0 v1:172.21.15.43:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 252829757 0 0) 0x7fc87415dd30 con 0x7fc8740aff80 2023-08-09T14:51:34.920 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:34.917+0000 7fc878ad3640 1 -- 172.21.15.43:0/1090836383 --> v1:172.21.15.43:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fc860003520 con 0x7fc8740aff80 2023-08-09T14:51:34.920 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:34.917+0000 7fc878ad3640 1 -- 172.21.15.43:0/1090836383 <== mon.0 v1:172.21.15.43:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 84572816 0 0) 0x7fc860003520 con 0x7fc8740aff80 2023-08-09T14:51:34.920 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:34.917+0000 7fc878ad3640 1 -- 172.21.15.43:0/1090836383 >> v2:172.21.15.203:3300/0 conn(0x7fc87415a330 msgr2=0x7fc8740c9780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-09T14:51:34.920 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:34.917+0000 7fc878ad3640 1 --2- 172.21.15.43:0/1090836383 >> v2:172.21.15.203:3300/0 conn(0x7fc87415a330 0x7fc8740c9780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T14:51:34.920 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:34.917+0000 7fc878ad3640 1 -- 172.21.15.43:0/1090836383 >> v1:172.21.15.203:6789/0 conn(0x7fc87415ea60 legacy=0x7fc8740af870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T14:51:34.920 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:34.917+0000 7fc878ad3640 1 -- 172.21.15.43:0/1090836383 --> v1:172.21.15.43:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc87415df80 con 0x7fc8740aff80 2023-08-09T14:51:34.921 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:34.917+0000 7fc878ad3640 1 -- 172.21.15.43:0/1090836383 <== mon.0 v1:172.21.15.43:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1285150740 0 0) 0x7fc86c0019f0 con 0x7fc8740aff80 2023-08-09T14:51:34.921 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:34.917+0000 7fc878ad3640 1 -- 172.21.15.43:0/1090836383 >> v1:172.21.15.43:6789/0 conn(0x7fc8740aff80 legacy=0x7fc87406b400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T14:51:34.921 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:34.917+0000 7fc878ad3640 1 --2- 172.21.15.43:0/1090836383 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc860003970 0x7fc860003d50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T14:51:34.921 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:34.917+0000 7fc878ad3640 1 --2- 172.21.15.43:0/1090836383 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fc860004370 0x7fc860004840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T14:51:34.921 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:34.917+0000 7fc878ad3640 1 --2- 172.21.15.43:0/1090836383 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fc860005040 0x7fc86000b980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T14:51:34.921 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:34.917+0000 7fc878ad3640 1 -- 172.21.15.43:0/1090836383 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc8740c9cc0 con 0x7fc860003970 2023-08-09T14:51:34.921 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:34.917+0000 7fc878ad3640 1 -- 172.21.15.43:0/1090836383 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc86000bfa0 con 0x7fc860004370 2023-08-09T14:51:34.922 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:34.917+0000 7fc878ad3640 1 -- 172.21.15.43:0/1090836383 --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc86000c160 con 0x7fc860005040 2023-08-09T14:51:34.922 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:34.918+0000 7fc879ad5640 1 --2- 172.21.15.43:0/1090836383 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc860003970 0x7fc860003d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T14:51:34.923 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:34.918+0000 7fc879ad5640 1 -- 172.21.15.43:0/1090836383 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fc860005040 msgr2=0x7fc86000b980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-09T14:51:34.924 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:34.918+0000 7fc87a2d6640 1 --2- 172.21.15.43:0/1090836383 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fc860005040 0x7fc86000b980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T14:51:34.924 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:34.918+0000 7fc8792d4640 1 --2- 172.21.15.43:0/1090836383 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fc860004370 0x7fc860004840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T14:51:34.924 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:34.918+0000 7fc879ad5640 1 --2- 172.21.15.43:0/1090836383 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fc860005040 0x7fc86000b980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T14:51:34.924 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:34.918+0000 7fc879ad5640 1 -- 172.21.15.43:0/1090836383 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fc860004370 msgr2=0x7fc860004840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-09T14:51:34.924 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:34.918+0000 7fc879ad5640 1 --2- 172.21.15.43:0/1090836383 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fc860004370 0x7fc860004840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T14:51:34.924 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:34.918+0000 7fc879ad5640 1 -- 172.21.15.43:0/1090836383 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc860003520 con 0x7fc860003970 2023-08-09T14:51:34.925 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:34.918+0000 7fc87a2d6640 1 -- 172.21.15.43:0/1090836383 reap_dead start 2023-08-09T14:51:34.925 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:34.918+0000 7fc8792d4640 1 --2- 172.21.15.43:0/1090836383 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fc860004370 0x7fc860004840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-08-09T14:51:34.925 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:34.919+0000 7fc879ad5640 1 --2- 172.21.15.43:0/1090836383 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc860003970 0x7fc860003d50 secure :-1 s=READY pgs=1804 cs=0 l=1 rev1=1 crypto rx=0x7fc870009ac0 tx=0x7fc8600057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d55fa1451d2e584d server_cookie=0 in_seq=0 out_seq=0 2023-08-09T14:51:34.925 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:34.919+0000 7fc878ad3640 1 -- 172.21.15.43:0/1090836383 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc87002c6b0 con 0x7fc860003970 2023-08-09T14:51:34.925 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:34.919+0000 7fc878ad3640 1 -- 172.21.15.43:0/1090836383 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc87002c810 con 0x7fc860003970 2023-08-09T14:51:34.925 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:34.920+0000 7fc878ad3640 1 -- 172.21.15.43:0/1090836383 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc87002cba0 con 0x7fc860003970 2023-08-09T14:51:34.926 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:34.920+0000 7fc87aad7640 1 -- 172.21.15.43:0/1090836383 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc860003970 msgr2=0x7fc860003d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T14:51:34.926 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:34.920+0000 7fc87aad7640 1 --2- 172.21.15.43:0/1090836383 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc860003970 0x7fc860003d50 secure :-1 s=READY pgs=1804 cs=0 l=1 rev1=1 crypto rx=0x7fc870009ac0 tx=0x7fc8600057f0 comp rx=0 tx=0).stop 2023-08-09T14:51:34.926 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:34.920+0000 7fc87aad7640 1 -- 172.21.15.43:0/1090836383 shutdown_connections 2023-08-09T14:51:34.926 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:34.920+0000 7fc87aad7640 1 --2- 172.21.15.43:0/1090836383 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc860003970 0x7fc860003d50 unknown :-1 s=CLOSED pgs=1804 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T14:51:34.926 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:34.920+0000 7fc87aad7640 1 -- 172.21.15.43:0/1090836383 >> 172.21.15.43:0/1090836383 conn(0x7fc8740ca870 msgr2=0x7fc8740cac50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-09T14:51:34.926 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:34.920+0000 7fc87aad7640 1 -- 172.21.15.43:0/1090836383 shutdown_connections 2023-08-09T14:51:34.926 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:34.920+0000 7fc87aad7640 1 -- 172.21.15.43:0/1090836383 wait complete. 2023-08-09T14:51:34.926 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:34.922+0000 7fc87aad7640 1 Processor -- start 2023-08-09T14:51:34.927 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:34.922+0000 7fc87aad7640 1 -- start start 2023-08-09T14:51:34.928 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:34.923+0000 7fc87aad7640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fc860003970 0x7fc8740aca50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T14:51:34.928 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:34.923+0000 7fc87aad7640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc8740acf90 0x7fc8740cbb10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T14:51:34.929 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:34.923+0000 7fc87aad7640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fc8740cc050 0x7fc8740c6a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T14:51:34.929 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:34.923+0000 7fc87aad7640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc8740cc860 con 0x7fc8740acf90 2023-08-09T14:51:34.929 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:34.923+0000 7fc87aad7640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc8740ad3d0 con 0x7fc860003970 2023-08-09T14:51:34.929 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:34.923+0000 7fc87aad7640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc8741f6000 con 0x7fc8740cc050 2023-08-09T14:51:34.929 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:34.924+0000 7fc8792d4640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc8740acf90 0x7fc8740cbb10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T14:51:34.929 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:34.924+0000 7fc8792d4640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc8740acf90 0x7fc8740cbb10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:40024/0 (socket says 172.21.15.43:40024) 2023-08-09T14:51:34.930 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:34.924+0000 7fc8792d4640 1 -- 172.21.15.43:0/273952401 learned_addr learned my addr 172.21.15.43:0/273952401 (peer_addr_for_me v2:172.21.15.43:0/0) 2023-08-09T14:51:34.930 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:34.924+0000 7fc879ad5640 1 --2- 172.21.15.43:0/273952401 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fc860003970 0x7fc8740aca50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T14:51:34.930 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:34.924+0000 7fc8792d4640 1 -- 172.21.15.43:0/273952401 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fc8740cc050 msgr2=0x7fc8740c6a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T14:51:34.930 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:34.924+0000 7fc8792d4640 1 --2- 172.21.15.43:0/273952401 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fc8740cc050 0x7fc8740c6a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T14:51:34.930 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:34.924+0000 7fc8792d4640 1 -- 172.21.15.43:0/273952401 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fc860003970 msgr2=0x7fc8740aca50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T14:51:34.930 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:34.924+0000 7fc8792d4640 1 --2- 172.21.15.43:0/273952401 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fc860003970 0x7fc8740aca50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T14:51:34.931 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:34.924+0000 7fc8792d4640 1 -- 172.21.15.43:0/273952401 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc8741f5240 con 0x7fc8740acf90 2023-08-09T14:51:34.931 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:34.924+0000 7fc8792d4640 1 --2- 172.21.15.43:0/273952401 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc8740acf90 0x7fc8740cbb10 secure :-1 s=READY pgs=1805 cs=0 l=1 rev1=1 crypto rx=0x7fc86400d5c0 tx=0x7fc86400da80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-09T14:51:34.931 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:34.925+0000 7fc86a7fc640 1 -- 172.21.15.43:0/273952401 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc864017070 con 0x7fc8740acf90 2023-08-09T14:51:34.931 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:34.925+0000 7fc86a7fc640 1 -- 172.21.15.43:0/273952401 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc864005280 con 0x7fc8740acf90 2023-08-09T14:51:34.931 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:34.925+0000 7fc86a7fc640 1 -- 172.21.15.43:0/273952401 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc864007030 con 0x7fc8740acf90 2023-08-09T14:51:34.931 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:34.925+0000 7fc87aad7640 1 -- 172.21.15.43:0/273952401 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc8741f5520 con 0x7fc8740acf90 2023-08-09T14:51:34.931 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:34.925+0000 7fc87aad7640 1 -- 172.21.15.43:0/273952401 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc8741f5ae0 con 0x7fc8740acf90 2023-08-09T14:51:34.932 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:34.926+0000 7fc87aad7640 1 -- 172.21.15.43:0/273952401 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc8740739f0 con 0x7fc8740acf90 2023-08-09T14:51:34.932 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:34.927+0000 7fc86a7fc640 1 -- 172.21.15.43:0/273952401 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 79) v1 ==== 88443+0+0 (secure 0 0 0) 0x7fc864007190 con 0x7fc8740acf90 2023-08-09T14:51:34.933 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:34.928+0000 7fc86a7fc640 1 --2- 172.21.15.43:0/273952401 >> [v2:172.21.15.203:6832/2566816332,v1:172.21.15.203:6833/2566816332] conn(0x7fc85806a1a0 0x7fc85806c660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T14:51:34.933 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:34.928+0000 7fc86a7fc640 1 -- 172.21.15.43:0/273952401 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(163..163 src has 1..163) v4 ==== 6820+0+0 (secure 0 0 0) 0x7fc864088230 con 0x7fc8740acf90 2023-08-09T14:51:34.933 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:34.928+0000 7fc879ad5640 1 --2- 172.21.15.43:0/273952401 >> [v2:172.21.15.203:6832/2566816332,v1:172.21.15.203:6833/2566816332] conn(0x7fc85806a1a0 0x7fc85806c660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T14:51:34.933 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:34.930+0000 7fc879ad5640 1 --2- 172.21.15.43:0/273952401 >> [v2:172.21.15.203:6832/2566816332,v1:172.21.15.203:6833/2566816332] conn(0x7fc85806a1a0 0x7fc85806c660 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7fc870036000 tx=0x7fc870030000 comp rx=0 tx=0).ready entity=mgr.7765 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-09T14:51:34.934 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:34.932+0000 7fc86a7fc640 1 -- 172.21.15.43:0/273952401 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178066 (secure 0 0 0) 0x7fc864053550 con 0x7fc8740acf90 2023-08-09T14:51:35.134 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:35.133+0000 7fc87aad7640 1 -- 172.21.15.43:0/273952401 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7fc87406c550 con 0x7fc8740acf90 2023-08-09T14:51:35.668 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:35.667+0000 7fc86a7fc640 1 -- 172.21.15.43:0/273952401 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data' removed v164) v1 ==== 169+0+0 (secure 0 0 0) 0x7fc864056b70 con 0x7fc8740acf90 2023-08-09T14:51:35.669 INFO:teuthology.orchestra.run.smithi043.stderr:pool 'cephfs_data' removed 2023-08-09T14:51:35.674 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:35.672+0000 7fc87aad7640 1 -- 172.21.15.43:0/273952401 >> [v2:172.21.15.203:6832/2566816332,v1:172.21.15.203:6833/2566816332] conn(0x7fc85806a1a0 msgr2=0x7fc85806c660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T14:51:35.675 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:35.672+0000 7fc87aad7640 1 --2- 172.21.15.43:0/273952401 >> [v2:172.21.15.203:6832/2566816332,v1:172.21.15.203:6833/2566816332] conn(0x7fc85806a1a0 0x7fc85806c660 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7fc870036000 tx=0x7fc870030000 comp rx=0 tx=0).stop 2023-08-09T14:51:35.675 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:35.672+0000 7fc87aad7640 1 -- 172.21.15.43:0/273952401 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc8740acf90 msgr2=0x7fc8740cbb10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T14:51:35.675 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:35.672+0000 7fc87aad7640 1 --2- 172.21.15.43:0/273952401 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc8740acf90 0x7fc8740cbb10 secure :-1 s=READY pgs=1805 cs=0 l=1 rev1=1 crypto rx=0x7fc86400d5c0 tx=0x7fc86400da80 comp rx=0 tx=0).stop 2023-08-09T14:51:35.675 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:35.672+0000 7fc87aad7640 1 -- 172.21.15.43:0/273952401 shutdown_connections 2023-08-09T14:51:35.675 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:35.672+0000 7fc87aad7640 1 --2- 172.21.15.43:0/273952401 >> [v2:172.21.15.203:6832/2566816332,v1:172.21.15.203:6833/2566816332] conn(0x7fc85806a1a0 0x7fc85806c660 unknown :-1 s=CLOSED pgs=35 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T14:51:35.675 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:35.672+0000 7fc87aad7640 1 --2- 172.21.15.43:0/273952401 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fc8740cc050 0x7fc8740c6a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T14:51:35.675 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:35.672+0000 7fc87aad7640 1 --2- 172.21.15.43:0/273952401 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc8740acf90 0x7fc8740cbb10 unknown :-1 s=CLOSED pgs=1805 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T14:51:35.676 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:35.672+0000 7fc87aad7640 1 --2- 172.21.15.43:0/273952401 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fc860003970 0x7fc8740aca50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T14:51:35.676 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:35.672+0000 7fc87aad7640 1 -- 172.21.15.43:0/273952401 >> 172.21.15.43:0/273952401 conn(0x7fc8740ca870 msgr2=0x7fc8740b3850 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-09T14:51:35.676 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:35.673+0000 7fc87aad7640 1 -- 172.21.15.43:0/273952401 shutdown_connections 2023-08-09T14:51:35.676 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:35.673+0000 7fc87aad7640 1 -- 172.21.15.43:0/273952401 wait complete. 2023-08-09T14:51:35.697 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config rm mgr mgr/snap_schedule/dump_on_update 2023-08-09T14:51:35.927 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:35.925+0000 7f4ebb59e640 1 Processor -- start 2023-08-09T14:51:35.927 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:35.925+0000 7f4ebb59e640 1 -- start start 2023-08-09T14:51:35.928 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:35.927+0000 7f4ebb59e640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f4ebc159400 0x7f4ebc0cb9a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T14:51:35.929 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:35.927+0000 7f4ebb59e640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f4ebc0c8900 0x7f4ebc15a270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T14:51:35.929 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:35.927+0000 7f4ebb59e640 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7f4ebc0c8e50 0x7f4ebc0c92e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T14:51:35.929 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:35.927+0000 7f4ebb59e640 1 -- --> v2:172.21.15.43:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4ebc0c9820 con 0x7f4ebc0c8900 2023-08-09T14:51:35.929 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:35.927+0000 7f4ebb59e640 1 -- --> v2:172.21.15.203:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4ebc15a7b0 con 0x7f4ebc0c8e50 2023-08-09T14:51:35.929 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:35.927+0000 7f4ebb59e640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4ebc070250 con 0x7f4ebc159400 2023-08-09T14:51:35.929 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:35.927+0000 7f4eb1d9b640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f4ebc0c8900 0x7f4ebc15a270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T14:51:35.930 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:35.927+0000 7f4eb1d9b640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f4ebc0c8900 0x7f4ebc15a270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:52308/0 (socket says 172.21.15.43:52308) 2023-08-09T14:51:35.930 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:35.927+0000 7f4eb1d9b640 1 -- 172.21.15.43:0/1311375232 learned_addr learned my addr 172.21.15.43:0/1311375232 (peer_addr_for_me v2:172.21.15.43:0/0) 2023-08-09T14:51:35.931 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:35.928+0000 7f4eba59c640 1 --2- 172.21.15.43:0/1311375232 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f4ebc159400 0x7f4ebc0cb9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T14:51:35.931 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:35.928+0000 7f4eb1d9b640 1 -- 172.21.15.43:0/1311375232 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f4ebc159400 msgr2=0x7f4ebc0cb9a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-09T14:51:35.931 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:35.928+0000 7f4ebad9d640 1 --2- 172.21.15.43:0/1311375232 >> v2:172.21.15.203:3300/0 conn(0x7f4ebc0c8e50 0x7f4ebc0c92e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T14:51:35.931 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:35.928+0000 7f4eb1d9b640 1 --2- 172.21.15.43:0/1311375232 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f4ebc159400 0x7f4ebc0cb9a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T14:51:35.932 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:35.928+0000 7f4eb1d9b640 1 -- 172.21.15.43:0/1311375232 >> v2:172.21.15.203:3300/0 conn(0x7f4ebc0c8e50 msgr2=0x7f4ebc0c92e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-09T14:51:35.932 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:35.928+0000 7f4eb1d9b640 1 --2- 172.21.15.43:0/1311375232 >> v2:172.21.15.203:3300/0 conn(0x7f4ebc0c8e50 0x7f4ebc0c92e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T14:51:35.932 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:35.928+0000 7f4eb1d9b640 1 -- 172.21.15.43:0/1311375232 --> v2:172.21.15.43:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4ebc0c86c0 con 0x7f4ebc0c8900 2023-08-09T14:51:35.932 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:35.928+0000 7f4eba59c640 1 --2- 172.21.15.43:0/1311375232 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f4ebc159400 0x7f4ebc0cb9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-08-09T14:51:35.932 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:35.928+0000 7f4ebad9d640 1 --2- 172.21.15.43:0/1311375232 >> v2:172.21.15.203:3300/0 conn(0x7f4ebc0c8e50 0x7f4ebc0c92e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-08-09T14:51:35.932 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:35.929+0000 7f4eb1d9b640 1 --2- 172.21.15.43:0/1311375232 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f4ebc0c8900 0x7f4ebc15a270 secure :-1 s=READY pgs=1806 cs=0 l=1 rev1=1 crypto rx=0x7f4ebc06e730 tx=0x7f4ea402be50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=11039f3a45874984 server_cookie=0 in_seq=0 out_seq=0 2023-08-09T14:51:35.932 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:35.929+0000 7f4eb9d9b640 1 -- 172.21.15.43:0/1311375232 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4ea402c9f0 con 0x7f4ebc0c8900 2023-08-09T14:51:35.933 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:35.929+0000 7f4eb9d9b640 1 -- 172.21.15.43:0/1311375232 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4ea402cb50 con 0x7f4ebc0c8900 2023-08-09T14:51:35.933 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:35.929+0000 7f4eb9d9b640 1 -- 172.21.15.43:0/1311375232 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4ea40343f0 con 0x7f4ebc0c8900 2023-08-09T14:51:35.934 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:35.930+0000 7f4ebb59e640 1 -- 172.21.15.43:0/1311375232 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f4ebc0c8900 msgr2=0x7f4ebc15a270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T14:51:35.934 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:35.930+0000 7f4ebb59e640 1 --2- 172.21.15.43:0/1311375232 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f4ebc0c8900 0x7f4ebc15a270 secure :-1 s=READY pgs=1806 cs=0 l=1 rev1=1 crypto rx=0x7f4ebc06e730 tx=0x7f4ea402be50 comp rx=0 tx=0).stop 2023-08-09T14:51:35.934 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:35.930+0000 7f4ebb59e640 1 -- 172.21.15.43:0/1311375232 shutdown_connections 2023-08-09T14:51:35.935 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:35.930+0000 7f4ebb59e640 1 --2- 172.21.15.43:0/1311375232 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f4ebc0c8900 0x7f4ebc15a270 unknown :-1 s=CLOSED pgs=1806 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T14:51:35.935 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:35.930+0000 7f4ebb59e640 1 --2- 172.21.15.43:0/1311375232 >> v2:172.21.15.203:3300/0 conn(0x7f4ebc0c8e50 0x7f4ebc0c92e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T14:51:35.935 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:35.930+0000 7f4ebb59e640 1 --2- 172.21.15.43:0/1311375232 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f4ebc159400 0x7f4ebc0cb9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T14:51:35.935 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:35.930+0000 7f4ebb59e640 1 -- 172.21.15.43:0/1311375232 >> 172.21.15.43:0/1311375232 conn(0x7f4ebc0cddd0 msgr2=0x7f4ebc0ce1e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-09T14:51:35.935 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:35.930+0000 7f4ebb59e640 1 -- 172.21.15.43:0/1311375232 shutdown_connections 2023-08-09T14:51:35.935 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:35.930+0000 7f4ebb59e640 1 -- 172.21.15.43:0/1311375232 wait complete. 2023-08-09T14:51:35.935 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:35.932+0000 7f4ebb59e640 1 Processor -- start 2023-08-09T14:51:35.936 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:35.932+0000 7f4ebb59e640 1 -- start start 2023-08-09T14:51:35.937 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:35.933+0000 7f4ebb59e640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f4ebc0c8900 0x7f4ebc0c3040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T14:51:35.937 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:35.933+0000 7f4ebb59e640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f4ebc0c8e50 0x7f4ebc15b290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T14:51:35.938 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:35.933+0000 7f4ebb59e640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f4ebc1561d0 0x7f4ebc156680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T14:51:35.938 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:35.933+0000 7f4ebb59e640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4ebc156bc0 con 0x7f4ebc1561d0 2023-08-09T14:51:35.938 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:35.933+0000 7f4ebb59e640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4ebc15d230 con 0x7f4ebc0c8e50 2023-08-09T14:51:35.938 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:35.933+0000 7f4ebb59e640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4ebc0c3580 con 0x7f4ebc0c8900 2023-08-09T14:51:35.938 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:35.933+0000 7f4ebad9d640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f4ebc1561d0 0x7f4ebc156680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T14:51:35.938 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:35.933+0000 7f4ebad9d640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f4ebc1561d0 0x7f4ebc156680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:52316/0 (socket says 172.21.15.43:52316) 2023-08-09T14:51:35.939 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:35.933+0000 7f4ebad9d640 1 -- 172.21.15.43:0/247974106 learned_addr learned my addr 172.21.15.43:0/247974106 (peer_addr_for_me v2:172.21.15.43:0/0) 2023-08-09T14:51:35.939 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:35.934+0000 7f4eba59c640 1 --2- 172.21.15.43:0/247974106 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f4ebc0c8900 0x7f4ebc0c3040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T14:51:35.939 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:35.934+0000 7f4eb1d9b640 1 --2- 172.21.15.43:0/247974106 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f4ebc0c8e50 0x7f4ebc15b290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T14:51:35.939 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:35.934+0000 7f4ebad9d640 1 -- 172.21.15.43:0/247974106 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f4ebc0c8900 msgr2=0x7f4ebc0c3040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T14:51:35.939 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:35.934+0000 7f4ebad9d640 1 --2- 172.21.15.43:0/247974106 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f4ebc0c8900 0x7f4ebc0c3040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T14:51:35.939 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:35.934+0000 7f4ebad9d640 1 -- 172.21.15.43:0/247974106 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f4ebc0c8e50 msgr2=0x7f4ebc15b290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T14:51:35.939 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:35.934+0000 7f4ebad9d640 1 --2- 172.21.15.43:0/247974106 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f4ebc0c8e50 0x7f4ebc15b290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T14:51:35.939 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:35.934+0000 7f4ebad9d640 1 -- 172.21.15.43:0/247974106 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4ea4009b30 con 0x7f4ebc1561d0 2023-08-09T14:51:35.940 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:35.934+0000 7f4ebad9d640 1 --2- 172.21.15.43:0/247974106 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f4ebc1561d0 0x7f4ebc156680 secure :-1 s=READY pgs=1807 cs=0 l=1 rev1=1 crypto rx=0x7f4eb400c820 tx=0x7f4eb400cce0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-09T14:51:35.940 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:35.934+0000 7f4eb37fe640 1 -- 172.21.15.43:0/247974106 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4eb4007a30 con 0x7f4ebc1561d0 2023-08-09T14:51:35.940 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:35.935+0000 7f4eb37fe640 1 -- 172.21.15.43:0/247974106 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4eb4007b90 con 0x7f4ebc1561d0 2023-08-09T14:51:35.940 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:35.935+0000 7f4eb37fe640 1 -- 172.21.15.43:0/247974106 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4eb400ce10 con 0x7f4ebc1561d0 2023-08-09T14:51:35.940 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:35.935+0000 7f4ebb59e640 1 -- 172.21.15.43:0/247974106 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4ebc15d6c0 con 0x7f4ebc1561d0 2023-08-09T14:51:35.940 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:35.935+0000 7f4ebb59e640 1 -- 172.21.15.43:0/247974106 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4ebc15c900 con 0x7f4ebc1561d0 2023-08-09T14:51:35.940 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:35.936+0000 7f4ebb59e640 1 -- 172.21.15.43:0/247974106 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4ebc04e390 con 0x7f4ebc1561d0 2023-08-09T14:51:35.941 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:35.937+0000 7f4eb37fe640 1 -- 172.21.15.43:0/247974106 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 79) v1 ==== 88443+0+0 (secure 0 0 0) 0x7f4eb40113f0 con 0x7f4ebc1561d0 2023-08-09T14:51:35.941 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:35.938+0000 7f4eb37fe640 1 --2- 172.21.15.43:0/247974106 >> [v2:172.21.15.203:6832/2566816332,v1:172.21.15.203:6833/2566816332] conn(0x7f4e9c06a1a0 0x7f4e9c06c660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T14:51:35.941 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:35.938+0000 7f4eb37fe640 1 -- 172.21.15.43:0/247974106 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(164..164 src has 1..164) v4 ==== 6348+0+0 (secure 0 0 0) 0x7f4eb4088a40 con 0x7f4ebc1561d0 2023-08-09T14:51:35.942 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:35.939+0000 7f4eba59c640 1 --2- 172.21.15.43:0/247974106 >> [v2:172.21.15.203:6832/2566816332,v1:172.21.15.203:6833/2566816332] conn(0x7f4e9c06a1a0 0x7f4e9c06c660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T14:51:35.942 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:35.940+0000 7f4eba59c640 1 --2- 172.21.15.43:0/247974106 >> [v2:172.21.15.203:6832/2566816332,v1:172.21.15.203:6833/2566816332] conn(0x7f4e9c06a1a0 0x7f4e9c06c660 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7f4eac00a950 tx=0x7f4eac008040 comp rx=0 tx=0).ready entity=mgr.7765 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-09T14:51:35.944 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:35.943+0000 7f4eb37fe640 1 -- 172.21.15.43:0/247974106 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178066 (secure 0 0 0) 0x7f4eb4053e10 con 0x7f4ebc1561d0 2023-08-09T14:51:36.152 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:36.151+0000 7f4ebb59e640 1 -- 172.21.15.43:0/247974106 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "config rm", "who": "mgr", "name": "mgr/snap_schedule/dump_on_update"} v 0) v1 -- 0x7f4ebc0cea30 con 0x7f4ebc1561d0 2023-08-09T14:51:36.158 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:36.157+0000 7f4eb37fe640 1 -- 172.21.15.43:0/247974106 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "config rm", "who": "mgr", "name": "mgr/snap_schedule/dump_on_update"}]=0 v49) v1 ==== 115+0+0 (secure 0 0 0) 0x7f4eb4057430 con 0x7f4ebc1561d0 2023-08-09T14:51:36.161 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:36.160+0000 7f4ebb59e640 1 -- 172.21.15.43:0/247974106 >> [v2:172.21.15.203:6832/2566816332,v1:172.21.15.203:6833/2566816332] conn(0x7f4e9c06a1a0 msgr2=0x7f4e9c06c660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T14:51:36.162 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:36.160+0000 7f4ebb59e640 1 --2- 172.21.15.43:0/247974106 >> [v2:172.21.15.203:6832/2566816332,v1:172.21.15.203:6833/2566816332] conn(0x7f4e9c06a1a0 0x7f4e9c06c660 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7f4eac00a950 tx=0x7f4eac008040 comp rx=0 tx=0).stop 2023-08-09T14:51:36.162 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:36.160+0000 7f4ebb59e640 1 -- 172.21.15.43:0/247974106 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f4ebc1561d0 msgr2=0x7f4ebc156680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T14:51:36.162 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:36.160+0000 7f4ebb59e640 1 --2- 172.21.15.43:0/247974106 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f4ebc1561d0 0x7f4ebc156680 secure :-1 s=READY pgs=1807 cs=0 l=1 rev1=1 crypto rx=0x7f4eb400c820 tx=0x7f4eb400cce0 comp rx=0 tx=0).stop 2023-08-09T14:51:36.162 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:36.160+0000 7f4ebb59e640 1 -- 172.21.15.43:0/247974106 shutdown_connections 2023-08-09T14:51:36.162 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:36.160+0000 7f4ebb59e640 1 --2- 172.21.15.43:0/247974106 >> [v2:172.21.15.203:6832/2566816332,v1:172.21.15.203:6833/2566816332] conn(0x7f4e9c06a1a0 0x7f4e9c06c660 unknown :-1 s=CLOSED pgs=36 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T14:51:36.162 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:36.160+0000 7f4ebb59e640 1 --2- 172.21.15.43:0/247974106 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f4ebc1561d0 0x7f4ebc156680 unknown :-1 s=CLOSED pgs=1807 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T14:51:36.162 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:36.160+0000 7f4ebb59e640 1 --2- 172.21.15.43:0/247974106 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f4ebc0c8e50 0x7f4ebc15b290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T14:51:36.162 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:36.160+0000 7f4ebb59e640 1 --2- 172.21.15.43:0/247974106 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f4ebc0c8900 0x7f4ebc0c3040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T14:51:36.163 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:36.160+0000 7f4ebb59e640 1 -- 172.21.15.43:0/247974106 >> 172.21.15.43:0/247974106 conn(0x7f4ebc0cddd0 msgr2=0x7f4ebc0ca850 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-09T14:51:36.163 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:36.160+0000 7f4ebb59e640 1 -- 172.21.15.43:0/247974106 shutdown_connections 2023-08-09T14:51:36.163 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:36.160+0000 7f4ebb59e640 1 -- 172.21.15.43:0/247974106 wait complete. 2023-08-09T14:51:36.181 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config rm mgr mgr/snap_schedule/allow_m_granularity 2023-08-09T14:51:36.400 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:36.399+0000 7f445ac91640 1 Processor -- start 2023-08-09T14:51:36.401 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:36.399+0000 7f445ac91640 1 -- start start 2023-08-09T14:51:36.402 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:36.401+0000 7f445ac91640 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7f44540c9ee0 0x7f445415a800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T14:51:36.402 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:36.401+0000 7f445ac91640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f44540c90a0 0x7f445415ad40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T14:51:36.402 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:36.401+0000 7f445ac91640 1 -- --> v1:172.21.15.43:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4454158280 con 0x7f4454157ea0 2023-08-09T14:51:36.403 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:36.401+0000 7f445ac91640 1 -- --> v2:172.21.15.203:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f445415b840 con 0x7f44540c9ee0 2023-08-09T14:51:36.403 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:36.401+0000 7f445ac91640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f44540b3810 con 0x7f44540c90a0 2023-08-09T14:51:36.403 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:36.401+0000 7f4459c8f640 1 --1- >> v1:172.21.15.43:6789/0 conn(0x7f4454157ea0 0x7f44540c97d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.43:6789/0 says I am v1:172.21.15.43:46610/0 (socket says 172.21.15.43:46610) 2023-08-09T14:51:36.404 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:36.401+0000 7f4459c8f640 1 -- 172.21.15.43:0/3752585449 learned_addr learned my addr 172.21.15.43:0/3752585449 (peer_addr_for_me v1:172.21.15.43:0/0) 2023-08-09T14:51:36.404 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:36.402+0000 7f445948e640 1 --2- 172.21.15.43:0/3752585449 >> v2:172.21.15.203:3300/0 conn(0x7f44540c9ee0 0x7f445415a800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T14:51:36.404 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:36.402+0000 7f445a490640 1 --2- 172.21.15.43:0/3752585449 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f44540c90a0 0x7f445415ad40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T14:51:36.404 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:36.402+0000 7f4458c8d640 1 -- 172.21.15.43:0/3752585449 <== mon.0 v1:172.21.15.43:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2288575721 0 0) 0x7f4450001110 con 0x7f4454157ea0 2023-08-09T14:51:36.404 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:36.402+0000 7f4458c8d640 1 -- 172.21.15.43:0/3752585449 --> v1:172.21.15.43:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f4440003520 con 0x7f4454157ea0 2023-08-09T14:51:36.404 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:36.402+0000 7f4458c8d640 1 -- 172.21.15.43:0/3752585449 <== mon.0 v1:172.21.15.43:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2235385286 0 0) 0x7f4440003520 con 0x7f4454157ea0 2023-08-09T14:51:36.404 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:36.402+0000 7f4458c8d640 1 -- 172.21.15.43:0/3752585449 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f44540c90a0 msgr2=0x7f445415ad40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-09T14:51:36.405 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:36.402+0000 7f4458c8d640 1 --2- 172.21.15.43:0/3752585449 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f44540c90a0 0x7f445415ad40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T14:51:36.405 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:36.402+0000 7f4458c8d640 1 -- 172.21.15.43:0/3752585449 >> v2:172.21.15.203:3300/0 conn(0x7f44540c9ee0 msgr2=0x7f445415a800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-09T14:51:36.405 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:36.402+0000 7f4458c8d640 1 --2- 172.21.15.43:0/3752585449 >> v2:172.21.15.203:3300/0 conn(0x7f44540c9ee0 0x7f445415a800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T14:51:36.406 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:36.402+0000 7f4458c8d640 1 -- 172.21.15.43:0/3752585449 --> v1:172.21.15.43:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f445415b560 con 0x7f4454157ea0 2023-08-09T14:51:36.406 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:36.402+0000 7f4458c8d640 1 -- 172.21.15.43:0/3752585449 <== mon.0 v1:172.21.15.43:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1285150740 0 0) 0x7f4450001800 con 0x7f4454157ea0 2023-08-09T14:51:36.406 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:36.402+0000 7f4458c8d640 1 -- 172.21.15.43:0/3752585449 >> v1:172.21.15.43:6789/0 conn(0x7f4454157ea0 legacy=0x7f44540c97d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T14:51:36.406 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:36.402+0000 7f4458c8d640 1 --2- 172.21.15.43:0/3752585449 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f44400039a0 0x7f4440003dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T14:51:36.407 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:36.402+0000 7f4458c8d640 1 --2- 172.21.15.43:0/3752585449 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f44540c90a0 0x7f44400047c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T14:51:36.407 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:36.402+0000 7f4458c8d640 1 --2- 172.21.15.43:0/3752585449 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f4440004d00 0x7f4440005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T14:51:36.407 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:36.402+0000 7f4458c8d640 1 -- 172.21.15.43:0/3752585449 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f445415b840 con 0x7f44400039a0 2023-08-09T14:51:36.407 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:36.402+0000 7f4458c8d640 1 -- 172.21.15.43:0/3752585449 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f44540b3810 con 0x7f4440004d00 2023-08-09T14:51:36.407 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:36.402+0000 7f4458c8d640 1 -- 172.21.15.43:0/3752585449 --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f444000fdc0 con 0x7f44540c90a0 2023-08-09T14:51:36.407 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:36.403+0000 7f4459c8f640 1 --2- 172.21.15.43:0/3752585449 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f44400039a0 0x7f4440003dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T14:51:36.407 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:36.403+0000 7f445a490640 1 --2- 172.21.15.43:0/3752585449 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f4440004d00 0x7f4440005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T14:51:36.408 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:36.403+0000 7f445948e640 1 --2- 172.21.15.43:0/3752585449 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f44540c90a0 0x7f44400047c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T14:51:36.408 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:36.403+0000 7f4459c8f640 1 -- 172.21.15.43:0/3752585449 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f44540c90a0 msgr2=0x7f44400047c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-09T14:51:36.408 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:36.403+0000 7f4459c8f640 1 --2- 172.21.15.43:0/3752585449 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f44540c90a0 0x7f44400047c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T14:51:36.408 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:36.403+0000 7f4459c8f640 1 -- 172.21.15.43:0/3752585449 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f4440004d00 msgr2=0x7f4440005180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-09T14:51:36.408 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:36.403+0000 7f4459c8f640 1 --2- 172.21.15.43:0/3752585449 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f4440004d00 0x7f4440005180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T14:51:36.408 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:36.403+0000 7f4459c8f640 1 -- 172.21.15.43:0/3752585449 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4440003520 con 0x7f44400039a0 2023-08-09T14:51:36.408 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:36.404+0000 7f4459c8f640 1 --2- 172.21.15.43:0/3752585449 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f44400039a0 0x7f4440003dc0 secure :-1 s=READY pgs=1809 cs=0 l=1 rev1=1 crypto rx=0x7f4450012950 tx=0x7f44400057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=54e9b662a3e47dab server_cookie=0 in_seq=0 out_seq=0 2023-08-09T14:51:36.408 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:36.405+0000 7f4458c8d640 1 -- 172.21.15.43:0/3752585449 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4450005ad0 con 0x7f44400039a0 2023-08-09T14:51:36.409 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:36.405+0000 7f4458c8d640 1 -- 172.21.15.43:0/3752585449 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4450037070 con 0x7f44400039a0 2023-08-09T14:51:36.409 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:36.405+0000 7f4458c8d640 1 -- 172.21.15.43:0/3752585449 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4450005e90 con 0x7f44400039a0 2023-08-09T14:51:36.410 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:36.405+0000 7f445ac91640 1 -- 172.21.15.43:0/3752585449 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f44400039a0 msgr2=0x7f4440003dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T14:51:36.410 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:36.405+0000 7f445ac91640 1 --2- 172.21.15.43:0/3752585449 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f44400039a0 0x7f4440003dc0 secure :-1 s=READY pgs=1809 cs=0 l=1 rev1=1 crypto rx=0x7f4450012950 tx=0x7f44400057f0 comp rx=0 tx=0).stop 2023-08-09T14:51:36.410 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:36.405+0000 7f445a490640 1 -- 172.21.15.43:0/3752585449 reap_dead start 2023-08-09T14:51:36.410 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:36.405+0000 7f445ac91640 1 -- 172.21.15.43:0/3752585449 shutdown_connections 2023-08-09T14:51:36.410 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:36.405+0000 7f445ac91640 1 -- 172.21.15.43:0/3752585449 >> 172.21.15.43:0/3752585449 conn(0x7f44540cddd0 msgr2=0x7f44540ce1e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-09T14:51:36.410 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:36.405+0000 7f445ac91640 1 -- 172.21.15.43:0/3752585449 shutdown_connections 2023-08-09T14:51:36.410 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:36.406+0000 7f445ac91640 1 -- 172.21.15.43:0/3752585449 wait complete. 2023-08-09T14:51:36.411 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:36.408+0000 7f445ac91640 1 Processor -- start 2023-08-09T14:51:36.411 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:36.408+0000 7f445ac91640 1 -- start start 2023-08-09T14:51:36.412 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:36.409+0000 7f445ac91640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f44400039a0 0x7f44540c7690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T14:51:36.412 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:36.409+0000 7f445ac91640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f445415d3c0 0x7f44540c4310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T14:51:36.412 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:36.409+0000 7f445ac91640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f4454158420 0x7f44541588b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T14:51:36.412 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:36.409+0000 7f445ac91640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f445415cfa0 con 0x7f44400039a0 2023-08-09T14:51:36.412 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:36.409+0000 7f445ac91640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4454158df0 con 0x7f4454158420 2023-08-09T14:51:36.412 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:36.409+0000 7f445ac91640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f44540c51d0 con 0x7f445415d3c0 2023-08-09T14:51:36.413 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:36.410+0000 7f4459c8f640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f44400039a0 0x7f44540c7690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T14:51:36.413 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:36.410+0000 7f4459c8f640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f44400039a0 0x7f44540c7690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:52336/0 (socket says 172.21.15.43:52336) 2023-08-09T14:51:36.413 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:36.410+0000 7f4459c8f640 1 -- 172.21.15.43:0/4223673363 learned_addr learned my addr 172.21.15.43:0/4223673363 (peer_addr_for_me v2:172.21.15.43:0/0) 2023-08-09T14:51:36.413 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:36.410+0000 7f445a490640 1 --2- 172.21.15.43:0/4223673363 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f4454158420 0x7f44541588b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T14:51:36.413 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:36.410+0000 7f445948e640 1 --2- 172.21.15.43:0/4223673363 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f445415d3c0 0x7f44540c4310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T14:51:36.413 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:36.410+0000 7f4459c8f640 1 -- 172.21.15.43:0/4223673363 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f445415d3c0 msgr2=0x7f44540c4310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T14:51:36.413 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:36.410+0000 7f4459c8f640 1 --2- 172.21.15.43:0/4223673363 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f445415d3c0 0x7f44540c4310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T14:51:36.414 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:36.410+0000 7f4459c8f640 1 -- 172.21.15.43:0/4223673363 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f4454158420 msgr2=0x7f44541588b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T14:51:36.414 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:36.410+0000 7f4459c8f640 1 --2- 172.21.15.43:0/4223673363 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f4454158420 0x7f44541588b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T14:51:36.414 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:36.410+0000 7f4459c8f640 1 -- 172.21.15.43:0/4223673363 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f445415c320 con 0x7f44400039a0 2023-08-09T14:51:36.414 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:36.410+0000 7f445a490640 1 --2- 172.21.15.43:0/4223673363 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f4454158420 0x7f44541588b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-08-09T14:51:36.414 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:36.410+0000 7f4459c8f640 1 --2- 172.21.15.43:0/4223673363 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f44400039a0 0x7f44540c7690 secure :-1 s=READY pgs=1810 cs=0 l=1 rev1=1 crypto rx=0x7f4450012950 tx=0x7f4450001b90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-09T14:51:36.415 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:36.411+0000 7f444a7fc640 1 -- 172.21.15.43:0/4223673363 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4450043070 con 0x7f44400039a0 2023-08-09T14:51:36.415 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:36.411+0000 7f444a7fc640 1 -- 172.21.15.43:0/4223673363 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4450041070 con 0x7f44400039a0 2023-08-09T14:51:36.415 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:36.411+0000 7f444a7fc640 1 -- 172.21.15.43:0/4223673363 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4450010cb0 con 0x7f44400039a0 2023-08-09T14:51:36.415 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:36.411+0000 7f445ac91640 1 -- 172.21.15.43:0/4223673363 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f445415c600 con 0x7f44400039a0 2023-08-09T14:51:36.415 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:36.411+0000 7f445ac91640 1 -- 172.21.15.43:0/4223673363 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f445415cbc0 con 0x7f44400039a0 2023-08-09T14:51:36.416 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:36.412+0000 7f445ac91640 1 -- 172.21.15.43:0/4223673363 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f445404e390 con 0x7f44400039a0 2023-08-09T14:51:36.416 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:36.413+0000 7f444a7fc640 1 -- 172.21.15.43:0/4223673363 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 79) v1 ==== 88443+0+0 (secure 0 0 0) 0x7f44500047e0 con 0x7f44400039a0 2023-08-09T14:51:36.416 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:36.414+0000 7f444a7fc640 1 --2- 172.21.15.43:0/4223673363 >> [v2:172.21.15.203:6832/2566816332,v1:172.21.15.203:6833/2566816332] conn(0x7f443806a170 0x7f443806c630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T14:51:36.416 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:36.414+0000 7f444a7fc640 1 -- 172.21.15.43:0/4223673363 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(164..164 src has 1..164) v4 ==== 6348+0+0 (secure 0 0 0) 0x7f44500b2090 con 0x7f44400039a0 2023-08-09T14:51:36.417 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:36.415+0000 7f445948e640 1 --2- 172.21.15.43:0/4223673363 >> [v2:172.21.15.203:6832/2566816332,v1:172.21.15.203:6833/2566816332] conn(0x7f443806a170 0x7f443806c630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T14:51:36.417 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:36.416+0000 7f445948e640 1 --2- 172.21.15.43:0/4223673363 >> [v2:172.21.15.203:6832/2566816332,v1:172.21.15.203:6833/2566816332] conn(0x7f443806a170 0x7f443806c630 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7f445406b790 tx=0x7f4444008040 comp rx=0 tx=0).ready entity=mgr.7765 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-09T14:51:36.419 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:36.418+0000 7f444a7fc640 1 -- 172.21.15.43:0/4223673363 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178066 (secure 0 0 0) 0x7f445007d5b0 con 0x7f44400039a0 2023-08-09T14:51:36.618 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:36.616+0000 7f445ac91640 1 -- 172.21.15.43:0/4223673363 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "config rm", "who": "mgr", "name": "mgr/snap_schedule/allow_m_granularity"} v 0) v1 -- 0x7f4454158280 con 0x7f44400039a0 2023-08-09T14:51:36.625 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:36.623+0000 7f444a7fc640 1 -- 172.21.15.43:0/4223673363 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "config rm", "who": "mgr", "name": "mgr/snap_schedule/allow_m_granularity"}]=0 v50) v1 ==== 120+0+0 (secure 0 0 0) 0x7f4454158280 con 0x7f44400039a0 2023-08-09T14:51:36.628 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:36.626+0000 7f445ac91640 1 -- 172.21.15.43:0/4223673363 >> [v2:172.21.15.203:6832/2566816332,v1:172.21.15.203:6833/2566816332] conn(0x7f443806a170 msgr2=0x7f443806c630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T14:51:36.629 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:36.626+0000 7f445ac91640 1 --2- 172.21.15.43:0/4223673363 >> [v2:172.21.15.203:6832/2566816332,v1:172.21.15.203:6833/2566816332] conn(0x7f443806a170 0x7f443806c630 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7f445406b790 tx=0x7f4444008040 comp rx=0 tx=0).stop 2023-08-09T14:51:36.629 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:36.626+0000 7f445ac91640 1 -- 172.21.15.43:0/4223673363 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f44400039a0 msgr2=0x7f44540c7690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T14:51:36.629 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:36.626+0000 7f445ac91640 1 --2- 172.21.15.43:0/4223673363 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f44400039a0 0x7f44540c7690 secure :-1 s=READY pgs=1810 cs=0 l=1 rev1=1 crypto rx=0x7f4450012950 tx=0x7f4450001b90 comp rx=0 tx=0).stop 2023-08-09T14:51:36.629 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:36.626+0000 7f445ac91640 1 -- 172.21.15.43:0/4223673363 shutdown_connections 2023-08-09T14:51:36.630 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:36.626+0000 7f445ac91640 1 --2- 172.21.15.43:0/4223673363 >> [v2:172.21.15.203:6832/2566816332,v1:172.21.15.203:6833/2566816332] conn(0x7f443806a170 0x7f443806c630 unknown :-1 s=CLOSED pgs=37 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T14:51:36.630 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:36.626+0000 7f445ac91640 1 --2- 172.21.15.43:0/4223673363 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f4454158420 0x7f44541588b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T14:51:36.630 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:36.626+0000 7f445ac91640 1 --2- 172.21.15.43:0/4223673363 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f445415d3c0 0x7f44540c4310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T14:51:36.630 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:36.626+0000 7f445ac91640 1 --2- 172.21.15.43:0/4223673363 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f44400039a0 0x7f44540c7690 unknown :-1 s=CLOSED pgs=1810 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T14:51:36.630 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:36.626+0000 7f445ac91640 1 -- 172.21.15.43:0/4223673363 >> 172.21.15.43:0/4223673363 conn(0x7f44540cddd0 msgr2=0x7f44540ce1e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-09T14:51:36.631 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:36.626+0000 7f445ac91640 1 -- 172.21.15.43:0/4223673363 shutdown_connections 2023-08-09T14:51:36.631 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:36.626+0000 7f445ac91640 1 -- 172.21.15.43:0/4223673363 wait complete. 2023-08-09T14:51:36.648 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config rm mon mon_allow_pool_delete 2023-08-09T14:51:36.875 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:36.873+0000 7f10f1773640 1 Processor -- start 2023-08-09T14:51:36.876 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:36.874+0000 7f10f1773640 1 -- start start 2023-08-09T14:51:36.876 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:36.875+0000 7f10f1773640 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7f10ec159400 0x7f10ec0cb9a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T14:51:36.877 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:36.875+0000 7f10f1773640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f10ec0c9560 0x7f10ec15a270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T14:51:36.877 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:36.875+0000 7f10f1773640 1 -- --> v1:172.21.15.203:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f10ec15a7b0 con 0x7f10ec0c8900 2023-08-09T14:51:36.877 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:36.875+0000 7f10f1773640 1 -- --> v2:172.21.15.43:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f10ec070250 con 0x7f10ec0c9560 2023-08-09T14:51:36.877 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:36.875+0000 7f10f1773640 1 -- --> v2:172.21.15.203:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f10ec0c86c0 con 0x7f10ec159400 2023-08-09T14:51:36.878 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:36.875+0000 7f10f0f72640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f10ec0c9560 0x7f10ec15a270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T14:51:36.878 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:36.875+0000 7f10f0f72640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f10ec0c9560 0x7f10ec15a270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:52352/0 (socket says 172.21.15.43:52352) 2023-08-09T14:51:36.878 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:36.875+0000 7f10f0f72640 1 -- 172.21.15.43:0/1180658487 learned_addr learned my addr 172.21.15.43:0/1180658487 (peer_addr_for_me v2:172.21.15.43:0/0) 2023-08-09T14:51:36.878 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:36.876+0000 7f10ebfff640 1 --2- 172.21.15.43:0/1180658487 >> v2:172.21.15.203:3300/0 conn(0x7f10ec159400 0x7f10ec0cb9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T14:51:36.878 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:36.876+0000 7f10f0f72640 1 -- 172.21.15.43:0/1180658487 >> v2:172.21.15.203:3300/0 conn(0x7f10ec159400 msgr2=0x7f10ec0cb9a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-09T14:51:36.879 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:36.876+0000 7f10f0f72640 1 --2- 172.21.15.43:0/1180658487 >> v2:172.21.15.203:3300/0 conn(0x7f10ec159400 0x7f10ec0cb9a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T14:51:36.879 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:36.876+0000 7f10f0f72640 1 -- 172.21.15.43:0/1180658487 >> v1:172.21.15.203:6789/0 conn(0x7f10ec0c8900 legacy=0x7f10ec0c8e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-09T14:51:36.879 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:36.876+0000 7f10f0f72640 1 -- 172.21.15.43:0/1180658487 --> v2:172.21.15.43:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f10ec06fc60 con 0x7f10ec0c9560 2023-08-09T14:51:36.880 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:36.877+0000 7f10f0f72640 1 --2- 172.21.15.43:0/1180658487 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f10ec0c9560 0x7f10ec15a270 secure :-1 s=READY pgs=1811 cs=0 l=1 rev1=1 crypto rx=0x7f10ec0cb920 tx=0x7f10e402be50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e9cfe08c76029015 server_cookie=0 in_seq=0 out_seq=0 2023-08-09T14:51:36.880 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:36.878+0000 7f10eb7fe640 1 -- 172.21.15.43:0/1180658487 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f10e402c9f0 con 0x7f10ec0c9560 2023-08-09T14:51:36.880 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:36.878+0000 7f10eb7fe640 1 -- 172.21.15.43:0/1180658487 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f10e402cb50 con 0x7f10ec0c9560 2023-08-09T14:51:36.880 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:36.878+0000 7f10eb7fe640 1 -- 172.21.15.43:0/1180658487 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f10e40343f0 con 0x7f10ec0c9560 2023-08-09T14:51:36.881 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:36.878+0000 7f10f1773640 1 -- 172.21.15.43:0/1180658487 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f10ec0c9560 msgr2=0x7f10ec15a270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T14:51:36.881 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:36.878+0000 7f10f1773640 1 --2- 172.21.15.43:0/1180658487 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f10ec0c9560 0x7f10ec15a270 secure :-1 s=READY pgs=1811 cs=0 l=1 rev1=1 crypto rx=0x7f10ec0cb920 tx=0x7f10e402be50 comp rx=0 tx=0).stop 2023-08-09T14:51:36.881 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:36.879+0000 7f10f1773640 1 -- 172.21.15.43:0/1180658487 shutdown_connections 2023-08-09T14:51:36.881 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:36.879+0000 7f10f1773640 1 --2- 172.21.15.43:0/1180658487 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f10ec0c9560 0x7f10ec15a270 unknown :-1 s=CLOSED pgs=1811 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T14:51:36.881 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:36.879+0000 7f10f1773640 1 --2- 172.21.15.43:0/1180658487 >> v2:172.21.15.203:3300/0 conn(0x7f10ec159400 0x7f10ec0cb9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T14:51:36.881 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:36.879+0000 7f10f1773640 1 -- 172.21.15.43:0/1180658487 >> 172.21.15.43:0/1180658487 conn(0x7f10ec0cddd0 msgr2=0x7f10ec0ce1e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-09T14:51:36.881 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:36.879+0000 7f10f1773640 1 -- 172.21.15.43:0/1180658487 shutdown_connections 2023-08-09T14:51:36.882 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:36.879+0000 7f10f1773640 1 -- 172.21.15.43:0/1180658487 wait complete. 2023-08-09T14:51:36.882 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:36.880+0000 7f10f1773640 1 Processor -- start 2023-08-09T14:51:36.882 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:36.881+0000 7f10f1773640 1 -- start start 2023-08-09T14:51:36.883 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:36.881+0000 7f10f1773640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f10ec0c8900 0x7f10ec0ac2b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T14:51:36.883 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:36.882+0000 7f10f1773640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f10ec0c9560 0x7f10ec1f11d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T14:51:36.884 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:36.882+0000 7f10f1773640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f10ec0cac80 0x7f10ec0cb110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T14:51:36.884 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:36.882+0000 7f10f1773640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f10ec0ac7f0 con 0x7f10ec0c8900 2023-08-09T14:51:36.884 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:36.882+0000 7f10f1773640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f10ec154f00 con 0x7f10ec0cac80 2023-08-09T14:51:36.884 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:36.882+0000 7f10f1773640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f10ec0ad290 con 0x7f10ec0c9560 2023-08-09T14:51:36.884 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:36.882+0000 7f10ebfff640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f10ec0c8900 0x7f10ec0ac2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T14:51:36.884 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:36.882+0000 7f10ebfff640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f10ec0c8900 0x7f10ec0ac2b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:52358/0 (socket says 172.21.15.43:52358) 2023-08-09T14:51:36.885 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:36.882+0000 7f10ebfff640 1 -- 172.21.15.43:0/344192034 learned_addr learned my addr 172.21.15.43:0/344192034 (peer_addr_for_me v2:172.21.15.43:0/0) 2023-08-09T14:51:36.885 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:36.882+0000 7f10e37fe640 1 --2- 172.21.15.43:0/344192034 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f10ec0c9560 0x7f10ec1f11d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T14:51:36.886 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:36.882+0000 7f10f0f72640 1 --2- 172.21.15.43:0/344192034 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f10ec0cac80 0x7f10ec0cb110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T14:51:36.886 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:36.882+0000 7f10ebfff640 1 -- 172.21.15.43:0/344192034 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f10ec0c9560 msgr2=0x7f10ec1f11d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T14:51:36.886 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:36.882+0000 7f10ebfff640 1 --2- 172.21.15.43:0/344192034 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f10ec0c9560 0x7f10ec1f11d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T14:51:36.886 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:36.882+0000 7f10ebfff640 1 -- 172.21.15.43:0/344192034 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f10ec0cac80 msgr2=0x7f10ec0cb110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T14:51:36.886 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:36.882+0000 7f10ebfff640 1 --2- 172.21.15.43:0/344192034 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f10ec0cac80 0x7f10ec0cb110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T14:51:36.886 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:36.882+0000 7f10ebfff640 1 -- 172.21.15.43:0/344192034 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f10e4009b30 con 0x7f10ec0c8900 2023-08-09T14:51:36.887 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:36.883+0000 7f10ebfff640 1 --2- 172.21.15.43:0/344192034 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f10ec0c8900 0x7f10ec0ac2b0 secure :-1 s=READY pgs=1812 cs=0 l=1 rev1=1 crypto rx=0x7f10dc00c820 tx=0x7f10dc00cce0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-09T14:51:36.887 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:36.883+0000 7f10e97fa640 1 -- 172.21.15.43:0/344192034 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f10dc004020 con 0x7f10ec0c8900 2023-08-09T14:51:36.887 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:36.883+0000 7f10e97fa640 1 -- 172.21.15.43:0/344192034 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f10dc004180 con 0x7f10ec0c8900 2023-08-09T14:51:36.887 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:36.883+0000 7f10e97fa640 1 -- 172.21.15.43:0/344192034 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f10dc007830 con 0x7f10ec0c8900 2023-08-09T14:51:36.887 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:36.883+0000 7f10f1773640 1 -- 172.21.15.43:0/344192034 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f10ec0ac9f0 con 0x7f10ec0c8900 2023-08-09T14:51:36.887 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:36.883+0000 7f10f1773640 1 -- 172.21.15.43:0/344192034 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f10ec0acfb0 con 0x7f10ec0c8900 2023-08-09T14:51:36.888 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:36.884+0000 7f10f1773640 1 -- 172.21.15.43:0/344192034 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f10ec154cd0 con 0x7f10ec0c8900 2023-08-09T14:51:36.888 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:36.885+0000 7f10e97fa640 1 -- 172.21.15.43:0/344192034 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 79) v1 ==== 88443+0+0 (secure 0 0 0) 0x7f10dc007990 con 0x7f10ec0c8900 2023-08-09T14:51:36.888 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:36.886+0000 7f10e97fa640 1 --2- 172.21.15.43:0/344192034 >> [v2:172.21.15.203:6832/2566816332,v1:172.21.15.203:6833/2566816332] conn(0x7f10cc06a1a0 0x7f10cc06c660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T14:51:36.888 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:36.886+0000 7f10e97fa640 1 -- 172.21.15.43:0/344192034 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(164..164 src has 1..164) v4 ==== 6348+0+0 (secure 0 0 0) 0x7f10dc0886a0 con 0x7f10ec0c8900 2023-08-09T14:51:36.889 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:36.887+0000 7f10e37fe640 1 --2- 172.21.15.43:0/344192034 >> [v2:172.21.15.203:6832/2566816332,v1:172.21.15.203:6833/2566816332] conn(0x7f10cc06a1a0 0x7f10cc06c660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T14:51:36.889 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:36.888+0000 7f10e37fe640 1 --2- 172.21.15.43:0/344192034 >> [v2:172.21.15.203:6832/2566816332,v1:172.21.15.203:6833/2566816332] conn(0x7f10cc06a1a0 0x7f10cc06c660 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7f10ec0cce90 tx=0x7f10d4009040 comp rx=0 tx=0).ready entity=mgr.7765 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-09T14:51:36.892 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:36.891+0000 7f10e97fa640 1 -- 172.21.15.43:0/344192034 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178066 (secure 0 0 0) 0x7f10ec154cd0 con 0x7f10ec0c8900 2023-08-09T14:51:37.085 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:37.084+0000 7f10f1773640 1 -- 172.21.15.43:0/344192034 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"} v 0) v1 -- 0x7f10ec154cd0 con 0x7f10ec0c8900 2023-08-09T14:51:37.092 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:37.091+0000 7f10e97fa640 1 -- 172.21.15.43:0/344192034 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"}]=0 v51) v1 ==== 104+0+0 (secure 0 0 0) 0x7f10ec154cd0 con 0x7f10ec0c8900 2023-08-09T14:51:37.095 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:37.093+0000 7f10f1773640 1 -- 172.21.15.43:0/344192034 >> [v2:172.21.15.203:6832/2566816332,v1:172.21.15.203:6833/2566816332] conn(0x7f10cc06a1a0 msgr2=0x7f10cc06c660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T14:51:37.095 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:37.093+0000 7f10f1773640 1 --2- 172.21.15.43:0/344192034 >> [v2:172.21.15.203:6832/2566816332,v1:172.21.15.203:6833/2566816332] conn(0x7f10cc06a1a0 0x7f10cc06c660 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7f10ec0cce90 tx=0x7f10d4009040 comp rx=0 tx=0).stop 2023-08-09T14:51:37.095 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:37.093+0000 7f10f1773640 1 -- 172.21.15.43:0/344192034 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f10ec0c8900 msgr2=0x7f10ec0ac2b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T14:51:37.096 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:37.093+0000 7f10f1773640 1 --2- 172.21.15.43:0/344192034 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f10ec0c8900 0x7f10ec0ac2b0 secure :-1 s=READY pgs=1812 cs=0 l=1 rev1=1 crypto rx=0x7f10dc00c820 tx=0x7f10dc00cce0 comp rx=0 tx=0).stop 2023-08-09T14:51:37.096 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:37.093+0000 7f10f1773640 1 -- 172.21.15.43:0/344192034 shutdown_connections 2023-08-09T14:51:37.096 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:37.093+0000 7f10f1773640 1 --2- 172.21.15.43:0/344192034 >> [v2:172.21.15.203:6832/2566816332,v1:172.21.15.203:6833/2566816332] conn(0x7f10cc06a1a0 0x7f10cc06c660 unknown :-1 s=CLOSED pgs=38 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T14:51:37.096 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:37.093+0000 7f10f1773640 1 --2- 172.21.15.43:0/344192034 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f10ec0cac80 0x7f10ec0cb110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T14:51:37.096 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:37.093+0000 7f10f1773640 1 --2- 172.21.15.43:0/344192034 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f10ec0c9560 0x7f10ec1f11d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T14:51:37.096 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:37.093+0000 7f10f1773640 1 --2- 172.21.15.43:0/344192034 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f10ec0c8900 0x7f10ec0ac2b0 unknown :-1 s=CLOSED pgs=1812 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T14:51:37.097 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:37.093+0000 7f10f1773640 1 -- 172.21.15.43:0/344192034 >> 172.21.15.43:0/344192034 conn(0x7f10ec0cddd0 msgr2=0x7f10ec15b880 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-09T14:51:37.097 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:37.094+0000 7f10f1773640 1 -- 172.21.15.43:0/344192034 shutdown_connections 2023-08-09T14:51:37.097 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:37.094+0000 7f10f1773640 1 -- 172.21.15.43:0/344192034 wait complete. 2023-08-09T14:51:37.113 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Ended test tasks.cephfs.test_snap_schedules.TestSnapSchedules.test_snap_schedule_for_number_of_snaps_retention' 2023-08-09T14:51:37.335 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:37.333+0000 7fda7e7eb640 1 Processor -- start 2023-08-09T14:51:37.335 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:37.334+0000 7fda7e7eb640 1 -- start start 2023-08-09T14:51:37.337 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:37.335+0000 7fda7e7eb640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fda7815c950 0x7fda780cecb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T14:51:37.337 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:37.335+0000 7fda7e7eb640 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7fda780d0e80 0x7fda780b6150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T14:51:37.337 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:37.335+0000 7fda7e7eb640 1 -- --> v1:172.21.15.43:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fda780cf650 con 0x7fda780cf1f0 2023-08-09T14:51:37.337 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:37.335+0000 7fda7e7eb640 1 -- --> v2:172.21.15.203:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fda780d1aa0 con 0x7fda780d0e80 2023-08-09T14:51:37.337 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:37.335+0000 7fda7e7eb640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fda780b56b0 con 0x7fda7815c950 2023-08-09T14:51:37.339 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:37.336+0000 7fda7cfe8640 1 --1- >> v1:172.21.15.43:6789/0 conn(0x7fda780cf1f0 0x7fda780d0770 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.43:6789/0 says I am v1:172.21.15.43:46618/0 (socket says 172.21.15.43:46618) 2023-08-09T14:51:37.339 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:37.336+0000 7fda7cfe8640 1 -- 172.21.15.43:0/1458798996 learned_addr learned my addr 172.21.15.43:0/1458798996 (peer_addr_for_me v1:172.21.15.43:0/0) 2023-08-09T14:51:37.339 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:37.336+0000 7fda7d7e9640 1 --2- 172.21.15.43:0/1458798996 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fda7815c950 0x7fda780cecb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T14:51:37.340 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:37.336+0000 7fda7dfea640 1 --2- 172.21.15.43:0/1458798996 >> v2:172.21.15.203:3300/0 conn(0x7fda780d0e80 0x7fda780b6150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T14:51:37.340 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:37.336+0000 7fda6ffff640 1 -- 172.21.15.43:0/1458798996 <== mon.0 v1:172.21.15.43:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3962654411 0 0) 0x7fda780cf650 con 0x7fda780cf1f0 2023-08-09T14:51:37.340 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:37.336+0000 7fda6ffff640 1 -- 172.21.15.43:0/1458798996 --> v1:172.21.15.43:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fda64003520 con 0x7fda780cf1f0 2023-08-09T14:51:37.340 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:37.337+0000 7fda6ffff640 1 -- 172.21.15.43:0/1458798996 <== mon.0 v1:172.21.15.43:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3823562075 0 0) 0x7fda64003520 con 0x7fda780cf1f0 2023-08-09T14:51:37.340 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:37.337+0000 7fda6ffff640 1 -- 172.21.15.43:0/1458798996 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fda7815c950 msgr2=0x7fda780cecb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-09T14:51:37.340 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:37.337+0000 7fda6ffff640 1 --2- 172.21.15.43:0/1458798996 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fda7815c950 0x7fda780cecb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T14:51:37.341 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:37.337+0000 7fda6ffff640 1 -- 172.21.15.43:0/1458798996 >> v2:172.21.15.203:3300/0 conn(0x7fda780d0e80 msgr2=0x7fda780b6150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-09T14:51:37.341 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:37.337+0000 7fda6ffff640 1 --2- 172.21.15.43:0/1458798996 >> v2:172.21.15.203:3300/0 conn(0x7fda780d0e80 0x7fda780b6150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T14:51:37.341 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:37.337+0000 7fda6ffff640 1 -- 172.21.15.43:0/1458798996 --> v1:172.21.15.43:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fda780d14e0 con 0x7fda780cf1f0 2023-08-09T14:51:37.341 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:37.337+0000 7fda6ffff640 1 -- 172.21.15.43:0/1458798996 <== mon.0 v1:172.21.15.43:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1285150740 0 0) 0x7fda680017f0 con 0x7fda780cf1f0 2023-08-09T14:51:37.341 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:37.337+0000 7fda6ffff640 1 -- 172.21.15.43:0/1458798996 >> v1:172.21.15.43:6789/0 conn(0x7fda780cf1f0 legacy=0x7fda780d0770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T14:51:37.341 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:37.337+0000 7fda6ffff640 1 --2- 172.21.15.43:0/1458798996 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fda64003970 0x7fda64003d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T14:51:37.341 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:37.337+0000 7fda6ffff640 1 --2- 172.21.15.43:0/1458798996 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fda640047c0 0x7fda64004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T14:51:37.341 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:37.337+0000 7fda6ffff640 1 --2- 172.21.15.43:0/1458798996 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fda7815c950 0x7fda64005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T14:51:37.342 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:37.337+0000 7fda6ffff640 1 -- 172.21.15.43:0/1458798996 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fda780d1aa0 con 0x7fda640047c0 2023-08-09T14:51:37.342 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:37.337+0000 7fda6ffff640 1 -- 172.21.15.43:0/1458798996 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fda780b56b0 con 0x7fda64003970 2023-08-09T14:51:37.342 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:37.337+0000 7fda6ffff640 1 -- 172.21.15.43:0/1458798996 --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fda6400b980 con 0x7fda7815c950 2023-08-09T14:51:37.342 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:37.337+0000 7fda7cfe8640 1 --2- 172.21.15.43:0/1458798996 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fda640047c0 0x7fda64004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T14:51:37.342 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:37.338+0000 7fda7d7e9640 1 --2- 172.21.15.43:0/1458798996 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fda64003970 0x7fda64003d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T14:51:37.342 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:37.338+0000 7fda7dfea640 1 --2- 172.21.15.43:0/1458798996 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fda7815c950 0x7fda64005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T14:51:37.342 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:37.338+0000 7fda7cfe8640 1 -- 172.21.15.43:0/1458798996 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fda7815c950 msgr2=0x7fda64005180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-09T14:51:37.343 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:37.338+0000 7fda7cfe8640 1 --2- 172.21.15.43:0/1458798996 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fda7815c950 0x7fda64005180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T14:51:37.343 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:37.338+0000 7fda7cfe8640 1 -- 172.21.15.43:0/1458798996 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fda64003970 msgr2=0x7fda64003d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-09T14:51:37.343 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:37.338+0000 7fda7cfe8640 1 --2- 172.21.15.43:0/1458798996 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fda64003970 0x7fda64003d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T14:51:37.343 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:37.338+0000 7fda7cfe8640 1 -- 172.21.15.43:0/1458798996 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fda64003520 con 0x7fda640047c0 2023-08-09T14:51:37.343 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:37.338+0000 7fda7dfea640 1 --2- 172.21.15.43:0/1458798996 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fda7815c950 0x7fda64005180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-08-09T14:51:37.345 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:37.339+0000 7fda7cfe8640 1 --2- 172.21.15.43:0/1458798996 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fda640047c0 0x7fda64004c40 secure :-1 s=READY pgs=1814 cs=0 l=1 rev1=1 crypto rx=0x7fda6800c1c0 tx=0x7fda640057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9b33421397a3c086 server_cookie=0 in_seq=0 out_seq=0 2023-08-09T14:51:37.345 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:37.339+0000 7fda6ffff640 1 -- 172.21.15.43:0/1458798996 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fda6802cae0 con 0x7fda640047c0 2023-08-09T14:51:37.345 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:37.339+0000 7fda6ffff640 1 -- 172.21.15.43:0/1458798996 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fda6802cc40 con 0x7fda640047c0 2023-08-09T14:51:37.345 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:37.339+0000 7fda6ffff640 1 -- 172.21.15.43:0/1458798996 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fda68007070 con 0x7fda640047c0 2023-08-09T14:51:37.345 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:37.340+0000 7fda7e7eb640 1 -- 172.21.15.43:0/1458798996 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fda640047c0 msgr2=0x7fda64004c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T14:51:37.345 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:37.340+0000 7fda7e7eb640 1 --2- 172.21.15.43:0/1458798996 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fda640047c0 0x7fda64004c40 secure :-1 s=READY pgs=1814 cs=0 l=1 rev1=1 crypto rx=0x7fda6800c1c0 tx=0x7fda640057f0 comp rx=0 tx=0).stop 2023-08-09T14:51:37.346 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:37.340+0000 7fda7dfea640 1 -- 172.21.15.43:0/1458798996 reap_dead start 2023-08-09T14:51:37.346 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:37.340+0000 7fda7e7eb640 1 -- 172.21.15.43:0/1458798996 shutdown_connections 2023-08-09T14:51:37.346 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:37.340+0000 7fda7e7eb640 1 -- 172.21.15.43:0/1458798996 >> 172.21.15.43:0/1458798996 conn(0x7fda7806a2e0 msgr2=0x7fda7806a6f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-09T14:51:37.346 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:37.340+0000 7fda7e7eb640 1 -- 172.21.15.43:0/1458798996 shutdown_connections 2023-08-09T14:51:37.346 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:37.340+0000 7fda7e7eb640 1 -- 172.21.15.43:0/1458798996 wait complete. 2023-08-09T14:51:37.346 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:37.342+0000 7fda7e7eb640 1 Processor -- start 2023-08-09T14:51:37.346 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:37.342+0000 7fda7e7eb640 1 -- start start 2023-08-09T14:51:37.346 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:37.343+0000 7fda7e7eb640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fda640047c0 0x7fda780c92e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T14:51:37.347 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:37.343+0000 7fda7e7eb640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fda7815f870 0x7fda781e5bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T14:51:37.347 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:37.343+0000 7fda7e7eb640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fda780cabd0 0x7fda781490f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T14:51:37.347 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:37.343+0000 7fda7e7eb640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fda780ca7b0 con 0x7fda780cabd0 2023-08-09T14:51:37.347 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:37.343+0000 7fda7e7eb640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fda780c9b30 con 0x7fda7815f870 2023-08-09T14:51:37.347 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:37.343+0000 7fda7e7eb640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fda780c9e10 con 0x7fda640047c0 2023-08-09T14:51:37.347 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:37.343+0000 7fda7dfea640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fda780cabd0 0x7fda781490f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T14:51:37.347 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:37.343+0000 7fda7dfea640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fda780cabd0 0x7fda781490f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:52378/0 (socket says 172.21.15.43:52378) 2023-08-09T14:51:37.347 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:37.343+0000 7fda7dfea640 1 -- 172.21.15.43:0/3891018058 learned_addr learned my addr 172.21.15.43:0/3891018058 (peer_addr_for_me v2:172.21.15.43:0/0) 2023-08-09T14:51:37.349 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:37.344+0000 7fda7d7e9640 1 --2- 172.21.15.43:0/3891018058 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fda640047c0 0x7fda780c92e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T14:51:37.349 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:37.344+0000 7fda7dfea640 1 -- 172.21.15.43:0/3891018058 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fda640047c0 msgr2=0x7fda780c92e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T14:51:37.349 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:37.344+0000 7fda7dfea640 1 --2- 172.21.15.43:0/3891018058 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fda640047c0 0x7fda780c92e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T14:51:37.349 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:37.344+0000 7fda7dfea640 1 -- 172.21.15.43:0/3891018058 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fda7815f870 msgr2=0x7fda781e5bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T14:51:37.349 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:37.344+0000 7fda7dfea640 1 --2- 172.21.15.43:0/3891018058 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fda7815f870 0x7fda781e5bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T14:51:37.350 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:37.344+0000 7fda7dfea640 1 -- 172.21.15.43:0/3891018058 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fda780ca0f0 con 0x7fda780cabd0 2023-08-09T14:51:37.350 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:37.344+0000 7fda7dfea640 1 --2- 172.21.15.43:0/3891018058 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fda780cabd0 0x7fda781490f0 secure :-1 s=READY pgs=1815 cs=0 l=1 rev1=1 crypto rx=0x7fda780b5180 tx=0x7fda7000f4a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-09T14:51:37.350 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:37.344+0000 7fda6dffb640 1 -- 172.21.15.43:0/3891018058 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fda70017400 con 0x7fda780cabd0 2023-08-09T14:51:37.350 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:37.344+0000 7fda6dffb640 1 -- 172.21.15.43:0/3891018058 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fda70011040 con 0x7fda780cabd0 2023-08-09T14:51:37.350 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:37.344+0000 7fda6dffb640 1 -- 172.21.15.43:0/3891018058 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fda70017780 con 0x7fda780cabd0 2023-08-09T14:51:37.350 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:37.344+0000 7fda7e7eb640 1 -- 172.21.15.43:0/3891018058 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fda780ca3d0 con 0x7fda780cabd0 2023-08-09T14:51:37.350 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:37.344+0000 7fda7e7eb640 1 -- 172.21.15.43:0/3891018058 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fda781e6110 con 0x7fda780cabd0 2023-08-09T14:51:37.350 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:37.346+0000 7fda7e7eb640 1 -- 172.21.15.43:0/3891018058 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fda780ca8f0 con 0x7fda780cabd0 2023-08-09T14:51:37.351 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:37.347+0000 7fda6dffb640 1 -- 172.21.15.43:0/3891018058 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 79) v1 ==== 88443+0+0 (secure 0 0 0) 0x7fda700203b0 con 0x7fda780cabd0 2023-08-09T14:51:37.351 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:37.348+0000 7fda6dffb640 1 --2- 172.21.15.43:0/3891018058 >> [v2:172.21.15.203:6832/2566816332,v1:172.21.15.203:6833/2566816332] conn(0x7fda5c06a1a0 0x7fda5c06c660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T14:51:37.351 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:37.348+0000 7fda6dffb640 1 -- 172.21.15.43:0/3891018058 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(164..164 src has 1..164) v4 ==== 6348+0+0 (secure 0 0 0) 0x7fda70088c70 con 0x7fda780cabd0 2023-08-09T14:51:37.351 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:37.348+0000 7fda7d7e9640 1 --2- 172.21.15.43:0/3891018058 >> [v2:172.21.15.203:6832/2566816332,v1:172.21.15.203:6833/2566816332] conn(0x7fda5c06a1a0 0x7fda5c06c660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T14:51:37.352 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:37.349+0000 7fda7d7e9640 1 --2- 172.21.15.43:0/3891018058 >> [v2:172.21.15.203:6832/2566816332,v1:172.21.15.203:6833/2566816332] conn(0x7fda5c06a1a0 0x7fda5c06c660 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7fda740097f0 tx=0x7fda74008040 comp rx=0 tx=0).ready entity=mgr.7765 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-09T14:51:37.353 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:37.352+0000 7fda6dffb640 1 -- 172.21.15.43:0/3891018058 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178066 (secure 0 0 0) 0x7fda780ca8f0 con 0x7fda780cabd0 2023-08-09T14:51:37.631 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:37.630+0000 7fda7e7eb640 1 -- 172.21.15.43:0/3891018058 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_snap_schedules.TestSnapSchedules.test_snap_schedule_for_number_of_snaps_retention"]} v 0) v1 -- 0x7fda780ca8f0 con 0x7fda780cabd0 2023-08-09T14:51:37.672 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:37.670+0000 7fda6dffb640 1 -- 172.21.15.43:0/3891018058 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_snap_schedules.TestSnapSchedules.test_snap_schedule_for_number_of_snaps_retention"]}]=0 v2551) v1 ==== 178+0+0 (secure 0 0 0) 0x7fda780ca8f0 con 0x7fda780cabd0 2023-08-09T14:51:37.675 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:37.673+0000 7fda7e7eb640 1 -- 172.21.15.43:0/3891018058 >> [v2:172.21.15.203:6832/2566816332,v1:172.21.15.203:6833/2566816332] conn(0x7fda5c06a1a0 msgr2=0x7fda5c06c660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T14:51:37.675 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:37.673+0000 7fda7e7eb640 1 --2- 172.21.15.43:0/3891018058 >> [v2:172.21.15.203:6832/2566816332,v1:172.21.15.203:6833/2566816332] conn(0x7fda5c06a1a0 0x7fda5c06c660 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7fda740097f0 tx=0x7fda74008040 comp rx=0 tx=0).stop 2023-08-09T14:51:37.675 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:37.673+0000 7fda7e7eb640 1 -- 172.21.15.43:0/3891018058 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fda780cabd0 msgr2=0x7fda781490f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T14:51:37.675 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:37.673+0000 7fda7e7eb640 1 --2- 172.21.15.43:0/3891018058 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fda780cabd0 0x7fda781490f0 secure :-1 s=READY pgs=1815 cs=0 l=1 rev1=1 crypto rx=0x7fda780b5180 tx=0x7fda7000f4a0 comp rx=0 tx=0).stop 2023-08-09T14:51:37.675 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:37.673+0000 7fda7e7eb640 1 -- 172.21.15.43:0/3891018058 shutdown_connections 2023-08-09T14:51:37.676 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:37.673+0000 7fda7e7eb640 1 --2- 172.21.15.43:0/3891018058 >> [v2:172.21.15.203:6832/2566816332,v1:172.21.15.203:6833/2566816332] conn(0x7fda5c06a1a0 0x7fda5c06c660 unknown :-1 s=CLOSED pgs=39 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T14:51:37.676 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:37.673+0000 7fda7e7eb640 1 --2- 172.21.15.43:0/3891018058 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fda780cabd0 0x7fda781490f0 unknown :-1 s=CLOSED pgs=1815 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T14:51:37.676 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:37.673+0000 7fda7e7eb640 1 --2- 172.21.15.43:0/3891018058 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fda7815f870 0x7fda781e5bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T14:51:37.676 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:37.673+0000 7fda7e7eb640 1 --2- 172.21.15.43:0/3891018058 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fda640047c0 0x7fda780c92e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T14:51:37.676 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:37.673+0000 7fda7e7eb640 1 -- 172.21.15.43:0/3891018058 >> 172.21.15.43:0/3891018058 conn(0x7fda7806a2e0 msgr2=0x7fda7815b9e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-09T14:51:37.677 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:37.674+0000 7fda7e7eb640 1 -- 172.21.15.43:0/3891018058 shutdown_connections 2023-08-09T14:51:37.677 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:37.674+0000 7fda7e7eb640 1 -- 172.21.15.43:0/3891018058 wait complete. 2023-08-09T14:51:37.696 INFO:tasks.cephfs_test_runner:test_snap_schedule_for_number_of_snaps_retention (tasks.cephfs.test_snap_schedules.TestSnapSchedules) ... ok 2023-08-09T14:51:37.697 INFO:tasks.cephfs_test_runner:Starting test: test_snap_schedule_list_post_schedule_remove (tasks.cephfs.test_snap_schedules.TestSnapSchedules) Test listing snap schedules post removal of a schedule 2023-08-09T14:51:37.697 INFO:tasks.cephfs_test_runner:test_snap_schedule_list_post_schedule_remove (tasks.cephfs.test_snap_schedules.TestSnapSchedules) 2023-08-09T14:51:37.697 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Starting test tasks.cephfs.test_snap_schedules.TestSnapSchedules.test_snap_schedule_list_post_schedule_remove' 2023-08-09T14:51:37.929 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:37.927+0000 7f13b2e14640 1 Processor -- start 2023-08-09T14:51:37.929 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:37.927+0000 7f13b2e14640 1 -- start start 2023-08-09T14:51:37.931 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:37.929+0000 7f13b2e14640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f13ac15c630 0x7f13ac0abef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T14:51:37.931 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:37.929+0000 7f13b2e14640 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7f13ac0cdc60 0x7f13ac15ca10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T14:51:37.931 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:37.929+0000 7f13b2e14640 1 -- --> v1:172.21.15.203:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f13ac0cca90 con 0x7f13ac0ccf20 2023-08-09T14:51:37.931 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:37.929+0000 7f13b2e14640 1 -- --> v2:172.21.15.43:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f13ac0cbc70 con 0x7f13ac15c630 2023-08-09T14:51:37.932 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:37.929+0000 7f13b2e14640 1 -- --> v2:172.21.15.203:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f13ac0cbe80 con 0x7f13ac0cdc60 2023-08-09T14:51:37.932 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:37.930+0000 7f13b0bb0640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f13ac15c630 0x7f13ac0abef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T14:51:37.932 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:37.930+0000 7f13b0bb0640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f13ac15c630 0x7f13ac0abef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:52388/0 (socket says 172.21.15.43:52388) 2023-08-09T14:51:37.932 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:37.930+0000 7f13b0bb0640 1 -- 172.21.15.43:0/100655761 learned_addr learned my addr 172.21.15.43:0/100655761 (peer_addr_for_me v2:172.21.15.43:0/0) 2023-08-09T14:51:37.932 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:37.930+0000 7f13b0bb0640 1 -- 172.21.15.43:0/100655761 >> v2:172.21.15.203:3300/0 conn(0x7f13ac0cdc60 msgr2=0x7f13ac15ca10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-09T14:51:37.933 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:37.930+0000 7f13b0bb0640 1 --2- 172.21.15.43:0/100655761 >> v2:172.21.15.203:3300/0 conn(0x7f13ac0cdc60 0x7f13ac15ca10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T14:51:37.933 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:37.930+0000 7f13b0bb0640 1 -- 172.21.15.43:0/100655761 >> v1:172.21.15.203:6789/0 conn(0x7f13ac0ccf20 legacy=0x7f13ac0cd550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-09T14:51:37.933 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:37.930+0000 7f13b0bb0640 1 -- 172.21.15.43:0/100655761 --> v2:172.21.15.43:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f13ac0cc160 con 0x7f13ac15c630 2023-08-09T14:51:37.933 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:37.931+0000 7f13b0bb0640 1 --2- 172.21.15.43:0/100655761 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f13ac15c630 0x7f13ac0abef0 secure :-1 s=READY pgs=1816 cs=0 l=1 rev1=1 crypto rx=0x7f13ac0ca9b0 tx=0x7f13a002b910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=14a256f4896c96fe server_cookie=0 in_seq=0 out_seq=0 2023-08-09T14:51:37.934 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:37.932+0000 7f13ab7fe640 1 -- 172.21.15.43:0/100655761 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f13a0031870 con 0x7f13ac15c630 2023-08-09T14:51:37.934 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:37.932+0000 7f13ab7fe640 1 -- 172.21.15.43:0/100655761 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f13a00319d0 con 0x7f13ac15c630 2023-08-09T14:51:37.935 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:37.932+0000 7f13ab7fe640 1 -- 172.21.15.43:0/100655761 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f13a0031d60 con 0x7f13ac15c630 2023-08-09T14:51:37.935 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:37.932+0000 7f13b2e14640 1 -- 172.21.15.43:0/100655761 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f13ac15c630 msgr2=0x7f13ac0abef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T14:51:37.935 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:37.932+0000 7f13b2e14640 1 --2- 172.21.15.43:0/100655761 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f13ac15c630 0x7f13ac0abef0 secure :-1 s=READY pgs=1816 cs=0 l=1 rev1=1 crypto rx=0x7f13ac0ca9b0 tx=0x7f13a002b910 comp rx=0 tx=0).stop 2023-08-09T14:51:37.935 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:37.933+0000 7f13b2e14640 1 -- 172.21.15.43:0/100655761 shutdown_connections 2023-08-09T14:51:37.935 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:37.933+0000 7f13b2e14640 1 --2- 172.21.15.43:0/100655761 >> v2:172.21.15.203:3300/0 conn(0x7f13ac0cdc60 0x7f13ac15ca10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T14:51:37.935 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:37.933+0000 7f13b2e14640 1 --2- 172.21.15.43:0/100655761 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f13ac15c630 0x7f13ac0abef0 unknown :-1 s=CLOSED pgs=1816 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T14:51:37.935 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:37.933+0000 7f13b2e14640 1 -- 172.21.15.43:0/100655761 >> 172.21.15.43:0/100655761 conn(0x7f13ac0ce0a0 msgr2=0x7f13ac0ce480 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-09T14:51:37.936 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:37.933+0000 7f13b2e14640 1 -- 172.21.15.43:0/100655761 shutdown_connections 2023-08-09T14:51:37.936 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:37.933+0000 7f13b2e14640 1 -- 172.21.15.43:0/100655761 wait complete. 2023-08-09T14:51:37.936 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:37.935+0000 7f13b2e14640 1 Processor -- start 2023-08-09T14:51:37.936 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:37.935+0000 7f13b2e14640 1 -- start start 2023-08-09T14:51:37.938 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:37.936+0000 7f13b2e14640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f13ac0ccf20 0x7f13ac0aeb30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T14:51:37.938 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:37.936+0000 7f13b2e14640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f13ac0cdc60 0x7f13ac06f940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T14:51:37.938 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:37.936+0000 7f13b2e14640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f13ac15c630 0x7f13ac0c6640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T14:51:37.938 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:37.936+0000 7f13b2e14640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f13ac14a6f0 con 0x7f13ac0ccf20 2023-08-09T14:51:37.938 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:37.936+0000 7f13b2e14640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f13ac070450 con 0x7f13ac0cdc60 2023-08-09T14:51:37.938 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:37.936+0000 7f13b2e14640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f13ac070730 con 0x7f13ac15c630 2023-08-09T14:51:37.940 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:37.937+0000 7f13b0bb0640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f13ac0ccf20 0x7f13ac0aeb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T14:51:37.940 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:37.937+0000 7f13b0bb0640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f13ac0ccf20 0x7f13ac0aeb30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:52394/0 (socket says 172.21.15.43:52394) 2023-08-09T14:51:37.940 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:37.937+0000 7f13b0bb0640 1 -- 172.21.15.43:0/1977281320 learned_addr learned my addr 172.21.15.43:0/1977281320 (peer_addr_for_me v2:172.21.15.43:0/0) 2023-08-09T14:51:37.940 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:37.937+0000 7f13abfff640 1 --2- 172.21.15.43:0/1977281320 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f13ac0cdc60 0x7f13ac06f940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T14:51:37.940 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:37.937+0000 7f13b13b1640 1 --2- 172.21.15.43:0/1977281320 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f13ac15c630 0x7f13ac0c6640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T14:51:37.941 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:37.937+0000 7f13b0bb0640 1 -- 172.21.15.43:0/1977281320 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f13ac15c630 msgr2=0x7f13ac0c6640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T14:51:37.941 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:37.937+0000 7f13b0bb0640 1 --2- 172.21.15.43:0/1977281320 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f13ac15c630 0x7f13ac0c6640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T14:51:37.941 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:37.937+0000 7f13b0bb0640 1 -- 172.21.15.43:0/1977281320 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f13ac0cdc60 msgr2=0x7f13ac06f940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T14:51:37.941 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:37.937+0000 7f13b0bb0640 1 --2- 172.21.15.43:0/1977281320 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f13ac0cdc60 0x7f13ac06f940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T14:51:37.941 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:37.937+0000 7f13b0bb0640 1 -- 172.21.15.43:0/1977281320 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f13a00094d0 con 0x7f13ac0ccf20 2023-08-09T14:51:37.941 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:37.937+0000 7f13b13b1640 1 --2- 172.21.15.43:0/1977281320 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f13ac15c630 0x7f13ac0c6640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-08-09T14:51:37.941 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:37.938+0000 7f13b0bb0640 1 --2- 172.21.15.43:0/1977281320 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f13ac0ccf20 0x7f13ac0aeb30 secure :-1 s=READY pgs=1817 cs=0 l=1 rev1=1 crypto rx=0x7f13a0000c00 tx=0x7f13a00046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-09T14:51:37.942 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:37.938+0000 7f13a97fa640 1 -- 172.21.15.43:0/1977281320 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f13a0031840 con 0x7f13ac0ccf20 2023-08-09T14:51:37.942 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:37.938+0000 7f13a97fa640 1 -- 172.21.15.43:0/1977281320 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f13a00443f0 con 0x7f13ac0ccf20 2023-08-09T14:51:37.942 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:37.938+0000 7f13a97fa640 1 -- 172.21.15.43:0/1977281320 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f13a00447a0 con 0x7f13ac0ccf20 2023-08-09T14:51:37.942 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:37.938+0000 7f13b2e14640 1 -- 172.21.15.43:0/1977281320 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f13ac070a10 con 0x7f13ac0ccf20 2023-08-09T14:51:37.942 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:37.938+0000 7f13b2e14640 1 -- 172.21.15.43:0/1977281320 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f13ac0af070 con 0x7f13ac0ccf20 2023-08-09T14:51:37.943 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:37.939+0000 7f13b2e14640 1 -- 172.21.15.43:0/1977281320 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f13ac15ca10 con 0x7f13ac0ccf20 2023-08-09T14:51:37.943 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:37.940+0000 7f13a97fa640 1 -- 172.21.15.43:0/1977281320 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 79) v1 ==== 88443+0+0 (secure 0 0 0) 0x7f13a004d450 con 0x7f13ac0ccf20 2023-08-09T14:51:37.943 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:37.941+0000 7f13a97fa640 1 --2- 172.21.15.43:0/1977281320 >> [v2:172.21.15.203:6832/2566816332,v1:172.21.15.203:6833/2566816332] conn(0x7f139006a170 0x7f139006c630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T14:51:37.943 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:37.941+0000 7f13a97fa640 1 -- 172.21.15.43:0/1977281320 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(164..164 src has 1..164) v4 ==== 6348+0+0 (secure 0 0 0) 0x7f13a002d040 con 0x7f13ac0ccf20 2023-08-09T14:51:37.944 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:37.942+0000 7f13abfff640 1 --2- 172.21.15.43:0/1977281320 >> [v2:172.21.15.203:6832/2566816332,v1:172.21.15.203:6833/2566816332] conn(0x7f139006a170 0x7f139006c630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T14:51:37.944 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:37.943+0000 7f13abfff640 1 --2- 172.21.15.43:0/1977281320 >> [v2:172.21.15.203:6832/2566816332,v1:172.21.15.203:6833/2566816332] conn(0x7f139006a170 0x7f139006c630 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7f1398007fd0 tx=0x7f1398009040 comp rx=0 tx=0).ready entity=mgr.7765 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-09T14:51:37.947 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:37.945+0000 7f13a97fa640 1 -- 172.21.15.43:0/1977281320 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178066 (secure 0 0 0) 0x7f13a0076460 con 0x7f13ac0ccf20 2023-08-09T14:51:38.216 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:38.214+0000 7f13b2e14640 1 -- 172.21.15.43:0/1977281320 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_snap_schedules.TestSnapSchedules.test_snap_schedule_list_post_schedule_remove"]} v 0) v1 -- 0x7f13ac0739f0 con 0x7f13ac0ccf20 2023-08-09T14:51:38.675 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:38.673+0000 7f13a97fa640 1 -- 172.21.15.43:0/1977281320 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_snap_schedules.TestSnapSchedules.test_snap_schedule_list_post_schedule_remove"]}]=0 v2552) v1 ==== 177+0+0 (secure 0 0 0) 0x7f13a00350f0 con 0x7f13ac0ccf20 2023-08-09T14:51:38.678 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:38.676+0000 7f13b2e14640 1 -- 172.21.15.43:0/1977281320 >> [v2:172.21.15.203:6832/2566816332,v1:172.21.15.203:6833/2566816332] conn(0x7f139006a170 msgr2=0x7f139006c630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T14:51:38.678 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:38.676+0000 7f13b2e14640 1 --2- 172.21.15.43:0/1977281320 >> [v2:172.21.15.203:6832/2566816332,v1:172.21.15.203:6833/2566816332] conn(0x7f139006a170 0x7f139006c630 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7f1398007fd0 tx=0x7f1398009040 comp rx=0 tx=0).stop 2023-08-09T14:51:38.678 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:38.676+0000 7f13b2e14640 1 -- 172.21.15.43:0/1977281320 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f13ac0ccf20 msgr2=0x7f13ac0aeb30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T14:51:38.679 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:38.676+0000 7f13b2e14640 1 --2- 172.21.15.43:0/1977281320 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f13ac0ccf20 0x7f13ac0aeb30 secure :-1 s=READY pgs=1817 cs=0 l=1 rev1=1 crypto rx=0x7f13a0000c00 tx=0x7f13a00046c0 comp rx=0 tx=0).stop 2023-08-09T14:51:38.679 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:38.676+0000 7f13b2e14640 1 -- 172.21.15.43:0/1977281320 shutdown_connections 2023-08-09T14:51:38.679 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:38.676+0000 7f13b2e14640 1 --2- 172.21.15.43:0/1977281320 >> [v2:172.21.15.203:6832/2566816332,v1:172.21.15.203:6833/2566816332] conn(0x7f139006a170 0x7f139006c630 unknown :-1 s=CLOSED pgs=40 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T14:51:38.679 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:38.676+0000 7f13b2e14640 1 --2- 172.21.15.43:0/1977281320 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f13ac15c630 0x7f13ac0c6640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T14:51:38.679 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:38.676+0000 7f13b2e14640 1 --2- 172.21.15.43:0/1977281320 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f13ac0cdc60 0x7f13ac06f940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T14:51:38.679 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:38.676+0000 7f13b2e14640 1 --2- 172.21.15.43:0/1977281320 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f13ac0ccf20 0x7f13ac0aeb30 unknown :-1 s=CLOSED pgs=1817 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T14:51:38.679 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:38.676+0000 7f13b2e14640 1 -- 172.21.15.43:0/1977281320 >> 172.21.15.43:0/1977281320 conn(0x7f13ac0ce0a0 msgr2=0x7f13ac0c8890 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-09T14:51:38.680 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:38.676+0000 7f13b2e14640 1 -- 172.21.15.43:0/1977281320 shutdown_connections 2023-08-09T14:51:38.680 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:38.676+0000 7f13b2e14640 1 -- 172.21.15.43:0/1977281320 wait complete. 2023-08-09T14:51:38.700 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-08-09T14:51:38.932 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:38.930+0000 7fc03227a640 1 Processor -- start 2023-08-09T14:51:38.932 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:38.930+0000 7fc03227a640 1 -- start start 2023-08-09T14:51:38.934 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:38.932+0000 7fc03227a640 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7fc02c15c630 0x7fc02c0abef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T14:51:38.934 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:38.932+0000 7fc03227a640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7fc02c0ccf20 0x7fc02c0cd550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T14:51:38.934 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:38.932+0000 7fc03227a640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fc02c0cda90 0x7fc02c0c2f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T14:51:38.934 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:38.932+0000 7fc03227a640 1 -- --> v2:172.21.15.43:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc02c0cbc70 con 0x7fc02c0ccf20 2023-08-09T14:51:38.934 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:38.932+0000 7fc03227a640 1 -- --> v2:172.21.15.203:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc02c0cdee0 con 0x7fc02c15c630 2023-08-09T14:51:38.935 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:38.932+0000 7fc03227a640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc02c0cca90 con 0x7fc02c0cda90 2023-08-09T14:51:38.935 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:38.933+0000 7fc0237fe640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7fc02c0ccf20 0x7fc02c0cd550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T14:51:38.936 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:38.933+0000 7fc0237fe640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7fc02c0ccf20 0x7fc02c0cd550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:52402/0 (socket says 172.21.15.43:52402) 2023-08-09T14:51:38.936 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:38.933+0000 7fc0237fe640 1 -- 172.21.15.43:0/3661794047 learned_addr learned my addr 172.21.15.43:0/3661794047 (peer_addr_for_me v2:172.21.15.43:0/0) 2023-08-09T14:51:38.936 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:38.933+0000 7fc030817640 1 --2- 172.21.15.43:0/3661794047 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fc02c0cda90 0x7fc02c0c2f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T14:51:38.936 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:38.933+0000 7fc02bfff640 1 --2- 172.21.15.43:0/3661794047 >> v2:172.21.15.203:3300/0 conn(0x7fc02c15c630 0x7fc02c0abef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T14:51:38.936 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:38.933+0000 7fc0237fe640 1 -- 172.21.15.43:0/3661794047 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fc02c0cda90 msgr2=0x7fc02c0c2f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-09T14:51:38.937 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:38.933+0000 7fc0237fe640 1 --2- 172.21.15.43:0/3661794047 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fc02c0cda90 0x7fc02c0c2f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T14:51:38.937 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:38.933+0000 7fc0237fe640 1 -- 172.21.15.43:0/3661794047 >> v2:172.21.15.203:3300/0 conn(0x7fc02c15c630 msgr2=0x7fc02c0abef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-09T14:51:38.937 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:38.933+0000 7fc0237fe640 1 --2- 172.21.15.43:0/3661794047 >> v2:172.21.15.203:3300/0 conn(0x7fc02c15c630 0x7fc02c0abef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T14:51:38.937 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:38.933+0000 7fc0237fe640 1 -- 172.21.15.43:0/3661794047 --> v2:172.21.15.43:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc02c0cbe80 con 0x7fc02c0ccf20 2023-08-09T14:51:38.938 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:38.934+0000 7fc0237fe640 1 --2- 172.21.15.43:0/3661794047 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc02c0ccf20 0x7fc02c0cd550 secure :-1 s=READY pgs=1818 cs=0 l=1 rev1=1 crypto rx=0x7fc02c0cbbb0 tx=0x7fc01402bee0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=445be966cb01f64 server_cookie=0 in_seq=0 out_seq=0 2023-08-09T14:51:38.938 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:38.935+0000 7fc02b7fe640 1 -- 172.21.15.43:0/3661794047 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc01402ca80 con 0x7fc02c0ccf20 2023-08-09T14:51:38.938 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:38.935+0000 7fc02b7fe640 1 -- 172.21.15.43:0/3661794047 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc01402cbe0 con 0x7fc02c0ccf20 2023-08-09T14:51:38.938 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:38.935+0000 7fc02b7fe640 1 -- 172.21.15.43:0/3661794047 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc0140343f0 con 0x7fc02c0ccf20 2023-08-09T14:51:38.945 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:38.935+0000 7fc03227a640 1 -- 172.21.15.43:0/3661794047 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc02c0ccf20 msgr2=0x7fc02c0cd550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T14:51:38.945 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:38.935+0000 7fc03227a640 1 --2- 172.21.15.43:0/3661794047 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc02c0ccf20 0x7fc02c0cd550 secure :-1 s=READY pgs=1818 cs=0 l=1 rev1=1 crypto rx=0x7fc02c0cbbb0 tx=0x7fc01402bee0 comp rx=0 tx=0).stop 2023-08-09T14:51:38.946 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:38.936+0000 7fc03227a640 1 -- 172.21.15.43:0/3661794047 shutdown_connections 2023-08-09T14:51:38.946 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:38.936+0000 7fc03227a640 1 --2- 172.21.15.43:0/3661794047 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc02c0ccf20 0x7fc02c0cd550 unknown :-1 s=CLOSED pgs=1818 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T14:51:38.946 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:38.936+0000 7fc03227a640 1 --2- 172.21.15.43:0/3661794047 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fc02c0cda90 0x7fc02c0c2f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T14:51:38.947 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:38.936+0000 7fc03227a640 1 --2- 172.21.15.43:0/3661794047 >> v2:172.21.15.203:3300/0 conn(0x7fc02c15c630 0x7fc02c0abef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T14:51:38.947 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:38.936+0000 7fc03227a640 1 -- 172.21.15.43:0/3661794047 >> 172.21.15.43:0/3661794047 conn(0x7fc02c0ce0a0 msgr2=0x7fc02c0ce480 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-09T14:51:38.947 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:38.936+0000 7fc03227a640 1 -- 172.21.15.43:0/3661794047 shutdown_connections 2023-08-09T14:51:38.947 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:38.936+0000 7fc03227a640 1 -- 172.21.15.43:0/3661794047 wait complete. 2023-08-09T14:51:38.948 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:38.937+0000 7fc03227a640 1 Processor -- start 2023-08-09T14:51:38.948 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:38.938+0000 7fc03227a640 1 -- start start 2023-08-09T14:51:38.948 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:38.939+0000 7fc03227a640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc02c0ccf20 0x7fc02c1f5dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T14:51:38.948 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:38.939+0000 7fc03227a640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fc02c0cda90 0x7fc02c0af3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T14:51:38.949 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:38.939+0000 7fc03227a640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fc02c15c630 0x7fc02c0c9720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T14:51:38.949 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:38.939+0000 7fc03227a640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc02c1f6300 con 0x7fc02c0ccf20 2023-08-09T14:51:38.949 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:38.939+0000 7fc03227a640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc02c06f7b0 con 0x7fc02c0cda90 2023-08-09T14:51:38.949 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:38.939+0000 7fc03227a640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc02c06fa90 con 0x7fc02c15c630 2023-08-09T14:51:38.949 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:38.939+0000 7fc02bfff640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc02c0ccf20 0x7fc02c1f5dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T14:51:38.949 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:38.939+0000 7fc02bfff640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc02c0ccf20 0x7fc02c1f5dc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:52418/0 (socket says 172.21.15.43:52418) 2023-08-09T14:51:38.949 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:38.939+0000 7fc02bfff640 1 -- 172.21.15.43:0/1424095792 learned_addr learned my addr 172.21.15.43:0/1424095792 (peer_addr_for_me v2:172.21.15.43:0/0) 2023-08-09T14:51:38.949 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:38.939+0000 7fc02bfff640 1 -- 172.21.15.43:0/1424095792 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fc02c15c630 msgr2=0x7fc02c0c9720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T14:51:38.950 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:38.939+0000 7fc02bfff640 1 --2- 172.21.15.43:0/1424095792 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fc02c15c630 0x7fc02c0c9720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T14:51:38.950 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:38.939+0000 7fc02bfff640 1 -- 172.21.15.43:0/1424095792 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fc02c0cda90 msgr2=0x7fc02c0af3f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T14:51:38.950 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:38.939+0000 7fc02bfff640 1 --2- 172.21.15.43:0/1424095792 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fc02c0cda90 0x7fc02c0af3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T14:51:38.950 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:38.939+0000 7fc02bfff640 1 -- 172.21.15.43:0/1424095792 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc014009b30 con 0x7fc02c0ccf20 2023-08-09T14:51:38.950 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:38.940+0000 7fc02bfff640 1 --2- 172.21.15.43:0/1424095792 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc02c0ccf20 0x7fc02c1f5dc0 secure :-1 s=READY pgs=1819 cs=0 l=1 rev1=1 crypto rx=0x7fc01c00d800 tx=0x7fc01c00dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-09T14:51:38.950 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:38.940+0000 7fc0297fa640 1 -- 172.21.15.43:0/1424095792 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc01c00b6c0 con 0x7fc02c0ccf20 2023-08-09T14:51:38.951 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:38.940+0000 7fc0297fa640 1 -- 172.21.15.43:0/1424095792 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc01c00b820 con 0x7fc02c0ccf20 2023-08-09T14:51:38.951 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:38.940+0000 7fc0297fa640 1 -- 172.21.15.43:0/1424095792 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc01c00bbb0 con 0x7fc02c0ccf20 2023-08-09T14:51:38.951 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:38.940+0000 7fc03227a640 1 -- 172.21.15.43:0/1424095792 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc02c06fd70 con 0x7fc02c0ccf20 2023-08-09T14:51:38.951 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:38.940+0000 7fc03227a640 1 -- 172.21.15.43:0/1424095792 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc02c070330 con 0x7fc02c0ccf20 2023-08-09T14:51:38.951 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:38.941+0000 7fc03227a640 1 -- 172.21.15.43:0/1424095792 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc02c0704c0 con 0x7fc02c0ccf20 2023-08-09T14:51:38.951 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:38.942+0000 7fc0297fa640 1 -- 172.21.15.43:0/1424095792 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 79) v1 ==== 88443+0+0 (secure 0 0 0) 0x7fc01c00bd10 con 0x7fc02c0ccf20 2023-08-09T14:51:38.952 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:38.943+0000 7fc0297fa640 1 --2- 172.21.15.43:0/1424095792 >> [v2:172.21.15.203:6832/2566816332,v1:172.21.15.203:6833/2566816332] conn(0x7fc00c06a170 0x7fc00c06c630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T14:51:38.952 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:38.943+0000 7fc0297fa640 1 -- 172.21.15.43:0/1424095792 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(164..164 src has 1..164) v4 ==== 6348+0+0 (secure 0 0 0) 0x7fc01c088d20 con 0x7fc02c0ccf20 2023-08-09T14:51:38.952 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:38.944+0000 7fc0237fe640 1 --2- 172.21.15.43:0/1424095792 >> [v2:172.21.15.203:6832/2566816332,v1:172.21.15.203:6833/2566816332] conn(0x7fc00c06a170 0x7fc00c06c630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T14:51:38.952 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:38.945+0000 7fc0237fe640 1 --2- 172.21.15.43:0/1424095792 >> [v2:172.21.15.203:6832/2566816332,v1:172.21.15.203:6833/2566816332] conn(0x7fc00c06a170 0x7fc00c06c630 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7fc014035000 tx=0x7fc01402e000 comp rx=0 tx=0).ready entity=mgr.7765 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-09T14:51:38.953 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:38.948+0000 7fc0297fa640 1 -- 172.21.15.43:0/1424095792 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178066 (secure 0 0 0) 0x7fc01c054270 con 0x7fc02c0ccf20 2023-08-09T14:51:39.146 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:39.144+0000 7fc03227a640 1 -- 172.21.15.43:0/1424095792 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fc02c0739f0 con 0x7fc02c0ccf20 2023-08-09T14:51:39.148 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:39.146+0000 7fc0297fa640 1 -- 172.21.15.43:0/1424095792 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v164) v1 ==== 74+0+16000 (secure 0 0 0) 0x7fc01c057890 con 0x7fc02c0ccf20 2023-08-09T14:51:39.148 INFO:teuthology.orchestra.run.smithi043.stdout: 2023-08-09T14:51:39.149 INFO:teuthology.orchestra.run.smithi043.stdout:{"epoch":164,"fsid":"daac87ee-1337-4250-a955-aea4e408a39b","created":"2023-08-09T13:33:41.701198+0000","modified":"2023-08-09T14:51:35.650599+0000","last_up_change":"2023-08-09T13:33:53.783453+0000","last_in_change":"2023-08-09T13:33:45.148904+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":19,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":"rbd","create_time":"2023-08-09T13:33:54.365137+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":".mgr","create_time":"2023-08-09T13:33:54.831803+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"cbd49ecd-e896-4fbd-864e-5c37a829b165","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":149,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6800","nonce":502667335},{"type":"v1","addr":"172.21.15.43:6801","nonce":502667335}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6802","nonce":502667335},{"type":"v1","addr":"172.21.15.43:6803","nonce":502667335}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6806","nonce":502667335},{"type":"v1","addr":"172.21.15.43:6807","nonce":502667335}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6804","nonce":502667335},{"type":"v1","addr":"172.21.15.43:6805","nonce":502667335}]},"public_addr":"172.21.15.43:6801/502667335","cluster_addr":"172.21.15.43:6803/502667335","heartbeat_back_addr":"172.21.15.43:6807/502667335","heartbeat_front_addr":"172.21.15.43:6805/502667335","state":["exists","up"]},{"osd":1,"uuid":"75c7e0fe-1ec7-4fe8-a2f2-3661a65b8500","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":145,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6808","nonce":3289134610},{"type":"v1","addr":"172.21.15.43:6809","nonce":3289134610}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6810","nonce":3289134610},{"type":"v1","addr":"172.21.15.43:6812","nonce":3289134610}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6821","nonce":3289134610},{"type":"v1","addr":"172.21.15.43:6824","nonce":3289134610}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6815","nonce":3289134610},{"type":"v1","addr":"172.21.15.43:6818","nonce":3289134610}]},"public_addr":"172.21.15.43:6809/3289134610","cluster_addr":"172.21.15.43:6812/3289134610","heartbeat_back_addr":"172.21.15.43:6824/3289134610","heartbeat_front_addr":"172.21.15.43:6818/3289134610","state":["exists","up"]},{"osd":2,"uuid":"bb5c41e9-7f5f-4125-9b01-3bb8bfc2ea83","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":145,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6811","nonce":4079709736},{"type":"v1","addr":"172.21.15.43:6814","nonce":4079709736}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6817","nonce":4079709736},{"type":"v1","addr":"172.21.15.43:6820","nonce":4079709736}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6829","nonce":4079709736},{"type":"v1","addr":"172.21.15.43:6831","nonce":4079709736}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6823","nonce":4079709736},{"type":"v1","addr":"172.21.15.43:6826","nonce":4079709736}]},"public_addr":"172.21.15.43:6814/4079709736","cluster_addr":"172.21.15.43:6820/4079709736","heartbeat_back_addr":"172.21.15.43:6831/4079709736","heartbeat_front_addr":"172.21.15.43:6826/4079709736","state":["exists","up"]},{"osd":3,"uuid":"bfe4c8b4-694a-465c-87e6-2b2aab852613","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":149,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6813","nonce":1709538759},{"type":"v1","addr":"172.21.15.43:6816","nonce":1709538759}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6819","nonce":1709538759},{"type":"v1","addr":"172.21.15.43:6822","nonce":1709538759}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6828","nonce":1709538759},{"type":"v1","addr":"172.21.15.43:6830","nonce":1709538759}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6825","nonce":1709538759},{"type":"v1","addr":"172.21.15.43:6827","nonce":1709538759}]},"public_addr":"172.21.15.43:6816/1709538759","cluster_addr":"172.21.15.43:6822/1709538759","heartbeat_back_addr":"172.21.15.43:6830/1709538759","heartbeat_front_addr":"172.21.15.43:6827/1709538759","state":["exists","up"]},{"osd":4,"uuid":"fa2e8a77-aba1-4756-853e-0d174a5bdeba","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":145,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6800","nonce":4196866481},{"type":"v1","addr":"172.21.15.203:6801","nonce":4196866481}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6802","nonce":4196866481},{"type":"v1","addr":"172.21.15.203:6803","nonce":4196866481}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6806","nonce":4196866481},{"type":"v1","addr":"172.21.15.203:6807","nonce":4196866481}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6804","nonce":4196866481},{"type":"v1","addr":"172.21.15.203:6805","nonce":4196866481}]},"public_addr":"172.21.15.203:6801/4196866481","cluster_addr":"172.21.15.203:6803/4196866481","heartbeat_back_addr":"172.21.15.203:6807/4196866481","heartbeat_front_addr":"172.21.15.203:6805/4196866481","state":["exists","up"]},{"osd":5,"uuid":"81811e4b-4d24-4f51-a82c-f1d700a2a00d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":145,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6816","nonce":3059290081},{"type":"v1","addr":"172.21.15.203:6817","nonce":3059290081}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6818","nonce":3059290081},{"type":"v1","addr":"172.21.15.203:6819","nonce":3059290081}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6822","nonce":3059290081},{"type":"v1","addr":"172.21.15.203:6823","nonce":3059290081}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6820","nonce":3059290081},{"type":"v1","addr":"172.21.15.203:6821","nonce":3059290081}]},"public_addr":"172.21.15.203:6817/3059290081","cluster_addr":"172.21.15.203:6819/3059290081","heartbeat_back_addr":"172.21.15.203:6823/3059290081","heartbeat_front_addr":"172.21.15.203:6821/3059290081","state":["exists","up"]},{"osd":6,"uuid":"307bac4d-6a5b-443d-af75-56d695d7674f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":145,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6824","nonce":512155569},{"type":"v1","addr":"172.21.15.203:6825","nonce":512155569}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6826","nonce":512155569},{"type":"v1","addr":"172.21.15.203:6827","nonce":512155569}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6830","nonce":512155569},{"type":"v1","addr":"172.21.15.203:6831","nonce":512155569}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6828","nonce":512155569},{"type":"v1","addr":"172.21.15.203:6829","nonce":512155569}]},"public_addr":"172.21.15.203:6825/512155569","cluster_addr":"172.21.15.203:6827/512155569","heartbeat_back_addr":"172.21.15.203:6831/512155569","heartbeat_front_addr":"172.21.15.203:6829/512155569","state":["exists","up"]},{"osd":7,"uuid":"a95dc7ca-9f1b-4295-af70-68a22a2d7d5e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":145,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6808","nonce":3331450438},{"type":"v1","addr":"172.21.15.203:6809","nonce":3331450438}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6810","nonce":3331450438},{"type":"v1","addr":"172.21.15.203:6811","nonce":3331450438}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6814","nonce":3331450438},{"type":"v1","addr":"172.21.15.203:6815","nonce":3331450438}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6812","nonce":3331450438},{"type":"v1","addr":"172.21.15.203:6813","nonce":3331450438}]},"public_addr":"172.21.15.203:6809/3331450438","cluster_addr":"172.21.15.203:6811/3331450438","heartbeat_back_addr":"172.21.15.203:6815/3331450438","heartbeat_front_addr":"172.21.15.203:6813/3331450438","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-09T13:33:50.547652+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-09T13:33:50.589469+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-09T13:33:50.561724+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-09T13:33:50.599650+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-09T13:33:50.411855+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-09T13:33:50.219681+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-09T13:33:50.504664+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-09T13:33:50.441248+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.43:6834/2023142010":"2023-08-10T14:51:32.630804+0000","172.21.15.43:6832/2023142010":"2023-08-10T14:51:32.630804+0000","172.21.15.203:0/1340171653":"2023-08-10T14:51:27.179649+0000","172.21.15.203:0/2575410559":"2023-08-10T14:51:27.179649+0000","172.21.15.203:0/1196185412":"2023-08-10T14:51:27.179649+0000","172.21.15.203:6833/372694067":"2023-08-10T14:51:27.179649+0000","172.21.15.203:6832/372694067":"2023-08-10T14:51:27.179649+0000","172.21.15.203:0/2726918685":"2023-08-10T14:51:27.179649+0000","172.21.15.203:0/3273594633":"2023-08-10T14:51:27.179649+0000","172.21.15.203:0/2477376154":"2023-08-10T14:51:27.179649+0000","172.21.15.203:0/2901879784":"2023-08-10T13:55:44.498027+0000","172.21.15.203:0/1501229236":"2023-08-10T13:55:44.498027+0000","172.21.15.203:0/3615051942":"2023-08-10T13:55:44.498027+0000","172.21.15.203:6833/3490266242":"2023-08-10T13:55:44.498027+0000","172.21.15.203:6832/3490266242":"2023-08-10T13:55:44.498027+0000","172.21.15.203:0/3892021074":"2023-08-10T13:55:44.498027+0000","172.21.15.203:0/750810033":"2023-08-10T13:55:44.498027+0000","172.21.15.203:0/1164133940":"2023-08-10T13:55:44.498027+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":7,"snaps":[{"begin":104,"length":1},{"begin":221,"length":1}]}],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-08-09T14:51:39.150 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:39.149+0000 7fc03227a640 1 -- 172.21.15.43:0/1424095792 >> [v2:172.21.15.203:6832/2566816332,v1:172.21.15.203:6833/2566816332] conn(0x7fc00c06a170 msgr2=0x7fc00c06c630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T14:51:39.151 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:39.149+0000 7fc03227a640 1 --2- 172.21.15.43:0/1424095792 >> [v2:172.21.15.203:6832/2566816332,v1:172.21.15.203:6833/2566816332] conn(0x7fc00c06a170 0x7fc00c06c630 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7fc014035000 tx=0x7fc01402e000 comp rx=0 tx=0).stop 2023-08-09T14:51:39.151 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:39.149+0000 7fc03227a640 1 -- 172.21.15.43:0/1424095792 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc02c0ccf20 msgr2=0x7fc02c1f5dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T14:51:39.151 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:39.149+0000 7fc03227a640 1 --2- 172.21.15.43:0/1424095792 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc02c0ccf20 0x7fc02c1f5dc0 secure :-1 s=READY pgs=1819 cs=0 l=1 rev1=1 crypto rx=0x7fc01c00d800 tx=0x7fc01c00dcc0 comp rx=0 tx=0).stop 2023-08-09T14:51:39.152 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:39.149+0000 7fc03227a640 1 -- 172.21.15.43:0/1424095792 shutdown_connections 2023-08-09T14:51:39.152 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:39.149+0000 7fc03227a640 1 --2- 172.21.15.43:0/1424095792 >> [v2:172.21.15.203:6832/2566816332,v1:172.21.15.203:6833/2566816332] conn(0x7fc00c06a170 0x7fc00c06c630 unknown :-1 s=CLOSED pgs=41 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T14:51:39.152 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:39.149+0000 7fc03227a640 1 --2- 172.21.15.43:0/1424095792 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fc02c15c630 0x7fc02c0c9720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T14:51:39.152 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:39.149+0000 7fc03227a640 1 --2- 172.21.15.43:0/1424095792 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fc02c0cda90 0x7fc02c0af3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T14:51:39.153 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:39.149+0000 7fc03227a640 1 --2- 172.21.15.43:0/1424095792 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc02c0ccf20 0x7fc02c1f5dc0 unknown :-1 s=CLOSED pgs=1819 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T14:51:39.153 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:39.149+0000 7fc03227a640 1 -- 172.21.15.43:0/1424095792 >> 172.21.15.43:0/1424095792 conn(0x7fc02c0ce0a0 msgr2=0x7fc02c067560 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-09T14:51:39.153 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:39.149+0000 7fc03227a640 1 -- 172.21.15.43:0/1424095792 shutdown_connections 2023-08-09T14:51:39.153 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:39.149+0000 7fc03227a640 1 -- 172.21.15.43:0/1424095792 wait complete. 2023-08-09T14:51:39.172 INFO:tasks.cephfs.cephfs_test_case.ceph_manager:[{'pool': 1, 'pool_name': 'rbd', 'create_time': '2023-08-09T13:33:54.365137+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}, 'read_balance': {'score_acting': 2, 'score_stable': 2, 'optimal_score': 1, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': '.mgr', 'create_time': '2023-08-09T13:33:54.831803+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-08-09T14:51:39.173 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get rbd pg_num 2023-08-09T14:51:39.410 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:39.408+0000 7f75dcb79640 1 Processor -- start 2023-08-09T14:51:39.410 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:39.408+0000 7f75dcb79640 1 -- start start 2023-08-09T14:51:39.412 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:39.410+0000 7f75dcb79640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f75d815c620 0x7f75d80cceb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T14:51:39.413 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:39.410+0000 7f75dcb79640 1 -- --> v1:172.21.15.43:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f75d80cd700 con 0x7f75d80b1260 2023-08-09T14:51:39.413 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:39.410+0000 7f75dcb79640 1 -- --> v1:172.21.15.203:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f75d80cd9e0 con 0x7f75d80cf510 2023-08-09T14:51:39.413 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:39.410+0000 7f75dcb79640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f75d80b1b70 con 0x7f75d815c620 2023-08-09T14:51:39.413 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:39.410+0000 7f75cdd9b640 1 --1- >> v1:172.21.15.43:6789/0 conn(0x7f75d80b1260 0x7f75d80cee00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.43:6789/0 says I am v1:172.21.15.43:46628/0 (socket says 172.21.15.43:46628) 2023-08-09T14:51:39.413 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:39.410+0000 7f75cdd9b640 1 -- 172.21.15.43:0/2994426479 learned_addr learned my addr 172.21.15.43:0/2994426479 (peer_addr_for_me v1:172.21.15.43:0/0) 2023-08-09T14:51:39.414 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:39.410+0000 7f75d659c640 1 --2- 172.21.15.43:0/2994426479 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f75d815c620 0x7f75d80cceb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T14:51:39.414 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:39.410+0000 7f75d5d9b640 1 -- 172.21.15.43:0/2994426479 <== mon.0 v1:172.21.15.43:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 318919457 0 0) 0x7f75d80cd700 con 0x7f75d80b1260 2023-08-09T14:51:39.414 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:39.411+0000 7f75d5d9b640 1 -- 172.21.15.43:0/2994426479 --> v1:172.21.15.43:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f75c4003520 con 0x7f75d80b1260 2023-08-09T14:51:39.415 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:39.411+0000 7f75d5d9b640 1 -- 172.21.15.43:0/2994426479 <== mon.0 v1:172.21.15.43:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 952408262 0 0) 0x7f75c4003520 con 0x7f75d80b1260 2023-08-09T14:51:39.415 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:39.411+0000 7f75d5d9b640 1 -- 172.21.15.43:0/2994426479 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f75d815c620 msgr2=0x7f75d80cceb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-09T14:51:39.415 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:39.411+0000 7f75d5d9b640 1 --2- 172.21.15.43:0/2994426479 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f75d815c620 0x7f75d80cceb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T14:51:39.415 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:39.411+0000 7f75d5d9b640 1 -- 172.21.15.43:0/2994426479 >> v1:172.21.15.203:6789/0 conn(0x7f75d80cf510 legacy=0x7f75d80d0350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T14:51:39.416 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:39.411+0000 7f75d5d9b640 1 -- 172.21.15.43:0/2994426479 --> v1:172.21.15.43:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f75d80cdcc0 con 0x7f75d80b1260 2023-08-09T14:51:39.417 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:39.411+0000 7f75d5d9b640 1 -- 172.21.15.43:0/2994426479 <== mon.0 v1:172.21.15.43:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1285150740 0 0) 0x7f75c0001670 con 0x7f75d80b1260 2023-08-09T14:51:39.418 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:39.411+0000 7f75d5d9b640 1 -- 172.21.15.43:0/2994426479 >> v1:172.21.15.43:6789/0 conn(0x7f75d80b1260 legacy=0x7f75d80cee00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T14:51:39.418 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:39.411+0000 7f75d5d9b640 1 --2- 172.21.15.43:0/2994426479 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f75c40039a0 0x7f75c4003dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T14:51:39.418 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:39.411+0000 7f75d5d9b640 1 --2- 172.21.15.43:0/2994426479 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f75c40047c0 0x7f75c4004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T14:51:39.418 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:39.411+0000 7f75d5d9b640 1 --2- 172.21.15.43:0/2994426479 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f75c40051d0 0x7f75c400b980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T14:51:39.418 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:39.411+0000 7f75d5d9b640 1 -- 172.21.15.43:0/2994426479 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f75c400c080 con 0x7f75c40047c0 2023-08-09T14:51:39.418 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:39.411+0000 7f75d5d9b640 1 -- 172.21.15.43:0/2994426479 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f75c400c1f0 con 0x7f75c40051d0 2023-08-09T14:51:39.419 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:39.411+0000 7f75d5d9b640 1 -- 172.21.15.43:0/2994426479 --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f75c400c3b0 con 0x7f75c40039a0 2023-08-09T14:51:39.419 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:39.411+0000 7f75cdd9b640 1 --2- 172.21.15.43:0/2994426479 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f75c40047c0 0x7f75c4004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T14:51:39.419 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:39.412+0000 7f75d659c640 1 --2- 172.21.15.43:0/2994426479 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f75c40039a0 0x7f75c4003dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T14:51:39.419 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:39.412+0000 7f75d6d9d640 1 --2- 172.21.15.43:0/2994426479 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f75c40051d0 0x7f75c400b980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T14:51:39.419 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:39.412+0000 7f75cdd9b640 1 -- 172.21.15.43:0/2994426479 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f75c40039a0 msgr2=0x7f75c4003dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-09T14:51:39.419 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:39.412+0000 7f75cdd9b640 1 --2- 172.21.15.43:0/2994426479 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f75c40039a0 0x7f75c4003dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T14:51:39.419 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:39.412+0000 7f75cdd9b640 1 -- 172.21.15.43:0/2994426479 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f75c40051d0 msgr2=0x7f75c400b980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-09T14:51:39.419 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:39.412+0000 7f75cdd9b640 1 --2- 172.21.15.43:0/2994426479 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f75c40051d0 0x7f75c400b980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T14:51:39.420 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:39.412+0000 7f75cdd9b640 1 -- 172.21.15.43:0/2994426479 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f75c4003520 con 0x7f75c40047c0 2023-08-09T14:51:39.420 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:39.412+0000 7f75d6d9d640 1 --2- 172.21.15.43:0/2994426479 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f75c40051d0 0x7f75c400b980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-08-09T14:51:39.420 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:39.413+0000 7f75cdd9b640 1 --2- 172.21.15.43:0/2994426479 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f75c40047c0 0x7f75c4004c40 secure :-1 s=READY pgs=1821 cs=0 l=1 rev1=1 crypto rx=0x7f75c000c240 tx=0x7f75c40057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=39c5cf35ec37462b server_cookie=0 in_seq=0 out_seq=0 2023-08-09T14:51:39.420 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:39.413+0000 7f75d5d9b640 1 -- 172.21.15.43:0/2994426479 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f75c0005ad0 con 0x7f75c40047c0 2023-08-09T14:51:39.420 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:39.413+0000 7f75d5d9b640 1 -- 172.21.15.43:0/2994426479 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f75c0005c30 con 0x7f75c40047c0 2023-08-09T14:51:39.420 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:39.413+0000 7f75d5d9b640 1 -- 172.21.15.43:0/2994426479 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f75c002caf0 con 0x7f75c40047c0 2023-08-09T14:51:39.420 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:39.414+0000 7f75dcb79640 1 -- 172.21.15.43:0/2994426479 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f75c40047c0 msgr2=0x7f75c4004c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T14:51:39.421 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:39.414+0000 7f75dcb79640 1 --2- 172.21.15.43:0/2994426479 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f75c40047c0 0x7f75c4004c40 secure :-1 s=READY pgs=1821 cs=0 l=1 rev1=1 crypto rx=0x7f75c000c240 tx=0x7f75c40057f0 comp rx=0 tx=0).stop 2023-08-09T14:51:39.421 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:39.414+0000 7f75d6d9d640 1 -- 172.21.15.43:0/2994426479 reap_dead start 2023-08-09T14:51:39.421 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:39.414+0000 7f75dcb79640 1 -- 172.21.15.43:0/2994426479 shutdown_connections 2023-08-09T14:51:39.421 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:39.414+0000 7f75dcb79640 1 -- 172.21.15.43:0/2994426479 >> 172.21.15.43:0/2994426479 conn(0x7f75d809b750 msgr2=0x7f75d80d2410 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-09T14:51:39.421 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:39.414+0000 7f75dcb79640 1 -- 172.21.15.43:0/2994426479 shutdown_connections 2023-08-09T14:51:39.421 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:39.414+0000 7f75dcb79640 1 -- 172.21.15.43:0/2994426479 wait complete. 2023-08-09T14:51:39.421 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:39.416+0000 7f75dcb79640 1 Processor -- start 2023-08-09T14:51:39.421 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:39.416+0000 7f75dcb79640 1 -- start start 2023-08-09T14:51:39.423 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:39.417+0000 7f75dcb79640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f75c40047c0 0x7f75d81f2890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T14:51:39.423 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:39.417+0000 7f75dcb79640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f75d80b7bc0 0x7f75d81f5cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T14:51:39.424 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:39.417+0000 7f75dcb79640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f75d80d2980 0x7f75d80d2e00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T14:51:39.424 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:39.417+0000 7f75dcb79640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f75d80d3340 con 0x7f75c40047c0 2023-08-09T14:51:39.424 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:39.417+0000 7f75dcb79640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f75d81f2dd0 con 0x7f75d80d2980 2023-08-09T14:51:39.424 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:39.417+0000 7f75dcb79640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f75d81f5830 con 0x7f75d80b7bc0 2023-08-09T14:51:39.424 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:39.418+0000 7f75d659c640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f75c40047c0 0x7f75d81f2890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T14:51:39.424 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:39.418+0000 7f75d659c640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f75c40047c0 0x7f75d81f2890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:52444/0 (socket says 172.21.15.43:52444) 2023-08-09T14:51:39.425 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:39.418+0000 7f75d659c640 1 -- 172.21.15.43:0/1928613568 learned_addr learned my addr 172.21.15.43:0/1928613568 (peer_addr_for_me v2:172.21.15.43:0/0) 2023-08-09T14:51:39.425 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:39.418+0000 7f75d6d9d640 1 --2- 172.21.15.43:0/1928613568 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f75d80d2980 0x7f75d80d2e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T14:51:39.425 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:39.418+0000 7f75cdd9b640 1 --2- 172.21.15.43:0/1928613568 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f75d80b7bc0 0x7f75d81f5cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T14:51:39.425 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:39.418+0000 7f75d659c640 1 -- 172.21.15.43:0/1928613568 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f75d80b7bc0 msgr2=0x7f75d81f5cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T14:51:39.425 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:39.418+0000 7f75d659c640 1 --2- 172.21.15.43:0/1928613568 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f75d80b7bc0 0x7f75d81f5cc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T14:51:39.425 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:39.418+0000 7f75d659c640 1 -- 172.21.15.43:0/1928613568 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f75d80d2980 msgr2=0x7f75d80d2e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T14:51:39.426 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:39.418+0000 7f75d659c640 1 --2- 172.21.15.43:0/1928613568 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f75d80d2980 0x7f75d80d2e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T14:51:39.426 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:39.418+0000 7f75d659c640 1 -- 172.21.15.43:0/1928613568 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f75d81f5530 con 0x7f75c40047c0 2023-08-09T14:51:39.426 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:39.418+0000 7f75d6d9d640 1 --2- 172.21.15.43:0/1928613568 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f75d80d2980 0x7f75d80d2e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-08-09T14:51:39.426 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:39.418+0000 7f75d659c640 1 --2- 172.21.15.43:0/1928613568 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f75c40047c0 0x7f75d81f2890 secure :-1 s=READY pgs=1822 cs=0 l=1 rev1=1 crypto rx=0x7f75c8011430 tx=0x7f75c80118f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-09T14:51:39.426 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:39.419+0000 7f75cf7fe640 1 -- 172.21.15.43:0/1928613568 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f75c800e030 con 0x7f75c40047c0 2023-08-09T14:51:39.426 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:39.419+0000 7f75cf7fe640 1 -- 172.21.15.43:0/1928613568 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f75c8016470 con 0x7f75c40047c0 2023-08-09T14:51:39.426 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:39.419+0000 7f75dcb79640 1 -- 172.21.15.43:0/1928613568 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f75d8158fa0 con 0x7f75c40047c0 2023-08-09T14:51:39.427 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:39.419+0000 7f75cf7fe640 1 -- 172.21.15.43:0/1928613568 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f75c8016820 con 0x7f75c40047c0 2023-08-09T14:51:39.427 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:39.419+0000 7f75dcb79640 1 -- 172.21.15.43:0/1928613568 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f75d80b7610 con 0x7f75c40047c0 2023-08-09T14:51:39.427 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:39.420+0000 7f75dcb79640 1 -- 172.21.15.43:0/1928613568 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f75d80739f0 con 0x7f75c40047c0 2023-08-09T14:51:39.427 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:39.421+0000 7f75cf7fe640 1 -- 172.21.15.43:0/1928613568 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 79) v1 ==== 88443+0+0 (secure 0 0 0) 0x7f75c8016980 con 0x7f75c40047c0 2023-08-09T14:51:39.427 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:39.422+0000 7f75cf7fe640 1 --2- 172.21.15.43:0/1928613568 >> [v2:172.21.15.203:6832/2566816332,v1:172.21.15.203:6833/2566816332] conn(0x7f75b806a170 0x7f75b806c630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T14:51:39.427 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:39.422+0000 7f75cf7fe640 1 -- 172.21.15.43:0/1928613568 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(164..164 src has 1..164) v4 ==== 6348+0+0 (secure 0 0 0) 0x7f75c800f070 con 0x7f75c40047c0 2023-08-09T14:51:39.428 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:39.423+0000 7f75cdd9b640 1 --2- 172.21.15.43:0/1928613568 >> [v2:172.21.15.203:6832/2566816332,v1:172.21.15.203:6833/2566816332] conn(0x7f75b806a170 0x7f75b806c630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T14:51:39.428 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:39.424+0000 7f75cdd9b640 1 --2- 172.21.15.43:0/1928613568 >> [v2:172.21.15.203:6832/2566816332,v1:172.21.15.203:6833/2566816332] conn(0x7f75b806a170 0x7f75b806c630 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7f75c0034000 tx=0x7f75c0009030 comp rx=0 tx=0).ready entity=mgr.7765 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-09T14:51:39.428 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:39.426+0000 7f75cf7fe640 1 -- 172.21.15.43:0/1928613568 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178066 (secure 0 0 0) 0x7f75c8054e80 con 0x7f75c40047c0 2023-08-09T14:51:39.624 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:39.622+0000 7f75dcb79640 1 -- 172.21.15.43:0/1928613568 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"} v 0) v1 -- 0x7f75d80cfda0 con 0x7f75c40047c0 2023-08-09T14:51:39.625 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:39.623+0000 7f75cf7fe640 1 -- 172.21.15.43:0/1928613568 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"}]=0 v164) v1 ==== 92+0+10 (secure 0 0 0) 0x7f75c80584a0 con 0x7f75c40047c0 2023-08-09T14:51:39.625 INFO:teuthology.orchestra.run.smithi043.stdout:pg_num: 8 2023-08-09T14:51:39.628 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:39.626+0000 7f75dcb79640 1 -- 172.21.15.43:0/1928613568 >> [v2:172.21.15.203:6832/2566816332,v1:172.21.15.203:6833/2566816332] conn(0x7f75b806a170 msgr2=0x7f75b806c630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T14:51:39.628 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:39.626+0000 7f75dcb79640 1 --2- 172.21.15.43:0/1928613568 >> [v2:172.21.15.203:6832/2566816332,v1:172.21.15.203:6833/2566816332] conn(0x7f75b806a170 0x7f75b806c630 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7f75c0034000 tx=0x7f75c0009030 comp rx=0 tx=0).stop 2023-08-09T14:51:39.629 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:39.626+0000 7f75dcb79640 1 -- 172.21.15.43:0/1928613568 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f75c40047c0 msgr2=0x7f75d81f2890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T14:51:39.629 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:39.626+0000 7f75dcb79640 1 --2- 172.21.15.43:0/1928613568 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f75c40047c0 0x7f75d81f2890 secure :-1 s=READY pgs=1822 cs=0 l=1 rev1=1 crypto rx=0x7f75c8011430 tx=0x7f75c80118f0 comp rx=0 tx=0).stop 2023-08-09T14:51:39.629 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:39.626+0000 7f75dcb79640 1 -- 172.21.15.43:0/1928613568 shutdown_connections 2023-08-09T14:51:39.629 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:39.626+0000 7f75dcb79640 1 --2- 172.21.15.43:0/1928613568 >> [v2:172.21.15.203:6832/2566816332,v1:172.21.15.203:6833/2566816332] conn(0x7f75b806a170 0x7f75b806c630 unknown :-1 s=CLOSED pgs=42 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T14:51:39.630 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:39.626+0000 7f75dcb79640 1 --2- 172.21.15.43:0/1928613568 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f75d80d2980 0x7f75d80d2e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T14:51:39.630 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:39.626+0000 7f75dcb79640 1 --2- 172.21.15.43:0/1928613568 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f75d80b7bc0 0x7f75d81f5cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T14:51:39.630 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:39.626+0000 7f75dcb79640 1 --2- 172.21.15.43:0/1928613568 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f75c40047c0 0x7f75d81f2890 unknown :-1 s=CLOSED pgs=1822 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T14:51:39.630 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:39.626+0000 7f75dcb79640 1 -- 172.21.15.43:0/1928613568 >> 172.21.15.43:0/1928613568 conn(0x7f75d809b750 msgr2=0x7f75d80b2370 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-09T14:51:39.631 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:39.626+0000 7f75dcb79640 1 -- 172.21.15.43:0/1928613568 shutdown_connections 2023-08-09T14:51:39.631 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:39.626+0000 7f75dcb79640 1 -- 172.21.15.43:0/1928613568 wait complete. 2023-08-09T14:51:39.646 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-08-09T14:51:39.867 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:39.866+0000 7ff16ee4d640 1 Processor -- start 2023-08-09T14:51:39.868 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:39.866+0000 7ff16ee4d640 1 -- start start 2023-08-09T14:51:39.870 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:39.868+0000 7ff16ee4d640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7ff168159470 0x7ff1680c9420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T14:51:39.870 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:39.868+0000 7ff16ee4d640 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7ff16815da80 0x7ff168070db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T14:51:39.871 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:39.868+0000 7ff16ee4d640 1 -- --> v1:172.21.15.43:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff1680c9d40 con 0x7ff1680712f0 2023-08-09T14:51:39.871 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:39.868+0000 7ff16ee4d640 1 -- --> v2:172.21.15.203:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff168070990 con 0x7ff16815da80 2023-08-09T14:51:39.871 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:39.868+0000 7ff16ee4d640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff1680ca020 con 0x7ff168159470 2023-08-09T14:51:39.871 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:39.869+0000 7ff16d3ea640 1 --1- >> v1:172.21.15.43:6789/0 conn(0x7ff1680712f0 0x7ff1680cc8b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.43:6789/0 says I am v1:172.21.15.43:46640/0 (socket says 172.21.15.43:46640) 2023-08-09T14:51:39.871 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:39.869+0000 7ff16d3ea640 1 -- 172.21.15.43:0/2799829367 learned_addr learned my addr 172.21.15.43:0/2799829367 (peer_addr_for_me v1:172.21.15.43:0/0) 2023-08-09T14:51:39.872 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:39.869+0000 7ff16cbe9640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7ff168159470 0x7ff1680c9420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T14:51:39.873 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:39.869+0000 7ff167fff640 1 --2- 172.21.15.43:0/2799829367 >> v2:172.21.15.203:3300/0 conn(0x7ff16815da80 0x7ff168070db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T14:51:39.873 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:39.870+0000 7ff1677fe640 1 -- 172.21.15.43:0/2799829367 <== mon.0 v1:172.21.15.43:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3273766550 0 0) 0x7ff1680c9d40 con 0x7ff1680712f0 2023-08-09T14:51:39.873 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:39.870+0000 7ff1677fe640 1 -- 172.21.15.43:0/2799829367 --> v1:172.21.15.43:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ff158003520 con 0x7ff1680712f0 2023-08-09T14:51:39.873 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:39.870+0000 7ff16cbe9640 1 -- 172.21.15.43:0/2799829367 >> v2:172.21.15.203:3300/0 conn(0x7ff16815da80 msgr2=0x7ff168070db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-09T14:51:39.874 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:39.870+0000 7ff16cbe9640 1 --2- 172.21.15.43:0/2799829367 >> v2:172.21.15.203:3300/0 conn(0x7ff16815da80 0x7ff168070db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T14:51:39.874 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:39.870+0000 7ff16cbe9640 1 -- 172.21.15.43:0/2799829367 >> v1:172.21.15.43:6789/0 conn(0x7ff1680712f0 legacy=0x7ff1680cc8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T14:51:39.874 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:39.870+0000 7ff16cbe9640 1 -- 172.21.15.43:0/2799829367 --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff1680705b0 con 0x7ff168159470 2023-08-09T14:51:39.874 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:39.871+0000 7ff16cbe9640 1 --2- 172.21.15.43:0/2799829367 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7ff168159470 0x7ff1680c9420 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7ff15c0097d0 tx=0x7ff15c02b8a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=26e2daa35375474d server_cookie=0 in_seq=0 out_seq=0 2023-08-09T14:51:39.875 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:39.872+0000 7ff1677fe640 1 -- 172.21.15.43:0/2799829367 <== mon.2 v2:172.21.15.203:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff15c031800 con 0x7ff168159470 2023-08-09T14:51:39.875 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:39.872+0000 7ff1677fe640 1 -- 172.21.15.43:0/2799829367 <== mon.2 v2:172.21.15.203:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff15c031960 con 0x7ff168159470 2023-08-09T14:51:39.875 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:39.872+0000 7ff1677fe640 1 -- 172.21.15.43:0/2799829367 <== mon.2 v2:172.21.15.203:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff15c031cd0 con 0x7ff168159470 2023-08-09T14:51:39.876 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:39.872+0000 7ff16ee4d640 1 -- 172.21.15.43:0/2799829367 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7ff168159470 msgr2=0x7ff1680c9420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T14:51:39.876 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:39.872+0000 7ff16ee4d640 1 --2- 172.21.15.43:0/2799829367 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7ff168159470 0x7ff1680c9420 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7ff15c0097d0 tx=0x7ff15c02b8a0 comp rx=0 tx=0).stop 2023-08-09T14:51:39.877 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:39.872+0000 7ff16ee4d640 1 -- 172.21.15.43:0/2799829367 shutdown_connections 2023-08-09T14:51:39.877 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:39.872+0000 7ff16ee4d640 1 --2- 172.21.15.43:0/2799829367 >> v2:172.21.15.203:3300/0 conn(0x7ff16815da80 0x7ff168070db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T14:51:39.877 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:39.872+0000 7ff16ee4d640 1 --2- 172.21.15.43:0/2799829367 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7ff168159470 0x7ff1680c9420 unknown :-1 s=CLOSED pgs=247 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T14:51:39.877 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:39.872+0000 7ff16ee4d640 1 -- 172.21.15.43:0/2799829367 >> 172.21.15.43:0/2799829367 conn(0x7ff1680aff50 msgr2=0x7ff1680b0360 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-09T14:51:39.877 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:39.873+0000 7ff16ee4d640 1 -- 172.21.15.43:0/2799829367 shutdown_connections 2023-08-09T14:51:39.877 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:39.873+0000 7ff16ee4d640 1 -- 172.21.15.43:0/2799829367 wait complete. 2023-08-09T14:51:39.877 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:39.874+0000 7ff16ee4d640 1 Processor -- start 2023-08-09T14:51:39.877 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:39.875+0000 7ff16ee4d640 1 -- start start 2023-08-09T14:51:39.879 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:39.875+0000 7ff16ee4d640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7ff1680712f0 0x7ff1680b38e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T14:51:39.879 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:39.876+0000 7ff16ee4d640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7ff168159470 0x7ff16806b510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T14:51:39.880 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:39.876+0000 7ff16ee4d640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff16815da80 0x7ff16806ba50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T14:51:39.880 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:39.876+0000 7ff16ee4d640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff1681556e0 con 0x7ff16815da80 2023-08-09T14:51:39.880 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:39.876+0000 7ff16ee4d640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff168154880 con 0x7ff1680712f0 2023-08-09T14:51:39.880 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:39.876+0000 7ff16ee4d640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff168154ad0 con 0x7ff168159470 2023-08-09T14:51:39.880 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:39.876+0000 7ff16d3ea640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff16815da80 0x7ff16806ba50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T14:51:39.880 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:39.876+0000 7ff16d3ea640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff16815da80 0x7ff16806ba50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:52450/0 (socket says 172.21.15.43:52450) 2023-08-09T14:51:39.880 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:39.876+0000 7ff16d3ea640 1 -- 172.21.15.43:0/1356119579 learned_addr learned my addr 172.21.15.43:0/1356119579 (peer_addr_for_me v2:172.21.15.43:0/0) 2023-08-09T14:51:39.881 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:39.876+0000 7ff16cbe9640 1 --2- 172.21.15.43:0/1356119579 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7ff1680712f0 0x7ff1680b38e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T14:51:39.881 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:39.876+0000 7ff167fff640 1 --2- 172.21.15.43:0/1356119579 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7ff168159470 0x7ff16806b510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T14:51:39.881 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:39.876+0000 7ff16d3ea640 1 -- 172.21.15.43:0/1356119579 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7ff168159470 msgr2=0x7ff16806b510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T14:51:39.881 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:39.876+0000 7ff16d3ea640 1 --2- 172.21.15.43:0/1356119579 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7ff168159470 0x7ff16806b510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T14:51:39.881 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:39.876+0000 7ff16d3ea640 1 -- 172.21.15.43:0/1356119579 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7ff1680712f0 msgr2=0x7ff1680b38e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T14:51:39.881 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:39.876+0000 7ff16d3ea640 1 --2- 172.21.15.43:0/1356119579 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7ff1680712f0 0x7ff1680b38e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T14:51:39.881 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:39.876+0000 7ff16d3ea640 1 -- 172.21.15.43:0/1356119579 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff15c009400 con 0x7ff16815da80 2023-08-09T14:51:39.882 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:39.876+0000 7ff16cbe9640 1 --2- 172.21.15.43:0/1356119579 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7ff1680712f0 0x7ff1680b38e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-08-09T14:51:39.882 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:39.877+0000 7ff16d3ea640 1 --2- 172.21.15.43:0/1356119579 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff16815da80 0x7ff16806ba50 secure :-1 s=READY pgs=1824 cs=0 l=1 rev1=1 crypto rx=0x7ff16000ca20 tx=0x7ff16000cee0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-09T14:51:39.882 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:39.877+0000 7ff1657fa640 1 -- 172.21.15.43:0/1356119579 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff16000abd0 con 0x7ff16815da80 2023-08-09T14:51:39.882 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:39.877+0000 7ff1657fa640 1 -- 172.21.15.43:0/1356119579 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff16000ad30 con 0x7ff16815da80 2023-08-09T14:51:39.882 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:39.877+0000 7ff1657fa640 1 -- 172.21.15.43:0/1356119579 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff160012620 con 0x7ff16815da80 2023-08-09T14:51:39.882 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:39.877+0000 7ff16ee4d640 1 -- 172.21.15.43:0/1356119579 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff168154db0 con 0x7ff16815da80 2023-08-09T14:51:39.882 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:39.877+0000 7ff16ee4d640 1 -- 172.21.15.43:0/1356119579 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff168155370 con 0x7ff16815da80 2023-08-09T14:51:39.883 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:39.878+0000 7ff16ee4d640 1 -- 172.21.15.43:0/1356119579 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff168155890 con 0x7ff16815da80 2023-08-09T14:51:39.883 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:39.879+0000 7ff1657fa640 1 -- 172.21.15.43:0/1356119579 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 79) v1 ==== 88443+0+0 (secure 0 0 0) 0x7ff160007070 con 0x7ff16815da80 2023-08-09T14:51:39.883 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:39.880+0000 7ff1657fa640 1 --2- 172.21.15.43:0/1356119579 >> [v2:172.21.15.203:6832/2566816332,v1:172.21.15.203:6833/2566816332] conn(0x7ff14c06a1a0 0x7ff14c06c660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T14:51:39.884 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:39.880+0000 7ff1657fa640 1 -- 172.21.15.43:0/1356119579 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(164..164 src has 1..164) v4 ==== 6348+0+0 (secure 0 0 0) 0x7ff1600886c0 con 0x7ff16815da80 2023-08-09T14:51:39.884 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:39.881+0000 7ff16cbe9640 1 --2- 172.21.15.43:0/1356119579 >> [v2:172.21.15.203:6832/2566816332,v1:172.21.15.203:6833/2566816332] conn(0x7ff14c06a1a0 0x7ff14c06c660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T14:51:39.884 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:39.882+0000 7ff16cbe9640 1 --2- 172.21.15.43:0/1356119579 >> [v2:172.21.15.203:6832/2566816332,v1:172.21.15.203:6833/2566816332] conn(0x7ff14c06a1a0 0x7ff14c06c660 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7ff15c036000 tx=0x7ff15c008030 comp rx=0 tx=0).ready entity=mgr.7765 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-09T14:51:39.886 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:39.884+0000 7ff1657fa640 1 -- 172.21.15.43:0/1356119579 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178066 (secure 0 0 0) 0x7ff168155890 con 0x7ff16815da80 2023-08-09T14:51:40.079 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:40.077+0000 7ff16ee4d640 1 -- 172.21.15.43:0/1356119579 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7ff168155890 con 0x7ff16815da80 2023-08-09T14:51:40.080 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:40.078+0000 7ff1657fa640 1 -- 172.21.15.43:0/1356119579 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v164) v1 ==== 93+0+10 (secure 0 0 0) 0x7ff160017020 con 0x7ff16815da80 2023-08-09T14:51:40.080 INFO:teuthology.orchestra.run.smithi043.stdout:pg_num: 1 2023-08-09T14:51:40.082 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:40.080+0000 7ff16ee4d640 1 -- 172.21.15.43:0/1356119579 >> [v2:172.21.15.203:6832/2566816332,v1:172.21.15.203:6833/2566816332] conn(0x7ff14c06a1a0 msgr2=0x7ff14c06c660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T14:51:40.082 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:40.080+0000 7ff16ee4d640 1 --2- 172.21.15.43:0/1356119579 >> [v2:172.21.15.203:6832/2566816332,v1:172.21.15.203:6833/2566816332] conn(0x7ff14c06a1a0 0x7ff14c06c660 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7ff15c036000 tx=0x7ff15c008030 comp rx=0 tx=0).stop 2023-08-09T14:51:40.082 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:40.080+0000 7ff16ee4d640 1 -- 172.21.15.43:0/1356119579 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff16815da80 msgr2=0x7ff16806ba50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T14:51:40.082 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:40.080+0000 7ff16ee4d640 1 --2- 172.21.15.43:0/1356119579 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff16815da80 0x7ff16806ba50 secure :-1 s=READY pgs=1824 cs=0 l=1 rev1=1 crypto rx=0x7ff16000ca20 tx=0x7ff16000cee0 comp rx=0 tx=0).stop 2023-08-09T14:51:40.083 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:40.081+0000 7ff16ee4d640 1 -- 172.21.15.43:0/1356119579 shutdown_connections 2023-08-09T14:51:40.083 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:40.081+0000 7ff16ee4d640 1 --2- 172.21.15.43:0/1356119579 >> [v2:172.21.15.203:6832/2566816332,v1:172.21.15.203:6833/2566816332] conn(0x7ff14c06a1a0 0x7ff14c06c660 unknown :-1 s=CLOSED pgs=43 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T14:51:40.083 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:40.081+0000 7ff16ee4d640 1 --2- 172.21.15.43:0/1356119579 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff16815da80 0x7ff16806ba50 unknown :-1 s=CLOSED pgs=1824 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T14:51:40.083 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:40.081+0000 7ff16ee4d640 1 --2- 172.21.15.43:0/1356119579 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7ff168159470 0x7ff16806b510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T14:51:40.083 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:40.081+0000 7ff16ee4d640 1 --2- 172.21.15.43:0/1356119579 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7ff1680712f0 0x7ff1680b38e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T14:51:40.083 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:40.081+0000 7ff16ee4d640 1 -- 172.21.15.43:0/1356119579 >> 172.21.15.43:0/1356119579 conn(0x7ff1680aff50 msgr2=0x7ff1680cb2d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-09T14:51:40.084 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:40.081+0000 7ff16ee4d640 1 -- 172.21.15.43:0/1356119579 shutdown_connections 2023-08-09T14:51:40.084 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:40.081+0000 7ff16ee4d640 1 -- 172.21.15.43:0/1356119579 wait complete. 2023-08-09T14:51:40.100 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mon mon_allow_pool_delete True 2023-08-09T14:51:40.327 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:40.326+0000 7f7b5fcda640 1 Processor -- start 2023-08-09T14:51:40.328 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:40.326+0000 7f7b5fcda640 1 -- start start 2023-08-09T14:51:40.329 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:40.328+0000 7f7b5fcda640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f7b580c86f0 0x7f7b580c8b50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T14:51:40.330 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:40.328+0000 7f7b5fcda640 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7f7b5806c8d0 0x7f7b5806cd60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T14:51:40.330 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:40.328+0000 7f7b5fcda640 1 -- --> v1:172.21.15.43:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7b580c9090 con 0x7f7b58143c20 2023-08-09T14:51:40.330 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:40.328+0000 7f7b5fcda640 1 -- --> v2:172.21.15.203:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7b58144000 con 0x7f7b5806c8d0 2023-08-09T14:51:40.330 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:40.328+0000 7f7b5fcda640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7b5806c440 con 0x7f7b580c86f0 2023-08-09T14:51:40.330 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:40.328+0000 7f7b5da76640 1 --1- >> v1:172.21.15.43:6789/0 conn(0x7f7b58143c20 0x7f7b5806f540 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.43:6789/0 says I am v1:172.21.15.43:46656/0 (socket says 172.21.15.43:46656) 2023-08-09T14:51:40.330 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:40.328+0000 7f7b5da76640 1 -- 172.21.15.43:0/757216942 learned_addr learned my addr 172.21.15.43:0/757216942 (peer_addr_for_me v1:172.21.15.43:0/0) 2023-08-09T14:51:40.332 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:40.329+0000 7f7b5d275640 1 --2- 172.21.15.43:0/757216942 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f7b580c86f0 0x7f7b580c8b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T14:51:40.332 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:40.329+0000 7f7b5e277640 1 --2- 172.21.15.43:0/757216942 >> v2:172.21.15.203:3300/0 conn(0x7f7b5806c8d0 0x7f7b5806cd60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T14:51:40.332 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:40.329+0000 7f7b5ca74640 1 -- 172.21.15.43:0/757216942 <== mon.0 v1:172.21.15.43:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3319619432 0 0) 0x7f7b580c9090 con 0x7f7b58143c20 2023-08-09T14:51:40.332 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:40.329+0000 7f7b5ca74640 1 -- 172.21.15.43:0/757216942 --> v1:172.21.15.43:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f7b44003520 con 0x7f7b58143c20 2023-08-09T14:51:40.333 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:40.329+0000 7f7b5ca74640 1 -- 172.21.15.43:0/757216942 <== mon.0 v1:172.21.15.43:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2738859252 0 0) 0x7f7b44003520 con 0x7f7b58143c20 2023-08-09T14:51:40.333 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:40.329+0000 7f7b5ca74640 1 -- 172.21.15.43:0/757216942 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f7b580c86f0 msgr2=0x7f7b580c8b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-09T14:51:40.333 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:40.329+0000 7f7b5ca74640 1 --2- 172.21.15.43:0/757216942 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f7b580c86f0 0x7f7b580c8b50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T14:51:40.333 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:40.329+0000 7f7b5ca74640 1 -- 172.21.15.43:0/757216942 >> v2:172.21.15.203:3300/0 conn(0x7f7b5806c8d0 msgr2=0x7f7b5806cd60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-09T14:51:40.333 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:40.329+0000 7f7b5ca74640 1 --2- 172.21.15.43:0/757216942 >> v2:172.21.15.203:3300/0 conn(0x7f7b5806c8d0 0x7f7b5806cd60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T14:51:40.333 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:40.329+0000 7f7b5ca74640 1 -- 172.21.15.43:0/757216942 --> v1:172.21.15.43:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7b5806fc50 con 0x7f7b58143c20 2023-08-09T14:51:40.333 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:40.329+0000 7f7b5ca74640 1 -- 172.21.15.43:0/757216942 <== mon.0 v1:172.21.15.43:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1285150740 0 0) 0x7f7b54001670 con 0x7f7b58143c20 2023-08-09T14:51:40.334 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:40.329+0000 7f7b5ca74640 1 -- 172.21.15.43:0/757216942 >> v1:172.21.15.43:6789/0 conn(0x7f7b58143c20 legacy=0x7f7b5806f540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T14:51:40.334 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:40.329+0000 7f7b5ca74640 1 --2- 172.21.15.43:0/757216942 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f7b580c86f0 0x7f7b440039e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T14:51:40.334 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:40.329+0000 7f7b5ca74640 1 --2- 172.21.15.43:0/757216942 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f7b44003f60 0x7f7b440047c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T14:51:40.334 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:40.329+0000 7f7b5ca74640 1 --2- 172.21.15.43:0/757216942 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f7b44004d00 0x7f7b44005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T14:51:40.334 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:40.329+0000 7f7b5ca74640 1 -- 172.21.15.43:0/757216942 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7b58144000 con 0x7f7b44004d00 2023-08-09T14:51:40.334 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:40.329+0000 7f7b5ca74640 1 -- 172.21.15.43:0/757216942 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7b5806c440 con 0x7f7b44003f60 2023-08-09T14:51:40.334 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:40.329+0000 7f7b5ca74640 1 -- 172.21.15.43:0/757216942 --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7b4400ba30 con 0x7f7b580c86f0 2023-08-09T14:51:40.335 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:40.330+0000 7f7b5e277640 1 --2- 172.21.15.43:0/757216942 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f7b44004d00 0x7f7b44005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T14:51:40.335 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:40.330+0000 7f7b5da76640 1 --2- 172.21.15.43:0/757216942 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f7b580c86f0 0x7f7b440039e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T14:51:40.335 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:40.330+0000 7f7b5e277640 1 -- 172.21.15.43:0/757216942 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f7b580c86f0 msgr2=0x7f7b440039e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-09T14:51:40.335 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:40.330+0000 7f7b5d275640 1 --2- 172.21.15.43:0/757216942 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f7b44003f60 0x7f7b440047c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T14:51:40.335 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:40.330+0000 7f7b5e277640 1 --2- 172.21.15.43:0/757216942 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f7b580c86f0 0x7f7b440039e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T14:51:40.335 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:40.330+0000 7f7b5e277640 1 -- 172.21.15.43:0/757216942 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f7b44003f60 msgr2=0x7f7b440047c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-09T14:51:40.335 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:40.330+0000 7f7b5e277640 1 --2- 172.21.15.43:0/757216942 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f7b44003f60 0x7f7b440047c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T14:51:40.336 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:40.330+0000 7f7b5e277640 1 -- 172.21.15.43:0/757216942 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7b44003520 con 0x7f7b44004d00 2023-08-09T14:51:40.336 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:40.330+0000 7f7b5da76640 1 --2- 172.21.15.43:0/757216942 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f7b580c86f0 0x7f7b440039e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-08-09T14:51:40.337 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:40.331+0000 7f7b5e277640 1 --2- 172.21.15.43:0/757216942 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f7b44004d00 0x7f7b44005180 secure :-1 s=READY pgs=1826 cs=0 l=1 rev1=1 crypto rx=0x7f7b50009fc0 tx=0x7f7b440057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e1d57bf70628f42f server_cookie=0 in_seq=0 out_seq=0 2023-08-09T14:51:40.337 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:40.332+0000 7f7b5ca74640 1 -- 172.21.15.43:0/757216942 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7b5002cc50 con 0x7f7b44004d00 2023-08-09T14:51:40.338 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:40.332+0000 7f7b5ca74640 1 -- 172.21.15.43:0/757216942 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7b5000f040 con 0x7f7b44004d00 2023-08-09T14:51:40.338 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:40.332+0000 7f7b5ca74640 1 -- 172.21.15.43:0/757216942 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7b50034440 con 0x7f7b44004d00 2023-08-09T14:51:40.338 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:40.332+0000 7f7b5fcda640 1 -- 172.21.15.43:0/757216942 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f7b44004d00 msgr2=0x7f7b44005180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T14:51:40.338 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:40.332+0000 7f7b5fcda640 1 --2- 172.21.15.43:0/757216942 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f7b44004d00 0x7f7b44005180 secure :-1 s=READY pgs=1826 cs=0 l=1 rev1=1 crypto rx=0x7f7b50009fc0 tx=0x7f7b440057f0 comp rx=0 tx=0).stop 2023-08-09T14:51:40.338 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:40.332+0000 7f7b5e277640 1 -- 172.21.15.43:0/757216942 reap_dead start 2023-08-09T14:51:40.338 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:40.332+0000 7f7b5fcda640 1 -- 172.21.15.43:0/757216942 shutdown_connections 2023-08-09T14:51:40.338 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:40.332+0000 7f7b5fcda640 1 -- 172.21.15.43:0/757216942 >> 172.21.15.43:0/757216942 conn(0x7f7b580ce0a0 msgr2=0x7f7b580ce480 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-09T14:51:40.339 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:40.332+0000 7f7b5fcda640 1 -- 172.21.15.43:0/757216942 shutdown_connections 2023-08-09T14:51:40.339 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:40.332+0000 7f7b5fcda640 1 -- 172.21.15.43:0/757216942 wait complete. 2023-08-09T14:51:40.339 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:40.334+0000 7f7b5fcda640 1 Processor -- start 2023-08-09T14:51:40.340 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:40.335+0000 7f7b5fcda640 1 -- start start 2023-08-09T14:51:40.340 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:40.336+0000 7f7b5fcda640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f7b44004d00 0x7f7b58070540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T14:51:40.340 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:40.336+0000 7f7b5fcda640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f7b580b0d80 0x7f7b580c5b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T14:51:40.340 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:40.336+0000 7f7b5fcda640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f7b580b1200 0x7f7b5815c390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T14:51:40.341 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:40.336+0000 7f7b5fcda640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7b5815bf70 con 0x7f7b580b0d80 2023-08-09T14:51:40.341 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:40.336+0000 7f7b5fcda640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7b5815b0a0 con 0x7f7b44004d00 2023-08-09T14:51:40.341 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:40.336+0000 7f7b5fcda640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7b580c7c60 con 0x7f7b580b1200 2023-08-09T14:51:40.341 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:40.336+0000 7f7b5d275640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f7b580b0d80 0x7f7b580c5b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T14:51:40.342 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:40.336+0000 7f7b5d275640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f7b580b0d80 0x7f7b580c5b50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:52456/0 (socket says 172.21.15.43:52456) 2023-08-09T14:51:40.342 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:40.336+0000 7f7b5d275640 1 -- 172.21.15.43:0/3598844304 learned_addr learned my addr 172.21.15.43:0/3598844304 (peer_addr_for_me v2:172.21.15.43:0/0) 2023-08-09T14:51:40.344 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:40.336+0000 7f7b5e277640 1 --2- 172.21.15.43:0/3598844304 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f7b580b1200 0x7f7b5815c390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T14:51:40.344 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:40.336+0000 7f7b5da76640 1 --2- 172.21.15.43:0/3598844304 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f7b44004d00 0x7f7b58070540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T14:51:40.344 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:40.336+0000 7f7b5d275640 1 -- 172.21.15.43:0/3598844304 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f7b580b1200 msgr2=0x7f7b5815c390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T14:51:40.344 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:40.336+0000 7f7b5d275640 1 --2- 172.21.15.43:0/3598844304 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f7b580b1200 0x7f7b5815c390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T14:51:40.344 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:40.336+0000 7f7b5d275640 1 -- 172.21.15.43:0/3598844304 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f7b44004d00 msgr2=0x7f7b58070540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T14:51:40.344 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:40.336+0000 7f7b5d275640 1 --2- 172.21.15.43:0/3598844304 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f7b44004d00 0x7f7b58070540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T14:51:40.345 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:40.336+0000 7f7b5d275640 1 -- 172.21.15.43:0/3598844304 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7b5815b2f0 con 0x7f7b580b0d80 2023-08-09T14:51:40.345 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:40.337+0000 7f7b5d275640 1 --2- 172.21.15.43:0/3598844304 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f7b580b0d80 0x7f7b580c5b50 secure :-1 s=READY pgs=1827 cs=0 l=1 rev1=1 crypto rx=0x7f7b480108c0 tx=0x7f7b48010d80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-09T14:51:40.345 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:40.337+0000 7f7b4e7fc640 1 -- 172.21.15.43:0/3598844304 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7b48009610 con 0x7f7b580b0d80 2023-08-09T14:51:40.345 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:40.337+0000 7f7b4e7fc640 1 -- 172.21.15.43:0/3598844304 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7b4800eb60 con 0x7f7b580b0d80 2023-08-09T14:51:40.345 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:40.337+0000 7f7b4e7fc640 1 -- 172.21.15.43:0/3598844304 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7b4801c400 con 0x7f7b580b0d80 2023-08-09T14:51:40.346 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:40.337+0000 7f7b5fcda640 1 -- 172.21.15.43:0/3598844304 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7b5815b5d0 con 0x7f7b580b0d80 2023-08-09T14:51:40.346 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:40.337+0000 7f7b5fcda640 1 -- 172.21.15.43:0/3598844304 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7b5815bb90 con 0x7f7b580b0d80 2023-08-09T14:51:40.346 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:40.338+0000 7f7b5fcda640 1 -- 172.21.15.43:0/3598844304 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7b5815c0b0 con 0x7f7b580b0d80 2023-08-09T14:51:40.346 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:40.339+0000 7f7b4e7fc640 1 -- 172.21.15.43:0/3598844304 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 79) v1 ==== 88443+0+0 (secure 0 0 0) 0x7f7b4801c560 con 0x7f7b580b0d80 2023-08-09T14:51:40.347 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:40.340+0000 7f7b4e7fc640 1 --2- 172.21.15.43:0/3598844304 >> [v2:172.21.15.203:6832/2566816332,v1:172.21.15.203:6833/2566816332] conn(0x7f7b3c06a1a0 0x7f7b3c06c660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T14:51:40.347 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:40.340+0000 7f7b4e7fc640 1 -- 172.21.15.43:0/3598844304 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(164..164 src has 1..164) v4 ==== 6348+0+0 (secure 0 0 0) 0x7f7b48088d40 con 0x7f7b580b0d80 2023-08-09T14:51:40.347 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:40.341+0000 7f7b5da76640 1 --2- 172.21.15.43:0/3598844304 >> [v2:172.21.15.203:6832/2566816332,v1:172.21.15.203:6833/2566816332] conn(0x7f7b3c06a1a0 0x7f7b3c06c660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T14:51:40.347 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:40.342+0000 7f7b5da76640 1 --2- 172.21.15.43:0/3598844304 >> [v2:172.21.15.203:6832/2566816332,v1:172.21.15.203:6833/2566816332] conn(0x7f7b3c06a1a0 0x7f7b3c06c660 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7f7b54001200 tx=0x7f7b54009040 comp rx=0 tx=0).ready entity=mgr.7765 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-09T14:51:40.347 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:40.344+0000 7f7b4e7fc640 1 -- 172.21.15.43:0/3598844304 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178066 (secure 0 0 0) 0x7f7b5815c0b0 con 0x7f7b580b0d80 2023-08-09T14:51:40.542 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:40.540+0000 7f7b5fcda640 1 -- 172.21.15.43:0/3598844304 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command([{prefix=config set, name=mon_allow_pool_delete}] v 0) v1 -- 0x7f7b5815c0b0 con 0x7f7b580b0d80 2023-08-09T14:51:40.548 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:40.546+0000 7f7b4e7fc640 1 -- 172.21.15.43:0/3598844304 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_allow_pool_delete}]=0 v52)=0 v52) v1 ==== 122+0+0 (secure 0 0 0) 0x7f7b5815c0b0 con 0x7f7b580b0d80 2023-08-09T14:51:40.551 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:40.549+0000 7f7b5fcda640 1 -- 172.21.15.43:0/3598844304 >> [v2:172.21.15.203:6832/2566816332,v1:172.21.15.203:6833/2566816332] conn(0x7f7b3c06a1a0 msgr2=0x7f7b3c06c660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T14:51:40.551 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:40.549+0000 7f7b5fcda640 1 --2- 172.21.15.43:0/3598844304 >> [v2:172.21.15.203:6832/2566816332,v1:172.21.15.203:6833/2566816332] conn(0x7f7b3c06a1a0 0x7f7b3c06c660 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7f7b54001200 tx=0x7f7b54009040 comp rx=0 tx=0).stop 2023-08-09T14:51:40.551 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:40.549+0000 7f7b5fcda640 1 -- 172.21.15.43:0/3598844304 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f7b580b0d80 msgr2=0x7f7b580c5b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T14:51:40.551 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:40.549+0000 7f7b5fcda640 1 --2- 172.21.15.43:0/3598844304 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f7b580b0d80 0x7f7b580c5b50 secure :-1 s=READY pgs=1827 cs=0 l=1 rev1=1 crypto rx=0x7f7b480108c0 tx=0x7f7b48010d80 comp rx=0 tx=0).stop 2023-08-09T14:51:40.552 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:40.549+0000 7f7b5fcda640 1 -- 172.21.15.43:0/3598844304 shutdown_connections 2023-08-09T14:51:40.552 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:40.549+0000 7f7b5fcda640 1 --2- 172.21.15.43:0/3598844304 >> [v2:172.21.15.203:6832/2566816332,v1:172.21.15.203:6833/2566816332] conn(0x7f7b3c06a1a0 0x7f7b3c06c660 unknown :-1 s=CLOSED pgs=44 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T14:51:40.552 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:40.549+0000 7f7b5fcda640 1 --2- 172.21.15.43:0/3598844304 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f7b580b1200 0x7f7b5815c390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T14:51:40.552 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:40.549+0000 7f7b5fcda640 1 --2- 172.21.15.43:0/3598844304 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f7b580b0d80 0x7f7b580c5b50 unknown :-1 s=CLOSED pgs=1827 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T14:51:40.552 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:40.549+0000 7f7b5fcda640 1 --2- 172.21.15.43:0/3598844304 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f7b44004d00 0x7f7b58070540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T14:51:40.552 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:40.549+0000 7f7b5fcda640 1 -- 172.21.15.43:0/3598844304 >> 172.21.15.43:0/3598844304 conn(0x7f7b580ce0a0 msgr2=0x7f7b580c8ad0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-09T14:51:40.552 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:40.549+0000 7f7b5fcda640 1 -- 172.21.15.43:0/3598844304 shutdown_connections 2023-08-09T14:51:40.553 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:40.549+0000 7f7b5fcda640 1 -- 172.21.15.43:0/3598844304 wait complete. 2023-08-09T14:51:40.570 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2023-08-09T14:51:40.571 DEBUG:teuthology.orchestra.run.smithi043:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2023-08-09T14:51:40.696 DEBUG:teuthology.orchestra.run.smithi203:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2023-08-09T14:51:40.819 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2023-08-09T14:51:40.820 DEBUG:teuthology.orchestra.run.smithi043:> set -ex 2023-08-09T14:51:40.820 DEBUG:teuthology.orchestra.run.smithi043:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-09T14:51:40.884 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4054441,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6498844k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=27,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3249420k,nr_inodes=812355,mode=700,uid=1000,gid=1275 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6498844k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2023-08-09T14:51:40.884 DEBUG:teuthology.orchestra.run.smithi203:> set -ex 2023-08-09T14:51:40.885 DEBUG:teuthology.orchestra.run.smithi203:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-09T14:51:40.946 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4053648,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497576k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=27,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248784k,nr_inodes=812196,mode=700,uid=1000,gid=1275 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497576k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2023-08-09T14:51:40.947 DEBUG:teuthology.orchestra.run.smithi043:> set -ex 2023-08-09T14:51:40.947 DEBUG:teuthology.orchestra.run.smithi043:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-09T14:51:41.015 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4054441,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6498844k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=27,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3249420k,nr_inodes=812355,mode=700,uid=1000,gid=1275 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6498844k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2023-08-09T14:51:41.015 DEBUG:teuthology.orchestra.run.smithi203:> set -ex 2023-08-09T14:51:41.015 DEBUG:teuthology.orchestra.run.smithi203:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-09T14:51:41.077 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4053648,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497576k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=27,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248784k,nr_inodes=812196,mode=700,uid=1000,gid=1275 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497576k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2023-08-09T14:51:41.078 INFO:tasks.cephfs.cephfs_test_case:[, , , ] 2023-08-09T14:51:41.079 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-08-09T14:51:41.320 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:41.318+0000 7fc5b1a04640 1 Processor -- start 2023-08-09T14:51:41.320 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:41.319+0000 7fc5b1a04640 1 -- start start 2023-08-09T14:51:41.322 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:41.320+0000 7fc5b1a04640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7fc5ac0cc380 0x7fc5ac160f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T14:51:41.322 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:41.321+0000 7fc5b1a04640 1 -- --> v1:172.21.15.43:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc5ac0c38a0 con 0x7fc5ac14a5c0 2023-08-09T14:51:41.323 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:41.321+0000 7fc5b1a04640 1 -- --> v1:172.21.15.203:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc5ac0c3b80 con 0x7fc5ac0c3130 2023-08-09T14:51:41.323 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:41.321+0000 7fc5b1a04640 1 -- --> v2:172.21.15.43:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc5ac0c4420 con 0x7fc5ac0cc380 2023-08-09T14:51:41.323 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:41.321+0000 7fc5aaffd640 1 --1- >> v1:172.21.15.43:6789/0 conn(0x7fc5ac14a5c0 0x7fc5ac0cbc70 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.43:6789/0 says I am v1:172.21.15.43:46660/0 (socket says 172.21.15.43:46660) 2023-08-09T14:51:41.323 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:41.321+0000 7fc5aaffd640 1 -- 172.21.15.43:0/2632553138 learned_addr learned my addr 172.21.15.43:0/2632553138 (peer_addr_for_me v1:172.21.15.43:0/0) 2023-08-09T14:51:41.325 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:41.321+0000 7fc5a3fff640 1 --2- 172.21.15.43:0/2632553138 >> v2:172.21.15.43:3300/0 conn(0x7fc5ac0cc380 0x7fc5ac160f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T14:51:41.326 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:41.322+0000 7fc5aa7fc640 1 -- 172.21.15.43:0/2632553138 <== mon.0 v1:172.21.15.43:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3287717332 0 0) 0x7fc5ac0c38a0 con 0x7fc5ac14a5c0 2023-08-09T14:51:41.326 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:41.322+0000 7fc5aa7fc640 1 -- 172.21.15.43:0/2632553138 --> v1:172.21.15.43:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fc598003520 con 0x7fc5ac14a5c0 2023-08-09T14:51:41.326 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:41.322+0000 7fc5a3fff640 1 -- 172.21.15.43:0/2632553138 >> v1:172.21.15.203:6789/0 conn(0x7fc5ac0c3130 legacy=0x7fc5ac06bb50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T14:51:41.326 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:41.322+0000 7fc5a3fff640 1 -- 172.21.15.43:0/2632553138 >> v1:172.21.15.43:6789/0 conn(0x7fc5ac14a5c0 legacy=0x7fc5ac0cbc70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T14:51:41.327 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:41.322+0000 7fc5a3fff640 1 -- 172.21.15.43:0/2632553138 --> v2:172.21.15.43:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc5ac0c3e60 con 0x7fc5ac0cc380 2023-08-09T14:51:41.327 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:41.323+0000 7fc5a3fff640 1 --2- 172.21.15.43:0/2632553138 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc5ac0cc380 0x7fc5ac160f20 secure :-1 s=READY pgs=1829 cs=0 l=1 rev1=1 crypto rx=0x7fc5940097a0 tx=0x7fc59402b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3a013a1c65182089 server_cookie=0 in_seq=0 out_seq=0 2023-08-09T14:51:41.327 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:41.323+0000 7fc5aa7fc640 1 -- 172.21.15.43:0/2632553138 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc594031830 con 0x7fc5ac0cc380 2023-08-09T14:51:41.327 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:41.323+0000 7fc5aa7fc640 1 -- 172.21.15.43:0/2632553138 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc594031990 con 0x7fc5ac0cc380 2023-08-09T14:51:41.327 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:41.323+0000 7fc5aa7fc640 1 -- 172.21.15.43:0/2632553138 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc594031d00 con 0x7fc5ac0cc380 2023-08-09T14:51:41.328 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:41.324+0000 7fc5b1a04640 1 -- 172.21.15.43:0/2632553138 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc5ac0cc380 msgr2=0x7fc5ac160f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T14:51:41.328 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:41.324+0000 7fc5b1a04640 1 --2- 172.21.15.43:0/2632553138 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc5ac0cc380 0x7fc5ac160f20 secure :-1 s=READY pgs=1829 cs=0 l=1 rev1=1 crypto rx=0x7fc5940097a0 tx=0x7fc59402b8d0 comp rx=0 tx=0).stop 2023-08-09T14:51:41.328 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:41.324+0000 7fc5b1a04640 1 -- 172.21.15.43:0/2632553138 shutdown_connections 2023-08-09T14:51:41.328 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:41.324+0000 7fc5b1a04640 1 --2- 172.21.15.43:0/2632553138 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc5ac0cc380 0x7fc5ac160f20 unknown :-1 s=CLOSED pgs=1829 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T14:51:41.329 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:41.324+0000 7fc5b1a04640 1 -- 172.21.15.43:0/2632553138 >> 172.21.15.43:0/2632553138 conn(0x7fc5ac0ce0a0 msgr2=0x7fc5ac0ce480 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-09T14:51:41.329 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:41.324+0000 7fc5b1a04640 1 -- 172.21.15.43:0/2632553138 shutdown_connections 2023-08-09T14:51:41.329 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:41.324+0000 7fc5b1a04640 1 -- 172.21.15.43:0/2632553138 wait complete. 2023-08-09T14:51:41.332 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:41.326+0000 7fc5b1a04640 1 Processor -- start 2023-08-09T14:51:41.332 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:41.326+0000 7fc5b1a04640 1 -- start start 2023-08-09T14:51:41.332 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:41.327+0000 7fc5b1a04640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fc5ac0c3130 0x7fc5ac1465d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T14:51:41.333 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:41.327+0000 7fc5b1a04640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc5ac0cc380 0x7fc5ac0b0d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T14:51:41.333 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:41.327+0000 7fc5b1a04640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fc5ac14a5c0 0x7fc5ac164290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T14:51:41.333 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:41.327+0000 7fc5b1a04640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc5ac0c6b70 con 0x7fc5ac0cc380 2023-08-09T14:51:41.333 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:41.327+0000 7fc5b1a04640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc5ac0b12c0 con 0x7fc5ac0c3130 2023-08-09T14:51:41.333 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:41.327+0000 7fc5b1a04640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc5ac0b1560 con 0x7fc5ac14a5c0 2023-08-09T14:51:41.334 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:41.327+0000 7fc5a3fff640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc5ac0cc380 0x7fc5ac0b0d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T14:51:41.334 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:41.327+0000 7fc5a3fff640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc5ac0cc380 0x7fc5ac0b0d80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:52472/0 (socket says 172.21.15.43:52472) 2023-08-09T14:51:41.334 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:41.327+0000 7fc5a3fff640 1 -- 172.21.15.43:0/1122758422 learned_addr learned my addr 172.21.15.43:0/1122758422 (peer_addr_for_me v2:172.21.15.43:0/0) 2023-08-09T14:51:41.334 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:41.328+0000 7fc5a3fff640 1 -- 172.21.15.43:0/1122758422 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fc5ac14a5c0 msgr2=0x7fc5ac164290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T14:51:41.334 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:41.328+0000 7fc5a3fff640 1 --2- 172.21.15.43:0/1122758422 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fc5ac14a5c0 0x7fc5ac164290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T14:51:41.334 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:41.328+0000 7fc5a3fff640 1 -- 172.21.15.43:0/1122758422 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fc5ac0c3130 msgr2=0x7fc5ac1465d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T14:51:41.334 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:41.328+0000 7fc5a3fff640 1 --2- 172.21.15.43:0/1122758422 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fc5ac0c3130 0x7fc5ac1465d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T14:51:41.335 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:41.328+0000 7fc5a3fff640 1 -- 172.21.15.43:0/1122758422 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc594009400 con 0x7fc5ac0cc380 2023-08-09T14:51:41.335 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:41.328+0000 7fc5a3fff640 1 --2- 172.21.15.43:0/1122758422 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc5ac0cc380 0x7fc5ac0b0d80 secure :-1 s=READY pgs=1830 cs=0 l=1 rev1=1 crypto rx=0x7fc59402bee0 tx=0x7fc5940046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-09T14:51:41.335 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:41.328+0000 7fc5a8ff9640 1 -- 172.21.15.43:0/1122758422 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc59403a6f0 con 0x7fc5ac0cc380 2023-08-09T14:51:41.335 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:41.328+0000 7fc5a8ff9640 1 -- 172.21.15.43:0/1122758422 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc59403a850 con 0x7fc5ac0cc380 2023-08-09T14:51:41.335 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:41.328+0000 7fc5a8ff9640 1 -- 172.21.15.43:0/1122758422 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc59403ac00 con 0x7fc5ac0cc380 2023-08-09T14:51:41.335 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:41.328+0000 7fc5b1a04640 1 -- 172.21.15.43:0/1122758422 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc5ac0b1840 con 0x7fc5ac0cc380 2023-08-09T14:51:41.335 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:41.329+0000 7fc5b1a04640 1 -- 172.21.15.43:0/1122758422 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc5ac0b1e00 con 0x7fc5ac0cc380 2023-08-09T14:51:41.335 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:41.330+0000 7fc5b1a04640 1 -- 172.21.15.43:0/1122758422 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc5ac0c6d20 con 0x7fc5ac0cc380 2023-08-09T14:51:41.336 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:41.331+0000 7fc5a8ff9640 1 -- 172.21.15.43:0/1122758422 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 79) v1 ==== 88443+0+0 (secure 0 0 0) 0x7fc594030380 con 0x7fc5ac0cc380 2023-08-09T14:51:41.336 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:41.331+0000 7fc5a8ff9640 1 --2- 172.21.15.43:0/1122758422 >> [v2:172.21.15.203:6832/2566816332,v1:172.21.15.203:6833/2566816332] conn(0x7fc58c06a170 0x7fc58c06c630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T14:51:41.337 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:41.332+0000 7fc5a8ff9640 1 -- 172.21.15.43:0/1122758422 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(164..164 src has 1..164) v4 ==== 6348+0+0 (secure 0 0 0) 0x7fc5940aa280 con 0x7fc5ac0cc380 2023-08-09T14:51:41.337 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:41.332+0000 7fc5aaffd640 1 --2- 172.21.15.43:0/1122758422 >> [v2:172.21.15.203:6832/2566816332,v1:172.21.15.203:6833/2566816332] conn(0x7fc58c06a170 0x7fc58c06c630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T14:51:41.337 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:41.333+0000 7fc5aaffd640 1 --2- 172.21.15.43:0/1122758422 >> [v2:172.21.15.203:6832/2566816332,v1:172.21.15.203:6833/2566816332] conn(0x7fc58c06a170 0x7fc58c06c630 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7fc59c001570 tx=0x7fc59c007930 comp rx=0 tx=0).ready entity=mgr.7765 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-09T14:51:41.337 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:41.336+0000 7fc5a8ff9640 1 -- 172.21.15.43:0/1122758422 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178066 (secure 0 0 0) 0x7fc5ac0c6d20 con 0x7fc5ac0cc380 2023-08-09T14:51:41.575 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:41.573+0000 7fc5b1a04640 1 -- 172.21.15.43:0/1122758422 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fc5ac0c4560 con 0x7fc5ac0cc380 2023-08-09T14:51:41.575 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:41.574+0000 7fc5a8ff9640 1 -- 172.21.15.43:0/1122758422 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 72 v72) v1 ==== 94+0+3705 (secure 0 0 0) 0x7fc594048e90 con 0x7fc5ac0cc380 2023-08-09T14:51:41.576 INFO:teuthology.orchestra.run.smithi043.stdout: 2023-08-09T14:51:41.576 INFO:teuthology.orchestra.run.smithi043.stdout:{"epoch":72,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6812,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.203:6835/2100579784","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6834","nonce":2100579784},{"type":"v1","addr":"172.21.15.203:6835","nonce":2100579784}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":65},{"gid":6815,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.203:6837/1689128061","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6836","nonce":1689128061},{"type":"v1","addr":"172.21.15.203:6837","nonce":1689128061}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":65},{"gid":7543,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.43:6835/1126540259","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6833","nonce":1126540259},{"type":"v1","addr":"172.21.15.43:6835","nonce":1126540259}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":65},{"gid":9549,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.43:6834/3955419003","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6832","nonce":3955419003},{"type":"v1","addr":"172.21.15.43:6834","nonce":3955419003}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":72}],"filesystems":[]} 2023-08-09T14:51:41.577 INFO:teuthology.orchestra.run.smithi043.stderr:dumped fsmap epoch 72 2023-08-09T14:51:41.578 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:41.577+0000 7fc5b1a04640 1 -- 172.21.15.43:0/1122758422 >> [v2:172.21.15.203:6832/2566816332,v1:172.21.15.203:6833/2566816332] conn(0x7fc58c06a170 msgr2=0x7fc58c06c630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T14:51:41.579 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:41.577+0000 7fc5b1a04640 1 --2- 172.21.15.43:0/1122758422 >> [v2:172.21.15.203:6832/2566816332,v1:172.21.15.203:6833/2566816332] conn(0x7fc58c06a170 0x7fc58c06c630 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7fc59c001570 tx=0x7fc59c007930 comp rx=0 tx=0).stop 2023-08-09T14:51:41.579 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:41.577+0000 7fc5b1a04640 1 -- 172.21.15.43:0/1122758422 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc5ac0cc380 msgr2=0x7fc5ac0b0d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T14:51:41.579 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:41.577+0000 7fc5b1a04640 1 --2- 172.21.15.43:0/1122758422 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc5ac0cc380 0x7fc5ac0b0d80 secure :-1 s=READY pgs=1830 cs=0 l=1 rev1=1 crypto rx=0x7fc59402bee0 tx=0x7fc5940046c0 comp rx=0 tx=0).stop 2023-08-09T14:51:41.579 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:41.577+0000 7fc5b1a04640 1 -- 172.21.15.43:0/1122758422 shutdown_connections 2023-08-09T14:51:41.579 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:41.577+0000 7fc5b1a04640 1 --2- 172.21.15.43:0/1122758422 >> [v2:172.21.15.203:6832/2566816332,v1:172.21.15.203:6833/2566816332] conn(0x7fc58c06a170 0x7fc58c06c630 unknown :-1 s=CLOSED pgs=45 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T14:51:41.579 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:41.577+0000 7fc5b1a04640 1 --2- 172.21.15.43:0/1122758422 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fc5ac14a5c0 0x7fc5ac164290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T14:51:41.579 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:41.577+0000 7fc5b1a04640 1 --2- 172.21.15.43:0/1122758422 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc5ac0cc380 0x7fc5ac0b0d80 unknown :-1 s=CLOSED pgs=1830 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T14:51:41.580 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:41.577+0000 7fc5b1a04640 1 --2- 172.21.15.43:0/1122758422 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fc5ac0c3130 0x7fc5ac1465d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T14:51:41.580 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:41.577+0000 7fc5b1a04640 1 -- 172.21.15.43:0/1122758422 >> 172.21.15.43:0/1122758422 conn(0x7fc5ac0ce0a0 msgr2=0x7fc5ac0ccab0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-09T14:51:41.580 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:41.577+0000 7fc5b1a04640 1 -- 172.21.15.43:0/1122758422 shutdown_connections 2023-08-09T14:51:41.580 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:41.577+0000 7fc5b1a04640 1 -- 172.21.15.43:0/1122758422 wait complete. 2023-08-09T14:51:41.598 INFO:tasks.ceph.mds.a:Restarting daemon 2023-08-09T14:51:41.598 INFO:tasks.ceph.mds.a:Stopping old one... 2023-08-09T14:51:41.598 DEBUG:tasks.ceph.mds.a:waiting for process to exit 2023-08-09T14:51:41.598 INFO:teuthology.orchestra.run:waiting for 300 2023-08-09T14:51:41.599 INFO:tasks.ceph.mds.c:Restarting daemon 2023-08-09T14:51:41.599 INFO:tasks.ceph.mds.c:Stopping old one... 2023-08-09T14:51:41.599 DEBUG:tasks.ceph.mds.c:waiting for process to exit 2023-08-09T14:51:41.599 INFO:teuthology.orchestra.run:waiting for 300 2023-08-09T14:51:41.600 INFO:tasks.ceph.mds.b:Restarting daemon 2023-08-09T14:51:41.600 INFO:tasks.ceph.mds.b:Stopping old one... 2023-08-09T14:51:41.600 DEBUG:tasks.ceph.mds.b:waiting for process to exit 2023-08-09T14:51:41.601 INFO:teuthology.orchestra.run:waiting for 300 2023-08-09T14:51:41.601 INFO:tasks.ceph.mds.d:Restarting daemon 2023-08-09T14:51:41.602 INFO:tasks.ceph.mds.d:Stopping old one... 2023-08-09T14:51:41.602 DEBUG:tasks.ceph.mds.d:waiting for process to exit 2023-08-09T14:51:41.602 INFO:teuthology.orchestra.run:waiting for 300 2023-08-09T14:51:41.617 INFO:tasks.ceph.mds.c:Stopped 2023-08-09T14:51:41.618 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i c 2023-08-09T14:51:41.620 INFO:tasks.ceph.mds.a:Stopped 2023-08-09T14:51:41.620 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i a 2023-08-09T14:51:41.626 INFO:tasks.ceph.mds.d:Stopped 2023-08-09T14:51:41.626 DEBUG:teuthology.orchestra.run.smithi203:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i d 2023-08-09T14:51:41.636 INFO:tasks.ceph.mds.b:Stopped 2023-08-09T14:51:41.637 DEBUG:teuthology.orchestra.run.smithi203:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i b 2023-08-09T14:51:41.668 INFO:tasks.ceph.mds.c:Started 2023-08-09T14:51:41.669 DEBUG:teuthology.parallel:result is None 2023-08-09T14:51:41.676 INFO:tasks.ceph.mds.a:Started 2023-08-09T14:51:41.677 DEBUG:teuthology.parallel:result is None 2023-08-09T14:51:41.684 INFO:tasks.ceph.mds.d:Started 2023-08-09T14:51:41.685 DEBUG:teuthology.parallel:result is None 2023-08-09T14:51:41.696 INFO:tasks.ceph.mds.b:Started 2023-08-09T14:51:41.696 DEBUG:teuthology.parallel:result is None 2023-08-09T14:51:41.697 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist clear 2023-08-09T14:51:41.835 INFO:tasks.ceph.mds.c.smithi043.stdout:starting mds.c at 2023-08-09T14:51:41.845 INFO:tasks.ceph.mds.a.smithi043.stdout:starting mds.a at 2023-08-09T14:51:41.855 INFO:tasks.ceph.mds.b.smithi203.stdout:starting mds.b at 2023-08-09T14:51:41.857 INFO:tasks.ceph.mds.d.smithi203.stdout:starting mds.d at 2023-08-09T14:51:41.902 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:41.901+0000 7f923fb0f640 1 Processor -- start 2023-08-09T14:51:41.903 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:41.901+0000 7f923fb0f640 1 -- start start 2023-08-09T14:51:41.904 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:41.902+0000 7f923fb0f640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f9238143c20 0x7f923806f540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T14:51:41.904 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:41.903+0000 7f923fb0f640 1 -- --> v1:172.21.15.43:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f923806c440 con 0x7f92380c86f0 2023-08-09T14:51:41.904 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:41.903+0000 7f923fb0f640 1 -- --> v1:172.21.15.203:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f923806b830 con 0x7f923806cfe0 2023-08-09T14:51:41.905 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:41.903+0000 7f923fb0f640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9238144000 con 0x7f9238143c20 2023-08-09T14:51:41.905 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:41.903+0000 7f923d0aa640 1 --1- >> v1:172.21.15.43:6789/0 conn(0x7f92380c86f0 0x7f923806c8d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.43:6789/0 says I am v1:172.21.15.43:46690/0 (socket says 172.21.15.43:46690) 2023-08-09T14:51:41.905 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:41.903+0000 7f923d0aa640 1 -- 172.21.15.43:0/409226218 learned_addr learned my addr 172.21.15.43:0/409226218 (peer_addr_for_me v1:172.21.15.43:0/0) 2023-08-09T14:51:41.905 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:41.903+0000 7f923d8ab640 1 --2- 172.21.15.43:0/409226218 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f9238143c20 0x7f923806f540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T14:51:41.906 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:41.903+0000 7f923c8a9640 1 -- 172.21.15.43:0/409226218 <== mon.0 v1:172.21.15.43:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3902012055 0 0) 0x7f923806c440 con 0x7f92380c86f0 2023-08-09T14:51:41.906 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:41.903+0000 7f923c8a9640 1 -- 172.21.15.43:0/409226218 --> v1:172.21.15.43:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f9224003520 con 0x7f92380c86f0 2023-08-09T14:51:41.906 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:41.904+0000 7f923d8ab640 1 -- 172.21.15.43:0/409226218 >> v1:172.21.15.203:6789/0 conn(0x7f923806cfe0 legacy=0x7f92380ccc00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T14:51:41.906 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:41.904+0000 7f923d8ab640 1 -- 172.21.15.43:0/409226218 >> v1:172.21.15.43:6789/0 conn(0x7f92380c86f0 legacy=0x7f923806c8d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T14:51:41.906 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:41.904+0000 7f923d8ab640 1 -- 172.21.15.43:0/409226218 --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f923806bb10 con 0x7f9238143c20 2023-08-09T14:51:41.907 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:41.905+0000 7f923d8ab640 1 --2- 172.21.15.43:0/409226218 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f9238143c20 0x7f923806f540 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7f92340097d0 tx=0x7f923402b8b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b42280e1b9efad4b server_cookie=0 in_seq=0 out_seq=0 2023-08-09T14:51:41.907 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:41.905+0000 7f923c8a9640 1 -- 172.21.15.43:0/409226218 <== mon.2 v2:172.21.15.203:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f923402d040 con 0x7f9238143c20 2023-08-09T14:51:41.907 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:41.905+0000 7f923c8a9640 1 -- 172.21.15.43:0/409226218 <== mon.2 v2:172.21.15.203:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9234031810 con 0x7f9238143c20 2023-08-09T14:51:41.908 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:41.905+0000 7f923c8a9640 1 -- 172.21.15.43:0/409226218 <== mon.2 v2:172.21.15.203:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9234031b80 con 0x7f9238143c20 2023-08-09T14:51:41.908 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:41.906+0000 7f923fb0f640 1 -- 172.21.15.43:0/409226218 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f9238143c20 msgr2=0x7f923806f540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T14:51:41.908 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:41.906+0000 7f923fb0f640 1 --2- 172.21.15.43:0/409226218 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f9238143c20 0x7f923806f540 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7f92340097d0 tx=0x7f923402b8b0 comp rx=0 tx=0).stop 2023-08-09T14:51:41.908 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:41.906+0000 7f923fb0f640 1 -- 172.21.15.43:0/409226218 shutdown_connections 2023-08-09T14:51:41.909 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:41.906+0000 7f923fb0f640 1 --2- 172.21.15.43:0/409226218 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f9238143c20 0x7f923806f540 unknown :-1 s=CLOSED pgs=251 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T14:51:41.909 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:41.906+0000 7f923fb0f640 1 -- 172.21.15.43:0/409226218 >> 172.21.15.43:0/409226218 conn(0x7f92380ce0a0 msgr2=0x7f92380ce480 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-09T14:51:41.909 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:41.906+0000 7f923fb0f640 1 -- 172.21.15.43:0/409226218 shutdown_connections 2023-08-09T14:51:41.909 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:41.906+0000 7f923fb0f640 1 -- 172.21.15.43:0/409226218 wait complete. 2023-08-09T14:51:41.909 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:41.908+0000 7f923fb0f640 1 Processor -- start 2023-08-09T14:51:41.910 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:41.908+0000 7f923fb0f640 1 -- start start 2023-08-09T14:51:41.911 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:41.909+0000 7f923fb0f640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f923806cfe0 0x7f92380b38d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T14:51:41.911 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:41.909+0000 7f923fb0f640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f92380c86f0 0x7f92380c7c30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T14:51:41.911 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:41.909+0000 7f923fb0f640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f9238143c20 0x7f923815c010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T14:51:41.912 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:41.910+0000 7f923fb0f640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f92381403a0 con 0x7f92380c86f0 2023-08-09T14:51:41.912 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:41.910+0000 7f923fb0f640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f92380b1b10 con 0x7f923806cfe0 2023-08-09T14:51:41.912 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:41.910+0000 7f923fb0f640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f92380b0f00 con 0x7f9238143c20 2023-08-09T14:51:41.912 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:41.910+0000 7f923d0aa640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f92380c86f0 0x7f92380c7c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T14:51:41.912 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:41.910+0000 7f923d0aa640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f92380c86f0 0x7f92380c7c30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:52506/0 (socket says 172.21.15.43:52506) 2023-08-09T14:51:41.912 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:41.910+0000 7f923d0aa640 1 -- 172.21.15.43:0/2224210962 learned_addr learned my addr 172.21.15.43:0/2224210962 (peer_addr_for_me v2:172.21.15.43:0/0) 2023-08-09T14:51:41.912 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:41.910+0000 7f923e0ac640 1 --2- 172.21.15.43:0/2224210962 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f9238143c20 0x7f923815c010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T14:51:41.913 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:41.910+0000 7f923d8ab640 1 --2- 172.21.15.43:0/2224210962 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f923806cfe0 0x7f92380b38d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T14:51:41.914 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:41.910+0000 7f923d0aa640 1 -- 172.21.15.43:0/2224210962 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f9238143c20 msgr2=0x7f923815c010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T14:51:41.914 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:41.910+0000 7f923d0aa640 1 --2- 172.21.15.43:0/2224210962 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f9238143c20 0x7f923815c010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T14:51:41.914 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:41.910+0000 7f923d0aa640 1 -- 172.21.15.43:0/2224210962 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f923806cfe0 msgr2=0x7f92380b38d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T14:51:41.914 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:41.910+0000 7f923d0aa640 1 --2- 172.21.15.43:0/2224210962 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f923806cfe0 0x7f92380b38d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T14:51:41.914 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:41.910+0000 7f923d0aa640 1 -- 172.21.15.43:0/2224210962 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9234009400 con 0x7f92380c86f0 2023-08-09T14:51:41.914 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:41.911+0000 7f923d0aa640 1 --2- 172.21.15.43:0/2224210962 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f92380c86f0 0x7f92380c7c30 secure :-1 s=READY pgs=1839 cs=0 l=1 rev1=1 crypto rx=0x7f922800c500 tx=0x7f922800c9c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-09T14:51:41.914 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:41.911+0000 7f922e7fc640 1 -- 172.21.15.43:0/2224210962 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9228005ce0 con 0x7f92380c86f0 2023-08-09T14:51:41.915 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:41.911+0000 7f922e7fc640 1 -- 172.21.15.43:0/2224210962 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9228005e40 con 0x7f92380c86f0 2023-08-09T14:51:41.915 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:41.911+0000 7f922e7fc640 1 -- 172.21.15.43:0/2224210962 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9228007330 con 0x7f92380c86f0 2023-08-09T14:51:41.915 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:41.911+0000 7f923fb0f640 1 -- 172.21.15.43:0/2224210962 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f92380b11e0 con 0x7f92380c86f0 2023-08-09T14:51:41.915 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:41.911+0000 7f923fb0f640 1 -- 172.21.15.43:0/2224210962 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f92380b17a0 con 0x7f92380c86f0 2023-08-09T14:51:41.915 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:41.912+0000 7f923fb0f640 1 -- 172.21.15.43:0/2224210962 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f92380b1cc0 con 0x7f92380c86f0 2023-08-09T14:51:41.916 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:41.913+0000 7f922e7fc640 1 -- 172.21.15.43:0/2224210962 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 79) v1 ==== 88443+0+0 (secure 0 0 0) 0x7f9228007550 con 0x7f92380c86f0 2023-08-09T14:51:41.916 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:41.914+0000 7f922e7fc640 1 --2- 172.21.15.43:0/2224210962 >> [v2:172.21.15.203:6832/2566816332,v1:172.21.15.203:6833/2566816332] conn(0x7f921c06a170 0x7f921c06c630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T14:51:41.916 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:41.914+0000 7f922e7fc640 1 -- 172.21.15.43:0/2224210962 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(164..164 src has 1..164) v4 ==== 6348+0+0 (secure 0 0 0) 0x7f9228087e10 con 0x7f92380c86f0 2023-08-09T14:51:41.916 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:41.915+0000 7f923d8ab640 1 --2- 172.21.15.43:0/2224210962 >> [v2:172.21.15.203:6832/2566816332,v1:172.21.15.203:6833/2566816332] conn(0x7f921c06a170 0x7f921c06c630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T14:51:41.917 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:41.916+0000 7f923d8ab640 1 --2- 172.21.15.43:0/2224210962 >> [v2:172.21.15.203:6832/2566816332,v1:172.21.15.203:6833/2566816332] conn(0x7f921c06a170 0x7f921c06c630 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7f9234036000 tx=0x7f923403c040 comp rx=0 tx=0).ready entity=mgr.7765 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-09T14:51:41.920 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:41.918+0000 7f922e7fc640 1 -- 172.21.15.43:0/2224210962 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178066 (secure 0 0 0) 0x7f92380b1cc0 con 0x7f92380c86f0 2023-08-09T14:51:42.134 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:42.133+0000 7f923fb0f640 1 -- 172.21.15.43:0/2224210962 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "osd blocklist clear"} v 0) v1 -- 0x7f923806c0d0 con 0x7f92380c86f0 2023-08-09T14:51:42.726 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:42.724+0000 7f922e7fc640 1 -- 172.21.15.43:0/2224210962 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist clear"}]=0 removed all blocklist entries v166) v1 ==== 97+0+0 (secure 0 0 0) 0x7f9228017020 con 0x7f92380c86f0 2023-08-09T14:51:42.726 INFO:teuthology.orchestra.run.smithi043.stderr: removed all blocklist entries 2023-08-09T14:51:42.730 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:42.728+0000 7f923fb0f640 1 -- 172.21.15.43:0/2224210962 >> [v2:172.21.15.203:6832/2566816332,v1:172.21.15.203:6833/2566816332] conn(0x7f921c06a170 msgr2=0x7f921c06c630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T14:51:42.730 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:42.728+0000 7f923fb0f640 1 --2- 172.21.15.43:0/2224210962 >> [v2:172.21.15.203:6832/2566816332,v1:172.21.15.203:6833/2566816332] conn(0x7f921c06a170 0x7f921c06c630 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7f9234036000 tx=0x7f923403c040 comp rx=0 tx=0).stop 2023-08-09T14:51:42.730 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:42.729+0000 7f923fb0f640 1 -- 172.21.15.43:0/2224210962 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f92380c86f0 msgr2=0x7f92380c7c30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T14:51:42.731 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:42.729+0000 7f923fb0f640 1 --2- 172.21.15.43:0/2224210962 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f92380c86f0 0x7f92380c7c30 secure :-1 s=READY pgs=1839 cs=0 l=1 rev1=1 crypto rx=0x7f922800c500 tx=0x7f922800c9c0 comp rx=0 tx=0).stop 2023-08-09T14:51:42.731 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:42.729+0000 7f923fb0f640 1 -- 172.21.15.43:0/2224210962 shutdown_connections 2023-08-09T14:51:42.731 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:42.729+0000 7f923fb0f640 1 --2- 172.21.15.43:0/2224210962 >> [v2:172.21.15.203:6832/2566816332,v1:172.21.15.203:6833/2566816332] conn(0x7f921c06a170 0x7f921c06c630 unknown :-1 s=CLOSED pgs=46 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T14:51:42.731 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:42.729+0000 7f923fb0f640 1 --2- 172.21.15.43:0/2224210962 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f9238143c20 0x7f923815c010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T14:51:42.732 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:42.729+0000 7f923fb0f640 1 --2- 172.21.15.43:0/2224210962 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f92380c86f0 0x7f92380c7c30 unknown :-1 s=CLOSED pgs=1839 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T14:51:42.732 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:42.729+0000 7f923fb0f640 1 --2- 172.21.15.43:0/2224210962 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f923806cfe0 0x7f92380b38d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T14:51:42.732 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:42.729+0000 7f923fb0f640 1 -- 172.21.15.43:0/2224210962 >> 172.21.15.43:0/2224210962 conn(0x7f92380ce0a0 msgr2=0x7f9238070ce0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-09T14:51:42.732 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:42.729+0000 7f923fb0f640 1 -- 172.21.15.43:0/2224210962 shutdown_connections 2023-08-09T14:51:42.732 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:42.729+0000 7f923fb0f640 1 -- 172.21.15.43:0/2224210962 wait complete. 2023-08-09T14:51:42.755 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth ls --format=json-pretty 2023-08-09T14:51:42.985 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:42.983+0000 7fa9688cd640 1 Processor -- start 2023-08-09T14:51:42.985 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:42.983+0000 7fa9688cd640 1 -- start start 2023-08-09T14:51:42.988 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:42.985+0000 7fa9688cd640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fa9640b1050 0x7fa9640b4760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T14:51:42.988 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:42.985+0000 7fa9688cd640 1 -- --> v1:172.21.15.43:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa9640b7bf0 con 0x7fa9640cd6b0 2023-08-09T14:51:42.989 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:42.985+0000 7fa9688cd640 1 -- --> v1:172.21.15.203:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa9640b7ed0 con 0x7fa96415f7b0 2023-08-09T14:51:42.989 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:42.985+0000 7fa9688cd640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa9640d0370 con 0x7fa9640b1050 2023-08-09T14:51:42.989 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:42.985+0000 7fa963fff640 1 --1- >> v1:172.21.15.43:6789/0 conn(0x7fa9640cd6b0 0x7fa9640d0e50 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.43:6789/0 says I am v1:172.21.15.43:46706/0 (socket says 172.21.15.43:46706) 2023-08-09T14:51:42.989 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:42.985+0000 7fa963fff640 1 -- 172.21.15.43:0/931750588 learned_addr learned my addr 172.21.15.43:0/931750588 (peer_addr_for_me v1:172.21.15.43:0/0) 2023-08-09T14:51:42.989 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:42.986+0000 7fa962ffd640 1 --2- 172.21.15.43:0/931750588 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fa9640b1050 0x7fa9640b4760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T14:51:42.989 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:42.986+0000 7fa9627fc640 1 -- 172.21.15.43:0/931750588 <== mon.0 v1:172.21.15.43:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1163083404 0 0) 0x7fa9640b7bf0 con 0x7fa9640cd6b0 2023-08-09T14:51:42.989 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:42.986+0000 7fa9627fc640 1 -- 172.21.15.43:0/931750588 --> v1:172.21.15.43:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fa94c003520 con 0x7fa9640cd6b0 2023-08-09T14:51:42.990 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:42.986+0000 7fa9627fc640 1 -- 172.21.15.43:0/931750588 <== mon.0 v1:172.21.15.43:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1426268381 0 0) 0x7fa94c003520 con 0x7fa9640cd6b0 2023-08-09T14:51:42.990 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:42.986+0000 7fa9627fc640 1 -- 172.21.15.43:0/931750588 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fa9640b1050 msgr2=0x7fa9640b4760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-09T14:51:42.990 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:42.986+0000 7fa9627fc640 1 --2- 172.21.15.43:0/931750588 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fa9640b1050 0x7fa9640b4760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T14:51:42.990 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:42.986+0000 7fa9627fc640 1 -- 172.21.15.43:0/931750588 >> v1:172.21.15.203:6789/0 conn(0x7fa96415f7b0 legacy=0x7fa9640ccfa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T14:51:42.990 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:42.986+0000 7fa9627fc640 1 -- 172.21.15.43:0/931750588 --> v1:172.21.15.43:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa9640d0650 con 0x7fa9640cd6b0 2023-08-09T14:51:42.990 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:42.986+0000 7fa9627fc640 1 -- 172.21.15.43:0/931750588 <== mon.0 v1:172.21.15.43:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1285150740 0 0) 0x7fa95c0019f0 con 0x7fa9640cd6b0 2023-08-09T14:51:42.990 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:42.986+0000 7fa9627fc640 1 -- 172.21.15.43:0/931750588 >> v1:172.21.15.43:6789/0 conn(0x7fa9640cd6b0 legacy=0x7fa9640d0e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T14:51:42.991 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:42.986+0000 7fa9627fc640 1 --2- 172.21.15.43:0/931750588 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fa94c0039a0 0x7fa94c003dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T14:51:42.991 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:42.987+0000 7fa9627fc640 1 --2- 172.21.15.43:0/931750588 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fa9640b1050 0x7fa94c0047c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T14:51:42.991 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:42.987+0000 7fa9627fc640 1 --2- 172.21.15.43:0/931750588 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa94c004d00 0x7fa94c005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T14:51:42.992 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:42.987+0000 7fa9627fc640 1 -- 172.21.15.43:0/931750588 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa9640d0370 con 0x7fa94c004d00 2023-08-09T14:51:42.992 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:42.987+0000 7fa9627fc640 1 -- 172.21.15.43:0/931750588 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa94c00ba30 con 0x7fa94c0039a0 2023-08-09T14:51:42.992 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:42.987+0000 7fa9627fc640 1 -- 172.21.15.43:0/931750588 --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa94c00bbf0 con 0x7fa9640b1050 2023-08-09T14:51:42.992 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:42.987+0000 7fa963fff640 1 --2- 172.21.15.43:0/931750588 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa94c004d00 0x7fa94c005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T14:51:42.993 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:42.987+0000 7fa962ffd640 1 --2- 172.21.15.43:0/931750588 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fa9640b1050 0x7fa94c0047c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T14:51:42.994 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:42.987+0000 7fa9637fe640 1 --2- 172.21.15.43:0/931750588 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fa94c0039a0 0x7fa94c003dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T14:51:42.994 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:42.987+0000 7fa963fff640 1 -- 172.21.15.43:0/931750588 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fa9640b1050 msgr2=0x7fa94c0047c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-09T14:51:42.994 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:42.987+0000 7fa963fff640 1 --2- 172.21.15.43:0/931750588 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fa9640b1050 0x7fa94c0047c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T14:51:42.994 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:42.987+0000 7fa963fff640 1 -- 172.21.15.43:0/931750588 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fa94c0039a0 msgr2=0x7fa94c003dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-09T14:51:42.994 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:42.987+0000 7fa963fff640 1 --2- 172.21.15.43:0/931750588 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fa94c0039a0 0x7fa94c003dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T14:51:42.994 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:42.987+0000 7fa963fff640 1 -- 172.21.15.43:0/931750588 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa94c003520 con 0x7fa94c004d00 2023-08-09T14:51:42.995 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:42.988+0000 7fa962ffd640 1 --2- 172.21.15.43:0/931750588 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fa9640b1050 0x7fa94c0047c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-08-09T14:51:42.995 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:42.989+0000 7fa963fff640 1 --2- 172.21.15.43:0/931750588 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa94c004d00 0x7fa94c005180 secure :-1 s=READY pgs=1841 cs=0 l=1 rev1=1 crypto rx=0x7fa95c00a800 tx=0x7fa94c0057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2545cb24fd21513a server_cookie=0 in_seq=0 out_seq=0 2023-08-09T14:51:42.995 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:42.989+0000 7fa9627fc640 1 -- 172.21.15.43:0/931750588 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa95c0072d0 con 0x7fa94c004d00 2023-08-09T14:51:42.995 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:42.989+0000 7fa9627fc640 1 -- 172.21.15.43:0/931750588 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa95c02f070 con 0x7fa94c004d00 2023-08-09T14:51:42.995 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:42.989+0000 7fa9627fc640 1 -- 172.21.15.43:0/931750588 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa95c007690 con 0x7fa94c004d00 2023-08-09T14:51:42.995 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:42.990+0000 7fa9688cd640 1 -- 172.21.15.43:0/931750588 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa94c004d00 msgr2=0x7fa94c005180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T14:51:42.995 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:42.990+0000 7fa9688cd640 1 --2- 172.21.15.43:0/931750588 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa94c004d00 0x7fa94c005180 secure :-1 s=READY pgs=1841 cs=0 l=1 rev1=1 crypto rx=0x7fa95c00a800 tx=0x7fa94c0057f0 comp rx=0 tx=0).stop 2023-08-09T14:51:42.995 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:42.990+0000 7fa963fff640 1 -- 172.21.15.43:0/931750588 reap_dead start 2023-08-09T14:51:42.996 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:42.990+0000 7fa9688cd640 1 -- 172.21.15.43:0/931750588 shutdown_connections 2023-08-09T14:51:42.996 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:42.990+0000 7fa9688cd640 1 -- 172.21.15.43:0/931750588 >> 172.21.15.43:0/931750588 conn(0x7fa96406a2b0 msgr2=0x7fa96406a690 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-09T14:51:42.996 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:42.990+0000 7fa9688cd640 1 -- 172.21.15.43:0/931750588 shutdown_connections 2023-08-09T14:51:42.996 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:42.990+0000 7fa9688cd640 1 -- 172.21.15.43:0/931750588 wait complete. 2023-08-09T14:51:42.996 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:42.992+0000 7fa9688cd640 1 Processor -- start 2023-08-09T14:51:42.996 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:42.992+0000 7fa9688cd640 1 -- start start 2023-08-09T14:51:42.998 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:42.993+0000 7fa9688cd640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fa94c004d00 0x7fa964148210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T14:51:42.998 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:42.993+0000 7fa9688cd640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fa9641f5c60 0x7fa9640c5400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T14:51:42.998 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:42.993+0000 7fa9688cd640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa9640c5940 0x7fa9641f24b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T14:51:42.998 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:42.993+0000 7fa9688cd640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa9641491b0 con 0x7fa9640c5940 2023-08-09T14:51:42.998 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:42.993+0000 7fa9688cd640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa9641f1d50 con 0x7fa94c004d00 2023-08-09T14:51:42.998 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:42.993+0000 7fa9688cd640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa964148910 con 0x7fa9641f5c60 2023-08-09T14:51:42.999 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:42.994+0000 7fa963fff640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa9640c5940 0x7fa9641f24b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T14:51:42.999 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:42.994+0000 7fa963fff640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa9640c5940 0x7fa9641f24b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:52514/0 (socket says 172.21.15.43:52514) 2023-08-09T14:51:42.999 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:42.994+0000 7fa963fff640 1 -- 172.21.15.43:0/2065695460 learned_addr learned my addr 172.21.15.43:0/2065695460 (peer_addr_for_me v2:172.21.15.43:0/0) 2023-08-09T14:51:42.999 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:42.994+0000 7fa962ffd640 1 --2- 172.21.15.43:0/2065695460 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fa9641f5c60 0x7fa9640c5400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T14:51:42.999 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:42.994+0000 7fa9637fe640 1 --2- 172.21.15.43:0/2065695460 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fa94c004d00 0x7fa964148210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T14:51:42.999 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:42.994+0000 7fa963fff640 1 -- 172.21.15.43:0/2065695460 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fa9641f5c60 msgr2=0x7fa9640c5400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T14:51:42.999 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:42.994+0000 7fa963fff640 1 --2- 172.21.15.43:0/2065695460 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fa9641f5c60 0x7fa9640c5400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T14:51:43.000 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:42.994+0000 7fa963fff640 1 -- 172.21.15.43:0/2065695460 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fa94c004d00 msgr2=0x7fa964148210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T14:51:43.000 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:42.994+0000 7fa963fff640 1 --2- 172.21.15.43:0/2065695460 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fa94c004d00 0x7fa964148210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T14:51:43.000 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:42.994+0000 7fa963fff640 1 -- 172.21.15.43:0/2065695460 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa964148bf0 con 0x7fa9640c5940 2023-08-09T14:51:43.000 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:42.994+0000 7fa963fff640 1 --2- 172.21.15.43:0/2065695460 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa9640c5940 0x7fa9641f24b0 secure :-1 s=READY pgs=1842 cs=0 l=1 rev1=1 crypto rx=0x7fa95c00a800 tx=0x7fa95c001d80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-09T14:51:43.000 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:42.995+0000 7fa94bfff640 1 -- 172.21.15.43:0/2065695460 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa95c0019b0 con 0x7fa9640c5940 2023-08-09T14:51:43.000 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:42.995+0000 7fa94bfff640 1 -- 172.21.15.43:0/2065695460 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa95c02ce90 con 0x7fa9640c5940 2023-08-09T14:51:43.000 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:42.995+0000 7fa94bfff640 1 -- 172.21.15.43:0/2065695460 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa95c007310 con 0x7fa9640c5940 2023-08-09T14:51:43.001 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:42.995+0000 7fa9688cd640 1 -- 172.21.15.43:0/2065695460 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa964148ed0 con 0x7fa9640c5940 2023-08-09T14:51:43.001 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:42.995+0000 7fa9688cd640 1 -- 172.21.15.43:0/2065695460 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa9641f60e0 con 0x7fa9640c5940 2023-08-09T14:51:43.001 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:42.996+0000 7fa9688cd640 1 -- 172.21.15.43:0/2065695460 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa9641f21d0 con 0x7fa9640c5940 2023-08-09T14:51:43.002 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:42.997+0000 7fa94bfff640 1 -- 172.21.15.43:0/2065695460 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 79) v1 ==== 88443+0+0 (secure 0 0 0) 0x7fa95c031030 con 0x7fa9640c5940 2023-08-09T14:51:43.002 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:42.998+0000 7fa94bfff640 1 --2- 172.21.15.43:0/2065695460 >> [v2:172.21.15.203:6832/2566816332,v1:172.21.15.203:6833/2566816332] conn(0x7fa95006a1a0 0x7fa95006c660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T14:51:43.002 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:42.998+0000 7fa94bfff640 1 -- 172.21.15.43:0/2065695460 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(168..168 src has 1..168) v4 ==== 5574+0+0 (secure 0 0 0) 0x7fa95c0a9fa0 con 0x7fa9640c5940 2023-08-09T14:51:43.002 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:42.999+0000 7fa9637fe640 1 --2- 172.21.15.43:0/2065695460 >> [v2:172.21.15.203:6832/2566816332,v1:172.21.15.203:6833/2566816332] conn(0x7fa95006a1a0 0x7fa95006c660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T14:51:43.002 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:43.000+0000 7fa9637fe640 1 --2- 172.21.15.43:0/2065695460 >> [v2:172.21.15.203:6832/2566816332,v1:172.21.15.203:6833/2566816332] conn(0x7fa95006a1a0 0x7fa95006c660 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7fa9540016d0 tx=0x7fa954009040 comp rx=0 tx=0).ready entity=mgr.7765 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-09T14:51:43.004 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:43.002+0000 7fa94bfff640 1 -- 172.21.15.43:0/2065695460 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178066 (secure 0 0 0) 0x7fa9641f21d0 con 0x7fa9640c5940 2023-08-09T14:51:43.269 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:43.268+0000 7fa9688cd640 1 -- 172.21.15.43:0/2065695460 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "auth ls", "format": "json-pretty"} v 0) v1 -- 0x7fa9641f21d0 con 0x7fa9640c5940 2023-08-09T14:51:43.270 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:43.269+0000 7fa94bfff640 1 -- 172.21.15.43:0/2065695460 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "auth ls", "format": "json-pretty"}]=0 v39) v1 ==== 80+0+5304 (secure 0 0 0) 0x7fa9641f21d0 con 0x7fa9640c5940 2023-08-09T14:51:43.271 INFO:teuthology.orchestra.run.smithi043.stdout: 2023-08-09T14:51:43.271 INFO:teuthology.orchestra.run.smithi043.stdout:{ 2023-08-09T14:51:43.271 INFO:teuthology.orchestra.run.smithi043.stdout: "auth_dump": [ 2023-08-09T14:51:43.271 INFO:teuthology.orchestra.run.smithi043.stdout: { 2023-08-09T14:51:43.272 INFO:teuthology.orchestra.run.smithi043.stdout: "entity": "mds.a", 2023-08-09T14:51:43.272 INFO:teuthology.orchestra.run.smithi043.stdout: "key": "AQCOldNkPu6GCRAAZecwctvMji0Zubb60KMlyQ==", 2023-08-09T14:51:43.272 INFO:teuthology.orchestra.run.smithi043.stdout: "caps": { 2023-08-09T14:51:43.272 INFO:teuthology.orchestra.run.smithi043.stdout: "mds": "allow", 2023-08-09T14:51:43.272 INFO:teuthology.orchestra.run.smithi043.stdout: "mgr": "allow *", 2023-08-09T14:51:43.272 INFO:teuthology.orchestra.run.smithi043.stdout: "mon": "allow *", 2023-08-09T14:51:43.272 INFO:teuthology.orchestra.run.smithi043.stdout: "osd": "allow *" 2023-08-09T14:51:43.272 INFO:teuthology.orchestra.run.smithi043.stdout: } 2023-08-09T14:51:43.273 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2023-08-09T14:51:43.273 INFO:teuthology.orchestra.run.smithi043.stdout: { 2023-08-09T14:51:43.273 INFO:teuthology.orchestra.run.smithi043.stdout: "entity": "mds.b", 2023-08-09T14:51:43.273 INFO:teuthology.orchestra.run.smithi043.stdout: "key": "AQCOldNkt2+5KxAAHcckB+S5NZHsF5z22StXmw==", 2023-08-09T14:51:43.273 INFO:teuthology.orchestra.run.smithi043.stdout: "caps": { 2023-08-09T14:51:43.273 INFO:teuthology.orchestra.run.smithi043.stdout: "mds": "allow", 2023-08-09T14:51:43.273 INFO:teuthology.orchestra.run.smithi043.stdout: "mgr": "allow *", 2023-08-09T14:51:43.274 INFO:teuthology.orchestra.run.smithi043.stdout: "mon": "allow *", 2023-08-09T14:51:43.274 INFO:teuthology.orchestra.run.smithi043.stdout: "osd": "allow *" 2023-08-09T14:51:43.274 INFO:teuthology.orchestra.run.smithi043.stdout: } 2023-08-09T14:51:43.274 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2023-08-09T14:51:43.274 INFO:teuthology.orchestra.run.smithi043.stdout: { 2023-08-09T14:51:43.274 INFO:teuthology.orchestra.run.smithi043.stdout: "entity": "mds.c", 2023-08-09T14:51:43.274 INFO:teuthology.orchestra.run.smithi043.stdout: "key": "AQCOldNk5V46GhAAQBhkvfjVSJCSHLS/YkhgCA==", 2023-08-09T14:51:43.274 INFO:teuthology.orchestra.run.smithi043.stdout: "caps": { 2023-08-09T14:51:43.275 INFO:teuthology.orchestra.run.smithi043.stdout: "mds": "allow", 2023-08-09T14:51:43.275 INFO:teuthology.orchestra.run.smithi043.stdout: "mgr": "allow *", 2023-08-09T14:51:43.275 INFO:teuthology.orchestra.run.smithi043.stdout: "mon": "allow *", 2023-08-09T14:51:43.275 INFO:teuthology.orchestra.run.smithi043.stdout: "osd": "allow *" 2023-08-09T14:51:43.275 INFO:teuthology.orchestra.run.smithi043.stdout: } 2023-08-09T14:51:43.275 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2023-08-09T14:51:43.275 INFO:teuthology.orchestra.run.smithi043.stdout: { 2023-08-09T14:51:43.276 INFO:teuthology.orchestra.run.smithi043.stdout: "entity": "mds.d", 2023-08-09T14:51:43.276 INFO:teuthology.orchestra.run.smithi043.stdout: "key": "AQCPldNkNBoTARAAMVW+rSyXQKaau6I90fUX9Q==", 2023-08-09T14:51:43.276 INFO:teuthology.orchestra.run.smithi043.stdout: "caps": { 2023-08-09T14:51:43.276 INFO:teuthology.orchestra.run.smithi043.stdout: "mds": "allow", 2023-08-09T14:51:43.276 INFO:teuthology.orchestra.run.smithi043.stdout: "mgr": "allow *", 2023-08-09T14:51:43.276 INFO:teuthology.orchestra.run.smithi043.stdout: "mon": "allow *", 2023-08-09T14:51:43.276 INFO:teuthology.orchestra.run.smithi043.stdout: "osd": "allow *" 2023-08-09T14:51:43.277 INFO:teuthology.orchestra.run.smithi043.stdout: } 2023-08-09T14:51:43.277 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2023-08-09T14:51:43.277 INFO:teuthology.orchestra.run.smithi043.stdout: { 2023-08-09T14:51:43.277 INFO:teuthology.orchestra.run.smithi043.stdout: "entity": "osd.0", 2023-08-09T14:51:43.277 INFO:teuthology.orchestra.run.smithi043.stdout: "key": "AQCRldNkltPlMhAAAt3rFvraCtI0aSqGvt6UxA==", 2023-08-09T14:51:43.277 INFO:teuthology.orchestra.run.smithi043.stdout: "caps": { 2023-08-09T14:51:43.277 INFO:teuthology.orchestra.run.smithi043.stdout: "mgr": "allow profile osd", 2023-08-09T14:51:43.277 INFO:teuthology.orchestra.run.smithi043.stdout: "mon": "allow profile osd", 2023-08-09T14:51:43.278 INFO:teuthology.orchestra.run.smithi043.stdout: "osd": "allow *" 2023-08-09T14:51:43.278 INFO:teuthology.orchestra.run.smithi043.stdout: } 2023-08-09T14:51:43.278 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2023-08-09T14:51:43.278 INFO:teuthology.orchestra.run.smithi043.stdout: { 2023-08-09T14:51:43.278 INFO:teuthology.orchestra.run.smithi043.stdout: "entity": "osd.1", 2023-08-09T14:51:43.278 INFO:teuthology.orchestra.run.smithi043.stdout: "key": "AQCVldNkzpKYEhAA+wen0kV/vUDLQztq3vhfOg==", 2023-08-09T14:51:43.278 INFO:teuthology.orchestra.run.smithi043.stdout: "caps": { 2023-08-09T14:51:43.278 INFO:teuthology.orchestra.run.smithi043.stdout: "mgr": "allow profile osd", 2023-08-09T14:51:43.279 INFO:teuthology.orchestra.run.smithi043.stdout: "mon": "allow profile osd", 2023-08-09T14:51:43.279 INFO:teuthology.orchestra.run.smithi043.stdout: "osd": "allow *" 2023-08-09T14:51:43.279 INFO:teuthology.orchestra.run.smithi043.stdout: } 2023-08-09T14:51:43.279 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2023-08-09T14:51:43.279 INFO:teuthology.orchestra.run.smithi043.stdout: { 2023-08-09T14:51:43.279 INFO:teuthology.orchestra.run.smithi043.stdout: "entity": "osd.2", 2023-08-09T14:51:43.279 INFO:teuthology.orchestra.run.smithi043.stdout: "key": "AQCYldNkYbwNHhAAQY8g9dldb1V9M2MM2SHKDg==", 2023-08-09T14:51:43.279 INFO:teuthology.orchestra.run.smithi043.stdout: "caps": { 2023-08-09T14:51:43.280 INFO:teuthology.orchestra.run.smithi043.stdout: "mgr": "allow profile osd", 2023-08-09T14:51:43.280 INFO:teuthology.orchestra.run.smithi043.stdout: "mon": "allow profile osd", 2023-08-09T14:51:43.280 INFO:teuthology.orchestra.run.smithi043.stdout: "osd": "allow *" 2023-08-09T14:51:43.280 INFO:teuthology.orchestra.run.smithi043.stdout: } 2023-08-09T14:51:43.280 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2023-08-09T14:51:43.280 INFO:teuthology.orchestra.run.smithi043.stdout: { 2023-08-09T14:51:43.280 INFO:teuthology.orchestra.run.smithi043.stdout: "entity": "osd.3", 2023-08-09T14:51:43.281 INFO:teuthology.orchestra.run.smithi043.stdout: "key": "AQCbldNkeyW6KRAAsd+kgqik2aD4U7PXwccWtA==", 2023-08-09T14:51:43.281 INFO:teuthology.orchestra.run.smithi043.stdout: "caps": { 2023-08-09T14:51:43.281 INFO:teuthology.orchestra.run.smithi043.stdout: "mgr": "allow profile osd", 2023-08-09T14:51:43.281 INFO:teuthology.orchestra.run.smithi043.stdout: "mon": "allow profile osd", 2023-08-09T14:51:43.281 INFO:teuthology.orchestra.run.smithi043.stdout: "osd": "allow *" 2023-08-09T14:51:43.281 INFO:teuthology.orchestra.run.smithi043.stdout: } 2023-08-09T14:51:43.281 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2023-08-09T14:51:43.281 INFO:teuthology.orchestra.run.smithi043.stdout: { 2023-08-09T14:51:43.282 INFO:teuthology.orchestra.run.smithi043.stdout: "entity": "osd.4", 2023-08-09T14:51:43.282 INFO:teuthology.orchestra.run.smithi043.stdout: "key": "AQCildNk80UkOBAAtX+4JGpOVcfZEHetJoeZKw==", 2023-08-09T14:51:43.282 INFO:teuthology.orchestra.run.smithi043.stdout: "caps": { 2023-08-09T14:51:43.282 INFO:teuthology.orchestra.run.smithi043.stdout: "mgr": "allow profile osd", 2023-08-09T14:51:43.282 INFO:teuthology.orchestra.run.smithi043.stdout: "mon": "allow profile osd", 2023-08-09T14:51:43.282 INFO:teuthology.orchestra.run.smithi043.stdout: "osd": "allow *" 2023-08-09T14:51:43.282 INFO:teuthology.orchestra.run.smithi043.stdout: } 2023-08-09T14:51:43.282 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2023-08-09T14:51:43.283 INFO:teuthology.orchestra.run.smithi043.stdout: { 2023-08-09T14:51:43.283 INFO:teuthology.orchestra.run.smithi043.stdout: "entity": "osd.5", 2023-08-09T14:51:43.283 INFO:teuthology.orchestra.run.smithi043.stdout: "key": "AQCmldNkCA9SBxAAZj4So6MRj2Eq1I9H9JzLDA==", 2023-08-09T14:51:43.283 INFO:teuthology.orchestra.run.smithi043.stdout: "caps": { 2023-08-09T14:51:43.283 INFO:teuthology.orchestra.run.smithi043.stdout: "mgr": "allow profile osd", 2023-08-09T14:51:43.283 INFO:teuthology.orchestra.run.smithi043.stdout: "mon": "allow profile osd", 2023-08-09T14:51:43.283 INFO:teuthology.orchestra.run.smithi043.stdout: "osd": "allow *" 2023-08-09T14:51:43.283 INFO:teuthology.orchestra.run.smithi043.stdout: } 2023-08-09T14:51:43.284 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2023-08-09T14:51:43.284 INFO:teuthology.orchestra.run.smithi043.stdout: { 2023-08-09T14:51:43.284 INFO:teuthology.orchestra.run.smithi043.stdout: "entity": "osd.6", 2023-08-09T14:51:43.284 INFO:teuthology.orchestra.run.smithi043.stdout: "key": "AQCpldNkaOwAExAAraV1tUwr9uU6GheZ+hvdqQ==", 2023-08-09T14:51:43.284 INFO:teuthology.orchestra.run.smithi043.stdout: "caps": { 2023-08-09T14:51:43.284 INFO:teuthology.orchestra.run.smithi043.stdout: "mgr": "allow profile osd", 2023-08-09T14:51:43.284 INFO:teuthology.orchestra.run.smithi043.stdout: "mon": "allow profile osd", 2023-08-09T14:51:43.285 INFO:teuthology.orchestra.run.smithi043.stdout: "osd": "allow *" 2023-08-09T14:51:43.285 INFO:teuthology.orchestra.run.smithi043.stdout: } 2023-08-09T14:51:43.285 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2023-08-09T14:51:43.285 INFO:teuthology.orchestra.run.smithi043.stdout: { 2023-08-09T14:51:43.285 INFO:teuthology.orchestra.run.smithi043.stdout: "entity": "osd.7", 2023-08-09T14:51:43.285 INFO:teuthology.orchestra.run.smithi043.stdout: "key": "AQCsldNk2/GwLBAAP42EXPivzLk44X8aMEVVCg==", 2023-08-09T14:51:43.285 INFO:teuthology.orchestra.run.smithi043.stdout: "caps": { 2023-08-09T14:51:43.285 INFO:teuthology.orchestra.run.smithi043.stdout: "mgr": "allow profile osd", 2023-08-09T14:51:43.286 INFO:teuthology.orchestra.run.smithi043.stdout: "mon": "allow profile osd", 2023-08-09T14:51:43.286 INFO:teuthology.orchestra.run.smithi043.stdout: "osd": "allow *" 2023-08-09T14:51:43.286 INFO:teuthology.orchestra.run.smithi043.stdout: } 2023-08-09T14:51:43.286 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2023-08-09T14:51:43.286 INFO:teuthology.orchestra.run.smithi043.stdout: { 2023-08-09T14:51:43.286 INFO:teuthology.orchestra.run.smithi043.stdout: "entity": "client.0", 2023-08-09T14:51:43.286 INFO:teuthology.orchestra.run.smithi043.stdout: "key": "AQCPldNkg8zrDxAAM/hnJaE6S9HU/fRwQeBV5w==", 2023-08-09T14:51:43.286 INFO:teuthology.orchestra.run.smithi043.stdout: "caps": { 2023-08-09T14:51:43.287 INFO:teuthology.orchestra.run.smithi043.stdout: "mds": "allow", 2023-08-09T14:51:43.287 INFO:teuthology.orchestra.run.smithi043.stdout: "mon": "allow r", 2023-08-09T14:51:43.287 INFO:teuthology.orchestra.run.smithi043.stdout: "osd": "allow rw tag cephfs data=cephfs" 2023-08-09T14:51:43.287 INFO:teuthology.orchestra.run.smithi043.stdout: } 2023-08-09T14:51:43.287 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2023-08-09T14:51:43.287 INFO:teuthology.orchestra.run.smithi043.stdout: { 2023-08-09T14:51:43.287 INFO:teuthology.orchestra.run.smithi043.stdout: "entity": "client.1", 2023-08-09T14:51:43.287 INFO:teuthology.orchestra.run.smithi043.stdout: "key": "AQCPldNklfFxJxAA19+L78rW6jU1uOyjey6WNA==", 2023-08-09T14:51:43.288 INFO:teuthology.orchestra.run.smithi043.stdout: "caps": { 2023-08-09T14:51:43.288 INFO:teuthology.orchestra.run.smithi043.stdout: "mds": "allow", 2023-08-09T14:51:43.288 INFO:teuthology.orchestra.run.smithi043.stdout: "mgr": "allow r", 2023-08-09T14:51:43.288 INFO:teuthology.orchestra.run.smithi043.stdout: "mon": "allow rw", 2023-08-09T14:51:43.288 INFO:teuthology.orchestra.run.smithi043.stdout: "osd": "allow rwx" 2023-08-09T14:51:43.288 INFO:teuthology.orchestra.run.smithi043.stdout: } 2023-08-09T14:51:43.288 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2023-08-09T14:51:43.289 INFO:teuthology.orchestra.run.smithi043.stdout: { 2023-08-09T14:51:43.289 INFO:teuthology.orchestra.run.smithi043.stdout: "entity": "client.2", 2023-08-09T14:51:43.289 INFO:teuthology.orchestra.run.smithi043.stdout: "key": "AQCPldNkC2lgGxAAZ8qZboOCUaQlcpkPL1ozHA==", 2023-08-09T14:51:43.289 INFO:teuthology.orchestra.run.smithi043.stdout: "caps": { 2023-08-09T14:51:43.289 INFO:teuthology.orchestra.run.smithi043.stdout: "mds": "allow", 2023-08-09T14:51:43.289 INFO:teuthology.orchestra.run.smithi043.stdout: "mgr": "allow r", 2023-08-09T14:51:43.289 INFO:teuthology.orchestra.run.smithi043.stdout: "mon": "allow rw", 2023-08-09T14:51:43.289 INFO:teuthology.orchestra.run.smithi043.stdout: "osd": "allow rwx" 2023-08-09T14:51:43.290 INFO:teuthology.orchestra.run.smithi043.stdout: } 2023-08-09T14:51:43.290 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2023-08-09T14:51:43.290 INFO:teuthology.orchestra.run.smithi043.stdout: { 2023-08-09T14:51:43.290 INFO:teuthology.orchestra.run.smithi043.stdout: "entity": "client.3", 2023-08-09T14:51:43.290 INFO:teuthology.orchestra.run.smithi043.stdout: "key": "AQCPldNkl+rXMhAA9rf3KdnSYbvngqmLYnIGGA==", 2023-08-09T14:51:43.290 INFO:teuthology.orchestra.run.smithi043.stdout: "caps": { 2023-08-09T14:51:43.290 INFO:teuthology.orchestra.run.smithi043.stdout: "mds": "allow", 2023-08-09T14:51:43.290 INFO:teuthology.orchestra.run.smithi043.stdout: "mgr": "allow r", 2023-08-09T14:51:43.291 INFO:teuthology.orchestra.run.smithi043.stdout: "mon": "allow rw", 2023-08-09T14:51:43.291 INFO:teuthology.orchestra.run.smithi043.stdout: "osd": "allow rwx" 2023-08-09T14:51:43.291 INFO:teuthology.orchestra.run.smithi043.stdout: } 2023-08-09T14:51:43.291 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2023-08-09T14:51:43.291 INFO:teuthology.orchestra.run.smithi043.stdout: { 2023-08-09T14:51:43.291 INFO:teuthology.orchestra.run.smithi043.stdout: "entity": "client.admin", 2023-08-09T14:51:43.291 INFO:teuthology.orchestra.run.smithi043.stdout: "key": "AQCMldNkUae6NRAA2FBTr0QPBdKNDoe1WK5zyg==", 2023-08-09T14:51:43.291 INFO:teuthology.orchestra.run.smithi043.stdout: "caps": { 2023-08-09T14:51:43.292 INFO:teuthology.orchestra.run.smithi043.stdout: "mds": "allow *", 2023-08-09T14:51:43.292 INFO:teuthology.orchestra.run.smithi043.stdout: "mgr": "allow *", 2023-08-09T14:51:43.292 INFO:teuthology.orchestra.run.smithi043.stdout: "mon": "allow *", 2023-08-09T14:51:43.292 INFO:teuthology.orchestra.run.smithi043.stdout: "osd": "allow *" 2023-08-09T14:51:43.292 INFO:teuthology.orchestra.run.smithi043.stdout: } 2023-08-09T14:51:43.292 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2023-08-09T14:51:43.292 INFO:teuthology.orchestra.run.smithi043.stdout: { 2023-08-09T14:51:43.292 INFO:teuthology.orchestra.run.smithi043.stdout: "entity": "mgr.x", 2023-08-09T14:51:43.293 INFO:teuthology.orchestra.run.smithi043.stdout: "key": "AQCNldNk2ufSNxAAU7msRkwLaoLZHTIsk0ck/g==", 2023-08-09T14:51:43.293 INFO:teuthology.orchestra.run.smithi043.stdout: "caps": { 2023-08-09T14:51:43.293 INFO:teuthology.orchestra.run.smithi043.stdout: "mds": "allow *", 2023-08-09T14:51:43.293 INFO:teuthology.orchestra.run.smithi043.stdout: "mon": "allow profile mgr", 2023-08-09T14:51:43.293 INFO:teuthology.orchestra.run.smithi043.stdout: "osd": "allow *" 2023-08-09T14:51:43.293 INFO:teuthology.orchestra.run.smithi043.stdout: } 2023-08-09T14:51:43.293 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2023-08-09T14:51:43.293 INFO:teuthology.orchestra.run.smithi043.stdout: { 2023-08-09T14:51:43.294 INFO:teuthology.orchestra.run.smithi043.stdout: "entity": "mgr.y", 2023-08-09T14:51:43.294 INFO:teuthology.orchestra.run.smithi043.stdout: "key": "AQCNldNk2YgzLBAAA11Q8uAMEgE/bUni+tXIKA==", 2023-08-09T14:51:43.294 INFO:teuthology.orchestra.run.smithi043.stdout: "caps": { 2023-08-09T14:51:43.294 INFO:teuthology.orchestra.run.smithi043.stdout: "mds": "allow *", 2023-08-09T14:51:43.294 INFO:teuthology.orchestra.run.smithi043.stdout: "mon": "allow profile mgr", 2023-08-09T14:51:43.294 INFO:teuthology.orchestra.run.smithi043.stdout: "osd": "allow *" 2023-08-09T14:51:43.294 INFO:teuthology.orchestra.run.smithi043.stdout: } 2023-08-09T14:51:43.295 INFO:teuthology.orchestra.run.smithi043.stdout: } 2023-08-09T14:51:43.295 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2023-08-09T14:51:43.295 INFO:teuthology.orchestra.run.smithi043.stdout:} 2023-08-09T14:51:43.296 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:43.272+0000 7fa9688cd640 1 -- 172.21.15.43:0/2065695460 >> [v2:172.21.15.203:6832/2566816332,v1:172.21.15.203:6833/2566816332] conn(0x7fa95006a1a0 msgr2=0x7fa95006c660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T14:51:43.296 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:43.272+0000 7fa9688cd640 1 --2- 172.21.15.43:0/2065695460 >> [v2:172.21.15.203:6832/2566816332,v1:172.21.15.203:6833/2566816332] conn(0x7fa95006a1a0 0x7fa95006c660 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7fa9540016d0 tx=0x7fa954009040 comp rx=0 tx=0).stop 2023-08-09T14:51:43.296 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:43.272+0000 7fa9688cd640 1 -- 172.21.15.43:0/2065695460 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa9640c5940 msgr2=0x7fa9641f24b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T14:51:43.296 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:43.272+0000 7fa9688cd640 1 --2- 172.21.15.43:0/2065695460 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa9640c5940 0x7fa9641f24b0 secure :-1 s=READY pgs=1842 cs=0 l=1 rev1=1 crypto rx=0x7fa95c00a800 tx=0x7fa95c001d80 comp rx=0 tx=0).stop 2023-08-09T14:51:43.297 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:43.272+0000 7fa9688cd640 1 -- 172.21.15.43:0/2065695460 shutdown_connections 2023-08-09T14:51:43.297 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:43.272+0000 7fa9688cd640 1 --2- 172.21.15.43:0/2065695460 >> [v2:172.21.15.203:6832/2566816332,v1:172.21.15.203:6833/2566816332] conn(0x7fa95006a1a0 0x7fa95006c660 unknown :-1 s=CLOSED pgs=48 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T14:51:43.297 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:43.272+0000 7fa9688cd640 1 --2- 172.21.15.43:0/2065695460 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa9640c5940 0x7fa9641f24b0 unknown :-1 s=CLOSED pgs=1842 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T14:51:43.297 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:43.272+0000 7fa9688cd640 1 --2- 172.21.15.43:0/2065695460 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fa9641f5c60 0x7fa9640c5400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T14:51:43.297 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:43.272+0000 7fa9688cd640 1 --2- 172.21.15.43:0/2065695460 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fa94c004d00 0x7fa964148210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T14:51:43.297 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:43.272+0000 7fa9688cd640 1 -- 172.21.15.43:0/2065695460 >> 172.21.15.43:0/2065695460 conn(0x7fa96406a2b0 msgr2=0x7fa9640d1dd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-09T14:51:43.297 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:43.272+0000 7fa9688cd640 1 -- 172.21.15.43:0/2065695460 shutdown_connections 2023-08-09T14:51:43.297 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:43.272+0000 7fa9688cd640 1 -- 172.21.15.43:0/2065695460 wait complete. 2023-08-09T14:51:43.298 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-08-09T14:51:43.531 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:43.530+0000 7f4a12b53640 1 Processor -- start 2023-08-09T14:51:43.532 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:43.530+0000 7f4a12b53640 1 -- start start 2023-08-09T14:51:43.533 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:43.531+0000 7f4a12b53640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f4a0c0b1050 0x7f4a0c0b4760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T14:51:43.533 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:43.531+0000 7f4a12b53640 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7f4a0c0cd6b0 0x7f4a0c0d0e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T14:51:43.533 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:43.531+0000 7f4a12b53640 1 -- --> v1:172.21.15.203:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4a0c0b7bf0 con 0x7f4a0c15f7b0 2023-08-09T14:51:43.533 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:43.531+0000 7f4a12b53640 1 -- --> v2:172.21.15.203:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4a0c0b7ed0 con 0x7f4a0c0cd6b0 2023-08-09T14:51:43.533 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:43.531+0000 7f4a12b53640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4a0c0d0370 con 0x7f4a0c0b1050 2023-08-09T14:51:43.534 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:43.532+0000 7f4a11350640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f4a0c0b1050 0x7f4a0c0b4760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T14:51:43.534 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:43.532+0000 7f4a11350640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f4a0c0b1050 0x7f4a0c0b4760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.203:3301/0 says I am v2:172.21.15.43:47252/0 (socket says 172.21.15.43:47252) 2023-08-09T14:51:43.534 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:43.532+0000 7f4a11350640 1 -- 172.21.15.43:0/73784268 learned_addr learned my addr 172.21.15.43:0/73784268 (peer_addr_for_me v2:172.21.15.43:0/0) 2023-08-09T14:51:43.534 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:43.532+0000 7f4a12352640 1 --2- 172.21.15.43:0/73784268 >> v2:172.21.15.203:3300/0 conn(0x7f4a0c0cd6b0 0x7f4a0c0d0e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T14:51:43.535 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:43.533+0000 7f4a11350640 1 -- 172.21.15.43:0/73784268 >> v2:172.21.15.203:3300/0 conn(0x7f4a0c0cd6b0 msgr2=0x7f4a0c0d0e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-09T14:51:43.535 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:43.533+0000 7f4a11350640 1 --2- 172.21.15.43:0/73784268 >> v2:172.21.15.203:3300/0 conn(0x7f4a0c0cd6b0 0x7f4a0c0d0e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T14:51:43.535 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:43.533+0000 7f4a11350640 1 -- 172.21.15.43:0/73784268 >> v1:172.21.15.203:6789/0 conn(0x7f4a0c15f7b0 legacy=0x7f4a0c0ccfa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T14:51:43.535 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:43.533+0000 7f4a11350640 1 -- 172.21.15.43:0/73784268 --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4a0c0d0650 con 0x7f4a0c0b1050 2023-08-09T14:51:43.536 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:43.534+0000 7f4a11350640 1 --2- 172.21.15.43:0/73784268 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f4a0c0b1050 0x7f4a0c0b4760 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7f49fc009870 tx=0x7f49fc02b980 comp rx=0 tx=0).ready entity=mon.2 client_cookie=9aa65304d091f89f server_cookie=0 in_seq=0 out_seq=0 2023-08-09T14:51:43.536 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:43.535+0000 7f4a10b4f640 1 -- 172.21.15.43:0/73784268 <== mon.2 v2:172.21.15.203:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f49fc030840 con 0x7f4a0c0b1050 2023-08-09T14:51:43.536 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:43.535+0000 7f4a10b4f640 1 -- 172.21.15.43:0/73784268 <== mon.2 v2:172.21.15.203:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f49fc0309a0 con 0x7f4a0c0b1050 2023-08-09T14:51:43.537 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:43.535+0000 7f4a10b4f640 1 -- 172.21.15.43:0/73784268 <== mon.2 v2:172.21.15.203:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f49fc030d30 con 0x7f4a0c0b1050 2023-08-09T14:51:43.537 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:43.535+0000 7f4a12b53640 1 -- 172.21.15.43:0/73784268 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f4a0c0b1050 msgr2=0x7f4a0c0b4760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T14:51:43.538 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:43.535+0000 7f4a12b53640 1 --2- 172.21.15.43:0/73784268 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f4a0c0b1050 0x7f4a0c0b4760 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7f49fc009870 tx=0x7f49fc02b980 comp rx=0 tx=0).stop 2023-08-09T14:51:43.538 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:43.536+0000 7f4a12b53640 1 -- 172.21.15.43:0/73784268 shutdown_connections 2023-08-09T14:51:43.538 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:43.536+0000 7f4a12b53640 1 --2- 172.21.15.43:0/73784268 >> v2:172.21.15.203:3300/0 conn(0x7f4a0c0cd6b0 0x7f4a0c0d0e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T14:51:43.538 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:43.536+0000 7f4a12b53640 1 --2- 172.21.15.43:0/73784268 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f4a0c0b1050 0x7f4a0c0b4760 unknown :-1 s=CLOSED pgs=252 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T14:51:43.538 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:43.536+0000 7f4a12b53640 1 -- 172.21.15.43:0/73784268 >> 172.21.15.43:0/73784268 conn(0x7f4a0c06a2b0 msgr2=0x7f4a0c06a690 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-09T14:51:43.538 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:43.536+0000 7f4a12b53640 1 -- 172.21.15.43:0/73784268 shutdown_connections 2023-08-09T14:51:43.538 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:43.536+0000 7f4a12b53640 1 -- 172.21.15.43:0/73784268 wait complete. 2023-08-09T14:51:43.539 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:43.537+0000 7f4a12b53640 1 Processor -- start 2023-08-09T14:51:43.539 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:43.538+0000 7f4a12b53640 1 -- start start 2023-08-09T14:51:43.540 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:43.539+0000 7f4a12b53640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f4a0c0b1050 0x7f4a0c0b3660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T14:51:43.540 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:43.539+0000 7f4a12b53640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f4a0c0cd6b0 0x7f4a0c1551d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T14:51:43.541 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:43.539+0000 7f4a12b53640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f4a0c15f7b0 0x7f4a0c1497c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T14:51:43.541 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:43.539+0000 7f4a12b53640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4a0c0b3ba0 con 0x7f4a0c0b1050 2023-08-09T14:51:43.541 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:43.539+0000 7f4a12b53640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4a0c1f4f80 con 0x7f4a0c0cd6b0 2023-08-09T14:51:43.541 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:43.539+0000 7f4a12b53640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4a0c1f5260 con 0x7f4a0c15f7b0 2023-08-09T14:51:43.541 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:43.539+0000 7f4a11b51640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f4a0c0b1050 0x7f4a0c0b3660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T14:51:43.541 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:43.539+0000 7f4a11b51640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f4a0c0b1050 0x7f4a0c0b3660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:52530/0 (socket says 172.21.15.43:52530) 2023-08-09T14:51:43.541 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:43.539+0000 7f4a11b51640 1 -- 172.21.15.43:0/180075171 learned_addr learned my addr 172.21.15.43:0/180075171 (peer_addr_for_me v2:172.21.15.43:0/0) 2023-08-09T14:51:43.543 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:43.539+0000 7f4a11b51640 1 -- 172.21.15.43:0/180075171 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f4a0c15f7b0 msgr2=0x7f4a0c1497c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T14:51:43.543 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:43.539+0000 7f4a11350640 1 --2- 172.21.15.43:0/180075171 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f4a0c0cd6b0 0x7f4a0c1551d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T14:51:43.543 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:43.539+0000 7f4a12352640 1 --2- 172.21.15.43:0/180075171 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f4a0c15f7b0 0x7f4a0c1497c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T14:51:43.543 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:43.539+0000 7f4a11b51640 1 --2- 172.21.15.43:0/180075171 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f4a0c15f7b0 0x7f4a0c1497c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T14:51:43.543 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:43.539+0000 7f4a11b51640 1 -- 172.21.15.43:0/180075171 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f4a0c0cd6b0 msgr2=0x7f4a0c1551d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T14:51:43.544 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:43.539+0000 7f4a11b51640 1 --2- 172.21.15.43:0/180075171 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f4a0c0cd6b0 0x7f4a0c1551d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T14:51:43.544 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:43.539+0000 7f4a11b51640 1 -- 172.21.15.43:0/180075171 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f49fc0094d0 con 0x7f4a0c0b1050 2023-08-09T14:51:43.544 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:43.540+0000 7f4a11b51640 1 --2- 172.21.15.43:0/180075171 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f4a0c0b1050 0x7f4a0c0b3660 secure :-1 s=READY pgs=1843 cs=0 l=1 rev1=1 crypto rx=0x7f4a0800c400 tx=0x7f4a0800c8c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-09T14:51:43.544 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:43.540+0000 7f4a027fc640 1 -- 172.21.15.43:0/180075171 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4a080052d0 con 0x7f4a0c0b1050 2023-08-09T14:51:43.544 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:43.540+0000 7f4a027fc640 1 -- 172.21.15.43:0/180075171 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4a08005430 con 0x7f4a0c0b1050 2023-08-09T14:51:43.545 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:43.540+0000 7f4a027fc640 1 -- 172.21.15.43:0/180075171 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4a08003830 con 0x7f4a0c0b1050 2023-08-09T14:51:43.545 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:43.540+0000 7f4a12b53640 1 -- 172.21.15.43:0/180075171 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4a0c1f5540 con 0x7f4a0c0b1050 2023-08-09T14:51:43.545 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:43.540+0000 7f4a12b53640 1 -- 172.21.15.43:0/180075171 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4a0c1f5b00 con 0x7f4a0c0b1050 2023-08-09T14:51:43.545 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:43.542+0000 7f4a12b53640 1 -- 172.21.15.43:0/180075171 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4a0c0739f0 con 0x7f4a0c0b1050 2023-08-09T14:51:43.546 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:43.542+0000 7f4a027fc640 1 -- 172.21.15.43:0/180075171 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 79) v1 ==== 88443+0+0 (secure 0 0 0) 0x7f4a08003990 con 0x7f4a0c0b1050 2023-08-09T14:51:43.546 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:43.543+0000 7f4a027fc640 1 --2- 172.21.15.43:0/180075171 >> [v2:172.21.15.203:6832/2566816332,v1:172.21.15.203:6833/2566816332] conn(0x7f49f006a170 0x7f49f006c630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T14:51:43.546 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:43.544+0000 7f4a027fc640 1 -- 172.21.15.43:0/180075171 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(168..168 src has 1..168) v4 ==== 5574+0+0 (secure 0 0 0) 0x7f4a08087b50 con 0x7f4a0c0b1050 2023-08-09T14:51:43.546 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:43.544+0000 7f4a11350640 1 --2- 172.21.15.43:0/180075171 >> [v2:172.21.15.203:6832/2566816332,v1:172.21.15.203:6833/2566816332] conn(0x7f49f006a170 0x7f49f006c630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T14:51:43.547 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:43.545+0000 7f4a11350640 1 --2- 172.21.15.43:0/180075171 >> [v2:172.21.15.203:6832/2566816332,v1:172.21.15.203:6833/2566816332] conn(0x7f49f006a170 0x7f49f006c630 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7f49fc036000 tx=0x7f49fc03b040 comp rx=0 tx=0).ready entity=mgr.7765 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-09T14:51:43.549 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:43.548+0000 7f4a027fc640 1 -- 172.21.15.43:0/180075171 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178066 (secure 0 0 0) 0x7f4a08053270 con 0x7f4a0c0b1050 2023-08-09T14:51:43.748 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:43.746+0000 7f4a12b53640 1 -- 172.21.15.43:0/180075171 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f4a0c15efd0 con 0x7f4a0c0b1050 2023-08-09T14:51:43.750 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:43.749+0000 7f4a027fc640 1 -- 172.21.15.43:0/180075171 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v168) v1 ==== 74+0+14854 (secure 0 0 0) 0x7f4a08056890 con 0x7f4a0c0b1050 2023-08-09T14:51:43.750 INFO:teuthology.orchestra.run.smithi043.stdout: 2023-08-09T14:51:43.751 INFO:teuthology.orchestra.run.smithi043.stdout:{"epoch":168,"fsid":"daac87ee-1337-4250-a955-aea4e408a39b","created":"2023-08-09T13:33:41.701198+0000","modified":"2023-08-09T14:51:42.736610+0000","last_up_change":"2023-08-09T13:33:53.783453+0000","last_in_change":"2023-08-09T13:33:45.148904+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":19,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":"rbd","create_time":"2023-08-09T13:33:54.365137+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":".mgr","create_time":"2023-08-09T13:33:54.831803+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"cbd49ecd-e896-4fbd-864e-5c37a829b165","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":149,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6800","nonce":502667335},{"type":"v1","addr":"172.21.15.43:6801","nonce":502667335}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6802","nonce":502667335},{"type":"v1","addr":"172.21.15.43:6803","nonce":502667335}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6806","nonce":502667335},{"type":"v1","addr":"172.21.15.43:6807","nonce":502667335}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6804","nonce":502667335},{"type":"v1","addr":"172.21.15.43:6805","nonce":502667335}]},"public_addr":"172.21.15.43:6801/502667335","cluster_addr":"172.21.15.43:6803/502667335","heartbeat_back_addr":"172.21.15.43:6807/502667335","heartbeat_front_addr":"172.21.15.43:6805/502667335","state":["exists","up"]},{"osd":1,"uuid":"75c7e0fe-1ec7-4fe8-a2f2-3661a65b8500","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":145,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6808","nonce":3289134610},{"type":"v1","addr":"172.21.15.43:6809","nonce":3289134610}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6810","nonce":3289134610},{"type":"v1","addr":"172.21.15.43:6812","nonce":3289134610}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6821","nonce":3289134610},{"type":"v1","addr":"172.21.15.43:6824","nonce":3289134610}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6815","nonce":3289134610},{"type":"v1","addr":"172.21.15.43:6818","nonce":3289134610}]},"public_addr":"172.21.15.43:6809/3289134610","cluster_addr":"172.21.15.43:6812/3289134610","heartbeat_back_addr":"172.21.15.43:6824/3289134610","heartbeat_front_addr":"172.21.15.43:6818/3289134610","state":["exists","up"]},{"osd":2,"uuid":"bb5c41e9-7f5f-4125-9b01-3bb8bfc2ea83","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":145,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6811","nonce":4079709736},{"type":"v1","addr":"172.21.15.43:6814","nonce":4079709736}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6817","nonce":4079709736},{"type":"v1","addr":"172.21.15.43:6820","nonce":4079709736}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6829","nonce":4079709736},{"type":"v1","addr":"172.21.15.43:6831","nonce":4079709736}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6823","nonce":4079709736},{"type":"v1","addr":"172.21.15.43:6826","nonce":4079709736}]},"public_addr":"172.21.15.43:6814/4079709736","cluster_addr":"172.21.15.43:6820/4079709736","heartbeat_back_addr":"172.21.15.43:6831/4079709736","heartbeat_front_addr":"172.21.15.43:6826/4079709736","state":["exists","up"]},{"osd":3,"uuid":"bfe4c8b4-694a-465c-87e6-2b2aab852613","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":149,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6813","nonce":1709538759},{"type":"v1","addr":"172.21.15.43:6816","nonce":1709538759}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6819","nonce":1709538759},{"type":"v1","addr":"172.21.15.43:6822","nonce":1709538759}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6828","nonce":1709538759},{"type":"v1","addr":"172.21.15.43:6830","nonce":1709538759}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6825","nonce":1709538759},{"type":"v1","addr":"172.21.15.43:6827","nonce":1709538759}]},"public_addr":"172.21.15.43:6816/1709538759","cluster_addr":"172.21.15.43:6822/1709538759","heartbeat_back_addr":"172.21.15.43:6830/1709538759","heartbeat_front_addr":"172.21.15.43:6827/1709538759","state":["exists","up"]},{"osd":4,"uuid":"fa2e8a77-aba1-4756-853e-0d174a5bdeba","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":145,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6800","nonce":4196866481},{"type":"v1","addr":"172.21.15.203:6801","nonce":4196866481}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6802","nonce":4196866481},{"type":"v1","addr":"172.21.15.203:6803","nonce":4196866481}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6806","nonce":4196866481},{"type":"v1","addr":"172.21.15.203:6807","nonce":4196866481}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6804","nonce":4196866481},{"type":"v1","addr":"172.21.15.203:6805","nonce":4196866481}]},"public_addr":"172.21.15.203:6801/4196866481","cluster_addr":"172.21.15.203:6803/4196866481","heartbeat_back_addr":"172.21.15.203:6807/4196866481","heartbeat_front_addr":"172.21.15.203:6805/4196866481","state":["exists","up"]},{"osd":5,"uuid":"81811e4b-4d24-4f51-a82c-f1d700a2a00d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":145,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6816","nonce":3059290081},{"type":"v1","addr":"172.21.15.203:6817","nonce":3059290081}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6818","nonce":3059290081},{"type":"v1","addr":"172.21.15.203:6819","nonce":3059290081}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6822","nonce":3059290081},{"type":"v1","addr":"172.21.15.203:6823","nonce":3059290081}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6820","nonce":3059290081},{"type":"v1","addr":"172.21.15.203:6821","nonce":3059290081}]},"public_addr":"172.21.15.203:6817/3059290081","cluster_addr":"172.21.15.203:6819/3059290081","heartbeat_back_addr":"172.21.15.203:6823/3059290081","heartbeat_front_addr":"172.21.15.203:6821/3059290081","state":["exists","up"]},{"osd":6,"uuid":"307bac4d-6a5b-443d-af75-56d695d7674f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":145,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6824","nonce":512155569},{"type":"v1","addr":"172.21.15.203:6825","nonce":512155569}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6826","nonce":512155569},{"type":"v1","addr":"172.21.15.203:6827","nonce":512155569}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6830","nonce":512155569},{"type":"v1","addr":"172.21.15.203:6831","nonce":512155569}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6828","nonce":512155569},{"type":"v1","addr":"172.21.15.203:6829","nonce":512155569}]},"public_addr":"172.21.15.203:6825/512155569","cluster_addr":"172.21.15.203:6827/512155569","heartbeat_back_addr":"172.21.15.203:6831/512155569","heartbeat_front_addr":"172.21.15.203:6829/512155569","state":["exists","up"]},{"osd":7,"uuid":"a95dc7ca-9f1b-4295-af70-68a22a2d7d5e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":145,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6808","nonce":3331450438},{"type":"v1","addr":"172.21.15.203:6809","nonce":3331450438}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6810","nonce":3331450438},{"type":"v1","addr":"172.21.15.203:6811","nonce":3331450438}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6814","nonce":3331450438},{"type":"v1","addr":"172.21.15.203:6815","nonce":3331450438}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6812","nonce":3331450438},{"type":"v1","addr":"172.21.15.203:6813","nonce":3331450438}]},"public_addr":"172.21.15.203:6809/3331450438","cluster_addr":"172.21.15.203:6811/3331450438","heartbeat_back_addr":"172.21.15.203:6815/3331450438","heartbeat_front_addr":"172.21.15.203:6813/3331450438","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-09T13:33:50.547652+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-09T13:33:50.589469+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-09T13:33:50.561724+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-09T13:33:50.599650+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-09T13:33:50.411855+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-09T13:33:50.219681+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-09T13:33:50.504664+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-09T13:33:50.441248+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":7,"snaps":[{"begin":104,"length":1},{"begin":221,"length":1}]}],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-08-09T14:51:43.753 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:43.751+0000 7f4a12b53640 1 -- 172.21.15.43:0/180075171 >> [v2:172.21.15.203:6832/2566816332,v1:172.21.15.203:6833/2566816332] conn(0x7f49f006a170 msgr2=0x7f49f006c630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T14:51:43.753 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:43.751+0000 7f4a12b53640 1 --2- 172.21.15.43:0/180075171 >> [v2:172.21.15.203:6832/2566816332,v1:172.21.15.203:6833/2566816332] conn(0x7f49f006a170 0x7f49f006c630 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7f49fc036000 tx=0x7f49fc03b040 comp rx=0 tx=0).stop 2023-08-09T14:51:43.753 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:43.751+0000 7f4a12b53640 1 -- 172.21.15.43:0/180075171 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f4a0c0b1050 msgr2=0x7f4a0c0b3660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T14:51:43.753 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:43.751+0000 7f4a12b53640 1 --2- 172.21.15.43:0/180075171 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f4a0c0b1050 0x7f4a0c0b3660 secure :-1 s=READY pgs=1843 cs=0 l=1 rev1=1 crypto rx=0x7f4a0800c400 tx=0x7f4a0800c8c0 comp rx=0 tx=0).stop 2023-08-09T14:51:43.753 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:43.751+0000 7f4a12b53640 1 -- 172.21.15.43:0/180075171 shutdown_connections 2023-08-09T14:51:43.754 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:43.751+0000 7f4a12b53640 1 --2- 172.21.15.43:0/180075171 >> [v2:172.21.15.203:6832/2566816332,v1:172.21.15.203:6833/2566816332] conn(0x7f49f006a170 0x7f49f006c630 unknown :-1 s=CLOSED pgs=49 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T14:51:43.754 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:43.751+0000 7f4a12b53640 1 --2- 172.21.15.43:0/180075171 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f4a0c15f7b0 0x7f4a0c1497c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T14:51:43.754 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:43.751+0000 7f4a12b53640 1 --2- 172.21.15.43:0/180075171 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f4a0c0cd6b0 0x7f4a0c1551d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T14:51:43.754 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:43.751+0000 7f4a12b53640 1 --2- 172.21.15.43:0/180075171 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f4a0c0b1050 0x7f4a0c0b3660 unknown :-1 s=CLOSED pgs=1843 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T14:51:43.754 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:43.751+0000 7f4a12b53640 1 -- 172.21.15.43:0/180075171 >> 172.21.15.43:0/180075171 conn(0x7f4a0c06a2b0 msgr2=0x7f4a0c15a9c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-09T14:51:43.754 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:43.752+0000 7f4a12b53640 1 -- 172.21.15.43:0/180075171 shutdown_connections 2023-08-09T14:51:43.754 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:43.752+0000 7f4a12b53640 1 -- 172.21.15.43:0/180075171 wait complete. 2023-08-09T14:51:43.772 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': 'rbd', 'create_time': '2023-08-09T13:33:54.365137+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}, 'read_balance': {'score_acting': 2, 'score_stable': 2, 'optimal_score': 1, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': '.mgr', 'create_time': '2023-08-09T13:33:54.831803+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-08-09T14:51:43.773 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get rbd pg_num 2023-08-09T14:51:44.000 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:43.999+0000 7f92b9463640 1 Processor -- start 2023-08-09T14:51:44.001 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:43.999+0000 7f92b9463640 1 -- start start 2023-08-09T14:51:44.002 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:44.000+0000 7f92b9463640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f92b40ae490 0x7f92b41670f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T14:51:44.002 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:44.000+0000 7f92b9463640 1 -- --> v1:172.21.15.43:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f92b40ae960 con 0x7f92b4153470 2023-08-09T14:51:44.002 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:44.001+0000 7f92b9463640 1 -- --> v1:172.21.15.203:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f92b415cb40 con 0x7f92b40b1530 2023-08-09T14:51:44.002 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:44.001+0000 7f92b9463640 1 -- --> v2:172.21.15.43:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f92b415cde0 con 0x7f92b40ae490 2023-08-09T14:51:44.003 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:44.001+0000 7f92b3fff640 1 --1- >> v1:172.21.15.43:6789/0 conn(0x7f92b4153470 0x7f92b40b0e20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.43:6789/0 says I am v1:172.21.15.43:46720/0 (socket says 172.21.15.43:46720) 2023-08-09T14:51:44.004 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:44.001+0000 7f92b3fff640 1 -- 172.21.15.43:0/4026510579 learned_addr learned my addr 172.21.15.43:0/4026510579 (peer_addr_for_me v1:172.21.15.43:0/0) 2023-08-09T14:51:44.004 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:44.001+0000 7f92b8c62640 1 --2- 172.21.15.43:0/4026510579 >> v2:172.21.15.43:3300/0 conn(0x7f92b40ae490 0x7f92b41670f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T14:51:44.004 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:44.001+0000 7f92b2ffd640 1 -- 172.21.15.43:0/4026510579 <== mon.0 v1:172.21.15.43:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1246146116 0 0) 0x7f92b40ae960 con 0x7f92b4153470 2023-08-09T14:51:44.005 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:44.002+0000 7f92b2ffd640 1 -- 172.21.15.43:0/4026510579 --> v1:172.21.15.43:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f929c003520 con 0x7f92b4153470 2023-08-09T14:51:44.005 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:44.002+0000 7f92b8c62640 1 -- 172.21.15.43:0/4026510579 >> v1:172.21.15.203:6789/0 conn(0x7f92b40b1530 legacy=0x7f92b40add80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T14:51:44.005 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:44.002+0000 7f92b8c62640 1 -- 172.21.15.43:0/4026510579 >> v1:172.21.15.43:6789/0 conn(0x7f92b4153470 legacy=0x7f92b40b0e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T14:51:44.005 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:44.002+0000 7f92b8c62640 1 -- 172.21.15.43:0/4026510579 --> v2:172.21.15.43:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f92b415d0c0 con 0x7f92b40ae490 2023-08-09T14:51:44.007 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:44.003+0000 7f92b8c62640 1 --2- 172.21.15.43:0/4026510579 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f92b40ae490 0x7f92b41670f0 secure :-1 s=READY pgs=1845 cs=0 l=1 rev1=1 crypto rx=0x7f92ac0097a0 tx=0x7f92ac02b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8808bf26cc5a4bb5 server_cookie=0 in_seq=0 out_seq=0 2023-08-09T14:51:44.007 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:44.003+0000 7f92b2ffd640 1 -- 172.21.15.43:0/4026510579 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f92ac031830 con 0x7f92b40ae490 2023-08-09T14:51:44.008 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:44.004+0000 7f92b2ffd640 1 -- 172.21.15.43:0/4026510579 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f92ac031990 con 0x7f92b40ae490 2023-08-09T14:51:44.008 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:44.004+0000 7f92b2ffd640 1 -- 172.21.15.43:0/4026510579 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f92ac031d00 con 0x7f92b40ae490 2023-08-09T14:51:44.008 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:44.004+0000 7f92b9463640 1 -- 172.21.15.43:0/4026510579 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f92b40ae490 msgr2=0x7f92b41670f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T14:51:44.008 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:44.004+0000 7f92b9463640 1 --2- 172.21.15.43:0/4026510579 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f92b40ae490 0x7f92b41670f0 secure :-1 s=READY pgs=1845 cs=0 l=1 rev1=1 crypto rx=0x7f92ac0097a0 tx=0x7f92ac02b8d0 comp rx=0 tx=0).stop 2023-08-09T14:51:44.008 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:44.004+0000 7f92b9463640 1 -- 172.21.15.43:0/4026510579 shutdown_connections 2023-08-09T14:51:44.009 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:44.004+0000 7f92b9463640 1 --2- 172.21.15.43:0/4026510579 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f92b40ae490 0x7f92b41670f0 unknown :-1 s=CLOSED pgs=1845 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T14:51:44.009 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:44.004+0000 7f92b9463640 1 -- 172.21.15.43:0/4026510579 >> 172.21.15.43:0/4026510579 conn(0x7f92b40b4400 msgr2=0x7f92b40b47e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-09T14:51:44.009 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:44.004+0000 7f92b9463640 1 -- 172.21.15.43:0/4026510579 shutdown_connections 2023-08-09T14:51:44.009 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:44.004+0000 7f92b9463640 1 -- 172.21.15.43:0/4026510579 wait complete. 2023-08-09T14:51:44.011 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:44.006+0000 7f92b9463640 1 Processor -- start 2023-08-09T14:51:44.011 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:44.006+0000 7f92b9463640 1 -- start start 2023-08-09T14:51:44.011 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:44.007+0000 7f92b9463640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f92b40ae490 0x7f92b4153e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T14:51:44.011 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:44.007+0000 7f92b9463640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f92b40b1530 0x7f92b413d2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T14:51:44.011 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:44.007+0000 7f92b9463640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f92b4153470 0x7f92b413d830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T14:51:44.011 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:44.007+0000 7f92b9463640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f92b413dd70 con 0x7f92b4153470 2023-08-09T14:51:44.012 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:44.007+0000 7f92b9463640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f92b413e020 con 0x7f92b40b1530 2023-08-09T14:51:44.012 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:44.007+0000 7f92b9463640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f92b4155dd0 con 0x7f92b40ae490 2023-08-09T14:51:44.012 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:44.008+0000 7f92b8c62640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f92b4153470 0x7f92b413d830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T14:51:44.012 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:44.008+0000 7f92b8c62640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f92b4153470 0x7f92b413d830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:52554/0 (socket says 172.21.15.43:52554) 2023-08-09T14:51:44.012 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:44.008+0000 7f92b8c62640 1 -- 172.21.15.43:0/595539387 learned_addr learned my addr 172.21.15.43:0/595539387 (peer_addr_for_me v2:172.21.15.43:0/0) 2023-08-09T14:51:44.012 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:44.008+0000 7f92b3fff640 1 --2- 172.21.15.43:0/595539387 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f92b40ae490 0x7f92b4153e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T14:51:44.013 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:44.008+0000 7f92b37fe640 1 --2- 172.21.15.43:0/595539387 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f92b40b1530 0x7f92b413d2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T14:51:44.013 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:44.008+0000 7f92b8c62640 1 -- 172.21.15.43:0/595539387 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f92b40ae490 msgr2=0x7f92b4153e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T14:51:44.013 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:44.008+0000 7f92b8c62640 1 --2- 172.21.15.43:0/595539387 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f92b40ae490 0x7f92b4153e60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T14:51:44.013 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:44.008+0000 7f92b8c62640 1 -- 172.21.15.43:0/595539387 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f92b40b1530 msgr2=0x7f92b413d2f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T14:51:44.013 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:44.008+0000 7f92b8c62640 1 --2- 172.21.15.43:0/595539387 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f92b40b1530 0x7f92b413d2f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T14:51:44.013 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:44.008+0000 7f92b8c62640 1 -- 172.21.15.43:0/595539387 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f92ac009400 con 0x7f92b4153470 2023-08-09T14:51:44.013 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:44.009+0000 7f92b8c62640 1 --2- 172.21.15.43:0/595539387 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f92b4153470 0x7f92b413d830 secure :-1 s=READY pgs=1846 cs=0 l=1 rev1=1 crypto rx=0x7f92ac034040 tx=0x7f92ac0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-09T14:51:44.013 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:44.009+0000 7f92b0ff9640 1 -- 172.21.15.43:0/595539387 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f92ac004230 con 0x7f92b4153470 2023-08-09T14:51:44.014 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:44.009+0000 7f92b0ff9640 1 -- 172.21.15.43:0/595539387 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f92ac004390 con 0x7f92b4153470 2023-08-09T14:51:44.014 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:44.009+0000 7f92b0ff9640 1 -- 172.21.15.43:0/595539387 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f92ac03a850 con 0x7f92b4153470 2023-08-09T14:51:44.014 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:44.009+0000 7f92b9463640 1 -- 172.21.15.43:0/595539387 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f92b41560b0 con 0x7f92b4153470 2023-08-09T14:51:44.014 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:44.009+0000 7f92b9463640 1 -- 172.21.15.43:0/595539387 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f92b4156670 con 0x7f92b4153470 2023-08-09T14:51:44.015 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:44.011+0000 7f92b9463640 1 -- 172.21.15.43:0/595539387 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f92b40b1350 con 0x7f92b4153470 2023-08-09T14:51:44.015 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:44.012+0000 7f92b0ff9640 1 -- 172.21.15.43:0/595539387 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 79) v1 ==== 88443+0+0 (secure 0 0 0) 0x7f92ac03a9b0 con 0x7f92b4153470 2023-08-09T14:51:44.015 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:44.012+0000 7f92b0ff9640 1 --2- 172.21.15.43:0/595539387 >> [v2:172.21.15.203:6832/2566816332,v1:172.21.15.203:6833/2566816332] conn(0x7f92a006a170 0x7f92a006c630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T14:51:44.015 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:44.013+0000 7f92b0ff9640 1 -- 172.21.15.43:0/595539387 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(168..168 src has 1..168) v4 ==== 5574+0+0 (secure 0 0 0) 0x7f92ac0aa8d0 con 0x7f92b4153470 2023-08-09T14:51:44.015 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:44.013+0000 7f92b3fff640 1 --2- 172.21.15.43:0/595539387 >> [v2:172.21.15.203:6832/2566816332,v1:172.21.15.203:6833/2566816332] conn(0x7f92a006a170 0x7f92a006c630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T14:51:44.016 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:44.015+0000 7f92b3fff640 1 --2- 172.21.15.43:0/595539387 >> [v2:172.21.15.203:6832/2566816332,v1:172.21.15.203:6833/2566816332] conn(0x7f92a006a170 0x7f92a006c630 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f92a4001560 tx=0x7f92a4009040 comp rx=0 tx=0).ready entity=mgr.7765 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-09T14:51:44.018 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:44.017+0000 7f92b0ff9640 1 -- 172.21.15.43:0/595539387 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178066 (secure 0 0 0) 0x7f92ac075ee0 con 0x7f92b4153470 2023-08-09T14:51:44.213 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:44.212+0000 7f92b9463640 1 -- 172.21.15.43:0/595539387 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"} v 0) v1 -- 0x7f92b40b2c10 con 0x7f92b4153470 2023-08-09T14:51:44.216 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:44.213+0000 7f92b0ff9640 1 -- 172.21.15.43:0/595539387 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"}]=0 v168) v1 ==== 92+0+10 (secure 0 0 0) 0x7f92ac079500 con 0x7f92b4153470 2023-08-09T14:51:44.217 INFO:teuthology.orchestra.run.smithi043.stdout:pg_num: 8 2023-08-09T14:51:44.217 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:44.215+0000 7f92b9463640 1 -- 172.21.15.43:0/595539387 >> [v2:172.21.15.203:6832/2566816332,v1:172.21.15.203:6833/2566816332] conn(0x7f92a006a170 msgr2=0x7f92a006c630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T14:51:44.217 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:44.215+0000 7f92b9463640 1 --2- 172.21.15.43:0/595539387 >> [v2:172.21.15.203:6832/2566816332,v1:172.21.15.203:6833/2566816332] conn(0x7f92a006a170 0x7f92a006c630 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f92a4001560 tx=0x7f92a4009040 comp rx=0 tx=0).stop 2023-08-09T14:51:44.218 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:44.215+0000 7f92b9463640 1 -- 172.21.15.43:0/595539387 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f92b4153470 msgr2=0x7f92b413d830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T14:51:44.218 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:44.215+0000 7f92b9463640 1 --2- 172.21.15.43:0/595539387 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f92b4153470 0x7f92b413d830 secure :-1 s=READY pgs=1846 cs=0 l=1 rev1=1 crypto rx=0x7f92ac034040 tx=0x7f92ac0046c0 comp rx=0 tx=0).stop 2023-08-09T14:51:44.218 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:44.215+0000 7f92b9463640 1 -- 172.21.15.43:0/595539387 shutdown_connections 2023-08-09T14:51:44.218 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:44.215+0000 7f92b9463640 1 --2- 172.21.15.43:0/595539387 >> [v2:172.21.15.203:6832/2566816332,v1:172.21.15.203:6833/2566816332] conn(0x7f92a006a170 0x7f92a006c630 unknown :-1 s=CLOSED pgs=53 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T14:51:44.218 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:44.215+0000 7f92b9463640 1 --2- 172.21.15.43:0/595539387 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f92b4153470 0x7f92b413d830 unknown :-1 s=CLOSED pgs=1846 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T14:51:44.218 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:44.215+0000 7f92b9463640 1 --2- 172.21.15.43:0/595539387 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f92b40b1530 0x7f92b413d2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T14:51:44.218 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:44.215+0000 7f92b9463640 1 --2- 172.21.15.43:0/595539387 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f92b40ae490 0x7f92b4153e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T14:51:44.219 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:44.215+0000 7f92b9463640 1 -- 172.21.15.43:0/595539387 >> 172.21.15.43:0/595539387 conn(0x7f92b40b4400 msgr2=0x7f92b40b47e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-09T14:51:44.219 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:44.216+0000 7f92b9463640 1 -- 172.21.15.43:0/595539387 shutdown_connections 2023-08-09T14:51:44.219 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:44.216+0000 7f92b9463640 1 -- 172.21.15.43:0/595539387 wait complete. 2023-08-09T14:51:44.235 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-08-09T14:51:44.475 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:44.473+0000 7f8526937640 1 Processor -- start 2023-08-09T14:51:44.475 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:44.473+0000 7f8526937640 1 -- start start 2023-08-09T14:51:44.477 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:44.475+0000 7f8526937640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f85200c9560 0x7f85200cb9a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T14:51:44.477 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:44.475+0000 7f8526937640 1 -- --> v1:172.21.15.43:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8520070250 con 0x7f85200c8900 2023-08-09T14:51:44.478 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:44.475+0000 7f8526937640 1 -- --> v1:172.21.15.203:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f85200c86c0 con 0x7f8520159400 2023-08-09T14:51:44.478 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:44.475+0000 7f8526937640 1 -- --> v2:172.21.15.43:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f852006fc60 con 0x7f85200c9560 2023-08-09T14:51:44.478 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:44.475+0000 7f8525134640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f85200c9560 0x7f85200cb9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T14:51:44.478 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:44.475+0000 7f8526136640 1 --1- >> v1:172.21.15.43:6789/0 conn(0x7f85200c8900 0x7f85200706e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.43:6789/0 says I am v1:172.21.15.43:46732/0 (socket says 172.21.15.43:46732) 2023-08-09T14:51:44.478 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:44.475+0000 7f8526136640 1 -- 172.21.15.43:0/1990962569 learned_addr learned my addr 172.21.15.43:0/1990962569 (peer_addr_for_me v1:172.21.15.43:0/0) 2023-08-09T14:51:44.479 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:44.476+0000 7f8525134640 1 -- 172.21.15.43:0/1990962569 >> v1:172.21.15.203:6789/0 conn(0x7f8520159400 legacy=0x7f85200c8e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T14:51:44.479 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:44.476+0000 7f8525134640 1 -- 172.21.15.43:0/1990962569 >> v1:172.21.15.43:6789/0 conn(0x7f85200c8900 legacy=0x7f85200706e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T14:51:44.480 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:44.476+0000 7f8525134640 1 -- 172.21.15.43:0/1990962569 --> v2:172.21.15.43:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f85200af290 con 0x7f85200c9560 2023-08-09T14:51:44.480 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:44.476+0000 7f8524933640 1 -- 172.21.15.43:0/1990962569 <== mon.0 v1:172.21.15.43:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2847102475 0 0) 0x7f8520070250 con 0x7f85200c8900 2023-08-09T14:51:44.480 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:44.477+0000 7f8525134640 1 --2- 172.21.15.43:0/1990962569 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f85200c9560 0x7f85200cb9a0 secure :-1 s=READY pgs=1848 cs=0 l=1 rev1=1 crypto rx=0x7f85100097a0 tx=0x7f851002b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ea32bc56290e3539 server_cookie=0 in_seq=0 out_seq=0 2023-08-09T14:51:44.480 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:44.477+0000 7f8524933640 1 -- 172.21.15.43:0/1990962569 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8510031830 con 0x7f85200c9560 2023-08-09T14:51:44.480 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:44.478+0000 7f8524933640 1 -- 172.21.15.43:0/1990962569 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8510031990 con 0x7f85200c9560 2023-08-09T14:51:44.480 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:44.478+0000 7f8524933640 1 -- 172.21.15.43:0/1990962569 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8510031d20 con 0x7f85200c9560 2023-08-09T14:51:44.481 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:44.478+0000 7f8526937640 1 -- 172.21.15.43:0/1990962569 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f85200c9560 msgr2=0x7f85200cb9a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T14:51:44.481 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:44.478+0000 7f8526937640 1 --2- 172.21.15.43:0/1990962569 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f85200c9560 0x7f85200cb9a0 secure :-1 s=READY pgs=1848 cs=0 l=1 rev1=1 crypto rx=0x7f85100097a0 tx=0x7f851002b8d0 comp rx=0 tx=0).stop 2023-08-09T14:51:44.482 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:44.478+0000 7f8526937640 1 -- 172.21.15.43:0/1990962569 shutdown_connections 2023-08-09T14:51:44.482 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:44.478+0000 7f8526937640 1 --2- 172.21.15.43:0/1990962569 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f85200c9560 0x7f85200cb9a0 unknown :-1 s=CLOSED pgs=1848 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T14:51:44.482 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:44.478+0000 7f8526937640 1 -- 172.21.15.43:0/1990962569 >> 172.21.15.43:0/1990962569 conn(0x7f85200cddd0 msgr2=0x7f85200ce1e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-09T14:51:44.482 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:44.478+0000 7f8526937640 1 -- 172.21.15.43:0/1990962569 shutdown_connections 2023-08-09T14:51:44.482 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:44.478+0000 7f8526937640 1 -- 172.21.15.43:0/1990962569 wait complete. 2023-08-09T14:51:44.482 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:44.480+0000 7f8526937640 1 Processor -- start 2023-08-09T14:51:44.483 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:44.480+0000 7f8526937640 1 -- start start 2023-08-09T14:51:44.484 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:44.481+0000 7f8526937640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f85200c8900 0x7f8520146400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T14:51:44.485 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:44.481+0000 7f8526937640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f85200c9560 0x7f85200c15a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T14:51:44.485 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:44.481+0000 7f8526937640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f85200c1ae0 0x7f852015c120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T14:51:44.485 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:44.481+0000 7f8526937640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f852015d6f0 con 0x7f85200c8900 2023-08-09T14:51:44.485 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:44.481+0000 7f8526937640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8520146940 con 0x7f85200c9560 2023-08-09T14:51:44.485 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:44.481+0000 7f8526937640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f852015d130 con 0x7f85200c1ae0 2023-08-09T14:51:44.486 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:44.482+0000 7f8525935640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f85200c8900 0x7f8520146400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T14:51:44.486 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:44.482+0000 7f8525935640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f85200c8900 0x7f8520146400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:52568/0 (socket says 172.21.15.43:52568) 2023-08-09T14:51:44.486 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:44.482+0000 7f8525935640 1 -- 172.21.15.43:0/1823259923 learned_addr learned my addr 172.21.15.43:0/1823259923 (peer_addr_for_me v2:172.21.15.43:0/0) 2023-08-09T14:51:44.486 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:44.482+0000 7f8526136640 1 --2- 172.21.15.43:0/1823259923 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f85200c1ae0 0x7f852015c120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T14:51:44.486 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:44.482+0000 7f8525134640 1 --2- 172.21.15.43:0/1823259923 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f85200c9560 0x7f85200c15a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T14:51:44.486 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:44.482+0000 7f8525935640 1 -- 172.21.15.43:0/1823259923 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f85200c1ae0 msgr2=0x7f852015c120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T14:51:44.486 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:44.482+0000 7f8525935640 1 --2- 172.21.15.43:0/1823259923 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f85200c1ae0 0x7f852015c120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T14:51:44.487 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:44.482+0000 7f8525935640 1 -- 172.21.15.43:0/1823259923 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f85200c9560 msgr2=0x7f85200c15a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T14:51:44.487 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:44.482+0000 7f8525935640 1 --2- 172.21.15.43:0/1823259923 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f85200c9560 0x7f85200c15a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T14:51:44.487 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:44.482+0000 7f8525935640 1 -- 172.21.15.43:0/1823259923 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8510009400 con 0x7f85200c8900 2023-08-09T14:51:44.487 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:44.482+0000 7f8525134640 1 --2- 172.21.15.43:0/1823259923 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f85200c9560 0x7f85200c15a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-08-09T14:51:44.487 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:44.482+0000 7f8525935640 1 --2- 172.21.15.43:0/1823259923 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f85200c8900 0x7f8520146400 secure :-1 s=READY pgs=1849 cs=0 l=1 rev1=1 crypto rx=0x7f851c00c730 tx=0x7f851c00cbf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-09T14:51:44.487 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:44.483+0000 7f85167fc640 1 -- 172.21.15.43:0/1823259923 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f851c003b70 con 0x7f85200c8900 2023-08-09T14:51:44.487 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:44.483+0000 7f85167fc640 1 -- 172.21.15.43:0/1823259923 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f851c003cf0 con 0x7f85200c8900 2023-08-09T14:51:44.488 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:44.483+0000 7f8526937640 1 -- 172.21.15.43:0/1823259923 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f852015d410 con 0x7f85200c8900 2023-08-09T14:51:44.488 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:44.483+0000 7f85167fc640 1 -- 172.21.15.43:0/1823259923 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f851c003ea0 con 0x7f85200c8900 2023-08-09T14:51:44.488 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:44.483+0000 7f8526937640 1 -- 172.21.15.43:0/1823259923 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f85200b0e70 con 0x7f85200c8900 2023-08-09T14:51:44.489 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:44.484+0000 7f8526937640 1 -- 172.21.15.43:0/1823259923 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f85200b09f0 con 0x7f85200c8900 2023-08-09T14:51:44.489 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:44.485+0000 7f85167fc640 1 -- 172.21.15.43:0/1823259923 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 79) v1 ==== 88443+0+0 (secure 0 0 0) 0x7f851c00aa90 con 0x7f85200c8900 2023-08-09T14:51:44.489 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:44.486+0000 7f85167fc640 1 --2- 172.21.15.43:0/1823259923 >> [v2:172.21.15.203:6832/2566816332,v1:172.21.15.203:6833/2566816332] conn(0x7f850406a170 0x7f850406c630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T14:51:44.489 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:44.486+0000 7f85167fc640 1 -- 172.21.15.43:0/1823259923 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(168..168 src has 1..168) v4 ==== 5574+0+0 (secure 0 0 0) 0x7f851c088400 con 0x7f85200c8900 2023-08-09T14:51:44.489 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:44.487+0000 7f8525134640 1 --2- 172.21.15.43:0/1823259923 >> [v2:172.21.15.203:6832/2566816332,v1:172.21.15.203:6833/2566816332] conn(0x7f850406a170 0x7f850406c630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T14:51:44.490 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:44.488+0000 7f8525134640 1 --2- 172.21.15.43:0/1823259923 >> [v2:172.21.15.203:6832/2566816332,v1:172.21.15.203:6833/2566816332] conn(0x7f850406a170 0x7f850406c630 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f8510036000 tx=0x7f8510008030 comp rx=0 tx=0).ready entity=mgr.7765 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-09T14:51:44.492 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:44.491+0000 7f85167fc640 1 -- 172.21.15.43:0/1823259923 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178066 (secure 0 0 0) 0x7f85200b09f0 con 0x7f85200c8900 2023-08-09T14:51:44.690 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:44.688+0000 7f8526937640 1 -- 172.21.15.43:0/1823259923 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f85200b09f0 con 0x7f85200c8900 2023-08-09T14:51:44.690 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:44.689+0000 7f85167fc640 1 -- 172.21.15.43:0/1823259923 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v168) v1 ==== 93+0+10 (secure 0 0 0) 0x7f85200b09f0 con 0x7f85200c8900 2023-08-09T14:51:44.690 INFO:teuthology.orchestra.run.smithi043.stdout:pg_num: 1 2023-08-09T14:51:44.693 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:44.691+0000 7f8526937640 1 -- 172.21.15.43:0/1823259923 >> [v2:172.21.15.203:6832/2566816332,v1:172.21.15.203:6833/2566816332] conn(0x7f850406a170 msgr2=0x7f850406c630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T14:51:44.693 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:44.691+0000 7f8526937640 1 --2- 172.21.15.43:0/1823259923 >> [v2:172.21.15.203:6832/2566816332,v1:172.21.15.203:6833/2566816332] conn(0x7f850406a170 0x7f850406c630 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f8510036000 tx=0x7f8510008030 comp rx=0 tx=0).stop 2023-08-09T14:51:44.693 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:44.691+0000 7f8526937640 1 -- 172.21.15.43:0/1823259923 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f85200c8900 msgr2=0x7f8520146400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T14:51:44.693 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:44.691+0000 7f8526937640 1 --2- 172.21.15.43:0/1823259923 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f85200c8900 0x7f8520146400 secure :-1 s=READY pgs=1849 cs=0 l=1 rev1=1 crypto rx=0x7f851c00c730 tx=0x7f851c00cbf0 comp rx=0 tx=0).stop 2023-08-09T14:51:44.693 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:44.691+0000 7f8526937640 1 -- 172.21.15.43:0/1823259923 shutdown_connections 2023-08-09T14:51:44.694 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:44.691+0000 7f8526937640 1 --2- 172.21.15.43:0/1823259923 >> [v2:172.21.15.203:6832/2566816332,v1:172.21.15.203:6833/2566816332] conn(0x7f850406a170 0x7f850406c630 unknown :-1 s=CLOSED pgs=54 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T14:51:44.694 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:44.691+0000 7f8526937640 1 --2- 172.21.15.43:0/1823259923 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f85200c1ae0 0x7f852015c120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T14:51:44.694 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:44.691+0000 7f8526937640 1 --2- 172.21.15.43:0/1823259923 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f85200c9560 0x7f85200c15a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T14:51:44.694 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:44.691+0000 7f8526937640 1 --2- 172.21.15.43:0/1823259923 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f85200c8900 0x7f8520146400 unknown :-1 s=CLOSED pgs=1849 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T14:51:44.694 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:44.691+0000 7f8526937640 1 -- 172.21.15.43:0/1823259923 >> 172.21.15.43:0/1823259923 conn(0x7f85200cddd0 msgr2=0x7f85200cb500 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-09T14:51:44.695 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:44.692+0000 7f8526937640 1 -- 172.21.15.43:0/1823259923 shutdown_connections 2023-08-09T14:51:44.695 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:44.692+0000 7f8526937640 1 -- 172.21.15.43:0/1823259923 wait complete. 2023-08-09T14:51:44.714 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --format=json-pretty osd lspools 2023-08-09T14:51:44.950 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:44.949+0000 7f839e457640 1 Processor -- start 2023-08-09T14:51:44.951 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:44.949+0000 7f839e457640 1 -- start start 2023-08-09T14:51:44.952 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:44.950+0000 7f839e457640 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7f83980cb6b0 0x7f8398165870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T14:51:44.952 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:44.950+0000 7f839e457640 1 -- --> v1:172.21.15.43:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f83980cd8a0 con 0x7f83980ce720 2023-08-09T14:51:44.952 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:44.950+0000 7f839e457640 1 -- --> v1:172.21.15.203:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f83980cf350 con 0x7f83980ce080 2023-08-09T14:51:44.952 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:44.950+0000 7f839e457640 1 -- --> v2:172.21.15.203:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8398071a70 con 0x7f83980cb6b0 2023-08-09T14:51:44.954 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:44.951+0000 7f8397fff640 1 --1- >> v1:172.21.15.43:6789/0 conn(0x7f83980ce720 0x7f83980ceb00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.43:6789/0 says I am v1:172.21.15.43:46738/0 (socket says 172.21.15.43:46738) 2023-08-09T14:51:44.954 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:44.951+0000 7f8397fff640 1 -- 172.21.15.43:0/3454906258 learned_addr learned my addr 172.21.15.43:0/3454906258 (peer_addr_for_me v1:172.21.15.43:0/0) 2023-08-09T14:51:44.954 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:44.951+0000 7f839c9f4640 1 --2- 172.21.15.43:0/3454906258 >> v2:172.21.15.203:3300/0 conn(0x7f83980cb6b0 0x7f8398165870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T14:51:44.955 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:44.951+0000 7f83977fe640 1 -- 172.21.15.43:0/3454906258 <== mon.0 v1:172.21.15.43:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3795294432 0 0) 0x7f83980cd8a0 con 0x7f83980ce720 2023-08-09T14:51:44.955 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:44.951+0000 7f83977fe640 1 -- 172.21.15.43:0/3454906258 --> v1:172.21.15.43:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f8384003520 con 0x7f83980ce720 2023-08-09T14:51:44.955 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:44.952+0000 7f83977fe640 1 -- 172.21.15.43:0/3454906258 <== mon.0 v1:172.21.15.43:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3425086128 0 0) 0x7f8384003520 con 0x7f83980ce720 2023-08-09T14:51:44.955 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:44.952+0000 7f83977fe640 1 -- 172.21.15.43:0/3454906258 >> v2:172.21.15.203:3300/0 conn(0x7f83980cb6b0 msgr2=0x7f8398165870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-09T14:51:44.955 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:44.952+0000 7f83977fe640 1 --2- 172.21.15.43:0/3454906258 >> v2:172.21.15.203:3300/0 conn(0x7f83980cb6b0 0x7f8398165870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T14:51:44.955 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:44.952+0000 7f83977fe640 1 -- 172.21.15.43:0/3454906258 >> v1:172.21.15.203:6789/0 conn(0x7f83980ce080 legacy=0x7f83980cafa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T14:51:44.955 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:44.952+0000 7f83977fe640 1 -- 172.21.15.43:0/3454906258 --> v1:172.21.15.43:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f83980af260 con 0x7f83980ce720 2023-08-09T14:51:44.956 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:44.952+0000 7f83977fe640 1 -- 172.21.15.43:0/3454906258 <== mon.0 v1:172.21.15.43:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1285150740 0 0) 0x7f83880017f0 con 0x7f83980ce720 2023-08-09T14:51:44.956 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:44.952+0000 7f83977fe640 1 -- 172.21.15.43:0/3454906258 >> v1:172.21.15.43:6789/0 conn(0x7f83980ce720 legacy=0x7f83980ceb00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T14:51:44.956 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:44.952+0000 7f83977fe640 1 --2- 172.21.15.43:0/3454906258 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f8384003a10 0x7f8384003e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T14:51:44.956 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:44.952+0000 7f83977fe640 1 --2- 172.21.15.43:0/3454906258 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f83840047c0 0x7f8384004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T14:51:44.956 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:44.952+0000 7f83977fe640 1 --2- 172.21.15.43:0/3454906258 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f83840052a0 0x7f838400b980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T14:51:44.956 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:44.952+0000 7f83977fe640 1 -- 172.21.15.43:0/3454906258 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8398071a70 con 0x7f83840052a0 2023-08-09T14:51:44.956 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:44.952+0000 7f83977fe640 1 -- 172.21.15.43:0/3454906258 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f838400c170 con 0x7f83840047c0 2023-08-09T14:51:44.957 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:44.952+0000 7f83977fe640 1 -- 172.21.15.43:0/3454906258 --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f838400c330 con 0x7f8384003a10 2023-08-09T14:51:44.957 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:44.952+0000 7f839c9f4640 1 --2- 172.21.15.43:0/3454906258 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f83840052a0 0x7f838400b980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T14:51:44.957 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:44.953+0000 7f8397fff640 1 --2- 172.21.15.43:0/3454906258 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f8384003a10 0x7f8384003e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T14:51:44.957 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:44.953+0000 7f839c9f4640 1 -- 172.21.15.43:0/3454906258 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f8384003a10 msgr2=0x7f8384003e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-09T14:51:44.957 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:44.953+0000 7f839c9f4640 1 --2- 172.21.15.43:0/3454906258 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f8384003a10 0x7f8384003e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T14:51:44.957 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:44.953+0000 7f839c9f4640 1 -- 172.21.15.43:0/3454906258 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f83840047c0 msgr2=0x7f8384004c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-09T14:51:44.958 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:44.953+0000 7f839c9f4640 1 --2- 172.21.15.43:0/3454906258 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f83840047c0 0x7f8384004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T14:51:44.958 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:44.953+0000 7f839c9f4640 1 -- 172.21.15.43:0/3454906258 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8384003520 con 0x7f83840052a0 2023-08-09T14:51:44.958 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:44.953+0000 7f8397fff640 1 --2- 172.21.15.43:0/3454906258 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f8384003a10 0x7f8384003e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-08-09T14:51:44.959 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:44.954+0000 7f839c9f4640 1 -- 172.21.15.43:0/3454906258 reap_dead start 2023-08-09T14:51:44.959 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:44.954+0000 7f839c9f4640 1 --2- 172.21.15.43:0/3454906258 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f83840052a0 0x7f838400b980 secure :-1 s=READY pgs=1851 cs=0 l=1 rev1=1 crypto rx=0x7f8390009f90 tx=0x7f83840057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b16d084314c16f8d server_cookie=0 in_seq=0 out_seq=0 2023-08-09T14:51:44.959 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:44.954+0000 7f83977fe640 1 -- 172.21.15.43:0/3454906258 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f839002cc20 con 0x7f83840052a0 2023-08-09T14:51:44.959 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:44.954+0000 7f83977fe640 1 -- 172.21.15.43:0/3454906258 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f839000f040 con 0x7f83840052a0 2023-08-09T14:51:44.960 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:44.954+0000 7f83977fe640 1 -- 172.21.15.43:0/3454906258 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8390003990 con 0x7f83840052a0 2023-08-09T14:51:44.960 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:44.955+0000 7f839e457640 1 -- 172.21.15.43:0/3454906258 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f83840052a0 msgr2=0x7f838400b980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T14:51:44.960 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:44.955+0000 7f839e457640 1 --2- 172.21.15.43:0/3454906258 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f83840052a0 0x7f838400b980 secure :-1 s=READY pgs=1851 cs=0 l=1 rev1=1 crypto rx=0x7f8390009f90 tx=0x7f83840057f0 comp rx=0 tx=0).stop 2023-08-09T14:51:44.961 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:44.955+0000 7f839e457640 1 -- 172.21.15.43:0/3454906258 shutdown_connections 2023-08-09T14:51:44.961 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:44.955+0000 7f839e457640 1 --2- 172.21.15.43:0/3454906258 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f83840052a0 0x7f838400b980 unknown :-1 s=CLOSED pgs=1851 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T14:51:44.961 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:44.955+0000 7f839e457640 1 -- 172.21.15.43:0/3454906258 >> 172.21.15.43:0/3454906258 conn(0x7f83980ccd80 msgr2=0x7f83980cd190 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-09T14:51:44.961 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:44.955+0000 7f839e457640 1 -- 172.21.15.43:0/3454906258 shutdown_connections 2023-08-09T14:51:44.961 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:44.955+0000 7f839e457640 1 -- 172.21.15.43:0/3454906258 wait complete. 2023-08-09T14:51:44.961 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:44.957+0000 7f839e457640 1 Processor -- start 2023-08-09T14:51:44.961 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:44.957+0000 7f839e457640 1 -- start start 2023-08-09T14:51:44.963 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:44.958+0000 7f839e457640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f83840052a0 0x7f83980b3000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T14:51:44.963 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:44.958+0000 7f839e457640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f839806b290 0x7f839814e290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T14:51:44.963 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:44.958+0000 7f839e457640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f83980b3540 0x7f839814e7d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T14:51:44.964 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:44.958+0000 7f839e457640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f839806c290 con 0x7f83840052a0 2023-08-09T14:51:44.964 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:44.958+0000 7f839e457640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f839806b710 con 0x7f83980b3540 2023-08-09T14:51:44.964 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:44.958+0000 7f839e457640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f839806b9f0 con 0x7f839806b290 2023-08-09T14:51:44.964 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:44.959+0000 7f8397fff640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f83840052a0 0x7f83980b3000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T14:51:44.964 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:44.959+0000 7f8397fff640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f83840052a0 0x7f83980b3000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:57788/0 (socket says 172.21.15.43:57788) 2023-08-09T14:51:44.964 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:44.959+0000 7f8397fff640 1 -- 172.21.15.43:0/1290995461 learned_addr learned my addr 172.21.15.43:0/1290995461 (peer_addr_for_me v2:172.21.15.43:0/0) 2023-08-09T14:51:44.964 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:44.959+0000 7f838f7fe640 1 --2- 172.21.15.43:0/1290995461 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f839806b290 0x7f839814e290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T14:51:44.964 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:44.959+0000 7f8397fff640 1 -- 172.21.15.43:0/1290995461 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f839806b290 msgr2=0x7f839814e290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T14:51:44.965 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:44.959+0000 7f839c9f4640 1 --2- 172.21.15.43:0/1290995461 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f83980b3540 0x7f839814e7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T14:51:44.965 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:44.959+0000 7f8397fff640 1 --2- 172.21.15.43:0/1290995461 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f839806b290 0x7f839814e290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T14:51:44.965 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:44.959+0000 7f8397fff640 1 -- 172.21.15.43:0/1290995461 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f83980b3540 msgr2=0x7f839814e7d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T14:51:44.965 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:44.959+0000 7f8397fff640 1 --2- 172.21.15.43:0/1290995461 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f83980b3540 0x7f839814e7d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T14:51:44.965 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:44.959+0000 7f8397fff640 1 -- 172.21.15.43:0/1290995461 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f839806bcd0 con 0x7f83840052a0 2023-08-09T14:51:44.965 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:44.959+0000 7f838f7fe640 1 --2- 172.21.15.43:0/1290995461 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f839806b290 0x7f839814e290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-08-09T14:51:44.965 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:44.959+0000 7f839c9f4640 1 --2- 172.21.15.43:0/1290995461 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f83980b3540 0x7f839814e7d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-08-09T14:51:44.966 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:44.960+0000 7f8397fff640 1 --2- 172.21.15.43:0/1290995461 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f83840052a0 0x7f83980b3000 secure :-1 s=READY pgs=1852 cs=0 l=1 rev1=1 crypto rx=0x7f838800d7a0 tx=0x7f838800dc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-09T14:51:44.966 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:44.960+0000 7f83957fa640 1 -- 172.21.15.43:0/1290995461 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8388007480 con 0x7f83840052a0 2023-08-09T14:51:44.966 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:44.960+0000 7f83957fa640 1 -- 172.21.15.43:0/1290995461 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f83880075e0 con 0x7f83840052a0 2023-08-09T14:51:44.966 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:44.960+0000 7f83957fa640 1 -- 172.21.15.43:0/1290995461 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f83880059a0 con 0x7f83840052a0 2023-08-09T14:51:44.966 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:44.960+0000 7f839e457640 1 -- 172.21.15.43:0/1290995461 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f839806bfb0 con 0x7f83840052a0 2023-08-09T14:51:44.966 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:44.960+0000 7f839e457640 1 -- 172.21.15.43:0/1290995461 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f83981f15f0 con 0x7f83840052a0 2023-08-09T14:51:44.966 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:44.961+0000 7f839e457640 1 -- 172.21.15.43:0/1290995461 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f83980739f0 con 0x7f83840052a0 2023-08-09T14:51:44.967 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:44.962+0000 7f83957fa640 1 -- 172.21.15.43:0/1290995461 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 79) v1 ==== 88443+0+0 (secure 0 0 0) 0x7f8388005b00 con 0x7f83840052a0 2023-08-09T14:51:44.967 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:44.964+0000 7f83957fa640 1 --2- 172.21.15.43:0/1290995461 >> [v2:172.21.15.203:6832/2566816332,v1:172.21.15.203:6833/2566816332] conn(0x7f837806a170 0x7f837806c630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T14:51:44.967 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:44.964+0000 7f83957fa640 1 -- 172.21.15.43:0/1290995461 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(168..168 src has 1..168) v4 ==== 5574+0+0 (secure 0 0 0) 0x7f83880887b0 con 0x7f83840052a0 2023-08-09T14:51:44.968 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:44.965+0000 7f838f7fe640 1 --2- 172.21.15.43:0/1290995461 >> [v2:172.21.15.203:6832/2566816332,v1:172.21.15.203:6833/2566816332] conn(0x7f837806a170 0x7f837806c630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T14:51:44.968 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:44.966+0000 7f838f7fe640 1 --2- 172.21.15.43:0/1290995461 >> [v2:172.21.15.203:6832/2566816332,v1:172.21.15.203:6833/2566816332] conn(0x7f837806a170 0x7f837806c630 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f839806c4c0 tx=0x7f8380009040 comp rx=0 tx=0).ready entity=mgr.7765 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-09T14:51:44.969 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:44.968+0000 7f83957fa640 1 -- 172.21.15.43:0/1290995461 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178066 (secure 0 0 0) 0x7f8388053ed0 con 0x7f83840052a0 2023-08-09T14:51:45.164 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:45.162+0000 7f839e457640 1 -- 172.21.15.43:0/1290995461 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) v1 -- 0x7f839814c230 con 0x7f83840052a0 2023-08-09T14:51:45.164 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:45.163+0000 7f83957fa640 1 -- 172.21.15.43:0/1290995461 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v168) v1 ==== 84+0+126 (secure 0 0 0) 0x7f8388005db0 con 0x7f83840052a0 2023-08-09T14:51:45.165 INFO:teuthology.orchestra.run.smithi043.stdout: 2023-08-09T14:51:45.165 INFO:teuthology.orchestra.run.smithi043.stdout:[ 2023-08-09T14:51:45.165 INFO:teuthology.orchestra.run.smithi043.stdout: { 2023-08-09T14:51:45.165 INFO:teuthology.orchestra.run.smithi043.stdout: "poolnum": 1, 2023-08-09T14:51:45.165 INFO:teuthology.orchestra.run.smithi043.stdout: "poolname": "rbd" 2023-08-09T14:51:45.166 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2023-08-09T14:51:45.166 INFO:teuthology.orchestra.run.smithi043.stdout: { 2023-08-09T14:51:45.166 INFO:teuthology.orchestra.run.smithi043.stdout: "poolnum": 2, 2023-08-09T14:51:45.166 INFO:teuthology.orchestra.run.smithi043.stdout: "poolname": ".mgr" 2023-08-09T14:51:45.166 INFO:teuthology.orchestra.run.smithi043.stdout: } 2023-08-09T14:51:45.166 INFO:teuthology.orchestra.run.smithi043.stdout:] 2023-08-09T14:51:45.167 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:45.165+0000 7f839e457640 1 -- 172.21.15.43:0/1290995461 >> [v2:172.21.15.203:6832/2566816332,v1:172.21.15.203:6833/2566816332] conn(0x7f837806a170 msgr2=0x7f837806c630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T14:51:45.167 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:45.165+0000 7f839e457640 1 --2- 172.21.15.43:0/1290995461 >> [v2:172.21.15.203:6832/2566816332,v1:172.21.15.203:6833/2566816332] conn(0x7f837806a170 0x7f837806c630 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f839806c4c0 tx=0x7f8380009040 comp rx=0 tx=0).stop 2023-08-09T14:51:45.168 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:45.165+0000 7f839e457640 1 -- 172.21.15.43:0/1290995461 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f83840052a0 msgr2=0x7f83980b3000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T14:51:45.168 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:45.165+0000 7f839e457640 1 --2- 172.21.15.43:0/1290995461 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f83840052a0 0x7f83980b3000 secure :-1 s=READY pgs=1852 cs=0 l=1 rev1=1 crypto rx=0x7f838800d7a0 tx=0x7f838800dc60 comp rx=0 tx=0).stop 2023-08-09T14:51:45.168 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:45.166+0000 7f839e457640 1 -- 172.21.15.43:0/1290995461 shutdown_connections 2023-08-09T14:51:45.169 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:45.166+0000 7f839e457640 1 --2- 172.21.15.43:0/1290995461 >> [v2:172.21.15.203:6832/2566816332,v1:172.21.15.203:6833/2566816332] conn(0x7f837806a170 0x7f837806c630 unknown :-1 s=CLOSED pgs=55 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T14:51:45.169 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:45.166+0000 7f839e457640 1 --2- 172.21.15.43:0/1290995461 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f83980b3540 0x7f839814e7d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T14:51:45.169 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:45.166+0000 7f839e457640 1 --2- 172.21.15.43:0/1290995461 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f839806b290 0x7f839814e290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T14:51:45.169 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:45.166+0000 7f839e457640 1 --2- 172.21.15.43:0/1290995461 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f83840052a0 0x7f83980b3000 unknown :-1 s=CLOSED pgs=1852 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T14:51:45.169 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:45.166+0000 7f839e457640 1 -- 172.21.15.43:0/1290995461 >> 172.21.15.43:0/1290995461 conn(0x7f83980ccd80 msgr2=0x7f839815da90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-09T14:51:45.169 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:45.166+0000 7f839e457640 1 -- 172.21.15.43:0/1290995461 shutdown_connections 2023-08-09T14:51:45.170 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:45.166+0000 7f839e457640 1 -- 172.21.15.43:0/1290995461 wait complete. 2023-08-09T14:51:45.188 DEBUG:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2023-08-09T14:51:45.189 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_metadata --pg_num_min 64 2023-08-09T14:51:45.416 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:45.414+0000 7fb6fb638640 1 Processor -- start 2023-08-09T14:51:45.417 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:45.415+0000 7fb6fb638640 1 -- start start 2023-08-09T14:51:45.418 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:45.416+0000 7fb6fb638640 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7fb6f406fb70 0x7fb6f40caab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T14:51:45.418 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:45.416+0000 7fb6fb638640 1 -- --> v1:172.21.15.43:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb6f415b9b0 con 0x7fb6f40ce4c0 2023-08-09T14:51:45.419 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:45.416+0000 7fb6fb638640 1 -- --> v1:172.21.15.203:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb6f415ae10 con 0x7fb6f40ca370 2023-08-09T14:51:45.419 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:45.416+0000 7fb6fb638640 1 -- --> v2:172.21.15.203:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb6f4154e70 con 0x7fb6f406fb70 2023-08-09T14:51:45.419 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:45.417+0000 7fb6f9bd5640 1 --1- >> v1:172.21.15.43:6789/0 conn(0x7fb6f40ce4c0 0x7fb6f40c86f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.43:6789/0 says I am v1:172.21.15.43:47246/0 (socket says 172.21.15.43:47246) 2023-08-09T14:51:45.420 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:45.417+0000 7fb6f9bd5640 1 -- 172.21.15.43:0/4287304577 learned_addr learned my addr 172.21.15.43:0/4287304577 (peer_addr_for_me v1:172.21.15.43:0/0) 2023-08-09T14:51:45.423 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:45.417+0000 7fb6f93d4640 1 --2- 172.21.15.43:0/4287304577 >> v2:172.21.15.203:3300/0 conn(0x7fb6f406fb70 0x7fb6f40caab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T14:51:45.424 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:45.417+0000 7fb6ebfff640 1 -- 172.21.15.43:0/4287304577 <== mon.0 v1:172.21.15.43:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2405075520 0 0) 0x7fb6ec001490 con 0x7fb6f40ce4c0 2023-08-09T14:51:45.424 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:45.418+0000 7fb6ebfff640 1 -- 172.21.15.43:0/4287304577 --> v1:172.21.15.43:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fb6e0003520 con 0x7fb6f40ce4c0 2023-08-09T14:51:45.424 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:45.418+0000 7fb6ebfff640 1 -- 172.21.15.43:0/4287304577 <== mon.0 v1:172.21.15.43:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2635661657 0 0) 0x7fb6e0003520 con 0x7fb6f40ce4c0 2023-08-09T14:51:45.424 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:45.418+0000 7fb6ebfff640 1 -- 172.21.15.43:0/4287304577 >> v2:172.21.15.203:3300/0 conn(0x7fb6f406fb70 msgr2=0x7fb6f40caab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-09T14:51:45.424 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:45.418+0000 7fb6ebfff640 1 --2- 172.21.15.43:0/4287304577 >> v2:172.21.15.203:3300/0 conn(0x7fb6f406fb70 0x7fb6f40caab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T14:51:45.424 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:45.418+0000 7fb6ebfff640 1 -- 172.21.15.43:0/4287304577 >> v1:172.21.15.203:6789/0 conn(0x7fb6f40ca370 legacy=0x7fb6f415b2a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T14:51:45.424 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:45.418+0000 7fb6ebfff640 1 -- 172.21.15.43:0/4287304577 --> v1:172.21.15.43:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb6f406ff50 con 0x7fb6f40ce4c0 2023-08-09T14:51:45.425 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:45.418+0000 7fb6ebfff640 1 -- 172.21.15.43:0/4287304577 <== mon.0 v1:172.21.15.43:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1285150740 0 0) 0x7fb6ec001b80 con 0x7fb6f40ce4c0 2023-08-09T14:51:45.425 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:45.418+0000 7fb6ebfff640 1 -- 172.21.15.43:0/4287304577 >> v1:172.21.15.43:6789/0 conn(0x7fb6f40ce4c0 legacy=0x7fb6f40c86f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T14:51:45.425 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:45.418+0000 7fb6ebfff640 1 --2- 172.21.15.43:0/4287304577 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fb6e00039e0 0x7fb6e0003e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T14:51:45.425 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:45.418+0000 7fb6ebfff640 1 --2- 172.21.15.43:0/4287304577 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb6e00047c0 0x7fb6e0004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T14:51:45.425 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:45.418+0000 7fb6ebfff640 1 --2- 172.21.15.43:0/4287304577 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fb6e0005180 0x7fb6e000b980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T14:51:45.425 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:45.418+0000 7fb6ebfff640 1 -- 172.21.15.43:0/4287304577 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb6f4154e70 con 0x7fb6e00047c0 2023-08-09T14:51:45.425 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:45.418+0000 7fb6ebfff640 1 -- 172.21.15.43:0/4287304577 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb6e000c0b0 con 0x7fb6e00039e0 2023-08-09T14:51:45.425 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:45.418+0000 7fb6ebfff640 1 -- 172.21.15.43:0/4287304577 --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb6e000c270 con 0x7fb6e0005180 2023-08-09T14:51:45.426 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:45.419+0000 7fb6f8bd3640 1 --2- 172.21.15.43:0/4287304577 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb6e00047c0 0x7fb6e0004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T14:51:45.426 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:45.419+0000 7fb6f93d4640 1 --2- 172.21.15.43:0/4287304577 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fb6e00039e0 0x7fb6e0003e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T14:51:45.426 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:45.419+0000 7fb6f9bd5640 1 --2- 172.21.15.43:0/4287304577 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fb6e0005180 0x7fb6e000b980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T14:51:45.426 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:45.419+0000 7fb6f8bd3640 1 -- 172.21.15.43:0/4287304577 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fb6e0005180 msgr2=0x7fb6e000b980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-09T14:51:45.426 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:45.419+0000 7fb6f8bd3640 1 --2- 172.21.15.43:0/4287304577 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fb6e0005180 0x7fb6e000b980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T14:51:45.426 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:45.419+0000 7fb6f8bd3640 1 -- 172.21.15.43:0/4287304577 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fb6e00039e0 msgr2=0x7fb6e0003e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-09T14:51:45.426 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:45.419+0000 7fb6f8bd3640 1 --2- 172.21.15.43:0/4287304577 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fb6e00039e0 0x7fb6e0003e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T14:51:45.427 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:45.419+0000 7fb6f8bd3640 1 -- 172.21.15.43:0/4287304577 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb6e0003520 con 0x7fb6e00047c0 2023-08-09T14:51:45.427 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:45.419+0000 7fb6f9bd5640 1 -- 172.21.15.43:0/4287304577 reap_dead start 2023-08-09T14:51:45.427 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:45.420+0000 7fb6f8bd3640 1 --2- 172.21.15.43:0/4287304577 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb6e00047c0 0x7fb6e0004c40 secure :-1 s=READY pgs=1854 cs=0 l=1 rev1=1 crypto rx=0x7fb6e4001750 tx=0x7fb6e00057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=99f73c28854286ae server_cookie=0 in_seq=0 out_seq=0 2023-08-09T14:51:45.427 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:45.421+0000 7fb6ebfff640 1 -- 172.21.15.43:0/4287304577 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb6e40052d0 con 0x7fb6e00047c0 2023-08-09T14:51:45.427 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:45.421+0000 7fb6ebfff640 1 -- 172.21.15.43:0/4287304577 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb6e402e050 con 0x7fb6e00047c0 2023-08-09T14:51:45.427 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:45.421+0000 7fb6ebfff640 1 -- 172.21.15.43:0/4287304577 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb6e4003830 con 0x7fb6e00047c0 2023-08-09T14:51:45.427 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:45.421+0000 7fb6fb638640 1 -- 172.21.15.43:0/4287304577 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb6e00047c0 msgr2=0x7fb6e0004c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T14:51:45.427 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:45.421+0000 7fb6fb638640 1 --2- 172.21.15.43:0/4287304577 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb6e00047c0 0x7fb6e0004c40 secure :-1 s=READY pgs=1854 cs=0 l=1 rev1=1 crypto rx=0x7fb6e4001750 tx=0x7fb6e00057f0 comp rx=0 tx=0).stop 2023-08-09T14:51:45.428 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:45.421+0000 7fb6fb638640 1 -- 172.21.15.43:0/4287304577 shutdown_connections 2023-08-09T14:51:45.428 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:45.421+0000 7fb6fb638640 1 --2- 172.21.15.43:0/4287304577 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb6e00047c0 0x7fb6e0004c40 unknown :-1 s=CLOSED pgs=1854 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T14:51:45.428 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:45.421+0000 7fb6fb638640 1 -- 172.21.15.43:0/4287304577 >> 172.21.15.43:0/4287304577 conn(0x7fb6f406b990 msgr2=0x7fb6f406bda0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-09T14:51:45.428 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:45.422+0000 7fb6fb638640 1 -- 172.21.15.43:0/4287304577 shutdown_connections 2023-08-09T14:51:45.428 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:45.422+0000 7fb6fb638640 1 -- 172.21.15.43:0/4287304577 wait complete. 2023-08-09T14:51:45.430 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:45.423+0000 7fb6fb638640 1 Processor -- start 2023-08-09T14:51:45.430 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:45.424+0000 7fb6fb638640 1 -- start start 2023-08-09T14:51:45.430 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:45.424+0000 7fb6fb638640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fb6e00047c0 0x7fb6f41f1230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T14:51:45.430 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:45.425+0000 7fb6fb638640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb6f41470a0 0x7fb6f415ccd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T14:51:45.430 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:45.425+0000 7fb6fb638640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fb6f4147520 0x7fb6f415d210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T14:51:45.431 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:45.425+0000 7fb6fb638640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb6f41f1770 con 0x7fb6f41470a0 2023-08-09T14:51:45.431 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:45.425+0000 7fb6fb638640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb6f40c4110 con 0x7fb6e00047c0 2023-08-09T14:51:45.431 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:45.425+0000 7fb6fb638640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb6f40c43f0 con 0x7fb6f4147520 2023-08-09T14:51:45.431 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:45.425+0000 7fb6f8bd3640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb6f41470a0 0x7fb6f415ccd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T14:51:45.431 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:45.425+0000 7fb6f8bd3640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb6f41470a0 0x7fb6f415ccd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:57804/0 (socket says 172.21.15.43:57804) 2023-08-09T14:51:45.431 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:45.425+0000 7fb6f8bd3640 1 -- 172.21.15.43:0/2544900798 learned_addr learned my addr 172.21.15.43:0/2544900798 (peer_addr_for_me v2:172.21.15.43:0/0) 2023-08-09T14:51:45.431 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:45.425+0000 7fb6f93d4640 1 --2- 172.21.15.43:0/2544900798 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fb6e00047c0 0x7fb6f41f1230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T14:51:45.432 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:45.425+0000 7fb6f9bd5640 1 --2- 172.21.15.43:0/2544900798 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fb6f4147520 0x7fb6f415d210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T14:51:45.432 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:45.425+0000 7fb6f8bd3640 1 -- 172.21.15.43:0/2544900798 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fb6f4147520 msgr2=0x7fb6f415d210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T14:51:45.432 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:45.425+0000 7fb6f8bd3640 1 --2- 172.21.15.43:0/2544900798 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fb6f4147520 0x7fb6f415d210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T14:51:45.432 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:45.425+0000 7fb6f8bd3640 1 -- 172.21.15.43:0/2544900798 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fb6e00047c0 msgr2=0x7fb6f41f1230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T14:51:45.432 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:45.425+0000 7fb6f8bd3640 1 --2- 172.21.15.43:0/2544900798 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fb6e00047c0 0x7fb6f41f1230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T14:51:45.433 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:45.425+0000 7fb6f8bd3640 1 -- 172.21.15.43:0/2544900798 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb6f40c46d0 con 0x7fb6f41470a0 2023-08-09T14:51:45.433 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:45.426+0000 7fb6f8bd3640 1 --2- 172.21.15.43:0/2544900798 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb6f41470a0 0x7fb6f415ccd0 secure :-1 s=READY pgs=1855 cs=0 l=1 rev1=1 crypto rx=0x7fb6e402bc00 tx=0x7fb6e4001120 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-09T14:51:45.433 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:45.426+0000 7fb6e9ffb640 1 -- 172.21.15.43:0/2544900798 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb6e40076c0 con 0x7fb6f41470a0 2023-08-09T14:51:45.433 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:45.426+0000 7fb6e9ffb640 1 -- 172.21.15.43:0/2544900798 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb6e4007a30 con 0x7fb6f41470a0 2023-08-09T14:51:45.433 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:45.426+0000 7fb6e9ffb640 1 -- 172.21.15.43:0/2544900798 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb6e4006be0 con 0x7fb6f41470a0 2023-08-09T14:51:45.433 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:45.426+0000 7fb6fb638640 1 -- 172.21.15.43:0/2544900798 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb6f40c49b0 con 0x7fb6f41470a0 2023-08-09T14:51:45.433 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:45.426+0000 7fb6fb638640 1 -- 172.21.15.43:0/2544900798 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb6f40c3dc0 con 0x7fb6f41470a0 2023-08-09T14:51:45.434 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:45.427+0000 7fb6fb638640 1 -- 172.21.15.43:0/2544900798 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb6f40739f0 con 0x7fb6f41470a0 2023-08-09T14:51:45.434 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:45.429+0000 7fb6e9ffb640 1 -- 172.21.15.43:0/2544900798 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 79) v1 ==== 88443+0+0 (secure 0 0 0) 0x7fb6e4006d40 con 0x7fb6f41470a0 2023-08-09T14:51:45.434 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:45.430+0000 7fb6e9ffb640 1 --2- 172.21.15.43:0/2544900798 >> [v2:172.21.15.203:6832/2566816332,v1:172.21.15.203:6833/2566816332] conn(0x7fb6d806a170 0x7fb6d806c630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T14:51:45.435 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:45.430+0000 7fb6e9ffb640 1 -- 172.21.15.43:0/2544900798 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(168..168 src has 1..168) v4 ==== 5574+0+0 (secure 0 0 0) 0x7fb6e40aa2d0 con 0x7fb6f41470a0 2023-08-09T14:51:45.435 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:45.431+0000 7fb6f93d4640 1 --2- 172.21.15.43:0/2544900798 >> [v2:172.21.15.203:6832/2566816332,v1:172.21.15.203:6833/2566816332] conn(0x7fb6d806a170 0x7fb6d806c630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T14:51:45.435 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:45.432+0000 7fb6f93d4640 1 --2- 172.21.15.43:0/2544900798 >> [v2:172.21.15.203:6832/2566816332,v1:172.21.15.203:6833/2566816332] conn(0x7fb6d806a170 0x7fb6d806c630 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7fb6f0009b80 tx=0x7fb6f0008040 comp rx=0 tx=0).ready entity=mgr.7765 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-09T14:51:45.436 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:45.434+0000 7fb6e9ffb640 1 -- 172.21.15.43:0/2544900798 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178066 (secure 0 0 0) 0x7fb6e4079400 con 0x7fb6f41470a0 2023-08-09T14:51:45.630 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:45.629+0000 7fb6fb638640 1 -- 172.21.15.43:0/2544900798 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64} v 0) v1 -- 0x7fb6f40c4530 con 0x7fb6f41470a0 2023-08-09T14:51:45.741 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:45.739+0000 7fb6e9ffb640 1 -- 172.21.15.43:0/2544900798 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]=0 pool 'cephfs_metadata' created v169) v1 ==== 138+0+0 (secure 0 0 0) 0x7fb6f40c4530 con 0x7fb6f41470a0 2023-08-09T14:51:45.741 INFO:teuthology.orchestra.run.smithi043.stderr:pool 'cephfs_metadata' created 2023-08-09T14:51:45.746 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:45.744+0000 7fb6fb638640 1 -- 172.21.15.43:0/2544900798 >> [v2:172.21.15.203:6832/2566816332,v1:172.21.15.203:6833/2566816332] conn(0x7fb6d806a170 msgr2=0x7fb6d806c630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T14:51:45.746 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:45.744+0000 7fb6fb638640 1 --2- 172.21.15.43:0/2544900798 >> [v2:172.21.15.203:6832/2566816332,v1:172.21.15.203:6833/2566816332] conn(0x7fb6d806a170 0x7fb6d806c630 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7fb6f0009b80 tx=0x7fb6f0008040 comp rx=0 tx=0).stop 2023-08-09T14:51:45.747 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:45.744+0000 7fb6fb638640 1 -- 172.21.15.43:0/2544900798 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb6f41470a0 msgr2=0x7fb6f415ccd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T14:51:45.747 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:45.744+0000 7fb6fb638640 1 --2- 172.21.15.43:0/2544900798 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb6f41470a0 0x7fb6f415ccd0 secure :-1 s=READY pgs=1855 cs=0 l=1 rev1=1 crypto rx=0x7fb6e402bc00 tx=0x7fb6e4001120 comp rx=0 tx=0).stop 2023-08-09T14:51:45.747 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:45.744+0000 7fb6fb638640 1 -- 172.21.15.43:0/2544900798 shutdown_connections 2023-08-09T14:51:45.747 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:45.744+0000 7fb6fb638640 1 --2- 172.21.15.43:0/2544900798 >> [v2:172.21.15.203:6832/2566816332,v1:172.21.15.203:6833/2566816332] conn(0x7fb6d806a170 0x7fb6d806c630 unknown :-1 s=CLOSED pgs=56 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T14:51:45.747 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:45.744+0000 7fb6fb638640 1 --2- 172.21.15.43:0/2544900798 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fb6f4147520 0x7fb6f415d210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T14:51:45.747 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:45.744+0000 7fb6fb638640 1 --2- 172.21.15.43:0/2544900798 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb6f41470a0 0x7fb6f415ccd0 unknown :-1 s=CLOSED pgs=1855 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T14:51:45.747 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:45.744+0000 7fb6fb638640 1 --2- 172.21.15.43:0/2544900798 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fb6e00047c0 0x7fb6f41f1230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T14:51:45.748 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:45.744+0000 7fb6fb638640 1 -- 172.21.15.43:0/2544900798 >> 172.21.15.43:0/2544900798 conn(0x7fb6f406b990 msgr2=0x7fb6f4067560 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-09T14:51:45.748 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:45.747+0000 7fb6fb638640 1 -- 172.21.15.43:0/2544900798 shutdown_connections 2023-08-09T14:51:45.749 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:45.747+0000 7fb6fb638640 1 -- 172.21.15.43:0/2544900798 wait complete. 2023-08-09T14:51:45.775 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data 64 --pg_num_min 64 --target_size_ratio 0.9 2023-08-09T14:51:46.012 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:46.010+0000 7f34cf7fd640 1 Processor -- start 2023-08-09T14:51:46.013 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:46.011+0000 7f34cf7fd640 1 -- start start 2023-08-09T14:51:46.014 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:46.013+0000 7f34cf7fd640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f34c80c2600 0x7f34c806cb80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T14:51:46.015 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:46.013+0000 7f34cf7fd640 1 -- --> v1:172.21.15.43:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f34c8071730 con 0x7f34c8069ed0 2023-08-09T14:51:46.015 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:46.013+0000 7f34cf7fd640 1 -- --> v1:172.21.15.203:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f34c80cedd0 con 0x7f34c80ccca0 2023-08-09T14:51:46.015 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:46.013+0000 7f34cf7fd640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f34c80c29e0 con 0x7f34c80c2600 2023-08-09T14:51:46.015 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:46.013+0000 7f34cdd9a640 1 --1- >> v1:172.21.15.43:6789/0 conn(0x7f34c8069ed0 0x7f34c80c9720 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.43:6789/0 says I am v1:172.21.15.43:47248/0 (socket says 172.21.15.43:47248) 2023-08-09T14:51:46.015 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:46.013+0000 7f34cdd9a640 1 -- 172.21.15.43:0/2891609991 learned_addr learned my addr 172.21.15.43:0/2891609991 (peer_addr_for_me v1:172.21.15.43:0/0) 2023-08-09T14:51:46.017 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:46.014+0000 7f34cd599640 1 --2- 172.21.15.43:0/2891609991 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f34c80c2600 0x7f34c806cb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T14:51:46.017 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:46.014+0000 7f34bffff640 1 -- 172.21.15.43:0/2891609991 <== mon.0 v1:172.21.15.43:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2217234487 0 0) 0x7f34c8071730 con 0x7f34c8069ed0 2023-08-09T14:51:46.017 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:46.014+0000 7f34bffff640 1 -- 172.21.15.43:0/2891609991 --> v1:172.21.15.43:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f34b4003520 con 0x7f34c8069ed0 2023-08-09T14:51:46.017 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:46.014+0000 7f34bffff640 1 -- 172.21.15.43:0/2891609991 <== mon.0 v1:172.21.15.43:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2170602220 0 0) 0x7f34b4003520 con 0x7f34c8069ed0 2023-08-09T14:51:46.017 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:46.014+0000 7f34bffff640 1 -- 172.21.15.43:0/2891609991 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f34c80c2600 msgr2=0x7f34c806cb80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-09T14:51:46.018 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:46.014+0000 7f34bffff640 1 --2- 172.21.15.43:0/2891609991 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f34c80c2600 0x7f34c806cb80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T14:51:46.018 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:46.014+0000 7f34bffff640 1 -- 172.21.15.43:0/2891609991 >> v1:172.21.15.203:6789/0 conn(0x7f34c80ccca0 legacy=0x7f34c80697c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T14:51:46.018 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:46.014+0000 7f34bffff640 1 -- 172.21.15.43:0/2891609991 --> v1:172.21.15.43:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f34c80cd130 con 0x7f34c8069ed0 2023-08-09T14:51:46.018 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:46.014+0000 7f34bffff640 1 -- 172.21.15.43:0/2891609991 <== mon.0 v1:172.21.15.43:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1285150740 0 0) 0x7f34c0001670 con 0x7f34c8069ed0 2023-08-09T14:51:46.018 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:46.014+0000 7f34bffff640 1 -- 172.21.15.43:0/2891609991 >> v1:172.21.15.43:6789/0 conn(0x7f34c8069ed0 legacy=0x7f34c80c9720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T14:51:46.018 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:46.014+0000 7f34bffff640 1 --2- 172.21.15.43:0/2891609991 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f34b40039a0 0x7f34b4003de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T14:51:46.019 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:46.014+0000 7f34bffff640 1 --2- 172.21.15.43:0/2891609991 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f34b40047c0 0x7f34b4004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T14:51:46.019 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:46.014+0000 7f34bffff640 1 --2- 172.21.15.43:0/2891609991 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f34c80c2600 0x7f34b4005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T14:51:46.019 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:46.014+0000 7f34bffff640 1 -- 172.21.15.43:0/2891609991 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f34c80c29e0 con 0x7f34b40039a0 2023-08-09T14:51:46.019 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:46.014+0000 7f34bffff640 1 -- 172.21.15.43:0/2891609991 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f34b400ba30 con 0x7f34b40047c0 2023-08-09T14:51:46.019 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:46.014+0000 7f34bffff640 1 -- 172.21.15.43:0/2891609991 --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f34b400bbf0 con 0x7f34c80c2600 2023-08-09T14:51:46.020 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:46.015+0000 7f34cd599640 1 --2- 172.21.15.43:0/2891609991 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f34b40039a0 0x7f34b4003de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T14:51:46.020 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:46.015+0000 7f34ccd98640 1 --2- 172.21.15.43:0/2891609991 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f34b40047c0 0x7f34b4004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T14:51:46.020 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:46.015+0000 7f34cdd9a640 1 --2- 172.21.15.43:0/2891609991 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f34c80c2600 0x7f34b4005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T14:51:46.020 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:46.015+0000 7f34cd599640 1 -- 172.21.15.43:0/2891609991 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f34c80c2600 msgr2=0x7f34b4005180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-09T14:51:46.020 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:46.015+0000 7f34cd599640 1 --2- 172.21.15.43:0/2891609991 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f34c80c2600 0x7f34b4005180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T14:51:46.020 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:46.015+0000 7f34cd599640 1 -- 172.21.15.43:0/2891609991 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f34b40047c0 msgr2=0x7f34b4004c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-09T14:51:46.021 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:46.015+0000 7f34cd599640 1 --2- 172.21.15.43:0/2891609991 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f34b40047c0 0x7f34b4004c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T14:51:46.021 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:46.015+0000 7f34cd599640 1 -- 172.21.15.43:0/2891609991 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f34b4003520 con 0x7f34b40039a0 2023-08-09T14:51:46.021 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:46.015+0000 7f34ccd98640 1 --2- 172.21.15.43:0/2891609991 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f34b40047c0 0x7f34b4004c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-08-09T14:51:46.024 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:46.016+0000 7f34cd599640 1 --2- 172.21.15.43:0/2891609991 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f34b40039a0 0x7f34b4003de0 secure :-1 s=READY pgs=1857 cs=0 l=1 rev1=1 crypto rx=0x7f34c4009f90 tx=0x7f34b40057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cf2620aa63c015c2 server_cookie=0 in_seq=0 out_seq=0 2023-08-09T14:51:46.024 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:46.017+0000 7f34bffff640 1 -- 172.21.15.43:0/2891609991 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f34c402cbd0 con 0x7f34b40039a0 2023-08-09T14:51:46.025 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:46.017+0000 7f34bffff640 1 -- 172.21.15.43:0/2891609991 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f34c400f040 con 0x7f34b40039a0 2023-08-09T14:51:46.025 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:46.017+0000 7f34bffff640 1 -- 172.21.15.43:0/2891609991 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f34c4034440 con 0x7f34b40039a0 2023-08-09T14:51:46.025 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:46.017+0000 7f34cf7fd640 1 -- 172.21.15.43:0/2891609991 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f34b40039a0 msgr2=0x7f34b4003de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T14:51:46.025 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:46.017+0000 7f34cf7fd640 1 --2- 172.21.15.43:0/2891609991 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f34b40039a0 0x7f34b4003de0 secure :-1 s=READY pgs=1857 cs=0 l=1 rev1=1 crypto rx=0x7f34c4009f90 tx=0x7f34b40057f0 comp rx=0 tx=0).stop 2023-08-09T14:51:46.025 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:46.017+0000 7f34cdd9a640 1 -- 172.21.15.43:0/2891609991 reap_dead start 2023-08-09T14:51:46.026 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:46.017+0000 7f34cf7fd640 1 -- 172.21.15.43:0/2891609991 shutdown_connections 2023-08-09T14:51:46.026 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:46.017+0000 7f34cf7fd640 1 -- 172.21.15.43:0/2891609991 >> 172.21.15.43:0/2891609991 conn(0x7f34c80b0cc0 msgr2=0x7f34c80b10d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-09T14:51:46.026 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:46.017+0000 7f34cf7fd640 1 -- 172.21.15.43:0/2891609991 shutdown_connections 2023-08-09T14:51:46.026 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:46.017+0000 7f34cf7fd640 1 -- 172.21.15.43:0/2891609991 wait complete. 2023-08-09T14:51:46.026 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:46.019+0000 7f34cf7fd640 1 Processor -- start 2023-08-09T14:51:46.026 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:46.019+0000 7f34cf7fd640 1 -- start start 2023-08-09T14:51:46.027 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:46.020+0000 7f34cf7fd640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f34b40039a0 0x7f34c815d8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T14:51:46.027 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:46.020+0000 7f34cf7fd640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f34c80c5050 0x7f34c80c75e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T14:51:46.027 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:46.020+0000 7f34cf7fd640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f34c80c54d0 0x7f34c80c7b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T14:51:46.027 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:46.021+0000 7f34cf7fd640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f34c80c71c0 con 0x7f34c80c5050 2023-08-09T14:51:46.027 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:46.021+0000 7f34cf7fd640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f34c81641e0 con 0x7f34c80c54d0 2023-08-09T14:51:46.027 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:46.021+0000 7f34cf7fd640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f34c80c6540 con 0x7f34b40039a0 2023-08-09T14:51:46.027 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:46.021+0000 7f34ccd98640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f34c80c5050 0x7f34c80c75e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T14:51:46.028 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:46.021+0000 7f34ccd98640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f34c80c5050 0x7f34c80c75e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:57814/0 (socket says 172.21.15.43:57814) 2023-08-09T14:51:46.028 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:46.021+0000 7f34ccd98640 1 -- 172.21.15.43:0/812393492 learned_addr learned my addr 172.21.15.43:0/812393492 (peer_addr_for_me v2:172.21.15.43:0/0) 2023-08-09T14:51:46.028 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:46.021+0000 7f34cd599640 1 --2- 172.21.15.43:0/812393492 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f34b40039a0 0x7f34c815d8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T14:51:46.028 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:46.021+0000 7f34cdd9a640 1 --2- 172.21.15.43:0/812393492 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f34c80c54d0 0x7f34c80c7b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T14:51:46.028 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:46.021+0000 7f34ccd98640 1 -- 172.21.15.43:0/812393492 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f34b40039a0 msgr2=0x7f34c815d8f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T14:51:46.028 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:46.021+0000 7f34ccd98640 1 --2- 172.21.15.43:0/812393492 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f34b40039a0 0x7f34c815d8f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T14:51:46.029 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:46.021+0000 7f34ccd98640 1 -- 172.21.15.43:0/812393492 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f34c80c54d0 msgr2=0x7f34c80c7b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T14:51:46.029 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:46.021+0000 7f34ccd98640 1 --2- 172.21.15.43:0/812393492 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f34c80c54d0 0x7f34c80c7b20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T14:51:46.029 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:46.021+0000 7f34ccd98640 1 -- 172.21.15.43:0/812393492 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f34c80c6820 con 0x7f34c80c5050 2023-08-09T14:51:46.029 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:46.022+0000 7f34ccd98640 1 --2- 172.21.15.43:0/812393492 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f34c80c5050 0x7f34c80c75e0 secure :-1 s=READY pgs=1858 cs=0 l=1 rev1=1 crypto rx=0x7f34b800d5f0 tx=0x7f34b800dab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-09T14:51:46.029 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:46.022+0000 7f34bdffb640 1 -- 172.21.15.43:0/812393492 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f34b8017070 con 0x7f34c80c5050 2023-08-09T14:51:46.029 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:46.022+0000 7f34bdffb640 1 -- 172.21.15.43:0/812393492 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f34b8005280 con 0x7f34c80c5050 2023-08-09T14:51:46.030 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:46.022+0000 7f34cf7fd640 1 -- 172.21.15.43:0/812393492 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f34c80c6b00 con 0x7f34c80c5050 2023-08-09T14:51:46.030 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:46.022+0000 7f34bdffb640 1 -- 172.21.15.43:0/812393492 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f34b8006820 con 0x7f34c80c5050 2023-08-09T14:51:46.030 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:46.022+0000 7f34cf7fd640 1 -- 172.21.15.43:0/812393492 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f34c80c61a0 con 0x7f34c80c5050 2023-08-09T14:51:46.031 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:46.023+0000 7f34cf7fd640 1 -- 172.21.15.43:0/812393492 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f34c80739f0 con 0x7f34c80c5050 2023-08-09T14:51:46.032 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:46.025+0000 7f34bdffb640 1 -- 172.21.15.43:0/812393492 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 79) v1 ==== 88443+0+0 (secure 0 0 0) 0x7f34b8006980 con 0x7f34c80c5050 2023-08-09T14:51:46.032 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:46.026+0000 7f34bdffb640 1 --2- 172.21.15.43:0/812393492 >> [v2:172.21.15.203:6832/2566816332,v1:172.21.15.203:6833/2566816332] conn(0x7f34ac06a170 0x7f34ac06c630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T14:51:46.032 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:46.026+0000 7f34bdffb640 1 -- 172.21.15.43:0/812393492 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(169..169 src has 1..169) v4 ==== 5944+0+0 (secure 0 0 0) 0x7f34b8088c40 con 0x7f34c80c5050 2023-08-09T14:51:46.033 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:46.027+0000 7f34cd599640 1 --2- 172.21.15.43:0/812393492 >> [v2:172.21.15.203:6832/2566816332,v1:172.21.15.203:6833/2566816332] conn(0x7f34ac06a170 0x7f34ac06c630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T14:51:46.033 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:46.028+0000 7f34cd599640 1 --2- 172.21.15.43:0/812393492 >> [v2:172.21.15.203:6832/2566816332,v1:172.21.15.203:6833/2566816332] conn(0x7f34ac06a170 0x7f34ac06c630 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f34c4036000 tx=0x7f34c402e000 comp rx=0 tx=0).ready entity=mgr.7765 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-09T14:51:46.033 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:46.030+0000 7f34bdffb640 1 -- 172.21.15.43:0/812393492 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178066 (secure 0 0 0) 0x7f34b80540d0 con 0x7f34c80c5050 2023-08-09T14:51:46.225 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:46.224+0000 7f34cf7fd640 1 -- 172.21.15.43:0/812393492 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9} v 0) v1 -- 0x7f34c80aeb40 con 0x7f34c80c5050 2023-08-09T14:51:46.763 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:46.758+0000 7f34bdffb640 1 -- 172.21.15.43:0/812393492 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]=0 pool 'cephfs_data' created v170) v1 ==== 170+0+0 (secure 0 0 0) 0x7f34b8001280 con 0x7f34c80c5050 2023-08-09T14:51:46.766 INFO:teuthology.orchestra.run.smithi043.stderr:pool 'cephfs_data' created 2023-08-09T14:51:46.768 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:46.766+0000 7f34cf7fd640 1 -- 172.21.15.43:0/812393492 >> [v2:172.21.15.203:6832/2566816332,v1:172.21.15.203:6833/2566816332] conn(0x7f34ac06a170 msgr2=0x7f34ac06c630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T14:51:46.768 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:46.766+0000 7f34cf7fd640 1 --2- 172.21.15.43:0/812393492 >> [v2:172.21.15.203:6832/2566816332,v1:172.21.15.203:6833/2566816332] conn(0x7f34ac06a170 0x7f34ac06c630 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f34c4036000 tx=0x7f34c402e000 comp rx=0 tx=0).stop 2023-08-09T14:51:46.768 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:46.766+0000 7f34cf7fd640 1 -- 172.21.15.43:0/812393492 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f34c80c5050 msgr2=0x7f34c80c75e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T14:51:46.769 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:46.766+0000 7f34cf7fd640 1 --2- 172.21.15.43:0/812393492 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f34c80c5050 0x7f34c80c75e0 secure :-1 s=READY pgs=1858 cs=0 l=1 rev1=1 crypto rx=0x7f34b800d5f0 tx=0x7f34b800dab0 comp rx=0 tx=0).stop 2023-08-09T14:51:46.771 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:46.768+0000 7f34cf7fd640 1 -- 172.21.15.43:0/812393492 shutdown_connections 2023-08-09T14:51:46.771 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:46.768+0000 7f34cf7fd640 1 --2- 172.21.15.43:0/812393492 >> [v2:172.21.15.203:6832/2566816332,v1:172.21.15.203:6833/2566816332] conn(0x7f34ac06a170 0x7f34ac06c630 unknown :-1 s=CLOSED pgs=57 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T14:51:46.771 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:46.768+0000 7f34cf7fd640 1 --2- 172.21.15.43:0/812393492 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f34c80c54d0 0x7f34c80c7b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T14:51:46.771 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:46.768+0000 7f34cf7fd640 1 --2- 172.21.15.43:0/812393492 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f34c80c5050 0x7f34c80c75e0 unknown :-1 s=CLOSED pgs=1858 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T14:51:46.772 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:46.768+0000 7f34cf7fd640 1 --2- 172.21.15.43:0/812393492 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f34b40039a0 0x7f34c815d8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T14:51:46.772 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:46.768+0000 7f34cf7fd640 1 -- 172.21.15.43:0/812393492 >> 172.21.15.43:0/812393492 conn(0x7f34c80b0cc0 msgr2=0x7f34c81606a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-09T14:51:46.772 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:46.769+0000 7f34cf7fd640 1 -- 172.21.15.43:0/812393492 shutdown_connections 2023-08-09T14:51:46.772 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:46.769+0000 7f34cf7fd640 1 -- 172.21.15.43:0/812393492 wait complete. 2023-08-09T14:51:46.802 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs new cephfs cephfs_metadata cephfs_data 2023-08-09T14:51:47.001 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:47.000+0000 7f6075825640 1 Processor -- start 2023-08-09T14:51:47.001 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:47.000+0000 7f6075825640 1 -- start start 2023-08-09T14:51:47.003 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:47.001+0000 7f6075825640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f607015c2f0 0x7f607015d410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T14:51:47.003 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:47.001+0000 7f6075825640 1 -- --> v1:172.21.15.43:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f60700cdc20 con 0x7f60700cfa70 2023-08-09T14:51:47.003 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:47.001+0000 7f6075825640 1 -- --> v1:172.21.15.203:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f607006c570 con 0x7f60700b21e0 2023-08-09T14:51:47.003 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:47.001+0000 7f6075825640 1 -- --> v2:172.21.15.43:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f607015d950 con 0x7f607015c2f0 2023-08-09T14:51:47.004 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:47.002+0000 7f6067fff640 1 --1- >> v1:172.21.15.43:6789/0 conn(0x7f60700cfa70 0x7f60700b1ad0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.43:6789/0 says I am v1:172.21.15.43:47252/0 (socket says 172.21.15.43:47252) 2023-08-09T14:51:47.004 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:47.002+0000 7f6067fff640 1 -- 172.21.15.43:0/3097721425 learned_addr learned my addr 172.21.15.43:0/3097721425 (peer_addr_for_me v1:172.21.15.43:0/0) 2023-08-09T14:51:47.004 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:47.002+0000 7f6074823640 1 --2- 172.21.15.43:0/3097721425 >> v2:172.21.15.43:3300/0 conn(0x7f607015c2f0 0x7f607015d410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T14:51:47.004 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:47.002+0000 7f6074823640 1 -- 172.21.15.43:0/3097721425 >> v1:172.21.15.203:6789/0 conn(0x7f60700b21e0 legacy=0x7f60700cb7b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-09T14:51:47.005 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:47.002+0000 7f6074823640 1 -- 172.21.15.43:0/3097721425 >> v1:172.21.15.43:6789/0 conn(0x7f60700cfa70 legacy=0x7f60700b1ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T14:51:47.005 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:47.002+0000 7f6074823640 1 -- 172.21.15.43:0/3097721425 --> v2:172.21.15.43:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f60700cde70 con 0x7f607015c2f0 2023-08-09T14:51:47.005 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:47.002+0000 7f60677fe640 1 -- 172.21.15.43:0/3097721425 <== mon.0 v1:172.21.15.43:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3175932646 0 0) 0x7f60700cdc20 con 0x7f60700cfa70 2023-08-09T14:51:47.006 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:47.003+0000 7f6074823640 1 --2- 172.21.15.43:0/3097721425 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f607015c2f0 0x7f607015d410 secure :-1 s=READY pgs=1860 cs=0 l=1 rev1=1 crypto rx=0x7f606c009d90 tx=0x7f606c02beb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a287685145884e51 server_cookie=0 in_seq=0 out_seq=0 2023-08-09T14:51:47.006 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:47.004+0000 7f60677fe640 1 -- 172.21.15.43:0/3097721425 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f606c02cc60 con 0x7f607015c2f0 2023-08-09T14:51:47.006 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:47.004+0000 7f60677fe640 1 -- 172.21.15.43:0/3097721425 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f606c02cdc0 con 0x7f607015c2f0 2023-08-09T14:51:47.006 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:47.004+0000 7f60677fe640 1 -- 172.21.15.43:0/3097721425 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f606c034660 con 0x7f607015c2f0 2023-08-09T14:51:47.006 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:47.004+0000 7f6075825640 1 -- 172.21.15.43:0/3097721425 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f607015c2f0 msgr2=0x7f607015d410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T14:51:47.007 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:47.004+0000 7f6075825640 1 --2- 172.21.15.43:0/3097721425 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f607015c2f0 0x7f607015d410 secure :-1 s=READY pgs=1860 cs=0 l=1 rev1=1 crypto rx=0x7f606c009d90 tx=0x7f606c02beb0 comp rx=0 tx=0).stop 2023-08-09T14:51:47.007 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:47.004+0000 7f6075825640 1 -- 172.21.15.43:0/3097721425 shutdown_connections 2023-08-09T14:51:47.007 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:47.004+0000 7f6075825640 1 --2- 172.21.15.43:0/3097721425 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f607015c2f0 0x7f607015d410 unknown :-1 s=CLOSED pgs=1860 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T14:51:47.007 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:47.004+0000 7f6075825640 1 -- 172.21.15.43:0/3097721425 >> 172.21.15.43:0/3097721425 conn(0x7f60700ccad0 msgr2=0x7f60700cceb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-09T14:51:47.007 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:47.004+0000 7f6075825640 1 -- 172.21.15.43:0/3097721425 shutdown_connections 2023-08-09T14:51:47.007 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:47.005+0000 7f6075825640 1 -- 172.21.15.43:0/3097721425 wait complete. 2023-08-09T14:51:47.008 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:47.006+0000 7f6075825640 1 Processor -- start 2023-08-09T14:51:47.008 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:47.006+0000 7f6075825640 1 -- start start 2023-08-09T14:51:47.009 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:47.007+0000 7f6075825640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f60700b21e0 0x7f6070146240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T14:51:47.009 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:47.007+0000 7f6075825640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f60700cfa70 0x7f607015a1c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T14:51:47.009 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:47.007+0000 7f6075825640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f607015c2f0 0x7f607015a700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T14:51:47.009 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:47.007+0000 7f6075825640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6070157a90 con 0x7f60700cfa70 2023-08-09T14:51:47.009 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:47.007+0000 7f6075825640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6070156230 con 0x7f60700b21e0 2023-08-09T14:51:47.010 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:47.007+0000 7f6075825640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6070157d30 con 0x7f607015c2f0 2023-08-09T14:51:47.011 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:47.008+0000 7f6067fff640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f60700cfa70 0x7f607015a1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T14:51:47.011 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:47.008+0000 7f6067fff640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f60700cfa70 0x7f607015a1c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:57838/0 (socket says 172.21.15.43:57838) 2023-08-09T14:51:47.011 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:47.008+0000 7f6067fff640 1 -- 172.21.15.43:0/1828645115 learned_addr learned my addr 172.21.15.43:0/1828645115 (peer_addr_for_me v2:172.21.15.43:0/0) 2023-08-09T14:51:47.011 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:47.008+0000 7f6074823640 1 --2- 172.21.15.43:0/1828645115 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f60700b21e0 0x7f6070146240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T14:51:47.011 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:47.008+0000 7f6075024640 1 --2- 172.21.15.43:0/1828645115 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f607015c2f0 0x7f607015a700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T14:51:47.011 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:47.008+0000 7f6067fff640 1 -- 172.21.15.43:0/1828645115 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f607015c2f0 msgr2=0x7f607015a700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T14:51:47.012 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:47.008+0000 7f6067fff640 1 --2- 172.21.15.43:0/1828645115 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f607015c2f0 0x7f607015a700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T14:51:47.012 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:47.008+0000 7f6067fff640 1 -- 172.21.15.43:0/1828645115 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f60700b21e0 msgr2=0x7f6070146240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T14:51:47.012 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:47.008+0000 7f6067fff640 1 --2- 172.21.15.43:0/1828645115 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f60700b21e0 0x7f6070146240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T14:51:47.012 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:47.008+0000 7f6067fff640 1 -- 172.21.15.43:0/1828645115 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f606c009a70 con 0x7f60700cfa70 2023-08-09T14:51:47.012 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:47.009+0000 7f6067fff640 1 --2- 172.21.15.43:0/1828645115 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f60700cfa70 0x7f607015a1c0 secure :-1 s=READY pgs=1861 cs=0 l=1 rev1=1 crypto rx=0x7f606000c4a0 tx=0x7f606000c960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-09T14:51:47.012 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:47.009+0000 7f60657fa640 1 -- 172.21.15.43:0/1828645115 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f60600074e0 con 0x7f60700cfa70 2023-08-09T14:51:47.012 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:47.009+0000 7f60657fa640 1 -- 172.21.15.43:0/1828645115 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6060007640 con 0x7f60700cfa70 2023-08-09T14:51:47.013 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:47.009+0000 7f60657fa640 1 -- 172.21.15.43:0/1828645115 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6060004a00 con 0x7f60700cfa70 2023-08-09T14:51:47.013 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:47.009+0000 7f6075825640 1 -- 172.21.15.43:0/1828645115 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6070158010 con 0x7f60700cfa70 2023-08-09T14:51:47.013 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:47.009+0000 7f6075825640 1 -- 172.21.15.43:0/1828645115 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6070146780 con 0x7f60700cfa70 2023-08-09T14:51:47.013 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:47.011+0000 7f6075825640 1 -- 172.21.15.43:0/1828645115 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6070158230 con 0x7f60700cfa70 2023-08-09T14:51:47.014 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:47.012+0000 7f60657fa640 1 -- 172.21.15.43:0/1828645115 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 79) v1 ==== 88443+0+0 (secure 0 0 0) 0x7f606001b360 con 0x7f60700cfa70 2023-08-09T14:51:47.015 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:47.014+0000 7f60657fa640 1 --2- 172.21.15.43:0/1828645115 >> [v2:172.21.15.203:6832/2566816332,v1:172.21.15.203:6833/2566816332] conn(0x7f605c06a170 0x7f605c06c630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T14:51:47.015 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:47.014+0000 7f60657fa640 1 -- 172.21.15.43:0/1828645115 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(170..170 src has 1..170) v4 ==== 6326+0+0 (secure 0 0 0) 0x7f6060087d00 con 0x7f60700cfa70 2023-08-09T14:51:47.016 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:47.015+0000 7f6074823640 1 --2- 172.21.15.43:0/1828645115 >> [v2:172.21.15.203:6832/2566816332,v1:172.21.15.203:6833/2566816332] conn(0x7f605c06a170 0x7f605c06c630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T14:51:47.017 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:47.016+0000 7f6074823640 1 --2- 172.21.15.43:0/1828645115 >> [v2:172.21.15.203:6832/2566816332,v1:172.21.15.203:6833/2566816332] conn(0x7f605c06a170 0x7f605c06c630 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f606c036000 tx=0x7f606c030000 comp rx=0 tx=0).ready entity=mgr.7765 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-09T14:51:47.019 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:47.018+0000 7f60657fa640 1 -- 172.21.15.43:0/1828645115 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178066 (secure 0 0 0) 0x7f6060053240 con 0x7f60700cfa70 2023-08-09T14:51:47.252 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:47.251+0000 7f6075825640 1 -- 172.21.15.43:0/1828645115 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"} v 0) v1 -- 0x7f607004e390 con 0x7f60700cfa70 2023-08-09T14:51:47.800 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:47.799+0000 7f60657fa640 1 -- 172.21.15.43:0/1828645115 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]=0 new fs with metadata pool 20 and data pool 21 v75) v1 ==== 174+0+0 (secure 0 0 0) 0x7f6060056860 con 0x7f60700cfa70 2023-08-09T14:51:47.802 INFO:teuthology.orchestra.run.smithi043.stderr:new fs with metadata pool 20 and data pool 21 2023-08-09T14:51:47.810 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:47.809+0000 7f6075825640 1 -- 172.21.15.43:0/1828645115 >> [v2:172.21.15.203:6832/2566816332,v1:172.21.15.203:6833/2566816332] conn(0x7f605c06a170 msgr2=0x7f605c06c630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T14:51:47.811 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:47.809+0000 7f6075825640 1 --2- 172.21.15.43:0/1828645115 >> [v2:172.21.15.203:6832/2566816332,v1:172.21.15.203:6833/2566816332] conn(0x7f605c06a170 0x7f605c06c630 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f606c036000 tx=0x7f606c030000 comp rx=0 tx=0).stop 2023-08-09T14:51:47.811 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:47.809+0000 7f6075825640 1 -- 172.21.15.43:0/1828645115 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f60700cfa70 msgr2=0x7f607015a1c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T14:51:47.811 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:47.809+0000 7f6075825640 1 --2- 172.21.15.43:0/1828645115 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f60700cfa70 0x7f607015a1c0 secure :-1 s=READY pgs=1861 cs=0 l=1 rev1=1 crypto rx=0x7f606000c4a0 tx=0x7f606000c960 comp rx=0 tx=0).stop 2023-08-09T14:51:47.812 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:47.810+0000 7f6075825640 1 -- 172.21.15.43:0/1828645115 shutdown_connections 2023-08-09T14:51:47.812 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:47.810+0000 7f6075825640 1 --2- 172.21.15.43:0/1828645115 >> [v2:172.21.15.203:6832/2566816332,v1:172.21.15.203:6833/2566816332] conn(0x7f605c06a170 0x7f605c06c630 unknown :-1 s=CLOSED pgs=58 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T14:51:47.812 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:47.810+0000 7f6075825640 1 --2- 172.21.15.43:0/1828645115 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f607015c2f0 0x7f607015a700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T14:51:47.812 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:47.810+0000 7f6075825640 1 --2- 172.21.15.43:0/1828645115 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f60700cfa70 0x7f607015a1c0 unknown :-1 s=CLOSED pgs=1861 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T14:51:47.813 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:47.810+0000 7f6075825640 1 --2- 172.21.15.43:0/1828645115 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f60700b21e0 0x7f6070146240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T14:51:47.813 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:47.810+0000 7f6075825640 1 -- 172.21.15.43:0/1828645115 >> 172.21.15.43:0/1828645115 conn(0x7f60700ccad0 msgr2=0x7f60700d0030 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-09T14:51:47.813 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:47.811+0000 7f6075825640 1 -- 172.21.15.43:0/1828645115 shutdown_connections 2023-08-09T14:51:47.813 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:47.811+0000 7f6075825640 1 -- 172.21.15.43:0/1828645115 wait complete. 2023-08-09T14:51:47.833 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-08-09T14:51:48.021 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:48.020+0000 7fde0f712640 1 Processor -- start 2023-08-09T14:51:48.022 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:48.020+0000 7fde0f712640 1 -- start start 2023-08-09T14:51:48.023 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:48.021+0000 7fde0f712640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fde080ce720 0x7fde080ceb00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T14:51:48.023 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:48.022+0000 7fde0f712640 1 -- --> v1:172.21.15.43:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fde080cd8a0 con 0x7fde0815ecc0 2023-08-09T14:51:48.023 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:48.022+0000 7fde0f712640 1 -- --> v1:172.21.15.203:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fde080cf350 con 0x7fde080ce080 2023-08-09T14:51:48.023 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:48.022+0000 7fde0f712640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fde080cf040 con 0x7fde080ce720 2023-08-09T14:51:48.025 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:48.022+0000 7fde0dcaf640 1 --1- >> v1:172.21.15.43:6789/0 conn(0x7fde0815ecc0 0x7fde0806d2a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.43:6789/0 says I am v1:172.21.15.43:47266/0 (socket says 172.21.15.43:47266) 2023-08-09T14:51:48.025 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:48.022+0000 7fde0dcaf640 1 -- 172.21.15.43:0/1503777030 learned_addr learned my addr 172.21.15.43:0/1503777030 (peer_addr_for_me v1:172.21.15.43:0/0) 2023-08-09T14:51:48.025 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:48.022+0000 7fde0d4ae640 1 --2- 172.21.15.43:0/1503777030 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fde080ce720 0x7fde080ceb00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T14:51:48.025 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:48.022+0000 7fddfffff640 1 -- 172.21.15.43:0/1503777030 <== mon.0 v1:172.21.15.43:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 567274115 0 0) 0x7fde080cd8a0 con 0x7fde0815ecc0 2023-08-09T14:51:48.025 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:48.023+0000 7fddfffff640 1 -- 172.21.15.43:0/1503777030 --> v1:172.21.15.43:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fddf4003520 con 0x7fde0815ecc0 2023-08-09T14:51:48.025 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:48.023+0000 7fddfffff640 1 -- 172.21.15.43:0/1503777030 <== mon.0 v1:172.21.15.43:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3495737665 0 0) 0x7fddf4003520 con 0x7fde0815ecc0 2023-08-09T14:51:48.026 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:48.023+0000 7fddfffff640 1 -- 172.21.15.43:0/1503777030 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fde080ce720 msgr2=0x7fde080ceb00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-09T14:51:48.026 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:48.023+0000 7fddfffff640 1 --2- 172.21.15.43:0/1503777030 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fde080ce720 0x7fde080ceb00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T14:51:48.026 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:48.023+0000 7fddfffff640 1 -- 172.21.15.43:0/1503777030 >> v1:172.21.15.203:6789/0 conn(0x7fde080ce080 legacy=0x7fde0815e5b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T14:51:48.026 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:48.023+0000 7fddfffff640 1 -- 172.21.15.43:0/1503777030 --> v1:172.21.15.43:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fde08071a70 con 0x7fde0815ecc0 2023-08-09T14:51:48.026 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:48.023+0000 7fddfffff640 1 -- 172.21.15.43:0/1503777030 <== mon.0 v1:172.21.15.43:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1285150740 0 0) 0x7fde00001670 con 0x7fde0815ecc0 2023-08-09T14:51:48.026 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:48.023+0000 7fddfffff640 1 -- 172.21.15.43:0/1503777030 >> v1:172.21.15.43:6789/0 conn(0x7fde0815ecc0 legacy=0x7fde0806d2a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T14:51:48.027 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:48.023+0000 7fddfffff640 1 --2- 172.21.15.43:0/1503777030 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fddf4003a10 0x7fddf4003e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T14:51:48.027 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:48.023+0000 7fddfffff640 1 --2- 172.21.15.43:0/1503777030 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fddf40047c0 0x7fddf4004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T14:51:48.027 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:48.023+0000 7fddfffff640 1 --2- 172.21.15.43:0/1503777030 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fde080ce720 0x7fddf400b980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T14:51:48.027 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:48.023+0000 7fddfffff640 1 -- 172.21.15.43:0/1503777030 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fde080cf040 con 0x7fddf4003a10 2023-08-09T14:51:48.028 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:48.023+0000 7fddfffff640 1 -- 172.21.15.43:0/1503777030 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fddf40054c0 con 0x7fddf40047c0 2023-08-09T14:51:48.028 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:48.023+0000 7fddfffff640 1 -- 172.21.15.43:0/1503777030 --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fddf4005680 con 0x7fde080ce720 2023-08-09T14:51:48.028 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:48.023+0000 7fde0d4ae640 1 --2- 172.21.15.43:0/1503777030 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fddf4003a10 0x7fddf4003e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T14:51:48.033 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:48.024+0000 7fde0d4ae640 1 -- 172.21.15.43:0/1503777030 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fde080ce720 msgr2=0x7fddf400b980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-09T14:51:48.033 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:48.024+0000 7fde0d4ae640 1 --2- 172.21.15.43:0/1503777030 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fde080ce720 0x7fddf400b980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T14:51:48.033 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:48.024+0000 7fde0d4ae640 1 -- 172.21.15.43:0/1503777030 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fddf40047c0 msgr2=0x7fddf4004c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-09T14:51:48.033 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:48.024+0000 7fde0d4ae640 1 --2- 172.21.15.43:0/1503777030 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fddf40047c0 0x7fddf4004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T14:51:48.034 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:48.024+0000 7fde0d4ae640 1 -- 172.21.15.43:0/1503777030 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fddf4003520 con 0x7fddf4003a10 2023-08-09T14:51:48.034 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:48.025+0000 7fde0d4ae640 1 --2- 172.21.15.43:0/1503777030 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fddf4003a10 0x7fddf4003e30 secure :-1 s=READY pgs=1863 cs=0 l=1 rev1=1 crypto rx=0x7fde0400e170 tx=0x7fddf40057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d0150681686b8bdd server_cookie=0 in_seq=0 out_seq=0 2023-08-09T14:51:48.034 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:48.025+0000 7fddfffff640 1 -- 172.21.15.43:0/1503777030 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fde0402cc70 con 0x7fddf4003a10 2023-08-09T14:51:48.035 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:48.025+0000 7fddfffff640 1 -- 172.21.15.43:0/1503777030 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fde0400f040 con 0x7fddf4003a10 2023-08-09T14:51:48.035 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:48.025+0000 7fddfffff640 1 -- 172.21.15.43:0/1503777030 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fde04034440 con 0x7fddf4003a10 2023-08-09T14:51:48.035 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:48.026+0000 7fde0f712640 1 -- 172.21.15.43:0/1503777030 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fddf4003a10 msgr2=0x7fddf4003e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T14:51:48.035 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:48.026+0000 7fde0f712640 1 --2- 172.21.15.43:0/1503777030 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fddf4003a10 0x7fddf4003e30 secure :-1 s=READY pgs=1863 cs=0 l=1 rev1=1 crypto rx=0x7fde0400e170 tx=0x7fddf40057f0 comp rx=0 tx=0).stop 2023-08-09T14:51:48.036 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:48.026+0000 7fde0dcaf640 1 -- 172.21.15.43:0/1503777030 reap_dead start 2023-08-09T14:51:48.036 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:48.026+0000 7fde0f712640 1 -- 172.21.15.43:0/1503777030 shutdown_connections 2023-08-09T14:51:48.036 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:48.026+0000 7fde0f712640 1 -- 172.21.15.43:0/1503777030 >> 172.21.15.43:0/1503777030 conn(0x7fde080ccd80 msgr2=0x7fde080cd190 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-09T14:51:48.036 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:48.026+0000 7fde0f712640 1 -- 172.21.15.43:0/1503777030 shutdown_connections 2023-08-09T14:51:48.037 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:48.026+0000 7fde0f712640 1 -- 172.21.15.43:0/1503777030 wait complete. 2023-08-09T14:51:48.037 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:48.028+0000 7fde0f712640 1 Processor -- start 2023-08-09T14:51:48.037 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:48.028+0000 7fde0f712640 1 -- start start 2023-08-09T14:51:48.038 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:48.029+0000 7fde0f712640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fddf4003a10 0x7fde0806b2e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T14:51:48.038 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:48.029+0000 7fde0f712640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fde0806c4e0 0x7fde080c5af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T14:51:48.038 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:48.029+0000 7fde0f712640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fde080c6030 0x7fde080afff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T14:51:48.038 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:48.029+0000 7fde0f712640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fde080c64c0 con 0x7fde0806c4e0 2023-08-09T14:51:48.039 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:48.029+0000 7fde0f712640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fde0806b820 con 0x7fde080c6030 2023-08-09T14:51:48.039 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:48.029+0000 7fde0f712640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fde08158f20 con 0x7fddf4003a10 2023-08-09T14:51:48.039 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:48.029+0000 7fde0ccad640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fde0806c4e0 0x7fde080c5af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T14:51:48.039 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:48.029+0000 7fde0ccad640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fde0806c4e0 0x7fde080c5af0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:57856/0 (socket says 172.21.15.43:57856) 2023-08-09T14:51:48.040 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:48.029+0000 7fde0ccad640 1 -- 172.21.15.43:0/1994068093 learned_addr learned my addr 172.21.15.43:0/1994068093 (peer_addr_for_me v2:172.21.15.43:0/0) 2023-08-09T14:51:48.040 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:48.030+0000 7fde0d4ae640 1 --2- 172.21.15.43:0/1994068093 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fddf4003a10 0x7fde0806b2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T14:51:48.040 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:48.030+0000 7fde0dcaf640 1 --2- 172.21.15.43:0/1994068093 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fde080c6030 0x7fde080afff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T14:51:48.041 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:48.030+0000 7fde0ccad640 1 -- 172.21.15.43:0/1994068093 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fddf4003a10 msgr2=0x7fde0806b2e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T14:51:48.041 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:48.030+0000 7fde0ccad640 1 --2- 172.21.15.43:0/1994068093 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fddf4003a10 0x7fde0806b2e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T14:51:48.041 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:48.030+0000 7fde0ccad640 1 -- 172.21.15.43:0/1994068093 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fde080c6030 msgr2=0x7fde080afff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T14:51:48.041 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:48.030+0000 7fde0ccad640 1 --2- 172.21.15.43:0/1994068093 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fde080c6030 0x7fde080afff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T14:51:48.041 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:48.030+0000 7fde0ccad640 1 -- 172.21.15.43:0/1994068093 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fde08159200 con 0x7fde0806c4e0 2023-08-09T14:51:48.041 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:48.030+0000 7fde0d4ae640 1 --2- 172.21.15.43:0/1994068093 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fddf4003a10 0x7fde0806b2e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-08-09T14:51:48.041 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:48.030+0000 7fde0ccad640 1 --2- 172.21.15.43:0/1994068093 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fde0806c4e0 0x7fde080c5af0 secure :-1 s=READY pgs=1864 cs=0 l=1 rev1=1 crypto rx=0x7fddf800c450 tx=0x7fddf800c910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-09T14:51:48.042 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:48.030+0000 7fddfdffb640 1 -- 172.21.15.43:0/1994068093 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fddf80052d0 con 0x7fde0806c4e0 2023-08-09T14:51:48.042 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:48.030+0000 7fddfdffb640 1 -- 172.21.15.43:0/1994068093 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fddf8005450 con 0x7fde0806c4e0 2023-08-09T14:51:48.042 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:48.030+0000 7fddfdffb640 1 -- 172.21.15.43:0/1994068093 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fddf8012870 con 0x7fde0806c4e0 2023-08-09T14:51:48.042 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:48.030+0000 7fde0f712640 1 -- 172.21.15.43:0/1994068093 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fde081594e0 con 0x7fde0806c4e0 2023-08-09T14:51:48.042 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:48.031+0000 7fde0f712640 1 -- 172.21.15.43:0/1994068093 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fde08159aa0 con 0x7fde0806c4e0 2023-08-09T14:51:48.043 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:48.032+0000 7fde0f712640 1 -- 172.21.15.43:0/1994068093 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fde080739f0 con 0x7fde0806c4e0 2023-08-09T14:51:48.043 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:48.033+0000 7fddfdffb640 1 -- 172.21.15.43:0/1994068093 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 79) v1 ==== 88443+0+0 (secure 0 0 0) 0x7fddf80129d0 con 0x7fde0806c4e0 2023-08-09T14:51:48.043 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:48.033+0000 7fddfdffb640 1 --2- 172.21.15.43:0/1994068093 >> [v2:172.21.15.203:6832/2566816332,v1:172.21.15.203:6833/2566816332] conn(0x7fddec06a170 0x7fddec06c630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T14:51:48.044 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:48.034+0000 7fddfdffb640 1 -- 172.21.15.43:0/1994068093 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(171..171 src has 1..171) v4 ==== 6426+0+0 (secure 0 0 0) 0x7fddf8088470 con 0x7fde0806c4e0 2023-08-09T14:51:48.044 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:48.034+0000 7fde0d4ae640 1 --2- 172.21.15.43:0/1994068093 >> [v2:172.21.15.203:6832/2566816332,v1:172.21.15.203:6833/2566816332] conn(0x7fddec06a170 0x7fddec06c630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T14:51:48.044 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:48.035+0000 7fde0d4ae640 1 --2- 172.21.15.43:0/1994068093 >> [v2:172.21.15.203:6832/2566816332,v1:172.21.15.203:6833/2566816332] conn(0x7fddec06a170 0x7fddec06c630 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7fde04036000 tx=0x7fde0402e000 comp rx=0 tx=0).ready entity=mgr.7765 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-09T14:51:48.044 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:48.038+0000 7fddfdffb640 1 -- 172.21.15.43:0/1994068093 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178066 (secure 0 0 0) 0x7fddf8053840 con 0x7fde0806c4e0 2023-08-09T14:51:48.236 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:48.235+0000 7fde0f712640 1 -- 172.21.15.43:0/1994068093 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fde0806c8c0 con 0x7fde0806c4e0 2023-08-09T14:51:48.238 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:48.237+0000 7fddfdffb640 1 -- 172.21.15.43:0/1994068093 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v171) v1 ==== 74+0+18579 (secure 0 0 0) 0x7fddf8056e60 con 0x7fde0806c4e0 2023-08-09T14:51:48.240 INFO:teuthology.orchestra.run.smithi043.stdout: 2023-08-09T14:51:48.240 INFO:teuthology.orchestra.run.smithi043.stdout:{"epoch":171,"fsid":"daac87ee-1337-4250-a955-aea4e408a39b","created":"2023-08-09T13:33:41.701198+0000","modified":"2023-08-09T14:51:47.253220+0000","last_up_change":"2023-08-09T13:33:53.783453+0000","last_in_change":"2023-08-09T13:33:45.148904+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":21,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":"rbd","create_time":"2023-08-09T13:33:54.365137+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":".mgr","create_time":"2023-08-09T13:33:54.831803+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":20,"pool_name":"cephfs_metadata","create_time":"2023-08-09T14:51:45.692613+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"171","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":21,"pool_name":"cephfs_data","create_time":"2023-08-09T14:51:46.286715+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"171","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"cbd49ecd-e896-4fbd-864e-5c37a829b165","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":170,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6800","nonce":502667335},{"type":"v1","addr":"172.21.15.43:6801","nonce":502667335}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6802","nonce":502667335},{"type":"v1","addr":"172.21.15.43:6803","nonce":502667335}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6806","nonce":502667335},{"type":"v1","addr":"172.21.15.43:6807","nonce":502667335}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6804","nonce":502667335},{"type":"v1","addr":"172.21.15.43:6805","nonce":502667335}]},"public_addr":"172.21.15.43:6801/502667335","cluster_addr":"172.21.15.43:6803/502667335","heartbeat_back_addr":"172.21.15.43:6807/502667335","heartbeat_front_addr":"172.21.15.43:6805/502667335","state":["exists","up"]},{"osd":1,"uuid":"75c7e0fe-1ec7-4fe8-a2f2-3661a65b8500","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":170,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6808","nonce":3289134610},{"type":"v1","addr":"172.21.15.43:6809","nonce":3289134610}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6810","nonce":3289134610},{"type":"v1","addr":"172.21.15.43:6812","nonce":3289134610}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6821","nonce":3289134610},{"type":"v1","addr":"172.21.15.43:6824","nonce":3289134610}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6815","nonce":3289134610},{"type":"v1","addr":"172.21.15.43:6818","nonce":3289134610}]},"public_addr":"172.21.15.43:6809/3289134610","cluster_addr":"172.21.15.43:6812/3289134610","heartbeat_back_addr":"172.21.15.43:6824/3289134610","heartbeat_front_addr":"172.21.15.43:6818/3289134610","state":["exists","up"]},{"osd":2,"uuid":"bb5c41e9-7f5f-4125-9b01-3bb8bfc2ea83","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":170,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6811","nonce":4079709736},{"type":"v1","addr":"172.21.15.43:6814","nonce":4079709736}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6817","nonce":4079709736},{"type":"v1","addr":"172.21.15.43:6820","nonce":4079709736}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6829","nonce":4079709736},{"type":"v1","addr":"172.21.15.43:6831","nonce":4079709736}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6823","nonce":4079709736},{"type":"v1","addr":"172.21.15.43:6826","nonce":4079709736}]},"public_addr":"172.21.15.43:6814/4079709736","cluster_addr":"172.21.15.43:6820/4079709736","heartbeat_back_addr":"172.21.15.43:6831/4079709736","heartbeat_front_addr":"172.21.15.43:6826/4079709736","state":["exists","up"]},{"osd":3,"uuid":"bfe4c8b4-694a-465c-87e6-2b2aab852613","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":170,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6813","nonce":1709538759},{"type":"v1","addr":"172.21.15.43:6816","nonce":1709538759}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6819","nonce":1709538759},{"type":"v1","addr":"172.21.15.43:6822","nonce":1709538759}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6828","nonce":1709538759},{"type":"v1","addr":"172.21.15.43:6830","nonce":1709538759}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6825","nonce":1709538759},{"type":"v1","addr":"172.21.15.43:6827","nonce":1709538759}]},"public_addr":"172.21.15.43:6816/1709538759","cluster_addr":"172.21.15.43:6822/1709538759","heartbeat_back_addr":"172.21.15.43:6830/1709538759","heartbeat_front_addr":"172.21.15.43:6827/1709538759","state":["exists","up"]},{"osd":4,"uuid":"fa2e8a77-aba1-4756-853e-0d174a5bdeba","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":170,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6800","nonce":4196866481},{"type":"v1","addr":"172.21.15.203:6801","nonce":4196866481}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6802","nonce":4196866481},{"type":"v1","addr":"172.21.15.203:6803","nonce":4196866481}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6806","nonce":4196866481},{"type":"v1","addr":"172.21.15.203:6807","nonce":4196866481}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6804","nonce":4196866481},{"type":"v1","addr":"172.21.15.203:6805","nonce":4196866481}]},"public_addr":"172.21.15.203:6801/4196866481","cluster_addr":"172.21.15.203:6803/4196866481","heartbeat_back_addr":"172.21.15.203:6807/4196866481","heartbeat_front_addr":"172.21.15.203:6805/4196866481","state":["exists","up"]},{"osd":5,"uuid":"81811e4b-4d24-4f51-a82c-f1d700a2a00d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":170,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6816","nonce":3059290081},{"type":"v1","addr":"172.21.15.203:6817","nonce":3059290081}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6818","nonce":3059290081},{"type":"v1","addr":"172.21.15.203:6819","nonce":3059290081}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6822","nonce":3059290081},{"type":"v1","addr":"172.21.15.203:6823","nonce":3059290081}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6820","nonce":3059290081},{"type":"v1","addr":"172.21.15.203:6821","nonce":3059290081}]},"public_addr":"172.21.15.203:6817/3059290081","cluster_addr":"172.21.15.203:6819/3059290081","heartbeat_back_addr":"172.21.15.203:6823/3059290081","heartbeat_front_addr":"172.21.15.203:6821/3059290081","state":["exists","up"]},{"osd":6,"uuid":"307bac4d-6a5b-443d-af75-56d695d7674f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":170,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6824","nonce":512155569},{"type":"v1","addr":"172.21.15.203:6825","nonce":512155569}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6826","nonce":512155569},{"type":"v1","addr":"172.21.15.203:6827","nonce":512155569}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6830","nonce":512155569},{"type":"v1","addr":"172.21.15.203:6831","nonce":512155569}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6828","nonce":512155569},{"type":"v1","addr":"172.21.15.203:6829","nonce":512155569}]},"public_addr":"172.21.15.203:6825/512155569","cluster_addr":"172.21.15.203:6827/512155569","heartbeat_back_addr":"172.21.15.203:6831/512155569","heartbeat_front_addr":"172.21.15.203:6829/512155569","state":["exists","up"]},{"osd":7,"uuid":"a95dc7ca-9f1b-4295-af70-68a22a2d7d5e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":170,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6808","nonce":3331450438},{"type":"v1","addr":"172.21.15.203:6809","nonce":3331450438}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6810","nonce":3331450438},{"type":"v1","addr":"172.21.15.203:6811","nonce":3331450438}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6814","nonce":3331450438},{"type":"v1","addr":"172.21.15.203:6815","nonce":3331450438}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6812","nonce":3331450438},{"type":"v1","addr":"172.21.15.203:6813","nonce":3331450438}]},"public_addr":"172.21.15.203:6809/3331450438","cluster_addr":"172.21.15.203:6811/3331450438","heartbeat_back_addr":"172.21.15.203:6815/3331450438","heartbeat_front_addr":"172.21.15.203:6813/3331450438","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-09T13:33:50.547652+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-09T13:33:50.589469+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-09T13:33:50.561724+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-09T13:33:50.599650+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-09T13:33:50.411855+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-09T13:33:50.219681+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-09T13:33:50.504664+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-09T13:33:50.441248+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":7,"snaps":[{"begin":104,"length":1},{"begin":221,"length":1}]}],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-08-09T14:51:48.242 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:48.240+0000 7fde0f712640 1 -- 172.21.15.43:0/1994068093 >> [v2:172.21.15.203:6832/2566816332,v1:172.21.15.203:6833/2566816332] conn(0x7fddec06a170 msgr2=0x7fddec06c630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T14:51:48.242 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:48.240+0000 7fde0f712640 1 --2- 172.21.15.43:0/1994068093 >> [v2:172.21.15.203:6832/2566816332,v1:172.21.15.203:6833/2566816332] conn(0x7fddec06a170 0x7fddec06c630 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7fde04036000 tx=0x7fde0402e000 comp rx=0 tx=0).stop 2023-08-09T14:51:48.243 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:48.240+0000 7fde0f712640 1 -- 172.21.15.43:0/1994068093 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fde0806c4e0 msgr2=0x7fde080c5af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T14:51:48.243 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:48.240+0000 7fde0f712640 1 --2- 172.21.15.43:0/1994068093 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fde0806c4e0 0x7fde080c5af0 secure :-1 s=READY pgs=1864 cs=0 l=1 rev1=1 crypto rx=0x7fddf800c450 tx=0x7fddf800c910 comp rx=0 tx=0).stop 2023-08-09T14:51:48.243 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:48.240+0000 7fde0f712640 1 -- 172.21.15.43:0/1994068093 shutdown_connections 2023-08-09T14:51:48.243 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:48.240+0000 7fde0f712640 1 --2- 172.21.15.43:0/1994068093 >> [v2:172.21.15.203:6832/2566816332,v1:172.21.15.203:6833/2566816332] conn(0x7fddec06a170 0x7fddec06c630 unknown :-1 s=CLOSED pgs=59 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T14:51:48.244 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:48.240+0000 7fde0f712640 1 --2- 172.21.15.43:0/1994068093 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fde080c6030 0x7fde080afff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T14:51:48.244 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:48.240+0000 7fde0f712640 1 --2- 172.21.15.43:0/1994068093 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fde0806c4e0 0x7fde080c5af0 unknown :-1 s=CLOSED pgs=1864 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T14:51:48.244 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:48.240+0000 7fde0f712640 1 --2- 172.21.15.43:0/1994068093 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fddf4003a10 0x7fde0806b2e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T14:51:48.245 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:48.240+0000 7fde0f712640 1 -- 172.21.15.43:0/1994068093 >> 172.21.15.43:0/1994068093 conn(0x7fde080ccd80 msgr2=0x7fde080cc040 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-09T14:51:48.245 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:48.240+0000 7fde0f712640 1 -- 172.21.15.43:0/1994068093 shutdown_connections 2023-08-09T14:51:48.245 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:48.241+0000 7fde0f712640 1 -- 172.21.15.43:0/1994068093 wait complete. 2023-08-09T14:51:48.262 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-08-09T14:51:48.489 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:48.487+0000 7fc37b401640 1 Processor -- start 2023-08-09T14:51:48.489 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:48.488+0000 7fc37b401640 1 -- start start 2023-08-09T14:51:48.491 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:48.489+0000 7fc37b401640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7fc3740b2790 0x7fc3740d2ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T14:51:48.491 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:48.489+0000 7fc37b401640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fc3740b2b70 0x7fc3740b5330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T14:51:48.491 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:48.489+0000 7fc37b401640 1 -- --> v1:172.21.15.203:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc3740ce090 con 0x7fc3740b3050 2023-08-09T14:51:48.491 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:48.489+0000 7fc37b401640 1 -- --> v2:172.21.15.43:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc3740b3950 con 0x7fc3740b2790 2023-08-09T14:51:48.491 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:48.489+0000 7fc37b401640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc3740cd7f0 con 0x7fc3740b2b70 2023-08-09T14:51:48.492 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:48.490+0000 7fc37919d640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7fc3740b2790 0x7fc3740d2ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T14:51:48.493 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:48.490+0000 7fc37919d640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7fc3740b2790 0x7fc3740d2ed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:57870/0 (socket says 172.21.15.43:57870) 2023-08-09T14:51:48.493 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:48.490+0000 7fc37919d640 1 -- 172.21.15.43:0/4089369140 learned_addr learned my addr 172.21.15.43:0/4089369140 (peer_addr_for_me v2:172.21.15.43:0/0) 2023-08-09T14:51:48.493 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:48.490+0000 7fc37899c640 1 --2- 172.21.15.43:0/4089369140 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fc3740b2b70 0x7fc3740b5330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T14:51:48.493 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:48.491+0000 7fc37919d640 1 -- 172.21.15.43:0/4089369140 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fc3740b2b70 msgr2=0x7fc3740b5330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-09T14:51:48.493 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:48.491+0000 7fc37919d640 1 --2- 172.21.15.43:0/4089369140 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fc3740b2b70 0x7fc3740b5330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T14:51:48.493 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:48.491+0000 7fc37919d640 1 -- 172.21.15.43:0/4089369140 >> v1:172.21.15.203:6789/0 conn(0x7fc3740b3050 legacy=0x7fc3740b3f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-09T14:51:48.493 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:48.491+0000 7fc37919d640 1 -- 172.21.15.43:0/4089369140 --> v2:172.21.15.43:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc3740cdad0 con 0x7fc3740b2790 2023-08-09T14:51:48.494 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:48.491+0000 7fc37899c640 1 --2- 172.21.15.43:0/4089369140 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fc3740b2b70 0x7fc3740b5330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-08-09T14:51:48.494 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:48.492+0000 7fc37919d640 1 --2- 172.21.15.43:0/4089369140 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc3740b2790 0x7fc3740d2ed0 secure :-1 s=READY pgs=1865 cs=0 l=1 rev1=1 crypto rx=0x7fc370009870 tx=0x7fc37002b8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=194c470ed86992f0 server_cookie=0 in_seq=0 out_seq=0 2023-08-09T14:51:48.495 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:48.493+0000 7fc36bfff640 1 -- 172.21.15.43:0/4089369140 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc370031640 con 0x7fc3740b2790 2023-08-09T14:51:48.495 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:48.493+0000 7fc36bfff640 1 -- 172.21.15.43:0/4089369140 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc3700317a0 con 0x7fc3740b2790 2023-08-09T14:51:48.495 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:48.493+0000 7fc36bfff640 1 -- 172.21.15.43:0/4089369140 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc370031b30 con 0x7fc3740b2790 2023-08-09T14:51:48.495 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:48.493+0000 7fc37b401640 1 -- 172.21.15.43:0/4089369140 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc3740b2790 msgr2=0x7fc3740d2ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T14:51:48.496 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:48.493+0000 7fc37b401640 1 --2- 172.21.15.43:0/4089369140 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc3740b2790 0x7fc3740d2ed0 secure :-1 s=READY pgs=1865 cs=0 l=1 rev1=1 crypto rx=0x7fc370009870 tx=0x7fc37002b8f0 comp rx=0 tx=0).stop 2023-08-09T14:51:48.496 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:48.494+0000 7fc37b401640 1 -- 172.21.15.43:0/4089369140 shutdown_connections 2023-08-09T14:51:48.496 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:48.494+0000 7fc37b401640 1 --2- 172.21.15.43:0/4089369140 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fc3740b2b70 0x7fc3740b5330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T14:51:48.496 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:48.494+0000 7fc37b401640 1 --2- 172.21.15.43:0/4089369140 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc3740b2790 0x7fc3740d2ed0 unknown :-1 s=CLOSED pgs=1865 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T14:51:48.496 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:48.494+0000 7fc37b401640 1 -- 172.21.15.43:0/4089369140 >> 172.21.15.43:0/4089369140 conn(0x7fc37409b7b0 msgr2=0x7fc3740d24e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-09T14:51:48.496 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:48.494+0000 7fc37b401640 1 -- 172.21.15.43:0/4089369140 shutdown_connections 2023-08-09T14:51:48.496 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:48.494+0000 7fc37b401640 1 -- 172.21.15.43:0/4089369140 wait complete. 2023-08-09T14:51:48.497 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:48.496+0000 7fc37b401640 1 Processor -- start 2023-08-09T14:51:48.497 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:48.496+0000 7fc37b401640 1 -- start start 2023-08-09T14:51:48.498 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:48.497+0000 7fc37b401640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fc3740b2790 0x7fc3740c8a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T14:51:48.498 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:48.497+0000 7fc37b401640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc3740b2b70 0x7fc37414a770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T14:51:48.498 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:48.497+0000 7fc37b401640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fc3740b3050 0x7fc37414acb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T14:51:48.499 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:48.497+0000 7fc37b401640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc3740b1820 con 0x7fc3740b2b70 2023-08-09T14:51:48.499 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:48.497+0000 7fc37b401640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc3740c8f80 con 0x7fc3740b3050 2023-08-09T14:51:48.499 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:48.497+0000 7fc37b401640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc3740b0ba0 con 0x7fc3740b2790 2023-08-09T14:51:48.500 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:48.497+0000 7fc37899c640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc3740b2b70 0x7fc37414a770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T14:51:48.500 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:48.497+0000 7fc37899c640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc3740b2b70 0x7fc37414a770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:57878/0 (socket says 172.21.15.43:57878) 2023-08-09T14:51:48.501 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:48.497+0000 7fc37899c640 1 -- 172.21.15.43:0/2775552608 learned_addr learned my addr 172.21.15.43:0/2775552608 (peer_addr_for_me v2:172.21.15.43:0/0) 2023-08-09T14:51:48.501 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:48.498+0000 7fc37899c640 1 -- 172.21.15.43:0/2775552608 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fc3740b2790 msgr2=0x7fc3740c8a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T14:51:48.501 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:48.498+0000 7fc37899c640 1 --2- 172.21.15.43:0/2775552608 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fc3740b2790 0x7fc3740c8a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T14:51:48.501 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:48.498+0000 7fc37899c640 1 -- 172.21.15.43:0/2775552608 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fc3740b3050 msgr2=0x7fc37414acb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T14:51:48.501 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:48.498+0000 7fc37899c640 1 --2- 172.21.15.43:0/2775552608 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fc3740b3050 0x7fc37414acb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T14:51:48.501 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:48.498+0000 7fc37899c640 1 -- 172.21.15.43:0/2775552608 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc3700094d0 con 0x7fc3740b2b70 2023-08-09T14:51:48.501 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:48.498+0000 7fc37899c640 1 --2- 172.21.15.43:0/2775552608 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc3740b2b70 0x7fc37414a770 secure :-1 s=READY pgs=1866 cs=0 l=1 rev1=1 crypto rx=0x7fc36400c7f0 tx=0x7fc36400ccb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-09T14:51:48.502 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:48.498+0000 7fc369ffb640 1 -- 172.21.15.43:0/2775552608 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc364007820 con 0x7fc3740b2b70 2023-08-09T14:51:48.502 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:48.498+0000 7fc369ffb640 1 -- 172.21.15.43:0/2775552608 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc364007980 con 0x7fc3740b2b70 2023-08-09T14:51:48.502 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:48.498+0000 7fc369ffb640 1 -- 172.21.15.43:0/2775552608 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc364007cf0 con 0x7fc3740b2b70 2023-08-09T14:51:48.502 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:48.498+0000 7fc37b401640 1 -- 172.21.15.43:0/2775552608 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc3740b0e80 con 0x7fc3740b2b70 2023-08-09T14:51:48.502 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:48.499+0000 7fc37b401640 1 -- 172.21.15.43:0/2775552608 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc3740b1440 con 0x7fc3740b2b70 2023-08-09T14:51:48.503 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:48.500+0000 7fc37b401640 1 -- 172.21.15.43:0/2775552608 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc3740739f0 con 0x7fc3740b2b70 2023-08-09T14:51:48.503 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:48.501+0000 7fc369ffb640 1 -- 172.21.15.43:0/2775552608 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 79) v1 ==== 88443+0+0 (secure 0 0 0) 0x7fc3640113f0 con 0x7fc3740b2b70 2023-08-09T14:51:48.503 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:48.502+0000 7fc369ffb640 1 --2- 172.21.15.43:0/2775552608 >> [v2:172.21.15.203:6832/2566816332,v1:172.21.15.203:6833/2566816332] conn(0x7fc35806a170 0x7fc35806c630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T14:51:48.503 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:48.502+0000 7fc369ffb640 1 -- 172.21.15.43:0/2775552608 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(171..171 src has 1..171) v4 ==== 6426+0+0 (secure 0 0 0) 0x7fc364088d80 con 0x7fc3740b2b70 2023-08-09T14:51:48.504 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:48.502+0000 7fc37919d640 1 --2- 172.21.15.43:0/2775552608 >> [v2:172.21.15.203:6832/2566816332,v1:172.21.15.203:6833/2566816332] conn(0x7fc35806a170 0x7fc35806c630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T14:51:48.505 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:48.504+0000 7fc37919d640 1 --2- 172.21.15.43:0/2775552608 >> [v2:172.21.15.203:6832/2566816332,v1:172.21.15.203:6833/2566816332] conn(0x7fc35806a170 0x7fc35806c630 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7fc370036000 tx=0x7fc370030000 comp rx=0 tx=0).ready entity=mgr.7765 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-09T14:51:48.507 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:48.506+0000 7fc369ffb640 1 -- 172.21.15.43:0/2775552608 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178066 (secure 0 0 0) 0x7fc364054040 con 0x7fc3740b2b70 2023-08-09T14:51:48.707 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:48.706+0000 7fc37b401640 1 -- 172.21.15.43:0/2775552608 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fc374065770 con 0x7fc3740b2b70 2023-08-09T14:51:48.710 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:48.709+0000 7fc369ffb640 1 -- 172.21.15.43:0/2775552608 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v171) v1 ==== 74+0+18579 (secure 0 0 0) 0x7fc364080e60 con 0x7fc3740b2b70 2023-08-09T14:51:48.711 INFO:teuthology.orchestra.run.smithi043.stdout: 2023-08-09T14:51:48.711 INFO:teuthology.orchestra.run.smithi043.stdout:{"epoch":171,"fsid":"daac87ee-1337-4250-a955-aea4e408a39b","created":"2023-08-09T13:33:41.701198+0000","modified":"2023-08-09T14:51:47.253220+0000","last_up_change":"2023-08-09T13:33:53.783453+0000","last_in_change":"2023-08-09T13:33:45.148904+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":21,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":"rbd","create_time":"2023-08-09T13:33:54.365137+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":".mgr","create_time":"2023-08-09T13:33:54.831803+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":20,"pool_name":"cephfs_metadata","create_time":"2023-08-09T14:51:45.692613+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"171","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":21,"pool_name":"cephfs_data","create_time":"2023-08-09T14:51:46.286715+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"171","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"cbd49ecd-e896-4fbd-864e-5c37a829b165","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":170,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6800","nonce":502667335},{"type":"v1","addr":"172.21.15.43:6801","nonce":502667335}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6802","nonce":502667335},{"type":"v1","addr":"172.21.15.43:6803","nonce":502667335}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6806","nonce":502667335},{"type":"v1","addr":"172.21.15.43:6807","nonce":502667335}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6804","nonce":502667335},{"type":"v1","addr":"172.21.15.43:6805","nonce":502667335}]},"public_addr":"172.21.15.43:6801/502667335","cluster_addr":"172.21.15.43:6803/502667335","heartbeat_back_addr":"172.21.15.43:6807/502667335","heartbeat_front_addr":"172.21.15.43:6805/502667335","state":["exists","up"]},{"osd":1,"uuid":"75c7e0fe-1ec7-4fe8-a2f2-3661a65b8500","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":170,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6808","nonce":3289134610},{"type":"v1","addr":"172.21.15.43:6809","nonce":3289134610}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6810","nonce":3289134610},{"type":"v1","addr":"172.21.15.43:6812","nonce":3289134610}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6821","nonce":3289134610},{"type":"v1","addr":"172.21.15.43:6824","nonce":3289134610}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6815","nonce":3289134610},{"type":"v1","addr":"172.21.15.43:6818","nonce":3289134610}]},"public_addr":"172.21.15.43:6809/3289134610","cluster_addr":"172.21.15.43:6812/3289134610","heartbeat_back_addr":"172.21.15.43:6824/3289134610","heartbeat_front_addr":"172.21.15.43:6818/3289134610","state":["exists","up"]},{"osd":2,"uuid":"bb5c41e9-7f5f-4125-9b01-3bb8bfc2ea83","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":170,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6811","nonce":4079709736},{"type":"v1","addr":"172.21.15.43:6814","nonce":4079709736}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6817","nonce":4079709736},{"type":"v1","addr":"172.21.15.43:6820","nonce":4079709736}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6829","nonce":4079709736},{"type":"v1","addr":"172.21.15.43:6831","nonce":4079709736}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6823","nonce":4079709736},{"type":"v1","addr":"172.21.15.43:6826","nonce":4079709736}]},"public_addr":"172.21.15.43:6814/4079709736","cluster_addr":"172.21.15.43:6820/4079709736","heartbeat_back_addr":"172.21.15.43:6831/4079709736","heartbeat_front_addr":"172.21.15.43:6826/4079709736","state":["exists","up"]},{"osd":3,"uuid":"bfe4c8b4-694a-465c-87e6-2b2aab852613","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":170,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6813","nonce":1709538759},{"type":"v1","addr":"172.21.15.43:6816","nonce":1709538759}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6819","nonce":1709538759},{"type":"v1","addr":"172.21.15.43:6822","nonce":1709538759}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6828","nonce":1709538759},{"type":"v1","addr":"172.21.15.43:6830","nonce":1709538759}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6825","nonce":1709538759},{"type":"v1","addr":"172.21.15.43:6827","nonce":1709538759}]},"public_addr":"172.21.15.43:6816/1709538759","cluster_addr":"172.21.15.43:6822/1709538759","heartbeat_back_addr":"172.21.15.43:6830/1709538759","heartbeat_front_addr":"172.21.15.43:6827/1709538759","state":["exists","up"]},{"osd":4,"uuid":"fa2e8a77-aba1-4756-853e-0d174a5bdeba","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":170,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6800","nonce":4196866481},{"type":"v1","addr":"172.21.15.203:6801","nonce":4196866481}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6802","nonce":4196866481},{"type":"v1","addr":"172.21.15.203:6803","nonce":4196866481}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6806","nonce":4196866481},{"type":"v1","addr":"172.21.15.203:6807","nonce":4196866481}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6804","nonce":4196866481},{"type":"v1","addr":"172.21.15.203:6805","nonce":4196866481}]},"public_addr":"172.21.15.203:6801/4196866481","cluster_addr":"172.21.15.203:6803/4196866481","heartbeat_back_addr":"172.21.15.203:6807/4196866481","heartbeat_front_addr":"172.21.15.203:6805/4196866481","state":["exists","up"]},{"osd":5,"uuid":"81811e4b-4d24-4f51-a82c-f1d700a2a00d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":170,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6816","nonce":3059290081},{"type":"v1","addr":"172.21.15.203:6817","nonce":3059290081}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6818","nonce":3059290081},{"type":"v1","addr":"172.21.15.203:6819","nonce":3059290081}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6822","nonce":3059290081},{"type":"v1","addr":"172.21.15.203:6823","nonce":3059290081}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6820","nonce":3059290081},{"type":"v1","addr":"172.21.15.203:6821","nonce":3059290081}]},"public_addr":"172.21.15.203:6817/3059290081","cluster_addr":"172.21.15.203:6819/3059290081","heartbeat_back_addr":"172.21.15.203:6823/3059290081","heartbeat_front_addr":"172.21.15.203:6821/3059290081","state":["exists","up"]},{"osd":6,"uuid":"307bac4d-6a5b-443d-af75-56d695d7674f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":170,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6824","nonce":512155569},{"type":"v1","addr":"172.21.15.203:6825","nonce":512155569}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6826","nonce":512155569},{"type":"v1","addr":"172.21.15.203:6827","nonce":512155569}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6830","nonce":512155569},{"type":"v1","addr":"172.21.15.203:6831","nonce":512155569}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6828","nonce":512155569},{"type":"v1","addr":"172.21.15.203:6829","nonce":512155569}]},"public_addr":"172.21.15.203:6825/512155569","cluster_addr":"172.21.15.203:6827/512155569","heartbeat_back_addr":"172.21.15.203:6831/512155569","heartbeat_front_addr":"172.21.15.203:6829/512155569","state":["exists","up"]},{"osd":7,"uuid":"a95dc7ca-9f1b-4295-af70-68a22a2d7d5e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":170,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6808","nonce":3331450438},{"type":"v1","addr":"172.21.15.203:6809","nonce":3331450438}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6810","nonce":3331450438},{"type":"v1","addr":"172.21.15.203:6811","nonce":3331450438}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6814","nonce":3331450438},{"type":"v1","addr":"172.21.15.203:6815","nonce":3331450438}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6812","nonce":3331450438},{"type":"v1","addr":"172.21.15.203:6813","nonce":3331450438}]},"public_addr":"172.21.15.203:6809/3331450438","cluster_addr":"172.21.15.203:6811/3331450438","heartbeat_back_addr":"172.21.15.203:6815/3331450438","heartbeat_front_addr":"172.21.15.203:6813/3331450438","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-09T13:33:50.547652+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-09T13:33:50.589469+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-09T13:33:50.561724+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-09T13:33:50.599650+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-09T13:33:50.411855+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-09T13:33:50.219681+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-09T13:33:50.504664+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-09T13:33:50.441248+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":7,"snaps":[{"begin":104,"length":1},{"begin":221,"length":1}]}],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-08-09T14:51:48.714 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:48.712+0000 7fc37b401640 1 -- 172.21.15.43:0/2775552608 >> [v2:172.21.15.203:6832/2566816332,v1:172.21.15.203:6833/2566816332] conn(0x7fc35806a170 msgr2=0x7fc35806c630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T14:51:48.714 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:48.712+0000 7fc37b401640 1 --2- 172.21.15.43:0/2775552608 >> [v2:172.21.15.203:6832/2566816332,v1:172.21.15.203:6833/2566816332] conn(0x7fc35806a170 0x7fc35806c630 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7fc370036000 tx=0x7fc370030000 comp rx=0 tx=0).stop 2023-08-09T14:51:48.714 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:48.712+0000 7fc37b401640 1 -- 172.21.15.43:0/2775552608 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc3740b2b70 msgr2=0x7fc37414a770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T14:51:48.714 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:48.712+0000 7fc37b401640 1 --2- 172.21.15.43:0/2775552608 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc3740b2b70 0x7fc37414a770 secure :-1 s=READY pgs=1866 cs=0 l=1 rev1=1 crypto rx=0x7fc36400c7f0 tx=0x7fc36400ccb0 comp rx=0 tx=0).stop 2023-08-09T14:51:48.714 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:48.712+0000 7fc37b401640 1 -- 172.21.15.43:0/2775552608 shutdown_connections 2023-08-09T14:51:48.714 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:48.712+0000 7fc37b401640 1 --2- 172.21.15.43:0/2775552608 >> [v2:172.21.15.203:6832/2566816332,v1:172.21.15.203:6833/2566816332] conn(0x7fc35806a170 0x7fc35806c630 unknown :-1 s=CLOSED pgs=60 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T14:51:48.715 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:48.712+0000 7fc37b401640 1 --2- 172.21.15.43:0/2775552608 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fc3740b3050 0x7fc37414acb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T14:51:48.715 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:48.712+0000 7fc37b401640 1 --2- 172.21.15.43:0/2775552608 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc3740b2b70 0x7fc37414a770 unknown :-1 s=CLOSED pgs=1866 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T14:51:48.715 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:48.712+0000 7fc37b401640 1 --2- 172.21.15.43:0/2775552608 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fc3740b2790 0x7fc3740c8a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T14:51:48.715 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:48.712+0000 7fc37b401640 1 -- 172.21.15.43:0/2775552608 >> 172.21.15.43:0/2775552608 conn(0x7fc37409b7b0 msgr2=0x7fc3740b5ca0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-09T14:51:48.715 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:48.712+0000 7fc37b401640 1 -- 172.21.15.43:0/2775552608 shutdown_connections 2023-08-09T14:51:48.715 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:48.712+0000 7fc37b401640 1 -- 172.21.15.43:0/2775552608 wait complete. 2023-08-09T14:51:48.731 INFO:tasks.ceph.mds.a.smithi043.stderr:2023-08-09T14:51:48.729+0000 7f4d461e1640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2023-08-09T14:51:48.734 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs standby_count_wanted 0 2023-08-09T14:51:48.967 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:48.966+0000 7f1fd40e7640 1 Processor -- start 2023-08-09T14:51:48.968 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:48.966+0000 7f1fd40e7640 1 -- start start 2023-08-09T14:51:48.969 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:48.968+0000 7f1fd40e7640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f1fcc0c86f0 0x7f1fcc0c8b50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T14:51:48.969 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:48.968+0000 7f1fd40e7640 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7f1fcc06c8d0 0x7f1fcc06cd00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T14:51:48.970 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:48.968+0000 7f1fd40e7640 1 -- --> v1:172.21.15.43:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1fcc0c9090 con 0x7f1fcc143c20 2023-08-09T14:51:48.970 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:48.968+0000 7f1fd40e7640 1 -- --> v2:172.21.15.43:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1fcc144000 con 0x7f1fcc0c86f0 2023-08-09T14:51:48.970 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:48.968+0000 7f1fd40e7640 1 -- --> v2:172.21.15.203:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1fcc06c440 con 0x7f1fcc06c8d0 2023-08-09T14:51:48.970 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:48.968+0000 7f1fd1e83640 1 --1- >> v1:172.21.15.43:6789/0 conn(0x7f1fcc143c20 0x7f1fcc06f540 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.43:6789/0 says I am v1:172.21.15.43:47278/0 (socket says 172.21.15.43:47278) 2023-08-09T14:51:48.970 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:48.968+0000 7f1fd1e83640 1 -- 172.21.15.43:0/3525067719 learned_addr learned my addr 172.21.15.43:0/3525067719 (peer_addr_for_me v1:172.21.15.43:0/0) 2023-08-09T14:51:48.970 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:48.968+0000 7f1fd1682640 1 --2- 172.21.15.43:0/3525067719 >> v2:172.21.15.43:3300/0 conn(0x7f1fcc0c86f0 0x7f1fcc0c8b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T14:51:48.972 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:48.968+0000 7f1fd2684640 1 --2- 172.21.15.43:0/3525067719 >> v2:172.21.15.203:3300/0 conn(0x7f1fcc06c8d0 0x7f1fcc06cd00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T14:51:48.972 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:48.968+0000 7f1fd1682640 1 -- 172.21.15.43:0/3525067719 >> v2:172.21.15.203:3300/0 conn(0x7f1fcc06c8d0 msgr2=0x7f1fcc06cd00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-09T14:51:48.972 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:48.969+0000 7f1fd1682640 1 --2- 172.21.15.43:0/3525067719 >> v2:172.21.15.203:3300/0 conn(0x7f1fcc06c8d0 0x7f1fcc06cd00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T14:51:48.973 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:48.969+0000 7f1fd1682640 1 -- 172.21.15.43:0/3525067719 >> v1:172.21.15.43:6789/0 conn(0x7f1fcc143c20 legacy=0x7f1fcc06f540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T14:51:48.973 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:48.969+0000 7f1fd1682640 1 -- 172.21.15.43:0/3525067719 --> v2:172.21.15.43:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1fcc06fc50 con 0x7f1fcc0c86f0 2023-08-09T14:51:48.973 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:48.969+0000 7f1fd2684640 1 --2- 172.21.15.43:0/3525067719 >> v2:172.21.15.203:3300/0 conn(0x7f1fcc06c8d0 0x7f1fcc06cd00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-08-09T14:51:48.973 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:48.969+0000 7f1fd0e81640 1 -- 172.21.15.43:0/3525067719 <== mon.0 v1:172.21.15.43:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 659442973 0 0) 0x7f1fcc0c9090 con 0x7f1fcc143c20 2023-08-09T14:51:48.973 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:48.970+0000 7f1fd1682640 1 --2- 172.21.15.43:0/3525067719 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f1fcc0c86f0 0x7f1fcc0c8b50 secure :-1 s=READY pgs=1868 cs=0 l=1 rev1=1 crypto rx=0x7f1fcc0caad0 tx=0x7f1fbc02b840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2a1fb70e67d80433 server_cookie=0 in_seq=0 out_seq=0 2023-08-09T14:51:48.973 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:48.970+0000 7f1fd0e81640 1 -- 172.21.15.43:0/3525067719 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1fbc02e070 con 0x7f1fcc0c86f0 2023-08-09T14:51:48.973 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:48.970+0000 7f1fd0e81640 1 -- 172.21.15.43:0/3525067719 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1fbc0315a0 con 0x7f1fcc0c86f0 2023-08-09T14:51:48.974 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:48.970+0000 7f1fd0e81640 1 -- 172.21.15.43:0/3525067719 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1fbc031930 con 0x7f1fcc0c86f0 2023-08-09T14:51:48.974 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:48.971+0000 7f1fd40e7640 1 -- 172.21.15.43:0/3525067719 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f1fcc0c86f0 msgr2=0x7f1fcc0c8b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T14:51:48.975 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:48.971+0000 7f1fd40e7640 1 --2- 172.21.15.43:0/3525067719 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f1fcc0c86f0 0x7f1fcc0c8b50 secure :-1 s=READY pgs=1868 cs=0 l=1 rev1=1 crypto rx=0x7f1fcc0caad0 tx=0x7f1fbc02b840 comp rx=0 tx=0).stop 2023-08-09T14:51:48.975 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:48.971+0000 7f1fd40e7640 1 -- 172.21.15.43:0/3525067719 shutdown_connections 2023-08-09T14:51:48.975 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:48.971+0000 7f1fd40e7640 1 --2- 172.21.15.43:0/3525067719 >> v2:172.21.15.203:3300/0 conn(0x7f1fcc06c8d0 0x7f1fcc06cd00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T14:51:48.975 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:48.971+0000 7f1fd40e7640 1 --2- 172.21.15.43:0/3525067719 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f1fcc0c86f0 0x7f1fcc0c8b50 unknown :-1 s=CLOSED pgs=1868 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T14:51:48.975 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:48.971+0000 7f1fd40e7640 1 -- 172.21.15.43:0/3525067719 >> 172.21.15.43:0/3525067719 conn(0x7f1fcc0ce0a0 msgr2=0x7f1fcc0ce480 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-09T14:51:48.975 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:48.971+0000 7f1fd40e7640 1 -- 172.21.15.43:0/3525067719 shutdown_connections 2023-08-09T14:51:48.975 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:48.971+0000 7f1fd40e7640 1 -- 172.21.15.43:0/3525067719 wait complete. 2023-08-09T14:51:48.976 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:48.973+0000 7f1fd40e7640 1 Processor -- start 2023-08-09T14:51:48.976 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:48.973+0000 7f1fd40e7640 1 -- start start 2023-08-09T14:51:48.977 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:48.974+0000 7f1fd40e7640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f1fcc06c8d0 0x7f1fcc0b0d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T14:51:48.977 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:48.974+0000 7f1fd40e7640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f1fcc0c86f0 0x7f1fcc0c6350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T14:51:48.978 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:48.974+0000 7f1fd40e7640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f1fcc143c20 0x7f1fcc13fa00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T14:51:48.978 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:48.974+0000 7f1fd40e7640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1fcc1f0d60 con 0x7f1fcc06c8d0 2023-08-09T14:51:48.978 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:48.974+0000 7f1fd40e7640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1fcc1f1040 con 0x7f1fcc0c86f0 2023-08-09T14:51:48.978 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:48.974+0000 7f1fd40e7640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1fcc1f1320 con 0x7f1fcc143c20 2023-08-09T14:51:48.978 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:48.975+0000 7f1fd1e83640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f1fcc06c8d0 0x7f1fcc0b0d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T14:51:48.978 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:48.975+0000 7f1fd1e83640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f1fcc06c8d0 0x7f1fcc0b0d80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:57884/0 (socket says 172.21.15.43:57884) 2023-08-09T14:51:48.978 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:48.975+0000 7f1fd1e83640 1 -- 172.21.15.43:0/1040715033 learned_addr learned my addr 172.21.15.43:0/1040715033 (peer_addr_for_me v2:172.21.15.43:0/0) 2023-08-09T14:51:48.979 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:48.975+0000 7f1fd1e83640 1 -- 172.21.15.43:0/1040715033 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f1fcc143c20 msgr2=0x7f1fcc13fa00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T14:51:48.979 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:48.975+0000 7f1fd1e83640 1 --2- 172.21.15.43:0/1040715033 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f1fcc143c20 0x7f1fcc13fa00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T14:51:48.979 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:48.975+0000 7f1fd1e83640 1 -- 172.21.15.43:0/1040715033 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f1fcc0c86f0 msgr2=0x7f1fcc0c6350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T14:51:48.979 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:48.975+0000 7f1fd1682640 1 --2- 172.21.15.43:0/1040715033 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f1fcc0c86f0 0x7f1fcc0c6350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T14:51:48.979 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:48.975+0000 7f1fd1e83640 1 --2- 172.21.15.43:0/1040715033 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f1fcc0c86f0 0x7f1fcc0c6350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T14:51:48.979 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:48.975+0000 7f1fd1e83640 1 -- 172.21.15.43:0/1040715033 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1fbc009400 con 0x7f1fcc06c8d0 2023-08-09T14:51:48.979 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:48.975+0000 7f1fd1682640 1 --2- 172.21.15.43:0/1040715033 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f1fcc0c86f0 0x7f1fcc0c6350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-08-09T14:51:48.979 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:48.976+0000 7f1fd1e83640 1 --2- 172.21.15.43:0/1040715033 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f1fcc06c8d0 0x7f1fcc0b0d80 secure :-1 s=READY pgs=1869 cs=0 l=1 rev1=1 crypto rx=0x7f1fc800c890 tx=0x7f1fc800cd50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-09T14:51:48.980 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:48.976+0000 7f1fc27fc640 1 -- 172.21.15.43:0/1040715033 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1fc800a640 con 0x7f1fcc06c8d0 2023-08-09T14:51:48.980 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:48.976+0000 7f1fc27fc640 1 -- 172.21.15.43:0/1040715033 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1fc800a7c0 con 0x7f1fcc06c8d0 2023-08-09T14:51:48.980 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:48.976+0000 7f1fc27fc640 1 -- 172.21.15.43:0/1040715033 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1fc800ab70 con 0x7f1fcc06c8d0 2023-08-09T14:51:48.980 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:48.976+0000 7f1fd40e7640 1 -- 172.21.15.43:0/1040715033 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1fcc1f1600 con 0x7f1fcc06c8d0 2023-08-09T14:51:48.980 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:48.976+0000 7f1fd40e7640 1 -- 172.21.15.43:0/1040715033 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1fcc1f0a10 con 0x7f1fcc06c8d0 2023-08-09T14:51:48.981 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:48.977+0000 7f1fd40e7640 1 -- 172.21.15.43:0/1040715033 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1fcc1f1b20 con 0x7f1fcc06c8d0 2023-08-09T14:51:48.981 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:48.978+0000 7f1fc27fc640 1 -- 172.21.15.43:0/1040715033 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 79) v1 ==== 88443+0+0 (secure 0 0 0) 0x7f1fc800acd0 con 0x7f1fcc06c8d0 2023-08-09T14:51:48.981 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:48.979+0000 7f1fc27fc640 1 --2- 172.21.15.43:0/1040715033 >> [v2:172.21.15.203:6832/2566816332,v1:172.21.15.203:6833/2566816332] conn(0x7f1fb006a170 0x7f1fb006c630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T14:51:48.981 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:48.979+0000 7f1fc27fc640 1 -- 172.21.15.43:0/1040715033 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(172..172 src has 1..172) v4 ==== 6426+0+0 (secure 0 0 0) 0x7f1fc8087e80 con 0x7f1fcc06c8d0 2023-08-09T14:51:48.982 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:48.980+0000 7f1fd1682640 1 --2- 172.21.15.43:0/1040715033 >> [v2:172.21.15.203:6832/2566816332,v1:172.21.15.203:6833/2566816332] conn(0x7f1fb006a170 0x7f1fb006c630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T14:51:48.982 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:48.981+0000 7f1fd1682640 1 --2- 172.21.15.43:0/1040715033 >> [v2:172.21.15.203:6832/2566816332,v1:172.21.15.203:6833/2566816332] conn(0x7f1fb006a170 0x7f1fb006c630 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f1fbc036000 tx=0x7f1fbc03d040 comp rx=0 tx=0).ready entity=mgr.7765 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-09T14:51:48.985 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:48.984+0000 7f1fc27fc640 1 -- 172.21.15.43:0/1040715033 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178066 (secure 0 0 0) 0x7f1fc8053250 con 0x7f1fcc06c8d0 2023-08-09T14:51:49.219 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:49.217+0000 7f1fd40e7640 1 -- 172.21.15.43:0/1040715033 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} v 0) v1 -- 0x7f1fcc0739f0 con 0x7f1fcc06c8d0 2023-08-09T14:51:49.811 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:49.809+0000 7f1fc27fc640 1 -- 172.21.15.43:0/1040715033 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]=0 v77) v1 ==== 118+0+0 (secure 0 0 0) 0x7f1fc80180b0 con 0x7f1fcc06c8d0 2023-08-09T14:51:49.814 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:49.813+0000 7f1fd40e7640 1 -- 172.21.15.43:0/1040715033 >> [v2:172.21.15.203:6832/2566816332,v1:172.21.15.203:6833/2566816332] conn(0x7f1fb006a170 msgr2=0x7f1fb006c630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T14:51:49.814 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:49.813+0000 7f1fd40e7640 1 --2- 172.21.15.43:0/1040715033 >> [v2:172.21.15.203:6832/2566816332,v1:172.21.15.203:6833/2566816332] conn(0x7f1fb006a170 0x7f1fb006c630 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f1fbc036000 tx=0x7f1fbc03d040 comp rx=0 tx=0).stop 2023-08-09T14:51:49.815 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:49.813+0000 7f1fd40e7640 1 -- 172.21.15.43:0/1040715033 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f1fcc06c8d0 msgr2=0x7f1fcc0b0d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T14:51:49.815 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:49.813+0000 7f1fd40e7640 1 --2- 172.21.15.43:0/1040715033 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f1fcc06c8d0 0x7f1fcc0b0d80 secure :-1 s=READY pgs=1869 cs=0 l=1 rev1=1 crypto rx=0x7f1fc800c890 tx=0x7f1fc800cd50 comp rx=0 tx=0).stop 2023-08-09T14:51:49.815 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:49.813+0000 7f1fd40e7640 1 -- 172.21.15.43:0/1040715033 shutdown_connections 2023-08-09T14:51:49.815 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:49.813+0000 7f1fd40e7640 1 --2- 172.21.15.43:0/1040715033 >> [v2:172.21.15.203:6832/2566816332,v1:172.21.15.203:6833/2566816332] conn(0x7f1fb006a170 0x7f1fb006c630 unknown :-1 s=CLOSED pgs=61 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T14:51:49.816 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:49.813+0000 7f1fd40e7640 1 --2- 172.21.15.43:0/1040715033 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f1fcc143c20 0x7f1fcc13fa00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T14:51:49.816 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:49.813+0000 7f1fd40e7640 1 --2- 172.21.15.43:0/1040715033 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f1fcc0c86f0 0x7f1fcc0c6350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T14:51:49.816 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:49.813+0000 7f1fd40e7640 1 --2- 172.21.15.43:0/1040715033 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f1fcc06c8d0 0x7f1fcc0b0d80 unknown :-1 s=CLOSED pgs=1869 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T14:51:49.816 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:49.813+0000 7f1fd40e7640 1 -- 172.21.15.43:0/1040715033 >> 172.21.15.43:0/1040715033 conn(0x7f1fcc0ce0a0 msgr2=0x7f1fcc15c9e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-09T14:51:49.816 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:49.813+0000 7f1fd40e7640 1 -- 172.21.15.43:0/1040715033 shutdown_connections 2023-08-09T14:51:49.816 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:49.813+0000 7f1fd40e7640 1 -- 172.21.15.43:0/1040715033 wait complete. 2023-08-09T14:51:49.838 DEBUG:tasks.cephfs.filesystem:fs_config: {} 2023-08-09T14:51:49.838 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs allow_standby_replay false 2023-08-09T14:51:50.064 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:50.063+0000 7f7a22e0f640 1 Processor -- start 2023-08-09T14:51:50.065 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:50.063+0000 7f7a22e0f640 1 -- start start 2023-08-09T14:51:50.066 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:50.064+0000 7f7a22e0f640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f7a1c15c3a0 0x7f7a1c1608a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T14:51:50.066 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:50.065+0000 7f7a22e0f640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f7a1c0cd600 0x7f7a1c0d0580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T14:51:50.066 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:50.065+0000 7f7a22e0f640 1 -- --> v1:172.21.15.43:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7a1c160de0 con 0x7f7a1c0b0f00 2023-08-09T14:51:50.067 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:50.065+0000 7f7a22e0f640 1 -- --> v2:172.21.15.43:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7a1c0b81c0 con 0x7f7a1c15c3a0 2023-08-09T14:51:50.067 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:50.065+0000 7f7a22e0f640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7a1c0b7c00 con 0x7f7a1c0cd600 2023-08-09T14:51:50.067 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:50.065+0000 7f7a2160c640 1 --1- >> v1:172.21.15.43:6789/0 conn(0x7f7a1c0b0f00 0x7f7a1c0ccef0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.43:6789/0 says I am v1:172.21.15.43:47288/0 (socket says 172.21.15.43:47288) 2023-08-09T14:51:50.068 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:50.065+0000 7f7a2160c640 1 -- 172.21.15.43:0/1185313908 learned_addr learned my addr 172.21.15.43:0/1185313908 (peer_addr_for_me v1:172.21.15.43:0/0) 2023-08-09T14:51:50.068 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:50.065+0000 7f7a21e0d640 1 --2- 172.21.15.43:0/1185313908 >> v2:172.21.15.43:3300/0 conn(0x7f7a1c15c3a0 0x7f7a1c1608a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T14:51:50.068 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:50.065+0000 7f7a2260e640 1 --2- 172.21.15.43:0/1185313908 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f7a1c0cd600 0x7f7a1c0d0580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T14:51:50.068 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:50.066+0000 7f7a21e0d640 1 -- 172.21.15.43:0/1185313908 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f7a1c0cd600 msgr2=0x7f7a1c0d0580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-09T14:51:50.068 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:50.066+0000 7f7a21e0d640 1 --2- 172.21.15.43:0/1185313908 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f7a1c0cd600 0x7f7a1c0d0580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T14:51:50.068 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:50.066+0000 7f7a20e0b640 1 -- 172.21.15.43:0/1185313908 <== mon.0 v1:172.21.15.43:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2100174966 0 0) 0x7f7a1c160de0 con 0x7f7a1c0b0f00 2023-08-09T14:51:50.069 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:50.066+0000 7f7a21e0d640 1 -- 172.21.15.43:0/1185313908 >> v1:172.21.15.43:6789/0 conn(0x7f7a1c0b0f00 legacy=0x7f7a1c0ccef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T14:51:50.069 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:50.066+0000 7f7a21e0d640 1 -- 172.21.15.43:0/1185313908 --> v2:172.21.15.43:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7a1c0b7ee0 con 0x7f7a1c15c3a0 2023-08-09T14:51:50.070 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:50.067+0000 7f7a21e0d640 1 --2- 172.21.15.43:0/1185313908 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f7a1c15c3a0 0x7f7a1c1608a0 secure :-1 s=READY pgs=1871 cs=0 l=1 rev1=1 crypto rx=0x7f7a180097a0 tx=0x7f7a1802b820 comp rx=0 tx=0).ready entity=mon.0 client_cookie=932c6894c45cdc94 server_cookie=0 in_seq=0 out_seq=0 2023-08-09T14:51:50.070 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:50.067+0000 7f7a20e0b640 1 -- 172.21.15.43:0/1185313908 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7a18031780 con 0x7f7a1c15c3a0 2023-08-09T14:51:50.070 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:50.068+0000 7f7a20e0b640 1 -- 172.21.15.43:0/1185313908 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7a180318e0 con 0x7f7a1c15c3a0 2023-08-09T14:51:50.070 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:50.068+0000 7f7a20e0b640 1 -- 172.21.15.43:0/1185313908 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7a18031c70 con 0x7f7a1c15c3a0 2023-08-09T14:51:50.070 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:50.068+0000 7f7a22e0f640 1 -- 172.21.15.43:0/1185313908 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f7a1c15c3a0 msgr2=0x7f7a1c1608a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T14:51:50.071 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:50.068+0000 7f7a22e0f640 1 --2- 172.21.15.43:0/1185313908 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f7a1c15c3a0 0x7f7a1c1608a0 secure :-1 s=READY pgs=1871 cs=0 l=1 rev1=1 crypto rx=0x7f7a180097a0 tx=0x7f7a1802b820 comp rx=0 tx=0).stop 2023-08-09T14:51:50.071 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:50.068+0000 7f7a22e0f640 1 -- 172.21.15.43:0/1185313908 shutdown_connections 2023-08-09T14:51:50.071 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:50.068+0000 7f7a22e0f640 1 --2- 172.21.15.43:0/1185313908 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f7a1c0cd600 0x7f7a1c0d0580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T14:51:50.071 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:50.068+0000 7f7a22e0f640 1 --2- 172.21.15.43:0/1185313908 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f7a1c15c3a0 0x7f7a1c1608a0 unknown :-1 s=CLOSED pgs=1871 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T14:51:50.071 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:50.068+0000 7f7a22e0f640 1 -- 172.21.15.43:0/1185313908 >> 172.21.15.43:0/1185313908 conn(0x7f7a1c06a210 msgr2=0x7f7a1c06a620 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-09T14:51:50.071 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:50.068+0000 7f7a22e0f640 1 -- 172.21.15.43:0/1185313908 shutdown_connections 2023-08-09T14:51:50.071 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:50.069+0000 7f7a22e0f640 1 -- 172.21.15.43:0/1185313908 wait complete. 2023-08-09T14:51:50.072 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:50.070+0000 7f7a22e0f640 1 Processor -- start 2023-08-09T14:51:50.072 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:50.070+0000 7f7a22e0f640 1 -- start start 2023-08-09T14:51:50.073 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:50.071+0000 7f7a22e0f640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f7a1c0b0f00 0x7f7a1c150610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T14:51:50.073 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:50.071+0000 7f7a22e0f640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f7a1c0cd600 0x7f7a1c0b1430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T14:51:50.073 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:50.071+0000 7f7a22e0f640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f7a1c15c3a0 0x7f7a1c15d2c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T14:51:50.074 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:50.071+0000 7f7a22e0f640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7a1c0b23b0 con 0x7f7a1c15c3a0 2023-08-09T14:51:50.074 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:50.071+0000 7f7a22e0f640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7a1c0b1df0 con 0x7f7a1c0cd600 2023-08-09T14:51:50.074 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:50.071+0000 7f7a22e0f640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7a1c0b20d0 con 0x7f7a1c0b0f00 2023-08-09T14:51:50.074 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:50.072+0000 7f7a2260e640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f7a1c15c3a0 0x7f7a1c15d2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T14:51:50.074 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:50.072+0000 7f7a2260e640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f7a1c15c3a0 0x7f7a1c15d2c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:57898/0 (socket says 172.21.15.43:57898) 2023-08-09T14:51:50.074 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:50.072+0000 7f7a2260e640 1 -- 172.21.15.43:0/3962930523 learned_addr learned my addr 172.21.15.43:0/3962930523 (peer_addr_for_me v2:172.21.15.43:0/0) 2023-08-09T14:51:50.075 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:50.072+0000 7f7a21e0d640 1 --2- 172.21.15.43:0/3962930523 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f7a1c0b0f00 0x7f7a1c150610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T14:51:50.076 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:50.072+0000 7f7a2160c640 1 --2- 172.21.15.43:0/3962930523 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f7a1c0cd600 0x7f7a1c0b1430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T14:51:50.076 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:50.072+0000 7f7a2260e640 1 -- 172.21.15.43:0/3962930523 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f7a1c0b0f00 msgr2=0x7f7a1c150610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T14:51:50.076 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:50.072+0000 7f7a2260e640 1 --2- 172.21.15.43:0/3962930523 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f7a1c0b0f00 0x7f7a1c150610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T14:51:50.076 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:50.072+0000 7f7a2260e640 1 -- 172.21.15.43:0/3962930523 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f7a1c0cd600 msgr2=0x7f7a1c0b1430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T14:51:50.077 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:50.072+0000 7f7a2260e640 1 --2- 172.21.15.43:0/3962930523 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f7a1c0cd600 0x7f7a1c0b1430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T14:51:50.077 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:50.072+0000 7f7a2260e640 1 -- 172.21.15.43:0/3962930523 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7a18009400 con 0x7f7a1c15c3a0 2023-08-09T14:51:50.077 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:50.072+0000 7f7a2160c640 1 --2- 172.21.15.43:0/3962930523 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f7a1c0cd600 0x7f7a1c0b1430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-08-09T14:51:50.077 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:50.073+0000 7f7a2260e640 1 --2- 172.21.15.43:0/3962930523 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f7a1c15c3a0 0x7f7a1c15d2c0 secure :-1 s=READY pgs=1872 cs=0 l=1 rev1=1 crypto rx=0x7f7a14009fd0 tx=0x7f7a1400e4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-09T14:51:50.077 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:50.073+0000 7f7a127fc640 1 -- 172.21.15.43:0/3962930523 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7a14014420 con 0x7f7a1c15c3a0 2023-08-09T14:51:50.077 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:50.073+0000 7f7a127fc640 1 -- 172.21.15.43:0/3962930523 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7a14014580 con 0x7f7a1c15c3a0 2023-08-09T14:51:50.077 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:50.073+0000 7f7a127fc640 1 -- 172.21.15.43:0/3962930523 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7a14014930 con 0x7f7a1c15c3a0 2023-08-09T14:51:50.077 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:50.073+0000 7f7a22e0f640 1 -- 172.21.15.43:0/3962930523 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7a1c0b2700 con 0x7f7a1c15c3a0 2023-08-09T14:51:50.078 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:50.073+0000 7f7a22e0f640 1 -- 172.21.15.43:0/3962930523 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7a1c150b50 con 0x7f7a1c15c3a0 2023-08-09T14:51:50.078 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:50.074+0000 7f7a22e0f640 1 -- 172.21.15.43:0/3962930523 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7a1c0d1fb0 con 0x7f7a1c15c3a0 2023-08-09T14:51:50.078 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:50.075+0000 7f7a127fc640 1 -- 172.21.15.43:0/3962930523 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 79) v1 ==== 88443+0+0 (secure 0 0 0) 0x7f7a14014a90 con 0x7f7a1c15c3a0 2023-08-09T14:51:50.078 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:50.076+0000 7f7a127fc640 1 --2- 172.21.15.43:0/3962930523 >> [v2:172.21.15.203:6832/2566816332,v1:172.21.15.203:6833/2566816332] conn(0x7f7a0006a1a0 0x7f7a0006c660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T14:51:50.079 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:50.076+0000 7f7a127fc640 1 -- 172.21.15.43:0/3962930523 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(172..172 src has 1..172) v4 ==== 6426+0+0 (secure 0 0 0) 0x7f7a14089210 con 0x7f7a1c15c3a0 2023-08-09T14:51:50.079 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:50.077+0000 7f7a21e0d640 1 --2- 172.21.15.43:0/3962930523 >> [v2:172.21.15.203:6832/2566816332,v1:172.21.15.203:6833/2566816332] conn(0x7f7a0006a1a0 0x7f7a0006c660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T14:51:50.079 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:50.078+0000 7f7a21e0d640 1 --2- 172.21.15.43:0/3962930523 >> [v2:172.21.15.203:6832/2566816332,v1:172.21.15.203:6833/2566816332] conn(0x7f7a0006a1a0 0x7f7a0006c660 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f7a18036000 tx=0x7f7a18008030 comp rx=0 tx=0).ready entity=mgr.7765 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-09T14:51:50.082 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:50.080+0000 7f7a127fc640 1 -- 172.21.15.43:0/3962930523 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178066 (secure 0 0 0) 0x7f7a14058070 con 0x7f7a1c15c3a0 2023-08-09T14:51:50.316 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:50.314+0000 7f7a22e0f640 1 -- 172.21.15.43:0/3962930523 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"} v 0) v1 -- 0x7f7a1c0b2c20 con 0x7f7a1c15c3a0 2023-08-09T14:51:50.818 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:50.816+0000 7f7a127fc640 1 -- 172.21.15.43:0/3962930523 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"}]=0 v78) v1 ==== 122+0+0 (secure 0 0 0) 0x7f7a1c0b2c20 con 0x7f7a1c15c3a0 2023-08-09T14:51:50.823 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:50.821+0000 7f7a22e0f640 1 -- 172.21.15.43:0/3962930523 >> [v2:172.21.15.203:6832/2566816332,v1:172.21.15.203:6833/2566816332] conn(0x7f7a0006a1a0 msgr2=0x7f7a0006c660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T14:51:50.823 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:50.821+0000 7f7a22e0f640 1 --2- 172.21.15.43:0/3962930523 >> [v2:172.21.15.203:6832/2566816332,v1:172.21.15.203:6833/2566816332] conn(0x7f7a0006a1a0 0x7f7a0006c660 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f7a18036000 tx=0x7f7a18008030 comp rx=0 tx=0).stop 2023-08-09T14:51:50.824 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:50.821+0000 7f7a22e0f640 1 -- 172.21.15.43:0/3962930523 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f7a1c15c3a0 msgr2=0x7f7a1c15d2c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T14:51:50.824 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:50.821+0000 7f7a22e0f640 1 --2- 172.21.15.43:0/3962930523 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f7a1c15c3a0 0x7f7a1c15d2c0 secure :-1 s=READY pgs=1872 cs=0 l=1 rev1=1 crypto rx=0x7f7a14009fd0 tx=0x7f7a1400e4b0 comp rx=0 tx=0).stop 2023-08-09T14:51:50.824 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:50.821+0000 7f7a22e0f640 1 -- 172.21.15.43:0/3962930523 shutdown_connections 2023-08-09T14:51:50.825 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:50.821+0000 7f7a22e0f640 1 --2- 172.21.15.43:0/3962930523 >> [v2:172.21.15.203:6832/2566816332,v1:172.21.15.203:6833/2566816332] conn(0x7f7a0006a1a0 0x7f7a0006c660 unknown :-1 s=CLOSED pgs=62 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T14:51:50.825 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:50.821+0000 7f7a22e0f640 1 --2- 172.21.15.43:0/3962930523 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f7a1c15c3a0 0x7f7a1c15d2c0 unknown :-1 s=CLOSED pgs=1872 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T14:51:50.825 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:50.821+0000 7f7a22e0f640 1 --2- 172.21.15.43:0/3962930523 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f7a1c0cd600 0x7f7a1c0b1430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T14:51:50.825 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:50.821+0000 7f7a22e0f640 1 --2- 172.21.15.43:0/3962930523 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f7a1c0b0f00 0x7f7a1c150610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T14:51:50.825 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:50.821+0000 7f7a22e0f640 1 -- 172.21.15.43:0/3962930523 >> 172.21.15.43:0/3962930523 conn(0x7f7a1c06a210 msgr2=0x7f7a1c160710 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-09T14:51:50.825 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:50.821+0000 7f7a22e0f640 1 -- 172.21.15.43:0/3962930523 shutdown_connections 2023-08-09T14:51:50.826 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:50.821+0000 7f7a22e0f640 1 -- 172.21.15.43:0/3962930523 wait complete. 2023-08-09T14:51:50.847 DEBUG:tasks.cephfs.filesystem:Not Creating any subvols for filesystem 'cephfs' 2023-08-09T14:51:50.847 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-08-09T14:51:51.078 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:51.076+0000 7f1e592ad640 1 Processor -- start 2023-08-09T14:51:51.078 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:51.077+0000 7f1e592ad640 1 -- start start 2023-08-09T14:51:51.080 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:51.078+0000 7f1e592ad640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f1e54142ca0 0x7f1e540c96e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T14:51:51.080 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:51.078+0000 7f1e592ad640 1 -- --> v1:172.21.15.43:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1e54143080 con 0x7f1e5415ce40 2023-08-09T14:51:51.080 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:51.078+0000 7f1e592ad640 1 -- --> v1:172.21.15.203:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1e5406d400 con 0x7f1e540cd510 2023-08-09T14:51:51.080 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:51.078+0000 7f1e592ad640 1 -- --> v2:172.21.15.43:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1e540c9d90 con 0x7f1e54142ca0 2023-08-09T14:51:51.081 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:51.079+0000 7f1e4bfff640 1 --1- >> v1:172.21.15.43:6789/0 conn(0x7f1e5415ce40 0x7f1e540cce00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.43:6789/0 says I am v1:172.21.15.43:47302/0 (socket says 172.21.15.43:47302) 2023-08-09T14:51:51.081 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:51.079+0000 7f1e4bfff640 1 -- 172.21.15.43:0/3875576918 learned_addr learned my addr 172.21.15.43:0/3875576918 (peer_addr_for_me v1:172.21.15.43:0/0) 2023-08-09T14:51:51.081 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:51.079+0000 7f1e52ffd640 1 --2- 172.21.15.43:0/3875576918 >> v2:172.21.15.43:3300/0 conn(0x7f1e54142ca0 0x7f1e540c96e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T14:51:51.082 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:51.080+0000 7f1e52ffd640 1 -- 172.21.15.43:0/3875576918 >> v1:172.21.15.203:6789/0 conn(0x7f1e540cd510 legacy=0x7f1e540c3290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-09T14:51:51.082 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:51.079+0000 7f1e527fc640 1 -- 172.21.15.43:0/3875576918 <== mon.0 v1:172.21.15.43:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2608417198 0 0) 0x7f1e54143080 con 0x7f1e5415ce40 2023-08-09T14:51:51.082 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:51.080+0000 7f1e52ffd640 1 -- 172.21.15.43:0/3875576918 >> v1:172.21.15.43:6789/0 conn(0x7f1e5415ce40 legacy=0x7f1e540cce00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T14:51:51.082 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:51.080+0000 7f1e52ffd640 1 -- 172.21.15.43:0/3875576918 --> v2:172.21.15.43:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1e540ca070 con 0x7f1e54142ca0 2023-08-09T14:51:51.083 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:51.081+0000 7f1e52ffd640 1 --2- 172.21.15.43:0/3875576918 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f1e54142ca0 0x7f1e540c96e0 secure :-1 s=READY pgs=1874 cs=0 l=1 rev1=1 crypto rx=0x7f1e440097a0 tx=0x7f1e4402b960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=eabc6c0845c3153a server_cookie=0 in_seq=0 out_seq=0 2023-08-09T14:51:51.083 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:51.081+0000 7f1e527fc640 1 -- 172.21.15.43:0/3875576918 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1e44030610 con 0x7f1e54142ca0 2023-08-09T14:51:51.083 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:51.081+0000 7f1e527fc640 1 -- 172.21.15.43:0/3875576918 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1e44030770 con 0x7f1e54142ca0 2023-08-09T14:51:51.083 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:51.081+0000 7f1e527fc640 1 -- 172.21.15.43:0/3875576918 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1e44030b00 con 0x7f1e54142ca0 2023-08-09T14:51:51.084 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:51.082+0000 7f1e592ad640 1 -- 172.21.15.43:0/3875576918 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f1e54142ca0 msgr2=0x7f1e540c96e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T14:51:51.084 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:51.082+0000 7f1e592ad640 1 --2- 172.21.15.43:0/3875576918 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f1e54142ca0 0x7f1e540c96e0 secure :-1 s=READY pgs=1874 cs=0 l=1 rev1=1 crypto rx=0x7f1e440097a0 tx=0x7f1e4402b960 comp rx=0 tx=0).stop 2023-08-09T14:51:51.084 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:51.082+0000 7f1e592ad640 1 -- 172.21.15.43:0/3875576918 shutdown_connections 2023-08-09T14:51:51.084 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:51.082+0000 7f1e592ad640 1 --2- 172.21.15.43:0/3875576918 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f1e54142ca0 0x7f1e540c96e0 unknown :-1 s=CLOSED pgs=1874 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T14:51:51.084 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:51.082+0000 7f1e592ad640 1 -- 172.21.15.43:0/3875576918 >> 172.21.15.43:0/3875576918 conn(0x7f1e540ce0a0 msgr2=0x7f1e540ce480 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-09T14:51:51.085 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:51.082+0000 7f1e592ad640 1 -- 172.21.15.43:0/3875576918 shutdown_connections 2023-08-09T14:51:51.085 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:51.082+0000 7f1e592ad640 1 -- 172.21.15.43:0/3875576918 wait complete. 2023-08-09T14:51:51.085 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:51.084+0000 7f1e592ad640 1 Processor -- start 2023-08-09T14:51:51.085 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:51.084+0000 7f1e592ad640 1 -- start start 2023-08-09T14:51:51.086 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:51.085+0000 7f1e592ad640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f1e540cd510 0x7f1e5413e400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T14:51:51.086 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:51.085+0000 7f1e592ad640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f1e54142ca0 0x7f1e5415b0f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T14:51:51.087 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:51.085+0000 7f1e592ad640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f1e5415ce40 0x7f1e5406c020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T14:51:51.087 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:51.085+0000 7f1e592ad640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1e5415a370 con 0x7f1e540cd510 2023-08-09T14:51:51.087 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:51.085+0000 7f1e592ad640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1e5415a610 con 0x7f1e5415ce40 2023-08-09T14:51:51.087 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:51.085+0000 7f1e592ad640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1e5415a8f0 con 0x7f1e54142ca0 2023-08-09T14:51:51.087 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:51.085+0000 7f1e52ffd640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f1e540cd510 0x7f1e5413e400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T14:51:51.087 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:51.085+0000 7f1e52ffd640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f1e540cd510 0x7f1e5413e400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:57914/0 (socket says 172.21.15.43:57914) 2023-08-09T14:51:51.087 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:51.085+0000 7f1e52ffd640 1 -- 172.21.15.43:0/3184598072 learned_addr learned my addr 172.21.15.43:0/3184598072 (peer_addr_for_me v2:172.21.15.43:0/0) 2023-08-09T14:51:51.088 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:51.085+0000 7f1e52ffd640 1 -- 172.21.15.43:0/3184598072 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f1e54142ca0 msgr2=0x7f1e5415b0f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T14:51:51.088 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:51.085+0000 7f1e52ffd640 1 --2- 172.21.15.43:0/3184598072 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f1e54142ca0 0x7f1e5415b0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T14:51:51.089 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:51.085+0000 7f1e52ffd640 1 -- 172.21.15.43:0/3184598072 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f1e5415ce40 msgr2=0x7f1e5406c020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T14:51:51.089 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:51.085+0000 7f1e52ffd640 1 --2- 172.21.15.43:0/3184598072 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f1e5415ce40 0x7f1e5406c020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T14:51:51.089 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:51.086+0000 7f1e52ffd640 1 -- 172.21.15.43:0/3184598072 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1e44009400 con 0x7f1e540cd510 2023-08-09T14:51:51.089 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:51.086+0000 7f1e52ffd640 1 --2- 172.21.15.43:0/3184598072 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f1e540cd510 0x7f1e5413e400 secure :-1 s=READY pgs=1875 cs=0 l=1 rev1=1 crypto rx=0x7f1e440098c0 tx=0x7f1e44004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-09T14:51:51.089 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:51.086+0000 7f1e4b7fe640 1 -- 172.21.15.43:0/3184598072 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1e4403e040 con 0x7f1e540cd510 2023-08-09T14:51:51.089 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:51.086+0000 7f1e4b7fe640 1 -- 172.21.15.43:0/3184598072 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1e44039cb0 con 0x7f1e540cd510 2023-08-09T14:51:51.089 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:51.086+0000 7f1e4b7fe640 1 -- 172.21.15.43:0/3184598072 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1e44044630 con 0x7f1e540cd510 2023-08-09T14:51:51.090 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:51.086+0000 7f1e592ad640 1 -- 172.21.15.43:0/3184598072 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1e5415abd0 con 0x7f1e540cd510 2023-08-09T14:51:51.090 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:51.086+0000 7f1e592ad640 1 -- 172.21.15.43:0/3184598072 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1e5413f580 con 0x7f1e540cd510 2023-08-09T14:51:51.090 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:51.088+0000 7f1e592ad640 1 -- 172.21.15.43:0/3184598072 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1e540739f0 con 0x7f1e540cd510 2023-08-09T14:51:51.090 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:51.088+0000 7f1e4b7fe640 1 -- 172.21.15.43:0/3184598072 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 79) v1 ==== 88443+0+0 (secure 0 0 0) 0x7f1e44043390 con 0x7f1e540cd510 2023-08-09T14:51:51.091 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:51.089+0000 7f1e4b7fe640 1 --2- 172.21.15.43:0/3184598072 >> [v2:172.21.15.203:6832/2566816332,v1:172.21.15.203:6833/2566816332] conn(0x7f1e3406a170 0x7f1e3406c630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T14:51:51.091 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:51.089+0000 7f1e4b7fe640 1 -- 172.21.15.43:0/3184598072 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(172..172 src has 1..172) v4 ==== 6426+0+0 (secure 0 0 0) 0x7f1e440aacf0 con 0x7f1e540cd510 2023-08-09T14:51:51.091 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:51.090+0000 7f1e4bfff640 1 --2- 172.21.15.43:0/3184598072 >> [v2:172.21.15.203:6832/2566816332,v1:172.21.15.203:6833/2566816332] conn(0x7f1e3406a170 0x7f1e3406c630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T14:51:51.093 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:51.091+0000 7f1e4bfff640 1 --2- 172.21.15.43:0/3184598072 >> [v2:172.21.15.203:6832/2566816332,v1:172.21.15.203:6833/2566816332] conn(0x7f1e3406a170 0x7f1e3406c630 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f1e3c000c00 tx=0x7f1e3c0079c0 comp rx=0 tx=0).ready entity=mgr.7765 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-09T14:51:51.095 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:51.094+0000 7f1e4b7fe640 1 -- 172.21.15.43:0/3184598072 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178066 (secure 0 0 0) 0x7f1e44076040 con 0x7f1e540cd510 2023-08-09T14:51:51.321 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:51.319+0000 7f1e592ad640 1 -- 172.21.15.43:0/3184598072 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f1e5415ad90 con 0x7f1e540cd510 2023-08-09T14:51:51.322 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:51.321+0000 7f1e4b7fe640 1 -- 172.21.15.43:0/3184598072 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 78 v78) v1 ==== 94+0+4880 (secure 0 0 0) 0x7f1e44079660 con 0x7f1e540cd510 2023-08-09T14:51:51.323 INFO:teuthology.orchestra.run.smithi043.stdout: 2023-08-09T14:51:51.323 INFO:teuthology.orchestra.run.smithi043.stdout:{"epoch":78,"default_fscid":9,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":7079,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.203:6836/807059349","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6834","nonce":807059349},{"type":"v1","addr":"172.21.15.203:6836","nonce":807059349}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":74},{"gid":7924,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.203:6837/3166829033","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6835","nonce":3166829033},{"type":"v1","addr":"172.21.15.203:6837","nonce":3166829033}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":74},{"gid":9642,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.43:6833/3803974919","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6832","nonce":3803974919},{"type":"v1","addr":"172.21.15.43:6833","nonce":3803974919}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":73}],"filesystems":[{"mdsmap":{"epoch":78,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-08-09T14:51:47.253704+0000","modified":"2023-08-09T14:51:50.806331+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":9648},"failed":[],"damaged":[],"stopped":[],"info":{"gid_9648":{"gid":9648,"name":"a","rank":0,"incarnation":75,"state":"up:active","state_seq":3,"addr":"172.21.15.43:6835/2435237803","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6834","nonce":2435237803},{"type":"v1","addr":"172.21.15.43:6835","nonce":2435237803}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[21],"metadata_pool":20,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":9}]} 2023-08-09T14:51:51.323 INFO:teuthology.orchestra.run.smithi043.stderr:dumped fsmap epoch 78 2023-08-09T14:51:51.325 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:51.324+0000 7f1e592ad640 1 -- 172.21.15.43:0/3184598072 >> [v2:172.21.15.203:6832/2566816332,v1:172.21.15.203:6833/2566816332] conn(0x7f1e3406a170 msgr2=0x7f1e3406c630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T14:51:51.325 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:51.324+0000 7f1e592ad640 1 --2- 172.21.15.43:0/3184598072 >> [v2:172.21.15.203:6832/2566816332,v1:172.21.15.203:6833/2566816332] conn(0x7f1e3406a170 0x7f1e3406c630 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f1e3c000c00 tx=0x7f1e3c0079c0 comp rx=0 tx=0).stop 2023-08-09T14:51:51.326 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:51.324+0000 7f1e592ad640 1 -- 172.21.15.43:0/3184598072 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f1e540cd510 msgr2=0x7f1e5413e400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T14:51:51.326 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:51.324+0000 7f1e592ad640 1 --2- 172.21.15.43:0/3184598072 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f1e540cd510 0x7f1e5413e400 secure :-1 s=READY pgs=1875 cs=0 l=1 rev1=1 crypto rx=0x7f1e440098c0 tx=0x7f1e44004730 comp rx=0 tx=0).stop 2023-08-09T14:51:51.326 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:51.324+0000 7f1e592ad640 1 -- 172.21.15.43:0/3184598072 shutdown_connections 2023-08-09T14:51:51.326 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:51.324+0000 7f1e592ad640 1 --2- 172.21.15.43:0/3184598072 >> [v2:172.21.15.203:6832/2566816332,v1:172.21.15.203:6833/2566816332] conn(0x7f1e3406a170 0x7f1e3406c630 unknown :-1 s=CLOSED pgs=63 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T14:51:51.326 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:51.324+0000 7f1e592ad640 1 --2- 172.21.15.43:0/3184598072 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f1e5415ce40 0x7f1e5406c020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T14:51:51.326 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:51.324+0000 7f1e592ad640 1 --2- 172.21.15.43:0/3184598072 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f1e54142ca0 0x7f1e5415b0f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T14:51:51.327 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:51.324+0000 7f1e592ad640 1 --2- 172.21.15.43:0/3184598072 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f1e540cd510 0x7f1e5413e400 unknown :-1 s=CLOSED pgs=1875 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T14:51:51.327 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:51.324+0000 7f1e592ad640 1 -- 172.21.15.43:0/3184598072 >> 172.21.15.43:0/3184598072 conn(0x7f1e540ce0a0 msgr2=0x7f1e5406f520 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-09T14:51:51.327 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:51.324+0000 7f1e592ad640 1 -- 172.21.15.43:0/3184598072 shutdown_connections 2023-08-09T14:51:51.327 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:51.324+0000 7f1e592ad640 1 -- 172.21.15.43:0/3184598072 wait complete. 2023-08-09T14:51:51.344 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-08-09T14:51:51.561 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:51.559+0000 7f4446574640 1 Processor -- start 2023-08-09T14:51:51.561 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:51.559+0000 7f4446574640 1 -- start start 2023-08-09T14:51:51.563 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:51.561+0000 7f4446574640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f444015c780 0x7f4440167440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T14:51:51.563 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:51.561+0000 7f4446574640 1 -- --> v1:172.21.15.43:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f44400c8680 con 0x7f4440160cb0 2023-08-09T14:51:51.563 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:51.561+0000 7f4446574640 1 -- --> v1:172.21.15.203:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f44400c8960 con 0x7f4440160380 2023-08-09T14:51:51.563 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:51.561+0000 7f4446574640 1 -- --> v2:172.21.15.43:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f44400ca7d0 con 0x7f444015c780 2023-08-09T14:51:51.564 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:51.562+0000 7f4445d73640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f444015c780 0x7f4440167440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T14:51:51.564 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:51.562+0000 7f4444d71640 1 --1- >> v1:172.21.15.43:6789/0 conn(0x7f4440160cb0 0x7f444015c070 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.43:6789/0 says I am v1:172.21.15.43:47308/0 (socket says 172.21.15.43:47308) 2023-08-09T14:51:51.564 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:51.562+0000 7f4444d71640 1 -- 172.21.15.43:0/3335604996 learned_addr learned my addr 172.21.15.43:0/3335604996 (peer_addr_for_me v1:172.21.15.43:0/0) 2023-08-09T14:51:51.564 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:51.562+0000 7f4445d73640 1 -- 172.21.15.43:0/3335604996 >> v1:172.21.15.203:6789/0 conn(0x7f4440160380 legacy=0x7f44400ca0c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T14:51:51.564 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:51.562+0000 7f4445d73640 1 -- 172.21.15.43:0/3335604996 >> v1:172.21.15.43:6789/0 conn(0x7f4440160cb0 legacy=0x7f444015c070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T14:51:51.565 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:51.562+0000 7f4445d73640 1 -- 172.21.15.43:0/3335604996 --> v2:172.21.15.43:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f44400aaf00 con 0x7f444015c780 2023-08-09T14:51:51.565 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:51.564+0000 7f4445d73640 1 --2- 172.21.15.43:0/3335604996 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f444015c780 0x7f4440167440 secure :-1 s=READY pgs=1877 cs=0 l=1 rev1=1 crypto rx=0x7f4438009d90 tx=0x7f443802bfd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f2589e60e6ad35c3 server_cookie=0 in_seq=0 out_seq=0 2023-08-09T14:51:51.565 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:51.564+0000 7f4437fff640 1 -- 172.21.15.43:0/3335604996 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f443802cd80 con 0x7f444015c780 2023-08-09T14:51:51.566 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:51.564+0000 7f4437fff640 1 -- 172.21.15.43:0/3335604996 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4438034440 con 0x7f444015c780 2023-08-09T14:51:51.566 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:51.564+0000 7f4437fff640 1 -- 172.21.15.43:0/3335604996 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f44380347d0 con 0x7f444015c780 2023-08-09T14:51:51.566 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:51.564+0000 7f4446574640 1 -- 172.21.15.43:0/3335604996 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f444015c780 msgr2=0x7f4440167440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T14:51:51.566 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:51.564+0000 7f4446574640 1 --2- 172.21.15.43:0/3335604996 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f444015c780 0x7f4440167440 secure :-1 s=READY pgs=1877 cs=0 l=1 rev1=1 crypto rx=0x7f4438009d90 tx=0x7f443802bfd0 comp rx=0 tx=0).stop 2023-08-09T14:51:51.567 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:51.565+0000 7f4446574640 1 -- 172.21.15.43:0/3335604996 shutdown_connections 2023-08-09T14:51:51.567 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:51.565+0000 7f4446574640 1 --2- 172.21.15.43:0/3335604996 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f444015c780 0x7f4440167440 unknown :-1 s=CLOSED pgs=1877 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T14:51:51.567 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:51.565+0000 7f4446574640 1 -- 172.21.15.43:0/3335604996 >> 172.21.15.43:0/3335604996 conn(0x7f44400cde90 msgr2=0x7f44400ce2a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-09T14:51:51.567 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:51.565+0000 7f4446574640 1 -- 172.21.15.43:0/3335604996 shutdown_connections 2023-08-09T14:51:51.567 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:51.565+0000 7f4446574640 1 -- 172.21.15.43:0/3335604996 wait complete. 2023-08-09T14:51:51.568 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:51.567+0000 7f4446574640 1 Processor -- start 2023-08-09T14:51:51.568 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:51.567+0000 7f4446574640 1 -- start start 2023-08-09T14:51:51.569 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:51.568+0000 7f4446574640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f444015c780 0x7f444009a750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T14:51:51.569 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:51.568+0000 7f4446574640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f4440160380 0x7f4440098c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T14:51:51.570 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:51.568+0000 7f4446574640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f4440160cb0 0x7f4440145c30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T14:51:51.570 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:51.568+0000 7f4446574640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f444015aa60 con 0x7f444015c780 2023-08-09T14:51:51.570 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:51.568+0000 7f4446574640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f444009ae10 con 0x7f4440160cb0 2023-08-09T14:51:51.570 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:51.568+0000 7f4446574640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f444009b0f0 con 0x7f4440160380 2023-08-09T14:51:51.570 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:51.568+0000 7f4445572640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f444015c780 0x7f444009a750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T14:51:51.570 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:51.568+0000 7f4445572640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f444015c780 0x7f444009a750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:57920/0 (socket says 172.21.15.43:57920) 2023-08-09T14:51:51.570 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:51.568+0000 7f4445572640 1 -- 172.21.15.43:0/3116270543 learned_addr learned my addr 172.21.15.43:0/3116270543 (peer_addr_for_me v2:172.21.15.43:0/0) 2023-08-09T14:51:51.571 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:51.569+0000 7f4445572640 1 -- 172.21.15.43:0/3116270543 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f4440160380 msgr2=0x7f4440098c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T14:51:51.572 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:51.569+0000 7f4445572640 1 --2- 172.21.15.43:0/3116270543 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f4440160380 0x7f4440098c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T14:51:51.572 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:51.569+0000 7f4445572640 1 -- 172.21.15.43:0/3116270543 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f4440160cb0 msgr2=0x7f4440145c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T14:51:51.572 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:51.569+0000 7f4445d73640 1 --2- 172.21.15.43:0/3116270543 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f4440160cb0 0x7f4440145c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T14:51:51.572 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:51.569+0000 7f4445572640 1 --2- 172.21.15.43:0/3116270543 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f4440160cb0 0x7f4440145c30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T14:51:51.572 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:51.569+0000 7f4445572640 1 -- 172.21.15.43:0/3116270543 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4438009a70 con 0x7f444015c780 2023-08-09T14:51:51.572 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:51.569+0000 7f4445572640 1 --2- 172.21.15.43:0/3116270543 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f444015c780 0x7f444009a750 secure :-1 s=READY pgs=1878 cs=0 l=1 rev1=1 crypto rx=0x7f443c00c450 tx=0x7f443c00c910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-09T14:51:51.572 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:51.569+0000 7f4435ffb640 1 -- 172.21.15.43:0/3116270543 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f443c0052d0 con 0x7f444015c780 2023-08-09T14:51:51.573 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:51.569+0000 7f4435ffb640 1 -- 172.21.15.43:0/3116270543 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f443c005430 con 0x7f444015c780 2023-08-09T14:51:51.573 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:51.569+0000 7f4435ffb640 1 -- 172.21.15.43:0/3116270543 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f443c012870 con 0x7f444015c780 2023-08-09T14:51:51.573 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:51.569+0000 7f4446574640 1 -- 172.21.15.43:0/3116270543 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f444009b3d0 con 0x7f444015c780 2023-08-09T14:51:51.573 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:51.570+0000 7f4446574640 1 -- 172.21.15.43:0/3116270543 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f444009b990 con 0x7f444015c780 2023-08-09T14:51:51.573 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:51.571+0000 7f4446574640 1 -- 172.21.15.43:0/3116270543 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f44400739f0 con 0x7f444015c780 2023-08-09T14:51:51.574 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:51.572+0000 7f4435ffb640 1 -- 172.21.15.43:0/3116270543 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 79) v1 ==== 88443+0+0 (secure 0 0 0) 0x7f443c0129d0 con 0x7f444015c780 2023-08-09T14:51:51.574 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:51.572+0000 7f4435ffb640 1 --2- 172.21.15.43:0/3116270543 >> [v2:172.21.15.203:6832/2566816332,v1:172.21.15.203:6833/2566816332] conn(0x7f442406a1a0 0x7f442406c660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T14:51:51.574 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:51.573+0000 7f4435ffb640 1 -- 172.21.15.43:0/3116270543 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(172..172 src has 1..172) v4 ==== 6426+0+0 (secure 0 0 0) 0x7f443c088420 con 0x7f444015c780 2023-08-09T14:51:51.575 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:51.573+0000 7f4444d71640 1 --2- 172.21.15.43:0/3116270543 >> [v2:172.21.15.203:6832/2566816332,v1:172.21.15.203:6833/2566816332] conn(0x7f442406a1a0 0x7f442406c660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T14:51:51.575 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:51.574+0000 7f4444d71640 1 --2- 172.21.15.43:0/3116270543 >> [v2:172.21.15.203:6832/2566816332,v1:172.21.15.203:6833/2566816332] conn(0x7f442406a1a0 0x7f442406c660 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f4430005880 tx=0x7f4430007a00 comp rx=0 tx=0).ready entity=mgr.7765 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-09T14:51:51.578 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:51.577+0000 7f4435ffb640 1 -- 172.21.15.43:0/3116270543 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178066 (secure 0 0 0) 0x7f443c0537f0 con 0x7f444015c780 2023-08-09T14:51:51.772 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:51.770+0000 7f4446574640 1 -- 172.21.15.43:0/3116270543 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f44401465a0 con 0x7f444015c780 2023-08-09T14:51:51.774 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:51.773+0000 7f4435ffb640 1 -- 172.21.15.43:0/3116270543 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v172) v1 ==== 74+0+18566 (secure 0 0 0) 0x7f443c056e10 con 0x7f444015c780 2023-08-09T14:51:51.775 INFO:teuthology.orchestra.run.smithi043.stdout: 2023-08-09T14:51:51.775 INFO:teuthology.orchestra.run.smithi043.stdout:{"epoch":172,"fsid":"daac87ee-1337-4250-a955-aea4e408a39b","created":"2023-08-09T13:33:41.701198+0000","modified":"2023-08-09T14:51:48.756254+0000","last_up_change":"2023-08-09T13:33:53.783453+0000","last_in_change":"2023-08-09T13:33:45.148904+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":21,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":"rbd","create_time":"2023-08-09T13:33:54.365137+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":".mgr","create_time":"2023-08-09T13:33:54.831803+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":20,"pool_name":"cephfs_metadata","create_time":"2023-08-09T14:51:45.692613+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"171","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":21,"pool_name":"cephfs_data","create_time":"2023-08-09T14:51:46.286715+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"172","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"cbd49ecd-e896-4fbd-864e-5c37a829b165","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":170,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6800","nonce":502667335},{"type":"v1","addr":"172.21.15.43:6801","nonce":502667335}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6802","nonce":502667335},{"type":"v1","addr":"172.21.15.43:6803","nonce":502667335}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6806","nonce":502667335},{"type":"v1","addr":"172.21.15.43:6807","nonce":502667335}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6804","nonce":502667335},{"type":"v1","addr":"172.21.15.43:6805","nonce":502667335}]},"public_addr":"172.21.15.43:6801/502667335","cluster_addr":"172.21.15.43:6803/502667335","heartbeat_back_addr":"172.21.15.43:6807/502667335","heartbeat_front_addr":"172.21.15.43:6805/502667335","state":["exists","up"]},{"osd":1,"uuid":"75c7e0fe-1ec7-4fe8-a2f2-3661a65b8500","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":170,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6808","nonce":3289134610},{"type":"v1","addr":"172.21.15.43:6809","nonce":3289134610}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6810","nonce":3289134610},{"type":"v1","addr":"172.21.15.43:6812","nonce":3289134610}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6821","nonce":3289134610},{"type":"v1","addr":"172.21.15.43:6824","nonce":3289134610}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6815","nonce":3289134610},{"type":"v1","addr":"172.21.15.43:6818","nonce":3289134610}]},"public_addr":"172.21.15.43:6809/3289134610","cluster_addr":"172.21.15.43:6812/3289134610","heartbeat_back_addr":"172.21.15.43:6824/3289134610","heartbeat_front_addr":"172.21.15.43:6818/3289134610","state":["exists","up"]},{"osd":2,"uuid":"bb5c41e9-7f5f-4125-9b01-3bb8bfc2ea83","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":170,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6811","nonce":4079709736},{"type":"v1","addr":"172.21.15.43:6814","nonce":4079709736}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6817","nonce":4079709736},{"type":"v1","addr":"172.21.15.43:6820","nonce":4079709736}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6829","nonce":4079709736},{"type":"v1","addr":"172.21.15.43:6831","nonce":4079709736}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6823","nonce":4079709736},{"type":"v1","addr":"172.21.15.43:6826","nonce":4079709736}]},"public_addr":"172.21.15.43:6814/4079709736","cluster_addr":"172.21.15.43:6820/4079709736","heartbeat_back_addr":"172.21.15.43:6831/4079709736","heartbeat_front_addr":"172.21.15.43:6826/4079709736","state":["exists","up"]},{"osd":3,"uuid":"bfe4c8b4-694a-465c-87e6-2b2aab852613","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":170,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6813","nonce":1709538759},{"type":"v1","addr":"172.21.15.43:6816","nonce":1709538759}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6819","nonce":1709538759},{"type":"v1","addr":"172.21.15.43:6822","nonce":1709538759}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6828","nonce":1709538759},{"type":"v1","addr":"172.21.15.43:6830","nonce":1709538759}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6825","nonce":1709538759},{"type":"v1","addr":"172.21.15.43:6827","nonce":1709538759}]},"public_addr":"172.21.15.43:6816/1709538759","cluster_addr":"172.21.15.43:6822/1709538759","heartbeat_back_addr":"172.21.15.43:6830/1709538759","heartbeat_front_addr":"172.21.15.43:6827/1709538759","state":["exists","up"]},{"osd":4,"uuid":"fa2e8a77-aba1-4756-853e-0d174a5bdeba","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":170,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6800","nonce":4196866481},{"type":"v1","addr":"172.21.15.203:6801","nonce":4196866481}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6802","nonce":4196866481},{"type":"v1","addr":"172.21.15.203:6803","nonce":4196866481}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6806","nonce":4196866481},{"type":"v1","addr":"172.21.15.203:6807","nonce":4196866481}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6804","nonce":4196866481},{"type":"v1","addr":"172.21.15.203:6805","nonce":4196866481}]},"public_addr":"172.21.15.203:6801/4196866481","cluster_addr":"172.21.15.203:6803/4196866481","heartbeat_back_addr":"172.21.15.203:6807/4196866481","heartbeat_front_addr":"172.21.15.203:6805/4196866481","state":["exists","up"]},{"osd":5,"uuid":"81811e4b-4d24-4f51-a82c-f1d700a2a00d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":170,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6816","nonce":3059290081},{"type":"v1","addr":"172.21.15.203:6817","nonce":3059290081}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6818","nonce":3059290081},{"type":"v1","addr":"172.21.15.203:6819","nonce":3059290081}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6822","nonce":3059290081},{"type":"v1","addr":"172.21.15.203:6823","nonce":3059290081}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6820","nonce":3059290081},{"type":"v1","addr":"172.21.15.203:6821","nonce":3059290081}]},"public_addr":"172.21.15.203:6817/3059290081","cluster_addr":"172.21.15.203:6819/3059290081","heartbeat_back_addr":"172.21.15.203:6823/3059290081","heartbeat_front_addr":"172.21.15.203:6821/3059290081","state":["exists","up"]},{"osd":6,"uuid":"307bac4d-6a5b-443d-af75-56d695d7674f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":170,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6824","nonce":512155569},{"type":"v1","addr":"172.21.15.203:6825","nonce":512155569}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6826","nonce":512155569},{"type":"v1","addr":"172.21.15.203:6827","nonce":512155569}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6830","nonce":512155569},{"type":"v1","addr":"172.21.15.203:6831","nonce":512155569}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6828","nonce":512155569},{"type":"v1","addr":"172.21.15.203:6829","nonce":512155569}]},"public_addr":"172.21.15.203:6825/512155569","cluster_addr":"172.21.15.203:6827/512155569","heartbeat_back_addr":"172.21.15.203:6831/512155569","heartbeat_front_addr":"172.21.15.203:6829/512155569","state":["exists","up"]},{"osd":7,"uuid":"a95dc7ca-9f1b-4295-af70-68a22a2d7d5e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":170,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6808","nonce":3331450438},{"type":"v1","addr":"172.21.15.203:6809","nonce":3331450438}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6810","nonce":3331450438},{"type":"v1","addr":"172.21.15.203:6811","nonce":3331450438}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6814","nonce":3331450438},{"type":"v1","addr":"172.21.15.203:6815","nonce":3331450438}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6812","nonce":3331450438},{"type":"v1","addr":"172.21.15.203:6813","nonce":3331450438}]},"public_addr":"172.21.15.203:6809/3331450438","cluster_addr":"172.21.15.203:6811/3331450438","heartbeat_back_addr":"172.21.15.203:6815/3331450438","heartbeat_front_addr":"172.21.15.203:6813/3331450438","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-09T13:33:50.547652+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-09T13:33:50.589469+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-09T13:33:50.561724+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-09T13:33:50.599650+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-09T13:33:50.411855+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-09T13:33:50.219681+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-09T13:33:50.504664+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-09T13:33:50.441248+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":7,"snaps":[{"begin":104,"length":1},{"begin":221,"length":1}]}],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-08-09T14:51:51.777 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:51.776+0000 7f4446574640 1 -- 172.21.15.43:0/3116270543 >> [v2:172.21.15.203:6832/2566816332,v1:172.21.15.203:6833/2566816332] conn(0x7f442406a1a0 msgr2=0x7f442406c660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T14:51:51.778 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:51.776+0000 7f4446574640 1 --2- 172.21.15.43:0/3116270543 >> [v2:172.21.15.203:6832/2566816332,v1:172.21.15.203:6833/2566816332] conn(0x7f442406a1a0 0x7f442406c660 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f4430005880 tx=0x7f4430007a00 comp rx=0 tx=0).stop 2023-08-09T14:51:51.778 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:51.776+0000 7f4446574640 1 -- 172.21.15.43:0/3116270543 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f444015c780 msgr2=0x7f444009a750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T14:51:51.778 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:51.776+0000 7f4446574640 1 --2- 172.21.15.43:0/3116270543 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f444015c780 0x7f444009a750 secure :-1 s=READY pgs=1878 cs=0 l=1 rev1=1 crypto rx=0x7f443c00c450 tx=0x7f443c00c910 comp rx=0 tx=0).stop 2023-08-09T14:51:51.778 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:51.776+0000 7f4446574640 1 -- 172.21.15.43:0/3116270543 shutdown_connections 2023-08-09T14:51:51.778 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:51.776+0000 7f4446574640 1 --2- 172.21.15.43:0/3116270543 >> [v2:172.21.15.203:6832/2566816332,v1:172.21.15.203:6833/2566816332] conn(0x7f442406a1a0 0x7f442406c660 unknown :-1 s=CLOSED pgs=64 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T14:51:51.778 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:51.776+0000 7f4446574640 1 --2- 172.21.15.43:0/3116270543 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f4440160cb0 0x7f4440145c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T14:51:51.779 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:51.776+0000 7f4446574640 1 --2- 172.21.15.43:0/3116270543 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f4440160380 0x7f4440098c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T14:51:51.779 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:51.776+0000 7f4446574640 1 --2- 172.21.15.43:0/3116270543 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f444015c780 0x7f444009a750 unknown :-1 s=CLOSED pgs=1878 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T14:51:51.779 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:51.776+0000 7f4446574640 1 -- 172.21.15.43:0/3116270543 >> 172.21.15.43:0/3116270543 conn(0x7f44400cde90 msgr2=0x7f44400ca0c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-09T14:51:51.779 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:51.776+0000 7f4446574640 1 -- 172.21.15.43:0/3116270543 shutdown_connections 2023-08-09T14:51:51.779 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:51.776+0000 7f4446574640 1 -- 172.21.15.43:0/3116270543 wait complete. 2023-08-09T14:51:51.797 INFO:tasks.cephfs.filesystem.ceph_manager:waiting for clean 2023-08-09T14:51:51.798 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-08-09T14:51:52.030 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:52.029+0000 7fc2f7d5b640 1 Processor -- start 2023-08-09T14:51:52.031 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:52.029+0000 7fc2f7d5b640 1 -- start start 2023-08-09T14:51:52.032 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:52.031+0000 7fc2f7d5b640 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7fc2f015c630 0x7fc2f00abef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T14:51:52.033 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:52.031+0000 7fc2f7d5b640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fc2f00ccf20 0x7fc2f00cd550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T14:51:52.033 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:52.031+0000 7fc2f7d5b640 1 -- --> v1:172.21.15.43:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc2f00cca90 con 0x7fc2f00cda90 2023-08-09T14:51:52.033 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:52.031+0000 7fc2f7d5b640 1 -- --> v2:172.21.15.203:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc2f00cbc70 con 0x7fc2f015c630 2023-08-09T14:51:52.033 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:52.031+0000 7fc2f7d5b640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc2f00cdf00 con 0x7fc2f00ccf20 2023-08-09T14:51:52.035 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:52.032+0000 7fc2f62f8640 1 --1- >> v1:172.21.15.43:6789/0 conn(0x7fc2f00cda90 0x7fc2f00c2f20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.43:6789/0 says I am v1:172.21.15.43:47312/0 (socket says 172.21.15.43:47312) 2023-08-09T14:51:52.035 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:52.032+0000 7fc2f62f8640 1 -- 172.21.15.43:0/1684900386 learned_addr learned my addr 172.21.15.43:0/1684900386 (peer_addr_for_me v1:172.21.15.43:0/0) 2023-08-09T14:51:52.035 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:52.032+0000 7fc2f52f6640 1 --2- 172.21.15.43:0/1684900386 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fc2f00ccf20 0x7fc2f00cd550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T14:51:52.035 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:52.032+0000 7fc2f5af7640 1 --2- 172.21.15.43:0/1684900386 >> v2:172.21.15.203:3300/0 conn(0x7fc2f015c630 0x7fc2f00abef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T14:51:52.036 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:52.032+0000 7fc2f4af5640 1 -- 172.21.15.43:0/1684900386 <== mon.0 v1:172.21.15.43:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1891775479 0 0) 0x7fc2f00cca90 con 0x7fc2f00cda90 2023-08-09T14:51:52.036 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:52.032+0000 7fc2f4af5640 1 -- 172.21.15.43:0/1684900386 --> v1:172.21.15.43:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fc2dc003520 con 0x7fc2f00cda90 2023-08-09T14:51:52.036 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:52.033+0000 7fc2f4af5640 1 -- 172.21.15.43:0/1684900386 <== mon.0 v1:172.21.15.43:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 775779416 0 0) 0x7fc2dc003520 con 0x7fc2f00cda90 2023-08-09T14:51:52.036 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:52.033+0000 7fc2f4af5640 1 -- 172.21.15.43:0/1684900386 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fc2f00ccf20 msgr2=0x7fc2f00cd550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-09T14:51:52.036 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:52.033+0000 7fc2f4af5640 1 --2- 172.21.15.43:0/1684900386 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fc2f00ccf20 0x7fc2f00cd550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T14:51:52.036 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:52.033+0000 7fc2f4af5640 1 -- 172.21.15.43:0/1684900386 >> v2:172.21.15.203:3300/0 conn(0x7fc2f015c630 msgr2=0x7fc2f00abef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-09T14:51:52.037 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:52.033+0000 7fc2f4af5640 1 --2- 172.21.15.43:0/1684900386 >> v2:172.21.15.203:3300/0 conn(0x7fc2f015c630 0x7fc2f00abef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T14:51:52.037 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:52.033+0000 7fc2f4af5640 1 -- 172.21.15.43:0/1684900386 --> v1:172.21.15.43:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc2f015da20 con 0x7fc2f00cda90 2023-08-09T14:51:52.037 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:52.033+0000 7fc2f4af5640 1 -- 172.21.15.43:0/1684900386 <== mon.0 v1:172.21.15.43:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1285150740 0 0) 0x7fc2e80019f0 con 0x7fc2f00cda90 2023-08-09T14:51:52.037 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:52.033+0000 7fc2f5af7640 1 --2- 172.21.15.43:0/1684900386 >> v2:172.21.15.203:3300/0 conn(0x7fc2f015c630 0x7fc2f00abef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-08-09T14:51:52.037 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:52.033+0000 7fc2f4af5640 1 -- 172.21.15.43:0/1684900386 >> v1:172.21.15.43:6789/0 conn(0x7fc2f00cda90 legacy=0x7fc2f00c2f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T14:51:52.037 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:52.033+0000 7fc2f4af5640 1 --2- 172.21.15.43:0/1684900386 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc2dc0039e0 0x7fc2dc003e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T14:51:52.037 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:52.033+0000 7fc2f4af5640 1 --2- 172.21.15.43:0/1684900386 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fc2dc0047c0 0x7fc2dc004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T14:51:52.038 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:52.033+0000 7fc2f4af5640 1 --2- 172.21.15.43:0/1684900386 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fc2f00ccf20 0x7fc2dc005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T14:51:52.038 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:52.033+0000 7fc2f4af5640 1 -- 172.21.15.43:0/1684900386 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc2f00cbc70 con 0x7fc2dc0039e0 2023-08-09T14:51:52.038 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:52.033+0000 7fc2f4af5640 1 -- 172.21.15.43:0/1684900386 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc2f00cdf00 con 0x7fc2dc0047c0 2023-08-09T14:51:52.038 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:52.033+0000 7fc2f4af5640 1 -- 172.21.15.43:0/1684900386 --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc2dc00b980 con 0x7fc2f00ccf20 2023-08-09T14:51:52.038 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:52.033+0000 7fc2f5af7640 1 --2- 172.21.15.43:0/1684900386 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc2dc0039e0 0x7fc2dc003e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T14:51:52.041 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:52.034+0000 7fc2f62f8640 1 --2- 172.21.15.43:0/1684900386 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fc2f00ccf20 0x7fc2dc005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T14:51:52.041 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:52.034+0000 7fc2f5af7640 1 -- 172.21.15.43:0/1684900386 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fc2f00ccf20 msgr2=0x7fc2dc005180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-09T14:51:52.041 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:52.034+0000 7fc2f5af7640 1 --2- 172.21.15.43:0/1684900386 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fc2f00ccf20 0x7fc2dc005180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T14:51:52.042 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:52.034+0000 7fc2f5af7640 1 -- 172.21.15.43:0/1684900386 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fc2dc0047c0 msgr2=0x7fc2dc004c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-09T14:51:52.042 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:52.034+0000 7fc2f5af7640 1 --2- 172.21.15.43:0/1684900386 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fc2dc0047c0 0x7fc2dc004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T14:51:52.042 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:52.034+0000 7fc2f5af7640 1 -- 172.21.15.43:0/1684900386 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc2dc003520 con 0x7fc2dc0039e0 2023-08-09T14:51:52.042 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:52.034+0000 7fc2f62f8640 1 --2- 172.21.15.43:0/1684900386 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fc2f00ccf20 0x7fc2dc005180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-08-09T14:51:52.042 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:52.035+0000 7fc2f5af7640 1 --2- 172.21.15.43:0/1684900386 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc2dc0039e0 0x7fc2dc003e00 secure :-1 s=READY pgs=1880 cs=0 l=1 rev1=1 crypto rx=0x7fc2ec009cc0 tx=0x7fc2dc0057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=25ad4a3e1115d9de server_cookie=0 in_seq=0 out_seq=0 2023-08-09T14:51:52.042 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:52.035+0000 7fc2f4af5640 1 -- 172.21.15.43:0/1684900386 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc2ec02c980 con 0x7fc2dc0039e0 2023-08-09T14:51:52.042 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:52.035+0000 7fc2f4af5640 1 -- 172.21.15.43:0/1684900386 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc2ec02cae0 con 0x7fc2dc0039e0 2023-08-09T14:51:52.043 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:52.035+0000 7fc2f4af5640 1 -- 172.21.15.43:0/1684900386 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc2ec02ce90 con 0x7fc2dc0039e0 2023-08-09T14:51:52.043 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:52.036+0000 7fc2f7d5b640 1 -- 172.21.15.43:0/1684900386 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc2dc0039e0 msgr2=0x7fc2dc003e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T14:51:52.043 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:52.036+0000 7fc2f7d5b640 1 --2- 172.21.15.43:0/1684900386 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc2dc0039e0 0x7fc2dc003e00 secure :-1 s=READY pgs=1880 cs=0 l=1 rev1=1 crypto rx=0x7fc2ec009cc0 tx=0x7fc2dc0057f0 comp rx=0 tx=0).stop 2023-08-09T14:51:52.043 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:52.036+0000 7fc2f62f8640 1 -- 172.21.15.43:0/1684900386 reap_dead start 2023-08-09T14:51:52.043 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:52.036+0000 7fc2f7d5b640 1 -- 172.21.15.43:0/1684900386 shutdown_connections 2023-08-09T14:51:52.043 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:52.036+0000 7fc2f7d5b640 1 -- 172.21.15.43:0/1684900386 >> 172.21.15.43:0/1684900386 conn(0x7fc2f00ce0a0 msgr2=0x7fc2f00ce480 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-09T14:51:52.043 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:52.036+0000 7fc2f7d5b640 1 -- 172.21.15.43:0/1684900386 shutdown_connections 2023-08-09T14:51:52.043 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:52.036+0000 7fc2f7d5b640 1 -- 172.21.15.43:0/1684900386 wait complete. 2023-08-09T14:51:52.044 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:52.038+0000 7fc2f7d5b640 1 Processor -- start 2023-08-09T14:51:52.044 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:52.038+0000 7fc2f7d5b640 1 -- start start 2023-08-09T14:51:52.044 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:52.039+0000 7fc2f7d5b640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fc2dc0039e0 0x7fc2f00af030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T14:51:52.044 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:52.039+0000 7fc2f7d5b640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fc2f01f64e0 0x7fc2f01f6960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T14:51:52.044 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:52.039+0000 7fc2f7d5b640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc2f00ad9c0 0x7fc2f00ade40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T14:51:52.044 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:52.039+0000 7fc2f7d5b640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc2f00c5170 con 0x7fc2f00ad9c0 2023-08-09T14:51:52.044 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:52.039+0000 7fc2f7d5b640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc2f00c56f0 con 0x7fc2f01f64e0 2023-08-09T14:51:52.045 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:52.039+0000 7fc2f7d5b640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc2f01f7010 con 0x7fc2dc0039e0 2023-08-09T14:51:52.045 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:52.040+0000 7fc2f62f8640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc2f00ad9c0 0x7fc2f00ade40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T14:51:52.045 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:52.040+0000 7fc2f62f8640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc2f00ad9c0 0x7fc2f00ade40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:57932/0 (socket says 172.21.15.43:57932) 2023-08-09T14:51:52.045 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:52.040+0000 7fc2f62f8640 1 -- 172.21.15.43:0/3165418025 learned_addr learned my addr 172.21.15.43:0/3165418025 (peer_addr_for_me v2:172.21.15.43:0/0) 2023-08-09T14:51:52.046 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:52.040+0000 7fc2f5af7640 1 --2- 172.21.15.43:0/3165418025 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fc2dc0039e0 0x7fc2f00af030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T14:51:52.047 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:52.040+0000 7fc2f52f6640 1 --2- 172.21.15.43:0/3165418025 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fc2f01f64e0 0x7fc2f01f6960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T14:51:52.047 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:52.040+0000 7fc2f62f8640 1 -- 172.21.15.43:0/3165418025 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fc2dc0039e0 msgr2=0x7fc2f00af030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T14:51:52.047 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:52.040+0000 7fc2f62f8640 1 --2- 172.21.15.43:0/3165418025 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fc2dc0039e0 0x7fc2f00af030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T14:51:52.047 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:52.040+0000 7fc2f62f8640 1 -- 172.21.15.43:0/3165418025 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fc2f01f64e0 msgr2=0x7fc2f01f6960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T14:51:52.047 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:52.040+0000 7fc2f62f8640 1 --2- 172.21.15.43:0/3165418025 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fc2f01f64e0 0x7fc2f01f6960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T14:51:52.047 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:52.040+0000 7fc2f62f8640 1 -- 172.21.15.43:0/3165418025 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc2f01f72f0 con 0x7fc2f00ad9c0 2023-08-09T14:51:52.048 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:52.040+0000 7fc2f52f6640 1 --2- 172.21.15.43:0/3165418025 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fc2f01f64e0 0x7fc2f01f6960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-08-09T14:51:52.048 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:52.041+0000 7fc2f62f8640 1 --2- 172.21.15.43:0/3165418025 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc2f00ad9c0 0x7fc2f00ade40 secure :-1 s=READY pgs=1881 cs=0 l=1 rev1=1 crypto rx=0x7fc2e8000f80 tx=0x7fc2e800df90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-09T14:51:52.048 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:52.041+0000 7fc2e67fc640 1 -- 172.21.15.43:0/3165418025 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc2e8001990 con 0x7fc2f00ad9c0 2023-08-09T14:51:52.048 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:52.041+0000 7fc2e67fc640 1 -- 172.21.15.43:0/3165418025 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc2e800bab0 con 0x7fc2f00ad9c0 2023-08-09T14:51:52.048 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:52.041+0000 7fc2f7d5b640 1 -- 172.21.15.43:0/3165418025 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc2f00c5380 con 0x7fc2f00ad9c0 2023-08-09T14:51:52.048 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:52.041+0000 7fc2e67fc640 1 -- 172.21.15.43:0/3165418025 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc2e800be40 con 0x7fc2f00ad9c0 2023-08-09T14:51:52.048 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:52.041+0000 7fc2f7d5b640 1 -- 172.21.15.43:0/3165418025 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc2f00ae380 con 0x7fc2f00ad9c0 2023-08-09T14:51:52.048 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:52.042+0000 7fc2f7d5b640 1 -- 172.21.15.43:0/3165418025 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc2f00ae570 con 0x7fc2f00ad9c0 2023-08-09T14:51:52.049 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:52.043+0000 7fc2e67fc640 1 -- 172.21.15.43:0/3165418025 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 79) v1 ==== 88443+0+0 (secure 0 0 0) 0x7fc2e8007110 con 0x7fc2f00ad9c0 2023-08-09T14:51:52.049 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:52.044+0000 7fc2e67fc640 1 --2- 172.21.15.43:0/3165418025 >> [v2:172.21.15.203:6832/2566816332,v1:172.21.15.203:6833/2566816332] conn(0x7fc2d406a170 0x7fc2d406c630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T14:51:52.049 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:52.044+0000 7fc2e67fc640 1 -- 172.21.15.43:0/3165418025 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(172..172 src has 1..172) v4 ==== 6426+0+0 (secure 0 0 0) 0x7fc2e8088e30 con 0x7fc2f00ad9c0 2023-08-09T14:51:52.049 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:52.045+0000 7fc2f5af7640 1 --2- 172.21.15.43:0/3165418025 >> [v2:172.21.15.203:6832/2566816332,v1:172.21.15.203:6833/2566816332] conn(0x7fc2d406a170 0x7fc2d406c630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T14:51:52.049 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:52.046+0000 7fc2f5af7640 1 --2- 172.21.15.43:0/3165418025 >> [v2:172.21.15.203:6832/2566816332,v1:172.21.15.203:6833/2566816332] conn(0x7fc2d406a170 0x7fc2d406c630 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7fc2ec036000 tx=0x7fc2ec030000 comp rx=0 tx=0).ready entity=mgr.7765 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-09T14:51:52.052 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:52.049+0000 7fc2e67fc640 1 -- 172.21.15.43:0/3165418025 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178066 (secure 0 0 0) 0x7fc2e80540f0 con 0x7fc2f00ad9c0 2023-08-09T14:51:52.244 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:52.243+0000 7fc2f7d5b640 1 -- 172.21.15.43:0/3165418025 --> [v2:172.21.15.203:6832/2566816332,v1:172.21.15.203:6833/2566816332] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7fc2f00cc160 con 0x7fc2d406a170 2023-08-09T14:51:52.275 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:52.273+0000 7fc2e67fc640 1 -- 172.21.15.43:0/3165418025 <== mgr.7765 v2:172.21.15.203:6832/2566816332 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+308933 (secure 0 0 0) 0x7fc2ec045090 con 0x7fc2d406a170 2023-08-09T14:51:52.277 INFO:teuthology.orchestra.run.smithi043.stdout: 2023-08-09T14:51:52.280 INFO:teuthology.orchestra.run.smithi043.stderr:dumped all 2023-08-09T14:51:52.282 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:52.281+0000 7fc2f7d5b640 1 -- 172.21.15.43:0/3165418025 >> [v2:172.21.15.203:6832/2566816332,v1:172.21.15.203:6833/2566816332] conn(0x7fc2d406a170 msgr2=0x7fc2d406c630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T14:51:52.282 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:52.281+0000 7fc2f7d5b640 1 --2- 172.21.15.43:0/3165418025 >> [v2:172.21.15.203:6832/2566816332,v1:172.21.15.203:6833/2566816332] conn(0x7fc2d406a170 0x7fc2d406c630 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7fc2ec036000 tx=0x7fc2ec030000 comp rx=0 tx=0).stop 2023-08-09T14:51:52.283 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:52.281+0000 7fc2f7d5b640 1 -- 172.21.15.43:0/3165418025 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc2f00ad9c0 msgr2=0x7fc2f00ade40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T14:51:52.283 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:52.281+0000 7fc2f7d5b640 1 --2- 172.21.15.43:0/3165418025 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc2f00ad9c0 0x7fc2f00ade40 secure :-1 s=READY pgs=1881 cs=0 l=1 rev1=1 crypto rx=0x7fc2e8000f80 tx=0x7fc2e800df90 comp rx=0 tx=0).stop 2023-08-09T14:51:52.283 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:52.281+0000 7fc2f7d5b640 1 -- 172.21.15.43:0/3165418025 shutdown_connections 2023-08-09T14:51:52.283 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:52.281+0000 7fc2f7d5b640 1 --2- 172.21.15.43:0/3165418025 >> [v2:172.21.15.203:6832/2566816332,v1:172.21.15.203:6833/2566816332] conn(0x7fc2d406a170 0x7fc2d406c630 unknown :-1 s=CLOSED pgs=65 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T14:51:52.283 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:52.281+0000 7fc2f7d5b640 1 --2- 172.21.15.43:0/3165418025 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc2f00ad9c0 0x7fc2f00ade40 unknown :-1 s=CLOSED pgs=1881 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T14:51:52.283 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:52.281+0000 7fc2f7d5b640 1 --2- 172.21.15.43:0/3165418025 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fc2f01f64e0 0x7fc2f01f6960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T14:51:52.283 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:52.281+0000 7fc2f7d5b640 1 --2- 172.21.15.43:0/3165418025 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fc2dc0039e0 0x7fc2f00af030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T14:51:52.283 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:52.281+0000 7fc2f7d5b640 1 -- 172.21.15.43:0/3165418025 >> 172.21.15.43:0/3165418025 conn(0x7fc2f00ce0a0 msgr2=0x7fc2f015bbe0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-09T14:51:52.284 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:52.281+0000 7fc2f7d5b640 1 -- 172.21.15.43:0/3165418025 shutdown_connections 2023-08-09T14:51:52.284 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:52.281+0000 7fc2f7d5b640 1 -- 172.21.15.43:0/3165418025 wait complete. 2023-08-09T14:51:52.306 INFO:teuthology.orchestra.run.smithi043.stdout:{"pg_ready":true,"pg_map":{"version":27,"stamp":"2023-08-09T14:51:51.226647+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":23,"num_object_clones":0,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":0,"num_read":1291,"num_read_kb":2654,"num_write":447,"num_write_kb":2430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":290,"ondisk_log_size":290,"up":210,"acting":210,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":95,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":406080,"kb_used_data":17720,"kb_used_omap":13,"kb_used_meta":388210,"kb_avail":754568640,"statfs":{"total":773094113280,"available":772678287360,"internally_reserved":0,"allocated":18145280,"data_stored":10355374,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":14248,"internal_metadata":397527128},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":28,"apply_latency_ms":28,"commit_latency_ns":28000000,"apply_latency_ns":28000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"5.491260"},"pg_stats":[{"pgid":"21.3f","version":"0'0","reported_seq":9,"reported_epoch":171,"state":"active+clean","last_fresh":"2023-08-09T14:51:47.794579+0000","last_change":"2023-08-09T14:51:47.792212+0000","last_active":"2023-08-09T14:51:47.794579+0000","last_peered":"2023-08-09T14:51:47.794579+0000","last_clean":"2023-08-09T14:51:47.794579+0000","last_became_active":"2023-08-09T14:51:47.791933+0000","last_became_peered":"2023-08-09T14:51:47.791933+0000","last_unstale":"2023-08-09T14:51:47.794579+0000","last_undegraded":"2023-08-09T14:51:47.794579+0000","last_fullsized":"2023-08-09T14:51:47.794579+0000","mapping_epoch":170,"log_start":"0'0","ondisk_log_start":"0'0","created":170,"last_epoch_clean":171,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T14:51:46.735123+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T14:51:46.735123+0000","last_clean_scrub_stamp":"2023-08-09T14:51:46.735123+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T23:10:26.794676+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"21.3e","version":"0'0","reported_seq":9,"reported_epoch":171,"state":"active+clean","last_fresh":"2023-08-09T14:51:47.788631+0000","last_change":"2023-08-09T14:51:47.784922+0000","last_active":"2023-08-09T14:51:47.788631+0000","last_peered":"2023-08-09T14:51:47.788631+0000","last_clean":"2023-08-09T14:51:47.788631+0000","last_became_active":"2023-08-09T14:51:47.784551+0000","last_became_peered":"2023-08-09T14:51:47.784551+0000","last_unstale":"2023-08-09T14:51:47.788631+0000","last_undegraded":"2023-08-09T14:51:47.788631+0000","last_fullsized":"2023-08-09T14:51:47.788631+0000","mapping_epoch":170,"log_start":"0'0","ondisk_log_start":"0'0","created":170,"last_epoch_clean":171,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T14:51:46.735123+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T14:51:46.735123+0000","last_clean_scrub_stamp":"2023-08-09T14:51:46.735123+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-11T01:01:21.767494+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"21.3d","version":"0'0","reported_seq":9,"reported_epoch":171,"state":"active+clean","last_fresh":"2023-08-09T14:51:47.797949+0000","last_change":"2023-08-09T14:51:47.793329+0000","last_active":"2023-08-09T14:51:47.797949+0000","last_peered":"2023-08-09T14:51:47.797949+0000","last_clean":"2023-08-09T14:51:47.797949+0000","last_became_active":"2023-08-09T14:51:47.793034+0000","last_became_peered":"2023-08-09T14:51:47.793034+0000","last_unstale":"2023-08-09T14:51:47.797949+0000","last_undegraded":"2023-08-09T14:51:47.797949+0000","last_fullsized":"2023-08-09T14:51:47.797949+0000","mapping_epoch":170,"log_start":"0'0","ondisk_log_start":"0'0","created":170,"last_epoch_clean":171,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T14:51:46.735123+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T14:51:46.735123+0000","last_clean_scrub_stamp":"2023-08-09T14:51:46.735123+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T18:53:38.906224+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"21.3c","version":"0'0","reported_seq":9,"reported_epoch":171,"state":"active+clean","last_fresh":"2023-08-09T14:51:47.807916+0000","last_change":"2023-08-09T14:51:47.800751+0000","last_active":"2023-08-09T14:51:47.807916+0000","last_peered":"2023-08-09T14:51:47.807916+0000","last_clean":"2023-08-09T14:51:47.807916+0000","last_became_active":"2023-08-09T14:51:47.800253+0000","last_became_peered":"2023-08-09T14:51:47.800253+0000","last_unstale":"2023-08-09T14:51:47.807916+0000","last_undegraded":"2023-08-09T14:51:47.807916+0000","last_fullsized":"2023-08-09T14:51:47.807916+0000","mapping_epoch":170,"log_start":"0'0","ondisk_log_start":"0'0","created":170,"last_epoch_clean":171,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T14:51:46.735123+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T14:51:46.735123+0000","last_clean_scrub_stamp":"2023-08-09T14:51:46.735123+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T23:19:38.181929+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"21.3b","version":"0'0","reported_seq":9,"reported_epoch":171,"state":"active+clean","last_fresh":"2023-08-09T14:51:47.796865+0000","last_change":"2023-08-09T14:51:47.792818+0000","last_active":"2023-08-09T14:51:47.796865+0000","last_peered":"2023-08-09T14:51:47.796865+0000","last_clean":"2023-08-09T14:51:47.796865+0000","last_became_active":"2023-08-09T14:51:47.792523+0000","last_became_peered":"2023-08-09T14:51:47.792523+0000","last_unstale":"2023-08-09T14:51:47.796865+0000","last_undegraded":"2023-08-09T14:51:47.796865+0000","last_fullsized":"2023-08-09T14:51:47.796865+0000","mapping_epoch":170,"log_start":"0'0","ondisk_log_start":"0'0","created":170,"last_epoch_clean":171,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T14:51:46.735123+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T14:51:46.735123+0000","last_clean_scrub_stamp":"2023-08-09T14:51:46.735123+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-11T02:37:22.012917+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"21.3a","version":"0'0","reported_seq":9,"reported_epoch":171,"state":"active+clean","last_fresh":"2023-08-09T14:51:47.801341+0000","last_change":"2023-08-09T14:51:47.795139+0000","last_active":"2023-08-09T14:51:47.801341+0000","last_peered":"2023-08-09T14:51:47.801341+0000","last_clean":"2023-08-09T14:51:47.801341+0000","last_became_active":"2023-08-09T14:51:47.794866+0000","last_became_peered":"2023-08-09T14:51:47.794866+0000","last_unstale":"2023-08-09T14:51:47.801341+0000","last_undegraded":"2023-08-09T14:51:47.801341+0000","last_fullsized":"2023-08-09T14:51:47.801341+0000","mapping_epoch":170,"log_start":"0'0","ondisk_log_start":"0'0","created":170,"last_epoch_clean":171,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T14:51:46.735123+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T14:51:46.735123+0000","last_clean_scrub_stamp":"2023-08-09T14:51:46.735123+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T19:03:21.686709+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"21.39","version":"0'0","reported_seq":9,"reported_epoch":171,"state":"active+clean","last_fresh":"2023-08-09T14:51:47.802119+0000","last_change":"2023-08-09T14:51:47.795696+0000","last_active":"2023-08-09T14:51:47.802119+0000","last_peered":"2023-08-09T14:51:47.802119+0000","last_clean":"2023-08-09T14:51:47.802119+0000","last_became_active":"2023-08-09T14:51:47.795019+0000","last_became_peered":"2023-08-09T14:51:47.795019+0000","last_unstale":"2023-08-09T14:51:47.802119+0000","last_undegraded":"2023-08-09T14:51:47.802119+0000","last_fullsized":"2023-08-09T14:51:47.802119+0000","mapping_epoch":170,"log_start":"0'0","ondisk_log_start":"0'0","created":170,"last_epoch_clean":171,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T14:51:46.735123+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T14:51:46.735123+0000","last_clean_scrub_stamp":"2023-08-09T14:51:46.735123+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T16:04:02.808388+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"21.38","version":"0'0","reported_seq":9,"reported_epoch":171,"state":"active+clean","last_fresh":"2023-08-09T14:51:47.787294+0000","last_change":"2023-08-09T14:51:47.785620+0000","last_active":"2023-08-09T14:51:47.787294+0000","last_peered":"2023-08-09T14:51:47.787294+0000","last_clean":"2023-08-09T14:51:47.787294+0000","last_became_active":"2023-08-09T14:51:47.783277+0000","last_became_peered":"2023-08-09T14:51:47.783277+0000","last_unstale":"2023-08-09T14:51:47.787294+0000","last_undegraded":"2023-08-09T14:51:47.787294+0000","last_fullsized":"2023-08-09T14:51:47.787294+0000","mapping_epoch":170,"log_start":"0'0","ondisk_log_start":"0'0","created":170,"last_epoch_clean":171,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T14:51:46.735123+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T14:51:46.735123+0000","last_clean_scrub_stamp":"2023-08-09T14:51:46.735123+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T19:31:07.562365+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"21.37","version":"0'0","reported_seq":9,"reported_epoch":171,"state":"active+clean","last_fresh":"2023-08-09T14:51:47.793300+0000","last_change":"2023-08-09T14:51:47.790654+0000","last_active":"2023-08-09T14:51:47.793300+0000","last_peered":"2023-08-09T14:51:47.793300+0000","last_clean":"2023-08-09T14:51:47.793300+0000","last_became_active":"2023-08-09T14:51:47.790356+0000","last_became_peered":"2023-08-09T14:51:47.790356+0000","last_unstale":"2023-08-09T14:51:47.793300+0000","last_undegraded":"2023-08-09T14:51:47.793300+0000","last_fullsized":"2023-08-09T14:51:47.793300+0000","mapping_epoch":170,"log_start":"0'0","ondisk_log_start":"0'0","created":170,"last_epoch_clean":171,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T14:51:46.735123+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T14:51:46.735123+0000","last_clean_scrub_stamp":"2023-08-09T14:51:46.735123+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-11T02:16:12.197435+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"21.36","version":"0'0","reported_seq":9,"reported_epoch":171,"state":"active+clean","last_fresh":"2023-08-09T14:51:47.805290+0000","last_change":"2023-08-09T14:51:47.795912+0000","last_active":"2023-08-09T14:51:47.805290+0000","last_peered":"2023-08-09T14:51:47.805290+0000","last_clean":"2023-08-09T14:51:47.805290+0000","last_became_active":"2023-08-09T14:51:47.795608+0000","last_became_peered":"2023-08-09T14:51:47.795608+0000","last_unstale":"2023-08-09T14:51:47.805290+0000","last_undegraded":"2023-08-09T14:51:47.805290+0000","last_fullsized":"2023-08-09T14:51:47.805290+0000","mapping_epoch":170,"log_start":"0'0","ondisk_log_start":"0'0","created":170,"last_epoch_clean":171,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T14:51:46.735123+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T14:51:46.735123+0000","last_clean_scrub_stamp":"2023-08-09T14:51:46.735123+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T17:35:49.467820+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"21.35","version":"0'0","reported_seq":9,"reported_epoch":171,"state":"active+clean","last_fresh":"2023-08-09T14:51:47.785607+0000","last_change":"2023-08-09T14:51:47.779209+0000","last_active":"2023-08-09T14:51:47.785607+0000","last_peered":"2023-08-09T14:51:47.785607+0000","last_clean":"2023-08-09T14:51:47.785607+0000","last_became_active":"2023-08-09T14:51:47.778879+0000","last_became_peered":"2023-08-09T14:51:47.778879+0000","last_unstale":"2023-08-09T14:51:47.785607+0000","last_undegraded":"2023-08-09T14:51:47.785607+0000","last_fullsized":"2023-08-09T14:51:47.785607+0000","mapping_epoch":170,"log_start":"0'0","ondisk_log_start":"0'0","created":170,"last_epoch_clean":171,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T14:51:46.735123+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T14:51:46.735123+0000","last_clean_scrub_stamp":"2023-08-09T14:51:46.735123+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T17:34:21.807704+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"21.34","version":"0'0","reported_seq":9,"reported_epoch":171,"state":"active+clean","last_fresh":"2023-08-09T14:51:47.808271+0000","last_change":"2023-08-09T14:51:47.801977+0000","last_active":"2023-08-09T14:51:47.808271+0000","last_peered":"2023-08-09T14:51:47.808271+0000","last_clean":"2023-08-09T14:51:47.808271+0000","last_became_active":"2023-08-09T14:51:47.801683+0000","last_became_peered":"2023-08-09T14:51:47.801683+0000","last_unstale":"2023-08-09T14:51:47.808271+0000","last_undegraded":"2023-08-09T14:51:47.808271+0000","last_fullsized":"2023-08-09T14:51:47.808271+0000","mapping_epoch":170,"log_start":"0'0","ondisk_log_start":"0'0","created":170,"last_epoch_clean":171,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T14:51:46.735123+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T14:51:46.735123+0000","last_clean_scrub_stamp":"2023-08-09T14:51:46.735123+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T23:01:06.813062+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"21.33","version":"0'0","reported_seq":9,"reported_epoch":171,"state":"active+clean","last_fresh":"2023-08-09T14:51:47.792932+0000","last_change":"2023-08-09T14:51:47.791347+0000","last_active":"2023-08-09T14:51:47.792932+0000","last_peered":"2023-08-09T14:51:47.792932+0000","last_clean":"2023-08-09T14:51:47.792932+0000","last_became_active":"2023-08-09T14:51:47.788285+0000","last_became_peered":"2023-08-09T14:51:47.788285+0000","last_unstale":"2023-08-09T14:51:47.792932+0000","last_undegraded":"2023-08-09T14:51:47.792932+0000","last_fullsized":"2023-08-09T14:51:47.792932+0000","mapping_epoch":170,"log_start":"0'0","ondisk_log_start":"0'0","created":170,"last_epoch_clean":171,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T14:51:46.735123+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T14:51:46.735123+0000","last_clean_scrub_stamp":"2023-08-09T14:51:46.735123+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T22:32:23.037398+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"21.32","version":"0'0","reported_seq":9,"reported_epoch":171,"state":"active+clean","last_fresh":"2023-08-09T14:51:47.793903+0000","last_change":"2023-08-09T14:51:47.790400+0000","last_active":"2023-08-09T14:51:47.793903+0000","last_peered":"2023-08-09T14:51:47.793903+0000","last_clean":"2023-08-09T14:51:47.793903+0000","last_became_active":"2023-08-09T14:51:47.790057+0000","last_became_peered":"2023-08-09T14:51:47.790057+0000","last_unstale":"2023-08-09T14:51:47.793903+0000","last_undegraded":"2023-08-09T14:51:47.793903+0000","last_fullsized":"2023-08-09T14:51:47.793903+0000","mapping_epoch":170,"log_start":"0'0","ondisk_log_start":"0'0","created":170,"last_epoch_clean":171,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T14:51:46.735123+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T14:51:46.735123+0000","last_clean_scrub_stamp":"2023-08-09T14:51:46.735123+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T19:01:52.632957+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"21.31","version":"0'0","reported_seq":9,"reported_epoch":171,"state":"active+clean","last_fresh":"2023-08-09T14:51:47.805565+0000","last_change":"2023-08-09T14:51:47.795574+0000","last_active":"2023-08-09T14:51:47.805565+0000","last_peered":"2023-08-09T14:51:47.805565+0000","last_clean":"2023-08-09T14:51:47.805565+0000","last_became_active":"2023-08-09T14:51:47.794319+0000","last_became_peered":"2023-08-09T14:51:47.794319+0000","last_unstale":"2023-08-09T14:51:47.805565+0000","last_undegraded":"2023-08-09T14:51:47.805565+0000","last_fullsized":"2023-08-09T14:51:47.805565+0000","mapping_epoch":170,"log_start":"0'0","ondisk_log_start":"0'0","created":170,"last_epoch_clean":171,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T14:51:46.735123+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T14:51:46.735123+0000","last_clean_scrub_stamp":"2023-08-09T14:51:46.735123+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T21:37:15.799330+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"21.30","version":"0'0","reported_seq":9,"reported_epoch":171,"state":"active+clean","last_fresh":"2023-08-09T14:51:47.798403+0000","last_change":"2023-08-09T14:51:47.791421+0000","last_active":"2023-08-09T14:51:47.798403+0000","last_peered":"2023-08-09T14:51:47.798403+0000","last_clean":"2023-08-09T14:51:47.798403+0000","last_became_active":"2023-08-09T14:51:47.791128+0000","last_became_peered":"2023-08-09T14:51:47.791128+0000","last_unstale":"2023-08-09T14:51:47.798403+0000","last_undegraded":"2023-08-09T14:51:47.798403+0000","last_fullsized":"2023-08-09T14:51:47.798403+0000","mapping_epoch":170,"log_start":"0'0","ondisk_log_start":"0'0","created":170,"last_epoch_clean":171,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T14:51:46.735123+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T14:51:46.735123+0000","last_clean_scrub_stamp":"2023-08-09T14:51:46.735123+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T21:06:30.391234+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"21.2f","version":"0'0","reported_seq":9,"reported_epoch":171,"state":"active+clean","last_fresh":"2023-08-09T14:51:47.805492+0000","last_change":"2023-08-09T14:51:47.798702+0000","last_active":"2023-08-09T14:51:47.805492+0000","last_peered":"2023-08-09T14:51:47.805492+0000","last_clean":"2023-08-09T14:51:47.805492+0000","last_became_active":"2023-08-09T14:51:47.798262+0000","last_became_peered":"2023-08-09T14:51:47.798262+0000","last_unstale":"2023-08-09T14:51:47.805492+0000","last_undegraded":"2023-08-09T14:51:47.805492+0000","last_fullsized":"2023-08-09T14:51:47.805492+0000","mapping_epoch":170,"log_start":"0'0","ondisk_log_start":"0'0","created":170,"last_epoch_clean":171,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T14:51:46.735123+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T14:51:46.735123+0000","last_clean_scrub_stamp":"2023-08-09T14:51:46.735123+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T19:58:42.741137+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"21.2e","version":"0'0","reported_seq":9,"reported_epoch":171,"state":"active+clean","last_fresh":"2023-08-09T14:51:47.799481+0000","last_change":"2023-08-09T14:51:47.797021+0000","last_active":"2023-08-09T14:51:47.799481+0000","last_peered":"2023-08-09T14:51:47.799481+0000","last_clean":"2023-08-09T14:51:47.799481+0000","last_became_active":"2023-08-09T14:51:47.794339+0000","last_became_peered":"2023-08-09T14:51:47.794339+0000","last_unstale":"2023-08-09T14:51:47.799481+0000","last_undegraded":"2023-08-09T14:51:47.799481+0000","last_fullsized":"2023-08-09T14:51:47.799481+0000","mapping_epoch":170,"log_start":"0'0","ondisk_log_start":"0'0","created":170,"last_epoch_clean":171,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T14:51:46.735123+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T14:51:46.735123+0000","last_clean_scrub_stamp":"2023-08-09T14:51:46.735123+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T19:32:47.594934+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"21.2d","version":"0'0","reported_seq":9,"reported_epoch":171,"state":"active+clean","last_fresh":"2023-08-09T14:51:47.792448+0000","last_change":"2023-08-09T14:51:47.789326+0000","last_active":"2023-08-09T14:51:47.792448+0000","last_peered":"2023-08-09T14:51:47.792448+0000","last_clean":"2023-08-09T14:51:47.792448+0000","last_became_active":"2023-08-09T14:51:47.788975+0000","last_became_peered":"2023-08-09T14:51:47.788975+0000","last_unstale":"2023-08-09T14:51:47.792448+0000","last_undegraded":"2023-08-09T14:51:47.792448+0000","last_fullsized":"2023-08-09T14:51:47.792448+0000","mapping_epoch":170,"log_start":"0'0","ondisk_log_start":"0'0","created":170,"last_epoch_clean":171,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T14:51:46.735123+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T14:51:46.735123+0000","last_clean_scrub_stamp":"2023-08-09T14:51:46.735123+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-11T00:15:00.756526+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"21.2c","version":"0'0","reported_seq":9,"reported_epoch":171,"state":"active+clean","last_fresh":"2023-08-09T14:51:47.806034+0000","last_change":"2023-08-09T14:51:47.798711+0000","last_active":"2023-08-09T14:51:47.806034+0000","last_peered":"2023-08-09T14:51:47.806034+0000","last_clean":"2023-08-09T14:51:47.806034+0000","last_became_active":"2023-08-09T14:51:47.798399+0000","last_became_peered":"2023-08-09T14:51:47.798399+0000","last_unstale":"2023-08-09T14:51:47.806034+0000","last_undegraded":"2023-08-09T14:51:47.806034+0000","last_fullsized":"2023-08-09T14:51:47.806034+0000","mapping_epoch":170,"log_start":"0'0","ondisk_log_start":"0'0","created":170,"last_epoch_clean":171,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T14:51:46.735123+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T14:51:46.735123+0000","last_clean_scrub_stamp":"2023-08-09T14:51:46.735123+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T23:16:39.005612+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"21.2b","version":"0'0","reported_seq":9,"reported_epoch":171,"state":"active+clean","last_fresh":"2023-08-09T14:51:47.805689+0000","last_change":"2023-08-09T14:51:47.796099+0000","last_active":"2023-08-09T14:51:47.805689+0000","last_peered":"2023-08-09T14:51:47.805689+0000","last_clean":"2023-08-09T14:51:47.805689+0000","last_became_active":"2023-08-09T14:51:47.795766+0000","last_became_peered":"2023-08-09T14:51:47.795766+0000","last_unstale":"2023-08-09T14:51:47.805689+0000","last_undegraded":"2023-08-09T14:51:47.805689+0000","last_fullsized":"2023-08-09T14:51:47.805689+0000","mapping_epoch":170,"log_start":"0'0","ondisk_log_start":"0'0","created":170,"last_epoch_clean":171,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T14:51:46.735123+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T14:51:46.735123+0000","last_clean_scrub_stamp":"2023-08-09T14:51:46.735123+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T16:18:50.656741+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"21.2a","version":"0'0","reported_seq":9,"reported_epoch":171,"state":"active+clean","last_fresh":"2023-08-09T14:51:47.791675+0000","last_change":"2023-08-09T14:51:47.789658+0000","last_active":"2023-08-09T14:51:47.791675+0000","last_peered":"2023-08-09T14:51:47.791675+0000","last_clean":"2023-08-09T14:51:47.791675+0000","last_became_active":"2023-08-09T14:51:47.784732+0000","last_became_peered":"2023-08-09T14:51:47.784732+0000","last_unstale":"2023-08-09T14:51:47.791675+0000","last_undegraded":"2023-08-09T14:51:47.791675+0000","last_fullsized":"2023-08-09T14:51:47.791675+0000","mapping_epoch":170,"log_start":"0'0","ondisk_log_start":"0'0","created":170,"last_epoch_clean":171,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T14:51:46.735123+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T14:51:46.735123+0000","last_clean_scrub_stamp":"2023-08-09T14:51:46.735123+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T22:41:30.767499+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"21.29","version":"0'0","reported_seq":9,"reported_epoch":171,"state":"active+clean","last_fresh":"2023-08-09T14:51:47.791613+0000","last_change":"2023-08-09T14:51:47.789361+0000","last_active":"2023-08-09T14:51:47.791613+0000","last_peered":"2023-08-09T14:51:47.791613+0000","last_clean":"2023-08-09T14:51:47.791613+0000","last_became_active":"2023-08-09T14:51:47.789071+0000","last_became_peered":"2023-08-09T14:51:47.789071+0000","last_unstale":"2023-08-09T14:51:47.791613+0000","last_undegraded":"2023-08-09T14:51:47.791613+0000","last_fullsized":"2023-08-09T14:51:47.791613+0000","mapping_epoch":170,"log_start":"0'0","ondisk_log_start":"0'0","created":170,"last_epoch_clean":171,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T14:51:46.735123+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T14:51:46.735123+0000","last_clean_scrub_stamp":"2023-08-09T14:51:46.735123+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-11T00:26:44.647860+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"21.28","version":"0'0","reported_seq":9,"reported_epoch":171,"state":"active+clean","last_fresh":"2023-08-09T14:51:47.801095+0000","last_change":"2023-08-09T14:51:47.792988+0000","last_active":"2023-08-09T14:51:47.801095+0000","last_peered":"2023-08-09T14:51:47.801095+0000","last_clean":"2023-08-09T14:51:47.801095+0000","last_became_active":"2023-08-09T14:51:47.792695+0000","last_became_peered":"2023-08-09T14:51:47.792695+0000","last_unstale":"2023-08-09T14:51:47.801095+0000","last_undegraded":"2023-08-09T14:51:47.801095+0000","last_fullsized":"2023-08-09T14:51:47.801095+0000","mapping_epoch":170,"log_start":"0'0","ondisk_log_start":"0'0","created":170,"last_epoch_clean":171,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T14:51:46.735123+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T14:51:46.735123+0000","last_clean_scrub_stamp":"2023-08-09T14:51:46.735123+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T22:08:25.169397+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"21.27","version":"0'0","reported_seq":9,"reported_epoch":171,"state":"active+clean","last_fresh":"2023-08-09T14:51:47.805881+0000","last_change":"2023-08-09T14:51:47.798390+0000","last_active":"2023-08-09T14:51:47.805881+0000","last_peered":"2023-08-09T14:51:47.805881+0000","last_clean":"2023-08-09T14:51:47.805881+0000","last_became_active":"2023-08-09T14:51:47.797984+0000","last_became_peered":"2023-08-09T14:51:47.797984+0000","last_unstale":"2023-08-09T14:51:47.805881+0000","last_undegraded":"2023-08-09T14:51:47.805881+0000","last_fullsized":"2023-08-09T14:51:47.805881+0000","mapping_epoch":170,"log_start":"0'0","ondisk_log_start":"0'0","created":170,"last_epoch_clean":171,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T14:51:46.735123+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T14:51:46.735123+0000","last_clean_scrub_stamp":"2023-08-09T14:51:46.735123+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-11T01:24:05.556919+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"21.26","version":"0'0","reported_seq":9,"reported_epoch":171,"state":"active+clean","last_fresh":"2023-08-09T14:51:47.796002+0000","last_change":"2023-08-09T14:51:47.790314+0000","last_active":"2023-08-09T14:51:47.796002+0000","last_peered":"2023-08-09T14:51:47.796002+0000","last_clean":"2023-08-09T14:51:47.796002+0000","last_became_active":"2023-08-09T14:51:47.790015+0000","last_became_peered":"2023-08-09T14:51:47.790015+0000","last_unstale":"2023-08-09T14:51:47.796002+0000","last_undegraded":"2023-08-09T14:51:47.796002+0000","last_fullsized":"2023-08-09T14:51:47.796002+0000","mapping_epoch":170,"log_start":"0'0","ondisk_log_start":"0'0","created":170,"last_epoch_clean":171,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T14:51:46.735123+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T14:51:46.735123+0000","last_clean_scrub_stamp":"2023-08-09T14:51:46.735123+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T19:31:26.638403+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"21.25","version":"0'0","reported_seq":9,"reported_epoch":171,"state":"active+clean","last_fresh":"2023-08-09T14:51:47.805731+0000","last_change":"2023-08-09T14:51:47.800254+0000","last_active":"2023-08-09T14:51:47.805731+0000","last_peered":"2023-08-09T14:51:47.805731+0000","last_clean":"2023-08-09T14:51:47.805731+0000","last_became_active":"2023-08-09T14:51:47.799969+0000","last_became_peered":"2023-08-09T14:51:47.799969+0000","last_unstale":"2023-08-09T14:51:47.805731+0000","last_undegraded":"2023-08-09T14:51:47.805731+0000","last_fullsized":"2023-08-09T14:51:47.805731+0000","mapping_epoch":170,"log_start":"0'0","ondisk_log_start":"0'0","created":170,"last_epoch_clean":171,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T14:51:46.735123+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T14:51:46.735123+0000","last_clean_scrub_stamp":"2023-08-09T14:51:46.735123+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-11T00:44:32.393045+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"21.24","version":"0'0","reported_seq":9,"reported_epoch":171,"state":"active+clean","last_fresh":"2023-08-09T14:51:47.803141+0000","last_change":"2023-08-09T14:51:47.797986+0000","last_active":"2023-08-09T14:51:47.803141+0000","last_peered":"2023-08-09T14:51:47.803141+0000","last_clean":"2023-08-09T14:51:47.803141+0000","last_became_active":"2023-08-09T14:51:47.797710+0000","last_became_peered":"2023-08-09T14:51:47.797710+0000","last_unstale":"2023-08-09T14:51:47.803141+0000","last_undegraded":"2023-08-09T14:51:47.803141+0000","last_fullsized":"2023-08-09T14:51:47.803141+0000","mapping_epoch":170,"log_start":"0'0","ondisk_log_start":"0'0","created":170,"last_epoch_clean":171,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T14:51:46.735123+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T14:51:46.735123+0000","last_clean_scrub_stamp":"2023-08-09T14:51:46.735123+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T22:55:26.964070+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"21.23","version":"0'0","reported_seq":9,"reported_epoch":171,"state":"active+clean","last_fresh":"2023-08-09T14:51:47.798314+0000","last_change":"2023-08-09T14:51:47.794542+0000","last_active":"2023-08-09T14:51:47.798314+0000","last_peered":"2023-08-09T14:51:47.798314+0000","last_clean":"2023-08-09T14:51:47.798314+0000","last_became_active":"2023-08-09T14:51:47.794236+0000","last_became_peered":"2023-08-09T14:51:47.794236+0000","last_unstale":"2023-08-09T14:51:47.798314+0000","last_undegraded":"2023-08-09T14:51:47.798314+0000","last_fullsized":"2023-08-09T14:51:47.798314+0000","mapping_epoch":170,"log_start":"0'0","ondisk_log_start":"0'0","created":170,"last_epoch_clean":171,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T14:51:46.735123+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T14:51:46.735123+0000","last_clean_scrub_stamp":"2023-08-09T14:51:46.735123+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T20:06:05.899184+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"21.22","version":"0'0","reported_seq":9,"reported_epoch":171,"state":"active+clean","last_fresh":"2023-08-09T14:51:47.789893+0000","last_change":"2023-08-09T14:51:47.781712+0000","last_active":"2023-08-09T14:51:47.789893+0000","last_peered":"2023-08-09T14:51:47.789893+0000","last_clean":"2023-08-09T14:51:47.789893+0000","last_became_active":"2023-08-09T14:51:47.781050+0000","last_became_peered":"2023-08-09T14:51:47.781050+0000","last_unstale":"2023-08-09T14:51:47.789893+0000","last_undegraded":"2023-08-09T14:51:47.789893+0000","last_fullsized":"2023-08-09T14:51:47.789893+0000","mapping_epoch":170,"log_start":"0'0","ondisk_log_start":"0'0","created":170,"last_epoch_clean":171,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T14:51:46.735123+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T14:51:46.735123+0000","last_clean_scrub_stamp":"2023-08-09T14:51:46.735123+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T16:20:39.930781+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"21.21","version":"0'0","reported_seq":9,"reported_epoch":171,"state":"active+clean","last_fresh":"2023-08-09T14:51:47.796637+0000","last_change":"2023-08-09T14:51:47.795155+0000","last_active":"2023-08-09T14:51:47.796637+0000","last_peered":"2023-08-09T14:51:47.796637+0000","last_clean":"2023-08-09T14:51:47.796637+0000","last_became_active":"2023-08-09T14:51:47.794847+0000","last_became_peered":"2023-08-09T14:51:47.794847+0000","last_unstale":"2023-08-09T14:51:47.796637+0000","last_undegraded":"2023-08-09T14:51:47.796637+0000","last_fullsized":"2023-08-09T14:51:47.796637+0000","mapping_epoch":170,"log_start":"0'0","ondisk_log_start":"0'0","created":170,"last_epoch_clean":171,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T14:51:46.735123+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T14:51:46.735123+0000","last_clean_scrub_stamp":"2023-08-09T14:51:46.735123+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-11T01:07:09.488460+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"21.20","version":"0'0","reported_seq":9,"reported_epoch":171,"state":"active+clean","last_fresh":"2023-08-09T14:51:47.792921+0000","last_change":"2023-08-09T14:51:47.787101+0000","last_active":"2023-08-09T14:51:47.792921+0000","last_peered":"2023-08-09T14:51:47.792921+0000","last_clean":"2023-08-09T14:51:47.792921+0000","last_became_active":"2023-08-09T14:51:47.786782+0000","last_became_peered":"2023-08-09T14:51:47.786782+0000","last_unstale":"2023-08-09T14:51:47.792921+0000","last_undegraded":"2023-08-09T14:51:47.792921+0000","last_fullsized":"2023-08-09T14:51:47.792921+0000","mapping_epoch":170,"log_start":"0'0","ondisk_log_start":"0'0","created":170,"last_epoch_clean":171,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T14:51:46.735123+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T14:51:46.735123+0000","last_clean_scrub_stamp":"2023-08-09T14:51:46.735123+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T15:18:45.272412+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"21.1e","version":"0'0","reported_seq":9,"reported_epoch":171,"state":"active+clean","last_fresh":"2023-08-09T14:51:47.808887+0000","last_change":"2023-08-09T14:51:47.803915+0000","last_active":"2023-08-09T14:51:47.808887+0000","last_peered":"2023-08-09T14:51:47.808887+0000","last_clean":"2023-08-09T14:51:47.808887+0000","last_became_active":"2023-08-09T14:51:47.802801+0000","last_became_peered":"2023-08-09T14:51:47.802801+0000","last_unstale":"2023-08-09T14:51:47.808887+0000","last_undegraded":"2023-08-09T14:51:47.808887+0000","last_fullsized":"2023-08-09T14:51:47.808887+0000","mapping_epoch":170,"log_start":"0'0","ondisk_log_start":"0'0","created":170,"last_epoch_clean":171,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T14:51:46.735123+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T14:51:46.735123+0000","last_clean_scrub_stamp":"2023-08-09T14:51:46.735123+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T22:54:21.263039+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"20.1f","version":"171'4","reported_seq":15,"reported_epoch":171,"state":"active+clean","last_fresh":"2023-08-09T14:51:47.816596+0000","last_change":"2023-08-09T14:51:46.786260+0000","last_active":"2023-08-09T14:51:47.816596+0000","last_peered":"2023-08-09T14:51:47.816596+0000","last_clean":"2023-08-09T14:51:47.816596+0000","last_became_active":"2023-08-09T14:51:46.785985+0000","last_became_peered":"2023-08-09T14:51:46.785985+0000","last_unstale":"2023-08-09T14:51:47.816596+0000","last_undegraded":"2023-08-09T14:51:47.816596+0000","last_fullsized":"2023-08-09T14:51:47.816596+0000","mapping_epoch":169,"log_start":"0'0","ondisk_log_start":"0'0","created":169,"last_epoch_clean":170,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T14:51:45.719739+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T14:51:45.719739+0000","last_clean_scrub_stamp":"2023-08-09T14:51:45.719739+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T18:00:32.806143+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"21.1f","version":"0'0","reported_seq":9,"reported_epoch":171,"state":"active+clean","last_fresh":"2023-08-09T14:51:47.791972+0000","last_change":"2023-08-09T14:51:47.786043+0000","last_active":"2023-08-09T14:51:47.791972+0000","last_peered":"2023-08-09T14:51:47.791972+0000","last_clean":"2023-08-09T14:51:47.791972+0000","last_became_active":"2023-08-09T14:51:47.785765+0000","last_became_peered":"2023-08-09T14:51:47.785765+0000","last_unstale":"2023-08-09T14:51:47.791972+0000","last_undegraded":"2023-08-09T14:51:47.791972+0000","last_fullsized":"2023-08-09T14:51:47.791972+0000","mapping_epoch":170,"log_start":"0'0","ondisk_log_start":"0'0","created":170,"last_epoch_clean":171,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T14:51:46.735123+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T14:51:46.735123+0000","last_clean_scrub_stamp":"2023-08-09T14:51:46.735123+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T17:44:30.221845+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"20.1e","version":"171'1","reported_seq":12,"reported_epoch":171,"state":"active+clean","last_fresh":"2023-08-09T14:51:47.814540+0000","last_change":"2023-08-09T14:51:46.791336+0000","last_active":"2023-08-09T14:51:47.814540+0000","last_peered":"2023-08-09T14:51:47.814540+0000","last_clean":"2023-08-09T14:51:47.814540+0000","last_became_active":"2023-08-09T14:51:46.791007+0000","last_became_peered":"2023-08-09T14:51:46.791007+0000","last_unstale":"2023-08-09T14:51:47.814540+0000","last_undegraded":"2023-08-09T14:51:47.814540+0000","last_fullsized":"2023-08-09T14:51:47.814540+0000","mapping_epoch":169,"log_start":"0'0","ondisk_log_start":"0'0","created":169,"last_epoch_clean":170,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T14:51:45.719739+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T14:51:45.719739+0000","last_clean_scrub_stamp":"2023-08-09T14:51:45.719739+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-11T02:11:10.030977+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"21.1c","version":"0'0","reported_seq":9,"reported_epoch":171,"state":"active+clean","last_fresh":"2023-08-09T14:51:47.782971+0000","last_change":"2023-08-09T14:51:47.774632+0000","last_active":"2023-08-09T14:51:47.782971+0000","last_peered":"2023-08-09T14:51:47.782971+0000","last_clean":"2023-08-09T14:51:47.782971+0000","last_became_active":"2023-08-09T14:51:47.773892+0000","last_became_peered":"2023-08-09T14:51:47.773892+0000","last_unstale":"2023-08-09T14:51:47.782971+0000","last_undegraded":"2023-08-09T14:51:47.782971+0000","last_fullsized":"2023-08-09T14:51:47.782971+0000","mapping_epoch":170,"log_start":"0'0","ondisk_log_start":"0'0","created":170,"last_epoch_clean":171,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T14:51:46.735123+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T14:51:46.735123+0000","last_clean_scrub_stamp":"2023-08-09T14:51:46.735123+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T15:24:05.666551+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"20.1d","version":"171'2","reported_seq":13,"reported_epoch":171,"state":"active+clean","last_fresh":"2023-08-09T14:51:47.812306+0000","last_change":"2023-08-09T14:51:46.782906+0000","last_active":"2023-08-09T14:51:47.812306+0000","last_peered":"2023-08-09T14:51:47.812306+0000","last_clean":"2023-08-09T14:51:47.812306+0000","last_became_active":"2023-08-09T14:51:46.782616+0000","last_became_peered":"2023-08-09T14:51:46.782616+0000","last_unstale":"2023-08-09T14:51:47.812306+0000","last_undegraded":"2023-08-09T14:51:47.812306+0000","last_fullsized":"2023-08-09T14:51:47.812306+0000","mapping_epoch":169,"log_start":"0'0","ondisk_log_start":"0'0","created":169,"last_epoch_clean":170,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T14:51:45.719739+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T14:51:45.719739+0000","last_clean_scrub_stamp":"2023-08-09T14:51:45.719739+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T20:36:40.482735+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"21.1d","version":"0'0","reported_seq":9,"reported_epoch":171,"state":"active+clean","last_fresh":"2023-08-09T14:51:47.808530+0000","last_change":"2023-08-09T14:51:47.802494+0000","last_active":"2023-08-09T14:51:47.808530+0000","last_peered":"2023-08-09T14:51:47.808530+0000","last_clean":"2023-08-09T14:51:47.808530+0000","last_became_active":"2023-08-09T14:51:47.802101+0000","last_became_peered":"2023-08-09T14:51:47.802101+0000","last_unstale":"2023-08-09T14:51:47.808530+0000","last_undegraded":"2023-08-09T14:51:47.808530+0000","last_fullsized":"2023-08-09T14:51:47.808530+0000","mapping_epoch":170,"log_start":"0'0","ondisk_log_start":"0'0","created":170,"last_epoch_clean":171,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T14:51:46.735123+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T14:51:46.735123+0000","last_clean_scrub_stamp":"2023-08-09T14:51:46.735123+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T17:07:44.856176+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"20.1c","version":"171'2","reported_seq":13,"reported_epoch":171,"state":"active+clean","last_fresh":"2023-08-09T14:51:47.811760+0000","last_change":"2023-08-09T14:51:46.782922+0000","last_active":"2023-08-09T14:51:47.811760+0000","last_peered":"2023-08-09T14:51:47.811760+0000","last_clean":"2023-08-09T14:51:47.811760+0000","last_became_active":"2023-08-09T14:51:46.782289+0000","last_became_peered":"2023-08-09T14:51:46.782289+0000","last_unstale":"2023-08-09T14:51:47.811760+0000","last_undegraded":"2023-08-09T14:51:47.811760+0000","last_fullsized":"2023-08-09T14:51:47.811760+0000","mapping_epoch":169,"log_start":"0'0","ondisk_log_start":"0'0","created":169,"last_epoch_clean":170,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T14:51:45.719739+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T14:51:45.719739+0000","last_clean_scrub_stamp":"2023-08-09T14:51:45.719739+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-11T00:00:24.424384+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"21.1a","version":"0'0","reported_seq":9,"reported_epoch":171,"state":"active+clean","last_fresh":"2023-08-09T14:51:47.797277+0000","last_change":"2023-08-09T14:51:47.795851+0000","last_active":"2023-08-09T14:51:47.797277+0000","last_peered":"2023-08-09T14:51:47.797277+0000","last_clean":"2023-08-09T14:51:47.797277+0000","last_became_active":"2023-08-09T14:51:47.795514+0000","last_became_peered":"2023-08-09T14:51:47.795514+0000","last_unstale":"2023-08-09T14:51:47.797277+0000","last_undegraded":"2023-08-09T14:51:47.797277+0000","last_fullsized":"2023-08-09T14:51:47.797277+0000","mapping_epoch":170,"log_start":"0'0","ondisk_log_start":"0'0","created":170,"last_epoch_clean":171,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T14:51:46.735123+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T14:51:46.735123+0000","last_clean_scrub_stamp":"2023-08-09T14:51:46.735123+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-11T02:50:37.653241+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"20.1b","version":"171'1","reported_seq":12,"reported_epoch":171,"state":"active+clean","last_fresh":"2023-08-09T14:51:47.811597+0000","last_change":"2023-08-09T14:51:46.786986+0000","last_active":"2023-08-09T14:51:47.811597+0000","last_peered":"2023-08-09T14:51:47.811597+0000","last_clean":"2023-08-09T14:51:47.811597+0000","last_became_active":"2023-08-09T14:51:46.786728+0000","last_became_peered":"2023-08-09T14:51:46.786728+0000","last_unstale":"2023-08-09T14:51:47.811597+0000","last_undegraded":"2023-08-09T14:51:47.811597+0000","last_fullsized":"2023-08-09T14:51:47.811597+0000","mapping_epoch":169,"log_start":"0'0","ondisk_log_start":"0'0","created":169,"last_epoch_clean":170,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T14:51:45.719739+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T14:51:45.719739+0000","last_clean_scrub_stamp":"2023-08-09T14:51:45.719739+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-11T00:13:44.288100+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"21.1b","version":"0'0","reported_seq":9,"reported_epoch":171,"state":"active+clean","last_fresh":"2023-08-09T14:51:47.807896+0000","last_change":"2023-08-09T14:51:47.800335+0000","last_active":"2023-08-09T14:51:47.807896+0000","last_peered":"2023-08-09T14:51:47.807896+0000","last_clean":"2023-08-09T14:51:47.807896+0000","last_became_active":"2023-08-09T14:51:47.800029+0000","last_became_peered":"2023-08-09T14:51:47.800029+0000","last_unstale":"2023-08-09T14:51:47.807896+0000","last_undegraded":"2023-08-09T14:51:47.807896+0000","last_fullsized":"2023-08-09T14:51:47.807896+0000","mapping_epoch":170,"log_start":"0'0","ondisk_log_start":"0'0","created":170,"last_epoch_clean":171,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T14:51:46.735123+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T14:51:46.735123+0000","last_clean_scrub_stamp":"2023-08-09T14:51:46.735123+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T18:10:10.281829+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"20.1a","version":"171'1","reported_seq":12,"reported_epoch":171,"state":"active+clean","last_fresh":"2023-08-09T14:51:47.811066+0000","last_change":"2023-08-09T14:51:46.786553+0000","last_active":"2023-08-09T14:51:47.811066+0000","last_peered":"2023-08-09T14:51:47.811066+0000","last_clean":"2023-08-09T14:51:47.811066+0000","last_became_active":"2023-08-09T14:51:46.786191+0000","last_became_peered":"2023-08-09T14:51:46.786191+0000","last_unstale":"2023-08-09T14:51:47.811066+0000","last_undegraded":"2023-08-09T14:51:47.811066+0000","last_fullsized":"2023-08-09T14:51:47.811066+0000","mapping_epoch":169,"log_start":"0'0","ondisk_log_start":"0'0","created":169,"last_epoch_clean":170,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T14:51:45.719739+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T14:51:45.719739+0000","last_clean_scrub_stamp":"2023-08-09T14:51:45.719739+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T17:52:01.935407+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"21.18","version":"0'0","reported_seq":9,"reported_epoch":171,"state":"active+clean","last_fresh":"2023-08-09T14:51:47.783494+0000","last_change":"2023-08-09T14:51:47.780937+0000","last_active":"2023-08-09T14:51:47.783494+0000","last_peered":"2023-08-09T14:51:47.783494+0000","last_clean":"2023-08-09T14:51:47.783494+0000","last_became_active":"2023-08-09T14:51:47.780284+0000","last_became_peered":"2023-08-09T14:51:47.780284+0000","last_unstale":"2023-08-09T14:51:47.783494+0000","last_undegraded":"2023-08-09T14:51:47.783494+0000","last_fullsized":"2023-08-09T14:51:47.783494+0000","mapping_epoch":170,"log_start":"0'0","ondisk_log_start":"0'0","created":170,"last_epoch_clean":171,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T14:51:46.735123+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T14:51:46.735123+0000","last_clean_scrub_stamp":"2023-08-09T14:51:46.735123+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T16:34:26.749551+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"20.19","version":"0'0","reported_seq":11,"reported_epoch":171,"state":"active+clean","last_fresh":"2023-08-09T14:51:47.780773+0000","last_change":"2023-08-09T14:51:46.786597+0000","last_active":"2023-08-09T14:51:47.780773+0000","last_peered":"2023-08-09T14:51:47.780773+0000","last_clean":"2023-08-09T14:51:47.780773+0000","last_became_active":"2023-08-09T14:51:46.786259+0000","last_became_peered":"2023-08-09T14:51:46.786259+0000","last_unstale":"2023-08-09T14:51:47.780773+0000","last_undegraded":"2023-08-09T14:51:47.780773+0000","last_fullsized":"2023-08-09T14:51:47.780773+0000","mapping_epoch":169,"log_start":"0'0","ondisk_log_start":"0'0","created":169,"last_epoch_clean":170,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T14:51:45.719739+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T14:51:45.719739+0000","last_clean_scrub_stamp":"2023-08-09T14:51:45.719739+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-11T00:34:30.127008+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"21.19","version":"0'0","reported_seq":9,"reported_epoch":171,"state":"active+clean","last_fresh":"2023-08-09T14:51:47.797688+0000","last_change":"2023-08-09T14:51:47.796944+0000","last_active":"2023-08-09T14:51:47.797688+0000","last_peered":"2023-08-09T14:51:47.797688+0000","last_clean":"2023-08-09T14:51:47.797688+0000","last_became_active":"2023-08-09T14:51:47.796656+0000","last_became_peered":"2023-08-09T14:51:47.796656+0000","last_unstale":"2023-08-09T14:51:47.797688+0000","last_undegraded":"2023-08-09T14:51:47.797688+0000","last_fullsized":"2023-08-09T14:51:47.797688+0000","mapping_epoch":170,"log_start":"0'0","ondisk_log_start":"0'0","created":170,"last_epoch_clean":171,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T14:51:46.735123+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T14:51:46.735123+0000","last_clean_scrub_stamp":"2023-08-09T14:51:46.735123+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T22:29:15.151030+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"20.18","version":"0'0","reported_seq":11,"reported_epoch":171,"state":"active+clean","last_fresh":"2023-08-09T14:51:47.764939+0000","last_change":"2023-08-09T14:51:46.791440+0000","last_active":"2023-08-09T14:51:47.764939+0000","last_peered":"2023-08-09T14:51:47.764939+0000","last_clean":"2023-08-09T14:51:47.764939+0000","last_became_active":"2023-08-09T14:51:46.791030+0000","last_became_peered":"2023-08-09T14:51:46.791030+0000","last_unstale":"2023-08-09T14:51:47.764939+0000","last_undegraded":"2023-08-09T14:51:47.764939+0000","last_fullsized":"2023-08-09T14:51:47.764939+0000","mapping_epoch":169,"log_start":"0'0","ondisk_log_start":"0'0","created":169,"last_epoch_clean":170,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T14:51:45.719739+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T14:51:45.719739+0000","last_clean_scrub_stamp":"2023-08-09T14:51:45.719739+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T15:41:47.070932+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"21.e","version":"0'0","reported_seq":9,"reported_epoch":171,"state":"active+clean","last_fresh":"2023-08-09T14:51:47.808907+0000","last_change":"2023-08-09T14:51:47.804113+0000","last_active":"2023-08-09T14:51:47.808907+0000","last_peered":"2023-08-09T14:51:47.808907+0000","last_clean":"2023-08-09T14:51:47.808907+0000","last_became_active":"2023-08-09T14:51:47.803657+0000","last_became_peered":"2023-08-09T14:51:47.803657+0000","last_unstale":"2023-08-09T14:51:47.808907+0000","last_undegraded":"2023-08-09T14:51:47.808907+0000","last_fullsized":"2023-08-09T14:51:47.808907+0000","mapping_epoch":170,"log_start":"0'0","ondisk_log_start":"0'0","created":170,"last_epoch_clean":171,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T14:51:46.735123+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T14:51:46.735123+0000","last_clean_scrub_stamp":"2023-08-09T14:51:46.735123+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-11T01:54:05.813801+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"20.f","version":"171'1","reported_seq":12,"reported_epoch":171,"state":"active+clean","last_fresh":"2023-08-09T14:51:47.812348+0000","last_change":"2023-08-09T14:51:46.787215+0000","last_active":"2023-08-09T14:51:47.812348+0000","last_peered":"2023-08-09T14:51:47.812348+0000","last_clean":"2023-08-09T14:51:47.812348+0000","last_became_active":"2023-08-09T14:51:46.786660+0000","last_became_peered":"2023-08-09T14:51:46.786660+0000","last_unstale":"2023-08-09T14:51:47.812348+0000","last_undegraded":"2023-08-09T14:51:47.812348+0000","last_fullsized":"2023-08-09T14:51:47.812348+0000","mapping_epoch":169,"log_start":"0'0","ondisk_log_start":"0'0","created":169,"last_epoch_clean":170,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T14:51:45.719739+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T14:51:45.719739+0000","last_clean_scrub_stamp":"2023-08-09T14:51:45.719739+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T22:44:23.546686+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"21.f","version":"0'0","reported_seq":9,"reported_epoch":171,"state":"active+clean","last_fresh":"2023-08-09T14:51:47.798508+0000","last_change":"2023-08-09T14:51:47.794357+0000","last_active":"2023-08-09T14:51:47.798508+0000","last_peered":"2023-08-09T14:51:47.798508+0000","last_clean":"2023-08-09T14:51:47.798508+0000","last_became_active":"2023-08-09T14:51:47.794078+0000","last_became_peered":"2023-08-09T14:51:47.794078+0000","last_unstale":"2023-08-09T14:51:47.798508+0000","last_undegraded":"2023-08-09T14:51:47.798508+0000","last_fullsized":"2023-08-09T14:51:47.798508+0000","mapping_epoch":170,"log_start":"0'0","ondisk_log_start":"0'0","created":170,"last_epoch_clean":171,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T14:51:46.735123+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T14:51:46.735123+0000","last_clean_scrub_stamp":"2023-08-09T14:51:46.735123+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T20:02:17.166890+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"20.e","version":"171'2","reported_seq":13,"reported_epoch":171,"state":"active+clean","last_fresh":"2023-08-09T14:51:47.814490+0000","last_change":"2023-08-09T14:51:46.784640+0000","last_active":"2023-08-09T14:51:47.814490+0000","last_peered":"2023-08-09T14:51:47.814490+0000","last_clean":"2023-08-09T14:51:47.814490+0000","last_became_active":"2023-08-09T14:51:46.784368+0000","last_became_peered":"2023-08-09T14:51:46.784368+0000","last_unstale":"2023-08-09T14:51:47.814490+0000","last_undegraded":"2023-08-09T14:51:47.814490+0000","last_fullsized":"2023-08-09T14:51:47.814490+0000","mapping_epoch":169,"log_start":"0'0","ondisk_log_start":"0'0","created":169,"last_epoch_clean":170,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T14:51:45.719739+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T14:51:45.719739+0000","last_clean_scrub_stamp":"2023-08-09T14:51:45.719739+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T21:48:55.522611+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"21.c","version":"0'0","reported_seq":9,"reported_epoch":171,"state":"active+clean","last_fresh":"2023-08-09T14:51:47.798115+0000","last_change":"2023-08-09T14:51:47.794010+0000","last_active":"2023-08-09T14:51:47.798115+0000","last_peered":"2023-08-09T14:51:47.798115+0000","last_clean":"2023-08-09T14:51:47.798115+0000","last_became_active":"2023-08-09T14:51:47.793683+0000","last_became_peered":"2023-08-09T14:51:47.793683+0000","last_unstale":"2023-08-09T14:51:47.798115+0000","last_undegraded":"2023-08-09T14:51:47.798115+0000","last_fullsized":"2023-08-09T14:51:47.798115+0000","mapping_epoch":170,"log_start":"0'0","ondisk_log_start":"0'0","created":170,"last_epoch_clean":171,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T14:51:46.735123+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T14:51:46.735123+0000","last_clean_scrub_stamp":"2023-08-09T14:51:46.735123+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T15:23:38.410151+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"20.d","version":"171'1","reported_seq":12,"reported_epoch":171,"state":"active+clean","last_fresh":"2023-08-09T14:51:47.812644+0000","last_change":"2023-08-09T14:51:46.787142+0000","last_active":"2023-08-09T14:51:47.812644+0000","last_peered":"2023-08-09T14:51:47.812644+0000","last_clean":"2023-08-09T14:51:47.812644+0000","last_became_active":"2023-08-09T14:51:46.786118+0000","last_became_peered":"2023-08-09T14:51:46.786118+0000","last_unstale":"2023-08-09T14:51:47.812644+0000","last_undegraded":"2023-08-09T14:51:47.812644+0000","last_fullsized":"2023-08-09T14:51:47.812644+0000","mapping_epoch":169,"log_start":"0'0","ondisk_log_start":"0'0","created":169,"last_epoch_clean":170,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T14:51:45.719739+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T14:51:45.719739+0000","last_clean_scrub_stamp":"2023-08-09T14:51:45.719739+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T17:32:00.642586+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"21.d","version":"0'0","reported_seq":9,"reported_epoch":171,"state":"active+clean","last_fresh":"2023-08-09T14:51:47.805528+0000","last_change":"2023-08-09T14:51:47.798659+0000","last_active":"2023-08-09T14:51:47.805528+0000","last_peered":"2023-08-09T14:51:47.805528+0000","last_clean":"2023-08-09T14:51:47.805528+0000","last_became_active":"2023-08-09T14:51:47.798125+0000","last_became_peered":"2023-08-09T14:51:47.798125+0000","last_unstale":"2023-08-09T14:51:47.805528+0000","last_undegraded":"2023-08-09T14:51:47.805528+0000","last_fullsized":"2023-08-09T14:51:47.805528+0000","mapping_epoch":170,"log_start":"0'0","ondisk_log_start":"0'0","created":170,"last_epoch_clean":171,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T14:51:46.735123+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T14:51:46.735123+0000","last_clean_scrub_stamp":"2023-08-09T14:51:46.735123+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-11T01:33:44.144950+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"20.c","version":"0'0","reported_seq":11,"reported_epoch":171,"state":"active+clean","last_fresh":"2023-08-09T14:51:47.771635+0000","last_change":"2023-08-09T14:51:46.782861+0000","last_active":"2023-08-09T14:51:47.771635+0000","last_peered":"2023-08-09T14:51:47.771635+0000","last_clean":"2023-08-09T14:51:47.771635+0000","last_became_active":"2023-08-09T14:51:46.782554+0000","last_became_peered":"2023-08-09T14:51:46.782554+0000","last_unstale":"2023-08-09T14:51:47.771635+0000","last_undegraded":"2023-08-09T14:51:47.771635+0000","last_fullsized":"2023-08-09T14:51:47.771635+0000","mapping_epoch":169,"log_start":"0'0","ondisk_log_start":"0'0","created":169,"last_epoch_clean":170,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T14:51:45.719739+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T14:51:45.719739+0000","last_clean_scrub_stamp":"2023-08-09T14:51:45.719739+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-11T00:37:55.636435+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"21.a","version":"0'0","reported_seq":9,"reported_epoch":171,"state":"active+clean","last_fresh":"2023-08-09T14:51:47.789542+0000","last_change":"2023-08-09T14:51:47.781446+0000","last_active":"2023-08-09T14:51:47.789542+0000","last_peered":"2023-08-09T14:51:47.789542+0000","last_clean":"2023-08-09T14:51:47.789542+0000","last_became_active":"2023-08-09T14:51:47.780558+0000","last_became_peered":"2023-08-09T14:51:47.780558+0000","last_unstale":"2023-08-09T14:51:47.789542+0000","last_undegraded":"2023-08-09T14:51:47.789542+0000","last_fullsized":"2023-08-09T14:51:47.789542+0000","mapping_epoch":170,"log_start":"0'0","ondisk_log_start":"0'0","created":170,"last_epoch_clean":171,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T14:51:46.735123+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T14:51:46.735123+0000","last_clean_scrub_stamp":"2023-08-09T14:51:46.735123+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T22:35:53.052714+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"20.b","version":"171'2","reported_seq":13,"reported_epoch":171,"state":"active+clean","last_fresh":"2023-08-09T14:51:47.811852+0000","last_change":"2023-08-09T14:51:46.784391+0000","last_active":"2023-08-09T14:51:47.811852+0000","last_peered":"2023-08-09T14:51:47.811852+0000","last_clean":"2023-08-09T14:51:47.811852+0000","last_became_active":"2023-08-09T14:51:46.784094+0000","last_became_peered":"2023-08-09T14:51:46.784094+0000","last_unstale":"2023-08-09T14:51:47.811852+0000","last_undegraded":"2023-08-09T14:51:47.811852+0000","last_fullsized":"2023-08-09T14:51:47.811852+0000","mapping_epoch":169,"log_start":"0'0","ondisk_log_start":"0'0","created":169,"last_epoch_clean":170,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T14:51:45.719739+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T14:51:45.719739+0000","last_clean_scrub_stamp":"2023-08-09T14:51:45.719739+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T22:35:52.037330+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"21.b","version":"0'0","reported_seq":9,"reported_epoch":171,"state":"active+clean","last_fresh":"2023-08-09T14:51:47.802180+0000","last_change":"2023-08-09T14:51:47.796879+0000","last_active":"2023-08-09T14:51:47.802180+0000","last_peered":"2023-08-09T14:51:47.802180+0000","last_clean":"2023-08-09T14:51:47.802180+0000","last_became_active":"2023-08-09T14:51:47.796468+0000","last_became_peered":"2023-08-09T14:51:47.796468+0000","last_unstale":"2023-08-09T14:51:47.802180+0000","last_undegraded":"2023-08-09T14:51:47.802180+0000","last_fullsized":"2023-08-09T14:51:47.802180+0000","mapping_epoch":170,"log_start":"0'0","ondisk_log_start":"0'0","created":170,"last_epoch_clean":171,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T14:51:46.735123+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T14:51:46.735123+0000","last_clean_scrub_stamp":"2023-08-09T14:51:46.735123+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-11T01:13:33.380484+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"20.a","version":"0'0","reported_seq":11,"reported_epoch":171,"state":"active+clean","last_fresh":"2023-08-09T14:51:47.785757+0000","last_change":"2023-08-09T14:51:46.784518+0000","last_active":"2023-08-09T14:51:47.785757+0000","last_peered":"2023-08-09T14:51:47.785757+0000","last_clean":"2023-08-09T14:51:47.785757+0000","last_became_active":"2023-08-09T14:51:46.783848+0000","last_became_peered":"2023-08-09T14:51:46.783848+0000","last_unstale":"2023-08-09T14:51:47.785757+0000","last_undegraded":"2023-08-09T14:51:47.785757+0000","last_fullsized":"2023-08-09T14:51:47.785757+0000","mapping_epoch":169,"log_start":"0'0","ondisk_log_start":"0'0","created":169,"last_epoch_clean":170,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T14:51:45.719739+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T14:51:45.719739+0000","last_clean_scrub_stamp":"2023-08-09T14:51:45.719739+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T17:58:22.192246+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"21.8","version":"0'0","reported_seq":9,"reported_epoch":171,"state":"active+clean","last_fresh":"2023-08-09T14:51:47.791388+0000","last_change":"2023-08-09T14:51:47.787488+0000","last_active":"2023-08-09T14:51:47.791388+0000","last_peered":"2023-08-09T14:51:47.791388+0000","last_clean":"2023-08-09T14:51:47.791388+0000","last_became_active":"2023-08-09T14:51:47.786938+0000","last_became_peered":"2023-08-09T14:51:47.786938+0000","last_unstale":"2023-08-09T14:51:47.791388+0000","last_undegraded":"2023-08-09T14:51:47.791388+0000","last_fullsized":"2023-08-09T14:51:47.791388+0000","mapping_epoch":170,"log_start":"0'0","ondisk_log_start":"0'0","created":170,"last_epoch_clean":171,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T14:51:46.735123+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T14:51:46.735123+0000","last_clean_scrub_stamp":"2023-08-09T14:51:46.735123+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-11T02:18:22.001820+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"20.9","version":"0'0","reported_seq":11,"reported_epoch":171,"state":"active+clean","last_fresh":"2023-08-09T14:51:47.788252+0000","last_change":"2023-08-09T14:51:46.789049+0000","last_active":"2023-08-09T14:51:47.788252+0000","last_peered":"2023-08-09T14:51:47.788252+0000","last_clean":"2023-08-09T14:51:47.788252+0000","last_became_active":"2023-08-09T14:51:46.788778+0000","last_became_peered":"2023-08-09T14:51:46.788778+0000","last_unstale":"2023-08-09T14:51:47.788252+0000","last_undegraded":"2023-08-09T14:51:47.788252+0000","last_fullsized":"2023-08-09T14:51:47.788252+0000","mapping_epoch":169,"log_start":"0'0","ondisk_log_start":"0'0","created":169,"last_epoch_clean":170,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T14:51:45.719739+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T14:51:45.719739+0000","last_clean_scrub_stamp":"2023-08-09T14:51:45.719739+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T21:29:02.270845+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"21.9","version":"0'0","reported_seq":9,"reported_epoch":171,"state":"active+clean","last_fresh":"2023-08-09T14:51:47.802145+0000","last_change":"2023-08-09T14:51:47.795891+0000","last_active":"2023-08-09T14:51:47.802145+0000","last_peered":"2023-08-09T14:51:47.802145+0000","last_clean":"2023-08-09T14:51:47.802145+0000","last_became_active":"2023-08-09T14:51:47.795419+0000","last_became_peered":"2023-08-09T14:51:47.795419+0000","last_unstale":"2023-08-09T14:51:47.802145+0000","last_undegraded":"2023-08-09T14:51:47.802145+0000","last_fullsized":"2023-08-09T14:51:47.802145+0000","mapping_epoch":170,"log_start":"0'0","ondisk_log_start":"0'0","created":170,"last_epoch_clean":171,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T14:51:46.735123+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T14:51:46.735123+0000","last_clean_scrub_stamp":"2023-08-09T14:51:46.735123+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T15:33:27.498627+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"20.8","version":"0'0","reported_seq":11,"reported_epoch":171,"state":"active+clean","last_fresh":"2023-08-09T14:51:47.775212+0000","last_change":"2023-08-09T14:51:46.781845+0000","last_active":"2023-08-09T14:51:47.775212+0000","last_peered":"2023-08-09T14:51:47.775212+0000","last_clean":"2023-08-09T14:51:47.775212+0000","last_became_active":"2023-08-09T14:51:46.781489+0000","last_became_peered":"2023-08-09T14:51:46.781489+0000","last_unstale":"2023-08-09T14:51:47.775212+0000","last_undegraded":"2023-08-09T14:51:47.775212+0000","last_fullsized":"2023-08-09T14:51:47.775212+0000","mapping_epoch":169,"log_start":"0'0","ondisk_log_start":"0'0","created":169,"last_epoch_clean":170,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T14:51:45.719739+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T14:51:45.719739+0000","last_clean_scrub_stamp":"2023-08-09T14:51:45.719739+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-11T01:31:08.610990+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"21.6","version":"0'0","reported_seq":9,"reported_epoch":171,"state":"active+clean","last_fresh":"2023-08-09T14:51:47.799323+0000","last_change":"2023-08-09T14:51:47.797548+0000","last_active":"2023-08-09T14:51:47.799323+0000","last_peered":"2023-08-09T14:51:47.799323+0000","last_clean":"2023-08-09T14:51:47.799323+0000","last_became_active":"2023-08-09T14:51:47.797272+0000","last_became_peered":"2023-08-09T14:51:47.797272+0000","last_unstale":"2023-08-09T14:51:47.799323+0000","last_undegraded":"2023-08-09T14:51:47.799323+0000","last_fullsized":"2023-08-09T14:51:47.799323+0000","mapping_epoch":170,"log_start":"0'0","ondisk_log_start":"0'0","created":170,"last_epoch_clean":171,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T14:51:46.735123+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T14:51:46.735123+0000","last_clean_scrub_stamp":"2023-08-09T14:51:46.735123+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T22:51:09.744725+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"20.7","version":"171'2","reported_seq":13,"reported_epoch":171,"state":"active+clean","last_fresh":"2023-08-09T14:51:47.812435+0000","last_change":"2023-08-09T14:51:46.785516+0000","last_active":"2023-08-09T14:51:47.812435+0000","last_peered":"2023-08-09T14:51:47.812435+0000","last_clean":"2023-08-09T14:51:47.812435+0000","last_became_active":"2023-08-09T14:51:46.785234+0000","last_became_peered":"2023-08-09T14:51:46.785234+0000","last_unstale":"2023-08-09T14:51:47.812435+0000","last_undegraded":"2023-08-09T14:51:47.812435+0000","last_fullsized":"2023-08-09T14:51:47.812435+0000","mapping_epoch":169,"log_start":"0'0","ondisk_log_start":"0'0","created":169,"last_epoch_clean":170,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T14:51:45.719739+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T14:51:45.719739+0000","last_clean_scrub_stamp":"2023-08-09T14:51:45.719739+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T18:11:10.873473+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"21.7","version":"0'0","reported_seq":9,"reported_epoch":171,"state":"active+clean","last_fresh":"2023-08-09T14:51:47.798153+0000","last_change":"2023-08-09T14:51:47.794010+0000","last_active":"2023-08-09T14:51:47.798153+0000","last_peered":"2023-08-09T14:51:47.798153+0000","last_clean":"2023-08-09T14:51:47.798153+0000","last_became_active":"2023-08-09T14:51:47.793661+0000","last_became_peered":"2023-08-09T14:51:47.793661+0000","last_unstale":"2023-08-09T14:51:47.798153+0000","last_undegraded":"2023-08-09T14:51:47.798153+0000","last_fullsized":"2023-08-09T14:51:47.798153+0000","mapping_epoch":170,"log_start":"0'0","ondisk_log_start":"0'0","created":170,"last_epoch_clean":171,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T14:51:46.735123+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T14:51:46.735123+0000","last_clean_scrub_stamp":"2023-08-09T14:51:46.735123+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T19:10:19.642841+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"20.6","version":"0'0","reported_seq":11,"reported_epoch":171,"state":"active+clean","last_fresh":"2023-08-09T14:51:47.764656+0000","last_change":"2023-08-09T14:51:46.785316+0000","last_active":"2023-08-09T14:51:47.764656+0000","last_peered":"2023-08-09T14:51:47.764656+0000","last_clean":"2023-08-09T14:51:47.764656+0000","last_became_active":"2023-08-09T14:51:46.785034+0000","last_became_peered":"2023-08-09T14:51:46.785034+0000","last_unstale":"2023-08-09T14:51:47.764656+0000","last_undegraded":"2023-08-09T14:51:47.764656+0000","last_fullsized":"2023-08-09T14:51:47.764656+0000","mapping_epoch":169,"log_start":"0'0","ondisk_log_start":"0'0","created":169,"last_epoch_clean":170,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T14:51:45.719739+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T14:51:45.719739+0000","last_clean_scrub_stamp":"2023-08-09T14:51:45.719739+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T18:41:24.093109+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"21.4","version":"0'0","reported_seq":9,"reported_epoch":171,"state":"active+clean","last_fresh":"2023-08-09T14:51:47.792165+0000","last_change":"2023-08-09T14:51:47.787985+0000","last_active":"2023-08-09T14:51:47.792165+0000","last_peered":"2023-08-09T14:51:47.792165+0000","last_clean":"2023-08-09T14:51:47.792165+0000","last_became_active":"2023-08-09T14:51:47.787565+0000","last_became_peered":"2023-08-09T14:51:47.787565+0000","last_unstale":"2023-08-09T14:51:47.792165+0000","last_undegraded":"2023-08-09T14:51:47.792165+0000","last_fullsized":"2023-08-09T14:51:47.792165+0000","mapping_epoch":170,"log_start":"0'0","ondisk_log_start":"0'0","created":170,"last_epoch_clean":171,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T14:51:46.735123+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T14:51:46.735123+0000","last_clean_scrub_stamp":"2023-08-09T14:51:46.735123+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T20:55:40.272873+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"20.5","version":"0'0","reported_seq":11,"reported_epoch":171,"state":"active+clean","last_fresh":"2023-08-09T14:51:47.766489+0000","last_change":"2023-08-09T14:51:46.772657+0000","last_active":"2023-08-09T14:51:47.766489+0000","last_peered":"2023-08-09T14:51:47.766489+0000","last_clean":"2023-08-09T14:51:47.766489+0000","last_became_active":"2023-08-09T14:51:46.772339+0000","last_became_peered":"2023-08-09T14:51:46.772339+0000","last_unstale":"2023-08-09T14:51:47.766489+0000","last_undegraded":"2023-08-09T14:51:47.766489+0000","last_fullsized":"2023-08-09T14:51:47.766489+0000","mapping_epoch":169,"log_start":"0'0","ondisk_log_start":"0'0","created":169,"last_epoch_clean":170,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T14:51:45.719739+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T14:51:45.719739+0000","last_clean_scrub_stamp":"2023-08-09T14:51:45.719739+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T15:51:44.315128+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"21.5","version":"0'0","reported_seq":9,"reported_epoch":171,"state":"active+clean","last_fresh":"2023-08-09T14:51:47.791633+0000","last_change":"2023-08-09T14:51:47.787962+0000","last_active":"2023-08-09T14:51:47.791633+0000","last_peered":"2023-08-09T14:51:47.791633+0000","last_clean":"2023-08-09T14:51:47.791633+0000","last_became_active":"2023-08-09T14:51:47.787595+0000","last_became_peered":"2023-08-09T14:51:47.787595+0000","last_unstale":"2023-08-09T14:51:47.791633+0000","last_undegraded":"2023-08-09T14:51:47.791633+0000","last_fullsized":"2023-08-09T14:51:47.791633+0000","mapping_epoch":170,"log_start":"0'0","ondisk_log_start":"0'0","created":170,"last_epoch_clean":171,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T14:51:46.735123+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T14:51:46.735123+0000","last_clean_scrub_stamp":"2023-08-09T14:51:46.735123+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T16:37:00.297341+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"20.4","version":"0'0","reported_seq":11,"reported_epoch":171,"state":"active+clean","last_fresh":"2023-08-09T14:51:47.763337+0000","last_change":"2023-08-09T14:51:46.772250+0000","last_active":"2023-08-09T14:51:47.763337+0000","last_peered":"2023-08-09T14:51:47.763337+0000","last_clean":"2023-08-09T14:51:47.763337+0000","last_became_active":"2023-08-09T14:51:46.771906+0000","last_became_peered":"2023-08-09T14:51:46.771906+0000","last_unstale":"2023-08-09T14:51:47.763337+0000","last_undegraded":"2023-08-09T14:51:47.763337+0000","last_fullsized":"2023-08-09T14:51:47.763337+0000","mapping_epoch":169,"log_start":"0'0","ondisk_log_start":"0'0","created":169,"last_epoch_clean":170,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T14:51:45.719739+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T14:51:45.719739+0000","last_clean_scrub_stamp":"2023-08-09T14:51:45.719739+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T15:27:12.471209+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"21.2","version":"0'0","reported_seq":9,"reported_epoch":171,"state":"active+clean","last_fresh":"2023-08-09T14:51:47.798567+0000","last_change":"2023-08-09T14:51:47.795963+0000","last_active":"2023-08-09T14:51:47.798567+0000","last_peered":"2023-08-09T14:51:47.798567+0000","last_clean":"2023-08-09T14:51:47.798567+0000","last_became_active":"2023-08-09T14:51:47.795647+0000","last_became_peered":"2023-08-09T14:51:47.795647+0000","last_unstale":"2023-08-09T14:51:47.798567+0000","last_undegraded":"2023-08-09T14:51:47.798567+0000","last_fullsized":"2023-08-09T14:51:47.798567+0000","mapping_epoch":170,"log_start":"0'0","ondisk_log_start":"0'0","created":170,"last_epoch_clean":171,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T14:51:46.735123+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T14:51:46.735123+0000","last_clean_scrub_stamp":"2023-08-09T14:51:46.735123+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T18:22:58.458491+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"20.3","version":"0'0","reported_seq":11,"reported_epoch":171,"state":"active+clean","last_fresh":"2023-08-09T14:51:47.778873+0000","last_change":"2023-08-09T14:51:46.784221+0000","last_active":"2023-08-09T14:51:47.778873+0000","last_peered":"2023-08-09T14:51:47.778873+0000","last_clean":"2023-08-09T14:51:47.778873+0000","last_became_active":"2023-08-09T14:51:46.783914+0000","last_became_peered":"2023-08-09T14:51:46.783914+0000","last_unstale":"2023-08-09T14:51:47.778873+0000","last_undegraded":"2023-08-09T14:51:47.778873+0000","last_fullsized":"2023-08-09T14:51:47.778873+0000","mapping_epoch":169,"log_start":"0'0","ondisk_log_start":"0'0","created":169,"last_epoch_clean":170,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T14:51:45.719739+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T14:51:45.719739+0000","last_clean_scrub_stamp":"2023-08-09T14:51:45.719739+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-11T00:26:43.632475+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"21.3","version":"0'0","reported_seq":9,"reported_epoch":171,"state":"active+clean","last_fresh":"2023-08-09T14:51:47.800072+0000","last_change":"2023-08-09T14:51:47.799128+0000","last_active":"2023-08-09T14:51:47.800072+0000","last_peered":"2023-08-09T14:51:47.800072+0000","last_clean":"2023-08-09T14:51:47.800072+0000","last_became_active":"2023-08-09T14:51:47.798771+0000","last_became_peered":"2023-08-09T14:51:47.798771+0000","last_unstale":"2023-08-09T14:51:47.800072+0000","last_undegraded":"2023-08-09T14:51:47.800072+0000","last_fullsized":"2023-08-09T14:51:47.800072+0000","mapping_epoch":170,"log_start":"0'0","ondisk_log_start":"0'0","created":170,"last_epoch_clean":171,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T14:51:46.735123+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T14:51:46.735123+0000","last_clean_scrub_stamp":"2023-08-09T14:51:46.735123+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T17:30:38.007650+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"20.2","version":"0'0","reported_seq":11,"reported_epoch":171,"state":"active+clean","last_fresh":"2023-08-09T14:51:47.785960+0000","last_change":"2023-08-09T14:51:46.784750+0000","last_active":"2023-08-09T14:51:47.785960+0000","last_peered":"2023-08-09T14:51:47.785960+0000","last_clean":"2023-08-09T14:51:47.785960+0000","last_became_active":"2023-08-09T14:51:46.784230+0000","last_became_peered":"2023-08-09T14:51:46.784230+0000","last_unstale":"2023-08-09T14:51:47.785960+0000","last_undegraded":"2023-08-09T14:51:47.785960+0000","last_fullsized":"2023-08-09T14:51:47.785960+0000","mapping_epoch":169,"log_start":"0'0","ondisk_log_start":"0'0","created":169,"last_epoch_clean":170,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T14:51:45.719739+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T14:51:45.719739+0000","last_clean_scrub_stamp":"2023-08-09T14:51:45.719739+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T21:49:35.633438+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"21.0","version":"0'0","reported_seq":9,"reported_epoch":171,"state":"active+clean","last_fresh":"2023-08-09T14:51:47.805333+0000","last_change":"2023-08-09T14:51:47.796432+0000","last_active":"2023-08-09T14:51:47.805333+0000","last_peered":"2023-08-09T14:51:47.805333+0000","last_clean":"2023-08-09T14:51:47.805333+0000","last_became_active":"2023-08-09T14:51:47.796129+0000","last_became_peered":"2023-08-09T14:51:47.796129+0000","last_unstale":"2023-08-09T14:51:47.805333+0000","last_undegraded":"2023-08-09T14:51:47.805333+0000","last_fullsized":"2023-08-09T14:51:47.805333+0000","mapping_epoch":170,"log_start":"0'0","ondisk_log_start":"0'0","created":170,"last_epoch_clean":171,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T14:51:46.735123+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T14:51:46.735123+0000","last_clean_scrub_stamp":"2023-08-09T14:51:46.735123+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T21:12:37.780571+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"20.1","version":"0'0","reported_seq":11,"reported_epoch":171,"state":"active+clean","last_fresh":"2023-08-09T14:51:47.776812+0000","last_change":"2023-08-09T14:51:46.783800+0000","last_active":"2023-08-09T14:51:47.776812+0000","last_peered":"2023-08-09T14:51:47.776812+0000","last_clean":"2023-08-09T14:51:47.776812+0000","last_became_active":"2023-08-09T14:51:46.783530+0000","last_became_peered":"2023-08-09T14:51:46.783530+0000","last_unstale":"2023-08-09T14:51:47.776812+0000","last_undegraded":"2023-08-09T14:51:47.776812+0000","last_fullsized":"2023-08-09T14:51:47.776812+0000","mapping_epoch":169,"log_start":"0'0","ondisk_log_start":"0'0","created":169,"last_epoch_clean":170,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T14:51:45.719739+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T14:51:45.719739+0000","last_clean_scrub_stamp":"2023-08-09T14:51:45.719739+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T21:43:59.706915+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"21.1","version":"0'0","reported_seq":9,"reported_epoch":171,"state":"active+clean","last_fresh":"2023-08-09T14:51:47.788039+0000","last_change":"2023-08-09T14:51:47.786151+0000","last_active":"2023-08-09T14:51:47.788039+0000","last_peered":"2023-08-09T14:51:47.788039+0000","last_clean":"2023-08-09T14:51:47.788039+0000","last_became_active":"2023-08-09T14:51:47.785803+0000","last_became_peered":"2023-08-09T14:51:47.785803+0000","last_unstale":"2023-08-09T14:51:47.788039+0000","last_undegraded":"2023-08-09T14:51:47.788039+0000","last_fullsized":"2023-08-09T14:51:47.788039+0000","mapping_epoch":170,"log_start":"0'0","ondisk_log_start":"0'0","created":170,"last_epoch_clean":171,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T14:51:46.735123+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T14:51:46.735123+0000","last_clean_scrub_stamp":"2023-08-09T14:51:46.735123+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T20:40:40.159710+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"20.0","version":"0'0","reported_seq":11,"reported_epoch":171,"state":"active+clean","last_fresh":"2023-08-09T14:51:47.770122+0000","last_change":"2023-08-09T14:51:46.791268+0000","last_active":"2023-08-09T14:51:47.770122+0000","last_peered":"2023-08-09T14:51:47.770122+0000","last_clean":"2023-08-09T14:51:47.770122+0000","last_became_active":"2023-08-09T14:51:46.790960+0000","last_became_peered":"2023-08-09T14:51:46.790960+0000","last_unstale":"2023-08-09T14:51:47.770122+0000","last_undegraded":"2023-08-09T14:51:47.770122+0000","last_fullsized":"2023-08-09T14:51:47.770122+0000","mapping_epoch":169,"log_start":"0'0","ondisk_log_start":"0'0","created":169,"last_epoch_clean":170,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T14:51:45.719739+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T14:51:45.719739+0000","last_clean_scrub_stamp":"2023-08-09T14:51:45.719739+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T17:43:27.675754+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"21.14","version":"0'0","reported_seq":9,"reported_epoch":171,"state":"active+clean","last_fresh":"2023-08-09T14:51:47.798347+0000","last_change":"2023-08-09T14:51:47.791201+0000","last_active":"2023-08-09T14:51:47.798347+0000","last_peered":"2023-08-09T14:51:47.798347+0000","last_clean":"2023-08-09T14:51:47.798347+0000","last_became_active":"2023-08-09T14:51:47.790897+0000","last_became_peered":"2023-08-09T14:51:47.790897+0000","last_unstale":"2023-08-09T14:51:47.798347+0000","last_undegraded":"2023-08-09T14:51:47.798347+0000","last_fullsized":"2023-08-09T14:51:47.798347+0000","mapping_epoch":170,"log_start":"0'0","ondisk_log_start":"0'0","created":170,"last_epoch_clean":171,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T14:51:46.735123+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T14:51:46.735123+0000","last_clean_scrub_stamp":"2023-08-09T14:51:46.735123+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T20:42:16.832652+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"20.15","version":"171'3","reported_seq":14,"reported_epoch":171,"state":"active+clean","last_fresh":"2023-08-09T14:51:47.814593+0000","last_change":"2023-08-09T14:51:46.782009+0000","last_active":"2023-08-09T14:51:47.814593+0000","last_peered":"2023-08-09T14:51:47.814593+0000","last_clean":"2023-08-09T14:51:47.814593+0000","last_became_active":"2023-08-09T14:51:46.781722+0000","last_became_peered":"2023-08-09T14:51:46.781722+0000","last_unstale":"2023-08-09T14:51:47.814593+0000","last_undegraded":"2023-08-09T14:51:47.814593+0000","last_fullsized":"2023-08-09T14:51:47.814593+0000","mapping_epoch":169,"log_start":"0'0","ondisk_log_start":"0'0","created":169,"last_epoch_clean":170,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T14:51:45.719739+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T14:51:45.719739+0000","last_clean_scrub_stamp":"2023-08-09T14:51:45.719739+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T16:54:04.005809+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":319,"reported_epoch":171,"state":"active+clean","last_fresh":"2023-08-09T14:51:47.786219+0000","last_change":"2023-08-09T13:33:56.401406+0000","last_active":"2023-08-09T14:51:47.786219+0000","last_peered":"2023-08-09T14:51:47.786219+0000","last_clean":"2023-08-09T14:51:47.786219+0000","last_became_active":"2023-08-09T13:33:56.400487+0000","last_became_peered":"2023-08-09T13:33:56.400487+0000","last_unstale":"2023-08-09T14:51:47.786219+0000","last_undegraded":"2023-08-09T14:51:47.786219+0000","last_fullsized":"2023-08-09T14:51:47.786219+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:33:54.845869+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:33:54.845869+0000","last_clean_scrub_stamp":"2023-08-09T13:33:54.845869+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T22:57:53.864171+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"21.15","version":"0'0","reported_seq":9,"reported_epoch":171,"state":"active+clean","last_fresh":"2023-08-09T14:51:47.795847+0000","last_change":"2023-08-09T14:51:47.788086+0000","last_active":"2023-08-09T14:51:47.795847+0000","last_peered":"2023-08-09T14:51:47.795847+0000","last_clean":"2023-08-09T14:51:47.795847+0000","last_became_active":"2023-08-09T14:51:47.787696+0000","last_became_peered":"2023-08-09T14:51:47.787696+0000","last_unstale":"2023-08-09T14:51:47.795847+0000","last_undegraded":"2023-08-09T14:51:47.795847+0000","last_fullsized":"2023-08-09T14:51:47.795847+0000","mapping_epoch":170,"log_start":"0'0","ondisk_log_start":"0'0","created":170,"last_epoch_clean":171,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T14:51:46.735123+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T14:51:46.735123+0000","last_clean_scrub_stamp":"2023-08-09T14:51:46.735123+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-11T02:37:18.590003+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"20.14","version":"171'6","reported_seq":17,"reported_epoch":171,"state":"active+clean","last_fresh":"2023-08-09T14:51:47.817171+0000","last_change":"2023-08-09T14:51:46.772775+0000","last_active":"2023-08-09T14:51:47.817171+0000","last_peered":"2023-08-09T14:51:47.817171+0000","last_clean":"2023-08-09T14:51:47.817171+0000","last_became_active":"2023-08-09T14:51:46.772463+0000","last_became_peered":"2023-08-09T14:51:46.772463+0000","last_unstale":"2023-08-09T14:51:47.817171+0000","last_undegraded":"2023-08-09T14:51:47.817171+0000","last_fullsized":"2023-08-09T14:51:47.817171+0000","mapping_epoch":169,"log_start":"0'0","ondisk_log_start":"0'0","created":169,"last_epoch_clean":170,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T14:51:45.719739+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T14:51:45.719739+0000","last_clean_scrub_stamp":"2023-08-09T14:51:45.719739+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-11T02:18:20.986435+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":326,"reported_epoch":171,"state":"active+clean","last_fresh":"2023-08-09T14:51:47.772371+0000","last_change":"2023-08-09T13:33:56.401123+0000","last_active":"2023-08-09T14:51:47.772371+0000","last_peered":"2023-08-09T14:51:47.772371+0000","last_clean":"2023-08-09T14:51:47.772371+0000","last_became_active":"2023-08-09T13:33:56.399933+0000","last_became_peered":"2023-08-09T13:33:56.399933+0000","last_unstale":"2023-08-09T14:51:47.772371+0000","last_undegraded":"2023-08-09T14:51:47.772371+0000","last_fullsized":"2023-08-09T14:51:47.772371+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:33:54.845869+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:33:54.845869+0000","last_clean_scrub_stamp":"2023-08-09T13:33:54.845869+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-11T01:21:08.313655+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"21.16","version":"0'0","reported_seq":9,"reported_epoch":171,"state":"active+clean","last_fresh":"2023-08-09T14:51:47.797665+0000","last_change":"2023-08-09T14:51:47.792022+0000","last_active":"2023-08-09T14:51:47.797665+0000","last_peered":"2023-08-09T14:51:47.797665+0000","last_clean":"2023-08-09T14:51:47.797665+0000","last_became_active":"2023-08-09T14:51:47.788230+0000","last_became_peered":"2023-08-09T14:51:47.788230+0000","last_unstale":"2023-08-09T14:51:47.797665+0000","last_undegraded":"2023-08-09T14:51:47.797665+0000","last_fullsized":"2023-08-09T14:51:47.797665+0000","mapping_epoch":170,"log_start":"0'0","ondisk_log_start":"0'0","created":170,"last_epoch_clean":171,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T14:51:46.735123+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T14:51:46.735123+0000","last_clean_scrub_stamp":"2023-08-09T14:51:46.735123+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T22:10:25.748586+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"20.17","version":"0'0","reported_seq":11,"reported_epoch":171,"state":"active+clean","last_fresh":"2023-08-09T14:51:47.772166+0000","last_change":"2023-08-09T14:51:46.780599+0000","last_active":"2023-08-09T14:51:47.772166+0000","last_peered":"2023-08-09T14:51:47.772166+0000","last_clean":"2023-08-09T14:51:47.772166+0000","last_became_active":"2023-08-09T14:51:46.780275+0000","last_became_peered":"2023-08-09T14:51:46.780275+0000","last_unstale":"2023-08-09T14:51:47.772166+0000","last_undegraded":"2023-08-09T14:51:47.772166+0000","last_fullsized":"2023-08-09T14:51:47.772166+0000","mapping_epoch":169,"log_start":"0'0","ondisk_log_start":"0'0","created":169,"last_epoch_clean":170,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T14:51:45.719739+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T14:51:45.719739+0000","last_clean_scrub_stamp":"2023-08-09T14:51:45.719739+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T18:46:48.463911+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.2","version":"0'0","reported_seq":318,"reported_epoch":171,"state":"active+clean","last_fresh":"2023-08-09T14:51:47.786081+0000","last_change":"2023-08-09T13:33:56.402794+0000","last_active":"2023-08-09T14:51:47.786081+0000","last_peered":"2023-08-09T14:51:47.786081+0000","last_clean":"2023-08-09T14:51:47.786081+0000","last_became_active":"2023-08-09T13:33:56.402038+0000","last_became_peered":"2023-08-09T13:33:56.402038+0000","last_unstale":"2023-08-09T14:51:47.786081+0000","last_undegraded":"2023-08-09T14:51:47.786081+0000","last_fullsized":"2023-08-09T14:51:47.786081+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:33:54.845869+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:33:54.845869+0000","last_clean_scrub_stamp":"2023-08-09T13:33:54.845869+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T22:32:04.366882+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"21.17","version":"0'0","reported_seq":9,"reported_epoch":171,"state":"active+clean","last_fresh":"2023-08-09T14:51:47.805471+0000","last_change":"2023-08-09T14:51:47.798518+0000","last_active":"2023-08-09T14:51:47.805471+0000","last_peered":"2023-08-09T14:51:47.805471+0000","last_clean":"2023-08-09T14:51:47.805471+0000","last_became_active":"2023-08-09T14:51:47.797907+0000","last_became_peered":"2023-08-09T14:51:47.797907+0000","last_unstale":"2023-08-09T14:51:47.805471+0000","last_undegraded":"2023-08-09T14:51:47.805471+0000","last_fullsized":"2023-08-09T14:51:47.805471+0000","mapping_epoch":170,"log_start":"0'0","ondisk_log_start":"0'0","created":170,"last_epoch_clean":171,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T14:51:46.735123+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T14:51:46.735123+0000","last_clean_scrub_stamp":"2023-08-09T14:51:46.735123+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T17:57:19.553018+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"20.16","version":"0'0","reported_seq":11,"reported_epoch":171,"state":"active+clean","last_fresh":"2023-08-09T14:51:47.788083+0000","last_change":"2023-08-09T14:51:46.790916+0000","last_active":"2023-08-09T14:51:47.788083+0000","last_peered":"2023-08-09T14:51:47.788083+0000","last_clean":"2023-08-09T14:51:47.788083+0000","last_became_active":"2023-08-09T14:51:46.790560+0000","last_became_peered":"2023-08-09T14:51:46.790560+0000","last_unstale":"2023-08-09T14:51:47.788083+0000","last_undegraded":"2023-08-09T14:51:47.788083+0000","last_fullsized":"2023-08-09T14:51:47.788083+0000","mapping_epoch":169,"log_start":"0'0","ondisk_log_start":"0'0","created":169,"last_epoch_clean":170,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T14:51:45.719739+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T14:51:45.719739+0000","last_clean_scrub_stamp":"2023-08-09T14:51:45.719739+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T21:59:29.373187+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.3","version":"0'0","reported_seq":326,"reported_epoch":171,"state":"active+clean","last_fresh":"2023-08-09T14:51:47.763518+0000","last_change":"2023-08-09T13:33:55.886749+0000","last_active":"2023-08-09T14:51:47.763518+0000","last_peered":"2023-08-09T14:51:47.763518+0000","last_clean":"2023-08-09T14:51:47.763518+0000","last_became_active":"2023-08-09T13:33:55.886116+0000","last_became_peered":"2023-08-09T13:33:55.886116+0000","last_unstale":"2023-08-09T14:51:47.763518+0000","last_undegraded":"2023-08-09T14:51:47.763518+0000","last_fullsized":"2023-08-09T14:51:47.763518+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:33:54.845869+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:33:54.845869+0000","last_clean_scrub_stamp":"2023-08-09T13:33:54.845869+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T23:32:37.214710+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"160'252","reported_seq":947,"reported_epoch":171,"state":"active+clean","last_fresh":"2023-08-09T14:51:47.785811+0000","last_change":"2023-08-09T13:33:55.886345+0000","last_active":"2023-08-09T14:51:47.785811+0000","last_peered":"2023-08-09T14:51:47.785811+0000","last_clean":"2023-08-09T14:51:47.785811+0000","last_became_active":"2023-08-09T13:33:55.885781+0000","last_became_peered":"2023-08-09T13:33:55.885781+0000","last_unstale":"2023-08-09T14:51:47.785811+0000","last_undegraded":"2023-08-09T14:51:47.785811+0000","last_fullsized":"2023-08-09T14:51:47.785811+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:33:54.845869+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:33:54.845869+0000","last_clean_scrub_stamp":"2023-08-09T13:33:54.845869+0000","objects_scrubbed":0,"log_size":252,"log_dups_size":0,"ondisk_log_size":252,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T14:13:22.909308+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1291,"num_read_kb":2654,"num_write":403,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"21.10","version":"0'0","reported_seq":9,"reported_epoch":171,"state":"active+clean","last_fresh":"2023-08-09T14:51:47.799484+0000","last_change":"2023-08-09T14:51:47.792011+0000","last_active":"2023-08-09T14:51:47.799484+0000","last_peered":"2023-08-09T14:51:47.799484+0000","last_clean":"2023-08-09T14:51:47.799484+0000","last_became_active":"2023-08-09T14:51:47.791690+0000","last_became_peered":"2023-08-09T14:51:47.791690+0000","last_unstale":"2023-08-09T14:51:47.799484+0000","last_undegraded":"2023-08-09T14:51:47.799484+0000","last_fullsized":"2023-08-09T14:51:47.799484+0000","mapping_epoch":170,"log_start":"0'0","ondisk_log_start":"0'0","created":170,"last_epoch_clean":171,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T14:51:46.735123+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T14:51:46.735123+0000","last_clean_scrub_stamp":"2023-08-09T14:51:46.735123+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T18:07:44.866766+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"20.11","version":"0'0","reported_seq":11,"reported_epoch":171,"state":"active+clean","last_fresh":"2023-08-09T14:51:47.762267+0000","last_change":"2023-08-09T14:51:46.771405+0000","last_active":"2023-08-09T14:51:47.762267+0000","last_peered":"2023-08-09T14:51:47.762267+0000","last_clean":"2023-08-09T14:51:47.762267+0000","last_became_active":"2023-08-09T14:51:46.770725+0000","last_became_peered":"2023-08-09T14:51:46.770725+0000","last_unstale":"2023-08-09T14:51:47.762267+0000","last_undegraded":"2023-08-09T14:51:47.762267+0000","last_fullsized":"2023-08-09T14:51:47.762267+0000","mapping_epoch":169,"log_start":"0'0","ondisk_log_start":"0'0","created":169,"last_epoch_clean":170,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T14:51:45.719739+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T14:51:45.719739+0000","last_clean_scrub_stamp":"2023-08-09T14:51:45.719739+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T23:49:28.629389+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":328,"reported_epoch":171,"state":"active+clean","last_fresh":"2023-08-09T14:51:47.771842+0000","last_change":"2023-08-09T13:33:55.889565+0000","last_active":"2023-08-09T14:51:47.771842+0000","last_peered":"2023-08-09T14:51:47.771842+0000","last_clean":"2023-08-09T14:51:47.771842+0000","last_became_active":"2023-08-09T13:33:55.889164+0000","last_became_peered":"2023-08-09T13:33:55.889164+0000","last_unstale":"2023-08-09T14:51:47.771842+0000","last_undegraded":"2023-08-09T14:51:47.771842+0000","last_fullsized":"2023-08-09T14:51:47.771842+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:33:54.845869+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:33:54.845869+0000","last_clean_scrub_stamp":"2023-08-09T13:33:54.845869+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T14:46:14.719093+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"21.11","version":"0'0","reported_seq":9,"reported_epoch":171,"state":"active+clean","last_fresh":"2023-08-09T14:51:47.804603+0000","last_change":"2023-08-09T14:51:47.799276+0000","last_active":"2023-08-09T14:51:47.804603+0000","last_peered":"2023-08-09T14:51:47.804603+0000","last_clean":"2023-08-09T14:51:47.804603+0000","last_became_active":"2023-08-09T14:51:47.798952+0000","last_became_peered":"2023-08-09T14:51:47.798952+0000","last_unstale":"2023-08-09T14:51:47.804603+0000","last_undegraded":"2023-08-09T14:51:47.804603+0000","last_fullsized":"2023-08-09T14:51:47.804603+0000","mapping_epoch":170,"log_start":"0'0","ondisk_log_start":"0'0","created":170,"last_epoch_clean":171,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T14:51:46.735123+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T14:51:46.735123+0000","last_clean_scrub_stamp":"2023-08-09T14:51:46.735123+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T16:55:27.430423+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"20.10","version":"171'8","reported_seq":19,"reported_epoch":171,"state":"active+clean","last_fresh":"2023-08-09T14:51:47.814870+0000","last_change":"2023-08-09T14:51:46.783858+0000","last_active":"2023-08-09T14:51:47.814870+0000","last_peered":"2023-08-09T14:51:47.814870+0000","last_clean":"2023-08-09T14:51:47.814870+0000","last_became_active":"2023-08-09T14:51:46.783565+0000","last_became_peered":"2023-08-09T14:51:46.783565+0000","last_unstale":"2023-08-09T14:51:47.814870+0000","last_undegraded":"2023-08-09T14:51:47.814870+0000","last_fullsized":"2023-08-09T14:51:47.814870+0000","mapping_epoch":169,"log_start":"0'0","ondisk_log_start":"0'0","created":169,"last_epoch_clean":170,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T14:51:45.719739+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T14:51:45.719739+0000","last_clean_scrub_stamp":"2023-08-09T14:51:45.719739+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-11T01:11:24.931250+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":324,"reported_epoch":171,"state":"active+clean","last_fresh":"2023-08-09T14:51:47.764985+0000","last_change":"2023-08-09T13:33:56.404236+0000","last_active":"2023-08-09T14:51:47.764985+0000","last_peered":"2023-08-09T14:51:47.764985+0000","last_clean":"2023-08-09T14:51:47.764985+0000","last_became_active":"2023-08-09T13:33:56.402485+0000","last_became_peered":"2023-08-09T13:33:56.402485+0000","last_unstale":"2023-08-09T14:51:47.764985+0000","last_undegraded":"2023-08-09T14:51:47.764985+0000","last_fullsized":"2023-08-09T14:51:47.764985+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:33:54.845869+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:33:54.845869+0000","last_clean_scrub_stamp":"2023-08-09T13:33:54.845869+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T14:26:21.684948+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"21.12","version":"0'0","reported_seq":9,"reported_epoch":171,"state":"active+clean","last_fresh":"2023-08-09T14:51:47.794052+0000","last_change":"2023-08-09T14:51:47.788323+0000","last_active":"2023-08-09T14:51:47.794052+0000","last_peered":"2023-08-09T14:51:47.794052+0000","last_clean":"2023-08-09T14:51:47.794052+0000","last_became_active":"2023-08-09T14:51:47.787761+0000","last_became_peered":"2023-08-09T14:51:47.787761+0000","last_unstale":"2023-08-09T14:51:47.794052+0000","last_undegraded":"2023-08-09T14:51:47.794052+0000","last_fullsized":"2023-08-09T14:51:47.794052+0000","mapping_epoch":170,"log_start":"0'0","ondisk_log_start":"0'0","created":170,"last_epoch_clean":171,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T14:51:46.735123+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T14:51:46.735123+0000","last_clean_scrub_stamp":"2023-08-09T14:51:46.735123+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T15:03:01.623215+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"20.13","version":"171'2","reported_seq":13,"reported_epoch":171,"state":"active+clean","last_fresh":"2023-08-09T14:51:47.811450+0000","last_change":"2023-08-09T14:51:46.784739+0000","last_active":"2023-08-09T14:51:47.811450+0000","last_peered":"2023-08-09T14:51:47.811450+0000","last_clean":"2023-08-09T14:51:47.811450+0000","last_became_active":"2023-08-09T14:51:46.784369+0000","last_became_peered":"2023-08-09T14:51:46.784369+0000","last_unstale":"2023-08-09T14:51:47.811450+0000","last_undegraded":"2023-08-09T14:51:47.811450+0000","last_fullsized":"2023-08-09T14:51:47.811450+0000","mapping_epoch":169,"log_start":"0'0","ondisk_log_start":"0'0","created":169,"last_epoch_clean":170,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T14:51:45.719739+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T14:51:45.719739+0000","last_clean_scrub_stamp":"2023-08-09T14:51:45.719739+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T23:19:23.402306+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":312,"reported_epoch":171,"state":"active+clean","last_fresh":"2023-08-09T14:51:47.785779+0000","last_change":"2023-08-09T13:33:56.401781+0000","last_active":"2023-08-09T14:51:47.785779+0000","last_peered":"2023-08-09T14:51:47.785779+0000","last_clean":"2023-08-09T14:51:47.785779+0000","last_became_active":"2023-08-09T13:33:56.400336+0000","last_became_peered":"2023-08-09T13:33:56.400336+0000","last_unstale":"2023-08-09T14:51:47.785779+0000","last_undegraded":"2023-08-09T14:51:47.785779+0000","last_fullsized":"2023-08-09T14:51:47.785779+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:33:54.845869+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:33:54.845869+0000","last_clean_scrub_stamp":"2023-08-09T13:33:54.845869+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T15:12:23.106020+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"21.13","version":"0'0","reported_seq":9,"reported_epoch":171,"state":"active+clean","last_fresh":"2023-08-09T14:51:47.788598+0000","last_change":"2023-08-09T14:51:47.784733+0000","last_active":"2023-08-09T14:51:47.788598+0000","last_peered":"2023-08-09T14:51:47.788598+0000","last_clean":"2023-08-09T14:51:47.788598+0000","last_became_active":"2023-08-09T14:51:47.784344+0000","last_became_peered":"2023-08-09T14:51:47.784344+0000","last_unstale":"2023-08-09T14:51:47.788598+0000","last_undegraded":"2023-08-09T14:51:47.788598+0000","last_fullsized":"2023-08-09T14:51:47.788598+0000","mapping_epoch":170,"log_start":"0'0","ondisk_log_start":"0'0","created":170,"last_epoch_clean":171,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T14:51:46.735123+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T14:51:46.735123+0000","last_clean_scrub_stamp":"2023-08-09T14:51:46.735123+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T22:55:45.066024+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"20.12","version":"0'0","reported_seq":11,"reported_epoch":171,"state":"active+clean","last_fresh":"2023-08-09T14:51:47.785348+0000","last_change":"2023-08-09T14:51:46.787485+0000","last_active":"2023-08-09T14:51:47.785348+0000","last_peered":"2023-08-09T14:51:47.785348+0000","last_clean":"2023-08-09T14:51:47.785348+0000","last_became_active":"2023-08-09T14:51:46.787196+0000","last_became_peered":"2023-08-09T14:51:46.787196+0000","last_unstale":"2023-08-09T14:51:47.785348+0000","last_undegraded":"2023-08-09T14:51:47.785348+0000","last_fullsized":"2023-08-09T14:51:47.785348+0000","mapping_epoch":169,"log_start":"0'0","ondisk_log_start":"0'0","created":169,"last_epoch_clean":170,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T14:51:45.719739+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T14:51:45.719739+0000","last_clean_scrub_stamp":"2023-08-09T14:51:45.719739+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T21:09:02.297470+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":312,"reported_epoch":171,"state":"active+clean","last_fresh":"2023-08-09T14:51:47.777642+0000","last_change":"2023-08-09T13:33:55.886389+0000","last_active":"2023-08-09T14:51:47.777642+0000","last_peered":"2023-08-09T14:51:47.777642+0000","last_clean":"2023-08-09T14:51:47.777642+0000","last_became_active":"2023-08-09T13:33:55.885638+0000","last_became_peered":"2023-08-09T13:33:55.885638+0000","last_unstale":"2023-08-09T14:51:47.777642+0000","last_undegraded":"2023-08-09T14:51:47.777642+0000","last_fullsized":"2023-08-09T14:51:47.777642+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:33:54.845869+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:33:54.845869+0000","last_clean_scrub_stamp":"2023-08-09T13:33:54.845869+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T13:38:55.954033+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]}],"pool_stats":[{"poolid":21,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":20,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":38,"ondisk_log_size":38,"up":64,"acting":64,"num_store_stats":8},{"poolid":2,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1291,"num_read_kb":2654,"num_write":403,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":252,"ondisk_log_size":252,"up":2,"acting":2,"num_store_stats":2},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":8}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834575779,"num_pgs":34,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":47736,"kb_used_data":2660,"kb_used_omap":1,"kb_used_meta":45054,"kb_avail":94324104,"statfs":{"total":96636764160,"available":96587882496,"internally_reserved":0,"allocated":2723840,"data_stored":1743157,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1781,"internal_metadata":46135563},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Aug 9 14:51:11 2023","interfaces":[{"interface":"back","average":{"1min":1.2729999999999999,"5min":1.2350000000000001,"15min":1.24},"min":{"1min":0.79200000000000004,"5min":0.76800000000000002,"15min":0.60499999999999998},"max":{"1min":1.625,"5min":2.2320000000000002,"15min":2.2320000000000002},"last":0.90700000000000003},{"interface":"front","average":{"1min":1.1990000000000001,"5min":1.2250000000000001,"15min":1.258},"min":{"1min":0.88500000000000001,"5min":0.82399999999999995,"15min":0.82399999999999995},"max":{"1min":1.796,"5min":1.871,"15min":1.871},"last":1.274}]},{"osd":1,"last update":"Wed Aug 9 14:51:11 2023","interfaces":[{"interface":"back","average":{"1min":1.244,"5min":1.292,"15min":1.325},"min":{"1min":0.89800000000000002,"5min":0.89800000000000002,"15min":0.79900000000000004},"max":{"1min":1.702,"5min":1.9239999999999999,"15min":1.9470000000000001},"last":0.72799999999999998},{"interface":"front","average":{"1min":1.351,"5min":1.319,"15min":1.3080000000000001},"min":{"1min":0.93999999999999995,"5min":0.874,"15min":0.874},"max":{"1min":1.7270000000000001,"5min":1.7270000000000001,"15min":1.9399999999999999},"last":1.377}]},{"osd":2,"last update":"Wed Aug 9 14:51:11 2023","interfaces":[{"interface":"back","average":{"1min":1.2729999999999999,"5min":1.306,"15min":1.3129999999999999},"min":{"1min":0.96199999999999997,"5min":0.89900000000000002,"15min":0.89500000000000002},"max":{"1min":1.6890000000000001,"5min":2.3149999999999999,"15min":2.3149999999999999},"last":1.3320000000000001},{"interface":"front","average":{"1min":1.375,"5min":1.3740000000000001,"15min":1.3899999999999999},"min":{"1min":1.0900000000000001,"5min":0.93200000000000005,"15min":0.92100000000000004},"max":{"1min":1.675,"5min":1.9279999999999999,"15min":2.0110000000000001},"last":1.04}]},{"osd":3,"last update":"Wed Aug 9 14:51:11 2023","interfaces":[{"interface":"back","average":{"1min":1.357,"5min":1.359,"15min":1.365},"min":{"1min":0.94999999999999996,"5min":0.94999999999999996,"15min":0.90600000000000003},"max":{"1min":1.6160000000000001,"5min":1.9910000000000001,"15min":1.9910000000000001},"last":1.4410000000000001},{"interface":"front","average":{"1min":1.5269999999999999,"5min":1.48,"15min":1.474},"min":{"1min":1.236,"5min":0.98599999999999999,"15min":0.98599999999999999},"max":{"1min":1.887,"5min":2.375,"15min":2.375},"last":1.407}]},{"osd":4,"last update":"Wed Aug 9 14:51:11 2023","interfaces":[{"interface":"back","average":{"1min":1.294,"5min":1.294,"15min":1.3069999999999999},"min":{"1min":0.82499999999999996,"5min":0.82499999999999996,"15min":0.79800000000000004},"max":{"1min":1.7090000000000001,"5min":1.901,"15min":2.254},"last":1.087},{"interface":"front","average":{"1min":1.4790000000000001,"5min":1.4099999999999999,"15min":1.3879999999999999},"min":{"1min":1.0940000000000001,"5min":0.90400000000000003,"15min":0.90400000000000003},"max":{"1min":1.776,"5min":2.1789999999999998,"15min":2.1789999999999998},"last":1.1899999999999999}]},{"osd":5,"last update":"Wed Aug 9 14:51:11 2023","interfaces":[{"interface":"back","average":{"1min":1.502,"5min":1.498,"15min":1.476},"min":{"1min":1.101,"5min":0.85599999999999998,"15min":0.76900000000000002},"max":{"1min":1.929,"5min":2.1589999999999998,"15min":2.1589999999999998},"last":1.133},{"interface":"front","average":{"1min":1.476,"5min":1.504,"15min":1.4910000000000001},"min":{"1min":1.0920000000000001,"5min":0.92400000000000004,"15min":0.92400000000000004},"max":{"1min":1.9430000000000001,"5min":2.4510000000000001,"15min":2.4510000000000001},"last":1.589}]},{"osd":6,"last update":"Wed Aug 9 14:51:11 2023","interfaces":[{"interface":"back","average":{"1min":1.6439999999999999,"5min":1.613,"15min":1.6020000000000001},"min":{"1min":1.369,"5min":1.0529999999999999,"15min":1.0529999999999999},"max":{"1min":1.8660000000000001,"5min":2.266,"15min":2.5419999999999998},"last":1.4890000000000001},{"interface":"front","average":{"1min":1.5489999999999999,"5min":1.542,"15min":1.55},"min":{"1min":1.222,"5min":0.98499999999999999,"15min":0.84899999999999998},"max":{"1min":1.8240000000000001,"5min":2.415,"15min":2.415},"last":1.5429999999999999}]}]},{"osd":6,"up_from":13,"seq":55834575779,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":41392,"kb_used_data":2076,"kb_used_omap":1,"kb_used_meta":39294,"kb_avail":94330448,"statfs":{"total":96636764160,"available":96594378752,"internally_reserved":0,"allocated":2125824,"data_stored":1151428,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1781,"internal_metadata":40237323},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Aug 9 14:51:20 2023","interfaces":[{"interface":"back","average":{"1min":1.246,"5min":1.181,"15min":1.202},"min":{"1min":0.92000000000000004,"5min":0.75,"15min":0.745},"max":{"1min":1.6080000000000001,"5min":1.6870000000000001,"15min":2.278},"last":1.6879999999999999},{"interface":"front","average":{"1min":1.262,"5min":1.262,"15min":1.2549999999999999},"min":{"1min":0.93000000000000005,"5min":0.84999999999999998,"15min":0.84199999999999997},"max":{"1min":2.2170000000000001,"5min":2.2170000000000001,"15min":2.2170000000000001},"last":1.3380000000000001}]},{"osd":1,"last update":"Wed Aug 9 14:51:20 2023","interfaces":[{"interface":"back","average":{"1min":1.226,"5min":1.2390000000000001,"15min":1.224},"min":{"1min":0.88400000000000001,"5min":0.88400000000000001,"15min":0.84799999999999998},"max":{"1min":1.7569999999999999,"5min":1.829,"15min":1.829},"last":1.1299999999999999},{"interface":"front","average":{"1min":1.1850000000000001,"5min":1.28,"15min":1.2989999999999999},"min":{"1min":0.88500000000000001,"5min":0.79400000000000004,"15min":0.79400000000000004},"max":{"1min":1.5369999999999999,"5min":1.8049999999999999,"15min":1.903},"last":1.9099999999999999}]},{"osd":2,"last update":"Wed Aug 9 14:51:20 2023","interfaces":[{"interface":"back","average":{"1min":1.4219999999999999,"5min":1.3500000000000001,"15min":1.3520000000000001},"min":{"1min":1.0309999999999999,"5min":0.89100000000000001,"15min":0.88100000000000001},"max":{"1min":1.863,"5min":1.863,"15min":2.0920000000000001},"last":1.401},{"interface":"front","average":{"1min":1.395,"5min":1.3700000000000001,"15min":1.3340000000000001},"min":{"1min":1.034,"5min":0.93200000000000005,"15min":0.90900000000000003},"max":{"1min":1.6890000000000001,"5min":1.857,"15min":1.968},"last":1.984}]},{"osd":3,"last update":"Wed Aug 9 14:51:20 2023","interfaces":[{"interface":"back","average":{"1min":1.427,"5min":1.4259999999999999,"15min":1.4159999999999999},"min":{"1min":1.167,"5min":1.085,"15min":1.0089999999999999},"max":{"1min":1.774,"5min":1.7929999999999999,"15min":1.8169999999999999},"last":1.4830000000000001},{"interface":"front","average":{"1min":1.4350000000000001,"5min":1.4099999999999999,"15min":1.3859999999999999},"min":{"1min":1.0529999999999999,"5min":0.97199999999999998,"15min":0.93300000000000005},"max":{"1min":1.7090000000000001,"5min":1.8280000000000001,"15min":1.8280000000000001},"last":1.609}]},{"osd":4,"last update":"Wed Aug 9 14:51:20 2023","interfaces":[{"interface":"back","average":{"1min":1.4330000000000001,"5min":1.3480000000000001,"15min":1.2989999999999999},"min":{"1min":0.82299999999999995,"5min":0.82299999999999995,"15min":0.82199999999999995},"max":{"1min":1.9810000000000001,"5min":1.9810000000000001,"15min":1.9810000000000001},"last":0.85399999999999998},{"interface":"front","average":{"1min":1.5309999999999999,"5min":1.444,"15min":1.371},"min":{"1min":1.002,"5min":0.871,"15min":0.71599999999999997},"max":{"1min":1.925,"5min":1.9530000000000001,"15min":1.9530000000000001},"last":1.742}]},{"osd":5,"last update":"Wed Aug 9 14:51:20 2023","interfaces":[{"interface":"back","average":{"1min":1.46,"5min":1.4299999999999999,"15min":1.3700000000000001},"min":{"1min":0.83599999999999997,"5min":0.83599999999999997,"15min":0.83599999999999997},"max":{"1min":2.0609999999999999,"5min":2.2170000000000001,"15min":2.2170000000000001},"last":1.821},{"interface":"front","average":{"1min":1.6859999999999999,"5min":1.6279999999999999,"15min":1.6020000000000001},"min":{"1min":1.363,"5min":1.331,"15min":0.93500000000000005},"max":{"1min":2.1619999999999999,"5min":2.1619999999999999,"15min":2.1619999999999999},"last":1.21}]},{"osd":7,"last update":"Wed Aug 9 14:51:20 2023","interfaces":[{"interface":"back","average":{"1min":1.7010000000000001,"5min":1.657,"15min":1.6000000000000001},"min":{"1min":1.327,"5min":0.96299999999999997,"15min":0.96299999999999997},"max":{"1min":2.476,"5min":2.476,"15min":2.5310000000000001},"last":1.8700000000000001},{"interface":"front","average":{"1min":1.675,"5min":1.625,"15min":1.5920000000000001},"min":{"1min":1.0549999999999999,"5min":0.997,"15min":0.96999999999999997},"max":{"1min":2.0670000000000002,"5min":2.286,"15min":2.286},"last":1.9399999999999999}]}]},{"osd":5,"up_from":13,"seq":55834575779,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":59428,"kb_used_data":2068,"kb_used_omap":1,"kb_used_meta":57342,"kb_avail":94312412,"statfs":{"total":96636764160,"available":96575909888,"internally_reserved":0,"allocated":2117632,"data_stored":1145297,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1781,"internal_metadata":58718475},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Aug 9 14:51:12 2023","interfaces":[{"interface":"back","average":{"1min":1.1819999999999999,"5min":1.1970000000000001,"15min":1.194},"min":{"1min":0.81999999999999995,"5min":0.72599999999999998,"15min":0.72599999999999998},"max":{"1min":1.6899999999999999,"5min":1.819,"15min":1.976},"last":1.5},{"interface":"front","average":{"1min":1.3320000000000001,"5min":1.24,"15min":1.2350000000000001},"min":{"1min":0.92700000000000005,"5min":0.84999999999999998,"15min":0.79200000000000004},"max":{"1min":1.9470000000000001,"5min":1.9470000000000001,"15min":1.9470000000000001},"last":1.5569999999999999}]},{"osd":1,"last update":"Wed Aug 9 14:51:12 2023","interfaces":[{"interface":"back","average":{"1min":1.361,"5min":1.2989999999999999,"15min":1.26},"min":{"1min":0.96699999999999997,"5min":0.89500000000000002,"15min":0.82199999999999995},"max":{"1min":1.7929999999999999,"5min":1.7929999999999999,"15min":1.7929999999999999},"last":1.0840000000000001},{"interface":"front","average":{"1min":1.274,"5min":1.327,"15min":1.327},"min":{"1min":0.99299999999999999,"5min":0.90000000000000002,"15min":0.877},"max":{"1min":1.544,"5min":1.8480000000000001,"15min":1.8480000000000001},"last":1.147}]},{"osd":2,"last update":"Wed Aug 9 14:51:12 2023","interfaces":[{"interface":"back","average":{"1min":1.3440000000000001,"5min":1.3500000000000001,"15min":1.329},"min":{"1min":1.004,"5min":0.96099999999999997,"15min":0.86199999999999999},"max":{"1min":1.7529999999999999,"5min":1.7529999999999999,"15min":1.8939999999999999},"last":1.2230000000000001},{"interface":"front","average":{"1min":1.4630000000000001,"5min":1.3819999999999999,"15min":1.351},"min":{"1min":1.093,"5min":0.98499999999999999,"15min":0.88800000000000001},"max":{"1min":1.694,"5min":1.8300000000000001,"15min":1.8540000000000001},"last":1.4450000000000001}]},{"osd":3,"last update":"Wed Aug 9 14:51:12 2023","interfaces":[{"interface":"back","average":{"1min":1.4039999999999999,"5min":1.3720000000000001,"15min":1.3799999999999999},"min":{"1min":1.079,"5min":0.95999999999999996,"15min":0.95999999999999996},"max":{"1min":1.766,"5min":1.7809999999999999,"15min":2.04},"last":1.619},{"interface":"front","average":{"1min":1.456,"5min":1.498,"15min":1.4690000000000001},"min":{"1min":1.1499999999999999,"5min":1.0009999999999999,"15min":0.98299999999999998},"max":{"1min":1.752,"5min":1.8109999999999999,"15min":2.0409999999999999},"last":1.6919999999999999}]},{"osd":4,"last update":"Wed Aug 9 14:51:12 2023","interfaces":[{"interface":"back","average":{"1min":1.421,"5min":1.3560000000000001,"15min":1.3080000000000001},"min":{"1min":1.0389999999999999,"5min":0.871,"15min":0.77600000000000002},"max":{"1min":1.905,"5min":1.905,"15min":1.905},"last":1.9690000000000001},{"interface":"front","average":{"1min":1.349,"5min":1.4330000000000001,"15min":1.423},"min":{"1min":0.94799999999999995,"5min":0.94799999999999995,"15min":0.84499999999999997},"max":{"1min":1.8280000000000001,"5min":1.911,"15min":2.1899999999999999},"last":1.8460000000000001}]},{"osd":6,"last update":"Wed Aug 9 14:51:12 2023","interfaces":[{"interface":"back","average":{"1min":1.415,"5min":1.403,"15min":1.4199999999999999},"min":{"1min":1.0469999999999999,"5min":0.88600000000000001,"15min":0.79300000000000004},"max":{"1min":1.8839999999999999,"5min":1.8839999999999999,"15min":2.0960000000000001},"last":1.9059999999999999},{"interface":"front","average":{"1min":1.512,"5min":1.4950000000000001,"15min":1.48},"min":{"1min":1.0589999999999999,"5min":1.0169999999999999,"15min":0.871},"max":{"1min":1.9630000000000001,"5min":1.9630000000000001,"15min":1.9730000000000001},"last":2.1280000000000001}]},{"osd":7,"last update":"Wed Aug 9 14:51:12 2023","interfaces":[{"interface":"back","average":{"1min":1.5129999999999999,"5min":1.554,"15min":1.5780000000000001},"min":{"1min":1.085,"5min":1.054,"15min":0.98899999999999999},"max":{"1min":1.845,"5min":1.9319999999999999,"15min":2.3170000000000002},"last":2.0680000000000001},{"interface":"front","average":{"1min":1.659,"5min":1.6299999999999999,"15min":1.637},"min":{"1min":1.1859999999999999,"5min":1.008,"15min":0.95099999999999996},"max":{"1min":2.0049999999999999,"5min":2.0649999999999999,"15min":2.3500000000000001},"last":2.105}]}]},{"osd":4,"up_from":13,"seq":55834575779,"num_pgs":13,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":59428,"kb_used_data":2068,"kb_used_omap":1,"kb_used_meta":57342,"kb_avail":94312412,"statfs":{"total":96636764160,"available":96575909888,"internally_reserved":0,"allocated":2117632,"data_stored":1145297,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1781,"internal_metadata":58718475},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Aug 9 14:51:38 2023","interfaces":[{"interface":"back","average":{"1min":1.1859999999999999,"5min":1.2070000000000001,"15min":1.2050000000000001},"min":{"1min":0.69999999999999996,"5min":0.69999999999999996,"15min":0.69999999999999996},"max":{"1min":2.573,"5min":2.573,"15min":2.573},"last":0.77300000000000002},{"interface":"front","average":{"1min":1.3460000000000001,"5min":1.341,"15min":1.3260000000000001},"min":{"1min":0.83799999999999997,"5min":0.83799999999999997,"15min":0.54000000000000004},"max":{"1min":2.1960000000000002,"5min":2.1960000000000002,"15min":2.1960000000000002},"last":1.661}]},{"osd":1,"last update":"Wed Aug 9 14:51:38 2023","interfaces":[{"interface":"back","average":{"1min":1.49,"5min":1.3799999999999999,"15min":1.3540000000000001},"min":{"1min":0.86599999999999999,"5min":0.86599999999999999,"15min":0.60599999999999998},"max":{"1min":4.4299999999999997,"5min":4.4299999999999997,"15min":4.4299999999999997},"last":1.7509999999999999},{"interface":"front","average":{"1min":1.204,"5min":1.292,"15min":1.2849999999999999},"min":{"1min":0.68200000000000005,"5min":0.68200000000000005,"15min":0.68200000000000005},"max":{"1min":2.4630000000000001,"5min":2.4630000000000001,"15min":2.4630000000000001},"last":0.83799999999999997}]},{"osd":2,"last update":"Wed Aug 9 14:51:38 2023","interfaces":[{"interface":"back","average":{"1min":1.335,"5min":1.3260000000000001,"15min":1.3109999999999999},"min":{"1min":0.81499999999999995,"5min":0.81499999999999995,"15min":0.74199999999999999},"max":{"1min":2.6920000000000002,"5min":2.6920000000000002,"15min":2.6920000000000002},"last":0.90000000000000002},{"interface":"front","average":{"1min":1.462,"5min":1.454,"15min":1.4530000000000001},"min":{"1min":0.70899999999999996,"5min":0.70899999999999996,"15min":0.70899999999999996},"max":{"1min":2.0830000000000002,"5min":2.1619999999999999,"15min":2.4359999999999999},"last":1.2889999999999999}]},{"osd":3,"last update":"Wed Aug 9 14:51:38 2023","interfaces":[{"interface":"back","average":{"1min":1.369,"5min":1.3360000000000001,"15min":1.339},"min":{"1min":0.85699999999999998,"5min":0.85699999999999998,"15min":0.85699999999999998},"max":{"1min":2.2069999999999999,"5min":2.2069999999999999,"15min":2.6429999999999998},"last":1.097},{"interface":"front","average":{"1min":1.534,"5min":1.4970000000000001,"15min":1.4750000000000001},"min":{"1min":0.75,"5min":0.75,"15min":0.75},"max":{"1min":3.0600000000000001,"5min":3.0600000000000001,"15min":3.0600000000000001},"last":1.044}]},{"osd":5,"last update":"Wed Aug 9 14:51:38 2023","interfaces":[{"interface":"back","average":{"1min":1.462,"5min":1.4099999999999999,"15min":1.377},"min":{"1min":0.77100000000000002,"5min":0.77100000000000002,"15min":0.70399999999999996},"max":{"1min":2.0710000000000002,"5min":2.5009999999999999,"15min":2.5009999999999999},"last":1.4470000000000001},{"interface":"front","average":{"1min":1.343,"5min":1.403,"15min":1.385},"min":{"1min":0.56999999999999995,"5min":0.56999999999999995,"15min":0.56999999999999995},"max":{"1min":1.9790000000000001,"5min":2.4129999999999998,"15min":3.0110000000000001},"last":1.399}]},{"osd":6,"last update":"Wed Aug 9 14:51:38 2023","interfaces":[{"interface":"back","average":{"1min":1.589,"5min":1.4470000000000001,"15min":1.431},"min":{"1min":0.93200000000000005,"5min":0.85499999999999998,"15min":0.79500000000000004},"max":{"1min":2.9540000000000002,"5min":2.9540000000000002,"15min":2.9540000000000002},"last":1.1499999999999999},{"interface":"front","average":{"1min":1.6739999999999999,"5min":1.619,"15min":1.5680000000000001},"min":{"1min":0.59899999999999998,"5min":0.59899999999999998,"15min":0.59899999999999998},"max":{"1min":2.0529999999999999,"5min":2.3260000000000001,"15min":2.5310000000000001},"last":1.5680000000000001}]},{"osd":7,"last update":"Wed Aug 9 14:51:38 2023","interfaces":[{"interface":"back","average":{"1min":1.5629999999999999,"5min":1.5569999999999999,"15min":1.5229999999999999},"min":{"1min":0.79100000000000004,"5min":0.79100000000000004,"15min":0.79100000000000004},"max":{"1min":2.1360000000000001,"5min":2.331,"15min":3.3149999999999999},"last":1.506},{"interface":"front","average":{"1min":1.75,"5min":1.6339999999999999,"15min":1.587},"min":{"1min":0.88400000000000001,"5min":0.88400000000000001,"15min":0.82599999999999996},"max":{"1min":3.274,"5min":3.274,"15min":3.274},"last":1.256}]}]},{"osd":3,"up_from":13,"seq":55834575780,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":59428,"kb_used_data":2068,"kb_used_omap":1,"kb_used_meta":57342,"kb_avail":94312412,"statfs":{"total":96636764160,"available":96575909888,"internally_reserved":0,"allocated":2117632,"data_stored":1145297,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1781,"internal_metadata":58718475},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Aug 9 14:51:19 2023","interfaces":[{"interface":"back","average":{"1min":0.97199999999999998,"5min":0.95999999999999996,"15min":0.95199999999999996},"min":{"1min":0.5,"5min":0.5,"15min":0.5},"max":{"1min":1.5620000000000001,"5min":1.722,"15min":1.8740000000000001},"last":0.90800000000000003},{"interface":"front","average":{"1min":1.091,"5min":1.133,"15min":1.1559999999999999},"min":{"1min":0.81599999999999995,"5min":0.748,"15min":0.748},"max":{"1min":1.4910000000000001,"5min":1.9179999999999999,"15min":1.9179999999999999},"last":0.69199999999999995}]},{"osd":1,"last update":"Wed Aug 9 14:51:19 2023","interfaces":[{"interface":"back","average":{"1min":1.042,"5min":1.073,"15min":1.077},"min":{"1min":0.56999999999999995,"5min":0.56999999999999995,"15min":0.56999999999999995},"max":{"1min":1.651,"5min":1.7589999999999999,"15min":1.903},"last":0.78000000000000003},{"interface":"front","average":{"1min":1.2450000000000001,"5min":1.2290000000000001,"15min":1.2190000000000001},"min":{"1min":0.68200000000000005,"5min":0.68200000000000005,"15min":0.68200000000000005},"max":{"1min":1.802,"5min":1.8320000000000001,"15min":1.903},"last":0.72799999999999998}]},{"osd":2,"last update":"Wed Aug 9 14:51:19 2023","interfaces":[{"interface":"back","average":{"1min":1.4259999999999999,"5min":1.3500000000000001,"15min":1.337},"min":{"1min":0.99099999999999999,"5min":0.92900000000000005,"15min":0.878},"max":{"1min":3.7770000000000001,"5min":3.7770000000000001,"15min":3.7770000000000001},"last":1.0429999999999999},{"interface":"front","average":{"1min":1.591,"5min":1.425,"15min":1.411},"min":{"1min":1.026,"5min":0.94899999999999995,"15min":0.92800000000000005},"max":{"1min":3.5750000000000002,"5min":3.5750000000000002,"15min":3.5750000000000002},"last":1.194}]},{"osd":4,"last update":"Wed Aug 9 14:51:19 2023","interfaces":[{"interface":"back","average":{"1min":1.3420000000000001,"5min":1.409,"15min":1.391},"min":{"1min":0.92200000000000004,"5min":0.88400000000000001,"15min":0.86499999999999999},"max":{"1min":1.714,"5min":1.9359999999999999,"15min":2.153},"last":0.82699999999999996},{"interface":"front","average":{"1min":1.4399999999999999,"5min":1.4690000000000001,"15min":1.472},"min":{"1min":0.79300000000000004,"5min":0.79300000000000004,"15min":0.79300000000000004},"max":{"1min":1.792,"5min":1.992,"15min":2.3439999999999999},"last":1.294}]},{"osd":5,"last update":"Wed Aug 9 14:51:19 2023","interfaces":[{"interface":"back","average":{"1min":1.54,"5min":1.5609999999999999,"15min":1.5620000000000001},"min":{"1min":0.85199999999999998,"5min":0.85199999999999998,"15min":0.85199999999999998},"max":{"1min":1.8580000000000001,"5min":2.0339999999999998,"15min":2.2629999999999999},"last":1.5760000000000001},{"interface":"front","average":{"1min":1.5940000000000001,"5min":1.5700000000000001,"15min":1.554},"min":{"1min":1.1619999999999999,"5min":1.052,"15min":1.052},"max":{"1min":2.3399999999999999,"5min":2.4630000000000001,"15min":2.4630000000000001},"last":1.274}]},{"osd":6,"last update":"Wed Aug 9 14:51:19 2023","interfaces":[{"interface":"back","average":{"1min":1.5229999999999999,"5min":1.5629999999999999,"15min":1.5309999999999999},"min":{"1min":0.93500000000000005,"5min":0.93500000000000005,"15min":0.93500000000000005},"max":{"1min":1.9470000000000001,"5min":2.0790000000000002,"15min":2.343},"last":1.1619999999999999},{"interface":"front","average":{"1min":1.6739999999999999,"5min":1.653,"15min":1.643},"min":{"1min":1.038,"5min":1.038,"15min":1.038},"max":{"1min":2.0819999999999999,"5min":2.3370000000000002,"15min":2.4140000000000001},"last":1.5149999999999999}]},{"osd":7,"last update":"Wed Aug 9 14:51:19 2023","interfaces":[{"interface":"back","average":{"1min":1.722,"5min":1.7150000000000001,"15min":1.6890000000000001},"min":{"1min":1.367,"5min":1.3169999999999999,"15min":1.2589999999999999},"max":{"1min":2.427,"5min":2.5430000000000001,"15min":2.5430000000000001},"last":1.546},{"interface":"front","average":{"1min":1.6950000000000001,"5min":1.724,"15min":1.681},"min":{"1min":1.2070000000000001,"5min":1.2070000000000001,"15min":1.1879999999999999},"max":{"1min":2.0750000000000002,"5min":2.2069999999999999,"15min":3.1339999999999999},"last":1.2410000000000001}]}]},{"osd":2,"up_from":13,"seq":55834575779,"num_pgs":5,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":41380,"kb_used_data":2068,"kb_used_omap":1,"kb_used_meta":39294,"kb_avail":94330460,"statfs":{"total":96636764160,"available":96594391040,"internally_reserved":0,"allocated":2117632,"data_stored":1145297,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1781,"internal_metadata":40237323},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Aug 9 14:51:11 2023","interfaces":[{"interface":"back","average":{"1min":0.93000000000000005,"5min":0.91200000000000003,"15min":0.92400000000000004},"min":{"1min":0.755,"5min":0.625,"15min":0.54400000000000004},"max":{"1min":1.4079999999999999,"5min":1.4079999999999999,"15min":1.7190000000000001},"last":0.86399999999999999},{"interface":"front","average":{"1min":1.119,"5min":1.1020000000000001,"15min":1.117},"min":{"1min":0.79800000000000004,"5min":0.67600000000000005,"15min":0.54800000000000004},"max":{"1min":1.8200000000000001,"5min":1.8300000000000001,"15min":1.8740000000000001},"last":1.4359999999999999}]},{"osd":1,"last update":"Wed Aug 9 14:51:11 2023","interfaces":[{"interface":"back","average":{"1min":1.2609999999999999,"5min":1.181,"15min":1.226},"min":{"1min":0.90900000000000003,"5min":0.59399999999999997,"15min":0.59399999999999997},"max":{"1min":1.514,"5min":1.732,"15min":2.1600000000000001},"last":1.099},{"interface":"front","average":{"1min":1.4770000000000001,"5min":1.446,"15min":1.4470000000000001},"min":{"1min":1.161,"5min":0.66700000000000004,"15min":0.66700000000000004},"max":{"1min":1.861,"5min":2.0350000000000001,"15min":2.2200000000000002},"last":1.534}]},{"osd":3,"last update":"Wed Aug 9 14:51:11 2023","interfaces":[{"interface":"back","average":{"1min":1.0600000000000001,"5min":1.103,"15min":1.1240000000000001},"min":{"1min":0.877,"5min":0.73399999999999999,"15min":0.70799999999999996},"max":{"1min":1.371,"5min":1.7350000000000001,"15min":1.9239999999999999},"last":1.472},{"interface":"front","average":{"1min":1.4039999999999999,"5min":1.411,"15min":1.375},"min":{"1min":1.04,"5min":0.876,"15min":0.876},"max":{"1min":1.7430000000000001,"5min":2.0299999999999998,"15min":2.0299999999999998},"last":1.2769999999999999}]},{"osd":4,"last update":"Wed Aug 9 14:51:11 2023","interfaces":[{"interface":"back","average":{"1min":1.5940000000000001,"5min":1.5249999999999999,"15min":1.52},"min":{"1min":1.2490000000000001,"5min":1.0609999999999999,"15min":0.95699999999999996},"max":{"1min":1.8819999999999999,"5min":2.1400000000000001,"15min":2.2389999999999999},"last":1.2150000000000001},{"interface":"front","average":{"1min":1.5169999999999999,"5min":1.474,"15min":1.4790000000000001},"min":{"1min":1.202,"5min":0.85599999999999998,"15min":0.85599999999999998},"max":{"1min":1.9810000000000001,"5min":2.2509999999999999,"15min":2.2509999999999999},"last":1.319}]},{"osd":5,"last update":"Wed Aug 9 14:51:11 2023","interfaces":[{"interface":"back","average":{"1min":1.623,"5min":1.5649999999999999,"15min":1.5389999999999999},"min":{"1min":1.194,"5min":0.90100000000000002,"15min":0.90100000000000002},"max":{"1min":2.0070000000000001,"5min":2.1190000000000002,"15min":2.1619999999999999},"last":1.645},{"interface":"front","average":{"1min":1.5289999999999999,"5min":1.478,"15min":1.478},"min":{"1min":1.1339999999999999,"5min":0.92200000000000004,"15min":0.92200000000000004},"max":{"1min":1.9239999999999999,"5min":1.9930000000000001,"15min":2.1869999999999998},"last":1.6890000000000001}]},{"osd":6,"last update":"Wed Aug 9 14:51:11 2023","interfaces":[{"interface":"back","average":{"1min":1.6140000000000001,"5min":1.5409999999999999,"15min":1.5129999999999999},"min":{"1min":1.26,"5min":0.82599999999999996,"15min":0.82599999999999996},"max":{"1min":2.0099999999999998,"5min":2.081,"15min":2.3210000000000002},"last":1.851},{"interface":"front","average":{"1min":1.673,"5min":1.639,"15min":1.6240000000000001},"min":{"1min":1.3,"5min":0.98699999999999999,"15min":0.98699999999999999},"max":{"1min":2.0339999999999998,"5min":2.2759999999999998,"15min":2.5070000000000001},"last":1.395}]},{"osd":7,"last update":"Wed Aug 9 14:51:11 2023","interfaces":[{"interface":"back","average":{"1min":1.724,"5min":1.704,"15min":1.665},"min":{"1min":1.3660000000000001,"5min":0.96099999999999997,"15min":0.96099999999999997},"max":{"1min":2.1269999999999998,"5min":2.419,"15min":2.419},"last":1.7310000000000001},{"interface":"front","average":{"1min":1.6950000000000001,"5min":1.649,"15min":1.6240000000000001},"min":{"1min":1.2569999999999999,"5min":1.018,"15min":1.018},"max":{"1min":2.0899999999999999,"5min":2.1349999999999998,"15min":2.3399999999999999},"last":2.3079999999999998}]}]},{"osd":1,"up_from":13,"seq":55834575779,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":51816,"kb_used_data":2648,"kb_used_omap":1,"kb_used_meta":49150,"kb_avail":94320024,"statfs":{"total":96636764160,"available":96583704576,"internally_reserved":0,"allocated":2711552,"data_stored":1735665,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1781,"internal_metadata":50329867},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Aug 9 14:51:22 2023","interfaces":[{"interface":"back","average":{"1min":0.97899999999999998,"5min":0.998,"15min":0.997},"min":{"1min":0.308,"5min":0.308,"15min":0.308},"max":{"1min":1.5009999999999999,"5min":1.879,"15min":1.931},"last":0.84499999999999997},{"interface":"front","average":{"1min":1.1759999999999999,"5min":1.139,"15min":1.131},"min":{"1min":0.77000000000000002,"5min":0.76900000000000002,"15min":0.71299999999999997},"max":{"1min":1.623,"5min":1.849,"15min":1.849},"last":0.91400000000000003}]},{"osd":2,"last update":"Wed Aug 9 14:51:22 2023","interfaces":[{"interface":"back","average":{"1min":1.173,"5min":1.111,"15min":1.087},"min":{"1min":0.79800000000000004,"5min":0.79700000000000004,"15min":0.76300000000000001},"max":{"1min":1.8400000000000001,"5min":2.71,"15min":2.71},"last":0.96899999999999997},{"interface":"front","average":{"1min":1.298,"5min":1.252,"15min":1.2609999999999999},"min":{"1min":0.72999999999999998,"5min":0.72999999999999998,"15min":0.72999999999999998},"max":{"1min":1.8919999999999999,"5min":1.8919999999999999,"15min":1.929},"last":1.341}]},{"osd":3,"last update":"Wed Aug 9 14:51:22 2023","interfaces":[{"interface":"back","average":{"1min":1.429,"5min":1.405,"15min":1.3979999999999999},"min":{"1min":0.83599999999999997,"5min":0.83599999999999997,"15min":0.83599999999999997},"max":{"1min":1.863,"5min":2.6070000000000002,"15min":2.6070000000000002},"last":1.1359999999999999},{"interface":"front","average":{"1min":1.4339999999999999,"5min":1.3999999999999999,"15min":1.375},"min":{"1min":0.86799999999999999,"5min":0.86799999999999999,"15min":0.80400000000000005},"max":{"1min":3.1629999999999998,"5min":3.1629999999999998,"15min":3.1629999999999998},"last":1.4810000000000001}]},{"osd":4,"last update":"Wed Aug 9 14:51:22 2023","interfaces":[{"interface":"back","average":{"1min":1.5189999999999999,"5min":1.512,"15min":1.4730000000000001},"min":{"1min":1.2509999999999999,"5min":1.1359999999999999,"15min":1.036},"max":{"1min":2.0489999999999999,"5min":2.0579999999999998,"15min":2.0579999999999998},"last":1.9850000000000001},{"interface":"front","average":{"1min":1.472,"5min":1.4690000000000001,"15min":1.4379999999999999},"min":{"1min":1.1539999999999999,"5min":1.0289999999999999,"15min":0.84699999999999998},"max":{"1min":1.899,"5min":3.2440000000000002,"15min":3.2440000000000002},"last":1.5509999999999999}]},{"osd":5,"last update":"Wed Aug 9 14:51:22 2023","interfaces":[{"interface":"back","average":{"1min":1.4199999999999999,"5min":1.462,"15min":1.4550000000000001},"min":{"1min":1.016,"5min":1.016,"15min":1.016},"max":{"1min":1.667,"5min":3.036,"15min":3.036},"last":1.7170000000000001},{"interface":"front","average":{"1min":1.641,"5min":1.6040000000000001,"15min":1.599},"min":{"1min":0.879,"5min":0.879,"15min":0.879},"max":{"1min":2.5019999999999998,"5min":2.5019999999999998,"15min":2.5019999999999998},"last":1.867}]},{"osd":6,"last update":"Wed Aug 9 14:51:22 2023","interfaces":[{"interface":"back","average":{"1min":1.6459999999999999,"5min":1.621,"15min":1.6020000000000001},"min":{"1min":1.3620000000000001,"5min":1.1639999999999999,"15min":1.1000000000000001},"max":{"1min":2.141,"5min":3.149,"15min":3.149},"last":2.1269999999999998},{"interface":"front","average":{"1min":1.581,"5min":1.611,"15min":1.5980000000000001},"min":{"1min":1.0629999999999999,"5min":1.0629999999999999,"15min":1.0629999999999999},"max":{"1min":1.9630000000000001,"5min":3.4460000000000002,"15min":3.4460000000000002},"last":1.9339999999999999}]},{"osd":7,"last update":"Wed Aug 9 14:51:22 2023","interfaces":[{"interface":"back","average":{"1min":1.681,"5min":1.6879999999999999,"15min":1.6919999999999999},"min":{"1min":0.79000000000000004,"5min":0.79000000000000004,"15min":0.79000000000000004},"max":{"1min":2.5939999999999999,"5min":2.5939999999999999,"15min":2.5939999999999999},"last":2.0529999999999999},{"interface":"front","average":{"1min":1.7769999999999999,"5min":1.7450000000000001,"15min":1.7230000000000001},"min":{"1min":1.1459999999999999,"5min":1.1459999999999999,"15min":1.1459999999999999},"max":{"1min":2.242,"5min":2.802,"15min":2.802},"last":2.2010000000000001}]}]},{"osd":0,"up_from":13,"seq":55834575779,"num_pgs":16,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":45472,"kb_used_data":2064,"kb_used_omap":1,"kb_used_meta":43390,"kb_avail":94326368,"statfs":{"total":96636764160,"available":96590200832,"internally_reserved":0,"allocated":2113536,"data_stored":1143936,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1781,"internal_metadata":44431627},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Wed Aug 9 14:51:07 2023","interfaces":[{"interface":"back","average":{"1min":0.878,"5min":0.94899999999999995,"15min":0.96699999999999997},"min":{"1min":0.71499999999999997,"5min":0.55000000000000004,"15min":0.55000000000000004},"max":{"1min":1.23,"5min":1.4650000000000001,"15min":1.893},"last":0.82999999999999996},{"interface":"front","average":{"1min":1.28,"5min":1.208,"15min":1.153},"min":{"1min":0.81399999999999995,"5min":0.70299999999999996,"15min":0.70299999999999996},"max":{"1min":1.6870000000000001,"5min":1.8520000000000001,"15min":1.8520000000000001},"last":0.71299999999999997}]},{"osd":2,"last update":"Wed Aug 9 14:51:07 2023","interfaces":[{"interface":"back","average":{"1min":1.171,"5min":1.1890000000000001,"15min":1.216},"min":{"1min":0.85599999999999998,"5min":0.65700000000000003,"15min":0.65700000000000003},"max":{"1min":1.6870000000000001,"5min":1.802,"15min":2.698},"last":0.96899999999999997},{"interface":"front","average":{"1min":1.248,"5min":1.2709999999999999,"15min":1.2390000000000001},"min":{"1min":0.94199999999999995,"5min":0.92400000000000004,"15min":0.82599999999999996},"max":{"1min":1.7170000000000001,"5min":1.895,"15min":1.895},"last":1.0309999999999999}]},{"osd":3,"last update":"Wed Aug 9 14:51:07 2023","interfaces":[{"interface":"back","average":{"1min":1.2250000000000001,"5min":1.2509999999999999,"15min":1.254},"min":{"1min":0.82199999999999995,"5min":0.68100000000000005,"15min":0.68100000000000005},"max":{"1min":1.74,"5min":2.0049999999999999,"15min":2.1230000000000002},"last":0.88700000000000001},{"interface":"front","average":{"1min":1.3959999999999999,"5min":1.4039999999999999,"15min":1.397},"min":{"1min":0.95399999999999996,"5min":0.95399999999999996,"15min":0.88800000000000001},"max":{"1min":1.748,"5min":1.988,"15min":2.3580000000000001},"last":1.3680000000000001}]},{"osd":4,"last update":"Wed Aug 9 14:51:07 2023","interfaces":[{"interface":"back","average":{"1min":1.3879999999999999,"5min":1.421,"15min":1.3899999999999999},"min":{"1min":1.022,"5min":0.96599999999999997,"15min":0.96099999999999997},"max":{"1min":1.9470000000000001,"5min":2.3730000000000002,"15min":2.3730000000000002},"last":1.018},{"interface":"front","average":{"1min":1.4630000000000001,"5min":1.5249999999999999,"15min":1.5169999999999999},"min":{"1min":1.137,"5min":1.137,"15min":1.052},"max":{"1min":1.758,"5min":2.125,"15min":2.149},"last":1.282}]},{"osd":5,"last update":"Wed Aug 9 14:51:07 2023","interfaces":[{"interface":"back","average":{"1min":1.518,"5min":1.5569999999999999,"15min":1.5269999999999999},"min":{"1min":1.127,"5min":0.93100000000000005,"15min":0.93100000000000005},"max":{"1min":1.972,"5min":2.0110000000000001,"15min":2.097},"last":1.153},{"interface":"front","average":{"1min":1.484,"5min":1.4770000000000001,"15min":1.5109999999999999},"min":{"1min":1.1739999999999999,"5min":1.075,"15min":0.99399999999999999},"max":{"1min":1.921,"5min":2.2320000000000002,"15min":2.266},"last":1.2370000000000001}]},{"osd":6,"last update":"Wed Aug 9 14:51:07 2023","interfaces":[{"interface":"back","average":{"1min":1.5660000000000001,"5min":1.6459999999999999,"15min":1.641},"min":{"1min":1.2190000000000001,"5min":0.997,"15min":0.997},"max":{"1min":2.3879999999999999,"5min":2.4260000000000002,"15min":3.0750000000000002},"last":1.464},{"interface":"front","average":{"1min":1.597,"5min":1.651,"15min":1.631},"min":{"1min":1.202,"5min":1.034,"15min":1.034},"max":{"1min":2.2429999999999999,"5min":2.2429999999999999,"15min":2.2429999999999999},"last":1.256}]},{"osd":7,"last update":"Wed Aug 9 14:51:07 2023","interfaces":[{"interface":"back","average":{"1min":1.7110000000000001,"5min":1.74,"15min":1.7090000000000001},"min":{"1min":1.1839999999999999,"5min":1.1839999999999999,"15min":1.1639999999999999},"max":{"1min":2.1259999999999999,"5min":2.2250000000000001,"15min":2.3490000000000002},"last":1.427},{"interface":"front","average":{"1min":1.6100000000000001,"5min":1.6339999999999999,"15min":1.6200000000000001},"min":{"1min":1.3460000000000001,"5min":1.0940000000000001,"15min":1.0940000000000001},"max":{"1min":2.0249999999999999,"5min":2.4500000000000002,"15min":2.468},"last":1.3220000000000001}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":20,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1646,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":20,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":20,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1682,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":20,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":20,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":20,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":22,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":20,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":146,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":20,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":652,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":21,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":21,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":21,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":21,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":21,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":21,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":21,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":21,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-08-09T14:51:52.313 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-08-09T14:51:52.585 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:52.584+0000 7f9491db7640 1 Processor -- start 2023-08-09T14:51:52.586 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:52.584+0000 7f9491db7640 1 -- start start 2023-08-09T14:51:52.587 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:52.586+0000 7f9491db7640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f948c0ce080 0x7f948c0cafa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T14:51:52.587 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:52.586+0000 7f9491db7640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f948c0cb4e0 0x7f948c0cb970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T14:51:52.587 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:52.586+0000 7f9491db7640 1 -- --> v1:172.21.15.203:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f948c0cbeb0 con 0x7f948c0ce720 2023-08-09T14:51:52.587 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:52.586+0000 7f9491db7640 1 -- --> v2:172.21.15.43:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f948c0cd8a0 con 0x7f948c0ce080 2023-08-09T14:51:52.588 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:52.586+0000 7f9491db7640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f948c0cf350 con 0x7f948c0cb4e0 2023-08-09T14:51:52.588 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:52.586+0000 7f9482ffd640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f948c0ce080 0x7f948c0cafa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T14:51:52.589 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:52.586+0000 7f9482ffd640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f948c0ce080 0x7f948c0cafa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:57938/0 (socket says 172.21.15.43:57938) 2023-08-09T14:51:52.589 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:52.586+0000 7f9482ffd640 1 -- 172.21.15.43:0/1959340307 learned_addr learned my addr 172.21.15.43:0/1959340307 (peer_addr_for_me v2:172.21.15.43:0/0) 2023-08-09T14:51:52.589 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:52.587+0000 7f948bfff640 1 --2- 172.21.15.43:0/1959340307 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f948c0cb4e0 0x7f948c0cb970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T14:51:52.589 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:52.587+0000 7f9482ffd640 1 -- 172.21.15.43:0/1959340307 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f948c0cb4e0 msgr2=0x7f948c0cb970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-09T14:51:52.589 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:52.587+0000 7f9482ffd640 1 --2- 172.21.15.43:0/1959340307 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f948c0cb4e0 0x7f948c0cb970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T14:51:52.589 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:52.587+0000 7f9482ffd640 1 -- 172.21.15.43:0/1959340307 >> v1:172.21.15.203:6789/0 conn(0x7f948c0ce720 legacy=0x7f948c0ceb00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T14:51:52.590 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:52.587+0000 7f9482ffd640 1 -- 172.21.15.43:0/1959340307 --> v2:172.21.15.43:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f948c071a70 con 0x7f948c0ce080 2023-08-09T14:51:52.590 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:52.588+0000 7f9482ffd640 1 --2- 172.21.15.43:0/1959340307 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f948c0ce080 0x7f948c0cafa0 secure :-1 s=READY pgs=1882 cs=0 l=1 rev1=1 crypto rx=0x7f9474009870 tx=0x7f947402b8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9aaf432c660b9a93 server_cookie=0 in_seq=0 out_seq=0 2023-08-09T14:51:52.591 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:52.588+0000 7f948affd640 1 -- 172.21.15.43:0/1959340307 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9474031640 con 0x7f948c0ce080 2023-08-09T14:51:52.591 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:52.589+0000 7f948affd640 1 -- 172.21.15.43:0/1959340307 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f94740317a0 con 0x7f948c0ce080 2023-08-09T14:51:52.591 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:52.589+0000 7f948affd640 1 -- 172.21.15.43:0/1959340307 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9474031b30 con 0x7f948c0ce080 2023-08-09T14:51:52.591 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:52.589+0000 7f9491db7640 1 -- 172.21.15.43:0/1959340307 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f948c0ce080 msgr2=0x7f948c0cafa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T14:51:52.591 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:52.589+0000 7f9491db7640 1 --2- 172.21.15.43:0/1959340307 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f948c0ce080 0x7f948c0cafa0 secure :-1 s=READY pgs=1882 cs=0 l=1 rev1=1 crypto rx=0x7f9474009870 tx=0x7f947402b8f0 comp rx=0 tx=0).stop 2023-08-09T14:51:52.591 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:52.589+0000 7f9491db7640 1 -- 172.21.15.43:0/1959340307 shutdown_connections 2023-08-09T14:51:52.591 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:52.589+0000 7f9491db7640 1 --2- 172.21.15.43:0/1959340307 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f948c0cb4e0 0x7f948c0cb970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T14:51:52.592 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:52.589+0000 7f9491db7640 1 --2- 172.21.15.43:0/1959340307 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f948c0ce080 0x7f948c0cafa0 unknown :-1 s=CLOSED pgs=1882 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T14:51:52.592 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:52.589+0000 7f9491db7640 1 -- 172.21.15.43:0/1959340307 >> 172.21.15.43:0/1959340307 conn(0x7f948c0ccd80 msgr2=0x7f948c0cd190 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-09T14:51:52.592 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:52.589+0000 7f9491db7640 1 -- 172.21.15.43:0/1959340307 shutdown_connections 2023-08-09T14:51:52.592 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:52.589+0000 7f9491db7640 1 -- 172.21.15.43:0/1959340307 wait complete. 2023-08-09T14:51:52.592 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:52.591+0000 7f9491db7640 1 Processor -- start 2023-08-09T14:51:52.593 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:52.591+0000 7f9491db7640 1 -- start start 2023-08-09T14:51:52.593 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:52.592+0000 7f9491db7640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f948c0cb4e0 0x7f948c0afd60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T14:51:52.594 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:52.592+0000 7f9491db7640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f948c0ce080 0x7f948c0b5510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T14:51:52.594 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:52.592+0000 7f9491db7640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f948c0ce720 0x7f948c0b5a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T14:51:52.594 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:52.592+0000 7f9491db7640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f948c0b02a0 con 0x7f948c0ce720 2023-08-09T14:51:52.594 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:52.592+0000 7f9491db7640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f948c0b04b0 con 0x7f948c0cb4e0 2023-08-09T14:51:52.594 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:52.592+0000 7f9491db7640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f948c0b0790 con 0x7f948c0ce080 2023-08-09T14:51:52.595 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:52.592+0000 7f948bfff640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f948c0ce720 0x7f948c0b5a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T14:51:52.596 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:52.593+0000 7f948bfff640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f948c0ce720 0x7f948c0b5a50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:57948/0 (socket says 172.21.15.43:57948) 2023-08-09T14:51:52.596 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:52.593+0000 7f948bfff640 1 -- 172.21.15.43:0/1489568427 learned_addr learned my addr 172.21.15.43:0/1489568427 (peer_addr_for_me v2:172.21.15.43:0/0) 2023-08-09T14:51:52.596 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:52.593+0000 7f9482ffd640 1 --2- 172.21.15.43:0/1489568427 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f948c0ce080 0x7f948c0b5510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T14:51:52.596 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:52.593+0000 7f948bfff640 1 -- 172.21.15.43:0/1489568427 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f948c0ce080 msgr2=0x7f948c0b5510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T14:51:52.596 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:52.593+0000 7f948b7fe640 1 --2- 172.21.15.43:0/1489568427 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f948c0cb4e0 0x7f948c0afd60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T14:51:52.596 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:52.593+0000 7f948bfff640 1 --2- 172.21.15.43:0/1489568427 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f948c0ce080 0x7f948c0b5510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T14:51:52.597 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:52.593+0000 7f948bfff640 1 -- 172.21.15.43:0/1489568427 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f948c0cb4e0 msgr2=0x7f948c0afd60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T14:51:52.597 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:52.593+0000 7f948bfff640 1 --2- 172.21.15.43:0/1489568427 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f948c0cb4e0 0x7f948c0afd60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T14:51:52.597 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:52.593+0000 7f948bfff640 1 -- 172.21.15.43:0/1489568427 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f94740094d0 con 0x7f948c0ce720 2023-08-09T14:51:52.597 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:52.593+0000 7f948b7fe640 1 --2- 172.21.15.43:0/1489568427 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f948c0cb4e0 0x7f948c0afd60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-08-09T14:51:52.597 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:52.593+0000 7f948bfff640 1 --2- 172.21.15.43:0/1489568427 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f948c0ce720 0x7f948c0b5a50 secure :-1 s=READY pgs=1883 cs=0 l=1 rev1=1 crypto rx=0x7f9484007e60 tx=0x7f948400e4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-09T14:51:52.597 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:52.594+0000 7f9488ff9640 1 -- 172.21.15.43:0/1489568427 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9484014420 con 0x7f948c0ce720 2023-08-09T14:51:52.597 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:52.594+0000 7f9488ff9640 1 -- 172.21.15.43:0/1489568427 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f94840145a0 con 0x7f948c0ce720 2023-08-09T14:51:52.597 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:52.594+0000 7f9491db7640 1 -- 172.21.15.43:0/1489568427 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f948c0b0a70 con 0x7f948c0ce720 2023-08-09T14:51:52.598 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:52.594+0000 7f9488ff9640 1 -- 172.21.15.43:0/1489568427 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9484014950 con 0x7f948c0ce720 2023-08-09T14:51:52.598 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:52.594+0000 7f9491db7640 1 -- 172.21.15.43:0/1489568427 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f948c0b1030 con 0x7f948c0ce720 2023-08-09T14:51:52.598 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:52.595+0000 7f9491db7640 1 -- 172.21.15.43:0/1489568427 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f948c0b1220 con 0x7f948c0ce720 2023-08-09T14:51:52.599 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:52.596+0000 7f9488ff9640 1 -- 172.21.15.43:0/1489568427 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 79) v1 ==== 88443+0+0 (secure 0 0 0) 0x7f9484014ab0 con 0x7f948c0ce720 2023-08-09T14:51:52.599 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:52.597+0000 7f9488ff9640 1 --2- 172.21.15.43:0/1489568427 >> [v2:172.21.15.203:6832/2566816332,v1:172.21.15.203:6833/2566816332] conn(0x7f947006a170 0x7f947006c630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T14:51:52.599 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:52.597+0000 7f9488ff9640 1 -- 172.21.15.43:0/1489568427 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(172..172 src has 1..172) v4 ==== 6426+0+0 (secure 0 0 0) 0x7f9484089270 con 0x7f948c0ce720 2023-08-09T14:51:52.600 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:52.598+0000 7f948b7fe640 1 --2- 172.21.15.43:0/1489568427 >> [v2:172.21.15.203:6832/2566816332,v1:172.21.15.203:6833/2566816332] conn(0x7f947006a170 0x7f947006c630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T14:51:52.600 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:52.599+0000 7f948b7fe640 1 --2- 172.21.15.43:0/1489568427 >> [v2:172.21.15.203:6832/2566816332,v1:172.21.15.203:6833/2566816332] conn(0x7f947006a170 0x7f947006c630 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f947c000fb0 tx=0x7f947c009040 comp rx=0 tx=0).ready entity=mgr.7765 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-09T14:51:52.603 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:52.601+0000 7f9488ff9640 1 -- 172.21.15.43:0/1489568427 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178066 (secure 0 0 0) 0x7f94840580d0 con 0x7f948c0ce720 2023-08-09T14:51:52.792 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:52.790+0000 7f9491db7640 1 -- 172.21.15.43:0/1489568427 --> [v2:172.21.15.203:6832/2566816332,v1:172.21.15.203:6833/2566816332] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f948c0b0900 con 0x7f947006a170 2023-08-09T14:51:52.817 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:52.815+0000 7f9488ff9640 1 -- 172.21.15.43:0/1489568427 <== mgr.7765 v2:172.21.15.203:6832/2566816332 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+308933 (secure 0 0 0) 0x7f948c0b0900 con 0x7f947006a170 2023-08-09T14:51:52.820 INFO:teuthology.orchestra.run.smithi043.stdout: 2023-08-09T14:51:52.821 INFO:teuthology.orchestra.run.smithi043.stderr:dumped all 2023-08-09T14:51:52.823 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:52.820+0000 7f9491db7640 1 -- 172.21.15.43:0/1489568427 >> [v2:172.21.15.203:6832/2566816332,v1:172.21.15.203:6833/2566816332] conn(0x7f947006a170 msgr2=0x7f947006c630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T14:51:52.823 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:52.820+0000 7f9491db7640 1 --2- 172.21.15.43:0/1489568427 >> [v2:172.21.15.203:6832/2566816332,v1:172.21.15.203:6833/2566816332] conn(0x7f947006a170 0x7f947006c630 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f947c000fb0 tx=0x7f947c009040 comp rx=0 tx=0).stop 2023-08-09T14:51:52.823 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:52.821+0000 7f9491db7640 1 -- 172.21.15.43:0/1489568427 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f948c0ce720 msgr2=0x7f948c0b5a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T14:51:52.824 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:52.821+0000 7f9491db7640 1 --2- 172.21.15.43:0/1489568427 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f948c0ce720 0x7f948c0b5a50 secure :-1 s=READY pgs=1883 cs=0 l=1 rev1=1 crypto rx=0x7f9484007e60 tx=0x7f948400e4b0 comp rx=0 tx=0).stop 2023-08-09T14:51:52.824 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:52.821+0000 7f9491db7640 1 -- 172.21.15.43:0/1489568427 shutdown_connections 2023-08-09T14:51:52.824 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:52.821+0000 7f9491db7640 1 --2- 172.21.15.43:0/1489568427 >> [v2:172.21.15.203:6832/2566816332,v1:172.21.15.203:6833/2566816332] conn(0x7f947006a170 0x7f947006c630 unknown :-1 s=CLOSED pgs=66 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T14:51:52.824 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:52.821+0000 7f9491db7640 1 --2- 172.21.15.43:0/1489568427 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f948c0ce720 0x7f948c0b5a50 unknown :-1 s=CLOSED pgs=1883 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T14:51:52.824 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:52.821+0000 7f9491db7640 1 --2- 172.21.15.43:0/1489568427 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f948c0ce080 0x7f948c0b5510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T14:51:52.824 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:52.821+0000 7f9491db7640 1 --2- 172.21.15.43:0/1489568427 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f948c0cb4e0 0x7f948c0afd60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T14:51:52.825 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:52.821+0000 7f9491db7640 1 -- 172.21.15.43:0/1489568427 >> 172.21.15.43:0/1489568427 conn(0x7f948c0ccd80 msgr2=0x7f948c15db40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-09T14:51:52.825 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:52.821+0000 7f9491db7640 1 -- 172.21.15.43:0/1489568427 shutdown_connections 2023-08-09T14:51:52.825 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:52.821+0000 7f9491db7640 1 -- 172.21.15.43:0/1489568427 wait complete. 2023-08-09T14:51:52.845 INFO:teuthology.orchestra.run.smithi043.stdout:{"pg_ready":true,"pg_map":{"version":27,"stamp":"2023-08-09T14:51:51.226647+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":23,"num_object_clones":0,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":0,"num_read":1291,"num_read_kb":2654,"num_write":447,"num_write_kb":2430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":290,"ondisk_log_size":290,"up":210,"acting":210,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":95,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":406080,"kb_used_data":17720,"kb_used_omap":13,"kb_used_meta":388210,"kb_avail":754568640,"statfs":{"total":773094113280,"available":772678287360,"internally_reserved":0,"allocated":18145280,"data_stored":10355374,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":14248,"internal_metadata":397527128},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":28,"apply_latency_ms":28,"commit_latency_ns":28000000,"apply_latency_ns":28000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"5.491260"},"pg_stats":[{"pgid":"21.3f","version":"0'0","reported_seq":9,"reported_epoch":171,"state":"active+clean","last_fresh":"2023-08-09T14:51:47.794579+0000","last_change":"2023-08-09T14:51:47.792212+0000","last_active":"2023-08-09T14:51:47.794579+0000","last_peered":"2023-08-09T14:51:47.794579+0000","last_clean":"2023-08-09T14:51:47.794579+0000","last_became_active":"2023-08-09T14:51:47.791933+0000","last_became_peered":"2023-08-09T14:51:47.791933+0000","last_unstale":"2023-08-09T14:51:47.794579+0000","last_undegraded":"2023-08-09T14:51:47.794579+0000","last_fullsized":"2023-08-09T14:51:47.794579+0000","mapping_epoch":170,"log_start":"0'0","ondisk_log_start":"0'0","created":170,"last_epoch_clean":171,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T14:51:46.735123+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T14:51:46.735123+0000","last_clean_scrub_stamp":"2023-08-09T14:51:46.735123+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T23:10:26.794676+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"21.3e","version":"0'0","reported_seq":9,"reported_epoch":171,"state":"active+clean","last_fresh":"2023-08-09T14:51:47.788631+0000","last_change":"2023-08-09T14:51:47.784922+0000","last_active":"2023-08-09T14:51:47.788631+0000","last_peered":"2023-08-09T14:51:47.788631+0000","last_clean":"2023-08-09T14:51:47.788631+0000","last_became_active":"2023-08-09T14:51:47.784551+0000","last_became_peered":"2023-08-09T14:51:47.784551+0000","last_unstale":"2023-08-09T14:51:47.788631+0000","last_undegraded":"2023-08-09T14:51:47.788631+0000","last_fullsized":"2023-08-09T14:51:47.788631+0000","mapping_epoch":170,"log_start":"0'0","ondisk_log_start":"0'0","created":170,"last_epoch_clean":171,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T14:51:46.735123+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T14:51:46.735123+0000","last_clean_scrub_stamp":"2023-08-09T14:51:46.735123+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-11T01:01:21.767494+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"21.3d","version":"0'0","reported_seq":9,"reported_epoch":171,"state":"active+clean","last_fresh":"2023-08-09T14:51:47.797949+0000","last_change":"2023-08-09T14:51:47.793329+0000","last_active":"2023-08-09T14:51:47.797949+0000","last_peered":"2023-08-09T14:51:47.797949+0000","last_clean":"2023-08-09T14:51:47.797949+0000","last_became_active":"2023-08-09T14:51:47.793034+0000","last_became_peered":"2023-08-09T14:51:47.793034+0000","last_unstale":"2023-08-09T14:51:47.797949+0000","last_undegraded":"2023-08-09T14:51:47.797949+0000","last_fullsized":"2023-08-09T14:51:47.797949+0000","mapping_epoch":170,"log_start":"0'0","ondisk_log_start":"0'0","created":170,"last_epoch_clean":171,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T14:51:46.735123+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T14:51:46.735123+0000","last_clean_scrub_stamp":"2023-08-09T14:51:46.735123+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T18:53:38.906224+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"21.3c","version":"0'0","reported_seq":9,"reported_epoch":171,"state":"active+clean","last_fresh":"2023-08-09T14:51:47.807916+0000","last_change":"2023-08-09T14:51:47.800751+0000","last_active":"2023-08-09T14:51:47.807916+0000","last_peered":"2023-08-09T14:51:47.807916+0000","last_clean":"2023-08-09T14:51:47.807916+0000","last_became_active":"2023-08-09T14:51:47.800253+0000","last_became_peered":"2023-08-09T14:51:47.800253+0000","last_unstale":"2023-08-09T14:51:47.807916+0000","last_undegraded":"2023-08-09T14:51:47.807916+0000","last_fullsized":"2023-08-09T14:51:47.807916+0000","mapping_epoch":170,"log_start":"0'0","ondisk_log_start":"0'0","created":170,"last_epoch_clean":171,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T14:51:46.735123+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T14:51:46.735123+0000","last_clean_scrub_stamp":"2023-08-09T14:51:46.735123+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T23:19:38.181929+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"21.3b","version":"0'0","reported_seq":9,"reported_epoch":171,"state":"active+clean","last_fresh":"2023-08-09T14:51:47.796865+0000","last_change":"2023-08-09T14:51:47.792818+0000","last_active":"2023-08-09T14:51:47.796865+0000","last_peered":"2023-08-09T14:51:47.796865+0000","last_clean":"2023-08-09T14:51:47.796865+0000","last_became_active":"2023-08-09T14:51:47.792523+0000","last_became_peered":"2023-08-09T14:51:47.792523+0000","last_unstale":"2023-08-09T14:51:47.796865+0000","last_undegraded":"2023-08-09T14:51:47.796865+0000","last_fullsized":"2023-08-09T14:51:47.796865+0000","mapping_epoch":170,"log_start":"0'0","ondisk_log_start":"0'0","created":170,"last_epoch_clean":171,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T14:51:46.735123+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T14:51:46.735123+0000","last_clean_scrub_stamp":"2023-08-09T14:51:46.735123+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-11T02:37:22.012917+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"21.3a","version":"0'0","reported_seq":9,"reported_epoch":171,"state":"active+clean","last_fresh":"2023-08-09T14:51:47.801341+0000","last_change":"2023-08-09T14:51:47.795139+0000","last_active":"2023-08-09T14:51:47.801341+0000","last_peered":"2023-08-09T14:51:47.801341+0000","last_clean":"2023-08-09T14:51:47.801341+0000","last_became_active":"2023-08-09T14:51:47.794866+0000","last_became_peered":"2023-08-09T14:51:47.794866+0000","last_unstale":"2023-08-09T14:51:47.801341+0000","last_undegraded":"2023-08-09T14:51:47.801341+0000","last_fullsized":"2023-08-09T14:51:47.801341+0000","mapping_epoch":170,"log_start":"0'0","ondisk_log_start":"0'0","created":170,"last_epoch_clean":171,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T14:51:46.735123+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T14:51:46.735123+0000","last_clean_scrub_stamp":"2023-08-09T14:51:46.735123+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T19:03:21.686709+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"21.39","version":"0'0","reported_seq":9,"reported_epoch":171,"state":"active+clean","last_fresh":"2023-08-09T14:51:47.802119+0000","last_change":"2023-08-09T14:51:47.795696+0000","last_active":"2023-08-09T14:51:47.802119+0000","last_peered":"2023-08-09T14:51:47.802119+0000","last_clean":"2023-08-09T14:51:47.802119+0000","last_became_active":"2023-08-09T14:51:47.795019+0000","last_became_peered":"2023-08-09T14:51:47.795019+0000","last_unstale":"2023-08-09T14:51:47.802119+0000","last_undegraded":"2023-08-09T14:51:47.802119+0000","last_fullsized":"2023-08-09T14:51:47.802119+0000","mapping_epoch":170,"log_start":"0'0","ondisk_log_start":"0'0","created":170,"last_epoch_clean":171,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T14:51:46.735123+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T14:51:46.735123+0000","last_clean_scrub_stamp":"2023-08-09T14:51:46.735123+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T16:04:02.808388+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"21.38","version":"0'0","reported_seq":9,"reported_epoch":171,"state":"active+clean","last_fresh":"2023-08-09T14:51:47.787294+0000","last_change":"2023-08-09T14:51:47.785620+0000","last_active":"2023-08-09T14:51:47.787294+0000","last_peered":"2023-08-09T14:51:47.787294+0000","last_clean":"2023-08-09T14:51:47.787294+0000","last_became_active":"2023-08-09T14:51:47.783277+0000","last_became_peered":"2023-08-09T14:51:47.783277+0000","last_unstale":"2023-08-09T14:51:47.787294+0000","last_undegraded":"2023-08-09T14:51:47.787294+0000","last_fullsized":"2023-08-09T14:51:47.787294+0000","mapping_epoch":170,"log_start":"0'0","ondisk_log_start":"0'0","created":170,"last_epoch_clean":171,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T14:51:46.735123+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T14:51:46.735123+0000","last_clean_scrub_stamp":"2023-08-09T14:51:46.735123+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T19:31:07.562365+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"21.37","version":"0'0","reported_seq":9,"reported_epoch":171,"state":"active+clean","last_fresh":"2023-08-09T14:51:47.793300+0000","last_change":"2023-08-09T14:51:47.790654+0000","last_active":"2023-08-09T14:51:47.793300+0000","last_peered":"2023-08-09T14:51:47.793300+0000","last_clean":"2023-08-09T14:51:47.793300+0000","last_became_active":"2023-08-09T14:51:47.790356+0000","last_became_peered":"2023-08-09T14:51:47.790356+0000","last_unstale":"2023-08-09T14:51:47.793300+0000","last_undegraded":"2023-08-09T14:51:47.793300+0000","last_fullsized":"2023-08-09T14:51:47.793300+0000","mapping_epoch":170,"log_start":"0'0","ondisk_log_start":"0'0","created":170,"last_epoch_clean":171,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T14:51:46.735123+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T14:51:46.735123+0000","last_clean_scrub_stamp":"2023-08-09T14:51:46.735123+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-11T02:16:12.197435+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"21.36","version":"0'0","reported_seq":9,"reported_epoch":171,"state":"active+clean","last_fresh":"2023-08-09T14:51:47.805290+0000","last_change":"2023-08-09T14:51:47.795912+0000","last_active":"2023-08-09T14:51:47.805290+0000","last_peered":"2023-08-09T14:51:47.805290+0000","last_clean":"2023-08-09T14:51:47.805290+0000","last_became_active":"2023-08-09T14:51:47.795608+0000","last_became_peered":"2023-08-09T14:51:47.795608+0000","last_unstale":"2023-08-09T14:51:47.805290+0000","last_undegraded":"2023-08-09T14:51:47.805290+0000","last_fullsized":"2023-08-09T14:51:47.805290+0000","mapping_epoch":170,"log_start":"0'0","ondisk_log_start":"0'0","created":170,"last_epoch_clean":171,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T14:51:46.735123+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T14:51:46.735123+0000","last_clean_scrub_stamp":"2023-08-09T14:51:46.735123+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T17:35:49.467820+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"21.35","version":"0'0","reported_seq":9,"reported_epoch":171,"state":"active+clean","last_fresh":"2023-08-09T14:51:47.785607+0000","last_change":"2023-08-09T14:51:47.779209+0000","last_active":"2023-08-09T14:51:47.785607+0000","last_peered":"2023-08-09T14:51:47.785607+0000","last_clean":"2023-08-09T14:51:47.785607+0000","last_became_active":"2023-08-09T14:51:47.778879+0000","last_became_peered":"2023-08-09T14:51:47.778879+0000","last_unstale":"2023-08-09T14:51:47.785607+0000","last_undegraded":"2023-08-09T14:51:47.785607+0000","last_fullsized":"2023-08-09T14:51:47.785607+0000","mapping_epoch":170,"log_start":"0'0","ondisk_log_start":"0'0","created":170,"last_epoch_clean":171,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T14:51:46.735123+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T14:51:46.735123+0000","last_clean_scrub_stamp":"2023-08-09T14:51:46.735123+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T17:34:21.807704+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"21.34","version":"0'0","reported_seq":9,"reported_epoch":171,"state":"active+clean","last_fresh":"2023-08-09T14:51:47.808271+0000","last_change":"2023-08-09T14:51:47.801977+0000","last_active":"2023-08-09T14:51:47.808271+0000","last_peered":"2023-08-09T14:51:47.808271+0000","last_clean":"2023-08-09T14:51:47.808271+0000","last_became_active":"2023-08-09T14:51:47.801683+0000","last_became_peered":"2023-08-09T14:51:47.801683+0000","last_unstale":"2023-08-09T14:51:47.808271+0000","last_undegraded":"2023-08-09T14:51:47.808271+0000","last_fullsized":"2023-08-09T14:51:47.808271+0000","mapping_epoch":170,"log_start":"0'0","ondisk_log_start":"0'0","created":170,"last_epoch_clean":171,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T14:51:46.735123+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T14:51:46.735123+0000","last_clean_scrub_stamp":"2023-08-09T14:51:46.735123+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T23:01:06.813062+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"21.33","version":"0'0","reported_seq":9,"reported_epoch":171,"state":"active+clean","last_fresh":"2023-08-09T14:51:47.792932+0000","last_change":"2023-08-09T14:51:47.791347+0000","last_active":"2023-08-09T14:51:47.792932+0000","last_peered":"2023-08-09T14:51:47.792932+0000","last_clean":"2023-08-09T14:51:47.792932+0000","last_became_active":"2023-08-09T14:51:47.788285+0000","last_became_peered":"2023-08-09T14:51:47.788285+0000","last_unstale":"2023-08-09T14:51:47.792932+0000","last_undegraded":"2023-08-09T14:51:47.792932+0000","last_fullsized":"2023-08-09T14:51:47.792932+0000","mapping_epoch":170,"log_start":"0'0","ondisk_log_start":"0'0","created":170,"last_epoch_clean":171,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T14:51:46.735123+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T14:51:46.735123+0000","last_clean_scrub_stamp":"2023-08-09T14:51:46.735123+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T22:32:23.037398+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"21.32","version":"0'0","reported_seq":9,"reported_epoch":171,"state":"active+clean","last_fresh":"2023-08-09T14:51:47.793903+0000","last_change":"2023-08-09T14:51:47.790400+0000","last_active":"2023-08-09T14:51:47.793903+0000","last_peered":"2023-08-09T14:51:47.793903+0000","last_clean":"2023-08-09T14:51:47.793903+0000","last_became_active":"2023-08-09T14:51:47.790057+0000","last_became_peered":"2023-08-09T14:51:47.790057+0000","last_unstale":"2023-08-09T14:51:47.793903+0000","last_undegraded":"2023-08-09T14:51:47.793903+0000","last_fullsized":"2023-08-09T14:51:47.793903+0000","mapping_epoch":170,"log_start":"0'0","ondisk_log_start":"0'0","created":170,"last_epoch_clean":171,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T14:51:46.735123+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T14:51:46.735123+0000","last_clean_scrub_stamp":"2023-08-09T14:51:46.735123+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T19:01:52.632957+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"21.31","version":"0'0","reported_seq":9,"reported_epoch":171,"state":"active+clean","last_fresh":"2023-08-09T14:51:47.805565+0000","last_change":"2023-08-09T14:51:47.795574+0000","last_active":"2023-08-09T14:51:47.805565+0000","last_peered":"2023-08-09T14:51:47.805565+0000","last_clean":"2023-08-09T14:51:47.805565+0000","last_became_active":"2023-08-09T14:51:47.794319+0000","last_became_peered":"2023-08-09T14:51:47.794319+0000","last_unstale":"2023-08-09T14:51:47.805565+0000","last_undegraded":"2023-08-09T14:51:47.805565+0000","last_fullsized":"2023-08-09T14:51:47.805565+0000","mapping_epoch":170,"log_start":"0'0","ondisk_log_start":"0'0","created":170,"last_epoch_clean":171,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T14:51:46.735123+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T14:51:46.735123+0000","last_clean_scrub_stamp":"2023-08-09T14:51:46.735123+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T21:37:15.799330+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"21.30","version":"0'0","reported_seq":9,"reported_epoch":171,"state":"active+clean","last_fresh":"2023-08-09T14:51:47.798403+0000","last_change":"2023-08-09T14:51:47.791421+0000","last_active":"2023-08-09T14:51:47.798403+0000","last_peered":"2023-08-09T14:51:47.798403+0000","last_clean":"2023-08-09T14:51:47.798403+0000","last_became_active":"2023-08-09T14:51:47.791128+0000","last_became_peered":"2023-08-09T14:51:47.791128+0000","last_unstale":"2023-08-09T14:51:47.798403+0000","last_undegraded":"2023-08-09T14:51:47.798403+0000","last_fullsized":"2023-08-09T14:51:47.798403+0000","mapping_epoch":170,"log_start":"0'0","ondisk_log_start":"0'0","created":170,"last_epoch_clean":171,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T14:51:46.735123+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T14:51:46.735123+0000","last_clean_scrub_stamp":"2023-08-09T14:51:46.735123+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T21:06:30.391234+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"21.2f","version":"0'0","reported_seq":9,"reported_epoch":171,"state":"active+clean","last_fresh":"2023-08-09T14:51:47.805492+0000","last_change":"2023-08-09T14:51:47.798702+0000","last_active":"2023-08-09T14:51:47.805492+0000","last_peered":"2023-08-09T14:51:47.805492+0000","last_clean":"2023-08-09T14:51:47.805492+0000","last_became_active":"2023-08-09T14:51:47.798262+0000","last_became_peered":"2023-08-09T14:51:47.798262+0000","last_unstale":"2023-08-09T14:51:47.805492+0000","last_undegraded":"2023-08-09T14:51:47.805492+0000","last_fullsized":"2023-08-09T14:51:47.805492+0000","mapping_epoch":170,"log_start":"0'0","ondisk_log_start":"0'0","created":170,"last_epoch_clean":171,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T14:51:46.735123+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T14:51:46.735123+0000","last_clean_scrub_stamp":"2023-08-09T14:51:46.735123+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T19:58:42.741137+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"21.2e","version":"0'0","reported_seq":9,"reported_epoch":171,"state":"active+clean","last_fresh":"2023-08-09T14:51:47.799481+0000","last_change":"2023-08-09T14:51:47.797021+0000","last_active":"2023-08-09T14:51:47.799481+0000","last_peered":"2023-08-09T14:51:47.799481+0000","last_clean":"2023-08-09T14:51:47.799481+0000","last_became_active":"2023-08-09T14:51:47.794339+0000","last_became_peered":"2023-08-09T14:51:47.794339+0000","last_unstale":"2023-08-09T14:51:47.799481+0000","last_undegraded":"2023-08-09T14:51:47.799481+0000","last_fullsized":"2023-08-09T14:51:47.799481+0000","mapping_epoch":170,"log_start":"0'0","ondisk_log_start":"0'0","created":170,"last_epoch_clean":171,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T14:51:46.735123+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T14:51:46.735123+0000","last_clean_scrub_stamp":"2023-08-09T14:51:46.735123+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T19:32:47.594934+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"21.2d","version":"0'0","reported_seq":9,"reported_epoch":171,"state":"active+clean","last_fresh":"2023-08-09T14:51:47.792448+0000","last_change":"2023-08-09T14:51:47.789326+0000","last_active":"2023-08-09T14:51:47.792448+0000","last_peered":"2023-08-09T14:51:47.792448+0000","last_clean":"2023-08-09T14:51:47.792448+0000","last_became_active":"2023-08-09T14:51:47.788975+0000","last_became_peered":"2023-08-09T14:51:47.788975+0000","last_unstale":"2023-08-09T14:51:47.792448+0000","last_undegraded":"2023-08-09T14:51:47.792448+0000","last_fullsized":"2023-08-09T14:51:47.792448+0000","mapping_epoch":170,"log_start":"0'0","ondisk_log_start":"0'0","created":170,"last_epoch_clean":171,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T14:51:46.735123+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T14:51:46.735123+0000","last_clean_scrub_stamp":"2023-08-09T14:51:46.735123+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-11T00:15:00.756526+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"21.2c","version":"0'0","reported_seq":9,"reported_epoch":171,"state":"active+clean","last_fresh":"2023-08-09T14:51:47.806034+0000","last_change":"2023-08-09T14:51:47.798711+0000","last_active":"2023-08-09T14:51:47.806034+0000","last_peered":"2023-08-09T14:51:47.806034+0000","last_clean":"2023-08-09T14:51:47.806034+0000","last_became_active":"2023-08-09T14:51:47.798399+0000","last_became_peered":"2023-08-09T14:51:47.798399+0000","last_unstale":"2023-08-09T14:51:47.806034+0000","last_undegraded":"2023-08-09T14:51:47.806034+0000","last_fullsized":"2023-08-09T14:51:47.806034+0000","mapping_epoch":170,"log_start":"0'0","ondisk_log_start":"0'0","created":170,"last_epoch_clean":171,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T14:51:46.735123+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T14:51:46.735123+0000","last_clean_scrub_stamp":"2023-08-09T14:51:46.735123+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T23:16:39.005612+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"21.2b","version":"0'0","reported_seq":9,"reported_epoch":171,"state":"active+clean","last_fresh":"2023-08-09T14:51:47.805689+0000","last_change":"2023-08-09T14:51:47.796099+0000","last_active":"2023-08-09T14:51:47.805689+0000","last_peered":"2023-08-09T14:51:47.805689+0000","last_clean":"2023-08-09T14:51:47.805689+0000","last_became_active":"2023-08-09T14:51:47.795766+0000","last_became_peered":"2023-08-09T14:51:47.795766+0000","last_unstale":"2023-08-09T14:51:47.805689+0000","last_undegraded":"2023-08-09T14:51:47.805689+0000","last_fullsized":"2023-08-09T14:51:47.805689+0000","mapping_epoch":170,"log_start":"0'0","ondisk_log_start":"0'0","created":170,"last_epoch_clean":171,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T14:51:46.735123+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T14:51:46.735123+0000","last_clean_scrub_stamp":"2023-08-09T14:51:46.735123+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T16:18:50.656741+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"21.2a","version":"0'0","reported_seq":9,"reported_epoch":171,"state":"active+clean","last_fresh":"2023-08-09T14:51:47.791675+0000","last_change":"2023-08-09T14:51:47.789658+0000","last_active":"2023-08-09T14:51:47.791675+0000","last_peered":"2023-08-09T14:51:47.791675+0000","last_clean":"2023-08-09T14:51:47.791675+0000","last_became_active":"2023-08-09T14:51:47.784732+0000","last_became_peered":"2023-08-09T14:51:47.784732+0000","last_unstale":"2023-08-09T14:51:47.791675+0000","last_undegraded":"2023-08-09T14:51:47.791675+0000","last_fullsized":"2023-08-09T14:51:47.791675+0000","mapping_epoch":170,"log_start":"0'0","ondisk_log_start":"0'0","created":170,"last_epoch_clean":171,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T14:51:46.735123+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T14:51:46.735123+0000","last_clean_scrub_stamp":"2023-08-09T14:51:46.735123+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T22:41:30.767499+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"21.29","version":"0'0","reported_seq":9,"reported_epoch":171,"state":"active+clean","last_fresh":"2023-08-09T14:51:47.791613+0000","last_change":"2023-08-09T14:51:47.789361+0000","last_active":"2023-08-09T14:51:47.791613+0000","last_peered":"2023-08-09T14:51:47.791613+0000","last_clean":"2023-08-09T14:51:47.791613+0000","last_became_active":"2023-08-09T14:51:47.789071+0000","last_became_peered":"2023-08-09T14:51:47.789071+0000","last_unstale":"2023-08-09T14:51:47.791613+0000","last_undegraded":"2023-08-09T14:51:47.791613+0000","last_fullsized":"2023-08-09T14:51:47.791613+0000","mapping_epoch":170,"log_start":"0'0","ondisk_log_start":"0'0","created":170,"last_epoch_clean":171,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T14:51:46.735123+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T14:51:46.735123+0000","last_clean_scrub_stamp":"2023-08-09T14:51:46.735123+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-11T00:26:44.647860+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"21.28","version":"0'0","reported_seq":9,"reported_epoch":171,"state":"active+clean","last_fresh":"2023-08-09T14:51:47.801095+0000","last_change":"2023-08-09T14:51:47.792988+0000","last_active":"2023-08-09T14:51:47.801095+0000","last_peered":"2023-08-09T14:51:47.801095+0000","last_clean":"2023-08-09T14:51:47.801095+0000","last_became_active":"2023-08-09T14:51:47.792695+0000","last_became_peered":"2023-08-09T14:51:47.792695+0000","last_unstale":"2023-08-09T14:51:47.801095+0000","last_undegraded":"2023-08-09T14:51:47.801095+0000","last_fullsized":"2023-08-09T14:51:47.801095+0000","mapping_epoch":170,"log_start":"0'0","ondisk_log_start":"0'0","created":170,"last_epoch_clean":171,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T14:51:46.735123+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T14:51:46.735123+0000","last_clean_scrub_stamp":"2023-08-09T14:51:46.735123+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T22:08:25.169397+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"21.27","version":"0'0","reported_seq":9,"reported_epoch":171,"state":"active+clean","last_fresh":"2023-08-09T14:51:47.805881+0000","last_change":"2023-08-09T14:51:47.798390+0000","last_active":"2023-08-09T14:51:47.805881+0000","last_peered":"2023-08-09T14:51:47.805881+0000","last_clean":"2023-08-09T14:51:47.805881+0000","last_became_active":"2023-08-09T14:51:47.797984+0000","last_became_peered":"2023-08-09T14:51:47.797984+0000","last_unstale":"2023-08-09T14:51:47.805881+0000","last_undegraded":"2023-08-09T14:51:47.805881+0000","last_fullsized":"2023-08-09T14:51:47.805881+0000","mapping_epoch":170,"log_start":"0'0","ondisk_log_start":"0'0","created":170,"last_epoch_clean":171,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T14:51:46.735123+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T14:51:46.735123+0000","last_clean_scrub_stamp":"2023-08-09T14:51:46.735123+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-11T01:24:05.556919+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"21.26","version":"0'0","reported_seq":9,"reported_epoch":171,"state":"active+clean","last_fresh":"2023-08-09T14:51:47.796002+0000","last_change":"2023-08-09T14:51:47.790314+0000","last_active":"2023-08-09T14:51:47.796002+0000","last_peered":"2023-08-09T14:51:47.796002+0000","last_clean":"2023-08-09T14:51:47.796002+0000","last_became_active":"2023-08-09T14:51:47.790015+0000","last_became_peered":"2023-08-09T14:51:47.790015+0000","last_unstale":"2023-08-09T14:51:47.796002+0000","last_undegraded":"2023-08-09T14:51:47.796002+0000","last_fullsized":"2023-08-09T14:51:47.796002+0000","mapping_epoch":170,"log_start":"0'0","ondisk_log_start":"0'0","created":170,"last_epoch_clean":171,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T14:51:46.735123+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T14:51:46.735123+0000","last_clean_scrub_stamp":"2023-08-09T14:51:46.735123+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T19:31:26.638403+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"21.25","version":"0'0","reported_seq":9,"reported_epoch":171,"state":"active+clean","last_fresh":"2023-08-09T14:51:47.805731+0000","last_change":"2023-08-09T14:51:47.800254+0000","last_active":"2023-08-09T14:51:47.805731+0000","last_peered":"2023-08-09T14:51:47.805731+0000","last_clean":"2023-08-09T14:51:47.805731+0000","last_became_active":"2023-08-09T14:51:47.799969+0000","last_became_peered":"2023-08-09T14:51:47.799969+0000","last_unstale":"2023-08-09T14:51:47.805731+0000","last_undegraded":"2023-08-09T14:51:47.805731+0000","last_fullsized":"2023-08-09T14:51:47.805731+0000","mapping_epoch":170,"log_start":"0'0","ondisk_log_start":"0'0","created":170,"last_epoch_clean":171,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T14:51:46.735123+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T14:51:46.735123+0000","last_clean_scrub_stamp":"2023-08-09T14:51:46.735123+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-11T00:44:32.393045+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"21.24","version":"0'0","reported_seq":9,"reported_epoch":171,"state":"active+clean","last_fresh":"2023-08-09T14:51:47.803141+0000","last_change":"2023-08-09T14:51:47.797986+0000","last_active":"2023-08-09T14:51:47.803141+0000","last_peered":"2023-08-09T14:51:47.803141+0000","last_clean":"2023-08-09T14:51:47.803141+0000","last_became_active":"2023-08-09T14:51:47.797710+0000","last_became_peered":"2023-08-09T14:51:47.797710+0000","last_unstale":"2023-08-09T14:51:47.803141+0000","last_undegraded":"2023-08-09T14:51:47.803141+0000","last_fullsized":"2023-08-09T14:51:47.803141+0000","mapping_epoch":170,"log_start":"0'0","ondisk_log_start":"0'0","created":170,"last_epoch_clean":171,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T14:51:46.735123+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T14:51:46.735123+0000","last_clean_scrub_stamp":"2023-08-09T14:51:46.735123+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T22:55:26.964070+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"21.23","version":"0'0","reported_seq":9,"reported_epoch":171,"state":"active+clean","last_fresh":"2023-08-09T14:51:47.798314+0000","last_change":"2023-08-09T14:51:47.794542+0000","last_active":"2023-08-09T14:51:47.798314+0000","last_peered":"2023-08-09T14:51:47.798314+0000","last_clean":"2023-08-09T14:51:47.798314+0000","last_became_active":"2023-08-09T14:51:47.794236+0000","last_became_peered":"2023-08-09T14:51:47.794236+0000","last_unstale":"2023-08-09T14:51:47.798314+0000","last_undegraded":"2023-08-09T14:51:47.798314+0000","last_fullsized":"2023-08-09T14:51:47.798314+0000","mapping_epoch":170,"log_start":"0'0","ondisk_log_start":"0'0","created":170,"last_epoch_clean":171,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T14:51:46.735123+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T14:51:46.735123+0000","last_clean_scrub_stamp":"2023-08-09T14:51:46.735123+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T20:06:05.899184+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"21.22","version":"0'0","reported_seq":9,"reported_epoch":171,"state":"active+clean","last_fresh":"2023-08-09T14:51:47.789893+0000","last_change":"2023-08-09T14:51:47.781712+0000","last_active":"2023-08-09T14:51:47.789893+0000","last_peered":"2023-08-09T14:51:47.789893+0000","last_clean":"2023-08-09T14:51:47.789893+0000","last_became_active":"2023-08-09T14:51:47.781050+0000","last_became_peered":"2023-08-09T14:51:47.781050+0000","last_unstale":"2023-08-09T14:51:47.789893+0000","last_undegraded":"2023-08-09T14:51:47.789893+0000","last_fullsized":"2023-08-09T14:51:47.789893+0000","mapping_epoch":170,"log_start":"0'0","ondisk_log_start":"0'0","created":170,"last_epoch_clean":171,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T14:51:46.735123+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T14:51:46.735123+0000","last_clean_scrub_stamp":"2023-08-09T14:51:46.735123+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T16:20:39.930781+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"21.21","version":"0'0","reported_seq":9,"reported_epoch":171,"state":"active+clean","last_fresh":"2023-08-09T14:51:47.796637+0000","last_change":"2023-08-09T14:51:47.795155+0000","last_active":"2023-08-09T14:51:47.796637+0000","last_peered":"2023-08-09T14:51:47.796637+0000","last_clean":"2023-08-09T14:51:47.796637+0000","last_became_active":"2023-08-09T14:51:47.794847+0000","last_became_peered":"2023-08-09T14:51:47.794847+0000","last_unstale":"2023-08-09T14:51:47.796637+0000","last_undegraded":"2023-08-09T14:51:47.796637+0000","last_fullsized":"2023-08-09T14:51:47.796637+0000","mapping_epoch":170,"log_start":"0'0","ondisk_log_start":"0'0","created":170,"last_epoch_clean":171,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T14:51:46.735123+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T14:51:46.735123+0000","last_clean_scrub_stamp":"2023-08-09T14:51:46.735123+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-11T01:07:09.488460+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"21.20","version":"0'0","reported_seq":9,"reported_epoch":171,"state":"active+clean","last_fresh":"2023-08-09T14:51:47.792921+0000","last_change":"2023-08-09T14:51:47.787101+0000","last_active":"2023-08-09T14:51:47.792921+0000","last_peered":"2023-08-09T14:51:47.792921+0000","last_clean":"2023-08-09T14:51:47.792921+0000","last_became_active":"2023-08-09T14:51:47.786782+0000","last_became_peered":"2023-08-09T14:51:47.786782+0000","last_unstale":"2023-08-09T14:51:47.792921+0000","last_undegraded":"2023-08-09T14:51:47.792921+0000","last_fullsized":"2023-08-09T14:51:47.792921+0000","mapping_epoch":170,"log_start":"0'0","ondisk_log_start":"0'0","created":170,"last_epoch_clean":171,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T14:51:46.735123+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T14:51:46.735123+0000","last_clean_scrub_stamp":"2023-08-09T14:51:46.735123+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T15:18:45.272412+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"21.1e","version":"0'0","reported_seq":9,"reported_epoch":171,"state":"active+clean","last_fresh":"2023-08-09T14:51:47.808887+0000","last_change":"2023-08-09T14:51:47.803915+0000","last_active":"2023-08-09T14:51:47.808887+0000","last_peered":"2023-08-09T14:51:47.808887+0000","last_clean":"2023-08-09T14:51:47.808887+0000","last_became_active":"2023-08-09T14:51:47.802801+0000","last_became_peered":"2023-08-09T14:51:47.802801+0000","last_unstale":"2023-08-09T14:51:47.808887+0000","last_undegraded":"2023-08-09T14:51:47.808887+0000","last_fullsized":"2023-08-09T14:51:47.808887+0000","mapping_epoch":170,"log_start":"0'0","ondisk_log_start":"0'0","created":170,"last_epoch_clean":171,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T14:51:46.735123+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T14:51:46.735123+0000","last_clean_scrub_stamp":"2023-08-09T14:51:46.735123+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T22:54:21.263039+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"20.1f","version":"171'4","reported_seq":15,"reported_epoch":171,"state":"active+clean","last_fresh":"2023-08-09T14:51:47.816596+0000","last_change":"2023-08-09T14:51:46.786260+0000","last_active":"2023-08-09T14:51:47.816596+0000","last_peered":"2023-08-09T14:51:47.816596+0000","last_clean":"2023-08-09T14:51:47.816596+0000","last_became_active":"2023-08-09T14:51:46.785985+0000","last_became_peered":"2023-08-09T14:51:46.785985+0000","last_unstale":"2023-08-09T14:51:47.816596+0000","last_undegraded":"2023-08-09T14:51:47.816596+0000","last_fullsized":"2023-08-09T14:51:47.816596+0000","mapping_epoch":169,"log_start":"0'0","ondisk_log_start":"0'0","created":169,"last_epoch_clean":170,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T14:51:45.719739+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T14:51:45.719739+0000","last_clean_scrub_stamp":"2023-08-09T14:51:45.719739+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T18:00:32.806143+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"21.1f","version":"0'0","reported_seq":9,"reported_epoch":171,"state":"active+clean","last_fresh":"2023-08-09T14:51:47.791972+0000","last_change":"2023-08-09T14:51:47.786043+0000","last_active":"2023-08-09T14:51:47.791972+0000","last_peered":"2023-08-09T14:51:47.791972+0000","last_clean":"2023-08-09T14:51:47.791972+0000","last_became_active":"2023-08-09T14:51:47.785765+0000","last_became_peered":"2023-08-09T14:51:47.785765+0000","last_unstale":"2023-08-09T14:51:47.791972+0000","last_undegraded":"2023-08-09T14:51:47.791972+0000","last_fullsized":"2023-08-09T14:51:47.791972+0000","mapping_epoch":170,"log_start":"0'0","ondisk_log_start":"0'0","created":170,"last_epoch_clean":171,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T14:51:46.735123+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T14:51:46.735123+0000","last_clean_scrub_stamp":"2023-08-09T14:51:46.735123+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T17:44:30.221845+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"20.1e","version":"171'1","reported_seq":12,"reported_epoch":171,"state":"active+clean","last_fresh":"2023-08-09T14:51:47.814540+0000","last_change":"2023-08-09T14:51:46.791336+0000","last_active":"2023-08-09T14:51:47.814540+0000","last_peered":"2023-08-09T14:51:47.814540+0000","last_clean":"2023-08-09T14:51:47.814540+0000","last_became_active":"2023-08-09T14:51:46.791007+0000","last_became_peered":"2023-08-09T14:51:46.791007+0000","last_unstale":"2023-08-09T14:51:47.814540+0000","last_undegraded":"2023-08-09T14:51:47.814540+0000","last_fullsized":"2023-08-09T14:51:47.814540+0000","mapping_epoch":169,"log_start":"0'0","ondisk_log_start":"0'0","created":169,"last_epoch_clean":170,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T14:51:45.719739+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T14:51:45.719739+0000","last_clean_scrub_stamp":"2023-08-09T14:51:45.719739+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-11T02:11:10.030977+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"21.1c","version":"0'0","reported_seq":9,"reported_epoch":171,"state":"active+clean","last_fresh":"2023-08-09T14:51:47.782971+0000","last_change":"2023-08-09T14:51:47.774632+0000","last_active":"2023-08-09T14:51:47.782971+0000","last_peered":"2023-08-09T14:51:47.782971+0000","last_clean":"2023-08-09T14:51:47.782971+0000","last_became_active":"2023-08-09T14:51:47.773892+0000","last_became_peered":"2023-08-09T14:51:47.773892+0000","last_unstale":"2023-08-09T14:51:47.782971+0000","last_undegraded":"2023-08-09T14:51:47.782971+0000","last_fullsized":"2023-08-09T14:51:47.782971+0000","mapping_epoch":170,"log_start":"0'0","ondisk_log_start":"0'0","created":170,"last_epoch_clean":171,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T14:51:46.735123+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T14:51:46.735123+0000","last_clean_scrub_stamp":"2023-08-09T14:51:46.735123+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T15:24:05.666551+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"20.1d","version":"171'2","reported_seq":13,"reported_epoch":171,"state":"active+clean","last_fresh":"2023-08-09T14:51:47.812306+0000","last_change":"2023-08-09T14:51:46.782906+0000","last_active":"2023-08-09T14:51:47.812306+0000","last_peered":"2023-08-09T14:51:47.812306+0000","last_clean":"2023-08-09T14:51:47.812306+0000","last_became_active":"2023-08-09T14:51:46.782616+0000","last_became_peered":"2023-08-09T14:51:46.782616+0000","last_unstale":"2023-08-09T14:51:47.812306+0000","last_undegraded":"2023-08-09T14:51:47.812306+0000","last_fullsized":"2023-08-09T14:51:47.812306+0000","mapping_epoch":169,"log_start":"0'0","ondisk_log_start":"0'0","created":169,"last_epoch_clean":170,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T14:51:45.719739+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T14:51:45.719739+0000","last_clean_scrub_stamp":"2023-08-09T14:51:45.719739+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T20:36:40.482735+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"21.1d","version":"0'0","reported_seq":9,"reported_epoch":171,"state":"active+clean","last_fresh":"2023-08-09T14:51:47.808530+0000","last_change":"2023-08-09T14:51:47.802494+0000","last_active":"2023-08-09T14:51:47.808530+0000","last_peered":"2023-08-09T14:51:47.808530+0000","last_clean":"2023-08-09T14:51:47.808530+0000","last_became_active":"2023-08-09T14:51:47.802101+0000","last_became_peered":"2023-08-09T14:51:47.802101+0000","last_unstale":"2023-08-09T14:51:47.808530+0000","last_undegraded":"2023-08-09T14:51:47.808530+0000","last_fullsized":"2023-08-09T14:51:47.808530+0000","mapping_epoch":170,"log_start":"0'0","ondisk_log_start":"0'0","created":170,"last_epoch_clean":171,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T14:51:46.735123+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T14:51:46.735123+0000","last_clean_scrub_stamp":"2023-08-09T14:51:46.735123+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T17:07:44.856176+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"20.1c","version":"171'2","reported_seq":13,"reported_epoch":171,"state":"active+clean","last_fresh":"2023-08-09T14:51:47.811760+0000","last_change":"2023-08-09T14:51:46.782922+0000","last_active":"2023-08-09T14:51:47.811760+0000","last_peered":"2023-08-09T14:51:47.811760+0000","last_clean":"2023-08-09T14:51:47.811760+0000","last_became_active":"2023-08-09T14:51:46.782289+0000","last_became_peered":"2023-08-09T14:51:46.782289+0000","last_unstale":"2023-08-09T14:51:47.811760+0000","last_undegraded":"2023-08-09T14:51:47.811760+0000","last_fullsized":"2023-08-09T14:51:47.811760+0000","mapping_epoch":169,"log_start":"0'0","ondisk_log_start":"0'0","created":169,"last_epoch_clean":170,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T14:51:45.719739+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T14:51:45.719739+0000","last_clean_scrub_stamp":"2023-08-09T14:51:45.719739+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-11T00:00:24.424384+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"21.1a","version":"0'0","reported_seq":9,"reported_epoch":171,"state":"active+clean","last_fresh":"2023-08-09T14:51:47.797277+0000","last_change":"2023-08-09T14:51:47.795851+0000","last_active":"2023-08-09T14:51:47.797277+0000","last_peered":"2023-08-09T14:51:47.797277+0000","last_clean":"2023-08-09T14:51:47.797277+0000","last_became_active":"2023-08-09T14:51:47.795514+0000","last_became_peered":"2023-08-09T14:51:47.795514+0000","last_unstale":"2023-08-09T14:51:47.797277+0000","last_undegraded":"2023-08-09T14:51:47.797277+0000","last_fullsized":"2023-08-09T14:51:47.797277+0000","mapping_epoch":170,"log_start":"0'0","ondisk_log_start":"0'0","created":170,"last_epoch_clean":171,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T14:51:46.735123+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T14:51:46.735123+0000","last_clean_scrub_stamp":"2023-08-09T14:51:46.735123+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-11T02:50:37.653241+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"20.1b","version":"171'1","reported_seq":12,"reported_epoch":171,"state":"active+clean","last_fresh":"2023-08-09T14:51:47.811597+0000","last_change":"2023-08-09T14:51:46.786986+0000","last_active":"2023-08-09T14:51:47.811597+0000","last_peered":"2023-08-09T14:51:47.811597+0000","last_clean":"2023-08-09T14:51:47.811597+0000","last_became_active":"2023-08-09T14:51:46.786728+0000","last_became_peered":"2023-08-09T14:51:46.786728+0000","last_unstale":"2023-08-09T14:51:47.811597+0000","last_undegraded":"2023-08-09T14:51:47.811597+0000","last_fullsized":"2023-08-09T14:51:47.811597+0000","mapping_epoch":169,"log_start":"0'0","ondisk_log_start":"0'0","created":169,"last_epoch_clean":170,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T14:51:45.719739+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T14:51:45.719739+0000","last_clean_scrub_stamp":"2023-08-09T14:51:45.719739+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-11T00:13:44.288100+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"21.1b","version":"0'0","reported_seq":9,"reported_epoch":171,"state":"active+clean","last_fresh":"2023-08-09T14:51:47.807896+0000","last_change":"2023-08-09T14:51:47.800335+0000","last_active":"2023-08-09T14:51:47.807896+0000","last_peered":"2023-08-09T14:51:47.807896+0000","last_clean":"2023-08-09T14:51:47.807896+0000","last_became_active":"2023-08-09T14:51:47.800029+0000","last_became_peered":"2023-08-09T14:51:47.800029+0000","last_unstale":"2023-08-09T14:51:47.807896+0000","last_undegraded":"2023-08-09T14:51:47.807896+0000","last_fullsized":"2023-08-09T14:51:47.807896+0000","mapping_epoch":170,"log_start":"0'0","ondisk_log_start":"0'0","created":170,"last_epoch_clean":171,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T14:51:46.735123+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T14:51:46.735123+0000","last_clean_scrub_stamp":"2023-08-09T14:51:46.735123+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T18:10:10.281829+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"20.1a","version":"171'1","reported_seq":12,"reported_epoch":171,"state":"active+clean","last_fresh":"2023-08-09T14:51:47.811066+0000","last_change":"2023-08-09T14:51:46.786553+0000","last_active":"2023-08-09T14:51:47.811066+0000","last_peered":"2023-08-09T14:51:47.811066+0000","last_clean":"2023-08-09T14:51:47.811066+0000","last_became_active":"2023-08-09T14:51:46.786191+0000","last_became_peered":"2023-08-09T14:51:46.786191+0000","last_unstale":"2023-08-09T14:51:47.811066+0000","last_undegraded":"2023-08-09T14:51:47.811066+0000","last_fullsized":"2023-08-09T14:51:47.811066+0000","mapping_epoch":169,"log_start":"0'0","ondisk_log_start":"0'0","created":169,"last_epoch_clean":170,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T14:51:45.719739+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T14:51:45.719739+0000","last_clean_scrub_stamp":"2023-08-09T14:51:45.719739+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T17:52:01.935407+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"21.18","version":"0'0","reported_seq":9,"reported_epoch":171,"state":"active+clean","last_fresh":"2023-08-09T14:51:47.783494+0000","last_change":"2023-08-09T14:51:47.780937+0000","last_active":"2023-08-09T14:51:47.783494+0000","last_peered":"2023-08-09T14:51:47.783494+0000","last_clean":"2023-08-09T14:51:47.783494+0000","last_became_active":"2023-08-09T14:51:47.780284+0000","last_became_peered":"2023-08-09T14:51:47.780284+0000","last_unstale":"2023-08-09T14:51:47.783494+0000","last_undegraded":"2023-08-09T14:51:47.783494+0000","last_fullsized":"2023-08-09T14:51:47.783494+0000","mapping_epoch":170,"log_start":"0'0","ondisk_log_start":"0'0","created":170,"last_epoch_clean":171,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T14:51:46.735123+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T14:51:46.735123+0000","last_clean_scrub_stamp":"2023-08-09T14:51:46.735123+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T16:34:26.749551+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"20.19","version":"0'0","reported_seq":11,"reported_epoch":171,"state":"active+clean","last_fresh":"2023-08-09T14:51:47.780773+0000","last_change":"2023-08-09T14:51:46.786597+0000","last_active":"2023-08-09T14:51:47.780773+0000","last_peered":"2023-08-09T14:51:47.780773+0000","last_clean":"2023-08-09T14:51:47.780773+0000","last_became_active":"2023-08-09T14:51:46.786259+0000","last_became_peered":"2023-08-09T14:51:46.786259+0000","last_unstale":"2023-08-09T14:51:47.780773+0000","last_undegraded":"2023-08-09T14:51:47.780773+0000","last_fullsized":"2023-08-09T14:51:47.780773+0000","mapping_epoch":169,"log_start":"0'0","ondisk_log_start":"0'0","created":169,"last_epoch_clean":170,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T14:51:45.719739+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T14:51:45.719739+0000","last_clean_scrub_stamp":"2023-08-09T14:51:45.719739+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-11T00:34:30.127008+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"21.19","version":"0'0","reported_seq":9,"reported_epoch":171,"state":"active+clean","last_fresh":"2023-08-09T14:51:47.797688+0000","last_change":"2023-08-09T14:51:47.796944+0000","last_active":"2023-08-09T14:51:47.797688+0000","last_peered":"2023-08-09T14:51:47.797688+0000","last_clean":"2023-08-09T14:51:47.797688+0000","last_became_active":"2023-08-09T14:51:47.796656+0000","last_became_peered":"2023-08-09T14:51:47.796656+0000","last_unstale":"2023-08-09T14:51:47.797688+0000","last_undegraded":"2023-08-09T14:51:47.797688+0000","last_fullsized":"2023-08-09T14:51:47.797688+0000","mapping_epoch":170,"log_start":"0'0","ondisk_log_start":"0'0","created":170,"last_epoch_clean":171,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T14:51:46.735123+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T14:51:46.735123+0000","last_clean_scrub_stamp":"2023-08-09T14:51:46.735123+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T22:29:15.151030+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"20.18","version":"0'0","reported_seq":11,"reported_epoch":171,"state":"active+clean","last_fresh":"2023-08-09T14:51:47.764939+0000","last_change":"2023-08-09T14:51:46.791440+0000","last_active":"2023-08-09T14:51:47.764939+0000","last_peered":"2023-08-09T14:51:47.764939+0000","last_clean":"2023-08-09T14:51:47.764939+0000","last_became_active":"2023-08-09T14:51:46.791030+0000","last_became_peered":"2023-08-09T14:51:46.791030+0000","last_unstale":"2023-08-09T14:51:47.764939+0000","last_undegraded":"2023-08-09T14:51:47.764939+0000","last_fullsized":"2023-08-09T14:51:47.764939+0000","mapping_epoch":169,"log_start":"0'0","ondisk_log_start":"0'0","created":169,"last_epoch_clean":170,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T14:51:45.719739+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T14:51:45.719739+0000","last_clean_scrub_stamp":"2023-08-09T14:51:45.719739+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T15:41:47.070932+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"21.e","version":"0'0","reported_seq":9,"reported_epoch":171,"state":"active+clean","last_fresh":"2023-08-09T14:51:47.808907+0000","last_change":"2023-08-09T14:51:47.804113+0000","last_active":"2023-08-09T14:51:47.808907+0000","last_peered":"2023-08-09T14:51:47.808907+0000","last_clean":"2023-08-09T14:51:47.808907+0000","last_became_active":"2023-08-09T14:51:47.803657+0000","last_became_peered":"2023-08-09T14:51:47.803657+0000","last_unstale":"2023-08-09T14:51:47.808907+0000","last_undegraded":"2023-08-09T14:51:47.808907+0000","last_fullsized":"2023-08-09T14:51:47.808907+0000","mapping_epoch":170,"log_start":"0'0","ondisk_log_start":"0'0","created":170,"last_epoch_clean":171,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T14:51:46.735123+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T14:51:46.735123+0000","last_clean_scrub_stamp":"2023-08-09T14:51:46.735123+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-11T01:54:05.813801+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"20.f","version":"171'1","reported_seq":12,"reported_epoch":171,"state":"active+clean","last_fresh":"2023-08-09T14:51:47.812348+0000","last_change":"2023-08-09T14:51:46.787215+0000","last_active":"2023-08-09T14:51:47.812348+0000","last_peered":"2023-08-09T14:51:47.812348+0000","last_clean":"2023-08-09T14:51:47.812348+0000","last_became_active":"2023-08-09T14:51:46.786660+0000","last_became_peered":"2023-08-09T14:51:46.786660+0000","last_unstale":"2023-08-09T14:51:47.812348+0000","last_undegraded":"2023-08-09T14:51:47.812348+0000","last_fullsized":"2023-08-09T14:51:47.812348+0000","mapping_epoch":169,"log_start":"0'0","ondisk_log_start":"0'0","created":169,"last_epoch_clean":170,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T14:51:45.719739+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T14:51:45.719739+0000","last_clean_scrub_stamp":"2023-08-09T14:51:45.719739+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T22:44:23.546686+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"21.f","version":"0'0","reported_seq":9,"reported_epoch":171,"state":"active+clean","last_fresh":"2023-08-09T14:51:47.798508+0000","last_change":"2023-08-09T14:51:47.794357+0000","last_active":"2023-08-09T14:51:47.798508+0000","last_peered":"2023-08-09T14:51:47.798508+0000","last_clean":"2023-08-09T14:51:47.798508+0000","last_became_active":"2023-08-09T14:51:47.794078+0000","last_became_peered":"2023-08-09T14:51:47.794078+0000","last_unstale":"2023-08-09T14:51:47.798508+0000","last_undegraded":"2023-08-09T14:51:47.798508+0000","last_fullsized":"2023-08-09T14:51:47.798508+0000","mapping_epoch":170,"log_start":"0'0","ondisk_log_start":"0'0","created":170,"last_epoch_clean":171,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T14:51:46.735123+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T14:51:46.735123+0000","last_clean_scrub_stamp":"2023-08-09T14:51:46.735123+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T20:02:17.166890+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"20.e","version":"171'2","reported_seq":13,"reported_epoch":171,"state":"active+clean","last_fresh":"2023-08-09T14:51:47.814490+0000","last_change":"2023-08-09T14:51:46.784640+0000","last_active":"2023-08-09T14:51:47.814490+0000","last_peered":"2023-08-09T14:51:47.814490+0000","last_clean":"2023-08-09T14:51:47.814490+0000","last_became_active":"2023-08-09T14:51:46.784368+0000","last_became_peered":"2023-08-09T14:51:46.784368+0000","last_unstale":"2023-08-09T14:51:47.814490+0000","last_undegraded":"2023-08-09T14:51:47.814490+0000","last_fullsized":"2023-08-09T14:51:47.814490+0000","mapping_epoch":169,"log_start":"0'0","ondisk_log_start":"0'0","created":169,"last_epoch_clean":170,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T14:51:45.719739+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T14:51:45.719739+0000","last_clean_scrub_stamp":"2023-08-09T14:51:45.719739+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T21:48:55.522611+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"21.c","version":"0'0","reported_seq":9,"reported_epoch":171,"state":"active+clean","last_fresh":"2023-08-09T14:51:47.798115+0000","last_change":"2023-08-09T14:51:47.794010+0000","last_active":"2023-08-09T14:51:47.798115+0000","last_peered":"2023-08-09T14:51:47.798115+0000","last_clean":"2023-08-09T14:51:47.798115+0000","last_became_active":"2023-08-09T14:51:47.793683+0000","last_became_peered":"2023-08-09T14:51:47.793683+0000","last_unstale":"2023-08-09T14:51:47.798115+0000","last_undegraded":"2023-08-09T14:51:47.798115+0000","last_fullsized":"2023-08-09T14:51:47.798115+0000","mapping_epoch":170,"log_start":"0'0","ondisk_log_start":"0'0","created":170,"last_epoch_clean":171,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T14:51:46.735123+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T14:51:46.735123+0000","last_clean_scrub_stamp":"2023-08-09T14:51:46.735123+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T15:23:38.410151+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"20.d","version":"171'1","reported_seq":12,"reported_epoch":171,"state":"active+clean","last_fresh":"2023-08-09T14:51:47.812644+0000","last_change":"2023-08-09T14:51:46.787142+0000","last_active":"2023-08-09T14:51:47.812644+0000","last_peered":"2023-08-09T14:51:47.812644+0000","last_clean":"2023-08-09T14:51:47.812644+0000","last_became_active":"2023-08-09T14:51:46.786118+0000","last_became_peered":"2023-08-09T14:51:46.786118+0000","last_unstale":"2023-08-09T14:51:47.812644+0000","last_undegraded":"2023-08-09T14:51:47.812644+0000","last_fullsized":"2023-08-09T14:51:47.812644+0000","mapping_epoch":169,"log_start":"0'0","ondisk_log_start":"0'0","created":169,"last_epoch_clean":170,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T14:51:45.719739+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T14:51:45.719739+0000","last_clean_scrub_stamp":"2023-08-09T14:51:45.719739+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T17:32:00.642586+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"21.d","version":"0'0","reported_seq":9,"reported_epoch":171,"state":"active+clean","last_fresh":"2023-08-09T14:51:47.805528+0000","last_change":"2023-08-09T14:51:47.798659+0000","last_active":"2023-08-09T14:51:47.805528+0000","last_peered":"2023-08-09T14:51:47.805528+0000","last_clean":"2023-08-09T14:51:47.805528+0000","last_became_active":"2023-08-09T14:51:47.798125+0000","last_became_peered":"2023-08-09T14:51:47.798125+0000","last_unstale":"2023-08-09T14:51:47.805528+0000","last_undegraded":"2023-08-09T14:51:47.805528+0000","last_fullsized":"2023-08-09T14:51:47.805528+0000","mapping_epoch":170,"log_start":"0'0","ondisk_log_start":"0'0","created":170,"last_epoch_clean":171,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T14:51:46.735123+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T14:51:46.735123+0000","last_clean_scrub_stamp":"2023-08-09T14:51:46.735123+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-11T01:33:44.144950+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"20.c","version":"0'0","reported_seq":11,"reported_epoch":171,"state":"active+clean","last_fresh":"2023-08-09T14:51:47.771635+0000","last_change":"2023-08-09T14:51:46.782861+0000","last_active":"2023-08-09T14:51:47.771635+0000","last_peered":"2023-08-09T14:51:47.771635+0000","last_clean":"2023-08-09T14:51:47.771635+0000","last_became_active":"2023-08-09T14:51:46.782554+0000","last_became_peered":"2023-08-09T14:51:46.782554+0000","last_unstale":"2023-08-09T14:51:47.771635+0000","last_undegraded":"2023-08-09T14:51:47.771635+0000","last_fullsized":"2023-08-09T14:51:47.771635+0000","mapping_epoch":169,"log_start":"0'0","ondisk_log_start":"0'0","created":169,"last_epoch_clean":170,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T14:51:45.719739+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T14:51:45.719739+0000","last_clean_scrub_stamp":"2023-08-09T14:51:45.719739+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-11T00:37:55.636435+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"21.a","version":"0'0","reported_seq":9,"reported_epoch":171,"state":"active+clean","last_fresh":"2023-08-09T14:51:47.789542+0000","last_change":"2023-08-09T14:51:47.781446+0000","last_active":"2023-08-09T14:51:47.789542+0000","last_peered":"2023-08-09T14:51:47.789542+0000","last_clean":"2023-08-09T14:51:47.789542+0000","last_became_active":"2023-08-09T14:51:47.780558+0000","last_became_peered":"2023-08-09T14:51:47.780558+0000","last_unstale":"2023-08-09T14:51:47.789542+0000","last_undegraded":"2023-08-09T14:51:47.789542+0000","last_fullsized":"2023-08-09T14:51:47.789542+0000","mapping_epoch":170,"log_start":"0'0","ondisk_log_start":"0'0","created":170,"last_epoch_clean":171,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T14:51:46.735123+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T14:51:46.735123+0000","last_clean_scrub_stamp":"2023-08-09T14:51:46.735123+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T22:35:53.052714+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"20.b","version":"171'2","reported_seq":13,"reported_epoch":171,"state":"active+clean","last_fresh":"2023-08-09T14:51:47.811852+0000","last_change":"2023-08-09T14:51:46.784391+0000","last_active":"2023-08-09T14:51:47.811852+0000","last_peered":"2023-08-09T14:51:47.811852+0000","last_clean":"2023-08-09T14:51:47.811852+0000","last_became_active":"2023-08-09T14:51:46.784094+0000","last_became_peered":"2023-08-09T14:51:46.784094+0000","last_unstale":"2023-08-09T14:51:47.811852+0000","last_undegraded":"2023-08-09T14:51:47.811852+0000","last_fullsized":"2023-08-09T14:51:47.811852+0000","mapping_epoch":169,"log_start":"0'0","ondisk_log_start":"0'0","created":169,"last_epoch_clean":170,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T14:51:45.719739+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T14:51:45.719739+0000","last_clean_scrub_stamp":"2023-08-09T14:51:45.719739+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T22:35:52.037330+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"21.b","version":"0'0","reported_seq":9,"reported_epoch":171,"state":"active+clean","last_fresh":"2023-08-09T14:51:47.802180+0000","last_change":"2023-08-09T14:51:47.796879+0000","last_active":"2023-08-09T14:51:47.802180+0000","last_peered":"2023-08-09T14:51:47.802180+0000","last_clean":"2023-08-09T14:51:47.802180+0000","last_became_active":"2023-08-09T14:51:47.796468+0000","last_became_peered":"2023-08-09T14:51:47.796468+0000","last_unstale":"2023-08-09T14:51:47.802180+0000","last_undegraded":"2023-08-09T14:51:47.802180+0000","last_fullsized":"2023-08-09T14:51:47.802180+0000","mapping_epoch":170,"log_start":"0'0","ondisk_log_start":"0'0","created":170,"last_epoch_clean":171,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T14:51:46.735123+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T14:51:46.735123+0000","last_clean_scrub_stamp":"2023-08-09T14:51:46.735123+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-11T01:13:33.380484+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"20.a","version":"0'0","reported_seq":11,"reported_epoch":171,"state":"active+clean","last_fresh":"2023-08-09T14:51:47.785757+0000","last_change":"2023-08-09T14:51:46.784518+0000","last_active":"2023-08-09T14:51:47.785757+0000","last_peered":"2023-08-09T14:51:47.785757+0000","last_clean":"2023-08-09T14:51:47.785757+0000","last_became_active":"2023-08-09T14:51:46.783848+0000","last_became_peered":"2023-08-09T14:51:46.783848+0000","last_unstale":"2023-08-09T14:51:47.785757+0000","last_undegraded":"2023-08-09T14:51:47.785757+0000","last_fullsized":"2023-08-09T14:51:47.785757+0000","mapping_epoch":169,"log_start":"0'0","ondisk_log_start":"0'0","created":169,"last_epoch_clean":170,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T14:51:45.719739+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T14:51:45.719739+0000","last_clean_scrub_stamp":"2023-08-09T14:51:45.719739+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T17:58:22.192246+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"21.8","version":"0'0","reported_seq":9,"reported_epoch":171,"state":"active+clean","last_fresh":"2023-08-09T14:51:47.791388+0000","last_change":"2023-08-09T14:51:47.787488+0000","last_active":"2023-08-09T14:51:47.791388+0000","last_peered":"2023-08-09T14:51:47.791388+0000","last_clean":"2023-08-09T14:51:47.791388+0000","last_became_active":"2023-08-09T14:51:47.786938+0000","last_became_peered":"2023-08-09T14:51:47.786938+0000","last_unstale":"2023-08-09T14:51:47.791388+0000","last_undegraded":"2023-08-09T14:51:47.791388+0000","last_fullsized":"2023-08-09T14:51:47.791388+0000","mapping_epoch":170,"log_start":"0'0","ondisk_log_start":"0'0","created":170,"last_epoch_clean":171,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T14:51:46.735123+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T14:51:46.735123+0000","last_clean_scrub_stamp":"2023-08-09T14:51:46.735123+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-11T02:18:22.001820+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"20.9","version":"0'0","reported_seq":11,"reported_epoch":171,"state":"active+clean","last_fresh":"2023-08-09T14:51:47.788252+0000","last_change":"2023-08-09T14:51:46.789049+0000","last_active":"2023-08-09T14:51:47.788252+0000","last_peered":"2023-08-09T14:51:47.788252+0000","last_clean":"2023-08-09T14:51:47.788252+0000","last_became_active":"2023-08-09T14:51:46.788778+0000","last_became_peered":"2023-08-09T14:51:46.788778+0000","last_unstale":"2023-08-09T14:51:47.788252+0000","last_undegraded":"2023-08-09T14:51:47.788252+0000","last_fullsized":"2023-08-09T14:51:47.788252+0000","mapping_epoch":169,"log_start":"0'0","ondisk_log_start":"0'0","created":169,"last_epoch_clean":170,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T14:51:45.719739+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T14:51:45.719739+0000","last_clean_scrub_stamp":"2023-08-09T14:51:45.719739+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T21:29:02.270845+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"21.9","version":"0'0","reported_seq":9,"reported_epoch":171,"state":"active+clean","last_fresh":"2023-08-09T14:51:47.802145+0000","last_change":"2023-08-09T14:51:47.795891+0000","last_active":"2023-08-09T14:51:47.802145+0000","last_peered":"2023-08-09T14:51:47.802145+0000","last_clean":"2023-08-09T14:51:47.802145+0000","last_became_active":"2023-08-09T14:51:47.795419+0000","last_became_peered":"2023-08-09T14:51:47.795419+0000","last_unstale":"2023-08-09T14:51:47.802145+0000","last_undegraded":"2023-08-09T14:51:47.802145+0000","last_fullsized":"2023-08-09T14:51:47.802145+0000","mapping_epoch":170,"log_start":"0'0","ondisk_log_start":"0'0","created":170,"last_epoch_clean":171,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T14:51:46.735123+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T14:51:46.735123+0000","last_clean_scrub_stamp":"2023-08-09T14:51:46.735123+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T15:33:27.498627+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"20.8","version":"0'0","reported_seq":11,"reported_epoch":171,"state":"active+clean","last_fresh":"2023-08-09T14:51:47.775212+0000","last_change":"2023-08-09T14:51:46.781845+0000","last_active":"2023-08-09T14:51:47.775212+0000","last_peered":"2023-08-09T14:51:47.775212+0000","last_clean":"2023-08-09T14:51:47.775212+0000","last_became_active":"2023-08-09T14:51:46.781489+0000","last_became_peered":"2023-08-09T14:51:46.781489+0000","last_unstale":"2023-08-09T14:51:47.775212+0000","last_undegraded":"2023-08-09T14:51:47.775212+0000","last_fullsized":"2023-08-09T14:51:47.775212+0000","mapping_epoch":169,"log_start":"0'0","ondisk_log_start":"0'0","created":169,"last_epoch_clean":170,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T14:51:45.719739+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T14:51:45.719739+0000","last_clean_scrub_stamp":"2023-08-09T14:51:45.719739+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-11T01:31:08.610990+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"21.6","version":"0'0","reported_seq":9,"reported_epoch":171,"state":"active+clean","last_fresh":"2023-08-09T14:51:47.799323+0000","last_change":"2023-08-09T14:51:47.797548+0000","last_active":"2023-08-09T14:51:47.799323+0000","last_peered":"2023-08-09T14:51:47.799323+0000","last_clean":"2023-08-09T14:51:47.799323+0000","last_became_active":"2023-08-09T14:51:47.797272+0000","last_became_peered":"2023-08-09T14:51:47.797272+0000","last_unstale":"2023-08-09T14:51:47.799323+0000","last_undegraded":"2023-08-09T14:51:47.799323+0000","last_fullsized":"2023-08-09T14:51:47.799323+0000","mapping_epoch":170,"log_start":"0'0","ondisk_log_start":"0'0","created":170,"last_epoch_clean":171,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T14:51:46.735123+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T14:51:46.735123+0000","last_clean_scrub_stamp":"2023-08-09T14:51:46.735123+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T22:51:09.744725+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"20.7","version":"171'2","reported_seq":13,"reported_epoch":171,"state":"active+clean","last_fresh":"2023-08-09T14:51:47.812435+0000","last_change":"2023-08-09T14:51:46.785516+0000","last_active":"2023-08-09T14:51:47.812435+0000","last_peered":"2023-08-09T14:51:47.812435+0000","last_clean":"2023-08-09T14:51:47.812435+0000","last_became_active":"2023-08-09T14:51:46.785234+0000","last_became_peered":"2023-08-09T14:51:46.785234+0000","last_unstale":"2023-08-09T14:51:47.812435+0000","last_undegraded":"2023-08-09T14:51:47.812435+0000","last_fullsized":"2023-08-09T14:51:47.812435+0000","mapping_epoch":169,"log_start":"0'0","ondisk_log_start":"0'0","created":169,"last_epoch_clean":170,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T14:51:45.719739+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T14:51:45.719739+0000","last_clean_scrub_stamp":"2023-08-09T14:51:45.719739+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T18:11:10.873473+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"21.7","version":"0'0","reported_seq":9,"reported_epoch":171,"state":"active+clean","last_fresh":"2023-08-09T14:51:47.798153+0000","last_change":"2023-08-09T14:51:47.794010+0000","last_active":"2023-08-09T14:51:47.798153+0000","last_peered":"2023-08-09T14:51:47.798153+0000","last_clean":"2023-08-09T14:51:47.798153+0000","last_became_active":"2023-08-09T14:51:47.793661+0000","last_became_peered":"2023-08-09T14:51:47.793661+0000","last_unstale":"2023-08-09T14:51:47.798153+0000","last_undegraded":"2023-08-09T14:51:47.798153+0000","last_fullsized":"2023-08-09T14:51:47.798153+0000","mapping_epoch":170,"log_start":"0'0","ondisk_log_start":"0'0","created":170,"last_epoch_clean":171,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T14:51:46.735123+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T14:51:46.735123+0000","last_clean_scrub_stamp":"2023-08-09T14:51:46.735123+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T19:10:19.642841+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"20.6","version":"0'0","reported_seq":11,"reported_epoch":171,"state":"active+clean","last_fresh":"2023-08-09T14:51:47.764656+0000","last_change":"2023-08-09T14:51:46.785316+0000","last_active":"2023-08-09T14:51:47.764656+0000","last_peered":"2023-08-09T14:51:47.764656+0000","last_clean":"2023-08-09T14:51:47.764656+0000","last_became_active":"2023-08-09T14:51:46.785034+0000","last_became_peered":"2023-08-09T14:51:46.785034+0000","last_unstale":"2023-08-09T14:51:47.764656+0000","last_undegraded":"2023-08-09T14:51:47.764656+0000","last_fullsized":"2023-08-09T14:51:47.764656+0000","mapping_epoch":169,"log_start":"0'0","ondisk_log_start":"0'0","created":169,"last_epoch_clean":170,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T14:51:45.719739+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T14:51:45.719739+0000","last_clean_scrub_stamp":"2023-08-09T14:51:45.719739+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T18:41:24.093109+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"21.4","version":"0'0","reported_seq":9,"reported_epoch":171,"state":"active+clean","last_fresh":"2023-08-09T14:51:47.792165+0000","last_change":"2023-08-09T14:51:47.787985+0000","last_active":"2023-08-09T14:51:47.792165+0000","last_peered":"2023-08-09T14:51:47.792165+0000","last_clean":"2023-08-09T14:51:47.792165+0000","last_became_active":"2023-08-09T14:51:47.787565+0000","last_became_peered":"2023-08-09T14:51:47.787565+0000","last_unstale":"2023-08-09T14:51:47.792165+0000","last_undegraded":"2023-08-09T14:51:47.792165+0000","last_fullsized":"2023-08-09T14:51:47.792165+0000","mapping_epoch":170,"log_start":"0'0","ondisk_log_start":"0'0","created":170,"last_epoch_clean":171,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T14:51:46.735123+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T14:51:46.735123+0000","last_clean_scrub_stamp":"2023-08-09T14:51:46.735123+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T20:55:40.272873+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"20.5","version":"0'0","reported_seq":11,"reported_epoch":171,"state":"active+clean","last_fresh":"2023-08-09T14:51:47.766489+0000","last_change":"2023-08-09T14:51:46.772657+0000","last_active":"2023-08-09T14:51:47.766489+0000","last_peered":"2023-08-09T14:51:47.766489+0000","last_clean":"2023-08-09T14:51:47.766489+0000","last_became_active":"2023-08-09T14:51:46.772339+0000","last_became_peered":"2023-08-09T14:51:46.772339+0000","last_unstale":"2023-08-09T14:51:47.766489+0000","last_undegraded":"2023-08-09T14:51:47.766489+0000","last_fullsized":"2023-08-09T14:51:47.766489+0000","mapping_epoch":169,"log_start":"0'0","ondisk_log_start":"0'0","created":169,"last_epoch_clean":170,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T14:51:45.719739+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T14:51:45.719739+0000","last_clean_scrub_stamp":"2023-08-09T14:51:45.719739+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T15:51:44.315128+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"21.5","version":"0'0","reported_seq":9,"reported_epoch":171,"state":"active+clean","last_fresh":"2023-08-09T14:51:47.791633+0000","last_change":"2023-08-09T14:51:47.787962+0000","last_active":"2023-08-09T14:51:47.791633+0000","last_peered":"2023-08-09T14:51:47.791633+0000","last_clean":"2023-08-09T14:51:47.791633+0000","last_became_active":"2023-08-09T14:51:47.787595+0000","last_became_peered":"2023-08-09T14:51:47.787595+0000","last_unstale":"2023-08-09T14:51:47.791633+0000","last_undegraded":"2023-08-09T14:51:47.791633+0000","last_fullsized":"2023-08-09T14:51:47.791633+0000","mapping_epoch":170,"log_start":"0'0","ondisk_log_start":"0'0","created":170,"last_epoch_clean":171,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T14:51:46.735123+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T14:51:46.735123+0000","last_clean_scrub_stamp":"2023-08-09T14:51:46.735123+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T16:37:00.297341+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"20.4","version":"0'0","reported_seq":11,"reported_epoch":171,"state":"active+clean","last_fresh":"2023-08-09T14:51:47.763337+0000","last_change":"2023-08-09T14:51:46.772250+0000","last_active":"2023-08-09T14:51:47.763337+0000","last_peered":"2023-08-09T14:51:47.763337+0000","last_clean":"2023-08-09T14:51:47.763337+0000","last_became_active":"2023-08-09T14:51:46.771906+0000","last_became_peered":"2023-08-09T14:51:46.771906+0000","last_unstale":"2023-08-09T14:51:47.763337+0000","last_undegraded":"2023-08-09T14:51:47.763337+0000","last_fullsized":"2023-08-09T14:51:47.763337+0000","mapping_epoch":169,"log_start":"0'0","ondisk_log_start":"0'0","created":169,"last_epoch_clean":170,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T14:51:45.719739+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T14:51:45.719739+0000","last_clean_scrub_stamp":"2023-08-09T14:51:45.719739+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T15:27:12.471209+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"21.2","version":"0'0","reported_seq":9,"reported_epoch":171,"state":"active+clean","last_fresh":"2023-08-09T14:51:47.798567+0000","last_change":"2023-08-09T14:51:47.795963+0000","last_active":"2023-08-09T14:51:47.798567+0000","last_peered":"2023-08-09T14:51:47.798567+0000","last_clean":"2023-08-09T14:51:47.798567+0000","last_became_active":"2023-08-09T14:51:47.795647+0000","last_became_peered":"2023-08-09T14:51:47.795647+0000","last_unstale":"2023-08-09T14:51:47.798567+0000","last_undegraded":"2023-08-09T14:51:47.798567+0000","last_fullsized":"2023-08-09T14:51:47.798567+0000","mapping_epoch":170,"log_start":"0'0","ondisk_log_start":"0'0","created":170,"last_epoch_clean":171,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T14:51:46.735123+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T14:51:46.735123+0000","last_clean_scrub_stamp":"2023-08-09T14:51:46.735123+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T18:22:58.458491+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"20.3","version":"0'0","reported_seq":11,"reported_epoch":171,"state":"active+clean","last_fresh":"2023-08-09T14:51:47.778873+0000","last_change":"2023-08-09T14:51:46.784221+0000","last_active":"2023-08-09T14:51:47.778873+0000","last_peered":"2023-08-09T14:51:47.778873+0000","last_clean":"2023-08-09T14:51:47.778873+0000","last_became_active":"2023-08-09T14:51:46.783914+0000","last_became_peered":"2023-08-09T14:51:46.783914+0000","last_unstale":"2023-08-09T14:51:47.778873+0000","last_undegraded":"2023-08-09T14:51:47.778873+0000","last_fullsized":"2023-08-09T14:51:47.778873+0000","mapping_epoch":169,"log_start":"0'0","ondisk_log_start":"0'0","created":169,"last_epoch_clean":170,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T14:51:45.719739+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T14:51:45.719739+0000","last_clean_scrub_stamp":"2023-08-09T14:51:45.719739+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-11T00:26:43.632475+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"21.3","version":"0'0","reported_seq":9,"reported_epoch":171,"state":"active+clean","last_fresh":"2023-08-09T14:51:47.800072+0000","last_change":"2023-08-09T14:51:47.799128+0000","last_active":"2023-08-09T14:51:47.800072+0000","last_peered":"2023-08-09T14:51:47.800072+0000","last_clean":"2023-08-09T14:51:47.800072+0000","last_became_active":"2023-08-09T14:51:47.798771+0000","last_became_peered":"2023-08-09T14:51:47.798771+0000","last_unstale":"2023-08-09T14:51:47.800072+0000","last_undegraded":"2023-08-09T14:51:47.800072+0000","last_fullsized":"2023-08-09T14:51:47.800072+0000","mapping_epoch":170,"log_start":"0'0","ondisk_log_start":"0'0","created":170,"last_epoch_clean":171,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T14:51:46.735123+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T14:51:46.735123+0000","last_clean_scrub_stamp":"2023-08-09T14:51:46.735123+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T17:30:38.007650+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"20.2","version":"0'0","reported_seq":11,"reported_epoch":171,"state":"active+clean","last_fresh":"2023-08-09T14:51:47.785960+0000","last_change":"2023-08-09T14:51:46.784750+0000","last_active":"2023-08-09T14:51:47.785960+0000","last_peered":"2023-08-09T14:51:47.785960+0000","last_clean":"2023-08-09T14:51:47.785960+0000","last_became_active":"2023-08-09T14:51:46.784230+0000","last_became_peered":"2023-08-09T14:51:46.784230+0000","last_unstale":"2023-08-09T14:51:47.785960+0000","last_undegraded":"2023-08-09T14:51:47.785960+0000","last_fullsized":"2023-08-09T14:51:47.785960+0000","mapping_epoch":169,"log_start":"0'0","ondisk_log_start":"0'0","created":169,"last_epoch_clean":170,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T14:51:45.719739+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T14:51:45.719739+0000","last_clean_scrub_stamp":"2023-08-09T14:51:45.719739+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T21:49:35.633438+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"21.0","version":"0'0","reported_seq":9,"reported_epoch":171,"state":"active+clean","last_fresh":"2023-08-09T14:51:47.805333+0000","last_change":"2023-08-09T14:51:47.796432+0000","last_active":"2023-08-09T14:51:47.805333+0000","last_peered":"2023-08-09T14:51:47.805333+0000","last_clean":"2023-08-09T14:51:47.805333+0000","last_became_active":"2023-08-09T14:51:47.796129+0000","last_became_peered":"2023-08-09T14:51:47.796129+0000","last_unstale":"2023-08-09T14:51:47.805333+0000","last_undegraded":"2023-08-09T14:51:47.805333+0000","last_fullsized":"2023-08-09T14:51:47.805333+0000","mapping_epoch":170,"log_start":"0'0","ondisk_log_start":"0'0","created":170,"last_epoch_clean":171,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T14:51:46.735123+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T14:51:46.735123+0000","last_clean_scrub_stamp":"2023-08-09T14:51:46.735123+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T21:12:37.780571+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"20.1","version":"0'0","reported_seq":11,"reported_epoch":171,"state":"active+clean","last_fresh":"2023-08-09T14:51:47.776812+0000","last_change":"2023-08-09T14:51:46.783800+0000","last_active":"2023-08-09T14:51:47.776812+0000","last_peered":"2023-08-09T14:51:47.776812+0000","last_clean":"2023-08-09T14:51:47.776812+0000","last_became_active":"2023-08-09T14:51:46.783530+0000","last_became_peered":"2023-08-09T14:51:46.783530+0000","last_unstale":"2023-08-09T14:51:47.776812+0000","last_undegraded":"2023-08-09T14:51:47.776812+0000","last_fullsized":"2023-08-09T14:51:47.776812+0000","mapping_epoch":169,"log_start":"0'0","ondisk_log_start":"0'0","created":169,"last_epoch_clean":170,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T14:51:45.719739+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T14:51:45.719739+0000","last_clean_scrub_stamp":"2023-08-09T14:51:45.719739+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T21:43:59.706915+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"21.1","version":"0'0","reported_seq":9,"reported_epoch":171,"state":"active+clean","last_fresh":"2023-08-09T14:51:47.788039+0000","last_change":"2023-08-09T14:51:47.786151+0000","last_active":"2023-08-09T14:51:47.788039+0000","last_peered":"2023-08-09T14:51:47.788039+0000","last_clean":"2023-08-09T14:51:47.788039+0000","last_became_active":"2023-08-09T14:51:47.785803+0000","last_became_peered":"2023-08-09T14:51:47.785803+0000","last_unstale":"2023-08-09T14:51:47.788039+0000","last_undegraded":"2023-08-09T14:51:47.788039+0000","last_fullsized":"2023-08-09T14:51:47.788039+0000","mapping_epoch":170,"log_start":"0'0","ondisk_log_start":"0'0","created":170,"last_epoch_clean":171,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T14:51:46.735123+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T14:51:46.735123+0000","last_clean_scrub_stamp":"2023-08-09T14:51:46.735123+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T20:40:40.159710+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"20.0","version":"0'0","reported_seq":11,"reported_epoch":171,"state":"active+clean","last_fresh":"2023-08-09T14:51:47.770122+0000","last_change":"2023-08-09T14:51:46.791268+0000","last_active":"2023-08-09T14:51:47.770122+0000","last_peered":"2023-08-09T14:51:47.770122+0000","last_clean":"2023-08-09T14:51:47.770122+0000","last_became_active":"2023-08-09T14:51:46.790960+0000","last_became_peered":"2023-08-09T14:51:46.790960+0000","last_unstale":"2023-08-09T14:51:47.770122+0000","last_undegraded":"2023-08-09T14:51:47.770122+0000","last_fullsized":"2023-08-09T14:51:47.770122+0000","mapping_epoch":169,"log_start":"0'0","ondisk_log_start":"0'0","created":169,"last_epoch_clean":170,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T14:51:45.719739+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T14:51:45.719739+0000","last_clean_scrub_stamp":"2023-08-09T14:51:45.719739+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T17:43:27.675754+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"21.14","version":"0'0","reported_seq":9,"reported_epoch":171,"state":"active+clean","last_fresh":"2023-08-09T14:51:47.798347+0000","last_change":"2023-08-09T14:51:47.791201+0000","last_active":"2023-08-09T14:51:47.798347+0000","last_peered":"2023-08-09T14:51:47.798347+0000","last_clean":"2023-08-09T14:51:47.798347+0000","last_became_active":"2023-08-09T14:51:47.790897+0000","last_became_peered":"2023-08-09T14:51:47.790897+0000","last_unstale":"2023-08-09T14:51:47.798347+0000","last_undegraded":"2023-08-09T14:51:47.798347+0000","last_fullsized":"2023-08-09T14:51:47.798347+0000","mapping_epoch":170,"log_start":"0'0","ondisk_log_start":"0'0","created":170,"last_epoch_clean":171,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T14:51:46.735123+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T14:51:46.735123+0000","last_clean_scrub_stamp":"2023-08-09T14:51:46.735123+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T20:42:16.832652+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"20.15","version":"171'3","reported_seq":14,"reported_epoch":171,"state":"active+clean","last_fresh":"2023-08-09T14:51:47.814593+0000","last_change":"2023-08-09T14:51:46.782009+0000","last_active":"2023-08-09T14:51:47.814593+0000","last_peered":"2023-08-09T14:51:47.814593+0000","last_clean":"2023-08-09T14:51:47.814593+0000","last_became_active":"2023-08-09T14:51:46.781722+0000","last_became_peered":"2023-08-09T14:51:46.781722+0000","last_unstale":"2023-08-09T14:51:47.814593+0000","last_undegraded":"2023-08-09T14:51:47.814593+0000","last_fullsized":"2023-08-09T14:51:47.814593+0000","mapping_epoch":169,"log_start":"0'0","ondisk_log_start":"0'0","created":169,"last_epoch_clean":170,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T14:51:45.719739+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T14:51:45.719739+0000","last_clean_scrub_stamp":"2023-08-09T14:51:45.719739+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T16:54:04.005809+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":319,"reported_epoch":171,"state":"active+clean","last_fresh":"2023-08-09T14:51:47.786219+0000","last_change":"2023-08-09T13:33:56.401406+0000","last_active":"2023-08-09T14:51:47.786219+0000","last_peered":"2023-08-09T14:51:47.786219+0000","last_clean":"2023-08-09T14:51:47.786219+0000","last_became_active":"2023-08-09T13:33:56.400487+0000","last_became_peered":"2023-08-09T13:33:56.400487+0000","last_unstale":"2023-08-09T14:51:47.786219+0000","last_undegraded":"2023-08-09T14:51:47.786219+0000","last_fullsized":"2023-08-09T14:51:47.786219+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:33:54.845869+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:33:54.845869+0000","last_clean_scrub_stamp":"2023-08-09T13:33:54.845869+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T22:57:53.864171+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"21.15","version":"0'0","reported_seq":9,"reported_epoch":171,"state":"active+clean","last_fresh":"2023-08-09T14:51:47.795847+0000","last_change":"2023-08-09T14:51:47.788086+0000","last_active":"2023-08-09T14:51:47.795847+0000","last_peered":"2023-08-09T14:51:47.795847+0000","last_clean":"2023-08-09T14:51:47.795847+0000","last_became_active":"2023-08-09T14:51:47.787696+0000","last_became_peered":"2023-08-09T14:51:47.787696+0000","last_unstale":"2023-08-09T14:51:47.795847+0000","last_undegraded":"2023-08-09T14:51:47.795847+0000","last_fullsized":"2023-08-09T14:51:47.795847+0000","mapping_epoch":170,"log_start":"0'0","ondisk_log_start":"0'0","created":170,"last_epoch_clean":171,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T14:51:46.735123+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T14:51:46.735123+0000","last_clean_scrub_stamp":"2023-08-09T14:51:46.735123+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-11T02:37:18.590003+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"20.14","version":"171'6","reported_seq":17,"reported_epoch":171,"state":"active+clean","last_fresh":"2023-08-09T14:51:47.817171+0000","last_change":"2023-08-09T14:51:46.772775+0000","last_active":"2023-08-09T14:51:47.817171+0000","last_peered":"2023-08-09T14:51:47.817171+0000","last_clean":"2023-08-09T14:51:47.817171+0000","last_became_active":"2023-08-09T14:51:46.772463+0000","last_became_peered":"2023-08-09T14:51:46.772463+0000","last_unstale":"2023-08-09T14:51:47.817171+0000","last_undegraded":"2023-08-09T14:51:47.817171+0000","last_fullsized":"2023-08-09T14:51:47.817171+0000","mapping_epoch":169,"log_start":"0'0","ondisk_log_start":"0'0","created":169,"last_epoch_clean":170,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T14:51:45.719739+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T14:51:45.719739+0000","last_clean_scrub_stamp":"2023-08-09T14:51:45.719739+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-11T02:18:20.986435+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":326,"reported_epoch":171,"state":"active+clean","last_fresh":"2023-08-09T14:51:47.772371+0000","last_change":"2023-08-09T13:33:56.401123+0000","last_active":"2023-08-09T14:51:47.772371+0000","last_peered":"2023-08-09T14:51:47.772371+0000","last_clean":"2023-08-09T14:51:47.772371+0000","last_became_active":"2023-08-09T13:33:56.399933+0000","last_became_peered":"2023-08-09T13:33:56.399933+0000","last_unstale":"2023-08-09T14:51:47.772371+0000","last_undegraded":"2023-08-09T14:51:47.772371+0000","last_fullsized":"2023-08-09T14:51:47.772371+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:33:54.845869+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:33:54.845869+0000","last_clean_scrub_stamp":"2023-08-09T13:33:54.845869+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-11T01:21:08.313655+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"21.16","version":"0'0","reported_seq":9,"reported_epoch":171,"state":"active+clean","last_fresh":"2023-08-09T14:51:47.797665+0000","last_change":"2023-08-09T14:51:47.792022+0000","last_active":"2023-08-09T14:51:47.797665+0000","last_peered":"2023-08-09T14:51:47.797665+0000","last_clean":"2023-08-09T14:51:47.797665+0000","last_became_active":"2023-08-09T14:51:47.788230+0000","last_became_peered":"2023-08-09T14:51:47.788230+0000","last_unstale":"2023-08-09T14:51:47.797665+0000","last_undegraded":"2023-08-09T14:51:47.797665+0000","last_fullsized":"2023-08-09T14:51:47.797665+0000","mapping_epoch":170,"log_start":"0'0","ondisk_log_start":"0'0","created":170,"last_epoch_clean":171,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T14:51:46.735123+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T14:51:46.735123+0000","last_clean_scrub_stamp":"2023-08-09T14:51:46.735123+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T22:10:25.748586+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"20.17","version":"0'0","reported_seq":11,"reported_epoch":171,"state":"active+clean","last_fresh":"2023-08-09T14:51:47.772166+0000","last_change":"2023-08-09T14:51:46.780599+0000","last_active":"2023-08-09T14:51:47.772166+0000","last_peered":"2023-08-09T14:51:47.772166+0000","last_clean":"2023-08-09T14:51:47.772166+0000","last_became_active":"2023-08-09T14:51:46.780275+0000","last_became_peered":"2023-08-09T14:51:46.780275+0000","last_unstale":"2023-08-09T14:51:47.772166+0000","last_undegraded":"2023-08-09T14:51:47.772166+0000","last_fullsized":"2023-08-09T14:51:47.772166+0000","mapping_epoch":169,"log_start":"0'0","ondisk_log_start":"0'0","created":169,"last_epoch_clean":170,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T14:51:45.719739+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T14:51:45.719739+0000","last_clean_scrub_stamp":"2023-08-09T14:51:45.719739+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T18:46:48.463911+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.2","version":"0'0","reported_seq":318,"reported_epoch":171,"state":"active+clean","last_fresh":"2023-08-09T14:51:47.786081+0000","last_change":"2023-08-09T13:33:56.402794+0000","last_active":"2023-08-09T14:51:47.786081+0000","last_peered":"2023-08-09T14:51:47.786081+0000","last_clean":"2023-08-09T14:51:47.786081+0000","last_became_active":"2023-08-09T13:33:56.402038+0000","last_became_peered":"2023-08-09T13:33:56.402038+0000","last_unstale":"2023-08-09T14:51:47.786081+0000","last_undegraded":"2023-08-09T14:51:47.786081+0000","last_fullsized":"2023-08-09T14:51:47.786081+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:33:54.845869+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:33:54.845869+0000","last_clean_scrub_stamp":"2023-08-09T13:33:54.845869+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T22:32:04.366882+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"21.17","version":"0'0","reported_seq":9,"reported_epoch":171,"state":"active+clean","last_fresh":"2023-08-09T14:51:47.805471+0000","last_change":"2023-08-09T14:51:47.798518+0000","last_active":"2023-08-09T14:51:47.805471+0000","last_peered":"2023-08-09T14:51:47.805471+0000","last_clean":"2023-08-09T14:51:47.805471+0000","last_became_active":"2023-08-09T14:51:47.797907+0000","last_became_peered":"2023-08-09T14:51:47.797907+0000","last_unstale":"2023-08-09T14:51:47.805471+0000","last_undegraded":"2023-08-09T14:51:47.805471+0000","last_fullsized":"2023-08-09T14:51:47.805471+0000","mapping_epoch":170,"log_start":"0'0","ondisk_log_start":"0'0","created":170,"last_epoch_clean":171,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T14:51:46.735123+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T14:51:46.735123+0000","last_clean_scrub_stamp":"2023-08-09T14:51:46.735123+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T17:57:19.553018+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"20.16","version":"0'0","reported_seq":11,"reported_epoch":171,"state":"active+clean","last_fresh":"2023-08-09T14:51:47.788083+0000","last_change":"2023-08-09T14:51:46.790916+0000","last_active":"2023-08-09T14:51:47.788083+0000","last_peered":"2023-08-09T14:51:47.788083+0000","last_clean":"2023-08-09T14:51:47.788083+0000","last_became_active":"2023-08-09T14:51:46.790560+0000","last_became_peered":"2023-08-09T14:51:46.790560+0000","last_unstale":"2023-08-09T14:51:47.788083+0000","last_undegraded":"2023-08-09T14:51:47.788083+0000","last_fullsized":"2023-08-09T14:51:47.788083+0000","mapping_epoch":169,"log_start":"0'0","ondisk_log_start":"0'0","created":169,"last_epoch_clean":170,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T14:51:45.719739+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T14:51:45.719739+0000","last_clean_scrub_stamp":"2023-08-09T14:51:45.719739+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T21:59:29.373187+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.3","version":"0'0","reported_seq":326,"reported_epoch":171,"state":"active+clean","last_fresh":"2023-08-09T14:51:47.763518+0000","last_change":"2023-08-09T13:33:55.886749+0000","last_active":"2023-08-09T14:51:47.763518+0000","last_peered":"2023-08-09T14:51:47.763518+0000","last_clean":"2023-08-09T14:51:47.763518+0000","last_became_active":"2023-08-09T13:33:55.886116+0000","last_became_peered":"2023-08-09T13:33:55.886116+0000","last_unstale":"2023-08-09T14:51:47.763518+0000","last_undegraded":"2023-08-09T14:51:47.763518+0000","last_fullsized":"2023-08-09T14:51:47.763518+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:33:54.845869+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:33:54.845869+0000","last_clean_scrub_stamp":"2023-08-09T13:33:54.845869+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T23:32:37.214710+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"160'252","reported_seq":947,"reported_epoch":171,"state":"active+clean","last_fresh":"2023-08-09T14:51:47.785811+0000","last_change":"2023-08-09T13:33:55.886345+0000","last_active":"2023-08-09T14:51:47.785811+0000","last_peered":"2023-08-09T14:51:47.785811+0000","last_clean":"2023-08-09T14:51:47.785811+0000","last_became_active":"2023-08-09T13:33:55.885781+0000","last_became_peered":"2023-08-09T13:33:55.885781+0000","last_unstale":"2023-08-09T14:51:47.785811+0000","last_undegraded":"2023-08-09T14:51:47.785811+0000","last_fullsized":"2023-08-09T14:51:47.785811+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:33:54.845869+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:33:54.845869+0000","last_clean_scrub_stamp":"2023-08-09T13:33:54.845869+0000","objects_scrubbed":0,"log_size":252,"log_dups_size":0,"ondisk_log_size":252,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T14:13:22.909308+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1291,"num_read_kb":2654,"num_write":403,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"21.10","version":"0'0","reported_seq":9,"reported_epoch":171,"state":"active+clean","last_fresh":"2023-08-09T14:51:47.799484+0000","last_change":"2023-08-09T14:51:47.792011+0000","last_active":"2023-08-09T14:51:47.799484+0000","last_peered":"2023-08-09T14:51:47.799484+0000","last_clean":"2023-08-09T14:51:47.799484+0000","last_became_active":"2023-08-09T14:51:47.791690+0000","last_became_peered":"2023-08-09T14:51:47.791690+0000","last_unstale":"2023-08-09T14:51:47.799484+0000","last_undegraded":"2023-08-09T14:51:47.799484+0000","last_fullsized":"2023-08-09T14:51:47.799484+0000","mapping_epoch":170,"log_start":"0'0","ondisk_log_start":"0'0","created":170,"last_epoch_clean":171,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T14:51:46.735123+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T14:51:46.735123+0000","last_clean_scrub_stamp":"2023-08-09T14:51:46.735123+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T18:07:44.866766+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"20.11","version":"0'0","reported_seq":11,"reported_epoch":171,"state":"active+clean","last_fresh":"2023-08-09T14:51:47.762267+0000","last_change":"2023-08-09T14:51:46.771405+0000","last_active":"2023-08-09T14:51:47.762267+0000","last_peered":"2023-08-09T14:51:47.762267+0000","last_clean":"2023-08-09T14:51:47.762267+0000","last_became_active":"2023-08-09T14:51:46.770725+0000","last_became_peered":"2023-08-09T14:51:46.770725+0000","last_unstale":"2023-08-09T14:51:47.762267+0000","last_undegraded":"2023-08-09T14:51:47.762267+0000","last_fullsized":"2023-08-09T14:51:47.762267+0000","mapping_epoch":169,"log_start":"0'0","ondisk_log_start":"0'0","created":169,"last_epoch_clean":170,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T14:51:45.719739+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T14:51:45.719739+0000","last_clean_scrub_stamp":"2023-08-09T14:51:45.719739+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T23:49:28.629389+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":328,"reported_epoch":171,"state":"active+clean","last_fresh":"2023-08-09T14:51:47.771842+0000","last_change":"2023-08-09T13:33:55.889565+0000","last_active":"2023-08-09T14:51:47.771842+0000","last_peered":"2023-08-09T14:51:47.771842+0000","last_clean":"2023-08-09T14:51:47.771842+0000","last_became_active":"2023-08-09T13:33:55.889164+0000","last_became_peered":"2023-08-09T13:33:55.889164+0000","last_unstale":"2023-08-09T14:51:47.771842+0000","last_undegraded":"2023-08-09T14:51:47.771842+0000","last_fullsized":"2023-08-09T14:51:47.771842+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:33:54.845869+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:33:54.845869+0000","last_clean_scrub_stamp":"2023-08-09T13:33:54.845869+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T14:46:14.719093+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"21.11","version":"0'0","reported_seq":9,"reported_epoch":171,"state":"active+clean","last_fresh":"2023-08-09T14:51:47.804603+0000","last_change":"2023-08-09T14:51:47.799276+0000","last_active":"2023-08-09T14:51:47.804603+0000","last_peered":"2023-08-09T14:51:47.804603+0000","last_clean":"2023-08-09T14:51:47.804603+0000","last_became_active":"2023-08-09T14:51:47.798952+0000","last_became_peered":"2023-08-09T14:51:47.798952+0000","last_unstale":"2023-08-09T14:51:47.804603+0000","last_undegraded":"2023-08-09T14:51:47.804603+0000","last_fullsized":"2023-08-09T14:51:47.804603+0000","mapping_epoch":170,"log_start":"0'0","ondisk_log_start":"0'0","created":170,"last_epoch_clean":171,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T14:51:46.735123+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T14:51:46.735123+0000","last_clean_scrub_stamp":"2023-08-09T14:51:46.735123+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T16:55:27.430423+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"20.10","version":"171'8","reported_seq":19,"reported_epoch":171,"state":"active+clean","last_fresh":"2023-08-09T14:51:47.814870+0000","last_change":"2023-08-09T14:51:46.783858+0000","last_active":"2023-08-09T14:51:47.814870+0000","last_peered":"2023-08-09T14:51:47.814870+0000","last_clean":"2023-08-09T14:51:47.814870+0000","last_became_active":"2023-08-09T14:51:46.783565+0000","last_became_peered":"2023-08-09T14:51:46.783565+0000","last_unstale":"2023-08-09T14:51:47.814870+0000","last_undegraded":"2023-08-09T14:51:47.814870+0000","last_fullsized":"2023-08-09T14:51:47.814870+0000","mapping_epoch":169,"log_start":"0'0","ondisk_log_start":"0'0","created":169,"last_epoch_clean":170,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T14:51:45.719739+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T14:51:45.719739+0000","last_clean_scrub_stamp":"2023-08-09T14:51:45.719739+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-11T01:11:24.931250+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":324,"reported_epoch":171,"state":"active+clean","last_fresh":"2023-08-09T14:51:47.764985+0000","last_change":"2023-08-09T13:33:56.404236+0000","last_active":"2023-08-09T14:51:47.764985+0000","last_peered":"2023-08-09T14:51:47.764985+0000","last_clean":"2023-08-09T14:51:47.764985+0000","last_became_active":"2023-08-09T13:33:56.402485+0000","last_became_peered":"2023-08-09T13:33:56.402485+0000","last_unstale":"2023-08-09T14:51:47.764985+0000","last_undegraded":"2023-08-09T14:51:47.764985+0000","last_fullsized":"2023-08-09T14:51:47.764985+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:33:54.845869+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:33:54.845869+0000","last_clean_scrub_stamp":"2023-08-09T13:33:54.845869+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T14:26:21.684948+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"21.12","version":"0'0","reported_seq":9,"reported_epoch":171,"state":"active+clean","last_fresh":"2023-08-09T14:51:47.794052+0000","last_change":"2023-08-09T14:51:47.788323+0000","last_active":"2023-08-09T14:51:47.794052+0000","last_peered":"2023-08-09T14:51:47.794052+0000","last_clean":"2023-08-09T14:51:47.794052+0000","last_became_active":"2023-08-09T14:51:47.787761+0000","last_became_peered":"2023-08-09T14:51:47.787761+0000","last_unstale":"2023-08-09T14:51:47.794052+0000","last_undegraded":"2023-08-09T14:51:47.794052+0000","last_fullsized":"2023-08-09T14:51:47.794052+0000","mapping_epoch":170,"log_start":"0'0","ondisk_log_start":"0'0","created":170,"last_epoch_clean":171,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T14:51:46.735123+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T14:51:46.735123+0000","last_clean_scrub_stamp":"2023-08-09T14:51:46.735123+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T15:03:01.623215+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"20.13","version":"171'2","reported_seq":13,"reported_epoch":171,"state":"active+clean","last_fresh":"2023-08-09T14:51:47.811450+0000","last_change":"2023-08-09T14:51:46.784739+0000","last_active":"2023-08-09T14:51:47.811450+0000","last_peered":"2023-08-09T14:51:47.811450+0000","last_clean":"2023-08-09T14:51:47.811450+0000","last_became_active":"2023-08-09T14:51:46.784369+0000","last_became_peered":"2023-08-09T14:51:46.784369+0000","last_unstale":"2023-08-09T14:51:47.811450+0000","last_undegraded":"2023-08-09T14:51:47.811450+0000","last_fullsized":"2023-08-09T14:51:47.811450+0000","mapping_epoch":169,"log_start":"0'0","ondisk_log_start":"0'0","created":169,"last_epoch_clean":170,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T14:51:45.719739+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T14:51:45.719739+0000","last_clean_scrub_stamp":"2023-08-09T14:51:45.719739+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T23:19:23.402306+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":312,"reported_epoch":171,"state":"active+clean","last_fresh":"2023-08-09T14:51:47.785779+0000","last_change":"2023-08-09T13:33:56.401781+0000","last_active":"2023-08-09T14:51:47.785779+0000","last_peered":"2023-08-09T14:51:47.785779+0000","last_clean":"2023-08-09T14:51:47.785779+0000","last_became_active":"2023-08-09T13:33:56.400336+0000","last_became_peered":"2023-08-09T13:33:56.400336+0000","last_unstale":"2023-08-09T14:51:47.785779+0000","last_undegraded":"2023-08-09T14:51:47.785779+0000","last_fullsized":"2023-08-09T14:51:47.785779+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:33:54.845869+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:33:54.845869+0000","last_clean_scrub_stamp":"2023-08-09T13:33:54.845869+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T15:12:23.106020+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"21.13","version":"0'0","reported_seq":9,"reported_epoch":171,"state":"active+clean","last_fresh":"2023-08-09T14:51:47.788598+0000","last_change":"2023-08-09T14:51:47.784733+0000","last_active":"2023-08-09T14:51:47.788598+0000","last_peered":"2023-08-09T14:51:47.788598+0000","last_clean":"2023-08-09T14:51:47.788598+0000","last_became_active":"2023-08-09T14:51:47.784344+0000","last_became_peered":"2023-08-09T14:51:47.784344+0000","last_unstale":"2023-08-09T14:51:47.788598+0000","last_undegraded":"2023-08-09T14:51:47.788598+0000","last_fullsized":"2023-08-09T14:51:47.788598+0000","mapping_epoch":170,"log_start":"0'0","ondisk_log_start":"0'0","created":170,"last_epoch_clean":171,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T14:51:46.735123+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T14:51:46.735123+0000","last_clean_scrub_stamp":"2023-08-09T14:51:46.735123+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T22:55:45.066024+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"20.12","version":"0'0","reported_seq":11,"reported_epoch":171,"state":"active+clean","last_fresh":"2023-08-09T14:51:47.785348+0000","last_change":"2023-08-09T14:51:46.787485+0000","last_active":"2023-08-09T14:51:47.785348+0000","last_peered":"2023-08-09T14:51:47.785348+0000","last_clean":"2023-08-09T14:51:47.785348+0000","last_became_active":"2023-08-09T14:51:46.787196+0000","last_became_peered":"2023-08-09T14:51:46.787196+0000","last_unstale":"2023-08-09T14:51:47.785348+0000","last_undegraded":"2023-08-09T14:51:47.785348+0000","last_fullsized":"2023-08-09T14:51:47.785348+0000","mapping_epoch":169,"log_start":"0'0","ondisk_log_start":"0'0","created":169,"last_epoch_clean":170,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T14:51:45.719739+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T14:51:45.719739+0000","last_clean_scrub_stamp":"2023-08-09T14:51:45.719739+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T21:09:02.297470+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":312,"reported_epoch":171,"state":"active+clean","last_fresh":"2023-08-09T14:51:47.777642+0000","last_change":"2023-08-09T13:33:55.886389+0000","last_active":"2023-08-09T14:51:47.777642+0000","last_peered":"2023-08-09T14:51:47.777642+0000","last_clean":"2023-08-09T14:51:47.777642+0000","last_became_active":"2023-08-09T13:33:55.885638+0000","last_became_peered":"2023-08-09T13:33:55.885638+0000","last_unstale":"2023-08-09T14:51:47.777642+0000","last_undegraded":"2023-08-09T14:51:47.777642+0000","last_fullsized":"2023-08-09T14:51:47.777642+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:33:54.845869+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:33:54.845869+0000","last_clean_scrub_stamp":"2023-08-09T13:33:54.845869+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T13:38:55.954033+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]}],"pool_stats":[{"poolid":21,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":20,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":38,"ondisk_log_size":38,"up":64,"acting":64,"num_store_stats":8},{"poolid":2,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1291,"num_read_kb":2654,"num_write":403,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":252,"ondisk_log_size":252,"up":2,"acting":2,"num_store_stats":2},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":8}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834575779,"num_pgs":34,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":47736,"kb_used_data":2660,"kb_used_omap":1,"kb_used_meta":45054,"kb_avail":94324104,"statfs":{"total":96636764160,"available":96587882496,"internally_reserved":0,"allocated":2723840,"data_stored":1743157,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1781,"internal_metadata":46135563},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Aug 9 14:51:11 2023","interfaces":[{"interface":"back","average":{"1min":1.2729999999999999,"5min":1.2350000000000001,"15min":1.24},"min":{"1min":0.79200000000000004,"5min":0.76800000000000002,"15min":0.60499999999999998},"max":{"1min":1.625,"5min":2.2320000000000002,"15min":2.2320000000000002},"last":0.90700000000000003},{"interface":"front","average":{"1min":1.1990000000000001,"5min":1.2250000000000001,"15min":1.258},"min":{"1min":0.88500000000000001,"5min":0.82399999999999995,"15min":0.82399999999999995},"max":{"1min":1.796,"5min":1.871,"15min":1.871},"last":1.274}]},{"osd":1,"last update":"Wed Aug 9 14:51:11 2023","interfaces":[{"interface":"back","average":{"1min":1.244,"5min":1.292,"15min":1.325},"min":{"1min":0.89800000000000002,"5min":0.89800000000000002,"15min":0.79900000000000004},"max":{"1min":1.702,"5min":1.9239999999999999,"15min":1.9470000000000001},"last":0.72799999999999998},{"interface":"front","average":{"1min":1.351,"5min":1.319,"15min":1.3080000000000001},"min":{"1min":0.93999999999999995,"5min":0.874,"15min":0.874},"max":{"1min":1.7270000000000001,"5min":1.7270000000000001,"15min":1.9399999999999999},"last":1.377}]},{"osd":2,"last update":"Wed Aug 9 14:51:11 2023","interfaces":[{"interface":"back","average":{"1min":1.2729999999999999,"5min":1.306,"15min":1.3129999999999999},"min":{"1min":0.96199999999999997,"5min":0.89900000000000002,"15min":0.89500000000000002},"max":{"1min":1.6890000000000001,"5min":2.3149999999999999,"15min":2.3149999999999999},"last":1.3320000000000001},{"interface":"front","average":{"1min":1.375,"5min":1.3740000000000001,"15min":1.3899999999999999},"min":{"1min":1.0900000000000001,"5min":0.93200000000000005,"15min":0.92100000000000004},"max":{"1min":1.675,"5min":1.9279999999999999,"15min":2.0110000000000001},"last":1.04}]},{"osd":3,"last update":"Wed Aug 9 14:51:11 2023","interfaces":[{"interface":"back","average":{"1min":1.357,"5min":1.359,"15min":1.365},"min":{"1min":0.94999999999999996,"5min":0.94999999999999996,"15min":0.90600000000000003},"max":{"1min":1.6160000000000001,"5min":1.9910000000000001,"15min":1.9910000000000001},"last":1.4410000000000001},{"interface":"front","average":{"1min":1.5269999999999999,"5min":1.48,"15min":1.474},"min":{"1min":1.236,"5min":0.98599999999999999,"15min":0.98599999999999999},"max":{"1min":1.887,"5min":2.375,"15min":2.375},"last":1.407}]},{"osd":4,"last update":"Wed Aug 9 14:51:11 2023","interfaces":[{"interface":"back","average":{"1min":1.294,"5min":1.294,"15min":1.3069999999999999},"min":{"1min":0.82499999999999996,"5min":0.82499999999999996,"15min":0.79800000000000004},"max":{"1min":1.7090000000000001,"5min":1.901,"15min":2.254},"last":1.087},{"interface":"front","average":{"1min":1.4790000000000001,"5min":1.4099999999999999,"15min":1.3879999999999999},"min":{"1min":1.0940000000000001,"5min":0.90400000000000003,"15min":0.90400000000000003},"max":{"1min":1.776,"5min":2.1789999999999998,"15min":2.1789999999999998},"last":1.1899999999999999}]},{"osd":5,"last update":"Wed Aug 9 14:51:11 2023","interfaces":[{"interface":"back","average":{"1min":1.502,"5min":1.498,"15min":1.476},"min":{"1min":1.101,"5min":0.85599999999999998,"15min":0.76900000000000002},"max":{"1min":1.929,"5min":2.1589999999999998,"15min":2.1589999999999998},"last":1.133},{"interface":"front","average":{"1min":1.476,"5min":1.504,"15min":1.4910000000000001},"min":{"1min":1.0920000000000001,"5min":0.92400000000000004,"15min":0.92400000000000004},"max":{"1min":1.9430000000000001,"5min":2.4510000000000001,"15min":2.4510000000000001},"last":1.589}]},{"osd":6,"last update":"Wed Aug 9 14:51:11 2023","interfaces":[{"interface":"back","average":{"1min":1.6439999999999999,"5min":1.613,"15min":1.6020000000000001},"min":{"1min":1.369,"5min":1.0529999999999999,"15min":1.0529999999999999},"max":{"1min":1.8660000000000001,"5min":2.266,"15min":2.5419999999999998},"last":1.4890000000000001},{"interface":"front","average":{"1min":1.5489999999999999,"5min":1.542,"15min":1.55},"min":{"1min":1.222,"5min":0.98499999999999999,"15min":0.84899999999999998},"max":{"1min":1.8240000000000001,"5min":2.415,"15min":2.415},"last":1.5429999999999999}]}]},{"osd":6,"up_from":13,"seq":55834575779,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":41392,"kb_used_data":2076,"kb_used_omap":1,"kb_used_meta":39294,"kb_avail":94330448,"statfs":{"total":96636764160,"available":96594378752,"internally_reserved":0,"allocated":2125824,"data_stored":1151428,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1781,"internal_metadata":40237323},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Aug 9 14:51:20 2023","interfaces":[{"interface":"back","average":{"1min":1.246,"5min":1.181,"15min":1.202},"min":{"1min":0.92000000000000004,"5min":0.75,"15min":0.745},"max":{"1min":1.6080000000000001,"5min":1.6870000000000001,"15min":2.278},"last":1.6879999999999999},{"interface":"front","average":{"1min":1.262,"5min":1.262,"15min":1.2549999999999999},"min":{"1min":0.93000000000000005,"5min":0.84999999999999998,"15min":0.84199999999999997},"max":{"1min":2.2170000000000001,"5min":2.2170000000000001,"15min":2.2170000000000001},"last":1.3380000000000001}]},{"osd":1,"last update":"Wed Aug 9 14:51:20 2023","interfaces":[{"interface":"back","average":{"1min":1.226,"5min":1.2390000000000001,"15min":1.224},"min":{"1min":0.88400000000000001,"5min":0.88400000000000001,"15min":0.84799999999999998},"max":{"1min":1.7569999999999999,"5min":1.829,"15min":1.829},"last":1.1299999999999999},{"interface":"front","average":{"1min":1.1850000000000001,"5min":1.28,"15min":1.2989999999999999},"min":{"1min":0.88500000000000001,"5min":0.79400000000000004,"15min":0.79400000000000004},"max":{"1min":1.5369999999999999,"5min":1.8049999999999999,"15min":1.903},"last":1.9099999999999999}]},{"osd":2,"last update":"Wed Aug 9 14:51:20 2023","interfaces":[{"interface":"back","average":{"1min":1.4219999999999999,"5min":1.3500000000000001,"15min":1.3520000000000001},"min":{"1min":1.0309999999999999,"5min":0.89100000000000001,"15min":0.88100000000000001},"max":{"1min":1.863,"5min":1.863,"15min":2.0920000000000001},"last":1.401},{"interface":"front","average":{"1min":1.395,"5min":1.3700000000000001,"15min":1.3340000000000001},"min":{"1min":1.034,"5min":0.93200000000000005,"15min":0.90900000000000003},"max":{"1min":1.6890000000000001,"5min":1.857,"15min":1.968},"last":1.984}]},{"osd":3,"last update":"Wed Aug 9 14:51:20 2023","interfaces":[{"interface":"back","average":{"1min":1.427,"5min":1.4259999999999999,"15min":1.4159999999999999},"min":{"1min":1.167,"5min":1.085,"15min":1.0089999999999999},"max":{"1min":1.774,"5min":1.7929999999999999,"15min":1.8169999999999999},"last":1.4830000000000001},{"interface":"front","average":{"1min":1.4350000000000001,"5min":1.4099999999999999,"15min":1.3859999999999999},"min":{"1min":1.0529999999999999,"5min":0.97199999999999998,"15min":0.93300000000000005},"max":{"1min":1.7090000000000001,"5min":1.8280000000000001,"15min":1.8280000000000001},"last":1.609}]},{"osd":4,"last update":"Wed Aug 9 14:51:20 2023","interfaces":[{"interface":"back","average":{"1min":1.4330000000000001,"5min":1.3480000000000001,"15min":1.2989999999999999},"min":{"1min":0.82299999999999995,"5min":0.82299999999999995,"15min":0.82199999999999995},"max":{"1min":1.9810000000000001,"5min":1.9810000000000001,"15min":1.9810000000000001},"last":0.85399999999999998},{"interface":"front","average":{"1min":1.5309999999999999,"5min":1.444,"15min":1.371},"min":{"1min":1.002,"5min":0.871,"15min":0.71599999999999997},"max":{"1min":1.925,"5min":1.9530000000000001,"15min":1.9530000000000001},"last":1.742}]},{"osd":5,"last update":"Wed Aug 9 14:51:20 2023","interfaces":[{"interface":"back","average":{"1min":1.46,"5min":1.4299999999999999,"15min":1.3700000000000001},"min":{"1min":0.83599999999999997,"5min":0.83599999999999997,"15min":0.83599999999999997},"max":{"1min":2.0609999999999999,"5min":2.2170000000000001,"15min":2.2170000000000001},"last":1.821},{"interface":"front","average":{"1min":1.6859999999999999,"5min":1.6279999999999999,"15min":1.6020000000000001},"min":{"1min":1.363,"5min":1.331,"15min":0.93500000000000005},"max":{"1min":2.1619999999999999,"5min":2.1619999999999999,"15min":2.1619999999999999},"last":1.21}]},{"osd":7,"last update":"Wed Aug 9 14:51:20 2023","interfaces":[{"interface":"back","average":{"1min":1.7010000000000001,"5min":1.657,"15min":1.6000000000000001},"min":{"1min":1.327,"5min":0.96299999999999997,"15min":0.96299999999999997},"max":{"1min":2.476,"5min":2.476,"15min":2.5310000000000001},"last":1.8700000000000001},{"interface":"front","average":{"1min":1.675,"5min":1.625,"15min":1.5920000000000001},"min":{"1min":1.0549999999999999,"5min":0.997,"15min":0.96999999999999997},"max":{"1min":2.0670000000000002,"5min":2.286,"15min":2.286},"last":1.9399999999999999}]}]},{"osd":5,"up_from":13,"seq":55834575779,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":59428,"kb_used_data":2068,"kb_used_omap":1,"kb_used_meta":57342,"kb_avail":94312412,"statfs":{"total":96636764160,"available":96575909888,"internally_reserved":0,"allocated":2117632,"data_stored":1145297,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1781,"internal_metadata":58718475},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Aug 9 14:51:12 2023","interfaces":[{"interface":"back","average":{"1min":1.1819999999999999,"5min":1.1970000000000001,"15min":1.194},"min":{"1min":0.81999999999999995,"5min":0.72599999999999998,"15min":0.72599999999999998},"max":{"1min":1.6899999999999999,"5min":1.819,"15min":1.976},"last":1.5},{"interface":"front","average":{"1min":1.3320000000000001,"5min":1.24,"15min":1.2350000000000001},"min":{"1min":0.92700000000000005,"5min":0.84999999999999998,"15min":0.79200000000000004},"max":{"1min":1.9470000000000001,"5min":1.9470000000000001,"15min":1.9470000000000001},"last":1.5569999999999999}]},{"osd":1,"last update":"Wed Aug 9 14:51:12 2023","interfaces":[{"interface":"back","average":{"1min":1.361,"5min":1.2989999999999999,"15min":1.26},"min":{"1min":0.96699999999999997,"5min":0.89500000000000002,"15min":0.82199999999999995},"max":{"1min":1.7929999999999999,"5min":1.7929999999999999,"15min":1.7929999999999999},"last":1.0840000000000001},{"interface":"front","average":{"1min":1.274,"5min":1.327,"15min":1.327},"min":{"1min":0.99299999999999999,"5min":0.90000000000000002,"15min":0.877},"max":{"1min":1.544,"5min":1.8480000000000001,"15min":1.8480000000000001},"last":1.147}]},{"osd":2,"last update":"Wed Aug 9 14:51:12 2023","interfaces":[{"interface":"back","average":{"1min":1.3440000000000001,"5min":1.3500000000000001,"15min":1.329},"min":{"1min":1.004,"5min":0.96099999999999997,"15min":0.86199999999999999},"max":{"1min":1.7529999999999999,"5min":1.7529999999999999,"15min":1.8939999999999999},"last":1.2230000000000001},{"interface":"front","average":{"1min":1.4630000000000001,"5min":1.3819999999999999,"15min":1.351},"min":{"1min":1.093,"5min":0.98499999999999999,"15min":0.88800000000000001},"max":{"1min":1.694,"5min":1.8300000000000001,"15min":1.8540000000000001},"last":1.4450000000000001}]},{"osd":3,"last update":"Wed Aug 9 14:51:12 2023","interfaces":[{"interface":"back","average":{"1min":1.4039999999999999,"5min":1.3720000000000001,"15min":1.3799999999999999},"min":{"1min":1.079,"5min":0.95999999999999996,"15min":0.95999999999999996},"max":{"1min":1.766,"5min":1.7809999999999999,"15min":2.04},"last":1.619},{"interface":"front","average":{"1min":1.456,"5min":1.498,"15min":1.4690000000000001},"min":{"1min":1.1499999999999999,"5min":1.0009999999999999,"15min":0.98299999999999998},"max":{"1min":1.752,"5min":1.8109999999999999,"15min":2.0409999999999999},"last":1.6919999999999999}]},{"osd":4,"last update":"Wed Aug 9 14:51:12 2023","interfaces":[{"interface":"back","average":{"1min":1.421,"5min":1.3560000000000001,"15min":1.3080000000000001},"min":{"1min":1.0389999999999999,"5min":0.871,"15min":0.77600000000000002},"max":{"1min":1.905,"5min":1.905,"15min":1.905},"last":1.9690000000000001},{"interface":"front","average":{"1min":1.349,"5min":1.4330000000000001,"15min":1.423},"min":{"1min":0.94799999999999995,"5min":0.94799999999999995,"15min":0.84499999999999997},"max":{"1min":1.8280000000000001,"5min":1.911,"15min":2.1899999999999999},"last":1.8460000000000001}]},{"osd":6,"last update":"Wed Aug 9 14:51:12 2023","interfaces":[{"interface":"back","average":{"1min":1.415,"5min":1.403,"15min":1.4199999999999999},"min":{"1min":1.0469999999999999,"5min":0.88600000000000001,"15min":0.79300000000000004},"max":{"1min":1.8839999999999999,"5min":1.8839999999999999,"15min":2.0960000000000001},"last":1.9059999999999999},{"interface":"front","average":{"1min":1.512,"5min":1.4950000000000001,"15min":1.48},"min":{"1min":1.0589999999999999,"5min":1.0169999999999999,"15min":0.871},"max":{"1min":1.9630000000000001,"5min":1.9630000000000001,"15min":1.9730000000000001},"last":2.1280000000000001}]},{"osd":7,"last update":"Wed Aug 9 14:51:12 2023","interfaces":[{"interface":"back","average":{"1min":1.5129999999999999,"5min":1.554,"15min":1.5780000000000001},"min":{"1min":1.085,"5min":1.054,"15min":0.98899999999999999},"max":{"1min":1.845,"5min":1.9319999999999999,"15min":2.3170000000000002},"last":2.0680000000000001},{"interface":"front","average":{"1min":1.659,"5min":1.6299999999999999,"15min":1.637},"min":{"1min":1.1859999999999999,"5min":1.008,"15min":0.95099999999999996},"max":{"1min":2.0049999999999999,"5min":2.0649999999999999,"15min":2.3500000000000001},"last":2.105}]}]},{"osd":4,"up_from":13,"seq":55834575779,"num_pgs":13,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":59428,"kb_used_data":2068,"kb_used_omap":1,"kb_used_meta":57342,"kb_avail":94312412,"statfs":{"total":96636764160,"available":96575909888,"internally_reserved":0,"allocated":2117632,"data_stored":1145297,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1781,"internal_metadata":58718475},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Aug 9 14:51:38 2023","interfaces":[{"interface":"back","average":{"1min":1.1859999999999999,"5min":1.2070000000000001,"15min":1.2050000000000001},"min":{"1min":0.69999999999999996,"5min":0.69999999999999996,"15min":0.69999999999999996},"max":{"1min":2.573,"5min":2.573,"15min":2.573},"last":0.77300000000000002},{"interface":"front","average":{"1min":1.3460000000000001,"5min":1.341,"15min":1.3260000000000001},"min":{"1min":0.83799999999999997,"5min":0.83799999999999997,"15min":0.54000000000000004},"max":{"1min":2.1960000000000002,"5min":2.1960000000000002,"15min":2.1960000000000002},"last":1.661}]},{"osd":1,"last update":"Wed Aug 9 14:51:38 2023","interfaces":[{"interface":"back","average":{"1min":1.49,"5min":1.3799999999999999,"15min":1.3540000000000001},"min":{"1min":0.86599999999999999,"5min":0.86599999999999999,"15min":0.60599999999999998},"max":{"1min":4.4299999999999997,"5min":4.4299999999999997,"15min":4.4299999999999997},"last":1.7509999999999999},{"interface":"front","average":{"1min":1.204,"5min":1.292,"15min":1.2849999999999999},"min":{"1min":0.68200000000000005,"5min":0.68200000000000005,"15min":0.68200000000000005},"max":{"1min":2.4630000000000001,"5min":2.4630000000000001,"15min":2.4630000000000001},"last":0.83799999999999997}]},{"osd":2,"last update":"Wed Aug 9 14:51:38 2023","interfaces":[{"interface":"back","average":{"1min":1.335,"5min":1.3260000000000001,"15min":1.3109999999999999},"min":{"1min":0.81499999999999995,"5min":0.81499999999999995,"15min":0.74199999999999999},"max":{"1min":2.6920000000000002,"5min":2.6920000000000002,"15min":2.6920000000000002},"last":0.90000000000000002},{"interface":"front","average":{"1min":1.462,"5min":1.454,"15min":1.4530000000000001},"min":{"1min":0.70899999999999996,"5min":0.70899999999999996,"15min":0.70899999999999996},"max":{"1min":2.0830000000000002,"5min":2.1619999999999999,"15min":2.4359999999999999},"last":1.2889999999999999}]},{"osd":3,"last update":"Wed Aug 9 14:51:38 2023","interfaces":[{"interface":"back","average":{"1min":1.369,"5min":1.3360000000000001,"15min":1.339},"min":{"1min":0.85699999999999998,"5min":0.85699999999999998,"15min":0.85699999999999998},"max":{"1min":2.2069999999999999,"5min":2.2069999999999999,"15min":2.6429999999999998},"last":1.097},{"interface":"front","average":{"1min":1.534,"5min":1.4970000000000001,"15min":1.4750000000000001},"min":{"1min":0.75,"5min":0.75,"15min":0.75},"max":{"1min":3.0600000000000001,"5min":3.0600000000000001,"15min":3.0600000000000001},"last":1.044}]},{"osd":5,"last update":"Wed Aug 9 14:51:38 2023","interfaces":[{"interface":"back","average":{"1min":1.462,"5min":1.4099999999999999,"15min":1.377},"min":{"1min":0.77100000000000002,"5min":0.77100000000000002,"15min":0.70399999999999996},"max":{"1min":2.0710000000000002,"5min":2.5009999999999999,"15min":2.5009999999999999},"last":1.4470000000000001},{"interface":"front","average":{"1min":1.343,"5min":1.403,"15min":1.385},"min":{"1min":0.56999999999999995,"5min":0.56999999999999995,"15min":0.56999999999999995},"max":{"1min":1.9790000000000001,"5min":2.4129999999999998,"15min":3.0110000000000001},"last":1.399}]},{"osd":6,"last update":"Wed Aug 9 14:51:38 2023","interfaces":[{"interface":"back","average":{"1min":1.589,"5min":1.4470000000000001,"15min":1.431},"min":{"1min":0.93200000000000005,"5min":0.85499999999999998,"15min":0.79500000000000004},"max":{"1min":2.9540000000000002,"5min":2.9540000000000002,"15min":2.9540000000000002},"last":1.1499999999999999},{"interface":"front","average":{"1min":1.6739999999999999,"5min":1.619,"15min":1.5680000000000001},"min":{"1min":0.59899999999999998,"5min":0.59899999999999998,"15min":0.59899999999999998},"max":{"1min":2.0529999999999999,"5min":2.3260000000000001,"15min":2.5310000000000001},"last":1.5680000000000001}]},{"osd":7,"last update":"Wed Aug 9 14:51:38 2023","interfaces":[{"interface":"back","average":{"1min":1.5629999999999999,"5min":1.5569999999999999,"15min":1.5229999999999999},"min":{"1min":0.79100000000000004,"5min":0.79100000000000004,"15min":0.79100000000000004},"max":{"1min":2.1360000000000001,"5min":2.331,"15min":3.3149999999999999},"last":1.506},{"interface":"front","average":{"1min":1.75,"5min":1.6339999999999999,"15min":1.587},"min":{"1min":0.88400000000000001,"5min":0.88400000000000001,"15min":0.82599999999999996},"max":{"1min":3.274,"5min":3.274,"15min":3.274},"last":1.256}]}]},{"osd":3,"up_from":13,"seq":55834575780,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":59428,"kb_used_data":2068,"kb_used_omap":1,"kb_used_meta":57342,"kb_avail":94312412,"statfs":{"total":96636764160,"available":96575909888,"internally_reserved":0,"allocated":2117632,"data_stored":1145297,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1781,"internal_metadata":58718475},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Aug 9 14:51:19 2023","interfaces":[{"interface":"back","average":{"1min":0.97199999999999998,"5min":0.95999999999999996,"15min":0.95199999999999996},"min":{"1min":0.5,"5min":0.5,"15min":0.5},"max":{"1min":1.5620000000000001,"5min":1.722,"15min":1.8740000000000001},"last":0.90800000000000003},{"interface":"front","average":{"1min":1.091,"5min":1.133,"15min":1.1559999999999999},"min":{"1min":0.81599999999999995,"5min":0.748,"15min":0.748},"max":{"1min":1.4910000000000001,"5min":1.9179999999999999,"15min":1.9179999999999999},"last":0.69199999999999995}]},{"osd":1,"last update":"Wed Aug 9 14:51:19 2023","interfaces":[{"interface":"back","average":{"1min":1.042,"5min":1.073,"15min":1.077},"min":{"1min":0.56999999999999995,"5min":0.56999999999999995,"15min":0.56999999999999995},"max":{"1min":1.651,"5min":1.7589999999999999,"15min":1.903},"last":0.78000000000000003},{"interface":"front","average":{"1min":1.2450000000000001,"5min":1.2290000000000001,"15min":1.2190000000000001},"min":{"1min":0.68200000000000005,"5min":0.68200000000000005,"15min":0.68200000000000005},"max":{"1min":1.802,"5min":1.8320000000000001,"15min":1.903},"last":0.72799999999999998}]},{"osd":2,"last update":"Wed Aug 9 14:51:19 2023","interfaces":[{"interface":"back","average":{"1min":1.4259999999999999,"5min":1.3500000000000001,"15min":1.337},"min":{"1min":0.99099999999999999,"5min":0.92900000000000005,"15min":0.878},"max":{"1min":3.7770000000000001,"5min":3.7770000000000001,"15min":3.7770000000000001},"last":1.0429999999999999},{"interface":"front","average":{"1min":1.591,"5min":1.425,"15min":1.411},"min":{"1min":1.026,"5min":0.94899999999999995,"15min":0.92800000000000005},"max":{"1min":3.5750000000000002,"5min":3.5750000000000002,"15min":3.5750000000000002},"last":1.194}]},{"osd":4,"last update":"Wed Aug 9 14:51:19 2023","interfaces":[{"interface":"back","average":{"1min":1.3420000000000001,"5min":1.409,"15min":1.391},"min":{"1min":0.92200000000000004,"5min":0.88400000000000001,"15min":0.86499999999999999},"max":{"1min":1.714,"5min":1.9359999999999999,"15min":2.153},"last":0.82699999999999996},{"interface":"front","average":{"1min":1.4399999999999999,"5min":1.4690000000000001,"15min":1.472},"min":{"1min":0.79300000000000004,"5min":0.79300000000000004,"15min":0.79300000000000004},"max":{"1min":1.792,"5min":1.992,"15min":2.3439999999999999},"last":1.294}]},{"osd":5,"last update":"Wed Aug 9 14:51:19 2023","interfaces":[{"interface":"back","average":{"1min":1.54,"5min":1.5609999999999999,"15min":1.5620000000000001},"min":{"1min":0.85199999999999998,"5min":0.85199999999999998,"15min":0.85199999999999998},"max":{"1min":1.8580000000000001,"5min":2.0339999999999998,"15min":2.2629999999999999},"last":1.5760000000000001},{"interface":"front","average":{"1min":1.5940000000000001,"5min":1.5700000000000001,"15min":1.554},"min":{"1min":1.1619999999999999,"5min":1.052,"15min":1.052},"max":{"1min":2.3399999999999999,"5min":2.4630000000000001,"15min":2.4630000000000001},"last":1.274}]},{"osd":6,"last update":"Wed Aug 9 14:51:19 2023","interfaces":[{"interface":"back","average":{"1min":1.5229999999999999,"5min":1.5629999999999999,"15min":1.5309999999999999},"min":{"1min":0.93500000000000005,"5min":0.93500000000000005,"15min":0.93500000000000005},"max":{"1min":1.9470000000000001,"5min":2.0790000000000002,"15min":2.343},"last":1.1619999999999999},{"interface":"front","average":{"1min":1.6739999999999999,"5min":1.653,"15min":1.643},"min":{"1min":1.038,"5min":1.038,"15min":1.038},"max":{"1min":2.0819999999999999,"5min":2.3370000000000002,"15min":2.4140000000000001},"last":1.5149999999999999}]},{"osd":7,"last update":"Wed Aug 9 14:51:19 2023","interfaces":[{"interface":"back","average":{"1min":1.722,"5min":1.7150000000000001,"15min":1.6890000000000001},"min":{"1min":1.367,"5min":1.3169999999999999,"15min":1.2589999999999999},"max":{"1min":2.427,"5min":2.5430000000000001,"15min":2.5430000000000001},"last":1.546},{"interface":"front","average":{"1min":1.6950000000000001,"5min":1.724,"15min":1.681},"min":{"1min":1.2070000000000001,"5min":1.2070000000000001,"15min":1.1879999999999999},"max":{"1min":2.0750000000000002,"5min":2.2069999999999999,"15min":3.1339999999999999},"last":1.2410000000000001}]}]},{"osd":2,"up_from":13,"seq":55834575779,"num_pgs":5,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":41380,"kb_used_data":2068,"kb_used_omap":1,"kb_used_meta":39294,"kb_avail":94330460,"statfs":{"total":96636764160,"available":96594391040,"internally_reserved":0,"allocated":2117632,"data_stored":1145297,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1781,"internal_metadata":40237323},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Aug 9 14:51:11 2023","interfaces":[{"interface":"back","average":{"1min":0.93000000000000005,"5min":0.91200000000000003,"15min":0.92400000000000004},"min":{"1min":0.755,"5min":0.625,"15min":0.54400000000000004},"max":{"1min":1.4079999999999999,"5min":1.4079999999999999,"15min":1.7190000000000001},"last":0.86399999999999999},{"interface":"front","average":{"1min":1.119,"5min":1.1020000000000001,"15min":1.117},"min":{"1min":0.79800000000000004,"5min":0.67600000000000005,"15min":0.54800000000000004},"max":{"1min":1.8200000000000001,"5min":1.8300000000000001,"15min":1.8740000000000001},"last":1.4359999999999999}]},{"osd":1,"last update":"Wed Aug 9 14:51:11 2023","interfaces":[{"interface":"back","average":{"1min":1.2609999999999999,"5min":1.181,"15min":1.226},"min":{"1min":0.90900000000000003,"5min":0.59399999999999997,"15min":0.59399999999999997},"max":{"1min":1.514,"5min":1.732,"15min":2.1600000000000001},"last":1.099},{"interface":"front","average":{"1min":1.4770000000000001,"5min":1.446,"15min":1.4470000000000001},"min":{"1min":1.161,"5min":0.66700000000000004,"15min":0.66700000000000004},"max":{"1min":1.861,"5min":2.0350000000000001,"15min":2.2200000000000002},"last":1.534}]},{"osd":3,"last update":"Wed Aug 9 14:51:11 2023","interfaces":[{"interface":"back","average":{"1min":1.0600000000000001,"5min":1.103,"15min":1.1240000000000001},"min":{"1min":0.877,"5min":0.73399999999999999,"15min":0.70799999999999996},"max":{"1min":1.371,"5min":1.7350000000000001,"15min":1.9239999999999999},"last":1.472},{"interface":"front","average":{"1min":1.4039999999999999,"5min":1.411,"15min":1.375},"min":{"1min":1.04,"5min":0.876,"15min":0.876},"max":{"1min":1.7430000000000001,"5min":2.0299999999999998,"15min":2.0299999999999998},"last":1.2769999999999999}]},{"osd":4,"last update":"Wed Aug 9 14:51:11 2023","interfaces":[{"interface":"back","average":{"1min":1.5940000000000001,"5min":1.5249999999999999,"15min":1.52},"min":{"1min":1.2490000000000001,"5min":1.0609999999999999,"15min":0.95699999999999996},"max":{"1min":1.8819999999999999,"5min":2.1400000000000001,"15min":2.2389999999999999},"last":1.2150000000000001},{"interface":"front","average":{"1min":1.5169999999999999,"5min":1.474,"15min":1.4790000000000001},"min":{"1min":1.202,"5min":0.85599999999999998,"15min":0.85599999999999998},"max":{"1min":1.9810000000000001,"5min":2.2509999999999999,"15min":2.2509999999999999},"last":1.319}]},{"osd":5,"last update":"Wed Aug 9 14:51:11 2023","interfaces":[{"interface":"back","average":{"1min":1.623,"5min":1.5649999999999999,"15min":1.5389999999999999},"min":{"1min":1.194,"5min":0.90100000000000002,"15min":0.90100000000000002},"max":{"1min":2.0070000000000001,"5min":2.1190000000000002,"15min":2.1619999999999999},"last":1.645},{"interface":"front","average":{"1min":1.5289999999999999,"5min":1.478,"15min":1.478},"min":{"1min":1.1339999999999999,"5min":0.92200000000000004,"15min":0.92200000000000004},"max":{"1min":1.9239999999999999,"5min":1.9930000000000001,"15min":2.1869999999999998},"last":1.6890000000000001}]},{"osd":6,"last update":"Wed Aug 9 14:51:11 2023","interfaces":[{"interface":"back","average":{"1min":1.6140000000000001,"5min":1.5409999999999999,"15min":1.5129999999999999},"min":{"1min":1.26,"5min":0.82599999999999996,"15min":0.82599999999999996},"max":{"1min":2.0099999999999998,"5min":2.081,"15min":2.3210000000000002},"last":1.851},{"interface":"front","average":{"1min":1.673,"5min":1.639,"15min":1.6240000000000001},"min":{"1min":1.3,"5min":0.98699999999999999,"15min":0.98699999999999999},"max":{"1min":2.0339999999999998,"5min":2.2759999999999998,"15min":2.5070000000000001},"last":1.395}]},{"osd":7,"last update":"Wed Aug 9 14:51:11 2023","interfaces":[{"interface":"back","average":{"1min":1.724,"5min":1.704,"15min":1.665},"min":{"1min":1.3660000000000001,"5min":0.96099999999999997,"15min":0.96099999999999997},"max":{"1min":2.1269999999999998,"5min":2.419,"15min":2.419},"last":1.7310000000000001},{"interface":"front","average":{"1min":1.6950000000000001,"5min":1.649,"15min":1.6240000000000001},"min":{"1min":1.2569999999999999,"5min":1.018,"15min":1.018},"max":{"1min":2.0899999999999999,"5min":2.1349999999999998,"15min":2.3399999999999999},"last":2.3079999999999998}]}]},{"osd":1,"up_from":13,"seq":55834575779,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":51816,"kb_used_data":2648,"kb_used_omap":1,"kb_used_meta":49150,"kb_avail":94320024,"statfs":{"total":96636764160,"available":96583704576,"internally_reserved":0,"allocated":2711552,"data_stored":1735665,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1781,"internal_metadata":50329867},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Aug 9 14:51:22 2023","interfaces":[{"interface":"back","average":{"1min":0.97899999999999998,"5min":0.998,"15min":0.997},"min":{"1min":0.308,"5min":0.308,"15min":0.308},"max":{"1min":1.5009999999999999,"5min":1.879,"15min":1.931},"last":0.84499999999999997},{"interface":"front","average":{"1min":1.1759999999999999,"5min":1.139,"15min":1.131},"min":{"1min":0.77000000000000002,"5min":0.76900000000000002,"15min":0.71299999999999997},"max":{"1min":1.623,"5min":1.849,"15min":1.849},"last":0.91400000000000003}]},{"osd":2,"last update":"Wed Aug 9 14:51:22 2023","interfaces":[{"interface":"back","average":{"1min":1.173,"5min":1.111,"15min":1.087},"min":{"1min":0.79800000000000004,"5min":0.79700000000000004,"15min":0.76300000000000001},"max":{"1min":1.8400000000000001,"5min":2.71,"15min":2.71},"last":0.96899999999999997},{"interface":"front","average":{"1min":1.298,"5min":1.252,"15min":1.2609999999999999},"min":{"1min":0.72999999999999998,"5min":0.72999999999999998,"15min":0.72999999999999998},"max":{"1min":1.8919999999999999,"5min":1.8919999999999999,"15min":1.929},"last":1.341}]},{"osd":3,"last update":"Wed Aug 9 14:51:22 2023","interfaces":[{"interface":"back","average":{"1min":1.429,"5min":1.405,"15min":1.3979999999999999},"min":{"1min":0.83599999999999997,"5min":0.83599999999999997,"15min":0.83599999999999997},"max":{"1min":1.863,"5min":2.6070000000000002,"15min":2.6070000000000002},"last":1.1359999999999999},{"interface":"front","average":{"1min":1.4339999999999999,"5min":1.3999999999999999,"15min":1.375},"min":{"1min":0.86799999999999999,"5min":0.86799999999999999,"15min":0.80400000000000005},"max":{"1min":3.1629999999999998,"5min":3.1629999999999998,"15min":3.1629999999999998},"last":1.4810000000000001}]},{"osd":4,"last update":"Wed Aug 9 14:51:22 2023","interfaces":[{"interface":"back","average":{"1min":1.5189999999999999,"5min":1.512,"15min":1.4730000000000001},"min":{"1min":1.2509999999999999,"5min":1.1359999999999999,"15min":1.036},"max":{"1min":2.0489999999999999,"5min":2.0579999999999998,"15min":2.0579999999999998},"last":1.9850000000000001},{"interface":"front","average":{"1min":1.472,"5min":1.4690000000000001,"15min":1.4379999999999999},"min":{"1min":1.1539999999999999,"5min":1.0289999999999999,"15min":0.84699999999999998},"max":{"1min":1.899,"5min":3.2440000000000002,"15min":3.2440000000000002},"last":1.5509999999999999}]},{"osd":5,"last update":"Wed Aug 9 14:51:22 2023","interfaces":[{"interface":"back","average":{"1min":1.4199999999999999,"5min":1.462,"15min":1.4550000000000001},"min":{"1min":1.016,"5min":1.016,"15min":1.016},"max":{"1min":1.667,"5min":3.036,"15min":3.036},"last":1.7170000000000001},{"interface":"front","average":{"1min":1.641,"5min":1.6040000000000001,"15min":1.599},"min":{"1min":0.879,"5min":0.879,"15min":0.879},"max":{"1min":2.5019999999999998,"5min":2.5019999999999998,"15min":2.5019999999999998},"last":1.867}]},{"osd":6,"last update":"Wed Aug 9 14:51:22 2023","interfaces":[{"interface":"back","average":{"1min":1.6459999999999999,"5min":1.621,"15min":1.6020000000000001},"min":{"1min":1.3620000000000001,"5min":1.1639999999999999,"15min":1.1000000000000001},"max":{"1min":2.141,"5min":3.149,"15min":3.149},"last":2.1269999999999998},{"interface":"front","average":{"1min":1.581,"5min":1.611,"15min":1.5980000000000001},"min":{"1min":1.0629999999999999,"5min":1.0629999999999999,"15min":1.0629999999999999},"max":{"1min":1.9630000000000001,"5min":3.4460000000000002,"15min":3.4460000000000002},"last":1.9339999999999999}]},{"osd":7,"last update":"Wed Aug 9 14:51:22 2023","interfaces":[{"interface":"back","average":{"1min":1.681,"5min":1.6879999999999999,"15min":1.6919999999999999},"min":{"1min":0.79000000000000004,"5min":0.79000000000000004,"15min":0.79000000000000004},"max":{"1min":2.5939999999999999,"5min":2.5939999999999999,"15min":2.5939999999999999},"last":2.0529999999999999},{"interface":"front","average":{"1min":1.7769999999999999,"5min":1.7450000000000001,"15min":1.7230000000000001},"min":{"1min":1.1459999999999999,"5min":1.1459999999999999,"15min":1.1459999999999999},"max":{"1min":2.242,"5min":2.802,"15min":2.802},"last":2.2010000000000001}]}]},{"osd":0,"up_from":13,"seq":55834575779,"num_pgs":16,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":45472,"kb_used_data":2064,"kb_used_omap":1,"kb_used_meta":43390,"kb_avail":94326368,"statfs":{"total":96636764160,"available":96590200832,"internally_reserved":0,"allocated":2113536,"data_stored":1143936,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1781,"internal_metadata":44431627},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Wed Aug 9 14:51:07 2023","interfaces":[{"interface":"back","average":{"1min":0.878,"5min":0.94899999999999995,"15min":0.96699999999999997},"min":{"1min":0.71499999999999997,"5min":0.55000000000000004,"15min":0.55000000000000004},"max":{"1min":1.23,"5min":1.4650000000000001,"15min":1.893},"last":0.82999999999999996},{"interface":"front","average":{"1min":1.28,"5min":1.208,"15min":1.153},"min":{"1min":0.81399999999999995,"5min":0.70299999999999996,"15min":0.70299999999999996},"max":{"1min":1.6870000000000001,"5min":1.8520000000000001,"15min":1.8520000000000001},"last":0.71299999999999997}]},{"osd":2,"last update":"Wed Aug 9 14:51:07 2023","interfaces":[{"interface":"back","average":{"1min":1.171,"5min":1.1890000000000001,"15min":1.216},"min":{"1min":0.85599999999999998,"5min":0.65700000000000003,"15min":0.65700000000000003},"max":{"1min":1.6870000000000001,"5min":1.802,"15min":2.698},"last":0.96899999999999997},{"interface":"front","average":{"1min":1.248,"5min":1.2709999999999999,"15min":1.2390000000000001},"min":{"1min":0.94199999999999995,"5min":0.92400000000000004,"15min":0.82599999999999996},"max":{"1min":1.7170000000000001,"5min":1.895,"15min":1.895},"last":1.0309999999999999}]},{"osd":3,"last update":"Wed Aug 9 14:51:07 2023","interfaces":[{"interface":"back","average":{"1min":1.2250000000000001,"5min":1.2509999999999999,"15min":1.254},"min":{"1min":0.82199999999999995,"5min":0.68100000000000005,"15min":0.68100000000000005},"max":{"1min":1.74,"5min":2.0049999999999999,"15min":2.1230000000000002},"last":0.88700000000000001},{"interface":"front","average":{"1min":1.3959999999999999,"5min":1.4039999999999999,"15min":1.397},"min":{"1min":0.95399999999999996,"5min":0.95399999999999996,"15min":0.88800000000000001},"max":{"1min":1.748,"5min":1.988,"15min":2.3580000000000001},"last":1.3680000000000001}]},{"osd":4,"last update":"Wed Aug 9 14:51:07 2023","interfaces":[{"interface":"back","average":{"1min":1.3879999999999999,"5min":1.421,"15min":1.3899999999999999},"min":{"1min":1.022,"5min":0.96599999999999997,"15min":0.96099999999999997},"max":{"1min":1.9470000000000001,"5min":2.3730000000000002,"15min":2.3730000000000002},"last":1.018},{"interface":"front","average":{"1min":1.4630000000000001,"5min":1.5249999999999999,"15min":1.5169999999999999},"min":{"1min":1.137,"5min":1.137,"15min":1.052},"max":{"1min":1.758,"5min":2.125,"15min":2.149},"last":1.282}]},{"osd":5,"last update":"Wed Aug 9 14:51:07 2023","interfaces":[{"interface":"back","average":{"1min":1.518,"5min":1.5569999999999999,"15min":1.5269999999999999},"min":{"1min":1.127,"5min":0.93100000000000005,"15min":0.93100000000000005},"max":{"1min":1.972,"5min":2.0110000000000001,"15min":2.097},"last":1.153},{"interface":"front","average":{"1min":1.484,"5min":1.4770000000000001,"15min":1.5109999999999999},"min":{"1min":1.1739999999999999,"5min":1.075,"15min":0.99399999999999999},"max":{"1min":1.921,"5min":2.2320000000000002,"15min":2.266},"last":1.2370000000000001}]},{"osd":6,"last update":"Wed Aug 9 14:51:07 2023","interfaces":[{"interface":"back","average":{"1min":1.5660000000000001,"5min":1.6459999999999999,"15min":1.641},"min":{"1min":1.2190000000000001,"5min":0.997,"15min":0.997},"max":{"1min":2.3879999999999999,"5min":2.4260000000000002,"15min":3.0750000000000002},"last":1.464},{"interface":"front","average":{"1min":1.597,"5min":1.651,"15min":1.631},"min":{"1min":1.202,"5min":1.034,"15min":1.034},"max":{"1min":2.2429999999999999,"5min":2.2429999999999999,"15min":2.2429999999999999},"last":1.256}]},{"osd":7,"last update":"Wed Aug 9 14:51:07 2023","interfaces":[{"interface":"back","average":{"1min":1.7110000000000001,"5min":1.74,"15min":1.7090000000000001},"min":{"1min":1.1839999999999999,"5min":1.1839999999999999,"15min":1.1639999999999999},"max":{"1min":2.1259999999999999,"5min":2.2250000000000001,"15min":2.3490000000000002},"last":1.427},{"interface":"front","average":{"1min":1.6100000000000001,"5min":1.6339999999999999,"15min":1.6200000000000001},"min":{"1min":1.3460000000000001,"5min":1.0940000000000001,"15min":1.0940000000000001},"max":{"1min":2.0249999999999999,"5min":2.4500000000000002,"15min":2.468},"last":1.3220000000000001}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":20,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1646,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":20,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":20,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1682,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":20,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":20,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":20,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":22,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":20,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":146,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":20,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":652,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":21,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":21,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":21,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":21,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":21,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":21,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":21,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":21,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-08-09T14:51:52.856 INFO:tasks.cephfs.filesystem.ceph_manager:clean! 2023-08-09T14:51:52.857 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth caps client.0 mon 'allow r' osd 'allow rw tag cephfs data=cephfs' mds allow 2023-08-09T14:51:53.129 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:53.128+0000 7f08855e1640 1 Processor -- start 2023-08-09T14:51:53.130 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:53.128+0000 7f08855e1640 1 -- start start 2023-08-09T14:51:53.131 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:53.129+0000 7f08855e1640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f08800ce1c0 0x7f08800ce5c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T14:51:53.131 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:53.129+0000 7f08855e1640 1 -- --> v1:172.21.15.43:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f088015a4a0 con 0x7f088015a0c0 2023-08-09T14:51:53.131 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:53.129+0000 7f08855e1640 1 -- --> v1:172.21.15.203:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f08800ceb00 con 0x7f08800cbb80 2023-08-09T14:51:53.131 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:53.129+0000 7f08855e1640 1 -- --> v2:172.21.15.43:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f08800cc030 con 0x7f08800ce1c0 2023-08-09T14:51:53.132 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:53.130+0000 7f087effd640 1 --1- >> v1:172.21.15.43:6789/0 conn(0x7f088015a0c0 0x7f088015d510 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.43:6789/0 says I am v1:172.21.15.43:47322/0 (socket says 172.21.15.43:47322) 2023-08-09T14:51:53.132 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:53.130+0000 7f087effd640 1 -- 172.21.15.43:0/4130660539 learned_addr learned my addr 172.21.15.43:0/4130660539 (peer_addr_for_me v1:172.21.15.43:0/0) 2023-08-09T14:51:53.132 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:53.130+0000 7f087f7fe640 1 --2- 172.21.15.43:0/4130660539 >> v2:172.21.15.43:3300/0 conn(0x7f08800ce1c0 0x7f08800ce5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T14:51:53.132 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:53.130+0000 7f087f7fe640 1 -- 172.21.15.43:0/4130660539 >> v1:172.21.15.203:6789/0 conn(0x7f08800cbb80 legacy=0x7f08800cb2b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T14:51:53.132 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:53.130+0000 7f087dffb640 1 -- 172.21.15.43:0/4130660539 <== mon.0 v1:172.21.15.43:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 321209946 0 0) 0x7f088015a4a0 con 0x7f088015a0c0 2023-08-09T14:51:53.133 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:53.130+0000 7f087f7fe640 1 -- 172.21.15.43:0/4130660539 >> v1:172.21.15.43:6789/0 conn(0x7f088015a0c0 legacy=0x7f088015d510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T14:51:53.133 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:53.130+0000 7f087f7fe640 1 -- 172.21.15.43:0/4130660539 --> v2:172.21.15.43:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f08800cc2d0 con 0x7f08800ce1c0 2023-08-09T14:51:53.133 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:53.131+0000 7f087f7fe640 1 --2- 172.21.15.43:0/4130660539 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f08800ce1c0 0x7f08800ce5c0 secure :-1 s=READY pgs=1885 cs=0 l=1 rev1=1 crypto rx=0x7f08780097a0 tx=0x7f087802b960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=84b8526eaf78b31f server_cookie=0 in_seq=0 out_seq=0 2023-08-09T14:51:53.134 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:53.132+0000 7f087dffb640 1 -- 172.21.15.43:0/4130660539 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0878030610 con 0x7f08800ce1c0 2023-08-09T14:51:53.134 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:53.132+0000 7f087dffb640 1 -- 172.21.15.43:0/4130660539 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0878030770 con 0x7f08800ce1c0 2023-08-09T14:51:53.134 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:53.132+0000 7f087dffb640 1 -- 172.21.15.43:0/4130660539 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0878030b00 con 0x7f08800ce1c0 2023-08-09T14:51:53.134 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:53.132+0000 7f08855e1640 1 -- 172.21.15.43:0/4130660539 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f08800ce1c0 msgr2=0x7f08800ce5c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T14:51:53.134 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:53.132+0000 7f08855e1640 1 --2- 172.21.15.43:0/4130660539 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f08800ce1c0 0x7f08800ce5c0 secure :-1 s=READY pgs=1885 cs=0 l=1 rev1=1 crypto rx=0x7f08780097a0 tx=0x7f087802b960 comp rx=0 tx=0).stop 2023-08-09T14:51:53.135 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:53.133+0000 7f08855e1640 1 -- 172.21.15.43:0/4130660539 shutdown_connections 2023-08-09T14:51:53.135 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:53.133+0000 7f08855e1640 1 --2- 172.21.15.43:0/4130660539 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f08800ce1c0 0x7f08800ce5c0 unknown :-1 s=CLOSED pgs=1885 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T14:51:53.135 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:53.133+0000 7f08855e1640 1 -- 172.21.15.43:0/4130660539 >> 172.21.15.43:0/4130660539 conn(0x7f08800aff50 msgr2=0x7f08800b0360 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-09T14:51:53.135 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:53.133+0000 7f08855e1640 1 -- 172.21.15.43:0/4130660539 shutdown_connections 2023-08-09T14:51:53.136 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:53.133+0000 7f08855e1640 1 -- 172.21.15.43:0/4130660539 wait complete. 2023-08-09T14:51:53.136 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:53.134+0000 7f08855e1640 1 Processor -- start 2023-08-09T14:51:53.136 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:53.135+0000 7f08855e1640 1 -- start start 2023-08-09T14:51:53.137 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:53.136+0000 7f08855e1640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f08800cbb80 0x7f08800c3c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T14:51:53.137 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:53.136+0000 7f08855e1640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f08800ce1c0 0x7f08801570b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T14:51:53.138 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:53.136+0000 7f08855e1640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f088015a0c0 0x7f08801575f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T14:51:53.138 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:53.136+0000 7f08855e1640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0880157b30 con 0x7f08800cbb80 2023-08-09T14:51:53.138 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:53.136+0000 7f08855e1640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f08800c2890 con 0x7f088015a0c0 2023-08-09T14:51:53.138 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:53.136+0000 7f08855e1640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f08800c2b70 con 0x7f08800ce1c0 2023-08-09T14:51:53.138 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:53.136+0000 7f087effd640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f08800cbb80 0x7f08800c3c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T14:51:53.138 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:53.136+0000 7f087effd640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f08800cbb80 0x7f08800c3c00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:57978/0 (socket says 172.21.15.43:57978) 2023-08-09T14:51:53.138 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:53.136+0000 7f087effd640 1 -- 172.21.15.43:0/2284725377 learned_addr learned my addr 172.21.15.43:0/2284725377 (peer_addr_for_me v2:172.21.15.43:0/0) 2023-08-09T14:51:53.139 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:53.137+0000 7f087effd640 1 -- 172.21.15.43:0/2284725377 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f08800ce1c0 msgr2=0x7f08801570b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T14:51:53.140 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:53.137+0000 7f087e7fc640 1 --2- 172.21.15.43:0/2284725377 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f08800ce1c0 0x7f08801570b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T14:51:53.140 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:53.137+0000 7f087f7fe640 1 --2- 172.21.15.43:0/2284725377 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f088015a0c0 0x7f08801575f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T14:51:53.140 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:53.137+0000 7f087effd640 1 --2- 172.21.15.43:0/2284725377 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f08800ce1c0 0x7f08801570b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T14:51:53.140 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:53.137+0000 7f087effd640 1 -- 172.21.15.43:0/2284725377 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f088015a0c0 msgr2=0x7f08801575f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T14:51:53.140 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:53.137+0000 7f087effd640 1 --2- 172.21.15.43:0/2284725377 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f088015a0c0 0x7f08801575f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T14:51:53.140 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:53.137+0000 7f087effd640 1 -- 172.21.15.43:0/2284725377 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0878009400 con 0x7f08800cbb80 2023-08-09T14:51:53.141 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:53.137+0000 7f087e7fc640 1 --2- 172.21.15.43:0/2284725377 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f08800ce1c0 0x7f08801570b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-08-09T14:51:53.141 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:53.137+0000 7f087f7fe640 1 --2- 172.21.15.43:0/2284725377 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f088015a0c0 0x7f08801575f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-08-09T14:51:53.141 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:53.137+0000 7f087effd640 1 --2- 172.21.15.43:0/2284725377 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f08800cbb80 0x7f08800c3c00 secure :-1 s=READY pgs=1886 cs=0 l=1 rev1=1 crypto rx=0x7f087000c8c0 tx=0x7f087000cd80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-09T14:51:53.141 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:53.137+0000 7f08677fe640 1 -- 172.21.15.43:0/2284725377 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f087000a640 con 0x7f08800cbb80 2023-08-09T14:51:53.141 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:53.137+0000 7f08677fe640 1 -- 172.21.15.43:0/2284725377 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f087000a7a0 con 0x7f08800cbb80 2023-08-09T14:51:53.141 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:53.137+0000 7f08677fe640 1 -- 172.21.15.43:0/2284725377 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f087000ab90 con 0x7f08800cbb80 2023-08-09T14:51:53.142 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:53.137+0000 7f08855e1640 1 -- 172.21.15.43:0/2284725377 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f08800c2e50 con 0x7f08800cbb80 2023-08-09T14:51:53.142 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:53.138+0000 7f08855e1640 1 -- 172.21.15.43:0/2284725377 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f08800c2520 con 0x7f08800cbb80 2023-08-09T14:51:53.142 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:53.139+0000 7f08855e1640 1 -- 172.21.15.43:0/2284725377 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f08800739f0 con 0x7f08800cbb80 2023-08-09T14:51:53.142 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:53.140+0000 7f08677fe640 1 -- 172.21.15.43:0/2284725377 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 79) v1 ==== 88443+0+0 (secure 0 0 0) 0x7f087000acf0 con 0x7f08800cbb80 2023-08-09T14:51:53.143 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:53.141+0000 7f08677fe640 1 --2- 172.21.15.43:0/2284725377 >> [v2:172.21.15.203:6832/2566816332,v1:172.21.15.203:6833/2566816332] conn(0x7f086c06a170 0x7f086c06c630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T14:51:53.143 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:53.141+0000 7f08677fe640 1 -- 172.21.15.43:0/2284725377 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(172..172 src has 1..172) v4 ==== 6426+0+0 (secure 0 0 0) 0x7f0870087ee0 con 0x7f08800cbb80 2023-08-09T14:51:53.143 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:53.141+0000 7f087e7fc640 1 --2- 172.21.15.43:0/2284725377 >> [v2:172.21.15.203:6832/2566816332,v1:172.21.15.203:6833/2566816332] conn(0x7f086c06a170 0x7f086c06c630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T14:51:53.144 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:53.143+0000 7f087e7fc640 1 --2- 172.21.15.43:0/2284725377 >> [v2:172.21.15.203:6832/2566816332,v1:172.21.15.203:6833/2566816332] conn(0x7f086c06a170 0x7f086c06c630 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f08800aed90 tx=0x7f0874009040 comp rx=0 tx=0).ready entity=mgr.7765 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-09T14:51:53.147 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:53.145+0000 7f08677fe640 1 -- 172.21.15.43:0/2284725377 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178066 (secure 0 0 0) 0x7f0870053340 con 0x7f08800cbb80 2023-08-09T14:51:53.408 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:53.407+0000 7f08855e1640 1 -- 172.21.15.43:0/2284725377 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]} v 0) v1 -- 0x7f088006a3d0 con 0x7f08800cbb80 2023-08-09T14:51:53.417 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:53.415+0000 7f08677fe640 1 -- 172.21.15.43:0/2284725377 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]}]=0 updated caps for client.0 v40) v1 ==== 190+0+147 (secure 0 0 0) 0x7f0870018070 con 0x7f08800cbb80 2023-08-09T14:51:53.417 INFO:teuthology.orchestra.run.smithi043.stdout:[client.0] 2023-08-09T14:51:53.417 INFO:teuthology.orchestra.run.smithi043.stdout: key = AQCPldNkg8zrDxAAM/hnJaE6S9HU/fRwQeBV5w== 2023-08-09T14:51:53.418 INFO:teuthology.orchestra.run.smithi043.stdout: caps mds = "allow" 2023-08-09T14:51:53.418 INFO:teuthology.orchestra.run.smithi043.stdout: caps mon = "allow r" 2023-08-09T14:51:53.418 INFO:teuthology.orchestra.run.smithi043.stdout: caps osd = "allow rw tag cephfs data=cephfs" 2023-08-09T14:51:53.418 INFO:teuthology.orchestra.run.smithi043.stderr:updated caps for client.0 2023-08-09T14:51:53.419 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:53.418+0000 7f08855e1640 1 -- 172.21.15.43:0/2284725377 >> [v2:172.21.15.203:6832/2566816332,v1:172.21.15.203:6833/2566816332] conn(0x7f086c06a170 msgr2=0x7f086c06c630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T14:51:53.419 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:53.418+0000 7f08855e1640 1 --2- 172.21.15.43:0/2284725377 >> [v2:172.21.15.203:6832/2566816332,v1:172.21.15.203:6833/2566816332] conn(0x7f086c06a170 0x7f086c06c630 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f08800aed90 tx=0x7f0874009040 comp rx=0 tx=0).stop 2023-08-09T14:51:53.419 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:53.418+0000 7f08855e1640 1 -- 172.21.15.43:0/2284725377 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f08800cbb80 msgr2=0x7f08800c3c00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T14:51:53.420 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:53.418+0000 7f08855e1640 1 --2- 172.21.15.43:0/2284725377 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f08800cbb80 0x7f08800c3c00 secure :-1 s=READY pgs=1886 cs=0 l=1 rev1=1 crypto rx=0x7f087000c8c0 tx=0x7f087000cd80 comp rx=0 tx=0).stop 2023-08-09T14:51:53.420 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:53.418+0000 7f08855e1640 1 -- 172.21.15.43:0/2284725377 shutdown_connections 2023-08-09T14:51:53.421 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:53.418+0000 7f08855e1640 1 --2- 172.21.15.43:0/2284725377 >> [v2:172.21.15.203:6832/2566816332,v1:172.21.15.203:6833/2566816332] conn(0x7f086c06a170 0x7f086c06c630 unknown :-1 s=CLOSED pgs=67 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T14:51:53.421 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:53.418+0000 7f08855e1640 1 --2- 172.21.15.43:0/2284725377 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f088015a0c0 0x7f08801575f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T14:51:53.421 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:53.418+0000 7f08855e1640 1 --2- 172.21.15.43:0/2284725377 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f08800ce1c0 0x7f08801570b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T14:51:53.421 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:53.418+0000 7f08855e1640 1 --2- 172.21.15.43:0/2284725377 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f08800cbb80 0x7f08800c3c00 unknown :-1 s=CLOSED pgs=1886 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T14:51:53.421 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:53.418+0000 7f08855e1640 1 -- 172.21.15.43:0/2284725377 >> 172.21.15.43:0/2284725377 conn(0x7f08800aff50 msgr2=0x7f0880159620 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-09T14:51:53.421 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:53.418+0000 7f08855e1640 1 -- 172.21.15.43:0/2284725377 shutdown_connections 2023-08-09T14:51:53.421 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:53.418+0000 7f08855e1640 1 -- 172.21.15.43:0/2284725377 wait complete. 2023-08-09T14:51:53.438 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-08-09T14:51:53.670 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:53.669+0000 7f4a28b53640 1 Processor -- start 2023-08-09T14:51:53.670 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:53.669+0000 7f4a28b53640 1 -- start start 2023-08-09T14:51:53.671 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:53.670+0000 7f4a28b53640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f4a2415f7b0 0x7f4a240b4760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T14:51:53.671 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:53.670+0000 7f4a28b53640 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7f4a240cd6b0 0x7f4a240d0e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T14:51:53.672 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:53.670+0000 7f4a28b53640 1 -- --> v1:172.21.15.43:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4a240b7bf0 con 0x7f4a240b1050 2023-08-09T14:51:53.672 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:53.670+0000 7f4a28b53640 1 -- --> v2:172.21.15.203:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4a240b7ed0 con 0x7f4a240cd6b0 2023-08-09T14:51:53.672 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:53.670+0000 7f4a28b53640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4a240d0370 con 0x7f4a2415f7b0 2023-08-09T14:51:53.673 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:53.670+0000 7f4a1affd640 1 --1- >> v1:172.21.15.43:6789/0 conn(0x7f4a240b1050 0x7f4a240ccfa0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.43:6789/0 says I am v1:172.21.15.43:47328/0 (socket says 172.21.15.43:47328) 2023-08-09T14:51:53.673 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:53.670+0000 7f4a1affd640 1 -- 172.21.15.43:0/696518010 learned_addr learned my addr 172.21.15.43:0/696518010 (peer_addr_for_me v1:172.21.15.43:0/0) 2023-08-09T14:51:53.673 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:53.671+0000 7f4a237fe640 1 --2- 172.21.15.43:0/696518010 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f4a2415f7b0 0x7f4a240b4760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T14:51:53.674 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:53.671+0000 7f4a23fff640 1 --2- 172.21.15.43:0/696518010 >> v2:172.21.15.203:3300/0 conn(0x7f4a240cd6b0 0x7f4a240d0e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T14:51:53.674 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:53.671+0000 7f4a22ffd640 1 -- 172.21.15.43:0/696518010 <== mon.0 v1:172.21.15.43:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4045701700 0 0) 0x7f4a240b7bf0 con 0x7f4a240b1050 2023-08-09T14:51:53.674 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:53.671+0000 7f4a22ffd640 1 -- 172.21.15.43:0/696518010 --> v1:172.21.15.43:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f4a10003520 con 0x7f4a240b1050 2023-08-09T14:51:53.674 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:53.671+0000 7f4a22ffd640 1 -- 172.21.15.43:0/696518010 <== mon.0 v1:172.21.15.43:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 4156128959 0 0) 0x7f4a10003520 con 0x7f4a240b1050 2023-08-09T14:51:53.674 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:53.671+0000 7f4a22ffd640 1 -- 172.21.15.43:0/696518010 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f4a2415f7b0 msgr2=0x7f4a240b4760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-09T14:51:53.674 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:53.671+0000 7f4a22ffd640 1 --2- 172.21.15.43:0/696518010 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f4a2415f7b0 0x7f4a240b4760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T14:51:53.674 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:53.671+0000 7f4a22ffd640 1 -- 172.21.15.43:0/696518010 >> v2:172.21.15.203:3300/0 conn(0x7f4a240cd6b0 msgr2=0x7f4a240d0e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-09T14:51:53.674 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:53.671+0000 7f4a22ffd640 1 --2- 172.21.15.43:0/696518010 >> v2:172.21.15.203:3300/0 conn(0x7f4a240cd6b0 0x7f4a240d0e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T14:51:53.675 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:53.671+0000 7f4a22ffd640 1 -- 172.21.15.43:0/696518010 --> v1:172.21.15.43:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4a240d0650 con 0x7f4a240b1050 2023-08-09T14:51:53.675 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:53.671+0000 7f4a22ffd640 1 -- 172.21.15.43:0/696518010 <== mon.0 v1:172.21.15.43:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1285150740 0 0) 0x7f4a0c001670 con 0x7f4a240b1050 2023-08-09T14:51:53.675 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:53.671+0000 7f4a22ffd640 1 -- 172.21.15.43:0/696518010 >> v1:172.21.15.43:6789/0 conn(0x7f4a240b1050 legacy=0x7f4a240ccfa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T14:51:53.675 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:53.671+0000 7f4a22ffd640 1 --2- 172.21.15.43:0/696518010 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f4a10003970 0x7f4a10003d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T14:51:53.675 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:53.671+0000 7f4a22ffd640 1 --2- 172.21.15.43:0/696518010 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f4a100047c0 0x7f4a10004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T14:51:53.675 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:53.672+0000 7f4a22ffd640 1 --2- 172.21.15.43:0/696518010 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f4a2415f7b0 0x7f4a10005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T14:51:53.675 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:53.672+0000 7f4a22ffd640 1 -- 172.21.15.43:0/696518010 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4a240b7ed0 con 0x7f4a10003970 2023-08-09T14:51:53.676 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:53.672+0000 7f4a22ffd640 1 -- 172.21.15.43:0/696518010 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4a240d0370 con 0x7f4a100047c0 2023-08-09T14:51:53.676 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:53.672+0000 7f4a22ffd640 1 -- 172.21.15.43:0/696518010 --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4a1000b980 con 0x7f4a2415f7b0 2023-08-09T14:51:53.676 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:53.672+0000 7f4a237fe640 1 --2- 172.21.15.43:0/696518010 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f4a10003970 0x7f4a10003d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T14:51:53.677 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:53.672+0000 7f4a1affd640 1 --2- 172.21.15.43:0/696518010 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f4a100047c0 0x7f4a10004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T14:51:53.677 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:53.672+0000 7f4a23fff640 1 --2- 172.21.15.43:0/696518010 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f4a2415f7b0 0x7f4a10005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T14:51:53.677 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:53.672+0000 7f4a237fe640 1 -- 172.21.15.43:0/696518010 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f4a2415f7b0 msgr2=0x7f4a10005180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-09T14:51:53.677 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:53.672+0000 7f4a237fe640 1 --2- 172.21.15.43:0/696518010 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f4a2415f7b0 0x7f4a10005180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T14:51:53.678 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:53.672+0000 7f4a237fe640 1 -- 172.21.15.43:0/696518010 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f4a100047c0 msgr2=0x7f4a10004c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-09T14:51:53.678 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:53.672+0000 7f4a237fe640 1 --2- 172.21.15.43:0/696518010 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f4a100047c0 0x7f4a10004c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T14:51:53.678 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:53.672+0000 7f4a237fe640 1 -- 172.21.15.43:0/696518010 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4a10003520 con 0x7f4a10003970 2023-08-09T14:51:53.678 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:53.672+0000 7f4a23fff640 1 --2- 172.21.15.43:0/696518010 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f4a2415f7b0 0x7f4a10005180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-08-09T14:51:53.678 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:53.673+0000 7f4a237fe640 1 --2- 172.21.15.43:0/696518010 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f4a10003970 0x7f4a10003d90 secure :-1 s=READY pgs=1888 cs=0 l=1 rev1=1 crypto rx=0x7f4a14009ac0 tx=0x7f4a100057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=da5b3e3ccf3ca51a server_cookie=0 in_seq=0 out_seq=0 2023-08-09T14:51:53.678 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:53.674+0000 7f4a22ffd640 1 -- 172.21.15.43:0/696518010 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4a1402c6b0 con 0x7f4a10003970 2023-08-09T14:51:53.678 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:53.674+0000 7f4a22ffd640 1 -- 172.21.15.43:0/696518010 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4a1402c810 con 0x7f4a10003970 2023-08-09T14:51:53.679 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:53.674+0000 7f4a22ffd640 1 -- 172.21.15.43:0/696518010 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4a1402cba0 con 0x7f4a10003970 2023-08-09T14:51:53.679 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:53.674+0000 7f4a28b53640 1 -- 172.21.15.43:0/696518010 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f4a10003970 msgr2=0x7f4a10003d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T14:51:53.679 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:53.674+0000 7f4a28b53640 1 --2- 172.21.15.43:0/696518010 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f4a10003970 0x7f4a10003d90 secure :-1 s=READY pgs=1888 cs=0 l=1 rev1=1 crypto rx=0x7f4a14009ac0 tx=0x7f4a100057f0 comp rx=0 tx=0).stop 2023-08-09T14:51:53.679 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:53.674+0000 7f4a23fff640 1 -- 172.21.15.43:0/696518010 reap_dead start 2023-08-09T14:51:53.679 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:53.674+0000 7f4a28b53640 1 -- 172.21.15.43:0/696518010 shutdown_connections 2023-08-09T14:51:53.679 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:53.675+0000 7f4a28b53640 1 -- 172.21.15.43:0/696518010 >> 172.21.15.43:0/696518010 conn(0x7f4a2406a2b0 msgr2=0x7f4a2406a690 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-09T14:51:53.679 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:53.675+0000 7f4a28b53640 1 -- 172.21.15.43:0/696518010 shutdown_connections 2023-08-09T14:51:53.680 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:53.675+0000 7f4a28b53640 1 -- 172.21.15.43:0/696518010 wait complete. 2023-08-09T14:51:53.681 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:53.676+0000 7f4a28b53640 1 Processor -- start 2023-08-09T14:51:53.681 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:53.677+0000 7f4a28b53640 1 -- start start 2023-08-09T14:51:53.681 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:53.677+0000 7f4a28b53640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f4a10003970 0x7f4a240c7bc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T14:51:53.681 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:53.677+0000 7f4a28b53640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f4a241e5920 0x7f4a2415dfb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T14:51:53.682 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:53.677+0000 7f4a28b53640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f4a241e5da0 0x7f4a2415e4f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T14:51:53.682 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:53.677+0000 7f4a28b53640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4a24152fc0 con 0x7f4a241e5da0 2023-08-09T14:51:53.682 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:53.677+0000 7f4a28b53640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4a241e68c0 con 0x7f4a10003970 2023-08-09T14:51:53.682 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:53.677+0000 7f4a28b53640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4a241e6300 con 0x7f4a241e5920 2023-08-09T14:51:53.682 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:53.678+0000 7f4a23fff640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f4a241e5da0 0x7f4a2415e4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T14:51:53.682 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:53.678+0000 7f4a23fff640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f4a241e5da0 0x7f4a2415e4f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:57996/0 (socket says 172.21.15.43:57996) 2023-08-09T14:51:53.682 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:53.678+0000 7f4a23fff640 1 -- 172.21.15.43:0/4287411268 learned_addr learned my addr 172.21.15.43:0/4287411268 (peer_addr_for_me v2:172.21.15.43:0/0) 2023-08-09T14:51:53.682 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:53.678+0000 7f4a23fff640 1 -- 172.21.15.43:0/4287411268 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f4a241e5920 msgr2=0x7f4a2415dfb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T14:51:53.683 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:53.678+0000 7f4a23fff640 1 --2- 172.21.15.43:0/4287411268 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f4a241e5920 0x7f4a2415dfb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T14:51:53.683 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:53.678+0000 7f4a23fff640 1 -- 172.21.15.43:0/4287411268 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f4a10003970 msgr2=0x7f4a240c7bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T14:51:53.683 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:53.678+0000 7f4a237fe640 1 --2- 172.21.15.43:0/4287411268 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f4a10003970 0x7f4a240c7bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T14:51:53.683 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:53.678+0000 7f4a23fff640 1 --2- 172.21.15.43:0/4287411268 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f4a10003970 0x7f4a240c7bc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T14:51:53.683 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:53.678+0000 7f4a23fff640 1 -- 172.21.15.43:0/4287411268 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4a241e65e0 con 0x7f4a241e5da0 2023-08-09T14:51:53.683 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:53.679+0000 7f4a23fff640 1 --2- 172.21.15.43:0/4287411268 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f4a241e5da0 0x7f4a2415e4f0 secure :-1 s=READY pgs=1889 cs=0 l=1 rev1=1 crypto rx=0x7f4a240c9310 tx=0x7f4a1c00f4a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-09T14:51:53.683 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:53.679+0000 7f4a20ff9640 1 -- 172.21.15.43:0/4287411268 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4a1c017400 con 0x7f4a241e5da0 2023-08-09T14:51:53.684 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:53.679+0000 7f4a20ff9640 1 -- 172.21.15.43:0/4287411268 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4a1c011040 con 0x7f4a241e5da0 2023-08-09T14:51:53.684 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:53.679+0000 7f4a20ff9640 1 -- 172.21.15.43:0/4287411268 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4a1c0177c0 con 0x7f4a241e5da0 2023-08-09T14:51:53.684 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:53.679+0000 7f4a28b53640 1 -- 172.21.15.43:0/4287411268 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4a24152c50 con 0x7f4a241e5da0 2023-08-09T14:51:53.684 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:53.679+0000 7f4a28b53640 1 -- 172.21.15.43:0/4287411268 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4a24151b50 con 0x7f4a241e5da0 2023-08-09T14:51:53.685 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:53.680+0000 7f4a28b53640 1 -- 172.21.15.43:0/4287411268 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4a24152de0 con 0x7f4a241e5da0 2023-08-09T14:51:53.685 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:53.681+0000 7f4a20ff9640 1 -- 172.21.15.43:0/4287411268 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 79) v1 ==== 88443+0+0 (secure 0 0 0) 0x7f4a1c0203b0 con 0x7f4a241e5da0 2023-08-09T14:51:53.685 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:53.682+0000 7f4a20ff9640 1 --2- 172.21.15.43:0/4287411268 >> [v2:172.21.15.203:6832/2566816332,v1:172.21.15.203:6833/2566816332] conn(0x7f4a0406a170 0x7f4a0406c630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T14:51:53.685 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:53.682+0000 7f4a20ff9640 1 -- 172.21.15.43:0/4287411268 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(172..172 src has 1..172) v4 ==== 6426+0+0 (secure 0 0 0) 0x7f4a1c088c90 con 0x7f4a241e5da0 2023-08-09T14:51:53.685 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:53.683+0000 7f4a237fe640 1 --2- 172.21.15.43:0/4287411268 >> [v2:172.21.15.203:6832/2566816332,v1:172.21.15.203:6833/2566816332] conn(0x7f4a0406a170 0x7f4a0406c630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T14:51:53.686 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:53.684+0000 7f4a237fe640 1 --2- 172.21.15.43:0/4287411268 >> [v2:172.21.15.203:6832/2566816332,v1:172.21.15.203:6833/2566816332] conn(0x7f4a0406a170 0x7f4a0406c630 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f4a14036000 tx=0x7f4a14030000 comp rx=0 tx=0).ready entity=mgr.7765 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-09T14:51:53.688 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:53.687+0000 7f4a20ff9640 1 -- 172.21.15.43:0/4287411268 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178066 (secure 0 0 0) 0x7f4a1c053f50 con 0x7f4a241e5da0 2023-08-09T14:51:53.924 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:53.923+0000 7f4a28b53640 1 -- 172.21.15.43:0/4287411268 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f4a240739f0 con 0x7f4a241e5da0 2023-08-09T14:51:53.926 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:53.924+0000 7f4a20ff9640 1 -- 172.21.15.43:0/4287411268 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 78 v78) v1 ==== 94+0+4880 (secure 0 0 0) 0x7f4a1c057570 con 0x7f4a241e5da0 2023-08-09T14:51:53.926 INFO:teuthology.orchestra.run.smithi043.stdout: 2023-08-09T14:51:53.926 INFO:teuthology.orchestra.run.smithi043.stdout:{"epoch":78,"default_fscid":9,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":7079,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.203:6836/807059349","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6834","nonce":807059349},{"type":"v1","addr":"172.21.15.203:6836","nonce":807059349}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":74},{"gid":7924,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.203:6837/3166829033","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6835","nonce":3166829033},{"type":"v1","addr":"172.21.15.203:6837","nonce":3166829033}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":74},{"gid":9642,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.43:6833/3803974919","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6832","nonce":3803974919},{"type":"v1","addr":"172.21.15.43:6833","nonce":3803974919}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":73}],"filesystems":[{"mdsmap":{"epoch":78,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-08-09T14:51:47.253704+0000","modified":"2023-08-09T14:51:50.806331+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":9648},"failed":[],"damaged":[],"stopped":[],"info":{"gid_9648":{"gid":9648,"name":"a","rank":0,"incarnation":75,"state":"up:active","state_seq":3,"addr":"172.21.15.43:6835/2435237803","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6834","nonce":2435237803},{"type":"v1","addr":"172.21.15.43:6835","nonce":2435237803}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[21],"metadata_pool":20,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":9}]} 2023-08-09T14:51:53.926 INFO:teuthology.orchestra.run.smithi043.stderr:dumped fsmap epoch 78 2023-08-09T14:51:53.928 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:53.927+0000 7f4a28b53640 1 -- 172.21.15.43:0/4287411268 >> [v2:172.21.15.203:6832/2566816332,v1:172.21.15.203:6833/2566816332] conn(0x7f4a0406a170 msgr2=0x7f4a0406c630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T14:51:53.928 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:53.927+0000 7f4a28b53640 1 --2- 172.21.15.43:0/4287411268 >> [v2:172.21.15.203:6832/2566816332,v1:172.21.15.203:6833/2566816332] conn(0x7f4a0406a170 0x7f4a0406c630 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f4a14036000 tx=0x7f4a14030000 comp rx=0 tx=0).stop 2023-08-09T14:51:53.929 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:53.927+0000 7f4a28b53640 1 -- 172.21.15.43:0/4287411268 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f4a241e5da0 msgr2=0x7f4a2415e4f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T14:51:53.929 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:53.927+0000 7f4a28b53640 1 --2- 172.21.15.43:0/4287411268 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f4a241e5da0 0x7f4a2415e4f0 secure :-1 s=READY pgs=1889 cs=0 l=1 rev1=1 crypto rx=0x7f4a240c9310 tx=0x7f4a1c00f4a0 comp rx=0 tx=0).stop 2023-08-09T14:51:53.929 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:53.927+0000 7f4a28b53640 1 -- 172.21.15.43:0/4287411268 shutdown_connections 2023-08-09T14:51:53.930 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:53.927+0000 7f4a28b53640 1 --2- 172.21.15.43:0/4287411268 >> [v2:172.21.15.203:6832/2566816332,v1:172.21.15.203:6833/2566816332] conn(0x7f4a0406a170 0x7f4a0406c630 unknown :-1 s=CLOSED pgs=68 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T14:51:53.930 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:53.927+0000 7f4a28b53640 1 --2- 172.21.15.43:0/4287411268 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f4a241e5da0 0x7f4a2415e4f0 unknown :-1 s=CLOSED pgs=1889 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T14:51:53.930 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:53.927+0000 7f4a28b53640 1 --2- 172.21.15.43:0/4287411268 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f4a241e5920 0x7f4a2415dfb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T14:51:53.930 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:53.927+0000 7f4a28b53640 1 --2- 172.21.15.43:0/4287411268 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f4a10003970 0x7f4a240c7bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T14:51:53.930 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:53.927+0000 7f4a28b53640 1 -- 172.21.15.43:0/4287411268 >> 172.21.15.43:0/4287411268 conn(0x7f4a2406a2b0 msgr2=0x7f4a240b59e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-09T14:51:53.930 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:53.927+0000 7f4a28b53640 1 -- 172.21.15.43:0/4287411268 shutdown_connections 2023-08-09T14:51:53.930 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:53.927+0000 7f4a28b53640 1 -- 172.21.15.43:0/4287411268 wait complete. 2023-08-09T14:51:54.348 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 78, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2023-08-09T14:51:47.253704+0000', 'modified': '2023-08-09T14:51:50.806331+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 9648}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_9648': {'gid': 9648, 'name': 'a', 'rank': 0, 'incarnation': 75, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.43:6835/2435237803', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.43:6834', 'nonce': 2435237803}, {'type': 'v1', 'addr': '172.21.15.43:6835', 'nonce': 2435237803}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138322906710015, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}}, 'data_pools': [21], 'metadata_pool': 20, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0} 2023-08-09T14:51:54.349 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2023-08-09T14:51:54.349 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.a status 2023-08-09T14:51:54.595 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:54.593+0000 7f4593fab640 1 Processor -- start 2023-08-09T14:51:54.595 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:54.593+0000 7f4593fab640 1 -- start start 2023-08-09T14:51:54.596 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:54.595+0000 7f4593fab640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f458c15c680 0x7f458c0ce760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T14:51:54.597 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:54.595+0000 7f4593fab640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f458c0d0a70 0x7f458c0cd7f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T14:51:54.597 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:54.595+0000 7f4593fab640 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7f458c0cdd30 0x7f458c0b3fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T14:51:54.597 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:54.595+0000 7f4593fab640 1 -- --> v2:172.21.15.43:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f458c0d0f70 con 0x7f458c0d0a70 2023-08-09T14:51:54.598 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:54.595+0000 7f4593fab640 1 -- --> v2:172.21.15.203:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f458c0ccd10 con 0x7f458c0cdd30 2023-08-09T14:51:54.598 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:54.595+0000 7f4593fab640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f458c0ccff0 con 0x7f458c15c680 2023-08-09T14:51:54.599 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:54.595+0000 7f4591546640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f458c0d0a70 0x7f458c0cd7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T14:51:54.599 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:54.595+0000 7f4591546640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f458c0d0a70 0x7f458c0cd7f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:58006/0 (socket says 172.21.15.43:58006) 2023-08-09T14:51:54.600 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:54.595+0000 7f4591546640 1 -- 172.21.15.43:0/37894157 learned_addr learned my addr 172.21.15.43:0/37894157 (peer_addr_for_me v2:172.21.15.43:0/0) 2023-08-09T14:51:54.600 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:54.596+0000 7f4591d47640 1 --2- 172.21.15.43:0/37894157 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f458c15c680 0x7f458c0ce760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T14:51:54.600 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:54.596+0000 7f4592548640 1 --2- 172.21.15.43:0/37894157 >> v2:172.21.15.203:3300/0 conn(0x7f458c0cdd30 0x7f458c0b3fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T14:51:54.600 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:54.596+0000 7f4591546640 1 -- 172.21.15.43:0/37894157 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f458c15c680 msgr2=0x7f458c0ce760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-09T14:51:54.600 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:54.596+0000 7f4591546640 1 --2- 172.21.15.43:0/37894157 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f458c15c680 0x7f458c0ce760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T14:51:54.601 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:54.596+0000 7f4591546640 1 -- 172.21.15.43:0/37894157 >> v2:172.21.15.203:3300/0 conn(0x7f458c0cdd30 msgr2=0x7f458c0b3fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-09T14:51:54.601 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:54.596+0000 7f4591546640 1 --2- 172.21.15.43:0/37894157 >> v2:172.21.15.203:3300/0 conn(0x7f458c0cdd30 0x7f458c0b3fa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T14:51:54.601 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:54.596+0000 7f4591546640 1 -- 172.21.15.43:0/37894157 --> v2:172.21.15.43:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f458c0cd2d0 con 0x7f458c0d0a70 2023-08-09T14:51:54.601 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:54.596+0000 7f4592548640 1 --2- 172.21.15.43:0/37894157 >> v2:172.21.15.203:3300/0 conn(0x7f458c0cdd30 0x7f458c0b3fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-08-09T14:51:54.601 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:54.598+0000 7f4591546640 1 --2- 172.21.15.43:0/37894157 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f458c0d0a70 0x7f458c0cd7f0 secure :-1 s=READY pgs=1890 cs=0 l=1 rev1=1 crypto rx=0x7f458c0d3370 tx=0x7f457c02b910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9a683fca7ac979f6 server_cookie=0 in_seq=0 out_seq=0 2023-08-09T14:51:54.602 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:54.598+0000 7f4590d45640 1 -- 172.21.15.43:0/37894157 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f457c031870 con 0x7f458c0d0a70 2023-08-09T14:51:54.603 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:54.598+0000 7f4590d45640 1 -- 172.21.15.43:0/37894157 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f457c0319d0 con 0x7f458c0d0a70 2023-08-09T14:51:54.603 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:54.598+0000 7f4590d45640 1 -- 172.21.15.43:0/37894157 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f457c031d60 con 0x7f458c0d0a70 2023-08-09T14:51:54.603 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:54.599+0000 7f4593fab640 1 -- 172.21.15.43:0/37894157 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f458c0d0a70 msgr2=0x7f458c0cd7f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T14:51:54.603 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:54.599+0000 7f4593fab640 1 --2- 172.21.15.43:0/37894157 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f458c0d0a70 0x7f458c0cd7f0 secure :-1 s=READY pgs=1890 cs=0 l=1 rev1=1 crypto rx=0x7f458c0d3370 tx=0x7f457c02b910 comp rx=0 tx=0).stop 2023-08-09T14:51:54.603 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:54.599+0000 7f4593fab640 1 -- 172.21.15.43:0/37894157 shutdown_connections 2023-08-09T14:51:54.603 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:54.599+0000 7f4593fab640 1 --2- 172.21.15.43:0/37894157 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f458c0d0a70 0x7f458c0cd7f0 unknown :-1 s=CLOSED pgs=1890 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T14:51:54.604 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:54.599+0000 7f4593fab640 1 --2- 172.21.15.43:0/37894157 >> v2:172.21.15.203:3300/0 conn(0x7f458c0cdd30 0x7f458c0b3fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T14:51:54.604 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:54.599+0000 7f4593fab640 1 --2- 172.21.15.43:0/37894157 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f458c15c680 0x7f458c0ce760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T14:51:54.604 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:54.599+0000 7f4593fab640 1 -- 172.21.15.43:0/37894157 >> 172.21.15.43:0/37894157 conn(0x7f458c071520 msgr2=0x7f458c0d2520 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-09T14:51:54.604 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:54.599+0000 7f4593fab640 1 -- 172.21.15.43:0/37894157 shutdown_connections 2023-08-09T14:51:54.604 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:54.599+0000 7f4593fab640 1 -- 172.21.15.43:0/37894157 wait complete. 2023-08-09T14:51:54.604 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:54.601+0000 7f4593fab640 1 Processor -- start 2023-08-09T14:51:54.604 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:54.601+0000 7f4593fab640 1 -- start start 2023-08-09T14:51:54.606 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:54.602+0000 7f4593fab640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f458c0d0a70 0x7f458c0ca2c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T14:51:54.606 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:54.602+0000 7f4593fab640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f458c15c680 0x7f458c0cb070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T14:51:54.606 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:54.602+0000 7f4593fab640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f458c0b6910 0x7f458c153530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T14:51:54.606 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:54.602+0000 7f4593fab640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f458c0cb5b0 con 0x7f458c0d0a70 2023-08-09T14:51:54.607 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:54.602+0000 7f4593fab640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f458c1f1960 con 0x7f458c15c680 2023-08-09T14:51:54.607 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:54.602+0000 7f4593fab640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f458c15cb00 con 0x7f458c0b6910 2023-08-09T14:51:54.607 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:54.603+0000 7f4591d47640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f458c0d0a70 0x7f458c0ca2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T14:51:54.607 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:54.603+0000 7f4591d47640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f458c0d0a70 0x7f458c0ca2c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:58016/0 (socket says 172.21.15.43:58016) 2023-08-09T14:51:54.607 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:54.603+0000 7f4591d47640 1 -- 172.21.15.43:0/1999881876 learned_addr learned my addr 172.21.15.43:0/1999881876 (peer_addr_for_me v2:172.21.15.43:0/0) 2023-08-09T14:51:54.607 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:54.603+0000 7f4591546640 1 --2- 172.21.15.43:0/1999881876 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f458c15c680 0x7f458c0cb070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T14:51:54.607 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:54.603+0000 7f4592548640 1 --2- 172.21.15.43:0/1999881876 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f458c0b6910 0x7f458c153530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T14:51:54.607 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:54.603+0000 7f4591d47640 1 -- 172.21.15.43:0/1999881876 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f458c0b6910 msgr2=0x7f458c153530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T14:51:54.608 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:54.603+0000 7f4591d47640 1 --2- 172.21.15.43:0/1999881876 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f458c0b6910 0x7f458c153530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T14:51:54.608 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:54.603+0000 7f4591d47640 1 -- 172.21.15.43:0/1999881876 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f458c15c680 msgr2=0x7f458c0cb070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T14:51:54.608 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:54.603+0000 7f4591d47640 1 --2- 172.21.15.43:0/1999881876 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f458c15c680 0x7f458c0cb070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T14:51:54.608 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:54.603+0000 7f4591d47640 1 -- 172.21.15.43:0/1999881876 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f457c0094d0 con 0x7f458c0d0a70 2023-08-09T14:51:54.608 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:54.603+0000 7f4592548640 1 --2- 172.21.15.43:0/1999881876 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f458c0b6910 0x7f458c153530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-08-09T14:51:54.608 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:54.603+0000 7f4591d47640 1 --2- 172.21.15.43:0/1999881876 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f458c0d0a70 0x7f458c0ca2c0 secure :-1 s=READY pgs=1891 cs=0 l=1 rev1=1 crypto rx=0x7f458800d5f0 tx=0x7f458800dab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-09T14:51:54.608 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:54.604+0000 7f45827fc640 1 -- 172.21.15.43:0/1999881876 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f45880125f0 con 0x7f458c0d0a70 2023-08-09T14:51:54.609 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:54.604+0000 7f45827fc640 1 -- 172.21.15.43:0/1999881876 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4588012750 con 0x7f458c0d0a70 2023-08-09T14:51:54.609 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:54.604+0000 7f45827fc640 1 -- 172.21.15.43:0/1999881876 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4588012b00 con 0x7f458c0d0a70 2023-08-09T14:51:54.609 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:54.604+0000 7f4593fab640 1 -- 172.21.15.43:0/1999881876 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f458c15cda0 con 0x7f458c0d0a70 2023-08-09T14:51:54.609 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:54.604+0000 7f4593fab640 1 -- 172.21.15.43:0/1999881876 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f458c15d360 con 0x7f458c0d0a70 2023-08-09T14:51:54.610 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:54.606+0000 7f45827fc640 1 -- 172.21.15.43:0/1999881876 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 79) v1 ==== 88443+0+0 (secure 0 0 0) 0x7f4588012c60 con 0x7f458c0d0a70 2023-08-09T14:51:54.610 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:54.607+0000 7f45827fc640 1 --2- 172.21.15.43:0/1999881876 >> [v2:172.21.15.203:6832/2566816332,v1:172.21.15.203:6833/2566816332] conn(0x7f457006a1a0 0x7f457006c660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T14:51:54.610 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:54.607+0000 7f45827fc640 1 -- 172.21.15.43:0/1999881876 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(172..172 src has 1..172) v4 ==== 6426+0+0 (secure 0 0 0) 0x7f4588088d00 con 0x7f458c0d0a70 2023-08-09T14:51:54.610 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:54.608+0000 7f4591546640 1 --2- 172.21.15.43:0/1999881876 >> [v2:172.21.15.203:6832/2566816332,v1:172.21.15.203:6833/2566816332] conn(0x7f457006a1a0 0x7f457006c660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T14:51:54.610 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:54.608+0000 7f4596bf9740 1 Processor -- start 2023-08-09T14:51:54.611 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:54.608+0000 7f4596bf9740 1 -- start start 2023-08-09T14:51:54.612 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:54.609+0000 7f4596bf9740 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55f04fef3c50 0x55f04fef4030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T14:51:54.613 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:54.609+0000 7f4596bf9740 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x55f04fec3bd0 0x55f04fef0ec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T14:51:54.613 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:54.609+0000 7f4596bf9740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55f04fec77d0 0x55f04fef1400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T14:51:54.613 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:54.609+0000 7f4596bf9740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f04fe358f0 con 0x55f04fec77d0 2023-08-09T14:51:54.613 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:54.609+0000 7f4596bf9740 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f04fe32fe0 con 0x55f04fef3c50 2023-08-09T14:51:54.613 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:54.609+0000 7f4596bf9740 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x55f04fe2e8d0 con 0x55f04fec3bd0 2023-08-09T14:51:54.613 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:54.609+0000 7f4591546640 1 --2- 172.21.15.43:0/1999881876 >> [v2:172.21.15.203:6832/2566816332,v1:172.21.15.203:6833/2566816332] conn(0x7f457006a1a0 0x7f457006c660 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f457c036000 tx=0x7f457c03c040 comp rx=0 tx=0).ready entity=mgr.7765 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-09T14:51:54.613 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:54.609+0000 7f4592548640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55f04fec77d0 0x55f04fef1400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T14:51:54.614 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:54.609+0000 7f4592548640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55f04fec77d0 0x55f04fef1400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:58018/0 (socket says 172.21.15.43:58018) 2023-08-09T14:51:54.614 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:54.609+0000 7f4592548640 1 -- 172.21.15.43:0/3506566838 learned_addr learned my addr 172.21.15.43:0/3506566838 (peer_addr_for_me v2:172.21.15.43:0/0) 2023-08-09T14:51:54.614 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:54.610+0000 7f4591d47640 1 --2- 172.21.15.43:0/3506566838 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55f04fef3c50 0x55f04fef4030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T14:51:54.614 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:54.610+0000 7f4591546640 1 --2- 172.21.15.43:0/3506566838 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x55f04fec3bd0 0x55f04fef0ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T14:51:54.614 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:54.610+0000 7f4592548640 1 -- 172.21.15.43:0/3506566838 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x55f04fec3bd0 msgr2=0x55f04fef0ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-09T14:51:54.614 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:54.610+0000 7f4592548640 1 --2- 172.21.15.43:0/3506566838 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x55f04fec3bd0 0x55f04fef0ec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T14:51:54.614 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:54.610+0000 7f4592548640 1 -- 172.21.15.43:0/3506566838 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55f04fef3c50 msgr2=0x55f04fef4030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-09T14:51:54.615 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:54.610+0000 7f4592548640 1 --2- 172.21.15.43:0/3506566838 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55f04fef3c50 0x55f04fef4030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T14:51:54.615 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:54.610+0000 7f4592548640 1 -- 172.21.15.43:0/3506566838 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55f04fde7ac0 con 0x55f04fec77d0 2023-08-09T14:51:54.615 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:54.610+0000 7f4591d47640 1 --2- 172.21.15.43:0/3506566838 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55f04fef3c50 0x55f04fef4030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-08-09T14:51:54.615 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:54.610+0000 7f4592548640 1 --2- 172.21.15.43:0/3506566838 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55f04fec77d0 0x55f04fef1400 secure :-1 s=READY pgs=1892 cs=0 l=1 rev1=1 crypto rx=0x7f458400d4e0 tx=0x7f458400d9a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cf5d232099e92e16 server_cookie=0 in_seq=0 out_seq=0 2023-08-09T14:51:54.615 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:54.610+0000 7f456f7fe640 1 -- 172.21.15.43:0/3506566838 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f458400b760 con 0x55f04fec77d0 2023-08-09T14:51:54.615 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:54.610+0000 7f456f7fe640 1 -- 172.21.15.43:0/3506566838 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f458400b8c0 con 0x55f04fec77d0 2023-08-09T14:51:54.615 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:54.610+0000 7f456f7fe640 1 -- 172.21.15.43:0/3506566838 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f458400bc30 con 0x55f04fec77d0 2023-08-09T14:51:54.615 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:54.611+0000 7f4596bf9740 1 -- 172.21.15.43:0/3506566838 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55f04fec77d0 msgr2=0x55f04fef1400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T14:51:54.616 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:54.611+0000 7f4596bf9740 1 --2- 172.21.15.43:0/3506566838 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55f04fec77d0 0x55f04fef1400 secure :-1 s=READY pgs=1892 cs=0 l=1 rev1=1 crypto rx=0x7f458400d4e0 tx=0x7f458400d9a0 comp rx=0 tx=0).stop 2023-08-09T14:51:54.616 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:54.611+0000 7f4596bf9740 1 -- 172.21.15.43:0/3506566838 shutdown_connections 2023-08-09T14:51:54.616 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:54.611+0000 7f4596bf9740 1 --2- 172.21.15.43:0/3506566838 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55f04fec77d0 0x55f04fef1400 unknown :-1 s=CLOSED pgs=1892 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T14:51:54.616 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:54.611+0000 7f4596bf9740 1 --2- 172.21.15.43:0/3506566838 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x55f04fec3bd0 0x55f04fef0ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T14:51:54.616 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:54.611+0000 7f4596bf9740 1 --2- 172.21.15.43:0/3506566838 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55f04fef3c50 0x55f04fef4030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T14:51:54.616 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:54.611+0000 7f4596bf9740 1 -- 172.21.15.43:0/3506566838 >> 172.21.15.43:0/3506566838 conn(0x55f04fec5ee0 msgr2=0x55f04ff0ea60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-09T14:51:54.616 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:54.611+0000 7f4596bf9740 1 -- 172.21.15.43:0/3506566838 shutdown_connections 2023-08-09T14:51:54.617 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:54.611+0000 7f4596bf9740 1 -- 172.21.15.43:0/3506566838 wait complete. 2023-08-09T14:51:54.619 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:54.612+0000 7f4596bf9740 1 -- start start 2023-08-09T14:51:54.619 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:54.612+0000 7f4596bf9740 1 Processor -- start 2023-08-09T14:51:54.620 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:54.613+0000 7f4596bf9740 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x55f04fec77d0 0x55f04fef4030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T14:51:54.620 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:54.613+0000 7f4596bf9740 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55f04fef3c50 0x55f04feed990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T14:51:54.620 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:54.613+0000 7f4596bf9740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55f04feeb330 0x55f04feeded0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T14:51:54.620 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:54.613+0000 7f4596bf9740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f04fdced40 con 0x55f04feeb330 2023-08-09T14:51:54.620 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:54.613+0000 7f4596bf9740 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f04fdce9e0 con 0x55f04fef3c50 2023-08-09T14:51:54.620 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:54.613+0000 7f4596bf9740 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x55f04fb94f20 con 0x55f04fec77d0 2023-08-09T14:51:54.620 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:54.613+0000 7f4596bf9740 10 client.0 _ll_register_callbacks cb 0x55f04fde7160 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-08-09T14:51:54.621 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:54.614+0000 7f4592548640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55f04feeb330 0x55f04feeded0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T14:51:54.621 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:54.614+0000 7f4592548640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55f04feeb330 0x55f04feeded0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:58028/0 (socket says 172.21.15.43:58028) 2023-08-09T14:51:54.621 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:54.614+0000 7f4592548640 1 -- 172.21.15.43:0/292426353 learned_addr learned my addr 172.21.15.43:0/292426353 (peer_addr_for_me v2:172.21.15.43:0/0) 2023-08-09T14:51:54.621 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:54.614+0000 7f4591546640 1 --2- 172.21.15.43:0/292426353 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55f04fef3c50 0x55f04feed990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T14:51:54.621 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:54.614+0000 7f4592548640 1 -- 172.21.15.43:0/292426353 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x55f04fec77d0 msgr2=0x55f04fef4030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-09T14:51:54.622 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:54.614+0000 7f4592548640 1 --2- 172.21.15.43:0/292426353 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x55f04fec77d0 0x55f04fef4030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T14:51:54.622 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:54.614+0000 7f4592548640 1 -- 172.21.15.43:0/292426353 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55f04fef3c50 msgr2=0x55f04feed990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-09T14:51:54.622 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:54.614+0000 7f4592548640 1 --2- 172.21.15.43:0/292426353 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55f04fef3c50 0x55f04feed990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T14:51:54.622 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:54.614+0000 7f4592548640 1 -- 172.21.15.43:0/292426353 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4584008c80 con 0x55f04feeb330 2023-08-09T14:51:54.622 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:54.614+0000 7f4591546640 1 --2- 172.21.15.43:0/292426353 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55f04fef3c50 0x55f04feed990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-08-09T14:51:54.622 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:54.614+0000 7f4592548640 1 --2- 172.21.15.43:0/292426353 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55f04feeb330 0x55f04feeded0 secure :-1 s=READY pgs=1893 cs=0 l=1 rev1=1 crypto rx=0x7f458400d9e0 tx=0x7f4584004ab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2af9dec7c93b2b8c server_cookie=0 in_seq=0 out_seq=0 2023-08-09T14:51:54.623 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:54.614+0000 7f456cff9640 10 client.0 ms_handle_connect on v2:172.21.15.43:3300/0 2023-08-09T14:51:54.623 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:54.615+0000 7f456cff9640 1 -- 172.21.15.43:0/292426353 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f458401f640 con 0x55f04feeb330 2023-08-09T14:51:54.623 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:54.615+0000 7f4596bf9740 1 -- 172.21.15.43:0/292426353 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55f04fe06500 con 0x55f04feeb330 2023-08-09T14:51:54.623 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:54.615+0000 7f456cff9640 1 -- 172.21.15.43:0/292426353 <== mon.0 v2:172.21.15.43:3300/0 2 ==== osd_map(172..172 src has 1..172) v4 ==== 6426+0+0 (secure 0 0 0) 0x7f458401f7a0 con 0x55f04feeb330 2023-08-09T14:51:54.623 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:54.615+0000 7f456cff9640 1 -- 172.21.15.43:0/292426353 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_get_version_reply(handle=1 version=78) v2 ==== 24+0+0 (secure 0 0 0) 0x7f458401f960 con 0x55f04feeb330 2023-08-09T14:51:54.623 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:54.615+0000 7f4596bf9740 10 client.9801 fetch_fsmap learned FSMap version 78 2023-08-09T14:51:54.623 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:54.615+0000 7f4596bf9740 1 -- 172.21.15.43:0/292426353 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,fsmap=78,monmap=2+}) v3 -- 0x55f04fe2cf50 con 0x55f04feeb330 2023-08-09T14:51:54.623 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:54.615+0000 7f456cff9640 1 -- 172.21.15.43:0/292426353 <== mon.0 v2:172.21.15.43:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x55f04fe06500 con 0x55f04feeb330 2023-08-09T14:51:54.624 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:54.615+0000 7f456cff9640 1 -- 172.21.15.43:0/292426353 <== mon.0 v2:172.21.15.43:3300/0 5 ==== fsmap(e 78) v1 ==== 3294+0+0 (secure 0 0 0) 0x7f4584028d20 con 0x55f04feeb330 2023-08-09T14:51:54.624 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:54.615+0000 7f4596bf9740 10 client.9801 fetch_fsmap finished waiting for FSMap version 78 2023-08-09T14:51:54.624 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:54.615+0000 7f4596bf9740 10 client.9801 resolve_mds: resolved name 'a' to daemon mds.a 2023-08-09T14:51:54.624 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:54.615+0000 7f4596bf9740 20 client.9801 populate_metadata read hostname 'smithi043' 2023-08-09T14:51:54.624 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:54.615+0000 7f4596bf9740 1 --2- 172.21.15.43:0/292426353 >> [v2:172.21.15.43:6834/2435237803,v1:172.21.15.43:6835/2435237803] conn(0x55f04ff2a7f0 0x55f04ff2cbe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T14:51:54.624 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:54.615+0000 7f4596bf9740 4 client.9801 mds_command: new command op to 9648 tid=0[{"prefix": "get_command_descriptions"}] 2023-08-09T14:51:54.624 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:54.615+0000 7f4596bf9740 1 -- 172.21.15.43:0/292426353 --> [v2:172.21.15.43:6834/2435237803,v1:172.21.15.43:6835/2435237803] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55f04fe03bc0 con 0x55f04ff2a7f0 2023-08-09T14:51:54.625 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:54.616+0000 7f4591d47640 1 --2- 172.21.15.43:0/292426353 >> [v2:172.21.15.43:6834/2435237803,v1:172.21.15.43:6835/2435237803] conn(0x55f04ff2a7f0 0x55f04ff2cbe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T14:51:54.625 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:54.616+0000 7f4591d47640 1 --2- 172.21.15.43:0/292426353 >> [v2:172.21.15.43:6834/2435237803,v1:172.21.15.43:6835/2435237803] conn(0x55f04ff2a7f0 0x55f04ff2cbe0 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=52c13350c91f347 server_cookie=cce6e1f360604d8e in_seq=0 out_seq=0 2023-08-09T14:51:54.625 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:54.616+0000 7f456cff9640 10 client.9801 ms_handle_connect on v2:172.21.15.43:6834/2435237803 2023-08-09T14:51:54.625 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:54.617+0000 7f456cff9640 1 -- 172.21.15.43:0/292426353 <== mds.0 v2:172.21.15.43:6834/2435237803 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18017 (crc 0 0 0) 0x55f04fe03bc0 con 0x55f04ff2a7f0 2023-08-09T14:51:54.625 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:54.617+0000 7f456cff9640 10 client.9801 handle_command_reply: tid=0 2023-08-09T14:51:54.625 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:54.617+0000 7f4596bf9740 1 client.9801 shutdown 2023-08-09T14:51:54.625 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:54.617+0000 7f4596bf9740 1 -- 172.21.15.43:0/292426353 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55f04feeb330 msgr2=0x55f04feeded0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T14:51:54.626 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:54.617+0000 7f4596bf9740 1 --2- 172.21.15.43:0/292426353 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55f04feeb330 0x55f04feeded0 secure :-1 s=READY pgs=1893 cs=0 l=1 rev1=1 crypto rx=0x7f458400d9e0 tx=0x7f4584004ab0 comp rx=0 tx=0).stop 2023-08-09T14:51:54.626 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:54.617+0000 7f4596bf9740 1 -- 172.21.15.43:0/292426353 shutdown_connections 2023-08-09T14:51:54.626 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:54.617+0000 7f4596bf9740 1 --2- 172.21.15.43:0/292426353 >> [v2:172.21.15.43:6834/2435237803,v1:172.21.15.43:6835/2435237803] conn(0x55f04ff2a7f0 0x55f04ff2cbe0 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T14:51:54.626 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:54.617+0000 7f4596bf9740 1 --2- 172.21.15.43:0/292426353 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55f04feeb330 0x55f04feeded0 unknown :-1 s=CLOSED pgs=1893 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T14:51:54.626 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:54.617+0000 7f4596bf9740 1 --2- 172.21.15.43:0/292426353 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55f04fef3c50 0x55f04feed990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T14:51:54.626 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:54.617+0000 7f4596bf9740 1 --2- 172.21.15.43:0/292426353 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x55f04fec77d0 0x55f04fef4030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T14:51:54.626 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:54.617+0000 7f456cff9640 0 client.9801 ms_handle_reset on v2:172.21.15.43:6834/2435237803 2023-08-09T14:51:54.626 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:54.617+0000 7f4596bf9740 1 -- 172.21.15.43:0/292426353 >> 172.21.15.43:0/292426353 conn(0x55f04fec3bd0 msgr2=0x55f04fec8080 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-09T14:51:54.627 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:54.618+0000 7f4596bf9740 1 -- 172.21.15.43:0/292426353 shutdown_connections 2023-08-09T14:51:54.627 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:54.618+0000 7f4596bf9740 1 -- 172.21.15.43:0/292426353 wait complete. 2023-08-09T14:51:54.627 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:54.618+0000 7f4596bf9740 20 client.9801 trim_cache size 0 max 16384 2023-08-09T14:51:54.636 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:54.635+0000 7f4596bf9740 1 Processor -- start 2023-08-09T14:51:54.637 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:54.635+0000 7f4596bf9740 1 -- start start 2023-08-09T14:51:54.639 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:54.636+0000 7f4596bf9740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55f04fec77d0 0x55f04fec7dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T14:51:54.640 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:54.636+0000 7f4596bf9740 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55f04feeb330 0x55f04ff0ea60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T14:51:54.640 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:54.636+0000 7f4596bf9740 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x55f04fef3c50 0x55f04ff0efa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T14:51:54.640 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:54.636+0000 7f4596bf9740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f04fef4de0 con 0x55f04fec77d0 2023-08-09T14:51:54.640 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:54.636+0000 7f4596bf9740 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f04fec3fb0 con 0x55f04feeb330 2023-08-09T14:51:54.640 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:54.636+0000 7f4596bf9740 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x55f04fecb2b0 con 0x55f04fef3c50 2023-08-09T14:51:54.641 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:54.636+0000 7f4591d47640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55f04fec77d0 0x55f04fec7dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T14:51:54.641 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:54.636+0000 7f4591d47640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55f04fec77d0 0x55f04fec7dc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:58030/0 (socket says 172.21.15.43:58030) 2023-08-09T14:51:54.641 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:54.636+0000 7f4591d47640 1 -- 172.21.15.43:0/4094507063 learned_addr learned my addr 172.21.15.43:0/4094507063 (peer_addr_for_me v2:172.21.15.43:0/0) 2023-08-09T14:51:54.641 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:54.636+0000 7f4591d47640 1 -- 172.21.15.43:0/4094507063 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x55f04fef3c50 msgr2=0x55f04ff0efa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-09T14:51:54.641 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:54.636+0000 7f4591d47640 1 --2- 172.21.15.43:0/4094507063 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x55f04fef3c50 0x55f04ff0efa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T14:51:54.641 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:54.636+0000 7f4591546640 1 --2- 172.21.15.43:0/4094507063 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55f04feeb330 0x55f04ff0ea60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T14:51:54.641 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:54.636+0000 7f4591d47640 1 -- 172.21.15.43:0/4094507063 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55f04feeb330 msgr2=0x55f04ff0ea60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-09T14:51:54.642 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:54.636+0000 7f4591d47640 1 --2- 172.21.15.43:0/4094507063 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55f04feeb330 0x55f04ff0ea60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T14:51:54.642 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:54.636+0000 7f4591d47640 1 -- 172.21.15.43:0/4094507063 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55f04fe60300 con 0x55f04fec77d0 2023-08-09T14:51:54.642 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:54.637+0000 7f4591d47640 1 --2- 172.21.15.43:0/4094507063 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55f04fec77d0 0x55f04fec7dc0 secure :-1 s=READY pgs=1894 cs=0 l=1 rev1=1 crypto rx=0x7f458800dfb0 tx=0x7f4588012f10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=212d7be6dd8d6975 server_cookie=0 in_seq=0 out_seq=0 2023-08-09T14:51:54.642 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:54.637+0000 7f456ffff640 1 -- 172.21.15.43:0/4094507063 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4588056020 con 0x55f04fec77d0 2023-08-09T14:51:54.642 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:54.637+0000 7f456ffff640 1 -- 172.21.15.43:0/4094507063 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4588054530 con 0x55f04fec77d0 2023-08-09T14:51:54.642 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:54.637+0000 7f456ffff640 1 -- 172.21.15.43:0/4094507063 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f45880870b0 con 0x55f04fec77d0 2023-08-09T14:51:54.642 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:54.637+0000 7f4596bf9740 1 -- 172.21.15.43:0/4094507063 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55f04fec77d0 msgr2=0x55f04fec7dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T14:51:54.642 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:54.637+0000 7f4596bf9740 1 --2- 172.21.15.43:0/4094507063 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55f04fec77d0 0x55f04fec7dc0 secure :-1 s=READY pgs=1894 cs=0 l=1 rev1=1 crypto rx=0x7f458800dfb0 tx=0x7f4588012f10 comp rx=0 tx=0).stop 2023-08-09T14:51:54.643 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:54.638+0000 7f4596bf9740 1 -- 172.21.15.43:0/4094507063 shutdown_connections 2023-08-09T14:51:54.643 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:54.638+0000 7f4596bf9740 1 --2- 172.21.15.43:0/4094507063 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x55f04fef3c50 0x55f04ff0efa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T14:51:54.643 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:54.638+0000 7f4596bf9740 1 --2- 172.21.15.43:0/4094507063 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55f04feeb330 0x55f04ff0ea60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T14:51:54.643 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:54.638+0000 7f4596bf9740 1 --2- 172.21.15.43:0/4094507063 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55f04fec77d0 0x55f04fec7dc0 unknown :-1 s=CLOSED pgs=1894 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T14:51:54.643 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:54.638+0000 7f4596bf9740 1 -- 172.21.15.43:0/4094507063 >> 172.21.15.43:0/4094507063 conn(0x55f04fec3bd0 msgr2=0x55f04fec9d40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-09T14:51:54.643 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:54.638+0000 7f4596bf9740 1 -- 172.21.15.43:0/4094507063 shutdown_connections 2023-08-09T14:51:54.643 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:54.638+0000 7f4596bf9740 1 -- 172.21.15.43:0/4094507063 wait complete. 2023-08-09T14:51:54.643 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:54.638+0000 7f4596bf9740 1 -- start start 2023-08-09T14:51:54.646 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:54.639+0000 7f4596bf9740 1 Processor -- start 2023-08-09T14:51:54.646 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:54.639+0000 7f4596bf9740 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55f04fec77d0 0x55f04ff77770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T14:51:54.647 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:54.639+0000 7f4596bf9740 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x55f04feeb330 0x55f04ff77cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T14:51:54.647 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:54.640+0000 7f4596bf9740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55f04fef3c50 0x55f04ff781f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T14:51:54.647 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:54.640+0000 7f4596bf9740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f04fee8520 con 0x55f04fef3c50 2023-08-09T14:51:54.647 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:54.640+0000 7f4596bf9740 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f04fee80d0 con 0x55f04fec77d0 2023-08-09T14:51:54.647 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:54.640+0000 7f4596bf9740 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x55f04fec3210 con 0x55f04feeb330 2023-08-09T14:51:54.647 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:54.640+0000 7f4596bf9740 10 client.0 _ll_register_callbacks cb 0x7f4584018070 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-08-09T14:51:54.647 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:54.640+0000 7f4592548640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55f04fef3c50 0x55f04ff781f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T14:51:54.647 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:54.640+0000 7f4592548640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55f04fef3c50 0x55f04ff781f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:58034/0 (socket says 172.21.15.43:58034) 2023-08-09T14:51:54.648 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:54.640+0000 7f4592548640 1 -- 172.21.15.43:0/833343414 learned_addr learned my addr 172.21.15.43:0/833343414 (peer_addr_for_me v2:172.21.15.43:0/0) 2023-08-09T14:51:54.648 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:54.640+0000 7f4591546640 1 --2- 172.21.15.43:0/833343414 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x55f04feeb330 0x55f04ff77cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T14:51:54.648 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:54.640+0000 7f4592548640 1 -- 172.21.15.43:0/833343414 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x55f04feeb330 msgr2=0x55f04ff77cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-09T14:51:54.648 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:54.640+0000 7f4591d47640 1 --2- 172.21.15.43:0/833343414 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55f04fec77d0 0x55f04ff77770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T14:51:54.648 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:54.640+0000 7f4592548640 1 --2- 172.21.15.43:0/833343414 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x55f04feeb330 0x55f04ff77cb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T14:51:54.648 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:54.640+0000 7f4592548640 1 -- 172.21.15.43:0/833343414 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55f04fec77d0 msgr2=0x55f04ff77770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-09T14:51:54.648 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:54.640+0000 7f4592548640 1 --2- 172.21.15.43:0/833343414 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55f04fec77d0 0x55f04ff77770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T14:51:54.649 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:54.640+0000 7f4592548640 1 -- 172.21.15.43:0/833343414 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f458c15d360 con 0x55f04fef3c50 2023-08-09T14:51:54.649 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:54.641+0000 7f4592548640 1 --2- 172.21.15.43:0/833343414 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55f04fef3c50 0x55f04ff781f0 secure :-1 s=READY pgs=1895 cs=0 l=1 rev1=1 crypto rx=0x7f4584007130 tx=0x7f4584004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cb08bf84aa8b8a8d server_cookie=0 in_seq=0 out_seq=0 2023-08-09T14:51:54.649 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:54.641+0000 7f456cff9640 10 client.0 ms_handle_connect on v2:172.21.15.43:3300/0 2023-08-09T14:51:54.649 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:54.641+0000 7f456cff9640 1 -- 172.21.15.43:0/833343414 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4584008ea0 con 0x55f04fef3c50 2023-08-09T14:51:54.649 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:54.641+0000 7f456cff9640 1 -- 172.21.15.43:0/833343414 <== mon.0 v2:172.21.15.43:3300/0 2 ==== osd_map(172..172 src has 1..172) v4 ==== 6426+0+0 (secure 0 0 0) 0x7f458401fe40 con 0x55f04fef3c50 2023-08-09T14:51:54.649 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:54.641+0000 7f4596bf9740 1 -- 172.21.15.43:0/833343414 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55f04fe56800 con 0x55f04fef3c50 2023-08-09T14:51:54.649 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:54.641+0000 7f456cff9640 1 -- 172.21.15.43:0/833343414 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_get_version_reply(handle=1 version=78) v2 ==== 24+0+0 (secure 0 0 0) 0x7f45840288a0 con 0x55f04fef3c50 2023-08-09T14:51:54.650 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:54.641+0000 7f4596bf9740 10 client.9807 fetch_fsmap learned FSMap version 78 2023-08-09T14:51:54.650 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:54.641+0000 7f4596bf9740 1 -- 172.21.15.43:0/833343414 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,fsmap=78,monmap=2+}) v3 -- 0x55f04fdcda10 con 0x55f04fef3c50 2023-08-09T14:51:54.650 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:54.642+0000 7f456cff9640 1 -- 172.21.15.43:0/833343414 <== mon.0 v2:172.21.15.43:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x55f04fe56800 con 0x55f04fef3c50 2023-08-09T14:51:54.650 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:54.642+0000 7f456cff9640 1 -- 172.21.15.43:0/833343414 <== mon.0 v2:172.21.15.43:3300/0 5 ==== fsmap(e 78) v1 ==== 3294+0+0 (secure 0 0 0) 0x7f4584008040 con 0x55f04fef3c50 2023-08-09T14:51:54.650 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:54.642+0000 7f4596bf9740 10 client.9807 fetch_fsmap finished waiting for FSMap version 78 2023-08-09T14:51:54.650 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:54.642+0000 7f4596bf9740 10 client.9807 resolve_mds: resolved name 'a' to daemon mds.a 2023-08-09T14:51:54.650 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:54.642+0000 7f4596bf9740 20 client.9807 populate_metadata read hostname 'smithi043' 2023-08-09T14:51:54.650 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:54.642+0000 7f4596bf9740 1 --2- 172.21.15.43:0/833343414 >> [v2:172.21.15.43:6834/2435237803,v1:172.21.15.43:6835/2435237803] conn(0x55f04fec8fa0 0x55f04fec9380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T14:51:54.651 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:54.642+0000 7f4596bf9740 4 client.9807 mds_command: new command op to 9648 tid=0[{"prefix": "status"}] 2023-08-09T14:51:54.651 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:54.642+0000 7f4596bf9740 1 -- 172.21.15.43:0/833343414 --> [v2:172.21.15.43:6834/2435237803,v1:172.21.15.43:6835/2435237803] -- command(tid 0: {"prefix": "status"}) v1 -- 0x55f04fe278b0 con 0x55f04fec8fa0 2023-08-09T14:51:54.651 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:54.642+0000 7f4591d47640 1 --2- 172.21.15.43:0/833343414 >> [v2:172.21.15.43:6834/2435237803,v1:172.21.15.43:6835/2435237803] conn(0x55f04fec8fa0 0x55f04fec9380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T14:51:54.651 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:54.642+0000 7f4591d47640 1 --2- 172.21.15.43:0/833343414 >> [v2:172.21.15.43:6834/2435237803,v1:172.21.15.43:6835/2435237803] conn(0x55f04fec8fa0 0x55f04fec9380 crc :-1 s=READY pgs=14 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=aa5bc9f8b6e54744 server_cookie=888221de0cf4bbbd in_seq=0 out_seq=0 2023-08-09T14:51:54.651 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:54.643+0000 7f456cff9640 10 client.9807 ms_handle_connect on v2:172.21.15.43:6834/2435237803 2023-08-09T14:51:54.651 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:54.643+0000 7f456cff9640 1 -- 172.21.15.43:0/833343414 <== mds.0 v2:172.21.15.43:6834/2435237803 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+333 (crc 0 0 0) 0x55f04fe278b0 con 0x55f04fec8fa0 2023-08-09T14:51:54.651 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:54.643+0000 7f456cff9640 10 client.9807 handle_command_reply: tid=0 2023-08-09T14:51:54.651 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:54.643+0000 7f4596bf9740 1 client.9807 shutdown 2023-08-09T14:51:54.652 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:54.643+0000 7f4596bf9740 1 -- 172.21.15.43:0/833343414 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55f04fef3c50 msgr2=0x55f04ff781f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T14:51:54.652 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:54.643+0000 7f4596bf9740 1 --2- 172.21.15.43:0/833343414 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55f04fef3c50 0x55f04ff781f0 secure :-1 s=READY pgs=1895 cs=0 l=1 rev1=1 crypto rx=0x7f4584007130 tx=0x7f4584004a40 comp rx=0 tx=0).stop 2023-08-09T14:51:54.652 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:54.643+0000 7f4596bf9740 1 -- 172.21.15.43:0/833343414 shutdown_connections 2023-08-09T14:51:54.652 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:54.643+0000 7f4596bf9740 1 --2- 172.21.15.43:0/833343414 >> [v2:172.21.15.43:6834/2435237803,v1:172.21.15.43:6835/2435237803] conn(0x55f04fec8fa0 0x55f04fec9380 crc :-1 s=READY pgs=14 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T14:51:54.652 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:54.643+0000 7f4596bf9740 1 --2- 172.21.15.43:0/833343414 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55f04fef3c50 0x55f04ff781f0 unknown :-1 s=CLOSED pgs=1895 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T14:51:54.652 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:54.643+0000 7f4596bf9740 1 --2- 172.21.15.43:0/833343414 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x55f04feeb330 0x55f04ff77cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T14:51:54.653 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:54.643+0000 7f456cff9640 0 client.9807 ms_handle_reset on v2:172.21.15.43:6834/2435237803 2023-08-09T14:51:54.653 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:54.643+0000 7f4596bf9740 1 --2- 172.21.15.43:0/833343414 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55f04fec77d0 0x55f04ff77770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T14:51:54.653 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:54.643+0000 7f4596bf9740 1 -- 172.21.15.43:0/833343414 >> 172.21.15.43:0/833343414 conn(0x55f04fec3bd0 msgr2=0x55f04fec7dc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-09T14:51:54.653 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:54.643+0000 7f4596bf9740 1 -- 172.21.15.43:0/833343414 shutdown_connections 2023-08-09T14:51:54.653 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:54.643+0000 7f4596bf9740 1 -- 172.21.15.43:0/833343414 wait complete. 2023-08-09T14:51:54.653 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:54.643+0000 7f4596bf9740 20 client.9807 trim_cache size 0 max 16384 2023-08-09T14:51:54.654 INFO:teuthology.orchestra.run.smithi043.stdout:{ 2023-08-09T14:51:54.654 INFO:teuthology.orchestra.run.smithi043.stdout: "cluster_fsid": "daac87ee-1337-4250-a955-aea4e408a39b", 2023-08-09T14:51:54.654 INFO:teuthology.orchestra.run.smithi043.stdout: "whoami": 0, 2023-08-09T14:51:54.654 INFO:teuthology.orchestra.run.smithi043.stdout: "id": 9648, 2023-08-09T14:51:54.654 INFO:teuthology.orchestra.run.smithi043.stdout: "want_state": "up:active", 2023-08-09T14:51:54.654 INFO:teuthology.orchestra.run.smithi043.stdout: "state": "up:active", 2023-08-09T14:51:54.655 INFO:teuthology.orchestra.run.smithi043.stdout: "fs_name": "cephfs", 2023-08-09T14:51:54.655 INFO:teuthology.orchestra.run.smithi043.stdout: "rank_uptime": 6.8519470719999997, 2023-08-09T14:51:54.655 INFO:teuthology.orchestra.run.smithi043.stdout: "mdsmap_epoch": 78, 2023-08-09T14:51:54.655 INFO:teuthology.orchestra.run.smithi043.stdout: "osdmap_epoch": 172, 2023-08-09T14:51:54.655 INFO:teuthology.orchestra.run.smithi043.stdout: "osdmap_epoch_barrier": 172, 2023-08-09T14:51:54.655 INFO:teuthology.orchestra.run.smithi043.stdout: "uptime": 12.794869717999999 2023-08-09T14:51:54.655 INFO:teuthology.orchestra.run.smithi043.stdout:} 2023-08-09T14:51:54.656 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:54.644+0000 7f4593fab640 1 -- 172.21.15.43:0/1999881876 >> [v2:172.21.15.203:6832/2566816332,v1:172.21.15.203:6833/2566816332] conn(0x7f457006a1a0 msgr2=0x7f457006c660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T14:51:54.656 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:54.644+0000 7f4593fab640 1 --2- 172.21.15.43:0/1999881876 >> [v2:172.21.15.203:6832/2566816332,v1:172.21.15.203:6833/2566816332] conn(0x7f457006a1a0 0x7f457006c660 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f457c036000 tx=0x7f457c03c040 comp rx=0 tx=0).stop 2023-08-09T14:51:54.656 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:54.644+0000 7f4593fab640 1 -- 172.21.15.43:0/1999881876 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f458c0d0a70 msgr2=0x7f458c0ca2c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T14:51:54.656 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:54.644+0000 7f4593fab640 1 --2- 172.21.15.43:0/1999881876 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f458c0d0a70 0x7f458c0ca2c0 secure :-1 s=READY pgs=1891 cs=0 l=1 rev1=1 crypto rx=0x7f458800d5f0 tx=0x7f458800dab0 comp rx=0 tx=0).stop 2023-08-09T14:51:54.657 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:54.644+0000 7f4593fab640 1 -- 172.21.15.43:0/1999881876 shutdown_connections 2023-08-09T14:51:54.657 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:54.644+0000 7f4593fab640 1 --2- 172.21.15.43:0/1999881876 >> [v2:172.21.15.203:6832/2566816332,v1:172.21.15.203:6833/2566816332] conn(0x7f457006a1a0 0x7f457006c660 unknown :-1 s=CLOSED pgs=69 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T14:51:54.657 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:54.644+0000 7f4593fab640 1 --2- 172.21.15.43:0/1999881876 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f458c0b6910 0x7f458c153530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T14:51:54.657 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:54.644+0000 7f4593fab640 1 --2- 172.21.15.43:0/1999881876 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f458c15c680 0x7f458c0cb070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T14:51:54.657 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:54.644+0000 7f4593fab640 1 --2- 172.21.15.43:0/1999881876 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f458c0d0a70 0x7f458c0ca2c0 unknown :-1 s=CLOSED pgs=1891 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T14:51:54.657 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:54.644+0000 7f4593fab640 1 -- 172.21.15.43:0/1999881876 >> 172.21.15.43:0/1999881876 conn(0x7f458c071520 msgr2=0x7f458c15a600 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-09T14:51:54.657 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:54.644+0000 7f4593fab640 1 -- 172.21.15.43:0/1999881876 shutdown_connections 2023-08-09T14:51:54.658 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:54.644+0000 7f4593fab640 1 -- 172.21.15.43:0/1999881876 wait complete. 2023-08-09T14:51:54.662 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2023-08-09T14:51:54.860 DEBUG:teuthology.orchestra.run.smithi043:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-08-09T14:51:54.869 DEBUG:teuthology.orchestra.run.smithi203:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-08-09T14:51:54.888 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:54.887+0000 7fe0fe2fa640 1 Processor -- start 2023-08-09T14:51:54.889 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:54.887+0000 7fe0fe2fa640 1 -- start start 2023-08-09T14:51:54.890 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:54.888+0000 7fe0fe2fa640 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7fe0f80b2790 0x7fe0f80d2ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T14:51:54.890 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:54.888+0000 7fe0fe2fa640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fe0f80b2b70 0x7fe0f80b5330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T14:51:54.890 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:54.888+0000 7fe0fe2fa640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7fe0f80b3050 0x7fe0f80b3f60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T14:51:54.890 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:54.888+0000 7fe0fe2fa640 1 -- --> v2:172.21.15.43:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe0f80b3950 con 0x7fe0f80b3050 2023-08-09T14:51:54.890 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:54.888+0000 7fe0fe2fa640 1 -- --> v2:172.21.15.203:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe0f80ce090 con 0x7fe0f80b2790 2023-08-09T14:51:54.890 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:54.888+0000 7fe0fe2fa640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe0f80cd7f0 con 0x7fe0f80b2b70 2023-08-09T14:51:54.891 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:54.889+0000 7fe0fc897640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7fe0f80b3050 0x7fe0f80b3f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T14:51:54.891 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:54.889+0000 7fe0fc897640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7fe0f80b3050 0x7fe0f80b3f60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:58050/0 (socket says 172.21.15.43:58050) 2023-08-09T14:51:54.891 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:54.889+0000 7fe0fc897640 1 -- 172.21.15.43:0/675832069 learned_addr learned my addr 172.21.15.43:0/675832069 (peer_addr_for_me v2:172.21.15.43:0/0) 2023-08-09T14:51:54.892 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:54.889+0000 7fe0f77fe640 1 --2- 172.21.15.43:0/675832069 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fe0f80b2b70 0x7fe0f80b5330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T14:51:54.892 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:54.889+0000 7fe0f7fff640 1 --2- 172.21.15.43:0/675832069 >> v2:172.21.15.203:3300/0 conn(0x7fe0f80b2790 0x7fe0f80d2ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T14:51:54.892 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:54.889+0000 7fe0fc897640 1 -- 172.21.15.43:0/675832069 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fe0f80b2b70 msgr2=0x7fe0f80b5330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-09T14:51:54.892 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:54.889+0000 7fe0fc897640 1 --2- 172.21.15.43:0/675832069 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fe0f80b2b70 0x7fe0f80b5330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T14:51:54.892 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:54.889+0000 7fe0fc897640 1 -- 172.21.15.43:0/675832069 >> v2:172.21.15.203:3300/0 conn(0x7fe0f80b2790 msgr2=0x7fe0f80d2ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-09T14:51:54.892 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:54.889+0000 7fe0fc897640 1 --2- 172.21.15.43:0/675832069 >> v2:172.21.15.203:3300/0 conn(0x7fe0f80b2790 0x7fe0f80d2ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T14:51:54.892 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:54.889+0000 7fe0fc897640 1 -- 172.21.15.43:0/675832069 --> v2:172.21.15.43:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe0f80cdad0 con 0x7fe0f80b3050 2023-08-09T14:51:54.893 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:54.889+0000 7fe0f7fff640 1 --2- 172.21.15.43:0/675832069 >> v2:172.21.15.203:3300/0 conn(0x7fe0f80b2790 0x7fe0f80d2ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-08-09T14:51:54.893 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:54.890+0000 7fe0fc897640 1 --2- 172.21.15.43:0/675832069 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fe0f80b3050 0x7fe0f80b3f60 secure :-1 s=READY pgs=1896 cs=0 l=1 rev1=1 crypto rx=0x7fe0f808a4a0 tx=0x7fe0f002bee0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e874284b2235c741 server_cookie=0 in_seq=0 out_seq=0 2023-08-09T14:51:54.894 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:54.891+0000 7fe0f6ffd640 1 -- 172.21.15.43:0/675832069 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe0f002ca80 con 0x7fe0f80b3050 2023-08-09T14:51:54.894 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:54.891+0000 7fe0f6ffd640 1 -- 172.21.15.43:0/675832069 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe0f002cbe0 con 0x7fe0f80b3050 2023-08-09T14:51:54.894 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:54.891+0000 7fe0f6ffd640 1 -- 172.21.15.43:0/675832069 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe0f00343f0 con 0x7fe0f80b3050 2023-08-09T14:51:54.894 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:54.891+0000 7fe0fe2fa640 1 -- 172.21.15.43:0/675832069 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fe0f80b3050 msgr2=0x7fe0f80b3f60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T14:51:54.894 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:54.891+0000 7fe0fe2fa640 1 --2- 172.21.15.43:0/675832069 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fe0f80b3050 0x7fe0f80b3f60 secure :-1 s=READY pgs=1896 cs=0 l=1 rev1=1 crypto rx=0x7fe0f808a4a0 tx=0x7fe0f002bee0 comp rx=0 tx=0).stop 2023-08-09T14:51:54.894 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:54.892+0000 7fe0fe2fa640 1 -- 172.21.15.43:0/675832069 shutdown_connections 2023-08-09T14:51:54.894 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:54.892+0000 7fe0fe2fa640 1 --2- 172.21.15.43:0/675832069 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fe0f80b3050 0x7fe0f80b3f60 unknown :-1 s=CLOSED pgs=1896 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T14:51:54.895 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:54.892+0000 7fe0fe2fa640 1 --2- 172.21.15.43:0/675832069 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fe0f80b2b70 0x7fe0f80b5330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T14:51:54.895 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:54.892+0000 7fe0fe2fa640 1 --2- 172.21.15.43:0/675832069 >> v2:172.21.15.203:3300/0 conn(0x7fe0f80b2790 0x7fe0f80d2ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T14:51:54.895 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:54.892+0000 7fe0fe2fa640 1 -- 172.21.15.43:0/675832069 >> 172.21.15.43:0/675832069 conn(0x7fe0f809b7b0 msgr2=0x7fe0f80d24e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-09T14:51:54.895 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:54.892+0000 7fe0fe2fa640 1 -- 172.21.15.43:0/675832069 shutdown_connections 2023-08-09T14:51:54.895 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:54.892+0000 7fe0fe2fa640 1 -- 172.21.15.43:0/675832069 wait complete. 2023-08-09T14:51:54.896 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:54.893+0000 7fe0fe2fa640 1 Processor -- start 2023-08-09T14:51:54.896 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:54.894+0000 7fe0fe2fa640 1 -- start start 2023-08-09T14:51:54.897 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:54.895+0000 7fe0fe2fa640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fe0f80b2790 0x7fe0f8149270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T14:51:54.897 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:54.895+0000 7fe0fe2fa640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fe0f80b2b70 0x7fe0f80ccb10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T14:51:54.897 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:54.895+0000 7fe0fe2fa640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fe0f80b3050 0x7fe0f80cd050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T14:51:54.897 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:54.895+0000 7fe0fe2fa640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe0f80c8ec0 con 0x7fe0f80b2790 2023-08-09T14:51:54.898 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:54.895+0000 7fe0fe2fa640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe0f81f6250 con 0x7fe0f80b2b70 2023-08-09T14:51:54.898 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:54.895+0000 7fe0fe2fa640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe0f814bc50 con 0x7fe0f80b3050 2023-08-09T14:51:54.898 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:54.895+0000 7fe0f7fff640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fe0f80b2790 0x7fe0f8149270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T14:51:54.898 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:54.895+0000 7fe0f7fff640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fe0f80b2790 0x7fe0f8149270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:58066/0 (socket says 172.21.15.43:58066) 2023-08-09T14:51:54.898 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:54.895+0000 7fe0f7fff640 1 -- 172.21.15.43:0/1005997995 learned_addr learned my addr 172.21.15.43:0/1005997995 (peer_addr_for_me v2:172.21.15.43:0/0) 2023-08-09T14:51:54.898 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:54.895+0000 7fe0f77fe640 1 --2- 172.21.15.43:0/1005997995 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fe0f80b2b70 0x7fe0f80ccb10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T14:51:54.898 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:54.895+0000 7fe0fc897640 1 --2- 172.21.15.43:0/1005997995 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fe0f80b3050 0x7fe0f80cd050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T14:51:54.899 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:54.896+0000 7fe0f7fff640 1 -- 172.21.15.43:0/1005997995 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fe0f80b3050 msgr2=0x7fe0f80cd050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T14:51:54.899 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:54.896+0000 7fe0f7fff640 1 --2- 172.21.15.43:0/1005997995 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fe0f80b3050 0x7fe0f80cd050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T14:51:54.899 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:54.896+0000 7fe0f7fff640 1 -- 172.21.15.43:0/1005997995 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fe0f80b2b70 msgr2=0x7fe0f80ccb10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T14:51:54.899 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:54.896+0000 7fe0f7fff640 1 --2- 172.21.15.43:0/1005997995 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fe0f80b2b70 0x7fe0f80ccb10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T14:51:54.899 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:54.896+0000 7fe0f7fff640 1 -- 172.21.15.43:0/1005997995 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe0f0009b30 con 0x7fe0f80b2790 2023-08-09T14:51:54.900 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:54.896+0000 7fe0f7fff640 1 --2- 172.21.15.43:0/1005997995 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fe0f80b2790 0x7fe0f8149270 secure :-1 s=READY pgs=1897 cs=0 l=1 rev1=1 crypto rx=0x7fe0e800d800 tx=0x7fe0e800dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-09T14:51:54.900 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:54.896+0000 7fe0f4ff9640 1 -- 172.21.15.43:0/1005997995 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe0e8004030 con 0x7fe0f80b2790 2023-08-09T14:51:54.900 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:54.896+0000 7fe0f4ff9640 1 -- 172.21.15.43:0/1005997995 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe0e8004190 con 0x7fe0f80b2790 2023-08-09T14:51:54.900 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:54.896+0000 7fe0fe2fa640 1 -- 172.21.15.43:0/1005997995 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe0f814bf30 con 0x7fe0f80b2790 2023-08-09T14:51:54.900 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:54.896+0000 7fe0f4ff9640 1 -- 172.21.15.43:0/1005997995 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe0e8004520 con 0x7fe0f80b2790 2023-08-09T14:51:54.901 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:54.897+0000 7fe0fe2fa640 1 -- 172.21.15.43:0/1005997995 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe0f814c4f0 con 0x7fe0f80b2790 2023-08-09T14:51:54.901 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:54.898+0000 7fe0fe2fa640 1 -- 172.21.15.43:0/1005997995 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe0f80739f0 con 0x7fe0f80b2790 2023-08-09T14:51:54.901 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:54.899+0000 7fe0f4ff9640 1 -- 172.21.15.43:0/1005997995 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 79) v1 ==== 88443+0+0 (secure 0 0 0) 0x7fe0e80096a0 con 0x7fe0f80b2790 2023-08-09T14:51:54.901 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:54.900+0000 7fe0f4ff9640 1 --2- 172.21.15.43:0/1005997995 >> [v2:172.21.15.203:6832/2566816332,v1:172.21.15.203:6833/2566816332] conn(0x7fe0e406a170 0x7fe0e406c630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T14:51:54.902 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:54.900+0000 7fe0f4ff9640 1 -- 172.21.15.43:0/1005997995 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(172..172 src has 1..172) v4 ==== 6426+0+0 (secure 0 0 0) 0x7fe0e8088b70 con 0x7fe0f80b2790 2023-08-09T14:51:54.902 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:54.901+0000 7fe0f77fe640 1 --2- 172.21.15.43:0/1005997995 >> [v2:172.21.15.203:6832/2566816332,v1:172.21.15.203:6833/2566816332] conn(0x7fe0e406a170 0x7fe0e406c630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T14:51:54.903 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:54.902+0000 7fe0f77fe640 1 --2- 172.21.15.43:0/1005997995 >> [v2:172.21.15.203:6832/2566816332,v1:172.21.15.203:6833/2566816332] conn(0x7fe0e406a170 0x7fe0e406c630 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7fe0f80d25e0 tx=0x7fe0ec008040 comp rx=0 tx=0).ready entity=mgr.7765 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-09T14:51:54.906 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:54.905+0000 7fe0f4ff9640 1 -- 172.21.15.43:0/1005997995 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178066 (secure 0 0 0) 0x7fe0e8053f40 con 0x7fe0f80b2790 2023-08-09T14:51:55.141 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:55.139+0000 7fe0fe2fa640 1 -- 172.21.15.43:0/1005997995 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7fe0f80d2c50 con 0x7fe0f80b2790 2023-08-09T14:51:55.142 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:55.140+0000 7fe0f4ff9640 1 -- 172.21.15.43:0/1005997995 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v78) v1 ==== 53+0+73 (secure 0 0 0) 0x7fe0e8057560 con 0x7fe0f80b2790 2023-08-09T14:51:55.142 INFO:teuthology.orchestra.run.smithi043.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2023-08-09T14:51:55.145 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:55.143+0000 7fe0fe2fa640 1 -- 172.21.15.43:0/1005997995 >> [v2:172.21.15.203:6832/2566816332,v1:172.21.15.203:6833/2566816332] conn(0x7fe0e406a170 msgr2=0x7fe0e406c630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T14:51:55.145 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:55.143+0000 7fe0fe2fa640 1 --2- 172.21.15.43:0/1005997995 >> [v2:172.21.15.203:6832/2566816332,v1:172.21.15.203:6833/2566816332] conn(0x7fe0e406a170 0x7fe0e406c630 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7fe0f80d25e0 tx=0x7fe0ec008040 comp rx=0 tx=0).stop 2023-08-09T14:51:55.146 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:55.143+0000 7fe0fe2fa640 1 -- 172.21.15.43:0/1005997995 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fe0f80b2790 msgr2=0x7fe0f8149270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T14:51:55.146 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:55.143+0000 7fe0fe2fa640 1 --2- 172.21.15.43:0/1005997995 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fe0f80b2790 0x7fe0f8149270 secure :-1 s=READY pgs=1897 cs=0 l=1 rev1=1 crypto rx=0x7fe0e800d800 tx=0x7fe0e800dcc0 comp rx=0 tx=0).stop 2023-08-09T14:51:55.147 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:55.143+0000 7fe0fe2fa640 1 -- 172.21.15.43:0/1005997995 shutdown_connections 2023-08-09T14:51:55.147 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:55.143+0000 7fe0fe2fa640 1 --2- 172.21.15.43:0/1005997995 >> [v2:172.21.15.203:6832/2566816332,v1:172.21.15.203:6833/2566816332] conn(0x7fe0e406a170 0x7fe0e406c630 unknown :-1 s=CLOSED pgs=70 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T14:51:55.147 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:55.143+0000 7fe0fe2fa640 1 --2- 172.21.15.43:0/1005997995 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fe0f80b3050 0x7fe0f80cd050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T14:51:55.148 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:55.143+0000 7fe0fe2fa640 1 --2- 172.21.15.43:0/1005997995 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fe0f80b2b70 0x7fe0f80ccb10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T14:51:55.148 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:55.143+0000 7fe0fe2fa640 1 --2- 172.21.15.43:0/1005997995 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fe0f80b2790 0x7fe0f8149270 unknown :-1 s=CLOSED pgs=1897 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T14:51:55.148 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:55.143+0000 7fe0fe2fa640 1 -- 172.21.15.43:0/1005997995 >> 172.21.15.43:0/1005997995 conn(0x7fe0f809b7b0 msgr2=0x7fe0f806b0a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-09T14:51:55.149 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:55.143+0000 7fe0fe2fa640 1 -- 172.21.15.43:0/1005997995 shutdown_connections 2023-08-09T14:51:55.149 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:55.143+0000 7fe0fe2fa640 1 -- 172.21.15.43:0/1005997995 wait complete. 2023-08-09T14:51:55.164 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2023-08-09T14:51:55.164 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2023-08-09T14:51:55.165 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi043.front.sepia.ceph.com 2023-08-09T14:51:55.165 INFO:tasks.cephfs.mount:self.client.name = client.0 2023-08-09T14:51:55.165 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2023-08-09T14:51:55.165 INFO:tasks.cephfs.mount:self.cephfs_name = None 2023-08-09T14:51:55.166 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2023-08-09T14:51:55.166 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2023-08-09T14:51:55.166 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2023-08-09T14:51:55.167 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-09T14:51:55.167 DEBUG:teuthology.orchestra.run.smithi043:> ip addr 2023-08-09T14:51:55.224 INFO:teuthology.orchestra.run.smithi043.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2023-08-09T14:51:55.225 INFO:teuthology.orchestra.run.smithi043.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2023-08-09T14:51:55.225 INFO:teuthology.orchestra.run.smithi043.stdout: inet 127.0.0.1/8 scope host lo 2023-08-09T14:51:55.225 INFO:teuthology.orchestra.run.smithi043.stdout: valid_lft forever preferred_lft forever 2023-08-09T14:51:55.225 INFO:teuthology.orchestra.run.smithi043.stdout: inet6 ::1/128 scope host 2023-08-09T14:51:55.225 INFO:teuthology.orchestra.run.smithi043.stdout: valid_lft forever preferred_lft forever 2023-08-09T14:51:55.225 INFO:teuthology.orchestra.run.smithi043.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-08-09T14:51:55.225 INFO:teuthology.orchestra.run.smithi043.stdout: link/ether 0c:c4:7a:6b:fd:9c brd ff:ff:ff:ff:ff:ff 2023-08-09T14:51:55.226 INFO:teuthology.orchestra.run.smithi043.stdout: altname enp7s0f0 2023-08-09T14:51:55.226 INFO:teuthology.orchestra.run.smithi043.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-08-09T14:51:55.226 INFO:teuthology.orchestra.run.smithi043.stdout: link/ether 0c:c4:7a:6b:fd:9d brd ff:ff:ff:ff:ff:ff 2023-08-09T14:51:55.226 INFO:teuthology.orchestra.run.smithi043.stdout: altname enp7s0f1 2023-08-09T14:51:55.226 INFO:teuthology.orchestra.run.smithi043.stdout:4: ens1f0: mtu 1500 qdisc mq state UP group default qlen 1000 2023-08-09T14:51:55.226 INFO:teuthology.orchestra.run.smithi043.stdout: link/ether 0c:c4:7a:bd:13:58 brd ff:ff:ff:ff:ff:ff 2023-08-09T14:51:55.226 INFO:teuthology.orchestra.run.smithi043.stdout: altname enp1s0f0 2023-08-09T14:51:55.226 INFO:teuthology.orchestra.run.smithi043.stdout: inet 172.21.15.43/20 brd 172.21.15.255 scope global dynamic noprefixroute ens1f0 2023-08-09T14:51:55.227 INFO:teuthology.orchestra.run.smithi043.stdout: valid_lft 37588sec preferred_lft 37588sec 2023-08-09T14:51:55.227 INFO:teuthology.orchestra.run.smithi043.stdout: inet6 fe80::ec4:7aff:febd:1358/64 scope link 2023-08-09T14:51:55.227 INFO:teuthology.orchestra.run.smithi043.stdout: valid_lft forever preferred_lft forever 2023-08-09T14:51:55.227 INFO:teuthology.orchestra.run.smithi043.stdout:5: ens1f1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-08-09T14:51:55.227 INFO:teuthology.orchestra.run.smithi043.stdout: link/ether 0c:c4:7a:bd:13:59 brd ff:ff:ff:ff:ff:ff 2023-08-09T14:51:55.227 INFO:teuthology.orchestra.run.smithi043.stdout: altname enp1s0f1 2023-08-09T14:51:55.227 INFO:teuthology.orchestra.run.smithi043.stdout:8: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2023-08-09T14:51:55.228 INFO:teuthology.orchestra.run.smithi043.stdout: link/ether 56:43:50:19:f4:0c brd ff:ff:ff:ff:ff:ff 2023-08-09T14:51:55.228 INFO:teuthology.orchestra.run.smithi043.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2023-08-09T14:51:55.228 INFO:teuthology.orchestra.run.smithi043.stdout: valid_lft forever preferred_lft forever 2023-08-09T14:51:55.228 INFO:teuthology.orchestra.run.smithi043.stdout: inet6 fe80::e494:65ff:fe92:a182/64 scope link 2023-08-09T14:51:55.228 INFO:teuthology.orchestra.run.smithi043.stdout: valid_lft forever preferred_lft forever 2023-08-09T14:51:55.228 INFO:teuthology.orchestra.run.smithi043.stdout:9: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2023-08-09T14:51:55.228 INFO:teuthology.orchestra.run.smithi043.stdout: link/ether 7e:93:26:fa:07:8f brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.0 2023-08-09T14:51:55.228 INFO:teuthology.orchestra.run.smithi043.stdout: inet6 fe80::7c93:26ff:fefa:78f/64 scope link 2023-08-09T14:51:55.229 INFO:teuthology.orchestra.run.smithi043.stdout: valid_lft forever preferred_lft forever 2023-08-09T14:51:55.229 INFO:teuthology.orchestra.run.smithi043.stdout:10: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2023-08-09T14:51:55.229 INFO:teuthology.orchestra.run.smithi043.stdout: link/ether 56:43:50:19:f4:0c brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.2 2023-08-09T14:51:55.229 INFO:teuthology.orchestra.run.smithi043.stdout: inet6 fe80::5443:50ff:fe19:f40c/64 scope link 2023-08-09T14:51:55.229 INFO:teuthology.orchestra.run.smithi043.stdout: valid_lft forever preferred_lft forever 2023-08-09T14:51:55.230 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-09T14:51:55.230 DEBUG:teuthology.orchestra.run.smithi043:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2023-08-09T14:51:55.327 INFO:teuthology.orchestra.run.smithi043.stdout:1 2023-08-09T14:51:55.331 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-09T14:51:55.332 DEBUG:teuthology.orchestra.run.smithi043:> route 2023-08-09T14:51:55.392 INFO:teuthology.orchestra.run.smithi043.stdout:Kernel IP routing table 2023-08-09T14:51:55.392 INFO:teuthology.orchestra.run.smithi043.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2023-08-09T14:51:55.392 INFO:teuthology.orchestra.run.smithi043.stdout:default _gateway 0.0.0.0 UG 100 0 0 ens1f0 2023-08-09T14:51:55.392 INFO:teuthology.orchestra.run.smithi043.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 ens1f0 2023-08-09T14:51:55.392 INFO:teuthology.orchestra.run.smithi043.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2023-08-09T14:51:55.393 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-09T14:51:55.393 DEBUG:teuthology.orchestra.run.smithi043:> (cd / && exec bash -c ' 2023-08-09T14:51:55.394 DEBUG:teuthology.orchestra.run.smithi043:> set -e 2023-08-09T14:51:55.394 DEBUG:teuthology.orchestra.run.smithi043:> sudo iptables -A FORWARD -o ens1f0 -i ceph-brx -j ACCEPT 2023-08-09T14:51:55.394 DEBUG:teuthology.orchestra.run.smithi043:> sudo iptables -A FORWARD -i ens1f0 -o ceph-brx -j ACCEPT 2023-08-09T14:51:55.394 DEBUG:teuthology.orchestra.run.smithi043:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o ens1f0 -j MASQUERADE 2023-08-09T14:51:55.395 DEBUG:teuthology.orchestra.run.smithi043:> ') 2023-08-09T14:51:55.607 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-09T14:51:55.607 DEBUG:teuthology.orchestra.run.smithi043:> ip netns list 2023-08-09T14:51:55.663 INFO:teuthology.orchestra.run.smithi043.stdout:ceph-ns--home-ubuntu-cephtest-mnt.2 (id: 1) 2023-08-09T14:51:55.664 INFO:teuthology.orchestra.run.smithi043.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2023-08-09T14:51:55.665 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2023-08-09T14:51:55.665 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-09T14:51:55.666 DEBUG:teuthology.orchestra.run.smithi043:> sudo ip link set brx.0 up 2023-08-09T14:51:55.762 INFO:tasks.cephfs.fuse_mount:Client client.0 config is {} 2023-08-09T14:51:55.762 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-08-09T14:51:55.763 DEBUG:teuthology.orchestra.run.smithi043:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2023-08-09T14:51:55.831 INFO:teuthology.orchestra.run.smithi043.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2023-08-09T14:51:55.831 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-08-09T14:51:55.832 DEBUG:teuthology.orchestra.run.smithi043:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2023-08-09T14:51:55.888 DEBUG:teuthology.orchestra.run.smithi043:> sudo modprobe fuse 2023-08-09T14:51:55.975 INFO:teuthology.orchestra.run:Running command with timeout 30 2023-08-09T14:51:55.976 DEBUG:teuthology.orchestra.run.smithi043:> sudo mount -t fusectl /sys/fs/fuse/connections /sys/fs/fuse/connections 2023-08-09T14:51:56.072 INFO:teuthology.orchestra.run.smithi043.stderr:mount: /sys/fs/fuse/connections: /sys/fs/fuse/connections already mounted or mount point busy. 2023-08-09T14:51:56.075 DEBUG:teuthology.orchestra.run:got remote process result: 32 2023-08-09T14:51:56.076 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-09T14:51:56.077 DEBUG:teuthology.orchestra.run.smithi043:> ls /sys/fs/fuse/connections 2023-08-09T14:51:56.134 INFO:tasks.cephfs.fuse_mount:Pre-mount connections: [] 2023-08-09T14:51:56.135 DEBUG:teuthology.orchestra.run.smithi043:> (cd /home/ubuntu/cephtest && exec sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-fuse -f --admin-socket '/var/run/ceph/$cluster-$name.$pid.asok' /home/ubuntu/cephtest/mnt.0 --id 0) 2023-08-09T14:51:56.141 DEBUG:teuthology.orchestra.run.smithi043:> sudo modprobe fuse 2023-08-09T14:51:56.222 INFO:teuthology.orchestra.run:Running command with timeout 30 2023-08-09T14:51:56.222 DEBUG:teuthology.orchestra.run.smithi043:> sudo mount -t fusectl /sys/fs/fuse/connections /sys/fs/fuse/connections 2023-08-09T14:51:56.302 INFO:teuthology.orchestra.run.smithi043.stderr:mount: /sys/fs/fuse/connections: /sys/fs/fuse/connections already mounted or mount point busy. 2023-08-09T14:51:56.306 DEBUG:teuthology.orchestra.run:got remote process result: 32 2023-08-09T14:51:56.307 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-09T14:51:56.307 DEBUG:teuthology.orchestra.run.smithi043:> ls /sys/fs/fuse/connections 2023-08-09T14:51:56.343 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi043.stderr:2023-08-09T14:51:56.341+0000 7f3fe4662480 -1 init, newargv = 0x5624adf78830 newargc=15 2023-08-09T14:51:56.343 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi043.stderr:ceph-fuse[140114]: starting ceph client 2023-08-09T14:51:56.351 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi043.stderr:ceph-fuse[140114]: starting fuse 2023-08-09T14:51:56.357 INFO:teuthology.orchestra.run.smithi043.stdout:40 2023-08-09T14:51:56.357 INFO:tasks.cephfs.fuse_mount:Post-mount connections: [40] 2023-08-09T14:51:56.357 DEBUG:teuthology.orchestra.run.smithi043:> sudo stdin-killer -- python3 -c ' 2023-08-09T14:51:56.358 DEBUG:teuthology.orchestra.run.smithi043:> import glob 2023-08-09T14:51:56.358 DEBUG:teuthology.orchestra.run.smithi043:> import re 2023-08-09T14:51:56.358 DEBUG:teuthology.orchestra.run.smithi043:> import os 2023-08-09T14:51:56.359 DEBUG:teuthology.orchestra.run.smithi043:> import subprocess 2023-08-09T14:51:56.359 DEBUG:teuthology.orchestra.run.smithi043:> 2023-08-09T14:51:56.359 DEBUG:teuthology.orchestra.run.smithi043:> def _find_admin_socket(client_name): 2023-08-09T14:51:56.360 DEBUG:teuthology.orchestra.run.smithi043:> asok_path = "/var/run/ceph/ceph-client.0.*.asok" 2023-08-09T14:51:56.360 DEBUG:teuthology.orchestra.run.smithi043:> files = glob.glob(asok_path) 2023-08-09T14:51:56.360 DEBUG:teuthology.orchestra.run.smithi043:> mountpoint = "/home/ubuntu/cephtest/mnt.0" 2023-08-09T14:51:56.361 DEBUG:teuthology.orchestra.run.smithi043:> 2023-08-09T14:51:56.361 DEBUG:teuthology.orchestra.run.smithi043:> # Given a non-glob path, it better be there 2023-08-09T14:51:56.361 DEBUG:teuthology.orchestra.run.smithi043:> if "*" not in asok_path: 2023-08-09T14:51:56.361 DEBUG:teuthology.orchestra.run.smithi043:> assert(len(files) == 1) 2023-08-09T14:51:56.362 DEBUG:teuthology.orchestra.run.smithi043:> return files[0] 2023-08-09T14:51:56.362 DEBUG:teuthology.orchestra.run.smithi043:> 2023-08-09T14:51:56.362 DEBUG:teuthology.orchestra.run.smithi043:> for f in files: 2023-08-09T14:51:56.363 DEBUG:teuthology.orchestra.run.smithi043:> pid = re.match(".*\.(\d+)\.asok$", f).group(1) 2023-08-09T14:51:56.363 DEBUG:teuthology.orchestra.run.smithi043:> if os.path.exists("/proc/{0}".format(pid)): 2023-08-09T14:51:56.363 DEBUG:teuthology.orchestra.run.smithi043:> with open("/proc/{0}/cmdline".format(pid), '"'"'r'"'"') as proc_f: 2023-08-09T14:51:56.363 DEBUG:teuthology.orchestra.run.smithi043:> contents = proc_f.read() 2023-08-09T14:51:56.364 DEBUG:teuthology.orchestra.run.smithi043:> if mountpoint in contents: 2023-08-09T14:51:56.364 DEBUG:teuthology.orchestra.run.smithi043:> return f 2023-08-09T14:51:56.364 DEBUG:teuthology.orchestra.run.smithi043:> raise RuntimeError("Client socket {0} not found".format(client_name)) 2023-08-09T14:51:56.365 DEBUG:teuthology.orchestra.run.smithi043:> 2023-08-09T14:51:56.365 DEBUG:teuthology.orchestra.run.smithi043:> print(_find_admin_socket("client.0")) 2023-08-09T14:51:56.365 DEBUG:teuthology.orchestra.run.smithi043:> ' 2023-08-09T14:51:56.567 INFO:teuthology.orchestra.run.smithi043.stdout:/var/run/ceph/ceph-client.0.140114.asok 2023-08-09T14:51:56.572 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:56 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2023-08-09T14:51:56.585 INFO:tasks.cephfs.fuse_mount:Found client admin socket at /var/run/ceph/ceph-client.0.140114.asok 2023-08-09T14:51:56.586 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-09T14:51:56.586 DEBUG:teuthology.orchestra.run.smithi043:> sudo ceph --admin-daemon /var/run/ceph/ceph-client.0.140114.asok status 2023-08-09T14:51:56.838 INFO:teuthology.orchestra.run.smithi043.stdout:{ 2023-08-09T14:51:56.838 INFO:teuthology.orchestra.run.smithi043.stdout: "metadata": { 2023-08-09T14:51:56.839 INFO:teuthology.orchestra.run.smithi043.stdout: "ceph_sha1": "668a86f232c9454b2ce5928d1432a29a72964d85", 2023-08-09T14:51:56.839 INFO:teuthology.orchestra.run.smithi043.stdout: "ceph_version": "ceph version 18.0.0-5389-g668a86f2 (668a86f232c9454b2ce5928d1432a29a72964d85) reef (dev)", 2023-08-09T14:51:56.839 INFO:teuthology.orchestra.run.smithi043.stdout: "entity_id": "0", 2023-08-09T14:51:56.839 INFO:teuthology.orchestra.run.smithi043.stdout: "hostname": "smithi043", 2023-08-09T14:51:56.839 INFO:teuthology.orchestra.run.smithi043.stdout: "mount_point": "/home/ubuntu/cephtest/mnt.0", 2023-08-09T14:51:56.839 INFO:teuthology.orchestra.run.smithi043.stdout: "pid": "140114", 2023-08-09T14:51:56.839 INFO:teuthology.orchestra.run.smithi043.stdout: "root": "/" 2023-08-09T14:51:56.839 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2023-08-09T14:51:56.840 INFO:teuthology.orchestra.run.smithi043.stdout: "dentry_count": 0, 2023-08-09T14:51:56.840 INFO:teuthology.orchestra.run.smithi043.stdout: "dentry_pinned_count": 0, 2023-08-09T14:51:56.840 INFO:teuthology.orchestra.run.smithi043.stdout: "id": 9819, 2023-08-09T14:51:56.840 INFO:teuthology.orchestra.run.smithi043.stdout: "inst": { 2023-08-09T14:51:56.840 INFO:teuthology.orchestra.run.smithi043.stdout: "name": { 2023-08-09T14:51:56.840 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "client", 2023-08-09T14:51:56.840 INFO:teuthology.orchestra.run.smithi043.stdout: "num": 9819 2023-08-09T14:51:56.841 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2023-08-09T14:51:56.841 INFO:teuthology.orchestra.run.smithi043.stdout: "addr": { 2023-08-09T14:51:56.841 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "v1", 2023-08-09T14:51:56.841 INFO:teuthology.orchestra.run.smithi043.stdout: "addr": "192.168.0.1:0", 2023-08-09T14:51:56.841 INFO:teuthology.orchestra.run.smithi043.stdout: "nonce": 1266703263 2023-08-09T14:51:56.841 INFO:teuthology.orchestra.run.smithi043.stdout: } 2023-08-09T14:51:56.841 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2023-08-09T14:51:56.841 INFO:teuthology.orchestra.run.smithi043.stdout: "addr": { 2023-08-09T14:51:56.842 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "v1", 2023-08-09T14:51:56.842 INFO:teuthology.orchestra.run.smithi043.stdout: "addr": "192.168.0.1:0", 2023-08-09T14:51:56.842 INFO:teuthology.orchestra.run.smithi043.stdout: "nonce": 1266703263 2023-08-09T14:51:56.842 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2023-08-09T14:51:56.842 INFO:teuthology.orchestra.run.smithi043.stdout: "inst_str": "client.9819 192.168.0.1:0/1266703263", 2023-08-09T14:51:56.842 INFO:teuthology.orchestra.run.smithi043.stdout: "addr_str": "192.168.0.1:0/1266703263", 2023-08-09T14:51:56.842 INFO:teuthology.orchestra.run.smithi043.stdout: "inode_count": 1, 2023-08-09T14:51:56.842 INFO:teuthology.orchestra.run.smithi043.stdout: "mds_epoch": 78, 2023-08-09T14:51:56.843 INFO:teuthology.orchestra.run.smithi043.stdout: "osd_epoch": 172, 2023-08-09T14:51:56.843 INFO:teuthology.orchestra.run.smithi043.stdout: "osd_epoch_barrier": 0, 2023-08-09T14:51:56.843 INFO:teuthology.orchestra.run.smithi043.stdout: "blocklisted": false, 2023-08-09T14:51:56.843 INFO:teuthology.orchestra.run.smithi043.stdout: "fs_name": "cephfs" 2023-08-09T14:51:56.843 INFO:teuthology.orchestra.run.smithi043.stdout:} 2023-08-09T14:51:56.853 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-09T14:51:56.854 DEBUG:teuthology.orchestra.run.smithi043:> stat --file-system '--printf=%T 2023-08-09T14:51:56.854 DEBUG:teuthology.orchestra.run.smithi043:> ' -- /home/ubuntu/cephtest/mnt.0 2023-08-09T14:51:56.916 INFO:teuthology.orchestra.run.smithi043.stdout:fuseblk 2023-08-09T14:51:56.916 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.0 2023-08-09T14:51:56.917 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-09T14:51:56.917 DEBUG:teuthology.orchestra.run.smithi043:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2023-08-09T14:51:57.018 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs volume ls 2023-08-09T14:51:57.247 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:57.245+0000 7f965d026640 1 Processor -- start 2023-08-09T14:51:57.247 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:57.245+0000 7f965d026640 1 -- start start 2023-08-09T14:51:57.248 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:57.247+0000 7f965d026640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f96580b2790 0x7f96580d2ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T14:51:57.248 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:57.247+0000 7f965d026640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f96580b3050 0x7f96580b3f60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T14:51:57.249 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:57.247+0000 7f965d026640 1 -- --> v1:172.21.15.203:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f96580ce090 con 0x7f96580b2b70 2023-08-09T14:51:57.249 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:57.247+0000 7f965d026640 1 -- --> v2:172.21.15.43:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f96580b3950 con 0x7f96580b3050 2023-08-09T14:51:57.249 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:57.247+0000 7f965d026640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f96580cd7f0 con 0x7f96580b2790 2023-08-09T14:51:57.249 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:57.247+0000 7f965759e640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f96580b3050 0x7f96580b3f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T14:51:57.249 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:57.247+0000 7f965759e640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f96580b3050 0x7f96580b3f60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:33868/0 (socket says 172.21.15.43:33868) 2023-08-09T14:51:57.249 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:57.247+0000 7f965759e640 1 -- 172.21.15.43:0/1783090969 learned_addr learned my addr 172.21.15.43:0/1783090969 (peer_addr_for_me v2:172.21.15.43:0/0) 2023-08-09T14:51:57.250 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:57.247+0000 7f9656d9d640 1 --2- 172.21.15.43:0/1783090969 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f96580b2790 0x7f96580d2ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T14:51:57.250 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:57.247+0000 7f965759e640 1 -- 172.21.15.43:0/1783090969 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f96580b2790 msgr2=0x7f96580d2ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-09T14:51:57.250 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:57.247+0000 7f965759e640 1 --2- 172.21.15.43:0/1783090969 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f96580b2790 0x7f96580d2ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T14:51:57.251 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:57.247+0000 7f965759e640 1 -- 172.21.15.43:0/1783090969 >> v1:172.21.15.203:6789/0 conn(0x7f96580b2b70 legacy=0x7f96580b5330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-09T14:51:57.251 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:57.248+0000 7f965759e640 1 -- 172.21.15.43:0/1783090969 --> v2:172.21.15.43:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f96580cdad0 con 0x7f96580b3050 2023-08-09T14:51:57.251 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:57.248+0000 7f9656d9d640 1 --2- 172.21.15.43:0/1783090969 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f96580b2790 0x7f96580d2ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-08-09T14:51:57.251 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:57.249+0000 7f965759e640 1 --2- 172.21.15.43:0/1783090969 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f96580b3050 0x7f96580b3f60 secure :-1 s=READY pgs=1901 cs=0 l=1 rev1=1 crypto rx=0x7f9650009e50 tx=0x7f965002bee0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8cc77cc84686c5c2 server_cookie=0 in_seq=0 out_seq=0 2023-08-09T14:51:57.251 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:57.249+0000 7f965659c640 1 -- 172.21.15.43:0/1783090969 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f965002ca80 con 0x7f96580b3050 2023-08-09T14:51:57.251 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:57.249+0000 7f965659c640 1 -- 172.21.15.43:0/1783090969 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f965002cbe0 con 0x7f96580b3050 2023-08-09T14:51:57.251 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:57.249+0000 7f965659c640 1 -- 172.21.15.43:0/1783090969 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f96500343f0 con 0x7f96580b3050 2023-08-09T14:51:57.252 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:57.250+0000 7f965d026640 1 -- 172.21.15.43:0/1783090969 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f96580b3050 msgr2=0x7f96580b3f60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T14:51:57.252 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:57.250+0000 7f965d026640 1 --2- 172.21.15.43:0/1783090969 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f96580b3050 0x7f96580b3f60 secure :-1 s=READY pgs=1901 cs=0 l=1 rev1=1 crypto rx=0x7f9650009e50 tx=0x7f965002bee0 comp rx=0 tx=0).stop 2023-08-09T14:51:57.253 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:57.250+0000 7f965d026640 1 -- 172.21.15.43:0/1783090969 shutdown_connections 2023-08-09T14:51:57.253 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:57.250+0000 7f965d026640 1 --2- 172.21.15.43:0/1783090969 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f96580b3050 0x7f96580b3f60 unknown :-1 s=CLOSED pgs=1901 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T14:51:57.253 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:57.250+0000 7f965d026640 1 --2- 172.21.15.43:0/1783090969 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f96580b2790 0x7f96580d2ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T14:51:57.253 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:57.250+0000 7f965d026640 1 -- 172.21.15.43:0/1783090969 >> 172.21.15.43:0/1783090969 conn(0x7f965809b7b0 msgr2=0x7f96580d24e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-09T14:51:57.253 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:57.250+0000 7f965d026640 1 -- 172.21.15.43:0/1783090969 shutdown_connections 2023-08-09T14:51:57.253 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:57.250+0000 7f965d026640 1 -- 172.21.15.43:0/1783090969 wait complete. 2023-08-09T14:51:57.254 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:57.252+0000 7f965d026640 1 Processor -- start 2023-08-09T14:51:57.254 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:57.252+0000 7f965d026640 1 -- start start 2023-08-09T14:51:57.255 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:57.253+0000 7f965d026640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f96580b2790 0x7f9658152220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T14:51:57.255 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:57.253+0000 7f965d026640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f96580b2b70 0x7f9658159fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T14:51:57.255 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:57.253+0000 7f965d026640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f96580b3050 0x7f96581542f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T14:51:57.255 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:57.253+0000 7f965d026640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9658156e40 con 0x7f96580b2790 2023-08-09T14:51:57.255 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:57.253+0000 7f965d026640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9658156870 con 0x7f96580b3050 2023-08-09T14:51:57.256 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:57.253+0000 7f965d026640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9658156aa0 con 0x7f96580b2b70 2023-08-09T14:51:57.256 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:57.253+0000 7f9656d9d640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f96580b2790 0x7f9658152220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T14:51:57.256 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:57.253+0000 7f9656d9d640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f96580b2790 0x7f9658152220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:33884/0 (socket says 172.21.15.43:33884) 2023-08-09T14:51:57.256 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:57.253+0000 7f9656d9d640 1 -- 172.21.15.43:0/3472930543 learned_addr learned my addr 172.21.15.43:0/3472930543 (peer_addr_for_me v2:172.21.15.43:0/0) 2023-08-09T14:51:57.256 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:57.253+0000 7f9656d9d640 1 -- 172.21.15.43:0/3472930543 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f96580b2b70 msgr2=0x7f9658159fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T14:51:57.256 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:57.253+0000 7f9656d9d640 1 --2- 172.21.15.43:0/3472930543 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f96580b2b70 0x7f9658159fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T14:51:57.256 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:57.253+0000 7f9656d9d640 1 -- 172.21.15.43:0/3472930543 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f96580b3050 msgr2=0x7f96581542f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T14:51:57.257 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:57.253+0000 7f9656d9d640 1 --2- 172.21.15.43:0/3472930543 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f96580b3050 0x7f96581542f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T14:51:57.257 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:57.253+0000 7f9656d9d640 1 -- 172.21.15.43:0/3472930543 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9650009b30 con 0x7f96580b2790 2023-08-09T14:51:57.257 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:57.254+0000 7f9656d9d640 1 --2- 172.21.15.43:0/3472930543 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f96580b2790 0x7f9658152220 secure :-1 s=READY pgs=1902 cs=0 l=1 rev1=1 crypto rx=0x7f964800c7f0 tx=0x7f964800ccb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-09T14:51:57.258 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:57.254+0000 7f964f7fe640 1 -- 172.21.15.43:0/3472930543 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9648007820 con 0x7f96580b2790 2023-08-09T14:51:57.258 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:57.254+0000 7f964f7fe640 1 -- 172.21.15.43:0/3472930543 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9648007980 con 0x7f96580b2790 2023-08-09T14:51:57.258 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:57.254+0000 7f965d026640 1 -- 172.21.15.43:0/3472930543 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9658152760 con 0x7f96580b2790 2023-08-09T14:51:57.258 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:57.254+0000 7f964f7fe640 1 -- 172.21.15.43:0/3472930543 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9648007cf0 con 0x7f96580b2790 2023-08-09T14:51:57.258 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:57.254+0000 7f965d026640 1 -- 172.21.15.43:0/3472930543 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9658155000 con 0x7f96580b2790 2023-08-09T14:51:57.258 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:57.255+0000 7f965d026640 1 -- 172.21.15.43:0/3472930543 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f96580739f0 con 0x7f96580b2790 2023-08-09T14:51:57.258 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:57.256+0000 7f964f7fe640 1 -- 172.21.15.43:0/3472930543 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 79) v1 ==== 88443+0+0 (secure 0 0 0) 0x7f96480113f0 con 0x7f96580b2790 2023-08-09T14:51:57.259 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:57.257+0000 7f964f7fe640 1 --2- 172.21.15.43:0/3472930543 >> [v2:172.21.15.203:6832/2566816332,v1:172.21.15.203:6833/2566816332] conn(0x7f963806a170 0x7f963806c630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T14:51:57.259 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:57.257+0000 7f964f7fe640 1 -- 172.21.15.43:0/3472930543 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(172..172 src has 1..172) v4 ==== 6426+0+0 (secure 0 0 0) 0x7f9648088d80 con 0x7f96580b2790 2023-08-09T14:51:57.259 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:57.258+0000 7f964ffff640 1 --2- 172.21.15.43:0/3472930543 >> [v2:172.21.15.203:6832/2566816332,v1:172.21.15.203:6833/2566816332] conn(0x7f963806a170 0x7f963806c630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T14:51:57.260 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:57.259+0000 7f964ffff640 1 --2- 172.21.15.43:0/3472930543 >> [v2:172.21.15.203:6832/2566816332,v1:172.21.15.203:6833/2566816332] conn(0x7f963806a170 0x7f963806c630 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f9640006fd0 tx=0x7f9640006bc0 comp rx=0 tx=0).ready entity=mgr.7765 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-09T14:51:57.262 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:57.261+0000 7f964f7fe640 1 -- 172.21.15.43:0/3472930543 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178066 (secure 0 0 0) 0x7f9648054040 con 0x7f96580b2790 2023-08-09T14:51:57.483 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:57.482+0000 7f965d026640 1 -- 172.21.15.43:0/3472930543 --> [v2:172.21.15.203:6832/2566816332,v1:172.21.15.203:6833/2566816332] -- mgr_command(tid 0: {"prefix": "fs volume ls", "target": ["mon-mgr", ""]}) v1 -- 0x7f96580d0930 con 0x7f963806a170 2023-08-09T14:51:57.487 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:57.486+0000 7f964f7fe640 1 -- 172.21.15.43:0/3472930543 <== mgr.7765 v2:172.21.15.203:6832/2566816332 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+41 (secure 0 0 0) 0x7f9640013430 con 0x7f963806a170 2023-08-09T14:51:57.487 INFO:teuthology.orchestra.run.smithi043.stdout:[ 2023-08-09T14:51:57.488 INFO:teuthology.orchestra.run.smithi043.stdout: { 2023-08-09T14:51:57.488 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "cephfs" 2023-08-09T14:51:57.488 INFO:teuthology.orchestra.run.smithi043.stdout: } 2023-08-09T14:51:57.488 INFO:teuthology.orchestra.run.smithi043.stdout:] 2023-08-09T14:51:57.490 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:57.488+0000 7f965d026640 1 -- 172.21.15.43:0/3472930543 >> [v2:172.21.15.203:6832/2566816332,v1:172.21.15.203:6833/2566816332] conn(0x7f963806a170 msgr2=0x7f963806c630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T14:51:57.490 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:57.488+0000 7f965d026640 1 --2- 172.21.15.43:0/3472930543 >> [v2:172.21.15.203:6832/2566816332,v1:172.21.15.203:6833/2566816332] conn(0x7f963806a170 0x7f963806c630 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f9640006fd0 tx=0x7f9640006bc0 comp rx=0 tx=0).stop 2023-08-09T14:51:57.491 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:57.489+0000 7f965d026640 1 -- 172.21.15.43:0/3472930543 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f96580b2790 msgr2=0x7f9658152220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T14:51:57.491 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:57.489+0000 7f965d026640 1 --2- 172.21.15.43:0/3472930543 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f96580b2790 0x7f9658152220 secure :-1 s=READY pgs=1902 cs=0 l=1 rev1=1 crypto rx=0x7f964800c7f0 tx=0x7f964800ccb0 comp rx=0 tx=0).stop 2023-08-09T14:51:57.491 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:57.489+0000 7f965d026640 1 -- 172.21.15.43:0/3472930543 shutdown_connections 2023-08-09T14:51:57.491 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:57.489+0000 7f965d026640 1 --2- 172.21.15.43:0/3472930543 >> [v2:172.21.15.203:6832/2566816332,v1:172.21.15.203:6833/2566816332] conn(0x7f963806a170 0x7f963806c630 unknown :-1 s=CLOSED pgs=71 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T14:51:57.491 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:57.489+0000 7f965d026640 1 --2- 172.21.15.43:0/3472930543 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f96580b3050 0x7f96581542f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T14:51:57.491 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:57.489+0000 7f965d026640 1 --2- 172.21.15.43:0/3472930543 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f96580b2b70 0x7f9658159fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T14:51:57.491 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:57.489+0000 7f965d026640 1 --2- 172.21.15.43:0/3472930543 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f96580b2790 0x7f9658152220 unknown :-1 s=CLOSED pgs=1902 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T14:51:57.492 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:57.489+0000 7f965d026640 1 -- 172.21.15.43:0/3472930543 >> 172.21.15.43:0/3472930543 conn(0x7f965809b7b0 msgr2=0x7f96580b1950 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-09T14:51:57.492 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:57.489+0000 7f965d026640 1 -- 172.21.15.43:0/3472930543 shutdown_connections 2023-08-09T14:51:57.492 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:57.489+0000 7f965d026640 1 -- 172.21.15.43:0/3472930543 wait complete. 2023-08-09T14:51:57.510 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph mgr module enable snap_schedule 2023-08-09T14:51:57.742 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:57.741+0000 7fb37e18b640 1 Processor -- start 2023-08-09T14:51:57.743 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:57.741+0000 7fb37e18b640 1 -- start start 2023-08-09T14:51:57.744 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:57.743+0000 7fb37e18b640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7fb37814f940 0x7fb37806c020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T14:51:57.745 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:57.743+0000 7fb37e18b640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fb37815ccd0 0x7fb3780c8e80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T14:51:57.745 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:57.743+0000 7fb37e18b640 1 -- --> v1:172.21.15.43:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb37806c600 con 0x7fb3780ccb20 2023-08-09T14:51:57.745 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:57.743+0000 7fb37e18b640 1 -- --> v2:172.21.15.43:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb37806c8e0 con 0x7fb37814f940 2023-08-09T14:51:57.745 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:57.743+0000 7fb37e18b640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb37806cbc0 con 0x7fb37815ccd0 2023-08-09T14:51:57.746 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:57.743+0000 7fb3777fe640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7fb37814f940 0x7fb37806c020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T14:51:57.746 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:57.744+0000 7fb3777fe640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7fb37814f940 0x7fb37806c020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:33900/0 (socket says 172.21.15.43:33900) 2023-08-09T14:51:57.746 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:57.744+0000 7fb3777fe640 1 -- 172.21.15.43:0/739464088 learned_addr learned my addr 172.21.15.43:0/739464088 (peer_addr_for_me v2:172.21.15.43:0/0) 2023-08-09T14:51:57.746 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:57.744+0000 7fb377fff640 1 --2- 172.21.15.43:0/739464088 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fb37815ccd0 0x7fb3780c8e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T14:51:57.746 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:57.744+0000 7fb3777fe640 1 -- 172.21.15.43:0/739464088 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fb37815ccd0 msgr2=0x7fb3780c8e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-09T14:51:57.746 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:57.744+0000 7fb3777fe640 1 --2- 172.21.15.43:0/739464088 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fb37815ccd0 0x7fb3780c8e80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T14:51:57.747 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:57.744+0000 7fb3777fe640 1 -- 172.21.15.43:0/739464088 >> v1:172.21.15.43:6789/0 conn(0x7fb3780ccb20 legacy=0x7fb37815c5c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T14:51:57.747 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:57.744+0000 7fb3777fe640 1 -- 172.21.15.43:0/739464088 --> v2:172.21.15.43:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb37806cea0 con 0x7fb37814f940 2023-08-09T14:51:57.747 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:57.745+0000 7fb3777fe640 1 --2- 172.21.15.43:0/739464088 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb37814f940 0x7fb37806c020 secure :-1 s=READY pgs=1904 cs=0 l=1 rev1=1 crypto rx=0x7fb368009e50 tx=0x7fb36802bf60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c2413218837e5c54 server_cookie=0 in_seq=0 out_seq=0 2023-08-09T14:51:57.748 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:57.746+0000 7fb376ffd640 1 -- 172.21.15.43:0/739464088 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb36802cb00 con 0x7fb37814f940 2023-08-09T14:51:57.748 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:57.746+0000 7fb376ffd640 1 -- 172.21.15.43:0/739464088 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb36802cc60 con 0x7fb37814f940 2023-08-09T14:51:57.748 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:57.746+0000 7fb376ffd640 1 -- 172.21.15.43:0/739464088 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb3680343f0 con 0x7fb37814f940 2023-08-09T14:51:57.748 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:57.746+0000 7fb37e18b640 1 -- 172.21.15.43:0/739464088 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb37814f940 msgr2=0x7fb37806c020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T14:51:57.748 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:57.746+0000 7fb37e18b640 1 --2- 172.21.15.43:0/739464088 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb37814f940 0x7fb37806c020 secure :-1 s=READY pgs=1904 cs=0 l=1 rev1=1 crypto rx=0x7fb368009e50 tx=0x7fb36802bf60 comp rx=0 tx=0).stop 2023-08-09T14:51:57.749 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:57.746+0000 7fb37e18b640 1 -- 172.21.15.43:0/739464088 shutdown_connections 2023-08-09T14:51:57.749 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:57.746+0000 7fb37e18b640 1 --2- 172.21.15.43:0/739464088 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fb37815ccd0 0x7fb3780c8e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T14:51:57.749 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:57.746+0000 7fb37e18b640 1 --2- 172.21.15.43:0/739464088 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb37814f940 0x7fb37806c020 unknown :-1 s=CLOSED pgs=1904 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T14:51:57.749 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:57.746+0000 7fb37e18b640 1 -- 172.21.15.43:0/739464088 >> 172.21.15.43:0/739464088 conn(0x7fb3780ce0a0 msgr2=0x7fb3780ce480 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-09T14:51:57.749 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:57.747+0000 7fb37e18b640 1 -- 172.21.15.43:0/739464088 shutdown_connections 2023-08-09T14:51:57.750 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:57.747+0000 7fb37e18b640 1 -- 172.21.15.43:0/739464088 wait complete. 2023-08-09T14:51:57.750 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:57.748+0000 7fb37e18b640 1 Processor -- start 2023-08-09T14:51:57.750 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:57.749+0000 7fb37e18b640 1 -- start start 2023-08-09T14:51:57.751 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:57.749+0000 7fb37e18b640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb3780ccb20 0x7fb378141740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T14:51:57.751 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:57.750+0000 7fb37e18b640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fb37814f940 0x7fb3780c3930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T14:51:57.751 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:57.750+0000 7fb37e18b640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fb37815ccd0 0x7fb3780c8d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T14:51:57.752 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:57.750+0000 7fb37e18b640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb3780c3e70 con 0x7fb3780ccb20 2023-08-09T14:51:57.752 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:57.750+0000 7fb37e18b640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb3780c92c0 con 0x7fb37815ccd0 2023-08-09T14:51:57.752 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:57.750+0000 7fb37e18b640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb378144810 con 0x7fb37814f940 2023-08-09T14:51:57.752 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:57.750+0000 7fb3777fe640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb3780ccb20 0x7fb378141740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T14:51:57.752 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:57.750+0000 7fb3777fe640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb3780ccb20 0x7fb378141740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:33902/0 (socket says 172.21.15.43:33902) 2023-08-09T14:51:57.752 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:57.750+0000 7fb3777fe640 1 -- 172.21.15.43:0/2048024587 learned_addr learned my addr 172.21.15.43:0/2048024587 (peer_addr_for_me v2:172.21.15.43:0/0) 2023-08-09T14:51:57.753 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:57.750+0000 7fb36effd640 1 --2- 172.21.15.43:0/2048024587 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fb37814f940 0x7fb3780c3930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T14:51:57.753 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:57.750+0000 7fb3777fe640 1 -- 172.21.15.43:0/2048024587 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fb37814f940 msgr2=0x7fb3780c3930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T14:51:57.754 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:57.750+0000 7fb3777fe640 1 --2- 172.21.15.43:0/2048024587 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fb37814f940 0x7fb3780c3930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T14:51:57.754 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:57.750+0000 7fb3777fe640 1 -- 172.21.15.43:0/2048024587 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fb37815ccd0 msgr2=0x7fb3780c8d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T14:51:57.754 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:57.750+0000 7fb3777fe640 1 --2- 172.21.15.43:0/2048024587 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fb37815ccd0 0x7fb3780c8d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T14:51:57.754 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:57.750+0000 7fb3777fe640 1 -- 172.21.15.43:0/2048024587 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb368009b30 con 0x7fb3780ccb20 2023-08-09T14:51:57.754 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:57.750+0000 7fb36effd640 1 --2- 172.21.15.43:0/2048024587 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fb37814f940 0x7fb3780c3930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-08-09T14:51:57.754 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:57.751+0000 7fb3777fe640 1 --2- 172.21.15.43:0/2048024587 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb3780ccb20 0x7fb378141740 secure :-1 s=READY pgs=1905 cs=0 l=1 rev1=1 crypto rx=0x7fb368000f80 tx=0x7fb368004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-09T14:51:57.754 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:57.751+0000 7fb374ff9640 1 -- 172.21.15.43:0/2048024587 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb36803e450 con 0x7fb3780ccb20 2023-08-09T14:51:57.754 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:57.751+0000 7fb374ff9640 1 -- 172.21.15.43:0/2048024587 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb36803e5f0 con 0x7fb3780ccb20 2023-08-09T14:51:57.755 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:57.751+0000 7fb374ff9640 1 -- 172.21.15.43:0/2048024587 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb36803e9a0 con 0x7fb3780ccb20 2023-08-09T14:51:57.755 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:57.751+0000 7fb37e18b640 1 -- 172.21.15.43:0/2048024587 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb378144af0 con 0x7fb3780ccb20 2023-08-09T14:51:57.755 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:57.751+0000 7fb37e18b640 1 -- 172.21.15.43:0/2048024587 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb3781450b0 con 0x7fb3780ccb20 2023-08-09T14:51:57.755 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:57.753+0000 7fb37e18b640 1 -- 172.21.15.43:0/2048024587 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb3780739f0 con 0x7fb3780ccb20 2023-08-09T14:51:57.756 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:57.753+0000 7fb374ff9640 1 -- 172.21.15.43:0/2048024587 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 79) v1 ==== 88443+0+0 (secure 0 0 0) 0x7fb36803eb00 con 0x7fb3780ccb20 2023-08-09T14:51:57.756 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:57.754+0000 7fb374ff9640 1 --2- 172.21.15.43:0/2048024587 >> [v2:172.21.15.203:6832/2566816332,v1:172.21.15.203:6833/2566816332] conn(0x7fb35806a170 0x7fb35806c630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T14:51:57.756 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:57.754+0000 7fb374ff9640 1 -- 172.21.15.43:0/2048024587 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(172..172 src has 1..172) v4 ==== 6426+0+0 (secure 0 0 0) 0x7fb3680ab8e0 con 0x7fb3780ccb20 2023-08-09T14:51:57.756 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:57.755+0000 7fb36effd640 1 --2- 172.21.15.43:0/2048024587 >> [v2:172.21.15.203:6832/2566816332,v1:172.21.15.203:6833/2566816332] conn(0x7fb35806a170 0x7fb35806c630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T14:51:57.757 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:57.756+0000 7fb36effd640 1 --2- 172.21.15.43:0/2048024587 >> [v2:172.21.15.203:6832/2566816332,v1:172.21.15.203:6833/2566816332] conn(0x7fb35806a170 0x7fb35806c630 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7fb37815b070 tx=0x7fb360009040 comp rx=0 tx=0).ready entity=mgr.7765 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-09T14:51:57.760 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:57.759+0000 7fb374ff9640 1 -- 172.21.15.43:0/2048024587 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178066 (secure 0 0 0) 0x7fb368076e80 con 0x7fb3780ccb20 2023-08-09T14:51:58.007 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:58.006+0000 7fb37e18b640 1 -- 172.21.15.43:0/2048024587 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "mgr module enable", "module": "snap_schedule"} v 0) v1 -- 0x7fb3781463e0 con 0x7fb3780ccb20 2023-08-09T14:51:58.899 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:58.898+0000 7fb374ff9640 1 -- 172.21.15.43:0/2048024587 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mgrmap(e 80) v1 ==== 88460+0+0 (secure 0 0 0) 0x7fb368077250 con 0x7fb3780ccb20 2023-08-09T14:51:58.903 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:58.902+0000 7fb374ff9640 1 -- 172.21.15.43:0/2048024587 <== mon.0 v2:172.21.15.43:3300/0 8 ==== mon_command_ack([{"prefix": "mgr module enable", "module": "snap_schedule"}]=0 v80) v1 ==== 92+0+0 (secure 0 0 0) 0x7fb368076e80 con 0x7fb3780ccb20 2023-08-09T14:51:58.908 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:58.906+0000 7fb37e18b640 1 -- 172.21.15.43:0/2048024587 >> [v2:172.21.15.203:6832/2566816332,v1:172.21.15.203:6833/2566816332] conn(0x7fb35806a170 msgr2=0x7fb35806c630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T14:51:58.908 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:58.906+0000 7fb37e18b640 1 --2- 172.21.15.43:0/2048024587 >> [v2:172.21.15.203:6832/2566816332,v1:172.21.15.203:6833/2566816332] conn(0x7fb35806a170 0x7fb35806c630 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7fb37815b070 tx=0x7fb360009040 comp rx=0 tx=0).stop 2023-08-09T14:51:58.908 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:58.906+0000 7fb37e18b640 1 -- 172.21.15.43:0/2048024587 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb3780ccb20 msgr2=0x7fb378141740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T14:51:58.909 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:58.906+0000 7fb37e18b640 1 --2- 172.21.15.43:0/2048024587 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb3780ccb20 0x7fb378141740 secure :-1 s=READY pgs=1905 cs=0 l=1 rev1=1 crypto rx=0x7fb368000f80 tx=0x7fb368004a40 comp rx=0 tx=0).stop 2023-08-09T14:51:58.909 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:58.907+0000 7fb37e18b640 1 -- 172.21.15.43:0/2048024587 shutdown_connections 2023-08-09T14:51:58.909 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:58.907+0000 7fb37e18b640 1 --2- 172.21.15.43:0/2048024587 >> [v2:172.21.15.203:6832/2566816332,v1:172.21.15.203:6833/2566816332] conn(0x7fb35806a170 0x7fb35806c630 unknown :-1 s=CLOSED pgs=72 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T14:51:58.909 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:58.907+0000 7fb37e18b640 1 --2- 172.21.15.43:0/2048024587 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fb37815ccd0 0x7fb3780c8d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T14:51:58.909 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:58.907+0000 7fb37e18b640 1 --2- 172.21.15.43:0/2048024587 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fb37814f940 0x7fb3780c3930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T14:51:58.909 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:58.907+0000 7fb37e18b640 1 --2- 172.21.15.43:0/2048024587 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb3780ccb20 0x7fb378141740 unknown :-1 s=CLOSED pgs=1905 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T14:51:58.909 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:58.907+0000 7fb37e18b640 1 -- 172.21.15.43:0/2048024587 >> 172.21.15.43:0/2048024587 conn(0x7fb3780ce0a0 msgr2=0x7fb3780cc060 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-09T14:51:58.910 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:58.907+0000 7fb37e18b640 1 -- 172.21.15.43:0/2048024587 shutdown_connections 2023-08-09T14:51:58.910 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:58.907+0000 7fb37e18b640 1 -- 172.21.15.43:0/2048024587 wait complete. 2023-08-09T14:51:58.928 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mgr mgr/snap_schedule/allow_m_granularity True 2023-08-09T14:51:59.113 INFO:tasks.ceph.mgr.x.smithi203.stderr:2023-08-09T14:51:59.112+0000 7fb8eb6d6040 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2023-08-09T14:51:59.121 INFO:tasks.ceph.mgr.y.smithi043.stderr:2023-08-09T14:51:59.119+0000 7f1c74493040 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2023-08-09T14:51:59.141 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:59.139+0000 7fe479698640 1 Processor -- start 2023-08-09T14:51:59.141 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:59.139+0000 7fe479698640 1 -- start start 2023-08-09T14:51:59.143 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:59.141+0000 7fe479698640 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7fe47415fd40 0x7fe4740c9220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T14:51:59.143 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:59.141+0000 7fe479698640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7fe4740c9760 0x7fe4740cca40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T14:51:59.143 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:59.141+0000 7fe479698640 1 -- --> v1:172.21.15.203:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe474070980 con 0x7fe4740c4850 2023-08-09T14:51:59.143 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:59.141+0000 7fe479698640 1 -- --> v2:172.21.15.43:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe4740caa30 con 0x7fe4740c9760 2023-08-09T14:51:59.144 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:59.141+0000 7fe479698640 1 -- --> v2:172.21.15.203:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe4740706a0 con 0x7fe47415fd40 2023-08-09T14:51:59.144 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:59.142+0000 7fe4737fe640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7fe4740c9760 0x7fe4740cca40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T14:51:59.144 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:59.142+0000 7fe4737fe640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7fe4740c9760 0x7fe4740cca40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:33914/0 (socket says 172.21.15.43:33914) 2023-08-09T14:51:59.144 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:59.142+0000 7fe4737fe640 1 -- 172.21.15.43:0/302289527 learned_addr learned my addr 172.21.15.43:0/302289527 (peer_addr_for_me v2:172.21.15.43:0/0) 2023-08-09T14:51:59.145 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:59.142+0000 7fe473fff640 1 --2- 172.21.15.43:0/302289527 >> v2:172.21.15.203:3300/0 conn(0x7fe47415fd40 0x7fe4740c9220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T14:51:59.145 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:59.142+0000 7fe4737fe640 1 -- 172.21.15.43:0/302289527 >> v2:172.21.15.203:3300/0 conn(0x7fe47415fd40 msgr2=0x7fe4740c9220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-09T14:51:59.145 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:59.142+0000 7fe4737fe640 1 --2- 172.21.15.43:0/302289527 >> v2:172.21.15.203:3300/0 conn(0x7fe47415fd40 0x7fe4740c9220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T14:51:59.145 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:59.142+0000 7fe4737fe640 1 -- 172.21.15.43:0/302289527 >> v1:172.21.15.203:6789/0 conn(0x7fe4740c4850 legacy=0x7fe4740c3be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-09T14:51:59.146 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:59.142+0000 7fe4737fe640 1 -- 172.21.15.43:0/302289527 --> v2:172.21.15.43:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe4740c36c0 con 0x7fe4740c9760 2023-08-09T14:51:59.146 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:59.143+0000 7fe4737fe640 1 --2- 172.21.15.43:0/302289527 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fe4740c9760 0x7fe4740cca40 secure :-1 s=READY pgs=1907 cs=0 l=1 rev1=1 crypto rx=0x7fe47406b500 tx=0x7fe46802b910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6f294b9bc3a2695d server_cookie=0 in_seq=0 out_seq=0 2023-08-09T14:51:59.147 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:59.144+0000 7fe472ffd640 1 -- 172.21.15.43:0/302289527 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe468031870 con 0x7fe4740c9760 2023-08-09T14:51:59.147 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:59.144+0000 7fe472ffd640 1 -- 172.21.15.43:0/302289527 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe4680319d0 con 0x7fe4740c9760 2023-08-09T14:51:59.147 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:59.144+0000 7fe472ffd640 1 -- 172.21.15.43:0/302289527 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe468031d60 con 0x7fe4740c9760 2023-08-09T14:51:59.147 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:59.144+0000 7fe479698640 1 -- 172.21.15.43:0/302289527 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fe4740c9760 msgr2=0x7fe4740cca40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T14:51:59.147 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:59.144+0000 7fe479698640 1 --2- 172.21.15.43:0/302289527 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fe4740c9760 0x7fe4740cca40 secure :-1 s=READY pgs=1907 cs=0 l=1 rev1=1 crypto rx=0x7fe47406b500 tx=0x7fe46802b910 comp rx=0 tx=0).stop 2023-08-09T14:51:59.147 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:59.145+0000 7fe479698640 1 -- 172.21.15.43:0/302289527 shutdown_connections 2023-08-09T14:51:59.148 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:59.145+0000 7fe479698640 1 --2- 172.21.15.43:0/302289527 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fe4740c9760 0x7fe4740cca40 unknown :-1 s=CLOSED pgs=1907 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T14:51:59.148 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:59.145+0000 7fe479698640 1 --2- 172.21.15.43:0/302289527 >> v2:172.21.15.203:3300/0 conn(0x7fe47415fd40 0x7fe4740c9220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T14:51:59.148 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:59.145+0000 7fe479698640 1 -- 172.21.15.43:0/302289527 >> 172.21.15.43:0/302289527 conn(0x7fe4740cb200 msgr2=0x7fe4740cb610 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-09T14:51:59.148 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:59.145+0000 7fe479698640 1 -- 172.21.15.43:0/302289527 shutdown_connections 2023-08-09T14:51:59.148 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:59.145+0000 7fe479698640 1 -- 172.21.15.43:0/302289527 wait complete. 2023-08-09T14:51:59.149 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:59.146+0000 7fe479698640 1 Processor -- start 2023-08-09T14:51:59.149 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:59.147+0000 7fe479698640 1 -- start start 2023-08-09T14:51:59.150 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:59.148+0000 7fe479698640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fe4740c4850 0x7fe47414fec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T14:51:59.150 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:59.148+0000 7fe479698640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fe4740c9760 0x7fe4740c73e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T14:51:59.150 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:59.148+0000 7fe479698640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fe47415fd40 0x7fe47414cb20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T14:51:59.150 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:59.148+0000 7fe479698640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe4740c87d0 con 0x7fe47415fd40 2023-08-09T14:51:59.150 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:59.148+0000 7fe479698640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe4740c7920 con 0x7fe4740c4850 2023-08-09T14:51:59.150 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:59.148+0000 7fe479698640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe47414d060 con 0x7fe4740c9760 2023-08-09T14:51:59.151 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:59.148+0000 7fe478e97640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fe47415fd40 0x7fe47414cb20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T14:51:59.151 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:59.148+0000 7fe478e97640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fe47415fd40 0x7fe47414cb20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:33926/0 (socket says 172.21.15.43:33926) 2023-08-09T14:51:59.151 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:59.148+0000 7fe478e97640 1 -- 172.21.15.43:0/1957189245 learned_addr learned my addr 172.21.15.43:0/1957189245 (peer_addr_for_me v2:172.21.15.43:0/0) 2023-08-09T14:51:59.151 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:59.148+0000 7fe473fff640 1 --2- 172.21.15.43:0/1957189245 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fe4740c4850 0x7fe47414fec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T14:51:59.152 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:59.148+0000 7fe478e97640 1 -- 172.21.15.43:0/1957189245 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fe4740c9760 msgr2=0x7fe4740c73e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T14:51:59.152 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:59.148+0000 7fe478e97640 1 --2- 172.21.15.43:0/1957189245 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fe4740c9760 0x7fe4740c73e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T14:51:59.152 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:59.148+0000 7fe478e97640 1 -- 172.21.15.43:0/1957189245 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fe4740c4850 msgr2=0x7fe47414fec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T14:51:59.153 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:59.148+0000 7fe478e97640 1 --2- 172.21.15.43:0/1957189245 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fe4740c4850 0x7fe47414fec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T14:51:59.153 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:59.148+0000 7fe478e97640 1 -- 172.21.15.43:0/1957189245 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe4680094d0 con 0x7fe47415fd40 2023-08-09T14:51:59.153 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:59.149+0000 7fe478e97640 1 --2- 172.21.15.43:0/1957189245 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fe47415fd40 0x7fe47414cb20 secure :-1 s=READY pgs=1908 cs=0 l=1 rev1=1 crypto rx=0x7fe46c00c7b0 tx=0x7fe46c00cc70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-09T14:51:59.153 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:59.149+0000 7fe470ff9640 1 -- 172.21.15.43:0/1957189245 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe46c004b60 con 0x7fe47415fd40 2023-08-09T14:51:59.153 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:59.149+0000 7fe470ff9640 1 -- 172.21.15.43:0/1957189245 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe46c004cc0 con 0x7fe47415fd40 2023-08-09T14:51:59.153 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:59.149+0000 7fe470ff9640 1 -- 172.21.15.43:0/1957189245 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe46c00aa90 con 0x7fe47415fd40 2023-08-09T14:51:59.153 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:59.149+0000 7fe479698640 1 -- 172.21.15.43:0/1957189245 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe4740c7b50 con 0x7fe47415fd40 2023-08-09T14:51:59.154 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:59.149+0000 7fe479698640 1 -- 172.21.15.43:0/1957189245 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe4740c8110 con 0x7fe47415fd40 2023-08-09T14:51:59.154 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:59.150+0000 7fe479698640 1 -- 172.21.15.43:0/1957189245 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe47404e390 con 0x7fe47415fd40 2023-08-09T14:51:59.154 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:59.151+0000 7fe470ff9640 1 -- 172.21.15.43:0/1957189245 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 80) v1 ==== 88460+0+0 (secure 0 0 0) 0x7fe46c00abf0 con 0x7fe47415fd40 2023-08-09T14:51:59.155 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:59.152+0000 7fe470ff9640 1 --2- 172.21.15.43:0/1957189245 >> [v2:172.21.15.203:6832/2566816332,v1:172.21.15.203:6833/2566816332] conn(0x7fe46006a1c0 0x7fe46006c680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T14:51:59.155 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:59.152+0000 7fe470ff9640 1 -- 172.21.15.43:0/1957189245 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(172..172 src has 1..172) v4 ==== 6426+0+0 (secure 0 0 0) 0x7fe46c088b60 con 0x7fe47415fd40 2023-08-09T14:51:59.155 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:59.153+0000 7fe473fff640 1 -- 172.21.15.43:0/1957189245 >> [v2:172.21.15.203:6832/2566816332,v1:172.21.15.203:6833/2566816332] conn(0x7fe46006a1c0 msgr2=0x7fe46006c680 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.203:6832/2566816332 2023-08-09T14:51:59.155 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:59.153+0000 7fe473fff640 1 --2- 172.21.15.43:0/1957189245 >> [v2:172.21.15.203:6832/2566816332,v1:172.21.15.203:6833/2566816332] conn(0x7fe46006a1c0 0x7fe46006c680 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-08-09T14:51:59.158 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:59.157+0000 7fe470ff9640 1 -- 172.21.15.43:0/1957189245 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178066 (secure 0 0 0) 0x7fe46c053f30 con 0x7fe47415fd40 2023-08-09T14:51:59.354 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:59.352+0000 7fe479698640 1 -- 172.21.15.43:0/1957189245 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command([{prefix=config set, name=mgr/snap_schedule/allow_m_granularity}] v 0) v1 -- 0x7fe4740c9420 con 0x7fe47415fd40 2023-08-09T14:51:59.355 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:59.354+0000 7fe473fff640 1 -- 172.21.15.43:0/1957189245 >> [v2:172.21.15.203:6832/2566816332,v1:172.21.15.203:6833/2566816332] conn(0x7fe46006a1c0 msgr2=0x7fe46006c680 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.203:6832/2566816332 2023-08-09T14:51:59.356 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:59.354+0000 7fe473fff640 1 --2- 172.21.15.43:0/1957189245 >> [v2:172.21.15.203:6832/2566816332,v1:172.21.15.203:6833/2566816332] conn(0x7fe46006a1c0 0x7fe46006c680 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2023-08-09T14:51:59.361 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:59.360+0000 7fe470ff9640 1 -- 172.21.15.43:0/1957189245 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mgr/snap_schedule/allow_m_granularity}]=0 v53)=0 v53) v1 ==== 138+0+0 (secure 0 0 0) 0x7fe46c057550 con 0x7fe47415fd40 2023-08-09T14:51:59.364 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:59.363+0000 7fe479698640 1 -- 172.21.15.43:0/1957189245 >> [v2:172.21.15.203:6832/2566816332,v1:172.21.15.203:6833/2566816332] conn(0x7fe46006a1c0 msgr2=0x7fe46006c680 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-08-09T14:51:59.365 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:59.363+0000 7fe479698640 1 --2- 172.21.15.43:0/1957189245 >> [v2:172.21.15.203:6832/2566816332,v1:172.21.15.203:6833/2566816332] conn(0x7fe46006a1c0 0x7fe46006c680 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T14:51:59.365 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:59.363+0000 7fe479698640 1 -- 172.21.15.43:0/1957189245 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fe47415fd40 msgr2=0x7fe47414cb20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T14:51:59.365 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:59.363+0000 7fe479698640 1 --2- 172.21.15.43:0/1957189245 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fe47415fd40 0x7fe47414cb20 secure :-1 s=READY pgs=1908 cs=0 l=1 rev1=1 crypto rx=0x7fe46c00c7b0 tx=0x7fe46c00cc70 comp rx=0 tx=0).stop 2023-08-09T14:51:59.365 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:59.363+0000 7fe479698640 1 -- 172.21.15.43:0/1957189245 shutdown_connections 2023-08-09T14:51:59.365 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:59.363+0000 7fe479698640 1 --2- 172.21.15.43:0/1957189245 >> [v2:172.21.15.203:6832/2566816332,v1:172.21.15.203:6833/2566816332] conn(0x7fe46006a1c0 0x7fe46006c680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T14:51:59.365 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:59.363+0000 7fe479698640 1 --2- 172.21.15.43:0/1957189245 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fe47415fd40 0x7fe47414cb20 unknown :-1 s=CLOSED pgs=1908 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T14:51:59.366 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:59.363+0000 7fe479698640 1 --2- 172.21.15.43:0/1957189245 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fe4740c9760 0x7fe4740c73e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T14:51:59.366 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:59.363+0000 7fe479698640 1 --2- 172.21.15.43:0/1957189245 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fe4740c4850 0x7fe47414fec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T14:51:59.366 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:59.363+0000 7fe479698640 1 -- 172.21.15.43:0/1957189245 >> 172.21.15.43:0/1957189245 conn(0x7fe4740cb200 msgr2=0x7fe4740af860 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-09T14:51:59.366 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:59.363+0000 7fe479698640 1 -- 172.21.15.43:0/1957189245 shutdown_connections 2023-08-09T14:51:59.366 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:59.363+0000 7fe479698640 1 -- 172.21.15.43:0/1957189245 wait complete. 2023-08-09T14:51:59.384 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mgr mgr/snap_schedule/dump_on_update True 2023-08-09T14:51:59.607 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:59.605+0000 7fe41561f640 1 Processor -- start 2023-08-09T14:51:59.607 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:59.606+0000 7fe41561f640 1 -- start start 2023-08-09T14:51:59.609 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:59.607+0000 7fe41561f640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fe41015c6f0 0x7fe4100ac630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T14:51:59.609 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:59.607+0000 7fe41561f640 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7fe4100acb70 0x7fe4100b23a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T14:51:59.609 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:59.607+0000 7fe41561f640 1 -- --> v1:172.21.15.203:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe4100b1ef0 con 0x7fe41015c310 2023-08-09T14:51:59.609 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:59.607+0000 7fe41561f640 1 -- --> v2:172.21.15.203:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe4100ad000 con 0x7fe4100acb70 2023-08-09T14:51:59.610 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:59.607+0000 7fe41561f640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe41006c8a0 con 0x7fe41015c6f0 2023-08-09T14:51:59.610 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:59.608+0000 7fe40f7fe640 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7fe4100acb70 0x7fe4100b23a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T14:51:59.611 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:59.608+0000 7fe40f7fe640 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7fe4100acb70 0x7fe4100b23a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.203:3300/0 says I am v2:172.21.15.43:59642/0 (socket says 172.21.15.43:59642) 2023-08-09T14:51:59.611 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:59.608+0000 7fe40e7fc640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fe41015c6f0 0x7fe4100ac630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T14:51:59.611 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:59.608+0000 7fe40f7fe640 1 -- 172.21.15.43:0/3409552690 learned_addr learned my addr 172.21.15.43:0/3409552690 (peer_addr_for_me v2:172.21.15.43:0/0) 2023-08-09T14:51:59.611 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:59.609+0000 7fe40f7fe640 1 -- 172.21.15.43:0/3409552690 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fe41015c6f0 msgr2=0x7fe4100ac630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-09T14:51:59.611 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:59.609+0000 7fe40f7fe640 1 --2- 172.21.15.43:0/3409552690 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fe41015c6f0 0x7fe4100ac630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T14:51:59.612 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:59.609+0000 7fe40f7fe640 1 -- 172.21.15.43:0/3409552690 >> v1:172.21.15.203:6789/0 conn(0x7fe41015c310 legacy=0x7fe41006c190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T14:51:59.612 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:59.609+0000 7fe40dffb640 1 -- 172.21.15.43:0/3409552690 <== mon.1 v1:172.21.15.203:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 551008425 0 0) 0x7fe4100b1ef0 con 0x7fe41015c310 2023-08-09T14:51:59.612 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:59.609+0000 7fe40f7fe640 1 -- 172.21.15.43:0/3409552690 --> v2:172.21.15.203:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe41006cb70 con 0x7fe4100acb70 2023-08-09T14:51:59.612 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:59.611+0000 7fe40f7fe640 1 --2- 172.21.15.43:0/3409552690 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fe4100acb70 0x7fe4100b23a0 secure :-1 s=READY pgs=695 cs=0 l=1 rev1=1 crypto rx=0x7fe408009e50 tx=0x7fe40802bf60 comp rx=0 tx=0).ready entity=mon.1 client_cookie=7685a846a7c0bb58 server_cookie=0 in_seq=0 out_seq=0 2023-08-09T14:51:59.613 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:59.611+0000 7fe40dffb640 1 -- 172.21.15.43:0/3409552690 <== mon.1 v2:172.21.15.203:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe40802cb00 con 0x7fe4100acb70 2023-08-09T14:51:59.613 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:59.611+0000 7fe40dffb640 1 -- 172.21.15.43:0/3409552690 <== mon.1 v2:172.21.15.203:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe40802cc60 con 0x7fe4100acb70 2023-08-09T14:51:59.614 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:59.611+0000 7fe40dffb640 1 -- 172.21.15.43:0/3409552690 <== mon.1 v2:172.21.15.203:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe4080343f0 con 0x7fe4100acb70 2023-08-09T14:51:59.614 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:59.612+0000 7fe41561f640 1 -- 172.21.15.43:0/3409552690 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fe4100acb70 msgr2=0x7fe4100b23a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T14:51:59.614 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:59.612+0000 7fe41561f640 1 --2- 172.21.15.43:0/3409552690 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fe4100acb70 0x7fe4100b23a0 secure :-1 s=READY pgs=695 cs=0 l=1 rev1=1 crypto rx=0x7fe408009e50 tx=0x7fe40802bf60 comp rx=0 tx=0).stop 2023-08-09T14:51:59.615 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:59.612+0000 7fe41561f640 1 -- 172.21.15.43:0/3409552690 shutdown_connections 2023-08-09T14:51:59.615 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:59.612+0000 7fe41561f640 1 --2- 172.21.15.43:0/3409552690 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fe4100acb70 0x7fe4100b23a0 unknown :-1 s=CLOSED pgs=695 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T14:51:59.615 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:59.612+0000 7fe41561f640 1 --2- 172.21.15.43:0/3409552690 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fe41015c6f0 0x7fe4100ac630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T14:51:59.615 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:59.612+0000 7fe41561f640 1 -- 172.21.15.43:0/3409552690 >> 172.21.15.43:0/3409552690 conn(0x7fe4100697c0 msgr2=0x7fe410069ba0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-09T14:51:59.615 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:59.612+0000 7fe41561f640 1 -- 172.21.15.43:0/3409552690 shutdown_connections 2023-08-09T14:51:59.615 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:59.612+0000 7fe41561f640 1 -- 172.21.15.43:0/3409552690 wait complete. 2023-08-09T14:51:59.617 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:59.614+0000 7fe41561f640 1 Processor -- start 2023-08-09T14:51:59.617 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:59.614+0000 7fe41561f640 1 -- start start 2023-08-09T14:51:59.617 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:59.615+0000 7fe41561f640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fe4100acb70 0x7fe41014cef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T14:51:59.617 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:59.615+0000 7fe41561f640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fe41015c310 0x7fe4100afe00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T14:51:59.617 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:59.615+0000 7fe41561f640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fe41015c6f0 0x7fe41015d510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T14:51:59.617 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:59.615+0000 7fe41561f640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe4100b0df0 con 0x7fe41015c6f0 2023-08-09T14:51:59.618 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:59.615+0000 7fe41561f640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe4100b0340 con 0x7fe41015c310 2023-08-09T14:51:59.618 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:59.615+0000 7fe41561f640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe410154aa0 con 0x7fe4100acb70 2023-08-09T14:51:59.618 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:59.615+0000 7fe40f7fe640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fe41015c6f0 0x7fe41015d510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T14:51:59.618 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:59.616+0000 7fe40f7fe640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fe41015c6f0 0x7fe41015d510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:33930/0 (socket says 172.21.15.43:33930) 2023-08-09T14:51:59.618 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:59.616+0000 7fe40f7fe640 1 -- 172.21.15.43:0/3964192443 learned_addr learned my addr 172.21.15.43:0/3964192443 (peer_addr_for_me v2:172.21.15.43:0/0) 2023-08-09T14:51:59.619 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:59.616+0000 7fe40effd640 1 --2- 172.21.15.43:0/3964192443 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fe4100acb70 0x7fe41014cef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T14:51:59.620 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:59.616+0000 7fe40e7fc640 1 --2- 172.21.15.43:0/3964192443 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fe41015c310 0x7fe4100afe00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T14:51:59.620 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:59.616+0000 7fe40f7fe640 1 -- 172.21.15.43:0/3964192443 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fe4100acb70 msgr2=0x7fe41014cef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T14:51:59.620 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:59.616+0000 7fe40f7fe640 1 --2- 172.21.15.43:0/3964192443 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fe4100acb70 0x7fe41014cef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T14:51:59.620 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:59.616+0000 7fe40f7fe640 1 -- 172.21.15.43:0/3964192443 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fe41015c310 msgr2=0x7fe4100afe00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T14:51:59.620 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:59.616+0000 7fe40f7fe640 1 --2- 172.21.15.43:0/3964192443 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fe41015c310 0x7fe4100afe00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T14:51:59.620 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:59.616+0000 7fe40f7fe640 1 -- 172.21.15.43:0/3964192443 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe408009b30 con 0x7fe41015c6f0 2023-08-09T14:51:59.620 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:59.616+0000 7fe40effd640 1 --2- 172.21.15.43:0/3964192443 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fe4100acb70 0x7fe41014cef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-08-09T14:51:59.621 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:59.616+0000 7fe40f7fe640 1 --2- 172.21.15.43:0/3964192443 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fe41015c6f0 0x7fe41015d510 secure :-1 s=READY pgs=1909 cs=0 l=1 rev1=1 crypto rx=0x7fe408036040 tx=0x7fe408004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-09T14:51:59.621 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:59.617+0000 7fe3f77fe640 1 -- 172.21.15.43:0/3964192443 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe408009510 con 0x7fe41015c6f0 2023-08-09T14:51:59.621 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:59.617+0000 7fe3f77fe640 1 -- 172.21.15.43:0/3964192443 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe40803e450 con 0x7fe41015c6f0 2023-08-09T14:51:59.621 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:59.617+0000 7fe3f77fe640 1 -- 172.21.15.43:0/3964192443 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe40803e5e0 con 0x7fe41015c6f0 2023-08-09T14:51:59.621 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:59.617+0000 7fe41561f640 1 -- 172.21.15.43:0/3964192443 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe41015da50 con 0x7fe41015c6f0 2023-08-09T14:51:59.621 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:59.617+0000 7fe41561f640 1 -- 172.21.15.43:0/3964192443 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe4100b0830 con 0x7fe41015c6f0 2023-08-09T14:51:59.621 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:59.618+0000 7fe41561f640 1 -- 172.21.15.43:0/3964192443 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe410154c50 con 0x7fe41015c6f0 2023-08-09T14:51:59.622 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:59.620+0000 7fe3f77fe640 1 -- 172.21.15.43:0/3964192443 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 80) v1 ==== 88460+0+0 (secure 0 0 0) 0x7fe40803e740 con 0x7fe41015c6f0 2023-08-09T14:51:59.623 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:59.621+0000 7fe3f77fe640 1 --2- 172.21.15.43:0/3964192443 >> [v2:172.21.15.203:6832/2566816332,v1:172.21.15.203:6833/2566816332] conn(0x7fe3fc06a1c0 0x7fe3fc06c680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T14:51:59.623 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:59.621+0000 7fe3f77fe640 1 -- 172.21.15.43:0/3964192443 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(172..172 src has 1..172) v4 ==== 6426+0+0 (secure 0 0 0) 0x7fe4080ab2d0 con 0x7fe41015c6f0 2023-08-09T14:51:59.623 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:59.621+0000 7fe40effd640 1 -- 172.21.15.43:0/3964192443 >> [v2:172.21.15.203:6832/2566816332,v1:172.21.15.203:6833/2566816332] conn(0x7fe3fc06a1c0 msgr2=0x7fe3fc06c680 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.203:6832/2566816332 2023-08-09T14:51:59.623 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:59.621+0000 7fe40effd640 1 --2- 172.21.15.43:0/3964192443 >> [v2:172.21.15.203:6832/2566816332,v1:172.21.15.203:6833/2566816332] conn(0x7fe3fc06a1c0 0x7fe3fc06c680 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-08-09T14:51:59.627 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:59.626+0000 7fe3f77fe640 1 -- 172.21.15.43:0/3964192443 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178066 (secure 0 0 0) 0x7fe410154c50 con 0x7fe41015c6f0 2023-08-09T14:51:59.811 INFO:tasks.ceph.mgr.x.smithi203.stderr:2023-08-09T14:51:59.810+0000 7fb8eb6d6040 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2023-08-09T14:51:59.818 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:59.817+0000 7fe41561f640 1 -- 172.21.15.43:0/3964192443 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command([{prefix=config set, name=mgr/snap_schedule/dump_on_update}] v 0) v1 -- 0x7fe410154c50 con 0x7fe41015c6f0 2023-08-09T14:51:59.823 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:59.822+0000 7fe40effd640 1 -- 172.21.15.43:0/3964192443 >> [v2:172.21.15.203:6832/2566816332,v1:172.21.15.203:6833/2566816332] conn(0x7fe3fc06a1c0 msgr2=0x7fe3fc06c680 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.203:6832/2566816332 2023-08-09T14:51:59.823 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:59.822+0000 7fe40effd640 1 --2- 172.21.15.43:0/3964192443 >> [v2:172.21.15.203:6832/2566816332,v1:172.21.15.203:6833/2566816332] conn(0x7fe3fc06a1c0 0x7fe3fc06c680 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2023-08-09T14:51:59.824 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:59.823+0000 7fe3f77fe640 1 -- 172.21.15.43:0/3964192443 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mgr/snap_schedule/dump_on_update}]=0 v54)=0 v54) v1 ==== 133+0+0 (secure 0 0 0) 0x7fe410154c50 con 0x7fe41015c6f0 2023-08-09T14:51:59.827 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:59.826+0000 7fe41561f640 1 -- 172.21.15.43:0/3964192443 >> [v2:172.21.15.203:6832/2566816332,v1:172.21.15.203:6833/2566816332] conn(0x7fe3fc06a1c0 msgr2=0x7fe3fc06c680 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-08-09T14:51:59.827 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:59.826+0000 7fe41561f640 1 --2- 172.21.15.43:0/3964192443 >> [v2:172.21.15.203:6832/2566816332,v1:172.21.15.203:6833/2566816332] conn(0x7fe3fc06a1c0 0x7fe3fc06c680 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T14:51:59.828 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:59.826+0000 7fe41561f640 1 -- 172.21.15.43:0/3964192443 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fe41015c6f0 msgr2=0x7fe41015d510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T14:51:59.828 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:59.826+0000 7fe41561f640 1 --2- 172.21.15.43:0/3964192443 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fe41015c6f0 0x7fe41015d510 secure :-1 s=READY pgs=1909 cs=0 l=1 rev1=1 crypto rx=0x7fe408036040 tx=0x7fe408004a40 comp rx=0 tx=0).stop 2023-08-09T14:51:59.828 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:59.826+0000 7fe41561f640 1 -- 172.21.15.43:0/3964192443 shutdown_connections 2023-08-09T14:51:59.828 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:59.826+0000 7fe41561f640 1 --2- 172.21.15.43:0/3964192443 >> [v2:172.21.15.203:6832/2566816332,v1:172.21.15.203:6833/2566816332] conn(0x7fe3fc06a1c0 0x7fe3fc06c680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T14:51:59.828 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:59.826+0000 7fe41561f640 1 --2- 172.21.15.43:0/3964192443 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fe41015c6f0 0x7fe41015d510 unknown :-1 s=CLOSED pgs=1909 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T14:51:59.828 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:59.826+0000 7fe41561f640 1 --2- 172.21.15.43:0/3964192443 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fe41015c310 0x7fe4100afe00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T14:51:59.828 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:59.826+0000 7fe41561f640 1 --2- 172.21.15.43:0/3964192443 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fe4100acb70 0x7fe41014cef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T14:51:59.829 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:59.826+0000 7fe41561f640 1 -- 172.21.15.43:0/3964192443 >> 172.21.15.43:0/3964192443 conn(0x7fe4100697c0 msgr2=0x7fe410159cc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-09T14:51:59.829 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:59.826+0000 7fe41561f640 1 -- 172.21.15.43:0/3964192443 shutdown_connections 2023-08-09T14:51:59.829 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:51:59.826+0000 7fe41561f640 1 -- 172.21.15.43:0/3964192443 wait complete. 2023-08-09T14:51:59.846 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-09T14:51:59.847 DEBUG:teuthology.orchestra.run.smithi043:> (cd /home/ubuntu/cephtest/mnt.0 && exec mkdir -p snap_test_dir1) 2023-08-09T14:51:59.880 INFO:tasks.ceph.mgr.y.smithi043.stderr:2023-08-09T14:51:59.879+0000 7f1c74493040 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2023-08-09T14:51:59.910 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs snap-schedule add --fs cephfs snap_test_dir1 1h 2023-08-09T14:51:59.934 INFO:tasks.ceph.mgr.x.smithi203.stderr:2023-08-09T14:51:59.932+0000 7fb8eb6d6040 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2023-08-09T14:51:59.989 INFO:tasks.ceph.mgr.x.smithi203.stderr:2023-08-09T14:51:59.988+0000 7fb8eb6d6040 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2023-08-09T14:52:00.010 INFO:tasks.ceph.mgr.y.smithi043.stderr:2023-08-09T14:52:00.008+0000 7f1c74493040 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2023-08-09T14:52:00.038 INFO:tasks.ceph.mgr.x.smithi203.stderr:2023-08-09T14:52:00.037+0000 7fb8eb6d6040 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2023-08-09T14:52:00.070 INFO:tasks.ceph.mgr.y.smithi043.stderr:2023-08-09T14:52:00.069+0000 7f1c74493040 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2023-08-09T14:52:00.123 INFO:tasks.ceph.mgr.y.smithi043.stderr:2023-08-09T14:52:00.121+0000 7f1c74493040 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2023-08-09T14:52:00.169 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:00.167+0000 7f91244c5640 1 Processor -- start 2023-08-09T14:52:00.169 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:00.168+0000 7f91244c5640 1 -- start start 2023-08-09T14:52:00.171 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:00.169+0000 7f91244c5640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f911c159390 0x7f911c0cae90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T14:52:00.171 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:00.169+0000 7f91244c5640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f911c06afe0 0x7f911c070600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T14:52:00.171 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:00.169+0000 7f91244c5640 1 -- --> v1:172.21.15.203:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f911c0cd4d0 con 0x7f911c0cb3d0 2023-08-09T14:52:00.171 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:00.169+0000 7f91244c5640 1 -- --> v2:172.21.15.43:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f911c0711e0 con 0x7f911c159390 2023-08-09T14:52:00.171 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:00.169+0000 7f91244c5640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f911c0714c0 con 0x7f911c06afe0 2023-08-09T14:52:00.172 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:00.170+0000 7f9122261640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f911c159390 0x7f911c0cae90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T14:52:00.173 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:00.170+0000 7f9122261640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f911c159390 0x7f911c0cae90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:33932/0 (socket says 172.21.15.43:33932) 2023-08-09T14:52:00.173 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:00.170+0000 7f9122261640 1 -- 172.21.15.43:0/3162650847 learned_addr learned my addr 172.21.15.43:0/3162650847 (peer_addr_for_me v2:172.21.15.43:0/0) 2023-08-09T14:52:00.173 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:00.170+0000 7f9122a62640 1 --2- 172.21.15.43:0/3162650847 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f911c06afe0 0x7f911c070600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T14:52:00.173 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:00.170+0000 7f9122261640 1 -- 172.21.15.43:0/3162650847 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f911c06afe0 msgr2=0x7f911c070600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-09T14:52:00.174 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:00.170+0000 7f9122261640 1 --2- 172.21.15.43:0/3162650847 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f911c06afe0 0x7f911c070600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T14:52:00.174 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:00.170+0000 7f9122261640 1 -- 172.21.15.43:0/3162650847 >> v1:172.21.15.203:6789/0 conn(0x7f911c0cb3d0 legacy=0x7f911c06a8d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-09T14:52:00.174 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:00.170+0000 7f9122261640 1 -- 172.21.15.43:0/3162650847 --> v2:172.21.15.43:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f911c0ca3b0 con 0x7f911c159390 2023-08-09T14:52:00.174 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:00.170+0000 7f9122a62640 1 --2- 172.21.15.43:0/3162650847 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f911c06afe0 0x7f911c070600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-08-09T14:52:00.174 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:00.171+0000 7f9122261640 1 --2- 172.21.15.43:0/3162650847 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f911c159390 0x7f911c0cae90 secure :-1 s=READY pgs=1910 cs=0 l=1 rev1=1 crypto rx=0x7f911800bb10 tx=0x7f911802e0c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1e563e3a66c64d2e server_cookie=0 in_seq=0 out_seq=0 2023-08-09T14:52:00.175 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:00.172+0000 7f912125f640 1 -- 172.21.15.43:0/3162650847 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f911802ee70 con 0x7f911c159390 2023-08-09T14:52:00.175 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:00.172+0000 7f912125f640 1 -- 172.21.15.43:0/3162650847 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9118036440 con 0x7f911c159390 2023-08-09T14:52:00.175 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:00.172+0000 7f912125f640 1 -- 172.21.15.43:0/3162650847 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f91180367d0 con 0x7f911c159390 2023-08-09T14:52:00.175 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:00.172+0000 7f91244c5640 1 -- 172.21.15.43:0/3162650847 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f911c159390 msgr2=0x7f911c0cae90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T14:52:00.176 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:00.172+0000 7f91244c5640 1 --2- 172.21.15.43:0/3162650847 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f911c159390 0x7f911c0cae90 secure :-1 s=READY pgs=1910 cs=0 l=1 rev1=1 crypto rx=0x7f911800bb10 tx=0x7f911802e0c0 comp rx=0 tx=0).stop 2023-08-09T14:52:00.176 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:00.172+0000 7f91244c5640 1 -- 172.21.15.43:0/3162650847 shutdown_connections 2023-08-09T14:52:00.176 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:00.173+0000 7f91244c5640 1 --2- 172.21.15.43:0/3162650847 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f911c06afe0 0x7f911c070600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T14:52:00.176 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:00.173+0000 7f91244c5640 1 --2- 172.21.15.43:0/3162650847 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f911c159390 0x7f911c0cae90 unknown :-1 s=CLOSED pgs=1910 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T14:52:00.176 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:00.173+0000 7f91244c5640 1 -- 172.21.15.43:0/3162650847 >> 172.21.15.43:0/3162650847 conn(0x7f911c0cf080 msgr2=0x7f911c0cf490 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-09T14:52:00.176 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:00.173+0000 7f91244c5640 1 -- 172.21.15.43:0/3162650847 shutdown_connections 2023-08-09T14:52:00.176 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:00.173+0000 7f91244c5640 1 -- 172.21.15.43:0/3162650847 wait complete. 2023-08-09T14:52:00.178 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:00.174+0000 7f91244c5640 1 Processor -- start 2023-08-09T14:52:00.178 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:00.174+0000 7f91244c5640 1 -- start start 2023-08-09T14:52:00.178 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:00.175+0000 7f91244c5640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f911c06afe0 0x7f911c0c83d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T14:52:00.178 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:00.175+0000 7f91244c5640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f911c0cb3d0 0x7f911c1f03f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T14:52:00.178 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:00.175+0000 7f91244c5640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f911c159390 0x7f911c1f0930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T14:52:00.179 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:00.175+0000 7f91244c5640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f911c15b500 con 0x7f911c06afe0 2023-08-09T14:52:00.179 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:00.175+0000 7f91244c5640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f911c15a8f0 con 0x7f911c0cb3d0 2023-08-09T14:52:00.179 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:00.175+0000 7f91244c5640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f911c15abd0 con 0x7f911c159390 2023-08-09T14:52:00.179 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:00.176+0000 7f9122261640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f911c06afe0 0x7f911c0c83d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T14:52:00.179 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:00.176+0000 7f9122261640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f911c06afe0 0x7f911c0c83d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:33938/0 (socket says 172.21.15.43:33938) 2023-08-09T14:52:00.179 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:00.176+0000 7f9122261640 1 -- 172.21.15.43:0/318138066 learned_addr learned my addr 172.21.15.43:0/318138066 (peer_addr_for_me v2:172.21.15.43:0/0) 2023-08-09T14:52:00.179 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:00.176+0000 7f9121a60640 1 --2- 172.21.15.43:0/318138066 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f911c0cb3d0 0x7f911c1f03f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T14:52:00.180 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:00.176+0000 7f9122a62640 1 --2- 172.21.15.43:0/318138066 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f911c159390 0x7f911c1f0930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T14:52:00.180 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:00.176+0000 7f9122261640 1 -- 172.21.15.43:0/318138066 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f911c159390 msgr2=0x7f911c1f0930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T14:52:00.180 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:00.176+0000 7f9122261640 1 --2- 172.21.15.43:0/318138066 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f911c159390 0x7f911c1f0930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T14:52:00.180 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:00.176+0000 7f9122261640 1 -- 172.21.15.43:0/318138066 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f911c0cb3d0 msgr2=0x7f911c1f03f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T14:52:00.180 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:00.176+0000 7f9122261640 1 --2- 172.21.15.43:0/318138066 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f911c0cb3d0 0x7f911c1f03f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T14:52:00.180 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:00.176+0000 7f9122261640 1 -- 172.21.15.43:0/318138066 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f911800b770 con 0x7f911c06afe0 2023-08-09T14:52:00.180 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:00.176+0000 7f9121a60640 1 --2- 172.21.15.43:0/318138066 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f911c0cb3d0 0x7f911c1f03f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-08-09T14:52:00.181 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:00.177+0000 7f9122261640 1 --2- 172.21.15.43:0/318138066 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f911c06afe0 0x7f911c0c83d0 secure :-1 s=READY pgs=1911 cs=0 l=1 rev1=1 crypto rx=0x7f911800b2c0 tx=0x7f91180046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-09T14:52:00.182 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:00.177+0000 7f9112ffd640 1 -- 172.21.15.43:0/318138066 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9118034b80 con 0x7f911c06afe0 2023-08-09T14:52:00.182 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:00.177+0000 7f9112ffd640 1 -- 172.21.15.43:0/318138066 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9118034d00 con 0x7f911c06afe0 2023-08-09T14:52:00.182 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:00.177+0000 7f9112ffd640 1 -- 172.21.15.43:0/318138066 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9118041680 con 0x7f911c06afe0 2023-08-09T14:52:00.182 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:00.177+0000 7f91244c5640 1 -- 172.21.15.43:0/318138066 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f911c15aeb0 con 0x7f911c06afe0 2023-08-09T14:52:00.182 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:00.177+0000 7f91244c5640 1 -- 172.21.15.43:0/318138066 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f911c0c8910 con 0x7f911c06afe0 2023-08-09T14:52:00.182 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:00.178+0000 7f91244c5640 1 -- 172.21.15.43:0/318138066 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f911c0c8aa0 con 0x7f911c06afe0 2023-08-09T14:52:00.182 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:00.179+0000 7f9112ffd640 1 -- 172.21.15.43:0/318138066 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 80) v1 ==== 88460+0+0 (secure 0 0 0) 0x7f91180403e0 con 0x7f911c06afe0 2023-08-09T14:52:00.183 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:00.181+0000 7f9112ffd640 1 --2- 172.21.15.43:0/318138066 >> [v2:172.21.15.203:6832/2566816332,v1:172.21.15.203:6833/2566816332] conn(0x7f910006a1c0 0x7f910006c680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T14:52:00.183 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:00.181+0000 7f9112ffd640 1 -- 172.21.15.43:0/318138066 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(172..172 src has 1..172) v4 ==== 6426+0+0 (secure 0 0 0) 0x7f91180aadb0 con 0x7f911c06afe0 2023-08-09T14:52:00.183 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:00.181+0000 7f9121a60640 1 -- 172.21.15.43:0/318138066 >> [v2:172.21.15.203:6832/2566816332,v1:172.21.15.203:6833/2566816332] conn(0x7f910006a1c0 msgr2=0x7f910006c680 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.203:6832/2566816332 2023-08-09T14:52:00.184 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:00.181+0000 7f9121a60640 1 --2- 172.21.15.43:0/318138066 >> [v2:172.21.15.203:6832/2566816332,v1:172.21.15.203:6833/2566816332] conn(0x7f910006a1c0 0x7f910006c680 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-08-09T14:52:00.186 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:00.184+0000 7f9112ffd640 1 -- 172.21.15.43:0/318138066 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178066 (secure 0 0 0) 0x7f91180af070 con 0x7f911c06afe0 2023-08-09T14:52:00.383 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:00.382+0000 7f9121a60640 1 -- 172.21.15.43:0/318138066 >> [v2:172.21.15.203:6832/2566816332,v1:172.21.15.203:6833/2566816332] conn(0x7f910006a1c0 msgr2=0x7f910006c680 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.203:6832/2566816332 2023-08-09T14:52:00.384 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:00.382+0000 7f9121a60640 1 --2- 172.21.15.43:0/318138066 >> [v2:172.21.15.203:6832/2566816332,v1:172.21.15.203:6833/2566816332] conn(0x7f910006a1c0 0x7f910006c680 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2023-08-09T14:52:00.399 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:00.397+0000 7f91244c5640 1 -- 172.21.15.43:0/318138066 --> [v2:172.21.15.203:6832/2566816332,v1:172.21.15.203:6833/2566816332] -- mgr_command(tid 0: {"prefix": "fs snap-schedule add", "fs": "cephfs", "path": "snap_test_dir1", "snap_schedule": "1h", "target": ["mon-mgr", ""]}) v1 -- 0x7f911c0ca970 con 0x7f910006a1c0 2023-08-09T14:52:00.731 INFO:tasks.ceph.mgr.x.smithi203.stderr:2023-08-09T14:52:00.730+0000 7fb8eb6d6040 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2023-08-09T14:52:00.785 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:00.783+0000 7f9121a60640 1 -- 172.21.15.43:0/318138066 >> [v2:172.21.15.203:6832/2566816332,v1:172.21.15.203:6833/2566816332] conn(0x7f910006a1c0 msgr2=0x7f910006c680 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.203:6832/2566816332 2023-08-09T14:52:00.785 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:00.783+0000 7f9121a60640 1 --2- 172.21.15.43:0/318138066 >> [v2:172.21.15.203:6832/2566816332,v1:172.21.15.203:6833/2566816332] conn(0x7f910006a1c0 0x7f910006c680 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2023-08-09T14:52:00.792 INFO:tasks.ceph.mgr.x.smithi203.stderr:2023-08-09T14:52:00.790+0000 7fb8eb6d6040 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2023-08-09T14:52:00.841 INFO:tasks.ceph.mgr.x.smithi203.stderr:2023-08-09T14:52:00.840+0000 7fb8eb6d6040 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2023-08-09T14:52:00.848 INFO:tasks.ceph.mgr.y.smithi043.stderr:2023-08-09T14:52:00.846+0000 7f1c74493040 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2023-08-09T14:52:00.911 INFO:tasks.ceph.mgr.y.smithi043.stderr:2023-08-09T14:52:00.910+0000 7f1c74493040 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2023-08-09T14:52:00.959 INFO:tasks.ceph.mgr.x.smithi203.stderr:2023-08-09T14:52:00.958+0000 7fb8eb6d6040 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2023-08-09T14:52:00.963 INFO:tasks.ceph.mgr.y.smithi043.stderr:2023-08-09T14:52:00.961+0000 7f1c74493040 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2023-08-09T14:52:01.085 INFO:tasks.ceph.mgr.y.smithi043.stderr:2023-08-09T14:52:01.083+0000 7f1c74493040 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2023-08-09T14:52:01.116 INFO:tasks.ceph.mgr.x.smithi203.stderr:2023-08-09T14:52:01.115+0000 7fb8eb6d6040 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2023-08-09T14:52:01.222 INFO:tasks.ceph.mgr.x.smithi203.stderr:2023-08-09T14:52:01.221+0000 7fb8eb6d6040 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2023-08-09T14:52:01.248 INFO:tasks.ceph.mgr.y.smithi043.stderr:2023-08-09T14:52:01.246+0000 7f1c74493040 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2023-08-09T14:52:01.271 INFO:tasks.ceph.mgr.x.smithi203.stderr:2023-08-09T14:52:01.270+0000 7fb8eb6d6040 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2023-08-09T14:52:01.360 INFO:tasks.ceph.mgr.y.smithi043.stderr:2023-08-09T14:52:01.359+0000 7f1c74493040 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2023-08-09T14:52:01.412 INFO:tasks.ceph.mgr.y.smithi043.stderr:2023-08-09T14:52:01.410+0000 7f1c74493040 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2023-08-09T14:52:01.431 INFO:tasks.ceph.mgr.x.smithi203.stderr:2023-08-09T14:52:01.429+0000 7fb8eb6d6040 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2023-08-09T14:52:01.579 INFO:tasks.ceph.mgr.y.smithi043.stderr:2023-08-09T14:52:01.577+0000 7f1c74493040 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2023-08-09T14:52:01.585 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:01.584+0000 7f9121a60640 1 -- 172.21.15.43:0/318138066 >> [v2:172.21.15.203:6832/2566816332,v1:172.21.15.203:6833/2566816332] conn(0x7f910006a1c0 msgr2=0x7f910006c680 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.203:6832/2566816332 2023-08-09T14:52:01.586 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:01.584+0000 7f9121a60640 1 --2- 172.21.15.43:0/318138066 >> [v2:172.21.15.203:6832/2566816332,v1:172.21.15.203:6833/2566816332] conn(0x7f910006a1c0 0x7f910006c680 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2023-08-09T14:52:01.812 INFO:tasks.ceph.mgr.x.smithi203.stderr:2023-08-09T14:52:01.810+0000 7fb8eb6d6040 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2023-08-09T14:52:01.981 INFO:tasks.ceph.mgr.y.smithi043.stderr:2023-08-09T14:52:01.980+0000 7f1c74493040 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2023-08-09T14:52:02.033 INFO:tasks.ceph.mgr.x.smithi203.stderr:2023-08-09T14:52:02.032+0000 7fb8eb6d6040 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2023-08-09T14:52:02.089 INFO:tasks.ceph.mgr.x.smithi203.stderr:2023-08-09T14:52:02.088+0000 7fb8eb6d6040 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2023-08-09T14:52:02.139 INFO:tasks.ceph.mgr.x.smithi203.stderr:2023-08-09T14:52:02.137+0000 7fb8eb6d6040 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2023-08-09T14:52:02.190 INFO:tasks.ceph.mgr.x.smithi203.stderr:2023-08-09T14:52:02.188+0000 7fb8eb6d6040 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2023-08-09T14:52:02.211 INFO:tasks.ceph.mgr.y.smithi043.stderr:2023-08-09T14:52:02.209+0000 7f1c74493040 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2023-08-09T14:52:02.271 INFO:tasks.ceph.mgr.y.smithi043.stderr:2023-08-09T14:52:02.269+0000 7f1c74493040 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2023-08-09T14:52:02.324 INFO:tasks.ceph.mgr.y.smithi043.stderr:2023-08-09T14:52:02.322+0000 7f1c74493040 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2023-08-09T14:52:02.378 INFO:tasks.ceph.mgr.y.smithi043.stderr:2023-08-09T14:52:02.377+0000 7f1c74493040 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2023-08-09T14:52:02.482 INFO:tasks.ceph.mgr.x.smithi203.stderr:2023-08-09T14:52:02.482+0000 7fb8eb6d6040 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2023-08-09T14:52:02.539 INFO:tasks.ceph.mgr.x.smithi203.stderr:2023-08-09T14:52:02.539+0000 7fb8eb6d6040 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2023-08-09T14:52:02.591 INFO:tasks.ceph.mgr.x.smithi203.stderr:2023-08-09T14:52:02.591+0000 7fb8eb6d6040 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2023-08-09T14:52:02.687 INFO:tasks.ceph.mgr.y.smithi043.stderr:2023-08-09T14:52:02.686+0000 7f1c74493040 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2023-08-09T14:52:02.747 INFO:tasks.ceph.mgr.y.smithi043.stderr:2023-08-09T14:52:02.746+0000 7f1c74493040 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2023-08-09T14:52:02.750 INFO:tasks.ceph.mgr.x.smithi203.stderr:2023-08-09T14:52:02.750+0000 7fb8eb6d6040 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2023-08-09T14:52:02.802 INFO:tasks.ceph.mgr.x.smithi203.stderr:2023-08-09T14:52:02.802+0000 7fb8eb6d6040 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2023-08-09T14:52:02.804 INFO:tasks.ceph.mgr.y.smithi043.stderr:2023-08-09T14:52:02.803+0000 7f1c74493040 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2023-08-09T14:52:02.855 INFO:tasks.ceph.mgr.x.smithi203.stderr:2023-08-09T14:52:02.855+0000 7fb8eb6d6040 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2023-08-09T14:52:02.905 INFO:tasks.ceph.mgr.x.smithi203.stderr:2023-08-09T14:52:02.905+0000 7fb8eb6d6040 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2023-08-09T14:52:02.930 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:02.927+0000 7f9112ffd640 1 -- 172.21.15.43:0/318138066 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mgrmap(e 81) v1 ==== 88069+0+0 (secure 0 0 0) 0x7f91180765e0 con 0x7f911c06afe0 2023-08-09T14:52:02.930 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:02.928+0000 7f9112ffd640 1 -- 172.21.15.43:0/318138066 >> [v2:172.21.15.203:6832/2566816332,v1:172.21.15.203:6833/2566816332] conn(0x7f910006a1c0 msgr2=0x7f910006c680 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-08-09T14:52:02.930 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:02.928+0000 7f9112ffd640 1 --2- 172.21.15.43:0/318138066 >> [v2:172.21.15.203:6832/2566816332,v1:172.21.15.203:6833/2566816332] conn(0x7f910006a1c0 0x7f910006c680 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T14:52:02.980 INFO:tasks.ceph.mgr.y.smithi043.stderr:2023-08-09T14:52:02.978+0000 7f1c74493040 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2023-08-09T14:52:03.039 INFO:tasks.ceph.mgr.y.smithi043.stderr:2023-08-09T14:52:03.037+0000 7f1c74493040 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2023-08-09T14:52:03.099 INFO:tasks.ceph.mgr.y.smithi043.stderr:2023-08-09T14:52:03.097+0000 7f1c74493040 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2023-08-09T14:52:03.110 INFO:tasks.ceph.mgr.x.smithi203.stderr:2023-08-09T14:52:03.110+0000 7fb86b1a6640 -1 client.0 error registering admin socket command: (17) File exists 2023-08-09T14:52:03.111 INFO:tasks.ceph.mgr.x.smithi203.stderr:2023-08-09T14:52:03.110+0000 7fb86b1a6640 -1 client.0 error registering admin socket command: (17) File exists 2023-08-09T14:52:03.111 INFO:tasks.ceph.mgr.x.smithi203.stderr:2023-08-09T14:52:03.110+0000 7fb86b1a6640 -1 client.0 error registering admin socket command: (17) File exists 2023-08-09T14:52:03.111 INFO:tasks.ceph.mgr.x.smithi203.stderr:2023-08-09T14:52:03.110+0000 7fb86b1a6640 -1 client.0 error registering admin socket command: (17) File exists 2023-08-09T14:52:03.111 INFO:tasks.ceph.mgr.x.smithi203.stderr:2023-08-09T14:52:03.110+0000 7fb86b1a6640 -1 client.0 error registering admin socket command: (17) File exists 2023-08-09T14:52:03.112 INFO:tasks.ceph.mgr.x.smithi203.stderr:2023-08-09T14:52:03.111+0000 7fb8689a1640 -1 client.0 error registering admin socket command: (17) File exists 2023-08-09T14:52:03.112 INFO:tasks.ceph.mgr.x.smithi203.stderr:2023-08-09T14:52:03.111+0000 7fb8689a1640 -1 client.0 error registering admin socket command: (17) File exists 2023-08-09T14:52:03.112 INFO:tasks.ceph.mgr.x.smithi203.stderr:2023-08-09T14:52:03.111+0000 7fb8689a1640 -1 client.0 error registering admin socket command: (17) File exists 2023-08-09T14:52:03.112 INFO:tasks.ceph.mgr.x.smithi203.stderr:2023-08-09T14:52:03.111+0000 7fb8689a1640 -1 client.0 error registering admin socket command: (17) File exists 2023-08-09T14:52:03.113 INFO:tasks.ceph.mgr.x.smithi203.stderr:2023-08-09T14:52:03.111+0000 7fb8689a1640 -1 client.0 error registering admin socket command: (17) File exists 2023-08-09T14:52:03.154 INFO:tasks.ceph.mgr.y.smithi043.stderr:2023-08-09T14:52:03.153+0000 7f1c74493040 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2023-08-09T14:52:03.939 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:03.937+0000 7f9112ffd640 1 -- 172.21.15.43:0/318138066 <== mon.0 v2:172.21.15.43:3300/0 8 ==== mgrmap(e 82) v1 ==== 88196+0+0 (secure 0 0 0) 0x7f91180a8f60 con 0x7f911c06afe0 2023-08-09T14:52:03.940 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:03.939+0000 7f9112ffd640 1 --2- 172.21.15.43:0/318138066 >> [v2:172.21.15.203:6832/843920641,v1:172.21.15.203:6833/843920641] conn(0x7f9100072cb0 0x7f91000750a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T14:52:03.940 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:03.939+0000 7f9112ffd640 1 -- 172.21.15.43:0/318138066 --> [v2:172.21.15.203:6832/843920641,v1:172.21.15.203:6833/843920641] -- mgr_command(tid 0: {"prefix": "fs snap-schedule add", "fs": "cephfs", "path": "snap_test_dir1", "snap_schedule": "1h", "target": ["mon-mgr", ""]}) v1 -- 0x7f9100036270 con 0x7f9100072cb0 2023-08-09T14:52:03.943 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:03.941+0000 7f9121a60640 1 --2- 172.21.15.43:0/318138066 >> [v2:172.21.15.203:6832/843920641,v1:172.21.15.203:6833/843920641] conn(0x7f9100072cb0 0x7f91000750a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T14:52:03.950 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:03.949+0000 7f9121a60640 1 --2- 172.21.15.43:0/318138066 >> [v2:172.21.15.203:6832/843920641,v1:172.21.15.203:6833/843920641] conn(0x7f9100072cb0 0x7f91000750a0 secure :-1 s=READY pgs=2 cs=0 l=1 rev1=1 crypto rx=0x7f911c0b4770 tx=0x7f910c009040 comp rx=0 tx=0).ready entity=mgr.8020 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-09T14:52:04.111 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:04.110+0000 7f9112ffd640 1 -- 172.21.15.43:0/318138066 <== mgr.8020 v2:172.21.15.203:6832/843920641 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+37 (secure 0 0 0) 0x7f910c003880 con 0x7f9100072cb0 2023-08-09T14:52:04.112 INFO:teuthology.orchestra.run.smithi043.stdout:Schedule set for path snap_test_dir1 2023-08-09T14:52:04.115 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:04.113+0000 7f91244c5640 1 -- 172.21.15.43:0/318138066 >> [v2:172.21.15.203:6832/843920641,v1:172.21.15.203:6833/843920641] conn(0x7f9100072cb0 msgr2=0x7f91000750a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T14:52:04.115 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:04.113+0000 7f91244c5640 1 --2- 172.21.15.43:0/318138066 >> [v2:172.21.15.203:6832/843920641,v1:172.21.15.203:6833/843920641] conn(0x7f9100072cb0 0x7f91000750a0 secure :-1 s=READY pgs=2 cs=0 l=1 rev1=1 crypto rx=0x7f911c0b4770 tx=0x7f910c009040 comp rx=0 tx=0).stop 2023-08-09T14:52:04.115 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:04.113+0000 7f91244c5640 1 -- 172.21.15.43:0/318138066 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f911c06afe0 msgr2=0x7f911c0c83d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T14:52:04.115 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:04.113+0000 7f91244c5640 1 --2- 172.21.15.43:0/318138066 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f911c06afe0 0x7f911c0c83d0 secure :-1 s=READY pgs=1911 cs=0 l=1 rev1=1 crypto rx=0x7f911800b2c0 tx=0x7f91180046c0 comp rx=0 tx=0).stop 2023-08-09T14:52:04.115 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:04.113+0000 7f9122a62640 1 -- 172.21.15.43:0/318138066 reap_dead start 2023-08-09T14:52:04.116 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:04.113+0000 7f91244c5640 1 -- 172.21.15.43:0/318138066 shutdown_connections 2023-08-09T14:52:04.116 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:04.113+0000 7f91244c5640 1 -- 172.21.15.43:0/318138066 >> 172.21.15.43:0/318138066 conn(0x7f911c0cf080 msgr2=0x7f911c157d40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-09T14:52:04.116 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:04.114+0000 7f91244c5640 1 -- 172.21.15.43:0/318138066 shutdown_connections 2023-08-09T14:52:04.116 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:04.114+0000 7f91244c5640 1 -- 172.21.15.43:0/318138066 wait complete. 2023-08-09T14:52:04.138 DEBUG:tasks.cephfs.test_snap_schedules:res=Schedule set for path snap_test_dir1 2023-08-09T14:52:04.138 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs snap-schedule remove --fs cephfs snap_test_dir1 2023-08-09T14:52:04.376 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:04.374+0000 7f9c20e25640 1 Processor -- start 2023-08-09T14:52:04.376 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:04.375+0000 7f9c20e25640 1 -- start start 2023-08-09T14:52:04.379 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:04.376+0000 7f9c20e25640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f9c1c15a330 0x7f9c1c0c9780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T14:52:04.379 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:04.376+0000 7f9c20e25640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f9c1c15ea60 0x7f9c1c070d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T14:52:04.379 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:04.376+0000 7f9c20e25640 1 -- --> v1:172.21.15.203:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9c1c15dd30 con 0x7f9c1c0af870 2023-08-09T14:52:04.379 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:04.376+0000 7f9c20e25640 1 -- --> v2:172.21.15.43:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9c1c0c9cc0 con 0x7f9c1c15a330 2023-08-09T14:52:04.380 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:04.376+0000 7f9c20e25640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9c1c071240 con 0x7f9c1c15ea60 2023-08-09T14:52:04.380 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:04.377+0000 7f9c1b7fe640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f9c1c15a330 0x7f9c1c0c9780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T14:52:04.380 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:04.377+0000 7f9c1b7fe640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f9c1c15a330 0x7f9c1c0c9780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:33952/0 (socket says 172.21.15.43:33952) 2023-08-09T14:52:04.380 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:04.377+0000 7f9c1b7fe640 1 -- 172.21.15.43:0/4128595165 learned_addr learned my addr 172.21.15.43:0/4128595165 (peer_addr_for_me v2:172.21.15.43:0/0) 2023-08-09T14:52:04.381 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:04.377+0000 7f9c1affd640 1 --2- 172.21.15.43:0/4128595165 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f9c1c15ea60 0x7f9c1c070d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T14:52:04.381 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:04.377+0000 7f9c1b7fe640 1 -- 172.21.15.43:0/4128595165 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f9c1c15ea60 msgr2=0x7f9c1c070d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-09T14:52:04.381 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:04.377+0000 7f9c1b7fe640 1 --2- 172.21.15.43:0/4128595165 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f9c1c15ea60 0x7f9c1c070d00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T14:52:04.381 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:04.377+0000 7f9c1b7fe640 1 -- 172.21.15.43:0/4128595165 >> v1:172.21.15.203:6789/0 conn(0x7f9c1c0af870 legacy=0x7f9c1c0c8440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-09T14:52:04.382 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:04.377+0000 7f9c1b7fe640 1 -- 172.21.15.43:0/4128595165 --> v2:172.21.15.43:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9c1c071490 con 0x7f9c1c15a330 2023-08-09T14:52:04.384 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:04.378+0000 7f9c1b7fe640 1 --2- 172.21.15.43:0/4128595165 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f9c1c15a330 0x7f9c1c0c9780 secure :-1 s=READY pgs=1912 cs=0 l=1 rev1=1 crypto rx=0x7f9c0c009870 tx=0x7f9c0c02b8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b4c62cd22a3b3152 server_cookie=0 in_seq=0 out_seq=0 2023-08-09T14:52:04.385 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:04.379+0000 7f9c1a7fc640 1 -- 172.21.15.43:0/4128595165 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9c0c031640 con 0x7f9c1c15a330 2023-08-09T14:52:04.385 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:04.379+0000 7f9c1a7fc640 1 -- 172.21.15.43:0/4128595165 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9c0c0317a0 con 0x7f9c1c15a330 2023-08-09T14:52:04.385 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:04.379+0000 7f9c1a7fc640 1 -- 172.21.15.43:0/4128595165 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9c0c031b30 con 0x7f9c1c15a330 2023-08-09T14:52:04.386 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:04.379+0000 7f9c20e25640 1 -- 172.21.15.43:0/4128595165 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f9c1c15a330 msgr2=0x7f9c1c0c9780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T14:52:04.386 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:04.379+0000 7f9c20e25640 1 --2- 172.21.15.43:0/4128595165 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f9c1c15a330 0x7f9c1c0c9780 secure :-1 s=READY pgs=1912 cs=0 l=1 rev1=1 crypto rx=0x7f9c0c009870 tx=0x7f9c0c02b8f0 comp rx=0 tx=0).stop 2023-08-09T14:52:04.386 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:04.379+0000 7f9c20e25640 1 -- 172.21.15.43:0/4128595165 shutdown_connections 2023-08-09T14:52:04.386 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:04.379+0000 7f9c20e25640 1 --2- 172.21.15.43:0/4128595165 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f9c1c15ea60 0x7f9c1c070d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T14:52:04.387 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:04.379+0000 7f9c20e25640 1 --2- 172.21.15.43:0/4128595165 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f9c1c15a330 0x7f9c1c0c9780 unknown :-1 s=CLOSED pgs=1912 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T14:52:04.387 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:04.379+0000 7f9c20e25640 1 -- 172.21.15.43:0/4128595165 >> 172.21.15.43:0/4128595165 conn(0x7f9c1c0ca870 msgr2=0x7f9c1c0cac50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-09T14:52:04.387 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:04.380+0000 7f9c20e25640 1 -- 172.21.15.43:0/4128595165 shutdown_connections 2023-08-09T14:52:04.387 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:04.380+0000 7f9c20e25640 1 -- 172.21.15.43:0/4128595165 wait complete. 2023-08-09T14:52:04.388 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:04.381+0000 7f9c20e25640 1 Processor -- start 2023-08-09T14:52:04.388 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:04.382+0000 7f9c20e25640 1 -- start start 2023-08-09T14:52:04.388 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:04.383+0000 7f9c20e25640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f9c1c0af870 0x7f9c1c1ef910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T14:52:04.388 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:04.383+0000 7f9c20e25640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f9c1c15a330 0x7f9c1c156f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T14:52:04.389 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:04.383+0000 7f9c20e25640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f9c1c15ea60 0x7f9c1c0a7d20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T14:52:04.389 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:04.383+0000 7f9c20e25640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9c1c1f0bb0 con 0x7f9c1c15a330 2023-08-09T14:52:04.389 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:04.383+0000 7f9c20e25640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9c1c1eff20 con 0x7f9c1c0af870 2023-08-09T14:52:04.390 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:04.383+0000 7f9c20e25640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9c1c1f0200 con 0x7f9c1c15ea60 2023-08-09T14:52:04.390 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:04.383+0000 7f9c1affd640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f9c1c15a330 0x7f9c1c156f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T14:52:04.390 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:04.383+0000 7f9c1affd640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f9c1c15a330 0x7f9c1c156f50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:33956/0 (socket says 172.21.15.43:33956) 2023-08-09T14:52:04.390 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:04.383+0000 7f9c1affd640 1 -- 172.21.15.43:0/1804444796 learned_addr learned my addr 172.21.15.43:0/1804444796 (peer_addr_for_me v2:172.21.15.43:0/0) 2023-08-09T14:52:04.393 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:04.383+0000 7f9c1bfff640 1 --2- 172.21.15.43:0/1804444796 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f9c1c15ea60 0x7f9c1c0a7d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T14:52:04.393 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:04.383+0000 7f9c1b7fe640 1 --2- 172.21.15.43:0/1804444796 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f9c1c0af870 0x7f9c1c1ef910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T14:52:04.394 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:04.383+0000 7f9c1affd640 1 -- 172.21.15.43:0/1804444796 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f9c1c15ea60 msgr2=0x7f9c1c0a7d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T14:52:04.394 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:04.383+0000 7f9c1affd640 1 --2- 172.21.15.43:0/1804444796 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f9c1c15ea60 0x7f9c1c0a7d20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T14:52:04.394 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:04.384+0000 7f9c1affd640 1 -- 172.21.15.43:0/1804444796 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f9c1c0af870 msgr2=0x7f9c1c1ef910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T14:52:04.394 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:04.384+0000 7f9c1affd640 1 --2- 172.21.15.43:0/1804444796 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f9c1c0af870 0x7f9c1c1ef910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T14:52:04.395 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:04.384+0000 7f9c1affd640 1 -- 172.21.15.43:0/1804444796 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9c0c0094d0 con 0x7f9c1c15a330 2023-08-09T14:52:04.395 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:04.384+0000 7f9c1b7fe640 1 --2- 172.21.15.43:0/1804444796 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f9c1c0af870 0x7f9c1c1ef910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-08-09T14:52:04.395 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:04.384+0000 7f9c1affd640 1 --2- 172.21.15.43:0/1804444796 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f9c1c15a330 0x7f9c1c156f50 secure :-1 s=READY pgs=1913 cs=0 l=1 rev1=1 crypto rx=0x7f9c1000c7f0 tx=0x7f9c1000ccb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-09T14:52:04.395 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:04.384+0000 7f9c03fff640 1 -- 172.21.15.43:0/1804444796 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9c10004220 con 0x7f9c1c15a330 2023-08-09T14:52:04.396 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:04.384+0000 7f9c03fff640 1 -- 172.21.15.43:0/1804444796 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9c100043a0 con 0x7f9c1c15a330 2023-08-09T14:52:04.396 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:04.384+0000 7f9c03fff640 1 -- 172.21.15.43:0/1804444796 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9c10007860 con 0x7f9c1c15a330 2023-08-09T14:52:04.396 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:04.384+0000 7f9c20e25640 1 -- 172.21.15.43:0/1804444796 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9c1c1f04e0 con 0x7f9c1c15a330 2023-08-09T14:52:04.396 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:04.384+0000 7f9c20e25640 1 -- 172.21.15.43:0/1804444796 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9c1c157490 con 0x7f9c1c15a330 2023-08-09T14:52:04.397 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:04.386+0000 7f9c20e25640 1 -- 172.21.15.43:0/1804444796 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9c1c0739f0 con 0x7f9c1c15a330 2023-08-09T14:52:04.397 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:04.387+0000 7f9c03fff640 1 -- 172.21.15.43:0/1804444796 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 82) v1 ==== 88196+0+0 (secure 0 0 0) 0x7f9c10007bd0 con 0x7f9c1c15a330 2023-08-09T14:52:04.397 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:04.388+0000 7f9c03fff640 1 --2- 172.21.15.43:0/1804444796 >> [v2:172.21.15.203:6832/843920641,v1:172.21.15.203:6833/843920641] conn(0x7f9c08069ef0 0x7f9c0806c3b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T14:52:04.398 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:04.388+0000 7f9c03fff640 1 -- 172.21.15.43:0/1804444796 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(173..173 src has 1..173) v4 ==== 6770+0+0 (secure 0 0 0) 0x7f9c10088590 con 0x7f9c1c15a330 2023-08-09T14:52:04.398 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:04.388+0000 7f9c1b7fe640 1 --2- 172.21.15.43:0/1804444796 >> [v2:172.21.15.203:6832/843920641,v1:172.21.15.203:6833/843920641] conn(0x7f9c08069ef0 0x7f9c0806c3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T14:52:04.398 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:04.390+0000 7f9c1b7fe640 1 --2- 172.21.15.43:0/1804444796 >> [v2:172.21.15.203:6832/843920641,v1:172.21.15.203:6833/843920641] conn(0x7f9c08069ef0 0x7f9c0806c3b0 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7f9c0c036000 tx=0x7f9c0c03c040 comp rx=0 tx=0).ready entity=mgr.8020 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-09T14:52:04.399 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:04.392+0000 7f9c03fff640 1 -- 172.21.15.43:0/1804444796 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178066 (secure 0 0 0) 0x7f9c10053c20 con 0x7f9c1c15a330 2023-08-09T14:52:04.621 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:04.620+0000 7f9c20e25640 1 -- 172.21.15.43:0/1804444796 --> [v2:172.21.15.203:6832/843920641,v1:172.21.15.203:6833/843920641] -- mgr_command(tid 0: {"prefix": "fs snap-schedule remove", "fs": "cephfs", "path": "snap_test_dir1", "target": ["mon-mgr", ""]}) v1 -- 0x7f9c1c15e260 con 0x7f9c08069ef0 2023-08-09T14:52:04.791 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:04.790+0000 7f9c03fff640 1 -- 172.21.15.43:0/1804444796 <== mgr.8020 v2:172.21.15.203:6832/843920641 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+41 (secure 0 0 0) 0x7f9c0c031a90 con 0x7f9c08069ef0 2023-08-09T14:52:04.792 INFO:teuthology.orchestra.run.smithi043.stdout:Schedule removed for path snap_test_dir1 2023-08-09T14:52:04.795 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:04.793+0000 7f9c20e25640 1 -- 172.21.15.43:0/1804444796 >> [v2:172.21.15.203:6832/843920641,v1:172.21.15.203:6833/843920641] conn(0x7f9c08069ef0 msgr2=0x7f9c0806c3b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T14:52:04.795 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:04.793+0000 7f9c20e25640 1 --2- 172.21.15.43:0/1804444796 >> [v2:172.21.15.203:6832/843920641,v1:172.21.15.203:6833/843920641] conn(0x7f9c08069ef0 0x7f9c0806c3b0 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7f9c0c036000 tx=0x7f9c0c03c040 comp rx=0 tx=0).stop 2023-08-09T14:52:04.795 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:04.793+0000 7f9c20e25640 1 -- 172.21.15.43:0/1804444796 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f9c1c15a330 msgr2=0x7f9c1c156f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T14:52:04.796 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:04.793+0000 7f9c20e25640 1 --2- 172.21.15.43:0/1804444796 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f9c1c15a330 0x7f9c1c156f50 secure :-1 s=READY pgs=1913 cs=0 l=1 rev1=1 crypto rx=0x7f9c1000c7f0 tx=0x7f9c1000ccb0 comp rx=0 tx=0).stop 2023-08-09T14:52:04.796 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:04.794+0000 7f9c20e25640 1 -- 172.21.15.43:0/1804444796 shutdown_connections 2023-08-09T14:52:04.796 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:04.794+0000 7f9c20e25640 1 --2- 172.21.15.43:0/1804444796 >> [v2:172.21.15.203:6832/843920641,v1:172.21.15.203:6833/843920641] conn(0x7f9c08069ef0 0x7f9c0806c3b0 unknown :-1 s=CLOSED pgs=23 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T14:52:04.796 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:04.794+0000 7f9c20e25640 1 --2- 172.21.15.43:0/1804444796 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f9c1c15ea60 0x7f9c1c0a7d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T14:52:04.796 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:04.794+0000 7f9c20e25640 1 --2- 172.21.15.43:0/1804444796 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f9c1c15a330 0x7f9c1c156f50 unknown :-1 s=CLOSED pgs=1913 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T14:52:04.796 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:04.794+0000 7f9c20e25640 1 --2- 172.21.15.43:0/1804444796 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f9c1c0af870 0x7f9c1c1ef910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T14:52:04.796 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:04.794+0000 7f9c20e25640 1 -- 172.21.15.43:0/1804444796 >> 172.21.15.43:0/1804444796 conn(0x7f9c1c0ca870 msgr2=0x7f9c1c0ae620 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-09T14:52:04.797 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:04.794+0000 7f9c20e25640 1 -- 172.21.15.43:0/1804444796 shutdown_connections 2023-08-09T14:52:04.797 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:04.794+0000 7f9c20e25640 1 -- 172.21.15.43:0/1804444796 wait complete. 2023-08-09T14:52:04.818 DEBUG:tasks.cephfs.test_snap_schedules:res=Schedule removed for path snap_test_dir1 2023-08-09T14:52:04.819 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs snap-schedule list --fs cephfs snap_test_dir1 2023-08-09T14:52:05.062 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:05.060+0000 7f9988376640 1 Processor -- start 2023-08-09T14:52:05.062 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:05.060+0000 7f9988376640 1 -- start start 2023-08-09T14:52:05.063 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:05.062+0000 7f9988376640 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7f9980069a30 0x7f9980159760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T14:52:05.064 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:05.062+0000 7f9988376640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f998015d5b0 0x7f9980069e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T14:52:05.064 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:05.062+0000 7f9988376640 1 -- --> v1:172.21.15.43:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f99800ca410 con 0x7f998006a350 2023-08-09T14:52:05.064 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:05.062+0000 7f9988376640 1 -- --> v2:172.21.15.203:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f998015ce80 con 0x7f9980069a30 2023-08-09T14:52:05.064 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:05.062+0000 7f9988376640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f998015a050 con 0x7f998015d5b0 2023-08-09T14:52:05.065 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:05.062+0000 7f9986913640 1 --1- >> v1:172.21.15.43:6789/0 conn(0x7f998006a350 0x7f99800ce4c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.43:6789/0 says I am v1:172.21.15.43:41486/0 (socket says 172.21.15.43:41486) 2023-08-09T14:52:05.065 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:05.062+0000 7f9986913640 1 -- 172.21.15.43:0/1381361102 learned_addr learned my addr 172.21.15.43:0/1381361102 (peer_addr_for_me v1:172.21.15.43:0/0) 2023-08-09T14:52:05.065 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:05.063+0000 7f9986112640 1 --2- 172.21.15.43:0/1381361102 >> v2:172.21.15.203:3300/0 conn(0x7f9980069a30 0x7f9980159760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T14:52:05.065 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:05.063+0000 7f9985911640 1 --2- 172.21.15.43:0/1381361102 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f998015d5b0 0x7f9980069e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T14:52:05.066 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:05.063+0000 7f9985110640 1 -- 172.21.15.43:0/1381361102 <== mon.0 v1:172.21.15.43:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1803433846 0 0) 0x7f99800ca410 con 0x7f998006a350 2023-08-09T14:52:05.066 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:05.063+0000 7f9985110640 1 -- 172.21.15.43:0/1381361102 --> v1:172.21.15.43:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f996c003520 con 0x7f998006a350 2023-08-09T14:52:05.066 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:05.063+0000 7f9986112640 1 -- 172.21.15.43:0/1381361102 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f998015d5b0 msgr2=0x7f9980069e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-09T14:52:05.066 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:05.063+0000 7f9986112640 1 --2- 172.21.15.43:0/1381361102 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f998015d5b0 0x7f9980069e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T14:52:05.066 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:05.063+0000 7f9986112640 1 -- 172.21.15.43:0/1381361102 >> v1:172.21.15.43:6789/0 conn(0x7f998006a350 legacy=0x7f99800ce4c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T14:52:05.066 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:05.063+0000 7f9986112640 1 -- 172.21.15.43:0/1381361102 --> v2:172.21.15.203:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f998015a330 con 0x7f9980069a30 2023-08-09T14:52:05.067 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:05.065+0000 7f9986112640 1 --2- 172.21.15.43:0/1381361102 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f9980069a30 0x7f9980159760 secure :-1 s=READY pgs=704 cs=0 l=1 rev1=1 crypto rx=0x7f997c0097a0 tx=0x7f997c02b820 comp rx=0 tx=0).ready entity=mon.1 client_cookie=506eb4136627613d server_cookie=0 in_seq=0 out_seq=0 2023-08-09T14:52:05.067 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:05.065+0000 7f9985110640 1 -- 172.21.15.43:0/1381361102 <== mon.1 v2:172.21.15.203:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f997c031780 con 0x7f9980069a30 2023-08-09T14:52:05.068 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:05.065+0000 7f9985110640 1 -- 172.21.15.43:0/1381361102 <== mon.1 v2:172.21.15.203:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f997c0318e0 con 0x7f9980069a30 2023-08-09T14:52:05.068 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:05.065+0000 7f9985110640 1 -- 172.21.15.43:0/1381361102 <== mon.1 v2:172.21.15.203:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f997c031c70 con 0x7f9980069a30 2023-08-09T14:52:05.068 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:05.065+0000 7f9988376640 1 -- 172.21.15.43:0/1381361102 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f9980069a30 msgr2=0x7f9980159760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T14:52:05.068 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:05.065+0000 7f9988376640 1 --2- 172.21.15.43:0/1381361102 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f9980069a30 0x7f9980159760 secure :-1 s=READY pgs=704 cs=0 l=1 rev1=1 crypto rx=0x7f997c0097a0 tx=0x7f997c02b820 comp rx=0 tx=0).stop 2023-08-09T14:52:05.068 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:05.066+0000 7f9988376640 1 -- 172.21.15.43:0/1381361102 shutdown_connections 2023-08-09T14:52:05.069 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:05.066+0000 7f9988376640 1 --2- 172.21.15.43:0/1381361102 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f998015d5b0 0x7f9980069e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T14:52:05.069 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:05.066+0000 7f9988376640 1 --2- 172.21.15.43:0/1381361102 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f9980069a30 0x7f9980159760 unknown :-1 s=CLOSED pgs=704 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T14:52:05.069 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:05.066+0000 7f9988376640 1 -- 172.21.15.43:0/1381361102 >> 172.21.15.43:0/1381361102 conn(0x7f998006b990 msgr2=0x7f998006bda0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-09T14:52:05.069 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:05.066+0000 7f9988376640 1 -- 172.21.15.43:0/1381361102 shutdown_connections 2023-08-09T14:52:05.069 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:05.066+0000 7f9988376640 1 -- 172.21.15.43:0/1381361102 wait complete. 2023-08-09T14:52:05.070 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:05.068+0000 7f9988376640 1 Processor -- start 2023-08-09T14:52:05.070 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:05.068+0000 7f9988376640 1 -- start start 2023-08-09T14:52:05.071 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:05.069+0000 7f9988376640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f9980069a30 0x7f99800abf00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T14:52:05.071 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:05.069+0000 7f9988376640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f998006a350 0x7f99800afe20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T14:52:05.071 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:05.069+0000 7f9988376640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f998015d5b0 0x7f99800b0360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T14:52:05.071 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:05.069+0000 7f9988376640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f99801f1d20 con 0x7f998006a350 2023-08-09T14:52:05.072 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:05.069+0000 7f9988376640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f99801f6000 con 0x7f9980069a30 2023-08-09T14:52:05.072 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:05.069+0000 7f9988376640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f99801f1f30 con 0x7f998015d5b0 2023-08-09T14:52:05.072 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:05.069+0000 7f9985911640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f998006a350 0x7f99800afe20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T14:52:05.072 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:05.069+0000 7f9985911640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f998006a350 0x7f99800afe20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:34766/0 (socket says 172.21.15.43:34766) 2023-08-09T14:52:05.072 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:05.069+0000 7f9985911640 1 -- 172.21.15.43:0/2170599646 learned_addr learned my addr 172.21.15.43:0/2170599646 (peer_addr_for_me v2:172.21.15.43:0/0) 2023-08-09T14:52:05.072 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:05.069+0000 7f9986112640 1 --2- 172.21.15.43:0/2170599646 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f9980069a30 0x7f99800abf00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T14:52:05.072 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:05.069+0000 7f9986913640 1 --2- 172.21.15.43:0/2170599646 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f998015d5b0 0x7f99800b0360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T14:52:05.073 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:05.070+0000 7f9985911640 1 -- 172.21.15.43:0/2170599646 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f998015d5b0 msgr2=0x7f99800b0360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T14:52:05.073 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:05.070+0000 7f9985911640 1 --2- 172.21.15.43:0/2170599646 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f998015d5b0 0x7f99800b0360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T14:52:05.073 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:05.070+0000 7f9985911640 1 -- 172.21.15.43:0/2170599646 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f9980069a30 msgr2=0x7f99800abf00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T14:52:05.073 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:05.070+0000 7f9985911640 1 --2- 172.21.15.43:0/2170599646 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f9980069a30 0x7f99800abf00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T14:52:05.073 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:05.070+0000 7f9985911640 1 -- 172.21.15.43:0/2170599646 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f997c009400 con 0x7f998006a350 2023-08-09T14:52:05.073 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:05.070+0000 7f9986913640 1 --2- 172.21.15.43:0/2170599646 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f998015d5b0 0x7f99800b0360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-08-09T14:52:05.073 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:05.070+0000 7f9985911640 1 --2- 172.21.15.43:0/2170599646 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f998006a350 0x7f99800afe20 secure :-1 s=READY pgs=1915 cs=0 l=1 rev1=1 crypto rx=0x7f997000e870 tx=0x7f997000ed30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-09T14:52:05.074 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:05.070+0000 7f9976ffd640 1 -- 172.21.15.43:0/2170599646 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f99700094f0 con 0x7f998006a350 2023-08-09T14:52:05.074 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:05.070+0000 7f9976ffd640 1 -- 172.21.15.43:0/2170599646 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f997000c940 con 0x7f998006a350 2023-08-09T14:52:05.074 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:05.070+0000 7f9976ffd640 1 -- 172.21.15.43:0/2170599646 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f997000ccf0 con 0x7f998006a350 2023-08-09T14:52:05.075 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:05.070+0000 7f9988376640 1 -- 172.21.15.43:0/2170599646 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f99801f2210 con 0x7f998006a350 2023-08-09T14:52:05.075 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:05.070+0000 7f9988376640 1 -- 172.21.15.43:0/2170599646 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f99801f2a10 con 0x7f998006a350 2023-08-09T14:52:05.075 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:05.071+0000 7f9988376640 1 -- 172.21.15.43:0/2170599646 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f998013fb30 con 0x7f998006a350 2023-08-09T14:52:05.075 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:05.072+0000 7f9976ffd640 1 -- 172.21.15.43:0/2170599646 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 83) v1 ==== 88461+0+0 (secure 0 0 0) 0x7f9970013470 con 0x7f998006a350 2023-08-09T14:52:05.076 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:05.073+0000 7f9976ffd640 1 --2- 172.21.15.43:0/2170599646 >> [v2:172.21.15.203:6832/843920641,v1:172.21.15.203:6833/843920641] conn(0x7f996406a1c0 0x7f996406c680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T14:52:05.076 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:05.074+0000 7f9976ffd640 1 -- 172.21.15.43:0/2170599646 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(173..173 src has 1..173) v4 ==== 6770+0+0 (secure 0 0 0) 0x7f9970088b80 con 0x7f998006a350 2023-08-09T14:52:05.076 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:05.074+0000 7f9986112640 1 --2- 172.21.15.43:0/2170599646 >> [v2:172.21.15.203:6832/843920641,v1:172.21.15.203:6833/843920641] conn(0x7f996406a1c0 0x7f996406c680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T14:52:05.076 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:05.075+0000 7f9986112640 1 --2- 172.21.15.43:0/2170599646 >> [v2:172.21.15.203:6832/843920641,v1:172.21.15.203:6833/843920641] conn(0x7f996406a1c0 0x7f996406c680 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7f997c036000 tx=0x7f997c03d040 comp rx=0 tx=0).ready entity=mgr.8020 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-09T14:52:05.080 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:05.079+0000 7f9976ffd640 1 -- 172.21.15.43:0/2170599646 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178066 (secure 0 0 0) 0x7f9970053df0 con 0x7f998006a350 2023-08-09T14:52:05.301 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:05.300+0000 7f9988376640 1 -- 172.21.15.43:0/2170599646 --> [v2:172.21.15.203:6832/843920641,v1:172.21.15.203:6833/843920641] -- mgr_command(tid 0: {"prefix": "fs snap-schedule list", "fs": "cephfs", "path": "snap_test_dir1", "target": ["mon-mgr", ""]}) v1 -- 0x7f998015a610 con 0x7f996406a1c0 2023-08-09T14:52:05.316 INFO:tasks.ceph.mgr.x.smithi203.stderr:2023-08-09T14:52:05.315+0000 7fb8729b5640 -1 mgr.server reply reply (2) No such file or directory SnapSchedule for snap_test_dir1 not found 2023-08-09T14:52:05.316 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:05.314+0000 7f9976ffd640 1 -- 172.21.15.43:0/2170599646 <== mgr.8020 v2:172.21.15.203:6832/843920641 1 ==== mgr_command_reply(tid 0: -2 SnapSchedule for snap_test_dir1 not found) v1 ==== 49+0+0 (secure 0 0 0) 0x7f997c031a50 con 0x7f996406a1c0 2023-08-09T14:52:05.317 INFO:teuthology.orchestra.run.smithi043.stderr:Error ENOENT: SnapSchedule for snap_test_dir1 not found 2023-08-09T14:52:05.329 DEBUG:teuthology.orchestra.run:got remote process result: 2 2023-08-09T14:52:05.330 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-09T14:52:05.331 DEBUG:teuthology.orchestra.run.smithi043:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1) 2023-08-09T14:52:05.389 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph mgr module disable snap_schedule 2023-08-09T14:52:05.611 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:05.609+0000 7f7eaa99b640 1 Processor -- start 2023-08-09T14:52:05.611 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:05.609+0000 7f7eaa99b640 1 -- start start 2023-08-09T14:52:05.612 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:05.611+0000 7f7eaa99b640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f7ea4157ce0 0x7f7ea415b0f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T14:52:05.612 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:05.611+0000 7f7eaa99b640 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7f7ea40c8470 0x7f7ea40c88c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T14:52:05.612 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:05.611+0000 7f7eaa99b640 1 -- --> v1:172.21.15.203:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7ea40c8e00 con 0x7f7ea406f2c0 2023-08-09T14:52:05.613 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:05.611+0000 7f7eaa99b640 1 -- --> v2:172.21.15.203:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7ea41580c0 con 0x7f7ea40c8470 2023-08-09T14:52:05.613 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:05.611+0000 7f7eaa99b640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7ea40c92c0 con 0x7f7ea4157ce0 2023-08-09T14:52:05.613 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:05.612+0000 7f7eaa19a640 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7f7ea40c8470 0x7f7ea40c88c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T14:52:05.613 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:05.612+0000 7f7ea9999640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f7ea4157ce0 0x7f7ea415b0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T14:52:05.614 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:05.612+0000 7f7eaa19a640 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7f7ea40c8470 0x7f7ea40c88c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.203:3300/0 says I am v2:172.21.15.43:50410/0 (socket says 172.21.15.43:50410) 2023-08-09T14:52:05.614 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:05.612+0000 7f7eaa19a640 1 -- 172.21.15.43:0/2723277031 learned_addr learned my addr 172.21.15.43:0/2723277031 (peer_addr_for_me v2:172.21.15.43:0/0) 2023-08-09T14:52:05.614 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:05.612+0000 7f7eaa19a640 1 -- 172.21.15.43:0/2723277031 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f7ea4157ce0 msgr2=0x7f7ea415b0f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-09T14:52:05.614 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:05.612+0000 7f7eaa19a640 1 --2- 172.21.15.43:0/2723277031 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f7ea4157ce0 0x7f7ea415b0f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T14:52:05.615 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:05.612+0000 7f7eaa19a640 1 -- 172.21.15.43:0/2723277031 >> v1:172.21.15.203:6789/0 conn(0x7f7ea406f2c0 legacy=0x7f7ea406bc40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T14:52:05.615 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:05.612+0000 7f7eaa19a640 1 -- 172.21.15.43:0/2723277031 --> v2:172.21.15.203:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7ea406b480 con 0x7f7ea40c8470 2023-08-09T14:52:05.615 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:05.614+0000 7f7eaa19a640 1 --2- 172.21.15.43:0/2723277031 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f7ea40c8470 0x7f7ea40c88c0 secure :-1 s=READY pgs=706 cs=0 l=1 rev1=1 crypto rx=0x7f7e9c009e50 tx=0x7f7e9c02bed0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=ff93f948195a00c1 server_cookie=0 in_seq=0 out_seq=0 2023-08-09T14:52:05.616 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:05.615+0000 7f7ea8997640 1 -- 172.21.15.43:0/2723277031 <== mon.1 v2:172.21.15.203:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7e9c02ca70 con 0x7f7ea40c8470 2023-08-09T14:52:05.616 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:05.615+0000 7f7ea8997640 1 -- 172.21.15.43:0/2723277031 <== mon.1 v2:172.21.15.203:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7e9c02cbd0 con 0x7f7ea40c8470 2023-08-09T14:52:05.616 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:05.615+0000 7f7ea8997640 1 -- 172.21.15.43:0/2723277031 <== mon.1 v2:172.21.15.203:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7e9c0343f0 con 0x7f7ea40c8470 2023-08-09T14:52:05.617 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:05.615+0000 7f7eaa99b640 1 -- 172.21.15.43:0/2723277031 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f7ea40c8470 msgr2=0x7f7ea40c88c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T14:52:05.617 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:05.615+0000 7f7eaa99b640 1 --2- 172.21.15.43:0/2723277031 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f7ea40c8470 0x7f7ea40c88c0 secure :-1 s=READY pgs=706 cs=0 l=1 rev1=1 crypto rx=0x7f7e9c009e50 tx=0x7f7e9c02bed0 comp rx=0 tx=0).stop 2023-08-09T14:52:05.617 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:05.615+0000 7f7eaa99b640 1 -- 172.21.15.43:0/2723277031 shutdown_connections 2023-08-09T14:52:05.618 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:05.615+0000 7f7eaa99b640 1 --2- 172.21.15.43:0/2723277031 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f7ea40c8470 0x7f7ea40c88c0 unknown :-1 s=CLOSED pgs=706 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T14:52:05.618 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:05.615+0000 7f7eaa99b640 1 --2- 172.21.15.43:0/2723277031 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f7ea4157ce0 0x7f7ea415b0f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T14:52:05.618 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:05.615+0000 7f7eaa99b640 1 -- 172.21.15.43:0/2723277031 >> 172.21.15.43:0/2723277031 conn(0x7f7ea40af9f0 msgr2=0x7f7ea40afdd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-09T14:52:05.618 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:05.616+0000 7f7eaa99b640 1 -- 172.21.15.43:0/2723277031 shutdown_connections 2023-08-09T14:52:05.618 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:05.616+0000 7f7eaa99b640 1 -- 172.21.15.43:0/2723277031 wait complete. 2023-08-09T14:52:05.619 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:05.618+0000 7f7eaa99b640 1 Processor -- start 2023-08-09T14:52:05.620 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:05.618+0000 7f7eaa99b640 1 -- start start 2023-08-09T14:52:05.621 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:05.620+0000 7f7eaa99b640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f7ea406f2c0 0x7f7ea415de90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T14:52:05.621 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:05.620+0000 7f7eaa99b640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f7ea40c8470 0x7f7ea4154c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T14:52:05.622 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:05.620+0000 7f7eaa99b640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f7ea4157ce0 0x7f7ea41551b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T14:52:05.622 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:05.620+0000 7f7eaa99b640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7ea41f5ee0 con 0x7f7ea40c8470 2023-08-09T14:52:05.622 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:05.620+0000 7f7eaa99b640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7ea40c60f0 con 0x7f7ea4157ce0 2023-08-09T14:52:05.622 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:05.620+0000 7f7eaa99b640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7ea41f4ff0 con 0x7f7ea406f2c0 2023-08-09T14:52:05.622 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:05.620+0000 7f7ea9198640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f7ea40c8470 0x7f7ea4154c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T14:52:05.622 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:05.620+0000 7f7ea9198640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f7ea40c8470 0x7f7ea4154c70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:34774/0 (socket says 172.21.15.43:34774) 2023-08-09T14:52:05.622 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:05.620+0000 7f7ea9198640 1 -- 172.21.15.43:0/3033679729 learned_addr learned my addr 172.21.15.43:0/3033679729 (peer_addr_for_me v2:172.21.15.43:0/0) 2023-08-09T14:52:05.623 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:05.620+0000 7f7ea9999640 1 --2- 172.21.15.43:0/3033679729 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f7ea406f2c0 0x7f7ea415de90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T14:52:05.624 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:05.620+0000 7f7eaa19a640 1 --2- 172.21.15.43:0/3033679729 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f7ea4157ce0 0x7f7ea41551b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T14:52:05.624 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:05.620+0000 7f7ea9198640 1 -- 172.21.15.43:0/3033679729 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f7ea406f2c0 msgr2=0x7f7ea415de90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T14:52:05.624 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:05.620+0000 7f7ea9198640 1 --2- 172.21.15.43:0/3033679729 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f7ea406f2c0 0x7f7ea415de90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T14:52:05.624 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:05.621+0000 7f7ea9198640 1 -- 172.21.15.43:0/3033679729 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f7ea4157ce0 msgr2=0x7f7ea41551b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T14:52:05.624 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:05.621+0000 7f7ea9198640 1 --2- 172.21.15.43:0/3033679729 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f7ea4157ce0 0x7f7ea41551b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T14:52:05.624 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:05.621+0000 7f7ea9198640 1 -- 172.21.15.43:0/3033679729 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7e9c009b30 con 0x7f7ea40c8470 2023-08-09T14:52:05.624 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:05.621+0000 7f7ea9198640 1 --2- 172.21.15.43:0/3033679729 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f7ea40c8470 0x7f7ea4154c70 secure :-1 s=READY pgs=1916 cs=0 l=1 rev1=1 crypto rx=0x7f7e9400c400 tx=0x7f7e9400c8c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-09T14:52:05.625 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:05.621+0000 7f7e9a7fc640 1 -- 172.21.15.43:0/3033679729 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7e940052d0 con 0x7f7ea40c8470 2023-08-09T14:52:05.625 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:05.621+0000 7f7e9a7fc640 1 -- 172.21.15.43:0/3033679729 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7e94005430 con 0x7f7ea40c8470 2023-08-09T14:52:05.625 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:05.621+0000 7f7e9a7fc640 1 -- 172.21.15.43:0/3033679729 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7e94003830 con 0x7f7ea40c8470 2023-08-09T14:52:05.625 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:05.621+0000 7f7eaa99b640 1 -- 172.21.15.43:0/3033679729 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7ea41f52d0 con 0x7f7ea40c8470 2023-08-09T14:52:05.625 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:05.621+0000 7f7eaa99b640 1 -- 172.21.15.43:0/3033679729 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7ea41f5890 con 0x7f7ea40c8470 2023-08-09T14:52:05.626 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:05.623+0000 7f7eaa99b640 1 -- 172.21.15.43:0/3033679729 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7ea404e390 con 0x7f7ea40c8470 2023-08-09T14:52:05.626 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:05.624+0000 7f7e9a7fc640 1 -- 172.21.15.43:0/3033679729 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 83) v1 ==== 88461+0+0 (secure 0 0 0) 0x7f7e94003990 con 0x7f7ea40c8470 2023-08-09T14:52:05.626 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:05.625+0000 7f7e9a7fc640 1 --2- 172.21.15.43:0/3033679729 >> [v2:172.21.15.203:6832/843920641,v1:172.21.15.203:6833/843920641] conn(0x7f7e8806a1f0 0x7f7e8806c6b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T14:52:05.626 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:05.625+0000 7f7e9a7fc640 1 -- 172.21.15.43:0/3033679729 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(173..173 src has 1..173) v4 ==== 6770+0+0 (secure 0 0 0) 0x7f7e94087df0 con 0x7f7ea40c8470 2023-08-09T14:52:05.627 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:05.625+0000 7f7ea9999640 1 --2- 172.21.15.43:0/3033679729 >> [v2:172.21.15.203:6832/843920641,v1:172.21.15.203:6833/843920641] conn(0x7f7e8806a1f0 0x7f7e8806c6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T14:52:05.628 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:05.627+0000 7f7ea9999640 1 --2- 172.21.15.43:0/3033679729 >> [v2:172.21.15.203:6832/843920641,v1:172.21.15.203:6833/843920641] conn(0x7f7e8806a1f0 0x7f7e8806c6b0 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7f7ea4157340 tx=0x7f7ea0008040 comp rx=0 tx=0).ready entity=mgr.8020 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-09T14:52:05.630 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:05.629+0000 7f7e9a7fc640 1 -- 172.21.15.43:0/3033679729 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178066 (secure 0 0 0) 0x7f7e94053100 con 0x7f7ea40c8470 2023-08-09T14:52:05.875 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:05.873+0000 7f7eaa99b640 1 -- 172.21.15.43:0/3033679729 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "mgr module disable", "module": "snap_schedule"} v 0) v1 -- 0x7f7ea40c8e00 con 0x7f7ea40c8470 2023-08-09T14:52:06.011 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:06.009+0000 7f7e9a7fc640 1 -- 172.21.15.43:0/3033679729 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mgrmap(e 84) v1 ==== 88444+0+0 (secure 0 0 0) 0x7f7e94085ee0 con 0x7f7ea40c8470 2023-08-09T14:52:06.014 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:06.012+0000 7f7e9a7fc640 1 -- 172.21.15.43:0/3033679729 <== mon.0 v2:172.21.15.43:3300/0 8 ==== mon_command_ack([{"prefix": "mgr module disable", "module": "snap_schedule"}]=0 v84) v1 ==== 93+0+0 (secure 0 0 0) 0x7f7e94056b90 con 0x7f7ea40c8470 2023-08-09T14:52:06.017 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:06.016+0000 7f7eaa99b640 1 -- 172.21.15.43:0/3033679729 >> [v2:172.21.15.203:6832/843920641,v1:172.21.15.203:6833/843920641] conn(0x7f7e8806a1f0 msgr2=0x7f7e8806c6b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T14:52:06.018 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:06.016+0000 7f7eaa99b640 1 --2- 172.21.15.43:0/3033679729 >> [v2:172.21.15.203:6832/843920641,v1:172.21.15.203:6833/843920641] conn(0x7f7e8806a1f0 0x7f7e8806c6b0 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7f7ea4157340 tx=0x7f7ea0008040 comp rx=0 tx=0).stop 2023-08-09T14:52:06.018 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:06.016+0000 7f7eaa99b640 1 -- 172.21.15.43:0/3033679729 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f7ea40c8470 msgr2=0x7f7ea4154c70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T14:52:06.018 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:06.016+0000 7f7eaa99b640 1 --2- 172.21.15.43:0/3033679729 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f7ea40c8470 0x7f7ea4154c70 secure :-1 s=READY pgs=1916 cs=0 l=1 rev1=1 crypto rx=0x7f7e9400c400 tx=0x7f7e9400c8c0 comp rx=0 tx=0).stop 2023-08-09T14:52:06.018 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:06.016+0000 7f7eaa99b640 1 -- 172.21.15.43:0/3033679729 shutdown_connections 2023-08-09T14:52:06.019 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:06.016+0000 7f7eaa99b640 1 --2- 172.21.15.43:0/3033679729 >> [v2:172.21.15.203:6832/843920641,v1:172.21.15.203:6833/843920641] conn(0x7f7e8806a1f0 0x7f7e8806c6b0 unknown :-1 s=CLOSED pgs=26 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T14:52:06.019 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:06.016+0000 7f7eaa99b640 1 --2- 172.21.15.43:0/3033679729 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f7ea4157ce0 0x7f7ea41551b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T14:52:06.019 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:06.016+0000 7f7eaa99b640 1 --2- 172.21.15.43:0/3033679729 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f7ea40c8470 0x7f7ea4154c70 unknown :-1 s=CLOSED pgs=1916 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T14:52:06.019 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:06.016+0000 7f7eaa99b640 1 --2- 172.21.15.43:0/3033679729 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f7ea406f2c0 0x7f7ea415de90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T14:52:06.019 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:06.016+0000 7f7eaa99b640 1 -- 172.21.15.43:0/3033679729 >> 172.21.15.43:0/3033679729 conn(0x7f7ea40af9f0 msgr2=0x7f7ea40c9430 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-09T14:52:06.020 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:06.017+0000 7f7eaa99b640 1 -- 172.21.15.43:0/3033679729 shutdown_connections 2023-08-09T14:52:06.020 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:06.017+0000 7f7eaa99b640 1 -- 172.21.15.43:0/3033679729 wait complete. 2023-08-09T14:52:06.040 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2023-08-09T14:52:06.041 DEBUG:teuthology.orchestra.run.smithi043:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2023-08-09T14:52:06.155 DEBUG:teuthology.orchestra.run.smithi203:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2023-08-09T14:52:06.232 INFO:tasks.ceph.mgr.y.smithi043.stderr:2023-08-09T14:52:06.231+0000 7f35b420d040 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2023-08-09T14:52:06.237 INFO:tasks.ceph.mgr.x.smithi203.stderr:2023-08-09T14:52:06.237+0000 7fb798253040 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2023-08-09T14:52:06.255 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2023-08-09T14:52:06.256 DEBUG:teuthology.orchestra.run.smithi043:> set -ex 2023-08-09T14:52:06.257 DEBUG:teuthology.orchestra.run.smithi043:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-09T14:52:06.318 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-08-09T14:52:06.535 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:06.534+0000 7f36d5566640 1 Processor -- start 2023-08-09T14:52:06.535 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:06.534+0000 7f36d5566640 1 -- start start 2023-08-09T14:52:06.537 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:06.535+0000 7f36d5566640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f36d00cc980 0x7f36d00ce620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T14:52:06.537 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:06.535+0000 7f36d5566640 1 -- --> v1:172.21.15.43:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f36d015a720 con 0x7f36d015a340 2023-08-09T14:52:06.537 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:06.535+0000 7f36d5566640 1 -- --> v1:172.21.15.203:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f36d00cd970 con 0x7f36d00ccdf0 2023-08-09T14:52:06.537 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:06.535+0000 7f36d5566640 1 -- --> v2:172.21.15.43:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f36d00b0190 con 0x7f36d00cc980 2023-08-09T14:52:06.538 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:06.536+0000 7f36cffff640 1 --1- >> v1:172.21.15.43:6789/0 conn(0x7f36d015a340 0x7f36d00cac60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.43:6789/0 says I am v1:172.21.15.43:41502/0 (socket says 172.21.15.43:41502) 2023-08-09T14:52:06.538 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:06.536+0000 7f36cffff640 1 -- 172.21.15.43:0/1829414828 learned_addr learned my addr 172.21.15.43:0/1829414828 (peer_addr_for_me v1:172.21.15.43:0/0) 2023-08-09T14:52:06.538 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:06.536+0000 7f36cf7fe640 1 --2- 172.21.15.43:0/1829414828 >> v2:172.21.15.43:3300/0 conn(0x7f36d00cc980 0x7f36d00ce620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T14:52:06.538 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:06.536+0000 7f36cf7fe640 1 -- 172.21.15.43:0/1829414828 >> v1:172.21.15.203:6789/0 conn(0x7f36d00ccdf0 legacy=0x7f36d00b0620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-09T14:52:06.538 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:06.536+0000 7f36cf7fe640 1 -- 172.21.15.43:0/1829414828 >> v1:172.21.15.43:6789/0 conn(0x7f36d015a340 legacy=0x7f36d00cac60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T14:52:06.538 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:06.536+0000 7f36cf7fe640 1 -- 172.21.15.43:0/1829414828 --> v2:172.21.15.43:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f36d00cd3b0 con 0x7f36d00cc980 2023-08-09T14:52:06.539 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:06.538+0000 7f36cf7fe640 1 --2- 172.21.15.43:0/1829414828 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f36d00cc980 0x7f36d00ce620 secure :-1 s=READY pgs=1922 cs=0 l=1 rev1=1 crypto rx=0x7f36b80097d0 tx=0x7f36b802b790 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2ed6ba246b532f65 server_cookie=0 in_seq=0 out_seq=0 2023-08-09T14:52:06.540 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:06.538+0000 7f36ceffd640 1 -- 172.21.15.43:0/1829414828 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f36b80316f0 con 0x7f36d00cc980 2023-08-09T14:52:06.540 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:06.538+0000 7f36ceffd640 1 -- 172.21.15.43:0/1829414828 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f36b8031850 con 0x7f36d00cc980 2023-08-09T14:52:06.540 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:06.538+0000 7f36ceffd640 1 -- 172.21.15.43:0/1829414828 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f36b8031be0 con 0x7f36d00cc980 2023-08-09T14:52:06.541 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:06.538+0000 7f36d5566640 1 -- 172.21.15.43:0/1829414828 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f36d00cc980 msgr2=0x7f36d00ce620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T14:52:06.541 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:06.538+0000 7f36d5566640 1 --2- 172.21.15.43:0/1829414828 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f36d00cc980 0x7f36d00ce620 secure :-1 s=READY pgs=1922 cs=0 l=1 rev1=1 crypto rx=0x7f36b80097d0 tx=0x7f36b802b790 comp rx=0 tx=0).stop 2023-08-09T14:52:06.541 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:06.539+0000 7f36d5566640 1 -- 172.21.15.43:0/1829414828 shutdown_connections 2023-08-09T14:52:06.541 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:06.539+0000 7f36d5566640 1 --2- 172.21.15.43:0/1829414828 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f36d00cc980 0x7f36d00ce620 unknown :-1 s=CLOSED pgs=1922 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T14:52:06.542 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:06.539+0000 7f36d5566640 1 -- 172.21.15.43:0/1829414828 >> 172.21.15.43:0/1829414828 conn(0x7f36d0089fe0 msgr2=0x7f36d0099470 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-09T14:52:06.542 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:06.539+0000 7f36d5566640 1 -- 172.21.15.43:0/1829414828 shutdown_connections 2023-08-09T14:52:06.542 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:06.539+0000 7f36d5566640 1 -- 172.21.15.43:0/1829414828 wait complete. 2023-08-09T14:52:06.545 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:06.540+0000 7f36d5566640 1 Processor -- start 2023-08-09T14:52:06.545 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:06.541+0000 7f36d5566640 1 -- start start 2023-08-09T14:52:06.546 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:06.541+0000 7f36d5566640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f36d00cc980 0x7f36d014e860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T14:52:06.546 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:06.541+0000 7f36d5566640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f36d00ccdf0 0x7f36d006b9d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T14:52:06.546 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:06.541+0000 7f36d5566640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f36d015a340 0x7f36d015dec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T14:52:06.547 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:06.541+0000 7f36d5566640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f36d015d7a0 con 0x7f36d00cc980 2023-08-09T14:52:06.547 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:06.541+0000 7f36d5566640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f36d014eda0 con 0x7f36d00ccdf0 2023-08-09T14:52:06.547 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:06.541+0000 7f36d5566640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f36d015a8c0 con 0x7f36d015a340 2023-08-09T14:52:06.547 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:06.542+0000 7f36cffff640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f36d00cc980 0x7f36d014e860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T14:52:06.548 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:06.542+0000 7f36cffff640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f36d00cc980 0x7f36d014e860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:34814/0 (socket says 172.21.15.43:34814) 2023-08-09T14:52:06.548 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:06.542+0000 7f36cffff640 1 -- 172.21.15.43:0/60345695 learned_addr learned my addr 172.21.15.43:0/60345695 (peer_addr_for_me v2:172.21.15.43:0/0) 2023-08-09T14:52:06.548 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:06.542+0000 7f36cf7fe640 1 --2- 172.21.15.43:0/60345695 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f36d00ccdf0 0x7f36d006b9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T14:52:06.548 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:06.542+0000 7f36d4d65640 1 --2- 172.21.15.43:0/60345695 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f36d015a340 0x7f36d015dec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T14:52:06.549 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:06.542+0000 7f36cffff640 1 -- 172.21.15.43:0/60345695 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f36d015a340 msgr2=0x7f36d015dec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T14:52:06.549 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:06.542+0000 7f36cffff640 1 --2- 172.21.15.43:0/60345695 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f36d015a340 0x7f36d015dec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T14:52:06.549 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:06.542+0000 7f36cffff640 1 -- 172.21.15.43:0/60345695 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f36d00ccdf0 msgr2=0x7f36d006b9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T14:52:06.549 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:06.542+0000 7f36cffff640 1 --2- 172.21.15.43:0/60345695 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f36d00ccdf0 0x7f36d006b9d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T14:52:06.550 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:06.542+0000 7f36cffff640 1 -- 172.21.15.43:0/60345695 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f36b8009400 con 0x7f36d00cc980 2023-08-09T14:52:06.550 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:06.542+0000 7f36cf7fe640 1 --2- 172.21.15.43:0/60345695 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f36d00ccdf0 0x7f36d006b9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-08-09T14:52:06.550 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:06.543+0000 7f36cffff640 1 --2- 172.21.15.43:0/60345695 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f36d00cc980 0x7f36d014e860 secure :-1 s=READY pgs=1923 cs=0 l=1 rev1=1 crypto rx=0x7f36c000c480 tx=0x7f36c000c940 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-09T14:52:06.550 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:06.543+0000 7f36ccff9640 1 -- 172.21.15.43:0/60345695 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f36c0005ce0 con 0x7f36d00cc980 2023-08-09T14:52:06.551 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:06.543+0000 7f36ccff9640 1 -- 172.21.15.43:0/60345695 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f36c0005e40 con 0x7f36d00cc980 2023-08-09T14:52:06.551 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:06.543+0000 7f36d5566640 1 -- 172.21.15.43:0/60345695 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f36d015ae80 con 0x7f36d00cc980 2023-08-09T14:52:06.551 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:06.543+0000 7f36ccff9640 1 -- 172.21.15.43:0/60345695 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f36c00072f0 con 0x7f36d00cc980 2023-08-09T14:52:06.551 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:06.543+0000 7f36d5566640 1 -- 172.21.15.43:0/60345695 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f36d006c770 con 0x7f36d00cc980 2023-08-09T14:52:06.553 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:06.544+0000 7f36d5566640 1 -- 172.21.15.43:0/60345695 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f36d00739f0 con 0x7f36d00cc980 2023-08-09T14:52:06.553 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:06.545+0000 7f36ccff9640 1 -- 172.21.15.43:0/60345695 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 84) v1 ==== 88444+0+0 (secure 0 0 0) 0x7f36c0007510 con 0x7f36d00cc980 2023-08-09T14:52:06.553 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:06.546+0000 7f36ccff9640 1 --2- 172.21.15.43:0/60345695 >> [v2:172.21.15.203:6832/843920641,v1:172.21.15.203:6833/843920641] conn(0x7f36b006a170 0x7f36b006c630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T14:52:06.553 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:06.546+0000 7f36ccff9640 1 -- 172.21.15.43:0/60345695 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(173..173 src has 1..173) v4 ==== 6770+0+0 (secure 0 0 0) 0x7f36c0087d40 con 0x7f36d00cc980 2023-08-09T14:52:06.554 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:06.546+0000 7f36cf7fe640 1 -- 172.21.15.43:0/60345695 >> [v2:172.21.15.203:6832/843920641,v1:172.21.15.203:6833/843920641] conn(0x7f36b006a170 msgr2=0x7f36b006c630 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.203:6832/843920641 2023-08-09T14:52:06.554 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:06.546+0000 7f36cf7fe640 1 --2- 172.21.15.43:0/60345695 >> [v2:172.21.15.203:6832/843920641,v1:172.21.15.203:6833/843920641] conn(0x7f36b006a170 0x7f36b006c630 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-08-09T14:52:06.554 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:06.550+0000 7f36ccff9640 1 -- 172.21.15.43:0/60345695 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178066 (secure 0 0 0) 0x7f36c00530f0 con 0x7f36d00cc980 2023-08-09T14:52:06.745 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:06.743+0000 7f36d5566640 1 -- 172.21.15.43:0/60345695 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f36d015a720 con 0x7f36d00cc980 2023-08-09T14:52:06.747 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:06.746+0000 7f36ccff9640 1 -- 172.21.15.43:0/60345695 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v173) v1 ==== 74+0+19074 (secure 0 0 0) 0x7f36c007fe60 con 0x7f36d00cc980 2023-08-09T14:52:06.748 INFO:teuthology.orchestra.run.smithi043.stdout: 2023-08-09T14:52:06.748 INFO:teuthology.orchestra.run.smithi043.stdout:{"epoch":173,"fsid":"daac87ee-1337-4250-a955-aea4e408a39b","created":"2023-08-09T13:33:41.701198+0000","modified":"2023-08-09T14:52:02.909364+0000","last_up_change":"2023-08-09T13:33:53.783453+0000","last_in_change":"2023-08-09T13:33:45.148904+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":21,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":"rbd","create_time":"2023-08-09T13:33:54.365137+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":".mgr","create_time":"2023-08-09T13:33:54.831803+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":20,"pool_name":"cephfs_metadata","create_time":"2023-08-09T14:51:45.692613+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"171","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":21,"pool_name":"cephfs_data","create_time":"2023-08-09T14:51:46.286715+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"172","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"cbd49ecd-e896-4fbd-864e-5c37a829b165","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":170,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6800","nonce":502667335},{"type":"v1","addr":"172.21.15.43:6801","nonce":502667335}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6802","nonce":502667335},{"type":"v1","addr":"172.21.15.43:6803","nonce":502667335}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6806","nonce":502667335},{"type":"v1","addr":"172.21.15.43:6807","nonce":502667335}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6804","nonce":502667335},{"type":"v1","addr":"172.21.15.43:6805","nonce":502667335}]},"public_addr":"172.21.15.43:6801/502667335","cluster_addr":"172.21.15.43:6803/502667335","heartbeat_back_addr":"172.21.15.43:6807/502667335","heartbeat_front_addr":"172.21.15.43:6805/502667335","state":["exists","up"]},{"osd":1,"uuid":"75c7e0fe-1ec7-4fe8-a2f2-3661a65b8500","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":170,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6808","nonce":3289134610},{"type":"v1","addr":"172.21.15.43:6809","nonce":3289134610}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6810","nonce":3289134610},{"type":"v1","addr":"172.21.15.43:6812","nonce":3289134610}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6821","nonce":3289134610},{"type":"v1","addr":"172.21.15.43:6824","nonce":3289134610}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6815","nonce":3289134610},{"type":"v1","addr":"172.21.15.43:6818","nonce":3289134610}]},"public_addr":"172.21.15.43:6809/3289134610","cluster_addr":"172.21.15.43:6812/3289134610","heartbeat_back_addr":"172.21.15.43:6824/3289134610","heartbeat_front_addr":"172.21.15.43:6818/3289134610","state":["exists","up"]},{"osd":2,"uuid":"bb5c41e9-7f5f-4125-9b01-3bb8bfc2ea83","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":170,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6811","nonce":4079709736},{"type":"v1","addr":"172.21.15.43:6814","nonce":4079709736}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6817","nonce":4079709736},{"type":"v1","addr":"172.21.15.43:6820","nonce":4079709736}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6829","nonce":4079709736},{"type":"v1","addr":"172.21.15.43:6831","nonce":4079709736}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6823","nonce":4079709736},{"type":"v1","addr":"172.21.15.43:6826","nonce":4079709736}]},"public_addr":"172.21.15.43:6814/4079709736","cluster_addr":"172.21.15.43:6820/4079709736","heartbeat_back_addr":"172.21.15.43:6831/4079709736","heartbeat_front_addr":"172.21.15.43:6826/4079709736","state":["exists","up"]},{"osd":3,"uuid":"bfe4c8b4-694a-465c-87e6-2b2aab852613","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":170,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6813","nonce":1709538759},{"type":"v1","addr":"172.21.15.43:6816","nonce":1709538759}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6819","nonce":1709538759},{"type":"v1","addr":"172.21.15.43:6822","nonce":1709538759}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6828","nonce":1709538759},{"type":"v1","addr":"172.21.15.43:6830","nonce":1709538759}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6825","nonce":1709538759},{"type":"v1","addr":"172.21.15.43:6827","nonce":1709538759}]},"public_addr":"172.21.15.43:6816/1709538759","cluster_addr":"172.21.15.43:6822/1709538759","heartbeat_back_addr":"172.21.15.43:6830/1709538759","heartbeat_front_addr":"172.21.15.43:6827/1709538759","state":["exists","up"]},{"osd":4,"uuid":"fa2e8a77-aba1-4756-853e-0d174a5bdeba","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":170,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6800","nonce":4196866481},{"type":"v1","addr":"172.21.15.203:6801","nonce":4196866481}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6802","nonce":4196866481},{"type":"v1","addr":"172.21.15.203:6803","nonce":4196866481}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6806","nonce":4196866481},{"type":"v1","addr":"172.21.15.203:6807","nonce":4196866481}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6804","nonce":4196866481},{"type":"v1","addr":"172.21.15.203:6805","nonce":4196866481}]},"public_addr":"172.21.15.203:6801/4196866481","cluster_addr":"172.21.15.203:6803/4196866481","heartbeat_back_addr":"172.21.15.203:6807/4196866481","heartbeat_front_addr":"172.21.15.203:6805/4196866481","state":["exists","up"]},{"osd":5,"uuid":"81811e4b-4d24-4f51-a82c-f1d700a2a00d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":170,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6816","nonce":3059290081},{"type":"v1","addr":"172.21.15.203:6817","nonce":3059290081}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6818","nonce":3059290081},{"type":"v1","addr":"172.21.15.203:6819","nonce":3059290081}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6822","nonce":3059290081},{"type":"v1","addr":"172.21.15.203:6823","nonce":3059290081}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6820","nonce":3059290081},{"type":"v1","addr":"172.21.15.203:6821","nonce":3059290081}]},"public_addr":"172.21.15.203:6817/3059290081","cluster_addr":"172.21.15.203:6819/3059290081","heartbeat_back_addr":"172.21.15.203:6823/3059290081","heartbeat_front_addr":"172.21.15.203:6821/3059290081","state":["exists","up"]},{"osd":6,"uuid":"307bac4d-6a5b-443d-af75-56d695d7674f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":170,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6824","nonce":512155569},{"type":"v1","addr":"172.21.15.203:6825","nonce":512155569}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6826","nonce":512155569},{"type":"v1","addr":"172.21.15.203:6827","nonce":512155569}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6830","nonce":512155569},{"type":"v1","addr":"172.21.15.203:6831","nonce":512155569}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6828","nonce":512155569},{"type":"v1","addr":"172.21.15.203:6829","nonce":512155569}]},"public_addr":"172.21.15.203:6825/512155569","cluster_addr":"172.21.15.203:6827/512155569","heartbeat_back_addr":"172.21.15.203:6831/512155569","heartbeat_front_addr":"172.21.15.203:6829/512155569","state":["exists","up"]},{"osd":7,"uuid":"a95dc7ca-9f1b-4295-af70-68a22a2d7d5e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":170,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6808","nonce":3331450438},{"type":"v1","addr":"172.21.15.203:6809","nonce":3331450438}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6810","nonce":3331450438},{"type":"v1","addr":"172.21.15.203:6811","nonce":3331450438}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6814","nonce":3331450438},{"type":"v1","addr":"172.21.15.203:6815","nonce":3331450438}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6812","nonce":3331450438},{"type":"v1","addr":"172.21.15.203:6813","nonce":3331450438}]},"public_addr":"172.21.15.203:6809/3331450438","cluster_addr":"172.21.15.203:6811/3331450438","heartbeat_back_addr":"172.21.15.203:6815/3331450438","heartbeat_front_addr":"172.21.15.203:6813/3331450438","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-09T13:33:50.547652+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-09T13:33:50.589469+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-09T13:33:50.561724+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-09T13:33:50.599650+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-09T13:33:50.411855+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-09T13:33:50.219681+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-09T13:33:50.504664+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-09T13:33:50.441248+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.203:0/2095212004":"2023-08-10T14:52:02.909250+0000","172.21.15.203:0/3749872091":"2023-08-10T14:52:02.909250+0000","172.21.15.203:0/651612122":"2023-08-10T14:52:02.909250+0000","172.21.15.203:0/4120954828":"2023-08-10T14:52:02.909250+0000","172.21.15.203:0/2783782601":"2023-08-10T14:52:02.909250+0000","172.21.15.203:0/2232659577":"2023-08-10T14:52:02.909250+0000","172.21.15.203:6833/2566816332":"2023-08-10T14:52:02.909250+0000","172.21.15.203:6832/2566816332":"2023-08-10T14:52:02.909250+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":7,"snaps":[{"begin":104,"length":1},{"begin":221,"length":1}]}],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-08-09T14:52:06.749 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:06.747+0000 7f36cf7fe640 1 -- 172.21.15.43:0/60345695 >> [v2:172.21.15.203:6832/843920641,v1:172.21.15.203:6833/843920641] conn(0x7f36b006a170 msgr2=0x7f36b006c630 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.203:6832/843920641 2023-08-09T14:52:06.749 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:06.747+0000 7f36cf7fe640 1 --2- 172.21.15.43:0/60345695 >> [v2:172.21.15.203:6832/843920641,v1:172.21.15.203:6833/843920641] conn(0x7f36b006a170 0x7f36b006c630 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2023-08-09T14:52:06.750 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:06.748+0000 7f36d5566640 1 -- 172.21.15.43:0/60345695 >> [v2:172.21.15.203:6832/843920641,v1:172.21.15.203:6833/843920641] conn(0x7f36b006a170 msgr2=0x7f36b006c630 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-08-09T14:52:06.750 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:06.748+0000 7f36d5566640 1 --2- 172.21.15.43:0/60345695 >> [v2:172.21.15.203:6832/843920641,v1:172.21.15.203:6833/843920641] conn(0x7f36b006a170 0x7f36b006c630 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T14:52:06.751 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:06.749+0000 7f36d5566640 1 -- 172.21.15.43:0/60345695 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f36d00cc980 msgr2=0x7f36d014e860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T14:52:06.751 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:06.749+0000 7f36d5566640 1 --2- 172.21.15.43:0/60345695 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f36d00cc980 0x7f36d014e860 secure :-1 s=READY pgs=1923 cs=0 l=1 rev1=1 crypto rx=0x7f36c000c480 tx=0x7f36c000c940 comp rx=0 tx=0).stop 2023-08-09T14:52:06.751 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:06.749+0000 7f36d5566640 1 -- 172.21.15.43:0/60345695 shutdown_connections 2023-08-09T14:52:06.751 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:06.749+0000 7f36d5566640 1 --2- 172.21.15.43:0/60345695 >> [v2:172.21.15.203:6832/843920641,v1:172.21.15.203:6833/843920641] conn(0x7f36b006a170 0x7f36b006c630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T14:52:06.751 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:06.749+0000 7f36d5566640 1 --2- 172.21.15.43:0/60345695 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f36d015a340 0x7f36d015dec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T14:52:06.751 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:06.749+0000 7f36d5566640 1 --2- 172.21.15.43:0/60345695 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f36d00ccdf0 0x7f36d006b9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T14:52:06.751 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:06.749+0000 7f36d5566640 1 --2- 172.21.15.43:0/60345695 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f36d00cc980 0x7f36d014e860 unknown :-1 s=CLOSED pgs=1923 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T14:52:06.751 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:06.749+0000 7f36d5566640 1 -- 172.21.15.43:0/60345695 >> 172.21.15.43:0/60345695 conn(0x7f36d0089fe0 msgr2=0x7f36d00b09b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-09T14:52:06.752 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:06.749+0000 7f36d5566640 1 -- 172.21.15.43:0/60345695 shutdown_connections 2023-08-09T14:52:06.752 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:06.749+0000 7f36d5566640 1 -- 172.21.15.43:0/60345695 wait complete. 2023-08-09T14:52:06.769 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': 'rbd', 'create_time': '2023-08-09T13:33:54.365137+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}, 'read_balance': {'score_acting': 2, 'score_stable': 2, 'optimal_score': 1, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': '.mgr', 'create_time': '2023-08-09T13:33:54.831803+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 20, 'pool_name': 'cephfs_metadata', 'create_time': '2023-08-09T14:51:45.692613+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '171', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 21, 'pool_name': 'cephfs_data', 'create_time': '2023-08-09T14:51:46.286715+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '172', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.3799999952316284, 'score_stable': 1.3799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.3799999952316284, 'raw_score_stable': 1.3799999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-08-09T14:52:06.770 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get rbd pg_num 2023-08-09T14:52:06.942 INFO:tasks.ceph.mgr.x.smithi203.stderr:2023-08-09T14:52:06.942+0000 7fb798253040 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2023-08-09T14:52:06.992 INFO:tasks.ceph.mgr.y.smithi043.stderr:2023-08-09T14:52:06.991+0000 7f35b420d040 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2023-08-09T14:52:06.993 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:06.992+0000 7fc6f9a77640 1 Processor -- start 2023-08-09T14:52:06.993 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:06.992+0000 7fc6f9a77640 1 -- start start 2023-08-09T14:52:06.995 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:06.993+0000 7fc6f9a77640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7fc6f415a1b0 0x7fc6f40b1ad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T14:52:06.995 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:06.993+0000 7fc6f9a77640 1 -- --> v1:172.21.15.43:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc6f40cea40 con 0x7fc6f415e9c0 2023-08-09T14:52:06.995 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:06.993+0000 7fc6f9a77640 1 -- --> v1:172.21.15.203:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc6f40cc6c0 con 0x7fc6f40cd630 2023-08-09T14:52:06.995 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:06.993+0000 7fc6f9a77640 1 -- --> v2:172.21.15.43:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc6f415dee0 con 0x7fc6f415a1b0 2023-08-09T14:52:06.996 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:06.994+0000 7fc6f8a75640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7fc6f415a1b0 0x7fc6f40b1ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T14:52:06.996 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:06.994+0000 7fc6f8a75640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7fc6f415a1b0 0x7fc6f40b1ad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:34822/0 (socket says 172.21.15.43:34822) 2023-08-09T14:52:06.996 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:06.994+0000 7fc6f9276640 1 --1- >> v1:172.21.15.43:6789/0 conn(0x7fc6f415e9c0 0x7fc6f40ce330 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.43:6789/0 says I am v1:172.21.15.43:41504/0 (socket says 172.21.15.43:41504) 2023-08-09T14:52:06.996 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:06.994+0000 7fc6f8a75640 1 -- 172.21.15.43:0/284362052 learned_addr learned my addr 172.21.15.43:0/284362052 (peer_addr_for_me v2:172.21.15.43:0/0) 2023-08-09T14:52:06.996 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:06.994+0000 7fc6f8a75640 1 -- 172.21.15.43:0/284362052 >> v1:172.21.15.203:6789/0 conn(0x7fc6f40cd630 legacy=0x7fc6f40cdc20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T14:52:06.997 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:06.994+0000 7fc6f8a75640 1 -- 172.21.15.43:0/284362052 >> v1:172.21.15.43:6789/0 conn(0x7fc6f415e9c0 legacy=0x7fc6f40ce330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T14:52:06.997 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:06.994+0000 7fc6f8a75640 1 -- 172.21.15.43:0/284362052 --> v2:172.21.15.43:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc6f415e1c0 con 0x7fc6f415a1b0 2023-08-09T14:52:06.997 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:06.995+0000 7fc6f8a75640 1 --2- 172.21.15.43:0/284362052 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc6f415a1b0 0x7fc6f40b1ad0 secure :-1 s=READY pgs=1925 cs=0 l=1 rev1=1 crypto rx=0x7fc6e8009870 tx=0x7fc6e802ba30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b8ae92e6b7a806f server_cookie=0 in_seq=0 out_seq=0 2023-08-09T14:52:06.997 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:06.996+0000 7fc6f37fe640 1 -- 172.21.15.43:0/284362052 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc6e802f850 con 0x7fc6f415a1b0 2023-08-09T14:52:06.998 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:06.996+0000 7fc6f37fe640 1 -- 172.21.15.43:0/284362052 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc6e802f9b0 con 0x7fc6f415a1b0 2023-08-09T14:52:06.998 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:06.996+0000 7fc6f37fe640 1 -- 172.21.15.43:0/284362052 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc6e802fd40 con 0x7fc6f415a1b0 2023-08-09T14:52:06.998 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:06.996+0000 7fc6f9a77640 1 -- 172.21.15.43:0/284362052 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc6f415a1b0 msgr2=0x7fc6f40b1ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T14:52:06.999 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:06.996+0000 7fc6f9a77640 1 --2- 172.21.15.43:0/284362052 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc6f415a1b0 0x7fc6f40b1ad0 secure :-1 s=READY pgs=1925 cs=0 l=1 rev1=1 crypto rx=0x7fc6e8009870 tx=0x7fc6e802ba30 comp rx=0 tx=0).stop 2023-08-09T14:52:06.999 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:06.997+0000 7fc6f9a77640 1 -- 172.21.15.43:0/284362052 shutdown_connections 2023-08-09T14:52:06.999 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:06.997+0000 7fc6f9a77640 1 --2- 172.21.15.43:0/284362052 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc6f415a1b0 0x7fc6f40b1ad0 unknown :-1 s=CLOSED pgs=1925 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T14:52:06.999 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:06.997+0000 7fc6f9a77640 1 -- 172.21.15.43:0/284362052 >> 172.21.15.43:0/284362052 conn(0x7fc6f40ccad0 msgr2=0x7fc6f40cceb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-09T14:52:06.999 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:06.997+0000 7fc6f9a77640 1 -- 172.21.15.43:0/284362052 shutdown_connections 2023-08-09T14:52:06.999 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:06.997+0000 7fc6f9a77640 1 -- 172.21.15.43:0/284362052 wait complete. 2023-08-09T14:52:07.000 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:06.999+0000 7fc6f9a77640 1 Processor -- start 2023-08-09T14:52:07.000 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:06.999+0000 7fc6f9a77640 1 -- start start 2023-08-09T14:52:07.001 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:07.000+0000 7fc6f9a77640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fc6f40cd630 0x7fc6f40c5a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T14:52:07.002 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:07.000+0000 7fc6f9a77640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc6f415a1b0 0x7fc6f414f7e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T14:52:07.002 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:07.000+0000 7fc6f9a77640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fc6f415e9c0 0x7fc6f4155320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T14:52:07.002 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:07.000+0000 7fc6f9a77640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc6f40c6cd0 con 0x7fc6f415a1b0 2023-08-09T14:52:07.002 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:07.000+0000 7fc6f9a77640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc6f40c60c0 con 0x7fc6f415e9c0 2023-08-09T14:52:07.002 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:07.000+0000 7fc6f9a77640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc6f40c63a0 con 0x7fc6f40cd630 2023-08-09T14:52:07.002 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:07.000+0000 7fc6f3fff640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc6f415a1b0 0x7fc6f414f7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T14:52:07.002 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:07.000+0000 7fc6f3fff640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc6f415a1b0 0x7fc6f414f7e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:34834/0 (socket says 172.21.15.43:34834) 2023-08-09T14:52:07.003 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:07.000+0000 7fc6f3fff640 1 -- 172.21.15.43:0/2503741472 learned_addr learned my addr 172.21.15.43:0/2503741472 (peer_addr_for_me v2:172.21.15.43:0/0) 2023-08-09T14:52:07.004 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:07.001+0000 7fc6f8a75640 1 --2- 172.21.15.43:0/2503741472 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fc6f40cd630 0x7fc6f40c5a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T14:52:07.004 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:07.001+0000 7fc6f9276640 1 --2- 172.21.15.43:0/2503741472 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fc6f415e9c0 0x7fc6f4155320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T14:52:07.004 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:07.001+0000 7fc6f3fff640 1 -- 172.21.15.43:0/2503741472 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fc6f40cd630 msgr2=0x7fc6f40c5a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T14:52:07.004 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:07.001+0000 7fc6f3fff640 1 --2- 172.21.15.43:0/2503741472 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fc6f40cd630 0x7fc6f40c5a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T14:52:07.004 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:07.001+0000 7fc6f3fff640 1 -- 172.21.15.43:0/2503741472 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fc6f415e9c0 msgr2=0x7fc6f4155320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T14:52:07.004 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:07.001+0000 7fc6f3fff640 1 --2- 172.21.15.43:0/2503741472 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fc6f415e9c0 0x7fc6f4155320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T14:52:07.004 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:07.001+0000 7fc6f3fff640 1 -- 172.21.15.43:0/2503741472 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc6e80094d0 con 0x7fc6f415a1b0 2023-08-09T14:52:07.005 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:07.001+0000 7fc6f3fff640 1 --2- 172.21.15.43:0/2503741472 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc6f415a1b0 0x7fc6f414f7e0 secure :-1 s=READY pgs=1926 cs=0 l=1 rev1=1 crypto rx=0x7fc6e000c450 tx=0x7fc6e000c910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-09T14:52:07.005 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:07.001+0000 7fc6f17fa640 1 -- 172.21.15.43:0/2503741472 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc6e00052d0 con 0x7fc6f415a1b0 2023-08-09T14:52:07.005 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:07.002+0000 7fc6f17fa640 1 -- 172.21.15.43:0/2503741472 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc6e0005430 con 0x7fc6f415a1b0 2023-08-09T14:52:07.005 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:07.002+0000 7fc6f17fa640 1 -- 172.21.15.43:0/2503741472 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc6e0012870 con 0x7fc6f415a1b0 2023-08-09T14:52:07.005 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:07.002+0000 7fc6f9a77640 1 -- 172.21.15.43:0/2503741472 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc6f40c6680 con 0x7fc6f415a1b0 2023-08-09T14:52:07.005 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:07.002+0000 7fc6f9a77640 1 -- 172.21.15.43:0/2503741472 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc6f414fd20 con 0x7fc6f415a1b0 2023-08-09T14:52:07.006 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:07.003+0000 7fc6f9a77640 1 -- 172.21.15.43:0/2503741472 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc6f40c6e80 con 0x7fc6f415a1b0 2023-08-09T14:52:07.006 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:07.004+0000 7fc6f17fa640 1 -- 172.21.15.43:0/2503741472 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 84) v1 ==== 88444+0+0 (secure 0 0 0) 0x7fc6e00129d0 con 0x7fc6f415a1b0 2023-08-09T14:52:07.007 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:07.005+0000 7fc6f17fa640 1 --2- 172.21.15.43:0/2503741472 >> [v2:172.21.15.203:6832/843920641,v1:172.21.15.203:6833/843920641] conn(0x7fc6d806a170 0x7fc6d806c630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T14:52:07.007 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:07.005+0000 7fc6f17fa640 1 -- 172.21.15.43:0/2503741472 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(173..173 src has 1..173) v4 ==== 6770+0+0 (secure 0 0 0) 0x7fc6e0088650 con 0x7fc6f415a1b0 2023-08-09T14:52:07.007 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:07.005+0000 7fc6f8a75640 1 -- 172.21.15.43:0/2503741472 >> [v2:172.21.15.203:6832/843920641,v1:172.21.15.203:6833/843920641] conn(0x7fc6d806a170 msgr2=0x7fc6d806c630 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.203:6832/843920641 2023-08-09T14:52:07.007 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:07.005+0000 7fc6f8a75640 1 --2- 172.21.15.43:0/2503741472 >> [v2:172.21.15.203:6832/843920641,v1:172.21.15.203:6833/843920641] conn(0x7fc6d806a170 0x7fc6d806c630 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-08-09T14:52:07.011 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:07.009+0000 7fc6f17fa640 1 -- 172.21.15.43:0/2503741472 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178066 (secure 0 0 0) 0x7fc6f40c6e80 con 0x7fc6f415a1b0 2023-08-09T14:52:07.064 INFO:tasks.ceph.mgr.x.smithi203.stderr:2023-08-09T14:52:07.064+0000 7fb798253040 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2023-08-09T14:52:07.120 INFO:tasks.ceph.mgr.x.smithi203.stderr:2023-08-09T14:52:07.120+0000 7fb798253040 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2023-08-09T14:52:07.121 INFO:tasks.ceph.mgr.y.smithi043.stderr:2023-08-09T14:52:07.119+0000 7f35b420d040 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2023-08-09T14:52:07.170 INFO:tasks.ceph.mgr.x.smithi203.stderr:2023-08-09T14:52:07.170+0000 7fb798253040 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2023-08-09T14:52:07.180 INFO:tasks.ceph.mgr.y.smithi043.stderr:2023-08-09T14:52:07.178+0000 7f35b420d040 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2023-08-09T14:52:07.202 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:07.201+0000 7fc6f9a77640 1 -- 172.21.15.43:0/2503741472 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"} v 0) v1 -- 0x7fc6f404e390 con 0x7fc6f415a1b0 2023-08-09T14:52:07.203 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:07.202+0000 7fc6f17fa640 1 -- 172.21.15.43:0/2503741472 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"}]=0 v173) v1 ==== 92+0+10 (secure 0 0 0) 0x7fc6e0053830 con 0x7fc6f415a1b0 2023-08-09T14:52:07.203 INFO:teuthology.orchestra.run.smithi043.stdout:pg_num: 8 2023-08-09T14:52:07.206 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:07.204+0000 7fc6f9a77640 1 -- 172.21.15.43:0/2503741472 >> [v2:172.21.15.203:6832/843920641,v1:172.21.15.203:6833/843920641] conn(0x7fc6d806a170 msgr2=0x7fc6d806c630 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-08-09T14:52:07.206 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:07.204+0000 7fc6f9a77640 1 --2- 172.21.15.43:0/2503741472 >> [v2:172.21.15.203:6832/843920641,v1:172.21.15.203:6833/843920641] conn(0x7fc6d806a170 0x7fc6d806c630 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T14:52:07.206 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:07.204+0000 7fc6f9a77640 1 -- 172.21.15.43:0/2503741472 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc6f415a1b0 msgr2=0x7fc6f414f7e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T14:52:07.206 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:07.204+0000 7fc6f9a77640 1 --2- 172.21.15.43:0/2503741472 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc6f415a1b0 0x7fc6f414f7e0 secure :-1 s=READY pgs=1926 cs=0 l=1 rev1=1 crypto rx=0x7fc6e000c450 tx=0x7fc6e000c910 comp rx=0 tx=0).stop 2023-08-09T14:52:07.206 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:07.204+0000 7fc6f9a77640 1 -- 172.21.15.43:0/2503741472 shutdown_connections 2023-08-09T14:52:07.207 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:07.204+0000 7fc6f9a77640 1 --2- 172.21.15.43:0/2503741472 >> [v2:172.21.15.203:6832/843920641,v1:172.21.15.203:6833/843920641] conn(0x7fc6d806a170 0x7fc6d806c630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T14:52:07.207 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:07.204+0000 7fc6f9a77640 1 --2- 172.21.15.43:0/2503741472 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fc6f415e9c0 0x7fc6f4155320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T14:52:07.207 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:07.204+0000 7fc6f9a77640 1 --2- 172.21.15.43:0/2503741472 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc6f415a1b0 0x7fc6f414f7e0 unknown :-1 s=CLOSED pgs=1926 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T14:52:07.207 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:07.204+0000 7fc6f9a77640 1 --2- 172.21.15.43:0/2503741472 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fc6f40cd630 0x7fc6f40c5a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T14:52:07.207 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:07.204+0000 7fc6f9a77640 1 -- 172.21.15.43:0/2503741472 >> 172.21.15.43:0/2503741472 conn(0x7fc6f40ccad0 msgr2=0x7fc6f40b1f60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-09T14:52:07.207 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:07.204+0000 7fc6f9a77640 1 -- 172.21.15.43:0/2503741472 shutdown_connections 2023-08-09T14:52:07.207 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:07.205+0000 7fc6f9a77640 1 -- 172.21.15.43:0/2503741472 wait complete. 2023-08-09T14:52:07.223 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-08-09T14:52:07.233 INFO:tasks.ceph.mgr.y.smithi043.stderr:2023-08-09T14:52:07.232+0000 7f35b420d040 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2023-08-09T14:52:07.430 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:07.428+0000 7fe6fa32f640 1 Processor -- start 2023-08-09T14:52:07.430 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:07.429+0000 7fe6fa32f640 1 -- start start 2023-08-09T14:52:07.431 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:07.430+0000 7fe6fa32f640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fe6f4159460 0x7fe6f40cb270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T14:52:07.432 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:07.430+0000 7fe6fa32f640 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7fe6f40706e0 0x7fe6f4070b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T14:52:07.432 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:07.430+0000 7fe6fa32f640 1 -- --> v1:172.21.15.43:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe6f40ce6c0 con 0x7fe6f40cb7b0 2023-08-09T14:52:07.432 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:07.430+0000 7fe6fa32f640 1 -- --> v2:172.21.15.203:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe6f40ce9a0 con 0x7fe6f40706e0 2023-08-09T14:52:07.432 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:07.430+0000 7fe6fa32f640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe6f40710b0 con 0x7fe6f4159460 2023-08-09T14:52:07.432 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:07.430+0000 7fe6f37fe640 1 --1- >> v1:172.21.15.43:6789/0 conn(0x7fe6f40cb7b0 0x7fe6f40cdc20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.43:6789/0 says I am v1:172.21.15.43:41512/0 (socket says 172.21.15.43:41512) 2023-08-09T14:52:07.432 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:07.430+0000 7fe6f37fe640 1 -- 172.21.15.43:0/298637568 learned_addr learned my addr 172.21.15.43:0/298637568 (peer_addr_for_me v1:172.21.15.43:0/0) 2023-08-09T14:52:07.434 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:07.431+0000 7fe6f88cc640 1 --2- 172.21.15.43:0/298637568 >> v2:172.21.15.203:3300/0 conn(0x7fe6f40706e0 0x7fe6f4070b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T14:52:07.434 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:07.431+0000 7fe6f3fff640 1 --2- 172.21.15.43:0/298637568 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fe6f4159460 0x7fe6f40cb270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T14:52:07.434 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:07.431+0000 7fe6f2ffd640 1 -- 172.21.15.43:0/298637568 <== mon.0 v1:172.21.15.43:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1337081416 0 0) 0x7fe6f40ce6c0 con 0x7fe6f40cb7b0 2023-08-09T14:52:07.434 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:07.431+0000 7fe6f2ffd640 1 -- 172.21.15.43:0/298637568 --> v1:172.21.15.43:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fe6dc003520 con 0x7fe6f40cb7b0 2023-08-09T14:52:07.435 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:07.431+0000 7fe6f2ffd640 1 -- 172.21.15.43:0/298637568 <== mon.0 v1:172.21.15.43:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3539601449 0 0) 0x7fe6dc003520 con 0x7fe6f40cb7b0 2023-08-09T14:52:07.435 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:07.431+0000 7fe6f2ffd640 1 -- 172.21.15.43:0/298637568 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fe6f4159460 msgr2=0x7fe6f40cb270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-09T14:52:07.435 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:07.431+0000 7fe6f2ffd640 1 --2- 172.21.15.43:0/298637568 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fe6f4159460 0x7fe6f40cb270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T14:52:07.435 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:07.431+0000 7fe6f2ffd640 1 -- 172.21.15.43:0/298637568 >> v2:172.21.15.203:3300/0 conn(0x7fe6f40706e0 msgr2=0x7fe6f4070b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-09T14:52:07.435 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:07.431+0000 7fe6f2ffd640 1 --2- 172.21.15.43:0/298637568 >> v2:172.21.15.203:3300/0 conn(0x7fe6f40706e0 0x7fe6f4070b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T14:52:07.435 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:07.431+0000 7fe6f2ffd640 1 -- 172.21.15.43:0/298637568 --> v1:172.21.15.43:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe6f40ca790 con 0x7fe6f40cb7b0 2023-08-09T14:52:07.435 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:07.431+0000 7fe6f2ffd640 1 -- 172.21.15.43:0/298637568 <== mon.0 v1:172.21.15.43:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1285150740 0 0) 0x7fe6e8001670 con 0x7fe6f40cb7b0 2023-08-09T14:52:07.435 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:07.431+0000 7fe6f2ffd640 1 -- 172.21.15.43:0/298637568 >> v1:172.21.15.43:6789/0 conn(0x7fe6f40cb7b0 legacy=0x7fe6f40cdc20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T14:52:07.436 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:07.431+0000 7fe6f2ffd640 1 --2- 172.21.15.43:0/298637568 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fe6dc0039a0 0x7fe6dc003dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T14:52:07.436 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:07.431+0000 7fe6f2ffd640 1 --2- 172.21.15.43:0/298637568 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fe6dc004300 0x7fe6dc0047c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T14:52:07.436 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:07.431+0000 7fe6f2ffd640 1 --2- 172.21.15.43:0/298637568 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fe6f4159460 0x7fe6dc004d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T14:52:07.436 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:07.431+0000 7fe6f2ffd640 1 -- 172.21.15.43:0/298637568 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe6f40710b0 con 0x7fe6dc004300 2023-08-09T14:52:07.436 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:07.431+0000 7fe6f2ffd640 1 -- 172.21.15.43:0/298637568 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe6dc005560 con 0x7fe6dc0039a0 2023-08-09T14:52:07.436 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:07.431+0000 7fe6f2ffd640 1 -- 172.21.15.43:0/298637568 --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe6dc00ff90 con 0x7fe6f4159460 2023-08-09T14:52:07.436 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:07.432+0000 7fe6f88cc640 1 --2- 172.21.15.43:0/298637568 >> v2:172.21.15.203:3300/0 conn(0x7fe6f40706e0 0x7fe6f4070b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-08-09T14:52:07.437 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:07.432+0000 7fe6f37fe640 1 --2- 172.21.15.43:0/298637568 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fe6dc004300 0x7fe6dc0047c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T14:52:07.437 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:07.432+0000 7fe6f3fff640 1 --2- 172.21.15.43:0/298637568 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fe6dc0039a0 0x7fe6dc003dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T14:52:07.437 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:07.432+0000 7fe6f88cc640 1 --2- 172.21.15.43:0/298637568 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fe6f4159460 0x7fe6dc004d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T14:52:07.437 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:07.432+0000 7fe6f3fff640 1 -- 172.21.15.43:0/298637568 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fe6f4159460 msgr2=0x7fe6dc004d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-09T14:52:07.437 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:07.432+0000 7fe6f3fff640 1 --2- 172.21.15.43:0/298637568 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fe6f4159460 0x7fe6dc004d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T14:52:07.437 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:07.432+0000 7fe6f3fff640 1 -- 172.21.15.43:0/298637568 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fe6dc004300 msgr2=0x7fe6dc0047c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-09T14:52:07.437 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:07.432+0000 7fe6f3fff640 1 --2- 172.21.15.43:0/298637568 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fe6dc004300 0x7fe6dc0047c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T14:52:07.437 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:07.432+0000 7fe6f3fff640 1 -- 172.21.15.43:0/298637568 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe6dc003520 con 0x7fe6dc0039a0 2023-08-09T14:52:07.438 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:07.432+0000 7fe6f37fe640 1 --2- 172.21.15.43:0/298637568 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fe6dc004300 0x7fe6dc0047c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-08-09T14:52:07.439 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:07.434+0000 7fe6f3fff640 1 --2- 172.21.15.43:0/298637568 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fe6dc0039a0 0x7fe6dc003dc0 secure :-1 s=READY pgs=709 cs=0 l=1 rev1=1 crypto rx=0x7fe6e4009ac0 tx=0x7fe6dc0057f0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=402f33ebb537cd02 server_cookie=0 in_seq=0 out_seq=0 2023-08-09T14:52:07.439 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:07.434+0000 7fe6f2ffd640 1 -- 172.21.15.43:0/298637568 <== mon.1 v2:172.21.15.203:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe6e402c6b0 con 0x7fe6dc0039a0 2023-08-09T14:52:07.439 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:07.434+0000 7fe6f2ffd640 1 -- 172.21.15.43:0/298637568 <== mon.1 v2:172.21.15.203:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe6e402c810 con 0x7fe6dc0039a0 2023-08-09T14:52:07.439 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:07.434+0000 7fe6f2ffd640 1 -- 172.21.15.43:0/298637568 <== mon.1 v2:172.21.15.203:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe6e402cba0 con 0x7fe6dc0039a0 2023-08-09T14:52:07.439 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:07.435+0000 7fe6fa32f640 1 -- 172.21.15.43:0/298637568 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fe6dc0039a0 msgr2=0x7fe6dc003dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T14:52:07.439 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:07.435+0000 7fe6fa32f640 1 --2- 172.21.15.43:0/298637568 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fe6dc0039a0 0x7fe6dc003dc0 secure :-1 s=READY pgs=709 cs=0 l=1 rev1=1 crypto rx=0x7fe6e4009ac0 tx=0x7fe6dc0057f0 comp rx=0 tx=0).stop 2023-08-09T14:52:07.439 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:07.435+0000 7fe6f88cc640 1 -- 172.21.15.43:0/298637568 reap_dead start 2023-08-09T14:52:07.440 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:07.435+0000 7fe6fa32f640 1 -- 172.21.15.43:0/298637568 shutdown_connections 2023-08-09T14:52:07.440 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:07.435+0000 7fe6fa32f640 1 -- 172.21.15.43:0/298637568 >> 172.21.15.43:0/298637568 conn(0x7fe6f40cbbd0 msgr2=0x7fe6f40cbfe0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-09T14:52:07.440 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:07.435+0000 7fe6fa32f640 1 -- 172.21.15.43:0/298637568 shutdown_connections 2023-08-09T14:52:07.440 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:07.435+0000 7fe6fa32f640 1 -- 172.21.15.43:0/298637568 wait complete. 2023-08-09T14:52:07.440 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:07.437+0000 7fe6fa32f640 1 Processor -- start 2023-08-09T14:52:07.440 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:07.437+0000 7fe6fa32f640 1 -- start start 2023-08-09T14:52:07.442 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:07.438+0000 7fe6fa32f640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fe6dc0039a0 0x7fe6f414cf50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T14:52:07.442 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:07.438+0000 7fe6fa32f640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fe6f4153bf0 0x7fe6f4154070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T14:52:07.442 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:07.438+0000 7fe6fa32f640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fe6f40c7920 0x7fe6f40c7da0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T14:52:07.442 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:07.438+0000 7fe6fa32f640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe6f41545b0 con 0x7fe6f40c7920 2023-08-09T14:52:07.442 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:07.438+0000 7fe6fa32f640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe6f4151490 con 0x7fe6f4153bf0 2023-08-09T14:52:07.442 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:07.438+0000 7fe6fa32f640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe6f40c8610 con 0x7fe6dc0039a0 2023-08-09T14:52:07.443 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:07.438+0000 7fe6f88cc640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fe6f40c7920 0x7fe6f40c7da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T14:52:07.443 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:07.439+0000 7fe6f88cc640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fe6f40c7920 0x7fe6f40c7da0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:34846/0 (socket says 172.21.15.43:34846) 2023-08-09T14:52:07.443 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:07.439+0000 7fe6f88cc640 1 -- 172.21.15.43:0/2121141005 learned_addr learned my addr 172.21.15.43:0/2121141005 (peer_addr_for_me v2:172.21.15.43:0/0) 2023-08-09T14:52:07.443 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:07.439+0000 7fe6f37fe640 1 --2- 172.21.15.43:0/2121141005 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fe6f4153bf0 0x7fe6f4154070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T14:52:07.443 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:07.439+0000 7fe6f3fff640 1 --2- 172.21.15.43:0/2121141005 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fe6dc0039a0 0x7fe6f414cf50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T14:52:07.443 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:07.439+0000 7fe6f88cc640 1 -- 172.21.15.43:0/2121141005 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fe6dc0039a0 msgr2=0x7fe6f414cf50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T14:52:07.443 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:07.439+0000 7fe6f88cc640 1 --2- 172.21.15.43:0/2121141005 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fe6dc0039a0 0x7fe6f414cf50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T14:52:07.444 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:07.439+0000 7fe6f88cc640 1 -- 172.21.15.43:0/2121141005 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fe6f4153bf0 msgr2=0x7fe6f4154070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T14:52:07.444 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:07.439+0000 7fe6f88cc640 1 --2- 172.21.15.43:0/2121141005 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fe6f4153bf0 0x7fe6f4154070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T14:52:07.444 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:07.439+0000 7fe6f88cc640 1 -- 172.21.15.43:0/2121141005 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe6f40c88f0 con 0x7fe6f40c7920 2023-08-09T14:52:07.444 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:07.439+0000 7fe6f88cc640 1 --2- 172.21.15.43:0/2121141005 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fe6f40c7920 0x7fe6f40c7da0 secure :-1 s=READY pgs=1928 cs=0 l=1 rev1=1 crypto rx=0x7fe6dc005770 tx=0x7fe6ec011890 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-09T14:52:07.444 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:07.440+0000 7fe6f0ff9640 1 -- 172.21.15.43:0/2121141005 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe6ec00e070 con 0x7fe6f40c7920 2023-08-09T14:52:07.444 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:07.440+0000 7fe6f0ff9640 1 -- 172.21.15.43:0/2121141005 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe6ec003990 con 0x7fe6f40c7920 2023-08-09T14:52:07.444 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:07.440+0000 7fe6f0ff9640 1 -- 172.21.15.43:0/2121141005 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe6ec003b40 con 0x7fe6f40c7920 2023-08-09T14:52:07.444 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:07.440+0000 7fe6fa32f640 1 -- 172.21.15.43:0/2121141005 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe6f41516e0 con 0x7fe6f40c7920 2023-08-09T14:52:07.445 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:07.440+0000 7fe6fa32f640 1 -- 172.21.15.43:0/2121141005 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe6f40c3520 con 0x7fe6f40c7920 2023-08-09T14:52:07.445 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:07.441+0000 7fe6fa32f640 1 -- 172.21.15.43:0/2121141005 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe6f40c2af0 con 0x7fe6f40c7920 2023-08-09T14:52:07.446 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:07.442+0000 7fe6f0ff9640 1 -- 172.21.15.43:0/2121141005 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 84) v1 ==== 88444+0+0 (secure 0 0 0) 0x7fe6ec003ca0 con 0x7fe6f40c7920 2023-08-09T14:52:07.446 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:07.443+0000 7fe6f0ff9640 1 --2- 172.21.15.43:0/2121141005 >> [v2:172.21.15.203:6832/843920641,v1:172.21.15.203:6833/843920641] conn(0x7fe6e006a170 0x7fe6e006c630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T14:52:07.446 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:07.443+0000 7fe6f0ff9640 1 -- 172.21.15.43:0/2121141005 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(173..173 src has 1..173) v4 ==== 6770+0+0 (secure 0 0 0) 0x7fe6ec089170 con 0x7fe6f40c7920 2023-08-09T14:52:07.446 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:07.443+0000 7fe6f3fff640 1 -- 172.21.15.43:0/2121141005 >> [v2:172.21.15.203:6832/843920641,v1:172.21.15.203:6833/843920641] conn(0x7fe6e006a170 msgr2=0x7fe6e006c630 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.203:6832/843920641 2023-08-09T14:52:07.446 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:07.443+0000 7fe6f3fff640 1 --2- 172.21.15.43:0/2121141005 >> [v2:172.21.15.203:6832/843920641,v1:172.21.15.203:6833/843920641] conn(0x7fe6e006a170 0x7fe6e006c630 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-08-09T14:52:07.448 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:07.447+0000 7fe6f0ff9640 1 -- 172.21.15.43:0/2121141005 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178066 (secure 0 0 0) 0x7fe6ec054360 con 0x7fe6f40c7920 2023-08-09T14:52:07.642 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:07.640+0000 7fe6fa32f640 1 -- 172.21.15.43:0/2121141005 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7fe6f4151c00 con 0x7fe6f40c7920 2023-08-09T14:52:07.642 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:07.641+0000 7fe6f0ff9640 1 -- 172.21.15.43:0/2121141005 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v173) v1 ==== 93+0+10 (secure 0 0 0) 0x7fe6f4151c00 con 0x7fe6f40c7920 2023-08-09T14:52:07.643 INFO:teuthology.orchestra.run.smithi043.stdout:pg_num: 1 2023-08-09T14:52:07.645 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:07.644+0000 7fe6fa32f640 1 -- 172.21.15.43:0/2121141005 >> [v2:172.21.15.203:6832/843920641,v1:172.21.15.203:6833/843920641] conn(0x7fe6e006a170 msgr2=0x7fe6e006c630 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-08-09T14:52:07.645 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:07.644+0000 7fe6fa32f640 1 --2- 172.21.15.43:0/2121141005 >> [v2:172.21.15.203:6832/843920641,v1:172.21.15.203:6833/843920641] conn(0x7fe6e006a170 0x7fe6e006c630 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T14:52:07.646 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:07.644+0000 7fe6fa32f640 1 -- 172.21.15.43:0/2121141005 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fe6f40c7920 msgr2=0x7fe6f40c7da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T14:52:07.646 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:07.644+0000 7fe6fa32f640 1 --2- 172.21.15.43:0/2121141005 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fe6f40c7920 0x7fe6f40c7da0 secure :-1 s=READY pgs=1928 cs=0 l=1 rev1=1 crypto rx=0x7fe6dc005770 tx=0x7fe6ec011890 comp rx=0 tx=0).stop 2023-08-09T14:52:07.646 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:07.644+0000 7fe6fa32f640 1 -- 172.21.15.43:0/2121141005 shutdown_connections 2023-08-09T14:52:07.646 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:07.644+0000 7fe6fa32f640 1 --2- 172.21.15.43:0/2121141005 >> [v2:172.21.15.203:6832/843920641,v1:172.21.15.203:6833/843920641] conn(0x7fe6e006a170 0x7fe6e006c630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T14:52:07.646 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:07.644+0000 7fe6fa32f640 1 --2- 172.21.15.43:0/2121141005 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fe6f40c7920 0x7fe6f40c7da0 unknown :-1 s=CLOSED pgs=1928 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T14:52:07.646 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:07.644+0000 7fe6fa32f640 1 --2- 172.21.15.43:0/2121141005 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fe6f4153bf0 0x7fe6f4154070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T14:52:07.646 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:07.644+0000 7fe6fa32f640 1 --2- 172.21.15.43:0/2121141005 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fe6dc0039a0 0x7fe6f414cf50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T14:52:07.647 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:07.644+0000 7fe6fa32f640 1 -- 172.21.15.43:0/2121141005 >> 172.21.15.43:0/2121141005 conn(0x7fe6f40cbbd0 msgr2=0x7fe6f40cda70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-09T14:52:07.647 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:07.644+0000 7fe6fa32f640 1 -- 172.21.15.43:0/2121141005 shutdown_connections 2023-08-09T14:52:07.647 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:07.644+0000 7fe6fa32f640 1 -- 172.21.15.43:0/2121141005 wait complete. 2023-08-09T14:52:07.664 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-08-09T14:52:07.856 INFO:tasks.ceph.mgr.x.smithi203.stderr:2023-08-09T14:52:07.856+0000 7fb798253040 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2023-08-09T14:52:07.912 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:07.910+0000 7fa9aae6a640 1 Processor -- start 2023-08-09T14:52:07.912 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:07.910+0000 7fa9aae6a640 1 -- start start 2023-08-09T14:52:07.913 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:07.912+0000 7fa9aae6a640 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7fa9a415c3a0 0x7fa9a41608a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T14:52:07.914 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:07.912+0000 7fa9aae6a640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7fa9a40b0f00 0x7fa9a40ccef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T14:52:07.914 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:07.912+0000 7fa9aae6a640 1 -- --> v1:172.21.15.43:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa9a4160de0 con 0x7fa9a40b1450 2023-08-09T14:52:07.914 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:07.912+0000 7fa9aae6a640 1 -- --> v2:172.21.15.43:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa9a40b81c0 con 0x7fa9a40b0f00 2023-08-09T14:52:07.914 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:07.912+0000 7fa9aae6a640 1 -- --> v2:172.21.15.203:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa9a40b7c00 con 0x7fa9a415c3a0 2023-08-09T14:52:07.915 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:07.912+0000 7fa9a9667640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7fa9a40b0f00 0x7fa9a40ccef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T14:52:07.915 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:07.912+0000 7fa9aa669640 1 --1- >> v1:172.21.15.43:6789/0 conn(0x7fa9a40b1450 0x7fa9a40d0580 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.43:6789/0 says I am v1:172.21.15.43:41522/0 (socket says 172.21.15.43:41522) 2023-08-09T14:52:07.915 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:07.912+0000 7fa9a9667640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7fa9a40b0f00 0x7fa9a40ccef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:34850/0 (socket says 172.21.15.43:34850) 2023-08-09T14:52:07.915 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:07.912+0000 7fa9aa669640 1 -- 172.21.15.43:0/1682138857 learned_addr learned my addr 172.21.15.43:0/1682138857 (peer_addr_for_me v1:172.21.15.43:0/0) 2023-08-09T14:52:07.916 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:07.913+0000 7fa9a9e68640 1 --2- 172.21.15.43:0/1682138857 >> v2:172.21.15.203:3300/0 conn(0x7fa9a415c3a0 0x7fa9a41608a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T14:52:07.916 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:07.913+0000 7fa9a9667640 1 -- 172.21.15.43:0/1682138857 >> v2:172.21.15.203:3300/0 conn(0x7fa9a415c3a0 msgr2=0x7fa9a41608a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-09T14:52:07.916 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:07.913+0000 7fa9a9667640 1 --2- 172.21.15.43:0/1682138857 >> v2:172.21.15.203:3300/0 conn(0x7fa9a415c3a0 0x7fa9a41608a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T14:52:07.916 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:07.913+0000 7fa9a9667640 1 -- 172.21.15.43:0/1682138857 >> v1:172.21.15.43:6789/0 conn(0x7fa9a40b1450 legacy=0x7fa9a40d0580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T14:52:07.916 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:07.913+0000 7fa9a9667640 1 -- 172.21.15.43:0/1682138857 --> v2:172.21.15.43:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa9a40b7ee0 con 0x7fa9a40b0f00 2023-08-09T14:52:07.917 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:07.914+0000 7fa9a9667640 1 --2- 172.21.15.43:0/1682138857 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa9a40b0f00 0x7fa9a40ccef0 secure :-1 s=READY pgs=1930 cs=0 l=1 rev1=1 crypto rx=0x7fa9a40cfd30 tx=0x7fa99402b840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=23c5ed7f9fbf8d59 server_cookie=0 in_seq=0 out_seq=0 2023-08-09T14:52:07.917 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:07.915+0000 7fa9a8e66640 1 -- 172.21.15.43:0/1682138857 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa99402e070 con 0x7fa9a40b0f00 2023-08-09T14:52:07.917 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:07.915+0000 7fa9a8e66640 1 -- 172.21.15.43:0/1682138857 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa9940315a0 con 0x7fa9a40b0f00 2023-08-09T14:52:07.917 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:07.915+0000 7fa9a8e66640 1 -- 172.21.15.43:0/1682138857 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa994031930 con 0x7fa9a40b0f00 2023-08-09T14:52:07.918 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:07.915+0000 7fa9aae6a640 1 -- 172.21.15.43:0/1682138857 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa9a40b0f00 msgr2=0x7fa9a40ccef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T14:52:07.918 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:07.915+0000 7fa9aae6a640 1 --2- 172.21.15.43:0/1682138857 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa9a40b0f00 0x7fa9a40ccef0 secure :-1 s=READY pgs=1930 cs=0 l=1 rev1=1 crypto rx=0x7fa9a40cfd30 tx=0x7fa99402b840 comp rx=0 tx=0).stop 2023-08-09T14:52:07.918 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:07.915+0000 7fa9aae6a640 1 -- 172.21.15.43:0/1682138857 shutdown_connections 2023-08-09T14:52:07.918 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:07.915+0000 7fa9aae6a640 1 --2- 172.21.15.43:0/1682138857 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa9a40b0f00 0x7fa9a40ccef0 unknown :-1 s=CLOSED pgs=1930 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T14:52:07.919 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:07.915+0000 7fa9aae6a640 1 --2- 172.21.15.43:0/1682138857 >> v2:172.21.15.203:3300/0 conn(0x7fa9a415c3a0 0x7fa9a41608a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T14:52:07.919 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:07.916+0000 7fa9aae6a640 1 -- 172.21.15.43:0/1682138857 >> 172.21.15.43:0/1682138857 conn(0x7fa9a406a210 msgr2=0x7fa9a406a620 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-09T14:52:07.919 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:07.916+0000 7fa9aae6a640 1 -- 172.21.15.43:0/1682138857 shutdown_connections 2023-08-09T14:52:07.919 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:07.916+0000 7fa9aae6a640 1 -- 172.21.15.43:0/1682138857 wait complete. 2023-08-09T14:52:07.920 INFO:tasks.ceph.mgr.x.smithi203.stderr:2023-08-09T14:52:07.916+0000 7fb798253040 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2023-08-09T14:52:07.921 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:07.917+0000 7fa9aae6a640 1 Processor -- start 2023-08-09T14:52:07.921 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:07.918+0000 7fa9aae6a640 1 -- start start 2023-08-09T14:52:07.921 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:07.919+0000 7fa9aae6a640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fa9a40b0f00 0x7fa9a41547b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T14:52:07.921 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:07.919+0000 7fa9aae6a640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fa9a40b1450 0x7fa9a4150630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T14:52:07.921 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:07.919+0000 7fa9aae6a640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa9a415c3a0 0x7fa9a40c7010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T14:52:07.922 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:07.919+0000 7fa9aae6a640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa9a40b3810 con 0x7fa9a415c3a0 2023-08-09T14:52:07.922 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:07.919+0000 7fa9aae6a640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa9a41f5600 con 0x7fa9a40b0f00 2023-08-09T14:52:07.922 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:07.919+0000 7fa9aae6a640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa9a40b27d0 con 0x7fa9a40b1450 2023-08-09T14:52:07.922 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:07.919+0000 7fa9aa669640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa9a415c3a0 0x7fa9a40c7010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T14:52:07.922 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:07.919+0000 7fa9aa669640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa9a415c3a0 0x7fa9a40c7010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:34852/0 (socket says 172.21.15.43:34852) 2023-08-09T14:52:07.922 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:07.919+0000 7fa9aa669640 1 -- 172.21.15.43:0/3144381431 learned_addr learned my addr 172.21.15.43:0/3144381431 (peer_addr_for_me v2:172.21.15.43:0/0) 2023-08-09T14:52:07.922 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:07.919+0000 7fa9a9e68640 1 --2- 172.21.15.43:0/3144381431 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fa9a40b0f00 0x7fa9a41547b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T14:52:07.922 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:07.919+0000 7fa9a9667640 1 --2- 172.21.15.43:0/3144381431 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fa9a40b1450 0x7fa9a4150630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T14:52:07.923 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:07.919+0000 7fa9aa669640 1 -- 172.21.15.43:0/3144381431 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fa9a40b1450 msgr2=0x7fa9a4150630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T14:52:07.923 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:07.919+0000 7fa9aa669640 1 --2- 172.21.15.43:0/3144381431 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fa9a40b1450 0x7fa9a4150630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T14:52:07.923 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:07.919+0000 7fa9aa669640 1 -- 172.21.15.43:0/3144381431 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fa9a40b0f00 msgr2=0x7fa9a41547b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T14:52:07.923 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:07.919+0000 7fa9aa669640 1 --2- 172.21.15.43:0/3144381431 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fa9a40b0f00 0x7fa9a41547b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T14:52:07.923 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:07.919+0000 7fa9aa669640 1 -- 172.21.15.43:0/3144381431 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa9940093a0 con 0x7fa9a415c3a0 2023-08-09T14:52:07.924 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:07.920+0000 7fa9aa669640 1 --2- 172.21.15.43:0/3144381431 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa9a415c3a0 0x7fa9a40c7010 secure :-1 s=READY pgs=1931 cs=0 l=1 rev1=1 crypto rx=0x7fa99c00cab0 tx=0x7fa99c00cf70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-09T14:52:07.925 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:07.920+0000 7fa99a7fc640 1 -- 172.21.15.43:0/3144381431 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa99c00ac30 con 0x7fa9a415c3a0 2023-08-09T14:52:07.925 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:07.920+0000 7fa99a7fc640 1 -- 172.21.15.43:0/3144381431 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa99c00ad90 con 0x7fa9a415c3a0 2023-08-09T14:52:07.925 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:07.920+0000 7fa99a7fc640 1 -- 172.21.15.43:0/3144381431 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa99c012600 con 0x7fa9a415c3a0 2023-08-09T14:52:07.925 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:07.920+0000 7fa9aae6a640 1 -- 172.21.15.43:0/3144381431 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa9a40b3150 con 0x7fa9a415c3a0 2023-08-09T14:52:07.925 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:07.920+0000 7fa9aae6a640 1 -- 172.21.15.43:0/3144381431 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa9a40b2ab0 con 0x7fa9a415c3a0 2023-08-09T14:52:07.925 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:07.921+0000 7fa9aae6a640 1 -- 172.21.15.43:0/3144381431 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa9a40b3950 con 0x7fa9a415c3a0 2023-08-09T14:52:07.925 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:07.922+0000 7fa99a7fc640 1 -- 172.21.15.43:0/3144381431 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 84) v1 ==== 88444+0+0 (secure 0 0 0) 0x7fa99c012760 con 0x7fa9a415c3a0 2023-08-09T14:52:07.926 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:07.923+0000 7fa99a7fc640 1 --2- 172.21.15.43:0/3144381431 >> [v2:172.21.15.203:6832/843920641,v1:172.21.15.203:6833/843920641] conn(0x7fa98806a1a0 0x7fa98806c660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T14:52:07.926 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:07.923+0000 7fa99a7fc640 1 -- 172.21.15.43:0/3144381431 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(173..173 src has 1..173) v4 ==== 6770+0+0 (secure 0 0 0) 0x7fa99c088760 con 0x7fa9a415c3a0 2023-08-09T14:52:07.926 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:07.923+0000 7fa9a9e68640 1 -- 172.21.15.43:0/3144381431 >> [v2:172.21.15.203:6832/843920641,v1:172.21.15.203:6833/843920641] conn(0x7fa98806a1a0 msgr2=0x7fa98806c660 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.203:6832/843920641 2023-08-09T14:52:07.926 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:07.924+0000 7fa9a9e68640 1 --2- 172.21.15.43:0/3144381431 >> [v2:172.21.15.203:6832/843920641,v1:172.21.15.203:6833/843920641] conn(0x7fa98806a1a0 0x7fa98806c660 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-08-09T14:52:07.929 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:07.928+0000 7fa99a7fc640 1 -- 172.21.15.43:0/3144381431 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178066 (secure 0 0 0) 0x7fa9a40b3950 con 0x7fa9a415c3a0 2023-08-09T14:52:07.967 INFO:tasks.ceph.mgr.x.smithi203.stderr:2023-08-09T14:52:07.966+0000 7fb798253040 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2023-08-09T14:52:07.977 INFO:tasks.ceph.mgr.y.smithi043.stderr:2023-08-09T14:52:07.976+0000 7f35b420d040 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2023-08-09T14:52:08.043 INFO:tasks.ceph.mgr.y.smithi043.stderr:2023-08-09T14:52:08.042+0000 7f35b420d040 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2023-08-09T14:52:08.085 INFO:tasks.ceph.mgr.x.smithi203.stderr:2023-08-09T14:52:08.084+0000 7fb798253040 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2023-08-09T14:52:08.095 INFO:tasks.ceph.mgr.y.smithi043.stderr:2023-08-09T14:52:08.093+0000 7f35b420d040 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2023-08-09T14:52:08.122 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:08.120+0000 7fa9aae6a640 1 -- 172.21.15.43:0/3144381431 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7fa9a40b3950 con 0x7fa9a415c3a0 2023-08-09T14:52:08.122 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:08.121+0000 7fa99a7fc640 1 -- 172.21.15.43:0/3144381431 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v173) v1 ==== 104+0+11 (secure 0 0 0) 0x7fa9a40b3950 con 0x7fa9a415c3a0 2023-08-09T14:52:08.123 INFO:teuthology.orchestra.run.smithi043.stdout:pg_num: 32 2023-08-09T14:52:08.125 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:08.123+0000 7fa9aae6a640 1 -- 172.21.15.43:0/3144381431 >> [v2:172.21.15.203:6832/843920641,v1:172.21.15.203:6833/843920641] conn(0x7fa98806a1a0 msgr2=0x7fa98806c660 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-08-09T14:52:08.125 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:08.123+0000 7fa9aae6a640 1 --2- 172.21.15.43:0/3144381431 >> [v2:172.21.15.203:6832/843920641,v1:172.21.15.203:6833/843920641] conn(0x7fa98806a1a0 0x7fa98806c660 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T14:52:08.125 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:08.123+0000 7fa9aae6a640 1 -- 172.21.15.43:0/3144381431 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa9a415c3a0 msgr2=0x7fa9a40c7010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T14:52:08.126 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:08.123+0000 7fa9aae6a640 1 --2- 172.21.15.43:0/3144381431 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa9a415c3a0 0x7fa9a40c7010 secure :-1 s=READY pgs=1931 cs=0 l=1 rev1=1 crypto rx=0x7fa99c00cab0 tx=0x7fa99c00cf70 comp rx=0 tx=0).stop 2023-08-09T14:52:08.126 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:08.123+0000 7fa9aae6a640 1 -- 172.21.15.43:0/3144381431 shutdown_connections 2023-08-09T14:52:08.126 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:08.123+0000 7fa9aae6a640 1 --2- 172.21.15.43:0/3144381431 >> [v2:172.21.15.203:6832/843920641,v1:172.21.15.203:6833/843920641] conn(0x7fa98806a1a0 0x7fa98806c660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T14:52:08.126 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:08.123+0000 7fa9aae6a640 1 --2- 172.21.15.43:0/3144381431 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa9a415c3a0 0x7fa9a40c7010 unknown :-1 s=CLOSED pgs=1931 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T14:52:08.126 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:08.123+0000 7fa9aae6a640 1 --2- 172.21.15.43:0/3144381431 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fa9a40b1450 0x7fa9a4150630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T14:52:08.126 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:08.123+0000 7fa9aae6a640 1 --2- 172.21.15.43:0/3144381431 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fa9a40b0f00 0x7fa9a41547b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T14:52:08.126 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:08.124+0000 7fa9aae6a640 1 -- 172.21.15.43:0/3144381431 >> 172.21.15.43:0/3144381431 conn(0x7fa9a406a210 msgr2=0x7fa9a40cd270 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-09T14:52:08.126 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:08.124+0000 7fa9aae6a640 1 -- 172.21.15.43:0/3144381431 shutdown_connections 2023-08-09T14:52:08.127 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:08.124+0000 7fa9aae6a640 1 -- 172.21.15.43:0/3144381431 wait complete. 2023-08-09T14:52:08.142 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-08-09T14:52:08.223 INFO:tasks.ceph.mgr.y.smithi043.stderr:2023-08-09T14:52:08.221+0000 7f35b420d040 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2023-08-09T14:52:08.241 INFO:tasks.ceph.mgr.x.smithi203.stderr:2023-08-09T14:52:08.240+0000 7fb798253040 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2023-08-09T14:52:08.347 INFO:tasks.ceph.mgr.x.smithi203.stderr:2023-08-09T14:52:08.347+0000 7fb798253040 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2023-08-09T14:52:08.353 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:08.351+0000 7fb6006e1640 1 Processor -- start 2023-08-09T14:52:08.353 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:08.351+0000 7fb6006e1640 1 -- start start 2023-08-09T14:52:08.354 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:08.353+0000 7fb6006e1640 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7fb5f815c940 0x7fb5f806ab70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T14:52:08.355 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:08.353+0000 7fb6006e1640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7fb5f80ca810 0x7fb5f80caca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T14:52:08.355 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:08.353+0000 7fb6006e1640 1 -- --> v1:172.21.15.203:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb5f80cefd0 con 0x7fb5f80b0de0 2023-08-09T14:52:08.355 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:08.353+0000 7fb6006e1640 1 -- --> v2:172.21.15.43:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb5f80b1660 con 0x7fb5f80ca810 2023-08-09T14:52:08.355 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:08.353+0000 7fb6006e1640 1 -- --> v2:172.21.15.203:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb5f80cf580 con 0x7fb5f815c940 2023-08-09T14:52:08.356 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:08.353+0000 7fb5fec7e640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7fb5f80ca810 0x7fb5f80caca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T14:52:08.356 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:08.353+0000 7fb5fec7e640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7fb5f80ca810 0x7fb5f80caca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:34868/0 (socket says 172.21.15.43:34868) 2023-08-09T14:52:08.356 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:08.353+0000 7fb5fec7e640 1 -- 172.21.15.43:0/1472607710 learned_addr learned my addr 172.21.15.43:0/1472607710 (peer_addr_for_me v2:172.21.15.43:0/0) 2023-08-09T14:52:08.356 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:08.354+0000 7fb5fe47d640 1 --2- 172.21.15.43:0/1472607710 >> v2:172.21.15.203:3300/0 conn(0x7fb5f815c940 0x7fb5f806ab70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T14:52:08.356 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:08.354+0000 7fb5fec7e640 1 -- 172.21.15.43:0/1472607710 >> v2:172.21.15.203:3300/0 conn(0x7fb5f815c940 msgr2=0x7fb5f806ab70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-09T14:52:08.357 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:08.354+0000 7fb5fec7e640 1 --2- 172.21.15.43:0/1472607710 >> v2:172.21.15.203:3300/0 conn(0x7fb5f815c940 0x7fb5f806ab70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T14:52:08.357 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:08.354+0000 7fb5fec7e640 1 -- 172.21.15.43:0/1472607710 >> v1:172.21.15.203:6789/0 conn(0x7fb5f80b0de0 legacy=0x7fb5f806b0b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-09T14:52:08.357 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:08.354+0000 7fb5fec7e640 1 -- 172.21.15.43:0/1472607710 --> v2:172.21.15.43:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb5f815cd20 con 0x7fb5f80ca810 2023-08-09T14:52:08.358 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:08.355+0000 7fb5fec7e640 1 --2- 172.21.15.43:0/1472607710 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb5f80ca810 0x7fb5f80caca0 secure :-1 s=READY pgs=1932 cs=0 l=1 rev1=1 crypto rx=0x7fb5f806d2c0 tx=0x7fb5f002b930 comp rx=0 tx=0).ready entity=mon.0 client_cookie=22b3d97d10487c07 server_cookie=0 in_seq=0 out_seq=0 2023-08-09T14:52:08.358 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:08.355+0000 7fb5fd47b640 1 -- 172.21.15.43:0/1472607710 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb5f0031890 con 0x7fb5f80ca810 2023-08-09T14:52:08.358 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:08.355+0000 7fb5fd47b640 1 -- 172.21.15.43:0/1472607710 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb5f00319f0 con 0x7fb5f80ca810 2023-08-09T14:52:08.358 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:08.355+0000 7fb5fd47b640 1 -- 172.21.15.43:0/1472607710 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb5f0031d80 con 0x7fb5f80ca810 2023-08-09T14:52:08.359 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:08.356+0000 7fb6006e1640 1 -- 172.21.15.43:0/1472607710 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb5f80ca810 msgr2=0x7fb5f80caca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T14:52:08.359 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:08.356+0000 7fb6006e1640 1 --2- 172.21.15.43:0/1472607710 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb5f80ca810 0x7fb5f80caca0 secure :-1 s=READY pgs=1932 cs=0 l=1 rev1=1 crypto rx=0x7fb5f806d2c0 tx=0x7fb5f002b930 comp rx=0 tx=0).stop 2023-08-09T14:52:08.359 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:08.356+0000 7fb6006e1640 1 -- 172.21.15.43:0/1472607710 shutdown_connections 2023-08-09T14:52:08.359 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:08.356+0000 7fb6006e1640 1 --2- 172.21.15.43:0/1472607710 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb5f80ca810 0x7fb5f80caca0 unknown :-1 s=CLOSED pgs=1932 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T14:52:08.360 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:08.356+0000 7fb6006e1640 1 --2- 172.21.15.43:0/1472607710 >> v2:172.21.15.203:3300/0 conn(0x7fb5f815c940 0x7fb5f806ab70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T14:52:08.360 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:08.356+0000 7fb6006e1640 1 -- 172.21.15.43:0/1472607710 >> 172.21.15.43:0/1472607710 conn(0x7fb5f8089fe0 msgr2=0x7fb5f80d01c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-09T14:52:08.360 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:08.356+0000 7fb6006e1640 1 -- 172.21.15.43:0/1472607710 shutdown_connections 2023-08-09T14:52:08.360 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:08.356+0000 7fb6006e1640 1 -- 172.21.15.43:0/1472607710 wait complete. 2023-08-09T14:52:08.361 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:08.359+0000 7fb6006e1640 1 Processor -- start 2023-08-09T14:52:08.361 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:08.359+0000 7fb6006e1640 1 -- start start 2023-08-09T14:52:08.362 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:08.361+0000 7fb6006e1640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fb5f80b0de0 0x7fb5f806e960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T14:52:08.363 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:08.361+0000 7fb6006e1640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fb5f80ca810 0x7fb5f8147980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T14:52:08.363 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:08.361+0000 7fb6006e1640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb5f815c940 0x7fb5f80cc020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T14:52:08.363 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:08.361+0000 7fb6006e1640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb5f8156120 con 0x7fb5f815c940 2023-08-09T14:52:08.364 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:08.361+0000 7fb6006e1640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb5f8156400 con 0x7fb5f80b0de0 2023-08-09T14:52:08.364 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:08.361+0000 7fb6006e1640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb5f81566e0 con 0x7fb5f80ca810 2023-08-09T14:52:08.364 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:08.361+0000 7fb5fec7e640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb5f815c940 0x7fb5f80cc020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T14:52:08.364 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:08.361+0000 7fb5fec7e640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb5f815c940 0x7fb5f80cc020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:34882/0 (socket says 172.21.15.43:34882) 2023-08-09T14:52:08.365 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:08.361+0000 7fb5fec7e640 1 -- 172.21.15.43:0/1439306711 learned_addr learned my addr 172.21.15.43:0/1439306711 (peer_addr_for_me v2:172.21.15.43:0/0) 2023-08-09T14:52:08.367 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:08.362+0000 7fb5fe47d640 1 --2- 172.21.15.43:0/1439306711 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fb5f80b0de0 0x7fb5f806e960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T14:52:08.367 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:08.362+0000 7fb5fdc7c640 1 --2- 172.21.15.43:0/1439306711 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fb5f80ca810 0x7fb5f8147980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T14:52:08.368 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:08.362+0000 7fb5fec7e640 1 -- 172.21.15.43:0/1439306711 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fb5f80ca810 msgr2=0x7fb5f8147980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T14:52:08.368 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:08.362+0000 7fb5fec7e640 1 --2- 172.21.15.43:0/1439306711 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fb5f80ca810 0x7fb5f8147980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T14:52:08.368 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:08.362+0000 7fb5fec7e640 1 -- 172.21.15.43:0/1439306711 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fb5f80b0de0 msgr2=0x7fb5f806e960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T14:52:08.368 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:08.362+0000 7fb5fec7e640 1 --2- 172.21.15.43:0/1439306711 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fb5f80b0de0 0x7fb5f806e960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T14:52:08.370 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:08.362+0000 7fb5fec7e640 1 -- 172.21.15.43:0/1439306711 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb5f00094d0 con 0x7fb5f815c940 2023-08-09T14:52:08.370 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:08.362+0000 7fb5fec7e640 1 --2- 172.21.15.43:0/1439306711 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb5f815c940 0x7fb5f80cc020 secure :-1 s=READY pgs=1933 cs=0 l=1 rev1=1 crypto rx=0x7fb5f00090b0 tx=0x7fb5f00046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-09T14:52:08.370 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:08.362+0000 7fb5eeffd640 1 -- 172.21.15.43:0/1439306711 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb5f003ac60 con 0x7fb5f815c940 2023-08-09T14:52:08.370 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:08.363+0000 7fb5eeffd640 1 -- 172.21.15.43:0/1439306711 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb5f003adc0 con 0x7fb5f815c940 2023-08-09T14:52:08.370 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:08.363+0000 7fb5eeffd640 1 -- 172.21.15.43:0/1439306711 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb5f0044630 con 0x7fb5f815c940 2023-08-09T14:52:08.370 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:08.363+0000 7fb6006e1640 1 -- 172.21.15.43:0/1439306711 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb5f8148400 con 0x7fb5f815c940 2023-08-09T14:52:08.371 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:08.363+0000 7fb6006e1640 1 -- 172.21.15.43:0/1439306711 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb5f806eea0 con 0x7fb5f815c940 2023-08-09T14:52:08.371 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:08.364+0000 7fb6006e1640 1 -- 172.21.15.43:0/1439306711 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb5f80b1ab0 con 0x7fb5f815c940 2023-08-09T14:52:08.371 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:08.365+0000 7fb5eeffd640 1 -- 172.21.15.43:0/1439306711 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 84) v1 ==== 88444+0+0 (secure 0 0 0) 0x7fb5f004d450 con 0x7fb5f815c940 2023-08-09T14:52:08.372 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:08.366+0000 7fb5eeffd640 1 --2- 172.21.15.43:0/1439306711 >> [v2:172.21.15.203:6832/843920641,v1:172.21.15.203:6833/843920641] conn(0x7fb5dc06a170 0x7fb5dc06c630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T14:52:08.372 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:08.366+0000 7fb5eeffd640 1 -- 172.21.15.43:0/1439306711 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(173..173 src has 1..173) v4 ==== 6770+0+0 (secure 0 0 0) 0x7fb5f00ab090 con 0x7fb5f815c940 2023-08-09T14:52:08.372 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:08.366+0000 7fb5fe47d640 1 -- 172.21.15.43:0/1439306711 >> [v2:172.21.15.203:6832/843920641,v1:172.21.15.203:6833/843920641] conn(0x7fb5dc06a170 msgr2=0x7fb5dc06c630 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.203:6832/843920641 2023-08-09T14:52:08.372 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:08.366+0000 7fb5fe47d640 1 --2- 172.21.15.43:0/1439306711 >> [v2:172.21.15.203:6832/843920641,v1:172.21.15.203:6833/843920641] conn(0x7fb5dc06a170 0x7fb5dc06c630 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-08-09T14:52:08.372 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:08.370+0000 7fb5eeffd640 1 -- 172.21.15.43:0/1439306711 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178066 (secure 0 0 0) 0x7fb5f00761f0 con 0x7fb5f815c940 2023-08-09T14:52:08.394 INFO:tasks.ceph.mgr.y.smithi043.stderr:2023-08-09T14:52:08.393+0000 7f35b420d040 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2023-08-09T14:52:08.397 INFO:tasks.ceph.mgr.x.smithi203.stderr:2023-08-09T14:52:08.397+0000 7fb798253040 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2023-08-09T14:52:08.508 INFO:tasks.ceph.mgr.y.smithi043.stderr:2023-08-09T14:52:08.507+0000 7f35b420d040 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2023-08-09T14:52:08.555 INFO:tasks.ceph.mgr.x.smithi203.stderr:2023-08-09T14:52:08.555+0000 7fb798253040 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2023-08-09T14:52:08.560 INFO:tasks.ceph.mgr.y.smithi043.stderr:2023-08-09T14:52:08.558+0000 7f35b420d040 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2023-08-09T14:52:08.562 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:08.560+0000 7fb6006e1640 1 -- 172.21.15.43:0/1439306711 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7fb5f80cefd0 con 0x7fb5f815c940 2023-08-09T14:52:08.562 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:08.561+0000 7fb5eeffd640 1 -- 172.21.15.43:0/1439306711 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v173) v1 ==== 100+0+11 (secure 0 0 0) 0x7fb5f80cefd0 con 0x7fb5f815c940 2023-08-09T14:52:08.563 INFO:teuthology.orchestra.run.smithi043.stdout:pg_num: 64 2023-08-09T14:52:08.565 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:08.563+0000 7fb6006e1640 1 -- 172.21.15.43:0/1439306711 >> [v2:172.21.15.203:6832/843920641,v1:172.21.15.203:6833/843920641] conn(0x7fb5dc06a170 msgr2=0x7fb5dc06c630 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-08-09T14:52:08.565 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:08.563+0000 7fb6006e1640 1 --2- 172.21.15.43:0/1439306711 >> [v2:172.21.15.203:6832/843920641,v1:172.21.15.203:6833/843920641] conn(0x7fb5dc06a170 0x7fb5dc06c630 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T14:52:08.565 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:08.563+0000 7fb6006e1640 1 -- 172.21.15.43:0/1439306711 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb5f815c940 msgr2=0x7fb5f80cc020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T14:52:08.565 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:08.563+0000 7fb6006e1640 1 --2- 172.21.15.43:0/1439306711 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb5f815c940 0x7fb5f80cc020 secure :-1 s=READY pgs=1933 cs=0 l=1 rev1=1 crypto rx=0x7fb5f00090b0 tx=0x7fb5f00046c0 comp rx=0 tx=0).stop 2023-08-09T14:52:08.566 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:08.564+0000 7fb6006e1640 1 -- 172.21.15.43:0/1439306711 shutdown_connections 2023-08-09T14:52:08.566 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:08.564+0000 7fb6006e1640 1 --2- 172.21.15.43:0/1439306711 >> [v2:172.21.15.203:6832/843920641,v1:172.21.15.203:6833/843920641] conn(0x7fb5dc06a170 0x7fb5dc06c630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T14:52:08.566 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:08.564+0000 7fb6006e1640 1 --2- 172.21.15.43:0/1439306711 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb5f815c940 0x7fb5f80cc020 unknown :-1 s=CLOSED pgs=1933 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T14:52:08.566 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:08.564+0000 7fb6006e1640 1 --2- 172.21.15.43:0/1439306711 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fb5f80ca810 0x7fb5f8147980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T14:52:08.566 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:08.564+0000 7fb6006e1640 1 --2- 172.21.15.43:0/1439306711 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fb5f80b0de0 0x7fb5f806e960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T14:52:08.567 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:08.564+0000 7fb6006e1640 1 -- 172.21.15.43:0/1439306711 >> 172.21.15.43:0/1439306711 conn(0x7fb5f8089fe0 msgr2=0x7fb5f80b1c40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-09T14:52:08.567 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:08.564+0000 7fb6006e1640 1 -- 172.21.15.43:0/1439306711 shutdown_connections 2023-08-09T14:52:08.567 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:08.564+0000 7fb6006e1640 1 -- 172.21.15.43:0/1439306711 wait complete. 2023-08-09T14:52:08.582 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2023-08-09T14:52:08.734 INFO:tasks.ceph.mgr.y.smithi043.stderr:2023-08-09T14:52:08.732+0000 7f35b420d040 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2023-08-09T14:52:08.791 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:08.790+0000 7f5e4135b640 1 Processor -- start 2023-08-09T14:52:08.791 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:08.790+0000 7f5e4135b640 1 -- start start 2023-08-09T14:52:08.793 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:08.791+0000 7f5e4135b640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f5e3c143c20 0x7f5e3c06f540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T14:52:08.793 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:08.792+0000 7f5e4135b640 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7f5e3c0c86f0 0x7f5e3c0c8b50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T14:52:08.794 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:08.792+0000 7f5e4135b640 1 -- --> v1:172.21.15.203:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5e3c0c9090 con 0x7f5e3c06c8d0 2023-08-09T14:52:08.794 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:08.792+0000 7f5e4135b640 1 -- --> v2:172.21.15.43:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5e3c144000 con 0x7f5e3c143c20 2023-08-09T14:52:08.794 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:08.792+0000 7f5e4135b640 1 -- --> v2:172.21.15.203:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5e3c06c440 con 0x7f5e3c0c86f0 2023-08-09T14:52:08.794 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:08.792+0000 7f5e3affd640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f5e3c143c20 0x7f5e3c06f540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T14:52:08.794 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:08.792+0000 7f5e3affd640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f5e3c143c20 0x7f5e3c06f540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:34892/0 (socket says 172.21.15.43:34892) 2023-08-09T14:52:08.794 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:08.792+0000 7f5e3affd640 1 -- 172.21.15.43:0/1796431738 learned_addr learned my addr 172.21.15.43:0/1796431738 (peer_addr_for_me v2:172.21.15.43:0/0) 2023-08-09T14:52:08.795 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:08.792+0000 7f5e3affd640 1 -- 172.21.15.43:0/1796431738 >> v2:172.21.15.203:3300/0 conn(0x7f5e3c0c86f0 msgr2=0x7f5e3c0c8b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-09T14:52:08.795 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:08.792+0000 7f5e3affd640 1 --2- 172.21.15.43:0/1796431738 >> v2:172.21.15.203:3300/0 conn(0x7f5e3c0c86f0 0x7f5e3c0c8b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T14:52:08.796 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:08.792+0000 7f5e3affd640 1 -- 172.21.15.43:0/1796431738 >> v1:172.21.15.203:6789/0 conn(0x7f5e3c06c8d0 legacy=0x7f5e3c06cd00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-09T14:52:08.796 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:08.792+0000 7f5e3affd640 1 -- 172.21.15.43:0/1796431738 --> v2:172.21.15.43:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5e3c06b830 con 0x7f5e3c143c20 2023-08-09T14:52:08.796 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:08.793+0000 7f5e3affd640 1 --2- 172.21.15.43:0/1796431738 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f5e3c143c20 0x7f5e3c06f540 secure :-1 s=READY pgs=1934 cs=0 l=1 rev1=1 crypto rx=0x7f5e3c0c9b70 tx=0x7f5e2c02b7f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2a330b7c51a3fe6 server_cookie=0 in_seq=0 out_seq=0 2023-08-09T14:52:08.796 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:08.794+0000 7f5e39ffb640 1 -- 172.21.15.43:0/1796431738 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5e2c031750 con 0x7f5e3c143c20 2023-08-09T14:52:08.796 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:08.794+0000 7f5e39ffb640 1 -- 172.21.15.43:0/1796431738 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5e2c0318b0 con 0x7f5e3c143c20 2023-08-09T14:52:08.796 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:08.794+0000 7f5e39ffb640 1 -- 172.21.15.43:0/1796431738 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5e2c031c40 con 0x7f5e3c143c20 2023-08-09T14:52:08.797 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:08.794+0000 7f5e4135b640 1 -- 172.21.15.43:0/1796431738 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f5e3c143c20 msgr2=0x7f5e3c06f540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T14:52:08.797 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:08.794+0000 7f5e4135b640 1 --2- 172.21.15.43:0/1796431738 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f5e3c143c20 0x7f5e3c06f540 secure :-1 s=READY pgs=1934 cs=0 l=1 rev1=1 crypto rx=0x7f5e3c0c9b70 tx=0x7f5e2c02b7f0 comp rx=0 tx=0).stop 2023-08-09T14:52:08.797 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:08.794+0000 7f5e4135b640 1 -- 172.21.15.43:0/1796431738 shutdown_connections 2023-08-09T14:52:08.797 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:08.794+0000 7f5e4135b640 1 --2- 172.21.15.43:0/1796431738 >> v2:172.21.15.203:3300/0 conn(0x7f5e3c0c86f0 0x7f5e3c0c8b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T14:52:08.798 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:08.794+0000 7f5e4135b640 1 --2- 172.21.15.43:0/1796431738 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f5e3c143c20 0x7f5e3c06f540 unknown :-1 s=CLOSED pgs=1934 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T14:52:08.798 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:08.794+0000 7f5e4135b640 1 -- 172.21.15.43:0/1796431738 >> 172.21.15.43:0/1796431738 conn(0x7f5e3c0ce0a0 msgr2=0x7f5e3c0ce480 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-09T14:52:08.798 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:08.795+0000 7f5e4135b640 1 -- 172.21.15.43:0/1796431738 shutdown_connections 2023-08-09T14:52:08.798 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:08.795+0000 7f5e4135b640 1 -- 172.21.15.43:0/1796431738 wait complete. 2023-08-09T14:52:08.799 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:08.796+0000 7f5e4135b640 1 Processor -- start 2023-08-09T14:52:08.799 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:08.796+0000 7f5e4135b640 1 -- start start 2023-08-09T14:52:08.799 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:08.797+0000 7f5e4135b640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f5e3c06c8d0 0x7f5e3c131970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T14:52:08.799 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:08.797+0000 7f5e4135b640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f5e3c0c86f0 0x7f5e3c13e840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T14:52:08.800 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:08.797+0000 7f5e4135b640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f5e3c143c20 0x7f5e3c13ed80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T14:52:08.800 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:08.797+0000 7f5e4135b640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5e3c140510 con 0x7f5e3c0c86f0 2023-08-09T14:52:08.800 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:08.797+0000 7f5e4135b640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5e3c0c7990 con 0x7f5e3c143c20 2023-08-09T14:52:08.800 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:08.797+0000 7f5e4135b640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5e3c0c6530 con 0x7f5e3c06c8d0 2023-08-09T14:52:08.800 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:08.798+0000 7f5e3a7fc640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f5e3c0c86f0 0x7f5e3c13e840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T14:52:08.800 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:08.798+0000 7f5e3a7fc640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f5e3c0c86f0 0x7f5e3c13e840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:34902/0 (socket says 172.21.15.43:34902) 2023-08-09T14:52:08.800 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:08.798+0000 7f5e3a7fc640 1 -- 172.21.15.43:0/1407675668 learned_addr learned my addr 172.21.15.43:0/1407675668 (peer_addr_for_me v2:172.21.15.43:0/0) 2023-08-09T14:52:08.801 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:08.798+0000 7f5e3b7fe640 1 --2- 172.21.15.43:0/1407675668 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f5e3c143c20 0x7f5e3c13ed80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T14:52:08.802 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:08.798+0000 7f5e3affd640 1 --2- 172.21.15.43:0/1407675668 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f5e3c06c8d0 0x7f5e3c131970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T14:52:08.802 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:08.798+0000 7f5e3a7fc640 1 -- 172.21.15.43:0/1407675668 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f5e3c06c8d0 msgr2=0x7f5e3c131970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T14:52:08.802 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:08.798+0000 7f5e3a7fc640 1 --2- 172.21.15.43:0/1407675668 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f5e3c06c8d0 0x7f5e3c131970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T14:52:08.802 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:08.798+0000 7f5e3a7fc640 1 -- 172.21.15.43:0/1407675668 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f5e3c143c20 msgr2=0x7f5e3c13ed80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T14:52:08.802 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:08.798+0000 7f5e3a7fc640 1 --2- 172.21.15.43:0/1407675668 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f5e3c143c20 0x7f5e3c13ed80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T14:52:08.802 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:08.798+0000 7f5e3a7fc640 1 -- 172.21.15.43:0/1407675668 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5e2c0094d0 con 0x7f5e3c0c86f0 2023-08-09T14:52:08.803 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:08.799+0000 7f5e3a7fc640 1 --2- 172.21.15.43:0/1407675668 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f5e3c0c86f0 0x7f5e3c13e840 secure :-1 s=READY pgs=1935 cs=0 l=1 rev1=1 crypto rx=0x7f5e3000b410 tx=0x7f5e3000b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-09T14:52:08.803 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:08.799+0000 7f5e237fe640 1 -- 172.21.15.43:0/1407675668 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5e30011830 con 0x7f5e3c0c86f0 2023-08-09T14:52:08.803 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:08.799+0000 7f5e237fe640 1 -- 172.21.15.43:0/1407675668 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5e300119b0 con 0x7f5e3c0c86f0 2023-08-09T14:52:08.803 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:08.799+0000 7f5e237fe640 1 -- 172.21.15.43:0/1407675668 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5e30011d60 con 0x7f5e3c0c86f0 2023-08-09T14:52:08.803 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:08.799+0000 7f5e4135b640 1 -- 172.21.15.43:0/1407675668 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5e3c131eb0 con 0x7f5e3c0c86f0 2023-08-09T14:52:08.803 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:08.799+0000 7f5e4135b640 1 -- 172.21.15.43:0/1407675668 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5e3c0c7620 con 0x7f5e3c0c86f0 2023-08-09T14:52:08.803 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:08.800+0000 7f5e4135b640 1 -- 172.21.15.43:0/1407675668 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5e3c0739f0 con 0x7f5e3c0c86f0 2023-08-09T14:52:08.804 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:08.802+0000 7f5e237fe640 1 -- 172.21.15.43:0/1407675668 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 84) v1 ==== 88444+0+0 (secure 0 0 0) 0x7f5e3001a400 con 0x7f5e3c0c86f0 2023-08-09T14:52:08.805 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:08.804+0000 7f5e237fe640 1 --2- 172.21.15.43:0/1407675668 >> [v2:172.21.15.203:6832/843920641,v1:172.21.15.203:6833/843920641] conn(0x7f5e2806a170 0x7f5e2806c630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T14:52:08.806 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:08.804+0000 7f5e237fe640 1 -- 172.21.15.43:0/1407675668 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(173..173 src has 1..173) v4 ==== 6770+0+0 (secure 0 0 0) 0x7f5e30088bd0 con 0x7f5e3c0c86f0 2023-08-09T14:52:08.806 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:08.804+0000 7f5e3affd640 1 -- 172.21.15.43:0/1407675668 >> [v2:172.21.15.203:6832/843920641,v1:172.21.15.203:6833/843920641] conn(0x7f5e2806a170 msgr2=0x7f5e2806c630 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.203:6832/843920641 2023-08-09T14:52:08.806 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:08.804+0000 7f5e3affd640 1 --2- 172.21.15.43:0/1407675668 >> [v2:172.21.15.203:6832/843920641,v1:172.21.15.203:6833/843920641] conn(0x7f5e2806a170 0x7f5e2806c630 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-08-09T14:52:08.810 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:08.809+0000 7f5e237fe640 1 -- 172.21.15.43:0/1407675668 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178066 (secure 0 0 0) 0x7f5e30053e40 con 0x7f5e3c0c86f0 2023-08-09T14:52:08.941 INFO:tasks.ceph.mgr.x.smithi203.stderr:2023-08-09T14:52:08.940+0000 7fb798253040 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2023-08-09T14:52:09.007 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:09.005+0000 7f5e3affd640 1 -- 172.21.15.43:0/1407675668 >> [v2:172.21.15.203:6832/843920641,v1:172.21.15.203:6833/843920641] conn(0x7f5e2806a170 msgr2=0x7f5e2806c630 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.203:6832/843920641 2023-08-09T14:52:09.007 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:09.005+0000 7f5e3affd640 1 --2- 172.21.15.43:0/1407675668 >> [v2:172.21.15.203:6832/843920641,v1:172.21.15.203:6833/843920641] conn(0x7f5e2806a170 0x7f5e2806c630 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2023-08-09T14:52:09.016 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:09.015+0000 7f5e4135b640 1 -- 172.21.15.43:0/1407675668 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7f5e3c0cec90 con 0x7f5e3c0c86f0 2023-08-09T14:52:09.017 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:09.016+0000 7f5e237fe640 1 -- 172.21.15.43:0/1407675668 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 8 entries v173) v1 ==== 80+0+477 (secure 0 0 0) 0x7f5e30015080 con 0x7f5e3c0c86f0 2023-08-09T14:52:09.017 INFO:teuthology.orchestra.run.smithi043.stdout:172.21.15.203:0/2095212004 2023-08-10T14:52:02.909250+0000 2023-08-09T14:52:09.017 INFO:teuthology.orchestra.run.smithi043.stdout:172.21.15.203:0/3749872091 2023-08-10T14:52:02.909250+0000 2023-08-09T14:52:09.017 INFO:teuthology.orchestra.run.smithi043.stdout:172.21.15.203:0/651612122 2023-08-10T14:52:02.909250+0000 2023-08-09T14:52:09.018 INFO:teuthology.orchestra.run.smithi043.stdout:172.21.15.203:0/4120954828 2023-08-10T14:52:02.909250+0000 2023-08-09T14:52:09.018 INFO:teuthology.orchestra.run.smithi043.stdout:172.21.15.203:0/2783782601 2023-08-10T14:52:02.909250+0000 2023-08-09T14:52:09.018 INFO:teuthology.orchestra.run.smithi043.stdout:172.21.15.203:0/2232659577 2023-08-10T14:52:02.909250+0000 2023-08-09T14:52:09.018 INFO:teuthology.orchestra.run.smithi043.stdout:172.21.15.203:6833/2566816332 2023-08-10T14:52:02.909250+0000 2023-08-09T14:52:09.018 INFO:teuthology.orchestra.run.smithi043.stdout:172.21.15.203:6832/2566816332 2023-08-10T14:52:02.909250+0000 2023-08-09T14:52:09.018 INFO:teuthology.orchestra.run.smithi043.stderr:listed 8 entries 2023-08-09T14:52:09.019 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:09.018+0000 7f5e4135b640 1 -- 172.21.15.43:0/1407675668 >> [v2:172.21.15.203:6832/843920641,v1:172.21.15.203:6833/843920641] conn(0x7f5e2806a170 msgr2=0x7f5e2806c630 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-08-09T14:52:09.020 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:09.018+0000 7f5e4135b640 1 --2- 172.21.15.43:0/1407675668 >> [v2:172.21.15.203:6832/843920641,v1:172.21.15.203:6833/843920641] conn(0x7f5e2806a170 0x7f5e2806c630 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T14:52:09.020 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:09.018+0000 7f5e4135b640 1 -- 172.21.15.43:0/1407675668 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f5e3c0c86f0 msgr2=0x7f5e3c13e840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T14:52:09.020 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:09.018+0000 7f5e4135b640 1 --2- 172.21.15.43:0/1407675668 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f5e3c0c86f0 0x7f5e3c13e840 secure :-1 s=READY pgs=1935 cs=0 l=1 rev1=1 crypto rx=0x7f5e3000b410 tx=0x7f5e3000b8d0 comp rx=0 tx=0).stop 2023-08-09T14:52:09.020 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:09.018+0000 7f5e4135b640 1 -- 172.21.15.43:0/1407675668 shutdown_connections 2023-08-09T14:52:09.020 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:09.018+0000 7f5e4135b640 1 --2- 172.21.15.43:0/1407675668 >> [v2:172.21.15.203:6832/843920641,v1:172.21.15.203:6833/843920641] conn(0x7f5e2806a170 0x7f5e2806c630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T14:52:09.020 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:09.018+0000 7f5e4135b640 1 --2- 172.21.15.43:0/1407675668 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f5e3c143c20 0x7f5e3c13ed80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T14:52:09.020 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:09.018+0000 7f5e4135b640 1 --2- 172.21.15.43:0/1407675668 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f5e3c0c86f0 0x7f5e3c13e840 unknown :-1 s=CLOSED pgs=1935 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T14:52:09.021 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:09.018+0000 7f5e4135b640 1 --2- 172.21.15.43:0/1407675668 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f5e3c06c8d0 0x7f5e3c131970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T14:52:09.021 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:09.018+0000 7f5e4135b640 1 -- 172.21.15.43:0/1407675668 >> 172.21.15.43:0/1407675668 conn(0x7f5e3c0ce0a0 msgr2=0x7f5e3c0c4390 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-09T14:52:09.021 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:09.018+0000 7f5e4135b640 1 -- 172.21.15.43:0/1407675668 shutdown_connections 2023-08-09T14:52:09.021 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:09.018+0000 7f5e4135b640 1 -- 172.21.15.43:0/1407675668 wait complete. 2023-08-09T14:52:09.038 INFO:tasks.cephfs.fuse_mount:Running fusermount -u on ubuntu@smithi043.front.sepia.ceph.com... 2023-08-09T14:52:09.038 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-09T14:52:09.038 DEBUG:teuthology.orchestra.run.smithi043:> sudo fusermount -u /home/ubuntu/cephtest/mnt.0 2023-08-09T14:52:09.127 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi043.stderr:ceph-fuse[140114]: fuse finished with error 0 and tester_r 0 2023-08-09T14:52:09.132 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi043.front.sepia.ceph.com 2023-08-09T14:52:09.133 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-09T14:52:09.133 DEBUG:teuthology.orchestra.run.smithi043:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2023-08-09T14:52:09.158 INFO:tasks.ceph.mgr.y.smithi043.stderr:2023-08-09T14:52:09.157+0000 7f35b420d040 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2023-08-09T14:52:09.163 INFO:tasks.ceph.mgr.x.smithi203.stderr:2023-08-09T14:52:09.162+0000 7fb798253040 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2023-08-09T14:52:09.221 INFO:tasks.ceph.mgr.x.smithi203.stderr:2023-08-09T14:52:09.219+0000 7fb798253040 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2023-08-09T14:52:09.271 INFO:tasks.ceph.mgr.x.smithi203.stderr:2023-08-09T14:52:09.270+0000 7fb798253040 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2023-08-09T14:52:09.305 DEBUG:teuthology.orchestra.run.smithi203:> set -ex 2023-08-09T14:52:09.306 DEBUG:teuthology.orchestra.run.smithi203:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-09T14:52:09.324 INFO:tasks.ceph.mgr.x.smithi203.stderr:2023-08-09T14:52:09.323+0000 7fb798253040 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2023-08-09T14:52:09.361 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4053648,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497576k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=27,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248784k,nr_inodes=812196,mode=700,uid=1000,gid=1275 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497576k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2023-08-09T14:52:09.361 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi203.front.sepia.ceph.com 2023-08-09T14:52:09.361 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-09T14:52:09.362 DEBUG:teuthology.orchestra.run.smithi203:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.1) 2023-08-09T14:52:09.391 INFO:tasks.ceph.mgr.y.smithi043.stderr:2023-08-09T14:52:09.389+0000 7f35b420d040 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2023-08-09T14:52:09.450 INFO:tasks.ceph.mgr.y.smithi043.stderr:2023-08-09T14:52:09.449+0000 7f35b420d040 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2023-08-09T14:52:09.459 INFO:teuthology.orchestra.run.smithi203.stderr:rmdir: failed to remove '/home/ubuntu/cephtest/mnt.1': No such file or directory 2023-08-09T14:52:09.460 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-08-09T14:52:09.461 DEBUG:teuthology.orchestra.run.smithi043:> set -ex 2023-08-09T14:52:09.461 DEBUG:teuthology.orchestra.run.smithi043:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-09T14:52:09.505 INFO:tasks.ceph.mgr.y.smithi043.stderr:2023-08-09T14:52:09.503+0000 7f35b420d040 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2023-08-09T14:52:09.517 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4054441,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6498844k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=27,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3249420k,nr_inodes=812355,mode=700,uid=1000,gid=1275 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6498844k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2023-08-09T14:52:09.517 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi043.front.sepia.ceph.com 2023-08-09T14:52:09.518 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-09T14:52:09.518 DEBUG:teuthology.orchestra.run.smithi043:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.2) 2023-08-09T14:52:09.564 INFO:tasks.ceph.mgr.y.smithi043.stderr:2023-08-09T14:52:09.559+0000 7f35b420d040 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2023-08-09T14:52:09.614 INFO:teuthology.orchestra.run.smithi043.stderr:rmdir: failed to remove '/home/ubuntu/cephtest/mnt.2': No such file or directory 2023-08-09T14:52:09.615 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-08-09T14:52:09.615 DEBUG:teuthology.orchestra.run.smithi203:> set -ex 2023-08-09T14:52:09.616 DEBUG:teuthology.orchestra.run.smithi203:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-09T14:52:09.627 INFO:tasks.ceph.mgr.x.smithi203.stderr:2023-08-09T14:52:09.626+0000 7fb798253040 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2023-08-09T14:52:09.674 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4053648,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497576k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=27,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248784k,nr_inodes=812196,mode=700,uid=1000,gid=1275 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497576k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2023-08-09T14:52:09.675 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi203.front.sepia.ceph.com 2023-08-09T14:52:09.675 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-09T14:52:09.675 DEBUG:teuthology.orchestra.run.smithi203:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.3) 2023-08-09T14:52:09.687 INFO:tasks.ceph.mgr.x.smithi203.stderr:2023-08-09T14:52:09.686+0000 7fb798253040 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2023-08-09T14:52:09.731 INFO:teuthology.orchestra.run.smithi203.stderr:rmdir: failed to remove '/home/ubuntu/cephtest/mnt.3': No such file or directory 2023-08-09T14:52:09.731 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-08-09T14:52:09.732 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-08-09T14:52:09.742 INFO:tasks.ceph.mgr.x.smithi203.stderr:2023-08-09T14:52:09.741+0000 7fb798253040 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2023-08-09T14:52:09.876 INFO:tasks.ceph.mgr.y.smithi043.stderr:2023-08-09T14:52:09.874+0000 7f35b420d040 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2023-08-09T14:52:09.902 INFO:tasks.ceph.mgr.x.smithi203.stderr:2023-08-09T14:52:09.901+0000 7fb798253040 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2023-08-09T14:52:09.937 INFO:tasks.ceph.mgr.y.smithi043.stderr:2023-08-09T14:52:09.935+0000 7f35b420d040 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2023-08-09T14:52:09.949 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:09.948+0000 7f0712aad640 1 Processor -- start 2023-08-09T14:52:09.950 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:09.948+0000 7f0712aad640 1 -- start start 2023-08-09T14:52:09.950 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:09.949+0000 7f0712aad640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f070c15f7b0 0x7f070c0b4760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T14:52:09.951 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:09.949+0000 7f0712aad640 1 -- --> v1:172.21.15.43:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f070c0b7bf0 con 0x7f070c0b1050 2023-08-09T14:52:09.951 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:09.949+0000 7f0712aad640 1 -- --> v1:172.21.15.203:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f070c0b7ed0 con 0x7f070c0cd6b0 2023-08-09T14:52:09.951 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:09.949+0000 7f0712aad640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f070c0d0370 con 0x7f070c15f7b0 2023-08-09T14:52:09.951 INFO:tasks.ceph.mgr.x.smithi203.stderr:2023-08-09T14:52:09.950+0000 7fb798253040 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2023-08-09T14:52:09.953 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:09.949+0000 7f07112aa640 1 --1- >> v1:172.21.15.43:6789/0 conn(0x7f070c0b1050 0x7f070c0ccfa0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.43:6789/0 says I am v1:172.21.15.43:41532/0 (socket says 172.21.15.43:41532) 2023-08-09T14:52:09.953 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:09.949+0000 7f07112aa640 1 -- 172.21.15.43:0/2093252679 learned_addr learned my addr 172.21.15.43:0/2093252679 (peer_addr_for_me v1:172.21.15.43:0/0) 2023-08-09T14:52:09.953 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:09.950+0000 7f0711aab640 1 --2- 172.21.15.43:0/2093252679 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f070c15f7b0 0x7f070c0b4760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T14:52:09.953 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:09.950+0000 7f0710aa9640 1 -- 172.21.15.43:0/2093252679 <== mon.0 v1:172.21.15.43:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3441067626 0 0) 0x7f070c0b7bf0 con 0x7f070c0b1050 2023-08-09T14:52:09.953 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:09.950+0000 7f0710aa9640 1 -- 172.21.15.43:0/2093252679 --> v1:172.21.15.43:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f06f8003520 con 0x7f070c0b1050 2023-08-09T14:52:09.954 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:09.950+0000 7f0710aa9640 1 -- 172.21.15.43:0/2093252679 <== mon.0 v1:172.21.15.43:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1946687172 0 0) 0x7f06f8003520 con 0x7f070c0b1050 2023-08-09T14:52:09.954 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:09.950+0000 7f0710aa9640 1 -- 172.21.15.43:0/2093252679 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f070c15f7b0 msgr2=0x7f070c0b4760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-09T14:52:09.954 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:09.950+0000 7f0710aa9640 1 --2- 172.21.15.43:0/2093252679 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f070c15f7b0 0x7f070c0b4760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T14:52:09.954 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:09.950+0000 7f0710aa9640 1 -- 172.21.15.43:0/2093252679 >> v1:172.21.15.203:6789/0 conn(0x7f070c0cd6b0 legacy=0x7f070c0d0e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T14:52:09.954 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:09.950+0000 7f0710aa9640 1 -- 172.21.15.43:0/2093252679 --> v1:172.21.15.43:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f070c0d0650 con 0x7f070c0b1050 2023-08-09T14:52:09.954 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:09.950+0000 7f0710aa9640 1 -- 172.21.15.43:0/2093252679 <== mon.0 v1:172.21.15.43:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1285150740 0 0) 0x7f06fc001670 con 0x7f070c0b1050 2023-08-09T14:52:09.954 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:09.950+0000 7f0710aa9640 1 -- 172.21.15.43:0/2093252679 >> v1:172.21.15.43:6789/0 conn(0x7f070c0b1050 legacy=0x7f070c0ccfa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T14:52:09.955 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:09.950+0000 7f0710aa9640 1 --2- 172.21.15.43:0/2093252679 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f06f80039a0 0x7f06f8003dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T14:52:09.955 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:09.950+0000 7f0710aa9640 1 --2- 172.21.15.43:0/2093252679 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f06f80047c0 0x7f06f8004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T14:52:09.955 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:09.950+0000 7f0710aa9640 1 --2- 172.21.15.43:0/2093252679 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f070c15f7b0 0x7f06f8005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T14:52:09.955 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:09.950+0000 7f0710aa9640 1 -- 172.21.15.43:0/2093252679 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f070c0d0370 con 0x7f06f80039a0 2023-08-09T14:52:09.955 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:09.950+0000 7f0710aa9640 1 -- 172.21.15.43:0/2093252679 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f06f800ba30 con 0x7f06f80047c0 2023-08-09T14:52:09.955 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:09.950+0000 7f0710aa9640 1 -- 172.21.15.43:0/2093252679 --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f06f800bbf0 con 0x7f070c15f7b0 2023-08-09T14:52:09.955 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:09.951+0000 7f0711aab640 1 --2- 172.21.15.43:0/2093252679 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f06f80039a0 0x7f06f8003dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T14:52:09.955 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:09.951+0000 7f07112aa640 1 --2- 172.21.15.43:0/2093252679 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f06f80047c0 0x7f06f8004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T14:52:09.956 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:09.951+0000 7f07122ac640 1 --2- 172.21.15.43:0/2093252679 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f070c15f7b0 0x7f06f8005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T14:52:09.956 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:09.951+0000 7f0711aab640 1 -- 172.21.15.43:0/2093252679 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f070c15f7b0 msgr2=0x7f06f8005180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-09T14:52:09.956 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:09.951+0000 7f0711aab640 1 --2- 172.21.15.43:0/2093252679 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f070c15f7b0 0x7f06f8005180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T14:52:09.956 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:09.951+0000 7f0711aab640 1 -- 172.21.15.43:0/2093252679 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f06f80047c0 msgr2=0x7f06f8004c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-09T14:52:09.956 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:09.951+0000 7f0711aab640 1 --2- 172.21.15.43:0/2093252679 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f06f80047c0 0x7f06f8004c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T14:52:09.956 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:09.951+0000 7f0711aab640 1 -- 172.21.15.43:0/2093252679 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f06f8003520 con 0x7f06f80039a0 2023-08-09T14:52:09.956 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:09.951+0000 7f07122ac640 1 --2- 172.21.15.43:0/2093252679 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f070c15f7b0 0x7f06f8005180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-08-09T14:52:09.958 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:09.952+0000 7f0711aab640 1 --2- 172.21.15.43:0/2093252679 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f06f80039a0 0x7f06f8003dc0 secure :-1 s=READY pgs=1937 cs=0 l=1 rev1=1 crypto rx=0x7f0708009a90 tx=0x7f06f80057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=877550f7b05a6488 server_cookie=0 in_seq=0 out_seq=0 2023-08-09T14:52:09.958 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:09.953+0000 7f0710aa9640 1 -- 172.21.15.43:0/2093252679 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f070802c660 con 0x7f06f80039a0 2023-08-09T14:52:09.958 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:09.953+0000 7f0710aa9640 1 -- 172.21.15.43:0/2093252679 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f070802c7c0 con 0x7f06f80039a0 2023-08-09T14:52:09.958 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:09.953+0000 7f0710aa9640 1 -- 172.21.15.43:0/2093252679 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f070802cb50 con 0x7f06f80039a0 2023-08-09T14:52:09.958 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:09.953+0000 7f0712aad640 1 -- 172.21.15.43:0/2093252679 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f06f80039a0 msgr2=0x7f06f8003dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T14:52:09.958 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:09.953+0000 7f0712aad640 1 --2- 172.21.15.43:0/2093252679 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f06f80039a0 0x7f06f8003dc0 secure :-1 s=READY pgs=1937 cs=0 l=1 rev1=1 crypto rx=0x7f0708009a90 tx=0x7f06f80057f0 comp rx=0 tx=0).stop 2023-08-09T14:52:09.958 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:09.953+0000 7f07122ac640 1 -- 172.21.15.43:0/2093252679 reap_dead start 2023-08-09T14:52:09.959 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:09.953+0000 7f0712aad640 1 -- 172.21.15.43:0/2093252679 shutdown_connections 2023-08-09T14:52:09.959 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:09.953+0000 7f0712aad640 1 -- 172.21.15.43:0/2093252679 >> 172.21.15.43:0/2093252679 conn(0x7f070c06a2b0 msgr2=0x7f070c06a690 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-09T14:52:09.959 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:09.954+0000 7f0712aad640 1 -- 172.21.15.43:0/2093252679 shutdown_connections 2023-08-09T14:52:09.959 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:09.954+0000 7f0712aad640 1 -- 172.21.15.43:0/2093252679 wait complete. 2023-08-09T14:52:09.959 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:09.955+0000 7f0712aad640 1 Processor -- start 2023-08-09T14:52:09.959 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:09.955+0000 7f0712aad640 1 -- start start 2023-08-09T14:52:09.959 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:09.956+0000 7f0712aad640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f06f80039a0 0x7f070c148230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T14:52:09.960 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:09.956+0000 7f0712aad640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f070c153d50 0x7f070c1591d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T14:52:09.960 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:09.956+0000 7f0712aad640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f070c0c7d80 0x7f070c15ab70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T14:52:09.960 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:09.956+0000 7f0712aad640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f070c153020 con 0x7f06f80039a0 2023-08-09T14:52:09.960 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:09.956+0000 7f0712aad640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f070c0c8230 con 0x7f070c153d50 2023-08-09T14:52:09.960 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:09.956+0000 7f0712aad640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f070c156760 con 0x7f070c0c7d80 2023-08-09T14:52:09.961 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:09.957+0000 7f0711aab640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f06f80039a0 0x7f070c148230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T14:52:09.961 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:09.957+0000 7f0711aab640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f06f80039a0 0x7f070c148230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:34926/0 (socket says 172.21.15.43:34926) 2023-08-09T14:52:09.962 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:09.957+0000 7f0711aab640 1 -- 172.21.15.43:0/3092807715 learned_addr learned my addr 172.21.15.43:0/3092807715 (peer_addr_for_me v2:172.21.15.43:0/0) 2023-08-09T14:52:09.962 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:09.957+0000 7f07112aa640 1 --2- 172.21.15.43:0/3092807715 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f070c153d50 0x7f070c1591d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T14:52:09.962 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:09.957+0000 7f07122ac640 1 --2- 172.21.15.43:0/3092807715 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f070c0c7d80 0x7f070c15ab70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T14:52:09.962 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:09.957+0000 7f0711aab640 1 -- 172.21.15.43:0/3092807715 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f070c0c7d80 msgr2=0x7f070c15ab70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T14:52:09.962 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:09.957+0000 7f0711aab640 1 --2- 172.21.15.43:0/3092807715 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f070c0c7d80 0x7f070c15ab70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T14:52:09.962 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:09.957+0000 7f0711aab640 1 -- 172.21.15.43:0/3092807715 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f070c153d50 msgr2=0x7f070c1591d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T14:52:09.962 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:09.957+0000 7f0711aab640 1 --2- 172.21.15.43:0/3092807715 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f070c153d50 0x7f070c1591d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T14:52:09.963 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:09.957+0000 7f0711aab640 1 -- 172.21.15.43:0/3092807715 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f070c148870 con 0x7f06f80039a0 2023-08-09T14:52:09.963 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:09.957+0000 7f07122ac640 1 --2- 172.21.15.43:0/3092807715 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f070c0c7d80 0x7f070c15ab70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-08-09T14:52:09.963 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:09.957+0000 7f0711aab640 1 --2- 172.21.15.43:0/3092807715 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f06f80039a0 0x7f070c148230 secure :-1 s=READY pgs=1938 cs=0 l=1 rev1=1 crypto rx=0x7f070802c200 tx=0x7f0708004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-09T14:52:09.963 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:09.958+0000 7f07027fc640 1 -- 172.21.15.43:0/3092807715 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0708003810 con 0x7f06f80039a0 2023-08-09T14:52:09.963 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:09.958+0000 7f07027fc640 1 -- 172.21.15.43:0/3092807715 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0708003970 con 0x7f06f80039a0 2023-08-09T14:52:09.963 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:09.958+0000 7f07027fc640 1 -- 172.21.15.43:0/3092807715 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0708003ce0 con 0x7f06f80039a0 2023-08-09T14:52:09.963 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:09.958+0000 7f0712aad640 1 -- 172.21.15.43:0/3092807715 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f070c153270 con 0x7f06f80039a0 2023-08-09T14:52:09.963 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:09.958+0000 7f0712aad640 1 -- 172.21.15.43:0/3092807715 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f070c153830 con 0x7f06f80039a0 2023-08-09T14:52:09.964 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:09.959+0000 7f0712aad640 1 -- 172.21.15.43:0/3092807715 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f070c1539c0 con 0x7f06f80039a0 2023-08-09T14:52:09.964 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:09.960+0000 7f07027fc640 1 -- 172.21.15.43:0/3092807715 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 84) v1 ==== 88444+0+0 (secure 0 0 0) 0x7f0708034440 con 0x7f06f80039a0 2023-08-09T14:52:09.964 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:09.961+0000 7f07027fc640 1 --2- 172.21.15.43:0/3092807715 >> [v2:172.21.15.203:6832/843920641,v1:172.21.15.203:6833/843920641] conn(0x7f06f006a1a0 0x7f06f006c660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T14:52:09.965 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:09.961+0000 7f07027fc640 1 -- 172.21.15.43:0/3092807715 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(173..173 src has 1..173) v4 ==== 6770+0+0 (secure 0 0 0) 0x7f07080aa810 con 0x7f06f80039a0 2023-08-09T14:52:09.965 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:09.961+0000 7f07112aa640 1 -- 172.21.15.43:0/3092807715 >> [v2:172.21.15.203:6832/843920641,v1:172.21.15.203:6833/843920641] conn(0x7f06f006a1a0 msgr2=0x7f06f006c660 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.203:6832/843920641 2023-08-09T14:52:09.965 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:09.961+0000 7f07112aa640 1 --2- 172.21.15.43:0/3092807715 >> [v2:172.21.15.203:6832/843920641,v1:172.21.15.203:6833/843920641] conn(0x7f06f006a1a0 0x7f06f006c660 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-08-09T14:52:09.966 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:09.965+0000 7f07027fc640 1 -- 172.21.15.43:0/3092807715 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178066 (secure 0 0 0) 0x7f0708075bc0 con 0x7f06f80039a0 2023-08-09T14:52:09.995 INFO:tasks.ceph.mgr.y.smithi043.stderr:2023-08-09T14:52:09.993+0000 7f35b420d040 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2023-08-09T14:52:10.005 INFO:tasks.ceph.mgr.x.smithi203.stderr:2023-08-09T14:52:10.004+0000 7fb798253040 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2023-08-09T14:52:10.055 INFO:tasks.ceph.mgr.x.smithi203.stderr:2023-08-09T14:52:10.053+0000 7fb798253040 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2023-08-09T14:52:10.083 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:10.082+0000 7f07027fc640 1 -- 172.21.15.43:0/3092807715 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mgrmap(e 85) v1 ==== 88052+0+0 (secure 0 0 0) 0x7f0708075f90 con 0x7f06f80039a0 2023-08-09T14:52:10.084 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:10.083+0000 7f07027fc640 1 -- 172.21.15.43:0/3092807715 >> [v2:172.21.15.203:6832/843920641,v1:172.21.15.203:6833/843920641] conn(0x7f06f006a1a0 msgr2=0x7f06f006c660 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-08-09T14:52:10.084 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:10.083+0000 7f07027fc640 1 --2- 172.21.15.43:0/3092807715 >> [v2:172.21.15.203:6832/843920641,v1:172.21.15.203:6833/843920641] conn(0x7f06f006a1a0 0x7f06f006c660 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T14:52:10.149 INFO:tasks.ceph.mgr.x.smithi203.stderr:2023-08-09T14:52:10.147+0000 7fb71e679640 -1 client.0 error registering admin socket command: (17) File exists 2023-08-09T14:52:10.149 INFO:tasks.ceph.mgr.x.smithi203.stderr:2023-08-09T14:52:10.147+0000 7fb71e679640 -1 client.0 error registering admin socket command: (17) File exists 2023-08-09T14:52:10.149 INFO:tasks.ceph.mgr.x.smithi203.stderr:2023-08-09T14:52:10.147+0000 7fb71e679640 -1 client.0 error registering admin socket command: (17) File exists 2023-08-09T14:52:10.149 INFO:tasks.ceph.mgr.x.smithi203.stderr:2023-08-09T14:52:10.147+0000 7fb71e679640 -1 client.0 error registering admin socket command: (17) File exists 2023-08-09T14:52:10.149 INFO:tasks.ceph.mgr.x.smithi203.stderr:2023-08-09T14:52:10.147+0000 7fb71e679640 -1 client.0 error registering admin socket command: (17) File exists 2023-08-09T14:52:10.149 INFO:tasks.ceph.mgr.x.smithi203.stderr:2023-08-09T14:52:10.147+0000 7fb71b673640 -1 client.0 error registering admin socket command: (17) File exists 2023-08-09T14:52:10.150 INFO:tasks.ceph.mgr.x.smithi203.stderr:2023-08-09T14:52:10.147+0000 7fb71b673640 -1 client.0 error registering admin socket command: (17) File exists 2023-08-09T14:52:10.150 INFO:tasks.ceph.mgr.x.smithi203.stderr:2023-08-09T14:52:10.147+0000 7fb71b673640 -1 client.0 error registering admin socket command: (17) File exists 2023-08-09T14:52:10.150 INFO:tasks.ceph.mgr.x.smithi203.stderr:2023-08-09T14:52:10.147+0000 7fb71b673640 -1 client.0 error registering admin socket command: (17) File exists 2023-08-09T14:52:10.150 INFO:tasks.ceph.mgr.x.smithi203.stderr:2023-08-09T14:52:10.147+0000 7fb71b673640 -1 client.0 error registering admin socket command: (17) File exists 2023-08-09T14:52:10.175 INFO:tasks.ceph.mgr.y.smithi043.stderr:2023-08-09T14:52:10.174+0000 7f35b420d040 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2023-08-09T14:52:10.214 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:10.212+0000 7f0712aad640 1 -- 172.21.15.43:0/3092807715 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f070c0739f0 con 0x7f06f80039a0 2023-08-09T14:52:10.215 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:10.214+0000 7f07027fc640 1 -- 172.21.15.43:0/3092807715 <== mon.0 v2:172.21.15.43:3300/0 8 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 78 v78) v1 ==== 94+0+4880 (secure 0 0 0) 0x7f070808cd90 con 0x7f06f80039a0 2023-08-09T14:52:10.215 INFO:teuthology.orchestra.run.smithi043.stdout: 2023-08-09T14:52:10.216 INFO:teuthology.orchestra.run.smithi043.stdout:{"epoch":78,"default_fscid":9,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":7079,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.203:6836/807059349","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6834","nonce":807059349},{"type":"v1","addr":"172.21.15.203:6836","nonce":807059349}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":74},{"gid":7924,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.203:6837/3166829033","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6835","nonce":3166829033},{"type":"v1","addr":"172.21.15.203:6837","nonce":3166829033}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":74},{"gid":9642,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.43:6833/3803974919","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6832","nonce":3803974919},{"type":"v1","addr":"172.21.15.43:6833","nonce":3803974919}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":73}],"filesystems":[{"mdsmap":{"epoch":78,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-08-09T14:51:47.253704+0000","modified":"2023-08-09T14:51:50.806331+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":9648},"failed":[],"damaged":[],"stopped":[],"info":{"gid_9648":{"gid":9648,"name":"a","rank":0,"incarnation":75,"state":"up:active","state_seq":3,"addr":"172.21.15.43:6835/2435237803","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6834","nonce":2435237803},{"type":"v1","addr":"172.21.15.43:6835","nonce":2435237803}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[21],"metadata_pool":20,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":9}]} 2023-08-09T14:52:10.216 INFO:teuthology.orchestra.run.smithi043.stderr:dumped fsmap epoch 78 2023-08-09T14:52:10.218 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:10.216+0000 7f0712aad640 1 -- 172.21.15.43:0/3092807715 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f06f80039a0 msgr2=0x7f070c148230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T14:52:10.218 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:10.216+0000 7f0712aad640 1 --2- 172.21.15.43:0/3092807715 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f06f80039a0 0x7f070c148230 secure :-1 s=READY pgs=1938 cs=0 l=1 rev1=1 crypto rx=0x7f070802c200 tx=0x7f0708004730 comp rx=0 tx=0).stop 2023-08-09T14:52:10.218 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:10.216+0000 7f0712aad640 1 -- 172.21.15.43:0/3092807715 shutdown_connections 2023-08-09T14:52:10.218 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:10.216+0000 7f0712aad640 1 --2- 172.21.15.43:0/3092807715 >> [v2:172.21.15.203:6832/843920641,v1:172.21.15.203:6833/843920641] conn(0x7f06f006a1a0 0x7f06f006c660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T14:52:10.218 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:10.216+0000 7f0712aad640 1 --2- 172.21.15.43:0/3092807715 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f070c0c7d80 0x7f070c15ab70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T14:52:10.218 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:10.216+0000 7f0712aad640 1 --2- 172.21.15.43:0/3092807715 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f070c153d50 0x7f070c1591d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T14:52:10.218 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:10.216+0000 7f0712aad640 1 --2- 172.21.15.43:0/3092807715 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f06f80039a0 0x7f070c148230 unknown :-1 s=CLOSED pgs=1938 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T14:52:10.219 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:10.216+0000 7f0712aad640 1 -- 172.21.15.43:0/3092807715 >> 172.21.15.43:0/3092807715 conn(0x7f070c06a2b0 msgr2=0x7f070c0c8640 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-09T14:52:10.219 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:10.216+0000 7f0712aad640 1 -- 172.21.15.43:0/3092807715 shutdown_connections 2023-08-09T14:52:10.219 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:10.216+0000 7f0712aad640 1 -- 172.21.15.43:0/3092807715 wait complete. 2023-08-09T14:52:10.228 INFO:tasks.ceph.mgr.y.smithi043.stderr:2023-08-09T14:52:10.227+0000 7f35b420d040 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2023-08-09T14:52:10.236 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-08-09T14:52:10.286 INFO:tasks.ceph.mgr.y.smithi043.stderr:2023-08-09T14:52:10.284+0000 7f35b420d040 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2023-08-09T14:52:10.339 INFO:tasks.ceph.mgr.y.smithi043.stderr:2023-08-09T14:52:10.338+0000 7f35b420d040 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2023-08-09T14:52:10.448 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:10.446+0000 7f2237fff640 1 Processor -- start 2023-08-09T14:52:10.448 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:10.447+0000 7f2237fff640 1 -- start start 2023-08-09T14:52:10.449 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:10.448+0000 7f2237fff640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f223815f7b0 0x7f22380b4760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T14:52:10.450 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:10.448+0000 7f2237fff640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f22380b1050 0x7f22380ccfa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T14:52:10.450 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:10.448+0000 7f2237fff640 1 -- --> v1:172.21.15.203:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f22380cd930 con 0x7f22380cd4e0 2023-08-09T14:52:10.450 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:10.448+0000 7f2237fff640 1 -- --> v2:172.21.15.43:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f22380b7bf0 con 0x7f22380b1050 2023-08-09T14:52:10.450 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:10.448+0000 7f2237fff640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f22380b7ed0 con 0x7f223815f7b0 2023-08-09T14:52:10.451 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:10.448+0000 7f222ffff640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f22380b1050 0x7f22380ccfa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T14:52:10.451 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:10.449+0000 7f222ffff640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f22380b1050 0x7f22380ccfa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:34940/0 (socket says 172.21.15.43:34940) 2023-08-09T14:52:10.451 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:10.449+0000 7f222ffff640 1 -- 172.21.15.43:0/1139047784 learned_addr learned my addr 172.21.15.43:0/1139047784 (peer_addr_for_me v2:172.21.15.43:0/0) 2023-08-09T14:52:10.451 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:10.449+0000 7f2236ffd640 1 --2- 172.21.15.43:0/1139047784 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f223815f7b0 0x7f22380b4760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T14:52:10.452 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:10.449+0000 7f222ffff640 1 -- 172.21.15.43:0/1139047784 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f223815f7b0 msgr2=0x7f22380b4760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-09T14:52:10.452 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:10.449+0000 7f222ffff640 1 --2- 172.21.15.43:0/1139047784 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f223815f7b0 0x7f22380b4760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T14:52:10.452 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:10.449+0000 7f222ffff640 1 -- 172.21.15.43:0/1139047784 >> v1:172.21.15.203:6789/0 conn(0x7f22380cd4e0 legacy=0x7f22380ceae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-09T14:52:10.452 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:10.449+0000 7f222ffff640 1 -- 172.21.15.43:0/1139047784 --> v2:172.21.15.43:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f22380d0370 con 0x7f22380b1050 2023-08-09T14:52:10.453 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:10.450+0000 7f222ffff640 1 --2- 172.21.15.43:0/1139047784 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f22380b1050 0x7f22380ccfa0 secure :-1 s=READY pgs=1939 cs=0 l=1 rev1=1 crypto rx=0x7f2220009870 tx=0x7f222002b8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=865bc6a6cd30f4fd server_cookie=0 in_seq=0 out_seq=0 2023-08-09T14:52:10.453 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:10.451+0000 7f22367fc640 1 -- 172.21.15.43:0/1139047784 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2220031640 con 0x7f22380b1050 2023-08-09T14:52:10.453 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:10.451+0000 7f22367fc640 1 -- 172.21.15.43:0/1139047784 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f22200317a0 con 0x7f22380b1050 2023-08-09T14:52:10.454 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:10.451+0000 7f22367fc640 1 -- 172.21.15.43:0/1139047784 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2220031b30 con 0x7f22380b1050 2023-08-09T14:52:10.454 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:10.451+0000 7f2237fff640 1 -- 172.21.15.43:0/1139047784 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f22380b1050 msgr2=0x7f22380ccfa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T14:52:10.454 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:10.451+0000 7f2237fff640 1 --2- 172.21.15.43:0/1139047784 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f22380b1050 0x7f22380ccfa0 secure :-1 s=READY pgs=1939 cs=0 l=1 rev1=1 crypto rx=0x7f2220009870 tx=0x7f222002b8f0 comp rx=0 tx=0).stop 2023-08-09T14:52:10.454 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:10.451+0000 7f2237fff640 1 -- 172.21.15.43:0/1139047784 shutdown_connections 2023-08-09T14:52:10.454 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:10.451+0000 7f2237fff640 1 --2- 172.21.15.43:0/1139047784 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f22380b1050 0x7f22380ccfa0 unknown :-1 s=CLOSED pgs=1939 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T14:52:10.454 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:10.451+0000 7f2237fff640 1 --2- 172.21.15.43:0/1139047784 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f223815f7b0 0x7f22380b4760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T14:52:10.454 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:10.451+0000 7f2237fff640 1 -- 172.21.15.43:0/1139047784 >> 172.21.15.43:0/1139047784 conn(0x7f223806a2b0 msgr2=0x7f223806a690 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-09T14:52:10.454 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:10.451+0000 7f2237fff640 1 -- 172.21.15.43:0/1139047784 shutdown_connections 2023-08-09T14:52:10.455 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:10.451+0000 7f2237fff640 1 -- 172.21.15.43:0/1139047784 wait complete. 2023-08-09T14:52:10.455 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:10.453+0000 7f2237fff640 1 Processor -- start 2023-08-09T14:52:10.455 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:10.453+0000 7f2237fff640 1 -- start start 2023-08-09T14:52:10.456 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:10.454+0000 7f2237fff640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f22380b1050 0x7f2238149670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T14:52:10.457 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:10.454+0000 7f2237fff640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f22380cd4e0 0x7f22380c6850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T14:52:10.457 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:10.454+0000 7f2237fff640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f223815f7b0 0x7f22380c6d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T14:52:10.457 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:10.454+0000 7f2237fff640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f22380d1f30 con 0x7f22380cd4e0 2023-08-09T14:52:10.457 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:10.454+0000 7f2237fff640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f22380d2180 con 0x7f223815f7b0 2023-08-09T14:52:10.457 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:10.454+0000 7f2237fff640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f22380d2460 con 0x7f22380b1050 2023-08-09T14:52:10.457 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:10.455+0000 7f222ffff640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f22380cd4e0 0x7f22380c6850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T14:52:10.458 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:10.455+0000 7f222ffff640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f22380cd4e0 0x7f22380c6850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:34942/0 (socket says 172.21.15.43:34942) 2023-08-09T14:52:10.458 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:10.455+0000 7f222ffff640 1 -- 172.21.15.43:0/929350585 learned_addr learned my addr 172.21.15.43:0/929350585 (peer_addr_for_me v2:172.21.15.43:0/0) 2023-08-09T14:52:10.458 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:10.455+0000 7f2236ffd640 1 --2- 172.21.15.43:0/929350585 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f22380b1050 0x7f2238149670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T14:52:10.458 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:10.455+0000 7f22377fe640 1 --2- 172.21.15.43:0/929350585 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f223815f7b0 0x7f22380c6d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T14:52:10.458 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:10.455+0000 7f222ffff640 1 -- 172.21.15.43:0/929350585 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f22380b1050 msgr2=0x7f2238149670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T14:52:10.458 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:10.455+0000 7f222ffff640 1 --2- 172.21.15.43:0/929350585 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f22380b1050 0x7f2238149670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T14:52:10.458 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:10.455+0000 7f222ffff640 1 -- 172.21.15.43:0/929350585 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f223815f7b0 msgr2=0x7f22380c6d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T14:52:10.458 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:10.455+0000 7f222ffff640 1 --2- 172.21.15.43:0/929350585 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f223815f7b0 0x7f22380c6d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T14:52:10.459 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:10.455+0000 7f222ffff640 1 -- 172.21.15.43:0/929350585 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f22200094d0 con 0x7f22380cd4e0 2023-08-09T14:52:10.459 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:10.455+0000 7f22377fe640 1 --2- 172.21.15.43:0/929350585 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f223815f7b0 0x7f22380c6d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-08-09T14:52:10.459 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:10.455+0000 7f222ffff640 1 --2- 172.21.15.43:0/929350585 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f22380cd4e0 0x7f22380c6850 secure :-1 s=READY pgs=1940 cs=0 l=1 rev1=1 crypto rx=0x7f2220009870 tx=0x7f22200046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-09T14:52:10.460 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:10.456+0000 7f222f7fe640 1 -- 172.21.15.43:0/929350585 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f222003ac60 con 0x7f22380cd4e0 2023-08-09T14:52:10.460 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:10.456+0000 7f222f7fe640 1 -- 172.21.15.43:0/929350585 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f222003adc0 con 0x7f22380cd4e0 2023-08-09T14:52:10.460 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:10.456+0000 7f222f7fe640 1 -- 172.21.15.43:0/929350585 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2220044630 con 0x7f22380cd4e0 2023-08-09T14:52:10.460 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:10.456+0000 7f2237fff640 1 -- 172.21.15.43:0/929350585 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f22380d2740 con 0x7f22380cd4e0 2023-08-09T14:52:10.460 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:10.456+0000 7f2237fff640 1 -- 172.21.15.43:0/929350585 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2238149bb0 con 0x7f22380cd4e0 2023-08-09T14:52:10.460 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:10.457+0000 7f2237fff640 1 -- 172.21.15.43:0/929350585 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f22380d2c60 con 0x7f22380cd4e0 2023-08-09T14:52:10.461 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:10.458+0000 7f222f7fe640 1 -- 172.21.15.43:0/929350585 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 85) v1 ==== 88052+0+0 (secure 0 0 0) 0x7f222002f050 con 0x7f22380cd4e0 2023-08-09T14:52:10.461 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:10.459+0000 7f222f7fe640 1 -- 172.21.15.43:0/929350585 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(174..174 src has 1..174) v4 ==== 7114+0+0 (secure 0 0 0) 0x7f22200aa240 con 0x7f22380cd4e0 2023-08-09T14:52:10.465 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:10.464+0000 7f222f7fe640 1 -- 172.21.15.43:0/929350585 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178066 (secure 0 0 0) 0x7f22380d2c60 con 0x7f22380cd4e0 2023-08-09T14:52:10.651 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:10.649+0000 7f2237fff640 1 -- 172.21.15.43:0/929350585 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f22380cd930 con 0x7f22380cd4e0 2023-08-09T14:52:10.654 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:10.653+0000 7f222f7fe640 1 -- 172.21.15.43:0/929350585 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v174) v1 ==== 74+0+19582 (secure 0 0 0) 0x7f22380cd930 con 0x7f22380cd4e0 2023-08-09T14:52:10.655 INFO:teuthology.orchestra.run.smithi043.stdout: 2023-08-09T14:52:10.656 INFO:teuthology.orchestra.run.smithi043.stdout:{"epoch":174,"fsid":"daac87ee-1337-4250-a955-aea4e408a39b","created":"2023-08-09T13:33:41.701198+0000","modified":"2023-08-09T14:52:10.059650+0000","last_up_change":"2023-08-09T13:33:53.783453+0000","last_in_change":"2023-08-09T13:33:45.148904+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":21,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":"rbd","create_time":"2023-08-09T13:33:54.365137+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":".mgr","create_time":"2023-08-09T13:33:54.831803+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":20,"pool_name":"cephfs_metadata","create_time":"2023-08-09T14:51:45.692613+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"171","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":21,"pool_name":"cephfs_data","create_time":"2023-08-09T14:51:46.286715+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"172","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"cbd49ecd-e896-4fbd-864e-5c37a829b165","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":170,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6800","nonce":502667335},{"type":"v1","addr":"172.21.15.43:6801","nonce":502667335}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6802","nonce":502667335},{"type":"v1","addr":"172.21.15.43:6803","nonce":502667335}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6806","nonce":502667335},{"type":"v1","addr":"172.21.15.43:6807","nonce":502667335}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6804","nonce":502667335},{"type":"v1","addr":"172.21.15.43:6805","nonce":502667335}]},"public_addr":"172.21.15.43:6801/502667335","cluster_addr":"172.21.15.43:6803/502667335","heartbeat_back_addr":"172.21.15.43:6807/502667335","heartbeat_front_addr":"172.21.15.43:6805/502667335","state":["exists","up"]},{"osd":1,"uuid":"75c7e0fe-1ec7-4fe8-a2f2-3661a65b8500","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":170,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6808","nonce":3289134610},{"type":"v1","addr":"172.21.15.43:6809","nonce":3289134610}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6810","nonce":3289134610},{"type":"v1","addr":"172.21.15.43:6812","nonce":3289134610}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6821","nonce":3289134610},{"type":"v1","addr":"172.21.15.43:6824","nonce":3289134610}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6815","nonce":3289134610},{"type":"v1","addr":"172.21.15.43:6818","nonce":3289134610}]},"public_addr":"172.21.15.43:6809/3289134610","cluster_addr":"172.21.15.43:6812/3289134610","heartbeat_back_addr":"172.21.15.43:6824/3289134610","heartbeat_front_addr":"172.21.15.43:6818/3289134610","state":["exists","up"]},{"osd":2,"uuid":"bb5c41e9-7f5f-4125-9b01-3bb8bfc2ea83","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":170,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6811","nonce":4079709736},{"type":"v1","addr":"172.21.15.43:6814","nonce":4079709736}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6817","nonce":4079709736},{"type":"v1","addr":"172.21.15.43:6820","nonce":4079709736}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6829","nonce":4079709736},{"type":"v1","addr":"172.21.15.43:6831","nonce":4079709736}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6823","nonce":4079709736},{"type":"v1","addr":"172.21.15.43:6826","nonce":4079709736}]},"public_addr":"172.21.15.43:6814/4079709736","cluster_addr":"172.21.15.43:6820/4079709736","heartbeat_back_addr":"172.21.15.43:6831/4079709736","heartbeat_front_addr":"172.21.15.43:6826/4079709736","state":["exists","up"]},{"osd":3,"uuid":"bfe4c8b4-694a-465c-87e6-2b2aab852613","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":170,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6813","nonce":1709538759},{"type":"v1","addr":"172.21.15.43:6816","nonce":1709538759}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6819","nonce":1709538759},{"type":"v1","addr":"172.21.15.43:6822","nonce":1709538759}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6828","nonce":1709538759},{"type":"v1","addr":"172.21.15.43:6830","nonce":1709538759}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6825","nonce":1709538759},{"type":"v1","addr":"172.21.15.43:6827","nonce":1709538759}]},"public_addr":"172.21.15.43:6816/1709538759","cluster_addr":"172.21.15.43:6822/1709538759","heartbeat_back_addr":"172.21.15.43:6830/1709538759","heartbeat_front_addr":"172.21.15.43:6827/1709538759","state":["exists","up"]},{"osd":4,"uuid":"fa2e8a77-aba1-4756-853e-0d174a5bdeba","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":170,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6800","nonce":4196866481},{"type":"v1","addr":"172.21.15.203:6801","nonce":4196866481}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6802","nonce":4196866481},{"type":"v1","addr":"172.21.15.203:6803","nonce":4196866481}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6806","nonce":4196866481},{"type":"v1","addr":"172.21.15.203:6807","nonce":4196866481}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6804","nonce":4196866481},{"type":"v1","addr":"172.21.15.203:6805","nonce":4196866481}]},"public_addr":"172.21.15.203:6801/4196866481","cluster_addr":"172.21.15.203:6803/4196866481","heartbeat_back_addr":"172.21.15.203:6807/4196866481","heartbeat_front_addr":"172.21.15.203:6805/4196866481","state":["exists","up"]},{"osd":5,"uuid":"81811e4b-4d24-4f51-a82c-f1d700a2a00d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":170,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6816","nonce":3059290081},{"type":"v1","addr":"172.21.15.203:6817","nonce":3059290081}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6818","nonce":3059290081},{"type":"v1","addr":"172.21.15.203:6819","nonce":3059290081}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6822","nonce":3059290081},{"type":"v1","addr":"172.21.15.203:6823","nonce":3059290081}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6820","nonce":3059290081},{"type":"v1","addr":"172.21.15.203:6821","nonce":3059290081}]},"public_addr":"172.21.15.203:6817/3059290081","cluster_addr":"172.21.15.203:6819/3059290081","heartbeat_back_addr":"172.21.15.203:6823/3059290081","heartbeat_front_addr":"172.21.15.203:6821/3059290081","state":["exists","up"]},{"osd":6,"uuid":"307bac4d-6a5b-443d-af75-56d695d7674f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":170,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6824","nonce":512155569},{"type":"v1","addr":"172.21.15.203:6825","nonce":512155569}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6826","nonce":512155569},{"type":"v1","addr":"172.21.15.203:6827","nonce":512155569}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6830","nonce":512155569},{"type":"v1","addr":"172.21.15.203:6831","nonce":512155569}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6828","nonce":512155569},{"type":"v1","addr":"172.21.15.203:6829","nonce":512155569}]},"public_addr":"172.21.15.203:6825/512155569","cluster_addr":"172.21.15.203:6827/512155569","heartbeat_back_addr":"172.21.15.203:6831/512155569","heartbeat_front_addr":"172.21.15.203:6829/512155569","state":["exists","up"]},{"osd":7,"uuid":"a95dc7ca-9f1b-4295-af70-68a22a2d7d5e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":170,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6808","nonce":3331450438},{"type":"v1","addr":"172.21.15.203:6809","nonce":3331450438}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6810","nonce":3331450438},{"type":"v1","addr":"172.21.15.203:6811","nonce":3331450438}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6814","nonce":3331450438},{"type":"v1","addr":"172.21.15.203:6815","nonce":3331450438}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6812","nonce":3331450438},{"type":"v1","addr":"172.21.15.203:6813","nonce":3331450438}]},"public_addr":"172.21.15.203:6809/3331450438","cluster_addr":"172.21.15.203:6811/3331450438","heartbeat_back_addr":"172.21.15.203:6815/3331450438","heartbeat_front_addr":"172.21.15.203:6813/3331450438","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-09T13:33:50.547652+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-09T13:33:50.589469+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-09T13:33:50.561724+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-09T13:33:50.599650+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-09T13:33:50.411855+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-09T13:33:50.219681+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-09T13:33:50.504664+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-09T13:33:50.441248+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.203:0/3301934490":"2023-08-10T14:52:10.059559+0000","172.21.15.203:0/1715525748":"2023-08-10T14:52:10.059559+0000","172.21.15.203:0/1477488908":"2023-08-10T14:52:10.059559+0000","172.21.15.203:6833/843920641":"2023-08-10T14:52:10.059559+0000","172.21.15.203:6832/843920641":"2023-08-10T14:52:10.059559+0000","172.21.15.203:0/2095212004":"2023-08-10T14:52:02.909250+0000","172.21.15.203:0/1196801979":"2023-08-10T14:52:10.059559+0000","172.21.15.203:0/3749872091":"2023-08-10T14:52:02.909250+0000","172.21.15.203:0/1305485142":"2023-08-10T14:52:10.059559+0000","172.21.15.203:0/651612122":"2023-08-10T14:52:02.909250+0000","172.21.15.203:0/4120954828":"2023-08-10T14:52:02.909250+0000","172.21.15.203:0/3552079285":"2023-08-10T14:52:10.059559+0000","172.21.15.203:0/2783782601":"2023-08-10T14:52:02.909250+0000","172.21.15.203:0/2232659577":"2023-08-10T14:52:02.909250+0000","172.21.15.203:6833/2566816332":"2023-08-10T14:52:02.909250+0000","172.21.15.203:6832/2566816332":"2023-08-10T14:52:02.909250+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":7,"snaps":[{"begin":104,"length":1},{"begin":221,"length":1}]}],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-08-09T14:52:10.657 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:10.655+0000 7f2237fff640 1 -- 172.21.15.43:0/929350585 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f22380cd4e0 msgr2=0x7f22380c6850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T14:52:10.658 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:10.655+0000 7f2237fff640 1 --2- 172.21.15.43:0/929350585 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f22380cd4e0 0x7f22380c6850 secure :-1 s=READY pgs=1940 cs=0 l=1 rev1=1 crypto rx=0x7f2220009870 tx=0x7f22200046c0 comp rx=0 tx=0).stop 2023-08-09T14:52:10.658 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:10.655+0000 7f2237fff640 1 -- 172.21.15.43:0/929350585 shutdown_connections 2023-08-09T14:52:10.658 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:10.655+0000 7f2237fff640 1 --2- 172.21.15.43:0/929350585 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f223815f7b0 0x7f22380c6d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T14:52:10.659 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:10.655+0000 7f2237fff640 1 --2- 172.21.15.43:0/929350585 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f22380cd4e0 0x7f22380c6850 unknown :-1 s=CLOSED pgs=1940 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T14:52:10.659 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:10.655+0000 7f2237fff640 1 --2- 172.21.15.43:0/929350585 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f22380b1050 0x7f2238149670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T14:52:10.659 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:10.655+0000 7f2237fff640 1 -- 172.21.15.43:0/929350585 >> 172.21.15.43:0/929350585 conn(0x7f223806a2b0 msgr2=0x7f223815ab60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-09T14:52:10.659 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:10.656+0000 7f2237fff640 1 -- 172.21.15.43:0/929350585 shutdown_connections 2023-08-09T14:52:10.660 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:10.656+0000 7f2237fff640 1 -- 172.21.15.43:0/929350585 wait complete. 2023-08-09T14:52:10.675 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': 'rbd', 'create_time': '2023-08-09T13:33:54.365137+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}, 'read_balance': {'score_acting': 2, 'score_stable': 2, 'optimal_score': 1, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': '.mgr', 'create_time': '2023-08-09T13:33:54.831803+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 20, 'pool_name': 'cephfs_metadata', 'create_time': '2023-08-09T14:51:45.692613+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '171', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 21, 'pool_name': 'cephfs_data', 'create_time': '2023-08-09T14:51:46.286715+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '172', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.3799999952316284, 'score_stable': 1.3799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.3799999952316284, 'raw_score_stable': 1.3799999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-08-09T14:52:10.677 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get rbd pg_num 2023-08-09T14:52:10.899 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:10.898+0000 7f7e9c4b8640 1 Processor -- start 2023-08-09T14:52:10.899 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:10.898+0000 7f7e9c4b8640 1 -- start start 2023-08-09T14:52:10.901 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:10.899+0000 7f7e9c4b8640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f7e9415a0c0 0x7f7e940cbb80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T14:52:10.901 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:10.899+0000 7f7e9c4b8640 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7f7e940ce1c0 0x7f7e940ce5f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T14:52:10.901 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:10.899+0000 7f7e9c4b8640 1 -- --> v1:172.21.15.43:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7e940ceb30 con 0x7f7e9415d510 2023-08-09T14:52:10.901 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:10.899+0000 7f7e9c4b8640 1 -- --> v2:172.21.15.203:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7e940cc0c0 con 0x7f7e940ce1c0 2023-08-09T14:52:10.901 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:10.899+0000 7f7e9c4b8640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7e9415a4a0 con 0x7f7e9415a0c0 2023-08-09T14:52:10.902 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:10.900+0000 7f7e99a53640 1 --1- >> v1:172.21.15.43:6789/0 conn(0x7f7e9415d510 0x7f7e940cb2b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.43:6789/0 says I am v1:172.21.15.43:41538/0 (socket says 172.21.15.43:41538) 2023-08-09T14:52:10.902 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:10.900+0000 7f7e99a53640 1 -- 172.21.15.43:0/2735594767 learned_addr learned my addr 172.21.15.43:0/2735594767 (peer_addr_for_me v1:172.21.15.43:0/0) 2023-08-09T14:52:10.903 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:10.900+0000 7f7e9a254640 1 --2- 172.21.15.43:0/2735594767 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f7e9415a0c0 0x7f7e940cbb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T14:52:10.903 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:10.900+0000 7f7e9aa55640 1 --2- 172.21.15.43:0/2735594767 >> v2:172.21.15.203:3300/0 conn(0x7f7e940ce1c0 0x7f7e940ce5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T14:52:10.904 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:10.900+0000 7f7e99252640 1 -- 172.21.15.43:0/2735594767 <== mon.0 v1:172.21.15.43:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3111533213 0 0) 0x7f7e940ceb30 con 0x7f7e9415d510 2023-08-09T14:52:10.904 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:10.900+0000 7f7e99252640 1 -- 172.21.15.43:0/2735594767 --> v1:172.21.15.43:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f7e80003520 con 0x7f7e9415d510 2023-08-09T14:52:10.904 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:10.901+0000 7f7e99252640 1 -- 172.21.15.43:0/2735594767 <== mon.0 v1:172.21.15.43:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 367236443 0 0) 0x7f7e80003520 con 0x7f7e9415d510 2023-08-09T14:52:10.904 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:10.901+0000 7f7e99252640 1 -- 172.21.15.43:0/2735594767 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f7e9415a0c0 msgr2=0x7f7e940cbb80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-09T14:52:10.904 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:10.901+0000 7f7e99252640 1 --2- 172.21.15.43:0/2735594767 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f7e9415a0c0 0x7f7e940cbb80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T14:52:10.904 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:10.901+0000 7f7e99252640 1 -- 172.21.15.43:0/2735594767 >> v2:172.21.15.203:3300/0 conn(0x7f7e940ce1c0 msgr2=0x7f7e940ce5f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-09T14:52:10.905 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:10.901+0000 7f7e99252640 1 --2- 172.21.15.43:0/2735594767 >> v2:172.21.15.203:3300/0 conn(0x7f7e940ce1c0 0x7f7e940ce5f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T14:52:10.905 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:10.901+0000 7f7e99252640 1 -- 172.21.15.43:0/2735594767 --> v1:172.21.15.43:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7e940cc2d0 con 0x7f7e9415d510 2023-08-09T14:52:10.905 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:10.901+0000 7f7e99252640 1 -- 172.21.15.43:0/2735594767 <== mon.0 v1:172.21.15.43:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1285150740 0 0) 0x7f7e84001690 con 0x7f7e9415d510 2023-08-09T14:52:10.905 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:10.901+0000 7f7e99252640 1 -- 172.21.15.43:0/2735594767 >> v1:172.21.15.43:6789/0 conn(0x7f7e9415d510 legacy=0x7f7e940cb2b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T14:52:10.905 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:10.901+0000 7f7e99252640 1 --2- 172.21.15.43:0/2735594767 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f7e80003970 0x7f7e80003d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T14:52:10.905 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:10.901+0000 7f7e99252640 1 --2- 172.21.15.43:0/2735594767 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f7e800047c0 0x7f7e80004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T14:52:10.905 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:10.901+0000 7f7e99252640 1 --2- 172.21.15.43:0/2735594767 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f7e9415a0c0 0x7f7e80005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T14:52:10.905 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:10.901+0000 7f7e99252640 1 -- 172.21.15.43:0/2735594767 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7e940cc0c0 con 0x7f7e800047c0 2023-08-09T14:52:10.906 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:10.901+0000 7f7e99252640 1 -- 172.21.15.43:0/2735594767 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7e9415a4a0 con 0x7f7e80003970 2023-08-09T14:52:10.906 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:10.901+0000 7f7e99252640 1 -- 172.21.15.43:0/2735594767 --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7e8000b980 con 0x7f7e9415a0c0 2023-08-09T14:52:10.906 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:10.901+0000 7f7e99a53640 1 --2- 172.21.15.43:0/2735594767 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f7e800047c0 0x7f7e80004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T14:52:10.906 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:10.902+0000 7f7e9a254640 1 --2- 172.21.15.43:0/2735594767 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f7e80003970 0x7f7e80003d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T14:52:10.906 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:10.902+0000 7f7e9aa55640 1 --2- 172.21.15.43:0/2735594767 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f7e9415a0c0 0x7f7e80005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T14:52:10.906 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:10.902+0000 7f7e99a53640 1 -- 172.21.15.43:0/2735594767 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f7e9415a0c0 msgr2=0x7f7e80005180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-09T14:52:10.906 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:10.902+0000 7f7e99a53640 1 --2- 172.21.15.43:0/2735594767 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f7e9415a0c0 0x7f7e80005180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T14:52:10.906 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:10.902+0000 7f7e99a53640 1 -- 172.21.15.43:0/2735594767 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f7e80003970 msgr2=0x7f7e80003d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-09T14:52:10.907 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:10.902+0000 7f7e99a53640 1 --2- 172.21.15.43:0/2735594767 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f7e80003970 0x7f7e80003d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T14:52:10.907 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:10.902+0000 7f7e99a53640 1 -- 172.21.15.43:0/2735594767 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7e80003520 con 0x7f7e800047c0 2023-08-09T14:52:10.911 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:10.903+0000 7f7e99a53640 1 --2- 172.21.15.43:0/2735594767 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f7e800047c0 0x7f7e80004c40 secure :-1 s=READY pgs=1942 cs=0 l=1 rev1=1 crypto rx=0x7f7e8400c210 tx=0x7f7e800057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e01f1774b2011d34 server_cookie=0 in_seq=0 out_seq=0 2023-08-09T14:52:10.911 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:10.903+0000 7f7e99252640 1 -- 172.21.15.43:0/2735594767 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7e84005ad0 con 0x7f7e800047c0 2023-08-09T14:52:10.912 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:10.903+0000 7f7e99252640 1 -- 172.21.15.43:0/2735594767 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7e84005c50 con 0x7f7e800047c0 2023-08-09T14:52:10.912 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:10.903+0000 7f7e99252640 1 -- 172.21.15.43:0/2735594767 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7e8402c8b0 con 0x7f7e800047c0 2023-08-09T14:52:10.912 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:10.904+0000 7f7e9c4b8640 1 -- 172.21.15.43:0/2735594767 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f7e800047c0 msgr2=0x7f7e80004c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T14:52:10.913 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:10.904+0000 7f7e9c4b8640 1 --2- 172.21.15.43:0/2735594767 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f7e800047c0 0x7f7e80004c40 secure :-1 s=READY pgs=1942 cs=0 l=1 rev1=1 crypto rx=0x7f7e8400c210 tx=0x7f7e800057f0 comp rx=0 tx=0).stop 2023-08-09T14:52:10.913 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:10.904+0000 7f7e9aa55640 1 -- 172.21.15.43:0/2735594767 reap_dead start 2023-08-09T14:52:10.913 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:10.904+0000 7f7e9c4b8640 1 -- 172.21.15.43:0/2735594767 shutdown_connections 2023-08-09T14:52:10.913 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:10.904+0000 7f7e9c4b8640 1 -- 172.21.15.43:0/2735594767 >> 172.21.15.43:0/2735594767 conn(0x7f7e940aff50 msgr2=0x7f7e940b0360 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-09T14:52:10.914 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:10.904+0000 7f7e9c4b8640 1 -- 172.21.15.43:0/2735594767 shutdown_connections 2023-08-09T14:52:10.914 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:10.904+0000 7f7e9c4b8640 1 -- 172.21.15.43:0/2735594767 wait complete. 2023-08-09T14:52:10.914 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:10.906+0000 7f7e9c4b8640 1 Processor -- start 2023-08-09T14:52:10.915 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:10.906+0000 7f7e9c4b8640 1 -- start start 2023-08-09T14:52:10.915 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:10.907+0000 7f7e9c4b8640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f7e800047c0 0x7f7e941567f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T14:52:10.915 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:10.907+0000 7f7e9c4b8640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f7e940af510 0x7f7e9415e980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T14:52:10.915 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:10.907+0000 7f7e9c4b8640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f7e940ab8d0 0x7f7e940b24d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T14:52:10.915 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:10.907+0000 7f7e9c4b8640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7e9415fa10 con 0x7f7e940af510 2023-08-09T14:52:10.916 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:10.907+0000 7f7e9c4b8640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7e94156d30 con 0x7f7e800047c0 2023-08-09T14:52:10.916 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:10.907+0000 7f7e9c4b8640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7e9415eec0 con 0x7f7e940ab8d0 2023-08-09T14:52:10.916 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:10.907+0000 7f7e99a53640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f7e940af510 0x7f7e9415e980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T14:52:10.916 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:10.908+0000 7f7e99a53640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f7e940af510 0x7f7e9415e980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:34964/0 (socket says 172.21.15.43:34964) 2023-08-09T14:52:10.917 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:10.908+0000 7f7e99a53640 1 -- 172.21.15.43:0/3775337736 learned_addr learned my addr 172.21.15.43:0/3775337736 (peer_addr_for_me v2:172.21.15.43:0/0) 2023-08-09T14:52:10.917 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:10.908+0000 7f7e9a254640 1 --2- 172.21.15.43:0/3775337736 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f7e800047c0 0x7f7e941567f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T14:52:10.917 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:10.908+0000 7f7e9aa55640 1 --2- 172.21.15.43:0/3775337736 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f7e940ab8d0 0x7f7e940b24d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T14:52:10.917 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:10.908+0000 7f7e99a53640 1 -- 172.21.15.43:0/3775337736 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f7e940ab8d0 msgr2=0x7f7e940b24d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T14:52:10.918 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:10.908+0000 7f7e99a53640 1 --2- 172.21.15.43:0/3775337736 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f7e940ab8d0 0x7f7e940b24d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T14:52:10.918 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:10.908+0000 7f7e99a53640 1 -- 172.21.15.43:0/3775337736 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f7e800047c0 msgr2=0x7f7e941567f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T14:52:10.918 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:10.908+0000 7f7e99a53640 1 --2- 172.21.15.43:0/3775337736 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f7e800047c0 0x7f7e941567f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T14:52:10.918 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:10.908+0000 7f7e99a53640 1 -- 172.21.15.43:0/3775337736 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7e9415f170 con 0x7f7e940af510 2023-08-09T14:52:10.919 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:10.908+0000 7f7e9a254640 1 --2- 172.21.15.43:0/3775337736 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f7e800047c0 0x7f7e941567f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-08-09T14:52:10.919 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:10.908+0000 7f7e99a53640 1 --2- 172.21.15.43:0/3775337736 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f7e940af510 0x7f7e9415e980 secure :-1 s=READY pgs=1943 cs=0 l=1 rev1=1 crypto rx=0x7f7e8400c330 tx=0x7f7e84001a20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-09T14:52:10.919 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:10.909+0000 7f7e8affd640 1 -- 172.21.15.43:0/3775337736 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7e8402ccc0 con 0x7f7e940af510 2023-08-09T14:52:10.920 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:10.909+0000 7f7e8affd640 1 -- 172.21.15.43:0/3775337736 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7e8402ce20 con 0x7f7e940af510 2023-08-09T14:52:10.920 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:10.909+0000 7f7e8affd640 1 -- 172.21.15.43:0/3775337736 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7e84007270 con 0x7f7e940af510 2023-08-09T14:52:10.920 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:10.909+0000 7f7e9c4b8640 1 -- 172.21.15.43:0/3775337736 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7e9415f450 con 0x7f7e940af510 2023-08-09T14:52:10.920 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:10.909+0000 7f7e9c4b8640 1 -- 172.21.15.43:0/3775337736 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7e9415ff90 con 0x7f7e940af510 2023-08-09T14:52:10.921 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:10.910+0000 7f7e9c4b8640 1 -- 172.21.15.43:0/3775337736 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7e9415fdf0 con 0x7f7e940af510 2023-08-09T14:52:10.922 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:10.911+0000 7f7e8affd640 1 -- 172.21.15.43:0/3775337736 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 85) v1 ==== 88052+0+0 (secure 0 0 0) 0x7f7e84032070 con 0x7f7e940af510 2023-08-09T14:52:10.922 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:10.912+0000 7f7e8affd640 1 -- 172.21.15.43:0/3775337736 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(174..174 src has 1..174) v4 ==== 7114+0+0 (secure 0 0 0) 0x7f7e8403c040 con 0x7f7e940af510 2023-08-09T14:52:10.922 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:10.916+0000 7f7e8affd640 1 -- 172.21.15.43:0/3775337736 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178066 (secure 0 0 0) 0x7f7e9415fdf0 con 0x7f7e940af510 2023-08-09T14:52:11.086 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:11.084+0000 7f7e8affd640 1 -- 172.21.15.43:0/3775337736 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mgrmap(e 86) v1 ==== 88179+0+0 (secure 0 0 0) 0x7f7e84074a90 con 0x7f7e940af510 2023-08-09T14:52:11.087 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:11.085+0000 7f7e8affd640 1 --2- 172.21.15.43:0/3775337736 >> [v2:172.21.15.203:6832/2959400181,v1:172.21.15.203:6833/2959400181] conn(0x7f7e78035c60 0x7f7e78072470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T14:52:11.091 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:11.089+0000 7f7e9a254640 1 --2- 172.21.15.43:0/3775337736 >> [v2:172.21.15.203:6832/2959400181,v1:172.21.15.203:6833/2959400181] conn(0x7f7e78035c60 0x7f7e78072470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T14:52:11.106 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:11.105+0000 7f7e9a254640 1 --2- 172.21.15.43:0/3775337736 >> [v2:172.21.15.203:6832/2959400181,v1:172.21.15.203:6833/2959400181] conn(0x7f7e78035c60 0x7f7e78072470 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7f7e900098b0 tx=0x7f7e90008040 comp rx=0 tx=0).ready entity=mgr.7223 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-09T14:52:11.115 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:11.113+0000 7f7e9c4b8640 1 -- 172.21.15.43:0/3775337736 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"} v 0) v1 -- 0x7f7e9415fdf0 con 0x7f7e940af510 2023-08-09T14:52:11.116 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:11.114+0000 7f7e8affd640 1 -- 172.21.15.43:0/3775337736 <== mon.0 v2:172.21.15.43:3300/0 8 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"}]=0 v174) v1 ==== 92+0+10 (secure 0 0 0) 0x7f7e9415fdf0 con 0x7f7e940af510 2023-08-09T14:52:11.116 INFO:teuthology.orchestra.run.smithi043.stdout:pg_num: 8 2023-08-09T14:52:11.118 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:11.117+0000 7f7e9c4b8640 1 -- 172.21.15.43:0/3775337736 >> [v2:172.21.15.203:6832/2959400181,v1:172.21.15.203:6833/2959400181] conn(0x7f7e78035c60 msgr2=0x7f7e78072470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T14:52:11.118 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:11.117+0000 7f7e9c4b8640 1 --2- 172.21.15.43:0/3775337736 >> [v2:172.21.15.203:6832/2959400181,v1:172.21.15.203:6833/2959400181] conn(0x7f7e78035c60 0x7f7e78072470 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7f7e900098b0 tx=0x7f7e90008040 comp rx=0 tx=0).stop 2023-08-09T14:52:11.118 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:11.117+0000 7f7e9c4b8640 1 -- 172.21.15.43:0/3775337736 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f7e940af510 msgr2=0x7f7e9415e980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T14:52:11.119 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:11.117+0000 7f7e9c4b8640 1 --2- 172.21.15.43:0/3775337736 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f7e940af510 0x7f7e9415e980 secure :-1 s=READY pgs=1943 cs=0 l=1 rev1=1 crypto rx=0x7f7e8400c330 tx=0x7f7e84001a20 comp rx=0 tx=0).stop 2023-08-09T14:52:11.119 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:11.117+0000 7f7e9c4b8640 1 -- 172.21.15.43:0/3775337736 shutdown_connections 2023-08-09T14:52:11.120 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:11.117+0000 7f7e9c4b8640 1 --2- 172.21.15.43:0/3775337736 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f7e940ab8d0 0x7f7e940b24d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T14:52:11.120 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:11.117+0000 7f7e9c4b8640 1 --2- 172.21.15.43:0/3775337736 >> [v2:172.21.15.203:6832/2959400181,v1:172.21.15.203:6833/2959400181] conn(0x7f7e78035c60 0x7f7e78072470 unknown :-1 s=CLOSED pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T14:52:11.120 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:11.117+0000 7f7e9c4b8640 1 --2- 172.21.15.43:0/3775337736 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f7e940af510 0x7f7e9415e980 unknown :-1 s=CLOSED pgs=1943 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T14:52:11.120 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:11.117+0000 7f7e9c4b8640 1 --2- 172.21.15.43:0/3775337736 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f7e800047c0 0x7f7e941567f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T14:52:11.120 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:11.117+0000 7f7e9c4b8640 1 -- 172.21.15.43:0/3775337736 >> 172.21.15.43:0/3775337736 conn(0x7f7e940aff50 msgr2=0x7f7e9415af80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-09T14:52:11.120 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:11.117+0000 7f7e9c4b8640 1 -- 172.21.15.43:0/3775337736 shutdown_connections 2023-08-09T14:52:11.120 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:11.117+0000 7f7e9c4b8640 1 -- 172.21.15.43:0/3775337736 wait complete. 2023-08-09T14:52:11.139 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-08-09T14:52:11.366 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:11.365+0000 7f758ed00640 1 Processor -- start 2023-08-09T14:52:11.367 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:11.365+0000 7f758ed00640 1 -- start start 2023-08-09T14:52:11.368 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:11.366+0000 7f758ed00640 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7f75880d04b0 0x7f75880cd6d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T14:52:11.368 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:11.366+0000 7f758ed00640 1 -- --> v1:172.21.15.43:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f75880b7bd0 con 0x7f75880b45f0 2023-08-09T14:52:11.369 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:11.366+0000 7f758ed00640 1 -- --> v1:172.21.15.203:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f75880d1c00 con 0x7f758815c550 2023-08-09T14:52:11.369 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:11.366+0000 7f758ed00640 1 -- --> v2:172.21.15.203:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f758815c930 con 0x7f75880d04b0 2023-08-09T14:52:11.369 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:11.367+0000 7f7587fff640 1 --1- >> v1:172.21.15.43:6789/0 conn(0x7f75880b45f0 0x7f75880b59d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.43:6789/0 says I am v1:172.21.15.43:41544/0 (socket says 172.21.15.43:41544) 2023-08-09T14:52:11.370 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:11.367+0000 7f7587fff640 1 -- 172.21.15.43:0/3939369171 learned_addr learned my addr 172.21.15.43:0/3939369171 (peer_addr_for_me v1:172.21.15.43:0/0) 2023-08-09T14:52:11.374 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:11.367+0000 7f758d29d640 1 --2- 172.21.15.43:0/3939369171 >> v2:172.21.15.203:3300/0 conn(0x7f75880d04b0 0x7f75880cd6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T14:52:11.374 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:11.367+0000 7f75877fe640 1 -- 172.21.15.43:0/3939369171 <== mon.0 v1:172.21.15.43:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4069708733 0 0) 0x7f75880b7bd0 con 0x7f75880b45f0 2023-08-09T14:52:11.374 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:11.368+0000 7f75877fe640 1 -- 172.21.15.43:0/3939369171 --> v1:172.21.15.43:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f7570003520 con 0x7f75880b45f0 2023-08-09T14:52:11.374 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:11.368+0000 7f75877fe640 1 -- 172.21.15.43:0/3939369171 <== mon.0 v1:172.21.15.43:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1004401659 0 0) 0x7f7570003520 con 0x7f75880b45f0 2023-08-09T14:52:11.375 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:11.368+0000 7f75877fe640 1 -- 172.21.15.43:0/3939369171 >> v2:172.21.15.203:3300/0 conn(0x7f75880d04b0 msgr2=0x7f75880cd6d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-09T14:52:11.375 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:11.368+0000 7f75877fe640 1 --2- 172.21.15.43:0/3939369171 >> v2:172.21.15.203:3300/0 conn(0x7f75880d04b0 0x7f75880cd6d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T14:52:11.375 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:11.368+0000 7f75877fe640 1 -- 172.21.15.43:0/3939369171 >> v1:172.21.15.203:6789/0 conn(0x7f758815c550 legacy=0x7f75880b3ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T14:52:11.376 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:11.368+0000 7f75877fe640 1 -- 172.21.15.43:0/3939369171 --> v1:172.21.15.43:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f75880b54f0 con 0x7f75880b45f0 2023-08-09T14:52:11.376 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:11.368+0000 7f75877fe640 1 -- 172.21.15.43:0/3939369171 <== mon.0 v1:172.21.15.43:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1285150740 0 0) 0x7f757c001670 con 0x7f75880b45f0 2023-08-09T14:52:11.376 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:11.368+0000 7f75877fe640 1 -- 172.21.15.43:0/3939369171 >> v1:172.21.15.43:6789/0 conn(0x7f75880b45f0 legacy=0x7f75880b59d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T14:52:11.376 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:11.368+0000 7f75877fe640 1 --2- 172.21.15.43:0/3939369171 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f7570003970 0x7f7570003d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T14:52:11.377 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:11.368+0000 7f75877fe640 1 --2- 172.21.15.43:0/3939369171 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f75700047c0 0x7f7570004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T14:52:11.377 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:11.368+0000 7f75877fe640 1 --2- 172.21.15.43:0/3939369171 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f7570005180 0x7f757000b980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T14:52:11.377 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:11.368+0000 7f75877fe640 1 -- 172.21.15.43:0/3939369171 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f758815c930 con 0x7f7570005180 2023-08-09T14:52:11.377 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:11.368+0000 7f75877fe640 1 -- 172.21.15.43:0/3939369171 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f757000c020 con 0x7f7570003970 2023-08-09T14:52:11.378 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:11.368+0000 7f75877fe640 1 -- 172.21.15.43:0/3939369171 --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f757000c1e0 con 0x7f75700047c0 2023-08-09T14:52:11.378 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:11.368+0000 7f758d29d640 1 --2- 172.21.15.43:0/3939369171 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f7570005180 0x7f757000b980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T14:52:11.378 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:11.369+0000 7f758ca9c640 1 --2- 172.21.15.43:0/3939369171 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f7570003970 0x7f7570003d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T14:52:11.379 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:11.369+0000 7f7587fff640 1 --2- 172.21.15.43:0/3939369171 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f75700047c0 0x7f7570004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T14:52:11.379 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:11.369+0000 7f758d29d640 1 -- 172.21.15.43:0/3939369171 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f75700047c0 msgr2=0x7f7570004c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-09T14:52:11.379 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:11.369+0000 7f758d29d640 1 --2- 172.21.15.43:0/3939369171 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f75700047c0 0x7f7570004c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T14:52:11.379 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:11.369+0000 7f758d29d640 1 -- 172.21.15.43:0/3939369171 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f7570003970 msgr2=0x7f7570003d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-09T14:52:11.380 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:11.369+0000 7f758d29d640 1 --2- 172.21.15.43:0/3939369171 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f7570003970 0x7f7570003d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T14:52:11.380 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:11.369+0000 7f758d29d640 1 -- 172.21.15.43:0/3939369171 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7570003520 con 0x7f7570005180 2023-08-09T14:52:11.380 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:11.369+0000 7f7587fff640 1 --2- 172.21.15.43:0/3939369171 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f75700047c0 0x7f7570004c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-08-09T14:52:11.380 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:11.369+0000 7f758ca9c640 1 --2- 172.21.15.43:0/3939369171 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f7570003970 0x7f7570003d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-08-09T14:52:11.381 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:11.370+0000 7f758d29d640 1 -- 172.21.15.43:0/3939369171 reap_dead start 2023-08-09T14:52:11.381 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:11.370+0000 7f758d29d640 1 --2- 172.21.15.43:0/3939369171 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f7570005180 0x7f757000b980 secure :-1 s=READY pgs=1945 cs=0 l=1 rev1=1 crypto rx=0x7f7580009ac0 tx=0x7f75700057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ba56bd9f401e3aef server_cookie=0 in_seq=0 out_seq=0 2023-08-09T14:52:11.381 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:11.370+0000 7f75877fe640 1 -- 172.21.15.43:0/3939369171 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7580004020 con 0x7f7570005180 2023-08-09T14:52:11.382 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:11.370+0000 7f75877fe640 1 -- 172.21.15.43:0/3939369171 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7580004180 con 0x7f7570005180 2023-08-09T14:52:11.382 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:11.370+0000 7f75877fe640 1 -- 172.21.15.43:0/3939369171 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f758002c710 con 0x7f7570005180 2023-08-09T14:52:11.382 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:11.371+0000 7f758ed00640 1 -- 172.21.15.43:0/3939369171 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f7570005180 msgr2=0x7f757000b980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T14:52:11.382 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:11.371+0000 7f758ed00640 1 --2- 172.21.15.43:0/3939369171 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f7570005180 0x7f757000b980 secure :-1 s=READY pgs=1945 cs=0 l=1 rev1=1 crypto rx=0x7f7580009ac0 tx=0x7f75700057f0 comp rx=0 tx=0).stop 2023-08-09T14:52:11.383 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:11.371+0000 7f758ed00640 1 -- 172.21.15.43:0/3939369171 shutdown_connections 2023-08-09T14:52:11.383 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:11.371+0000 7f758ed00640 1 --2- 172.21.15.43:0/3939369171 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f7570005180 0x7f757000b980 unknown :-1 s=CLOSED pgs=1945 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T14:52:11.383 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:11.371+0000 7f758ed00640 1 -- 172.21.15.43:0/3939369171 >> 172.21.15.43:0/3939369171 conn(0x7f7588071520 msgr2=0x7f75880ceee0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-09T14:52:11.383 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:11.371+0000 7f758ed00640 1 -- 172.21.15.43:0/3939369171 shutdown_connections 2023-08-09T14:52:11.384 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:11.371+0000 7f758ed00640 1 -- 172.21.15.43:0/3939369171 wait complete. 2023-08-09T14:52:11.386 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:11.373+0000 7f758ed00640 1 Processor -- start 2023-08-09T14:52:11.386 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:11.373+0000 7f758ed00640 1 -- start start 2023-08-09T14:52:11.387 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:11.374+0000 7f758ed00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f7570005180 0x7f7588149770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T14:52:11.387 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:11.374+0000 7f758ed00640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f75880b5d50 0x7f75880b61d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T14:52:11.387 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:11.374+0000 7f758ed00640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f75881570d0 0x7f7588157550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T14:52:11.387 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:11.374+0000 7f758ed00640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f75880b6710 con 0x7f7570005180 2023-08-09T14:52:11.388 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:11.374+0000 7f758ed00640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f75880c9530 con 0x7f75881570d0 2023-08-09T14:52:11.388 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:11.374+0000 7f758ed00640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7588157d90 con 0x7f75880b5d50 2023-08-09T14:52:11.388 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:11.374+0000 7f758ca9c640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f7570005180 0x7f7588149770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T14:52:11.388 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:11.375+0000 7f758ca9c640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f7570005180 0x7f7588149770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:34986/0 (socket says 172.21.15.43:34986) 2023-08-09T14:52:11.388 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:11.375+0000 7f758ca9c640 1 -- 172.21.15.43:0/3599643574 learned_addr learned my addr 172.21.15.43:0/3599643574 (peer_addr_for_me v2:172.21.15.43:0/0) 2023-08-09T14:52:11.388 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:11.375+0000 7f7587fff640 1 --2- 172.21.15.43:0/3599643574 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f75880b5d50 0x7f75880b61d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T14:52:11.388 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:11.375+0000 7f758d29d640 1 --2- 172.21.15.43:0/3599643574 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f75881570d0 0x7f7588157550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T14:52:11.389 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:11.375+0000 7f758ca9c640 1 -- 172.21.15.43:0/3599643574 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f75880b5d50 msgr2=0x7f75880b61d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T14:52:11.389 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:11.375+0000 7f758ca9c640 1 --2- 172.21.15.43:0/3599643574 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f75880b5d50 0x7f75880b61d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T14:52:11.389 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:11.375+0000 7f758ca9c640 1 -- 172.21.15.43:0/3599643574 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f75881570d0 msgr2=0x7f7588157550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T14:52:11.389 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:11.375+0000 7f758ca9c640 1 --2- 172.21.15.43:0/3599643574 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f75881570d0 0x7f7588157550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T14:52:11.389 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:11.375+0000 7f758ca9c640 1 -- 172.21.15.43:0/3599643574 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7588158070 con 0x7f7570005180 2023-08-09T14:52:11.389 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:11.375+0000 7f7587fff640 1 --2- 172.21.15.43:0/3599643574 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f75880b5d50 0x7f75880b61d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-08-09T14:52:11.389 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:11.375+0000 7f758d29d640 1 --2- 172.21.15.43:0/3599643574 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f75881570d0 0x7f7588157550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-08-09T14:52:11.389 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:11.375+0000 7f758ca9c640 1 --2- 172.21.15.43:0/3599643574 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f7570005180 0x7f7588149770 secure :-1 s=READY pgs=1946 cs=0 l=1 rev1=1 crypto rx=0x7f757800d8b0 tx=0x7f757800dd70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-09T14:52:11.390 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:11.376+0000 7f75857fa640 1 -- 172.21.15.43:0/3599643574 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7578003da0 con 0x7f7570005180 2023-08-09T14:52:11.390 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:11.376+0000 7f75857fa640 1 -- 172.21.15.43:0/3599643574 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f757800b6b0 con 0x7f7570005180 2023-08-09T14:52:11.390 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:11.376+0000 7f758ed00640 1 -- 172.21.15.43:0/3599643574 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7588153b50 con 0x7f7570005180 2023-08-09T14:52:11.390 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:11.376+0000 7f75857fa640 1 -- 172.21.15.43:0/3599643574 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f757800ba80 con 0x7f7570005180 2023-08-09T14:52:11.390 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:11.376+0000 7f758ed00640 1 -- 172.21.15.43:0/3599643574 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7588154110 con 0x7f7570005180 2023-08-09T14:52:11.390 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:11.377+0000 7f758ed00640 1 -- 172.21.15.43:0/3599643574 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f75881542a0 con 0x7f7570005180 2023-08-09T14:52:11.390 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:11.378+0000 7f75857fa640 1 -- 172.21.15.43:0/3599643574 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 86) v1 ==== 88179+0+0 (secure 0 0 0) 0x7f757800bbe0 con 0x7f7570005180 2023-08-09T14:52:11.390 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:11.379+0000 7f75857fa640 1 --2- 172.21.15.43:0/3599643574 >> [v2:172.21.15.203:6832/2959400181,v1:172.21.15.203:6833/2959400181] conn(0x7f7574069ea0 0x7f757406c360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T14:52:11.391 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:11.379+0000 7f75857fa640 1 -- 172.21.15.43:0/3599643574 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(174..174 src has 1..174) v4 ==== 7114+0+0 (secure 0 0 0) 0x7f7578088940 con 0x7f7570005180 2023-08-09T14:52:11.391 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:11.380+0000 7f7587fff640 1 --2- 172.21.15.43:0/3599643574 >> [v2:172.21.15.203:6832/2959400181,v1:172.21.15.203:6833/2959400181] conn(0x7f7574069ea0 0x7f757406c360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T14:52:11.391 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:11.381+0000 7f7587fff640 1 --2- 172.21.15.43:0/3599643574 >> [v2:172.21.15.203:6832/2959400181,v1:172.21.15.203:6833/2959400181] conn(0x7f7574069ea0 0x7f757406c360 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7f7588158c70 tx=0x7f757c001470 comp rx=0 tx=0).ready entity=mgr.7223 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-09T14:52:11.391 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:11.383+0000 7f75857fa640 1 -- 172.21.15.43:0/3599643574 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178066 (secure 0 0 0) 0x7f7578053e80 con 0x7f7570005180 2023-08-09T14:52:11.586 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:11.585+0000 7f758ed00640 1 -- 172.21.15.43:0/3599643574 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f75880739f0 con 0x7f7570005180 2023-08-09T14:52:11.587 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:11.585+0000 7f75857fa640 1 -- 172.21.15.43:0/3599643574 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v174) v1 ==== 93+0+10 (secure 0 0 0) 0x7f75780574a0 con 0x7f7570005180 2023-08-09T14:52:11.588 INFO:teuthology.orchestra.run.smithi043.stdout:pg_num: 1 2023-08-09T14:52:11.590 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:11.588+0000 7f758ed00640 1 -- 172.21.15.43:0/3599643574 >> [v2:172.21.15.203:6832/2959400181,v1:172.21.15.203:6833/2959400181] conn(0x7f7574069ea0 msgr2=0x7f757406c360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T14:52:11.590 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:11.588+0000 7f758ed00640 1 --2- 172.21.15.43:0/3599643574 >> [v2:172.21.15.203:6832/2959400181,v1:172.21.15.203:6833/2959400181] conn(0x7f7574069ea0 0x7f757406c360 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7f7588158c70 tx=0x7f757c001470 comp rx=0 tx=0).stop 2023-08-09T14:52:11.590 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:11.588+0000 7f758ed00640 1 -- 172.21.15.43:0/3599643574 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f7570005180 msgr2=0x7f7588149770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T14:52:11.591 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:11.588+0000 7f758ed00640 1 --2- 172.21.15.43:0/3599643574 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f7570005180 0x7f7588149770 secure :-1 s=READY pgs=1946 cs=0 l=1 rev1=1 crypto rx=0x7f757800d8b0 tx=0x7f757800dd70 comp rx=0 tx=0).stop 2023-08-09T14:52:11.591 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:11.588+0000 7f758ed00640 1 -- 172.21.15.43:0/3599643574 shutdown_connections 2023-08-09T14:52:11.591 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:11.588+0000 7f758ed00640 1 --2- 172.21.15.43:0/3599643574 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f75881570d0 0x7f7588157550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T14:52:11.592 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:11.588+0000 7f758ed00640 1 --2- 172.21.15.43:0/3599643574 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f75880b5d50 0x7f75880b61d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T14:52:11.592 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:11.588+0000 7f758ed00640 1 --2- 172.21.15.43:0/3599643574 >> [v2:172.21.15.203:6832/2959400181,v1:172.21.15.203:6833/2959400181] conn(0x7f7574069ea0 0x7f757406c360 unknown :-1 s=CLOSED pgs=22 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T14:52:11.592 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:11.588+0000 7f758ed00640 1 --2- 172.21.15.43:0/3599643574 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f7570005180 0x7f7588149770 unknown :-1 s=CLOSED pgs=1946 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T14:52:11.592 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:11.588+0000 7f758ed00640 1 -- 172.21.15.43:0/3599643574 >> 172.21.15.43:0/3599643574 conn(0x7f7588071520 msgr2=0x7f75880ccc70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-09T14:52:11.593 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:11.588+0000 7f758ed00640 1 -- 172.21.15.43:0/3599643574 shutdown_connections 2023-08-09T14:52:11.593 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:11.588+0000 7f758ed00640 1 -- 172.21.15.43:0/3599643574 wait complete. 2023-08-09T14:52:11.609 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-08-09T14:52:11.837 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:11.836+0000 7fe65759e640 1 Processor -- start 2023-08-09T14:52:11.837 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:11.836+0000 7fe65759e640 1 -- start start 2023-08-09T14:52:11.839 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:11.837+0000 7fe65759e640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fe6580c8b80 0x7fe65815b0f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T14:52:11.839 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:11.837+0000 7fe65759e640 1 -- --> v1:172.21.15.43:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe65806b480 con 0x7fe658157ce0 2023-08-09T14:52:11.839 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:11.837+0000 7fe65759e640 1 -- --> v1:172.21.15.203:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe658071580 con 0x7fe65806f2c0 2023-08-09T14:52:11.840 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:11.837+0000 7fe65759e640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe6581580c0 con 0x7fe6580c8b80 2023-08-09T14:52:11.840 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:11.838+0000 7fe65659c640 1 --1- >> v1:172.21.15.43:6789/0 conn(0x7fe658157ce0 0x7fe65806bc40 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.43:6789/0 says I am v1:172.21.15.43:41548/0 (socket says 172.21.15.43:41548) 2023-08-09T14:52:11.840 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:11.838+0000 7fe65659c640 1 -- 172.21.15.43:0/2586804784 learned_addr learned my addr 172.21.15.43:0/2586804784 (peer_addr_for_me v1:172.21.15.43:0/0) 2023-08-09T14:52:11.841 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:11.838+0000 7fe655d9b640 1 -- 172.21.15.43:0/2586804784 <== mon.0 v1:172.21.15.43:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2930385955 0 0) 0x7fe65806b480 con 0x7fe658157ce0 2023-08-09T14:52:11.841 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:11.838+0000 7fe655d9b640 1 -- 172.21.15.43:0/2586804784 --> v1:172.21.15.43:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fe644003520 con 0x7fe658157ce0 2023-08-09T14:52:11.842 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:11.838+0000 7fe656d9d640 1 --2- 172.21.15.43:0/2586804784 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fe6580c8b80 0x7fe65815b0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T14:52:11.842 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:11.838+0000 7fe655d9b640 1 -- 172.21.15.43:0/2586804784 <== mon.0 v1:172.21.15.43:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3319539710 0 0) 0x7fe644003520 con 0x7fe658157ce0 2023-08-09T14:52:11.842 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:11.838+0000 7fe655d9b640 1 -- 172.21.15.43:0/2586804784 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fe6580c8b80 msgr2=0x7fe65815b0f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-09T14:52:11.842 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:11.838+0000 7fe655d9b640 1 --2- 172.21.15.43:0/2586804784 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fe6580c8b80 0x7fe65815b0f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T14:52:11.842 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:11.838+0000 7fe655d9b640 1 -- 172.21.15.43:0/2586804784 >> v1:172.21.15.203:6789/0 conn(0x7fe65806f2c0 legacy=0x7fe6580c8470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-09T14:52:11.842 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:11.838+0000 7fe655d9b640 1 -- 172.21.15.43:0/2586804784 --> v1:172.21.15.43:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe65806c350 con 0x7fe658157ce0 2023-08-09T14:52:11.842 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:11.839+0000 7fe655d9b640 1 -- 172.21.15.43:0/2586804784 <== mon.0 v1:172.21.15.43:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1285150740 0 0) 0x7fe648001690 con 0x7fe658157ce0 2023-08-09T14:52:11.843 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:11.839+0000 7fe655d9b640 1 -- 172.21.15.43:0/2586804784 >> v1:172.21.15.43:6789/0 conn(0x7fe658157ce0 legacy=0x7fe65806bc40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T14:52:11.843 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:11.839+0000 7fe655d9b640 1 --2- 172.21.15.43:0/2586804784 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fe6440039a0 0x7fe644003da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T14:52:11.843 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:11.839+0000 7fe655d9b640 1 --2- 172.21.15.43:0/2586804784 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fe6440047c0 0x7fe644004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T14:52:11.843 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:11.839+0000 7fe655d9b640 1 --2- 172.21.15.43:0/2586804784 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fe6580c8b80 0x7fe644005250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T14:52:11.843 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:11.839+0000 7fe655d9b640 1 -- 172.21.15.43:0/2586804784 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe6581580c0 con 0x7fe6440047c0 2023-08-09T14:52:11.843 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:11.839+0000 7fe655d9b640 1 -- 172.21.15.43:0/2586804784 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe64400bb70 con 0x7fe6440039a0 2023-08-09T14:52:11.843 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:11.839+0000 7fe655d9b640 1 -- 172.21.15.43:0/2586804784 --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe64400bd30 con 0x7fe6580c8b80 2023-08-09T14:52:11.843 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:11.839+0000 7fe64ffff640 1 --2- 172.21.15.43:0/2586804784 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fe6440047c0 0x7fe644004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T14:52:11.844 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:11.839+0000 7fe65659c640 1 --2- 172.21.15.43:0/2586804784 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fe6440039a0 0x7fe644003da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T14:52:11.844 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:11.839+0000 7fe656d9d640 1 --2- 172.21.15.43:0/2586804784 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fe6580c8b80 0x7fe644005250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T14:52:11.844 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:11.839+0000 7fe64ffff640 1 -- 172.21.15.43:0/2586804784 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fe6580c8b80 msgr2=0x7fe644005250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-09T14:52:11.844 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:11.839+0000 7fe64ffff640 1 --2- 172.21.15.43:0/2586804784 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fe6580c8b80 0x7fe644005250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T14:52:11.844 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:11.840+0000 7fe64ffff640 1 -- 172.21.15.43:0/2586804784 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fe6440039a0 msgr2=0x7fe644003da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-09T14:52:11.844 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:11.840+0000 7fe64ffff640 1 --2- 172.21.15.43:0/2586804784 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fe6440039a0 0x7fe644003da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T14:52:11.844 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:11.840+0000 7fe64ffff640 1 -- 172.21.15.43:0/2586804784 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe644003520 con 0x7fe6440047c0 2023-08-09T14:52:11.845 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:11.840+0000 7fe65659c640 1 --2- 172.21.15.43:0/2586804784 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fe6440039a0 0x7fe644003da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-08-09T14:52:11.845 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:11.841+0000 7fe64ffff640 1 --2- 172.21.15.43:0/2586804784 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fe6440047c0 0x7fe644004c40 secure :-1 s=READY pgs=1948 cs=0 l=1 rev1=1 crypto rx=0x7fe640003db0 tx=0x7fe6440057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=571ca16454de7481 server_cookie=0 in_seq=0 out_seq=0 2023-08-09T14:52:11.846 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:11.841+0000 7fe655d9b640 1 -- 172.21.15.43:0/2586804784 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe640032600 con 0x7fe6440047c0 2023-08-09T14:52:11.846 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:11.841+0000 7fe655d9b640 1 -- 172.21.15.43:0/2586804784 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe64002e050 con 0x7fe6440047c0 2023-08-09T14:52:11.846 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:11.841+0000 7fe655d9b640 1 -- 172.21.15.43:0/2586804784 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe6400329a0 con 0x7fe6440047c0 2023-08-09T14:52:11.846 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:11.842+0000 7fe65759e640 1 -- 172.21.15.43:0/2586804784 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fe6440047c0 msgr2=0x7fe644004c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T14:52:11.846 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:11.842+0000 7fe65759e640 1 --2- 172.21.15.43:0/2586804784 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fe6440047c0 0x7fe644004c40 secure :-1 s=READY pgs=1948 cs=0 l=1 rev1=1 crypto rx=0x7fe640003db0 tx=0x7fe6440057f0 comp rx=0 tx=0).stop 2023-08-09T14:52:11.846 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:11.842+0000 7fe656d9d640 1 -- 172.21.15.43:0/2586804784 reap_dead start 2023-08-09T14:52:11.847 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:11.842+0000 7fe65759e640 1 -- 172.21.15.43:0/2586804784 shutdown_connections 2023-08-09T14:52:11.847 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:11.842+0000 7fe65759e640 1 -- 172.21.15.43:0/2586804784 >> 172.21.15.43:0/2586804784 conn(0x7fe6580af9f0 msgr2=0x7fe6580afdd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-09T14:52:11.847 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:11.842+0000 7fe65759e640 1 -- 172.21.15.43:0/2586804784 shutdown_connections 2023-08-09T14:52:11.847 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:11.842+0000 7fe65759e640 1 -- 172.21.15.43:0/2586804784 wait complete. 2023-08-09T14:52:11.847 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:11.845+0000 7fe65759e640 1 Processor -- start 2023-08-09T14:52:11.847 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:11.845+0000 7fe65759e640 1 -- start start 2023-08-09T14:52:11.849 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:11.845+0000 7fe65759e640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fe6440047c0 0x7fe6580b10c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T14:52:11.849 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:11.845+0000 7fe65759e640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fe6581f5d70 0x7fe658155190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T14:52:11.849 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:11.845+0000 7fe65759e640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fe658155c50 0x7fe6581560e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T14:52:11.849 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:11.845+0000 7fe65759e640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe658156620 con 0x7fe658155c50 2023-08-09T14:52:11.849 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:11.845+0000 7fe65759e640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe6580c7260 con 0x7fe6440047c0 2023-08-09T14:52:11.849 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:11.845+0000 7fe65759e640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe6581f6270 con 0x7fe6581f5d70 2023-08-09T14:52:11.850 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:11.845+0000 7fe656d9d640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fe658155c50 0x7fe6581560e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T14:52:11.850 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:11.845+0000 7fe656d9d640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fe658155c50 0x7fe6581560e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:35014/0 (socket says 172.21.15.43:35014) 2023-08-09T14:52:11.850 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:11.845+0000 7fe656d9d640 1 -- 172.21.15.43:0/1533131769 learned_addr learned my addr 172.21.15.43:0/1533131769 (peer_addr_for_me v2:172.21.15.43:0/0) 2023-08-09T14:52:11.850 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:11.846+0000 7fe65659c640 1 --2- 172.21.15.43:0/1533131769 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fe6440047c0 0x7fe6580b10c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T14:52:11.850 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:11.846+0000 7fe656d9d640 1 -- 172.21.15.43:0/1533131769 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fe6581f5d70 msgr2=0x7fe658155190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T14:52:11.850 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:11.846+0000 7fe64ffff640 1 --2- 172.21.15.43:0/1533131769 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fe6581f5d70 0x7fe658155190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T14:52:11.850 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:11.846+0000 7fe656d9d640 1 --2- 172.21.15.43:0/1533131769 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fe6581f5d70 0x7fe658155190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T14:52:11.850 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:11.846+0000 7fe656d9d640 1 -- 172.21.15.43:0/1533131769 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fe6440047c0 msgr2=0x7fe6580b10c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T14:52:11.851 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:11.846+0000 7fe656d9d640 1 --2- 172.21.15.43:0/1533131769 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fe6440047c0 0x7fe6580b10c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T14:52:11.851 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:11.846+0000 7fe656d9d640 1 -- 172.21.15.43:0/1533131769 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe6581f6550 con 0x7fe658155c50 2023-08-09T14:52:11.851 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:11.846+0000 7fe65659c640 1 --2- 172.21.15.43:0/1533131769 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fe6440047c0 0x7fe6580b10c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-08-09T14:52:11.851 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:11.846+0000 7fe64ffff640 1 --2- 172.21.15.43:0/1533131769 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fe6581f5d70 0x7fe658155190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-08-09T14:52:11.851 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:11.846+0000 7fe656d9d640 1 --2- 172.21.15.43:0/1533131769 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fe658155c50 0x7fe6581560e0 secure :-1 s=READY pgs=1949 cs=0 l=1 rev1=1 crypto rx=0x7fe6580cb950 tx=0x7fe650009d50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-09T14:52:11.851 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:11.846+0000 7fe64effd640 1 -- 172.21.15.43:0/1533131769 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe650007cf0 con 0x7fe658155c50 2023-08-09T14:52:11.851 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:11.846+0000 7fe64effd640 1 -- 172.21.15.43:0/1533131769 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe65000f050 con 0x7fe658155c50 2023-08-09T14:52:11.852 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:11.846+0000 7fe64effd640 1 -- 172.21.15.43:0/1533131769 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe650014650 con 0x7fe658155c50 2023-08-09T14:52:11.852 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:11.846+0000 7fe65759e640 1 -- 172.21.15.43:0/1533131769 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe6581f6830 con 0x7fe658155c50 2023-08-09T14:52:11.852 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:11.847+0000 7fe65759e640 1 -- 172.21.15.43:0/1533131769 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe65815dc00 con 0x7fe658155c50 2023-08-09T14:52:11.852 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:11.848+0000 7fe65759e640 1 -- 172.21.15.43:0/1533131769 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe6580c7420 con 0x7fe658155c50 2023-08-09T14:52:11.853 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:11.849+0000 7fe64effd640 1 -- 172.21.15.43:0/1533131769 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 86) v1 ==== 88179+0+0 (secure 0 0 0) 0x7fe6500147b0 con 0x7fe658155c50 2023-08-09T14:52:11.853 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:11.849+0000 7fe64effd640 1 --2- 172.21.15.43:0/1533131769 >> [v2:172.21.15.203:6832/2959400181,v1:172.21.15.203:6833/2959400181] conn(0x7fe638069ea0 0x7fe63806c360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T14:52:11.853 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:11.850+0000 7fe64effd640 1 -- 172.21.15.43:0/1533131769 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(174..174 src has 1..174) v4 ==== 7114+0+0 (secure 0 0 0) 0x7fe650088a70 con 0x7fe658155c50 2023-08-09T14:52:11.853 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:11.850+0000 7fe65659c640 1 --2- 172.21.15.43:0/1533131769 >> [v2:172.21.15.203:6832/2959400181,v1:172.21.15.203:6833/2959400181] conn(0x7fe638069ea0 0x7fe63806c360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T14:52:11.853 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:11.852+0000 7fe65659c640 1 --2- 172.21.15.43:0/1533131769 >> [v2:172.21.15.203:6832/2959400181,v1:172.21.15.203:6833/2959400181] conn(0x7fe638069ea0 0x7fe63806c360 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7fe6480017c0 tx=0x7fe648009040 comp rx=0 tx=0).ready entity=mgr.7223 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-09T14:52:11.856 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:11.854+0000 7fe64effd640 1 -- 172.21.15.43:0/1533131769 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178066 (secure 0 0 0) 0x7fe650054080 con 0x7fe658155c50 2023-08-09T14:52:12.049 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:12.047+0000 7fe65759e640 1 -- 172.21.15.43:0/1533131769 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7fe65804e390 con 0x7fe658155c50 2023-08-09T14:52:12.050 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:12.048+0000 7fe64effd640 1 -- 172.21.15.43:0/1533131769 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v174) v1 ==== 104+0+11 (secure 0 0 0) 0x7fe6500576a0 con 0x7fe658155c50 2023-08-09T14:52:12.050 INFO:teuthology.orchestra.run.smithi043.stdout:pg_num: 32 2023-08-09T14:52:12.052 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:12.051+0000 7fe65759e640 1 -- 172.21.15.43:0/1533131769 >> [v2:172.21.15.203:6832/2959400181,v1:172.21.15.203:6833/2959400181] conn(0x7fe638069ea0 msgr2=0x7fe63806c360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T14:52:12.053 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:12.051+0000 7fe65759e640 1 --2- 172.21.15.43:0/1533131769 >> [v2:172.21.15.203:6832/2959400181,v1:172.21.15.203:6833/2959400181] conn(0x7fe638069ea0 0x7fe63806c360 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7fe6480017c0 tx=0x7fe648009040 comp rx=0 tx=0).stop 2023-08-09T14:52:12.053 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:12.051+0000 7fe65759e640 1 -- 172.21.15.43:0/1533131769 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fe658155c50 msgr2=0x7fe6581560e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T14:52:12.053 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:12.051+0000 7fe65759e640 1 --2- 172.21.15.43:0/1533131769 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fe658155c50 0x7fe6581560e0 secure :-1 s=READY pgs=1949 cs=0 l=1 rev1=1 crypto rx=0x7fe6580cb950 tx=0x7fe650009d50 comp rx=0 tx=0).stop 2023-08-09T14:52:12.053 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:12.051+0000 7fe65759e640 1 -- 172.21.15.43:0/1533131769 shutdown_connections 2023-08-09T14:52:12.053 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:12.051+0000 7fe65759e640 1 --2- 172.21.15.43:0/1533131769 >> [v2:172.21.15.203:6832/2959400181,v1:172.21.15.203:6833/2959400181] conn(0x7fe638069ea0 0x7fe63806c360 unknown :-1 s=CLOSED pgs=23 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T14:52:12.053 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:12.051+0000 7fe65759e640 1 --2- 172.21.15.43:0/1533131769 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fe658155c50 0x7fe6581560e0 unknown :-1 s=CLOSED pgs=1949 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T14:52:12.054 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:12.051+0000 7fe65759e640 1 --2- 172.21.15.43:0/1533131769 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fe6581f5d70 0x7fe658155190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T14:52:12.054 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:12.051+0000 7fe65759e640 1 --2- 172.21.15.43:0/1533131769 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fe6440047c0 0x7fe6580b10c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T14:52:12.054 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:12.051+0000 7fe65759e640 1 -- 172.21.15.43:0/1533131769 >> 172.21.15.43:0/1533131769 conn(0x7fe6580af9f0 msgr2=0x7fe6580c3d60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-09T14:52:12.054 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:12.051+0000 7fe65759e640 1 -- 172.21.15.43:0/1533131769 shutdown_connections 2023-08-09T14:52:12.054 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:12.051+0000 7fe65759e640 1 -- 172.21.15.43:0/1533131769 wait complete. 2023-08-09T14:52:12.072 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-08-09T14:52:12.302 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:12.301+0000 7f102af54640 1 Processor -- start 2023-08-09T14:52:12.303 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:12.301+0000 7f102af54640 1 -- start start 2023-08-09T14:52:12.304 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:12.303+0000 7f102af54640 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7f10240b45f0 0x7f10240cd6d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T14:52:12.304 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:12.303+0000 7f102af54640 1 -- --> v1:172.21.15.43:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f10240b7bd0 con 0x7f102415c550 2023-08-09T14:52:12.305 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:12.303+0000 7f102af54640 1 -- --> v1:172.21.15.203:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f10240d1c00 con 0x7f10240d04b0 2023-08-09T14:52:12.305 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:12.303+0000 7f102af54640 1 -- --> v2:172.21.15.203:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f102415c930 con 0x7f10240b45f0 2023-08-09T14:52:12.305 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:12.303+0000 7f1028cf0640 1 --1- >> v1:172.21.15.43:6789/0 conn(0x7f102415c550 0x7f10240b3ee0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.43:6789/0 says I am v1:172.21.15.43:41550/0 (socket says 172.21.15.43:41550) 2023-08-09T14:52:12.306 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:12.303+0000 7f1028cf0640 1 -- 172.21.15.43:0/1046412485 learned_addr learned my addr 172.21.15.43:0/1046412485 (peer_addr_for_me v1:172.21.15.43:0/0) 2023-08-09T14:52:12.310 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:12.303+0000 7f1023fff640 1 --2- 172.21.15.43:0/1046412485 >> v2:172.21.15.203:3300/0 conn(0x7f10240b45f0 0x7f10240cd6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T14:52:12.311 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:12.303+0000 7f10237fe640 1 -- 172.21.15.43:0/1046412485 <== mon.0 v1:172.21.15.43:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 708800727 0 0) 0x7f10240b7bd0 con 0x7f102415c550 2023-08-09T14:52:12.311 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:12.303+0000 7f10237fe640 1 -- 172.21.15.43:0/1046412485 --> v1:172.21.15.43:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f1014003520 con 0x7f102415c550 2023-08-09T14:52:12.311 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:12.304+0000 7f10237fe640 1 -- 172.21.15.43:0/1046412485 <== mon.0 v1:172.21.15.43:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 951790234 0 0) 0x7f1014003520 con 0x7f102415c550 2023-08-09T14:52:12.311 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:12.304+0000 7f10237fe640 1 -- 172.21.15.43:0/1046412485 >> v2:172.21.15.203:3300/0 conn(0x7f10240b45f0 msgr2=0x7f10240cd6d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-09T14:52:12.312 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:12.304+0000 7f10237fe640 1 --2- 172.21.15.43:0/1046412485 >> v2:172.21.15.203:3300/0 conn(0x7f10240b45f0 0x7f10240cd6d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T14:52:12.312 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:12.304+0000 7f10237fe640 1 -- 172.21.15.43:0/1046412485 >> v1:172.21.15.203:6789/0 conn(0x7f10240d04b0 legacy=0x7f10240b59d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T14:52:12.312 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:12.304+0000 7f10237fe640 1 -- 172.21.15.43:0/1046412485 --> v1:172.21.15.43:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f10240b54f0 con 0x7f102415c550 2023-08-09T14:52:12.313 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:12.304+0000 7f10237fe640 1 -- 172.21.15.43:0/1046412485 <== mon.0 v1:172.21.15.43:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1285150740 0 0) 0x7f1018001670 con 0x7f102415c550 2023-08-09T14:52:12.313 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:12.304+0000 7f10237fe640 1 -- 172.21.15.43:0/1046412485 >> v1:172.21.15.43:6789/0 conn(0x7f102415c550 legacy=0x7f10240b3ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T14:52:12.313 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:12.304+0000 7f10237fe640 1 --2- 172.21.15.43:0/1046412485 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f1014003970 0x7f1014003d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T14:52:12.313 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:12.304+0000 7f10237fe640 1 --2- 172.21.15.43:0/1046412485 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f10140047c0 0x7f1014004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T14:52:12.314 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:12.304+0000 7f10237fe640 1 --2- 172.21.15.43:0/1046412485 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f1014005180 0x7f101400b980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T14:52:12.314 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:12.304+0000 7f10237fe640 1 -- 172.21.15.43:0/1046412485 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f102415c930 con 0x7f1014003970 2023-08-09T14:52:12.314 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:12.304+0000 7f10237fe640 1 -- 172.21.15.43:0/1046412485 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f101400c020 con 0x7f1014005180 2023-08-09T14:52:12.315 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:12.304+0000 7f10237fe640 1 -- 172.21.15.43:0/1046412485 --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f101400c1e0 con 0x7f10140047c0 2023-08-09T14:52:12.315 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:12.304+0000 7f1028cf0640 1 --2- 172.21.15.43:0/1046412485 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f1014003970 0x7f1014003d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T14:52:12.315 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:12.305+0000 7f1023fff640 1 --2- 172.21.15.43:0/1046412485 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f10140047c0 0x7f1014004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T14:52:12.315 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:12.305+0000 7f1028cf0640 1 -- 172.21.15.43:0/1046412485 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f10140047c0 msgr2=0x7f1014004c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-09T14:52:12.316 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:12.305+0000 7f10294f1640 1 --2- 172.21.15.43:0/1046412485 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f1014005180 0x7f101400b980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T14:52:12.316 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:12.305+0000 7f1028cf0640 1 --2- 172.21.15.43:0/1046412485 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f10140047c0 0x7f1014004c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T14:52:12.316 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:12.305+0000 7f1028cf0640 1 -- 172.21.15.43:0/1046412485 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f1014005180 msgr2=0x7f101400b980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-09T14:52:12.316 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:12.305+0000 7f1028cf0640 1 --2- 172.21.15.43:0/1046412485 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f1014005180 0x7f101400b980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T14:52:12.316 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:12.305+0000 7f1028cf0640 1 -- 172.21.15.43:0/1046412485 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1014003520 con 0x7f1014003970 2023-08-09T14:52:12.316 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:12.305+0000 7f1023fff640 1 --2- 172.21.15.43:0/1046412485 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f10140047c0 0x7f1014004c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-08-09T14:52:12.317 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:12.305+0000 7f10294f1640 1 --2- 172.21.15.43:0/1046412485 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f1014005180 0x7f101400b980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-08-09T14:52:12.317 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:12.305+0000 7f10294f1640 1 -- 172.21.15.43:0/1046412485 reap_dead start 2023-08-09T14:52:12.317 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:12.306+0000 7f1028cf0640 1 --2- 172.21.15.43:0/1046412485 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f1014003970 0x7f1014003d90 secure :-1 s=READY pgs=1951 cs=0 l=1 rev1=1 crypto rx=0x7f101800c190 tx=0x7f10140057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b97d9cc6e58bf1fb server_cookie=0 in_seq=0 out_seq=0 2023-08-09T14:52:12.317 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:12.306+0000 7f10237fe640 1 -- 172.21.15.43:0/1046412485 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1018005ad0 con 0x7f1014003970 2023-08-09T14:52:12.317 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:12.306+0000 7f10237fe640 1 -- 172.21.15.43:0/1046412485 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1018005c30 con 0x7f1014003970 2023-08-09T14:52:12.318 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:12.306+0000 7f10237fe640 1 -- 172.21.15.43:0/1046412485 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f101802caf0 con 0x7f1014003970 2023-08-09T14:52:12.318 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:12.307+0000 7f102af54640 1 -- 172.21.15.43:0/1046412485 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f1014003970 msgr2=0x7f1014003d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T14:52:12.318 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:12.307+0000 7f102af54640 1 --2- 172.21.15.43:0/1046412485 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f1014003970 0x7f1014003d90 secure :-1 s=READY pgs=1951 cs=0 l=1 rev1=1 crypto rx=0x7f101800c190 tx=0x7f10140057f0 comp rx=0 tx=0).stop 2023-08-09T14:52:12.318 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:12.307+0000 7f102af54640 1 -- 172.21.15.43:0/1046412485 shutdown_connections 2023-08-09T14:52:12.318 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:12.307+0000 7f102af54640 1 --2- 172.21.15.43:0/1046412485 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f1014003970 0x7f1014003d90 unknown :-1 s=CLOSED pgs=1951 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T14:52:12.318 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:12.307+0000 7f102af54640 1 -- 172.21.15.43:0/1046412485 >> 172.21.15.43:0/1046412485 conn(0x7f1024071520 msgr2=0x7f10240ceee0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-09T14:52:12.319 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:12.307+0000 7f102af54640 1 -- 172.21.15.43:0/1046412485 shutdown_connections 2023-08-09T14:52:12.319 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:12.307+0000 7f102af54640 1 -- 172.21.15.43:0/1046412485 wait complete. 2023-08-09T14:52:12.319 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:12.309+0000 7f102af54640 1 Processor -- start 2023-08-09T14:52:12.319 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:12.309+0000 7f102af54640 1 -- start start 2023-08-09T14:52:12.322 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:12.310+0000 7f102af54640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f1014003970 0x7f10240a9940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T14:52:12.322 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:12.310+0000 7f102af54640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f1024149f90 0x7f102414a410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T14:52:12.322 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:12.310+0000 7f102af54640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f10240cc590 0x7f10240cca10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T14:52:12.323 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:12.310+0000 7f102af54640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1024150f30 con 0x7f1024149f90 2023-08-09T14:52:12.323 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:12.310+0000 7f102af54640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f10240ccf50 con 0x7f1014003970 2023-08-09T14:52:12.323 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:12.310+0000 7f102af54640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f102414b7e0 con 0x7f10240cc590 2023-08-09T14:52:12.323 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:12.310+0000 7f1023fff640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f1024149f90 0x7f102414a410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T14:52:12.323 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:12.310+0000 7f1023fff640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f1024149f90 0x7f102414a410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:35018/0 (socket says 172.21.15.43:35018) 2023-08-09T14:52:12.324 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:12.310+0000 7f1023fff640 1 -- 172.21.15.43:0/2837421080 learned_addr learned my addr 172.21.15.43:0/2837421080 (peer_addr_for_me v2:172.21.15.43:0/0) 2023-08-09T14:52:12.324 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:12.311+0000 7f1028cf0640 1 --2- 172.21.15.43:0/2837421080 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f1014003970 0x7f10240a9940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T14:52:12.324 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:12.311+0000 7f10294f1640 1 --2- 172.21.15.43:0/2837421080 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f10240cc590 0x7f10240cca10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T14:52:12.324 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:12.311+0000 7f1023fff640 1 -- 172.21.15.43:0/2837421080 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f10240cc590 msgr2=0x7f10240cca10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T14:52:12.324 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:12.311+0000 7f1023fff640 1 --2- 172.21.15.43:0/2837421080 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f10240cc590 0x7f10240cca10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T14:52:12.325 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:12.311+0000 7f1023fff640 1 -- 172.21.15.43:0/2837421080 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f1014003970 msgr2=0x7f10240a9940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T14:52:12.325 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:12.311+0000 7f1023fff640 1 --2- 172.21.15.43:0/2837421080 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f1014003970 0x7f10240a9940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T14:52:12.325 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:12.311+0000 7f1023fff640 1 -- 172.21.15.43:0/2837421080 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f10241f5d40 con 0x7f1024149f90 2023-08-09T14:52:12.325 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:12.311+0000 7f1028cf0640 1 --2- 172.21.15.43:0/2837421080 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f1014003970 0x7f10240a9940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-08-09T14:52:12.325 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:12.311+0000 7f10294f1640 1 --2- 172.21.15.43:0/2837421080 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f10240cc590 0x7f10240cca10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-08-09T14:52:12.326 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:12.311+0000 7f1023fff640 1 --2- 172.21.15.43:0/2837421080 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f1024149f90 0x7f102414a410 secure :-1 s=READY pgs=1952 cs=0 l=1 rev1=1 crypto rx=0x7f10100107c0 tx=0x7f1010010c80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-09T14:52:12.326 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:12.311+0000 7f10217fa640 1 -- 172.21.15.43:0/2837421080 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f101000eb60 con 0x7f1024149f90 2023-08-09T14:52:12.326 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:12.312+0000 7f10217fa640 1 -- 172.21.15.43:0/2837421080 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1010010ed0 con 0x7f1024149f90 2023-08-09T14:52:12.326 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:12.312+0000 7f10217fa640 1 -- 172.21.15.43:0/2837421080 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f101001c400 con 0x7f1024149f90 2023-08-09T14:52:12.327 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:12.312+0000 7f102af54640 1 -- 172.21.15.43:0/2837421080 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f10241f6020 con 0x7f1024149f90 2023-08-09T14:52:12.327 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:12.312+0000 7f102af54640 1 -- 172.21.15.43:0/2837421080 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f10240a9140 con 0x7f1024149f90 2023-08-09T14:52:12.327 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:12.313+0000 7f102af54640 1 -- 172.21.15.43:0/2837421080 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f10240739f0 con 0x7f1024149f90 2023-08-09T14:52:12.327 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:12.314+0000 7f10217fa640 1 -- 172.21.15.43:0/2837421080 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 87) v1 ==== 88387+0+0 (secure 0 0 0) 0x7f101001c560 con 0x7f1024149f90 2023-08-09T14:52:12.328 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:12.315+0000 7f10217fa640 1 --2- 172.21.15.43:0/2837421080 >> [v2:172.21.15.203:6832/2959400181,v1:172.21.15.203:6833/2959400181] conn(0x7f100806a110 0x7f100806c5d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T14:52:12.328 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:12.315+0000 7f10217fa640 1 -- 172.21.15.43:0/2837421080 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(174..174 src has 1..174) v4 ==== 7114+0+0 (secure 0 0 0) 0x7f1010088e00 con 0x7f1024149f90 2023-08-09T14:52:12.328 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:12.316+0000 7f1028cf0640 1 --2- 172.21.15.43:0/2837421080 >> [v2:172.21.15.203:6832/2959400181,v1:172.21.15.203:6833/2959400181] conn(0x7f100806a110 0x7f100806c5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T14:52:12.328 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:12.317+0000 7f1028cf0640 1 --2- 172.21.15.43:0/2837421080 >> [v2:172.21.15.203:6832/2959400181,v1:172.21.15.203:6833/2959400181] conn(0x7f100806a110 0x7f100806c5d0 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7f1018034000 tx=0x7f1018009030 comp rx=0 tx=0).ready entity=mgr.7223 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-09T14:52:12.329 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:12.319+0000 7f10217fa640 1 -- 172.21.15.43:0/2837421080 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178066 (secure 0 0 0) 0x7f1010054120 con 0x7f1024149f90 2023-08-09T14:52:12.514 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:12.512+0000 7f102af54640 1 -- 172.21.15.43:0/2837421080 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f10240cf210 con 0x7f1024149f90 2023-08-09T14:52:12.514 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:12.513+0000 7f10217fa640 1 -- 172.21.15.43:0/2837421080 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v174) v1 ==== 100+0+11 (secure 0 0 0) 0x7f1010057740 con 0x7f1024149f90 2023-08-09T14:52:12.515 INFO:teuthology.orchestra.run.smithi043.stdout:pg_num: 64 2023-08-09T14:52:12.517 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:12.515+0000 7f102af54640 1 -- 172.21.15.43:0/2837421080 >> [v2:172.21.15.203:6832/2959400181,v1:172.21.15.203:6833/2959400181] conn(0x7f100806a110 msgr2=0x7f100806c5d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T14:52:12.517 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:12.515+0000 7f102af54640 1 --2- 172.21.15.43:0/2837421080 >> [v2:172.21.15.203:6832/2959400181,v1:172.21.15.203:6833/2959400181] conn(0x7f100806a110 0x7f100806c5d0 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7f1018034000 tx=0x7f1018009030 comp rx=0 tx=0).stop 2023-08-09T14:52:12.517 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:12.515+0000 7f102af54640 1 -- 172.21.15.43:0/2837421080 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f1024149f90 msgr2=0x7f102414a410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T14:52:12.517 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:12.515+0000 7f102af54640 1 --2- 172.21.15.43:0/2837421080 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f1024149f90 0x7f102414a410 secure :-1 s=READY pgs=1952 cs=0 l=1 rev1=1 crypto rx=0x7f10100107c0 tx=0x7f1010010c80 comp rx=0 tx=0).stop 2023-08-09T14:52:12.518 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:12.516+0000 7f102af54640 1 -- 172.21.15.43:0/2837421080 shutdown_connections 2023-08-09T14:52:12.518 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:12.516+0000 7f102af54640 1 --2- 172.21.15.43:0/2837421080 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f10240cc590 0x7f10240cca10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T14:52:12.518 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:12.516+0000 7f102af54640 1 --2- 172.21.15.43:0/2837421080 >> [v2:172.21.15.203:6832/2959400181,v1:172.21.15.203:6833/2959400181] conn(0x7f100806a110 0x7f100806c5d0 unknown :-1 s=CLOSED pgs=25 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T14:52:12.518 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:12.516+0000 7f102af54640 1 --2- 172.21.15.43:0/2837421080 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f1024149f90 0x7f102414a410 unknown :-1 s=CLOSED pgs=1952 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T14:52:12.518 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:12.516+0000 7f102af54640 1 --2- 172.21.15.43:0/2837421080 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f1014003970 0x7f10240a9940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T14:52:12.518 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:12.516+0000 7f102af54640 1 -- 172.21.15.43:0/2837421080 >> 172.21.15.43:0/2837421080 conn(0x7f1024071520 msgr2=0x7f102415e110 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-09T14:52:12.519 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:12.516+0000 7f102af54640 1 -- 172.21.15.43:0/2837421080 shutdown_connections 2023-08-09T14:52:12.519 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:12.516+0000 7f102af54640 1 -- 172.21.15.43:0/2837421080 wait complete. 2023-08-09T14:52:12.536 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-08-09T14:52:12.769 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:12.768+0000 7f27867e8640 1 Processor -- start 2023-08-09T14:52:12.770 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:12.768+0000 7f27867e8640 1 -- start start 2023-08-09T14:52:12.772 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:12.770+0000 7f27867e8640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f278014f940 0x7f27800ccb20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T14:52:12.772 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:12.770+0000 7f27867e8640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f278015c5c0 0x7f278015ca20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T14:52:12.772 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:12.770+0000 7f27867e8640 1 -- --> v1:172.21.15.43:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f278015cf60 con 0x7f278006b790 2023-08-09T14:52:12.772 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:12.770+0000 7f27867e8640 1 -- --> v2:172.21.15.43:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f278006c330 con 0x7f278015c5c0 2023-08-09T14:52:12.772 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:12.770+0000 7f27867e8640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f278006c600 con 0x7f278014f940 2023-08-09T14:52:12.772 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:12.770+0000 7f2784d85640 1 --1- >> v1:172.21.15.43:6789/0 conn(0x7f278006b790 0x7f278006bc20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.43:6789/0 says I am v1:172.21.15.43:41564/0 (socket says 172.21.15.43:41564) 2023-08-09T14:52:12.773 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:12.770+0000 7f2784d85640 1 -- 172.21.15.43:0/2955609948 learned_addr learned my addr 172.21.15.43:0/2955609948 (peer_addr_for_me v1:172.21.15.43:0/0) 2023-08-09T14:52:12.773 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:12.770+0000 7f27777fe640 1 --2- 172.21.15.43:0/2955609948 >> v2:172.21.15.43:3300/0 conn(0x7f278015c5c0 0x7f278015ca20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T14:52:12.773 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:12.771+0000 7f277ffff640 1 --2- 172.21.15.43:0/2955609948 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f278014f940 0x7f27800ccb20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T14:52:12.774 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:12.771+0000 7f27777fe640 1 -- 172.21.15.43:0/2955609948 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f278014f940 msgr2=0x7f27800ccb20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-09T14:52:12.774 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:12.771+0000 7f27777fe640 1 --2- 172.21.15.43:0/2955609948 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f278014f940 0x7f27800ccb20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T14:52:12.774 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:12.771+0000 7f27777fe640 1 -- 172.21.15.43:0/2955609948 >> v1:172.21.15.43:6789/0 conn(0x7f278006b790 legacy=0x7f278006bc20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T14:52:12.774 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:12.771+0000 7f277f7fe640 1 -- 172.21.15.43:0/2955609948 <== mon.0 v1:172.21.15.43:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 446023047 0 0) 0x7f278015cf60 con 0x7f278006b790 2023-08-09T14:52:12.774 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:12.771+0000 7f27777fe640 1 -- 172.21.15.43:0/2955609948 --> v2:172.21.15.43:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f278006c8e0 con 0x7f278015c5c0 2023-08-09T14:52:12.775 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:12.773+0000 7f27777fe640 1 --2- 172.21.15.43:0/2955609948 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f278015c5c0 0x7f278015ca20 secure :-1 s=READY pgs=1954 cs=0 l=1 rev1=1 crypto rx=0x7f2768009dc0 tx=0x7f276802be90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e232f729773968a8 server_cookie=0 in_seq=0 out_seq=0 2023-08-09T14:52:12.775 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:12.774+0000 7f277f7fe640 1 -- 172.21.15.43:0/2955609948 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f276802cc40 con 0x7f278015c5c0 2023-08-09T14:52:12.775 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:12.774+0000 7f277f7fe640 1 -- 172.21.15.43:0/2955609948 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f276802cda0 con 0x7f278015c5c0 2023-08-09T14:52:12.775 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:12.774+0000 7f277f7fe640 1 -- 172.21.15.43:0/2955609948 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2768034660 con 0x7f278015c5c0 2023-08-09T14:52:12.776 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:12.774+0000 7f27867e8640 1 -- 172.21.15.43:0/2955609948 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f278015c5c0 msgr2=0x7f278015ca20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T14:52:12.776 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:12.774+0000 7f27867e8640 1 --2- 172.21.15.43:0/2955609948 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f278015c5c0 0x7f278015ca20 secure :-1 s=READY pgs=1954 cs=0 l=1 rev1=1 crypto rx=0x7f2768009dc0 tx=0x7f276802be90 comp rx=0 tx=0).stop 2023-08-09T14:52:12.776 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:12.774+0000 7f27867e8640 1 -- 172.21.15.43:0/2955609948 shutdown_connections 2023-08-09T14:52:12.777 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:12.774+0000 7f27867e8640 1 --2- 172.21.15.43:0/2955609948 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f278015c5c0 0x7f278015ca20 unknown :-1 s=CLOSED pgs=1954 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T14:52:12.777 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:12.774+0000 7f27867e8640 1 --2- 172.21.15.43:0/2955609948 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f278014f940 0x7f27800ccb20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T14:52:12.777 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:12.774+0000 7f27867e8640 1 -- 172.21.15.43:0/2955609948 >> 172.21.15.43:0/2955609948 conn(0x7f27800ce0a0 msgr2=0x7f27800ce480 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-09T14:52:12.777 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:12.774+0000 7f27867e8640 1 -- 172.21.15.43:0/2955609948 shutdown_connections 2023-08-09T14:52:12.777 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:12.775+0000 7f27867e8640 1 -- 172.21.15.43:0/2955609948 wait complete. 2023-08-09T14:52:12.778 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:12.777+0000 7f27867e8640 1 Processor -- start 2023-08-09T14:52:12.778 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:12.777+0000 7f27867e8640 1 -- start start 2023-08-09T14:52:12.779 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:12.777+0000 7f27867e8640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f278006b790 0x7f27800c5b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T14:52:12.779 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:12.777+0000 7f27867e8640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f278014f940 0x7f2780151c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T14:52:12.779 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:12.777+0000 7f27867e8640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f278015c5c0 0x7f27800cb0f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T14:52:12.779 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:12.778+0000 7f27867e8640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f27800c6e00 con 0x7f278014f940 2023-08-09T14:52:12.780 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:12.778+0000 7f27867e8640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2780152180 con 0x7f278006b790 2023-08-09T14:52:12.780 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:12.778+0000 7f27867e8640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f27800c6180 con 0x7f278015c5c0 2023-08-09T14:52:12.780 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:12.778+0000 7f27777fe640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f278014f940 0x7f2780151c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T14:52:12.780 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:12.778+0000 7f27777fe640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f278014f940 0x7f2780151c40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:35044/0 (socket says 172.21.15.43:35044) 2023-08-09T14:52:12.780 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:12.778+0000 7f27777fe640 1 -- 172.21.15.43:0/3810858846 learned_addr learned my addr 172.21.15.43:0/3810858846 (peer_addr_for_me v2:172.21.15.43:0/0) 2023-08-09T14:52:12.782 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:12.778+0000 7f27777fe640 1 -- 172.21.15.43:0/3810858846 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f278015c5c0 msgr2=0x7f27800cb0f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T14:52:12.782 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:12.778+0000 7f27777fe640 1 --2- 172.21.15.43:0/3810858846 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f278015c5c0 0x7f27800cb0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T14:52:12.782 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:12.778+0000 7f27777fe640 1 -- 172.21.15.43:0/3810858846 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f278006b790 msgr2=0x7f27800c5b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T14:52:12.782 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:12.778+0000 7f27777fe640 1 --2- 172.21.15.43:0/3810858846 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f278006b790 0x7f27800c5b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T14:52:12.782 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:12.778+0000 7f27777fe640 1 -- 172.21.15.43:0/3810858846 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2768009a70 con 0x7f278014f940 2023-08-09T14:52:12.782 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:12.779+0000 7f27777fe640 1 --2- 172.21.15.43:0/3810858846 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f278014f940 0x7f2780151c40 secure :-1 s=READY pgs=1955 cs=0 l=1 rev1=1 crypto rx=0x7f2768009dc0 tx=0x7f2768004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-09T14:52:12.782 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:12.779+0000 7f277d7fa640 1 -- 172.21.15.43:0/3810858846 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2768040650 con 0x7f278014f940 2023-08-09T14:52:12.783 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:12.779+0000 7f277d7fa640 1 -- 172.21.15.43:0/3810858846 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f27680407b0 con 0x7f278014f940 2023-08-09T14:52:12.783 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:12.779+0000 7f27867e8640 1 -- 172.21.15.43:0/3810858846 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f27800c6460 con 0x7f278014f940 2023-08-09T14:52:12.783 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:12.779+0000 7f277d7fa640 1 -- 172.21.15.43:0/3810858846 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2768040b60 con 0x7f278014f940 2023-08-09T14:52:12.783 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:12.779+0000 7f27867e8640 1 -- 172.21.15.43:0/3810858846 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f27800c6a20 con 0x7f278014f940 2023-08-09T14:52:12.783 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:12.780+0000 7f27867e8640 1 -- 172.21.15.43:0/3810858846 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f27800739f0 con 0x7f278014f940 2023-08-09T14:52:12.784 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:12.781+0000 7f277d7fa640 1 -- 172.21.15.43:0/3810858846 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 87) v1 ==== 88387+0+0 (secure 0 0 0) 0x7f2768040cc0 con 0x7f278014f940 2023-08-09T14:52:12.784 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:12.782+0000 7f277d7fa640 1 --2- 172.21.15.43:0/3810858846 >> [v2:172.21.15.203:6832/2959400181,v1:172.21.15.203:6833/2959400181] conn(0x7f276006a0e0 0x7f276006c5a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T14:52:12.784 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:12.782+0000 7f277d7fa640 1 -- 172.21.15.43:0/3810858846 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(174..174 src has 1..174) v4 ==== 7114+0+0 (secure 0 0 0) 0x7f27680ab7d0 con 0x7f278014f940 2023-08-09T14:52:12.784 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:12.783+0000 7f277ffff640 1 --2- 172.21.15.43:0/3810858846 >> [v2:172.21.15.203:6832/2959400181,v1:172.21.15.203:6833/2959400181] conn(0x7f276006a0e0 0x7f276006c5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T14:52:12.785 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:12.785+0000 7f277ffff640 1 --2- 172.21.15.43:0/3810858846 >> [v2:172.21.15.203:6832/2959400181,v1:172.21.15.203:6833/2959400181] conn(0x7f276006a0e0 0x7f276006c5a0 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7f2770006a90 tx=0x7f2770006a20 comp rx=0 tx=0).ready entity=mgr.7223 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-09T14:52:12.788 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:12.787+0000 7f277d7fa640 1 -- 172.21.15.43:0/3810858846 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178066 (secure 0 0 0) 0x7f2768076b40 con 0x7f278014f940 2023-08-09T14:52:13.019 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:13.018+0000 7f27867e8640 1 -- 172.21.15.43:0/3810858846 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f278006bfa0 con 0x7f278014f940 2023-08-09T14:52:13.021 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:13.020+0000 7f277d7fa640 1 -- 172.21.15.43:0/3810858846 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 78 v78) v1 ==== 94+0+4880 (secure 0 0 0) 0x7f276807a160 con 0x7f278014f940 2023-08-09T14:52:13.022 INFO:teuthology.orchestra.run.smithi043.stdout: 2023-08-09T14:52:13.022 INFO:teuthology.orchestra.run.smithi043.stdout:{"epoch":78,"default_fscid":9,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":7079,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.203:6836/807059349","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6834","nonce":807059349},{"type":"v1","addr":"172.21.15.203:6836","nonce":807059349}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":74},{"gid":7924,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.203:6837/3166829033","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6835","nonce":3166829033},{"type":"v1","addr":"172.21.15.203:6837","nonce":3166829033}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":74},{"gid":9642,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.43:6833/3803974919","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6832","nonce":3803974919},{"type":"v1","addr":"172.21.15.43:6833","nonce":3803974919}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":73}],"filesystems":[{"mdsmap":{"epoch":78,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-08-09T14:51:47.253704+0000","modified":"2023-08-09T14:51:50.806331+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":9648},"failed":[],"damaged":[],"stopped":[],"info":{"gid_9648":{"gid":9648,"name":"a","rank":0,"incarnation":75,"state":"up:active","state_seq":3,"addr":"172.21.15.43:6835/2435237803","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6834","nonce":2435237803},{"type":"v1","addr":"172.21.15.43:6835","nonce":2435237803}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[21],"metadata_pool":20,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":9}]} 2023-08-09T14:52:13.023 INFO:teuthology.orchestra.run.smithi043.stderr:dumped fsmap epoch 78 2023-08-09T14:52:13.024 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:13.022+0000 7f27867e8640 1 -- 172.21.15.43:0/3810858846 >> [v2:172.21.15.203:6832/2959400181,v1:172.21.15.203:6833/2959400181] conn(0x7f276006a0e0 msgr2=0x7f276006c5a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T14:52:13.024 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:13.022+0000 7f27867e8640 1 --2- 172.21.15.43:0/3810858846 >> [v2:172.21.15.203:6832/2959400181,v1:172.21.15.203:6833/2959400181] conn(0x7f276006a0e0 0x7f276006c5a0 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7f2770006a90 tx=0x7f2770006a20 comp rx=0 tx=0).stop 2023-08-09T14:52:13.024 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:13.022+0000 7f27867e8640 1 -- 172.21.15.43:0/3810858846 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f278014f940 msgr2=0x7f2780151c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T14:52:13.024 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:13.022+0000 7f27867e8640 1 --2- 172.21.15.43:0/3810858846 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f278014f940 0x7f2780151c40 secure :-1 s=READY pgs=1955 cs=0 l=1 rev1=1 crypto rx=0x7f2768009dc0 tx=0x7f2768004a40 comp rx=0 tx=0).stop 2023-08-09T14:52:13.024 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:13.023+0000 7f27867e8640 1 -- 172.21.15.43:0/3810858846 shutdown_connections 2023-08-09T14:52:13.024 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:13.023+0000 7f27867e8640 1 --2- 172.21.15.43:0/3810858846 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f278015c5c0 0x7f27800cb0f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T14:52:13.024 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:13.023+0000 7f27867e8640 1 --2- 172.21.15.43:0/3810858846 >> [v2:172.21.15.203:6832/2959400181,v1:172.21.15.203:6833/2959400181] conn(0x7f276006a0e0 0x7f276006c5a0 unknown :-1 s=CLOSED pgs=26 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T14:52:13.025 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:13.023+0000 7f27867e8640 1 --2- 172.21.15.43:0/3810858846 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f278014f940 0x7f2780151c40 unknown :-1 s=CLOSED pgs=1955 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T14:52:13.025 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:13.023+0000 7f27867e8640 1 --2- 172.21.15.43:0/3810858846 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f278006b790 0x7f27800c5b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T14:52:13.025 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:13.023+0000 7f27867e8640 1 -- 172.21.15.43:0/3810858846 >> 172.21.15.43:0/3810858846 conn(0x7f27800ce0a0 msgr2=0x7f27800c44a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-09T14:52:13.025 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:13.023+0000 7f27867e8640 1 -- 172.21.15.43:0/3810858846 shutdown_connections 2023-08-09T14:52:13.025 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:13.023+0000 7f27867e8640 1 -- 172.21.15.43:0/3810858846 wait complete. 2023-08-09T14:52:13.042 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-08-09T14:52:13.275 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:13.274+0000 7f597481e640 1 Processor -- start 2023-08-09T14:52:13.275 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:13.274+0000 7f597481e640 1 -- start start 2023-08-09T14:52:13.276 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:13.276+0000 7f597481e640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f597015f7b0 0x7f59700b4760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T14:52:13.277 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:13.276+0000 7f597481e640 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7f59700cd6b0 0x7f59700d0e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T14:52:13.277 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:13.276+0000 7f597481e640 1 -- --> v1:172.21.15.203:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f59700b7bf0 con 0x7f59700b1050 2023-08-09T14:52:13.277 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:13.276+0000 7f597481e640 1 -- --> v2:172.21.15.43:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f59700b7ed0 con 0x7f597015f7b0 2023-08-09T14:52:13.277 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:13.276+0000 7f597481e640 1 -- --> v2:172.21.15.203:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f59700d0370 con 0x7f59700cd6b0 2023-08-09T14:52:13.278 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:13.277+0000 7f596ed9d640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f597015f7b0 0x7f59700b4760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T14:52:13.278 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:13.277+0000 7f596ed9d640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f597015f7b0 0x7f59700b4760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:35052/0 (socket says 172.21.15.43:35052) 2023-08-09T14:52:13.278 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:13.277+0000 7f596ed9d640 1 -- 172.21.15.43:0/4125703015 learned_addr learned my addr 172.21.15.43:0/4125703015 (peer_addr_for_me v2:172.21.15.43:0/0) 2023-08-09T14:52:13.278 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:13.277+0000 7f596f59e640 1 --2- 172.21.15.43:0/4125703015 >> v2:172.21.15.203:3300/0 conn(0x7f59700cd6b0 0x7f59700d0e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T14:52:13.279 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:13.277+0000 7f596ed9d640 1 -- 172.21.15.43:0/4125703015 >> v2:172.21.15.203:3300/0 conn(0x7f59700cd6b0 msgr2=0x7f59700d0e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-09T14:52:13.279 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:13.277+0000 7f596ed9d640 1 --2- 172.21.15.43:0/4125703015 >> v2:172.21.15.203:3300/0 conn(0x7f59700cd6b0 0x7f59700d0e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T14:52:13.279 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:13.277+0000 7f596ed9d640 1 -- 172.21.15.43:0/4125703015 >> v1:172.21.15.203:6789/0 conn(0x7f59700b1050 legacy=0x7f59700ccfa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-09T14:52:13.279 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:13.277+0000 7f596ed9d640 1 -- 172.21.15.43:0/4125703015 --> v2:172.21.15.43:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f59700d0650 con 0x7f597015f7b0 2023-08-09T14:52:13.280 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:13.278+0000 7f596ed9d640 1 --2- 172.21.15.43:0/4125703015 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f597015f7b0 0x7f59700b4760 secure :-1 s=READY pgs=1956 cs=0 l=1 rev1=1 crypto rx=0x7f5960009870 tx=0x7f596002b910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7aae15f70b91847b server_cookie=0 in_seq=0 out_seq=0 2023-08-09T14:52:13.280 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:13.279+0000 7f596dd9b640 1 -- 172.21.15.43:0/4125703015 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5960031870 con 0x7f597015f7b0 2023-08-09T14:52:13.280 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:13.279+0000 7f596dd9b640 1 -- 172.21.15.43:0/4125703015 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f59600319d0 con 0x7f597015f7b0 2023-08-09T14:52:13.281 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:13.279+0000 7f596dd9b640 1 -- 172.21.15.43:0/4125703015 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5960031d60 con 0x7f597015f7b0 2023-08-09T14:52:13.281 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:13.279+0000 7f597481e640 1 -- 172.21.15.43:0/4125703015 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f597015f7b0 msgr2=0x7f59700b4760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T14:52:13.281 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:13.279+0000 7f597481e640 1 --2- 172.21.15.43:0/4125703015 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f597015f7b0 0x7f59700b4760 secure :-1 s=READY pgs=1956 cs=0 l=1 rev1=1 crypto rx=0x7f5960009870 tx=0x7f596002b910 comp rx=0 tx=0).stop 2023-08-09T14:52:13.281 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:13.279+0000 7f597481e640 1 -- 172.21.15.43:0/4125703015 shutdown_connections 2023-08-09T14:52:13.281 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:13.279+0000 7f597481e640 1 --2- 172.21.15.43:0/4125703015 >> v2:172.21.15.203:3300/0 conn(0x7f59700cd6b0 0x7f59700d0e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T14:52:13.282 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:13.279+0000 7f597481e640 1 --2- 172.21.15.43:0/4125703015 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f597015f7b0 0x7f59700b4760 unknown :-1 s=CLOSED pgs=1956 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T14:52:13.282 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:13.279+0000 7f597481e640 1 -- 172.21.15.43:0/4125703015 >> 172.21.15.43:0/4125703015 conn(0x7f597006a2b0 msgr2=0x7f597006a690 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-09T14:52:13.282 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:13.279+0000 7f597481e640 1 -- 172.21.15.43:0/4125703015 shutdown_connections 2023-08-09T14:52:13.282 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:13.280+0000 7f597481e640 1 -- 172.21.15.43:0/4125703015 wait complete. 2023-08-09T14:52:13.282 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:13.282+0000 7f597481e640 1 Processor -- start 2023-08-09T14:52:13.283 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:13.282+0000 7f597481e640 1 -- start start 2023-08-09T14:52:13.283 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:13.283+0000 7f597481e640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f59700b1050 0x7f59700c67e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T14:52:13.284 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:13.283+0000 7f597481e640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f59700cd6b0 0x7f59700c85e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T14:52:13.284 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:13.283+0000 7f597481e640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f597015f7b0 0x7f59701f1e80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T14:52:13.284 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:13.283+0000 7f597481e640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f59700c5e00 con 0x7f597015f7b0 2023-08-09T14:52:13.284 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:13.283+0000 7f597481e640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f59700c6350 con 0x7f59700b1050 2023-08-09T14:52:13.284 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:13.283+0000 7f597481e640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f597014efd0 con 0x7f59700cd6b0 2023-08-09T14:52:13.285 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:13.283+0000 7f596f59e640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f597015f7b0 0x7f59701f1e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T14:52:13.286 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:13.283+0000 7f596f59e640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f597015f7b0 0x7f59701f1e80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:35062/0 (socket says 172.21.15.43:35062) 2023-08-09T14:52:13.286 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:13.284+0000 7f596f59e640 1 -- 172.21.15.43:0/1424380886 learned_addr learned my addr 172.21.15.43:0/1424380886 (peer_addr_for_me v2:172.21.15.43:0/0) 2023-08-09T14:52:13.286 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:13.284+0000 7f596ed9d640 1 --2- 172.21.15.43:0/1424380886 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f59700b1050 0x7f59700c67e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T14:52:13.286 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:13.284+0000 7f596f59e640 1 -- 172.21.15.43:0/1424380886 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f59700cd6b0 msgr2=0x7f59700c85e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T14:52:13.286 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:13.284+0000 7f596f59e640 1 --2- 172.21.15.43:0/1424380886 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f59700cd6b0 0x7f59700c85e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T14:52:13.286 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:13.284+0000 7f596f59e640 1 -- 172.21.15.43:0/1424380886 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f59700b1050 msgr2=0x7f59700c67e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T14:52:13.286 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:13.284+0000 7f596f59e640 1 --2- 172.21.15.43:0/1424380886 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f59700b1050 0x7f59700c67e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T14:52:13.287 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:13.284+0000 7f596f59e640 1 -- 172.21.15.43:0/1424380886 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f59600094d0 con 0x7f597015f7b0 2023-08-09T14:52:13.287 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:13.284+0000 7f596f59e640 1 --2- 172.21.15.43:0/1424380886 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f597015f7b0 0x7f59701f1e80 secure :-1 s=READY pgs=1957 cs=0 l=1 rev1=1 crypto rx=0x7f5968007e70 tx=0x7f596800e4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-09T14:52:13.287 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:13.285+0000 7f59577fe640 1 -- 172.21.15.43:0/1424380886 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5968014620 con 0x7f597015f7b0 2023-08-09T14:52:13.287 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:13.285+0000 7f59577fe640 1 -- 172.21.15.43:0/1424380886 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f59680147a0 con 0x7f597015f7b0 2023-08-09T14:52:13.287 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:13.285+0000 7f59577fe640 1 -- 172.21.15.43:0/1424380886 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5968014b50 con 0x7f597015f7b0 2023-08-09T14:52:13.287 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:13.285+0000 7f597481e640 1 -- 172.21.15.43:0/1424380886 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f59701f2950 con 0x7f597015f7b0 2023-08-09T14:52:13.288 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:13.285+0000 7f597481e640 1 -- 172.21.15.43:0/1424380886 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f59700c5fe0 con 0x7f597015f7b0 2023-08-09T14:52:13.288 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:13.286+0000 7f597481e640 1 -- 172.21.15.43:0/1424380886 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f59701f2e70 con 0x7f597015f7b0 2023-08-09T14:52:13.288 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:13.287+0000 7f59577fe640 1 -- 172.21.15.43:0/1424380886 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 87) v1 ==== 88387+0+0 (secure 0 0 0) 0x7f5968014cb0 con 0x7f597015f7b0 2023-08-09T14:52:13.289 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:13.288+0000 7f59577fe640 1 --2- 172.21.15.43:0/1424380886 >> [v2:172.21.15.203:6832/2959400181,v1:172.21.15.203:6833/2959400181] conn(0x7f595c06a0e0 0x7f595c06c5a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T14:52:13.289 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:13.288+0000 7f59577fe640 1 -- 172.21.15.43:0/1424380886 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(174..174 src has 1..174) v4 ==== 7114+0+0 (secure 0 0 0) 0x7f59680894b0 con 0x7f597015f7b0 2023-08-09T14:52:13.289 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:13.289+0000 7f596ed9d640 1 --2- 172.21.15.43:0/1424380886 >> [v2:172.21.15.203:6832/2959400181,v1:172.21.15.203:6833/2959400181] conn(0x7f595c06a0e0 0x7f595c06c5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T14:52:13.290 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:13.290+0000 7f596ed9d640 1 --2- 172.21.15.43:0/1424380886 >> [v2:172.21.15.203:6832/2959400181,v1:172.21.15.203:6833/2959400181] conn(0x7f595c06a0e0 0x7f595c06c5a0 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7f5960036000 tx=0x7f596003c040 comp rx=0 tx=0).ready entity=mgr.7223 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-09T14:52:13.292 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:13.292+0000 7f59577fe640 1 -- 172.21.15.43:0/1424380886 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178066 (secure 0 0 0) 0x7f5968058120 con 0x7f597015f7b0 2023-08-09T14:52:13.479 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:13.479+0000 7f597481e640 1 -- 172.21.15.43:0/1424380886 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f59700739f0 con 0x7f597015f7b0 2023-08-09T14:52:13.483 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:13.482+0000 7f59577fe640 1 -- 172.21.15.43:0/1424380886 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v174) v1 ==== 74+0+19582 (secure 0 0 0) 0x7f5968057cb0 con 0x7f597015f7b0 2023-08-09T14:52:13.485 INFO:teuthology.orchestra.run.smithi043.stdout: 2023-08-09T14:52:13.485 INFO:teuthology.orchestra.run.smithi043.stdout:{"epoch":174,"fsid":"daac87ee-1337-4250-a955-aea4e408a39b","created":"2023-08-09T13:33:41.701198+0000","modified":"2023-08-09T14:52:10.059650+0000","last_up_change":"2023-08-09T13:33:53.783453+0000","last_in_change":"2023-08-09T13:33:45.148904+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":21,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":"rbd","create_time":"2023-08-09T13:33:54.365137+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":".mgr","create_time":"2023-08-09T13:33:54.831803+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":20,"pool_name":"cephfs_metadata","create_time":"2023-08-09T14:51:45.692613+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"171","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":21,"pool_name":"cephfs_data","create_time":"2023-08-09T14:51:46.286715+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"172","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"cbd49ecd-e896-4fbd-864e-5c37a829b165","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":170,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6800","nonce":502667335},{"type":"v1","addr":"172.21.15.43:6801","nonce":502667335}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6802","nonce":502667335},{"type":"v1","addr":"172.21.15.43:6803","nonce":502667335}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6806","nonce":502667335},{"type":"v1","addr":"172.21.15.43:6807","nonce":502667335}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6804","nonce":502667335},{"type":"v1","addr":"172.21.15.43:6805","nonce":502667335}]},"public_addr":"172.21.15.43:6801/502667335","cluster_addr":"172.21.15.43:6803/502667335","heartbeat_back_addr":"172.21.15.43:6807/502667335","heartbeat_front_addr":"172.21.15.43:6805/502667335","state":["exists","up"]},{"osd":1,"uuid":"75c7e0fe-1ec7-4fe8-a2f2-3661a65b8500","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":170,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6808","nonce":3289134610},{"type":"v1","addr":"172.21.15.43:6809","nonce":3289134610}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6810","nonce":3289134610},{"type":"v1","addr":"172.21.15.43:6812","nonce":3289134610}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6821","nonce":3289134610},{"type":"v1","addr":"172.21.15.43:6824","nonce":3289134610}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6815","nonce":3289134610},{"type":"v1","addr":"172.21.15.43:6818","nonce":3289134610}]},"public_addr":"172.21.15.43:6809/3289134610","cluster_addr":"172.21.15.43:6812/3289134610","heartbeat_back_addr":"172.21.15.43:6824/3289134610","heartbeat_front_addr":"172.21.15.43:6818/3289134610","state":["exists","up"]},{"osd":2,"uuid":"bb5c41e9-7f5f-4125-9b01-3bb8bfc2ea83","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":170,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6811","nonce":4079709736},{"type":"v1","addr":"172.21.15.43:6814","nonce":4079709736}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6817","nonce":4079709736},{"type":"v1","addr":"172.21.15.43:6820","nonce":4079709736}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6829","nonce":4079709736},{"type":"v1","addr":"172.21.15.43:6831","nonce":4079709736}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6823","nonce":4079709736},{"type":"v1","addr":"172.21.15.43:6826","nonce":4079709736}]},"public_addr":"172.21.15.43:6814/4079709736","cluster_addr":"172.21.15.43:6820/4079709736","heartbeat_back_addr":"172.21.15.43:6831/4079709736","heartbeat_front_addr":"172.21.15.43:6826/4079709736","state":["exists","up"]},{"osd":3,"uuid":"bfe4c8b4-694a-465c-87e6-2b2aab852613","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":170,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6813","nonce":1709538759},{"type":"v1","addr":"172.21.15.43:6816","nonce":1709538759}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6819","nonce":1709538759},{"type":"v1","addr":"172.21.15.43:6822","nonce":1709538759}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6828","nonce":1709538759},{"type":"v1","addr":"172.21.15.43:6830","nonce":1709538759}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6825","nonce":1709538759},{"type":"v1","addr":"172.21.15.43:6827","nonce":1709538759}]},"public_addr":"172.21.15.43:6816/1709538759","cluster_addr":"172.21.15.43:6822/1709538759","heartbeat_back_addr":"172.21.15.43:6830/1709538759","heartbeat_front_addr":"172.21.15.43:6827/1709538759","state":["exists","up"]},{"osd":4,"uuid":"fa2e8a77-aba1-4756-853e-0d174a5bdeba","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":170,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6800","nonce":4196866481},{"type":"v1","addr":"172.21.15.203:6801","nonce":4196866481}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6802","nonce":4196866481},{"type":"v1","addr":"172.21.15.203:6803","nonce":4196866481}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6806","nonce":4196866481},{"type":"v1","addr":"172.21.15.203:6807","nonce":4196866481}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6804","nonce":4196866481},{"type":"v1","addr":"172.21.15.203:6805","nonce":4196866481}]},"public_addr":"172.21.15.203:6801/4196866481","cluster_addr":"172.21.15.203:6803/4196866481","heartbeat_back_addr":"172.21.15.203:6807/4196866481","heartbeat_front_addr":"172.21.15.203:6805/4196866481","state":["exists","up"]},{"osd":5,"uuid":"81811e4b-4d24-4f51-a82c-f1d700a2a00d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":170,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6816","nonce":3059290081},{"type":"v1","addr":"172.21.15.203:6817","nonce":3059290081}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6818","nonce":3059290081},{"type":"v1","addr":"172.21.15.203:6819","nonce":3059290081}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6822","nonce":3059290081},{"type":"v1","addr":"172.21.15.203:6823","nonce":3059290081}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6820","nonce":3059290081},{"type":"v1","addr":"172.21.15.203:6821","nonce":3059290081}]},"public_addr":"172.21.15.203:6817/3059290081","cluster_addr":"172.21.15.203:6819/3059290081","heartbeat_back_addr":"172.21.15.203:6823/3059290081","heartbeat_front_addr":"172.21.15.203:6821/3059290081","state":["exists","up"]},{"osd":6,"uuid":"307bac4d-6a5b-443d-af75-56d695d7674f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":170,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6824","nonce":512155569},{"type":"v1","addr":"172.21.15.203:6825","nonce":512155569}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6826","nonce":512155569},{"type":"v1","addr":"172.21.15.203:6827","nonce":512155569}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6830","nonce":512155569},{"type":"v1","addr":"172.21.15.203:6831","nonce":512155569}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6828","nonce":512155569},{"type":"v1","addr":"172.21.15.203:6829","nonce":512155569}]},"public_addr":"172.21.15.203:6825/512155569","cluster_addr":"172.21.15.203:6827/512155569","heartbeat_back_addr":"172.21.15.203:6831/512155569","heartbeat_front_addr":"172.21.15.203:6829/512155569","state":["exists","up"]},{"osd":7,"uuid":"a95dc7ca-9f1b-4295-af70-68a22a2d7d5e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":170,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6808","nonce":3331450438},{"type":"v1","addr":"172.21.15.203:6809","nonce":3331450438}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6810","nonce":3331450438},{"type":"v1","addr":"172.21.15.203:6811","nonce":3331450438}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6814","nonce":3331450438},{"type":"v1","addr":"172.21.15.203:6815","nonce":3331450438}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6812","nonce":3331450438},{"type":"v1","addr":"172.21.15.203:6813","nonce":3331450438}]},"public_addr":"172.21.15.203:6809/3331450438","cluster_addr":"172.21.15.203:6811/3331450438","heartbeat_back_addr":"172.21.15.203:6815/3331450438","heartbeat_front_addr":"172.21.15.203:6813/3331450438","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-09T13:33:50.547652+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-09T13:33:50.589469+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-09T13:33:50.561724+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-09T13:33:50.599650+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-09T13:33:50.411855+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-09T13:33:50.219681+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-09T13:33:50.504664+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-09T13:33:50.441248+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.203:0/3301934490":"2023-08-10T14:52:10.059559+0000","172.21.15.203:0/1715525748":"2023-08-10T14:52:10.059559+0000","172.21.15.203:0/1477488908":"2023-08-10T14:52:10.059559+0000","172.21.15.203:6833/843920641":"2023-08-10T14:52:10.059559+0000","172.21.15.203:6832/843920641":"2023-08-10T14:52:10.059559+0000","172.21.15.203:0/2095212004":"2023-08-10T14:52:02.909250+0000","172.21.15.203:0/1196801979":"2023-08-10T14:52:10.059559+0000","172.21.15.203:0/3749872091":"2023-08-10T14:52:02.909250+0000","172.21.15.203:0/1305485142":"2023-08-10T14:52:10.059559+0000","172.21.15.203:0/651612122":"2023-08-10T14:52:02.909250+0000","172.21.15.203:0/4120954828":"2023-08-10T14:52:02.909250+0000","172.21.15.203:0/3552079285":"2023-08-10T14:52:10.059559+0000","172.21.15.203:0/2783782601":"2023-08-10T14:52:02.909250+0000","172.21.15.203:0/2232659577":"2023-08-10T14:52:02.909250+0000","172.21.15.203:6833/2566816332":"2023-08-10T14:52:02.909250+0000","172.21.15.203:6832/2566816332":"2023-08-10T14:52:02.909250+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":7,"snaps":[{"begin":104,"length":1},{"begin":221,"length":1}]}],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-08-09T14:52:13.486 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:13.485+0000 7f597481e640 1 -- 172.21.15.43:0/1424380886 >> [v2:172.21.15.203:6832/2959400181,v1:172.21.15.203:6833/2959400181] conn(0x7f595c06a0e0 msgr2=0x7f595c06c5a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T14:52:13.487 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:13.485+0000 7f597481e640 1 --2- 172.21.15.43:0/1424380886 >> [v2:172.21.15.203:6832/2959400181,v1:172.21.15.203:6833/2959400181] conn(0x7f595c06a0e0 0x7f595c06c5a0 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7f5960036000 tx=0x7f596003c040 comp rx=0 tx=0).stop 2023-08-09T14:52:13.487 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:13.485+0000 7f597481e640 1 -- 172.21.15.43:0/1424380886 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f597015f7b0 msgr2=0x7f59701f1e80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T14:52:13.487 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:13.485+0000 7f597481e640 1 --2- 172.21.15.43:0/1424380886 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f597015f7b0 0x7f59701f1e80 secure :-1 s=READY pgs=1957 cs=0 l=1 rev1=1 crypto rx=0x7f5968007e70 tx=0x7f596800e4b0 comp rx=0 tx=0).stop 2023-08-09T14:52:13.487 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:13.485+0000 7f597481e640 1 -- 172.21.15.43:0/1424380886 shutdown_connections 2023-08-09T14:52:13.487 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:13.485+0000 7f597481e640 1 --2- 172.21.15.43:0/1424380886 >> [v2:172.21.15.203:6832/2959400181,v1:172.21.15.203:6833/2959400181] conn(0x7f595c06a0e0 0x7f595c06c5a0 unknown :-1 s=CLOSED pgs=27 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T14:52:13.488 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:13.485+0000 7f597481e640 1 --2- 172.21.15.43:0/1424380886 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f597015f7b0 0x7f59701f1e80 unknown :-1 s=CLOSED pgs=1957 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T14:52:13.488 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:13.485+0000 7f597481e640 1 --2- 172.21.15.43:0/1424380886 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f59700cd6b0 0x7f59700c85e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T14:52:13.488 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:13.485+0000 7f597481e640 1 --2- 172.21.15.43:0/1424380886 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f59700b1050 0x7f59700c67e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T14:52:13.488 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:13.485+0000 7f597481e640 1 -- 172.21.15.43:0/1424380886 >> 172.21.15.43:0/1424380886 conn(0x7f597006a2b0 msgr2=0x7f597015e150 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-09T14:52:13.489 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:13.485+0000 7f597481e640 1 -- 172.21.15.43:0/1424380886 shutdown_connections 2023-08-09T14:52:13.489 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:13.486+0000 7f597481e640 1 -- 172.21.15.43:0/1424380886 wait complete. 2023-08-09T14:52:13.504 INFO:tasks.cephfs.filesystem:Destroying file system cephfs and related pools 2023-08-09T14:52:13.505 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-08-09T14:52:13.707 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:13.707+0000 7f49e3b12640 1 Processor -- start 2023-08-09T14:52:13.707 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:13.707+0000 7f49e3b12640 1 -- start start 2023-08-09T14:52:13.709 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:13.708+0000 7f49e3b12640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f49dc0ce720 0x7f49dc0ceb00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T14:52:13.709 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:13.708+0000 7f49e3b12640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f49dc0ce080 0x7f49dc15e5b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T14:52:13.709 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:13.708+0000 7f49e3b12640 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7f49dc15eaf0 0x7f49dc15efa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T14:52:13.709 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:13.708+0000 7f49e3b12640 1 -- --> v2:172.21.15.43:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f49dc0cd8a0 con 0x7f49dc0ce080 2023-08-09T14:52:13.709 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:13.708+0000 7f49e3b12640 1 -- --> v2:172.21.15.203:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f49dc0cf350 con 0x7f49dc15eaf0 2023-08-09T14:52:13.710 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:13.708+0000 7f49e3b12640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f49dc0cf040 con 0x7f49dc0ce720 2023-08-09T14:52:13.710 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:13.709+0000 7f49e10ad640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f49dc0ce080 0x7f49dc15e5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T14:52:13.711 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:13.709+0000 7f49e10ad640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f49dc0ce080 0x7f49dc15e5b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:35066/0 (socket says 172.21.15.43:35066) 2023-08-09T14:52:13.711 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:13.709+0000 7f49e10ad640 1 -- 172.21.15.43:0/732644724 learned_addr learned my addr 172.21.15.43:0/732644724 (peer_addr_for_me v2:172.21.15.43:0/0) 2023-08-09T14:52:13.711 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:13.709+0000 7f49e18ae640 1 --2- 172.21.15.43:0/732644724 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f49dc0ce720 0x7f49dc0ceb00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T14:52:13.711 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:13.709+0000 7f49e20af640 1 --2- 172.21.15.43:0/732644724 >> v2:172.21.15.203:3300/0 conn(0x7f49dc15eaf0 0x7f49dc15efa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T14:52:13.711 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:13.710+0000 7f49e10ad640 1 -- 172.21.15.43:0/732644724 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f49dc0ce720 msgr2=0x7f49dc0ceb00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-09T14:52:13.711 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:13.710+0000 7f49e10ad640 1 --2- 172.21.15.43:0/732644724 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f49dc0ce720 0x7f49dc0ceb00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T14:52:13.711 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:13.710+0000 7f49e10ad640 1 -- 172.21.15.43:0/732644724 >> v2:172.21.15.203:3300/0 conn(0x7f49dc15eaf0 msgr2=0x7f49dc15efa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-09T14:52:13.712 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:13.710+0000 7f49e10ad640 1 --2- 172.21.15.43:0/732644724 >> v2:172.21.15.203:3300/0 conn(0x7f49dc15eaf0 0x7f49dc15efa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T14:52:13.712 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:13.710+0000 7f49e10ad640 1 -- 172.21.15.43:0/732644724 --> v2:172.21.15.43:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f49dc071a70 con 0x7f49dc0ce080 2023-08-09T14:52:13.713 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:13.711+0000 7f49e10ad640 1 --2- 172.21.15.43:0/732644724 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f49dc0ce080 0x7f49dc15e5b0 secure :-1 s=READY pgs=1958 cs=0 l=1 rev1=1 crypto rx=0x7f49dc0ca370 tx=0x7f49cc02b880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e837b0e91fa41eaf server_cookie=0 in_seq=0 out_seq=0 2023-08-09T14:52:13.713 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:13.712+0000 7f49e08ac640 1 -- 172.21.15.43:0/732644724 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f49cc0317e0 con 0x7f49dc0ce080 2023-08-09T14:52:13.713 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:13.712+0000 7f49e08ac640 1 -- 172.21.15.43:0/732644724 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f49cc031940 con 0x7f49dc0ce080 2023-08-09T14:52:13.713 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:13.712+0000 7f49e08ac640 1 -- 172.21.15.43:0/732644724 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f49cc031cd0 con 0x7f49dc0ce080 2023-08-09T14:52:13.713 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:13.712+0000 7f49e3b12640 1 -- 172.21.15.43:0/732644724 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f49dc0ce080 msgr2=0x7f49dc15e5b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T14:52:13.713 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:13.712+0000 7f49e3b12640 1 --2- 172.21.15.43:0/732644724 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f49dc0ce080 0x7f49dc15e5b0 secure :-1 s=READY pgs=1958 cs=0 l=1 rev1=1 crypto rx=0x7f49dc0ca370 tx=0x7f49cc02b880 comp rx=0 tx=0).stop 2023-08-09T14:52:13.714 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:13.712+0000 7f49e3b12640 1 -- 172.21.15.43:0/732644724 shutdown_connections 2023-08-09T14:52:13.714 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:13.712+0000 7f49e3b12640 1 --2- 172.21.15.43:0/732644724 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f49dc0ce080 0x7f49dc15e5b0 unknown :-1 s=CLOSED pgs=1958 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T14:52:13.714 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:13.712+0000 7f49e3b12640 1 --2- 172.21.15.43:0/732644724 >> v2:172.21.15.203:3300/0 conn(0x7f49dc15eaf0 0x7f49dc15efa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T14:52:13.714 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:13.712+0000 7f49e3b12640 1 --2- 172.21.15.43:0/732644724 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f49dc0ce720 0x7f49dc0ceb00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T14:52:13.714 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:13.712+0000 7f49e3b12640 1 -- 172.21.15.43:0/732644724 >> 172.21.15.43:0/732644724 conn(0x7f49dc0ccd80 msgr2=0x7f49dc0cd190 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-09T14:52:13.714 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:13.712+0000 7f49e3b12640 1 -- 172.21.15.43:0/732644724 shutdown_connections 2023-08-09T14:52:13.714 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:13.713+0000 7f49e3b12640 1 -- 172.21.15.43:0/732644724 wait complete. 2023-08-09T14:52:13.715 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:13.714+0000 7f49e3b12640 1 Processor -- start 2023-08-09T14:52:13.715 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:13.715+0000 7f49e3b12640 1 -- start start 2023-08-09T14:52:13.716 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:13.715+0000 7f49e3b12640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f49dc0ce080 0x7f49dc1f1d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T14:52:13.716 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:13.716+0000 7f49e3b12640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f49dc0ce720 0x7f49dc1593e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T14:52:13.716 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:13.716+0000 7f49e3b12640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f49dc15eaf0 0x7f49dc1f5c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T14:52:13.717 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:13.716+0000 7f49e3b12640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f49dc1f2360 con 0x7f49dc0ce720 2023-08-09T14:52:13.717 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:13.716+0000 7f49e3b12640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f49dc1f2640 con 0x7f49dc15eaf0 2023-08-09T14:52:13.717 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:13.716+0000 7f49e3b12640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f49dc1f2920 con 0x7f49dc0ce080 2023-08-09T14:52:13.717 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:13.716+0000 7f49e10ad640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f49dc0ce720 0x7f49dc1593e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T14:52:13.717 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:13.716+0000 7f49e10ad640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f49dc0ce720 0x7f49dc1593e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:35080/0 (socket says 172.21.15.43:35080) 2023-08-09T14:52:13.717 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:13.716+0000 7f49e10ad640 1 -- 172.21.15.43:0/427089968 learned_addr learned my addr 172.21.15.43:0/427089968 (peer_addr_for_me v2:172.21.15.43:0/0) 2023-08-09T14:52:13.718 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:13.716+0000 7f49e10ad640 1 -- 172.21.15.43:0/427089968 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f49dc0ce080 msgr2=0x7f49dc1f1d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T14:52:13.718 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:13.716+0000 7f49e10ad640 1 --2- 172.21.15.43:0/427089968 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f49dc0ce080 0x7f49dc1f1d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T14:52:13.718 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:13.716+0000 7f49e10ad640 1 -- 172.21.15.43:0/427089968 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f49dc15eaf0 msgr2=0x7f49dc1f5c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T14:52:13.719 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:13.716+0000 7f49e10ad640 1 --2- 172.21.15.43:0/427089968 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f49dc15eaf0 0x7f49dc1f5c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T14:52:13.719 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:13.716+0000 7f49e10ad640 1 -- 172.21.15.43:0/427089968 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f49cc0094d0 con 0x7f49dc0ce720 2023-08-09T14:52:13.719 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:13.717+0000 7f49e10ad640 1 --2- 172.21.15.43:0/427089968 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f49dc0ce720 0x7f49dc1593e0 secure :-1 s=READY pgs=1959 cs=0 l=1 rev1=1 crypto rx=0x7f49cc000c00 tx=0x7f49cc0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-09T14:52:13.719 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:13.717+0000 7f49d27fc640 1 -- 172.21.15.43:0/427089968 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f49cc032880 con 0x7f49dc0ce720 2023-08-09T14:52:13.719 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:13.717+0000 7f49d27fc640 1 -- 172.21.15.43:0/427089968 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f49cc032a20 con 0x7f49dc0ce720 2023-08-09T14:52:13.719 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:13.717+0000 7f49e3b12640 1 -- 172.21.15.43:0/427089968 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f49dc0c3df0 con 0x7f49dc0ce720 2023-08-09T14:52:13.719 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:13.717+0000 7f49d27fc640 1 -- 172.21.15.43:0/427089968 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f49cc032dd0 con 0x7f49dc0ce720 2023-08-09T14:52:13.719 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:13.717+0000 7f49e3b12640 1 -- 172.21.15.43:0/427089968 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f49dc1f6190 con 0x7f49dc0ce720 2023-08-09T14:52:13.720 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:13.718+0000 7f49e3b12640 1 -- 172.21.15.43:0/427089968 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f49dc15e1c0 con 0x7f49dc0ce720 2023-08-09T14:52:13.720 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:13.719+0000 7f49d27fc640 1 -- 172.21.15.43:0/427089968 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 87) v1 ==== 88387+0+0 (secure 0 0 0) 0x7f49cc04e450 con 0x7f49dc0ce720 2023-08-09T14:52:13.720 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:13.720+0000 7f49d27fc640 1 --2- 172.21.15.43:0/427089968 >> [v2:172.21.15.203:6832/2959400181,v1:172.21.15.203:6833/2959400181] conn(0x7f49c006a0e0 0x7f49c006c5a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T14:52:13.721 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:13.720+0000 7f49d27fc640 1 -- 172.21.15.43:0/427089968 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(174..174 src has 1..174) v4 ==== 7114+0+0 (secure 0 0 0) 0x7f49cc0aad20 con 0x7f49dc0ce720 2023-08-09T14:52:13.721 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:13.721+0000 7f49e18ae640 1 --2- 172.21.15.43:0/427089968 >> [v2:172.21.15.203:6832/2959400181,v1:172.21.15.203:6833/2959400181] conn(0x7f49c006a0e0 0x7f49c006c5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T14:52:13.722 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:13.722+0000 7f49e18ae640 1 --2- 172.21.15.43:0/427089968 >> [v2:172.21.15.203:6832/2959400181,v1:172.21.15.203:6833/2959400181] conn(0x7f49c006a0e0 0x7f49c006c5a0 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7f49d8009540 tx=0x7f49d8009450 comp rx=0 tx=0).ready entity=mgr.7223 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-09T14:52:13.725 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:13.725+0000 7f49d27fc640 1 -- 172.21.15.43:0/427089968 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178066 (secure 0 0 0) 0x7f49cc075eb0 con 0x7f49dc0ce720 2023-08-09T14:52:13.952 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:13.951+0000 7f49e3b12640 1 -- 172.21.15.43:0/427089968 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f49dc0739f0 con 0x7f49dc0ce720 2023-08-09T14:52:13.953 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:13.952+0000 7f49d27fc640 1 -- 172.21.15.43:0/427089968 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 78 v78) v1 ==== 94+0+4880 (secure 0 0 0) 0x7f49cc0794d0 con 0x7f49dc0ce720 2023-08-09T14:52:13.953 INFO:teuthology.orchestra.run.smithi043.stdout: 2023-08-09T14:52:13.953 INFO:teuthology.orchestra.run.smithi043.stdout:{"epoch":78,"default_fscid":9,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":7079,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.203:6836/807059349","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6834","nonce":807059349},{"type":"v1","addr":"172.21.15.203:6836","nonce":807059349}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":74},{"gid":7924,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.203:6837/3166829033","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6835","nonce":3166829033},{"type":"v1","addr":"172.21.15.203:6837","nonce":3166829033}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":74},{"gid":9642,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.43:6833/3803974919","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6832","nonce":3803974919},{"type":"v1","addr":"172.21.15.43:6833","nonce":3803974919}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":73}],"filesystems":[{"mdsmap":{"epoch":78,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-08-09T14:51:47.253704+0000","modified":"2023-08-09T14:51:50.806331+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":9648},"failed":[],"damaged":[],"stopped":[],"info":{"gid_9648":{"gid":9648,"name":"a","rank":0,"incarnation":75,"state":"up:active","state_seq":3,"addr":"172.21.15.43:6835/2435237803","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6834","nonce":2435237803},{"type":"v1","addr":"172.21.15.43:6835","nonce":2435237803}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[21],"metadata_pool":20,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":9}]} 2023-08-09T14:52:13.954 INFO:teuthology.orchestra.run.smithi043.stderr:dumped fsmap epoch 78 2023-08-09T14:52:13.956 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:13.956+0000 7f49e3b12640 1 -- 172.21.15.43:0/427089968 >> [v2:172.21.15.203:6832/2959400181,v1:172.21.15.203:6833/2959400181] conn(0x7f49c006a0e0 msgr2=0x7f49c006c5a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T14:52:13.957 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:13.956+0000 7f49e3b12640 1 --2- 172.21.15.43:0/427089968 >> [v2:172.21.15.203:6832/2959400181,v1:172.21.15.203:6833/2959400181] conn(0x7f49c006a0e0 0x7f49c006c5a0 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7f49d8009540 tx=0x7f49d8009450 comp rx=0 tx=0).stop 2023-08-09T14:52:13.957 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:13.956+0000 7f49e3b12640 1 -- 172.21.15.43:0/427089968 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f49dc0ce720 msgr2=0x7f49dc1593e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T14:52:13.957 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:13.956+0000 7f49e3b12640 1 --2- 172.21.15.43:0/427089968 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f49dc0ce720 0x7f49dc1593e0 secure :-1 s=READY pgs=1959 cs=0 l=1 rev1=1 crypto rx=0x7f49cc000c00 tx=0x7f49cc0046c0 comp rx=0 tx=0).stop 2023-08-09T14:52:13.957 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:13.956+0000 7f49e3b12640 1 -- 172.21.15.43:0/427089968 shutdown_connections 2023-08-09T14:52:13.957 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:13.956+0000 7f49e3b12640 1 --2- 172.21.15.43:0/427089968 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f49dc15eaf0 0x7f49dc1f5c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T14:52:13.957 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:13.956+0000 7f49e3b12640 1 --2- 172.21.15.43:0/427089968 >> [v2:172.21.15.203:6832/2959400181,v1:172.21.15.203:6833/2959400181] conn(0x7f49c006a0e0 0x7f49c006c5a0 unknown :-1 s=CLOSED pgs=28 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T14:52:13.957 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:13.956+0000 7f49e3b12640 1 --2- 172.21.15.43:0/427089968 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f49dc0ce720 0x7f49dc1593e0 unknown :-1 s=CLOSED pgs=1959 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T14:52:13.958 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:13.956+0000 7f49e3b12640 1 --2- 172.21.15.43:0/427089968 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f49dc0ce080 0x7f49dc1f1d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T14:52:13.958 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:13.956+0000 7f49e3b12640 1 -- 172.21.15.43:0/427089968 >> 172.21.15.43:0/427089968 conn(0x7f49dc0ccd80 msgr2=0x7f49dc15dab0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-09T14:52:13.958 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:13.956+0000 7f49e3b12640 1 -- 172.21.15.43:0/427089968 shutdown_connections 2023-08-09T14:52:13.958 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:13.956+0000 7f49e3b12640 1 -- 172.21.15.43:0/427089968 wait complete. 2023-08-09T14:52:13.975 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-08-09T14:52:14.193 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:14.192+0000 7f45e759e640 1 Processor -- start 2023-08-09T14:52:14.193 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:14.193+0000 7f45e759e640 1 -- start start 2023-08-09T14:52:14.194 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:14.194+0000 7f45e759e640 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7f45e815c460 0x7f45e80d0df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T14:52:14.195 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:14.194+0000 7f45e759e640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f45e80d1330 0x7f45e80ccbd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T14:52:14.195 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:14.194+0000 7f45e759e640 1 -- --> v1:172.21.15.43:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f45e80d1c50 con 0x7f45e80cd110 2023-08-09T14:52:14.195 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:14.194+0000 7f45e759e640 1 -- --> v2:172.21.15.43:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f45e80cd5a0 con 0x7f45e80d1330 2023-08-09T14:52:14.195 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:14.194+0000 7f45e759e640 1 -- --> v2:172.21.15.203:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f45e80d2cf0 con 0x7f45e815c460 2023-08-09T14:52:14.196 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:14.195+0000 7f45dffff640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f45e80d1330 0x7f45e80ccbd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T14:52:14.196 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:14.195+0000 7f45e6d9d640 1 --1- >> v1:172.21.15.43:6789/0 conn(0x7f45e80cd110 0x7f45e80b3df0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.43:6789/0 says I am v1:172.21.15.43:41568/0 (socket says 172.21.15.43:41568) 2023-08-09T14:52:14.196 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:14.195+0000 7f45e6d9d640 1 -- 172.21.15.43:0/3345668601 learned_addr learned my addr 172.21.15.43:0/3345668601 (peer_addr_for_me v1:172.21.15.43:0/0) 2023-08-09T14:52:14.196 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:14.195+0000 7f45e659c640 1 --2- 172.21.15.43:0/3345668601 >> v2:172.21.15.203:3300/0 conn(0x7f45e815c460 0x7f45e80d0df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T14:52:14.196 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:14.195+0000 7f45dffff640 1 -- 172.21.15.43:0/3345668601 >> v2:172.21.15.203:3300/0 conn(0x7f45e815c460 msgr2=0x7f45e80d0df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-09T14:52:14.197 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:14.195+0000 7f45dffff640 1 --2- 172.21.15.43:0/3345668601 >> v2:172.21.15.203:3300/0 conn(0x7f45e815c460 0x7f45e80d0df0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T14:52:14.197 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:14.195+0000 7f45dffff640 1 -- 172.21.15.43:0/3345668601 >> v1:172.21.15.43:6789/0 conn(0x7f45e80cd110 legacy=0x7f45e80b3df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T14:52:14.197 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:14.195+0000 7f45dffff640 1 -- 172.21.15.43:0/3345668601 --> v2:172.21.15.43:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f45e80d1f30 con 0x7f45e80d1330 2023-08-09T14:52:14.197 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:14.195+0000 7f45e5d9b640 1 -- 172.21.15.43:0/3345668601 <== mon.0 v1:172.21.15.43:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 341390130 0 0) 0x7f45e80d1c50 con 0x7f45e80cd110 2023-08-09T14:52:14.197 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:14.197+0000 7f45dffff640 1 --2- 172.21.15.43:0/3345668601 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f45e80d1330 0x7f45e80ccbd0 secure :-1 s=READY pgs=1961 cs=0 l=1 rev1=1 crypto rx=0x7f45e80ced50 tx=0x7f45d002b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4a594a472ed32062 server_cookie=0 in_seq=0 out_seq=0 2023-08-09T14:52:14.198 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:14.197+0000 7f45e5d9b640 1 -- 172.21.15.43:0/3345668601 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f45d0031830 con 0x7f45e80d1330 2023-08-09T14:52:14.198 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:14.197+0000 7f45e5d9b640 1 -- 172.21.15.43:0/3345668601 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f45d0031990 con 0x7f45e80d1330 2023-08-09T14:52:14.198 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:14.197+0000 7f45e5d9b640 1 -- 172.21.15.43:0/3345668601 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f45d0031d20 con 0x7f45e80d1330 2023-08-09T14:52:14.199 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:14.198+0000 7f45e759e640 1 -- 172.21.15.43:0/3345668601 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f45e80d1330 msgr2=0x7f45e80ccbd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T14:52:14.199 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:14.198+0000 7f45e759e640 1 --2- 172.21.15.43:0/3345668601 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f45e80d1330 0x7f45e80ccbd0 secure :-1 s=READY pgs=1961 cs=0 l=1 rev1=1 crypto rx=0x7f45e80ced50 tx=0x7f45d002b8d0 comp rx=0 tx=0).stop 2023-08-09T14:52:14.199 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:14.198+0000 7f45e759e640 1 -- 172.21.15.43:0/3345668601 shutdown_connections 2023-08-09T14:52:14.199 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:14.198+0000 7f45e759e640 1 --2- 172.21.15.43:0/3345668601 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f45e80d1330 0x7f45e80ccbd0 unknown :-1 s=CLOSED pgs=1961 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T14:52:14.199 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:14.198+0000 7f45e759e640 1 --2- 172.21.15.43:0/3345668601 >> v2:172.21.15.203:3300/0 conn(0x7f45e815c460 0x7f45e80d0df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T14:52:14.199 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:14.198+0000 7f45e759e640 1 -- 172.21.15.43:0/3345668601 >> 172.21.15.43:0/3345668601 conn(0x7f45e8071310 msgr2=0x7f45e8071720 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-09T14:52:14.200 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:14.198+0000 7f45e759e640 1 -- 172.21.15.43:0/3345668601 shutdown_connections 2023-08-09T14:52:14.200 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:14.198+0000 7f45e759e640 1 -- 172.21.15.43:0/3345668601 wait complete. 2023-08-09T14:52:14.200 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:14.200+0000 7f45e759e640 1 Processor -- start 2023-08-09T14:52:14.200 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:14.200+0000 7f45e759e640 1 -- start start 2023-08-09T14:52:14.201 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:14.201+0000 7f45e759e640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f45e80cd110 0x7f45e81f2210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T14:52:14.202 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:14.201+0000 7f45e759e640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f45e80d1330 0x7f45e8156700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T14:52:14.202 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:14.201+0000 7f45e759e640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f45e815c460 0x7f45e8156c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T14:52:14.202 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:14.201+0000 7f45e759e640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f45e81f2750 con 0x7f45e80cd110 2023-08-09T14:52:14.202 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:14.201+0000 7f45e759e640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f45e8156330 con 0x7f45e80d1330 2023-08-09T14:52:14.202 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:14.201+0000 7f45e759e640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f45e8155d70 con 0x7f45e815c460 2023-08-09T14:52:14.202 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:14.201+0000 7f45e659c640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f45e80cd110 0x7f45e81f2210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T14:52:14.202 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:14.201+0000 7f45e659c640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f45e80cd110 0x7f45e81f2210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:35096/0 (socket says 172.21.15.43:35096) 2023-08-09T14:52:14.203 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:14.201+0000 7f45e659c640 1 -- 172.21.15.43:0/2564073524 learned_addr learned my addr 172.21.15.43:0/2564073524 (peer_addr_for_me v2:172.21.15.43:0/0) 2023-08-09T14:52:14.203 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:14.202+0000 7f45e659c640 1 -- 172.21.15.43:0/2564073524 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f45e815c460 msgr2=0x7f45e8156c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T14:52:14.204 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:14.202+0000 7f45e659c640 1 --2- 172.21.15.43:0/2564073524 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f45e815c460 0x7f45e8156c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T14:52:14.204 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:14.202+0000 7f45e659c640 1 -- 172.21.15.43:0/2564073524 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f45e80d1330 msgr2=0x7f45e8156700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T14:52:14.204 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:14.202+0000 7f45e659c640 1 --2- 172.21.15.43:0/2564073524 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f45e80d1330 0x7f45e8156700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T14:52:14.204 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:14.202+0000 7f45e659c640 1 -- 172.21.15.43:0/2564073524 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f45d0009400 con 0x7f45e80cd110 2023-08-09T14:52:14.204 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:14.202+0000 7f45e659c640 1 --2- 172.21.15.43:0/2564073524 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f45e80cd110 0x7f45e81f2210 secure :-1 s=READY pgs=1962 cs=0 l=1 rev1=1 crypto rx=0x7f45d800e8a0 tx=0x7f45d800ed60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-09T14:52:14.204 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:14.202+0000 7f45deffd640 1 -- 172.21.15.43:0/2564073524 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f45d80094f0 con 0x7f45e80cd110 2023-08-09T14:52:14.205 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:14.202+0000 7f45deffd640 1 -- 172.21.15.43:0/2564073524 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f45d800c940 con 0x7f45e80cd110 2023-08-09T14:52:14.205 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:14.202+0000 7f45e759e640 1 -- 172.21.15.43:0/2564073524 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f45e8156050 con 0x7f45e80cd110 2023-08-09T14:52:14.205 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:14.202+0000 7f45deffd640 1 -- 172.21.15.43:0/2564073524 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f45d800ccf0 con 0x7f45e80cd110 2023-08-09T14:52:14.205 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:14.202+0000 7f45e759e640 1 -- 172.21.15.43:0/2564073524 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f45e8158250 con 0x7f45e80cd110 2023-08-09T14:52:14.205 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:14.203+0000 7f45e759e640 1 -- 172.21.15.43:0/2564073524 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f45e80d1920 con 0x7f45e80cd110 2023-08-09T14:52:14.206 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:14.204+0000 7f45deffd640 1 -- 172.21.15.43:0/2564073524 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 87) v1 ==== 88387+0+0 (secure 0 0 0) 0x7f45d8013470 con 0x7f45e80cd110 2023-08-09T14:52:14.206 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:14.205+0000 7f45deffd640 1 --2- 172.21.15.43:0/2564073524 >> [v2:172.21.15.203:6832/2959400181,v1:172.21.15.203:6833/2959400181] conn(0x7f45c806a0e0 0x7f45c806c5a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T14:52:14.206 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:14.206+0000 7f45deffd640 1 -- 172.21.15.43:0/2564073524 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(174..174 src has 1..174) v4 ==== 7114+0+0 (secure 0 0 0) 0x7f45d8088af0 con 0x7f45e80cd110 2023-08-09T14:52:14.206 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:14.206+0000 7f45dffff640 1 --2- 172.21.15.43:0/2564073524 >> [v2:172.21.15.203:6832/2959400181,v1:172.21.15.203:6833/2959400181] conn(0x7f45c806a0e0 0x7f45c806c5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T14:52:14.208 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:14.207+0000 7f45dffff640 1 --2- 172.21.15.43:0/2564073524 >> [v2:172.21.15.203:6832/2959400181,v1:172.21.15.203:6833/2959400181] conn(0x7f45c806a0e0 0x7f45c806c5a0 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7f45d0036000 tx=0x7f45d0030000 comp rx=0 tx=0).ready entity=mgr.7223 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-09T14:52:14.210 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:14.210+0000 7f45deffd640 1 -- 172.21.15.43:0/2564073524 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178066 (secure 0 0 0) 0x7f45d8053d60 con 0x7f45e80cd110 2023-08-09T14:52:14.435 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:14.434+0000 7f45e759e640 1 -- 172.21.15.43:0/2564073524 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f45e80739f0 con 0x7f45e80cd110 2023-08-09T14:52:14.436 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:14.435+0000 7f45deffd640 1 -- 172.21.15.43:0/2564073524 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 78 v78) v1 ==== 94+0+4880 (secure 0 0 0) 0x7f45d8021090 con 0x7f45e80cd110 2023-08-09T14:52:14.436 INFO:teuthology.orchestra.run.smithi043.stdout: 2023-08-09T14:52:14.436 INFO:teuthology.orchestra.run.smithi043.stdout:{"epoch":78,"default_fscid":9,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":7079,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.203:6836/807059349","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6834","nonce":807059349},{"type":"v1","addr":"172.21.15.203:6836","nonce":807059349}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":74},{"gid":7924,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.203:6837/3166829033","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6835","nonce":3166829033},{"type":"v1","addr":"172.21.15.203:6837","nonce":3166829033}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":74},{"gid":9642,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.43:6833/3803974919","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6832","nonce":3803974919},{"type":"v1","addr":"172.21.15.43:6833","nonce":3803974919}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":73}],"filesystems":[{"mdsmap":{"epoch":78,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-08-09T14:51:47.253704+0000","modified":"2023-08-09T14:51:50.806331+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":9648},"failed":[],"damaged":[],"stopped":[],"info":{"gid_9648":{"gid":9648,"name":"a","rank":0,"incarnation":75,"state":"up:active","state_seq":3,"addr":"172.21.15.43:6835/2435237803","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6834","nonce":2435237803},{"type":"v1","addr":"172.21.15.43:6835","nonce":2435237803}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[21],"metadata_pool":20,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":9}]} 2023-08-09T14:52:14.437 INFO:teuthology.orchestra.run.smithi043.stderr:dumped fsmap epoch 78 2023-08-09T14:52:14.438 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:14.438+0000 7f45e759e640 1 -- 172.21.15.43:0/2564073524 >> [v2:172.21.15.203:6832/2959400181,v1:172.21.15.203:6833/2959400181] conn(0x7f45c806a0e0 msgr2=0x7f45c806c5a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T14:52:14.439 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:14.438+0000 7f45e759e640 1 --2- 172.21.15.43:0/2564073524 >> [v2:172.21.15.203:6832/2959400181,v1:172.21.15.203:6833/2959400181] conn(0x7f45c806a0e0 0x7f45c806c5a0 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7f45d0036000 tx=0x7f45d0030000 comp rx=0 tx=0).stop 2023-08-09T14:52:14.439 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:14.438+0000 7f45e759e640 1 -- 172.21.15.43:0/2564073524 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f45e80cd110 msgr2=0x7f45e81f2210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T14:52:14.439 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:14.438+0000 7f45e759e640 1 --2- 172.21.15.43:0/2564073524 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f45e80cd110 0x7f45e81f2210 secure :-1 s=READY pgs=1962 cs=0 l=1 rev1=1 crypto rx=0x7f45d800e8a0 tx=0x7f45d800ed60 comp rx=0 tx=0).stop 2023-08-09T14:52:14.439 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:14.438+0000 7f45e759e640 1 -- 172.21.15.43:0/2564073524 shutdown_connections 2023-08-09T14:52:14.439 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:14.438+0000 7f45e759e640 1 --2- 172.21.15.43:0/2564073524 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f45e815c460 0x7f45e8156c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T14:52:14.439 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:14.438+0000 7f45e759e640 1 --2- 172.21.15.43:0/2564073524 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f45e80d1330 0x7f45e8156700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T14:52:14.439 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:14.438+0000 7f45e759e640 1 --2- 172.21.15.43:0/2564073524 >> [v2:172.21.15.203:6832/2959400181,v1:172.21.15.203:6833/2959400181] conn(0x7f45c806a0e0 0x7f45c806c5a0 unknown :-1 s=CLOSED pgs=29 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T14:52:14.440 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:14.438+0000 7f45e759e640 1 --2- 172.21.15.43:0/2564073524 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f45e80cd110 0x7f45e81f2210 unknown :-1 s=CLOSED pgs=1962 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T14:52:14.440 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:14.438+0000 7f45e759e640 1 -- 172.21.15.43:0/2564073524 >> 172.21.15.43:0/2564073524 conn(0x7f45e8071310 msgr2=0x7f45e815d600 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-09T14:52:14.440 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:14.438+0000 7f45e759e640 1 -- 172.21.15.43:0/2564073524 shutdown_connections 2023-08-09T14:52:14.440 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:14.438+0000 7f45e759e640 1 -- 172.21.15.43:0/2564073524 wait complete. 2023-08-09T14:52:14.458 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-08-09T14:52:14.680 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:14.679+0000 7f7ed3fff640 1 Processor -- start 2023-08-09T14:52:14.680 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:14.679+0000 7f7ed3fff640 1 -- start start 2023-08-09T14:52:14.681 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:14.681+0000 7f7ed3fff640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f7ed40c8b30 0x7f7ed40ad340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T14:52:14.682 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:14.681+0000 7f7ed3fff640 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7f7ed40acdf0 0x7f7ed415b2c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T14:52:14.682 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:14.681+0000 7f7ed3fff640 1 -- --> v1:172.21.15.203:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7ed40cbd90 con 0x7f7ed40c8750 2023-08-09T14:52:14.682 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:14.681+0000 7f7ed3fff640 1 -- --> v2:172.21.15.203:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7ed415e810 con 0x7f7ed40acdf0 2023-08-09T14:52:14.682 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:14.681+0000 7f7ed3fff640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7ed40c9f70 con 0x7f7ed40c8b30 2023-08-09T14:52:14.683 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:14.682+0000 7f7eca7fc640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f7ed40c8b30 0x7f7ed40ad340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T14:52:14.683 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:14.682+0000 7f7eca7fc640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f7ed40c8b30 0x7f7ed40ad340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.203:3301/0 says I am v2:172.21.15.43:39050/0 (socket says 172.21.15.43:39050) 2023-08-09T14:52:14.683 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:14.682+0000 7f7eca7fc640 1 -- 172.21.15.43:0/2971995421 learned_addr learned my addr 172.21.15.43:0/2971995421 (peer_addr_for_me v2:172.21.15.43:0/0) 2023-08-09T14:52:14.683 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:14.682+0000 7f7ed37fe640 1 --2- 172.21.15.43:0/2971995421 >> v2:172.21.15.203:3300/0 conn(0x7f7ed40acdf0 0x7f7ed415b2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T14:52:14.683 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:14.683+0000 7f7eca7fc640 1 -- 172.21.15.43:0/2971995421 >> v2:172.21.15.203:3300/0 conn(0x7f7ed40acdf0 msgr2=0x7f7ed415b2c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-09T14:52:14.684 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:14.683+0000 7f7eca7fc640 1 --2- 172.21.15.43:0/2971995421 >> v2:172.21.15.203:3300/0 conn(0x7f7ed40acdf0 0x7f7ed415b2c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T14:52:14.684 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:14.683+0000 7f7eca7fc640 1 -- 172.21.15.43:0/2971995421 >> v1:172.21.15.203:6789/0 conn(0x7f7ed40c8750 legacy=0x7f7ed406f410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-09T14:52:14.684 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:14.683+0000 7f7eca7fc640 1 -- 172.21.15.43:0/2971995421 --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7ed40ca2b0 con 0x7f7ed40c8b30 2023-08-09T14:52:14.685 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:14.684+0000 7f7eca7fc640 1 --2- 172.21.15.43:0/2971995421 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f7ed40c8b30 0x7f7ed40ad340 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7f7ebc009e50 tx=0x7f7ebc02be50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=4eba28c504adef4c server_cookie=0 in_seq=0 out_seq=0 2023-08-09T14:52:14.685 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:14.684+0000 7f7ed27fc640 1 -- 172.21.15.43:0/2971995421 <== mon.2 v2:172.21.15.203:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7ebc02c9f0 con 0x7f7ed40c8b30 2023-08-09T14:52:14.685 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:14.684+0000 7f7ed27fc640 1 -- 172.21.15.43:0/2971995421 <== mon.2 v2:172.21.15.203:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7ebc02cb50 con 0x7f7ed40c8b30 2023-08-09T14:52:14.685 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:14.684+0000 7f7ed27fc640 1 -- 172.21.15.43:0/2971995421 <== mon.2 v2:172.21.15.203:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7ebc0343f0 con 0x7f7ed40c8b30 2023-08-09T14:52:14.686 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:14.685+0000 7f7ed3fff640 1 -- 172.21.15.43:0/2971995421 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f7ed40c8b30 msgr2=0x7f7ed40ad340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T14:52:14.686 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:14.685+0000 7f7ed3fff640 1 --2- 172.21.15.43:0/2971995421 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f7ed40c8b30 0x7f7ed40ad340 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7f7ebc009e50 tx=0x7f7ebc02be50 comp rx=0 tx=0).stop 2023-08-09T14:52:14.686 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:14.685+0000 7f7ed3fff640 1 -- 172.21.15.43:0/2971995421 shutdown_connections 2023-08-09T14:52:14.686 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:14.685+0000 7f7ed3fff640 1 --2- 172.21.15.43:0/2971995421 >> v2:172.21.15.203:3300/0 conn(0x7f7ed40acdf0 0x7f7ed415b2c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T14:52:14.687 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:14.685+0000 7f7ed3fff640 1 --2- 172.21.15.43:0/2971995421 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f7ed40c8b30 0x7f7ed40ad340 unknown :-1 s=CLOSED pgs=272 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T14:52:14.687 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:14.685+0000 7f7ed3fff640 1 -- 172.21.15.43:0/2971995421 >> 172.21.15.43:0/2971995421 conn(0x7f7ed40cde90 msgr2=0x7f7ed40ce2a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-09T14:52:14.687 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:14.685+0000 7f7ed3fff640 1 -- 172.21.15.43:0/2971995421 shutdown_connections 2023-08-09T14:52:14.687 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:14.686+0000 7f7ed3fff640 1 -- 172.21.15.43:0/2971995421 wait complete. 2023-08-09T14:52:14.688 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:14.688+0000 7f7ed3fff640 1 Processor -- start 2023-08-09T14:52:14.688 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:14.688+0000 7f7ed3fff640 1 -- start start 2023-08-09T14:52:14.690 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:14.689+0000 7f7ed3fff640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f7ed40acdf0 0x7f7ed415d2d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T14:52:14.690 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:14.689+0000 7f7ed3fff640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f7ed40c8750 0x7f7ed40b0ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T14:52:14.690 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:14.689+0000 7f7ed3fff640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f7ed40c8b30 0x7f7ed41fa440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T14:52:14.690 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:14.689+0000 7f7ed3fff640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7ed415d810 con 0x7f7ed40c8750 2023-08-09T14:52:14.690 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:14.689+0000 7f7ed3fff640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7ed41f77c0 con 0x7f7ed40acdf0 2023-08-09T14:52:14.691 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:14.689+0000 7f7ed3fff640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7ed41f6f20 con 0x7f7ed40c8b30 2023-08-09T14:52:14.691 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:14.690+0000 7f7eca7fc640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f7ed40c8750 0x7f7ed40b0ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T14:52:14.691 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:14.690+0000 7f7eca7fc640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f7ed40c8750 0x7f7ed40b0ab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:35102/0 (socket says 172.21.15.43:35102) 2023-08-09T14:52:14.691 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:14.690+0000 7f7eca7fc640 1 -- 172.21.15.43:0/1196541951 learned_addr learned my addr 172.21.15.43:0/1196541951 (peer_addr_for_me v2:172.21.15.43:0/0) 2023-08-09T14:52:14.692 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:14.690+0000 7f7ed2ffd640 1 --2- 172.21.15.43:0/1196541951 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f7ed40acdf0 0x7f7ed415d2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T14:52:14.692 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:14.690+0000 7f7ed37fe640 1 --2- 172.21.15.43:0/1196541951 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f7ed40c8b30 0x7f7ed41fa440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T14:52:14.692 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:14.690+0000 7f7eca7fc640 1 -- 172.21.15.43:0/1196541951 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f7ed40c8b30 msgr2=0x7f7ed41fa440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T14:52:14.693 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:14.690+0000 7f7eca7fc640 1 --2- 172.21.15.43:0/1196541951 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f7ed40c8b30 0x7f7ed41fa440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T14:52:14.693 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:14.690+0000 7f7eca7fc640 1 -- 172.21.15.43:0/1196541951 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f7ed40acdf0 msgr2=0x7f7ed415d2d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T14:52:14.693 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:14.690+0000 7f7eca7fc640 1 --2- 172.21.15.43:0/1196541951 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f7ed40acdf0 0x7f7ed415d2d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T14:52:14.693 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:14.690+0000 7f7eca7fc640 1 -- 172.21.15.43:0/1196541951 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7ebc009b30 con 0x7f7ed40c8750 2023-08-09T14:52:14.693 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:14.690+0000 7f7ed37fe640 1 --2- 172.21.15.43:0/1196541951 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f7ed40c8b30 0x7f7ed41fa440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-08-09T14:52:14.693 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:14.690+0000 7f7ed2ffd640 1 --2- 172.21.15.43:0/1196541951 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f7ed40acdf0 0x7f7ed415d2d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-08-09T14:52:14.693 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:14.691+0000 7f7eca7fc640 1 --2- 172.21.15.43:0/1196541951 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f7ed40c8750 0x7f7ed40b0ab0 secure :-1 s=READY pgs=1963 cs=0 l=1 rev1=1 crypto rx=0x7f7ebc009e50 tx=0x7f7ebc004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-09T14:52:14.694 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:14.691+0000 7f7ecbfff640 1 -- 172.21.15.43:0/1196541951 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7ebc040670 con 0x7f7ed40c8750 2023-08-09T14:52:14.694 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:14.691+0000 7f7ecbfff640 1 -- 172.21.15.43:0/1196541951 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7ebc0407f0 con 0x7f7ed40c8750 2023-08-09T14:52:14.694 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:14.691+0000 7f7ecbfff640 1 -- 172.21.15.43:0/1196541951 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7ebc040ba0 con 0x7f7ed40c8750 2023-08-09T14:52:14.694 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:14.691+0000 7f7ed3fff640 1 -- 172.21.15.43:0/1196541951 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7ed41f7200 con 0x7f7ed40c8750 2023-08-09T14:52:14.694 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:14.691+0000 7f7ed3fff640 1 -- 172.21.15.43:0/1196541951 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7ed41f9f20 con 0x7f7ed40c8750 2023-08-09T14:52:14.694 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:14.692+0000 7f7ed3fff640 1 -- 172.21.15.43:0/1196541951 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7ed40c9230 con 0x7f7ed40c8750 2023-08-09T14:52:14.695 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:14.693+0000 7f7ecbfff640 1 -- 172.21.15.43:0/1196541951 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 88) v1 ==== 88443+0+0 (secure 0 0 0) 0x7f7ebc040d00 con 0x7f7ed40c8750 2023-08-09T14:52:14.695 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:14.695+0000 7f7ecbfff640 1 --2- 172.21.15.43:0/1196541951 >> [v2:172.21.15.203:6832/2959400181,v1:172.21.15.203:6833/2959400181] conn(0x7f7eb406a170 0x7f7eb406c630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T14:52:14.696 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:14.695+0000 7f7ecbfff640 1 -- 172.21.15.43:0/1196541951 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(174..174 src has 1..174) v4 ==== 7114+0+0 (secure 0 0 0) 0x7f7ebc0abb30 con 0x7f7ed40c8750 2023-08-09T14:52:14.696 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:14.695+0000 7f7ed2ffd640 1 --2- 172.21.15.43:0/1196541951 >> [v2:172.21.15.203:6832/2959400181,v1:172.21.15.203:6833/2959400181] conn(0x7f7eb406a170 0x7f7eb406c630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T14:52:14.697 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:14.697+0000 7f7ed2ffd640 1 --2- 172.21.15.43:0/1196541951 >> [v2:172.21.15.203:6832/2959400181,v1:172.21.15.203:6833/2959400181] conn(0x7f7eb406a170 0x7f7eb406c630 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0x7f7ec4006fd0 tx=0x7f7ec4008040 comp rx=0 tx=0).ready entity=mgr.7223 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-09T14:52:14.699 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:14.699+0000 7f7ecbfff640 1 -- 172.21.15.43:0/1196541951 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178066 (secure 0 0 0) 0x7f7ebc076ce0 con 0x7f7ed40c8750 2023-08-09T14:52:14.896 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:14.895+0000 7f7ed3fff640 1 -- 172.21.15.43:0/1196541951 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f7ed40739f0 con 0x7f7ed40c8750 2023-08-09T14:52:14.899 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:14.899+0000 7f7ecbfff640 1 -- 172.21.15.43:0/1196541951 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v174) v1 ==== 74+0+19582 (secure 0 0 0) 0x7f7ebc0a9e60 con 0x7f7ed40c8750 2023-08-09T14:52:14.900 INFO:teuthology.orchestra.run.smithi043.stdout: 2023-08-09T14:52:14.900 INFO:teuthology.orchestra.run.smithi043.stdout:{"epoch":174,"fsid":"daac87ee-1337-4250-a955-aea4e408a39b","created":"2023-08-09T13:33:41.701198+0000","modified":"2023-08-09T14:52:10.059650+0000","last_up_change":"2023-08-09T13:33:53.783453+0000","last_in_change":"2023-08-09T13:33:45.148904+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":21,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":"rbd","create_time":"2023-08-09T13:33:54.365137+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":".mgr","create_time":"2023-08-09T13:33:54.831803+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":20,"pool_name":"cephfs_metadata","create_time":"2023-08-09T14:51:45.692613+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"171","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":21,"pool_name":"cephfs_data","create_time":"2023-08-09T14:51:46.286715+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"172","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"cbd49ecd-e896-4fbd-864e-5c37a829b165","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":170,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6800","nonce":502667335},{"type":"v1","addr":"172.21.15.43:6801","nonce":502667335}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6802","nonce":502667335},{"type":"v1","addr":"172.21.15.43:6803","nonce":502667335}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6806","nonce":502667335},{"type":"v1","addr":"172.21.15.43:6807","nonce":502667335}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6804","nonce":502667335},{"type":"v1","addr":"172.21.15.43:6805","nonce":502667335}]},"public_addr":"172.21.15.43:6801/502667335","cluster_addr":"172.21.15.43:6803/502667335","heartbeat_back_addr":"172.21.15.43:6807/502667335","heartbeat_front_addr":"172.21.15.43:6805/502667335","state":["exists","up"]},{"osd":1,"uuid":"75c7e0fe-1ec7-4fe8-a2f2-3661a65b8500","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":170,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6808","nonce":3289134610},{"type":"v1","addr":"172.21.15.43:6809","nonce":3289134610}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6810","nonce":3289134610},{"type":"v1","addr":"172.21.15.43:6812","nonce":3289134610}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6821","nonce":3289134610},{"type":"v1","addr":"172.21.15.43:6824","nonce":3289134610}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6815","nonce":3289134610},{"type":"v1","addr":"172.21.15.43:6818","nonce":3289134610}]},"public_addr":"172.21.15.43:6809/3289134610","cluster_addr":"172.21.15.43:6812/3289134610","heartbeat_back_addr":"172.21.15.43:6824/3289134610","heartbeat_front_addr":"172.21.15.43:6818/3289134610","state":["exists","up"]},{"osd":2,"uuid":"bb5c41e9-7f5f-4125-9b01-3bb8bfc2ea83","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":170,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6811","nonce":4079709736},{"type":"v1","addr":"172.21.15.43:6814","nonce":4079709736}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6817","nonce":4079709736},{"type":"v1","addr":"172.21.15.43:6820","nonce":4079709736}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6829","nonce":4079709736},{"type":"v1","addr":"172.21.15.43:6831","nonce":4079709736}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6823","nonce":4079709736},{"type":"v1","addr":"172.21.15.43:6826","nonce":4079709736}]},"public_addr":"172.21.15.43:6814/4079709736","cluster_addr":"172.21.15.43:6820/4079709736","heartbeat_back_addr":"172.21.15.43:6831/4079709736","heartbeat_front_addr":"172.21.15.43:6826/4079709736","state":["exists","up"]},{"osd":3,"uuid":"bfe4c8b4-694a-465c-87e6-2b2aab852613","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":170,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6813","nonce":1709538759},{"type":"v1","addr":"172.21.15.43:6816","nonce":1709538759}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6819","nonce":1709538759},{"type":"v1","addr":"172.21.15.43:6822","nonce":1709538759}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6828","nonce":1709538759},{"type":"v1","addr":"172.21.15.43:6830","nonce":1709538759}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6825","nonce":1709538759},{"type":"v1","addr":"172.21.15.43:6827","nonce":1709538759}]},"public_addr":"172.21.15.43:6816/1709538759","cluster_addr":"172.21.15.43:6822/1709538759","heartbeat_back_addr":"172.21.15.43:6830/1709538759","heartbeat_front_addr":"172.21.15.43:6827/1709538759","state":["exists","up"]},{"osd":4,"uuid":"fa2e8a77-aba1-4756-853e-0d174a5bdeba","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":170,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6800","nonce":4196866481},{"type":"v1","addr":"172.21.15.203:6801","nonce":4196866481}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6802","nonce":4196866481},{"type":"v1","addr":"172.21.15.203:6803","nonce":4196866481}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6806","nonce":4196866481},{"type":"v1","addr":"172.21.15.203:6807","nonce":4196866481}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6804","nonce":4196866481},{"type":"v1","addr":"172.21.15.203:6805","nonce":4196866481}]},"public_addr":"172.21.15.203:6801/4196866481","cluster_addr":"172.21.15.203:6803/4196866481","heartbeat_back_addr":"172.21.15.203:6807/4196866481","heartbeat_front_addr":"172.21.15.203:6805/4196866481","state":["exists","up"]},{"osd":5,"uuid":"81811e4b-4d24-4f51-a82c-f1d700a2a00d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":170,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6816","nonce":3059290081},{"type":"v1","addr":"172.21.15.203:6817","nonce":3059290081}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6818","nonce":3059290081},{"type":"v1","addr":"172.21.15.203:6819","nonce":3059290081}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6822","nonce":3059290081},{"type":"v1","addr":"172.21.15.203:6823","nonce":3059290081}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6820","nonce":3059290081},{"type":"v1","addr":"172.21.15.203:6821","nonce":3059290081}]},"public_addr":"172.21.15.203:6817/3059290081","cluster_addr":"172.21.15.203:6819/3059290081","heartbeat_back_addr":"172.21.15.203:6823/3059290081","heartbeat_front_addr":"172.21.15.203:6821/3059290081","state":["exists","up"]},{"osd":6,"uuid":"307bac4d-6a5b-443d-af75-56d695d7674f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":170,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6824","nonce":512155569},{"type":"v1","addr":"172.21.15.203:6825","nonce":512155569}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6826","nonce":512155569},{"type":"v1","addr":"172.21.15.203:6827","nonce":512155569}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6830","nonce":512155569},{"type":"v1","addr":"172.21.15.203:6831","nonce":512155569}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6828","nonce":512155569},{"type":"v1","addr":"172.21.15.203:6829","nonce":512155569}]},"public_addr":"172.21.15.203:6825/512155569","cluster_addr":"172.21.15.203:6827/512155569","heartbeat_back_addr":"172.21.15.203:6831/512155569","heartbeat_front_addr":"172.21.15.203:6829/512155569","state":["exists","up"]},{"osd":7,"uuid":"a95dc7ca-9f1b-4295-af70-68a22a2d7d5e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":170,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6808","nonce":3331450438},{"type":"v1","addr":"172.21.15.203:6809","nonce":3331450438}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6810","nonce":3331450438},{"type":"v1","addr":"172.21.15.203:6811","nonce":3331450438}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6814","nonce":3331450438},{"type":"v1","addr":"172.21.15.203:6815","nonce":3331450438}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6812","nonce":3331450438},{"type":"v1","addr":"172.21.15.203:6813","nonce":3331450438}]},"public_addr":"172.21.15.203:6809/3331450438","cluster_addr":"172.21.15.203:6811/3331450438","heartbeat_back_addr":"172.21.15.203:6815/3331450438","heartbeat_front_addr":"172.21.15.203:6813/3331450438","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-09T13:33:50.547652+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-09T13:33:50.589469+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-09T13:33:50.561724+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-09T13:33:50.599650+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-09T13:33:50.411855+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-09T13:33:50.219681+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-09T13:33:50.504664+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-09T13:33:50.441248+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.203:0/3301934490":"2023-08-10T14:52:10.059559+0000","172.21.15.203:0/1715525748":"2023-08-10T14:52:10.059559+0000","172.21.15.203:0/1477488908":"2023-08-10T14:52:10.059559+0000","172.21.15.203:6833/843920641":"2023-08-10T14:52:10.059559+0000","172.21.15.203:6832/843920641":"2023-08-10T14:52:10.059559+0000","172.21.15.203:0/2095212004":"2023-08-10T14:52:02.909250+0000","172.21.15.203:0/1196801979":"2023-08-10T14:52:10.059559+0000","172.21.15.203:0/3749872091":"2023-08-10T14:52:02.909250+0000","172.21.15.203:0/1305485142":"2023-08-10T14:52:10.059559+0000","172.21.15.203:0/651612122":"2023-08-10T14:52:02.909250+0000","172.21.15.203:0/4120954828":"2023-08-10T14:52:02.909250+0000","172.21.15.203:0/3552079285":"2023-08-10T14:52:10.059559+0000","172.21.15.203:0/2783782601":"2023-08-10T14:52:02.909250+0000","172.21.15.203:0/2232659577":"2023-08-10T14:52:02.909250+0000","172.21.15.203:6833/2566816332":"2023-08-10T14:52:02.909250+0000","172.21.15.203:6832/2566816332":"2023-08-10T14:52:02.909250+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":7,"snaps":[{"begin":104,"length":1},{"begin":221,"length":1}]}],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-08-09T14:52:14.902 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:14.902+0000 7f7ed3fff640 1 -- 172.21.15.43:0/1196541951 >> [v2:172.21.15.203:6832/2959400181,v1:172.21.15.203:6833/2959400181] conn(0x7f7eb406a170 msgr2=0x7f7eb406c630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T14:52:14.902 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:14.902+0000 7f7ed3fff640 1 --2- 172.21.15.43:0/1196541951 >> [v2:172.21.15.203:6832/2959400181,v1:172.21.15.203:6833/2959400181] conn(0x7f7eb406a170 0x7f7eb406c630 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0x7f7ec4006fd0 tx=0x7f7ec4008040 comp rx=0 tx=0).stop 2023-08-09T14:52:14.903 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:14.902+0000 7f7ed3fff640 1 -- 172.21.15.43:0/1196541951 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f7ed40c8750 msgr2=0x7f7ed40b0ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T14:52:14.903 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:14.902+0000 7f7ed3fff640 1 --2- 172.21.15.43:0/1196541951 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f7ed40c8750 0x7f7ed40b0ab0 secure :-1 s=READY pgs=1963 cs=0 l=1 rev1=1 crypto rx=0x7f7ebc009e50 tx=0x7f7ebc004a40 comp rx=0 tx=0).stop 2023-08-09T14:52:14.903 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:14.902+0000 7f7ed3fff640 1 -- 172.21.15.43:0/1196541951 shutdown_connections 2023-08-09T14:52:14.903 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:14.902+0000 7f7ed3fff640 1 --2- 172.21.15.43:0/1196541951 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f7ed40c8b30 0x7f7ed41fa440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T14:52:14.903 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:14.902+0000 7f7ed3fff640 1 --2- 172.21.15.43:0/1196541951 >> [v2:172.21.15.203:6832/2959400181,v1:172.21.15.203:6833/2959400181] conn(0x7f7eb406a170 0x7f7eb406c630 unknown :-1 s=CLOSED pgs=30 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T14:52:14.903 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:14.902+0000 7f7ed3fff640 1 --2- 172.21.15.43:0/1196541951 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f7ed40c8750 0x7f7ed40b0ab0 unknown :-1 s=CLOSED pgs=1963 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T14:52:14.903 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:14.902+0000 7f7ed3fff640 1 --2- 172.21.15.43:0/1196541951 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f7ed40acdf0 0x7f7ed415d2d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T14:52:14.903 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:14.902+0000 7f7ed3fff640 1 -- 172.21.15.43:0/1196541951 >> 172.21.15.43:0/1196541951 conn(0x7f7ed40cde90 msgr2=0x7f7ed415ae30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-09T14:52:14.904 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:14.902+0000 7f7ed3fff640 1 -- 172.21.15.43:0/1196541951 shutdown_connections 2023-08-09T14:52:14.904 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:14.902+0000 7f7ed3fff640 1 -- 172.21.15.43:0/1196541951 wait complete. 2023-08-09T14:52:14.923 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2023-08-09T14:52:15.135 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:15.135+0000 7f8c8d302640 1 Processor -- start 2023-08-09T14:52:15.135 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:15.135+0000 7f8c8d302640 1 -- start start 2023-08-09T14:52:15.137 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:15.137+0000 7f8c8d302640 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7f8c88070ab0 0x7f8c88070f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T14:52:15.137 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:15.137+0000 7f8c8d302640 1 -- --> v1:172.21.15.43:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8c8806d770 con 0x7f8c881546f0 2023-08-09T14:52:15.137 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:15.137+0000 7f8c8d302640 1 -- --> v1:172.21.15.203:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8c8806bae0 con 0x7f8c8815b460 2023-08-09T14:52:15.138 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:15.137+0000 7f8c8d302640 1 -- --> v2:172.21.15.203:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8c88071480 con 0x7f8c88070ab0 2023-08-09T14:52:15.138 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:15.137+0000 7f8c87fff640 1 --1- >> v1:172.21.15.43:6789/0 conn(0x7f8c881546f0 0x7f8c8815e640 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.43:6789/0 says I am v1:172.21.15.43:48374/0 (socket says 172.21.15.43:48374) 2023-08-09T14:52:15.138 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:15.137+0000 7f8c87fff640 1 -- 172.21.15.43:0/4082386306 learned_addr learned my addr 172.21.15.43:0/4082386306 (peer_addr_for_me v1:172.21.15.43:0/0) 2023-08-09T14:52:15.138 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:15.137+0000 7f8c8cb01640 1 --2- 172.21.15.43:0/4082386306 >> v2:172.21.15.203:3300/0 conn(0x7f8c88070ab0 0x7f8c88070f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T14:52:15.139 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:15.138+0000 7f8c8cb01640 1 -- 172.21.15.43:0/4082386306 >> v1:172.21.15.203:6789/0 conn(0x7f8c8815b460 legacy=0x7f8c8806b3d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T14:52:15.139 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:15.138+0000 7f8c8cb01640 1 -- 172.21.15.43:0/4082386306 >> v1:172.21.15.43:6789/0 conn(0x7f8c881546f0 legacy=0x7f8c8815e640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T14:52:15.139 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:15.138+0000 7f8c8cb01640 1 -- 172.21.15.43:0/4082386306 --> v2:172.21.15.203:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8c8806cf30 con 0x7f8c88070ab0 2023-08-09T14:52:15.141 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:15.140+0000 7f8c8cb01640 1 --2- 172.21.15.43:0/4082386306 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f8c88070ab0 0x7f8c88070f40 secure :-1 s=READY pgs=726 cs=0 l=1 rev1=1 crypto rx=0x7f8c800097d0 tx=0x7f8c8002b8b0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=1dd6f833212679a6 server_cookie=0 in_seq=0 out_seq=0 2023-08-09T14:52:15.145 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:15.145+0000 7f8c86ffd640 1 -- 172.21.15.43:0/4082386306 <== mon.1 v2:172.21.15.203:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8c8002d040 con 0x7f8c88070ab0 2023-08-09T14:52:15.145 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:15.145+0000 7f8c86ffd640 1 -- 172.21.15.43:0/4082386306 <== mon.1 v2:172.21.15.203:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8c80031810 con 0x7f8c88070ab0 2023-08-09T14:52:15.146 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:15.145+0000 7f8c86ffd640 1 -- 172.21.15.43:0/4082386306 <== mon.1 v2:172.21.15.203:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8c80031ba0 con 0x7f8c88070ab0 2023-08-09T14:52:15.146 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:15.146+0000 7f8c8d302640 1 -- 172.21.15.43:0/4082386306 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f8c88070ab0 msgr2=0x7f8c88070f40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T14:52:15.146 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:15.146+0000 7f8c8d302640 1 --2- 172.21.15.43:0/4082386306 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f8c88070ab0 0x7f8c88070f40 secure :-1 s=READY pgs=726 cs=0 l=1 rev1=1 crypto rx=0x7f8c800097d0 tx=0x7f8c8002b8b0 comp rx=0 tx=0).stop 2023-08-09T14:52:15.147 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:15.146+0000 7f8c8d302640 1 -- 172.21.15.43:0/4082386306 shutdown_connections 2023-08-09T14:52:15.147 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:15.146+0000 7f8c8d302640 1 --2- 172.21.15.43:0/4082386306 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f8c88070ab0 0x7f8c88070f40 unknown :-1 s=CLOSED pgs=726 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T14:52:15.147 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:15.146+0000 7f8c8d302640 1 -- 172.21.15.43:0/4082386306 >> 172.21.15.43:0/4082386306 conn(0x7f8c880af950 msgr2=0x7f8c880afd60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-09T14:52:15.147 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:15.146+0000 7f8c8d302640 1 -- 172.21.15.43:0/4082386306 shutdown_connections 2023-08-09T14:52:15.147 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:15.146+0000 7f8c8d302640 1 -- 172.21.15.43:0/4082386306 wait complete. 2023-08-09T14:52:15.148 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:15.148+0000 7f8c8d302640 1 Processor -- start 2023-08-09T14:52:15.148 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:15.148+0000 7f8c8d302640 1 -- start start 2023-08-09T14:52:15.149 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:15.149+0000 7f8c8d302640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f8c88070ab0 0x7f8c881f21b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T14:52:15.150 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:15.149+0000 7f8c8d302640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f8c881546f0 0x7f8c881460e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T14:52:15.150 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:15.149+0000 7f8c8d302640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f8c880c4270 0x7f8c88158ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T14:52:15.150 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:15.149+0000 7f8c8d302640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8c881f26f0 con 0x7f8c88070ab0 2023-08-09T14:52:15.150 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:15.149+0000 7f8c8d302640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8c880b2df0 con 0x7f8c881546f0 2023-08-09T14:52:15.151 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:15.149+0000 7f8c8d302640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8c880b30d0 con 0x7f8c880c4270 2023-08-09T14:52:15.151 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:15.149+0000 7f8c87fff640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f8c88070ab0 0x7f8c881f21b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T14:52:15.151 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:15.149+0000 7f8c87fff640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f8c88070ab0 0x7f8c881f21b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:37780/0 (socket says 172.21.15.43:37780) 2023-08-09T14:52:15.151 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:15.149+0000 7f8c87fff640 1 -- 172.21.15.43:0/2249060109 learned_addr learned my addr 172.21.15.43:0/2249060109 (peer_addr_for_me v2:172.21.15.43:0/0) 2023-08-09T14:52:15.153 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:15.150+0000 7f8c877fe640 1 --2- 172.21.15.43:0/2249060109 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f8c881546f0 0x7f8c881460e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T14:52:15.154 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:15.150+0000 7f8c87fff640 1 -- 172.21.15.43:0/2249060109 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f8c880c4270 msgr2=0x7f8c88158ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T14:52:15.154 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:15.150+0000 7f8c87fff640 1 --2- 172.21.15.43:0/2249060109 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f8c880c4270 0x7f8c88158ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T14:52:15.154 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:15.150+0000 7f8c87fff640 1 -- 172.21.15.43:0/2249060109 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f8c881546f0 msgr2=0x7f8c881460e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T14:52:15.154 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:15.150+0000 7f8c87fff640 1 --2- 172.21.15.43:0/2249060109 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f8c881546f0 0x7f8c881460e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T14:52:15.155 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:15.150+0000 7f8c87fff640 1 -- 172.21.15.43:0/2249060109 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8c80009400 con 0x7f8c88070ab0 2023-08-09T14:52:15.155 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:15.150+0000 7f8c877fe640 1 --2- 172.21.15.43:0/2249060109 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f8c881546f0 0x7f8c881460e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-08-09T14:52:15.155 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:15.150+0000 7f8c87fff640 1 --2- 172.21.15.43:0/2249060109 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f8c88070ab0 0x7f8c881f21b0 secure :-1 s=READY pgs=1965 cs=0 l=1 rev1=1 crypto rx=0x7f8c7800cb20 tx=0x7f8c7800a060 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-09T14:52:15.156 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:15.150+0000 7f8c84ff9640 1 -- 172.21.15.43:0/2249060109 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8c7800aca0 con 0x7f8c88070ab0 2023-08-09T14:52:15.156 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:15.150+0000 7f8c84ff9640 1 -- 172.21.15.43:0/2249060109 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8c7800ae00 con 0x7f8c88070ab0 2023-08-09T14:52:15.156 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:15.150+0000 7f8c84ff9640 1 -- 172.21.15.43:0/2249060109 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8c78012600 con 0x7f8c88070ab0 2023-08-09T14:52:15.156 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:15.150+0000 7f8c8d302640 1 -- 172.21.15.43:0/2249060109 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8c880b33b0 con 0x7f8c88070ab0 2023-08-09T14:52:15.157 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:15.151+0000 7f8c8d302640 1 -- 172.21.15.43:0/2249060109 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8c88146620 con 0x7f8c88070ab0 2023-08-09T14:52:15.157 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:15.152+0000 7f8c8d302640 1 -- 172.21.15.43:0/2249060109 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8c880739f0 con 0x7f8c88070ab0 2023-08-09T14:52:15.157 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:15.153+0000 7f8c84ff9640 1 -- 172.21.15.43:0/2249060109 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 88) v1 ==== 88443+0+0 (secure 0 0 0) 0x7f8c78012760 con 0x7f8c88070ab0 2023-08-09T14:52:15.158 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:15.154+0000 7f8c84ff9640 1 --2- 172.21.15.43:0/2249060109 >> [v2:172.21.15.203:6832/2959400181,v1:172.21.15.203:6833/2959400181] conn(0x7f8c7406a170 0x7f8c7406c630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T14:52:15.158 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:15.154+0000 7f8c84ff9640 1 -- 172.21.15.43:0/2249060109 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(174..174 src has 1..174) v4 ==== 7114+0+0 (secure 0 0 0) 0x7f8c78088850 con 0x7f8c88070ab0 2023-08-09T14:52:15.158 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:15.155+0000 7f8c877fe640 1 --2- 172.21.15.43:0/2249060109 >> [v2:172.21.15.203:6832/2959400181,v1:172.21.15.203:6833/2959400181] conn(0x7f8c7406a170 0x7f8c7406c630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T14:52:15.158 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:15.156+0000 7f8c877fe640 1 --2- 172.21.15.43:0/2249060109 >> [v2:172.21.15.203:6832/2959400181,v1:172.21.15.203:6833/2959400181] conn(0x7f8c7406a170 0x7f8c7406c630 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0x7f8c88159af0 tx=0x7f8c7c009040 comp rx=0 tx=0).ready entity=mgr.7223 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-09T14:52:15.159 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:15.158+0000 7f8c84ff9640 1 -- 172.21.15.43:0/2249060109 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178066 (secure 0 0 0) 0x7f8c78053ac0 con 0x7f8c88070ab0 2023-08-09T14:52:15.389 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:15.388+0000 7f8c8d302640 1 -- 172.21.15.43:0/2249060109 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) v1 -- 0x7f8c8806d770 con 0x7f8c88070ab0 2023-08-09T14:52:15.390 INFO:tasks.ceph.mon.a.smithi043.stderr:2023-08-09T14:52:15.390+0000 7f32867f3640 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2023-08-09T14:52:15.400 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:15.399+0000 7f8c84ff9640 1 -- 172.21.15.43:0/2249060109 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v79) v1 ==== 161+0+0 (secure 0 0 0) 0x7f8c8806d770 con 0x7f8c88070ab0 2023-08-09T14:52:15.400 INFO:teuthology.orchestra.run.smithi043.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2023-08-09T14:52:15.404 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:15.403+0000 7f8c8d302640 1 -- 172.21.15.43:0/2249060109 >> [v2:172.21.15.203:6832/2959400181,v1:172.21.15.203:6833/2959400181] conn(0x7f8c7406a170 msgr2=0x7f8c7406c630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T14:52:15.404 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:15.403+0000 7f8c8d302640 1 --2- 172.21.15.43:0/2249060109 >> [v2:172.21.15.203:6832/2959400181,v1:172.21.15.203:6833/2959400181] conn(0x7f8c7406a170 0x7f8c7406c630 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0x7f8c88159af0 tx=0x7f8c7c009040 comp rx=0 tx=0).stop 2023-08-09T14:52:15.404 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:15.403+0000 7f8c8d302640 1 -- 172.21.15.43:0/2249060109 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f8c88070ab0 msgr2=0x7f8c881f21b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T14:52:15.404 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:15.403+0000 7f8c8d302640 1 --2- 172.21.15.43:0/2249060109 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f8c88070ab0 0x7f8c881f21b0 secure :-1 s=READY pgs=1965 cs=0 l=1 rev1=1 crypto rx=0x7f8c7800cb20 tx=0x7f8c7800a060 comp rx=0 tx=0).stop 2023-08-09T14:52:15.404 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:15.404+0000 7f8c8d302640 1 -- 172.21.15.43:0/2249060109 shutdown_connections 2023-08-09T14:52:15.405 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:15.404+0000 7f8c8d302640 1 --2- 172.21.15.43:0/2249060109 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f8c880c4270 0x7f8c88158ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T14:52:15.405 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:15.404+0000 7f8c8d302640 1 --2- 172.21.15.43:0/2249060109 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f8c881546f0 0x7f8c881460e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T14:52:15.405 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:15.404+0000 7f8c8d302640 1 --2- 172.21.15.43:0/2249060109 >> [v2:172.21.15.203:6832/2959400181,v1:172.21.15.203:6833/2959400181] conn(0x7f8c7406a170 0x7f8c7406c630 unknown :-1 s=CLOSED pgs=31 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T14:52:15.406 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:15.404+0000 7f8c8d302640 1 --2- 172.21.15.43:0/2249060109 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f8c88070ab0 0x7f8c881f21b0 unknown :-1 s=CLOSED pgs=1965 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T14:52:15.406 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:15.404+0000 7f8c8d302640 1 -- 172.21.15.43:0/2249060109 >> 172.21.15.43:0/2249060109 conn(0x7f8c880af950 msgr2=0x7f8c88151ce0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-09T14:52:15.406 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:15.404+0000 7f8c8d302640 1 -- 172.21.15.43:0/2249060109 shutdown_connections 2023-08-09T14:52:15.406 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:15.404+0000 7f8c8d302640 1 -- 172.21.15.43:0/2249060109 wait complete. 2023-08-09T14:52:15.410 INFO:tasks.ceph.mds.a.smithi043.stderr: -3335> 2023-08-09T14:51:48.729+0000 7f4d461e1640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2023-08-09T14:52:15.430 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs rm cephfs --yes-i-really-mean-it 2023-08-09T14:52:15.470 INFO:tasks.ceph.mds.a.smithi043.stdout:starting mds.a at 2023-08-09T14:52:15.636 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:15.635+0000 7f50cffff640 1 Processor -- start 2023-08-09T14:52:15.636 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:15.635+0000 7f50cffff640 1 -- start start 2023-08-09T14:52:15.637 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:15.637+0000 7f50cffff640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f50d00c9ee0 0x7f50d015a800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T14:52:15.638 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:15.637+0000 7f50cffff640 1 -- --> v1:172.21.15.43:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f50d0158280 con 0x7f50d00c90a0 2023-08-09T14:52:15.638 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:15.637+0000 7f50cffff640 1 -- --> v1:172.21.15.203:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f50d015b560 con 0x7f50d0157ea0 2023-08-09T14:52:15.638 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:15.637+0000 7f50cffff640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f50d015b840 con 0x7f50d00c9ee0 2023-08-09T14:52:15.639 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:15.637+0000 7f50cf7fe640 1 --1- >> v1:172.21.15.43:6789/0 conn(0x7f50d00c90a0 0x7f50d015ad40 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.43:6789/0 says I am v1:172.21.15.43:48388/0 (socket says 172.21.15.43:48388) 2023-08-09T14:52:15.639 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:15.637+0000 7f50cf7fe640 1 -- 172.21.15.43:0/1004372785 learned_addr learned my addr 172.21.15.43:0/1004372785 (peer_addr_for_me v1:172.21.15.43:0/0) 2023-08-09T14:52:15.639 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:15.638+0000 7f50c67fc640 1 --2- 172.21.15.43:0/1004372785 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f50d00c9ee0 0x7f50d015a800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T14:52:15.639 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:15.638+0000 7f50ce7fc640 1 -- 172.21.15.43:0/1004372785 <== mon.0 v1:172.21.15.43:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 710518091 0 0) 0x7f50c8001490 con 0x7f50d00c90a0 2023-08-09T14:52:15.640 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:15.638+0000 7f50ce7fc640 1 -- 172.21.15.43:0/1004372785 --> v1:172.21.15.43:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f50b0003520 con 0x7f50d00c90a0 2023-08-09T14:52:15.640 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:15.638+0000 7f50ce7fc640 1 -- 172.21.15.43:0/1004372785 <== mon.0 v1:172.21.15.43:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1249406919 0 0) 0x7f50b0003520 con 0x7f50d00c90a0 2023-08-09T14:52:15.640 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:15.638+0000 7f50ce7fc640 1 -- 172.21.15.43:0/1004372785 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f50d00c9ee0 msgr2=0x7f50d015a800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-09T14:52:15.640 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:15.638+0000 7f50ce7fc640 1 --2- 172.21.15.43:0/1004372785 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f50d00c9ee0 0x7f50d015a800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T14:52:15.640 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:15.638+0000 7f50ce7fc640 1 -- 172.21.15.43:0/1004372785 >> v1:172.21.15.203:6789/0 conn(0x7f50d0157ea0 legacy=0x7f50d00c97d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T14:52:15.641 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:15.638+0000 7f50ce7fc640 1 -- 172.21.15.43:0/1004372785 --> v1:172.21.15.43:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f50d00c85c0 con 0x7f50d00c90a0 2023-08-09T14:52:15.641 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:15.638+0000 7f50ce7fc640 1 -- 172.21.15.43:0/1004372785 <== mon.0 v1:172.21.15.43:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1285150740 0 0) 0x7f50c8001b80 con 0x7f50d00c90a0 2023-08-09T14:52:15.641 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:15.638+0000 7f50ce7fc640 1 -- 172.21.15.43:0/1004372785 >> v1:172.21.15.43:6789/0 conn(0x7f50d00c90a0 legacy=0x7f50d015ad40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T14:52:15.641 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:15.638+0000 7f50ce7fc640 1 --2- 172.21.15.43:0/1004372785 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f50b00039a0 0x7f50b0003dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T14:52:15.641 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:15.638+0000 7f50ce7fc640 1 --2- 172.21.15.43:0/1004372785 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f50d00c9ee0 0x7f50b00047c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T14:52:15.642 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:15.638+0000 7f50ce7fc640 1 --2- 172.21.15.43:0/1004372785 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f50b0004d00 0x7f50b0005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T14:52:15.642 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:15.638+0000 7f50ce7fc640 1 -- 172.21.15.43:0/1004372785 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f50d015b840 con 0x7f50b0004d00 2023-08-09T14:52:15.642 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:15.638+0000 7f50ce7fc640 1 -- 172.21.15.43:0/1004372785 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f50b000ba60 con 0x7f50b00039a0 2023-08-09T14:52:15.642 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:15.638+0000 7f50ce7fc640 1 -- 172.21.15.43:0/1004372785 --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f50b000bc20 con 0x7f50d00c9ee0 2023-08-09T14:52:15.642 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:15.639+0000 7f50cf7fe640 1 --2- 172.21.15.43:0/1004372785 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f50b0004d00 0x7f50b0005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T14:52:15.643 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:15.639+0000 7f50ceffd640 1 --2- 172.21.15.43:0/1004372785 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f50b00039a0 0x7f50b0003dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T14:52:15.644 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:15.639+0000 7f50c67fc640 1 --2- 172.21.15.43:0/1004372785 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f50d00c9ee0 0x7f50b00047c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T14:52:15.644 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:15.639+0000 7f50cf7fe640 1 -- 172.21.15.43:0/1004372785 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f50d00c9ee0 msgr2=0x7f50b00047c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-09T14:52:15.644 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:15.639+0000 7f50cf7fe640 1 --2- 172.21.15.43:0/1004372785 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f50d00c9ee0 0x7f50b00047c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T14:52:15.644 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:15.639+0000 7f50cf7fe640 1 -- 172.21.15.43:0/1004372785 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f50b00039a0 msgr2=0x7f50b0003dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-09T14:52:15.644 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:15.639+0000 7f50cf7fe640 1 --2- 172.21.15.43:0/1004372785 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f50b00039a0 0x7f50b0003dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T14:52:15.644 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:15.639+0000 7f50cf7fe640 1 -- 172.21.15.43:0/1004372785 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f50b0003520 con 0x7f50b0004d00 2023-08-09T14:52:15.644 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:15.639+0000 7f50c67fc640 1 --2- 172.21.15.43:0/1004372785 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f50d00c9ee0 0x7f50b00047c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-08-09T14:52:15.645 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:15.640+0000 7f50cf7fe640 1 --2- 172.21.15.43:0/1004372785 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f50b0004d00 0x7f50b0005180 secure :-1 s=READY pgs=1969 cs=0 l=1 rev1=1 crypto rx=0x7f50c800a800 tx=0x7f50b00057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e7330901f2605fde server_cookie=0 in_seq=0 out_seq=0 2023-08-09T14:52:15.645 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:15.641+0000 7f50ce7fc640 1 -- 172.21.15.43:0/1004372785 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f50c8034630 con 0x7f50b0004d00 2023-08-09T14:52:15.645 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:15.641+0000 7f50ce7fc640 1 -- 172.21.15.43:0/1004372785 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f50c802f070 con 0x7f50b0004d00 2023-08-09T14:52:15.645 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:15.641+0000 7f50ce7fc640 1 -- 172.21.15.43:0/1004372785 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f50c80349f0 con 0x7f50b0004d00 2023-08-09T14:52:15.645 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:15.641+0000 7f50cffff640 1 -- 172.21.15.43:0/1004372785 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f50b0004d00 msgr2=0x7f50b0005180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T14:52:15.645 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:15.641+0000 7f50cffff640 1 --2- 172.21.15.43:0/1004372785 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f50b0004d00 0x7f50b0005180 secure :-1 s=READY pgs=1969 cs=0 l=1 rev1=1 crypto rx=0x7f50c800a800 tx=0x7f50b00057f0 comp rx=0 tx=0).stop 2023-08-09T14:52:15.645 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:15.641+0000 7f50cf7fe640 1 -- 172.21.15.43:0/1004372785 reap_dead start 2023-08-09T14:52:15.646 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:15.642+0000 7f50cffff640 1 -- 172.21.15.43:0/1004372785 shutdown_connections 2023-08-09T14:52:15.646 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:15.642+0000 7f50cffff640 1 -- 172.21.15.43:0/1004372785 >> 172.21.15.43:0/1004372785 conn(0x7f50d00cddd0 msgr2=0x7f50d00ce1e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-09T14:52:15.646 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:15.642+0000 7f50cffff640 1 -- 172.21.15.43:0/1004372785 shutdown_connections 2023-08-09T14:52:15.646 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:15.642+0000 7f50cffff640 1 -- 172.21.15.43:0/1004372785 wait complete. 2023-08-09T14:52:15.648 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:15.643+0000 7f50cffff640 1 Processor -- start 2023-08-09T14:52:15.648 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:15.644+0000 7f50cffff640 1 -- start start 2023-08-09T14:52:15.648 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:15.645+0000 7f50cffff640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f50b0004d00 0x7f50d00c22f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T14:52:15.648 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:15.645+0000 7f50cffff640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f50d01ef560 0x7f50d00c3aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T14:52:15.648 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:15.645+0000 7f50cffff640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f50d00c7820 0x7f50d014cf60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T14:52:15.648 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:15.645+0000 7f50cffff640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f50d00c4ea0 con 0x7f50d00c7820 2023-08-09T14:52:15.649 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:15.645+0000 7f50cffff640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f50d00c2bf0 con 0x7f50d01ef560 2023-08-09T14:52:15.649 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:15.645+0000 7f50cffff640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f50d00c2ed0 con 0x7f50b0004d00 2023-08-09T14:52:15.649 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:15.645+0000 7f50cf7fe640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f50d00c7820 0x7f50d014cf60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T14:52:15.649 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:15.645+0000 7f50cf7fe640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f50d00c7820 0x7f50d014cf60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:37814/0 (socket says 172.21.15.43:37814) 2023-08-09T14:52:15.649 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:15.645+0000 7f50cf7fe640 1 -- 172.21.15.43:0/2797392069 learned_addr learned my addr 172.21.15.43:0/2797392069 (peer_addr_for_me v2:172.21.15.43:0/0) 2023-08-09T14:52:15.649 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:15.645+0000 7f50c67fc640 1 --2- 172.21.15.43:0/2797392069 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f50d01ef560 0x7f50d00c3aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T14:52:15.649 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:15.645+0000 7f50cf7fe640 1 -- 172.21.15.43:0/2797392069 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f50b0004d00 msgr2=0x7f50d00c22f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T14:52:15.650 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:15.645+0000 7f50cf7fe640 1 --2- 172.21.15.43:0/2797392069 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f50b0004d00 0x7f50d00c22f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T14:52:15.650 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:15.645+0000 7f50cf7fe640 1 -- 172.21.15.43:0/2797392069 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f50d01ef560 msgr2=0x7f50d00c3aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T14:52:15.650 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:15.645+0000 7f50cf7fe640 1 --2- 172.21.15.43:0/2797392069 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f50d01ef560 0x7f50d00c3aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T14:52:15.650 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:15.646+0000 7f50cf7fe640 1 -- 172.21.15.43:0/2797392069 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f50d00c50f0 con 0x7f50d00c7820 2023-08-09T14:52:15.650 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:15.646+0000 7f50c67fc640 1 --2- 172.21.15.43:0/2797392069 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f50d01ef560 0x7f50d00c3aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-08-09T14:52:15.650 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:15.646+0000 7f50cf7fe640 1 --2- 172.21.15.43:0/2797392069 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f50d00c7820 0x7f50d014cf60 secure :-1 s=READY pgs=1970 cs=0 l=1 rev1=1 crypto rx=0x7f50c800a800 tx=0x7f50c8001f10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-09T14:52:15.650 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:15.646+0000 7f50ccff9640 1 -- 172.21.15.43:0/2797392069 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f50c8001b40 con 0x7f50d00c7820 2023-08-09T14:52:15.650 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:15.646+0000 7f50ccff9640 1 -- 172.21.15.43:0/2797392069 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f50c8004e90 con 0x7f50d00c7820 2023-08-09T14:52:15.651 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:15.646+0000 7f50ccff9640 1 -- 172.21.15.43:0/2797392069 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f50c8034630 con 0x7f50d00c7820 2023-08-09T14:52:15.651 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:15.646+0000 7f50cffff640 1 -- 172.21.15.43:0/2797392069 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f50d00c53d0 con 0x7f50d00c7820 2023-08-09T14:52:15.651 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:15.646+0000 7f50cffff640 1 -- 172.21.15.43:0/2797392069 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f50d00c3fe0 con 0x7f50d00c7820 2023-08-09T14:52:15.651 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:15.647+0000 7f50cffff640 1 -- 172.21.15.43:0/2797392069 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f50d004e390 con 0x7f50d00c7820 2023-08-09T14:52:15.652 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:15.648+0000 7f50ccff9640 1 -- 172.21.15.43:0/2797392069 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 88) v1 ==== 88443+0+0 (secure 0 0 0) 0x7f50c8031030 con 0x7f50d00c7820 2023-08-09T14:52:15.652 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:15.650+0000 7f50ccff9640 1 --2- 172.21.15.43:0/2797392069 >> [v2:172.21.15.203:6832/2959400181,v1:172.21.15.203:6833/2959400181] conn(0x7f50b406a170 0x7f50b406c630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T14:52:15.652 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:15.650+0000 7f50ccff9640 1 -- 172.21.15.43:0/2797392069 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(175..175 src has 1..175) v4 ==== 7200+0+0 (secure 0 0 0) 0x7f50c80aa630 con 0x7f50d00c7820 2023-08-09T14:52:15.652 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:15.650+0000 7f50ceffd640 1 --2- 172.21.15.43:0/2797392069 >> [v2:172.21.15.203:6832/2959400181,v1:172.21.15.203:6833/2959400181] conn(0x7f50b406a170 0x7f50b406c630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T14:52:15.652 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:15.652+0000 7f50ceffd640 1 --2- 172.21.15.43:0/2797392069 >> [v2:172.21.15.203:6832/2959400181,v1:172.21.15.203:6833/2959400181] conn(0x7f50b406a170 0x7f50b406c630 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0x7f50c0007fd0 tx=0x7f50c0009040 comp rx=0 tx=0).ready entity=mgr.7223 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-09T14:52:15.654 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:15.654+0000 7f50ccff9640 1 -- 172.21.15.43:0/2797392069 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178066 (secure 0 0 0) 0x7f50c8075720 con 0x7f50d00c7820 2023-08-09T14:52:15.891 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:15.890+0000 7f50cffff640 1 -- 172.21.15.43:0/2797392069 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} v 0) v1 -- 0x7f50d004e390 con 0x7f50d00c7820 2023-08-09T14:52:16.402 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:16.401+0000 7f50ccff9640 1 -- 172.21.15.43:0/2797392069 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]=0 v80) v1 ==== 104+0+0 (secure 0 0 0) 0x7f50c80011c0 con 0x7f50d00c7820 2023-08-09T14:52:16.405 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:16.405+0000 7f50cffff640 1 -- 172.21.15.43:0/2797392069 >> [v2:172.21.15.203:6832/2959400181,v1:172.21.15.203:6833/2959400181] conn(0x7f50b406a170 msgr2=0x7f50b406c630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T14:52:16.405 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:16.405+0000 7f50cffff640 1 --2- 172.21.15.43:0/2797392069 >> [v2:172.21.15.203:6832/2959400181,v1:172.21.15.203:6833/2959400181] conn(0x7f50b406a170 0x7f50b406c630 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0x7f50c0007fd0 tx=0x7f50c0009040 comp rx=0 tx=0).stop 2023-08-09T14:52:16.405 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:16.405+0000 7f50cffff640 1 -- 172.21.15.43:0/2797392069 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f50d00c7820 msgr2=0x7f50d014cf60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T14:52:16.406 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:16.405+0000 7f50cffff640 1 --2- 172.21.15.43:0/2797392069 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f50d00c7820 0x7f50d014cf60 secure :-1 s=READY pgs=1970 cs=0 l=1 rev1=1 crypto rx=0x7f50c800a800 tx=0x7f50c8001f10 comp rx=0 tx=0).stop 2023-08-09T14:52:16.406 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:16.405+0000 7f50cffff640 1 -- 172.21.15.43:0/2797392069 shutdown_connections 2023-08-09T14:52:16.406 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:16.405+0000 7f50cffff640 1 --2- 172.21.15.43:0/2797392069 >> [v2:172.21.15.203:6832/2959400181,v1:172.21.15.203:6833/2959400181] conn(0x7f50b406a170 0x7f50b406c630 unknown :-1 s=CLOSED pgs=32 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T14:52:16.407 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:16.405+0000 7f50cffff640 1 --2- 172.21.15.43:0/2797392069 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f50d00c7820 0x7f50d014cf60 unknown :-1 s=CLOSED pgs=1970 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T14:52:16.407 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:16.405+0000 7f50cffff640 1 --2- 172.21.15.43:0/2797392069 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f50d01ef560 0x7f50d00c3aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T14:52:16.407 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:16.405+0000 7f50cffff640 1 --2- 172.21.15.43:0/2797392069 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f50b0004d00 0x7f50d00c22f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T14:52:16.407 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:16.405+0000 7f50cffff640 1 -- 172.21.15.43:0/2797392069 >> 172.21.15.43:0/2797392069 conn(0x7f50d00cddd0 msgr2=0x7f50d0158420 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-09T14:52:16.407 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:16.405+0000 7f50cffff640 1 -- 172.21.15.43:0/2797392069 shutdown_connections 2023-08-09T14:52:16.407 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:16.405+0000 7f50cffff640 1 -- 172.21.15.43:0/2797392069 wait complete. 2023-08-09T14:52:16.428 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_metadata cephfs_metadata --yes-i-really-really-mean-it 2023-08-09T14:52:16.710 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:16.710+0000 7f9422722640 1 Processor -- start 2023-08-09T14:52:16.711 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:16.710+0000 7f9422722640 1 -- start start 2023-08-09T14:52:16.712 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:16.711+0000 7f9422722640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f941c15a330 0x7f941c0c9780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T14:52:16.712 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:16.711+0000 7f9422722640 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7f941c15ea60 0x7f941c070d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T14:52:16.712 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:16.711+0000 7f9422722640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f941c0af870 0x7f941c0afd00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T14:52:16.713 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:16.711+0000 7f9422722640 1 -- --> v2:172.21.15.43:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f941c0b0240 con 0x7f941c15a330 2023-08-09T14:52:16.713 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:16.711+0000 7f9422722640 1 -- --> v2:172.21.15.203:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f941c0c9cc0 con 0x7f941c15ea60 2023-08-09T14:52:16.713 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:16.711+0000 7f9422722640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f941c15dd30 con 0x7f941c0af870 2023-08-09T14:52:16.714 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:16.712+0000 7f9421720640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f941c15a330 0x7f941c0c9780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T14:52:16.714 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:16.712+0000 7f9421720640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f941c15a330 0x7f941c0c9780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:37828/0 (socket says 172.21.15.43:37828) 2023-08-09T14:52:16.714 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:16.712+0000 7f9421720640 1 -- 172.21.15.43:0/834569432 learned_addr learned my addr 172.21.15.43:0/834569432 (peer_addr_for_me v2:172.21.15.43:0/0) 2023-08-09T14:52:16.714 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:16.712+0000 7f9421f21640 1 --2- 172.21.15.43:0/834569432 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f941c0af870 0x7f941c0afd00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T14:52:16.715 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:16.712+0000 7f9420f1f640 1 --2- 172.21.15.43:0/834569432 >> v2:172.21.15.203:3300/0 conn(0x7f941c15ea60 0x7f941c070d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T14:52:16.715 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:16.712+0000 7f9421720640 1 -- 172.21.15.43:0/834569432 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f941c0af870 msgr2=0x7f941c0afd00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-09T14:52:16.715 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:16.712+0000 7f9421720640 1 --2- 172.21.15.43:0/834569432 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f941c0af870 0x7f941c0afd00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T14:52:16.715 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:16.712+0000 7f9421720640 1 -- 172.21.15.43:0/834569432 >> v2:172.21.15.203:3300/0 conn(0x7f941c15ea60 msgr2=0x7f941c070d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-09T14:52:16.715 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:16.712+0000 7f9421720640 1 --2- 172.21.15.43:0/834569432 >> v2:172.21.15.203:3300/0 conn(0x7f941c15ea60 0x7f941c070d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T14:52:16.716 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:16.712+0000 7f9421720640 1 -- 172.21.15.43:0/834569432 --> v2:172.21.15.43:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f941c071240 con 0x7f941c15a330 2023-08-09T14:52:16.717 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:16.714+0000 7f9421720640 1 --2- 172.21.15.43:0/834569432 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f941c15a330 0x7f941c0c9780 secure :-1 s=READY pgs=1971 cs=0 l=1 rev1=1 crypto rx=0x7f941c0ceef0 tx=0x7f941802b8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b9c5ec56fb29a1e5 server_cookie=0 in_seq=0 out_seq=0 2023-08-09T14:52:16.717 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:16.714+0000 7f9413fff640 1 -- 172.21.15.43:0/834569432 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9418031640 con 0x7f941c15a330 2023-08-09T14:52:16.717 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:16.714+0000 7f9413fff640 1 -- 172.21.15.43:0/834569432 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f94180317a0 con 0x7f941c15a330 2023-08-09T14:52:16.717 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:16.715+0000 7f9413fff640 1 -- 172.21.15.43:0/834569432 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9418031b30 con 0x7f941c15a330 2023-08-09T14:52:16.717 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:16.715+0000 7f9422722640 1 -- 172.21.15.43:0/834569432 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f941c15a330 msgr2=0x7f941c0c9780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T14:52:16.718 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:16.715+0000 7f9422722640 1 --2- 172.21.15.43:0/834569432 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f941c15a330 0x7f941c0c9780 secure :-1 s=READY pgs=1971 cs=0 l=1 rev1=1 crypto rx=0x7f941c0ceef0 tx=0x7f941802b8f0 comp rx=0 tx=0).stop 2023-08-09T14:52:16.718 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:16.715+0000 7f9422722640 1 -- 172.21.15.43:0/834569432 shutdown_connections 2023-08-09T14:52:16.718 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:16.715+0000 7f9422722640 1 --2- 172.21.15.43:0/834569432 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f941c0af870 0x7f941c0afd00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T14:52:16.718 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:16.715+0000 7f9422722640 1 --2- 172.21.15.43:0/834569432 >> v2:172.21.15.203:3300/0 conn(0x7f941c15ea60 0x7f941c070d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T14:52:16.718 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:16.715+0000 7f9422722640 1 --2- 172.21.15.43:0/834569432 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f941c15a330 0x7f941c0c9780 unknown :-1 s=CLOSED pgs=1971 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T14:52:16.719 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:16.715+0000 7f9422722640 1 -- 172.21.15.43:0/834569432 >> 172.21.15.43:0/834569432 conn(0x7f941c0ca870 msgr2=0x7f941c0cac50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-09T14:52:16.719 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:16.715+0000 7f9422722640 1 -- 172.21.15.43:0/834569432 shutdown_connections 2023-08-09T14:52:16.719 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:16.715+0000 7f9422722640 1 -- 172.21.15.43:0/834569432 wait complete. 2023-08-09T14:52:16.722 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:16.717+0000 7f9422722640 1 Processor -- start 2023-08-09T14:52:16.722 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:16.717+0000 7f9422722640 1 -- start start 2023-08-09T14:52:16.723 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:16.718+0000 7f9422722640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f941c0af870 0x7f941c0cd350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T14:52:16.723 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:16.718+0000 7f9422722640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f941c15a330 0x7f941c149e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T14:52:16.723 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:16.718+0000 7f9422722640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f941c15ea60 0x7f941c1472b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T14:52:16.723 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:16.718+0000 7f9422722640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f941c145100 con 0x7f941c15a330 2023-08-09T14:52:16.724 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:16.718+0000 7f9422722640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f941c14a350 con 0x7f941c15ea60 2023-08-09T14:52:16.724 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:16.718+0000 7f9422722640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f941c0cc810 con 0x7f941c0af870 2023-08-09T14:52:16.724 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:16.718+0000 7f9420f1f640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f941c15a330 0x7f941c149e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T14:52:16.724 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:16.719+0000 7f9420f1f640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f941c15a330 0x7f941c149e10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:37836/0 (socket says 172.21.15.43:37836) 2023-08-09T14:52:16.725 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:16.719+0000 7f9420f1f640 1 -- 172.21.15.43:0/2253301403 learned_addr learned my addr 172.21.15.43:0/2253301403 (peer_addr_for_me v2:172.21.15.43:0/0) 2023-08-09T14:52:16.725 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:16.719+0000 7f9421720640 1 --2- 172.21.15.43:0/2253301403 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f941c0af870 0x7f941c0cd350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T14:52:16.725 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:16.719+0000 7f9420f1f640 1 -- 172.21.15.43:0/2253301403 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f941c0af870 msgr2=0x7f941c0cd350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T14:52:16.725 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:16.719+0000 7f9420f1f640 1 --2- 172.21.15.43:0/2253301403 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f941c0af870 0x7f941c0cd350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T14:52:16.725 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:16.719+0000 7f9420f1f640 1 -- 172.21.15.43:0/2253301403 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f941c15ea60 msgr2=0x7f941c1472b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T14:52:16.726 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:16.719+0000 7f9420f1f640 1 --2- 172.21.15.43:0/2253301403 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f941c15ea60 0x7f941c1472b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T14:52:16.726 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:16.719+0000 7f9420f1f640 1 -- 172.21.15.43:0/2253301403 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f94180094d0 con 0x7f941c15a330 2023-08-09T14:52:16.726 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:16.719+0000 7f9420f1f640 1 --2- 172.21.15.43:0/2253301403 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f941c15a330 0x7f941c149e10 secure :-1 s=READY pgs=1972 cs=0 l=1 rev1=1 crypto rx=0x7f940c00d800 tx=0x7f940c00dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-09T14:52:16.726 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:16.719+0000 7f9411ffb640 1 -- 172.21.15.43:0/2253301403 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f940c00b660 con 0x7f941c15a330 2023-08-09T14:52:16.726 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:16.719+0000 7f9411ffb640 1 -- 172.21.15.43:0/2253301403 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f940c00b7c0 con 0x7f941c15a330 2023-08-09T14:52:16.726 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:16.720+0000 7f9422722640 1 -- 172.21.15.43:0/2253301403 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f941c0ccab0 con 0x7f941c15a330 2023-08-09T14:52:16.726 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:16.720+0000 7f9411ffb640 1 -- 172.21.15.43:0/2253301403 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f940c00bb50 con 0x7f941c15a330 2023-08-09T14:52:16.726 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:16.720+0000 7f9422722640 1 -- 172.21.15.43:0/2253301403 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f941c0cd9f0 con 0x7f941c15a330 2023-08-09T14:52:16.727 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:16.721+0000 7f9422722640 1 -- 172.21.15.43:0/2253301403 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f941c0cb410 con 0x7f941c15a330 2023-08-09T14:52:16.727 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:16.722+0000 7f9411ffb640 1 -- 172.21.15.43:0/2253301403 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 88) v1 ==== 88443+0+0 (secure 0 0 0) 0x7f940c00bcb0 con 0x7f941c15a330 2023-08-09T14:52:16.727 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:16.723+0000 7f9411ffb640 1 --2- 172.21.15.43:0/2253301403 >> [v2:172.21.15.203:6832/2959400181,v1:172.21.15.203:6833/2959400181] conn(0x7f940006a1a0 0x7f940006c660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T14:52:16.728 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:16.723+0000 7f9411ffb640 1 -- 172.21.15.43:0/2253301403 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(175..175 src has 1..175) v4 ==== 7200+0+0 (secure 0 0 0) 0x7f940c089040 con 0x7f941c15a330 2023-08-09T14:52:16.728 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:16.724+0000 7f9421720640 1 --2- 172.21.15.43:0/2253301403 >> [v2:172.21.15.203:6832/2959400181,v1:172.21.15.203:6833/2959400181] conn(0x7f940006a1a0 0x7f940006c660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T14:52:16.728 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:16.725+0000 7f9421720640 1 --2- 172.21.15.43:0/2253301403 >> [v2:172.21.15.203:6832/2959400181,v1:172.21.15.203:6833/2959400181] conn(0x7f940006a1a0 0x7f940006c660 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0x7f9418036000 tx=0x7f941803c040 comp rx=0 tx=0).ready entity=mgr.7223 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-09T14:52:16.728 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:16.727+0000 7f9411ffb640 1 -- 172.21.15.43:0/2253301403 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178066 (secure 0 0 0) 0x7f940c0bb7b0 con 0x7f941c15a330 2023-08-09T14:52:16.924 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:16.924+0000 7f9422722640 1 -- 172.21.15.43:0/2253301403 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7f941c0739f0 con 0x7f941c15a330 2023-08-09T14:52:17.414 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:17.413+0000 7f9411ffb640 1 -- 172.21.15.43:0/2253301403 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_metadata' removed v176) v1 ==== 181+0+0 (secure 0 0 0) 0x7f940c0579c0 con 0x7f941c15a330 2023-08-09T14:52:17.414 INFO:teuthology.orchestra.run.smithi043.stderr:pool 'cephfs_metadata' removed 2023-08-09T14:52:17.421 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:17.420+0000 7f9422722640 1 -- 172.21.15.43:0/2253301403 >> [v2:172.21.15.203:6832/2959400181,v1:172.21.15.203:6833/2959400181] conn(0x7f940006a1a0 msgr2=0x7f940006c660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T14:52:17.421 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:17.420+0000 7f9422722640 1 --2- 172.21.15.43:0/2253301403 >> [v2:172.21.15.203:6832/2959400181,v1:172.21.15.203:6833/2959400181] conn(0x7f940006a1a0 0x7f940006c660 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0x7f9418036000 tx=0x7f941803c040 comp rx=0 tx=0).stop 2023-08-09T14:52:17.421 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:17.420+0000 7f9422722640 1 -- 172.21.15.43:0/2253301403 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f941c15a330 msgr2=0x7f941c149e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T14:52:17.422 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:17.420+0000 7f9422722640 1 --2- 172.21.15.43:0/2253301403 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f941c15a330 0x7f941c149e10 secure :-1 s=READY pgs=1972 cs=0 l=1 rev1=1 crypto rx=0x7f940c00d800 tx=0x7f940c00dcc0 comp rx=0 tx=0).stop 2023-08-09T14:52:17.423 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:17.423+0000 7f9422722640 1 -- 172.21.15.43:0/2253301403 shutdown_connections 2023-08-09T14:52:17.423 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:17.423+0000 7f9422722640 1 --2- 172.21.15.43:0/2253301403 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f941c15ea60 0x7f941c1472b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T14:52:17.423 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:17.423+0000 7f9422722640 1 --2- 172.21.15.43:0/2253301403 >> [v2:172.21.15.203:6832/2959400181,v1:172.21.15.203:6833/2959400181] conn(0x7f940006a1a0 0x7f940006c660 unknown :-1 s=CLOSED pgs=34 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T14:52:17.424 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:17.423+0000 7f9422722640 1 --2- 172.21.15.43:0/2253301403 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f941c15a330 0x7f941c149e10 unknown :-1 s=CLOSED pgs=1972 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T14:52:17.424 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:17.423+0000 7f9422722640 1 --2- 172.21.15.43:0/2253301403 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f941c0af870 0x7f941c0cd350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T14:52:17.424 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:17.423+0000 7f9422722640 1 -- 172.21.15.43:0/2253301403 >> 172.21.15.43:0/2253301403 conn(0x7f941c0ca870 msgr2=0x7f941c0b2f90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-09T14:52:17.424 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:17.423+0000 7f9422722640 1 -- 172.21.15.43:0/2253301403 shutdown_connections 2023-08-09T14:52:17.425 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:17.424+0000 7f9422722640 1 -- 172.21.15.43:0/2253301403 wait complete. 2023-08-09T14:52:17.449 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_data cephfs_data --yes-i-really-really-mean-it 2023-08-09T14:52:17.675 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:17.674+0000 7f668359e640 1 Processor -- start 2023-08-09T14:52:17.675 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:17.674+0000 7f668359e640 1 -- start start 2023-08-09T14:52:17.677 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:17.676+0000 7f668359e640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f66841606f0 0x7f668406bd60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T14:52:17.677 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:17.676+0000 7f668359e640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f66840c8440 0x7f66840af870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T14:52:17.677 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:17.676+0000 7f668359e640 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7f66840afdb0 0x7f6684164300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T14:52:17.677 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:17.676+0000 7f668359e640 1 -- --> v2:172.21.15.43:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f66840cdc90 con 0x7f66840c8440 2023-08-09T14:52:17.677 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:17.676+0000 7f668359e640 1 -- --> v2:172.21.15.203:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f668406c2a0 con 0x7f66840afdb0 2023-08-09T14:52:17.677 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:17.676+0000 7f668359e640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f66840b0200 con 0x7f66841606f0 2023-08-09T14:52:17.678 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:17.677+0000 7f6681d9b640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f66840c8440 0x7f66840af870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T14:52:17.678 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:17.677+0000 7f6681d9b640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f66840c8440 0x7f66840af870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:37846/0 (socket says 172.21.15.43:37846) 2023-08-09T14:52:17.678 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:17.677+0000 7f6681d9b640 1 -- 172.21.15.43:0/3374706345 learned_addr learned my addr 172.21.15.43:0/3374706345 (peer_addr_for_me v2:172.21.15.43:0/0) 2023-08-09T14:52:17.679 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:17.677+0000 7f668259c640 1 --2- 172.21.15.43:0/3374706345 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f66841606f0 0x7f668406bd60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T14:52:17.679 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:17.677+0000 7f6682d9d640 1 --2- 172.21.15.43:0/3374706345 >> v2:172.21.15.203:3300/0 conn(0x7f66840afdb0 0x7f6684164300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T14:52:17.679 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:17.677+0000 7f6681d9b640 1 -- 172.21.15.43:0/3374706345 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f66841606f0 msgr2=0x7f668406bd60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-09T14:52:17.679 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:17.677+0000 7f6681d9b640 1 --2- 172.21.15.43:0/3374706345 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f66841606f0 0x7f668406bd60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T14:52:17.679 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:17.677+0000 7f6681d9b640 1 -- 172.21.15.43:0/3374706345 >> v2:172.21.15.203:3300/0 conn(0x7f66840afdb0 msgr2=0x7f6684164300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-09T14:52:17.679 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:17.677+0000 7f6681d9b640 1 --2- 172.21.15.43:0/3374706345 >> v2:172.21.15.203:3300/0 conn(0x7f66840afdb0 0x7f6684164300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T14:52:17.679 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:17.677+0000 7f6681d9b640 1 -- 172.21.15.43:0/3374706345 --> v2:172.21.15.43:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f668406ce30 con 0x7f66840c8440 2023-08-09T14:52:17.680 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:17.679+0000 7f6681d9b640 1 --2- 172.21.15.43:0/3374706345 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f66840c8440 0x7f66840af870 secure :-1 s=READY pgs=1973 cs=0 l=1 rev1=1 crypto rx=0x7f66840b0590 tx=0x7f667802b910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f588d2edfd8090ca server_cookie=0 in_seq=0 out_seq=0 2023-08-09T14:52:17.681 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:17.679+0000 7f668159a640 1 -- 172.21.15.43:0/3374706345 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6678031870 con 0x7f66840c8440 2023-08-09T14:52:17.681 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:17.679+0000 7f668159a640 1 -- 172.21.15.43:0/3374706345 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f66780319d0 con 0x7f66840c8440 2023-08-09T14:52:17.681 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:17.679+0000 7f668159a640 1 -- 172.21.15.43:0/3374706345 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6678031d60 con 0x7f66840c8440 2023-08-09T14:52:17.681 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:17.680+0000 7f668359e640 1 -- 172.21.15.43:0/3374706345 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f66840c8440 msgr2=0x7f66840af870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T14:52:17.681 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:17.680+0000 7f668359e640 1 --2- 172.21.15.43:0/3374706345 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f66840c8440 0x7f66840af870 secure :-1 s=READY pgs=1973 cs=0 l=1 rev1=1 crypto rx=0x7f66840b0590 tx=0x7f667802b910 comp rx=0 tx=0).stop 2023-08-09T14:52:17.681 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:17.680+0000 7f668359e640 1 -- 172.21.15.43:0/3374706345 shutdown_connections 2023-08-09T14:52:17.681 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:17.680+0000 7f668359e640 1 --2- 172.21.15.43:0/3374706345 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f66840c8440 0x7f66840af870 unknown :-1 s=CLOSED pgs=1973 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T14:52:17.682 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:17.680+0000 7f668359e640 1 --2- 172.21.15.43:0/3374706345 >> v2:172.21.15.203:3300/0 conn(0x7f66840afdb0 0x7f6684164300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T14:52:17.682 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:17.680+0000 7f668359e640 1 --2- 172.21.15.43:0/3374706345 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f66841606f0 0x7f668406bd60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T14:52:17.682 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:17.680+0000 7f668359e640 1 -- 172.21.15.43:0/3374706345 >> 172.21.15.43:0/3374706345 conn(0x7f66840ca870 msgr2=0x7f66840cac50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-09T14:52:17.682 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:17.680+0000 7f668359e640 1 -- 172.21.15.43:0/3374706345 shutdown_connections 2023-08-09T14:52:17.682 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:17.680+0000 7f668359e640 1 -- 172.21.15.43:0/3374706345 wait complete. 2023-08-09T14:52:17.683 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:17.682+0000 7f668359e640 1 Processor -- start 2023-08-09T14:52:17.683 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:17.682+0000 7f668359e640 1 -- start start 2023-08-09T14:52:17.685 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:17.683+0000 7f668359e640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f66840afdb0 0x7f66841f9870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T14:52:17.685 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:17.683+0000 7f668359e640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f66840c8440 0x7f66841f9db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T14:52:17.685 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:17.683+0000 7f668359e640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f66841606f0 0x7f66840c4ec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T14:52:17.685 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:17.683+0000 7f668359e640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f66840c7be0 con 0x7f66841606f0 2023-08-09T14:52:17.685 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:17.683+0000 7f668359e640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f66840c66b0 con 0x7f66840afdb0 2023-08-09T14:52:17.685 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:17.683+0000 7f668359e640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f66840c5aa0 con 0x7f66840c8440 2023-08-09T14:52:17.686 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:17.683+0000 7f6682d9d640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f66841606f0 0x7f66840c4ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T14:52:17.686 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:17.683+0000 7f6682d9d640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f66841606f0 0x7f66840c4ec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:37854/0 (socket says 172.21.15.43:37854) 2023-08-09T14:52:17.686 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:17.683+0000 7f6682d9d640 1 -- 172.21.15.43:0/3851013987 learned_addr learned my addr 172.21.15.43:0/3851013987 (peer_addr_for_me v2:172.21.15.43:0/0) 2023-08-09T14:52:17.686 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:17.683+0000 7f668259c640 1 --2- 172.21.15.43:0/3851013987 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f66840afdb0 0x7f66841f9870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T14:52:17.686 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:17.684+0000 7f6682d9d640 1 -- 172.21.15.43:0/3851013987 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f66840c8440 msgr2=0x7f66841f9db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T14:52:17.687 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:17.684+0000 7f6682d9d640 1 --2- 172.21.15.43:0/3851013987 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f66840c8440 0x7f66841f9db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T14:52:17.687 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:17.684+0000 7f6682d9d640 1 -- 172.21.15.43:0/3851013987 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f66840afdb0 msgr2=0x7f66841f9870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T14:52:17.687 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:17.684+0000 7f6682d9d640 1 --2- 172.21.15.43:0/3851013987 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f66840afdb0 0x7f66841f9870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T14:52:17.687 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:17.684+0000 7f6682d9d640 1 -- 172.21.15.43:0/3851013987 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f66780094d0 con 0x7f66841606f0 2023-08-09T14:52:17.687 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:17.684+0000 7f6682d9d640 1 --2- 172.21.15.43:0/3851013987 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f66841606f0 0x7f66840c4ec0 secure :-1 s=READY pgs=1974 cs=0 l=1 rev1=1 crypto rx=0x7f667c007e60 tx=0x7f667c00e4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-09T14:52:17.687 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:17.684+0000 7f6672ffd640 1 -- 172.21.15.43:0/3851013987 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f667c014420 con 0x7f66841606f0 2023-08-09T14:52:17.687 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:17.684+0000 7f6672ffd640 1 -- 172.21.15.43:0/3851013987 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f667c0145a0 con 0x7f66841606f0 2023-08-09T14:52:17.687 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:17.684+0000 7f6672ffd640 1 -- 172.21.15.43:0/3851013987 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f667c014950 con 0x7f66841606f0 2023-08-09T14:52:17.688 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:17.684+0000 7f668359e640 1 -- 172.21.15.43:0/3851013987 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f66840c5d80 con 0x7f66841606f0 2023-08-09T14:52:17.688 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:17.684+0000 7f668359e640 1 -- 172.21.15.43:0/3851013987 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f66840c6340 con 0x7f66841606f0 2023-08-09T14:52:17.688 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:17.685+0000 7f668359e640 1 -- 172.21.15.43:0/3851013987 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f66840739f0 con 0x7f66841606f0 2023-08-09T14:52:17.689 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:17.686+0000 7f6672ffd640 1 -- 172.21.15.43:0/3851013987 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 88) v1 ==== 88443+0+0 (secure 0 0 0) 0x7f667c014ab0 con 0x7f66841606f0 2023-08-09T14:52:17.689 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:17.687+0000 7f6672ffd640 1 --2- 172.21.15.43:0/3851013987 >> [v2:172.21.15.203:6832/2959400181,v1:172.21.15.203:6833/2959400181] conn(0x7f666406a170 0x7f666406c630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T14:52:17.689 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:17.687+0000 7f6672ffd640 1 -- 172.21.15.43:0/3851013987 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(176..176 src has 1..176) v4 ==== 6762+0+0 (secure 0 0 0) 0x7f667c0893c0 con 0x7f66841606f0 2023-08-09T14:52:17.689 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:17.688+0000 7f668259c640 1 --2- 172.21.15.43:0/3851013987 >> [v2:172.21.15.203:6832/2959400181,v1:172.21.15.203:6833/2959400181] conn(0x7f666406a170 0x7f666406c630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T14:52:17.689 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:17.689+0000 7f668259c640 1 --2- 172.21.15.43:0/3851013987 >> [v2:172.21.15.203:6832/2959400181,v1:172.21.15.203:6833/2959400181] conn(0x7f666406a170 0x7f666406c630 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7f6674009990 tx=0x7f6674008040 comp rx=0 tx=0).ready entity=mgr.7223 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-09T14:52:17.692 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:17.692+0000 7f6672ffd640 1 -- 172.21.15.43:0/3851013987 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178066 (secure 0 0 0) 0x7f667c0580d0 con 0x7f66841606f0 2023-08-09T14:52:17.886 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:17.885+0000 7f668359e640 1 -- 172.21.15.43:0/3851013987 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7f66840c67f0 con 0x7f66841606f0 2023-08-09T14:52:18.437 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:18.436+0000 7f6672ffd640 1 -- 172.21.15.43:0/3851013987 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data' removed v177) v1 ==== 169+0+0 (secure 0 0 0) 0x7f667c057c60 con 0x7f66841606f0 2023-08-09T14:52:18.440 INFO:teuthology.orchestra.run.smithi043.stderr:pool 'cephfs_data' removed 2023-08-09T14:52:18.444 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:18.444+0000 7f668359e640 1 -- 172.21.15.43:0/3851013987 >> [v2:172.21.15.203:6832/2959400181,v1:172.21.15.203:6833/2959400181] conn(0x7f666406a170 msgr2=0x7f666406c630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T14:52:18.445 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:18.444+0000 7f668359e640 1 --2- 172.21.15.43:0/3851013987 >> [v2:172.21.15.203:6832/2959400181,v1:172.21.15.203:6833/2959400181] conn(0x7f666406a170 0x7f666406c630 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7f6674009990 tx=0x7f6674008040 comp rx=0 tx=0).stop 2023-08-09T14:52:18.445 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:18.444+0000 7f668359e640 1 -- 172.21.15.43:0/3851013987 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f66841606f0 msgr2=0x7f66840c4ec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T14:52:18.445 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:18.444+0000 7f668359e640 1 --2- 172.21.15.43:0/3851013987 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f66841606f0 0x7f66840c4ec0 secure :-1 s=READY pgs=1974 cs=0 l=1 rev1=1 crypto rx=0x7f667c007e60 tx=0x7f667c00e4b0 comp rx=0 tx=0).stop 2023-08-09T14:52:18.446 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:18.445+0000 7f668359e640 1 -- 172.21.15.43:0/3851013987 shutdown_connections 2023-08-09T14:52:18.446 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:18.445+0000 7f668359e640 1 --2- 172.21.15.43:0/3851013987 >> [v2:172.21.15.203:6832/2959400181,v1:172.21.15.203:6833/2959400181] conn(0x7f666406a170 0x7f666406c630 unknown :-1 s=CLOSED pgs=36 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T14:52:18.446 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:18.445+0000 7f668359e640 1 --2- 172.21.15.43:0/3851013987 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f66841606f0 0x7f66840c4ec0 unknown :-1 s=CLOSED pgs=1974 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T14:52:18.446 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:18.445+0000 7f668359e640 1 --2- 172.21.15.43:0/3851013987 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f66840c8440 0x7f66841f9db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T14:52:18.446 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:18.445+0000 7f668359e640 1 --2- 172.21.15.43:0/3851013987 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f66840afdb0 0x7f66841f9870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T14:52:18.446 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:18.445+0000 7f668359e640 1 -- 172.21.15.43:0/3851013987 >> 172.21.15.43:0/3851013987 conn(0x7f66840ca870 msgr2=0x7f66840c3e00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-09T14:52:18.447 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:18.447+0000 7f668359e640 1 -- 172.21.15.43:0/3851013987 shutdown_connections 2023-08-09T14:52:18.447 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:18.447+0000 7f668359e640 1 -- 172.21.15.43:0/3851013987 wait complete. 2023-08-09T14:52:18.468 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config rm mgr mgr/snap_schedule/dump_on_update 2023-08-09T14:52:18.675 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:18.674+0000 7fbbe8ccc640 1 Processor -- start 2023-08-09T14:52:18.675 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:18.674+0000 7fbbe8ccc640 1 -- start start 2023-08-09T14:52:18.676 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:18.676+0000 7fbbe8ccc640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7fbbe40cbb80 0x7fbbe40cb2b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T14:52:18.677 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:18.676+0000 7fbbe8ccc640 1 -- --> v1:172.21.15.43:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbbe415a4a0 con 0x7fbbe40ce1c0 2023-08-09T14:52:18.677 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:18.676+0000 7fbbe8ccc640 1 -- --> v1:172.21.15.203:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbbe40cc030 con 0x7fbbe415a0c0 2023-08-09T14:52:18.677 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:18.676+0000 7fbbe8ccc640 1 -- --> v2:172.21.15.43:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbbe40cc2d0 con 0x7fbbe40cbb80 2023-08-09T14:52:18.678 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:18.676+0000 7fbbe1d9b640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7fbbe40cbb80 0x7fbbe40cb2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T14:52:18.678 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:18.676+0000 7fbbe2d9d640 1 --1- >> v1:172.21.15.43:6789/0 conn(0x7fbbe40ce1c0 0x7fbbe40ce5c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.43:6789/0 says I am v1:172.21.15.43:48402/0 (socket says 172.21.15.43:48402) 2023-08-09T14:52:18.678 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:18.677+0000 7fbbe2d9d640 1 -- 172.21.15.43:0/2477415165 learned_addr learned my addr 172.21.15.43:0/2477415165 (peer_addr_for_me v1:172.21.15.43:0/0) 2023-08-09T14:52:18.678 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:18.677+0000 7fbbe159a640 1 -- 172.21.15.43:0/2477415165 <== mon.0 v1:172.21.15.43:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 261954290 0 0) 0x7fbbe415a4a0 con 0x7fbbe40ce1c0 2023-08-09T14:52:18.678 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:18.677+0000 7fbbe1d9b640 1 -- 172.21.15.43:0/2477415165 >> v1:172.21.15.203:6789/0 conn(0x7fbbe415a0c0 legacy=0x7fbbe415d510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-09T14:52:18.678 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:18.677+0000 7fbbe1d9b640 1 -- 172.21.15.43:0/2477415165 >> v1:172.21.15.43:6789/0 conn(0x7fbbe40ce1c0 legacy=0x7fbbe40ce5c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T14:52:18.678 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:18.677+0000 7fbbe1d9b640 1 -- 172.21.15.43:0/2477415165 --> v2:172.21.15.43:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbbe40cc5b0 con 0x7fbbe40cbb80 2023-08-09T14:52:18.679 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:18.678+0000 7fbbe1d9b640 1 --2- 172.21.15.43:0/2477415165 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fbbe40cbb80 0x7fbbe40cb2b0 secure :-1 s=READY pgs=1976 cs=0 l=1 rev1=1 crypto rx=0x7fbbd80097a0 tx=0x7fbbd802b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=348bb37fad3cda8d server_cookie=0 in_seq=0 out_seq=0 2023-08-09T14:52:18.679 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:18.679+0000 7fbbe159a640 1 -- 172.21.15.43:0/2477415165 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbbd8031830 con 0x7fbbe40cbb80 2023-08-09T14:52:18.679 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:18.679+0000 7fbbe159a640 1 -- 172.21.15.43:0/2477415165 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbbd8031990 con 0x7fbbe40cbb80 2023-08-09T14:52:18.679 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:18.679+0000 7fbbe159a640 1 -- 172.21.15.43:0/2477415165 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbbd8031d20 con 0x7fbbe40cbb80 2023-08-09T14:52:18.680 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:18.679+0000 7fbbe8ccc640 1 -- 172.21.15.43:0/2477415165 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fbbe40cbb80 msgr2=0x7fbbe40cb2b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T14:52:18.680 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:18.679+0000 7fbbe8ccc640 1 --2- 172.21.15.43:0/2477415165 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fbbe40cbb80 0x7fbbe40cb2b0 secure :-1 s=READY pgs=1976 cs=0 l=1 rev1=1 crypto rx=0x7fbbd80097a0 tx=0x7fbbd802b8d0 comp rx=0 tx=0).stop 2023-08-09T14:52:18.680 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:18.679+0000 7fbbe8ccc640 1 -- 172.21.15.43:0/2477415165 shutdown_connections 2023-08-09T14:52:18.681 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:18.679+0000 7fbbe8ccc640 1 --2- 172.21.15.43:0/2477415165 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fbbe40cbb80 0x7fbbe40cb2b0 unknown :-1 s=CLOSED pgs=1976 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T14:52:18.681 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:18.679+0000 7fbbe8ccc640 1 -- 172.21.15.43:0/2477415165 >> 172.21.15.43:0/2477415165 conn(0x7fbbe40aff50 msgr2=0x7fbbe40b0360 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-09T14:52:18.681 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:18.679+0000 7fbbe8ccc640 1 -- 172.21.15.43:0/2477415165 shutdown_connections 2023-08-09T14:52:18.681 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:18.679+0000 7fbbe8ccc640 1 -- 172.21.15.43:0/2477415165 wait complete. 2023-08-09T14:52:18.681 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:18.681+0000 7fbbe8ccc640 1 Processor -- start 2023-08-09T14:52:18.682 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:18.681+0000 7fbbe8ccc640 1 -- start start 2023-08-09T14:52:18.683 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:18.682+0000 7fbbe8ccc640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fbbe40cbb80 0x7fbbe4156f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T14:52:18.683 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:18.682+0000 7fbbe8ccc640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fbbe40ce1c0 0x7fbbe4070b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T14:52:18.683 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:18.682+0000 7fbbe8ccc640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fbbe415a0c0 0x7fbbe40b1440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T14:52:18.683 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:18.682+0000 7fbbe8ccc640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbbe4152e60 con 0x7fbbe40ce1c0 2023-08-09T14:52:18.683 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:18.682+0000 7fbbe8ccc640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbbe4153140 con 0x7fbbe40cbb80 2023-08-09T14:52:18.683 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:18.682+0000 7fbbe8ccc640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbbe4153420 con 0x7fbbe415a0c0 2023-08-09T14:52:18.684 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:18.683+0000 7fbbe1d9b640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fbbe40ce1c0 0x7fbbe4070b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T14:52:18.685 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:18.683+0000 7fbbe1d9b640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fbbe40ce1c0 0x7fbbe4070b30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:37866/0 (socket says 172.21.15.43:37866) 2023-08-09T14:52:18.685 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:18.683+0000 7fbbe1d9b640 1 -- 172.21.15.43:0/1441926077 learned_addr learned my addr 172.21.15.43:0/1441926077 (peer_addr_for_me v2:172.21.15.43:0/0) 2023-08-09T14:52:18.685 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:18.683+0000 7fbbe1d9b640 1 -- 172.21.15.43:0/1441926077 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fbbe415a0c0 msgr2=0x7fbbe40b1440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T14:52:18.685 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:18.683+0000 7fbbe1d9b640 1 --2- 172.21.15.43:0/1441926077 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fbbe415a0c0 0x7fbbe40b1440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T14:52:18.685 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:18.683+0000 7fbbe1d9b640 1 -- 172.21.15.43:0/1441926077 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fbbe40cbb80 msgr2=0x7fbbe4156f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T14:52:18.685 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:18.683+0000 7fbbe259c640 1 --2- 172.21.15.43:0/1441926077 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fbbe40cbb80 0x7fbbe4156f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T14:52:18.685 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:18.683+0000 7fbbe1d9b640 1 --2- 172.21.15.43:0/1441926077 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fbbe40cbb80 0x7fbbe4156f30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T14:52:18.686 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:18.683+0000 7fbbe1d9b640 1 -- 172.21.15.43:0/1441926077 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbbd8009400 con 0x7fbbe40ce1c0 2023-08-09T14:52:18.686 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:18.683+0000 7fbbe1d9b640 1 --2- 172.21.15.43:0/1441926077 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fbbe40ce1c0 0x7fbbe4070b30 secure :-1 s=READY pgs=1977 cs=0 l=1 rev1=1 crypto rx=0x7fbbd80098c0 tx=0x7fbbd80046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-09T14:52:18.686 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:18.684+0000 7fbbd2ffd640 1 -- 172.21.15.43:0/1441926077 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbbd803ba70 con 0x7fbbe40ce1c0 2023-08-09T14:52:18.686 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:18.684+0000 7fbbd2ffd640 1 -- 172.21.15.43:0/1441926077 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbbd803bbd0 con 0x7fbbe40ce1c0 2023-08-09T14:52:18.686 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:18.684+0000 7fbbd2ffd640 1 -- 172.21.15.43:0/1441926077 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbbd80453f0 con 0x7fbbe40ce1c0 2023-08-09T14:52:18.686 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:18.684+0000 7fbbe8ccc640 1 -- 172.21.15.43:0/1441926077 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbbe4153700 con 0x7fbbe40ce1c0 2023-08-09T14:52:18.686 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:18.684+0000 7fbbe8ccc640 1 -- 172.21.15.43:0/1441926077 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbbe4153cc0 con 0x7fbbe40ce1c0 2023-08-09T14:52:18.687 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:18.685+0000 7fbbe8ccc640 1 -- 172.21.15.43:0/1441926077 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbbe4153eb0 con 0x7fbbe40ce1c0 2023-08-09T14:52:18.687 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:18.686+0000 7fbbd2ffd640 1 -- 172.21.15.43:0/1441926077 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 88) v1 ==== 88443+0+0 (secure 0 0 0) 0x7fbbd804e450 con 0x7fbbe40ce1c0 2023-08-09T14:52:18.688 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:18.687+0000 7fbbd2ffd640 1 --2- 172.21.15.43:0/1441926077 >> [v2:172.21.15.203:6832/2959400181,v1:172.21.15.203:6833/2959400181] conn(0x7fbbc406a170 0x7fbbc406c630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T14:52:18.688 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:18.688+0000 7fbbd2ffd640 1 -- 172.21.15.43:0/1441926077 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(177..177 src has 1..177) v4 ==== 6348+0+0 (secure 0 0 0) 0x7fbbd80aabe0 con 0x7fbbe40ce1c0 2023-08-09T14:52:18.689 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:18.688+0000 7fbbe259c640 1 --2- 172.21.15.43:0/1441926077 >> [v2:172.21.15.203:6832/2959400181,v1:172.21.15.203:6833/2959400181] conn(0x7fbbc406a170 0x7fbbc406c630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T14:52:18.690 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:18.689+0000 7fbbe259c640 1 --2- 172.21.15.43:0/1441926077 >> [v2:172.21.15.203:6832/2959400181,v1:172.21.15.203:6833/2959400181] conn(0x7fbbc406a170 0x7fbbc406c630 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7fbbd400a750 tx=0x7fbbd4008040 comp rx=0 tx=0).ready entity=mgr.7223 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-09T14:52:18.692 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:18.692+0000 7fbbd2ffd640 1 -- 172.21.15.43:0/1441926077 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178066 (secure 0 0 0) 0x7fbbd8075f90 con 0x7fbbe40ce1c0 2023-08-09T14:52:18.900 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:18.900+0000 7fbbe8ccc640 1 -- 172.21.15.43:0/1441926077 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "config rm", "who": "mgr", "name": "mgr/snap_schedule/dump_on_update"} v 0) v1 -- 0x7fbbe40739f0 con 0x7fbbe40ce1c0 2023-08-09T14:52:18.909 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:18.909+0000 7fbbd2ffd640 1 -- 172.21.15.43:0/1441926077 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "config rm", "who": "mgr", "name": "mgr/snap_schedule/dump_on_update"}]=0 v55) v1 ==== 115+0+0 (secure 0 0 0) 0x7fbbd80795b0 con 0x7fbbe40ce1c0 2023-08-09T14:52:18.912 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:18.911+0000 7fbbe8ccc640 1 -- 172.21.15.43:0/1441926077 >> [v2:172.21.15.203:6832/2959400181,v1:172.21.15.203:6833/2959400181] conn(0x7fbbc406a170 msgr2=0x7fbbc406c630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T14:52:18.912 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:18.911+0000 7fbbe8ccc640 1 --2- 172.21.15.43:0/1441926077 >> [v2:172.21.15.203:6832/2959400181,v1:172.21.15.203:6833/2959400181] conn(0x7fbbc406a170 0x7fbbc406c630 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7fbbd400a750 tx=0x7fbbd4008040 comp rx=0 tx=0).stop 2023-08-09T14:52:18.913 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:18.911+0000 7fbbe8ccc640 1 -- 172.21.15.43:0/1441926077 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fbbe40ce1c0 msgr2=0x7fbbe4070b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T14:52:18.913 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:18.911+0000 7fbbe8ccc640 1 --2- 172.21.15.43:0/1441926077 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fbbe40ce1c0 0x7fbbe4070b30 secure :-1 s=READY pgs=1977 cs=0 l=1 rev1=1 crypto rx=0x7fbbd80098c0 tx=0x7fbbd80046c0 comp rx=0 tx=0).stop 2023-08-09T14:52:18.913 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:18.912+0000 7fbbe8ccc640 1 -- 172.21.15.43:0/1441926077 shutdown_connections 2023-08-09T14:52:18.913 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:18.912+0000 7fbbe8ccc640 1 --2- 172.21.15.43:0/1441926077 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fbbe415a0c0 0x7fbbe40b1440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T14:52:18.913 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:18.912+0000 7fbbe8ccc640 1 --2- 172.21.15.43:0/1441926077 >> [v2:172.21.15.203:6832/2959400181,v1:172.21.15.203:6833/2959400181] conn(0x7fbbc406a170 0x7fbbc406c630 unknown :-1 s=CLOSED pgs=37 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T14:52:18.913 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:18.912+0000 7fbbe8ccc640 1 --2- 172.21.15.43:0/1441926077 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fbbe40ce1c0 0x7fbbe4070b30 unknown :-1 s=CLOSED pgs=1977 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T14:52:18.913 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:18.912+0000 7fbbe8ccc640 1 --2- 172.21.15.43:0/1441926077 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fbbe40cbb80 0x7fbbe4156f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T14:52:18.914 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:18.912+0000 7fbbe8ccc640 1 -- 172.21.15.43:0/1441926077 >> 172.21.15.43:0/1441926077 conn(0x7fbbe40aff50 msgr2=0x7fbbe415b5b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-09T14:52:18.914 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:18.912+0000 7fbbe8ccc640 1 -- 172.21.15.43:0/1441926077 shutdown_connections 2023-08-09T14:52:18.914 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:18.912+0000 7fbbe8ccc640 1 -- 172.21.15.43:0/1441926077 wait complete. 2023-08-09T14:52:18.931 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config rm mgr mgr/snap_schedule/allow_m_granularity 2023-08-09T14:52:19.160 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:19.159+0000 7f80b8d66640 1 Processor -- start 2023-08-09T14:52:19.160 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:19.160+0000 7f80b8d66640 1 -- start start 2023-08-09T14:52:19.162 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:19.161+0000 7f80b8d66640 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7f80b4159400 0x7f80b40cb9a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T14:52:19.162 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:19.161+0000 7f80b8d66640 1 -- --> v1:172.21.15.43:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f80b4070250 con 0x7f80b40c8900 2023-08-09T14:52:19.162 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:19.161+0000 7f80b8d66640 1 -- --> v1:172.21.15.203:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f80b40c86c0 con 0x7f80b40c9560 2023-08-09T14:52:19.162 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:19.161+0000 7f80b8d66640 1 -- --> v2:172.21.15.203:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f80b406fc60 con 0x7f80b4159400 2023-08-09T14:52:19.163 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:19.162+0000 7f80abfff640 1 --1- >> v1:172.21.15.43:6789/0 conn(0x7f80b40c8900 0x7f80b40c8e50 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.43:6789/0 says I am v1:172.21.15.43:48410/0 (socket says 172.21.15.43:48410) 2023-08-09T14:52:19.164 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:19.162+0000 7f80abfff640 1 -- 172.21.15.43:0/1174115611 learned_addr learned my addr 172.21.15.43:0/1174115611 (peer_addr_for_me v1:172.21.15.43:0/0) 2023-08-09T14:52:19.164 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:19.162+0000 7f80b37fe640 1 --2- 172.21.15.43:0/1174115611 >> v2:172.21.15.203:3300/0 conn(0x7f80b4159400 0x7f80b40cb9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T14:52:19.164 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:19.162+0000 7f80b2ffd640 1 -- 172.21.15.43:0/1174115611 <== mon.0 v1:172.21.15.43:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2999093004 0 0) 0x7f80b4070250 con 0x7f80b40c8900 2023-08-09T14:52:19.164 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:19.162+0000 7f80b2ffd640 1 -- 172.21.15.43:0/1174115611 --> v1:172.21.15.43:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f80a0003520 con 0x7f80b40c8900 2023-08-09T14:52:19.164 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:19.163+0000 7f80b2ffd640 1 -- 172.21.15.43:0/1174115611 <== mon.0 v1:172.21.15.43:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2675829564 0 0) 0x7f80a0003520 con 0x7f80b40c8900 2023-08-09T14:52:19.164 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:19.163+0000 7f80b2ffd640 1 -- 172.21.15.43:0/1174115611 >> v2:172.21.15.203:3300/0 conn(0x7f80b4159400 msgr2=0x7f80b40cb9a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-09T14:52:19.164 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:19.163+0000 7f80b2ffd640 1 --2- 172.21.15.43:0/1174115611 >> v2:172.21.15.203:3300/0 conn(0x7f80b4159400 0x7f80b40cb9a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T14:52:19.165 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:19.163+0000 7f80b2ffd640 1 -- 172.21.15.43:0/1174115611 >> v1:172.21.15.203:6789/0 conn(0x7f80b40c9560 legacy=0x7f80b40706e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-09T14:52:19.165 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:19.163+0000 7f80b2ffd640 1 -- 172.21.15.43:0/1174115611 --> v1:172.21.15.43:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f80b40af290 con 0x7f80b40c8900 2023-08-09T14:52:19.165 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:19.163+0000 7f80b2ffd640 1 -- 172.21.15.43:0/1174115611 <== mon.0 v1:172.21.15.43:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1285150740 0 0) 0x7f809c001670 con 0x7f80b40c8900 2023-08-09T14:52:19.165 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:19.163+0000 7f80b2ffd640 1 -- 172.21.15.43:0/1174115611 >> v1:172.21.15.43:6789/0 conn(0x7f80b40c8900 legacy=0x7f80b40c8e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T14:52:19.165 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:19.163+0000 7f80b2ffd640 1 --2- 172.21.15.43:0/1174115611 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f80a0003970 0x7f80a0003d70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T14:52:19.165 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:19.163+0000 7f80b2ffd640 1 --2- 172.21.15.43:0/1174115611 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f80a00047c0 0x7f80a0004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T14:52:19.165 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:19.163+0000 7f80b2ffd640 1 --2- 172.21.15.43:0/1174115611 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f80a0005180 0x7f80a000b980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T14:52:19.165 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:19.163+0000 7f80b2ffd640 1 -- 172.21.15.43:0/1174115611 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f80b406fc60 con 0x7f80a0003970 2023-08-09T14:52:19.166 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:19.163+0000 7f80b2ffd640 1 -- 172.21.15.43:0/1174115611 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f80a000c0b0 con 0x7f80a00047c0 2023-08-09T14:52:19.166 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:19.163+0000 7f80b2ffd640 1 -- 172.21.15.43:0/1174115611 --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f80a000c270 con 0x7f80a0005180 2023-08-09T14:52:19.166 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:19.163+0000 7f80b37fe640 1 --2- 172.21.15.43:0/1174115611 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f80a0003970 0x7f80a0003d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T14:52:19.167 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:19.164+0000 7f80abfff640 1 --2- 172.21.15.43:0/1174115611 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f80a00047c0 0x7f80a0004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T14:52:19.167 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:19.164+0000 7f80b37fe640 1 -- 172.21.15.43:0/1174115611 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f80a0005180 msgr2=0x7f80a000b980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-09T14:52:19.167 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:19.164+0000 7f80b37fe640 1 --2- 172.21.15.43:0/1174115611 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f80a0005180 0x7f80a000b980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T14:52:19.168 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:19.164+0000 7f80b37fe640 1 -- 172.21.15.43:0/1174115611 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f80a00047c0 msgr2=0x7f80a0004c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-09T14:52:19.168 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:19.164+0000 7f80b37fe640 1 --2- 172.21.15.43:0/1174115611 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f80a00047c0 0x7f80a0004c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T14:52:19.168 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:19.164+0000 7f80b37fe640 1 -- 172.21.15.43:0/1174115611 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f80a0003520 con 0x7f80a0003970 2023-08-09T14:52:19.168 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:19.164+0000 7f80abfff640 1 --2- 172.21.15.43:0/1174115611 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f80a00047c0 0x7f80a0004c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-08-09T14:52:19.168 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:19.164+0000 7f80b3fff640 1 -- 172.21.15.43:0/1174115611 reap_dead start 2023-08-09T14:52:19.168 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:19.165+0000 7f80b37fe640 1 --2- 172.21.15.43:0/1174115611 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f80a0003970 0x7f80a0003d70 secure :-1 s=READY pgs=1979 cs=0 l=1 rev1=1 crypto rx=0x7f80a4009ac0 tx=0x7f80a00057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=938188f56c899a50 server_cookie=0 in_seq=0 out_seq=0 2023-08-09T14:52:19.168 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:19.165+0000 7f80b2ffd640 1 -- 172.21.15.43:0/1174115611 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f80a402c710 con 0x7f80a0003970 2023-08-09T14:52:19.169 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:19.165+0000 7f80b2ffd640 1 -- 172.21.15.43:0/1174115611 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f80a402c870 con 0x7f80a0003970 2023-08-09T14:52:19.169 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:19.165+0000 7f80b2ffd640 1 -- 172.21.15.43:0/1174115611 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f80a402cc00 con 0x7f80a0003970 2023-08-09T14:52:19.169 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:19.165+0000 7f80b8d66640 1 -- 172.21.15.43:0/1174115611 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f80a0003970 msgr2=0x7f80a0003d70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T14:52:19.169 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:19.165+0000 7f80b8d66640 1 --2- 172.21.15.43:0/1174115611 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f80a0003970 0x7f80a0003d70 secure :-1 s=READY pgs=1979 cs=0 l=1 rev1=1 crypto rx=0x7f80a4009ac0 tx=0x7f80a00057f0 comp rx=0 tx=0).stop 2023-08-09T14:52:19.169 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:19.166+0000 7f80b8d66640 1 -- 172.21.15.43:0/1174115611 shutdown_connections 2023-08-09T14:52:19.169 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:19.166+0000 7f80b8d66640 1 --2- 172.21.15.43:0/1174115611 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f80a0003970 0x7f80a0003d70 unknown :-1 s=CLOSED pgs=1979 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T14:52:19.169 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:19.166+0000 7f80b8d66640 1 -- 172.21.15.43:0/1174115611 >> 172.21.15.43:0/1174115611 conn(0x7f80b40cddd0 msgr2=0x7f80b40ce1e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-09T14:52:19.169 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:19.166+0000 7f80b8d66640 1 -- 172.21.15.43:0/1174115611 shutdown_connections 2023-08-09T14:52:19.170 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:19.166+0000 7f80b8d66640 1 -- 172.21.15.43:0/1174115611 wait complete. 2023-08-09T14:52:19.171 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:19.168+0000 7f80b8d66640 1 Processor -- start 2023-08-09T14:52:19.171 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:19.168+0000 7f80b8d66640 1 -- start start 2023-08-09T14:52:19.171 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:19.169+0000 7f80b8d66640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f80a0003970 0x7f80b40bf280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T14:52:19.172 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:19.169+0000 7f80b8d66640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f80b40c7ad0 0x7f80b406c360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T14:52:19.172 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:19.169+0000 7f80b8d66640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f80b406c8a0 0x7f80b415c270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T14:52:19.172 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:19.169+0000 7f80b8d66640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f80b406cd20 con 0x7f80b40c7ad0 2023-08-09T14:52:19.172 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:19.169+0000 7f80b8d66640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f80b40c0770 con 0x7f80a0003970 2023-08-09T14:52:19.172 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:19.169+0000 7f80b8d66640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f80b40c4ae0 con 0x7f80b406c8a0 2023-08-09T14:52:19.172 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:19.169+0000 7f80abfff640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f80b40c7ad0 0x7f80b406c360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T14:52:19.172 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:19.169+0000 7f80abfff640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f80b40c7ad0 0x7f80b406c360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:37880/0 (socket says 172.21.15.43:37880) 2023-08-09T14:52:19.173 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:19.169+0000 7f80abfff640 1 -- 172.21.15.43:0/450528042 learned_addr learned my addr 172.21.15.43:0/450528042 (peer_addr_for_me v2:172.21.15.43:0/0) 2023-08-09T14:52:19.173 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:19.169+0000 7f80b37fe640 1 --2- 172.21.15.43:0/450528042 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f80a0003970 0x7f80b40bf280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T14:52:19.173 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:19.169+0000 7f80abfff640 1 -- 172.21.15.43:0/450528042 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f80b406c8a0 msgr2=0x7f80b415c270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T14:52:19.173 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:19.169+0000 7f80b3fff640 1 --2- 172.21.15.43:0/450528042 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f80b406c8a0 0x7f80b415c270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T14:52:19.173 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:19.169+0000 7f80abfff640 1 --2- 172.21.15.43:0/450528042 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f80b406c8a0 0x7f80b415c270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T14:52:19.173 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:19.169+0000 7f80abfff640 1 -- 172.21.15.43:0/450528042 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f80a0003970 msgr2=0x7f80b40bf280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T14:52:19.173 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:19.169+0000 7f80abfff640 1 --2- 172.21.15.43:0/450528042 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f80a0003970 0x7f80b40bf280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T14:52:19.174 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:19.170+0000 7f80abfff640 1 -- 172.21.15.43:0/450528042 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f80b40bfb60 con 0x7f80b40c7ad0 2023-08-09T14:52:19.174 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:19.170+0000 7f80b37fe640 1 --2- 172.21.15.43:0/450528042 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f80a0003970 0x7f80b40bf280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-08-09T14:52:19.174 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:19.170+0000 7f80b3fff640 1 --2- 172.21.15.43:0/450528042 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f80b406c8a0 0x7f80b415c270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-08-09T14:52:19.174 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:19.170+0000 7f80abfff640 1 --2- 172.21.15.43:0/450528042 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f80b40c7ad0 0x7f80b406c360 secure :-1 s=READY pgs=1980 cs=0 l=1 rev1=1 crypto rx=0x7f809c00db40 tx=0x7f809c007e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-09T14:52:19.174 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:19.170+0000 7f80b0ff9640 1 -- 172.21.15.43:0/450528042 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f809c0074e0 con 0x7f80b40c7ad0 2023-08-09T14:52:19.174 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:19.170+0000 7f80b0ff9640 1 -- 172.21.15.43:0/450528042 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f809c007640 con 0x7f80b40c7ad0 2023-08-09T14:52:19.174 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:19.170+0000 7f80b8d66640 1 -- 172.21.15.43:0/450528042 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f80b40bfe40 con 0x7f80b40c7ad0 2023-08-09T14:52:19.174 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:19.170+0000 7f80b0ff9640 1 -- 172.21.15.43:0/450528042 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f809c0049e0 con 0x7f80b40c7ad0 2023-08-09T14:52:19.175 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:19.170+0000 7f80b8d66640 1 -- 172.21.15.43:0/450528042 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f80b40c0400 con 0x7f80b40c7ad0 2023-08-09T14:52:19.175 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:19.172+0000 7f80b8d66640 1 -- 172.21.15.43:0/450528042 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f80b404e390 con 0x7f80b40c7ad0 2023-08-09T14:52:19.175 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:19.173+0000 7f80b0ff9640 1 -- 172.21.15.43:0/450528042 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 88) v1 ==== 88443+0+0 (secure 0 0 0) 0x7f809c004b40 con 0x7f80b40c7ad0 2023-08-09T14:52:19.176 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:19.174+0000 7f80b0ff9640 1 --2- 172.21.15.43:0/450528042 >> [v2:172.21.15.203:6832/2959400181,v1:172.21.15.203:6833/2959400181] conn(0x7f809406a1a0 0x7f809406c660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T14:52:19.176 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:19.174+0000 7f80b0ff9640 1 -- 172.21.15.43:0/450528042 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(177..177 src has 1..177) v4 ==== 6348+0+0 (secure 0 0 0) 0x7f809c0884b0 con 0x7f80b40c7ad0 2023-08-09T14:52:19.176 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:19.175+0000 7f80b37fe640 1 --2- 172.21.15.43:0/450528042 >> [v2:172.21.15.203:6832/2959400181,v1:172.21.15.203:6833/2959400181] conn(0x7f809406a1a0 0x7f809406c660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T14:52:19.176 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:19.176+0000 7f80b37fe640 1 --2- 172.21.15.43:0/450528042 >> [v2:172.21.15.203:6832/2959400181,v1:172.21.15.203:6833/2959400181] conn(0x7f809406a1a0 0x7f809406c660 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7f80a4036000 tx=0x7f80a4030000 comp rx=0 tx=0).ready entity=mgr.7223 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-09T14:52:19.178 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:19.178+0000 7f80b0ff9640 1 -- 172.21.15.43:0/450528042 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178066 (secure 0 0 0) 0x7f809c0537c0 con 0x7f80b40c7ad0 2023-08-09T14:52:19.385 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:19.384+0000 7f80b8d66640 1 -- 172.21.15.43:0/450528042 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "config rm", "who": "mgr", "name": "mgr/snap_schedule/allow_m_granularity"} v 0) v1 -- 0x7f80b40cea30 con 0x7f80b40c7ad0 2023-08-09T14:52:19.394 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:19.393+0000 7f80b0ff9640 1 -- 172.21.15.43:0/450528042 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "config rm", "who": "mgr", "name": "mgr/snap_schedule/allow_m_granularity"}]=0 v56) v1 ==== 120+0+0 (secure 0 0 0) 0x7f809c056de0 con 0x7f80b40c7ad0 2023-08-09T14:52:19.397 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:19.396+0000 7f80b8d66640 1 -- 172.21.15.43:0/450528042 >> [v2:172.21.15.203:6832/2959400181,v1:172.21.15.203:6833/2959400181] conn(0x7f809406a1a0 msgr2=0x7f809406c660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T14:52:19.397 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:19.396+0000 7f80b8d66640 1 --2- 172.21.15.43:0/450528042 >> [v2:172.21.15.203:6832/2959400181,v1:172.21.15.203:6833/2959400181] conn(0x7f809406a1a0 0x7f809406c660 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7f80a4036000 tx=0x7f80a4030000 comp rx=0 tx=0).stop 2023-08-09T14:52:19.397 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:19.396+0000 7f80b8d66640 1 -- 172.21.15.43:0/450528042 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f80b40c7ad0 msgr2=0x7f80b406c360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T14:52:19.397 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:19.396+0000 7f80b8d66640 1 --2- 172.21.15.43:0/450528042 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f80b40c7ad0 0x7f80b406c360 secure :-1 s=READY pgs=1980 cs=0 l=1 rev1=1 crypto rx=0x7f809c00db40 tx=0x7f809c007e30 comp rx=0 tx=0).stop 2023-08-09T14:52:19.397 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:19.396+0000 7f80b8d66640 1 -- 172.21.15.43:0/450528042 shutdown_connections 2023-08-09T14:52:19.398 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:19.396+0000 7f80b8d66640 1 --2- 172.21.15.43:0/450528042 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f80b406c8a0 0x7f80b415c270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T14:52:19.398 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:19.396+0000 7f80b8d66640 1 --2- 172.21.15.43:0/450528042 >> [v2:172.21.15.203:6832/2959400181,v1:172.21.15.203:6833/2959400181] conn(0x7f809406a1a0 0x7f809406c660 unknown :-1 s=CLOSED pgs=38 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T14:52:19.398 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:19.396+0000 7f80b8d66640 1 --2- 172.21.15.43:0/450528042 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f80b40c7ad0 0x7f80b406c360 unknown :-1 s=CLOSED pgs=1980 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T14:52:19.398 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:19.396+0000 7f80b8d66640 1 --2- 172.21.15.43:0/450528042 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f80a0003970 0x7f80b40bf280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T14:52:19.398 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:19.396+0000 7f80b8d66640 1 -- 172.21.15.43:0/450528042 >> 172.21.15.43:0/450528042 conn(0x7f80b40cddd0 msgr2=0x7f80b4067560 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-09T14:52:19.398 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:19.396+0000 7f80b8d66640 1 -- 172.21.15.43:0/450528042 shutdown_connections 2023-08-09T14:52:19.398 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:19.396+0000 7f80b8d66640 1 -- 172.21.15.43:0/450528042 wait complete. 2023-08-09T14:52:19.415 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config rm mon mon_allow_pool_delete 2023-08-09T14:52:19.645 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:19.645+0000 7f68f25ab640 1 Processor -- start 2023-08-09T14:52:19.645 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:19.645+0000 7f68f25ab640 1 -- start start 2023-08-09T14:52:19.647 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:19.646+0000 7f68f25ab640 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7f68ec15c550 0x7f68ec0cd6d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T14:52:19.647 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:19.647+0000 7f68f25ab640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f68ec0b45f0 0x7f68ec0b59d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T14:52:19.647 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:19.647+0000 7f68f25ab640 1 -- --> v1:172.21.15.43:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f68ec0b7bd0 con 0x7f68ec0d04b0 2023-08-09T14:52:19.647 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:19.647+0000 7f68f25ab640 1 -- --> v2:172.21.15.43:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f68ec0d1c00 con 0x7f68ec0b45f0 2023-08-09T14:52:19.648 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:19.647+0000 7f68f25ab640 1 -- --> v2:172.21.15.203:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f68ec15c930 con 0x7f68ec15c550 2023-08-09T14:52:19.648 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:19.647+0000 7f68eb7fe640 1 --1- >> v1:172.21.15.43:6789/0 conn(0x7f68ec0d04b0 0x7f68ec0b3ee0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.43:6789/0 says I am v1:172.21.15.43:48420/0 (socket says 172.21.15.43:48420) 2023-08-09T14:52:19.649 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:19.647+0000 7f68eb7fe640 1 -- 172.21.15.43:0/1664852511 learned_addr learned my addr 172.21.15.43:0/1664852511 (peer_addr_for_me v1:172.21.15.43:0/0) 2023-08-09T14:52:19.649 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:19.647+0000 7f68f0b48640 1 --2- 172.21.15.43:0/1664852511 >> v2:172.21.15.43:3300/0 conn(0x7f68ec0b45f0 0x7f68ec0b59d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T14:52:19.649 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:19.647+0000 7f68ebfff640 1 --2- 172.21.15.43:0/1664852511 >> v2:172.21.15.203:3300/0 conn(0x7f68ec15c550 0x7f68ec0cd6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T14:52:19.649 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:19.647+0000 7f68eaffd640 1 -- 172.21.15.43:0/1664852511 <== mon.0 v1:172.21.15.43:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 883440352 0 0) 0x7f68ec0b7bd0 con 0x7f68ec0d04b0 2023-08-09T14:52:19.649 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:19.647+0000 7f68eaffd640 1 -- 172.21.15.43:0/1664852511 --> v1:172.21.15.43:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f68d4003520 con 0x7f68ec0d04b0 2023-08-09T14:52:19.649 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:19.647+0000 7f68f0b48640 1 -- 172.21.15.43:0/1664852511 >> v2:172.21.15.203:3300/0 conn(0x7f68ec15c550 msgr2=0x7f68ec0cd6d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-09T14:52:19.649 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:19.647+0000 7f68f0b48640 1 --2- 172.21.15.43:0/1664852511 >> v2:172.21.15.203:3300/0 conn(0x7f68ec15c550 0x7f68ec0cd6d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T14:52:19.650 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:19.647+0000 7f68f0b48640 1 -- 172.21.15.43:0/1664852511 >> v1:172.21.15.43:6789/0 conn(0x7f68ec0d04b0 legacy=0x7f68ec0b3ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T14:52:19.650 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:19.648+0000 7f68f0b48640 1 -- 172.21.15.43:0/1664852511 --> v2:172.21.15.43:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f68ec0b54f0 con 0x7f68ec0b45f0 2023-08-09T14:52:19.650 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:19.648+0000 7f68ebfff640 1 --2- 172.21.15.43:0/1664852511 >> v2:172.21.15.203:3300/0 conn(0x7f68ec15c550 0x7f68ec0cd6d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-08-09T14:52:19.651 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:19.649+0000 7f68f0b48640 1 --2- 172.21.15.43:0/1664852511 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f68ec0b45f0 0x7f68ec0b59d0 secure :-1 s=READY pgs=1982 cs=0 l=1 rev1=1 crypto rx=0x7f68ec0d01c0 tx=0x7f68e402b840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=37daa056853f5918 server_cookie=0 in_seq=0 out_seq=0 2023-08-09T14:52:19.651 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:19.649+0000 7f68eaffd640 1 -- 172.21.15.43:0/1664852511 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f68e402e070 con 0x7f68ec0b45f0 2023-08-09T14:52:19.651 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:19.649+0000 7f68eaffd640 1 -- 172.21.15.43:0/1664852511 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f68e40315a0 con 0x7f68ec0b45f0 2023-08-09T14:52:19.651 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:19.649+0000 7f68eaffd640 1 -- 172.21.15.43:0/1664852511 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f68e4031910 con 0x7f68ec0b45f0 2023-08-09T14:52:19.651 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:19.650+0000 7f68f25ab640 1 -- 172.21.15.43:0/1664852511 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f68ec0b45f0 msgr2=0x7f68ec0b59d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T14:52:19.652 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:19.650+0000 7f68f25ab640 1 --2- 172.21.15.43:0/1664852511 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f68ec0b45f0 0x7f68ec0b59d0 secure :-1 s=READY pgs=1982 cs=0 l=1 rev1=1 crypto rx=0x7f68ec0d01c0 tx=0x7f68e402b840 comp rx=0 tx=0).stop 2023-08-09T14:52:19.652 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:19.650+0000 7f68f25ab640 1 -- 172.21.15.43:0/1664852511 shutdown_connections 2023-08-09T14:52:19.652 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:19.650+0000 7f68f25ab640 1 --2- 172.21.15.43:0/1664852511 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f68ec0b45f0 0x7f68ec0b59d0 unknown :-1 s=CLOSED pgs=1982 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T14:52:19.652 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:19.650+0000 7f68f25ab640 1 --2- 172.21.15.43:0/1664852511 >> v2:172.21.15.203:3300/0 conn(0x7f68ec15c550 0x7f68ec0cd6d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T14:52:19.652 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:19.650+0000 7f68f25ab640 1 -- 172.21.15.43:0/1664852511 >> 172.21.15.43:0/1664852511 conn(0x7f68ec071520 msgr2=0x7f68ec0ceee0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-09T14:52:19.652 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:19.650+0000 7f68f25ab640 1 -- 172.21.15.43:0/1664852511 shutdown_connections 2023-08-09T14:52:19.653 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:19.650+0000 7f68f25ab640 1 -- 172.21.15.43:0/1664852511 wait complete. 2023-08-09T14:52:19.654 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:19.652+0000 7f68f25ab640 1 Processor -- start 2023-08-09T14:52:19.654 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:19.652+0000 7f68f25ab640 1 -- start start 2023-08-09T14:52:19.654 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:19.653+0000 7f68f25ab640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f68ec0b45f0 0x7f68ec14f250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T14:52:19.654 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:19.653+0000 7f68f25ab640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f68ec0d04b0 0x7f68ec06e7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T14:52:19.654 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:19.653+0000 7f68f25ab640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f68ec15c550 0x7f68ec06cae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T14:52:19.654 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:19.653+0000 7f68f25ab640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f68ec1504e0 con 0x7f68ec0d04b0 2023-08-09T14:52:19.655 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:19.653+0000 7f68f25ab640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f68ec06d3a0 con 0x7f68ec0b45f0 2023-08-09T14:52:19.655 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:19.653+0000 7f68f25ab640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f68ec14f8f0 con 0x7f68ec15c550 2023-08-09T14:52:19.655 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:19.653+0000 7f68eb7fe640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f68ec0d04b0 0x7f68ec06e7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T14:52:19.655 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:19.653+0000 7f68eb7fe640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f68ec0d04b0 0x7f68ec06e7a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:37902/0 (socket says 172.21.15.43:37902) 2023-08-09T14:52:19.655 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:19.653+0000 7f68eb7fe640 1 -- 172.21.15.43:0/3275861205 learned_addr learned my addr 172.21.15.43:0/3275861205 (peer_addr_for_me v2:172.21.15.43:0/0) 2023-08-09T14:52:19.656 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:19.654+0000 7f68ebfff640 1 --2- 172.21.15.43:0/3275861205 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f68ec0b45f0 0x7f68ec14f250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T14:52:19.656 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:19.654+0000 7f68eb7fe640 1 -- 172.21.15.43:0/3275861205 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f68ec15c550 msgr2=0x7f68ec06cae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T14:52:19.657 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:19.654+0000 7f68eb7fe640 1 --2- 172.21.15.43:0/3275861205 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f68ec15c550 0x7f68ec06cae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T14:52:19.657 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:19.654+0000 7f68eb7fe640 1 -- 172.21.15.43:0/3275861205 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f68ec0b45f0 msgr2=0x7f68ec14f250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T14:52:19.657 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:19.654+0000 7f68eb7fe640 1 --2- 172.21.15.43:0/3275861205 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f68ec0b45f0 0x7f68ec14f250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T14:52:19.657 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:19.654+0000 7f68eb7fe640 1 -- 172.21.15.43:0/3275861205 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f68e4009400 con 0x7f68ec0d04b0 2023-08-09T14:52:19.657 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:19.654+0000 7f68ebfff640 1 --2- 172.21.15.43:0/3275861205 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f68ec0b45f0 0x7f68ec14f250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-08-09T14:52:19.657 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:19.654+0000 7f68eb7fe640 1 --2- 172.21.15.43:0/3275861205 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f68ec0d04b0 0x7f68ec06e7a0 secure :-1 s=READY pgs=1983 cs=0 l=1 rev1=1 crypto rx=0x7f68e000cbb0 tx=0x7f68e000a430 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-09T14:52:19.657 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:19.654+0000 7f68e8ff9640 1 -- 172.21.15.43:0/3275861205 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f68e000ace0 con 0x7f68ec0d04b0 2023-08-09T14:52:19.658 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:19.654+0000 7f68e8ff9640 1 -- 172.21.15.43:0/3275861205 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f68e000ae60 con 0x7f68ec0d04b0 2023-08-09T14:52:19.658 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:19.654+0000 7f68e8ff9640 1 -- 172.21.15.43:0/3275861205 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f68e0012640 con 0x7f68ec0d04b0 2023-08-09T14:52:19.658 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:19.655+0000 7f68f25ab640 1 -- 172.21.15.43:0/3275861205 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f68ec14fb40 con 0x7f68ec0d04b0 2023-08-09T14:52:19.658 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:19.655+0000 7f68f25ab640 1 -- 172.21.15.43:0/3275861205 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f68ec150100 con 0x7f68ec0d04b0 2023-08-09T14:52:19.658 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:19.656+0000 7f68f25ab640 1 -- 172.21.15.43:0/3275861205 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f68ec150620 con 0x7f68ec0d04b0 2023-08-09T14:52:19.659 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:19.657+0000 7f68e8ff9640 1 -- 172.21.15.43:0/3275861205 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 88) v1 ==== 88443+0+0 (secure 0 0 0) 0x7f68e0007070 con 0x7f68ec0d04b0 2023-08-09T14:52:19.659 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:19.658+0000 7f68e8ff9640 1 --2- 172.21.15.43:0/3275861205 >> [v2:172.21.15.203:6832/2959400181,v1:172.21.15.203:6833/2959400181] conn(0x7f68d806a170 0x7f68d806c630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T14:52:19.659 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:19.658+0000 7f68e8ff9640 1 -- 172.21.15.43:0/3275861205 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(177..177 src has 1..177) v4 ==== 6348+0+0 (secure 0 0 0) 0x7f68e0088950 con 0x7f68ec0d04b0 2023-08-09T14:52:19.659 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:19.659+0000 7f68ebfff640 1 --2- 172.21.15.43:0/3275861205 >> [v2:172.21.15.203:6832/2959400181,v1:172.21.15.203:6833/2959400181] conn(0x7f68d806a170 0x7f68d806c630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T14:52:19.660 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:19.660+0000 7f68ebfff640 1 --2- 172.21.15.43:0/3275861205 >> [v2:172.21.15.203:6832/2959400181,v1:172.21.15.203:6833/2959400181] conn(0x7f68d806a170 0x7f68d806c630 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7f68dc006fd0 tx=0x7f68dc008040 comp rx=0 tx=0).ready entity=mgr.7223 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-09T14:52:19.662 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:19.662+0000 7f68e8ff9640 1 -- 172.21.15.43:0/3275861205 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178066 (secure 0 0 0) 0x7f68ec150620 con 0x7f68ec0d04b0 2023-08-09T14:52:19.858 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:19.858+0000 7f68f25ab640 1 -- 172.21.15.43:0/3275861205 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"} v 0) v1 -- 0x7f68ec150620 con 0x7f68ec0d04b0 2023-08-09T14:52:19.864 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:19.864+0000 7f68e8ff9640 1 -- 172.21.15.43:0/3275861205 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"}]=0 v57) v1 ==== 104+0+0 (secure 0 0 0) 0x7f68ec150620 con 0x7f68ec0d04b0 2023-08-09T14:52:19.868 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:19.866+0000 7f68f25ab640 1 -- 172.21.15.43:0/3275861205 >> [v2:172.21.15.203:6832/2959400181,v1:172.21.15.203:6833/2959400181] conn(0x7f68d806a170 msgr2=0x7f68d806c630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T14:52:19.868 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:19.866+0000 7f68f25ab640 1 --2- 172.21.15.43:0/3275861205 >> [v2:172.21.15.203:6832/2959400181,v1:172.21.15.203:6833/2959400181] conn(0x7f68d806a170 0x7f68d806c630 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7f68dc006fd0 tx=0x7f68dc008040 comp rx=0 tx=0).stop 2023-08-09T14:52:19.868 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:19.866+0000 7f68f25ab640 1 -- 172.21.15.43:0/3275861205 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f68ec0d04b0 msgr2=0x7f68ec06e7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T14:52:19.869 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:19.866+0000 7f68f25ab640 1 --2- 172.21.15.43:0/3275861205 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f68ec0d04b0 0x7f68ec06e7a0 secure :-1 s=READY pgs=1983 cs=0 l=1 rev1=1 crypto rx=0x7f68e000cbb0 tx=0x7f68e000a430 comp rx=0 tx=0).stop 2023-08-09T14:52:19.869 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:19.867+0000 7f68f25ab640 1 -- 172.21.15.43:0/3275861205 shutdown_connections 2023-08-09T14:52:19.869 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:19.867+0000 7f68f25ab640 1 --2- 172.21.15.43:0/3275861205 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f68ec15c550 0x7f68ec06cae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T14:52:19.869 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:19.867+0000 7f68f25ab640 1 --2- 172.21.15.43:0/3275861205 >> [v2:172.21.15.203:6832/2959400181,v1:172.21.15.203:6833/2959400181] conn(0x7f68d806a170 0x7f68d806c630 unknown :-1 s=CLOSED pgs=39 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T14:52:19.870 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:19.867+0000 7f68f25ab640 1 --2- 172.21.15.43:0/3275861205 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f68ec0d04b0 0x7f68ec06e7a0 unknown :-1 s=CLOSED pgs=1983 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T14:52:19.870 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:19.867+0000 7f68f25ab640 1 --2- 172.21.15.43:0/3275861205 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f68ec0b45f0 0x7f68ec14f250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T14:52:19.870 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:19.867+0000 7f68f25ab640 1 -- 172.21.15.43:0/3275861205 >> 172.21.15.43:0/3275861205 conn(0x7f68ec071520 msgr2=0x7f68ec0c7f80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-09T14:52:19.870 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:19.867+0000 7f68f25ab640 1 -- 172.21.15.43:0/3275861205 shutdown_connections 2023-08-09T14:52:19.871 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:19.867+0000 7f68f25ab640 1 -- 172.21.15.43:0/3275861205 wait complete. 2023-08-09T14:52:19.888 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Ended test tasks.cephfs.test_snap_schedules.TestSnapSchedules.test_snap_schedule_list_post_schedule_remove' 2023-08-09T14:52:20.115 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:20.115+0000 7f18e6ee5640 1 Processor -- start 2023-08-09T14:52:20.116 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:20.115+0000 7f18e6ee5640 1 -- start start 2023-08-09T14:52:20.116 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:20.116+0000 7f18e6ee5640 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7f18e015f7b0 0x7f18e00b4760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T14:52:20.117 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:20.116+0000 7f18e6ee5640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f18e00cd6b0 0x7f18e00d0e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T14:52:20.117 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:20.116+0000 7f18e6ee5640 1 -- --> v1:172.21.15.43:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f18e00b7bf0 con 0x7f18e00b1050 2023-08-09T14:52:20.117 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:20.116+0000 7f18e6ee5640 1 -- --> v2:172.21.15.203:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f18e00b7ed0 con 0x7f18e015f7b0 2023-08-09T14:52:20.117 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:20.116+0000 7f18e6ee5640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f18e00d0370 con 0x7f18e00cd6b0 2023-08-09T14:52:20.119 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:20.117+0000 7f18e56e2640 1 --1- >> v1:172.21.15.43:6789/0 conn(0x7f18e00b1050 0x7f18e00ccfa0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.43:6789/0 says I am v1:172.21.15.43:48430/0 (socket says 172.21.15.43:48430) 2023-08-09T14:52:20.119 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:20.117+0000 7f18e56e2640 1 -- 172.21.15.43:0/2113113627 learned_addr learned my addr 172.21.15.43:0/2113113627 (peer_addr_for_me v1:172.21.15.43:0/0) 2023-08-09T14:52:20.119 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:20.117+0000 7f18e5ee3640 1 --2- 172.21.15.43:0/2113113627 >> v2:172.21.15.203:3300/0 conn(0x7f18e015f7b0 0x7f18e00b4760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T14:52:20.119 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:20.117+0000 7f18e66e4640 1 --2- 172.21.15.43:0/2113113627 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f18e00cd6b0 0x7f18e00d0e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T14:52:20.119 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:20.117+0000 7f18e4ee1640 1 -- 172.21.15.43:0/2113113627 <== mon.0 v1:172.21.15.43:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1069892971 0 0) 0x7f18e00b7bf0 con 0x7f18e00b1050 2023-08-09T14:52:20.120 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:20.117+0000 7f18e4ee1640 1 -- 172.21.15.43:0/2113113627 --> v1:172.21.15.43:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f18cc003520 con 0x7f18e00b1050 2023-08-09T14:52:20.120 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:20.118+0000 7f18e4ee1640 1 -- 172.21.15.43:0/2113113627 <== mon.0 v1:172.21.15.43:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2102251928 0 0) 0x7f18cc003520 con 0x7f18e00b1050 2023-08-09T14:52:20.120 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:20.118+0000 7f18e4ee1640 1 -- 172.21.15.43:0/2113113627 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f18e00cd6b0 msgr2=0x7f18e00d0e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-09T14:52:20.120 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:20.118+0000 7f18e4ee1640 1 --2- 172.21.15.43:0/2113113627 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f18e00cd6b0 0x7f18e00d0e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T14:52:20.120 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:20.118+0000 7f18e4ee1640 1 -- 172.21.15.43:0/2113113627 >> v2:172.21.15.203:3300/0 conn(0x7f18e015f7b0 msgr2=0x7f18e00b4760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-09T14:52:20.120 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:20.118+0000 7f18e4ee1640 1 --2- 172.21.15.43:0/2113113627 >> v2:172.21.15.203:3300/0 conn(0x7f18e015f7b0 0x7f18e00b4760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T14:52:20.120 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:20.118+0000 7f18e4ee1640 1 -- 172.21.15.43:0/2113113627 --> v1:172.21.15.43:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f18e00d0650 con 0x7f18e00b1050 2023-08-09T14:52:20.120 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:20.118+0000 7f18e4ee1640 1 -- 172.21.15.43:0/2113113627 <== mon.0 v1:172.21.15.43:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1285150740 0 0) 0x7f18d0001670 con 0x7f18e00b1050 2023-08-09T14:52:20.121 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:20.118+0000 7f18e4ee1640 1 -- 172.21.15.43:0/2113113627 >> v1:172.21.15.43:6789/0 conn(0x7f18e00b1050 legacy=0x7f18e00ccfa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T14:52:20.121 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:20.118+0000 7f18e4ee1640 1 --2- 172.21.15.43:0/2113113627 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f18cc003970 0x7f18cc003d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T14:52:20.121 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:20.118+0000 7f18e4ee1640 1 --2- 172.21.15.43:0/2113113627 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f18e00cd6b0 0x7f18cc0047c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T14:52:20.121 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:20.118+0000 7f18e4ee1640 1 --2- 172.21.15.43:0/2113113627 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f18cc004d00 0x7f18cc005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T14:52:20.121 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:20.118+0000 7f18e4ee1640 1 -- 172.21.15.43:0/2113113627 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f18e00b7ed0 con 0x7f18cc003970 2023-08-09T14:52:20.121 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:20.118+0000 7f18e4ee1640 1 -- 172.21.15.43:0/2113113627 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f18e00d0370 con 0x7f18cc004d00 2023-08-09T14:52:20.121 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:20.118+0000 7f18e4ee1640 1 -- 172.21.15.43:0/2113113627 --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f18cc00b9b0 con 0x7f18e00cd6b0 2023-08-09T14:52:20.121 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:20.118+0000 7f18e5ee3640 1 --2- 172.21.15.43:0/2113113627 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f18cc003970 0x7f18cc003d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T14:52:20.122 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:20.118+0000 7f18e5ee3640 1 -- 172.21.15.43:0/2113113627 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f18e00cd6b0 msgr2=0x7f18cc0047c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-09T14:52:20.122 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:20.118+0000 7f18e5ee3640 1 --2- 172.21.15.43:0/2113113627 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f18e00cd6b0 0x7f18cc0047c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T14:52:20.122 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:20.118+0000 7f18e66e4640 1 --2- 172.21.15.43:0/2113113627 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f18cc004d00 0x7f18cc005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T14:52:20.122 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:20.118+0000 7f18e5ee3640 1 -- 172.21.15.43:0/2113113627 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f18cc004d00 msgr2=0x7f18cc005180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-09T14:52:20.122 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:20.118+0000 7f18e5ee3640 1 --2- 172.21.15.43:0/2113113627 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f18cc004d00 0x7f18cc005180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T14:52:20.122 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:20.119+0000 7f18e5ee3640 1 -- 172.21.15.43:0/2113113627 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f18cc003520 con 0x7f18cc003970 2023-08-09T14:52:20.123 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:20.120+0000 7f18e5ee3640 1 --2- 172.21.15.43:0/2113113627 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f18cc003970 0x7f18cc003d90 secure :-1 s=READY pgs=1985 cs=0 l=1 rev1=1 crypto rx=0x7f18dc009ac0 tx=0x7f18cc0057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=391c30096ebb7d20 server_cookie=0 in_seq=0 out_seq=0 2023-08-09T14:52:20.123 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:20.120+0000 7f18e4ee1640 1 -- 172.21.15.43:0/2113113627 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f18dc02c710 con 0x7f18cc003970 2023-08-09T14:52:20.124 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:20.120+0000 7f18e4ee1640 1 -- 172.21.15.43:0/2113113627 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f18dc02c870 con 0x7f18cc003970 2023-08-09T14:52:20.124 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:20.120+0000 7f18e4ee1640 1 -- 172.21.15.43:0/2113113627 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f18dc02cc00 con 0x7f18cc003970 2023-08-09T14:52:20.124 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:20.120+0000 7f18e6ee5640 1 -- 172.21.15.43:0/2113113627 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f18cc003970 msgr2=0x7f18cc003d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T14:52:20.124 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:20.120+0000 7f18e6ee5640 1 --2- 172.21.15.43:0/2113113627 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f18cc003970 0x7f18cc003d90 secure :-1 s=READY pgs=1985 cs=0 l=1 rev1=1 crypto rx=0x7f18dc009ac0 tx=0x7f18cc0057f0 comp rx=0 tx=0).stop 2023-08-09T14:52:20.124 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:20.120+0000 7f18e66e4640 1 -- 172.21.15.43:0/2113113627 reap_dead start 2023-08-09T14:52:20.124 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:20.121+0000 7f18e6ee5640 1 -- 172.21.15.43:0/2113113627 shutdown_connections 2023-08-09T14:52:20.124 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:20.121+0000 7f18e6ee5640 1 -- 172.21.15.43:0/2113113627 >> 172.21.15.43:0/2113113627 conn(0x7f18e006a2b0 msgr2=0x7f18e006a690 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-09T14:52:20.125 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:20.121+0000 7f18e6ee5640 1 -- 172.21.15.43:0/2113113627 shutdown_connections 2023-08-09T14:52:20.125 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:20.121+0000 7f18e6ee5640 1 -- 172.21.15.43:0/2113113627 wait complete. 2023-08-09T14:52:20.125 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:20.122+0000 7f18e6ee5640 1 Processor -- start 2023-08-09T14:52:20.125 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:20.123+0000 7f18e6ee5640 1 -- start start 2023-08-09T14:52:20.126 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:20.124+0000 7f18e6ee5640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f18cc003970 0x7f18e01f4580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T14:52:20.127 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:20.124+0000 7f18e6ee5640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f18e00d2080 0x7f18e00d2500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T14:52:20.127 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:20.124+0000 7f18e6ee5640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f18e01f1be0 0x7f18e01f2090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T14:52:20.127 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:20.124+0000 7f18e6ee5640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f18e01f5b60 con 0x7f18e01f1be0 2023-08-09T14:52:20.127 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:20.124+0000 7f18e6ee5640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f18e00d2a40 con 0x7f18e00d2080 2023-08-09T14:52:20.127 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:20.124+0000 7f18e6ee5640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f18e01f4ee0 con 0x7f18cc003970 2023-08-09T14:52:20.127 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:20.124+0000 7f18e66e4640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f18e01f1be0 0x7f18e01f2090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T14:52:20.127 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:20.124+0000 7f18e66e4640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f18e01f1be0 0x7f18e01f2090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:37926/0 (socket says 172.21.15.43:37926) 2023-08-09T14:52:20.128 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:20.124+0000 7f18e66e4640 1 -- 172.21.15.43:0/3295011693 learned_addr learned my addr 172.21.15.43:0/3295011693 (peer_addr_for_me v2:172.21.15.43:0/0) 2023-08-09T14:52:20.128 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:20.124+0000 7f18e56e2640 1 --2- 172.21.15.43:0/3295011693 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f18e00d2080 0x7f18e00d2500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T14:52:20.128 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:20.125+0000 7f18e5ee3640 1 --2- 172.21.15.43:0/3295011693 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f18cc003970 0x7f18e01f4580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T14:52:20.128 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:20.125+0000 7f18e66e4640 1 -- 172.21.15.43:0/3295011693 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f18cc003970 msgr2=0x7f18e01f4580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T14:52:20.128 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:20.125+0000 7f18e66e4640 1 --2- 172.21.15.43:0/3295011693 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f18cc003970 0x7f18e01f4580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T14:52:20.128 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:20.125+0000 7f18e66e4640 1 -- 172.21.15.43:0/3295011693 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f18e00d2080 msgr2=0x7f18e00d2500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T14:52:20.128 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:20.125+0000 7f18e66e4640 1 --2- 172.21.15.43:0/3295011693 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f18e00d2080 0x7f18e00d2500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T14:52:20.129 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:20.125+0000 7f18e66e4640 1 -- 172.21.15.43:0/3295011693 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f18e01f51c0 con 0x7f18e01f1be0 2023-08-09T14:52:20.129 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:20.125+0000 7f18e66e4640 1 --2- 172.21.15.43:0/3295011693 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f18e01f1be0 0x7f18e01f2090 secure :-1 s=READY pgs=1986 cs=0 l=1 rev1=1 crypto rx=0x7f18e006af30 tx=0x7f18d8009d40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-09T14:52:20.129 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:20.125+0000 7f18d67fc640 1 -- 172.21.15.43:0/3295011693 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f18d8007cf0 con 0x7f18e01f1be0 2023-08-09T14:52:20.129 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:20.125+0000 7f18d67fc640 1 -- 172.21.15.43:0/3295011693 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f18d8007e70 con 0x7f18e01f1be0 2023-08-09T14:52:20.129 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:20.125+0000 7f18d67fc640 1 -- 172.21.15.43:0/3295011693 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f18d8014620 con 0x7f18e01f1be0 2023-08-09T14:52:20.129 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:20.125+0000 7f18e6ee5640 1 -- 172.21.15.43:0/3295011693 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f18e01f54a0 con 0x7f18e01f1be0 2023-08-09T14:52:20.129 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:20.126+0000 7f18e6ee5640 1 -- 172.21.15.43:0/3295011693 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f18e01f25d0 con 0x7f18e01f1be0 2023-08-09T14:52:20.130 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:20.127+0000 7f18e6ee5640 1 -- 172.21.15.43:0/3295011693 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f18e00739f0 con 0x7f18e01f1be0 2023-08-09T14:52:20.130 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:20.128+0000 7f18d67fc640 1 -- 172.21.15.43:0/3295011693 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 88) v1 ==== 88443+0+0 (secure 0 0 0) 0x7f18d8014780 con 0x7f18e01f1be0 2023-08-09T14:52:20.130 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:20.129+0000 7f18d67fc640 1 --2- 172.21.15.43:0/3295011693 >> [v2:172.21.15.203:6832/2959400181,v1:172.21.15.203:6833/2959400181] conn(0x7f18c406a170 0x7f18c406c630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T14:52:20.130 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:20.129+0000 7f18d67fc640 1 -- 172.21.15.43:0/3295011693 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(177..177 src has 1..177) v4 ==== 6348+0+0 (secure 0 0 0) 0x7f18d8088400 con 0x7f18e01f1be0 2023-08-09T14:52:20.131 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:20.129+0000 7f18e5ee3640 1 --2- 172.21.15.43:0/3295011693 >> [v2:172.21.15.203:6832/2959400181,v1:172.21.15.203:6833/2959400181] conn(0x7f18c406a170 0x7f18c406c630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T14:52:20.131 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:20.130+0000 7f18e5ee3640 1 --2- 172.21.15.43:0/3295011693 >> [v2:172.21.15.203:6832/2959400181,v1:172.21.15.203:6833/2959400181] conn(0x7f18c406a170 0x7f18c406c630 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7f18dc009ac0 tx=0x7f18dc030000 comp rx=0 tx=0).ready entity=mgr.7223 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-09T14:52:20.133 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:20.133+0000 7f18d67fc640 1 -- 172.21.15.43:0/3295011693 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178066 (secure 0 0 0) 0x7f18d80573f0 con 0x7f18e01f1be0 2023-08-09T14:52:20.400 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:20.400+0000 7f18e6ee5640 1 -- 172.21.15.43:0/3295011693 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_snap_schedules.TestSnapSchedules.test_snap_schedule_list_post_schedule_remove"]} v 0) v1 -- 0x7f18e015c840 con 0x7f18e01f1be0 2023-08-09T14:52:20.433 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:20.432+0000 7f18d67fc640 1 -- 172.21.15.43:0/3295011693 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_snap_schedules.TestSnapSchedules.test_snap_schedule_list_post_schedule_remove"]}]=0 v2590) v1 ==== 174+0+0 (secure 0 0 0) 0x7f18d8056f80 con 0x7f18e01f1be0 2023-08-09T14:52:20.435 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:20.435+0000 7f18e6ee5640 1 -- 172.21.15.43:0/3295011693 >> [v2:172.21.15.203:6832/2959400181,v1:172.21.15.203:6833/2959400181] conn(0x7f18c406a170 msgr2=0x7f18c406c630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T14:52:20.436 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:20.435+0000 7f18e6ee5640 1 --2- 172.21.15.43:0/3295011693 >> [v2:172.21.15.203:6832/2959400181,v1:172.21.15.203:6833/2959400181] conn(0x7f18c406a170 0x7f18c406c630 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7f18dc009ac0 tx=0x7f18dc030000 comp rx=0 tx=0).stop 2023-08-09T14:52:20.436 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:20.435+0000 7f18e6ee5640 1 -- 172.21.15.43:0/3295011693 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f18e01f1be0 msgr2=0x7f18e01f2090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T14:52:20.436 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:20.435+0000 7f18e6ee5640 1 --2- 172.21.15.43:0/3295011693 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f18e01f1be0 0x7f18e01f2090 secure :-1 s=READY pgs=1986 cs=0 l=1 rev1=1 crypto rx=0x7f18e006af30 tx=0x7f18d8009d40 comp rx=0 tx=0).stop 2023-08-09T14:52:20.436 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:20.435+0000 7f18e6ee5640 1 -- 172.21.15.43:0/3295011693 shutdown_connections 2023-08-09T14:52:20.436 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:20.435+0000 7f18e6ee5640 1 --2- 172.21.15.43:0/3295011693 >> [v2:172.21.15.203:6832/2959400181,v1:172.21.15.203:6833/2959400181] conn(0x7f18c406a170 0x7f18c406c630 unknown :-1 s=CLOSED pgs=40 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T14:52:20.436 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:20.435+0000 7f18e6ee5640 1 --2- 172.21.15.43:0/3295011693 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f18e01f1be0 0x7f18e01f2090 unknown :-1 s=CLOSED pgs=1986 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T14:52:20.437 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:20.435+0000 7f18e6ee5640 1 --2- 172.21.15.43:0/3295011693 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f18e00d2080 0x7f18e00d2500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T14:52:20.437 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:20.435+0000 7f18e6ee5640 1 --2- 172.21.15.43:0/3295011693 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f18cc003970 0x7f18e01f4580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T14:52:20.437 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:20.435+0000 7f18e6ee5640 1 -- 172.21.15.43:0/3295011693 >> 172.21.15.43:0/3295011693 conn(0x7f18e006a2b0 msgr2=0x7f18e00cfc10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-09T14:52:20.437 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:20.435+0000 7f18e6ee5640 1 -- 172.21.15.43:0/3295011693 shutdown_connections 2023-08-09T14:52:20.437 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:20.435+0000 7f18e6ee5640 1 -- 172.21.15.43:0/3295011693 wait complete. 2023-08-09T14:52:20.453 INFO:tasks.cephfs_test_runner:Test listing snap schedules post removal of a schedule ... ok 2023-08-09T14:52:20.454 INFO:tasks.cephfs_test_runner:Starting test: test_snap_schedule_with_mgr_restart (tasks.cephfs.test_snap_schedules.TestSnapSchedules) Test that snap schedule is resumed after mgr restart 2023-08-09T14:52:20.454 INFO:tasks.cephfs_test_runner:test_snap_schedule_with_mgr_restart (tasks.cephfs.test_snap_schedules.TestSnapSchedules) 2023-08-09T14:52:20.454 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Starting test tasks.cephfs.test_snap_schedules.TestSnapSchedules.test_snap_schedule_with_mgr_restart' 2023-08-09T14:52:20.678 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:20.677+0000 7fac53f42640 1 Processor -- start 2023-08-09T14:52:20.678 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:20.677+0000 7fac53f42640 1 -- start start 2023-08-09T14:52:20.680 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:20.679+0000 7fac53f42640 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7fac4c143c20 0x7fac4c06f540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T14:52:20.680 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:20.679+0000 7fac53f42640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fac4c06cfe0 0x7fac4c0c8b50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T14:52:20.680 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:20.679+0000 7fac53f42640 1 -- --> v1:172.21.15.203:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fac4c0c9090 con 0x7fac4c0c86f0 2023-08-09T14:52:20.680 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:20.679+0000 7fac53f42640 1 -- --> v2:172.21.15.203:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fac4c144000 con 0x7fac4c143c20 2023-08-09T14:52:20.681 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:20.679+0000 7fac53f42640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fac4c06c440 con 0x7fac4c06cfe0 2023-08-09T14:52:20.681 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:20.680+0000 7fac51cde640 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7fac4c143c20 0x7fac4c06f540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T14:52:20.682 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:20.680+0000 7fac51cde640 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7fac4c143c20 0x7fac4c06f540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.203:3300/0 says I am v2:172.21.15.43:49650/0 (socket says 172.21.15.43:49650) 2023-08-09T14:52:20.682 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:20.680+0000 7fac51cde640 1 -- 172.21.15.43:0/526434704 learned_addr learned my addr 172.21.15.43:0/526434704 (peer_addr_for_me v2:172.21.15.43:0/0) 2023-08-09T14:52:20.682 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:20.680+0000 7fac524df640 1 --2- 172.21.15.43:0/526434704 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fac4c06cfe0 0x7fac4c0c8b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T14:52:20.682 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:20.681+0000 7fac51cde640 1 -- 172.21.15.43:0/526434704 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fac4c06cfe0 msgr2=0x7fac4c0c8b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-09T14:52:20.683 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:20.681+0000 7fac51cde640 1 --2- 172.21.15.43:0/526434704 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fac4c06cfe0 0x7fac4c0c8b50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T14:52:20.683 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:20.681+0000 7fac51cde640 1 -- 172.21.15.43:0/526434704 >> v1:172.21.15.203:6789/0 conn(0x7fac4c0c86f0 legacy=0x7fac4c06c8d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T14:52:20.683 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:20.681+0000 7fac51cde640 1 -- 172.21.15.43:0/526434704 --> v2:172.21.15.203:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fac4c06b830 con 0x7fac4c143c20 2023-08-09T14:52:20.683 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:20.681+0000 7fac524df640 1 --2- 172.21.15.43:0/526434704 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fac4c06cfe0 0x7fac4c0c8b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-08-09T14:52:20.684 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:20.682+0000 7fac51cde640 1 --2- 172.21.15.43:0/526434704 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fac4c143c20 0x7fac4c06f540 secure :-1 s=READY pgs=731 cs=0 l=1 rev1=1 crypto rx=0x7fac48009870 tx=0x7fac4802b8f0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=b7130804a89da96d server_cookie=0 in_seq=0 out_seq=0 2023-08-09T14:52:20.685 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:20.682+0000 7fac50cdc640 1 -- 172.21.15.43:0/526434704 <== mon.1 v2:172.21.15.203:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fac48031640 con 0x7fac4c143c20 2023-08-09T14:52:20.685 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:20.683+0000 7fac50cdc640 1 -- 172.21.15.43:0/526434704 <== mon.1 v2:172.21.15.203:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fac480317a0 con 0x7fac4c143c20 2023-08-09T14:52:20.685 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:20.683+0000 7fac50cdc640 1 -- 172.21.15.43:0/526434704 <== mon.1 v2:172.21.15.203:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fac48031b30 con 0x7fac4c143c20 2023-08-09T14:52:20.685 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:20.683+0000 7fac53f42640 1 -- 172.21.15.43:0/526434704 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fac4c143c20 msgr2=0x7fac4c06f540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T14:52:20.685 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:20.683+0000 7fac53f42640 1 --2- 172.21.15.43:0/526434704 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fac4c143c20 0x7fac4c06f540 secure :-1 s=READY pgs=731 cs=0 l=1 rev1=1 crypto rx=0x7fac48009870 tx=0x7fac4802b8f0 comp rx=0 tx=0).stop 2023-08-09T14:52:20.685 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:20.683+0000 7fac53f42640 1 -- 172.21.15.43:0/526434704 shutdown_connections 2023-08-09T14:52:20.686 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:20.683+0000 7fac53f42640 1 --2- 172.21.15.43:0/526434704 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fac4c06cfe0 0x7fac4c0c8b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T14:52:20.686 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:20.683+0000 7fac53f42640 1 --2- 172.21.15.43:0/526434704 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fac4c143c20 0x7fac4c06f540 unknown :-1 s=CLOSED pgs=731 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T14:52:20.686 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:20.683+0000 7fac53f42640 1 -- 172.21.15.43:0/526434704 >> 172.21.15.43:0/526434704 conn(0x7fac4c0ce0a0 msgr2=0x7fac4c0ce480 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-09T14:52:20.686 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:20.683+0000 7fac53f42640 1 -- 172.21.15.43:0/526434704 shutdown_connections 2023-08-09T14:52:20.686 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:20.683+0000 7fac53f42640 1 -- 172.21.15.43:0/526434704 wait complete. 2023-08-09T14:52:20.687 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:20.686+0000 7fac53f42640 1 Processor -- start 2023-08-09T14:52:20.687 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:20.686+0000 7fac53f42640 1 -- start start 2023-08-09T14:52:20.688 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:20.687+0000 7fac53f42640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fac4c06cfe0 0x7fac4c15cba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T14:52:20.688 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:20.687+0000 7fac53f42640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fac4c0c86f0 0x7fac4c159bc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T14:52:20.688 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:20.687+0000 7fac53f42640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fac4c143c20 0x7fac4c1415c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T14:52:20.688 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:20.687+0000 7fac53f42640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fac4c0b1f50 con 0x7fac4c0c86f0 2023-08-09T14:52:20.688 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:20.687+0000 7fac53f42640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fac4c0b1340 con 0x7fac4c06cfe0 2023-08-09T14:52:20.688 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:20.687+0000 7fac53f42640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fac4c0b1620 con 0x7fac4c143c20 2023-08-09T14:52:20.689 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:20.687+0000 7fac514dd640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fac4c0c86f0 0x7fac4c159bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T14:52:20.689 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:20.688+0000 7fac514dd640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fac4c0c86f0 0x7fac4c159bc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:37932/0 (socket says 172.21.15.43:37932) 2023-08-09T14:52:20.689 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:20.688+0000 7fac514dd640 1 -- 172.21.15.43:0/1007667421 learned_addr learned my addr 172.21.15.43:0/1007667421 (peer_addr_for_me v2:172.21.15.43:0/0) 2023-08-09T14:52:20.690 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:20.688+0000 7fac51cde640 1 --2- 172.21.15.43:0/1007667421 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fac4c06cfe0 0x7fac4c15cba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T14:52:20.690 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:20.688+0000 7fac524df640 1 --2- 172.21.15.43:0/1007667421 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fac4c143c20 0x7fac4c1415c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T14:52:20.690 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:20.688+0000 7fac514dd640 1 -- 172.21.15.43:0/1007667421 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fac4c143c20 msgr2=0x7fac4c1415c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T14:52:20.690 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:20.688+0000 7fac514dd640 1 --2- 172.21.15.43:0/1007667421 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fac4c143c20 0x7fac4c1415c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T14:52:20.691 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:20.688+0000 7fac514dd640 1 -- 172.21.15.43:0/1007667421 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fac4c06cfe0 msgr2=0x7fac4c15cba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T14:52:20.691 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:20.688+0000 7fac514dd640 1 --2- 172.21.15.43:0/1007667421 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fac4c06cfe0 0x7fac4c15cba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T14:52:20.691 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:20.688+0000 7fac514dd640 1 -- 172.21.15.43:0/1007667421 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fac44009a70 con 0x7fac4c0c86f0 2023-08-09T14:52:20.692 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:20.688+0000 7fac514dd640 1 --2- 172.21.15.43:0/1007667421 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fac4c0c86f0 0x7fac4c159bc0 secure :-1 s=READY pgs=1987 cs=0 l=1 rev1=1 crypto rx=0x7fac3c00c3c0 tx=0x7fac3c00c880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-09T14:52:20.692 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:20.689+0000 7fac427fc640 1 -- 172.21.15.43:0/1007667421 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fac3c0052d0 con 0x7fac4c0c86f0 2023-08-09T14:52:20.692 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:20.689+0000 7fac427fc640 1 -- 172.21.15.43:0/1007667421 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fac3c005450 con 0x7fac4c0c86f0 2023-08-09T14:52:20.692 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:20.689+0000 7fac427fc640 1 -- 172.21.15.43:0/1007667421 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fac3c007260 con 0x7fac4c0c86f0 2023-08-09T14:52:20.692 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:20.689+0000 7fac53f42640 1 -- 172.21.15.43:0/1007667421 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fac480094d0 con 0x7fac4c0c86f0 2023-08-09T14:52:20.693 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:20.689+0000 7fac53f42640 1 -- 172.21.15.43:0/1007667421 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fac4c0b1be0 con 0x7fac4c0c86f0 2023-08-09T14:52:20.693 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:20.690+0000 7fac53f42640 1 -- 172.21.15.43:0/1007667421 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fac4c0b2100 con 0x7fac4c0c86f0 2023-08-09T14:52:20.694 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:20.691+0000 7fac427fc640 1 -- 172.21.15.43:0/1007667421 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 88) v1 ==== 88443+0+0 (secure 0 0 0) 0x7fac3c0073c0 con 0x7fac4c0c86f0 2023-08-09T14:52:20.694 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:20.692+0000 7fac427fc640 1 --2- 172.21.15.43:0/1007667421 >> [v2:172.21.15.203:6832/2959400181,v1:172.21.15.203:6833/2959400181] conn(0x7fac3006a170 0x7fac3006c630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T14:52:20.694 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:20.692+0000 7fac427fc640 1 -- 172.21.15.43:0/1007667421 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(177..177 src has 1..177) v4 ==== 6348+0+0 (secure 0 0 0) 0x7fac3c087c20 con 0x7fac4c0c86f0 2023-08-09T14:52:20.694 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:20.693+0000 7fac51cde640 1 --2- 172.21.15.43:0/1007667421 >> [v2:172.21.15.203:6832/2959400181,v1:172.21.15.203:6833/2959400181] conn(0x7fac3006a170 0x7fac3006c630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T14:52:20.695 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:20.694+0000 7fac51cde640 1 --2- 172.21.15.43:0/1007667421 >> [v2:172.21.15.203:6832/2959400181,v1:172.21.15.203:6833/2959400181] conn(0x7fac3006a170 0x7fac3006c630 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7fac48036000 tx=0x7fac4803c040 comp rx=0 tx=0).ready entity=mgr.7223 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-09T14:52:20.698 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:20.698+0000 7fac427fc640 1 -- 172.21.15.43:0/1007667421 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178066 (secure 0 0 0) 0x7fac4c0b2100 con 0x7fac4c0c86f0 2023-08-09T14:52:20.964 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:20.963+0000 7fac53f42640 1 -- 172.21.15.43:0/1007667421 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_snap_schedules.TestSnapSchedules.test_snap_schedule_with_mgr_restart"]} v 0) v1 -- 0x7fac4c0b2100 con 0x7fac4c0c86f0 2023-08-09T14:52:21.437 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:21.436+0000 7fac427fc640 1 -- 172.21.15.43:0/1007667421 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_snap_schedules.TestSnapSchedules.test_snap_schedule_with_mgr_restart"]}]=0 v2591) v1 ==== 168+0+0 (secure 0 0 0) 0x7fac4c0b2100 con 0x7fac4c0c86f0 2023-08-09T14:52:21.441 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:21.440+0000 7fac53f42640 1 -- 172.21.15.43:0/1007667421 >> [v2:172.21.15.203:6832/2959400181,v1:172.21.15.203:6833/2959400181] conn(0x7fac3006a170 msgr2=0x7fac3006c630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T14:52:21.441 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:21.440+0000 7fac53f42640 1 --2- 172.21.15.43:0/1007667421 >> [v2:172.21.15.203:6832/2959400181,v1:172.21.15.203:6833/2959400181] conn(0x7fac3006a170 0x7fac3006c630 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7fac48036000 tx=0x7fac4803c040 comp rx=0 tx=0).stop 2023-08-09T14:52:21.441 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:21.440+0000 7fac53f42640 1 -- 172.21.15.43:0/1007667421 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fac4c0c86f0 msgr2=0x7fac4c159bc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T14:52:21.441 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:21.440+0000 7fac53f42640 1 --2- 172.21.15.43:0/1007667421 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fac4c0c86f0 0x7fac4c159bc0 secure :-1 s=READY pgs=1987 cs=0 l=1 rev1=1 crypto rx=0x7fac3c00c3c0 tx=0x7fac3c00c880 comp rx=0 tx=0).stop 2023-08-09T14:52:21.442 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:21.440+0000 7fac53f42640 1 -- 172.21.15.43:0/1007667421 shutdown_connections 2023-08-09T14:52:21.442 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:21.440+0000 7fac53f42640 1 --2- 172.21.15.43:0/1007667421 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fac4c143c20 0x7fac4c1415c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T14:52:21.442 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:21.440+0000 7fac53f42640 1 --2- 172.21.15.43:0/1007667421 >> [v2:172.21.15.203:6832/2959400181,v1:172.21.15.203:6833/2959400181] conn(0x7fac3006a170 0x7fac3006c630 unknown :-1 s=CLOSED pgs=41 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T14:52:21.442 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:21.440+0000 7fac53f42640 1 --2- 172.21.15.43:0/1007667421 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fac4c0c86f0 0x7fac4c159bc0 unknown :-1 s=CLOSED pgs=1987 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T14:52:21.442 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:21.440+0000 7fac53f42640 1 --2- 172.21.15.43:0/1007667421 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fac4c06cfe0 0x7fac4c15cba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T14:52:21.442 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:21.440+0000 7fac53f42640 1 -- 172.21.15.43:0/1007667421 >> 172.21.15.43:0/1007667421 conn(0x7fac4c0ce0a0 msgr2=0x7fac4c0c4610 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-09T14:52:21.442 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:21.441+0000 7fac53f42640 1 -- 172.21.15.43:0/1007667421 shutdown_connections 2023-08-09T14:52:21.443 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:21.441+0000 7fac53f42640 1 -- 172.21.15.43:0/1007667421 wait complete. 2023-08-09T14:52:21.466 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-08-09T14:52:21.686 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:21.685+0000 7ff27e7bd640 1 Processor -- start 2023-08-09T14:52:21.686 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:21.686+0000 7ff27e7bd640 1 -- start start 2023-08-09T14:52:21.691 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:21.687+0000 7ff27e7bd640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7ff27806b190 0x7ff2780cbd90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T14:52:21.691 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:21.687+0000 7ff27e7bd640 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7ff2780cd160 0x7ff27806f5c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T14:52:21.692 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:21.687+0000 7ff27e7bd640 1 -- --> v1:172.21.15.43:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff27806c2b0 con 0x7ff2780ccce0 2023-08-09T14:52:21.692 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:21.687+0000 7ff27e7bd640 1 -- --> v2:172.21.15.203:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff278158eb0 con 0x7ff2780cd160 2023-08-09T14:52:21.692 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:21.687+0000 7ff27e7bd640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff27806b6a0 con 0x7ff27806b190 2023-08-09T14:52:21.692 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:21.688+0000 7ff27cfba640 1 --1- >> v1:172.21.15.43:6789/0 conn(0x7ff2780ccce0 0x7ff27806c740 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.43:6789/0 says I am v1:172.21.15.43:48444/0 (socket says 172.21.15.43:48444) 2023-08-09T14:52:21.693 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:21.688+0000 7ff27cfba640 1 -- 172.21.15.43:0/3529655399 learned_addr learned my addr 172.21.15.43:0/3529655399 (peer_addr_for_me v1:172.21.15.43:0/0) 2023-08-09T14:52:21.693 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:21.688+0000 7ff27d7bb640 1 --2- 172.21.15.43:0/3529655399 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7ff27806b190 0x7ff2780cbd90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T14:52:21.693 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:21.688+0000 7ff26ffff640 1 -- 172.21.15.43:0/3529655399 <== mon.0 v1:172.21.15.43:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 302503834 0 0) 0x7ff27806c2b0 con 0x7ff2780ccce0 2023-08-09T14:52:21.694 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:21.688+0000 7ff26ffff640 1 -- 172.21.15.43:0/3529655399 --> v1:172.21.15.43:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ff264003520 con 0x7ff2780ccce0 2023-08-09T14:52:21.694 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:21.688+0000 7ff27dfbc640 1 --2- 172.21.15.43:0/3529655399 >> v2:172.21.15.203:3300/0 conn(0x7ff2780cd160 0x7ff27806f5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T14:52:21.694 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:21.688+0000 7ff26ffff640 1 -- 172.21.15.43:0/3529655399 <== mon.0 v1:172.21.15.43:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 4250207798 0 0) 0x7ff264003520 con 0x7ff2780ccce0 2023-08-09T14:52:21.694 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:21.688+0000 7ff26ffff640 1 -- 172.21.15.43:0/3529655399 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7ff27806b190 msgr2=0x7ff2780cbd90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-09T14:52:21.695 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:21.688+0000 7ff26ffff640 1 --2- 172.21.15.43:0/3529655399 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7ff27806b190 0x7ff2780cbd90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T14:52:21.695 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:21.688+0000 7ff26ffff640 1 -- 172.21.15.43:0/3529655399 >> v2:172.21.15.203:3300/0 conn(0x7ff2780cd160 msgr2=0x7ff27806f5c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-09T14:52:21.695 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:21.688+0000 7ff26ffff640 1 --2- 172.21.15.43:0/3529655399 >> v2:172.21.15.203:3300/0 conn(0x7ff2780cd160 0x7ff27806f5c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T14:52:21.696 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:21.689+0000 7ff26ffff640 1 -- 172.21.15.43:0/3529655399 --> v1:172.21.15.43:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff27806b980 con 0x7ff2780ccce0 2023-08-09T14:52:21.696 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:21.689+0000 7ff26ffff640 1 -- 172.21.15.43:0/3529655399 <== mon.0 v1:172.21.15.43:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1285150740 0 0) 0x7ff268001670 con 0x7ff2780ccce0 2023-08-09T14:52:21.696 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:21.689+0000 7ff26ffff640 1 -- 172.21.15.43:0/3529655399 >> v1:172.21.15.43:6789/0 conn(0x7ff2780ccce0 legacy=0x7ff27806c740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T14:52:21.696 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:21.689+0000 7ff26ffff640 1 --2- 172.21.15.43:0/3529655399 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7ff27806b190 0x7ff264003970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T14:52:21.697 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:21.689+0000 7ff26ffff640 1 --2- 172.21.15.43:0/3529655399 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff264003ef0 0x7ff2640047c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T14:52:21.697 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:21.689+0000 7ff26ffff640 1 --2- 172.21.15.43:0/3529655399 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7ff264004d00 0x7ff264005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T14:52:21.697 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:21.689+0000 7ff26ffff640 1 -- 172.21.15.43:0/3529655399 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff278158eb0 con 0x7ff264003ef0 2023-08-09T14:52:21.698 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:21.689+0000 7ff26ffff640 1 -- 172.21.15.43:0/3529655399 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff27806b6a0 con 0x7ff264004d00 2023-08-09T14:52:21.698 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:21.689+0000 7ff26ffff640 1 -- 172.21.15.43:0/3529655399 --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff26400bb10 con 0x7ff27806b190 2023-08-09T14:52:21.698 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:21.689+0000 7ff27cfba640 1 --2- 172.21.15.43:0/3529655399 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff264003ef0 0x7ff2640047c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T14:52:21.698 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:21.689+0000 7ff27d7bb640 1 --2- 172.21.15.43:0/3529655399 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7ff27806b190 0x7ff264003970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T14:52:21.699 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:21.689+0000 7ff27dfbc640 1 --2- 172.21.15.43:0/3529655399 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7ff264004d00 0x7ff264005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T14:52:21.699 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:21.689+0000 7ff27cfba640 1 -- 172.21.15.43:0/3529655399 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7ff27806b190 msgr2=0x7ff264003970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-09T14:52:21.699 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:21.689+0000 7ff27cfba640 1 --2- 172.21.15.43:0/3529655399 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7ff27806b190 0x7ff264003970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T14:52:21.699 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:21.689+0000 7ff27cfba640 1 -- 172.21.15.43:0/3529655399 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7ff264004d00 msgr2=0x7ff264005180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-09T14:52:21.700 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:21.689+0000 7ff27cfba640 1 --2- 172.21.15.43:0/3529655399 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7ff264004d00 0x7ff264005180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T14:52:21.700 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:21.689+0000 7ff27cfba640 1 -- 172.21.15.43:0/3529655399 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff264003520 con 0x7ff264003ef0 2023-08-09T14:52:21.700 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:21.691+0000 7ff27cfba640 1 --2- 172.21.15.43:0/3529655399 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff264003ef0 0x7ff2640047c0 secure :-1 s=READY pgs=1989 cs=0 l=1 rev1=1 crypto rx=0x7ff26800c240 tx=0x7ff2640057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=869bfedabc83d047 server_cookie=0 in_seq=0 out_seq=0 2023-08-09T14:52:21.706 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:21.691+0000 7ff26ffff640 1 -- 172.21.15.43:0/3529655399 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff26802cb00 con 0x7ff264003ef0 2023-08-09T14:52:21.706 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:21.691+0000 7ff26ffff640 1 -- 172.21.15.43:0/3529655399 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff26802cc60 con 0x7ff264003ef0 2023-08-09T14:52:21.706 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:21.691+0000 7ff26ffff640 1 -- 172.21.15.43:0/3529655399 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff268005af0 con 0x7ff264003ef0 2023-08-09T14:52:21.707 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:21.691+0000 7ff27e7bd640 1 -- 172.21.15.43:0/3529655399 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff264003ef0 msgr2=0x7ff2640047c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T14:52:21.707 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:21.691+0000 7ff27e7bd640 1 --2- 172.21.15.43:0/3529655399 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff264003ef0 0x7ff2640047c0 secure :-1 s=READY pgs=1989 cs=0 l=1 rev1=1 crypto rx=0x7ff26800c240 tx=0x7ff2640057f0 comp rx=0 tx=0).stop 2023-08-09T14:52:21.707 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:21.691+0000 7ff27dfbc640 1 -- 172.21.15.43:0/3529655399 reap_dead start 2023-08-09T14:52:21.707 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:21.691+0000 7ff27e7bd640 1 -- 172.21.15.43:0/3529655399 shutdown_connections 2023-08-09T14:52:21.708 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:21.691+0000 7ff27e7bd640 1 -- 172.21.15.43:0/3529655399 >> 172.21.15.43:0/3529655399 conn(0x7ff2780cde90 msgr2=0x7ff2780ce2a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-09T14:52:21.708 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:21.692+0000 7ff27e7bd640 1 -- 172.21.15.43:0/3529655399 shutdown_connections 2023-08-09T14:52:21.708 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:21.692+0000 7ff27e7bd640 1 -- 172.21.15.43:0/3529655399 wait complete. 2023-08-09T14:52:21.709 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:21.693+0000 7ff27e7bd640 1 Processor -- start 2023-08-09T14:52:21.709 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:21.694+0000 7ff27e7bd640 1 -- start start 2023-08-09T14:52:21.709 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:21.694+0000 7ff27e7bd640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff264003ef0 0x7ff2780ace80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T14:52:21.709 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:21.695+0000 7ff27e7bd640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7ff27815bc70 0x7ff2780abfb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T14:52:21.709 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:21.695+0000 7ff27e7bd640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7ff2780ac4f0 0x7ff278142e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T14:52:21.709 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:21.695+0000 7ff27e7bd640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff2780c4f60 con 0x7ff264003ef0 2023-08-09T14:52:21.710 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:21.695+0000 7ff27e7bd640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff2780c5ae0 con 0x7ff2780ac4f0 2023-08-09T14:52:21.710 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:21.695+0000 7ff27e7bd640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff2780c51b0 con 0x7ff27815bc70 2023-08-09T14:52:21.710 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:21.695+0000 7ff27d7bb640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff264003ef0 0x7ff2780ace80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T14:52:21.710 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:21.695+0000 7ff27d7bb640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff264003ef0 0x7ff2780ace80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:37946/0 (socket says 172.21.15.43:37946) 2023-08-09T14:52:21.710 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:21.695+0000 7ff27d7bb640 1 -- 172.21.15.43:0/1741843174 learned_addr learned my addr 172.21.15.43:0/1741843174 (peer_addr_for_me v2:172.21.15.43:0/0) 2023-08-09T14:52:21.710 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:21.695+0000 7ff27d7bb640 1 -- 172.21.15.43:0/1741843174 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7ff27815bc70 msgr2=0x7ff2780abfb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T14:52:21.710 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:21.695+0000 7ff27d7bb640 1 --2- 172.21.15.43:0/1741843174 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7ff27815bc70 0x7ff2780abfb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T14:52:21.711 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:21.695+0000 7ff27d7bb640 1 -- 172.21.15.43:0/1741843174 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7ff2780ac4f0 msgr2=0x7ff278142e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T14:52:21.711 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:21.695+0000 7ff27dfbc640 1 --2- 172.21.15.43:0/1741843174 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7ff2780ac4f0 0x7ff278142e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T14:52:21.711 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:21.695+0000 7ff27d7bb640 1 --2- 172.21.15.43:0/1741843174 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7ff2780ac4f0 0x7ff278142e70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T14:52:21.711 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:21.695+0000 7ff27d7bb640 1 -- 172.21.15.43:0/1741843174 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff2780c5490 con 0x7ff264003ef0 2023-08-09T14:52:21.712 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:21.696+0000 7ff27d7bb640 1 --2- 172.21.15.43:0/1741843174 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff264003ef0 0x7ff2780ace80 secure :-1 s=READY pgs=1990 cs=0 l=1 rev1=1 crypto rx=0x7ff274007f60 tx=0x7ff274007f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-09T14:52:21.712 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:21.696+0000 7ff26dffb640 1 -- 172.21.15.43:0/1741843174 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff27400e030 con 0x7ff264003ef0 2023-08-09T14:52:21.712 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:21.696+0000 7ff26dffb640 1 -- 172.21.15.43:0/1741843174 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff274016470 con 0x7ff264003ef0 2023-08-09T14:52:21.712 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:21.696+0000 7ff26dffb640 1 -- 172.21.15.43:0/1741843174 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff274016820 con 0x7ff264003ef0 2023-08-09T14:52:21.712 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:21.696+0000 7ff27e7bd640 1 -- 172.21.15.43:0/1741843174 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff2780c5770 con 0x7ff264003ef0 2023-08-09T14:52:21.713 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:21.696+0000 7ff27e7bd640 1 -- 172.21.15.43:0/1741843174 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff2781433b0 con 0x7ff264003ef0 2023-08-09T14:52:21.713 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:21.697+0000 7ff27e7bd640 1 -- 172.21.15.43:0/1741843174 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff2780739f0 con 0x7ff264003ef0 2023-08-09T14:52:21.713 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:21.698+0000 7ff26dffb640 1 -- 172.21.15.43:0/1741843174 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 88) v1 ==== 88443+0+0 (secure 0 0 0) 0x7ff274016980 con 0x7ff264003ef0 2023-08-09T14:52:21.713 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:21.699+0000 7ff26dffb640 1 --2- 172.21.15.43:0/1741843174 >> [v2:172.21.15.203:6832/2959400181,v1:172.21.15.203:6833/2959400181] conn(0x7ff25c06a170 0x7ff25c06c630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T14:52:21.713 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:21.699+0000 7ff26dffb640 1 -- 172.21.15.43:0/1741843174 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(177..177 src has 1..177) v4 ==== 6348+0+0 (secure 0 0 0) 0x7ff27400f070 con 0x7ff264003ef0 2023-08-09T14:52:21.713 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:21.700+0000 7ff27cfba640 1 --2- 172.21.15.43:0/1741843174 >> [v2:172.21.15.203:6832/2959400181,v1:172.21.15.203:6833/2959400181] conn(0x7ff25c06a170 0x7ff25c06c630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T14:52:21.713 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:21.701+0000 7ff27cfba640 1 --2- 172.21.15.43:0/1741843174 >> [v2:172.21.15.203:6832/2959400181,v1:172.21.15.203:6833/2959400181] conn(0x7ff25c06a170 0x7ff25c06c630 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7ff268034000 tx=0x7ff26802e000 comp rx=0 tx=0).ready entity=mgr.7223 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-09T14:52:21.713 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:21.703+0000 7ff26dffb640 1 -- 172.21.15.43:0/1741843174 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178066 (secure 0 0 0) 0x7ff274054b80 con 0x7ff264003ef0 2023-08-09T14:52:21.900 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:21.899+0000 7ff27e7bd640 1 -- 172.21.15.43:0/1741843174 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7ff27806bf10 con 0x7ff264003ef0 2023-08-09T14:52:21.902 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:21.902+0000 7ff26dffb640 1 -- 172.21.15.43:0/1741843174 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v177) v1 ==== 74+0+16000 (secure 0 0 0) 0x7ff2740581a0 con 0x7ff264003ef0 2023-08-09T14:52:21.903 INFO:teuthology.orchestra.run.smithi043.stdout: 2023-08-09T14:52:21.903 INFO:teuthology.orchestra.run.smithi043.stdout:{"epoch":177,"fsid":"daac87ee-1337-4250-a955-aea4e408a39b","created":"2023-08-09T13:33:41.701198+0000","modified":"2023-08-09T14:52:18.409659+0000","last_up_change":"2023-08-09T13:33:53.783453+0000","last_in_change":"2023-08-09T13:33:45.148904+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":21,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":"rbd","create_time":"2023-08-09T13:33:54.365137+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":".mgr","create_time":"2023-08-09T13:33:54.831803+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"cbd49ecd-e896-4fbd-864e-5c37a829b165","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":170,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6800","nonce":502667335},{"type":"v1","addr":"172.21.15.43:6801","nonce":502667335}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6802","nonce":502667335},{"type":"v1","addr":"172.21.15.43:6803","nonce":502667335}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6806","nonce":502667335},{"type":"v1","addr":"172.21.15.43:6807","nonce":502667335}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6804","nonce":502667335},{"type":"v1","addr":"172.21.15.43:6805","nonce":502667335}]},"public_addr":"172.21.15.43:6801/502667335","cluster_addr":"172.21.15.43:6803/502667335","heartbeat_back_addr":"172.21.15.43:6807/502667335","heartbeat_front_addr":"172.21.15.43:6805/502667335","state":["exists","up"]},{"osd":1,"uuid":"75c7e0fe-1ec7-4fe8-a2f2-3661a65b8500","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":170,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6808","nonce":3289134610},{"type":"v1","addr":"172.21.15.43:6809","nonce":3289134610}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6810","nonce":3289134610},{"type":"v1","addr":"172.21.15.43:6812","nonce":3289134610}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6821","nonce":3289134610},{"type":"v1","addr":"172.21.15.43:6824","nonce":3289134610}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6815","nonce":3289134610},{"type":"v1","addr":"172.21.15.43:6818","nonce":3289134610}]},"public_addr":"172.21.15.43:6809/3289134610","cluster_addr":"172.21.15.43:6812/3289134610","heartbeat_back_addr":"172.21.15.43:6824/3289134610","heartbeat_front_addr":"172.21.15.43:6818/3289134610","state":["exists","up"]},{"osd":2,"uuid":"bb5c41e9-7f5f-4125-9b01-3bb8bfc2ea83","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":170,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6811","nonce":4079709736},{"type":"v1","addr":"172.21.15.43:6814","nonce":4079709736}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6817","nonce":4079709736},{"type":"v1","addr":"172.21.15.43:6820","nonce":4079709736}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6829","nonce":4079709736},{"type":"v1","addr":"172.21.15.43:6831","nonce":4079709736}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6823","nonce":4079709736},{"type":"v1","addr":"172.21.15.43:6826","nonce":4079709736}]},"public_addr":"172.21.15.43:6814/4079709736","cluster_addr":"172.21.15.43:6820/4079709736","heartbeat_back_addr":"172.21.15.43:6831/4079709736","heartbeat_front_addr":"172.21.15.43:6826/4079709736","state":["exists","up"]},{"osd":3,"uuid":"bfe4c8b4-694a-465c-87e6-2b2aab852613","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":170,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6813","nonce":1709538759},{"type":"v1","addr":"172.21.15.43:6816","nonce":1709538759}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6819","nonce":1709538759},{"type":"v1","addr":"172.21.15.43:6822","nonce":1709538759}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6828","nonce":1709538759},{"type":"v1","addr":"172.21.15.43:6830","nonce":1709538759}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6825","nonce":1709538759},{"type":"v1","addr":"172.21.15.43:6827","nonce":1709538759}]},"public_addr":"172.21.15.43:6816/1709538759","cluster_addr":"172.21.15.43:6822/1709538759","heartbeat_back_addr":"172.21.15.43:6830/1709538759","heartbeat_front_addr":"172.21.15.43:6827/1709538759","state":["exists","up"]},{"osd":4,"uuid":"fa2e8a77-aba1-4756-853e-0d174a5bdeba","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":170,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6800","nonce":4196866481},{"type":"v1","addr":"172.21.15.203:6801","nonce":4196866481}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6802","nonce":4196866481},{"type":"v1","addr":"172.21.15.203:6803","nonce":4196866481}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6806","nonce":4196866481},{"type":"v1","addr":"172.21.15.203:6807","nonce":4196866481}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6804","nonce":4196866481},{"type":"v1","addr":"172.21.15.203:6805","nonce":4196866481}]},"public_addr":"172.21.15.203:6801/4196866481","cluster_addr":"172.21.15.203:6803/4196866481","heartbeat_back_addr":"172.21.15.203:6807/4196866481","heartbeat_front_addr":"172.21.15.203:6805/4196866481","state":["exists","up"]},{"osd":5,"uuid":"81811e4b-4d24-4f51-a82c-f1d700a2a00d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":170,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6816","nonce":3059290081},{"type":"v1","addr":"172.21.15.203:6817","nonce":3059290081}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6818","nonce":3059290081},{"type":"v1","addr":"172.21.15.203:6819","nonce":3059290081}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6822","nonce":3059290081},{"type":"v1","addr":"172.21.15.203:6823","nonce":3059290081}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6820","nonce":3059290081},{"type":"v1","addr":"172.21.15.203:6821","nonce":3059290081}]},"public_addr":"172.21.15.203:6817/3059290081","cluster_addr":"172.21.15.203:6819/3059290081","heartbeat_back_addr":"172.21.15.203:6823/3059290081","heartbeat_front_addr":"172.21.15.203:6821/3059290081","state":["exists","up"]},{"osd":6,"uuid":"307bac4d-6a5b-443d-af75-56d695d7674f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":170,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6824","nonce":512155569},{"type":"v1","addr":"172.21.15.203:6825","nonce":512155569}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6826","nonce":512155569},{"type":"v1","addr":"172.21.15.203:6827","nonce":512155569}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6830","nonce":512155569},{"type":"v1","addr":"172.21.15.203:6831","nonce":512155569}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6828","nonce":512155569},{"type":"v1","addr":"172.21.15.203:6829","nonce":512155569}]},"public_addr":"172.21.15.203:6825/512155569","cluster_addr":"172.21.15.203:6827/512155569","heartbeat_back_addr":"172.21.15.203:6831/512155569","heartbeat_front_addr":"172.21.15.203:6829/512155569","state":["exists","up"]},{"osd":7,"uuid":"a95dc7ca-9f1b-4295-af70-68a22a2d7d5e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":170,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6808","nonce":3331450438},{"type":"v1","addr":"172.21.15.203:6809","nonce":3331450438}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6810","nonce":3331450438},{"type":"v1","addr":"172.21.15.203:6811","nonce":3331450438}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6814","nonce":3331450438},{"type":"v1","addr":"172.21.15.203:6815","nonce":3331450438}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6812","nonce":3331450438},{"type":"v1","addr":"172.21.15.203:6813","nonce":3331450438}]},"public_addr":"172.21.15.203:6809/3331450438","cluster_addr":"172.21.15.203:6811/3331450438","heartbeat_back_addr":"172.21.15.203:6815/3331450438","heartbeat_front_addr":"172.21.15.203:6813/3331450438","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-09T13:33:50.547652+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-09T13:33:50.589469+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-09T13:33:50.561724+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-09T13:33:50.599650+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-09T13:33:50.411855+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-09T13:33:50.219681+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-09T13:33:50.504664+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-09T13:33:50.441248+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.43:6835/2435237803":"2023-08-10T14:52:15.389877+0000","172.21.15.43:6834/2435237803":"2023-08-10T14:52:15.389877+0000","172.21.15.203:0/3301934490":"2023-08-10T14:52:10.059559+0000","172.21.15.203:0/1715525748":"2023-08-10T14:52:10.059559+0000","172.21.15.203:0/1477488908":"2023-08-10T14:52:10.059559+0000","172.21.15.203:6833/843920641":"2023-08-10T14:52:10.059559+0000","172.21.15.203:6832/843920641":"2023-08-10T14:52:10.059559+0000","172.21.15.203:0/2095212004":"2023-08-10T14:52:02.909250+0000","172.21.15.203:0/1196801979":"2023-08-10T14:52:10.059559+0000","172.21.15.203:0/3749872091":"2023-08-10T14:52:02.909250+0000","172.21.15.203:0/1305485142":"2023-08-10T14:52:10.059559+0000","172.21.15.203:0/651612122":"2023-08-10T14:52:02.909250+0000","172.21.15.203:0/4120954828":"2023-08-10T14:52:02.909250+0000","172.21.15.203:0/3552079285":"2023-08-10T14:52:10.059559+0000","172.21.15.203:0/2783782601":"2023-08-10T14:52:02.909250+0000","172.21.15.203:0/2232659577":"2023-08-10T14:52:02.909250+0000","172.21.15.203:6833/2566816332":"2023-08-10T14:52:02.909250+0000","172.21.15.203:6832/2566816332":"2023-08-10T14:52:02.909250+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":7,"snaps":[{"begin":104,"length":1},{"begin":221,"length":1}]}],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-08-09T14:52:21.905 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:21.905+0000 7ff27e7bd640 1 -- 172.21.15.43:0/1741843174 >> [v2:172.21.15.203:6832/2959400181,v1:172.21.15.203:6833/2959400181] conn(0x7ff25c06a170 msgr2=0x7ff25c06c630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T14:52:21.906 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:21.905+0000 7ff27e7bd640 1 --2- 172.21.15.43:0/1741843174 >> [v2:172.21.15.203:6832/2959400181,v1:172.21.15.203:6833/2959400181] conn(0x7ff25c06a170 0x7ff25c06c630 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7ff268034000 tx=0x7ff26802e000 comp rx=0 tx=0).stop 2023-08-09T14:52:21.906 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:21.905+0000 7ff27e7bd640 1 -- 172.21.15.43:0/1741843174 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff264003ef0 msgr2=0x7ff2780ace80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T14:52:21.906 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:21.905+0000 7ff27e7bd640 1 --2- 172.21.15.43:0/1741843174 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff264003ef0 0x7ff2780ace80 secure :-1 s=READY pgs=1990 cs=0 l=1 rev1=1 crypto rx=0x7ff274007f60 tx=0x7ff274007f90 comp rx=0 tx=0).stop 2023-08-09T14:52:21.906 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:21.905+0000 7ff27e7bd640 1 -- 172.21.15.43:0/1741843174 shutdown_connections 2023-08-09T14:52:21.906 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:21.905+0000 7ff27e7bd640 1 --2- 172.21.15.43:0/1741843174 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7ff2780ac4f0 0x7ff278142e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T14:52:21.906 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:21.905+0000 7ff27e7bd640 1 --2- 172.21.15.43:0/1741843174 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7ff27815bc70 0x7ff2780abfb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T14:52:21.906 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:21.905+0000 7ff27e7bd640 1 --2- 172.21.15.43:0/1741843174 >> [v2:172.21.15.203:6832/2959400181,v1:172.21.15.203:6833/2959400181] conn(0x7ff25c06a170 0x7ff25c06c630 unknown :-1 s=CLOSED pgs=42 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T14:52:21.907 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:21.905+0000 7ff27e7bd640 1 --2- 172.21.15.43:0/1741843174 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff264003ef0 0x7ff2780ace80 unknown :-1 s=CLOSED pgs=1990 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T14:52:21.907 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:21.905+0000 7ff27e7bd640 1 -- 172.21.15.43:0/1741843174 >> 172.21.15.43:0/1741843174 conn(0x7ff2780cde90 msgr2=0x7ff27814f2d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-09T14:52:21.907 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:21.905+0000 7ff27e7bd640 1 -- 172.21.15.43:0/1741843174 shutdown_connections 2023-08-09T14:52:21.907 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:21.905+0000 7ff27e7bd640 1 -- 172.21.15.43:0/1741843174 wait complete. 2023-08-09T14:52:21.925 INFO:tasks.cephfs.cephfs_test_case.ceph_manager:[{'pool': 1, 'pool_name': 'rbd', 'create_time': '2023-08-09T13:33:54.365137+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}, 'read_balance': {'score_acting': 2, 'score_stable': 2, 'optimal_score': 1, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': '.mgr', 'create_time': '2023-08-09T13:33:54.831803+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-08-09T14:52:21.925 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get rbd pg_num 2023-08-09T14:52:22.160 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:22.159+0000 7f7219225640 1 Processor -- start 2023-08-09T14:52:22.160 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:22.160+0000 7f7219225640 1 -- start start 2023-08-09T14:52:22.161 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:22.161+0000 7f7219225640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f721415c2d0 0x7f72140b7df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T14:52:22.162 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:22.161+0000 7f7219225640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f72140cebf0 0x7f72140cf080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T14:52:22.162 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:22.161+0000 7f7219225640 1 -- --> v1:172.21.15.43:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f72140d16a0 con 0x7f72140d1b30 2023-08-09T14:52:22.162 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:22.161+0000 7f7219225640 1 -- --> v2:172.21.15.43:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f72140cf5c0 con 0x7f72140cebf0 2023-08-09T14:52:22.162 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:22.161+0000 7f7219225640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f72140d1f70 con 0x7f721415c2d0 2023-08-09T14:52:22.162 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:22.161+0000 7f7218a24640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f72140cebf0 0x7f72140cf080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T14:52:22.162 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:22.161+0000 7f7218a24640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f72140cebf0 0x7f72140cf080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:37950/0 (socket says 172.21.15.43:37950) 2023-08-09T14:52:22.162 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:22.161+0000 7f7218a24640 1 -- 172.21.15.43:0/1741194240 learned_addr learned my addr 172.21.15.43:0/1741194240 (peer_addr_for_me v2:172.21.15.43:0/0) 2023-08-09T14:52:22.163 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:22.161+0000 7f72137fe640 1 --1- 172.21.15.43:0/1741194240 >> v1:172.21.15.43:6789/0 conn(0x7f72140d1b30 0x7f72140cd020 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.43:6789/0 says I am v1:172.21.15.43:48456/0 (socket says 172.21.15.43:48456) 2023-08-09T14:52:22.164 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:22.162+0000 7f7213fff640 1 --2- 172.21.15.43:0/1741194240 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f721415c2d0 0x7f72140b7df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T14:52:22.164 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:22.162+0000 7f7218a24640 1 -- 172.21.15.43:0/1741194240 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f721415c2d0 msgr2=0x7f72140b7df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-09T14:52:22.164 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:22.162+0000 7f7218a24640 1 --2- 172.21.15.43:0/1741194240 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f721415c2d0 0x7f72140b7df0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T14:52:22.164 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:22.162+0000 7f7218a24640 1 -- 172.21.15.43:0/1741194240 >> v1:172.21.15.43:6789/0 conn(0x7f72140d1b30 legacy=0x7f72140cd020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T14:52:22.164 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:22.162+0000 7f7218a24640 1 -- 172.21.15.43:0/1741194240 --> v2:172.21.15.43:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f72140d0d70 con 0x7f72140cebf0 2023-08-09T14:52:22.164 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:22.162+0000 7f7212ffd640 1 -- 172.21.15.43:0/1741194240 <== mon.0 v1:172.21.15.43:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3634558165 0 0) 0x7f72140d16a0 con 0x7f72140d1b30 2023-08-09T14:52:22.165 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:22.162+0000 7f7213fff640 1 --2- 172.21.15.43:0/1741194240 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f721415c2d0 0x7f72140b7df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-08-09T14:52:22.165 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:22.163+0000 7f7218a24640 1 --2- 172.21.15.43:0/1741194240 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f72140cebf0 0x7f72140cf080 secure :-1 s=READY pgs=1992 cs=0 l=1 rev1=1 crypto rx=0x7f720c009870 tx=0x7f720c02b980 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2d3b0154c4bd519e server_cookie=0 in_seq=0 out_seq=0 2023-08-09T14:52:22.165 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:22.163+0000 7f7212ffd640 1 -- 172.21.15.43:0/1741194240 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f720c030840 con 0x7f72140cebf0 2023-08-09T14:52:22.165 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:22.163+0000 7f7212ffd640 1 -- 172.21.15.43:0/1741194240 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f720c0309a0 con 0x7f72140cebf0 2023-08-09T14:52:22.165 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:22.163+0000 7f7212ffd640 1 -- 172.21.15.43:0/1741194240 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f720c030d30 con 0x7f72140cebf0 2023-08-09T14:52:22.166 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:22.164+0000 7f7219225640 1 -- 172.21.15.43:0/1741194240 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f72140cebf0 msgr2=0x7f72140cf080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T14:52:22.166 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:22.164+0000 7f7219225640 1 --2- 172.21.15.43:0/1741194240 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f72140cebf0 0x7f72140cf080 secure :-1 s=READY pgs=1992 cs=0 l=1 rev1=1 crypto rx=0x7f720c009870 tx=0x7f720c02b980 comp rx=0 tx=0).stop 2023-08-09T14:52:22.166 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:22.164+0000 7f7219225640 1 -- 172.21.15.43:0/1741194240 shutdown_connections 2023-08-09T14:52:22.166 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:22.164+0000 7f7219225640 1 --2- 172.21.15.43:0/1741194240 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f72140cebf0 0x7f72140cf080 unknown :-1 s=CLOSED pgs=1992 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T14:52:22.167 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:22.164+0000 7f7219225640 1 --2- 172.21.15.43:0/1741194240 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f721415c2d0 0x7f72140b7df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T14:52:22.167 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:22.164+0000 7f7219225640 1 -- 172.21.15.43:0/1741194240 >> 172.21.15.43:0/1741194240 conn(0x7f721406a230 msgr2=0x7f721406a640 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-09T14:52:22.167 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:22.164+0000 7f7219225640 1 -- 172.21.15.43:0/1741194240 shutdown_connections 2023-08-09T14:52:22.167 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:22.164+0000 7f7219225640 1 -- 172.21.15.43:0/1741194240 wait complete. 2023-08-09T14:52:22.167 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:22.166+0000 7f7219225640 1 Processor -- start 2023-08-09T14:52:22.168 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:22.166+0000 7f7219225640 1 -- start start 2023-08-09T14:52:22.169 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:22.167+0000 7f7219225640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f72140cebf0 0x7f721415f130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T14:52:22.170 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:22.167+0000 7f7219225640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f72140d1b30 0x7f72141f5030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T14:52:22.170 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:22.167+0000 7f7219225640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f721415c2d0 0x7f72141f2290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T14:52:22.170 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:22.167+0000 7f7219225640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f72141f5570 con 0x7f72140cebf0 2023-08-09T14:52:22.170 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:22.167+0000 7f7219225640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f72141f5810 con 0x7f721415c2d0 2023-08-09T14:52:22.170 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:22.167+0000 7f7219225640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f72141f5af0 con 0x7f72140d1b30 2023-08-09T14:52:22.170 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:22.167+0000 7f7213fff640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f72140cebf0 0x7f721415f130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T14:52:22.171 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:22.167+0000 7f7213fff640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f72140cebf0 0x7f721415f130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:37954/0 (socket says 172.21.15.43:37954) 2023-08-09T14:52:22.171 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:22.167+0000 7f7213fff640 1 -- 172.21.15.43:0/3071113703 learned_addr learned my addr 172.21.15.43:0/3071113703 (peer_addr_for_me v2:172.21.15.43:0/0) 2023-08-09T14:52:22.171 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:22.168+0000 7f7213fff640 1 -- 172.21.15.43:0/3071113703 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f72140d1b30 msgr2=0x7f72141f5030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T14:52:22.171 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:22.168+0000 7f7213fff640 1 --2- 172.21.15.43:0/3071113703 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f72140d1b30 0x7f72141f5030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T14:52:22.172 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:22.168+0000 7f7213fff640 1 -- 172.21.15.43:0/3071113703 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f721415c2d0 msgr2=0x7f72141f2290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T14:52:22.172 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:22.168+0000 7f7218a24640 1 --2- 172.21.15.43:0/3071113703 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f721415c2d0 0x7f72141f2290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T14:52:22.172 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:22.168+0000 7f7213fff640 1 --2- 172.21.15.43:0/3071113703 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f721415c2d0 0x7f72141f2290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T14:52:22.173 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:22.168+0000 7f7213fff640 1 -- 172.21.15.43:0/3071113703 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f720c0094d0 con 0x7f72140cebf0 2023-08-09T14:52:22.173 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:22.168+0000 7f7213fff640 1 --2- 172.21.15.43:0/3071113703 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f72140cebf0 0x7f721415f130 secure :-1 s=READY pgs=1993 cs=0 l=1 rev1=1 crypto rx=0x7f720400cf60 tx=0x7f720400cf90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-09T14:52:22.173 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:22.168+0000 7f7210ff9640 1 -- 172.21.15.43:0/3071113703 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f720400d840 con 0x7f72140cebf0 2023-08-09T14:52:22.173 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:22.168+0000 7f7210ff9640 1 -- 172.21.15.43:0/3071113703 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f720400d9a0 con 0x7f72140cebf0 2023-08-09T14:52:22.174 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:22.168+0000 7f7210ff9640 1 -- 172.21.15.43:0/3071113703 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f720400dd30 con 0x7f72140cebf0 2023-08-09T14:52:22.174 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:22.168+0000 7f7219225640 1 -- 172.21.15.43:0/3071113703 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f72141f5dd0 con 0x7f72140cebf0 2023-08-09T14:52:22.174 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:22.168+0000 7f7219225640 1 -- 172.21.15.43:0/3071113703 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f72141f27d0 con 0x7f72140cebf0 2023-08-09T14:52:22.176 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:22.169+0000 7f7219225640 1 -- 172.21.15.43:0/3071113703 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f72141f29c0 con 0x7f72140cebf0 2023-08-09T14:52:22.176 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:22.170+0000 7f7210ff9640 1 -- 172.21.15.43:0/3071113703 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 88) v1 ==== 88443+0+0 (secure 0 0 0) 0x7f7204015440 con 0x7f72140cebf0 2023-08-09T14:52:22.176 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:22.171+0000 7f7210ff9640 1 --2- 172.21.15.43:0/3071113703 >> [v2:172.21.15.203:6832/2959400181,v1:172.21.15.203:6833/2959400181] conn(0x7f71f406a170 0x7f71f406c630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T14:52:22.176 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:22.171+0000 7f7210ff9640 1 -- 172.21.15.43:0/3071113703 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(177..177 src has 1..177) v4 ==== 6348+0+0 (secure 0 0 0) 0x7f7204089240 con 0x7f72140cebf0 2023-08-09T14:52:22.176 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:22.172+0000 7f72137fe640 1 --2- 172.21.15.43:0/3071113703 >> [v2:172.21.15.203:6832/2959400181,v1:172.21.15.203:6833/2959400181] conn(0x7f71f406a170 0x7f71f406c630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T14:52:22.177 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:22.173+0000 7f72137fe640 1 --2- 172.21.15.43:0/3071113703 >> [v2:172.21.15.203:6832/2959400181,v1:172.21.15.203:6833/2959400181] conn(0x7f71f406a170 0x7f71f406c630 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7f71fc007fd0 tx=0x7f71fc007960 comp rx=0 tx=0).ready entity=mgr.7223 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-09T14:52:22.177 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:22.176+0000 7f7210ff9640 1 -- 172.21.15.43:0/3071113703 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178066 (secure 0 0 0) 0x7f7204054710 con 0x7f72140cebf0 2023-08-09T14:52:22.372 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:22.372+0000 7f7219225640 1 -- 172.21.15.43:0/3071113703 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"} v 0) v1 -- 0x7f72141f1fb0 con 0x7f72140cebf0 2023-08-09T14:52:22.373 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:22.373+0000 7f7210ff9640 1 -- 172.21.15.43:0/3071113703 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"}]=0 v177) v1 ==== 92+0+10 (secure 0 0 0) 0x7f720401a0a0 con 0x7f72140cebf0 2023-08-09T14:52:22.374 INFO:teuthology.orchestra.run.smithi043.stdout:pg_num: 8 2023-08-09T14:52:22.375 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:22.375+0000 7f7219225640 1 -- 172.21.15.43:0/3071113703 >> [v2:172.21.15.203:6832/2959400181,v1:172.21.15.203:6833/2959400181] conn(0x7f71f406a170 msgr2=0x7f71f406c630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T14:52:22.376 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:22.375+0000 7f7219225640 1 --2- 172.21.15.43:0/3071113703 >> [v2:172.21.15.203:6832/2959400181,v1:172.21.15.203:6833/2959400181] conn(0x7f71f406a170 0x7f71f406c630 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7f71fc007fd0 tx=0x7f71fc007960 comp rx=0 tx=0).stop 2023-08-09T14:52:22.376 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:22.375+0000 7f7219225640 1 -- 172.21.15.43:0/3071113703 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f72140cebf0 msgr2=0x7f721415f130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T14:52:22.376 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:22.375+0000 7f7219225640 1 --2- 172.21.15.43:0/3071113703 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f72140cebf0 0x7f721415f130 secure :-1 s=READY pgs=1993 cs=0 l=1 rev1=1 crypto rx=0x7f720400cf60 tx=0x7f720400cf90 comp rx=0 tx=0).stop 2023-08-09T14:52:22.377 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:22.376+0000 7f7219225640 1 -- 172.21.15.43:0/3071113703 shutdown_connections 2023-08-09T14:52:22.377 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:22.376+0000 7f7219225640 1 --2- 172.21.15.43:0/3071113703 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f721415c2d0 0x7f72141f2290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T14:52:22.377 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:22.376+0000 7f7219225640 1 --2- 172.21.15.43:0/3071113703 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f72140d1b30 0x7f72141f5030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T14:52:22.377 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:22.376+0000 7f7219225640 1 --2- 172.21.15.43:0/3071113703 >> [v2:172.21.15.203:6832/2959400181,v1:172.21.15.203:6833/2959400181] conn(0x7f71f406a170 0x7f71f406c630 unknown :-1 s=CLOSED pgs=43 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T14:52:22.377 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:22.376+0000 7f7219225640 1 --2- 172.21.15.43:0/3071113703 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f72140cebf0 0x7f721415f130 unknown :-1 s=CLOSED pgs=1993 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T14:52:22.378 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:22.376+0000 7f7219225640 1 -- 172.21.15.43:0/3071113703 >> 172.21.15.43:0/3071113703 conn(0x7f721406a230 msgr2=0x7f72140cccc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-09T14:52:22.378 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:22.376+0000 7f7219225640 1 -- 172.21.15.43:0/3071113703 shutdown_connections 2023-08-09T14:52:22.378 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:22.376+0000 7f7219225640 1 -- 172.21.15.43:0/3071113703 wait complete. 2023-08-09T14:52:22.397 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-08-09T14:52:22.630 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:22.629+0000 7f868f59e640 1 Processor -- start 2023-08-09T14:52:22.630 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:22.630+0000 7f868f59e640 1 -- start start 2023-08-09T14:52:22.632 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:22.631+0000 7f868f59e640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f8690157ea0 0x7f86900c97d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T14:52:22.632 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:22.631+0000 7f868f59e640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f869015af10 0x7f86900c9d10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T14:52:22.632 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:22.631+0000 7f868f59e640 1 -- --> v1:172.21.15.43:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8690158280 con 0x7f86900c90a0 2023-08-09T14:52:22.632 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:22.631+0000 7f868f59e640 1 -- --> v2:172.21.15.43:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f869015b840 con 0x7f8690157ea0 2023-08-09T14:52:22.632 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:22.631+0000 7f868f59e640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f86900b3810 con 0x7f869015af10 2023-08-09T14:52:22.633 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:22.632+0000 7f868e59c640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f8690157ea0 0x7f86900c97d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T14:52:22.633 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:22.632+0000 7f8687fff640 1 --1- >> v1:172.21.15.43:6789/0 conn(0x7f86900c90a0 0x7f869015a800 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.43:6789/0 says I am v1:172.21.15.43:48462/0 (socket says 172.21.15.43:48462) 2023-08-09T14:52:22.633 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:22.632+0000 7f8687fff640 1 -- 172.21.15.43:0/1824367325 learned_addr learned my addr 172.21.15.43:0/1824367325 (peer_addr_for_me v1:172.21.15.43:0/0) 2023-08-09T14:52:22.634 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:22.632+0000 7f868ed9d640 1 --2- 172.21.15.43:0/1824367325 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f869015af10 0x7f86900c9d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T14:52:22.634 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:22.632+0000 7f868e59c640 1 -- 172.21.15.43:0/1824367325 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f869015af10 msgr2=0x7f86900c9d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-09T14:52:22.634 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:22.632+0000 7f868e59c640 1 --2- 172.21.15.43:0/1824367325 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f869015af10 0x7f86900c9d10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T14:52:22.634 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:22.632+0000 7f868e59c640 1 -- 172.21.15.43:0/1824367325 >> v1:172.21.15.43:6789/0 conn(0x7f86900c90a0 legacy=0x7f869015a800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T14:52:22.634 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:22.632+0000 7f868e59c640 1 -- 172.21.15.43:0/1824367325 --> v2:172.21.15.43:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f869015b560 con 0x7f8690157ea0 2023-08-09T14:52:22.634 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:22.632+0000 7f868ed9d640 1 --2- 172.21.15.43:0/1824367325 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f869015af10 0x7f86900c9d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-08-09T14:52:22.635 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:22.634+0000 7f868e59c640 1 --2- 172.21.15.43:0/1824367325 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f8690157ea0 0x7f86900c97d0 secure :-1 s=READY pgs=1995 cs=0 l=1 rev1=1 crypto rx=0x7f86800097d0 tx=0x7f868002b820 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2deaf76a8287b6af server_cookie=0 in_seq=0 out_seq=0 2023-08-09T14:52:22.635 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:22.634+0000 7f868dd9b640 1 -- 172.21.15.43:0/1824367325 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8680031780 con 0x7f8690157ea0 2023-08-09T14:52:22.635 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:22.634+0000 7f868dd9b640 1 -- 172.21.15.43:0/1824367325 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f86800318e0 con 0x7f8690157ea0 2023-08-09T14:52:22.635 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:22.634+0000 7f868dd9b640 1 -- 172.21.15.43:0/1824367325 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8680031c70 con 0x7f8690157ea0 2023-08-09T14:52:22.636 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:22.635+0000 7f868f59e640 1 -- 172.21.15.43:0/1824367325 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f8690157ea0 msgr2=0x7f86900c97d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T14:52:22.636 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:22.635+0000 7f868f59e640 1 --2- 172.21.15.43:0/1824367325 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f8690157ea0 0x7f86900c97d0 secure :-1 s=READY pgs=1995 cs=0 l=1 rev1=1 crypto rx=0x7f86800097d0 tx=0x7f868002b820 comp rx=0 tx=0).stop 2023-08-09T14:52:22.636 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:22.635+0000 7f868f59e640 1 -- 172.21.15.43:0/1824367325 shutdown_connections 2023-08-09T14:52:22.636 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:22.635+0000 7f868f59e640 1 --2- 172.21.15.43:0/1824367325 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f869015af10 0x7f86900c9d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T14:52:22.637 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:22.635+0000 7f868f59e640 1 --2- 172.21.15.43:0/1824367325 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f8690157ea0 0x7f86900c97d0 unknown :-1 s=CLOSED pgs=1995 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T14:52:22.637 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:22.635+0000 7f868f59e640 1 -- 172.21.15.43:0/1824367325 >> 172.21.15.43:0/1824367325 conn(0x7f86900cddd0 msgr2=0x7f86900ce1e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-09T14:52:22.637 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:22.635+0000 7f868f59e640 1 -- 172.21.15.43:0/1824367325 shutdown_connections 2023-08-09T14:52:22.637 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:22.636+0000 7f868f59e640 1 -- 172.21.15.43:0/1824367325 wait complete. 2023-08-09T14:52:22.638 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:22.638+0000 7f868f59e640 1 Processor -- start 2023-08-09T14:52:22.638 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:22.638+0000 7f868f59e640 1 -- start start 2023-08-09T14:52:22.639 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:22.639+0000 7f868f59e640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f86900c90a0 0x7f8690157350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T14:52:22.640 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:22.639+0000 7f868f59e640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f8690157ea0 0x7f86900ae3a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T14:52:22.640 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:22.639+0000 7f868f59e640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f869015af10 0x7f86900b1a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T14:52:22.640 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:22.639+0000 7f868f59e640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f86900c6360 con 0x7f86900c90a0 2023-08-09T14:52:22.640 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:22.639+0000 7f868f59e640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f86901f4ea0 con 0x7f8690157ea0 2023-08-09T14:52:22.641 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:22.639+0000 7f868f59e640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f86901f5180 con 0x7f869015af10 2023-08-09T14:52:22.642 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:22.639+0000 7f868e59c640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f86900c90a0 0x7f8690157350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T14:52:22.642 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:22.640+0000 7f868e59c640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f86900c90a0 0x7f8690157350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:37970/0 (socket says 172.21.15.43:37970) 2023-08-09T14:52:22.642 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:22.640+0000 7f868e59c640 1 -- 172.21.15.43:0/1782929802 learned_addr learned my addr 172.21.15.43:0/1782929802 (peer_addr_for_me v2:172.21.15.43:0/0) 2023-08-09T14:52:22.642 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:22.640+0000 7f868ed9d640 1 --2- 172.21.15.43:0/1782929802 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f869015af10 0x7f86900b1a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T14:52:22.642 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:22.640+0000 7f868e59c640 1 -- 172.21.15.43:0/1782929802 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f869015af10 msgr2=0x7f86900b1a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T14:52:22.642 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:22.640+0000 7f8687fff640 1 --2- 172.21.15.43:0/1782929802 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f8690157ea0 0x7f86900ae3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T14:52:22.643 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:22.640+0000 7f868e59c640 1 --2- 172.21.15.43:0/1782929802 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f869015af10 0x7f86900b1a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T14:52:22.643 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:22.640+0000 7f868e59c640 1 -- 172.21.15.43:0/1782929802 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f8690157ea0 msgr2=0x7f86900ae3a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T14:52:22.643 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:22.640+0000 7f868e59c640 1 --2- 172.21.15.43:0/1782929802 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f8690157ea0 0x7f86900ae3a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T14:52:22.643 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:22.640+0000 7f868e59c640 1 -- 172.21.15.43:0/1782929802 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8680009400 con 0x7f86900c90a0 2023-08-09T14:52:22.643 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:22.640+0000 7f8687fff640 1 --2- 172.21.15.43:0/1782929802 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f8690157ea0 0x7f86900ae3a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-08-09T14:52:22.643 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:22.640+0000 7f868ed9d640 1 --2- 172.21.15.43:0/1782929802 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f869015af10 0x7f86900b1a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-08-09T14:52:22.643 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:22.641+0000 7f868e59c640 1 --2- 172.21.15.43:0/1782929802 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f86900c90a0 0x7f8690157350 secure :-1 s=READY pgs=1996 cs=0 l=1 rev1=1 crypto rx=0x7f86800097d0 tx=0x7f86800046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-09T14:52:22.644 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:22.641+0000 7f8686ffd640 1 -- 172.21.15.43:0/1782929802 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8680032960 con 0x7f86900c90a0 2023-08-09T14:52:22.644 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:22.641+0000 7f8686ffd640 1 -- 172.21.15.43:0/1782929802 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8680032ae0 con 0x7f86900c90a0 2023-08-09T14:52:22.644 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:22.641+0000 7f8686ffd640 1 -- 172.21.15.43:0/1782929802 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8680032e90 con 0x7f86900c90a0 2023-08-09T14:52:22.644 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:22.641+0000 7f868f59e640 1 -- 172.21.15.43:0/1782929802 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f86901f5460 con 0x7f86900c90a0 2023-08-09T14:52:22.644 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:22.641+0000 7f868f59e640 1 -- 172.21.15.43:0/1782929802 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f86901f5a20 con 0x7f86900c90a0 2023-08-09T14:52:22.645 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:22.642+0000 7f868f59e640 1 -- 172.21.15.43:0/1782929802 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f86901f5bb0 con 0x7f86900c90a0 2023-08-09T14:52:22.645 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:22.643+0000 7f8686ffd640 1 -- 172.21.15.43:0/1782929802 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 88) v1 ==== 88443+0+0 (secure 0 0 0) 0x7f868004e440 con 0x7f86900c90a0 2023-08-09T14:52:22.645 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:22.644+0000 7f8686ffd640 1 --2- 172.21.15.43:0/1782929802 >> [v2:172.21.15.203:6832/2959400181,v1:172.21.15.203:6833/2959400181] conn(0x7f867006a1a0 0x7f867006c660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T14:52:22.645 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:22.644+0000 7f8686ffd640 1 -- 172.21.15.43:0/1782929802 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(177..177 src has 1..177) v4 ==== 6348+0+0 (secure 0 0 0) 0x7f86800aac80 con 0x7f86900c90a0 2023-08-09T14:52:22.646 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:22.645+0000 7f8687fff640 1 --2- 172.21.15.43:0/1782929802 >> [v2:172.21.15.203:6832/2959400181,v1:172.21.15.203:6833/2959400181] conn(0x7f867006a1a0 0x7f867006c660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T14:52:22.647 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:22.646+0000 7f8687fff640 1 --2- 172.21.15.43:0/1782929802 >> [v2:172.21.15.203:6832/2959400181,v1:172.21.15.203:6833/2959400181] conn(0x7f867006a1a0 0x7f867006c660 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7f8678007fd0 tx=0x7f8678009040 comp rx=0 tx=0).ready entity=mgr.7223 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-09T14:52:22.650 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:22.650+0000 7f8686ffd640 1 -- 172.21.15.43:0/1782929802 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178066 (secure 0 0 0) 0x7f8680075f90 con 0x7f86900c90a0 2023-08-09T14:52:22.846 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:22.846+0000 7f868f59e640 1 -- 172.21.15.43:0/1782929802 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f869004e390 con 0x7f86900c90a0 2023-08-09T14:52:22.847 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:22.847+0000 7f8686ffd640 1 -- 172.21.15.43:0/1782929802 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v177) v1 ==== 93+0+10 (secure 0 0 0) 0x7f86800795b0 con 0x7f86900c90a0 2023-08-09T14:52:22.848 INFO:teuthology.orchestra.run.smithi043.stdout:pg_num: 1 2023-08-09T14:52:22.850 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:22.849+0000 7f868f59e640 1 -- 172.21.15.43:0/1782929802 >> [v2:172.21.15.203:6832/2959400181,v1:172.21.15.203:6833/2959400181] conn(0x7f867006a1a0 msgr2=0x7f867006c660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T14:52:22.850 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:22.849+0000 7f868f59e640 1 --2- 172.21.15.43:0/1782929802 >> [v2:172.21.15.203:6832/2959400181,v1:172.21.15.203:6833/2959400181] conn(0x7f867006a1a0 0x7f867006c660 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7f8678007fd0 tx=0x7f8678009040 comp rx=0 tx=0).stop 2023-08-09T14:52:22.850 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:22.849+0000 7f868f59e640 1 -- 172.21.15.43:0/1782929802 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f86900c90a0 msgr2=0x7f8690157350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T14:52:22.851 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:22.849+0000 7f868f59e640 1 --2- 172.21.15.43:0/1782929802 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f86900c90a0 0x7f8690157350 secure :-1 s=READY pgs=1996 cs=0 l=1 rev1=1 crypto rx=0x7f86800097d0 tx=0x7f86800046c0 comp rx=0 tx=0).stop 2023-08-09T14:52:22.851 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:22.849+0000 7f868f59e640 1 -- 172.21.15.43:0/1782929802 shutdown_connections 2023-08-09T14:52:22.851 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:22.849+0000 7f868f59e640 1 --2- 172.21.15.43:0/1782929802 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f869015af10 0x7f86900b1a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T14:52:22.851 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:22.849+0000 7f868f59e640 1 --2- 172.21.15.43:0/1782929802 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f8690157ea0 0x7f86900ae3a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T14:52:22.851 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:22.849+0000 7f868f59e640 1 --2- 172.21.15.43:0/1782929802 >> [v2:172.21.15.203:6832/2959400181,v1:172.21.15.203:6833/2959400181] conn(0x7f867006a1a0 0x7f867006c660 unknown :-1 s=CLOSED pgs=44 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T14:52:22.851 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:22.849+0000 7f868f59e640 1 --2- 172.21.15.43:0/1782929802 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f86900c90a0 0x7f8690157350 unknown :-1 s=CLOSED pgs=1996 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T14:52:22.851 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:22.850+0000 7f868f59e640 1 -- 172.21.15.43:0/1782929802 >> 172.21.15.43:0/1782929802 conn(0x7f86900cddd0 msgr2=0x7f869015bea0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-09T14:52:22.852 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:22.850+0000 7f868f59e640 1 -- 172.21.15.43:0/1782929802 shutdown_connections 2023-08-09T14:52:22.852 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:22.850+0000 7f868f59e640 1 -- 172.21.15.43:0/1782929802 wait complete. 2023-08-09T14:52:22.868 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mon mon_allow_pool_delete True 2023-08-09T14:52:23.099 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:23.099+0000 7f0cd7fff640 1 Processor -- start 2023-08-09T14:52:23.100 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:23.099+0000 7f0cd7fff640 1 -- start start 2023-08-09T14:52:23.101 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:23.101+0000 7f0cd7fff640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f0cd815c3a0 0x7f0cd81608a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T14:52:23.101 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:23.101+0000 7f0cd7fff640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f0cd80b0f00 0x7f0cd80ccef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T14:52:23.102 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:23.101+0000 7f0cd7fff640 1 -- --> v1:172.21.15.203:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0cd8160de0 con 0x7f0cd80b1450 2023-08-09T14:52:23.102 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:23.101+0000 7f0cd7fff640 1 -- --> v2:172.21.15.43:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0cd80b81c0 con 0x7f0cd815c3a0 2023-08-09T14:52:23.102 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:23.101+0000 7f0cd7fff640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0cd80b7c00 con 0x7f0cd80b0f00 2023-08-09T14:52:23.102 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:23.101+0000 7f0cd6ffd640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f0cd815c3a0 0x7f0cd81608a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T14:52:23.103 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:23.101+0000 7f0cd6ffd640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f0cd815c3a0 0x7f0cd81608a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:37982/0 (socket says 172.21.15.43:37982) 2023-08-09T14:52:23.103 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:23.101+0000 7f0cd6ffd640 1 -- 172.21.15.43:0/2176956333 learned_addr learned my addr 172.21.15.43:0/2176956333 (peer_addr_for_me v2:172.21.15.43:0/0) 2023-08-09T14:52:23.103 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:23.102+0000 7f0cd67fc640 1 --2- 172.21.15.43:0/2176956333 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f0cd80b0f00 0x7f0cd80ccef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T14:52:23.103 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:23.102+0000 7f0cd6ffd640 1 -- 172.21.15.43:0/2176956333 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f0cd80b0f00 msgr2=0x7f0cd80ccef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-09T14:52:23.103 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:23.102+0000 7f0cd6ffd640 1 --2- 172.21.15.43:0/2176956333 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f0cd80b0f00 0x7f0cd80ccef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T14:52:23.103 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:23.102+0000 7f0cd6ffd640 1 -- 172.21.15.43:0/2176956333 >> v1:172.21.15.203:6789/0 conn(0x7f0cd80b1450 legacy=0x7f0cd80d0580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-09T14:52:23.104 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:23.102+0000 7f0cd6ffd640 1 -- 172.21.15.43:0/2176956333 --> v2:172.21.15.43:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0cd80b7ee0 con 0x7f0cd815c3a0 2023-08-09T14:52:23.104 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:23.103+0000 7f0cd6ffd640 1 --2- 172.21.15.43:0/2176956333 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f0cd815c3a0 0x7f0cd81608a0 secure :-1 s=READY pgs=1997 cs=0 l=1 rev1=1 crypto rx=0x7f0cc8009870 tx=0x7f0cc802b860 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1dfe4c26cc7a58b1 server_cookie=0 in_seq=0 out_seq=0 2023-08-09T14:52:23.105 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:23.103+0000 7f0cd5ffb640 1 -- 172.21.15.43:0/2176956333 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0cc80317c0 con 0x7f0cd815c3a0 2023-08-09T14:52:23.105 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:23.103+0000 7f0cd5ffb640 1 -- 172.21.15.43:0/2176956333 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0cc8031920 con 0x7f0cd815c3a0 2023-08-09T14:52:23.105 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:23.104+0000 7f0cd5ffb640 1 -- 172.21.15.43:0/2176956333 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0cc8031cb0 con 0x7f0cd815c3a0 2023-08-09T14:52:23.105 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:23.104+0000 7f0cd7fff640 1 -- 172.21.15.43:0/2176956333 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f0cd815c3a0 msgr2=0x7f0cd81608a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T14:52:23.105 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:23.104+0000 7f0cd7fff640 1 --2- 172.21.15.43:0/2176956333 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f0cd815c3a0 0x7f0cd81608a0 secure :-1 s=READY pgs=1997 cs=0 l=1 rev1=1 crypto rx=0x7f0cc8009870 tx=0x7f0cc802b860 comp rx=0 tx=0).stop 2023-08-09T14:52:23.105 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:23.104+0000 7f0cd7fff640 1 -- 172.21.15.43:0/2176956333 shutdown_connections 2023-08-09T14:52:23.106 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:23.104+0000 7f0cd7fff640 1 --2- 172.21.15.43:0/2176956333 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f0cd80b0f00 0x7f0cd80ccef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T14:52:23.106 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:23.104+0000 7f0cd7fff640 1 --2- 172.21.15.43:0/2176956333 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f0cd815c3a0 0x7f0cd81608a0 unknown :-1 s=CLOSED pgs=1997 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T14:52:23.106 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:23.104+0000 7f0cd7fff640 1 -- 172.21.15.43:0/2176956333 >> 172.21.15.43:0/2176956333 conn(0x7f0cd806a210 msgr2=0x7f0cd806a620 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-09T14:52:23.106 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:23.104+0000 7f0cd7fff640 1 -- 172.21.15.43:0/2176956333 shutdown_connections 2023-08-09T14:52:23.106 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:23.104+0000 7f0cd7fff640 1 -- 172.21.15.43:0/2176956333 wait complete. 2023-08-09T14:52:23.106 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:23.106+0000 7f0cd7fff640 1 Processor -- start 2023-08-09T14:52:23.107 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:23.106+0000 7f0cd7fff640 1 -- start start 2023-08-09T14:52:23.107 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:23.107+0000 7f0cd7fff640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f0cd80b0f00 0x7f0cd8157630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T14:52:23.108 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:23.107+0000 7f0cd7fff640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f0cd80b1450 0x7f0cd80cace0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T14:52:23.108 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:23.107+0000 7f0cd7fff640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f0cd815c3a0 0x7f0cd80b5cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T14:52:23.108 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:23.107+0000 7f0cd7fff640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0cd80b1a10 con 0x7f0cd80b0f00 2023-08-09T14:52:23.108 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:23.107+0000 7f0cd7fff640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0cd815c820 con 0x7f0cd815c3a0 2023-08-09T14:52:23.109 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:23.107+0000 7f0cd7fff640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0cd80cb220 con 0x7f0cd80b1450 2023-08-09T14:52:23.110 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:23.107+0000 7f0cd6ffd640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f0cd80b0f00 0x7f0cd8157630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T14:52:23.110 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:23.108+0000 7f0cd6ffd640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f0cd80b0f00 0x7f0cd8157630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:37998/0 (socket says 172.21.15.43:37998) 2023-08-09T14:52:23.110 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:23.108+0000 7f0cd6ffd640 1 -- 172.21.15.43:0/3961956636 learned_addr learned my addr 172.21.15.43:0/3961956636 (peer_addr_for_me v2:172.21.15.43:0/0) 2023-08-09T14:52:23.110 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:23.108+0000 7f0cd67fc640 1 --2- 172.21.15.43:0/3961956636 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f0cd80b1450 0x7f0cd80cace0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T14:52:23.110 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:23.108+0000 7f0cd6ffd640 1 -- 172.21.15.43:0/3961956636 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f0cd80b1450 msgr2=0x7f0cd80cace0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T14:52:23.111 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:23.108+0000 7f0cd6ffd640 1 --2- 172.21.15.43:0/3961956636 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f0cd80b1450 0x7f0cd80cace0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T14:52:23.111 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:23.108+0000 7f0cd6ffd640 1 -- 172.21.15.43:0/3961956636 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f0cd815c3a0 msgr2=0x7f0cd80b5cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T14:52:23.111 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:23.108+0000 7f0cd6ffd640 1 --2- 172.21.15.43:0/3961956636 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f0cd815c3a0 0x7f0cd80b5cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T14:52:23.111 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:23.108+0000 7f0cd6ffd640 1 -- 172.21.15.43:0/3961956636 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0cc80094d0 con 0x7f0cd80b0f00 2023-08-09T14:52:23.111 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:23.108+0000 7f0cd6ffd640 1 --2- 172.21.15.43:0/3961956636 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f0cd80b0f00 0x7f0cd8157630 secure :-1 s=READY pgs=1998 cs=0 l=1 rev1=1 crypto rx=0x7f0cc8000c00 tx=0x7f0cc80046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-09T14:52:23.111 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:23.109+0000 7f0ccf7fe640 1 -- 172.21.15.43:0/3961956636 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0cc80329a0 con 0x7f0cd80b0f00 2023-08-09T14:52:23.111 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:23.109+0000 7f0ccf7fe640 1 -- 172.21.15.43:0/3961956636 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0cc8032b20 con 0x7f0cd80b0f00 2023-08-09T14:52:23.111 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:23.109+0000 7f0ccf7fe640 1 -- 172.21.15.43:0/3961956636 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0cc80453f0 con 0x7f0cd80b0f00 2023-08-09T14:52:23.112 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:23.109+0000 7f0cd7fff640 1 -- 172.21.15.43:0/3961956636 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0cd80d10f0 con 0x7f0cd80b0f00 2023-08-09T14:52:23.112 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:23.109+0000 7f0cd7fff640 1 -- 172.21.15.43:0/3961956636 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0cd815cac0 con 0x7f0cd80b0f00 2023-08-09T14:52:23.112 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:23.110+0000 7f0cd7fff640 1 -- 172.21.15.43:0/3961956636 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0cd804e390 con 0x7f0cd80b0f00 2023-08-09T14:52:23.113 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:23.111+0000 7f0ccf7fe640 1 -- 172.21.15.43:0/3961956636 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 88) v1 ==== 88443+0+0 (secure 0 0 0) 0x7f0cc804d3f0 con 0x7f0cd80b0f00 2023-08-09T14:52:23.113 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:23.112+0000 7f0ccf7fe640 1 --2- 172.21.15.43:0/3961956636 >> [v2:172.21.15.203:6832/2959400181,v1:172.21.15.203:6833/2959400181] conn(0x7f0cb806a170 0x7f0cb806c630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T14:52:23.113 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:23.112+0000 7f0ccf7fe640 1 -- 172.21.15.43:0/3961956636 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(177..177 src has 1..177) v4 ==== 6348+0+0 (secure 0 0 0) 0x7f0cc80aa880 con 0x7f0cd80b0f00 2023-08-09T14:52:23.113 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:23.113+0000 7f0cd67fc640 1 --2- 172.21.15.43:0/3961956636 >> [v2:172.21.15.203:6832/2959400181,v1:172.21.15.203:6833/2959400181] conn(0x7f0cb806a170 0x7f0cb806c630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T14:52:23.114 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:23.114+0000 7f0cd67fc640 1 --2- 172.21.15.43:0/3961956636 >> [v2:172.21.15.203:6832/2959400181,v1:172.21.15.203:6833/2959400181] conn(0x7f0cb806a170 0x7f0cb806c630 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7f0cc00070b0 tx=0x7f0cc0007040 comp rx=0 tx=0).ready entity=mgr.7223 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-09T14:52:23.116 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:23.116+0000 7f0ccf7fe640 1 -- 172.21.15.43:0/3961956636 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178066 (secure 0 0 0) 0x7f0cc8075cb0 con 0x7f0cd80b0f00 2023-08-09T14:52:23.311 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:23.310+0000 7f0cd7fff640 1 -- 172.21.15.43:0/3961956636 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command([{prefix=config set, name=mon_allow_pool_delete}] v 0) v1 -- 0x7f0cd80cfe50 con 0x7f0cd80b0f00 2023-08-09T14:52:23.320 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:23.320+0000 7f0ccf7fe640 1 -- 172.21.15.43:0/3961956636 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_allow_pool_delete}]=0 v58)=0 v58) v1 ==== 122+0+0 (secure 0 0 0) 0x7f0cc80792d0 con 0x7f0cd80b0f00 2023-08-09T14:52:23.324 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:23.323+0000 7f0cd7fff640 1 -- 172.21.15.43:0/3961956636 >> [v2:172.21.15.203:6832/2959400181,v1:172.21.15.203:6833/2959400181] conn(0x7f0cb806a170 msgr2=0x7f0cb806c630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T14:52:23.324 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:23.323+0000 7f0cd7fff640 1 --2- 172.21.15.43:0/3961956636 >> [v2:172.21.15.203:6832/2959400181,v1:172.21.15.203:6833/2959400181] conn(0x7f0cb806a170 0x7f0cb806c630 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7f0cc00070b0 tx=0x7f0cc0007040 comp rx=0 tx=0).stop 2023-08-09T14:52:23.324 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:23.323+0000 7f0cd7fff640 1 -- 172.21.15.43:0/3961956636 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f0cd80b0f00 msgr2=0x7f0cd8157630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T14:52:23.325 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:23.323+0000 7f0cd7fff640 1 --2- 172.21.15.43:0/3961956636 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f0cd80b0f00 0x7f0cd8157630 secure :-1 s=READY pgs=1998 cs=0 l=1 rev1=1 crypto rx=0x7f0cc8000c00 tx=0x7f0cc80046c0 comp rx=0 tx=0).stop 2023-08-09T14:52:23.325 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:23.323+0000 7f0cd7fff640 1 -- 172.21.15.43:0/3961956636 shutdown_connections 2023-08-09T14:52:23.325 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:23.323+0000 7f0cd7fff640 1 --2- 172.21.15.43:0/3961956636 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f0cd815c3a0 0x7f0cd80b5cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T14:52:23.325 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:23.323+0000 7f0cd7fff640 1 --2- 172.21.15.43:0/3961956636 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f0cd80b1450 0x7f0cd80cace0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T14:52:23.326 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:23.323+0000 7f0cd7fff640 1 --2- 172.21.15.43:0/3961956636 >> [v2:172.21.15.203:6832/2959400181,v1:172.21.15.203:6833/2959400181] conn(0x7f0cb806a170 0x7f0cb806c630 unknown :-1 s=CLOSED pgs=45 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T14:52:23.326 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:23.323+0000 7f0cd7fff640 1 --2- 172.21.15.43:0/3961956636 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f0cd80b0f00 0x7f0cd8157630 unknown :-1 s=CLOSED pgs=1998 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T14:52:23.326 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:23.323+0000 7f0cd7fff640 1 -- 172.21.15.43:0/3961956636 >> 172.21.15.43:0/3961956636 conn(0x7f0cd806a210 msgr2=0x7f0cd80b7190 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-09T14:52:23.326 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:23.323+0000 7f0cd7fff640 1 -- 172.21.15.43:0/3961956636 shutdown_connections 2023-08-09T14:52:23.326 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:23.323+0000 7f0cd7fff640 1 -- 172.21.15.43:0/3961956636 wait complete. 2023-08-09T14:52:23.344 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2023-08-09T14:52:23.345 DEBUG:teuthology.orchestra.run.smithi043:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2023-08-09T14:52:23.467 DEBUG:teuthology.orchestra.run.smithi203:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2023-08-09T14:52:23.598 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2023-08-09T14:52:23.599 DEBUG:teuthology.orchestra.run.smithi043:> set -ex 2023-08-09T14:52:23.599 DEBUG:teuthology.orchestra.run.smithi043:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-09T14:52:23.663 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4054441,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6498844k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=27,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3249420k,nr_inodes=812355,mode=700,uid=1000,gid=1275 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6498844k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2023-08-09T14:52:23.663 DEBUG:teuthology.orchestra.run.smithi203:> set -ex 2023-08-09T14:52:23.664 DEBUG:teuthology.orchestra.run.smithi203:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-09T14:52:23.725 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4053648,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497576k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=27,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248784k,nr_inodes=812196,mode=700,uid=1000,gid=1275 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497576k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2023-08-09T14:52:23.725 DEBUG:teuthology.orchestra.run.smithi043:> set -ex 2023-08-09T14:52:23.726 DEBUG:teuthology.orchestra.run.smithi043:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-09T14:52:23.789 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4054441,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6498844k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=27,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3249420k,nr_inodes=812355,mode=700,uid=1000,gid=1275 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6498844k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2023-08-09T14:52:23.789 DEBUG:teuthology.orchestra.run.smithi203:> set -ex 2023-08-09T14:52:23.790 DEBUG:teuthology.orchestra.run.smithi203:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-09T14:52:23.848 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4053648,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497576k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=27,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248784k,nr_inodes=812196,mode=700,uid=1000,gid=1275 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497576k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2023-08-09T14:52:23.848 INFO:tasks.cephfs.cephfs_test_case:[, , , ] 2023-08-09T14:52:23.849 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-08-09T14:52:24.087 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:24.087+0000 7fb7dd1c4640 1 Processor -- start 2023-08-09T14:52:24.088 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:24.087+0000 7fb7dd1c4640 1 -- start start 2023-08-09T14:52:24.089 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:24.088+0000 7fb7dd1c4640 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7fb7d815f7b0 0x7fb7d80b4760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T14:52:24.089 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:24.088+0000 7fb7dd1c4640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7fb7d80b1050 0x7fb7d80ccfa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T14:52:24.089 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:24.088+0000 7fb7dd1c4640 1 -- --> v1:172.21.15.203:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb7d80b7bf0 con 0x7fb7d80cd4e0 2023-08-09T14:52:24.090 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:24.088+0000 7fb7dd1c4640 1 -- --> v2:172.21.15.43:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb7d80cd970 con 0x7fb7d80b1050 2023-08-09T14:52:24.090 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:24.088+0000 7fb7dd1c4640 1 -- --> v2:172.21.15.203:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb7d80b7ed0 con 0x7fb7d815f7b0 2023-08-09T14:52:24.093 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:24.089+0000 7fb7cf7fe640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7fb7d80b1050 0x7fb7d80ccfa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T14:52:24.093 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:24.089+0000 7fb7cf7fe640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7fb7d80b1050 0x7fb7d80ccfa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:38010/0 (socket says 172.21.15.43:38010) 2023-08-09T14:52:24.094 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:24.089+0000 7fb7cf7fe640 1 -- 172.21.15.43:0/2311327384 learned_addr learned my addr 172.21.15.43:0/2311327384 (peer_addr_for_me v2:172.21.15.43:0/0) 2023-08-09T14:52:24.094 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:24.089+0000 7fb7d7fff640 1 --2- 172.21.15.43:0/2311327384 >> v2:172.21.15.203:3300/0 conn(0x7fb7d815f7b0 0x7fb7d80b4760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T14:52:24.094 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:24.089+0000 7fb7cf7fe640 1 -- 172.21.15.43:0/2311327384 >> v2:172.21.15.203:3300/0 conn(0x7fb7d815f7b0 msgr2=0x7fb7d80b4760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-09T14:52:24.094 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:24.089+0000 7fb7cf7fe640 1 --2- 172.21.15.43:0/2311327384 >> v2:172.21.15.203:3300/0 conn(0x7fb7d815f7b0 0x7fb7d80b4760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T14:52:24.095 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:24.089+0000 7fb7cf7fe640 1 -- 172.21.15.43:0/2311327384 >> v1:172.21.15.203:6789/0 conn(0x7fb7d80cd4e0 legacy=0x7fb7d80ceae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-09T14:52:24.095 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:24.089+0000 7fb7cf7fe640 1 -- 172.21.15.43:0/2311327384 --> v2:172.21.15.43:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb7d80d0370 con 0x7fb7d80b1050 2023-08-09T14:52:24.095 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:24.089+0000 7fb7d7fff640 1 --2- 172.21.15.43:0/2311327384 >> v2:172.21.15.203:3300/0 conn(0x7fb7d815f7b0 0x7fb7d80b4760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-08-09T14:52:24.095 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:24.091+0000 7fb7cf7fe640 1 --2- 172.21.15.43:0/2311327384 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb7d80b1050 0x7fb7d80ccfa0 secure :-1 s=READY pgs=1999 cs=0 l=1 rev1=1 crypto rx=0x7fb7d808aa40 tx=0x7fb7c002b910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a785dc7eec8972bd server_cookie=0 in_seq=0 out_seq=0 2023-08-09T14:52:24.095 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:24.091+0000 7fb7d77fe640 1 -- 172.21.15.43:0/2311327384 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb7c0031870 con 0x7fb7d80b1050 2023-08-09T14:52:24.095 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:24.091+0000 7fb7d77fe640 1 -- 172.21.15.43:0/2311327384 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb7c00319d0 con 0x7fb7d80b1050 2023-08-09T14:52:24.096 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:24.091+0000 7fb7d77fe640 1 -- 172.21.15.43:0/2311327384 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb7c0031d60 con 0x7fb7d80b1050 2023-08-09T14:52:24.096 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:24.092+0000 7fb7dd1c4640 1 -- 172.21.15.43:0/2311327384 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb7d80b1050 msgr2=0x7fb7d80ccfa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T14:52:24.096 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:24.092+0000 7fb7dd1c4640 1 --2- 172.21.15.43:0/2311327384 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb7d80b1050 0x7fb7d80ccfa0 secure :-1 s=READY pgs=1999 cs=0 l=1 rev1=1 crypto rx=0x7fb7d808aa40 tx=0x7fb7c002b910 comp rx=0 tx=0).stop 2023-08-09T14:52:24.096 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:24.092+0000 7fb7dd1c4640 1 -- 172.21.15.43:0/2311327384 shutdown_connections 2023-08-09T14:52:24.096 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:24.092+0000 7fb7dd1c4640 1 --2- 172.21.15.43:0/2311327384 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb7d80b1050 0x7fb7d80ccfa0 unknown :-1 s=CLOSED pgs=1999 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T14:52:24.096 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:24.092+0000 7fb7dd1c4640 1 --2- 172.21.15.43:0/2311327384 >> v2:172.21.15.203:3300/0 conn(0x7fb7d815f7b0 0x7fb7d80b4760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T14:52:24.096 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:24.092+0000 7fb7dd1c4640 1 -- 172.21.15.43:0/2311327384 >> 172.21.15.43:0/2311327384 conn(0x7fb7d806a2b0 msgr2=0x7fb7d806a690 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-09T14:52:24.097 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:24.092+0000 7fb7dd1c4640 1 -- 172.21.15.43:0/2311327384 shutdown_connections 2023-08-09T14:52:24.097 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:24.092+0000 7fb7dd1c4640 1 -- 172.21.15.43:0/2311327384 wait complete. 2023-08-09T14:52:24.099 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:24.094+0000 7fb7dd1c4640 1 Processor -- start 2023-08-09T14:52:24.099 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:24.094+0000 7fb7dd1c4640 1 -- start start 2023-08-09T14:52:24.099 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:24.095+0000 7fb7dd1c4640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb7d80b1050 0x7fb7d8155ec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T14:52:24.099 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:24.095+0000 7fb7dd1c4640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fb7d80cd4e0 0x7fb7d80a9670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T14:52:24.099 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:24.095+0000 7fb7dd1c4640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fb7d815f7b0 0x7fb7d80b2bb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T14:52:24.100 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:24.095+0000 7fb7dd1c4640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb7d81f5bc0 con 0x7fb7d80b1050 2023-08-09T14:52:24.100 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:24.095+0000 7fb7dd1c4640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb7d81f4cf0 con 0x7fb7d815f7b0 2023-08-09T14:52:24.100 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:24.095+0000 7fb7dd1c4640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb7d81f4f40 con 0x7fb7d80cd4e0 2023-08-09T14:52:24.100 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:24.095+0000 7fb7d7fff640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb7d80b1050 0x7fb7d8155ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T14:52:24.100 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:24.095+0000 7fb7d7fff640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb7d80b1050 0x7fb7d8155ec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:38012/0 (socket says 172.21.15.43:38012) 2023-08-09T14:52:24.100 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:24.095+0000 7fb7d7fff640 1 -- 172.21.15.43:0/2425518505 learned_addr learned my addr 172.21.15.43:0/2425518505 (peer_addr_for_me v2:172.21.15.43:0/0) 2023-08-09T14:52:24.100 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:24.096+0000 7fb7cf7fe640 1 --2- 172.21.15.43:0/2425518505 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fb7d80cd4e0 0x7fb7d80a9670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T14:52:24.101 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:24.096+0000 7fb7d7fff640 1 -- 172.21.15.43:0/2425518505 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fb7d80cd4e0 msgr2=0x7fb7d80a9670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T14:52:24.101 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:24.096+0000 7fb7dc9c3640 1 --2- 172.21.15.43:0/2425518505 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fb7d815f7b0 0x7fb7d80b2bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T14:52:24.101 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:24.096+0000 7fb7d7fff640 1 --2- 172.21.15.43:0/2425518505 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fb7d80cd4e0 0x7fb7d80a9670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T14:52:24.101 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:24.096+0000 7fb7d7fff640 1 -- 172.21.15.43:0/2425518505 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fb7d815f7b0 msgr2=0x7fb7d80b2bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T14:52:24.101 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:24.096+0000 7fb7d7fff640 1 --2- 172.21.15.43:0/2425518505 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fb7d815f7b0 0x7fb7d80b2bb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T14:52:24.101 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:24.096+0000 7fb7d7fff640 1 -- 172.21.15.43:0/2425518505 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb7c00094d0 con 0x7fb7d80b1050 2023-08-09T14:52:24.101 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:24.096+0000 7fb7dc9c3640 1 --2- 172.21.15.43:0/2425518505 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fb7d815f7b0 0x7fb7d80b2bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-08-09T14:52:24.101 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:24.096+0000 7fb7d7fff640 1 --2- 172.21.15.43:0/2425518505 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb7d80b1050 0x7fb7d8155ec0 secure :-1 s=READY pgs=2000 cs=0 l=1 rev1=1 crypto rx=0x7fb7c800c7f0 tx=0x7fb7c800ccb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-09T14:52:24.102 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:24.096+0000 7fb7d57fa640 1 -- 172.21.15.43:0/2425518505 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb7c8007820 con 0x7fb7d80b1050 2023-08-09T14:52:24.102 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:24.096+0000 7fb7d57fa640 1 -- 172.21.15.43:0/2425518505 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb7c8007980 con 0x7fb7d80b1050 2023-08-09T14:52:24.102 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:24.096+0000 7fb7d57fa640 1 -- 172.21.15.43:0/2425518505 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb7c8007cf0 con 0x7fb7d80b1050 2023-08-09T14:52:24.102 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:24.096+0000 7fb7dd1c4640 1 -- 172.21.15.43:0/2425518505 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb7d81f5220 con 0x7fb7d80b1050 2023-08-09T14:52:24.102 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:24.097+0000 7fb7dd1c4640 1 -- 172.21.15.43:0/2425518505 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb7d81f57e0 con 0x7fb7d80b1050 2023-08-09T14:52:24.102 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:24.098+0000 7fb7dd1c4640 1 -- 172.21.15.43:0/2425518505 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb7d80cb230 con 0x7fb7d80b1050 2023-08-09T14:52:24.103 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:24.099+0000 7fb7d57fa640 1 -- 172.21.15.43:0/2425518505 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 88) v1 ==== 88443+0+0 (secure 0 0 0) 0x7fb7c80113f0 con 0x7fb7d80b1050 2023-08-09T14:52:24.104 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:24.100+0000 7fb7d57fa640 1 --2- 172.21.15.43:0/2425518505 >> [v2:172.21.15.203:6832/2959400181,v1:172.21.15.203:6833/2959400181] conn(0x7fb7b806a170 0x7fb7b806c630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T14:52:24.104 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:24.100+0000 7fb7d57fa640 1 -- 172.21.15.43:0/2425518505 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(177..177 src has 1..177) v4 ==== 6348+0+0 (secure 0 0 0) 0x7fb7c8088ba0 con 0x7fb7d80b1050 2023-08-09T14:52:24.104 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:24.101+0000 7fb7cf7fe640 1 --2- 172.21.15.43:0/2425518505 >> [v2:172.21.15.203:6832/2959400181,v1:172.21.15.203:6833/2959400181] conn(0x7fb7b806a170 0x7fb7b806c630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T14:52:24.104 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:24.102+0000 7fb7cf7fe640 1 --2- 172.21.15.43:0/2425518505 >> [v2:172.21.15.203:6832/2959400181,v1:172.21.15.203:6833/2959400181] conn(0x7fb7b806a170 0x7fb7b806c630 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7fb7c0036000 tx=0x7fb7c003c040 comp rx=0 tx=0).ready entity=mgr.7223 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-09T14:52:24.105 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:24.104+0000 7fb7d57fa640 1 -- 172.21.15.43:0/2425518505 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178066 (secure 0 0 0) 0x7fb7c8053fe0 con 0x7fb7d80b1050 2023-08-09T14:52:24.336 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:24.335+0000 7fb7dd1c4640 1 -- 172.21.15.43:0/2425518505 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fb7d80739f0 con 0x7fb7d80b1050 2023-08-09T14:52:24.336 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:24.336+0000 7fb7d57fa640 1 -- 172.21.15.43:0/2425518505 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 80 v80) v1 ==== 94+0+3703 (secure 0 0 0) 0x7fb7c801fe90 con 0x7fb7d80b1050 2023-08-09T14:52:24.337 INFO:teuthology.orchestra.run.smithi043.stdout: 2023-08-09T14:52:24.337 INFO:teuthology.orchestra.run.smithi043.stdout:{"epoch":80,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":7079,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.203:6836/807059349","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6834","nonce":807059349},{"type":"v1","addr":"172.21.15.203:6836","nonce":807059349}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":74},{"gid":7924,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.203:6837/3166829033","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6835","nonce":3166829033},{"type":"v1","addr":"172.21.15.203:6837","nonce":3166829033}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":74},{"gid":9642,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.43:6833/3803974919","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6832","nonce":3803974919},{"type":"v1","addr":"172.21.15.43:6833","nonce":3803974919}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":73},{"gid":10077,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.43:6835/1417053935","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6834","nonce":1417053935},{"type":"v1","addr":"172.21.15.43:6835","nonce":1417053935}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":80}],"filesystems":[]} 2023-08-09T14:52:24.337 INFO:teuthology.orchestra.run.smithi043.stderr:dumped fsmap epoch 80 2023-08-09T14:52:24.339 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:24.339+0000 7fb7dd1c4640 1 -- 172.21.15.43:0/2425518505 >> [v2:172.21.15.203:6832/2959400181,v1:172.21.15.203:6833/2959400181] conn(0x7fb7b806a170 msgr2=0x7fb7b806c630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T14:52:24.340 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:24.339+0000 7fb7dd1c4640 1 --2- 172.21.15.43:0/2425518505 >> [v2:172.21.15.203:6832/2959400181,v1:172.21.15.203:6833/2959400181] conn(0x7fb7b806a170 0x7fb7b806c630 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7fb7c0036000 tx=0x7fb7c003c040 comp rx=0 tx=0).stop 2023-08-09T14:52:24.340 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:24.339+0000 7fb7dd1c4640 1 -- 172.21.15.43:0/2425518505 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb7d80b1050 msgr2=0x7fb7d8155ec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T14:52:24.340 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:24.339+0000 7fb7dd1c4640 1 --2- 172.21.15.43:0/2425518505 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb7d80b1050 0x7fb7d8155ec0 secure :-1 s=READY pgs=2000 cs=0 l=1 rev1=1 crypto rx=0x7fb7c800c7f0 tx=0x7fb7c800ccb0 comp rx=0 tx=0).stop 2023-08-09T14:52:24.340 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:24.339+0000 7fb7dd1c4640 1 -- 172.21.15.43:0/2425518505 shutdown_connections 2023-08-09T14:52:24.340 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:24.339+0000 7fb7dd1c4640 1 --2- 172.21.15.43:0/2425518505 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fb7d815f7b0 0x7fb7d80b2bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T14:52:24.340 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:24.339+0000 7fb7dd1c4640 1 --2- 172.21.15.43:0/2425518505 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fb7d80cd4e0 0x7fb7d80a9670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T14:52:24.341 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:24.339+0000 7fb7dd1c4640 1 --2- 172.21.15.43:0/2425518505 >> [v2:172.21.15.203:6832/2959400181,v1:172.21.15.203:6833/2959400181] conn(0x7fb7b806a170 0x7fb7b806c630 unknown :-1 s=CLOSED pgs=46 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T14:52:24.341 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:24.339+0000 7fb7dd1c4640 1 --2- 172.21.15.43:0/2425518505 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb7d80b1050 0x7fb7d8155ec0 unknown :-1 s=CLOSED pgs=2000 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T14:52:24.341 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:24.339+0000 7fb7dd1c4640 1 -- 172.21.15.43:0/2425518505 >> 172.21.15.43:0/2425518505 conn(0x7fb7d806a2b0 msgr2=0x7fb7d815ead0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-09T14:52:24.341 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:24.339+0000 7fb7dd1c4640 1 -- 172.21.15.43:0/2425518505 shutdown_connections 2023-08-09T14:52:24.341 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:24.339+0000 7fb7dd1c4640 1 -- 172.21.15.43:0/2425518505 wait complete. 2023-08-09T14:52:24.358 INFO:tasks.ceph.mds.a:Restarting daemon 2023-08-09T14:52:24.358 INFO:tasks.ceph.mds.a:Stopping old one... 2023-08-09T14:52:24.359 DEBUG:tasks.ceph.mds.a:waiting for process to exit 2023-08-09T14:52:24.359 INFO:teuthology.orchestra.run:waiting for 300 2023-08-09T14:52:24.359 INFO:tasks.ceph.mds.c:Restarting daemon 2023-08-09T14:52:24.359 INFO:tasks.ceph.mds.c:Stopping old one... 2023-08-09T14:52:24.360 DEBUG:tasks.ceph.mds.c:waiting for process to exit 2023-08-09T14:52:24.360 INFO:teuthology.orchestra.run:waiting for 300 2023-08-09T14:52:24.360 INFO:tasks.ceph.mds.b:Restarting daemon 2023-08-09T14:52:24.360 INFO:tasks.ceph.mds.b:Stopping old one... 2023-08-09T14:52:24.360 DEBUG:tasks.ceph.mds.b:waiting for process to exit 2023-08-09T14:52:24.361 INFO:teuthology.orchestra.run:waiting for 300 2023-08-09T14:52:24.361 INFO:tasks.ceph.mds.d:Restarting daemon 2023-08-09T14:52:24.361 INFO:tasks.ceph.mds.d:Stopping old one... 2023-08-09T14:52:24.361 DEBUG:tasks.ceph.mds.d:waiting for process to exit 2023-08-09T14:52:24.361 INFO:teuthology.orchestra.run:waiting for 300 2023-08-09T14:52:24.376 INFO:tasks.ceph.mds.a:Stopped 2023-08-09T14:52:24.377 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i a 2023-08-09T14:52:24.382 INFO:tasks.ceph.mds.d:Stopped 2023-08-09T14:52:24.383 DEBUG:teuthology.orchestra.run.smithi203:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i d 2023-08-09T14:52:24.384 INFO:tasks.ceph.mds.b:Stopped 2023-08-09T14:52:24.384 DEBUG:teuthology.orchestra.run.smithi203:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i b 2023-08-09T14:52:24.388 INFO:tasks.ceph.mds.c:Stopped 2023-08-09T14:52:24.389 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i c 2023-08-09T14:52:24.432 INFO:tasks.ceph.mds.d:Started 2023-08-09T14:52:24.432 DEBUG:teuthology.parallel:result is None 2023-08-09T14:52:24.436 INFO:tasks.ceph.mds.a:Started 2023-08-09T14:52:24.437 DEBUG:teuthology.parallel:result is None 2023-08-09T14:52:24.439 INFO:tasks.ceph.mds.b:Started 2023-08-09T14:52:24.440 DEBUG:teuthology.parallel:result is None 2023-08-09T14:52:24.444 INFO:tasks.ceph.mds.c:Started 2023-08-09T14:52:24.445 DEBUG:teuthology.parallel:result is None 2023-08-09T14:52:24.445 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist clear 2023-08-09T14:52:24.596 INFO:tasks.ceph.mds.b.smithi203.stdout:starting mds.b at 2023-08-09T14:52:24.597 INFO:tasks.ceph.mds.d.smithi203.stdout:starting mds.d at 2023-08-09T14:52:24.607 INFO:tasks.ceph.mds.c.smithi043.stdout:starting mds.c at 2023-08-09T14:52:24.608 INFO:tasks.ceph.mds.a.smithi043.stdout:starting mds.a at 2023-08-09T14:52:24.650 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:24.650+0000 7fc52bfff640 1 Processor -- start 2023-08-09T14:52:24.651 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:24.650+0000 7fc52bfff640 1 -- start start 2023-08-09T14:52:24.652 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:24.651+0000 7fc52bfff640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fc52c142b40 0x7fc52c0697c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T14:52:24.652 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:24.651+0000 7fc52bfff640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7fc52c0704c0 0x7fc52c069d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T14:52:24.652 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:24.652+0000 7fc52bfff640 1 -- --> v1:172.21.15.43:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc52c071620 con 0x7fc52c0c9630 2023-08-09T14:52:24.653 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:24.652+0000 7fc52bfff640 1 -- --> v2:172.21.15.43:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc52c071ab0 con 0x7fc52c0704c0 2023-08-09T14:52:24.653 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:24.652+0000 7fc52bfff640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc52c142f20 con 0x7fc52c142b40 2023-08-09T14:52:24.653 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:24.652+0000 7fc52b7fe640 1 --1- >> v1:172.21.15.43:6789/0 conn(0x7fc52c0c9630 0x7fc52c0c9ac0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.43:6789/0 says I am v1:172.21.15.43:48468/0 (socket says 172.21.15.43:48468) 2023-08-09T14:52:24.654 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:24.652+0000 7fc52a7fc640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7fc52c0704c0 0x7fc52c069d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T14:52:24.654 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:24.652+0000 7fc52b7fe640 1 -- 172.21.15.43:0/5627844 learned_addr learned my addr 172.21.15.43:0/5627844 (peer_addr_for_me v1:172.21.15.43:0/0) 2023-08-09T14:52:24.654 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:24.652+0000 7fc52affd640 1 --2- 172.21.15.43:0/5627844 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fc52c142b40 0x7fc52c0697c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T14:52:24.654 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:24.652+0000 7fc52a7fc640 1 -- 172.21.15.43:0/5627844 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fc52c142b40 msgr2=0x7fc52c0697c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-09T14:52:24.654 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:24.652+0000 7fc52a7fc640 1 --2- 172.21.15.43:0/5627844 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fc52c142b40 0x7fc52c0697c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T14:52:24.654 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:24.652+0000 7fc52a7fc640 1 -- 172.21.15.43:0/5627844 >> v1:172.21.15.43:6789/0 conn(0x7fc52c0c9630 legacy=0x7fc52c0c9ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T14:52:24.654 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:24.653+0000 7fc52a7fc640 1 -- 172.21.15.43:0/5627844 --> v2:172.21.15.43:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc52c070a10 con 0x7fc52c0704c0 2023-08-09T14:52:24.655 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:24.652+0000 7fc529ffb640 1 -- 172.21.15.43:0/5627844 <== mon.0 v1:172.21.15.43:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2891139244 0 0) 0x7fc52c071620 con 0x7fc52c0c9630 2023-08-09T14:52:24.655 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:24.654+0000 7fc52a7fc640 1 --2- 172.21.15.43:0/5627844 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc52c0704c0 0x7fc52c069d00 secure :-1 s=READY pgs=2004 cs=0 l=1 rev1=1 crypto rx=0x7fc5200097a0 tx=0x7fc52002b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d5329c6686af633 server_cookie=0 in_seq=0 out_seq=0 2023-08-09T14:52:24.656 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:24.654+0000 7fc529ffb640 1 -- 172.21.15.43:0/5627844 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc520031830 con 0x7fc52c0704c0 2023-08-09T14:52:24.656 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:24.654+0000 7fc529ffb640 1 -- 172.21.15.43:0/5627844 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc520031990 con 0x7fc52c0704c0 2023-08-09T14:52:24.656 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:24.654+0000 7fc529ffb640 1 -- 172.21.15.43:0/5627844 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc520031d20 con 0x7fc52c0704c0 2023-08-09T14:52:24.656 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:24.655+0000 7fc52bfff640 1 -- 172.21.15.43:0/5627844 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc52c0704c0 msgr2=0x7fc52c069d00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T14:52:24.656 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:24.655+0000 7fc52bfff640 1 --2- 172.21.15.43:0/5627844 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc52c0704c0 0x7fc52c069d00 secure :-1 s=READY pgs=2004 cs=0 l=1 rev1=1 crypto rx=0x7fc5200097a0 tx=0x7fc52002b8d0 comp rx=0 tx=0).stop 2023-08-09T14:52:24.656 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:24.655+0000 7fc52bfff640 1 -- 172.21.15.43:0/5627844 shutdown_connections 2023-08-09T14:52:24.657 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:24.655+0000 7fc52bfff640 1 --2- 172.21.15.43:0/5627844 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc52c0704c0 0x7fc52c069d00 unknown :-1 s=CLOSED pgs=2004 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T14:52:24.657 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:24.655+0000 7fc52bfff640 1 --2- 172.21.15.43:0/5627844 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fc52c142b40 0x7fc52c0697c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T14:52:24.657 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:24.655+0000 7fc52bfff640 1 -- 172.21.15.43:0/5627844 >> 172.21.15.43:0/5627844 conn(0x7fc52c0b0c50 msgr2=0x7fc52c0b1060 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-09T14:52:24.657 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:24.655+0000 7fc52bfff640 1 -- 172.21.15.43:0/5627844 shutdown_connections 2023-08-09T14:52:24.657 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:24.655+0000 7fc52bfff640 1 -- 172.21.15.43:0/5627844 wait complete. 2023-08-09T14:52:24.658 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:24.657+0000 7fc52bfff640 1 Processor -- start 2023-08-09T14:52:24.658 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:24.657+0000 7fc52bfff640 1 -- start start 2023-08-09T14:52:24.659 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:24.658+0000 7fc52bfff640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fc52c0704c0 0x7fc52c1e6960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T14:52:24.659 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:24.658+0000 7fc52bfff640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fc52c0c9630 0x7fc52c0c7910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T14:52:24.659 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:24.658+0000 7fc52bfff640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc52c142b40 0x7fc52c15ba60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T14:52:24.659 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:24.658+0000 7fc52bfff640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc52c13b160 con 0x7fc52c142b40 2023-08-09T14:52:24.660 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:24.658+0000 7fc52bfff640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc52c13ae80 con 0x7fc52c0704c0 2023-08-09T14:52:24.660 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:24.658+0000 7fc52bfff640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc52c1f95b0 con 0x7fc52c0c9630 2023-08-09T14:52:24.660 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:24.658+0000 7fc52b7fe640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc52c142b40 0x7fc52c15ba60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T14:52:24.660 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:24.658+0000 7fc52a7fc640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fc52c0c9630 0x7fc52c0c7910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T14:52:24.660 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:24.658+0000 7fc52b7fe640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc52c142b40 0x7fc52c15ba60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:38040/0 (socket says 172.21.15.43:38040) 2023-08-09T14:52:24.660 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:24.658+0000 7fc52b7fe640 1 -- 172.21.15.43:0/2412221277 learned_addr learned my addr 172.21.15.43:0/2412221277 (peer_addr_for_me v2:172.21.15.43:0/0) 2023-08-09T14:52:24.660 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:24.658+0000 7fc52affd640 1 --2- 172.21.15.43:0/2412221277 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fc52c0704c0 0x7fc52c1e6960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T14:52:24.661 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:24.659+0000 7fc52b7fe640 1 -- 172.21.15.43:0/2412221277 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fc52c0c9630 msgr2=0x7fc52c0c7910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T14:52:24.661 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:24.659+0000 7fc52b7fe640 1 --2- 172.21.15.43:0/2412221277 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fc52c0c9630 0x7fc52c0c7910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T14:52:24.661 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:24.659+0000 7fc52b7fe640 1 -- 172.21.15.43:0/2412221277 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fc52c0704c0 msgr2=0x7fc52c1e6960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T14:52:24.661 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:24.659+0000 7fc52b7fe640 1 --2- 172.21.15.43:0/2412221277 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fc52c0704c0 0x7fc52c1e6960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T14:52:24.661 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:24.659+0000 7fc52b7fe640 1 -- 172.21.15.43:0/2412221277 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc520009400 con 0x7fc52c142b40 2023-08-09T14:52:24.662 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:24.659+0000 7fc52b7fe640 1 --2- 172.21.15.43:0/2412221277 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc52c142b40 0x7fc52c15ba60 secure :-1 s=READY pgs=2005 cs=0 l=1 rev1=1 crypto rx=0x7fc52400c7e0 tx=0x7fc52400cca0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-09T14:52:24.662 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:24.659+0000 7fc5137fe640 1 -- 172.21.15.43:0/2412221277 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc52400a640 con 0x7fc52c142b40 2023-08-09T14:52:24.663 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:24.659+0000 7fc5137fe640 1 -- 172.21.15.43:0/2412221277 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc52400a7c0 con 0x7fc52c142b40 2023-08-09T14:52:24.663 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:24.659+0000 7fc52bfff640 1 -- 172.21.15.43:0/2412221277 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc52c1f9890 con 0x7fc52c142b40 2023-08-09T14:52:24.663 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:24.659+0000 7fc5137fe640 1 -- 172.21.15.43:0/2412221277 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc52400ab70 con 0x7fc52c142b40 2023-08-09T14:52:24.663 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:24.660+0000 7fc52bfff640 1 -- 172.21.15.43:0/2412221277 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc52c13eae0 con 0x7fc52c142b40 2023-08-09T14:52:24.663 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:24.661+0000 7fc52bfff640 1 -- 172.21.15.43:0/2412221277 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc52c0c4ad0 con 0x7fc52c142b40 2023-08-09T14:52:24.663 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:24.662+0000 7fc5137fe640 1 -- 172.21.15.43:0/2412221277 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 88) v1 ==== 88443+0+0 (secure 0 0 0) 0x7fc52400acd0 con 0x7fc52c142b40 2023-08-09T14:52:24.664 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:24.663+0000 7fc5137fe640 1 --2- 172.21.15.43:0/2412221277 >> [v2:172.21.15.203:6832/2959400181,v1:172.21.15.203:6833/2959400181] conn(0x7fc51806a1a0 0x7fc51806c660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T14:52:24.665 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:24.663+0000 7fc5137fe640 1 -- 172.21.15.43:0/2412221277 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(181..181 src has 1..181) v4 ==== 6348+0+0 (secure 0 0 0) 0x7fc524087be0 con 0x7fc52c142b40 2023-08-09T14:52:24.665 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:24.663+0000 7fc52affd640 1 --2- 172.21.15.43:0/2412221277 >> [v2:172.21.15.203:6832/2959400181,v1:172.21.15.203:6833/2959400181] conn(0x7fc51806a1a0 0x7fc51806c660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T14:52:24.665 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:24.664+0000 7fc52affd640 1 --2- 172.21.15.43:0/2412221277 >> [v2:172.21.15.203:6832/2959400181,v1:172.21.15.203:6833/2959400181] conn(0x7fc51806a1a0 0x7fc51806c660 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7fc51c009aa0 tx=0x7fc51c009190 comp rx=0 tx=0).ready entity=mgr.7223 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-09T14:52:24.667 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:24.667+0000 7fc5137fe640 1 -- 172.21.15.43:0/2412221277 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178066 (secure 0 0 0) 0x7fc524053140 con 0x7fc52c142b40 2023-08-09T14:52:24.882 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:24.882+0000 7fc52bfff640 1 -- 172.21.15.43:0/2412221277 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "osd blocklist clear"} v 0) v1 -- 0x7fc52c0739f0 con 0x7fc52c142b40 2023-08-09T14:52:25.066 DEBUG:teuthology.orchestra.run.smithi043:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-08-09T14:52:25.076 DEBUG:teuthology.orchestra.run.smithi203:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-08-09T14:52:25.653 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:25.652+0000 7fc5137fe640 1 -- 172.21.15.43:0/2412221277 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist clear"}]=0 removed all blocklist entries v182) v1 ==== 97+0+0 (secure 0 0 0) 0x7fc524018080 con 0x7fc52c142b40 2023-08-09T14:52:25.653 INFO:teuthology.orchestra.run.smithi043.stderr: removed all blocklist entries 2023-08-09T14:52:25.656 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:25.656+0000 7fc52bfff640 1 -- 172.21.15.43:0/2412221277 >> [v2:172.21.15.203:6832/2959400181,v1:172.21.15.203:6833/2959400181] conn(0x7fc51806a1a0 msgr2=0x7fc51806c660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T14:52:25.657 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:25.656+0000 7fc52bfff640 1 --2- 172.21.15.43:0/2412221277 >> [v2:172.21.15.203:6832/2959400181,v1:172.21.15.203:6833/2959400181] conn(0x7fc51806a1a0 0x7fc51806c660 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7fc51c009aa0 tx=0x7fc51c009190 comp rx=0 tx=0).stop 2023-08-09T14:52:25.657 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:25.656+0000 7fc52bfff640 1 -- 172.21.15.43:0/2412221277 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc52c142b40 msgr2=0x7fc52c15ba60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T14:52:25.657 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:25.656+0000 7fc52bfff640 1 --2- 172.21.15.43:0/2412221277 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc52c142b40 0x7fc52c15ba60 secure :-1 s=READY pgs=2005 cs=0 l=1 rev1=1 crypto rx=0x7fc52400c7e0 tx=0x7fc52400cca0 comp rx=0 tx=0).stop 2023-08-09T14:52:25.657 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:25.656+0000 7fc52bfff640 1 -- 172.21.15.43:0/2412221277 shutdown_connections 2023-08-09T14:52:25.657 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:25.656+0000 7fc52bfff640 1 --2- 172.21.15.43:0/2412221277 >> [v2:172.21.15.203:6832/2959400181,v1:172.21.15.203:6833/2959400181] conn(0x7fc51806a1a0 0x7fc51806c660 unknown :-1 s=CLOSED pgs=48 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T14:52:25.657 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:25.656+0000 7fc52bfff640 1 --2- 172.21.15.43:0/2412221277 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc52c142b40 0x7fc52c15ba60 unknown :-1 s=CLOSED pgs=2005 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T14:52:25.658 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:25.656+0000 7fc52bfff640 1 --2- 172.21.15.43:0/2412221277 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fc52c0c9630 0x7fc52c0c7910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T14:52:25.658 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:25.656+0000 7fc52bfff640 1 --2- 172.21.15.43:0/2412221277 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fc52c0704c0 0x7fc52c1e6960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T14:52:25.658 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:25.656+0000 7fc52bfff640 1 -- 172.21.15.43:0/2412221277 >> 172.21.15.43:0/2412221277 conn(0x7fc52c0b0c50 msgr2=0x7fc52c0af870 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-09T14:52:25.658 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:25.656+0000 7fc52bfff640 1 -- 172.21.15.43:0/2412221277 shutdown_connections 2023-08-09T14:52:25.658 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:25.656+0000 7fc52bfff640 1 -- 172.21.15.43:0/2412221277 wait complete. 2023-08-09T14:52:25.678 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth ls --format=json-pretty 2023-08-09T14:52:25.906 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:25.905+0000 7f76cf59e640 1 Processor -- start 2023-08-09T14:52:25.906 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:25.905+0000 7f76cf59e640 1 -- start start 2023-08-09T14:52:25.908 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:25.907+0000 7f76cf59e640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f76d00c0e60 0x7f76d0160e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T14:52:25.908 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:25.907+0000 7f76cf59e640 1 -- --> v1:172.21.15.43:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f76d01614b0 con 0x7f76d0149e10 2023-08-09T14:52:25.908 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:25.907+0000 7f76cf59e640 1 -- --> v1:172.21.15.203:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f76d0161790 con 0x7f76d00c9620 2023-08-09T14:52:25.908 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:25.907+0000 7f76cf59e640 1 -- --> v2:172.21.15.43:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f76d0164540 con 0x7f76d00c0e60 2023-08-09T14:52:25.909 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:25.908+0000 7f76ce59c640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f76d00c0e60 0x7f76d0160e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T14:52:25.909 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:25.908+0000 7f76cdd9b640 1 --1- >> v1:172.21.15.43:6789/0 conn(0x7f76d0149e10 0x7f76d00c1240 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.43:6789/0 says I am v1:172.21.15.43:50408/0 (socket says 172.21.15.43:50408) 2023-08-09T14:52:25.909 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:25.908+0000 7f76cdd9b640 1 -- 172.21.15.43:0/3708844037 learned_addr learned my addr 172.21.15.43:0/3708844037 (peer_addr_for_me v1:172.21.15.43:0/0) 2023-08-09T14:52:25.909 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:25.908+0000 7f76ce59c640 1 -- 172.21.15.43:0/3708844037 >> v1:172.21.15.203:6789/0 conn(0x7f76d00c9620 legacy=0x7f76d0164960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-09T14:52:25.910 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:25.908+0000 7f76ce59c640 1 -- 172.21.15.43:0/3708844037 >> v1:172.21.15.43:6789/0 conn(0x7f76d0149e10 legacy=0x7f76d00c1240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T14:52:25.910 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:25.908+0000 7f76ce59c640 1 -- 172.21.15.43:0/3708844037 --> v2:172.21.15.43:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f76d0161a70 con 0x7f76d00c0e60 2023-08-09T14:52:25.910 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:25.908+0000 7f76cd59a640 1 -- 172.21.15.43:0/3708844037 <== mon.0 v1:172.21.15.43:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3404605560 0 0) 0x7f76d01614b0 con 0x7f76d0149e10 2023-08-09T14:52:25.911 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:25.910+0000 7f76ce59c640 1 --2- 172.21.15.43:0/3708844037 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f76d00c0e60 0x7f76d0160e70 secure :-1 s=READY pgs=2007 cs=0 l=1 rev1=1 crypto rx=0x7f76c0009d90 tx=0x7f76c002beb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=240744aaa23dc10a server_cookie=0 in_seq=0 out_seq=0 2023-08-09T14:52:25.911 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:25.910+0000 7f76cd59a640 1 -- 172.21.15.43:0/3708844037 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f76c002cc60 con 0x7f76d00c0e60 2023-08-09T14:52:25.911 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:25.910+0000 7f76cd59a640 1 -- 172.21.15.43:0/3708844037 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f76c002cdc0 con 0x7f76d00c0e60 2023-08-09T14:52:25.911 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:25.910+0000 7f76cd59a640 1 -- 172.21.15.43:0/3708844037 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f76c0034660 con 0x7f76d00c0e60 2023-08-09T14:52:25.912 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:25.910+0000 7f76cf59e640 1 -- 172.21.15.43:0/3708844037 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f76d00c0e60 msgr2=0x7f76d0160e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T14:52:25.912 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:25.911+0000 7f76cf59e640 1 --2- 172.21.15.43:0/3708844037 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f76d00c0e60 0x7f76d0160e70 secure :-1 s=READY pgs=2007 cs=0 l=1 rev1=1 crypto rx=0x7f76c0009d90 tx=0x7f76c002beb0 comp rx=0 tx=0).stop 2023-08-09T14:52:25.912 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:25.911+0000 7f76cf59e640 1 -- 172.21.15.43:0/3708844037 shutdown_connections 2023-08-09T14:52:25.912 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:25.911+0000 7f76cf59e640 1 --2- 172.21.15.43:0/3708844037 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f76d00c0e60 0x7f76d0160e70 unknown :-1 s=CLOSED pgs=2007 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T14:52:25.912 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:25.911+0000 7f76cf59e640 1 -- 172.21.15.43:0/3708844037 >> 172.21.15.43:0/3708844037 conn(0x7f76d00cde90 msgr2=0x7f76d00ce2a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-09T14:52:25.912 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:25.911+0000 7f76cf59e640 1 -- 172.21.15.43:0/3708844037 shutdown_connections 2023-08-09T14:52:25.913 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:25.911+0000 7f76cf59e640 1 -- 172.21.15.43:0/3708844037 wait complete. 2023-08-09T14:52:25.913 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:25.913+0000 7f76cf59e640 1 Processor -- start 2023-08-09T14:52:25.913 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:25.913+0000 7f76cf59e640 1 -- start start 2023-08-09T14:52:25.914 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:25.914+0000 7f76cf59e640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f76d00c0e60 0x7f76d01f5e80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T14:52:25.914 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:25.914+0000 7f76cf59e640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f76d00c9620 0x7f76d00b2bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T14:52:25.914 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:25.914+0000 7f76cf59e640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f76d0149e10 0x7f76d0163560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T14:52:25.914 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:25.914+0000 7f76cf59e640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f76d014a2a0 con 0x7f76d0149e10 2023-08-09T14:52:25.915 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:25.914+0000 7f76cf59e640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f76d0152b50 con 0x7f76d00c0e60 2023-08-09T14:52:25.915 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:25.914+0000 7f76cf59e640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f76d01f63c0 con 0x7f76d00c9620 2023-08-09T14:52:25.916 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:25.914+0000 7f76ced9d640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f76d0149e10 0x7f76d0163560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T14:52:25.916 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:25.914+0000 7f76ced9d640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f76d0149e10 0x7f76d0163560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:55382/0 (socket says 172.21.15.43:55382) 2023-08-09T14:52:25.916 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:25.914+0000 7f76ced9d640 1 -- 172.21.15.43:0/2083141240 learned_addr learned my addr 172.21.15.43:0/2083141240 (peer_addr_for_me v2:172.21.15.43:0/0) 2023-08-09T14:52:25.916 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:25.914+0000 7f76ce59c640 1 --2- 172.21.15.43:0/2083141240 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f76d00c0e60 0x7f76d01f5e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T14:52:25.916 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:25.914+0000 7f76cdd9b640 1 --2- 172.21.15.43:0/2083141240 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f76d00c9620 0x7f76d00b2bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T14:52:25.917 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:25.915+0000 7f76ced9d640 1 -- 172.21.15.43:0/2083141240 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f76d00c9620 msgr2=0x7f76d00b2bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T14:52:25.917 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:25.915+0000 7f76ced9d640 1 --2- 172.21.15.43:0/2083141240 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f76d00c9620 0x7f76d00b2bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T14:52:25.917 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:25.915+0000 7f76ced9d640 1 -- 172.21.15.43:0/2083141240 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f76d00c0e60 msgr2=0x7f76d01f5e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T14:52:25.917 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:25.915+0000 7f76ced9d640 1 --2- 172.21.15.43:0/2083141240 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f76d00c0e60 0x7f76d01f5e80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T14:52:25.917 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:25.915+0000 7f76ced9d640 1 -- 172.21.15.43:0/2083141240 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f76c0009a70 con 0x7f76d0149e10 2023-08-09T14:52:25.917 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:25.915+0000 7f76ced9d640 1 --2- 172.21.15.43:0/2083141240 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f76d0149e10 0x7f76d0163560 secure :-1 s=READY pgs=2008 cs=0 l=1 rev1=1 crypto rx=0x7f76c800c440 tx=0x7f76c800c900 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-09T14:52:25.917 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:25.915+0000 7f76beffd640 1 -- 172.21.15.43:0/2083141240 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f76c8011790 con 0x7f76d0149e10 2023-08-09T14:52:25.918 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:25.915+0000 7f76beffd640 1 -- 172.21.15.43:0/2083141240 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f76c80118f0 con 0x7f76d0149e10 2023-08-09T14:52:25.918 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:25.915+0000 7f76beffd640 1 -- 172.21.15.43:0/2083141240 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f76c8011ca0 con 0x7f76d0149e10 2023-08-09T14:52:25.918 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:25.915+0000 7f76cf59e640 1 -- 172.21.15.43:0/2083141240 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f76d00b34f0 con 0x7f76d0149e10 2023-08-09T14:52:25.918 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:25.916+0000 7f76cf59e640 1 -- 172.21.15.43:0/2083141240 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f76d00b3ab0 con 0x7f76d0149e10 2023-08-09T14:52:25.918 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:25.917+0000 7f76cf59e640 1 -- 172.21.15.43:0/2083141240 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f76d00739f0 con 0x7f76d0149e10 2023-08-09T14:52:25.919 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:25.918+0000 7f76beffd640 1 -- 172.21.15.43:0/2083141240 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 88) v1 ==== 88443+0+0 (secure 0 0 0) 0x7f76c801c460 con 0x7f76d0149e10 2023-08-09T14:52:25.919 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:25.918+0000 7f76beffd640 1 --2- 172.21.15.43:0/2083141240 >> [v2:172.21.15.203:6832/2959400181,v1:172.21.15.203:6833/2959400181] conn(0x7f76b006a170 0x7f76b006c630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T14:52:25.919 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:25.919+0000 7f76beffd640 1 -- 172.21.15.43:0/2083141240 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(182..182 src has 1..182) v4 ==== 5574+0+0 (secure 0 0 0) 0x7f76c8088430 con 0x7f76d0149e10 2023-08-09T14:52:25.920 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:25.919+0000 7f76ce59c640 1 --2- 172.21.15.43:0/2083141240 >> [v2:172.21.15.203:6832/2959400181,v1:172.21.15.203:6833/2959400181] conn(0x7f76b006a170 0x7f76b006c630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T14:52:25.921 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:25.920+0000 7f76ce59c640 1 --2- 172.21.15.43:0/2083141240 >> [v2:172.21.15.203:6832/2959400181,v1:172.21.15.203:6833/2959400181] conn(0x7f76b006a170 0x7f76b006c630 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7f76c0036000 tx=0x7f76c0030000 comp rx=0 tx=0).ready entity=mgr.7223 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-09T14:52:25.923 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:25.923+0000 7f76beffd640 1 -- 172.21.15.43:0/2083141240 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178066 (secure 0 0 0) 0x7f76c8053be0 con 0x7f76d0149e10 2023-08-09T14:52:26.190 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:26.189+0000 7f76cf59e640 1 -- 172.21.15.43:0/2083141240 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "auth ls", "format": "json-pretty"} v 0) v1 -- 0x7f76d00b8430 con 0x7f76d0149e10 2023-08-09T14:52:26.191 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:26.191+0000 7f76beffd640 1 -- 172.21.15.43:0/2083141240 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "auth ls", "format": "json-pretty"}]=0 v40) v1 ==== 80+0+5304 (secure 0 0 0) 0x7f76c8057200 con 0x7f76d0149e10 2023-08-09T14:52:26.191 INFO:teuthology.orchestra.run.smithi043.stdout: 2023-08-09T14:52:26.191 INFO:teuthology.orchestra.run.smithi043.stdout:{ 2023-08-09T14:52:26.192 INFO:teuthology.orchestra.run.smithi043.stdout: "auth_dump": [ 2023-08-09T14:52:26.192 INFO:teuthology.orchestra.run.smithi043.stdout: { 2023-08-09T14:52:26.192 INFO:teuthology.orchestra.run.smithi043.stdout: "entity": "mds.a", 2023-08-09T14:52:26.192 INFO:teuthology.orchestra.run.smithi043.stdout: "key": "AQCOldNkPu6GCRAAZecwctvMji0Zubb60KMlyQ==", 2023-08-09T14:52:26.192 INFO:teuthology.orchestra.run.smithi043.stdout: "caps": { 2023-08-09T14:52:26.192 INFO:teuthology.orchestra.run.smithi043.stdout: "mds": "allow", 2023-08-09T14:52:26.193 INFO:teuthology.orchestra.run.smithi043.stdout: "mgr": "allow *", 2023-08-09T14:52:26.193 INFO:teuthology.orchestra.run.smithi043.stdout: "mon": "allow *", 2023-08-09T14:52:26.193 INFO:teuthology.orchestra.run.smithi043.stdout: "osd": "allow *" 2023-08-09T14:52:26.193 INFO:teuthology.orchestra.run.smithi043.stdout: } 2023-08-09T14:52:26.193 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2023-08-09T14:52:26.193 INFO:teuthology.orchestra.run.smithi043.stdout: { 2023-08-09T14:52:26.193 INFO:teuthology.orchestra.run.smithi043.stdout: "entity": "mds.b", 2023-08-09T14:52:26.194 INFO:teuthology.orchestra.run.smithi043.stdout: "key": "AQCOldNkt2+5KxAAHcckB+S5NZHsF5z22StXmw==", 2023-08-09T14:52:26.194 INFO:teuthology.orchestra.run.smithi043.stdout: "caps": { 2023-08-09T14:52:26.194 INFO:teuthology.orchestra.run.smithi043.stdout: "mds": "allow", 2023-08-09T14:52:26.194 INFO:teuthology.orchestra.run.smithi043.stdout: "mgr": "allow *", 2023-08-09T14:52:26.194 INFO:teuthology.orchestra.run.smithi043.stdout: "mon": "allow *", 2023-08-09T14:52:26.194 INFO:teuthology.orchestra.run.smithi043.stdout: "osd": "allow *" 2023-08-09T14:52:26.194 INFO:teuthology.orchestra.run.smithi043.stdout: } 2023-08-09T14:52:26.194 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2023-08-09T14:52:26.195 INFO:teuthology.orchestra.run.smithi043.stdout: { 2023-08-09T14:52:26.195 INFO:teuthology.orchestra.run.smithi043.stdout: "entity": "mds.c", 2023-08-09T14:52:26.195 INFO:teuthology.orchestra.run.smithi043.stdout: "key": "AQCOldNk5V46GhAAQBhkvfjVSJCSHLS/YkhgCA==", 2023-08-09T14:52:26.195 INFO:teuthology.orchestra.run.smithi043.stdout: "caps": { 2023-08-09T14:52:26.195 INFO:teuthology.orchestra.run.smithi043.stdout: "mds": "allow", 2023-08-09T14:52:26.195 INFO:teuthology.orchestra.run.smithi043.stdout: "mgr": "allow *", 2023-08-09T14:52:26.195 INFO:teuthology.orchestra.run.smithi043.stdout: "mon": "allow *", 2023-08-09T14:52:26.195 INFO:teuthology.orchestra.run.smithi043.stdout: "osd": "allow *" 2023-08-09T14:52:26.196 INFO:teuthology.orchestra.run.smithi043.stdout: } 2023-08-09T14:52:26.196 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2023-08-09T14:52:26.196 INFO:teuthology.orchestra.run.smithi043.stdout: { 2023-08-09T14:52:26.196 INFO:teuthology.orchestra.run.smithi043.stdout: "entity": "mds.d", 2023-08-09T14:52:26.196 INFO:teuthology.orchestra.run.smithi043.stdout: "key": "AQCPldNkNBoTARAAMVW+rSyXQKaau6I90fUX9Q==", 2023-08-09T14:52:26.196 INFO:teuthology.orchestra.run.smithi043.stdout: "caps": { 2023-08-09T14:52:26.197 INFO:teuthology.orchestra.run.smithi043.stdout: "mds": "allow", 2023-08-09T14:52:26.197 INFO:teuthology.orchestra.run.smithi043.stdout: "mgr": "allow *", 2023-08-09T14:52:26.197 INFO:teuthology.orchestra.run.smithi043.stdout: "mon": "allow *", 2023-08-09T14:52:26.197 INFO:teuthology.orchestra.run.smithi043.stdout: "osd": "allow *" 2023-08-09T14:52:26.197 INFO:teuthology.orchestra.run.smithi043.stdout: } 2023-08-09T14:52:26.197 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2023-08-09T14:52:26.198 INFO:teuthology.orchestra.run.smithi043.stdout: { 2023-08-09T14:52:26.198 INFO:teuthology.orchestra.run.smithi043.stdout: "entity": "osd.0", 2023-08-09T14:52:26.198 INFO:teuthology.orchestra.run.smithi043.stdout: "key": "AQCRldNkltPlMhAAAt3rFvraCtI0aSqGvt6UxA==", 2023-08-09T14:52:26.198 INFO:teuthology.orchestra.run.smithi043.stdout: "caps": { 2023-08-09T14:52:26.198 INFO:teuthology.orchestra.run.smithi043.stdout: "mgr": "allow profile osd", 2023-08-09T14:52:26.198 INFO:teuthology.orchestra.run.smithi043.stdout: "mon": "allow profile osd", 2023-08-09T14:52:26.198 INFO:teuthology.orchestra.run.smithi043.stdout: "osd": "allow *" 2023-08-09T14:52:26.198 INFO:teuthology.orchestra.run.smithi043.stdout: } 2023-08-09T14:52:26.199 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2023-08-09T14:52:26.199 INFO:teuthology.orchestra.run.smithi043.stdout: { 2023-08-09T14:52:26.199 INFO:teuthology.orchestra.run.smithi043.stdout: "entity": "osd.1", 2023-08-09T14:52:26.199 INFO:teuthology.orchestra.run.smithi043.stdout: "key": "AQCVldNkzpKYEhAA+wen0kV/vUDLQztq3vhfOg==", 2023-08-09T14:52:26.199 INFO:teuthology.orchestra.run.smithi043.stdout: "caps": { 2023-08-09T14:52:26.199 INFO:teuthology.orchestra.run.smithi043.stdout: "mgr": "allow profile osd", 2023-08-09T14:52:26.199 INFO:teuthology.orchestra.run.smithi043.stdout: "mon": "allow profile osd", 2023-08-09T14:52:26.199 INFO:teuthology.orchestra.run.smithi043.stdout: "osd": "allow *" 2023-08-09T14:52:26.200 INFO:teuthology.orchestra.run.smithi043.stdout: } 2023-08-09T14:52:26.200 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2023-08-09T14:52:26.200 INFO:teuthology.orchestra.run.smithi043.stdout: { 2023-08-09T14:52:26.200 INFO:teuthology.orchestra.run.smithi043.stdout: "entity": "osd.2", 2023-08-09T14:52:26.200 INFO:teuthology.orchestra.run.smithi043.stdout: "key": "AQCYldNkYbwNHhAAQY8g9dldb1V9M2MM2SHKDg==", 2023-08-09T14:52:26.200 INFO:teuthology.orchestra.run.smithi043.stdout: "caps": { 2023-08-09T14:52:26.200 INFO:teuthology.orchestra.run.smithi043.stdout: "mgr": "allow profile osd", 2023-08-09T14:52:26.201 INFO:teuthology.orchestra.run.smithi043.stdout: "mon": "allow profile osd", 2023-08-09T14:52:26.201 INFO:teuthology.orchestra.run.smithi043.stdout: "osd": "allow *" 2023-08-09T14:52:26.201 INFO:teuthology.orchestra.run.smithi043.stdout: } 2023-08-09T14:52:26.201 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2023-08-09T14:52:26.201 INFO:teuthology.orchestra.run.smithi043.stdout: { 2023-08-09T14:52:26.202 INFO:teuthology.orchestra.run.smithi043.stdout: "entity": "osd.3", 2023-08-09T14:52:26.202 INFO:teuthology.orchestra.run.smithi043.stdout: "key": "AQCbldNkeyW6KRAAsd+kgqik2aD4U7PXwccWtA==", 2023-08-09T14:52:26.202 INFO:teuthology.orchestra.run.smithi043.stdout: "caps": { 2023-08-09T14:52:26.202 INFO:teuthology.orchestra.run.smithi043.stdout: "mgr": "allow profile osd", 2023-08-09T14:52:26.202 INFO:teuthology.orchestra.run.smithi043.stdout: "mon": "allow profile osd", 2023-08-09T14:52:26.202 INFO:teuthology.orchestra.run.smithi043.stdout: "osd": "allow *" 2023-08-09T14:52:26.202 INFO:teuthology.orchestra.run.smithi043.stdout: } 2023-08-09T14:52:26.202 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2023-08-09T14:52:26.203 INFO:teuthology.orchestra.run.smithi043.stdout: { 2023-08-09T14:52:26.203 INFO:teuthology.orchestra.run.smithi043.stdout: "entity": "osd.4", 2023-08-09T14:52:26.203 INFO:teuthology.orchestra.run.smithi043.stdout: "key": "AQCildNk80UkOBAAtX+4JGpOVcfZEHetJoeZKw==", 2023-08-09T14:52:26.203 INFO:teuthology.orchestra.run.smithi043.stdout: "caps": { 2023-08-09T14:52:26.203 INFO:teuthology.orchestra.run.smithi043.stdout: "mgr": "allow profile osd", 2023-08-09T14:52:26.203 INFO:teuthology.orchestra.run.smithi043.stdout: "mon": "allow profile osd", 2023-08-09T14:52:26.203 INFO:teuthology.orchestra.run.smithi043.stdout: "osd": "allow *" 2023-08-09T14:52:26.203 INFO:teuthology.orchestra.run.smithi043.stdout: } 2023-08-09T14:52:26.204 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2023-08-09T14:52:26.204 INFO:teuthology.orchestra.run.smithi043.stdout: { 2023-08-09T14:52:26.204 INFO:teuthology.orchestra.run.smithi043.stdout: "entity": "osd.5", 2023-08-09T14:52:26.204 INFO:teuthology.orchestra.run.smithi043.stdout: "key": "AQCmldNkCA9SBxAAZj4So6MRj2Eq1I9H9JzLDA==", 2023-08-09T14:52:26.204 INFO:teuthology.orchestra.run.smithi043.stdout: "caps": { 2023-08-09T14:52:26.204 INFO:teuthology.orchestra.run.smithi043.stdout: "mgr": "allow profile osd", 2023-08-09T14:52:26.204 INFO:teuthology.orchestra.run.smithi043.stdout: "mon": "allow profile osd", 2023-08-09T14:52:26.205 INFO:teuthology.orchestra.run.smithi043.stdout: "osd": "allow *" 2023-08-09T14:52:26.205 INFO:teuthology.orchestra.run.smithi043.stdout: } 2023-08-09T14:52:26.205 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2023-08-09T14:52:26.205 INFO:teuthology.orchestra.run.smithi043.stdout: { 2023-08-09T14:52:26.205 INFO:teuthology.orchestra.run.smithi043.stdout: "entity": "osd.6", 2023-08-09T14:52:26.205 INFO:teuthology.orchestra.run.smithi043.stdout: "key": "AQCpldNkaOwAExAAraV1tUwr9uU6GheZ+hvdqQ==", 2023-08-09T14:52:26.205 INFO:teuthology.orchestra.run.smithi043.stdout: "caps": { 2023-08-09T14:52:26.205 INFO:teuthology.orchestra.run.smithi043.stdout: "mgr": "allow profile osd", 2023-08-09T14:52:26.206 INFO:teuthology.orchestra.run.smithi043.stdout: "mon": "allow profile osd", 2023-08-09T14:52:26.206 INFO:teuthology.orchestra.run.smithi043.stdout: "osd": "allow *" 2023-08-09T14:52:26.206 INFO:teuthology.orchestra.run.smithi043.stdout: } 2023-08-09T14:52:26.206 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2023-08-09T14:52:26.206 INFO:teuthology.orchestra.run.smithi043.stdout: { 2023-08-09T14:52:26.206 INFO:teuthology.orchestra.run.smithi043.stdout: "entity": "osd.7", 2023-08-09T14:52:26.206 INFO:teuthology.orchestra.run.smithi043.stdout: "key": "AQCsldNk2/GwLBAAP42EXPivzLk44X8aMEVVCg==", 2023-08-09T14:52:26.207 INFO:teuthology.orchestra.run.smithi043.stdout: "caps": { 2023-08-09T14:52:26.207 INFO:teuthology.orchestra.run.smithi043.stdout: "mgr": "allow profile osd", 2023-08-09T14:52:26.207 INFO:teuthology.orchestra.run.smithi043.stdout: "mon": "allow profile osd", 2023-08-09T14:52:26.207 INFO:teuthology.orchestra.run.smithi043.stdout: "osd": "allow *" 2023-08-09T14:52:26.207 INFO:teuthology.orchestra.run.smithi043.stdout: } 2023-08-09T14:52:26.208 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2023-08-09T14:52:26.208 INFO:teuthology.orchestra.run.smithi043.stdout: { 2023-08-09T14:52:26.208 INFO:teuthology.orchestra.run.smithi043.stdout: "entity": "client.0", 2023-08-09T14:52:26.208 INFO:teuthology.orchestra.run.smithi043.stdout: "key": "AQCPldNkg8zrDxAAM/hnJaE6S9HU/fRwQeBV5w==", 2023-08-09T14:52:26.208 INFO:teuthology.orchestra.run.smithi043.stdout: "caps": { 2023-08-09T14:52:26.208 INFO:teuthology.orchestra.run.smithi043.stdout: "mds": "allow", 2023-08-09T14:52:26.209 INFO:teuthology.orchestra.run.smithi043.stdout: "mon": "allow r", 2023-08-09T14:52:26.209 INFO:teuthology.orchestra.run.smithi043.stdout: "osd": "allow rw tag cephfs data=cephfs" 2023-08-09T14:52:26.209 INFO:teuthology.orchestra.run.smithi043.stdout: } 2023-08-09T14:52:26.209 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2023-08-09T14:52:26.209 INFO:teuthology.orchestra.run.smithi043.stdout: { 2023-08-09T14:52:26.209 INFO:teuthology.orchestra.run.smithi043.stdout: "entity": "client.1", 2023-08-09T14:52:26.209 INFO:teuthology.orchestra.run.smithi043.stdout: "key": "AQCPldNklfFxJxAA19+L78rW6jU1uOyjey6WNA==", 2023-08-09T14:52:26.210 INFO:teuthology.orchestra.run.smithi043.stdout: "caps": { 2023-08-09T14:52:26.210 INFO:teuthology.orchestra.run.smithi043.stdout: "mds": "allow", 2023-08-09T14:52:26.210 INFO:teuthology.orchestra.run.smithi043.stdout: "mgr": "allow r", 2023-08-09T14:52:26.210 INFO:teuthology.orchestra.run.smithi043.stdout: "mon": "allow rw", 2023-08-09T14:52:26.210 INFO:teuthology.orchestra.run.smithi043.stdout: "osd": "allow rwx" 2023-08-09T14:52:26.210 INFO:teuthology.orchestra.run.smithi043.stdout: } 2023-08-09T14:52:26.210 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2023-08-09T14:52:26.211 INFO:teuthology.orchestra.run.smithi043.stdout: { 2023-08-09T14:52:26.211 INFO:teuthology.orchestra.run.smithi043.stdout: "entity": "client.2", 2023-08-09T14:52:26.211 INFO:teuthology.orchestra.run.smithi043.stdout: "key": "AQCPldNkC2lgGxAAZ8qZboOCUaQlcpkPL1ozHA==", 2023-08-09T14:52:26.211 INFO:teuthology.orchestra.run.smithi043.stdout: "caps": { 2023-08-09T14:52:26.211 INFO:teuthology.orchestra.run.smithi043.stdout: "mds": "allow", 2023-08-09T14:52:26.211 INFO:teuthology.orchestra.run.smithi043.stdout: "mgr": "allow r", 2023-08-09T14:52:26.212 INFO:teuthology.orchestra.run.smithi043.stdout: "mon": "allow rw", 2023-08-09T14:52:26.212 INFO:teuthology.orchestra.run.smithi043.stdout: "osd": "allow rwx" 2023-08-09T14:52:26.212 INFO:teuthology.orchestra.run.smithi043.stdout: } 2023-08-09T14:52:26.212 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2023-08-09T14:52:26.212 INFO:teuthology.orchestra.run.smithi043.stdout: { 2023-08-09T14:52:26.212 INFO:teuthology.orchestra.run.smithi043.stdout: "entity": "client.3", 2023-08-09T14:52:26.213 INFO:teuthology.orchestra.run.smithi043.stdout: "key": "AQCPldNkl+rXMhAA9rf3KdnSYbvngqmLYnIGGA==", 2023-08-09T14:52:26.213 INFO:teuthology.orchestra.run.smithi043.stdout: "caps": { 2023-08-09T14:52:26.213 INFO:teuthology.orchestra.run.smithi043.stdout: "mds": "allow", 2023-08-09T14:52:26.213 INFO:teuthology.orchestra.run.smithi043.stdout: "mgr": "allow r", 2023-08-09T14:52:26.213 INFO:teuthology.orchestra.run.smithi043.stdout: "mon": "allow rw", 2023-08-09T14:52:26.213 INFO:teuthology.orchestra.run.smithi043.stdout: "osd": "allow rwx" 2023-08-09T14:52:26.214 INFO:teuthology.orchestra.run.smithi043.stdout: } 2023-08-09T14:52:26.214 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2023-08-09T14:52:26.214 INFO:teuthology.orchestra.run.smithi043.stdout: { 2023-08-09T14:52:26.214 INFO:teuthology.orchestra.run.smithi043.stdout: "entity": "client.admin", 2023-08-09T14:52:26.214 INFO:teuthology.orchestra.run.smithi043.stdout: "key": "AQCMldNkUae6NRAA2FBTr0QPBdKNDoe1WK5zyg==", 2023-08-09T14:52:26.214 INFO:teuthology.orchestra.run.smithi043.stdout: "caps": { 2023-08-09T14:52:26.214 INFO:teuthology.orchestra.run.smithi043.stdout: "mds": "allow *", 2023-08-09T14:52:26.214 INFO:teuthology.orchestra.run.smithi043.stdout: "mgr": "allow *", 2023-08-09T14:52:26.215 INFO:teuthology.orchestra.run.smithi043.stdout: "mon": "allow *", 2023-08-09T14:52:26.215 INFO:teuthology.orchestra.run.smithi043.stdout: "osd": "allow *" 2023-08-09T14:52:26.215 INFO:teuthology.orchestra.run.smithi043.stdout: } 2023-08-09T14:52:26.215 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2023-08-09T14:52:26.215 INFO:teuthology.orchestra.run.smithi043.stdout: { 2023-08-09T14:52:26.215 INFO:teuthology.orchestra.run.smithi043.stdout: "entity": "mgr.x", 2023-08-09T14:52:26.215 INFO:teuthology.orchestra.run.smithi043.stdout: "key": "AQCNldNk2ufSNxAAU7msRkwLaoLZHTIsk0ck/g==", 2023-08-09T14:52:26.215 INFO:teuthology.orchestra.run.smithi043.stdout: "caps": { 2023-08-09T14:52:26.216 INFO:teuthology.orchestra.run.smithi043.stdout: "mds": "allow *", 2023-08-09T14:52:26.216 INFO:teuthology.orchestra.run.smithi043.stdout: "mon": "allow profile mgr", 2023-08-09T14:52:26.216 INFO:teuthology.orchestra.run.smithi043.stdout: "osd": "allow *" 2023-08-09T14:52:26.216 INFO:teuthology.orchestra.run.smithi043.stdout: } 2023-08-09T14:52:26.216 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2023-08-09T14:52:26.216 INFO:teuthology.orchestra.run.smithi043.stdout: { 2023-08-09T14:52:26.216 INFO:teuthology.orchestra.run.smithi043.stdout: "entity": "mgr.y", 2023-08-09T14:52:26.217 INFO:teuthology.orchestra.run.smithi043.stdout: "key": "AQCNldNk2YgzLBAAA11Q8uAMEgE/bUni+tXIKA==", 2023-08-09T14:52:26.217 INFO:teuthology.orchestra.run.smithi043.stdout: "caps": { 2023-08-09T14:52:26.217 INFO:teuthology.orchestra.run.smithi043.stdout: "mds": "allow *", 2023-08-09T14:52:26.217 INFO:teuthology.orchestra.run.smithi043.stdout: "mon": "allow profile mgr", 2023-08-09T14:52:26.217 INFO:teuthology.orchestra.run.smithi043.stdout: "osd": "allow *" 2023-08-09T14:52:26.217 INFO:teuthology.orchestra.run.smithi043.stdout: } 2023-08-09T14:52:26.218 INFO:teuthology.orchestra.run.smithi043.stdout: } 2023-08-09T14:52:26.218 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2023-08-09T14:52:26.218 INFO:teuthology.orchestra.run.smithi043.stdout:} 2023-08-09T14:52:26.219 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:26.193+0000 7f76cf59e640 1 -- 172.21.15.43:0/2083141240 >> [v2:172.21.15.203:6832/2959400181,v1:172.21.15.203:6833/2959400181] conn(0x7f76b006a170 msgr2=0x7f76b006c630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T14:52:26.220 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:26.193+0000 7f76cf59e640 1 --2- 172.21.15.43:0/2083141240 >> [v2:172.21.15.203:6832/2959400181,v1:172.21.15.203:6833/2959400181] conn(0x7f76b006a170 0x7f76b006c630 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7f76c0036000 tx=0x7f76c0030000 comp rx=0 tx=0).stop 2023-08-09T14:52:26.220 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:26.193+0000 7f76cf59e640 1 -- 172.21.15.43:0/2083141240 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f76d0149e10 msgr2=0x7f76d0163560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T14:52:26.220 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:26.193+0000 7f76cf59e640 1 --2- 172.21.15.43:0/2083141240 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f76d0149e10 0x7f76d0163560 secure :-1 s=READY pgs=2008 cs=0 l=1 rev1=1 crypto rx=0x7f76c800c440 tx=0x7f76c800c900 comp rx=0 tx=0).stop 2023-08-09T14:52:26.220 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:26.194+0000 7f76cf59e640 1 -- 172.21.15.43:0/2083141240 shutdown_connections 2023-08-09T14:52:26.220 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:26.194+0000 7f76cf59e640 1 --2- 172.21.15.43:0/2083141240 >> [v2:172.21.15.203:6832/2959400181,v1:172.21.15.203:6833/2959400181] conn(0x7f76b006a170 0x7f76b006c630 unknown :-1 s=CLOSED pgs=52 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T14:52:26.220 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:26.194+0000 7f76cf59e640 1 --2- 172.21.15.43:0/2083141240 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f76d0149e10 0x7f76d0163560 unknown :-1 s=CLOSED pgs=2008 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T14:52:26.221 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:26.194+0000 7f76cf59e640 1 --2- 172.21.15.43:0/2083141240 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f76d00c9620 0x7f76d00b2bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T14:52:26.221 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:26.194+0000 7f76cf59e640 1 --2- 172.21.15.43:0/2083141240 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f76d00c0e60 0x7f76d01f5e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T14:52:26.221 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:26.194+0000 7f76cf59e640 1 -- 172.21.15.43:0/2083141240 >> 172.21.15.43:0/2083141240 conn(0x7f76d00cde90 msgr2=0x7f76d00ad930 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-09T14:52:26.221 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:26.194+0000 7f76cf59e640 1 -- 172.21.15.43:0/2083141240 shutdown_connections 2023-08-09T14:52:26.221 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:26.194+0000 7f76cf59e640 1 -- 172.21.15.43:0/2083141240 wait complete. 2023-08-09T14:52:26.222 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-08-09T14:52:26.447 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:26.446+0000 7f51019e1640 1 Processor -- start 2023-08-09T14:52:26.448 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:26.447+0000 7f51019e1640 1 -- start start 2023-08-09T14:52:26.450 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:26.448+0000 7f51019e1640 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7f50fc142ca0 0x7f50fc0c96e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T14:52:26.450 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:26.448+0000 7f51019e1640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f50fc15ce40 0x7f50fc0cce00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T14:52:26.450 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:26.448+0000 7f51019e1640 1 -- --> v1:172.21.15.43:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f50fc143080 con 0x7f50fc0cd340 2023-08-09T14:52:26.451 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:26.448+0000 7f51019e1640 1 -- --> v2:172.21.15.43:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f50fc0cd750 con 0x7f50fc15ce40 2023-08-09T14:52:26.451 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:26.448+0000 7f51019e1640 1 -- --> v2:172.21.15.203:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f50fc06d400 con 0x7f50fc142ca0 2023-08-09T14:52:26.451 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:26.449+0000 7f50fa7fc640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f50fc15ce40 0x7f50fc0cce00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T14:52:26.451 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:26.449+0000 7f50fa7fc640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f50fc15ce40 0x7f50fc0cce00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:55396/0 (socket says 172.21.15.43:55396) 2023-08-09T14:52:26.451 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:26.449+0000 7f50fa7fc640 1 -- 172.21.15.43:0/2899537370 learned_addr learned my addr 172.21.15.43:0/2899537370 (peer_addr_for_me v2:172.21.15.43:0/0) 2023-08-09T14:52:26.451 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:26.449+0000 7f50faffd640 1 --2- 172.21.15.43:0/2899537370 >> v2:172.21.15.203:3300/0 conn(0x7f50fc142ca0 0x7f50fc0c96e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T14:52:26.451 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:26.449+0000 7f50fa7fc640 1 -- 172.21.15.43:0/2899537370 >> v2:172.21.15.203:3300/0 conn(0x7f50fc142ca0 msgr2=0x7f50fc0c96e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-09T14:52:26.452 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:26.449+0000 7f50fa7fc640 1 --2- 172.21.15.43:0/2899537370 >> v2:172.21.15.203:3300/0 conn(0x7f50fc142ca0 0x7f50fc0c96e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T14:52:26.452 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:26.449+0000 7f50fa7fc640 1 -- 172.21.15.43:0/2899537370 >> v1:172.21.15.43:6789/0 conn(0x7f50fc0cd340 legacy=0x7f50fc0cbc70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T14:52:26.452 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:26.449+0000 7f50fa7fc640 1 -- 172.21.15.43:0/2899537370 --> v2:172.21.15.43:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f50fc0c9d90 con 0x7f50fc15ce40 2023-08-09T14:52:26.453 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:26.451+0000 7f50fa7fc640 1 --2- 172.21.15.43:0/2899537370 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f50fc15ce40 0x7f50fc0cce00 secure :-1 s=READY pgs=2010 cs=0 l=1 rev1=1 crypto rx=0x7f50fc0c5120 tx=0x7f50f002b9b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=39f34389bbe1e2a2 server_cookie=0 in_seq=0 out_seq=0 2023-08-09T14:52:26.453 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:26.451+0000 7f50f9ffb640 1 -- 172.21.15.43:0/2899537370 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f50f0030870 con 0x7f50fc15ce40 2023-08-09T14:52:26.453 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:26.451+0000 7f50f9ffb640 1 -- 172.21.15.43:0/2899537370 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f50f00309d0 con 0x7f50fc15ce40 2023-08-09T14:52:26.453 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:26.451+0000 7f50f9ffb640 1 -- 172.21.15.43:0/2899537370 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f50f0030d60 con 0x7f50fc15ce40 2023-08-09T14:52:26.453 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:26.451+0000 7f51019e1640 1 -- 172.21.15.43:0/2899537370 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f50fc15ce40 msgr2=0x7f50fc0cce00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T14:52:26.454 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:26.451+0000 7f51019e1640 1 --2- 172.21.15.43:0/2899537370 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f50fc15ce40 0x7f50fc0cce00 secure :-1 s=READY pgs=2010 cs=0 l=1 rev1=1 crypto rx=0x7f50fc0c5120 tx=0x7f50f002b9b0 comp rx=0 tx=0).stop 2023-08-09T14:52:26.454 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:26.452+0000 7f51019e1640 1 -- 172.21.15.43:0/2899537370 shutdown_connections 2023-08-09T14:52:26.454 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:26.452+0000 7f51019e1640 1 --2- 172.21.15.43:0/2899537370 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f50fc15ce40 0x7f50fc0cce00 unknown :-1 s=CLOSED pgs=2010 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T14:52:26.454 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:26.452+0000 7f51019e1640 1 --2- 172.21.15.43:0/2899537370 >> v2:172.21.15.203:3300/0 conn(0x7f50fc142ca0 0x7f50fc0c96e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T14:52:26.454 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:26.452+0000 7f51019e1640 1 -- 172.21.15.43:0/2899537370 >> 172.21.15.43:0/2899537370 conn(0x7f50fc0ce0a0 msgr2=0x7f50fc0ce480 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-09T14:52:26.454 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:26.452+0000 7f51019e1640 1 -- 172.21.15.43:0/2899537370 shutdown_connections 2023-08-09T14:52:26.454 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:26.452+0000 7f51019e1640 1 -- 172.21.15.43:0/2899537370 wait complete. 2023-08-09T14:52:26.455 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:26.453+0000 7f51019e1640 1 Processor -- start 2023-08-09T14:52:26.456 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:26.454+0000 7f51019e1640 1 -- start start 2023-08-09T14:52:26.456 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:26.454+0000 7f51019e1640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f50fc0cd340 0x7f50fc1578b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T14:52:26.456 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:26.454+0000 7f51019e1640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f50fc142ca0 0x7f50fc1379f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T14:52:26.456 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:26.455+0000 7f51019e1640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f50fc15ce40 0x7f50fc0addb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T14:52:26.456 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:26.455+0000 7f51019e1640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f50fc159150 con 0x7f50fc142ca0 2023-08-09T14:52:26.456 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:26.455+0000 7f51019e1640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f50fc0c6b40 con 0x7f50fc15ce40 2023-08-09T14:52:26.457 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:26.455+0000 7f51019e1640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f50fc158c90 con 0x7f50fc0cd340 2023-08-09T14:52:26.457 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:26.455+0000 7f50fa7fc640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f50fc142ca0 0x7f50fc1379f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T14:52:26.457 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:26.455+0000 7f50fa7fc640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f50fc142ca0 0x7f50fc1379f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:55406/0 (socket says 172.21.15.43:55406) 2023-08-09T14:52:26.457 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:26.455+0000 7f50fa7fc640 1 -- 172.21.15.43:0/2692342519 learned_addr learned my addr 172.21.15.43:0/2692342519 (peer_addr_for_me v2:172.21.15.43:0/0) 2023-08-09T14:52:26.457 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:26.455+0000 7f50faffd640 1 --2- 172.21.15.43:0/2692342519 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f50fc0cd340 0x7f50fc1578b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T14:52:26.457 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:26.455+0000 7f50fa7fc640 1 -- 172.21.15.43:0/2692342519 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f50fc0cd340 msgr2=0x7f50fc1578b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T14:52:26.457 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:26.455+0000 7f50fa7fc640 1 --2- 172.21.15.43:0/2692342519 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f50fc0cd340 0x7f50fc1578b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T14:52:26.458 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:26.455+0000 7f50fa7fc640 1 -- 172.21.15.43:0/2692342519 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f50fc15ce40 msgr2=0x7f50fc0addb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T14:52:26.458 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:26.455+0000 7f50fb7fe640 1 --2- 172.21.15.43:0/2692342519 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f50fc15ce40 0x7f50fc0addb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T14:52:26.458 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:26.455+0000 7f50fa7fc640 1 --2- 172.21.15.43:0/2692342519 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f50fc15ce40 0x7f50fc0addb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T14:52:26.458 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:26.455+0000 7f50fa7fc640 1 -- 172.21.15.43:0/2692342519 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f50f00094d0 con 0x7f50fc142ca0 2023-08-09T14:52:26.458 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:26.455+0000 7f50fb7fe640 1 --2- 172.21.15.43:0/2692342519 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f50fc15ce40 0x7f50fc0addb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-08-09T14:52:26.459 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:26.456+0000 7f50fa7fc640 1 --2- 172.21.15.43:0/2692342519 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f50fc142ca0 0x7f50fc1379f0 secure :-1 s=READY pgs=2011 cs=0 l=1 rev1=1 crypto rx=0x7f50f002b9f0 tx=0x7f50f0004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-09T14:52:26.459 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:26.456+0000 7f50e3fff640 1 -- 172.21.15.43:0/2692342519 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f50f0039d60 con 0x7f50fc142ca0 2023-08-09T14:52:26.459 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:26.456+0000 7f50e3fff640 1 -- 172.21.15.43:0/2692342519 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f50f0043400 con 0x7f50fc142ca0 2023-08-09T14:52:26.459 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:26.456+0000 7f50e3fff640 1 -- 172.21.15.43:0/2692342519 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f50f00437b0 con 0x7f50fc142ca0 2023-08-09T14:52:26.460 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:26.456+0000 7f51019e1640 1 -- 172.21.15.43:0/2692342519 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f50fc158110 con 0x7f50fc142ca0 2023-08-09T14:52:26.460 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:26.456+0000 7f51019e1640 1 -- 172.21.15.43:0/2692342519 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f50fc1586d0 con 0x7f50fc142ca0 2023-08-09T14:52:26.460 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:26.458+0000 7f51019e1640 1 -- 172.21.15.43:0/2692342519 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f50fc0c6cd0 con 0x7f50fc142ca0 2023-08-09T14:52:26.460 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:26.459+0000 7f50e3fff640 1 -- 172.21.15.43:0/2692342519 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 88) v1 ==== 88443+0+0 (secure 0 0 0) 0x7f50f0043910 con 0x7f50fc142ca0 2023-08-09T14:52:26.461 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:26.460+0000 7f50e3fff640 1 --2- 172.21.15.43:0/2692342519 >> [v2:172.21.15.203:6832/2959400181,v1:172.21.15.203:6833/2959400181] conn(0x7f50e806a170 0x7f50e806c630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T14:52:26.461 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:26.460+0000 7f50e3fff640 1 -- 172.21.15.43:0/2692342519 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(182..182 src has 1..182) v4 ==== 5574+0+0 (secure 0 0 0) 0x7f50f003e070 con 0x7f50fc142ca0 2023-08-09T14:52:26.461 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:26.460+0000 7f50faffd640 1 --2- 172.21.15.43:0/2692342519 >> [v2:172.21.15.203:6832/2959400181,v1:172.21.15.203:6833/2959400181] conn(0x7f50e806a170 0x7f50e806c630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T14:52:26.462 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:26.462+0000 7f50faffd640 1 --2- 172.21.15.43:0/2692342519 >> [v2:172.21.15.203:6832/2959400181,v1:172.21.15.203:6833/2959400181] conn(0x7f50e806a170 0x7f50e806c630 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f50ec006fd0 tx=0x7f50ec008040 comp rx=0 tx=0).ready entity=mgr.7223 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-09T14:52:26.464 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:26.464+0000 7f50e3fff640 1 -- 172.21.15.43:0/2692342519 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178066 (secure 0 0 0) 0x7f50fc0c6cd0 con 0x7f50fc142ca0 2023-08-09T14:52:26.664 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:26.663+0000 7f51019e1640 1 -- 172.21.15.43:0/2692342519 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f50fc0c6c80 con 0x7f50fc142ca0 2023-08-09T14:52:26.665 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:26.664+0000 7f50e3fff640 1 -- 172.21.15.43:0/2692342519 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v182) v1 ==== 74+0+14854 (secure 0 0 0) 0x7f50f0034090 con 0x7f50fc142ca0 2023-08-09T14:52:26.666 INFO:teuthology.orchestra.run.smithi043.stdout: 2023-08-09T14:52:26.666 INFO:teuthology.orchestra.run.smithi043.stdout:{"epoch":182,"fsid":"daac87ee-1337-4250-a955-aea4e408a39b","created":"2023-08-09T13:33:41.701198+0000","modified":"2023-08-09T14:52:25.645117+0000","last_up_change":"2023-08-09T13:33:53.783453+0000","last_in_change":"2023-08-09T13:33:45.148904+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":21,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":"rbd","create_time":"2023-08-09T13:33:54.365137+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":".mgr","create_time":"2023-08-09T13:33:54.831803+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"cbd49ecd-e896-4fbd-864e-5c37a829b165","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":170,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6800","nonce":502667335},{"type":"v1","addr":"172.21.15.43:6801","nonce":502667335}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6802","nonce":502667335},{"type":"v1","addr":"172.21.15.43:6803","nonce":502667335}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6806","nonce":502667335},{"type":"v1","addr":"172.21.15.43:6807","nonce":502667335}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6804","nonce":502667335},{"type":"v1","addr":"172.21.15.43:6805","nonce":502667335}]},"public_addr":"172.21.15.43:6801/502667335","cluster_addr":"172.21.15.43:6803/502667335","heartbeat_back_addr":"172.21.15.43:6807/502667335","heartbeat_front_addr":"172.21.15.43:6805/502667335","state":["exists","up"]},{"osd":1,"uuid":"75c7e0fe-1ec7-4fe8-a2f2-3661a65b8500","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":170,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6808","nonce":3289134610},{"type":"v1","addr":"172.21.15.43:6809","nonce":3289134610}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6810","nonce":3289134610},{"type":"v1","addr":"172.21.15.43:6812","nonce":3289134610}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6821","nonce":3289134610},{"type":"v1","addr":"172.21.15.43:6824","nonce":3289134610}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6815","nonce":3289134610},{"type":"v1","addr":"172.21.15.43:6818","nonce":3289134610}]},"public_addr":"172.21.15.43:6809/3289134610","cluster_addr":"172.21.15.43:6812/3289134610","heartbeat_back_addr":"172.21.15.43:6824/3289134610","heartbeat_front_addr":"172.21.15.43:6818/3289134610","state":["exists","up"]},{"osd":2,"uuid":"bb5c41e9-7f5f-4125-9b01-3bb8bfc2ea83","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":170,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6811","nonce":4079709736},{"type":"v1","addr":"172.21.15.43:6814","nonce":4079709736}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6817","nonce":4079709736},{"type":"v1","addr":"172.21.15.43:6820","nonce":4079709736}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6829","nonce":4079709736},{"type":"v1","addr":"172.21.15.43:6831","nonce":4079709736}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6823","nonce":4079709736},{"type":"v1","addr":"172.21.15.43:6826","nonce":4079709736}]},"public_addr":"172.21.15.43:6814/4079709736","cluster_addr":"172.21.15.43:6820/4079709736","heartbeat_back_addr":"172.21.15.43:6831/4079709736","heartbeat_front_addr":"172.21.15.43:6826/4079709736","state":["exists","up"]},{"osd":3,"uuid":"bfe4c8b4-694a-465c-87e6-2b2aab852613","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":170,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6813","nonce":1709538759},{"type":"v1","addr":"172.21.15.43:6816","nonce":1709538759}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6819","nonce":1709538759},{"type":"v1","addr":"172.21.15.43:6822","nonce":1709538759}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6828","nonce":1709538759},{"type":"v1","addr":"172.21.15.43:6830","nonce":1709538759}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6825","nonce":1709538759},{"type":"v1","addr":"172.21.15.43:6827","nonce":1709538759}]},"public_addr":"172.21.15.43:6816/1709538759","cluster_addr":"172.21.15.43:6822/1709538759","heartbeat_back_addr":"172.21.15.43:6830/1709538759","heartbeat_front_addr":"172.21.15.43:6827/1709538759","state":["exists","up"]},{"osd":4,"uuid":"fa2e8a77-aba1-4756-853e-0d174a5bdeba","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":170,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6800","nonce":4196866481},{"type":"v1","addr":"172.21.15.203:6801","nonce":4196866481}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6802","nonce":4196866481},{"type":"v1","addr":"172.21.15.203:6803","nonce":4196866481}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6806","nonce":4196866481},{"type":"v1","addr":"172.21.15.203:6807","nonce":4196866481}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6804","nonce":4196866481},{"type":"v1","addr":"172.21.15.203:6805","nonce":4196866481}]},"public_addr":"172.21.15.203:6801/4196866481","cluster_addr":"172.21.15.203:6803/4196866481","heartbeat_back_addr":"172.21.15.203:6807/4196866481","heartbeat_front_addr":"172.21.15.203:6805/4196866481","state":["exists","up"]},{"osd":5,"uuid":"81811e4b-4d24-4f51-a82c-f1d700a2a00d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":170,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6816","nonce":3059290081},{"type":"v1","addr":"172.21.15.203:6817","nonce":3059290081}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6818","nonce":3059290081},{"type":"v1","addr":"172.21.15.203:6819","nonce":3059290081}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6822","nonce":3059290081},{"type":"v1","addr":"172.21.15.203:6823","nonce":3059290081}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6820","nonce":3059290081},{"type":"v1","addr":"172.21.15.203:6821","nonce":3059290081}]},"public_addr":"172.21.15.203:6817/3059290081","cluster_addr":"172.21.15.203:6819/3059290081","heartbeat_back_addr":"172.21.15.203:6823/3059290081","heartbeat_front_addr":"172.21.15.203:6821/3059290081","state":["exists","up"]},{"osd":6,"uuid":"307bac4d-6a5b-443d-af75-56d695d7674f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":170,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6824","nonce":512155569},{"type":"v1","addr":"172.21.15.203:6825","nonce":512155569}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6826","nonce":512155569},{"type":"v1","addr":"172.21.15.203:6827","nonce":512155569}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6830","nonce":512155569},{"type":"v1","addr":"172.21.15.203:6831","nonce":512155569}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6828","nonce":512155569},{"type":"v1","addr":"172.21.15.203:6829","nonce":512155569}]},"public_addr":"172.21.15.203:6825/512155569","cluster_addr":"172.21.15.203:6827/512155569","heartbeat_back_addr":"172.21.15.203:6831/512155569","heartbeat_front_addr":"172.21.15.203:6829/512155569","state":["exists","up"]},{"osd":7,"uuid":"a95dc7ca-9f1b-4295-af70-68a22a2d7d5e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":170,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6808","nonce":3331450438},{"type":"v1","addr":"172.21.15.203:6809","nonce":3331450438}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6810","nonce":3331450438},{"type":"v1","addr":"172.21.15.203:6811","nonce":3331450438}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6814","nonce":3331450438},{"type":"v1","addr":"172.21.15.203:6815","nonce":3331450438}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6812","nonce":3331450438},{"type":"v1","addr":"172.21.15.203:6813","nonce":3331450438}]},"public_addr":"172.21.15.203:6809/3331450438","cluster_addr":"172.21.15.203:6811/3331450438","heartbeat_back_addr":"172.21.15.203:6815/3331450438","heartbeat_front_addr":"172.21.15.203:6813/3331450438","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-09T13:33:50.547652+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-09T13:33:50.589469+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-09T13:33:50.561724+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-09T13:33:50.599650+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-09T13:33:50.411855+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-09T13:33:50.219681+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-09T13:33:50.504664+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-09T13:33:50.441248+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":7,"snaps":[{"begin":104,"length":1},{"begin":221,"length":1}]}],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-08-09T14:52:26.668 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:26.667+0000 7f51019e1640 1 -- 172.21.15.43:0/2692342519 >> [v2:172.21.15.203:6832/2959400181,v1:172.21.15.203:6833/2959400181] conn(0x7f50e806a170 msgr2=0x7f50e806c630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T14:52:26.668 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:26.667+0000 7f51019e1640 1 --2- 172.21.15.43:0/2692342519 >> [v2:172.21.15.203:6832/2959400181,v1:172.21.15.203:6833/2959400181] conn(0x7f50e806a170 0x7f50e806c630 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f50ec006fd0 tx=0x7f50ec008040 comp rx=0 tx=0).stop 2023-08-09T14:52:26.668 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:26.667+0000 7f51019e1640 1 -- 172.21.15.43:0/2692342519 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f50fc142ca0 msgr2=0x7f50fc1379f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T14:52:26.668 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:26.667+0000 7f51019e1640 1 --2- 172.21.15.43:0/2692342519 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f50fc142ca0 0x7f50fc1379f0 secure :-1 s=READY pgs=2011 cs=0 l=1 rev1=1 crypto rx=0x7f50f002b9f0 tx=0x7f50f0004730 comp rx=0 tx=0).stop 2023-08-09T14:52:26.669 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:26.667+0000 7f51019e1640 1 -- 172.21.15.43:0/2692342519 shutdown_connections 2023-08-09T14:52:26.669 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:26.667+0000 7f51019e1640 1 --2- 172.21.15.43:0/2692342519 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f50fc15ce40 0x7f50fc0addb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T14:52:26.669 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:26.667+0000 7f51019e1640 1 --2- 172.21.15.43:0/2692342519 >> [v2:172.21.15.203:6832/2959400181,v1:172.21.15.203:6833/2959400181] conn(0x7f50e806a170 0x7f50e806c630 unknown :-1 s=CLOSED pgs=53 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T14:52:26.669 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:26.667+0000 7f51019e1640 1 --2- 172.21.15.43:0/2692342519 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f50fc142ca0 0x7f50fc1379f0 unknown :-1 s=CLOSED pgs=2011 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T14:52:26.669 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:26.667+0000 7f51019e1640 1 --2- 172.21.15.43:0/2692342519 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f50fc0cd340 0x7f50fc1578b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T14:52:26.669 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:26.667+0000 7f51019e1640 1 -- 172.21.15.43:0/2692342519 >> 172.21.15.43:0/2692342519 conn(0x7f50fc0ce0a0 msgr2=0x7f50fc141ee0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-09T14:52:26.669 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:26.668+0000 7f51019e1640 1 -- 172.21.15.43:0/2692342519 shutdown_connections 2023-08-09T14:52:26.670 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:26.668+0000 7f51019e1640 1 -- 172.21.15.43:0/2692342519 wait complete. 2023-08-09T14:52:26.687 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': 'rbd', 'create_time': '2023-08-09T13:33:54.365137+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}, 'read_balance': {'score_acting': 2, 'score_stable': 2, 'optimal_score': 1, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': '.mgr', 'create_time': '2023-08-09T13:33:54.831803+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-08-09T14:52:26.688 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get rbd pg_num 2023-08-09T14:52:26.919 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:26.918+0000 7fab0bfff640 1 Processor -- start 2023-08-09T14:52:26.919 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:26.918+0000 7fab0bfff640 1 -- start start 2023-08-09T14:52:26.920 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:26.920+0000 7fab0bfff640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fab0c070ef0 0x7fab0c0ab8d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T14:52:26.921 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:26.920+0000 7fab0bfff640 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7fab0c0cc0b0 0x7fab0c06f310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T14:52:26.921 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:26.920+0000 7fab0bfff640 1 -- --> v1:172.21.15.203:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fab0c0acfa0 con 0x7fab0c0abe10 2023-08-09T14:52:26.921 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:26.920+0000 7fab0bfff640 1 -- --> v2:172.21.15.203:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fab0c15c940 con 0x7fab0c0cc0b0 2023-08-09T14:52:26.921 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:26.920+0000 7fab0bfff640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fab0c0ac390 con 0x7fab0c070ef0 2023-08-09T14:52:26.922 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:26.921+0000 7fab0affd640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fab0c070ef0 0x7fab0c0ab8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T14:52:26.922 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:26.921+0000 7fab0affd640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fab0c070ef0 0x7fab0c0ab8d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.203:3301/0 says I am v2:172.21.15.43:34702/0 (socket says 172.21.15.43:34702) 2023-08-09T14:52:26.922 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:26.921+0000 7fab0affd640 1 -- 172.21.15.43:0/1068926561 learned_addr learned my addr 172.21.15.43:0/1068926561 (peer_addr_for_me v2:172.21.15.43:0/0) 2023-08-09T14:52:26.922 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:26.921+0000 7fab0b7fe640 1 --2- 172.21.15.43:0/1068926561 >> v2:172.21.15.203:3300/0 conn(0x7fab0c0cc0b0 0x7fab0c06f310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T14:52:26.923 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:26.922+0000 7fab0affd640 1 -- 172.21.15.43:0/1068926561 >> v2:172.21.15.203:3300/0 conn(0x7fab0c0cc0b0 msgr2=0x7fab0c06f310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-09T14:52:26.923 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:26.922+0000 7fab0affd640 1 --2- 172.21.15.43:0/1068926561 >> v2:172.21.15.203:3300/0 conn(0x7fab0c0cc0b0 0x7fab0c06f310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T14:52:26.923 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:26.922+0000 7fab0affd640 1 -- 172.21.15.43:0/1068926561 >> v1:172.21.15.203:6789/0 conn(0x7fab0c0abe10 legacy=0x7fab0c0cb9a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T14:52:26.923 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:26.922+0000 7fab0affd640 1 -- 172.21.15.43:0/1068926561 --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fab0c0ac670 con 0x7fab0c070ef0 2023-08-09T14:52:26.923 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:26.923+0000 7fab0affd640 1 --2- 172.21.15.43:0/1068926561 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fab0c070ef0 0x7fab0c0ab8d0 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7faafc009870 tx=0x7faafc02ba20 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b714e7351e5dbd4e server_cookie=0 in_seq=0 out_seq=0 2023-08-09T14:52:26.924 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:26.924+0000 7fab09ffb640 1 -- 172.21.15.43:0/1068926561 <== mon.2 v2:172.21.15.203:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faafc02f840 con 0x7fab0c070ef0 2023-08-09T14:52:26.924 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:26.924+0000 7fab09ffb640 1 -- 172.21.15.43:0/1068926561 <== mon.2 v2:172.21.15.203:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7faafc02f9a0 con 0x7fab0c070ef0 2023-08-09T14:52:26.924 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:26.924+0000 7fab09ffb640 1 -- 172.21.15.43:0/1068926561 <== mon.2 v2:172.21.15.203:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faafc02fd30 con 0x7fab0c070ef0 2023-08-09T14:52:26.925 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:26.924+0000 7fab0bfff640 1 -- 172.21.15.43:0/1068926561 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fab0c070ef0 msgr2=0x7fab0c0ab8d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T14:52:26.925 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:26.924+0000 7fab0bfff640 1 --2- 172.21.15.43:0/1068926561 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fab0c070ef0 0x7fab0c0ab8d0 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7faafc009870 tx=0x7faafc02ba20 comp rx=0 tx=0).stop 2023-08-09T14:52:26.925 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:26.925+0000 7fab0bfff640 1 -- 172.21.15.43:0/1068926561 shutdown_connections 2023-08-09T14:52:26.926 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:26.925+0000 7fab0bfff640 1 --2- 172.21.15.43:0/1068926561 >> v2:172.21.15.203:3300/0 conn(0x7fab0c0cc0b0 0x7fab0c06f310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T14:52:26.926 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:26.925+0000 7fab0bfff640 1 --2- 172.21.15.43:0/1068926561 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fab0c070ef0 0x7fab0c0ab8d0 unknown :-1 s=CLOSED pgs=277 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T14:52:26.926 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:26.925+0000 7fab0bfff640 1 -- 172.21.15.43:0/1068926561 >> 172.21.15.43:0/1068926561 conn(0x7fab0c0cddd0 msgr2=0x7fab0c0ce1e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-09T14:52:26.926 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:26.925+0000 7fab0bfff640 1 -- 172.21.15.43:0/1068926561 shutdown_connections 2023-08-09T14:52:26.926 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:26.925+0000 7fab0bfff640 1 -- 172.21.15.43:0/1068926561 wait complete. 2023-08-09T14:52:26.927 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:26.927+0000 7fab0bfff640 1 Processor -- start 2023-08-09T14:52:26.928 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:26.927+0000 7fab0bfff640 1 -- start start 2023-08-09T14:52:26.929 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:26.929+0000 7fab0bfff640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fab0c070ef0 0x7fab0c157e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T14:52:26.929 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:26.929+0000 7fab0bfff640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fab0c0abe10 0x7fab0c15dcd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T14:52:26.929 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:26.929+0000 7fab0bfff640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fab0c0cc0b0 0x7fab0c0add40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T14:52:26.929 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:26.929+0000 7fab0bfff640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fab0c15d390 con 0x7fab0c0cc0b0 2023-08-09T14:52:26.930 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:26.929+0000 7fab0bfff640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fab0c15ea50 con 0x7fab0c0abe10 2023-08-09T14:52:26.930 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:26.929+0000 7fab0bfff640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fab0c1583a0 con 0x7fab0c070ef0 2023-08-09T14:52:26.931 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:26.929+0000 7fab0b7fe640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fab0c0cc0b0 0x7fab0c0add40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T14:52:26.931 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:26.929+0000 7fab0b7fe640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fab0c0cc0b0 0x7fab0c0add40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:55408/0 (socket says 172.21.15.43:55408) 2023-08-09T14:52:26.931 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:26.929+0000 7fab0b7fe640 1 -- 172.21.15.43:0/4281227506 learned_addr learned my addr 172.21.15.43:0/4281227506 (peer_addr_for_me v2:172.21.15.43:0/0) 2023-08-09T14:52:26.931 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:26.929+0000 7fab0affd640 1 --2- 172.21.15.43:0/4281227506 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fab0c070ef0 0x7fab0c157e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T14:52:26.931 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:26.929+0000 7fab0a7fc640 1 --2- 172.21.15.43:0/4281227506 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fab0c0abe10 0x7fab0c15dcd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T14:52:26.932 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:26.930+0000 7fab0b7fe640 1 -- 172.21.15.43:0/4281227506 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fab0c070ef0 msgr2=0x7fab0c157e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T14:52:26.932 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:26.930+0000 7fab0b7fe640 1 --2- 172.21.15.43:0/4281227506 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fab0c070ef0 0x7fab0c157e60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T14:52:26.932 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:26.930+0000 7fab0b7fe640 1 -- 172.21.15.43:0/4281227506 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fab0c0abe10 msgr2=0x7fab0c15dcd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T14:52:26.932 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:26.930+0000 7fab0b7fe640 1 --2- 172.21.15.43:0/4281227506 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fab0c0abe10 0x7fab0c15dcd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T14:52:26.932 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:26.930+0000 7fab0b7fe640 1 -- 172.21.15.43:0/4281227506 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faafc0094d0 con 0x7fab0c0cc0b0 2023-08-09T14:52:26.932 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:26.930+0000 7fab0b7fe640 1 --2- 172.21.15.43:0/4281227506 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fab0c0cc0b0 0x7fab0c0add40 secure :-1 s=READY pgs=2012 cs=0 l=1 rev1=1 crypto rx=0x7fab0400efc0 tx=0x7fab0400c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-09T14:52:26.932 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:26.930+0000 7faaf3fff640 1 -- 172.21.15.43:0/4281227506 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fab04009ca0 con 0x7fab0c0cc0b0 2023-08-09T14:52:26.933 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:26.930+0000 7faaf3fff640 1 -- 172.21.15.43:0/4281227506 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fab0400f040 con 0x7fab0c0cc0b0 2023-08-09T14:52:26.933 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:26.930+0000 7faaf3fff640 1 -- 172.21.15.43:0/4281227506 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fab040146c0 con 0x7fab0c0cc0b0 2023-08-09T14:52:26.933 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:26.930+0000 7fab0bfff640 1 -- 172.21.15.43:0/4281227506 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fab0c15e770 con 0x7fab0c0cc0b0 2023-08-09T14:52:26.933 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:26.931+0000 7fab0bfff640 1 -- 172.21.15.43:0/4281227506 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fab0c15cfb0 con 0x7fab0c0cc0b0 2023-08-09T14:52:26.933 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:26.932+0000 7fab0bfff640 1 -- 172.21.15.43:0/4281227506 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fab0c15cb30 con 0x7fab0c0cc0b0 2023-08-09T14:52:26.934 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:26.933+0000 7faaf3fff640 1 -- 172.21.15.43:0/4281227506 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 88) v1 ==== 88443+0+0 (secure 0 0 0) 0x7fab04014820 con 0x7fab0c0cc0b0 2023-08-09T14:52:26.934 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:26.934+0000 7faaf3fff640 1 --2- 172.21.15.43:0/4281227506 >> [v2:172.21.15.203:6832/2959400181,v1:172.21.15.203:6833/2959400181] conn(0x7faaf806a170 0x7faaf806c630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T14:52:26.934 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:26.934+0000 7faaf3fff640 1 -- 172.21.15.43:0/4281227506 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(182..182 src has 1..182) v4 ==== 5574+0+0 (secure 0 0 0) 0x7fab04089570 con 0x7fab0c0cc0b0 2023-08-09T14:52:26.935 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:26.934+0000 7fab0affd640 1 --2- 172.21.15.43:0/4281227506 >> [v2:172.21.15.203:6832/2959400181,v1:172.21.15.203:6833/2959400181] conn(0x7faaf806a170 0x7faaf806c630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T14:52:26.936 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:26.936+0000 7fab0affd640 1 --2- 172.21.15.43:0/4281227506 >> [v2:172.21.15.203:6832/2959400181,v1:172.21.15.203:6833/2959400181] conn(0x7faaf806a170 0x7faaf806c630 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7faafc036000 tx=0x7faafc008030 comp rx=0 tx=0).ready entity=mgr.7223 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-09T14:52:26.938 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:26.938+0000 7faaf3fff640 1 -- 172.21.15.43:0/4281227506 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178066 (secure 0 0 0) 0x7fab0c15cb30 con 0x7fab0c0cc0b0 2023-08-09T14:52:27.129 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:27.129+0000 7fab0bfff640 1 -- 172.21.15.43:0/4281227506 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"} v 0) v1 -- 0x7fab0c15d4d0 con 0x7fab0c0cc0b0 2023-08-09T14:52:27.130 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:27.130+0000 7faaf3fff640 1 -- 172.21.15.43:0/4281227506 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"}]=0 v182) v1 ==== 92+0+10 (secure 0 0 0) 0x7fab0c15d4d0 con 0x7fab0c0cc0b0 2023-08-09T14:52:27.130 INFO:teuthology.orchestra.run.smithi043.stdout:pg_num: 8 2023-08-09T14:52:27.133 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:27.132+0000 7fab0bfff640 1 -- 172.21.15.43:0/4281227506 >> [v2:172.21.15.203:6832/2959400181,v1:172.21.15.203:6833/2959400181] conn(0x7faaf806a170 msgr2=0x7faaf806c630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T14:52:27.133 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:27.132+0000 7fab0bfff640 1 --2- 172.21.15.43:0/4281227506 >> [v2:172.21.15.203:6832/2959400181,v1:172.21.15.203:6833/2959400181] conn(0x7faaf806a170 0x7faaf806c630 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7faafc036000 tx=0x7faafc008030 comp rx=0 tx=0).stop 2023-08-09T14:52:27.134 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:27.132+0000 7fab0bfff640 1 -- 172.21.15.43:0/4281227506 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fab0c0cc0b0 msgr2=0x7fab0c0add40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T14:52:27.134 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:27.132+0000 7fab0bfff640 1 --2- 172.21.15.43:0/4281227506 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fab0c0cc0b0 0x7fab0c0add40 secure :-1 s=READY pgs=2012 cs=0 l=1 rev1=1 crypto rx=0x7fab0400efc0 tx=0x7fab0400c490 comp rx=0 tx=0).stop 2023-08-09T14:52:27.134 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:27.133+0000 7fab0bfff640 1 -- 172.21.15.43:0/4281227506 shutdown_connections 2023-08-09T14:52:27.134 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:27.133+0000 7fab0bfff640 1 --2- 172.21.15.43:0/4281227506 >> [v2:172.21.15.203:6832/2959400181,v1:172.21.15.203:6833/2959400181] conn(0x7faaf806a170 0x7faaf806c630 unknown :-1 s=CLOSED pgs=54 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T14:52:27.134 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:27.133+0000 7fab0bfff640 1 --2- 172.21.15.43:0/4281227506 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fab0c0cc0b0 0x7fab0c0add40 unknown :-1 s=CLOSED pgs=2012 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T14:52:27.134 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:27.133+0000 7fab0bfff640 1 --2- 172.21.15.43:0/4281227506 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fab0c0abe10 0x7fab0c15dcd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T14:52:27.134 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:27.133+0000 7fab0bfff640 1 --2- 172.21.15.43:0/4281227506 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fab0c070ef0 0x7fab0c157e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T14:52:27.135 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:27.133+0000 7fab0bfff640 1 -- 172.21.15.43:0/4281227506 >> 172.21.15.43:0/4281227506 conn(0x7fab0c0cddd0 msgr2=0x7fab0c160600 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-09T14:52:27.135 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:27.133+0000 7fab0bfff640 1 -- 172.21.15.43:0/4281227506 shutdown_connections 2023-08-09T14:52:27.135 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:27.133+0000 7fab0bfff640 1 -- 172.21.15.43:0/4281227506 wait complete. 2023-08-09T14:52:27.152 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-08-09T14:52:27.372 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:27.371+0000 7f40cd2da640 1 Processor -- start 2023-08-09T14:52:27.372 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:27.372+0000 7f40cd2da640 1 -- start start 2023-08-09T14:52:27.374 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:27.373+0000 7f40cd2da640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f40c80cb2d0 0x7f40c815b380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T14:52:27.374 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:27.373+0000 7f40cd2da640 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7f40c80ce8d0 0x7f40c80c9f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T14:52:27.374 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:27.374+0000 7f40cd2da640 1 -- --> v1:172.21.15.203:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f40c815aef0 con 0x7f40c815df60 2023-08-09T14:52:27.374 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:27.374+0000 7f40cd2da640 1 -- --> v2:172.21.15.203:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f40c815a970 con 0x7f40c80ce8d0 2023-08-09T14:52:27.375 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:27.374+0000 7f40cd2da640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f40c815b8c0 con 0x7f40c80cb2d0 2023-08-09T14:52:27.376 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:27.374+0000 7f40c6ffd640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f40c80cb2d0 0x7f40c815b380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T14:52:27.376 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:27.374+0000 7f40c6ffd640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f40c80cb2d0 0x7f40c815b380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.203:3301/0 says I am v2:172.21.15.43:34726/0 (socket says 172.21.15.43:34726) 2023-08-09T14:52:27.376 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:27.374+0000 7f40c6ffd640 1 -- 172.21.15.43:0/2003460722 learned_addr learned my addr 172.21.15.43:0/2003460722 (peer_addr_for_me v2:172.21.15.43:0/0) 2023-08-09T14:52:27.376 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:27.375+0000 7f40c77fe640 1 --2- 172.21.15.43:0/2003460722 >> v2:172.21.15.203:3300/0 conn(0x7f40c80ce8d0 0x7f40c80c9f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T14:52:27.376 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:27.375+0000 7f40c6ffd640 1 -- 172.21.15.43:0/2003460722 >> v2:172.21.15.203:3300/0 conn(0x7f40c80ce8d0 msgr2=0x7f40c80c9f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-09T14:52:27.377 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:27.375+0000 7f40c6ffd640 1 --2- 172.21.15.43:0/2003460722 >> v2:172.21.15.203:3300/0 conn(0x7f40c80ce8d0 0x7f40c80c9f90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T14:52:27.377 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:27.375+0000 7f40c6ffd640 1 -- 172.21.15.43:0/2003460722 >> v1:172.21.15.203:6789/0 conn(0x7f40c815df60 legacy=0x7f40c80ce1c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T14:52:27.377 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:27.375+0000 7f40c6ffd640 1 -- 172.21.15.43:0/2003460722 --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f40c815e940 con 0x7f40c80cb2d0 2023-08-09T14:52:27.377 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:27.376+0000 7f40c6ffd640 1 --2- 172.21.15.43:0/2003460722 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f40c80cb2d0 0x7f40c815b380 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7f40b8009870 tx=0x7f40b802b980 comp rx=0 tx=0).ready entity=mon.2 client_cookie=4f4924065895154e server_cookie=0 in_seq=0 out_seq=0 2023-08-09T14:52:27.378 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:27.377+0000 7f40c67fc640 1 -- 172.21.15.43:0/2003460722 <== mon.2 v2:172.21.15.203:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f40b8030840 con 0x7f40c80cb2d0 2023-08-09T14:52:27.378 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:27.377+0000 7f40c67fc640 1 -- 172.21.15.43:0/2003460722 <== mon.2 v2:172.21.15.203:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f40b80309a0 con 0x7f40c80cb2d0 2023-08-09T14:52:27.378 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:27.377+0000 7f40c67fc640 1 -- 172.21.15.43:0/2003460722 <== mon.2 v2:172.21.15.203:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f40b8030d30 con 0x7f40c80cb2d0 2023-08-09T14:52:27.378 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:27.378+0000 7f40cd2da640 1 -- 172.21.15.43:0/2003460722 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f40c80cb2d0 msgr2=0x7f40c815b380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T14:52:27.379 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:27.378+0000 7f40cd2da640 1 --2- 172.21.15.43:0/2003460722 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f40c80cb2d0 0x7f40c815b380 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7f40b8009870 tx=0x7f40b802b980 comp rx=0 tx=0).stop 2023-08-09T14:52:27.379 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:27.378+0000 7f40cd2da640 1 -- 172.21.15.43:0/2003460722 shutdown_connections 2023-08-09T14:52:27.379 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:27.378+0000 7f40cd2da640 1 --2- 172.21.15.43:0/2003460722 >> v2:172.21.15.203:3300/0 conn(0x7f40c80ce8d0 0x7f40c80c9f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T14:52:27.379 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:27.378+0000 7f40cd2da640 1 --2- 172.21.15.43:0/2003460722 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f40c80cb2d0 0x7f40c815b380 unknown :-1 s=CLOSED pgs=278 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T14:52:27.379 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:27.378+0000 7f40cd2da640 1 -- 172.21.15.43:0/2003460722 >> 172.21.15.43:0/2003460722 conn(0x7f40c80aff50 msgr2=0x7f40c80b0360 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-09T14:52:27.379 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:27.378+0000 7f40cd2da640 1 -- 172.21.15.43:0/2003460722 shutdown_connections 2023-08-09T14:52:27.379 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:27.378+0000 7f40cd2da640 1 -- 172.21.15.43:0/2003460722 wait complete. 2023-08-09T14:52:27.380 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:27.380+0000 7f40cd2da640 1 Processor -- start 2023-08-09T14:52:27.381 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:27.381+0000 7f40cd2da640 1 -- start start 2023-08-09T14:52:27.382 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:27.382+0000 7f40cd2da640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f40c80cb2d0 0x7f40c81f2a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T14:52:27.383 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:27.382+0000 7f40cd2da640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f40c80ce8d0 0x7f40c8153630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T14:52:27.383 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:27.382+0000 7f40cd2da640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f40c815df60 0x7f40c80c9490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T14:52:27.383 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:27.382+0000 7f40cd2da640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f40c81f5dc0 con 0x7f40c80cb2d0 2023-08-09T14:52:27.383 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:27.382+0000 7f40cd2da640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f40c8155d50 con 0x7f40c80ce8d0 2023-08-09T14:52:27.383 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:27.382+0000 7f40cd2da640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f40c8156030 con 0x7f40c815df60 2023-08-09T14:52:27.383 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:27.382+0000 7f40c6ffd640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f40c80cb2d0 0x7f40c81f2a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T14:52:27.384 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:27.382+0000 7f40c6ffd640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f40c80cb2d0 0x7f40c81f2a30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:55412/0 (socket says 172.21.15.43:55412) 2023-08-09T14:52:27.384 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:27.382+0000 7f40c6ffd640 1 -- 172.21.15.43:0/2185760534 learned_addr learned my addr 172.21.15.43:0/2185760534 (peer_addr_for_me v2:172.21.15.43:0/0) 2023-08-09T14:52:27.384 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:27.383+0000 7f40c6ffd640 1 -- 172.21.15.43:0/2185760534 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f40c815df60 msgr2=0x7f40c80c9490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T14:52:27.385 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:27.383+0000 7f40c6ffd640 1 --2- 172.21.15.43:0/2185760534 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f40c815df60 0x7f40c80c9490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T14:52:27.385 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:27.383+0000 7f40c6ffd640 1 -- 172.21.15.43:0/2185760534 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f40c80ce8d0 msgr2=0x7f40c8153630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T14:52:27.385 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:27.383+0000 7f40c6ffd640 1 --2- 172.21.15.43:0/2185760534 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f40c80ce8d0 0x7f40c8153630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T14:52:27.385 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:27.383+0000 7f40c6ffd640 1 -- 172.21.15.43:0/2185760534 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f40b80094d0 con 0x7f40c80cb2d0 2023-08-09T14:52:27.385 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:27.383+0000 7f40c6ffd640 1 --2- 172.21.15.43:0/2185760534 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f40c80cb2d0 0x7f40c81f2a30 secure :-1 s=READY pgs=2013 cs=0 l=1 rev1=1 crypto rx=0x7f40b8000c00 tx=0x7f40b8004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-09T14:52:27.385 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:27.383+0000 7f40bffff640 1 -- 172.21.15.43:0/2185760534 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f40b8039c90 con 0x7f40c80cb2d0 2023-08-09T14:52:27.386 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:27.383+0000 7f40bffff640 1 -- 172.21.15.43:0/2185760534 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f40b8039df0 con 0x7f40c80cb2d0 2023-08-09T14:52:27.386 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:27.383+0000 7f40bffff640 1 -- 172.21.15.43:0/2185760534 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f40b8043640 con 0x7f40c80cb2d0 2023-08-09T14:52:27.386 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:27.383+0000 7f40cd2da640 1 -- 172.21.15.43:0/2185760534 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f40c8156310 con 0x7f40c80cb2d0 2023-08-09T14:52:27.386 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:27.384+0000 7f40cd2da640 1 -- 172.21.15.43:0/2185760534 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f40c81f2f70 con 0x7f40c80cb2d0 2023-08-09T14:52:27.387 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:27.385+0000 7f40cd2da640 1 -- 172.21.15.43:0/2185760534 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f40c8156830 con 0x7f40c80cb2d0 2023-08-09T14:52:27.387 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:27.385+0000 7f40bffff640 1 -- 172.21.15.43:0/2185760534 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 88) v1 ==== 88443+0+0 (secure 0 0 0) 0x7f40b80437a0 con 0x7f40c80cb2d0 2023-08-09T14:52:27.387 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:27.386+0000 7f40bffff640 1 --2- 172.21.15.43:0/2185760534 >> [v2:172.21.15.203:6832/2959400181,v1:172.21.15.203:6833/2959400181] conn(0x7f40a806a1a0 0x7f40a806c660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T14:52:27.387 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:27.387+0000 7f40bffff640 1 -- 172.21.15.43:0/2185760534 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(182..182 src has 1..182) v4 ==== 5574+0+0 (secure 0 0 0) 0x7f40b803e070 con 0x7f40c80cb2d0 2023-08-09T14:52:27.387 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:27.387+0000 7f40be7fc640 1 --2- 172.21.15.43:0/2185760534 >> [v2:172.21.15.203:6832/2959400181,v1:172.21.15.203:6833/2959400181] conn(0x7f40a806a1a0 0x7f40a806c660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T14:52:27.388 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:27.388+0000 7f40be7fc640 1 --2- 172.21.15.43:0/2185760534 >> [v2:172.21.15.203:6832/2959400181,v1:172.21.15.203:6833/2959400181] conn(0x7f40a806a1a0 0x7f40a806c660 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f40b0000f60 tx=0x7f40b00079b0 comp rx=0 tx=0).ready entity=mgr.7223 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-09T14:52:27.391 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:27.391+0000 7f40bffff640 1 -- 172.21.15.43:0/2185760534 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178066 (secure 0 0 0) 0x7f40c8156830 con 0x7f40c80cb2d0 2023-08-09T14:52:27.586 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:27.586+0000 7f40cd2da640 1 -- 172.21.15.43:0/2185760534 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f40c8156830 con 0x7f40c80cb2d0 2023-08-09T14:52:27.587 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:27.587+0000 7f40bffff640 1 -- 172.21.15.43:0/2185760534 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v182) v1 ==== 93+0+10 (secure 0 0 0) 0x7f40c8156830 con 0x7f40c80cb2d0 2023-08-09T14:52:27.588 INFO:teuthology.orchestra.run.smithi043.stdout:pg_num: 1 2023-08-09T14:52:27.590 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:27.589+0000 7f40cd2da640 1 -- 172.21.15.43:0/2185760534 >> [v2:172.21.15.203:6832/2959400181,v1:172.21.15.203:6833/2959400181] conn(0x7f40a806a1a0 msgr2=0x7f40a806c660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T14:52:27.590 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:27.589+0000 7f40cd2da640 1 --2- 172.21.15.43:0/2185760534 >> [v2:172.21.15.203:6832/2959400181,v1:172.21.15.203:6833/2959400181] conn(0x7f40a806a1a0 0x7f40a806c660 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f40b0000f60 tx=0x7f40b00079b0 comp rx=0 tx=0).stop 2023-08-09T14:52:27.590 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:27.589+0000 7f40cd2da640 1 -- 172.21.15.43:0/2185760534 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f40c80cb2d0 msgr2=0x7f40c81f2a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T14:52:27.590 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:27.589+0000 7f40cd2da640 1 --2- 172.21.15.43:0/2185760534 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f40c80cb2d0 0x7f40c81f2a30 secure :-1 s=READY pgs=2013 cs=0 l=1 rev1=1 crypto rx=0x7f40b8000c00 tx=0x7f40b8004730 comp rx=0 tx=0).stop 2023-08-09T14:52:27.590 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:27.589+0000 7f40cd2da640 1 -- 172.21.15.43:0/2185760534 shutdown_connections 2023-08-09T14:52:27.590 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:27.589+0000 7f40cd2da640 1 --2- 172.21.15.43:0/2185760534 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f40c815df60 0x7f40c80c9490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T14:52:27.591 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:27.589+0000 7f40cd2da640 1 --2- 172.21.15.43:0/2185760534 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f40c80ce8d0 0x7f40c8153630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T14:52:27.591 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:27.589+0000 7f40cd2da640 1 --2- 172.21.15.43:0/2185760534 >> [v2:172.21.15.203:6832/2959400181,v1:172.21.15.203:6833/2959400181] conn(0x7f40a806a1a0 0x7f40a806c660 unknown :-1 s=CLOSED pgs=55 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T14:52:27.591 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:27.589+0000 7f40cd2da640 1 --2- 172.21.15.43:0/2185760534 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f40c80cb2d0 0x7f40c81f2a30 unknown :-1 s=CLOSED pgs=2013 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T14:52:27.591 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:27.589+0000 7f40cd2da640 1 -- 172.21.15.43:0/2185760534 >> 172.21.15.43:0/2185760534 conn(0x7f40c80aff50 msgr2=0x7f40c80ad9d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-09T14:52:27.591 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:27.589+0000 7f40cd2da640 1 -- 172.21.15.43:0/2185760534 shutdown_connections 2023-08-09T14:52:27.591 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:27.589+0000 7f40cd2da640 1 -- 172.21.15.43:0/2185760534 wait complete. 2023-08-09T14:52:27.607 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --format=json-pretty osd lspools 2023-08-09T14:52:27.822 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:27.822+0000 7ffa6228a640 1 Processor -- start 2023-08-09T14:52:27.823 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:27.822+0000 7ffa6228a640 1 -- start start 2023-08-09T14:52:27.824 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:27.824+0000 7ffa6228a640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7ffa5c160cb0 0x7ffa5c15c070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T14:52:27.825 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:27.824+0000 7ffa6228a640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7ffa5c160760 0x7ffa5c15c5b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T14:52:27.825 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:27.824+0000 7ffa6228a640 1 -- --> v1:172.21.15.43:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ffa5c0c8680 con 0x7ffa5c160380 2023-08-09T14:52:27.825 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:27.824+0000 7ffa6228a640 1 -- --> v2:172.21.15.43:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ffa5c0ca7d0 con 0x7ffa5c160cb0 2023-08-09T14:52:27.825 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:27.824+0000 7ffa6228a640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ffa5c0c8960 con 0x7ffa5c160760 2023-08-09T14:52:27.825 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:27.824+0000 7ffa61288640 1 --1- >> v1:172.21.15.43:6789/0 conn(0x7ffa5c160380 0x7ffa5c0ca0c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.43:6789/0 says I am v1:172.21.15.43:50424/0 (socket says 172.21.15.43:50424) 2023-08-09T14:52:27.825 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:27.824+0000 7ffa61288640 1 -- 172.21.15.43:0/2445737132 learned_addr learned my addr 172.21.15.43:0/2445737132 (peer_addr_for_me v1:172.21.15.43:0/0) 2023-08-09T14:52:27.826 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:27.824+0000 7ffa60a87640 1 --2- 172.21.15.43:0/2445737132 >> v2:172.21.15.43:3300/0 conn(0x7ffa5c160cb0 0x7ffa5c15c070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T14:52:27.826 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:27.825+0000 7ffa53fff640 1 -- 172.21.15.43:0/2445737132 <== mon.0 v1:172.21.15.43:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3528804372 0 0) 0x7ffa5c0c8680 con 0x7ffa5c160380 2023-08-09T14:52:27.826 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:27.825+0000 7ffa61a89640 1 --2- 172.21.15.43:0/2445737132 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7ffa5c160760 0x7ffa5c15c5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T14:52:27.827 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:27.825+0000 7ffa53fff640 1 -- 172.21.15.43:0/2445737132 --> v1:172.21.15.43:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ffa48003520 con 0x7ffa5c160380 2023-08-09T14:52:27.827 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:27.825+0000 7ffa60a87640 1 -- 172.21.15.43:0/2445737132 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7ffa5c160760 msgr2=0x7ffa5c15c5b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-09T14:52:27.827 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:27.825+0000 7ffa60a87640 1 --2- 172.21.15.43:0/2445737132 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7ffa5c160760 0x7ffa5c15c5b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T14:52:27.827 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:27.825+0000 7ffa60a87640 1 -- 172.21.15.43:0/2445737132 >> v1:172.21.15.43:6789/0 conn(0x7ffa5c160380 legacy=0x7ffa5c0ca0c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T14:52:27.827 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:27.825+0000 7ffa60a87640 1 -- 172.21.15.43:0/2445737132 --> v2:172.21.15.43:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ffa5c0aaf00 con 0x7ffa5c160cb0 2023-08-09T14:52:27.828 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:27.827+0000 7ffa60a87640 1 --2- 172.21.15.43:0/2445737132 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ffa5c160cb0 0x7ffa5c15c070 secure :-1 s=READY pgs=2015 cs=0 l=1 rev1=1 crypto rx=0x7ffa4c0097a0 tx=0x7ffa4c02b940 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7ee3da7ac6abe2f5 server_cookie=0 in_seq=0 out_seq=0 2023-08-09T14:52:27.828 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:27.827+0000 7ffa53fff640 1 -- 172.21.15.43:0/2445737132 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffa4c030800 con 0x7ffa5c160cb0 2023-08-09T14:52:27.829 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:27.827+0000 7ffa53fff640 1 -- 172.21.15.43:0/2445737132 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ffa4c030960 con 0x7ffa5c160cb0 2023-08-09T14:52:27.829 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:27.827+0000 7ffa53fff640 1 -- 172.21.15.43:0/2445737132 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffa4c030cd0 con 0x7ffa5c160cb0 2023-08-09T14:52:27.829 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:27.827+0000 7ffa6228a640 1 -- 172.21.15.43:0/2445737132 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ffa5c160cb0 msgr2=0x7ffa5c15c070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T14:52:27.829 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:27.827+0000 7ffa6228a640 1 --2- 172.21.15.43:0/2445737132 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ffa5c160cb0 0x7ffa5c15c070 secure :-1 s=READY pgs=2015 cs=0 l=1 rev1=1 crypto rx=0x7ffa4c0097a0 tx=0x7ffa4c02b940 comp rx=0 tx=0).stop 2023-08-09T14:52:27.829 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:27.828+0000 7ffa6228a640 1 -- 172.21.15.43:0/2445737132 shutdown_connections 2023-08-09T14:52:27.829 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:27.828+0000 7ffa6228a640 1 --2- 172.21.15.43:0/2445737132 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7ffa5c160760 0x7ffa5c15c5b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T14:52:27.829 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:27.828+0000 7ffa6228a640 1 --2- 172.21.15.43:0/2445737132 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ffa5c160cb0 0x7ffa5c15c070 unknown :-1 s=CLOSED pgs=2015 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T14:52:27.829 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:27.828+0000 7ffa6228a640 1 -- 172.21.15.43:0/2445737132 >> 172.21.15.43:0/2445737132 conn(0x7ffa5c0cde90 msgr2=0x7ffa5c0ce2a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-09T14:52:27.830 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:27.828+0000 7ffa6228a640 1 -- 172.21.15.43:0/2445737132 shutdown_connections 2023-08-09T14:52:27.830 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:27.828+0000 7ffa6228a640 1 -- 172.21.15.43:0/2445737132 wait complete. 2023-08-09T14:52:27.830 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:27.829+0000 7ffa6228a640 1 Processor -- start 2023-08-09T14:52:27.830 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:27.830+0000 7ffa6228a640 1 -- start start 2023-08-09T14:52:27.832 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:27.831+0000 7ffa6228a640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7ffa5c160380 0x7ffa5c0c7d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T14:52:27.832 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:27.831+0000 7ffa6228a640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ffa5c160760 0x7ffa5c1590b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T14:52:27.832 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:27.831+0000 7ffa6228a640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7ffa5c160cb0 0x7ffa5c0ca910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T14:52:27.833 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:27.831+0000 7ffa6228a640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffa5c1f5000 con 0x7ffa5c160760 2023-08-09T14:52:27.833 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:27.831+0000 7ffa6228a640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffa5c1f52e0 con 0x7ffa5c160cb0 2023-08-09T14:52:27.833 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:27.831+0000 7ffa6228a640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ffa5c1f55c0 con 0x7ffa5c160380 2023-08-09T14:52:27.833 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:27.831+0000 7ffa60a87640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ffa5c160760 0x7ffa5c1590b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T14:52:27.833 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:27.831+0000 7ffa60a87640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ffa5c160760 0x7ffa5c1590b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:55422/0 (socket says 172.21.15.43:55422) 2023-08-09T14:52:27.833 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:27.831+0000 7ffa60a87640 1 -- 172.21.15.43:0/681950544 learned_addr learned my addr 172.21.15.43:0/681950544 (peer_addr_for_me v2:172.21.15.43:0/0) 2023-08-09T14:52:27.833 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:27.831+0000 7ffa61288640 1 --2- 172.21.15.43:0/681950544 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7ffa5c160380 0x7ffa5c0c7d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T14:52:27.834 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:27.831+0000 7ffa61a89640 1 --2- 172.21.15.43:0/681950544 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7ffa5c160cb0 0x7ffa5c0ca910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T14:52:27.834 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:27.831+0000 7ffa60a87640 1 -- 172.21.15.43:0/681950544 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7ffa5c160380 msgr2=0x7ffa5c0c7d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T14:52:27.834 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:27.831+0000 7ffa60a87640 1 --2- 172.21.15.43:0/681950544 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7ffa5c160380 0x7ffa5c0c7d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T14:52:27.834 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:27.831+0000 7ffa60a87640 1 -- 172.21.15.43:0/681950544 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7ffa5c160cb0 msgr2=0x7ffa5c0ca910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T14:52:27.834 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:27.831+0000 7ffa60a87640 1 --2- 172.21.15.43:0/681950544 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7ffa5c160cb0 0x7ffa5c0ca910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T14:52:27.834 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:27.831+0000 7ffa60a87640 1 -- 172.21.15.43:0/681950544 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ffa4c009400 con 0x7ffa5c160760 2023-08-09T14:52:27.834 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:27.832+0000 7ffa60a87640 1 --2- 172.21.15.43:0/681950544 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ffa5c160760 0x7ffa5c1590b0 secure :-1 s=READY pgs=2016 cs=0 l=1 rev1=1 crypto rx=0x7ffa4c0098c0 tx=0x7ffa4c0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-09T14:52:27.835 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:27.832+0000 7ffa51ffb640 1 -- 172.21.15.43:0/681950544 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffa4c03e070 con 0x7ffa5c160760 2023-08-09T14:52:27.835 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:27.832+0000 7ffa51ffb640 1 -- 172.21.15.43:0/681950544 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ffa4c039a10 con 0x7ffa5c160760 2023-08-09T14:52:27.835 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:27.832+0000 7ffa51ffb640 1 -- 172.21.15.43:0/681950544 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffa4c039dc0 con 0x7ffa5c160760 2023-08-09T14:52:27.836 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:27.832+0000 7ffa6228a640 1 -- 172.21.15.43:0/681950544 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ffa5c1f58a0 con 0x7ffa5c160760 2023-08-09T14:52:27.836 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:27.832+0000 7ffa6228a640 1 -- 172.21.15.43:0/681950544 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ffa5c0b3750 con 0x7ffa5c160760 2023-08-09T14:52:27.836 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:27.833+0000 7ffa6228a640 1 -- 172.21.15.43:0/681950544 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ffa5c0739f0 con 0x7ffa5c160760 2023-08-09T14:52:27.836 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:27.834+0000 7ffa51ffb640 1 -- 172.21.15.43:0/681950544 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 88) v1 ==== 88443+0+0 (secure 0 0 0) 0x7ffa4c0423f0 con 0x7ffa5c160760 2023-08-09T14:52:27.837 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:27.835+0000 7ffa51ffb640 1 --2- 172.21.15.43:0/681950544 >> [v2:172.21.15.203:6832/2959400181,v1:172.21.15.203:6833/2959400181] conn(0x7ffa4006a170 0x7ffa4006c630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T14:52:27.837 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:27.835+0000 7ffa51ffb640 1 -- 172.21.15.43:0/681950544 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(182..182 src has 1..182) v4 ==== 5574+0+0 (secure 0 0 0) 0x7ffa4c0aa1c0 con 0x7ffa5c160760 2023-08-09T14:52:27.837 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:27.836+0000 7ffa61288640 1 --2- 172.21.15.43:0/681950544 >> [v2:172.21.15.203:6832/2959400181,v1:172.21.15.203:6833/2959400181] conn(0x7ffa4006a170 0x7ffa4006c630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T14:52:27.838 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:27.837+0000 7ffa61288640 1 --2- 172.21.15.43:0/681950544 >> [v2:172.21.15.203:6832/2959400181,v1:172.21.15.203:6833/2959400181] conn(0x7ffa4006a170 0x7ffa4006c630 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7ffa58007fd0 tx=0x7ffa58009040 comp rx=0 tx=0).ready entity=mgr.7223 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-09T14:52:27.840 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:27.840+0000 7ffa51ffb640 1 -- 172.21.15.43:0/681950544 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178066 (secure 0 0 0) 0x7ffa4c0757d0 con 0x7ffa5c160760 2023-08-09T14:52:28.038 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:28.038+0000 7ffa6228a640 1 -- 172.21.15.43:0/681950544 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) v1 -- 0x7ffa5c0cdb30 con 0x7ffa5c160760 2023-08-09T14:52:28.039 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:28.039+0000 7ffa51ffb640 1 -- 172.21.15.43:0/681950544 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v182) v1 ==== 84+0+126 (secure 0 0 0) 0x7ffa4c048090 con 0x7ffa5c160760 2023-08-09T14:52:28.039 INFO:teuthology.orchestra.run.smithi043.stdout: 2023-08-09T14:52:28.040 INFO:teuthology.orchestra.run.smithi043.stdout:[ 2023-08-09T14:52:28.040 INFO:teuthology.orchestra.run.smithi043.stdout: { 2023-08-09T14:52:28.040 INFO:teuthology.orchestra.run.smithi043.stdout: "poolnum": 1, 2023-08-09T14:52:28.040 INFO:teuthology.orchestra.run.smithi043.stdout: "poolname": "rbd" 2023-08-09T14:52:28.040 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2023-08-09T14:52:28.040 INFO:teuthology.orchestra.run.smithi043.stdout: { 2023-08-09T14:52:28.041 INFO:teuthology.orchestra.run.smithi043.stdout: "poolnum": 2, 2023-08-09T14:52:28.041 INFO:teuthology.orchestra.run.smithi043.stdout: "poolname": ".mgr" 2023-08-09T14:52:28.041 INFO:teuthology.orchestra.run.smithi043.stdout: } 2023-08-09T14:52:28.041 INFO:teuthology.orchestra.run.smithi043.stdout:] 2023-08-09T14:52:28.042 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:28.041+0000 7ffa6228a640 1 -- 172.21.15.43:0/681950544 >> [v2:172.21.15.203:6832/2959400181,v1:172.21.15.203:6833/2959400181] conn(0x7ffa4006a170 msgr2=0x7ffa4006c630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T14:52:28.042 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:28.041+0000 7ffa6228a640 1 --2- 172.21.15.43:0/681950544 >> [v2:172.21.15.203:6832/2959400181,v1:172.21.15.203:6833/2959400181] conn(0x7ffa4006a170 0x7ffa4006c630 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7ffa58007fd0 tx=0x7ffa58009040 comp rx=0 tx=0).stop 2023-08-09T14:52:28.042 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:28.041+0000 7ffa6228a640 1 -- 172.21.15.43:0/681950544 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ffa5c160760 msgr2=0x7ffa5c1590b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T14:52:28.042 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:28.041+0000 7ffa6228a640 1 --2- 172.21.15.43:0/681950544 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ffa5c160760 0x7ffa5c1590b0 secure :-1 s=READY pgs=2016 cs=0 l=1 rev1=1 crypto rx=0x7ffa4c0098c0 tx=0x7ffa4c0046c0 comp rx=0 tx=0).stop 2023-08-09T14:52:28.042 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:28.041+0000 7ffa6228a640 1 -- 172.21.15.43:0/681950544 shutdown_connections 2023-08-09T14:52:28.043 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:28.041+0000 7ffa6228a640 1 --2- 172.21.15.43:0/681950544 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7ffa5c160cb0 0x7ffa5c0ca910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T14:52:28.043 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:28.041+0000 7ffa6228a640 1 --2- 172.21.15.43:0/681950544 >> [v2:172.21.15.203:6832/2959400181,v1:172.21.15.203:6833/2959400181] conn(0x7ffa4006a170 0x7ffa4006c630 unknown :-1 s=CLOSED pgs=56 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T14:52:28.043 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:28.041+0000 7ffa6228a640 1 --2- 172.21.15.43:0/681950544 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ffa5c160760 0x7ffa5c1590b0 unknown :-1 s=CLOSED pgs=2016 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T14:52:28.043 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:28.041+0000 7ffa6228a640 1 --2- 172.21.15.43:0/681950544 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7ffa5c160380 0x7ffa5c0c7d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T14:52:28.043 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:28.041+0000 7ffa6228a640 1 -- 172.21.15.43:0/681950544 >> 172.21.15.43:0/681950544 conn(0x7ffa5c0cde90 msgr2=0x7ffa5c06f450 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-09T14:52:28.043 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:28.042+0000 7ffa6228a640 1 -- 172.21.15.43:0/681950544 shutdown_connections 2023-08-09T14:52:28.044 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:28.042+0000 7ffa6228a640 1 -- 172.21.15.43:0/681950544 wait complete. 2023-08-09T14:52:28.061 DEBUG:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2023-08-09T14:52:28.061 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_metadata --pg_num_min 64 2023-08-09T14:52:28.297 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:28.296+0000 7fc1396ec640 1 Processor -- start 2023-08-09T14:52:28.298 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:28.297+0000 7fc1396ec640 1 -- start start 2023-08-09T14:52:28.299 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:28.298+0000 7fc1396ec640 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7fc1340b80a0 0x7fc1340b5470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T14:52:28.299 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:28.298+0000 7fc1396ec640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fc1340b59b0 0x7fc1340b5e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T14:52:28.299 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:28.298+0000 7fc1396ec640 1 -- --> v1:172.21.15.43:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc1340d1890 con 0x7fc13415c6f0 2023-08-09T14:52:28.299 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:28.298+0000 7fc1396ec640 1 -- --> v2:172.21.15.203:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc1340d2d30 con 0x7fc1340b80a0 2023-08-09T14:52:28.299 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:28.298+0000 7fc1396ec640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc13415cad0 con 0x7fc1340b59b0 2023-08-09T14:52:28.300 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:28.299+0000 7fc132ffd640 1 --1- >> v1:172.21.15.43:6789/0 conn(0x7fc13415c6f0 0x7fc1340d03a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.43:6789/0 says I am v1:172.21.15.43:50438/0 (socket says 172.21.15.43:50438) 2023-08-09T14:52:28.300 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:28.299+0000 7fc132ffd640 1 -- 172.21.15.43:0/3277683626 learned_addr learned my addr 172.21.15.43:0/3277683626 (peer_addr_for_me v1:172.21.15.43:0/0) 2023-08-09T14:52:28.301 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:28.299+0000 7fc1337fe640 1 --2- 172.21.15.43:0/3277683626 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fc1340b59b0 0x7fc1340b5e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T14:52:28.301 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:28.299+0000 7fc12bfff640 1 --2- 172.21.15.43:0/3277683626 >> v2:172.21.15.203:3300/0 conn(0x7fc1340b80a0 0x7fc1340b5470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T14:52:28.301 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:28.299+0000 7fc1327fc640 1 -- 172.21.15.43:0/3277683626 <== mon.0 v1:172.21.15.43:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 323387182 0 0) 0x7fc1340d1890 con 0x7fc13415c6f0 2023-08-09T14:52:28.301 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:28.299+0000 7fc1327fc640 1 -- 172.21.15.43:0/3277683626 --> v1:172.21.15.43:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fc120003520 con 0x7fc13415c6f0 2023-08-09T14:52:28.301 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:28.300+0000 7fc1327fc640 1 -- 172.21.15.43:0/3277683626 <== mon.0 v1:172.21.15.43:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 4048131532 0 0) 0x7fc120003520 con 0x7fc13415c6f0 2023-08-09T14:52:28.301 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:28.300+0000 7fc1327fc640 1 -- 172.21.15.43:0/3277683626 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fc1340b59b0 msgr2=0x7fc1340b5e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-09T14:52:28.301 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:28.300+0000 7fc1327fc640 1 --2- 172.21.15.43:0/3277683626 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fc1340b59b0 0x7fc1340b5e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T14:52:28.302 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:28.300+0000 7fc1327fc640 1 -- 172.21.15.43:0/3277683626 >> v2:172.21.15.203:3300/0 conn(0x7fc1340b80a0 msgr2=0x7fc1340b5470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-09T14:52:28.302 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:28.300+0000 7fc1327fc640 1 --2- 172.21.15.43:0/3277683626 >> v2:172.21.15.203:3300/0 conn(0x7fc1340b80a0 0x7fc1340b5470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T14:52:28.302 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:28.300+0000 7fc1327fc640 1 -- 172.21.15.43:0/3277683626 --> v1:172.21.15.43:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc1340d0c80 con 0x7fc13415c6f0 2023-08-09T14:52:28.302 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:28.300+0000 7fc1327fc640 1 -- 172.21.15.43:0/3277683626 <== mon.0 v1:172.21.15.43:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1285150740 0 0) 0x7fc124001670 con 0x7fc13415c6f0 2023-08-09T14:52:28.302 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:28.300+0000 7fc1327fc640 1 -- 172.21.15.43:0/3277683626 >> v1:172.21.15.43:6789/0 conn(0x7fc13415c6f0 legacy=0x7fc1340d03a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T14:52:28.302 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:28.300+0000 7fc1327fc640 1 --2- 172.21.15.43:0/3277683626 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc1200039e0 0x7fc120003e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T14:52:28.303 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:28.300+0000 7fc1327fc640 1 --2- 172.21.15.43:0/3277683626 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fc1340b59b0 0x7fc1200047c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T14:52:28.303 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:28.300+0000 7fc1327fc640 1 --2- 172.21.15.43:0/3277683626 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fc120004d00 0x7fc120005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T14:52:28.303 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:28.300+0000 7fc1327fc640 1 -- 172.21.15.43:0/3277683626 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc1340d2d30 con 0x7fc1200039e0 2023-08-09T14:52:28.303 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:28.300+0000 7fc1327fc640 1 -- 172.21.15.43:0/3277683626 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc13415cad0 con 0x7fc120004d00 2023-08-09T14:52:28.303 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:28.300+0000 7fc1327fc640 1 -- 172.21.15.43:0/3277683626 --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc12000b980 con 0x7fc1340b59b0 2023-08-09T14:52:28.304 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:28.300+0000 7fc132ffd640 1 --2- 172.21.15.43:0/3277683626 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc1200039e0 0x7fc120003e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T14:52:28.305 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:28.301+0000 7fc12bfff640 1 --2- 172.21.15.43:0/3277683626 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fc1340b59b0 0x7fc1200047c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T14:52:28.305 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:28.301+0000 7fc1337fe640 1 --2- 172.21.15.43:0/3277683626 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fc120004d00 0x7fc120005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T14:52:28.305 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:28.301+0000 7fc132ffd640 1 -- 172.21.15.43:0/3277683626 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fc1340b59b0 msgr2=0x7fc1200047c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-09T14:52:28.305 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:28.301+0000 7fc132ffd640 1 --2- 172.21.15.43:0/3277683626 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fc1340b59b0 0x7fc1200047c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T14:52:28.305 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:28.301+0000 7fc132ffd640 1 -- 172.21.15.43:0/3277683626 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fc120004d00 msgr2=0x7fc120005180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-09T14:52:28.305 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:28.301+0000 7fc132ffd640 1 --2- 172.21.15.43:0/3277683626 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fc120004d00 0x7fc120005180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T14:52:28.305 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:28.301+0000 7fc132ffd640 1 -- 172.21.15.43:0/3277683626 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc120003520 con 0x7fc1200039e0 2023-08-09T14:52:28.306 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:28.302+0000 7fc132ffd640 1 --2- 172.21.15.43:0/3277683626 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc1200039e0 0x7fc120003e00 secure :-1 s=READY pgs=2018 cs=0 l=1 rev1=1 crypto rx=0x7fc12400c240 tx=0x7fc1200057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c62ec5b038b190a6 server_cookie=0 in_seq=0 out_seq=0 2023-08-09T14:52:28.306 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:28.302+0000 7fc1327fc640 1 -- 172.21.15.43:0/3277683626 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc12402cb00 con 0x7fc1200039e0 2023-08-09T14:52:28.306 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:28.302+0000 7fc1327fc640 1 -- 172.21.15.43:0/3277683626 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc12402cc60 con 0x7fc1200039e0 2023-08-09T14:52:28.306 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:28.302+0000 7fc1327fc640 1 -- 172.21.15.43:0/3277683626 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc124005af0 con 0x7fc1200039e0 2023-08-09T14:52:28.306 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:28.303+0000 7fc1396ec640 1 -- 172.21.15.43:0/3277683626 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc1200039e0 msgr2=0x7fc120003e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T14:52:28.306 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:28.303+0000 7fc1396ec640 1 --2- 172.21.15.43:0/3277683626 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc1200039e0 0x7fc120003e00 secure :-1 s=READY pgs=2018 cs=0 l=1 rev1=1 crypto rx=0x7fc12400c240 tx=0x7fc1200057f0 comp rx=0 tx=0).stop 2023-08-09T14:52:28.306 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:28.303+0000 7fc1337fe640 1 -- 172.21.15.43:0/3277683626 reap_dead start 2023-08-09T14:52:28.307 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:28.303+0000 7fc1396ec640 1 -- 172.21.15.43:0/3277683626 shutdown_connections 2023-08-09T14:52:28.307 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:28.303+0000 7fc1396ec640 1 -- 172.21.15.43:0/3277683626 >> 172.21.15.43:0/3277683626 conn(0x7fc1340712e0 msgr2=0x7fc1340716f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-09T14:52:28.307 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:28.303+0000 7fc1396ec640 1 -- 172.21.15.43:0/3277683626 shutdown_connections 2023-08-09T14:52:28.307 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:28.303+0000 7fc1396ec640 1 -- 172.21.15.43:0/3277683626 wait complete. 2023-08-09T14:52:28.308 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:28.305+0000 7fc1396ec640 1 Processor -- start 2023-08-09T14:52:28.309 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:28.305+0000 7fc1396ec640 1 -- start start 2023-08-09T14:52:28.309 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:28.306+0000 7fc1396ec640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc1200039e0 0x7fc1340c8520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T14:52:28.309 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:28.306+0000 7fc1396ec640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fc1340ca500 0x7fc134144420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T14:52:28.309 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:28.306+0000 7fc1396ec640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fc1340ca980 0x7fc134152570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T14:52:28.309 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:28.306+0000 7fc1396ec640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc134154a40 con 0x7fc1200039e0 2023-08-09T14:52:28.309 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:28.306+0000 7fc1396ec640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc1340c8a60 con 0x7fc1340ca500 2023-08-09T14:52:28.310 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:28.306+0000 7fc1396ec640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc134144ee0 con 0x7fc1340ca980 2023-08-09T14:52:28.310 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:28.306+0000 7fc132ffd640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc1200039e0 0x7fc1340c8520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T14:52:28.310 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:28.306+0000 7fc132ffd640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc1200039e0 0x7fc1340c8520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:55444/0 (socket says 172.21.15.43:55444) 2023-08-09T14:52:28.310 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:28.307+0000 7fc132ffd640 1 -- 172.21.15.43:0/1548467934 learned_addr learned my addr 172.21.15.43:0/1548467934 (peer_addr_for_me v2:172.21.15.43:0/0) 2023-08-09T14:52:28.310 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:28.307+0000 7fc12bfff640 1 --2- 172.21.15.43:0/1548467934 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fc1340ca500 0x7fc134144420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T14:52:28.310 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:28.307+0000 7fc1337fe640 1 --2- 172.21.15.43:0/1548467934 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fc1340ca980 0x7fc134152570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T14:52:28.310 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:28.307+0000 7fc132ffd640 1 -- 172.21.15.43:0/1548467934 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fc1340ca980 msgr2=0x7fc134152570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T14:52:28.311 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:28.307+0000 7fc132ffd640 1 --2- 172.21.15.43:0/1548467934 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fc1340ca980 0x7fc134152570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T14:52:28.311 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:28.307+0000 7fc132ffd640 1 -- 172.21.15.43:0/1548467934 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fc1340ca500 msgr2=0x7fc134144420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T14:52:28.311 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:28.307+0000 7fc132ffd640 1 --2- 172.21.15.43:0/1548467934 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fc1340ca500 0x7fc134144420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T14:52:28.311 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:28.307+0000 7fc132ffd640 1 -- 172.21.15.43:0/1548467934 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc1341451c0 con 0x7fc1200039e0 2023-08-09T14:52:28.311 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:28.307+0000 7fc12bfff640 1 --2- 172.21.15.43:0/1548467934 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fc1340ca500 0x7fc134144420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-08-09T14:52:28.311 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:28.307+0000 7fc132ffd640 1 --2- 172.21.15.43:0/1548467934 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc1200039e0 0x7fc1340c8520 secure :-1 s=READY pgs=2019 cs=0 l=1 rev1=1 crypto rx=0x7fc12400c360 tx=0x7fc124001a00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-09T14:52:28.311 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:28.307+0000 7fc12b7fe640 1 -- 172.21.15.43:0/1548467934 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc124039070 con 0x7fc1200039e0 2023-08-09T14:52:28.312 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:28.308+0000 7fc12b7fe640 1 -- 172.21.15.43:0/1548467934 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc124005c80 con 0x7fc1200039e0 2023-08-09T14:52:28.312 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:28.308+0000 7fc12b7fe640 1 -- 172.21.15.43:0/1548467934 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc12403c040 con 0x7fc1200039e0 2023-08-09T14:52:28.312 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:28.308+0000 7fc1396ec640 1 -- 172.21.15.43:0/1548467934 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc1341546d0 con 0x7fc1200039e0 2023-08-09T14:52:28.312 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:28.308+0000 7fc1396ec640 1 -- 172.21.15.43:0/1548467934 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc1340b1ec0 con 0x7fc1200039e0 2023-08-09T14:52:28.313 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:28.309+0000 7fc1396ec640 1 -- 172.21.15.43:0/1548467934 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc134066000 con 0x7fc1200039e0 2023-08-09T14:52:28.313 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:28.310+0000 7fc12b7fe640 1 -- 172.21.15.43:0/1548467934 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 88) v1 ==== 88443+0+0 (secure 0 0 0) 0x7fc124032070 con 0x7fc1200039e0 2023-08-09T14:52:28.313 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:28.311+0000 7fc12b7fe640 1 --2- 172.21.15.43:0/1548467934 >> [v2:172.21.15.203:6832/2959400181,v1:172.21.15.203:6833/2959400181] conn(0x7fc11406a1a0 0x7fc11406c660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T14:52:28.313 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:28.311+0000 7fc12b7fe640 1 -- 172.21.15.43:0/1548467934 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(182..182 src has 1..182) v4 ==== 5574+0+0 (secure 0 0 0) 0x7fc1240a9a10 con 0x7fc1200039e0 2023-08-09T14:52:28.313 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:28.312+0000 7fc12bfff640 1 --2- 172.21.15.43:0/1548467934 >> [v2:172.21.15.203:6832/2959400181,v1:172.21.15.203:6833/2959400181] conn(0x7fc11406a1a0 0x7fc11406c660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T14:52:28.314 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:28.313+0000 7fc12bfff640 1 --2- 172.21.15.43:0/1548467934 >> [v2:172.21.15.203:6832/2959400181,v1:172.21.15.203:6833/2959400181] conn(0x7fc11406a1a0 0x7fc11406c660 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7fc1340cc5b0 tx=0x7fc11c008040 comp rx=0 tx=0).ready entity=mgr.7223 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-09T14:52:28.315 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:28.315+0000 7fc12b7fe640 1 -- 172.21.15.43:0/1548467934 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178066 (secure 0 0 0) 0x7fc1240751e0 con 0x7fc1200039e0 2023-08-09T14:52:28.510 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:28.509+0000 7fc1396ec640 1 -- 172.21.15.43:0/1548467934 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64} v 0) v1 -- 0x7fc134145350 con 0x7fc1200039e0 2023-08-09T14:52:28.671 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:28.670+0000 7fc12b7fe640 1 -- 172.21.15.43:0/1548467934 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]=0 pool 'cephfs_metadata' created v183) v1 ==== 138+0+0 (secure 0 0 0) 0x7fc124046020 con 0x7fc1200039e0 2023-08-09T14:52:28.672 INFO:teuthology.orchestra.run.smithi043.stderr:pool 'cephfs_metadata' created 2023-08-09T14:52:28.676 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:28.676+0000 7fc1396ec640 1 -- 172.21.15.43:0/1548467934 >> [v2:172.21.15.203:6832/2959400181,v1:172.21.15.203:6833/2959400181] conn(0x7fc11406a1a0 msgr2=0x7fc11406c660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T14:52:28.677 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:28.676+0000 7fc1396ec640 1 --2- 172.21.15.43:0/1548467934 >> [v2:172.21.15.203:6832/2959400181,v1:172.21.15.203:6833/2959400181] conn(0x7fc11406a1a0 0x7fc11406c660 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7fc1340cc5b0 tx=0x7fc11c008040 comp rx=0 tx=0).stop 2023-08-09T14:52:28.677 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:28.676+0000 7fc1396ec640 1 -- 172.21.15.43:0/1548467934 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc1200039e0 msgr2=0x7fc1340c8520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T14:52:28.677 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:28.676+0000 7fc1396ec640 1 --2- 172.21.15.43:0/1548467934 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc1200039e0 0x7fc1340c8520 secure :-1 s=READY pgs=2019 cs=0 l=1 rev1=1 crypto rx=0x7fc12400c360 tx=0x7fc124001a00 comp rx=0 tx=0).stop 2023-08-09T14:52:28.677 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:28.676+0000 7fc1396ec640 1 -- 172.21.15.43:0/1548467934 shutdown_connections 2023-08-09T14:52:28.677 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:28.676+0000 7fc1396ec640 1 --2- 172.21.15.43:0/1548467934 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fc1340ca980 0x7fc134152570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T14:52:28.677 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:28.676+0000 7fc1396ec640 1 --2- 172.21.15.43:0/1548467934 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fc1340ca500 0x7fc134144420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T14:52:28.677 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:28.676+0000 7fc1396ec640 1 --2- 172.21.15.43:0/1548467934 >> [v2:172.21.15.203:6832/2959400181,v1:172.21.15.203:6833/2959400181] conn(0x7fc11406a1a0 0x7fc11406c660 unknown :-1 s=CLOSED pgs=57 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T14:52:28.678 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:28.676+0000 7fc1396ec640 1 --2- 172.21.15.43:0/1548467934 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc1200039e0 0x7fc1340c8520 unknown :-1 s=CLOSED pgs=2019 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T14:52:28.678 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:28.676+0000 7fc1396ec640 1 -- 172.21.15.43:0/1548467934 >> 172.21.15.43:0/1548467934 conn(0x7fc1340712e0 msgr2=0x7fc13415cd10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-09T14:52:28.678 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:28.676+0000 7fc1396ec640 1 -- 172.21.15.43:0/1548467934 shutdown_connections 2023-08-09T14:52:28.678 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:28.676+0000 7fc1396ec640 1 -- 172.21.15.43:0/1548467934 wait complete. 2023-08-09T14:52:28.699 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data 64 --pg_num_min 64 --target_size_ratio 0.9 2023-08-09T14:52:28.935 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:28.934+0000 7fe84de37640 1 Processor -- start 2023-08-09T14:52:28.935 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:28.935+0000 7fe84de37640 1 -- start start 2023-08-09T14:52:28.937 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:28.936+0000 7fe84de37640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7fe8480b7df0 0x7fe8480d1b30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T14:52:28.937 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:28.936+0000 7fe84de37640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fe8480cebf0 0x7fe8480cf040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T14:52:28.937 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:28.936+0000 7fe84de37640 1 -- --> v1:172.21.15.203:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe8480cf580 con 0x7fe84815c2d0 2023-08-09T14:52:28.937 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:28.936+0000 7fe84de37640 1 -- --> v2:172.21.15.43:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe8480b8230 con 0x7fe8480b7df0 2023-08-09T14:52:28.937 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:28.936+0000 7fe84de37640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe8480d16a0 con 0x7fe8480cebf0 2023-08-09T14:52:28.938 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:28.937+0000 7fe847fff640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7fe8480b7df0 0x7fe8480d1b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T14:52:28.938 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:28.937+0000 7fe847fff640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7fe8480b7df0 0x7fe8480d1b30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:55458/0 (socket says 172.21.15.43:55458) 2023-08-09T14:52:28.938 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:28.937+0000 7fe847fff640 1 -- 172.21.15.43:0/3920532673 learned_addr learned my addr 172.21.15.43:0/3920532673 (peer_addr_for_me v2:172.21.15.43:0/0) 2023-08-09T14:52:28.938 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:28.937+0000 7fe84d636640 1 --2- 172.21.15.43:0/3920532673 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fe8480cebf0 0x7fe8480cf040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T14:52:28.939 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:28.937+0000 7fe847fff640 1 -- 172.21.15.43:0/3920532673 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fe8480cebf0 msgr2=0x7fe8480cf040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-09T14:52:28.939 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:28.937+0000 7fe847fff640 1 --2- 172.21.15.43:0/3920532673 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fe8480cebf0 0x7fe8480cf040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T14:52:28.939 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:28.937+0000 7fe847fff640 1 -- 172.21.15.43:0/3920532673 >> v1:172.21.15.203:6789/0 conn(0x7fe84815c2d0 legacy=0x7fe8480cd020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-09T14:52:28.939 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:28.937+0000 7fe847fff640 1 -- 172.21.15.43:0/3920532673 --> v2:172.21.15.43:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe8480d0d70 con 0x7fe8480b7df0 2023-08-09T14:52:28.939 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:28.937+0000 7fe84d636640 1 --2- 172.21.15.43:0/3920532673 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fe8480cebf0 0x7fe8480cf040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-08-09T14:52:28.940 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:28.939+0000 7fe847fff640 1 --2- 172.21.15.43:0/3920532673 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fe8480b7df0 0x7fe8480d1b30 secure :-1 s=READY pgs=2020 cs=0 l=1 rev1=1 crypto rx=0x7fe8480c5ce0 tx=0x7fe83402b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dc20d843f02cc1f1 server_cookie=0 in_seq=0 out_seq=0 2023-08-09T14:52:28.940 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:28.939+0000 7fe8477fe640 1 -- 172.21.15.43:0/3920532673 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe834031830 con 0x7fe8480b7df0 2023-08-09T14:52:28.940 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:28.939+0000 7fe8477fe640 1 -- 172.21.15.43:0/3920532673 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe834031990 con 0x7fe8480b7df0 2023-08-09T14:52:28.940 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:28.939+0000 7fe8477fe640 1 -- 172.21.15.43:0/3920532673 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe834031d20 con 0x7fe8480b7df0 2023-08-09T14:52:28.941 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:28.940+0000 7fe84de37640 1 -- 172.21.15.43:0/3920532673 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fe8480b7df0 msgr2=0x7fe8480d1b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T14:52:28.941 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:28.940+0000 7fe84de37640 1 --2- 172.21.15.43:0/3920532673 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fe8480b7df0 0x7fe8480d1b30 secure :-1 s=READY pgs=2020 cs=0 l=1 rev1=1 crypto rx=0x7fe8480c5ce0 tx=0x7fe83402b8d0 comp rx=0 tx=0).stop 2023-08-09T14:52:28.941 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:28.940+0000 7fe84de37640 1 -- 172.21.15.43:0/3920532673 shutdown_connections 2023-08-09T14:52:28.941 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:28.940+0000 7fe84de37640 1 --2- 172.21.15.43:0/3920532673 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fe8480cebf0 0x7fe8480cf040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T14:52:28.941 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:28.940+0000 7fe84de37640 1 --2- 172.21.15.43:0/3920532673 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fe8480b7df0 0x7fe8480d1b30 unknown :-1 s=CLOSED pgs=2020 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T14:52:28.942 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:28.940+0000 7fe84de37640 1 -- 172.21.15.43:0/3920532673 >> 172.21.15.43:0/3920532673 conn(0x7fe84806a230 msgr2=0x7fe84806a640 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-09T14:52:28.942 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:28.940+0000 7fe84de37640 1 -- 172.21.15.43:0/3920532673 shutdown_connections 2023-08-09T14:52:28.942 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:28.940+0000 7fe84de37640 1 -- 172.21.15.43:0/3920532673 wait complete. 2023-08-09T14:52:28.942 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:28.942+0000 7fe84de37640 1 Processor -- start 2023-08-09T14:52:28.943 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:28.942+0000 7fe84de37640 1 -- start start 2023-08-09T14:52:28.943 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:28.943+0000 7fe84de37640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fe8480b7df0 0x7fe84815f3d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T14:52:28.944 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:28.943+0000 7fe84de37640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fe8480cebf0 0x7fe84815d050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T14:52:28.944 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:28.943+0000 7fe84de37640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fe84815c2d0 0x7fe8481f1bb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T14:52:28.944 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:28.943+0000 7fe84de37640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe8481f2b30 con 0x7fe8480b7df0 2023-08-09T14:52:28.944 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:28.943+0000 7fe84de37640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe84815f910 con 0x7fe84815c2d0 2023-08-09T14:52:28.944 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:28.943+0000 7fe84de37640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe8481f2570 con 0x7fe8480cebf0 2023-08-09T14:52:28.945 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:28.943+0000 7fe84ce35640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fe8480b7df0 0x7fe84815f3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T14:52:28.946 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:28.943+0000 7fe84ce35640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fe8480b7df0 0x7fe84815f3d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:55464/0 (socket says 172.21.15.43:55464) 2023-08-09T14:52:28.946 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:28.943+0000 7fe84ce35640 1 -- 172.21.15.43:0/1054534688 learned_addr learned my addr 172.21.15.43:0/1054534688 (peer_addr_for_me v2:172.21.15.43:0/0) 2023-08-09T14:52:28.946 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:28.944+0000 7fe847fff640 1 --2- 172.21.15.43:0/1054534688 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fe8480cebf0 0x7fe84815d050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T14:52:28.946 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:28.944+0000 7fe84d636640 1 --2- 172.21.15.43:0/1054534688 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fe84815c2d0 0x7fe8481f1bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T14:52:28.946 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:28.944+0000 7fe84ce35640 1 -- 172.21.15.43:0/1054534688 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fe8480cebf0 msgr2=0x7fe84815d050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T14:52:28.946 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:28.944+0000 7fe84ce35640 1 --2- 172.21.15.43:0/1054534688 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fe8480cebf0 0x7fe84815d050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T14:52:28.946 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:28.944+0000 7fe84ce35640 1 -- 172.21.15.43:0/1054534688 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fe84815c2d0 msgr2=0x7fe8481f1bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T14:52:28.947 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:28.944+0000 7fe84ce35640 1 --2- 172.21.15.43:0/1054534688 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fe84815c2d0 0x7fe8481f1bb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T14:52:28.947 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:28.944+0000 7fe84ce35640 1 -- 172.21.15.43:0/1054534688 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe8340094d0 con 0x7fe8480b7df0 2023-08-09T14:52:28.947 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:28.944+0000 7fe84d636640 1 --2- 172.21.15.43:0/1054534688 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fe84815c2d0 0x7fe8481f1bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-08-09T14:52:28.947 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:28.944+0000 7fe84ce35640 1 --2- 172.21.15.43:0/1054534688 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fe8480b7df0 0x7fe84815f3d0 secure :-1 s=READY pgs=2021 cs=0 l=1 rev1=1 crypto rx=0x7fe83c00c430 tx=0x7fe83c00c8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-09T14:52:28.947 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:28.945+0000 7fe8457fa640 1 -- 172.21.15.43:0/1054534688 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe83c011790 con 0x7fe8480b7df0 2023-08-09T14:52:28.947 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:28.945+0000 7fe8457fa640 1 -- 172.21.15.43:0/1054534688 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe83c0118f0 con 0x7fe8480b7df0 2023-08-09T14:52:28.947 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:28.945+0000 7fe84de37640 1 -- 172.21.15.43:0/1054534688 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe8481f2850 con 0x7fe8480b7df0 2023-08-09T14:52:28.948 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:28.945+0000 7fe8457fa640 1 -- 172.21.15.43:0/1054534688 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe83c011ce0 con 0x7fe8480b7df0 2023-08-09T14:52:28.948 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:28.945+0000 7fe84de37640 1 -- 172.21.15.43:0/1054534688 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe84815cb30 con 0x7fe8480b7df0 2023-08-09T14:52:28.948 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:28.946+0000 7fe84de37640 1 -- 172.21.15.43:0/1054534688 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe84815c6b0 con 0x7fe8480b7df0 2023-08-09T14:52:28.948 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:28.948+0000 7fe8457fa640 1 -- 172.21.15.43:0/1054534688 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 88) v1 ==== 88443+0+0 (secure 0 0 0) 0x7fe83c01c460 con 0x7fe8480b7df0 2023-08-09T14:52:28.949 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:28.949+0000 7fe8457fa640 1 --2- 172.21.15.43:0/1054534688 >> [v2:172.21.15.203:6832/2959400181,v1:172.21.15.203:6833/2959400181] conn(0x7fe82c06a170 0x7fe82c06c630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T14:52:28.950 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:28.949+0000 7fe8457fa640 1 -- 172.21.15.43:0/1054534688 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(183..183 src has 1..183) v4 ==== 5944+0+0 (secure 0 0 0) 0x7fe83c088fd0 con 0x7fe8480b7df0 2023-08-09T14:52:28.950 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:28.950+0000 7fe847fff640 1 --2- 172.21.15.43:0/1054534688 >> [v2:172.21.15.203:6832/2959400181,v1:172.21.15.203:6833/2959400181] conn(0x7fe82c06a170 0x7fe82c06c630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T14:52:28.951 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:28.951+0000 7fe847fff640 1 --2- 172.21.15.43:0/1054534688 >> [v2:172.21.15.203:6832/2959400181,v1:172.21.15.203:6833/2959400181] conn(0x7fe82c06a170 0x7fe82c06c630 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7fe834036000 tx=0x7fe83403c040 comp rx=0 tx=0).ready entity=mgr.7223 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-09T14:52:28.954 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:28.954+0000 7fe8457fa640 1 -- 172.21.15.43:0/1054534688 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178066 (secure 0 0 0) 0x7fe83c054600 con 0x7fe8480b7df0 2023-08-09T14:52:29.152 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:29.152+0000 7fe84de37640 1 -- 172.21.15.43:0/1054534688 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9} v 0) v1 -- 0x7fe84815c6b0 con 0x7fe8480b7df0 2023-08-09T14:52:29.698 INFO:teuthology.orchestra.run.smithi043.stderr:pool 'cephfs_data' created 2023-08-09T14:52:29.705 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:29.696+0000 7fe8457fa640 1 -- 172.21.15.43:0/1054534688 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]=0 pool 'cephfs_data' created v184) v1 ==== 170+0+0 (secure 0 0 0) 0x7fe84815c6b0 con 0x7fe8480b7df0 2023-08-09T14:52:29.711 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:29.706+0000 7fe84de37640 1 -- 172.21.15.43:0/1054534688 >> [v2:172.21.15.203:6832/2959400181,v1:172.21.15.203:6833/2959400181] conn(0x7fe82c06a170 msgr2=0x7fe82c06c630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T14:52:29.711 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:29.706+0000 7fe84de37640 1 --2- 172.21.15.43:0/1054534688 >> [v2:172.21.15.203:6832/2959400181,v1:172.21.15.203:6833/2959400181] conn(0x7fe82c06a170 0x7fe82c06c630 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7fe834036000 tx=0x7fe83403c040 comp rx=0 tx=0).stop 2023-08-09T14:52:29.712 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:29.706+0000 7fe84de37640 1 -- 172.21.15.43:0/1054534688 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fe8480b7df0 msgr2=0x7fe84815f3d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T14:52:29.712 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:29.706+0000 7fe84de37640 1 --2- 172.21.15.43:0/1054534688 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fe8480b7df0 0x7fe84815f3d0 secure :-1 s=READY pgs=2021 cs=0 l=1 rev1=1 crypto rx=0x7fe83c00c430 tx=0x7fe83c00c8f0 comp rx=0 tx=0).stop 2023-08-09T14:52:29.712 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:29.707+0000 7fe84de37640 1 -- 172.21.15.43:0/1054534688 shutdown_connections 2023-08-09T14:52:29.712 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:29.707+0000 7fe84de37640 1 --2- 172.21.15.43:0/1054534688 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fe84815c2d0 0x7fe8481f1bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T14:52:29.713 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:29.707+0000 7fe84de37640 1 --2- 172.21.15.43:0/1054534688 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fe8480cebf0 0x7fe84815d050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T14:52:29.713 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:29.707+0000 7fe84de37640 1 --2- 172.21.15.43:0/1054534688 >> [v2:172.21.15.203:6832/2959400181,v1:172.21.15.203:6833/2959400181] conn(0x7fe82c06a170 0x7fe82c06c630 secure :-1 s=CLOSED pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7fe834036000 tx=0x7fe83403c040 comp rx=0 tx=0).stop 2023-08-09T14:52:29.713 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:29.707+0000 7fe84de37640 1 --2- 172.21.15.43:0/1054534688 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fe8480b7df0 0x7fe84815f3d0 unknown :-1 s=CLOSED pgs=2021 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T14:52:29.713 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:29.707+0000 7fe84de37640 1 -- 172.21.15.43:0/1054534688 >> 172.21.15.43:0/1054534688 conn(0x7fe84806a230 msgr2=0x7fe8480b2070 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-09T14:52:29.714 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:29.707+0000 7fe84de37640 1 -- 172.21.15.43:0/1054534688 shutdown_connections 2023-08-09T14:52:29.714 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:29.708+0000 7fe84de37640 1 -- 172.21.15.43:0/1054534688 wait complete. 2023-08-09T14:52:29.731 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs new cephfs cephfs_metadata cephfs_data 2023-08-09T14:52:29.932 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:29.931+0000 7fe172742640 1 Processor -- start 2023-08-09T14:52:29.932 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:29.932+0000 7fe172742640 1 -- start start 2023-08-09T14:52:29.933 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:29.933+0000 7fe172742640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7fe16c15a0c0 0x7fe16c0cbb80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T14:52:29.934 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:29.933+0000 7fe172742640 1 -- --> v1:172.21.15.43:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe16c15a4a0 con 0x7fe16c15d510 2023-08-09T14:52:29.934 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:29.933+0000 7fe172742640 1 -- --> v1:172.21.15.203:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe16c0cc2d0 con 0x7fe16c0ce1c0 2023-08-09T14:52:29.934 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:29.933+0000 7fe172742640 1 -- --> v2:172.21.15.43:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe16c0cc0c0 con 0x7fe16c15a0c0 2023-08-09T14:52:29.935 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:29.933+0000 7fe16bfff640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7fe16c15a0c0 0x7fe16c0cbb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T14:52:29.935 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:29.934+0000 7fe16bfff640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7fe16c15a0c0 0x7fe16c0cbb80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:55468/0 (socket says 172.21.15.43:55468) 2023-08-09T14:52:29.935 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:29.934+0000 7fe163fff640 1 --1- >> v1:172.21.15.43:6789/0 conn(0x7fe16c15d510 0x7fe16c0cb2b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.43:6789/0 says I am v1:172.21.15.43:50442/0 (socket says 172.21.15.43:50442) 2023-08-09T14:52:29.935 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:29.934+0000 7fe16bfff640 1 -- 172.21.15.43:0/3789215779 learned_addr learned my addr 172.21.15.43:0/3789215779 (peer_addr_for_me v2:172.21.15.43:0/0) 2023-08-09T14:52:29.935 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:29.934+0000 7fe16bfff640 1 -- 172.21.15.43:0/3789215779 >> v1:172.21.15.203:6789/0 conn(0x7fe16c0ce1c0 legacy=0x7fe16c0ce5c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-09T14:52:29.936 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:29.934+0000 7fe16bfff640 1 -- 172.21.15.43:0/3789215779 >> v1:172.21.15.43:6789/0 conn(0x7fe16c15d510 legacy=0x7fe16c0cb2b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T14:52:29.936 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:29.934+0000 7fe16bfff640 1 -- 172.21.15.43:0/3789215779 --> v2:172.21.15.43:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe16c0cc5b0 con 0x7fe16c15a0c0 2023-08-09T14:52:29.936 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:29.934+0000 7fe16b7fe640 1 -- 172.21.15.43:0/3789215779 <== mon.0 v1:172.21.15.43:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2838775630 0 0) 0x7fe16c15a4a0 con 0x7fe16c15d510 2023-08-09T14:52:29.936 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:29.936+0000 7fe16bfff640 1 --2- 172.21.15.43:0/3789215779 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fe16c15a0c0 0x7fe16c0cbb80 secure :-1 s=READY pgs=2023 cs=0 l=1 rev1=1 crypto rx=0x7fe15c009870 tx=0x7fe15c02b990 comp rx=0 tx=0).ready entity=mon.0 client_cookie=aaace35fce56545d server_cookie=0 in_seq=0 out_seq=0 2023-08-09T14:52:29.937 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:29.936+0000 7fe16b7fe640 1 -- 172.21.15.43:0/3789215779 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe15c030850 con 0x7fe16c15a0c0 2023-08-09T14:52:29.937 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:29.936+0000 7fe16b7fe640 1 -- 172.21.15.43:0/3789215779 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe15c0309b0 con 0x7fe16c15a0c0 2023-08-09T14:52:29.937 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:29.936+0000 7fe16b7fe640 1 -- 172.21.15.43:0/3789215779 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe15c030d40 con 0x7fe16c15a0c0 2023-08-09T14:52:29.938 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:29.937+0000 7fe172742640 1 -- 172.21.15.43:0/3789215779 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fe16c15a0c0 msgr2=0x7fe16c0cbb80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T14:52:29.938 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:29.937+0000 7fe172742640 1 --2- 172.21.15.43:0/3789215779 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fe16c15a0c0 0x7fe16c0cbb80 secure :-1 s=READY pgs=2023 cs=0 l=1 rev1=1 crypto rx=0x7fe15c009870 tx=0x7fe15c02b990 comp rx=0 tx=0).stop 2023-08-09T14:52:29.938 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:29.937+0000 7fe172742640 1 -- 172.21.15.43:0/3789215779 shutdown_connections 2023-08-09T14:52:29.938 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:29.937+0000 7fe172742640 1 --2- 172.21.15.43:0/3789215779 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fe16c15a0c0 0x7fe16c0cbb80 unknown :-1 s=CLOSED pgs=2023 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T14:52:29.938 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:29.937+0000 7fe172742640 1 -- 172.21.15.43:0/3789215779 >> 172.21.15.43:0/3789215779 conn(0x7fe16c0aff50 msgr2=0x7fe16c0b0360 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-09T14:52:29.939 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:29.937+0000 7fe172742640 1 -- 172.21.15.43:0/3789215779 shutdown_connections 2023-08-09T14:52:29.939 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:29.937+0000 7fe172742640 1 -- 172.21.15.43:0/3789215779 wait complete. 2023-08-09T14:52:29.939 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:29.939+0000 7fe172742640 1 Processor -- start 2023-08-09T14:52:29.940 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:29.939+0000 7fe172742640 1 -- start start 2023-08-09T14:52:29.941 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:29.941+0000 7fe172742640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fe16c0ce1c0 0x7fe16c070020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T14:52:29.941 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:29.941+0000 7fe172742640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fe16c15a0c0 0x7fe16c0c7680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T14:52:29.942 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:29.941+0000 7fe172742640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fe16c15d510 0x7fe16c0c7bc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T14:52:29.942 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:29.941+0000 7fe172742640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe16c15ea10 con 0x7fe16c0ce1c0 2023-08-09T14:52:29.942 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:29.941+0000 7fe172742640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe16c15f590 con 0x7fe16c15d510 2023-08-09T14:52:29.942 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:29.941+0000 7fe172742640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe16c1f2cd0 con 0x7fe16c15a0c0 2023-08-09T14:52:29.942 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:29.941+0000 7fe16bfff640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fe16c0ce1c0 0x7fe16c070020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T14:52:29.943 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:29.941+0000 7fe16bfff640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fe16c0ce1c0 0x7fe16c070020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:55480/0 (socket says 172.21.15.43:55480) 2023-08-09T14:52:29.944 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:29.941+0000 7fe16bfff640 1 -- 172.21.15.43:0/2676065878 learned_addr learned my addr 172.21.15.43:0/2676065878 (peer_addr_for_me v2:172.21.15.43:0/0) 2023-08-09T14:52:29.944 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:29.942+0000 7fe163fff640 1 --2- 172.21.15.43:0/2676065878 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fe16c15a0c0 0x7fe16c0c7680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T14:52:29.944 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:29.942+0000 7fe170cdf640 1 --2- 172.21.15.43:0/2676065878 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fe16c15d510 0x7fe16c0c7bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T14:52:29.944 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:29.942+0000 7fe16bfff640 1 -- 172.21.15.43:0/2676065878 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fe16c15a0c0 msgr2=0x7fe16c0c7680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T14:52:29.944 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:29.942+0000 7fe16bfff640 1 --2- 172.21.15.43:0/2676065878 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fe16c15a0c0 0x7fe16c0c7680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T14:52:29.945 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:29.942+0000 7fe16bfff640 1 -- 172.21.15.43:0/2676065878 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fe16c15d510 msgr2=0x7fe16c0c7bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T14:52:29.945 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:29.942+0000 7fe16bfff640 1 --2- 172.21.15.43:0/2676065878 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fe16c15d510 0x7fe16c0c7bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T14:52:29.945 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:29.942+0000 7fe16bfff640 1 -- 172.21.15.43:0/2676065878 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe15c0094d0 con 0x7fe16c0ce1c0 2023-08-09T14:52:29.945 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:29.942+0000 7fe16bfff640 1 --2- 172.21.15.43:0/2676065878 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fe16c0ce1c0 0x7fe16c070020 secure :-1 s=READY pgs=2024 cs=0 l=1 rev1=1 crypto rx=0x7fe15c009870 tx=0x7fe15c004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-09T14:52:29.945 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:29.943+0000 7fe1697fa640 1 -- 172.21.15.43:0/2676065878 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe15c03e040 con 0x7fe16c0ce1c0 2023-08-09T14:52:29.945 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:29.943+0000 7fe172742640 1 -- 172.21.15.43:0/2676065878 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe16c0c8290 con 0x7fe16c0ce1c0 2023-08-09T14:52:29.946 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:29.943+0000 7fe1697fa640 1 -- 172.21.15.43:0/2676065878 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe15c039d80 con 0x7fe16c0ce1c0 2023-08-09T14:52:29.946 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:29.943+0000 7fe1697fa640 1 -- 172.21.15.43:0/2676065878 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe15c004270 con 0x7fe16c0ce1c0 2023-08-09T14:52:29.946 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:29.943+0000 7fe172742640 1 -- 172.21.15.43:0/2676065878 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe16c15eed0 con 0x7fe16c0ce1c0 2023-08-09T14:52:29.946 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:29.944+0000 7fe172742640 1 -- 172.21.15.43:0/2676065878 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe16c0739f0 con 0x7fe16c0ce1c0 2023-08-09T14:52:29.947 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:29.947+0000 7fe1697fa640 1 -- 172.21.15.43:0/2676065878 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 88) v1 ==== 88443+0+0 (secure 0 0 0) 0x7fe15c02e070 con 0x7fe16c0ce1c0 2023-08-09T14:52:29.948 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:29.948+0000 7fe1697fa640 1 --2- 172.21.15.43:0/2676065878 >> [v2:172.21.15.203:6832/2959400181,v1:172.21.15.203:6833/2959400181] conn(0x7fe14c06a170 0x7fe14c06c630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T14:52:29.948 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:29.948+0000 7fe1697fa640 1 -- 172.21.15.43:0/2676065878 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(184..184 src has 1..184) v4 ==== 6326+0+0 (secure 0 0 0) 0x7fe15c0ab2f0 con 0x7fe16c0ce1c0 2023-08-09T14:52:29.949 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:29.949+0000 7fe163fff640 1 --2- 172.21.15.43:0/2676065878 >> [v2:172.21.15.203:6832/2959400181,v1:172.21.15.203:6833/2959400181] conn(0x7fe14c06a170 0x7fe14c06c630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T14:52:29.950 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:29.950+0000 7fe163fff640 1 --2- 172.21.15.43:0/2676065878 >> [v2:172.21.15.203:6832/2959400181,v1:172.21.15.203:6833/2959400181] conn(0x7fe14c06a170 0x7fe14c06c630 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7fe16c0ca0e0 tx=0x7fe154007b50 comp rx=0 tx=0).ready entity=mgr.7223 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-09T14:52:29.952 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:29.952+0000 7fe1697fa640 1 -- 172.21.15.43:0/2676065878 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178066 (secure 0 0 0) 0x7fe15c076720 con 0x7fe16c0ce1c0 2023-08-09T14:52:30.187 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:30.186+0000 7fe172742640 1 -- 172.21.15.43:0/2676065878 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"} v 0) v1 -- 0x7fe16c15f270 con 0x7fe16c0ce1c0 2023-08-09T14:52:30.745 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:30.744+0000 7fe1697fa640 1 -- 172.21.15.43:0/2676065878 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]=0 new fs with metadata pool 22 and data pool 23 v83) v1 ==== 174+0+0 (secure 0 0 0) 0x7fe15c079d40 con 0x7fe16c0ce1c0 2023-08-09T14:52:30.756 INFO:teuthology.orchestra.run.smithi043.stderr:new fs with metadata pool 22 and data pool 23 2023-08-09T14:52:30.768 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:30.767+0000 7fe172742640 1 -- 172.21.15.43:0/2676065878 >> [v2:172.21.15.203:6832/2959400181,v1:172.21.15.203:6833/2959400181] conn(0x7fe14c06a170 msgr2=0x7fe14c06c630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T14:52:30.769 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:30.767+0000 7fe172742640 1 --2- 172.21.15.43:0/2676065878 >> [v2:172.21.15.203:6832/2959400181,v1:172.21.15.203:6833/2959400181] conn(0x7fe14c06a170 0x7fe14c06c630 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7fe16c0ca0e0 tx=0x7fe154007b50 comp rx=0 tx=0).stop 2023-08-09T14:52:30.769 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:30.767+0000 7fe172742640 1 -- 172.21.15.43:0/2676065878 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fe16c0ce1c0 msgr2=0x7fe16c070020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T14:52:30.770 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:30.767+0000 7fe172742640 1 --2- 172.21.15.43:0/2676065878 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fe16c0ce1c0 0x7fe16c070020 secure :-1 s=READY pgs=2024 cs=0 l=1 rev1=1 crypto rx=0x7fe15c009870 tx=0x7fe15c004730 comp rx=0 tx=0).stop 2023-08-09T14:52:30.770 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:30.767+0000 7fe172742640 1 -- 172.21.15.43:0/2676065878 shutdown_connections 2023-08-09T14:52:30.770 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:30.767+0000 7fe172742640 1 --2- 172.21.15.43:0/2676065878 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fe16c15d510 0x7fe16c0c7bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T14:52:30.771 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:30.767+0000 7fe172742640 1 --2- 172.21.15.43:0/2676065878 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fe16c15a0c0 0x7fe16c0c7680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T14:52:30.771 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:30.767+0000 7fe172742640 1 --2- 172.21.15.43:0/2676065878 >> [v2:172.21.15.203:6832/2959400181,v1:172.21.15.203:6833/2959400181] conn(0x7fe14c06a170 0x7fe14c06c630 unknown :-1 s=CLOSED pgs=59 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T14:52:30.771 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:30.767+0000 7fe172742640 1 --2- 172.21.15.43:0/2676065878 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fe16c0ce1c0 0x7fe16c070020 unknown :-1 s=CLOSED pgs=2024 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T14:52:30.772 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:30.767+0000 7fe172742640 1 -- 172.21.15.43:0/2676065878 >> 172.21.15.43:0/2676065878 conn(0x7fe16c0aff50 msgr2=0x7fe16c0b0330 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-09T14:52:30.772 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:30.768+0000 7fe172742640 1 -- 172.21.15.43:0/2676065878 shutdown_connections 2023-08-09T14:52:30.772 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:30.768+0000 7fe172742640 1 -- 172.21.15.43:0/2676065878 wait complete. 2023-08-09T14:52:30.789 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-08-09T14:52:30.984 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:30.984+0000 7fe097271640 1 Processor -- start 2023-08-09T14:52:30.984 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:30.984+0000 7fe097271640 1 -- start start 2023-08-09T14:52:30.986 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:30.985+0000 7fe097271640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fe0900ce720 0x7fe0900ceb00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T14:52:30.986 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:30.985+0000 7fe097271640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7fe09015ecc0 0x7fe09006d2a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T14:52:30.986 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:30.985+0000 7fe097271640 1 -- --> v1:172.21.15.43:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe0900cd8a0 con 0x7fe0900ce080 2023-08-09T14:52:30.986 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:30.985+0000 7fe097271640 1 -- --> v2:172.21.15.43:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe0900cf350 con 0x7fe09015ecc0 2023-08-09T14:52:30.986 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:30.985+0000 7fe097271640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe0900cf040 con 0x7fe0900ce720 2023-08-09T14:52:30.987 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:30.986+0000 7fe09580e640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7fe09015ecc0 0x7fe09006d2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T14:52:30.987 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:30.986+0000 7fe09480c640 1 --1- >> v1:172.21.15.43:6789/0 conn(0x7fe0900ce080 0x7fe09015e5b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.43:6789/0 says I am v1:172.21.15.43:50452/0 (socket says 172.21.15.43:50452) 2023-08-09T14:52:30.987 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:30.986+0000 7fe09480c640 1 -- 172.21.15.43:0/2340949412 learned_addr learned my addr 172.21.15.43:0/2340949412 (peer_addr_for_me v1:172.21.15.43:0/0) 2023-08-09T14:52:30.987 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:30.986+0000 7fe09500d640 1 --2- 172.21.15.43:0/2340949412 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fe0900ce720 0x7fe0900ceb00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T14:52:30.988 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:30.986+0000 7fe09580e640 1 -- 172.21.15.43:0/2340949412 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fe0900ce720 msgr2=0x7fe0900ceb00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-09T14:52:30.988 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:30.986+0000 7fe09580e640 1 --2- 172.21.15.43:0/2340949412 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fe0900ce720 0x7fe0900ceb00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T14:52:30.988 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:30.986+0000 7fe09580e640 1 -- 172.21.15.43:0/2340949412 >> v1:172.21.15.43:6789/0 conn(0x7fe0900ce080 legacy=0x7fe09015e5b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T14:52:30.988 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:30.986+0000 7fe09580e640 1 -- 172.21.15.43:0/2340949412 --> v2:172.21.15.43:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe090071a70 con 0x7fe09015ecc0 2023-08-09T14:52:30.988 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:30.986+0000 7fe083fff640 1 -- 172.21.15.43:0/2340949412 <== mon.0 v1:172.21.15.43:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3736368438 0 0) 0x7fe0900cd8a0 con 0x7fe0900ce080 2023-08-09T14:52:30.989 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:30.988+0000 7fe09580e640 1 --2- 172.21.15.43:0/2340949412 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fe09015ecc0 0x7fe09006d2a0 secure :-1 s=READY pgs=2026 cs=0 l=1 rev1=1 crypto rx=0x7fe088009d90 tx=0x7fe08802bd80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9ccb250346abe9e3 server_cookie=0 in_seq=0 out_seq=0 2023-08-09T14:52:30.989 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:30.988+0000 7fe083fff640 1 -- 172.21.15.43:0/2340949412 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe08802cb30 con 0x7fe09015ecc0 2023-08-09T14:52:30.989 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:30.988+0000 7fe083fff640 1 -- 172.21.15.43:0/2340949412 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe08802cc90 con 0x7fe09015ecc0 2023-08-09T14:52:30.990 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:30.988+0000 7fe083fff640 1 -- 172.21.15.43:0/2340949412 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe088034440 con 0x7fe09015ecc0 2023-08-09T14:52:30.990 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:30.988+0000 7fe097271640 1 -- 172.21.15.43:0/2340949412 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fe09015ecc0 msgr2=0x7fe09006d2a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T14:52:30.990 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:30.988+0000 7fe097271640 1 --2- 172.21.15.43:0/2340949412 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fe09015ecc0 0x7fe09006d2a0 secure :-1 s=READY pgs=2026 cs=0 l=1 rev1=1 crypto rx=0x7fe088009d90 tx=0x7fe08802bd80 comp rx=0 tx=0).stop 2023-08-09T14:52:30.990 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:30.989+0000 7fe097271640 1 -- 172.21.15.43:0/2340949412 shutdown_connections 2023-08-09T14:52:30.990 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:30.989+0000 7fe097271640 1 --2- 172.21.15.43:0/2340949412 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fe09015ecc0 0x7fe09006d2a0 unknown :-1 s=CLOSED pgs=2026 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T14:52:30.990 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:30.989+0000 7fe097271640 1 --2- 172.21.15.43:0/2340949412 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fe0900ce720 0x7fe0900ceb00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T14:52:30.990 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:30.989+0000 7fe097271640 1 -- 172.21.15.43:0/2340949412 >> 172.21.15.43:0/2340949412 conn(0x7fe0900ccd80 msgr2=0x7fe0900cd190 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-09T14:52:30.991 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:30.989+0000 7fe097271640 1 -- 172.21.15.43:0/2340949412 shutdown_connections 2023-08-09T14:52:30.991 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:30.989+0000 7fe097271640 1 -- 172.21.15.43:0/2340949412 wait complete. 2023-08-09T14:52:30.991 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:30.990+0000 7fe097271640 1 Processor -- start 2023-08-09T14:52:30.991 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:30.991+0000 7fe097271640 1 -- start start 2023-08-09T14:52:30.992 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:30.992+0000 7fe097271640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fe0900ce080 0x7fe0900c56d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T14:52:30.992 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:30.992+0000 7fe097271640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fe0900ce720 0x7fe0900b58c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T14:52:30.993 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:30.992+0000 7fe097271640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fe09015ecc0 0x7fe0901f2770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T14:52:30.993 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:30.992+0000 7fe097271640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe0900b5e00 con 0x7fe0900ce080 2023-08-09T14:52:30.993 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:30.992+0000 7fe097271640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe0901f2cb0 con 0x7fe0900ce720 2023-08-09T14:52:30.993 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:30.992+0000 7fe097271640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe0900afc30 con 0x7fe09015ecc0 2023-08-09T14:52:30.993 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:30.992+0000 7fe09500d640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fe0900ce080 0x7fe0900c56d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T14:52:30.993 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:30.992+0000 7fe09500d640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fe0900ce080 0x7fe0900c56d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:55490/0 (socket says 172.21.15.43:55490) 2023-08-09T14:52:30.993 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:30.992+0000 7fe09500d640 1 -- 172.21.15.43:0/3070295208 learned_addr learned my addr 172.21.15.43:0/3070295208 (peer_addr_for_me v2:172.21.15.43:0/0) 2023-08-09T14:52:30.994 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:30.992+0000 7fe09500d640 1 -- 172.21.15.43:0/3070295208 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fe09015ecc0 msgr2=0x7fe0901f2770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T14:52:30.994 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:30.992+0000 7fe09500d640 1 --2- 172.21.15.43:0/3070295208 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fe09015ecc0 0x7fe0901f2770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T14:52:30.995 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:30.992+0000 7fe09500d640 1 -- 172.21.15.43:0/3070295208 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fe0900ce720 msgr2=0x7fe0900b58c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T14:52:30.995 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:30.992+0000 7fe09500d640 1 --2- 172.21.15.43:0/3070295208 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fe0900ce720 0x7fe0900b58c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T14:52:30.995 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:30.992+0000 7fe09500d640 1 -- 172.21.15.43:0/3070295208 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe088009a70 con 0x7fe0900ce080 2023-08-09T14:52:30.995 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:30.993+0000 7fe09500d640 1 --2- 172.21.15.43:0/3070295208 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fe0900ce080 0x7fe0900c56d0 secure :-1 s=READY pgs=2027 cs=0 l=1 rev1=1 crypto rx=0x7fe08c00d7d0 tx=0x7fe08c00dc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-09T14:52:30.995 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:30.993+0000 7fe081ffb640 1 -- 172.21.15.43:0/3070295208 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe08c0096c0 con 0x7fe0900ce080 2023-08-09T14:52:30.995 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:30.993+0000 7fe081ffb640 1 -- 172.21.15.43:0/3070295208 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe08c009820 con 0x7fe0900ce080 2023-08-09T14:52:30.995 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:30.993+0000 7fe081ffb640 1 -- 172.21.15.43:0/3070295208 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe08c009bd0 con 0x7fe0900ce080 2023-08-09T14:52:30.996 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:30.993+0000 7fe097271640 1 -- 172.21.15.43:0/3070295208 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe0900aff10 con 0x7fe0900ce080 2023-08-09T14:52:30.996 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:30.993+0000 7fe097271640 1 -- 172.21.15.43:0/3070295208 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe0900b04d0 con 0x7fe0900ce080 2023-08-09T14:52:30.996 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:30.995+0000 7fe097271640 1 -- 172.21.15.43:0/3070295208 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe0900739f0 con 0x7fe0900ce080 2023-08-09T14:52:30.996 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:30.996+0000 7fe081ffb640 1 -- 172.21.15.43:0/3070295208 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 88) v1 ==== 88443+0+0 (secure 0 0 0) 0x7fe08c009d30 con 0x7fe0900ce080 2023-08-09T14:52:30.997 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:30.996+0000 7fe081ffb640 1 --2- 172.21.15.43:0/3070295208 >> [v2:172.21.15.203:6832/2959400181,v1:172.21.15.203:6833/2959400181] conn(0x7fe07406a1a0 0x7fe07406c660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T14:52:30.997 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:30.997+0000 7fe081ffb640 1 -- 172.21.15.43:0/3070295208 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(185..185 src has 1..185) v4 ==== 6426+0+0 (secure 0 0 0) 0x7fe08c089210 con 0x7fe0900ce080 2023-08-09T14:52:30.997 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:30.997+0000 7fe09480c640 1 --2- 172.21.15.43:0/3070295208 >> [v2:172.21.15.203:6832/2959400181,v1:172.21.15.203:6833/2959400181] conn(0x7fe07406a1a0 0x7fe07406c660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T14:52:30.998 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:30.998+0000 7fe09480c640 1 --2- 172.21.15.43:0/3070295208 >> [v2:172.21.15.203:6832/2959400181,v1:172.21.15.203:6833/2959400181] conn(0x7fe07406a1a0 0x7fe07406c660 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7fe084007fd0 tx=0x7fe084007960 comp rx=0 tx=0).ready entity=mgr.7223 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-09T14:52:31.001 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:31.001+0000 7fe081ffb640 1 -- 172.21.15.43:0/3070295208 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178066 (secure 0 0 0) 0x7fe08c054550 con 0x7fe0900ce080 2023-08-09T14:52:31.196 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:31.195+0000 7fe097271640 1 -- 172.21.15.43:0/3070295208 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fe0900b2be0 con 0x7fe0900ce080 2023-08-09T14:52:31.198 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:31.198+0000 7fe081ffb640 1 -- 172.21.15.43:0/3070295208 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v185) v1 ==== 74+0+18571 (secure 0 0 0) 0x7fe08c057b70 con 0x7fe0900ce080 2023-08-09T14:52:31.199 INFO:teuthology.orchestra.run.smithi043.stdout: 2023-08-09T14:52:31.199 INFO:teuthology.orchestra.run.smithi043.stdout:{"epoch":185,"fsid":"daac87ee-1337-4250-a955-aea4e408a39b","created":"2023-08-09T13:33:41.701198+0000","modified":"2023-08-09T14:52:30.188055+0000","last_up_change":"2023-08-09T13:33:53.783453+0000","last_in_change":"2023-08-09T13:33:45.148904+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":23,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":"rbd","create_time":"2023-08-09T13:33:54.365137+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":".mgr","create_time":"2023-08-09T13:33:54.831803+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":22,"pool_name":"cephfs_metadata","create_time":"2023-08-09T14:52:28.569334+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"185","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":23,"pool_name":"cephfs_data","create_time":"2023-08-09T14:52:29.209973+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"185","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"cbd49ecd-e896-4fbd-864e-5c37a829b165","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":184,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6800","nonce":502667335},{"type":"v1","addr":"172.21.15.43:6801","nonce":502667335}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6802","nonce":502667335},{"type":"v1","addr":"172.21.15.43:6803","nonce":502667335}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6806","nonce":502667335},{"type":"v1","addr":"172.21.15.43:6807","nonce":502667335}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6804","nonce":502667335},{"type":"v1","addr":"172.21.15.43:6805","nonce":502667335}]},"public_addr":"172.21.15.43:6801/502667335","cluster_addr":"172.21.15.43:6803/502667335","heartbeat_back_addr":"172.21.15.43:6807/502667335","heartbeat_front_addr":"172.21.15.43:6805/502667335","state":["exists","up"]},{"osd":1,"uuid":"75c7e0fe-1ec7-4fe8-a2f2-3661a65b8500","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":184,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6808","nonce":3289134610},{"type":"v1","addr":"172.21.15.43:6809","nonce":3289134610}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6810","nonce":3289134610},{"type":"v1","addr":"172.21.15.43:6812","nonce":3289134610}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6821","nonce":3289134610},{"type":"v1","addr":"172.21.15.43:6824","nonce":3289134610}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6815","nonce":3289134610},{"type":"v1","addr":"172.21.15.43:6818","nonce":3289134610}]},"public_addr":"172.21.15.43:6809/3289134610","cluster_addr":"172.21.15.43:6812/3289134610","heartbeat_back_addr":"172.21.15.43:6824/3289134610","heartbeat_front_addr":"172.21.15.43:6818/3289134610","state":["exists","up"]},{"osd":2,"uuid":"bb5c41e9-7f5f-4125-9b01-3bb8bfc2ea83","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":184,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6811","nonce":4079709736},{"type":"v1","addr":"172.21.15.43:6814","nonce":4079709736}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6817","nonce":4079709736},{"type":"v1","addr":"172.21.15.43:6820","nonce":4079709736}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6829","nonce":4079709736},{"type":"v1","addr":"172.21.15.43:6831","nonce":4079709736}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6823","nonce":4079709736},{"type":"v1","addr":"172.21.15.43:6826","nonce":4079709736}]},"public_addr":"172.21.15.43:6814/4079709736","cluster_addr":"172.21.15.43:6820/4079709736","heartbeat_back_addr":"172.21.15.43:6831/4079709736","heartbeat_front_addr":"172.21.15.43:6826/4079709736","state":["exists","up"]},{"osd":3,"uuid":"bfe4c8b4-694a-465c-87e6-2b2aab852613","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":184,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6813","nonce":1709538759},{"type":"v1","addr":"172.21.15.43:6816","nonce":1709538759}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6819","nonce":1709538759},{"type":"v1","addr":"172.21.15.43:6822","nonce":1709538759}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6828","nonce":1709538759},{"type":"v1","addr":"172.21.15.43:6830","nonce":1709538759}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6825","nonce":1709538759},{"type":"v1","addr":"172.21.15.43:6827","nonce":1709538759}]},"public_addr":"172.21.15.43:6816/1709538759","cluster_addr":"172.21.15.43:6822/1709538759","heartbeat_back_addr":"172.21.15.43:6830/1709538759","heartbeat_front_addr":"172.21.15.43:6827/1709538759","state":["exists","up"]},{"osd":4,"uuid":"fa2e8a77-aba1-4756-853e-0d174a5bdeba","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":184,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6800","nonce":4196866481},{"type":"v1","addr":"172.21.15.203:6801","nonce":4196866481}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6802","nonce":4196866481},{"type":"v1","addr":"172.21.15.203:6803","nonce":4196866481}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6806","nonce":4196866481},{"type":"v1","addr":"172.21.15.203:6807","nonce":4196866481}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6804","nonce":4196866481},{"type":"v1","addr":"172.21.15.203:6805","nonce":4196866481}]},"public_addr":"172.21.15.203:6801/4196866481","cluster_addr":"172.21.15.203:6803/4196866481","heartbeat_back_addr":"172.21.15.203:6807/4196866481","heartbeat_front_addr":"172.21.15.203:6805/4196866481","state":["exists","up"]},{"osd":5,"uuid":"81811e4b-4d24-4f51-a82c-f1d700a2a00d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":184,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6816","nonce":3059290081},{"type":"v1","addr":"172.21.15.203:6817","nonce":3059290081}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6818","nonce":3059290081},{"type":"v1","addr":"172.21.15.203:6819","nonce":3059290081}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6822","nonce":3059290081},{"type":"v1","addr":"172.21.15.203:6823","nonce":3059290081}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6820","nonce":3059290081},{"type":"v1","addr":"172.21.15.203:6821","nonce":3059290081}]},"public_addr":"172.21.15.203:6817/3059290081","cluster_addr":"172.21.15.203:6819/3059290081","heartbeat_back_addr":"172.21.15.203:6823/3059290081","heartbeat_front_addr":"172.21.15.203:6821/3059290081","state":["exists","up"]},{"osd":6,"uuid":"307bac4d-6a5b-443d-af75-56d695d7674f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":184,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6824","nonce":512155569},{"type":"v1","addr":"172.21.15.203:6825","nonce":512155569}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6826","nonce":512155569},{"type":"v1","addr":"172.21.15.203:6827","nonce":512155569}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6830","nonce":512155569},{"type":"v1","addr":"172.21.15.203:6831","nonce":512155569}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6828","nonce":512155569},{"type":"v1","addr":"172.21.15.203:6829","nonce":512155569}]},"public_addr":"172.21.15.203:6825/512155569","cluster_addr":"172.21.15.203:6827/512155569","heartbeat_back_addr":"172.21.15.203:6831/512155569","heartbeat_front_addr":"172.21.15.203:6829/512155569","state":["exists","up"]},{"osd":7,"uuid":"a95dc7ca-9f1b-4295-af70-68a22a2d7d5e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":184,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6808","nonce":3331450438},{"type":"v1","addr":"172.21.15.203:6809","nonce":3331450438}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6810","nonce":3331450438},{"type":"v1","addr":"172.21.15.203:6811","nonce":3331450438}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6814","nonce":3331450438},{"type":"v1","addr":"172.21.15.203:6815","nonce":3331450438}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6812","nonce":3331450438},{"type":"v1","addr":"172.21.15.203:6813","nonce":3331450438}]},"public_addr":"172.21.15.203:6809/3331450438","cluster_addr":"172.21.15.203:6811/3331450438","heartbeat_back_addr":"172.21.15.203:6815/3331450438","heartbeat_front_addr":"172.21.15.203:6813/3331450438","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-09T13:33:50.547652+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-09T13:33:50.589469+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-09T13:33:50.561724+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-09T13:33:50.599650+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-09T13:33:50.411855+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-09T13:33:50.219681+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-09T13:33:50.504664+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-09T13:33:50.441248+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":7,"snaps":[{"begin":104,"length":1},{"begin":221,"length":1}]}],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-08-09T14:52:31.201 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:31.201+0000 7fe097271640 1 -- 172.21.15.43:0/3070295208 >> [v2:172.21.15.203:6832/2959400181,v1:172.21.15.203:6833/2959400181] conn(0x7fe07406a1a0 msgr2=0x7fe07406c660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T14:52:31.202 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:31.201+0000 7fe097271640 1 --2- 172.21.15.43:0/3070295208 >> [v2:172.21.15.203:6832/2959400181,v1:172.21.15.203:6833/2959400181] conn(0x7fe07406a1a0 0x7fe07406c660 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7fe084007fd0 tx=0x7fe084007960 comp rx=0 tx=0).stop 2023-08-09T14:52:31.202 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:31.201+0000 7fe097271640 1 -- 172.21.15.43:0/3070295208 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fe0900ce080 msgr2=0x7fe0900c56d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T14:52:31.202 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:31.201+0000 7fe097271640 1 --2- 172.21.15.43:0/3070295208 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fe0900ce080 0x7fe0900c56d0 secure :-1 s=READY pgs=2027 cs=0 l=1 rev1=1 crypto rx=0x7fe08c00d7d0 tx=0x7fe08c00dc90 comp rx=0 tx=0).stop 2023-08-09T14:52:31.202 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:31.201+0000 7fe097271640 1 -- 172.21.15.43:0/3070295208 shutdown_connections 2023-08-09T14:52:31.202 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:31.201+0000 7fe097271640 1 --2- 172.21.15.43:0/3070295208 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fe09015ecc0 0x7fe0901f2770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T14:52:31.203 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:31.201+0000 7fe097271640 1 --2- 172.21.15.43:0/3070295208 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fe0900ce720 0x7fe0900b58c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T14:52:31.203 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:31.201+0000 7fe097271640 1 --2- 172.21.15.43:0/3070295208 >> [v2:172.21.15.203:6832/2959400181,v1:172.21.15.203:6833/2959400181] conn(0x7fe07406a1a0 0x7fe07406c660 unknown :-1 s=CLOSED pgs=60 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T14:52:31.203 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:31.201+0000 7fe097271640 1 --2- 172.21.15.43:0/3070295208 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fe0900ce080 0x7fe0900c56d0 unknown :-1 s=CLOSED pgs=2027 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T14:52:31.203 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:31.201+0000 7fe097271640 1 -- 172.21.15.43:0/3070295208 >> 172.21.15.43:0/3070295208 conn(0x7fe0900ccd80 msgr2=0x7fe0900ad4b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-09T14:52:31.203 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:31.201+0000 7fe097271640 1 -- 172.21.15.43:0/3070295208 shutdown_connections 2023-08-09T14:52:31.203 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:31.201+0000 7fe097271640 1 -- 172.21.15.43:0/3070295208 wait complete. 2023-08-09T14:52:31.221 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-08-09T14:52:31.442 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:31.442+0000 7f714c048640 1 Processor -- start 2023-08-09T14:52:31.442 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:31.442+0000 7f714c048640 1 -- start start 2023-08-09T14:52:31.444 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:31.443+0000 7f714c048640 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7f71440cee50 0x7f714416c120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T14:52:31.444 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:31.443+0000 7f714c048640 1 -- --> v1:172.21.15.43:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f71440af660 con 0x7f714406ab40 2023-08-09T14:52:31.444 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:31.444+0000 7f714c048640 1 -- --> v1:172.21.15.203:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f714406b6e0 con 0x7f71440cfeb0 2023-08-09T14:52:31.445 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:31.444+0000 7f714c048640 1 -- --> v2:172.21.15.203:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f71440af990 con 0x7f71440cee50 2023-08-09T14:52:31.445 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:31.444+0000 7f7149de4640 1 --1- >> v1:172.21.15.43:6789/0 conn(0x7f714406ab40 0x7f714406af20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.43:6789/0 says I am v1:172.21.15.43:50462/0 (socket says 172.21.15.43:50462) 2023-08-09T14:52:31.445 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:31.444+0000 7f7149de4640 1 -- 172.21.15.43:0/841241300 learned_addr learned my addr 172.21.15.43:0/841241300 (peer_addr_for_me v1:172.21.15.43:0/0) 2023-08-09T14:52:31.446 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:31.444+0000 7f714a5e5640 1 --2- 172.21.15.43:0/841241300 >> v2:172.21.15.203:3300/0 conn(0x7f71440cee50 0x7f714416c120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T14:52:31.447 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:31.444+0000 7f7148de2640 1 -- 172.21.15.43:0/841241300 <== mon.0 v1:172.21.15.43:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 56850571 0 0) 0x7f71440af660 con 0x7f714406ab40 2023-08-09T14:52:31.447 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:31.444+0000 7f7148de2640 1 -- 172.21.15.43:0/841241300 --> v1:172.21.15.43:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f7130003520 con 0x7f714406ab40 2023-08-09T14:52:31.447 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:31.445+0000 7f7148de2640 1 -- 172.21.15.43:0/841241300 <== mon.0 v1:172.21.15.43:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2179321557 0 0) 0x7f7130003520 con 0x7f714406ab40 2023-08-09T14:52:31.447 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:31.445+0000 7f7148de2640 1 -- 172.21.15.43:0/841241300 >> v2:172.21.15.203:3300/0 conn(0x7f71440cee50 msgr2=0x7f714416c120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-09T14:52:31.447 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:31.445+0000 7f7148de2640 1 --2- 172.21.15.43:0/841241300 >> v2:172.21.15.203:3300/0 conn(0x7f71440cee50 0x7f714416c120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T14:52:31.447 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:31.445+0000 7f7148de2640 1 -- 172.21.15.43:0/841241300 >> v1:172.21.15.203:6789/0 conn(0x7f71440cfeb0 legacy=0x7f714406d2a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T14:52:31.448 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:31.445+0000 7f7148de2640 1 -- 172.21.15.43:0/841241300 --> v1:172.21.15.43:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f714406d9b0 con 0x7f714406ab40 2023-08-09T14:52:31.448 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:31.445+0000 7f7148de2640 1 -- 172.21.15.43:0/841241300 <== mon.0 v1:172.21.15.43:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1285150740 0 0) 0x7f7140001b70 con 0x7f714406ab40 2023-08-09T14:52:31.448 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:31.445+0000 7f7148de2640 1 -- 172.21.15.43:0/841241300 >> v1:172.21.15.43:6789/0 conn(0x7f714406ab40 legacy=0x7f714406af20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T14:52:31.448 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:31.445+0000 7f7148de2640 1 --2- 172.21.15.43:0/841241300 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f71300039e0 0x7f7130003e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T14:52:31.448 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:31.445+0000 7f7148de2640 1 --2- 172.21.15.43:0/841241300 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f71300047c0 0x7f7130004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T14:52:31.448 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:31.445+0000 7f7148de2640 1 --2- 172.21.15.43:0/841241300 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f71300052a0 0x7f713000b980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T14:52:31.448 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:31.445+0000 7f7148de2640 1 -- 172.21.15.43:0/841241300 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f71440af990 con 0x7f71300047c0 2023-08-09T14:52:31.448 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:31.445+0000 7f7148de2640 1 -- 172.21.15.43:0/841241300 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f713000c150 con 0x7f71300039e0 2023-08-09T14:52:31.449 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:31.445+0000 7f7148de2640 1 -- 172.21.15.43:0/841241300 --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f713000c310 con 0x7f71300052a0 2023-08-09T14:52:31.449 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:31.445+0000 7f71495e3640 1 --2- 172.21.15.43:0/841241300 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f71300047c0 0x7f7130004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T14:52:31.449 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:31.446+0000 7f7149de4640 1 --2- 172.21.15.43:0/841241300 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f71300039e0 0x7f7130003e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T14:52:31.449 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:31.446+0000 7f714a5e5640 1 --2- 172.21.15.43:0/841241300 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f71300052a0 0x7f713000b980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T14:52:31.449 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:31.446+0000 7f71495e3640 1 -- 172.21.15.43:0/841241300 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f71300052a0 msgr2=0x7f713000b980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-09T14:52:31.449 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:31.446+0000 7f71495e3640 1 --2- 172.21.15.43:0/841241300 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f71300052a0 0x7f713000b980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T14:52:31.450 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:31.446+0000 7f71495e3640 1 -- 172.21.15.43:0/841241300 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f71300039e0 msgr2=0x7f7130003e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-09T14:52:31.450 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:31.446+0000 7f71495e3640 1 --2- 172.21.15.43:0/841241300 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f71300039e0 0x7f7130003e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T14:52:31.450 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:31.446+0000 7f71495e3640 1 -- 172.21.15.43:0/841241300 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7130003520 con 0x7f71300047c0 2023-08-09T14:52:31.450 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:31.446+0000 7f714a5e5640 1 -- 172.21.15.43:0/841241300 reap_dead start 2023-08-09T14:52:31.451 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:31.447+0000 7f71495e3640 1 --2- 172.21.15.43:0/841241300 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f71300047c0 0x7f7130004c40 secure :-1 s=READY pgs=2029 cs=0 l=1 rev1=1 crypto rx=0x7f71440b1da0 tx=0x7f71300057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=117a9a76ca803c server_cookie=0 in_seq=0 out_seq=0 2023-08-09T14:52:31.451 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:31.447+0000 7f7148de2640 1 -- 172.21.15.43:0/841241300 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f71340052d0 con 0x7f71300047c0 2023-08-09T14:52:31.452 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:31.447+0000 7f7148de2640 1 -- 172.21.15.43:0/841241300 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f713402e050 con 0x7f71300047c0 2023-08-09T14:52:31.452 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:31.447+0000 7f7148de2640 1 -- 172.21.15.43:0/841241300 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7134003830 con 0x7f71300047c0 2023-08-09T14:52:31.452 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:31.448+0000 7f714c048640 1 -- 172.21.15.43:0/841241300 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f71300047c0 msgr2=0x7f7130004c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T14:52:31.452 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:31.448+0000 7f714c048640 1 --2- 172.21.15.43:0/841241300 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f71300047c0 0x7f7130004c40 secure :-1 s=READY pgs=2029 cs=0 l=1 rev1=1 crypto rx=0x7f71440b1da0 tx=0x7f71300057f0 comp rx=0 tx=0).stop 2023-08-09T14:52:31.452 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:31.448+0000 7f714c048640 1 -- 172.21.15.43:0/841241300 shutdown_connections 2023-08-09T14:52:31.452 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:31.448+0000 7f714c048640 1 --2- 172.21.15.43:0/841241300 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f71300047c0 0x7f7130004c40 unknown :-1 s=CLOSED pgs=2029 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T14:52:31.452 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:31.448+0000 7f714c048640 1 -- 172.21.15.43:0/841241300 >> 172.21.15.43:0/841241300 conn(0x7f71440ccd80 msgr2=0x7f71440cd190 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-09T14:52:31.453 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:31.448+0000 7f714c048640 1 -- 172.21.15.43:0/841241300 shutdown_connections 2023-08-09T14:52:31.453 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:31.448+0000 7f714c048640 1 -- 172.21.15.43:0/841241300 wait complete. 2023-08-09T14:52:31.453 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:31.450+0000 7f714c048640 1 Processor -- start 2023-08-09T14:52:31.453 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:31.450+0000 7f714c048640 1 -- start start 2023-08-09T14:52:31.453 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:31.451+0000 7f714c048640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f71300047c0 0x7f71440b58b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T14:52:31.453 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:31.451+0000 7f714c048640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f71440c8680 0x7f71440cbe00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T14:52:31.453 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:31.451+0000 7f714c048640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f71440c8b00 0x7f71440cc340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T14:52:31.453 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:31.451+0000 7f714c048640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f71440c8260 con 0x7f71440c8b00 2023-08-09T14:52:31.454 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:31.451+0000 7f714c048640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f71440c7340 con 0x7f71300047c0 2023-08-09T14:52:31.454 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:31.451+0000 7f714c048640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f71440c75e0 con 0x7f71440c8680 2023-08-09T14:52:31.456 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:31.451+0000 7f714a5e5640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f71440c8b00 0x7f71440cc340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T14:52:31.456 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:31.451+0000 7f714a5e5640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f71440c8b00 0x7f71440cc340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:55514/0 (socket says 172.21.15.43:55514) 2023-08-09T14:52:31.456 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:31.451+0000 7f714a5e5640 1 -- 172.21.15.43:0/565760239 learned_addr learned my addr 172.21.15.43:0/565760239 (peer_addr_for_me v2:172.21.15.43:0/0) 2023-08-09T14:52:31.456 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:31.452+0000 7f7149de4640 1 --2- 172.21.15.43:0/565760239 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f71300047c0 0x7f71440b58b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T14:52:31.456 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:31.452+0000 7f71495e3640 1 --2- 172.21.15.43:0/565760239 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f71440c8680 0x7f71440cbe00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T14:52:31.457 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:31.452+0000 7f714a5e5640 1 -- 172.21.15.43:0/565760239 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f71440c8680 msgr2=0x7f71440cbe00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T14:52:31.457 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:31.452+0000 7f714a5e5640 1 --2- 172.21.15.43:0/565760239 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f71440c8680 0x7f71440cbe00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T14:52:31.457 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:31.452+0000 7f714a5e5640 1 -- 172.21.15.43:0/565760239 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f71300047c0 msgr2=0x7f71440b58b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T14:52:31.457 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:31.452+0000 7f714a5e5640 1 --2- 172.21.15.43:0/565760239 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f71300047c0 0x7f71440b58b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T14:52:31.457 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:31.452+0000 7f714a5e5640 1 -- 172.21.15.43:0/565760239 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f714406a630 con 0x7f71440c8b00 2023-08-09T14:52:31.457 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:31.452+0000 7f7149de4640 1 --2- 172.21.15.43:0/565760239 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f71300047c0 0x7f71440b58b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-08-09T14:52:31.457 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:31.452+0000 7f714a5e5640 1 --2- 172.21.15.43:0/565760239 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f71440c8b00 0x7f71440cc340 secure :-1 s=READY pgs=2030 cs=0 l=1 rev1=1 crypto rx=0x7f7144072640 tx=0x7f713c010c20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-09T14:52:31.458 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:31.452+0000 7f713a7fc640 1 -- 172.21.15.43:0/565760239 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f713c00e8a0 con 0x7f71440c8b00 2023-08-09T14:52:31.458 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:31.452+0000 7f713a7fc640 1 -- 172.21.15.43:0/565760239 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f713c00ea20 con 0x7f71440c8b00 2023-08-09T14:52:31.458 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:31.452+0000 7f713a7fc640 1 -- 172.21.15.43:0/565760239 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f713c00edd0 con 0x7f71440c8b00 2023-08-09T14:52:31.458 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:31.452+0000 7f714c048640 1 -- 172.21.15.43:0/565760239 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f71440c78c0 con 0x7f71440c8b00 2023-08-09T14:52:31.458 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:31.453+0000 7f714c048640 1 -- 172.21.15.43:0/565760239 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f71440c7e80 con 0x7f71440c8b00 2023-08-09T14:52:31.458 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:31.454+0000 7f714c048640 1 -- 172.21.15.43:0/565760239 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f71440c83a0 con 0x7f71440c8b00 2023-08-09T14:52:31.458 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:31.455+0000 7f713a7fc640 1 -- 172.21.15.43:0/565760239 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 88) v1 ==== 88443+0+0 (secure 0 0 0) 0x7f713c021460 con 0x7f71440c8b00 2023-08-09T14:52:31.458 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:31.455+0000 7f713a7fc640 1 --2- 172.21.15.43:0/565760239 >> [v2:172.21.15.203:6832/2959400181,v1:172.21.15.203:6833/2959400181] conn(0x7f712806a170 0x7f712806c630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T14:52:31.459 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:31.456+0000 7f713a7fc640 1 -- 172.21.15.43:0/565760239 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(185..185 src has 1..185) v4 ==== 6426+0+0 (secure 0 0 0) 0x7f713c088670 con 0x7f71440c8b00 2023-08-09T14:52:31.459 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:31.456+0000 7f7149de4640 1 --2- 172.21.15.43:0/565760239 >> [v2:172.21.15.203:6832/2959400181,v1:172.21.15.203:6833/2959400181] conn(0x7f712806a170 0x7f712806c630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T14:52:31.459 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:31.457+0000 7f7149de4640 1 --2- 172.21.15.43:0/565760239 >> [v2:172.21.15.203:6832/2959400181,v1:172.21.15.203:6833/2959400181] conn(0x7f712806a170 0x7f712806c630 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f7144068600 tx=0x7f7140009040 comp rx=0 tx=0).ready entity=mgr.7223 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-09T14:52:31.460 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:31.460+0000 7f713a7fc640 1 -- 172.21.15.43:0/565760239 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178066 (secure 0 0 0) 0x7f71440c83a0 con 0x7f71440c8b00 2023-08-09T14:52:31.679 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:31.678+0000 7f714c048640 1 -- 172.21.15.43:0/565760239 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f71440c83a0 con 0x7f71440c8b00 2023-08-09T14:52:31.681 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:31.680+0000 7f713a7fc640 1 -- 172.21.15.43:0/565760239 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v185) v1 ==== 74+0+18571 (secure 0 0 0) 0x7f71440c83a0 con 0x7f71440c8b00 2023-08-09T14:52:31.682 INFO:teuthology.orchestra.run.smithi043.stdout: 2023-08-09T14:52:31.682 INFO:teuthology.orchestra.run.smithi043.stdout:{"epoch":185,"fsid":"daac87ee-1337-4250-a955-aea4e408a39b","created":"2023-08-09T13:33:41.701198+0000","modified":"2023-08-09T14:52:30.188055+0000","last_up_change":"2023-08-09T13:33:53.783453+0000","last_in_change":"2023-08-09T13:33:45.148904+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":23,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":"rbd","create_time":"2023-08-09T13:33:54.365137+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":".mgr","create_time":"2023-08-09T13:33:54.831803+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":22,"pool_name":"cephfs_metadata","create_time":"2023-08-09T14:52:28.569334+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"185","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":23,"pool_name":"cephfs_data","create_time":"2023-08-09T14:52:29.209973+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"185","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"cbd49ecd-e896-4fbd-864e-5c37a829b165","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":184,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6800","nonce":502667335},{"type":"v1","addr":"172.21.15.43:6801","nonce":502667335}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6802","nonce":502667335},{"type":"v1","addr":"172.21.15.43:6803","nonce":502667335}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6806","nonce":502667335},{"type":"v1","addr":"172.21.15.43:6807","nonce":502667335}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6804","nonce":502667335},{"type":"v1","addr":"172.21.15.43:6805","nonce":502667335}]},"public_addr":"172.21.15.43:6801/502667335","cluster_addr":"172.21.15.43:6803/502667335","heartbeat_back_addr":"172.21.15.43:6807/502667335","heartbeat_front_addr":"172.21.15.43:6805/502667335","state":["exists","up"]},{"osd":1,"uuid":"75c7e0fe-1ec7-4fe8-a2f2-3661a65b8500","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":184,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6808","nonce":3289134610},{"type":"v1","addr":"172.21.15.43:6809","nonce":3289134610}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6810","nonce":3289134610},{"type":"v1","addr":"172.21.15.43:6812","nonce":3289134610}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6821","nonce":3289134610},{"type":"v1","addr":"172.21.15.43:6824","nonce":3289134610}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6815","nonce":3289134610},{"type":"v1","addr":"172.21.15.43:6818","nonce":3289134610}]},"public_addr":"172.21.15.43:6809/3289134610","cluster_addr":"172.21.15.43:6812/3289134610","heartbeat_back_addr":"172.21.15.43:6824/3289134610","heartbeat_front_addr":"172.21.15.43:6818/3289134610","state":["exists","up"]},{"osd":2,"uuid":"bb5c41e9-7f5f-4125-9b01-3bb8bfc2ea83","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":184,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6811","nonce":4079709736},{"type":"v1","addr":"172.21.15.43:6814","nonce":4079709736}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6817","nonce":4079709736},{"type":"v1","addr":"172.21.15.43:6820","nonce":4079709736}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6829","nonce":4079709736},{"type":"v1","addr":"172.21.15.43:6831","nonce":4079709736}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6823","nonce":4079709736},{"type":"v1","addr":"172.21.15.43:6826","nonce":4079709736}]},"public_addr":"172.21.15.43:6814/4079709736","cluster_addr":"172.21.15.43:6820/4079709736","heartbeat_back_addr":"172.21.15.43:6831/4079709736","heartbeat_front_addr":"172.21.15.43:6826/4079709736","state":["exists","up"]},{"osd":3,"uuid":"bfe4c8b4-694a-465c-87e6-2b2aab852613","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":184,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6813","nonce":1709538759},{"type":"v1","addr":"172.21.15.43:6816","nonce":1709538759}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6819","nonce":1709538759},{"type":"v1","addr":"172.21.15.43:6822","nonce":1709538759}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6828","nonce":1709538759},{"type":"v1","addr":"172.21.15.43:6830","nonce":1709538759}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6825","nonce":1709538759},{"type":"v1","addr":"172.21.15.43:6827","nonce":1709538759}]},"public_addr":"172.21.15.43:6816/1709538759","cluster_addr":"172.21.15.43:6822/1709538759","heartbeat_back_addr":"172.21.15.43:6830/1709538759","heartbeat_front_addr":"172.21.15.43:6827/1709538759","state":["exists","up"]},{"osd":4,"uuid":"fa2e8a77-aba1-4756-853e-0d174a5bdeba","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":184,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6800","nonce":4196866481},{"type":"v1","addr":"172.21.15.203:6801","nonce":4196866481}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6802","nonce":4196866481},{"type":"v1","addr":"172.21.15.203:6803","nonce":4196866481}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6806","nonce":4196866481},{"type":"v1","addr":"172.21.15.203:6807","nonce":4196866481}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6804","nonce":4196866481},{"type":"v1","addr":"172.21.15.203:6805","nonce":4196866481}]},"public_addr":"172.21.15.203:6801/4196866481","cluster_addr":"172.21.15.203:6803/4196866481","heartbeat_back_addr":"172.21.15.203:6807/4196866481","heartbeat_front_addr":"172.21.15.203:6805/4196866481","state":["exists","up"]},{"osd":5,"uuid":"81811e4b-4d24-4f51-a82c-f1d700a2a00d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":184,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6816","nonce":3059290081},{"type":"v1","addr":"172.21.15.203:6817","nonce":3059290081}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6818","nonce":3059290081},{"type":"v1","addr":"172.21.15.203:6819","nonce":3059290081}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6822","nonce":3059290081},{"type":"v1","addr":"172.21.15.203:6823","nonce":3059290081}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6820","nonce":3059290081},{"type":"v1","addr":"172.21.15.203:6821","nonce":3059290081}]},"public_addr":"172.21.15.203:6817/3059290081","cluster_addr":"172.21.15.203:6819/3059290081","heartbeat_back_addr":"172.21.15.203:6823/3059290081","heartbeat_front_addr":"172.21.15.203:6821/3059290081","state":["exists","up"]},{"osd":6,"uuid":"307bac4d-6a5b-443d-af75-56d695d7674f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":184,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6824","nonce":512155569},{"type":"v1","addr":"172.21.15.203:6825","nonce":512155569}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6826","nonce":512155569},{"type":"v1","addr":"172.21.15.203:6827","nonce":512155569}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6830","nonce":512155569},{"type":"v1","addr":"172.21.15.203:6831","nonce":512155569}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6828","nonce":512155569},{"type":"v1","addr":"172.21.15.203:6829","nonce":512155569}]},"public_addr":"172.21.15.203:6825/512155569","cluster_addr":"172.21.15.203:6827/512155569","heartbeat_back_addr":"172.21.15.203:6831/512155569","heartbeat_front_addr":"172.21.15.203:6829/512155569","state":["exists","up"]},{"osd":7,"uuid":"a95dc7ca-9f1b-4295-af70-68a22a2d7d5e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":184,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6808","nonce":3331450438},{"type":"v1","addr":"172.21.15.203:6809","nonce":3331450438}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6810","nonce":3331450438},{"type":"v1","addr":"172.21.15.203:6811","nonce":3331450438}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6814","nonce":3331450438},{"type":"v1","addr":"172.21.15.203:6815","nonce":3331450438}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6812","nonce":3331450438},{"type":"v1","addr":"172.21.15.203:6813","nonce":3331450438}]},"public_addr":"172.21.15.203:6809/3331450438","cluster_addr":"172.21.15.203:6811/3331450438","heartbeat_back_addr":"172.21.15.203:6815/3331450438","heartbeat_front_addr":"172.21.15.203:6813/3331450438","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-09T13:33:50.547652+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-09T13:33:50.589469+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-09T13:33:50.561724+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-09T13:33:50.599650+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-09T13:33:50.411855+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-09T13:33:50.219681+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-09T13:33:50.504664+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-09T13:33:50.441248+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":7,"snaps":[{"begin":104,"length":1},{"begin":221,"length":1}]}],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-08-09T14:52:31.684 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:31.683+0000 7f714c048640 1 -- 172.21.15.43:0/565760239 >> [v2:172.21.15.203:6832/2959400181,v1:172.21.15.203:6833/2959400181] conn(0x7f712806a170 msgr2=0x7f712806c630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T14:52:31.685 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:31.683+0000 7f714c048640 1 --2- 172.21.15.43:0/565760239 >> [v2:172.21.15.203:6832/2959400181,v1:172.21.15.203:6833/2959400181] conn(0x7f712806a170 0x7f712806c630 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f7144068600 tx=0x7f7140009040 comp rx=0 tx=0).stop 2023-08-09T14:52:31.685 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:31.683+0000 7f714c048640 1 -- 172.21.15.43:0/565760239 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f71440c8b00 msgr2=0x7f71440cc340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T14:52:31.685 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:31.683+0000 7f714c048640 1 --2- 172.21.15.43:0/565760239 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f71440c8b00 0x7f71440cc340 secure :-1 s=READY pgs=2030 cs=0 l=1 rev1=1 crypto rx=0x7f7144072640 tx=0x7f713c010c20 comp rx=0 tx=0).stop 2023-08-09T14:52:31.686 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:31.683+0000 7f714c048640 1 -- 172.21.15.43:0/565760239 shutdown_connections 2023-08-09T14:52:31.686 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:31.683+0000 7f714c048640 1 --2- 172.21.15.43:0/565760239 >> [v2:172.21.15.203:6832/2959400181,v1:172.21.15.203:6833/2959400181] conn(0x7f712806a170 0x7f712806c630 unknown :-1 s=CLOSED pgs=61 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T14:52:31.686 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:31.683+0000 7f714c048640 1 --2- 172.21.15.43:0/565760239 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f71440c8b00 0x7f71440cc340 unknown :-1 s=CLOSED pgs=2030 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T14:52:31.687 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:31.683+0000 7f714c048640 1 --2- 172.21.15.43:0/565760239 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f71440c8680 0x7f71440cbe00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T14:52:31.687 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:31.683+0000 7f714c048640 1 --2- 172.21.15.43:0/565760239 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f71300047c0 0x7f71440b58b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T14:52:31.687 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:31.683+0000 7f714c048640 1 -- 172.21.15.43:0/565760239 >> 172.21.15.43:0/565760239 conn(0x7f71440ccd80 msgr2=0x7f714406dba0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-09T14:52:31.687 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:31.684+0000 7f714c048640 1 -- 172.21.15.43:0/565760239 shutdown_connections 2023-08-09T14:52:31.688 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:31.684+0000 7f714c048640 1 -- 172.21.15.43:0/565760239 wait complete. 2023-08-09T14:52:31.704 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs standby_count_wanted 0 2023-08-09T14:52:31.934 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:31.933+0000 7fdcbd949640 1 Processor -- start 2023-08-09T14:52:31.934 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:31.933+0000 7fdcbd949640 1 -- start start 2023-08-09T14:52:31.935 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:31.935+0000 7fdcbd949640 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7fdcb80c96a0 0x7fdcb80cafa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T14:52:31.935 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:31.935+0000 7fdcbd949640 1 -- --> v1:172.21.15.43:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fdcb80b07e0 con 0x7fdcb81590b0 2023-08-09T14:52:31.936 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:31.935+0000 7fdcbd949640 1 -- --> v1:172.21.15.203:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fdcb8070150 con 0x7fdcb806fd40 2023-08-09T14:52:31.936 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:31.935+0000 7fdcbd949640 1 -- --> v2:172.21.15.203:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fdcb8070420 con 0x7fdcb80c96a0 2023-08-09T14:52:31.937 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:31.935+0000 7fdcbc947640 1 --1- >> v1:172.21.15.43:6789/0 conn(0x7fdcb81590b0 0x7fdcb80c8f90 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.43:6789/0 says I am v1:172.21.15.43:50478/0 (socket says 172.21.15.43:50478) 2023-08-09T14:52:31.937 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:31.935+0000 7fdcbc947640 1 -- 172.21.15.43:0/3212986287 learned_addr learned my addr 172.21.15.43:0/3212986287 (peer_addr_for_me v1:172.21.15.43:0/0) 2023-08-09T14:52:31.937 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:31.936+0000 7fdcb7fff640 1 --2- 172.21.15.43:0/3212986287 >> v2:172.21.15.203:3300/0 conn(0x7fdcb80c96a0 0x7fdcb80cafa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T14:52:31.937 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:31.936+0000 7fdcb77fe640 1 -- 172.21.15.43:0/3212986287 <== mon.0 v1:172.21.15.43:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2518068742 0 0) 0x7fdcb80b07e0 con 0x7fdcb81590b0 2023-08-09T14:52:31.938 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:31.936+0000 7fdcb77fe640 1 -- 172.21.15.43:0/3212986287 --> v1:172.21.15.43:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fdca0003520 con 0x7fdcb81590b0 2023-08-09T14:52:31.938 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:31.936+0000 7fdcb77fe640 1 -- 172.21.15.43:0/3212986287 <== mon.0 v1:172.21.15.43:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2255175539 0 0) 0x7fdca0003520 con 0x7fdcb81590b0 2023-08-09T14:52:31.938 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:31.936+0000 7fdcb77fe640 1 -- 172.21.15.43:0/3212986287 >> v2:172.21.15.203:3300/0 conn(0x7fdcb80c96a0 msgr2=0x7fdcb80cafa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-09T14:52:31.938 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:31.936+0000 7fdcb77fe640 1 --2- 172.21.15.43:0/3212986287 >> v2:172.21.15.203:3300/0 conn(0x7fdcb80c96a0 0x7fdcb80cafa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T14:52:31.938 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:31.936+0000 7fdcb77fe640 1 -- 172.21.15.43:0/3212986287 >> v1:172.21.15.203:6789/0 conn(0x7fdcb806fd40 legacy=0x7fdcb80cb9f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T14:52:31.938 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:31.936+0000 7fdcb77fe640 1 -- 172.21.15.43:0/3212986287 --> v1:172.21.15.43:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdcb8070700 con 0x7fdcb81590b0 2023-08-09T14:52:31.938 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:31.936+0000 7fdcb77fe640 1 -- 172.21.15.43:0/3212986287 <== mon.0 v1:172.21.15.43:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1285150740 0 0) 0x7fdca8001670 con 0x7fdcb81590b0 2023-08-09T14:52:31.938 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:31.936+0000 7fdcb77fe640 1 -- 172.21.15.43:0/3212986287 >> v1:172.21.15.43:6789/0 conn(0x7fdcb81590b0 legacy=0x7fdcb80c8f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T14:52:31.939 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:31.937+0000 7fdcb77fe640 1 --2- 172.21.15.43:0/3212986287 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fdca0003970 0x7fdca0003d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T14:52:31.939 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:31.937+0000 7fdcb77fe640 1 --2- 172.21.15.43:0/3212986287 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fdca00047c0 0x7fdca0004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T14:52:31.939 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:31.937+0000 7fdcb77fe640 1 --2- 172.21.15.43:0/3212986287 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fdca0005180 0x7fdca000b980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T14:52:31.939 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:31.937+0000 7fdcb77fe640 1 -- 172.21.15.43:0/3212986287 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdcb8070420 con 0x7fdca0003970 2023-08-09T14:52:31.939 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:31.937+0000 7fdcb77fe640 1 -- 172.21.15.43:0/3212986287 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdca000c020 con 0x7fdca00047c0 2023-08-09T14:52:31.939 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:31.937+0000 7fdcb77fe640 1 -- 172.21.15.43:0/3212986287 --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdca000c1e0 con 0x7fdca0005180 2023-08-09T14:52:31.939 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:31.937+0000 7fdcbc947640 1 --2- 172.21.15.43:0/3212986287 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fdca0003970 0x7fdca0003d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T14:52:31.941 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:31.937+0000 7fdcb7fff640 1 --2- 172.21.15.43:0/3212986287 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fdca00047c0 0x7fdca0004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T14:52:31.941 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:31.937+0000 7fdcbc947640 1 -- 172.21.15.43:0/3212986287 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fdca0005180 msgr2=0x7fdca000b980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-09T14:52:31.941 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:31.937+0000 7fdcbc947640 1 --2- 172.21.15.43:0/3212986287 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fdca0005180 0x7fdca000b980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T14:52:31.941 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:31.937+0000 7fdcbc947640 1 -- 172.21.15.43:0/3212986287 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fdca00047c0 msgr2=0x7fdca0004c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-09T14:52:31.941 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:31.937+0000 7fdcbc947640 1 --2- 172.21.15.43:0/3212986287 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fdca00047c0 0x7fdca0004c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T14:52:31.941 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:31.937+0000 7fdcbc947640 1 -- 172.21.15.43:0/3212986287 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdca0003520 con 0x7fdca0003970 2023-08-09T14:52:31.942 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:31.937+0000 7fdcbd148640 1 -- 172.21.15.43:0/3212986287 reap_dead start 2023-08-09T14:52:31.942 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:31.937+0000 7fdcb7fff640 1 --2- 172.21.15.43:0/3212986287 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fdca00047c0 0x7fdca0004c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-08-09T14:52:31.942 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:31.938+0000 7fdcbc947640 1 --2- 172.21.15.43:0/3212986287 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fdca0003970 0x7fdca0003d90 secure :-1 s=READY pgs=2032 cs=0 l=1 rev1=1 crypto rx=0x7fdca800c240 tx=0x7fdca00057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d470508ae1b1d394 server_cookie=0 in_seq=0 out_seq=0 2023-08-09T14:52:31.942 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:31.939+0000 7fdcb77fe640 1 -- 172.21.15.43:0/3212986287 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdca8005ad0 con 0x7fdca0003970 2023-08-09T14:52:31.942 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:31.939+0000 7fdcb77fe640 1 -- 172.21.15.43:0/3212986287 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdca8005c50 con 0x7fdca0003970 2023-08-09T14:52:31.942 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:31.939+0000 7fdcb77fe640 1 -- 172.21.15.43:0/3212986287 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdca802c8b0 con 0x7fdca0003970 2023-08-09T14:52:31.942 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:31.939+0000 7fdcbd949640 1 -- 172.21.15.43:0/3212986287 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fdca0003970 msgr2=0x7fdca0003d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T14:52:31.942 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:31.939+0000 7fdcbd949640 1 --2- 172.21.15.43:0/3212986287 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fdca0003970 0x7fdca0003d90 secure :-1 s=READY pgs=2032 cs=0 l=1 rev1=1 crypto rx=0x7fdca800c240 tx=0x7fdca00057f0 comp rx=0 tx=0).stop 2023-08-09T14:52:31.943 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:31.939+0000 7fdcbd949640 1 -- 172.21.15.43:0/3212986287 shutdown_connections 2023-08-09T14:52:31.943 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:31.939+0000 7fdcbd949640 1 --2- 172.21.15.43:0/3212986287 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fdca0003970 0x7fdca0003d90 unknown :-1 s=CLOSED pgs=2032 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T14:52:31.943 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:31.939+0000 7fdcbd949640 1 -- 172.21.15.43:0/3212986287 >> 172.21.15.43:0/3212986287 conn(0x7fdcb80af9f0 msgr2=0x7fdcb80afdd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-09T14:52:31.943 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:31.940+0000 7fdcbd949640 1 -- 172.21.15.43:0/3212986287 shutdown_connections 2023-08-09T14:52:31.943 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:31.940+0000 7fdcbd949640 1 -- 172.21.15.43:0/3212986287 wait complete. 2023-08-09T14:52:31.945 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:31.941+0000 7fdcbd949640 1 Processor -- start 2023-08-09T14:52:31.945 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:31.942+0000 7fdcbd949640 1 -- start start 2023-08-09T14:52:31.945 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:31.942+0000 7fdcbd949640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fdca0003970 0x7fdcb80c6e80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T14:52:31.945 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:31.942+0000 7fdcbd949640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fdcb8154ad0 0x7fdcb8148130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T14:52:31.945 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:31.942+0000 7fdcbd949640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fdcb81574c0 0x7fdcb8157950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T14:52:31.946 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:31.942+0000 7fdcbd949640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdcb80c73c0 con 0x7fdca0003970 2023-08-09T14:52:31.946 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:31.942+0000 7fdcbd949640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdcb8157e90 con 0x7fdcb8154ad0 2023-08-09T14:52:31.946 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:31.942+0000 7fdcbd949640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdcb8155b70 con 0x7fdcb81574c0 2023-08-09T14:52:31.946 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:31.943+0000 7fdcbc947640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fdca0003970 0x7fdcb80c6e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T14:52:31.946 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:31.943+0000 7fdcbc947640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fdca0003970 0x7fdcb80c6e80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:55526/0 (socket says 172.21.15.43:55526) 2023-08-09T14:52:31.946 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:31.943+0000 7fdcbc947640 1 -- 172.21.15.43:0/2711796537 learned_addr learned my addr 172.21.15.43:0/2711796537 (peer_addr_for_me v2:172.21.15.43:0/0) 2023-08-09T14:52:31.946 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:31.943+0000 7fdcb7fff640 1 --2- 172.21.15.43:0/2711796537 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fdcb8154ad0 0x7fdcb8148130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T14:52:31.947 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:31.943+0000 7fdcbd148640 1 --2- 172.21.15.43:0/2711796537 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fdcb81574c0 0x7fdcb8157950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T14:52:31.947 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:31.943+0000 7fdcbc947640 1 -- 172.21.15.43:0/2711796537 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fdcb81574c0 msgr2=0x7fdcb8157950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T14:52:31.947 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:31.943+0000 7fdcbc947640 1 --2- 172.21.15.43:0/2711796537 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fdcb81574c0 0x7fdcb8157950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T14:52:31.947 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:31.943+0000 7fdcbc947640 1 -- 172.21.15.43:0/2711796537 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fdcb8154ad0 msgr2=0x7fdcb8148130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T14:52:31.947 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:31.943+0000 7fdcbc947640 1 --2- 172.21.15.43:0/2711796537 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fdcb8154ad0 0x7fdcb8148130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T14:52:31.947 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:31.943+0000 7fdcbc947640 1 -- 172.21.15.43:0/2711796537 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdcb8154f60 con 0x7fdca0003970 2023-08-09T14:52:31.947 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:31.943+0000 7fdcbd148640 1 --2- 172.21.15.43:0/2711796537 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fdcb81574c0 0x7fdcb8157950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-08-09T14:52:31.948 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:31.944+0000 7fdcbc947640 1 --2- 172.21.15.43:0/2711796537 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fdca0003970 0x7fdcb80c6e80 secure :-1 s=READY pgs=2033 cs=0 l=1 rev1=1 crypto rx=0x7fdca8001930 tx=0x7fdca802e070 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-09T14:52:31.948 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:31.944+0000 7fdcb57fa640 1 -- 172.21.15.43:0/2711796537 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdca803c040 con 0x7fdca0003970 2023-08-09T14:52:31.948 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:31.944+0000 7fdcb57fa640 1 -- 172.21.15.43:0/2711796537 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdca802cd80 con 0x7fdca0003970 2023-08-09T14:52:31.948 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:31.944+0000 7fdcb57fa640 1 -- 172.21.15.43:0/2711796537 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdca8007290 con 0x7fdca0003970 2023-08-09T14:52:31.948 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:31.944+0000 7fdcbd949640 1 -- 172.21.15.43:0/2711796537 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdcb8155240 con 0x7fdca0003970 2023-08-09T14:52:31.948 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:31.944+0000 7fdcbd949640 1 -- 172.21.15.43:0/2711796537 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdcb8155800 con 0x7fdca0003970 2023-08-09T14:52:31.949 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:31.946+0000 7fdcbd949640 1 -- 172.21.15.43:0/2711796537 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdcb804e390 con 0x7fdca0003970 2023-08-09T14:52:31.949 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:31.946+0000 7fdcb57fa640 1 -- 172.21.15.43:0/2711796537 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 88) v1 ==== 88443+0+0 (secure 0 0 0) 0x7fdca8032070 con 0x7fdca0003970 2023-08-09T14:52:31.949 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:31.947+0000 7fdcb57fa640 1 --2- 172.21.15.43:0/2711796537 >> [v2:172.21.15.203:6832/2959400181,v1:172.21.15.203:6833/2959400181] conn(0x7fdca406a170 0x7fdca406c630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T14:52:31.949 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:31.947+0000 7fdcb57fa640 1 -- 172.21.15.43:0/2711796537 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(186..186 src has 1..186) v4 ==== 6426+0+0 (secure 0 0 0) 0x7fdca80a9ef0 con 0x7fdca0003970 2023-08-09T14:52:31.950 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:31.948+0000 7fdcb7fff640 1 --2- 172.21.15.43:0/2711796537 >> [v2:172.21.15.203:6832/2959400181,v1:172.21.15.203:6833/2959400181] conn(0x7fdca406a170 0x7fdca406c630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T14:52:31.950 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:31.949+0000 7fdcb7fff640 1 --2- 172.21.15.43:0/2711796537 >> [v2:172.21.15.203:6832/2959400181,v1:172.21.15.203:6833/2959400181] conn(0x7fdca406a170 0x7fdca406c630 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7fdcb80c42b0 tx=0x7fdcac008040 comp rx=0 tx=0).ready entity=mgr.7223 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-09T14:52:31.952 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:31.952+0000 7fdcb57fa640 1 -- 172.21.15.43:0/2711796537 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178066 (secure 0 0 0) 0x7fdca80752c0 con 0x7fdca0003970 2023-08-09T14:52:32.191 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:32.191+0000 7fdcbd949640 1 -- 172.21.15.43:0/2711796537 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} v 0) v1 -- 0x7fdcb815c150 con 0x7fdca0003970 2023-08-09T14:52:32.777 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:32.776+0000 7fdcb57fa640 1 -- 172.21.15.43:0/2711796537 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]=0 v86) v1 ==== 118+0+0 (secure 0 0 0) 0x7fdca80788e0 con 0x7fdca0003970 2023-08-09T14:52:32.781 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:32.780+0000 7fdcbd949640 1 -- 172.21.15.43:0/2711796537 >> [v2:172.21.15.203:6832/2959400181,v1:172.21.15.203:6833/2959400181] conn(0x7fdca406a170 msgr2=0x7fdca406c630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T14:52:32.781 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:32.780+0000 7fdcbd949640 1 --2- 172.21.15.43:0/2711796537 >> [v2:172.21.15.203:6832/2959400181,v1:172.21.15.203:6833/2959400181] conn(0x7fdca406a170 0x7fdca406c630 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7fdcb80c42b0 tx=0x7fdcac008040 comp rx=0 tx=0).stop 2023-08-09T14:52:32.781 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:32.780+0000 7fdcbd949640 1 -- 172.21.15.43:0/2711796537 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fdca0003970 msgr2=0x7fdcb80c6e80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T14:52:32.782 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:32.780+0000 7fdcbd949640 1 --2- 172.21.15.43:0/2711796537 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fdca0003970 0x7fdcb80c6e80 secure :-1 s=READY pgs=2033 cs=0 l=1 rev1=1 crypto rx=0x7fdca8001930 tx=0x7fdca802e070 comp rx=0 tx=0).stop 2023-08-09T14:52:32.782 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:32.780+0000 7fdcbd949640 1 -- 172.21.15.43:0/2711796537 shutdown_connections 2023-08-09T14:52:32.782 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:32.780+0000 7fdcbd949640 1 --2- 172.21.15.43:0/2711796537 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fdcb81574c0 0x7fdcb8157950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T14:52:32.782 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:32.780+0000 7fdcbd949640 1 --2- 172.21.15.43:0/2711796537 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fdcb8154ad0 0x7fdcb8148130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T14:52:32.782 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:32.780+0000 7fdcbd949640 1 --2- 172.21.15.43:0/2711796537 >> [v2:172.21.15.203:6832/2959400181,v1:172.21.15.203:6833/2959400181] conn(0x7fdca406a170 0x7fdca406c630 unknown :-1 s=CLOSED pgs=62 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T14:52:32.782 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:32.780+0000 7fdcbd949640 1 --2- 172.21.15.43:0/2711796537 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fdca0003970 0x7fdcb80c6e80 unknown :-1 s=CLOSED pgs=2033 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T14:52:32.782 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:32.780+0000 7fdcbd949640 1 -- 172.21.15.43:0/2711796537 >> 172.21.15.43:0/2711796537 conn(0x7fdcb80af9f0 msgr2=0x7fdcb80cdd40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-09T14:52:32.783 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:32.781+0000 7fdcbd949640 1 -- 172.21.15.43:0/2711796537 shutdown_connections 2023-08-09T14:52:32.783 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:32.781+0000 7fdcbd949640 1 -- 172.21.15.43:0/2711796537 wait complete. 2023-08-09T14:52:32.805 DEBUG:tasks.cephfs.filesystem:fs_config: {} 2023-08-09T14:52:32.805 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs allow_standby_replay false 2023-08-09T14:52:33.031 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:33.031+0000 7f4359966640 1 Processor -- start 2023-08-09T14:52:33.032 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:33.031+0000 7f4359966640 1 -- start start 2023-08-09T14:52:33.033 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:33.033+0000 7f4359966640 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7f435406f2c0 0x7f435415b0f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T14:52:33.033 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:33.033+0000 7f4359966640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f43540c8470 0x7f43540c88c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T14:52:33.034 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:33.033+0000 7f4359966640 1 -- --> v1:172.21.15.43:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f43540c8e00 con 0x7f4354157ce0 2023-08-09T14:52:33.034 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:33.033+0000 7f4359966640 1 -- --> v2:172.21.15.203:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f43541580c0 con 0x7f435406f2c0 2023-08-09T14:52:33.034 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:33.033+0000 7f4359966640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f43540c92c0 con 0x7f43540c8470 2023-08-09T14:52:33.036 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:33.033+0000 7f4358964640 1 --1- >> v1:172.21.15.43:6789/0 conn(0x7f4354157ce0 0x7f435406bc40 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.43:6789/0 says I am v1:172.21.15.43:50494/0 (socket says 172.21.15.43:50494) 2023-08-09T14:52:33.036 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:33.033+0000 7f4358964640 1 -- 172.21.15.43:0/4172212218 learned_addr learned my addr 172.21.15.43:0/4172212218 (peer_addr_for_me v1:172.21.15.43:0/0) 2023-08-09T14:52:33.036 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:33.034+0000 7f4359165640 1 --2- 172.21.15.43:0/4172212218 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f43540c8470 0x7f43540c88c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T14:52:33.036 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:33.034+0000 7f4353fff640 1 --2- 172.21.15.43:0/4172212218 >> v2:172.21.15.203:3300/0 conn(0x7f435406f2c0 0x7f435415b0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T14:52:33.036 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:33.034+0000 7f43537fe640 1 -- 172.21.15.43:0/4172212218 <== mon.0 v1:172.21.15.43:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1427554514 0 0) 0x7f4348001110 con 0x7f4354157ce0 2023-08-09T14:52:33.036 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:33.034+0000 7f43537fe640 1 -- 172.21.15.43:0/4172212218 --> v1:172.21.15.43:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f4344003520 con 0x7f4354157ce0 2023-08-09T14:52:33.037 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:33.034+0000 7f43537fe640 1 -- 172.21.15.43:0/4172212218 <== mon.0 v1:172.21.15.43:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3427411379 0 0) 0x7f4344003520 con 0x7f4354157ce0 2023-08-09T14:52:33.037 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:33.034+0000 7f43537fe640 1 -- 172.21.15.43:0/4172212218 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f43540c8470 msgr2=0x7f43540c88c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-09T14:52:33.037 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:33.034+0000 7f43537fe640 1 --2- 172.21.15.43:0/4172212218 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f43540c8470 0x7f43540c88c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T14:52:33.037 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:33.034+0000 7f43537fe640 1 -- 172.21.15.43:0/4172212218 >> v2:172.21.15.203:3300/0 conn(0x7f435406f2c0 msgr2=0x7f435415b0f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-09T14:52:33.037 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:33.034+0000 7f43537fe640 1 --2- 172.21.15.43:0/4172212218 >> v2:172.21.15.203:3300/0 conn(0x7f435406f2c0 0x7f435415b0f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T14:52:33.037 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:33.034+0000 7f43537fe640 1 -- 172.21.15.43:0/4172212218 --> v1:172.21.15.43:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f435406b480 con 0x7f4354157ce0 2023-08-09T14:52:33.038 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:33.034+0000 7f43537fe640 1 -- 172.21.15.43:0/4172212218 <== mon.0 v1:172.21.15.43:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1285150740 0 0) 0x7f4348001800 con 0x7f4354157ce0 2023-08-09T14:52:33.038 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:33.034+0000 7f43537fe640 1 -- 172.21.15.43:0/4172212218 >> v1:172.21.15.43:6789/0 conn(0x7f4354157ce0 legacy=0x7f435406bc40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T14:52:33.038 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:33.034+0000 7f43537fe640 1 --2- 172.21.15.43:0/4172212218 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f43540c8470 0x7f4344003970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T14:52:33.038 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:33.034+0000 7f43537fe640 1 --2- 172.21.15.43:0/4172212218 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f4344003ef0 0x7f43440047c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T14:52:33.038 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:33.034+0000 7f43537fe640 1 --2- 172.21.15.43:0/4172212218 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f4344004d00 0x7f4344005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T14:52:33.038 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:33.035+0000 7f43537fe640 1 -- 172.21.15.43:0/4172212218 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f43541580c0 con 0x7f4344004d00 2023-08-09T14:52:33.038 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:33.035+0000 7f43537fe640 1 -- 172.21.15.43:0/4172212218 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f43540c92c0 con 0x7f4344003ef0 2023-08-09T14:52:33.039 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:33.035+0000 7f43537fe640 1 -- 172.21.15.43:0/4172212218 --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f434400b9d0 con 0x7f43540c8470 2023-08-09T14:52:33.039 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:33.035+0000 7f4359165640 1 --2- 172.21.15.43:0/4172212218 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f4344004d00 0x7f4344005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T14:52:33.039 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:33.035+0000 7f4358964640 1 --2- 172.21.15.43:0/4172212218 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f43540c8470 0x7f4344003970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T14:52:33.039 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:33.035+0000 7f4359165640 1 -- 172.21.15.43:0/4172212218 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f43540c8470 msgr2=0x7f4344003970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-09T14:52:33.039 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:33.035+0000 7f4359165640 1 --2- 172.21.15.43:0/4172212218 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f43540c8470 0x7f4344003970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T14:52:33.039 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:33.035+0000 7f4359165640 1 -- 172.21.15.43:0/4172212218 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f4344003ef0 msgr2=0x7f43440047c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-09T14:52:33.039 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:33.035+0000 7f4359165640 1 --2- 172.21.15.43:0/4172212218 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f4344003ef0 0x7f43440047c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T14:52:33.039 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:33.035+0000 7f4359165640 1 -- 172.21.15.43:0/4172212218 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4344003520 con 0x7f4344004d00 2023-08-09T14:52:33.040 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:33.035+0000 7f4358964640 1 --2- 172.21.15.43:0/4172212218 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f43540c8470 0x7f4344003970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-08-09T14:52:33.042 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:33.036+0000 7f4359165640 1 --2- 172.21.15.43:0/4172212218 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f4344004d00 0x7f4344005180 secure :-1 s=READY pgs=2035 cs=0 l=1 rev1=1 crypto rx=0x7f434c00e170 tx=0x7f43440057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=69ba5314e86177c4 server_cookie=0 in_seq=0 out_seq=0 2023-08-09T14:52:33.042 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:33.037+0000 7f43537fe640 1 -- 172.21.15.43:0/4172212218 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f434c02cc00 con 0x7f4344004d00 2023-08-09T14:52:33.042 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:33.037+0000 7f43537fe640 1 -- 172.21.15.43:0/4172212218 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f434c00f040 con 0x7f4344004d00 2023-08-09T14:52:33.042 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:33.037+0000 7f43537fe640 1 -- 172.21.15.43:0/4172212218 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f434c034440 con 0x7f4344004d00 2023-08-09T14:52:33.043 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:33.037+0000 7f4359966640 1 -- 172.21.15.43:0/4172212218 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f4344004d00 msgr2=0x7f4344005180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T14:52:33.043 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:33.037+0000 7f4359966640 1 --2- 172.21.15.43:0/4172212218 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f4344004d00 0x7f4344005180 secure :-1 s=READY pgs=2035 cs=0 l=1 rev1=1 crypto rx=0x7f434c00e170 tx=0x7f43440057f0 comp rx=0 tx=0).stop 2023-08-09T14:52:33.043 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:33.037+0000 7f4359165640 1 -- 172.21.15.43:0/4172212218 reap_dead start 2023-08-09T14:52:33.043 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:33.038+0000 7f4359966640 1 -- 172.21.15.43:0/4172212218 shutdown_connections 2023-08-09T14:52:33.044 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:33.038+0000 7f4359966640 1 -- 172.21.15.43:0/4172212218 >> 172.21.15.43:0/4172212218 conn(0x7f43540af9f0 msgr2=0x7f43540afdd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-09T14:52:33.044 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:33.038+0000 7f4359966640 1 -- 172.21.15.43:0/4172212218 shutdown_connections 2023-08-09T14:52:33.044 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:33.038+0000 7f4359966640 1 -- 172.21.15.43:0/4172212218 wait complete. 2023-08-09T14:52:33.045 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:33.039+0000 7f4359966640 1 Processor -- start 2023-08-09T14:52:33.045 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:33.040+0000 7f4359966640 1 -- start start 2023-08-09T14:52:33.045 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:33.041+0000 7f4359966640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f4344004d00 0x7f435415d2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T14:52:33.046 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:33.041+0000 7f4359966640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f43540c2360 0x7f43541f1180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T14:52:33.046 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:33.041+0000 7f4359966640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f43541f16c0 0x7f43540cc410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T14:52:33.046 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:33.041+0000 7f4359966640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f43540c3730 con 0x7f43541f16c0 2023-08-09T14:52:33.046 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:33.041+0000 7f4359966640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f43540c2bb0 con 0x7f43540c2360 2023-08-09T14:52:33.047 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:33.041+0000 7f4359966640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f43540c2e90 con 0x7f4344004d00 2023-08-09T14:52:33.047 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:33.041+0000 7f4359165640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f43541f16c0 0x7f43540cc410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T14:52:33.047 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:33.041+0000 7f4359165640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f43541f16c0 0x7f43540cc410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:55532/0 (socket says 172.21.15.43:55532) 2023-08-09T14:52:33.048 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:33.041+0000 7f4359165640 1 -- 172.21.15.43:0/4158141810 learned_addr learned my addr 172.21.15.43:0/4158141810 (peer_addr_for_me v2:172.21.15.43:0/0) 2023-08-09T14:52:33.048 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:33.041+0000 7f4353fff640 1 --2- 172.21.15.43:0/4158141810 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f43540c2360 0x7f43541f1180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T14:52:33.048 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:33.041+0000 7f4359165640 1 -- 172.21.15.43:0/4158141810 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f4344004d00 msgr2=0x7f435415d2f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T14:52:33.048 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:33.041+0000 7f4358964640 1 --2- 172.21.15.43:0/4158141810 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f4344004d00 0x7f435415d2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T14:52:33.049 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:33.041+0000 7f4359165640 1 --2- 172.21.15.43:0/4158141810 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f4344004d00 0x7f435415d2f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T14:52:33.049 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:33.041+0000 7f4359165640 1 -- 172.21.15.43:0/4158141810 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f43540c2360 msgr2=0x7f43541f1180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T14:52:33.049 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:33.041+0000 7f4359165640 1 --2- 172.21.15.43:0/4158141810 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f43540c2360 0x7f43541f1180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T14:52:33.050 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:33.041+0000 7f4359165640 1 -- 172.21.15.43:0/4158141810 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f43540c3170 con 0x7f43541f16c0 2023-08-09T14:52:33.052 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:33.042+0000 7f4359165640 1 --2- 172.21.15.43:0/4158141810 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f43541f16c0 0x7f43540cc410 secure :-1 s=READY pgs=2036 cs=0 l=1 rev1=1 crypto rx=0x7f434c00e170 tx=0x7f434c009750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-09T14:52:33.052 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:33.042+0000 7f43517fa640 1 -- 172.21.15.43:0/4158141810 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f434c043070 con 0x7f43541f16c0 2023-08-09T14:52:33.053 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:33.042+0000 7f43517fa640 1 -- 172.21.15.43:0/4158141810 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f434c034b50 con 0x7f43541f16c0 2023-08-09T14:52:33.053 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:33.042+0000 7f43517fa640 1 -- 172.21.15.43:0/4158141810 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f434c003990 con 0x7f43541f16c0 2023-08-09T14:52:33.053 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:33.042+0000 7f4359966640 1 -- 172.21.15.43:0/4158141810 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f43540c3450 con 0x7f43541f16c0 2023-08-09T14:52:33.054 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:33.042+0000 7f4359966640 1 -- 172.21.15.43:0/4158141810 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f43540cc950 con 0x7f43541f16c0 2023-08-09T14:52:33.054 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:33.043+0000 7f4359966640 1 -- 172.21.15.43:0/4158141810 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f435404e390 con 0x7f43541f16c0 2023-08-09T14:52:33.054 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:33.044+0000 7f43517fa640 1 -- 172.21.15.43:0/4158141810 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 88) v1 ==== 88443+0+0 (secure 0 0 0) 0x7f434c03e440 con 0x7f43541f16c0 2023-08-09T14:52:33.054 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:33.045+0000 7f43517fa640 1 --2- 172.21.15.43:0/4158141810 >> [v2:172.21.15.203:6832/2959400181,v1:172.21.15.203:6833/2959400181] conn(0x7f433806a170 0x7f433806c630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T14:52:33.054 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:33.046+0000 7f43517fa640 1 -- 172.21.15.43:0/4158141810 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(186..186 src has 1..186) v4 ==== 6426+0+0 (secure 0 0 0) 0x7f434c0aa150 con 0x7f43541f16c0 2023-08-09T14:52:33.055 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:33.046+0000 7f4358964640 1 --2- 172.21.15.43:0/4158141810 >> [v2:172.21.15.203:6832/2959400181,v1:172.21.15.203:6833/2959400181] conn(0x7f433806a170 0x7f433806c630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T14:52:33.055 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:33.047+0000 7f4358964640 1 --2- 172.21.15.43:0/4158141810 >> [v2:172.21.15.203:6832/2959400181,v1:172.21.15.203:6833/2959400181] conn(0x7f433806a170 0x7f433806c630 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f4348000f60 tx=0x7f4348009040 comp rx=0 tx=0).ready entity=mgr.7223 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-09T14:52:33.055 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:33.050+0000 7f43517fa640 1 -- 172.21.15.43:0/4158141810 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178066 (secure 0 0 0) 0x7f434c0756d0 con 0x7f43541f16c0 2023-08-09T14:52:33.281 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:33.280+0000 7f4359966640 1 -- 172.21.15.43:0/4158141810 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"} v 0) v1 -- 0x7f435406db10 con 0x7f43541f16c0 2023-08-09T14:52:33.780 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:33.779+0000 7f43517fa640 1 -- 172.21.15.43:0/4158141810 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"}]=0 v87) v1 ==== 122+0+0 (secure 0 0 0) 0x7f434c078cf0 con 0x7f43541f16c0 2023-08-09T14:52:33.784 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:33.783+0000 7f4359966640 1 -- 172.21.15.43:0/4158141810 >> [v2:172.21.15.203:6832/2959400181,v1:172.21.15.203:6833/2959400181] conn(0x7f433806a170 msgr2=0x7f433806c630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T14:52:33.784 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:33.783+0000 7f4359966640 1 --2- 172.21.15.43:0/4158141810 >> [v2:172.21.15.203:6832/2959400181,v1:172.21.15.203:6833/2959400181] conn(0x7f433806a170 0x7f433806c630 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f4348000f60 tx=0x7f4348009040 comp rx=0 tx=0).stop 2023-08-09T14:52:33.784 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:33.783+0000 7f4359966640 1 -- 172.21.15.43:0/4158141810 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f43541f16c0 msgr2=0x7f43540cc410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T14:52:33.784 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:33.783+0000 7f4359966640 1 --2- 172.21.15.43:0/4158141810 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f43541f16c0 0x7f43540cc410 secure :-1 s=READY pgs=2036 cs=0 l=1 rev1=1 crypto rx=0x7f434c00e170 tx=0x7f434c009750 comp rx=0 tx=0).stop 2023-08-09T14:52:33.785 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:33.783+0000 7f4359966640 1 -- 172.21.15.43:0/4158141810 shutdown_connections 2023-08-09T14:52:33.785 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:33.783+0000 7f4359966640 1 --2- 172.21.15.43:0/4158141810 >> [v2:172.21.15.203:6832/2959400181,v1:172.21.15.203:6833/2959400181] conn(0x7f433806a170 0x7f433806c630 unknown :-1 s=CLOSED pgs=63 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T14:52:33.785 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:33.783+0000 7f4359966640 1 --2- 172.21.15.43:0/4158141810 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f43541f16c0 0x7f43540cc410 unknown :-1 s=CLOSED pgs=2036 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T14:52:33.785 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:33.783+0000 7f4359966640 1 --2- 172.21.15.43:0/4158141810 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f43540c2360 0x7f43541f1180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T14:52:33.785 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:33.783+0000 7f4359966640 1 --2- 172.21.15.43:0/4158141810 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f4344004d00 0x7f435415d2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T14:52:33.785 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:33.783+0000 7f4359966640 1 -- 172.21.15.43:0/4158141810 >> 172.21.15.43:0/4158141810 conn(0x7f43540af9f0 msgr2=0x7f43540b0f80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-09T14:52:33.785 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:33.784+0000 7f4359966640 1 -- 172.21.15.43:0/4158141810 shutdown_connections 2023-08-09T14:52:33.786 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:33.784+0000 7f4359966640 1 -- 172.21.15.43:0/4158141810 wait complete. 2023-08-09T14:52:33.806 DEBUG:tasks.cephfs.filesystem:Not Creating any subvols for filesystem 'cephfs' 2023-08-09T14:52:33.806 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-08-09T14:52:34.032 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:34.031+0000 7f498ffff640 1 Processor -- start 2023-08-09T14:52:34.033 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:34.032+0000 7f498ffff640 1 -- start start 2023-08-09T14:52:34.035 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:34.033+0000 7f498ffff640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f49900713b0 0x7f49900c8fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T14:52:34.035 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:34.033+0000 7f498ffff640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f49900c94f0 0x7f499015ad20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T14:52:34.035 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:34.033+0000 7f498ffff640 1 -- --> v1:172.21.15.43:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f49900cdc70 con 0x7f499015a6a0 2023-08-09T14:52:34.036 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:34.033+0000 7f498ffff640 1 -- --> v2:172.21.15.43:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f49900c9960 con 0x7f49900c94f0 2023-08-09T14:52:34.036 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:34.033+0000 7f498ffff640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4990070910 con 0x7f49900713b0 2023-08-09T14:52:34.036 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:34.033+0000 7f498effd640 1 --1- >> v1:172.21.15.43:6789/0 conn(0x7f499015a6a0 0x7f49900cbd90 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.43:6789/0 says I am v1:172.21.15.43:50508/0 (socket says 172.21.15.43:50508) 2023-08-09T14:52:34.036 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:34.034+0000 7f498effd640 1 -- 172.21.15.43:0/283909741 learned_addr learned my addr 172.21.15.43:0/283909741 (peer_addr_for_me v1:172.21.15.43:0/0) 2023-08-09T14:52:34.037 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:34.034+0000 7f498f7fe640 1 --2- 172.21.15.43:0/283909741 >> v2:172.21.15.43:3300/0 conn(0x7f49900c94f0 0x7f499015ad20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T14:52:34.037 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:34.034+0000 7f4987fff640 1 --2- 172.21.15.43:0/283909741 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f49900713b0 0x7f49900c8fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T14:52:34.037 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:34.034+0000 7f498e7fc640 1 -- 172.21.15.43:0/283909741 <== mon.0 v1:172.21.15.43:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 938786755 0 0) 0x7f49900cdc70 con 0x7f499015a6a0 2023-08-09T14:52:34.037 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:34.034+0000 7f498e7fc640 1 -- 172.21.15.43:0/283909741 --> v1:172.21.15.43:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f497c003520 con 0x7f499015a6a0 2023-08-09T14:52:34.038 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:34.034+0000 7f498f7fe640 1 -- 172.21.15.43:0/283909741 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f49900713b0 msgr2=0x7f49900c8fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-09T14:52:34.038 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:34.034+0000 7f498f7fe640 1 --2- 172.21.15.43:0/283909741 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f49900713b0 0x7f49900c8fb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T14:52:34.038 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:34.034+0000 7f498f7fe640 1 -- 172.21.15.43:0/283909741 >> v1:172.21.15.43:6789/0 conn(0x7f499015a6a0 legacy=0x7f49900cbd90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T14:52:34.038 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:34.034+0000 7f498f7fe640 1 -- 172.21.15.43:0/283909741 --> v2:172.21.15.43:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f49900c88c0 con 0x7f49900c94f0 2023-08-09T14:52:34.041 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:34.035+0000 7f498f7fe640 1 --2- 172.21.15.43:0/283909741 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f49900c94f0 0x7f499015ad20 secure :-1 s=READY pgs=2038 cs=0 l=1 rev1=1 crypto rx=0x7f4988009d90 tx=0x7f498802be20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=133ad97aac4ce249 server_cookie=0 in_seq=0 out_seq=0 2023-08-09T14:52:34.041 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:34.036+0000 7f498e7fc640 1 -- 172.21.15.43:0/283909741 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f498802cbd0 con 0x7f49900c94f0 2023-08-09T14:52:34.041 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:34.036+0000 7f498e7fc640 1 -- 172.21.15.43:0/283909741 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f498802cd30 con 0x7f49900c94f0 2023-08-09T14:52:34.041 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:34.036+0000 7f498e7fc640 1 -- 172.21.15.43:0/283909741 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f498802cea0 con 0x7f49900c94f0 2023-08-09T14:52:34.041 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:34.036+0000 7f498ffff640 1 -- 172.21.15.43:0/283909741 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f49900c94f0 msgr2=0x7f499015ad20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T14:52:34.041 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:34.036+0000 7f498ffff640 1 --2- 172.21.15.43:0/283909741 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f49900c94f0 0x7f499015ad20 secure :-1 s=READY pgs=2038 cs=0 l=1 rev1=1 crypto rx=0x7f4988009d90 tx=0x7f498802be20 comp rx=0 tx=0).stop 2023-08-09T14:52:34.042 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:34.036+0000 7f498ffff640 1 -- 172.21.15.43:0/283909741 shutdown_connections 2023-08-09T14:52:34.042 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:34.036+0000 7f498ffff640 1 --2- 172.21.15.43:0/283909741 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f49900c94f0 0x7f499015ad20 unknown :-1 s=CLOSED pgs=2038 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T14:52:34.042 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:34.036+0000 7f498ffff640 1 --2- 172.21.15.43:0/283909741 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f49900713b0 0x7f49900c8fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T14:52:34.042 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:34.036+0000 7f498ffff640 1 -- 172.21.15.43:0/283909741 >> 172.21.15.43:0/283909741 conn(0x7f49900cde90 msgr2=0x7f49900ce2a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-09T14:52:34.042 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:34.036+0000 7f498ffff640 1 -- 172.21.15.43:0/283909741 shutdown_connections 2023-08-09T14:52:34.042 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:34.037+0000 7f498ffff640 1 -- 172.21.15.43:0/283909741 wait complete. 2023-08-09T14:52:34.042 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:34.038+0000 7f498ffff640 1 Processor -- start 2023-08-09T14:52:34.042 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:34.038+0000 7f498ffff640 1 -- start start 2023-08-09T14:52:34.043 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:34.039+0000 7f498ffff640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f49900713b0 0x7f499009b3a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T14:52:34.043 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:34.039+0000 7f498ffff640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f49900c94f0 0x7f49900b3410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T14:52:34.043 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:34.039+0000 7f498ffff640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f499015a6a0 0x7f4990099460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T14:52:34.043 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:34.039+0000 7f498ffff640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f499009b8e0 con 0x7f49900713b0 2023-08-09T14:52:34.043 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:34.039+0000 7f498ffff640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f499009af10 con 0x7f49900c94f0 2023-08-09T14:52:34.043 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:34.039+0000 7f498ffff640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f49900b3950 con 0x7f499015a6a0 2023-08-09T14:52:34.043 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:34.040+0000 7f498effd640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f49900713b0 0x7f499009b3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T14:52:34.044 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:34.040+0000 7f498effd640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f49900713b0 0x7f499009b3a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:55546/0 (socket says 172.21.15.43:55546) 2023-08-09T14:52:34.044 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:34.040+0000 7f498effd640 1 -- 172.21.15.43:0/3365124307 learned_addr learned my addr 172.21.15.43:0/3365124307 (peer_addr_for_me v2:172.21.15.43:0/0) 2023-08-09T14:52:34.044 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:34.040+0000 7f4987fff640 1 --2- 172.21.15.43:0/3365124307 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f49900c94f0 0x7f49900b3410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T14:52:34.044 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:34.040+0000 7f498f7fe640 1 --2- 172.21.15.43:0/3365124307 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f499015a6a0 0x7f4990099460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T14:52:34.044 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:34.040+0000 7f498effd640 1 -- 172.21.15.43:0/3365124307 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f499015a6a0 msgr2=0x7f4990099460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T14:52:34.044 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:34.040+0000 7f498effd640 1 --2- 172.21.15.43:0/3365124307 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f499015a6a0 0x7f4990099460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T14:52:34.044 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:34.040+0000 7f498effd640 1 -- 172.21.15.43:0/3365124307 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f49900c94f0 msgr2=0x7f49900b3410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T14:52:34.044 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:34.040+0000 7f498effd640 1 --2- 172.21.15.43:0/3365124307 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f49900c94f0 0x7f49900b3410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T14:52:34.045 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:34.040+0000 7f498effd640 1 -- 172.21.15.43:0/3365124307 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4988009a70 con 0x7f49900713b0 2023-08-09T14:52:34.045 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:34.040+0000 7f4987fff640 1 --2- 172.21.15.43:0/3365124307 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f49900c94f0 0x7f49900b3410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-08-09T14:52:34.045 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:34.040+0000 7f498effd640 1 --2- 172.21.15.43:0/3365124307 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f49900713b0 0x7f499009b3a0 secure :-1 s=READY pgs=2039 cs=0 l=1 rev1=1 crypto rx=0x7f498000c500 tx=0x7f498000c9c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-09T14:52:34.046 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:34.041+0000 7f49877fe640 1 -- 172.21.15.43:0/3365124307 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4980005ce0 con 0x7f49900713b0 2023-08-09T14:52:34.046 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:34.041+0000 7f49877fe640 1 -- 172.21.15.43:0/3365124307 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4980005e40 con 0x7f49900713b0 2023-08-09T14:52:34.046 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:34.041+0000 7f49877fe640 1 -- 172.21.15.43:0/3365124307 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4980007330 con 0x7f49900713b0 2023-08-09T14:52:34.046 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:34.041+0000 7f498ffff640 1 -- 172.21.15.43:0/3365124307 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f499009a300 con 0x7f49900713b0 2023-08-09T14:52:34.046 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:34.041+0000 7f498ffff640 1 -- 172.21.15.43:0/3365124307 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f499009a8c0 con 0x7f49900713b0 2023-08-09T14:52:34.047 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:34.042+0000 7f498ffff640 1 -- 172.21.15.43:0/3365124307 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f49900739f0 con 0x7f49900713b0 2023-08-09T14:52:34.047 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:34.043+0000 7f49877fe640 1 -- 172.21.15.43:0/3365124307 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 88) v1 ==== 88443+0+0 (secure 0 0 0) 0x7f4980007550 con 0x7f49900713b0 2023-08-09T14:52:34.047 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:34.044+0000 7f49877fe640 1 --2- 172.21.15.43:0/3365124307 >> [v2:172.21.15.203:6832/2959400181,v1:172.21.15.203:6833/2959400181] conn(0x7f497006a170 0x7f497006c630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T14:52:34.047 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:34.044+0000 7f49877fe640 1 -- 172.21.15.43:0/3365124307 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(186..186 src has 1..186) v4 ==== 6426+0+0 (secure 0 0 0) 0x7f4980087f10 con 0x7f49900713b0 2023-08-09T14:52:34.047 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:34.045+0000 7f4987fff640 1 --2- 172.21.15.43:0/3365124307 >> [v2:172.21.15.203:6832/2959400181,v1:172.21.15.203:6833/2959400181] conn(0x7f497006a170 0x7f497006c630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T14:52:34.048 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:34.046+0000 7f4987fff640 1 --2- 172.21.15.43:0/3365124307 >> [v2:172.21.15.203:6832/2959400181,v1:172.21.15.203:6833/2959400181] conn(0x7f497006a170 0x7f497006c630 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f4978006fd0 tx=0x7f4978008040 comp rx=0 tx=0).ready entity=mgr.7223 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-09T14:52:34.049 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:34.048+0000 7f49877fe640 1 -- 172.21.15.43:0/3365124307 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178066 (secure 0 0 0) 0x7f4980053430 con 0x7f49900713b0 2023-08-09T14:52:34.279 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:34.278+0000 7f498ffff640 1 -- 172.21.15.43:0/3365124307 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f49900cec10 con 0x7f49900713b0 2023-08-09T14:52:34.280 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:34.280+0000 7f49877fe640 1 -- 172.21.15.43:0/3365124307 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 87 v87) v1 ==== 94+0+4882 (secure 0 0 0) 0x7f4980017020 con 0x7f49900713b0 2023-08-09T14:52:34.281 INFO:teuthology.orchestra.run.smithi043.stdout: 2023-08-09T14:52:34.281 INFO:teuthology.orchestra.run.smithi043.stdout:{"epoch":87,"default_fscid":10,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":7328,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.203:6836/3701396136","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6834","nonce":3701396136},{"type":"v1","addr":"172.21.15.203:6836","nonce":3701396136}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":82},{"gid":7343,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.43:6835/2096611841","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6833","nonce":2096611841},{"type":"v1","addr":"172.21.15.43:6835","nonce":2096611841}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":82},{"gid":8242,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.203:6837/18754135","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6835","nonce":18754135},{"type":"v1","addr":"172.21.15.203:6837","nonce":18754135}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":81}],"filesystems":[{"mdsmap":{"epoch":87,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-08-09T14:52:30.188965+0000","modified":"2023-08-09T14:52:33.772528+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":10185},"failed":[],"damaged":[],"stopped":[],"info":{"gid_10185":{"gid":10185,"name":"c","rank":0,"incarnation":83,"state":"up:active","state_seq":3,"addr":"172.21.15.43:6834/2866936881","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6832","nonce":2866936881},{"type":"v1","addr":"172.21.15.43:6834","nonce":2866936881}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[23],"metadata_pool":22,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":10}]} 2023-08-09T14:52:34.281 INFO:teuthology.orchestra.run.smithi043.stderr:dumped fsmap epoch 87 2023-08-09T14:52:34.283 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:34.282+0000 7f498ffff640 1 -- 172.21.15.43:0/3365124307 >> [v2:172.21.15.203:6832/2959400181,v1:172.21.15.203:6833/2959400181] conn(0x7f497006a170 msgr2=0x7f497006c630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T14:52:34.283 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:34.282+0000 7f498ffff640 1 --2- 172.21.15.43:0/3365124307 >> [v2:172.21.15.203:6832/2959400181,v1:172.21.15.203:6833/2959400181] conn(0x7f497006a170 0x7f497006c630 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f4978006fd0 tx=0x7f4978008040 comp rx=0 tx=0).stop 2023-08-09T14:52:34.283 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:34.282+0000 7f498ffff640 1 -- 172.21.15.43:0/3365124307 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f49900713b0 msgr2=0x7f499009b3a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T14:52:34.283 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:34.282+0000 7f498ffff640 1 --2- 172.21.15.43:0/3365124307 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f49900713b0 0x7f499009b3a0 secure :-1 s=READY pgs=2039 cs=0 l=1 rev1=1 crypto rx=0x7f498000c500 tx=0x7f498000c9c0 comp rx=0 tx=0).stop 2023-08-09T14:52:34.284 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:34.283+0000 7f498ffff640 1 -- 172.21.15.43:0/3365124307 shutdown_connections 2023-08-09T14:52:34.284 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:34.283+0000 7f498ffff640 1 --2- 172.21.15.43:0/3365124307 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f499015a6a0 0x7f4990099460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T14:52:34.284 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:34.283+0000 7f498ffff640 1 --2- 172.21.15.43:0/3365124307 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f49900c94f0 0x7f49900b3410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T14:52:34.284 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:34.283+0000 7f498ffff640 1 --2- 172.21.15.43:0/3365124307 >> [v2:172.21.15.203:6832/2959400181,v1:172.21.15.203:6833/2959400181] conn(0x7f497006a170 0x7f497006c630 unknown :-1 s=CLOSED pgs=64 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T14:52:34.285 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:34.283+0000 7f498ffff640 1 --2- 172.21.15.43:0/3365124307 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f49900713b0 0x7f499009b3a0 unknown :-1 s=CLOSED pgs=2039 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T14:52:34.285 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:34.283+0000 7f498ffff640 1 -- 172.21.15.43:0/3365124307 >> 172.21.15.43:0/3365124307 conn(0x7f49900cde90 msgr2=0x7f4990157c90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-09T14:52:34.285 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:34.283+0000 7f498ffff640 1 -- 172.21.15.43:0/3365124307 shutdown_connections 2023-08-09T14:52:34.285 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:34.283+0000 7f498ffff640 1 -- 172.21.15.43:0/3365124307 wait complete. 2023-08-09T14:52:34.303 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-08-09T14:52:34.521 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:34.520+0000 7f185b59e640 1 Processor -- start 2023-08-09T14:52:34.521 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:34.521+0000 7f185b59e640 1 -- start start 2023-08-09T14:52:34.522 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:34.522+0000 7f185b59e640 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7f185c15f7b0 0x7f185c0b4760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T14:52:34.523 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:34.522+0000 7f185b59e640 1 -- --> v1:172.21.15.43:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f185c0b7bf0 con 0x7f185c0b1050 2023-08-09T14:52:34.523 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:34.522+0000 7f185b59e640 1 -- --> v1:172.21.15.203:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f185c0b7ed0 con 0x7f185c0cd6b0 2023-08-09T14:52:34.524 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:34.523+0000 7f185b59e640 1 -- --> v2:172.21.15.203:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f185c0d0370 con 0x7f185c15f7b0 2023-08-09T14:52:34.524 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:34.523+0000 7f1853fff640 1 --1- >> v1:172.21.15.43:6789/0 conn(0x7f185c0b1050 0x7f185c0ccfa0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.43:6789/0 says I am v1:172.21.15.43:50510/0 (socket says 172.21.15.43:50510) 2023-08-09T14:52:34.524 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:34.523+0000 7f1853fff640 1 -- 172.21.15.43:0/3276893034 learned_addr learned my addr 172.21.15.43:0/3276893034 (peer_addr_for_me v1:172.21.15.43:0/0) 2023-08-09T14:52:34.524 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:34.523+0000 7f185a59c640 1 --2- 172.21.15.43:0/3276893034 >> v2:172.21.15.203:3300/0 conn(0x7f185c15f7b0 0x7f185c0b4760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T14:52:34.526 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:34.523+0000 7f1859d9b640 1 -- 172.21.15.43:0/3276893034 <== mon.0 v1:172.21.15.43:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2140689430 0 0) 0x7f185c0b7bf0 con 0x7f185c0b1050 2023-08-09T14:52:34.526 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:34.523+0000 7f1859d9b640 1 -- 172.21.15.43:0/3276893034 --> v1:172.21.15.43:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f1848003520 con 0x7f185c0b1050 2023-08-09T14:52:34.526 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:34.524+0000 7f1859d9b640 1 -- 172.21.15.43:0/3276893034 <== mon.0 v1:172.21.15.43:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2851321541 0 0) 0x7f1848003520 con 0x7f185c0b1050 2023-08-09T14:52:34.526 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:34.524+0000 7f1859d9b640 1 -- 172.21.15.43:0/3276893034 >> v2:172.21.15.203:3300/0 conn(0x7f185c15f7b0 msgr2=0x7f185c0b4760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-09T14:52:34.526 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:34.524+0000 7f1859d9b640 1 --2- 172.21.15.43:0/3276893034 >> v2:172.21.15.203:3300/0 conn(0x7f185c15f7b0 0x7f185c0b4760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T14:52:34.526 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:34.524+0000 7f1859d9b640 1 -- 172.21.15.43:0/3276893034 >> v1:172.21.15.203:6789/0 conn(0x7f185c0cd6b0 legacy=0x7f185c0d0e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T14:52:34.527 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:34.524+0000 7f1859d9b640 1 -- 172.21.15.43:0/3276893034 --> v1:172.21.15.43:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f185c0d0650 con 0x7f185c0b1050 2023-08-09T14:52:34.527 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:34.524+0000 7f1859d9b640 1 -- 172.21.15.43:0/3276893034 <== mon.0 v1:172.21.15.43:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1285150740 0 0) 0x7f1844001670 con 0x7f185c0b1050 2023-08-09T14:52:34.527 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:34.524+0000 7f1859d9b640 1 -- 172.21.15.43:0/3276893034 >> v1:172.21.15.43:6789/0 conn(0x7f185c0b1050 legacy=0x7f185c0ccfa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T14:52:34.527 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:34.524+0000 7f1859d9b640 1 --2- 172.21.15.43:0/3276893034 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f1848003970 0x7f1848003d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T14:52:34.527 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:34.524+0000 7f1859d9b640 1 --2- 172.21.15.43:0/3276893034 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f18480047c0 0x7f1848004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T14:52:34.527 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:34.524+0000 7f1859d9b640 1 --2- 172.21.15.43:0/3276893034 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f1848005180 0x7f184800b980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T14:52:34.527 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:34.524+0000 7f1859d9b640 1 -- 172.21.15.43:0/3276893034 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f185c0d0370 con 0x7f18480047c0 2023-08-09T14:52:34.528 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:34.524+0000 7f1859d9b640 1 -- 172.21.15.43:0/3276893034 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f184800c020 con 0x7f1848005180 2023-08-09T14:52:34.528 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:34.524+0000 7f1859d9b640 1 -- 172.21.15.43:0/3276893034 --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f184800c1e0 con 0x7f1848003970 2023-08-09T14:52:34.528 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:34.524+0000 7f1853fff640 1 --2- 172.21.15.43:0/3276893034 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f18480047c0 0x7f1848004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T14:52:34.528 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:34.525+0000 7f185a59c640 1 --2- 172.21.15.43:0/3276893034 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f1848003970 0x7f1848003d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T14:52:34.528 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:34.525+0000 7f1853fff640 1 -- 172.21.15.43:0/3276893034 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f1848003970 msgr2=0x7f1848003d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-09T14:52:34.528 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:34.525+0000 7f1853fff640 1 --2- 172.21.15.43:0/3276893034 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f1848003970 0x7f1848003d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T14:52:34.528 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:34.525+0000 7f1853fff640 1 -- 172.21.15.43:0/3276893034 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f1848005180 msgr2=0x7f184800b980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-09T14:52:34.529 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:34.525+0000 7f1853fff640 1 --2- 172.21.15.43:0/3276893034 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f1848005180 0x7f184800b980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T14:52:34.529 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:34.525+0000 7f1853fff640 1 -- 172.21.15.43:0/3276893034 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1848003520 con 0x7f18480047c0 2023-08-09T14:52:34.529 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:34.525+0000 7f185a59c640 1 --2- 172.21.15.43:0/3276893034 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f1848003970 0x7f1848003d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-08-09T14:52:34.529 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:34.525+0000 7f185ad9d640 1 -- 172.21.15.43:0/3276893034 reap_dead start 2023-08-09T14:52:34.532 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:34.526+0000 7f1853fff640 1 --2- 172.21.15.43:0/3276893034 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f18480047c0 0x7f1848004c40 secure :-1 s=READY pgs=2041 cs=0 l=1 rev1=1 crypto rx=0x7f185c0b0eb0 tx=0x7f18480057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ea72424c3cf7d096 server_cookie=0 in_seq=0 out_seq=0 2023-08-09T14:52:34.532 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:34.526+0000 7f1859d9b640 1 -- 172.21.15.43:0/3276893034 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1844005ad0 con 0x7f18480047c0 2023-08-09T14:52:34.532 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:34.526+0000 7f1859d9b640 1 -- 172.21.15.43:0/3276893034 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1844005c30 con 0x7f18480047c0 2023-08-09T14:52:34.533 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:34.526+0000 7f1859d9b640 1 -- 172.21.15.43:0/3276893034 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f184402ca90 con 0x7f18480047c0 2023-08-09T14:52:34.533 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:34.527+0000 7f185b59e640 1 -- 172.21.15.43:0/3276893034 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f18480047c0 msgr2=0x7f1848004c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T14:52:34.533 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:34.527+0000 7f185b59e640 1 --2- 172.21.15.43:0/3276893034 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f18480047c0 0x7f1848004c40 secure :-1 s=READY pgs=2041 cs=0 l=1 rev1=1 crypto rx=0x7f185c0b0eb0 tx=0x7f18480057f0 comp rx=0 tx=0).stop 2023-08-09T14:52:34.533 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:34.527+0000 7f185b59e640 1 -- 172.21.15.43:0/3276893034 shutdown_connections 2023-08-09T14:52:34.533 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:34.527+0000 7f185b59e640 1 --2- 172.21.15.43:0/3276893034 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f18480047c0 0x7f1848004c40 unknown :-1 s=CLOSED pgs=2041 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T14:52:34.533 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:34.527+0000 7f185b59e640 1 -- 172.21.15.43:0/3276893034 >> 172.21.15.43:0/3276893034 conn(0x7f185c06a2b0 msgr2=0x7f185c06a690 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-09T14:52:34.533 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:34.527+0000 7f185b59e640 1 -- 172.21.15.43:0/3276893034 shutdown_connections 2023-08-09T14:52:34.534 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:34.527+0000 7f185b59e640 1 -- 172.21.15.43:0/3276893034 wait complete. 2023-08-09T14:52:34.534 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:34.529+0000 7f185b59e640 1 Processor -- start 2023-08-09T14:52:34.534 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:34.529+0000 7f185b59e640 1 -- start start 2023-08-09T14:52:34.534 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:34.530+0000 7f185b59e640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f18480047c0 0x7f185c0b04c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T14:52:34.534 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:34.530+0000 7f185b59e640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f185c1f1730 0x7f185c1f1bb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T14:52:34.534 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:34.530+0000 7f185b59e640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f185c0ca2b0 0x7f185c0ca740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T14:52:34.534 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:34.530+0000 7f185b59e640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f185c0b0a00 con 0x7f185c0ca2b0 2023-08-09T14:52:34.534 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:34.530+0000 7f185b59e640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f185c0cae50 con 0x7f18480047c0 2023-08-09T14:52:34.535 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:34.530+0000 7f185b59e640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f185c0cc660 con 0x7f185c1f1730 2023-08-09T14:52:34.535 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:34.530+0000 7f185ad9d640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f185c0ca2b0 0x7f185c0ca740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T14:52:34.535 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:34.530+0000 7f185ad9d640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f185c0ca2b0 0x7f185c0ca740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:55564/0 (socket says 172.21.15.43:55564) 2023-08-09T14:52:34.535 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:34.530+0000 7f185ad9d640 1 -- 172.21.15.43:0/3049749213 learned_addr learned my addr 172.21.15.43:0/3049749213 (peer_addr_for_me v2:172.21.15.43:0/0) 2023-08-09T14:52:34.535 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:34.531+0000 7f1853fff640 1 --2- 172.21.15.43:0/3049749213 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f185c1f1730 0x7f185c1f1bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T14:52:34.535 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:34.531+0000 7f185a59c640 1 --2- 172.21.15.43:0/3049749213 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f18480047c0 0x7f185c0b04c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T14:52:34.535 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:34.531+0000 7f185ad9d640 1 -- 172.21.15.43:0/3049749213 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f185c1f1730 msgr2=0x7f185c1f1bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T14:52:34.535 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:34.531+0000 7f185ad9d640 1 --2- 172.21.15.43:0/3049749213 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f185c1f1730 0x7f185c1f1bb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T14:52:34.536 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:34.531+0000 7f185ad9d640 1 -- 172.21.15.43:0/3049749213 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f18480047c0 msgr2=0x7f185c0b04c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T14:52:34.536 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:34.531+0000 7f185ad9d640 1 --2- 172.21.15.43:0/3049749213 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f18480047c0 0x7f185c0b04c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T14:52:34.536 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:34.531+0000 7f185ad9d640 1 -- 172.21.15.43:0/3049749213 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f185c157e20 con 0x7f185c0ca2b0 2023-08-09T14:52:34.536 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:34.531+0000 7f1853fff640 1 --2- 172.21.15.43:0/3049749213 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f185c1f1730 0x7f185c1f1bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-08-09T14:52:34.536 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:34.531+0000 7f185a59c640 1 --2- 172.21.15.43:0/3049749213 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f18480047c0 0x7f185c0b04c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-08-09T14:52:34.537 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:34.531+0000 7f185ad9d640 1 --2- 172.21.15.43:0/3049749213 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f185c0ca2b0 0x7f185c0ca740 secure :-1 s=READY pgs=2042 cs=0 l=1 rev1=1 crypto rx=0x7f185c06af30 tx=0x7f185400c960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-09T14:52:34.537 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:34.531+0000 7f1852ffd640 1 -- 172.21.15.43:0/3049749213 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1854007070 con 0x7f185c0ca2b0 2023-08-09T14:52:34.537 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:34.531+0000 7f1852ffd640 1 -- 172.21.15.43:0/3049749213 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f18540071d0 con 0x7f185c0ca2b0 2023-08-09T14:52:34.537 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:34.531+0000 7f1852ffd640 1 -- 172.21.15.43:0/3049749213 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1854007560 con 0x7f185c0ca2b0 2023-08-09T14:52:34.537 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:34.531+0000 7f185b59e640 1 -- 172.21.15.43:0/3049749213 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f185c158100 con 0x7f185c0ca2b0 2023-08-09T14:52:34.537 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:34.532+0000 7f185b59e640 1 -- 172.21.15.43:0/3049749213 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f185c1586c0 con 0x7f185c0ca2b0 2023-08-09T14:52:34.538 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:34.533+0000 7f185b59e640 1 -- 172.21.15.43:0/3049749213 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f185c1588e0 con 0x7f185c0ca2b0 2023-08-09T14:52:34.538 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:34.534+0000 7f1852ffd640 1 -- 172.21.15.43:0/3049749213 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 88) v1 ==== 88443+0+0 (secure 0 0 0) 0x7f18540076c0 con 0x7f185c0ca2b0 2023-08-09T14:52:34.538 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:34.535+0000 7f1852ffd640 1 --2- 172.21.15.43:0/3049749213 >> [v2:172.21.15.203:6832/2959400181,v1:172.21.15.203:6833/2959400181] conn(0x7f183c06a170 0x7f183c06c630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T14:52:34.539 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:34.535+0000 7f1852ffd640 1 -- 172.21.15.43:0/3049749213 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(186..186 src has 1..186) v4 ==== 6426+0+0 (secure 0 0 0) 0x7f1854087d20 con 0x7f185c0ca2b0 2023-08-09T14:52:34.539 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:34.536+0000 7f185a59c640 1 --2- 172.21.15.43:0/3049749213 >> [v2:172.21.15.203:6832/2959400181,v1:172.21.15.203:6833/2959400181] conn(0x7f183c06a170 0x7f183c06c630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T14:52:34.539 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:34.537+0000 7f185a59c640 1 --2- 172.21.15.43:0/3049749213 >> [v2:172.21.15.203:6832/2959400181,v1:172.21.15.203:6833/2959400181] conn(0x7f183c06a170 0x7f183c06c630 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f185c0b0d60 tx=0x7f184c008040 comp rx=0 tx=0).ready entity=mgr.7223 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-09T14:52:34.539 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:34.539+0000 7f1852ffd640 1 -- 172.21.15.43:0/3049749213 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178066 (secure 0 0 0) 0x7f18540530f0 con 0x7f185c0ca2b0 2023-08-09T14:52:34.733 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:34.732+0000 7f185b59e640 1 -- 172.21.15.43:0/3049749213 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f185c0739f0 con 0x7f185c0ca2b0 2023-08-09T14:52:34.735 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:34.735+0000 7f1852ffd640 1 -- 172.21.15.43:0/3049749213 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v186) v1 ==== 74+0+18558 (secure 0 0 0) 0x7f1854017020 con 0x7f185c0ca2b0 2023-08-09T14:52:34.736 INFO:teuthology.orchestra.run.smithi043.stdout: 2023-08-09T14:52:34.736 INFO:teuthology.orchestra.run.smithi043.stdout:{"epoch":186,"fsid":"daac87ee-1337-4250-a955-aea4e408a39b","created":"2023-08-09T13:33:41.701198+0000","modified":"2023-08-09T14:52:31.739471+0000","last_up_change":"2023-08-09T13:33:53.783453+0000","last_in_change":"2023-08-09T13:33:45.148904+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":23,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":"rbd","create_time":"2023-08-09T13:33:54.365137+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":".mgr","create_time":"2023-08-09T13:33:54.831803+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":22,"pool_name":"cephfs_metadata","create_time":"2023-08-09T14:52:28.569334+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"185","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":23,"pool_name":"cephfs_data","create_time":"2023-08-09T14:52:29.209973+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"186","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"cbd49ecd-e896-4fbd-864e-5c37a829b165","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":184,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6800","nonce":502667335},{"type":"v1","addr":"172.21.15.43:6801","nonce":502667335}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6802","nonce":502667335},{"type":"v1","addr":"172.21.15.43:6803","nonce":502667335}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6806","nonce":502667335},{"type":"v1","addr":"172.21.15.43:6807","nonce":502667335}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6804","nonce":502667335},{"type":"v1","addr":"172.21.15.43:6805","nonce":502667335}]},"public_addr":"172.21.15.43:6801/502667335","cluster_addr":"172.21.15.43:6803/502667335","heartbeat_back_addr":"172.21.15.43:6807/502667335","heartbeat_front_addr":"172.21.15.43:6805/502667335","state":["exists","up"]},{"osd":1,"uuid":"75c7e0fe-1ec7-4fe8-a2f2-3661a65b8500","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":184,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6808","nonce":3289134610},{"type":"v1","addr":"172.21.15.43:6809","nonce":3289134610}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6810","nonce":3289134610},{"type":"v1","addr":"172.21.15.43:6812","nonce":3289134610}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6821","nonce":3289134610},{"type":"v1","addr":"172.21.15.43:6824","nonce":3289134610}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6815","nonce":3289134610},{"type":"v1","addr":"172.21.15.43:6818","nonce":3289134610}]},"public_addr":"172.21.15.43:6809/3289134610","cluster_addr":"172.21.15.43:6812/3289134610","heartbeat_back_addr":"172.21.15.43:6824/3289134610","heartbeat_front_addr":"172.21.15.43:6818/3289134610","state":["exists","up"]},{"osd":2,"uuid":"bb5c41e9-7f5f-4125-9b01-3bb8bfc2ea83","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":184,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6811","nonce":4079709736},{"type":"v1","addr":"172.21.15.43:6814","nonce":4079709736}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6817","nonce":4079709736},{"type":"v1","addr":"172.21.15.43:6820","nonce":4079709736}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6829","nonce":4079709736},{"type":"v1","addr":"172.21.15.43:6831","nonce":4079709736}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6823","nonce":4079709736},{"type":"v1","addr":"172.21.15.43:6826","nonce":4079709736}]},"public_addr":"172.21.15.43:6814/4079709736","cluster_addr":"172.21.15.43:6820/4079709736","heartbeat_back_addr":"172.21.15.43:6831/4079709736","heartbeat_front_addr":"172.21.15.43:6826/4079709736","state":["exists","up"]},{"osd":3,"uuid":"bfe4c8b4-694a-465c-87e6-2b2aab852613","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":184,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6813","nonce":1709538759},{"type":"v1","addr":"172.21.15.43:6816","nonce":1709538759}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6819","nonce":1709538759},{"type":"v1","addr":"172.21.15.43:6822","nonce":1709538759}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6828","nonce":1709538759},{"type":"v1","addr":"172.21.15.43:6830","nonce":1709538759}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6825","nonce":1709538759},{"type":"v1","addr":"172.21.15.43:6827","nonce":1709538759}]},"public_addr":"172.21.15.43:6816/1709538759","cluster_addr":"172.21.15.43:6822/1709538759","heartbeat_back_addr":"172.21.15.43:6830/1709538759","heartbeat_front_addr":"172.21.15.43:6827/1709538759","state":["exists","up"]},{"osd":4,"uuid":"fa2e8a77-aba1-4756-853e-0d174a5bdeba","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":184,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6800","nonce":4196866481},{"type":"v1","addr":"172.21.15.203:6801","nonce":4196866481}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6802","nonce":4196866481},{"type":"v1","addr":"172.21.15.203:6803","nonce":4196866481}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6806","nonce":4196866481},{"type":"v1","addr":"172.21.15.203:6807","nonce":4196866481}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6804","nonce":4196866481},{"type":"v1","addr":"172.21.15.203:6805","nonce":4196866481}]},"public_addr":"172.21.15.203:6801/4196866481","cluster_addr":"172.21.15.203:6803/4196866481","heartbeat_back_addr":"172.21.15.203:6807/4196866481","heartbeat_front_addr":"172.21.15.203:6805/4196866481","state":["exists","up"]},{"osd":5,"uuid":"81811e4b-4d24-4f51-a82c-f1d700a2a00d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":184,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6816","nonce":3059290081},{"type":"v1","addr":"172.21.15.203:6817","nonce":3059290081}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6818","nonce":3059290081},{"type":"v1","addr":"172.21.15.203:6819","nonce":3059290081}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6822","nonce":3059290081},{"type":"v1","addr":"172.21.15.203:6823","nonce":3059290081}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6820","nonce":3059290081},{"type":"v1","addr":"172.21.15.203:6821","nonce":3059290081}]},"public_addr":"172.21.15.203:6817/3059290081","cluster_addr":"172.21.15.203:6819/3059290081","heartbeat_back_addr":"172.21.15.203:6823/3059290081","heartbeat_front_addr":"172.21.15.203:6821/3059290081","state":["exists","up"]},{"osd":6,"uuid":"307bac4d-6a5b-443d-af75-56d695d7674f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":184,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6824","nonce":512155569},{"type":"v1","addr":"172.21.15.203:6825","nonce":512155569}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6826","nonce":512155569},{"type":"v1","addr":"172.21.15.203:6827","nonce":512155569}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6830","nonce":512155569},{"type":"v1","addr":"172.21.15.203:6831","nonce":512155569}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6828","nonce":512155569},{"type":"v1","addr":"172.21.15.203:6829","nonce":512155569}]},"public_addr":"172.21.15.203:6825/512155569","cluster_addr":"172.21.15.203:6827/512155569","heartbeat_back_addr":"172.21.15.203:6831/512155569","heartbeat_front_addr":"172.21.15.203:6829/512155569","state":["exists","up"]},{"osd":7,"uuid":"a95dc7ca-9f1b-4295-af70-68a22a2d7d5e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":184,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6808","nonce":3331450438},{"type":"v1","addr":"172.21.15.203:6809","nonce":3331450438}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6810","nonce":3331450438},{"type":"v1","addr":"172.21.15.203:6811","nonce":3331450438}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6814","nonce":3331450438},{"type":"v1","addr":"172.21.15.203:6815","nonce":3331450438}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6812","nonce":3331450438},{"type":"v1","addr":"172.21.15.203:6813","nonce":3331450438}]},"public_addr":"172.21.15.203:6809/3331450438","cluster_addr":"172.21.15.203:6811/3331450438","heartbeat_back_addr":"172.21.15.203:6815/3331450438","heartbeat_front_addr":"172.21.15.203:6813/3331450438","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-09T13:33:50.547652+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-09T13:33:50.589469+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-09T13:33:50.561724+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-09T13:33:50.599650+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-09T13:33:50.411855+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-09T13:33:50.219681+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-09T13:33:50.504664+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-09T13:33:50.441248+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":7,"snaps":[{"begin":104,"length":1},{"begin":221,"length":1}]}],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-08-09T14:52:34.738 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:34.738+0000 7f185b59e640 1 -- 172.21.15.43:0/3049749213 >> [v2:172.21.15.203:6832/2959400181,v1:172.21.15.203:6833/2959400181] conn(0x7f183c06a170 msgr2=0x7f183c06c630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T14:52:34.739 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:34.738+0000 7f185b59e640 1 --2- 172.21.15.43:0/3049749213 >> [v2:172.21.15.203:6832/2959400181,v1:172.21.15.203:6833/2959400181] conn(0x7f183c06a170 0x7f183c06c630 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f185c0b0d60 tx=0x7f184c008040 comp rx=0 tx=0).stop 2023-08-09T14:52:34.739 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:34.738+0000 7f185b59e640 1 -- 172.21.15.43:0/3049749213 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f185c0ca2b0 msgr2=0x7f185c0ca740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T14:52:34.739 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:34.738+0000 7f185b59e640 1 --2- 172.21.15.43:0/3049749213 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f185c0ca2b0 0x7f185c0ca740 secure :-1 s=READY pgs=2042 cs=0 l=1 rev1=1 crypto rx=0x7f185c06af30 tx=0x7f185400c960 comp rx=0 tx=0).stop 2023-08-09T14:52:34.739 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:34.738+0000 7f185b59e640 1 -- 172.21.15.43:0/3049749213 shutdown_connections 2023-08-09T14:52:34.739 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:34.738+0000 7f185b59e640 1 --2- 172.21.15.43:0/3049749213 >> [v2:172.21.15.203:6832/2959400181,v1:172.21.15.203:6833/2959400181] conn(0x7f183c06a170 0x7f183c06c630 unknown :-1 s=CLOSED pgs=65 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T14:52:34.739 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:34.738+0000 7f185b59e640 1 --2- 172.21.15.43:0/3049749213 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f185c0ca2b0 0x7f185c0ca740 unknown :-1 s=CLOSED pgs=2042 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T14:52:34.740 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:34.738+0000 7f185b59e640 1 --2- 172.21.15.43:0/3049749213 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f185c1f1730 0x7f185c1f1bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T14:52:34.740 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:34.738+0000 7f185b59e640 1 --2- 172.21.15.43:0/3049749213 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f18480047c0 0x7f185c0b04c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T14:52:34.740 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:34.738+0000 7f185b59e640 1 -- 172.21.15.43:0/3049749213 >> 172.21.15.43:0/3049749213 conn(0x7f185c06a2b0 msgr2=0x7f185c0d0de0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-09T14:52:34.740 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:34.738+0000 7f185b59e640 1 -- 172.21.15.43:0/3049749213 shutdown_connections 2023-08-09T14:52:34.740 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:34.738+0000 7f185b59e640 1 -- 172.21.15.43:0/3049749213 wait complete. 2023-08-09T14:52:34.758 INFO:tasks.cephfs.filesystem.ceph_manager:waiting for clean 2023-08-09T14:52:34.758 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-08-09T14:52:34.974 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:34.973+0000 7f92eab86640 1 Processor -- start 2023-08-09T14:52:34.975 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:34.974+0000 7f92eab86640 1 -- start start 2023-08-09T14:52:34.976 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:34.975+0000 7f92eab86640 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7f92e4142ca0 0x7f92e40c96e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T14:52:34.976 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:34.975+0000 7f92eab86640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f92e415ce40 0x7f92e40cce00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T14:52:34.976 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:34.975+0000 7f92eab86640 1 -- --> v1:172.21.15.43:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f92e4143080 con 0x7f92e40cd340 2023-08-09T14:52:34.976 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:34.975+0000 7f92eab86640 1 -- --> v2:172.21.15.43:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f92e40cd750 con 0x7f92e415ce40 2023-08-09T14:52:34.976 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:34.975+0000 7f92eab86640 1 -- --> v2:172.21.15.203:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f92e406d400 con 0x7f92e4142ca0 2023-08-09T14:52:34.977 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:34.976+0000 7f92e9123640 1 --1- >> v1:172.21.15.43:6789/0 conn(0x7f92e40cd340 0x7f92e40cbc70 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.43:6789/0 says I am v1:172.21.15.43:46216/0 (socket says 172.21.15.43:46216) 2023-08-09T14:52:34.977 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:34.976+0000 7f92e9123640 1 -- 172.21.15.43:0/1424102159 learned_addr learned my addr 172.21.15.43:0/1424102159 (peer_addr_for_me v1:172.21.15.43:0/0) 2023-08-09T14:52:34.978 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:34.976+0000 7f92e3fff640 1 --2- 172.21.15.43:0/1424102159 >> v2:172.21.15.43:3300/0 conn(0x7f92e415ce40 0x7f92e40cce00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T14:52:34.978 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:34.976+0000 7f92e8922640 1 --2- 172.21.15.43:0/1424102159 >> v2:172.21.15.203:3300/0 conn(0x7f92e4142ca0 0x7f92e40c96e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T14:52:34.978 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:34.976+0000 7f92e3fff640 1 -- 172.21.15.43:0/1424102159 >> v2:172.21.15.203:3300/0 conn(0x7f92e4142ca0 msgr2=0x7f92e40c96e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-09T14:52:34.978 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:34.976+0000 7f92e3fff640 1 --2- 172.21.15.43:0/1424102159 >> v2:172.21.15.203:3300/0 conn(0x7f92e4142ca0 0x7f92e40c96e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T14:52:34.978 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:34.976+0000 7f92e3fff640 1 -- 172.21.15.43:0/1424102159 >> v1:172.21.15.43:6789/0 conn(0x7f92e40cd340 legacy=0x7f92e40cbc70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T14:52:34.978 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:34.976+0000 7f92e3fff640 1 -- 172.21.15.43:0/1424102159 --> v2:172.21.15.43:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f92e40c9d90 con 0x7f92e415ce40 2023-08-09T14:52:34.978 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:34.976+0000 7f92e37fe640 1 -- 172.21.15.43:0/1424102159 <== mon.0 v1:172.21.15.43:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3205881068 0 0) 0x7f92e4143080 con 0x7f92e40cd340 2023-08-09T14:52:34.979 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:34.978+0000 7f92e3fff640 1 --2- 172.21.15.43:0/1424102159 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f92e415ce40 0x7f92e40cce00 secure :-1 s=READY pgs=2044 cs=0 l=1 rev1=1 crypto rx=0x7f92e40c5120 tx=0x7f92d002b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1f1fcb18c41ee1ce server_cookie=0 in_seq=0 out_seq=0 2023-08-09T14:52:34.980 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:34.978+0000 7f92e37fe640 1 -- 172.21.15.43:0/1424102159 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f92d0031830 con 0x7f92e415ce40 2023-08-09T14:52:34.980 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:34.978+0000 7f92e37fe640 1 -- 172.21.15.43:0/1424102159 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f92d0031990 con 0x7f92e415ce40 2023-08-09T14:52:34.980 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:34.978+0000 7f92e37fe640 1 -- 172.21.15.43:0/1424102159 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f92d0031d20 con 0x7f92e415ce40 2023-08-09T14:52:34.980 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:34.979+0000 7f92eab86640 1 -- 172.21.15.43:0/1424102159 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f92e415ce40 msgr2=0x7f92e40cce00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T14:52:34.980 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:34.979+0000 7f92eab86640 1 --2- 172.21.15.43:0/1424102159 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f92e415ce40 0x7f92e40cce00 secure :-1 s=READY pgs=2044 cs=0 l=1 rev1=1 crypto rx=0x7f92e40c5120 tx=0x7f92d002b8d0 comp rx=0 tx=0).stop 2023-08-09T14:52:34.980 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:34.979+0000 7f92eab86640 1 -- 172.21.15.43:0/1424102159 shutdown_connections 2023-08-09T14:52:34.980 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:34.979+0000 7f92eab86640 1 --2- 172.21.15.43:0/1424102159 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f92e415ce40 0x7f92e40cce00 unknown :-1 s=CLOSED pgs=2044 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T14:52:34.981 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:34.979+0000 7f92eab86640 1 --2- 172.21.15.43:0/1424102159 >> v2:172.21.15.203:3300/0 conn(0x7f92e4142ca0 0x7f92e40c96e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T14:52:34.981 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:34.979+0000 7f92eab86640 1 -- 172.21.15.43:0/1424102159 >> 172.21.15.43:0/1424102159 conn(0x7f92e40ce0a0 msgr2=0x7f92e40ce480 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-09T14:52:34.981 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:34.979+0000 7f92eab86640 1 -- 172.21.15.43:0/1424102159 shutdown_connections 2023-08-09T14:52:34.981 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:34.979+0000 7f92eab86640 1 -- 172.21.15.43:0/1424102159 wait complete. 2023-08-09T14:52:34.981 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:34.981+0000 7f92eab86640 1 Processor -- start 2023-08-09T14:52:34.982 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:34.981+0000 7f92eab86640 1 -- start start 2023-08-09T14:52:34.983 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:34.982+0000 7f92eab86640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f92e40cd340 0x7f92e4158c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T14:52:34.983 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:34.982+0000 7f92eab86640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f92e4142ca0 0x7f92e40701b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T14:52:34.983 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:34.982+0000 7f92eab86640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f92e415ce40 0x7f92e40c6280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T14:52:34.983 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:34.982+0000 7f92eab86640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f92e4071880 con 0x7f92e4142ca0 2023-08-09T14:52:34.983 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:34.982+0000 7f92eab86640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f92e41591b0 con 0x7f92e415ce40 2023-08-09T14:52:34.983 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:34.982+0000 7f92eab86640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f92e4070c70 con 0x7f92e40cd340 2023-08-09T14:52:34.985 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:34.983+0000 7f92e3fff640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f92e4142ca0 0x7f92e40701b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T14:52:34.985 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:34.983+0000 7f92e3fff640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f92e4142ca0 0x7f92e40701b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:36294/0 (socket says 172.21.15.43:36294) 2023-08-09T14:52:34.985 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:34.983+0000 7f92e3fff640 1 -- 172.21.15.43:0/2962023291 learned_addr learned my addr 172.21.15.43:0/2962023291 (peer_addr_for_me v2:172.21.15.43:0/0) 2023-08-09T14:52:34.985 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:34.983+0000 7f92e8922640 1 --2- 172.21.15.43:0/2962023291 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f92e40cd340 0x7f92e4158c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T14:52:34.985 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:34.983+0000 7f92e9123640 1 --2- 172.21.15.43:0/2962023291 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f92e415ce40 0x7f92e40c6280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T14:52:34.986 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:34.983+0000 7f92e3fff640 1 -- 172.21.15.43:0/2962023291 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f92e40cd340 msgr2=0x7f92e4158c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T14:52:34.986 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:34.983+0000 7f92e3fff640 1 --2- 172.21.15.43:0/2962023291 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f92e40cd340 0x7f92e4158c70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T14:52:34.986 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:34.983+0000 7f92e3fff640 1 -- 172.21.15.43:0/2962023291 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f92e415ce40 msgr2=0x7f92e40c6280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T14:52:34.986 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:34.983+0000 7f92e3fff640 1 --2- 172.21.15.43:0/2962023291 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f92e415ce40 0x7f92e40c6280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T14:52:34.986 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:34.983+0000 7f92e3fff640 1 -- 172.21.15.43:0/2962023291 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f92d0009400 con 0x7f92e4142ca0 2023-08-09T14:52:34.986 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:34.984+0000 7f92e3fff640 1 --2- 172.21.15.43:0/2962023291 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f92e4142ca0 0x7f92e40701b0 secure :-1 s=READY pgs=2045 cs=0 l=1 rev1=1 crypto rx=0x7f92d0009890 tx=0x7f92d00046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-09T14:52:34.986 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:34.984+0000 7f92e17fa640 1 -- 172.21.15.43:0/2962023291 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f92d003ba70 con 0x7f92e4142ca0 2023-08-09T14:52:34.986 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:34.984+0000 7f92e17fa640 1 -- 172.21.15.43:0/2962023291 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f92d003bbf0 con 0x7f92e4142ca0 2023-08-09T14:52:34.987 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:34.984+0000 7f92e17fa640 1 -- 172.21.15.43:0/2962023291 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f92d0045400 con 0x7f92e4142ca0 2023-08-09T14:52:34.987 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:34.984+0000 7f92eab86640 1 -- 172.21.15.43:0/2962023291 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f92e4070f50 con 0x7f92e4142ca0 2023-08-09T14:52:34.987 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:34.984+0000 7f92eab86640 1 -- 172.21.15.43:0/2962023291 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f92e4071510 con 0x7f92e4142ca0 2023-08-09T14:52:34.987 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:34.986+0000 7f92eab86640 1 -- 172.21.15.43:0/2962023291 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f92e40ab950 con 0x7f92e4142ca0 2023-08-09T14:52:34.988 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:34.987+0000 7f92e17fa640 1 -- 172.21.15.43:0/2962023291 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 88) v1 ==== 88443+0+0 (secure 0 0 0) 0x7f92d004e440 con 0x7f92e4142ca0 2023-08-09T14:52:34.988 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:34.988+0000 7f92e17fa640 1 --2- 172.21.15.43:0/2962023291 >> [v2:172.21.15.203:6832/2959400181,v1:172.21.15.203:6833/2959400181] conn(0x7f92c806a170 0x7f92c806c630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T14:52:34.988 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:34.988+0000 7f92e17fa640 1 -- 172.21.15.43:0/2962023291 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(186..186 src has 1..186) v4 ==== 6426+0+0 (secure 0 0 0) 0x7f92d00aac40 con 0x7f92e4142ca0 2023-08-09T14:52:34.989 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:34.989+0000 7f92e8922640 1 --2- 172.21.15.43:0/2962023291 >> [v2:172.21.15.203:6832/2959400181,v1:172.21.15.203:6833/2959400181] conn(0x7f92c806a170 0x7f92c806c630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T14:52:34.990 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:34.990+0000 7f92e8922640 1 --2- 172.21.15.43:0/2962023291 >> [v2:172.21.15.203:6832/2959400181,v1:172.21.15.203:6833/2959400181] conn(0x7f92c806a170 0x7f92c806c630 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f92d8006fd0 tx=0x7f92d8008040 comp rx=0 tx=0).ready entity=mgr.7223 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-09T14:52:34.992 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:34.992+0000 7f92e17fa640 1 -- 172.21.15.43:0/2962023291 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178066 (secure 0 0 0) 0x7f92e40ab950 con 0x7f92e4142ca0 2023-08-09T14:52:35.185 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:35.185+0000 7f92eab86640 1 -- 172.21.15.43:0/2962023291 --> [v2:172.21.15.203:6832/2959400181,v1:172.21.15.203:6833/2959400181] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f92e40716a0 con 0x7f92c806a170 2023-08-09T14:52:35.217 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:35.216+0000 7f92e17fa640 1 -- 172.21.15.43:0/2962023291 <== mgr.7223 v2:172.21.15.203:6832/2959400181 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+309550 (secure 0 0 0) 0x7f92e40716a0 con 0x7f92c806a170 2023-08-09T14:52:35.220 INFO:teuthology.orchestra.run.smithi043.stdout: 2023-08-09T14:52:35.221 INFO:teuthology.orchestra.run.smithi043.stderr:dumped all 2023-08-09T14:52:35.223 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:35.221+0000 7f92eab86640 1 -- 172.21.15.43:0/2962023291 >> [v2:172.21.15.203:6832/2959400181,v1:172.21.15.203:6833/2959400181] conn(0x7f92c806a170 msgr2=0x7f92c806c630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T14:52:35.223 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:35.221+0000 7f92eab86640 1 --2- 172.21.15.43:0/2962023291 >> [v2:172.21.15.203:6832/2959400181,v1:172.21.15.203:6833/2959400181] conn(0x7f92c806a170 0x7f92c806c630 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f92d8006fd0 tx=0x7f92d8008040 comp rx=0 tx=0).stop 2023-08-09T14:52:35.223 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:35.222+0000 7f92eab86640 1 -- 172.21.15.43:0/2962023291 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f92e4142ca0 msgr2=0x7f92e40701b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T14:52:35.224 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:35.222+0000 7f92eab86640 1 --2- 172.21.15.43:0/2962023291 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f92e4142ca0 0x7f92e40701b0 secure :-1 s=READY pgs=2045 cs=0 l=1 rev1=1 crypto rx=0x7f92d0009890 tx=0x7f92d00046c0 comp rx=0 tx=0).stop 2023-08-09T14:52:35.224 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:35.222+0000 7f92eab86640 1 -- 172.21.15.43:0/2962023291 shutdown_connections 2023-08-09T14:52:35.224 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:35.222+0000 7f92eab86640 1 --2- 172.21.15.43:0/2962023291 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f92e415ce40 0x7f92e40c6280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T14:52:35.224 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:35.222+0000 7f92eab86640 1 --2- 172.21.15.43:0/2962023291 >> [v2:172.21.15.203:6832/2959400181,v1:172.21.15.203:6833/2959400181] conn(0x7f92c806a170 0x7f92c806c630 unknown :-1 s=CLOSED pgs=66 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T14:52:35.224 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:35.222+0000 7f92eab86640 1 --2- 172.21.15.43:0/2962023291 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f92e4142ca0 0x7f92e40701b0 unknown :-1 s=CLOSED pgs=2045 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T14:52:35.224 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:35.222+0000 7f92eab86640 1 --2- 172.21.15.43:0/2962023291 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f92e40cd340 0x7f92e4158c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T14:52:35.224 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:35.222+0000 7f92eab86640 1 -- 172.21.15.43:0/2962023291 >> 172.21.15.43:0/2962023291 conn(0x7f92e40ce0a0 msgr2=0x7f92e4141b90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-09T14:52:35.225 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:35.222+0000 7f92eab86640 1 -- 172.21.15.43:0/2962023291 shutdown_connections 2023-08-09T14:52:35.225 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:35.222+0000 7f92eab86640 1 -- 172.21.15.43:0/2962023291 wait complete. 2023-08-09T14:52:35.243 INFO:teuthology.orchestra.run.smithi043.stdout:{"pg_ready":true,"pg_map":{"version":27,"stamp":"2023-08-09T14:52:34.100798+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":23,"num_object_clones":0,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":0,"num_read":1483,"num_read_kb":3066,"num_write":487,"num_write_kb":2430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":312,"ondisk_log_size":312,"up":210,"acting":210,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":69,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":409128,"kb_used_data":19132,"kb_used_omap":13,"kb_used_meta":389874,"kb_avail":754565592,"statfs":{"total":773094113280,"available":772675166208,"internally_reserved":0,"allocated":19591168,"data_stored":11135132,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":14248,"internal_metadata":399231064},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":34,"apply_latency_ms":34,"commit_latency_ns":34000000,"apply_latency_ns":34000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"5.432769"},"pg_stats":[{"pgid":"23.3f","version":"0'0","reported_seq":9,"reported_epoch":185,"state":"active+clean","last_fresh":"2023-08-09T14:52:30.761556+0000","last_change":"2023-08-09T14:52:30.753912+0000","last_active":"2023-08-09T14:52:30.761556+0000","last_peered":"2023-08-09T14:52:30.761556+0000","last_clean":"2023-08-09T14:52:30.761556+0000","last_became_active":"2023-08-09T14:52:30.753644+0000","last_became_peered":"2023-08-09T14:52:30.753644+0000","last_unstale":"2023-08-09T14:52:30.761556+0000","last_undegraded":"2023-08-09T14:52:30.761556+0000","last_fullsized":"2023-08-09T14:52:30.761556+0000","mapping_epoch":184,"log_start":"0'0","ondisk_log_start":"0'0","created":184,"last_epoch_clean":185,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T14:52:29.665328+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T14:52:29.665328+0000","last_clean_scrub_stamp":"2023-08-09T14:52:29.665328+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T23:36:56.791787+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"23.3e","version":"0'0","reported_seq":9,"reported_epoch":185,"state":"active+clean","last_fresh":"2023-08-09T14:52:30.770118+0000","last_change":"2023-08-09T14:52:30.763355+0000","last_active":"2023-08-09T14:52:30.770118+0000","last_peered":"2023-08-09T14:52:30.770118+0000","last_clean":"2023-08-09T14:52:30.770118+0000","last_became_active":"2023-08-09T14:52:30.762723+0000","last_became_peered":"2023-08-09T14:52:30.762723+0000","last_unstale":"2023-08-09T14:52:30.770118+0000","last_undegraded":"2023-08-09T14:52:30.770118+0000","last_fullsized":"2023-08-09T14:52:30.770118+0000","mapping_epoch":184,"log_start":"0'0","ondisk_log_start":"0'0","created":184,"last_epoch_clean":185,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T14:52:29.665328+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T14:52:29.665328+0000","last_clean_scrub_stamp":"2023-08-09T14:52:29.665328+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-11T01:02:04.697698+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"23.3d","version":"0'0","reported_seq":9,"reported_epoch":185,"state":"active+clean","last_fresh":"2023-08-09T14:52:30.763514+0000","last_change":"2023-08-09T14:52:30.758091+0000","last_active":"2023-08-09T14:52:30.763514+0000","last_peered":"2023-08-09T14:52:30.763514+0000","last_clean":"2023-08-09T14:52:30.763514+0000","last_became_active":"2023-08-09T14:52:30.757610+0000","last_became_peered":"2023-08-09T14:52:30.757610+0000","last_unstale":"2023-08-09T14:52:30.763514+0000","last_undegraded":"2023-08-09T14:52:30.763514+0000","last_fullsized":"2023-08-09T14:52:30.763514+0000","mapping_epoch":184,"log_start":"0'0","ondisk_log_start":"0'0","created":184,"last_epoch_clean":185,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T14:52:29.665328+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T14:52:29.665328+0000","last_clean_scrub_stamp":"2023-08-09T14:52:29.665328+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T23:49:55.143349+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"23.3c","version":"0'0","reported_seq":9,"reported_epoch":185,"state":"active+clean","last_fresh":"2023-08-09T14:52:30.766489+0000","last_change":"2023-08-09T14:52:30.760459+0000","last_active":"2023-08-09T14:52:30.766489+0000","last_peered":"2023-08-09T14:52:30.766489+0000","last_clean":"2023-08-09T14:52:30.766489+0000","last_became_active":"2023-08-09T14:52:30.760019+0000","last_became_peered":"2023-08-09T14:52:30.760019+0000","last_unstale":"2023-08-09T14:52:30.766489+0000","last_undegraded":"2023-08-09T14:52:30.766489+0000","last_fullsized":"2023-08-09T14:52:30.766489+0000","mapping_epoch":184,"log_start":"0'0","ondisk_log_start":"0'0","created":184,"last_epoch_clean":185,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T14:52:29.665328+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T14:52:29.665328+0000","last_clean_scrub_stamp":"2023-08-09T14:52:29.665328+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T17:50:23.747187+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"23.3b","version":"0'0","reported_seq":9,"reported_epoch":185,"state":"active+clean","last_fresh":"2023-08-09T14:52:30.769487+0000","last_change":"2023-08-09T14:52:30.762742+0000","last_active":"2023-08-09T14:52:30.769487+0000","last_peered":"2023-08-09T14:52:30.769487+0000","last_clean":"2023-08-09T14:52:30.769487+0000","last_became_active":"2023-08-09T14:52:30.762436+0000","last_became_peered":"2023-08-09T14:52:30.762436+0000","last_unstale":"2023-08-09T14:52:30.769487+0000","last_undegraded":"2023-08-09T14:52:30.769487+0000","last_fullsized":"2023-08-09T14:52:30.769487+0000","mapping_epoch":184,"log_start":"0'0","ondisk_log_start":"0'0","created":184,"last_epoch_clean":185,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T14:52:29.665328+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T14:52:29.665328+0000","last_clean_scrub_stamp":"2023-08-09T14:52:29.665328+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T17:44:11.621343+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"23.3a","version":"0'0","reported_seq":9,"reported_epoch":185,"state":"active+clean","last_fresh":"2023-08-09T14:52:30.761488+0000","last_change":"2023-08-09T14:52:30.753787+0000","last_active":"2023-08-09T14:52:30.761488+0000","last_peered":"2023-08-09T14:52:30.761488+0000","last_clean":"2023-08-09T14:52:30.761488+0000","last_became_active":"2023-08-09T14:52:30.753510+0000","last_became_peered":"2023-08-09T14:52:30.753510+0000","last_unstale":"2023-08-09T14:52:30.761488+0000","last_undegraded":"2023-08-09T14:52:30.761488+0000","last_fullsized":"2023-08-09T14:52:30.761488+0000","mapping_epoch":184,"log_start":"0'0","ondisk_log_start":"0'0","created":184,"last_epoch_clean":185,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T14:52:29.665328+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T14:52:29.665328+0000","last_clean_scrub_stamp":"2023-08-09T14:52:29.665328+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-11T00:26:08.275980+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"23.39","version":"0'0","reported_seq":9,"reported_epoch":185,"state":"active+clean","last_fresh":"2023-08-09T14:52:30.748065+0000","last_change":"2023-08-09T14:52:30.742190+0000","last_active":"2023-08-09T14:52:30.748065+0000","last_peered":"2023-08-09T14:52:30.748065+0000","last_clean":"2023-08-09T14:52:30.748065+0000","last_became_active":"2023-08-09T14:52:30.741907+0000","last_became_peered":"2023-08-09T14:52:30.741907+0000","last_unstale":"2023-08-09T14:52:30.748065+0000","last_undegraded":"2023-08-09T14:52:30.748065+0000","last_fullsized":"2023-08-09T14:52:30.748065+0000","mapping_epoch":184,"log_start":"0'0","ondisk_log_start":"0'0","created":184,"last_epoch_clean":185,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T14:52:29.665328+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T14:52:29.665328+0000","last_clean_scrub_stamp":"2023-08-09T14:52:29.665328+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T21:49:39.468201+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"23.38","version":"0'0","reported_seq":9,"reported_epoch":185,"state":"active+clean","last_fresh":"2023-08-09T14:52:30.762473+0000","last_change":"2023-08-09T14:52:30.753012+0000","last_active":"2023-08-09T14:52:30.762473+0000","last_peered":"2023-08-09T14:52:30.762473+0000","last_clean":"2023-08-09T14:52:30.762473+0000","last_became_active":"2023-08-09T14:52:30.752563+0000","last_became_peered":"2023-08-09T14:52:30.752563+0000","last_unstale":"2023-08-09T14:52:30.762473+0000","last_undegraded":"2023-08-09T14:52:30.762473+0000","last_fullsized":"2023-08-09T14:52:30.762473+0000","mapping_epoch":184,"log_start":"0'0","ondisk_log_start":"0'0","created":184,"last_epoch_clean":185,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T14:52:29.665328+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T14:52:29.665328+0000","last_clean_scrub_stamp":"2023-08-09T14:52:29.665328+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T18:14:27.098381+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"23.37","version":"0'0","reported_seq":9,"reported_epoch":185,"state":"active+clean","last_fresh":"2023-08-09T14:52:30.769904+0000","last_change":"2023-08-09T14:52:30.764106+0000","last_active":"2023-08-09T14:52:30.769904+0000","last_peered":"2023-08-09T14:52:30.769904+0000","last_clean":"2023-08-09T14:52:30.769904+0000","last_became_active":"2023-08-09T14:52:30.763814+0000","last_became_peered":"2023-08-09T14:52:30.763814+0000","last_unstale":"2023-08-09T14:52:30.769904+0000","last_undegraded":"2023-08-09T14:52:30.769904+0000","last_fullsized":"2023-08-09T14:52:30.769904+0000","mapping_epoch":184,"log_start":"0'0","ondisk_log_start":"0'0","created":184,"last_epoch_clean":185,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T14:52:29.665328+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T14:52:29.665328+0000","last_clean_scrub_stamp":"2023-08-09T14:52:29.665328+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T19:52:04.468524+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"23.36","version":"0'0","reported_seq":9,"reported_epoch":185,"state":"active+clean","last_fresh":"2023-08-09T14:52:30.770142+0000","last_change":"2023-08-09T14:52:30.763562+0000","last_active":"2023-08-09T14:52:30.770142+0000","last_peered":"2023-08-09T14:52:30.770142+0000","last_clean":"2023-08-09T14:52:30.770142+0000","last_became_active":"2023-08-09T14:52:30.763093+0000","last_became_peered":"2023-08-09T14:52:30.763093+0000","last_unstale":"2023-08-09T14:52:30.770142+0000","last_undegraded":"2023-08-09T14:52:30.770142+0000","last_fullsized":"2023-08-09T14:52:30.770142+0000","mapping_epoch":184,"log_start":"0'0","ondisk_log_start":"0'0","created":184,"last_epoch_clean":185,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T14:52:29.665328+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T14:52:29.665328+0000","last_clean_scrub_stamp":"2023-08-09T14:52:29.665328+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T16:35:09.679756+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"23.35","version":"0'0","reported_seq":9,"reported_epoch":185,"state":"active+clean","last_fresh":"2023-08-09T14:52:30.769321+0000","last_change":"2023-08-09T14:52:30.762674+0000","last_active":"2023-08-09T14:52:30.769321+0000","last_peered":"2023-08-09T14:52:30.769321+0000","last_clean":"2023-08-09T14:52:30.769321+0000","last_became_active":"2023-08-09T14:52:30.762378+0000","last_became_peered":"2023-08-09T14:52:30.762378+0000","last_unstale":"2023-08-09T14:52:30.769321+0000","last_undegraded":"2023-08-09T14:52:30.769321+0000","last_fullsized":"2023-08-09T14:52:30.769321+0000","mapping_epoch":184,"log_start":"0'0","ondisk_log_start":"0'0","created":184,"last_epoch_clean":185,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T14:52:29.665328+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T14:52:29.665328+0000","last_clean_scrub_stamp":"2023-08-09T14:52:29.665328+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-11T02:51:20.583446+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"23.34","version":"0'0","reported_seq":9,"reported_epoch":185,"state":"active+clean","last_fresh":"2023-08-09T14:52:30.769647+0000","last_change":"2023-08-09T14:52:30.763378+0000","last_active":"2023-08-09T14:52:30.769647+0000","last_peered":"2023-08-09T14:52:30.769647+0000","last_clean":"2023-08-09T14:52:30.769647+0000","last_became_active":"2023-08-09T14:52:30.763084+0000","last_became_peered":"2023-08-09T14:52:30.763084+0000","last_unstale":"2023-08-09T14:52:30.769647+0000","last_undegraded":"2023-08-09T14:52:30.769647+0000","last_fullsized":"2023-08-09T14:52:30.769647+0000","mapping_epoch":184,"log_start":"0'0","ondisk_log_start":"0'0","created":184,"last_epoch_clean":185,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T14:52:29.665328+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T14:52:29.665328+0000","last_clean_scrub_stamp":"2023-08-09T14:52:29.665328+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T20:56:23.203078+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"23.33","version":"0'0","reported_seq":9,"reported_epoch":185,"state":"active+clean","last_fresh":"2023-08-09T14:52:30.747579+0000","last_change":"2023-08-09T14:52:30.743697+0000","last_active":"2023-08-09T14:52:30.747579+0000","last_peered":"2023-08-09T14:52:30.747579+0000","last_clean":"2023-08-09T14:52:30.747579+0000","last_became_active":"2023-08-09T14:52:30.743303+0000","last_became_peered":"2023-08-09T14:52:30.743303+0000","last_unstale":"2023-08-09T14:52:30.747579+0000","last_undegraded":"2023-08-09T14:52:30.747579+0000","last_fullsized":"2023-08-09T14:52:30.747579+0000","mapping_epoch":184,"log_start":"0'0","ondisk_log_start":"0'0","created":184,"last_epoch_clean":185,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T14:52:29.665328+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T14:52:29.665328+0000","last_clean_scrub_stamp":"2023-08-09T14:52:29.665328+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T18:13:44.615831+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"23.32","version":"0'0","reported_seq":9,"reported_epoch":185,"state":"active+clean","last_fresh":"2023-08-09T14:52:30.761338+0000","last_change":"2023-08-09T14:52:30.757194+0000","last_active":"2023-08-09T14:52:30.761338+0000","last_peered":"2023-08-09T14:52:30.761338+0000","last_clean":"2023-08-09T14:52:30.761338+0000","last_became_active":"2023-08-09T14:52:30.756885+0000","last_became_peered":"2023-08-09T14:52:30.756885+0000","last_unstale":"2023-08-09T14:52:30.761338+0000","last_undegraded":"2023-08-09T14:52:30.761338+0000","last_fullsized":"2023-08-09T14:52:30.761338+0000","mapping_epoch":184,"log_start":"0'0","ondisk_log_start":"0'0","created":184,"last_epoch_clean":185,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T14:52:29.665328+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T14:52:29.665328+0000","last_clean_scrub_stamp":"2023-08-09T14:52:29.665328+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-11T02:14:40.604088+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"23.31","version":"0'0","reported_seq":9,"reported_epoch":185,"state":"active+clean","last_fresh":"2023-08-09T14:52:30.754358+0000","last_change":"2023-08-09T14:52:30.744546+0000","last_active":"2023-08-09T14:52:30.754358+0000","last_peered":"2023-08-09T14:52:30.754358+0000","last_clean":"2023-08-09T14:52:30.754358+0000","last_became_active":"2023-08-09T14:52:30.744270+0000","last_became_peered":"2023-08-09T14:52:30.744270+0000","last_unstale":"2023-08-09T14:52:30.754358+0000","last_undegraded":"2023-08-09T14:52:30.754358+0000","last_fullsized":"2023-08-09T14:52:30.754358+0000","mapping_epoch":184,"log_start":"0'0","ondisk_log_start":"0'0","created":184,"last_epoch_clean":185,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T14:52:29.665328+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T14:52:29.665328+0000","last_clean_scrub_stamp":"2023-08-09T14:52:29.665328+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T16:48:24.213005+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"23.30","version":"0'0","reported_seq":9,"reported_epoch":185,"state":"active+clean","last_fresh":"2023-08-09T14:52:30.757165+0000","last_change":"2023-08-09T14:52:30.750803+0000","last_active":"2023-08-09T14:52:30.757165+0000","last_peered":"2023-08-09T14:52:30.757165+0000","last_clean":"2023-08-09T14:52:30.757165+0000","last_became_active":"2023-08-09T14:52:30.750326+0000","last_became_peered":"2023-08-09T14:52:30.750326+0000","last_unstale":"2023-08-09T14:52:30.757165+0000","last_undegraded":"2023-08-09T14:52:30.757165+0000","last_fullsized":"2023-08-09T14:52:30.757165+0000","mapping_epoch":184,"log_start":"0'0","ondisk_log_start":"0'0","created":184,"last_epoch_clean":185,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T14:52:29.665328+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T14:52:29.665328+0000","last_clean_scrub_stamp":"2023-08-09T14:52:29.665328+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T16:04:49.538552+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"23.2f","version":"0'0","reported_seq":9,"reported_epoch":185,"state":"active+clean","last_fresh":"2023-08-09T14:52:30.761542+0000","last_change":"2023-08-09T14:52:30.752136+0000","last_active":"2023-08-09T14:52:30.761542+0000","last_peered":"2023-08-09T14:52:30.761542+0000","last_clean":"2023-08-09T14:52:30.761542+0000","last_became_active":"2023-08-09T14:52:30.751800+0000","last_became_peered":"2023-08-09T14:52:30.751800+0000","last_unstale":"2023-08-09T14:52:30.761542+0000","last_undegraded":"2023-08-09T14:52:30.761542+0000","last_fullsized":"2023-08-09T14:52:30.761542+0000","mapping_epoch":184,"log_start":"0'0","ondisk_log_start":"0'0","created":184,"last_epoch_clean":185,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T14:52:29.665328+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T14:52:29.665328+0000","last_clean_scrub_stamp":"2023-08-09T14:52:29.665328+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T19:33:48.201185+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"23.2e","version":"0'0","reported_seq":9,"reported_epoch":185,"state":"active+clean","last_fresh":"2023-08-09T14:52:30.749047+0000","last_change":"2023-08-09T14:52:30.746878+0000","last_active":"2023-08-09T14:52:30.749047+0000","last_peered":"2023-08-09T14:52:30.749047+0000","last_clean":"2023-08-09T14:52:30.749047+0000","last_became_active":"2023-08-09T14:52:30.746389+0000","last_became_peered":"2023-08-09T14:52:30.746389+0000","last_unstale":"2023-08-09T14:52:30.749047+0000","last_undegraded":"2023-08-09T14:52:30.749047+0000","last_fullsized":"2023-08-09T14:52:30.749047+0000","mapping_epoch":184,"log_start":"0'0","ondisk_log_start":"0'0","created":184,"last_epoch_clean":185,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T14:52:29.665328+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T14:52:29.665328+0000","last_clean_scrub_stamp":"2023-08-09T14:52:29.665328+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-11T01:08:47.452806+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"23.2d","version":"0'0","reported_seq":9,"reported_epoch":185,"state":"active+clean","last_fresh":"2023-08-09T14:52:30.746114+0000","last_change":"2023-08-09T14:52:30.744109+0000","last_active":"2023-08-09T14:52:30.746114+0000","last_peered":"2023-08-09T14:52:30.746114+0000","last_clean":"2023-08-09T14:52:30.746114+0000","last_became_active":"2023-08-09T14:52:30.743824+0000","last_became_peered":"2023-08-09T14:52:30.743824+0000","last_unstale":"2023-08-09T14:52:30.746114+0000","last_undegraded":"2023-08-09T14:52:30.746114+0000","last_fullsized":"2023-08-09T14:52:30.746114+0000","mapping_epoch":184,"log_start":"0'0","ondisk_log_start":"0'0","created":184,"last_epoch_clean":185,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T14:52:29.665328+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T14:52:29.665328+0000","last_clean_scrub_stamp":"2023-08-09T14:52:29.665328+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T17:47:28.862639+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"23.2c","version":"0'0","reported_seq":9,"reported_epoch":185,"state":"active+clean","last_fresh":"2023-08-09T14:52:30.761365+0000","last_change":"2023-08-09T14:52:30.757201+0000","last_active":"2023-08-09T14:52:30.761365+0000","last_peered":"2023-08-09T14:52:30.761365+0000","last_clean":"2023-08-09T14:52:30.761365+0000","last_became_active":"2023-08-09T14:52:30.756843+0000","last_became_peered":"2023-08-09T14:52:30.756843+0000","last_unstale":"2023-08-09T14:52:30.761365+0000","last_undegraded":"2023-08-09T14:52:30.761365+0000","last_fullsized":"2023-08-09T14:52:30.761365+0000","mapping_epoch":184,"log_start":"0'0","ondisk_log_start":"0'0","created":184,"last_epoch_clean":185,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T14:52:29.665328+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T14:52:29.665328+0000","last_clean_scrub_stamp":"2023-08-09T14:52:29.665328+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T22:17:13.182406+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"23.2b","version":"0'0","reported_seq":9,"reported_epoch":185,"state":"active+clean","last_fresh":"2023-08-09T14:52:30.760468+0000","last_change":"2023-08-09T14:52:30.749933+0000","last_active":"2023-08-09T14:52:30.760468+0000","last_peered":"2023-08-09T14:52:30.760468+0000","last_clean":"2023-08-09T14:52:30.760468+0000","last_became_active":"2023-08-09T14:52:30.749664+0000","last_became_peered":"2023-08-09T14:52:30.749664+0000","last_unstale":"2023-08-09T14:52:30.760468+0000","last_undegraded":"2023-08-09T14:52:30.760468+0000","last_fullsized":"2023-08-09T14:52:30.760468+0000","mapping_epoch":184,"log_start":"0'0","ondisk_log_start":"0'0","created":184,"last_epoch_clean":185,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T14:52:29.665328+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T14:52:29.665328+0000","last_clean_scrub_stamp":"2023-08-09T14:52:29.665328+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T20:26:05.313360+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"23.2a","version":"0'0","reported_seq":9,"reported_epoch":185,"state":"active+clean","last_fresh":"2023-08-09T14:52:30.769863+0000","last_change":"2023-08-09T14:52:30.767768+0000","last_active":"2023-08-09T14:52:30.769863+0000","last_peered":"2023-08-09T14:52:30.769863+0000","last_clean":"2023-08-09T14:52:30.769863+0000","last_became_active":"2023-08-09T14:52:30.766333+0000","last_became_peered":"2023-08-09T14:52:30.766333+0000","last_unstale":"2023-08-09T14:52:30.769863+0000","last_undegraded":"2023-08-09T14:52:30.769863+0000","last_fullsized":"2023-08-09T14:52:30.769863+0000","mapping_epoch":184,"log_start":"0'0","ondisk_log_start":"0'0","created":184,"last_epoch_clean":185,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T14:52:29.665328+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T14:52:29.665328+0000","last_clean_scrub_stamp":"2023-08-09T14:52:29.665328+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-11T02:16:55.127639+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"23.29","version":"0'0","reported_seq":9,"reported_epoch":185,"state":"active+clean","last_fresh":"2023-08-09T14:52:30.765442+0000","last_change":"2023-08-09T14:52:30.763747+0000","last_active":"2023-08-09T14:52:30.765442+0000","last_peered":"2023-08-09T14:52:30.765442+0000","last_clean":"2023-08-09T14:52:30.765442+0000","last_became_active":"2023-08-09T14:52:30.763460+0000","last_became_peered":"2023-08-09T14:52:30.763460+0000","last_unstale":"2023-08-09T14:52:30.765442+0000","last_undegraded":"2023-08-09T14:52:30.765442+0000","last_fullsized":"2023-08-09T14:52:30.765442+0000","mapping_epoch":184,"log_start":"0'0","ondisk_log_start":"0'0","created":184,"last_epoch_clean":185,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T14:52:29.665328+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T14:52:29.665328+0000","last_clean_scrub_stamp":"2023-08-09T14:52:29.665328+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T20:00:10.628893+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"23.28","version":"0'0","reported_seq":9,"reported_epoch":185,"state":"active+clean","last_fresh":"2023-08-09T14:52:30.762451+0000","last_change":"2023-08-09T14:52:30.752836+0000","last_active":"2023-08-09T14:52:30.762451+0000","last_peered":"2023-08-09T14:52:30.762451+0000","last_clean":"2023-08-09T14:52:30.762451+0000","last_became_active":"2023-08-09T14:52:30.752181+0000","last_became_peered":"2023-08-09T14:52:30.752181+0000","last_unstale":"2023-08-09T14:52:30.762451+0000","last_undegraded":"2023-08-09T14:52:30.762451+0000","last_fullsized":"2023-08-09T14:52:30.762451+0000","mapping_epoch":184,"log_start":"0'0","ondisk_log_start":"0'0","created":184,"last_epoch_clean":185,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T14:52:29.665328+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T14:52:29.665328+0000","last_clean_scrub_stamp":"2023-08-09T14:52:29.665328+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-11T00:54:31.624155+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"23.27","version":"0'0","reported_seq":9,"reported_epoch":185,"state":"active+clean","last_fresh":"2023-08-09T14:52:30.761422+0000","last_change":"2023-08-09T14:52:30.756217+0000","last_active":"2023-08-09T14:52:30.761422+0000","last_peered":"2023-08-09T14:52:30.761422+0000","last_clean":"2023-08-09T14:52:30.761422+0000","last_became_active":"2023-08-09T14:52:30.755921+0000","last_became_peered":"2023-08-09T14:52:30.755921+0000","last_unstale":"2023-08-09T14:52:30.761422+0000","last_undegraded":"2023-08-09T14:52:30.761422+0000","last_fullsized":"2023-08-09T14:52:30.761422+0000","mapping_epoch":184,"log_start":"0'0","ondisk_log_start":"0'0","created":184,"last_epoch_clean":185,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T14:52:29.665328+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T14:52:29.665328+0000","last_clean_scrub_stamp":"2023-08-09T14:52:29.665328+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-11T00:21:08.700639+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"23.26","version":"0'0","reported_seq":9,"reported_epoch":185,"state":"active+clean","last_fresh":"2023-08-09T14:52:30.749091+0000","last_change":"2023-08-09T14:52:30.747505+0000","last_active":"2023-08-09T14:52:30.749091+0000","last_peered":"2023-08-09T14:52:30.749091+0000","last_clean":"2023-08-09T14:52:30.749091+0000","last_became_active":"2023-08-09T14:52:30.747196+0000","last_became_peered":"2023-08-09T14:52:30.747196+0000","last_unstale":"2023-08-09T14:52:30.749091+0000","last_undegraded":"2023-08-09T14:52:30.749091+0000","last_fullsized":"2023-08-09T14:52:30.749091+0000","mapping_epoch":184,"log_start":"0'0","ondisk_log_start":"0'0","created":184,"last_epoch_clean":185,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T14:52:29.665328+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T14:52:29.665328+0000","last_clean_scrub_stamp":"2023-08-09T14:52:29.665328+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-11T01:19:24.212841+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"23.25","version":"0'0","reported_seq":9,"reported_epoch":185,"state":"active+clean","last_fresh":"2023-08-09T14:52:30.770421+0000","last_change":"2023-08-09T14:52:30.767487+0000","last_active":"2023-08-09T14:52:30.770421+0000","last_peered":"2023-08-09T14:52:30.770421+0000","last_clean":"2023-08-09T14:52:30.770421+0000","last_became_active":"2023-08-09T14:52:30.767201+0000","last_became_peered":"2023-08-09T14:52:30.767201+0000","last_unstale":"2023-08-09T14:52:30.770421+0000","last_undegraded":"2023-08-09T14:52:30.770421+0000","last_fullsized":"2023-08-09T14:52:30.770421+0000","mapping_epoch":184,"log_start":"0'0","ondisk_log_start":"0'0","created":184,"last_epoch_clean":185,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T14:52:29.665328+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T14:52:29.665328+0000","last_clean_scrub_stamp":"2023-08-09T14:52:29.665328+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-11T00:42:15.257036+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"23.24","version":"0'0","reported_seq":9,"reported_epoch":185,"state":"active+clean","last_fresh":"2023-08-09T14:52:30.754309+0000","last_change":"2023-08-09T14:52:30.744360+0000","last_active":"2023-08-09T14:52:30.754309+0000","last_peered":"2023-08-09T14:52:30.754309+0000","last_clean":"2023-08-09T14:52:30.754309+0000","last_became_active":"2023-08-09T14:52:30.744052+0000","last_became_peered":"2023-08-09T14:52:30.744052+0000","last_unstale":"2023-08-09T14:52:30.754309+0000","last_undegraded":"2023-08-09T14:52:30.754309+0000","last_fullsized":"2023-08-09T14:52:30.754309+0000","mapping_epoch":184,"log_start":"0'0","ondisk_log_start":"0'0","created":184,"last_epoch_clean":185,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T14:52:29.665328+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T14:52:29.665328+0000","last_clean_scrub_stamp":"2023-08-09T14:52:29.665328+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T21:56:25.594544+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"23.23","version":"0'0","reported_seq":9,"reported_epoch":185,"state":"active+clean","last_fresh":"2023-08-09T14:52:30.770461+0000","last_change":"2023-08-09T14:52:30.768224+0000","last_active":"2023-08-09T14:52:30.770461+0000","last_peered":"2023-08-09T14:52:30.770461+0000","last_clean":"2023-08-09T14:52:30.770461+0000","last_became_active":"2023-08-09T14:52:30.767828+0000","last_became_peered":"2023-08-09T14:52:30.767828+0000","last_unstale":"2023-08-09T14:52:30.770461+0000","last_undegraded":"2023-08-09T14:52:30.770461+0000","last_fullsized":"2023-08-09T14:52:30.770461+0000","mapping_epoch":184,"log_start":"0'0","ondisk_log_start":"0'0","created":184,"last_epoch_clean":185,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T14:52:29.665328+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T14:52:29.665328+0000","last_clean_scrub_stamp":"2023-08-09T14:52:29.665328+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T18:38:13.297526+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"23.22","version":"0'0","reported_seq":9,"reported_epoch":185,"state":"active+clean","last_fresh":"2023-08-09T14:52:30.759868+0000","last_change":"2023-08-09T14:52:30.756106+0000","last_active":"2023-08-09T14:52:30.759868+0000","last_peered":"2023-08-09T14:52:30.759868+0000","last_clean":"2023-08-09T14:52:30.759868+0000","last_became_active":"2023-08-09T14:52:30.755144+0000","last_became_peered":"2023-08-09T14:52:30.755144+0000","last_unstale":"2023-08-09T14:52:30.759868+0000","last_undegraded":"2023-08-09T14:52:30.759868+0000","last_fullsized":"2023-08-09T14:52:30.759868+0000","mapping_epoch":184,"log_start":"0'0","ondisk_log_start":"0'0","created":184,"last_epoch_clean":185,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T14:52:29.665328+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T14:52:29.665328+0000","last_clean_scrub_stamp":"2023-08-09T14:52:29.665328+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T19:16:10.586060+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"23.21","version":"0'0","reported_seq":9,"reported_epoch":185,"state":"active+clean","last_fresh":"2023-08-09T14:52:30.759977+0000","last_change":"2023-08-09T14:52:30.756197+0000","last_active":"2023-08-09T14:52:30.759977+0000","last_peered":"2023-08-09T14:52:30.759977+0000","last_clean":"2023-08-09T14:52:30.759977+0000","last_became_active":"2023-08-09T14:52:30.755871+0000","last_became_peered":"2023-08-09T14:52:30.755871+0000","last_unstale":"2023-08-09T14:52:30.759977+0000","last_undegraded":"2023-08-09T14:52:30.759977+0000","last_fullsized":"2023-08-09T14:52:30.759977+0000","mapping_epoch":184,"log_start":"0'0","ondisk_log_start":"0'0","created":184,"last_epoch_clean":185,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T14:52:29.665328+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T14:52:29.665328+0000","last_clean_scrub_stamp":"2023-08-09T14:52:29.665328+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-11T01:31:05.174962+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"23.20","version":"0'0","reported_seq":9,"reported_epoch":185,"state":"active+clean","last_fresh":"2023-08-09T14:52:30.745972+0000","last_change":"2023-08-09T14:52:30.743192+0000","last_active":"2023-08-09T14:52:30.745972+0000","last_peered":"2023-08-09T14:52:30.745972+0000","last_clean":"2023-08-09T14:52:30.745972+0000","last_became_active":"2023-08-09T14:52:30.742836+0000","last_became_peered":"2023-08-09T14:52:30.742836+0000","last_unstale":"2023-08-09T14:52:30.745972+0000","last_undegraded":"2023-08-09T14:52:30.745972+0000","last_fullsized":"2023-08-09T14:52:30.745972+0000","mapping_epoch":184,"log_start":"0'0","ondisk_log_start":"0'0","created":184,"last_epoch_clean":185,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T14:52:29.665328+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T14:52:29.665328+0000","last_clean_scrub_stamp":"2023-08-09T14:52:29.665328+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T22:46:41.082755+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"23.1e","version":"0'0","reported_seq":9,"reported_epoch":185,"state":"active+clean","last_fresh":"2023-08-09T14:52:30.763699+0000","last_change":"2023-08-09T14:52:30.760085+0000","last_active":"2023-08-09T14:52:30.763699+0000","last_peered":"2023-08-09T14:52:30.763699+0000","last_clean":"2023-08-09T14:52:30.763699+0000","last_became_active":"2023-08-09T14:52:30.759756+0000","last_became_peered":"2023-08-09T14:52:30.759756+0000","last_unstale":"2023-08-09T14:52:30.763699+0000","last_undegraded":"2023-08-09T14:52:30.763699+0000","last_fullsized":"2023-08-09T14:52:30.763699+0000","mapping_epoch":184,"log_start":"0'0","ondisk_log_start":"0'0","created":184,"last_epoch_clean":185,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T14:52:29.665328+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T14:52:29.665328+0000","last_clean_scrub_stamp":"2023-08-09T14:52:29.665328+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T15:46:43.388518+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"22.1f","version":"185'4","reported_seq":15,"reported_epoch":185,"state":"active+clean","last_fresh":"2023-08-09T14:52:30.774910+0000","last_change":"2023-08-09T14:52:29.715240+0000","last_active":"2023-08-09T14:52:30.774910+0000","last_peered":"2023-08-09T14:52:30.774910+0000","last_clean":"2023-08-09T14:52:30.774910+0000","last_became_active":"2023-08-09T14:52:29.714931+0000","last_became_peered":"2023-08-09T14:52:29.714931+0000","last_unstale":"2023-08-09T14:52:30.774910+0000","last_undegraded":"2023-08-09T14:52:30.774910+0000","last_fullsized":"2023-08-09T14:52:30.774910+0000","mapping_epoch":183,"log_start":"0'0","ondisk_log_start":"0'0","created":183,"last_epoch_clean":184,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T14:52:28.654282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T14:52:28.654282+0000","last_clean_scrub_stamp":"2023-08-09T14:52:28.654282+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T20:41:46.046693+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"23.1f","version":"0'0","reported_seq":9,"reported_epoch":185,"state":"active+clean","last_fresh":"2023-08-09T14:52:30.747456+0000","last_change":"2023-08-09T14:52:30.744316+0000","last_active":"2023-08-09T14:52:30.747456+0000","last_peered":"2023-08-09T14:52:30.747456+0000","last_clean":"2023-08-09T14:52:30.747456+0000","last_became_active":"2023-08-09T14:52:30.743964+0000","last_became_peered":"2023-08-09T14:52:30.743964+0000","last_unstale":"2023-08-09T14:52:30.747456+0000","last_undegraded":"2023-08-09T14:52:30.747456+0000","last_fullsized":"2023-08-09T14:52:30.747456+0000","mapping_epoch":184,"log_start":"0'0","ondisk_log_start":"0'0","created":184,"last_epoch_clean":185,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T14:52:29.665328+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T14:52:29.665328+0000","last_clean_scrub_stamp":"2023-08-09T14:52:29.665328+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-11T01:27:11.688539+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"22.1e","version":"185'1","reported_seq":12,"reported_epoch":185,"state":"active+clean","last_fresh":"2023-08-09T14:52:30.771037+0000","last_change":"2023-08-09T14:52:29.714217+0000","last_active":"2023-08-09T14:52:30.771037+0000","last_peered":"2023-08-09T14:52:30.771037+0000","last_clean":"2023-08-09T14:52:30.771037+0000","last_became_active":"2023-08-09T14:52:29.713881+0000","last_became_peered":"2023-08-09T14:52:29.713881+0000","last_unstale":"2023-08-09T14:52:30.771037+0000","last_undegraded":"2023-08-09T14:52:30.771037+0000","last_fullsized":"2023-08-09T14:52:30.771037+0000","mapping_epoch":183,"log_start":"0'0","ondisk_log_start":"0'0","created":183,"last_epoch_clean":184,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T14:52:28.654282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T14:52:28.654282+0000","last_clean_scrub_stamp":"2023-08-09T14:52:28.654282+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-11T02:08:41.542781+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"23.1c","version":"0'0","reported_seq":9,"reported_epoch":185,"state":"active+clean","last_fresh":"2023-08-09T14:52:30.756814+0000","last_change":"2023-08-09T14:52:30.745487+0000","last_active":"2023-08-09T14:52:30.756814+0000","last_peered":"2023-08-09T14:52:30.756814+0000","last_clean":"2023-08-09T14:52:30.756814+0000","last_became_active":"2023-08-09T14:52:30.744752+0000","last_became_peered":"2023-08-09T14:52:30.744752+0000","last_unstale":"2023-08-09T14:52:30.756814+0000","last_undegraded":"2023-08-09T14:52:30.756814+0000","last_fullsized":"2023-08-09T14:52:30.756814+0000","mapping_epoch":184,"log_start":"0'0","ondisk_log_start":"0'0","created":184,"last_epoch_clean":185,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T14:52:29.665328+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T14:52:29.665328+0000","last_clean_scrub_stamp":"2023-08-09T14:52:29.665328+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T15:49:51.175691+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"22.1d","version":"185'2","reported_seq":13,"reported_epoch":185,"state":"active+clean","last_fresh":"2023-08-09T14:52:30.772025+0000","last_change":"2023-08-09T14:52:29.712716+0000","last_active":"2023-08-09T14:52:30.772025+0000","last_peered":"2023-08-09T14:52:30.772025+0000","last_clean":"2023-08-09T14:52:30.772025+0000","last_became_active":"2023-08-09T14:52:29.711958+0000","last_became_peered":"2023-08-09T14:52:29.711958+0000","last_unstale":"2023-08-09T14:52:30.772025+0000","last_undegraded":"2023-08-09T14:52:30.772025+0000","last_fullsized":"2023-08-09T14:52:30.772025+0000","mapping_epoch":183,"log_start":"0'0","ondisk_log_start":"0'0","created":183,"last_epoch_clean":184,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T14:52:28.654282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T14:52:28.654282+0000","last_clean_scrub_stamp":"2023-08-09T14:52:28.654282+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-11T02:01:06.667578+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"23.1d","version":"0'0","reported_seq":9,"reported_epoch":185,"state":"active+clean","last_fresh":"2023-08-09T14:52:30.763532+0000","last_change":"2023-08-09T14:52:30.758424+0000","last_active":"2023-08-09T14:52:30.763532+0000","last_peered":"2023-08-09T14:52:30.763532+0000","last_clean":"2023-08-09T14:52:30.763532+0000","last_became_active":"2023-08-09T14:52:30.758145+0000","last_became_peered":"2023-08-09T14:52:30.758145+0000","last_unstale":"2023-08-09T14:52:30.763532+0000","last_undegraded":"2023-08-09T14:52:30.763532+0000","last_fullsized":"2023-08-09T14:52:30.763532+0000","mapping_epoch":184,"log_start":"0'0","ondisk_log_start":"0'0","created":184,"last_epoch_clean":185,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T14:52:29.665328+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T14:52:29.665328+0000","last_clean_scrub_stamp":"2023-08-09T14:52:29.665328+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T17:23:02.106000+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"22.1c","version":"185'2","reported_seq":13,"reported_epoch":185,"state":"active+clean","last_fresh":"2023-08-09T14:52:30.774998+0000","last_change":"2023-08-09T14:52:29.716345+0000","last_active":"2023-08-09T14:52:30.774998+0000","last_peered":"2023-08-09T14:52:30.774998+0000","last_clean":"2023-08-09T14:52:30.774998+0000","last_became_active":"2023-08-09T14:52:29.715940+0000","last_became_peered":"2023-08-09T14:52:29.715940+0000","last_unstale":"2023-08-09T14:52:30.774998+0000","last_undegraded":"2023-08-09T14:52:30.774998+0000","last_fullsized":"2023-08-09T14:52:30.774998+0000","mapping_epoch":183,"log_start":"0'0","ondisk_log_start":"0'0","created":183,"last_epoch_clean":184,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T14:52:28.654282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T14:52:28.654282+0000","last_clean_scrub_stamp":"2023-08-09T14:52:28.654282+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T19:50:22.634529+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"23.1a","version":"0'0","reported_seq":9,"reported_epoch":185,"state":"active+clean","last_fresh":"2023-08-09T14:52:30.756758+0000","last_change":"2023-08-09T14:52:30.752369+0000","last_active":"2023-08-09T14:52:30.756758+0000","last_peered":"2023-08-09T14:52:30.756758+0000","last_clean":"2023-08-09T14:52:30.756758+0000","last_became_active":"2023-08-09T14:52:30.747909+0000","last_became_peered":"2023-08-09T14:52:30.747909+0000","last_unstale":"2023-08-09T14:52:30.756758+0000","last_undegraded":"2023-08-09T14:52:30.756758+0000","last_fullsized":"2023-08-09T14:52:30.756758+0000","mapping_epoch":184,"log_start":"0'0","ondisk_log_start":"0'0","created":184,"last_epoch_clean":185,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T14:52:29.665328+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T14:52:29.665328+0000","last_clean_scrub_stamp":"2023-08-09T14:52:29.665328+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T22:59:22.827689+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"22.1b","version":"185'1","reported_seq":12,"reported_epoch":185,"state":"active+clean","last_fresh":"2023-08-09T14:52:30.768049+0000","last_change":"2023-08-09T14:52:29.704885+0000","last_active":"2023-08-09T14:52:30.768049+0000","last_peered":"2023-08-09T14:52:30.768049+0000","last_clean":"2023-08-09T14:52:30.768049+0000","last_became_active":"2023-08-09T14:52:29.704567+0000","last_became_peered":"2023-08-09T14:52:29.704567+0000","last_unstale":"2023-08-09T14:52:30.768049+0000","last_undegraded":"2023-08-09T14:52:30.768049+0000","last_fullsized":"2023-08-09T14:52:30.768049+0000","mapping_epoch":183,"log_start":"0'0","ondisk_log_start":"0'0","created":183,"last_epoch_clean":184,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T14:52:28.654282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T14:52:28.654282+0000","last_clean_scrub_stamp":"2023-08-09T14:52:28.654282+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-11T00:13:48.503140+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"23.1b","version":"0'0","reported_seq":9,"reported_epoch":185,"state":"active+clean","last_fresh":"2023-08-09T14:52:30.766589+0000","last_change":"2023-08-09T14:52:30.760189+0000","last_active":"2023-08-09T14:52:30.766589+0000","last_peered":"2023-08-09T14:52:30.766589+0000","last_clean":"2023-08-09T14:52:30.766589+0000","last_became_active":"2023-08-09T14:52:30.759907+0000","last_became_peered":"2023-08-09T14:52:30.759907+0000","last_unstale":"2023-08-09T14:52:30.766589+0000","last_undegraded":"2023-08-09T14:52:30.766589+0000","last_fullsized":"2023-08-09T14:52:30.766589+0000","mapping_epoch":184,"log_start":"0'0","ondisk_log_start":"0'0","created":184,"last_epoch_clean":185,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T14:52:29.665328+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T14:52:29.665328+0000","last_clean_scrub_stamp":"2023-08-09T14:52:29.665328+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T23:48:29.448541+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"22.1a","version":"185'1","reported_seq":12,"reported_epoch":185,"state":"active+clean","last_fresh":"2023-08-09T14:52:30.771819+0000","last_change":"2023-08-09T14:52:29.698683+0000","last_active":"2023-08-09T14:52:30.771819+0000","last_peered":"2023-08-09T14:52:30.771819+0000","last_clean":"2023-08-09T14:52:30.771819+0000","last_became_active":"2023-08-09T14:52:29.698300+0000","last_became_peered":"2023-08-09T14:52:29.698300+0000","last_unstale":"2023-08-09T14:52:30.771819+0000","last_undegraded":"2023-08-09T14:52:30.771819+0000","last_fullsized":"2023-08-09T14:52:30.771819+0000","mapping_epoch":183,"log_start":"0'0","ondisk_log_start":"0'0","created":183,"last_epoch_clean":184,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T14:52:28.654282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T14:52:28.654282+0000","last_clean_scrub_stamp":"2023-08-09T14:52:28.654282+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T16:33:30.903318+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"23.18","version":"0'0","reported_seq":9,"reported_epoch":185,"state":"active+clean","last_fresh":"2023-08-09T14:52:30.765180+0000","last_change":"2023-08-09T14:52:30.756817+0000","last_active":"2023-08-09T14:52:30.765180+0000","last_peered":"2023-08-09T14:52:30.765180+0000","last_clean":"2023-08-09T14:52:30.765180+0000","last_became_active":"2023-08-09T14:52:30.756054+0000","last_became_peered":"2023-08-09T14:52:30.756054+0000","last_unstale":"2023-08-09T14:52:30.765180+0000","last_undegraded":"2023-08-09T14:52:30.765180+0000","last_fullsized":"2023-08-09T14:52:30.765180+0000","mapping_epoch":184,"log_start":"0'0","ondisk_log_start":"0'0","created":184,"last_epoch_clean":185,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T14:52:29.665328+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T14:52:29.665328+0000","last_clean_scrub_stamp":"2023-08-09T14:52:29.665328+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T16:48:27.700071+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"22.19","version":"0'0","reported_seq":11,"reported_epoch":185,"state":"active+clean","last_fresh":"2023-08-09T14:52:30.744368+0000","last_change":"2023-08-09T14:52:29.717797+0000","last_active":"2023-08-09T14:52:30.744368+0000","last_peered":"2023-08-09T14:52:30.744368+0000","last_clean":"2023-08-09T14:52:30.744368+0000","last_became_active":"2023-08-09T14:52:29.717365+0000","last_became_peered":"2023-08-09T14:52:29.717365+0000","last_unstale":"2023-08-09T14:52:30.744368+0000","last_undegraded":"2023-08-09T14:52:30.744368+0000","last_fullsized":"2023-08-09T14:52:30.744368+0000","mapping_epoch":183,"log_start":"0'0","ondisk_log_start":"0'0","created":183,"last_epoch_clean":184,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T14:52:28.654282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T14:52:28.654282+0000","last_clean_scrub_stamp":"2023-08-09T14:52:28.654282+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-11T00:38:38.570978+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"23.19","version":"0'0","reported_seq":9,"reported_epoch":185,"state":"active+clean","last_fresh":"2023-08-09T14:52:30.761570+0000","last_change":"2023-08-09T14:52:30.752017+0000","last_active":"2023-08-09T14:52:30.761570+0000","last_peered":"2023-08-09T14:52:30.761570+0000","last_clean":"2023-08-09T14:52:30.761570+0000","last_became_active":"2023-08-09T14:52:30.751744+0000","last_became_peered":"2023-08-09T14:52:30.751744+0000","last_unstale":"2023-08-09T14:52:30.761570+0000","last_undegraded":"2023-08-09T14:52:30.761570+0000","last_fullsized":"2023-08-09T14:52:30.761570+0000","mapping_epoch":184,"log_start":"0'0","ondisk_log_start":"0'0","created":184,"last_epoch_clean":185,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T14:52:29.665328+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T14:52:29.665328+0000","last_clean_scrub_stamp":"2023-08-09T14:52:29.665328+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T21:33:27.199787+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"22.18","version":"0'0","reported_seq":11,"reported_epoch":185,"state":"active+clean","last_fresh":"2023-08-09T14:52:30.721059+0000","last_change":"2023-08-09T14:52:29.705213+0000","last_active":"2023-08-09T14:52:30.721059+0000","last_peered":"2023-08-09T14:52:30.721059+0000","last_clean":"2023-08-09T14:52:30.721059+0000","last_became_active":"2023-08-09T14:52:29.704926+0000","last_became_peered":"2023-08-09T14:52:29.704926+0000","last_unstale":"2023-08-09T14:52:30.721059+0000","last_undegraded":"2023-08-09T14:52:30.721059+0000","last_fullsized":"2023-08-09T14:52:30.721059+0000","mapping_epoch":183,"log_start":"0'0","ondisk_log_start":"0'0","created":183,"last_epoch_clean":184,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T14:52:28.654282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T14:52:28.654282+0000","last_clean_scrub_stamp":"2023-08-09T14:52:28.654282+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T18:47:31.398455+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"23.e","version":"0'0","reported_seq":9,"reported_epoch":185,"state":"active+clean","last_fresh":"2023-08-09T14:52:30.766595+0000","last_change":"2023-08-09T14:52:30.759493+0000","last_active":"2023-08-09T14:52:30.766595+0000","last_peered":"2023-08-09T14:52:30.766595+0000","last_clean":"2023-08-09T14:52:30.766595+0000","last_became_active":"2023-08-09T14:52:30.759170+0000","last_became_peered":"2023-08-09T14:52:30.759170+0000","last_unstale":"2023-08-09T14:52:30.766595+0000","last_undegraded":"2023-08-09T14:52:30.766595+0000","last_fullsized":"2023-08-09T14:52:30.766595+0000","mapping_epoch":184,"log_start":"0'0","ondisk_log_start":"0'0","created":184,"last_epoch_clean":185,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T14:52:29.665328+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T14:52:29.665328+0000","last_clean_scrub_stamp":"2023-08-09T14:52:29.665328+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T16:25:08.863039+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"22.f","version":"185'1","reported_seq":12,"reported_epoch":185,"state":"active+clean","last_fresh":"2023-08-09T14:52:30.769891+0000","last_change":"2023-08-09T14:52:29.711508+0000","last_active":"2023-08-09T14:52:30.769891+0000","last_peered":"2023-08-09T14:52:30.769891+0000","last_clean":"2023-08-09T14:52:30.769891+0000","last_became_active":"2023-08-09T14:52:29.711242+0000","last_became_peered":"2023-08-09T14:52:29.711242+0000","last_unstale":"2023-08-09T14:52:30.769891+0000","last_undegraded":"2023-08-09T14:52:30.769891+0000","last_fullsized":"2023-08-09T14:52:30.769891+0000","mapping_epoch":183,"log_start":"0'0","ondisk_log_start":"0'0","created":183,"last_epoch_clean":184,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T14:52:28.654282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T14:52:28.654282+0000","last_clean_scrub_stamp":"2023-08-09T14:52:28.654282+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-11T02:17:49.407925+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"23.f","version":"0'0","reported_seq":9,"reported_epoch":185,"state":"active+clean","last_fresh":"2023-08-09T14:52:30.757435+0000","last_change":"2023-08-09T14:52:30.742721+0000","last_active":"2023-08-09T14:52:30.757435+0000","last_peered":"2023-08-09T14:52:30.757435+0000","last_clean":"2023-08-09T14:52:30.757435+0000","last_became_active":"2023-08-09T14:52:30.741032+0000","last_became_peered":"2023-08-09T14:52:30.741032+0000","last_unstale":"2023-08-09T14:52:30.757435+0000","last_undegraded":"2023-08-09T14:52:30.757435+0000","last_fullsized":"2023-08-09T14:52:30.757435+0000","mapping_epoch":184,"log_start":"0'0","ondisk_log_start":"0'0","created":184,"last_epoch_clean":185,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T14:52:29.665328+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T14:52:29.665328+0000","last_clean_scrub_stamp":"2023-08-09T14:52:29.665328+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-11T01:15:53.748020+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"22.e","version":"185'2","reported_seq":13,"reported_epoch":185,"state":"active+clean","last_fresh":"2023-08-09T14:52:30.767028+0000","last_change":"2023-08-09T14:52:29.708857+0000","last_active":"2023-08-09T14:52:30.767028+0000","last_peered":"2023-08-09T14:52:30.767028+0000","last_clean":"2023-08-09T14:52:30.767028+0000","last_became_active":"2023-08-09T14:52:29.708531+0000","last_became_peered":"2023-08-09T14:52:29.708531+0000","last_unstale":"2023-08-09T14:52:30.767028+0000","last_undegraded":"2023-08-09T14:52:30.767028+0000","last_fullsized":"2023-08-09T14:52:30.767028+0000","mapping_epoch":183,"log_start":"0'0","ondisk_log_start":"0'0","created":183,"last_epoch_clean":184,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T14:52:28.654282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T14:52:28.654282+0000","last_clean_scrub_stamp":"2023-08-09T14:52:28.654282+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T18:44:32.435595+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"23.c","version":"0'0","reported_seq":9,"reported_epoch":185,"state":"active+clean","last_fresh":"2023-08-09T14:52:30.766515+0000","last_change":"2023-08-09T14:52:30.761119+0000","last_active":"2023-08-09T14:52:30.766515+0000","last_peered":"2023-08-09T14:52:30.766515+0000","last_clean":"2023-08-09T14:52:30.766515+0000","last_became_active":"2023-08-09T14:52:30.760825+0000","last_became_peered":"2023-08-09T14:52:30.760825+0000","last_unstale":"2023-08-09T14:52:30.766515+0000","last_undegraded":"2023-08-09T14:52:30.766515+0000","last_fullsized":"2023-08-09T14:52:30.766515+0000","mapping_epoch":184,"log_start":"0'0","ondisk_log_start":"0'0","created":184,"last_epoch_clean":185,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T14:52:29.665328+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T14:52:29.665328+0000","last_clean_scrub_stamp":"2023-08-09T14:52:29.665328+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-11T01:18:15.509351+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"22.d","version":"185'1","reported_seq":12,"reported_epoch":185,"state":"active+clean","last_fresh":"2023-08-09T14:52:30.773042+0000","last_change":"2023-08-09T14:52:29.708127+0000","last_active":"2023-08-09T14:52:30.773042+0000","last_peered":"2023-08-09T14:52:30.773042+0000","last_clean":"2023-08-09T14:52:30.773042+0000","last_became_active":"2023-08-09T14:52:29.707832+0000","last_became_peered":"2023-08-09T14:52:29.707832+0000","last_unstale":"2023-08-09T14:52:30.773042+0000","last_undegraded":"2023-08-09T14:52:30.773042+0000","last_fullsized":"2023-08-09T14:52:30.773042+0000","mapping_epoch":183,"log_start":"0'0","ondisk_log_start":"0'0","created":183,"last_epoch_clean":184,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T14:52:28.654282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T14:52:28.654282+0000","last_clean_scrub_stamp":"2023-08-09T14:52:28.654282+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T20:44:44.511125+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"23.d","version":"0'0","reported_seq":9,"reported_epoch":185,"state":"active+clean","last_fresh":"2023-08-09T14:52:30.749118+0000","last_change":"2023-08-09T14:52:30.747608+0000","last_active":"2023-08-09T14:52:30.749118+0000","last_peered":"2023-08-09T14:52:30.749118+0000","last_clean":"2023-08-09T14:52:30.749118+0000","last_became_active":"2023-08-09T14:52:30.747289+0000","last_became_peered":"2023-08-09T14:52:30.747289+0000","last_unstale":"2023-08-09T14:52:30.749118+0000","last_undegraded":"2023-08-09T14:52:30.749118+0000","last_fullsized":"2023-08-09T14:52:30.749118+0000","mapping_epoch":184,"log_start":"0'0","ondisk_log_start":"0'0","created":184,"last_epoch_clean":185,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T14:52:29.665328+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T14:52:29.665328+0000","last_clean_scrub_stamp":"2023-08-09T14:52:29.665328+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-11T02:11:53.976567+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"22.c","version":"0'0","reported_seq":11,"reported_epoch":185,"state":"active+clean","last_fresh":"2023-08-09T14:52:30.738608+0000","last_change":"2023-08-09T14:52:29.711505+0000","last_active":"2023-08-09T14:52:30.738608+0000","last_peered":"2023-08-09T14:52:30.738608+0000","last_clean":"2023-08-09T14:52:30.738608+0000","last_became_active":"2023-08-09T14:52:29.711234+0000","last_became_peered":"2023-08-09T14:52:29.711234+0000","last_unstale":"2023-08-09T14:52:30.738608+0000","last_undegraded":"2023-08-09T14:52:30.738608+0000","last_fullsized":"2023-08-09T14:52:30.738608+0000","mapping_epoch":183,"log_start":"0'0","ondisk_log_start":"0'0","created":183,"last_epoch_clean":184,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T14:52:28.654282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T14:52:28.654282+0000","last_clean_scrub_stamp":"2023-08-09T14:52:28.654282+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T20:33:36.515170+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"23.a","version":"0'0","reported_seq":9,"reported_epoch":185,"state":"active+clean","last_fresh":"2023-08-09T14:52:30.755373+0000","last_change":"2023-08-09T14:52:30.753084+0000","last_active":"2023-08-09T14:52:30.755373+0000","last_peered":"2023-08-09T14:52:30.755373+0000","last_clean":"2023-08-09T14:52:30.755373+0000","last_became_active":"2023-08-09T14:52:30.752808+0000","last_became_peered":"2023-08-09T14:52:30.752808+0000","last_unstale":"2023-08-09T14:52:30.755373+0000","last_undegraded":"2023-08-09T14:52:30.755373+0000","last_fullsized":"2023-08-09T14:52:30.755373+0000","mapping_epoch":184,"log_start":"0'0","ondisk_log_start":"0'0","created":184,"last_epoch_clean":185,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T14:52:29.665328+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T14:52:29.665328+0000","last_clean_scrub_stamp":"2023-08-09T14:52:29.665328+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T21:21:12.933823+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"22.b","version":"185'2","reported_seq":13,"reported_epoch":185,"state":"active+clean","last_fresh":"2023-08-09T14:52:30.773187+0000","last_change":"2023-08-09T14:52:29.713164+0000","last_active":"2023-08-09T14:52:30.773187+0000","last_peered":"2023-08-09T14:52:30.773187+0000","last_clean":"2023-08-09T14:52:30.773187+0000","last_became_active":"2023-08-09T14:52:29.712262+0000","last_became_peered":"2023-08-09T14:52:29.712262+0000","last_unstale":"2023-08-09T14:52:30.773187+0000","last_undegraded":"2023-08-09T14:52:30.773187+0000","last_fullsized":"2023-08-09T14:52:30.773187+0000","mapping_epoch":183,"log_start":"0'0","ondisk_log_start":"0'0","created":183,"last_epoch_clean":184,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T14:52:28.654282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T14:52:28.654282+0000","last_clean_scrub_stamp":"2023-08-09T14:52:28.654282+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T17:30:59.228973+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"23.b","version":"0'0","reported_seq":9,"reported_epoch":185,"state":"active+clean","last_fresh":"2023-08-09T14:52:30.756720+0000","last_change":"2023-08-09T14:52:30.749683+0000","last_active":"2023-08-09T14:52:30.756720+0000","last_peered":"2023-08-09T14:52:30.756720+0000","last_clean":"2023-08-09T14:52:30.756720+0000","last_became_active":"2023-08-09T14:52:30.749196+0000","last_became_peered":"2023-08-09T14:52:30.749196+0000","last_unstale":"2023-08-09T14:52:30.756720+0000","last_undegraded":"2023-08-09T14:52:30.756720+0000","last_fullsized":"2023-08-09T14:52:30.756720+0000","mapping_epoch":184,"log_start":"0'0","ondisk_log_start":"0'0","created":184,"last_epoch_clean":185,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T14:52:29.665328+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T14:52:29.665328+0000","last_clean_scrub_stamp":"2023-08-09T14:52:29.665328+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T19:56:10.498391+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"22.a","version":"0'0","reported_seq":11,"reported_epoch":185,"state":"active+clean","last_fresh":"2023-08-09T14:52:30.735818+0000","last_change":"2023-08-09T14:52:29.706557+0000","last_active":"2023-08-09T14:52:30.735818+0000","last_peered":"2023-08-09T14:52:30.735818+0000","last_clean":"2023-08-09T14:52:30.735818+0000","last_became_active":"2023-08-09T14:52:29.706258+0000","last_became_peered":"2023-08-09T14:52:29.706258+0000","last_unstale":"2023-08-09T14:52:30.735818+0000","last_undegraded":"2023-08-09T14:52:30.735818+0000","last_fullsized":"2023-08-09T14:52:30.735818+0000","mapping_epoch":183,"log_start":"0'0","ondisk_log_start":"0'0","created":183,"last_epoch_clean":184,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T14:52:28.654282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T14:52:28.654282+0000","last_clean_scrub_stamp":"2023-08-09T14:52:28.654282+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-11T00:27:57.653513+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"23.8","version":"0'0","reported_seq":9,"reported_epoch":185,"state":"active+clean","last_fresh":"2023-08-09T14:52:30.774738+0000","last_change":"2023-08-09T14:52:30.772364+0000","last_active":"2023-08-09T14:52:30.774738+0000","last_peered":"2023-08-09T14:52:30.774738+0000","last_clean":"2023-08-09T14:52:30.774738+0000","last_became_active":"2023-08-09T14:52:30.768062+0000","last_became_peered":"2023-08-09T14:52:30.768062+0000","last_unstale":"2023-08-09T14:52:30.774738+0000","last_undegraded":"2023-08-09T14:52:30.774738+0000","last_fullsized":"2023-08-09T14:52:30.774738+0000","mapping_epoch":184,"log_start":"0'0","ondisk_log_start":"0'0","created":184,"last_epoch_clean":185,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T14:52:29.665328+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T14:52:29.665328+0000","last_clean_scrub_stamp":"2023-08-09T14:52:29.665328+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T16:43:04.079456+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"22.9","version":"0'0","reported_seq":11,"reported_epoch":185,"state":"active+clean","last_fresh":"2023-08-09T14:52:30.734789+0000","last_change":"2023-08-09T14:52:29.716865+0000","last_active":"2023-08-09T14:52:30.734789+0000","last_peered":"2023-08-09T14:52:30.734789+0000","last_clean":"2023-08-09T14:52:30.734789+0000","last_became_active":"2023-08-09T14:52:29.716554+0000","last_became_peered":"2023-08-09T14:52:29.716554+0000","last_unstale":"2023-08-09T14:52:30.734789+0000","last_undegraded":"2023-08-09T14:52:30.734789+0000","last_fullsized":"2023-08-09T14:52:30.734789+0000","mapping_epoch":183,"log_start":"0'0","ondisk_log_start":"0'0","created":183,"last_epoch_clean":184,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T14:52:28.654282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T14:52:28.654282+0000","last_clean_scrub_stamp":"2023-08-09T14:52:28.654282+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T15:20:59.645144+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"23.9","version":"0'0","reported_seq":9,"reported_epoch":185,"state":"active+clean","last_fresh":"2023-08-09T14:52:30.757256+0000","last_change":"2023-08-09T14:52:30.748423+0000","last_active":"2023-08-09T14:52:30.757256+0000","last_peered":"2023-08-09T14:52:30.757256+0000","last_clean":"2023-08-09T14:52:30.757256+0000","last_became_active":"2023-08-09T14:52:30.748134+0000","last_became_peered":"2023-08-09T14:52:30.748134+0000","last_unstale":"2023-08-09T14:52:30.757256+0000","last_undegraded":"2023-08-09T14:52:30.757256+0000","last_fullsized":"2023-08-09T14:52:30.757256+0000","mapping_epoch":184,"log_start":"0'0","ondisk_log_start":"0'0","created":184,"last_epoch_clean":185,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T14:52:29.665328+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T14:52:29.665328+0000","last_clean_scrub_stamp":"2023-08-09T14:52:29.665328+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T18:28:37.865422+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"22.8","version":"0'0","reported_seq":11,"reported_epoch":185,"state":"active+clean","last_fresh":"2023-08-09T14:52:30.737010+0000","last_change":"2023-08-09T14:52:29.705724+0000","last_active":"2023-08-09T14:52:30.737010+0000","last_peered":"2023-08-09T14:52:30.737010+0000","last_clean":"2023-08-09T14:52:30.737010+0000","last_became_active":"2023-08-09T14:52:29.705214+0000","last_became_peered":"2023-08-09T14:52:29.705214+0000","last_unstale":"2023-08-09T14:52:30.737010+0000","last_undegraded":"2023-08-09T14:52:30.737010+0000","last_fullsized":"2023-08-09T14:52:30.737010+0000","mapping_epoch":183,"log_start":"0'0","ondisk_log_start":"0'0","created":183,"last_epoch_clean":184,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T14:52:28.654282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T14:52:28.654282+0000","last_clean_scrub_stamp":"2023-08-09T14:52:28.654282+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T16:41:30.543988+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"23.6","version":"0'0","reported_seq":9,"reported_epoch":185,"state":"active+clean","last_fresh":"2023-08-09T14:52:30.749020+0000","last_change":"2023-08-09T14:52:30.746688+0000","last_active":"2023-08-09T14:52:30.749020+0000","last_peered":"2023-08-09T14:52:30.749020+0000","last_clean":"2023-08-09T14:52:30.749020+0000","last_became_active":"2023-08-09T14:52:30.745831+0000","last_became_peered":"2023-08-09T14:52:30.745831+0000","last_unstale":"2023-08-09T14:52:30.749020+0000","last_undegraded":"2023-08-09T14:52:30.749020+0000","last_fullsized":"2023-08-09T14:52:30.749020+0000","mapping_epoch":184,"log_start":"0'0","ondisk_log_start":"0'0","created":184,"last_epoch_clean":185,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T14:52:29.665328+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T14:52:29.665328+0000","last_clean_scrub_stamp":"2023-08-09T14:52:29.665328+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T17:21:33.948301+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"22.7","version":"185'2","reported_seq":13,"reported_epoch":185,"state":"active+clean","last_fresh":"2023-08-09T14:52:30.771469+0000","last_change":"2023-08-09T14:52:29.708596+0000","last_active":"2023-08-09T14:52:30.771469+0000","last_peered":"2023-08-09T14:52:30.771469+0000","last_clean":"2023-08-09T14:52:30.771469+0000","last_became_active":"2023-08-09T14:52:29.707759+0000","last_became_peered":"2023-08-09T14:52:29.707759+0000","last_unstale":"2023-08-09T14:52:30.771469+0000","last_undegraded":"2023-08-09T14:52:30.771469+0000","last_fullsized":"2023-08-09T14:52:30.771469+0000","mapping_epoch":183,"log_start":"0'0","ondisk_log_start":"0'0","created":183,"last_epoch_clean":184,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T14:52:28.654282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T14:52:28.654282+0000","last_clean_scrub_stamp":"2023-08-09T14:52:28.654282+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T18:11:32.859448+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"23.7","version":"0'0","reported_seq":9,"reported_epoch":185,"state":"active+clean","last_fresh":"2023-08-09T14:52:30.763571+0000","last_change":"2023-08-09T14:52:30.759239+0000","last_active":"2023-08-09T14:52:30.763571+0000","last_peered":"2023-08-09T14:52:30.763571+0000","last_clean":"2023-08-09T14:52:30.763571+0000","last_became_active":"2023-08-09T14:52:30.758934+0000","last_became_peered":"2023-08-09T14:52:30.758934+0000","last_unstale":"2023-08-09T14:52:30.763571+0000","last_undegraded":"2023-08-09T14:52:30.763571+0000","last_fullsized":"2023-08-09T14:52:30.763571+0000","mapping_epoch":184,"log_start":"0'0","ondisk_log_start":"0'0","created":184,"last_epoch_clean":185,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T14:52:29.665328+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T14:52:29.665328+0000","last_clean_scrub_stamp":"2023-08-09T14:52:29.665328+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T15:17:59.713739+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"22.6","version":"0'0","reported_seq":11,"reported_epoch":185,"state":"active+clean","last_fresh":"2023-08-09T14:52:30.739115+0000","last_change":"2023-08-09T14:52:29.703991+0000","last_active":"2023-08-09T14:52:30.739115+0000","last_peered":"2023-08-09T14:52:30.739115+0000","last_clean":"2023-08-09T14:52:30.739115+0000","last_became_active":"2023-08-09T14:52:29.703708+0000","last_became_peered":"2023-08-09T14:52:29.703708+0000","last_unstale":"2023-08-09T14:52:30.739115+0000","last_undegraded":"2023-08-09T14:52:30.739115+0000","last_fullsized":"2023-08-09T14:52:30.739115+0000","mapping_epoch":183,"log_start":"0'0","ondisk_log_start":"0'0","created":183,"last_epoch_clean":184,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T14:52:28.654282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T14:52:28.654282+0000","last_clean_scrub_stamp":"2023-08-09T14:52:28.654282+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T22:56:34.647423+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"23.4","version":"0'0","reported_seq":9,"reported_epoch":185,"state":"active+clean","last_fresh":"2023-08-09T14:52:30.756835+0000","last_change":"2023-08-09T14:52:30.745677+0000","last_active":"2023-08-09T14:52:30.756835+0000","last_peered":"2023-08-09T14:52:30.756835+0000","last_clean":"2023-08-09T14:52:30.756835+0000","last_became_active":"2023-08-09T14:52:30.745168+0000","last_became_peered":"2023-08-09T14:52:30.745168+0000","last_unstale":"2023-08-09T14:52:30.756835+0000","last_undegraded":"2023-08-09T14:52:30.756835+0000","last_fullsized":"2023-08-09T14:52:30.756835+0000","mapping_epoch":184,"log_start":"0'0","ondisk_log_start":"0'0","created":184,"last_epoch_clean":185,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T14:52:29.665328+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T14:52:29.665328+0000","last_clean_scrub_stamp":"2023-08-09T14:52:29.665328+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T18:31:29.821581+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"22.5","version":"0'0","reported_seq":11,"reported_epoch":185,"state":"active+clean","last_fresh":"2023-08-09T14:52:30.743005+0000","last_change":"2023-08-09T14:52:29.712162+0000","last_active":"2023-08-09T14:52:30.743005+0000","last_peered":"2023-08-09T14:52:30.743005+0000","last_clean":"2023-08-09T14:52:30.743005+0000","last_became_active":"2023-08-09T14:52:29.711832+0000","last_became_peered":"2023-08-09T14:52:29.711832+0000","last_unstale":"2023-08-09T14:52:30.743005+0000","last_undegraded":"2023-08-09T14:52:30.743005+0000","last_fullsized":"2023-08-09T14:52:30.743005+0000","mapping_epoch":183,"log_start":"0'0","ondisk_log_start":"0'0","created":183,"last_epoch_clean":184,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T14:52:28.654282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T14:52:28.654282+0000","last_clean_scrub_stamp":"2023-08-09T14:52:28.654282+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T17:45:12.141004+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"23.5","version":"0'0","reported_seq":9,"reported_epoch":185,"state":"active+clean","last_fresh":"2023-08-09T14:52:30.757335+0000","last_change":"2023-08-09T14:52:30.755065+0000","last_active":"2023-08-09T14:52:30.757335+0000","last_peered":"2023-08-09T14:52:30.757335+0000","last_clean":"2023-08-09T14:52:30.757335+0000","last_became_active":"2023-08-09T14:52:30.754799+0000","last_became_peered":"2023-08-09T14:52:30.754799+0000","last_unstale":"2023-08-09T14:52:30.757335+0000","last_undegraded":"2023-08-09T14:52:30.757335+0000","last_fullsized":"2023-08-09T14:52:30.757335+0000","mapping_epoch":184,"log_start":"0'0","ondisk_log_start":"0'0","created":184,"last_epoch_clean":185,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T14:52:29.665328+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T14:52:29.665328+0000","last_clean_scrub_stamp":"2023-08-09T14:52:29.665328+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T19:02:24.807175+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"22.4","version":"0'0","reported_seq":11,"reported_epoch":185,"state":"active+clean","last_fresh":"2023-08-09T14:52:30.727289+0000","last_change":"2023-08-09T14:52:29.714203+0000","last_active":"2023-08-09T14:52:30.727289+0000","last_peered":"2023-08-09T14:52:30.727289+0000","last_clean":"2023-08-09T14:52:30.727289+0000","last_became_active":"2023-08-09T14:52:29.713927+0000","last_became_peered":"2023-08-09T14:52:29.713927+0000","last_unstale":"2023-08-09T14:52:30.727289+0000","last_undegraded":"2023-08-09T14:52:30.727289+0000","last_fullsized":"2023-08-09T14:52:30.727289+0000","mapping_epoch":183,"log_start":"0'0","ondisk_log_start":"0'0","created":183,"last_epoch_clean":184,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T14:52:28.654282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T14:52:28.654282+0000","last_clean_scrub_stamp":"2023-08-09T14:52:28.654282+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T20:08:49.741258+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"23.2","version":"0'0","reported_seq":9,"reported_epoch":185,"state":"active+clean","last_fresh":"2023-08-09T14:52:30.754415+0000","last_change":"2023-08-09T14:52:30.746639+0000","last_active":"2023-08-09T14:52:30.754415+0000","last_peered":"2023-08-09T14:52:30.754415+0000","last_clean":"2023-08-09T14:52:30.754415+0000","last_became_active":"2023-08-09T14:52:30.746337+0000","last_became_peered":"2023-08-09T14:52:30.746337+0000","last_unstale":"2023-08-09T14:52:30.754415+0000","last_undegraded":"2023-08-09T14:52:30.754415+0000","last_fullsized":"2023-08-09T14:52:30.754415+0000","mapping_epoch":184,"log_start":"0'0","ondisk_log_start":"0'0","created":184,"last_epoch_clean":185,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T14:52:29.665328+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T14:52:29.665328+0000","last_clean_scrub_stamp":"2023-08-09T14:52:29.665328+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T22:12:54.884632+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"22.3","version":"0'0","reported_seq":11,"reported_epoch":185,"state":"active+clean","last_fresh":"2023-08-09T14:52:30.740203+0000","last_change":"2023-08-09T14:52:29.702701+0000","last_active":"2023-08-09T14:52:30.740203+0000","last_peered":"2023-08-09T14:52:30.740203+0000","last_clean":"2023-08-09T14:52:30.740203+0000","last_became_active":"2023-08-09T14:52:29.702431+0000","last_became_peered":"2023-08-09T14:52:29.702431+0000","last_unstale":"2023-08-09T14:52:30.740203+0000","last_undegraded":"2023-08-09T14:52:30.740203+0000","last_fullsized":"2023-08-09T14:52:30.740203+0000","mapping_epoch":183,"log_start":"0'0","ondisk_log_start":"0'0","created":183,"last_epoch_clean":184,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T14:52:28.654282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T14:52:28.654282+0000","last_clean_scrub_stamp":"2023-08-09T14:52:28.654282+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T15:06:20.990122+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"23.3","version":"0'0","reported_seq":9,"reported_epoch":185,"state":"active+clean","last_fresh":"2023-08-09T14:52:30.766359+0000","last_change":"2023-08-09T14:52:30.759160+0000","last_active":"2023-08-09T14:52:30.766359+0000","last_peered":"2023-08-09T14:52:30.766359+0000","last_clean":"2023-08-09T14:52:30.766359+0000","last_became_active":"2023-08-09T14:52:30.758862+0000","last_became_peered":"2023-08-09T14:52:30.758862+0000","last_unstale":"2023-08-09T14:52:30.766359+0000","last_undegraded":"2023-08-09T14:52:30.766359+0000","last_fullsized":"2023-08-09T14:52:30.766359+0000","mapping_epoch":184,"log_start":"0'0","ondisk_log_start":"0'0","created":184,"last_epoch_clean":185,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T14:52:29.665328+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T14:52:29.665328+0000","last_clean_scrub_stamp":"2023-08-09T14:52:29.665328+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T18:55:43.404564+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"22.2","version":"0'0","reported_seq":11,"reported_epoch":185,"state":"active+clean","last_fresh":"2023-08-09T14:52:30.734096+0000","last_change":"2023-08-09T14:52:29.692393+0000","last_active":"2023-08-09T14:52:30.734096+0000","last_peered":"2023-08-09T14:52:30.734096+0000","last_clean":"2023-08-09T14:52:30.734096+0000","last_became_active":"2023-08-09T14:52:29.691789+0000","last_became_peered":"2023-08-09T14:52:29.691789+0000","last_unstale":"2023-08-09T14:52:30.734096+0000","last_undegraded":"2023-08-09T14:52:30.734096+0000","last_fullsized":"2023-08-09T14:52:30.734096+0000","mapping_epoch":183,"log_start":"0'0","ondisk_log_start":"0'0","created":183,"last_epoch_clean":184,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T14:52:28.654282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T14:52:28.654282+0000","last_clean_scrub_stamp":"2023-08-09T14:52:28.654282+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T17:38:05.365447+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"23.0","version":"0'0","reported_seq":9,"reported_epoch":185,"state":"active+clean","last_fresh":"2023-08-09T14:52:30.757142+0000","last_change":"2023-08-09T14:52:30.750614+0000","last_active":"2023-08-09T14:52:30.757142+0000","last_peered":"2023-08-09T14:52:30.757142+0000","last_clean":"2023-08-09T14:52:30.757142+0000","last_became_active":"2023-08-09T14:52:30.749983+0000","last_became_peered":"2023-08-09T14:52:30.749983+0000","last_unstale":"2023-08-09T14:52:30.757142+0000","last_undegraded":"2023-08-09T14:52:30.757142+0000","last_fullsized":"2023-08-09T14:52:30.757142+0000","mapping_epoch":184,"log_start":"0'0","ondisk_log_start":"0'0","created":184,"last_epoch_clean":185,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T14:52:29.665328+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T14:52:29.665328+0000","last_clean_scrub_stamp":"2023-08-09T14:52:29.665328+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-11T00:38:39.582024+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"22.1","version":"0'0","reported_seq":11,"reported_epoch":185,"state":"active+clean","last_fresh":"2023-08-09T14:52:30.744520+0000","last_change":"2023-08-09T14:52:29.709333+0000","last_active":"2023-08-09T14:52:30.744520+0000","last_peered":"2023-08-09T14:52:30.744520+0000","last_clean":"2023-08-09T14:52:30.744520+0000","last_became_active":"2023-08-09T14:52:29.708994+0000","last_became_peered":"2023-08-09T14:52:29.708994+0000","last_unstale":"2023-08-09T14:52:30.744520+0000","last_undegraded":"2023-08-09T14:52:30.744520+0000","last_fullsized":"2023-08-09T14:52:30.744520+0000","mapping_epoch":183,"log_start":"0'0","ondisk_log_start":"0'0","created":183,"last_epoch_clean":184,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T14:52:28.654282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T14:52:28.654282+0000","last_clean_scrub_stamp":"2023-08-09T14:52:28.654282+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T16:04:48.527506+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"23.1","version":"0'0","reported_seq":9,"reported_epoch":185,"state":"active+clean","last_fresh":"2023-08-09T14:52:30.748409+0000","last_change":"2023-08-09T14:52:30.747011+0000","last_active":"2023-08-09T14:52:30.748409+0000","last_peered":"2023-08-09T14:52:30.748409+0000","last_clean":"2023-08-09T14:52:30.748409+0000","last_became_active":"2023-08-09T14:52:30.746102+0000","last_became_peered":"2023-08-09T14:52:30.746102+0000","last_unstale":"2023-08-09T14:52:30.748409+0000","last_undegraded":"2023-08-09T14:52:30.748409+0000","last_fullsized":"2023-08-09T14:52:30.748409+0000","mapping_epoch":184,"log_start":"0'0","ondisk_log_start":"0'0","created":184,"last_epoch_clean":185,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T14:52:29.665328+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T14:52:29.665328+0000","last_clean_scrub_stamp":"2023-08-09T14:52:29.665328+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T19:10:33.908606+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"22.0","version":"0'0","reported_seq":11,"reported_epoch":185,"state":"active+clean","last_fresh":"2023-08-09T14:52:30.735628+0000","last_change":"2023-08-09T14:52:29.699378+0000","last_active":"2023-08-09T14:52:30.735628+0000","last_peered":"2023-08-09T14:52:30.735628+0000","last_clean":"2023-08-09T14:52:30.735628+0000","last_became_active":"2023-08-09T14:52:29.698999+0000","last_became_peered":"2023-08-09T14:52:29.698999+0000","last_unstale":"2023-08-09T14:52:30.735628+0000","last_undegraded":"2023-08-09T14:52:30.735628+0000","last_fullsized":"2023-08-09T14:52:30.735628+0000","mapping_epoch":183,"log_start":"0'0","ondisk_log_start":"0'0","created":183,"last_epoch_clean":184,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T14:52:28.654282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T14:52:28.654282+0000","last_clean_scrub_stamp":"2023-08-09T14:52:28.654282+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-11T01:22:08.109189+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"23.16","version":"0'0","reported_seq":9,"reported_epoch":185,"state":"active+clean","last_fresh":"2023-08-09T14:52:30.757340+0000","last_change":"2023-08-09T14:52:30.748843+0000","last_active":"2023-08-09T14:52:30.757340+0000","last_peered":"2023-08-09T14:52:30.757340+0000","last_clean":"2023-08-09T14:52:30.757340+0000","last_became_active":"2023-08-09T14:52:30.747365+0000","last_became_peered":"2023-08-09T14:52:30.747365+0000","last_unstale":"2023-08-09T14:52:30.757340+0000","last_undegraded":"2023-08-09T14:52:30.757340+0000","last_fullsized":"2023-08-09T14:52:30.757340+0000","mapping_epoch":184,"log_start":"0'0","ondisk_log_start":"0'0","created":184,"last_epoch_clean":185,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T14:52:29.665328+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T14:52:29.665328+0000","last_clean_scrub_stamp":"2023-08-09T14:52:29.665328+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T18:06:08.837776+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"22.17","version":"0'0","reported_seq":11,"reported_epoch":185,"state":"active+clean","last_fresh":"2023-08-09T14:52:30.742688+0000","last_change":"2023-08-09T14:52:29.712808+0000","last_active":"2023-08-09T14:52:30.742688+0000","last_peered":"2023-08-09T14:52:30.742688+0000","last_clean":"2023-08-09T14:52:30.742688+0000","last_became_active":"2023-08-09T14:52:29.712507+0000","last_became_peered":"2023-08-09T14:52:29.712507+0000","last_unstale":"2023-08-09T14:52:30.742688+0000","last_undegraded":"2023-08-09T14:52:30.742688+0000","last_fullsized":"2023-08-09T14:52:30.742688+0000","mapping_epoch":183,"log_start":"0'0","ondisk_log_start":"0'0","created":183,"last_epoch_clean":184,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T14:52:28.654282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T14:52:28.654282+0000","last_clean_scrub_stamp":"2023-08-09T14:52:28.654282+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T22:31:25.507328+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":340,"reported_epoch":185,"state":"active+clean","last_fresh":"2023-08-09T14:52:30.737319+0000","last_change":"2023-08-09T13:33:56.401406+0000","last_active":"2023-08-09T14:52:30.737319+0000","last_peered":"2023-08-09T14:52:30.737319+0000","last_clean":"2023-08-09T14:52:30.737319+0000","last_became_active":"2023-08-09T13:33:56.400487+0000","last_became_peered":"2023-08-09T13:33:56.400487+0000","last_unstale":"2023-08-09T14:52:30.737319+0000","last_undegraded":"2023-08-09T14:52:30.737319+0000","last_fullsized":"2023-08-09T14:52:30.737319+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:33:54.845869+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:33:54.845869+0000","last_clean_scrub_stamp":"2023-08-09T13:33:54.845869+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T16:55:01.819791+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"23.17","version":"0'0","reported_seq":9,"reported_epoch":185,"state":"active+clean","last_fresh":"2023-08-09T14:52:30.757405+0000","last_change":"2023-08-09T14:52:30.742487+0000","last_active":"2023-08-09T14:52:30.757405+0000","last_peered":"2023-08-09T14:52:30.757405+0000","last_clean":"2023-08-09T14:52:30.757405+0000","last_became_active":"2023-08-09T14:52:30.741798+0000","last_became_peered":"2023-08-09T14:52:30.741798+0000","last_unstale":"2023-08-09T14:52:30.757405+0000","last_undegraded":"2023-08-09T14:52:30.757405+0000","last_fullsized":"2023-08-09T14:52:30.757405+0000","mapping_epoch":184,"log_start":"0'0","ondisk_log_start":"0'0","created":184,"last_epoch_clean":185,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T14:52:29.665328+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T14:52:29.665328+0000","last_clean_scrub_stamp":"2023-08-09T14:52:29.665328+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T21:57:20.199759+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"22.16","version":"0'0","reported_seq":11,"reported_epoch":185,"state":"active+clean","last_fresh":"2023-08-09T14:52:30.724008+0000","last_change":"2023-08-09T14:52:29.706553+0000","last_active":"2023-08-09T14:52:30.724008+0000","last_peered":"2023-08-09T14:52:30.724008+0000","last_clean":"2023-08-09T14:52:30.724008+0000","last_became_active":"2023-08-09T14:52:29.706199+0000","last_became_peered":"2023-08-09T14:52:29.706199+0000","last_unstale":"2023-08-09T14:52:30.724008+0000","last_undegraded":"2023-08-09T14:52:30.724008+0000","last_fullsized":"2023-08-09T14:52:30.724008+0000","mapping_epoch":183,"log_start":"0'0","ondisk_log_start":"0'0","created":183,"last_epoch_clean":184,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T14:52:28.654282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T14:52:28.654282+0000","last_clean_scrub_stamp":"2023-08-09T14:52:28.654282+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T20:20:06.900048+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":348,"reported_epoch":185,"state":"active+clean","last_fresh":"2023-08-09T14:52:30.732967+0000","last_change":"2023-08-09T13:33:56.401123+0000","last_active":"2023-08-09T14:52:30.732967+0000","last_peered":"2023-08-09T14:52:30.732967+0000","last_clean":"2023-08-09T14:52:30.732967+0000","last_became_active":"2023-08-09T13:33:56.399933+0000","last_became_peered":"2023-08-09T13:33:56.399933+0000","last_unstale":"2023-08-09T14:52:30.732967+0000","last_undegraded":"2023-08-09T14:52:30.732967+0000","last_fullsized":"2023-08-09T14:52:30.732967+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:33:54.845869+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:33:54.845869+0000","last_clean_scrub_stamp":"2023-08-09T13:33:54.845869+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-11T01:14:31.338701+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"23.14","version":"0'0","reported_seq":9,"reported_epoch":185,"state":"active+clean","last_fresh":"2023-08-09T14:52:30.747632+0000","last_change":"2023-08-09T14:52:30.746166+0000","last_active":"2023-08-09T14:52:30.747632+0000","last_peered":"2023-08-09T14:52:30.747632+0000","last_clean":"2023-08-09T14:52:30.747632+0000","last_became_active":"2023-08-09T14:52:30.745820+0000","last_became_peered":"2023-08-09T14:52:30.745820+0000","last_unstale":"2023-08-09T14:52:30.747632+0000","last_undegraded":"2023-08-09T14:52:30.747632+0000","last_fullsized":"2023-08-09T14:52:30.747632+0000","mapping_epoch":184,"log_start":"0'0","ondisk_log_start":"0'0","created":184,"last_epoch_clean":185,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T14:52:29.665328+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T14:52:29.665328+0000","last_clean_scrub_stamp":"2023-08-09T14:52:29.665328+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T23:19:45.490259+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"22.15","version":"185'3","reported_seq":14,"reported_epoch":185,"state":"active+clean","last_fresh":"2023-08-09T14:52:30.773557+0000","last_change":"2023-08-09T14:52:29.710014+0000","last_active":"2023-08-09T14:52:30.773557+0000","last_peered":"2023-08-09T14:52:30.773557+0000","last_clean":"2023-08-09T14:52:30.773557+0000","last_became_active":"2023-08-09T14:52:29.709737+0000","last_became_peered":"2023-08-09T14:52:29.709737+0000","last_unstale":"2023-08-09T14:52:30.773557+0000","last_undegraded":"2023-08-09T14:52:30.773557+0000","last_fullsized":"2023-08-09T14:52:30.773557+0000","mapping_epoch":183,"log_start":"0'0","ondisk_log_start":"0'0","created":183,"last_epoch_clean":184,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T14:52:28.654282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T14:52:28.654282+0000","last_clean_scrub_stamp":"2023-08-09T14:52:28.654282+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-11T00:13:22.178599+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.2","version":"0'0","reported_seq":338,"reported_epoch":185,"state":"active+clean","last_fresh":"2023-08-09T14:52:30.735966+0000","last_change":"2023-08-09T13:33:56.402794+0000","last_active":"2023-08-09T14:52:30.735966+0000","last_peered":"2023-08-09T14:52:30.735966+0000","last_clean":"2023-08-09T14:52:30.735966+0000","last_became_active":"2023-08-09T13:33:56.402038+0000","last_became_peered":"2023-08-09T13:33:56.402038+0000","last_unstale":"2023-08-09T14:52:30.735966+0000","last_undegraded":"2023-08-09T14:52:30.735966+0000","last_fullsized":"2023-08-09T14:52:30.735966+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:33:54.845869+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:33:54.845869+0000","last_clean_scrub_stamp":"2023-08-09T13:33:54.845869+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T19:04:24.398873+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"23.15","version":"0'0","reported_seq":9,"reported_epoch":185,"state":"active+clean","last_fresh":"2023-08-09T14:52:30.760545+0000","last_change":"2023-08-09T14:52:30.756276+0000","last_active":"2023-08-09T14:52:30.760545+0000","last_peered":"2023-08-09T14:52:30.760545+0000","last_clean":"2023-08-09T14:52:30.760545+0000","last_became_active":"2023-08-09T14:52:30.755934+0000","last_became_peered":"2023-08-09T14:52:30.755934+0000","last_unstale":"2023-08-09T14:52:30.760545+0000","last_undegraded":"2023-08-09T14:52:30.760545+0000","last_fullsized":"2023-08-09T14:52:30.760545+0000","mapping_epoch":184,"log_start":"0'0","ondisk_log_start":"0'0","created":184,"last_epoch_clean":185,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T14:52:29.665328+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T14:52:29.665328+0000","last_clean_scrub_stamp":"2023-08-09T14:52:29.665328+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-11T00:27:44.822389+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"22.14","version":"185'6","reported_seq":17,"reported_epoch":185,"state":"active+clean","last_fresh":"2023-08-09T14:52:30.775627+0000","last_change":"2023-08-09T14:52:29.706374+0000","last_active":"2023-08-09T14:52:30.775627+0000","last_peered":"2023-08-09T14:52:30.775627+0000","last_clean":"2023-08-09T14:52:30.775627+0000","last_became_active":"2023-08-09T14:52:29.706004+0000","last_became_peered":"2023-08-09T14:52:29.706004+0000","last_unstale":"2023-08-09T14:52:30.775627+0000","last_undegraded":"2023-08-09T14:52:30.775627+0000","last_fullsized":"2023-08-09T14:52:30.775627+0000","mapping_epoch":183,"log_start":"0'0","ondisk_log_start":"0'0","created":183,"last_epoch_clean":184,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T14:52:28.654282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T14:52:28.654282+0000","last_clean_scrub_stamp":"2023-08-09T14:52:28.654282+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-11T00:40:29.631853+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.3","version":"0'0","reported_seq":350,"reported_epoch":185,"state":"active+clean","last_fresh":"2023-08-09T14:52:30.736532+0000","last_change":"2023-08-09T13:33:55.886749+0000","last_active":"2023-08-09T14:52:30.736532+0000","last_peered":"2023-08-09T14:52:30.736532+0000","last_clean":"2023-08-09T14:52:30.736532+0000","last_became_active":"2023-08-09T13:33:55.886116+0000","last_became_peered":"2023-08-09T13:33:55.886116+0000","last_unstale":"2023-08-09T14:52:30.736532+0000","last_undegraded":"2023-08-09T14:52:30.736532+0000","last_fullsized":"2023-08-09T14:52:30.736532+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:33:54.845869+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:33:54.845869+0000","last_clean_scrub_stamp":"2023-08-09T13:33:54.845869+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-11T00:13:44.684470+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"174'274","reported_seq":1046,"reported_epoch":185,"state":"active+clean","last_fresh":"2023-08-09T14:52:30.737131+0000","last_change":"2023-08-09T13:33:55.886345+0000","last_active":"2023-08-09T14:52:30.737131+0000","last_peered":"2023-08-09T14:52:30.737131+0000","last_clean":"2023-08-09T14:52:30.737131+0000","last_became_active":"2023-08-09T13:33:55.885781+0000","last_became_peered":"2023-08-09T13:33:55.885781+0000","last_unstale":"2023-08-09T14:52:30.737131+0000","last_undegraded":"2023-08-09T14:52:30.737131+0000","last_fullsized":"2023-08-09T14:52:30.737131+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:33:54.845869+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:33:54.845869+0000","last_clean_scrub_stamp":"2023-08-09T13:33:54.845869+0000","objects_scrubbed":0,"log_size":274,"log_dups_size":0,"ondisk_log_size":274,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-11T00:50:15.334951+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1483,"num_read_kb":3066,"num_write":443,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"23.12","version":"0'0","reported_seq":9,"reported_epoch":185,"state":"active+clean","last_fresh":"2023-08-09T14:52:30.762910+0000","last_change":"2023-08-09T14:52:30.761825+0000","last_active":"2023-08-09T14:52:30.762910+0000","last_peered":"2023-08-09T14:52:30.762910+0000","last_clean":"2023-08-09T14:52:30.762910+0000","last_became_active":"2023-08-09T14:52:30.761412+0000","last_became_peered":"2023-08-09T14:52:30.761412+0000","last_unstale":"2023-08-09T14:52:30.762910+0000","last_undegraded":"2023-08-09T14:52:30.762910+0000","last_fullsized":"2023-08-09T14:52:30.762910+0000","mapping_epoch":184,"log_start":"0'0","ondisk_log_start":"0'0","created":184,"last_epoch_clean":185,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T14:52:29.665328+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T14:52:29.665328+0000","last_clean_scrub_stamp":"2023-08-09T14:52:29.665328+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-11T00:41:24.209798+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"22.13","version":"185'2","reported_seq":13,"reported_epoch":185,"state":"active+clean","last_fresh":"2023-08-09T14:52:30.772863+0000","last_change":"2023-08-09T14:52:29.697008+0000","last_active":"2023-08-09T14:52:30.772863+0000","last_peered":"2023-08-09T14:52:30.772863+0000","last_clean":"2023-08-09T14:52:30.772863+0000","last_became_active":"2023-08-09T14:52:29.696501+0000","last_became_peered":"2023-08-09T14:52:29.696501+0000","last_unstale":"2023-08-09T14:52:30.772863+0000","last_undegraded":"2023-08-09T14:52:30.772863+0000","last_fullsized":"2023-08-09T14:52:30.772863+0000","mapping_epoch":183,"log_start":"0'0","ondisk_log_start":"0'0","created":183,"last_epoch_clean":184,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T14:52:28.654282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T14:52:28.654282+0000","last_clean_scrub_stamp":"2023-08-09T14:52:28.654282+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-11T00:03:28.802518+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":350,"reported_epoch":185,"state":"active+clean","last_fresh":"2023-08-09T14:52:30.727531+0000","last_change":"2023-08-09T13:33:55.889565+0000","last_active":"2023-08-09T14:52:30.727531+0000","last_peered":"2023-08-09T14:52:30.727531+0000","last_clean":"2023-08-09T14:52:30.727531+0000","last_became_active":"2023-08-09T13:33:55.889164+0000","last_became_peered":"2023-08-09T13:33:55.889164+0000","last_unstale":"2023-08-09T14:52:30.727531+0000","last_undegraded":"2023-08-09T14:52:30.727531+0000","last_fullsized":"2023-08-09T14:52:30.727531+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:33:54.845869+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:33:54.845869+0000","last_clean_scrub_stamp":"2023-08-09T13:33:54.845869+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T13:45:16.129340+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"23.13","version":"0'0","reported_seq":9,"reported_epoch":185,"state":"active+clean","last_fresh":"2023-08-09T14:52:30.756698+0000","last_change":"2023-08-09T14:52:30.749497+0000","last_active":"2023-08-09T14:52:30.756698+0000","last_peered":"2023-08-09T14:52:30.756698+0000","last_clean":"2023-08-09T14:52:30.756698+0000","last_became_active":"2023-08-09T14:52:30.746836+0000","last_became_peered":"2023-08-09T14:52:30.746836+0000","last_unstale":"2023-08-09T14:52:30.756698+0000","last_undegraded":"2023-08-09T14:52:30.756698+0000","last_fullsized":"2023-08-09T14:52:30.756698+0000","mapping_epoch":184,"log_start":"0'0","ondisk_log_start":"0'0","created":184,"last_epoch_clean":185,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T14:52:29.665328+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T14:52:29.665328+0000","last_clean_scrub_stamp":"2023-08-09T14:52:29.665328+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T17:45:30.550869+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"22.12","version":"0'0","reported_seq":11,"reported_epoch":185,"state":"active+clean","last_fresh":"2023-08-09T14:52:30.737083+0000","last_change":"2023-08-09T14:52:29.708545+0000","last_active":"2023-08-09T14:52:30.737083+0000","last_peered":"2023-08-09T14:52:30.737083+0000","last_clean":"2023-08-09T14:52:30.737083+0000","last_became_active":"2023-08-09T14:52:29.707008+0000","last_became_peered":"2023-08-09T14:52:29.707008+0000","last_unstale":"2023-08-09T14:52:30.737083+0000","last_undegraded":"2023-08-09T14:52:30.737083+0000","last_fullsized":"2023-08-09T14:52:30.737083+0000","mapping_epoch":183,"log_start":"0'0","ondisk_log_start":"0'0","created":183,"last_epoch_clean":184,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T14:52:28.654282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T14:52:28.654282+0000","last_clean_scrub_stamp":"2023-08-09T14:52:28.654282+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-11T02:20:03.687422+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":348,"reported_epoch":185,"state":"active+clean","last_fresh":"2023-08-09T14:52:30.742114+0000","last_change":"2023-08-09T13:33:56.404236+0000","last_active":"2023-08-09T14:52:30.742114+0000","last_peered":"2023-08-09T14:52:30.742114+0000","last_clean":"2023-08-09T14:52:30.742114+0000","last_became_active":"2023-08-09T13:33:56.402485+0000","last_became_peered":"2023-08-09T13:33:56.402485+0000","last_unstale":"2023-08-09T14:52:30.742114+0000","last_undegraded":"2023-08-09T14:52:30.742114+0000","last_fullsized":"2023-08-09T14:52:30.742114+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:33:54.845869+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:33:54.845869+0000","last_clean_scrub_stamp":"2023-08-09T13:33:54.845869+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T22:13:54.485808+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"23.10","version":"0'0","reported_seq":9,"reported_epoch":185,"state":"active+clean","last_fresh":"2023-08-09T14:52:30.765202+0000","last_change":"2023-08-09T14:52:30.757015+0000","last_active":"2023-08-09T14:52:30.765202+0000","last_peered":"2023-08-09T14:52:30.765202+0000","last_clean":"2023-08-09T14:52:30.765202+0000","last_became_active":"2023-08-09T14:52:30.756469+0000","last_became_peered":"2023-08-09T14:52:30.756469+0000","last_unstale":"2023-08-09T14:52:30.765202+0000","last_undegraded":"2023-08-09T14:52:30.765202+0000","last_fullsized":"2023-08-09T14:52:30.765202+0000","mapping_epoch":184,"log_start":"0'0","ondisk_log_start":"0'0","created":184,"last_epoch_clean":185,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T14:52:29.665328+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T14:52:29.665328+0000","last_clean_scrub_stamp":"2023-08-09T14:52:29.665328+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T23:01:36.233201+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"22.11","version":"0'0","reported_seq":11,"reported_epoch":185,"state":"active+clean","last_fresh":"2023-08-09T14:52:30.735757+0000","last_change":"2023-08-09T14:52:29.712599+0000","last_active":"2023-08-09T14:52:30.735757+0000","last_peered":"2023-08-09T14:52:30.735757+0000","last_clean":"2023-08-09T14:52:30.735757+0000","last_became_active":"2023-08-09T14:52:29.712276+0000","last_became_peered":"2023-08-09T14:52:29.712276+0000","last_unstale":"2023-08-09T14:52:30.735757+0000","last_undegraded":"2023-08-09T14:52:30.735757+0000","last_fullsized":"2023-08-09T14:52:30.735757+0000","mapping_epoch":183,"log_start":"0'0","ondisk_log_start":"0'0","created":183,"last_epoch_clean":184,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T14:52:28.654282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T14:52:28.654282+0000","last_clean_scrub_stamp":"2023-08-09T14:52:28.654282+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T23:27:30.639308+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":337,"reported_epoch":185,"state":"active+clean","last_fresh":"2023-08-09T14:52:30.741653+0000","last_change":"2023-08-09T13:33:56.401781+0000","last_active":"2023-08-09T14:52:30.741653+0000","last_peered":"2023-08-09T14:52:30.741653+0000","last_clean":"2023-08-09T14:52:30.741653+0000","last_became_active":"2023-08-09T13:33:56.400336+0000","last_became_peered":"2023-08-09T13:33:56.400336+0000","last_unstale":"2023-08-09T14:52:30.741653+0000","last_undegraded":"2023-08-09T14:52:30.741653+0000","last_fullsized":"2023-08-09T14:52:30.741653+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:33:54.845869+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:33:54.845869+0000","last_clean_scrub_stamp":"2023-08-09T13:33:54.845869+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T23:49:17.599206+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"23.11","version":"0'0","reported_seq":9,"reported_epoch":185,"state":"active+clean","last_fresh":"2023-08-09T14:52:30.750485+0000","last_change":"2023-08-09T14:52:30.749874+0000","last_active":"2023-08-09T14:52:30.750485+0000","last_peered":"2023-08-09T14:52:30.750485+0000","last_clean":"2023-08-09T14:52:30.750485+0000","last_became_active":"2023-08-09T14:52:30.749195+0000","last_became_peered":"2023-08-09T14:52:30.749195+0000","last_unstale":"2023-08-09T14:52:30.750485+0000","last_undegraded":"2023-08-09T14:52:30.750485+0000","last_fullsized":"2023-08-09T14:52:30.750485+0000","mapping_epoch":184,"log_start":"0'0","ondisk_log_start":"0'0","created":184,"last_epoch_clean":185,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T14:52:29.665328+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T14:52:29.665328+0000","last_clean_scrub_stamp":"2023-08-09T14:52:29.665328+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T21:24:37.801551+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"22.10","version":"185'8","reported_seq":19,"reported_epoch":185,"state":"active+clean","last_fresh":"2023-08-09T14:52:30.775295+0000","last_change":"2023-08-09T14:52:29.706475+0000","last_active":"2023-08-09T14:52:30.775295+0000","last_peered":"2023-08-09T14:52:30.775295+0000","last_clean":"2023-08-09T14:52:30.775295+0000","last_became_active":"2023-08-09T14:52:29.706135+0000","last_became_peered":"2023-08-09T14:52:29.706135+0000","last_unstale":"2023-08-09T14:52:30.775295+0000","last_undegraded":"2023-08-09T14:52:30.775295+0000","last_fullsized":"2023-08-09T14:52:30.775295+0000","mapping_epoch":183,"log_start":"0'0","ondisk_log_start":"0'0","created":183,"last_epoch_clean":184,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T14:52:28.654282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T14:52:28.654282+0000","last_clean_scrub_stamp":"2023-08-09T14:52:28.654282+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T22:53:32.087258+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":337,"reported_epoch":185,"state":"active+clean","last_fresh":"2023-08-09T14:52:30.737797+0000","last_change":"2023-08-09T13:33:55.886389+0000","last_active":"2023-08-09T14:52:30.737797+0000","last_peered":"2023-08-09T14:52:30.737797+0000","last_clean":"2023-08-09T14:52:30.737797+0000","last_became_active":"2023-08-09T13:33:55.885638+0000","last_became_peered":"2023-08-09T13:33:55.885638+0000","last_unstale":"2023-08-09T14:52:30.737797+0000","last_undegraded":"2023-08-09T14:52:30.737797+0000","last_fullsized":"2023-08-09T14:52:30.737797+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:33:54.845869+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:33:54.845869+0000","last_clean_scrub_stamp":"2023-08-09T13:33:54.845869+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T21:37:53.176770+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]}],"pool_stats":[{"poolid":23,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":22,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":38,"ondisk_log_size":38,"up":64,"acting":64,"num_store_stats":8},{"poolid":2,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1483,"num_read_kb":3066,"num_write":443,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":274,"ondisk_log_size":274,"up":2,"acting":2,"num_store_stats":2},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":8}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834575787,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":47908,"kb_used_data":2836,"kb_used_omap":1,"kb_used_meta":45054,"kb_avail":94323932,"statfs":{"total":96636764160,"available":96587706368,"internally_reserved":0,"allocated":2904064,"data_stored":1840624,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1781,"internal_metadata":46135563},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Aug 9 14:52:11 2023","interfaces":[{"interface":"back","average":{"1min":1.129,"5min":1.1990000000000001,"15min":1.23},"min":{"1min":0.57899999999999996,"5min":0.57899999999999996,"15min":0.57899999999999996},"max":{"1min":1.3939999999999999,"5min":1.6930000000000001,"15min":2.2320000000000002},"last":1.208},{"interface":"front","average":{"1min":1.202,"5min":1.2250000000000001,"15min":1.264},"min":{"1min":0.76100000000000001,"5min":0.76100000000000001,"15min":0.76100000000000001},"max":{"1min":1.883,"5min":1.883,"15min":1.883},"last":1.036}]},{"osd":1,"last update":"Wed Aug 9 14:52:11 2023","interfaces":[{"interface":"back","average":{"1min":1.2150000000000001,"5min":1.2889999999999999,"15min":1.319},"min":{"1min":0.72799999999999998,"5min":0.72799999999999998,"15min":0.72799999999999998},"max":{"1min":1.7350000000000001,"5min":1.9239999999999999,"15min":1.9470000000000001},"last":1.391},{"interface":"front","average":{"1min":1.262,"5min":1.3089999999999999,"15min":1.3089999999999999},"min":{"1min":0.52000000000000002,"5min":0.52000000000000002,"15min":0.52000000000000002},"max":{"1min":1.976,"5min":1.976,"15min":1.976},"last":1.151}]},{"osd":2,"last update":"Wed Aug 9 14:52:11 2023","interfaces":[{"interface":"back","average":{"1min":1.2,"5min":1.276,"15min":1.304},"min":{"1min":0.83099999999999996,"5min":0.83099999999999996,"15min":0.83099999999999996},"max":{"1min":1.577,"5min":1.764,"15min":2.3149999999999999},"last":0.77200000000000002},{"interface":"front","average":{"1min":1.294,"5min":1.365,"15min":1.387},"min":{"1min":1.0149999999999999,"5min":0.93200000000000005,"15min":0.92100000000000004},"max":{"1min":1.6499999999999999,"5min":1.9279999999999999,"15min":2.0110000000000001},"last":0.89800000000000002}]},{"osd":3,"last update":"Wed Aug 9 14:52:11 2023","interfaces":[{"interface":"back","average":{"1min":1.206,"5min":1.331,"15min":1.355},"min":{"1min":0.68200000000000005,"5min":0.68200000000000005,"15min":0.68200000000000005},"max":{"1min":1.4710000000000001,"5min":1.9910000000000001,"15min":1.9910000000000001},"last":1.083},{"interface":"front","average":{"1min":1.333,"5min":1.4410000000000001,"15min":1.4650000000000001},"min":{"1min":0.98299999999999998,"5min":0.98299999999999998,"15min":0.98299999999999998},"max":{"1min":1.6020000000000001,"5min":1.887,"15min":2.375},"last":1.5629999999999999}]},{"osd":4,"last update":"Wed Aug 9 14:52:11 2023","interfaces":[{"interface":"back","average":{"1min":1.401,"5min":1.3129999999999999,"15min":1.3200000000000001},"min":{"1min":0.93400000000000005,"5min":0.82499999999999996,"15min":0.79800000000000004},"max":{"1min":2.0510000000000002,"5min":2.0510000000000002,"15min":2.254},"last":1.2609999999999999},{"interface":"front","average":{"1min":1.341,"5min":1.3959999999999999,"15min":1.387},"min":{"1min":0.92900000000000005,"5min":0.90400000000000003,"15min":0.90400000000000003},"max":{"1min":1.6799999999999999,"5min":1.859,"15min":2.1789999999999998},"last":0.96999999999999997}]},{"osd":5,"last update":"Wed Aug 9 14:52:11 2023","interfaces":[{"interface":"back","average":{"1min":1.47,"5min":1.4930000000000001,"15min":1.4750000000000001},"min":{"1min":1.0820000000000001,"5min":0.85599999999999998,"15min":0.76900000000000002},"max":{"1min":1.7789999999999999,"5min":1.9770000000000001,"15min":2.1589999999999998},"last":1.498},{"interface":"front","average":{"1min":1.5449999999999999,"5min":1.508,"15min":1.5},"min":{"1min":0.998,"5min":0.92400000000000004,"15min":0.92400000000000004},"max":{"1min":2.1400000000000001,"5min":2.1949999999999998,"15min":2.4510000000000001},"last":1.6040000000000001}]},{"osd":6,"last update":"Wed Aug 9 14:52:11 2023","interfaces":[{"interface":"back","average":{"1min":1.7190000000000001,"5min":1.6359999999999999,"15min":1.615},"min":{"1min":0.97099999999999997,"5min":0.97099999999999997,"15min":0.97099999999999997},"max":{"1min":3.0680000000000001,"5min":3.0680000000000001,"15min":3.0680000000000001},"last":1.4419999999999999},{"interface":"front","average":{"1min":1.7310000000000001,"5min":1.579,"15min":1.5620000000000001},"min":{"1min":1.254,"5min":0.98499999999999999,"15min":0.84899999999999998},"max":{"1min":4.516,"5min":4.516,"15min":4.516},"last":1.3140000000000001}]}]},{"osd":6,"up_from":13,"seq":55834575787,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":59612,"kb_used_data":2256,"kb_used_omap":1,"kb_used_meta":57342,"kb_avail":94312228,"statfs":{"total":96636764160,"available":96575721472,"internally_reserved":0,"allocated":2310144,"data_stored":1248917,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1781,"internal_metadata":58718475},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Aug 9 14:52:22 2023","interfaces":[{"interface":"back","average":{"1min":1.3260000000000001,"5min":1.2170000000000001,"15min":1.214},"min":{"1min":0.85699999999999998,"5min":0.75,"15min":0.75},"max":{"1min":1.73,"5min":1.73,"15min":2.278},"last":1.2050000000000001},{"interface":"front","average":{"1min":1.089,"5min":1.224,"15min":1.242},"min":{"1min":0.72899999999999998,"5min":0.72899999999999998,"15min":0.72899999999999998},"max":{"1min":1.732,"5min":2.2170000000000001,"15min":2.2170000000000001},"last":1.3080000000000001}]},{"osd":1,"last update":"Wed Aug 9 14:52:22 2023","interfaces":[{"interface":"back","average":{"1min":1.2629999999999999,"5min":1.246,"15min":1.2310000000000001},"min":{"1min":0.89900000000000002,"5min":0.88400000000000001,"15min":0.88400000000000001},"max":{"1min":2.0139999999999998,"5min":2.0139999999999998,"15min":2.0139999999999998},"last":1.1160000000000001},{"interface":"front","average":{"1min":1.2789999999999999,"5min":1.2629999999999999,"15min":1.2929999999999999},"min":{"1min":0.76500000000000001,"5min":0.76500000000000001,"15min":0.76500000000000001},"max":{"1min":1.9099999999999999,"5min":1.9099999999999999,"15min":1.9099999999999999},"last":1.712}]},{"osd":2,"last update":"Wed Aug 9 14:52:22 2023","interfaces":[{"interface":"back","average":{"1min":1.7090000000000001,"5min":1.421,"15min":1.3779999999999999},"min":{"1min":0.95799999999999996,"5min":0.94399999999999995,"15min":0.88100000000000001},"max":{"1min":4.9560000000000004,"5min":4.9560000000000004,"15min":4.9560000000000004},"last":1.8420000000000001},{"interface":"front","average":{"1min":1.669,"5min":1.4359999999999999,"15min":1.353},"min":{"1min":1.109,"5min":0.95999999999999996,"15min":0.90900000000000003},"max":{"1min":4.0140000000000002,"5min":4.0140000000000002,"15min":4.0140000000000002},"last":1.6579999999999999}]},{"osd":3,"last update":"Wed Aug 9 14:52:22 2023","interfaces":[{"interface":"back","average":{"1min":1.5149999999999999,"5min":1.446,"15min":1.419},"min":{"1min":0.90800000000000003,"5min":0.90800000000000003,"15min":0.90800000000000003},"max":{"1min":1.913,"5min":1.913,"15min":1.913},"last":1.786},{"interface":"front","average":{"1min":1.466,"5min":1.4279999999999999,"15min":1.3919999999999999},"min":{"1min":1.0369999999999999,"5min":0.97199999999999998,"15min":0.93300000000000005},"max":{"1min":2.1520000000000001,"5min":2.1520000000000001,"15min":2.1520000000000001},"last":1.6040000000000001}]},{"osd":4,"last update":"Wed Aug 9 14:52:22 2023","interfaces":[{"interface":"back","average":{"1min":1.5309999999999999,"5min":1.3839999999999999,"15min":1.3140000000000001},"min":{"1min":0.85399999999999998,"5min":0.82299999999999995,"15min":0.82199999999999995},"max":{"1min":2.2040000000000002,"5min":2.2040000000000002,"15min":2.2040000000000002},"last":2.0350000000000001},{"interface":"front","average":{"1min":1.6759999999999999,"5min":1.4990000000000001,"15min":1.3899999999999999},"min":{"1min":1.2030000000000001,"5min":0.871,"15min":0.71599999999999997},"max":{"1min":2.0430000000000001,"5min":2.0430000000000001,"15min":2.0430000000000001},"last":2.2280000000000002}]},{"osd":5,"last update":"Wed Aug 9 14:52:22 2023","interfaces":[{"interface":"back","average":{"1min":1.667,"5min":1.476,"15min":1.387},"min":{"1min":1.0900000000000001,"5min":0.83599999999999997,"15min":0.83599999999999997},"max":{"1min":2.2549999999999999,"5min":2.2549999999999999,"15min":2.2549999999999999},"last":1.9279999999999999},{"interface":"front","average":{"1min":1.7330000000000001,"5min":1.6519999999999999,"15min":1.6060000000000001},"min":{"1min":1.21,"5min":1.21,"15min":0.93500000000000005},"max":{"1min":2.3140000000000001,"5min":2.3140000000000001,"15min":2.3140000000000001},"last":2.1059999999999999}]},{"osd":7,"last update":"Wed Aug 9 14:52:22 2023","interfaces":[{"interface":"back","average":{"1min":1.798,"5min":1.7030000000000001,"15min":1.6120000000000001},"min":{"1min":1.409,"5min":1.175,"15min":0.96299999999999997},"max":{"1min":2.153,"5min":2.476,"15min":2.5310000000000001},"last":2.145},{"interface":"front","average":{"1min":1.851,"5min":1.6779999999999999,"15min":1.605},"min":{"1min":1.4119999999999999,"5min":1.0549999999999999,"15min":0.96999999999999997},"max":{"1min":2.3500000000000001,"5min":2.3500000000000001,"15min":2.3500000000000001},"last":2.0600000000000001}]}]},{"osd":5,"up_from":13,"seq":55834575787,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":59604,"kb_used_data":2244,"kb_used_omap":1,"kb_used_meta":57342,"kb_avail":94312236,"statfs":{"total":96636764160,"available":96575729664,"internally_reserved":0,"allocated":2297856,"data_stored":1242764,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1781,"internal_metadata":58718475},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Aug 9 14:52:14 2023","interfaces":[{"interface":"back","average":{"1min":1.2210000000000001,"5min":1.1830000000000001,"15min":1.1950000000000001},"min":{"1min":0.83399999999999996,"5min":0.72599999999999998,"15min":0.72599999999999998},"max":{"1min":1.714,"5min":1.792,"15min":1.976},"last":0.95399999999999996},{"interface":"front","average":{"1min":1.1679999999999999,"5min":1.2430000000000001,"15min":1.234},"min":{"1min":0.81000000000000005,"5min":0.81000000000000005,"15min":0.81000000000000005},"max":{"1min":1.79,"5min":1.9470000000000001,"15min":1.9470000000000001},"last":1.0169999999999999}]},{"osd":1,"last update":"Wed Aug 9 14:52:14 2023","interfaces":[{"interface":"back","average":{"1min":1.167,"5min":1.2769999999999999,"15min":1.258},"min":{"1min":0.85799999999999998,"5min":0.85799999999999998,"15min":0.82199999999999995},"max":{"1min":1.512,"5min":1.7929999999999999,"15min":1.7929999999999999},"last":1.071},{"interface":"front","average":{"1min":1.258,"5min":1.292,"15min":1.3200000000000001},"min":{"1min":0.72699999999999998,"5min":0.72699999999999998,"15min":0.72699999999999998},"max":{"1min":1.704,"5min":1.704,"15min":1.8480000000000001},"last":1.262}]},{"osd":2,"last update":"Wed Aug 9 14:52:14 2023","interfaces":[{"interface":"back","average":{"1min":1.2909999999999999,"5min":1.343,"15min":1.331},"min":{"1min":0.875,"5min":0.875,"15min":0.86199999999999999},"max":{"1min":1.7829999999999999,"5min":1.7829999999999999,"15min":1.8939999999999999},"last":1.393},{"interface":"front","average":{"1min":1.3120000000000001,"5min":1.355,"15min":1.3520000000000001},"min":{"1min":0.96699999999999997,"5min":0.96699999999999997,"15min":0.88800000000000001},"max":{"1min":1.7050000000000001,"5min":1.8300000000000001,"15min":1.8540000000000001},"last":1.444}]},{"osd":3,"last update":"Wed Aug 9 14:52:14 2023","interfaces":[{"interface":"back","average":{"1min":1.369,"5min":1.375,"15min":1.381},"min":{"1min":0.71899999999999997,"5min":0.71899999999999997,"15min":0.71899999999999997},"max":{"1min":1.794,"5min":1.794,"15min":2.04},"last":1.655},{"interface":"front","average":{"1min":1.504,"5min":1.4970000000000001,"15min":1.474},"min":{"1min":1.034,"5min":1.0009999999999999,"15min":0.98299999999999998},"max":{"1min":1.9710000000000001,"5min":1.9710000000000001,"15min":2.0409999999999999},"last":1.5600000000000001}]},{"osd":4,"last update":"Wed Aug 9 14:52:14 2023","interfaces":[{"interface":"back","average":{"1min":1.3999999999999999,"5min":1.349,"15min":1.3180000000000001},"min":{"1min":0.68300000000000005,"5min":0.68300000000000005,"15min":0.68300000000000005},"max":{"1min":2.2839999999999998,"5min":2.2839999999999998,"15min":2.2839999999999998},"last":1.329},{"interface":"front","average":{"1min":1.601,"5min":1.4450000000000001,"15min":1.4359999999999999},"min":{"1min":0.97199999999999998,"5min":0.94799999999999995,"15min":0.84499999999999997},"max":{"1min":2.2450000000000001,"5min":2.2450000000000001,"15min":2.2450000000000001},"last":1.796}]},{"osd":6,"last update":"Wed Aug 9 14:52:14 2023","interfaces":[{"interface":"back","average":{"1min":1.6879999999999999,"5min":1.4610000000000001,"15min":1.4410000000000001},"min":{"1min":1.1279999999999999,"5min":0.88600000000000001,"15min":0.79300000000000004},"max":{"1min":3.194,"5min":3.194,"15min":3.194},"last":1.744},{"interface":"front","average":{"1min":1.5860000000000001,"5min":1.502,"15min":1.4890000000000001},"min":{"1min":1.163,"5min":1.0169999999999999,"15min":0.871},"max":{"1min":2.1739999999999999,"5min":2.1739999999999999,"15min":2.1739999999999999},"last":1.621}]},{"osd":7,"last update":"Wed Aug 9 14:52:14 2023","interfaces":[{"interface":"back","average":{"1min":1.804,"5min":1.591,"15min":1.595},"min":{"1min":1.3420000000000001,"5min":1.054,"15min":0.98899999999999999},"max":{"1min":2.6150000000000002,"5min":2.6150000000000002,"15min":2.6150000000000002},"last":1.869},{"interface":"front","average":{"1min":1.8759999999999999,"5min":1.6719999999999999,"15min":1.6559999999999999},"min":{"1min":1.345,"5min":1.008,"15min":0.95099999999999996},"max":{"1min":3.6480000000000001,"5min":3.6480000000000001,"15min":3.6480000000000001},"last":1.9510000000000001}]}]},{"osd":4,"up_from":13,"seq":55834575787,"num_pgs":17,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":59604,"kb_used_data":2244,"kb_used_omap":1,"kb_used_meta":57342,"kb_avail":94312236,"statfs":{"total":96636764160,"available":96575729664,"internally_reserved":0,"allocated":2297856,"data_stored":1242764,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1781,"internal_metadata":58718475},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Aug 9 14:51:38 2023","interfaces":[{"interface":"back","average":{"1min":1.1859999999999999,"5min":1.2070000000000001,"15min":1.2050000000000001},"min":{"1min":0.69999999999999996,"5min":0.69999999999999996,"15min":0.69999999999999996},"max":{"1min":2.573,"5min":2.573,"15min":2.573},"last":1.548},{"interface":"front","average":{"1min":1.3460000000000001,"5min":1.341,"15min":1.3260000000000001},"min":{"1min":0.83799999999999997,"5min":0.83799999999999997,"15min":0.54000000000000004},"max":{"1min":2.1960000000000002,"5min":2.1960000000000002,"15min":2.1960000000000002},"last":1.8220000000000001}]},{"osd":1,"last update":"Wed Aug 9 14:51:38 2023","interfaces":[{"interface":"back","average":{"1min":1.49,"5min":1.3799999999999999,"15min":1.3540000000000001},"min":{"1min":0.86599999999999999,"5min":0.86599999999999999,"15min":0.60599999999999998},"max":{"1min":4.4299999999999997,"5min":4.4299999999999997,"15min":4.4299999999999997},"last":1.415},{"interface":"front","average":{"1min":1.204,"5min":1.292,"15min":1.2849999999999999},"min":{"1min":0.68200000000000005,"5min":0.68200000000000005,"15min":0.68200000000000005},"max":{"1min":2.4630000000000001,"5min":2.4630000000000001,"15min":2.4630000000000001},"last":1.0700000000000001}]},{"osd":2,"last update":"Wed Aug 9 14:51:38 2023","interfaces":[{"interface":"back","average":{"1min":1.335,"5min":1.3260000000000001,"15min":1.3109999999999999},"min":{"1min":0.81499999999999995,"5min":0.81499999999999995,"15min":0.74199999999999999},"max":{"1min":2.6920000000000002,"5min":2.6920000000000002,"15min":2.6920000000000002},"last":1.794},{"interface":"front","average":{"1min":1.462,"5min":1.454,"15min":1.4530000000000001},"min":{"1min":0.70899999999999996,"5min":0.70899999999999996,"15min":0.70899999999999996},"max":{"1min":2.0830000000000002,"5min":2.1619999999999999,"15min":2.4359999999999999},"last":1.141}]},{"osd":3,"last update":"Wed Aug 9 14:51:38 2023","interfaces":[{"interface":"back","average":{"1min":1.369,"5min":1.3360000000000001,"15min":1.339},"min":{"1min":0.85699999999999998,"5min":0.85699999999999998,"15min":0.85699999999999998},"max":{"1min":2.2069999999999999,"5min":2.2069999999999999,"15min":2.6429999999999998},"last":1.764},{"interface":"front","average":{"1min":1.534,"5min":1.4970000000000001,"15min":1.4750000000000001},"min":{"1min":0.75,"5min":0.75,"15min":0.75},"max":{"1min":3.0600000000000001,"5min":3.0600000000000001,"15min":3.0600000000000001},"last":1.365}]},{"osd":5,"last update":"Wed Aug 9 14:51:38 2023","interfaces":[{"interface":"back","average":{"1min":1.462,"5min":1.4099999999999999,"15min":1.377},"min":{"1min":0.77100000000000002,"5min":0.77100000000000002,"15min":0.70399999999999996},"max":{"1min":2.0710000000000002,"5min":2.5009999999999999,"15min":2.5009999999999999},"last":1.659},{"interface":"front","average":{"1min":1.343,"5min":1.403,"15min":1.385},"min":{"1min":0.56999999999999995,"5min":0.56999999999999995,"15min":0.56999999999999995},"max":{"1min":1.9790000000000001,"5min":2.4129999999999998,"15min":3.0110000000000001},"last":1.607}]},{"osd":6,"last update":"Wed Aug 9 14:51:38 2023","interfaces":[{"interface":"back","average":{"1min":1.589,"5min":1.4470000000000001,"15min":1.431},"min":{"1min":0.93200000000000005,"5min":0.85499999999999998,"15min":0.79500000000000004},"max":{"1min":2.9540000000000002,"5min":2.9540000000000002,"15min":2.9540000000000002},"last":1.208},{"interface":"front","average":{"1min":1.6739999999999999,"5min":1.619,"15min":1.5680000000000001},"min":{"1min":0.59899999999999998,"5min":0.59899999999999998,"15min":0.59899999999999998},"max":{"1min":2.0529999999999999,"5min":2.3260000000000001,"15min":2.5310000000000001},"last":1.2949999999999999}]},{"osd":7,"last update":"Wed Aug 9 14:51:38 2023","interfaces":[{"interface":"back","average":{"1min":1.5629999999999999,"5min":1.5569999999999999,"15min":1.5229999999999999},"min":{"1min":0.79100000000000004,"5min":0.79100000000000004,"15min":0.79100000000000004},"max":{"1min":2.1360000000000001,"5min":2.331,"15min":3.3149999999999999},"last":1.4650000000000001},{"interface":"front","average":{"1min":1.75,"5min":1.6339999999999999,"15min":1.587},"min":{"1min":0.88400000000000001,"5min":0.88400000000000001,"15min":0.82599999999999996},"max":{"1min":3.274,"5min":3.274,"15min":3.274},"last":1.7190000000000001}]}]},{"osd":3,"up_from":13,"seq":55834575788,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":59604,"kb_used_data":2244,"kb_used_omap":1,"kb_used_meta":57342,"kb_avail":94312236,"statfs":{"total":96636764160,"available":96575729664,"internally_reserved":0,"allocated":2297856,"data_stored":1242764,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1781,"internal_metadata":58718475},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Aug 9 14:52:20 2023","interfaces":[{"interface":"back","average":{"1min":0.86099999999999999,"5min":0.93799999999999994,"15min":0.94699999999999995},"min":{"1min":0.35699999999999998,"5min":0.35699999999999998,"15min":0.35699999999999998},"max":{"1min":1.26,"5min":1.722,"15min":1.8740000000000001},"last":0.70299999999999996},{"interface":"front","average":{"1min":0.91900000000000004,"5min":1.0820000000000001,"15min":1.1339999999999999},"min":{"1min":0.50600000000000001,"5min":0.50600000000000001,"15min":0.50600000000000001},"max":{"1min":1.613,"5min":1.9179999999999999,"15min":1.9179999999999999},"last":1.2889999999999999}]},{"osd":1,"last update":"Wed Aug 9 14:52:20 2023","interfaces":[{"interface":"back","average":{"1min":0.88900000000000001,"5min":1.0329999999999999,"15min":1.0640000000000001},"min":{"1min":0.50600000000000001,"5min":0.50600000000000001,"15min":0.50600000000000001},"max":{"1min":1.405,"5min":1.7050000000000001,"15min":1.903},"last":0.93300000000000005},{"interface":"front","average":{"1min":0.997,"5min":1.194,"15min":1.2090000000000001},"min":{"1min":0.58199999999999996,"5min":0.58199999999999996,"15min":0.58199999999999996},"max":{"1min":1.4630000000000001,"5min":1.8320000000000001,"15min":1.903},"last":1.0609999999999999}]},{"osd":2,"last update":"Wed Aug 9 14:52:20 2023","interfaces":[{"interface":"back","average":{"1min":1.038,"5min":1.3029999999999999,"15min":1.319},"min":{"1min":0.64400000000000002,"5min":0.64400000000000002,"15min":0.64400000000000002},"max":{"1min":1.4770000000000001,"5min":3.7770000000000001,"15min":3.7770000000000001},"last":1.2190000000000001},{"interface":"front","average":{"1min":1.1679999999999999,"5min":1.373,"15min":1.3999999999999999},"min":{"1min":0.58999999999999997,"5min":0.58999999999999997,"15min":0.58999999999999997},"max":{"1min":2.117,"5min":3.5750000000000002,"15min":3.5750000000000002},"last":1.581}]},{"osd":4,"last update":"Wed Aug 9 14:52:20 2023","interfaces":[{"interface":"back","average":{"1min":1.2869999999999999,"5min":1.3600000000000001,"15min":1.385},"min":{"1min":0.82699999999999996,"5min":0.82699999999999996,"15min":0.82699999999999996},"max":{"1min":1.6619999999999999,"5min":1.879,"15min":1.974},"last":0.97499999999999998},{"interface":"front","average":{"1min":1.2769999999999999,"5min":1.415,"15min":1.458},"min":{"1min":0.81499999999999995,"5min":0.79300000000000004,"15min":0.79300000000000004},"max":{"1min":1.613,"5min":1.992,"15min":2.3439999999999999},"last":1.361}]},{"osd":5,"last update":"Wed Aug 9 14:52:20 2023","interfaces":[{"interface":"back","average":{"1min":1.482,"5min":1.5349999999999999,"15min":1.554},"min":{"1min":0.746,"5min":0.746,"15min":0.746},"max":{"1min":1.9299999999999999,"5min":2.0339999999999998,"15min":2.2629999999999999},"last":1.48},{"interface":"front","average":{"1min":1.444,"5min":1.5289999999999999,"15min":1.55},"min":{"1min":0.97899999999999998,"5min":0.97899999999999998,"15min":0.97899999999999998},"max":{"1min":2.0539999999999998,"5min":2.3399999999999999,"15min":2.4630000000000001},"last":1.4139999999999999}]},{"osd":6,"last update":"Wed Aug 9 14:52:20 2023","interfaces":[{"interface":"back","average":{"1min":1.365,"5min":1.5169999999999999,"15min":1.524},"min":{"1min":0.69199999999999995,"5min":0.69199999999999995,"15min":0.69199999999999995},"max":{"1min":1.97,"5min":2.0790000000000002,"15min":2.1659999999999999},"last":1.5409999999999999},{"interface":"front","average":{"1min":1.4950000000000001,"5min":1.6120000000000001,"15min":1.631},"min":{"1min":0.84999999999999998,"5min":0.84999999999999998,"15min":0.84999999999999998},"max":{"1min":1.972,"5min":2.3140000000000001,"15min":2.4140000000000001},"last":1.629}]},{"osd":7,"last update":"Wed Aug 9 14:52:20 2023","interfaces":[{"interface":"back","average":{"1min":1.573,"5min":1.669,"15min":1.6850000000000001},"min":{"1min":1.032,"5min":1.032,"15min":1.032},"max":{"1min":2.0209999999999999,"5min":2.427,"15min":2.5430000000000001},"last":1.698},{"interface":"front","average":{"1min":1.492,"5min":1.679,"15min":1.667},"min":{"1min":1.0389999999999999,"5min":1.0389999999999999,"15min":1.0389999999999999},"max":{"1min":1.9790000000000001,"5min":2.2069999999999999,"15min":2.2069999999999999},"last":1.7250000000000001}]}]},{"osd":2,"up_from":13,"seq":55834575787,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":41548,"kb_used_data":2244,"kb_used_omap":1,"kb_used_meta":39294,"kb_avail":94330292,"statfs":{"total":96636764160,"available":96594219008,"internally_reserved":0,"allocated":2297856,"data_stored":1242764,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1781,"internal_metadata":40237323},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Aug 9 14:52:13 2023","interfaces":[{"interface":"back","average":{"1min":0.91700000000000004,"5min":0.91300000000000003,"15min":0.92300000000000004},"min":{"1min":0.46300000000000002,"5min":0.46300000000000002,"15min":0.46300000000000002},"max":{"1min":1.984,"5min":1.984,"15min":1.984},"last":1.3759999999999999},{"interface":"front","average":{"1min":0.94899999999999995,"5min":1.0660000000000001,"15min":1.101},"min":{"1min":0.41399999999999998,"5min":0.41399999999999998,"15min":0.41399999999999998},"max":{"1min":1.4359999999999999,"5min":1.8300000000000001,"15min":1.8740000000000001},"last":1.238}]},{"osd":1,"last update":"Wed Aug 9 14:52:13 2023","interfaces":[{"interface":"back","average":{"1min":0.997,"5min":1.143,"15min":1.2050000000000001},"min":{"1min":0.60099999999999998,"5min":0.59399999999999997,"15min":0.59399999999999997},"max":{"1min":1.573,"5min":1.6739999999999999,"15min":2.1600000000000001},"last":1.0840000000000001},{"interface":"front","average":{"1min":1.0620000000000001,"5min":1.3620000000000001,"15min":1.421},"min":{"1min":0.58199999999999996,"5min":0.58199999999999996,"15min":0.58199999999999996},"max":{"1min":1.74,"5min":2.0350000000000001,"15min":2.2200000000000002},"last":1.054}]},{"osd":3,"last update":"Wed Aug 9 14:52:13 2023","interfaces":[{"interface":"back","average":{"1min":0.93400000000000005,"5min":1.056,"15min":1.111},"min":{"1min":0.46000000000000002,"5min":0.46000000000000002,"15min":0.46000000000000002},"max":{"1min":1.472,"5min":1.7350000000000001,"15min":1.9239999999999999},"last":0.83399999999999996},{"interface":"front","average":{"1min":1.1839999999999999,"5min":1.345,"15min":1.3660000000000001},"min":{"1min":0.64500000000000002,"5min":0.64500000000000002,"15min":0.64500000000000002},"max":{"1min":1.877,"5min":2.0299999999999998,"15min":2.0299999999999998},"last":1.5189999999999999}]},{"osd":4,"last update":"Wed Aug 9 14:52:13 2023","interfaces":[{"interface":"back","average":{"1min":1.347,"5min":1.4910000000000001,"15min":1.508},"min":{"1min":0.89600000000000002,"5min":0.89600000000000002,"15min":0.89600000000000002},"max":{"1min":2.5750000000000002,"5min":2.5750000000000002,"15min":2.5750000000000002},"last":1.5840000000000001},{"interface":"front","average":{"1min":1.306,"5min":1.4390000000000001,"15min":1.4710000000000001},"min":{"1min":0.85799999999999998,"5min":0.85599999999999998,"15min":0.85599999999999998},"max":{"1min":2.1459999999999999,"5min":2.2509999999999999,"15min":2.2509999999999999},"last":1.3400000000000001}]},{"osd":5,"last update":"Wed Aug 9 14:52:13 2023","interfaces":[{"interface":"back","average":{"1min":1.4370000000000001,"5min":1.538,"15min":1.5389999999999999},"min":{"1min":0.97099999999999997,"5min":0.90100000000000002,"15min":0.90100000000000002},"max":{"1min":2.3919999999999999,"5min":2.3919999999999999,"15min":2.3919999999999999},"last":1.4390000000000001},{"interface":"front","average":{"1min":1.323,"5min":1.4530000000000001,"15min":1.472},"min":{"1min":0.85999999999999999,"5min":0.85999999999999999,"15min":0.85999999999999999},"max":{"1min":2.202,"5min":2.202,"15min":2.202},"last":1.1839999999999999}]},{"osd":6,"last update":"Wed Aug 9 14:52:13 2023","interfaces":[{"interface":"back","average":{"1min":1.421,"5min":1.51,"15min":1.508},"min":{"1min":0.67800000000000005,"5min":0.67800000000000005,"15min":0.67800000000000005},"max":{"1min":2.5289999999999999,"5min":2.5289999999999999,"15min":2.5289999999999999},"last":1.2929999999999999},{"interface":"front","average":{"1min":1.518,"5min":1.6000000000000001,"15min":1.615},"min":{"1min":1.012,"5min":0.98699999999999999,"15min":0.98699999999999999},"max":{"1min":2.448,"5min":2.448,"15min":2.5070000000000001},"last":1.871}]},{"osd":7,"last update":"Wed Aug 9 14:52:13 2023","interfaces":[{"interface":"back","average":{"1min":1.4710000000000001,"5min":1.6479999999999999,"15min":1.6519999999999999},"min":{"1min":0.82899999999999996,"5min":0.82899999999999996,"15min":0.82899999999999996},"max":{"1min":2.258,"5min":2.419,"15min":2.419},"last":1.5600000000000001},{"interface":"front","average":{"1min":1.52,"5min":1.615,"15min":1.6180000000000001},"min":{"1min":0.98499999999999999,"5min":0.98499999999999999,"15min":0.98499999999999999},"max":{"1min":2.3290000000000002,"5min":2.3290000000000002,"15min":2.3399999999999999},"last":1.407}]}]},{"osd":1,"up_from":13,"seq":55834575787,"num_pgs":19,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":51992,"kb_used_data":2824,"kb_used_omap":1,"kb_used_meta":49150,"kb_avail":94319848,"statfs":{"total":96636764160,"available":96583524352,"internally_reserved":0,"allocated":2891776,"data_stored":1833132,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1781,"internal_metadata":50329867},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Aug 9 14:52:26 2023","interfaces":[{"interface":"back","average":{"1min":0.96499999999999997,"5min":0.995,"15min":0.999},"min":{"1min":0.39000000000000001,"5min":0.308,"15min":0.308},"max":{"1min":3.169,"5min":3.169,"15min":3.169},"last":0.42399999999999999},{"interface":"front","average":{"1min":1.0029999999999999,"5min":1.135,"15min":1.1259999999999999},"min":{"1min":0.50600000000000001,"5min":0.50600000000000001,"15min":0.50600000000000001},"max":{"1min":2.895,"5min":2.895,"15min":2.895},"last":0.57799999999999996}]},{"osd":2,"last update":"Wed Aug 9 14:52:26 2023","interfaces":[{"interface":"back","average":{"1min":0.94699999999999995,"5min":1.0780000000000001,"15min":1.079},"min":{"1min":0.38200000000000001,"5min":0.38200000000000001,"15min":0.38200000000000001},"max":{"1min":1.298,"5min":2.71,"15min":2.71},"last":0.60099999999999998},{"interface":"front","average":{"1min":1.1659999999999999,"5min":1.238,"15min":1.2589999999999999},"min":{"1min":0.46300000000000002,"5min":0.46300000000000002,"15min":0.46300000000000002},"max":{"1min":3.2919999999999998,"5min":3.2919999999999998,"15min":3.2919999999999998},"last":0.67900000000000005}]},{"osd":3,"last update":"Wed Aug 9 14:52:26 2023","interfaces":[{"interface":"back","average":{"1min":1.298,"5min":1.383,"15min":1.397},"min":{"1min":0.53400000000000003,"5min":0.53400000000000003,"15min":0.53400000000000003},"max":{"1min":3.5790000000000002,"5min":3.5790000000000002,"15min":3.5790000000000002},"last":0.629},{"interface":"front","average":{"1min":1.3169999999999999,"5min":1.3759999999999999,"15min":1.3720000000000001},"min":{"1min":0.56999999999999995,"5min":0.56999999999999995,"15min":0.56999999999999995},"max":{"1min":3.4580000000000002,"5min":3.4580000000000002,"15min":3.4580000000000002},"last":0.54100000000000004}]},{"osd":4,"last update":"Wed Aug 9 14:52:26 2023","interfaces":[{"interface":"back","average":{"1min":1.425,"5min":1.504,"15min":1.476},"min":{"1min":0.73399999999999999,"5min":0.73399999999999999,"15min":0.73399999999999999},"max":{"1min":2.6949999999999998,"5min":2.6949999999999998,"15min":2.6949999999999998},"last":0.751},{"interface":"front","average":{"1min":1.3500000000000001,"5min":1.452,"15min":1.431},"min":{"1min":0.71899999999999997,"5min":0.71899999999999997,"15min":0.71899999999999997},"max":{"1min":1.855,"5min":3.2440000000000002,"15min":3.2440000000000002},"last":0.94599999999999995}]},{"osd":5,"last update":"Wed Aug 9 14:52:26 2023","interfaces":[{"interface":"back","average":{"1min":1.3799999999999999,"5min":1.4610000000000001,"15min":1.4510000000000001},"min":{"1min":0.61499999999999999,"5min":0.61499999999999999,"15min":0.61499999999999999},"max":{"1min":1.9830000000000001,"5min":3.036,"15min":3.036},"last":0.86099999999999999},{"interface":"front","average":{"1min":1.5960000000000001,"5min":1.611,"15min":1.601},"min":{"1min":0.70799999999999996,"5min":0.70799999999999996,"15min":0.70799999999999996},"max":{"1min":3.6930000000000001,"5min":3.6930000000000001,"15min":3.6930000000000001},"last":0.78700000000000003}]},{"osd":6,"last update":"Wed Aug 9 14:52:26 2023","interfaces":[{"interface":"back","average":{"1min":1.601,"5min":1.625,"15min":1.6080000000000001},"min":{"1min":0.78000000000000003,"5min":0.78000000000000003,"15min":0.78000000000000003},"max":{"1min":2.5750000000000002,"5min":3.149,"15min":3.149},"last":0.997},{"interface":"front","average":{"1min":1.48,"5min":1.585,"15min":1.5900000000000001},"min":{"1min":0.51800000000000002,"5min":0.51800000000000002,"15min":0.51800000000000002},"max":{"1min":2.2189999999999999,"5min":3.4460000000000002,"15min":3.4460000000000002},"last":0.91000000000000003}]},{"osd":7,"last update":"Wed Aug 9 14:52:26 2023","interfaces":[{"interface":"back","average":{"1min":1.708,"5min":1.6879999999999999,"15min":1.694},"min":{"1min":0.90900000000000003,"5min":0.79000000000000004,"15min":0.79000000000000004},"max":{"1min":3.835,"5min":3.835,"15min":3.835},"last":0.92700000000000005},{"interface":"front","average":{"1min":1.6870000000000001,"5min":1.738,"15min":1.728},"min":{"1min":0.88700000000000001,"5min":0.88700000000000001,"15min":0.88700000000000001},"max":{"1min":2.8029999999999999,"5min":2.8029999999999999,"15min":2.8029999999999999},"last":1.1459999999999999}]}]},{"osd":0,"up_from":13,"seq":55834575787,"num_pgs":17,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":29256,"kb_used_data":2240,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94342584,"statfs":{"total":96636764160,"available":96606806016,"internally_reserved":0,"allocated":2293760,"data_stored":1241403,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1781,"internal_metadata":27654411},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Wed Aug 9 14:52:11 2023","interfaces":[{"interface":"back","average":{"1min":0.78700000000000003,"5min":0.91300000000000003,"15min":0.96099999999999997},"min":{"1min":0.38500000000000001,"5min":0.38500000000000001,"15min":0.38500000000000001},"max":{"1min":1.444,"5min":1.456,"15min":1.893},"last":0.60299999999999998},{"interface":"front","average":{"1min":0.93799999999999994,"5min":1.1579999999999999,"15min":1.1399999999999999},"min":{"1min":0.40999999999999998,"5min":0.40999999999999998,"15min":0.40999999999999998},"max":{"1min":1.4930000000000001,"5min":1.8520000000000001,"15min":1.8520000000000001},"last":0.44}]},{"osd":2,"last update":"Wed Aug 9 14:52:11 2023","interfaces":[{"interface":"back","average":{"1min":1.0329999999999999,"5min":1.157,"15min":1.2050000000000001},"min":{"1min":0.54300000000000004,"5min":0.54300000000000004,"15min":0.54300000000000004},"max":{"1min":1.5700000000000001,"5min":1.6910000000000001,"15min":2.698},"last":0.66500000000000004},{"interface":"front","average":{"1min":1.0309999999999999,"5min":1.2270000000000001,"15min":1.2250000000000001},"min":{"1min":0.42699999999999999,"5min":0.42699999999999999,"15min":0.42699999999999999},"max":{"1min":1.649,"5min":1.895,"15min":1.895},"last":0.67800000000000005}]},{"osd":3,"last update":"Wed Aug 9 14:52:11 2023","interfaces":[{"interface":"back","average":{"1min":0.89000000000000001,"5min":1.1719999999999999,"15min":1.2230000000000001},"min":{"1min":0.47299999999999998,"5min":0.47299999999999998,"15min":0.47299999999999998},"max":{"1min":1.5249999999999999,"5min":2.0049999999999999,"15min":2.1230000000000002},"last":0.627},{"interface":"front","average":{"1min":1.1839999999999999,"5min":1.349,"15min":1.393},"min":{"1min":0.44600000000000001,"5min":0.44600000000000001,"15min":0.44600000000000001},"max":{"1min":1.728,"5min":1.829,"15min":2.3580000000000001},"last":0.71599999999999997}]},{"osd":4,"last update":"Wed Aug 9 14:52:11 2023","interfaces":[{"interface":"back","average":{"1min":1.3,"5min":1.387,"15min":1.3799999999999999},"min":{"1min":0.52800000000000002,"5min":0.52800000000000002,"15min":0.52800000000000002},"max":{"1min":1.893,"5min":2.3730000000000002,"15min":2.3730000000000002},"last":0.92900000000000005},{"interface":"front","average":{"1min":1.3089999999999999,"5min":1.4770000000000001,"15min":1.51},"min":{"1min":0.58699999999999997,"5min":0.58699999999999997,"15min":0.58699999999999997},"max":{"1min":1.843,"5min":2.125,"15min":2.149},"last":0.86799999999999999}]},{"osd":5,"last update":"Wed Aug 9 14:52:11 2023","interfaces":[{"interface":"back","average":{"1min":1.3340000000000001,"5min":1.512,"15min":1.52},"min":{"1min":0.55700000000000005,"5min":0.55700000000000005,"15min":0.55700000000000005},"max":{"1min":1.901,"5min":2.0110000000000001,"15min":2.097},"last":1.0149999999999999},{"interface":"front","average":{"1min":1.4179999999999999,"5min":1.4650000000000001,"15min":1.504},"min":{"1min":0.60699999999999998,"5min":0.60699999999999998,"15min":0.60699999999999998},"max":{"1min":2.0099999999999998,"5min":2.2320000000000002,"15min":2.266},"last":0.88400000000000001}]},{"osd":6,"last update":"Wed Aug 9 14:52:11 2023","interfaces":[{"interface":"back","average":{"1min":1.456,"5min":1.6080000000000001,"15min":1.639},"min":{"1min":0.63900000000000001,"5min":0.63900000000000001,"15min":0.63900000000000001},"max":{"1min":1.956,"5min":2.4260000000000002,"15min":3.0750000000000002},"last":0.97699999999999998},{"interface":"front","average":{"1min":1.377,"5min":1.6000000000000001,"15min":1.6180000000000001},"min":{"1min":0.67200000000000004,"5min":0.67200000000000004,"15min":0.67200000000000004},"max":{"1min":2.0800000000000001,"5min":2.2429999999999999,"15min":2.2429999999999999},"last":0.84499999999999997}]},{"osd":7,"last update":"Wed Aug 9 14:52:11 2023","interfaces":[{"interface":"back","average":{"1min":1.5129999999999999,"5min":1.6910000000000001,"15min":1.7010000000000001},"min":{"1min":0.70299999999999996,"5min":0.70299999999999996,"15min":0.70299999999999996},"max":{"1min":1.984,"5min":2.1339999999999999,"15min":2.3490000000000002},"last":0.90900000000000003},{"interface":"front","average":{"1min":1.4590000000000001,"5min":1.597,"15min":1.6100000000000001},"min":{"1min":0.65500000000000003,"5min":0.65500000000000003,"15min":0.65500000000000003},"max":{"1min":2.3940000000000001,"5min":2.4500000000000002,"15min":2.468},"last":0.999}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":22,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":22,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":34,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":22,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":22,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":34,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":22,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1682,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":22,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":604,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":22,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":22,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":22,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":2354,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":23,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":23,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":23,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":23,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":23,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":23,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":23,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":23,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-08-09T14:52:35.294 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-08-09T14:52:35.531 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:35.531+0000 7f78c3f0f640 1 Processor -- start 2023-08-09T14:52:35.532 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:35.531+0000 7f78c3f0f640 1 -- start start 2023-08-09T14:52:35.533 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:35.532+0000 7f78c3f0f640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f78bc0b2790 0x7f78bc0d2ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T14:52:35.533 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:35.533+0000 7f78c3f0f640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f78bc0b3050 0x7f78bc0b3f60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T14:52:35.533 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:35.533+0000 7f78c3f0f640 1 -- --> v1:172.21.15.203:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f78bc0ce090 con 0x7f78bc0b2b70 2023-08-09T14:52:35.534 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:35.533+0000 7f78c3f0f640 1 -- --> v2:172.21.15.43:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f78bc0b3950 con 0x7f78bc0b3050 2023-08-09T14:52:35.534 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:35.533+0000 7f78c3f0f640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f78bc0cd7f0 con 0x7f78bc0b2790 2023-08-09T14:52:35.534 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:35.533+0000 7f78c24ac640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f78bc0b3050 0x7f78bc0b3f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T14:52:35.535 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:35.533+0000 7f78c24ac640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f78bc0b3050 0x7f78bc0b3f60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:36296/0 (socket says 172.21.15.43:36296) 2023-08-09T14:52:35.535 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:35.533+0000 7f78c24ac640 1 -- 172.21.15.43:0/256893688 learned_addr learned my addr 172.21.15.43:0/256893688 (peer_addr_for_me v2:172.21.15.43:0/0) 2023-08-09T14:52:35.535 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:35.534+0000 7f78c1cab640 1 --2- 172.21.15.43:0/256893688 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f78bc0b2790 0x7f78bc0d2ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T14:52:35.535 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:35.534+0000 7f78c24ac640 1 -- 172.21.15.43:0/256893688 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f78bc0b2790 msgr2=0x7f78bc0d2ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-09T14:52:35.535 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:35.534+0000 7f78c24ac640 1 --2- 172.21.15.43:0/256893688 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f78bc0b2790 0x7f78bc0d2ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T14:52:35.535 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:35.534+0000 7f78c24ac640 1 -- 172.21.15.43:0/256893688 >> v1:172.21.15.203:6789/0 conn(0x7f78bc0b2b70 legacy=0x7f78bc0b5330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-09T14:52:35.535 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:35.534+0000 7f78c24ac640 1 -- 172.21.15.43:0/256893688 --> v2:172.21.15.43:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f78bc0cdad0 con 0x7f78bc0b3050 2023-08-09T14:52:35.536 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:35.534+0000 7f78c1cab640 1 --2- 172.21.15.43:0/256893688 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f78bc0b2790 0x7f78bc0d2ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-08-09T14:52:35.537 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:35.535+0000 7f78c24ac640 1 --2- 172.21.15.43:0/256893688 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f78bc0b3050 0x7f78bc0b3f60 secure :-1 s=READY pgs=2046 cs=0 l=1 rev1=1 crypto rx=0x7f78b4009870 tx=0x7f78b402b8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=42efc13a092fd406 server_cookie=0 in_seq=0 out_seq=0 2023-08-09T14:52:35.537 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:35.535+0000 7f78c0ca9640 1 -- 172.21.15.43:0/256893688 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f78b4031640 con 0x7f78bc0b3050 2023-08-09T14:52:35.537 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:35.535+0000 7f78c0ca9640 1 -- 172.21.15.43:0/256893688 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f78b40317a0 con 0x7f78bc0b3050 2023-08-09T14:52:35.537 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:35.535+0000 7f78c0ca9640 1 -- 172.21.15.43:0/256893688 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f78b4031b30 con 0x7f78bc0b3050 2023-08-09T14:52:35.537 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:35.536+0000 7f78c3f0f640 1 -- 172.21.15.43:0/256893688 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f78bc0b3050 msgr2=0x7f78bc0b3f60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T14:52:35.537 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:35.536+0000 7f78c3f0f640 1 --2- 172.21.15.43:0/256893688 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f78bc0b3050 0x7f78bc0b3f60 secure :-1 s=READY pgs=2046 cs=0 l=1 rev1=1 crypto rx=0x7f78b4009870 tx=0x7f78b402b8f0 comp rx=0 tx=0).stop 2023-08-09T14:52:35.538 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:35.536+0000 7f78c3f0f640 1 -- 172.21.15.43:0/256893688 shutdown_connections 2023-08-09T14:52:35.538 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:35.536+0000 7f78c3f0f640 1 --2- 172.21.15.43:0/256893688 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f78bc0b3050 0x7f78bc0b3f60 unknown :-1 s=CLOSED pgs=2046 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T14:52:35.538 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:35.536+0000 7f78c3f0f640 1 --2- 172.21.15.43:0/256893688 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f78bc0b2790 0x7f78bc0d2ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T14:52:35.538 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:35.536+0000 7f78c3f0f640 1 -- 172.21.15.43:0/256893688 >> 172.21.15.43:0/256893688 conn(0x7f78bc09b7b0 msgr2=0x7f78bc0d24e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-09T14:52:35.538 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:35.536+0000 7f78c3f0f640 1 -- 172.21.15.43:0/256893688 shutdown_connections 2023-08-09T14:52:35.538 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:35.536+0000 7f78c3f0f640 1 -- 172.21.15.43:0/256893688 wait complete. 2023-08-09T14:52:35.539 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:35.538+0000 7f78c3f0f640 1 Processor -- start 2023-08-09T14:52:35.539 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:35.538+0000 7f78c3f0f640 1 -- start start 2023-08-09T14:52:35.540 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:35.539+0000 7f78c3f0f640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f78bc0b2790 0x7f78bc0cb290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T14:52:35.541 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:35.539+0000 7f78c3f0f640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f78bc0b2b70 0x7f78bc0cb7d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T14:52:35.541 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:35.539+0000 7f78c3f0f640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f78bc0b3050 0x7f78bc0c7870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T14:52:35.541 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:35.539+0000 7f78c3f0f640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f78bc0cbd10 con 0x7f78bc0b2790 2023-08-09T14:52:35.541 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:35.539+0000 7f78c3f0f640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f78bc0c67d0 con 0x7f78bc0b3050 2023-08-09T14:52:35.541 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:35.539+0000 7f78c3f0f640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f78bc0c6ab0 con 0x7f78bc0b2b70 2023-08-09T14:52:35.542 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:35.540+0000 7f78c1cab640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f78bc0b2790 0x7f78bc0cb290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T14:52:35.542 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:35.540+0000 7f78c1cab640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f78bc0b2790 0x7f78bc0cb290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:36304/0 (socket says 172.21.15.43:36304) 2023-08-09T14:52:35.542 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:35.540+0000 7f78c1cab640 1 -- 172.21.15.43:0/719529690 learned_addr learned my addr 172.21.15.43:0/719529690 (peer_addr_for_me v2:172.21.15.43:0/0) 2023-08-09T14:52:35.542 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:35.540+0000 7f78c14aa640 1 --2- 172.21.15.43:0/719529690 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f78bc0b2b70 0x7f78bc0cb7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T14:52:35.542 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:35.540+0000 7f78c24ac640 1 --2- 172.21.15.43:0/719529690 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f78bc0b3050 0x7f78bc0c7870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T14:52:35.542 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:35.540+0000 7f78c1cab640 1 -- 172.21.15.43:0/719529690 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f78bc0b2b70 msgr2=0x7f78bc0cb7d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T14:52:35.542 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:35.540+0000 7f78c1cab640 1 --2- 172.21.15.43:0/719529690 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f78bc0b2b70 0x7f78bc0cb7d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T14:52:35.543 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:35.540+0000 7f78c1cab640 1 -- 172.21.15.43:0/719529690 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f78bc0b3050 msgr2=0x7f78bc0c7870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T14:52:35.543 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:35.540+0000 7f78c1cab640 1 --2- 172.21.15.43:0/719529690 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f78bc0b3050 0x7f78bc0c7870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T14:52:35.543 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:35.540+0000 7f78c1cab640 1 -- 172.21.15.43:0/719529690 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f78b40094d0 con 0x7f78bc0b2790 2023-08-09T14:52:35.543 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:35.540+0000 7f78c1cab640 1 --2- 172.21.15.43:0/719529690 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f78bc0b2790 0x7f78bc0cb290 secure :-1 s=READY pgs=2047 cs=0 l=1 rev1=1 crypto rx=0x7f78ac00ce00 tx=0x7f78ac00a650 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-09T14:52:35.544 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:35.541+0000 7f78ba7fc640 1 -- 172.21.15.43:0/719529690 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f78ac00f890 con 0x7f78bc0b2790 2023-08-09T14:52:35.544 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:35.541+0000 7f78ba7fc640 1 -- 172.21.15.43:0/719529690 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f78ac00f9f0 con 0x7f78bc0b2790 2023-08-09T14:52:35.544 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:35.541+0000 7f78ba7fc640 1 -- 172.21.15.43:0/719529690 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f78ac00fdc0 con 0x7f78bc0b2790 2023-08-09T14:52:35.544 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:35.541+0000 7f78c3f0f640 1 -- 172.21.15.43:0/719529690 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f78bc0c6d90 con 0x7f78bc0b2790 2023-08-09T14:52:35.544 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:35.541+0000 7f78c3f0f640 1 -- 172.21.15.43:0/719529690 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f78bc0c7350 con 0x7f78bc0b2790 2023-08-09T14:52:35.545 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:35.542+0000 7f78c3f0f640 1 -- 172.21.15.43:0/719529690 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f78bc0c7510 con 0x7f78bc0b2790 2023-08-09T14:52:35.545 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:35.543+0000 7f78ba7fc640 1 -- 172.21.15.43:0/719529690 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 88) v1 ==== 88443+0+0 (secure 0 0 0) 0x7f78ac00b040 con 0x7f78bc0b2790 2023-08-09T14:52:35.545 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:35.544+0000 7f78ba7fc640 1 --2- 172.21.15.43:0/719529690 >> [v2:172.21.15.203:6832/2959400181,v1:172.21.15.203:6833/2959400181] conn(0x7f78a806a170 0x7f78a806c630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T14:52:35.545 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:35.544+0000 7f78ba7fc640 1 -- 172.21.15.43:0/719529690 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(186..186 src has 1..186) v4 ==== 6426+0+0 (secure 0 0 0) 0x7f78ac089410 con 0x7f78bc0b2790 2023-08-09T14:52:35.546 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:35.545+0000 7f78c14aa640 1 --2- 172.21.15.43:0/719529690 >> [v2:172.21.15.203:6832/2959400181,v1:172.21.15.203:6833/2959400181] conn(0x7f78a806a170 0x7f78a806c630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T14:52:35.546 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:35.546+0000 7f78c14aa640 1 --2- 172.21.15.43:0/719529690 >> [v2:172.21.15.203:6832/2959400181,v1:172.21.15.203:6833/2959400181] conn(0x7f78a806a170 0x7f78a806c630 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f78b0006fd0 tx=0x7f78b0008040 comp rx=0 tx=0).ready entity=mgr.7223 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-09T14:52:35.549 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:35.549+0000 7f78ba7fc640 1 -- 172.21.15.43:0/719529690 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178066 (secure 0 0 0) 0x7f78ac0546d0 con 0x7f78bc0b2790 2023-08-09T14:52:35.654 INFO:tasks.ceph.mds.c.smithi043.stderr:2023-08-09T14:52:35.653+0000 7fdbfc635640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2023-08-09T14:52:35.739 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:35.738+0000 7f78c3f0f640 1 -- 172.21.15.43:0/719529690 --> [v2:172.21.15.203:6832/2959400181,v1:172.21.15.203:6833/2959400181] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f78bc0c6c20 con 0x7f78a806a170 2023-08-09T14:52:35.763 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:35.762+0000 7f78ba7fc640 1 -- 172.21.15.43:0/719529690 <== mgr.7223 v2:172.21.15.203:6832/2959400181 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+309550 (secure 0 0 0) 0x7f78bc0c6c20 con 0x7f78a806a170 2023-08-09T14:52:35.766 INFO:teuthology.orchestra.run.smithi043.stdout: 2023-08-09T14:52:35.768 INFO:teuthology.orchestra.run.smithi043.stderr:dumped all 2023-08-09T14:52:35.769 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:35.768+0000 7f78c3f0f640 1 -- 172.21.15.43:0/719529690 >> [v2:172.21.15.203:6832/2959400181,v1:172.21.15.203:6833/2959400181] conn(0x7f78a806a170 msgr2=0x7f78a806c630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T14:52:35.770 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:35.768+0000 7f78c3f0f640 1 --2- 172.21.15.43:0/719529690 >> [v2:172.21.15.203:6832/2959400181,v1:172.21.15.203:6833/2959400181] conn(0x7f78a806a170 0x7f78a806c630 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f78b0006fd0 tx=0x7f78b0008040 comp rx=0 tx=0).stop 2023-08-09T14:52:35.770 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:35.768+0000 7f78c3f0f640 1 -- 172.21.15.43:0/719529690 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f78bc0b2790 msgr2=0x7f78bc0cb290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T14:52:35.770 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:35.768+0000 7f78c3f0f640 1 --2- 172.21.15.43:0/719529690 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f78bc0b2790 0x7f78bc0cb290 secure :-1 s=READY pgs=2047 cs=0 l=1 rev1=1 crypto rx=0x7f78ac00ce00 tx=0x7f78ac00a650 comp rx=0 tx=0).stop 2023-08-09T14:52:35.770 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:35.768+0000 7f78c3f0f640 1 -- 172.21.15.43:0/719529690 shutdown_connections 2023-08-09T14:52:35.770 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:35.768+0000 7f78c3f0f640 1 --2- 172.21.15.43:0/719529690 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f78bc0b3050 0x7f78bc0c7870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T14:52:35.770 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:35.768+0000 7f78c3f0f640 1 --2- 172.21.15.43:0/719529690 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f78bc0b2b70 0x7f78bc0cb7d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T14:52:35.770 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:35.768+0000 7f78c3f0f640 1 --2- 172.21.15.43:0/719529690 >> [v2:172.21.15.203:6832/2959400181,v1:172.21.15.203:6833/2959400181] conn(0x7f78a806a170 0x7f78a806c630 unknown :-1 s=CLOSED pgs=67 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T14:52:35.771 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:35.768+0000 7f78c3f0f640 1 --2- 172.21.15.43:0/719529690 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f78bc0b2790 0x7f78bc0cb290 unknown :-1 s=CLOSED pgs=2047 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T14:52:35.771 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:35.768+0000 7f78c3f0f640 1 -- 172.21.15.43:0/719529690 >> 172.21.15.43:0/719529690 conn(0x7f78bc09b7b0 msgr2=0x7f78bc0b5100 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-09T14:52:35.771 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:35.768+0000 7f78c3f0f640 1 -- 172.21.15.43:0/719529690 shutdown_connections 2023-08-09T14:52:35.771 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:35.768+0000 7f78c3f0f640 1 -- 172.21.15.43:0/719529690 wait complete. 2023-08-09T14:52:35.791 INFO:teuthology.orchestra.run.smithi043.stdout:{"pg_ready":true,"pg_map":{"version":27,"stamp":"2023-08-09T14:52:34.100798+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":23,"num_object_clones":0,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":0,"num_read":1483,"num_read_kb":3066,"num_write":487,"num_write_kb":2430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":312,"ondisk_log_size":312,"up":210,"acting":210,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":69,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":409128,"kb_used_data":19132,"kb_used_omap":13,"kb_used_meta":389874,"kb_avail":754565592,"statfs":{"total":773094113280,"available":772675166208,"internally_reserved":0,"allocated":19591168,"data_stored":11135132,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":14248,"internal_metadata":399231064},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":34,"apply_latency_ms":34,"commit_latency_ns":34000000,"apply_latency_ns":34000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"5.432769"},"pg_stats":[{"pgid":"23.3f","version":"0'0","reported_seq":9,"reported_epoch":185,"state":"active+clean","last_fresh":"2023-08-09T14:52:30.761556+0000","last_change":"2023-08-09T14:52:30.753912+0000","last_active":"2023-08-09T14:52:30.761556+0000","last_peered":"2023-08-09T14:52:30.761556+0000","last_clean":"2023-08-09T14:52:30.761556+0000","last_became_active":"2023-08-09T14:52:30.753644+0000","last_became_peered":"2023-08-09T14:52:30.753644+0000","last_unstale":"2023-08-09T14:52:30.761556+0000","last_undegraded":"2023-08-09T14:52:30.761556+0000","last_fullsized":"2023-08-09T14:52:30.761556+0000","mapping_epoch":184,"log_start":"0'0","ondisk_log_start":"0'0","created":184,"last_epoch_clean":185,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T14:52:29.665328+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T14:52:29.665328+0000","last_clean_scrub_stamp":"2023-08-09T14:52:29.665328+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T23:36:56.791787+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"23.3e","version":"0'0","reported_seq":9,"reported_epoch":185,"state":"active+clean","last_fresh":"2023-08-09T14:52:30.770118+0000","last_change":"2023-08-09T14:52:30.763355+0000","last_active":"2023-08-09T14:52:30.770118+0000","last_peered":"2023-08-09T14:52:30.770118+0000","last_clean":"2023-08-09T14:52:30.770118+0000","last_became_active":"2023-08-09T14:52:30.762723+0000","last_became_peered":"2023-08-09T14:52:30.762723+0000","last_unstale":"2023-08-09T14:52:30.770118+0000","last_undegraded":"2023-08-09T14:52:30.770118+0000","last_fullsized":"2023-08-09T14:52:30.770118+0000","mapping_epoch":184,"log_start":"0'0","ondisk_log_start":"0'0","created":184,"last_epoch_clean":185,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T14:52:29.665328+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T14:52:29.665328+0000","last_clean_scrub_stamp":"2023-08-09T14:52:29.665328+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-11T01:02:04.697698+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"23.3d","version":"0'0","reported_seq":9,"reported_epoch":185,"state":"active+clean","last_fresh":"2023-08-09T14:52:30.763514+0000","last_change":"2023-08-09T14:52:30.758091+0000","last_active":"2023-08-09T14:52:30.763514+0000","last_peered":"2023-08-09T14:52:30.763514+0000","last_clean":"2023-08-09T14:52:30.763514+0000","last_became_active":"2023-08-09T14:52:30.757610+0000","last_became_peered":"2023-08-09T14:52:30.757610+0000","last_unstale":"2023-08-09T14:52:30.763514+0000","last_undegraded":"2023-08-09T14:52:30.763514+0000","last_fullsized":"2023-08-09T14:52:30.763514+0000","mapping_epoch":184,"log_start":"0'0","ondisk_log_start":"0'0","created":184,"last_epoch_clean":185,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T14:52:29.665328+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T14:52:29.665328+0000","last_clean_scrub_stamp":"2023-08-09T14:52:29.665328+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T23:49:55.143349+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"23.3c","version":"0'0","reported_seq":9,"reported_epoch":185,"state":"active+clean","last_fresh":"2023-08-09T14:52:30.766489+0000","last_change":"2023-08-09T14:52:30.760459+0000","last_active":"2023-08-09T14:52:30.766489+0000","last_peered":"2023-08-09T14:52:30.766489+0000","last_clean":"2023-08-09T14:52:30.766489+0000","last_became_active":"2023-08-09T14:52:30.760019+0000","last_became_peered":"2023-08-09T14:52:30.760019+0000","last_unstale":"2023-08-09T14:52:30.766489+0000","last_undegraded":"2023-08-09T14:52:30.766489+0000","last_fullsized":"2023-08-09T14:52:30.766489+0000","mapping_epoch":184,"log_start":"0'0","ondisk_log_start":"0'0","created":184,"last_epoch_clean":185,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T14:52:29.665328+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T14:52:29.665328+0000","last_clean_scrub_stamp":"2023-08-09T14:52:29.665328+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T17:50:23.747187+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"23.3b","version":"0'0","reported_seq":9,"reported_epoch":185,"state":"active+clean","last_fresh":"2023-08-09T14:52:30.769487+0000","last_change":"2023-08-09T14:52:30.762742+0000","last_active":"2023-08-09T14:52:30.769487+0000","last_peered":"2023-08-09T14:52:30.769487+0000","last_clean":"2023-08-09T14:52:30.769487+0000","last_became_active":"2023-08-09T14:52:30.762436+0000","last_became_peered":"2023-08-09T14:52:30.762436+0000","last_unstale":"2023-08-09T14:52:30.769487+0000","last_undegraded":"2023-08-09T14:52:30.769487+0000","last_fullsized":"2023-08-09T14:52:30.769487+0000","mapping_epoch":184,"log_start":"0'0","ondisk_log_start":"0'0","created":184,"last_epoch_clean":185,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T14:52:29.665328+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T14:52:29.665328+0000","last_clean_scrub_stamp":"2023-08-09T14:52:29.665328+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T17:44:11.621343+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"23.3a","version":"0'0","reported_seq":9,"reported_epoch":185,"state":"active+clean","last_fresh":"2023-08-09T14:52:30.761488+0000","last_change":"2023-08-09T14:52:30.753787+0000","last_active":"2023-08-09T14:52:30.761488+0000","last_peered":"2023-08-09T14:52:30.761488+0000","last_clean":"2023-08-09T14:52:30.761488+0000","last_became_active":"2023-08-09T14:52:30.753510+0000","last_became_peered":"2023-08-09T14:52:30.753510+0000","last_unstale":"2023-08-09T14:52:30.761488+0000","last_undegraded":"2023-08-09T14:52:30.761488+0000","last_fullsized":"2023-08-09T14:52:30.761488+0000","mapping_epoch":184,"log_start":"0'0","ondisk_log_start":"0'0","created":184,"last_epoch_clean":185,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T14:52:29.665328+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T14:52:29.665328+0000","last_clean_scrub_stamp":"2023-08-09T14:52:29.665328+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-11T00:26:08.275980+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"23.39","version":"0'0","reported_seq":9,"reported_epoch":185,"state":"active+clean","last_fresh":"2023-08-09T14:52:30.748065+0000","last_change":"2023-08-09T14:52:30.742190+0000","last_active":"2023-08-09T14:52:30.748065+0000","last_peered":"2023-08-09T14:52:30.748065+0000","last_clean":"2023-08-09T14:52:30.748065+0000","last_became_active":"2023-08-09T14:52:30.741907+0000","last_became_peered":"2023-08-09T14:52:30.741907+0000","last_unstale":"2023-08-09T14:52:30.748065+0000","last_undegraded":"2023-08-09T14:52:30.748065+0000","last_fullsized":"2023-08-09T14:52:30.748065+0000","mapping_epoch":184,"log_start":"0'0","ondisk_log_start":"0'0","created":184,"last_epoch_clean":185,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T14:52:29.665328+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T14:52:29.665328+0000","last_clean_scrub_stamp":"2023-08-09T14:52:29.665328+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T21:49:39.468201+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"23.38","version":"0'0","reported_seq":9,"reported_epoch":185,"state":"active+clean","last_fresh":"2023-08-09T14:52:30.762473+0000","last_change":"2023-08-09T14:52:30.753012+0000","last_active":"2023-08-09T14:52:30.762473+0000","last_peered":"2023-08-09T14:52:30.762473+0000","last_clean":"2023-08-09T14:52:30.762473+0000","last_became_active":"2023-08-09T14:52:30.752563+0000","last_became_peered":"2023-08-09T14:52:30.752563+0000","last_unstale":"2023-08-09T14:52:30.762473+0000","last_undegraded":"2023-08-09T14:52:30.762473+0000","last_fullsized":"2023-08-09T14:52:30.762473+0000","mapping_epoch":184,"log_start":"0'0","ondisk_log_start":"0'0","created":184,"last_epoch_clean":185,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T14:52:29.665328+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T14:52:29.665328+0000","last_clean_scrub_stamp":"2023-08-09T14:52:29.665328+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T18:14:27.098381+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"23.37","version":"0'0","reported_seq":9,"reported_epoch":185,"state":"active+clean","last_fresh":"2023-08-09T14:52:30.769904+0000","last_change":"2023-08-09T14:52:30.764106+0000","last_active":"2023-08-09T14:52:30.769904+0000","last_peered":"2023-08-09T14:52:30.769904+0000","last_clean":"2023-08-09T14:52:30.769904+0000","last_became_active":"2023-08-09T14:52:30.763814+0000","last_became_peered":"2023-08-09T14:52:30.763814+0000","last_unstale":"2023-08-09T14:52:30.769904+0000","last_undegraded":"2023-08-09T14:52:30.769904+0000","last_fullsized":"2023-08-09T14:52:30.769904+0000","mapping_epoch":184,"log_start":"0'0","ondisk_log_start":"0'0","created":184,"last_epoch_clean":185,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T14:52:29.665328+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T14:52:29.665328+0000","last_clean_scrub_stamp":"2023-08-09T14:52:29.665328+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T19:52:04.468524+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"23.36","version":"0'0","reported_seq":9,"reported_epoch":185,"state":"active+clean","last_fresh":"2023-08-09T14:52:30.770142+0000","last_change":"2023-08-09T14:52:30.763562+0000","last_active":"2023-08-09T14:52:30.770142+0000","last_peered":"2023-08-09T14:52:30.770142+0000","last_clean":"2023-08-09T14:52:30.770142+0000","last_became_active":"2023-08-09T14:52:30.763093+0000","last_became_peered":"2023-08-09T14:52:30.763093+0000","last_unstale":"2023-08-09T14:52:30.770142+0000","last_undegraded":"2023-08-09T14:52:30.770142+0000","last_fullsized":"2023-08-09T14:52:30.770142+0000","mapping_epoch":184,"log_start":"0'0","ondisk_log_start":"0'0","created":184,"last_epoch_clean":185,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T14:52:29.665328+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T14:52:29.665328+0000","last_clean_scrub_stamp":"2023-08-09T14:52:29.665328+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T16:35:09.679756+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"23.35","version":"0'0","reported_seq":9,"reported_epoch":185,"state":"active+clean","last_fresh":"2023-08-09T14:52:30.769321+0000","last_change":"2023-08-09T14:52:30.762674+0000","last_active":"2023-08-09T14:52:30.769321+0000","last_peered":"2023-08-09T14:52:30.769321+0000","last_clean":"2023-08-09T14:52:30.769321+0000","last_became_active":"2023-08-09T14:52:30.762378+0000","last_became_peered":"2023-08-09T14:52:30.762378+0000","last_unstale":"2023-08-09T14:52:30.769321+0000","last_undegraded":"2023-08-09T14:52:30.769321+0000","last_fullsized":"2023-08-09T14:52:30.769321+0000","mapping_epoch":184,"log_start":"0'0","ondisk_log_start":"0'0","created":184,"last_epoch_clean":185,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T14:52:29.665328+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T14:52:29.665328+0000","last_clean_scrub_stamp":"2023-08-09T14:52:29.665328+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-11T02:51:20.583446+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"23.34","version":"0'0","reported_seq":9,"reported_epoch":185,"state":"active+clean","last_fresh":"2023-08-09T14:52:30.769647+0000","last_change":"2023-08-09T14:52:30.763378+0000","last_active":"2023-08-09T14:52:30.769647+0000","last_peered":"2023-08-09T14:52:30.769647+0000","last_clean":"2023-08-09T14:52:30.769647+0000","last_became_active":"2023-08-09T14:52:30.763084+0000","last_became_peered":"2023-08-09T14:52:30.763084+0000","last_unstale":"2023-08-09T14:52:30.769647+0000","last_undegraded":"2023-08-09T14:52:30.769647+0000","last_fullsized":"2023-08-09T14:52:30.769647+0000","mapping_epoch":184,"log_start":"0'0","ondisk_log_start":"0'0","created":184,"last_epoch_clean":185,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T14:52:29.665328+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T14:52:29.665328+0000","last_clean_scrub_stamp":"2023-08-09T14:52:29.665328+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T20:56:23.203078+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"23.33","version":"0'0","reported_seq":9,"reported_epoch":185,"state":"active+clean","last_fresh":"2023-08-09T14:52:30.747579+0000","last_change":"2023-08-09T14:52:30.743697+0000","last_active":"2023-08-09T14:52:30.747579+0000","last_peered":"2023-08-09T14:52:30.747579+0000","last_clean":"2023-08-09T14:52:30.747579+0000","last_became_active":"2023-08-09T14:52:30.743303+0000","last_became_peered":"2023-08-09T14:52:30.743303+0000","last_unstale":"2023-08-09T14:52:30.747579+0000","last_undegraded":"2023-08-09T14:52:30.747579+0000","last_fullsized":"2023-08-09T14:52:30.747579+0000","mapping_epoch":184,"log_start":"0'0","ondisk_log_start":"0'0","created":184,"last_epoch_clean":185,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T14:52:29.665328+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T14:52:29.665328+0000","last_clean_scrub_stamp":"2023-08-09T14:52:29.665328+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T18:13:44.615831+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"23.32","version":"0'0","reported_seq":9,"reported_epoch":185,"state":"active+clean","last_fresh":"2023-08-09T14:52:30.761338+0000","last_change":"2023-08-09T14:52:30.757194+0000","last_active":"2023-08-09T14:52:30.761338+0000","last_peered":"2023-08-09T14:52:30.761338+0000","last_clean":"2023-08-09T14:52:30.761338+0000","last_became_active":"2023-08-09T14:52:30.756885+0000","last_became_peered":"2023-08-09T14:52:30.756885+0000","last_unstale":"2023-08-09T14:52:30.761338+0000","last_undegraded":"2023-08-09T14:52:30.761338+0000","last_fullsized":"2023-08-09T14:52:30.761338+0000","mapping_epoch":184,"log_start":"0'0","ondisk_log_start":"0'0","created":184,"last_epoch_clean":185,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T14:52:29.665328+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T14:52:29.665328+0000","last_clean_scrub_stamp":"2023-08-09T14:52:29.665328+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-11T02:14:40.604088+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"23.31","version":"0'0","reported_seq":9,"reported_epoch":185,"state":"active+clean","last_fresh":"2023-08-09T14:52:30.754358+0000","last_change":"2023-08-09T14:52:30.744546+0000","last_active":"2023-08-09T14:52:30.754358+0000","last_peered":"2023-08-09T14:52:30.754358+0000","last_clean":"2023-08-09T14:52:30.754358+0000","last_became_active":"2023-08-09T14:52:30.744270+0000","last_became_peered":"2023-08-09T14:52:30.744270+0000","last_unstale":"2023-08-09T14:52:30.754358+0000","last_undegraded":"2023-08-09T14:52:30.754358+0000","last_fullsized":"2023-08-09T14:52:30.754358+0000","mapping_epoch":184,"log_start":"0'0","ondisk_log_start":"0'0","created":184,"last_epoch_clean":185,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T14:52:29.665328+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T14:52:29.665328+0000","last_clean_scrub_stamp":"2023-08-09T14:52:29.665328+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T16:48:24.213005+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"23.30","version":"0'0","reported_seq":9,"reported_epoch":185,"state":"active+clean","last_fresh":"2023-08-09T14:52:30.757165+0000","last_change":"2023-08-09T14:52:30.750803+0000","last_active":"2023-08-09T14:52:30.757165+0000","last_peered":"2023-08-09T14:52:30.757165+0000","last_clean":"2023-08-09T14:52:30.757165+0000","last_became_active":"2023-08-09T14:52:30.750326+0000","last_became_peered":"2023-08-09T14:52:30.750326+0000","last_unstale":"2023-08-09T14:52:30.757165+0000","last_undegraded":"2023-08-09T14:52:30.757165+0000","last_fullsized":"2023-08-09T14:52:30.757165+0000","mapping_epoch":184,"log_start":"0'0","ondisk_log_start":"0'0","created":184,"last_epoch_clean":185,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T14:52:29.665328+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T14:52:29.665328+0000","last_clean_scrub_stamp":"2023-08-09T14:52:29.665328+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T16:04:49.538552+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"23.2f","version":"0'0","reported_seq":9,"reported_epoch":185,"state":"active+clean","last_fresh":"2023-08-09T14:52:30.761542+0000","last_change":"2023-08-09T14:52:30.752136+0000","last_active":"2023-08-09T14:52:30.761542+0000","last_peered":"2023-08-09T14:52:30.761542+0000","last_clean":"2023-08-09T14:52:30.761542+0000","last_became_active":"2023-08-09T14:52:30.751800+0000","last_became_peered":"2023-08-09T14:52:30.751800+0000","last_unstale":"2023-08-09T14:52:30.761542+0000","last_undegraded":"2023-08-09T14:52:30.761542+0000","last_fullsized":"2023-08-09T14:52:30.761542+0000","mapping_epoch":184,"log_start":"0'0","ondisk_log_start":"0'0","created":184,"last_epoch_clean":185,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T14:52:29.665328+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T14:52:29.665328+0000","last_clean_scrub_stamp":"2023-08-09T14:52:29.665328+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T19:33:48.201185+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"23.2e","version":"0'0","reported_seq":9,"reported_epoch":185,"state":"active+clean","last_fresh":"2023-08-09T14:52:30.749047+0000","last_change":"2023-08-09T14:52:30.746878+0000","last_active":"2023-08-09T14:52:30.749047+0000","last_peered":"2023-08-09T14:52:30.749047+0000","last_clean":"2023-08-09T14:52:30.749047+0000","last_became_active":"2023-08-09T14:52:30.746389+0000","last_became_peered":"2023-08-09T14:52:30.746389+0000","last_unstale":"2023-08-09T14:52:30.749047+0000","last_undegraded":"2023-08-09T14:52:30.749047+0000","last_fullsized":"2023-08-09T14:52:30.749047+0000","mapping_epoch":184,"log_start":"0'0","ondisk_log_start":"0'0","created":184,"last_epoch_clean":185,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T14:52:29.665328+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T14:52:29.665328+0000","last_clean_scrub_stamp":"2023-08-09T14:52:29.665328+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-11T01:08:47.452806+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"23.2d","version":"0'0","reported_seq":9,"reported_epoch":185,"state":"active+clean","last_fresh":"2023-08-09T14:52:30.746114+0000","last_change":"2023-08-09T14:52:30.744109+0000","last_active":"2023-08-09T14:52:30.746114+0000","last_peered":"2023-08-09T14:52:30.746114+0000","last_clean":"2023-08-09T14:52:30.746114+0000","last_became_active":"2023-08-09T14:52:30.743824+0000","last_became_peered":"2023-08-09T14:52:30.743824+0000","last_unstale":"2023-08-09T14:52:30.746114+0000","last_undegraded":"2023-08-09T14:52:30.746114+0000","last_fullsized":"2023-08-09T14:52:30.746114+0000","mapping_epoch":184,"log_start":"0'0","ondisk_log_start":"0'0","created":184,"last_epoch_clean":185,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T14:52:29.665328+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T14:52:29.665328+0000","last_clean_scrub_stamp":"2023-08-09T14:52:29.665328+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T17:47:28.862639+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"23.2c","version":"0'0","reported_seq":9,"reported_epoch":185,"state":"active+clean","last_fresh":"2023-08-09T14:52:30.761365+0000","last_change":"2023-08-09T14:52:30.757201+0000","last_active":"2023-08-09T14:52:30.761365+0000","last_peered":"2023-08-09T14:52:30.761365+0000","last_clean":"2023-08-09T14:52:30.761365+0000","last_became_active":"2023-08-09T14:52:30.756843+0000","last_became_peered":"2023-08-09T14:52:30.756843+0000","last_unstale":"2023-08-09T14:52:30.761365+0000","last_undegraded":"2023-08-09T14:52:30.761365+0000","last_fullsized":"2023-08-09T14:52:30.761365+0000","mapping_epoch":184,"log_start":"0'0","ondisk_log_start":"0'0","created":184,"last_epoch_clean":185,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T14:52:29.665328+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T14:52:29.665328+0000","last_clean_scrub_stamp":"2023-08-09T14:52:29.665328+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T22:17:13.182406+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"23.2b","version":"0'0","reported_seq":9,"reported_epoch":185,"state":"active+clean","last_fresh":"2023-08-09T14:52:30.760468+0000","last_change":"2023-08-09T14:52:30.749933+0000","last_active":"2023-08-09T14:52:30.760468+0000","last_peered":"2023-08-09T14:52:30.760468+0000","last_clean":"2023-08-09T14:52:30.760468+0000","last_became_active":"2023-08-09T14:52:30.749664+0000","last_became_peered":"2023-08-09T14:52:30.749664+0000","last_unstale":"2023-08-09T14:52:30.760468+0000","last_undegraded":"2023-08-09T14:52:30.760468+0000","last_fullsized":"2023-08-09T14:52:30.760468+0000","mapping_epoch":184,"log_start":"0'0","ondisk_log_start":"0'0","created":184,"last_epoch_clean":185,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T14:52:29.665328+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T14:52:29.665328+0000","last_clean_scrub_stamp":"2023-08-09T14:52:29.665328+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T20:26:05.313360+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"23.2a","version":"0'0","reported_seq":9,"reported_epoch":185,"state":"active+clean","last_fresh":"2023-08-09T14:52:30.769863+0000","last_change":"2023-08-09T14:52:30.767768+0000","last_active":"2023-08-09T14:52:30.769863+0000","last_peered":"2023-08-09T14:52:30.769863+0000","last_clean":"2023-08-09T14:52:30.769863+0000","last_became_active":"2023-08-09T14:52:30.766333+0000","last_became_peered":"2023-08-09T14:52:30.766333+0000","last_unstale":"2023-08-09T14:52:30.769863+0000","last_undegraded":"2023-08-09T14:52:30.769863+0000","last_fullsized":"2023-08-09T14:52:30.769863+0000","mapping_epoch":184,"log_start":"0'0","ondisk_log_start":"0'0","created":184,"last_epoch_clean":185,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T14:52:29.665328+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T14:52:29.665328+0000","last_clean_scrub_stamp":"2023-08-09T14:52:29.665328+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-11T02:16:55.127639+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"23.29","version":"0'0","reported_seq":9,"reported_epoch":185,"state":"active+clean","last_fresh":"2023-08-09T14:52:30.765442+0000","last_change":"2023-08-09T14:52:30.763747+0000","last_active":"2023-08-09T14:52:30.765442+0000","last_peered":"2023-08-09T14:52:30.765442+0000","last_clean":"2023-08-09T14:52:30.765442+0000","last_became_active":"2023-08-09T14:52:30.763460+0000","last_became_peered":"2023-08-09T14:52:30.763460+0000","last_unstale":"2023-08-09T14:52:30.765442+0000","last_undegraded":"2023-08-09T14:52:30.765442+0000","last_fullsized":"2023-08-09T14:52:30.765442+0000","mapping_epoch":184,"log_start":"0'0","ondisk_log_start":"0'0","created":184,"last_epoch_clean":185,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T14:52:29.665328+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T14:52:29.665328+0000","last_clean_scrub_stamp":"2023-08-09T14:52:29.665328+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T20:00:10.628893+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"23.28","version":"0'0","reported_seq":9,"reported_epoch":185,"state":"active+clean","last_fresh":"2023-08-09T14:52:30.762451+0000","last_change":"2023-08-09T14:52:30.752836+0000","last_active":"2023-08-09T14:52:30.762451+0000","last_peered":"2023-08-09T14:52:30.762451+0000","last_clean":"2023-08-09T14:52:30.762451+0000","last_became_active":"2023-08-09T14:52:30.752181+0000","last_became_peered":"2023-08-09T14:52:30.752181+0000","last_unstale":"2023-08-09T14:52:30.762451+0000","last_undegraded":"2023-08-09T14:52:30.762451+0000","last_fullsized":"2023-08-09T14:52:30.762451+0000","mapping_epoch":184,"log_start":"0'0","ondisk_log_start":"0'0","created":184,"last_epoch_clean":185,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T14:52:29.665328+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T14:52:29.665328+0000","last_clean_scrub_stamp":"2023-08-09T14:52:29.665328+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-11T00:54:31.624155+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"23.27","version":"0'0","reported_seq":9,"reported_epoch":185,"state":"active+clean","last_fresh":"2023-08-09T14:52:30.761422+0000","last_change":"2023-08-09T14:52:30.756217+0000","last_active":"2023-08-09T14:52:30.761422+0000","last_peered":"2023-08-09T14:52:30.761422+0000","last_clean":"2023-08-09T14:52:30.761422+0000","last_became_active":"2023-08-09T14:52:30.755921+0000","last_became_peered":"2023-08-09T14:52:30.755921+0000","last_unstale":"2023-08-09T14:52:30.761422+0000","last_undegraded":"2023-08-09T14:52:30.761422+0000","last_fullsized":"2023-08-09T14:52:30.761422+0000","mapping_epoch":184,"log_start":"0'0","ondisk_log_start":"0'0","created":184,"last_epoch_clean":185,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T14:52:29.665328+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T14:52:29.665328+0000","last_clean_scrub_stamp":"2023-08-09T14:52:29.665328+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-11T00:21:08.700639+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"23.26","version":"0'0","reported_seq":9,"reported_epoch":185,"state":"active+clean","last_fresh":"2023-08-09T14:52:30.749091+0000","last_change":"2023-08-09T14:52:30.747505+0000","last_active":"2023-08-09T14:52:30.749091+0000","last_peered":"2023-08-09T14:52:30.749091+0000","last_clean":"2023-08-09T14:52:30.749091+0000","last_became_active":"2023-08-09T14:52:30.747196+0000","last_became_peered":"2023-08-09T14:52:30.747196+0000","last_unstale":"2023-08-09T14:52:30.749091+0000","last_undegraded":"2023-08-09T14:52:30.749091+0000","last_fullsized":"2023-08-09T14:52:30.749091+0000","mapping_epoch":184,"log_start":"0'0","ondisk_log_start":"0'0","created":184,"last_epoch_clean":185,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T14:52:29.665328+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T14:52:29.665328+0000","last_clean_scrub_stamp":"2023-08-09T14:52:29.665328+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-11T01:19:24.212841+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"23.25","version":"0'0","reported_seq":9,"reported_epoch":185,"state":"active+clean","last_fresh":"2023-08-09T14:52:30.770421+0000","last_change":"2023-08-09T14:52:30.767487+0000","last_active":"2023-08-09T14:52:30.770421+0000","last_peered":"2023-08-09T14:52:30.770421+0000","last_clean":"2023-08-09T14:52:30.770421+0000","last_became_active":"2023-08-09T14:52:30.767201+0000","last_became_peered":"2023-08-09T14:52:30.767201+0000","last_unstale":"2023-08-09T14:52:30.770421+0000","last_undegraded":"2023-08-09T14:52:30.770421+0000","last_fullsized":"2023-08-09T14:52:30.770421+0000","mapping_epoch":184,"log_start":"0'0","ondisk_log_start":"0'0","created":184,"last_epoch_clean":185,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T14:52:29.665328+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T14:52:29.665328+0000","last_clean_scrub_stamp":"2023-08-09T14:52:29.665328+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-11T00:42:15.257036+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"23.24","version":"0'0","reported_seq":9,"reported_epoch":185,"state":"active+clean","last_fresh":"2023-08-09T14:52:30.754309+0000","last_change":"2023-08-09T14:52:30.744360+0000","last_active":"2023-08-09T14:52:30.754309+0000","last_peered":"2023-08-09T14:52:30.754309+0000","last_clean":"2023-08-09T14:52:30.754309+0000","last_became_active":"2023-08-09T14:52:30.744052+0000","last_became_peered":"2023-08-09T14:52:30.744052+0000","last_unstale":"2023-08-09T14:52:30.754309+0000","last_undegraded":"2023-08-09T14:52:30.754309+0000","last_fullsized":"2023-08-09T14:52:30.754309+0000","mapping_epoch":184,"log_start":"0'0","ondisk_log_start":"0'0","created":184,"last_epoch_clean":185,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T14:52:29.665328+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T14:52:29.665328+0000","last_clean_scrub_stamp":"2023-08-09T14:52:29.665328+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T21:56:25.594544+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"23.23","version":"0'0","reported_seq":9,"reported_epoch":185,"state":"active+clean","last_fresh":"2023-08-09T14:52:30.770461+0000","last_change":"2023-08-09T14:52:30.768224+0000","last_active":"2023-08-09T14:52:30.770461+0000","last_peered":"2023-08-09T14:52:30.770461+0000","last_clean":"2023-08-09T14:52:30.770461+0000","last_became_active":"2023-08-09T14:52:30.767828+0000","last_became_peered":"2023-08-09T14:52:30.767828+0000","last_unstale":"2023-08-09T14:52:30.770461+0000","last_undegraded":"2023-08-09T14:52:30.770461+0000","last_fullsized":"2023-08-09T14:52:30.770461+0000","mapping_epoch":184,"log_start":"0'0","ondisk_log_start":"0'0","created":184,"last_epoch_clean":185,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T14:52:29.665328+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T14:52:29.665328+0000","last_clean_scrub_stamp":"2023-08-09T14:52:29.665328+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T18:38:13.297526+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"23.22","version":"0'0","reported_seq":9,"reported_epoch":185,"state":"active+clean","last_fresh":"2023-08-09T14:52:30.759868+0000","last_change":"2023-08-09T14:52:30.756106+0000","last_active":"2023-08-09T14:52:30.759868+0000","last_peered":"2023-08-09T14:52:30.759868+0000","last_clean":"2023-08-09T14:52:30.759868+0000","last_became_active":"2023-08-09T14:52:30.755144+0000","last_became_peered":"2023-08-09T14:52:30.755144+0000","last_unstale":"2023-08-09T14:52:30.759868+0000","last_undegraded":"2023-08-09T14:52:30.759868+0000","last_fullsized":"2023-08-09T14:52:30.759868+0000","mapping_epoch":184,"log_start":"0'0","ondisk_log_start":"0'0","created":184,"last_epoch_clean":185,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T14:52:29.665328+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T14:52:29.665328+0000","last_clean_scrub_stamp":"2023-08-09T14:52:29.665328+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T19:16:10.586060+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"23.21","version":"0'0","reported_seq":9,"reported_epoch":185,"state":"active+clean","last_fresh":"2023-08-09T14:52:30.759977+0000","last_change":"2023-08-09T14:52:30.756197+0000","last_active":"2023-08-09T14:52:30.759977+0000","last_peered":"2023-08-09T14:52:30.759977+0000","last_clean":"2023-08-09T14:52:30.759977+0000","last_became_active":"2023-08-09T14:52:30.755871+0000","last_became_peered":"2023-08-09T14:52:30.755871+0000","last_unstale":"2023-08-09T14:52:30.759977+0000","last_undegraded":"2023-08-09T14:52:30.759977+0000","last_fullsized":"2023-08-09T14:52:30.759977+0000","mapping_epoch":184,"log_start":"0'0","ondisk_log_start":"0'0","created":184,"last_epoch_clean":185,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T14:52:29.665328+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T14:52:29.665328+0000","last_clean_scrub_stamp":"2023-08-09T14:52:29.665328+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-11T01:31:05.174962+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"23.20","version":"0'0","reported_seq":9,"reported_epoch":185,"state":"active+clean","last_fresh":"2023-08-09T14:52:30.745972+0000","last_change":"2023-08-09T14:52:30.743192+0000","last_active":"2023-08-09T14:52:30.745972+0000","last_peered":"2023-08-09T14:52:30.745972+0000","last_clean":"2023-08-09T14:52:30.745972+0000","last_became_active":"2023-08-09T14:52:30.742836+0000","last_became_peered":"2023-08-09T14:52:30.742836+0000","last_unstale":"2023-08-09T14:52:30.745972+0000","last_undegraded":"2023-08-09T14:52:30.745972+0000","last_fullsized":"2023-08-09T14:52:30.745972+0000","mapping_epoch":184,"log_start":"0'0","ondisk_log_start":"0'0","created":184,"last_epoch_clean":185,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T14:52:29.665328+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T14:52:29.665328+0000","last_clean_scrub_stamp":"2023-08-09T14:52:29.665328+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T22:46:41.082755+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"23.1e","version":"0'0","reported_seq":9,"reported_epoch":185,"state":"active+clean","last_fresh":"2023-08-09T14:52:30.763699+0000","last_change":"2023-08-09T14:52:30.760085+0000","last_active":"2023-08-09T14:52:30.763699+0000","last_peered":"2023-08-09T14:52:30.763699+0000","last_clean":"2023-08-09T14:52:30.763699+0000","last_became_active":"2023-08-09T14:52:30.759756+0000","last_became_peered":"2023-08-09T14:52:30.759756+0000","last_unstale":"2023-08-09T14:52:30.763699+0000","last_undegraded":"2023-08-09T14:52:30.763699+0000","last_fullsized":"2023-08-09T14:52:30.763699+0000","mapping_epoch":184,"log_start":"0'0","ondisk_log_start":"0'0","created":184,"last_epoch_clean":185,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T14:52:29.665328+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T14:52:29.665328+0000","last_clean_scrub_stamp":"2023-08-09T14:52:29.665328+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T15:46:43.388518+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"22.1f","version":"185'4","reported_seq":15,"reported_epoch":185,"state":"active+clean","last_fresh":"2023-08-09T14:52:30.774910+0000","last_change":"2023-08-09T14:52:29.715240+0000","last_active":"2023-08-09T14:52:30.774910+0000","last_peered":"2023-08-09T14:52:30.774910+0000","last_clean":"2023-08-09T14:52:30.774910+0000","last_became_active":"2023-08-09T14:52:29.714931+0000","last_became_peered":"2023-08-09T14:52:29.714931+0000","last_unstale":"2023-08-09T14:52:30.774910+0000","last_undegraded":"2023-08-09T14:52:30.774910+0000","last_fullsized":"2023-08-09T14:52:30.774910+0000","mapping_epoch":183,"log_start":"0'0","ondisk_log_start":"0'0","created":183,"last_epoch_clean":184,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T14:52:28.654282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T14:52:28.654282+0000","last_clean_scrub_stamp":"2023-08-09T14:52:28.654282+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T20:41:46.046693+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"23.1f","version":"0'0","reported_seq":9,"reported_epoch":185,"state":"active+clean","last_fresh":"2023-08-09T14:52:30.747456+0000","last_change":"2023-08-09T14:52:30.744316+0000","last_active":"2023-08-09T14:52:30.747456+0000","last_peered":"2023-08-09T14:52:30.747456+0000","last_clean":"2023-08-09T14:52:30.747456+0000","last_became_active":"2023-08-09T14:52:30.743964+0000","last_became_peered":"2023-08-09T14:52:30.743964+0000","last_unstale":"2023-08-09T14:52:30.747456+0000","last_undegraded":"2023-08-09T14:52:30.747456+0000","last_fullsized":"2023-08-09T14:52:30.747456+0000","mapping_epoch":184,"log_start":"0'0","ondisk_log_start":"0'0","created":184,"last_epoch_clean":185,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T14:52:29.665328+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T14:52:29.665328+0000","last_clean_scrub_stamp":"2023-08-09T14:52:29.665328+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-11T01:27:11.688539+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"22.1e","version":"185'1","reported_seq":12,"reported_epoch":185,"state":"active+clean","last_fresh":"2023-08-09T14:52:30.771037+0000","last_change":"2023-08-09T14:52:29.714217+0000","last_active":"2023-08-09T14:52:30.771037+0000","last_peered":"2023-08-09T14:52:30.771037+0000","last_clean":"2023-08-09T14:52:30.771037+0000","last_became_active":"2023-08-09T14:52:29.713881+0000","last_became_peered":"2023-08-09T14:52:29.713881+0000","last_unstale":"2023-08-09T14:52:30.771037+0000","last_undegraded":"2023-08-09T14:52:30.771037+0000","last_fullsized":"2023-08-09T14:52:30.771037+0000","mapping_epoch":183,"log_start":"0'0","ondisk_log_start":"0'0","created":183,"last_epoch_clean":184,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T14:52:28.654282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T14:52:28.654282+0000","last_clean_scrub_stamp":"2023-08-09T14:52:28.654282+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-11T02:08:41.542781+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"23.1c","version":"0'0","reported_seq":9,"reported_epoch":185,"state":"active+clean","last_fresh":"2023-08-09T14:52:30.756814+0000","last_change":"2023-08-09T14:52:30.745487+0000","last_active":"2023-08-09T14:52:30.756814+0000","last_peered":"2023-08-09T14:52:30.756814+0000","last_clean":"2023-08-09T14:52:30.756814+0000","last_became_active":"2023-08-09T14:52:30.744752+0000","last_became_peered":"2023-08-09T14:52:30.744752+0000","last_unstale":"2023-08-09T14:52:30.756814+0000","last_undegraded":"2023-08-09T14:52:30.756814+0000","last_fullsized":"2023-08-09T14:52:30.756814+0000","mapping_epoch":184,"log_start":"0'0","ondisk_log_start":"0'0","created":184,"last_epoch_clean":185,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T14:52:29.665328+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T14:52:29.665328+0000","last_clean_scrub_stamp":"2023-08-09T14:52:29.665328+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T15:49:51.175691+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"22.1d","version":"185'2","reported_seq":13,"reported_epoch":185,"state":"active+clean","last_fresh":"2023-08-09T14:52:30.772025+0000","last_change":"2023-08-09T14:52:29.712716+0000","last_active":"2023-08-09T14:52:30.772025+0000","last_peered":"2023-08-09T14:52:30.772025+0000","last_clean":"2023-08-09T14:52:30.772025+0000","last_became_active":"2023-08-09T14:52:29.711958+0000","last_became_peered":"2023-08-09T14:52:29.711958+0000","last_unstale":"2023-08-09T14:52:30.772025+0000","last_undegraded":"2023-08-09T14:52:30.772025+0000","last_fullsized":"2023-08-09T14:52:30.772025+0000","mapping_epoch":183,"log_start":"0'0","ondisk_log_start":"0'0","created":183,"last_epoch_clean":184,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T14:52:28.654282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T14:52:28.654282+0000","last_clean_scrub_stamp":"2023-08-09T14:52:28.654282+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-11T02:01:06.667578+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"23.1d","version":"0'0","reported_seq":9,"reported_epoch":185,"state":"active+clean","last_fresh":"2023-08-09T14:52:30.763532+0000","last_change":"2023-08-09T14:52:30.758424+0000","last_active":"2023-08-09T14:52:30.763532+0000","last_peered":"2023-08-09T14:52:30.763532+0000","last_clean":"2023-08-09T14:52:30.763532+0000","last_became_active":"2023-08-09T14:52:30.758145+0000","last_became_peered":"2023-08-09T14:52:30.758145+0000","last_unstale":"2023-08-09T14:52:30.763532+0000","last_undegraded":"2023-08-09T14:52:30.763532+0000","last_fullsized":"2023-08-09T14:52:30.763532+0000","mapping_epoch":184,"log_start":"0'0","ondisk_log_start":"0'0","created":184,"last_epoch_clean":185,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T14:52:29.665328+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T14:52:29.665328+0000","last_clean_scrub_stamp":"2023-08-09T14:52:29.665328+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T17:23:02.106000+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"22.1c","version":"185'2","reported_seq":13,"reported_epoch":185,"state":"active+clean","last_fresh":"2023-08-09T14:52:30.774998+0000","last_change":"2023-08-09T14:52:29.716345+0000","last_active":"2023-08-09T14:52:30.774998+0000","last_peered":"2023-08-09T14:52:30.774998+0000","last_clean":"2023-08-09T14:52:30.774998+0000","last_became_active":"2023-08-09T14:52:29.715940+0000","last_became_peered":"2023-08-09T14:52:29.715940+0000","last_unstale":"2023-08-09T14:52:30.774998+0000","last_undegraded":"2023-08-09T14:52:30.774998+0000","last_fullsized":"2023-08-09T14:52:30.774998+0000","mapping_epoch":183,"log_start":"0'0","ondisk_log_start":"0'0","created":183,"last_epoch_clean":184,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T14:52:28.654282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T14:52:28.654282+0000","last_clean_scrub_stamp":"2023-08-09T14:52:28.654282+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T19:50:22.634529+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"23.1a","version":"0'0","reported_seq":9,"reported_epoch":185,"state":"active+clean","last_fresh":"2023-08-09T14:52:30.756758+0000","last_change":"2023-08-09T14:52:30.752369+0000","last_active":"2023-08-09T14:52:30.756758+0000","last_peered":"2023-08-09T14:52:30.756758+0000","last_clean":"2023-08-09T14:52:30.756758+0000","last_became_active":"2023-08-09T14:52:30.747909+0000","last_became_peered":"2023-08-09T14:52:30.747909+0000","last_unstale":"2023-08-09T14:52:30.756758+0000","last_undegraded":"2023-08-09T14:52:30.756758+0000","last_fullsized":"2023-08-09T14:52:30.756758+0000","mapping_epoch":184,"log_start":"0'0","ondisk_log_start":"0'0","created":184,"last_epoch_clean":185,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T14:52:29.665328+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T14:52:29.665328+0000","last_clean_scrub_stamp":"2023-08-09T14:52:29.665328+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T22:59:22.827689+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"22.1b","version":"185'1","reported_seq":12,"reported_epoch":185,"state":"active+clean","last_fresh":"2023-08-09T14:52:30.768049+0000","last_change":"2023-08-09T14:52:29.704885+0000","last_active":"2023-08-09T14:52:30.768049+0000","last_peered":"2023-08-09T14:52:30.768049+0000","last_clean":"2023-08-09T14:52:30.768049+0000","last_became_active":"2023-08-09T14:52:29.704567+0000","last_became_peered":"2023-08-09T14:52:29.704567+0000","last_unstale":"2023-08-09T14:52:30.768049+0000","last_undegraded":"2023-08-09T14:52:30.768049+0000","last_fullsized":"2023-08-09T14:52:30.768049+0000","mapping_epoch":183,"log_start":"0'0","ondisk_log_start":"0'0","created":183,"last_epoch_clean":184,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T14:52:28.654282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T14:52:28.654282+0000","last_clean_scrub_stamp":"2023-08-09T14:52:28.654282+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-11T00:13:48.503140+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"23.1b","version":"0'0","reported_seq":9,"reported_epoch":185,"state":"active+clean","last_fresh":"2023-08-09T14:52:30.766589+0000","last_change":"2023-08-09T14:52:30.760189+0000","last_active":"2023-08-09T14:52:30.766589+0000","last_peered":"2023-08-09T14:52:30.766589+0000","last_clean":"2023-08-09T14:52:30.766589+0000","last_became_active":"2023-08-09T14:52:30.759907+0000","last_became_peered":"2023-08-09T14:52:30.759907+0000","last_unstale":"2023-08-09T14:52:30.766589+0000","last_undegraded":"2023-08-09T14:52:30.766589+0000","last_fullsized":"2023-08-09T14:52:30.766589+0000","mapping_epoch":184,"log_start":"0'0","ondisk_log_start":"0'0","created":184,"last_epoch_clean":185,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T14:52:29.665328+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T14:52:29.665328+0000","last_clean_scrub_stamp":"2023-08-09T14:52:29.665328+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T23:48:29.448541+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"22.1a","version":"185'1","reported_seq":12,"reported_epoch":185,"state":"active+clean","last_fresh":"2023-08-09T14:52:30.771819+0000","last_change":"2023-08-09T14:52:29.698683+0000","last_active":"2023-08-09T14:52:30.771819+0000","last_peered":"2023-08-09T14:52:30.771819+0000","last_clean":"2023-08-09T14:52:30.771819+0000","last_became_active":"2023-08-09T14:52:29.698300+0000","last_became_peered":"2023-08-09T14:52:29.698300+0000","last_unstale":"2023-08-09T14:52:30.771819+0000","last_undegraded":"2023-08-09T14:52:30.771819+0000","last_fullsized":"2023-08-09T14:52:30.771819+0000","mapping_epoch":183,"log_start":"0'0","ondisk_log_start":"0'0","created":183,"last_epoch_clean":184,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T14:52:28.654282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T14:52:28.654282+0000","last_clean_scrub_stamp":"2023-08-09T14:52:28.654282+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T16:33:30.903318+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"23.18","version":"0'0","reported_seq":9,"reported_epoch":185,"state":"active+clean","last_fresh":"2023-08-09T14:52:30.765180+0000","last_change":"2023-08-09T14:52:30.756817+0000","last_active":"2023-08-09T14:52:30.765180+0000","last_peered":"2023-08-09T14:52:30.765180+0000","last_clean":"2023-08-09T14:52:30.765180+0000","last_became_active":"2023-08-09T14:52:30.756054+0000","last_became_peered":"2023-08-09T14:52:30.756054+0000","last_unstale":"2023-08-09T14:52:30.765180+0000","last_undegraded":"2023-08-09T14:52:30.765180+0000","last_fullsized":"2023-08-09T14:52:30.765180+0000","mapping_epoch":184,"log_start":"0'0","ondisk_log_start":"0'0","created":184,"last_epoch_clean":185,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T14:52:29.665328+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T14:52:29.665328+0000","last_clean_scrub_stamp":"2023-08-09T14:52:29.665328+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T16:48:27.700071+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"22.19","version":"0'0","reported_seq":11,"reported_epoch":185,"state":"active+clean","last_fresh":"2023-08-09T14:52:30.744368+0000","last_change":"2023-08-09T14:52:29.717797+0000","last_active":"2023-08-09T14:52:30.744368+0000","last_peered":"2023-08-09T14:52:30.744368+0000","last_clean":"2023-08-09T14:52:30.744368+0000","last_became_active":"2023-08-09T14:52:29.717365+0000","last_became_peered":"2023-08-09T14:52:29.717365+0000","last_unstale":"2023-08-09T14:52:30.744368+0000","last_undegraded":"2023-08-09T14:52:30.744368+0000","last_fullsized":"2023-08-09T14:52:30.744368+0000","mapping_epoch":183,"log_start":"0'0","ondisk_log_start":"0'0","created":183,"last_epoch_clean":184,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T14:52:28.654282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T14:52:28.654282+0000","last_clean_scrub_stamp":"2023-08-09T14:52:28.654282+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-11T00:38:38.570978+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"23.19","version":"0'0","reported_seq":9,"reported_epoch":185,"state":"active+clean","last_fresh":"2023-08-09T14:52:30.761570+0000","last_change":"2023-08-09T14:52:30.752017+0000","last_active":"2023-08-09T14:52:30.761570+0000","last_peered":"2023-08-09T14:52:30.761570+0000","last_clean":"2023-08-09T14:52:30.761570+0000","last_became_active":"2023-08-09T14:52:30.751744+0000","last_became_peered":"2023-08-09T14:52:30.751744+0000","last_unstale":"2023-08-09T14:52:30.761570+0000","last_undegraded":"2023-08-09T14:52:30.761570+0000","last_fullsized":"2023-08-09T14:52:30.761570+0000","mapping_epoch":184,"log_start":"0'0","ondisk_log_start":"0'0","created":184,"last_epoch_clean":185,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T14:52:29.665328+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T14:52:29.665328+0000","last_clean_scrub_stamp":"2023-08-09T14:52:29.665328+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T21:33:27.199787+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"22.18","version":"0'0","reported_seq":11,"reported_epoch":185,"state":"active+clean","last_fresh":"2023-08-09T14:52:30.721059+0000","last_change":"2023-08-09T14:52:29.705213+0000","last_active":"2023-08-09T14:52:30.721059+0000","last_peered":"2023-08-09T14:52:30.721059+0000","last_clean":"2023-08-09T14:52:30.721059+0000","last_became_active":"2023-08-09T14:52:29.704926+0000","last_became_peered":"2023-08-09T14:52:29.704926+0000","last_unstale":"2023-08-09T14:52:30.721059+0000","last_undegraded":"2023-08-09T14:52:30.721059+0000","last_fullsized":"2023-08-09T14:52:30.721059+0000","mapping_epoch":183,"log_start":"0'0","ondisk_log_start":"0'0","created":183,"last_epoch_clean":184,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T14:52:28.654282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T14:52:28.654282+0000","last_clean_scrub_stamp":"2023-08-09T14:52:28.654282+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T18:47:31.398455+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"23.e","version":"0'0","reported_seq":9,"reported_epoch":185,"state":"active+clean","last_fresh":"2023-08-09T14:52:30.766595+0000","last_change":"2023-08-09T14:52:30.759493+0000","last_active":"2023-08-09T14:52:30.766595+0000","last_peered":"2023-08-09T14:52:30.766595+0000","last_clean":"2023-08-09T14:52:30.766595+0000","last_became_active":"2023-08-09T14:52:30.759170+0000","last_became_peered":"2023-08-09T14:52:30.759170+0000","last_unstale":"2023-08-09T14:52:30.766595+0000","last_undegraded":"2023-08-09T14:52:30.766595+0000","last_fullsized":"2023-08-09T14:52:30.766595+0000","mapping_epoch":184,"log_start":"0'0","ondisk_log_start":"0'0","created":184,"last_epoch_clean":185,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T14:52:29.665328+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T14:52:29.665328+0000","last_clean_scrub_stamp":"2023-08-09T14:52:29.665328+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T16:25:08.863039+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"22.f","version":"185'1","reported_seq":12,"reported_epoch":185,"state":"active+clean","last_fresh":"2023-08-09T14:52:30.769891+0000","last_change":"2023-08-09T14:52:29.711508+0000","last_active":"2023-08-09T14:52:30.769891+0000","last_peered":"2023-08-09T14:52:30.769891+0000","last_clean":"2023-08-09T14:52:30.769891+0000","last_became_active":"2023-08-09T14:52:29.711242+0000","last_became_peered":"2023-08-09T14:52:29.711242+0000","last_unstale":"2023-08-09T14:52:30.769891+0000","last_undegraded":"2023-08-09T14:52:30.769891+0000","last_fullsized":"2023-08-09T14:52:30.769891+0000","mapping_epoch":183,"log_start":"0'0","ondisk_log_start":"0'0","created":183,"last_epoch_clean":184,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T14:52:28.654282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T14:52:28.654282+0000","last_clean_scrub_stamp":"2023-08-09T14:52:28.654282+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-11T02:17:49.407925+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"23.f","version":"0'0","reported_seq":9,"reported_epoch":185,"state":"active+clean","last_fresh":"2023-08-09T14:52:30.757435+0000","last_change":"2023-08-09T14:52:30.742721+0000","last_active":"2023-08-09T14:52:30.757435+0000","last_peered":"2023-08-09T14:52:30.757435+0000","last_clean":"2023-08-09T14:52:30.757435+0000","last_became_active":"2023-08-09T14:52:30.741032+0000","last_became_peered":"2023-08-09T14:52:30.741032+0000","last_unstale":"2023-08-09T14:52:30.757435+0000","last_undegraded":"2023-08-09T14:52:30.757435+0000","last_fullsized":"2023-08-09T14:52:30.757435+0000","mapping_epoch":184,"log_start":"0'0","ondisk_log_start":"0'0","created":184,"last_epoch_clean":185,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T14:52:29.665328+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T14:52:29.665328+0000","last_clean_scrub_stamp":"2023-08-09T14:52:29.665328+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-11T01:15:53.748020+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"22.e","version":"185'2","reported_seq":13,"reported_epoch":185,"state":"active+clean","last_fresh":"2023-08-09T14:52:30.767028+0000","last_change":"2023-08-09T14:52:29.708857+0000","last_active":"2023-08-09T14:52:30.767028+0000","last_peered":"2023-08-09T14:52:30.767028+0000","last_clean":"2023-08-09T14:52:30.767028+0000","last_became_active":"2023-08-09T14:52:29.708531+0000","last_became_peered":"2023-08-09T14:52:29.708531+0000","last_unstale":"2023-08-09T14:52:30.767028+0000","last_undegraded":"2023-08-09T14:52:30.767028+0000","last_fullsized":"2023-08-09T14:52:30.767028+0000","mapping_epoch":183,"log_start":"0'0","ondisk_log_start":"0'0","created":183,"last_epoch_clean":184,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T14:52:28.654282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T14:52:28.654282+0000","last_clean_scrub_stamp":"2023-08-09T14:52:28.654282+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T18:44:32.435595+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"23.c","version":"0'0","reported_seq":9,"reported_epoch":185,"state":"active+clean","last_fresh":"2023-08-09T14:52:30.766515+0000","last_change":"2023-08-09T14:52:30.761119+0000","last_active":"2023-08-09T14:52:30.766515+0000","last_peered":"2023-08-09T14:52:30.766515+0000","last_clean":"2023-08-09T14:52:30.766515+0000","last_became_active":"2023-08-09T14:52:30.760825+0000","last_became_peered":"2023-08-09T14:52:30.760825+0000","last_unstale":"2023-08-09T14:52:30.766515+0000","last_undegraded":"2023-08-09T14:52:30.766515+0000","last_fullsized":"2023-08-09T14:52:30.766515+0000","mapping_epoch":184,"log_start":"0'0","ondisk_log_start":"0'0","created":184,"last_epoch_clean":185,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T14:52:29.665328+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T14:52:29.665328+0000","last_clean_scrub_stamp":"2023-08-09T14:52:29.665328+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-11T01:18:15.509351+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"22.d","version":"185'1","reported_seq":12,"reported_epoch":185,"state":"active+clean","last_fresh":"2023-08-09T14:52:30.773042+0000","last_change":"2023-08-09T14:52:29.708127+0000","last_active":"2023-08-09T14:52:30.773042+0000","last_peered":"2023-08-09T14:52:30.773042+0000","last_clean":"2023-08-09T14:52:30.773042+0000","last_became_active":"2023-08-09T14:52:29.707832+0000","last_became_peered":"2023-08-09T14:52:29.707832+0000","last_unstale":"2023-08-09T14:52:30.773042+0000","last_undegraded":"2023-08-09T14:52:30.773042+0000","last_fullsized":"2023-08-09T14:52:30.773042+0000","mapping_epoch":183,"log_start":"0'0","ondisk_log_start":"0'0","created":183,"last_epoch_clean":184,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T14:52:28.654282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T14:52:28.654282+0000","last_clean_scrub_stamp":"2023-08-09T14:52:28.654282+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T20:44:44.511125+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"23.d","version":"0'0","reported_seq":9,"reported_epoch":185,"state":"active+clean","last_fresh":"2023-08-09T14:52:30.749118+0000","last_change":"2023-08-09T14:52:30.747608+0000","last_active":"2023-08-09T14:52:30.749118+0000","last_peered":"2023-08-09T14:52:30.749118+0000","last_clean":"2023-08-09T14:52:30.749118+0000","last_became_active":"2023-08-09T14:52:30.747289+0000","last_became_peered":"2023-08-09T14:52:30.747289+0000","last_unstale":"2023-08-09T14:52:30.749118+0000","last_undegraded":"2023-08-09T14:52:30.749118+0000","last_fullsized":"2023-08-09T14:52:30.749118+0000","mapping_epoch":184,"log_start":"0'0","ondisk_log_start":"0'0","created":184,"last_epoch_clean":185,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T14:52:29.665328+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T14:52:29.665328+0000","last_clean_scrub_stamp":"2023-08-09T14:52:29.665328+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-11T02:11:53.976567+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"22.c","version":"0'0","reported_seq":11,"reported_epoch":185,"state":"active+clean","last_fresh":"2023-08-09T14:52:30.738608+0000","last_change":"2023-08-09T14:52:29.711505+0000","last_active":"2023-08-09T14:52:30.738608+0000","last_peered":"2023-08-09T14:52:30.738608+0000","last_clean":"2023-08-09T14:52:30.738608+0000","last_became_active":"2023-08-09T14:52:29.711234+0000","last_became_peered":"2023-08-09T14:52:29.711234+0000","last_unstale":"2023-08-09T14:52:30.738608+0000","last_undegraded":"2023-08-09T14:52:30.738608+0000","last_fullsized":"2023-08-09T14:52:30.738608+0000","mapping_epoch":183,"log_start":"0'0","ondisk_log_start":"0'0","created":183,"last_epoch_clean":184,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T14:52:28.654282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T14:52:28.654282+0000","last_clean_scrub_stamp":"2023-08-09T14:52:28.654282+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T20:33:36.515170+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"23.a","version":"0'0","reported_seq":9,"reported_epoch":185,"state":"active+clean","last_fresh":"2023-08-09T14:52:30.755373+0000","last_change":"2023-08-09T14:52:30.753084+0000","last_active":"2023-08-09T14:52:30.755373+0000","last_peered":"2023-08-09T14:52:30.755373+0000","last_clean":"2023-08-09T14:52:30.755373+0000","last_became_active":"2023-08-09T14:52:30.752808+0000","last_became_peered":"2023-08-09T14:52:30.752808+0000","last_unstale":"2023-08-09T14:52:30.755373+0000","last_undegraded":"2023-08-09T14:52:30.755373+0000","last_fullsized":"2023-08-09T14:52:30.755373+0000","mapping_epoch":184,"log_start":"0'0","ondisk_log_start":"0'0","created":184,"last_epoch_clean":185,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T14:52:29.665328+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T14:52:29.665328+0000","last_clean_scrub_stamp":"2023-08-09T14:52:29.665328+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T21:21:12.933823+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"22.b","version":"185'2","reported_seq":13,"reported_epoch":185,"state":"active+clean","last_fresh":"2023-08-09T14:52:30.773187+0000","last_change":"2023-08-09T14:52:29.713164+0000","last_active":"2023-08-09T14:52:30.773187+0000","last_peered":"2023-08-09T14:52:30.773187+0000","last_clean":"2023-08-09T14:52:30.773187+0000","last_became_active":"2023-08-09T14:52:29.712262+0000","last_became_peered":"2023-08-09T14:52:29.712262+0000","last_unstale":"2023-08-09T14:52:30.773187+0000","last_undegraded":"2023-08-09T14:52:30.773187+0000","last_fullsized":"2023-08-09T14:52:30.773187+0000","mapping_epoch":183,"log_start":"0'0","ondisk_log_start":"0'0","created":183,"last_epoch_clean":184,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T14:52:28.654282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T14:52:28.654282+0000","last_clean_scrub_stamp":"2023-08-09T14:52:28.654282+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T17:30:59.228973+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"23.b","version":"0'0","reported_seq":9,"reported_epoch":185,"state":"active+clean","last_fresh":"2023-08-09T14:52:30.756720+0000","last_change":"2023-08-09T14:52:30.749683+0000","last_active":"2023-08-09T14:52:30.756720+0000","last_peered":"2023-08-09T14:52:30.756720+0000","last_clean":"2023-08-09T14:52:30.756720+0000","last_became_active":"2023-08-09T14:52:30.749196+0000","last_became_peered":"2023-08-09T14:52:30.749196+0000","last_unstale":"2023-08-09T14:52:30.756720+0000","last_undegraded":"2023-08-09T14:52:30.756720+0000","last_fullsized":"2023-08-09T14:52:30.756720+0000","mapping_epoch":184,"log_start":"0'0","ondisk_log_start":"0'0","created":184,"last_epoch_clean":185,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T14:52:29.665328+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T14:52:29.665328+0000","last_clean_scrub_stamp":"2023-08-09T14:52:29.665328+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T19:56:10.498391+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"22.a","version":"0'0","reported_seq":11,"reported_epoch":185,"state":"active+clean","last_fresh":"2023-08-09T14:52:30.735818+0000","last_change":"2023-08-09T14:52:29.706557+0000","last_active":"2023-08-09T14:52:30.735818+0000","last_peered":"2023-08-09T14:52:30.735818+0000","last_clean":"2023-08-09T14:52:30.735818+0000","last_became_active":"2023-08-09T14:52:29.706258+0000","last_became_peered":"2023-08-09T14:52:29.706258+0000","last_unstale":"2023-08-09T14:52:30.735818+0000","last_undegraded":"2023-08-09T14:52:30.735818+0000","last_fullsized":"2023-08-09T14:52:30.735818+0000","mapping_epoch":183,"log_start":"0'0","ondisk_log_start":"0'0","created":183,"last_epoch_clean":184,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T14:52:28.654282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T14:52:28.654282+0000","last_clean_scrub_stamp":"2023-08-09T14:52:28.654282+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-11T00:27:57.653513+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"23.8","version":"0'0","reported_seq":9,"reported_epoch":185,"state":"active+clean","last_fresh":"2023-08-09T14:52:30.774738+0000","last_change":"2023-08-09T14:52:30.772364+0000","last_active":"2023-08-09T14:52:30.774738+0000","last_peered":"2023-08-09T14:52:30.774738+0000","last_clean":"2023-08-09T14:52:30.774738+0000","last_became_active":"2023-08-09T14:52:30.768062+0000","last_became_peered":"2023-08-09T14:52:30.768062+0000","last_unstale":"2023-08-09T14:52:30.774738+0000","last_undegraded":"2023-08-09T14:52:30.774738+0000","last_fullsized":"2023-08-09T14:52:30.774738+0000","mapping_epoch":184,"log_start":"0'0","ondisk_log_start":"0'0","created":184,"last_epoch_clean":185,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T14:52:29.665328+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T14:52:29.665328+0000","last_clean_scrub_stamp":"2023-08-09T14:52:29.665328+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T16:43:04.079456+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"22.9","version":"0'0","reported_seq":11,"reported_epoch":185,"state":"active+clean","last_fresh":"2023-08-09T14:52:30.734789+0000","last_change":"2023-08-09T14:52:29.716865+0000","last_active":"2023-08-09T14:52:30.734789+0000","last_peered":"2023-08-09T14:52:30.734789+0000","last_clean":"2023-08-09T14:52:30.734789+0000","last_became_active":"2023-08-09T14:52:29.716554+0000","last_became_peered":"2023-08-09T14:52:29.716554+0000","last_unstale":"2023-08-09T14:52:30.734789+0000","last_undegraded":"2023-08-09T14:52:30.734789+0000","last_fullsized":"2023-08-09T14:52:30.734789+0000","mapping_epoch":183,"log_start":"0'0","ondisk_log_start":"0'0","created":183,"last_epoch_clean":184,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T14:52:28.654282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T14:52:28.654282+0000","last_clean_scrub_stamp":"2023-08-09T14:52:28.654282+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T15:20:59.645144+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"23.9","version":"0'0","reported_seq":9,"reported_epoch":185,"state":"active+clean","last_fresh":"2023-08-09T14:52:30.757256+0000","last_change":"2023-08-09T14:52:30.748423+0000","last_active":"2023-08-09T14:52:30.757256+0000","last_peered":"2023-08-09T14:52:30.757256+0000","last_clean":"2023-08-09T14:52:30.757256+0000","last_became_active":"2023-08-09T14:52:30.748134+0000","last_became_peered":"2023-08-09T14:52:30.748134+0000","last_unstale":"2023-08-09T14:52:30.757256+0000","last_undegraded":"2023-08-09T14:52:30.757256+0000","last_fullsized":"2023-08-09T14:52:30.757256+0000","mapping_epoch":184,"log_start":"0'0","ondisk_log_start":"0'0","created":184,"last_epoch_clean":185,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T14:52:29.665328+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T14:52:29.665328+0000","last_clean_scrub_stamp":"2023-08-09T14:52:29.665328+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T18:28:37.865422+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"22.8","version":"0'0","reported_seq":11,"reported_epoch":185,"state":"active+clean","last_fresh":"2023-08-09T14:52:30.737010+0000","last_change":"2023-08-09T14:52:29.705724+0000","last_active":"2023-08-09T14:52:30.737010+0000","last_peered":"2023-08-09T14:52:30.737010+0000","last_clean":"2023-08-09T14:52:30.737010+0000","last_became_active":"2023-08-09T14:52:29.705214+0000","last_became_peered":"2023-08-09T14:52:29.705214+0000","last_unstale":"2023-08-09T14:52:30.737010+0000","last_undegraded":"2023-08-09T14:52:30.737010+0000","last_fullsized":"2023-08-09T14:52:30.737010+0000","mapping_epoch":183,"log_start":"0'0","ondisk_log_start":"0'0","created":183,"last_epoch_clean":184,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T14:52:28.654282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T14:52:28.654282+0000","last_clean_scrub_stamp":"2023-08-09T14:52:28.654282+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T16:41:30.543988+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"23.6","version":"0'0","reported_seq":9,"reported_epoch":185,"state":"active+clean","last_fresh":"2023-08-09T14:52:30.749020+0000","last_change":"2023-08-09T14:52:30.746688+0000","last_active":"2023-08-09T14:52:30.749020+0000","last_peered":"2023-08-09T14:52:30.749020+0000","last_clean":"2023-08-09T14:52:30.749020+0000","last_became_active":"2023-08-09T14:52:30.745831+0000","last_became_peered":"2023-08-09T14:52:30.745831+0000","last_unstale":"2023-08-09T14:52:30.749020+0000","last_undegraded":"2023-08-09T14:52:30.749020+0000","last_fullsized":"2023-08-09T14:52:30.749020+0000","mapping_epoch":184,"log_start":"0'0","ondisk_log_start":"0'0","created":184,"last_epoch_clean":185,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T14:52:29.665328+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T14:52:29.665328+0000","last_clean_scrub_stamp":"2023-08-09T14:52:29.665328+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T17:21:33.948301+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"22.7","version":"185'2","reported_seq":13,"reported_epoch":185,"state":"active+clean","last_fresh":"2023-08-09T14:52:30.771469+0000","last_change":"2023-08-09T14:52:29.708596+0000","last_active":"2023-08-09T14:52:30.771469+0000","last_peered":"2023-08-09T14:52:30.771469+0000","last_clean":"2023-08-09T14:52:30.771469+0000","last_became_active":"2023-08-09T14:52:29.707759+0000","last_became_peered":"2023-08-09T14:52:29.707759+0000","last_unstale":"2023-08-09T14:52:30.771469+0000","last_undegraded":"2023-08-09T14:52:30.771469+0000","last_fullsized":"2023-08-09T14:52:30.771469+0000","mapping_epoch":183,"log_start":"0'0","ondisk_log_start":"0'0","created":183,"last_epoch_clean":184,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T14:52:28.654282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T14:52:28.654282+0000","last_clean_scrub_stamp":"2023-08-09T14:52:28.654282+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T18:11:32.859448+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"23.7","version":"0'0","reported_seq":9,"reported_epoch":185,"state":"active+clean","last_fresh":"2023-08-09T14:52:30.763571+0000","last_change":"2023-08-09T14:52:30.759239+0000","last_active":"2023-08-09T14:52:30.763571+0000","last_peered":"2023-08-09T14:52:30.763571+0000","last_clean":"2023-08-09T14:52:30.763571+0000","last_became_active":"2023-08-09T14:52:30.758934+0000","last_became_peered":"2023-08-09T14:52:30.758934+0000","last_unstale":"2023-08-09T14:52:30.763571+0000","last_undegraded":"2023-08-09T14:52:30.763571+0000","last_fullsized":"2023-08-09T14:52:30.763571+0000","mapping_epoch":184,"log_start":"0'0","ondisk_log_start":"0'0","created":184,"last_epoch_clean":185,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T14:52:29.665328+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T14:52:29.665328+0000","last_clean_scrub_stamp":"2023-08-09T14:52:29.665328+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T15:17:59.713739+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"22.6","version":"0'0","reported_seq":11,"reported_epoch":185,"state":"active+clean","last_fresh":"2023-08-09T14:52:30.739115+0000","last_change":"2023-08-09T14:52:29.703991+0000","last_active":"2023-08-09T14:52:30.739115+0000","last_peered":"2023-08-09T14:52:30.739115+0000","last_clean":"2023-08-09T14:52:30.739115+0000","last_became_active":"2023-08-09T14:52:29.703708+0000","last_became_peered":"2023-08-09T14:52:29.703708+0000","last_unstale":"2023-08-09T14:52:30.739115+0000","last_undegraded":"2023-08-09T14:52:30.739115+0000","last_fullsized":"2023-08-09T14:52:30.739115+0000","mapping_epoch":183,"log_start":"0'0","ondisk_log_start":"0'0","created":183,"last_epoch_clean":184,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T14:52:28.654282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T14:52:28.654282+0000","last_clean_scrub_stamp":"2023-08-09T14:52:28.654282+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T22:56:34.647423+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"23.4","version":"0'0","reported_seq":9,"reported_epoch":185,"state":"active+clean","last_fresh":"2023-08-09T14:52:30.756835+0000","last_change":"2023-08-09T14:52:30.745677+0000","last_active":"2023-08-09T14:52:30.756835+0000","last_peered":"2023-08-09T14:52:30.756835+0000","last_clean":"2023-08-09T14:52:30.756835+0000","last_became_active":"2023-08-09T14:52:30.745168+0000","last_became_peered":"2023-08-09T14:52:30.745168+0000","last_unstale":"2023-08-09T14:52:30.756835+0000","last_undegraded":"2023-08-09T14:52:30.756835+0000","last_fullsized":"2023-08-09T14:52:30.756835+0000","mapping_epoch":184,"log_start":"0'0","ondisk_log_start":"0'0","created":184,"last_epoch_clean":185,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T14:52:29.665328+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T14:52:29.665328+0000","last_clean_scrub_stamp":"2023-08-09T14:52:29.665328+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T18:31:29.821581+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"22.5","version":"0'0","reported_seq":11,"reported_epoch":185,"state":"active+clean","last_fresh":"2023-08-09T14:52:30.743005+0000","last_change":"2023-08-09T14:52:29.712162+0000","last_active":"2023-08-09T14:52:30.743005+0000","last_peered":"2023-08-09T14:52:30.743005+0000","last_clean":"2023-08-09T14:52:30.743005+0000","last_became_active":"2023-08-09T14:52:29.711832+0000","last_became_peered":"2023-08-09T14:52:29.711832+0000","last_unstale":"2023-08-09T14:52:30.743005+0000","last_undegraded":"2023-08-09T14:52:30.743005+0000","last_fullsized":"2023-08-09T14:52:30.743005+0000","mapping_epoch":183,"log_start":"0'0","ondisk_log_start":"0'0","created":183,"last_epoch_clean":184,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T14:52:28.654282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T14:52:28.654282+0000","last_clean_scrub_stamp":"2023-08-09T14:52:28.654282+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T17:45:12.141004+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"23.5","version":"0'0","reported_seq":9,"reported_epoch":185,"state":"active+clean","last_fresh":"2023-08-09T14:52:30.757335+0000","last_change":"2023-08-09T14:52:30.755065+0000","last_active":"2023-08-09T14:52:30.757335+0000","last_peered":"2023-08-09T14:52:30.757335+0000","last_clean":"2023-08-09T14:52:30.757335+0000","last_became_active":"2023-08-09T14:52:30.754799+0000","last_became_peered":"2023-08-09T14:52:30.754799+0000","last_unstale":"2023-08-09T14:52:30.757335+0000","last_undegraded":"2023-08-09T14:52:30.757335+0000","last_fullsized":"2023-08-09T14:52:30.757335+0000","mapping_epoch":184,"log_start":"0'0","ondisk_log_start":"0'0","created":184,"last_epoch_clean":185,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T14:52:29.665328+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T14:52:29.665328+0000","last_clean_scrub_stamp":"2023-08-09T14:52:29.665328+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T19:02:24.807175+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"22.4","version":"0'0","reported_seq":11,"reported_epoch":185,"state":"active+clean","last_fresh":"2023-08-09T14:52:30.727289+0000","last_change":"2023-08-09T14:52:29.714203+0000","last_active":"2023-08-09T14:52:30.727289+0000","last_peered":"2023-08-09T14:52:30.727289+0000","last_clean":"2023-08-09T14:52:30.727289+0000","last_became_active":"2023-08-09T14:52:29.713927+0000","last_became_peered":"2023-08-09T14:52:29.713927+0000","last_unstale":"2023-08-09T14:52:30.727289+0000","last_undegraded":"2023-08-09T14:52:30.727289+0000","last_fullsized":"2023-08-09T14:52:30.727289+0000","mapping_epoch":183,"log_start":"0'0","ondisk_log_start":"0'0","created":183,"last_epoch_clean":184,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T14:52:28.654282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T14:52:28.654282+0000","last_clean_scrub_stamp":"2023-08-09T14:52:28.654282+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T20:08:49.741258+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"23.2","version":"0'0","reported_seq":9,"reported_epoch":185,"state":"active+clean","last_fresh":"2023-08-09T14:52:30.754415+0000","last_change":"2023-08-09T14:52:30.746639+0000","last_active":"2023-08-09T14:52:30.754415+0000","last_peered":"2023-08-09T14:52:30.754415+0000","last_clean":"2023-08-09T14:52:30.754415+0000","last_became_active":"2023-08-09T14:52:30.746337+0000","last_became_peered":"2023-08-09T14:52:30.746337+0000","last_unstale":"2023-08-09T14:52:30.754415+0000","last_undegraded":"2023-08-09T14:52:30.754415+0000","last_fullsized":"2023-08-09T14:52:30.754415+0000","mapping_epoch":184,"log_start":"0'0","ondisk_log_start":"0'0","created":184,"last_epoch_clean":185,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T14:52:29.665328+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T14:52:29.665328+0000","last_clean_scrub_stamp":"2023-08-09T14:52:29.665328+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T22:12:54.884632+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"22.3","version":"0'0","reported_seq":11,"reported_epoch":185,"state":"active+clean","last_fresh":"2023-08-09T14:52:30.740203+0000","last_change":"2023-08-09T14:52:29.702701+0000","last_active":"2023-08-09T14:52:30.740203+0000","last_peered":"2023-08-09T14:52:30.740203+0000","last_clean":"2023-08-09T14:52:30.740203+0000","last_became_active":"2023-08-09T14:52:29.702431+0000","last_became_peered":"2023-08-09T14:52:29.702431+0000","last_unstale":"2023-08-09T14:52:30.740203+0000","last_undegraded":"2023-08-09T14:52:30.740203+0000","last_fullsized":"2023-08-09T14:52:30.740203+0000","mapping_epoch":183,"log_start":"0'0","ondisk_log_start":"0'0","created":183,"last_epoch_clean":184,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T14:52:28.654282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T14:52:28.654282+0000","last_clean_scrub_stamp":"2023-08-09T14:52:28.654282+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T15:06:20.990122+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"23.3","version":"0'0","reported_seq":9,"reported_epoch":185,"state":"active+clean","last_fresh":"2023-08-09T14:52:30.766359+0000","last_change":"2023-08-09T14:52:30.759160+0000","last_active":"2023-08-09T14:52:30.766359+0000","last_peered":"2023-08-09T14:52:30.766359+0000","last_clean":"2023-08-09T14:52:30.766359+0000","last_became_active":"2023-08-09T14:52:30.758862+0000","last_became_peered":"2023-08-09T14:52:30.758862+0000","last_unstale":"2023-08-09T14:52:30.766359+0000","last_undegraded":"2023-08-09T14:52:30.766359+0000","last_fullsized":"2023-08-09T14:52:30.766359+0000","mapping_epoch":184,"log_start":"0'0","ondisk_log_start":"0'0","created":184,"last_epoch_clean":185,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T14:52:29.665328+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T14:52:29.665328+0000","last_clean_scrub_stamp":"2023-08-09T14:52:29.665328+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T18:55:43.404564+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"22.2","version":"0'0","reported_seq":11,"reported_epoch":185,"state":"active+clean","last_fresh":"2023-08-09T14:52:30.734096+0000","last_change":"2023-08-09T14:52:29.692393+0000","last_active":"2023-08-09T14:52:30.734096+0000","last_peered":"2023-08-09T14:52:30.734096+0000","last_clean":"2023-08-09T14:52:30.734096+0000","last_became_active":"2023-08-09T14:52:29.691789+0000","last_became_peered":"2023-08-09T14:52:29.691789+0000","last_unstale":"2023-08-09T14:52:30.734096+0000","last_undegraded":"2023-08-09T14:52:30.734096+0000","last_fullsized":"2023-08-09T14:52:30.734096+0000","mapping_epoch":183,"log_start":"0'0","ondisk_log_start":"0'0","created":183,"last_epoch_clean":184,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T14:52:28.654282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T14:52:28.654282+0000","last_clean_scrub_stamp":"2023-08-09T14:52:28.654282+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T17:38:05.365447+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"23.0","version":"0'0","reported_seq":9,"reported_epoch":185,"state":"active+clean","last_fresh":"2023-08-09T14:52:30.757142+0000","last_change":"2023-08-09T14:52:30.750614+0000","last_active":"2023-08-09T14:52:30.757142+0000","last_peered":"2023-08-09T14:52:30.757142+0000","last_clean":"2023-08-09T14:52:30.757142+0000","last_became_active":"2023-08-09T14:52:30.749983+0000","last_became_peered":"2023-08-09T14:52:30.749983+0000","last_unstale":"2023-08-09T14:52:30.757142+0000","last_undegraded":"2023-08-09T14:52:30.757142+0000","last_fullsized":"2023-08-09T14:52:30.757142+0000","mapping_epoch":184,"log_start":"0'0","ondisk_log_start":"0'0","created":184,"last_epoch_clean":185,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T14:52:29.665328+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T14:52:29.665328+0000","last_clean_scrub_stamp":"2023-08-09T14:52:29.665328+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-11T00:38:39.582024+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"22.1","version":"0'0","reported_seq":11,"reported_epoch":185,"state":"active+clean","last_fresh":"2023-08-09T14:52:30.744520+0000","last_change":"2023-08-09T14:52:29.709333+0000","last_active":"2023-08-09T14:52:30.744520+0000","last_peered":"2023-08-09T14:52:30.744520+0000","last_clean":"2023-08-09T14:52:30.744520+0000","last_became_active":"2023-08-09T14:52:29.708994+0000","last_became_peered":"2023-08-09T14:52:29.708994+0000","last_unstale":"2023-08-09T14:52:30.744520+0000","last_undegraded":"2023-08-09T14:52:30.744520+0000","last_fullsized":"2023-08-09T14:52:30.744520+0000","mapping_epoch":183,"log_start":"0'0","ondisk_log_start":"0'0","created":183,"last_epoch_clean":184,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T14:52:28.654282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T14:52:28.654282+0000","last_clean_scrub_stamp":"2023-08-09T14:52:28.654282+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T16:04:48.527506+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"23.1","version":"0'0","reported_seq":9,"reported_epoch":185,"state":"active+clean","last_fresh":"2023-08-09T14:52:30.748409+0000","last_change":"2023-08-09T14:52:30.747011+0000","last_active":"2023-08-09T14:52:30.748409+0000","last_peered":"2023-08-09T14:52:30.748409+0000","last_clean":"2023-08-09T14:52:30.748409+0000","last_became_active":"2023-08-09T14:52:30.746102+0000","last_became_peered":"2023-08-09T14:52:30.746102+0000","last_unstale":"2023-08-09T14:52:30.748409+0000","last_undegraded":"2023-08-09T14:52:30.748409+0000","last_fullsized":"2023-08-09T14:52:30.748409+0000","mapping_epoch":184,"log_start":"0'0","ondisk_log_start":"0'0","created":184,"last_epoch_clean":185,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T14:52:29.665328+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T14:52:29.665328+0000","last_clean_scrub_stamp":"2023-08-09T14:52:29.665328+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T19:10:33.908606+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"22.0","version":"0'0","reported_seq":11,"reported_epoch":185,"state":"active+clean","last_fresh":"2023-08-09T14:52:30.735628+0000","last_change":"2023-08-09T14:52:29.699378+0000","last_active":"2023-08-09T14:52:30.735628+0000","last_peered":"2023-08-09T14:52:30.735628+0000","last_clean":"2023-08-09T14:52:30.735628+0000","last_became_active":"2023-08-09T14:52:29.698999+0000","last_became_peered":"2023-08-09T14:52:29.698999+0000","last_unstale":"2023-08-09T14:52:30.735628+0000","last_undegraded":"2023-08-09T14:52:30.735628+0000","last_fullsized":"2023-08-09T14:52:30.735628+0000","mapping_epoch":183,"log_start":"0'0","ondisk_log_start":"0'0","created":183,"last_epoch_clean":184,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T14:52:28.654282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T14:52:28.654282+0000","last_clean_scrub_stamp":"2023-08-09T14:52:28.654282+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-11T01:22:08.109189+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"23.16","version":"0'0","reported_seq":9,"reported_epoch":185,"state":"active+clean","last_fresh":"2023-08-09T14:52:30.757340+0000","last_change":"2023-08-09T14:52:30.748843+0000","last_active":"2023-08-09T14:52:30.757340+0000","last_peered":"2023-08-09T14:52:30.757340+0000","last_clean":"2023-08-09T14:52:30.757340+0000","last_became_active":"2023-08-09T14:52:30.747365+0000","last_became_peered":"2023-08-09T14:52:30.747365+0000","last_unstale":"2023-08-09T14:52:30.757340+0000","last_undegraded":"2023-08-09T14:52:30.757340+0000","last_fullsized":"2023-08-09T14:52:30.757340+0000","mapping_epoch":184,"log_start":"0'0","ondisk_log_start":"0'0","created":184,"last_epoch_clean":185,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T14:52:29.665328+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T14:52:29.665328+0000","last_clean_scrub_stamp":"2023-08-09T14:52:29.665328+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T18:06:08.837776+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"22.17","version":"0'0","reported_seq":11,"reported_epoch":185,"state":"active+clean","last_fresh":"2023-08-09T14:52:30.742688+0000","last_change":"2023-08-09T14:52:29.712808+0000","last_active":"2023-08-09T14:52:30.742688+0000","last_peered":"2023-08-09T14:52:30.742688+0000","last_clean":"2023-08-09T14:52:30.742688+0000","last_became_active":"2023-08-09T14:52:29.712507+0000","last_became_peered":"2023-08-09T14:52:29.712507+0000","last_unstale":"2023-08-09T14:52:30.742688+0000","last_undegraded":"2023-08-09T14:52:30.742688+0000","last_fullsized":"2023-08-09T14:52:30.742688+0000","mapping_epoch":183,"log_start":"0'0","ondisk_log_start":"0'0","created":183,"last_epoch_clean":184,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T14:52:28.654282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T14:52:28.654282+0000","last_clean_scrub_stamp":"2023-08-09T14:52:28.654282+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T22:31:25.507328+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":340,"reported_epoch":185,"state":"active+clean","last_fresh":"2023-08-09T14:52:30.737319+0000","last_change":"2023-08-09T13:33:56.401406+0000","last_active":"2023-08-09T14:52:30.737319+0000","last_peered":"2023-08-09T14:52:30.737319+0000","last_clean":"2023-08-09T14:52:30.737319+0000","last_became_active":"2023-08-09T13:33:56.400487+0000","last_became_peered":"2023-08-09T13:33:56.400487+0000","last_unstale":"2023-08-09T14:52:30.737319+0000","last_undegraded":"2023-08-09T14:52:30.737319+0000","last_fullsized":"2023-08-09T14:52:30.737319+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:33:54.845869+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:33:54.845869+0000","last_clean_scrub_stamp":"2023-08-09T13:33:54.845869+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T16:55:01.819791+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"23.17","version":"0'0","reported_seq":9,"reported_epoch":185,"state":"active+clean","last_fresh":"2023-08-09T14:52:30.757405+0000","last_change":"2023-08-09T14:52:30.742487+0000","last_active":"2023-08-09T14:52:30.757405+0000","last_peered":"2023-08-09T14:52:30.757405+0000","last_clean":"2023-08-09T14:52:30.757405+0000","last_became_active":"2023-08-09T14:52:30.741798+0000","last_became_peered":"2023-08-09T14:52:30.741798+0000","last_unstale":"2023-08-09T14:52:30.757405+0000","last_undegraded":"2023-08-09T14:52:30.757405+0000","last_fullsized":"2023-08-09T14:52:30.757405+0000","mapping_epoch":184,"log_start":"0'0","ondisk_log_start":"0'0","created":184,"last_epoch_clean":185,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T14:52:29.665328+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T14:52:29.665328+0000","last_clean_scrub_stamp":"2023-08-09T14:52:29.665328+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T21:57:20.199759+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"22.16","version":"0'0","reported_seq":11,"reported_epoch":185,"state":"active+clean","last_fresh":"2023-08-09T14:52:30.724008+0000","last_change":"2023-08-09T14:52:29.706553+0000","last_active":"2023-08-09T14:52:30.724008+0000","last_peered":"2023-08-09T14:52:30.724008+0000","last_clean":"2023-08-09T14:52:30.724008+0000","last_became_active":"2023-08-09T14:52:29.706199+0000","last_became_peered":"2023-08-09T14:52:29.706199+0000","last_unstale":"2023-08-09T14:52:30.724008+0000","last_undegraded":"2023-08-09T14:52:30.724008+0000","last_fullsized":"2023-08-09T14:52:30.724008+0000","mapping_epoch":183,"log_start":"0'0","ondisk_log_start":"0'0","created":183,"last_epoch_clean":184,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T14:52:28.654282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T14:52:28.654282+0000","last_clean_scrub_stamp":"2023-08-09T14:52:28.654282+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T20:20:06.900048+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":348,"reported_epoch":185,"state":"active+clean","last_fresh":"2023-08-09T14:52:30.732967+0000","last_change":"2023-08-09T13:33:56.401123+0000","last_active":"2023-08-09T14:52:30.732967+0000","last_peered":"2023-08-09T14:52:30.732967+0000","last_clean":"2023-08-09T14:52:30.732967+0000","last_became_active":"2023-08-09T13:33:56.399933+0000","last_became_peered":"2023-08-09T13:33:56.399933+0000","last_unstale":"2023-08-09T14:52:30.732967+0000","last_undegraded":"2023-08-09T14:52:30.732967+0000","last_fullsized":"2023-08-09T14:52:30.732967+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:33:54.845869+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:33:54.845869+0000","last_clean_scrub_stamp":"2023-08-09T13:33:54.845869+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-11T01:14:31.338701+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"23.14","version":"0'0","reported_seq":9,"reported_epoch":185,"state":"active+clean","last_fresh":"2023-08-09T14:52:30.747632+0000","last_change":"2023-08-09T14:52:30.746166+0000","last_active":"2023-08-09T14:52:30.747632+0000","last_peered":"2023-08-09T14:52:30.747632+0000","last_clean":"2023-08-09T14:52:30.747632+0000","last_became_active":"2023-08-09T14:52:30.745820+0000","last_became_peered":"2023-08-09T14:52:30.745820+0000","last_unstale":"2023-08-09T14:52:30.747632+0000","last_undegraded":"2023-08-09T14:52:30.747632+0000","last_fullsized":"2023-08-09T14:52:30.747632+0000","mapping_epoch":184,"log_start":"0'0","ondisk_log_start":"0'0","created":184,"last_epoch_clean":185,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T14:52:29.665328+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T14:52:29.665328+0000","last_clean_scrub_stamp":"2023-08-09T14:52:29.665328+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T23:19:45.490259+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"22.15","version":"185'3","reported_seq":14,"reported_epoch":185,"state":"active+clean","last_fresh":"2023-08-09T14:52:30.773557+0000","last_change":"2023-08-09T14:52:29.710014+0000","last_active":"2023-08-09T14:52:30.773557+0000","last_peered":"2023-08-09T14:52:30.773557+0000","last_clean":"2023-08-09T14:52:30.773557+0000","last_became_active":"2023-08-09T14:52:29.709737+0000","last_became_peered":"2023-08-09T14:52:29.709737+0000","last_unstale":"2023-08-09T14:52:30.773557+0000","last_undegraded":"2023-08-09T14:52:30.773557+0000","last_fullsized":"2023-08-09T14:52:30.773557+0000","mapping_epoch":183,"log_start":"0'0","ondisk_log_start":"0'0","created":183,"last_epoch_clean":184,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T14:52:28.654282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T14:52:28.654282+0000","last_clean_scrub_stamp":"2023-08-09T14:52:28.654282+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-11T00:13:22.178599+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.2","version":"0'0","reported_seq":338,"reported_epoch":185,"state":"active+clean","last_fresh":"2023-08-09T14:52:30.735966+0000","last_change":"2023-08-09T13:33:56.402794+0000","last_active":"2023-08-09T14:52:30.735966+0000","last_peered":"2023-08-09T14:52:30.735966+0000","last_clean":"2023-08-09T14:52:30.735966+0000","last_became_active":"2023-08-09T13:33:56.402038+0000","last_became_peered":"2023-08-09T13:33:56.402038+0000","last_unstale":"2023-08-09T14:52:30.735966+0000","last_undegraded":"2023-08-09T14:52:30.735966+0000","last_fullsized":"2023-08-09T14:52:30.735966+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:33:54.845869+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:33:54.845869+0000","last_clean_scrub_stamp":"2023-08-09T13:33:54.845869+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T19:04:24.398873+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"23.15","version":"0'0","reported_seq":9,"reported_epoch":185,"state":"active+clean","last_fresh":"2023-08-09T14:52:30.760545+0000","last_change":"2023-08-09T14:52:30.756276+0000","last_active":"2023-08-09T14:52:30.760545+0000","last_peered":"2023-08-09T14:52:30.760545+0000","last_clean":"2023-08-09T14:52:30.760545+0000","last_became_active":"2023-08-09T14:52:30.755934+0000","last_became_peered":"2023-08-09T14:52:30.755934+0000","last_unstale":"2023-08-09T14:52:30.760545+0000","last_undegraded":"2023-08-09T14:52:30.760545+0000","last_fullsized":"2023-08-09T14:52:30.760545+0000","mapping_epoch":184,"log_start":"0'0","ondisk_log_start":"0'0","created":184,"last_epoch_clean":185,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T14:52:29.665328+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T14:52:29.665328+0000","last_clean_scrub_stamp":"2023-08-09T14:52:29.665328+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-11T00:27:44.822389+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"22.14","version":"185'6","reported_seq":17,"reported_epoch":185,"state":"active+clean","last_fresh":"2023-08-09T14:52:30.775627+0000","last_change":"2023-08-09T14:52:29.706374+0000","last_active":"2023-08-09T14:52:30.775627+0000","last_peered":"2023-08-09T14:52:30.775627+0000","last_clean":"2023-08-09T14:52:30.775627+0000","last_became_active":"2023-08-09T14:52:29.706004+0000","last_became_peered":"2023-08-09T14:52:29.706004+0000","last_unstale":"2023-08-09T14:52:30.775627+0000","last_undegraded":"2023-08-09T14:52:30.775627+0000","last_fullsized":"2023-08-09T14:52:30.775627+0000","mapping_epoch":183,"log_start":"0'0","ondisk_log_start":"0'0","created":183,"last_epoch_clean":184,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T14:52:28.654282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T14:52:28.654282+0000","last_clean_scrub_stamp":"2023-08-09T14:52:28.654282+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-11T00:40:29.631853+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.3","version":"0'0","reported_seq":350,"reported_epoch":185,"state":"active+clean","last_fresh":"2023-08-09T14:52:30.736532+0000","last_change":"2023-08-09T13:33:55.886749+0000","last_active":"2023-08-09T14:52:30.736532+0000","last_peered":"2023-08-09T14:52:30.736532+0000","last_clean":"2023-08-09T14:52:30.736532+0000","last_became_active":"2023-08-09T13:33:55.886116+0000","last_became_peered":"2023-08-09T13:33:55.886116+0000","last_unstale":"2023-08-09T14:52:30.736532+0000","last_undegraded":"2023-08-09T14:52:30.736532+0000","last_fullsized":"2023-08-09T14:52:30.736532+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:33:54.845869+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:33:54.845869+0000","last_clean_scrub_stamp":"2023-08-09T13:33:54.845869+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-11T00:13:44.684470+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"174'274","reported_seq":1046,"reported_epoch":185,"state":"active+clean","last_fresh":"2023-08-09T14:52:30.737131+0000","last_change":"2023-08-09T13:33:55.886345+0000","last_active":"2023-08-09T14:52:30.737131+0000","last_peered":"2023-08-09T14:52:30.737131+0000","last_clean":"2023-08-09T14:52:30.737131+0000","last_became_active":"2023-08-09T13:33:55.885781+0000","last_became_peered":"2023-08-09T13:33:55.885781+0000","last_unstale":"2023-08-09T14:52:30.737131+0000","last_undegraded":"2023-08-09T14:52:30.737131+0000","last_fullsized":"2023-08-09T14:52:30.737131+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:33:54.845869+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:33:54.845869+0000","last_clean_scrub_stamp":"2023-08-09T13:33:54.845869+0000","objects_scrubbed":0,"log_size":274,"log_dups_size":0,"ondisk_log_size":274,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-11T00:50:15.334951+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1483,"num_read_kb":3066,"num_write":443,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"23.12","version":"0'0","reported_seq":9,"reported_epoch":185,"state":"active+clean","last_fresh":"2023-08-09T14:52:30.762910+0000","last_change":"2023-08-09T14:52:30.761825+0000","last_active":"2023-08-09T14:52:30.762910+0000","last_peered":"2023-08-09T14:52:30.762910+0000","last_clean":"2023-08-09T14:52:30.762910+0000","last_became_active":"2023-08-09T14:52:30.761412+0000","last_became_peered":"2023-08-09T14:52:30.761412+0000","last_unstale":"2023-08-09T14:52:30.762910+0000","last_undegraded":"2023-08-09T14:52:30.762910+0000","last_fullsized":"2023-08-09T14:52:30.762910+0000","mapping_epoch":184,"log_start":"0'0","ondisk_log_start":"0'0","created":184,"last_epoch_clean":185,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T14:52:29.665328+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T14:52:29.665328+0000","last_clean_scrub_stamp":"2023-08-09T14:52:29.665328+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-11T00:41:24.209798+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"22.13","version":"185'2","reported_seq":13,"reported_epoch":185,"state":"active+clean","last_fresh":"2023-08-09T14:52:30.772863+0000","last_change":"2023-08-09T14:52:29.697008+0000","last_active":"2023-08-09T14:52:30.772863+0000","last_peered":"2023-08-09T14:52:30.772863+0000","last_clean":"2023-08-09T14:52:30.772863+0000","last_became_active":"2023-08-09T14:52:29.696501+0000","last_became_peered":"2023-08-09T14:52:29.696501+0000","last_unstale":"2023-08-09T14:52:30.772863+0000","last_undegraded":"2023-08-09T14:52:30.772863+0000","last_fullsized":"2023-08-09T14:52:30.772863+0000","mapping_epoch":183,"log_start":"0'0","ondisk_log_start":"0'0","created":183,"last_epoch_clean":184,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T14:52:28.654282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T14:52:28.654282+0000","last_clean_scrub_stamp":"2023-08-09T14:52:28.654282+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-11T00:03:28.802518+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":350,"reported_epoch":185,"state":"active+clean","last_fresh":"2023-08-09T14:52:30.727531+0000","last_change":"2023-08-09T13:33:55.889565+0000","last_active":"2023-08-09T14:52:30.727531+0000","last_peered":"2023-08-09T14:52:30.727531+0000","last_clean":"2023-08-09T14:52:30.727531+0000","last_became_active":"2023-08-09T13:33:55.889164+0000","last_became_peered":"2023-08-09T13:33:55.889164+0000","last_unstale":"2023-08-09T14:52:30.727531+0000","last_undegraded":"2023-08-09T14:52:30.727531+0000","last_fullsized":"2023-08-09T14:52:30.727531+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:33:54.845869+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:33:54.845869+0000","last_clean_scrub_stamp":"2023-08-09T13:33:54.845869+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T13:45:16.129340+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"23.13","version":"0'0","reported_seq":9,"reported_epoch":185,"state":"active+clean","last_fresh":"2023-08-09T14:52:30.756698+0000","last_change":"2023-08-09T14:52:30.749497+0000","last_active":"2023-08-09T14:52:30.756698+0000","last_peered":"2023-08-09T14:52:30.756698+0000","last_clean":"2023-08-09T14:52:30.756698+0000","last_became_active":"2023-08-09T14:52:30.746836+0000","last_became_peered":"2023-08-09T14:52:30.746836+0000","last_unstale":"2023-08-09T14:52:30.756698+0000","last_undegraded":"2023-08-09T14:52:30.756698+0000","last_fullsized":"2023-08-09T14:52:30.756698+0000","mapping_epoch":184,"log_start":"0'0","ondisk_log_start":"0'0","created":184,"last_epoch_clean":185,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T14:52:29.665328+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T14:52:29.665328+0000","last_clean_scrub_stamp":"2023-08-09T14:52:29.665328+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T17:45:30.550869+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"22.12","version":"0'0","reported_seq":11,"reported_epoch":185,"state":"active+clean","last_fresh":"2023-08-09T14:52:30.737083+0000","last_change":"2023-08-09T14:52:29.708545+0000","last_active":"2023-08-09T14:52:30.737083+0000","last_peered":"2023-08-09T14:52:30.737083+0000","last_clean":"2023-08-09T14:52:30.737083+0000","last_became_active":"2023-08-09T14:52:29.707008+0000","last_became_peered":"2023-08-09T14:52:29.707008+0000","last_unstale":"2023-08-09T14:52:30.737083+0000","last_undegraded":"2023-08-09T14:52:30.737083+0000","last_fullsized":"2023-08-09T14:52:30.737083+0000","mapping_epoch":183,"log_start":"0'0","ondisk_log_start":"0'0","created":183,"last_epoch_clean":184,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T14:52:28.654282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T14:52:28.654282+0000","last_clean_scrub_stamp":"2023-08-09T14:52:28.654282+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-11T02:20:03.687422+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":348,"reported_epoch":185,"state":"active+clean","last_fresh":"2023-08-09T14:52:30.742114+0000","last_change":"2023-08-09T13:33:56.404236+0000","last_active":"2023-08-09T14:52:30.742114+0000","last_peered":"2023-08-09T14:52:30.742114+0000","last_clean":"2023-08-09T14:52:30.742114+0000","last_became_active":"2023-08-09T13:33:56.402485+0000","last_became_peered":"2023-08-09T13:33:56.402485+0000","last_unstale":"2023-08-09T14:52:30.742114+0000","last_undegraded":"2023-08-09T14:52:30.742114+0000","last_fullsized":"2023-08-09T14:52:30.742114+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:33:54.845869+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:33:54.845869+0000","last_clean_scrub_stamp":"2023-08-09T13:33:54.845869+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T22:13:54.485808+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"23.10","version":"0'0","reported_seq":9,"reported_epoch":185,"state":"active+clean","last_fresh":"2023-08-09T14:52:30.765202+0000","last_change":"2023-08-09T14:52:30.757015+0000","last_active":"2023-08-09T14:52:30.765202+0000","last_peered":"2023-08-09T14:52:30.765202+0000","last_clean":"2023-08-09T14:52:30.765202+0000","last_became_active":"2023-08-09T14:52:30.756469+0000","last_became_peered":"2023-08-09T14:52:30.756469+0000","last_unstale":"2023-08-09T14:52:30.765202+0000","last_undegraded":"2023-08-09T14:52:30.765202+0000","last_fullsized":"2023-08-09T14:52:30.765202+0000","mapping_epoch":184,"log_start":"0'0","ondisk_log_start":"0'0","created":184,"last_epoch_clean":185,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T14:52:29.665328+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T14:52:29.665328+0000","last_clean_scrub_stamp":"2023-08-09T14:52:29.665328+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T23:01:36.233201+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"22.11","version":"0'0","reported_seq":11,"reported_epoch":185,"state":"active+clean","last_fresh":"2023-08-09T14:52:30.735757+0000","last_change":"2023-08-09T14:52:29.712599+0000","last_active":"2023-08-09T14:52:30.735757+0000","last_peered":"2023-08-09T14:52:30.735757+0000","last_clean":"2023-08-09T14:52:30.735757+0000","last_became_active":"2023-08-09T14:52:29.712276+0000","last_became_peered":"2023-08-09T14:52:29.712276+0000","last_unstale":"2023-08-09T14:52:30.735757+0000","last_undegraded":"2023-08-09T14:52:30.735757+0000","last_fullsized":"2023-08-09T14:52:30.735757+0000","mapping_epoch":183,"log_start":"0'0","ondisk_log_start":"0'0","created":183,"last_epoch_clean":184,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T14:52:28.654282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T14:52:28.654282+0000","last_clean_scrub_stamp":"2023-08-09T14:52:28.654282+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T23:27:30.639308+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":337,"reported_epoch":185,"state":"active+clean","last_fresh":"2023-08-09T14:52:30.741653+0000","last_change":"2023-08-09T13:33:56.401781+0000","last_active":"2023-08-09T14:52:30.741653+0000","last_peered":"2023-08-09T14:52:30.741653+0000","last_clean":"2023-08-09T14:52:30.741653+0000","last_became_active":"2023-08-09T13:33:56.400336+0000","last_became_peered":"2023-08-09T13:33:56.400336+0000","last_unstale":"2023-08-09T14:52:30.741653+0000","last_undegraded":"2023-08-09T14:52:30.741653+0000","last_fullsized":"2023-08-09T14:52:30.741653+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:33:54.845869+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:33:54.845869+0000","last_clean_scrub_stamp":"2023-08-09T13:33:54.845869+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T23:49:17.599206+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"23.11","version":"0'0","reported_seq":9,"reported_epoch":185,"state":"active+clean","last_fresh":"2023-08-09T14:52:30.750485+0000","last_change":"2023-08-09T14:52:30.749874+0000","last_active":"2023-08-09T14:52:30.750485+0000","last_peered":"2023-08-09T14:52:30.750485+0000","last_clean":"2023-08-09T14:52:30.750485+0000","last_became_active":"2023-08-09T14:52:30.749195+0000","last_became_peered":"2023-08-09T14:52:30.749195+0000","last_unstale":"2023-08-09T14:52:30.750485+0000","last_undegraded":"2023-08-09T14:52:30.750485+0000","last_fullsized":"2023-08-09T14:52:30.750485+0000","mapping_epoch":184,"log_start":"0'0","ondisk_log_start":"0'0","created":184,"last_epoch_clean":185,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T14:52:29.665328+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T14:52:29.665328+0000","last_clean_scrub_stamp":"2023-08-09T14:52:29.665328+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T21:24:37.801551+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"22.10","version":"185'8","reported_seq":19,"reported_epoch":185,"state":"active+clean","last_fresh":"2023-08-09T14:52:30.775295+0000","last_change":"2023-08-09T14:52:29.706475+0000","last_active":"2023-08-09T14:52:30.775295+0000","last_peered":"2023-08-09T14:52:30.775295+0000","last_clean":"2023-08-09T14:52:30.775295+0000","last_became_active":"2023-08-09T14:52:29.706135+0000","last_became_peered":"2023-08-09T14:52:29.706135+0000","last_unstale":"2023-08-09T14:52:30.775295+0000","last_undegraded":"2023-08-09T14:52:30.775295+0000","last_fullsized":"2023-08-09T14:52:30.775295+0000","mapping_epoch":183,"log_start":"0'0","ondisk_log_start":"0'0","created":183,"last_epoch_clean":184,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T14:52:28.654282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T14:52:28.654282+0000","last_clean_scrub_stamp":"2023-08-09T14:52:28.654282+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T22:53:32.087258+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":337,"reported_epoch":185,"state":"active+clean","last_fresh":"2023-08-09T14:52:30.737797+0000","last_change":"2023-08-09T13:33:55.886389+0000","last_active":"2023-08-09T14:52:30.737797+0000","last_peered":"2023-08-09T14:52:30.737797+0000","last_clean":"2023-08-09T14:52:30.737797+0000","last_became_active":"2023-08-09T13:33:55.885638+0000","last_became_peered":"2023-08-09T13:33:55.885638+0000","last_unstale":"2023-08-09T14:52:30.737797+0000","last_undegraded":"2023-08-09T14:52:30.737797+0000","last_fullsized":"2023-08-09T14:52:30.737797+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:33:54.845869+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:33:54.845869+0000","last_clean_scrub_stamp":"2023-08-09T13:33:54.845869+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T21:37:53.176770+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]}],"pool_stats":[{"poolid":23,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":22,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":38,"ondisk_log_size":38,"up":64,"acting":64,"num_store_stats":8},{"poolid":2,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1483,"num_read_kb":3066,"num_write":443,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":274,"ondisk_log_size":274,"up":2,"acting":2,"num_store_stats":2},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":8}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834575787,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":47908,"kb_used_data":2836,"kb_used_omap":1,"kb_used_meta":45054,"kb_avail":94323932,"statfs":{"total":96636764160,"available":96587706368,"internally_reserved":0,"allocated":2904064,"data_stored":1840624,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1781,"internal_metadata":46135563},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Aug 9 14:52:11 2023","interfaces":[{"interface":"back","average":{"1min":1.129,"5min":1.1990000000000001,"15min":1.23},"min":{"1min":0.57899999999999996,"5min":0.57899999999999996,"15min":0.57899999999999996},"max":{"1min":1.3939999999999999,"5min":1.6930000000000001,"15min":2.2320000000000002},"last":1.208},{"interface":"front","average":{"1min":1.202,"5min":1.2250000000000001,"15min":1.264},"min":{"1min":0.76100000000000001,"5min":0.76100000000000001,"15min":0.76100000000000001},"max":{"1min":1.883,"5min":1.883,"15min":1.883},"last":1.036}]},{"osd":1,"last update":"Wed Aug 9 14:52:11 2023","interfaces":[{"interface":"back","average":{"1min":1.2150000000000001,"5min":1.2889999999999999,"15min":1.319},"min":{"1min":0.72799999999999998,"5min":0.72799999999999998,"15min":0.72799999999999998},"max":{"1min":1.7350000000000001,"5min":1.9239999999999999,"15min":1.9470000000000001},"last":1.391},{"interface":"front","average":{"1min":1.262,"5min":1.3089999999999999,"15min":1.3089999999999999},"min":{"1min":0.52000000000000002,"5min":0.52000000000000002,"15min":0.52000000000000002},"max":{"1min":1.976,"5min":1.976,"15min":1.976},"last":1.151}]},{"osd":2,"last update":"Wed Aug 9 14:52:11 2023","interfaces":[{"interface":"back","average":{"1min":1.2,"5min":1.276,"15min":1.304},"min":{"1min":0.83099999999999996,"5min":0.83099999999999996,"15min":0.83099999999999996},"max":{"1min":1.577,"5min":1.764,"15min":2.3149999999999999},"last":0.77200000000000002},{"interface":"front","average":{"1min":1.294,"5min":1.365,"15min":1.387},"min":{"1min":1.0149999999999999,"5min":0.93200000000000005,"15min":0.92100000000000004},"max":{"1min":1.6499999999999999,"5min":1.9279999999999999,"15min":2.0110000000000001},"last":0.89800000000000002}]},{"osd":3,"last update":"Wed Aug 9 14:52:11 2023","interfaces":[{"interface":"back","average":{"1min":1.206,"5min":1.331,"15min":1.355},"min":{"1min":0.68200000000000005,"5min":0.68200000000000005,"15min":0.68200000000000005},"max":{"1min":1.4710000000000001,"5min":1.9910000000000001,"15min":1.9910000000000001},"last":1.083},{"interface":"front","average":{"1min":1.333,"5min":1.4410000000000001,"15min":1.4650000000000001},"min":{"1min":0.98299999999999998,"5min":0.98299999999999998,"15min":0.98299999999999998},"max":{"1min":1.6020000000000001,"5min":1.887,"15min":2.375},"last":1.5629999999999999}]},{"osd":4,"last update":"Wed Aug 9 14:52:11 2023","interfaces":[{"interface":"back","average":{"1min":1.401,"5min":1.3129999999999999,"15min":1.3200000000000001},"min":{"1min":0.93400000000000005,"5min":0.82499999999999996,"15min":0.79800000000000004},"max":{"1min":2.0510000000000002,"5min":2.0510000000000002,"15min":2.254},"last":1.2609999999999999},{"interface":"front","average":{"1min":1.341,"5min":1.3959999999999999,"15min":1.387},"min":{"1min":0.92900000000000005,"5min":0.90400000000000003,"15min":0.90400000000000003},"max":{"1min":1.6799999999999999,"5min":1.859,"15min":2.1789999999999998},"last":0.96999999999999997}]},{"osd":5,"last update":"Wed Aug 9 14:52:11 2023","interfaces":[{"interface":"back","average":{"1min":1.47,"5min":1.4930000000000001,"15min":1.4750000000000001},"min":{"1min":1.0820000000000001,"5min":0.85599999999999998,"15min":0.76900000000000002},"max":{"1min":1.7789999999999999,"5min":1.9770000000000001,"15min":2.1589999999999998},"last":1.498},{"interface":"front","average":{"1min":1.5449999999999999,"5min":1.508,"15min":1.5},"min":{"1min":0.998,"5min":0.92400000000000004,"15min":0.92400000000000004},"max":{"1min":2.1400000000000001,"5min":2.1949999999999998,"15min":2.4510000000000001},"last":1.6040000000000001}]},{"osd":6,"last update":"Wed Aug 9 14:52:11 2023","interfaces":[{"interface":"back","average":{"1min":1.7190000000000001,"5min":1.6359999999999999,"15min":1.615},"min":{"1min":0.97099999999999997,"5min":0.97099999999999997,"15min":0.97099999999999997},"max":{"1min":3.0680000000000001,"5min":3.0680000000000001,"15min":3.0680000000000001},"last":1.4419999999999999},{"interface":"front","average":{"1min":1.7310000000000001,"5min":1.579,"15min":1.5620000000000001},"min":{"1min":1.254,"5min":0.98499999999999999,"15min":0.84899999999999998},"max":{"1min":4.516,"5min":4.516,"15min":4.516},"last":1.3140000000000001}]}]},{"osd":6,"up_from":13,"seq":55834575787,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":59612,"kb_used_data":2256,"kb_used_omap":1,"kb_used_meta":57342,"kb_avail":94312228,"statfs":{"total":96636764160,"available":96575721472,"internally_reserved":0,"allocated":2310144,"data_stored":1248917,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1781,"internal_metadata":58718475},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Aug 9 14:52:22 2023","interfaces":[{"interface":"back","average":{"1min":1.3260000000000001,"5min":1.2170000000000001,"15min":1.214},"min":{"1min":0.85699999999999998,"5min":0.75,"15min":0.75},"max":{"1min":1.73,"5min":1.73,"15min":2.278},"last":1.2050000000000001},{"interface":"front","average":{"1min":1.089,"5min":1.224,"15min":1.242},"min":{"1min":0.72899999999999998,"5min":0.72899999999999998,"15min":0.72899999999999998},"max":{"1min":1.732,"5min":2.2170000000000001,"15min":2.2170000000000001},"last":1.3080000000000001}]},{"osd":1,"last update":"Wed Aug 9 14:52:22 2023","interfaces":[{"interface":"back","average":{"1min":1.2629999999999999,"5min":1.246,"15min":1.2310000000000001},"min":{"1min":0.89900000000000002,"5min":0.88400000000000001,"15min":0.88400000000000001},"max":{"1min":2.0139999999999998,"5min":2.0139999999999998,"15min":2.0139999999999998},"last":1.1160000000000001},{"interface":"front","average":{"1min":1.2789999999999999,"5min":1.2629999999999999,"15min":1.2929999999999999},"min":{"1min":0.76500000000000001,"5min":0.76500000000000001,"15min":0.76500000000000001},"max":{"1min":1.9099999999999999,"5min":1.9099999999999999,"15min":1.9099999999999999},"last":1.712}]},{"osd":2,"last update":"Wed Aug 9 14:52:22 2023","interfaces":[{"interface":"back","average":{"1min":1.7090000000000001,"5min":1.421,"15min":1.3779999999999999},"min":{"1min":0.95799999999999996,"5min":0.94399999999999995,"15min":0.88100000000000001},"max":{"1min":4.9560000000000004,"5min":4.9560000000000004,"15min":4.9560000000000004},"last":1.8420000000000001},{"interface":"front","average":{"1min":1.669,"5min":1.4359999999999999,"15min":1.353},"min":{"1min":1.109,"5min":0.95999999999999996,"15min":0.90900000000000003},"max":{"1min":4.0140000000000002,"5min":4.0140000000000002,"15min":4.0140000000000002},"last":1.6579999999999999}]},{"osd":3,"last update":"Wed Aug 9 14:52:22 2023","interfaces":[{"interface":"back","average":{"1min":1.5149999999999999,"5min":1.446,"15min":1.419},"min":{"1min":0.90800000000000003,"5min":0.90800000000000003,"15min":0.90800000000000003},"max":{"1min":1.913,"5min":1.913,"15min":1.913},"last":1.786},{"interface":"front","average":{"1min":1.466,"5min":1.4279999999999999,"15min":1.3919999999999999},"min":{"1min":1.0369999999999999,"5min":0.97199999999999998,"15min":0.93300000000000005},"max":{"1min":2.1520000000000001,"5min":2.1520000000000001,"15min":2.1520000000000001},"last":1.6040000000000001}]},{"osd":4,"last update":"Wed Aug 9 14:52:22 2023","interfaces":[{"interface":"back","average":{"1min":1.5309999999999999,"5min":1.3839999999999999,"15min":1.3140000000000001},"min":{"1min":0.85399999999999998,"5min":0.82299999999999995,"15min":0.82199999999999995},"max":{"1min":2.2040000000000002,"5min":2.2040000000000002,"15min":2.2040000000000002},"last":2.0350000000000001},{"interface":"front","average":{"1min":1.6759999999999999,"5min":1.4990000000000001,"15min":1.3899999999999999},"min":{"1min":1.2030000000000001,"5min":0.871,"15min":0.71599999999999997},"max":{"1min":2.0430000000000001,"5min":2.0430000000000001,"15min":2.0430000000000001},"last":2.2280000000000002}]},{"osd":5,"last update":"Wed Aug 9 14:52:22 2023","interfaces":[{"interface":"back","average":{"1min":1.667,"5min":1.476,"15min":1.387},"min":{"1min":1.0900000000000001,"5min":0.83599999999999997,"15min":0.83599999999999997},"max":{"1min":2.2549999999999999,"5min":2.2549999999999999,"15min":2.2549999999999999},"last":1.9279999999999999},{"interface":"front","average":{"1min":1.7330000000000001,"5min":1.6519999999999999,"15min":1.6060000000000001},"min":{"1min":1.21,"5min":1.21,"15min":0.93500000000000005},"max":{"1min":2.3140000000000001,"5min":2.3140000000000001,"15min":2.3140000000000001},"last":2.1059999999999999}]},{"osd":7,"last update":"Wed Aug 9 14:52:22 2023","interfaces":[{"interface":"back","average":{"1min":1.798,"5min":1.7030000000000001,"15min":1.6120000000000001},"min":{"1min":1.409,"5min":1.175,"15min":0.96299999999999997},"max":{"1min":2.153,"5min":2.476,"15min":2.5310000000000001},"last":2.145},{"interface":"front","average":{"1min":1.851,"5min":1.6779999999999999,"15min":1.605},"min":{"1min":1.4119999999999999,"5min":1.0549999999999999,"15min":0.96999999999999997},"max":{"1min":2.3500000000000001,"5min":2.3500000000000001,"15min":2.3500000000000001},"last":2.0600000000000001}]}]},{"osd":5,"up_from":13,"seq":55834575787,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":59604,"kb_used_data":2244,"kb_used_omap":1,"kb_used_meta":57342,"kb_avail":94312236,"statfs":{"total":96636764160,"available":96575729664,"internally_reserved":0,"allocated":2297856,"data_stored":1242764,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1781,"internal_metadata":58718475},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Aug 9 14:52:14 2023","interfaces":[{"interface":"back","average":{"1min":1.2210000000000001,"5min":1.1830000000000001,"15min":1.1950000000000001},"min":{"1min":0.83399999999999996,"5min":0.72599999999999998,"15min":0.72599999999999998},"max":{"1min":1.714,"5min":1.792,"15min":1.976},"last":0.95399999999999996},{"interface":"front","average":{"1min":1.1679999999999999,"5min":1.2430000000000001,"15min":1.234},"min":{"1min":0.81000000000000005,"5min":0.81000000000000005,"15min":0.81000000000000005},"max":{"1min":1.79,"5min":1.9470000000000001,"15min":1.9470000000000001},"last":1.0169999999999999}]},{"osd":1,"last update":"Wed Aug 9 14:52:14 2023","interfaces":[{"interface":"back","average":{"1min":1.167,"5min":1.2769999999999999,"15min":1.258},"min":{"1min":0.85799999999999998,"5min":0.85799999999999998,"15min":0.82199999999999995},"max":{"1min":1.512,"5min":1.7929999999999999,"15min":1.7929999999999999},"last":1.071},{"interface":"front","average":{"1min":1.258,"5min":1.292,"15min":1.3200000000000001},"min":{"1min":0.72699999999999998,"5min":0.72699999999999998,"15min":0.72699999999999998},"max":{"1min":1.704,"5min":1.704,"15min":1.8480000000000001},"last":1.262}]},{"osd":2,"last update":"Wed Aug 9 14:52:14 2023","interfaces":[{"interface":"back","average":{"1min":1.2909999999999999,"5min":1.343,"15min":1.331},"min":{"1min":0.875,"5min":0.875,"15min":0.86199999999999999},"max":{"1min":1.7829999999999999,"5min":1.7829999999999999,"15min":1.8939999999999999},"last":1.393},{"interface":"front","average":{"1min":1.3120000000000001,"5min":1.355,"15min":1.3520000000000001},"min":{"1min":0.96699999999999997,"5min":0.96699999999999997,"15min":0.88800000000000001},"max":{"1min":1.7050000000000001,"5min":1.8300000000000001,"15min":1.8540000000000001},"last":1.444}]},{"osd":3,"last update":"Wed Aug 9 14:52:14 2023","interfaces":[{"interface":"back","average":{"1min":1.369,"5min":1.375,"15min":1.381},"min":{"1min":0.71899999999999997,"5min":0.71899999999999997,"15min":0.71899999999999997},"max":{"1min":1.794,"5min":1.794,"15min":2.04},"last":1.655},{"interface":"front","average":{"1min":1.504,"5min":1.4970000000000001,"15min":1.474},"min":{"1min":1.034,"5min":1.0009999999999999,"15min":0.98299999999999998},"max":{"1min":1.9710000000000001,"5min":1.9710000000000001,"15min":2.0409999999999999},"last":1.5600000000000001}]},{"osd":4,"last update":"Wed Aug 9 14:52:14 2023","interfaces":[{"interface":"back","average":{"1min":1.3999999999999999,"5min":1.349,"15min":1.3180000000000001},"min":{"1min":0.68300000000000005,"5min":0.68300000000000005,"15min":0.68300000000000005},"max":{"1min":2.2839999999999998,"5min":2.2839999999999998,"15min":2.2839999999999998},"last":1.329},{"interface":"front","average":{"1min":1.601,"5min":1.4450000000000001,"15min":1.4359999999999999},"min":{"1min":0.97199999999999998,"5min":0.94799999999999995,"15min":0.84499999999999997},"max":{"1min":2.2450000000000001,"5min":2.2450000000000001,"15min":2.2450000000000001},"last":1.796}]},{"osd":6,"last update":"Wed Aug 9 14:52:14 2023","interfaces":[{"interface":"back","average":{"1min":1.6879999999999999,"5min":1.4610000000000001,"15min":1.4410000000000001},"min":{"1min":1.1279999999999999,"5min":0.88600000000000001,"15min":0.79300000000000004},"max":{"1min":3.194,"5min":3.194,"15min":3.194},"last":1.744},{"interface":"front","average":{"1min":1.5860000000000001,"5min":1.502,"15min":1.4890000000000001},"min":{"1min":1.163,"5min":1.0169999999999999,"15min":0.871},"max":{"1min":2.1739999999999999,"5min":2.1739999999999999,"15min":2.1739999999999999},"last":1.621}]},{"osd":7,"last update":"Wed Aug 9 14:52:14 2023","interfaces":[{"interface":"back","average":{"1min":1.804,"5min":1.591,"15min":1.595},"min":{"1min":1.3420000000000001,"5min":1.054,"15min":0.98899999999999999},"max":{"1min":2.6150000000000002,"5min":2.6150000000000002,"15min":2.6150000000000002},"last":1.869},{"interface":"front","average":{"1min":1.8759999999999999,"5min":1.6719999999999999,"15min":1.6559999999999999},"min":{"1min":1.345,"5min":1.008,"15min":0.95099999999999996},"max":{"1min":3.6480000000000001,"5min":3.6480000000000001,"15min":3.6480000000000001},"last":1.9510000000000001}]}]},{"osd":4,"up_from":13,"seq":55834575787,"num_pgs":17,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":59604,"kb_used_data":2244,"kb_used_omap":1,"kb_used_meta":57342,"kb_avail":94312236,"statfs":{"total":96636764160,"available":96575729664,"internally_reserved":0,"allocated":2297856,"data_stored":1242764,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1781,"internal_metadata":58718475},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Aug 9 14:51:38 2023","interfaces":[{"interface":"back","average":{"1min":1.1859999999999999,"5min":1.2070000000000001,"15min":1.2050000000000001},"min":{"1min":0.69999999999999996,"5min":0.69999999999999996,"15min":0.69999999999999996},"max":{"1min":2.573,"5min":2.573,"15min":2.573},"last":1.548},{"interface":"front","average":{"1min":1.3460000000000001,"5min":1.341,"15min":1.3260000000000001},"min":{"1min":0.83799999999999997,"5min":0.83799999999999997,"15min":0.54000000000000004},"max":{"1min":2.1960000000000002,"5min":2.1960000000000002,"15min":2.1960000000000002},"last":1.8220000000000001}]},{"osd":1,"last update":"Wed Aug 9 14:51:38 2023","interfaces":[{"interface":"back","average":{"1min":1.49,"5min":1.3799999999999999,"15min":1.3540000000000001},"min":{"1min":0.86599999999999999,"5min":0.86599999999999999,"15min":0.60599999999999998},"max":{"1min":4.4299999999999997,"5min":4.4299999999999997,"15min":4.4299999999999997},"last":1.415},{"interface":"front","average":{"1min":1.204,"5min":1.292,"15min":1.2849999999999999},"min":{"1min":0.68200000000000005,"5min":0.68200000000000005,"15min":0.68200000000000005},"max":{"1min":2.4630000000000001,"5min":2.4630000000000001,"15min":2.4630000000000001},"last":1.0700000000000001}]},{"osd":2,"last update":"Wed Aug 9 14:51:38 2023","interfaces":[{"interface":"back","average":{"1min":1.335,"5min":1.3260000000000001,"15min":1.3109999999999999},"min":{"1min":0.81499999999999995,"5min":0.81499999999999995,"15min":0.74199999999999999},"max":{"1min":2.6920000000000002,"5min":2.6920000000000002,"15min":2.6920000000000002},"last":1.794},{"interface":"front","average":{"1min":1.462,"5min":1.454,"15min":1.4530000000000001},"min":{"1min":0.70899999999999996,"5min":0.70899999999999996,"15min":0.70899999999999996},"max":{"1min":2.0830000000000002,"5min":2.1619999999999999,"15min":2.4359999999999999},"last":1.141}]},{"osd":3,"last update":"Wed Aug 9 14:51:38 2023","interfaces":[{"interface":"back","average":{"1min":1.369,"5min":1.3360000000000001,"15min":1.339},"min":{"1min":0.85699999999999998,"5min":0.85699999999999998,"15min":0.85699999999999998},"max":{"1min":2.2069999999999999,"5min":2.2069999999999999,"15min":2.6429999999999998},"last":1.764},{"interface":"front","average":{"1min":1.534,"5min":1.4970000000000001,"15min":1.4750000000000001},"min":{"1min":0.75,"5min":0.75,"15min":0.75},"max":{"1min":3.0600000000000001,"5min":3.0600000000000001,"15min":3.0600000000000001},"last":1.365}]},{"osd":5,"last update":"Wed Aug 9 14:51:38 2023","interfaces":[{"interface":"back","average":{"1min":1.462,"5min":1.4099999999999999,"15min":1.377},"min":{"1min":0.77100000000000002,"5min":0.77100000000000002,"15min":0.70399999999999996},"max":{"1min":2.0710000000000002,"5min":2.5009999999999999,"15min":2.5009999999999999},"last":1.659},{"interface":"front","average":{"1min":1.343,"5min":1.403,"15min":1.385},"min":{"1min":0.56999999999999995,"5min":0.56999999999999995,"15min":0.56999999999999995},"max":{"1min":1.9790000000000001,"5min":2.4129999999999998,"15min":3.0110000000000001},"last":1.607}]},{"osd":6,"last update":"Wed Aug 9 14:51:38 2023","interfaces":[{"interface":"back","average":{"1min":1.589,"5min":1.4470000000000001,"15min":1.431},"min":{"1min":0.93200000000000005,"5min":0.85499999999999998,"15min":0.79500000000000004},"max":{"1min":2.9540000000000002,"5min":2.9540000000000002,"15min":2.9540000000000002},"last":1.208},{"interface":"front","average":{"1min":1.6739999999999999,"5min":1.619,"15min":1.5680000000000001},"min":{"1min":0.59899999999999998,"5min":0.59899999999999998,"15min":0.59899999999999998},"max":{"1min":2.0529999999999999,"5min":2.3260000000000001,"15min":2.5310000000000001},"last":1.2949999999999999}]},{"osd":7,"last update":"Wed Aug 9 14:51:38 2023","interfaces":[{"interface":"back","average":{"1min":1.5629999999999999,"5min":1.5569999999999999,"15min":1.5229999999999999},"min":{"1min":0.79100000000000004,"5min":0.79100000000000004,"15min":0.79100000000000004},"max":{"1min":2.1360000000000001,"5min":2.331,"15min":3.3149999999999999},"last":1.4650000000000001},{"interface":"front","average":{"1min":1.75,"5min":1.6339999999999999,"15min":1.587},"min":{"1min":0.88400000000000001,"5min":0.88400000000000001,"15min":0.82599999999999996},"max":{"1min":3.274,"5min":3.274,"15min":3.274},"last":1.7190000000000001}]}]},{"osd":3,"up_from":13,"seq":55834575788,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":59604,"kb_used_data":2244,"kb_used_omap":1,"kb_used_meta":57342,"kb_avail":94312236,"statfs":{"total":96636764160,"available":96575729664,"internally_reserved":0,"allocated":2297856,"data_stored":1242764,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1781,"internal_metadata":58718475},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Aug 9 14:52:20 2023","interfaces":[{"interface":"back","average":{"1min":0.86099999999999999,"5min":0.93799999999999994,"15min":0.94699999999999995},"min":{"1min":0.35699999999999998,"5min":0.35699999999999998,"15min":0.35699999999999998},"max":{"1min":1.26,"5min":1.722,"15min":1.8740000000000001},"last":0.70299999999999996},{"interface":"front","average":{"1min":0.91900000000000004,"5min":1.0820000000000001,"15min":1.1339999999999999},"min":{"1min":0.50600000000000001,"5min":0.50600000000000001,"15min":0.50600000000000001},"max":{"1min":1.613,"5min":1.9179999999999999,"15min":1.9179999999999999},"last":1.2889999999999999}]},{"osd":1,"last update":"Wed Aug 9 14:52:20 2023","interfaces":[{"interface":"back","average":{"1min":0.88900000000000001,"5min":1.0329999999999999,"15min":1.0640000000000001},"min":{"1min":0.50600000000000001,"5min":0.50600000000000001,"15min":0.50600000000000001},"max":{"1min":1.405,"5min":1.7050000000000001,"15min":1.903},"last":0.93300000000000005},{"interface":"front","average":{"1min":0.997,"5min":1.194,"15min":1.2090000000000001},"min":{"1min":0.58199999999999996,"5min":0.58199999999999996,"15min":0.58199999999999996},"max":{"1min":1.4630000000000001,"5min":1.8320000000000001,"15min":1.903},"last":1.0609999999999999}]},{"osd":2,"last update":"Wed Aug 9 14:52:20 2023","interfaces":[{"interface":"back","average":{"1min":1.038,"5min":1.3029999999999999,"15min":1.319},"min":{"1min":0.64400000000000002,"5min":0.64400000000000002,"15min":0.64400000000000002},"max":{"1min":1.4770000000000001,"5min":3.7770000000000001,"15min":3.7770000000000001},"last":1.2190000000000001},{"interface":"front","average":{"1min":1.1679999999999999,"5min":1.373,"15min":1.3999999999999999},"min":{"1min":0.58999999999999997,"5min":0.58999999999999997,"15min":0.58999999999999997},"max":{"1min":2.117,"5min":3.5750000000000002,"15min":3.5750000000000002},"last":1.581}]},{"osd":4,"last update":"Wed Aug 9 14:52:20 2023","interfaces":[{"interface":"back","average":{"1min":1.2869999999999999,"5min":1.3600000000000001,"15min":1.385},"min":{"1min":0.82699999999999996,"5min":0.82699999999999996,"15min":0.82699999999999996},"max":{"1min":1.6619999999999999,"5min":1.879,"15min":1.974},"last":0.97499999999999998},{"interface":"front","average":{"1min":1.2769999999999999,"5min":1.415,"15min":1.458},"min":{"1min":0.81499999999999995,"5min":0.79300000000000004,"15min":0.79300000000000004},"max":{"1min":1.613,"5min":1.992,"15min":2.3439999999999999},"last":1.361}]},{"osd":5,"last update":"Wed Aug 9 14:52:20 2023","interfaces":[{"interface":"back","average":{"1min":1.482,"5min":1.5349999999999999,"15min":1.554},"min":{"1min":0.746,"5min":0.746,"15min":0.746},"max":{"1min":1.9299999999999999,"5min":2.0339999999999998,"15min":2.2629999999999999},"last":1.48},{"interface":"front","average":{"1min":1.444,"5min":1.5289999999999999,"15min":1.55},"min":{"1min":0.97899999999999998,"5min":0.97899999999999998,"15min":0.97899999999999998},"max":{"1min":2.0539999999999998,"5min":2.3399999999999999,"15min":2.4630000000000001},"last":1.4139999999999999}]},{"osd":6,"last update":"Wed Aug 9 14:52:20 2023","interfaces":[{"interface":"back","average":{"1min":1.365,"5min":1.5169999999999999,"15min":1.524},"min":{"1min":0.69199999999999995,"5min":0.69199999999999995,"15min":0.69199999999999995},"max":{"1min":1.97,"5min":2.0790000000000002,"15min":2.1659999999999999},"last":1.5409999999999999},{"interface":"front","average":{"1min":1.4950000000000001,"5min":1.6120000000000001,"15min":1.631},"min":{"1min":0.84999999999999998,"5min":0.84999999999999998,"15min":0.84999999999999998},"max":{"1min":1.972,"5min":2.3140000000000001,"15min":2.4140000000000001},"last":1.629}]},{"osd":7,"last update":"Wed Aug 9 14:52:20 2023","interfaces":[{"interface":"back","average":{"1min":1.573,"5min":1.669,"15min":1.6850000000000001},"min":{"1min":1.032,"5min":1.032,"15min":1.032},"max":{"1min":2.0209999999999999,"5min":2.427,"15min":2.5430000000000001},"last":1.698},{"interface":"front","average":{"1min":1.492,"5min":1.679,"15min":1.667},"min":{"1min":1.0389999999999999,"5min":1.0389999999999999,"15min":1.0389999999999999},"max":{"1min":1.9790000000000001,"5min":2.2069999999999999,"15min":2.2069999999999999},"last":1.7250000000000001}]}]},{"osd":2,"up_from":13,"seq":55834575787,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":41548,"kb_used_data":2244,"kb_used_omap":1,"kb_used_meta":39294,"kb_avail":94330292,"statfs":{"total":96636764160,"available":96594219008,"internally_reserved":0,"allocated":2297856,"data_stored":1242764,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1781,"internal_metadata":40237323},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Aug 9 14:52:13 2023","interfaces":[{"interface":"back","average":{"1min":0.91700000000000004,"5min":0.91300000000000003,"15min":0.92300000000000004},"min":{"1min":0.46300000000000002,"5min":0.46300000000000002,"15min":0.46300000000000002},"max":{"1min":1.984,"5min":1.984,"15min":1.984},"last":1.3759999999999999},{"interface":"front","average":{"1min":0.94899999999999995,"5min":1.0660000000000001,"15min":1.101},"min":{"1min":0.41399999999999998,"5min":0.41399999999999998,"15min":0.41399999999999998},"max":{"1min":1.4359999999999999,"5min":1.8300000000000001,"15min":1.8740000000000001},"last":1.238}]},{"osd":1,"last update":"Wed Aug 9 14:52:13 2023","interfaces":[{"interface":"back","average":{"1min":0.997,"5min":1.143,"15min":1.2050000000000001},"min":{"1min":0.60099999999999998,"5min":0.59399999999999997,"15min":0.59399999999999997},"max":{"1min":1.573,"5min":1.6739999999999999,"15min":2.1600000000000001},"last":1.0840000000000001},{"interface":"front","average":{"1min":1.0620000000000001,"5min":1.3620000000000001,"15min":1.421},"min":{"1min":0.58199999999999996,"5min":0.58199999999999996,"15min":0.58199999999999996},"max":{"1min":1.74,"5min":2.0350000000000001,"15min":2.2200000000000002},"last":1.054}]},{"osd":3,"last update":"Wed Aug 9 14:52:13 2023","interfaces":[{"interface":"back","average":{"1min":0.93400000000000005,"5min":1.056,"15min":1.111},"min":{"1min":0.46000000000000002,"5min":0.46000000000000002,"15min":0.46000000000000002},"max":{"1min":1.472,"5min":1.7350000000000001,"15min":1.9239999999999999},"last":0.83399999999999996},{"interface":"front","average":{"1min":1.1839999999999999,"5min":1.345,"15min":1.3660000000000001},"min":{"1min":0.64500000000000002,"5min":0.64500000000000002,"15min":0.64500000000000002},"max":{"1min":1.877,"5min":2.0299999999999998,"15min":2.0299999999999998},"last":1.5189999999999999}]},{"osd":4,"last update":"Wed Aug 9 14:52:13 2023","interfaces":[{"interface":"back","average":{"1min":1.347,"5min":1.4910000000000001,"15min":1.508},"min":{"1min":0.89600000000000002,"5min":0.89600000000000002,"15min":0.89600000000000002},"max":{"1min":2.5750000000000002,"5min":2.5750000000000002,"15min":2.5750000000000002},"last":1.5840000000000001},{"interface":"front","average":{"1min":1.306,"5min":1.4390000000000001,"15min":1.4710000000000001},"min":{"1min":0.85799999999999998,"5min":0.85599999999999998,"15min":0.85599999999999998},"max":{"1min":2.1459999999999999,"5min":2.2509999999999999,"15min":2.2509999999999999},"last":1.3400000000000001}]},{"osd":5,"last update":"Wed Aug 9 14:52:13 2023","interfaces":[{"interface":"back","average":{"1min":1.4370000000000001,"5min":1.538,"15min":1.5389999999999999},"min":{"1min":0.97099999999999997,"5min":0.90100000000000002,"15min":0.90100000000000002},"max":{"1min":2.3919999999999999,"5min":2.3919999999999999,"15min":2.3919999999999999},"last":1.4390000000000001},{"interface":"front","average":{"1min":1.323,"5min":1.4530000000000001,"15min":1.472},"min":{"1min":0.85999999999999999,"5min":0.85999999999999999,"15min":0.85999999999999999},"max":{"1min":2.202,"5min":2.202,"15min":2.202},"last":1.1839999999999999}]},{"osd":6,"last update":"Wed Aug 9 14:52:13 2023","interfaces":[{"interface":"back","average":{"1min":1.421,"5min":1.51,"15min":1.508},"min":{"1min":0.67800000000000005,"5min":0.67800000000000005,"15min":0.67800000000000005},"max":{"1min":2.5289999999999999,"5min":2.5289999999999999,"15min":2.5289999999999999},"last":1.2929999999999999},{"interface":"front","average":{"1min":1.518,"5min":1.6000000000000001,"15min":1.615},"min":{"1min":1.012,"5min":0.98699999999999999,"15min":0.98699999999999999},"max":{"1min":2.448,"5min":2.448,"15min":2.5070000000000001},"last":1.871}]},{"osd":7,"last update":"Wed Aug 9 14:52:13 2023","interfaces":[{"interface":"back","average":{"1min":1.4710000000000001,"5min":1.6479999999999999,"15min":1.6519999999999999},"min":{"1min":0.82899999999999996,"5min":0.82899999999999996,"15min":0.82899999999999996},"max":{"1min":2.258,"5min":2.419,"15min":2.419},"last":1.5600000000000001},{"interface":"front","average":{"1min":1.52,"5min":1.615,"15min":1.6180000000000001},"min":{"1min":0.98499999999999999,"5min":0.98499999999999999,"15min":0.98499999999999999},"max":{"1min":2.3290000000000002,"5min":2.3290000000000002,"15min":2.3399999999999999},"last":1.407}]}]},{"osd":1,"up_from":13,"seq":55834575787,"num_pgs":19,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":51992,"kb_used_data":2824,"kb_used_omap":1,"kb_used_meta":49150,"kb_avail":94319848,"statfs":{"total":96636764160,"available":96583524352,"internally_reserved":0,"allocated":2891776,"data_stored":1833132,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1781,"internal_metadata":50329867},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Aug 9 14:52:26 2023","interfaces":[{"interface":"back","average":{"1min":0.96499999999999997,"5min":0.995,"15min":0.999},"min":{"1min":0.39000000000000001,"5min":0.308,"15min":0.308},"max":{"1min":3.169,"5min":3.169,"15min":3.169},"last":0.42399999999999999},{"interface":"front","average":{"1min":1.0029999999999999,"5min":1.135,"15min":1.1259999999999999},"min":{"1min":0.50600000000000001,"5min":0.50600000000000001,"15min":0.50600000000000001},"max":{"1min":2.895,"5min":2.895,"15min":2.895},"last":0.57799999999999996}]},{"osd":2,"last update":"Wed Aug 9 14:52:26 2023","interfaces":[{"interface":"back","average":{"1min":0.94699999999999995,"5min":1.0780000000000001,"15min":1.079},"min":{"1min":0.38200000000000001,"5min":0.38200000000000001,"15min":0.38200000000000001},"max":{"1min":1.298,"5min":2.71,"15min":2.71},"last":0.60099999999999998},{"interface":"front","average":{"1min":1.1659999999999999,"5min":1.238,"15min":1.2589999999999999},"min":{"1min":0.46300000000000002,"5min":0.46300000000000002,"15min":0.46300000000000002},"max":{"1min":3.2919999999999998,"5min":3.2919999999999998,"15min":3.2919999999999998},"last":0.67900000000000005}]},{"osd":3,"last update":"Wed Aug 9 14:52:26 2023","interfaces":[{"interface":"back","average":{"1min":1.298,"5min":1.383,"15min":1.397},"min":{"1min":0.53400000000000003,"5min":0.53400000000000003,"15min":0.53400000000000003},"max":{"1min":3.5790000000000002,"5min":3.5790000000000002,"15min":3.5790000000000002},"last":0.629},{"interface":"front","average":{"1min":1.3169999999999999,"5min":1.3759999999999999,"15min":1.3720000000000001},"min":{"1min":0.56999999999999995,"5min":0.56999999999999995,"15min":0.56999999999999995},"max":{"1min":3.4580000000000002,"5min":3.4580000000000002,"15min":3.4580000000000002},"last":0.54100000000000004}]},{"osd":4,"last update":"Wed Aug 9 14:52:26 2023","interfaces":[{"interface":"back","average":{"1min":1.425,"5min":1.504,"15min":1.476},"min":{"1min":0.73399999999999999,"5min":0.73399999999999999,"15min":0.73399999999999999},"max":{"1min":2.6949999999999998,"5min":2.6949999999999998,"15min":2.6949999999999998},"last":0.751},{"interface":"front","average":{"1min":1.3500000000000001,"5min":1.452,"15min":1.431},"min":{"1min":0.71899999999999997,"5min":0.71899999999999997,"15min":0.71899999999999997},"max":{"1min":1.855,"5min":3.2440000000000002,"15min":3.2440000000000002},"last":0.94599999999999995}]},{"osd":5,"last update":"Wed Aug 9 14:52:26 2023","interfaces":[{"interface":"back","average":{"1min":1.3799999999999999,"5min":1.4610000000000001,"15min":1.4510000000000001},"min":{"1min":0.61499999999999999,"5min":0.61499999999999999,"15min":0.61499999999999999},"max":{"1min":1.9830000000000001,"5min":3.036,"15min":3.036},"last":0.86099999999999999},{"interface":"front","average":{"1min":1.5960000000000001,"5min":1.611,"15min":1.601},"min":{"1min":0.70799999999999996,"5min":0.70799999999999996,"15min":0.70799999999999996},"max":{"1min":3.6930000000000001,"5min":3.6930000000000001,"15min":3.6930000000000001},"last":0.78700000000000003}]},{"osd":6,"last update":"Wed Aug 9 14:52:26 2023","interfaces":[{"interface":"back","average":{"1min":1.601,"5min":1.625,"15min":1.6080000000000001},"min":{"1min":0.78000000000000003,"5min":0.78000000000000003,"15min":0.78000000000000003},"max":{"1min":2.5750000000000002,"5min":3.149,"15min":3.149},"last":0.997},{"interface":"front","average":{"1min":1.48,"5min":1.585,"15min":1.5900000000000001},"min":{"1min":0.51800000000000002,"5min":0.51800000000000002,"15min":0.51800000000000002},"max":{"1min":2.2189999999999999,"5min":3.4460000000000002,"15min":3.4460000000000002},"last":0.91000000000000003}]},{"osd":7,"last update":"Wed Aug 9 14:52:26 2023","interfaces":[{"interface":"back","average":{"1min":1.708,"5min":1.6879999999999999,"15min":1.694},"min":{"1min":0.90900000000000003,"5min":0.79000000000000004,"15min":0.79000000000000004},"max":{"1min":3.835,"5min":3.835,"15min":3.835},"last":0.92700000000000005},{"interface":"front","average":{"1min":1.6870000000000001,"5min":1.738,"15min":1.728},"min":{"1min":0.88700000000000001,"5min":0.88700000000000001,"15min":0.88700000000000001},"max":{"1min":2.8029999999999999,"5min":2.8029999999999999,"15min":2.8029999999999999},"last":1.1459999999999999}]}]},{"osd":0,"up_from":13,"seq":55834575787,"num_pgs":17,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":29256,"kb_used_data":2240,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94342584,"statfs":{"total":96636764160,"available":96606806016,"internally_reserved":0,"allocated":2293760,"data_stored":1241403,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1781,"internal_metadata":27654411},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Wed Aug 9 14:52:11 2023","interfaces":[{"interface":"back","average":{"1min":0.78700000000000003,"5min":0.91300000000000003,"15min":0.96099999999999997},"min":{"1min":0.38500000000000001,"5min":0.38500000000000001,"15min":0.38500000000000001},"max":{"1min":1.444,"5min":1.456,"15min":1.893},"last":0.60299999999999998},{"interface":"front","average":{"1min":0.93799999999999994,"5min":1.1579999999999999,"15min":1.1399999999999999},"min":{"1min":0.40999999999999998,"5min":0.40999999999999998,"15min":0.40999999999999998},"max":{"1min":1.4930000000000001,"5min":1.8520000000000001,"15min":1.8520000000000001},"last":0.44}]},{"osd":2,"last update":"Wed Aug 9 14:52:11 2023","interfaces":[{"interface":"back","average":{"1min":1.0329999999999999,"5min":1.157,"15min":1.2050000000000001},"min":{"1min":0.54300000000000004,"5min":0.54300000000000004,"15min":0.54300000000000004},"max":{"1min":1.5700000000000001,"5min":1.6910000000000001,"15min":2.698},"last":0.66500000000000004},{"interface":"front","average":{"1min":1.0309999999999999,"5min":1.2270000000000001,"15min":1.2250000000000001},"min":{"1min":0.42699999999999999,"5min":0.42699999999999999,"15min":0.42699999999999999},"max":{"1min":1.649,"5min":1.895,"15min":1.895},"last":0.67800000000000005}]},{"osd":3,"last update":"Wed Aug 9 14:52:11 2023","interfaces":[{"interface":"back","average":{"1min":0.89000000000000001,"5min":1.1719999999999999,"15min":1.2230000000000001},"min":{"1min":0.47299999999999998,"5min":0.47299999999999998,"15min":0.47299999999999998},"max":{"1min":1.5249999999999999,"5min":2.0049999999999999,"15min":2.1230000000000002},"last":0.627},{"interface":"front","average":{"1min":1.1839999999999999,"5min":1.349,"15min":1.393},"min":{"1min":0.44600000000000001,"5min":0.44600000000000001,"15min":0.44600000000000001},"max":{"1min":1.728,"5min":1.829,"15min":2.3580000000000001},"last":0.71599999999999997}]},{"osd":4,"last update":"Wed Aug 9 14:52:11 2023","interfaces":[{"interface":"back","average":{"1min":1.3,"5min":1.387,"15min":1.3799999999999999},"min":{"1min":0.52800000000000002,"5min":0.52800000000000002,"15min":0.52800000000000002},"max":{"1min":1.893,"5min":2.3730000000000002,"15min":2.3730000000000002},"last":0.92900000000000005},{"interface":"front","average":{"1min":1.3089999999999999,"5min":1.4770000000000001,"15min":1.51},"min":{"1min":0.58699999999999997,"5min":0.58699999999999997,"15min":0.58699999999999997},"max":{"1min":1.843,"5min":2.125,"15min":2.149},"last":0.86799999999999999}]},{"osd":5,"last update":"Wed Aug 9 14:52:11 2023","interfaces":[{"interface":"back","average":{"1min":1.3340000000000001,"5min":1.512,"15min":1.52},"min":{"1min":0.55700000000000005,"5min":0.55700000000000005,"15min":0.55700000000000005},"max":{"1min":1.901,"5min":2.0110000000000001,"15min":2.097},"last":1.0149999999999999},{"interface":"front","average":{"1min":1.4179999999999999,"5min":1.4650000000000001,"15min":1.504},"min":{"1min":0.60699999999999998,"5min":0.60699999999999998,"15min":0.60699999999999998},"max":{"1min":2.0099999999999998,"5min":2.2320000000000002,"15min":2.266},"last":0.88400000000000001}]},{"osd":6,"last update":"Wed Aug 9 14:52:11 2023","interfaces":[{"interface":"back","average":{"1min":1.456,"5min":1.6080000000000001,"15min":1.639},"min":{"1min":0.63900000000000001,"5min":0.63900000000000001,"15min":0.63900000000000001},"max":{"1min":1.956,"5min":2.4260000000000002,"15min":3.0750000000000002},"last":0.97699999999999998},{"interface":"front","average":{"1min":1.377,"5min":1.6000000000000001,"15min":1.6180000000000001},"min":{"1min":0.67200000000000004,"5min":0.67200000000000004,"15min":0.67200000000000004},"max":{"1min":2.0800000000000001,"5min":2.2429999999999999,"15min":2.2429999999999999},"last":0.84499999999999997}]},{"osd":7,"last update":"Wed Aug 9 14:52:11 2023","interfaces":[{"interface":"back","average":{"1min":1.5129999999999999,"5min":1.6910000000000001,"15min":1.7010000000000001},"min":{"1min":0.70299999999999996,"5min":0.70299999999999996,"15min":0.70299999999999996},"max":{"1min":1.984,"5min":2.1339999999999999,"15min":2.3490000000000002},"last":0.90900000000000003},{"interface":"front","average":{"1min":1.4590000000000001,"5min":1.597,"15min":1.6100000000000001},"min":{"1min":0.65500000000000003,"5min":0.65500000000000003,"15min":0.65500000000000003},"max":{"1min":2.3940000000000001,"5min":2.4500000000000002,"15min":2.468},"last":0.999}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":22,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":22,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":34,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":22,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":22,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":34,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":22,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1682,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":22,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":604,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":22,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":22,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":22,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":2354,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":23,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":23,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":23,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":23,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":23,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":23,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":23,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":23,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-08-09T14:52:35.798 INFO:tasks.cephfs.filesystem.ceph_manager:clean! 2023-08-09T14:52:35.798 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth caps client.0 mon 'allow r' osd 'allow rw tag cephfs data=cephfs' mds allow 2023-08-09T14:52:36.077 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:36.077+0000 7f8431aad640 1 Processor -- start 2023-08-09T14:52:36.078 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:36.077+0000 7f8431aad640 1 -- start start 2023-08-09T14:52:36.079 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:36.079+0000 7f8431aad640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f842c0d04b0 0x7f842c0cd6d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T14:52:36.079 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:36.079+0000 7f8431aad640 1 -- --> v1:172.21.15.43:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f842c0b7bd0 con 0x7f842c0b45f0 2023-08-09T14:52:36.079 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:36.079+0000 7f8431aad640 1 -- --> v1:172.21.15.203:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f842c0d1c00 con 0x7f842c15c550 2023-08-09T14:52:36.080 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:36.079+0000 7f8431aad640 1 -- --> v2:172.21.15.43:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f842c15c930 con 0x7f842c0d04b0 2023-08-09T14:52:36.080 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:36.079+0000 7f8422ffd640 1 --1- >> v1:172.21.15.43:6789/0 conn(0x7f842c0b45f0 0x7f842c0b59d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.43:6789/0 says I am v1:172.21.15.43:46224/0 (socket says 172.21.15.43:46224) 2023-08-09T14:52:36.080 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:36.079+0000 7f8422ffd640 1 -- 172.21.15.43:0/3761411442 learned_addr learned my addr 172.21.15.43:0/3761411442 (peer_addr_for_me v1:172.21.15.43:0/0) 2023-08-09T14:52:36.081 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:36.079+0000 7f842bfff640 1 --2- 172.21.15.43:0/3761411442 >> v2:172.21.15.43:3300/0 conn(0x7f842c0d04b0 0x7f842c0cd6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T14:52:36.081 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:36.080+0000 7f842affd640 1 -- 172.21.15.43:0/3761411442 <== mon.0 v1:172.21.15.43:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1676180356 0 0) 0x7f842c0b7bd0 con 0x7f842c0b45f0 2023-08-09T14:52:36.081 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:36.080+0000 7f842bfff640 1 -- 172.21.15.43:0/3761411442 >> v1:172.21.15.203:6789/0 conn(0x7f842c15c550 legacy=0x7f842c0b3ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T14:52:36.081 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:36.080+0000 7f842bfff640 1 -- 172.21.15.43:0/3761411442 >> v1:172.21.15.43:6789/0 conn(0x7f842c0b45f0 legacy=0x7f842c0b59d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T14:52:36.081 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:36.080+0000 7f842bfff640 1 -- 172.21.15.43:0/3761411442 --> v2:172.21.15.43:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f842c0b54f0 con 0x7f842c0d04b0 2023-08-09T14:52:36.082 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:36.081+0000 7f842bfff640 1 --2- 172.21.15.43:0/3761411442 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f842c0d04b0 0x7f842c0cd6d0 secure :-1 s=READY pgs=2049 cs=0 l=1 rev1=1 crypto rx=0x7f84240097a0 tx=0x7f842402b960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=963daf08fe689941 server_cookie=0 in_seq=0 out_seq=0 2023-08-09T14:52:36.082 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:36.081+0000 7f842affd640 1 -- 172.21.15.43:0/3761411442 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8424030610 con 0x7f842c0d04b0 2023-08-09T14:52:36.082 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:36.082+0000 7f842affd640 1 -- 172.21.15.43:0/3761411442 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8424030770 con 0x7f842c0d04b0 2023-08-09T14:52:36.082 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:36.082+0000 7f842affd640 1 -- 172.21.15.43:0/3761411442 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8424030b00 con 0x7f842c0d04b0 2023-08-09T14:52:36.083 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:36.082+0000 7f8431aad640 1 -- 172.21.15.43:0/3761411442 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f842c0d04b0 msgr2=0x7f842c0cd6d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T14:52:36.083 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:36.082+0000 7f8431aad640 1 --2- 172.21.15.43:0/3761411442 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f842c0d04b0 0x7f842c0cd6d0 secure :-1 s=READY pgs=2049 cs=0 l=1 rev1=1 crypto rx=0x7f84240097a0 tx=0x7f842402b960 comp rx=0 tx=0).stop 2023-08-09T14:52:36.083 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:36.082+0000 7f8431aad640 1 -- 172.21.15.43:0/3761411442 shutdown_connections 2023-08-09T14:52:36.083 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:36.082+0000 7f8431aad640 1 --2- 172.21.15.43:0/3761411442 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f842c0d04b0 0x7f842c0cd6d0 unknown :-1 s=CLOSED pgs=2049 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T14:52:36.083 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:36.082+0000 7f8431aad640 1 -- 172.21.15.43:0/3761411442 >> 172.21.15.43:0/3761411442 conn(0x7f842c071520 msgr2=0x7f842c0ceee0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-09T14:52:36.084 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:36.082+0000 7f8431aad640 1 -- 172.21.15.43:0/3761411442 shutdown_connections 2023-08-09T14:52:36.084 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:36.082+0000 7f8431aad640 1 -- 172.21.15.43:0/3761411442 wait complete. 2023-08-09T14:52:36.084 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:36.084+0000 7f8431aad640 1 Processor -- start 2023-08-09T14:52:36.084 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:36.084+0000 7f8431aad640 1 -- start start 2023-08-09T14:52:36.085 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:36.085+0000 7f8431aad640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f842c0b45f0 0x7f842c0a8dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T14:52:36.086 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:36.085+0000 7f8431aad640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f842c0d04b0 0x7f842c0cb0b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T14:52:36.086 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:36.085+0000 7f8431aad640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f842c15c550 0x7f842c1f29b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T14:52:36.086 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:36.085+0000 7f8431aad640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f842c0a94c0 con 0x7f842c0d04b0 2023-08-09T14:52:36.086 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:36.085+0000 7f8431aad640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f842c0a97a0 con 0x7f842c0b45f0 2023-08-09T14:52:36.086 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:36.085+0000 7f8431aad640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f842c0a9a80 con 0x7f842c15c550 2023-08-09T14:52:36.087 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:36.086+0000 7f8422ffd640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f842c0d04b0 0x7f842c0cb0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T14:52:36.088 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:36.086+0000 7f8422ffd640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f842c0d04b0 0x7f842c0cb0b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:36318/0 (socket says 172.21.15.43:36318) 2023-08-09T14:52:36.088 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:36.086+0000 7f8422ffd640 1 -- 172.21.15.43:0/3115589431 learned_addr learned my addr 172.21.15.43:0/3115589431 (peer_addr_for_me v2:172.21.15.43:0/0) 2023-08-09T14:52:36.088 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:36.086+0000 7f842bfff640 1 --2- 172.21.15.43:0/3115589431 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f842c15c550 0x7f842c1f29b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T14:52:36.088 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:36.086+0000 7f842b7fe640 1 --2- 172.21.15.43:0/3115589431 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f842c0b45f0 0x7f842c0a8dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T14:52:36.088 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:36.086+0000 7f8422ffd640 1 -- 172.21.15.43:0/3115589431 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f842c15c550 msgr2=0x7f842c1f29b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T14:52:36.088 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:36.086+0000 7f8422ffd640 1 --2- 172.21.15.43:0/3115589431 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f842c15c550 0x7f842c1f29b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T14:52:36.089 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:36.086+0000 7f8422ffd640 1 -- 172.21.15.43:0/3115589431 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f842c0b45f0 msgr2=0x7f842c0a8dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T14:52:36.089 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:36.086+0000 7f8422ffd640 1 --2- 172.21.15.43:0/3115589431 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f842c0b45f0 0x7f842c0a8dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T14:52:36.089 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:36.086+0000 7f8422ffd640 1 -- 172.21.15.43:0/3115589431 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8424009400 con 0x7f842c0d04b0 2023-08-09T14:52:36.089 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:36.086+0000 7f842bfff640 1 --2- 172.21.15.43:0/3115589431 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f842c15c550 0x7f842c1f29b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-08-09T14:52:36.089 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:36.086+0000 7f842b7fe640 1 --2- 172.21.15.43:0/3115589431 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f842c0b45f0 0x7f842c0a8dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-08-09T14:52:36.089 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:36.086+0000 7f8422ffd640 1 --2- 172.21.15.43:0/3115589431 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f842c0d04b0 0x7f842c0cb0b0 secure :-1 s=READY pgs=2050 cs=0 l=1 rev1=1 crypto rx=0x7f841400c4a0 tx=0x7f841400c960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-09T14:52:36.089 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:36.087+0000 7f8428ff9640 1 -- 172.21.15.43:0/3115589431 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f84140074e0 con 0x7f842c0d04b0 2023-08-09T14:52:36.090 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:36.087+0000 7f8428ff9640 1 -- 172.21.15.43:0/3115589431 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8414007640 con 0x7f842c0d04b0 2023-08-09T14:52:36.090 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:36.087+0000 7f8431aad640 1 -- 172.21.15.43:0/3115589431 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f842c0a9d60 con 0x7f842c0d04b0 2023-08-09T14:52:36.090 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:36.087+0000 7f8428ff9640 1 -- 172.21.15.43:0/3115589431 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8414004a00 con 0x7f842c0d04b0 2023-08-09T14:52:36.090 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:36.087+0000 7f8431aad640 1 -- 172.21.15.43:0/3115589431 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f842c1f2ef0 con 0x7f842c0d04b0 2023-08-09T14:52:36.091 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:36.088+0000 7f8431aad640 1 -- 172.21.15.43:0/3115589431 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f842c0739f0 con 0x7f842c0d04b0 2023-08-09T14:52:36.091 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:36.089+0000 7f8428ff9640 1 -- 172.21.15.43:0/3115589431 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 88) v1 ==== 88443+0+0 (secure 0 0 0) 0x7f8414004b60 con 0x7f842c0d04b0 2023-08-09T14:52:36.091 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:36.090+0000 7f8428ff9640 1 --2- 172.21.15.43:0/3115589431 >> [v2:172.21.15.203:6832/2959400181,v1:172.21.15.203:6833/2959400181] conn(0x7f840c06a170 0x7f840c06c630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T14:52:36.091 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:36.090+0000 7f8428ff9640 1 -- 172.21.15.43:0/3115589431 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(186..186 src has 1..186) v4 ==== 6426+0+0 (secure 0 0 0) 0x7f8414087e20 con 0x7f842c0d04b0 2023-08-09T14:52:36.091 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:36.091+0000 7f842b7fe640 1 --2- 172.21.15.43:0/3115589431 >> [v2:172.21.15.203:6832/2959400181,v1:172.21.15.203:6833/2959400181] conn(0x7f840c06a170 0x7f840c06c630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T14:52:36.092 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:36.092+0000 7f842b7fe640 1 --2- 172.21.15.43:0/3115589431 >> [v2:172.21.15.203:6832/2959400181,v1:172.21.15.203:6833/2959400181] conn(0x7f840c06a170 0x7f840c06c630 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f841c0012e0 tx=0x7f841c009040 comp rx=0 tx=0).ready entity=mgr.7223 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-09T14:52:36.095 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:36.094+0000 7f8428ff9640 1 -- 172.21.15.43:0/3115589431 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178066 (secure 0 0 0) 0x7f84140531f0 con 0x7f842c0d04b0 2023-08-09T14:52:36.362 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:36.362+0000 7f8431aad640 1 -- 172.21.15.43:0/3115589431 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]} v 0) v1 -- 0x7f842c0aa280 con 0x7f842c0d04b0 2023-08-09T14:52:36.372 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:36.372+0000 7f8428ff9640 1 -- 172.21.15.43:0/3115589431 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]}]=0 updated caps for client.0 v41) v1 ==== 190+0+147 (secure 0 0 0) 0x7f84140180e0 con 0x7f842c0d04b0 2023-08-09T14:52:36.373 INFO:teuthology.orchestra.run.smithi043.stdout:[client.0] 2023-08-09T14:52:36.373 INFO:teuthology.orchestra.run.smithi043.stdout: key = AQCPldNkg8zrDxAAM/hnJaE6S9HU/fRwQeBV5w== 2023-08-09T14:52:36.373 INFO:teuthology.orchestra.run.smithi043.stdout: caps mds = "allow" 2023-08-09T14:52:36.373 INFO:teuthology.orchestra.run.smithi043.stdout: caps mon = "allow r" 2023-08-09T14:52:36.373 INFO:teuthology.orchestra.run.smithi043.stdout: caps osd = "allow rw tag cephfs data=cephfs" 2023-08-09T14:52:36.374 INFO:teuthology.orchestra.run.smithi043.stderr:updated caps for client.0 2023-08-09T14:52:36.375 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:36.374+0000 7f8431aad640 1 -- 172.21.15.43:0/3115589431 >> [v2:172.21.15.203:6832/2959400181,v1:172.21.15.203:6833/2959400181] conn(0x7f840c06a170 msgr2=0x7f840c06c630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T14:52:36.376 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:36.374+0000 7f8431aad640 1 --2- 172.21.15.43:0/3115589431 >> [v2:172.21.15.203:6832/2959400181,v1:172.21.15.203:6833/2959400181] conn(0x7f840c06a170 0x7f840c06c630 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f841c0012e0 tx=0x7f841c009040 comp rx=0 tx=0).stop 2023-08-09T14:52:36.376 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:36.374+0000 7f8431aad640 1 -- 172.21.15.43:0/3115589431 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f842c0d04b0 msgr2=0x7f842c0cb0b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T14:52:36.376 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:36.374+0000 7f8431aad640 1 --2- 172.21.15.43:0/3115589431 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f842c0d04b0 0x7f842c0cb0b0 secure :-1 s=READY pgs=2050 cs=0 l=1 rev1=1 crypto rx=0x7f841400c4a0 tx=0x7f841400c960 comp rx=0 tx=0).stop 2023-08-09T14:52:36.376 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:36.374+0000 7f8431aad640 1 -- 172.21.15.43:0/3115589431 shutdown_connections 2023-08-09T14:52:36.376 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:36.374+0000 7f8431aad640 1 --2- 172.21.15.43:0/3115589431 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f842c15c550 0x7f842c1f29b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T14:52:36.376 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:36.374+0000 7f8431aad640 1 --2- 172.21.15.43:0/3115589431 >> [v2:172.21.15.203:6832/2959400181,v1:172.21.15.203:6833/2959400181] conn(0x7f840c06a170 0x7f840c06c630 unknown :-1 s=CLOSED pgs=68 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T14:52:36.377 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:36.374+0000 7f8431aad640 1 --2- 172.21.15.43:0/3115589431 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f842c0d04b0 0x7f842c0cb0b0 unknown :-1 s=CLOSED pgs=2050 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T14:52:36.377 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:36.374+0000 7f8431aad640 1 --2- 172.21.15.43:0/3115589431 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f842c0b45f0 0x7f842c0a8dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T14:52:36.377 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:36.374+0000 7f8431aad640 1 -- 172.21.15.43:0/3115589431 >> 172.21.15.43:0/3115589431 conn(0x7f842c071520 msgr2=0x7f842c0c8d90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-09T14:52:36.377 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:36.375+0000 7f8431aad640 1 -- 172.21.15.43:0/3115589431 shutdown_connections 2023-08-09T14:52:36.377 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:36.375+0000 7f8431aad640 1 -- 172.21.15.43:0/3115589431 wait complete. 2023-08-09T14:52:36.395 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-08-09T14:52:36.627 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:36.626+0000 7f30a5bdd640 1 Processor -- start 2023-08-09T14:52:36.627 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:36.627+0000 7f30a5bdd640 1 -- start start 2023-08-09T14:52:36.629 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:36.628+0000 7f30a5bdd640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f30a00acdf0 0x7f30a00ad340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T14:52:36.629 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:36.628+0000 7f30a5bdd640 1 -- --> v1:172.21.15.43:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f30a00cbd90 con 0x7f30a00c8b30 2023-08-09T14:52:36.630 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:36.629+0000 7f30a5bdd640 1 -- --> v1:172.21.15.203:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f30a00ca2b0 con 0x7f30a00c8750 2023-08-09T14:52:36.630 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:36.629+0000 7f30a5bdd640 1 -- --> v2:172.21.15.43:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f30a015e810 con 0x7f30a00acdf0 2023-08-09T14:52:36.630 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:36.629+0000 7f3097fff640 1 --1- >> v1:172.21.15.43:6789/0 conn(0x7f30a00c8b30 0x7f30a0070f00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.43:6789/0 says I am v1:172.21.15.43:46238/0 (socket says 172.21.15.43:46238) 2023-08-09T14:52:36.630 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:36.629+0000 7f3097fff640 1 -- 172.21.15.43:0/398007774 learned_addr learned my addr 172.21.15.43:0/398007774 (peer_addr_for_me v1:172.21.15.43:0/0) 2023-08-09T14:52:36.630 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:36.629+0000 7f30a53dc640 1 --2- 172.21.15.43:0/398007774 >> v2:172.21.15.43:3300/0 conn(0x7f30a00acdf0 0x7f30a00ad340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T14:52:36.631 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:36.629+0000 7f30977fe640 1 -- 172.21.15.43:0/398007774 <== mon.0 v1:172.21.15.43:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4053389416 0 0) 0x7f30a00cbd90 con 0x7f30a00c8b30 2023-08-09T14:52:36.632 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:36.629+0000 7f30a53dc640 1 -- 172.21.15.43:0/398007774 >> v1:172.21.15.203:6789/0 conn(0x7f30a00c8750 legacy=0x7f30a006f410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-09T14:52:36.632 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:36.630+0000 7f30a53dc640 1 -- 172.21.15.43:0/398007774 >> v1:172.21.15.43:6789/0 conn(0x7f30a00c8b30 legacy=0x7f30a0070f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T14:52:36.632 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:36.630+0000 7f30a53dc640 1 -- 172.21.15.43:0/398007774 --> v2:172.21.15.43:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f30a00c9a20 con 0x7f30a00acdf0 2023-08-09T14:52:36.632 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:36.631+0000 7f30a53dc640 1 --2- 172.21.15.43:0/398007774 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f30a00acdf0 0x7f30a00ad340 secure :-1 s=READY pgs=2052 cs=0 l=1 rev1=1 crypto rx=0x7f3098009d90 tx=0x7f309802beb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=370df5cb0ed0a973 server_cookie=0 in_seq=0 out_seq=0 2023-08-09T14:52:36.632 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:36.631+0000 7f30977fe640 1 -- 172.21.15.43:0/398007774 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f309802cc60 con 0x7f30a00acdf0 2023-08-09T14:52:36.633 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:36.631+0000 7f30977fe640 1 -- 172.21.15.43:0/398007774 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f309802cdc0 con 0x7f30a00acdf0 2023-08-09T14:52:36.633 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:36.631+0000 7f30977fe640 1 -- 172.21.15.43:0/398007774 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3098034660 con 0x7f30a00acdf0 2023-08-09T14:52:36.633 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:36.632+0000 7f30a5bdd640 1 -- 172.21.15.43:0/398007774 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f30a00acdf0 msgr2=0x7f30a00ad340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T14:52:36.633 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:36.632+0000 7f30a5bdd640 1 --2- 172.21.15.43:0/398007774 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f30a00acdf0 0x7f30a00ad340 secure :-1 s=READY pgs=2052 cs=0 l=1 rev1=1 crypto rx=0x7f3098009d90 tx=0x7f309802beb0 comp rx=0 tx=0).stop 2023-08-09T14:52:36.634 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:36.632+0000 7f30a5bdd640 1 -- 172.21.15.43:0/398007774 shutdown_connections 2023-08-09T14:52:36.634 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:36.632+0000 7f30a5bdd640 1 --2- 172.21.15.43:0/398007774 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f30a00acdf0 0x7f30a00ad340 unknown :-1 s=CLOSED pgs=2052 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T14:52:36.634 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:36.632+0000 7f30a5bdd640 1 -- 172.21.15.43:0/398007774 >> 172.21.15.43:0/398007774 conn(0x7f30a00cde90 msgr2=0x7f30a00ce2a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-09T14:52:36.634 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:36.632+0000 7f30a5bdd640 1 -- 172.21.15.43:0/398007774 shutdown_connections 2023-08-09T14:52:36.634 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:36.632+0000 7f30a5bdd640 1 -- 172.21.15.43:0/398007774 wait complete. 2023-08-09T14:52:36.635 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:36.634+0000 7f30a5bdd640 1 Processor -- start 2023-08-09T14:52:36.635 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:36.634+0000 7f30a5bdd640 1 -- start start 2023-08-09T14:52:36.635 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:36.635+0000 7f30a5bdd640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f30a00acdf0 0x7f30a00b0ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T14:52:36.636 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:36.635+0000 7f30a5bdd640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f30a00c8750 0x7f30a00c42a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T14:52:36.636 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:36.635+0000 7f30a5bdd640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f30a00c8b30 0x7f30a00c19c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T14:52:36.636 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:36.635+0000 7f30a5bdd640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f30a00c9f70 con 0x7f30a00c8b30 2023-08-09T14:52:36.636 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:36.635+0000 7f30a5bdd640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f30a01f29d0 con 0x7f30a00c8750 2023-08-09T14:52:36.636 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:36.635+0000 7f30a5bdd640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f30a00c6250 con 0x7f30a00acdf0 2023-08-09T14:52:36.637 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:36.636+0000 7f30a53dc640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f30a00c8b30 0x7f30a00c19c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T14:52:36.638 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:36.636+0000 7f30a53dc640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f30a00c8b30 0x7f30a00c19c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:36338/0 (socket says 172.21.15.43:36338) 2023-08-09T14:52:36.638 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:36.636+0000 7f30a53dc640 1 -- 172.21.15.43:0/1569646188 learned_addr learned my addr 172.21.15.43:0/1569646188 (peer_addr_for_me v2:172.21.15.43:0/0) 2023-08-09T14:52:36.638 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:36.636+0000 7f30a4bdb640 1 --2- 172.21.15.43:0/1569646188 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f30a00acdf0 0x7f30a00b0ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T14:52:36.638 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:36.636+0000 7f3097fff640 1 --2- 172.21.15.43:0/1569646188 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f30a00c8750 0x7f30a00c42a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T14:52:36.638 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:36.636+0000 7f30a53dc640 1 -- 172.21.15.43:0/1569646188 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f30a00acdf0 msgr2=0x7f30a00b0ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T14:52:36.638 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:36.636+0000 7f30a53dc640 1 --2- 172.21.15.43:0/1569646188 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f30a00acdf0 0x7f30a00b0ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T14:52:36.639 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:36.636+0000 7f30a53dc640 1 -- 172.21.15.43:0/1569646188 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f30a00c8750 msgr2=0x7f30a00c42a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T14:52:36.639 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:36.636+0000 7f30a53dc640 1 --2- 172.21.15.43:0/1569646188 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f30a00c8750 0x7f30a00c42a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T14:52:36.639 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:36.636+0000 7f30a53dc640 1 -- 172.21.15.43:0/1569646188 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3098009a70 con 0x7f30a00c8b30 2023-08-09T14:52:36.639 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:36.636+0000 7f30a4bdb640 1 --2- 172.21.15.43:0/1569646188 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f30a00acdf0 0x7f30a00b0ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-08-09T14:52:36.639 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:36.637+0000 7f30a53dc640 1 --2- 172.21.15.43:0/1569646188 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f30a00c8b30 0x7f30a00c19c0 secure :-1 s=READY pgs=2053 cs=0 l=1 rev1=1 crypto rx=0x7f3098009eb0 tx=0x7f3098004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-09T14:52:36.639 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:36.637+0000 7f30957fa640 1 -- 172.21.15.43:0/1569646188 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3098003b90 con 0x7f30a00c8b30 2023-08-09T14:52:36.639 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:36.637+0000 7f30957fa640 1 -- 172.21.15.43:0/1569646188 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3098003cf0 con 0x7f30a00c8b30 2023-08-09T14:52:36.639 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:36.637+0000 7f30957fa640 1 -- 172.21.15.43:0/1569646188 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3098004230 con 0x7f30a00c8b30 2023-08-09T14:52:36.640 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:36.637+0000 7f30a5bdd640 1 -- 172.21.15.43:0/1569646188 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f30a00c6530 con 0x7f30a00c8b30 2023-08-09T14:52:36.640 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:36.637+0000 7f30a5bdd640 1 -- 172.21.15.43:0/1569646188 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f30a00c6af0 con 0x7f30a00c8b30 2023-08-09T14:52:36.640 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:36.638+0000 7f30a5bdd640 1 -- 172.21.15.43:0/1569646188 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f30a006eef0 con 0x7f30a00c8b30 2023-08-09T14:52:36.640 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:36.640+0000 7f30957fa640 1 -- 172.21.15.43:0/1569646188 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 88) v1 ==== 88443+0+0 (secure 0 0 0) 0x7f309803b050 con 0x7f30a00c8b30 2023-08-09T14:52:36.641 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:36.641+0000 7f30957fa640 1 --2- 172.21.15.43:0/1569646188 >> [v2:172.21.15.203:6832/2959400181,v1:172.21.15.203:6833/2959400181] conn(0x7f308c06a170 0x7f308c06c630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T14:52:36.641 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:36.641+0000 7f30957fa640 1 -- 172.21.15.43:0/1569646188 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(186..186 src has 1..186) v4 ==== 6426+0+0 (secure 0 0 0) 0x7f30980ab270 con 0x7f30a00c8b30 2023-08-09T14:52:36.642 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:36.641+0000 7f30a4bdb640 1 --2- 172.21.15.43:0/1569646188 >> [v2:172.21.15.203:6832/2959400181,v1:172.21.15.203:6833/2959400181] conn(0x7f308c06a170 0x7f308c06c630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T14:52:36.642 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:36.642+0000 7f30a4bdb640 1 --2- 172.21.15.43:0/1569646188 >> [v2:172.21.15.203:6832/2959400181,v1:172.21.15.203:6833/2959400181] conn(0x7f308c06a170 0x7f308c06c630 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f309c001700 tx=0x7f309c009040 comp rx=0 tx=0).ready entity=mgr.7223 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-09T14:52:36.648 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:36.645+0000 7f30957fa640 1 -- 172.21.15.43:0/1569646188 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178066 (secure 0 0 0) 0x7f3098076640 con 0x7f30a00c8b30 2023-08-09T14:52:36.892 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:36.891+0000 7f30a5bdd640 1 -- 172.21.15.43:0/1569646188 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f30a00c1760 con 0x7f30a00c8b30 2023-08-09T14:52:36.893 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:36.893+0000 7f30957fa640 1 -- 172.21.15.43:0/1569646188 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 87 v87) v1 ==== 94+0+4882 (secure 0 0 0) 0x7f3098079c60 con 0x7f30a00c8b30 2023-08-09T14:52:36.894 INFO:teuthology.orchestra.run.smithi043.stdout: 2023-08-09T14:52:36.894 INFO:teuthology.orchestra.run.smithi043.stdout:{"epoch":87,"default_fscid":10,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":7328,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.203:6836/3701396136","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6834","nonce":3701396136},{"type":"v1","addr":"172.21.15.203:6836","nonce":3701396136}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":82},{"gid":7343,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.43:6835/2096611841","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6833","nonce":2096611841},{"type":"v1","addr":"172.21.15.43:6835","nonce":2096611841}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":82},{"gid":8242,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.203:6837/18754135","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6835","nonce":18754135},{"type":"v1","addr":"172.21.15.203:6837","nonce":18754135}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":81}],"filesystems":[{"mdsmap":{"epoch":87,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-08-09T14:52:30.188965+0000","modified":"2023-08-09T14:52:33.772528+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":10185},"failed":[],"damaged":[],"stopped":[],"info":{"gid_10185":{"gid":10185,"name":"c","rank":0,"incarnation":83,"state":"up:active","state_seq":3,"addr":"172.21.15.43:6834/2866936881","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6832","nonce":2866936881},{"type":"v1","addr":"172.21.15.43:6834","nonce":2866936881}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[23],"metadata_pool":22,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":10}]} 2023-08-09T14:52:36.894 INFO:teuthology.orchestra.run.smithi043.stderr:dumped fsmap epoch 87 2023-08-09T14:52:36.896 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:36.895+0000 7f30a5bdd640 1 -- 172.21.15.43:0/1569646188 >> [v2:172.21.15.203:6832/2959400181,v1:172.21.15.203:6833/2959400181] conn(0x7f308c06a170 msgr2=0x7f308c06c630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T14:52:36.896 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:36.895+0000 7f30a5bdd640 1 --2- 172.21.15.43:0/1569646188 >> [v2:172.21.15.203:6832/2959400181,v1:172.21.15.203:6833/2959400181] conn(0x7f308c06a170 0x7f308c06c630 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f309c001700 tx=0x7f309c009040 comp rx=0 tx=0).stop 2023-08-09T14:52:36.896 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:36.895+0000 7f30a5bdd640 1 -- 172.21.15.43:0/1569646188 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f30a00c8b30 msgr2=0x7f30a00c19c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T14:52:36.896 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:36.895+0000 7f30a5bdd640 1 --2- 172.21.15.43:0/1569646188 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f30a00c8b30 0x7f30a00c19c0 secure :-1 s=READY pgs=2053 cs=0 l=1 rev1=1 crypto rx=0x7f3098009eb0 tx=0x7f3098004a40 comp rx=0 tx=0).stop 2023-08-09T14:52:36.897 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:36.896+0000 7f30a5bdd640 1 -- 172.21.15.43:0/1569646188 shutdown_connections 2023-08-09T14:52:36.897 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:36.896+0000 7f30a5bdd640 1 --2- 172.21.15.43:0/1569646188 >> [v2:172.21.15.203:6832/2959400181,v1:172.21.15.203:6833/2959400181] conn(0x7f308c06a170 0x7f308c06c630 unknown :-1 s=CLOSED pgs=69 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T14:52:36.897 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:36.896+0000 7f30a5bdd640 1 --2- 172.21.15.43:0/1569646188 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f30a00c8b30 0x7f30a00c19c0 unknown :-1 s=CLOSED pgs=2053 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T14:52:36.897 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:36.896+0000 7f30a5bdd640 1 --2- 172.21.15.43:0/1569646188 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f30a00c8750 0x7f30a00c42a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T14:52:36.897 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:36.896+0000 7f30a5bdd640 1 --2- 172.21.15.43:0/1569646188 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f30a00acdf0 0x7f30a00b0ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T14:52:36.898 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:36.896+0000 7f30a5bdd640 1 -- 172.21.15.43:0/1569646188 >> 172.21.15.43:0/1569646188 conn(0x7f30a00cde90 msgr2=0x7f30a00cc240 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-09T14:52:36.898 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:36.896+0000 7f30a5bdd640 1 -- 172.21.15.43:0/1569646188 shutdown_connections 2023-08-09T14:52:36.898 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:36.896+0000 7f30a5bdd640 1 -- 172.21.15.43:0/1569646188 wait complete. 2023-08-09T14:52:37.318 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 87, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2023-08-09T14:52:30.188965+0000', 'modified': '2023-08-09T14:52:33.772528+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 10185}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_10185': {'gid': 10185, 'name': 'c', 'rank': 0, 'incarnation': 83, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.43:6834/2866936881', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.43:6832', 'nonce': 2866936881}, {'type': 'v1', 'addr': '172.21.15.43:6834', 'nonce': 2866936881}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138322906710015, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}}, 'data_pools': [23], 'metadata_pool': 22, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0} 2023-08-09T14:52:37.319 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2023-08-09T14:52:37.319 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.c status 2023-08-09T14:52:37.563 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:37.562+0000 7f0ffbc94640 1 Processor -- start 2023-08-09T14:52:37.563 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:37.562+0000 7f0ffbc94640 1 -- start start 2023-08-09T14:52:37.565 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:37.564+0000 7f0ffbc94640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f0ff406cfe0 0x7f0ff40c8b50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T14:52:37.565 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:37.564+0000 7f0ffbc94640 1 -- --> v1:172.21.15.43:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0ff40c9090 con 0x7f0ff4143c20 2023-08-09T14:52:37.565 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:37.564+0000 7f0ffbc94640 1 -- --> v1:172.21.15.203:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0ff406c440 con 0x7f0ff40c86f0 2023-08-09T14:52:37.565 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:37.564+0000 7f0ffbc94640 1 -- --> v2:172.21.15.43:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0ff4144000 con 0x7f0ff406cfe0 2023-08-09T14:52:37.566 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:37.565+0000 7f0ff9a30640 1 --1- >> v1:172.21.15.43:6789/0 conn(0x7f0ff4143c20 0x7f0ff406f540 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.43:6789/0 says I am v1:172.21.15.43:46254/0 (socket says 172.21.15.43:46254) 2023-08-09T14:52:37.566 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:37.565+0000 7f0ff9a30640 1 -- 172.21.15.43:0/2398567235 learned_addr learned my addr 172.21.15.43:0/2398567235 (peer_addr_for_me v1:172.21.15.43:0/0) 2023-08-09T14:52:37.566 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:37.565+0000 7f0ffa231640 1 --2- 172.21.15.43:0/2398567235 >> v2:172.21.15.43:3300/0 conn(0x7f0ff406cfe0 0x7f0ff40c8b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T14:52:37.567 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:37.565+0000 7f0ff8a2e640 1 -- 172.21.15.43:0/2398567235 <== mon.0 v1:172.21.15.43:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1079743358 0 0) 0x7f0ff40c9090 con 0x7f0ff4143c20 2023-08-09T14:52:37.567 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:37.565+0000 7f0ff8a2e640 1 -- 172.21.15.43:0/2398567235 --> v1:172.21.15.43:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f0fe0003520 con 0x7f0ff4143c20 2023-08-09T14:52:37.567 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:37.565+0000 7f0ffa231640 1 -- 172.21.15.43:0/2398567235 >> v1:172.21.15.203:6789/0 conn(0x7f0ff40c86f0 legacy=0x7f0ff406c8d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T14:52:37.567 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:37.565+0000 7f0ffa231640 1 -- 172.21.15.43:0/2398567235 >> v1:172.21.15.43:6789/0 conn(0x7f0ff4143c20 legacy=0x7f0ff406f540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T14:52:37.567 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:37.566+0000 7f0ffa231640 1 -- 172.21.15.43:0/2398567235 --> v2:172.21.15.43:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0ff406fc50 con 0x7f0ff406cfe0 2023-08-09T14:52:37.568 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:37.567+0000 7f0ffa231640 1 --2- 172.21.15.43:0/2398567235 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f0ff406cfe0 0x7f0ff40c8b50 secure :-1 s=READY pgs=2055 cs=0 l=1 rev1=1 crypto rx=0x7f0fec009d90 tx=0x7f0fec02be20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=72aa5f0653e1893e server_cookie=0 in_seq=0 out_seq=0 2023-08-09T14:52:37.568 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:37.567+0000 7f0ff8a2e640 1 -- 172.21.15.43:0/2398567235 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0fec02cbd0 con 0x7f0ff406cfe0 2023-08-09T14:52:37.568 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:37.567+0000 7f0ff8a2e640 1 -- 172.21.15.43:0/2398567235 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0fec02cd30 con 0x7f0ff406cfe0 2023-08-09T14:52:37.568 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:37.567+0000 7f0ff8a2e640 1 -- 172.21.15.43:0/2398567235 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0fec02cea0 con 0x7f0ff406cfe0 2023-08-09T14:52:37.569 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:37.568+0000 7f0ffbc94640 1 -- 172.21.15.43:0/2398567235 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f0ff406cfe0 msgr2=0x7f0ff40c8b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T14:52:37.569 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:37.568+0000 7f0ffbc94640 1 --2- 172.21.15.43:0/2398567235 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f0ff406cfe0 0x7f0ff40c8b50 secure :-1 s=READY pgs=2055 cs=0 l=1 rev1=1 crypto rx=0x7f0fec009d90 tx=0x7f0fec02be20 comp rx=0 tx=0).stop 2023-08-09T14:52:37.569 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:37.568+0000 7f0ffbc94640 1 -- 172.21.15.43:0/2398567235 shutdown_connections 2023-08-09T14:52:37.569 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:37.568+0000 7f0ffbc94640 1 --2- 172.21.15.43:0/2398567235 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f0ff406cfe0 0x7f0ff40c8b50 unknown :-1 s=CLOSED pgs=2055 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T14:52:37.570 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:37.568+0000 7f0ffbc94640 1 -- 172.21.15.43:0/2398567235 >> 172.21.15.43:0/2398567235 conn(0x7f0ff40ce0a0 msgr2=0x7f0ff40ce480 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-09T14:52:37.570 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:37.568+0000 7f0ffbc94640 1 -- 172.21.15.43:0/2398567235 shutdown_connections 2023-08-09T14:52:37.570 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:37.568+0000 7f0ffbc94640 1 -- 172.21.15.43:0/2398567235 wait complete. 2023-08-09T14:52:37.570 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:37.570+0000 7f0ffbc94640 1 Processor -- start 2023-08-09T14:52:37.571 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:37.570+0000 7f0ffbc94640 1 -- start start 2023-08-09T14:52:37.571 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:37.571+0000 7f0ffbc94640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f0ff406cfe0 0x7f0ff413cab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T14:52:37.571 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:37.571+0000 7f0ffbc94640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f0ff40c86f0 0x7f0ff40b1700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T14:52:37.572 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:37.571+0000 7f0ffbc94640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f0ff4143c20 0x7f0ff40c3f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T14:52:37.572 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:37.571+0000 7f0ffbc94640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0ff40b3c00 con 0x7f0ff406cfe0 2023-08-09T14:52:37.572 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:37.571+0000 7f0ffbc94640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0ff40c3af0 con 0x7f0ff40c86f0 2023-08-09T14:52:37.572 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:37.571+0000 7f0ffbc94640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0ff40b3640 con 0x7f0ff4143c20 2023-08-09T14:52:37.572 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:37.571+0000 7f0ff9a30640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f0ff406cfe0 0x7f0ff413cab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T14:52:37.572 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:37.571+0000 7f0ff9a30640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f0ff406cfe0 0x7f0ff413cab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:36352/0 (socket says 172.21.15.43:36352) 2023-08-09T14:52:37.573 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:37.571+0000 7f0ff9a30640 1 -- 172.21.15.43:0/1927182610 learned_addr learned my addr 172.21.15.43:0/1927182610 (peer_addr_for_me v2:172.21.15.43:0/0) 2023-08-09T14:52:37.573 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:37.572+0000 7f0ff9a30640 1 -- 172.21.15.43:0/1927182610 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f0ff4143c20 msgr2=0x7f0ff40c3f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T14:52:37.574 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:37.572+0000 7f0ff9a30640 1 --2- 172.21.15.43:0/1927182610 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f0ff4143c20 0x7f0ff40c3f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T14:52:37.574 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:37.572+0000 7f0ff9a30640 1 -- 172.21.15.43:0/1927182610 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f0ff40c86f0 msgr2=0x7f0ff40b1700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T14:52:37.574 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:37.572+0000 7f0ff922f640 1 --2- 172.21.15.43:0/1927182610 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f0ff40c86f0 0x7f0ff40b1700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T14:52:37.574 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:37.572+0000 7f0ff9a30640 1 --2- 172.21.15.43:0/1927182610 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f0ff40c86f0 0x7f0ff40b1700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T14:52:37.574 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:37.572+0000 7f0ff9a30640 1 -- 172.21.15.43:0/1927182610 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0fec009a70 con 0x7f0ff406cfe0 2023-08-09T14:52:37.574 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:37.572+0000 7f0ff922f640 1 --2- 172.21.15.43:0/1927182610 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f0ff40c86f0 0x7f0ff40b1700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-08-09T14:52:37.575 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:37.572+0000 7f0ff9a30640 1 --2- 172.21.15.43:0/1927182610 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f0ff406cfe0 0x7f0ff413cab0 secure :-1 s=READY pgs=2056 cs=0 l=1 rev1=1 crypto rx=0x7f0ff000c530 tx=0x7f0ff000c9f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-09T14:52:37.575 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:37.572+0000 7f0fea7fc640 1 -- 172.21.15.43:0/1927182610 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0ff0005ce0 con 0x7f0ff406cfe0 2023-08-09T14:52:37.575 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:37.572+0000 7f0fea7fc640 1 -- 172.21.15.43:0/1927182610 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0ff0005e40 con 0x7f0ff406cfe0 2023-08-09T14:52:37.575 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:37.572+0000 7f0fea7fc640 1 -- 172.21.15.43:0/1927182610 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0ff00072b0 con 0x7f0ff406cfe0 2023-08-09T14:52:37.575 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:37.572+0000 7f0ffbc94640 1 -- 172.21.15.43:0/1927182610 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0ff40b3920 con 0x7f0ff406cfe0 2023-08-09T14:52:37.575 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:37.573+0000 7f0ffbc94640 1 -- 172.21.15.43:0/1927182610 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0ff40b1c40 con 0x7f0ff406cfe0 2023-08-09T14:52:37.576 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:37.575+0000 7f0fea7fc640 1 -- 172.21.15.43:0/1927182610 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 88) v1 ==== 88443+0+0 (secure 0 0 0) 0x7f0ff0007410 con 0x7f0ff406cfe0 2023-08-09T14:52:37.576 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:37.576+0000 7f0fea7fc640 1 --2- 172.21.15.43:0/1927182610 >> [v2:172.21.15.203:6832/2959400181,v1:172.21.15.203:6833/2959400181] conn(0x7f0fd806a170 0x7f0fd806c630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T14:52:37.576 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:37.576+0000 7f0fea7fc640 1 -- 172.21.15.43:0/1927182610 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(186..186 src has 1..186) v4 ==== 6426+0+0 (secure 0 0 0) 0x7f0ff0087ec0 con 0x7f0ff406cfe0 2023-08-09T14:52:37.577 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:37.577+0000 7f0ff922f640 1 --2- 172.21.15.43:0/1927182610 >> [v2:172.21.15.203:6832/2959400181,v1:172.21.15.203:6833/2959400181] conn(0x7f0fd806a170 0x7f0fd806c630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T14:52:37.577 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:37.577+0000 7f0ffe802740 1 Processor -- start 2023-08-09T14:52:37.577 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:37.577+0000 7f0ffe802740 1 -- start start 2023-08-09T14:52:37.578 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:37.578+0000 7f0ffe802740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x56031efc2c50 0x56031efc3030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T14:52:37.579 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:37.578+0000 7f0ffe802740 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x56031ef92bd0 0x56031efbfec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T14:52:37.579 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:37.578+0000 7f0ffe802740 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x56031ef967d0 0x56031efc0400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T14:52:37.579 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:37.578+0000 7f0ffe802740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x56031ef048f0 con 0x56031efc2c50 2023-08-09T14:52:37.579 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:37.578+0000 7f0ffe802740 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x56031ef01fe0 con 0x56031ef967d0 2023-08-09T14:52:37.579 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:37.578+0000 7f0ffe802740 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x56031eefd8d0 con 0x56031ef92bd0 2023-08-09T14:52:37.579 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:37.578+0000 7f0ff922f640 1 --2- 172.21.15.43:0/1927182610 >> [v2:172.21.15.203:6832/2959400181,v1:172.21.15.203:6833/2959400181] conn(0x7f0fd806a170 0x7f0fd806c630 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f0ff41430a0 tx=0x7f0fe4009040 comp rx=0 tx=0).ready entity=mgr.7223 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-09T14:52:37.579 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:37.578+0000 7f0ff9a30640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x56031efc2c50 0x56031efc3030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T14:52:37.579 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:37.578+0000 7f0ff9a30640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x56031efc2c50 0x56031efc3030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:36360/0 (socket says 172.21.15.43:36360) 2023-08-09T14:52:37.580 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:37.578+0000 7f0ff9a30640 1 -- 172.21.15.43:0/1617114110 learned_addr learned my addr 172.21.15.43:0/1617114110 (peer_addr_for_me v2:172.21.15.43:0/0) 2023-08-09T14:52:37.581 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:37.579+0000 7f0ff922f640 1 --2- 172.21.15.43:0/1617114110 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x56031ef92bd0 0x56031efbfec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T14:52:37.581 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:37.579+0000 7f0ffa231640 1 --2- 172.21.15.43:0/1617114110 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x56031ef967d0 0x56031efc0400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T14:52:37.581 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:37.579+0000 7f0ff9a30640 1 -- 172.21.15.43:0/1617114110 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x56031ef92bd0 msgr2=0x56031efbfec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-09T14:52:37.581 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:37.579+0000 7f0ff9a30640 1 --2- 172.21.15.43:0/1617114110 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x56031ef92bd0 0x56031efbfec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T14:52:37.582 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:37.579+0000 7f0ff9a30640 1 -- 172.21.15.43:0/1617114110 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x56031ef967d0 msgr2=0x56031efc0400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-09T14:52:37.582 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:37.579+0000 7f0ff9a30640 1 --2- 172.21.15.43:0/1617114110 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x56031ef967d0 0x56031efc0400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T14:52:37.582 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:37.579+0000 7f0ff9a30640 1 -- 172.21.15.43:0/1617114110 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x56031eeb6ac0 con 0x56031efc2c50 2023-08-09T14:52:37.582 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:37.579+0000 7f0ff922f640 1 --2- 172.21.15.43:0/1617114110 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x56031ef92bd0 0x56031efbfec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-08-09T14:52:37.582 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:37.579+0000 7f0ffa231640 1 --2- 172.21.15.43:0/1617114110 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x56031ef967d0 0x56031efc0400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-08-09T14:52:37.582 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:37.579+0000 7f0ff9a30640 1 --2- 172.21.15.43:0/1617114110 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x56031efc2c50 0x56031efc3030 secure :-1 s=READY pgs=2057 cs=0 l=1 rev1=1 crypto rx=0x7f0ff0057030 tx=0x7f0ff005ee70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7d8dc45bd6779fa0 server_cookie=0 in_seq=0 out_seq=0 2023-08-09T14:52:37.582 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:37.579+0000 7f0fd77fe640 1 -- 172.21.15.43:0/1617114110 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0ff0007780 con 0x56031efc2c50 2023-08-09T14:52:37.582 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:37.579+0000 7f0fd77fe640 1 -- 172.21.15.43:0/1617114110 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0ff0081030 con 0x56031efc2c50 2023-08-09T14:52:37.583 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:37.579+0000 7f0fd77fe640 1 -- 172.21.15.43:0/1617114110 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0ff005e920 con 0x56031efc2c50 2023-08-09T14:52:37.583 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:37.580+0000 7f0ffe802740 1 -- 172.21.15.43:0/1617114110 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x56031efc2c50 msgr2=0x56031efc3030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T14:52:37.583 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:37.580+0000 7f0ffe802740 1 --2- 172.21.15.43:0/1617114110 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x56031efc2c50 0x56031efc3030 secure :-1 s=READY pgs=2057 cs=0 l=1 rev1=1 crypto rx=0x7f0ff0057030 tx=0x7f0ff005ee70 comp rx=0 tx=0).stop 2023-08-09T14:52:37.583 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:37.580+0000 7f0ffe802740 1 -- 172.21.15.43:0/1617114110 shutdown_connections 2023-08-09T14:52:37.583 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:37.580+0000 7f0ffe802740 1 --2- 172.21.15.43:0/1617114110 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x56031ef967d0 0x56031efc0400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T14:52:37.583 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:37.580+0000 7f0ffe802740 1 --2- 172.21.15.43:0/1617114110 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x56031ef92bd0 0x56031efbfec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T14:52:37.583 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:37.580+0000 7f0ffe802740 1 --2- 172.21.15.43:0/1617114110 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x56031efc2c50 0x56031efc3030 unknown :-1 s=CLOSED pgs=2057 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T14:52:37.583 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:37.580+0000 7f0ffe802740 1 -- 172.21.15.43:0/1617114110 >> 172.21.15.43:0/1617114110 conn(0x56031ef94ee0 msgr2=0x56031efdda60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-09T14:52:37.584 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:37.580+0000 7f0ffe802740 1 -- 172.21.15.43:0/1617114110 shutdown_connections 2023-08-09T14:52:37.584 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:37.580+0000 7f0ffe802740 1 -- 172.21.15.43:0/1617114110 wait complete. 2023-08-09T14:52:37.584 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:37.581+0000 7f0ffe802740 1 -- start start 2023-08-09T14:52:37.586 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:37.581+0000 7f0ffe802740 1 Processor -- start 2023-08-09T14:52:37.586 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:37.582+0000 7f0ffe802740 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x56031ef967d0 0x56031efddee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T14:52:37.586 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:37.582+0000 7f0ffe802740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x56031efc2c50 0x56031efbae90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T14:52:37.586 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:37.582+0000 7f0ffe802740 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x56031efbb3d0 0x56031efb8b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T14:52:37.586 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:37.582+0000 7f0ffe802740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x56031ee9dd40 con 0x56031efc2c50 2023-08-09T14:52:37.587 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:37.582+0000 7f0ffe802740 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x56031ee9d9e0 con 0x56031ef967d0 2023-08-09T14:52:37.587 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:37.582+0000 7f0ffe802740 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x56031ec63f20 con 0x56031efbb3d0 2023-08-09T14:52:37.587 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:37.582+0000 7f0ffe802740 10 client.0 _ll_register_callbacks cb 0x56031eeb6160 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-08-09T14:52:37.587 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:37.582+0000 7f0ff922f640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x56031efc2c50 0x56031efbae90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T14:52:37.587 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:37.582+0000 7f0ff922f640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x56031efc2c50 0x56031efbae90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:36372/0 (socket says 172.21.15.43:36372) 2023-08-09T14:52:37.587 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:37.582+0000 7f0ff922f640 1 -- 172.21.15.43:0/1816488649 learned_addr learned my addr 172.21.15.43:0/1816488649 (peer_addr_for_me v2:172.21.15.43:0/0) 2023-08-09T14:52:37.587 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:37.583+0000 7f0ff9a30640 1 --2- 172.21.15.43:0/1816488649 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x56031ef967d0 0x56031efddee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T14:52:37.587 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:37.583+0000 7f0ff922f640 1 -- 172.21.15.43:0/1816488649 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x56031efbb3d0 msgr2=0x56031efb8b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-09T14:52:37.588 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:37.583+0000 7f0ff922f640 1 --2- 172.21.15.43:0/1816488649 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x56031efbb3d0 0x56031efb8b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T14:52:37.588 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:37.583+0000 7f0ff922f640 1 -- 172.21.15.43:0/1816488649 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x56031ef967d0 msgr2=0x56031efddee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-09T14:52:37.588 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:37.583+0000 7f0ff922f640 1 --2- 172.21.15.43:0/1816488649 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x56031ef967d0 0x56031efddee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T14:52:37.588 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:37.583+0000 7f0ff922f640 1 -- 172.21.15.43:0/1816488649 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0ff40b1c40 con 0x56031efc2c50 2023-08-09T14:52:37.588 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:37.583+0000 7f0ff922f640 1 --2- 172.21.15.43:0/1816488649 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x56031efc2c50 0x56031efbae90 secure :-1 s=READY pgs=2058 cs=0 l=1 rev1=1 crypto rx=0x7f0fe400dd20 tx=0x7f0fe4000ed0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3c40631c5d54fcd7 server_cookie=0 in_seq=0 out_seq=0 2023-08-09T14:52:37.588 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:37.583+0000 7f0fd4ff9640 10 client.0 ms_handle_connect on v2:172.21.15.43:3300/0 2023-08-09T14:52:37.588 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:37.584+0000 7f0fd4ff9640 1 -- 172.21.15.43:0/1816488649 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0fe401f410 con 0x56031efc2c50 2023-08-09T14:52:37.589 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:37.584+0000 7f0ffe802740 1 -- 172.21.15.43:0/1816488649 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x56031eed5500 con 0x56031efc2c50 2023-08-09T14:52:37.589 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:37.584+0000 7f0fd4ff9640 1 -- 172.21.15.43:0/1816488649 <== mon.0 v2:172.21.15.43:3300/0 2 ==== osd_map(186..186 src has 1..186) v4 ==== 6426+0+0 (secure 0 0 0) 0x7f0fe401f570 con 0x56031efc2c50 2023-08-09T14:52:37.589 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:37.584+0000 7f0fd4ff9640 1 -- 172.21.15.43:0/1816488649 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_get_version_reply(handle=1 version=87) v2 ==== 24+0+0 (secure 0 0 0) 0x7f0fe401f730 con 0x56031efc2c50 2023-08-09T14:52:37.589 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:37.584+0000 7f0ffe802740 10 client.10341 fetch_fsmap learned FSMap version 87 2023-08-09T14:52:37.589 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:37.584+0000 7f0ffe802740 1 -- 172.21.15.43:0/1816488649 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,fsmap=87,monmap=2+}) v3 -- 0x56031eefbf50 con 0x56031efc2c50 2023-08-09T14:52:37.589 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:37.584+0000 7f0fd4ff9640 1 -- 172.21.15.43:0/1816488649 <== mon.0 v2:172.21.15.43:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x56031eed5500 con 0x56031efc2c50 2023-08-09T14:52:37.589 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:37.584+0000 7f0fd4ff9640 1 -- 172.21.15.43:0/1816488649 <== mon.0 v2:172.21.15.43:3300/0 5 ==== fsmap(e 87) v1 ==== 3294+0+0 (secure 0 0 0) 0x7f0fe401f890 con 0x56031efc2c50 2023-08-09T14:52:37.589 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:37.584+0000 7f0ffe802740 10 client.10341 fetch_fsmap finished waiting for FSMap version 87 2023-08-09T14:52:37.590 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:37.585+0000 7f0ffe802740 10 client.10341 resolve_mds: resolved name 'c' to daemon mds.c 2023-08-09T14:52:37.590 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:37.585+0000 7f0ffe802740 20 client.10341 populate_metadata read hostname 'smithi043' 2023-08-09T14:52:37.590 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:37.585+0000 7f0ffe802740 1 --2- 172.21.15.43:0/1816488649 >> [v2:172.21.15.43:6832/2866936881,v1:172.21.15.43:6834/2866936881] conn(0x56031eff14e0 0x56031eff18c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T14:52:37.590 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:37.585+0000 7f0ffe802740 4 client.10341 mds_command: new command op to 10185 tid=0[{"prefix": "get_command_descriptions"}] 2023-08-09T14:52:37.590 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:37.585+0000 7f0ffe802740 1 -- 172.21.15.43:0/1816488649 --> [v2:172.21.15.43:6832/2866936881,v1:172.21.15.43:6834/2866936881] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x56031eed2bc0 con 0x56031eff14e0 2023-08-09T14:52:37.590 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:37.585+0000 7f0ff9a30640 1 --2- 172.21.15.43:0/1816488649 >> [v2:172.21.15.43:6832/2866936881,v1:172.21.15.43:6834/2866936881] conn(0x56031eff14e0 0x56031eff18c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T14:52:37.590 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:37.585+0000 7f0ff9a30640 1 --2- 172.21.15.43:0/1816488649 >> [v2:172.21.15.43:6832/2866936881,v1:172.21.15.43:6834/2866936881] conn(0x56031eff14e0 0x56031eff18c0 crc :-1 s=READY pgs=11 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=db86ef95819e3362 server_cookie=c480132fd6da3447 in_seq=0 out_seq=0 2023-08-09T14:52:37.590 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:37.585+0000 7f0fd4ff9640 10 client.10341 ms_handle_connect on v2:172.21.15.43:6832/2866936881 2023-08-09T14:52:37.591 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:37.586+0000 7f0fd4ff9640 1 -- 172.21.15.43:0/1816488649 <== mds.0 v2:172.21.15.43:6832/2866936881 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18017 (crc 0 0 0) 0x56031eed2bc0 con 0x56031eff14e0 2023-08-09T14:52:37.592 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:37.586+0000 7f0fd4ff9640 10 client.10341 handle_command_reply: tid=0 2023-08-09T14:52:37.592 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:37.586+0000 7f0ffe802740 1 client.10341 shutdown 2023-08-09T14:52:37.592 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:37.586+0000 7f0ffe802740 1 -- 172.21.15.43:0/1816488649 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x56031efc2c50 msgr2=0x56031efbae90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T14:52:37.592 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:37.586+0000 7f0ffe802740 1 --2- 172.21.15.43:0/1816488649 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x56031efc2c50 0x56031efbae90 secure :-1 s=READY pgs=2058 cs=0 l=1 rev1=1 crypto rx=0x7f0fe400dd20 tx=0x7f0fe4000ed0 comp rx=0 tx=0).stop 2023-08-09T14:52:37.592 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:37.586+0000 7f0ffe802740 1 -- 172.21.15.43:0/1816488649 shutdown_connections 2023-08-09T14:52:37.592 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:37.586+0000 7f0ffe802740 1 --2- 172.21.15.43:0/1816488649 >> [v2:172.21.15.43:6832/2866936881,v1:172.21.15.43:6834/2866936881] conn(0x56031eff14e0 0x56031eff18c0 crc :-1 s=READY pgs=11 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T14:52:37.593 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:37.587+0000 7f0ffe802740 1 --2- 172.21.15.43:0/1816488649 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x56031efbb3d0 0x56031efb8b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T14:52:37.593 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:37.587+0000 7f0ffe802740 1 --2- 172.21.15.43:0/1816488649 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x56031efc2c50 0x56031efbae90 unknown :-1 s=CLOSED pgs=2058 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T14:52:37.593 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:37.587+0000 7f0ffe802740 1 --2- 172.21.15.43:0/1816488649 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x56031ef967d0 0x56031efddee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T14:52:37.593 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:37.587+0000 7f0fd4ff9640 0 client.10341 ms_handle_reset on v2:172.21.15.43:6832/2866936881 2023-08-09T14:52:37.593 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:37.587+0000 7f0ffe802740 1 -- 172.21.15.43:0/1816488649 >> 172.21.15.43:0/1816488649 conn(0x56031ef92bd0 msgr2=0x56031efc67d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-09T14:52:37.593 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:37.587+0000 7f0ffe802740 1 -- 172.21.15.43:0/1816488649 shutdown_connections 2023-08-09T14:52:37.593 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:37.587+0000 7f0ffe802740 1 -- 172.21.15.43:0/1816488649 wait complete. 2023-08-09T14:52:37.593 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:37.587+0000 7f0ffe802740 20 client.10341 trim_cache size 0 max 16384 2023-08-09T14:52:37.604 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:37.604+0000 7f0ffe802740 1 Processor -- start 2023-08-09T14:52:37.604 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:37.604+0000 7f0ffe802740 1 -- start start 2023-08-09T14:52:37.605 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:37.605+0000 7f0ffe802740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x56031ef967d0 0x56031ef98fd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T14:52:37.605 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:37.605+0000 7f0ffe802740 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x56031efbb3d0 0x56031efc5890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T14:52:37.605 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:37.605+0000 7f0ffe802740 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x56031efc2c50 0x56031efbf990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T14:52:37.606 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:37.605+0000 7f0ffe802740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x56031ef97840 con 0x56031ef967d0 2023-08-09T14:52:37.606 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:37.605+0000 7f0ffe802740 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x56031efdfea0 con 0x56031efc2c50 2023-08-09T14:52:37.606 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:37.605+0000 7f0ffe802740 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x56031ef2f150 con 0x56031efbb3d0 2023-08-09T14:52:37.606 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:37.605+0000 7f0ff9a30640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x56031ef967d0 0x56031ef98fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T14:52:37.606 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:37.605+0000 7f0ff9a30640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x56031ef967d0 0x56031ef98fd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:36374/0 (socket says 172.21.15.43:36374) 2023-08-09T14:52:37.606 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:37.605+0000 7f0ff9a30640 1 -- 172.21.15.43:0/2294363874 learned_addr learned my addr 172.21.15.43:0/2294363874 (peer_addr_for_me v2:172.21.15.43:0/0) 2023-08-09T14:52:37.607 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:37.605+0000 7f0ff9a30640 1 -- 172.21.15.43:0/2294363874 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x56031efbb3d0 msgr2=0x56031efc5890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-09T14:52:37.608 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:37.605+0000 7f0ff9a30640 1 --2- 172.21.15.43:0/2294363874 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x56031efbb3d0 0x56031efc5890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T14:52:37.608 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:37.605+0000 7f0ff9a30640 1 -- 172.21.15.43:0/2294363874 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x56031efc2c50 msgr2=0x56031efbf990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-09T14:52:37.608 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:37.605+0000 7f0ff9a30640 1 --2- 172.21.15.43:0/2294363874 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x56031efc2c50 0x56031efbf990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T14:52:37.608 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:37.605+0000 7f0ff9a30640 1 -- 172.21.15.43:0/2294363874 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x56031ef2f300 con 0x56031ef967d0 2023-08-09T14:52:37.608 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:37.606+0000 7f0ff9a30640 1 --2- 172.21.15.43:0/2294363874 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x56031ef967d0 0x56031ef98fd0 secure :-1 s=READY pgs=2059 cs=0 l=1 rev1=1 crypto rx=0x7f0ff0007fd0 tx=0x7f0ff0013000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1d2507a95fa0530b server_cookie=0 in_seq=0 out_seq=0 2023-08-09T14:52:37.608 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:37.606+0000 7f0fd7fff640 1 -- 172.21.15.43:0/2294363874 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0ff00537b0 con 0x56031ef967d0 2023-08-09T14:52:37.609 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:37.606+0000 7f0fd7fff640 1 -- 172.21.15.43:0/2294363874 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0ff0081070 con 0x56031ef967d0 2023-08-09T14:52:37.609 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:37.606+0000 7f0fd7fff640 1 -- 172.21.15.43:0/2294363874 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0ff005eb40 con 0x56031ef967d0 2023-08-09T14:52:37.609 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:37.606+0000 7f0ffe802740 1 -- 172.21.15.43:0/2294363874 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x56031ef967d0 msgr2=0x56031ef98fd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T14:52:37.609 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:37.606+0000 7f0ffe802740 1 --2- 172.21.15.43:0/2294363874 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x56031ef967d0 0x56031ef98fd0 secure :-1 s=READY pgs=2059 cs=0 l=1 rev1=1 crypto rx=0x7f0ff0007fd0 tx=0x7f0ff0013000 comp rx=0 tx=0).stop 2023-08-09T14:52:37.609 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:37.607+0000 7f0ffe802740 1 -- 172.21.15.43:0/2294363874 shutdown_connections 2023-08-09T14:52:37.609 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:37.607+0000 7f0ffe802740 1 --2- 172.21.15.43:0/2294363874 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x56031efc2c50 0x56031efbf990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T14:52:37.609 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:37.607+0000 7f0ffe802740 1 --2- 172.21.15.43:0/2294363874 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x56031efbb3d0 0x56031efc5890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T14:52:37.609 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:37.607+0000 7f0ffe802740 1 --2- 172.21.15.43:0/2294363874 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x56031ef967d0 0x56031ef98fd0 unknown :-1 s=CLOSED pgs=2059 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T14:52:37.610 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:37.607+0000 7f0ffe802740 1 -- 172.21.15.43:0/2294363874 >> 172.21.15.43:0/2294363874 conn(0x56031ef92bd0 msgr2=0x56031efe02f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-09T14:52:37.610 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:37.607+0000 7f0ffe802740 1 -- 172.21.15.43:0/2294363874 shutdown_connections 2023-08-09T14:52:37.610 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:37.607+0000 7f0ffe802740 1 -- 172.21.15.43:0/2294363874 wait complete. 2023-08-09T14:52:37.610 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:37.607+0000 7f0ffe802740 1 -- start start 2023-08-09T14:52:37.613 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:37.608+0000 7f0ffe802740 1 Processor -- start 2023-08-09T14:52:37.613 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:37.609+0000 7f0ffe802740 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x56031ef967d0 0x56031efe0f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T14:52:37.613 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:37.609+0000 7f0ffe802740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x56031efbb3d0 0x56031efe14b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T14:52:37.613 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:37.609+0000 7f0ffe802740 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x56031efc2c50 0x56031efec6f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T14:52:37.613 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:37.609+0000 7f0ffe802740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x56031f02d1c0 con 0x56031efbb3d0 2023-08-09T14:52:37.613 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:37.609+0000 7f0ffe802740 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x56031efc67f0 con 0x56031efc2c50 2023-08-09T14:52:37.613 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:37.609+0000 7f0ffe802740 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x56031efebf80 con 0x56031ef967d0 2023-08-09T14:52:37.614 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:37.609+0000 7f0ffe802740 10 client.0 _ll_register_callbacks cb 0x7f0fe4001d30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-08-09T14:52:37.614 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:37.609+0000 7f0ff922f640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x56031efbb3d0 0x56031efe14b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T14:52:37.614 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:37.609+0000 7f0ff922f640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x56031efbb3d0 0x56031efe14b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:36380/0 (socket says 172.21.15.43:36380) 2023-08-09T14:52:37.614 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:37.609+0000 7f0ff922f640 1 -- 172.21.15.43:0/3678172812 learned_addr learned my addr 172.21.15.43:0/3678172812 (peer_addr_for_me v2:172.21.15.43:0/0) 2023-08-09T14:52:37.614 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:37.609+0000 7f0ff922f640 1 -- 172.21.15.43:0/3678172812 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x56031ef967d0 msgr2=0x56031efe0f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-09T14:52:37.614 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:37.609+0000 7f0ff922f640 1 --2- 172.21.15.43:0/3678172812 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x56031ef967d0 0x56031efe0f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T14:52:37.614 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:37.609+0000 7f0ff922f640 1 -- 172.21.15.43:0/3678172812 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x56031efc2c50 msgr2=0x56031efec6f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-09T14:52:37.615 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:37.609+0000 7f0ff922f640 1 --2- 172.21.15.43:0/3678172812 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x56031efc2c50 0x56031efec6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T14:52:37.615 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:37.609+0000 7f0ff922f640 1 -- 172.21.15.43:0/3678172812 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x56031eeb6ac0 con 0x56031efbb3d0 2023-08-09T14:52:37.615 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:37.610+0000 7f0ff922f640 1 --2- 172.21.15.43:0/3678172812 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x56031efbb3d0 0x56031efe14b0 secure :-1 s=READY pgs=2060 cs=0 l=1 rev1=1 crypto rx=0x7f0fe400dd20 tx=0x7f0fe4001120 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4096ddb8df12c9ae server_cookie=0 in_seq=0 out_seq=0 2023-08-09T14:52:37.615 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:37.610+0000 7f0fd4ff9640 10 client.0 ms_handle_connect on v2:172.21.15.43:3300/0 2023-08-09T14:52:37.615 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:37.610+0000 7f0fd4ff9640 1 -- 172.21.15.43:0/3678172812 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0fe4027760 con 0x56031efbb3d0 2023-08-09T14:52:37.615 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:37.610+0000 7f0fd4ff9640 1 -- 172.21.15.43:0/3678172812 <== mon.0 v2:172.21.15.43:3300/0 2 ==== osd_map(186..186 src has 1..186) v4 ==== 6426+0+0 (secure 0 0 0) 0x7f0fe40278c0 con 0x56031efbb3d0 2023-08-09T14:52:37.615 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:37.610+0000 7f0ffe802740 1 -- 172.21.15.43:0/3678172812 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x56031ef25800 con 0x56031efbb3d0 2023-08-09T14:52:37.615 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:37.610+0000 7f0fd4ff9640 1 -- 172.21.15.43:0/3678172812 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_get_version_reply(handle=1 version=87) v2 ==== 24+0+0 (secure 0 0 0) 0x7f0fe400d5c0 con 0x56031efbb3d0 2023-08-09T14:52:37.616 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:37.610+0000 7f0ffe802740 10 client.10347 fetch_fsmap learned FSMap version 87 2023-08-09T14:52:37.616 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:37.610+0000 7f0ffe802740 1 -- 172.21.15.43:0/3678172812 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,fsmap=87,monmap=2+}) v3 -- 0x56031ee9ca10 con 0x56031efbb3d0 2023-08-09T14:52:37.616 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:37.610+0000 7f0fd4ff9640 1 -- 172.21.15.43:0/3678172812 <== mon.0 v2:172.21.15.43:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x56031ef25800 con 0x56031efbb3d0 2023-08-09T14:52:37.616 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:37.611+0000 7f0fd4ff9640 1 -- 172.21.15.43:0/3678172812 <== mon.0 v2:172.21.15.43:3300/0 5 ==== fsmap(e 87) v1 ==== 3294+0+0 (secure 0 0 0) 0x7f0fe40303f0 con 0x56031efbb3d0 2023-08-09T14:52:37.616 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:37.611+0000 7f0ffe802740 10 client.10347 fetch_fsmap finished waiting for FSMap version 87 2023-08-09T14:52:37.616 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:37.611+0000 7f0ffe802740 10 client.10347 resolve_mds: resolved name 'c' to daemon mds.c 2023-08-09T14:52:37.616 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:37.611+0000 7f0ffe802740 20 client.10347 populate_metadata read hostname 'smithi043' 2023-08-09T14:52:37.616 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:37.611+0000 7f0ffe802740 1 --2- 172.21.15.43:0/3678172812 >> [v2:172.21.15.43:6832/2866936881,v1:172.21.15.43:6834/2866936881] conn(0x56031efecc30 0x56031ef97a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T14:52:37.617 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:37.611+0000 7f0ffe802740 4 client.10347 mds_command: new command op to 10185 tid=0[{"prefix": "status"}] 2023-08-09T14:52:37.617 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:37.611+0000 7f0ffe802740 1 -- 172.21.15.43:0/3678172812 --> [v2:172.21.15.43:6832/2866936881,v1:172.21.15.43:6834/2866936881] -- command(tid 0: {"prefix": "status"}) v1 -- 0x56031eef68b0 con 0x56031efecc30 2023-08-09T14:52:37.617 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:37.611+0000 7f0ff9a30640 1 --2- 172.21.15.43:0/3678172812 >> [v2:172.21.15.43:6832/2866936881,v1:172.21.15.43:6834/2866936881] conn(0x56031efecc30 0x56031ef97a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T14:52:37.617 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:37.611+0000 7f0ff9a30640 1 --2- 172.21.15.43:0/3678172812 >> [v2:172.21.15.43:6832/2866936881,v1:172.21.15.43:6834/2866936881] conn(0x56031efecc30 0x56031ef97a10 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a6154f2069a93872 server_cookie=dc791456b06bd8ca in_seq=0 out_seq=0 2023-08-09T14:52:37.617 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:37.611+0000 7f0fd4ff9640 10 client.10347 ms_handle_connect on v2:172.21.15.43:6832/2866936881 2023-08-09T14:52:37.617 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:37.612+0000 7f0fd4ff9640 1 -- 172.21.15.43:0/3678172812 <== mds.0 v2:172.21.15.43:6832/2866936881 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+334 (crc 0 0 0) 0x56031eef68b0 con 0x56031efecc30 2023-08-09T14:52:37.617 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:37.612+0000 7f0fd4ff9640 10 client.10347 handle_command_reply: tid=0 2023-08-09T14:52:37.618 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:37.612+0000 7f0ffe802740 1 client.10347 shutdown 2023-08-09T14:52:37.618 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:37.612+0000 7f0ffe802740 1 -- 172.21.15.43:0/3678172812 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x56031efbb3d0 msgr2=0x56031efe14b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T14:52:37.618 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:37.612+0000 7f0ffe802740 1 --2- 172.21.15.43:0/3678172812 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x56031efbb3d0 0x56031efe14b0 secure :-1 s=READY pgs=2060 cs=0 l=1 rev1=1 crypto rx=0x7f0fe400dd20 tx=0x7f0fe4001120 comp rx=0 tx=0).stop 2023-08-09T14:52:37.618 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:37.612+0000 7f0ffe802740 1 -- 172.21.15.43:0/3678172812 shutdown_connections 2023-08-09T14:52:37.618 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:37.612+0000 7f0ffe802740 1 --2- 172.21.15.43:0/3678172812 >> [v2:172.21.15.43:6832/2866936881,v1:172.21.15.43:6834/2866936881] conn(0x56031efecc30 0x56031ef97a10 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T14:52:37.618 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:37.612+0000 7f0ffe802740 1 --2- 172.21.15.43:0/3678172812 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x56031efc2c50 0x56031efec6f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T14:52:37.618 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:37.612+0000 7f0ffe802740 1 --2- 172.21.15.43:0/3678172812 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x56031efbb3d0 0x56031efe14b0 unknown :-1 s=CLOSED pgs=2060 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T14:52:37.618 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:37.612+0000 7f0fd4ff9640 0 client.10347 ms_handle_reset on v2:172.21.15.43:6832/2866936881 2023-08-09T14:52:37.619 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:37.612+0000 7f0ffe802740 1 --2- 172.21.15.43:0/3678172812 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x56031ef967d0 0x56031efe0f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T14:52:37.619 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:37.612+0000 7f0ffe802740 1 -- 172.21.15.43:0/3678172812 >> 172.21.15.43:0/3678172812 conn(0x56031ef92bd0 msgr2=0x56031eff78f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-09T14:52:37.619 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:37.612+0000 7f0ffe802740 1 -- 172.21.15.43:0/3678172812 shutdown_connections 2023-08-09T14:52:37.619 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:37.612+0000 7f0ffe802740 1 -- 172.21.15.43:0/3678172812 wait complete. 2023-08-09T14:52:37.619 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:37.612+0000 7f0ffe802740 20 client.10347 trim_cache size 0 max 16384 2023-08-09T14:52:37.619 INFO:teuthology.orchestra.run.smithi043.stdout:{ 2023-08-09T14:52:37.620 INFO:teuthology.orchestra.run.smithi043.stdout: "cluster_fsid": "daac87ee-1337-4250-a955-aea4e408a39b", 2023-08-09T14:52:37.620 INFO:teuthology.orchestra.run.smithi043.stdout: "whoami": 0, 2023-08-09T14:52:37.620 INFO:teuthology.orchestra.run.smithi043.stdout: "id": 10185, 2023-08-09T14:52:37.620 INFO:teuthology.orchestra.run.smithi043.stdout: "want_state": "up:active", 2023-08-09T14:52:37.620 INFO:teuthology.orchestra.run.smithi043.stdout: "state": "up:active", 2023-08-09T14:52:37.620 INFO:teuthology.orchestra.run.smithi043.stdout: "fs_name": "cephfs", 2023-08-09T14:52:37.620 INFO:teuthology.orchestra.run.smithi043.stdout: "rank_uptime": 6.8706345339999997, 2023-08-09T14:52:37.621 INFO:teuthology.orchestra.run.smithi043.stdout: "mdsmap_epoch": 87, 2023-08-09T14:52:37.621 INFO:teuthology.orchestra.run.smithi043.stdout: "osdmap_epoch": 186, 2023-08-09T14:52:37.621 INFO:teuthology.orchestra.run.smithi043.stdout: "osdmap_epoch_barrier": 186, 2023-08-09T14:52:37.621 INFO:teuthology.orchestra.run.smithi043.stdout: "uptime": 12.999177511999999 2023-08-09T14:52:37.621 INFO:teuthology.orchestra.run.smithi043.stdout:} 2023-08-09T14:52:37.622 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:37.613+0000 7f0ffbc94640 1 -- 172.21.15.43:0/1927182610 >> [v2:172.21.15.203:6832/2959400181,v1:172.21.15.203:6833/2959400181] conn(0x7f0fd806a170 msgr2=0x7f0fd806c630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T14:52:37.622 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:37.613+0000 7f0ffbc94640 1 --2- 172.21.15.43:0/1927182610 >> [v2:172.21.15.203:6832/2959400181,v1:172.21.15.203:6833/2959400181] conn(0x7f0fd806a170 0x7f0fd806c630 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f0ff41430a0 tx=0x7f0fe4009040 comp rx=0 tx=0).stop 2023-08-09T14:52:37.622 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:37.613+0000 7f0ffbc94640 1 -- 172.21.15.43:0/1927182610 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f0ff406cfe0 msgr2=0x7f0ff413cab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T14:52:37.622 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:37.613+0000 7f0ffbc94640 1 --2- 172.21.15.43:0/1927182610 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f0ff406cfe0 0x7f0ff413cab0 secure :-1 s=READY pgs=2056 cs=0 l=1 rev1=1 crypto rx=0x7f0ff000c530 tx=0x7f0ff000c9f0 comp rx=0 tx=0).stop 2023-08-09T14:52:37.622 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:37.613+0000 7f0ffbc94640 1 -- 172.21.15.43:0/1927182610 shutdown_connections 2023-08-09T14:52:37.622 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:37.613+0000 7f0ffbc94640 1 --2- 172.21.15.43:0/1927182610 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f0ff4143c20 0x7f0ff40c3f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T14:52:37.623 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:37.613+0000 7f0ffbc94640 1 --2- 172.21.15.43:0/1927182610 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f0ff40c86f0 0x7f0ff40b1700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T14:52:37.623 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:37.613+0000 7f0ffbc94640 1 --2- 172.21.15.43:0/1927182610 >> [v2:172.21.15.203:6832/2959400181,v1:172.21.15.203:6833/2959400181] conn(0x7f0fd806a170 0x7f0fd806c630 unknown :-1 s=CLOSED pgs=70 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T14:52:37.623 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:37.613+0000 7f0ffbc94640 1 --2- 172.21.15.43:0/1927182610 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f0ff406cfe0 0x7f0ff413cab0 unknown :-1 s=CLOSED pgs=2056 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T14:52:37.623 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:37.613+0000 7f0ffbc94640 1 -- 172.21.15.43:0/1927182610 >> 172.21.15.43:0/1927182610 conn(0x7f0ff40ce0a0 msgr2=0x7f0ff40c8b50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-09T14:52:37.623 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:37.613+0000 7f0ffbc94640 1 -- 172.21.15.43:0/1927182610 shutdown_connections 2023-08-09T14:52:37.623 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:37.613+0000 7f0ffbc94640 1 -- 172.21.15.43:0/1927182610 wait complete. 2023-08-09T14:52:37.633 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2023-08-09T14:52:37.864 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:37.864+0000 7f6f68457640 1 Processor -- start 2023-08-09T14:52:37.865 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:37.864+0000 7f6f68457640 1 -- start start 2023-08-09T14:52:37.866 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:37.865+0000 7f6f68457640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f6f600b2790 0x7f6f600d2ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T14:52:37.866 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:37.865+0000 7f6f68457640 1 -- --> v1:172.21.15.43:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6f600ce090 con 0x7f6f600b3050 2023-08-09T14:52:37.866 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:37.866+0000 7f6f68457640 1 -- --> v1:172.21.15.203:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6f600cd7f0 con 0x7f6f600b2b70 2023-08-09T14:52:37.867 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:37.866+0000 7f6f68457640 1 -- --> v2:172.21.15.43:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6f600b3950 con 0x7f6f600b2790 2023-08-09T14:52:37.867 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:37.866+0000 7f6f661f3640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f6f600b2790 0x7f6f600d2ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T14:52:37.867 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:37.866+0000 7f6f669f4640 1 --1- >> v1:172.21.15.43:6789/0 conn(0x7f6f600b3050 0x7f6f600b3f60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.43:6789/0 says I am v1:172.21.15.43:46268/0 (socket says 172.21.15.43:46268) 2023-08-09T14:52:37.867 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:37.866+0000 7f6f669f4640 1 -- 172.21.15.43:0/2672195211 learned_addr learned my addr 172.21.15.43:0/2672195211 (peer_addr_for_me v1:172.21.15.43:0/0) 2023-08-09T14:52:37.868 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:37.866+0000 7f6f661f3640 1 -- 172.21.15.43:0/2672195211 >> v1:172.21.15.203:6789/0 conn(0x7f6f600b2b70 legacy=0x7f6f600b5330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-09T14:52:37.868 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:37.866+0000 7f6f661f3640 1 -- 172.21.15.43:0/2672195211 >> v1:172.21.15.43:6789/0 conn(0x7f6f600b3050 legacy=0x7f6f600b3f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T14:52:37.868 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:37.866+0000 7f6f661f3640 1 -- 172.21.15.43:0/2672195211 --> v2:172.21.15.43:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6f600cdad0 con 0x7f6f600b2790 2023-08-09T14:52:37.868 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:37.866+0000 7f6f651f1640 1 -- 172.21.15.43:0/2672195211 <== mon.0 v1:172.21.15.43:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4157491616 0 0) 0x7f6f600ce090 con 0x7f6f600b3050 2023-08-09T14:52:37.869 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:37.868+0000 7f6f661f3640 1 --2- 172.21.15.43:0/2672195211 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f6f600b2790 0x7f6f600d2ed0 secure :-1 s=READY pgs=2062 cs=0 l=1 rev1=1 crypto rx=0x7f6f5c0097a0 tx=0x7f6f5c02b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2b77582122990c6c server_cookie=0 in_seq=0 out_seq=0 2023-08-09T14:52:37.869 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:37.868+0000 7f6f651f1640 1 -- 172.21.15.43:0/2672195211 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6f5c031830 con 0x7f6f600b2790 2023-08-09T14:52:37.869 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:37.868+0000 7f6f651f1640 1 -- 172.21.15.43:0/2672195211 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6f5c031990 con 0x7f6f600b2790 2023-08-09T14:52:37.869 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:37.868+0000 7f6f651f1640 1 -- 172.21.15.43:0/2672195211 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6f5c031d20 con 0x7f6f600b2790 2023-08-09T14:52:37.869 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:37.869+0000 7f6f68457640 1 -- 172.21.15.43:0/2672195211 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f6f600b2790 msgr2=0x7f6f600d2ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T14:52:37.869 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:37.869+0000 7f6f68457640 1 --2- 172.21.15.43:0/2672195211 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f6f600b2790 0x7f6f600d2ed0 secure :-1 s=READY pgs=2062 cs=0 l=1 rev1=1 crypto rx=0x7f6f5c0097a0 tx=0x7f6f5c02b8d0 comp rx=0 tx=0).stop 2023-08-09T14:52:37.870 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:37.869+0000 7f6f68457640 1 -- 172.21.15.43:0/2672195211 shutdown_connections 2023-08-09T14:52:37.870 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:37.869+0000 7f6f68457640 1 --2- 172.21.15.43:0/2672195211 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f6f600b2790 0x7f6f600d2ed0 unknown :-1 s=CLOSED pgs=2062 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T14:52:37.870 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:37.869+0000 7f6f68457640 1 -- 172.21.15.43:0/2672195211 >> 172.21.15.43:0/2672195211 conn(0x7f6f6009b7b0 msgr2=0x7f6f600d24e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-09T14:52:37.870 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:37.869+0000 7f6f68457640 1 -- 172.21.15.43:0/2672195211 shutdown_connections 2023-08-09T14:52:37.871 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:37.869+0000 7f6f68457640 1 -- 172.21.15.43:0/2672195211 wait complete. 2023-08-09T14:52:37.871 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:37.871+0000 7f6f68457640 1 Processor -- start 2023-08-09T14:52:37.871 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:37.871+0000 7f6f68457640 1 -- start start 2023-08-09T14:52:37.872 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:37.872+0000 7f6f68457640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f6f600b2790 0x7f6f601f60b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T14:52:37.873 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:37.872+0000 7f6f68457640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f6f600b2b70 0x7f6f6009f6f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T14:52:37.873 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:37.872+0000 7f6f68457640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f6f600b3050 0x7f6f600c8120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T14:52:37.873 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:37.872+0000 7f6f68457640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6f601f5c20 con 0x7f6f600b2790 2023-08-09T14:52:37.873 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:37.872+0000 7f6f68457640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6f601f5610 con 0x7f6f600b2b70 2023-08-09T14:52:37.873 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:37.872+0000 7f6f68457640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6f600c8be0 con 0x7f6f600b3050 2023-08-09T14:52:37.873 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:37.872+0000 7f6f661f3640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f6f600b2790 0x7f6f601f60b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T14:52:37.873 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:37.872+0000 7f6f661f3640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f6f600b2790 0x7f6f601f60b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:36394/0 (socket says 172.21.15.43:36394) 2023-08-09T14:52:37.873 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:37.872+0000 7f6f661f3640 1 -- 172.21.15.43:0/3848713568 learned_addr learned my addr 172.21.15.43:0/3848713568 (peer_addr_for_me v2:172.21.15.43:0/0) 2023-08-09T14:52:37.875 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:37.872+0000 7f6f659f2640 1 --2- 172.21.15.43:0/3848713568 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f6f600b2b70 0x7f6f6009f6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T14:52:37.875 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:37.873+0000 7f6f669f4640 1 --2- 172.21.15.43:0/3848713568 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f6f600b3050 0x7f6f600c8120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T14:52:37.875 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:37.873+0000 7f6f661f3640 1 -- 172.21.15.43:0/3848713568 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f6f600b3050 msgr2=0x7f6f600c8120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T14:52:37.875 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:37.873+0000 7f6f661f3640 1 --2- 172.21.15.43:0/3848713568 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f6f600b3050 0x7f6f600c8120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T14:52:37.875 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:37.873+0000 7f6f661f3640 1 -- 172.21.15.43:0/3848713568 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f6f600b2b70 msgr2=0x7f6f6009f6f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T14:52:37.875 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:37.873+0000 7f6f661f3640 1 --2- 172.21.15.43:0/3848713568 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f6f600b2b70 0x7f6f6009f6f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T14:52:37.875 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:37.873+0000 7f6f661f3640 1 -- 172.21.15.43:0/3848713568 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6f5c009400 con 0x7f6f600b2790 2023-08-09T14:52:37.876 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:37.873+0000 7f6f659f2640 1 --2- 172.21.15.43:0/3848713568 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f6f600b2b70 0x7f6f6009f6f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-08-09T14:52:37.876 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:37.873+0000 7f6f661f3640 1 --2- 172.21.15.43:0/3848713568 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f6f600b2790 0x7f6f601f60b0 secure :-1 s=READY pgs=2063 cs=0 l=1 rev1=1 crypto rx=0x7f6f5c000c00 tx=0x7f6f5c0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-09T14:52:37.876 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:37.873+0000 7f6f56ffd640 1 -- 172.21.15.43:0/3848713568 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6f5c004230 con 0x7f6f600b2790 2023-08-09T14:52:37.876 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:37.873+0000 7f6f56ffd640 1 -- 172.21.15.43:0/3848713568 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6f5c004390 con 0x7f6f600b2790 2023-08-09T14:52:37.876 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:37.873+0000 7f6f68457640 1 -- 172.21.15.43:0/3848713568 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6f601f58b0 con 0x7f6f600b2790 2023-08-09T14:52:37.876 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:37.873+0000 7f6f56ffd640 1 -- 172.21.15.43:0/3848713568 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6f5c03a910 con 0x7f6f600b2790 2023-08-09T14:52:37.876 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:37.874+0000 7f6f68457640 1 -- 172.21.15.43:0/3848713568 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6f6009fc30 con 0x7f6f600b2790 2023-08-09T14:52:37.877 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:37.875+0000 7f6f68457640 1 -- 172.21.15.43:0/3848713568 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6f600739f0 con 0x7f6f600b2790 2023-08-09T14:52:37.877 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:37.876+0000 7f6f56ffd640 1 -- 172.21.15.43:0/3848713568 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 88) v1 ==== 88443+0+0 (secure 0 0 0) 0x7f6f5c03aa70 con 0x7f6f600b2790 2023-08-09T14:52:37.878 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:37.877+0000 7f6f56ffd640 1 --2- 172.21.15.43:0/3848713568 >> [v2:172.21.15.203:6832/2959400181,v1:172.21.15.203:6833/2959400181] conn(0x7f6f4406a170 0x7f6f4406c630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T14:52:37.878 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:37.877+0000 7f6f56ffd640 1 -- 172.21.15.43:0/3848713568 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(186..186 src has 1..186) v4 ==== 6426+0+0 (secure 0 0 0) 0x7f6f5c0ab150 con 0x7f6f600b2790 2023-08-09T14:52:37.878 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:37.878+0000 7f6f659f2640 1 --2- 172.21.15.43:0/3848713568 >> [v2:172.21.15.203:6832/2959400181,v1:172.21.15.203:6833/2959400181] conn(0x7f6f4406a170 0x7f6f4406c630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T14:52:37.879 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:37.879+0000 7f6f659f2640 1 --2- 172.21.15.43:0/3848713568 >> [v2:172.21.15.203:6832/2959400181,v1:172.21.15.203:6833/2959400181] conn(0x7f6f4406a170 0x7f6f4406c630 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f6f60159d70 tx=0x7f6f50008040 comp rx=0 tx=0).ready entity=mgr.7223 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-09T14:52:37.882 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:37.881+0000 7f6f56ffd640 1 -- 172.21.15.43:0/3848713568 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178066 (secure 0 0 0) 0x7f6f5c0765b0 con 0x7f6f600b2790 2023-08-09T14:52:38.115 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:38.115+0000 7f6f68457640 1 -- 172.21.15.43:0/3848713568 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7f6f600739f0 con 0x7f6f600b2790 2023-08-09T14:52:38.116 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:38.116+0000 7f6f56ffd640 1 -- 172.21.15.43:0/3848713568 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v87) v1 ==== 53+0+73 (secure 0 0 0) 0x7f6f5c079bd0 con 0x7f6f600b2790 2023-08-09T14:52:38.116 INFO:teuthology.orchestra.run.smithi043.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2023-08-09T14:52:38.119 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:38.118+0000 7f6f68457640 1 -- 172.21.15.43:0/3848713568 >> [v2:172.21.15.203:6832/2959400181,v1:172.21.15.203:6833/2959400181] conn(0x7f6f4406a170 msgr2=0x7f6f4406c630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T14:52:38.119 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:38.118+0000 7f6f68457640 1 --2- 172.21.15.43:0/3848713568 >> [v2:172.21.15.203:6832/2959400181,v1:172.21.15.203:6833/2959400181] conn(0x7f6f4406a170 0x7f6f4406c630 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f6f60159d70 tx=0x7f6f50008040 comp rx=0 tx=0).stop 2023-08-09T14:52:38.119 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:38.119+0000 7f6f68457640 1 -- 172.21.15.43:0/3848713568 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f6f600b2790 msgr2=0x7f6f601f60b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T14:52:38.120 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:38.119+0000 7f6f68457640 1 --2- 172.21.15.43:0/3848713568 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f6f600b2790 0x7f6f601f60b0 secure :-1 s=READY pgs=2063 cs=0 l=1 rev1=1 crypto rx=0x7f6f5c000c00 tx=0x7f6f5c0046c0 comp rx=0 tx=0).stop 2023-08-09T14:52:38.120 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:38.119+0000 7f6f68457640 1 -- 172.21.15.43:0/3848713568 shutdown_connections 2023-08-09T14:52:38.120 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:38.119+0000 7f6f68457640 1 --2- 172.21.15.43:0/3848713568 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f6f600b3050 0x7f6f600c8120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T14:52:38.120 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:38.119+0000 7f6f68457640 1 --2- 172.21.15.43:0/3848713568 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f6f600b2b70 0x7f6f6009f6f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T14:52:38.120 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:38.119+0000 7f6f68457640 1 --2- 172.21.15.43:0/3848713568 >> [v2:172.21.15.203:6832/2959400181,v1:172.21.15.203:6833/2959400181] conn(0x7f6f4406a170 0x7f6f4406c630 unknown :-1 s=CLOSED pgs=71 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T14:52:38.120 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:38.119+0000 7f6f68457640 1 --2- 172.21.15.43:0/3848713568 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f6f600b2790 0x7f6f601f60b0 unknown :-1 s=CLOSED pgs=2063 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T14:52:38.120 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:38.119+0000 7f6f68457640 1 -- 172.21.15.43:0/3848713568 >> 172.21.15.43:0/3848713568 conn(0x7f6f6009b7b0 msgr2=0x7f6f6006bc70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-09T14:52:38.121 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:38.119+0000 7f6f68457640 1 -- 172.21.15.43:0/3848713568 shutdown_connections 2023-08-09T14:52:38.121 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:38.119+0000 7f6f68457640 1 -- 172.21.15.43:0/3848713568 wait complete. 2023-08-09T14:52:38.140 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2023-08-09T14:52:38.140 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2023-08-09T14:52:38.141 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi043.front.sepia.ceph.com 2023-08-09T14:52:38.141 INFO:tasks.cephfs.mount:self.client.name = client.0 2023-08-09T14:52:38.141 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2023-08-09T14:52:38.141 INFO:tasks.cephfs.mount:self.cephfs_name = None 2023-08-09T14:52:38.142 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2023-08-09T14:52:38.142 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2023-08-09T14:52:38.142 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2023-08-09T14:52:38.143 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-09T14:52:38.143 DEBUG:teuthology.orchestra.run.smithi043:> ip addr 2023-08-09T14:52:38.204 INFO:teuthology.orchestra.run.smithi043.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2023-08-09T14:52:38.204 INFO:teuthology.orchestra.run.smithi043.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2023-08-09T14:52:38.204 INFO:teuthology.orchestra.run.smithi043.stdout: inet 127.0.0.1/8 scope host lo 2023-08-09T14:52:38.204 INFO:teuthology.orchestra.run.smithi043.stdout: valid_lft forever preferred_lft forever 2023-08-09T14:52:38.204 INFO:teuthology.orchestra.run.smithi043.stdout: inet6 ::1/128 scope host 2023-08-09T14:52:38.205 INFO:teuthology.orchestra.run.smithi043.stdout: valid_lft forever preferred_lft forever 2023-08-09T14:52:38.205 INFO:teuthology.orchestra.run.smithi043.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-08-09T14:52:38.205 INFO:teuthology.orchestra.run.smithi043.stdout: link/ether 0c:c4:7a:6b:fd:9c brd ff:ff:ff:ff:ff:ff 2023-08-09T14:52:38.205 INFO:teuthology.orchestra.run.smithi043.stdout: altname enp7s0f0 2023-08-09T14:52:38.205 INFO:teuthology.orchestra.run.smithi043.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-08-09T14:52:38.205 INFO:teuthology.orchestra.run.smithi043.stdout: link/ether 0c:c4:7a:6b:fd:9d brd ff:ff:ff:ff:ff:ff 2023-08-09T14:52:38.205 INFO:teuthology.orchestra.run.smithi043.stdout: altname enp7s0f1 2023-08-09T14:52:38.206 INFO:teuthology.orchestra.run.smithi043.stdout:4: ens1f0: mtu 1500 qdisc mq state UP group default qlen 1000 2023-08-09T14:52:38.206 INFO:teuthology.orchestra.run.smithi043.stdout: link/ether 0c:c4:7a:bd:13:58 brd ff:ff:ff:ff:ff:ff 2023-08-09T14:52:38.206 INFO:teuthology.orchestra.run.smithi043.stdout: altname enp1s0f0 2023-08-09T14:52:38.206 INFO:teuthology.orchestra.run.smithi043.stdout: inet 172.21.15.43/20 brd 172.21.15.255 scope global dynamic noprefixroute ens1f0 2023-08-09T14:52:38.206 INFO:teuthology.orchestra.run.smithi043.stdout: valid_lft 37545sec preferred_lft 37545sec 2023-08-09T14:52:38.206 INFO:teuthology.orchestra.run.smithi043.stdout: inet6 fe80::ec4:7aff:febd:1358/64 scope link 2023-08-09T14:52:38.206 INFO:teuthology.orchestra.run.smithi043.stdout: valid_lft forever preferred_lft forever 2023-08-09T14:52:38.206 INFO:teuthology.orchestra.run.smithi043.stdout:5: ens1f1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-08-09T14:52:38.207 INFO:teuthology.orchestra.run.smithi043.stdout: link/ether 0c:c4:7a:bd:13:59 brd ff:ff:ff:ff:ff:ff 2023-08-09T14:52:38.207 INFO:teuthology.orchestra.run.smithi043.stdout: altname enp1s0f1 2023-08-09T14:52:38.207 INFO:teuthology.orchestra.run.smithi043.stdout:8: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2023-08-09T14:52:38.207 INFO:teuthology.orchestra.run.smithi043.stdout: link/ether 56:43:50:19:f4:0c brd ff:ff:ff:ff:ff:ff 2023-08-09T14:52:38.207 INFO:teuthology.orchestra.run.smithi043.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2023-08-09T14:52:38.207 INFO:teuthology.orchestra.run.smithi043.stdout: valid_lft forever preferred_lft forever 2023-08-09T14:52:38.207 INFO:teuthology.orchestra.run.smithi043.stdout: inet6 fe80::e494:65ff:fe92:a182/64 scope link 2023-08-09T14:52:38.207 INFO:teuthology.orchestra.run.smithi043.stdout: valid_lft forever preferred_lft forever 2023-08-09T14:52:38.208 INFO:teuthology.orchestra.run.smithi043.stdout:9: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2023-08-09T14:52:38.209 INFO:teuthology.orchestra.run.smithi043.stdout: link/ether 7e:93:26:fa:07:8f brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.0 2023-08-09T14:52:38.209 INFO:teuthology.orchestra.run.smithi043.stdout: inet6 fe80::7c93:26ff:fefa:78f/64 scope link 2023-08-09T14:52:38.209 INFO:teuthology.orchestra.run.smithi043.stdout: valid_lft forever preferred_lft forever 2023-08-09T14:52:38.209 INFO:teuthology.orchestra.run.smithi043.stdout:10: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2023-08-09T14:52:38.209 INFO:teuthology.orchestra.run.smithi043.stdout: link/ether 56:43:50:19:f4:0c brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.2 2023-08-09T14:52:38.209 INFO:teuthology.orchestra.run.smithi043.stdout: inet6 fe80::5443:50ff:fe19:f40c/64 scope link 2023-08-09T14:52:38.209 INFO:teuthology.orchestra.run.smithi043.stdout: valid_lft forever preferred_lft forever 2023-08-09T14:52:38.210 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-09T14:52:38.210 DEBUG:teuthology.orchestra.run.smithi043:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2023-08-09T14:52:38.310 INFO:teuthology.orchestra.run.smithi043.stdout:1 2023-08-09T14:52:38.313 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-09T14:52:38.314 DEBUG:teuthology.orchestra.run.smithi043:> route 2023-08-09T14:52:38.370 INFO:teuthology.orchestra.run.smithi043.stdout:Kernel IP routing table 2023-08-09T14:52:38.370 INFO:teuthology.orchestra.run.smithi043.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2023-08-09T14:52:38.370 INFO:teuthology.orchestra.run.smithi043.stdout:default _gateway 0.0.0.0 UG 100 0 0 ens1f0 2023-08-09T14:52:38.370 INFO:teuthology.orchestra.run.smithi043.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 ens1f0 2023-08-09T14:52:38.370 INFO:teuthology.orchestra.run.smithi043.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2023-08-09T14:52:38.371 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-09T14:52:38.372 DEBUG:teuthology.orchestra.run.smithi043:> (cd / && exec bash -c ' 2023-08-09T14:52:38.372 DEBUG:teuthology.orchestra.run.smithi043:> set -e 2023-08-09T14:52:38.372 DEBUG:teuthology.orchestra.run.smithi043:> sudo iptables -A FORWARD -o ens1f0 -i ceph-brx -j ACCEPT 2023-08-09T14:52:38.373 DEBUG:teuthology.orchestra.run.smithi043:> sudo iptables -A FORWARD -i ens1f0 -o ceph-brx -j ACCEPT 2023-08-09T14:52:38.373 DEBUG:teuthology.orchestra.run.smithi043:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o ens1f0 -j MASQUERADE 2023-08-09T14:52:38.373 DEBUG:teuthology.orchestra.run.smithi043:> ') 2023-08-09T14:52:38.569 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-09T14:52:38.569 DEBUG:teuthology.orchestra.run.smithi043:> ip netns list 2023-08-09T14:52:38.628 INFO:teuthology.orchestra.run.smithi043.stdout:ceph-ns--home-ubuntu-cephtest-mnt.2 (id: 1) 2023-08-09T14:52:38.628 INFO:teuthology.orchestra.run.smithi043.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2023-08-09T14:52:38.628 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2023-08-09T14:52:38.629 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-09T14:52:38.629 DEBUG:teuthology.orchestra.run.smithi043:> sudo ip link set brx.0 up 2023-08-09T14:52:38.736 INFO:tasks.cephfs.fuse_mount:Client client.0 config is {} 2023-08-09T14:52:38.737 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-08-09T14:52:38.737 DEBUG:teuthology.orchestra.run.smithi043:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2023-08-09T14:52:38.799 INFO:teuthology.orchestra.run.smithi043.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2023-08-09T14:52:38.800 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-08-09T14:52:38.800 DEBUG:teuthology.orchestra.run.smithi043:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2023-08-09T14:52:38.855 DEBUG:teuthology.orchestra.run.smithi043:> sudo modprobe fuse 2023-08-09T14:52:38.961 INFO:teuthology.orchestra.run:Running command with timeout 30 2023-08-09T14:52:38.962 DEBUG:teuthology.orchestra.run.smithi043:> sudo mount -t fusectl /sys/fs/fuse/connections /sys/fs/fuse/connections 2023-08-09T14:52:39.067 INFO:teuthology.orchestra.run.smithi043.stderr:mount: /sys/fs/fuse/connections: /sys/fs/fuse/connections already mounted or mount point busy. 2023-08-09T14:52:39.071 DEBUG:teuthology.orchestra.run:got remote process result: 32 2023-08-09T14:52:39.071 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-09T14:52:39.072 DEBUG:teuthology.orchestra.run.smithi043:> ls /sys/fs/fuse/connections 2023-08-09T14:52:39.132 INFO:tasks.cephfs.fuse_mount:Pre-mount connections: [] 2023-08-09T14:52:39.133 DEBUG:teuthology.orchestra.run.smithi043:> (cd /home/ubuntu/cephtest && exec sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-fuse -f --admin-socket '/var/run/ceph/$cluster-$name.$pid.asok' /home/ubuntu/cephtest/mnt.0 --id 0) 2023-08-09T14:52:39.139 DEBUG:teuthology.orchestra.run.smithi043:> sudo modprobe fuse 2023-08-09T14:52:39.219 INFO:teuthology.orchestra.run:Running command with timeout 30 2023-08-09T14:52:39.220 DEBUG:teuthology.orchestra.run.smithi043:> sudo mount -t fusectl /sys/fs/fuse/connections /sys/fs/fuse/connections 2023-08-09T14:52:39.296 INFO:teuthology.orchestra.run.smithi043.stderr:mount: /sys/fs/fuse/connections: /sys/fs/fuse/connections already mounted or mount point busy. 2023-08-09T14:52:39.299 DEBUG:teuthology.orchestra.run:got remote process result: 32 2023-08-09T14:52:39.300 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-09T14:52:39.300 DEBUG:teuthology.orchestra.run.smithi043:> ls /sys/fs/fuse/connections 2023-08-09T14:52:39.332 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi043.stderr:2023-08-09T14:52:39.332+0000 7f2f96c2a480 -1 init, newargv = 0x558805ed9540 newargc=15 2023-08-09T14:52:39.333 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi043.stderr:ceph-fuse[143425]: starting ceph client 2023-08-09T14:52:39.342 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi043.stderr:ceph-fuse[143425]: starting fuse 2023-08-09T14:52:39.349 INFO:teuthology.orchestra.run.smithi043.stdout:40 2023-08-09T14:52:39.350 INFO:tasks.cephfs.fuse_mount:Post-mount connections: [40] 2023-08-09T14:52:39.351 DEBUG:teuthology.orchestra.run.smithi043:> sudo stdin-killer -- python3 -c ' 2023-08-09T14:52:39.351 DEBUG:teuthology.orchestra.run.smithi043:> import glob 2023-08-09T14:52:39.351 DEBUG:teuthology.orchestra.run.smithi043:> import re 2023-08-09T14:52:39.352 DEBUG:teuthology.orchestra.run.smithi043:> import os 2023-08-09T14:52:39.352 DEBUG:teuthology.orchestra.run.smithi043:> import subprocess 2023-08-09T14:52:39.352 DEBUG:teuthology.orchestra.run.smithi043:> 2023-08-09T14:52:39.353 DEBUG:teuthology.orchestra.run.smithi043:> def _find_admin_socket(client_name): 2023-08-09T14:52:39.353 DEBUG:teuthology.orchestra.run.smithi043:> asok_path = "/var/run/ceph/ceph-client.0.*.asok" 2023-08-09T14:52:39.353 DEBUG:teuthology.orchestra.run.smithi043:> files = glob.glob(asok_path) 2023-08-09T14:52:39.354 DEBUG:teuthology.orchestra.run.smithi043:> mountpoint = "/home/ubuntu/cephtest/mnt.0" 2023-08-09T14:52:39.354 DEBUG:teuthology.orchestra.run.smithi043:> 2023-08-09T14:52:39.354 DEBUG:teuthology.orchestra.run.smithi043:> # Given a non-glob path, it better be there 2023-08-09T14:52:39.355 DEBUG:teuthology.orchestra.run.smithi043:> if "*" not in asok_path: 2023-08-09T14:52:39.355 DEBUG:teuthology.orchestra.run.smithi043:> assert(len(files) == 1) 2023-08-09T14:52:39.355 DEBUG:teuthology.orchestra.run.smithi043:> return files[0] 2023-08-09T14:52:39.356 DEBUG:teuthology.orchestra.run.smithi043:> 2023-08-09T14:52:39.356 DEBUG:teuthology.orchestra.run.smithi043:> for f in files: 2023-08-09T14:52:39.356 DEBUG:teuthology.orchestra.run.smithi043:> pid = re.match(".*\.(\d+)\.asok$", f).group(1) 2023-08-09T14:52:39.357 DEBUG:teuthology.orchestra.run.smithi043:> if os.path.exists("/proc/{0}".format(pid)): 2023-08-09T14:52:39.357 DEBUG:teuthology.orchestra.run.smithi043:> with open("/proc/{0}/cmdline".format(pid), '"'"'r'"'"') as proc_f: 2023-08-09T14:52:39.357 DEBUG:teuthology.orchestra.run.smithi043:> contents = proc_f.read() 2023-08-09T14:52:39.358 DEBUG:teuthology.orchestra.run.smithi043:> if mountpoint in contents: 2023-08-09T14:52:39.358 DEBUG:teuthology.orchestra.run.smithi043:> return f 2023-08-09T14:52:39.358 DEBUG:teuthology.orchestra.run.smithi043:> raise RuntimeError("Client socket {0} not found".format(client_name)) 2023-08-09T14:52:39.358 DEBUG:teuthology.orchestra.run.smithi043:> 2023-08-09T14:52:39.359 DEBUG:teuthology.orchestra.run.smithi043:> print(_find_admin_socket("client.0")) 2023-08-09T14:52:39.359 DEBUG:teuthology.orchestra.run.smithi043:> ' 2023-08-09T14:52:39.578 INFO:teuthology.orchestra.run.smithi043.stdout:/var/run/ceph/ceph-client.0.143425.asok 2023-08-09T14:52:39.583 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:39 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2023-08-09T14:52:39.597 INFO:tasks.cephfs.fuse_mount:Found client admin socket at /var/run/ceph/ceph-client.0.143425.asok 2023-08-09T14:52:39.598 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-09T14:52:39.598 DEBUG:teuthology.orchestra.run.smithi043:> sudo ceph --admin-daemon /var/run/ceph/ceph-client.0.143425.asok status 2023-08-09T14:52:39.852 INFO:teuthology.orchestra.run.smithi043.stdout:{ 2023-08-09T14:52:39.852 INFO:teuthology.orchestra.run.smithi043.stdout: "metadata": { 2023-08-09T14:52:39.852 INFO:teuthology.orchestra.run.smithi043.stdout: "ceph_sha1": "668a86f232c9454b2ce5928d1432a29a72964d85", 2023-08-09T14:52:39.853 INFO:teuthology.orchestra.run.smithi043.stdout: "ceph_version": "ceph version 18.0.0-5389-g668a86f2 (668a86f232c9454b2ce5928d1432a29a72964d85) reef (dev)", 2023-08-09T14:52:39.853 INFO:teuthology.orchestra.run.smithi043.stdout: "entity_id": "0", 2023-08-09T14:52:39.853 INFO:teuthology.orchestra.run.smithi043.stdout: "hostname": "smithi043", 2023-08-09T14:52:39.854 INFO:teuthology.orchestra.run.smithi043.stdout: "mount_point": "/home/ubuntu/cephtest/mnt.0", 2023-08-09T14:52:39.854 INFO:teuthology.orchestra.run.smithi043.stdout: "pid": "143425", 2023-08-09T14:52:39.854 INFO:teuthology.orchestra.run.smithi043.stdout: "root": "/" 2023-08-09T14:52:39.854 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2023-08-09T14:52:39.854 INFO:teuthology.orchestra.run.smithi043.stdout: "dentry_count": 0, 2023-08-09T14:52:39.855 INFO:teuthology.orchestra.run.smithi043.stdout: "dentry_pinned_count": 0, 2023-08-09T14:52:39.855 INFO:teuthology.orchestra.run.smithi043.stdout: "id": 10365, 2023-08-09T14:52:39.855 INFO:teuthology.orchestra.run.smithi043.stdout: "inst": { 2023-08-09T14:52:39.855 INFO:teuthology.orchestra.run.smithi043.stdout: "name": { 2023-08-09T14:52:39.855 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "client", 2023-08-09T14:52:39.855 INFO:teuthology.orchestra.run.smithi043.stdout: "num": 10365 2023-08-09T14:52:39.855 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2023-08-09T14:52:39.855 INFO:teuthology.orchestra.run.smithi043.stdout: "addr": { 2023-08-09T14:52:39.856 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "v1", 2023-08-09T14:52:39.856 INFO:teuthology.orchestra.run.smithi043.stdout: "addr": "192.168.0.1:0", 2023-08-09T14:52:39.856 INFO:teuthology.orchestra.run.smithi043.stdout: "nonce": 1546800009 2023-08-09T14:52:39.856 INFO:teuthology.orchestra.run.smithi043.stdout: } 2023-08-09T14:52:39.856 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2023-08-09T14:52:39.856 INFO:teuthology.orchestra.run.smithi043.stdout: "addr": { 2023-08-09T14:52:39.856 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "v1", 2023-08-09T14:52:39.857 INFO:teuthology.orchestra.run.smithi043.stdout: "addr": "192.168.0.1:0", 2023-08-09T14:52:39.857 INFO:teuthology.orchestra.run.smithi043.stdout: "nonce": 1546800009 2023-08-09T14:52:39.857 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2023-08-09T14:52:39.857 INFO:teuthology.orchestra.run.smithi043.stdout: "inst_str": "client.10365 192.168.0.1:0/1546800009", 2023-08-09T14:52:39.857 INFO:teuthology.orchestra.run.smithi043.stdout: "addr_str": "192.168.0.1:0/1546800009", 2023-08-09T14:52:39.857 INFO:teuthology.orchestra.run.smithi043.stdout: "inode_count": 1, 2023-08-09T14:52:39.857 INFO:teuthology.orchestra.run.smithi043.stdout: "mds_epoch": 87, 2023-08-09T14:52:39.857 INFO:teuthology.orchestra.run.smithi043.stdout: "osd_epoch": 186, 2023-08-09T14:52:39.858 INFO:teuthology.orchestra.run.smithi043.stdout: "osd_epoch_barrier": 0, 2023-08-09T14:52:39.858 INFO:teuthology.orchestra.run.smithi043.stdout: "blocklisted": false, 2023-08-09T14:52:39.858 INFO:teuthology.orchestra.run.smithi043.stdout: "fs_name": "cephfs" 2023-08-09T14:52:39.858 INFO:teuthology.orchestra.run.smithi043.stdout:} 2023-08-09T14:52:39.866 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-09T14:52:39.867 DEBUG:teuthology.orchestra.run.smithi043:> stat --file-system '--printf=%T 2023-08-09T14:52:39.867 DEBUG:teuthology.orchestra.run.smithi043:> ' -- /home/ubuntu/cephtest/mnt.0 2023-08-09T14:52:39.927 INFO:teuthology.orchestra.run.smithi043.stdout:fuseblk 2023-08-09T14:52:39.927 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.0 2023-08-09T14:52:39.928 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-09T14:52:39.928 DEBUG:teuthology.orchestra.run.smithi043:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2023-08-09T14:52:40.028 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs volume ls 2023-08-09T14:52:40.262 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:40.262+0000 7fb8ade52640 1 Processor -- start 2023-08-09T14:52:40.262 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:40.262+0000 7fb8ade52640 1 -- start start 2023-08-09T14:52:40.264 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:40.263+0000 7fb8ade52640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fb8a80c8670 0x7fb8a815c8e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T14:52:40.264 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:40.264+0000 7fb8ade52640 1 -- --> v1:172.21.15.43:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb8a8070730 con 0x7fb8a80abbb0 2023-08-09T14:52:40.264 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:40.264+0000 7fb8ade52640 1 -- --> v1:172.21.15.203:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb8a8071620 con 0x7fb8a806f3d0 2023-08-09T14:52:40.264 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:40.264+0000 7fb8ade52640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb8a8156e60 con 0x7fb8a80c8670 2023-08-09T14:52:40.265 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:40.264+0000 7fb8ad651640 1 --1- >> v1:172.21.15.43:6789/0 conn(0x7fb8a80abbb0 0x7fb8a80ac040 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.43:6789/0 says I am v1:172.21.15.43:46282/0 (socket says 172.21.15.43:46282) 2023-08-09T14:52:40.266 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:40.264+0000 7fb8ad651640 1 -- 172.21.15.43:0/214433305 learned_addr learned my addr 172.21.15.43:0/214433305 (peer_addr_for_me v1:172.21.15.43:0/0) 2023-08-09T14:52:40.266 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:40.264+0000 7fb8ace50640 1 --2- 172.21.15.43:0/214433305 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fb8a80c8670 0x7fb8a815c8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T14:52:40.266 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:40.265+0000 7fb8a77fe640 1 -- 172.21.15.43:0/214433305 <== mon.0 v1:172.21.15.43:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3228999938 0 0) 0x7fb8a8070730 con 0x7fb8a80abbb0 2023-08-09T14:52:40.266 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:40.265+0000 7fb8a77fe640 1 -- 172.21.15.43:0/214433305 --> v1:172.21.15.43:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fb898003520 con 0x7fb8a80abbb0 2023-08-09T14:52:40.266 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:40.265+0000 7fb8a77fe640 1 -- 172.21.15.43:0/214433305 <== mon.0 v1:172.21.15.43:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1284867165 0 0) 0x7fb898003520 con 0x7fb8a80abbb0 2023-08-09T14:52:40.266 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:40.265+0000 7fb8a77fe640 1 -- 172.21.15.43:0/214433305 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fb8a80c8670 msgr2=0x7fb8a815c8e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-09T14:52:40.267 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:40.265+0000 7fb8a77fe640 1 --2- 172.21.15.43:0/214433305 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fb8a80c8670 0x7fb8a815c8e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T14:52:40.267 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:40.265+0000 7fb8a77fe640 1 -- 172.21.15.43:0/214433305 >> v1:172.21.15.203:6789/0 conn(0x7fb8a806f3d0 legacy=0x7fb8a80c8a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T14:52:40.267 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:40.265+0000 7fb8a77fe640 1 -- 172.21.15.43:0/214433305 --> v1:172.21.15.43:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb8a806fb20 con 0x7fb8a80abbb0 2023-08-09T14:52:40.267 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:40.265+0000 7fb8a77fe640 1 -- 172.21.15.43:0/214433305 <== mon.0 v1:172.21.15.43:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1285150740 0 0) 0x7fb8a00019f0 con 0x7fb8a80abbb0 2023-08-09T14:52:40.267 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:40.265+0000 7fb8a77fe640 1 -- 172.21.15.43:0/214433305 >> v1:172.21.15.43:6789/0 conn(0x7fb8a80abbb0 legacy=0x7fb8a80ac040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T14:52:40.267 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:40.265+0000 7fb8a77fe640 1 --2- 172.21.15.43:0/214433305 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fb8980039a0 0x7fb898003dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T14:52:40.267 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:40.265+0000 7fb8a77fe640 1 --2- 172.21.15.43:0/214433305 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb8980047c0 0x7fb898004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T14:52:40.267 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:40.265+0000 7fb8a77fe640 1 --2- 172.21.15.43:0/214433305 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fb8a80c8670 0x7fb898005250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T14:52:40.268 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:40.265+0000 7fb8a77fe640 1 -- 172.21.15.43:0/214433305 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb8a8156e60 con 0x7fb8980047c0 2023-08-09T14:52:40.268 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:40.265+0000 7fb8a77fe640 1 -- 172.21.15.43:0/214433305 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb89800bb40 con 0x7fb8980039a0 2023-08-09T14:52:40.268 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:40.265+0000 7fb8a77fe640 1 -- 172.21.15.43:0/214433305 --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb89800bd00 con 0x7fb8a80c8670 2023-08-09T14:52:40.269 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:40.265+0000 7fb8a7fff640 1 --2- 172.21.15.43:0/214433305 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb8980047c0 0x7fb898004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T14:52:40.269 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:40.266+0000 7fb8ad651640 1 --2- 172.21.15.43:0/214433305 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fb8a80c8670 0x7fb898005250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T14:52:40.270 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:40.266+0000 7fb8a7fff640 1 -- 172.21.15.43:0/214433305 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fb8a80c8670 msgr2=0x7fb898005250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-09T14:52:40.270 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:40.266+0000 7fb8ace50640 1 --2- 172.21.15.43:0/214433305 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fb8980039a0 0x7fb898003dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T14:52:40.270 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:40.266+0000 7fb8a7fff640 1 --2- 172.21.15.43:0/214433305 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fb8a80c8670 0x7fb898005250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T14:52:40.270 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:40.266+0000 7fb8a7fff640 1 -- 172.21.15.43:0/214433305 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fb8980039a0 msgr2=0x7fb898003dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-09T14:52:40.270 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:40.266+0000 7fb8a7fff640 1 --2- 172.21.15.43:0/214433305 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fb8980039a0 0x7fb898003dc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T14:52:40.270 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:40.266+0000 7fb8a7fff640 1 -- 172.21.15.43:0/214433305 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb898003520 con 0x7fb8980047c0 2023-08-09T14:52:40.270 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:40.266+0000 7fb8ad651640 1 --2- 172.21.15.43:0/214433305 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fb8a80c8670 0x7fb898005250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-08-09T14:52:40.271 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:40.267+0000 7fb8a7fff640 1 --2- 172.21.15.43:0/214433305 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb8980047c0 0x7fb898004c40 secure :-1 s=READY pgs=2068 cs=0 l=1 rev1=1 crypto rx=0x7fb894001750 tx=0x7fb8980057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a848cfbd675cbf16 server_cookie=0 in_seq=0 out_seq=0 2023-08-09T14:52:40.271 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:40.268+0000 7fb8a77fe640 1 -- 172.21.15.43:0/214433305 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb8940052d0 con 0x7fb8980047c0 2023-08-09T14:52:40.271 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:40.268+0000 7fb8a77fe640 1 -- 172.21.15.43:0/214433305 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb89402e050 con 0x7fb8980047c0 2023-08-09T14:52:40.271 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:40.268+0000 7fb8a77fe640 1 -- 172.21.15.43:0/214433305 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb8940035f0 con 0x7fb8980047c0 2023-08-09T14:52:40.271 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:40.268+0000 7fb8ade52640 1 -- 172.21.15.43:0/214433305 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb8980047c0 msgr2=0x7fb898004c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T14:52:40.271 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:40.268+0000 7fb8ade52640 1 --2- 172.21.15.43:0/214433305 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb8980047c0 0x7fb898004c40 secure :-1 s=READY pgs=2068 cs=0 l=1 rev1=1 crypto rx=0x7fb894001750 tx=0x7fb8980057f0 comp rx=0 tx=0).stop 2023-08-09T14:52:40.271 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:40.268+0000 7fb8ad651640 1 -- 172.21.15.43:0/214433305 reap_dead start 2023-08-09T14:52:40.271 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:40.268+0000 7fb8ade52640 1 -- 172.21.15.43:0/214433305 shutdown_connections 2023-08-09T14:52:40.272 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:40.268+0000 7fb8ade52640 1 -- 172.21.15.43:0/214433305 >> 172.21.15.43:0/214433305 conn(0x7fb8a80af950 msgr2=0x7fb8a80afd60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-09T14:52:40.272 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:40.269+0000 7fb8ade52640 1 -- 172.21.15.43:0/214433305 shutdown_connections 2023-08-09T14:52:40.272 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:40.269+0000 7fb8ade52640 1 -- 172.21.15.43:0/214433305 wait complete. 2023-08-09T14:52:40.273 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:40.270+0000 7fb8ade52640 1 Processor -- start 2023-08-09T14:52:40.274 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:40.271+0000 7fb8ade52640 1 -- start start 2023-08-09T14:52:40.274 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:40.271+0000 7fb8ade52640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fb8980047c0 0x7fb8a8153340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T14:52:40.274 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:40.271+0000 7fb8ade52640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb8a8151e30 0x7fb8a8151460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T14:52:40.274 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:40.272+0000 7fb8ade52640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fb8a80b1500 0x7fb8a80b2c30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T14:52:40.274 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:40.272+0000 7fb8ade52640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb8a8153880 con 0x7fb8a8151e30 2023-08-09T14:52:40.274 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:40.272+0000 7fb8ade52640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb8a81522c0 con 0x7fb8a80b1500 2023-08-09T14:52:40.274 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:40.272+0000 7fb8ade52640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb8a81525a0 con 0x7fb8980047c0 2023-08-09T14:52:40.275 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:40.272+0000 7fb8a7fff640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb8a8151e30 0x7fb8a8151460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T14:52:40.275 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:40.272+0000 7fb8a7fff640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb8a8151e30 0x7fb8a8151460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:36404/0 (socket says 172.21.15.43:36404) 2023-08-09T14:52:40.275 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:40.272+0000 7fb8a7fff640 1 -- 172.21.15.43:0/2829111435 learned_addr learned my addr 172.21.15.43:0/2829111435 (peer_addr_for_me v2:172.21.15.43:0/0) 2023-08-09T14:52:40.275 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:40.272+0000 7fb8ace50640 1 --2- 172.21.15.43:0/2829111435 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fb8980047c0 0x7fb8a8153340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T14:52:40.275 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:40.272+0000 7fb8ad651640 1 --2- 172.21.15.43:0/2829111435 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fb8a80b1500 0x7fb8a80b2c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T14:52:40.275 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:40.272+0000 7fb8a7fff640 1 -- 172.21.15.43:0/2829111435 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fb8980047c0 msgr2=0x7fb8a8153340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T14:52:40.275 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:40.272+0000 7fb8a7fff640 1 --2- 172.21.15.43:0/2829111435 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fb8980047c0 0x7fb8a8153340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T14:52:40.275 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:40.272+0000 7fb8a7fff640 1 -- 172.21.15.43:0/2829111435 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fb8a80b1500 msgr2=0x7fb8a80b2c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T14:52:40.276 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:40.272+0000 7fb8a7fff640 1 --2- 172.21.15.43:0/2829111435 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fb8a80b1500 0x7fb8a80b2c30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T14:52:40.276 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:40.272+0000 7fb8a7fff640 1 -- 172.21.15.43:0/2829111435 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb8a8152880 con 0x7fb8a8151e30 2023-08-09T14:52:40.276 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:40.272+0000 7fb8ad651640 1 --2- 172.21.15.43:0/2829111435 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fb8a80b1500 0x7fb8a80b2c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-08-09T14:52:40.276 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:40.273+0000 7fb8a7fff640 1 --2- 172.21.15.43:0/2829111435 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb8a8151e30 0x7fb8a8151460 secure :-1 s=READY pgs=2069 cs=0 l=1 rev1=1 crypto rx=0x7fb894034040 tx=0x7fb894001120 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-09T14:52:40.276 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:40.273+0000 7fb8a57fa640 1 -- 172.21.15.43:0/2829111435 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb89403c630 con 0x7fb8a8151e30 2023-08-09T14:52:40.276 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:40.273+0000 7fb8a57fa640 1 -- 172.21.15.43:0/2829111435 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb89403c7b0 con 0x7fb8a8151e30 2023-08-09T14:52:40.276 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:40.273+0000 7fb8a57fa640 1 -- 172.21.15.43:0/2829111435 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb89403cb60 con 0x7fb8a8151e30 2023-08-09T14:52:40.277 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:40.273+0000 7fb8ade52640 1 -- 172.21.15.43:0/2829111435 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb8a8152b60 con 0x7fb8a8151e30 2023-08-09T14:52:40.277 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:40.273+0000 7fb8ade52640 1 -- 172.21.15.43:0/2829111435 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb8a81519a0 con 0x7fb8a8151e30 2023-08-09T14:52:40.278 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:40.275+0000 7fb8ade52640 1 -- 172.21.15.43:0/2829111435 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb8a8151b30 con 0x7fb8a8151e30 2023-08-09T14:52:40.278 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:40.276+0000 7fb8a57fa640 1 -- 172.21.15.43:0/2829111435 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 88) v1 ==== 88443+0+0 (secure 0 0 0) 0x7fb89403ccc0 con 0x7fb8a8151e30 2023-08-09T14:52:40.278 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:40.276+0000 7fb8a57fa640 1 --2- 172.21.15.43:0/2829111435 >> [v2:172.21.15.203:6832/2959400181,v1:172.21.15.203:6833/2959400181] conn(0x7fb88c06a170 0x7fb88c06c630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T14:52:40.278 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:40.277+0000 7fb8a57fa640 1 -- 172.21.15.43:0/2829111435 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(186..186 src has 1..186) v4 ==== 6426+0+0 (secure 0 0 0) 0x7fb8940aa630 con 0x7fb8a8151e30 2023-08-09T14:52:40.278 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:40.277+0000 7fb8ace50640 1 --2- 172.21.15.43:0/2829111435 >> [v2:172.21.15.203:6832/2959400181,v1:172.21.15.203:6833/2959400181] conn(0x7fb88c06a170 0x7fb88c06c630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T14:52:40.279 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:40.279+0000 7fb8ace50640 1 --2- 172.21.15.43:0/2829111435 >> [v2:172.21.15.203:6832/2959400181,v1:172.21.15.203:6833/2959400181] conn(0x7fb88c06a170 0x7fb88c06c630 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7fb89c009a10 tx=0x7fb89c008040 comp rx=0 tx=0).ready entity=mgr.7223 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-09T14:52:40.282 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:40.281+0000 7fb8a57fa640 1 -- 172.21.15.43:0/2829111435 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178066 (secure 0 0 0) 0x7fb894075ab0 con 0x7fb8a8151e30 2023-08-09T14:52:40.505 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:40.504+0000 7fb8ade52640 1 -- 172.21.15.43:0/2829111435 --> [v2:172.21.15.203:6832/2959400181,v1:172.21.15.203:6833/2959400181] -- mgr_command(tid 0: {"prefix": "fs volume ls", "target": ["mon-mgr", ""]}) v1 -- 0x7fb8a8153880 con 0x7fb88c06a170 2023-08-09T14:52:40.509 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:40.508+0000 7fb8a57fa640 1 -- 172.21.15.43:0/2829111435 <== mgr.7223 v2:172.21.15.203:6832/2959400181 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+41 (secure 0 0 0) 0x7fb8a8153880 con 0x7fb88c06a170 2023-08-09T14:52:40.509 INFO:teuthology.orchestra.run.smithi043.stdout:[ 2023-08-09T14:52:40.509 INFO:teuthology.orchestra.run.smithi043.stdout: { 2023-08-09T14:52:40.510 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "cephfs" 2023-08-09T14:52:40.510 INFO:teuthology.orchestra.run.smithi043.stdout: } 2023-08-09T14:52:40.510 INFO:teuthology.orchestra.run.smithi043.stdout:] 2023-08-09T14:52:40.512 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:40.511+0000 7fb8ade52640 1 -- 172.21.15.43:0/2829111435 >> [v2:172.21.15.203:6832/2959400181,v1:172.21.15.203:6833/2959400181] conn(0x7fb88c06a170 msgr2=0x7fb88c06c630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T14:52:40.513 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:40.511+0000 7fb8ade52640 1 --2- 172.21.15.43:0/2829111435 >> [v2:172.21.15.203:6832/2959400181,v1:172.21.15.203:6833/2959400181] conn(0x7fb88c06a170 0x7fb88c06c630 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7fb89c009a10 tx=0x7fb89c008040 comp rx=0 tx=0).stop 2023-08-09T14:52:40.513 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:40.511+0000 7fb8ade52640 1 -- 172.21.15.43:0/2829111435 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb8a8151e30 msgr2=0x7fb8a8151460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T14:52:40.514 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:40.511+0000 7fb8ade52640 1 --2- 172.21.15.43:0/2829111435 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb8a8151e30 0x7fb8a8151460 secure :-1 s=READY pgs=2069 cs=0 l=1 rev1=1 crypto rx=0x7fb894034040 tx=0x7fb894001120 comp rx=0 tx=0).stop 2023-08-09T14:52:40.514 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:40.511+0000 7fb8ade52640 1 -- 172.21.15.43:0/2829111435 shutdown_connections 2023-08-09T14:52:40.514 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:40.511+0000 7fb8ade52640 1 --2- 172.21.15.43:0/2829111435 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fb8a80b1500 0x7fb8a80b2c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T14:52:40.514 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:40.511+0000 7fb8ade52640 1 --2- 172.21.15.43:0/2829111435 >> [v2:172.21.15.203:6832/2959400181,v1:172.21.15.203:6833/2959400181] conn(0x7fb88c06a170 0x7fb88c06c630 unknown :-1 s=CLOSED pgs=72 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T14:52:40.515 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:40.511+0000 7fb8ade52640 1 --2- 172.21.15.43:0/2829111435 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb8a8151e30 0x7fb8a8151460 unknown :-1 s=CLOSED pgs=2069 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T14:52:40.515 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:40.511+0000 7fb8ade52640 1 --2- 172.21.15.43:0/2829111435 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fb8980047c0 0x7fb8a8153340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T14:52:40.515 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:40.511+0000 7fb8ade52640 1 -- 172.21.15.43:0/2829111435 >> 172.21.15.43:0/2829111435 conn(0x7fb8a80af950 msgr2=0x7fb8a80ac630 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-09T14:52:40.515 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:40.511+0000 7fb8ade52640 1 -- 172.21.15.43:0/2829111435 shutdown_connections 2023-08-09T14:52:40.515 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:40.511+0000 7fb8ade52640 1 -- 172.21.15.43:0/2829111435 wait complete. 2023-08-09T14:52:40.530 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph mgr module enable snap_schedule 2023-08-09T14:52:40.761 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:40.761+0000 7f0db982d640 1 Processor -- start 2023-08-09T14:52:40.761 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:40.761+0000 7f0db982d640 1 -- start start 2023-08-09T14:52:40.763 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:40.762+0000 7f0db982d640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f0db4154720 0x7f0db415e760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T14:52:40.763 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:40.762+0000 7f0db982d640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f0db406a130 0x7f0db40ce3b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T14:52:40.763 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:40.763+0000 7f0db982d640 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7f0db40ce8f0 0x7f0db40c9d50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T14:52:40.763 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:40.763+0000 7f0db982d640 1 -- --> v2:172.21.15.43:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0db415eca0 con 0x7f0db406a130 2023-08-09T14:52:40.764 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:40.763+0000 7f0db982d640 1 -- --> v2:172.21.15.203:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0db40ced60 con 0x7f0db40ce8f0 2023-08-09T14:52:40.764 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:40.763+0000 7f0db982d640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0db406fd90 con 0x7f0db4154720 2023-08-09T14:52:40.765 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:40.763+0000 7f0daa7fc640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f0db406a130 0x7f0db40ce3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T14:52:40.765 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:40.763+0000 7f0daa7fc640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f0db406a130 0x7f0db40ce3b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:36416/0 (socket says 172.21.15.43:36416) 2023-08-09T14:52:40.765 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:40.763+0000 7f0daa7fc640 1 -- 172.21.15.43:0/1740177131 learned_addr learned my addr 172.21.15.43:0/1740177131 (peer_addr_for_me v2:172.21.15.43:0/0) 2023-08-09T14:52:40.765 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:40.763+0000 7f0db2ffd640 1 --2- 172.21.15.43:0/1740177131 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f0db4154720 0x7f0db415e760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T14:52:40.765 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:40.763+0000 7f0db37fe640 1 --2- 172.21.15.43:0/1740177131 >> v2:172.21.15.203:3300/0 conn(0x7f0db40ce8f0 0x7f0db40c9d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T14:52:40.765 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:40.764+0000 7f0daa7fc640 1 -- 172.21.15.43:0/1740177131 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f0db4154720 msgr2=0x7f0db415e760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-09T14:52:40.765 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:40.764+0000 7f0daa7fc640 1 --2- 172.21.15.43:0/1740177131 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f0db4154720 0x7f0db415e760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T14:52:40.766 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:40.764+0000 7f0daa7fc640 1 -- 172.21.15.43:0/1740177131 >> v2:172.21.15.203:3300/0 conn(0x7f0db40ce8f0 msgr2=0x7f0db40c9d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-09T14:52:40.766 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:40.764+0000 7f0daa7fc640 1 --2- 172.21.15.43:0/1740177131 >> v2:172.21.15.203:3300/0 conn(0x7f0db40ce8f0 0x7f0db40c9d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T14:52:40.766 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:40.764+0000 7f0daa7fc640 1 -- 172.21.15.43:0/1740177131 --> v2:172.21.15.43:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0db406f4f0 con 0x7f0db406a130 2023-08-09T14:52:40.766 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:40.764+0000 7f0db37fe640 1 --2- 172.21.15.43:0/1740177131 >> v2:172.21.15.203:3300/0 conn(0x7f0db40ce8f0 0x7f0db40c9d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-08-09T14:52:40.767 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:40.765+0000 7f0daa7fc640 1 --2- 172.21.15.43:0/1740177131 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f0db406a130 0x7f0db40ce3b0 secure :-1 s=READY pgs=2070 cs=0 l=1 rev1=1 crypto rx=0x7f0db40698e0 tx=0x7f0d9c02b8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e708b7e6bea3db3f server_cookie=0 in_seq=0 out_seq=0 2023-08-09T14:52:40.767 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:40.766+0000 7f0db27fc640 1 -- 172.21.15.43:0/1740177131 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0d9c031640 con 0x7f0db406a130 2023-08-09T14:52:40.767 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:40.766+0000 7f0db27fc640 1 -- 172.21.15.43:0/1740177131 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0d9c0317a0 con 0x7f0db406a130 2023-08-09T14:52:40.767 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:40.766+0000 7f0db27fc640 1 -- 172.21.15.43:0/1740177131 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0d9c031b30 con 0x7f0db406a130 2023-08-09T14:52:40.768 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:40.766+0000 7f0db982d640 1 -- 172.21.15.43:0/1740177131 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f0db406a130 msgr2=0x7f0db40ce3b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T14:52:40.768 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:40.766+0000 7f0db982d640 1 --2- 172.21.15.43:0/1740177131 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f0db406a130 0x7f0db40ce3b0 secure :-1 s=READY pgs=2070 cs=0 l=1 rev1=1 crypto rx=0x7f0db40698e0 tx=0x7f0d9c02b8f0 comp rx=0 tx=0).stop 2023-08-09T14:52:40.768 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:40.766+0000 7f0db982d640 1 -- 172.21.15.43:0/1740177131 shutdown_connections 2023-08-09T14:52:40.768 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:40.766+0000 7f0db982d640 1 --2- 172.21.15.43:0/1740177131 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f0db406a130 0x7f0db40ce3b0 unknown :-1 s=CLOSED pgs=2070 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T14:52:40.768 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:40.766+0000 7f0db982d640 1 --2- 172.21.15.43:0/1740177131 >> v2:172.21.15.203:3300/0 conn(0x7f0db40ce8f0 0x7f0db40c9d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T14:52:40.768 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:40.767+0000 7f0db982d640 1 --2- 172.21.15.43:0/1740177131 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f0db4154720 0x7f0db415e760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T14:52:40.768 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:40.767+0000 7f0db982d640 1 -- 172.21.15.43:0/1740177131 >> 172.21.15.43:0/1740177131 conn(0x7f0db406b970 msgr2=0x7f0db406bd80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-09T14:52:40.769 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:40.767+0000 7f0db982d640 1 -- 172.21.15.43:0/1740177131 shutdown_connections 2023-08-09T14:52:40.769 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:40.767+0000 7f0db982d640 1 -- 172.21.15.43:0/1740177131 wait complete. 2023-08-09T14:52:40.769 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:40.769+0000 7f0db982d640 1 Processor -- start 2023-08-09T14:52:40.770 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:40.769+0000 7f0db982d640 1 -- start start 2023-08-09T14:52:40.770 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:40.770+0000 7f0db982d640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f0db406a130 0x7f0db415a860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T14:52:40.771 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:40.770+0000 7f0db982d640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f0db40ce8f0 0x7f0db415ada0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T14:52:40.771 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:40.770+0000 7f0db982d640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f0db4154720 0x7f0db4151f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T14:52:40.771 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:40.770+0000 7f0db982d640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0db40c2130 con 0x7f0db406a130 2023-08-09T14:52:40.771 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:40.770+0000 7f0db982d640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0db40c2410 con 0x7f0db40ce8f0 2023-08-09T14:52:40.771 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:40.770+0000 7f0db982d640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0db40c26f0 con 0x7f0db4154720 2023-08-09T14:52:40.771 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:40.770+0000 7f0db2ffd640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f0db406a130 0x7f0db415a860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T14:52:40.771 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:40.770+0000 7f0db2ffd640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f0db406a130 0x7f0db415a860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:36418/0 (socket says 172.21.15.43:36418) 2023-08-09T14:52:40.772 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:40.770+0000 7f0db2ffd640 1 -- 172.21.15.43:0/3031395896 learned_addr learned my addr 172.21.15.43:0/3031395896 (peer_addr_for_me v2:172.21.15.43:0/0) 2023-08-09T14:52:40.772 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:40.770+0000 7f0db2ffd640 1 -- 172.21.15.43:0/3031395896 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f0db4154720 msgr2=0x7f0db4151f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T14:52:40.773 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:40.770+0000 7f0db2ffd640 1 --2- 172.21.15.43:0/3031395896 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f0db4154720 0x7f0db4151f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T14:52:40.773 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:40.770+0000 7f0db2ffd640 1 -- 172.21.15.43:0/3031395896 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f0db40ce8f0 msgr2=0x7f0db415ada0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T14:52:40.773 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:40.770+0000 7f0db2ffd640 1 --2- 172.21.15.43:0/3031395896 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f0db40ce8f0 0x7f0db415ada0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T14:52:40.773 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:40.770+0000 7f0db2ffd640 1 -- 172.21.15.43:0/3031395896 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0d9c0094d0 con 0x7f0db406a130 2023-08-09T14:52:40.773 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:40.771+0000 7f0db2ffd640 1 --2- 172.21.15.43:0/3031395896 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f0db406a130 0x7f0db415a860 secure :-1 s=READY pgs=2071 cs=0 l=1 rev1=1 crypto rx=0x7f0da400e870 tx=0x7f0da400ed30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-09T14:52:40.773 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:40.771+0000 7f0db0ff9640 1 -- 172.21.15.43:0/3031395896 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0da40094f0 con 0x7f0db406a130 2023-08-09T14:52:40.773 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:40.771+0000 7f0db0ff9640 1 -- 172.21.15.43:0/3031395896 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0da400c940 con 0x7f0db406a130 2023-08-09T14:52:40.774 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:40.771+0000 7f0db982d640 1 -- 172.21.15.43:0/3031395896 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0db40c29d0 con 0x7f0db406a130 2023-08-09T14:52:40.774 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:40.771+0000 7f0db0ff9640 1 -- 172.21.15.43:0/3031395896 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0da400ccf0 con 0x7f0db406a130 2023-08-09T14:52:40.774 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:40.771+0000 7f0db982d640 1 -- 172.21.15.43:0/3031395896 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0db414fb80 con 0x7f0db406a130 2023-08-09T14:52:40.774 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:40.773+0000 7f0db982d640 1 -- 172.21.15.43:0/3031395896 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0db4069df0 con 0x7f0db406a130 2023-08-09T14:52:40.774 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:40.774+0000 7f0db0ff9640 1 -- 172.21.15.43:0/3031395896 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 88) v1 ==== 88443+0+0 (secure 0 0 0) 0x7f0da4013470 con 0x7f0db406a130 2023-08-09T14:52:40.775 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:40.774+0000 7f0db0ff9640 1 --2- 172.21.15.43:0/3031395896 >> [v2:172.21.15.203:6832/2959400181,v1:172.21.15.203:6833/2959400181] conn(0x7f0d9406a170 0x7f0d9406c630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T14:52:40.775 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:40.775+0000 7f0db0ff9640 1 -- 172.21.15.43:0/3031395896 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(186..186 src has 1..186) v4 ==== 6426+0+0 (secure 0 0 0) 0x7f0da40889a0 con 0x7f0db406a130 2023-08-09T14:52:40.776 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:40.775+0000 7f0daa7fc640 1 --2- 172.21.15.43:0/3031395896 >> [v2:172.21.15.203:6832/2959400181,v1:172.21.15.203:6833/2959400181] conn(0x7f0d9406a170 0x7f0d9406c630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T14:52:40.776 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:40.776+0000 7f0daa7fc640 1 --2- 172.21.15.43:0/3031395896 >> [v2:172.21.15.203:6832/2959400181,v1:172.21.15.203:6833/2959400181] conn(0x7f0d9406a170 0x7f0d9406c630 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f0d9c036000 tx=0x7f0d9c030000 comp rx=0 tx=0).ready entity=mgr.7223 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-09T14:52:40.779 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:40.779+0000 7f0db0ff9640 1 -- 172.21.15.43:0/3031395896 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178066 (secure 0 0 0) 0x7f0da4053e10 con 0x7f0db406a130 2023-08-09T14:52:41.030 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:41.029+0000 7f0db982d640 1 -- 172.21.15.43:0/3031395896 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "mgr module enable", "module": "snap_schedule"} v 0) v1 -- 0x7f0db415a1e0 con 0x7f0db406a130 2023-08-09T14:52:41.210 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:41.209+0000 7f0db0ff9640 1 -- 172.21.15.43:0/3031395896 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mgrmap(e 89) v1 ==== 88460+0+0 (secure 0 0 0) 0x7f0da4086bf0 con 0x7f0db406a130 2023-08-09T14:52:41.213 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:41.212+0000 7f0db0ff9640 1 -- 172.21.15.43:0/3031395896 <== mon.0 v2:172.21.15.43:3300/0 8 ==== mon_command_ack([{"prefix": "mgr module enable", "module": "snap_schedule"}]=0 v89) v1 ==== 92+0+0 (secure 0 0 0) 0x7f0da4053e10 con 0x7f0db406a130 2023-08-09T14:52:41.216 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:41.215+0000 7f0db982d640 1 -- 172.21.15.43:0/3031395896 >> [v2:172.21.15.203:6832/2959400181,v1:172.21.15.203:6833/2959400181] conn(0x7f0d9406a170 msgr2=0x7f0d9406c630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T14:52:41.216 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:41.215+0000 7f0db982d640 1 --2- 172.21.15.43:0/3031395896 >> [v2:172.21.15.203:6832/2959400181,v1:172.21.15.203:6833/2959400181] conn(0x7f0d9406a170 0x7f0d9406c630 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f0d9c036000 tx=0x7f0d9c030000 comp rx=0 tx=0).stop 2023-08-09T14:52:41.216 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:41.215+0000 7f0db982d640 1 -- 172.21.15.43:0/3031395896 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f0db406a130 msgr2=0x7f0db415a860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T14:52:41.216 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:41.215+0000 7f0db982d640 1 --2- 172.21.15.43:0/3031395896 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f0db406a130 0x7f0db415a860 secure :-1 s=READY pgs=2071 cs=0 l=1 rev1=1 crypto rx=0x7f0da400e870 tx=0x7f0da400ed30 comp rx=0 tx=0).stop 2023-08-09T14:52:41.217 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:41.216+0000 7f0db982d640 1 -- 172.21.15.43:0/3031395896 shutdown_connections 2023-08-09T14:52:41.217 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:41.216+0000 7f0db982d640 1 --2- 172.21.15.43:0/3031395896 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f0db4154720 0x7f0db4151f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T14:52:41.217 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:41.216+0000 7f0db982d640 1 --2- 172.21.15.43:0/3031395896 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f0db40ce8f0 0x7f0db415ada0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T14:52:41.218 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:41.216+0000 7f0db982d640 1 --2- 172.21.15.43:0/3031395896 >> [v2:172.21.15.203:6832/2959400181,v1:172.21.15.203:6833/2959400181] conn(0x7f0d9406a170 0x7f0d9406c630 unknown :-1 s=CLOSED pgs=73 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T14:52:41.218 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:41.216+0000 7f0db982d640 1 --2- 172.21.15.43:0/3031395896 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f0db406a130 0x7f0db415a860 unknown :-1 s=CLOSED pgs=2071 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T14:52:41.218 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:41.216+0000 7f0db982d640 1 -- 172.21.15.43:0/3031395896 >> 172.21.15.43:0/3031395896 conn(0x7f0db406b970 msgr2=0x7f0db406bd50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-09T14:52:41.218 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:41.216+0000 7f0db982d640 1 -- 172.21.15.43:0/3031395896 shutdown_connections 2023-08-09T14:52:41.218 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:41.216+0000 7f0db982d640 1 -- 172.21.15.43:0/3031395896 wait complete. 2023-08-09T14:52:41.237 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mgr mgr/snap_schedule/allow_m_granularity True 2023-08-09T14:52:41.429 INFO:tasks.ceph.mgr.x.smithi203.stderr:2023-08-09T14:52:41.428+0000 7f25af26f040 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2023-08-09T14:52:41.433 INFO:tasks.ceph.mgr.y.smithi043.stderr:2023-08-09T14:52:41.432+0000 7f1ac4c20040 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2023-08-09T14:52:41.449 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:41.448+0000 7f4ea71fd640 1 Processor -- start 2023-08-09T14:52:41.449 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:41.449+0000 7f4ea71fd640 1 -- start start 2023-08-09T14:52:41.450 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:41.450+0000 7f4ea71fd640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f4ea015ffc0 0x7f4ea00d0040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T14:52:41.451 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:41.450+0000 7f4ea71fd640 1 -- --> v1:172.21.15.43:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4ea00b1730 con 0x7f4ea015fbe0 2023-08-09T14:52:41.451 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:41.450+0000 7f4ea71fd640 1 -- --> v1:172.21.15.203:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4ea00b0250 con 0x7f4ea00b1270 2023-08-09T14:52:41.451 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:41.450+0000 7f4ea71fd640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4ea00b1a10 con 0x7f4ea015ffc0 2023-08-09T14:52:41.452 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:41.451+0000 7f4ea4f99640 1 --1- >> v1:172.21.15.43:6789/0 conn(0x7f4ea015fbe0 0x7f4ea00af0a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.43:6789/0 says I am v1:172.21.15.43:46288/0 (socket says 172.21.15.43:46288) 2023-08-09T14:52:41.452 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:41.451+0000 7f4ea4f99640 1 -- 172.21.15.43:0/179569506 learned_addr learned my addr 172.21.15.43:0/179569506 (peer_addr_for_me v1:172.21.15.43:0/0) 2023-08-09T14:52:41.452 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:41.451+0000 7f4ea579a640 1 --2- 172.21.15.43:0/179569506 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f4ea015ffc0 0x7f4ea00d0040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T14:52:41.453 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:41.451+0000 7f4e9f7fe640 1 -- 172.21.15.43:0/179569506 <== mon.0 v1:172.21.15.43:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3711749084 0 0) 0x7f4ea00b1730 con 0x7f4ea015fbe0 2023-08-09T14:52:41.453 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:41.451+0000 7f4e9f7fe640 1 -- 172.21.15.43:0/179569506 --> v1:172.21.15.43:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f4e90003520 con 0x7f4ea015fbe0 2023-08-09T14:52:41.453 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:41.451+0000 7f4e9f7fe640 1 -- 172.21.15.43:0/179569506 <== mon.0 v1:172.21.15.43:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 878028264 0 0) 0x7f4e90003520 con 0x7f4ea015fbe0 2023-08-09T14:52:41.453 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:41.452+0000 7f4e9f7fe640 1 -- 172.21.15.43:0/179569506 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f4ea015ffc0 msgr2=0x7f4ea00d0040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-09T14:52:41.453 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:41.452+0000 7f4e9f7fe640 1 --2- 172.21.15.43:0/179569506 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f4ea015ffc0 0x7f4ea00d0040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T14:52:41.453 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:41.452+0000 7f4e9f7fe640 1 -- 172.21.15.43:0/179569506 >> v1:172.21.15.203:6789/0 conn(0x7f4ea00b1270 legacy=0x7f4ea00af7b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T14:52:41.453 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:41.452+0000 7f4e9f7fe640 1 -- 172.21.15.43:0/179569506 --> v1:172.21.15.43:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4ea00b0530 con 0x7f4ea015fbe0 2023-08-09T14:52:41.454 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:41.452+0000 7f4e9f7fe640 1 -- 172.21.15.43:0/179569506 <== mon.0 v1:172.21.15.43:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1285150740 0 0) 0x7f4e940017f0 con 0x7f4ea015fbe0 2023-08-09T14:52:41.454 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:41.452+0000 7f4e9f7fe640 1 -- 172.21.15.43:0/179569506 >> v1:172.21.15.43:6789/0 conn(0x7f4ea015fbe0 legacy=0x7f4ea00af0a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T14:52:41.454 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:41.452+0000 7f4e9f7fe640 1 --2- 172.21.15.43:0/179569506 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f4e900039a0 0x7f4e90003dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T14:52:41.454 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:41.452+0000 7f4e9f7fe640 1 --2- 172.21.15.43:0/179569506 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f4ea015ffc0 0x7f4e900047c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T14:52:41.454 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:41.452+0000 7f4e9f7fe640 1 --2- 172.21.15.43:0/179569506 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f4e90004d00 0x7f4e90005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T14:52:41.454 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:41.452+0000 7f4e9f7fe640 1 -- 172.21.15.43:0/179569506 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4ea00b1a10 con 0x7f4e900039a0 2023-08-09T14:52:41.454 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:41.452+0000 7f4e9f7fe640 1 -- 172.21.15.43:0/179569506 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4e9000ba30 con 0x7f4e90004d00 2023-08-09T14:52:41.454 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:41.452+0000 7f4e9f7fe640 1 -- 172.21.15.43:0/179569506 --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4e9000bbf0 con 0x7f4ea015ffc0 2023-08-09T14:52:41.456 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:41.452+0000 7f4ea4f99640 1 --2- 172.21.15.43:0/179569506 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f4e900039a0 0x7f4e90003dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T14:52:41.456 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:41.452+0000 7f4ea579a640 1 --2- 172.21.15.43:0/179569506 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f4e90004d00 0x7f4e90005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T14:52:41.456 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:41.452+0000 7f4ea4f99640 1 -- 172.21.15.43:0/179569506 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f4ea015ffc0 msgr2=0x7f4e900047c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-09T14:52:41.456 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:41.452+0000 7f4e9ffff640 1 --2- 172.21.15.43:0/179569506 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f4ea015ffc0 0x7f4e900047c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T14:52:41.456 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:41.453+0000 7f4ea4f99640 1 --2- 172.21.15.43:0/179569506 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f4ea015ffc0 0x7f4e900047c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T14:52:41.457 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:41.453+0000 7f4ea4f99640 1 -- 172.21.15.43:0/179569506 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f4e90004d00 msgr2=0x7f4e90005180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-09T14:52:41.457 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:41.453+0000 7f4ea4f99640 1 --2- 172.21.15.43:0/179569506 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f4e90004d00 0x7f4e90005180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T14:52:41.457 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:41.453+0000 7f4ea4f99640 1 -- 172.21.15.43:0/179569506 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4e90003520 con 0x7f4e900039a0 2023-08-09T14:52:41.457 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:41.453+0000 7f4ea579a640 1 --2- 172.21.15.43:0/179569506 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f4e90004d00 0x7f4e90005180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-08-09T14:52:41.457 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:41.453+0000 7f4e9ffff640 1 --2- 172.21.15.43:0/179569506 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f4ea015ffc0 0x7f4e900047c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-08-09T14:52:41.457 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:41.454+0000 7f4ea4f99640 1 --2- 172.21.15.43:0/179569506 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f4e900039a0 0x7f4e90003dc0 secure :-1 s=READY pgs=2077 cs=0 l=1 rev1=1 crypto rx=0x7f4e9400c160 tx=0x7f4e900057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b92d82be511211b4 server_cookie=0 in_seq=0 out_seq=0 2023-08-09T14:52:41.457 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:41.454+0000 7f4e9f7fe640 1 -- 172.21.15.43:0/179569506 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4e94007270 con 0x7f4e900039a0 2023-08-09T14:52:41.458 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:41.454+0000 7f4e9f7fe640 1 -- 172.21.15.43:0/179569506 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4e940073f0 con 0x7f4e900039a0 2023-08-09T14:52:41.458 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:41.454+0000 7f4e9f7fe640 1 -- 172.21.15.43:0/179569506 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4e9402c870 con 0x7f4e900039a0 2023-08-09T14:52:41.458 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:41.455+0000 7f4ea71fd640 1 -- 172.21.15.43:0/179569506 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f4e900039a0 msgr2=0x7f4e90003dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T14:52:41.458 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:41.455+0000 7f4ea71fd640 1 --2- 172.21.15.43:0/179569506 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f4e900039a0 0x7f4e90003dc0 secure :-1 s=READY pgs=2077 cs=0 l=1 rev1=1 crypto rx=0x7f4e9400c160 tx=0x7f4e900057f0 comp rx=0 tx=0).stop 2023-08-09T14:52:41.458 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:41.455+0000 7f4ea579a640 1 -- 172.21.15.43:0/179569506 reap_dead start 2023-08-09T14:52:41.458 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:41.455+0000 7f4ea71fd640 1 -- 172.21.15.43:0/179569506 shutdown_connections 2023-08-09T14:52:41.458 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:41.455+0000 7f4ea71fd640 1 -- 172.21.15.43:0/179569506 >> 172.21.15.43:0/179569506 conn(0x7f4ea009b7b0 msgr2=0x7f4ea00d2470 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-09T14:52:41.458 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:41.455+0000 7f4ea71fd640 1 -- 172.21.15.43:0/179569506 shutdown_connections 2023-08-09T14:52:41.459 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:41.455+0000 7f4ea71fd640 1 -- 172.21.15.43:0/179569506 wait complete. 2023-08-09T14:52:41.460 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:41.457+0000 7f4ea71fd640 1 Processor -- start 2023-08-09T14:52:41.460 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:41.457+0000 7f4ea71fd640 1 -- start start 2023-08-09T14:52:41.461 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:41.458+0000 7f4ea71fd640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f4e900039a0 0x7f4ea015df10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T14:52:41.461 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:41.458+0000 7f4ea71fd640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f4ea015e6f0 0x7f4ea00cfc20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T14:52:41.461 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:41.458+0000 7f4ea71fd640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f4ea00cac00 0x7f4ea00cb080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T14:52:41.461 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:41.458+0000 7f4ea71fd640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4ea00b0910 con 0x7f4ea015e6f0 2023-08-09T14:52:41.461 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:41.458+0000 7f4ea71fd640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4ea015e450 con 0x7f4e900039a0 2023-08-09T14:52:41.461 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:41.458+0000 7f4ea71fd640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4ea00cdf60 con 0x7f4ea00cac00 2023-08-09T14:52:41.461 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:41.459+0000 7f4e9ffff640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f4ea015e6f0 0x7f4ea00cfc20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T14:52:41.461 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:41.459+0000 7f4e9ffff640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f4ea015e6f0 0x7f4ea00cfc20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:36468/0 (socket says 172.21.15.43:36468) 2023-08-09T14:52:41.462 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:41.459+0000 7f4e9ffff640 1 -- 172.21.15.43:0/1892963321 learned_addr learned my addr 172.21.15.43:0/1892963321 (peer_addr_for_me v2:172.21.15.43:0/0) 2023-08-09T14:52:41.462 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:41.459+0000 7f4ea579a640 1 --2- 172.21.15.43:0/1892963321 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f4ea00cac00 0x7f4ea00cb080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T14:52:41.462 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:41.459+0000 7f4ea4f99640 1 --2- 172.21.15.43:0/1892963321 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f4e900039a0 0x7f4ea015df10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T14:52:41.462 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:41.459+0000 7f4e9ffff640 1 -- 172.21.15.43:0/1892963321 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f4ea00cac00 msgr2=0x7f4ea00cb080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T14:52:41.462 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:41.459+0000 7f4e9ffff640 1 --2- 172.21.15.43:0/1892963321 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f4ea00cac00 0x7f4ea00cb080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T14:52:41.462 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:41.459+0000 7f4e9ffff640 1 -- 172.21.15.43:0/1892963321 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f4e900039a0 msgr2=0x7f4ea015df10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T14:52:41.462 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:41.459+0000 7f4e9ffff640 1 --2- 172.21.15.43:0/1892963321 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f4e900039a0 0x7f4ea015df10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T14:52:41.463 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:41.459+0000 7f4e9ffff640 1 -- 172.21.15.43:0/1892963321 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4ea00ce240 con 0x7f4ea015e6f0 2023-08-09T14:52:41.463 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:41.460+0000 7f4e9ffff640 1 --2- 172.21.15.43:0/1892963321 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f4ea015e6f0 0x7f4ea00cfc20 secure :-1 s=READY pgs=2078 cs=0 l=1 rev1=1 crypto rx=0x7f4e8c00d770 tx=0x7f4e8c00dc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-09T14:52:41.463 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:41.460+0000 7f4e9d7fa640 1 -- 172.21.15.43:0/1892963321 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4e8c005270 con 0x7f4ea015e6f0 2023-08-09T14:52:41.463 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:41.460+0000 7f4e9d7fa640 1 -- 172.21.15.43:0/1892963321 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4e8c0053d0 con 0x7f4ea015e6f0 2023-08-09T14:52:41.463 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:41.460+0000 7f4ea71fd640 1 -- 172.21.15.43:0/1892963321 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4ea00ce520 con 0x7f4ea015e6f0 2023-08-09T14:52:41.463 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:41.460+0000 7f4e9d7fa640 1 -- 172.21.15.43:0/1892963321 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4e8c00b8f0 con 0x7f4ea015e6f0 2023-08-09T14:52:41.464 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:41.460+0000 7f4ea71fd640 1 -- 172.21.15.43:0/1892963321 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4ea00ceae0 con 0x7f4ea015e6f0 2023-08-09T14:52:41.464 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:41.461+0000 7f4ea71fd640 1 -- 172.21.15.43:0/1892963321 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4ea00cec70 con 0x7f4ea015e6f0 2023-08-09T14:52:41.464 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:41.462+0000 7f4e9d7fa640 1 -- 172.21.15.43:0/1892963321 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 89) v1 ==== 88460+0+0 (secure 0 0 0) 0x7f4e8c00ba50 con 0x7f4ea015e6f0 2023-08-09T14:52:41.464 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:41.463+0000 7f4e9d7fa640 1 --2- 172.21.15.43:0/1892963321 >> [v2:172.21.15.203:6832/2959400181,v1:172.21.15.203:6833/2959400181] conn(0x7f4e8406a1f0 0x7f4e8406c6b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T14:52:41.465 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:41.464+0000 7f4e9d7fa640 1 -- 172.21.15.43:0/1892963321 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(186..186 src has 1..186) v4 ==== 6426+0+0 (secure 0 0 0) 0x7f4e8c088940 con 0x7f4ea015e6f0 2023-08-09T14:52:41.465 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:41.464+0000 7f4ea4f99640 1 -- 172.21.15.43:0/1892963321 >> [v2:172.21.15.203:6832/2959400181,v1:172.21.15.203:6833/2959400181] conn(0x7f4e8406a1f0 msgr2=0x7f4e8406c6b0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.203:6832/2959400181 2023-08-09T14:52:41.465 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:41.464+0000 7f4ea4f99640 1 --2- 172.21.15.43:0/1892963321 >> [v2:172.21.15.203:6832/2959400181,v1:172.21.15.203:6833/2959400181] conn(0x7f4e8406a1f0 0x7f4e8406c6b0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-08-09T14:52:41.468 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:41.467+0000 7f4e9d7fa640 1 -- 172.21.15.43:0/1892963321 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178066 (secure 0 0 0) 0x7f4e8c053c90 con 0x7f4ea015e6f0 2023-08-09T14:52:41.663 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:41.662+0000 7f4ea71fd640 1 -- 172.21.15.43:0/1892963321 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command([{prefix=config set, name=mgr/snap_schedule/allow_m_granularity}] v 0) v1 -- 0x7f4ea00739f0 con 0x7f4ea015e6f0 2023-08-09T14:52:41.665 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:41.665+0000 7f4ea4f99640 1 -- 172.21.15.43:0/1892963321 >> [v2:172.21.15.203:6832/2959400181,v1:172.21.15.203:6833/2959400181] conn(0x7f4e8406a1f0 msgr2=0x7f4e8406c6b0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.203:6832/2959400181 2023-08-09T14:52:41.665 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:41.665+0000 7f4ea4f99640 1 --2- 172.21.15.43:0/1892963321 >> [v2:172.21.15.203:6832/2959400181,v1:172.21.15.203:6833/2959400181] conn(0x7f4e8406a1f0 0x7f4e8406c6b0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2023-08-09T14:52:41.668 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:41.668+0000 7f4e9d7fa640 1 -- 172.21.15.43:0/1892963321 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mgr/snap_schedule/allow_m_granularity}]=0 v59)=0 v59) v1 ==== 138+0+0 (secure 0 0 0) 0x7f4e8c019310 con 0x7f4ea015e6f0 2023-08-09T14:52:41.671 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:41.671+0000 7f4ea71fd640 1 -- 172.21.15.43:0/1892963321 >> [v2:172.21.15.203:6832/2959400181,v1:172.21.15.203:6833/2959400181] conn(0x7f4e8406a1f0 msgr2=0x7f4e8406c6b0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-08-09T14:52:41.671 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:41.671+0000 7f4ea71fd640 1 --2- 172.21.15.43:0/1892963321 >> [v2:172.21.15.203:6832/2959400181,v1:172.21.15.203:6833/2959400181] conn(0x7f4e8406a1f0 0x7f4e8406c6b0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T14:52:41.672 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:41.671+0000 7f4ea71fd640 1 -- 172.21.15.43:0/1892963321 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f4ea015e6f0 msgr2=0x7f4ea00cfc20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T14:52:41.672 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:41.671+0000 7f4ea71fd640 1 --2- 172.21.15.43:0/1892963321 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f4ea015e6f0 0x7f4ea00cfc20 secure :-1 s=READY pgs=2078 cs=0 l=1 rev1=1 crypto rx=0x7f4e8c00d770 tx=0x7f4e8c00dc30 comp rx=0 tx=0).stop 2023-08-09T14:52:41.672 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:41.671+0000 7f4ea71fd640 1 -- 172.21.15.43:0/1892963321 shutdown_connections 2023-08-09T14:52:41.673 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:41.671+0000 7f4ea71fd640 1 --2- 172.21.15.43:0/1892963321 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f4ea00cac00 0x7f4ea00cb080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T14:52:41.673 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:41.671+0000 7f4ea71fd640 1 --2- 172.21.15.43:0/1892963321 >> [v2:172.21.15.203:6832/2959400181,v1:172.21.15.203:6833/2959400181] conn(0x7f4e8406a1f0 0x7f4e8406c6b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T14:52:41.673 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:41.671+0000 7f4ea71fd640 1 --2- 172.21.15.43:0/1892963321 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f4ea015e6f0 0x7f4ea00cfc20 unknown :-1 s=CLOSED pgs=2078 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T14:52:41.673 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:41.671+0000 7f4ea71fd640 1 --2- 172.21.15.43:0/1892963321 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f4e900039a0 0x7f4ea015df10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T14:52:41.673 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:41.671+0000 7f4ea71fd640 1 -- 172.21.15.43:0/1892963321 >> 172.21.15.43:0/1892963321 conn(0x7f4ea009b7b0 msgr2=0x7f4ea006be80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-09T14:52:41.673 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:41.671+0000 7f4ea71fd640 1 -- 172.21.15.43:0/1892963321 shutdown_connections 2023-08-09T14:52:41.673 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:41.671+0000 7f4ea71fd640 1 -- 172.21.15.43:0/1892963321 wait complete. 2023-08-09T14:52:41.690 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mgr mgr/snap_schedule/dump_on_update True 2023-08-09T14:52:41.909 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:41.909+0000 7f25769b7640 1 Processor -- start 2023-08-09T14:52:41.910 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:41.909+0000 7f25769b7640 1 -- start start 2023-08-09T14:52:41.911 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:41.911+0000 7f25769b7640 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7f25700b1260 0x7f25700cee00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T14:52:41.911 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:41.911+0000 7f25769b7640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f25700cf340 0x7f25700d0350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T14:52:41.911 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:41.911+0000 7f25769b7640 1 -- --> v1:172.21.15.203:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f25700cf790 con 0x7f257015c620 2023-08-09T14:52:41.912 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:41.911+0000 7f25769b7640 1 -- --> v2:172.21.15.203:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f25700b1b70 con 0x7f25700b1260 2023-08-09T14:52:41.912 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:41.911+0000 7f25769b7640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f25700cd700 con 0x7f25700cf340 2023-08-09T14:52:41.912 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:41.911+0000 7f2574f54640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f25700cf340 0x7f25700d0350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T14:52:41.913 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:41.911+0000 7f256ffff640 1 --1- >> v1:172.21.15.203:6789/0 conn(0x7f257015c620 0x7f25700cceb0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.203:6789/0 says I am v1:172.21.15.43:34826/0 (socket says 172.21.15.43:34826) 2023-08-09T14:52:41.913 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:41.911+0000 7f256ffff640 1 -- 172.21.15.43:0/3726918102 learned_addr learned my addr 172.21.15.43:0/3726918102 (peer_addr_for_me v1:172.21.15.43:0/0) 2023-08-09T14:52:41.913 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:41.911+0000 7f256f7fe640 1 --2- 172.21.15.43:0/3726918102 >> v2:172.21.15.203:3300/0 conn(0x7f25700b1260 0x7f25700cee00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T14:52:41.913 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:41.912+0000 7f256effd640 1 -- 172.21.15.43:0/3726918102 <== mon.1 v1:172.21.15.203:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 968020253 0 0) 0x7f2560001160 con 0x7f257015c620 2023-08-09T14:52:41.913 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:41.912+0000 7f256effd640 1 -- 172.21.15.43:0/3726918102 --> v1:172.21.15.203:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f2558003520 con 0x7f257015c620 2023-08-09T14:52:41.913 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:41.912+0000 7f2574f54640 1 -- 172.21.15.43:0/3726918102 >> v2:172.21.15.203:3300/0 conn(0x7f25700b1260 msgr2=0x7f25700cee00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-09T14:52:41.914 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:41.912+0000 7f2574f54640 1 --2- 172.21.15.43:0/3726918102 >> v2:172.21.15.203:3300/0 conn(0x7f25700b1260 0x7f25700cee00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T14:52:41.914 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:41.912+0000 7f2574f54640 1 -- 172.21.15.43:0/3726918102 >> v1:172.21.15.203:6789/0 conn(0x7f257015c620 legacy=0x7f25700cceb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T14:52:41.914 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:41.912+0000 7f2574f54640 1 -- 172.21.15.43:0/3726918102 --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f25700cd9e0 con 0x7f25700cf340 2023-08-09T14:52:41.914 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:41.913+0000 7f2574f54640 1 --2- 172.21.15.43:0/3726918102 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f25700cf340 0x7f25700d0350 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7f2568009d90 tx=0x7f256802bd80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=1b36516ca95ff66 server_cookie=0 in_seq=0 out_seq=0 2023-08-09T14:52:41.914 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:41.914+0000 7f256effd640 1 -- 172.21.15.43:0/3726918102 <== mon.2 v2:172.21.15.203:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f256802cb30 con 0x7f25700cf340 2023-08-09T14:52:41.915 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:41.914+0000 7f256effd640 1 -- 172.21.15.43:0/3726918102 <== mon.2 v2:172.21.15.203:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f256802cc90 con 0x7f25700cf340 2023-08-09T14:52:41.915 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:41.914+0000 7f256effd640 1 -- 172.21.15.43:0/3726918102 <== mon.2 v2:172.21.15.203:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2568034440 con 0x7f25700cf340 2023-08-09T14:52:41.916 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:41.914+0000 7f25769b7640 1 -- 172.21.15.43:0/3726918102 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f25700cf340 msgr2=0x7f25700d0350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T14:52:41.916 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:41.914+0000 7f25769b7640 1 --2- 172.21.15.43:0/3726918102 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f25700cf340 0x7f25700d0350 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7f2568009d90 tx=0x7f256802bd80 comp rx=0 tx=0).stop 2023-08-09T14:52:41.916 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:41.915+0000 7f25769b7640 1 -- 172.21.15.43:0/3726918102 shutdown_connections 2023-08-09T14:52:41.916 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:41.915+0000 7f25769b7640 1 --2- 172.21.15.43:0/3726918102 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f25700cf340 0x7f25700d0350 unknown :-1 s=CLOSED pgs=279 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T14:52:41.916 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:41.915+0000 7f25769b7640 1 --2- 172.21.15.43:0/3726918102 >> v2:172.21.15.203:3300/0 conn(0x7f25700b1260 0x7f25700cee00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T14:52:41.916 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:41.915+0000 7f25769b7640 1 -- 172.21.15.43:0/3726918102 >> 172.21.15.43:0/3726918102 conn(0x7f257009b750 msgr2=0x7f25700d2410 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-09T14:52:41.917 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:41.915+0000 7f25769b7640 1 -- 172.21.15.43:0/3726918102 shutdown_connections 2023-08-09T14:52:41.917 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:41.915+0000 7f25769b7640 1 -- 172.21.15.43:0/3726918102 wait complete. 2023-08-09T14:52:41.917 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:41.916+0000 7f25769b7640 1 Processor -- start 2023-08-09T14:52:41.917 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:41.917+0000 7f25769b7640 1 -- start start 2023-08-09T14:52:41.918 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:41.918+0000 7f25769b7640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f25700b1260 0x7f257006ae10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T14:52:41.918 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:41.918+0000 7f25769b7640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f25700cf340 0x7f257006b350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T14:52:41.918 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:41.918+0000 7f25769b7640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f257015c620 0x7f257009d7b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T14:52:41.919 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:41.918+0000 7f25769b7640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f25701f5020 con 0x7f25700b1260 2023-08-09T14:52:41.919 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:41.918+0000 7f25769b7640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f25701f5fc0 con 0x7f25700cf340 2023-08-09T14:52:41.919 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:41.918+0000 7f25769b7640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f25701f5b30 con 0x7f257015c620 2023-08-09T14:52:41.919 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:41.918+0000 7f256ffff640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f25700b1260 0x7f257006ae10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T14:52:41.919 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:41.918+0000 7f256ffff640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f25700b1260 0x7f257006ae10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:36484/0 (socket says 172.21.15.43:36484) 2023-08-09T14:52:41.919 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:41.918+0000 7f256ffff640 1 -- 172.21.15.43:0/3014005100 learned_addr learned my addr 172.21.15.43:0/3014005100 (peer_addr_for_me v2:172.21.15.43:0/0) 2023-08-09T14:52:41.920 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:41.918+0000 7f256f7fe640 1 --2- 172.21.15.43:0/3014005100 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f25700cf340 0x7f257006b350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T14:52:41.921 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:41.918+0000 7f2574f54640 1 --2- 172.21.15.43:0/3014005100 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f257015c620 0x7f257009d7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T14:52:41.921 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:41.919+0000 7f256ffff640 1 -- 172.21.15.43:0/3014005100 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f257015c620 msgr2=0x7f257009d7b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T14:52:41.921 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:41.919+0000 7f256ffff640 1 --2- 172.21.15.43:0/3014005100 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f257015c620 0x7f257009d7b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T14:52:41.921 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:41.919+0000 7f256ffff640 1 -- 172.21.15.43:0/3014005100 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f25700cf340 msgr2=0x7f257006b350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T14:52:41.921 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:41.919+0000 7f256ffff640 1 --2- 172.21.15.43:0/3014005100 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f25700cf340 0x7f257006b350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T14:52:41.921 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:41.919+0000 7f256ffff640 1 -- 172.21.15.43:0/3014005100 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2568009a70 con 0x7f25700b1260 2023-08-09T14:52:41.921 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:41.919+0000 7f256ffff640 1 --2- 172.21.15.43:0/3014005100 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f25700b1260 0x7f257006ae10 secure :-1 s=READY pgs=2079 cs=0 l=1 rev1=1 crypto rx=0x7f256000c400 tx=0x7f256000c8c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-09T14:52:41.922 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:41.919+0000 7f256cff9640 1 -- 172.21.15.43:0/3014005100 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2560005d30 con 0x7f25700b1260 2023-08-09T14:52:41.922 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:41.919+0000 7f256cff9640 1 -- 172.21.15.43:0/3014005100 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2560005e90 con 0x7f25700b1260 2023-08-09T14:52:41.922 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:41.919+0000 7f256cff9640 1 -- 172.21.15.43:0/3014005100 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f25600072f0 con 0x7f25700b1260 2023-08-09T14:52:41.922 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:41.919+0000 7f25769b7640 1 -- 172.21.15.43:0/3014005100 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f25700cc120 con 0x7f25700b1260 2023-08-09T14:52:41.922 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:41.919+0000 7f25769b7640 1 -- 172.21.15.43:0/3014005100 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f25701f54e0 con 0x7f25700b1260 2023-08-09T14:52:41.923 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:41.921+0000 7f25769b7640 1 -- 172.21.15.43:0/3014005100 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f25700739f0 con 0x7f25700b1260 2023-08-09T14:52:41.923 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:41.921+0000 7f256cff9640 1 -- 172.21.15.43:0/3014005100 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 89) v1 ==== 88460+0+0 (secure 0 0 0) 0x7f2560007450 con 0x7f25700b1260 2023-08-09T14:52:41.923 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:41.922+0000 7f256cff9640 1 --2- 172.21.15.43:0/3014005100 >> [v2:172.21.15.203:6832/2959400181,v1:172.21.15.203:6833/2959400181] conn(0x7f255c06a1f0 0x7f255c06c6b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T14:52:41.924 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:41.922+0000 7f256cff9640 1 -- 172.21.15.43:0/3014005100 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(186..186 src has 1..186) v4 ==== 6426+0+0 (secure 0 0 0) 0x7f2560087da0 con 0x7f25700b1260 2023-08-09T14:52:41.924 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:41.922+0000 7f256f7fe640 1 -- 172.21.15.43:0/3014005100 >> [v2:172.21.15.203:6832/2959400181,v1:172.21.15.203:6833/2959400181] conn(0x7f255c06a1f0 msgr2=0x7f255c06c6b0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.203:6832/2959400181 2023-08-09T14:52:41.924 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:41.923+0000 7f256f7fe640 1 --2- 172.21.15.43:0/3014005100 >> [v2:172.21.15.203:6832/2959400181,v1:172.21.15.203:6833/2959400181] conn(0x7f255c06a1f0 0x7f255c06c6b0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-08-09T14:52:41.927 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:41.927+0000 7f256cff9640 1 -- 172.21.15.43:0/3014005100 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178066 (secure 0 0 0) 0x7f2560056c00 con 0x7f25700b1260 2023-08-09T14:52:42.117 INFO:tasks.ceph.mgr.x.smithi203.stderr:2023-08-09T14:52:42.117+0000 7f25af26f040 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2023-08-09T14:52:42.119 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:42.119+0000 7f25769b7640 1 -- 172.21.15.43:0/3014005100 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command([{prefix=config set, name=mgr/snap_schedule/dump_on_update}] v 0) v1 -- 0x7f25700cf790 con 0x7f25700b1260 2023-08-09T14:52:42.124 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:42.124+0000 7f256f7fe640 1 -- 172.21.15.43:0/3014005100 >> [v2:172.21.15.203:6832/2959400181,v1:172.21.15.203:6833/2959400181] conn(0x7f255c06a1f0 msgr2=0x7f255c06c6b0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.203:6832/2959400181 2023-08-09T14:52:42.125 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:42.124+0000 7f256cff9640 1 -- 172.21.15.43:0/3014005100 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mgr/snap_schedule/dump_on_update}]=0 v60)=0 v60) v1 ==== 133+0+0 (secure 0 0 0) 0x7f25700cf790 con 0x7f25700b1260 2023-08-09T14:52:42.125 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:42.124+0000 7f256f7fe640 1 --2- 172.21.15.43:0/3014005100 >> [v2:172.21.15.203:6832/2959400181,v1:172.21.15.203:6833/2959400181] conn(0x7f255c06a1f0 0x7f255c06c6b0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2023-08-09T14:52:42.128 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:42.127+0000 7f25769b7640 1 -- 172.21.15.43:0/3014005100 >> [v2:172.21.15.203:6832/2959400181,v1:172.21.15.203:6833/2959400181] conn(0x7f255c06a1f0 msgr2=0x7f255c06c6b0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-08-09T14:52:42.128 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:42.127+0000 7f25769b7640 1 --2- 172.21.15.43:0/3014005100 >> [v2:172.21.15.203:6832/2959400181,v1:172.21.15.203:6833/2959400181] conn(0x7f255c06a1f0 0x7f255c06c6b0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T14:52:42.128 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:42.127+0000 7f25769b7640 1 -- 172.21.15.43:0/3014005100 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f25700b1260 msgr2=0x7f257006ae10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T14:52:42.128 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:42.127+0000 7f25769b7640 1 --2- 172.21.15.43:0/3014005100 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f25700b1260 0x7f257006ae10 secure :-1 s=READY pgs=2079 cs=0 l=1 rev1=1 crypto rx=0x7f256000c400 tx=0x7f256000c8c0 comp rx=0 tx=0).stop 2023-08-09T14:52:42.129 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:42.127+0000 7f25769b7640 1 -- 172.21.15.43:0/3014005100 shutdown_connections 2023-08-09T14:52:42.129 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:42.128+0000 7f25769b7640 1 --2- 172.21.15.43:0/3014005100 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f257015c620 0x7f257009d7b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T14:52:42.129 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:42.128+0000 7f25769b7640 1 --2- 172.21.15.43:0/3014005100 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f25700cf340 0x7f257006b350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T14:52:42.129 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:42.128+0000 7f25769b7640 1 --2- 172.21.15.43:0/3014005100 >> [v2:172.21.15.203:6832/2959400181,v1:172.21.15.203:6833/2959400181] conn(0x7f255c06a1f0 0x7f255c06c6b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T14:52:42.129 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:42.128+0000 7f25769b7640 1 --2- 172.21.15.43:0/3014005100 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f25700b1260 0x7f257006ae10 unknown :-1 s=CLOSED pgs=2079 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T14:52:42.129 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:42.128+0000 7f25769b7640 1 -- 172.21.15.43:0/3014005100 >> 172.21.15.43:0/3014005100 conn(0x7f257009b750 msgr2=0x7f257006ba40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-09T14:52:42.129 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:42.128+0000 7f25769b7640 1 -- 172.21.15.43:0/3014005100 shutdown_connections 2023-08-09T14:52:42.130 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:42.128+0000 7f25769b7640 1 -- 172.21.15.43:0/3014005100 wait complete. 2023-08-09T14:52:42.155 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-09T14:52:42.155 DEBUG:teuthology.orchestra.run.smithi043:> (cd /home/ubuntu/cephtest/mnt.0 && exec mkdir -p snap_test_dir1) 2023-08-09T14:52:42.188 INFO:tasks.ceph.mgr.y.smithi043.stderr:2023-08-09T14:52:42.188+0000 7f1ac4c20040 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2023-08-09T14:52:42.223 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-09T14:52:42.224 DEBUG:teuthology.orchestra.run.smithi043:> (cd /home/ubuntu/cephtest/mnt.0 && exec mkdir -p snap_test_dir1/test_restart) 2023-08-09T14:52:42.238 INFO:tasks.ceph.mgr.x.smithi203.stderr:2023-08-09T14:52:42.237+0000 7f25af26f040 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2023-08-09T14:52:42.294 INFO:tasks.ceph.mgr.x.smithi203.stderr:2023-08-09T14:52:42.294+0000 7f25af26f040 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2023-08-09T14:52:42.316 INFO:tasks.ceph.mgr.y.smithi043.stderr:2023-08-09T14:52:42.315+0000 7f1ac4c20040 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2023-08-09T14:52:42.327 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs snap-schedule add --fs cephfs /snap_test_dir1/test_restart 1M 2023-08-09T14:52:42.344 INFO:tasks.ceph.mgr.x.smithi203.stderr:2023-08-09T14:52:42.343+0000 7f25af26f040 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2023-08-09T14:52:42.377 INFO:tasks.ceph.mgr.y.smithi043.stderr:2023-08-09T14:52:42.376+0000 7f1ac4c20040 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2023-08-09T14:52:42.430 INFO:tasks.ceph.mgr.y.smithi043.stderr:2023-08-09T14:52:42.429+0000 7f1ac4c20040 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2023-08-09T14:52:42.531 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:42.530+0000 7f951ffff640 1 Processor -- start 2023-08-09T14:52:42.531 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:42.531+0000 7f951ffff640 1 -- start start 2023-08-09T14:52:42.535 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:42.532+0000 7f951ffff640 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7f95200cd3c0 0x7f95200cec70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T14:52:42.536 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:42.532+0000 7f951ffff640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f95200cf1b0 0x7f95200b5880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T14:52:42.536 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:42.532+0000 7f951ffff640 1 -- --> v1:172.21.15.43:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f95200cf5e0 con 0x7f952015c400 2023-08-09T14:52:42.536 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:42.532+0000 7f951ffff640 1 -- --> v2:172.21.15.203:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f952015c7e0 con 0x7f95200cd3c0 2023-08-09T14:52:42.536 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:42.532+0000 7f951ffff640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f95200d13e0 con 0x7f95200cf1b0 2023-08-09T14:52:42.536 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:42.532+0000 7f951effd640 1 --1- >> v1:172.21.15.43:6789/0 conn(0x7f952015c400 0x7f95200d0cd0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.43:6789/0 says I am v1:172.21.15.43:46298/0 (socket says 172.21.15.43:46298) 2023-08-09T14:52:42.536 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:42.532+0000 7f951effd640 1 -- 172.21.15.43:0/15550343 learned_addr learned my addr 172.21.15.43:0/15550343 (peer_addr_for_me v1:172.21.15.43:0/0) 2023-08-09T14:52:42.537 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:42.533+0000 7f951e7fc640 1 --2- 172.21.15.43:0/15550343 >> v2:172.21.15.203:3300/0 conn(0x7f95200cd3c0 0x7f95200cec70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T14:52:42.537 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:42.533+0000 7f951dffb640 1 -- 172.21.15.43:0/15550343 <== mon.0 v1:172.21.15.43:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3128636301 0 0) 0x7f95200cf5e0 con 0x7f952015c400 2023-08-09T14:52:42.537 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:42.533+0000 7f951dffb640 1 -- 172.21.15.43:0/15550343 --> v1:172.21.15.43:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f9508003520 con 0x7f952015c400 2023-08-09T14:52:42.537 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:42.533+0000 7f951f7fe640 1 --2- 172.21.15.43:0/15550343 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f95200cf1b0 0x7f95200b5880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T14:52:42.537 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:42.533+0000 7f951dffb640 1 -- 172.21.15.43:0/15550343 <== mon.0 v1:172.21.15.43:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1184895639 0 0) 0x7f9508003520 con 0x7f952015c400 2023-08-09T14:52:42.537 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:42.533+0000 7f951dffb640 1 -- 172.21.15.43:0/15550343 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f95200cf1b0 msgr2=0x7f95200b5880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-09T14:52:42.537 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:42.533+0000 7f951dffb640 1 --2- 172.21.15.43:0/15550343 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f95200cf1b0 0x7f95200b5880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T14:52:42.538 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:42.533+0000 7f951dffb640 1 -- 172.21.15.43:0/15550343 >> v2:172.21.15.203:3300/0 conn(0x7f95200cd3c0 msgr2=0x7f95200cec70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-09T14:52:42.538 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:42.533+0000 7f951dffb640 1 --2- 172.21.15.43:0/15550343 >> v2:172.21.15.203:3300/0 conn(0x7f95200cd3c0 0x7f95200cec70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T14:52:42.538 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:42.533+0000 7f951dffb640 1 -- 172.21.15.43:0/15550343 --> v1:172.21.15.43:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f95200d1690 con 0x7f952015c400 2023-08-09T14:52:42.538 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:42.533+0000 7f951dffb640 1 -- 172.21.15.43:0/15550343 <== mon.0 v1:172.21.15.43:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1285150740 0 0) 0x7f9510001690 con 0x7f952015c400 2023-08-09T14:52:42.538 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:42.533+0000 7f951dffb640 1 -- 172.21.15.43:0/15550343 >> v1:172.21.15.43:6789/0 conn(0x7f952015c400 legacy=0x7f95200d0cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T14:52:42.538 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:42.534+0000 7f951dffb640 1 --2- 172.21.15.43:0/15550343 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f9508003970 0x7f9508003d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T14:52:42.538 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:42.534+0000 7f951dffb640 1 --2- 172.21.15.43:0/15550343 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f95080047c0 0x7f9508004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T14:52:42.538 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:42.534+0000 7f951dffb640 1 --2- 172.21.15.43:0/15550343 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f95200cf1b0 0x7f9508005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T14:52:42.539 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:42.534+0000 7f951dffb640 1 -- 172.21.15.43:0/15550343 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f952015c7e0 con 0x7f9508003970 2023-08-09T14:52:42.539 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:42.534+0000 7f951dffb640 1 -- 172.21.15.43:0/15550343 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f95200d13e0 con 0x7f95080047c0 2023-08-09T14:52:42.539 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:42.534+0000 7f951dffb640 1 -- 172.21.15.43:0/15550343 --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f950800ba60 con 0x7f95200cf1b0 2023-08-09T14:52:42.539 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:42.534+0000 7f951effd640 1 --2- 172.21.15.43:0/15550343 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f9508003970 0x7f9508003d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T14:52:42.539 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:42.534+0000 7f951e7fc640 1 --2- 172.21.15.43:0/15550343 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f95080047c0 0x7f9508004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T14:52:42.539 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:42.534+0000 7f951f7fe640 1 --2- 172.21.15.43:0/15550343 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f95200cf1b0 0x7f9508005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T14:52:42.539 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:42.534+0000 7f951effd640 1 -- 172.21.15.43:0/15550343 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f95200cf1b0 msgr2=0x7f9508005180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-09T14:52:42.539 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:42.534+0000 7f951effd640 1 --2- 172.21.15.43:0/15550343 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f95200cf1b0 0x7f9508005180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T14:52:42.540 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:42.534+0000 7f951effd640 1 -- 172.21.15.43:0/15550343 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f95080047c0 msgr2=0x7f9508004c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-09T14:52:42.540 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:42.534+0000 7f951effd640 1 --2- 172.21.15.43:0/15550343 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f95080047c0 0x7f9508004c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T14:52:42.540 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:42.534+0000 7f951effd640 1 -- 172.21.15.43:0/15550343 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9508003520 con 0x7f9508003970 2023-08-09T14:52:42.544 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:42.536+0000 7f951effd640 1 --2- 172.21.15.43:0/15550343 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f9508003970 0x7f9508003d90 secure :-1 s=READY pgs=2081 cs=0 l=1 rev1=1 crypto rx=0x7f95200c5d80 tx=0x7f95080057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=330e8ce1344cd55b server_cookie=0 in_seq=0 out_seq=0 2023-08-09T14:52:42.544 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:42.536+0000 7f951dffb640 1 -- 172.21.15.43:0/15550343 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9510005ad0 con 0x7f9508003970 2023-08-09T14:52:42.545 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:42.536+0000 7f951dffb640 1 -- 172.21.15.43:0/15550343 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9510005c30 con 0x7f9508003970 2023-08-09T14:52:42.545 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:42.536+0000 7f951dffb640 1 -- 172.21.15.43:0/15550343 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f951002c8a0 con 0x7f9508003970 2023-08-09T14:52:42.545 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:42.537+0000 7f951ffff640 1 -- 172.21.15.43:0/15550343 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f9508003970 msgr2=0x7f9508003d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T14:52:42.546 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:42.537+0000 7f951ffff640 1 --2- 172.21.15.43:0/15550343 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f9508003970 0x7f9508003d90 secure :-1 s=READY pgs=2081 cs=0 l=1 rev1=1 crypto rx=0x7f95200c5d80 tx=0x7f95080057f0 comp rx=0 tx=0).stop 2023-08-09T14:52:42.546 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:42.537+0000 7f951f7fe640 1 -- 172.21.15.43:0/15550343 reap_dead start 2023-08-09T14:52:42.546 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:42.537+0000 7f951ffff640 1 -- 172.21.15.43:0/15550343 shutdown_connections 2023-08-09T14:52:42.546 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:42.537+0000 7f951ffff640 1 -- 172.21.15.43:0/15550343 >> 172.21.15.43:0/15550343 conn(0x7f952006a320 msgr2=0x7f952006a730 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-09T14:52:42.547 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:42.537+0000 7f951ffff640 1 -- 172.21.15.43:0/15550343 shutdown_connections 2023-08-09T14:52:42.547 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:42.537+0000 7f951ffff640 1 -- 172.21.15.43:0/15550343 wait complete. 2023-08-09T14:52:42.547 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:42.539+0000 7f951ffff640 1 Processor -- start 2023-08-09T14:52:42.547 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:42.539+0000 7f951ffff640 1 -- start start 2023-08-09T14:52:42.548 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:42.540+0000 7f951ffff640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f9508003970 0x7f952014a400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T14:52:42.548 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:42.540+0000 7f951ffff640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f95200b1a20 0x7f95200cb8d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T14:52:42.548 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:42.540+0000 7f951ffff640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f95200b1ea0 0x7f95200cbe10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T14:52:42.549 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:42.540+0000 7f951ffff640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f95200cc350 con 0x7f95200b1ea0 2023-08-09T14:52:42.549 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:42.540+0000 7f951ffff640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f95200b2470 con 0x7f9508003970 2023-08-09T14:52:42.549 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:42.540+0000 7f951ffff640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f95200b2750 con 0x7f95200b1a20 2023-08-09T14:52:42.549 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:42.540+0000 7f951f7fe640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f95200b1ea0 0x7f95200cbe10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T14:52:42.550 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:42.540+0000 7f951f7fe640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f95200b1ea0 0x7f95200cbe10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:36502/0 (socket says 172.21.15.43:36502) 2023-08-09T14:52:42.550 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:42.540+0000 7f951f7fe640 1 -- 172.21.15.43:0/2908302261 learned_addr learned my addr 172.21.15.43:0/2908302261 (peer_addr_for_me v2:172.21.15.43:0/0) 2023-08-09T14:52:42.550 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:42.541+0000 7f951effd640 1 --2- 172.21.15.43:0/2908302261 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f9508003970 0x7f952014a400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T14:52:42.551 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:42.541+0000 7f951e7fc640 1 --2- 172.21.15.43:0/2908302261 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f95200b1a20 0x7f95200cb8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T14:52:42.551 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:42.541+0000 7f951f7fe640 1 -- 172.21.15.43:0/2908302261 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f95200b1a20 msgr2=0x7f95200cb8d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T14:52:42.551 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:42.541+0000 7f951f7fe640 1 --2- 172.21.15.43:0/2908302261 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f95200b1a20 0x7f95200cb8d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T14:52:42.551 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:42.541+0000 7f951f7fe640 1 -- 172.21.15.43:0/2908302261 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f9508003970 msgr2=0x7f952014a400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T14:52:42.552 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:42.541+0000 7f951f7fe640 1 --2- 172.21.15.43:0/2908302261 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f9508003970 0x7f952014a400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T14:52:42.552 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:42.541+0000 7f951f7fe640 1 -- 172.21.15.43:0/2908302261 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f95200b2a30 con 0x7f95200b1ea0 2023-08-09T14:52:42.552 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:42.541+0000 7f951f7fe640 1 --2- 172.21.15.43:0/2908302261 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f95200b1ea0 0x7f95200cbe10 secure :-1 s=READY pgs=2082 cs=0 l=1 rev1=1 crypto rx=0x7f952015fdd0 tx=0x7f9518009d20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-09T14:52:42.553 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:42.541+0000 7f95077fe640 1 -- 172.21.15.43:0/2908302261 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9518007cf0 con 0x7f95200b1ea0 2023-08-09T14:52:42.553 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:42.542+0000 7f95077fe640 1 -- 172.21.15.43:0/2908302261 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f951800f050 con 0x7f95200b1ea0 2023-08-09T14:52:42.553 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:42.542+0000 7f95077fe640 1 -- 172.21.15.43:0/2908302261 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9518014650 con 0x7f95200b1ea0 2023-08-09T14:52:42.553 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:42.542+0000 7f951ffff640 1 -- 172.21.15.43:0/2908302261 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f95200b2d10 con 0x7f95200b1ea0 2023-08-09T14:52:42.554 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:42.542+0000 7f951ffff640 1 -- 172.21.15.43:0/2908302261 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f952014a940 con 0x7f95200b1ea0 2023-08-09T14:52:42.554 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:42.543+0000 7f951ffff640 1 -- 172.21.15.43:0/2908302261 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f95200b3230 con 0x7f95200b1ea0 2023-08-09T14:52:42.555 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:42.545+0000 7f95077fe640 1 -- 172.21.15.43:0/2908302261 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 89) v1 ==== 88460+0+0 (secure 0 0 0) 0x7f95180147b0 con 0x7f95200b1ea0 2023-08-09T14:52:42.556 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:42.546+0000 7f95077fe640 1 --2- 172.21.15.43:0/2908302261 >> [v2:172.21.15.203:6832/2959400181,v1:172.21.15.203:6833/2959400181] conn(0x7f950c06a1c0 0x7f950c06c680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T14:52:42.556 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:42.546+0000 7f95077fe640 1 -- 172.21.15.43:0/2908302261 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(186..186 src has 1..186) v4 ==== 6426+0+0 (secure 0 0 0) 0x7f9518088e90 con 0x7f95200b1ea0 2023-08-09T14:52:42.556 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:42.546+0000 7f951effd640 1 -- 172.21.15.43:0/2908302261 >> [v2:172.21.15.203:6832/2959400181,v1:172.21.15.203:6833/2959400181] conn(0x7f950c06a1c0 msgr2=0x7f950c06c680 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.203:6832/2959400181 2023-08-09T14:52:42.556 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:42.546+0000 7f951effd640 1 --2- 172.21.15.43:0/2908302261 >> [v2:172.21.15.203:6832/2959400181,v1:172.21.15.203:6833/2959400181] conn(0x7f950c06a1c0 0x7f950c06c680 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-08-09T14:52:42.556 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:42.551+0000 7f95077fe640 1 -- 172.21.15.43:0/2908302261 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178066 (secure 0 0 0) 0x7f95200b3230 con 0x7f95200b1ea0 2023-08-09T14:52:42.748 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:42.747+0000 7f951effd640 1 -- 172.21.15.43:0/2908302261 >> [v2:172.21.15.203:6832/2959400181,v1:172.21.15.203:6833/2959400181] conn(0x7f950c06a1c0 msgr2=0x7f950c06c680 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.203:6832/2959400181 2023-08-09T14:52:42.748 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:42.747+0000 7f951effd640 1 --2- 172.21.15.43:0/2908302261 >> [v2:172.21.15.203:6832/2959400181,v1:172.21.15.203:6833/2959400181] conn(0x7f950c06a1c0 0x7f950c06c680 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2023-08-09T14:52:42.768 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:42.768+0000 7f951ffff640 1 -- 172.21.15.43:0/2908302261 --> [v2:172.21.15.203:6832/2959400181,v1:172.21.15.203:6833/2959400181] -- mgr_command(tid 0: {"prefix": "fs snap-schedule add", "fs": "cephfs", "path": "/snap_test_dir1/test_restart", "snap_schedule": "1M", "target": ["mon-mgr", ""]}) v1 -- 0x7f95200d1c50 con 0x7f950c06a1c0 2023-08-09T14:52:43.026 INFO:tasks.ceph.mgr.x.smithi203.stderr:2023-08-09T14:52:43.026+0000 7f25af26f040 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2023-08-09T14:52:43.086 INFO:tasks.ceph.mgr.x.smithi203.stderr:2023-08-09T14:52:43.086+0000 7f25af26f040 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2023-08-09T14:52:43.135 INFO:tasks.ceph.mgr.x.smithi203.stderr:2023-08-09T14:52:43.135+0000 7f25af26f040 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2023-08-09T14:52:43.148 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:43.148+0000 7f951effd640 1 -- 172.21.15.43:0/2908302261 >> [v2:172.21.15.203:6832/2959400181,v1:172.21.15.203:6833/2959400181] conn(0x7f950c06a1c0 msgr2=0x7f950c06c680 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.203:6832/2959400181 2023-08-09T14:52:43.149 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:43.148+0000 7f951effd640 1 --2- 172.21.15.43:0/2908302261 >> [v2:172.21.15.203:6832/2959400181,v1:172.21.15.203:6833/2959400181] conn(0x7f950c06a1c0 0x7f950c06c680 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2023-08-09T14:52:43.164 INFO:tasks.ceph.mgr.y.smithi043.stderr:2023-08-09T14:52:43.163+0000 7f1ac4c20040 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2023-08-09T14:52:43.228 INFO:tasks.ceph.mgr.y.smithi043.stderr:2023-08-09T14:52:43.228+0000 7f1ac4c20040 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2023-08-09T14:52:43.252 INFO:tasks.ceph.mgr.x.smithi203.stderr:2023-08-09T14:52:43.251+0000 7f25af26f040 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2023-08-09T14:52:43.280 INFO:tasks.ceph.mgr.y.smithi043.stderr:2023-08-09T14:52:43.280+0000 7f1ac4c20040 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2023-08-09T14:52:43.406 INFO:tasks.ceph.mgr.x.smithi203.stderr:2023-08-09T14:52:43.406+0000 7f25af26f040 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2023-08-09T14:52:43.410 INFO:tasks.ceph.mgr.y.smithi043.stderr:2023-08-09T14:52:43.410+0000 7f1ac4c20040 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2023-08-09T14:52:43.513 INFO:tasks.ceph.mgr.x.smithi203.stderr:2023-08-09T14:52:43.512+0000 7f25af26f040 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2023-08-09T14:52:43.562 INFO:tasks.ceph.mgr.x.smithi203.stderr:2023-08-09T14:52:43.561+0000 7f25af26f040 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2023-08-09T14:52:43.575 INFO:tasks.ceph.mgr.y.smithi043.stderr:2023-08-09T14:52:43.574+0000 7f1ac4c20040 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2023-08-09T14:52:43.689 INFO:tasks.ceph.mgr.y.smithi043.stderr:2023-08-09T14:52:43.688+0000 7f1ac4c20040 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2023-08-09T14:52:43.721 INFO:tasks.ceph.mgr.x.smithi203.stderr:2023-08-09T14:52:43.721+0000 7f25af26f040 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2023-08-09T14:52:43.740 INFO:tasks.ceph.mgr.y.smithi043.stderr:2023-08-09T14:52:43.740+0000 7f1ac4c20040 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2023-08-09T14:52:43.918 INFO:tasks.ceph.mgr.y.smithi043.stderr:2023-08-09T14:52:43.917+0000 7f1ac4c20040 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2023-08-09T14:52:43.950 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:43.949+0000 7f951effd640 1 -- 172.21.15.43:0/2908302261 >> [v2:172.21.15.203:6832/2959400181,v1:172.21.15.203:6833/2959400181] conn(0x7f950c06a1c0 msgr2=0x7f950c06c680 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.203:6832/2959400181 2023-08-09T14:52:43.950 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:43.949+0000 7f951effd640 1 --2- 172.21.15.43:0/2908302261 >> [v2:172.21.15.203:6832/2959400181,v1:172.21.15.203:6833/2959400181] conn(0x7f950c06a1c0 0x7f950c06c680 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2023-08-09T14:52:44.098 INFO:tasks.ceph.mgr.x.smithi203.stderr:2023-08-09T14:52:44.097+0000 7f25af26f040 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2023-08-09T14:52:44.317 INFO:tasks.ceph.mgr.x.smithi203.stderr:2023-08-09T14:52:44.317+0000 7f25af26f040 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2023-08-09T14:52:44.320 INFO:tasks.ceph.mgr.y.smithi043.stderr:2023-08-09T14:52:44.319+0000 7f1ac4c20040 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2023-08-09T14:52:44.374 INFO:tasks.ceph.mgr.x.smithi203.stderr:2023-08-09T14:52:44.373+0000 7f25af26f040 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2023-08-09T14:52:44.423 INFO:tasks.ceph.mgr.x.smithi203.stderr:2023-08-09T14:52:44.423+0000 7f25af26f040 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2023-08-09T14:52:44.475 INFO:tasks.ceph.mgr.x.smithi203.stderr:2023-08-09T14:52:44.474+0000 7f25af26f040 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2023-08-09T14:52:44.552 INFO:tasks.ceph.mgr.y.smithi043.stderr:2023-08-09T14:52:44.550+0000 7f1ac4c20040 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2023-08-09T14:52:44.612 INFO:tasks.ceph.mgr.y.smithi043.stderr:2023-08-09T14:52:44.610+0000 7f1ac4c20040 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2023-08-09T14:52:44.666 INFO:tasks.ceph.mgr.y.smithi043.stderr:2023-08-09T14:52:44.664+0000 7f1ac4c20040 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2023-08-09T14:52:44.720 INFO:tasks.ceph.mgr.y.smithi043.stderr:2023-08-09T14:52:44.718+0000 7f1ac4c20040 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2023-08-09T14:52:44.768 INFO:tasks.ceph.mgr.x.smithi203.stderr:2023-08-09T14:52:44.767+0000 7f25af26f040 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2023-08-09T14:52:44.825 INFO:tasks.ceph.mgr.x.smithi203.stderr:2023-08-09T14:52:44.825+0000 7f25af26f040 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2023-08-09T14:52:44.877 INFO:tasks.ceph.mgr.x.smithi203.stderr:2023-08-09T14:52:44.876+0000 7f25af26f040 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2023-08-09T14:52:45.030 INFO:tasks.ceph.mgr.y.smithi043.stderr:2023-08-09T14:52:45.029+0000 7f1ac4c20040 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2023-08-09T14:52:45.034 INFO:tasks.ceph.mgr.x.smithi203.stderr:2023-08-09T14:52:45.034+0000 7f25af26f040 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2023-08-09T14:52:45.083 INFO:tasks.ceph.mgr.x.smithi203.stderr:2023-08-09T14:52:45.083+0000 7f25af26f040 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2023-08-09T14:52:45.091 INFO:tasks.ceph.mgr.y.smithi043.stderr:2023-08-09T14:52:45.089+0000 7f1ac4c20040 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2023-08-09T14:52:45.135 INFO:tasks.ceph.mgr.x.smithi203.stderr:2023-08-09T14:52:45.135+0000 7f25af26f040 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2023-08-09T14:52:45.145 INFO:tasks.ceph.mgr.y.smithi043.stderr:2023-08-09T14:52:45.144+0000 7f1ac4c20040 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2023-08-09T14:52:45.185 INFO:tasks.ceph.mgr.x.smithi203.stderr:2023-08-09T14:52:45.184+0000 7f25af26f040 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2023-08-09T14:52:45.210 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:45.208+0000 7f95077fe640 1 -- 172.21.15.43:0/2908302261 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mgrmap(e 90) v1 ==== 88069+0+0 (secure 0 0 0) 0x7f95180545b0 con 0x7f95200b1ea0 2023-08-09T14:52:45.211 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:45.209+0000 7f95077fe640 1 -- 172.21.15.43:0/2908302261 >> [v2:172.21.15.203:6832/2959400181,v1:172.21.15.203:6833/2959400181] conn(0x7f950c06a1c0 msgr2=0x7f950c06c680 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-08-09T14:52:45.211 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:45.209+0000 7f95077fe640 1 --2- 172.21.15.43:0/2908302261 >> [v2:172.21.15.203:6832/2959400181,v1:172.21.15.203:6833/2959400181] conn(0x7f950c06a1c0 0x7f950c06c680 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T14:52:45.323 INFO:tasks.ceph.mgr.y.smithi043.stderr:2023-08-09T14:52:45.321+0000 7f1ac4c20040 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2023-08-09T14:52:45.332 INFO:tasks.ceph.mgr.x.smithi203.stderr:2023-08-09T14:52:45.332+0000 7f252f539640 -1 client.0 error registering admin socket command: (17) File exists 2023-08-09T14:52:45.332 INFO:tasks.ceph.mgr.x.smithi203.stderr:2023-08-09T14:52:45.332+0000 7f252f539640 -1 client.0 error registering admin socket command: (17) File exists 2023-08-09T14:52:45.332 INFO:tasks.ceph.mgr.x.smithi203.stderr:2023-08-09T14:52:45.332+0000 7f252f539640 -1 client.0 error registering admin socket command: (17) File exists 2023-08-09T14:52:45.333 INFO:tasks.ceph.mgr.x.smithi203.stderr:2023-08-09T14:52:45.332+0000 7f252f539640 -1 client.0 error registering admin socket command: (17) File exists 2023-08-09T14:52:45.333 INFO:tasks.ceph.mgr.x.smithi203.stderr:2023-08-09T14:52:45.332+0000 7f252f539640 -1 client.0 error registering admin socket command: (17) File exists 2023-08-09T14:52:45.333 INFO:tasks.ceph.mgr.x.smithi203.stderr:2023-08-09T14:52:45.332+0000 7f252c533640 -1 client.0 error registering admin socket command: (17) File exists 2023-08-09T14:52:45.334 INFO:tasks.ceph.mgr.x.smithi203.stderr:2023-08-09T14:52:45.332+0000 7f252c533640 -1 client.0 error registering admin socket command: (17) File exists 2023-08-09T14:52:45.334 INFO:tasks.ceph.mgr.x.smithi203.stderr:2023-08-09T14:52:45.332+0000 7f252c533640 -1 client.0 error registering admin socket command: (17) File exists 2023-08-09T14:52:45.334 INFO:tasks.ceph.mgr.x.smithi203.stderr:2023-08-09T14:52:45.332+0000 7f252c533640 -1 client.0 error registering admin socket command: (17) File exists 2023-08-09T14:52:45.334 INFO:tasks.ceph.mgr.x.smithi203.stderr:2023-08-09T14:52:45.332+0000 7f252c533640 -1 client.0 error registering admin socket command: (17) File exists 2023-08-09T14:52:45.376 INFO:tasks.ceph.mgr.y.smithi043.stderr:2023-08-09T14:52:45.373+0000 7f1ac4c20040 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2023-08-09T14:52:45.433 INFO:tasks.ceph.mgr.y.smithi043.stderr:2023-08-09T14:52:45.431+0000 7f1ac4c20040 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2023-08-09T14:52:45.485 INFO:tasks.ceph.mgr.y.smithi043.stderr:2023-08-09T14:52:45.484+0000 7f1ac4c20040 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2023-08-09T14:52:46.218 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:46.217+0000 7f95077fe640 1 -- 172.21.15.43:0/2908302261 <== mon.0 v2:172.21.15.43:3300/0 8 ==== mgrmap(e 91) v1 ==== 88196+0+0 (secure 0 0 0) 0x7f951806f230 con 0x7f95200b1ea0 2023-08-09T14:52:46.219 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:46.218+0000 7f95077fe640 1 --2- 172.21.15.43:0/2908302261 >> [v2:172.21.15.203:6832/40883597,v1:172.21.15.203:6833/40883597] conn(0x7f950c072cb0 0x7f950c0750a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T14:52:46.219 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:46.218+0000 7f95077fe640 1 -- 172.21.15.43:0/2908302261 --> [v2:172.21.15.203:6832/40883597,v1:172.21.15.203:6833/40883597] -- mgr_command(tid 0: {"prefix": "fs snap-schedule add", "fs": "cephfs", "path": "/snap_test_dir1/test_restart", "snap_schedule": "1M", "target": ["mon-mgr", ""]}) v1 -- 0x7f950c027d90 con 0x7f950c072cb0 2023-08-09T14:52:46.220 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:46.219+0000 7f951effd640 1 --2- 172.21.15.43:0/2908302261 >> [v2:172.21.15.203:6832/40883597,v1:172.21.15.203:6833/40883597] conn(0x7f950c072cb0 0x7f950c0750a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T14:52:46.233 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:46.231+0000 7f951effd640 1 --2- 172.21.15.43:0/2908302261 >> [v2:172.21.15.203:6832/40883597,v1:172.21.15.203:6833/40883597] conn(0x7f950c072cb0 0x7f950c0750a0 secure :-1 s=READY pgs=4 cs=0 l=1 rev1=1 crypto rx=0x7f9510034000 tx=0x7f9510037040 comp rx=0 tx=0).ready entity=mgr.8347 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-09T14:52:46.359 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:46.358+0000 7f95077fe640 1 -- 172.21.15.43:0/2908302261 <== mgr.8347 v2:172.21.15.203:6832/40883597 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+51 (secure 0 0 0) 0x7f950c027d90 con 0x7f950c072cb0 2023-08-09T14:52:46.360 INFO:teuthology.orchestra.run.smithi043.stdout:Schedule set for path /snap_test_dir1/test_restart 2023-08-09T14:52:46.363 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:46.361+0000 7f951ffff640 1 -- 172.21.15.43:0/2908302261 >> [v2:172.21.15.203:6832/40883597,v1:172.21.15.203:6833/40883597] conn(0x7f950c072cb0 msgr2=0x7f950c0750a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T14:52:46.363 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:46.361+0000 7f951ffff640 1 --2- 172.21.15.43:0/2908302261 >> [v2:172.21.15.203:6832/40883597,v1:172.21.15.203:6833/40883597] conn(0x7f950c072cb0 0x7f950c0750a0 secure :-1 s=READY pgs=4 cs=0 l=1 rev1=1 crypto rx=0x7f9510034000 tx=0x7f9510037040 comp rx=0 tx=0).stop 2023-08-09T14:52:46.363 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:46.361+0000 7f951ffff640 1 -- 172.21.15.43:0/2908302261 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f95200b1ea0 msgr2=0x7f95200cbe10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T14:52:46.364 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:46.362+0000 7f951ffff640 1 --2- 172.21.15.43:0/2908302261 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f95200b1ea0 0x7f95200cbe10 secure :-1 s=READY pgs=2082 cs=0 l=1 rev1=1 crypto rx=0x7f952015fdd0 tx=0x7f9518009d20 comp rx=0 tx=0).stop 2023-08-09T14:52:46.364 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:46.362+0000 7f951f7fe640 1 -- 172.21.15.43:0/2908302261 reap_dead start 2023-08-09T14:52:46.364 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:46.362+0000 7f951ffff640 1 -- 172.21.15.43:0/2908302261 shutdown_connections 2023-08-09T14:52:46.364 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:46.362+0000 7f951ffff640 1 -- 172.21.15.43:0/2908302261 >> 172.21.15.43:0/2908302261 conn(0x7f952006a320 msgr2=0x7f95200b7790 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-09T14:52:46.364 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:46.362+0000 7f951ffff640 1 -- 172.21.15.43:0/2908302261 shutdown_connections 2023-08-09T14:52:46.365 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:46.362+0000 7f951ffff640 1 -- 172.21.15.43:0/2908302261 wait complete. 2023-08-09T14:52:46.384 DEBUG:tasks.cephfs.test_snap_schedules:res=Schedule set for path /snap_test_dir1/test_restart 2023-08-09T14:52:46.385 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs snap-schedule activate --fs cephfs /snap_test_dir1/test_restart 1M 2023-08-09T14:52:46.625 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:46.623+0000 7f6ab4a33640 1 Processor -- start 2023-08-09T14:52:46.625 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:46.623+0000 7f6ab4a33640 1 -- start start 2023-08-09T14:52:46.628 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:46.625+0000 7f6ab4a33640 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7f6ab006b400 0x7f6ab006cae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T14:52:46.628 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:46.625+0000 7f6ab4a33640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f6ab015c8e0 0x7f6ab015cd50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T14:52:46.628 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:46.625+0000 7f6ab4a33640 1 -- --> v1:172.21.15.203:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6ab00ad260 con 0x7f6ab015c4f0 2023-08-09T14:52:46.628 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:46.625+0000 7f6ab4a33640 1 -- --> v2:172.21.15.43:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6ab006c280 con 0x7f6ab015c8e0 2023-08-09T14:52:46.629 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:46.625+0000 7f6ab4a33640 1 -- --> v2:172.21.15.203:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6ab015d290 con 0x7f6ab006b400 2023-08-09T14:52:46.629 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:46.626+0000 7f6aaf59e640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f6ab015c8e0 0x7f6ab015cd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T14:52:46.629 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:46.626+0000 7f6aaf59e640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f6ab015c8e0 0x7f6ab015cd50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:59078/0 (socket says 172.21.15.43:59078) 2023-08-09T14:52:46.629 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:46.626+0000 7f6aaf59e640 1 -- 172.21.15.43:0/2727442843 learned_addr learned my addr 172.21.15.43:0/2727442843 (peer_addr_for_me v2:172.21.15.43:0/0) 2023-08-09T14:52:46.630 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:46.626+0000 7f6aa7fff640 1 --2- 172.21.15.43:0/2727442843 >> v2:172.21.15.203:3300/0 conn(0x7f6ab006b400 0x7f6ab006cae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T14:52:46.630 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:46.626+0000 7f6aaf59e640 1 -- 172.21.15.43:0/2727442843 >> v2:172.21.15.203:3300/0 conn(0x7f6ab006b400 msgr2=0x7f6ab006cae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-09T14:52:46.630 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:46.626+0000 7f6aaf59e640 1 --2- 172.21.15.43:0/2727442843 >> v2:172.21.15.203:3300/0 conn(0x7f6ab006b400 0x7f6ab006cae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T14:52:46.631 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:46.626+0000 7f6aaf59e640 1 -- 172.21.15.43:0/2727442843 >> v1:172.21.15.203:6789/0 conn(0x7f6ab015c4f0 legacy=0x7f6ab0160590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-09T14:52:46.631 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:46.626+0000 7f6aaf59e640 1 -- 172.21.15.43:0/2727442843 --> v2:172.21.15.43:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6ab006b9e0 con 0x7f6ab015c8e0 2023-08-09T14:52:46.631 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:46.626+0000 7f6aa7fff640 1 --2- 172.21.15.43:0/2727442843 >> v2:172.21.15.203:3300/0 conn(0x7f6ab006b400 0x7f6ab006cae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-08-09T14:52:46.632 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:46.627+0000 7f6aaf59e640 1 --2- 172.21.15.43:0/2727442843 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f6ab015c8e0 0x7f6ab015cd50 secure :-1 s=READY pgs=2083 cs=0 l=1 rev1=1 crypto rx=0x7f6aa8009870 tx=0x7f6aa802b900 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9a19254be95d1594 server_cookie=0 in_seq=0 out_seq=0 2023-08-09T14:52:46.632 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:46.628+0000 7f6aae59c640 1 -- 172.21.15.43:0/2727442843 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6aa8031860 con 0x7f6ab015c8e0 2023-08-09T14:52:46.633 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:46.628+0000 7f6aae59c640 1 -- 172.21.15.43:0/2727442843 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6aa80319c0 con 0x7f6ab015c8e0 2023-08-09T14:52:46.633 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:46.628+0000 7f6aae59c640 1 -- 172.21.15.43:0/2727442843 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6aa8031d50 con 0x7f6ab015c8e0 2023-08-09T14:52:46.633 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:46.628+0000 7f6ab4a33640 1 -- 172.21.15.43:0/2727442843 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f6ab015c8e0 msgr2=0x7f6ab015cd50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T14:52:46.633 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:46.628+0000 7f6ab4a33640 1 --2- 172.21.15.43:0/2727442843 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f6ab015c8e0 0x7f6ab015cd50 secure :-1 s=READY pgs=2083 cs=0 l=1 rev1=1 crypto rx=0x7f6aa8009870 tx=0x7f6aa802b900 comp rx=0 tx=0).stop 2023-08-09T14:52:46.633 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:46.629+0000 7f6ab4a33640 1 -- 172.21.15.43:0/2727442843 shutdown_connections 2023-08-09T14:52:46.633 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:46.629+0000 7f6ab4a33640 1 --2- 172.21.15.43:0/2727442843 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f6ab015c8e0 0x7f6ab015cd50 unknown :-1 s=CLOSED pgs=2083 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T14:52:46.633 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:46.629+0000 7f6ab4a33640 1 --2- 172.21.15.43:0/2727442843 >> v2:172.21.15.203:3300/0 conn(0x7f6ab006b400 0x7f6ab006cae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T14:52:46.634 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:46.629+0000 7f6ab4a33640 1 -- 172.21.15.43:0/2727442843 >> 172.21.15.43:0/2727442843 conn(0x7f6ab00ca870 msgr2=0x7f6ab00cac50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-09T14:52:46.634 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:46.629+0000 7f6ab4a33640 1 -- 172.21.15.43:0/2727442843 shutdown_connections 2023-08-09T14:52:46.634 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:46.629+0000 7f6ab4a33640 1 -- 172.21.15.43:0/2727442843 wait complete. 2023-08-09T14:52:46.634 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:46.631+0000 7f6ab4a33640 1 Processor -- start 2023-08-09T14:52:46.634 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:46.631+0000 7f6ab4a33640 1 -- start start 2023-08-09T14:52:46.636 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:46.632+0000 7f6ab4a33640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f6ab006b400 0x7f6ab01f5310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T14:52:46.636 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:46.632+0000 7f6ab4a33640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f6ab015c8e0 0x7f6ab00ad820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T14:52:46.636 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:46.632+0000 7f6ab4a33640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f6ab00add60 0x7f6ab00b0a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T14:52:46.636 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:46.632+0000 7f6ab4a33640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6ab00aeeb0 con 0x7f6ab006b400 2023-08-09T14:52:46.636 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:46.632+0000 7f6ab4a33640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6ab00ae2a0 con 0x7f6ab00add60 2023-08-09T14:52:46.637 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:46.632+0000 7f6ab4a33640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6ab00ae580 con 0x7f6ab015c8e0 2023-08-09T14:52:46.637 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:46.632+0000 7f6aaed9d640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f6ab006b400 0x7f6ab01f5310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T14:52:46.637 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:46.632+0000 7f6aaed9d640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f6ab006b400 0x7f6ab01f5310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:59088/0 (socket says 172.21.15.43:59088) 2023-08-09T14:52:46.637 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:46.632+0000 7f6aaed9d640 1 -- 172.21.15.43:0/1283119771 learned_addr learned my addr 172.21.15.43:0/1283119771 (peer_addr_for_me v2:172.21.15.43:0/0) 2023-08-09T14:52:46.637 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:46.632+0000 7f6aa7fff640 1 --2- 172.21.15.43:0/1283119771 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f6ab015c8e0 0x7f6ab00ad820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T14:52:46.637 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:46.632+0000 7f6aaf59e640 1 --2- 172.21.15.43:0/1283119771 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f6ab00add60 0x7f6ab00b0a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T14:52:46.637 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:46.632+0000 7f6aaed9d640 1 -- 172.21.15.43:0/1283119771 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f6ab015c8e0 msgr2=0x7f6ab00ad820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T14:52:46.638 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:46.632+0000 7f6aaed9d640 1 --2- 172.21.15.43:0/1283119771 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f6ab015c8e0 0x7f6ab00ad820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T14:52:46.638 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:46.632+0000 7f6aaed9d640 1 -- 172.21.15.43:0/1283119771 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f6ab00add60 msgr2=0x7f6ab00b0a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T14:52:46.638 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:46.632+0000 7f6aaed9d640 1 --2- 172.21.15.43:0/1283119771 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f6ab00add60 0x7f6ab00b0a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T14:52:46.638 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:46.633+0000 7f6aaed9d640 1 -- 172.21.15.43:0/1283119771 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6aa80094d0 con 0x7f6ab006b400 2023-08-09T14:52:46.638 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:46.633+0000 7f6aa7fff640 1 --2- 172.21.15.43:0/1283119771 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f6ab015c8e0 0x7f6ab00ad820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-08-09T14:52:46.638 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:46.633+0000 7f6aaf59e640 1 --2- 172.21.15.43:0/1283119771 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f6ab00add60 0x7f6ab00b0a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-08-09T14:52:46.638 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:46.633+0000 7f6aaed9d640 1 --2- 172.21.15.43:0/1283119771 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f6ab006b400 0x7f6ab01f5310 secure :-1 s=READY pgs=2084 cs=0 l=1 rev1=1 crypto rx=0x7f6aa000c7c0 tx=0x7f6aa000cc80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-09T14:52:46.638 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:46.633+0000 7f6aa77fe640 1 -- 172.21.15.43:0/1283119771 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6aa000aa50 con 0x7f6ab006b400 2023-08-09T14:52:46.639 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:46.633+0000 7f6aa77fe640 1 -- 172.21.15.43:0/1283119771 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6aa000abb0 con 0x7f6ab006b400 2023-08-09T14:52:46.639 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:46.633+0000 7f6aa77fe640 1 -- 172.21.15.43:0/1283119771 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6aa0012420 con 0x7f6ab006b400 2023-08-09T14:52:46.639 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:46.633+0000 7f6ab4a33640 1 -- 172.21.15.43:0/1283119771 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6ab00ae860 con 0x7f6ab006b400 2023-08-09T14:52:46.639 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:46.633+0000 7f6ab4a33640 1 -- 172.21.15.43:0/1283119771 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6ab0155c70 con 0x7f6ab006b400 2023-08-09T14:52:46.639 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:46.634+0000 7f6ab4a33640 1 -- 172.21.15.43:0/1283119771 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6ab00af020 con 0x7f6ab006b400 2023-08-09T14:52:46.640 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:46.635+0000 7f6aa77fe640 1 -- 172.21.15.43:0/1283119771 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 91) v1 ==== 88196+0+0 (secure 0 0 0) 0x7f6aa0012580 con 0x7f6ab006b400 2023-08-09T14:52:46.640 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:46.636+0000 7f6aa77fe640 1 --2- 172.21.15.43:0/1283119771 >> [v2:172.21.15.203:6832/40883597,v1:172.21.15.203:6833/40883597] conn(0x7f6a90069ef0 0x7f6a9006c3b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T14:52:46.640 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:46.636+0000 7f6aa77fe640 1 -- 172.21.15.43:0/1283119771 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(187..187 src has 1..187) v4 ==== 6770+0+0 (secure 0 0 0) 0x7f6aa0088650 con 0x7f6ab006b400 2023-08-09T14:52:46.640 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:46.637+0000 7f6aa7fff640 1 --2- 172.21.15.43:0/1283119771 >> [v2:172.21.15.203:6832/40883597,v1:172.21.15.203:6833/40883597] conn(0x7f6a90069ef0 0x7f6a9006c3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T14:52:46.640 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:46.638+0000 7f6aa7fff640 1 --2- 172.21.15.43:0/1283119771 >> [v2:172.21.15.203:6832/40883597,v1:172.21.15.203:6833/40883597] conn(0x7f6a90069ef0 0x7f6a9006c3b0 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7f6a98006fd0 tx=0x7f6a98008040 comp rx=0 tx=0).ready entity=mgr.8347 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-09T14:52:46.642 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:46.641+0000 7f6aa77fe640 1 -- 172.21.15.43:0/1283119771 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178066 (secure 0 0 0) 0x7f6aa0053c50 con 0x7f6ab006b400 2023-08-09T14:52:46.862 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:46.860+0000 7f6ab4a33640 1 -- 172.21.15.43:0/1283119771 --> [v2:172.21.15.203:6832/40883597,v1:172.21.15.203:6833/40883597] -- mgr_command(tid 0: {"prefix": "fs snap-schedule activate", "fs": "cephfs", "path": "/snap_test_dir1/test_restart", "repeat": "1M", "target": ["mon-mgr", ""]}) v1 -- 0x7f6ab006bfa0 con 0x7f6a90069ef0 2023-08-09T14:52:46.976 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:46.975+0000 7f6aa77fe640 1 -- 172.21.15.43:0/1283119771 <== mgr.8347 v2:172.21.15.203:6832/40883597 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+57 (secure 0 0 0) 0x7f6a9800c040 con 0x7f6a90069ef0 2023-08-09T14:52:46.977 INFO:teuthology.orchestra.run.smithi043.stdout:Schedule activated for path /snap_test_dir1/test_restart 2023-08-09T14:52:46.979 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:46.978+0000 7f6ab4a33640 1 -- 172.21.15.43:0/1283119771 >> [v2:172.21.15.203:6832/40883597,v1:172.21.15.203:6833/40883597] conn(0x7f6a90069ef0 msgr2=0x7f6a9006c3b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T14:52:46.980 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:46.978+0000 7f6ab4a33640 1 --2- 172.21.15.43:0/1283119771 >> [v2:172.21.15.203:6832/40883597,v1:172.21.15.203:6833/40883597] conn(0x7f6a90069ef0 0x7f6a9006c3b0 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7f6a98006fd0 tx=0x7f6a98008040 comp rx=0 tx=0).stop 2023-08-09T14:52:46.980 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:46.978+0000 7f6ab4a33640 1 -- 172.21.15.43:0/1283119771 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f6ab006b400 msgr2=0x7f6ab01f5310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T14:52:46.980 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:46.978+0000 7f6ab4a33640 1 --2- 172.21.15.43:0/1283119771 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f6ab006b400 0x7f6ab01f5310 secure :-1 s=READY pgs=2084 cs=0 l=1 rev1=1 crypto rx=0x7f6aa000c7c0 tx=0x7f6aa000cc80 comp rx=0 tx=0).stop 2023-08-09T14:52:46.980 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:46.978+0000 7f6ab4a33640 1 -- 172.21.15.43:0/1283119771 shutdown_connections 2023-08-09T14:52:46.980 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:46.978+0000 7f6ab4a33640 1 --2- 172.21.15.43:0/1283119771 >> [v2:172.21.15.203:6832/40883597,v1:172.21.15.203:6833/40883597] conn(0x7f6a90069ef0 0x7f6a9006c3b0 unknown :-1 s=CLOSED pgs=23 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T14:52:46.980 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:46.978+0000 7f6ab4a33640 1 --2- 172.21.15.43:0/1283119771 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f6ab00add60 0x7f6ab00b0a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T14:52:46.981 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:46.978+0000 7f6ab4a33640 1 --2- 172.21.15.43:0/1283119771 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f6ab015c8e0 0x7f6ab00ad820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T14:52:46.981 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:46.978+0000 7f6ab4a33640 1 --2- 172.21.15.43:0/1283119771 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f6ab006b400 0x7f6ab01f5310 unknown :-1 s=CLOSED pgs=2084 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T14:52:46.981 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:46.978+0000 7f6ab4a33640 1 -- 172.21.15.43:0/1283119771 >> 172.21.15.43:0/1283119771 conn(0x7f6ab00ca870 msgr2=0x7f6ab006feb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-09T14:52:46.981 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:46.978+0000 7f6ab4a33640 1 -- 172.21.15.43:0/1283119771 shutdown_connections 2023-08-09T14:52:46.981 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T14:52:46.979+0000 7f6ab4a33640 1 -- 172.21.15.43:0/1283119771 wait complete. 2023-08-09T14:52:46.999 DEBUG:tasks.cephfs.test_snap_schedules:res=Schedule activated for path /snap_test_dir1/test_restart 2023-08-09T14:52:55.366 DEBUG:teuthology.orchestra.run.smithi043:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-08-09T14:52:55.375 DEBUG:teuthology.orchestra.run.smithi203:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-08-09T14:53:00.997 INFO:tasks.ceph.mgr.x.smithi203.stderr:2023-08-09T14:53:00.995+0000 7f251e517640 -1 client.0 error registering admin socket command: (17) File exists 2023-08-09T14:53:00.997 INFO:tasks.ceph.mgr.x.smithi203.stderr:2023-08-09T14:53:00.995+0000 7f251e517640 -1 client.0 error registering admin socket command: (17) File exists 2023-08-09T14:53:00.997 INFO:tasks.ceph.mgr.x.smithi203.stderr:2023-08-09T14:53:00.995+0000 7f251e517640 -1 client.0 error registering admin socket command: (17) File exists 2023-08-09T14:53:00.997 INFO:tasks.ceph.mgr.x.smithi203.stderr:2023-08-09T14:53:00.995+0000 7f251e517640 -1 client.0 error registering admin socket command: (17) File exists 2023-08-09T14:53:00.998 INFO:tasks.ceph.mgr.x.smithi203.stderr:2023-08-09T14:53:00.995+0000 7f251e517640 -1 client.0 error registering admin socket command: (17) File exists 2023-08-09T14:53:25.522 DEBUG:teuthology.orchestra.run.smithi043:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-08-09T14:53:25.531 DEBUG:teuthology.orchestra.run.smithi203:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-08-09T14:53:55.805 DEBUG:teuthology.orchestra.run.smithi043:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-08-09T14:53:55.815 DEBUG:teuthology.orchestra.run.smithi203:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-08-09T14:54:25.971 DEBUG:teuthology.orchestra.run.smithi043:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-08-09T14:54:25.981 DEBUG:teuthology.orchestra.run.smithi203:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-08-09T14:54:56.126 DEBUG:teuthology.orchestra.run.smithi043:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-08-09T14:54:56.140 DEBUG:teuthology.orchestra.run.smithi203:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-08-09T14:55:26.358 DEBUG:teuthology.orchestra.run.smithi043:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-08-09T14:55:26.469 DEBUG:teuthology.orchestra.run.smithi203:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-08-09T14:55:56.614 DEBUG:teuthology.orchestra.run.smithi043:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-08-09T14:55:56.624 DEBUG:teuthology.orchestra.run.smithi203:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-08-09T14:56:26.762 DEBUG:teuthology.orchestra.run.smithi043:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-08-09T14:56:26.772 DEBUG:teuthology.orchestra.run.smithi203:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-08-09T14:56:56.920 DEBUG:teuthology.orchestra.run.smithi043:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-08-09T14:56:56.929 DEBUG:teuthology.orchestra.run.smithi203:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-08-09T14:57:27.093 DEBUG:teuthology.orchestra.run.smithi043:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-08-09T14:57:27.103 DEBUG:teuthology.orchestra.run.smithi203:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-08-09T14:57:57.298 DEBUG:teuthology.orchestra.run.smithi043:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-08-09T14:57:57.308 DEBUG:teuthology.orchestra.run.smithi203:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-08-09T14:58:27.449 DEBUG:teuthology.orchestra.run.smithi043:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-08-09T14:58:27.460 DEBUG:teuthology.orchestra.run.smithi203:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-08-09T14:58:57.594 DEBUG:teuthology.orchestra.run.smithi043:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-08-09T14:58:57.604 DEBUG:teuthology.orchestra.run.smithi203:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-08-09T14:59:27.750 DEBUG:teuthology.orchestra.run.smithi043:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-08-09T14:59:27.760 DEBUG:teuthology.orchestra.run.smithi203:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-08-09T14:59:57.949 DEBUG:teuthology.orchestra.run.smithi043:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-08-09T14:59:57.959 DEBUG:teuthology.orchestra.run.smithi203:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-08-09T15:00:28.113 DEBUG:teuthology.orchestra.run.smithi043:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-08-09T15:00:28.124 DEBUG:teuthology.orchestra.run.smithi203:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-08-09T15:00:58.263 DEBUG:teuthology.orchestra.run.smithi043:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-08-09T15:00:58.272 DEBUG:teuthology.orchestra.run.smithi203:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-08-09T15:01:28.413 DEBUG:teuthology.orchestra.run.smithi043:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-08-09T15:01:28.423 DEBUG:teuthology.orchestra.run.smithi203:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-08-09T15:01:58.565 DEBUG:teuthology.orchestra.run.smithi043:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-08-09T15:01:58.576 DEBUG:teuthology.orchestra.run.smithi203:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-08-09T15:02:28.731 DEBUG:teuthology.orchestra.run.smithi043:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-08-09T15:02:28.741 DEBUG:teuthology.orchestra.run.smithi203:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-08-09T15:02:58.907 DEBUG:teuthology.orchestra.run.smithi043:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-08-09T15:02:58.916 DEBUG:teuthology.orchestra.run.smithi203:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-08-09T15:03:16.000 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-09T15:03:16.001 DEBUG:teuthology.orchestra.run.smithi043:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/test_restart/.snap) 2023-08-09T15:03:16.081 INFO:teuthology.orchestra.run.smithi043.stdout:scheduled-2023-08-09-14_53_00_UTC 2023-08-09T15:03:16.082 INFO:teuthology.orchestra.run.smithi043.stdout:scheduled-2023-08-09-14_54_00_UTC 2023-08-09T15:03:16.082 INFO:teuthology.orchestra.run.smithi043.stdout:scheduled-2023-08-09-14_55_00_UTC 2023-08-09T15:03:16.082 INFO:teuthology.orchestra.run.smithi043.stdout:scheduled-2023-08-09-14_56_00_UTC 2023-08-09T15:03:16.082 INFO:teuthology.orchestra.run.smithi043.stdout:scheduled-2023-08-09-14_57_00_UTC 2023-08-09T15:03:16.082 INFO:teuthology.orchestra.run.smithi043.stdout:scheduled-2023-08-09-14_58_00_UTC 2023-08-09T15:03:16.083 INFO:teuthology.orchestra.run.smithi043.stdout:scheduled-2023-08-09-14_59_00_UTC 2023-08-09T15:03:16.083 INFO:teuthology.orchestra.run.smithi043.stdout:scheduled-2023-08-09-15_00_00_UTC 2023-08-09T15:03:16.083 INFO:teuthology.orchestra.run.smithi043.stdout:scheduled-2023-08-09-15_01_00_UTC 2023-08-09T15:03:16.083 INFO:teuthology.orchestra.run.smithi043.stdout:scheduled-2023-08-09-15_02_00_UTC 2023-08-09T15:03:16.083 INFO:teuthology.orchestra.run.smithi043.stdout:scheduled-2023-08-09-15_03_00_UTC 2023-08-09T15:03:16.084 DEBUG:tasks.cephfs.test_snap_schedules:snapshots: ['scheduled-2023-08-09-14_53_00_UTC', 'scheduled-2023-08-09-14_54_00_UTC', 'scheduled-2023-08-09-14_55_00_UTC', 'scheduled-2023-08-09-14_56_00_UTC', 'scheduled-2023-08-09-14_57_00_UTC', 'scheduled-2023-08-09-14_58_00_UTC', 'scheduled-2023-08-09-14_59_00_UTC', 'scheduled-2023-08-09-15_00_00_UTC', 'scheduled-2023-08-09-15_01_00_UTC', 'scheduled-2023-08-09-15_02_00_UTC', 'scheduled-2023-08-09-15_03_00_UTC'] 2023-08-09T15:03:16.085 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs snap-schedule status --fs cephfs --format json /snap_test_dir1/test_restart 2023-08-09T15:03:16.325 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:03:16.323+0000 7fd2cdfa4640 1 Processor -- start 2023-08-09T15:03:16.326 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:03:16.323+0000 7fd2cdfa4640 1 -- start start 2023-08-09T15:03:16.327 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:03:16.324+0000 7fd2cdfa4640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7fd2c815c720 0x7fd2c80cd850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T15:03:16.327 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:03:16.324+0000 7fd2cdfa4640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fd2c80cf370 0x7fd2c80b5de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T15:03:16.327 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:03:16.324+0000 7fd2cdfa4640 1 -- --> v1:172.21.15.43:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd2c80b5660 con 0x7fd2c80d0900 2023-08-09T15:03:16.327 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:03:16.324+0000 7fd2cdfa4640 1 -- --> v2:172.21.15.43:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd2c815cb00 con 0x7fd2c815c720 2023-08-09T15:03:16.327 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:03:16.324+0000 7fd2cdfa4640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd2c80d1960 con 0x7fd2c80cf370 2023-08-09T15:03:16.328 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:03:16.325+0000 7fd2ccfa2640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7fd2c815c720 0x7fd2c80cd850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T15:03:16.328 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:03:16.325+0000 7fd2ccfa2640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7fd2c815c720 0x7fd2c80cd850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:58100/0 (socket says 172.21.15.43:58100) 2023-08-09T15:03:16.328 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:03:16.325+0000 7fd2ccfa2640 1 -- 172.21.15.43:0/1638207531 learned_addr learned my addr 172.21.15.43:0/1638207531 (peer_addr_for_me v2:172.21.15.43:0/0) 2023-08-09T15:03:16.328 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:03:16.325+0000 7fd2ccfa2640 1 -- 172.21.15.43:0/1638207531 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fd2c80cf370 msgr2=0x7fd2c80b5de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-09T15:03:16.328 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:03:16.325+0000 7fd2cd7a3640 1 --2- 172.21.15.43:0/1638207531 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fd2c80cf370 0x7fd2c80b5de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T15:03:16.328 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:03:16.325+0000 7fd2c77fe640 1 -- 172.21.15.43:0/1638207531 <== mon.0 v1:172.21.15.43:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 757146608 0 0) 0x7fd2c80b5660 con 0x7fd2c80d0900 2023-08-09T15:03:16.329 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:03:16.325+0000 7fd2ccfa2640 1 --2- 172.21.15.43:0/1638207531 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fd2c80cf370 0x7fd2c80b5de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T15:03:16.329 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:03:16.325+0000 7fd2ccfa2640 1 -- 172.21.15.43:0/1638207531 >> v1:172.21.15.43:6789/0 conn(0x7fd2c80d0900 legacy=0x7fd2c80cec60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T15:03:16.329 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:03:16.325+0000 7fd2ccfa2640 1 -- 172.21.15.43:0/1638207531 --> v2:172.21.15.43:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd2c80b5900 con 0x7fd2c815c720 2023-08-09T15:03:16.329 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:03:16.326+0000 7fd2cd7a3640 1 --2- 172.21.15.43:0/1638207531 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fd2c80cf370 0x7fd2c80b5de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-08-09T15:03:16.330 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:03:16.327+0000 7fd2ccfa2640 1 --2- 172.21.15.43:0/1638207531 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fd2c815c720 0x7fd2c80cd850 secure :-1 s=READY pgs=2086 cs=0 l=1 rev1=1 crypto rx=0x7fd2b80098a0 tx=0x7fd2b802b980 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f5f004226982a7e1 server_cookie=0 in_seq=0 out_seq=0 2023-08-09T15:03:16.330 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:03:16.327+0000 7fd2c77fe640 1 -- 172.21.15.43:0/1638207531 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd2b8030840 con 0x7fd2c815c720 2023-08-09T15:03:16.331 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:03:16.327+0000 7fd2c77fe640 1 -- 172.21.15.43:0/1638207531 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd2b80309a0 con 0x7fd2c815c720 2023-08-09T15:03:16.331 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:03:16.327+0000 7fd2c77fe640 1 -- 172.21.15.43:0/1638207531 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd2b8030d30 con 0x7fd2c815c720 2023-08-09T15:03:16.331 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:03:16.327+0000 7fd2cdfa4640 1 -- 172.21.15.43:0/1638207531 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fd2c815c720 msgr2=0x7fd2c80cd850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T15:03:16.331 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:03:16.327+0000 7fd2cdfa4640 1 --2- 172.21.15.43:0/1638207531 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fd2c815c720 0x7fd2c80cd850 secure :-1 s=READY pgs=2086 cs=0 l=1 rev1=1 crypto rx=0x7fd2b80098a0 tx=0x7fd2b802b980 comp rx=0 tx=0).stop 2023-08-09T15:03:16.331 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:03:16.328+0000 7fd2cdfa4640 1 -- 172.21.15.43:0/1638207531 shutdown_connections 2023-08-09T15:03:16.331 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:03:16.328+0000 7fd2cdfa4640 1 --2- 172.21.15.43:0/1638207531 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fd2c80cf370 0x7fd2c80b5de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T15:03:16.331 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:03:16.328+0000 7fd2cdfa4640 1 --2- 172.21.15.43:0/1638207531 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fd2c815c720 0x7fd2c80cd850 unknown :-1 s=CLOSED pgs=2086 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T15:03:16.332 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:03:16.328+0000 7fd2cdfa4640 1 -- 172.21.15.43:0/1638207531 >> 172.21.15.43:0/1638207531 conn(0x7fd2c806a2a0 msgr2=0x7fd2c806a680 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-09T15:03:16.332 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:03:16.328+0000 7fd2cdfa4640 1 -- 172.21.15.43:0/1638207531 shutdown_connections 2023-08-09T15:03:16.332 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:03:16.328+0000 7fd2cdfa4640 1 -- 172.21.15.43:0/1638207531 wait complete. 2023-08-09T15:03:16.332 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:03:16.329+0000 7fd2cdfa4640 1 Processor -- start 2023-08-09T15:03:16.334 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:03:16.330+0000 7fd2cdfa4640 1 -- start start 2023-08-09T15:03:16.334 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:03:16.331+0000 7fd2cdfa4640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fd2c80cf370 0x7fd2c80b2a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T15:03:16.334 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:03:16.331+0000 7fd2cdfa4640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fd2c80d0900 0x7fd2c80b35c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T15:03:16.335 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:03:16.331+0000 7fd2cdfa4640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fd2c815c720 0x7fd2c80b3b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T15:03:16.335 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:03:16.331+0000 7fd2cdfa4640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd2c80b33d0 con 0x7fd2c80d0900 2023-08-09T15:03:16.335 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:03:16.331+0000 7fd2cdfa4640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd2c815d710 con 0x7fd2c80cf370 2023-08-09T15:03:16.335 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:03:16.331+0000 7fd2cdfa4640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd2c815cde0 con 0x7fd2c815c720 2023-08-09T15:03:16.335 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:03:16.331+0000 7fd2c7fff640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fd2c80d0900 0x7fd2c80b35c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T15:03:16.335 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:03:16.331+0000 7fd2c7fff640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fd2c80d0900 0x7fd2c80b35c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:58108/0 (socket says 172.21.15.43:58108) 2023-08-09T15:03:16.335 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:03:16.331+0000 7fd2c7fff640 1 -- 172.21.15.43:0/1658661050 learned_addr learned my addr 172.21.15.43:0/1658661050 (peer_addr_for_me v2:172.21.15.43:0/0) 2023-08-09T15:03:16.336 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:03:16.331+0000 7fd2ccfa2640 1 --2- 172.21.15.43:0/1658661050 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fd2c80cf370 0x7fd2c80b2a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T15:03:16.336 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:03:16.331+0000 7fd2cd7a3640 1 --2- 172.21.15.43:0/1658661050 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fd2c815c720 0x7fd2c80b3b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T15:03:16.336 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:03:16.331+0000 7fd2c7fff640 1 -- 172.21.15.43:0/1658661050 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fd2c815c720 msgr2=0x7fd2c80b3b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T15:03:16.336 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:03:16.331+0000 7fd2c7fff640 1 --2- 172.21.15.43:0/1658661050 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fd2c815c720 0x7fd2c80b3b00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T15:03:16.336 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:03:16.331+0000 7fd2c7fff640 1 -- 172.21.15.43:0/1658661050 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fd2c80cf370 msgr2=0x7fd2c80b2a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T15:03:16.336 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:03:16.331+0000 7fd2c7fff640 1 --2- 172.21.15.43:0/1658661050 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fd2c80cf370 0x7fd2c80b2a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T15:03:16.336 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:03:16.331+0000 7fd2c7fff640 1 -- 172.21.15.43:0/1658661050 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd2b80094d0 con 0x7fd2c80d0900 2023-08-09T15:03:16.337 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:03:16.332+0000 7fd2c7fff640 1 --2- 172.21.15.43:0/1658661050 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fd2c80d0900 0x7fd2c80b35c0 secure :-1 s=READY pgs=2087 cs=0 l=1 rev1=1 crypto rx=0x7fd2bc00c4b0 tx=0x7fd2bc00c970 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-09T15:03:16.337 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:03:16.332+0000 7fd2c57fa640 1 -- 172.21.15.43:0/1658661050 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd2bc005480 con 0x7fd2c80d0900 2023-08-09T15:03:16.337 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:03:16.332+0000 7fd2c57fa640 1 -- 172.21.15.43:0/1658661050 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd2bc0035f0 con 0x7fd2c80d0900 2023-08-09T15:03:16.337 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:03:16.332+0000 7fd2c57fa640 1 -- 172.21.15.43:0/1658661050 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd2bc0039e0 con 0x7fd2c80d0900 2023-08-09T15:03:16.337 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:03:16.332+0000 7fd2cdfa4640 1 -- 172.21.15.43:0/1658661050 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd2c815d0c0 con 0x7fd2c80d0900 2023-08-09T15:03:16.337 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:03:16.332+0000 7fd2cdfa4640 1 -- 172.21.15.43:0/1658661050 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd2c80b2fc0 con 0x7fd2c80d0900 2023-08-09T15:03:16.338 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:03:16.333+0000 7fd2cdfa4640 1 -- 172.21.15.43:0/1658661050 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd2c815d880 con 0x7fd2c80d0900 2023-08-09T15:03:16.339 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:03:16.335+0000 7fd2c57fa640 1 -- 172.21.15.43:0/1658661050 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 95) v1 ==== 88472+0+0 (secure 0 0 0) 0x7fd2bc003b40 con 0x7fd2c80d0900 2023-08-09T15:03:16.339 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:03:16.335+0000 7fd2c57fa640 1 --2- 172.21.15.43:0/1658661050 >> [v2:172.21.15.203:6832/40883597,v1:172.21.15.203:6833/40883597] conn(0x7fd2b406a1c0 0x7fd2b406c680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T15:03:16.339 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:03:16.336+0000 7fd2c57fa640 1 -- 172.21.15.43:0/1658661050 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(187..187 src has 1..187) v4 ==== 6770+0+0 (secure 0 0 0) 0x7fd2bc088010 con 0x7fd2c80d0900 2023-08-09T15:03:16.339 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:03:16.336+0000 7fd2ccfa2640 1 --2- 172.21.15.43:0/1658661050 >> [v2:172.21.15.203:6832/40883597,v1:172.21.15.203:6833/40883597] conn(0x7fd2b406a1c0 0x7fd2b406c680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T15:03:16.340 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:03:16.337+0000 7fd2ccfa2640 1 --2- 172.21.15.43:0/1658661050 >> [v2:172.21.15.203:6832/40883597,v1:172.21.15.203:6833/40883597] conn(0x7fd2b406a1c0 0x7fd2b406c680 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7fd2b8036000 tx=0x7fd2b803b040 comp rx=0 tx=0).ready entity=mgr.8347 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-09T15:03:16.342 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:03:16.341+0000 7fd2c57fa640 1 -- 172.21.15.43:0/1658661050 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178066 (secure 0 0 0) 0x7fd2bc053320 con 0x7fd2c80d0900 2023-08-09T15:03:16.556 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:03:16.555+0000 7fd2cdfa4640 1 -- 172.21.15.43:0/1658661050 --> [v2:172.21.15.203:6832/40883597,v1:172.21.15.203:6833/40883597] -- mgr_command(tid 0: {"prefix": "fs snap-schedule status", "fs": "cephfs", "path": "/snap_test_dir1/test_restart", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7fd2c815cf50 con 0x7fd2b406a1c0 2023-08-09T15:03:16.574 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:03:16.573+0000 7fd2c57fa640 1 -- 172.21.15.43:0/1658661050 <== mgr.8347 v2:172.21.15.203:6832/40883597 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+360 (secure 0 0 0) 0x7fd2c815cf50 con 0x7fd2b406a1c0 2023-08-09T15:03:16.574 INFO:teuthology.orchestra.run.smithi043.stdout: 2023-08-09T15:03:16.574 INFO:teuthology.orchestra.run.smithi043.stdout:[{"fs": "cephfs", "subvol": null, "path": "/snap_test_dir1/test_restart", "rel_path": "/snap_test_dir1/test_restart", "schedule": "1M", "retention": {}, "start": "2023-08-09T00:00:00", "created": "2023-08-09T14:52:46", "first": "2023-08-09T14:53:00", "last": "2023-08-09T15:03:00", "last_pruned": null, "created_count": 11, "pruned_count": 0, "active": true}] 2023-08-09T15:03:16.576 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:03:16.575+0000 7fd2cdfa4640 1 -- 172.21.15.43:0/1658661050 >> [v2:172.21.15.203:6832/40883597,v1:172.21.15.203:6833/40883597] conn(0x7fd2b406a1c0 msgr2=0x7fd2b406c680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T15:03:16.576 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:03:16.575+0000 7fd2cdfa4640 1 --2- 172.21.15.43:0/1658661050 >> [v2:172.21.15.203:6832/40883597,v1:172.21.15.203:6833/40883597] conn(0x7fd2b406a1c0 0x7fd2b406c680 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7fd2b8036000 tx=0x7fd2b803b040 comp rx=0 tx=0).stop 2023-08-09T15:03:16.577 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:03:16.575+0000 7fd2cdfa4640 1 -- 172.21.15.43:0/1658661050 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fd2c80d0900 msgr2=0x7fd2c80b35c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T15:03:16.577 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:03:16.575+0000 7fd2cdfa4640 1 --2- 172.21.15.43:0/1658661050 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fd2c80d0900 0x7fd2c80b35c0 secure :-1 s=READY pgs=2087 cs=0 l=1 rev1=1 crypto rx=0x7fd2bc00c4b0 tx=0x7fd2bc00c970 comp rx=0 tx=0).stop 2023-08-09T15:03:16.577 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:03:16.575+0000 7fd2cdfa4640 1 -- 172.21.15.43:0/1658661050 shutdown_connections 2023-08-09T15:03:16.577 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:03:16.575+0000 7fd2cdfa4640 1 --2- 172.21.15.43:0/1658661050 >> [v2:172.21.15.203:6832/40883597,v1:172.21.15.203:6833/40883597] conn(0x7fd2b406a1c0 0x7fd2b406c680 unknown :-1 s=CLOSED pgs=25 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T15:03:16.577 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:03:16.575+0000 7fd2cdfa4640 1 --2- 172.21.15.43:0/1658661050 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fd2c815c720 0x7fd2c80b3b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T15:03:16.577 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:03:16.575+0000 7fd2cdfa4640 1 --2- 172.21.15.43:0/1658661050 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fd2c80d0900 0x7fd2c80b35c0 unknown :-1 s=CLOSED pgs=2087 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T15:03:16.577 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:03:16.575+0000 7fd2cdfa4640 1 --2- 172.21.15.43:0/1658661050 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fd2c80cf370 0x7fd2c80b2a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T15:03:16.578 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:03:16.575+0000 7fd2cdfa4640 1 -- 172.21.15.43:0/1658661050 >> 172.21.15.43:0/1658661050 conn(0x7fd2c806a2a0 msgr2=0x7fd2c80d2b90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-09T15:03:16.578 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:03:16.575+0000 7fd2cdfa4640 1 -- 172.21.15.43:0/1658661050 shutdown_connections 2023-08-09T15:03:16.578 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:03:16.575+0000 7fd2cdfa4640 1 -- 172.21.15.43:0/1658661050 wait complete. 2023-08-09T15:03:16.595 DEBUG:tasks.cephfs.test_snap_schedules:res= [{"fs": "cephfs", "subvol": null, "path": "/snap_test_dir1/test_restart", "rel_path": "/snap_test_dir1/test_restart", "schedule": "1M", "retention": {}, "start": "2023-08-09T00:00:00", "created": "2023-08-09T14:52:46", "first": "2023-08-09T14:53:00", "last": "2023-08-09T15:03:00", "last_pruned": null, "created_count": 11, "pruned_count": 0, "active": true}] 2023-08-09T15:03:16.595 DEBUG:tasks.cephfs.test_snap_schedules:json_res: {'fs': 'cephfs', 'subvol': None, 'path': '/snap_test_dir1/test_restart', 'rel_path': '/snap_test_dir1/test_restart', 'schedule': '1M', 'retention': {}, 'start': '2023-08-09T00:00:00', 'created': '2023-08-09T14:52:46', 'first': '2023-08-09T14:53:00', 'last': '2023-08-09T15:03:00', 'last_pruned': None, 'created_count': 11, 'pruned_count': 0, 'active': True} 2023-08-09T15:03:16.596 DEBUG:tasks.cephfs.test_snap_schedules:fs_count: 11 2023-08-09T15:03:16.596 DEBUG:tasks.cephfs.test_snap_schedules:db_count: 11 2023-08-09T15:03:16.597 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph mgr dump --format=json 2023-08-09T15:03:16.823 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:03:16.822+0000 7f637b322640 1 Processor -- start 2023-08-09T15:03:16.824 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:03:16.822+0000 7f637b322640 1 -- start start 2023-08-09T15:03:16.826 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:03:16.824+0000 7f637b322640 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7f637415f7b0 0x7f63740b4760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T15:03:16.827 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:03:16.824+0000 7f637b322640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f63740cd6b0 0x7f63740d0e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T15:03:16.827 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:03:16.824+0000 7f637b322640 1 -- --> v1:172.21.15.203:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f63740b7bf0 con 0x7f63740b1050 2023-08-09T15:03:16.827 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:03:16.824+0000 7f637b322640 1 -- --> v2:172.21.15.43:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f63740b7ed0 con 0x7f63740cd6b0 2023-08-09T15:03:16.827 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:03:16.824+0000 7f637b322640 1 -- --> v2:172.21.15.203:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f63740d0370 con 0x7f637415f7b0 2023-08-09T15:03:16.828 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:03:16.824+0000 7f637ab21640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f63740cd6b0 0x7f63740d0e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T15:03:16.828 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:03:16.824+0000 7f637ab21640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f63740cd6b0 0x7f63740d0e50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:58122/0 (socket says 172.21.15.43:58122) 2023-08-09T15:03:16.828 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:03:16.824+0000 7f637ab21640 1 -- 172.21.15.43:0/344198908 learned_addr learned my addr 172.21.15.43:0/344198908 (peer_addr_for_me v2:172.21.15.43:0/0) 2023-08-09T15:03:16.829 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:03:16.825+0000 7f637a320640 1 --2- 172.21.15.43:0/344198908 >> v2:172.21.15.203:3300/0 conn(0x7f637415f7b0 0x7f63740b4760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T15:03:16.829 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:03:16.825+0000 7f637ab21640 1 -- 172.21.15.43:0/344198908 >> v2:172.21.15.203:3300/0 conn(0x7f637415f7b0 msgr2=0x7f63740b4760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-09T15:03:16.829 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:03:16.825+0000 7f637ab21640 1 --2- 172.21.15.43:0/344198908 >> v2:172.21.15.203:3300/0 conn(0x7f637415f7b0 0x7f63740b4760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T15:03:16.829 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:03:16.825+0000 7f637ab21640 1 -- 172.21.15.43:0/344198908 >> v1:172.21.15.203:6789/0 conn(0x7f63740b1050 legacy=0x7f63740ccfa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-09T15:03:16.830 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:03:16.825+0000 7f637ab21640 1 -- 172.21.15.43:0/344198908 --> v2:172.21.15.43:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f63740d0650 con 0x7f63740cd6b0 2023-08-09T15:03:16.832 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:03:16.826+0000 7f637ab21640 1 --2- 172.21.15.43:0/344198908 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f63740cd6b0 0x7f63740d0e50 secure :-1 s=READY pgs=2088 cs=0 l=1 rev1=1 crypto rx=0x7f636c009e50 tx=0x7f636c02bee0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b7a126038b0b910f server_cookie=0 in_seq=0 out_seq=0 2023-08-09T15:03:16.832 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:03:16.826+0000 7f637931e640 1 -- 172.21.15.43:0/344198908 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f636c02ca80 con 0x7f63740cd6b0 2023-08-09T15:03:16.833 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:03:16.827+0000 7f637931e640 1 -- 172.21.15.43:0/344198908 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f636c02cbe0 con 0x7f63740cd6b0 2023-08-09T15:03:16.833 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:03:16.827+0000 7f637931e640 1 -- 172.21.15.43:0/344198908 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f636c0343f0 con 0x7f63740cd6b0 2023-08-09T15:03:16.833 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:03:16.827+0000 7f637b322640 1 -- 172.21.15.43:0/344198908 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f63740cd6b0 msgr2=0x7f63740d0e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T15:03:16.834 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:03:16.827+0000 7f637b322640 1 --2- 172.21.15.43:0/344198908 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f63740cd6b0 0x7f63740d0e50 secure :-1 s=READY pgs=2088 cs=0 l=1 rev1=1 crypto rx=0x7f636c009e50 tx=0x7f636c02bee0 comp rx=0 tx=0).stop 2023-08-09T15:03:16.834 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:03:16.827+0000 7f637b322640 1 -- 172.21.15.43:0/344198908 shutdown_connections 2023-08-09T15:03:16.834 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:03:16.827+0000 7f637b322640 1 --2- 172.21.15.43:0/344198908 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f63740cd6b0 0x7f63740d0e50 unknown :-1 s=CLOSED pgs=2088 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T15:03:16.834 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:03:16.827+0000 7f637b322640 1 --2- 172.21.15.43:0/344198908 >> v2:172.21.15.203:3300/0 conn(0x7f637415f7b0 0x7f63740b4760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T15:03:16.834 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:03:16.827+0000 7f637b322640 1 -- 172.21.15.43:0/344198908 >> 172.21.15.43:0/344198908 conn(0x7f637406a2b0 msgr2=0x7f637406a690 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-09T15:03:16.834 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:03:16.827+0000 7f637b322640 1 -- 172.21.15.43:0/344198908 shutdown_connections 2023-08-09T15:03:16.835 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:03:16.827+0000 7f637b322640 1 -- 172.21.15.43:0/344198908 wait complete. 2023-08-09T15:03:16.835 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:03:16.829+0000 7f637b322640 1 Processor -- start 2023-08-09T15:03:16.835 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:03:16.829+0000 7f637b322640 1 -- start start 2023-08-09T15:03:16.835 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:03:16.830+0000 7f637b322640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f63740b1050 0x7f63741549f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T15:03:16.835 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:03:16.830+0000 7f637b322640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f63740cd6b0 0x7f6374152a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T15:03:16.835 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:03:16.830+0000 7f637b322640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f637415f7b0 0x7f63740b3100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T15:03:16.835 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:03:16.830+0000 7f637b322640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6374152f40 con 0x7f637415f7b0 2023-08-09T15:03:16.836 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:03:16.830+0000 7f637b322640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f63741f4ee0 con 0x7f63740b1050 2023-08-09T15:03:16.836 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:03:16.830+0000 7f637b322640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f63741f51c0 con 0x7f63740cd6b0 2023-08-09T15:03:16.836 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:03:16.831+0000 7f637ab21640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f637415f7b0 0x7f63740b3100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T15:03:16.836 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:03:16.831+0000 7f637ab21640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f637415f7b0 0x7f63740b3100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:58136/0 (socket says 172.21.15.43:58136) 2023-08-09T15:03:16.836 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:03:16.831+0000 7f637ab21640 1 -- 172.21.15.43:0/2445644513 learned_addr learned my addr 172.21.15.43:0/2445644513 (peer_addr_for_me v2:172.21.15.43:0/0) 2023-08-09T15:03:16.836 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:03:16.831+0000 7f637a320640 1 --2- 172.21.15.43:0/2445644513 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f63740b1050 0x7f63741549f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T15:03:16.836 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:03:16.831+0000 7f6379b1f640 1 --2- 172.21.15.43:0/2445644513 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f63740cd6b0 0x7f6374152a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T15:03:16.838 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:03:16.832+0000 7f637ab21640 1 -- 172.21.15.43:0/2445644513 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f63740cd6b0 msgr2=0x7f6374152a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T15:03:16.838 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:03:16.832+0000 7f637ab21640 1 --2- 172.21.15.43:0/2445644513 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f63740cd6b0 0x7f6374152a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T15:03:16.838 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:03:16.832+0000 7f637ab21640 1 -- 172.21.15.43:0/2445644513 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f63740b1050 msgr2=0x7f63741549f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T15:03:16.838 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:03:16.832+0000 7f637ab21640 1 --2- 172.21.15.43:0/2445644513 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f63740b1050 0x7f63741549f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T15:03:16.839 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:03:16.832+0000 7f637ab21640 1 -- 172.21.15.43:0/2445644513 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f636c009b30 con 0x7f637415f7b0 2023-08-09T15:03:16.839 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:03:16.832+0000 7f637ab21640 1 --2- 172.21.15.43:0/2445644513 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f637415f7b0 0x7f63740b3100 secure :-1 s=READY pgs=2089 cs=0 l=1 rev1=1 crypto rx=0x7f636c009e50 tx=0x7f636c004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-09T15:03:16.839 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:03:16.832+0000 7f636affd640 1 -- 172.21.15.43:0/2445644513 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f636c009510 con 0x7f637415f7b0 2023-08-09T15:03:16.839 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:03:16.832+0000 7f636affd640 1 -- 172.21.15.43:0/2445644513 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f636c03e450 con 0x7f637415f7b0 2023-08-09T15:03:16.839 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:03:16.832+0000 7f637b322640 1 -- 172.21.15.43:0/2445644513 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f63741f54a0 con 0x7f637415f7b0 2023-08-09T15:03:16.839 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:03:16.832+0000 7f636affd640 1 -- 172.21.15.43:0/2445644513 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f636c03e7e0 con 0x7f637415f7b0 2023-08-09T15:03:16.839 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:03:16.833+0000 7f637b322640 1 -- 172.21.15.43:0/2445644513 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f63741f5a60 con 0x7f637415f7b0 2023-08-09T15:03:16.840 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:03:16.834+0000 7f637b322640 1 -- 172.21.15.43:0/2445644513 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f63741f5c50 con 0x7f637415f7b0 2023-08-09T15:03:16.840 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:03:16.835+0000 7f636affd640 1 -- 172.21.15.43:0/2445644513 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 95) v1 ==== 88472+0+0 (secure 0 0 0) 0x7f636c02cd50 con 0x7f637415f7b0 2023-08-09T15:03:16.840 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:03:16.836+0000 7f636affd640 1 --2- 172.21.15.43:0/2445644513 >> [v2:172.21.15.203:6832/40883597,v1:172.21.15.203:6833/40883597] conn(0x7f635806a1c0 0x7f635806c680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T15:03:16.840 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:03:16.836+0000 7f636affd640 1 -- 172.21.15.43:0/2445644513 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(187..187 src has 1..187) v4 ==== 6770+0+0 (secure 0 0 0) 0x7f636c0ab4e0 con 0x7f637415f7b0 2023-08-09T15:03:16.840 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:03:16.836+0000 7f637a320640 1 --2- 172.21.15.43:0/2445644513 >> [v2:172.21.15.203:6832/40883597,v1:172.21.15.203:6833/40883597] conn(0x7f635806a1c0 0x7f635806c680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T15:03:16.841 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:03:16.837+0000 7f637a320640 1 --2- 172.21.15.43:0/2445644513 >> [v2:172.21.15.203:6832/40883597,v1:172.21.15.203:6833/40883597] conn(0x7f635806a1c0 0x7f635806c680 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7f6370006fd0 tx=0x7f6370008040 comp rx=0 tx=0).ready entity=mgr.8347 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-09T15:03:16.841 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:03:16.840+0000 7f636affd640 1 -- 172.21.15.43:0/2445644513 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178066 (secure 0 0 0) 0x7f636c076900 con 0x7f637415f7b0 2023-08-09T15:03:17.083 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:03:17.082+0000 7f637b322640 1 -- 172.21.15.43:0/2445644513 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "mgr dump", "format": "json"} v 0) v1 -- 0x7f63740739f0 con 0x7f637415f7b0 2023-08-09T15:03:17.095 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:03:17.093+0000 7f636affd640 1 -- 172.21.15.43:0/2445644513 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "mgr dump", "format": "json"}]=0 v95) v1 ==== 74+0+169698 (secure 0 0 0) 0x7f636c079f20 con 0x7f637415f7b0 2023-08-09T15:03:17.099 INFO:teuthology.orchestra.run.smithi043.stdout: 2023-08-09T15:03:17.100 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:03:17.099+0000 7f637b322640 1 -- 172.21.15.43:0/2445644513 >> [v2:172.21.15.203:6832/40883597,v1:172.21.15.203:6833/40883597] conn(0x7f635806a1c0 msgr2=0x7f635806c680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T15:03:17.100 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:03:17.099+0000 7f637b322640 1 --2- 172.21.15.43:0/2445644513 >> [v2:172.21.15.203:6832/40883597,v1:172.21.15.203:6833/40883597] conn(0x7f635806a1c0 0x7f635806c680 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7f6370006fd0 tx=0x7f6370008040 comp rx=0 tx=0).stop 2023-08-09T15:03:17.100 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:03:17.099+0000 7f637b322640 1 -- 172.21.15.43:0/2445644513 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f637415f7b0 msgr2=0x7f63740b3100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T15:03:17.101 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:03:17.099+0000 7f637b322640 1 --2- 172.21.15.43:0/2445644513 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f637415f7b0 0x7f63740b3100 secure :-1 s=READY pgs=2089 cs=0 l=1 rev1=1 crypto rx=0x7f636c009e50 tx=0x7f636c004a40 comp rx=0 tx=0).stop 2023-08-09T15:03:17.101 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:03:17.099+0000 7f637b322640 1 -- 172.21.15.43:0/2445644513 shutdown_connections 2023-08-09T15:03:17.102 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:03:17.099+0000 7f637b322640 1 --2- 172.21.15.43:0/2445644513 >> [v2:172.21.15.203:6832/40883597,v1:172.21.15.203:6833/40883597] conn(0x7f635806a1c0 0x7f635806c680 unknown :-1 s=CLOSED pgs=26 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T15:03:17.102 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:03:17.099+0000 7f637b322640 1 --2- 172.21.15.43:0/2445644513 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f637415f7b0 0x7f63740b3100 unknown :-1 s=CLOSED pgs=2089 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T15:03:17.102 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:03:17.099+0000 7f637b322640 1 --2- 172.21.15.43:0/2445644513 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f63740cd6b0 0x7f6374152a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T15:03:17.102 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:03:17.099+0000 7f637b322640 1 --2- 172.21.15.43:0/2445644513 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f63740b1050 0x7f63741549f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T15:03:17.102 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:03:17.099+0000 7f637b322640 1 -- 172.21.15.43:0/2445644513 >> 172.21.15.43:0/2445644513 conn(0x7f637406a2b0 msgr2=0x7f637415b790 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-09T15:03:17.103 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:03:17.099+0000 7f637b322640 1 -- 172.21.15.43:0/2445644513 shutdown_connections 2023-08-09T15:03:17.103 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:03:17.099+0000 7f637b322640 1 -- 172.21.15.43:0/2445644513 wait complete. 2023-08-09T15:03:17.119 INFO:teuthology.orchestra.run.smithi043.stdout:{"epoch":95,"active_gid":8347,"active_name":"x","active_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6832","nonce":40883597},{"type":"v1","addr":"172.21.15.203:6833","nonce":40883597}]},"active_addr":"172.21.15.203:6833/40883597","active_change":"2023-08-09T14:52:45.189016+0000","active_mgr_features":4540138322906710015,"available":true,"standbys":[{"gid":10389,"name":"y","mgr_features":4540138322906710015,"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2359","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"upmap","min":"","max":"","enum_allowed":["crush-compat","none","upmap"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"cephadm","can_run":true,"error_string":"","module_options":{"agent_down_multiplier":{"name":"agent_down_multiplier","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"","max":"","enum_allowed":[],"desc":"Multiplied by agent refresh rate to calculate how long agent must not report before being marked down","long_desc":"","tags":[],"see_also":[]},"agent_refresh_rate":{"name":"agent_refresh_rate","type":"secs","level":"advanced","flags":0,"default_value":"20","min":"","max":"","enum_allowed":[],"desc":"How often agent on each host will try to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"agent_starting_port":{"name":"agent_starting_port","type":"int","level":"advanced","flags":0,"default_value":"4721","min":"","max":"","enum_allowed":[],"desc":"First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)","long_desc":"","tags":[],"see_also":[]},"allow_ptrace":{"name":"allow_ptrace","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow SYS_PTRACE capability on ceph containers","long_desc":"The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.","tags":[],"see_also":[]},"autotune_interval":{"name":"autotune_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to autotune daemon memory","long_desc":"","tags":[],"see_also":[]},"autotune_memory_target_ratio":{"name":"autotune_memory_target_ratio","type":"float","level":"advanced","flags":0,"default_value":"0.7","min":"","max":"","enum_allowed":[],"desc":"ratio of total system memory to divide amongst autotuned daemons","long_desc":"","tags":[],"see_also":[]},"cgroups_split":{"name":"cgroups_split","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Pass --cgroups=split when cephadm creates containers (currently podman only)","long_desc":"","tags":[],"see_also":[]},"config_checks_enabled":{"name":"config_checks_enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable or disable the cephadm configuration analysis","long_desc":"","tags":[],"see_also":[]},"config_dashboard":{"name":"config_dashboard","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"manage configs like API endpoints in Dashboard.","long_desc":"","tags":[],"see_also":[]},"container_image_alertmanager":{"name":"container_image_alertmanager","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/alertmanager:v0.25.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_base":{"name":"container_image_base","type":"str","level":"advanced","flags":1,"default_value":"quay.io/ceph/ceph","min":"","max":"","enum_allowed":[],"desc":"Container image name, without the tag","long_desc":"","tags":[],"see_also":[]},"container_image_elasticsearch":{"name":"container_image_elasticsearch","type":"str","level":"advanced","flags":0,"default_value":"quay.io/omrizeneva/elasticsearch:6.8.23","min":"","max":"","enum_allowed":[],"desc":"elasticsearch container image","long_desc":"","tags":[],"see_also":[]},"container_image_grafana":{"name":"container_image_grafana","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/ceph-grafana:9.4.7","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_haproxy":{"name":"container_image_haproxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/haproxy:2.3","min":"","max":"","enum_allowed":[],"desc":"HAproxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_agent":{"name":"container_image_jaeger_agent","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-agent:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger agent container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_collector":{"name":"container_image_jaeger_collector","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-collector:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger collector container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_query":{"name":"container_image_jaeger_query","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-query:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger query container image","long_desc":"","tags":[],"see_also":[]},"container_image_keepalived":{"name":"container_image_keepalived","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/keepalived:2.2.4","min":"","max":"","enum_allowed":[],"desc":"Keepalived container image","long_desc":"","tags":[],"see_also":[]},"container_image_loki":{"name":"container_image_loki","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/loki:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Loki container image","long_desc":"","tags":[],"see_also":[]},"container_image_node_exporter":{"name":"container_image_node_exporter","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/node-exporter:v1.5.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_nvmeof":{"name":"container_image_nvmeof","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/nvmeof:0.0.1","min":"","max":"","enum_allowed":[],"desc":"Nvme-of container image","long_desc":"","tags":[],"see_also":[]},"container_image_prometheus":{"name":"container_image_prometheus","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/prometheus:v2.43.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_promtail":{"name":"container_image_promtail","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/promtail:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Promtail container image","long_desc":"","tags":[],"see_also":[]},"container_image_snmp_gateway":{"name":"container_image_snmp_gateway","type":"str","level":"advanced","flags":0,"default_value":"docker.io/maxwo/snmp-notifier:v1.2.1","min":"","max":"","enum_allowed":[],"desc":"SNMP Gateway container image","long_desc":"","tags":[],"see_also":[]},"container_init":{"name":"container_init","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Run podman/docker with `--init`","long_desc":"","tags":[],"see_also":[]},"daemon_cache_timeout":{"name":"daemon_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"seconds to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"default_cephadm_command_timeout":{"name":"default_cephadm_command_timeout","type":"secs","level":"advanced","flags":0,"default_value":"900","min":"","max":"","enum_allowed":[],"desc":"Default timeout applied to cephadm commands run directly on the host (in seconds)","long_desc":"","tags":[],"see_also":[]},"default_registry":{"name":"default_registry","type":"str","level":"advanced","flags":0,"default_value":"docker.io","min":"","max":"","enum_allowed":[],"desc":"Search-registry to which we should normalize unqualified image names. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"device_cache_timeout":{"name":"device_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"seconds to cache device inventory","long_desc":"","tags":[],"see_also":[]},"device_enhanced_scan":{"name":"device_enhanced_scan","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use libstoragemgmt during device scans","long_desc":"","tags":[],"see_also":[]},"facts_cache_timeout":{"name":"facts_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"seconds to cache host facts data","long_desc":"","tags":[],"see_also":[]},"host_check_interval":{"name":"host_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to perform a host check","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_refresh_metadata":{"name":"log_refresh_metadata","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"log to the \"cephadm\" cluster log channel\"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf":{"name":"manage_etc_ceph_ceph_conf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Manage and own /etc/ceph/ceph.conf on the hosts.","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf_hosts":{"name":"manage_etc_ceph_ceph_conf_hosts","type":"str","level":"advanced","flags":0,"default_value":"*","min":"","max":"","enum_allowed":[],"desc":"PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf","long_desc":"","tags":[],"see_also":[]},"max_count_per_host":{"name":"max_count_per_host","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of daemons per service per host","long_desc":"","tags":[],"see_also":[]},"max_osd_draining_count":{"name":"max_osd_draining_count","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of osds that will be drained simultaneously when osds are removed","long_desc":"","tags":[],"see_also":[]},"migration_current":{"name":"migration_current","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"internal - do not modify","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["cephadm-package","root"],"desc":"mode for remote execution of cephadm","long_desc":"","tags":[],"see_also":[]},"prometheus_alerts_path":{"name":"prometheus_alerts_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/prometheus/ceph/ceph_default_alerts.yml","min":"","max":"","enum_allowed":[],"desc":"location of alerts to include in prometheus deployments","long_desc":"","tags":[],"see_also":[]},"registry_insecure":{"name":"registry_insecure","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Registry is to be considered insecure (no TLS available). Only for development purposes.","long_desc":"","tags":[],"see_also":[]},"registry_password":{"name":"registry_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository password. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"registry_url":{"name":"registry_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Registry url for login purposes. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"registry_username":{"name":"registry_username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository username. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"secure_monitoring_stack":{"name":"secure_monitoring_stack","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable TLS security for all the monitoring stack daemons","long_desc":"","tags":[],"see_also":[]},"service_discovery_port":{"name":"service_discovery_port","type":"int","level":"advanced","flags":0,"default_value":"8765","min":"","max":"","enum_allowed":[],"desc":"cephadm service discovery port","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"customized SSH config file to connect to managed hosts","long_desc":"","tags":[],"see_also":[]},"use_agent":{"name":"use_agent","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use cephadm agent on each host to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"use_repo_digest":{"name":"use_repo_digest","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Automatically convert image tags to image digest. Make sure all daemons use the same image","long_desc":"","tags":[],"see_also":[]},"warn_on_failed_host_check":{"name":"warn_on_failed_host_check","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if the host check fails","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_daemons":{"name":"warn_on_stray_daemons","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected that are not managed by cephadm","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_hosts":{"name":"warn_on_stray_hosts","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected on a host that is not managed by cephadm","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ACCOUNT_LOCKOUT_ATTEMPTS":{"name":"ACCOUNT_LOCKOUT_ATTEMPTS","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_SSL_VERIFY":{"name":"ALERTMANAGER_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_CEPHFS":{"name":"FEATURE_TOGGLE_CEPHFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_DASHBOARD":{"name":"FEATURE_TOGGLE_DASHBOARD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_ISCSI":{"name":"FEATURE_TOGGLE_ISCSI","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_MIRRORING":{"name":"FEATURE_TOGGLE_MIRRORING","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_NFS":{"name":"FEATURE_TOGGLE_NFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RBD":{"name":"FEATURE_TOGGLE_RBD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RGW":{"name":"FEATURE_TOGGLE_RGW","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_FRONTEND_API_URL":{"name":"GRAFANA_FRONTEND_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISSUE_TRACKER_API_KEY":{"name":"ISSUE_TRACKER_API_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_SSL_VERIFY":{"name":"PROMETHEUS_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"cross_origin_url":{"name":"cross_origin_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"motd":{"name":"motd","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"The message of the day","long_desc":"","tags":[],"see_also":[]},"redirect_resolve_ip_addr":{"name":"redirect_resolve_ip_addr","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"diskprediction_local","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predict_interval":{"name":"predict_interval","type":"str","level":"advanced","flags":0,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predictor_model":{"name":"predictor_model","type":"str","level":"advanced","flags":0,"default_value":"prophetstor","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"int","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"How big batches of data points should be when sending to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server hostname","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"30","min":"5","max":"","enum_allowed":[],"desc":"Time between reports to InfluxDB. Default 30 seconds.","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"password of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"int","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server port","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"Use https connection for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"int","level":"advanced","flags":0,"default_value":"5","min":"1","max":"32","enum_allowed":[],"desc":"How many worker threads should be spawned for sending data to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"username of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"Verify https cert for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"what failure domain we should separate data replicas across.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"value to set min_size to (unchanged from Ceph's default if this option is not set)","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"which CRUSH subtree type the module should create a pool for.","tags":[],"see_also":[]}}},{"name":"mds_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"mirroring","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"nfs","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"fail_fs":{"name":"fail_fs","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Fail filesystem for rapid multi-rank mds upgrade","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_perf_query","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"noautoscale":{"name":"noautoscale","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"global autoscale flag","long_desc":"Option to turn on/off the autoscaler for all pools","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threshold":{"name":"threshold","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"1.0","max":"","enum_allowed":[],"desc":"scaling threshold","long_desc":"The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"allow_pg_recovery_event":{"name":"allow_pg_recovery_event","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow the module to show pg recovery progress","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how long the module is going to sleep","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"cache":{"name":"cache","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"exclude_perf_counters":{"name":"exclude_perf_counters","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Do not include perf-counters in the metrics output","long_desc":"Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"float","level":"advanced","flags":0,"default_value":"15.0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"the IPv4 or IPv6 address on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":1,"default_value":"9283","min":"","max":"","enum_allowed":[],"desc":"the port on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"log","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":1,"default_value":"default","min":"","max":"","enum_allowed":["default","error"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":1,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"restful","can_run":true,"error_string":"","module_options":{"enable_auth":{"name":"enable_auth","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rgw","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rook","can_run":true,"error_string":"","module_options":{"drive_group_interval":{"name":"drive_group_interval","type":"float","level":"advanced","flags":0,"default_value":"300.0","min":"","max":"","enum_allowed":[],"desc":"interval in seconds between re-application of applied drive_groups","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"storage_class":{"name":"storage_class","type":"str","level":"advanced","flags":0,"default_value":"local","min":"","max":"","enum_allowed":[],"desc":"storage class name for LSO-discovered PVs","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption7":{"name":"rwoption7","type":"int","level":"advanced","flags":0,"default_value":"","min":"1","max":"42","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"snap_schedule","can_run":true,"error_string":"","module_options":{"allow_m_granularity":{"name":"allow_m_granularity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow minute scheduled snapshots","long_desc":"","tags":[],"see_also":[]},"dump_on_update":{"name":"dump_on_update","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"dump database to debug log on update","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"stats","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share metadata about Ceph daemon crashes (version, stack straces, etc)","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share a user-provided description and/or contact email for the cluster","long_desc":"","tags":[],"see_also":[]},"channel_perf":{"name":"channel_perf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share various performance metrics of a cluster","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard_description":{"name":"leaderboard_description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_delay":{"name":"snapshot_clone_delay","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"Delay clone begin operation by snapshot_clone_delay seconds","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"discovery_interval":{"name":"discovery_interval","type":"uint","level":"advanced","flags":0,"default_value":"100","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}]}],"modules":["iostat","nfs","restful","snap_schedule"],"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2359","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"upmap","min":"","max":"","enum_allowed":["crush-compat","none","upmap"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"cephadm","can_run":true,"error_string":"","module_options":{"agent_down_multiplier":{"name":"agent_down_multiplier","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"","max":"","enum_allowed":[],"desc":"Multiplied by agent refresh rate to calculate how long agent must not report before being marked down","long_desc":"","tags":[],"see_also":[]},"agent_refresh_rate":{"name":"agent_refresh_rate","type":"secs","level":"advanced","flags":0,"default_value":"20","min":"","max":"","enum_allowed":[],"desc":"How often agent on each host will try to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"agent_starting_port":{"name":"agent_starting_port","type":"int","level":"advanced","flags":0,"default_value":"4721","min":"","max":"","enum_allowed":[],"desc":"First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)","long_desc":"","tags":[],"see_also":[]},"allow_ptrace":{"name":"allow_ptrace","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow SYS_PTRACE capability on ceph containers","long_desc":"The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.","tags":[],"see_also":[]},"autotune_interval":{"name":"autotune_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to autotune daemon memory","long_desc":"","tags":[],"see_also":[]},"autotune_memory_target_ratio":{"name":"autotune_memory_target_ratio","type":"float","level":"advanced","flags":0,"default_value":"0.7","min":"","max":"","enum_allowed":[],"desc":"ratio of total system memory to divide amongst autotuned daemons","long_desc":"","tags":[],"see_also":[]},"cgroups_split":{"name":"cgroups_split","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Pass --cgroups=split when cephadm creates containers (currently podman only)","long_desc":"","tags":[],"see_also":[]},"config_checks_enabled":{"name":"config_checks_enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable or disable the cephadm configuration analysis","long_desc":"","tags":[],"see_also":[]},"config_dashboard":{"name":"config_dashboard","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"manage configs like API endpoints in Dashboard.","long_desc":"","tags":[],"see_also":[]},"container_image_alertmanager":{"name":"container_image_alertmanager","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/alertmanager:v0.25.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_base":{"name":"container_image_base","type":"str","level":"advanced","flags":1,"default_value":"quay.io/ceph/ceph","min":"","max":"","enum_allowed":[],"desc":"Container image name, without the tag","long_desc":"","tags":[],"see_also":[]},"container_image_elasticsearch":{"name":"container_image_elasticsearch","type":"str","level":"advanced","flags":0,"default_value":"quay.io/omrizeneva/elasticsearch:6.8.23","min":"","max":"","enum_allowed":[],"desc":"elasticsearch container image","long_desc":"","tags":[],"see_also":[]},"container_image_grafana":{"name":"container_image_grafana","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/ceph-grafana:9.4.7","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_haproxy":{"name":"container_image_haproxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/haproxy:2.3","min":"","max":"","enum_allowed":[],"desc":"HAproxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_agent":{"name":"container_image_jaeger_agent","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-agent:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger agent container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_collector":{"name":"container_image_jaeger_collector","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-collector:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger collector container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_query":{"name":"container_image_jaeger_query","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-query:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger query container image","long_desc":"","tags":[],"see_also":[]},"container_image_keepalived":{"name":"container_image_keepalived","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/keepalived:2.2.4","min":"","max":"","enum_allowed":[],"desc":"Keepalived container image","long_desc":"","tags":[],"see_also":[]},"container_image_loki":{"name":"container_image_loki","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/loki:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Loki container image","long_desc":"","tags":[],"see_also":[]},"container_image_node_exporter":{"name":"container_image_node_exporter","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/node-exporter:v1.5.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_nvmeof":{"name":"container_image_nvmeof","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/nvmeof:0.0.1","min":"","max":"","enum_allowed":[],"desc":"Nvme-of container image","long_desc":"","tags":[],"see_also":[]},"container_image_prometheus":{"name":"container_image_prometheus","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/prometheus:v2.43.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_promtail":{"name":"container_image_promtail","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/promtail:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Promtail container image","long_desc":"","tags":[],"see_also":[]},"container_image_snmp_gateway":{"name":"container_image_snmp_gateway","type":"str","level":"advanced","flags":0,"default_value":"docker.io/maxwo/snmp-notifier:v1.2.1","min":"","max":"","enum_allowed":[],"desc":"SNMP Gateway container image","long_desc":"","tags":[],"see_also":[]},"container_init":{"name":"container_init","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Run podman/docker with `--init`","long_desc":"","tags":[],"see_also":[]},"daemon_cache_timeout":{"name":"daemon_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"seconds to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"default_cephadm_command_timeout":{"name":"default_cephadm_command_timeout","type":"secs","level":"advanced","flags":0,"default_value":"900","min":"","max":"","enum_allowed":[],"desc":"Default timeout applied to cephadm commands run directly on the host (in seconds)","long_desc":"","tags":[],"see_also":[]},"default_registry":{"name":"default_registry","type":"str","level":"advanced","flags":0,"default_value":"docker.io","min":"","max":"","enum_allowed":[],"desc":"Search-registry to which we should normalize unqualified image names. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"device_cache_timeout":{"name":"device_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"seconds to cache device inventory","long_desc":"","tags":[],"see_also":[]},"device_enhanced_scan":{"name":"device_enhanced_scan","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use libstoragemgmt during device scans","long_desc":"","tags":[],"see_also":[]},"facts_cache_timeout":{"name":"facts_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"seconds to cache host facts data","long_desc":"","tags":[],"see_also":[]},"host_check_interval":{"name":"host_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to perform a host check","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_refresh_metadata":{"name":"log_refresh_metadata","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"log to the \"cephadm\" cluster log channel\"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf":{"name":"manage_etc_ceph_ceph_conf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Manage and own /etc/ceph/ceph.conf on the hosts.","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf_hosts":{"name":"manage_etc_ceph_ceph_conf_hosts","type":"str","level":"advanced","flags":0,"default_value":"*","min":"","max":"","enum_allowed":[],"desc":"PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf","long_desc":"","tags":[],"see_also":[]},"max_count_per_host":{"name":"max_count_per_host","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of daemons per service per host","long_desc":"","tags":[],"see_also":[]},"max_osd_draining_count":{"name":"max_osd_draining_count","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of osds that will be drained simultaneously when osds are removed","long_desc":"","tags":[],"see_also":[]},"migration_current":{"name":"migration_current","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"internal - do not modify","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["cephadm-package","root"],"desc":"mode for remote execution of cephadm","long_desc":"","tags":[],"see_also":[]},"prometheus_alerts_path":{"name":"prometheus_alerts_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/prometheus/ceph/ceph_default_alerts.yml","min":"","max":"","enum_allowed":[],"desc":"location of alerts to include in prometheus deployments","long_desc":"","tags":[],"see_also":[]},"registry_insecure":{"name":"registry_insecure","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Registry is to be considered insecure (no TLS available). Only for development purposes.","long_desc":"","tags":[],"see_also":[]},"registry_password":{"name":"registry_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository password. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"registry_url":{"name":"registry_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Registry url for login purposes. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"registry_username":{"name":"registry_username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository username. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"secure_monitoring_stack":{"name":"secure_monitoring_stack","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable TLS security for all the monitoring stack daemons","long_desc":"","tags":[],"see_also":[]},"service_discovery_port":{"name":"service_discovery_port","type":"int","level":"advanced","flags":0,"default_value":"8765","min":"","max":"","enum_allowed":[],"desc":"cephadm service discovery port","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"customized SSH config file to connect to managed hosts","long_desc":"","tags":[],"see_also":[]},"use_agent":{"name":"use_agent","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use cephadm agent on each host to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"use_repo_digest":{"name":"use_repo_digest","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Automatically convert image tags to image digest. Make sure all daemons use the same image","long_desc":"","tags":[],"see_also":[]},"warn_on_failed_host_check":{"name":"warn_on_failed_host_check","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if the host check fails","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_daemons":{"name":"warn_on_stray_daemons","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected that are not managed by cephadm","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_hosts":{"name":"warn_on_stray_hosts","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected on a host that is not managed by cephadm","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ACCOUNT_LOCKOUT_ATTEMPTS":{"name":"ACCOUNT_LOCKOUT_ATTEMPTS","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_SSL_VERIFY":{"name":"ALERTMANAGER_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_CEPHFS":{"name":"FEATURE_TOGGLE_CEPHFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_DASHBOARD":{"name":"FEATURE_TOGGLE_DASHBOARD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_ISCSI":{"name":"FEATURE_TOGGLE_ISCSI","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_MIRRORING":{"name":"FEATURE_TOGGLE_MIRRORING","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_NFS":{"name":"FEATURE_TOGGLE_NFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RBD":{"name":"FEATURE_TOGGLE_RBD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RGW":{"name":"FEATURE_TOGGLE_RGW","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_FRONTEND_API_URL":{"name":"GRAFANA_FRONTEND_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISSUE_TRACKER_API_KEY":{"name":"ISSUE_TRACKER_API_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_SSL_VERIFY":{"name":"PROMETHEUS_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"cross_origin_url":{"name":"cross_origin_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"motd":{"name":"motd","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"The message of the day","long_desc":"","tags":[],"see_also":[]},"redirect_resolve_ip_addr":{"name":"redirect_resolve_ip_addr","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"diskprediction_local","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predict_interval":{"name":"predict_interval","type":"str","level":"advanced","flags":0,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predictor_model":{"name":"predictor_model","type":"str","level":"advanced","flags":0,"default_value":"prophetstor","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"int","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"How big batches of data points should be when sending to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server hostname","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"30","min":"5","max":"","enum_allowed":[],"desc":"Time between reports to InfluxDB. Default 30 seconds.","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"password of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"int","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server port","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"Use https connection for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"int","level":"advanced","flags":0,"default_value":"5","min":"1","max":"32","enum_allowed":[],"desc":"How many worker threads should be spawned for sending data to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"username of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"Verify https cert for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"what failure domain we should separate data replicas across.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"value to set min_size to (unchanged from Ceph's default if this option is not set)","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"which CRUSH subtree type the module should create a pool for.","tags":[],"see_also":[]}}},{"name":"mds_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"mirroring","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"nfs","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"fail_fs":{"name":"fail_fs","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Fail filesystem for rapid multi-rank mds upgrade","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_perf_query","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"noautoscale":{"name":"noautoscale","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"global autoscale flag","long_desc":"Option to turn on/off the autoscaler for all pools","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threshold":{"name":"threshold","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"1.0","max":"","enum_allowed":[],"desc":"scaling threshold","long_desc":"The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"allow_pg_recovery_event":{"name":"allow_pg_recovery_event","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow the module to show pg recovery progress","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how long the module is going to sleep","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"cache":{"name":"cache","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"exclude_perf_counters":{"name":"exclude_perf_counters","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Do not include perf-counters in the metrics output","long_desc":"Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"float","level":"advanced","flags":0,"default_value":"15.0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"the IPv4 or IPv6 address on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":1,"default_value":"9283","min":"","max":"","enum_allowed":[],"desc":"the port on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"log","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":1,"default_value":"default","min":"","max":"","enum_allowed":["default","error"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":1,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"restful","can_run":true,"error_string":"","module_options":{"enable_auth":{"name":"enable_auth","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rgw","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rook","can_run":true,"error_string":"","module_options":{"drive_group_interval":{"name":"drive_group_interval","type":"float","level":"advanced","flags":0,"default_value":"300.0","min":"","max":"","enum_allowed":[],"desc":"interval in seconds between re-application of applied drive_groups","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"storage_class":{"name":"storage_class","type":"str","level":"advanced","flags":0,"default_value":"local","min":"","max":"","enum_allowed":[],"desc":"storage class name for LSO-discovered PVs","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption7":{"name":"rwoption7","type":"int","level":"advanced","flags":0,"default_value":"","min":"1","max":"42","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"snap_schedule","can_run":true,"error_string":"","module_options":{"allow_m_granularity":{"name":"allow_m_granularity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow minute scheduled snapshots","long_desc":"","tags":[],"see_also":[]},"dump_on_update":{"name":"dump_on_update","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"dump database to debug log on update","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"stats","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share metadata about Ceph daemon crashes (version, stack straces, etc)","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share a user-provided description and/or contact email for the cluster","long_desc":"","tags":[],"see_also":[]},"channel_perf":{"name":"channel_perf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share various performance metrics of a cluster","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard_description":{"name":"leaderboard_description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_delay":{"name":"snapshot_clone_delay","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"Delay clone begin operation by snapshot_clone_delay seconds","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"discovery_interval":{"name":"discovery_interval","type":"uint","level":"advanced","flags":0,"default_value":"100","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}],"services":{},"always_on_modules":{"octopus":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"pacific":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"quincy":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"reef":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"]},"last_failure_osd_epoch":187,"active_clients":[{"name":"devicehealth","addrvec":[{"type":"v2","addr":"172.21.15.203:0","nonce":3690197627}]},{"name":"libcephsqlite","addrvec":[{"type":"v2","addr":"172.21.15.203:0","nonce":82655847}]},{"name":"rbd_support","addrvec":[{"type":"v2","addr":"172.21.15.203:0","nonce":2219953609}]},{"name":"snap_schedule","addrvec":[{"type":"v2","addr":"172.21.15.203:0","nonce":1082753911}]},{"name":"snap_schedule","addrvec":[{"type":"v2","addr":"172.21.15.203:0","nonce":169967264}]},{"name":"volumes","addrvec":[{"type":"v2","addr":"172.21.15.203:0","nonce":69497804}]}]} 2023-08-09T15:03:17.125 DEBUG:tasks.cephfs.test_snap_schedules:restarting active mgr: x 2023-08-09T15:03:17.125 INFO:tasks.ceph.mgr.x:Restarting daemon 2023-08-09T15:03:17.126 INFO:tasks.ceph.mgr.x:Stopping old one... 2023-08-09T15:03:17.126 DEBUG:tasks.ceph.mgr.x:waiting for process to exit 2023-08-09T15:03:17.127 INFO:teuthology.orchestra.run:waiting for 300 2023-08-09T15:03:17.202 INFO:tasks.ceph.mgr.x:Stopped 2023-08-09T15:03:17.203 DEBUG:teuthology.orchestra.run.smithi203:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mgr -f --cluster ceph -i x 2023-08-09T15:03:17.208 INFO:tasks.ceph.mgr.x:Started 2023-08-09T15:03:17.508 INFO:tasks.ceph.mgr.x.smithi203.stderr:2023-08-09T15:03:17.508+0000 7f8699c0e040 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2023-08-09T15:03:18.228 INFO:tasks.ceph.mgr.x.smithi203.stderr:2023-08-09T15:03:18.228+0000 7f8699c0e040 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2023-08-09T15:03:18.348 INFO:tasks.ceph.mgr.x.smithi203.stderr:2023-08-09T15:03:18.348+0000 7f8699c0e040 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2023-08-09T15:03:18.405 INFO:tasks.ceph.mgr.x.smithi203.stderr:2023-08-09T15:03:18.404+0000 7f8699c0e040 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2023-08-09T15:03:18.454 INFO:tasks.ceph.mgr.x.smithi203.stderr:2023-08-09T15:03:18.454+0000 7f8699c0e040 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2023-08-09T15:03:19.161 INFO:tasks.ceph.mgr.x.smithi203.stderr:2023-08-09T15:03:19.161+0000 7f8699c0e040 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2023-08-09T15:03:19.223 INFO:tasks.ceph.mgr.x.smithi203.stderr:2023-08-09T15:03:19.223+0000 7f8699c0e040 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2023-08-09T15:03:19.273 INFO:tasks.ceph.mgr.x.smithi203.stderr:2023-08-09T15:03:19.272+0000 7f8699c0e040 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2023-08-09T15:03:19.391 INFO:tasks.ceph.mgr.x.smithi203.stderr:2023-08-09T15:03:19.390+0000 7f8699c0e040 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2023-08-09T15:03:19.546 INFO:tasks.ceph.mgr.x.smithi203.stderr:2023-08-09T15:03:19.546+0000 7f8699c0e040 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2023-08-09T15:03:19.654 INFO:tasks.ceph.mgr.x.smithi203.stderr:2023-08-09T15:03:19.654+0000 7f8699c0e040 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2023-08-09T15:03:19.703 INFO:tasks.ceph.mgr.x.smithi203.stderr:2023-08-09T15:03:19.703+0000 7f8699c0e040 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2023-08-09T15:03:19.864 INFO:tasks.ceph.mgr.x.smithi203.stderr:2023-08-09T15:03:19.863+0000 7f8699c0e040 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2023-08-09T15:03:20.249 INFO:tasks.ceph.mgr.x.smithi203.stderr:2023-08-09T15:03:20.249+0000 7f8699c0e040 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2023-08-09T15:03:20.471 INFO:tasks.ceph.mgr.x.smithi203.stderr:2023-08-09T15:03:20.470+0000 7f8699c0e040 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2023-08-09T15:03:20.528 INFO:tasks.ceph.mgr.x.smithi203.stderr:2023-08-09T15:03:20.528+0000 7f8699c0e040 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2023-08-09T15:03:20.578 INFO:tasks.ceph.mgr.x.smithi203.stderr:2023-08-09T15:03:20.578+0000 7f8699c0e040 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2023-08-09T15:03:20.630 INFO:tasks.ceph.mgr.x.smithi203.stderr:2023-08-09T15:03:20.630+0000 7f8699c0e040 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2023-08-09T15:03:20.925 INFO:tasks.ceph.mgr.x.smithi203.stderr:2023-08-09T15:03:20.925+0000 7f8699c0e040 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2023-08-09T15:03:20.983 INFO:tasks.ceph.mgr.x.smithi203.stderr:2023-08-09T15:03:20.983+0000 7f8699c0e040 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2023-08-09T15:03:21.036 INFO:tasks.ceph.mgr.x.smithi203.stderr:2023-08-09T15:03:21.035+0000 7f8699c0e040 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2023-08-09T15:03:21.195 INFO:tasks.ceph.mgr.x.smithi203.stderr:2023-08-09T15:03:21.195+0000 7f8699c0e040 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2023-08-09T15:03:21.245 INFO:tasks.ceph.mgr.x.smithi203.stderr:2023-08-09T15:03:21.245+0000 7f8699c0e040 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2023-08-09T15:03:21.298 INFO:tasks.ceph.mgr.x.smithi203.stderr:2023-08-09T15:03:21.297+0000 7f8699c0e040 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2023-08-09T15:03:21.348 INFO:tasks.ceph.mgr.x.smithi203.stderr:2023-08-09T15:03:21.348+0000 7f8699c0e040 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2023-08-09T15:03:21.546 INFO:tasks.ceph.mgr.x.smithi203.stderr:2023-08-09T15:03:21.546+0000 7f8615ca3640 -1 client.0 error registering admin socket command: (17) File exists 2023-08-09T15:03:21.547 INFO:tasks.ceph.mgr.x.smithi203.stderr:2023-08-09T15:03:21.546+0000 7f8615ca3640 -1 client.0 error registering admin socket command: (17) File exists 2023-08-09T15:03:21.547 INFO:tasks.ceph.mgr.x.smithi203.stderr:2023-08-09T15:03:21.546+0000 7f8615ca3640 -1 client.0 error registering admin socket command: (17) File exists 2023-08-09T15:03:21.547 INFO:tasks.ceph.mgr.x.smithi203.stderr:2023-08-09T15:03:21.546+0000 7f8615ca3640 -1 client.0 error registering admin socket command: (17) File exists 2023-08-09T15:03:21.547 INFO:tasks.ceph.mgr.x.smithi203.stderr:2023-08-09T15:03:21.546+0000 7f8615ca3640 -1 client.0 error registering admin socket command: (17) File exists 2023-08-09T15:03:21.547 INFO:tasks.ceph.mgr.x.smithi203.stderr:2023-08-09T15:03:21.546+0000 7f8618ca9640 -1 client.0 error registering admin socket command: (17) File exists 2023-08-09T15:03:21.547 INFO:tasks.ceph.mgr.x.smithi203.stderr:2023-08-09T15:03:21.546+0000 7f8618ca9640 -1 client.0 error registering admin socket command: (17) File exists 2023-08-09T15:03:21.548 INFO:tasks.ceph.mgr.x.smithi203.stderr:2023-08-09T15:03:21.546+0000 7f8618ca9640 -1 client.0 error registering admin socket command: (17) File exists 2023-08-09T15:03:21.548 INFO:tasks.ceph.mgr.x.smithi203.stderr:2023-08-09T15:03:21.546+0000 7f8618ca9640 -1 client.0 error registering admin socket command: (17) File exists 2023-08-09T15:03:21.548 INFO:tasks.ceph.mgr.x.smithi203.stderr:2023-08-09T15:03:21.546+0000 7f8618ca9640 -1 client.0 error registering admin socket command: (17) File exists 2023-08-09T15:03:29.070 DEBUG:teuthology.orchestra.run.smithi043:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-08-09T15:03:29.080 DEBUG:teuthology.orchestra.run.smithi203:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-08-09T15:03:59.258 DEBUG:teuthology.orchestra.run.smithi043:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-08-09T15:03:59.271 DEBUG:teuthology.orchestra.run.smithi203:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-08-09T15:04:00.554 INFO:tasks.ceph.mgr.x.smithi203.stderr:2023-08-09T15:04:00.554+0000 7f8620cb9640 -1 client.0 error registering admin socket command: (17) File exists 2023-08-09T15:04:00.554 INFO:tasks.ceph.mgr.x.smithi203.stderr:2023-08-09T15:04:00.554+0000 7f8620cb9640 -1 client.0 error registering admin socket command: (17) File exists 2023-08-09T15:04:00.554 INFO:tasks.ceph.mgr.x.smithi203.stderr:2023-08-09T15:04:00.554+0000 7f8620cb9640 -1 client.0 error registering admin socket command: (17) File exists 2023-08-09T15:04:00.555 INFO:tasks.ceph.mgr.x.smithi203.stderr:2023-08-09T15:04:00.554+0000 7f8620cb9640 -1 client.0 error registering admin socket command: (17) File exists 2023-08-09T15:04:00.555 INFO:tasks.ceph.mgr.x.smithi203.stderr:2023-08-09T15:04:00.554+0000 7f8620cb9640 -1 client.0 error registering admin socket command: (17) File exists 2023-08-09T15:04:29.442 DEBUG:teuthology.orchestra.run.smithi043:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-08-09T15:04:29.452 DEBUG:teuthology.orchestra.run.smithi203:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-08-09T15:04:59.595 DEBUG:teuthology.orchestra.run.smithi043:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-08-09T15:04:59.604 DEBUG:teuthology.orchestra.run.smithi203:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-08-09T15:05:29.871 DEBUG:teuthology.orchestra.run.smithi043:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-08-09T15:05:29.882 DEBUG:teuthology.orchestra.run.smithi203:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-08-09T15:06:00.037 DEBUG:teuthology.orchestra.run.smithi043:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-08-09T15:06:00.047 DEBUG:teuthology.orchestra.run.smithi203:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-08-09T15:06:30.267 DEBUG:teuthology.orchestra.run.smithi043:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-08-09T15:06:30.277 DEBUG:teuthology.orchestra.run.smithi203:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-08-09T15:07:00.424 DEBUG:teuthology.orchestra.run.smithi043:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-08-09T15:07:00.433 DEBUG:teuthology.orchestra.run.smithi203:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-08-09T15:07:30.617 DEBUG:teuthology.orchestra.run.smithi043:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-08-09T15:07:30.627 DEBUG:teuthology.orchestra.run.smithi203:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-08-09T15:08:00.820 DEBUG:teuthology.orchestra.run.smithi043:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-08-09T15:08:00.830 DEBUG:teuthology.orchestra.run.smithi203:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-08-09T15:08:17.210 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs snap-schedule deactivate --fs cephfs /snap_test_dir1/test_restart 1M 2023-08-09T15:08:17.456 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:17.454+0000 7f9668da4640 1 Processor -- start 2023-08-09T15:08:17.457 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:17.455+0000 7f9668da4640 1 -- start start 2023-08-09T15:08:17.458 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:17.456+0000 7f9668da4640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f966415c720 0x7f96640cd850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T15:08:17.458 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:17.456+0000 7f9668da4640 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7f96640d0900 0x7f96640cec60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T15:08:17.458 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:17.456+0000 7f9668da4640 1 -- --> v1:172.21.15.203:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f96640b5660 con 0x7f96640cf1a0 2023-08-09T15:08:17.458 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:17.456+0000 7f9668da4640 1 -- --> v2:172.21.15.203:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f966415cb00 con 0x7f96640d0900 2023-08-09T15:08:17.458 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:17.456+0000 7f9668da4640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f96640cf630 con 0x7f966415c720 2023-08-09T15:08:17.459 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:17.457+0000 7f96637fe640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f966415c720 0x7f96640cd850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T15:08:17.459 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:17.457+0000 7f9662ffd640 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7f96640d0900 0x7f96640cec60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T15:08:17.459 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:17.457+0000 7f96637fe640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f966415c720 0x7f96640cd850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.203:3301/0 says I am v2:172.21.15.43:52498/0 (socket says 172.21.15.43:52498) 2023-08-09T15:08:17.459 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:17.457+0000 7f96637fe640 1 -- 172.21.15.43:0/4184211076 learned_addr learned my addr 172.21.15.43:0/4184211076 (peer_addr_for_me v2:172.21.15.43:0/0) 2023-08-09T15:08:17.460 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:17.458+0000 7f96637fe640 1 -- 172.21.15.43:0/4184211076 >> v2:172.21.15.203:3300/0 conn(0x7f96640d0900 msgr2=0x7f96640cec60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-09T15:08:17.460 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:17.458+0000 7f96637fe640 1 --2- 172.21.15.43:0/4184211076 >> v2:172.21.15.203:3300/0 conn(0x7f96640d0900 0x7f96640cec60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T15:08:17.460 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:17.458+0000 7f96637fe640 1 -- 172.21.15.43:0/4184211076 >> v1:172.21.15.203:6789/0 conn(0x7f96640cf1a0 legacy=0x7f96640b3cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T15:08:17.460 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:17.458+0000 7f96637fe640 1 -- 172.21.15.43:0/4184211076 --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f96640b5900 con 0x7f966415c720 2023-08-09T15:08:17.460 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:17.459+0000 7f96637fe640 1 --2- 172.21.15.43:0/4184211076 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f966415c720 0x7f96640cd850 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x7f9654009870 tx=0x7f965402b980 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c900cc8f82ebeea6 server_cookie=0 in_seq=0 out_seq=0 2023-08-09T15:08:17.461 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:17.460+0000 7f96627fc640 1 -- 172.21.15.43:0/4184211076 <== mon.2 v2:172.21.15.203:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9654030840 con 0x7f966415c720 2023-08-09T15:08:17.462 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:17.460+0000 7f96627fc640 1 -- 172.21.15.43:0/4184211076 <== mon.2 v2:172.21.15.203:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f96540309a0 con 0x7f966415c720 2023-08-09T15:08:17.462 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:17.460+0000 7f96627fc640 1 -- 172.21.15.43:0/4184211076 <== mon.2 v2:172.21.15.203:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9654030d30 con 0x7f966415c720 2023-08-09T15:08:17.462 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:17.461+0000 7f9668da4640 1 -- 172.21.15.43:0/4184211076 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f966415c720 msgr2=0x7f96640cd850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T15:08:17.463 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:17.461+0000 7f9668da4640 1 --2- 172.21.15.43:0/4184211076 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f966415c720 0x7f96640cd850 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x7f9654009870 tx=0x7f965402b980 comp rx=0 tx=0).stop 2023-08-09T15:08:17.463 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:17.461+0000 7f9668da4640 1 -- 172.21.15.43:0/4184211076 shutdown_connections 2023-08-09T15:08:17.463 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:17.461+0000 7f9668da4640 1 --2- 172.21.15.43:0/4184211076 >> v2:172.21.15.203:3300/0 conn(0x7f96640d0900 0x7f96640cec60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T15:08:17.463 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:17.461+0000 7f9668da4640 1 --2- 172.21.15.43:0/4184211076 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f966415c720 0x7f96640cd850 unknown :-1 s=CLOSED pgs=296 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T15:08:17.463 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:17.461+0000 7f9668da4640 1 -- 172.21.15.43:0/4184211076 >> 172.21.15.43:0/4184211076 conn(0x7f966406a2a0 msgr2=0x7f966406a680 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-09T15:08:17.463 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:17.461+0000 7f9668da4640 1 -- 172.21.15.43:0/4184211076 shutdown_connections 2023-08-09T15:08:17.464 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:17.461+0000 7f9668da4640 1 -- 172.21.15.43:0/4184211076 wait complete. 2023-08-09T15:08:17.464 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:17.463+0000 7f9668da4640 1 Processor -- start 2023-08-09T15:08:17.464 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:17.463+0000 7f9668da4640 1 -- start start 2023-08-09T15:08:17.465 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:17.464+0000 7f9668da4640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f96640cf1a0 0x7f96641f2a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T15:08:17.465 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:17.464+0000 7f9668da4640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f96640d0900 0x7f96641f4790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T15:08:17.465 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:17.464+0000 7f9668da4640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f966415c720 0x7f96640c7bc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T15:08:17.465 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:17.464+0000 7f9668da4640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f96640c8100 con 0x7f96640d0900 2023-08-09T15:08:17.466 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:17.464+0000 7f9668da4640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f96640c9c20 con 0x7f96640cf1a0 2023-08-09T15:08:17.466 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:17.464+0000 7f9668da4640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f96640c9330 con 0x7f966415c720 2023-08-09T15:08:17.467 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:17.464+0000 7f9662ffd640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f96640d0900 0x7f96641f4790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T15:08:17.467 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:17.464+0000 7f9662ffd640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f96640d0900 0x7f96641f4790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:46362/0 (socket says 172.21.15.43:46362) 2023-08-09T15:08:17.468 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:17.464+0000 7f9662ffd640 1 -- 172.21.15.43:0/4131263355 learned_addr learned my addr 172.21.15.43:0/4131263355 (peer_addr_for_me v2:172.21.15.43:0/0) 2023-08-09T15:08:17.468 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:17.465+0000 7f9663fff640 1 --2- 172.21.15.43:0/4131263355 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f966415c720 0x7f96640c7bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T15:08:17.468 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:17.465+0000 7f96637fe640 1 --2- 172.21.15.43:0/4131263355 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f96640cf1a0 0x7f96641f2a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T15:08:17.468 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:17.465+0000 7f9662ffd640 1 -- 172.21.15.43:0/4131263355 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f966415c720 msgr2=0x7f96640c7bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T15:08:17.468 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:17.465+0000 7f9662ffd640 1 --2- 172.21.15.43:0/4131263355 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f966415c720 0x7f96640c7bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T15:08:17.468 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:17.465+0000 7f9662ffd640 1 -- 172.21.15.43:0/4131263355 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f96640cf1a0 msgr2=0x7f96641f2a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T15:08:17.469 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:17.465+0000 7f9662ffd640 1 --2- 172.21.15.43:0/4131263355 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f96640cf1a0 0x7f96641f2a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T15:08:17.469 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:17.465+0000 7f9662ffd640 1 -- 172.21.15.43:0/4131263355 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f96540094d0 con 0x7f96640d0900 2023-08-09T15:08:17.469 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:17.465+0000 7f96637fe640 1 --2- 172.21.15.43:0/4131263355 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f96640cf1a0 0x7f96641f2a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-08-09T15:08:17.469 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:17.465+0000 7f9663fff640 1 --2- 172.21.15.43:0/4131263355 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f966415c720 0x7f96640c7bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-08-09T15:08:17.469 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:17.465+0000 7f9662ffd640 1 --2- 172.21.15.43:0/4131263355 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f96640d0900 0x7f96641f4790 secure :-1 s=READY pgs=2090 cs=0 l=1 rev1=1 crypto rx=0x7f965800e9b0 tx=0x7f965800ee70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-09T15:08:17.469 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:17.466+0000 7f964bfff640 1 -- 172.21.15.43:0/4131263355 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9658009670 con 0x7f96640d0900 2023-08-09T15:08:17.469 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:17.466+0000 7f964bfff640 1 -- 172.21.15.43:0/4131263355 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f965800c9a0 con 0x7f96640d0900 2023-08-09T15:08:17.470 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:17.466+0000 7f964bfff640 1 -- 172.21.15.43:0/4131263355 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f965800cd50 con 0x7f96640d0900 2023-08-09T15:08:17.470 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:17.466+0000 7f9668da4640 1 -- 172.21.15.43:0/4131263355 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f96640c95d0 con 0x7f96640d0900 2023-08-09T15:08:17.470 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:17.466+0000 7f9668da4640 1 -- 172.21.15.43:0/4131263355 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f96640cb910 con 0x7f96640d0900 2023-08-09T15:08:17.470 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:17.467+0000 7f9668da4640 1 -- 172.21.15.43:0/4131263355 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f96640739f0 con 0x7f96640d0900 2023-08-09T15:08:17.471 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:17.468+0000 7f964bfff640 1 -- 172.21.15.43:0/4131263355 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 101) v1 ==== 88472+0+0 (secure 0 0 0) 0x7f9658013470 con 0x7f96640d0900 2023-08-09T15:08:17.471 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:17.469+0000 7f964bfff640 1 --2- 172.21.15.43:0/4131263355 >> [v2:172.21.15.203:6832/115830229,v1:172.21.15.203:6833/115830229] conn(0x7f965006a1c0 0x7f965006c680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T15:08:17.471 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:17.469+0000 7f964bfff640 1 -- 172.21.15.43:0/4131263355 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(188..188 src has 1..188) v4 ==== 7114+0+0 (secure 0 0 0) 0x7f9658088e60 con 0x7f96640d0900 2023-08-09T15:08:17.471 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:17.470+0000 7f96637fe640 1 --2- 172.21.15.43:0/4131263355 >> [v2:172.21.15.203:6832/115830229,v1:172.21.15.203:6833/115830229] conn(0x7f965006a1c0 0x7f965006c680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T15:08:17.472 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:17.471+0000 7f96637fe640 1 --2- 172.21.15.43:0/4131263355 >> [v2:172.21.15.203:6832/115830229,v1:172.21.15.203:6833/115830229] conn(0x7f965006a1c0 0x7f965006c680 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7f9654036000 tx=0x7f965403b040 comp rx=0 tx=0).ready entity=mgr.7487 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-09T15:08:17.476 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:17.475+0000 7f964bfff640 1 -- 172.21.15.43:0/4131263355 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178066 (secure 0 0 0) 0x7f9658053f80 con 0x7f96640d0900 2023-08-09T15:08:17.700 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:17.698+0000 7f9668da4640 1 -- 172.21.15.43:0/4131263355 --> [v2:172.21.15.203:6832/115830229,v1:172.21.15.203:6833/115830229] -- mgr_command(tid 0: {"prefix": "fs snap-schedule deactivate", "fs": "cephfs", "path": "/snap_test_dir1/test_restart", "repeat": "1M", "target": ["mon-mgr", ""]}) v1 -- 0x7f96640d10c0 con 0x7f965006a1c0 2023-08-09T15:08:17.851 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:17.850+0000 7f964bfff640 1 -- 172.21.15.43:0/4131263355 <== mgr.7487 v2:172.21.15.203:6832/115830229 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+59 (secure 0 0 0) 0x7f9654030c10 con 0x7f965006a1c0 2023-08-09T15:08:17.851 INFO:teuthology.orchestra.run.smithi043.stdout:Schedule deactivated for path /snap_test_dir1/test_restart 2023-08-09T15:08:17.854 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:17.853+0000 7f9668da4640 1 -- 172.21.15.43:0/4131263355 >> [v2:172.21.15.203:6832/115830229,v1:172.21.15.203:6833/115830229] conn(0x7f965006a1c0 msgr2=0x7f965006c680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T15:08:17.854 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:17.853+0000 7f9668da4640 1 --2- 172.21.15.43:0/4131263355 >> [v2:172.21.15.203:6832/115830229,v1:172.21.15.203:6833/115830229] conn(0x7f965006a1c0 0x7f965006c680 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7f9654036000 tx=0x7f965403b040 comp rx=0 tx=0).stop 2023-08-09T15:08:17.855 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:17.853+0000 7f9668da4640 1 -- 172.21.15.43:0/4131263355 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f96640d0900 msgr2=0x7f96641f4790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T15:08:17.855 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:17.853+0000 7f9668da4640 1 --2- 172.21.15.43:0/4131263355 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f96640d0900 0x7f96641f4790 secure :-1 s=READY pgs=2090 cs=0 l=1 rev1=1 crypto rx=0x7f965800e9b0 tx=0x7f965800ee70 comp rx=0 tx=0).stop 2023-08-09T15:08:17.855 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:17.853+0000 7f9668da4640 1 -- 172.21.15.43:0/4131263355 shutdown_connections 2023-08-09T15:08:17.855 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:17.853+0000 7f9668da4640 1 --2- 172.21.15.43:0/4131263355 >> [v2:172.21.15.203:6832/115830229,v1:172.21.15.203:6833/115830229] conn(0x7f965006a1c0 0x7f965006c680 unknown :-1 s=CLOSED pgs=23 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T15:08:17.855 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:17.853+0000 7f9668da4640 1 --2- 172.21.15.43:0/4131263355 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f966415c720 0x7f96640c7bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T15:08:17.855 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:17.853+0000 7f9668da4640 1 --2- 172.21.15.43:0/4131263355 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f96640d0900 0x7f96641f4790 unknown :-1 s=CLOSED pgs=2090 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T15:08:17.855 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:17.853+0000 7f9668da4640 1 --2- 172.21.15.43:0/4131263355 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f96640cf1a0 0x7f96641f2a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T15:08:17.856 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:17.853+0000 7f9668da4640 1 -- 172.21.15.43:0/4131263355 >> 172.21.15.43:0/4131263355 conn(0x7f966406a2a0 msgr2=0x7f96640b7ba0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-09T15:08:17.856 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:17.853+0000 7f9668da4640 1 -- 172.21.15.43:0/4131263355 shutdown_connections 2023-08-09T15:08:17.856 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:17.853+0000 7f9668da4640 1 -- 172.21.15.43:0/4131263355 wait complete. 2023-08-09T15:08:17.876 DEBUG:tasks.cephfs.test_snap_schedules:res=Schedule deactivated for path /snap_test_dir1/test_restart 2023-08-09T15:08:17.876 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-09T15:08:17.877 DEBUG:teuthology.orchestra.run.smithi043:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/test_restart/.snap) 2023-08-09T15:08:17.947 INFO:teuthology.orchestra.run.smithi043.stdout:scheduled-2023-08-09-14_53_00_UTC 2023-08-09T15:08:17.947 INFO:teuthology.orchestra.run.smithi043.stdout:scheduled-2023-08-09-14_54_00_UTC 2023-08-09T15:08:17.947 INFO:teuthology.orchestra.run.smithi043.stdout:scheduled-2023-08-09-14_55_00_UTC 2023-08-09T15:08:17.947 INFO:teuthology.orchestra.run.smithi043.stdout:scheduled-2023-08-09-14_56_00_UTC 2023-08-09T15:08:17.948 INFO:teuthology.orchestra.run.smithi043.stdout:scheduled-2023-08-09-14_57_00_UTC 2023-08-09T15:08:17.948 INFO:teuthology.orchestra.run.smithi043.stdout:scheduled-2023-08-09-14_58_00_UTC 2023-08-09T15:08:17.948 INFO:teuthology.orchestra.run.smithi043.stdout:scheduled-2023-08-09-14_59_00_UTC 2023-08-09T15:08:17.948 INFO:teuthology.orchestra.run.smithi043.stdout:scheduled-2023-08-09-15_00_00_UTC 2023-08-09T15:08:17.948 INFO:teuthology.orchestra.run.smithi043.stdout:scheduled-2023-08-09-15_01_00_UTC 2023-08-09T15:08:17.948 INFO:teuthology.orchestra.run.smithi043.stdout:scheduled-2023-08-09-15_02_00_UTC 2023-08-09T15:08:17.948 INFO:teuthology.orchestra.run.smithi043.stdout:scheduled-2023-08-09-15_03_00_UTC 2023-08-09T15:08:17.949 INFO:teuthology.orchestra.run.smithi043.stdout:scheduled-2023-08-09-15_04_00_UTC 2023-08-09T15:08:17.949 INFO:teuthology.orchestra.run.smithi043.stdout:scheduled-2023-08-09-15_05_00_UTC 2023-08-09T15:08:17.949 INFO:teuthology.orchestra.run.smithi043.stdout:scheduled-2023-08-09-15_06_00_UTC 2023-08-09T15:08:17.949 INFO:teuthology.orchestra.run.smithi043.stdout:scheduled-2023-08-09-15_07_00_UTC 2023-08-09T15:08:17.949 INFO:teuthology.orchestra.run.smithi043.stdout:scheduled-2023-08-09-15_08_00_UTC 2023-08-09T15:08:17.950 DEBUG:tasks.cephfs.test_snap_schedules:snapshots: ['scheduled-2023-08-09-14_53_00_UTC', 'scheduled-2023-08-09-14_54_00_UTC', 'scheduled-2023-08-09-14_55_00_UTC', 'scheduled-2023-08-09-14_56_00_UTC', 'scheduled-2023-08-09-14_57_00_UTC', 'scheduled-2023-08-09-14_58_00_UTC', 'scheduled-2023-08-09-14_59_00_UTC', 'scheduled-2023-08-09-15_00_00_UTC', 'scheduled-2023-08-09-15_01_00_UTC', 'scheduled-2023-08-09-15_02_00_UTC', 'scheduled-2023-08-09-15_03_00_UTC', 'scheduled-2023-08-09-15_04_00_UTC', 'scheduled-2023-08-09-15_05_00_UTC', 'scheduled-2023-08-09-15_06_00_UTC', 'scheduled-2023-08-09-15_07_00_UTC', 'scheduled-2023-08-09-15_08_00_UTC'] 2023-08-09T15:08:17.950 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs snap-schedule status --fs cephfs --format json /snap_test_dir1/test_restart 2023-08-09T15:08:18.194 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:18.193+0000 7f485c4ec640 1 Processor -- start 2023-08-09T15:08:18.194 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:18.193+0000 7f485c4ec640 1 -- start start 2023-08-09T15:08:18.196 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:18.194+0000 7f485c4ec640 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7f48540b80a0 0x7f48540b5470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T15:08:18.196 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:18.195+0000 7f485c4ec640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f48540b59b0 0x7f48540b5e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T15:08:18.196 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:18.195+0000 7f485c4ec640 1 -- --> v1:172.21.15.43:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f48540d1890 con 0x7f485415c6f0 2023-08-09T15:08:18.196 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:18.195+0000 7f485c4ec640 1 -- --> v2:172.21.15.203:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f48540d2d30 con 0x7f48540b80a0 2023-08-09T15:08:18.196 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:18.195+0000 7f485c4ec640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f485415cad0 con 0x7f48540b59b0 2023-08-09T15:08:18.198 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:18.195+0000 7f485a288640 1 --1- >> v1:172.21.15.43:6789/0 conn(0x7f485415c6f0 0x7f48540d03a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.43:6789/0 says I am v1:172.21.15.43:47322/0 (socket says 172.21.15.43:47322) 2023-08-09T15:08:18.198 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:18.195+0000 7f485a288640 1 -- 172.21.15.43:0/2992339839 learned_addr learned my addr 172.21.15.43:0/2992339839 (peer_addr_for_me v1:172.21.15.43:0/0) 2023-08-09T15:08:18.198 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:18.195+0000 7f4859a87640 1 --2- 172.21.15.43:0/2992339839 >> v2:172.21.15.203:3300/0 conn(0x7f48540b80a0 0x7f48540b5470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T15:08:18.198 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:18.195+0000 7f485aa89640 1 --2- 172.21.15.43:0/2992339839 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f48540b59b0 0x7f48540b5e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T15:08:18.199 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:18.195+0000 7f4859286640 1 -- 172.21.15.43:0/2992339839 <== mon.0 v1:172.21.15.43:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2538493999 0 0) 0x7f48540d1890 con 0x7f485415c6f0 2023-08-09T15:08:18.199 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:18.196+0000 7f4859286640 1 -- 172.21.15.43:0/2992339839 --> v1:172.21.15.43:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f4840003520 con 0x7f485415c6f0 2023-08-09T15:08:18.199 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:18.196+0000 7f4859286640 1 -- 172.21.15.43:0/2992339839 <== mon.0 v1:172.21.15.43:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1222702435 0 0) 0x7f4840003520 con 0x7f485415c6f0 2023-08-09T15:08:18.199 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:18.196+0000 7f4859286640 1 -- 172.21.15.43:0/2992339839 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f48540b59b0 msgr2=0x7f48540b5e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-09T15:08:18.199 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:18.196+0000 7f4859286640 1 --2- 172.21.15.43:0/2992339839 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f48540b59b0 0x7f48540b5e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T15:08:18.199 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:18.196+0000 7f4859286640 1 -- 172.21.15.43:0/2992339839 >> v2:172.21.15.203:3300/0 conn(0x7f48540b80a0 msgr2=0x7f48540b5470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-09T15:08:18.199 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:18.196+0000 7f4859286640 1 --2- 172.21.15.43:0/2992339839 >> v2:172.21.15.203:3300/0 conn(0x7f48540b80a0 0x7f48540b5470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T15:08:18.199 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:18.196+0000 7f4859286640 1 -- 172.21.15.43:0/2992339839 --> v1:172.21.15.43:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f48540d0c80 con 0x7f485415c6f0 2023-08-09T15:08:18.200 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:18.196+0000 7f4859286640 1 -- 172.21.15.43:0/2992339839 <== mon.0 v1:172.21.15.43:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1285150740 0 0) 0x7f48500019f0 con 0x7f485415c6f0 2023-08-09T15:08:18.200 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:18.196+0000 7f4859286640 1 -- 172.21.15.43:0/2992339839 >> v1:172.21.15.43:6789/0 conn(0x7f485415c6f0 legacy=0x7f48540d03a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T15:08:18.200 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:18.196+0000 7f4859286640 1 --2- 172.21.15.43:0/2992339839 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f48540b59b0 0x7f48400039e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T15:08:18.200 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:18.196+0000 7f4859286640 1 --2- 172.21.15.43:0/2992339839 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f4840003f60 0x7f48400047c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T15:08:18.200 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:18.196+0000 7f4859286640 1 --2- 172.21.15.43:0/2992339839 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f4840004d00 0x7f4840005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T15:08:18.200 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:18.196+0000 7f4859286640 1 -- 172.21.15.43:0/2992339839 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f48540d2d30 con 0x7f4840003f60 2023-08-09T15:08:18.200 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:18.196+0000 7f4859286640 1 -- 172.21.15.43:0/2992339839 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f485415cad0 con 0x7f4840004d00 2023-08-09T15:08:18.201 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:18.196+0000 7f4859286640 1 -- 172.21.15.43:0/2992339839 --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f484000b9b0 con 0x7f48540b59b0 2023-08-09T15:08:18.201 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:18.196+0000 7f4859a87640 1 --2- 172.21.15.43:0/2992339839 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f4840003f60 0x7f48400047c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T15:08:18.201 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:18.197+0000 7f485aa89640 1 --2- 172.21.15.43:0/2992339839 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f4840004d00 0x7f4840005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T15:08:18.201 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:18.197+0000 7f485a288640 1 --2- 172.21.15.43:0/2992339839 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f48540b59b0 0x7f48400039e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T15:08:18.201 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:18.197+0000 7f4859a87640 1 -- 172.21.15.43:0/2992339839 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f48540b59b0 msgr2=0x7f48400039e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-09T15:08:18.201 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:18.197+0000 7f4859a87640 1 --2- 172.21.15.43:0/2992339839 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f48540b59b0 0x7f48400039e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T15:08:18.201 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:18.197+0000 7f4859a87640 1 -- 172.21.15.43:0/2992339839 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f4840004d00 msgr2=0x7f4840005180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-09T15:08:18.201 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:18.197+0000 7f4859a87640 1 --2- 172.21.15.43:0/2992339839 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f4840004d00 0x7f4840005180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T15:08:18.202 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:18.197+0000 7f4859a87640 1 -- 172.21.15.43:0/2992339839 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4840003520 con 0x7f4840003f60 2023-08-09T15:08:18.202 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:18.197+0000 7f485aa89640 1 --2- 172.21.15.43:0/2992339839 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f4840004d00 0x7f4840005180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-08-09T15:08:18.203 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:18.198+0000 7f4859a87640 1 --2- 172.21.15.43:0/2992339839 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f4840003f60 0x7f48400047c0 secure :-1 s=READY pgs=2092 cs=0 l=1 rev1=1 crypto rx=0x7f4844009ac0 tx=0x7f48400057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ac860700f4564772 server_cookie=0 in_seq=0 out_seq=0 2023-08-09T15:08:18.203 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:18.198+0000 7f4859286640 1 -- 172.21.15.43:0/2992339839 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f484402c710 con 0x7f4840003f60 2023-08-09T15:08:18.203 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:18.198+0000 7f4859286640 1 -- 172.21.15.43:0/2992339839 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f484402c870 con 0x7f4840003f60 2023-08-09T15:08:18.204 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:18.198+0000 7f4859286640 1 -- 172.21.15.43:0/2992339839 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f484402cc00 con 0x7f4840003f60 2023-08-09T15:08:18.204 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:18.199+0000 7f485c4ec640 1 -- 172.21.15.43:0/2992339839 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f4840003f60 msgr2=0x7f48400047c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T15:08:18.204 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:18.199+0000 7f485c4ec640 1 --2- 172.21.15.43:0/2992339839 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f4840003f60 0x7f48400047c0 secure :-1 s=READY pgs=2092 cs=0 l=1 rev1=1 crypto rx=0x7f4844009ac0 tx=0x7f48400057f0 comp rx=0 tx=0).stop 2023-08-09T15:08:18.204 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:18.199+0000 7f485aa89640 1 -- 172.21.15.43:0/2992339839 reap_dead start 2023-08-09T15:08:18.204 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:18.199+0000 7f485c4ec640 1 -- 172.21.15.43:0/2992339839 shutdown_connections 2023-08-09T15:08:18.204 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:18.199+0000 7f485c4ec640 1 -- 172.21.15.43:0/2992339839 >> 172.21.15.43:0/2992339839 conn(0x7f48540712e0 msgr2=0x7f48540716f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-09T15:08:18.204 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:18.199+0000 7f485c4ec640 1 -- 172.21.15.43:0/2992339839 shutdown_connections 2023-08-09T15:08:18.205 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:18.199+0000 7f485c4ec640 1 -- 172.21.15.43:0/2992339839 wait complete. 2023-08-09T15:08:18.205 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:18.201+0000 7f485c4ec640 1 Processor -- start 2023-08-09T15:08:18.205 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:18.201+0000 7f485c4ec640 1 -- start start 2023-08-09T15:08:18.206 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:18.202+0000 7f485c4ec640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f4840003f60 0x7f4854156520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T15:08:18.206 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:18.202+0000 7f485c4ec640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f4854155f90 0x7f48541556d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T15:08:18.206 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:18.202+0000 7f485c4ec640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f4854159aa0 0x7f4854159f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T15:08:18.207 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:18.202+0000 7f485c4ec640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f485415a460 con 0x7f4840003f60 2023-08-09T15:08:18.207 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:18.202+0000 7f485c4ec640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4854156b20 con 0x7f4854159aa0 2023-08-09T15:08:18.207 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:18.202+0000 7f485c4ec640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4854156e00 con 0x7f4854155f90 2023-08-09T15:08:18.207 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:18.202+0000 7f485a288640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f4840003f60 0x7f4854156520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T15:08:18.207 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:18.202+0000 7f485a288640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f4840003f60 0x7f4854156520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:46392/0 (socket says 172.21.15.43:46392) 2023-08-09T15:08:18.207 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:18.202+0000 7f485a288640 1 -- 172.21.15.43:0/2308424025 learned_addr learned my addr 172.21.15.43:0/2308424025 (peer_addr_for_me v2:172.21.15.43:0/0) 2023-08-09T15:08:18.207 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:18.203+0000 7f485a288640 1 -- 172.21.15.43:0/2308424025 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f4854155f90 msgr2=0x7f48541556d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T15:08:18.207 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:18.203+0000 7f485a288640 1 --2- 172.21.15.43:0/2308424025 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f4854155f90 0x7f48541556d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T15:08:18.208 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:18.203+0000 7f485a288640 1 -- 172.21.15.43:0/2308424025 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f4854159aa0 msgr2=0x7f4854159f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T15:08:18.208 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:18.203+0000 7f485a288640 1 --2- 172.21.15.43:0/2308424025 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f4854159aa0 0x7f4854159f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T15:08:18.208 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:18.203+0000 7f485a288640 1 -- 172.21.15.43:0/2308424025 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f48541570e0 con 0x7f4840003f60 2023-08-09T15:08:18.208 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:18.203+0000 7f485a288640 1 --2- 172.21.15.43:0/2308424025 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f4840003f60 0x7f4854156520 secure :-1 s=READY pgs=2093 cs=0 l=1 rev1=1 crypto rx=0x7f4850001a00 tx=0x7f485000b4a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-09T15:08:18.208 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:18.203+0000 7f484affd640 1 -- 172.21.15.43:0/2308424025 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f485000bb30 con 0x7f4840003f60 2023-08-09T15:08:18.208 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:18.203+0000 7f484affd640 1 -- 172.21.15.43:0/2308424025 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f485000bc90 con 0x7f4840003f60 2023-08-09T15:08:18.208 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:18.203+0000 7f484affd640 1 -- 172.21.15.43:0/2308424025 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f48500072d0 con 0x7f4840003f60 2023-08-09T15:08:18.209 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:18.204+0000 7f485c4ec640 1 -- 172.21.15.43:0/2308424025 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f48541573c0 con 0x7f4840003f60 2023-08-09T15:08:18.209 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:18.204+0000 7f485c4ec640 1 -- 172.21.15.43:0/2308424025 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f48541f1840 con 0x7f4840003f60 2023-08-09T15:08:18.209 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:18.205+0000 7f485c4ec640 1 -- 172.21.15.43:0/2308424025 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f48541578e0 con 0x7f4840003f60 2023-08-09T15:08:18.209 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:18.206+0000 7f484affd640 1 -- 172.21.15.43:0/2308424025 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 101) v1 ==== 88472+0+0 (secure 0 0 0) 0x7f4850007430 con 0x7f4840003f60 2023-08-09T15:08:18.210 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:18.207+0000 7f484affd640 1 --2- 172.21.15.43:0/2308424025 >> [v2:172.21.15.203:6832/115830229,v1:172.21.15.203:6833/115830229] conn(0x7f483806a1f0 0x7f483806c6b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T15:08:18.210 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:18.207+0000 7f484affd640 1 -- 172.21.15.43:0/2308424025 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(188..188 src has 1..188) v4 ==== 7114+0+0 (secure 0 0 0) 0x7f4850088c50 con 0x7f4840003f60 2023-08-09T15:08:18.210 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:18.207+0000 7f4859a87640 1 --2- 172.21.15.43:0/2308424025 >> [v2:172.21.15.203:6832/115830229,v1:172.21.15.203:6833/115830229] conn(0x7f483806a1f0 0x7f483806c6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T15:08:18.210 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:18.208+0000 7f4859a87640 1 --2- 172.21.15.43:0/2308424025 >> [v2:172.21.15.203:6832/115830229,v1:172.21.15.203:6833/115830229] conn(0x7f483806a1f0 0x7f483806c6b0 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7f4844036000 tx=0x7f484402e000 comp rx=0 tx=0).ready entity=mgr.7487 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-09T15:08:18.212 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:18.211+0000 7f484affd640 1 -- 172.21.15.43:0/2308424025 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178066 (secure 0 0 0) 0x7f48541578e0 con 0x7f4840003f60 2023-08-09T15:08:18.429 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:18.427+0000 7f485c4ec640 1 -- 172.21.15.43:0/2308424025 --> [v2:172.21.15.203:6832/115830229,v1:172.21.15.203:6833/115830229] -- mgr_command(tid 0: {"prefix": "fs snap-schedule status", "fs": "cephfs", "path": "/snap_test_dir1/test_restart", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f48540d1240 con 0x7f483806a1f0 2023-08-09T15:08:18.447 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:18.446+0000 7f484affd640 1 -- 172.21.15.43:0/2308424025 <== mgr.7487 v2:172.21.15.203:6832/115830229 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+361 (secure 0 0 0) 0x7f484402da70 con 0x7f483806a1f0 2023-08-09T15:08:18.447 INFO:teuthology.orchestra.run.smithi043.stdout: 2023-08-09T15:08:18.448 INFO:teuthology.orchestra.run.smithi043.stdout:[{"fs": "cephfs", "subvol": null, "path": "/snap_test_dir1/test_restart", "rel_path": "/snap_test_dir1/test_restart", "schedule": "1M", "retention": {}, "start": "2023-08-09T00:00:00", "created": "2023-08-09T14:52:46", "first": "2023-08-09T14:53:00", "last": "2023-08-09T15:08:00", "last_pruned": null, "created_count": 16, "pruned_count": 0, "active": false}] 2023-08-09T15:08:18.450 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:18.448+0000 7f485c4ec640 1 -- 172.21.15.43:0/2308424025 >> [v2:172.21.15.203:6832/115830229,v1:172.21.15.203:6833/115830229] conn(0x7f483806a1f0 msgr2=0x7f483806c6b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T15:08:18.450 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:18.448+0000 7f485c4ec640 1 --2- 172.21.15.43:0/2308424025 >> [v2:172.21.15.203:6832/115830229,v1:172.21.15.203:6833/115830229] conn(0x7f483806a1f0 0x7f483806c6b0 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7f4844036000 tx=0x7f484402e000 comp rx=0 tx=0).stop 2023-08-09T15:08:18.450 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:18.448+0000 7f485c4ec640 1 -- 172.21.15.43:0/2308424025 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f4840003f60 msgr2=0x7f4854156520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T15:08:18.450 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:18.448+0000 7f485c4ec640 1 --2- 172.21.15.43:0/2308424025 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f4840003f60 0x7f4854156520 secure :-1 s=READY pgs=2093 cs=0 l=1 rev1=1 crypto rx=0x7f4850001a00 tx=0x7f485000b4a0 comp rx=0 tx=0).stop 2023-08-09T15:08:18.451 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:18.449+0000 7f485c4ec640 1 -- 172.21.15.43:0/2308424025 shutdown_connections 2023-08-09T15:08:18.451 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:18.449+0000 7f485c4ec640 1 --2- 172.21.15.43:0/2308424025 >> [v2:172.21.15.203:6832/115830229,v1:172.21.15.203:6833/115830229] conn(0x7f483806a1f0 0x7f483806c6b0 unknown :-1 s=CLOSED pgs=24 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T15:08:18.451 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:18.449+0000 7f485c4ec640 1 --2- 172.21.15.43:0/2308424025 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f4854159aa0 0x7f4854159f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T15:08:18.451 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:18.449+0000 7f485c4ec640 1 --2- 172.21.15.43:0/2308424025 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f4854155f90 0x7f48541556d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T15:08:18.451 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:18.449+0000 7f485c4ec640 1 --2- 172.21.15.43:0/2308424025 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f4840003f60 0x7f4854156520 unknown :-1 s=CLOSED pgs=2093 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T15:08:18.451 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:18.449+0000 7f485c4ec640 1 -- 172.21.15.43:0/2308424025 >> 172.21.15.43:0/2308424025 conn(0x7f48540712e0 msgr2=0x7f48540b67d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-09T15:08:18.452 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:18.449+0000 7f485c4ec640 1 -- 172.21.15.43:0/2308424025 shutdown_connections 2023-08-09T15:08:18.455 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:18.449+0000 7f485c4ec640 1 -- 172.21.15.43:0/2308424025 wait complete. 2023-08-09T15:08:18.471 DEBUG:tasks.cephfs.test_snap_schedules:res= [{"fs": "cephfs", "subvol": null, "path": "/snap_test_dir1/test_restart", "rel_path": "/snap_test_dir1/test_restart", "schedule": "1M", "retention": {}, "start": "2023-08-09T00:00:00", "created": "2023-08-09T14:52:46", "first": "2023-08-09T14:53:00", "last": "2023-08-09T15:08:00", "last_pruned": null, "created_count": 16, "pruned_count": 0, "active": false}] 2023-08-09T15:08:18.472 DEBUG:tasks.cephfs.test_snap_schedules:json_res: {'fs': 'cephfs', 'subvol': None, 'path': '/snap_test_dir1/test_restart', 'rel_path': '/snap_test_dir1/test_restart', 'schedule': '1M', 'retention': {}, 'start': '2023-08-09T00:00:00', 'created': '2023-08-09T14:52:46', 'first': '2023-08-09T14:53:00', 'last': '2023-08-09T15:08:00', 'last_pruned': None, 'created_count': 16, 'pruned_count': 0, 'active': False} 2023-08-09T15:08:18.472 DEBUG:tasks.cephfs.test_snap_schedules:fs_count: 16 2023-08-09T15:08:18.472 DEBUG:tasks.cephfs.test_snap_schedules:db_count: 16 2023-08-09T15:08:18.472 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs snap-schedule remove --fs cephfs /snap_test_dir1/test_restart 1M 2023-08-09T15:08:18.711 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:18.710+0000 7fc42080f640 1 Processor -- start 2023-08-09T15:08:18.711 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:18.710+0000 7fc42080f640 1 -- start start 2023-08-09T15:08:18.713 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:18.711+0000 7fc42080f640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7fc41c0c99b0 0x7fc41c15d3c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T15:08:18.713 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:18.711+0000 7fc42080f640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fc41c06c510 0x7fc41c06c960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T15:08:18.713 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:18.711+0000 7fc42080f640 1 -- --> v1:172.21.15.203:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc41c06cea0 con 0x7fc41c0c86c0 2023-08-09T15:08:18.713 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:18.711+0000 7fc42080f640 1 -- --> v2:172.21.15.43:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc41c157000 con 0x7fc41c0c99b0 2023-08-09T15:08:18.713 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:18.711+0000 7fc42080f640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc41c159c40 con 0x7fc41c06c510 2023-08-09T15:08:18.713 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:18.712+0000 7fc41b7fe640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7fc41c0c99b0 0x7fc41c15d3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T15:08:18.714 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:18.712+0000 7fc41b7fe640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7fc41c0c99b0 0x7fc41c15d3c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:46408/0 (socket says 172.21.15.43:46408) 2023-08-09T15:08:18.714 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:18.712+0000 7fc41b7fe640 1 -- 172.21.15.43:0/3914162553 learned_addr learned my addr 172.21.15.43:0/3914162553 (peer_addr_for_me v2:172.21.15.43:0/0) 2023-08-09T15:08:18.715 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:18.712+0000 7fc41bfff640 1 --2- 172.21.15.43:0/3914162553 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fc41c06c510 0x7fc41c06c960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T15:08:18.715 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:18.712+0000 7fc41b7fe640 1 -- 172.21.15.43:0/3914162553 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fc41c06c510 msgr2=0x7fc41c06c960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-09T15:08:18.716 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:18.712+0000 7fc41b7fe640 1 --2- 172.21.15.43:0/3914162553 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fc41c06c510 0x7fc41c06c960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T15:08:18.716 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:18.712+0000 7fc41b7fe640 1 -- 172.21.15.43:0/3914162553 >> v1:172.21.15.203:6789/0 conn(0x7fc41c0c86c0 legacy=0x7fc41c159530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-09T15:08:18.716 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:18.712+0000 7fc41b7fe640 1 -- 172.21.15.43:0/3914162553 --> v2:172.21.15.43:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc41c159ea0 con 0x7fc41c0c99b0 2023-08-09T15:08:18.718 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:18.713+0000 7fc41b7fe640 1 --2- 172.21.15.43:0/3914162553 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc41c0c99b0 0x7fc41c15d3c0 secure :-1 s=READY pgs=2094 cs=0 l=1 rev1=1 crypto rx=0x7fc40c009870 tx=0x7fc40c02b8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7ad3c89be0e19754 server_cookie=0 in_seq=0 out_seq=0 2023-08-09T15:08:18.718 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:18.714+0000 7fc41a7fc640 1 -- 172.21.15.43:0/3914162553 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc40c031640 con 0x7fc41c0c99b0 2023-08-09T15:08:18.718 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:18.714+0000 7fc41a7fc640 1 -- 172.21.15.43:0/3914162553 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc40c0317a0 con 0x7fc41c0c99b0 2023-08-09T15:08:18.719 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:18.714+0000 7fc41a7fc640 1 -- 172.21.15.43:0/3914162553 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc40c031b30 con 0x7fc41c0c99b0 2023-08-09T15:08:18.719 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:18.714+0000 7fc42080f640 1 -- 172.21.15.43:0/3914162553 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc41c0c99b0 msgr2=0x7fc41c15d3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T15:08:18.719 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:18.714+0000 7fc42080f640 1 --2- 172.21.15.43:0/3914162553 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc41c0c99b0 0x7fc41c15d3c0 secure :-1 s=READY pgs=2094 cs=0 l=1 rev1=1 crypto rx=0x7fc40c009870 tx=0x7fc40c02b8f0 comp rx=0 tx=0).stop 2023-08-09T15:08:18.721 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:18.714+0000 7fc42080f640 1 -- 172.21.15.43:0/3914162553 shutdown_connections 2023-08-09T15:08:18.721 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:18.714+0000 7fc42080f640 1 --2- 172.21.15.43:0/3914162553 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fc41c06c510 0x7fc41c06c960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T15:08:18.721 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:18.714+0000 7fc42080f640 1 --2- 172.21.15.43:0/3914162553 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc41c0c99b0 0x7fc41c15d3c0 unknown :-1 s=CLOSED pgs=2094 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T15:08:18.722 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:18.714+0000 7fc42080f640 1 -- 172.21.15.43:0/3914162553 >> 172.21.15.43:0/3914162553 conn(0x7fc41c0ca870 msgr2=0x7fc41c0cac50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-09T15:08:18.722 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:18.715+0000 7fc42080f640 1 -- 172.21.15.43:0/3914162553 shutdown_connections 2023-08-09T15:08:18.722 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:18.715+0000 7fc42080f640 1 -- 172.21.15.43:0/3914162553 wait complete. 2023-08-09T15:08:18.722 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:18.716+0000 7fc42080f640 1 Processor -- start 2023-08-09T15:08:18.722 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:18.717+0000 7fc42080f640 1 -- start start 2023-08-09T15:08:18.722 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:18.717+0000 7fc42080f640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fc41c06c510 0x7fc41c0c4410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T15:08:18.722 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:18.718+0000 7fc42080f640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc41c0c86c0 0x7fc41c0c4ec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T15:08:18.723 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:18.718+0000 7fc42080f640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fc41c0c99b0 0x7fc41c154470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T15:08:18.723 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:18.718+0000 7fc42080f640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc41c0c3190 con 0x7fc41c0c86c0 2023-08-09T15:08:18.723 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:18.718+0000 7fc42080f640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc41c0c5400 con 0x7fc41c06c510 2023-08-09T15:08:18.723 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:18.718+0000 7fc42080f640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc41c0b06d0 con 0x7fc41c0c99b0 2023-08-09T15:08:18.723 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:18.718+0000 7fc41affd640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc41c0c86c0 0x7fc41c0c4ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T15:08:18.723 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:18.718+0000 7fc41affd640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc41c0c86c0 0x7fc41c0c4ec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:46424/0 (socket says 172.21.15.43:46424) 2023-08-09T15:08:18.723 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:18.718+0000 7fc41affd640 1 -- 172.21.15.43:0/1919020739 learned_addr learned my addr 172.21.15.43:0/1919020739 (peer_addr_for_me v2:172.21.15.43:0/0) 2023-08-09T15:08:18.724 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:18.718+0000 7fc41b7fe640 1 --2- 172.21.15.43:0/1919020739 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fc41c06c510 0x7fc41c0c4410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T15:08:18.724 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:18.718+0000 7fc41bfff640 1 --2- 172.21.15.43:0/1919020739 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fc41c0c99b0 0x7fc41c154470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T15:08:18.724 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:18.718+0000 7fc41affd640 1 -- 172.21.15.43:0/1919020739 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fc41c0c99b0 msgr2=0x7fc41c154470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T15:08:18.724 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:18.718+0000 7fc41affd640 1 --2- 172.21.15.43:0/1919020739 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fc41c0c99b0 0x7fc41c154470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T15:08:18.724 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:18.718+0000 7fc41affd640 1 -- 172.21.15.43:0/1919020739 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fc41c06c510 msgr2=0x7fc41c0c4410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T15:08:18.724 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:18.718+0000 7fc41affd640 1 --2- 172.21.15.43:0/1919020739 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fc41c06c510 0x7fc41c0c4410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T15:08:18.724 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:18.718+0000 7fc41affd640 1 -- 172.21.15.43:0/1919020739 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc40c0094d0 con 0x7fc41c0c86c0 2023-08-09T15:08:18.724 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:18.718+0000 7fc41b7fe640 1 --2- 172.21.15.43:0/1919020739 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fc41c06c510 0x7fc41c0c4410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-08-09T15:08:18.725 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:18.719+0000 7fc41affd640 1 --2- 172.21.15.43:0/1919020739 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc41c0c86c0 0x7fc41c0c4ec0 secure :-1 s=READY pgs=2095 cs=0 l=1 rev1=1 crypto rx=0x7fc41000b420 tx=0x7fc41000b8e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-09T15:08:18.725 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:18.719+0000 7fc403fff640 1 -- 172.21.15.43:0/1919020739 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc410011830 con 0x7fc41c0c86c0 2023-08-09T15:08:18.725 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:18.719+0000 7fc403fff640 1 -- 172.21.15.43:0/1919020739 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc410011990 con 0x7fc41c0c86c0 2023-08-09T15:08:18.725 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:18.719+0000 7fc403fff640 1 -- 172.21.15.43:0/1919020739 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc410011d40 con 0x7fc41c0c86c0 2023-08-09T15:08:18.725 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:18.719+0000 7fc42080f640 1 -- 172.21.15.43:0/1919020739 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc41c0b03f0 con 0x7fc41c0c86c0 2023-08-09T15:08:18.725 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:18.719+0000 7fc42080f640 1 -- 172.21.15.43:0/1919020739 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc41c0c2e20 con 0x7fc41c0c86c0 2023-08-09T15:08:18.726 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:18.720+0000 7fc42080f640 1 -- 172.21.15.43:0/1919020739 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc41c0c3340 con 0x7fc41c0c86c0 2023-08-09T15:08:18.726 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:18.721+0000 7fc403fff640 1 -- 172.21.15.43:0/1919020739 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 101) v1 ==== 88472+0+0 (secure 0 0 0) 0x7fc41001b460 con 0x7fc41c0c86c0 2023-08-09T15:08:18.726 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:18.722+0000 7fc403fff640 1 --2- 172.21.15.43:0/1919020739 >> [v2:172.21.15.203:6832/115830229,v1:172.21.15.203:6833/115830229] conn(0x7fc40806a1c0 0x7fc40806c680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T15:08:18.727 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:18.722+0000 7fc403fff640 1 -- 172.21.15.43:0/1919020739 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(188..188 src has 1..188) v4 ==== 7114+0+0 (secure 0 0 0) 0x7fc410088740 con 0x7fc41c0c86c0 2023-08-09T15:08:18.727 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:18.723+0000 7fc41b7fe640 1 --2- 172.21.15.43:0/1919020739 >> [v2:172.21.15.203:6832/115830229,v1:172.21.15.203:6833/115830229] conn(0x7fc40806a1c0 0x7fc40806c680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T15:08:18.727 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:18.724+0000 7fc41b7fe640 1 --2- 172.21.15.43:0/1919020739 >> [v2:172.21.15.203:6832/115830229,v1:172.21.15.203:6833/115830229] conn(0x7fc40806a1c0 0x7fc40806c680 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7fc40c036000 tx=0x7fc40c03c040 comp rx=0 tx=0).ready entity=mgr.7487 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-09T15:08:18.728 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:18.727+0000 7fc403fff640 1 -- 172.21.15.43:0/1919020739 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178066 (secure 0 0 0) 0x7fc41c0c3340 con 0x7fc41c0c86c0 2023-08-09T15:08:18.963 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:18.962+0000 7fc42080f640 1 -- 172.21.15.43:0/1919020739 --> [v2:172.21.15.203:6832/115830229,v1:172.21.15.203:6833/115830229] -- mgr_command(tid 0: {"prefix": "fs snap-schedule remove", "fs": "cephfs", "path": "/snap_test_dir1/test_restart", "repeat": "1M", "target": ["mon-mgr", ""]}) v1 -- 0x7fc41c15a460 con 0x7fc40806a1c0 2023-08-09T15:08:19.127 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:19.126+0000 7fc403fff640 1 -- 172.21.15.43:0/1919020739 <== mgr.7487 v2:172.21.15.203:6832/115830229 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+55 (secure 0 0 0) 0x7fc40c031ad0 con 0x7fc40806a1c0 2023-08-09T15:08:19.127 INFO:teuthology.orchestra.run.smithi043.stdout:Schedule removed for path /snap_test_dir1/test_restart 2023-08-09T15:08:19.130 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:19.129+0000 7fc42080f640 1 -- 172.21.15.43:0/1919020739 >> [v2:172.21.15.203:6832/115830229,v1:172.21.15.203:6833/115830229] conn(0x7fc40806a1c0 msgr2=0x7fc40806c680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T15:08:19.130 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:19.129+0000 7fc42080f640 1 --2- 172.21.15.43:0/1919020739 >> [v2:172.21.15.203:6832/115830229,v1:172.21.15.203:6833/115830229] conn(0x7fc40806a1c0 0x7fc40806c680 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7fc40c036000 tx=0x7fc40c03c040 comp rx=0 tx=0).stop 2023-08-09T15:08:19.130 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:19.129+0000 7fc42080f640 1 -- 172.21.15.43:0/1919020739 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc41c0c86c0 msgr2=0x7fc41c0c4ec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T15:08:19.131 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:19.129+0000 7fc42080f640 1 --2- 172.21.15.43:0/1919020739 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc41c0c86c0 0x7fc41c0c4ec0 secure :-1 s=READY pgs=2095 cs=0 l=1 rev1=1 crypto rx=0x7fc41000b420 tx=0x7fc41000b8e0 comp rx=0 tx=0).stop 2023-08-09T15:08:19.131 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:19.129+0000 7fc42080f640 1 -- 172.21.15.43:0/1919020739 shutdown_connections 2023-08-09T15:08:19.131 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:19.129+0000 7fc42080f640 1 --2- 172.21.15.43:0/1919020739 >> [v2:172.21.15.203:6832/115830229,v1:172.21.15.203:6833/115830229] conn(0x7fc40806a1c0 0x7fc40806c680 unknown :-1 s=CLOSED pgs=25 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T15:08:19.131 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:19.129+0000 7fc42080f640 1 --2- 172.21.15.43:0/1919020739 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fc41c0c99b0 0x7fc41c154470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T15:08:19.131 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:19.129+0000 7fc42080f640 1 --2- 172.21.15.43:0/1919020739 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc41c0c86c0 0x7fc41c0c4ec0 unknown :-1 s=CLOSED pgs=2095 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T15:08:19.131 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:19.129+0000 7fc42080f640 1 --2- 172.21.15.43:0/1919020739 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fc41c06c510 0x7fc41c0c4410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T15:08:19.131 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:19.129+0000 7fc42080f640 1 -- 172.21.15.43:0/1919020739 >> 172.21.15.43:0/1919020739 conn(0x7fc41c0ca870 msgr2=0x7fc41c159160 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-09T15:08:19.132 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:19.129+0000 7fc42080f640 1 -- 172.21.15.43:0/1919020739 shutdown_connections 2023-08-09T15:08:19.132 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:19.129+0000 7fc42080f640 1 -- 172.21.15.43:0/1919020739 wait complete. 2023-08-09T15:08:19.151 DEBUG:tasks.cephfs.test_snap_schedules:res=Schedule removed for path /snap_test_dir1/test_restart 2023-08-09T15:08:19.152 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-09T15:08:19.152 DEBUG:teuthology.orchestra.run.smithi043:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/test_restart/.snap) 2023-08-09T15:08:19.217 INFO:teuthology.orchestra.run.smithi043.stdout:scheduled-2023-08-09-14_53_00_UTC 2023-08-09T15:08:19.217 INFO:teuthology.orchestra.run.smithi043.stdout:scheduled-2023-08-09-14_54_00_UTC 2023-08-09T15:08:19.217 INFO:teuthology.orchestra.run.smithi043.stdout:scheduled-2023-08-09-14_55_00_UTC 2023-08-09T15:08:19.217 INFO:teuthology.orchestra.run.smithi043.stdout:scheduled-2023-08-09-14_56_00_UTC 2023-08-09T15:08:19.217 INFO:teuthology.orchestra.run.smithi043.stdout:scheduled-2023-08-09-14_57_00_UTC 2023-08-09T15:08:19.218 INFO:teuthology.orchestra.run.smithi043.stdout:scheduled-2023-08-09-14_58_00_UTC 2023-08-09T15:08:19.218 INFO:teuthology.orchestra.run.smithi043.stdout:scheduled-2023-08-09-14_59_00_UTC 2023-08-09T15:08:19.218 INFO:teuthology.orchestra.run.smithi043.stdout:scheduled-2023-08-09-15_00_00_UTC 2023-08-09T15:08:19.218 INFO:teuthology.orchestra.run.smithi043.stdout:scheduled-2023-08-09-15_01_00_UTC 2023-08-09T15:08:19.218 INFO:teuthology.orchestra.run.smithi043.stdout:scheduled-2023-08-09-15_02_00_UTC 2023-08-09T15:08:19.218 INFO:teuthology.orchestra.run.smithi043.stdout:scheduled-2023-08-09-15_03_00_UTC 2023-08-09T15:08:19.219 INFO:teuthology.orchestra.run.smithi043.stdout:scheduled-2023-08-09-15_04_00_UTC 2023-08-09T15:08:19.219 INFO:teuthology.orchestra.run.smithi043.stdout:scheduled-2023-08-09-15_05_00_UTC 2023-08-09T15:08:19.219 INFO:teuthology.orchestra.run.smithi043.stdout:scheduled-2023-08-09-15_06_00_UTC 2023-08-09T15:08:19.219 INFO:teuthology.orchestra.run.smithi043.stdout:scheduled-2023-08-09-15_07_00_UTC 2023-08-09T15:08:19.219 INFO:teuthology.orchestra.run.smithi043.stdout:scheduled-2023-08-09-15_08_00_UTC 2023-08-09T15:08:19.220 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/test_restart/.snap/scheduled-2023-08-09-14_53_00_UTC 2023-08-09T15:08:19.220 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-09T15:08:19.221 DEBUG:teuthology.orchestra.run.smithi043:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/test_restart/.snap/scheduled-2023-08-09-14_53_00_UTC) 2023-08-09T15:08:19.300 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/test_restart/.snap/scheduled-2023-08-09-14_54_00_UTC 2023-08-09T15:08:19.300 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-09T15:08:19.301 DEBUG:teuthology.orchestra.run.smithi043:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/test_restart/.snap/scheduled-2023-08-09-14_54_00_UTC) 2023-08-09T15:08:19.385 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/test_restart/.snap/scheduled-2023-08-09-14_55_00_UTC 2023-08-09T15:08:19.385 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-09T15:08:19.386 DEBUG:teuthology.orchestra.run.smithi043:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/test_restart/.snap/scheduled-2023-08-09-14_55_00_UTC) 2023-08-09T15:08:19.468 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/test_restart/.snap/scheduled-2023-08-09-14_56_00_UTC 2023-08-09T15:08:19.468 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-09T15:08:19.469 DEBUG:teuthology.orchestra.run.smithi043:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/test_restart/.snap/scheduled-2023-08-09-14_56_00_UTC) 2023-08-09T15:08:19.547 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/test_restart/.snap/scheduled-2023-08-09-14_57_00_UTC 2023-08-09T15:08:19.547 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-09T15:08:19.548 DEBUG:teuthology.orchestra.run.smithi043:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/test_restart/.snap/scheduled-2023-08-09-14_57_00_UTC) 2023-08-09T15:08:19.627 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/test_restart/.snap/scheduled-2023-08-09-14_58_00_UTC 2023-08-09T15:08:19.628 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-09T15:08:19.629 DEBUG:teuthology.orchestra.run.smithi043:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/test_restart/.snap/scheduled-2023-08-09-14_58_00_UTC) 2023-08-09T15:08:19.707 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/test_restart/.snap/scheduled-2023-08-09-14_59_00_UTC 2023-08-09T15:08:19.708 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-09T15:08:19.708 DEBUG:teuthology.orchestra.run.smithi043:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/test_restart/.snap/scheduled-2023-08-09-14_59_00_UTC) 2023-08-09T15:08:19.791 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/test_restart/.snap/scheduled-2023-08-09-15_00_00_UTC 2023-08-09T15:08:19.791 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-09T15:08:19.792 DEBUG:teuthology.orchestra.run.smithi043:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/test_restart/.snap/scheduled-2023-08-09-15_00_00_UTC) 2023-08-09T15:08:19.880 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/test_restart/.snap/scheduled-2023-08-09-15_01_00_UTC 2023-08-09T15:08:19.880 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-09T15:08:19.881 DEBUG:teuthology.orchestra.run.smithi043:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/test_restart/.snap/scheduled-2023-08-09-15_01_00_UTC) 2023-08-09T15:08:19.958 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/test_restart/.snap/scheduled-2023-08-09-15_02_00_UTC 2023-08-09T15:08:19.958 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-09T15:08:19.959 DEBUG:teuthology.orchestra.run.smithi043:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/test_restart/.snap/scheduled-2023-08-09-15_02_00_UTC) 2023-08-09T15:08:20.033 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/test_restart/.snap/scheduled-2023-08-09-15_03_00_UTC 2023-08-09T15:08:20.034 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-09T15:08:20.034 DEBUG:teuthology.orchestra.run.smithi043:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/test_restart/.snap/scheduled-2023-08-09-15_03_00_UTC) 2023-08-09T15:08:20.112 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/test_restart/.snap/scheduled-2023-08-09-15_04_00_UTC 2023-08-09T15:08:20.113 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-09T15:08:20.113 DEBUG:teuthology.orchestra.run.smithi043:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/test_restart/.snap/scheduled-2023-08-09-15_04_00_UTC) 2023-08-09T15:08:20.192 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/test_restart/.snap/scheduled-2023-08-09-15_05_00_UTC 2023-08-09T15:08:20.192 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-09T15:08:20.193 DEBUG:teuthology.orchestra.run.smithi043:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/test_restart/.snap/scheduled-2023-08-09-15_05_00_UTC) 2023-08-09T15:08:20.272 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/test_restart/.snap/scheduled-2023-08-09-15_06_00_UTC 2023-08-09T15:08:20.272 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-09T15:08:20.272 DEBUG:teuthology.orchestra.run.smithi043:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/test_restart/.snap/scheduled-2023-08-09-15_06_00_UTC) 2023-08-09T15:08:20.356 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/test_restart/.snap/scheduled-2023-08-09-15_07_00_UTC 2023-08-09T15:08:20.357 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-09T15:08:20.357 DEBUG:teuthology.orchestra.run.smithi043:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/test_restart/.snap/scheduled-2023-08-09-15_07_00_UTC) 2023-08-09T15:08:20.449 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/test_restart/.snap/scheduled-2023-08-09-15_08_00_UTC 2023-08-09T15:08:20.449 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-09T15:08:20.450 DEBUG:teuthology.orchestra.run.smithi043:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/test_restart/.snap/scheduled-2023-08-09-15_08_00_UTC) 2023-08-09T15:08:20.521 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-09T15:08:20.522 DEBUG:teuthology.orchestra.run.smithi043:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/test_restart) 2023-08-09T15:08:20.588 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph mgr module disable snap_schedule 2023-08-09T15:08:20.820 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:20.819+0000 7f84fe9a2640 1 Processor -- start 2023-08-09T15:08:20.821 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:20.820+0000 7f84fe9a2640 1 -- start start 2023-08-09T15:08:20.822 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:20.821+0000 7f84fe9a2640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f84f815c3a0 0x7f84f81608a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T15:08:20.822 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:20.821+0000 7f84fe9a2640 1 -- --> v1:172.21.15.43:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f84f8160de0 con 0x7f84f80cd600 2023-08-09T15:08:20.822 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:20.821+0000 7f84fe9a2640 1 -- --> v1:172.21.15.203:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f84f80b7c00 con 0x7f84f80b0f00 2023-08-09T15:08:20.823 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:20.821+0000 7f84fe9a2640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f84f80b81c0 con 0x7f84f815c3a0 2023-08-09T15:08:20.824 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:20.822+0000 7f84fe1a1640 1 --1- >> v1:172.21.15.43:6789/0 conn(0x7f84f80cd600 0x7f84f80d0580 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.43:6789/0 says I am v1:172.21.15.43:47334/0 (socket says 172.21.15.43:47334) 2023-08-09T15:08:20.824 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:20.822+0000 7f84fe1a1640 1 -- 172.21.15.43:0/7314619 learned_addr learned my addr 172.21.15.43:0/7314619 (peer_addr_for_me v1:172.21.15.43:0/0) 2023-08-09T15:08:20.824 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:20.822+0000 7f84fd9a0640 1 --2- 172.21.15.43:0/7314619 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f84f815c3a0 0x7f84f81608a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T15:08:20.824 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:20.822+0000 7f84fc99e640 1 -- 172.21.15.43:0/7314619 <== mon.0 v1:172.21.15.43:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 732174644 0 0) 0x7f84f8160de0 con 0x7f84f80cd600 2023-08-09T15:08:20.824 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:20.822+0000 7f84fc99e640 1 -- 172.21.15.43:0/7314619 --> v1:172.21.15.43:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f84e0003520 con 0x7f84f80cd600 2023-08-09T15:08:20.825 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:20.823+0000 7f84fc99e640 1 -- 172.21.15.43:0/7314619 <== mon.0 v1:172.21.15.43:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3812848635 0 0) 0x7f84e0003520 con 0x7f84f80cd600 2023-08-09T15:08:20.825 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:20.823+0000 7f84fc99e640 1 -- 172.21.15.43:0/7314619 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f84f815c3a0 msgr2=0x7f84f81608a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-09T15:08:20.825 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:20.823+0000 7f84fc99e640 1 --2- 172.21.15.43:0/7314619 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f84f815c3a0 0x7f84f81608a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T15:08:20.825 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:20.823+0000 7f84fc99e640 1 -- 172.21.15.43:0/7314619 >> v1:172.21.15.203:6789/0 conn(0x7f84f80b0f00 legacy=0x7f84f80ccef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T15:08:20.825 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:20.823+0000 7f84fc99e640 1 -- 172.21.15.43:0/7314619 --> v1:172.21.15.43:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f84f80b7ee0 con 0x7f84f80cd600 2023-08-09T15:08:20.825 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:20.823+0000 7f84fc99e640 1 -- 172.21.15.43:0/7314619 <== mon.0 v1:172.21.15.43:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1285150740 0 0) 0x7f84f00019f0 con 0x7f84f80cd600 2023-08-09T15:08:20.825 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:20.823+0000 7f84fc99e640 1 -- 172.21.15.43:0/7314619 >> v1:172.21.15.43:6789/0 conn(0x7f84f80cd600 legacy=0x7f84f80d0580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T15:08:20.826 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:20.823+0000 7f84fc99e640 1 --2- 172.21.15.43:0/7314619 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f84e00039a0 0x7f84e0003dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T15:08:20.826 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:20.823+0000 7f84fc99e640 1 --2- 172.21.15.43:0/7314619 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f84e0004390 0x7f84e0004860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T15:08:20.826 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:20.823+0000 7f84fc99e640 1 --2- 172.21.15.43:0/7314619 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f84e0005100 0x7f84e000b980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T15:08:20.826 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:20.823+0000 7f84fc99e640 1 -- 172.21.15.43:0/7314619 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f84f80b81c0 con 0x7f84e0005100 2023-08-09T15:08:20.826 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:20.823+0000 7f84fc99e640 1 -- 172.21.15.43:0/7314619 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f84e000c020 con 0x7f84e0004390 2023-08-09T15:08:20.826 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:20.823+0000 7f84fc99e640 1 -- 172.21.15.43:0/7314619 --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f84e000c1e0 con 0x7f84e00039a0 2023-08-09T15:08:20.826 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:20.823+0000 7f84fe1a1640 1 --2- 172.21.15.43:0/7314619 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f84e0005100 0x7f84e000b980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T15:08:20.827 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:20.823+0000 7f84fd19f640 1 --2- 172.21.15.43:0/7314619 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f84e0004390 0x7f84e0004860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T15:08:20.828 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:20.823+0000 7f84fd9a0640 1 --2- 172.21.15.43:0/7314619 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f84e00039a0 0x7f84e0003dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T15:08:20.828 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:20.823+0000 7f84fe1a1640 1 -- 172.21.15.43:0/7314619 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f84e00039a0 msgr2=0x7f84e0003dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-09T15:08:20.828 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:20.823+0000 7f84fe1a1640 1 --2- 172.21.15.43:0/7314619 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f84e00039a0 0x7f84e0003dc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T15:08:20.828 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:20.823+0000 7f84fe1a1640 1 -- 172.21.15.43:0/7314619 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f84e0004390 msgr2=0x7f84e0004860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-09T15:08:20.828 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:20.823+0000 7f84fe1a1640 1 --2- 172.21.15.43:0/7314619 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f84e0004390 0x7f84e0004860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T15:08:20.828 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:20.823+0000 7f84fe1a1640 1 -- 172.21.15.43:0/7314619 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f84e0003520 con 0x7f84e0005100 2023-08-09T15:08:20.829 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:20.823+0000 7f84fd19f640 1 --2- 172.21.15.43:0/7314619 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f84e0004390 0x7f84e0004860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-08-09T15:08:20.829 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:20.825+0000 7f84fe1a1640 1 --2- 172.21.15.43:0/7314619 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f84e0005100 0x7f84e000b980 secure :-1 s=READY pgs=2097 cs=0 l=1 rev1=1 crypto rx=0x7f84f000a800 tx=0x7f84e00057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9af6add8064632ef server_cookie=0 in_seq=0 out_seq=0 2023-08-09T15:08:20.829 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:20.825+0000 7f84fc99e640 1 -- 172.21.15.43:0/7314619 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f84f00072d0 con 0x7f84e0005100 2023-08-09T15:08:20.829 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:20.825+0000 7f84fc99e640 1 -- 172.21.15.43:0/7314619 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f84f002f070 con 0x7f84e0005100 2023-08-09T15:08:20.829 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:20.825+0000 7f84fc99e640 1 -- 172.21.15.43:0/7314619 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f84f00076f0 con 0x7f84e0005100 2023-08-09T15:08:20.829 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:20.826+0000 7f84fe9a2640 1 -- 172.21.15.43:0/7314619 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f84e0005100 msgr2=0x7f84e000b980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T15:08:20.829 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:20.826+0000 7f84fe9a2640 1 --2- 172.21.15.43:0/7314619 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f84e0005100 0x7f84e000b980 secure :-1 s=READY pgs=2097 cs=0 l=1 rev1=1 crypto rx=0x7f84f000a800 tx=0x7f84e00057f0 comp rx=0 tx=0).stop 2023-08-09T15:08:20.830 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:20.826+0000 7f84fe1a1640 1 -- 172.21.15.43:0/7314619 reap_dead start 2023-08-09T15:08:20.830 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:20.826+0000 7f84fe9a2640 1 -- 172.21.15.43:0/7314619 shutdown_connections 2023-08-09T15:08:20.830 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:20.826+0000 7f84fe9a2640 1 -- 172.21.15.43:0/7314619 >> 172.21.15.43:0/7314619 conn(0x7f84f806a210 msgr2=0x7f84f806a620 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-09T15:08:20.830 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:20.826+0000 7f84fe9a2640 1 -- 172.21.15.43:0/7314619 shutdown_connections 2023-08-09T15:08:20.830 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:20.826+0000 7f84fe9a2640 1 -- 172.21.15.43:0/7314619 wait complete. 2023-08-09T15:08:20.831 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:20.828+0000 7f84fe9a2640 1 Processor -- start 2023-08-09T15:08:20.831 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:20.828+0000 7f84fe9a2640 1 -- start start 2023-08-09T15:08:20.831 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:20.830+0000 7f84fe9a2640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f84e0005100 0x7f84f815e520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T15:08:20.831 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:20.830+0000 7f84fe9a2640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f84f80d0070 0x7f84f80d04f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T15:08:20.832 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:20.830+0000 7f84fe9a2640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f84f815d000 0x7f84f815d480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T15:08:20.832 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:20.830+0000 7f84fe9a2640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f84f80d0a30 con 0x7f84f815d000 2023-08-09T15:08:20.832 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:20.830+0000 7f84fe9a2640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f84f815d9c0 con 0x7f84e0005100 2023-08-09T15:08:20.832 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:20.830+0000 7f84fe9a2640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f84f80b1430 con 0x7f84f80d0070 2023-08-09T15:08:20.833 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:20.830+0000 7f84fe1a1640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f84f815d000 0x7f84f815d480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T15:08:20.833 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:20.830+0000 7f84fe1a1640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f84f815d000 0x7f84f815d480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:46444/0 (socket says 172.21.15.43:46444) 2023-08-09T15:08:20.833 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:20.830+0000 7f84fe1a1640 1 -- 172.21.15.43:0/1100793640 learned_addr learned my addr 172.21.15.43:0/1100793640 (peer_addr_for_me v2:172.21.15.43:0/0) 2023-08-09T15:08:20.834 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:20.830+0000 7f84fd9a0640 1 --2- 172.21.15.43:0/1100793640 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f84e0005100 0x7f84f815e520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T15:08:20.834 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:20.831+0000 7f84fd19f640 1 --2- 172.21.15.43:0/1100793640 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f84f80d0070 0x7f84f80d04f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T15:08:20.834 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:20.831+0000 7f84fe1a1640 1 -- 172.21.15.43:0/1100793640 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f84f80d0070 msgr2=0x7f84f80d04f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T15:08:20.834 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:20.831+0000 7f84fe1a1640 1 --2- 172.21.15.43:0/1100793640 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f84f80d0070 0x7f84f80d04f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T15:08:20.834 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:20.831+0000 7f84fe1a1640 1 -- 172.21.15.43:0/1100793640 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f84e0005100 msgr2=0x7f84f815e520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T15:08:20.834 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:20.831+0000 7f84fe1a1640 1 --2- 172.21.15.43:0/1100793640 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f84e0005100 0x7f84f815e520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T15:08:20.834 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:20.831+0000 7f84fe1a1640 1 -- 172.21.15.43:0/1100793640 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f84f815ee90 con 0x7f84f815d000 2023-08-09T15:08:20.835 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:20.831+0000 7f84fe1a1640 1 --2- 172.21.15.43:0/1100793640 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f84f815d000 0x7f84f815d480 secure :-1 s=READY pgs=2098 cs=0 l=1 rev1=1 crypto rx=0x7f84f002cb50 tx=0x7f84f0001d80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-09T15:08:20.835 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:20.831+0000 7f84f67fc640 1 -- 172.21.15.43:0/1100793640 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f84f0004d60 con 0x7f84f815d000 2023-08-09T15:08:20.835 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:20.831+0000 7f84f67fc640 1 -- 172.21.15.43:0/1100793640 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f84f002ce90 con 0x7f84f815d000 2023-08-09T15:08:20.835 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:20.831+0000 7f84f67fc640 1 -- 172.21.15.43:0/1100793640 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f84f00072f0 con 0x7f84f815d000 2023-08-09T15:08:20.835 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:20.831+0000 7f84fe9a2640 1 -- 172.21.15.43:0/1100793640 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f84f815f170 con 0x7f84f815d000 2023-08-09T15:08:20.835 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:20.832+0000 7f84fe9a2640 1 -- 172.21.15.43:0/1100793640 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f84f815f730 con 0x7f84f815d000 2023-08-09T15:08:20.836 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:20.833+0000 7f84fe9a2640 1 -- 172.21.15.43:0/1100793640 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f84f804e390 con 0x7f84f815d000 2023-08-09T15:08:20.836 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:20.834+0000 7f84f67fc640 1 -- 172.21.15.43:0/1100793640 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 101) v1 ==== 88472+0+0 (secure 0 0 0) 0x7f84f0031030 con 0x7f84f815d000 2023-08-09T15:08:20.836 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:20.835+0000 7f84f67fc640 1 --2- 172.21.15.43:0/1100793640 >> [v2:172.21.15.203:6832/115830229,v1:172.21.15.203:6833/115830229] conn(0x7f84e406a1c0 0x7f84e406c680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T15:08:20.836 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:20.835+0000 7f84f67fc640 1 -- 172.21.15.43:0/1100793640 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(189..189 src has 1..189) v4 ==== 7202+0+0 (secure 0 0 0) 0x7f84f00a9c50 con 0x7f84f815d000 2023-08-09T15:08:20.837 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:20.835+0000 7f84fd9a0640 1 --2- 172.21.15.43:0/1100793640 >> [v2:172.21.15.203:6832/115830229,v1:172.21.15.203:6833/115830229] conn(0x7f84e406a1c0 0x7f84e406c680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T15:08:20.837 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:20.837+0000 7f84fd9a0640 1 --2- 172.21.15.43:0/1100793640 >> [v2:172.21.15.203:6832/115830229,v1:172.21.15.203:6833/115830229] conn(0x7f84e406a1c0 0x7f84e406c680 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7f84f80cfd30 tx=0x7f84e8008040 comp rx=0 tx=0).ready entity=mgr.7487 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-09T15:08:20.840 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:20.839+0000 7f84f67fc640 1 -- 172.21.15.43:0/1100793640 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178066 (secure 0 0 0) 0x7f84f0074c90 con 0x7f84f815d000 2023-08-09T15:08:21.083 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:21.082+0000 7f84fe9a2640 1 -- 172.21.15.43:0/1100793640 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "mgr module disable", "module": "snap_schedule"} v 0) v1 -- 0x7f84f80cfe50 con 0x7f84f815d000 2023-08-09T15:08:21.369 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:21.367+0000 7f84f67fc640 1 -- 172.21.15.43:0/1100793640 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mgrmap(e 102) v1 ==== 88455+0+0 (secure 0 0 0) 0x7f84f0075060 con 0x7f84f815d000 2023-08-09T15:08:21.371 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:21.370+0000 7f84f67fc640 1 -- 172.21.15.43:0/1100793640 <== mon.0 v2:172.21.15.43:3300/0 8 ==== mon_command_ack([{"prefix": "mgr module disable", "module": "snap_schedule"}]=0 v102) v1 ==== 93+0+0 (secure 0 0 0) 0x7f84f0078720 con 0x7f84f815d000 2023-08-09T15:08:21.375 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:21.373+0000 7f84fe9a2640 1 -- 172.21.15.43:0/1100793640 >> [v2:172.21.15.203:6832/115830229,v1:172.21.15.203:6833/115830229] conn(0x7f84e406a1c0 msgr2=0x7f84e406c680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T15:08:21.375 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:21.373+0000 7f84fe9a2640 1 --2- 172.21.15.43:0/1100793640 >> [v2:172.21.15.203:6832/115830229,v1:172.21.15.203:6833/115830229] conn(0x7f84e406a1c0 0x7f84e406c680 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7f84f80cfd30 tx=0x7f84e8008040 comp rx=0 tx=0).stop 2023-08-09T15:08:21.375 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:21.373+0000 7f84fe9a2640 1 -- 172.21.15.43:0/1100793640 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f84f815d000 msgr2=0x7f84f815d480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T15:08:21.375 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:21.373+0000 7f84fe9a2640 1 --2- 172.21.15.43:0/1100793640 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f84f815d000 0x7f84f815d480 secure :-1 s=READY pgs=2098 cs=0 l=1 rev1=1 crypto rx=0x7f84f002cb50 tx=0x7f84f0001d80 comp rx=0 tx=0).stop 2023-08-09T15:08:21.375 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:21.373+0000 7f84fe9a2640 1 -- 172.21.15.43:0/1100793640 shutdown_connections 2023-08-09T15:08:21.375 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:21.373+0000 7f84fe9a2640 1 --2- 172.21.15.43:0/1100793640 >> [v2:172.21.15.203:6832/115830229,v1:172.21.15.203:6833/115830229] conn(0x7f84e406a1c0 0x7f84e406c680 unknown :-1 s=CLOSED pgs=26 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T15:08:21.376 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:21.373+0000 7f84fe9a2640 1 --2- 172.21.15.43:0/1100793640 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f84f815d000 0x7f84f815d480 unknown :-1 s=CLOSED pgs=2098 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T15:08:21.376 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:21.373+0000 7f84fe9a2640 1 --2- 172.21.15.43:0/1100793640 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f84f80d0070 0x7f84f80d04f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T15:08:21.376 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:21.373+0000 7f84fe9a2640 1 --2- 172.21.15.43:0/1100793640 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f84e0005100 0x7f84f815e520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T15:08:21.376 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:21.373+0000 7f84fe9a2640 1 -- 172.21.15.43:0/1100793640 >> 172.21.15.43:0/1100793640 conn(0x7f84f806a210 msgr2=0x7f84f815b9b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-09T15:08:21.376 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:21.374+0000 7f84fe9a2640 1 -- 172.21.15.43:0/1100793640 shutdown_connections 2023-08-09T15:08:21.377 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:21.374+0000 7f84fe9a2640 1 -- 172.21.15.43:0/1100793640 wait complete. 2023-08-09T15:08:21.405 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2023-08-09T15:08:21.406 DEBUG:teuthology.orchestra.run.smithi043:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2023-08-09T15:08:21.517 DEBUG:teuthology.orchestra.run.smithi203:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2023-08-09T15:08:21.613 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2023-08-09T15:08:21.614 DEBUG:teuthology.orchestra.run.smithi043:> set -ex 2023-08-09T15:08:21.614 DEBUG:teuthology.orchestra.run.smithi043:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-09T15:08:21.615 INFO:tasks.ceph.mgr.x.smithi203.stderr:2023-08-09T15:08:21.613+0000 7fbc59c0f040 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2023-08-09T15:08:21.627 INFO:tasks.ceph.mgr.y.smithi043.stderr:2023-08-09T15:08:21.626+0000 7fe393e53040 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2023-08-09T15:08:21.670 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-08-09T15:08:21.924 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:21.922+0000 7f31b64cc640 1 Processor -- start 2023-08-09T15:08:21.924 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:21.923+0000 7f31b64cc640 1 -- start start 2023-08-09T15:08:21.926 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:21.924+0000 7f31b64cc640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f31b0143c20 0x7f31b006f540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T15:08:21.926 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:21.924+0000 7f31b64cc640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f31b006cfe0 0x7f31b00c8b50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T15:08:21.926 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:21.924+0000 7f31b64cc640 1 -- --> v1:172.21.15.203:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f31b00c9090 con 0x7f31b00c86f0 2023-08-09T15:08:21.926 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:21.924+0000 7f31b64cc640 1 -- --> v2:172.21.15.43:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f31b0144000 con 0x7f31b006cfe0 2023-08-09T15:08:21.926 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:21.924+0000 7f31b64cc640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f31b006c440 con 0x7f31b0143c20 2023-08-09T15:08:21.927 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:21.925+0000 7f31b4a69640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f31b006cfe0 0x7f31b00c8b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T15:08:21.927 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:21.925+0000 7f31b4a69640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f31b006cfe0 0x7f31b00c8b50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:46450/0 (socket says 172.21.15.43:46450) 2023-08-09T15:08:21.927 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:21.925+0000 7f31b4a69640 1 -- 172.21.15.43:0/1134282205 learned_addr learned my addr 172.21.15.43:0/1134282205 (peer_addr_for_me v2:172.21.15.43:0/0) 2023-08-09T15:08:21.927 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:21.925+0000 7f31affff640 1 --2- 172.21.15.43:0/1134282205 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f31b0143c20 0x7f31b006f540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T15:08:21.928 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:21.925+0000 7f31b4a69640 1 -- 172.21.15.43:0/1134282205 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f31b0143c20 msgr2=0x7f31b006f540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-09T15:08:21.928 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:21.925+0000 7f31b4a69640 1 --2- 172.21.15.43:0/1134282205 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f31b0143c20 0x7f31b006f540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T15:08:21.928 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:21.925+0000 7f31b4a69640 1 -- 172.21.15.43:0/1134282205 >> v1:172.21.15.203:6789/0 conn(0x7f31b00c86f0 legacy=0x7f31b006c8d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-09T15:08:21.928 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:21.925+0000 7f31b4a69640 1 -- 172.21.15.43:0/1134282205 --> v2:172.21.15.43:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f31b006b830 con 0x7f31b006cfe0 2023-08-09T15:08:21.929 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:21.927+0000 7f31b4a69640 1 --2- 172.21.15.43:0/1134282205 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f31b006cfe0 0x7f31b00c8b50 secure :-1 s=READY pgs=2101 cs=0 l=1 rev1=1 crypto rx=0x7f31a8009e50 tx=0x7f31a802be40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d4c7316f6fe58f1 server_cookie=0 in_seq=0 out_seq=0 2023-08-09T15:08:21.929 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:21.927+0000 7f31af7fe640 1 -- 172.21.15.43:0/1134282205 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f31a802c9e0 con 0x7f31b006cfe0 2023-08-09T15:08:21.929 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:21.927+0000 7f31af7fe640 1 -- 172.21.15.43:0/1134282205 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f31a802cb40 con 0x7f31b006cfe0 2023-08-09T15:08:21.929 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:21.927+0000 7f31af7fe640 1 -- 172.21.15.43:0/1134282205 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f31a80343f0 con 0x7f31b006cfe0 2023-08-09T15:08:21.929 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:21.928+0000 7f31b64cc640 1 -- 172.21.15.43:0/1134282205 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f31b006cfe0 msgr2=0x7f31b00c8b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T15:08:21.930 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:21.928+0000 7f31b64cc640 1 --2- 172.21.15.43:0/1134282205 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f31b006cfe0 0x7f31b00c8b50 secure :-1 s=READY pgs=2101 cs=0 l=1 rev1=1 crypto rx=0x7f31a8009e50 tx=0x7f31a802be40 comp rx=0 tx=0).stop 2023-08-09T15:08:21.930 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:21.928+0000 7f31b64cc640 1 -- 172.21.15.43:0/1134282205 shutdown_connections 2023-08-09T15:08:21.931 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:21.928+0000 7f31b64cc640 1 --2- 172.21.15.43:0/1134282205 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f31b006cfe0 0x7f31b00c8b50 unknown :-1 s=CLOSED pgs=2101 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T15:08:21.931 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:21.928+0000 7f31b64cc640 1 --2- 172.21.15.43:0/1134282205 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f31b0143c20 0x7f31b006f540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T15:08:21.931 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:21.928+0000 7f31b64cc640 1 -- 172.21.15.43:0/1134282205 >> 172.21.15.43:0/1134282205 conn(0x7f31b00ce0a0 msgr2=0x7f31b00ce480 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-09T15:08:21.931 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:21.928+0000 7f31b64cc640 1 -- 172.21.15.43:0/1134282205 shutdown_connections 2023-08-09T15:08:21.931 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:21.928+0000 7f31b64cc640 1 -- 172.21.15.43:0/1134282205 wait complete. 2023-08-09T15:08:21.932 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:21.930+0000 7f31b64cc640 1 Processor -- start 2023-08-09T15:08:21.932 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:21.930+0000 7f31b64cc640 1 -- start start 2023-08-09T15:08:21.933 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:21.931+0000 7f31b64cc640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f31b006cfe0 0x7f31b00acb80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T15:08:21.933 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:21.931+0000 7f31b64cc640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f31b00c86f0 0x7f31b00b3300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T15:08:21.933 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:21.931+0000 7f31b64cc640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f31b0143c20 0x7f31b00c4360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T15:08:21.933 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:21.931+0000 7f31b64cc640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f31b00c7190 con 0x7f31b006cfe0 2023-08-09T15:08:21.933 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:21.931+0000 7f31b64cc640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f31b00c67f0 con 0x7f31b0143c20 2023-08-09T15:08:21.934 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:21.931+0000 7f31b64cc640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f31b00c6ad0 con 0x7f31b00c86f0 2023-08-09T15:08:21.934 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:21.931+0000 7f31affff640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f31b006cfe0 0x7f31b00acb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T15:08:21.934 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:21.932+0000 7f31affff640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f31b006cfe0 0x7f31b00acb80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:46458/0 (socket says 172.21.15.43:46458) 2023-08-09T15:08:21.934 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:21.932+0000 7f31affff640 1 -- 172.21.15.43:0/3655665212 learned_addr learned my addr 172.21.15.43:0/3655665212 (peer_addr_for_me v2:172.21.15.43:0/0) 2023-08-09T15:08:21.934 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:21.932+0000 7f31affff640 1 -- 172.21.15.43:0/3655665212 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f31b00c86f0 msgr2=0x7f31b00b3300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T15:08:21.934 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:21.932+0000 7f31affff640 1 --2- 172.21.15.43:0/3655665212 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f31b00c86f0 0x7f31b00b3300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T15:08:21.934 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:21.932+0000 7f31affff640 1 -- 172.21.15.43:0/3655665212 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f31b0143c20 msgr2=0x7f31b00c4360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T15:08:21.935 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:21.932+0000 7f31affff640 1 --2- 172.21.15.43:0/3655665212 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f31b0143c20 0x7f31b00c4360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T15:08:21.935 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:21.932+0000 7f31affff640 1 -- 172.21.15.43:0/3655665212 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f31a8009b30 con 0x7f31b006cfe0 2023-08-09T15:08:21.935 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:21.932+0000 7f31affff640 1 --2- 172.21.15.43:0/3655665212 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f31b006cfe0 0x7f31b00acb80 secure :-1 s=READY pgs=2102 cs=0 l=1 rev1=1 crypto rx=0x7f31a000d800 tx=0x7f31a000dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-09T15:08:21.936 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:21.932+0000 7f31ad7fa640 1 -- 172.21.15.43:0/3655665212 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f31a000b6c0 con 0x7f31b006cfe0 2023-08-09T15:08:21.936 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:21.933+0000 7f31ad7fa640 1 -- 172.21.15.43:0/3655665212 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f31a000b820 con 0x7f31b006cfe0 2023-08-09T15:08:21.936 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:21.933+0000 7f31ad7fa640 1 -- 172.21.15.43:0/3655665212 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f31a000bbb0 con 0x7f31b006cfe0 2023-08-09T15:08:21.936 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:21.933+0000 7f31b64cc640 1 -- 172.21.15.43:0/3655665212 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f31b00c6db0 con 0x7f31b006cfe0 2023-08-09T15:08:21.936 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:21.933+0000 7f31b64cc640 1 -- 172.21.15.43:0/3655665212 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f31b00ad0c0 con 0x7f31b006cfe0 2023-08-09T15:08:21.936 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:21.934+0000 7f31b64cc640 1 -- 172.21.15.43:0/3655665212 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f31b00ad2b0 con 0x7f31b006cfe0 2023-08-09T15:08:21.937 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:21.935+0000 7f31ad7fa640 1 -- 172.21.15.43:0/3655665212 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 102) v1 ==== 88455+0+0 (secure 0 0 0) 0x7f31a000bd10 con 0x7f31b006cfe0 2023-08-09T15:08:21.937 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:21.936+0000 7f31ad7fa640 1 --2- 172.21.15.43:0/3655665212 >> [v2:172.21.15.203:6832/115830229,v1:172.21.15.203:6833/115830229] conn(0x7f319006a170 0x7f319006c630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T15:08:21.938 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:21.936+0000 7f31ad7fa640 1 -- 172.21.15.43:0/3655665212 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(190..190 src has 1..190) v4 ==== 7202+0+0 (secure 0 0 0) 0x7f31a00891c0 con 0x7f31b006cfe0 2023-08-09T15:08:21.938 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:21.937+0000 7f31a7fff640 1 -- 172.21.15.43:0/3655665212 >> [v2:172.21.15.203:6832/115830229,v1:172.21.15.203:6833/115830229] conn(0x7f319006a170 msgr2=0x7f319006c630 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.203:6832/115830229 2023-08-09T15:08:21.938 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:21.937+0000 7f31a7fff640 1 --2- 172.21.15.43:0/3655665212 >> [v2:172.21.15.203:6832/115830229,v1:172.21.15.203:6833/115830229] conn(0x7f319006a170 0x7f319006c630 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-08-09T15:08:21.941 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:21.940+0000 7f31ad7fa640 1 -- 172.21.15.43:0/3655665212 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178066 (secure 0 0 0) 0x7f31a0054280 con 0x7f31b006cfe0 2023-08-09T15:08:22.139 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:22.138+0000 7f31a7fff640 1 -- 172.21.15.43:0/3655665212 >> [v2:172.21.15.203:6832/115830229,v1:172.21.15.203:6833/115830229] conn(0x7f319006a170 msgr2=0x7f319006c630 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.203:6832/115830229 2023-08-09T15:08:22.139 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:22.138+0000 7f31a7fff640 1 --2- 172.21.15.43:0/3655665212 >> [v2:172.21.15.203:6832/115830229,v1:172.21.15.203:6833/115830229] conn(0x7f319006a170 0x7f319006c630 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2023-08-09T15:08:22.159 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:22.158+0000 7f31b64cc640 1 -- 172.21.15.43:0/3655665212 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f31b00c84d0 con 0x7f31b006cfe0 2023-08-09T15:08:22.161 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:22.160+0000 7f31ad7fa640 1 -- 172.21.15.43:0/3655665212 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v190) v1 ==== 74+0+19731 (secure 0 0 0) 0x7f31a00578a0 con 0x7f31b006cfe0 2023-08-09T15:08:22.162 INFO:teuthology.orchestra.run.smithi043.stdout: 2023-08-09T15:08:22.162 INFO:teuthology.orchestra.run.smithi043.stdout:{"epoch":190,"fsid":"daac87ee-1337-4250-a955-aea4e408a39b","created":"2023-08-09T13:33:41.701198+0000","modified":"2023-08-09T15:08:21.356706+0000","last_up_change":"2023-08-09T13:33:53.783453+0000","last_in_change":"2023-08-09T13:33:45.148904+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":23,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":"rbd","create_time":"2023-08-09T13:33:54.365137+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":".mgr","create_time":"2023-08-09T13:33:54.831803+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":22,"pool_name":"cephfs_metadata","create_time":"2023-08-09T14:52:28.569334+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"185","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":23,"pool_name":"cephfs_data","create_time":"2023-08-09T14:52:29.209973+0000","flags":8193,"flags_names":"hashpspool,selfmanaged_snaps","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"190","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":30,"snap_epoch":190,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"cbd49ecd-e896-4fbd-864e-5c37a829b165","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":184,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6800","nonce":502667335},{"type":"v1","addr":"172.21.15.43:6801","nonce":502667335}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6802","nonce":502667335},{"type":"v1","addr":"172.21.15.43:6803","nonce":502667335}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6806","nonce":502667335},{"type":"v1","addr":"172.21.15.43:6807","nonce":502667335}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6804","nonce":502667335},{"type":"v1","addr":"172.21.15.43:6805","nonce":502667335}]},"public_addr":"172.21.15.43:6801/502667335","cluster_addr":"172.21.15.43:6803/502667335","heartbeat_back_addr":"172.21.15.43:6807/502667335","heartbeat_front_addr":"172.21.15.43:6805/502667335","state":["exists","up"]},{"osd":1,"uuid":"75c7e0fe-1ec7-4fe8-a2f2-3661a65b8500","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":184,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6808","nonce":3289134610},{"type":"v1","addr":"172.21.15.43:6809","nonce":3289134610}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6810","nonce":3289134610},{"type":"v1","addr":"172.21.15.43:6812","nonce":3289134610}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6821","nonce":3289134610},{"type":"v1","addr":"172.21.15.43:6824","nonce":3289134610}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6815","nonce":3289134610},{"type":"v1","addr":"172.21.15.43:6818","nonce":3289134610}]},"public_addr":"172.21.15.43:6809/3289134610","cluster_addr":"172.21.15.43:6812/3289134610","heartbeat_back_addr":"172.21.15.43:6824/3289134610","heartbeat_front_addr":"172.21.15.43:6818/3289134610","state":["exists","up"]},{"osd":2,"uuid":"bb5c41e9-7f5f-4125-9b01-3bb8bfc2ea83","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":184,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6811","nonce":4079709736},{"type":"v1","addr":"172.21.15.43:6814","nonce":4079709736}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6817","nonce":4079709736},{"type":"v1","addr":"172.21.15.43:6820","nonce":4079709736}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6829","nonce":4079709736},{"type":"v1","addr":"172.21.15.43:6831","nonce":4079709736}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6823","nonce":4079709736},{"type":"v1","addr":"172.21.15.43:6826","nonce":4079709736}]},"public_addr":"172.21.15.43:6814/4079709736","cluster_addr":"172.21.15.43:6820/4079709736","heartbeat_back_addr":"172.21.15.43:6831/4079709736","heartbeat_front_addr":"172.21.15.43:6826/4079709736","state":["exists","up"]},{"osd":3,"uuid":"bfe4c8b4-694a-465c-87e6-2b2aab852613","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":184,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6813","nonce":1709538759},{"type":"v1","addr":"172.21.15.43:6816","nonce":1709538759}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6819","nonce":1709538759},{"type":"v1","addr":"172.21.15.43:6822","nonce":1709538759}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6828","nonce":1709538759},{"type":"v1","addr":"172.21.15.43:6830","nonce":1709538759}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6825","nonce":1709538759},{"type":"v1","addr":"172.21.15.43:6827","nonce":1709538759}]},"public_addr":"172.21.15.43:6816/1709538759","cluster_addr":"172.21.15.43:6822/1709538759","heartbeat_back_addr":"172.21.15.43:6830/1709538759","heartbeat_front_addr":"172.21.15.43:6827/1709538759","state":["exists","up"]},{"osd":4,"uuid":"fa2e8a77-aba1-4756-853e-0d174a5bdeba","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":184,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6800","nonce":4196866481},{"type":"v1","addr":"172.21.15.203:6801","nonce":4196866481}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6802","nonce":4196866481},{"type":"v1","addr":"172.21.15.203:6803","nonce":4196866481}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6806","nonce":4196866481},{"type":"v1","addr":"172.21.15.203:6807","nonce":4196866481}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6804","nonce":4196866481},{"type":"v1","addr":"172.21.15.203:6805","nonce":4196866481}]},"public_addr":"172.21.15.203:6801/4196866481","cluster_addr":"172.21.15.203:6803/4196866481","heartbeat_back_addr":"172.21.15.203:6807/4196866481","heartbeat_front_addr":"172.21.15.203:6805/4196866481","state":["exists","up"]},{"osd":5,"uuid":"81811e4b-4d24-4f51-a82c-f1d700a2a00d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":184,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6816","nonce":3059290081},{"type":"v1","addr":"172.21.15.203:6817","nonce":3059290081}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6818","nonce":3059290081},{"type":"v1","addr":"172.21.15.203:6819","nonce":3059290081}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6822","nonce":3059290081},{"type":"v1","addr":"172.21.15.203:6823","nonce":3059290081}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6820","nonce":3059290081},{"type":"v1","addr":"172.21.15.203:6821","nonce":3059290081}]},"public_addr":"172.21.15.203:6817/3059290081","cluster_addr":"172.21.15.203:6819/3059290081","heartbeat_back_addr":"172.21.15.203:6823/3059290081","heartbeat_front_addr":"172.21.15.203:6821/3059290081","state":["exists","up"]},{"osd":6,"uuid":"307bac4d-6a5b-443d-af75-56d695d7674f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":184,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6824","nonce":512155569},{"type":"v1","addr":"172.21.15.203:6825","nonce":512155569}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6826","nonce":512155569},{"type":"v1","addr":"172.21.15.203:6827","nonce":512155569}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6830","nonce":512155569},{"type":"v1","addr":"172.21.15.203:6831","nonce":512155569}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6828","nonce":512155569},{"type":"v1","addr":"172.21.15.203:6829","nonce":512155569}]},"public_addr":"172.21.15.203:6825/512155569","cluster_addr":"172.21.15.203:6827/512155569","heartbeat_back_addr":"172.21.15.203:6831/512155569","heartbeat_front_addr":"172.21.15.203:6829/512155569","state":["exists","up"]},{"osd":7,"uuid":"a95dc7ca-9f1b-4295-af70-68a22a2d7d5e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":184,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6808","nonce":3331450438},{"type":"v1","addr":"172.21.15.203:6809","nonce":3331450438}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6810","nonce":3331450438},{"type":"v1","addr":"172.21.15.203:6811","nonce":3331450438}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6814","nonce":3331450438},{"type":"v1","addr":"172.21.15.203:6815","nonce":3331450438}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6812","nonce":3331450438},{"type":"v1","addr":"172.21.15.203:6813","nonce":3331450438}]},"public_addr":"172.21.15.203:6809/3331450438","cluster_addr":"172.21.15.203:6811/3331450438","heartbeat_back_addr":"172.21.15.203:6815/3331450438","heartbeat_front_addr":"172.21.15.203:6813/3331450438","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-09T13:33:50.547652+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-09T13:33:50.589469+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-09T13:33:50.561724+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-09T13:33:50.599650+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-09T13:33:50.411855+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-09T13:33:50.219681+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-09T13:33:50.504664+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-09T13:33:50.441248+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.203:0/69497804":"2023-08-10T15:03:21.353235+0000","172.21.15.203:0/169967264":"2023-08-10T15:03:21.353235+0000","172.21.15.203:6833/40883597":"2023-08-10T15:03:21.353235+0000","172.21.15.203:6832/40883597":"2023-08-10T15:03:21.353235+0000","172.21.15.203:0/3690197627":"2023-08-10T15:03:21.353235+0000","172.21.15.203:0/1082753911":"2023-08-10T15:03:21.353235+0000","172.21.15.203:6832/2959400181":"2023-08-10T14:52:45.188641+0000","172.21.15.203:0/82655847":"2023-08-10T15:03:21.353235+0000","172.21.15.203:0/1935803093":"2023-08-10T14:52:45.188641+0000","172.21.15.203:0/3150685329":"2023-08-10T14:52:45.188641+0000","172.21.15.203:0/2219953609":"2023-08-10T15:03:21.353235+0000","172.21.15.203:0/3429376127":"2023-08-10T14:52:45.188641+0000","172.21.15.203:0/2197573025":"2023-08-10T14:52:45.188641+0000","172.21.15.203:0/2458603841":"2023-08-10T14:52:45.188641+0000","172.21.15.203:6833/2959400181":"2023-08-10T14:52:45.188641+0000","172.21.15.203:0/4177046277":"2023-08-10T14:52:45.188641+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":7,"snaps":[{"begin":104,"length":1},{"begin":221,"length":1}]},{"pool":23,"snaps":[{"begin":2,"length":13},{"begin":18,"length":13}]}],"new_removed_snaps":[{"pool":23,"snaps":[{"begin":6,"length":9},{"begin":22,"length":9}]}],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-08-09T15:08:22.163 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:22.162+0000 7f31b64cc640 1 -- 172.21.15.43:0/3655665212 >> [v2:172.21.15.203:6832/115830229,v1:172.21.15.203:6833/115830229] conn(0x7f319006a170 msgr2=0x7f319006c630 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-08-09T15:08:22.163 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:22.162+0000 7f31b64cc640 1 --2- 172.21.15.43:0/3655665212 >> [v2:172.21.15.203:6832/115830229,v1:172.21.15.203:6833/115830229] conn(0x7f319006a170 0x7f319006c630 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T15:08:22.164 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:22.162+0000 7f31b64cc640 1 -- 172.21.15.43:0/3655665212 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f31b006cfe0 msgr2=0x7f31b00acb80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T15:08:22.164 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:22.162+0000 7f31b64cc640 1 --2- 172.21.15.43:0/3655665212 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f31b006cfe0 0x7f31b00acb80 secure :-1 s=READY pgs=2102 cs=0 l=1 rev1=1 crypto rx=0x7f31a000d800 tx=0x7f31a000dcc0 comp rx=0 tx=0).stop 2023-08-09T15:08:22.164 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:22.162+0000 7f31b64cc640 1 -- 172.21.15.43:0/3655665212 shutdown_connections 2023-08-09T15:08:22.164 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:22.162+0000 7f31b64cc640 1 --2- 172.21.15.43:0/3655665212 >> [v2:172.21.15.203:6832/115830229,v1:172.21.15.203:6833/115830229] conn(0x7f319006a170 0x7f319006c630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T15:08:22.164 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:22.162+0000 7f31b64cc640 1 --2- 172.21.15.43:0/3655665212 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f31b0143c20 0x7f31b00c4360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T15:08:22.164 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:22.162+0000 7f31b64cc640 1 --2- 172.21.15.43:0/3655665212 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f31b00c86f0 0x7f31b00b3300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T15:08:22.165 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:22.162+0000 7f31b64cc640 1 --2- 172.21.15.43:0/3655665212 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f31b006cfe0 0x7f31b00acb80 unknown :-1 s=CLOSED pgs=2102 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T15:08:22.165 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:22.162+0000 7f31b64cc640 1 -- 172.21.15.43:0/3655665212 >> 172.21.15.43:0/3655665212 conn(0x7f31b00ce0a0 msgr2=0x7f31b00ae6f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-09T15:08:22.165 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:22.162+0000 7f31b64cc640 1 -- 172.21.15.43:0/3655665212 shutdown_connections 2023-08-09T15:08:22.165 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:22.162+0000 7f31b64cc640 1 -- 172.21.15.43:0/3655665212 wait complete. 2023-08-09T15:08:22.181 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': 'rbd', 'create_time': '2023-08-09T13:33:54.365137+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}, 'read_balance': {'score_acting': 2, 'score_stable': 2, 'optimal_score': 1, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': '.mgr', 'create_time': '2023-08-09T13:33:54.831803+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 22, 'pool_name': 'cephfs_metadata', 'create_time': '2023-08-09T14:52:28.569334+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '185', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 2, 'score_stable': 2, 'optimal_score': 1, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 23, 'pool_name': 'cephfs_data', 'create_time': '2023-08-09T14:52:29.209973+0000', 'flags': 8193, 'flags_names': 'hashpspool,selfmanaged_snaps', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '190', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 30, 'snap_epoch': 190, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.3799999952316284, 'score_stable': 1.3799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.3799999952316284, 'raw_score_stable': 1.3799999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-08-09T15:08:22.182 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get rbd pg_num 2023-08-09T15:08:22.403 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:22.402+0000 7fedae176640 1 Processor -- start 2023-08-09T15:08:22.404 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:22.402+0000 7fedae176640 1 -- start start 2023-08-09T15:08:22.405 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:22.404+0000 7fedae176640 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7feda815a0c0 0x7feda80cbb80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T15:08:22.406 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:22.404+0000 7fedae176640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7feda815d510 0x7feda80cb2b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T15:08:22.406 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:22.404+0000 7fedae176640 1 -- --> v1:172.21.15.203:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7feda815a4a0 con 0x7feda80ce1c0 2023-08-09T15:08:22.406 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:22.404+0000 7fedae176640 1 -- --> v2:172.21.15.203:3300/0 -- mon_getmap magic: 0 v1 -- 0x7feda80cc0c0 con 0x7feda815a0c0 2023-08-09T15:08:22.406 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:22.404+0000 7fedae176640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7feda80cc2d0 con 0x7feda815d510 2023-08-09T15:08:22.406 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:22.405+0000 7feda77fe640 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7feda815a0c0 0x7feda80cbb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T15:08:22.407 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:22.405+0000 7feda77fe640 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7feda815a0c0 0x7feda80cbb80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.203:3300/0 says I am v2:172.21.15.43:35120/0 (socket says 172.21.15.43:35120) 2023-08-09T15:08:22.407 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:22.405+0000 7feda77fe640 1 -- 172.21.15.43:0/919513472 learned_addr learned my addr 172.21.15.43:0/919513472 (peer_addr_for_me v2:172.21.15.43:0/0) 2023-08-09T15:08:22.407 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:22.405+0000 7feda6ffd640 1 --2- 172.21.15.43:0/919513472 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7feda815d510 0x7feda80cb2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T15:08:22.409 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:22.406+0000 7feda77fe640 1 -- 172.21.15.43:0/919513472 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7feda815d510 msgr2=0x7feda80cb2b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-09T15:08:22.409 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:22.406+0000 7feda77fe640 1 --2- 172.21.15.43:0/919513472 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7feda815d510 0x7feda80cb2b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T15:08:22.409 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:22.406+0000 7feda77fe640 1 -- 172.21.15.43:0/919513472 >> v1:172.21.15.203:6789/0 conn(0x7feda80ce1c0 legacy=0x7feda80c8600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T15:08:22.409 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:22.406+0000 7feda77fe640 1 -- 172.21.15.43:0/919513472 --> v2:172.21.15.203:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7feda80cc5b0 con 0x7feda815a0c0 2023-08-09T15:08:22.409 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:22.408+0000 7feda6ffd640 1 --2- 172.21.15.43:0/919513472 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7feda815d510 0x7feda80cb2b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-08-09T15:08:22.410 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:22.409+0000 7feda77fe640 1 --2- 172.21.15.43:0/919513472 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7feda815a0c0 0x7feda80cbb80 secure :-1 s=READY pgs=784 cs=0 l=1 rev1=1 crypto rx=0x7feda0009e50 tx=0x7feda002bf80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=582c5052a9eb1209 server_cookie=0 in_seq=0 out_seq=0 2023-08-09T15:08:22.410 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:22.409+0000 7feda67fc640 1 -- 172.21.15.43:0/919513472 <== mon.1 v2:172.21.15.203:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feda002cb20 con 0x7feda815a0c0 2023-08-09T15:08:22.411 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:22.409+0000 7feda67fc640 1 -- 172.21.15.43:0/919513472 <== mon.1 v2:172.21.15.203:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7feda002cc80 con 0x7feda815a0c0 2023-08-09T15:08:22.411 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:22.409+0000 7feda67fc640 1 -- 172.21.15.43:0/919513472 <== mon.1 v2:172.21.15.203:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feda0034610 con 0x7feda815a0c0 2023-08-09T15:08:22.411 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:22.410+0000 7fedae176640 1 -- 172.21.15.43:0/919513472 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7feda815a0c0 msgr2=0x7feda80cbb80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T15:08:22.412 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:22.410+0000 7fedae176640 1 --2- 172.21.15.43:0/919513472 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7feda815a0c0 0x7feda80cbb80 secure :-1 s=READY pgs=784 cs=0 l=1 rev1=1 crypto rx=0x7feda0009e50 tx=0x7feda002bf80 comp rx=0 tx=0).stop 2023-08-09T15:08:22.412 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:22.411+0000 7fedae176640 1 -- 172.21.15.43:0/919513472 shutdown_connections 2023-08-09T15:08:22.412 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:22.411+0000 7fedae176640 1 --2- 172.21.15.43:0/919513472 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7feda815d510 0x7feda80cb2b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T15:08:22.412 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:22.411+0000 7fedae176640 1 --2- 172.21.15.43:0/919513472 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7feda815a0c0 0x7feda80cbb80 unknown :-1 s=CLOSED pgs=784 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T15:08:22.413 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:22.411+0000 7fedae176640 1 -- 172.21.15.43:0/919513472 >> 172.21.15.43:0/919513472 conn(0x7feda80aff50 msgr2=0x7feda80b0360 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-09T15:08:22.413 INFO:tasks.ceph.mgr.x.smithi203.stderr:2023-08-09T15:08:22.412+0000 7fbc59c0f040 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2023-08-09T15:08:22.414 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:22.412+0000 7fedae176640 1 -- 172.21.15.43:0/919513472 shutdown_connections 2023-08-09T15:08:22.414 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:22.412+0000 7fedae176640 1 -- 172.21.15.43:0/919513472 wait complete. 2023-08-09T15:08:22.415 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:22.414+0000 7fedae176640 1 Processor -- start 2023-08-09T15:08:22.416 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:22.415+0000 7fedae176640 1 -- start start 2023-08-09T15:08:22.417 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:22.416+0000 7fedae176640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7feda80ce1c0 0x7feda81f3480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T15:08:22.417 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:22.416+0000 7fedae176640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7feda815a0c0 0x7feda80c7b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T15:08:22.418 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:22.416+0000 7fedae176640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7feda815d510 0x7feda80c8040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T15:08:22.418 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:22.416+0000 7fedae176640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feda81f6440 con 0x7feda815a0c0 2023-08-09T15:08:22.418 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:22.416+0000 7fedae176640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feda81f7040 con 0x7feda80ce1c0 2023-08-09T15:08:22.418 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:22.416+0000 7fedae176640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7feda80c4350 con 0x7feda815d510 2023-08-09T15:08:22.418 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:22.416+0000 7feda6ffd640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7feda815a0c0 0x7feda80c7b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T15:08:22.419 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:22.417+0000 7feda6ffd640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7feda815a0c0 0x7feda80c7b00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:46474/0 (socket says 172.21.15.43:46474) 2023-08-09T15:08:22.419 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:22.417+0000 7feda6ffd640 1 -- 172.21.15.43:0/1491983935 learned_addr learned my addr 172.21.15.43:0/1491983935 (peer_addr_for_me v2:172.21.15.43:0/0) 2023-08-09T15:08:22.419 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:22.417+0000 7feda7fff640 1 --2- 172.21.15.43:0/1491983935 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7feda815d510 0x7feda80c8040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T15:08:22.419 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:22.417+0000 7feda77fe640 1 --2- 172.21.15.43:0/1491983935 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7feda80ce1c0 0x7feda81f3480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T15:08:22.419 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:22.417+0000 7feda6ffd640 1 -- 172.21.15.43:0/1491983935 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7feda815d510 msgr2=0x7feda80c8040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T15:08:22.419 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:22.417+0000 7feda6ffd640 1 --2- 172.21.15.43:0/1491983935 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7feda815d510 0x7feda80c8040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T15:08:22.420 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:22.417+0000 7feda6ffd640 1 -- 172.21.15.43:0/1491983935 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7feda80ce1c0 msgr2=0x7feda81f3480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T15:08:22.420 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:22.417+0000 7feda6ffd640 1 --2- 172.21.15.43:0/1491983935 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7feda80ce1c0 0x7feda81f3480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T15:08:22.420 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:22.417+0000 7feda6ffd640 1 -- 172.21.15.43:0/1491983935 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7feda0009b30 con 0x7feda815a0c0 2023-08-09T15:08:22.420 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:22.418+0000 7feda6ffd640 1 --2- 172.21.15.43:0/1491983935 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7feda815a0c0 0x7feda80c7b00 secure :-1 s=READY pgs=2103 cs=0 l=1 rev1=1 crypto rx=0x7fed9c00c780 tx=0x7fed9c00cc40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-09T15:08:22.421 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:22.418+0000 7fed8ffff640 1 -- 172.21.15.43:0/1491983935 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fed9c007a30 con 0x7feda815a0c0 2023-08-09T15:08:22.421 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:22.418+0000 7fed8ffff640 1 -- 172.21.15.43:0/1491983935 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fed9c00ce90 con 0x7feda815a0c0 2023-08-09T15:08:22.421 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:22.418+0000 7fed8ffff640 1 -- 172.21.15.43:0/1491983935 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fed9c007dd0 con 0x7feda815a0c0 2023-08-09T15:08:22.421 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:22.418+0000 7fedae176640 1 -- 172.21.15.43:0/1491983935 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7feda80c3740 con 0x7feda815a0c0 2023-08-09T15:08:22.421 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:22.419+0000 7fedae176640 1 -- 172.21.15.43:0/1491983935 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7feda80c3d00 con 0x7feda815a0c0 2023-08-09T15:08:22.422 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:22.421+0000 7fedae176640 1 -- 172.21.15.43:0/1491983935 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7feda80c44c0 con 0x7feda815a0c0 2023-08-09T15:08:22.423 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:22.422+0000 7fed8ffff640 1 -- 172.21.15.43:0/1491983935 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 102) v1 ==== 88455+0+0 (secure 0 0 0) 0x7fed9c0113f0 con 0x7feda815a0c0 2023-08-09T15:08:22.425 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:22.424+0000 7fed8ffff640 1 --2- 172.21.15.43:0/1491983935 >> [v2:172.21.15.203:6832/115830229,v1:172.21.15.203:6833/115830229] conn(0x7fed9406a1c0 0x7fed9406c680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T15:08:22.425 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:22.424+0000 7fed8ffff640 1 -- 172.21.15.43:0/1491983935 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(191..191 src has 1..191) v4 ==== 7202+0+0 (secure 0 0 0) 0x7fed9c088ce0 con 0x7feda815a0c0 2023-08-09T15:08:22.425 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:22.424+0000 7feda77fe640 1 -- 172.21.15.43:0/1491983935 >> [v2:172.21.15.203:6832/115830229,v1:172.21.15.203:6833/115830229] conn(0x7fed9406a1c0 msgr2=0x7fed9406c680 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.203:6832/115830229 2023-08-09T15:08:22.426 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:22.424+0000 7feda77fe640 1 --2- 172.21.15.43:0/1491983935 >> [v2:172.21.15.203:6832/115830229,v1:172.21.15.203:6833/115830229] conn(0x7fed9406a1c0 0x7fed9406c680 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-08-09T15:08:22.431 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:22.430+0000 7fed8ffff640 1 -- 172.21.15.43:0/1491983935 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178066 (secure 0 0 0) 0x7fed9c053e50 con 0x7feda815a0c0 2023-08-09T15:08:22.451 INFO:tasks.ceph.mgr.y.smithi043.stderr:2023-08-09T15:08:22.450+0000 7fe393e53040 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2023-08-09T15:08:22.537 INFO:tasks.ceph.mgr.x.smithi203.stderr:2023-08-09T15:08:22.535+0000 7fbc59c0f040 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2023-08-09T15:08:22.585 INFO:tasks.ceph.mgr.y.smithi043.stderr:2023-08-09T15:08:22.584+0000 7fe393e53040 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2023-08-09T15:08:22.594 INFO:tasks.ceph.mgr.x.smithi203.stderr:2023-08-09T15:08:22.593+0000 7fbc59c0f040 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2023-08-09T15:08:22.626 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:22.625+0000 7feda77fe640 1 -- 172.21.15.43:0/1491983935 >> [v2:172.21.15.203:6832/115830229,v1:172.21.15.203:6833/115830229] conn(0x7fed9406a1c0 msgr2=0x7fed9406c680 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.203:6832/115830229 2023-08-09T15:08:22.627 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:22.625+0000 7feda77fe640 1 --2- 172.21.15.43:0/1491983935 >> [v2:172.21.15.203:6832/115830229,v1:172.21.15.203:6833/115830229] conn(0x7fed9406a1c0 0x7fed9406c680 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2023-08-09T15:08:22.632 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:22.631+0000 7fedae176640 1 -- 172.21.15.43:0/1491983935 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"} v 0) v1 -- 0x7feda80739f0 con 0x7feda815a0c0 2023-08-09T15:08:22.633 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:22.632+0000 7fed8ffff640 1 -- 172.21.15.43:0/1491983935 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"}]=0 v191) v1 ==== 92+0+10 (secure 0 0 0) 0x7fed9c01e0a0 con 0x7feda815a0c0 2023-08-09T15:08:22.633 INFO:teuthology.orchestra.run.smithi043.stdout:pg_num: 8 2023-08-09T15:08:22.637 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:22.635+0000 7fedae176640 1 -- 172.21.15.43:0/1491983935 >> [v2:172.21.15.203:6832/115830229,v1:172.21.15.203:6833/115830229] conn(0x7fed9406a1c0 msgr2=0x7fed9406c680 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-08-09T15:08:22.637 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:22.635+0000 7fedae176640 1 --2- 172.21.15.43:0/1491983935 >> [v2:172.21.15.203:6832/115830229,v1:172.21.15.203:6833/115830229] conn(0x7fed9406a1c0 0x7fed9406c680 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T15:08:22.637 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:22.636+0000 7fedae176640 1 -- 172.21.15.43:0/1491983935 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7feda815a0c0 msgr2=0x7feda80c7b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T15:08:22.637 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:22.636+0000 7fedae176640 1 --2- 172.21.15.43:0/1491983935 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7feda815a0c0 0x7feda80c7b00 secure :-1 s=READY pgs=2103 cs=0 l=1 rev1=1 crypto rx=0x7fed9c00c780 tx=0x7fed9c00cc40 comp rx=0 tx=0).stop 2023-08-09T15:08:22.638 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:22.636+0000 7fedae176640 1 -- 172.21.15.43:0/1491983935 shutdown_connections 2023-08-09T15:08:22.638 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:22.636+0000 7fedae176640 1 --2- 172.21.15.43:0/1491983935 >> [v2:172.21.15.203:6832/115830229,v1:172.21.15.203:6833/115830229] conn(0x7fed9406a1c0 0x7fed9406c680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T15:08:22.638 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:22.636+0000 7fedae176640 1 --2- 172.21.15.43:0/1491983935 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7feda815d510 0x7feda80c8040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T15:08:22.638 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:22.636+0000 7fedae176640 1 --2- 172.21.15.43:0/1491983935 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7feda815a0c0 0x7feda80c7b00 unknown :-1 s=CLOSED pgs=2103 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T15:08:22.638 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:22.636+0000 7fedae176640 1 --2- 172.21.15.43:0/1491983935 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7feda80ce1c0 0x7feda81f3480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T15:08:22.638 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:22.636+0000 7fedae176640 1 -- 172.21.15.43:0/1491983935 >> 172.21.15.43:0/1491983935 conn(0x7feda80aff50 msgr2=0x7feda815bb70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-09T15:08:22.639 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:22.636+0000 7fedae176640 1 -- 172.21.15.43:0/1491983935 shutdown_connections 2023-08-09T15:08:22.639 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:22.636+0000 7fedae176640 1 -- 172.21.15.43:0/1491983935 wait complete. 2023-08-09T15:08:22.652 INFO:tasks.ceph.mgr.y.smithi043.stderr:2023-08-09T15:08:22.651+0000 7fe393e53040 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2023-08-09T15:08:22.661 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-08-09T15:08:22.662 INFO:tasks.ceph.mgr.x.smithi203.stderr:2023-08-09T15:08:22.657+0000 7fbc59c0f040 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2023-08-09T15:08:22.706 INFO:tasks.ceph.mgr.y.smithi043.stderr:2023-08-09T15:08:22.705+0000 7fe393e53040 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2023-08-09T15:08:22.869 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:22.867+0000 7f7c6ee7a640 1 Processor -- start 2023-08-09T15:08:22.869 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:22.868+0000 7f7c6ee7a640 1 -- start start 2023-08-09T15:08:22.870 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:22.869+0000 7f7c6ee7a640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f7c6815c550 0x7f7c680cd6d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T15:08:22.871 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:22.869+0000 7f7c6ee7a640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f7c680d04b0 0x7f7c680b3ee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T15:08:22.871 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:22.869+0000 7f7c6ee7a640 1 -- --> v1:172.21.15.43:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7c680b4870 con 0x7f7c680b4420 2023-08-09T15:08:22.871 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:22.869+0000 7f7c6ee7a640 1 -- --> v2:172.21.15.43:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7c680b7bd0 con 0x7f7c6815c550 2023-08-09T15:08:22.871 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:22.869+0000 7f7c6ee7a640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7c680d1c00 con 0x7f7c680d04b0 2023-08-09T15:08:22.871 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:22.869+0000 7f7c6cc16640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f7c6815c550 0x7f7c680cd6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T15:08:22.871 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:22.869+0000 7f7c6d417640 1 --1- >> v1:172.21.15.43:6789/0 conn(0x7f7c680b4420 0x7f7c6806a760 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.43:6789/0 says I am v1:172.21.15.43:47348/0 (socket says 172.21.15.43:47348) 2023-08-09T15:08:22.872 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:22.870+0000 7f7c6d417640 1 -- 172.21.15.43:0/3438208740 learned_addr learned my addr 172.21.15.43:0/3438208740 (peer_addr_for_me v1:172.21.15.43:0/0) 2023-08-09T15:08:22.872 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:22.870+0000 7f7c67fff640 1 --2- 172.21.15.43:0/3438208740 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f7c680d04b0 0x7f7c680b3ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T15:08:22.873 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:22.870+0000 7f7c6cc16640 1 -- 172.21.15.43:0/3438208740 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f7c680d04b0 msgr2=0x7f7c680b3ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-09T15:08:22.873 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:22.870+0000 7f7c6cc16640 1 --2- 172.21.15.43:0/3438208740 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f7c680d04b0 0x7f7c680b3ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T15:08:22.873 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:22.870+0000 7f7c6cc16640 1 -- 172.21.15.43:0/3438208740 >> v1:172.21.15.43:6789/0 conn(0x7f7c680b4420 legacy=0x7f7c6806a760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T15:08:22.873 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:22.870+0000 7f7c6cc16640 1 -- 172.21.15.43:0/3438208740 --> v2:172.21.15.43:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7c6815c930 con 0x7f7c6815c550 2023-08-09T15:08:22.873 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:22.871+0000 7f7c6cc16640 1 --2- 172.21.15.43:0/3438208740 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f7c6815c550 0x7f7c680cd6d0 secure :-1 s=READY pgs=2105 cs=0 l=1 rev1=1 crypto rx=0x7f7c5c0097a0 tx=0x7f7c5c02b8b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=56a62df5958f48f1 server_cookie=0 in_seq=0 out_seq=0 2023-08-09T15:08:22.873 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:22.871+0000 7f7c677fe640 1 -- 172.21.15.43:0/3438208740 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7c5c02d040 con 0x7f7c6815c550 2023-08-09T15:08:22.873 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:22.871+0000 7f7c677fe640 1 -- 172.21.15.43:0/3438208740 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7c5c031810 con 0x7f7c6815c550 2023-08-09T15:08:22.874 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:22.871+0000 7f7c677fe640 1 -- 172.21.15.43:0/3438208740 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7c5c031ba0 con 0x7f7c6815c550 2023-08-09T15:08:22.875 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:22.872+0000 7f7c6ee7a640 1 -- 172.21.15.43:0/3438208740 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f7c6815c550 msgr2=0x7f7c680cd6d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T15:08:22.875 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:22.872+0000 7f7c6ee7a640 1 --2- 172.21.15.43:0/3438208740 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f7c6815c550 0x7f7c680cd6d0 secure :-1 s=READY pgs=2105 cs=0 l=1 rev1=1 crypto rx=0x7f7c5c0097a0 tx=0x7f7c5c02b8b0 comp rx=0 tx=0).stop 2023-08-09T15:08:22.875 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:22.872+0000 7f7c6ee7a640 1 -- 172.21.15.43:0/3438208740 shutdown_connections 2023-08-09T15:08:22.875 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:22.872+0000 7f7c6ee7a640 1 --2- 172.21.15.43:0/3438208740 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f7c680d04b0 0x7f7c680b3ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T15:08:22.876 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:22.872+0000 7f7c6ee7a640 1 --2- 172.21.15.43:0/3438208740 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f7c6815c550 0x7f7c680cd6d0 unknown :-1 s=CLOSED pgs=2105 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T15:08:22.876 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:22.872+0000 7f7c6ee7a640 1 -- 172.21.15.43:0/3438208740 >> 172.21.15.43:0/3438208740 conn(0x7f7c68071520 msgr2=0x7f7c680ceee0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-09T15:08:22.876 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:22.872+0000 7f7c6ee7a640 1 -- 172.21.15.43:0/3438208740 shutdown_connections 2023-08-09T15:08:22.877 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:22.872+0000 7f7c6ee7a640 1 -- 172.21.15.43:0/3438208740 wait complete. 2023-08-09T15:08:22.879 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:22.874+0000 7f7c6ee7a640 1 Processor -- start 2023-08-09T15:08:22.879 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:22.874+0000 7f7c6ee7a640 1 -- start start 2023-08-09T15:08:22.879 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:22.875+0000 7f7c6ee7a640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f7c680b4420 0x7f7c681f5500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T15:08:22.879 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:22.875+0000 7f7c6ee7a640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f7c680d04b0 0x7f7c681f5a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T15:08:22.879 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:22.875+0000 7f7c6ee7a640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f7c6815c550 0x7f7c680aaa90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T15:08:22.880 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:22.875+0000 7f7c6ee7a640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7c681f30d0 con 0x7f7c680b4420 2023-08-09T15:08:22.880 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:22.875+0000 7f7c6ee7a640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7c680ab7f0 con 0x7f7c6815c550 2023-08-09T15:08:22.880 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:22.875+0000 7f7c6ee7a640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7c681f3370 con 0x7f7c680d04b0 2023-08-09T15:08:22.881 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:22.875+0000 7f7c6cc16640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f7c680b4420 0x7f7c681f5500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T15:08:22.881 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:22.875+0000 7f7c6cc16640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f7c680b4420 0x7f7c681f5500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:46500/0 (socket says 172.21.15.43:46500) 2023-08-09T15:08:22.881 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:22.875+0000 7f7c6cc16640 1 -- 172.21.15.43:0/489298345 learned_addr learned my addr 172.21.15.43:0/489298345 (peer_addr_for_me v2:172.21.15.43:0/0) 2023-08-09T15:08:22.881 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:22.876+0000 7f7c67fff640 1 --2- 172.21.15.43:0/489298345 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f7c680d04b0 0x7f7c681f5a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T15:08:22.882 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:22.876+0000 7f7c6d417640 1 --2- 172.21.15.43:0/489298345 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f7c6815c550 0x7f7c680aaa90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T15:08:22.882 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:22.876+0000 7f7c6cc16640 1 -- 172.21.15.43:0/489298345 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f7c680d04b0 msgr2=0x7f7c681f5a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T15:08:22.882 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:22.876+0000 7f7c6cc16640 1 --2- 172.21.15.43:0/489298345 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f7c680d04b0 0x7f7c681f5a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T15:08:22.882 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:22.876+0000 7f7c6cc16640 1 -- 172.21.15.43:0/489298345 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f7c6815c550 msgr2=0x7f7c680aaa90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T15:08:22.883 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:22.876+0000 7f7c6cc16640 1 --2- 172.21.15.43:0/489298345 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f7c6815c550 0x7f7c680aaa90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T15:08:22.883 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:22.876+0000 7f7c6cc16640 1 -- 172.21.15.43:0/489298345 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7c5c009400 con 0x7f7c680b4420 2023-08-09T15:08:22.883 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:22.876+0000 7f7c67fff640 1 --2- 172.21.15.43:0/489298345 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f7c680d04b0 0x7f7c681f5a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-08-09T15:08:22.883 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:22.876+0000 7f7c6d417640 1 --2- 172.21.15.43:0/489298345 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f7c6815c550 0x7f7c680aaa90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-08-09T15:08:22.884 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:22.876+0000 7f7c6cc16640 1 --2- 172.21.15.43:0/489298345 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f7c680b4420 0x7f7c681f5500 secure :-1 s=READY pgs=2106 cs=0 l=1 rev1=1 crypto rx=0x7f7c5c000c00 tx=0x7f7c5c0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-09T15:08:22.884 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:22.876+0000 7f7c657fa640 1 -- 172.21.15.43:0/489298345 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7c5c03ad60 con 0x7f7c680b4420 2023-08-09T15:08:22.884 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:22.877+0000 7f7c657fa640 1 -- 172.21.15.43:0/489298345 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7c5c0443f0 con 0x7f7c680b4420 2023-08-09T15:08:22.885 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:22.877+0000 7f7c657fa640 1 -- 172.21.15.43:0/489298345 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7c5c0447a0 con 0x7f7c680b4420 2023-08-09T15:08:22.885 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:22.877+0000 7f7c6ee7a640 1 -- 172.21.15.43:0/489298345 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7c681f3650 con 0x7f7c680b4420 2023-08-09T15:08:22.885 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:22.877+0000 7f7c6ee7a640 1 -- 172.21.15.43:0/489298345 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7c680c9180 con 0x7f7c680b4420 2023-08-09T15:08:22.885 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:22.878+0000 7f7c6ee7a640 1 -- 172.21.15.43:0/489298345 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7c680739f0 con 0x7f7c680b4420 2023-08-09T15:08:22.887 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:22.879+0000 7f7c657fa640 1 -- 172.21.15.43:0/489298345 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 102) v1 ==== 88455+0+0 (secure 0 0 0) 0x7f7c5c04d450 con 0x7f7c680b4420 2023-08-09T15:08:22.887 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:22.880+0000 7f7c657fa640 1 --2- 172.21.15.43:0/489298345 >> [v2:172.21.15.203:6832/115830229,v1:172.21.15.203:6833/115830229] conn(0x7f7c4c06a1a0 0x7f7c4c06c660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T15:08:22.887 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:22.880+0000 7f7c657fa640 1 -- 172.21.15.43:0/489298345 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(191..191 src has 1..191) v4 ==== 7202+0+0 (secure 0 0 0) 0x7f7c5c0aafb0 con 0x7f7c680b4420 2023-08-09T15:08:22.888 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:22.880+0000 7f7c67fff640 1 -- 172.21.15.43:0/489298345 >> [v2:172.21.15.203:6832/115830229,v1:172.21.15.203:6833/115830229] conn(0x7f7c4c06a1a0 msgr2=0x7f7c4c06c660 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.203:6832/115830229 2023-08-09T15:08:22.888 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:22.880+0000 7f7c67fff640 1 --2- 172.21.15.43:0/489298345 >> [v2:172.21.15.203:6832/115830229,v1:172.21.15.203:6833/115830229] conn(0x7f7c4c06a1a0 0x7f7c4c06c660 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-08-09T15:08:22.888 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:22.884+0000 7f7c657fa640 1 -- 172.21.15.43:0/489298345 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178066 (secure 0 0 0) 0x7f7c5c075f60 con 0x7f7c680b4420 2023-08-09T15:08:23.075 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:23.074+0000 7f7c6ee7a640 1 -- 172.21.15.43:0/489298345 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f7c680d1020 con 0x7f7c680b4420 2023-08-09T15:08:23.076 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:23.075+0000 7f7c657fa640 1 -- 172.21.15.43:0/489298345 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v191) v1 ==== 93+0+10 (secure 0 0 0) 0x7f7c5c0350a0 con 0x7f7c680b4420 2023-08-09T15:08:23.076 INFO:teuthology.orchestra.run.smithi043.stdout:pg_num: 1 2023-08-09T15:08:23.078 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:23.077+0000 7f7c6ee7a640 1 -- 172.21.15.43:0/489298345 >> [v2:172.21.15.203:6832/115830229,v1:172.21.15.203:6833/115830229] conn(0x7f7c4c06a1a0 msgr2=0x7f7c4c06c660 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-08-09T15:08:23.079 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:23.077+0000 7f7c6ee7a640 1 --2- 172.21.15.43:0/489298345 >> [v2:172.21.15.203:6832/115830229,v1:172.21.15.203:6833/115830229] conn(0x7f7c4c06a1a0 0x7f7c4c06c660 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T15:08:23.079 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:23.077+0000 7f7c6ee7a640 1 -- 172.21.15.43:0/489298345 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f7c680b4420 msgr2=0x7f7c681f5500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T15:08:23.079 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:23.077+0000 7f7c6ee7a640 1 --2- 172.21.15.43:0/489298345 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f7c680b4420 0x7f7c681f5500 secure :-1 s=READY pgs=2106 cs=0 l=1 rev1=1 crypto rx=0x7f7c5c000c00 tx=0x7f7c5c0046c0 comp rx=0 tx=0).stop 2023-08-09T15:08:23.079 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:23.077+0000 7f7c6ee7a640 1 -- 172.21.15.43:0/489298345 shutdown_connections 2023-08-09T15:08:23.079 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:23.077+0000 7f7c6ee7a640 1 --2- 172.21.15.43:0/489298345 >> [v2:172.21.15.203:6832/115830229,v1:172.21.15.203:6833/115830229] conn(0x7f7c4c06a1a0 0x7f7c4c06c660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T15:08:23.079 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:23.077+0000 7f7c6ee7a640 1 --2- 172.21.15.43:0/489298345 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f7c6815c550 0x7f7c680aaa90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T15:08:23.079 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:23.077+0000 7f7c6ee7a640 1 --2- 172.21.15.43:0/489298345 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f7c680d04b0 0x7f7c681f5a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T15:08:23.080 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:23.078+0000 7f7c6ee7a640 1 --2- 172.21.15.43:0/489298345 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f7c680b4420 0x7f7c681f5500 unknown :-1 s=CLOSED pgs=2106 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T15:08:23.080 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:23.078+0000 7f7c6ee7a640 1 -- 172.21.15.43:0/489298345 >> 172.21.15.43:0/489298345 conn(0x7f7c68071520 msgr2=0x7f7c680cc980 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-09T15:08:23.080 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:23.078+0000 7f7c6ee7a640 1 -- 172.21.15.43:0/489298345 shutdown_connections 2023-08-09T15:08:23.080 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:23.078+0000 7f7c6ee7a640 1 -- 172.21.15.43:0/489298345 wait complete. 2023-08-09T15:08:23.098 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-08-09T15:08:23.358 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:23.356+0000 7f57ebec5640 1 Processor -- start 2023-08-09T15:08:23.358 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:23.357+0000 7f57ebec5640 1 -- start start 2023-08-09T15:08:23.360 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:23.358+0000 7f57ebec5640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f57e40cfa40 0x7f57e406d460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T15:08:23.361 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:23.358+0000 7f57ebec5640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f57e40ccd10 0x7f57e40cd160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T15:08:23.361 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:23.358+0000 7f57ebec5640 1 -- --> v1:172.21.15.43:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f57e40cd6a0 con 0x7f57e4160160 2023-08-09T15:08:23.361 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:23.358+0000 7f57ebec5640 1 -- --> v2:172.21.15.43:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f57e415cb70 con 0x7f57e40cfa40 2023-08-09T15:08:23.361 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:23.358+0000 7f57ebec5640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f57e4160610 con 0x7f57e40ccd10 2023-08-09T15:08:23.362 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:23.358+0000 7f57e9c61640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f57e40cfa40 0x7f57e406d460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T15:08:23.362 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:23.359+0000 7f57e9c61640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f57e40cfa40 0x7f57e406d460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:46502/0 (socket says 172.21.15.43:46502) 2023-08-09T15:08:23.362 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:23.359+0000 7f57e9c61640 1 -- 172.21.15.43:0/388727673 learned_addr learned my addr 172.21.15.43:0/388727673 (peer_addr_for_me v2:172.21.15.43:0/0) 2023-08-09T15:08:23.362 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:23.359+0000 7f57ea462640 1 --2- 172.21.15.43:0/388727673 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f57e40ccd10 0x7f57e40cd160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T15:08:23.363 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:23.359+0000 7f57e9c61640 1 -- 172.21.15.43:0/388727673 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f57e40ccd10 msgr2=0x7f57e40cd160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-09T15:08:23.363 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:23.359+0000 7f57e9c61640 1 --2- 172.21.15.43:0/388727673 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f57e40ccd10 0x7f57e40cd160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T15:08:23.363 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:23.359+0000 7f57e9c61640 1 -- 172.21.15.43:0/388727673 >> v1:172.21.15.43:6789/0 conn(0x7f57e4160160 legacy=0x7f57e415d310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T15:08:23.364 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:23.359+0000 7f57e9c61640 1 -- 172.21.15.43:0/388727673 --> v2:172.21.15.43:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f57e4160890 con 0x7f57e40cfa40 2023-08-09T15:08:23.364 INFO:tasks.ceph.mgr.x.smithi203.stderr:2023-08-09T15:08:23.358+0000 7fbc59c0f040 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2023-08-09T15:08:23.368 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:23.360+0000 7f57e9c61640 1 --2- 172.21.15.43:0/388727673 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f57e40cfa40 0x7f57e406d460 secure :-1 s=READY pgs=2108 cs=0 l=1 rev1=1 crypto rx=0x7f57e0009870 tx=0x7f57e002b980 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d20c6b1e126ab89e server_cookie=0 in_seq=0 out_seq=0 2023-08-09T15:08:23.369 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:23.360+0000 7f57e8c5f640 1 -- 172.21.15.43:0/388727673 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f57e0030840 con 0x7f57e40cfa40 2023-08-09T15:08:23.369 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:23.360+0000 7f57e8c5f640 1 -- 172.21.15.43:0/388727673 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f57e00309a0 con 0x7f57e40cfa40 2023-08-09T15:08:23.369 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:23.360+0000 7f57e8c5f640 1 -- 172.21.15.43:0/388727673 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f57e0030d30 con 0x7f57e40cfa40 2023-08-09T15:08:23.369 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:23.361+0000 7f57ebec5640 1 -- 172.21.15.43:0/388727673 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f57e40cfa40 msgr2=0x7f57e406d460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T15:08:23.369 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:23.361+0000 7f57ebec5640 1 --2- 172.21.15.43:0/388727673 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f57e40cfa40 0x7f57e406d460 secure :-1 s=READY pgs=2108 cs=0 l=1 rev1=1 crypto rx=0x7f57e0009870 tx=0x7f57e002b980 comp rx=0 tx=0).stop 2023-08-09T15:08:23.369 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:23.361+0000 7f57ebec5640 1 -- 172.21.15.43:0/388727673 shutdown_connections 2023-08-09T15:08:23.369 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:23.361+0000 7f57ebec5640 1 --2- 172.21.15.43:0/388727673 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f57e40ccd10 0x7f57e40cd160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T15:08:23.370 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:23.361+0000 7f57ebec5640 1 --2- 172.21.15.43:0/388727673 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f57e40cfa40 0x7f57e406d460 unknown :-1 s=CLOSED pgs=2108 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T15:08:23.370 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:23.361+0000 7f57ebec5640 1 -- 172.21.15.43:0/388727673 >> 172.21.15.43:0/388727673 conn(0x7f57e406acc0 msgr2=0x7f57e406b0d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-09T15:08:23.370 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:23.361+0000 7f57ebec5640 1 -- 172.21.15.43:0/388727673 shutdown_connections 2023-08-09T15:08:23.370 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:23.361+0000 7f57ebec5640 1 -- 172.21.15.43:0/388727673 wait complete. 2023-08-09T15:08:23.370 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:23.363+0000 7f57ebec5640 1 Processor -- start 2023-08-09T15:08:23.370 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:23.363+0000 7f57ebec5640 1 -- start start 2023-08-09T15:08:23.370 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:23.364+0000 7f57ebec5640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f57e40ccd10 0x7f57e41f27a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T15:08:23.371 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:23.364+0000 7f57ebec5640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f57e40cfa40 0x7f57e4159e90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T15:08:23.371 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:23.364+0000 7f57ebec5640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f57e4160160 0x7f57e40c2f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T15:08:23.371 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:23.364+0000 7f57ebec5640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f57e40c57b0 con 0x7f57e40cfa40 2023-08-09T15:08:23.371 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:23.364+0000 7f57ebec5640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f57e41f2ce0 con 0x7f57e40ccd10 2023-08-09T15:08:23.371 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:23.364+0000 7f57ebec5640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f57e40c4b50 con 0x7f57e4160160 2023-08-09T15:08:23.371 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:23.364+0000 7f57e9460640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f57e40cfa40 0x7f57e4159e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T15:08:23.371 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:23.364+0000 7f57e9460640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f57e40cfa40 0x7f57e4159e90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:46518/0 (socket says 172.21.15.43:46518) 2023-08-09T15:08:23.371 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:23.364+0000 7f57e9460640 1 -- 172.21.15.43:0/1820390099 learned_addr learned my addr 172.21.15.43:0/1820390099 (peer_addr_for_me v2:172.21.15.43:0/0) 2023-08-09T15:08:23.372 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:23.365+0000 7f57e9c61640 1 --2- 172.21.15.43:0/1820390099 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f57e40ccd10 0x7f57e41f27a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T15:08:23.372 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:23.365+0000 7f57ea462640 1 --2- 172.21.15.43:0/1820390099 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f57e4160160 0x7f57e40c2f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T15:08:23.372 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:23.365+0000 7f57e9460640 1 -- 172.21.15.43:0/1820390099 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f57e4160160 msgr2=0x7f57e40c2f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T15:08:23.372 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:23.365+0000 7f57e9460640 1 --2- 172.21.15.43:0/1820390099 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f57e4160160 0x7f57e40c2f80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T15:08:23.372 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:23.365+0000 7f57e9460640 1 -- 172.21.15.43:0/1820390099 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f57e40ccd10 msgr2=0x7f57e41f27a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T15:08:23.372 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:23.365+0000 7f57e9460640 1 --2- 172.21.15.43:0/1820390099 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f57e40ccd10 0x7f57e41f27a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T15:08:23.372 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:23.365+0000 7f57e9460640 1 -- 172.21.15.43:0/1820390099 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f57e00094d0 con 0x7f57e40cfa40 2023-08-09T15:08:23.373 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:23.365+0000 7f57e9460640 1 --2- 172.21.15.43:0/1820390099 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f57e40cfa40 0x7f57e4159e90 secure :-1 s=READY pgs=2109 cs=0 l=1 rev1=1 crypto rx=0x7f57d400c450 tx=0x7f57d400c910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-09T15:08:23.373 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:23.366+0000 7f57da7fc640 1 -- 172.21.15.43:0/1820390099 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f57d40052d0 con 0x7f57e40cfa40 2023-08-09T15:08:23.373 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:23.366+0000 7f57da7fc640 1 -- 172.21.15.43:0/1820390099 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f57d4005430 con 0x7f57e40cfa40 2023-08-09T15:08:23.373 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:23.366+0000 7f57da7fc640 1 -- 172.21.15.43:0/1820390099 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f57d4012870 con 0x7f57e40cfa40 2023-08-09T15:08:23.373 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:23.366+0000 7f57ebec5640 1 -- 172.21.15.43:0/1820390099 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f57e40c4e10 con 0x7f57e40cfa40 2023-08-09T15:08:23.373 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:23.366+0000 7f57ebec5640 1 -- 172.21.15.43:0/1820390099 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f57e40c53d0 con 0x7f57e40cfa40 2023-08-09T15:08:23.373 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:23.367+0000 7f57ebec5640 1 -- 172.21.15.43:0/1820390099 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f57e40739f0 con 0x7f57e40cfa40 2023-08-09T15:08:23.374 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:23.368+0000 7f57da7fc640 1 -- 172.21.15.43:0/1820390099 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 102) v1 ==== 88455+0+0 (secure 0 0 0) 0x7f57d40129d0 con 0x7f57e40cfa40 2023-08-09T15:08:23.374 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:23.369+0000 7f57da7fc640 1 --2- 172.21.15.43:0/1820390099 >> [v2:172.21.15.203:6832/115830229,v1:172.21.15.203:6833/115830229] conn(0x7f57c806a170 0x7f57c806c630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T15:08:23.375 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:23.369+0000 7f57da7fc640 1 -- 172.21.15.43:0/1820390099 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(191..191 src has 1..191) v4 ==== 7202+0+0 (secure 0 0 0) 0x7f57d4088560 con 0x7f57e40cfa40 2023-08-09T15:08:23.375 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:23.369+0000 7f57e9c61640 1 -- 172.21.15.43:0/1820390099 >> [v2:172.21.15.203:6832/115830229,v1:172.21.15.203:6833/115830229] conn(0x7f57c806a170 msgr2=0x7f57c806c630 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.203:6832/115830229 2023-08-09T15:08:23.375 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:23.369+0000 7f57e9c61640 1 --2- 172.21.15.43:0/1820390099 >> [v2:172.21.15.203:6832/115830229,v1:172.21.15.203:6833/115830229] conn(0x7f57c806a170 0x7f57c806c630 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-08-09T15:08:23.375 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:23.373+0000 7f57da7fc640 1 -- 172.21.15.43:0/1820390099 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178066 (secure 0 0 0) 0x7f57d40536b0 con 0x7f57e40cfa40 2023-08-09T15:08:23.423 INFO:tasks.ceph.mgr.x.smithi203.stderr:2023-08-09T15:08:23.422+0000 7fbc59c0f040 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2023-08-09T15:08:23.463 INFO:tasks.ceph.mgr.y.smithi043.stderr:2023-08-09T15:08:23.462+0000 7fe393e53040 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2023-08-09T15:08:23.472 INFO:tasks.ceph.mgr.x.smithi203.stderr:2023-08-09T15:08:23.471+0000 7fbc59c0f040 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2023-08-09T15:08:23.529 INFO:tasks.ceph.mgr.y.smithi043.stderr:2023-08-09T15:08:23.528+0000 7fe393e53040 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2023-08-09T15:08:23.563 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:23.562+0000 7f57ebec5640 1 -- 172.21.15.43:0/1820390099 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f57e40cd6a0 con 0x7f57e40cfa40 2023-08-09T15:08:23.564 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:23.563+0000 7f57da7fc640 1 -- 172.21.15.43:0/1820390099 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v191) v1 ==== 104+0+11 (secure 0 0 0) 0x7f57e40cd6a0 con 0x7f57e40cfa40 2023-08-09T15:08:23.565 INFO:teuthology.orchestra.run.smithi043.stdout:pg_num: 32 2023-08-09T15:08:23.567 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:23.565+0000 7f57ebec5640 1 -- 172.21.15.43:0/1820390099 >> [v2:172.21.15.203:6832/115830229,v1:172.21.15.203:6833/115830229] conn(0x7f57c806a170 msgr2=0x7f57c806c630 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-08-09T15:08:23.567 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:23.565+0000 7f57ebec5640 1 --2- 172.21.15.43:0/1820390099 >> [v2:172.21.15.203:6832/115830229,v1:172.21.15.203:6833/115830229] conn(0x7f57c806a170 0x7f57c806c630 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T15:08:23.567 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:23.565+0000 7f57ebec5640 1 -- 172.21.15.43:0/1820390099 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f57e40cfa40 msgr2=0x7f57e4159e90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T15:08:23.567 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:23.565+0000 7f57ebec5640 1 --2- 172.21.15.43:0/1820390099 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f57e40cfa40 0x7f57e4159e90 secure :-1 s=READY pgs=2109 cs=0 l=1 rev1=1 crypto rx=0x7f57d400c450 tx=0x7f57d400c910 comp rx=0 tx=0).stop 2023-08-09T15:08:23.567 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:23.565+0000 7f57ebec5640 1 -- 172.21.15.43:0/1820390099 shutdown_connections 2023-08-09T15:08:23.567 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:23.565+0000 7f57ebec5640 1 --2- 172.21.15.43:0/1820390099 >> [v2:172.21.15.203:6832/115830229,v1:172.21.15.203:6833/115830229] conn(0x7f57c806a170 0x7f57c806c630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T15:08:23.567 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:23.565+0000 7f57ebec5640 1 --2- 172.21.15.43:0/1820390099 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f57e4160160 0x7f57e40c2f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T15:08:23.568 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:23.565+0000 7f57ebec5640 1 --2- 172.21.15.43:0/1820390099 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f57e40cfa40 0x7f57e4159e90 unknown :-1 s=CLOSED pgs=2109 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T15:08:23.568 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:23.565+0000 7f57ebec5640 1 --2- 172.21.15.43:0/1820390099 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f57e40ccd10 0x7f57e41f27a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T15:08:23.568 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:23.565+0000 7f57ebec5640 1 -- 172.21.15.43:0/1820390099 >> 172.21.15.43:0/1820390099 conn(0x7f57e406acc0 msgr2=0x7f57e415f140 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-09T15:08:23.568 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:23.565+0000 7f57ebec5640 1 -- 172.21.15.43:0/1820390099 shutdown_connections 2023-08-09T15:08:23.568 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:23.566+0000 7f57ebec5640 1 -- 172.21.15.43:0/1820390099 wait complete. 2023-08-09T15:08:23.585 INFO:tasks.ceph.mgr.y.smithi043.stderr:2023-08-09T15:08:23.583+0000 7fe393e53040 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2023-08-09T15:08:23.586 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-08-09T15:08:23.588 INFO:tasks.ceph.mgr.x.smithi203.stderr:2023-08-09T15:08:23.587+0000 7fbc59c0f040 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2023-08-09T15:08:23.711 INFO:tasks.ceph.mgr.y.smithi043.stderr:2023-08-09T15:08:23.710+0000 7fe393e53040 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2023-08-09T15:08:23.744 INFO:tasks.ceph.mgr.x.smithi203.stderr:2023-08-09T15:08:23.742+0000 7fbc59c0f040 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2023-08-09T15:08:23.796 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:23.794+0000 7f6b7759e640 1 Processor -- start 2023-08-09T15:08:23.796 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:23.795+0000 7f6b7759e640 1 -- start start 2023-08-09T15:08:23.798 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:23.796+0000 7f6b7759e640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f6b780cd630 0x7f6b7806ed40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T15:08:23.798 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:23.797+0000 7f6b7759e640 1 -- --> v1:172.21.15.43:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6b780cc6c0 con 0x7f6b7815a1b0 2023-08-09T15:08:23.798 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:23.797+0000 7f6b7759e640 1 -- --> v1:172.21.15.203:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6b7815dee0 con 0x7f6b780b21e0 2023-08-09T15:08:23.798 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:23.797+0000 7f6b7759e640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6b7815e1c0 con 0x7f6b780cd630 2023-08-09T15:08:23.798 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:23.797+0000 7f6b7659c640 1 --1- >> v1:172.21.15.43:6789/0 conn(0x7f6b7815a1b0 0x7f6b780b1ad0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.43:6789/0 says I am v1:172.21.15.43:47372/0 (socket says 172.21.15.43:47372) 2023-08-09T15:08:23.799 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:23.797+0000 7f6b7659c640 1 -- 172.21.15.43:0/1118985666 learned_addr learned my addr 172.21.15.43:0/1118985666 (peer_addr_for_me v1:172.21.15.43:0/0) 2023-08-09T15:08:23.800 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:23.797+0000 7f6b75d9b640 1 --2- 172.21.15.43:0/1118985666 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f6b780cd630 0x7f6b7806ed40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T15:08:23.800 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:23.797+0000 7f6b7559a640 1 -- 172.21.15.43:0/1118985666 <== mon.0 v1:172.21.15.43:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1718065690 0 0) 0x7f6b780cc6c0 con 0x7f6b7815a1b0 2023-08-09T15:08:23.800 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:23.797+0000 7f6b7559a640 1 -- 172.21.15.43:0/1118985666 --> v1:172.21.15.43:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f6b60003520 con 0x7f6b7815a1b0 2023-08-09T15:08:23.801 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:23.798+0000 7f6b7559a640 1 -- 172.21.15.43:0/1118985666 <== mon.0 v1:172.21.15.43:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 4237631950 0 0) 0x7f6b60003520 con 0x7f6b7815a1b0 2023-08-09T15:08:23.801 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:23.798+0000 7f6b7559a640 1 -- 172.21.15.43:0/1118985666 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f6b780cd630 msgr2=0x7f6b7806ed40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-09T15:08:23.801 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:23.798+0000 7f6b7559a640 1 --2- 172.21.15.43:0/1118985666 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f6b780cd630 0x7f6b7806ed40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T15:08:23.801 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:23.798+0000 7f6b7559a640 1 -- 172.21.15.43:0/1118985666 >> v1:172.21.15.203:6789/0 conn(0x7f6b780b21e0 legacy=0x7f6b7806f280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T15:08:23.801 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:23.798+0000 7f6b7559a640 1 -- 172.21.15.43:0/1118985666 --> v1:172.21.15.43:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6b7815e4a0 con 0x7f6b7815a1b0 2023-08-09T15:08:23.801 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:23.798+0000 7f6b7559a640 1 -- 172.21.15.43:0/1118985666 <== mon.0 v1:172.21.15.43:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1285150740 0 0) 0x7f6b68001670 con 0x7f6b7815a1b0 2023-08-09T15:08:23.801 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:23.798+0000 7f6b7559a640 1 -- 172.21.15.43:0/1118985666 >> v1:172.21.15.43:6789/0 conn(0x7f6b7815a1b0 legacy=0x7f6b780b1ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T15:08:23.802 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:23.798+0000 7f6b7559a640 1 --2- 172.21.15.43:0/1118985666 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f6b600039a0 0x7f6b60003dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T15:08:23.802 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:23.798+0000 7f6b7559a640 1 --2- 172.21.15.43:0/1118985666 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f6b780cd630 0x7f6b600047c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T15:08:23.802 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:23.798+0000 7f6b7559a640 1 --2- 172.21.15.43:0/1118985666 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f6b60004d00 0x7f6b60005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T15:08:23.802 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:23.798+0000 7f6b7559a640 1 -- 172.21.15.43:0/1118985666 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6b7815e1c0 con 0x7f6b60004d00 2023-08-09T15:08:23.802 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:23.798+0000 7f6b7559a640 1 -- 172.21.15.43:0/1118985666 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6b6000ba30 con 0x7f6b600039a0 2023-08-09T15:08:23.802 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:23.798+0000 7f6b7559a640 1 -- 172.21.15.43:0/1118985666 --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6b6000bbf0 con 0x7f6b780cd630 2023-08-09T15:08:23.802 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:23.798+0000 7f6b76d9d640 1 --2- 172.21.15.43:0/1118985666 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f6b60004d00 0x7f6b60005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T15:08:23.803 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:23.799+0000 7f6b7659c640 1 --2- 172.21.15.43:0/1118985666 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f6b600039a0 0x7f6b60003dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T15:08:23.803 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:23.799+0000 7f6b75d9b640 1 --2- 172.21.15.43:0/1118985666 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f6b780cd630 0x7f6b600047c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T15:08:23.803 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:23.799+0000 7f6b76d9d640 1 -- 172.21.15.43:0/1118985666 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f6b780cd630 msgr2=0x7f6b600047c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-09T15:08:23.803 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:23.799+0000 7f6b76d9d640 1 --2- 172.21.15.43:0/1118985666 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f6b780cd630 0x7f6b600047c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T15:08:23.803 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:23.799+0000 7f6b76d9d640 1 -- 172.21.15.43:0/1118985666 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f6b600039a0 msgr2=0x7f6b60003dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-09T15:08:23.803 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:23.799+0000 7f6b76d9d640 1 --2- 172.21.15.43:0/1118985666 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f6b600039a0 0x7f6b60003dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T15:08:23.803 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:23.799+0000 7f6b76d9d640 1 -- 172.21.15.43:0/1118985666 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6b60003520 con 0x7f6b60004d00 2023-08-09T15:08:23.803 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:23.799+0000 7f6b75d9b640 1 --2- 172.21.15.43:0/1118985666 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f6b780cd630 0x7f6b600047c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-08-09T15:08:23.804 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:23.801+0000 7f6b76d9d640 1 --2- 172.21.15.43:0/1118985666 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f6b60004d00 0x7f6b60005180 secure :-1 s=READY pgs=2111 cs=0 l=1 rev1=1 crypto rx=0x7f6b7804dcd0 tx=0x7f6b600057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1e7becb7b9f4f61c server_cookie=0 in_seq=0 out_seq=0 2023-08-09T15:08:23.805 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:23.801+0000 7f6b7559a640 1 -- 172.21.15.43:0/1118985666 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6b70003b70 con 0x7f6b60004d00 2023-08-09T15:08:23.805 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:23.801+0000 7f6b7559a640 1 -- 172.21.15.43:0/1118985666 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6b70003cd0 con 0x7f6b60004d00 2023-08-09T15:08:23.805 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:23.801+0000 7f6b7559a640 1 -- 172.21.15.43:0/1118985666 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6b70005310 con 0x7f6b60004d00 2023-08-09T15:08:23.805 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:23.802+0000 7f6b7759e640 1 -- 172.21.15.43:0/1118985666 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f6b60004d00 msgr2=0x7f6b60005180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T15:08:23.805 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:23.802+0000 7f6b7759e640 1 --2- 172.21.15.43:0/1118985666 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f6b60004d00 0x7f6b60005180 secure :-1 s=READY pgs=2111 cs=0 l=1 rev1=1 crypto rx=0x7f6b7804dcd0 tx=0x7f6b600057f0 comp rx=0 tx=0).stop 2023-08-09T15:08:23.805 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:23.802+0000 7f6b76d9d640 1 -- 172.21.15.43:0/1118985666 reap_dead start 2023-08-09T15:08:23.805 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:23.802+0000 7f6b7759e640 1 -- 172.21.15.43:0/1118985666 shutdown_connections 2023-08-09T15:08:23.806 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:23.802+0000 7f6b7759e640 1 -- 172.21.15.43:0/1118985666 >> 172.21.15.43:0/1118985666 conn(0x7f6b780ccad0 msgr2=0x7f6b780cceb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-09T15:08:23.806 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:23.802+0000 7f6b7759e640 1 -- 172.21.15.43:0/1118985666 shutdown_connections 2023-08-09T15:08:23.806 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:23.802+0000 7f6b7759e640 1 -- 172.21.15.43:0/1118985666 wait complete. 2023-08-09T15:08:23.812 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:23.805+0000 7f6b7759e640 1 Processor -- start 2023-08-09T15:08:23.813 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:23.805+0000 7f6b7759e640 1 -- start start 2023-08-09T15:08:23.813 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:23.806+0000 7f6b7759e640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f6b60004d00 0x7f6b7815a710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T15:08:23.813 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:23.807+0000 7f6b7759e640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f6b7814e5d0 0x7f6b780c8c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T15:08:23.813 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:23.807+0000 7f6b7759e640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f6b78158ec0 0x7f6b78159340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T15:08:23.813 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:23.807+0000 7f6b7759e640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6b78159880 con 0x7f6b78158ec0 2023-08-09T15:08:23.814 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:23.807+0000 7f6b7759e640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6b780c9150 con 0x7f6b60004d00 2023-08-09T15:08:23.814 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:23.807+0000 7f6b7759e640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6b7814ea50 con 0x7f6b7814e5d0 2023-08-09T15:08:23.814 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:23.807+0000 7f6b76d9d640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f6b78158ec0 0x7f6b78159340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T15:08:23.814 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:23.807+0000 7f6b76d9d640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f6b78158ec0 0x7f6b78159340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:46534/0 (socket says 172.21.15.43:46534) 2023-08-09T15:08:23.814 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:23.807+0000 7f6b76d9d640 1 -- 172.21.15.43:0/382989098 learned_addr learned my addr 172.21.15.43:0/382989098 (peer_addr_for_me v2:172.21.15.43:0/0) 2023-08-09T15:08:23.814 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:23.807+0000 7f6b7659c640 1 --2- 172.21.15.43:0/382989098 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f6b60004d00 0x7f6b7815a710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T15:08:23.814 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:23.807+0000 7f6b75d9b640 1 --2- 172.21.15.43:0/382989098 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f6b7814e5d0 0x7f6b780c8c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T15:08:23.815 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:23.807+0000 7f6b76d9d640 1 -- 172.21.15.43:0/382989098 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f6b7814e5d0 msgr2=0x7f6b780c8c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T15:08:23.815 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:23.807+0000 7f6b76d9d640 1 --2- 172.21.15.43:0/382989098 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f6b7814e5d0 0x7f6b780c8c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T15:08:23.815 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:23.807+0000 7f6b76d9d640 1 -- 172.21.15.43:0/382989098 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f6b60004d00 msgr2=0x7f6b7815a710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T15:08:23.815 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:23.807+0000 7f6b76d9d640 1 --2- 172.21.15.43:0/382989098 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f6b60004d00 0x7f6b7815a710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T15:08:23.815 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:23.807+0000 7f6b76d9d640 1 -- 172.21.15.43:0/382989098 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6b7815add0 con 0x7f6b78158ec0 2023-08-09T15:08:23.816 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:23.808+0000 7f6b7659c640 1 --2- 172.21.15.43:0/382989098 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f6b60004d00 0x7f6b7815a710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-08-09T15:08:23.816 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:23.808+0000 7f6b76d9d640 1 --2- 172.21.15.43:0/382989098 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f6b78158ec0 0x7f6b78159340 secure :-1 s=READY pgs=2112 cs=0 l=1 rev1=1 crypto rx=0x7f6b70001890 tx=0x7f6b700014a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-09T15:08:23.816 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:23.808+0000 7f6b66ffd640 1 -- 172.21.15.43:0/382989098 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6b70006d50 con 0x7f6b78158ec0 2023-08-09T15:08:23.816 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:23.808+0000 7f6b66ffd640 1 -- 172.21.15.43:0/382989098 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6b70003e40 con 0x7f6b78158ec0 2023-08-09T15:08:23.816 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:23.808+0000 7f6b66ffd640 1 -- 172.21.15.43:0/382989098 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6b70007230 con 0x7f6b78158ec0 2023-08-09T15:08:23.816 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:23.808+0000 7f6b7759e640 1 -- 172.21.15.43:0/382989098 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6b7815b0b0 con 0x7f6b78158ec0 2023-08-09T15:08:23.816 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:23.808+0000 7f6b7759e640 1 -- 172.21.15.43:0/382989098 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6b7815b670 con 0x7f6b78158ec0 2023-08-09T15:08:23.817 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:23.810+0000 7f6b7759e640 1 -- 172.21.15.43:0/382989098 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6b7804e390 con 0x7f6b78158ec0 2023-08-09T15:08:23.817 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:23.811+0000 7f6b66ffd640 1 -- 172.21.15.43:0/382989098 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 102) v1 ==== 88455+0+0 (secure 0 0 0) 0x7f6b70007390 con 0x7f6b78158ec0 2023-08-09T15:08:23.817 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:23.812+0000 7f6b66ffd640 1 --2- 172.21.15.43:0/382989098 >> [v2:172.21.15.203:6832/115830229,v1:172.21.15.203:6833/115830229] conn(0x7f6b5806a170 0x7f6b5806c630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T15:08:23.818 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:23.812+0000 7f6b66ffd640 1 -- 172.21.15.43:0/382989098 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(191..191 src has 1..191) v4 ==== 7202+0+0 (secure 0 0 0) 0x7f6b700aa2c0 con 0x7f6b78158ec0 2023-08-09T15:08:23.818 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:23.813+0000 7f6b7659c640 1 -- 172.21.15.43:0/382989098 >> [v2:172.21.15.203:6832/115830229,v1:172.21.15.203:6833/115830229] conn(0x7f6b5806a170 msgr2=0x7f6b5806c630 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.203:6832/115830229 2023-08-09T15:08:23.818 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:23.813+0000 7f6b7659c640 1 --2- 172.21.15.43:0/382989098 >> [v2:172.21.15.203:6832/115830229,v1:172.21.15.203:6833/115830229] conn(0x7f6b5806a170 0x7f6b5806c630 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-08-09T15:08:23.818 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:23.817+0000 7f6b66ffd640 1 -- 172.21.15.43:0/382989098 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178066 (secure 0 0 0) 0x7f6b70075380 con 0x7f6b78158ec0 2023-08-09T15:08:23.853 INFO:tasks.ceph.mgr.x.smithi203.stderr:2023-08-09T15:08:23.851+0000 7fbc59c0f040 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2023-08-09T15:08:23.887 INFO:tasks.ceph.mgr.y.smithi043.stderr:2023-08-09T15:08:23.885+0000 7fe393e53040 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2023-08-09T15:08:23.902 INFO:tasks.ceph.mgr.x.smithi203.stderr:2023-08-09T15:08:23.900+0000 7fbc59c0f040 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2023-08-09T15:08:24.001 INFO:tasks.ceph.mgr.y.smithi043.stderr:2023-08-09T15:08:24.000+0000 7fe393e53040 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2023-08-09T15:08:24.007 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:24.006+0000 7f6b7759e640 1 -- 172.21.15.43:0/382989098 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f6b780b2870 con 0x7f6b78158ec0 2023-08-09T15:08:24.007 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:24.006+0000 7f6b66ffd640 1 -- 172.21.15.43:0/382989098 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v191) v1 ==== 100+0+11 (secure 0 0 0) 0x7f6b700789a0 con 0x7f6b78158ec0 2023-08-09T15:08:24.008 INFO:teuthology.orchestra.run.smithi043.stdout:pg_num: 64 2023-08-09T15:08:24.010 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:24.009+0000 7f6b7759e640 1 -- 172.21.15.43:0/382989098 >> [v2:172.21.15.203:6832/115830229,v1:172.21.15.203:6833/115830229] conn(0x7f6b5806a170 msgr2=0x7f6b5806c630 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-08-09T15:08:24.011 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:24.009+0000 7f6b7759e640 1 --2- 172.21.15.43:0/382989098 >> [v2:172.21.15.203:6832/115830229,v1:172.21.15.203:6833/115830229] conn(0x7f6b5806a170 0x7f6b5806c630 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T15:08:24.011 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:24.009+0000 7f6b7759e640 1 -- 172.21.15.43:0/382989098 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f6b78158ec0 msgr2=0x7f6b78159340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T15:08:24.011 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:24.009+0000 7f6b7759e640 1 --2- 172.21.15.43:0/382989098 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f6b78158ec0 0x7f6b78159340 secure :-1 s=READY pgs=2112 cs=0 l=1 rev1=1 crypto rx=0x7f6b70001890 tx=0x7f6b700014a0 comp rx=0 tx=0).stop 2023-08-09T15:08:24.011 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:24.010+0000 7f6b7759e640 1 -- 172.21.15.43:0/382989098 shutdown_connections 2023-08-09T15:08:24.011 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:24.010+0000 7f6b7759e640 1 --2- 172.21.15.43:0/382989098 >> [v2:172.21.15.203:6832/115830229,v1:172.21.15.203:6833/115830229] conn(0x7f6b5806a170 0x7f6b5806c630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T15:08:24.011 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:24.010+0000 7f6b7759e640 1 --2- 172.21.15.43:0/382989098 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f6b78158ec0 0x7f6b78159340 unknown :-1 s=CLOSED pgs=2112 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T15:08:24.012 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:24.010+0000 7f6b7759e640 1 --2- 172.21.15.43:0/382989098 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f6b7814e5d0 0x7f6b780c8c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T15:08:24.012 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:24.010+0000 7f6b7759e640 1 --2- 172.21.15.43:0/382989098 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f6b60004d00 0x7f6b7815a710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T15:08:24.012 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:24.010+0000 7f6b7759e640 1 -- 172.21.15.43:0/382989098 >> 172.21.15.43:0/382989098 conn(0x7f6b780ccad0 msgr2=0x7f6b780b1990 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-09T15:08:24.012 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:24.010+0000 7f6b7759e640 1 -- 172.21.15.43:0/382989098 shutdown_connections 2023-08-09T15:08:24.013 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:24.010+0000 7f6b7759e640 1 -- 172.21.15.43:0/382989098 wait complete. 2023-08-09T15:08:24.031 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2023-08-09T15:08:24.060 INFO:tasks.ceph.mgr.y.smithi043.stderr:2023-08-09T15:08:24.058+0000 7fe393e53040 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2023-08-09T15:08:24.064 INFO:tasks.ceph.mgr.x.smithi203.stderr:2023-08-09T15:08:24.063+0000 7fbc59c0f040 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2023-08-09T15:08:24.232 INFO:tasks.ceph.mgr.y.smithi043.stderr:2023-08-09T15:08:24.230+0000 7fe393e53040 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2023-08-09T15:08:24.245 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:24.244+0000 7f231494f640 1 Processor -- start 2023-08-09T15:08:24.245 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:24.244+0000 7f231494f640 1 -- start start 2023-08-09T15:08:24.247 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:24.245+0000 7f231494f640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f23100cf550 0x7f2310167280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T15:08:24.247 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:24.245+0000 7f231494f640 1 -- --> v1:172.21.15.43:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f23100d0700 con 0x7f231015c930 2023-08-09T15:08:24.247 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:24.245+0000 7f231494f640 1 -- --> v1:172.21.15.203:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f23100d09e0 con 0x7f231015c550 2023-08-09T15:08:24.247 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:24.245+0000 7f231494f640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f23100d0cc0 con 0x7f23100cf550 2023-08-09T15:08:24.247 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:24.246+0000 7f2305d9b640 1 --1- >> v1:172.21.15.43:6789/0 conn(0x7f231015c930 0x7f23100cee40 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.43:6789/0 says I am v1:172.21.15.43:47384/0 (socket says 172.21.15.43:47384) 2023-08-09T15:08:24.247 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:24.246+0000 7f2305d9b640 1 -- 172.21.15.43:0/2319080630 learned_addr learned my addr 172.21.15.43:0/2319080630 (peer_addr_for_me v1:172.21.15.43:0/0) 2023-08-09T15:08:24.249 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:24.246+0000 7f230ed9d640 1 --2- 172.21.15.43:0/2319080630 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f23100cf550 0x7f2310167280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T15:08:24.249 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:24.246+0000 7f230dd9b640 1 -- 172.21.15.43:0/2319080630 <== mon.0 v1:172.21.15.43:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1477571616 0 0) 0x7f23100d0700 con 0x7f231015c930 2023-08-09T15:08:24.249 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:24.246+0000 7f230dd9b640 1 -- 172.21.15.43:0/2319080630 --> v1:172.21.15.43:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f22fc003520 con 0x7f231015c930 2023-08-09T15:08:24.250 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:24.247+0000 7f230dd9b640 1 -- 172.21.15.43:0/2319080630 <== mon.1 v1:172.21.15.203:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3624096764 0 0) 0x7f23100d09e0 con 0x7f231015c550 2023-08-09T15:08:24.250 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:24.247+0000 7f230dd9b640 1 -- 172.21.15.43:0/2319080630 --> v1:172.21.15.203:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f23100d0700 con 0x7f231015c550 2023-08-09T15:08:24.250 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:24.247+0000 7f230dd9b640 1 -- 172.21.15.43:0/2319080630 <== mon.0 v1:172.21.15.43:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1388482233 0 0) 0x7f22fc003520 con 0x7f231015c930 2023-08-09T15:08:24.250 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:24.247+0000 7f230dd9b640 1 -- 172.21.15.43:0/2319080630 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f23100cf550 msgr2=0x7f2310167280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-09T15:08:24.250 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:24.247+0000 7f230dd9b640 1 --2- 172.21.15.43:0/2319080630 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f23100cf550 0x7f2310167280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T15:08:24.250 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:24.247+0000 7f230dd9b640 1 -- 172.21.15.43:0/2319080630 >> v1:172.21.15.203:6789/0 conn(0x7f231015c550 legacy=0x7f23100d1a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T15:08:24.250 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:24.247+0000 7f230dd9b640 1 -- 172.21.15.43:0/2319080630 --> v1:172.21.15.43:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f23100d0fa0 con 0x7f231015c930 2023-08-09T15:08:24.250 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:24.247+0000 7f230dd9b640 1 -- 172.21.15.43:0/2319080630 <== mon.0 v1:172.21.15.43:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1285150740 0 0) 0x7f22f8001970 con 0x7f231015c930 2023-08-09T15:08:24.251 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:24.247+0000 7f230dd9b640 1 -- 172.21.15.43:0/2319080630 >> v1:172.21.15.43:6789/0 conn(0x7f231015c930 legacy=0x7f23100cee40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T15:08:24.251 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:24.247+0000 7f230dd9b640 1 --2- 172.21.15.43:0/2319080630 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f22fc003ba0 0x7f22fc003fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T15:08:24.251 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:24.247+0000 7f230dd9b640 1 --2- 172.21.15.43:0/2319080630 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f22fc0047c0 0x7f22fc004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T15:08:24.251 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:24.247+0000 7f230dd9b640 1 --2- 172.21.15.43:0/2319080630 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f23100cf550 0x7f22fc0051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T15:08:24.251 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:24.247+0000 7f230dd9b640 1 -- 172.21.15.43:0/2319080630 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f23100d0cc0 con 0x7f22fc003ba0 2023-08-09T15:08:24.251 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:24.247+0000 7f230dd9b640 1 -- 172.21.15.43:0/2319080630 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f22fc00bae0 con 0x7f22fc0047c0 2023-08-09T15:08:24.251 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:24.247+0000 7f230dd9b640 1 -- 172.21.15.43:0/2319080630 --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f22fc00bca0 con 0x7f23100cf550 2023-08-09T15:08:24.252 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:24.247+0000 7f230e59c640 1 --2- 172.21.15.43:0/2319080630 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f22fc003ba0 0x7f22fc003fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T15:08:24.252 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:24.248+0000 7f2305d9b640 1 --2- 172.21.15.43:0/2319080630 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f22fc0047c0 0x7f22fc004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T15:08:24.252 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:24.248+0000 7f230ed9d640 1 --2- 172.21.15.43:0/2319080630 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f23100cf550 0x7f22fc0051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T15:08:24.252 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:24.248+0000 7f230e59c640 1 -- 172.21.15.43:0/2319080630 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f23100cf550 msgr2=0x7f22fc0051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-09T15:08:24.252 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:24.248+0000 7f230e59c640 1 --2- 172.21.15.43:0/2319080630 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f23100cf550 0x7f22fc0051c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T15:08:24.252 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:24.248+0000 7f230e59c640 1 -- 172.21.15.43:0/2319080630 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f22fc0047c0 msgr2=0x7f22fc004c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-09T15:08:24.252 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:24.248+0000 7f230e59c640 1 --2- 172.21.15.43:0/2319080630 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f22fc0047c0 0x7f22fc004c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T15:08:24.252 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:24.248+0000 7f230e59c640 1 -- 172.21.15.43:0/2319080630 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f22fc003520 con 0x7f22fc003ba0 2023-08-09T15:08:24.253 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:24.249+0000 7f230e59c640 1 --2- 172.21.15.43:0/2319080630 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f22fc003ba0 0x7f22fc003fa0 secure :-1 s=READY pgs=2114 cs=0 l=1 rev1=1 crypto rx=0x7f23000018d0 tx=0x7f22fc0057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dd5319fd00556da5 server_cookie=0 in_seq=0 out_seq=0 2023-08-09T15:08:24.254 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:24.249+0000 7f230dd9b640 1 -- 172.21.15.43:0/2319080630 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f23000052d0 con 0x7f22fc003ba0 2023-08-09T15:08:24.254 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:24.249+0000 7f230dd9b640 1 -- 172.21.15.43:0/2319080630 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2300005430 con 0x7f22fc003ba0 2023-08-09T15:08:24.254 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:24.249+0000 7f230dd9b640 1 -- 172.21.15.43:0/2319080630 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2300003810 con 0x7f22fc003ba0 2023-08-09T15:08:24.254 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:24.250+0000 7f231494f640 1 -- 172.21.15.43:0/2319080630 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f22fc003ba0 msgr2=0x7f22fc003fa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T15:08:24.254 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:24.250+0000 7f231494f640 1 --2- 172.21.15.43:0/2319080630 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f22fc003ba0 0x7f22fc003fa0 secure :-1 s=READY pgs=2114 cs=0 l=1 rev1=1 crypto rx=0x7f23000018d0 tx=0x7f22fc0057f0 comp rx=0 tx=0).stop 2023-08-09T15:08:24.254 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:24.250+0000 7f230ed9d640 1 -- 172.21.15.43:0/2319080630 reap_dead start 2023-08-09T15:08:24.254 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:24.250+0000 7f231494f640 1 -- 172.21.15.43:0/2319080630 shutdown_connections 2023-08-09T15:08:24.255 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:24.250+0000 7f231494f640 1 -- 172.21.15.43:0/2319080630 >> 172.21.15.43:0/2319080630 conn(0x7f231006a4e0 msgr2=0x7f231006a8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-09T15:08:24.255 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:24.250+0000 7f231494f640 1 -- 172.21.15.43:0/2319080630 shutdown_connections 2023-08-09T15:08:24.255 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:24.250+0000 7f231494f640 1 -- 172.21.15.43:0/2319080630 wait complete. 2023-08-09T15:08:24.255 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:24.252+0000 7f231494f640 1 Processor -- start 2023-08-09T15:08:24.255 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:24.252+0000 7f231494f640 1 -- start start 2023-08-09T15:08:24.257 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:24.253+0000 7f231494f640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f22fc003ba0 0x7f23101599e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T15:08:24.257 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:24.253+0000 7f231494f640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f23100c7580 0x7f23100c7a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T15:08:24.257 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:24.253+0000 7f231494f640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f23100b5290 0x7f23100b5710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T15:08:24.257 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:24.253+0000 7f231494f640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f23100c7f40 con 0x7f22fc003ba0 2023-08-09T15:08:24.257 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:24.253+0000 7f231494f640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f23100b5c50 con 0x7f23100b5290 2023-08-09T15:08:24.258 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:24.253+0000 7f231494f640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f23100c8d50 con 0x7f23100c7580 2023-08-09T15:08:24.258 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:24.253+0000 7f230e59c640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f22fc003ba0 0x7f23101599e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T15:08:24.258 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:24.253+0000 7f230e59c640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f22fc003ba0 0x7f23101599e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:46554/0 (socket says 172.21.15.43:46554) 2023-08-09T15:08:24.258 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:24.253+0000 7f230e59c640 1 -- 172.21.15.43:0/201388251 learned_addr learned my addr 172.21.15.43:0/201388251 (peer_addr_for_me v2:172.21.15.43:0/0) 2023-08-09T15:08:24.258 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:24.253+0000 7f230ed9d640 1 --2- 172.21.15.43:0/201388251 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f23100b5290 0x7f23100b5710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T15:08:24.258 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:24.253+0000 7f2305d9b640 1 --2- 172.21.15.43:0/201388251 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f23100c7580 0x7f23100c7a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T15:08:24.258 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:24.254+0000 7f230e59c640 1 -- 172.21.15.43:0/201388251 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f23100c7580 msgr2=0x7f23100c7a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T15:08:24.259 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:24.254+0000 7f230e59c640 1 --2- 172.21.15.43:0/201388251 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f23100c7580 0x7f23100c7a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T15:08:24.259 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:24.254+0000 7f230e59c640 1 -- 172.21.15.43:0/201388251 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f23100b5290 msgr2=0x7f23100b5710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T15:08:24.259 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:24.254+0000 7f230e59c640 1 --2- 172.21.15.43:0/201388251 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f23100b5290 0x7f23100b5710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T15:08:24.259 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:24.254+0000 7f230e59c640 1 -- 172.21.15.43:0/201388251 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f23101f5620 con 0x7f22fc003ba0 2023-08-09T15:08:24.259 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:24.254+0000 7f2305d9b640 1 --2- 172.21.15.43:0/201388251 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f23100c7580 0x7f23100c7a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-08-09T15:08:24.259 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:24.254+0000 7f230e59c640 1 --2- 172.21.15.43:0/201388251 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f22fc003ba0 0x7f23101599e0 secure :-1 s=READY pgs=2115 cs=0 l=1 rev1=1 crypto rx=0x7f2300005040 tx=0x7f23000012d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-09T15:08:24.259 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:24.254+0000 7f23077fe640 1 -- 172.21.15.43:0/201388251 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2300007450 con 0x7f22fc003ba0 2023-08-09T15:08:24.259 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:24.254+0000 7f23077fe640 1 -- 172.21.15.43:0/201388251 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f23000075b0 con 0x7f22fc003ba0 2023-08-09T15:08:24.260 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:24.254+0000 7f23077fe640 1 -- 172.21.15.43:0/201388251 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2300007720 con 0x7f22fc003ba0 2023-08-09T15:08:24.260 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:24.254+0000 7f231494f640 1 -- 172.21.15.43:0/201388251 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f23100c8140 con 0x7f22fc003ba0 2023-08-09T15:08:24.260 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:24.255+0000 7f231494f640 1 -- 172.21.15.43:0/201388251 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f23100c8700 con 0x7f22fc003ba0 2023-08-09T15:08:24.260 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:24.256+0000 7f231494f640 1 -- 172.21.15.43:0/201388251 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f23100739f0 con 0x7f22fc003ba0 2023-08-09T15:08:24.261 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:24.257+0000 7f23077fe640 1 -- 172.21.15.43:0/201388251 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 102) v1 ==== 88455+0+0 (secure 0 0 0) 0x7f23000037f0 con 0x7f22fc003ba0 2023-08-09T15:08:24.261 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:24.258+0000 7f23077fe640 1 --2- 172.21.15.43:0/201388251 >> [v2:172.21.15.203:6832/115830229,v1:172.21.15.203:6833/115830229] conn(0x7f22f006a1a0 0x7f22f006c660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T15:08:24.261 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:24.258+0000 7f23077fe640 1 -- 172.21.15.43:0/201388251 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(191..191 src has 1..191) v4 ==== 7202+0+0 (secure 0 0 0) 0x7f23000ab040 con 0x7f22fc003ba0 2023-08-09T15:08:24.261 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:24.258+0000 7f2305d9b640 1 -- 172.21.15.43:0/201388251 >> [v2:172.21.15.203:6832/115830229,v1:172.21.15.203:6833/115830229] conn(0x7f22f006a1a0 msgr2=0x7f22f006c660 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.203:6832/115830229 2023-08-09T15:08:24.261 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:24.258+0000 7f2305d9b640 1 --2- 172.21.15.43:0/201388251 >> [v2:172.21.15.203:6832/115830229,v1:172.21.15.203:6833/115830229] conn(0x7f22f006a1a0 0x7f22f006c660 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-08-09T15:08:24.262 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:24.261+0000 7f23077fe640 1 -- 172.21.15.43:0/201388251 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178066 (secure 0 0 0) 0x7f2300076230 con 0x7f22fc003ba0 2023-08-09T15:08:24.450 INFO:tasks.ceph.mgr.x.smithi203.stderr:2023-08-09T15:08:24.449+0000 7fbc59c0f040 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2023-08-09T15:08:24.460 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:24.459+0000 7f2305d9b640 1 -- 172.21.15.43:0/201388251 >> [v2:172.21.15.203:6832/115830229,v1:172.21.15.203:6833/115830229] conn(0x7f22f006a1a0 msgr2=0x7f22f006c660 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.203:6832/115830229 2023-08-09T15:08:24.461 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:24.459+0000 7f2305d9b640 1 --2- 172.21.15.43:0/201388251 >> [v2:172.21.15.203:6832/115830229,v1:172.21.15.203:6833/115830229] conn(0x7f22f006a1a0 0x7f22f006c660 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2023-08-09T15:08:24.469 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:24.467+0000 7f231494f640 1 -- 172.21.15.43:0/201388251 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7f23100d2160 con 0x7f22fc003ba0 2023-08-09T15:08:24.469 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:24.468+0000 7f23077fe640 1 -- 172.21.15.43:0/201388251 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 16 entries v191) v1 ==== 81+0+947 (secure 0 0 0) 0x7f2300037080 con 0x7f22fc003ba0 2023-08-09T15:08:24.470 INFO:teuthology.orchestra.run.smithi043.stdout:172.21.15.203:0/69497804 2023-08-10T15:03:21.353235+0000 2023-08-09T15:08:24.470 INFO:teuthology.orchestra.run.smithi043.stdout:172.21.15.203:0/169967264 2023-08-10T15:03:21.353235+0000 2023-08-09T15:08:24.470 INFO:teuthology.orchestra.run.smithi043.stdout:172.21.15.203:6833/40883597 2023-08-10T15:03:21.353235+0000 2023-08-09T15:08:24.470 INFO:teuthology.orchestra.run.smithi043.stdout:172.21.15.203:6832/40883597 2023-08-10T15:03:21.353235+0000 2023-08-09T15:08:24.470 INFO:teuthology.orchestra.run.smithi043.stdout:172.21.15.203:0/3690197627 2023-08-10T15:03:21.353235+0000 2023-08-09T15:08:24.470 INFO:teuthology.orchestra.run.smithi043.stdout:172.21.15.203:0/1082753911 2023-08-10T15:03:21.353235+0000 2023-08-09T15:08:24.471 INFO:teuthology.orchestra.run.smithi043.stdout:172.21.15.203:6832/2959400181 2023-08-10T14:52:45.188641+0000 2023-08-09T15:08:24.471 INFO:teuthology.orchestra.run.smithi043.stdout:172.21.15.203:0/82655847 2023-08-10T15:03:21.353235+0000 2023-08-09T15:08:24.471 INFO:teuthology.orchestra.run.smithi043.stdout:172.21.15.203:0/1935803093 2023-08-10T14:52:45.188641+0000 2023-08-09T15:08:24.471 INFO:teuthology.orchestra.run.smithi043.stdout:172.21.15.203:0/3150685329 2023-08-10T14:52:45.188641+0000 2023-08-09T15:08:24.471 INFO:teuthology.orchestra.run.smithi043.stdout:172.21.15.203:0/2219953609 2023-08-10T15:03:21.353235+0000 2023-08-09T15:08:24.471 INFO:teuthology.orchestra.run.smithi043.stdout:172.21.15.203:0/3429376127 2023-08-10T14:52:45.188641+0000 2023-08-09T15:08:24.471 INFO:teuthology.orchestra.run.smithi043.stdout:172.21.15.203:0/2197573025 2023-08-10T14:52:45.188641+0000 2023-08-09T15:08:24.472 INFO:teuthology.orchestra.run.smithi043.stdout:172.21.15.203:0/2458603841 2023-08-10T14:52:45.188641+0000 2023-08-09T15:08:24.472 INFO:teuthology.orchestra.run.smithi043.stdout:172.21.15.203:6833/2959400181 2023-08-10T14:52:45.188641+0000 2023-08-09T15:08:24.472 INFO:teuthology.orchestra.run.smithi043.stdout:172.21.15.203:0/4177046277 2023-08-10T14:52:45.188641+0000 2023-08-09T15:08:24.473 INFO:teuthology.orchestra.run.smithi043.stderr:listed 16 entries 2023-08-09T15:08:24.473 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:24.470+0000 7f231494f640 1 -- 172.21.15.43:0/201388251 >> [v2:172.21.15.203:6832/115830229,v1:172.21.15.203:6833/115830229] conn(0x7f22f006a1a0 msgr2=0x7f22f006c660 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-08-09T15:08:24.473 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:24.470+0000 7f231494f640 1 --2- 172.21.15.43:0/201388251 >> [v2:172.21.15.203:6832/115830229,v1:172.21.15.203:6833/115830229] conn(0x7f22f006a1a0 0x7f22f006c660 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T15:08:24.473 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:24.470+0000 7f231494f640 1 -- 172.21.15.43:0/201388251 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f22fc003ba0 msgr2=0x7f23101599e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T15:08:24.473 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:24.470+0000 7f231494f640 1 --2- 172.21.15.43:0/201388251 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f22fc003ba0 0x7f23101599e0 secure :-1 s=READY pgs=2115 cs=0 l=1 rev1=1 crypto rx=0x7f2300005040 tx=0x7f23000012d0 comp rx=0 tx=0).stop 2023-08-09T15:08:24.473 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:24.470+0000 7f231494f640 1 -- 172.21.15.43:0/201388251 shutdown_connections 2023-08-09T15:08:24.474 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:24.470+0000 7f231494f640 1 --2- 172.21.15.43:0/201388251 >> [v2:172.21.15.203:6832/115830229,v1:172.21.15.203:6833/115830229] conn(0x7f22f006a1a0 0x7f22f006c660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T15:08:24.474 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:24.471+0000 7f231494f640 1 --2- 172.21.15.43:0/201388251 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f23100b5290 0x7f23100b5710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T15:08:24.474 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:24.471+0000 7f231494f640 1 --2- 172.21.15.43:0/201388251 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f23100c7580 0x7f23100c7a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T15:08:24.474 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:24.471+0000 7f231494f640 1 --2- 172.21.15.43:0/201388251 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f22fc003ba0 0x7f23101599e0 unknown :-1 s=CLOSED pgs=2115 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T15:08:24.474 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:24.471+0000 7f231494f640 1 -- 172.21.15.43:0/201388251 >> 172.21.15.43:0/201388251 conn(0x7f231006a4e0 msgr2=0x7f231015f140 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-09T15:08:24.474 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:24.471+0000 7f231494f640 1 -- 172.21.15.43:0/201388251 shutdown_connections 2023-08-09T15:08:24.474 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:24.471+0000 7f231494f640 1 -- 172.21.15.43:0/201388251 wait complete. 2023-08-09T15:08:24.493 INFO:tasks.cephfs.fuse_mount:Running fusermount -u on ubuntu@smithi043.front.sepia.ceph.com... 2023-08-09T15:08:24.494 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-09T15:08:24.494 DEBUG:teuthology.orchestra.run.smithi043:> sudo fusermount -u /home/ubuntu/cephtest/mnt.0 2023-08-09T15:08:24.626 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi043.stderr:ceph-fuse[143425]: fuse finished with error 0 and tester_r 0 2023-08-09T15:08:24.631 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi043.front.sepia.ceph.com 2023-08-09T15:08:24.631 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-09T15:08:24.631 DEBUG:teuthology.orchestra.run.smithi043:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2023-08-09T15:08:24.657 INFO:tasks.ceph.mgr.y.smithi043.stderr:2023-08-09T15:08:24.656+0000 7fe393e53040 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2023-08-09T15:08:24.673 INFO:tasks.ceph.mgr.x.smithi203.stderr:2023-08-09T15:08:24.672+0000 7fbc59c0f040 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2023-08-09T15:08:24.730 INFO:tasks.ceph.mgr.x.smithi203.stderr:2023-08-09T15:08:24.729+0000 7fbc59c0f040 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2023-08-09T15:08:24.780 INFO:tasks.ceph.mgr.x.smithi203.stderr:2023-08-09T15:08:24.779+0000 7fbc59c0f040 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2023-08-09T15:08:24.817 DEBUG:teuthology.orchestra.run.smithi203:> set -ex 2023-08-09T15:08:24.818 DEBUG:teuthology.orchestra.run.smithi203:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-09T15:08:24.832 INFO:tasks.ceph.mgr.x.smithi203.stderr:2023-08-09T15:08:24.831+0000 7fbc59c0f040 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2023-08-09T15:08:24.874 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4053648,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497576k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=27,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248784k,nr_inodes=812196,mode=700,uid=1000,gid=1275 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497576k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2023-08-09T15:08:24.875 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi203.front.sepia.ceph.com 2023-08-09T15:08:24.875 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-09T15:08:24.875 DEBUG:teuthology.orchestra.run.smithi203:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.1) 2023-08-09T15:08:24.895 INFO:tasks.ceph.mgr.y.smithi043.stderr:2023-08-09T15:08:24.893+0000 7fe393e53040 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2023-08-09T15:08:24.956 INFO:tasks.ceph.mgr.y.smithi043.stderr:2023-08-09T15:08:24.954+0000 7fe393e53040 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2023-08-09T15:08:24.969 INFO:teuthology.orchestra.run.smithi203.stderr:rmdir: failed to remove '/home/ubuntu/cephtest/mnt.1': No such file or directory 2023-08-09T15:08:24.970 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-08-09T15:08:24.971 DEBUG:teuthology.orchestra.run.smithi043:> set -ex 2023-08-09T15:08:24.972 DEBUG:teuthology.orchestra.run.smithi043:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-09T15:08:25.013 INFO:tasks.ceph.mgr.y.smithi043.stderr:2023-08-09T15:08:25.012+0000 7fe393e53040 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2023-08-09T15:08:25.031 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4054441,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6498844k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=27,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3249420k,nr_inodes=812355,mode=700,uid=1000,gid=1275 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6498844k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2023-08-09T15:08:25.032 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi043.front.sepia.ceph.com 2023-08-09T15:08:25.032 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-09T15:08:25.032 DEBUG:teuthology.orchestra.run.smithi043:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.2) 2023-08-09T15:08:25.069 INFO:tasks.ceph.mgr.y.smithi043.stderr:2023-08-09T15:08:25.068+0000 7fe393e53040 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2023-08-09T15:08:25.124 INFO:teuthology.orchestra.run.smithi043.stderr:rmdir: failed to remove '/home/ubuntu/cephtest/mnt.2': No such file or directory 2023-08-09T15:08:25.125 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-08-09T15:08:25.126 DEBUG:teuthology.orchestra.run.smithi203:> set -ex 2023-08-09T15:08:25.126 DEBUG:teuthology.orchestra.run.smithi203:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-09T15:08:25.135 INFO:tasks.ceph.mgr.x.smithi203.stderr:2023-08-09T15:08:25.131+0000 7fbc59c0f040 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2023-08-09T15:08:25.181 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4053648,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497576k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=27,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248784k,nr_inodes=812196,mode=700,uid=1000,gid=1275 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497576k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2023-08-09T15:08:25.182 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi203.front.sepia.ceph.com 2023-08-09T15:08:25.182 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-09T15:08:25.183 DEBUG:teuthology.orchestra.run.smithi203:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.3) 2023-08-09T15:08:25.193 INFO:tasks.ceph.mgr.x.smithi203.stderr:2023-08-09T15:08:25.192+0000 7fbc59c0f040 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2023-08-09T15:08:25.230 INFO:teuthology.orchestra.run.smithi203.stderr:rmdir: failed to remove '/home/ubuntu/cephtest/mnt.3': No such file or directory 2023-08-09T15:08:25.231 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-08-09T15:08:25.232 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-08-09T15:08:25.248 INFO:tasks.ceph.mgr.x.smithi203.stderr:2023-08-09T15:08:25.247+0000 7fbc59c0f040 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2023-08-09T15:08:25.393 INFO:tasks.ceph.mgr.y.smithi043.stderr:2023-08-09T15:08:25.391+0000 7fe393e53040 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2023-08-09T15:08:25.407 INFO:tasks.ceph.mgr.x.smithi203.stderr:2023-08-09T15:08:25.405+0000 7fbc59c0f040 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2023-08-09T15:08:25.458 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:25.457+0000 7fb90986a640 1 Processor -- start 2023-08-09T15:08:25.459 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:25.457+0000 7fb90986a640 1 -- start start 2023-08-09T15:08:25.460 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:25.458+0000 7fb90986a640 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7fb9040b2b70 0x7fb9040d2ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T15:08:25.460 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:25.458+0000 7fb90986a640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7fb9040b3050 0x7fb9040b3f60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T15:08:25.461 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:25.458+0000 7fb90986a640 1 -- --> v1:172.21.15.43:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb9040ce090 con 0x7fb9040b2790 2023-08-09T15:08:25.461 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:25.458+0000 7fb90986a640 1 -- --> v2:172.21.15.43:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb9040b3950 con 0x7fb9040b3050 2023-08-09T15:08:25.461 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:25.458+0000 7fb90986a640 1 -- --> v2:172.21.15.203:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb9040cd7f0 con 0x7fb9040b2b70 2023-08-09T15:08:25.461 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:25.458+0000 7fb902ffd640 1 --1- >> v1:172.21.15.43:6789/0 conn(0x7fb9040b2790 0x7fb9040b5330 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.43:6789/0 says I am v1:172.21.15.43:41684/0 (socket says 172.21.15.43:41684) 2023-08-09T15:08:25.461 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:25.458+0000 7fb902ffd640 1 -- 172.21.15.43:0/3026936501 learned_addr learned my addr 172.21.15.43:0/3026936501 (peer_addr_for_me v1:172.21.15.43:0/0) 2023-08-09T15:08:25.461 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:25.458+0000 7fb9037fe640 1 --2- 172.21.15.43:0/3026936501 >> v2:172.21.15.43:3300/0 conn(0x7fb9040b3050 0x7fb9040b3f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T15:08:25.461 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:25.459+0000 7fb8fbfff640 1 --2- 172.21.15.43:0/3026936501 >> v2:172.21.15.203:3300/0 conn(0x7fb9040b2b70 0x7fb9040d2ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T15:08:25.462 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:25.459+0000 7fb9027fc640 1 -- 172.21.15.43:0/3026936501 <== mon.0 v1:172.21.15.43:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3504161525 0 0) 0x7fb9040ce090 con 0x7fb9040b2790 2023-08-09T15:08:25.462 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:25.459+0000 7fb9027fc640 1 -- 172.21.15.43:0/3026936501 --> v1:172.21.15.43:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fb8f0003520 con 0x7fb9040b2790 2023-08-09T15:08:25.462 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:25.459+0000 7fb9037fe640 1 -- 172.21.15.43:0/3026936501 >> v2:172.21.15.203:3300/0 conn(0x7fb9040b2b70 msgr2=0x7fb9040d2ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-09T15:08:25.462 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:25.459+0000 7fb9037fe640 1 --2- 172.21.15.43:0/3026936501 >> v2:172.21.15.203:3300/0 conn(0x7fb9040b2b70 0x7fb9040d2ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T15:08:25.463 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:25.459+0000 7fb9037fe640 1 -- 172.21.15.43:0/3026936501 >> v1:172.21.15.43:6789/0 conn(0x7fb9040b2790 legacy=0x7fb9040b5330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T15:08:25.463 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:25.459+0000 7fb9037fe640 1 -- 172.21.15.43:0/3026936501 --> v2:172.21.15.43:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb9040cdad0 con 0x7fb9040b3050 2023-08-09T15:08:25.463 INFO:tasks.ceph.mgr.x.smithi203.stderr:2023-08-09T15:08:25.459+0000 7fbc59c0f040 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2023-08-09T15:08:25.464 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:25.461+0000 7fb9037fe640 1 --2- 172.21.15.43:0/3026936501 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb9040b3050 0x7fb9040b3f60 secure :-1 s=READY pgs=2117 cs=0 l=1 rev1=1 crypto rx=0x7fb904066af0 tx=0x7fb8fc02bed0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=45eb77b3b452c2b4 server_cookie=0 in_seq=0 out_seq=0 2023-08-09T15:08:25.464 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:25.461+0000 7fb9027fc640 1 -- 172.21.15.43:0/3026936501 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb8fc02cc80 con 0x7fb9040b3050 2023-08-09T15:08:25.464 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:25.461+0000 7fb9027fc640 1 -- 172.21.15.43:0/3026936501 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb8fc02cde0 con 0x7fb9040b3050 2023-08-09T15:08:25.465 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:25.461+0000 7fb9027fc640 1 -- 172.21.15.43:0/3026936501 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb8fc034640 con 0x7fb9040b3050 2023-08-09T15:08:25.465 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:25.462+0000 7fb90986a640 1 -- 172.21.15.43:0/3026936501 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb9040b3050 msgr2=0x7fb9040b3f60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T15:08:25.465 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:25.462+0000 7fb90986a640 1 --2- 172.21.15.43:0/3026936501 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb9040b3050 0x7fb9040b3f60 secure :-1 s=READY pgs=2117 cs=0 l=1 rev1=1 crypto rx=0x7fb904066af0 tx=0x7fb8fc02bed0 comp rx=0 tx=0).stop 2023-08-09T15:08:25.465 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:25.462+0000 7fb90986a640 1 -- 172.21.15.43:0/3026936501 shutdown_connections 2023-08-09T15:08:25.465 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:25.462+0000 7fb90986a640 1 --2- 172.21.15.43:0/3026936501 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb9040b3050 0x7fb9040b3f60 unknown :-1 s=CLOSED pgs=2117 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T15:08:25.465 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:25.462+0000 7fb90986a640 1 --2- 172.21.15.43:0/3026936501 >> v2:172.21.15.203:3300/0 conn(0x7fb9040b2b70 0x7fb9040d2ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T15:08:25.465 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:25.462+0000 7fb90986a640 1 -- 172.21.15.43:0/3026936501 >> 172.21.15.43:0/3026936501 conn(0x7fb90409b7b0 msgr2=0x7fb9040d24e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-09T15:08:25.466 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:25.462+0000 7fb90986a640 1 -- 172.21.15.43:0/3026936501 shutdown_connections 2023-08-09T15:08:25.466 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:25.462+0000 7fb90986a640 1 -- 172.21.15.43:0/3026936501 wait complete. 2023-08-09T15:08:25.466 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:25.464+0000 7fb90986a640 1 Processor -- start 2023-08-09T15:08:25.466 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:25.464+0000 7fb90986a640 1 -- start start 2023-08-09T15:08:25.467 INFO:tasks.ceph.mgr.y.smithi043.stderr:2023-08-09T15:08:25.464+0000 7fe393e53040 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2023-08-09T15:08:25.468 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:25.465+0000 7fb90986a640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb9040b2790 0x7fb9040cba50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T15:08:25.468 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:25.465+0000 7fb90986a640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fb9040b2b70 0x7fb9040c95b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T15:08:25.468 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:25.465+0000 7fb90986a640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fb9040b3050 0x7fb9040c9af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T15:08:25.468 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:25.465+0000 7fb90986a640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb9040caca0 con 0x7fb9040b2790 2023-08-09T15:08:25.469 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:25.465+0000 7fb90986a640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb9040ca120 con 0x7fb9040b3050 2023-08-09T15:08:25.469 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:25.465+0000 7fb90986a640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb9040ca400 con 0x7fb9040b2b70 2023-08-09T15:08:25.469 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:25.465+0000 7fb902ffd640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb9040b2790 0x7fb9040cba50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T15:08:25.469 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:25.465+0000 7fb902ffd640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb9040b2790 0x7fb9040cba50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:35934/0 (socket says 172.21.15.43:35934) 2023-08-09T15:08:25.469 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:25.465+0000 7fb902ffd640 1 -- 172.21.15.43:0/1172436515 learned_addr learned my addr 172.21.15.43:0/1172436515 (peer_addr_for_me v2:172.21.15.43:0/0) 2023-08-09T15:08:25.469 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:25.465+0000 7fb8fbfff640 1 --2- 172.21.15.43:0/1172436515 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fb9040b2b70 0x7fb9040c95b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T15:08:25.469 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:25.465+0000 7fb9037fe640 1 --2- 172.21.15.43:0/1172436515 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fb9040b3050 0x7fb9040c9af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T15:08:25.470 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:25.466+0000 7fb902ffd640 1 -- 172.21.15.43:0/1172436515 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fb9040b2b70 msgr2=0x7fb9040c95b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T15:08:25.470 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:25.466+0000 7fb902ffd640 1 --2- 172.21.15.43:0/1172436515 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fb9040b2b70 0x7fb9040c95b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T15:08:25.470 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:25.466+0000 7fb902ffd640 1 -- 172.21.15.43:0/1172436515 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fb9040b3050 msgr2=0x7fb9040c9af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T15:08:25.470 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:25.466+0000 7fb902ffd640 1 --2- 172.21.15.43:0/1172436515 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fb9040b3050 0x7fb9040c9af0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T15:08:25.470 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:25.466+0000 7fb902ffd640 1 -- 172.21.15.43:0/1172436515 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb8fc009a70 con 0x7fb9040b2790 2023-08-09T15:08:25.470 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:25.466+0000 7fb902ffd640 1 --2- 172.21.15.43:0/1172436515 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb9040b2790 0x7fb9040cba50 secure :-1 s=READY pgs=2118 cs=0 l=1 rev1=1 crypto rx=0x7fb8f400c390 tx=0x7fb8f400c850 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-09T15:08:25.470 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:25.466+0000 7fb900ff9640 1 -- 172.21.15.43:0/1172436515 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb8f4005ce0 con 0x7fb9040b2790 2023-08-09T15:08:25.470 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:25.466+0000 7fb900ff9640 1 -- 172.21.15.43:0/1172436515 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb8f4005e40 con 0x7fb9040b2790 2023-08-09T15:08:25.471 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:25.466+0000 7fb900ff9640 1 -- 172.21.15.43:0/1172436515 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb8f40072f0 con 0x7fb9040b2790 2023-08-09T15:08:25.471 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:25.466+0000 7fb90986a640 1 -- 172.21.15.43:0/1172436515 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb9040ca6e0 con 0x7fb9040b2790 2023-08-09T15:08:25.471 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:25.466+0000 7fb90986a640 1 -- 172.21.15.43:0/1172436515 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb9040c8fe0 con 0x7fb9040b2790 2023-08-09T15:08:25.472 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:25.467+0000 7fb90986a640 1 -- 172.21.15.43:0/1172436515 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb9040739f0 con 0x7fb9040b2790 2023-08-09T15:08:25.472 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:25.469+0000 7fb900ff9640 1 -- 172.21.15.43:0/1172436515 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 102) v1 ==== 88455+0+0 (secure 0 0 0) 0x7fb8f4007450 con 0x7fb9040b2790 2023-08-09T15:08:25.472 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:25.469+0000 7fb900ff9640 1 --2- 172.21.15.43:0/1172436515 >> [v2:172.21.15.203:6832/115830229,v1:172.21.15.203:6833/115830229] conn(0x7fb8e406a170 0x7fb8e406c630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T15:08:25.472 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:25.470+0000 7fb900ff9640 1 -- 172.21.15.43:0/1172436515 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(192..192 src has 1..192) v4 ==== 7186+0+0 (secure 0 0 0) 0x7fb8f4088220 con 0x7fb9040b2790 2023-08-09T15:08:25.472 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:25.470+0000 7fb8fbfff640 1 -- 172.21.15.43:0/1172436515 >> [v2:172.21.15.203:6832/115830229,v1:172.21.15.203:6833/115830229] conn(0x7fb8e406a170 msgr2=0x7fb8e406c630 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.203:6832/115830229 2023-08-09T15:08:25.472 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:25.470+0000 7fb8fbfff640 1 --2- 172.21.15.43:0/1172436515 >> [v2:172.21.15.203:6832/115830229,v1:172.21.15.203:6833/115830229] conn(0x7fb8e406a170 0x7fb8e406c630 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-08-09T15:08:25.475 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:25.474+0000 7fb900ff9640 1 -- 172.21.15.43:0/1172436515 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178066 (secure 0 0 0) 0x7fb8f40532f0 con 0x7fb9040b2790 2023-08-09T15:08:25.514 INFO:tasks.ceph.mgr.x.smithi203.stderr:2023-08-09T15:08:25.513+0000 7fbc59c0f040 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2023-08-09T15:08:25.523 INFO:tasks.ceph.mgr.y.smithi043.stderr:2023-08-09T15:08:25.521+0000 7fe393e53040 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2023-08-09T15:08:25.564 INFO:tasks.ceph.mgr.x.smithi203.stderr:2023-08-09T15:08:25.563+0000 7fbc59c0f040 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2023-08-09T15:08:25.587 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:25.586+0000 7fb900ff9640 1 -- 172.21.15.43:0/1172436515 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mgrmap(e 103) v1 ==== 88052+0+0 (secure 0 0 0) 0x7fb8f40536c0 con 0x7fb9040b2790 2023-08-09T15:08:25.588 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:25.587+0000 7fb900ff9640 1 -- 172.21.15.43:0/1172436515 >> [v2:172.21.15.203:6832/115830229,v1:172.21.15.203:6833/115830229] conn(0x7fb8e406a170 msgr2=0x7fb8e406c630 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-08-09T15:08:25.588 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:25.587+0000 7fb900ff9640 1 --2- 172.21.15.43:0/1172436515 >> [v2:172.21.15.203:6832/115830229,v1:172.21.15.203:6833/115830229] conn(0x7fb8e406a170 0x7fb8e406c630 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T15:08:25.663 INFO:tasks.ceph.mgr.x.smithi203.stderr:2023-08-09T15:08:25.661+0000 7fbbdfc74640 -1 client.0 error registering admin socket command: (17) File exists 2023-08-09T15:08:25.663 INFO:tasks.ceph.mgr.x.smithi203.stderr:2023-08-09T15:08:25.661+0000 7fbbdfc74640 -1 client.0 error registering admin socket command: (17) File exists 2023-08-09T15:08:25.663 INFO:tasks.ceph.mgr.x.smithi203.stderr:2023-08-09T15:08:25.661+0000 7fbbdfc74640 -1 client.0 error registering admin socket command: (17) File exists 2023-08-09T15:08:25.663 INFO:tasks.ceph.mgr.x.smithi203.stderr:2023-08-09T15:08:25.661+0000 7fbbdfc74640 -1 client.0 error registering admin socket command: (17) File exists 2023-08-09T15:08:25.663 INFO:tasks.ceph.mgr.x.smithi203.stderr:2023-08-09T15:08:25.661+0000 7fbbdfc74640 -1 client.0 error registering admin socket command: (17) File exists 2023-08-09T15:08:25.664 INFO:tasks.ceph.mgr.x.smithi203.stderr:2023-08-09T15:08:25.662+0000 7fbbdcc6e640 -1 client.0 error registering admin socket command: (17) File exists 2023-08-09T15:08:25.664 INFO:tasks.ceph.mgr.x.smithi203.stderr:2023-08-09T15:08:25.662+0000 7fbbdcc6e640 -1 client.0 error registering admin socket command: (17) File exists 2023-08-09T15:08:25.664 INFO:tasks.ceph.mgr.x.smithi203.stderr:2023-08-09T15:08:25.662+0000 7fbbdcc6e640 -1 client.0 error registering admin socket command: (17) File exists 2023-08-09T15:08:25.664 INFO:tasks.ceph.mgr.x.smithi203.stderr:2023-08-09T15:08:25.662+0000 7fbbdcc6e640 -1 client.0 error registering admin socket command: (17) File exists 2023-08-09T15:08:25.664 INFO:tasks.ceph.mgr.x.smithi203.stderr:2023-08-09T15:08:25.662+0000 7fbbdcc6e640 -1 client.0 error registering admin socket command: (17) File exists 2023-08-09T15:08:25.707 INFO:tasks.ceph.mgr.y.smithi043.stderr:2023-08-09T15:08:25.706+0000 7fe393e53040 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2023-08-09T15:08:25.720 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:25.717+0000 7fb90986a640 1 -- 172.21.15.43:0/1172436515 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fb9040c5750 con 0x7fb9040b2790 2023-08-09T15:08:25.720 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:25.718+0000 7fb900ff9640 1 -- 172.21.15.43:0/1172436515 <== mon.0 v2:172.21.15.43:3300/0 8 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 87 v87) v1 ==== 94+0+4882 (secure 0 0 0) 0x7fb8f4070d90 con 0x7fb9040b2790 2023-08-09T15:08:25.721 INFO:teuthology.orchestra.run.smithi043.stdout: 2023-08-09T15:08:25.721 INFO:teuthology.orchestra.run.smithi043.stdout:{"epoch":87,"default_fscid":10,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":7328,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.203:6836/3701396136","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6834","nonce":3701396136},{"type":"v1","addr":"172.21.15.203:6836","nonce":3701396136}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":82},{"gid":7343,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.43:6835/2096611841","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6833","nonce":2096611841},{"type":"v1","addr":"172.21.15.43:6835","nonce":2096611841}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":82},{"gid":8242,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.203:6837/18754135","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6835","nonce":18754135},{"type":"v1","addr":"172.21.15.203:6837","nonce":18754135}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":81}],"filesystems":[{"mdsmap":{"epoch":87,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-08-09T14:52:30.188965+0000","modified":"2023-08-09T14:52:33.772528+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":10185},"failed":[],"damaged":[],"stopped":[],"info":{"gid_10185":{"gid":10185,"name":"c","rank":0,"incarnation":83,"state":"up:active","state_seq":3,"addr":"172.21.15.43:6834/2866936881","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6832","nonce":2866936881},{"type":"v1","addr":"172.21.15.43:6834","nonce":2866936881}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[23],"metadata_pool":22,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":10}]} 2023-08-09T15:08:25.721 INFO:teuthology.orchestra.run.smithi043.stderr:dumped fsmap epoch 87 2023-08-09T15:08:25.722 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:25.721+0000 7fb90986a640 1 -- 172.21.15.43:0/1172436515 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb9040b2790 msgr2=0x7fb9040cba50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T15:08:25.722 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:25.721+0000 7fb90986a640 1 --2- 172.21.15.43:0/1172436515 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb9040b2790 0x7fb9040cba50 secure :-1 s=READY pgs=2118 cs=0 l=1 rev1=1 crypto rx=0x7fb8f400c390 tx=0x7fb8f400c850 comp rx=0 tx=0).stop 2023-08-09T15:08:25.723 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:25.721+0000 7fb90986a640 1 -- 172.21.15.43:0/1172436515 shutdown_connections 2023-08-09T15:08:25.723 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:25.721+0000 7fb90986a640 1 --2- 172.21.15.43:0/1172436515 >> [v2:172.21.15.203:6832/115830229,v1:172.21.15.203:6833/115830229] conn(0x7fb8e406a170 0x7fb8e406c630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T15:08:25.723 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:25.721+0000 7fb90986a640 1 --2- 172.21.15.43:0/1172436515 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fb9040b3050 0x7fb9040c9af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T15:08:25.723 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:25.721+0000 7fb90986a640 1 --2- 172.21.15.43:0/1172436515 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fb9040b2b70 0x7fb9040c95b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T15:08:25.724 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:25.721+0000 7fb90986a640 1 --2- 172.21.15.43:0/1172436515 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb9040b2790 0x7fb9040cba50 unknown :-1 s=CLOSED pgs=2118 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T15:08:25.724 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:25.721+0000 7fb90986a640 1 -- 172.21.15.43:0/1172436515 >> 172.21.15.43:0/1172436515 conn(0x7fb90409b7b0 msgr2=0x7fb90406bbd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-09T15:08:25.724 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:25.721+0000 7fb90986a640 1 -- 172.21.15.43:0/1172436515 shutdown_connections 2023-08-09T15:08:25.724 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:25.721+0000 7fb90986a640 1 -- 172.21.15.43:0/1172436515 wait complete. 2023-08-09T15:08:25.740 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-08-09T15:08:25.762 INFO:tasks.ceph.mgr.y.smithi043.stderr:2023-08-09T15:08:25.761+0000 7fe393e53040 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2023-08-09T15:08:25.819 INFO:tasks.ceph.mgr.y.smithi043.stderr:2023-08-09T15:08:25.818+0000 7fe393e53040 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2023-08-09T15:08:25.873 INFO:tasks.ceph.mgr.y.smithi043.stderr:2023-08-09T15:08:25.872+0000 7fe393e53040 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2023-08-09T15:08:25.959 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:25.958+0000 7fa6b759e640 1 Processor -- start 2023-08-09T15:08:25.960 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:25.958+0000 7fa6b759e640 1 -- start start 2023-08-09T15:08:25.961 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:25.960+0000 7fa6b759e640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7fa6b80aef20 0x7fa6b80afff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T15:08:25.962 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:25.960+0000 7fa6b759e640 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7fa6b80af300 0x7fa6b80cd430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T15:08:25.962 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:25.960+0000 7fa6b759e640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fa6b8160110 0x7fa6b81605c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T15:08:25.962 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:25.960+0000 7fa6b759e640 1 -- --> v2:172.21.15.43:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa6b8160b00 con 0x7fa6b80aef20 2023-08-09T15:08:25.962 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:25.960+0000 7fa6b759e640 1 -- --> v2:172.21.15.203:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa6b80cd970 con 0x7fa6b80af300 2023-08-09T15:08:25.962 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:25.960+0000 7fa6b759e640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa6b80b0600 con 0x7fa6b8160110 2023-08-09T15:08:25.962 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:25.960+0000 7fa6b659c640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7fa6b80aef20 0x7fa6b80afff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T15:08:25.962 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:25.960+0000 7fa6b659c640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7fa6b80aef20 0x7fa6b80afff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:35938/0 (socket says 172.21.15.43:35938) 2023-08-09T15:08:25.963 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:25.960+0000 7fa6b659c640 1 -- 172.21.15.43:0/3222492142 learned_addr learned my addr 172.21.15.43:0/3222492142 (peer_addr_for_me v2:172.21.15.43:0/0) 2023-08-09T15:08:25.963 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:25.961+0000 7fa6affff640 1 --2- 172.21.15.43:0/3222492142 >> v2:172.21.15.203:3300/0 conn(0x7fa6b80af300 0x7fa6b80cd430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T15:08:25.963 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:25.961+0000 7fa6b659c640 1 -- 172.21.15.43:0/3222492142 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fa6b8160110 msgr2=0x7fa6b81605c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-09T15:08:25.963 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:25.961+0000 7fa6b6d9d640 1 --2- 172.21.15.43:0/3222492142 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fa6b8160110 0x7fa6b81605c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T15:08:25.964 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:25.961+0000 7fa6b659c640 1 --2- 172.21.15.43:0/3222492142 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fa6b8160110 0x7fa6b81605c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T15:08:25.964 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:25.961+0000 7fa6b659c640 1 -- 172.21.15.43:0/3222492142 >> v2:172.21.15.203:3300/0 conn(0x7fa6b80af300 msgr2=0x7fa6b80cd430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-09T15:08:25.964 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:25.961+0000 7fa6b659c640 1 --2- 172.21.15.43:0/3222492142 >> v2:172.21.15.203:3300/0 conn(0x7fa6b80af300 0x7fa6b80cd430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T15:08:25.964 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:25.961+0000 7fa6b659c640 1 -- 172.21.15.43:0/3222492142 --> v2:172.21.15.43:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa6b80b08e0 con 0x7fa6b80aef20 2023-08-09T15:08:25.964 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:25.962+0000 7fa6b659c640 1 --2- 172.21.15.43:0/3222492142 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa6b80aef20 0x7fa6b80afff0 secure :-1 s=READY pgs=2119 cs=0 l=1 rev1=1 crypto rx=0x7fa6b80cdbb0 tx=0x7fa6a802b920 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b7a430de1a7a83bb server_cookie=0 in_seq=0 out_seq=0 2023-08-09T15:08:25.965 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:25.962+0000 7fa6b5d9b640 1 -- 172.21.15.43:0/3222492142 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa6a8031880 con 0x7fa6b80aef20 2023-08-09T15:08:25.966 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:25.962+0000 7fa6b5d9b640 1 -- 172.21.15.43:0/3222492142 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa6a80319e0 con 0x7fa6b80aef20 2023-08-09T15:08:25.966 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:25.962+0000 7fa6b5d9b640 1 -- 172.21.15.43:0/3222492142 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa6a8031d70 con 0x7fa6b80aef20 2023-08-09T15:08:25.966 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:25.963+0000 7fa6b759e640 1 -- 172.21.15.43:0/3222492142 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa6b80aef20 msgr2=0x7fa6b80afff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T15:08:25.966 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:25.963+0000 7fa6b759e640 1 --2- 172.21.15.43:0/3222492142 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa6b80aef20 0x7fa6b80afff0 secure :-1 s=READY pgs=2119 cs=0 l=1 rev1=1 crypto rx=0x7fa6b80cdbb0 tx=0x7fa6a802b920 comp rx=0 tx=0).stop 2023-08-09T15:08:25.966 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:25.963+0000 7fa6b759e640 1 -- 172.21.15.43:0/3222492142 shutdown_connections 2023-08-09T15:08:25.966 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:25.963+0000 7fa6b759e640 1 --2- 172.21.15.43:0/3222492142 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fa6b8160110 0x7fa6b81605c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T15:08:25.967 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:25.963+0000 7fa6b759e640 1 --2- 172.21.15.43:0/3222492142 >> v2:172.21.15.203:3300/0 conn(0x7fa6b80af300 0x7fa6b80cd430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T15:08:25.967 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:25.963+0000 7fa6b759e640 1 --2- 172.21.15.43:0/3222492142 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa6b80aef20 0x7fa6b80afff0 unknown :-1 s=CLOSED pgs=2119 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T15:08:25.967 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:25.963+0000 7fa6b759e640 1 -- 172.21.15.43:0/3222492142 >> 172.21.15.43:0/3222492142 conn(0x7fa6b8089fe0 msgr2=0x7fa6b8099470 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-09T15:08:25.967 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:25.964+0000 7fa6b759e640 1 -- 172.21.15.43:0/3222492142 shutdown_connections 2023-08-09T15:08:25.967 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:25.964+0000 7fa6b759e640 1 -- 172.21.15.43:0/3222492142 wait complete. 2023-08-09T15:08:25.969 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:25.965+0000 7fa6b759e640 1 Processor -- start 2023-08-09T15:08:25.969 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:25.965+0000 7fa6b759e640 1 -- start start 2023-08-09T15:08:25.969 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:25.966+0000 7fa6b759e640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fa6b80aef20 0x7fa6b80c4a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T15:08:25.969 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:25.966+0000 7fa6b759e640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa6b80af300 0x7fa6b80c6000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T15:08:25.969 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:25.966+0000 7fa6b759e640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fa6b8160110 0x7fa6b815aa20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T15:08:25.970 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:25.966+0000 7fa6b759e640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa6b80c6860 con 0x7fa6b80af300 2023-08-09T15:08:25.970 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:25.966+0000 7fa6b759e640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa6b80c3c40 con 0x7fa6b80aef20 2023-08-09T15:08:25.970 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:25.966+0000 7fa6b759e640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa6b80c3f20 con 0x7fa6b8160110 2023-08-09T15:08:25.970 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:25.967+0000 7fa6affff640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa6b80af300 0x7fa6b80c6000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T15:08:25.970 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:25.967+0000 7fa6affff640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa6b80af300 0x7fa6b80c6000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:35940/0 (socket says 172.21.15.43:35940) 2023-08-09T15:08:25.970 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:25.967+0000 7fa6affff640 1 -- 172.21.15.43:0/1917258226 learned_addr learned my addr 172.21.15.43:0/1917258226 (peer_addr_for_me v2:172.21.15.43:0/0) 2023-08-09T15:08:25.970 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:25.967+0000 7fa6b6d9d640 1 --2- 172.21.15.43:0/1917258226 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fa6b8160110 0x7fa6b815aa20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T15:08:25.970 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:25.967+0000 7fa6b659c640 1 --2- 172.21.15.43:0/1917258226 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fa6b80aef20 0x7fa6b80c4a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T15:08:25.971 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:25.967+0000 7fa6affff640 1 -- 172.21.15.43:0/1917258226 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fa6b8160110 msgr2=0x7fa6b815aa20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T15:08:25.971 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:25.967+0000 7fa6affff640 1 --2- 172.21.15.43:0/1917258226 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fa6b8160110 0x7fa6b815aa20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T15:08:25.971 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:25.967+0000 7fa6affff640 1 -- 172.21.15.43:0/1917258226 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fa6b80aef20 msgr2=0x7fa6b80c4a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T15:08:25.971 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:25.967+0000 7fa6affff640 1 --2- 172.21.15.43:0/1917258226 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fa6b80aef20 0x7fa6b80c4a00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T15:08:25.971 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:25.967+0000 7fa6affff640 1 -- 172.21.15.43:0/1917258226 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa6a80094d0 con 0x7fa6b80af300 2023-08-09T15:08:25.971 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:25.967+0000 7fa6affff640 1 --2- 172.21.15.43:0/1917258226 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa6b80af300 0x7fa6b80c6000 secure :-1 s=READY pgs=2120 cs=0 l=1 rev1=1 crypto rx=0x7fa6a000c570 tx=0x7fa6a000ca30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-09T15:08:25.971 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:25.968+0000 7fa6aeffd640 1 -- 172.21.15.43:0/1917258226 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa6a0004220 con 0x7fa6b80af300 2023-08-09T15:08:25.972 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:25.968+0000 7fa6aeffd640 1 -- 172.21.15.43:0/1917258226 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa6a0004380 con 0x7fa6b80af300 2023-08-09T15:08:25.972 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:25.968+0000 7fa6aeffd640 1 -- 172.21.15.43:0/1917258226 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa6a0003850 con 0x7fa6b80af300 2023-08-09T15:08:25.972 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:25.968+0000 7fa6b759e640 1 -- 172.21.15.43:0/1917258226 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa6b80c4200 con 0x7fa6b80af300 2023-08-09T15:08:25.972 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:25.968+0000 7fa6b759e640 1 -- 172.21.15.43:0/1917258226 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa6b80c6540 con 0x7fa6b80af300 2023-08-09T15:08:25.972 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:25.969+0000 7fa6b759e640 1 -- 172.21.15.43:0/1917258226 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa6b80739f0 con 0x7fa6b80af300 2023-08-09T15:08:25.973 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:25.970+0000 7fa6aeffd640 1 -- 172.21.15.43:0/1917258226 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 103) v1 ==== 88052+0+0 (secure 0 0 0) 0x7fa6a00039b0 con 0x7fa6b80af300 2023-08-09T15:08:25.973 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:25.971+0000 7fa6aeffd640 1 -- 172.21.15.43:0/1917258226 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(193..193 src has 1..193) v4 ==== 7486+0+0 (secure 0 0 0) 0x7fa6a0088360 con 0x7fa6b80af300 2023-08-09T15:08:25.976 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:25.975+0000 7fa6aeffd640 1 -- 172.21.15.43:0/1917258226 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178066 (secure 0 0 0) 0x7fa6a0053820 con 0x7fa6b80af300 2023-08-09T15:08:26.174 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:26.173+0000 7fa6b759e640 1 -- 172.21.15.43:0/1917258226 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fa6b815bc10 con 0x7fa6b80af300 2023-08-09T15:08:26.176 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:26.175+0000 7fa6aeffd640 1 -- 172.21.15.43:0/1917258226 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v193) v1 ==== 74+0+20146 (secure 0 0 0) 0x7fa6a0056e40 con 0x7fa6b80af300 2023-08-09T15:08:26.177 INFO:teuthology.orchestra.run.smithi043.stdout: 2023-08-09T15:08:26.177 INFO:teuthology.orchestra.run.smithi043.stdout:{"epoch":193,"fsid":"daac87ee-1337-4250-a955-aea4e408a39b","created":"2023-08-09T13:33:41.701198+0000","modified":"2023-08-09T15:08:25.567949+0000","last_up_change":"2023-08-09T13:33:53.783453+0000","last_in_change":"2023-08-09T13:33:45.148904+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":23,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":"rbd","create_time":"2023-08-09T13:33:54.365137+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":".mgr","create_time":"2023-08-09T13:33:54.831803+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":22,"pool_name":"cephfs_metadata","create_time":"2023-08-09T14:52:28.569334+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"185","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":23,"pool_name":"cephfs_data","create_time":"2023-08-09T14:52:29.209973+0000","flags":8193,"flags_names":"hashpspool,selfmanaged_snaps","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"192","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":33,"snap_epoch":192,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"cbd49ecd-e896-4fbd-864e-5c37a829b165","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":184,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6800","nonce":502667335},{"type":"v1","addr":"172.21.15.43:6801","nonce":502667335}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6802","nonce":502667335},{"type":"v1","addr":"172.21.15.43:6803","nonce":502667335}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6806","nonce":502667335},{"type":"v1","addr":"172.21.15.43:6807","nonce":502667335}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6804","nonce":502667335},{"type":"v1","addr":"172.21.15.43:6805","nonce":502667335}]},"public_addr":"172.21.15.43:6801/502667335","cluster_addr":"172.21.15.43:6803/502667335","heartbeat_back_addr":"172.21.15.43:6807/502667335","heartbeat_front_addr":"172.21.15.43:6805/502667335","state":["exists","up"]},{"osd":1,"uuid":"75c7e0fe-1ec7-4fe8-a2f2-3661a65b8500","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":184,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6808","nonce":3289134610},{"type":"v1","addr":"172.21.15.43:6809","nonce":3289134610}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6810","nonce":3289134610},{"type":"v1","addr":"172.21.15.43:6812","nonce":3289134610}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6821","nonce":3289134610},{"type":"v1","addr":"172.21.15.43:6824","nonce":3289134610}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6815","nonce":3289134610},{"type":"v1","addr":"172.21.15.43:6818","nonce":3289134610}]},"public_addr":"172.21.15.43:6809/3289134610","cluster_addr":"172.21.15.43:6812/3289134610","heartbeat_back_addr":"172.21.15.43:6824/3289134610","heartbeat_front_addr":"172.21.15.43:6818/3289134610","state":["exists","up"]},{"osd":2,"uuid":"bb5c41e9-7f5f-4125-9b01-3bb8bfc2ea83","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":184,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6811","nonce":4079709736},{"type":"v1","addr":"172.21.15.43:6814","nonce":4079709736}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6817","nonce":4079709736},{"type":"v1","addr":"172.21.15.43:6820","nonce":4079709736}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6829","nonce":4079709736},{"type":"v1","addr":"172.21.15.43:6831","nonce":4079709736}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6823","nonce":4079709736},{"type":"v1","addr":"172.21.15.43:6826","nonce":4079709736}]},"public_addr":"172.21.15.43:6814/4079709736","cluster_addr":"172.21.15.43:6820/4079709736","heartbeat_back_addr":"172.21.15.43:6831/4079709736","heartbeat_front_addr":"172.21.15.43:6826/4079709736","state":["exists","up"]},{"osd":3,"uuid":"bfe4c8b4-694a-465c-87e6-2b2aab852613","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":184,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6813","nonce":1709538759},{"type":"v1","addr":"172.21.15.43:6816","nonce":1709538759}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6819","nonce":1709538759},{"type":"v1","addr":"172.21.15.43:6822","nonce":1709538759}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6828","nonce":1709538759},{"type":"v1","addr":"172.21.15.43:6830","nonce":1709538759}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6825","nonce":1709538759},{"type":"v1","addr":"172.21.15.43:6827","nonce":1709538759}]},"public_addr":"172.21.15.43:6816/1709538759","cluster_addr":"172.21.15.43:6822/1709538759","heartbeat_back_addr":"172.21.15.43:6830/1709538759","heartbeat_front_addr":"172.21.15.43:6827/1709538759","state":["exists","up"]},{"osd":4,"uuid":"fa2e8a77-aba1-4756-853e-0d174a5bdeba","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":184,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6800","nonce":4196866481},{"type":"v1","addr":"172.21.15.203:6801","nonce":4196866481}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6802","nonce":4196866481},{"type":"v1","addr":"172.21.15.203:6803","nonce":4196866481}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6806","nonce":4196866481},{"type":"v1","addr":"172.21.15.203:6807","nonce":4196866481}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6804","nonce":4196866481},{"type":"v1","addr":"172.21.15.203:6805","nonce":4196866481}]},"public_addr":"172.21.15.203:6801/4196866481","cluster_addr":"172.21.15.203:6803/4196866481","heartbeat_back_addr":"172.21.15.203:6807/4196866481","heartbeat_front_addr":"172.21.15.203:6805/4196866481","state":["exists","up"]},{"osd":5,"uuid":"81811e4b-4d24-4f51-a82c-f1d700a2a00d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":184,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6816","nonce":3059290081},{"type":"v1","addr":"172.21.15.203:6817","nonce":3059290081}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6818","nonce":3059290081},{"type":"v1","addr":"172.21.15.203:6819","nonce":3059290081}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6822","nonce":3059290081},{"type":"v1","addr":"172.21.15.203:6823","nonce":3059290081}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6820","nonce":3059290081},{"type":"v1","addr":"172.21.15.203:6821","nonce":3059290081}]},"public_addr":"172.21.15.203:6817/3059290081","cluster_addr":"172.21.15.203:6819/3059290081","heartbeat_back_addr":"172.21.15.203:6823/3059290081","heartbeat_front_addr":"172.21.15.203:6821/3059290081","state":["exists","up"]},{"osd":6,"uuid":"307bac4d-6a5b-443d-af75-56d695d7674f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":184,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6824","nonce":512155569},{"type":"v1","addr":"172.21.15.203:6825","nonce":512155569}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6826","nonce":512155569},{"type":"v1","addr":"172.21.15.203:6827","nonce":512155569}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6830","nonce":512155569},{"type":"v1","addr":"172.21.15.203:6831","nonce":512155569}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6828","nonce":512155569},{"type":"v1","addr":"172.21.15.203:6829","nonce":512155569}]},"public_addr":"172.21.15.203:6825/512155569","cluster_addr":"172.21.15.203:6827/512155569","heartbeat_back_addr":"172.21.15.203:6831/512155569","heartbeat_front_addr":"172.21.15.203:6829/512155569","state":["exists","up"]},{"osd":7,"uuid":"a95dc7ca-9f1b-4295-af70-68a22a2d7d5e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":184,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6808","nonce":3331450438},{"type":"v1","addr":"172.21.15.203:6809","nonce":3331450438}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6810","nonce":3331450438},{"type":"v1","addr":"172.21.15.203:6811","nonce":3331450438}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6814","nonce":3331450438},{"type":"v1","addr":"172.21.15.203:6815","nonce":3331450438}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6812","nonce":3331450438},{"type":"v1","addr":"172.21.15.203:6813","nonce":3331450438}]},"public_addr":"172.21.15.203:6809/3331450438","cluster_addr":"172.21.15.203:6811/3331450438","heartbeat_back_addr":"172.21.15.203:6815/3331450438","heartbeat_front_addr":"172.21.15.203:6813/3331450438","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-09T13:33:50.547652+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-09T13:33:50.589469+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-09T13:33:50.561724+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-09T13:33:50.599650+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-09T13:33:50.411855+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-09T13:33:50.219681+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-09T13:33:50.504664+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-09T13:33:50.441248+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.203:0/1820136184":"2023-08-10T15:08:25.567876+0000","172.21.15.203:6833/115830229":"2023-08-10T15:08:25.567876+0000","172.21.15.203:6832/115830229":"2023-08-10T15:08:25.567876+0000","172.21.15.203:0/3317788314":"2023-08-10T15:08:25.567876+0000","172.21.15.203:0/69497804":"2023-08-10T15:03:21.353235+0000","172.21.15.203:0/169967264":"2023-08-10T15:03:21.353235+0000","172.21.15.203:6833/40883597":"2023-08-10T15:03:21.353235+0000","172.21.15.203:0/1738813745":"2023-08-10T15:08:25.567876+0000","172.21.15.203:0/1306452763":"2023-08-10T15:08:25.567876+0000","172.21.15.203:6832/40883597":"2023-08-10T15:03:21.353235+0000","172.21.15.203:0/1225284485":"2023-08-10T15:08:25.567876+0000","172.21.15.203:0/3690197627":"2023-08-10T15:03:21.353235+0000","172.21.15.203:0/1082753911":"2023-08-10T15:03:21.353235+0000","172.21.15.203:6832/2959400181":"2023-08-10T14:52:45.188641+0000","172.21.15.203:0/82655847":"2023-08-10T15:03:21.353235+0000","172.21.15.203:0/1935803093":"2023-08-10T14:52:45.188641+0000","172.21.15.203:0/3150685329":"2023-08-10T14:52:45.188641+0000","172.21.15.203:0/2219953609":"2023-08-10T15:03:21.353235+0000","172.21.15.203:0/3429376127":"2023-08-10T14:52:45.188641+0000","172.21.15.203:0/1394945671":"2023-08-10T15:08:25.567876+0000","172.21.15.203:0/2197573025":"2023-08-10T14:52:45.188641+0000","172.21.15.203:0/2458603841":"2023-08-10T14:52:45.188641+0000","172.21.15.203:6833/2959400181":"2023-08-10T14:52:45.188641+0000","172.21.15.203:0/4177046277":"2023-08-10T14:52:45.188641+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":7,"snaps":[{"begin":104,"length":1},{"begin":221,"length":1}]},{"pool":23,"snaps":[{"begin":2,"length":32}]}],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-08-09T15:08:26.179 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:26.177+0000 7fa6b759e640 1 -- 172.21.15.43:0/1917258226 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa6b80af300 msgr2=0x7fa6b80c6000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T15:08:26.179 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:26.177+0000 7fa6b759e640 1 --2- 172.21.15.43:0/1917258226 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa6b80af300 0x7fa6b80c6000 secure :-1 s=READY pgs=2120 cs=0 l=1 rev1=1 crypto rx=0x7fa6a000c570 tx=0x7fa6a000ca30 comp rx=0 tx=0).stop 2023-08-09T15:08:26.179 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:26.178+0000 7fa6b759e640 1 -- 172.21.15.43:0/1917258226 shutdown_connections 2023-08-09T15:08:26.179 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:26.178+0000 7fa6b759e640 1 --2- 172.21.15.43:0/1917258226 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fa6b8160110 0x7fa6b815aa20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T15:08:26.179 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:26.178+0000 7fa6b759e640 1 --2- 172.21.15.43:0/1917258226 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa6b80af300 0x7fa6b80c6000 unknown :-1 s=CLOSED pgs=2120 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T15:08:26.180 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:26.178+0000 7fa6b759e640 1 --2- 172.21.15.43:0/1917258226 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fa6b80aef20 0x7fa6b80c4a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T15:08:26.180 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:26.178+0000 7fa6b759e640 1 -- 172.21.15.43:0/1917258226 >> 172.21.15.43:0/1917258226 conn(0x7fa6b8089fe0 msgr2=0x7fa6b806d6e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-09T15:08:26.180 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:26.178+0000 7fa6b759e640 1 -- 172.21.15.43:0/1917258226 shutdown_connections 2023-08-09T15:08:26.180 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:26.178+0000 7fa6b759e640 1 -- 172.21.15.43:0/1917258226 wait complete. 2023-08-09T15:08:26.197 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': 'rbd', 'create_time': '2023-08-09T13:33:54.365137+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}, 'read_balance': {'score_acting': 2, 'score_stable': 2, 'optimal_score': 1, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': '.mgr', 'create_time': '2023-08-09T13:33:54.831803+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 22, 'pool_name': 'cephfs_metadata', 'create_time': '2023-08-09T14:52:28.569334+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '185', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 2, 'score_stable': 2, 'optimal_score': 1, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 23, 'pool_name': 'cephfs_data', 'create_time': '2023-08-09T14:52:29.209973+0000', 'flags': 8193, 'flags_names': 'hashpspool,selfmanaged_snaps', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '192', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 33, 'snap_epoch': 192, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.3799999952316284, 'score_stable': 1.3799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.3799999952316284, 'raw_score_stable': 1.3799999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-08-09T15:08:26.198 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get rbd pg_num 2023-08-09T15:08:26.477 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:26.475+0000 7f543dd22640 1 Processor -- start 2023-08-09T15:08:26.477 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:26.476+0000 7f543dd22640 1 -- start start 2023-08-09T15:08:26.479 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:26.477+0000 7f543dd22640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f54380706e0 0x7f5438159740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T15:08:26.479 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:26.477+0000 7f543dd22640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f54380b1540 0x7f54380b2530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T15:08:26.479 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:26.477+0000 7f543dd22640 1 -- --> v1:172.21.15.43:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f54380ae9a0 con 0x7f54380b1160 2023-08-09T15:08:26.479 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:26.477+0000 7f543dd22640 1 -- --> v2:172.21.15.43:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5438159c80 con 0x7f54380706e0 2023-08-09T15:08:26.479 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:26.477+0000 7f543dd22640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f543815a730 con 0x7f54380b1540 2023-08-09T15:08:26.480 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:26.478+0000 7f5437fff640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f54380706e0 0x7f5438159740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T15:08:26.480 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:26.478+0000 7f543cd20640 1 --1- >> v1:172.21.15.43:6789/0 conn(0x7f54380b1160 0x7f543815a900 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.43:6789/0 says I am v1:172.21.15.43:41700/0 (socket says 172.21.15.43:41700) 2023-08-09T15:08:26.481 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:26.478+0000 7f543cd20640 1 -- 172.21.15.43:0/1288201965 learned_addr learned my addr 172.21.15.43:0/1288201965 (peer_addr_for_me v1:172.21.15.43:0/0) 2023-08-09T15:08:26.481 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:26.478+0000 7f543d521640 1 --2- 172.21.15.43:0/1288201965 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f54380b1540 0x7f54380b2530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T15:08:26.481 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:26.479+0000 7f5437fff640 1 -- 172.21.15.43:0/1288201965 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f54380b1540 msgr2=0x7f54380b2530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-09T15:08:26.481 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:26.479+0000 7f5437fff640 1 --2- 172.21.15.43:0/1288201965 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f54380b1540 0x7f54380b2530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T15:08:26.481 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:26.479+0000 7f5437fff640 1 -- 172.21.15.43:0/1288201965 >> v1:172.21.15.43:6789/0 conn(0x7f54380b1160 legacy=0x7f543815a900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T15:08:26.481 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:26.479+0000 7f5437fff640 1 -- 172.21.15.43:0/1288201965 --> v2:172.21.15.43:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5438159e90 con 0x7f54380706e0 2023-08-09T15:08:26.481 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:26.479+0000 7f54377fe640 1 -- 172.21.15.43:0/1288201965 <== mon.0 v1:172.21.15.43:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3439287573 0 0) 0x7f54380ae9a0 con 0x7f54380b1160 2023-08-09T15:08:26.482 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:26.480+0000 7f5437fff640 1 --2- 172.21.15.43:0/1288201965 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f54380706e0 0x7f5438159740 secure :-1 s=READY pgs=2122 cs=0 l=1 rev1=1 crypto rx=0x7f54240097a0 tx=0x7f542402b840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=782b652bf152863a server_cookie=0 in_seq=0 out_seq=0 2023-08-09T15:08:26.483 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:26.480+0000 7f54377fe640 1 -- 172.21.15.43:0/1288201965 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f542402e070 con 0x7f54380706e0 2023-08-09T15:08:26.483 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:26.480+0000 7f54377fe640 1 -- 172.21.15.43:0/1288201965 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f54240315a0 con 0x7f54380706e0 2023-08-09T15:08:26.483 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:26.480+0000 7f54377fe640 1 -- 172.21.15.43:0/1288201965 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5424031930 con 0x7f54380706e0 2023-08-09T15:08:26.483 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:26.481+0000 7f543dd22640 1 -- 172.21.15.43:0/1288201965 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f54380706e0 msgr2=0x7f5438159740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T15:08:26.483 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:26.481+0000 7f543dd22640 1 --2- 172.21.15.43:0/1288201965 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f54380706e0 0x7f5438159740 secure :-1 s=READY pgs=2122 cs=0 l=1 rev1=1 crypto rx=0x7f54240097a0 tx=0x7f542402b840 comp rx=0 tx=0).stop 2023-08-09T15:08:26.483 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:26.481+0000 7f543dd22640 1 -- 172.21.15.43:0/1288201965 shutdown_connections 2023-08-09T15:08:26.483 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:26.481+0000 7f543dd22640 1 --2- 172.21.15.43:0/1288201965 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f54380b1540 0x7f54380b2530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T15:08:26.483 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:26.481+0000 7f543dd22640 1 --2- 172.21.15.43:0/1288201965 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f54380706e0 0x7f5438159740 unknown :-1 s=CLOSED pgs=2122 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T15:08:26.484 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:26.481+0000 7f543dd22640 1 -- 172.21.15.43:0/1288201965 >> 172.21.15.43:0/1288201965 conn(0x7f54380b4400 msgr2=0x7f54380b47e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-09T15:08:26.484 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:26.481+0000 7f543dd22640 1 -- 172.21.15.43:0/1288201965 shutdown_connections 2023-08-09T15:08:26.484 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:26.481+0000 7f543dd22640 1 -- 172.21.15.43:0/1288201965 wait complete. 2023-08-09T15:08:26.484 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:26.483+0000 7f543dd22640 1 Processor -- start 2023-08-09T15:08:26.485 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:26.483+0000 7f543dd22640 1 -- start start 2023-08-09T15:08:26.486 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:26.484+0000 7f543dd22640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f54380706e0 0x7f5438156730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T15:08:26.486 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:26.484+0000 7f543dd22640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f54380b1160 0x7f5438154a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T15:08:26.486 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:26.484+0000 7f543dd22640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f54380b1540 0x7f5438148520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T15:08:26.486 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:26.484+0000 7f543dd22640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5438143ef0 con 0x7f54380b1540 2023-08-09T15:08:26.486 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:26.484+0000 7f543dd22640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f54381441c0 con 0x7f54380b1160 2023-08-09T15:08:26.487 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:26.484+0000 7f543dd22640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f54381444a0 con 0x7f54380706e0 2023-08-09T15:08:26.487 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:26.484+0000 7f543d521640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f54380b1540 0x7f5438148520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T15:08:26.487 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:26.484+0000 7f543d521640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f54380b1540 0x7f5438148520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:35952/0 (socket says 172.21.15.43:35952) 2023-08-09T15:08:26.487 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:26.484+0000 7f543d521640 1 -- 172.21.15.43:0/2268695005 learned_addr learned my addr 172.21.15.43:0/2268695005 (peer_addr_for_me v2:172.21.15.43:0/0) 2023-08-09T15:08:26.487 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:26.485+0000 7f5437fff640 1 --2- 172.21.15.43:0/2268695005 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f54380b1160 0x7f5438154a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T15:08:26.487 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:26.485+0000 7f543d521640 1 -- 172.21.15.43:0/2268695005 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f54380706e0 msgr2=0x7f5438156730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T15:08:26.487 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:26.485+0000 7f543d521640 1 --2- 172.21.15.43:0/2268695005 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f54380706e0 0x7f5438156730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T15:08:26.488 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:26.485+0000 7f543d521640 1 -- 172.21.15.43:0/2268695005 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f54380b1160 msgr2=0x7f5438154a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T15:08:26.488 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:26.485+0000 7f543d521640 1 --2- 172.21.15.43:0/2268695005 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f54380b1160 0x7f5438154a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T15:08:26.488 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:26.485+0000 7f543d521640 1 -- 172.21.15.43:0/2268695005 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5424009400 con 0x7f54380b1540 2023-08-09T15:08:26.488 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:26.485+0000 7f5437fff640 1 --2- 172.21.15.43:0/2268695005 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f54380b1160 0x7f5438154a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-08-09T15:08:26.489 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:26.485+0000 7f543d521640 1 --2- 172.21.15.43:0/2268695005 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f54380b1540 0x7f5438148520 secure :-1 s=READY pgs=2123 cs=0 l=1 rev1=1 crypto rx=0x7f5430007e40 tx=0x7f543000e4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-09T15:08:26.489 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:26.485+0000 7f54357fa640 1 -- 172.21.15.43:0/2268695005 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5430014620 con 0x7f54380b1540 2023-08-09T15:08:26.489 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:26.485+0000 7f54357fa640 1 -- 172.21.15.43:0/2268695005 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f543000ee90 con 0x7f54380b1540 2023-08-09T15:08:26.489 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:26.485+0000 7f54357fa640 1 -- 172.21.15.43:0/2268695005 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f54300149e0 con 0x7f54380b1540 2023-08-09T15:08:26.490 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:26.485+0000 7f543dd22640 1 -- 172.21.15.43:0/2268695005 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5438144780 con 0x7f54380b1540 2023-08-09T15:08:26.490 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:26.486+0000 7f543dd22640 1 -- 172.21.15.43:0/2268695005 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5438144d40 con 0x7f54380b1540 2023-08-09T15:08:26.490 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:26.487+0000 7f543dd22640 1 -- 172.21.15.43:0/2268695005 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5438144f60 con 0x7f54380b1540 2023-08-09T15:08:26.490 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:26.488+0000 7f54357fa640 1 -- 172.21.15.43:0/2268695005 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 103) v1 ==== 88052+0+0 (secure 0 0 0) 0x7f5430014b40 con 0x7f54380b1540 2023-08-09T15:08:26.490 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:26.489+0000 7f54357fa640 1 -- 172.21.15.43:0/2268695005 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(193..193 src has 1..193) v4 ==== 7486+0+0 (secure 0 0 0) 0x7f5430089080 con 0x7f54380b1540 2023-08-09T15:08:26.494 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:26.493+0000 7f54357fa640 1 -- 172.21.15.43:0/2268695005 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178066 (secure 0 0 0) 0x7f54300bb7b0 con 0x7f54380b1540 2023-08-09T15:08:26.595 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:26.594+0000 7f54357fa640 1 -- 172.21.15.43:0/2268695005 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mgrmap(e 104) v1 ==== 88179+0+0 (secure 0 0 0) 0x7f5430053ff0 con 0x7f54380b1540 2023-08-09T15:08:26.596 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:26.594+0000 7f54357fa640 1 --2- 172.21.15.43:0/2268695005 >> [v2:172.21.15.203:6832/3082732769,v1:172.21.15.203:6833/3082732769] conn(0x7f541c070710 0x7f541c072b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T15:08:26.596 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:26.595+0000 7f543cd20640 1 --2- 172.21.15.43:0/2268695005 >> [v2:172.21.15.203:6832/3082732769,v1:172.21.15.203:6833/3082732769] conn(0x7f541c070710 0x7f541c072b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T15:08:26.598 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:26.597+0000 7f543cd20640 1 --2- 172.21.15.43:0/2268695005 >> [v2:172.21.15.203:6832/3082732769,v1:172.21.15.203:6833/3082732769] conn(0x7f541c070710 0x7f541c072b00 secure :-1 s=READY pgs=3 cs=0 l=1 rev1=1 crypto rx=0x7f542c007fd0 tx=0x7f542c007920 comp rx=0 tx=0).ready entity=mgr.8500 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-09T15:08:26.686 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:26.685+0000 7f543dd22640 1 -- 172.21.15.43:0/2268695005 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"} v 0) v1 -- 0x7f543815a420 con 0x7f54380b1540 2023-08-09T15:08:26.687 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:26.686+0000 7f54357fa640 1 -- 172.21.15.43:0/2268695005 <== mon.0 v2:172.21.15.43:3300/0 8 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"}]=0 v193) v1 ==== 92+0+10 (secure 0 0 0) 0x7f543006f760 con 0x7f54380b1540 2023-08-09T15:08:26.687 INFO:teuthology.orchestra.run.smithi043.stdout:pg_num: 8 2023-08-09T15:08:26.691 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:26.688+0000 7f543dd22640 1 -- 172.21.15.43:0/2268695005 >> [v2:172.21.15.203:6832/3082732769,v1:172.21.15.203:6833/3082732769] conn(0x7f541c070710 msgr2=0x7f541c072b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T15:08:26.691 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:26.688+0000 7f543dd22640 1 --2- 172.21.15.43:0/2268695005 >> [v2:172.21.15.203:6832/3082732769,v1:172.21.15.203:6833/3082732769] conn(0x7f541c070710 0x7f541c072b00 secure :-1 s=READY pgs=3 cs=0 l=1 rev1=1 crypto rx=0x7f542c007fd0 tx=0x7f542c007920 comp rx=0 tx=0).stop 2023-08-09T15:08:26.692 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:26.688+0000 7f543dd22640 1 -- 172.21.15.43:0/2268695005 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f54380b1540 msgr2=0x7f5438148520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T15:08:26.692 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:26.688+0000 7f543dd22640 1 --2- 172.21.15.43:0/2268695005 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f54380b1540 0x7f5438148520 secure :-1 s=READY pgs=2123 cs=0 l=1 rev1=1 crypto rx=0x7f5430007e40 tx=0x7f543000e4b0 comp rx=0 tx=0).stop 2023-08-09T15:08:26.692 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:26.688+0000 7f543dd22640 1 -- 172.21.15.43:0/2268695005 shutdown_connections 2023-08-09T15:08:26.692 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:26.689+0000 7f543dd22640 1 --2- 172.21.15.43:0/2268695005 >> [v2:172.21.15.203:6832/3082732769,v1:172.21.15.203:6833/3082732769] conn(0x7f541c070710 0x7f541c072b00 unknown :-1 s=CLOSED pgs=3 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T15:08:26.693 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:26.689+0000 7f543dd22640 1 --2- 172.21.15.43:0/2268695005 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f54380b1540 0x7f5438148520 unknown :-1 s=CLOSED pgs=2123 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T15:08:26.693 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:26.689+0000 7f543dd22640 1 --2- 172.21.15.43:0/2268695005 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f54380b1160 0x7f5438154a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T15:08:26.693 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:26.689+0000 7f543dd22640 1 --2- 172.21.15.43:0/2268695005 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f54380706e0 0x7f5438156730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T15:08:26.694 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:26.689+0000 7f543dd22640 1 -- 172.21.15.43:0/2268695005 >> 172.21.15.43:0/2268695005 conn(0x7f54380b4400 msgr2=0x7f543815a870 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-09T15:08:26.694 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:26.689+0000 7f543dd22640 1 -- 172.21.15.43:0/2268695005 shutdown_connections 2023-08-09T15:08:26.694 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:26.689+0000 7f543dd22640 1 -- 172.21.15.43:0/2268695005 wait complete. 2023-08-09T15:08:26.709 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-08-09T15:08:26.930 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:26.928+0000 7fe712823640 1 Processor -- start 2023-08-09T15:08:26.930 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:26.929+0000 7fe712823640 1 -- start start 2023-08-09T15:08:26.931 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:26.930+0000 7fe712823640 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7fe70c070ef0 0x7fe70c0ab8d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T15:08:26.932 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:26.930+0000 7fe712823640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7fe70c0cc0b0 0x7fe70c06f310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T15:08:26.932 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:26.930+0000 7fe712823640 1 -- --> v1:172.21.15.203:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe70c0acfa0 con 0x7fe70c0abe10 2023-08-09T15:08:26.932 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:26.930+0000 7fe712823640 1 -- --> v2:172.21.15.43:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe70c15c940 con 0x7fe70c0cc0b0 2023-08-09T15:08:26.932 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:26.930+0000 7fe712823640 1 -- --> v2:172.21.15.203:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe70c0ac390 con 0x7fe70c070ef0 2023-08-09T15:08:26.933 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:26.931+0000 7fe712022640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7fe70c0cc0b0 0x7fe70c06f310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T15:08:26.933 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:26.931+0000 7fe712022640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7fe70c0cc0b0 0x7fe70c06f310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:35956/0 (socket says 172.21.15.43:35956) 2023-08-09T15:08:26.933 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:26.931+0000 7fe712022640 1 -- 172.21.15.43:0/890967537 learned_addr learned my addr 172.21.15.43:0/890967537 (peer_addr_for_me v2:172.21.15.43:0/0) 2023-08-09T15:08:26.933 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:26.931+0000 7fe711821640 1 --2- 172.21.15.43:0/890967537 >> v2:172.21.15.203:3300/0 conn(0x7fe70c070ef0 0x7fe70c0ab8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T15:08:26.933 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:26.931+0000 7fe712022640 1 -- 172.21.15.43:0/890967537 >> v2:172.21.15.203:3300/0 conn(0x7fe70c070ef0 msgr2=0x7fe70c0ab8d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-09T15:08:26.934 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:26.931+0000 7fe712022640 1 --2- 172.21.15.43:0/890967537 >> v2:172.21.15.203:3300/0 conn(0x7fe70c070ef0 0x7fe70c0ab8d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T15:08:26.934 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:26.931+0000 7fe712022640 1 -- 172.21.15.43:0/890967537 >> v1:172.21.15.203:6789/0 conn(0x7fe70c0abe10 legacy=0x7fe70c0cb9a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-09T15:08:26.934 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:26.931+0000 7fe712022640 1 -- 172.21.15.43:0/890967537 --> v2:172.21.15.43:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe70c0ac670 con 0x7fe70c0cc0b0 2023-08-09T15:08:26.934 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:26.932+0000 7fe711821640 1 --2- 172.21.15.43:0/890967537 >> v2:172.21.15.203:3300/0 conn(0x7fe70c070ef0 0x7fe70c0ab8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-08-09T15:08:26.935 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:26.933+0000 7fe712022640 1 --2- 172.21.15.43:0/890967537 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fe70c0cc0b0 0x7fe70c06f310 secure :-1 s=READY pgs=2124 cs=0 l=1 rev1=1 crypto rx=0x7fe70c0cb920 tx=0x7fe70402bf80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c64820b8399c830 server_cookie=0 in_seq=0 out_seq=0 2023-08-09T15:08:26.935 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:26.933+0000 7fe71081f640 1 -- 172.21.15.43:0/890967537 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe70402cb20 con 0x7fe70c0cc0b0 2023-08-09T15:08:26.935 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:26.933+0000 7fe71081f640 1 -- 172.21.15.43:0/890967537 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe70402cc80 con 0x7fe70c0cc0b0 2023-08-09T15:08:26.935 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:26.933+0000 7fe71081f640 1 -- 172.21.15.43:0/890967537 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe7040343f0 con 0x7fe70c0cc0b0 2023-08-09T15:08:26.936 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:26.934+0000 7fe712823640 1 -- 172.21.15.43:0/890967537 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fe70c0cc0b0 msgr2=0x7fe70c06f310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T15:08:26.936 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:26.934+0000 7fe712823640 1 --2- 172.21.15.43:0/890967537 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fe70c0cc0b0 0x7fe70c06f310 secure :-1 s=READY pgs=2124 cs=0 l=1 rev1=1 crypto rx=0x7fe70c0cb920 tx=0x7fe70402bf80 comp rx=0 tx=0).stop 2023-08-09T15:08:26.936 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:26.934+0000 7fe712823640 1 -- 172.21.15.43:0/890967537 shutdown_connections 2023-08-09T15:08:26.937 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:26.934+0000 7fe712823640 1 --2- 172.21.15.43:0/890967537 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fe70c0cc0b0 0x7fe70c06f310 unknown :-1 s=CLOSED pgs=2124 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T15:08:26.937 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:26.934+0000 7fe712823640 1 --2- 172.21.15.43:0/890967537 >> v2:172.21.15.203:3300/0 conn(0x7fe70c070ef0 0x7fe70c0ab8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T15:08:26.937 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:26.934+0000 7fe712823640 1 -- 172.21.15.43:0/890967537 >> 172.21.15.43:0/890967537 conn(0x7fe70c0cddd0 msgr2=0x7fe70c0ce1e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-09T15:08:26.937 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:26.934+0000 7fe712823640 1 -- 172.21.15.43:0/890967537 shutdown_connections 2023-08-09T15:08:26.937 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:26.934+0000 7fe712823640 1 -- 172.21.15.43:0/890967537 wait complete. 2023-08-09T15:08:26.938 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:26.937+0000 7fe712823640 1 Processor -- start 2023-08-09T15:08:26.938 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:26.937+0000 7fe712823640 1 -- start start 2023-08-09T15:08:26.939 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:26.938+0000 7fe712823640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fe70c070ef0 0x7fe70c1582d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T15:08:26.940 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:26.938+0000 7fe712823640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fe70c0abe10 0x7fe70c144de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T15:08:26.940 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:26.938+0000 7fe712823640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fe70c0cc0b0 0x7fe70c1f1660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T15:08:26.940 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:26.938+0000 7fe712823640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe70c15e630 con 0x7fe70c070ef0 2023-08-09T15:08:26.940 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:26.938+0000 7fe712823640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe70c15da20 con 0x7fe70c0cc0b0 2023-08-09T15:08:26.940 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:26.938+0000 7fe712823640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe70c15dd00 con 0x7fe70c0abe10 2023-08-09T15:08:26.940 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:26.938+0000 7fe711821640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fe70c070ef0 0x7fe70c1582d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T15:08:26.941 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:26.938+0000 7fe711821640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fe70c070ef0 0x7fe70c1582d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:35962/0 (socket says 172.21.15.43:35962) 2023-08-09T15:08:26.941 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:26.939+0000 7fe711821640 1 -- 172.21.15.43:0/1831102064 learned_addr learned my addr 172.21.15.43:0/1831102064 (peer_addr_for_me v2:172.21.15.43:0/0) 2023-08-09T15:08:26.942 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:26.939+0000 7fe712022640 1 --2- 172.21.15.43:0/1831102064 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fe70c0cc0b0 0x7fe70c1f1660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T15:08:26.942 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:26.939+0000 7fe711821640 1 -- 172.21.15.43:0/1831102064 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fe70c0abe10 msgr2=0x7fe70c144de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T15:08:26.942 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:26.939+0000 7fe711821640 1 --2- 172.21.15.43:0/1831102064 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fe70c0abe10 0x7fe70c144de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T15:08:26.942 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:26.939+0000 7fe711821640 1 -- 172.21.15.43:0/1831102064 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fe70c0cc0b0 msgr2=0x7fe70c1f1660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T15:08:26.942 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:26.939+0000 7fe711821640 1 --2- 172.21.15.43:0/1831102064 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fe70c0cc0b0 0x7fe70c1f1660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T15:08:26.942 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:26.939+0000 7fe711821640 1 -- 172.21.15.43:0/1831102064 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe704009b30 con 0x7fe70c070ef0 2023-08-09T15:08:26.942 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:26.939+0000 7fe712022640 1 --2- 172.21.15.43:0/1831102064 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fe70c0cc0b0 0x7fe70c1f1660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-08-09T15:08:26.943 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:26.939+0000 7fe711821640 1 --2- 172.21.15.43:0/1831102064 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fe70c070ef0 0x7fe70c1582d0 secure :-1 s=READY pgs=2125 cs=0 l=1 rev1=1 crypto rx=0x7fe70800c7f0 tx=0x7fe70800ccb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-09T15:08:26.943 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:26.940+0000 7fe7027fc640 1 -- 172.21.15.43:0/1831102064 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe708007820 con 0x7fe70c070ef0 2023-08-09T15:08:26.943 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:26.940+0000 7fe7027fc640 1 -- 172.21.15.43:0/1831102064 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe708007980 con 0x7fe70c070ef0 2023-08-09T15:08:26.943 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:26.940+0000 7fe7027fc640 1 -- 172.21.15.43:0/1831102064 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe708007cf0 con 0x7fe70c070ef0 2023-08-09T15:08:26.943 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:26.940+0000 7fe712823640 1 -- 172.21.15.43:0/1831102064 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe70c15dfe0 con 0x7fe70c070ef0 2023-08-09T15:08:26.943 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:26.940+0000 7fe712823640 1 -- 172.21.15.43:0/1831102064 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe70c158810 con 0x7fe70c070ef0 2023-08-09T15:08:26.944 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:26.941+0000 7fe712823640 1 -- 172.21.15.43:0/1831102064 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe70c04e390 con 0x7fe70c070ef0 2023-08-09T15:08:26.944 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:26.942+0000 7fe7027fc640 1 -- 172.21.15.43:0/1831102064 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 104) v1 ==== 88179+0+0 (secure 0 0 0) 0x7fe7080113f0 con 0x7fe70c070ef0 2023-08-09T15:08:26.944 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:26.943+0000 7fe7027fc640 1 --2- 172.21.15.43:0/1831102064 >> [v2:172.21.15.203:6832/3082732769,v1:172.21.15.203:6833/3082732769] conn(0x7fe6f0069ea0 0x7fe6f006c360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T15:08:26.945 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:26.944+0000 7fe7027fc640 1 -- 172.21.15.43:0/1831102064 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(193..193 src has 1..193) v4 ==== 7486+0+0 (secure 0 0 0) 0x7fe708088ba0 con 0x7fe70c070ef0 2023-08-09T15:08:26.945 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:26.944+0000 7fe711020640 1 --2- 172.21.15.43:0/1831102064 >> [v2:172.21.15.203:6832/3082732769,v1:172.21.15.203:6833/3082732769] conn(0x7fe6f0069ea0 0x7fe6f006c360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T15:08:26.946 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:26.945+0000 7fe711020640 1 --2- 172.21.15.43:0/1831102064 >> [v2:172.21.15.203:6832/3082732769,v1:172.21.15.203:6833/3082732769] conn(0x7fe6f0069ea0 0x7fe6f006c360 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7fe6fc006fd0 tx=0x7fe6fc006bc0 comp rx=0 tx=0).ready entity=mgr.8500 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-09T15:08:26.949 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:26.948+0000 7fe7027fc640 1 -- 172.21.15.43:0/1831102064 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178066 (secure 0 0 0) 0x7fe708053ee0 con 0x7fe70c070ef0 2023-08-09T15:08:27.138 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:27.136+0000 7fe712823640 1 -- 172.21.15.43:0/1831102064 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7fe70c0b3d50 con 0x7fe70c070ef0 2023-08-09T15:08:27.138 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:27.137+0000 7fe7027fc640 1 -- 172.21.15.43:0/1831102064 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v193) v1 ==== 93+0+10 (secure 0 0 0) 0x7fe70801f0a0 con 0x7fe70c070ef0 2023-08-09T15:08:27.139 INFO:teuthology.orchestra.run.smithi043.stdout:pg_num: 1 2023-08-09T15:08:27.141 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:27.139+0000 7fe712823640 1 -- 172.21.15.43:0/1831102064 >> [v2:172.21.15.203:6832/3082732769,v1:172.21.15.203:6833/3082732769] conn(0x7fe6f0069ea0 msgr2=0x7fe6f006c360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T15:08:27.141 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:27.139+0000 7fe712823640 1 --2- 172.21.15.43:0/1831102064 >> [v2:172.21.15.203:6832/3082732769,v1:172.21.15.203:6833/3082732769] conn(0x7fe6f0069ea0 0x7fe6f006c360 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7fe6fc006fd0 tx=0x7fe6fc006bc0 comp rx=0 tx=0).stop 2023-08-09T15:08:27.141 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:27.139+0000 7fe712823640 1 -- 172.21.15.43:0/1831102064 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fe70c070ef0 msgr2=0x7fe70c1582d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T15:08:27.141 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:27.139+0000 7fe712823640 1 --2- 172.21.15.43:0/1831102064 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fe70c070ef0 0x7fe70c1582d0 secure :-1 s=READY pgs=2125 cs=0 l=1 rev1=1 crypto rx=0x7fe70800c7f0 tx=0x7fe70800ccb0 comp rx=0 tx=0).stop 2023-08-09T15:08:27.141 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:27.139+0000 7fe712823640 1 -- 172.21.15.43:0/1831102064 shutdown_connections 2023-08-09T15:08:27.141 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:27.139+0000 7fe712823640 1 --2- 172.21.15.43:0/1831102064 >> [v2:172.21.15.203:6832/3082732769,v1:172.21.15.203:6833/3082732769] conn(0x7fe6f0069ea0 0x7fe6f006c360 unknown :-1 s=CLOSED pgs=22 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T15:08:27.141 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:27.139+0000 7fe712823640 1 --2- 172.21.15.43:0/1831102064 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fe70c0cc0b0 0x7fe70c1f1660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T15:08:27.142 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:27.139+0000 7fe712823640 1 --2- 172.21.15.43:0/1831102064 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fe70c0abe10 0x7fe70c144de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T15:08:27.142 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:27.139+0000 7fe712823640 1 --2- 172.21.15.43:0/1831102064 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fe70c070ef0 0x7fe70c1582d0 unknown :-1 s=CLOSED pgs=2125 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T15:08:27.142 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:27.139+0000 7fe712823640 1 -- 172.21.15.43:0/1831102064 >> 172.21.15.43:0/1831102064 conn(0x7fe70c0cddd0 msgr2=0x7fe70c0b2780 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-09T15:08:27.142 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:27.140+0000 7fe712823640 1 -- 172.21.15.43:0/1831102064 shutdown_connections 2023-08-09T15:08:27.142 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:27.140+0000 7fe712823640 1 -- 172.21.15.43:0/1831102064 wait complete. 2023-08-09T15:08:27.157 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-08-09T15:08:27.376 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:27.375+0000 7f17f32e5640 1 Processor -- start 2023-08-09T15:08:27.377 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:27.375+0000 7f17f32e5640 1 -- start start 2023-08-09T15:08:27.378 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:27.377+0000 7f17f32e5640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f17ec06fbb0 0x7f17ec0add10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T15:08:27.379 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:27.377+0000 7f17f32e5640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f17ec0cc210 0x7f17ec06cff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T15:08:27.379 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:27.377+0000 7f17f32e5640 1 -- --> v1:172.21.15.203:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f17ec06cb60 con 0x7f17ec06e0a0 2023-08-09T15:08:27.380 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:27.377+0000 7f17f32e5640 1 -- --> v2:172.21.15.43:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f17ec0cc6a0 con 0x7f17ec0cc210 2023-08-09T15:08:27.380 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:27.377+0000 7f17f32e5640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f17ec0ac8c0 con 0x7f17ec06fbb0 2023-08-09T15:08:27.381 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:27.378+0000 7f17f2ae4640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f17ec0cc210 0x7f17ec06cff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T15:08:27.381 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:27.378+0000 7f17f2ae4640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f17ec0cc210 0x7f17ec06cff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:35978/0 (socket says 172.21.15.43:35978) 2023-08-09T15:08:27.381 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:27.378+0000 7f17f2ae4640 1 -- 172.21.15.43:0/528294744 learned_addr learned my addr 172.21.15.43:0/528294744 (peer_addr_for_me v2:172.21.15.43:0/0) 2023-08-09T15:08:27.381 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:27.378+0000 7f17f22e3640 1 --2- 172.21.15.43:0/528294744 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f17ec06fbb0 0x7f17ec0add10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T15:08:27.381 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:27.378+0000 7f17f2ae4640 1 -- 172.21.15.43:0/528294744 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f17ec06fbb0 msgr2=0x7f17ec0add10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-09T15:08:27.382 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:27.378+0000 7f17f2ae4640 1 --2- 172.21.15.43:0/528294744 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f17ec06fbb0 0x7f17ec0add10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T15:08:27.382 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:27.378+0000 7f17f2ae4640 1 -- 172.21.15.43:0/528294744 >> v1:172.21.15.203:6789/0 conn(0x7f17ec06e0a0 legacy=0x7f17ec06ff90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-09T15:08:27.382 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:27.378+0000 7f17f2ae4640 1 -- 172.21.15.43:0/528294744 --> v2:172.21.15.43:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f17ec0ac300 con 0x7f17ec0cc210 2023-08-09T15:08:27.382 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:27.379+0000 7f17f2ae4640 1 --2- 172.21.15.43:0/528294744 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f17ec0cc210 0x7f17ec06cff0 secure :-1 s=READY pgs=2126 cs=0 l=1 rev1=1 crypto rx=0x7f17e4009e50 tx=0x7f17e402bee0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c4abc355be9e9170 server_cookie=0 in_seq=0 out_seq=0 2023-08-09T15:08:27.382 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:27.380+0000 7f17f12e1640 1 -- 172.21.15.43:0/528294744 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f17e402ca80 con 0x7f17ec0cc210 2023-08-09T15:08:27.382 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:27.380+0000 7f17f12e1640 1 -- 172.21.15.43:0/528294744 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f17e402cbe0 con 0x7f17ec0cc210 2023-08-09T15:08:27.382 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:27.380+0000 7f17f12e1640 1 -- 172.21.15.43:0/528294744 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f17e40343f0 con 0x7f17ec0cc210 2023-08-09T15:08:27.383 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:27.380+0000 7f17f32e5640 1 -- 172.21.15.43:0/528294744 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f17ec0cc210 msgr2=0x7f17ec06cff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T15:08:27.383 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:27.380+0000 7f17f32e5640 1 --2- 172.21.15.43:0/528294744 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f17ec0cc210 0x7f17ec06cff0 secure :-1 s=READY pgs=2126 cs=0 l=1 rev1=1 crypto rx=0x7f17e4009e50 tx=0x7f17e402bee0 comp rx=0 tx=0).stop 2023-08-09T15:08:27.383 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:27.381+0000 7f17f32e5640 1 -- 172.21.15.43:0/528294744 shutdown_connections 2023-08-09T15:08:27.383 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:27.381+0000 7f17f32e5640 1 --2- 172.21.15.43:0/528294744 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f17ec0cc210 0x7f17ec06cff0 unknown :-1 s=CLOSED pgs=2126 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T15:08:27.384 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:27.381+0000 7f17f32e5640 1 --2- 172.21.15.43:0/528294744 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f17ec06fbb0 0x7f17ec0add10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T15:08:27.384 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:27.381+0000 7f17f32e5640 1 -- 172.21.15.43:0/528294744 >> 172.21.15.43:0/528294744 conn(0x7f17ec0ce640 msgr2=0x7f17ec0cea50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-09T15:08:27.384 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:27.381+0000 7f17f32e5640 1 -- 172.21.15.43:0/528294744 shutdown_connections 2023-08-09T15:08:27.384 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:27.381+0000 7f17f32e5640 1 -- 172.21.15.43:0/528294744 wait complete. 2023-08-09T15:08:27.385 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:27.382+0000 7f17f32e5640 1 Processor -- start 2023-08-09T15:08:27.386 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:27.383+0000 7f17f32e5640 1 -- start start 2023-08-09T15:08:27.386 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:27.384+0000 7f17f32e5640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f17ec06e0a0 0x7f17ec15e180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T15:08:27.386 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:27.384+0000 7f17f32e5640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f17ec06fbb0 0x7f17ec15c900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T15:08:27.386 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:27.384+0000 7f17f32e5640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f17ec0cc210 0x7f17ec15a840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T15:08:27.386 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:27.384+0000 7f17f32e5640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f17ec15ce40 con 0x7f17ec06e0a0 2023-08-09T15:08:27.386 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:27.384+0000 7f17f32e5640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f17ec15d0e0 con 0x7f17ec06fbb0 2023-08-09T15:08:27.386 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:27.384+0000 7f17f32e5640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f17ec15d3c0 con 0x7f17ec0cc210 2023-08-09T15:08:27.387 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:27.384+0000 7f17f22e3640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f17ec06e0a0 0x7f17ec15e180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T15:08:27.387 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:27.384+0000 7f17f22e3640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f17ec06e0a0 0x7f17ec15e180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:35994/0 (socket says 172.21.15.43:35994) 2023-08-09T15:08:27.387 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:27.384+0000 7f17f22e3640 1 -- 172.21.15.43:0/3121613803 learned_addr learned my addr 172.21.15.43:0/3121613803 (peer_addr_for_me v2:172.21.15.43:0/0) 2023-08-09T15:08:27.387 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:27.384+0000 7f17f2ae4640 1 --2- 172.21.15.43:0/3121613803 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f17ec0cc210 0x7f17ec15a840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T15:08:27.387 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:27.384+0000 7f17f1ae2640 1 --2- 172.21.15.43:0/3121613803 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f17ec06fbb0 0x7f17ec15c900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T15:08:27.387 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:27.384+0000 7f17f22e3640 1 -- 172.21.15.43:0/3121613803 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f17ec0cc210 msgr2=0x7f17ec15a840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T15:08:27.388 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:27.384+0000 7f17f22e3640 1 --2- 172.21.15.43:0/3121613803 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f17ec0cc210 0x7f17ec15a840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T15:08:27.388 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:27.384+0000 7f17f22e3640 1 -- 172.21.15.43:0/3121613803 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f17ec06fbb0 msgr2=0x7f17ec15c900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T15:08:27.388 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:27.384+0000 7f17f22e3640 1 --2- 172.21.15.43:0/3121613803 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f17ec06fbb0 0x7f17ec15c900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T15:08:27.388 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:27.384+0000 7f17f22e3640 1 -- 172.21.15.43:0/3121613803 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f17e4009b30 con 0x7f17ec06e0a0 2023-08-09T15:08:27.388 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:27.385+0000 7f17f1ae2640 1 --2- 172.21.15.43:0/3121613803 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f17ec06fbb0 0x7f17ec15c900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-08-09T15:08:27.389 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:27.385+0000 7f17f22e3640 1 --2- 172.21.15.43:0/3121613803 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f17ec06e0a0 0x7f17ec15e180 secure :-1 s=READY pgs=2127 cs=0 l=1 rev1=1 crypto rx=0x7f17e800d5f0 tx=0x7f17e800dab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-09T15:08:27.389 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:27.385+0000 7f17e2ffd640 1 -- 172.21.15.43:0/3121613803 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f17e80125f0 con 0x7f17ec06e0a0 2023-08-09T15:08:27.389 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:27.385+0000 7f17e2ffd640 1 -- 172.21.15.43:0/3121613803 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f17e8012750 con 0x7f17ec06e0a0 2023-08-09T15:08:27.389 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:27.385+0000 7f17f32e5640 1 -- 172.21.15.43:0/3121613803 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f17ec15d6a0 con 0x7f17ec06e0a0 2023-08-09T15:08:27.390 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:27.385+0000 7f17e2ffd640 1 -- 172.21.15.43:0/3121613803 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f17e8012b00 con 0x7f17ec06e0a0 2023-08-09T15:08:27.390 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:27.385+0000 7f17f32e5640 1 -- 172.21.15.43:0/3121613803 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f17ec15dc60 con 0x7f17ec06e0a0 2023-08-09T15:08:27.390 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:27.387+0000 7f17f32e5640 1 -- 172.21.15.43:0/3121613803 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f17ec15de20 con 0x7f17ec06e0a0 2023-08-09T15:08:27.390 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:27.387+0000 7f17e2ffd640 1 -- 172.21.15.43:0/3121613803 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 104) v1 ==== 88179+0+0 (secure 0 0 0) 0x7f17e8012c60 con 0x7f17ec06e0a0 2023-08-09T15:08:27.391 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:27.388+0000 7f17e2ffd640 1 --2- 172.21.15.43:0/3121613803 >> [v2:172.21.15.203:6832/3082732769,v1:172.21.15.203:6833/3082732769] conn(0x7f17d0069ea0 0x7f17d006c360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T15:08:27.392 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:27.388+0000 7f17e2ffd640 1 -- 172.21.15.43:0/3121613803 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(193..193 src has 1..193) v4 ==== 7486+0+0 (secure 0 0 0) 0x7f17e8088cd0 con 0x7f17ec06e0a0 2023-08-09T15:08:27.392 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:27.389+0000 7f17f1ae2640 1 --2- 172.21.15.43:0/3121613803 >> [v2:172.21.15.203:6832/3082732769,v1:172.21.15.203:6833/3082732769] conn(0x7f17d0069ea0 0x7f17d006c360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T15:08:27.392 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:27.390+0000 7f17f1ae2640 1 --2- 172.21.15.43:0/3121613803 >> [v2:172.21.15.203:6832/3082732769,v1:172.21.15.203:6833/3082732769] conn(0x7f17d0069ea0 0x7f17d006c360 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7f17ec1559a0 tx=0x7f17dc009040 comp rx=0 tx=0).ready entity=mgr.8500 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-09T15:08:27.394 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:27.393+0000 7f17e2ffd640 1 -- 172.21.15.43:0/3121613803 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178066 (secure 0 0 0) 0x7f17e80540a0 con 0x7f17ec06e0a0 2023-08-09T15:08:27.587 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:27.586+0000 7f17f32e5640 1 -- 172.21.15.43:0/3121613803 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f17ec073020 con 0x7f17ec06e0a0 2023-08-09T15:08:27.588 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:27.586+0000 7f17e2ffd640 1 -- 172.21.15.43:0/3121613803 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v193) v1 ==== 104+0+11 (secure 0 0 0) 0x7f17e80576c0 con 0x7f17ec06e0a0 2023-08-09T15:08:27.588 INFO:teuthology.orchestra.run.smithi043.stdout:pg_num: 32 2023-08-09T15:08:27.590 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:27.589+0000 7f17f32e5640 1 -- 172.21.15.43:0/3121613803 >> [v2:172.21.15.203:6832/3082732769,v1:172.21.15.203:6833/3082732769] conn(0x7f17d0069ea0 msgr2=0x7f17d006c360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T15:08:27.590 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:27.589+0000 7f17f32e5640 1 --2- 172.21.15.43:0/3121613803 >> [v2:172.21.15.203:6832/3082732769,v1:172.21.15.203:6833/3082732769] conn(0x7f17d0069ea0 0x7f17d006c360 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7f17ec1559a0 tx=0x7f17dc009040 comp rx=0 tx=0).stop 2023-08-09T15:08:27.591 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:27.589+0000 7f17f32e5640 1 -- 172.21.15.43:0/3121613803 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f17ec06e0a0 msgr2=0x7f17ec15e180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T15:08:27.591 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:27.589+0000 7f17f32e5640 1 --2- 172.21.15.43:0/3121613803 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f17ec06e0a0 0x7f17ec15e180 secure :-1 s=READY pgs=2127 cs=0 l=1 rev1=1 crypto rx=0x7f17e800d5f0 tx=0x7f17e800dab0 comp rx=0 tx=0).stop 2023-08-09T15:08:27.591 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:27.589+0000 7f17f32e5640 1 -- 172.21.15.43:0/3121613803 shutdown_connections 2023-08-09T15:08:27.591 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:27.589+0000 7f17f32e5640 1 --2- 172.21.15.43:0/3121613803 >> [v2:172.21.15.203:6832/3082732769,v1:172.21.15.203:6833/3082732769] conn(0x7f17d0069ea0 0x7f17d006c360 unknown :-1 s=CLOSED pgs=23 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T15:08:27.592 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:27.589+0000 7f17f32e5640 1 --2- 172.21.15.43:0/3121613803 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f17ec0cc210 0x7f17ec15a840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T15:08:27.592 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:27.589+0000 7f17f32e5640 1 --2- 172.21.15.43:0/3121613803 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f17ec06fbb0 0x7f17ec15c900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T15:08:27.592 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:27.589+0000 7f17f32e5640 1 --2- 172.21.15.43:0/3121613803 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f17ec06e0a0 0x7f17ec15e180 unknown :-1 s=CLOSED pgs=2127 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T15:08:27.592 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:27.589+0000 7f17f32e5640 1 -- 172.21.15.43:0/3121613803 >> 172.21.15.43:0/3121613803 conn(0x7f17ec0ce640 msgr2=0x7f17ec0b34b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-09T15:08:27.592 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:27.590+0000 7f17f32e5640 1 -- 172.21.15.43:0/3121613803 shutdown_connections 2023-08-09T15:08:27.592 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:27.590+0000 7f17f32e5640 1 -- 172.21.15.43:0/3121613803 wait complete. 2023-08-09T15:08:27.610 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-08-09T15:08:27.844 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:27.843+0000 7f30f359e640 1 Processor -- start 2023-08-09T15:08:27.844 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:27.843+0000 7f30f359e640 1 -- start start 2023-08-09T15:08:27.845 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:27.844+0000 7f30f359e640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f30f4158940 0x7f30f40c9040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T15:08:27.846 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:27.844+0000 7f30f359e640 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7f30f40ae7f0 0x7f30f4070720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T15:08:27.846 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:27.844+0000 7f30f359e640 1 -- --> v1:172.21.15.43:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f30f4071630 con 0x7f30f4070e50 2023-08-09T15:08:27.846 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:27.844+0000 7f30f359e640 1 -- --> v2:172.21.15.43:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f30f406f4a0 con 0x7f30f4158940 2023-08-09T15:08:27.846 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:27.844+0000 7f30f359e640 1 -- --> v2:172.21.15.203:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f30f4070c60 con 0x7f30f40ae7f0 2023-08-09T15:08:27.847 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:27.845+0000 7f30f259c640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f30f4158940 0x7f30f40c9040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T15:08:27.847 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:27.845+0000 7f30f259c640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f30f4158940 0x7f30f40c9040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:36006/0 (socket says 172.21.15.43:36006) 2023-08-09T15:08:27.847 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:27.845+0000 7f30f259c640 1 -- 172.21.15.43:0/2433778920 learned_addr learned my addr 172.21.15.43:0/2433778920 (peer_addr_for_me v2:172.21.15.43:0/0) 2023-08-09T15:08:27.847 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:27.845+0000 7f30f259c640 1 -- 172.21.15.43:0/2433778920 >> v2:172.21.15.203:3300/0 conn(0x7f30f40ae7f0 msgr2=0x7f30f4070720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-09T15:08:27.848 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:27.845+0000 7f30f2d9d640 1 --2- 172.21.15.43:0/2433778920 >> v2:172.21.15.203:3300/0 conn(0x7f30f40ae7f0 0x7f30f4070720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T15:08:27.848 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:27.845+0000 7f30f259c640 1 --2- 172.21.15.43:0/2433778920 >> v2:172.21.15.203:3300/0 conn(0x7f30f40ae7f0 0x7f30f4070720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T15:08:27.848 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:27.845+0000 7f30f259c640 1 -- 172.21.15.43:0/2433778920 >> v1:172.21.15.43:6789/0 conn(0x7f30f4070e50 legacy=0x7f30f40ae0e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T15:08:27.848 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:27.845+0000 7f30f259c640 1 -- 172.21.15.43:0/2433778920 --> v2:172.21.15.43:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f30f406f680 con 0x7f30f4158940 2023-08-09T15:08:27.848 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:27.845+0000 7f30f159a640 1 -- 172.21.15.43:0/2433778920 <== mon.0 v1:172.21.15.43:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 624540947 0 0) 0x7f30f4071630 con 0x7f30f4070e50 2023-08-09T15:08:27.848 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:27.845+0000 7f30f2d9d640 1 --2- 172.21.15.43:0/2433778920 >> v2:172.21.15.203:3300/0 conn(0x7f30f40ae7f0 0x7f30f4070720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-08-09T15:08:27.849 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:27.847+0000 7f30f259c640 1 --2- 172.21.15.43:0/2433778920 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f30f4158940 0x7f30f40c9040 secure :-1 s=READY pgs=2129 cs=0 l=1 rev1=1 crypto rx=0x7f30f40b3f10 tx=0x7f30e402e0f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=570579e774f81347 server_cookie=0 in_seq=0 out_seq=0 2023-08-09T15:08:27.849 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:27.847+0000 7f30f159a640 1 -- 172.21.15.43:0/2433778920 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f30e402ec90 con 0x7f30f4158940 2023-08-09T15:08:27.849 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:27.847+0000 7f30f159a640 1 -- 172.21.15.43:0/2433778920 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f30e402edf0 con 0x7f30f4158940 2023-08-09T15:08:27.850 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:27.847+0000 7f30f159a640 1 -- 172.21.15.43:0/2433778920 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f30e4036610 con 0x7f30f4158940 2023-08-09T15:08:27.850 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:27.847+0000 7f30f359e640 1 -- 172.21.15.43:0/2433778920 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f30f4158940 msgr2=0x7f30f40c9040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T15:08:27.850 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:27.847+0000 7f30f359e640 1 --2- 172.21.15.43:0/2433778920 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f30f4158940 0x7f30f40c9040 secure :-1 s=READY pgs=2129 cs=0 l=1 rev1=1 crypto rx=0x7f30f40b3f10 tx=0x7f30e402e0f0 comp rx=0 tx=0).stop 2023-08-09T15:08:27.850 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:27.848+0000 7f30f359e640 1 -- 172.21.15.43:0/2433778920 shutdown_connections 2023-08-09T15:08:27.850 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:27.848+0000 7f30f359e640 1 --2- 172.21.15.43:0/2433778920 >> v2:172.21.15.203:3300/0 conn(0x7f30f40ae7f0 0x7f30f4070720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T15:08:27.850 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:27.848+0000 7f30f359e640 1 --2- 172.21.15.43:0/2433778920 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f30f4158940 0x7f30f40c9040 unknown :-1 s=CLOSED pgs=2129 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T15:08:27.850 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:27.848+0000 7f30f359e640 1 -- 172.21.15.43:0/2433778920 >> 172.21.15.43:0/2433778920 conn(0x7f30f40cddd0 msgr2=0x7f30f40ce1e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-09T15:08:27.851 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:27.848+0000 7f30f359e640 1 -- 172.21.15.43:0/2433778920 shutdown_connections 2023-08-09T15:08:27.851 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:27.848+0000 7f30f359e640 1 -- 172.21.15.43:0/2433778920 wait complete. 2023-08-09T15:08:27.851 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:27.849+0000 7f30f359e640 1 Processor -- start 2023-08-09T15:08:27.851 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:27.850+0000 7f30f359e640 1 -- start start 2023-08-09T15:08:27.852 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:27.851+0000 7f30f359e640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f30f4070e50 0x7f30f4153860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T15:08:27.853 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:27.851+0000 7f30f359e640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f30f40ae7f0 0x7f30f4158d20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T15:08:27.853 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:27.851+0000 7f30f359e640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f30f4158940 0x7f30f40b1510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T15:08:27.853 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:27.851+0000 7f30f359e640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f30f4159ca0 con 0x7f30f4158940 2023-08-09T15:08:27.853 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:27.851+0000 7f30f359e640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f30f41596e0 con 0x7f30f40ae7f0 2023-08-09T15:08:27.853 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:27.851+0000 7f30f359e640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f30f41599c0 con 0x7f30f4070e50 2023-08-09T15:08:27.853 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:27.851+0000 7f30f2d9d640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f30f4158940 0x7f30f40b1510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T15:08:27.853 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:27.851+0000 7f30f2d9d640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f30f4158940 0x7f30f40b1510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:36012/0 (socket says 172.21.15.43:36012) 2023-08-09T15:08:27.854 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:27.851+0000 7f30f2d9d640 1 -- 172.21.15.43:0/4242694784 learned_addr learned my addr 172.21.15.43:0/4242694784 (peer_addr_for_me v2:172.21.15.43:0/0) 2023-08-09T15:08:27.854 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:27.851+0000 7f30f1d9b640 1 --2- 172.21.15.43:0/4242694784 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f30f40ae7f0 0x7f30f4158d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T15:08:27.854 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:27.851+0000 7f30f259c640 1 --2- 172.21.15.43:0/4242694784 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f30f4070e50 0x7f30f4153860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T15:08:27.854 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:27.851+0000 7f30f2d9d640 1 -- 172.21.15.43:0/4242694784 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f30f4070e50 msgr2=0x7f30f4153860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T15:08:27.854 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:27.851+0000 7f30f2d9d640 1 --2- 172.21.15.43:0/4242694784 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f30f4070e50 0x7f30f4153860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T15:08:27.854 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:27.851+0000 7f30f2d9d640 1 -- 172.21.15.43:0/4242694784 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f30f40ae7f0 msgr2=0x7f30f4158d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T15:08:27.854 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:27.851+0000 7f30f2d9d640 1 --2- 172.21.15.43:0/4242694784 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f30f40ae7f0 0x7f30f4158d20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T15:08:27.855 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:27.851+0000 7f30f2d9d640 1 -- 172.21.15.43:0/4242694784 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f30e4009b30 con 0x7f30f4158940 2023-08-09T15:08:27.855 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:27.852+0000 7f30f2d9d640 1 --2- 172.21.15.43:0/4242694784 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f30f4158940 0x7f30f40b1510 secure :-1 s=READY pgs=2130 cs=0 l=1 rev1=1 crypto rx=0x7f30ec00c7f0 tx=0x7f30ec00ccb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-09T15:08:27.855 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:27.852+0000 7f30e2ffd640 1 -- 172.21.15.43:0/4242694784 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f30ec007820 con 0x7f30f4158940 2023-08-09T15:08:27.856 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:27.852+0000 7f30e2ffd640 1 -- 172.21.15.43:0/4242694784 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f30ec007980 con 0x7f30f4158940 2023-08-09T15:08:27.856 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:27.852+0000 7f30e2ffd640 1 -- 172.21.15.43:0/4242694784 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f30ec007d70 con 0x7f30f4158940 2023-08-09T15:08:27.856 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:27.852+0000 7f30f359e640 1 -- 172.21.15.43:0/4242694784 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f30f415a010 con 0x7f30f4158940 2023-08-09T15:08:27.856 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:27.852+0000 7f30f359e640 1 -- 172.21.15.43:0/4242694784 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f30f40b3760 con 0x7f30f4158940 2023-08-09T15:08:27.856 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:27.854+0000 7f30f359e640 1 -- 172.21.15.43:0/4242694784 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f30f40b3950 con 0x7f30f4158940 2023-08-09T15:08:27.857 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:27.855+0000 7f30e2ffd640 1 -- 172.21.15.43:0/4242694784 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 105) v1 ==== 88387+0+0 (secure 0 0 0) 0x7f30ec0113f0 con 0x7f30f4158940 2023-08-09T15:08:27.857 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:27.855+0000 7f30e2ffd640 1 --2- 172.21.15.43:0/4242694784 >> [v2:172.21.15.203:6832/3082732769,v1:172.21.15.203:6833/3082732769] conn(0x7f30d406a0e0 0x7f30d406c5a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T15:08:27.857 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:27.856+0000 7f30e2ffd640 1 -- 172.21.15.43:0/4242694784 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(193..193 src has 1..193) v4 ==== 7486+0+0 (secure 0 0 0) 0x7f30ec089070 con 0x7f30f4158940 2023-08-09T15:08:27.857 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:27.856+0000 7f30f259c640 1 --2- 172.21.15.43:0/4242694784 >> [v2:172.21.15.203:6832/3082732769,v1:172.21.15.203:6833/3082732769] conn(0x7f30d406a0e0 0x7f30d406c5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T15:08:27.858 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:27.857+0000 7f30f259c640 1 --2- 172.21.15.43:0/4242694784 >> [v2:172.21.15.203:6832/3082732769,v1:172.21.15.203:6833/3082732769] conn(0x7f30d406a0e0 0x7f30d406c5a0 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7f30e4037000 tx=0x7f30e400d040 comp rx=0 tx=0).ready entity=mgr.8500 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-09T15:08:27.861 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:27.860+0000 7f30e2ffd640 1 -- 172.21.15.43:0/4242694784 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178066 (secure 0 0 0) 0x7f30ec054060 con 0x7f30f4158940 2023-08-09T15:08:28.050 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:28.049+0000 7f30f359e640 1 -- 172.21.15.43:0/4242694784 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f30f404e390 con 0x7f30f4158940 2023-08-09T15:08:28.051 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:28.049+0000 7f30e2ffd640 1 -- 172.21.15.43:0/4242694784 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v193) v1 ==== 100+0+11 (secure 0 0 0) 0x7f30ec01f0a0 con 0x7f30f4158940 2023-08-09T15:08:28.051 INFO:teuthology.orchestra.run.smithi043.stdout:pg_num: 64 2023-08-09T15:08:28.053 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:28.052+0000 7f30f359e640 1 -- 172.21.15.43:0/4242694784 >> [v2:172.21.15.203:6832/3082732769,v1:172.21.15.203:6833/3082732769] conn(0x7f30d406a0e0 msgr2=0x7f30d406c5a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T15:08:28.053 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:28.052+0000 7f30f359e640 1 --2- 172.21.15.43:0/4242694784 >> [v2:172.21.15.203:6832/3082732769,v1:172.21.15.203:6833/3082732769] conn(0x7f30d406a0e0 0x7f30d406c5a0 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7f30e4037000 tx=0x7f30e400d040 comp rx=0 tx=0).stop 2023-08-09T15:08:28.053 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:28.052+0000 7f30f359e640 1 -- 172.21.15.43:0/4242694784 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f30f4158940 msgr2=0x7f30f40b1510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T15:08:28.054 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:28.052+0000 7f30f359e640 1 --2- 172.21.15.43:0/4242694784 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f30f4158940 0x7f30f40b1510 secure :-1 s=READY pgs=2130 cs=0 l=1 rev1=1 crypto rx=0x7f30ec00c7f0 tx=0x7f30ec00ccb0 comp rx=0 tx=0).stop 2023-08-09T15:08:28.054 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:28.052+0000 7f30f359e640 1 -- 172.21.15.43:0/4242694784 shutdown_connections 2023-08-09T15:08:28.054 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:28.052+0000 7f30f359e640 1 --2- 172.21.15.43:0/4242694784 >> [v2:172.21.15.203:6832/3082732769,v1:172.21.15.203:6833/3082732769] conn(0x7f30d406a0e0 0x7f30d406c5a0 unknown :-1 s=CLOSED pgs=25 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T15:08:28.054 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:28.052+0000 7f30f359e640 1 --2- 172.21.15.43:0/4242694784 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f30f4158940 0x7f30f40b1510 unknown :-1 s=CLOSED pgs=2130 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T15:08:28.054 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:28.052+0000 7f30f359e640 1 --2- 172.21.15.43:0/4242694784 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f30f40ae7f0 0x7f30f4158d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T15:08:28.054 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:28.052+0000 7f30f359e640 1 --2- 172.21.15.43:0/4242694784 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f30f4070e50 0x7f30f4153860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T15:08:28.054 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:28.052+0000 7f30f359e640 1 -- 172.21.15.43:0/4242694784 >> 172.21.15.43:0/4242694784 conn(0x7f30f40cddd0 msgr2=0x7f30f4157a10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-09T15:08:28.055 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:28.052+0000 7f30f359e640 1 -- 172.21.15.43:0/4242694784 shutdown_connections 2023-08-09T15:08:28.055 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:28.052+0000 7f30f359e640 1 -- 172.21.15.43:0/4242694784 wait complete. 2023-08-09T15:08:28.071 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-08-09T15:08:28.278 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:28.277+0000 7f8762514640 1 Processor -- start 2023-08-09T15:08:28.278 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:28.277+0000 7f8762514640 1 -- start start 2023-08-09T15:08:28.280 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:28.279+0000 7f8762514640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f875c0c8750 0x7f875c0ad340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T15:08:28.280 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:28.279+0000 7f8762514640 1 -- --> v1:172.21.15.43:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f875c0cbd90 con 0x7f875c0acdf0 2023-08-09T15:08:28.280 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:28.279+0000 7f8762514640 1 -- --> v1:172.21.15.203:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f875c0ca2b0 con 0x7f875c0c8b30 2023-08-09T15:08:28.280 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:28.279+0000 7f8762514640 1 -- --> v2:172.21.15.43:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f875c15e810 con 0x7f875c0c8750 2023-08-09T15:08:28.281 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:28.279+0000 7f8761512640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f875c0c8750 0x7f875c0ad340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T15:08:28.281 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:28.279+0000 7f8761d13640 1 --1- >> v1:172.21.15.43:6789/0 conn(0x7f875c0acdf0 0x7f875c070f00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.43:6789/0 says I am v1:172.21.15.43:41724/0 (socket says 172.21.15.43:41724) 2023-08-09T15:08:28.281 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:28.279+0000 7f8761d13640 1 -- 172.21.15.43:0/2347496264 learned_addr learned my addr 172.21.15.43:0/2347496264 (peer_addr_for_me v1:172.21.15.43:0/0) 2023-08-09T15:08:28.282 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:28.279+0000 7f8761512640 1 -- 172.21.15.43:0/2347496264 >> v1:172.21.15.203:6789/0 conn(0x7f875c0c8b30 legacy=0x7f875c06f410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-09T15:08:28.282 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:28.279+0000 7f8761512640 1 -- 172.21.15.43:0/2347496264 >> v1:172.21.15.43:6789/0 conn(0x7f875c0acdf0 legacy=0x7f875c070f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T15:08:28.282 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:28.280+0000 7f8761512640 1 -- 172.21.15.43:0/2347496264 --> v2:172.21.15.43:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f875c0c9a20 con 0x7f875c0c8750 2023-08-09T15:08:28.282 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:28.279+0000 7f8753fff640 1 -- 172.21.15.43:0/2347496264 <== mon.0 v1:172.21.15.43:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 402029998 0 0) 0x7f875c0cbd90 con 0x7f875c0acdf0 2023-08-09T15:08:28.283 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:28.281+0000 7f8761512640 1 --2- 172.21.15.43:0/2347496264 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f875c0c8750 0x7f875c0ad340 secure :-1 s=READY pgs=2132 cs=0 l=1 rev1=1 crypto rx=0x7f87580097a0 tx=0x7f875802b840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=82489bc550b52720 server_cookie=0 in_seq=0 out_seq=0 2023-08-09T15:08:28.283 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:28.281+0000 7f8753fff640 1 -- 172.21.15.43:0/2347496264 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f875802e070 con 0x7f875c0c8750 2023-08-09T15:08:28.283 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:28.281+0000 7f8753fff640 1 -- 172.21.15.43:0/2347496264 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f87580315a0 con 0x7f875c0c8750 2023-08-09T15:08:28.283 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:28.281+0000 7f8753fff640 1 -- 172.21.15.43:0/2347496264 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8758031930 con 0x7f875c0c8750 2023-08-09T15:08:28.284 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:28.281+0000 7f8762514640 1 -- 172.21.15.43:0/2347496264 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f875c0c8750 msgr2=0x7f875c0ad340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T15:08:28.284 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:28.281+0000 7f8762514640 1 --2- 172.21.15.43:0/2347496264 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f875c0c8750 0x7f875c0ad340 secure :-1 s=READY pgs=2132 cs=0 l=1 rev1=1 crypto rx=0x7f87580097a0 tx=0x7f875802b840 comp rx=0 tx=0).stop 2023-08-09T15:08:28.284 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:28.282+0000 7f8762514640 1 -- 172.21.15.43:0/2347496264 shutdown_connections 2023-08-09T15:08:28.284 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:28.282+0000 7f8762514640 1 --2- 172.21.15.43:0/2347496264 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f875c0c8750 0x7f875c0ad340 unknown :-1 s=CLOSED pgs=2132 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T15:08:28.284 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:28.282+0000 7f8762514640 1 -- 172.21.15.43:0/2347496264 >> 172.21.15.43:0/2347496264 conn(0x7f875c0cde90 msgr2=0x7f875c0ce2a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-09T15:08:28.285 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:28.282+0000 7f8762514640 1 -- 172.21.15.43:0/2347496264 shutdown_connections 2023-08-09T15:08:28.285 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:28.282+0000 7f8762514640 1 -- 172.21.15.43:0/2347496264 wait complete. 2023-08-09T15:08:28.285 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:28.284+0000 7f8762514640 1 Processor -- start 2023-08-09T15:08:28.285 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:28.284+0000 7f8762514640 1 -- start start 2023-08-09T15:08:28.286 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:28.285+0000 7f8762514640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f875c0acdf0 0x7f875c1f2ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T15:08:28.287 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:28.285+0000 7f8762514640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f875c0c8750 0x7f875c1f1c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T15:08:28.287 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:28.285+0000 7f8762514640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f875c0c8b30 0x7f875c1f2150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T15:08:28.287 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:28.285+0000 7f8762514640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f875c0c9f70 con 0x7f875c0c8750 2023-08-09T15:08:28.287 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:28.285+0000 7f8762514640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f875c1f6c00 con 0x7f875c0acdf0 2023-08-09T15:08:28.287 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:28.285+0000 7f8762514640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f875c1f6080 con 0x7f875c0c8b30 2023-08-09T15:08:28.287 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:28.286+0000 7f8760d11640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f875c0c8750 0x7f875c1f1c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T15:08:28.288 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:28.286+0000 7f8760d11640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f875c0c8750 0x7f875c1f1c10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:36036/0 (socket says 172.21.15.43:36036) 2023-08-09T15:08:28.288 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:28.286+0000 7f8760d11640 1 -- 172.21.15.43:0/451110843 learned_addr learned my addr 172.21.15.43:0/451110843 (peer_addr_for_me v2:172.21.15.43:0/0) 2023-08-09T15:08:28.289 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:28.286+0000 7f8761512640 1 --2- 172.21.15.43:0/451110843 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f875c0acdf0 0x7f875c1f2ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T15:08:28.289 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:28.286+0000 7f8761d13640 1 --2- 172.21.15.43:0/451110843 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f875c0c8b30 0x7f875c1f2150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T15:08:28.289 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:28.286+0000 7f8760d11640 1 -- 172.21.15.43:0/451110843 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f875c0c8b30 msgr2=0x7f875c1f2150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T15:08:28.289 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:28.286+0000 7f8760d11640 1 --2- 172.21.15.43:0/451110843 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f875c0c8b30 0x7f875c1f2150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T15:08:28.290 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:28.286+0000 7f8760d11640 1 -- 172.21.15.43:0/451110843 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f875c0acdf0 msgr2=0x7f875c1f2ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T15:08:28.290 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:28.286+0000 7f8760d11640 1 --2- 172.21.15.43:0/451110843 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f875c0acdf0 0x7f875c1f2ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T15:08:28.290 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:28.286+0000 7f8760d11640 1 -- 172.21.15.43:0/451110843 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8758009400 con 0x7f875c0c8750 2023-08-09T15:08:28.290 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:28.286+0000 7f8761d13640 1 --2- 172.21.15.43:0/451110843 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f875c0c8b30 0x7f875c1f2150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-08-09T15:08:28.290 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:28.286+0000 7f8760d11640 1 --2- 172.21.15.43:0/451110843 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f875c0c8750 0x7f875c1f1c10 secure :-1 s=READY pgs=2133 cs=0 l=1 rev1=1 crypto rx=0x7f874c00b3f0 tx=0x7f874c00b8b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-09T15:08:28.290 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:28.287+0000 7f8751ffb640 1 -- 172.21.15.43:0/451110843 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f874c011830 con 0x7f875c0c8750 2023-08-09T15:08:28.290 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:28.287+0000 7f8751ffb640 1 -- 172.21.15.43:0/451110843 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f874c011990 con 0x7f875c0c8750 2023-08-09T15:08:28.291 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:28.287+0000 7f8751ffb640 1 -- 172.21.15.43:0/451110843 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f874c011d40 con 0x7f875c0c8750 2023-08-09T15:08:28.291 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:28.287+0000 7f8762514640 1 -- 172.21.15.43:0/451110843 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f875c1f6360 con 0x7f875c0c8750 2023-08-09T15:08:28.291 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:28.287+0000 7f8762514640 1 -- 172.21.15.43:0/451110843 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f875c1f6920 con 0x7f875c0c8750 2023-08-09T15:08:28.292 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:28.288+0000 7f8762514640 1 -- 172.21.15.43:0/451110843 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f875c0c9230 con 0x7f875c0c8750 2023-08-09T15:08:28.292 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:28.289+0000 7f8751ffb640 1 -- 172.21.15.43:0/451110843 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 105) v1 ==== 88387+0+0 (secure 0 0 0) 0x7f874c01a400 con 0x7f875c0c8750 2023-08-09T15:08:28.292 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:28.290+0000 7f8751ffb640 1 --2- 172.21.15.43:0/451110843 >> [v2:172.21.15.203:6832/3082732769,v1:172.21.15.203:6833/3082732769] conn(0x7f874006a0e0 0x7f874006c5a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T15:08:28.292 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:28.290+0000 7f8751ffb640 1 -- 172.21.15.43:0/451110843 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(193..193 src has 1..193) v4 ==== 7486+0+0 (secure 0 0 0) 0x7f874c088ab0 con 0x7f875c0c8750 2023-08-09T15:08:28.293 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:28.290+0000 7f8761512640 1 --2- 172.21.15.43:0/451110843 >> [v2:172.21.15.203:6832/3082732769,v1:172.21.15.203:6833/3082732769] conn(0x7f874006a0e0 0x7f874006c5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T15:08:28.293 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:28.292+0000 7f8761512640 1 --2- 172.21.15.43:0/451110843 >> [v2:172.21.15.203:6832/3082732769,v1:172.21.15.203:6833/3082732769] conn(0x7f874006a0e0 0x7f874006c5a0 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7f8758036000 tx=0x7f8758008030 comp rx=0 tx=0).ready entity=mgr.8500 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-09T15:08:28.296 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:28.294+0000 7f8751ffb640 1 -- 172.21.15.43:0/451110843 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178066 (secure 0 0 0) 0x7f874c053b20 con 0x7f875c0c8750 2023-08-09T15:08:28.523 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:28.522+0000 7f8762514640 1 -- 172.21.15.43:0/451110843 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f875c0739f0 con 0x7f875c0c8750 2023-08-09T15:08:28.524 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:28.523+0000 7f8751ffb640 1 -- 172.21.15.43:0/451110843 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 87 v87) v1 ==== 94+0+4882 (secure 0 0 0) 0x7f874c015090 con 0x7f875c0c8750 2023-08-09T15:08:28.525 INFO:teuthology.orchestra.run.smithi043.stdout: 2023-08-09T15:08:28.525 INFO:teuthology.orchestra.run.smithi043.stdout:{"epoch":87,"default_fscid":10,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":7328,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.203:6836/3701396136","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6834","nonce":3701396136},{"type":"v1","addr":"172.21.15.203:6836","nonce":3701396136}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":82},{"gid":7343,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.43:6835/2096611841","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6833","nonce":2096611841},{"type":"v1","addr":"172.21.15.43:6835","nonce":2096611841}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":82},{"gid":8242,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.203:6837/18754135","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6835","nonce":18754135},{"type":"v1","addr":"172.21.15.203:6837","nonce":18754135}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":81}],"filesystems":[{"mdsmap":{"epoch":87,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-08-09T14:52:30.188965+0000","modified":"2023-08-09T14:52:33.772528+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":10185},"failed":[],"damaged":[],"stopped":[],"info":{"gid_10185":{"gid":10185,"name":"c","rank":0,"incarnation":83,"state":"up:active","state_seq":3,"addr":"172.21.15.43:6834/2866936881","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6832","nonce":2866936881},{"type":"v1","addr":"172.21.15.43:6834","nonce":2866936881}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[23],"metadata_pool":22,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":10}]} 2023-08-09T15:08:28.525 INFO:teuthology.orchestra.run.smithi043.stderr:dumped fsmap epoch 87 2023-08-09T15:08:28.527 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:28.526+0000 7f8762514640 1 -- 172.21.15.43:0/451110843 >> [v2:172.21.15.203:6832/3082732769,v1:172.21.15.203:6833/3082732769] conn(0x7f874006a0e0 msgr2=0x7f874006c5a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T15:08:28.527 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:28.526+0000 7f8762514640 1 --2- 172.21.15.43:0/451110843 >> [v2:172.21.15.203:6832/3082732769,v1:172.21.15.203:6833/3082732769] conn(0x7f874006a0e0 0x7f874006c5a0 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7f8758036000 tx=0x7f8758008030 comp rx=0 tx=0).stop 2023-08-09T15:08:28.528 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:28.526+0000 7f8762514640 1 -- 172.21.15.43:0/451110843 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f875c0c8750 msgr2=0x7f875c1f1c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T15:08:28.528 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:28.526+0000 7f8762514640 1 --2- 172.21.15.43:0/451110843 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f875c0c8750 0x7f875c1f1c10 secure :-1 s=READY pgs=2133 cs=0 l=1 rev1=1 crypto rx=0x7f874c00b3f0 tx=0x7f874c00b8b0 comp rx=0 tx=0).stop 2023-08-09T15:08:28.528 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:28.526+0000 7f8762514640 1 -- 172.21.15.43:0/451110843 shutdown_connections 2023-08-09T15:08:28.528 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:28.526+0000 7f8762514640 1 --2- 172.21.15.43:0/451110843 >> [v2:172.21.15.203:6832/3082732769,v1:172.21.15.203:6833/3082732769] conn(0x7f874006a0e0 0x7f874006c5a0 unknown :-1 s=CLOSED pgs=26 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T15:08:28.528 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:28.526+0000 7f8762514640 1 --2- 172.21.15.43:0/451110843 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f875c0c8b30 0x7f875c1f2150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T15:08:28.528 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:28.526+0000 7f8762514640 1 --2- 172.21.15.43:0/451110843 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f875c0c8750 0x7f875c1f1c10 unknown :-1 s=CLOSED pgs=2133 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T15:08:28.528 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:28.526+0000 7f8762514640 1 --2- 172.21.15.43:0/451110843 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f875c0acdf0 0x7f875c1f2ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T15:08:28.529 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:28.526+0000 7f8762514640 1 -- 172.21.15.43:0/451110843 >> 172.21.15.43:0/451110843 conn(0x7f875c0cde90 msgr2=0x7f875c159500 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-09T15:08:28.529 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:28.526+0000 7f8762514640 1 -- 172.21.15.43:0/451110843 shutdown_connections 2023-08-09T15:08:28.529 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:28.526+0000 7f8762514640 1 -- 172.21.15.43:0/451110843 wait complete. 2023-08-09T15:08:28.547 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-08-09T15:08:28.776 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:28.774+0000 7f0bc5e7c640 1 Processor -- start 2023-08-09T15:08:28.776 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:28.775+0000 7f0bc5e7c640 1 -- start start 2023-08-09T15:08:28.779 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:28.776+0000 7f0bc5e7c640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f0bc00c86f0 0x7f0bc00c8b50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T15:08:28.780 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:28.776+0000 7f0bc5e7c640 1 -- --> v1:172.21.15.43:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0bc00c9090 con 0x7f0bc0143c20 2023-08-09T15:08:28.780 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:28.776+0000 7f0bc5e7c640 1 -- --> v1:172.21.15.203:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0bc006c440 con 0x7f0bc006c8d0 2023-08-09T15:08:28.780 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:28.776+0000 7f0bc5e7c640 1 -- --> v2:172.21.15.43:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0bc0144000 con 0x7f0bc00c86f0 2023-08-09T15:08:28.781 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:28.777+0000 7f0bb7fff640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f0bc00c86f0 0x7f0bc00c8b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T15:08:28.781 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:28.777+0000 7f0bb7fff640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f0bc00c86f0 0x7f0bc00c8b50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:36050/0 (socket says 172.21.15.43:36050) 2023-08-09T15:08:28.781 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:28.777+0000 7f0bb7fff640 1 -- 172.21.15.43:0/2307487440 learned_addr learned my addr 172.21.15.43:0/2307487440 (peer_addr_for_me v2:172.21.15.43:0/0) 2023-08-09T15:08:28.781 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:28.777+0000 7f0bb7fff640 1 -- 172.21.15.43:0/2307487440 >> v1:172.21.15.203:6789/0 conn(0x7f0bc006c8d0 legacy=0x7f0bc006cd00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-09T15:08:28.782 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:28.777+0000 7f0bb7fff640 1 -- 172.21.15.43:0/2307487440 >> v1:172.21.15.43:6789/0 conn(0x7f0bc0143c20 legacy=0x7f0bc006f540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T15:08:28.782 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:28.777+0000 7f0bb7fff640 1 -- 172.21.15.43:0/2307487440 --> v2:172.21.15.43:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0bc006fc50 con 0x7f0bc00c86f0 2023-08-09T15:08:28.782 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:28.778+0000 7f0bb7fff640 1 --2- 172.21.15.43:0/2307487440 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f0bc00c86f0 0x7f0bc00c8b50 secure :-1 s=READY pgs=2135 cs=0 l=1 rev1=1 crypto rx=0x7f0ba8009870 tx=0x7f0ba802b880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e29f1eeaeed4a9e0 server_cookie=0 in_seq=0 out_seq=0 2023-08-09T15:08:28.787 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:28.778+0000 7f0bbeffd640 1 -- 172.21.15.43:0/2307487440 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0ba80317e0 con 0x7f0bc00c86f0 2023-08-09T15:08:28.788 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:28.779+0000 7f0bbeffd640 1 -- 172.21.15.43:0/2307487440 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0ba8031940 con 0x7f0bc00c86f0 2023-08-09T15:08:28.788 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:28.779+0000 7f0bbeffd640 1 -- 172.21.15.43:0/2307487440 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0ba8031cd0 con 0x7f0bc00c86f0 2023-08-09T15:08:28.788 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:28.779+0000 7f0bc5e7c640 1 -- 172.21.15.43:0/2307487440 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f0bc00c86f0 msgr2=0x7f0bc00c8b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T15:08:28.789 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:28.779+0000 7f0bc5e7c640 1 --2- 172.21.15.43:0/2307487440 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f0bc00c86f0 0x7f0bc00c8b50 secure :-1 s=READY pgs=2135 cs=0 l=1 rev1=1 crypto rx=0x7f0ba8009870 tx=0x7f0ba802b880 comp rx=0 tx=0).stop 2023-08-09T15:08:28.789 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:28.779+0000 7f0bc5e7c640 1 -- 172.21.15.43:0/2307487440 shutdown_connections 2023-08-09T15:08:28.789 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:28.779+0000 7f0bc5e7c640 1 --2- 172.21.15.43:0/2307487440 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f0bc00c86f0 0x7f0bc00c8b50 unknown :-1 s=CLOSED pgs=2135 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T15:08:28.789 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:28.779+0000 7f0bc5e7c640 1 -- 172.21.15.43:0/2307487440 >> 172.21.15.43:0/2307487440 conn(0x7f0bc00ce0a0 msgr2=0x7f0bc00ce480 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-09T15:08:28.790 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:28.779+0000 7f0bc5e7c640 1 -- 172.21.15.43:0/2307487440 shutdown_connections 2023-08-09T15:08:28.790 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:28.779+0000 7f0bc5e7c640 1 -- 172.21.15.43:0/2307487440 wait complete. 2023-08-09T15:08:28.790 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:28.781+0000 7f0bc5e7c640 1 Processor -- start 2023-08-09T15:08:28.791 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:28.782+0000 7f0bc5e7c640 1 -- start start 2023-08-09T15:08:28.791 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:28.783+0000 7f0bc5e7c640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f0bc006c8d0 0x7f0bc013ff80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T15:08:28.791 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:28.783+0000 7f0bc5e7c640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f0bc00c86f0 0x7f0bc00b22a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T15:08:28.791 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:28.783+0000 7f0bc5e7c640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f0bc0143c20 0x7f0bc00c3ba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T15:08:28.792 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:28.783+0000 7f0bc5e7c640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0bc00b37b0 con 0x7f0bc00c86f0 2023-08-09T15:08:28.792 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:28.783+0000 7f0bc5e7c640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0bc01404c0 con 0x7f0bc006c8d0 2023-08-09T15:08:28.792 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:28.783+0000 7f0bc5e7c640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0bc00b2ba0 con 0x7f0bc0143c20 2023-08-09T15:08:28.792 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:28.783+0000 7f0bb7fff640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f0bc00c86f0 0x7f0bc00b22a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T15:08:28.793 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:28.783+0000 7f0bb7fff640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f0bc00c86f0 0x7f0bc00b22a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:36060/0 (socket says 172.21.15.43:36060) 2023-08-09T15:08:28.793 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:28.783+0000 7f0bb7fff640 1 -- 172.21.15.43:0/4066073970 learned_addr learned my addr 172.21.15.43:0/4066073970 (peer_addr_for_me v2:172.21.15.43:0/0) 2023-08-09T15:08:28.793 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:28.783+0000 7f0bbf7fe640 1 --2- 172.21.15.43:0/4066073970 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f0bc006c8d0 0x7f0bc013ff80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T15:08:28.794 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:28.784+0000 7f0bb7fff640 1 -- 172.21.15.43:0/4066073970 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f0bc0143c20 msgr2=0x7f0bc00c3ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T15:08:28.794 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:28.784+0000 7f0bb7fff640 1 --2- 172.21.15.43:0/4066073970 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f0bc0143c20 0x7f0bc00c3ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T15:08:28.794 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:28.784+0000 7f0bb7fff640 1 -- 172.21.15.43:0/4066073970 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f0bc006c8d0 msgr2=0x7f0bc013ff80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T15:08:28.795 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:28.784+0000 7f0bb7fff640 1 --2- 172.21.15.43:0/4066073970 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f0bc006c8d0 0x7f0bc013ff80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T15:08:28.795 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:28.784+0000 7f0bb7fff640 1 -- 172.21.15.43:0/4066073970 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0ba80094d0 con 0x7f0bc00c86f0 2023-08-09T15:08:28.795 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:28.784+0000 7f0bbf7fe640 1 --2- 172.21.15.43:0/4066073970 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f0bc006c8d0 0x7f0bc013ff80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-08-09T15:08:28.796 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:28.784+0000 7f0bb7fff640 1 --2- 172.21.15.43:0/4066073970 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f0bc00c86f0 0x7f0bc00b22a0 secure :-1 s=READY pgs=2136 cs=0 l=1 rev1=1 crypto rx=0x7f0ba8000c00 tx=0x7f0ba80046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-09T15:08:28.796 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:28.784+0000 7f0bbcff9640 1 -- 172.21.15.43:0/4066073970 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0ba8032980 con 0x7f0bc00c86f0 2023-08-09T15:08:28.796 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:28.784+0000 7f0bbcff9640 1 -- 172.21.15.43:0/4066073970 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0ba8032b00 con 0x7f0bc00c86f0 2023-08-09T15:08:28.797 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:28.784+0000 7f0bbcff9640 1 -- 172.21.15.43:0/4066073970 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0ba80453f0 con 0x7f0bc00c86f0 2023-08-09T15:08:28.797 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:28.784+0000 7f0bc5e7c640 1 -- 172.21.15.43:0/4066073970 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0bc00b2e80 con 0x7f0bc00c86f0 2023-08-09T15:08:28.797 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:28.785+0000 7f0bc5e7c640 1 -- 172.21.15.43:0/4066073970 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0bc00b3440 con 0x7f0bc00c86f0 2023-08-09T15:08:28.797 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:28.786+0000 7f0bc5e7c640 1 -- 172.21.15.43:0/4066073970 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0bc00739f0 con 0x7f0bc00c86f0 2023-08-09T15:08:28.799 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:28.787+0000 7f0bbcff9640 1 -- 172.21.15.43:0/4066073970 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 105) v1 ==== 88387+0+0 (secure 0 0 0) 0x7f0ba804d3f0 con 0x7f0bc00c86f0 2023-08-09T15:08:28.799 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:28.788+0000 7f0bbcff9640 1 --2- 172.21.15.43:0/4066073970 >> [v2:172.21.15.203:6832/3082732769,v1:172.21.15.203:6833/3082732769] conn(0x7f0ba006a0e0 0x7f0ba006c5a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T15:08:28.800 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:28.788+0000 7f0bbcff9640 1 -- 172.21.15.43:0/4066073970 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(193..193 src has 1..193) v4 ==== 7486+0+0 (secure 0 0 0) 0x7f0ba80aad00 con 0x7f0bc00c86f0 2023-08-09T15:08:28.800 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:28.788+0000 7f0bbf7fe640 1 --2- 172.21.15.43:0/4066073970 >> [v2:172.21.15.203:6832/3082732769,v1:172.21.15.203:6833/3082732769] conn(0x7f0ba006a0e0 0x7f0ba006c5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T15:08:28.800 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:28.790+0000 7f0bbf7fe640 1 --2- 172.21.15.43:0/4066073970 >> [v2:172.21.15.203:6832/3082732769,v1:172.21.15.203:6833/3082732769] conn(0x7f0ba006a0e0 0x7f0ba006c5a0 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7f0bb0000f60 tx=0x7f0bb0009040 comp rx=0 tx=0).ready entity=mgr.8500 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-09T15:08:28.801 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:28.792+0000 7f0bbcff9640 1 -- 172.21.15.43:0/4066073970 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178066 (secure 0 0 0) 0x7f0ba8075d10 con 0x7f0bc00c86f0 2023-08-09T15:08:28.988 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:28.987+0000 7f0bc5e7c640 1 -- 172.21.15.43:0/4066073970 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f0bc00739f0 con 0x7f0bc00c86f0 2023-08-09T15:08:28.991 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:28.989+0000 7f0bbcff9640 1 -- 172.21.15.43:0/4066073970 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v193) v1 ==== 74+0+20146 (secure 0 0 0) 0x7f0ba8079330 con 0x7f0bc00c86f0 2023-08-09T15:08:28.991 INFO:teuthology.orchestra.run.smithi043.stdout: 2023-08-09T15:08:28.991 INFO:teuthology.orchestra.run.smithi043.stdout:{"epoch":193,"fsid":"daac87ee-1337-4250-a955-aea4e408a39b","created":"2023-08-09T13:33:41.701198+0000","modified":"2023-08-09T15:08:25.567949+0000","last_up_change":"2023-08-09T13:33:53.783453+0000","last_in_change":"2023-08-09T13:33:45.148904+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":23,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":"rbd","create_time":"2023-08-09T13:33:54.365137+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":".mgr","create_time":"2023-08-09T13:33:54.831803+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":22,"pool_name":"cephfs_metadata","create_time":"2023-08-09T14:52:28.569334+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"185","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":23,"pool_name":"cephfs_data","create_time":"2023-08-09T14:52:29.209973+0000","flags":8193,"flags_names":"hashpspool,selfmanaged_snaps","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"192","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":33,"snap_epoch":192,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"cbd49ecd-e896-4fbd-864e-5c37a829b165","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":184,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6800","nonce":502667335},{"type":"v1","addr":"172.21.15.43:6801","nonce":502667335}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6802","nonce":502667335},{"type":"v1","addr":"172.21.15.43:6803","nonce":502667335}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6806","nonce":502667335},{"type":"v1","addr":"172.21.15.43:6807","nonce":502667335}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6804","nonce":502667335},{"type":"v1","addr":"172.21.15.43:6805","nonce":502667335}]},"public_addr":"172.21.15.43:6801/502667335","cluster_addr":"172.21.15.43:6803/502667335","heartbeat_back_addr":"172.21.15.43:6807/502667335","heartbeat_front_addr":"172.21.15.43:6805/502667335","state":["exists","up"]},{"osd":1,"uuid":"75c7e0fe-1ec7-4fe8-a2f2-3661a65b8500","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":184,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6808","nonce":3289134610},{"type":"v1","addr":"172.21.15.43:6809","nonce":3289134610}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6810","nonce":3289134610},{"type":"v1","addr":"172.21.15.43:6812","nonce":3289134610}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6821","nonce":3289134610},{"type":"v1","addr":"172.21.15.43:6824","nonce":3289134610}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6815","nonce":3289134610},{"type":"v1","addr":"172.21.15.43:6818","nonce":3289134610}]},"public_addr":"172.21.15.43:6809/3289134610","cluster_addr":"172.21.15.43:6812/3289134610","heartbeat_back_addr":"172.21.15.43:6824/3289134610","heartbeat_front_addr":"172.21.15.43:6818/3289134610","state":["exists","up"]},{"osd":2,"uuid":"bb5c41e9-7f5f-4125-9b01-3bb8bfc2ea83","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":184,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6811","nonce":4079709736},{"type":"v1","addr":"172.21.15.43:6814","nonce":4079709736}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6817","nonce":4079709736},{"type":"v1","addr":"172.21.15.43:6820","nonce":4079709736}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6829","nonce":4079709736},{"type":"v1","addr":"172.21.15.43:6831","nonce":4079709736}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6823","nonce":4079709736},{"type":"v1","addr":"172.21.15.43:6826","nonce":4079709736}]},"public_addr":"172.21.15.43:6814/4079709736","cluster_addr":"172.21.15.43:6820/4079709736","heartbeat_back_addr":"172.21.15.43:6831/4079709736","heartbeat_front_addr":"172.21.15.43:6826/4079709736","state":["exists","up"]},{"osd":3,"uuid":"bfe4c8b4-694a-465c-87e6-2b2aab852613","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":184,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6813","nonce":1709538759},{"type":"v1","addr":"172.21.15.43:6816","nonce":1709538759}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6819","nonce":1709538759},{"type":"v1","addr":"172.21.15.43:6822","nonce":1709538759}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6828","nonce":1709538759},{"type":"v1","addr":"172.21.15.43:6830","nonce":1709538759}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6825","nonce":1709538759},{"type":"v1","addr":"172.21.15.43:6827","nonce":1709538759}]},"public_addr":"172.21.15.43:6816/1709538759","cluster_addr":"172.21.15.43:6822/1709538759","heartbeat_back_addr":"172.21.15.43:6830/1709538759","heartbeat_front_addr":"172.21.15.43:6827/1709538759","state":["exists","up"]},{"osd":4,"uuid":"fa2e8a77-aba1-4756-853e-0d174a5bdeba","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":184,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6800","nonce":4196866481},{"type":"v1","addr":"172.21.15.203:6801","nonce":4196866481}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6802","nonce":4196866481},{"type":"v1","addr":"172.21.15.203:6803","nonce":4196866481}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6806","nonce":4196866481},{"type":"v1","addr":"172.21.15.203:6807","nonce":4196866481}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6804","nonce":4196866481},{"type":"v1","addr":"172.21.15.203:6805","nonce":4196866481}]},"public_addr":"172.21.15.203:6801/4196866481","cluster_addr":"172.21.15.203:6803/4196866481","heartbeat_back_addr":"172.21.15.203:6807/4196866481","heartbeat_front_addr":"172.21.15.203:6805/4196866481","state":["exists","up"]},{"osd":5,"uuid":"81811e4b-4d24-4f51-a82c-f1d700a2a00d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":184,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6816","nonce":3059290081},{"type":"v1","addr":"172.21.15.203:6817","nonce":3059290081}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6818","nonce":3059290081},{"type":"v1","addr":"172.21.15.203:6819","nonce":3059290081}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6822","nonce":3059290081},{"type":"v1","addr":"172.21.15.203:6823","nonce":3059290081}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6820","nonce":3059290081},{"type":"v1","addr":"172.21.15.203:6821","nonce":3059290081}]},"public_addr":"172.21.15.203:6817/3059290081","cluster_addr":"172.21.15.203:6819/3059290081","heartbeat_back_addr":"172.21.15.203:6823/3059290081","heartbeat_front_addr":"172.21.15.203:6821/3059290081","state":["exists","up"]},{"osd":6,"uuid":"307bac4d-6a5b-443d-af75-56d695d7674f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":184,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6824","nonce":512155569},{"type":"v1","addr":"172.21.15.203:6825","nonce":512155569}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6826","nonce":512155569},{"type":"v1","addr":"172.21.15.203:6827","nonce":512155569}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6830","nonce":512155569},{"type":"v1","addr":"172.21.15.203:6831","nonce":512155569}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6828","nonce":512155569},{"type":"v1","addr":"172.21.15.203:6829","nonce":512155569}]},"public_addr":"172.21.15.203:6825/512155569","cluster_addr":"172.21.15.203:6827/512155569","heartbeat_back_addr":"172.21.15.203:6831/512155569","heartbeat_front_addr":"172.21.15.203:6829/512155569","state":["exists","up"]},{"osd":7,"uuid":"a95dc7ca-9f1b-4295-af70-68a22a2d7d5e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":184,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6808","nonce":3331450438},{"type":"v1","addr":"172.21.15.203:6809","nonce":3331450438}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6810","nonce":3331450438},{"type":"v1","addr":"172.21.15.203:6811","nonce":3331450438}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6814","nonce":3331450438},{"type":"v1","addr":"172.21.15.203:6815","nonce":3331450438}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6812","nonce":3331450438},{"type":"v1","addr":"172.21.15.203:6813","nonce":3331450438}]},"public_addr":"172.21.15.203:6809/3331450438","cluster_addr":"172.21.15.203:6811/3331450438","heartbeat_back_addr":"172.21.15.203:6815/3331450438","heartbeat_front_addr":"172.21.15.203:6813/3331450438","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-09T13:33:50.547652+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-09T13:33:50.589469+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-09T13:33:50.561724+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-09T13:33:50.599650+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-09T13:33:50.411855+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-09T13:33:50.219681+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-09T13:33:50.504664+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-09T13:33:50.441248+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.203:0/1820136184":"2023-08-10T15:08:25.567876+0000","172.21.15.203:6833/115830229":"2023-08-10T15:08:25.567876+0000","172.21.15.203:6832/115830229":"2023-08-10T15:08:25.567876+0000","172.21.15.203:0/3317788314":"2023-08-10T15:08:25.567876+0000","172.21.15.203:0/69497804":"2023-08-10T15:03:21.353235+0000","172.21.15.203:0/169967264":"2023-08-10T15:03:21.353235+0000","172.21.15.203:6833/40883597":"2023-08-10T15:03:21.353235+0000","172.21.15.203:0/1738813745":"2023-08-10T15:08:25.567876+0000","172.21.15.203:0/1306452763":"2023-08-10T15:08:25.567876+0000","172.21.15.203:6832/40883597":"2023-08-10T15:03:21.353235+0000","172.21.15.203:0/1225284485":"2023-08-10T15:08:25.567876+0000","172.21.15.203:0/3690197627":"2023-08-10T15:03:21.353235+0000","172.21.15.203:0/1082753911":"2023-08-10T15:03:21.353235+0000","172.21.15.203:6832/2959400181":"2023-08-10T14:52:45.188641+0000","172.21.15.203:0/82655847":"2023-08-10T15:03:21.353235+0000","172.21.15.203:0/1935803093":"2023-08-10T14:52:45.188641+0000","172.21.15.203:0/3150685329":"2023-08-10T14:52:45.188641+0000","172.21.15.203:0/2219953609":"2023-08-10T15:03:21.353235+0000","172.21.15.203:0/3429376127":"2023-08-10T14:52:45.188641+0000","172.21.15.203:0/1394945671":"2023-08-10T15:08:25.567876+0000","172.21.15.203:0/2197573025":"2023-08-10T14:52:45.188641+0000","172.21.15.203:0/2458603841":"2023-08-10T14:52:45.188641+0000","172.21.15.203:6833/2959400181":"2023-08-10T14:52:45.188641+0000","172.21.15.203:0/4177046277":"2023-08-10T14:52:45.188641+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":7,"snaps":[{"begin":104,"length":1},{"begin":221,"length":1}]},{"pool":23,"snaps":[{"begin":2,"length":32}]}],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-08-09T15:08:28.993 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:28.992+0000 7f0bc5e7c640 1 -- 172.21.15.43:0/4066073970 >> [v2:172.21.15.203:6832/3082732769,v1:172.21.15.203:6833/3082732769] conn(0x7f0ba006a0e0 msgr2=0x7f0ba006c5a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T15:08:28.994 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:28.992+0000 7f0bc5e7c640 1 --2- 172.21.15.43:0/4066073970 >> [v2:172.21.15.203:6832/3082732769,v1:172.21.15.203:6833/3082732769] conn(0x7f0ba006a0e0 0x7f0ba006c5a0 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7f0bb0000f60 tx=0x7f0bb0009040 comp rx=0 tx=0).stop 2023-08-09T15:08:28.994 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:28.992+0000 7f0bc5e7c640 1 -- 172.21.15.43:0/4066073970 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f0bc00c86f0 msgr2=0x7f0bc00b22a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T15:08:28.994 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:28.992+0000 7f0bc5e7c640 1 --2- 172.21.15.43:0/4066073970 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f0bc00c86f0 0x7f0bc00b22a0 secure :-1 s=READY pgs=2136 cs=0 l=1 rev1=1 crypto rx=0x7f0ba8000c00 tx=0x7f0ba80046c0 comp rx=0 tx=0).stop 2023-08-09T15:08:28.994 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:28.992+0000 7f0bc5e7c640 1 -- 172.21.15.43:0/4066073970 shutdown_connections 2023-08-09T15:08:28.994 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:28.992+0000 7f0bc5e7c640 1 --2- 172.21.15.43:0/4066073970 >> [v2:172.21.15.203:6832/3082732769,v1:172.21.15.203:6833/3082732769] conn(0x7f0ba006a0e0 0x7f0ba006c5a0 unknown :-1 s=CLOSED pgs=27 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T15:08:28.994 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:28.992+0000 7f0bc5e7c640 1 --2- 172.21.15.43:0/4066073970 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f0bc0143c20 0x7f0bc00c3ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T15:08:28.994 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:28.992+0000 7f0bc5e7c640 1 --2- 172.21.15.43:0/4066073970 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f0bc00c86f0 0x7f0bc00b22a0 unknown :-1 s=CLOSED pgs=2136 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T15:08:28.995 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:28.992+0000 7f0bc5e7c640 1 --2- 172.21.15.43:0/4066073970 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f0bc006c8d0 0x7f0bc013ff80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T15:08:28.995 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:28.992+0000 7f0bc5e7c640 1 -- 172.21.15.43:0/4066073970 >> 172.21.15.43:0/4066073970 conn(0x7f0bc00ce0a0 msgr2=0x7f0bc00c3200 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-09T15:08:28.995 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:28.992+0000 7f0bc5e7c640 1 -- 172.21.15.43:0/4066073970 shutdown_connections 2023-08-09T15:08:28.995 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:28.992+0000 7f0bc5e7c640 1 -- 172.21.15.43:0/4066073970 wait complete. 2023-08-09T15:08:29.013 INFO:tasks.cephfs.filesystem:Destroying file system cephfs and related pools 2023-08-09T15:08:29.014 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-08-09T15:08:29.242 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:29.241+0000 7f06536fa640 1 Processor -- start 2023-08-09T15:08:29.243 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:29.241+0000 7f06536fa640 1 -- start start 2023-08-09T15:08:29.244 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:29.243+0000 7f06536fa640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f064c14a5c0 0x7f064c160f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T15:08:29.244 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:29.243+0000 7f06536fa640 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7f064c0cc380 0x7f064c06bb50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T15:08:29.245 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:29.243+0000 7f06536fa640 1 -- --> v1:172.21.15.43:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f064c0c38a0 con 0x7f064c0c3130 2023-08-09T15:08:29.245 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:29.243+0000 7f06536fa640 1 -- --> v2:172.21.15.203:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f064c0c4420 con 0x7f064c0cc380 2023-08-09T15:08:29.245 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:29.243+0000 7f06536fa640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f064c0c3b80 con 0x7f064c14a5c0 2023-08-09T15:08:29.245 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:29.244+0000 7f0650c95640 1 --1- >> v1:172.21.15.43:6789/0 conn(0x7f064c0c3130 0x7f064c0cbc70 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.43:6789/0 says I am v1:172.21.15.43:41740/0 (socket says 172.21.15.43:41740) 2023-08-09T15:08:29.246 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:29.244+0000 7f0650c95640 1 -- 172.21.15.43:0/3105073981 learned_addr learned my addr 172.21.15.43:0/3105073981 (peer_addr_for_me v1:172.21.15.43:0/0) 2023-08-09T15:08:29.246 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:29.244+0000 7f0651496640 1 --2- 172.21.15.43:0/3105073981 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f064c14a5c0 0x7f064c160f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T15:08:29.246 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:29.244+0000 7f0651c97640 1 --2- 172.21.15.43:0/3105073981 >> v2:172.21.15.203:3300/0 conn(0x7f064c0cc380 0x7f064c06bb50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T15:08:29.246 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:29.244+0000 7f0643fff640 1 -- 172.21.15.43:0/3105073981 <== mon.0 v1:172.21.15.43:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2831482863 0 0) 0x7f064c0c38a0 con 0x7f064c0c3130 2023-08-09T15:08:29.246 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:29.244+0000 7f0643fff640 1 -- 172.21.15.43:0/3105073981 --> v1:172.21.15.43:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f0638003520 con 0x7f064c0c3130 2023-08-09T15:08:29.246 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:29.245+0000 7f0651496640 1 -- 172.21.15.43:0/3105073981 >> v2:172.21.15.203:3300/0 conn(0x7f064c0cc380 msgr2=0x7f064c06bb50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-09T15:08:29.247 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:29.245+0000 7f0651496640 1 --2- 172.21.15.43:0/3105073981 >> v2:172.21.15.203:3300/0 conn(0x7f064c0cc380 0x7f064c06bb50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T15:08:29.247 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:29.245+0000 7f0651496640 1 -- 172.21.15.43:0/3105073981 >> v1:172.21.15.43:6789/0 conn(0x7f064c0c3130 legacy=0x7f064c0cbc70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T15:08:29.247 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:29.245+0000 7f0643fff640 1 -- 172.21.15.43:0/3105073981 <== mon.0 v1:172.21.15.43:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 303918166 0 0) 0x7f0638003520 con 0x7f064c0c3130 2023-08-09T15:08:29.248 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:29.245+0000 7f0651496640 1 -- 172.21.15.43:0/3105073981 --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f064c0c3e60 con 0x7f064c14a5c0 2023-08-09T15:08:29.248 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:29.246+0000 7f0651496640 1 --2- 172.21.15.43:0/3105073981 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f064c14a5c0 0x7f064c160f20 secure :-1 s=READY pgs=308 cs=0 l=1 rev1=1 crypto rx=0x7f06480097a0 tx=0x7f064802b710 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a9e51b35d9293b12 server_cookie=0 in_seq=0 out_seq=0 2023-08-09T15:08:29.248 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:29.246+0000 7f0643fff640 1 -- 172.21.15.43:0/3105073981 <== mon.2 v2:172.21.15.203:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f063c001670 con 0x7f064c14a5c0 2023-08-09T15:08:29.249 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:29.247+0000 7f0643fff640 1 -- 172.21.15.43:0/3105073981 <== mon.2 v2:172.21.15.203:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f064802e050 con 0x7f064c14a5c0 2023-08-09T15:08:29.249 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:29.247+0000 7f0643fff640 1 -- 172.21.15.43:0/3105073981 <== mon.2 v2:172.21.15.203:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0648030710 con 0x7f064c14a5c0 2023-08-09T15:08:29.250 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:29.247+0000 7f06536fa640 1 -- 172.21.15.43:0/3105073981 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f064c14a5c0 msgr2=0x7f064c160f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T15:08:29.250 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:29.247+0000 7f06536fa640 1 --2- 172.21.15.43:0/3105073981 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f064c14a5c0 0x7f064c160f20 secure :-1 s=READY pgs=308 cs=0 l=1 rev1=1 crypto rx=0x7f06480097a0 tx=0x7f064802b710 comp rx=0 tx=0).stop 2023-08-09T15:08:29.250 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:29.247+0000 7f06536fa640 1 -- 172.21.15.43:0/3105073981 shutdown_connections 2023-08-09T15:08:29.250 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:29.247+0000 7f06536fa640 1 --2- 172.21.15.43:0/3105073981 >> v2:172.21.15.203:3300/0 conn(0x7f064c0cc380 0x7f064c06bb50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T15:08:29.250 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:29.247+0000 7f06536fa640 1 --2- 172.21.15.43:0/3105073981 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f064c14a5c0 0x7f064c160f20 unknown :-1 s=CLOSED pgs=308 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T15:08:29.251 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:29.248+0000 7f06536fa640 1 -- 172.21.15.43:0/3105073981 >> 172.21.15.43:0/3105073981 conn(0x7f064c0ce0a0 msgr2=0x7f064c0ce480 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-09T15:08:29.251 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:29.248+0000 7f06536fa640 1 -- 172.21.15.43:0/3105073981 shutdown_connections 2023-08-09T15:08:29.251 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:29.248+0000 7f06536fa640 1 -- 172.21.15.43:0/3105073981 wait complete. 2023-08-09T15:08:29.251 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:29.250+0000 7f06536fa640 1 Processor -- start 2023-08-09T15:08:29.252 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:29.250+0000 7f06536fa640 1 -- start start 2023-08-09T15:08:29.252 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:29.251+0000 7f06536fa640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f064c0c3130 0x7f064c1f5b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T15:08:29.253 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:29.251+0000 7f06536fa640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f064c0cc380 0x7f064c15e150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T15:08:29.253 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:29.251+0000 7f06536fa640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f064c14a5c0 0x7f064c070680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T15:08:29.253 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:29.251+0000 7f06536fa640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f064c1fa7a0 con 0x7f064c0c3130 2023-08-09T15:08:29.253 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:29.251+0000 7f06536fa640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f064c1f99e0 con 0x7f064c14a5c0 2023-08-09T15:08:29.253 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:29.251+0000 7f06536fa640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f064c1f9cc0 con 0x7f064c0cc380 2023-08-09T15:08:29.255 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:29.252+0000 7f0651496640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f064c0c3130 0x7f064c1f5b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T15:08:29.255 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:29.252+0000 7f0651496640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f064c0c3130 0x7f064c1f5b50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:36074/0 (socket says 172.21.15.43:36074) 2023-08-09T15:08:29.255 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:29.252+0000 7f0651496640 1 -- 172.21.15.43:0/3406131370 learned_addr learned my addr 172.21.15.43:0/3406131370 (peer_addr_for_me v2:172.21.15.43:0/0) 2023-08-09T15:08:29.255 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:29.252+0000 7f0650c95640 1 --2- 172.21.15.43:0/3406131370 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f064c0cc380 0x7f064c15e150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T15:08:29.255 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:29.252+0000 7f0651c97640 1 --2- 172.21.15.43:0/3406131370 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f064c14a5c0 0x7f064c070680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T15:08:29.255 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:29.252+0000 7f0651496640 1 -- 172.21.15.43:0/3406131370 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f064c0cc380 msgr2=0x7f064c15e150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T15:08:29.256 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:29.252+0000 7f0651496640 1 --2- 172.21.15.43:0/3406131370 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f064c0cc380 0x7f064c15e150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T15:08:29.256 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:29.252+0000 7f0651496640 1 -- 172.21.15.43:0/3406131370 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f064c14a5c0 msgr2=0x7f064c070680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T15:08:29.256 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:29.252+0000 7f0651496640 1 --2- 172.21.15.43:0/3406131370 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f064c14a5c0 0x7f064c070680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T15:08:29.256 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:29.252+0000 7f0651496640 1 -- 172.21.15.43:0/3406131370 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0648009400 con 0x7f064c0c3130 2023-08-09T15:08:29.256 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:29.253+0000 7f0651496640 1 --2- 172.21.15.43:0/3406131370 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f064c0c3130 0x7f064c1f5b50 secure :-1 s=READY pgs=2138 cs=0 l=1 rev1=1 crypto rx=0x7f064802b750 tx=0x7f06480046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-09T15:08:29.256 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:29.253+0000 7f0641ffb640 1 -- 172.21.15.43:0/3406131370 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f06480044c0 con 0x7f064c0c3130 2023-08-09T15:08:29.256 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:29.253+0000 7f0641ffb640 1 -- 172.21.15.43:0/3406131370 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0648003e20 con 0x7f064c0c3130 2023-08-09T15:08:29.257 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:29.253+0000 7f0641ffb640 1 -- 172.21.15.43:0/3406131370 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0648030730 con 0x7f064c0c3130 2023-08-09T15:08:29.257 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:29.253+0000 7f06536fa640 1 -- 172.21.15.43:0/3406131370 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f064c1f9fa0 con 0x7f064c0c3130 2023-08-09T15:08:29.257 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:29.253+0000 7f06536fa640 1 -- 172.21.15.43:0/3406131370 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f064c1f71f0 con 0x7f064c0c3130 2023-08-09T15:08:29.257 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:29.255+0000 7f06536fa640 1 -- 172.21.15.43:0/3406131370 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f064c1fa4c0 con 0x7f064c0c3130 2023-08-09T15:08:29.257 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:29.256+0000 7f0641ffb640 1 -- 172.21.15.43:0/3406131370 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 105) v1 ==== 88387+0+0 (secure 0 0 0) 0x7f0648039070 con 0x7f064c0c3130 2023-08-09T15:08:29.258 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:29.257+0000 7f0641ffb640 1 --2- 172.21.15.43:0/3406131370 >> [v2:172.21.15.203:6832/3082732769,v1:172.21.15.203:6833/3082732769] conn(0x7f063006a0e0 0x7f063006c5a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T15:08:29.258 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:29.257+0000 7f0641ffb640 1 -- 172.21.15.43:0/3406131370 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(194..194 src has 1..194) v4 ==== 7486+0+0 (secure 0 0 0) 0x7f06480aa450 con 0x7f064c0c3130 2023-08-09T15:08:29.259 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:29.257+0000 7f0650c95640 1 --2- 172.21.15.43:0/3406131370 >> [v2:172.21.15.203:6832/3082732769,v1:172.21.15.203:6833/3082732769] conn(0x7f063006a0e0 0x7f063006c5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T15:08:29.260 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:29.259+0000 7f0650c95640 1 --2- 172.21.15.43:0/3406131370 >> [v2:172.21.15.203:6832/3082732769,v1:172.21.15.203:6833/3082732769] conn(0x7f063006a0e0 0x7f063006c5a0 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7f064c0b3230 tx=0x7f063c0016d0 comp rx=0 tx=0).ready entity=mgr.8500 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-09T15:08:29.262 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:29.262+0000 7f0641ffb640 1 -- 172.21.15.43:0/3406131370 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178066 (secure 0 0 0) 0x7f064c1fa4c0 con 0x7f064c0c3130 2023-08-09T15:08:29.505 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:29.503+0000 7f06536fa640 1 -- 172.21.15.43:0/3406131370 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f064c0c3570 con 0x7f064c0c3130 2023-08-09T15:08:29.506 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:29.505+0000 7f0641ffb640 1 -- 172.21.15.43:0/3406131370 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 87 v87) v1 ==== 94+0+4882 (secure 0 0 0) 0x7f0648075460 con 0x7f064c0c3130 2023-08-09T15:08:29.506 INFO:teuthology.orchestra.run.smithi043.stdout: 2023-08-09T15:08:29.507 INFO:teuthology.orchestra.run.smithi043.stdout:{"epoch":87,"default_fscid":10,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":7328,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.203:6836/3701396136","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6834","nonce":3701396136},{"type":"v1","addr":"172.21.15.203:6836","nonce":3701396136}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":82},{"gid":7343,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.43:6835/2096611841","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6833","nonce":2096611841},{"type":"v1","addr":"172.21.15.43:6835","nonce":2096611841}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":82},{"gid":8242,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.203:6837/18754135","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6835","nonce":18754135},{"type":"v1","addr":"172.21.15.203:6837","nonce":18754135}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":81}],"filesystems":[{"mdsmap":{"epoch":87,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-08-09T14:52:30.188965+0000","modified":"2023-08-09T14:52:33.772528+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":10185},"failed":[],"damaged":[],"stopped":[],"info":{"gid_10185":{"gid":10185,"name":"c","rank":0,"incarnation":83,"state":"up:active","state_seq":3,"addr":"172.21.15.43:6834/2866936881","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6832","nonce":2866936881},{"type":"v1","addr":"172.21.15.43:6834","nonce":2866936881}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[23],"metadata_pool":22,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":10}]} 2023-08-09T15:08:29.507 INFO:teuthology.orchestra.run.smithi043.stderr:dumped fsmap epoch 87 2023-08-09T15:08:29.509 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:29.508+0000 7f06536fa640 1 -- 172.21.15.43:0/3406131370 >> [v2:172.21.15.203:6832/3082732769,v1:172.21.15.203:6833/3082732769] conn(0x7f063006a0e0 msgr2=0x7f063006c5a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T15:08:29.509 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:29.508+0000 7f06536fa640 1 --2- 172.21.15.43:0/3406131370 >> [v2:172.21.15.203:6832/3082732769,v1:172.21.15.203:6833/3082732769] conn(0x7f063006a0e0 0x7f063006c5a0 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7f064c0b3230 tx=0x7f063c0016d0 comp rx=0 tx=0).stop 2023-08-09T15:08:29.509 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:29.508+0000 7f06536fa640 1 -- 172.21.15.43:0/3406131370 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f064c0c3130 msgr2=0x7f064c1f5b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T15:08:29.510 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:29.508+0000 7f06536fa640 1 --2- 172.21.15.43:0/3406131370 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f064c0c3130 0x7f064c1f5b50 secure :-1 s=READY pgs=2138 cs=0 l=1 rev1=1 crypto rx=0x7f064802b750 tx=0x7f06480046c0 comp rx=0 tx=0).stop 2023-08-09T15:08:29.510 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:29.508+0000 7f06536fa640 1 -- 172.21.15.43:0/3406131370 shutdown_connections 2023-08-09T15:08:29.510 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:29.508+0000 7f06536fa640 1 --2- 172.21.15.43:0/3406131370 >> [v2:172.21.15.203:6832/3082732769,v1:172.21.15.203:6833/3082732769] conn(0x7f063006a0e0 0x7f063006c5a0 unknown :-1 s=CLOSED pgs=28 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T15:08:29.510 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:29.508+0000 7f06536fa640 1 --2- 172.21.15.43:0/3406131370 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f064c14a5c0 0x7f064c070680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T15:08:29.510 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:29.508+0000 7f06536fa640 1 --2- 172.21.15.43:0/3406131370 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f064c0cc380 0x7f064c15e150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T15:08:29.510 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:29.508+0000 7f06536fa640 1 --2- 172.21.15.43:0/3406131370 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f064c0c3130 0x7f064c1f5b50 unknown :-1 s=CLOSED pgs=2138 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T15:08:29.511 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:29.508+0000 7f06536fa640 1 -- 172.21.15.43:0/3406131370 >> 172.21.15.43:0/3406131370 conn(0x7f064c0ce0a0 msgr2=0x7f064c163850 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-09T15:08:29.511 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:29.508+0000 7f06536fa640 1 -- 172.21.15.43:0/3406131370 shutdown_connections 2023-08-09T15:08:29.511 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:29.508+0000 7f06536fa640 1 -- 172.21.15.43:0/3406131370 wait complete. 2023-08-09T15:08:29.532 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-08-09T15:08:29.759 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:29.758+0000 7f3647fff640 1 Processor -- start 2023-08-09T15:08:29.759 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:29.758+0000 7f3647fff640 1 -- start start 2023-08-09T15:08:29.760 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:29.759+0000 7f3647fff640 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7f364815c620 0x7f36480cfe70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T15:08:29.760 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:29.759+0000 7f3647fff640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f36480d0ef0 0x7f36480b27c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T15:08:29.761 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:29.759+0000 7f3647fff640 1 -- --> v1:172.21.15.203:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f36480b2330 con 0x7f36480cf4f0 2023-08-09T15:08:29.761 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:29.759+0000 7f3647fff640 1 -- --> v2:172.21.15.203:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f36480d2ec0 con 0x7f364815c620 2023-08-09T15:08:29.761 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:29.759+0000 7f3647fff640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f36480d0500 con 0x7f36480d0ef0 2023-08-09T15:08:29.762 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:29.760+0000 7f36477fe640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f36480d0ef0 0x7f36480b27c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T15:08:29.762 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:29.760+0000 7f36477fe640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f36480d0ef0 0x7f36480b27c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.203:3301/0 says I am v2:172.21.15.43:45708/0 (socket says 172.21.15.43:45708) 2023-08-09T15:08:29.762 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:29.760+0000 7f36477fe640 1 -- 172.21.15.43:0/1408091403 learned_addr learned my addr 172.21.15.43:0/1408091403 (peer_addr_for_me v2:172.21.15.43:0/0) 2023-08-09T15:08:29.762 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:29.760+0000 7f3646ffd640 1 --2- 172.21.15.43:0/1408091403 >> v2:172.21.15.203:3300/0 conn(0x7f364815c620 0x7f36480cfe70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T15:08:29.762 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:29.761+0000 7f36477fe640 1 -- 172.21.15.43:0/1408091403 >> v2:172.21.15.203:3300/0 conn(0x7f364815c620 msgr2=0x7f36480cfe70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-09T15:08:29.762 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:29.761+0000 7f36477fe640 1 --2- 172.21.15.43:0/1408091403 >> v2:172.21.15.203:3300/0 conn(0x7f364815c620 0x7f36480cfe70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T15:08:29.763 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:29.761+0000 7f36477fe640 1 -- 172.21.15.43:0/1408091403 >> v1:172.21.15.203:6789/0 conn(0x7f36480cf4f0 legacy=0x7f36480d07e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T15:08:29.763 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:29.761+0000 7f36477fe640 1 -- 172.21.15.43:0/1408091403 --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f36480b1a00 con 0x7f36480d0ef0 2023-08-09T15:08:29.763 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:29.761+0000 7f3646ffd640 1 --2- 172.21.15.43:0/1408091403 >> v2:172.21.15.203:3300/0 conn(0x7f364815c620 0x7f36480cfe70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-08-09T15:08:29.763 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:29.762+0000 7f36477fe640 1 --2- 172.21.15.43:0/1408091403 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f36480d0ef0 0x7f36480b27c0 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 crypto rx=0x7f3640009e50 tx=0x7f364002bee0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c909dd8329376f92 server_cookie=0 in_seq=0 out_seq=0 2023-08-09T15:08:29.764 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:29.763+0000 7f3645ffb640 1 -- 172.21.15.43:0/1408091403 <== mon.2 v2:172.21.15.203:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f364002ca80 con 0x7f36480d0ef0 2023-08-09T15:08:29.764 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:29.763+0000 7f3645ffb640 1 -- 172.21.15.43:0/1408091403 <== mon.2 v2:172.21.15.203:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f364002cbe0 con 0x7f36480d0ef0 2023-08-09T15:08:29.764 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:29.763+0000 7f3645ffb640 1 -- 172.21.15.43:0/1408091403 <== mon.2 v2:172.21.15.203:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f36400343f0 con 0x7f36480d0ef0 2023-08-09T15:08:29.765 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:29.763+0000 7f3647fff640 1 -- 172.21.15.43:0/1408091403 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f36480d0ef0 msgr2=0x7f36480b27c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T15:08:29.765 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:29.763+0000 7f3647fff640 1 --2- 172.21.15.43:0/1408091403 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f36480d0ef0 0x7f36480b27c0 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 crypto rx=0x7f3640009e50 tx=0x7f364002bee0 comp rx=0 tx=0).stop 2023-08-09T15:08:29.765 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:29.763+0000 7f3647fff640 1 -- 172.21.15.43:0/1408091403 shutdown_connections 2023-08-09T15:08:29.765 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:29.763+0000 7f3647fff640 1 --2- 172.21.15.43:0/1408091403 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f36480d0ef0 0x7f36480b27c0 unknown :-1 s=CLOSED pgs=309 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T15:08:29.765 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:29.763+0000 7f3647fff640 1 --2- 172.21.15.43:0/1408091403 >> v2:172.21.15.203:3300/0 conn(0x7f364815c620 0x7f36480cfe70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T15:08:29.765 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:29.763+0000 7f3647fff640 1 -- 172.21.15.43:0/1408091403 >> 172.21.15.43:0/1408091403 conn(0x7f364809b650 msgr2=0x7f36480d22a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-09T15:08:29.766 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:29.763+0000 7f3647fff640 1 -- 172.21.15.43:0/1408091403 shutdown_connections 2023-08-09T15:08:29.766 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:29.763+0000 7f3647fff640 1 -- 172.21.15.43:0/1408091403 wait complete. 2023-08-09T15:08:29.766 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:29.765+0000 7f3647fff640 1 Processor -- start 2023-08-09T15:08:29.766 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:29.765+0000 7f3647fff640 1 -- start start 2023-08-09T15:08:29.767 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:29.766+0000 7f3647fff640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f36480cf4f0 0x7f3648155900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T15:08:29.767 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:29.766+0000 7f3647fff640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f36480d0ef0 0x7f36480b65a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T15:08:29.768 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:29.766+0000 7f3647fff640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f364815c620 0x7f36480c80f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T15:08:29.768 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:29.766+0000 7f3647fff640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3648155e40 con 0x7f36480cf4f0 2023-08-09T15:08:29.768 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:29.766+0000 7f3647fff640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f36480c5dd0 con 0x7f364815c620 2023-08-09T15:08:29.768 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:29.766+0000 7f3647fff640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f36481f5e50 con 0x7f36480d0ef0 2023-08-09T15:08:29.769 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:29.767+0000 7f3646ffd640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f36480cf4f0 0x7f3648155900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T15:08:29.770 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:29.767+0000 7f3646ffd640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f36480cf4f0 0x7f3648155900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:36090/0 (socket says 172.21.15.43:36090) 2023-08-09T15:08:29.770 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:29.767+0000 7f3646ffd640 1 -- 172.21.15.43:0/4101246116 learned_addr learned my addr 172.21.15.43:0/4101246116 (peer_addr_for_me v2:172.21.15.43:0/0) 2023-08-09T15:08:29.770 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:29.767+0000 7f36467fc640 1 --2- 172.21.15.43:0/4101246116 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f36480d0ef0 0x7f36480b65a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T15:08:29.770 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:29.767+0000 7f36477fe640 1 --2- 172.21.15.43:0/4101246116 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f364815c620 0x7f36480c80f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T15:08:29.770 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:29.767+0000 7f3646ffd640 1 -- 172.21.15.43:0/4101246116 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f36480d0ef0 msgr2=0x7f36480b65a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T15:08:29.770 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:29.767+0000 7f3646ffd640 1 --2- 172.21.15.43:0/4101246116 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f36480d0ef0 0x7f36480b65a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T15:08:29.771 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:29.767+0000 7f3646ffd640 1 -- 172.21.15.43:0/4101246116 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f364815c620 msgr2=0x7f36480c80f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T15:08:29.771 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:29.767+0000 7f3646ffd640 1 --2- 172.21.15.43:0/4101246116 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f364815c620 0x7f36480c80f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T15:08:29.771 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:29.767+0000 7f3646ffd640 1 -- 172.21.15.43:0/4101246116 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3640009b30 con 0x7f36480cf4f0 2023-08-09T15:08:29.771 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:29.767+0000 7f36477fe640 1 --2- 172.21.15.43:0/4101246116 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f364815c620 0x7f36480c80f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-08-09T15:08:29.771 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:29.767+0000 7f3646ffd640 1 --2- 172.21.15.43:0/4101246116 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f36480cf4f0 0x7f3648155900 secure :-1 s=READY pgs=2139 cs=0 l=1 rev1=1 crypto rx=0x7f363800d800 tx=0x7f363800dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-09T15:08:29.771 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:29.768+0000 7f362ffff640 1 -- 172.21.15.43:0/4101246116 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3638004030 con 0x7f36480cf4f0 2023-08-09T15:08:29.771 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:29.768+0000 7f362ffff640 1 -- 172.21.15.43:0/4101246116 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3638004190 con 0x7f36480cf4f0 2023-08-09T15:08:29.771 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:29.768+0000 7f362ffff640 1 -- 172.21.15.43:0/4101246116 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3638004520 con 0x7f36480cf4f0 2023-08-09T15:08:29.772 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:29.768+0000 7f3647fff640 1 -- 172.21.15.43:0/4101246116 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f36480c8c60 con 0x7f36480cf4f0 2023-08-09T15:08:29.772 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:29.768+0000 7f3647fff640 1 -- 172.21.15.43:0/4101246116 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f36480c6300 con 0x7f36480cf4f0 2023-08-09T15:08:29.772 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:29.769+0000 7f3647fff640 1 -- 172.21.15.43:0/4101246116 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f36480739f0 con 0x7f36480cf4f0 2023-08-09T15:08:29.772 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:29.770+0000 7f362ffff640 1 -- 172.21.15.43:0/4101246116 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 105) v1 ==== 88387+0+0 (secure 0 0 0) 0x7f3638005990 con 0x7f36480cf4f0 2023-08-09T15:08:29.773 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:29.772+0000 7f362ffff640 1 --2- 172.21.15.43:0/4101246116 >> [v2:172.21.15.203:6832/3082732769,v1:172.21.15.203:6833/3082732769] conn(0x7f363406a0e0 0x7f363406c5a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T15:08:29.773 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:29.772+0000 7f362ffff640 1 -- 172.21.15.43:0/4101246116 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(194..194 src has 1..194) v4 ==== 7486+0+0 (secure 0 0 0) 0x7f3638088b70 con 0x7f36480cf4f0 2023-08-09T15:08:29.773 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:29.772+0000 7f36467fc640 1 --2- 172.21.15.43:0/4101246116 >> [v2:172.21.15.203:6832/3082732769,v1:172.21.15.203:6833/3082732769] conn(0x7f363406a0e0 0x7f363406c5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T15:08:29.774 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:29.774+0000 7f36467fc640 1 --2- 172.21.15.43:0/4101246116 >> [v2:172.21.15.203:6832/3082732769,v1:172.21.15.203:6833/3082732769] conn(0x7f363406a0e0 0x7f363406c5a0 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7f364815a8d0 tx=0x7f363c009040 comp rx=0 tx=0).ready entity=mgr.8500 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-09T15:08:29.778 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:29.777+0000 7f362ffff640 1 -- 172.21.15.43:0/4101246116 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178066 (secure 0 0 0) 0x7f3638057700 con 0x7f36480cf4f0 2023-08-09T15:08:30.010 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:30.009+0000 7f3647fff640 1 -- 172.21.15.43:0/4101246116 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f364806aa20 con 0x7f36480cf4f0 2023-08-09T15:08:30.011 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:30.010+0000 7f362ffff640 1 -- 172.21.15.43:0/4101246116 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 87 v87) v1 ==== 94+0+4882 (secure 0 0 0) 0x7f3638057290 con 0x7f36480cf4f0 2023-08-09T15:08:30.011 INFO:teuthology.orchestra.run.smithi043.stdout: 2023-08-09T15:08:30.011 INFO:teuthology.orchestra.run.smithi043.stdout:{"epoch":87,"default_fscid":10,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":7328,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.203:6836/3701396136","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6834","nonce":3701396136},{"type":"v1","addr":"172.21.15.203:6836","nonce":3701396136}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":82},{"gid":7343,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.43:6835/2096611841","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6833","nonce":2096611841},{"type":"v1","addr":"172.21.15.43:6835","nonce":2096611841}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":82},{"gid":8242,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.203:6837/18754135","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6835","nonce":18754135},{"type":"v1","addr":"172.21.15.203:6837","nonce":18754135}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":81}],"filesystems":[{"mdsmap":{"epoch":87,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-08-09T14:52:30.188965+0000","modified":"2023-08-09T14:52:33.772528+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":10185},"failed":[],"damaged":[],"stopped":[],"info":{"gid_10185":{"gid":10185,"name":"c","rank":0,"incarnation":83,"state":"up:active","state_seq":3,"addr":"172.21.15.43:6834/2866936881","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6832","nonce":2866936881},{"type":"v1","addr":"172.21.15.43:6834","nonce":2866936881}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[23],"metadata_pool":22,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":10}]} 2023-08-09T15:08:30.012 INFO:teuthology.orchestra.run.smithi043.stderr:dumped fsmap epoch 87 2023-08-09T15:08:30.013 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:30.012+0000 7f3647fff640 1 -- 172.21.15.43:0/4101246116 >> [v2:172.21.15.203:6832/3082732769,v1:172.21.15.203:6833/3082732769] conn(0x7f363406a0e0 msgr2=0x7f363406c5a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T15:08:30.014 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:30.012+0000 7f3647fff640 1 --2- 172.21.15.43:0/4101246116 >> [v2:172.21.15.203:6832/3082732769,v1:172.21.15.203:6833/3082732769] conn(0x7f363406a0e0 0x7f363406c5a0 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7f364815a8d0 tx=0x7f363c009040 comp rx=0 tx=0).stop 2023-08-09T15:08:30.014 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:30.012+0000 7f3647fff640 1 -- 172.21.15.43:0/4101246116 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f36480cf4f0 msgr2=0x7f3648155900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T15:08:30.014 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:30.012+0000 7f3647fff640 1 --2- 172.21.15.43:0/4101246116 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f36480cf4f0 0x7f3648155900 secure :-1 s=READY pgs=2139 cs=0 l=1 rev1=1 crypto rx=0x7f363800d800 tx=0x7f363800dcc0 comp rx=0 tx=0).stop 2023-08-09T15:08:30.014 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:30.012+0000 7f3647fff640 1 -- 172.21.15.43:0/4101246116 shutdown_connections 2023-08-09T15:08:30.014 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:30.012+0000 7f3647fff640 1 --2- 172.21.15.43:0/4101246116 >> [v2:172.21.15.203:6832/3082732769,v1:172.21.15.203:6833/3082732769] conn(0x7f363406a0e0 0x7f363406c5a0 unknown :-1 s=CLOSED pgs=29 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T15:08:30.015 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:30.013+0000 7f3647fff640 1 --2- 172.21.15.43:0/4101246116 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f364815c620 0x7f36480c80f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T15:08:30.015 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:30.013+0000 7f3647fff640 1 --2- 172.21.15.43:0/4101246116 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f36480d0ef0 0x7f36480b65a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T15:08:30.015 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:30.013+0000 7f3647fff640 1 --2- 172.21.15.43:0/4101246116 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f36480cf4f0 0x7f3648155900 unknown :-1 s=CLOSED pgs=2139 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T15:08:30.015 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:30.013+0000 7f3647fff640 1 -- 172.21.15.43:0/4101246116 >> 172.21.15.43:0/4101246116 conn(0x7f364809b650 msgr2=0x7f36480d1420 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-09T15:08:30.015 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:30.013+0000 7f3647fff640 1 -- 172.21.15.43:0/4101246116 shutdown_connections 2023-08-09T15:08:30.015 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:30.013+0000 7f3647fff640 1 -- 172.21.15.43:0/4101246116 wait complete. 2023-08-09T15:08:30.034 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-08-09T15:08:30.240 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:30.238+0000 7fbcf0b99640 1 Processor -- start 2023-08-09T15:08:30.240 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:30.239+0000 7fbcf0b99640 1 -- start start 2023-08-09T15:08:30.241 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:30.240+0000 7fbcf0b99640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fbcec0b4670 0x7fbcec0d2ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T15:08:30.242 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:30.240+0000 7fbcf0b99640 1 -- --> v1:172.21.15.43:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbcec0ce090 con 0x7fbcec0b2790 2023-08-09T15:08:30.242 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:30.240+0000 7fbcf0b99640 1 -- --> v1:172.21.15.203:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbcec0cd7f0 con 0x7fbcec0b2b70 2023-08-09T15:08:30.242 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:30.240+0000 7fbcf0b99640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbcec0b3950 con 0x7fbcec0b4670 2023-08-09T15:08:30.243 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:30.241+0000 7fbcea59c640 1 --1- >> v1:172.21.15.43:6789/0 conn(0x7fbcec0b2790 0x7fbcec0b5330 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.43:6789/0 says I am v1:172.21.15.43:41742/0 (socket says 172.21.15.43:41742) 2023-08-09T15:08:30.243 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:30.241+0000 7fbcea59c640 1 -- 172.21.15.43:0/3987111059 learned_addr learned my addr 172.21.15.43:0/3987111059 (peer_addr_for_me v1:172.21.15.43:0/0) 2023-08-09T15:08:30.244 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:30.241+0000 7fbcead9d640 1 --2- 172.21.15.43:0/3987111059 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fbcec0b4670 0x7fbcec0d2ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T15:08:30.244 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:30.241+0000 7fbce959a640 1 -- 172.21.15.43:0/3987111059 <== mon.0 v1:172.21.15.43:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1531291106 0 0) 0x7fbcec0ce090 con 0x7fbcec0b2790 2023-08-09T15:08:30.244 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:30.241+0000 7fbce959a640 1 -- 172.21.15.43:0/3987111059 --> v1:172.21.15.43:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fbcd8003520 con 0x7fbcec0b2790 2023-08-09T15:08:30.244 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:30.242+0000 7fbce959a640 1 -- 172.21.15.43:0/3987111059 <== mon.0 v1:172.21.15.43:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 4030698457 0 0) 0x7fbcd8003520 con 0x7fbcec0b2790 2023-08-09T15:08:30.244 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:30.242+0000 7fbce959a640 1 -- 172.21.15.43:0/3987111059 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fbcec0b4670 msgr2=0x7fbcec0d2ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-09T15:08:30.245 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:30.242+0000 7fbce959a640 1 --2- 172.21.15.43:0/3987111059 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fbcec0b4670 0x7fbcec0d2ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T15:08:30.245 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:30.242+0000 7fbce959a640 1 -- 172.21.15.43:0/3987111059 >> v1:172.21.15.203:6789/0 conn(0x7fbcec0b2b70 legacy=0x7fbcec0b3f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T15:08:30.245 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:30.242+0000 7fbce959a640 1 -- 172.21.15.43:0/3987111059 --> v1:172.21.15.43:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbcec0cdad0 con 0x7fbcec0b2790 2023-08-09T15:08:30.245 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:30.242+0000 7fbce959a640 1 -- 172.21.15.43:0/3987111059 <== mon.0 v1:172.21.15.43:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1285150740 0 0) 0x7fbcdc001670 con 0x7fbcec0b2790 2023-08-09T15:08:30.245 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:30.242+0000 7fbce959a640 1 -- 172.21.15.43:0/3987111059 >> v1:172.21.15.43:6789/0 conn(0x7fbcec0b2790 legacy=0x7fbcec0b5330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T15:08:30.245 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:30.242+0000 7fbce959a640 1 --2- 172.21.15.43:0/3987111059 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fbcd8003a10 0x7fbcd8003e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T15:08:30.245 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:30.242+0000 7fbce959a640 1 --2- 172.21.15.43:0/3987111059 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fbcec0b4670 0x7fbcd80047c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T15:08:30.246 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:30.242+0000 7fbce959a640 1 --2- 172.21.15.43:0/3987111059 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fbcd8004d00 0x7fbcd8005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T15:08:30.246 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:30.242+0000 7fbce959a640 1 -- 172.21.15.43:0/3987111059 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbcec0b3950 con 0x7fbcd8004d00 2023-08-09T15:08:30.246 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:30.242+0000 7fbce959a640 1 -- 172.21.15.43:0/3987111059 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbcd800bac0 con 0x7fbcd8003a10 2023-08-09T15:08:30.246 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:30.242+0000 7fbce959a640 1 -- 172.21.15.43:0/3987111059 --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbcd800bc80 con 0x7fbcec0b4670 2023-08-09T15:08:30.246 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:30.242+0000 7fbcead9d640 1 --2- 172.21.15.43:0/3987111059 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fbcd8004d00 0x7fbcd8005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T15:08:30.247 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:30.243+0000 7fbce9d9b640 1 --2- 172.21.15.43:0/3987111059 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fbcec0b4670 0x7fbcd80047c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T15:08:30.248 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:30.243+0000 7fbcea59c640 1 --2- 172.21.15.43:0/3987111059 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fbcd8003a10 0x7fbcd8003e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T15:08:30.248 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:30.243+0000 7fbcead9d640 1 -- 172.21.15.43:0/3987111059 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fbcec0b4670 msgr2=0x7fbcd80047c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-09T15:08:30.248 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:30.243+0000 7fbcead9d640 1 --2- 172.21.15.43:0/3987111059 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fbcec0b4670 0x7fbcd80047c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T15:08:30.248 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:30.243+0000 7fbcead9d640 1 -- 172.21.15.43:0/3987111059 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fbcd8003a10 msgr2=0x7fbcd8003e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-09T15:08:30.248 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:30.243+0000 7fbcead9d640 1 --2- 172.21.15.43:0/3987111059 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fbcd8003a10 0x7fbcd8003e30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T15:08:30.248 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:30.243+0000 7fbcead9d640 1 -- 172.21.15.43:0/3987111059 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbcd8003520 con 0x7fbcd8004d00 2023-08-09T15:08:30.248 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:30.243+0000 7fbce9d9b640 1 --2- 172.21.15.43:0/3987111059 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fbcec0b4670 0x7fbcd80047c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-08-09T15:08:30.249 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:30.244+0000 7fbcead9d640 1 --2- 172.21.15.43:0/3987111059 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fbcd8004d00 0x7fbcd8005180 secure :-1 s=READY pgs=2141 cs=0 l=1 rev1=1 crypto rx=0x7fbce4009f90 tx=0x7fbcd80057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dbefc240c3b8dbe1 server_cookie=0 in_seq=0 out_seq=0 2023-08-09T15:08:30.249 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:30.244+0000 7fbce959a640 1 -- 172.21.15.43:0/3987111059 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbce402cbd0 con 0x7fbcd8004d00 2023-08-09T15:08:30.249 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:30.244+0000 7fbce959a640 1 -- 172.21.15.43:0/3987111059 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbce400f040 con 0x7fbcd8004d00 2023-08-09T15:08:30.249 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:30.244+0000 7fbce959a640 1 -- 172.21.15.43:0/3987111059 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbce4034440 con 0x7fbcd8004d00 2023-08-09T15:08:30.249 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:30.245+0000 7fbcf0b99640 1 -- 172.21.15.43:0/3987111059 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fbcd8004d00 msgr2=0x7fbcd8005180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T15:08:30.249 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:30.245+0000 7fbcf0b99640 1 --2- 172.21.15.43:0/3987111059 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fbcd8004d00 0x7fbcd8005180 secure :-1 s=READY pgs=2141 cs=0 l=1 rev1=1 crypto rx=0x7fbce4009f90 tx=0x7fbcd80057f0 comp rx=0 tx=0).stop 2023-08-09T15:08:30.249 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:30.245+0000 7fbcead9d640 1 -- 172.21.15.43:0/3987111059 reap_dead start 2023-08-09T15:08:30.249 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:30.245+0000 7fbcf0b99640 1 -- 172.21.15.43:0/3987111059 shutdown_connections 2023-08-09T15:08:30.250 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:30.245+0000 7fbcf0b99640 1 -- 172.21.15.43:0/3987111059 >> 172.21.15.43:0/3987111059 conn(0x7fbcec09b7b0 msgr2=0x7fbcec0d24e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-09T15:08:30.250 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:30.245+0000 7fbcf0b99640 1 -- 172.21.15.43:0/3987111059 shutdown_connections 2023-08-09T15:08:30.250 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:30.245+0000 7fbcf0b99640 1 -- 172.21.15.43:0/3987111059 wait complete. 2023-08-09T15:08:30.251 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:30.247+0000 7fbcf0b99640 1 Processor -- start 2023-08-09T15:08:30.252 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:30.247+0000 7fbcf0b99640 1 -- start start 2023-08-09T15:08:30.252 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:30.248+0000 7fbcf0b99640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fbcd8004d00 0x7fbcec151a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T15:08:30.252 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:30.248+0000 7fbcf0b99640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fbcec0b0b00 0x7fbcec0b0f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T15:08:30.252 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:30.248+0000 7fbcf0b99640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fbcec1609d0 0x7fbcec160e80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T15:08:30.253 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:30.248+0000 7fbcf0b99640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbcec1613c0 con 0x7fbcec1609d0 2023-08-09T15:08:30.253 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:30.248+0000 7fbcf0b99640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbcec0b1550 con 0x7fbcec0b0b00 2023-08-09T15:08:30.253 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:30.248+0000 7fbcf0b99640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbcec0b1830 con 0x7fbcd8004d00 2023-08-09T15:08:30.253 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:30.248+0000 7fbcead9d640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fbcec1609d0 0x7fbcec160e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T15:08:30.253 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:30.248+0000 7fbcead9d640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fbcec1609d0 0x7fbcec160e80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:36106/0 (socket says 172.21.15.43:36106) 2023-08-09T15:08:30.254 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:30.248+0000 7fbcead9d640 1 -- 172.21.15.43:0/158565641 learned_addr learned my addr 172.21.15.43:0/158565641 (peer_addr_for_me v2:172.21.15.43:0/0) 2023-08-09T15:08:30.254 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:30.249+0000 7fbcea59c640 1 --2- 172.21.15.43:0/158565641 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fbcd8004d00 0x7fbcec151a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T15:08:30.254 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:30.249+0000 7fbce9d9b640 1 --2- 172.21.15.43:0/158565641 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fbcec0b0b00 0x7fbcec0b0f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T15:08:30.254 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:30.249+0000 7fbcead9d640 1 -- 172.21.15.43:0/158565641 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fbcd8004d00 msgr2=0x7fbcec151a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T15:08:30.254 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:30.249+0000 7fbcead9d640 1 --2- 172.21.15.43:0/158565641 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fbcd8004d00 0x7fbcec151a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T15:08:30.254 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:30.249+0000 7fbcead9d640 1 -- 172.21.15.43:0/158565641 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fbcec0b0b00 msgr2=0x7fbcec0b0f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T15:08:30.255 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:30.249+0000 7fbcead9d640 1 --2- 172.21.15.43:0/158565641 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fbcec0b0b00 0x7fbcec0b0f60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T15:08:30.255 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:30.249+0000 7fbcead9d640 1 -- 172.21.15.43:0/158565641 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbcec0b1b10 con 0x7fbcec1609d0 2023-08-09T15:08:30.255 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:30.249+0000 7fbcea59c640 1 --2- 172.21.15.43:0/158565641 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fbcd8004d00 0x7fbcec151a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-08-09T15:08:30.255 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:30.249+0000 7fbce9d9b640 1 --2- 172.21.15.43:0/158565641 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fbcec0b0b00 0x7fbcec0b0f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-08-09T15:08:30.255 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:30.249+0000 7fbcead9d640 1 --2- 172.21.15.43:0/158565641 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fbcec1609d0 0x7fbcec160e80 secure :-1 s=READY pgs=2142 cs=0 l=1 rev1=1 crypto rx=0x7fbce4002230 tx=0x7fbce4009750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-09T15:08:30.255 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:30.249+0000 7fbce2ffd640 1 -- 172.21.15.43:0/158565641 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbce4043070 con 0x7fbcec1609d0 2023-08-09T15:08:30.255 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:30.249+0000 7fbce2ffd640 1 -- 172.21.15.43:0/158565641 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbce402de40 con 0x7fbcec1609d0 2023-08-09T15:08:30.256 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:30.249+0000 7fbce2ffd640 1 -- 172.21.15.43:0/158565641 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbce4034d10 con 0x7fbcec1609d0 2023-08-09T15:08:30.256 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:30.249+0000 7fbcf0b99640 1 -- 172.21.15.43:0/158565641 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbcec0c9a00 con 0x7fbcec1609d0 2023-08-09T15:08:30.256 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:30.250+0000 7fbcf0b99640 1 -- 172.21.15.43:0/158565641 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbcec151f40 con 0x7fbcec1609d0 2023-08-09T15:08:30.257 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:30.251+0000 7fbcf0b99640 1 -- 172.21.15.43:0/158565641 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbcec0c9f20 con 0x7fbcec1609d0 2023-08-09T15:08:30.257 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:30.252+0000 7fbce2ffd640 1 -- 172.21.15.43:0/158565641 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 106) v1 ==== 88443+0+0 (secure 0 0 0) 0x7fbce403e440 con 0x7fbcec1609d0 2023-08-09T15:08:30.257 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:30.253+0000 7fbce2ffd640 1 --2- 172.21.15.43:0/158565641 >> [v2:172.21.15.203:6832/3082732769,v1:172.21.15.203:6833/3082732769] conn(0x7fbccc06a170 0x7fbccc06c630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T15:08:30.257 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:30.253+0000 7fbce2ffd640 1 -- 172.21.15.43:0/158565641 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(194..194 src has 1..194) v4 ==== 7486+0+0 (secure 0 0 0) 0x7fbce40aa7e0 con 0x7fbcec1609d0 2023-08-09T15:08:30.257 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:30.254+0000 7fbcea59c640 1 --2- 172.21.15.43:0/158565641 >> [v2:172.21.15.203:6832/3082732769,v1:172.21.15.203:6833/3082732769] conn(0x7fbccc06a170 0x7fbccc06c630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T15:08:30.257 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:30.255+0000 7fbcea59c640 1 --2- 172.21.15.43:0/158565641 >> [v2:172.21.15.203:6832/3082732769,v1:172.21.15.203:6833/3082732769] conn(0x7fbccc06a170 0x7fbccc06c630 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0x7fbcdc007970 tx=0x7fbcdc0014d0 comp rx=0 tx=0).ready entity=mgr.8500 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-09T15:08:30.258 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:30.257+0000 7fbce2ffd640 1 -- 172.21.15.43:0/158565641 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178066 (secure 0 0 0) 0x7fbcec0c9f20 con 0x7fbcec1609d0 2023-08-09T15:08:30.453 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:30.452+0000 7fbcf0b99640 1 -- 172.21.15.43:0/158565641 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fbcec0c9f20 con 0x7fbcec1609d0 2023-08-09T15:08:30.455 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:30.454+0000 7fbce2ffd640 1 -- 172.21.15.43:0/158565641 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v194) v1 ==== 74+0+20145 (secure 0 0 0) 0x7fbce4075790 con 0x7fbcec1609d0 2023-08-09T15:08:30.456 INFO:teuthology.orchestra.run.smithi043.stdout: 2023-08-09T15:08:30.456 INFO:teuthology.orchestra.run.smithi043.stdout:{"epoch":194,"fsid":"daac87ee-1337-4250-a955-aea4e408a39b","created":"2023-08-09T13:33:41.701198+0000","modified":"2023-08-09T15:08:29.013644+0000","last_up_change":"2023-08-09T13:33:53.783453+0000","last_in_change":"2023-08-09T13:33:45.148904+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":23,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":"rbd","create_time":"2023-08-09T13:33:54.365137+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":".mgr","create_time":"2023-08-09T13:33:54.831803+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":22,"pool_name":"cephfs_metadata","create_time":"2023-08-09T14:52:28.569334+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"185","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":23,"pool_name":"cephfs_data","create_time":"2023-08-09T14:52:29.209973+0000","flags":8193,"flags_names":"hashpspool,selfmanaged_snaps","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"192","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":33,"snap_epoch":192,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"cbd49ecd-e896-4fbd-864e-5c37a829b165","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":184,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6800","nonce":502667335},{"type":"v1","addr":"172.21.15.43:6801","nonce":502667335}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6802","nonce":502667335},{"type":"v1","addr":"172.21.15.43:6803","nonce":502667335}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6806","nonce":502667335},{"type":"v1","addr":"172.21.15.43:6807","nonce":502667335}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6804","nonce":502667335},{"type":"v1","addr":"172.21.15.43:6805","nonce":502667335}]},"public_addr":"172.21.15.43:6801/502667335","cluster_addr":"172.21.15.43:6803/502667335","heartbeat_back_addr":"172.21.15.43:6807/502667335","heartbeat_front_addr":"172.21.15.43:6805/502667335","state":["exists","up"]},{"osd":1,"uuid":"75c7e0fe-1ec7-4fe8-a2f2-3661a65b8500","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":184,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6808","nonce":3289134610},{"type":"v1","addr":"172.21.15.43:6809","nonce":3289134610}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6810","nonce":3289134610},{"type":"v1","addr":"172.21.15.43:6812","nonce":3289134610}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6821","nonce":3289134610},{"type":"v1","addr":"172.21.15.43:6824","nonce":3289134610}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6815","nonce":3289134610},{"type":"v1","addr":"172.21.15.43:6818","nonce":3289134610}]},"public_addr":"172.21.15.43:6809/3289134610","cluster_addr":"172.21.15.43:6812/3289134610","heartbeat_back_addr":"172.21.15.43:6824/3289134610","heartbeat_front_addr":"172.21.15.43:6818/3289134610","state":["exists","up"]},{"osd":2,"uuid":"bb5c41e9-7f5f-4125-9b01-3bb8bfc2ea83","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":184,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6811","nonce":4079709736},{"type":"v1","addr":"172.21.15.43:6814","nonce":4079709736}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6817","nonce":4079709736},{"type":"v1","addr":"172.21.15.43:6820","nonce":4079709736}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6829","nonce":4079709736},{"type":"v1","addr":"172.21.15.43:6831","nonce":4079709736}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6823","nonce":4079709736},{"type":"v1","addr":"172.21.15.43:6826","nonce":4079709736}]},"public_addr":"172.21.15.43:6814/4079709736","cluster_addr":"172.21.15.43:6820/4079709736","heartbeat_back_addr":"172.21.15.43:6831/4079709736","heartbeat_front_addr":"172.21.15.43:6826/4079709736","state":["exists","up"]},{"osd":3,"uuid":"bfe4c8b4-694a-465c-87e6-2b2aab852613","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":184,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6813","nonce":1709538759},{"type":"v1","addr":"172.21.15.43:6816","nonce":1709538759}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6819","nonce":1709538759},{"type":"v1","addr":"172.21.15.43:6822","nonce":1709538759}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6828","nonce":1709538759},{"type":"v1","addr":"172.21.15.43:6830","nonce":1709538759}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6825","nonce":1709538759},{"type":"v1","addr":"172.21.15.43:6827","nonce":1709538759}]},"public_addr":"172.21.15.43:6816/1709538759","cluster_addr":"172.21.15.43:6822/1709538759","heartbeat_back_addr":"172.21.15.43:6830/1709538759","heartbeat_front_addr":"172.21.15.43:6827/1709538759","state":["exists","up"]},{"osd":4,"uuid":"fa2e8a77-aba1-4756-853e-0d174a5bdeba","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":184,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6800","nonce":4196866481},{"type":"v1","addr":"172.21.15.203:6801","nonce":4196866481}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6802","nonce":4196866481},{"type":"v1","addr":"172.21.15.203:6803","nonce":4196866481}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6806","nonce":4196866481},{"type":"v1","addr":"172.21.15.203:6807","nonce":4196866481}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6804","nonce":4196866481},{"type":"v1","addr":"172.21.15.203:6805","nonce":4196866481}]},"public_addr":"172.21.15.203:6801/4196866481","cluster_addr":"172.21.15.203:6803/4196866481","heartbeat_back_addr":"172.21.15.203:6807/4196866481","heartbeat_front_addr":"172.21.15.203:6805/4196866481","state":["exists","up"]},{"osd":5,"uuid":"81811e4b-4d24-4f51-a82c-f1d700a2a00d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":184,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6816","nonce":3059290081},{"type":"v1","addr":"172.21.15.203:6817","nonce":3059290081}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6818","nonce":3059290081},{"type":"v1","addr":"172.21.15.203:6819","nonce":3059290081}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6822","nonce":3059290081},{"type":"v1","addr":"172.21.15.203:6823","nonce":3059290081}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6820","nonce":3059290081},{"type":"v1","addr":"172.21.15.203:6821","nonce":3059290081}]},"public_addr":"172.21.15.203:6817/3059290081","cluster_addr":"172.21.15.203:6819/3059290081","heartbeat_back_addr":"172.21.15.203:6823/3059290081","heartbeat_front_addr":"172.21.15.203:6821/3059290081","state":["exists","up"]},{"osd":6,"uuid":"307bac4d-6a5b-443d-af75-56d695d7674f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":184,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6824","nonce":512155569},{"type":"v1","addr":"172.21.15.203:6825","nonce":512155569}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6826","nonce":512155569},{"type":"v1","addr":"172.21.15.203:6827","nonce":512155569}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6830","nonce":512155569},{"type":"v1","addr":"172.21.15.203:6831","nonce":512155569}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6828","nonce":512155569},{"type":"v1","addr":"172.21.15.203:6829","nonce":512155569}]},"public_addr":"172.21.15.203:6825/512155569","cluster_addr":"172.21.15.203:6827/512155569","heartbeat_back_addr":"172.21.15.203:6831/512155569","heartbeat_front_addr":"172.21.15.203:6829/512155569","state":["exists","up"]},{"osd":7,"uuid":"a95dc7ca-9f1b-4295-af70-68a22a2d7d5e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":184,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6808","nonce":3331450438},{"type":"v1","addr":"172.21.15.203:6809","nonce":3331450438}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6810","nonce":3331450438},{"type":"v1","addr":"172.21.15.203:6811","nonce":3331450438}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6814","nonce":3331450438},{"type":"v1","addr":"172.21.15.203:6815","nonce":3331450438}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6812","nonce":3331450438},{"type":"v1","addr":"172.21.15.203:6813","nonce":3331450438}]},"public_addr":"172.21.15.203:6809/3331450438","cluster_addr":"172.21.15.203:6811/3331450438","heartbeat_back_addr":"172.21.15.203:6815/3331450438","heartbeat_front_addr":"172.21.15.203:6813/3331450438","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-09T13:33:50.547652+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-09T13:33:50.589469+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-09T13:33:50.561724+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-09T13:33:50.599650+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-09T13:33:50.411855+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-09T13:33:50.219681+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-09T13:33:50.504664+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-09T13:33:50.441248+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.203:0/1820136184":"2023-08-10T15:08:25.567876+0000","172.21.15.203:6833/115830229":"2023-08-10T15:08:25.567876+0000","172.21.15.203:6832/115830229":"2023-08-10T15:08:25.567876+0000","172.21.15.203:0/3317788314":"2023-08-10T15:08:25.567876+0000","172.21.15.203:0/69497804":"2023-08-10T15:03:21.353235+0000","172.21.15.203:0/169967264":"2023-08-10T15:03:21.353235+0000","172.21.15.203:6833/40883597":"2023-08-10T15:03:21.353235+0000","172.21.15.203:0/1738813745":"2023-08-10T15:08:25.567876+0000","172.21.15.203:0/1306452763":"2023-08-10T15:08:25.567876+0000","172.21.15.203:6832/40883597":"2023-08-10T15:03:21.353235+0000","172.21.15.203:0/1225284485":"2023-08-10T15:08:25.567876+0000","172.21.15.203:0/3690197627":"2023-08-10T15:03:21.353235+0000","172.21.15.203:0/1082753911":"2023-08-10T15:03:21.353235+0000","172.21.15.203:6832/2959400181":"2023-08-10T14:52:45.188641+0000","172.21.15.203:0/82655847":"2023-08-10T15:03:21.353235+0000","172.21.15.203:0/1935803093":"2023-08-10T14:52:45.188641+0000","172.21.15.203:0/3150685329":"2023-08-10T14:52:45.188641+0000","172.21.15.203:0/2219953609":"2023-08-10T15:03:21.353235+0000","172.21.15.203:0/3429376127":"2023-08-10T14:52:45.188641+0000","172.21.15.203:0/1394945671":"2023-08-10T15:08:25.567876+0000","172.21.15.203:0/2197573025":"2023-08-10T14:52:45.188641+0000","172.21.15.203:0/2458603841":"2023-08-10T14:52:45.188641+0000","172.21.15.203:6833/2959400181":"2023-08-10T14:52:45.188641+0000","172.21.15.203:0/4177046277":"2023-08-10T14:52:45.188641+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":7,"snaps":[{"begin":104,"length":1},{"begin":221,"length":1}]}],"new_removed_snaps":[],"new_purged_snaps":[{"pool":23,"snaps":[{"begin":2,"length":32}]}],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-08-09T15:08:30.458 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:30.456+0000 7fbcf0b99640 1 -- 172.21.15.43:0/158565641 >> [v2:172.21.15.203:6832/3082732769,v1:172.21.15.203:6833/3082732769] conn(0x7fbccc06a170 msgr2=0x7fbccc06c630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T15:08:30.458 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:30.456+0000 7fbcf0b99640 1 --2- 172.21.15.43:0/158565641 >> [v2:172.21.15.203:6832/3082732769,v1:172.21.15.203:6833/3082732769] conn(0x7fbccc06a170 0x7fbccc06c630 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0x7fbcdc007970 tx=0x7fbcdc0014d0 comp rx=0 tx=0).stop 2023-08-09T15:08:30.458 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:30.456+0000 7fbcf0b99640 1 -- 172.21.15.43:0/158565641 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fbcec1609d0 msgr2=0x7fbcec160e80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T15:08:30.458 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:30.456+0000 7fbcf0b99640 1 --2- 172.21.15.43:0/158565641 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fbcec1609d0 0x7fbcec160e80 secure :-1 s=READY pgs=2142 cs=0 l=1 rev1=1 crypto rx=0x7fbce4002230 tx=0x7fbce4009750 comp rx=0 tx=0).stop 2023-08-09T15:08:30.458 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:30.456+0000 7fbcf0b99640 1 -- 172.21.15.43:0/158565641 shutdown_connections 2023-08-09T15:08:30.459 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:30.456+0000 7fbcf0b99640 1 --2- 172.21.15.43:0/158565641 >> [v2:172.21.15.203:6832/3082732769,v1:172.21.15.203:6833/3082732769] conn(0x7fbccc06a170 0x7fbccc06c630 unknown :-1 s=CLOSED pgs=30 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T15:08:30.459 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:30.456+0000 7fbcf0b99640 1 --2- 172.21.15.43:0/158565641 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fbcec1609d0 0x7fbcec160e80 unknown :-1 s=CLOSED pgs=2142 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T15:08:30.459 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:30.456+0000 7fbcf0b99640 1 --2- 172.21.15.43:0/158565641 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fbcec0b0b00 0x7fbcec0b0f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T15:08:30.460 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:30.456+0000 7fbcf0b99640 1 --2- 172.21.15.43:0/158565641 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fbcd8004d00 0x7fbcec151a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T15:08:30.460 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:30.456+0000 7fbcf0b99640 1 -- 172.21.15.43:0/158565641 >> 172.21.15.43:0/158565641 conn(0x7fbcec09b7b0 msgr2=0x7fbcec06bba0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-09T15:08:30.460 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:30.456+0000 7fbcf0b99640 1 -- 172.21.15.43:0/158565641 shutdown_connections 2023-08-09T15:08:30.460 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:30.456+0000 7fbcf0b99640 1 -- 172.21.15.43:0/158565641 wait complete. 2023-08-09T15:08:30.476 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2023-08-09T15:08:30.697 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:30.695+0000 7f4c67406640 1 Processor -- start 2023-08-09T15:08:30.697 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:30.696+0000 7f4c67406640 1 -- start start 2023-08-09T15:08:30.699 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:30.698+0000 7f4c67406640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f4c600c86f0 0x7f4c600c8b50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T15:08:30.699 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:30.698+0000 7f4c67406640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f4c6006c8d0 0x7f4c6006cd40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T15:08:30.699 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:30.698+0000 7f4c67406640 1 -- --> v1:172.21.15.43:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4c600c9090 con 0x7f4c60143c20 2023-08-09T15:08:30.699 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:30.698+0000 7f4c67406640 1 -- --> v2:172.21.15.43:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4c60144000 con 0x7f4c600c86f0 2023-08-09T15:08:30.699 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:30.698+0000 7f4c67406640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4c6006c440 con 0x7f4c6006c8d0 2023-08-09T15:08:30.700 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:30.698+0000 7f4c651a2640 1 --1- >> v1:172.21.15.43:6789/0 conn(0x7f4c60143c20 0x7f4c6006f540 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.43:6789/0 says I am v1:172.21.15.43:41744/0 (socket says 172.21.15.43:41744) 2023-08-09T15:08:30.700 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:30.698+0000 7f4c651a2640 1 -- 172.21.15.43:0/4140737199 learned_addr learned my addr 172.21.15.43:0/4140737199 (peer_addr_for_me v1:172.21.15.43:0/0) 2023-08-09T15:08:30.701 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:30.698+0000 7f4c649a1640 1 --2- 172.21.15.43:0/4140737199 >> v2:172.21.15.43:3300/0 conn(0x7f4c600c86f0 0x7f4c600c8b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T15:08:30.701 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:30.698+0000 7f4c659a3640 1 --2- 172.21.15.43:0/4140737199 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f4c6006c8d0 0x7f4c6006cd40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T15:08:30.701 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:30.699+0000 7f4c57fff640 1 -- 172.21.15.43:0/4140737199 <== mon.0 v1:172.21.15.43:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3495638582 0 0) 0x7f4c600c9090 con 0x7f4c60143c20 2023-08-09T15:08:30.701 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:30.699+0000 7f4c57fff640 1 -- 172.21.15.43:0/4140737199 --> v1:172.21.15.43:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f4c4c003520 con 0x7f4c60143c20 2023-08-09T15:08:30.701 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:30.699+0000 7f4c649a1640 1 -- 172.21.15.43:0/4140737199 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f4c6006c8d0 msgr2=0x7f4c6006cd40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-09T15:08:30.701 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:30.699+0000 7f4c649a1640 1 --2- 172.21.15.43:0/4140737199 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f4c6006c8d0 0x7f4c6006cd40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T15:08:30.701 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:30.699+0000 7f4c649a1640 1 -- 172.21.15.43:0/4140737199 >> v1:172.21.15.43:6789/0 conn(0x7f4c60143c20 legacy=0x7f4c6006f540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T15:08:30.702 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:30.699+0000 7f4c649a1640 1 -- 172.21.15.43:0/4140737199 --> v2:172.21.15.43:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4c6006fc50 con 0x7f4c600c86f0 2023-08-09T15:08:30.702 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:30.700+0000 7f4c649a1640 1 --2- 172.21.15.43:0/4140737199 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f4c600c86f0 0x7f4c600c8b50 secure :-1 s=READY pgs=2144 cs=0 l=1 rev1=1 crypto rx=0x7f4c500097a0 tx=0x7f4c5002b820 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ed3f75348d55842b server_cookie=0 in_seq=0 out_seq=0 2023-08-09T15:08:30.703 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:30.700+0000 7f4c57fff640 1 -- 172.21.15.43:0/4140737199 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4c50031780 con 0x7f4c600c86f0 2023-08-09T15:08:30.703 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:30.700+0000 7f4c57fff640 1 -- 172.21.15.43:0/4140737199 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4c500318e0 con 0x7f4c600c86f0 2023-08-09T15:08:30.703 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:30.700+0000 7f4c57fff640 1 -- 172.21.15.43:0/4140737199 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4c50031c50 con 0x7f4c600c86f0 2023-08-09T15:08:30.703 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:30.701+0000 7f4c67406640 1 -- 172.21.15.43:0/4140737199 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f4c600c86f0 msgr2=0x7f4c600c8b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T15:08:30.703 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:30.701+0000 7f4c67406640 1 --2- 172.21.15.43:0/4140737199 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f4c600c86f0 0x7f4c600c8b50 secure :-1 s=READY pgs=2144 cs=0 l=1 rev1=1 crypto rx=0x7f4c500097a0 tx=0x7f4c5002b820 comp rx=0 tx=0).stop 2023-08-09T15:08:30.703 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:30.701+0000 7f4c67406640 1 -- 172.21.15.43:0/4140737199 shutdown_connections 2023-08-09T15:08:30.703 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:30.701+0000 7f4c67406640 1 --2- 172.21.15.43:0/4140737199 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f4c6006c8d0 0x7f4c6006cd40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T15:08:30.704 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:30.701+0000 7f4c67406640 1 --2- 172.21.15.43:0/4140737199 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f4c600c86f0 0x7f4c600c8b50 unknown :-1 s=CLOSED pgs=2144 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T15:08:30.704 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:30.701+0000 7f4c67406640 1 -- 172.21.15.43:0/4140737199 >> 172.21.15.43:0/4140737199 conn(0x7f4c600ce0a0 msgr2=0x7f4c600ce480 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-09T15:08:30.704 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:30.701+0000 7f4c67406640 1 -- 172.21.15.43:0/4140737199 shutdown_connections 2023-08-09T15:08:30.704 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:30.701+0000 7f4c67406640 1 -- 172.21.15.43:0/4140737199 wait complete. 2023-08-09T15:08:30.704 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:30.703+0000 7f4c67406640 1 Processor -- start 2023-08-09T15:08:30.705 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:30.704+0000 7f4c67406640 1 -- start start 2023-08-09T15:08:30.706 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:30.705+0000 7f4c67406640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f4c6006c8d0 0x7f4c60142620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T15:08:30.706 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:30.705+0000 7f4c67406640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f4c600c86f0 0x7f4c60141eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T15:08:30.706 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:30.705+0000 7f4c67406640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f4c60143c20 0x7f4c600ac910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T15:08:30.706 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:30.705+0000 7f4c67406640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4c600c6840 con 0x7f4c60143c20 2023-08-09T15:08:30.706 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:30.705+0000 7f4c67406640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4c601f64b0 con 0x7f4c6006c8d0 2023-08-09T15:08:30.707 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:30.705+0000 7f4c67406640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4c601423f0 con 0x7f4c600c86f0 2023-08-09T15:08:30.708 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:30.705+0000 7f4c659a3640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f4c60143c20 0x7f4c600ac910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T15:08:30.708 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:30.705+0000 7f4c659a3640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f4c60143c20 0x7f4c600ac910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:36112/0 (socket says 172.21.15.43:36112) 2023-08-09T15:08:30.708 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:30.705+0000 7f4c659a3640 1 -- 172.21.15.43:0/757918132 learned_addr learned my addr 172.21.15.43:0/757918132 (peer_addr_for_me v2:172.21.15.43:0/0) 2023-08-09T15:08:30.708 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:30.705+0000 7f4c649a1640 1 --2- 172.21.15.43:0/757918132 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f4c600c86f0 0x7f4c60141eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T15:08:30.708 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:30.705+0000 7f4c651a2640 1 --2- 172.21.15.43:0/757918132 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f4c6006c8d0 0x7f4c60142620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T15:08:30.709 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:30.706+0000 7f4c659a3640 1 -- 172.21.15.43:0/757918132 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f4c600c86f0 msgr2=0x7f4c60141eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T15:08:30.709 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:30.706+0000 7f4c659a3640 1 --2- 172.21.15.43:0/757918132 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f4c600c86f0 0x7f4c60141eb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T15:08:30.709 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:30.706+0000 7f4c659a3640 1 -- 172.21.15.43:0/757918132 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f4c6006c8d0 msgr2=0x7f4c60142620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T15:08:30.709 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:30.706+0000 7f4c659a3640 1 --2- 172.21.15.43:0/757918132 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f4c6006c8d0 0x7f4c60142620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T15:08:30.709 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:30.706+0000 7f4c659a3640 1 -- 172.21.15.43:0/757918132 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4c50009400 con 0x7f4c60143c20 2023-08-09T15:08:30.709 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:30.706+0000 7f4c651a2640 1 --2- 172.21.15.43:0/757918132 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f4c6006c8d0 0x7f4c60142620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-08-09T15:08:30.709 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:30.706+0000 7f4c659a3640 1 --2- 172.21.15.43:0/757918132 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f4c60143c20 0x7f4c600ac910 secure :-1 s=READY pgs=2145 cs=0 l=1 rev1=1 crypto rx=0x7f4c5800ec10 tx=0x7f4c5800c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-09T15:08:30.709 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:30.706+0000 7f4c55ffb640 1 -- 172.21.15.43:0/757918132 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4c5800cd20 con 0x7f4c60143c20 2023-08-09T15:08:30.710 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:30.706+0000 7f4c55ffb640 1 -- 172.21.15.43:0/757918132 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4c5800f040 con 0x7f4c60143c20 2023-08-09T15:08:30.710 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:30.706+0000 7f4c67406640 1 -- 172.21.15.43:0/757918132 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4c601f56f0 con 0x7f4c60143c20 2023-08-09T15:08:30.710 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:30.706+0000 7f4c55ffb640 1 -- 172.21.15.43:0/757918132 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4c580146c0 con 0x7f4c60143c20 2023-08-09T15:08:30.710 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:30.706+0000 7f4c67406640 1 -- 172.21.15.43:0/757918132 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4c601f5cb0 con 0x7f4c60143c20 2023-08-09T15:08:30.711 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:30.707+0000 7f4c67406640 1 -- 172.21.15.43:0/757918132 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4c600739f0 con 0x7f4c60143c20 2023-08-09T15:08:30.711 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:30.708+0000 7f4c55ffb640 1 -- 172.21.15.43:0/757918132 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 106) v1 ==== 88443+0+0 (secure 0 0 0) 0x7f4c58014a30 con 0x7f4c60143c20 2023-08-09T15:08:30.711 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:30.709+0000 7f4c55ffb640 1 --2- 172.21.15.43:0/757918132 >> [v2:172.21.15.203:6832/3082732769,v1:172.21.15.203:6833/3082732769] conn(0x7f4c4406a170 0x7f4c4406c630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T15:08:30.711 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:30.710+0000 7f4c55ffb640 1 -- 172.21.15.43:0/757918132 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(194..194 src has 1..194) v4 ==== 7486+0+0 (secure 0 0 0) 0x7f4c58089a80 con 0x7f4c60143c20 2023-08-09T15:08:30.712 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:30.710+0000 7f4c651a2640 1 --2- 172.21.15.43:0/757918132 >> [v2:172.21.15.203:6832/3082732769,v1:172.21.15.203:6833/3082732769] conn(0x7f4c4406a170 0x7f4c4406c630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T15:08:30.712 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:30.711+0000 7f4c651a2640 1 --2- 172.21.15.43:0/757918132 >> [v2:172.21.15.203:6832/3082732769,v1:172.21.15.203:6833/3082732769] conn(0x7f4c4406a170 0x7f4c4406c630 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0x7f4c5c005880 tx=0x7f4c5c009040 comp rx=0 tx=0).ready entity=mgr.8500 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-09T15:08:30.715 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:30.714+0000 7f4c55ffb640 1 -- 172.21.15.43:0/757918132 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178066 (secure 0 0 0) 0x7f4c580549b0 con 0x7f4c60143c20 2023-08-09T15:08:30.956 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:30.954+0000 7f4c67406640 1 -- 172.21.15.43:0/757918132 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) v1 -- 0x7f4c600cec90 con 0x7f4c60143c20 2023-08-09T15:08:30.956 INFO:tasks.ceph.mon.a.smithi043.stderr:2023-08-09T15:08:30.955+0000 7f32867f3640 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2023-08-09T15:08:30.966 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:30.965+0000 7f4c55ffb640 1 -- 172.21.15.43:0/757918132 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v88) v1 ==== 161+0+0 (secure 0 0 0) 0x7f4c58057fd0 con 0x7f4c60143c20 2023-08-09T15:08:30.966 INFO:teuthology.orchestra.run.smithi043.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2023-08-09T15:08:30.969 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:30.968+0000 7f4c67406640 1 -- 172.21.15.43:0/757918132 >> [v2:172.21.15.203:6832/3082732769,v1:172.21.15.203:6833/3082732769] conn(0x7f4c4406a170 msgr2=0x7f4c4406c630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T15:08:30.969 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:30.968+0000 7f4c67406640 1 --2- 172.21.15.43:0/757918132 >> [v2:172.21.15.203:6832/3082732769,v1:172.21.15.203:6833/3082732769] conn(0x7f4c4406a170 0x7f4c4406c630 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0x7f4c5c005880 tx=0x7f4c5c009040 comp rx=0 tx=0).stop 2023-08-09T15:08:30.970 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:30.968+0000 7f4c67406640 1 -- 172.21.15.43:0/757918132 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f4c60143c20 msgr2=0x7f4c600ac910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T15:08:30.970 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:30.968+0000 7f4c67406640 1 --2- 172.21.15.43:0/757918132 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f4c60143c20 0x7f4c600ac910 secure :-1 s=READY pgs=2145 cs=0 l=1 rev1=1 crypto rx=0x7f4c5800ec10 tx=0x7f4c5800c490 comp rx=0 tx=0).stop 2023-08-09T15:08:30.970 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:30.968+0000 7f4c67406640 1 -- 172.21.15.43:0/757918132 shutdown_connections 2023-08-09T15:08:30.970 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:30.968+0000 7f4c67406640 1 --2- 172.21.15.43:0/757918132 >> [v2:172.21.15.203:6832/3082732769,v1:172.21.15.203:6833/3082732769] conn(0x7f4c4406a170 0x7f4c4406c630 unknown :-1 s=CLOSED pgs=31 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T15:08:30.970 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:30.968+0000 7f4c67406640 1 --2- 172.21.15.43:0/757918132 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f4c60143c20 0x7f4c600ac910 unknown :-1 s=CLOSED pgs=2145 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T15:08:30.970 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:30.968+0000 7f4c67406640 1 --2- 172.21.15.43:0/757918132 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f4c600c86f0 0x7f4c60141eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T15:08:30.970 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:30.968+0000 7f4c67406640 1 --2- 172.21.15.43:0/757918132 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f4c6006c8d0 0x7f4c60142620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T15:08:30.970 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:30.968+0000 7f4c67406640 1 -- 172.21.15.43:0/757918132 >> 172.21.15.43:0/757918132 conn(0x7f4c600ce0a0 msgr2=0x7f4c600c8ad0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-09T15:08:30.971 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:30.968+0000 7f4c67406640 1 -- 172.21.15.43:0/757918132 shutdown_connections 2023-08-09T15:08:30.971 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:30.968+0000 7f4c67406640 1 -- 172.21.15.43:0/757918132 wait complete. 2023-08-09T15:08:30.972 DEBUG:teuthology.orchestra.run.smithi043:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-08-09T15:08:30.977 DEBUG:teuthology.orchestra.run.smithi203:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-08-09T15:08:30.990 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs rm cephfs --yes-i-really-mean-it 2023-08-09T15:08:31.053 INFO:tasks.ceph.mds.c.smithi043.stdout:starting mds.c at 2023-08-09T15:08:31.241 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:31.240+0000 7f9b9d722640 1 Processor -- start 2023-08-09T15:08:31.242 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:31.240+0000 7f9b9d722640 1 -- start start 2023-08-09T15:08:31.243 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:31.242+0000 7f9b9d722640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f9b9815da80 0x7f9b98070db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T15:08:31.243 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:31.242+0000 7f9b9d722640 1 -- --> v1:172.21.15.43:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9b980c9d40 con 0x7f9b98159470 2023-08-09T15:08:31.243 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:31.242+0000 7f9b9d722640 1 -- --> v1:172.21.15.203:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9b980ca020 con 0x7f9b980712f0 2023-08-09T15:08:31.243 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:31.242+0000 7f9b9d722640 1 -- --> v2:172.21.15.43:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9b98070990 con 0x7f9b9815da80 2023-08-09T15:08:31.244 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:31.242+0000 7f9b96ffd640 1 --1- >> v1:172.21.15.43:6789/0 conn(0x7f9b98159470 0x7f9b980c9420 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.43:6789/0 says I am v1:172.21.15.43:41756/0 (socket says 172.21.15.43:41756) 2023-08-09T15:08:31.244 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:31.242+0000 7f9b96ffd640 1 -- 172.21.15.43:0/2101111915 learned_addr learned my addr 172.21.15.43:0/2101111915 (peer_addr_for_me v1:172.21.15.43:0/0) 2023-08-09T15:08:31.244 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:31.242+0000 7f9b967fc640 1 --2- 172.21.15.43:0/2101111915 >> v2:172.21.15.43:3300/0 conn(0x7f9b9815da80 0x7f9b98070db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T15:08:31.245 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:31.243+0000 7f9b967fc640 1 -- 172.21.15.43:0/2101111915 >> v1:172.21.15.203:6789/0 conn(0x7f9b980712f0 legacy=0x7f9b980cc8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-09T15:08:31.245 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:31.243+0000 7f9b967fc640 1 -- 172.21.15.43:0/2101111915 >> v1:172.21.15.43:6789/0 conn(0x7f9b98159470 legacy=0x7f9b980c9420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T15:08:31.245 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:31.243+0000 7f9b967fc640 1 -- 172.21.15.43:0/2101111915 --> v2:172.21.15.43:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9b980705b0 con 0x7f9b9815da80 2023-08-09T15:08:31.245 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:31.244+0000 7f9b967fc640 1 --2- 172.21.15.43:0/2101111915 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f9b9815da80 0x7f9b98070db0 secure :-1 s=READY pgs=2149 cs=0 l=1 rev1=1 crypto rx=0x7f9b8c0097a0 tx=0x7f9b8c02b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=12c981728b7fe81 server_cookie=0 in_seq=0 out_seq=0 2023-08-09T15:08:31.246 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:31.244+0000 7f9b95ffb640 1 -- 172.21.15.43:0/2101111915 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9b8c031830 con 0x7f9b9815da80 2023-08-09T15:08:31.246 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:31.244+0000 7f9b95ffb640 1 -- 172.21.15.43:0/2101111915 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9b8c031990 con 0x7f9b9815da80 2023-08-09T15:08:31.246 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:31.244+0000 7f9b95ffb640 1 -- 172.21.15.43:0/2101111915 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9b8c031d20 con 0x7f9b9815da80 2023-08-09T15:08:31.246 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:31.245+0000 7f9b9d722640 1 -- 172.21.15.43:0/2101111915 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f9b9815da80 msgr2=0x7f9b98070db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T15:08:31.246 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:31.245+0000 7f9b9d722640 1 --2- 172.21.15.43:0/2101111915 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f9b9815da80 0x7f9b98070db0 secure :-1 s=READY pgs=2149 cs=0 l=1 rev1=1 crypto rx=0x7f9b8c0097a0 tx=0x7f9b8c02b8d0 comp rx=0 tx=0).stop 2023-08-09T15:08:31.247 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:31.245+0000 7f9b9d722640 1 -- 172.21.15.43:0/2101111915 shutdown_connections 2023-08-09T15:08:31.247 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:31.245+0000 7f9b9d722640 1 --2- 172.21.15.43:0/2101111915 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f9b9815da80 0x7f9b98070db0 unknown :-1 s=CLOSED pgs=2149 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T15:08:31.247 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:31.245+0000 7f9b9d722640 1 -- 172.21.15.43:0/2101111915 >> 172.21.15.43:0/2101111915 conn(0x7f9b980aff50 msgr2=0x7f9b980b0360 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-09T15:08:31.247 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:31.245+0000 7f9b9d722640 1 -- 172.21.15.43:0/2101111915 shutdown_connections 2023-08-09T15:08:31.247 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:31.245+0000 7f9b9d722640 1 -- 172.21.15.43:0/2101111915 wait complete. 2023-08-09T15:08:31.248 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:31.247+0000 7f9b9d722640 1 Processor -- start 2023-08-09T15:08:31.248 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:31.247+0000 7f9b9d722640 1 -- start start 2023-08-09T15:08:31.249 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:31.248+0000 7f9b9d722640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f9b980712f0 0x7f9b98151ba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T15:08:31.249 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:31.248+0000 7f9b9d722640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f9b98159470 0x7f9b9815b1c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T15:08:31.250 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:31.248+0000 7f9b9d722640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f9b9815da80 0x7f9b980ca6b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T15:08:31.250 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:31.248+0000 7f9b9d722640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9b981520e0 con 0x7f9b98159470 2023-08-09T15:08:31.250 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:31.248+0000 7f9b9d722640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9b9815a500 con 0x7f9b9815da80 2023-08-09T15:08:31.250 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:31.248+0000 7f9b9d722640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9b981598f0 con 0x7f9b980712f0 2023-08-09T15:08:31.250 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:31.248+0000 7f9b967fc640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f9b98159470 0x7f9b9815b1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T15:08:31.250 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:31.248+0000 7f9b967fc640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f9b98159470 0x7f9b9815b1c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:36152/0 (socket says 172.21.15.43:36152) 2023-08-09T15:08:31.250 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:31.248+0000 7f9b967fc640 1 -- 172.21.15.43:0/455612315 learned_addr learned my addr 172.21.15.43:0/455612315 (peer_addr_for_me v2:172.21.15.43:0/0) 2023-08-09T15:08:31.251 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:31.249+0000 7f9b977fe640 1 --2- 172.21.15.43:0/455612315 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f9b9815da80 0x7f9b980ca6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T15:08:31.251 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:31.249+0000 7f9b96ffd640 1 --2- 172.21.15.43:0/455612315 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f9b980712f0 0x7f9b98151ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T15:08:31.252 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:31.249+0000 7f9b967fc640 1 -- 172.21.15.43:0/455612315 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f9b980712f0 msgr2=0x7f9b98151ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T15:08:31.252 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:31.249+0000 7f9b967fc640 1 --2- 172.21.15.43:0/455612315 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f9b980712f0 0x7f9b98151ba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T15:08:31.252 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:31.249+0000 7f9b967fc640 1 -- 172.21.15.43:0/455612315 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f9b9815da80 msgr2=0x7f9b980ca6b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T15:08:31.252 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:31.249+0000 7f9b967fc640 1 --2- 172.21.15.43:0/455612315 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f9b9815da80 0x7f9b980ca6b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T15:08:31.252 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:31.249+0000 7f9b967fc640 1 -- 172.21.15.43:0/455612315 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9b8c009400 con 0x7f9b98159470 2023-08-09T15:08:31.252 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:31.249+0000 7f9b977fe640 1 --2- 172.21.15.43:0/455612315 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f9b9815da80 0x7f9b980ca6b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-08-09T15:08:31.252 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:31.249+0000 7f9b967fc640 1 --2- 172.21.15.43:0/455612315 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f9b98159470 0x7f9b9815b1c0 secure :-1 s=READY pgs=2150 cs=0 l=1 rev1=1 crypto rx=0x7f9b8c0098c0 tx=0x7f9b8c0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-09T15:08:31.253 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:31.250+0000 7f9b7f7fe640 1 -- 172.21.15.43:0/455612315 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9b8c03ba70 con 0x7f9b98159470 2023-08-09T15:08:31.253 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:31.250+0000 7f9b7f7fe640 1 -- 172.21.15.43:0/455612315 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9b8c03bbf0 con 0x7f9b98159470 2023-08-09T15:08:31.253 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:31.250+0000 7f9b7f7fe640 1 -- 172.21.15.43:0/455612315 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9b8c0453f0 con 0x7f9b98159470 2023-08-09T15:08:31.253 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:31.250+0000 7f9b9d722640 1 -- 172.21.15.43:0/455612315 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9b98159bd0 con 0x7f9b98159470 2023-08-09T15:08:31.253 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:31.250+0000 7f9b9d722640 1 -- 172.21.15.43:0/455612315 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9b9815a190 con 0x7f9b98159470 2023-08-09T15:08:31.254 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:31.251+0000 7f9b9d722640 1 -- 172.21.15.43:0/455612315 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9b980739f0 con 0x7f9b98159470 2023-08-09T15:08:31.254 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:31.252+0000 7f9b7f7fe640 1 -- 172.21.15.43:0/455612315 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 106) v1 ==== 88443+0+0 (secure 0 0 0) 0x7f9b8c045550 con 0x7f9b98159470 2023-08-09T15:08:31.254 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:31.253+0000 7f9b7f7fe640 1 --2- 172.21.15.43:0/455612315 >> [v2:172.21.15.203:6832/3082732769,v1:172.21.15.203:6833/3082732769] conn(0x7f9b8406a170 0x7f9b8406c630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T15:08:31.254 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:31.253+0000 7f9b7f7fe640 1 -- 172.21.15.43:0/455612315 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(195..195 src has 1..195) v4 ==== 7544+0+0 (secure 0 0 0) 0x7f9b8c0ab2a0 con 0x7f9b98159470 2023-08-09T15:08:31.255 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:31.253+0000 7f9b96ffd640 1 --2- 172.21.15.43:0/455612315 >> [v2:172.21.15.203:6832/3082732769,v1:172.21.15.203:6833/3082732769] conn(0x7f9b8406a170 0x7f9b8406c630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T15:08:31.256 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:31.255+0000 7f9b96ffd640 1 --2- 172.21.15.43:0/455612315 >> [v2:172.21.15.203:6832/3082732769,v1:172.21.15.203:6833/3082732769] conn(0x7f9b8406a170 0x7f9b8406c630 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0x7f9b88005880 tx=0x7f9b88009040 comp rx=0 tx=0).ready entity=mgr.8500 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-09T15:08:31.258 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:31.257+0000 7f9b7f7fe640 1 -- 172.21.15.43:0/455612315 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178066 (secure 0 0 0) 0x7f9b8c076220 con 0x7f9b98159470 2023-08-09T15:08:31.493 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:31.492+0000 7f9b9d722640 1 -- 172.21.15.43:0/455612315 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} v 0) v1 -- 0x7f9b98066070 con 0x7f9b98159470 2023-08-09T15:08:31.968 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:31.966+0000 7f9b7f7fe640 1 -- 172.21.15.43:0/455612315 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]=0 v89) v1 ==== 104+0+0 (secure 0 0 0) 0x7f9b8c079840 con 0x7f9b98159470 2023-08-09T15:08:31.971 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:31.970+0000 7f9b9d722640 1 -- 172.21.15.43:0/455612315 >> [v2:172.21.15.203:6832/3082732769,v1:172.21.15.203:6833/3082732769] conn(0x7f9b8406a170 msgr2=0x7f9b8406c630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T15:08:31.971 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:31.970+0000 7f9b9d722640 1 --2- 172.21.15.43:0/455612315 >> [v2:172.21.15.203:6832/3082732769,v1:172.21.15.203:6833/3082732769] conn(0x7f9b8406a170 0x7f9b8406c630 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0x7f9b88005880 tx=0x7f9b88009040 comp rx=0 tx=0).stop 2023-08-09T15:08:31.971 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:31.970+0000 7f9b9d722640 1 -- 172.21.15.43:0/455612315 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f9b98159470 msgr2=0x7f9b9815b1c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T15:08:31.971 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:31.970+0000 7f9b9d722640 1 --2- 172.21.15.43:0/455612315 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f9b98159470 0x7f9b9815b1c0 secure :-1 s=READY pgs=2150 cs=0 l=1 rev1=1 crypto rx=0x7f9b8c0098c0 tx=0x7f9b8c0046c0 comp rx=0 tx=0).stop 2023-08-09T15:08:31.972 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:31.970+0000 7f9b9d722640 1 -- 172.21.15.43:0/455612315 shutdown_connections 2023-08-09T15:08:31.972 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:31.970+0000 7f9b9d722640 1 --2- 172.21.15.43:0/455612315 >> [v2:172.21.15.203:6832/3082732769,v1:172.21.15.203:6833/3082732769] conn(0x7f9b8406a170 0x7f9b8406c630 unknown :-1 s=CLOSED pgs=32 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T15:08:31.972 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:31.970+0000 7f9b9d722640 1 --2- 172.21.15.43:0/455612315 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f9b9815da80 0x7f9b980ca6b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T15:08:31.973 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:31.970+0000 7f9b9d722640 1 --2- 172.21.15.43:0/455612315 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f9b98159470 0x7f9b9815b1c0 unknown :-1 s=CLOSED pgs=2150 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T15:08:31.973 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:31.970+0000 7f9b9d722640 1 --2- 172.21.15.43:0/455612315 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f9b980712f0 0x7f9b98151ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T15:08:31.973 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:31.970+0000 7f9b9d722640 1 -- 172.21.15.43:0/455612315 >> 172.21.15.43:0/455612315 conn(0x7f9b980aff50 msgr2=0x7f9b9815bb00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-09T15:08:31.973 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:31.970+0000 7f9b9d722640 1 -- 172.21.15.43:0/455612315 shutdown_connections 2023-08-09T15:08:31.973 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:31.970+0000 7f9b9d722640 1 -- 172.21.15.43:0/455612315 wait complete. 2023-08-09T15:08:31.993 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_metadata cephfs_metadata --yes-i-really-really-mean-it 2023-08-09T15:08:32.230 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:32.229+0000 7fcea6e63640 1 Processor -- start 2023-08-09T15:08:32.230 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:32.229+0000 7fcea6e63640 1 -- start start 2023-08-09T15:08:32.231 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:32.230+0000 7fcea6e63640 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7fcea015c720 0x7fcea00cd850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T15:08:32.231 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:32.230+0000 7fcea6e63640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fcea00cf370 0x7fcea00b5de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T15:08:32.231 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:32.230+0000 7fcea6e63640 1 -- --> v1:172.21.15.43:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fcea00b5660 con 0x7fcea00d0900 2023-08-09T15:08:32.232 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:32.230+0000 7fcea6e63640 1 -- --> v2:172.21.15.203:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fcea015cb00 con 0x7fcea015c720 2023-08-09T15:08:32.232 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:32.230+0000 7fcea6e63640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcea00d1960 con 0x7fcea00cf370 2023-08-09T15:08:32.232 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:32.230+0000 7fcea5660640 1 --1- >> v1:172.21.15.43:6789/0 conn(0x7fcea00d0900 0x7fcea00cec60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.43:6789/0 says I am v1:172.21.15.43:41760/0 (socket says 172.21.15.43:41760) 2023-08-09T15:08:32.233 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:32.230+0000 7fcea5660640 1 -- 172.21.15.43:0/513329969 learned_addr learned my addr 172.21.15.43:0/513329969 (peer_addr_for_me v1:172.21.15.43:0/0) 2023-08-09T15:08:32.233 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:32.231+0000 7fcea5e61640 1 --2- 172.21.15.43:0/513329969 >> v2:172.21.15.203:3300/0 conn(0x7fcea015c720 0x7fcea00cd850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T15:08:32.233 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:32.231+0000 7fcea6662640 1 --2- 172.21.15.43:0/513329969 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fcea00cf370 0x7fcea00b5de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T15:08:32.233 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:32.231+0000 7fcea4e5f640 1 -- 172.21.15.43:0/513329969 <== mon.0 v1:172.21.15.43:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3864662862 0 0) 0x7fcea00b5660 con 0x7fcea00d0900 2023-08-09T15:08:32.233 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:32.231+0000 7fcea4e5f640 1 -- 172.21.15.43:0/513329969 --> v1:172.21.15.43:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fce8c003520 con 0x7fcea00d0900 2023-08-09T15:08:32.243 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:32.231+0000 7fcea4e5f640 1 -- 172.21.15.43:0/513329969 <== mon.0 v1:172.21.15.43:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1700364292 0 0) 0x7fce8c003520 con 0x7fcea00d0900 2023-08-09T15:08:32.243 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:32.231+0000 7fcea4e5f640 1 -- 172.21.15.43:0/513329969 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fcea00cf370 msgr2=0x7fcea00b5de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-09T15:08:32.243 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:32.231+0000 7fcea4e5f640 1 --2- 172.21.15.43:0/513329969 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fcea00cf370 0x7fcea00b5de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T15:08:32.244 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:32.231+0000 7fcea4e5f640 1 -- 172.21.15.43:0/513329969 >> v2:172.21.15.203:3300/0 conn(0x7fcea015c720 msgr2=0x7fcea00cd850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-09T15:08:32.244 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:32.231+0000 7fcea4e5f640 1 --2- 172.21.15.43:0/513329969 >> v2:172.21.15.203:3300/0 conn(0x7fcea015c720 0x7fcea00cd850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T15:08:32.244 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:32.231+0000 7fcea4e5f640 1 -- 172.21.15.43:0/513329969 --> v1:172.21.15.43:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcea00b5900 con 0x7fcea00d0900 2023-08-09T15:08:32.244 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:32.231+0000 7fcea4e5f640 1 -- 172.21.15.43:0/513329969 <== mon.0 v1:172.21.15.43:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1285150740 0 0) 0x7fce90001670 con 0x7fcea00d0900 2023-08-09T15:08:32.244 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:32.231+0000 7fcea6662640 1 --2- 172.21.15.43:0/513329969 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fcea00cf370 0x7fcea00b5de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-08-09T15:08:32.244 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:32.231+0000 7fcea5e61640 1 --2- 172.21.15.43:0/513329969 >> v2:172.21.15.203:3300/0 conn(0x7fcea015c720 0x7fcea00cd850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-08-09T15:08:32.244 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:32.231+0000 7fcea4e5f640 1 -- 172.21.15.43:0/513329969 >> v1:172.21.15.43:6789/0 conn(0x7fcea00d0900 legacy=0x7fcea00cec60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T15:08:32.245 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:32.232+0000 7fcea4e5f640 1 --2- 172.21.15.43:0/513329969 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fce8c0039a0 0x7fce8c003dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T15:08:32.245 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:32.232+0000 7fcea4e5f640 1 --2- 172.21.15.43:0/513329969 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fce8c0047c0 0x7fce8c004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T15:08:32.245 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:32.232+0000 7fcea4e5f640 1 --2- 172.21.15.43:0/513329969 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fcea00cf370 0x7fce8c005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T15:08:32.245 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:32.232+0000 7fcea4e5f640 1 -- 172.21.15.43:0/513329969 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcea015cb00 con 0x7fce8c0039a0 2023-08-09T15:08:32.245 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:32.232+0000 7fcea4e5f640 1 -- 172.21.15.43:0/513329969 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcea00d1960 con 0x7fce8c0047c0 2023-08-09T15:08:32.245 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:32.232+0000 7fcea4e5f640 1 -- 172.21.15.43:0/513329969 --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fce8c00ba60 con 0x7fcea00cf370 2023-08-09T15:08:32.245 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:32.232+0000 7fcea5e61640 1 --2- 172.21.15.43:0/513329969 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fce8c0039a0 0x7fce8c003dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T15:08:32.245 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:32.232+0000 7fcea5e61640 1 -- 172.21.15.43:0/513329969 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fcea00cf370 msgr2=0x7fce8c005180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-09T15:08:32.246 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:32.232+0000 7fcea5e61640 1 --2- 172.21.15.43:0/513329969 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fcea00cf370 0x7fce8c005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T15:08:32.246 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:32.232+0000 7fcea5e61640 1 -- 172.21.15.43:0/513329969 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fce8c0047c0 msgr2=0x7fce8c004c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-09T15:08:32.246 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:32.232+0000 7fcea5e61640 1 --2- 172.21.15.43:0/513329969 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fce8c0047c0 0x7fce8c004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T15:08:32.246 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:32.232+0000 7fcea5e61640 1 -- 172.21.15.43:0/513329969 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fce8c003520 con 0x7fce8c0039a0 2023-08-09T15:08:32.246 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:32.233+0000 7fcea5e61640 1 --2- 172.21.15.43:0/513329969 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fce8c0039a0 0x7fce8c003dc0 secure :-1 s=READY pgs=2152 cs=0 l=1 rev1=1 crypto rx=0x7fce9c006fd0 tx=0x7fce8c0057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c3bcd26634425cc0 server_cookie=0 in_seq=0 out_seq=0 2023-08-09T15:08:32.246 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:32.234+0000 7fcea4e5f640 1 -- 172.21.15.43:0/513329969 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fce9c004230 con 0x7fce8c0039a0 2023-08-09T15:08:32.246 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:32.234+0000 7fcea4e5f640 1 -- 172.21.15.43:0/513329969 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fce9c004390 con 0x7fce8c0039a0 2023-08-09T15:08:32.247 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:32.234+0000 7fcea4e5f640 1 -- 172.21.15.43:0/513329969 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fce9c003830 con 0x7fce8c0039a0 2023-08-09T15:08:32.247 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:32.234+0000 7fcea6e63640 1 -- 172.21.15.43:0/513329969 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fce8c0039a0 msgr2=0x7fce8c003dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T15:08:32.247 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:32.234+0000 7fcea6e63640 1 --2- 172.21.15.43:0/513329969 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fce8c0039a0 0x7fce8c003dc0 secure :-1 s=READY pgs=2152 cs=0 l=1 rev1=1 crypto rx=0x7fce9c006fd0 tx=0x7fce8c0057f0 comp rx=0 tx=0).stop 2023-08-09T15:08:32.247 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:32.234+0000 7fcea6662640 1 -- 172.21.15.43:0/513329969 reap_dead start 2023-08-09T15:08:32.247 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:32.234+0000 7fcea6e63640 1 -- 172.21.15.43:0/513329969 shutdown_connections 2023-08-09T15:08:32.247 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:32.234+0000 7fcea6e63640 1 -- 172.21.15.43:0/513329969 >> 172.21.15.43:0/513329969 conn(0x7fcea006a2a0 msgr2=0x7fcea006a680 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-09T15:08:32.247 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:32.234+0000 7fcea6e63640 1 -- 172.21.15.43:0/513329969 shutdown_connections 2023-08-09T15:08:32.247 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:32.234+0000 7fcea6e63640 1 -- 172.21.15.43:0/513329969 wait complete. 2023-08-09T15:08:32.248 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:32.236+0000 7fcea6e63640 1 Processor -- start 2023-08-09T15:08:32.248 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:32.236+0000 7fcea6e63640 1 -- start start 2023-08-09T15:08:32.248 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:32.237+0000 7fcea6e63640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fce8c0039a0 0x7fcea014fda0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T15:08:32.248 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:32.237+0000 7fcea6e63640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fcea0150360 0x7fcea015d650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T15:08:32.248 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:32.237+0000 7fcea6e63640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fcea00b3820 0x7fcea00b3ca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T15:08:32.248 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:32.237+0000 7fcea6e63640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcea015ebc0 con 0x7fcea0150360 2023-08-09T15:08:32.248 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:32.237+0000 7fcea6e63640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcea00b2dd0 con 0x7fcea00b3820 2023-08-09T15:08:32.249 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:32.237+0000 7fcea6e63640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcea00b3390 con 0x7fce8c0039a0 2023-08-09T15:08:32.249 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:32.238+0000 7fcea5660640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fcea0150360 0x7fcea015d650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T15:08:32.249 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:32.238+0000 7fcea5660640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fcea0150360 0x7fcea015d650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:36162/0 (socket says 172.21.15.43:36162) 2023-08-09T15:08:32.249 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:32.238+0000 7fcea5660640 1 -- 172.21.15.43:0/1805142134 learned_addr learned my addr 172.21.15.43:0/1805142134 (peer_addr_for_me v2:172.21.15.43:0/0) 2023-08-09T15:08:32.249 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:32.238+0000 7fcea5660640 1 -- 172.21.15.43:0/1805142134 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fce8c0039a0 msgr2=0x7fcea014fda0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T15:08:32.249 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:32.238+0000 7fcea5660640 1 --2- 172.21.15.43:0/1805142134 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fce8c0039a0 0x7fcea014fda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T15:08:32.249 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:32.238+0000 7fcea5660640 1 -- 172.21.15.43:0/1805142134 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fcea00b3820 msgr2=0x7fcea00b3ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T15:08:32.250 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:32.238+0000 7fcea5660640 1 --2- 172.21.15.43:0/1805142134 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fcea00b3820 0x7fcea00b3ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T15:08:32.250 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:32.238+0000 7fcea5660640 1 -- 172.21.15.43:0/1805142134 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcea015ee10 con 0x7fcea0150360 2023-08-09T15:08:32.250 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:32.238+0000 7fcea5660640 1 --2- 172.21.15.43:0/1805142134 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fcea0150360 0x7fcea015d650 secure :-1 s=READY pgs=2153 cs=0 l=1 rev1=1 crypto rx=0x7fce9000c730 tx=0x7fce9000cbf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-09T15:08:32.250 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:32.239+0000 7fce967fc640 1 -- 172.21.15.43:0/1805142134 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fce900074e0 con 0x7fcea0150360 2023-08-09T15:08:32.250 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:32.239+0000 7fce967fc640 1 -- 172.21.15.43:0/1805142134 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fce90007640 con 0x7fcea0150360 2023-08-09T15:08:32.250 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:32.239+0000 7fce967fc640 1 -- 172.21.15.43:0/1805142134 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fce90004a20 con 0x7fcea0150360 2023-08-09T15:08:32.250 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:32.239+0000 7fcea6e63640 1 -- 172.21.15.43:0/1805142134 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcea015f0f0 con 0x7fcea0150360 2023-08-09T15:08:32.251 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:32.239+0000 7fcea6e63640 1 -- 172.21.15.43:0/1805142134 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcea00cb250 con 0x7fcea0150360 2023-08-09T15:08:32.251 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:32.240+0000 7fcea6e63640 1 -- 172.21.15.43:0/1805142134 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcea00739f0 con 0x7fcea0150360 2023-08-09T15:08:32.251 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:32.241+0000 7fce967fc640 1 -- 172.21.15.43:0/1805142134 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 106) v1 ==== 88443+0+0 (secure 0 0 0) 0x7fce90004b80 con 0x7fcea0150360 2023-08-09T15:08:32.251 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:32.242+0000 7fce967fc640 1 --2- 172.21.15.43:0/1805142134 >> [v2:172.21.15.203:6832/3082732769,v1:172.21.15.203:6833/3082732769] conn(0x7fce8406a170 0x7fce8406c630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T15:08:32.251 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:32.242+0000 7fce967fc640 1 -- 172.21.15.43:0/1805142134 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(195..195 src has 1..195) v4 ==== 7544+0+0 (secure 0 0 0) 0x7fce90088110 con 0x7fcea0150360 2023-08-09T15:08:32.252 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:32.242+0000 7fcea5e61640 1 --2- 172.21.15.43:0/1805142134 >> [v2:172.21.15.203:6832/3082732769,v1:172.21.15.203:6833/3082732769] conn(0x7fce8406a170 0x7fce8406c630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T15:08:32.252 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:32.244+0000 7fcea5e61640 1 --2- 172.21.15.43:0/1805142134 >> [v2:172.21.15.203:6832/3082732769,v1:172.21.15.203:6833/3082732769] conn(0x7fce8406a170 0x7fce8406c630 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0x7fce9c035000 tx=0x7fce9c030000 comp rx=0 tx=0).ready entity=mgr.8500 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-09T15:08:32.252 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:32.246+0000 7fce967fc640 1 -- 172.21.15.43:0/1805142134 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178066 (secure 0 0 0) 0x7fce90053080 con 0x7fcea0150360 2023-08-09T15:08:32.443 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:32.442+0000 7fcea6e63640 1 -- 172.21.15.43:0/1805142134 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7fcea015bc80 con 0x7fcea0150360 2023-08-09T15:08:32.993 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:32.992+0000 7fce967fc640 1 -- 172.21.15.43:0/1805142134 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_metadata' removed v196) v1 ==== 181+0+0 (secure 0 0 0) 0x7fce900566a0 con 0x7fcea0150360 2023-08-09T15:08:32.994 INFO:teuthology.orchestra.run.smithi043.stderr:pool 'cephfs_metadata' removed 2023-08-09T15:08:32.998 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:32.996+0000 7fcea6e63640 1 -- 172.21.15.43:0/1805142134 >> [v2:172.21.15.203:6832/3082732769,v1:172.21.15.203:6833/3082732769] conn(0x7fce8406a170 msgr2=0x7fce8406c630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T15:08:32.998 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:32.996+0000 7fcea6e63640 1 --2- 172.21.15.43:0/1805142134 >> [v2:172.21.15.203:6832/3082732769,v1:172.21.15.203:6833/3082732769] conn(0x7fce8406a170 0x7fce8406c630 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0x7fce9c035000 tx=0x7fce9c030000 comp rx=0 tx=0).stop 2023-08-09T15:08:32.998 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:32.996+0000 7fcea6e63640 1 -- 172.21.15.43:0/1805142134 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fcea0150360 msgr2=0x7fcea015d650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T15:08:32.998 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:32.996+0000 7fcea6e63640 1 --2- 172.21.15.43:0/1805142134 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fcea0150360 0x7fcea015d650 secure :-1 s=READY pgs=2153 cs=0 l=1 rev1=1 crypto rx=0x7fce9000c730 tx=0x7fce9000cbf0 comp rx=0 tx=0).stop 2023-08-09T15:08:32.999 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:32.997+0000 7fcea6e63640 1 -- 172.21.15.43:0/1805142134 shutdown_connections 2023-08-09T15:08:32.999 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:32.997+0000 7fcea6e63640 1 --2- 172.21.15.43:0/1805142134 >> [v2:172.21.15.203:6832/3082732769,v1:172.21.15.203:6833/3082732769] conn(0x7fce8406a170 0x7fce8406c630 unknown :-1 s=CLOSED pgs=34 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T15:08:32.999 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:32.997+0000 7fcea6e63640 1 --2- 172.21.15.43:0/1805142134 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fcea00b3820 0x7fcea00b3ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T15:08:32.999 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:32.997+0000 7fcea6e63640 1 --2- 172.21.15.43:0/1805142134 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fcea0150360 0x7fcea015d650 unknown :-1 s=CLOSED pgs=2153 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T15:08:32.999 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:32.997+0000 7fcea6e63640 1 --2- 172.21.15.43:0/1805142134 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fce8c0039a0 0x7fcea014fda0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T15:08:33.000 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:32.997+0000 7fcea6e63640 1 -- 172.21.15.43:0/1805142134 >> 172.21.15.43:0/1805142134 conn(0x7fcea006a2a0 msgr2=0x7fcea00b7c40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-09T15:08:33.000 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:32.997+0000 7fcea6e63640 1 -- 172.21.15.43:0/1805142134 shutdown_connections 2023-08-09T15:08:33.000 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:32.998+0000 7fcea6e63640 1 -- 172.21.15.43:0/1805142134 wait complete. 2023-08-09T15:08:33.021 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_data cephfs_data --yes-i-really-really-mean-it 2023-08-09T15:08:33.249 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:33.248+0000 7f9a26afb640 1 Processor -- start 2023-08-09T15:08:33.250 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:33.248+0000 7f9a26afb640 1 -- start start 2023-08-09T15:08:33.251 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:33.250+0000 7f9a26afb640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f9a200c99b0 0x7f9a2015d3c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T15:08:33.251 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:33.250+0000 7f9a26afb640 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7f9a2006c510 0x7f9a2006c920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T15:08:33.251 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:33.250+0000 7f9a26afb640 1 -- --> v1:172.21.15.203:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9a20159c40 con 0x7f9a200c86c0 2023-08-09T15:08:33.251 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:33.250+0000 7f9a26afb640 1 -- --> v2:172.21.15.43:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9a20157000 con 0x7f9a200c99b0 2023-08-09T15:08:33.251 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:33.250+0000 7f9a26afb640 1 -- --> v2:172.21.15.203:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9a2006ce60 con 0x7f9a2006c510 2023-08-09T15:08:33.252 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:33.250+0000 7f9a25af9640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f9a200c99b0 0x7f9a2015d3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T15:08:33.252 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:33.250+0000 7f9a25af9640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f9a200c99b0 0x7f9a2015d3c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:36166/0 (socket says 172.21.15.43:36166) 2023-08-09T15:08:33.253 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:33.250+0000 7f9a25af9640 1 -- 172.21.15.43:0/4133441024 learned_addr learned my addr 172.21.15.43:0/4133441024 (peer_addr_for_me v2:172.21.15.43:0/0) 2023-08-09T15:08:33.253 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:33.250+0000 7f9a25af9640 1 -- 172.21.15.43:0/4133441024 >> v2:172.21.15.203:3300/0 conn(0x7f9a2006c510 msgr2=0x7f9a2006c920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-09T15:08:33.253 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:33.250+0000 7f9a25af9640 1 --2- 172.21.15.43:0/4133441024 >> v2:172.21.15.203:3300/0 conn(0x7f9a2006c510 0x7f9a2006c920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T15:08:33.253 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:33.251+0000 7f9a25af9640 1 -- 172.21.15.43:0/4133441024 >> v1:172.21.15.203:6789/0 conn(0x7f9a200c86c0 legacy=0x7f9a20159530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-09T15:08:33.253 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:33.251+0000 7f9a25af9640 1 -- 172.21.15.43:0/4133441024 --> v2:172.21.15.43:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9a20159ea0 con 0x7f9a200c99b0 2023-08-09T15:08:33.254 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:33.252+0000 7f9a25af9640 1 --2- 172.21.15.43:0/4133441024 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f9a200c99b0 0x7f9a2015d3c0 secure :-1 s=READY pgs=2154 cs=0 l=1 rev1=1 crypto rx=0x7f9a200717c0 tx=0x7f9a1c02e0f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=149be8e23d3d8fc4 server_cookie=0 in_seq=0 out_seq=0 2023-08-09T15:08:33.254 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:33.252+0000 7f9a24af7640 1 -- 172.21.15.43:0/4133441024 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9a1c02ec90 con 0x7f9a200c99b0 2023-08-09T15:08:33.255 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:33.252+0000 7f9a24af7640 1 -- 172.21.15.43:0/4133441024 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9a1c02edf0 con 0x7f9a200c99b0 2023-08-09T15:08:33.255 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:33.252+0000 7f9a24af7640 1 -- 172.21.15.43:0/4133441024 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9a1c036610 con 0x7f9a200c99b0 2023-08-09T15:08:33.255 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:33.253+0000 7f9a26afb640 1 -- 172.21.15.43:0/4133441024 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f9a200c99b0 msgr2=0x7f9a2015d3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T15:08:33.255 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:33.253+0000 7f9a26afb640 1 --2- 172.21.15.43:0/4133441024 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f9a200c99b0 0x7f9a2015d3c0 secure :-1 s=READY pgs=2154 cs=0 l=1 rev1=1 crypto rx=0x7f9a200717c0 tx=0x7f9a1c02e0f0 comp rx=0 tx=0).stop 2023-08-09T15:08:33.255 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:33.253+0000 7f9a26afb640 1 -- 172.21.15.43:0/4133441024 shutdown_connections 2023-08-09T15:08:33.255 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:33.253+0000 7f9a26afb640 1 --2- 172.21.15.43:0/4133441024 >> v2:172.21.15.203:3300/0 conn(0x7f9a2006c510 0x7f9a2006c920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T15:08:33.255 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:33.253+0000 7f9a26afb640 1 --2- 172.21.15.43:0/4133441024 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f9a200c99b0 0x7f9a2015d3c0 unknown :-1 s=CLOSED pgs=2154 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T15:08:33.256 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:33.253+0000 7f9a26afb640 1 -- 172.21.15.43:0/4133441024 >> 172.21.15.43:0/4133441024 conn(0x7f9a200ca870 msgr2=0x7f9a200cac50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-09T15:08:33.256 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:33.253+0000 7f9a26afb640 1 -- 172.21.15.43:0/4133441024 shutdown_connections 2023-08-09T15:08:33.256 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:33.253+0000 7f9a26afb640 1 -- 172.21.15.43:0/4133441024 wait complete. 2023-08-09T15:08:33.256 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:33.255+0000 7f9a26afb640 1 Processor -- start 2023-08-09T15:08:33.257 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:33.255+0000 7f9a26afb640 1 -- start start 2023-08-09T15:08:33.258 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:33.256+0000 7f9a26afb640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f9a2006c510 0x7f9a20145410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T15:08:33.258 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:33.256+0000 7f9a26afb640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f9a200c86c0 0x7f9a200c4ec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T15:08:33.258 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:33.256+0000 7f9a26afb640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f9a200c99b0 0x7f9a200c4130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T15:08:33.258 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:33.256+0000 7f9a26afb640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9a20145950 con 0x7f9a200c99b0 2023-08-09T15:08:33.259 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:33.256+0000 7f9a26afb640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9a200aeb00 con 0x7f9a2006c510 2023-08-09T15:08:33.259 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:33.256+0000 7f9a26afb640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9a200aede0 con 0x7f9a200c86c0 2023-08-09T15:08:33.259 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:33.256+0000 7f9a262fa640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f9a200c99b0 0x7f9a200c4130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T15:08:33.259 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:33.256+0000 7f9a262fa640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f9a200c99b0 0x7f9a200c4130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:36180/0 (socket says 172.21.15.43:36180) 2023-08-09T15:08:33.259 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:33.256+0000 7f9a262fa640 1 -- 172.21.15.43:0/2962686143 learned_addr learned my addr 172.21.15.43:0/2962686143 (peer_addr_for_me v2:172.21.15.43:0/0) 2023-08-09T15:08:33.260 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:33.256+0000 7f9a252f8640 1 --2- 172.21.15.43:0/2962686143 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f9a200c86c0 0x7f9a200c4ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T15:08:33.260 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:33.257+0000 7f9a25af9640 1 --2- 172.21.15.43:0/2962686143 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f9a2006c510 0x7f9a20145410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T15:08:33.260 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:33.257+0000 7f9a262fa640 1 -- 172.21.15.43:0/2962686143 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f9a200c86c0 msgr2=0x7f9a200c4ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T15:08:33.260 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:33.257+0000 7f9a262fa640 1 --2- 172.21.15.43:0/2962686143 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f9a200c86c0 0x7f9a200c4ec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T15:08:33.260 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:33.257+0000 7f9a262fa640 1 -- 172.21.15.43:0/2962686143 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f9a2006c510 msgr2=0x7f9a20145410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T15:08:33.260 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:33.257+0000 7f9a262fa640 1 --2- 172.21.15.43:0/2962686143 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f9a2006c510 0x7f9a20145410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T15:08:33.260 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:33.257+0000 7f9a262fa640 1 -- 172.21.15.43:0/2962686143 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9a1c009b30 con 0x7f9a200c99b0 2023-08-09T15:08:33.260 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:33.257+0000 7f9a252f8640 1 --2- 172.21.15.43:0/2962686143 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f9a200c86c0 0x7f9a200c4ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-08-09T15:08:33.261 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:33.257+0000 7f9a25af9640 1 --2- 172.21.15.43:0/2962686143 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f9a2006c510 0x7f9a20145410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-08-09T15:08:33.261 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:33.257+0000 7f9a262fa640 1 --2- 172.21.15.43:0/2962686143 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f9a200c99b0 0x7f9a200c4130 secure :-1 s=READY pgs=2155 cs=0 l=1 rev1=1 crypto rx=0x7f9a1800b410 tx=0x7f9a1800b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-09T15:08:33.262 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:33.257+0000 7f9a167fc640 1 -- 172.21.15.43:0/2962686143 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9a18011830 con 0x7f9a200c99b0 2023-08-09T15:08:33.262 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:33.257+0000 7f9a167fc640 1 -- 172.21.15.43:0/2962686143 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9a180119b0 con 0x7f9a200c99b0 2023-08-09T15:08:33.262 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:33.257+0000 7f9a167fc640 1 -- 172.21.15.43:0/2962686143 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9a18011d60 con 0x7f9a200c99b0 2023-08-09T15:08:33.262 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:33.257+0000 7f9a26afb640 1 -- 172.21.15.43:0/2962686143 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9a200af0c0 con 0x7f9a200c99b0 2023-08-09T15:08:33.262 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:33.257+0000 7f9a26afb640 1 -- 172.21.15.43:0/2962686143 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9a200af680 con 0x7f9a200c99b0 2023-08-09T15:08:33.262 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:33.258+0000 7f9a26afb640 1 -- 172.21.15.43:0/2962686143 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9a200739f0 con 0x7f9a200c99b0 2023-08-09T15:08:33.263 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:33.260+0000 7f9a167fc640 1 -- 172.21.15.43:0/2962686143 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 106) v1 ==== 88443+0+0 (secure 0 0 0) 0x7f9a1801a400 con 0x7f9a200c99b0 2023-08-09T15:08:33.263 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:33.260+0000 7f9a167fc640 1 --2- 172.21.15.43:0/2962686143 >> [v2:172.21.15.203:6832/3082732769,v1:172.21.15.203:6833/3082732769] conn(0x7f9a0406a170 0x7f9a0406c630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T15:08:33.263 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:33.261+0000 7f9a167fc640 1 -- 172.21.15.43:0/2962686143 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(196..196 src has 1..196) v4 ==== 7106+0+0 (secure 0 0 0) 0x7f9a18088dd0 con 0x7f9a200c99b0 2023-08-09T15:08:33.263 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:33.261+0000 7f9a25af9640 1 --2- 172.21.15.43:0/2962686143 >> [v2:172.21.15.203:6832/3082732769,v1:172.21.15.203:6833/3082732769] conn(0x7f9a0406a170 0x7f9a0406c630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T15:08:33.263 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:33.262+0000 7f9a25af9640 1 --2- 172.21.15.43:0/2962686143 >> [v2:172.21.15.203:6832/3082732769,v1:172.21.15.203:6833/3082732769] conn(0x7f9a0406a170 0x7f9a0406c630 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7f9a1c037000 tx=0x7f9a1c032000 comp rx=0 tx=0).ready entity=mgr.8500 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-09T15:08:33.266 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:33.265+0000 7f9a167fc640 1 -- 172.21.15.43:0/2962686143 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178066 (secure 0 0 0) 0x7f9a18053e60 con 0x7f9a200c99b0 2023-08-09T15:08:33.460 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:33.459+0000 7f9a26afb640 1 -- 172.21.15.43:0/2962686143 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7f9a200cc0c0 con 0x7f9a200c99b0 2023-08-09T15:08:34.001 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:34.000+0000 7f9a167fc640 1 -- 172.21.15.43:0/2962686143 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data' removed v197) v1 ==== 169+0+0 (secure 0 0 0) 0x7f9a18015070 con 0x7f9a200c99b0 2023-08-09T15:08:34.003 INFO:teuthology.orchestra.run.smithi043.stderr:pool 'cephfs_data' removed 2023-08-09T15:08:34.009 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:34.007+0000 7f9a26afb640 1 -- 172.21.15.43:0/2962686143 >> [v2:172.21.15.203:6832/3082732769,v1:172.21.15.203:6833/3082732769] conn(0x7f9a0406a170 msgr2=0x7f9a0406c630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T15:08:34.009 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:34.007+0000 7f9a26afb640 1 --2- 172.21.15.43:0/2962686143 >> [v2:172.21.15.203:6832/3082732769,v1:172.21.15.203:6833/3082732769] conn(0x7f9a0406a170 0x7f9a0406c630 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7f9a1c037000 tx=0x7f9a1c032000 comp rx=0 tx=0).stop 2023-08-09T15:08:34.009 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:34.007+0000 7f9a26afb640 1 -- 172.21.15.43:0/2962686143 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f9a200c99b0 msgr2=0x7f9a200c4130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T15:08:34.009 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:34.007+0000 7f9a26afb640 1 --2- 172.21.15.43:0/2962686143 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f9a200c99b0 0x7f9a200c4130 secure :-1 s=READY pgs=2155 cs=0 l=1 rev1=1 crypto rx=0x7f9a1800b410 tx=0x7f9a1800b8d0 comp rx=0 tx=0).stop 2023-08-09T15:08:34.011 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:34.010+0000 7f9a26afb640 1 -- 172.21.15.43:0/2962686143 shutdown_connections 2023-08-09T15:08:34.011 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:34.010+0000 7f9a26afb640 1 --2- 172.21.15.43:0/2962686143 >> [v2:172.21.15.203:6832/3082732769,v1:172.21.15.203:6833/3082732769] conn(0x7f9a0406a170 0x7f9a0406c630 unknown :-1 s=CLOSED pgs=36 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T15:08:34.011 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:34.010+0000 7f9a26afb640 1 --2- 172.21.15.43:0/2962686143 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f9a200c99b0 0x7f9a200c4130 unknown :-1 s=CLOSED pgs=2155 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T15:08:34.012 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:34.010+0000 7f9a26afb640 1 --2- 172.21.15.43:0/2962686143 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f9a200c86c0 0x7f9a200c4ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T15:08:34.012 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:34.010+0000 7f9a26afb640 1 --2- 172.21.15.43:0/2962686143 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f9a2006c510 0x7f9a20145410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T15:08:34.012 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:34.010+0000 7f9a26afb640 1 -- 172.21.15.43:0/2962686143 >> 172.21.15.43:0/2962686143 conn(0x7f9a200ca870 msgr2=0x7f9a20159160 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-09T15:08:34.012 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:34.010+0000 7f9a26afb640 1 -- 172.21.15.43:0/2962686143 shutdown_connections 2023-08-09T15:08:34.012 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:34.010+0000 7f9a26afb640 1 -- 172.21.15.43:0/2962686143 wait complete. 2023-08-09T15:08:34.036 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config rm mgr mgr/snap_schedule/dump_on_update 2023-08-09T15:08:34.265 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:34.263+0000 7f2b8e786640 1 Processor -- start 2023-08-09T15:08:34.265 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:34.264+0000 7f2b8e786640 1 -- start start 2023-08-09T15:08:34.266 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:34.265+0000 7f2b8e786640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f2b880cbde0 0x7f2b8806fa20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T15:08:34.266 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:34.265+0000 7f2b8e786640 1 -- --> v1:172.21.15.43:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2b880acfa0 con 0x7f2b880cb9a0 2023-08-09T15:08:34.266 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:34.265+0000 7f2b8e786640 1 -- --> v1:172.21.15.203:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2b8815c940 con 0x7f2b88070ef0 2023-08-09T15:08:34.267 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:34.265+0000 7f2b8e786640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2b880ac390 con 0x7f2b880cbde0 2023-08-09T15:08:34.267 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:34.265+0000 7f2b8cf83640 1 --1- >> v1:172.21.15.43:6789/0 conn(0x7f2b880cb9a0 0x7f2b8806f310 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.43:6789/0 says I am v1:172.21.15.43:41762/0 (socket says 172.21.15.43:41762) 2023-08-09T15:08:34.267 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:34.265+0000 7f2b8cf83640 1 -- 172.21.15.43:0/2148935255 learned_addr learned my addr 172.21.15.43:0/2148935255 (peer_addr_for_me v1:172.21.15.43:0/0) 2023-08-09T15:08:34.267 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:34.266+0000 7f2b8df85640 1 --2- 172.21.15.43:0/2148935255 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f2b880cbde0 0x7f2b8806fa20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T15:08:34.268 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:34.266+0000 7f2b87fff640 1 -- 172.21.15.43:0/2148935255 <== mon.0 v1:172.21.15.43:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3277893699 0 0) 0x7f2b880acfa0 con 0x7f2b880cb9a0 2023-08-09T15:08:34.268 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:34.266+0000 7f2b87fff640 1 -- 172.21.15.43:0/2148935255 --> v1:172.21.15.43:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f2b70003520 con 0x7f2b880cb9a0 2023-08-09T15:08:34.269 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:34.266+0000 7f2b87fff640 1 -- 172.21.15.43:0/2148935255 <== mon.0 v1:172.21.15.43:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2421993402 0 0) 0x7f2b70003520 con 0x7f2b880cb9a0 2023-08-09T15:08:34.269 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:34.266+0000 7f2b87fff640 1 -- 172.21.15.43:0/2148935255 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f2b880cbde0 msgr2=0x7f2b8806fa20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-09T15:08:34.269 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:34.266+0000 7f2b87fff640 1 --2- 172.21.15.43:0/2148935255 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f2b880cbde0 0x7f2b8806fa20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T15:08:34.269 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:34.266+0000 7f2b87fff640 1 -- 172.21.15.43:0/2148935255 >> v1:172.21.15.203:6789/0 conn(0x7f2b88070ef0 legacy=0x7f2b880ab8d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T15:08:34.270 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:34.266+0000 7f2b87fff640 1 -- 172.21.15.43:0/2148935255 --> v1:172.21.15.43:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2b880ac670 con 0x7f2b880cb9a0 2023-08-09T15:08:34.270 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:34.266+0000 7f2b87fff640 1 -- 172.21.15.43:0/2148935255 <== mon.0 v1:172.21.15.43:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1285150740 0 0) 0x7f2b7c0019f0 con 0x7f2b880cb9a0 2023-08-09T15:08:34.270 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:34.266+0000 7f2b87fff640 1 -- 172.21.15.43:0/2148935255 >> v1:172.21.15.43:6789/0 conn(0x7f2b880cb9a0 legacy=0x7f2b8806f310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T15:08:34.270 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:34.266+0000 7f2b87fff640 1 --2- 172.21.15.43:0/2148935255 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f2b880cbde0 0x7f2b700039a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T15:08:34.270 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:34.267+0000 7f2b87fff640 1 --2- 172.21.15.43:0/2148935255 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f2b70003f20 0x7f2b700047c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T15:08:34.270 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:34.267+0000 7f2b87fff640 1 --2- 172.21.15.43:0/2148935255 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f2b70004d00 0x7f2b70005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T15:08:34.270 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:34.267+0000 7f2b87fff640 1 -- 172.21.15.43:0/2148935255 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2b880ac390 con 0x7f2b70003f20 2023-08-09T15:08:34.271 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:34.267+0000 7f2b87fff640 1 -- 172.21.15.43:0/2148935255 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2b7000baf0 con 0x7f2b70004d00 2023-08-09T15:08:34.271 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:34.267+0000 7f2b87fff640 1 -- 172.21.15.43:0/2148935255 --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2b7000bcb0 con 0x7f2b880cbde0 2023-08-09T15:08:34.271 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:34.267+0000 7f2b8cf83640 1 --2- 172.21.15.43:0/2148935255 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f2b70003f20 0x7f2b700047c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T15:08:34.271 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:34.267+0000 7f2b8d784640 1 --2- 172.21.15.43:0/2148935255 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f2b880cbde0 0x7f2b700039a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T15:08:34.271 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:34.267+0000 7f2b8df85640 1 --2- 172.21.15.43:0/2148935255 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f2b70004d00 0x7f2b70005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T15:08:34.271 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:34.267+0000 7f2b8cf83640 1 -- 172.21.15.43:0/2148935255 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f2b880cbde0 msgr2=0x7f2b700039a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-09T15:08:34.271 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:34.267+0000 7f2b8cf83640 1 --2- 172.21.15.43:0/2148935255 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f2b880cbde0 0x7f2b700039a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T15:08:34.271 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:34.267+0000 7f2b8cf83640 1 -- 172.21.15.43:0/2148935255 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f2b70004d00 msgr2=0x7f2b70005180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-09T15:08:34.272 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:34.267+0000 7f2b8cf83640 1 --2- 172.21.15.43:0/2148935255 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f2b70004d00 0x7f2b70005180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T15:08:34.272 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:34.267+0000 7f2b8cf83640 1 -- 172.21.15.43:0/2148935255 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2b70003520 con 0x7f2b70003f20 2023-08-09T15:08:34.272 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:34.268+0000 7f2b8df85640 1 --2- 172.21.15.43:0/2148935255 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f2b70004d00 0x7f2b70005180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-08-09T15:08:34.273 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:34.269+0000 7f2b8cf83640 1 --2- 172.21.15.43:0/2148935255 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f2b70003f20 0x7f2b700047c0 secure :-1 s=READY pgs=2157 cs=0 l=1 rev1=1 crypto rx=0x7f2b7c00a830 tx=0x7f2b700057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6ecdb6ab4bf249fa server_cookie=0 in_seq=0 out_seq=0 2023-08-09T15:08:34.273 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:34.269+0000 7f2b87fff640 1 -- 172.21.15.43:0/2148935255 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2b7c0072d0 con 0x7f2b70003f20 2023-08-09T15:08:34.273 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:34.269+0000 7f2b87fff640 1 -- 172.21.15.43:0/2148935255 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2b7c02f070 con 0x7f2b70003f20 2023-08-09T15:08:34.273 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:34.269+0000 7f2b87fff640 1 -- 172.21.15.43:0/2148935255 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2b7c0076f0 con 0x7f2b70003f20 2023-08-09T15:08:34.274 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:34.270+0000 7f2b8e786640 1 -- 172.21.15.43:0/2148935255 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f2b70003f20 msgr2=0x7f2b700047c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T15:08:34.274 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:34.270+0000 7f2b8e786640 1 --2- 172.21.15.43:0/2148935255 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f2b70003f20 0x7f2b700047c0 secure :-1 s=READY pgs=2157 cs=0 l=1 rev1=1 crypto rx=0x7f2b7c00a830 tx=0x7f2b700057f0 comp rx=0 tx=0).stop 2023-08-09T15:08:34.274 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:34.270+0000 7f2b8df85640 1 -- 172.21.15.43:0/2148935255 reap_dead start 2023-08-09T15:08:34.274 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:34.270+0000 7f2b8e786640 1 -- 172.21.15.43:0/2148935255 shutdown_connections 2023-08-09T15:08:34.274 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:34.270+0000 7f2b8e786640 1 -- 172.21.15.43:0/2148935255 >> 172.21.15.43:0/2148935255 conn(0x7f2b880cddd0 msgr2=0x7f2b880ce1e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-09T15:08:34.274 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:34.270+0000 7f2b8e786640 1 -- 172.21.15.43:0/2148935255 shutdown_connections 2023-08-09T15:08:34.274 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:34.270+0000 7f2b8e786640 1 -- 172.21.15.43:0/2148935255 wait complete. 2023-08-09T15:08:34.274 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:34.272+0000 7f2b8e786640 1 Processor -- start 2023-08-09T15:08:34.275 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:34.272+0000 7f2b8e786640 1 -- start start 2023-08-09T15:08:34.276 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:34.273+0000 7f2b8e786640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f2b70003f20 0x7f2b881f1cb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T15:08:34.276 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:34.273+0000 7f2b8e786640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f2b880b1700 0x7f2b881f49d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T15:08:34.276 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:34.273+0000 7f2b8e786640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f2b880c5d00 0x7f2b8815ed00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T15:08:34.277 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:34.273+0000 7f2b8e786640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2b881f21f0 con 0x7f2b70003f20 2023-08-09T15:08:34.278 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:34.273+0000 7f2b8e786640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2b880c6950 con 0x7f2b880c5d00 2023-08-09T15:08:34.278 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:34.273+0000 7f2b8e786640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2b880c6670 con 0x7f2b880b1700 2023-08-09T15:08:34.278 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:34.273+0000 7f2b8d784640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f2b70003f20 0x7f2b881f1cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T15:08:34.278 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:34.273+0000 7f2b8d784640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f2b70003f20 0x7f2b881f1cb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:36192/0 (socket says 172.21.15.43:36192) 2023-08-09T15:08:34.278 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:34.273+0000 7f2b8d784640 1 -- 172.21.15.43:0/976234776 learned_addr learned my addr 172.21.15.43:0/976234776 (peer_addr_for_me v2:172.21.15.43:0/0) 2023-08-09T15:08:34.278 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:34.273+0000 7f2b8cf83640 1 --2- 172.21.15.43:0/976234776 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f2b880b1700 0x7f2b881f49d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T15:08:34.278 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:34.274+0000 7f2b8d784640 1 -- 172.21.15.43:0/976234776 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f2b880b1700 msgr2=0x7f2b881f49d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T15:08:34.279 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:34.274+0000 7f2b8d784640 1 --2- 172.21.15.43:0/976234776 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f2b880b1700 0x7f2b881f49d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T15:08:34.279 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:34.274+0000 7f2b8d784640 1 -- 172.21.15.43:0/976234776 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f2b880c5d00 msgr2=0x7f2b8815ed00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T15:08:34.279 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:34.274+0000 7f2b8df85640 1 --2- 172.21.15.43:0/976234776 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f2b880c5d00 0x7f2b8815ed00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T15:08:34.279 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:34.274+0000 7f2b8d784640 1 --2- 172.21.15.43:0/976234776 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f2b880c5d00 0x7f2b8815ed00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T15:08:34.279 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:34.274+0000 7f2b8d784640 1 -- 172.21.15.43:0/976234776 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2b880b23e0 con 0x7f2b70003f20 2023-08-09T15:08:34.279 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:34.274+0000 7f2b8cf83640 1 --2- 172.21.15.43:0/976234776 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f2b880b1700 0x7f2b881f49d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-08-09T15:08:34.279 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:34.274+0000 7f2b8df85640 1 --2- 172.21.15.43:0/976234776 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f2b880c5d00 0x7f2b8815ed00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-08-09T15:08:34.279 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:34.274+0000 7f2b8d784640 1 --2- 172.21.15.43:0/976234776 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f2b70003f20 0x7f2b881f1cb0 secure :-1 s=READY pgs=2158 cs=0 l=1 rev1=1 crypto rx=0x7f2b7800e550 tx=0x7f2b7800ea10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-09T15:08:34.280 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:34.274+0000 7f2b85ffb640 1 -- 172.21.15.43:0/976234776 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2b780052e0 con 0x7f2b70003f20 2023-08-09T15:08:34.280 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:34.274+0000 7f2b85ffb640 1 -- 172.21.15.43:0/976234776 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2b78005440 con 0x7f2b70003f20 2023-08-09T15:08:34.280 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:34.274+0000 7f2b85ffb640 1 -- 172.21.15.43:0/976234776 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2b78003810 con 0x7f2b70003f20 2023-08-09T15:08:34.280 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:34.274+0000 7f2b8e786640 1 -- 172.21.15.43:0/976234776 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2b880b26c0 con 0x7f2b70003f20 2023-08-09T15:08:34.280 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:34.275+0000 7f2b8e786640 1 -- 172.21.15.43:0/976234776 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2b881f4f10 con 0x7f2b70003f20 2023-08-09T15:08:34.281 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:34.276+0000 7f2b8e786640 1 -- 172.21.15.43:0/976234776 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2b881f50a0 con 0x7f2b70003f20 2023-08-09T15:08:34.281 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:34.277+0000 7f2b85ffb640 1 -- 172.21.15.43:0/976234776 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 106) v1 ==== 88443+0+0 (secure 0 0 0) 0x7f2b78003970 con 0x7f2b70003f20 2023-08-09T15:08:34.281 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:34.277+0000 7f2b85ffb640 1 --2- 172.21.15.43:0/976234776 >> [v2:172.21.15.203:6832/3082732769,v1:172.21.15.203:6833/3082732769] conn(0x7f2b7406a1a0 0x7f2b7406c660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T15:08:34.281 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:34.278+0000 7f2b85ffb640 1 -- 172.21.15.43:0/976234776 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(197..197 src has 1..197) v4 ==== 6692+0+0 (secure 0 0 0) 0x7f2b78088710 con 0x7f2b70003f20 2023-08-09T15:08:34.281 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:34.278+0000 7f2b8cf83640 1 --2- 172.21.15.43:0/976234776 >> [v2:172.21.15.203:6832/3082732769,v1:172.21.15.203:6833/3082732769] conn(0x7f2b7406a1a0 0x7f2b7406c660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T15:08:34.282 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:34.279+0000 7f2b8cf83640 1 --2- 172.21.15.43:0/976234776 >> [v2:172.21.15.203:6832/3082732769,v1:172.21.15.203:6833/3082732769] conn(0x7f2b7406a1a0 0x7f2b7406c660 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7f2b7c034000 tx=0x7f2b7c007000 comp rx=0 tx=0).ready entity=mgr.8500 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-09T15:08:34.283 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:34.282+0000 7f2b85ffb640 1 -- 172.21.15.43:0/976234776 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178066 (secure 0 0 0) 0x7f2b78053a70 con 0x7f2b70003f20 2023-08-09T15:08:34.482 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:34.481+0000 7f2b8e786640 1 -- 172.21.15.43:0/976234776 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "config rm", "who": "mgr", "name": "mgr/snap_schedule/dump_on_update"} v 0) v1 -- 0x7f2b880b2240 con 0x7f2b70003f20 2023-08-09T15:08:34.491 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:34.489+0000 7f2b85ffb640 1 -- 172.21.15.43:0/976234776 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "config rm", "who": "mgr", "name": "mgr/snap_schedule/dump_on_update"}]=0 v61) v1 ==== 115+0+0 (secure 0 0 0) 0x7f2b880b2240 con 0x7f2b70003f20 2023-08-09T15:08:34.493 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:34.492+0000 7f2b8e786640 1 -- 172.21.15.43:0/976234776 >> [v2:172.21.15.203:6832/3082732769,v1:172.21.15.203:6833/3082732769] conn(0x7f2b7406a1a0 msgr2=0x7f2b7406c660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T15:08:34.494 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:34.492+0000 7f2b8e786640 1 --2- 172.21.15.43:0/976234776 >> [v2:172.21.15.203:6832/3082732769,v1:172.21.15.203:6833/3082732769] conn(0x7f2b7406a1a0 0x7f2b7406c660 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7f2b7c034000 tx=0x7f2b7c007000 comp rx=0 tx=0).stop 2023-08-09T15:08:34.494 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:34.492+0000 7f2b8e786640 1 -- 172.21.15.43:0/976234776 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f2b70003f20 msgr2=0x7f2b881f1cb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T15:08:34.494 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:34.492+0000 7f2b8e786640 1 --2- 172.21.15.43:0/976234776 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f2b70003f20 0x7f2b881f1cb0 secure :-1 s=READY pgs=2158 cs=0 l=1 rev1=1 crypto rx=0x7f2b7800e550 tx=0x7f2b7800ea10 comp rx=0 tx=0).stop 2023-08-09T15:08:34.494 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:34.492+0000 7f2b8e786640 1 -- 172.21.15.43:0/976234776 shutdown_connections 2023-08-09T15:08:34.494 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:34.492+0000 7f2b8e786640 1 --2- 172.21.15.43:0/976234776 >> [v2:172.21.15.203:6832/3082732769,v1:172.21.15.203:6833/3082732769] conn(0x7f2b7406a1a0 0x7f2b7406c660 unknown :-1 s=CLOSED pgs=37 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T15:08:34.494 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:34.492+0000 7f2b8e786640 1 --2- 172.21.15.43:0/976234776 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f2b880c5d00 0x7f2b8815ed00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T15:08:34.494 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:34.492+0000 7f2b8e786640 1 --2- 172.21.15.43:0/976234776 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f2b880b1700 0x7f2b881f49d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T15:08:34.495 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:34.492+0000 7f2b8e786640 1 --2- 172.21.15.43:0/976234776 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f2b70003f20 0x7f2b881f1cb0 unknown :-1 s=CLOSED pgs=2158 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T15:08:34.495 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:34.492+0000 7f2b8e786640 1 -- 172.21.15.43:0/976234776 >> 172.21.15.43:0/976234776 conn(0x7f2b880cddd0 msgr2=0x7f2b880ab8d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-09T15:08:34.495 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:34.492+0000 7f2b8e786640 1 -- 172.21.15.43:0/976234776 shutdown_connections 2023-08-09T15:08:34.495 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:34.492+0000 7f2b8e786640 1 -- 172.21.15.43:0/976234776 wait complete. 2023-08-09T15:08:34.513 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config rm mgr mgr/snap_schedule/allow_m_granularity 2023-08-09T15:08:34.733 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:34.732+0000 7f5fec9e0640 1 Processor -- start 2023-08-09T15:08:34.734 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:34.732+0000 7f5fec9e0640 1 -- start start 2023-08-09T15:08:34.735 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:34.734+0000 7f5fec9e0640 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7f5fe8157ea0 0x7f5fe80c97d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T15:08:34.735 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:34.734+0000 7f5fec9e0640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f5fe80c90a0 0x7f5fe80c9d10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T15:08:34.735 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:34.734+0000 7f5fec9e0640 1 -- --> v1:172.21.15.203:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5fe8158280 con 0x7f5fe815a800 2023-08-09T15:08:34.736 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:34.734+0000 7f5fec9e0640 1 -- --> v2:172.21.15.203:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5fe815b380 con 0x7f5fe8157ea0 2023-08-09T15:08:34.736 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:34.734+0000 7f5fec9e0640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5fe815b840 con 0x7f5fe80c90a0 2023-08-09T15:08:34.736 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:34.735+0000 7f5fdffff640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f5fe80c90a0 0x7f5fe80c9d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T15:08:34.736 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:34.735+0000 7f5fe759e640 1 --1- >> v1:172.21.15.203:6789/0 conn(0x7f5fe815a800 0x7f5fe815ac70 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.203:6789/0 says I am v1:172.21.15.43:53358/0 (socket says 172.21.15.43:53358) 2023-08-09T15:08:34.737 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:34.735+0000 7f5fe759e640 1 -- 172.21.15.43:0/3528223407 learned_addr learned my addr 172.21.15.43:0/3528223407 (peer_addr_for_me v1:172.21.15.43:0/0) 2023-08-09T15:08:34.737 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:34.735+0000 7f5fe6d9d640 1 --2- 172.21.15.43:0/3528223407 >> v2:172.21.15.203:3300/0 conn(0x7f5fe8157ea0 0x7f5fe80c97d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T15:08:34.737 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:34.736+0000 7f5fdffff640 1 -- 172.21.15.43:0/3528223407 >> v2:172.21.15.203:3300/0 conn(0x7f5fe8157ea0 msgr2=0x7f5fe80c97d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-09T15:08:34.737 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:34.736+0000 7f5fdffff640 1 --2- 172.21.15.43:0/3528223407 >> v2:172.21.15.203:3300/0 conn(0x7f5fe8157ea0 0x7f5fe80c97d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T15:08:34.738 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:34.736+0000 7f5fdffff640 1 -- 172.21.15.43:0/3528223407 >> v1:172.21.15.203:6789/0 conn(0x7f5fe815a800 legacy=0x7f5fe815ac70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T15:08:34.738 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:34.736+0000 7f5fdffff640 1 -- 172.21.15.43:0/3528223407 --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5fe815b560 con 0x7f5fe80c90a0 2023-08-09T15:08:34.738 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:34.737+0000 7f5fdffff640 1 --2- 172.21.15.43:0/3528223407 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f5fe80c90a0 0x7f5fe80c9d10 secure :-1 s=READY pgs=310 cs=0 l=1 rev1=1 crypto rx=0x7f5fd00097a0 tx=0x7f5fd002b820 comp rx=0 tx=0).ready entity=mon.2 client_cookie=6f6e92ee4b283efd server_cookie=0 in_seq=0 out_seq=0 2023-08-09T15:08:34.739 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:34.738+0000 7f5fe659c640 1 -- 172.21.15.43:0/3528223407 <== mon.2 v2:172.21.15.203:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5fd0031780 con 0x7f5fe80c90a0 2023-08-09T15:08:34.740 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:34.738+0000 7f5fe659c640 1 -- 172.21.15.43:0/3528223407 <== mon.2 v2:172.21.15.203:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5fd00318e0 con 0x7f5fe80c90a0 2023-08-09T15:08:34.740 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:34.738+0000 7f5fe659c640 1 -- 172.21.15.43:0/3528223407 <== mon.2 v2:172.21.15.203:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5fd0031c70 con 0x7f5fe80c90a0 2023-08-09T15:08:34.740 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:34.739+0000 7f5fec9e0640 1 -- 172.21.15.43:0/3528223407 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f5fe80c90a0 msgr2=0x7f5fe80c9d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T15:08:34.740 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:34.739+0000 7f5fec9e0640 1 --2- 172.21.15.43:0/3528223407 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f5fe80c90a0 0x7f5fe80c9d10 secure :-1 s=READY pgs=310 cs=0 l=1 rev1=1 crypto rx=0x7f5fd00097a0 tx=0x7f5fd002b820 comp rx=0 tx=0).stop 2023-08-09T15:08:34.741 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:34.739+0000 7f5fec9e0640 1 -- 172.21.15.43:0/3528223407 shutdown_connections 2023-08-09T15:08:34.741 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:34.739+0000 7f5fec9e0640 1 --2- 172.21.15.43:0/3528223407 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f5fe80c90a0 0x7f5fe80c9d10 unknown :-1 s=CLOSED pgs=310 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T15:08:34.741 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:34.739+0000 7f5fec9e0640 1 --2- 172.21.15.43:0/3528223407 >> v2:172.21.15.203:3300/0 conn(0x7f5fe8157ea0 0x7f5fe80c97d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T15:08:34.741 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:34.739+0000 7f5fec9e0640 1 -- 172.21.15.43:0/3528223407 >> 172.21.15.43:0/3528223407 conn(0x7f5fe80cddd0 msgr2=0x7f5fe80ce1e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-09T15:08:34.741 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:34.739+0000 7f5fec9e0640 1 -- 172.21.15.43:0/3528223407 shutdown_connections 2023-08-09T15:08:34.741 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:34.739+0000 7f5fec9e0640 1 -- 172.21.15.43:0/3528223407 wait complete. 2023-08-09T15:08:34.742 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:34.741+0000 7f5fec9e0640 1 Processor -- start 2023-08-09T15:08:34.742 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:34.741+0000 7f5fec9e0640 1 -- start start 2023-08-09T15:08:34.743 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:34.742+0000 7f5fec9e0640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f5fe80c90a0 0x7f5fe81f13c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T15:08:34.743 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:34.742+0000 7f5fec9e0640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f5fe8157ea0 0x7f5fe80b1a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T15:08:34.743 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:34.742+0000 7f5fec9e0640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f5fe815a800 0x7f5fe80b1f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T15:08:34.743 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:34.742+0000 7f5fec9e0640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5fe80c2440 con 0x7f5fe815a800 2023-08-09T15:08:34.743 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:34.742+0000 7f5fec9e0640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5fe80c2860 con 0x7f5fe80c90a0 2023-08-09T15:08:34.744 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:34.742+0000 7f5fec9e0640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5fe80cd870 con 0x7f5fe8157ea0 2023-08-09T15:08:34.745 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:34.742+0000 7f5fe759e640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f5fe815a800 0x7f5fe80b1f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T15:08:34.745 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:34.742+0000 7f5fe759e640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f5fe815a800 0x7f5fe80b1f80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:36194/0 (socket says 172.21.15.43:36194) 2023-08-09T15:08:34.745 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:34.742+0000 7f5fe759e640 1 -- 172.21.15.43:0/835956555 learned_addr learned my addr 172.21.15.43:0/835956555 (peer_addr_for_me v2:172.21.15.43:0/0) 2023-08-09T15:08:34.745 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:34.743+0000 7f5fdffff640 1 --2- 172.21.15.43:0/835956555 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f5fe8157ea0 0x7f5fe80b1a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T15:08:34.745 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:34.743+0000 7f5fe6d9d640 1 --2- 172.21.15.43:0/835956555 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f5fe80c90a0 0x7f5fe81f13c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T15:08:34.746 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:34.743+0000 7f5fe759e640 1 -- 172.21.15.43:0/835956555 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f5fe8157ea0 msgr2=0x7f5fe80b1a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T15:08:34.746 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:34.743+0000 7f5fe759e640 1 --2- 172.21.15.43:0/835956555 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f5fe8157ea0 0x7f5fe80b1a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T15:08:34.746 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:34.743+0000 7f5fe759e640 1 -- 172.21.15.43:0/835956555 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f5fe80c90a0 msgr2=0x7f5fe81f13c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T15:08:34.746 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:34.743+0000 7f5fe759e640 1 --2- 172.21.15.43:0/835956555 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f5fe80c90a0 0x7f5fe81f13c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T15:08:34.746 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:34.743+0000 7f5fe759e640 1 -- 172.21.15.43:0/835956555 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5fd0009400 con 0x7f5fe815a800 2023-08-09T15:08:34.746 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:34.743+0000 7f5fe759e640 1 --2- 172.21.15.43:0/835956555 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f5fe815a800 0x7f5fe80b1f80 secure :-1 s=READY pgs=2159 cs=0 l=1 rev1=1 crypto rx=0x7f5fe000cab0 tx=0x7f5fe000cf70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-09T15:08:34.746 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:34.743+0000 7f5fdf7fe640 1 -- 172.21.15.43:0/835956555 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5fe000abd0 con 0x7f5fe815a800 2023-08-09T15:08:34.746 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:34.744+0000 7f5fdf7fe640 1 -- 172.21.15.43:0/835956555 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5fe000ad30 con 0x7f5fe815a800 2023-08-09T15:08:34.747 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:34.744+0000 7f5fdf7fe640 1 -- 172.21.15.43:0/835956555 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5fe000aea0 con 0x7f5fe815a800 2023-08-09T15:08:34.747 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:34.744+0000 7f5fec9e0640 1 -- 172.21.15.43:0/835956555 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5fe81f1900 con 0x7f5fe815a800 2023-08-09T15:08:34.747 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:34.744+0000 7f5fec9e0640 1 -- 172.21.15.43:0/835956555 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5fe80c1d80 con 0x7f5fe815a800 2023-08-09T15:08:34.747 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:34.745+0000 7f5fec9e0640 1 -- 172.21.15.43:0/835956555 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5fe80c2580 con 0x7f5fe815a800 2023-08-09T15:08:34.747 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:34.746+0000 7f5fdf7fe640 1 -- 172.21.15.43:0/835956555 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 106) v1 ==== 88443+0+0 (secure 0 0 0) 0x7f5fe0007070 con 0x7f5fe815a800 2023-08-09T15:08:34.748 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:34.747+0000 7f5fdf7fe640 1 --2- 172.21.15.43:0/835956555 >> [v2:172.21.15.203:6832/3082732769,v1:172.21.15.203:6833/3082732769] conn(0x7f5fc806a170 0x7f5fc806c630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T15:08:34.748 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:34.747+0000 7f5fdf7fe640 1 -- 172.21.15.43:0/835956555 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(197..197 src has 1..197) v4 ==== 6692+0+0 (secure 0 0 0) 0x7f5fe0088620 con 0x7f5fe815a800 2023-08-09T15:08:34.748 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:34.747+0000 7f5fe6d9d640 1 --2- 172.21.15.43:0/835956555 >> [v2:172.21.15.203:6832/3082732769,v1:172.21.15.203:6833/3082732769] conn(0x7f5fc806a170 0x7f5fc806c630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T15:08:34.750 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:34.749+0000 7f5fe6d9d640 1 --2- 172.21.15.43:0/835956555 >> [v2:172.21.15.203:6832/3082732769,v1:172.21.15.203:6833/3082732769] conn(0x7f5fc806a170 0x7f5fc806c630 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7f5fe815b290 tx=0x7f5fd8008040 comp rx=0 tx=0).ready entity=mgr.8500 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-09T15:08:34.753 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:34.752+0000 7f5fdf7fe640 1 -- 172.21.15.43:0/835956555 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178066 (secure 0 0 0) 0x7f5fe80c2580 con 0x7f5fe815a800 2023-08-09T15:08:34.955 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:34.953+0000 7f5fec9e0640 1 -- 172.21.15.43:0/835956555 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "config rm", "who": "mgr", "name": "mgr/snap_schedule/allow_m_granularity"} v 0) v1 -- 0x7f5fe8158280 con 0x7f5fe815a800 2023-08-09T15:08:34.960 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:34.959+0000 7f5fdf7fe640 1 -- 172.21.15.43:0/835956555 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "config rm", "who": "mgr", "name": "mgr/snap_schedule/allow_m_granularity"}]=0 v62) v1 ==== 120+0+0 (secure 0 0 0) 0x7f5fe8158280 con 0x7f5fe815a800 2023-08-09T15:08:34.964 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:34.962+0000 7f5fec9e0640 1 -- 172.21.15.43:0/835956555 >> [v2:172.21.15.203:6832/3082732769,v1:172.21.15.203:6833/3082732769] conn(0x7f5fc806a170 msgr2=0x7f5fc806c630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T15:08:34.964 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:34.962+0000 7f5fec9e0640 1 --2- 172.21.15.43:0/835956555 >> [v2:172.21.15.203:6832/3082732769,v1:172.21.15.203:6833/3082732769] conn(0x7f5fc806a170 0x7f5fc806c630 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7f5fe815b290 tx=0x7f5fd8008040 comp rx=0 tx=0).stop 2023-08-09T15:08:34.964 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:34.962+0000 7f5fec9e0640 1 -- 172.21.15.43:0/835956555 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f5fe815a800 msgr2=0x7f5fe80b1f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T15:08:34.964 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:34.962+0000 7f5fec9e0640 1 --2- 172.21.15.43:0/835956555 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f5fe815a800 0x7f5fe80b1f80 secure :-1 s=READY pgs=2159 cs=0 l=1 rev1=1 crypto rx=0x7f5fe000cab0 tx=0x7f5fe000cf70 comp rx=0 tx=0).stop 2023-08-09T15:08:34.964 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:34.962+0000 7f5fec9e0640 1 -- 172.21.15.43:0/835956555 shutdown_connections 2023-08-09T15:08:34.964 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:34.962+0000 7f5fec9e0640 1 --2- 172.21.15.43:0/835956555 >> [v2:172.21.15.203:6832/3082732769,v1:172.21.15.203:6833/3082732769] conn(0x7f5fc806a170 0x7f5fc806c630 unknown :-1 s=CLOSED pgs=38 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T15:08:34.965 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:34.962+0000 7f5fec9e0640 1 --2- 172.21.15.43:0/835956555 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f5fe815a800 0x7f5fe80b1f80 unknown :-1 s=CLOSED pgs=2159 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T15:08:34.965 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:34.962+0000 7f5fec9e0640 1 --2- 172.21.15.43:0/835956555 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f5fe8157ea0 0x7f5fe80b1a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T15:08:34.965 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:34.962+0000 7f5fec9e0640 1 --2- 172.21.15.43:0/835956555 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f5fe80c90a0 0x7f5fe81f13c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T15:08:34.965 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:34.962+0000 7f5fec9e0640 1 -- 172.21.15.43:0/835956555 >> 172.21.15.43:0/835956555 conn(0x7f5fe80cddd0 msgr2=0x7f5fe80ce1b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-09T15:08:34.966 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:34.962+0000 7f5fec9e0640 1 -- 172.21.15.43:0/835956555 shutdown_connections 2023-08-09T15:08:34.966 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:34.962+0000 7f5fec9e0640 1 -- 172.21.15.43:0/835956555 wait complete. 2023-08-09T15:08:34.982 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config rm mon mon_allow_pool_delete 2023-08-09T15:08:35.197 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:35.196+0000 7f4c9e056640 1 Processor -- start 2023-08-09T15:08:35.197 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:35.196+0000 7f4c9e056640 1 -- start start 2023-08-09T15:08:35.199 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:35.198+0000 7f4c9e056640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f4c98070ef0 0x7f4c980ab8d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T15:08:35.199 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:35.198+0000 7f4c9e056640 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7f4c980abe10 0x7f4c980cb9a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T15:08:35.199 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:35.198+0000 7f4c9e056640 1 -- --> v1:172.21.15.203:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4c980acfa0 con 0x7f4c980cbee0 2023-08-09T15:08:35.199 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:35.198+0000 7f4c9e056640 1 -- --> v2:172.21.15.203:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4c980cc370 con 0x7f4c980abe10 2023-08-09T15:08:35.199 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:35.198+0000 7f4c9e056640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4c9815c940 con 0x7f4c98070ef0 2023-08-09T15:08:35.200 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:35.199+0000 7f4c9d054640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f4c98070ef0 0x7f4c980ab8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T15:08:35.200 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:35.199+0000 7f4c9d855640 1 --1- >> v1:172.21.15.203:6789/0 conn(0x7f4c980cbee0 0x7f4c98141e50 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.203:6789/0 says I am v1:172.21.15.43:49906/0 (socket says 172.21.15.43:49906) 2023-08-09T15:08:35.200 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:35.199+0000 7f4c9d855640 1 -- 172.21.15.43:0/2185843231 learned_addr learned my addr 172.21.15.43:0/2185843231 (peer_addr_for_me v1:172.21.15.43:0/0) 2023-08-09T15:08:35.200 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:35.199+0000 7f4c9c853640 1 --2- 172.21.15.43:0/2185843231 >> v2:172.21.15.203:3300/0 conn(0x7f4c980abe10 0x7f4c980cb9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T15:08:35.201 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:35.199+0000 7f4c9d054640 1 -- 172.21.15.43:0/2185843231 >> v2:172.21.15.203:3300/0 conn(0x7f4c980abe10 msgr2=0x7f4c980cb9a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-09T15:08:35.201 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:35.199+0000 7f4c9d054640 1 --2- 172.21.15.43:0/2185843231 >> v2:172.21.15.203:3300/0 conn(0x7f4c980abe10 0x7f4c980cb9a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T15:08:35.201 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:35.199+0000 7f4c9d054640 1 -- 172.21.15.43:0/2185843231 >> v1:172.21.15.203:6789/0 conn(0x7f4c980cbee0 legacy=0x7f4c98141e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T15:08:35.202 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:35.200+0000 7f4c9d054640 1 -- 172.21.15.43:0/2185843231 --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4c980ac390 con 0x7f4c98070ef0 2023-08-09T15:08:35.202 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:35.201+0000 7f4c9d054640 1 --2- 172.21.15.43:0/2185843231 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f4c98070ef0 0x7f4c980ab8d0 secure :-1 s=READY pgs=311 cs=0 l=1 rev1=1 crypto rx=0x7f4c940097a0 tx=0x7f4c9402b940 comp rx=0 tx=0).ready entity=mon.2 client_cookie=83373026ab99f0c4 server_cookie=0 in_seq=0 out_seq=0 2023-08-09T15:08:35.202 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:35.201+0000 7f4c8ffff640 1 -- 172.21.15.43:0/2185843231 <== mon.2 v2:172.21.15.203:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4c94030800 con 0x7f4c98070ef0 2023-08-09T15:08:35.203 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:35.201+0000 7f4c8ffff640 1 -- 172.21.15.43:0/2185843231 <== mon.2 v2:172.21.15.203:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4c94030960 con 0x7f4c98070ef0 2023-08-09T15:08:35.203 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:35.202+0000 7f4c8ffff640 1 -- 172.21.15.43:0/2185843231 <== mon.2 v2:172.21.15.203:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4c94030cf0 con 0x7f4c98070ef0 2023-08-09T15:08:35.203 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:35.202+0000 7f4c9e056640 1 -- 172.21.15.43:0/2185843231 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f4c98070ef0 msgr2=0x7f4c980ab8d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T15:08:35.204 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:35.202+0000 7f4c9e056640 1 --2- 172.21.15.43:0/2185843231 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f4c98070ef0 0x7f4c980ab8d0 secure :-1 s=READY pgs=311 cs=0 l=1 rev1=1 crypto rx=0x7f4c940097a0 tx=0x7f4c9402b940 comp rx=0 tx=0).stop 2023-08-09T15:08:35.204 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:35.202+0000 7f4c9e056640 1 -- 172.21.15.43:0/2185843231 shutdown_connections 2023-08-09T15:08:35.204 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:35.202+0000 7f4c9e056640 1 --2- 172.21.15.43:0/2185843231 >> v2:172.21.15.203:3300/0 conn(0x7f4c980abe10 0x7f4c980cb9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T15:08:35.204 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:35.202+0000 7f4c9e056640 1 --2- 172.21.15.43:0/2185843231 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f4c98070ef0 0x7f4c980ab8d0 unknown :-1 s=CLOSED pgs=311 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T15:08:35.204 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:35.202+0000 7f4c9e056640 1 -- 172.21.15.43:0/2185843231 >> 172.21.15.43:0/2185843231 conn(0x7f4c980cddd0 msgr2=0x7f4c980ce1e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-09T15:08:35.205 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:35.202+0000 7f4c9e056640 1 -- 172.21.15.43:0/2185843231 shutdown_connections 2023-08-09T15:08:35.205 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:35.202+0000 7f4c9e056640 1 -- 172.21.15.43:0/2185843231 wait complete. 2023-08-09T15:08:35.206 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:35.205+0000 7f4c9e056640 1 Processor -- start 2023-08-09T15:08:35.206 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:35.205+0000 7f4c9e056640 1 -- start start 2023-08-09T15:08:35.210 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:35.206+0000 7f4c9e056640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f4c98070ef0 0x7f4c981f16a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T15:08:35.210 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:35.206+0000 7f4c9e056640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f4c980abe10 0x7f4c980c6200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T15:08:35.210 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:35.206+0000 7f4c9e056640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f4c980cbee0 0x7f4c980c6740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T15:08:35.211 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:35.206+0000 7f4c9e056640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4c981f1be0 con 0x7f4c980abe10 2023-08-09T15:08:35.211 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:35.206+0000 7f4c9e056640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4c981f29c0 con 0x7f4c98070ef0 2023-08-09T15:08:35.211 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:35.206+0000 7f4c9e056640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4c981f1df0 con 0x7f4c980cbee0 2023-08-09T15:08:35.211 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:35.207+0000 7f4c9c853640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f4c980abe10 0x7f4c980c6200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T15:08:35.211 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:35.207+0000 7f4c9c853640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f4c980abe10 0x7f4c980c6200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:39250/0 (socket says 172.21.15.43:39250) 2023-08-09T15:08:35.211 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:35.207+0000 7f4c9c853640 1 -- 172.21.15.43:0/2730598949 learned_addr learned my addr 172.21.15.43:0/2730598949 (peer_addr_for_me v2:172.21.15.43:0/0) 2023-08-09T15:08:35.212 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:35.207+0000 7f4c9d855640 1 --2- 172.21.15.43:0/2730598949 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f4c980cbee0 0x7f4c980c6740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T15:08:35.212 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:35.207+0000 7f4c9d054640 1 --2- 172.21.15.43:0/2730598949 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f4c98070ef0 0x7f4c981f16a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T15:08:35.212 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:35.207+0000 7f4c9c853640 1 -- 172.21.15.43:0/2730598949 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f4c980cbee0 msgr2=0x7f4c980c6740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T15:08:35.212 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:35.207+0000 7f4c9c853640 1 --2- 172.21.15.43:0/2730598949 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f4c980cbee0 0x7f4c980c6740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T15:08:35.212 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:35.207+0000 7f4c9c853640 1 -- 172.21.15.43:0/2730598949 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f4c98070ef0 msgr2=0x7f4c981f16a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T15:08:35.212 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:35.207+0000 7f4c9c853640 1 --2- 172.21.15.43:0/2730598949 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f4c98070ef0 0x7f4c981f16a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T15:08:35.212 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:35.207+0000 7f4c9c853640 1 -- 172.21.15.43:0/2730598949 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4c94009400 con 0x7f4c980abe10 2023-08-09T15:08:35.213 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:35.207+0000 7f4c9d855640 1 --2- 172.21.15.43:0/2730598949 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f4c980cbee0 0x7f4c980c6740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-08-09T15:08:35.213 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:35.208+0000 7f4c9c853640 1 --2- 172.21.15.43:0/2730598949 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f4c980abe10 0x7f4c980c6200 secure :-1 s=READY pgs=2160 cs=0 l=1 rev1=1 crypto rx=0x7f4c8800e9b0 tx=0x7f4c8800ee70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-09T15:08:35.213 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:35.208+0000 7f4c8dffb640 1 -- 172.21.15.43:0/2730598949 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4c88009670 con 0x7f4c980abe10 2023-08-09T15:08:35.213 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:35.208+0000 7f4c8dffb640 1 -- 172.21.15.43:0/2730598949 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4c8800c9a0 con 0x7f4c980abe10 2023-08-09T15:08:35.213 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:35.208+0000 7f4c8dffb640 1 -- 172.21.15.43:0/2730598949 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4c8800cd50 con 0x7f4c980abe10 2023-08-09T15:08:35.213 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:35.208+0000 7f4c9e056640 1 -- 172.21.15.43:0/2730598949 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4c981f2090 con 0x7f4c980abe10 2023-08-09T15:08:35.213 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:35.208+0000 7f4c9e056640 1 -- 172.21.15.43:0/2730598949 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4c981f2650 con 0x7f4c980abe10 2023-08-09T15:08:35.214 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:35.209+0000 7f4c9e056640 1 -- 172.21.15.43:0/2730598949 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4c9804e390 con 0x7f4c980abe10 2023-08-09T15:08:35.214 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:35.210+0000 7f4c8dffb640 1 -- 172.21.15.43:0/2730598949 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 106) v1 ==== 88443+0+0 (secure 0 0 0) 0x7f4c88013470 con 0x7f4c980abe10 2023-08-09T15:08:35.215 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:35.211+0000 7f4c8dffb640 1 --2- 172.21.15.43:0/2730598949 >> [v2:172.21.15.203:6832/3082732769,v1:172.21.15.203:6833/3082732769] conn(0x7f4c7c06a170 0x7f4c7c06c630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T15:08:35.215 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:35.212+0000 7f4c8dffb640 1 -- 172.21.15.43:0/2730598949 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(197..197 src has 1..197) v4 ==== 6692+0+0 (secure 0 0 0) 0x7f4c88088cd0 con 0x7f4c980abe10 2023-08-09T15:08:35.215 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:35.212+0000 7f4c9d054640 1 --2- 172.21.15.43:0/2730598949 >> [v2:172.21.15.203:6832/3082732769,v1:172.21.15.203:6833/3082732769] conn(0x7f4c7c06a170 0x7f4c7c06c630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T15:08:35.215 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:35.213+0000 7f4c9d054640 1 --2- 172.21.15.43:0/2730598949 >> [v2:172.21.15.203:6832/3082732769,v1:172.21.15.203:6833/3082732769] conn(0x7f4c7c06a170 0x7f4c7c06c630 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7f4c94036000 tx=0x7f4c94008030 comp rx=0 tx=0).ready entity=mgr.8500 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-09T15:08:35.217 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:35.216+0000 7f4c8dffb640 1 -- 172.21.15.43:0/2730598949 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178066 (secure 0 0 0) 0x7f4c88053f90 con 0x7f4c980abe10 2023-08-09T15:08:35.420 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:35.419+0000 7f4c9e056640 1 -- 172.21.15.43:0/2730598949 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"} v 0) v1 -- 0x7f4c980b3d50 con 0x7f4c980abe10 2023-08-09T15:08:35.427 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:35.426+0000 7f4c8dffb640 1 -- 172.21.15.43:0/2730598949 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"}]=0 v63) v1 ==== 104+0+0 (secure 0 0 0) 0x7f4c880575b0 con 0x7f4c980abe10 2023-08-09T15:08:35.430 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:35.428+0000 7f4c9e056640 1 -- 172.21.15.43:0/2730598949 >> [v2:172.21.15.203:6832/3082732769,v1:172.21.15.203:6833/3082732769] conn(0x7f4c7c06a170 msgr2=0x7f4c7c06c630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T15:08:35.431 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:35.429+0000 7f4c9e056640 1 --2- 172.21.15.43:0/2730598949 >> [v2:172.21.15.203:6832/3082732769,v1:172.21.15.203:6833/3082732769] conn(0x7f4c7c06a170 0x7f4c7c06c630 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7f4c94036000 tx=0x7f4c94008030 comp rx=0 tx=0).stop 2023-08-09T15:08:35.431 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:35.429+0000 7f4c9e056640 1 -- 172.21.15.43:0/2730598949 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f4c980abe10 msgr2=0x7f4c980c6200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T15:08:35.431 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:35.429+0000 7f4c9e056640 1 --2- 172.21.15.43:0/2730598949 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f4c980abe10 0x7f4c980c6200 secure :-1 s=READY pgs=2160 cs=0 l=1 rev1=1 crypto rx=0x7f4c8800e9b0 tx=0x7f4c8800ee70 comp rx=0 tx=0).stop 2023-08-09T15:08:35.431 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:35.429+0000 7f4c9e056640 1 -- 172.21.15.43:0/2730598949 shutdown_connections 2023-08-09T15:08:35.432 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:35.429+0000 7f4c9e056640 1 --2- 172.21.15.43:0/2730598949 >> [v2:172.21.15.203:6832/3082732769,v1:172.21.15.203:6833/3082732769] conn(0x7f4c7c06a170 0x7f4c7c06c630 unknown :-1 s=CLOSED pgs=39 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T15:08:35.432 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:35.429+0000 7f4c9e056640 1 --2- 172.21.15.43:0/2730598949 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f4c980cbee0 0x7f4c980c6740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T15:08:35.432 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:35.429+0000 7f4c9e056640 1 --2- 172.21.15.43:0/2730598949 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f4c980abe10 0x7f4c980c6200 unknown :-1 s=CLOSED pgs=2160 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T15:08:35.432 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:35.429+0000 7f4c9e056640 1 --2- 172.21.15.43:0/2730598949 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f4c98070ef0 0x7f4c981f16a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T15:08:35.432 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:35.429+0000 7f4c9e056640 1 -- 172.21.15.43:0/2730598949 >> 172.21.15.43:0/2730598949 conn(0x7f4c980cddd0 msgr2=0x7f4c9806ce40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-09T15:08:35.433 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:35.429+0000 7f4c9e056640 1 -- 172.21.15.43:0/2730598949 shutdown_connections 2023-08-09T15:08:35.433 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:35.429+0000 7f4c9e056640 1 -- 172.21.15.43:0/2730598949 wait complete. 2023-08-09T15:08:35.450 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Ended test tasks.cephfs.test_snap_schedules.TestSnapSchedules.test_snap_schedule_with_mgr_restart' 2023-08-09T15:08:35.677 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:35.676+0000 7f2051213640 1 Processor -- start 2023-08-09T15:08:35.677 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:35.676+0000 7f2051213640 1 -- start start 2023-08-09T15:08:35.678 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:35.677+0000 7f2051213640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f204c0b2790 0x7f204c0d2ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T15:08:35.678 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:35.677+0000 7f2051213640 1 -- --> v1:172.21.15.43:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f204c0ce090 con 0x7f204c0b3050 2023-08-09T15:08:35.679 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:35.677+0000 7f2051213640 1 -- --> v1:172.21.15.203:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f204c0cd7f0 con 0x7f204c0b2b70 2023-08-09T15:08:35.679 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:35.677+0000 7f2051213640 1 -- --> v2:172.21.15.43:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f204c0b3950 con 0x7f204c0b2790 2023-08-09T15:08:35.679 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:35.677+0000 7f204b59e640 1 --1- >> v1:172.21.15.43:6789/0 conn(0x7f204c0b3050 0x7f204c0b3f60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.43:6789/0 says I am v1:172.21.15.43:59562/0 (socket says 172.21.15.43:59562) 2023-08-09T15:08:35.680 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:35.677+0000 7f204ad9d640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f204c0b2790 0x7f204c0d2ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T15:08:35.680 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:35.678+0000 7f204b59e640 1 -- 172.21.15.43:0/3666789020 learned_addr learned my addr 172.21.15.43:0/3666789020 (peer_addr_for_me v1:172.21.15.43:0/0) 2023-08-09T15:08:35.680 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:35.678+0000 7f204ad9d640 1 -- 172.21.15.43:0/3666789020 >> v1:172.21.15.203:6789/0 conn(0x7f204c0b2b70 legacy=0x7f204c0b5330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-09T15:08:35.681 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:35.678+0000 7f204ad9d640 1 -- 172.21.15.43:0/3666789020 >> v1:172.21.15.43:6789/0 conn(0x7f204c0b3050 legacy=0x7f204c0b3f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T15:08:35.681 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:35.678+0000 7f204ad9d640 1 -- 172.21.15.43:0/3666789020 --> v2:172.21.15.43:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f204c0cdad0 con 0x7f204c0b2790 2023-08-09T15:08:35.681 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:35.678+0000 7f2049d9b640 1 -- 172.21.15.43:0/3666789020 <== mon.0 v1:172.21.15.43:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3335096043 0 0) 0x7f204c0ce090 con 0x7f204c0b3050 2023-08-09T15:08:35.683 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:35.679+0000 7f204ad9d640 1 --2- 172.21.15.43:0/3666789020 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f204c0b2790 0x7f204c0d2ed0 secure :-1 s=READY pgs=2162 cs=0 l=1 rev1=1 crypto rx=0x7f203c009d90 tx=0x7f203c02beb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=954f7bd8555bfac8 server_cookie=0 in_seq=0 out_seq=0 2023-08-09T15:08:35.683 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:35.680+0000 7f2049d9b640 1 -- 172.21.15.43:0/3666789020 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f203c02cc60 con 0x7f204c0b2790 2023-08-09T15:08:35.683 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:35.680+0000 7f2049d9b640 1 -- 172.21.15.43:0/3666789020 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f203c02cdc0 con 0x7f204c0b2790 2023-08-09T15:08:35.684 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:35.680+0000 7f2049d9b640 1 -- 172.21.15.43:0/3666789020 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f203c034660 con 0x7f204c0b2790 2023-08-09T15:08:35.684 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:35.680+0000 7f2051213640 1 -- 172.21.15.43:0/3666789020 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f204c0b2790 msgr2=0x7f204c0d2ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T15:08:35.684 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:35.680+0000 7f2051213640 1 --2- 172.21.15.43:0/3666789020 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f204c0b2790 0x7f204c0d2ed0 secure :-1 s=READY pgs=2162 cs=0 l=1 rev1=1 crypto rx=0x7f203c009d90 tx=0x7f203c02beb0 comp rx=0 tx=0).stop 2023-08-09T15:08:35.684 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:35.680+0000 7f2051213640 1 -- 172.21.15.43:0/3666789020 shutdown_connections 2023-08-09T15:08:35.684 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:35.680+0000 7f2051213640 1 --2- 172.21.15.43:0/3666789020 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f204c0b2790 0x7f204c0d2ed0 unknown :-1 s=CLOSED pgs=2162 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T15:08:35.685 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:35.680+0000 7f2051213640 1 -- 172.21.15.43:0/3666789020 >> 172.21.15.43:0/3666789020 conn(0x7f204c09b7b0 msgr2=0x7f204c0d24e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-09T15:08:35.685 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:35.681+0000 7f2051213640 1 -- 172.21.15.43:0/3666789020 shutdown_connections 2023-08-09T15:08:35.685 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:35.681+0000 7f2051213640 1 -- 172.21.15.43:0/3666789020 wait complete. 2023-08-09T15:08:35.687 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:35.682+0000 7f2051213640 1 Processor -- start 2023-08-09T15:08:35.688 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:35.682+0000 7f2051213640 1 -- start start 2023-08-09T15:08:35.688 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:35.683+0000 7f2051213640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f204c0b2790 0x7f204c1570a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T15:08:35.688 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:35.683+0000 7f2051213640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f204c0b2b70 0x7f204c0b53c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T15:08:35.688 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:35.683+0000 7f2051213640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f204c0b3050 0x7f204c0b5900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T15:08:35.689 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:35.684+0000 7f2051213640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f204c1578f0 con 0x7f204c0b2b70 2023-08-09T15:08:35.689 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:35.684+0000 7f2051213640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f204c157bd0 con 0x7f204c0b2790 2023-08-09T15:08:35.689 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:35.684+0000 7f2051213640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f204c157eb0 con 0x7f204c0b3050 2023-08-09T15:08:35.689 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:35.684+0000 7f204a59c640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f204c0b2b70 0x7f204c0b53c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T15:08:35.690 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:35.684+0000 7f204a59c640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f204c0b2b70 0x7f204c0b53c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:39272/0 (socket says 172.21.15.43:39272) 2023-08-09T15:08:35.690 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:35.684+0000 7f204a59c640 1 -- 172.21.15.43:0/3884872572 learned_addr learned my addr 172.21.15.43:0/3884872572 (peer_addr_for_me v2:172.21.15.43:0/0) 2023-08-09T15:08:35.690 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:35.684+0000 7f204a59c640 1 -- 172.21.15.43:0/3884872572 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f204c0b3050 msgr2=0x7f204c0b5900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T15:08:35.690 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:35.684+0000 7f204b59e640 1 --2- 172.21.15.43:0/3884872572 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f204c0b3050 0x7f204c0b5900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T15:08:35.691 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:35.684+0000 7f204ad9d640 1 --2- 172.21.15.43:0/3884872572 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f204c0b2790 0x7f204c1570a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T15:08:35.691 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:35.684+0000 7f204a59c640 1 --2- 172.21.15.43:0/3884872572 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f204c0b3050 0x7f204c0b5900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T15:08:35.691 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:35.684+0000 7f204a59c640 1 -- 172.21.15.43:0/3884872572 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f204c0b2790 msgr2=0x7f204c1570a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T15:08:35.691 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:35.684+0000 7f204a59c640 1 --2- 172.21.15.43:0/3884872572 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f204c0b2790 0x7f204c1570a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T15:08:35.691 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:35.684+0000 7f204a59c640 1 -- 172.21.15.43:0/3884872572 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f203c009a70 con 0x7f204c0b2b70 2023-08-09T15:08:35.692 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:35.684+0000 7f204b59e640 1 --2- 172.21.15.43:0/3884872572 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f204c0b3050 0x7f204c0b5900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-08-09T15:08:35.692 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:35.685+0000 7f204a59c640 1 --2- 172.21.15.43:0/3884872572 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f204c0b2b70 0x7f204c0b53c0 secure :-1 s=READY pgs=2163 cs=0 l=1 rev1=1 crypto rx=0x7f204000b3f0 tx=0x7f204000b8b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-09T15:08:35.692 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:35.685+0000 7f20337fe640 1 -- 172.21.15.43:0/3884872572 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2040011830 con 0x7f204c0b2b70 2023-08-09T15:08:35.692 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:35.685+0000 7f20337fe640 1 -- 172.21.15.43:0/3884872572 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2040011990 con 0x7f204c0b2b70 2023-08-09T15:08:35.693 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:35.685+0000 7f20337fe640 1 -- 172.21.15.43:0/3884872572 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2040011d40 con 0x7f204c0b2b70 2023-08-09T15:08:35.693 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:35.685+0000 7f2051213640 1 -- 172.21.15.43:0/3884872572 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f204c158190 con 0x7f204c0b2b70 2023-08-09T15:08:35.693 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:35.685+0000 7f2051213640 1 -- 172.21.15.43:0/3884872572 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f204c1575e0 con 0x7f204c0b2b70 2023-08-09T15:08:35.693 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:35.686+0000 7f2051213640 1 -- 172.21.15.43:0/3884872572 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f204c0739f0 con 0x7f204c0b2b70 2023-08-09T15:08:35.694 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:35.688+0000 7f20337fe640 1 -- 172.21.15.43:0/3884872572 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 106) v1 ==== 88443+0+0 (secure 0 0 0) 0x7f204001a400 con 0x7f204c0b2b70 2023-08-09T15:08:35.695 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:35.688+0000 7f20337fe640 1 --2- 172.21.15.43:0/3884872572 >> [v2:172.21.15.203:6832/3082732769,v1:172.21.15.203:6833/3082732769] conn(0x7f203806a170 0x7f203806c630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T15:08:35.695 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:35.689+0000 7f20337fe640 1 -- 172.21.15.43:0/3884872572 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(197..197 src has 1..197) v4 ==== 6692+0+0 (secure 0 0 0) 0x7f2040088890 con 0x7f204c0b2b70 2023-08-09T15:08:35.695 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:35.689+0000 7f204ad9d640 1 --2- 172.21.15.43:0/3884872572 >> [v2:172.21.15.203:6832/3082732769,v1:172.21.15.203:6833/3082732769] conn(0x7f203806a170 0x7f203806c630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T15:08:35.695 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:35.690+0000 7f204ad9d640 1 --2- 172.21.15.43:0/3884872572 >> [v2:172.21.15.203:6832/3082732769,v1:172.21.15.203:6833/3082732769] conn(0x7f203806a170 0x7f203806c630 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7f203c036000 tx=0x7f203c030000 comp rx=0 tx=0).ready entity=mgr.8500 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-09T15:08:35.696 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:35.693+0000 7f20337fe640 1 -- 172.21.15.43:0/3884872572 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178066 (secure 0 0 0) 0x7f2040053b50 con 0x7f204c0b2b70 2023-08-09T15:08:35.958 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:35.956+0000 7f2051213640 1 -- 172.21.15.43:0/3884872572 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_snap_schedules.TestSnapSchedules.test_snap_schedule_with_mgr_restart"]} v 0) v1 -- 0x7f204c0b4b00 con 0x7f204c0b2b70 2023-08-09T15:08:35.996 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:35.995+0000 7f20337fe640 1 -- 172.21.15.43:0/3884872572 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_snap_schedules.TestSnapSchedules.test_snap_schedule_with_mgr_restart"]}]=0 v3098) v1 ==== 165+0+0 (secure 0 0 0) 0x7f2040057170 con 0x7f204c0b2b70 2023-08-09T15:08:35.999 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:35.998+0000 7f2051213640 1 -- 172.21.15.43:0/3884872572 >> [v2:172.21.15.203:6832/3082732769,v1:172.21.15.203:6833/3082732769] conn(0x7f203806a170 msgr2=0x7f203806c630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T15:08:36.000 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:35.998+0000 7f2051213640 1 --2- 172.21.15.43:0/3884872572 >> [v2:172.21.15.203:6832/3082732769,v1:172.21.15.203:6833/3082732769] conn(0x7f203806a170 0x7f203806c630 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7f203c036000 tx=0x7f203c030000 comp rx=0 tx=0).stop 2023-08-09T15:08:36.000 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:35.998+0000 7f2051213640 1 -- 172.21.15.43:0/3884872572 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f204c0b2b70 msgr2=0x7f204c0b53c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T15:08:36.000 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:35.998+0000 7f2051213640 1 --2- 172.21.15.43:0/3884872572 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f204c0b2b70 0x7f204c0b53c0 secure :-1 s=READY pgs=2163 cs=0 l=1 rev1=1 crypto rx=0x7f204000b3f0 tx=0x7f204000b8b0 comp rx=0 tx=0).stop 2023-08-09T15:08:36.000 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:35.998+0000 7f2051213640 1 -- 172.21.15.43:0/3884872572 shutdown_connections 2023-08-09T15:08:36.001 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:35.998+0000 7f2051213640 1 --2- 172.21.15.43:0/3884872572 >> [v2:172.21.15.203:6832/3082732769,v1:172.21.15.203:6833/3082732769] conn(0x7f203806a170 0x7f203806c630 unknown :-1 s=CLOSED pgs=40 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T15:08:36.001 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:35.998+0000 7f2051213640 1 --2- 172.21.15.43:0/3884872572 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f204c0b3050 0x7f204c0b5900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T15:08:36.001 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:35.998+0000 7f2051213640 1 --2- 172.21.15.43:0/3884872572 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f204c0b2b70 0x7f204c0b53c0 unknown :-1 s=CLOSED pgs=2163 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T15:08:36.001 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:35.998+0000 7f2051213640 1 --2- 172.21.15.43:0/3884872572 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f204c0b2790 0x7f204c1570a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T15:08:36.001 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:35.998+0000 7f2051213640 1 -- 172.21.15.43:0/3884872572 >> 172.21.15.43:0/3884872572 conn(0x7f204c09b7b0 msgr2=0x7f204c06b390 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-09T15:08:36.001 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:35.998+0000 7f2051213640 1 -- 172.21.15.43:0/3884872572 shutdown_connections 2023-08-09T15:08:36.001 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:35.998+0000 7f2051213640 1 -- 172.21.15.43:0/3884872572 wait complete. 2023-08-09T15:08:36.019 INFO:tasks.cephfs_test_runner:Test that snap schedule is resumed after mgr restart ... ok 2023-08-09T15:08:36.020 INFO:tasks.cephfs_test_runner:Starting test: test_snap_schedule_with_retention (tasks.cephfs.test_snap_schedules.TestSnapSchedules) Test scheduled snapshots along with rentention policy 2023-08-09T15:08:36.021 INFO:tasks.cephfs_test_runner:test_snap_schedule_with_retention (tasks.cephfs.test_snap_schedules.TestSnapSchedules) 2023-08-09T15:08:36.022 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Starting test tasks.cephfs.test_snap_schedules.TestSnapSchedules.test_snap_schedule_with_retention' 2023-08-09T15:08:36.228 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:36.227+0000 7f403d1ea640 1 Processor -- start 2023-08-09T15:08:36.229 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:36.227+0000 7f403d1ea640 1 -- start start 2023-08-09T15:08:36.230 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:36.229+0000 7f403d1ea640 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7f40380c86f0 0x7f40380c8b50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T15:08:36.231 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:36.229+0000 7f403d1ea640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f403806c8d0 0x7f403806cd00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T15:08:36.231 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:36.229+0000 7f403d1ea640 1 -- --> v1:172.21.15.43:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f40380c9090 con 0x7f4038143c20 2023-08-09T15:08:36.231 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:36.229+0000 7f403d1ea640 1 -- --> v2:172.21.15.43:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4038144000 con 0x7f403806c8d0 2023-08-09T15:08:36.231 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:36.229+0000 7f403d1ea640 1 -- --> v2:172.21.15.203:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f403806c440 con 0x7f40380c86f0 2023-08-09T15:08:36.232 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:36.230+0000 7f403759e640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f403806c8d0 0x7f403806cd00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T15:08:36.232 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:36.230+0000 7f4036d9d640 1 --1- >> v1:172.21.15.43:6789/0 conn(0x7f4038143c20 0x7f403806f540 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.43:6789/0 says I am v1:172.21.15.43:59576/0 (socket says 172.21.15.43:59576) 2023-08-09T15:08:36.232 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:36.230+0000 7f4036d9d640 1 -- 172.21.15.43:0/30109430 learned_addr learned my addr 172.21.15.43:0/30109430 (peer_addr_for_me v1:172.21.15.43:0/0) 2023-08-09T15:08:36.233 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:36.230+0000 7f403659c640 1 --2- 172.21.15.43:0/30109430 >> v2:172.21.15.203:3300/0 conn(0x7f40380c86f0 0x7f40380c8b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T15:08:36.233 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:36.230+0000 7f403759e640 1 -- 172.21.15.43:0/30109430 >> v2:172.21.15.203:3300/0 conn(0x7f40380c86f0 msgr2=0x7f40380c8b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-09T15:08:36.233 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:36.230+0000 7f403759e640 1 --2- 172.21.15.43:0/30109430 >> v2:172.21.15.203:3300/0 conn(0x7f40380c86f0 0x7f40380c8b50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T15:08:36.233 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:36.230+0000 7f403759e640 1 -- 172.21.15.43:0/30109430 >> v1:172.21.15.43:6789/0 conn(0x7f4038143c20 legacy=0x7f403806f540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T15:08:36.233 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:36.230+0000 7f403759e640 1 -- 172.21.15.43:0/30109430 --> v2:172.21.15.43:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f403806fc50 con 0x7f403806c8d0 2023-08-09T15:08:36.233 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:36.230+0000 7f4035d9b640 1 -- 172.21.15.43:0/30109430 <== mon.0 v1:172.21.15.43:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 651609477 0 0) 0x7f40380c9090 con 0x7f4038143c20 2023-08-09T15:08:36.234 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:36.232+0000 7f403759e640 1 --2- 172.21.15.43:0/30109430 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f403806c8d0 0x7f403806cd00 secure :-1 s=READY pgs=2165 cs=0 l=1 rev1=1 crypto rx=0x7f40380c9b70 tx=0x7f403002be40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a62602de7a77676f server_cookie=0 in_seq=0 out_seq=0 2023-08-09T15:08:36.235 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:36.232+0000 7f4035d9b640 1 -- 172.21.15.43:0/30109430 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f403002cbf0 con 0x7f403806c8d0 2023-08-09T15:08:36.235 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:36.232+0000 7f4035d9b640 1 -- 172.21.15.43:0/30109430 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f403002cd50 con 0x7f403806c8d0 2023-08-09T15:08:36.235 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:36.232+0000 7f4035d9b640 1 -- 172.21.15.43:0/30109430 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4030034660 con 0x7f403806c8d0 2023-08-09T15:08:36.235 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:36.232+0000 7f403d1ea640 1 -- 172.21.15.43:0/30109430 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f403806c8d0 msgr2=0x7f403806cd00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T15:08:36.235 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:36.232+0000 7f403d1ea640 1 --2- 172.21.15.43:0/30109430 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f403806c8d0 0x7f403806cd00 secure :-1 s=READY pgs=2165 cs=0 l=1 rev1=1 crypto rx=0x7f40380c9b70 tx=0x7f403002be40 comp rx=0 tx=0).stop 2023-08-09T15:08:36.235 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:36.233+0000 7f403d1ea640 1 -- 172.21.15.43:0/30109430 shutdown_connections 2023-08-09T15:08:36.235 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:36.233+0000 7f403d1ea640 1 --2- 172.21.15.43:0/30109430 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f403806c8d0 0x7f403806cd00 unknown :-1 s=CLOSED pgs=2165 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T15:08:36.236 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:36.233+0000 7f403d1ea640 1 --2- 172.21.15.43:0/30109430 >> v2:172.21.15.203:3300/0 conn(0x7f40380c86f0 0x7f40380c8b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T15:08:36.236 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:36.233+0000 7f403d1ea640 1 -- 172.21.15.43:0/30109430 >> 172.21.15.43:0/30109430 conn(0x7f40380ce0a0 msgr2=0x7f40380ce480 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-09T15:08:36.236 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:36.233+0000 7f403d1ea640 1 -- 172.21.15.43:0/30109430 shutdown_connections 2023-08-09T15:08:36.236 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:36.233+0000 7f403d1ea640 1 -- 172.21.15.43:0/30109430 wait complete. 2023-08-09T15:08:36.237 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:36.234+0000 7f403d1ea640 1 Processor -- start 2023-08-09T15:08:36.237 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:36.235+0000 7f403d1ea640 1 -- start start 2023-08-09T15:08:36.238 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:36.235+0000 7f403d1ea640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f403806c8d0 0x7f40381f0f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T15:08:36.238 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:36.235+0000 7f403d1ea640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f40380c86f0 0x7f40380b2970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T15:08:36.238 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:36.236+0000 7f403d1ea640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f4038143c20 0x7f40380b2eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T15:08:36.238 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:36.236+0000 7f403d1ea640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f40380b2760 con 0x7f403806c8d0 2023-08-09T15:08:36.238 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:36.236+0000 7f403d1ea640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f40380c56c0 con 0x7f40380c86f0 2023-08-09T15:08:36.238 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:36.236+0000 7f403d1ea640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f403815aae0 con 0x7f4038143c20 2023-08-09T15:08:36.238 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:36.236+0000 7f4036d9d640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f403806c8d0 0x7f40381f0f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T15:08:36.239 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:36.236+0000 7f4036d9d640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f403806c8d0 0x7f40381f0f70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:39280/0 (socket says 172.21.15.43:39280) 2023-08-09T15:08:36.239 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:36.236+0000 7f4036d9d640 1 -- 172.21.15.43:0/3818724556 learned_addr learned my addr 172.21.15.43:0/3818724556 (peer_addr_for_me v2:172.21.15.43:0/0) 2023-08-09T15:08:36.239 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:36.236+0000 7f403659c640 1 --2- 172.21.15.43:0/3818724556 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f40380c86f0 0x7f40380b2970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T15:08:36.239 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:36.236+0000 7f403759e640 1 --2- 172.21.15.43:0/3818724556 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f4038143c20 0x7f40380b2eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T15:08:36.239 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:36.236+0000 7f4036d9d640 1 -- 172.21.15.43:0/3818724556 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f4038143c20 msgr2=0x7f40380b2eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T15:08:36.239 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:36.236+0000 7f4036d9d640 1 --2- 172.21.15.43:0/3818724556 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f4038143c20 0x7f40380b2eb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T15:08:36.239 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:36.236+0000 7f4036d9d640 1 -- 172.21.15.43:0/3818724556 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f40380c86f0 msgr2=0x7f40380b2970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T15:08:36.240 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:36.236+0000 7f4036d9d640 1 --2- 172.21.15.43:0/3818724556 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f40380c86f0 0x7f40380b2970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T15:08:36.240 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:36.236+0000 7f4036d9d640 1 -- 172.21.15.43:0/3818724556 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4030009a70 con 0x7f403806c8d0 2023-08-09T15:08:36.240 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:36.237+0000 7f4036d9d640 1 --2- 172.21.15.43:0/3818724556 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f403806c8d0 0x7f40381f0f70 secure :-1 s=READY pgs=2166 cs=0 l=1 rev1=1 crypto rx=0x7f402800c500 tx=0x7f402800c9c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-09T15:08:36.241 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:36.237+0000 7f401f7fe640 1 -- 172.21.15.43:0/3818724556 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4028007560 con 0x7f403806c8d0 2023-08-09T15:08:36.241 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:36.237+0000 7f401f7fe640 1 -- 172.21.15.43:0/3818724556 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f40280047e0 con 0x7f403806c8d0 2023-08-09T15:08:36.241 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:36.237+0000 7f401f7fe640 1 -- 172.21.15.43:0/3818724556 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4028004b90 con 0x7f403806c8d0 2023-08-09T15:08:36.241 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:36.237+0000 7f403d1ea640 1 -- 172.21.15.43:0/3818724556 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f403815adc0 con 0x7f403806c8d0 2023-08-09T15:08:36.241 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:36.237+0000 7f403d1ea640 1 -- 172.21.15.43:0/3818724556 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f403815b380 con 0x7f403806c8d0 2023-08-09T15:08:36.241 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:36.238+0000 7f403d1ea640 1 -- 172.21.15.43:0/3818724556 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f40380739f0 con 0x7f403806c8d0 2023-08-09T15:08:36.242 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:36.239+0000 7f401f7fe640 1 -- 172.21.15.43:0/3818724556 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 106) v1 ==== 88443+0+0 (secure 0 0 0) 0x7f4028004cf0 con 0x7f403806c8d0 2023-08-09T15:08:36.242 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:36.241+0000 7f401f7fe640 1 --2- 172.21.15.43:0/3818724556 >> [v2:172.21.15.203:6832/3082732769,v1:172.21.15.203:6833/3082732769] conn(0x7f402406a170 0x7f402406c630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T15:08:36.242 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:36.241+0000 7f401f7fe640 1 -- 172.21.15.43:0/3818724556 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(197..197 src has 1..197) v4 ==== 6692+0+0 (secure 0 0 0) 0x7f4028087b30 con 0x7f403806c8d0 2023-08-09T15:08:36.242 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:36.241+0000 7f403659c640 1 --2- 172.21.15.43:0/3818724556 >> [v2:172.21.15.203:6832/3082732769,v1:172.21.15.203:6833/3082732769] conn(0x7f402406a170 0x7f402406c630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T15:08:36.243 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:36.242+0000 7f403659c640 1 --2- 172.21.15.43:0/3818724556 >> [v2:172.21.15.203:6832/3082732769,v1:172.21.15.203:6833/3082732769] conn(0x7f402406a170 0x7f402406c630 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7f40380b3710 tx=0x7f402c008040 comp rx=0 tx=0).ready entity=mgr.8500 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-09T15:08:36.246 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:36.245+0000 7f401f7fe640 1 -- 172.21.15.43:0/3818724556 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178066 (secure 0 0 0) 0x7f4028052e80 con 0x7f403806c8d0 2023-08-09T15:08:36.518 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:36.517+0000 7f403d1ea640 1 -- 172.21.15.43:0/3818724556 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_snap_schedules.TestSnapSchedules.test_snap_schedule_with_retention"]} v 0) v1 -- 0x7f40380c4c40 con 0x7f403806c8d0 2023-08-09T15:08:37.002 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:37.000+0000 7f401f7fe640 1 -- 172.21.15.43:0/3818724556 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_snap_schedules.TestSnapSchedules.test_snap_schedule_with_retention"]}]=0 v3099) v1 ==== 166+0+0 (secure 0 0 0) 0x7f40280564a0 con 0x7f403806c8d0 2023-08-09T15:08:37.006 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:37.004+0000 7f403d1ea640 1 -- 172.21.15.43:0/3818724556 >> [v2:172.21.15.203:6832/3082732769,v1:172.21.15.203:6833/3082732769] conn(0x7f402406a170 msgr2=0x7f402406c630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T15:08:37.006 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:37.004+0000 7f403d1ea640 1 --2- 172.21.15.43:0/3818724556 >> [v2:172.21.15.203:6832/3082732769,v1:172.21.15.203:6833/3082732769] conn(0x7f402406a170 0x7f402406c630 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7f40380b3710 tx=0x7f402c008040 comp rx=0 tx=0).stop 2023-08-09T15:08:37.006 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:37.004+0000 7f403d1ea640 1 -- 172.21.15.43:0/3818724556 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f403806c8d0 msgr2=0x7f40381f0f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T15:08:37.006 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:37.004+0000 7f403d1ea640 1 --2- 172.21.15.43:0/3818724556 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f403806c8d0 0x7f40381f0f70 secure :-1 s=READY pgs=2166 cs=0 l=1 rev1=1 crypto rx=0x7f402800c500 tx=0x7f402800c9c0 comp rx=0 tx=0).stop 2023-08-09T15:08:37.006 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:37.004+0000 7f403d1ea640 1 -- 172.21.15.43:0/3818724556 shutdown_connections 2023-08-09T15:08:37.006 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:37.004+0000 7f403d1ea640 1 --2- 172.21.15.43:0/3818724556 >> [v2:172.21.15.203:6832/3082732769,v1:172.21.15.203:6833/3082732769] conn(0x7f402406a170 0x7f402406c630 unknown :-1 s=CLOSED pgs=41 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T15:08:37.007 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:37.004+0000 7f403d1ea640 1 --2- 172.21.15.43:0/3818724556 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f4038143c20 0x7f40380b2eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T15:08:37.007 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:37.004+0000 7f403d1ea640 1 --2- 172.21.15.43:0/3818724556 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f40380c86f0 0x7f40380b2970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T15:08:37.007 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:37.004+0000 7f403d1ea640 1 --2- 172.21.15.43:0/3818724556 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f403806c8d0 0x7f40381f0f70 unknown :-1 s=CLOSED pgs=2166 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T15:08:37.007 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:37.004+0000 7f403d1ea640 1 -- 172.21.15.43:0/3818724556 >> 172.21.15.43:0/3818724556 conn(0x7f40380ce0a0 msgr2=0x7f40380ade10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-09T15:08:37.007 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:37.004+0000 7f403d1ea640 1 -- 172.21.15.43:0/3818724556 shutdown_connections 2023-08-09T15:08:37.007 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:37.004+0000 7f403d1ea640 1 -- 172.21.15.43:0/3818724556 wait complete. 2023-08-09T15:08:37.025 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-08-09T15:08:37.248 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:37.247+0000 7ff942776640 1 Processor -- start 2023-08-09T15:08:37.248 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:37.247+0000 7ff942776640 1 -- start start 2023-08-09T15:08:37.250 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:37.249+0000 7ff942776640 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7ff93c143c20 0x7ff93c06f540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T15:08:37.250 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:37.249+0000 7ff942776640 1 -- --> v1:172.21.15.43:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff93c06c440 con 0x7ff93c0c86f0 2023-08-09T15:08:37.251 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:37.249+0000 7ff942776640 1 -- --> v1:172.21.15.203:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff93c06b830 con 0x7ff93c06cfe0 2023-08-09T15:08:37.251 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:37.249+0000 7ff942776640 1 -- --> v2:172.21.15.203:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff93c144000 con 0x7ff93c143c20 2023-08-09T15:08:37.251 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:37.249+0000 7ff933fff640 1 --1- >> v1:172.21.15.43:6789/0 conn(0x7ff93c0c86f0 0x7ff93c06c8d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.43:6789/0 says I am v1:172.21.15.43:59578/0 (socket says 172.21.15.43:59578) 2023-08-09T15:08:37.251 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:37.249+0000 7ff933fff640 1 -- 172.21.15.43:0/479721470 learned_addr learned my addr 172.21.15.43:0/479721470 (peer_addr_for_me v1:172.21.15.43:0/0) 2023-08-09T15:08:37.253 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:37.249+0000 7ff93bfff640 1 --2- 172.21.15.43:0/479721470 >> v2:172.21.15.203:3300/0 conn(0x7ff93c143c20 0x7ff93c06f540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T15:08:37.253 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:37.250+0000 7ff93b7fe640 1 -- 172.21.15.43:0/479721470 <== mon.0 v1:172.21.15.43:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 275156293 0 0) 0x7ff93c06c440 con 0x7ff93c0c86f0 2023-08-09T15:08:37.253 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:37.250+0000 7ff93b7fe640 1 -- 172.21.15.43:0/479721470 --> v1:172.21.15.43:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ff928003520 con 0x7ff93c0c86f0 2023-08-09T15:08:37.253 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:37.250+0000 7ff93b7fe640 1 -- 172.21.15.43:0/479721470 <== mon.0 v1:172.21.15.43:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2958924260 0 0) 0x7ff928003520 con 0x7ff93c0c86f0 2023-08-09T15:08:37.253 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:37.250+0000 7ff93b7fe640 1 -- 172.21.15.43:0/479721470 >> v2:172.21.15.203:3300/0 conn(0x7ff93c143c20 msgr2=0x7ff93c06f540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-09T15:08:37.254 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:37.250+0000 7ff93b7fe640 1 --2- 172.21.15.43:0/479721470 >> v2:172.21.15.203:3300/0 conn(0x7ff93c143c20 0x7ff93c06f540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T15:08:37.254 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:37.250+0000 7ff93b7fe640 1 -- 172.21.15.43:0/479721470 >> v1:172.21.15.203:6789/0 conn(0x7ff93c06cfe0 legacy=0x7ff93c0ccc00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T15:08:37.254 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:37.250+0000 7ff93b7fe640 1 -- 172.21.15.43:0/479721470 --> v1:172.21.15.43:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff93c06bb10 con 0x7ff93c0c86f0 2023-08-09T15:08:37.254 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:37.250+0000 7ff93b7fe640 1 -- 172.21.15.43:0/479721470 <== mon.0 v1:172.21.15.43:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1285150740 0 0) 0x7ff924001670 con 0x7ff93c0c86f0 2023-08-09T15:08:37.255 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:37.250+0000 7ff93b7fe640 1 -- 172.21.15.43:0/479721470 >> v1:172.21.15.43:6789/0 conn(0x7ff93c0c86f0 legacy=0x7ff93c06c8d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T15:08:37.255 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:37.250+0000 7ff93b7fe640 1 --2- 172.21.15.43:0/479721470 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7ff9280039e0 0x7ff928003e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T15:08:37.255 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:37.250+0000 7ff93b7fe640 1 --2- 172.21.15.43:0/479721470 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7ff9280047c0 0x7ff928004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T15:08:37.255 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:37.250+0000 7ff93b7fe640 1 --2- 172.21.15.43:0/479721470 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff9280052a0 0x7ff92800b980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T15:08:37.255 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:37.250+0000 7ff93b7fe640 1 -- 172.21.15.43:0/479721470 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff93c144000 con 0x7ff9280052a0 2023-08-09T15:08:37.255 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:37.250+0000 7ff93b7fe640 1 -- 172.21.15.43:0/479721470 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff92800c170 con 0x7ff9280039e0 2023-08-09T15:08:37.256 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:37.250+0000 7ff93b7fe640 1 -- 172.21.15.43:0/479721470 --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff92800c330 con 0x7ff9280047c0 2023-08-09T15:08:37.257 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:37.251+0000 7ff940d13640 1 --2- 172.21.15.43:0/479721470 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff9280052a0 0x7ff92800b980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T15:08:37.257 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:37.251+0000 7ff93bfff640 1 --2- 172.21.15.43:0/479721470 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7ff9280039e0 0x7ff928003e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T15:08:37.257 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:37.251+0000 7ff933fff640 1 --2- 172.21.15.43:0/479721470 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7ff9280047c0 0x7ff928004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T15:08:37.257 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:37.251+0000 7ff940d13640 1 -- 172.21.15.43:0/479721470 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7ff9280047c0 msgr2=0x7ff928004c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-09T15:08:37.257 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:37.251+0000 7ff940d13640 1 --2- 172.21.15.43:0/479721470 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7ff9280047c0 0x7ff928004c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T15:08:37.257 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:37.251+0000 7ff940d13640 1 -- 172.21.15.43:0/479721470 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7ff9280039e0 msgr2=0x7ff928003e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-09T15:08:37.258 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:37.251+0000 7ff940d13640 1 --2- 172.21.15.43:0/479721470 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7ff9280039e0 0x7ff928003e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T15:08:37.258 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:37.251+0000 7ff940d13640 1 -- 172.21.15.43:0/479721470 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff93c06bb10 con 0x7ff9280052a0 2023-08-09T15:08:37.263 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:37.252+0000 7ff940d13640 1 -- 172.21.15.43:0/479721470 reap_dead start 2023-08-09T15:08:37.264 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:37.252+0000 7ff940d13640 1 --2- 172.21.15.43:0/479721470 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff9280052a0 0x7ff92800b980 secure :-1 s=READY pgs=2168 cs=0 l=1 rev1=1 crypto rx=0x7ff934001e50 tx=0x7ff9280057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9dcf57433799316e server_cookie=0 in_seq=0 out_seq=0 2023-08-09T15:08:37.264 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:37.253+0000 7ff93b7fe640 1 -- 172.21.15.43:0/479721470 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff934003b70 con 0x7ff9280052a0 2023-08-09T15:08:37.264 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:37.253+0000 7ff93b7fe640 1 -- 172.21.15.43:0/479721470 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff934003d10 con 0x7ff9280052a0 2023-08-09T15:08:37.264 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:37.253+0000 7ff93b7fe640 1 -- 172.21.15.43:0/479721470 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff934005310 con 0x7ff9280052a0 2023-08-09T15:08:37.265 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:37.253+0000 7ff942776640 1 -- 172.21.15.43:0/479721470 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff9280052a0 msgr2=0x7ff92800b980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T15:08:37.265 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:37.253+0000 7ff942776640 1 --2- 172.21.15.43:0/479721470 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff9280052a0 0x7ff92800b980 secure :-1 s=READY pgs=2168 cs=0 l=1 rev1=1 crypto rx=0x7ff934001e50 tx=0x7ff9280057f0 comp rx=0 tx=0).stop 2023-08-09T15:08:37.265 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:37.254+0000 7ff942776640 1 -- 172.21.15.43:0/479721470 shutdown_connections 2023-08-09T15:08:37.266 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:37.254+0000 7ff942776640 1 --2- 172.21.15.43:0/479721470 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff9280052a0 0x7ff92800b980 unknown :-1 s=CLOSED pgs=2168 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T15:08:37.266 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:37.254+0000 7ff942776640 1 -- 172.21.15.43:0/479721470 >> 172.21.15.43:0/479721470 conn(0x7ff93c0ce0a0 msgr2=0x7ff93c0ce480 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-09T15:08:37.266 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:37.254+0000 7ff942776640 1 -- 172.21.15.43:0/479721470 shutdown_connections 2023-08-09T15:08:37.266 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:37.254+0000 7ff942776640 1 -- 172.21.15.43:0/479721470 wait complete. 2023-08-09T15:08:37.266 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:37.256+0000 7ff942776640 1 Processor -- start 2023-08-09T15:08:37.266 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:37.256+0000 7ff942776640 1 -- start start 2023-08-09T15:08:37.267 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:37.257+0000 7ff942776640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7ff9280052a0 0x7ff93c141fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T15:08:37.267 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:37.257+0000 7ff942776640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff93c0c4100 0x7ff93c0c4560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T15:08:37.267 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:37.257+0000 7ff942776640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7ff93c0b2070 0x7ff93c0b2500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T15:08:37.267 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:37.257+0000 7ff942776640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff93c0b2de0 con 0x7ff93c0c4100 2023-08-09T15:08:37.267 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:37.257+0000 7ff942776640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff93c0c4aa0 con 0x7ff93c0b2070 2023-08-09T15:08:37.267 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:37.257+0000 7ff942776640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff93c0ae350 con 0x7ff9280052a0 2023-08-09T15:08:37.267 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:37.257+0000 7ff933fff640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff93c0c4100 0x7ff93c0c4560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T15:08:37.268 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:37.257+0000 7ff933fff640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff93c0c4100 0x7ff93c0c4560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:39294/0 (socket says 172.21.15.43:39294) 2023-08-09T15:08:37.268 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:37.257+0000 7ff933fff640 1 -- 172.21.15.43:0/2416345075 learned_addr learned my addr 172.21.15.43:0/2416345075 (peer_addr_for_me v2:172.21.15.43:0/0) 2023-08-09T15:08:37.268 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:37.257+0000 7ff93bfff640 1 --2- 172.21.15.43:0/2416345075 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7ff9280052a0 0x7ff93c141fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T15:08:37.268 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:37.257+0000 7ff940d13640 1 --2- 172.21.15.43:0/2416345075 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7ff93c0b2070 0x7ff93c0b2500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T15:08:37.268 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:37.258+0000 7ff933fff640 1 -- 172.21.15.43:0/2416345075 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7ff9280052a0 msgr2=0x7ff93c141fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T15:08:37.268 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:37.258+0000 7ff933fff640 1 --2- 172.21.15.43:0/2416345075 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7ff9280052a0 0x7ff93c141fc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T15:08:37.268 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:37.258+0000 7ff933fff640 1 -- 172.21.15.43:0/2416345075 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7ff93c0b2070 msgr2=0x7ff93c0b2500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T15:08:37.269 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:37.258+0000 7ff933fff640 1 --2- 172.21.15.43:0/2416345075 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7ff93c0b2070 0x7ff93c0b2500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T15:08:37.269 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:37.258+0000 7ff933fff640 1 -- 172.21.15.43:0/2416345075 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff93c0ad590 con 0x7ff93c0c4100 2023-08-09T15:08:37.269 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:37.258+0000 7ff933fff640 1 --2- 172.21.15.43:0/2416345075 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff93c0c4100 0x7ff93c0c4560 secure :-1 s=READY pgs=2169 cs=0 l=1 rev1=1 crypto rx=0x7ff92400e750 tx=0x7ff92400ec10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-09T15:08:37.269 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:37.258+0000 7ff9397fa640 1 -- 172.21.15.43:0/2416345075 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff92400ee60 con 0x7ff93c0c4100 2023-08-09T15:08:37.269 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:37.258+0000 7ff9397fa640 1 -- 172.21.15.43:0/2416345075 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff92400c8d0 con 0x7ff93c0c4100 2023-08-09T15:08:37.269 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:37.258+0000 7ff9397fa640 1 -- 172.21.15.43:0/2416345075 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff92400cc60 con 0x7ff93c0c4100 2023-08-09T15:08:37.269 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:37.258+0000 7ff942776640 1 -- 172.21.15.43:0/2416345075 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff93c0ad870 con 0x7ff93c0c4100 2023-08-09T15:08:37.269 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:37.258+0000 7ff942776640 1 -- 172.21.15.43:0/2416345075 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff93c0ade30 con 0x7ff93c0c4100 2023-08-09T15:08:37.270 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:37.260+0000 7ff942776640 1 -- 172.21.15.43:0/2416345075 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff93c0739f0 con 0x7ff93c0c4100 2023-08-09T15:08:37.271 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:37.261+0000 7ff9397fa640 1 -- 172.21.15.43:0/2416345075 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 106) v1 ==== 88443+0+0 (secure 0 0 0) 0x7ff92401d460 con 0x7ff93c0c4100 2023-08-09T15:08:37.271 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:37.262+0000 7ff9397fa640 1 --2- 172.21.15.43:0/2416345075 >> [v2:172.21.15.203:6832/3082732769,v1:172.21.15.203:6833/3082732769] conn(0x7ff91c06a170 0x7ff91c06c630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T15:08:37.271 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:37.262+0000 7ff9397fa640 1 -- 172.21.15.43:0/2416345075 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(197..197 src has 1..197) v4 ==== 6692+0+0 (secure 0 0 0) 0x7ff924088980 con 0x7ff93c0c4100 2023-08-09T15:08:37.271 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:37.263+0000 7ff93bfff640 1 --2- 172.21.15.43:0/2416345075 >> [v2:172.21.15.203:6832/3082732769,v1:172.21.15.203:6833/3082732769] conn(0x7ff91c06a170 0x7ff91c06c630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T15:08:37.271 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:37.264+0000 7ff93bfff640 1 --2- 172.21.15.43:0/2416345075 >> [v2:172.21.15.203:6832/3082732769,v1:172.21.15.203:6833/3082732769] conn(0x7ff91c06a170 0x7ff91c06c630 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7ff92c009990 tx=0x7ff92c008040 comp rx=0 tx=0).ready entity=mgr.8500 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-09T15:08:37.271 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:37.266+0000 7ff9397fa640 1 -- 172.21.15.43:0/2416345075 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178066 (secure 0 0 0) 0x7ff924053cd0 con 0x7ff93c0c4100 2023-08-09T15:08:37.464 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:37.463+0000 7ff942776640 1 -- 172.21.15.43:0/2416345075 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7ff93c0b3b70 con 0x7ff93c0c4100 2023-08-09T15:08:37.465 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:37.464+0000 7ff9397fa640 1 -- 172.21.15.43:0/2416345075 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v197) v1 ==== 74+0+16502 (secure 0 0 0) 0x7ff9240572f0 con 0x7ff93c0c4100 2023-08-09T15:08:37.466 INFO:teuthology.orchestra.run.smithi043.stdout: 2023-08-09T15:08:37.466 INFO:teuthology.orchestra.run.smithi043.stdout:{"epoch":197,"fsid":"daac87ee-1337-4250-a955-aea4e408a39b","created":"2023-08-09T13:33:41.701198+0000","modified":"2023-08-09T15:08:33.977927+0000","last_up_change":"2023-08-09T13:33:53.783453+0000","last_in_change":"2023-08-09T13:33:45.148904+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":23,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":"rbd","create_time":"2023-08-09T13:33:54.365137+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":".mgr","create_time":"2023-08-09T13:33:54.831803+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"cbd49ecd-e896-4fbd-864e-5c37a829b165","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":184,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6800","nonce":502667335},{"type":"v1","addr":"172.21.15.43:6801","nonce":502667335}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6802","nonce":502667335},{"type":"v1","addr":"172.21.15.43:6803","nonce":502667335}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6806","nonce":502667335},{"type":"v1","addr":"172.21.15.43:6807","nonce":502667335}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6804","nonce":502667335},{"type":"v1","addr":"172.21.15.43:6805","nonce":502667335}]},"public_addr":"172.21.15.43:6801/502667335","cluster_addr":"172.21.15.43:6803/502667335","heartbeat_back_addr":"172.21.15.43:6807/502667335","heartbeat_front_addr":"172.21.15.43:6805/502667335","state":["exists","up"]},{"osd":1,"uuid":"75c7e0fe-1ec7-4fe8-a2f2-3661a65b8500","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":184,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6808","nonce":3289134610},{"type":"v1","addr":"172.21.15.43:6809","nonce":3289134610}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6810","nonce":3289134610},{"type":"v1","addr":"172.21.15.43:6812","nonce":3289134610}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6821","nonce":3289134610},{"type":"v1","addr":"172.21.15.43:6824","nonce":3289134610}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6815","nonce":3289134610},{"type":"v1","addr":"172.21.15.43:6818","nonce":3289134610}]},"public_addr":"172.21.15.43:6809/3289134610","cluster_addr":"172.21.15.43:6812/3289134610","heartbeat_back_addr":"172.21.15.43:6824/3289134610","heartbeat_front_addr":"172.21.15.43:6818/3289134610","state":["exists","up"]},{"osd":2,"uuid":"bb5c41e9-7f5f-4125-9b01-3bb8bfc2ea83","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":184,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6811","nonce":4079709736},{"type":"v1","addr":"172.21.15.43:6814","nonce":4079709736}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6817","nonce":4079709736},{"type":"v1","addr":"172.21.15.43:6820","nonce":4079709736}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6829","nonce":4079709736},{"type":"v1","addr":"172.21.15.43:6831","nonce":4079709736}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6823","nonce":4079709736},{"type":"v1","addr":"172.21.15.43:6826","nonce":4079709736}]},"public_addr":"172.21.15.43:6814/4079709736","cluster_addr":"172.21.15.43:6820/4079709736","heartbeat_back_addr":"172.21.15.43:6831/4079709736","heartbeat_front_addr":"172.21.15.43:6826/4079709736","state":["exists","up"]},{"osd":3,"uuid":"bfe4c8b4-694a-465c-87e6-2b2aab852613","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":184,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6813","nonce":1709538759},{"type":"v1","addr":"172.21.15.43:6816","nonce":1709538759}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6819","nonce":1709538759},{"type":"v1","addr":"172.21.15.43:6822","nonce":1709538759}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6828","nonce":1709538759},{"type":"v1","addr":"172.21.15.43:6830","nonce":1709538759}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6825","nonce":1709538759},{"type":"v1","addr":"172.21.15.43:6827","nonce":1709538759}]},"public_addr":"172.21.15.43:6816/1709538759","cluster_addr":"172.21.15.43:6822/1709538759","heartbeat_back_addr":"172.21.15.43:6830/1709538759","heartbeat_front_addr":"172.21.15.43:6827/1709538759","state":["exists","up"]},{"osd":4,"uuid":"fa2e8a77-aba1-4756-853e-0d174a5bdeba","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":184,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6800","nonce":4196866481},{"type":"v1","addr":"172.21.15.203:6801","nonce":4196866481}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6802","nonce":4196866481},{"type":"v1","addr":"172.21.15.203:6803","nonce":4196866481}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6806","nonce":4196866481},{"type":"v1","addr":"172.21.15.203:6807","nonce":4196866481}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6804","nonce":4196866481},{"type":"v1","addr":"172.21.15.203:6805","nonce":4196866481}]},"public_addr":"172.21.15.203:6801/4196866481","cluster_addr":"172.21.15.203:6803/4196866481","heartbeat_back_addr":"172.21.15.203:6807/4196866481","heartbeat_front_addr":"172.21.15.203:6805/4196866481","state":["exists","up"]},{"osd":5,"uuid":"81811e4b-4d24-4f51-a82c-f1d700a2a00d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":184,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6816","nonce":3059290081},{"type":"v1","addr":"172.21.15.203:6817","nonce":3059290081}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6818","nonce":3059290081},{"type":"v1","addr":"172.21.15.203:6819","nonce":3059290081}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6822","nonce":3059290081},{"type":"v1","addr":"172.21.15.203:6823","nonce":3059290081}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6820","nonce":3059290081},{"type":"v1","addr":"172.21.15.203:6821","nonce":3059290081}]},"public_addr":"172.21.15.203:6817/3059290081","cluster_addr":"172.21.15.203:6819/3059290081","heartbeat_back_addr":"172.21.15.203:6823/3059290081","heartbeat_front_addr":"172.21.15.203:6821/3059290081","state":["exists","up"]},{"osd":6,"uuid":"307bac4d-6a5b-443d-af75-56d695d7674f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":184,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6824","nonce":512155569},{"type":"v1","addr":"172.21.15.203:6825","nonce":512155569}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6826","nonce":512155569},{"type":"v1","addr":"172.21.15.203:6827","nonce":512155569}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6830","nonce":512155569},{"type":"v1","addr":"172.21.15.203:6831","nonce":512155569}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6828","nonce":512155569},{"type":"v1","addr":"172.21.15.203:6829","nonce":512155569}]},"public_addr":"172.21.15.203:6825/512155569","cluster_addr":"172.21.15.203:6827/512155569","heartbeat_back_addr":"172.21.15.203:6831/512155569","heartbeat_front_addr":"172.21.15.203:6829/512155569","state":["exists","up"]},{"osd":7,"uuid":"a95dc7ca-9f1b-4295-af70-68a22a2d7d5e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":184,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6808","nonce":3331450438},{"type":"v1","addr":"172.21.15.203:6809","nonce":3331450438}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6810","nonce":3331450438},{"type":"v1","addr":"172.21.15.203:6811","nonce":3331450438}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6814","nonce":3331450438},{"type":"v1","addr":"172.21.15.203:6815","nonce":3331450438}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6812","nonce":3331450438},{"type":"v1","addr":"172.21.15.203:6813","nonce":3331450438}]},"public_addr":"172.21.15.203:6809/3331450438","cluster_addr":"172.21.15.203:6811/3331450438","heartbeat_back_addr":"172.21.15.203:6815/3331450438","heartbeat_front_addr":"172.21.15.203:6813/3331450438","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-09T13:33:50.547652+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-09T13:33:50.589469+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-09T13:33:50.561724+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-09T13:33:50.599650+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-09T13:33:50.411855+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-09T13:33:50.219681+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-09T13:33:50.504664+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-09T13:33:50.441248+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.43:6834/2866936881":"2023-08-10T15:08:30.956759+0000","172.21.15.203:0/1820136184":"2023-08-10T15:08:25.567876+0000","172.21.15.203:6833/115830229":"2023-08-10T15:08:25.567876+0000","172.21.15.203:6832/115830229":"2023-08-10T15:08:25.567876+0000","172.21.15.203:0/3317788314":"2023-08-10T15:08:25.567876+0000","172.21.15.203:0/69497804":"2023-08-10T15:03:21.353235+0000","172.21.15.203:0/169967264":"2023-08-10T15:03:21.353235+0000","172.21.15.203:6833/40883597":"2023-08-10T15:03:21.353235+0000","172.21.15.203:0/1738813745":"2023-08-10T15:08:25.567876+0000","172.21.15.203:0/1306452763":"2023-08-10T15:08:25.567876+0000","172.21.15.203:6832/40883597":"2023-08-10T15:03:21.353235+0000","172.21.15.43:6832/2866936881":"2023-08-10T15:08:30.956759+0000","172.21.15.203:0/1225284485":"2023-08-10T15:08:25.567876+0000","172.21.15.203:0/3690197627":"2023-08-10T15:03:21.353235+0000","172.21.15.203:0/1082753911":"2023-08-10T15:03:21.353235+0000","172.21.15.203:6832/2959400181":"2023-08-10T14:52:45.188641+0000","172.21.15.203:0/82655847":"2023-08-10T15:03:21.353235+0000","172.21.15.203:0/1935803093":"2023-08-10T14:52:45.188641+0000","172.21.15.203:0/3150685329":"2023-08-10T14:52:45.188641+0000","172.21.15.203:0/2219953609":"2023-08-10T15:03:21.353235+0000","172.21.15.203:0/3429376127":"2023-08-10T14:52:45.188641+0000","172.21.15.203:0/1394945671":"2023-08-10T15:08:25.567876+0000","172.21.15.203:0/2197573025":"2023-08-10T14:52:45.188641+0000","172.21.15.203:0/2458603841":"2023-08-10T14:52:45.188641+0000","172.21.15.203:6833/2959400181":"2023-08-10T14:52:45.188641+0000","172.21.15.203:0/4177046277":"2023-08-10T14:52:45.188641+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":7,"snaps":[{"begin":104,"length":1},{"begin":221,"length":1}]}],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-08-09T15:08:37.468 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:37.467+0000 7ff942776640 1 -- 172.21.15.43:0/2416345075 >> [v2:172.21.15.203:6832/3082732769,v1:172.21.15.203:6833/3082732769] conn(0x7ff91c06a170 msgr2=0x7ff91c06c630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T15:08:37.468 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:37.467+0000 7ff942776640 1 --2- 172.21.15.43:0/2416345075 >> [v2:172.21.15.203:6832/3082732769,v1:172.21.15.203:6833/3082732769] conn(0x7ff91c06a170 0x7ff91c06c630 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7ff92c009990 tx=0x7ff92c008040 comp rx=0 tx=0).stop 2023-08-09T15:08:37.468 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:37.467+0000 7ff942776640 1 -- 172.21.15.43:0/2416345075 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff93c0c4100 msgr2=0x7ff93c0c4560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T15:08:37.469 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:37.467+0000 7ff942776640 1 --2- 172.21.15.43:0/2416345075 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff93c0c4100 0x7ff93c0c4560 secure :-1 s=READY pgs=2169 cs=0 l=1 rev1=1 crypto rx=0x7ff92400e750 tx=0x7ff92400ec10 comp rx=0 tx=0).stop 2023-08-09T15:08:37.469 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:37.467+0000 7ff942776640 1 -- 172.21.15.43:0/2416345075 shutdown_connections 2023-08-09T15:08:37.469 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:37.467+0000 7ff942776640 1 --2- 172.21.15.43:0/2416345075 >> [v2:172.21.15.203:6832/3082732769,v1:172.21.15.203:6833/3082732769] conn(0x7ff91c06a170 0x7ff91c06c630 unknown :-1 s=CLOSED pgs=42 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T15:08:37.469 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:37.467+0000 7ff942776640 1 --2- 172.21.15.43:0/2416345075 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7ff93c0b2070 0x7ff93c0b2500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T15:08:37.470 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:37.467+0000 7ff942776640 1 --2- 172.21.15.43:0/2416345075 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff93c0c4100 0x7ff93c0c4560 unknown :-1 s=CLOSED pgs=2169 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T15:08:37.470 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:37.467+0000 7ff942776640 1 --2- 172.21.15.43:0/2416345075 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7ff9280052a0 0x7ff93c141fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T15:08:37.470 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:37.467+0000 7ff942776640 1 -- 172.21.15.43:0/2416345075 >> 172.21.15.43:0/2416345075 conn(0x7ff93c0ce0a0 msgr2=0x7ff93c0ca9d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-09T15:08:37.470 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:37.467+0000 7ff942776640 1 -- 172.21.15.43:0/2416345075 shutdown_connections 2023-08-09T15:08:37.470 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:37.467+0000 7ff942776640 1 -- 172.21.15.43:0/2416345075 wait complete. 2023-08-09T15:08:37.489 INFO:tasks.cephfs.cephfs_test_case.ceph_manager:[{'pool': 1, 'pool_name': 'rbd', 'create_time': '2023-08-09T13:33:54.365137+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}, 'read_balance': {'score_acting': 2, 'score_stable': 2, 'optimal_score': 1, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': '.mgr', 'create_time': '2023-08-09T13:33:54.831803+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-08-09T15:08:37.490 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get rbd pg_num 2023-08-09T15:08:37.726 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:37.725+0000 7f77197eb640 1 Processor -- start 2023-08-09T15:08:37.726 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:37.725+0000 7f77197eb640 1 -- start start 2023-08-09T15:08:37.727 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:37.726+0000 7f77197eb640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f771415c2d0 0x7f77140b7df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T15:08:37.728 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:37.726+0000 7f77197eb640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f77140cebf0 0x7f77140cf080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T15:08:37.728 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:37.726+0000 7f77197eb640 1 -- --> v1:172.21.15.203:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f77140d16a0 con 0x7f77140d1b30 2023-08-09T15:08:37.728 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:37.726+0000 7f77197eb640 1 -- --> v2:172.21.15.43:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f77140cf5c0 con 0x7f77140cebf0 2023-08-09T15:08:37.728 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:37.726+0000 7f77197eb640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f77140d1f70 con 0x7f771415c2d0 2023-08-09T15:08:37.729 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:37.727+0000 7f7718fea640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f77140cebf0 0x7f77140cf080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T15:08:37.729 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:37.727+0000 7f7718fea640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f77140cebf0 0x7f77140cf080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:39298/0 (socket says 172.21.15.43:39298) 2023-08-09T15:08:37.729 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:37.727+0000 7f7718fea640 1 -- 172.21.15.43:0/1956297000 learned_addr learned my addr 172.21.15.43:0/1956297000 (peer_addr_for_me v2:172.21.15.43:0/0) 2023-08-09T15:08:37.729 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:37.727+0000 7f7713fff640 1 --2- 172.21.15.43:0/1956297000 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f771415c2d0 0x7f77140b7df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T15:08:37.729 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:37.727+0000 7f7718fea640 1 -- 172.21.15.43:0/1956297000 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f771415c2d0 msgr2=0x7f77140b7df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-09T15:08:37.730 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:37.727+0000 7f7718fea640 1 --2- 172.21.15.43:0/1956297000 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f771415c2d0 0x7f77140b7df0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T15:08:37.730 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:37.727+0000 7f7718fea640 1 -- 172.21.15.43:0/1956297000 >> v1:172.21.15.203:6789/0 conn(0x7f77140d1b30 legacy=0x7f77140cd020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-09T15:08:37.730 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:37.727+0000 7f7718fea640 1 -- 172.21.15.43:0/1956297000 --> v2:172.21.15.43:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f77140d0d70 con 0x7f77140cebf0 2023-08-09T15:08:37.731 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:37.729+0000 7f7718fea640 1 --2- 172.21.15.43:0/1956297000 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f77140cebf0 0x7f77140cf080 secure :-1 s=READY pgs=2170 cs=0 l=1 rev1=1 crypto rx=0x7f770c009870 tx=0x7f770c02b8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8fc47c999a90efe2 server_cookie=0 in_seq=0 out_seq=0 2023-08-09T15:08:37.731 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:37.729+0000 7f7712ffd640 1 -- 172.21.15.43:0/1956297000 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f770c031640 con 0x7f77140cebf0 2023-08-09T15:08:37.731 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:37.729+0000 7f7712ffd640 1 -- 172.21.15.43:0/1956297000 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f770c0317a0 con 0x7f77140cebf0 2023-08-09T15:08:37.731 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:37.729+0000 7f7712ffd640 1 -- 172.21.15.43:0/1956297000 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f770c031b30 con 0x7f77140cebf0 2023-08-09T15:08:37.731 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:37.729+0000 7f77197eb640 1 -- 172.21.15.43:0/1956297000 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f77140cebf0 msgr2=0x7f77140cf080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T15:08:37.732 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:37.729+0000 7f77197eb640 1 --2- 172.21.15.43:0/1956297000 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f77140cebf0 0x7f77140cf080 secure :-1 s=READY pgs=2170 cs=0 l=1 rev1=1 crypto rx=0x7f770c009870 tx=0x7f770c02b8f0 comp rx=0 tx=0).stop 2023-08-09T15:08:37.732 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:37.730+0000 7f77197eb640 1 -- 172.21.15.43:0/1956297000 shutdown_connections 2023-08-09T15:08:37.732 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:37.730+0000 7f77197eb640 1 --2- 172.21.15.43:0/1956297000 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f77140cebf0 0x7f77140cf080 unknown :-1 s=CLOSED pgs=2170 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T15:08:37.732 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:37.730+0000 7f77197eb640 1 --2- 172.21.15.43:0/1956297000 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f771415c2d0 0x7f77140b7df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T15:08:37.732 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:37.730+0000 7f77197eb640 1 -- 172.21.15.43:0/1956297000 >> 172.21.15.43:0/1956297000 conn(0x7f771406a230 msgr2=0x7f771406a640 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-09T15:08:37.732 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:37.730+0000 7f77197eb640 1 -- 172.21.15.43:0/1956297000 shutdown_connections 2023-08-09T15:08:37.732 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:37.730+0000 7f77197eb640 1 -- 172.21.15.43:0/1956297000 wait complete. 2023-08-09T15:08:37.733 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:37.731+0000 7f77197eb640 1 Processor -- start 2023-08-09T15:08:37.733 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:37.732+0000 7f77197eb640 1 -- start start 2023-08-09T15:08:37.735 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:37.732+0000 7f77197eb640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f77140cebf0 0x7f77140b68e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T15:08:37.735 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:37.732+0000 7f77197eb640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f77140d1b30 0x7f771415e350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T15:08:37.735 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:37.733+0000 7f77197eb640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f771415c2d0 0x7f771415e890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T15:08:37.735 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:37.733+0000 7f77197eb640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7714156640 con 0x7f77140cebf0 2023-08-09T15:08:37.735 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:37.733+0000 7f77197eb640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f771415eec0 con 0x7f771415c2d0 2023-08-09T15:08:37.735 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:37.733+0000 7f77197eb640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f771415f1a0 con 0x7f77140d1b30 2023-08-09T15:08:37.736 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:37.733+0000 7f7713fff640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f77140cebf0 0x7f77140b68e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T15:08:37.736 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:37.733+0000 7f7713fff640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f77140cebf0 0x7f77140b68e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:39300/0 (socket says 172.21.15.43:39300) 2023-08-09T15:08:37.736 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:37.733+0000 7f7713fff640 1 -- 172.21.15.43:0/3613180707 learned_addr learned my addr 172.21.15.43:0/3613180707 (peer_addr_for_me v2:172.21.15.43:0/0) 2023-08-09T15:08:37.736 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:37.733+0000 7f77137fe640 1 --2- 172.21.15.43:0/3613180707 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f77140d1b30 0x7f771415e350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T15:08:37.736 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:37.733+0000 7f7718fea640 1 --2- 172.21.15.43:0/3613180707 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f771415c2d0 0x7f771415e890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T15:08:37.736 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:37.733+0000 7f7713fff640 1 -- 172.21.15.43:0/3613180707 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f77140d1b30 msgr2=0x7f771415e350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T15:08:37.736 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:37.733+0000 7f7713fff640 1 --2- 172.21.15.43:0/3613180707 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f77140d1b30 0x7f771415e350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T15:08:37.736 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:37.733+0000 7f7713fff640 1 -- 172.21.15.43:0/3613180707 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f771415c2d0 msgr2=0x7f771415e890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T15:08:37.737 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:37.733+0000 7f7713fff640 1 --2- 172.21.15.43:0/3613180707 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f771415c2d0 0x7f771415e890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T15:08:37.737 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:37.733+0000 7f7713fff640 1 -- 172.21.15.43:0/3613180707 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f770c0094d0 con 0x7f77140cebf0 2023-08-09T15:08:37.737 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:37.733+0000 7f77137fe640 1 --2- 172.21.15.43:0/3613180707 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f77140d1b30 0x7f771415e350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-08-09T15:08:37.737 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:37.733+0000 7f7718fea640 1 --2- 172.21.15.43:0/3613180707 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f771415c2d0 0x7f771415e890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-08-09T15:08:37.737 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:37.734+0000 7f7713fff640 1 --2- 172.21.15.43:0/3613180707 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f77140cebf0 0x7f77140b68e0 secure :-1 s=READY pgs=2171 cs=0 l=1 rev1=1 crypto rx=0x7f7704007ef0 tx=0x7f770400e4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-09T15:08:37.738 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:37.734+0000 7f7710ff9640 1 -- 172.21.15.43:0/3613180707 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7704014620 con 0x7f77140cebf0 2023-08-09T15:08:37.738 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:37.734+0000 7f7710ff9640 1 -- 172.21.15.43:0/3613180707 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7704014780 con 0x7f77140cebf0 2023-08-09T15:08:37.738 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:37.734+0000 7f7710ff9640 1 -- 172.21.15.43:0/3613180707 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7704014b30 con 0x7f77140cebf0 2023-08-09T15:08:37.738 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:37.734+0000 7f77197eb640 1 -- 172.21.15.43:0/3613180707 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f771415f480 con 0x7f77140cebf0 2023-08-09T15:08:37.738 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:37.734+0000 7f77197eb640 1 -- 172.21.15.43:0/3613180707 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7714152b40 con 0x7f77140cebf0 2023-08-09T15:08:37.739 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:37.735+0000 7f77197eb640 1 -- 172.21.15.43:0/3613180707 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f771404e390 con 0x7f77140cebf0 2023-08-09T15:08:37.739 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:37.736+0000 7f7710ff9640 1 -- 172.21.15.43:0/3613180707 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 106) v1 ==== 88443+0+0 (secure 0 0 0) 0x7f7704014c90 con 0x7f77140cebf0 2023-08-09T15:08:37.739 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:37.737+0000 7f7710ff9640 1 --2- 172.21.15.43:0/3613180707 >> [v2:172.21.15.203:6832/3082732769,v1:172.21.15.203:6833/3082732769] conn(0x7f770006a170 0x7f770006c630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T15:08:37.739 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:37.737+0000 7f7710ff9640 1 -- 172.21.15.43:0/3613180707 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(197..197 src has 1..197) v4 ==== 6692+0+0 (secure 0 0 0) 0x7f77040894a0 con 0x7f77140cebf0 2023-08-09T15:08:37.740 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:37.738+0000 7f77137fe640 1 --2- 172.21.15.43:0/3613180707 >> [v2:172.21.15.203:6832/3082732769,v1:172.21.15.203:6833/3082732769] conn(0x7f770006a170 0x7f770006c630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T15:08:37.740 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:37.739+0000 7f77137fe640 1 --2- 172.21.15.43:0/3613180707 >> [v2:172.21.15.203:6832/3082732769,v1:172.21.15.203:6833/3082732769] conn(0x7f770006a170 0x7f770006c630 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7f77141f5220 tx=0x7f7708009040 comp rx=0 tx=0).ready entity=mgr.8500 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-09T15:08:37.743 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:37.742+0000 7f7710ff9640 1 -- 172.21.15.43:0/3613180707 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178066 (secure 0 0 0) 0x7f7704054760 con 0x7f77140cebf0 2023-08-09T15:08:37.941 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:37.935+0000 7f77197eb640 1 -- 172.21.15.43:0/3613180707 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"} v 0) v1 -- 0x7f77140cf3a0 con 0x7f77140cebf0 2023-08-09T15:08:37.942 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:37.936+0000 7f7710ff9640 1 -- 172.21.15.43:0/3613180707 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"}]=0 v197) v1 ==== 92+0+10 (secure 0 0 0) 0x7f7704057d80 con 0x7f77140cebf0 2023-08-09T15:08:37.942 INFO:teuthology.orchestra.run.smithi043.stdout:pg_num: 8 2023-08-09T15:08:37.943 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:37.938+0000 7f77197eb640 1 -- 172.21.15.43:0/3613180707 >> [v2:172.21.15.203:6832/3082732769,v1:172.21.15.203:6833/3082732769] conn(0x7f770006a170 msgr2=0x7f770006c630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T15:08:37.943 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:37.938+0000 7f77197eb640 1 --2- 172.21.15.43:0/3613180707 >> [v2:172.21.15.203:6832/3082732769,v1:172.21.15.203:6833/3082732769] conn(0x7f770006a170 0x7f770006c630 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7f77141f5220 tx=0x7f7708009040 comp rx=0 tx=0).stop 2023-08-09T15:08:37.943 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:37.938+0000 7f77197eb640 1 -- 172.21.15.43:0/3613180707 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f77140cebf0 msgr2=0x7f77140b68e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T15:08:37.943 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:37.938+0000 7f77197eb640 1 --2- 172.21.15.43:0/3613180707 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f77140cebf0 0x7f77140b68e0 secure :-1 s=READY pgs=2171 cs=0 l=1 rev1=1 crypto rx=0x7f7704007ef0 tx=0x7f770400e4b0 comp rx=0 tx=0).stop 2023-08-09T15:08:37.943 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:37.939+0000 7f77197eb640 1 -- 172.21.15.43:0/3613180707 shutdown_connections 2023-08-09T15:08:37.943 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:37.939+0000 7f77197eb640 1 --2- 172.21.15.43:0/3613180707 >> [v2:172.21.15.203:6832/3082732769,v1:172.21.15.203:6833/3082732769] conn(0x7f770006a170 0x7f770006c630 unknown :-1 s=CLOSED pgs=43 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T15:08:37.944 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:37.939+0000 7f77197eb640 1 --2- 172.21.15.43:0/3613180707 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f771415c2d0 0x7f771415e890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T15:08:37.944 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:37.939+0000 7f77197eb640 1 --2- 172.21.15.43:0/3613180707 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f77140d1b30 0x7f771415e350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T15:08:37.944 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:37.939+0000 7f77197eb640 1 --2- 172.21.15.43:0/3613180707 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f77140cebf0 0x7f77140b68e0 unknown :-1 s=CLOSED pgs=2171 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T15:08:37.944 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:37.939+0000 7f77197eb640 1 -- 172.21.15.43:0/3613180707 >> 172.21.15.43:0/3613180707 conn(0x7f771406a230 msgr2=0x7f77140cccc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-09T15:08:37.944 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:37.939+0000 7f77197eb640 1 -- 172.21.15.43:0/3613180707 shutdown_connections 2023-08-09T15:08:37.944 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:37.939+0000 7f77197eb640 1 -- 172.21.15.43:0/3613180707 wait complete. 2023-08-09T15:08:37.961 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-08-09T15:08:38.194 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:38.193+0000 7f2ac5100640 1 Processor -- start 2023-08-09T15:08:38.194 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:38.193+0000 7f2ac5100640 1 -- start start 2023-08-09T15:08:38.196 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:38.195+0000 7f2ac5100640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f2ac00b45f0 0x7f2ac00cd6d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T15:08:38.196 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:38.195+0000 7f2ac5100640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f2ac00d04b0 0x7f2ac00b59d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T15:08:38.196 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:38.195+0000 7f2ac5100640 1 -- --> v1:172.21.15.43:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2ac00b7bd0 con 0x7f2ac015c550 2023-08-09T15:08:38.197 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:38.195+0000 7f2ac5100640 1 -- --> v2:172.21.15.43:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2ac00d1c00 con 0x7f2ac00b45f0 2023-08-09T15:08:38.197 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:38.195+0000 7f2ac5100640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2ac015c930 con 0x7f2ac00d04b0 2023-08-09T15:08:38.198 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:38.195+0000 7f2abed9d640 1 --1- >> v1:172.21.15.43:6789/0 conn(0x7f2ac015c550 0x7f2ac00b3ee0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.43:6789/0 says I am v1:172.21.15.43:59582/0 (socket says 172.21.15.43:59582) 2023-08-09T15:08:38.198 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:38.195+0000 7f2abed9d640 1 -- 172.21.15.43:0/1900513260 learned_addr learned my addr 172.21.15.43:0/1900513260 (peer_addr_for_me v1:172.21.15.43:0/0) 2023-08-09T15:08:38.198 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:38.195+0000 7f2abe59c640 1 --2- 172.21.15.43:0/1900513260 >> v2:172.21.15.43:3300/0 conn(0x7f2ac00b45f0 0x7f2ac00cd6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T15:08:38.198 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:38.195+0000 7f2abf59e640 1 --2- 172.21.15.43:0/1900513260 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f2ac00d04b0 0x7f2ac00b59d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T15:08:38.199 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:38.196+0000 7f2abe59c640 1 -- 172.21.15.43:0/1900513260 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f2ac00d04b0 msgr2=0x7f2ac00b59d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-09T15:08:38.199 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:38.196+0000 7f2abe59c640 1 --2- 172.21.15.43:0/1900513260 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f2ac00d04b0 0x7f2ac00b59d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T15:08:38.199 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:38.196+0000 7f2abe59c640 1 -- 172.21.15.43:0/1900513260 >> v1:172.21.15.43:6789/0 conn(0x7f2ac015c550 legacy=0x7f2ac00b3ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T15:08:38.199 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:38.196+0000 7f2abe59c640 1 -- 172.21.15.43:0/1900513260 --> v2:172.21.15.43:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2ac00b54f0 con 0x7f2ac00b45f0 2023-08-09T15:08:38.199 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:38.196+0000 7f2abdd9b640 1 -- 172.21.15.43:0/1900513260 <== mon.0 v1:172.21.15.43:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1442009478 0 0) 0x7f2ac00b7bd0 con 0x7f2ac015c550 2023-08-09T15:08:38.199 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:38.196+0000 7f2abf59e640 1 --2- 172.21.15.43:0/1900513260 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f2ac00d04b0 0x7f2ac00b59d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-08-09T15:08:38.199 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:38.197+0000 7f2abe59c640 1 --2- 172.21.15.43:0/1900513260 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f2ac00b45f0 0x7f2ac00cd6d0 secure :-1 s=READY pgs=2173 cs=0 l=1 rev1=1 crypto rx=0x7f2ab40097a0 tx=0x7f2ab402b8b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8bf0654f77575291 server_cookie=0 in_seq=0 out_seq=0 2023-08-09T15:08:38.200 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:38.197+0000 7f2abdd9b640 1 -- 172.21.15.43:0/1900513260 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2ab402d040 con 0x7f2ac00b45f0 2023-08-09T15:08:38.201 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:38.197+0000 7f2abdd9b640 1 -- 172.21.15.43:0/1900513260 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2ab4031810 con 0x7f2ac00b45f0 2023-08-09T15:08:38.201 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:38.197+0000 7f2abdd9b640 1 -- 172.21.15.43:0/1900513260 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2ab4031ba0 con 0x7f2ac00b45f0 2023-08-09T15:08:38.201 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:38.198+0000 7f2ac5100640 1 -- 172.21.15.43:0/1900513260 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f2ac00b45f0 msgr2=0x7f2ac00cd6d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T15:08:38.201 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:38.198+0000 7f2ac5100640 1 --2- 172.21.15.43:0/1900513260 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f2ac00b45f0 0x7f2ac00cd6d0 secure :-1 s=READY pgs=2173 cs=0 l=1 rev1=1 crypto rx=0x7f2ab40097a0 tx=0x7f2ab402b8b0 comp rx=0 tx=0).stop 2023-08-09T15:08:38.201 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:38.198+0000 7f2ac5100640 1 -- 172.21.15.43:0/1900513260 shutdown_connections 2023-08-09T15:08:38.201 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:38.198+0000 7f2ac5100640 1 --2- 172.21.15.43:0/1900513260 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f2ac00d04b0 0x7f2ac00b59d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T15:08:38.201 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:38.198+0000 7f2ac5100640 1 --2- 172.21.15.43:0/1900513260 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f2ac00b45f0 0x7f2ac00cd6d0 unknown :-1 s=CLOSED pgs=2173 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T15:08:38.202 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:38.198+0000 7f2ac5100640 1 -- 172.21.15.43:0/1900513260 >> 172.21.15.43:0/1900513260 conn(0x7f2ac0071520 msgr2=0x7f2ac00ceee0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-09T15:08:38.202 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:38.198+0000 7f2ac5100640 1 -- 172.21.15.43:0/1900513260 shutdown_connections 2023-08-09T15:08:38.202 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:38.198+0000 7f2ac5100640 1 -- 172.21.15.43:0/1900513260 wait complete. 2023-08-09T15:08:38.202 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:38.200+0000 7f2ac5100640 1 Processor -- start 2023-08-09T15:08:38.204 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:38.200+0000 7f2ac5100640 1 -- start start 2023-08-09T15:08:38.204 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:38.201+0000 7f2ac5100640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f2ac00b45f0 0x7f2ac0156d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T15:08:38.204 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:38.201+0000 7f2ac5100640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f2ac00d04b0 0x7f2ac00a9790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T15:08:38.205 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:38.201+0000 7f2ac5100640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f2ac015c550 0x7f2ac015ce60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T15:08:38.205 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:38.201+0000 7f2ac5100640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2ac0157640 con 0x7f2ac015c550 2023-08-09T15:08:38.205 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:38.201+0000 7f2ac5100640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2ac00ca770 con 0x7f2ac00d04b0 2023-08-09T15:08:38.205 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:38.201+0000 7f2ac5100640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2ac00aa0d0 con 0x7f2ac00b45f0 2023-08-09T15:08:38.205 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:38.201+0000 7f2abf59e640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f2ac015c550 0x7f2ac015ce60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T15:08:38.205 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:38.201+0000 7f2abf59e640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f2ac015c550 0x7f2ac015ce60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:39318/0 (socket says 172.21.15.43:39318) 2023-08-09T15:08:38.205 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:38.201+0000 7f2abf59e640 1 -- 172.21.15.43:0/1769855364 learned_addr learned my addr 172.21.15.43:0/1769855364 (peer_addr_for_me v2:172.21.15.43:0/0) 2023-08-09T15:08:38.206 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:38.201+0000 7f2abe59c640 1 --2- 172.21.15.43:0/1769855364 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f2ac00d04b0 0x7f2ac00a9790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T15:08:38.206 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:38.202+0000 7f2abed9d640 1 --2- 172.21.15.43:0/1769855364 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f2ac00b45f0 0x7f2ac0156d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T15:08:38.206 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:38.202+0000 7f2abf59e640 1 -- 172.21.15.43:0/1769855364 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f2ac00b45f0 msgr2=0x7f2ac0156d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T15:08:38.206 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:38.202+0000 7f2abf59e640 1 --2- 172.21.15.43:0/1769855364 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f2ac00b45f0 0x7f2ac0156d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T15:08:38.206 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:38.202+0000 7f2abf59e640 1 -- 172.21.15.43:0/1769855364 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f2ac00d04b0 msgr2=0x7f2ac00a9790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T15:08:38.207 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:38.202+0000 7f2abf59e640 1 --2- 172.21.15.43:0/1769855364 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f2ac00d04b0 0x7f2ac00a9790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T15:08:38.207 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:38.202+0000 7f2abf59e640 1 -- 172.21.15.43:0/1769855364 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2ab4009400 con 0x7f2ac015c550 2023-08-09T15:08:38.207 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:38.202+0000 7f2abe59c640 1 --2- 172.21.15.43:0/1769855364 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f2ac00d04b0 0x7f2ac00a9790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-08-09T15:08:38.207 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:38.202+0000 7f2abf59e640 1 --2- 172.21.15.43:0/1769855364 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f2ac015c550 0x7f2ac015ce60 secure :-1 s=READY pgs=2174 cs=0 l=1 rev1=1 crypto rx=0x7f2ab800c820 tx=0x7f2ab800cce0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-09T15:08:38.207 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:38.202+0000 7f2aa77fe640 1 -- 172.21.15.43:0/1769855364 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2ab8007a30 con 0x7f2ac015c550 2023-08-09T15:08:38.207 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:38.202+0000 7f2aa77fe640 1 -- 172.21.15.43:0/1769855364 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2ab8007b90 con 0x7f2ac015c550 2023-08-09T15:08:38.208 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:38.202+0000 7f2aa77fe640 1 -- 172.21.15.43:0/1769855364 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2ab800ce10 con 0x7f2ac015c550 2023-08-09T15:08:38.208 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:38.202+0000 7f2ac5100640 1 -- 172.21.15.43:0/1769855364 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2ac00ca9e0 con 0x7f2ac015c550 2023-08-09T15:08:38.208 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:38.203+0000 7f2ac5100640 1 -- 172.21.15.43:0/1769855364 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2ac00aa3b0 con 0x7f2ac015c550 2023-08-09T15:08:38.209 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:38.204+0000 7f2ac5100640 1 -- 172.21.15.43:0/1769855364 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2ac00739f0 con 0x7f2ac015c550 2023-08-09T15:08:38.209 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:38.205+0000 7f2aa77fe640 1 -- 172.21.15.43:0/1769855364 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 106) v1 ==== 88443+0+0 (secure 0 0 0) 0x7f2ab80113f0 con 0x7f2ac015c550 2023-08-09T15:08:38.209 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:38.206+0000 7f2aa77fe640 1 --2- 172.21.15.43:0/1769855364 >> [v2:172.21.15.203:6832/3082732769,v1:172.21.15.203:6833/3082732769] conn(0x7f2aac06a170 0x7f2aac06c630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T15:08:38.209 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:38.206+0000 7f2aa77fe640 1 -- 172.21.15.43:0/1769855364 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(197..197 src has 1..197) v4 ==== 6692+0+0 (secure 0 0 0) 0x7f2ab8088c20 con 0x7f2ac015c550 2023-08-09T15:08:38.209 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:38.206+0000 7f2abed9d640 1 --2- 172.21.15.43:0/1769855364 >> [v2:172.21.15.203:6832/3082732769,v1:172.21.15.203:6833/3082732769] conn(0x7f2aac06a170 0x7f2aac06c630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T15:08:38.209 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:38.207+0000 7f2abed9d640 1 --2- 172.21.15.43:0/1769855364 >> [v2:172.21.15.203:6832/3082732769,v1:172.21.15.203:6833/3082732769] conn(0x7f2aac06a170 0x7f2aac06c630 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7f2ab0007ed0 tx=0x7f2ab0007e60 comp rx=0 tx=0).ready entity=mgr.8500 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-09T15:08:38.211 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:38.210+0000 7f2aa77fe640 1 -- 172.21.15.43:0/1769855364 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178066 (secure 0 0 0) 0x7f2ab8053e60 con 0x7f2ac015c550 2023-08-09T15:08:38.407 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:38.406+0000 7f2ac5100640 1 -- 172.21.15.43:0/1769855364 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f2ac00caf00 con 0x7f2ac015c550 2023-08-09T15:08:38.408 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:38.407+0000 7f2aa77fe640 1 -- 172.21.15.43:0/1769855364 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v197) v1 ==== 93+0+10 (secure 0 0 0) 0x7f2ab8057480 con 0x7f2ac015c550 2023-08-09T15:08:38.409 INFO:teuthology.orchestra.run.smithi043.stdout:pg_num: 1 2023-08-09T15:08:38.411 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:38.409+0000 7f2ac5100640 1 -- 172.21.15.43:0/1769855364 >> [v2:172.21.15.203:6832/3082732769,v1:172.21.15.203:6833/3082732769] conn(0x7f2aac06a170 msgr2=0x7f2aac06c630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T15:08:38.411 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:38.409+0000 7f2ac5100640 1 --2- 172.21.15.43:0/1769855364 >> [v2:172.21.15.203:6832/3082732769,v1:172.21.15.203:6833/3082732769] conn(0x7f2aac06a170 0x7f2aac06c630 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7f2ab0007ed0 tx=0x7f2ab0007e60 comp rx=0 tx=0).stop 2023-08-09T15:08:38.411 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:38.410+0000 7f2ac5100640 1 -- 172.21.15.43:0/1769855364 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f2ac015c550 msgr2=0x7f2ac015ce60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T15:08:38.412 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:38.410+0000 7f2ac5100640 1 --2- 172.21.15.43:0/1769855364 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f2ac015c550 0x7f2ac015ce60 secure :-1 s=READY pgs=2174 cs=0 l=1 rev1=1 crypto rx=0x7f2ab800c820 tx=0x7f2ab800cce0 comp rx=0 tx=0).stop 2023-08-09T15:08:38.412 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:38.410+0000 7f2ac5100640 1 -- 172.21.15.43:0/1769855364 shutdown_connections 2023-08-09T15:08:38.412 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:38.410+0000 7f2ac5100640 1 --2- 172.21.15.43:0/1769855364 >> [v2:172.21.15.203:6832/3082732769,v1:172.21.15.203:6833/3082732769] conn(0x7f2aac06a170 0x7f2aac06c630 unknown :-1 s=CLOSED pgs=44 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T15:08:38.412 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:38.410+0000 7f2ac5100640 1 --2- 172.21.15.43:0/1769855364 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f2ac015c550 0x7f2ac015ce60 unknown :-1 s=CLOSED pgs=2174 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T15:08:38.412 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:38.410+0000 7f2ac5100640 1 --2- 172.21.15.43:0/1769855364 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f2ac00d04b0 0x7f2ac00a9790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T15:08:38.412 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:38.410+0000 7f2ac5100640 1 --2- 172.21.15.43:0/1769855364 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f2ac00b45f0 0x7f2ac0156d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T15:08:38.413 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:38.410+0000 7f2ac5100640 1 -- 172.21.15.43:0/1769855364 >> 172.21.15.43:0/1769855364 conn(0x7f2ac0071520 msgr2=0x7f2ac015ec70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-09T15:08:38.413 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:38.410+0000 7f2ac5100640 1 -- 172.21.15.43:0/1769855364 shutdown_connections 2023-08-09T15:08:38.413 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:38.410+0000 7f2ac5100640 1 -- 172.21.15.43:0/1769855364 wait complete. 2023-08-09T15:08:38.431 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mon mon_allow_pool_delete True 2023-08-09T15:08:38.667 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:38.666+0000 7f2518c26640 1 Processor -- start 2023-08-09T15:08:38.668 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:38.666+0000 7f2518c26640 1 -- start start 2023-08-09T15:08:38.669 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:38.668+0000 7f2518c26640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f25140ccf20 0x7f25140cd550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T15:08:38.669 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:38.668+0000 7f2518c26640 1 -- --> v1:172.21.15.43:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f25140cca90 con 0x7f251415c630 2023-08-09T15:08:38.669 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:38.668+0000 7f2518c26640 1 -- --> v1:172.21.15.203:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f25140cdea0 con 0x7f25140cda90 2023-08-09T15:08:38.670 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:38.668+0000 7f2518c26640 1 -- --> v2:172.21.15.43:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f25140ac600 con 0x7f25140ccf20 2023-08-09T15:08:38.670 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:38.668+0000 7f251259c640 1 --1- >> v1:172.21.15.43:6789/0 conn(0x7f251415c630 0x7f25140abef0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.43:6789/0 says I am v1:172.21.15.43:59586/0 (socket says 172.21.15.43:59586) 2023-08-09T15:08:38.671 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:38.668+0000 7f251259c640 1 -- 172.21.15.43:0/2429318363 learned_addr learned my addr 172.21.15.43:0/2429318363 (peer_addr_for_me v1:172.21.15.43:0/0) 2023-08-09T15:08:38.671 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:38.668+0000 7f250bfff640 1 --2- 172.21.15.43:0/2429318363 >> v2:172.21.15.43:3300/0 conn(0x7f25140ccf20 0x7f25140cd550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T15:08:38.671 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:38.669+0000 7f2511d9b640 1 -- 172.21.15.43:0/2429318363 <== mon.0 v1:172.21.15.43:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 760170881 0 0) 0x7f25140cca90 con 0x7f251415c630 2023-08-09T15:08:38.671 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:38.669+0000 7f2511d9b640 1 -- 172.21.15.43:0/2429318363 --> v1:172.21.15.43:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f2500003520 con 0x7f251415c630 2023-08-09T15:08:38.671 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:38.669+0000 7f250bfff640 1 -- 172.21.15.43:0/2429318363 >> v1:172.21.15.203:6789/0 conn(0x7f25140cda90 legacy=0x7f251415ca10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-09T15:08:38.671 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:38.669+0000 7f250bfff640 1 -- 172.21.15.43:0/2429318363 >> v1:172.21.15.43:6789/0 conn(0x7f251415c630 legacy=0x7f25140abef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T15:08:38.672 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:38.669+0000 7f250bfff640 1 -- 172.21.15.43:0/2429318363 --> v2:172.21.15.43:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f25140cbe80 con 0x7f25140ccf20 2023-08-09T15:08:38.673 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:38.670+0000 7f250bfff640 1 --2- 172.21.15.43:0/2429318363 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f25140ccf20 0x7f25140cd550 secure :-1 s=READY pgs=2176 cs=0 l=1 rev1=1 crypto rx=0x7f24fc0097a0 tx=0x7f24fc02b960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=73f85522995b02ac server_cookie=0 in_seq=0 out_seq=0 2023-08-09T15:08:38.673 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:38.670+0000 7f2511d9b640 1 -- 172.21.15.43:0/2429318363 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f24fc030610 con 0x7f25140ccf20 2023-08-09T15:08:38.673 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:38.670+0000 7f2511d9b640 1 -- 172.21.15.43:0/2429318363 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f24fc030770 con 0x7f25140ccf20 2023-08-09T15:08:38.673 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:38.670+0000 7f2511d9b640 1 -- 172.21.15.43:0/2429318363 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f24fc030ae0 con 0x7f25140ccf20 2023-08-09T15:08:38.673 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:38.671+0000 7f2518c26640 1 -- 172.21.15.43:0/2429318363 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f25140ccf20 msgr2=0x7f25140cd550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T15:08:38.673 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:38.671+0000 7f2518c26640 1 --2- 172.21.15.43:0/2429318363 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f25140ccf20 0x7f25140cd550 secure :-1 s=READY pgs=2176 cs=0 l=1 rev1=1 crypto rx=0x7f24fc0097a0 tx=0x7f24fc02b960 comp rx=0 tx=0).stop 2023-08-09T15:08:38.674 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:38.671+0000 7f2518c26640 1 -- 172.21.15.43:0/2429318363 shutdown_connections 2023-08-09T15:08:38.674 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:38.671+0000 7f2518c26640 1 --2- 172.21.15.43:0/2429318363 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f25140ccf20 0x7f25140cd550 unknown :-1 s=CLOSED pgs=2176 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T15:08:38.674 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:38.671+0000 7f2518c26640 1 -- 172.21.15.43:0/2429318363 >> 172.21.15.43:0/2429318363 conn(0x7f25140ce0a0 msgr2=0x7f25140ce480 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-09T15:08:38.674 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:38.671+0000 7f2518c26640 1 -- 172.21.15.43:0/2429318363 shutdown_connections 2023-08-09T15:08:38.674 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:38.671+0000 7f2518c26640 1 -- 172.21.15.43:0/2429318363 wait complete. 2023-08-09T15:08:38.674 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:38.673+0000 7f2518c26640 1 Processor -- start 2023-08-09T15:08:38.675 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:38.673+0000 7f2518c26640 1 -- start start 2023-08-09T15:08:38.676 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:38.674+0000 7f2518c26640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f25140ccf20 0x7f25140c5b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T15:08:38.677 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:38.674+0000 7f2518c26640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f25140cda90 0x7f25140c9120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T15:08:38.677 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:38.674+0000 7f2518c26640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f251415c630 0x7f25140c9660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T15:08:38.677 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:38.674+0000 7f2518c26640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f25140c6fe0 con 0x7f25140cda90 2023-08-09T15:08:38.677 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:38.674+0000 7f2518c26640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f25140c63d0 con 0x7f251415c630 2023-08-09T15:08:38.677 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:38.674+0000 7f2518c26640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f25140c66b0 con 0x7f25140ccf20 2023-08-09T15:08:38.677 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:38.675+0000 7f250bfff640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f25140cda90 0x7f25140c9120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T15:08:38.677 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:38.675+0000 7f250bfff640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f25140cda90 0x7f25140c9120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:39336/0 (socket says 172.21.15.43:39336) 2023-08-09T15:08:38.678 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:38.675+0000 7f250bfff640 1 -- 172.21.15.43:0/1884878713 learned_addr learned my addr 172.21.15.43:0/1884878713 (peer_addr_for_me v2:172.21.15.43:0/0) 2023-08-09T15:08:38.678 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:38.675+0000 7f250bfff640 1 -- 172.21.15.43:0/1884878713 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f25140ccf20 msgr2=0x7f25140c5b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T15:08:38.678 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:38.675+0000 7f250bfff640 1 --2- 172.21.15.43:0/1884878713 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f25140ccf20 0x7f25140c5b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T15:08:38.678 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:38.675+0000 7f250bfff640 1 -- 172.21.15.43:0/1884878713 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f251415c630 msgr2=0x7f25140c9660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T15:08:38.678 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:38.675+0000 7f250bfff640 1 --2- 172.21.15.43:0/1884878713 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f251415c630 0x7f25140c9660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T15:08:38.678 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:38.675+0000 7f250bfff640 1 -- 172.21.15.43:0/1884878713 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f24fc009400 con 0x7f25140cda90 2023-08-09T15:08:38.678 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:38.676+0000 7f250bfff640 1 --2- 172.21.15.43:0/1884878713 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f25140cda90 0x7f25140c9120 secure :-1 s=READY pgs=2177 cs=0 l=1 rev1=1 crypto rx=0x7f24fc02bf70 tx=0x7f24fc004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-09T15:08:38.679 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:38.676+0000 7f250affd640 1 -- 172.21.15.43:0/1884878713 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f24fc03e040 con 0x7f25140cda90 2023-08-09T15:08:38.679 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:38.676+0000 7f250affd640 1 -- 172.21.15.43:0/1884878713 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f24fc039c90 con 0x7f25140cda90 2023-08-09T15:08:38.679 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:38.676+0000 7f250affd640 1 -- 172.21.15.43:0/1884878713 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f24fc030e10 con 0x7f25140cda90 2023-08-09T15:08:38.680 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:38.676+0000 7f2518c26640 1 -- 172.21.15.43:0/1884878713 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f25140c6990 con 0x7f25140cda90 2023-08-09T15:08:38.680 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:38.676+0000 7f2518c26640 1 -- 172.21.15.43:0/1884878713 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f25140c60c0 con 0x7f25140cda90 2023-08-09T15:08:38.680 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:38.677+0000 7f2518c26640 1 -- 172.21.15.43:0/1884878713 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f25140b1240 con 0x7f25140cda90 2023-08-09T15:08:38.680 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:38.678+0000 7f250affd640 1 -- 172.21.15.43:0/1884878713 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 106) v1 ==== 88443+0+0 (secure 0 0 0) 0x7f24fc043390 con 0x7f25140cda90 2023-08-09T15:08:38.681 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:38.679+0000 7f250affd640 1 --2- 172.21.15.43:0/1884878713 >> [v2:172.21.15.203:6832/3082732769,v1:172.21.15.203:6833/3082732769] conn(0x7f24f406a170 0x7f24f406c630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T15:08:38.681 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:38.680+0000 7f250affd640 1 -- 172.21.15.43:0/1884878713 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(197..197 src has 1..197) v4 ==== 6692+0+0 (secure 0 0 0) 0x7f24fc0aacc0 con 0x7f25140cda90 2023-08-09T15:08:38.681 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:38.680+0000 7f251259c640 1 --2- 172.21.15.43:0/1884878713 >> [v2:172.21.15.203:6832/3082732769,v1:172.21.15.203:6833/3082732769] conn(0x7f24f406a170 0x7f24f406c630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T15:08:38.682 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:38.681+0000 7f251259c640 1 --2- 172.21.15.43:0/1884878713 >> [v2:172.21.15.203:6832/3082732769,v1:172.21.15.203:6833/3082732769] conn(0x7f24f406a170 0x7f24f406c630 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7f2504001570 tx=0x7f2504007930 comp rx=0 tx=0).ready entity=mgr.8500 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-09T15:08:38.685 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:38.684+0000 7f250affd640 1 -- 172.21.15.43:0/1884878713 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178066 (secure 0 0 0) 0x7f24fc0b1080 con 0x7f25140cda90 2023-08-09T15:08:38.886 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:38.884+0000 7f2518c26640 1 -- 172.21.15.43:0/1884878713 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command([{prefix=config set, name=mon_allow_pool_delete}] v 0) v1 -- 0x7f25140739f0 con 0x7f25140cda90 2023-08-09T15:08:38.893 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:38.892+0000 7f250affd640 1 -- 172.21.15.43:0/1884878713 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_allow_pool_delete}]=0 v64)=0 v64) v1 ==== 122+0+0 (secure 0 0 0) 0x7f24fc075e70 con 0x7f25140cda90 2023-08-09T15:08:38.896 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:38.894+0000 7f2518c26640 1 -- 172.21.15.43:0/1884878713 >> [v2:172.21.15.203:6832/3082732769,v1:172.21.15.203:6833/3082732769] conn(0x7f24f406a170 msgr2=0x7f24f406c630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T15:08:38.897 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:38.894+0000 7f2518c26640 1 --2- 172.21.15.43:0/1884878713 >> [v2:172.21.15.203:6832/3082732769,v1:172.21.15.203:6833/3082732769] conn(0x7f24f406a170 0x7f24f406c630 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7f2504001570 tx=0x7f2504007930 comp rx=0 tx=0).stop 2023-08-09T15:08:38.897 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:38.895+0000 7f2518c26640 1 -- 172.21.15.43:0/1884878713 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f25140cda90 msgr2=0x7f25140c9120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T15:08:38.897 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:38.895+0000 7f2518c26640 1 --2- 172.21.15.43:0/1884878713 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f25140cda90 0x7f25140c9120 secure :-1 s=READY pgs=2177 cs=0 l=1 rev1=1 crypto rx=0x7f24fc02bf70 tx=0x7f24fc004730 comp rx=0 tx=0).stop 2023-08-09T15:08:38.897 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:38.895+0000 7f2518c26640 1 -- 172.21.15.43:0/1884878713 shutdown_connections 2023-08-09T15:08:38.897 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:38.895+0000 7f2518c26640 1 --2- 172.21.15.43:0/1884878713 >> [v2:172.21.15.203:6832/3082732769,v1:172.21.15.203:6833/3082732769] conn(0x7f24f406a170 0x7f24f406c630 unknown :-1 s=CLOSED pgs=45 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T15:08:38.897 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:38.895+0000 7f2518c26640 1 --2- 172.21.15.43:0/1884878713 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f251415c630 0x7f25140c9660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T15:08:38.897 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:38.895+0000 7f2518c26640 1 --2- 172.21.15.43:0/1884878713 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f25140cda90 0x7f25140c9120 unknown :-1 s=CLOSED pgs=2177 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T15:08:38.898 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:38.895+0000 7f2518c26640 1 --2- 172.21.15.43:0/1884878713 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f25140ccf20 0x7f25140c5b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T15:08:38.898 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:38.895+0000 7f2518c26640 1 -- 172.21.15.43:0/1884878713 >> 172.21.15.43:0/1884878713 conn(0x7f25140ce0a0 msgr2=0x7f25140ac740 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-09T15:08:38.898 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:38.895+0000 7f2518c26640 1 -- 172.21.15.43:0/1884878713 shutdown_connections 2023-08-09T15:08:38.898 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:38.895+0000 7f2518c26640 1 -- 172.21.15.43:0/1884878713 wait complete. 2023-08-09T15:08:38.917 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2023-08-09T15:08:38.917 DEBUG:teuthology.orchestra.run.smithi043:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2023-08-09T15:08:39.041 DEBUG:teuthology.orchestra.run.smithi203:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2023-08-09T15:08:39.159 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2023-08-09T15:08:39.160 DEBUG:teuthology.orchestra.run.smithi043:> set -ex 2023-08-09T15:08:39.160 DEBUG:teuthology.orchestra.run.smithi043:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-09T15:08:39.226 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4054441,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6498844k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=27,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3249420k,nr_inodes=812355,mode=700,uid=1000,gid=1275 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6498844k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2023-08-09T15:08:39.226 DEBUG:teuthology.orchestra.run.smithi203:> set -ex 2023-08-09T15:08:39.227 DEBUG:teuthology.orchestra.run.smithi203:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-09T15:08:39.287 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4053648,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497576k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=27,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248784k,nr_inodes=812196,mode=700,uid=1000,gid=1275 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497576k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2023-08-09T15:08:39.287 DEBUG:teuthology.orchestra.run.smithi043:> set -ex 2023-08-09T15:08:39.288 DEBUG:teuthology.orchestra.run.smithi043:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-09T15:08:39.353 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4054441,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6498844k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=27,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3249420k,nr_inodes=812355,mode=700,uid=1000,gid=1275 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6498844k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2023-08-09T15:08:39.353 DEBUG:teuthology.orchestra.run.smithi203:> set -ex 2023-08-09T15:08:39.353 DEBUG:teuthology.orchestra.run.smithi203:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-09T15:08:39.414 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4053648,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497576k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=27,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248784k,nr_inodes=812196,mode=700,uid=1000,gid=1275 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497576k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2023-08-09T15:08:39.414 INFO:tasks.cephfs.cephfs_test_case:[, , , ] 2023-08-09T15:08:39.414 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-08-09T15:08:39.652 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:39.651+0000 7ff05cc4c640 1 Processor -- start 2023-08-09T15:08:39.653 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:39.651+0000 7ff05cc4c640 1 -- start start 2023-08-09T15:08:39.654 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:39.653+0000 7ff05cc4c640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7ff058143c20 0x7ff05806f540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T15:08:39.654 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:39.653+0000 7ff05cc4c640 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7ff0580c86f0 0x7ff0580c8b50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T15:08:39.654 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:39.653+0000 7ff05cc4c640 1 -- --> v1:172.21.15.203:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff0580c9090 con 0x7ff05806c8d0 2023-08-09T15:08:39.655 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:39.653+0000 7ff05cc4c640 1 -- --> v2:172.21.15.203:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff058144000 con 0x7ff0580c86f0 2023-08-09T15:08:39.655 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:39.653+0000 7ff05cc4c640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff05806c440 con 0x7ff058143c20 2023-08-09T15:08:39.655 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:39.654+0000 7ff05659c640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7ff058143c20 0x7ff05806f540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T15:08:39.655 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:39.654+0000 7ff056d9d640 1 --1- >> v1:172.21.15.203:6789/0 conn(0x7ff05806c8d0 0x7ff05806cd60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.203:6789/0 says I am v1:172.21.15.43:49930/0 (socket says 172.21.15.43:49930) 2023-08-09T15:08:39.655 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:39.654+0000 7ff04dd9b640 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7ff0580c86f0 0x7ff0580c8b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T15:08:39.656 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:39.654+0000 7ff056d9d640 1 -- 172.21.15.43:0/3591184952 learned_addr learned my addr 172.21.15.43:0/3591184952 (peer_addr_for_me v1:172.21.15.43:0/0) 2023-08-09T15:08:39.656 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:39.655+0000 7ff055d9b640 1 -- 172.21.15.43:0/3591184952 <== mon.1 v1:172.21.15.203:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1178481342 0 0) 0x7ff0580c9090 con 0x7ff05806c8d0 2023-08-09T15:08:39.657 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:39.655+0000 7ff055d9b640 1 -- 172.21.15.43:0/3591184952 --> v1:172.21.15.203:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ff044003520 con 0x7ff05806c8d0 2023-08-09T15:08:39.657 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:39.655+0000 7ff05659c640 1 -- 172.21.15.43:0/3591184952 >> v2:172.21.15.203:3300/0 conn(0x7ff0580c86f0 msgr2=0x7ff0580c8b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-09T15:08:39.657 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:39.655+0000 7ff05659c640 1 --2- 172.21.15.43:0/3591184952 >> v2:172.21.15.203:3300/0 conn(0x7ff0580c86f0 0x7ff0580c8b50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T15:08:39.657 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:39.655+0000 7ff05659c640 1 -- 172.21.15.43:0/3591184952 >> v1:172.21.15.203:6789/0 conn(0x7ff05806c8d0 legacy=0x7ff05806cd60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T15:08:39.657 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:39.655+0000 7ff05659c640 1 -- 172.21.15.43:0/3591184952 --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff05806b830 con 0x7ff058143c20 2023-08-09T15:08:39.657 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:39.655+0000 7ff04dd9b640 1 --2- 172.21.15.43:0/3591184952 >> v2:172.21.15.203:3300/0 conn(0x7ff0580c86f0 0x7ff0580c8b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-08-09T15:08:39.658 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:39.657+0000 7ff05659c640 1 --2- 172.21.15.43:0/3591184952 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7ff058143c20 0x7ff05806f540 secure :-1 s=READY pgs=312 cs=0 l=1 rev1=1 crypto rx=0x7ff048009d90 tx=0x7ff04802be10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=744e508e31ddf43d server_cookie=0 in_seq=0 out_seq=0 2023-08-09T15:08:39.658 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:39.657+0000 7ff055d9b640 1 -- 172.21.15.43:0/3591184952 <== mon.2 v2:172.21.15.203:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff04802cbc0 con 0x7ff058143c20 2023-08-09T15:08:39.658 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:39.657+0000 7ff055d9b640 1 -- 172.21.15.43:0/3591184952 <== mon.2 v2:172.21.15.203:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff04802cd20 con 0x7ff058143c20 2023-08-09T15:08:39.658 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:39.657+0000 7ff055d9b640 1 -- 172.21.15.43:0/3591184952 <== mon.2 v2:172.21.15.203:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff048034660 con 0x7ff058143c20 2023-08-09T15:08:39.659 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:39.658+0000 7ff05cc4c640 1 -- 172.21.15.43:0/3591184952 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7ff058143c20 msgr2=0x7ff05806f540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T15:08:39.659 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:39.658+0000 7ff05cc4c640 1 --2- 172.21.15.43:0/3591184952 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7ff058143c20 0x7ff05806f540 secure :-1 s=READY pgs=312 cs=0 l=1 rev1=1 crypto rx=0x7ff048009d90 tx=0x7ff04802be10 comp rx=0 tx=0).stop 2023-08-09T15:08:39.659 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:39.658+0000 7ff05cc4c640 1 -- 172.21.15.43:0/3591184952 shutdown_connections 2023-08-09T15:08:39.660 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:39.658+0000 7ff05cc4c640 1 --2- 172.21.15.43:0/3591184952 >> v2:172.21.15.203:3300/0 conn(0x7ff0580c86f0 0x7ff0580c8b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T15:08:39.660 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:39.658+0000 7ff05cc4c640 1 --2- 172.21.15.43:0/3591184952 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7ff058143c20 0x7ff05806f540 unknown :-1 s=CLOSED pgs=312 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T15:08:39.660 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:39.658+0000 7ff05cc4c640 1 -- 172.21.15.43:0/3591184952 >> 172.21.15.43:0/3591184952 conn(0x7ff0580ce0a0 msgr2=0x7ff0580ce480 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-09T15:08:39.660 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:39.658+0000 7ff05cc4c640 1 -- 172.21.15.43:0/3591184952 shutdown_connections 2023-08-09T15:08:39.660 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:39.658+0000 7ff05cc4c640 1 -- 172.21.15.43:0/3591184952 wait complete. 2023-08-09T15:08:39.661 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:39.660+0000 7ff05cc4c640 1 Processor -- start 2023-08-09T15:08:39.661 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:39.660+0000 7ff05cc4c640 1 -- start start 2023-08-09T15:08:39.662 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:39.661+0000 7ff05cc4c640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff05806c8d0 0x7ff058070a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T15:08:39.663 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:39.661+0000 7ff05cc4c640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7ff0580c86f0 0x7ff058142030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T15:08:39.663 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:39.661+0000 7ff05cc4c640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7ff058143c20 0x7ff0580b0d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T15:08:39.663 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:39.661+0000 7ff05cc4c640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff0581f1ff0 con 0x7ff05806c8d0 2023-08-09T15:08:39.663 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:39.661+0000 7ff05cc4c640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff0580b3d40 con 0x7ff058143c20 2023-08-09T15:08:39.663 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:39.661+0000 7ff05cc4c640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff058070fa0 con 0x7ff0580c86f0 2023-08-09T15:08:39.663 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:39.662+0000 7ff05659c640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff05806c8d0 0x7ff058070a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T15:08:39.663 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:39.662+0000 7ff05659c640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff05806c8d0 0x7ff058070a60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:39348/0 (socket says 172.21.15.43:39348) 2023-08-09T15:08:39.664 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:39.662+0000 7ff05659c640 1 -- 172.21.15.43:0/1361288428 learned_addr learned my addr 172.21.15.43:0/1361288428 (peer_addr_for_me v2:172.21.15.43:0/0) 2023-08-09T15:08:39.665 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:39.662+0000 7ff056d9d640 1 --2- 172.21.15.43:0/1361288428 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7ff058143c20 0x7ff0580b0d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T15:08:39.665 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:39.662+0000 7ff04dd9b640 1 --2- 172.21.15.43:0/1361288428 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7ff0580c86f0 0x7ff058142030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T15:08:39.665 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:39.662+0000 7ff05659c640 1 -- 172.21.15.43:0/1361288428 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7ff0580c86f0 msgr2=0x7ff058142030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T15:08:39.665 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:39.662+0000 7ff05659c640 1 --2- 172.21.15.43:0/1361288428 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7ff0580c86f0 0x7ff058142030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T15:08:39.665 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:39.662+0000 7ff05659c640 1 -- 172.21.15.43:0/1361288428 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7ff058143c20 msgr2=0x7ff0580b0d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T15:08:39.665 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:39.662+0000 7ff05659c640 1 --2- 172.21.15.43:0/1361288428 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7ff058143c20 0x7ff0580b0d80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T15:08:39.665 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:39.662+0000 7ff05659c640 1 -- 172.21.15.43:0/1361288428 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff040009400 con 0x7ff05806c8d0 2023-08-09T15:08:39.666 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:39.662+0000 7ff05659c640 1 --2- 172.21.15.43:0/1361288428 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff05806c8d0 0x7ff058070a60 secure :-1 s=READY pgs=2178 cs=0 l=1 rev1=1 crypto rx=0x7ff048009d90 tx=0x7ff048004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-09T15:08:39.666 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:39.663+0000 7ff04f7fe640 1 -- 172.21.15.43:0/1361288428 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff04803e610 con 0x7ff05806c8d0 2023-08-09T15:08:39.666 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:39.663+0000 7ff04f7fe640 1 -- 172.21.15.43:0/1361288428 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff04802ce90 con 0x7ff05806c8d0 2023-08-09T15:08:39.666 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:39.663+0000 7ff05cc4c640 1 -- 172.21.15.43:0/1361288428 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff048009a70 con 0x7ff05806c8d0 2023-08-09T15:08:39.666 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:39.663+0000 7ff04f7fe640 1 -- 172.21.15.43:0/1361288428 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff04803e9d0 con 0x7ff05806c8d0 2023-08-09T15:08:39.666 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:39.663+0000 7ff05cc4c640 1 -- 172.21.15.43:0/1361288428 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff0580b3410 con 0x7ff05806c8d0 2023-08-09T15:08:39.667 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:39.664+0000 7ff05cc4c640 1 -- 172.21.15.43:0/1361288428 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff0580739f0 con 0x7ff05806c8d0 2023-08-09T15:08:39.667 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:39.665+0000 7ff04f7fe640 1 -- 172.21.15.43:0/1361288428 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 106) v1 ==== 88443+0+0 (secure 0 0 0) 0x7ff04803b050 con 0x7ff05806c8d0 2023-08-09T15:08:39.667 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:39.666+0000 7ff04f7fe640 1 --2- 172.21.15.43:0/1361288428 >> [v2:172.21.15.203:6832/3082732769,v1:172.21.15.203:6833/3082732769] conn(0x7ff03806a170 0x7ff03806c630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T15:08:39.668 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:39.666+0000 7ff04f7fe640 1 -- 172.21.15.43:0/1361288428 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(197..197 src has 1..197) v4 ==== 6692+0+0 (secure 0 0 0) 0x7ff0480ab090 con 0x7ff05806c8d0 2023-08-09T15:08:39.668 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:39.667+0000 7ff04dd9b640 1 --2- 172.21.15.43:0/1361288428 >> [v2:172.21.15.203:6832/3082732769,v1:172.21.15.203:6833/3082732769] conn(0x7ff03806a170 0x7ff03806c630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T15:08:39.669 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:39.668+0000 7ff04dd9b640 1 --2- 172.21.15.43:0/1361288428 >> [v2:172.21.15.203:6832/3082732769,v1:172.21.15.203:6833/3082732769] conn(0x7ff03806a170 0x7ff03806c630 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7ff0580ad3c0 tx=0x7ff040008040 comp rx=0 tx=0).ready entity=mgr.8500 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-09T15:08:39.672 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:39.671+0000 7ff04f7fe640 1 -- 172.21.15.43:0/1361288428 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178066 (secure 0 0 0) 0x7ff0480763e0 con 0x7ff05806c8d0 2023-08-09T15:08:39.905 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:39.903+0000 7ff05cc4c640 1 -- 172.21.15.43:0/1361288428 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7ff0580cec90 con 0x7ff05806c8d0 2023-08-09T15:08:39.906 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:39.905+0000 7ff04f7fe640 1 -- 172.21.15.43:0/1361288428 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 89 v89) v1 ==== 94+0+3700 (secure 0 0 0) 0x7ff04804e090 con 0x7ff05806c8d0 2023-08-09T15:08:39.907 INFO:teuthology.orchestra.run.smithi043.stdout: 2023-08-09T15:08:39.907 INFO:teuthology.orchestra.run.smithi043.stdout:{"epoch":89,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":7328,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.203:6836/3701396136","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6834","nonce":3701396136},{"type":"v1","addr":"172.21.15.203:6836","nonce":3701396136}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":82},{"gid":7343,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.43:6835/2096611841","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6833","nonce":2096611841},{"type":"v1","addr":"172.21.15.43:6835","nonce":2096611841}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":82},{"gid":8242,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.203:6837/18754135","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6835","nonce":18754135},{"type":"v1","addr":"172.21.15.203:6837","nonce":18754135}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":81},{"gid":10656,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.43:6834/2183511990","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6832","nonce":2183511990},{"type":"v1","addr":"172.21.15.43:6834","nonce":2183511990}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":89}],"filesystems":[]} 2023-08-09T15:08:39.907 INFO:teuthology.orchestra.run.smithi043.stderr:dumped fsmap epoch 89 2023-08-09T15:08:39.909 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:39.908+0000 7ff05cc4c640 1 -- 172.21.15.43:0/1361288428 >> [v2:172.21.15.203:6832/3082732769,v1:172.21.15.203:6833/3082732769] conn(0x7ff03806a170 msgr2=0x7ff03806c630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T15:08:39.909 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:39.908+0000 7ff05cc4c640 1 --2- 172.21.15.43:0/1361288428 >> [v2:172.21.15.203:6832/3082732769,v1:172.21.15.203:6833/3082732769] conn(0x7ff03806a170 0x7ff03806c630 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7ff0580ad3c0 tx=0x7ff040008040 comp rx=0 tx=0).stop 2023-08-09T15:08:39.909 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:39.908+0000 7ff05cc4c640 1 -- 172.21.15.43:0/1361288428 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff05806c8d0 msgr2=0x7ff058070a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T15:08:39.910 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:39.908+0000 7ff05cc4c640 1 --2- 172.21.15.43:0/1361288428 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff05806c8d0 0x7ff058070a60 secure :-1 s=READY pgs=2178 cs=0 l=1 rev1=1 crypto rx=0x7ff048009d90 tx=0x7ff048004a40 comp rx=0 tx=0).stop 2023-08-09T15:08:39.910 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:39.908+0000 7ff05cc4c640 1 -- 172.21.15.43:0/1361288428 shutdown_connections 2023-08-09T15:08:39.910 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:39.908+0000 7ff05cc4c640 1 --2- 172.21.15.43:0/1361288428 >> [v2:172.21.15.203:6832/3082732769,v1:172.21.15.203:6833/3082732769] conn(0x7ff03806a170 0x7ff03806c630 unknown :-1 s=CLOSED pgs=46 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T15:08:39.910 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:39.908+0000 7ff05cc4c640 1 --2- 172.21.15.43:0/1361288428 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7ff058143c20 0x7ff0580b0d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T15:08:39.910 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:39.908+0000 7ff05cc4c640 1 --2- 172.21.15.43:0/1361288428 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7ff0580c86f0 0x7ff058142030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T15:08:39.910 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:39.908+0000 7ff05cc4c640 1 --2- 172.21.15.43:0/1361288428 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff05806c8d0 0x7ff058070a60 unknown :-1 s=CLOSED pgs=2178 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T15:08:39.910 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:39.908+0000 7ff05cc4c640 1 -- 172.21.15.43:0/1361288428 >> 172.21.15.43:0/1361288428 conn(0x7ff0580ce0a0 msgr2=0x7ff0580c8ef0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-09T15:08:39.910 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:39.908+0000 7ff05cc4c640 1 -- 172.21.15.43:0/1361288428 shutdown_connections 2023-08-09T15:08:39.911 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:39.908+0000 7ff05cc4c640 1 -- 172.21.15.43:0/1361288428 wait complete. 2023-08-09T15:08:39.930 INFO:tasks.ceph.mds.a:Restarting daemon 2023-08-09T15:08:39.930 INFO:tasks.ceph.mds.a:Stopping old one... 2023-08-09T15:08:39.931 DEBUG:tasks.ceph.mds.a:waiting for process to exit 2023-08-09T15:08:39.931 INFO:teuthology.orchestra.run:waiting for 300 2023-08-09T15:08:39.931 INFO:tasks.ceph.mds.c:Restarting daemon 2023-08-09T15:08:39.931 INFO:tasks.ceph.mds.c:Stopping old one... 2023-08-09T15:08:39.932 DEBUG:tasks.ceph.mds.c:waiting for process to exit 2023-08-09T15:08:39.932 INFO:teuthology.orchestra.run:waiting for 300 2023-08-09T15:08:39.932 INFO:tasks.ceph.mds.b:Restarting daemon 2023-08-09T15:08:39.932 INFO:tasks.ceph.mds.b:Stopping old one... 2023-08-09T15:08:39.932 DEBUG:tasks.ceph.mds.b:waiting for process to exit 2023-08-09T15:08:39.933 INFO:teuthology.orchestra.run:waiting for 300 2023-08-09T15:08:39.933 INFO:tasks.ceph.mds.d:Restarting daemon 2023-08-09T15:08:39.933 INFO:tasks.ceph.mds.d:Stopping old one... 2023-08-09T15:08:39.933 DEBUG:tasks.ceph.mds.d:waiting for process to exit 2023-08-09T15:08:39.933 INFO:teuthology.orchestra.run:waiting for 300 2023-08-09T15:08:39.951 INFO:tasks.ceph.mds.a:Stopped 2023-08-09T15:08:39.951 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i a 2023-08-09T15:08:39.953 INFO:tasks.ceph.mds.c:Stopped 2023-08-09T15:08:39.953 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i c 2023-08-09T15:08:39.955 INFO:tasks.ceph.mds.d:Stopped 2023-08-09T15:08:39.955 DEBUG:teuthology.orchestra.run.smithi203:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i d 2023-08-09T15:08:39.970 INFO:tasks.ceph.mds.b:Stopped 2023-08-09T15:08:39.970 DEBUG:teuthology.orchestra.run.smithi203:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i b 2023-08-09T15:08:40.001 INFO:tasks.ceph.mds.a:Started 2023-08-09T15:08:40.002 DEBUG:teuthology.parallel:result is None 2023-08-09T15:08:40.010 INFO:tasks.ceph.mds.c:Started 2023-08-09T15:08:40.010 DEBUG:teuthology.parallel:result is None 2023-08-09T15:08:40.016 INFO:tasks.ceph.mds.d:Started 2023-08-09T15:08:40.016 DEBUG:teuthology.parallel:result is None 2023-08-09T15:08:40.024 INFO:tasks.ceph.mds.b:Started 2023-08-09T15:08:40.025 DEBUG:teuthology.parallel:result is None 2023-08-09T15:08:40.025 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist clear 2023-08-09T15:08:40.172 INFO:tasks.ceph.mds.c.smithi043.stdout:starting mds.c at 2023-08-09T15:08:40.172 INFO:tasks.ceph.mds.a.smithi043.stdout:starting mds.a at 2023-08-09T15:08:40.187 INFO:tasks.ceph.mds.d.smithi203.stdout:starting mds.d at 2023-08-09T15:08:40.188 INFO:tasks.ceph.mds.b.smithi203.stdout:starting mds.b at 2023-08-09T15:08:40.221 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:40.220+0000 7fcd2515b640 1 Processor -- start 2023-08-09T15:08:40.221 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:40.220+0000 7fcd2515b640 1 -- start start 2023-08-09T15:08:40.223 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:40.222+0000 7fcd2515b640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fcd20143c20 0x7fcd2006f540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T15:08:40.223 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:40.222+0000 7fcd2515b640 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7fcd200c86f0 0x7fcd200c8b50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T15:08:40.224 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:40.222+0000 7fcd2515b640 1 -- --> v1:172.21.15.43:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fcd200c9090 con 0x7fcd2006c8d0 2023-08-09T15:08:40.224 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:40.222+0000 7fcd2515b640 1 -- --> v2:172.21.15.203:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fcd20144000 con 0x7fcd200c86f0 2023-08-09T15:08:40.224 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:40.222+0000 7fcd2515b640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcd2006c440 con 0x7fcd20143c20 2023-08-09T15:08:40.224 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:40.222+0000 7fcd1f59e640 1 --1- >> v1:172.21.15.43:6789/0 conn(0x7fcd2006c8d0 0x7fcd2006cd60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.43:6789/0 says I am v1:172.21.15.43:59604/0 (socket says 172.21.15.43:59604) 2023-08-09T15:08:40.224 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:40.222+0000 7fcd1f59e640 1 -- 172.21.15.43:0/3406681932 learned_addr learned my addr 172.21.15.43:0/3406681932 (peer_addr_for_me v1:172.21.15.43:0/0) 2023-08-09T15:08:40.225 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:40.222+0000 7fcd1ed9d640 1 --2- 172.21.15.43:0/3406681932 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fcd20143c20 0x7fcd2006f540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T15:08:40.227 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:40.222+0000 7fcd17fff640 1 --2- 172.21.15.43:0/3406681932 >> v2:172.21.15.203:3300/0 conn(0x7fcd200c86f0 0x7fcd200c8b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T15:08:40.227 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:40.222+0000 7fcd1e59c640 1 -- 172.21.15.43:0/3406681932 <== mon.0 v1:172.21.15.43:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 548589198 0 0) 0x7fcd200c9090 con 0x7fcd2006c8d0 2023-08-09T15:08:40.228 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:40.222+0000 7fcd1e59c640 1 -- 172.21.15.43:0/3406681932 --> v1:172.21.15.43:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fcd0c003520 con 0x7fcd2006c8d0 2023-08-09T15:08:40.228 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:40.222+0000 7fcd1ed9d640 1 -- 172.21.15.43:0/3406681932 >> v2:172.21.15.203:3300/0 conn(0x7fcd200c86f0 msgr2=0x7fcd200c8b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-09T15:08:40.228 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:40.222+0000 7fcd1ed9d640 1 --2- 172.21.15.43:0/3406681932 >> v2:172.21.15.203:3300/0 conn(0x7fcd200c86f0 0x7fcd200c8b50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T15:08:40.229 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:40.222+0000 7fcd1ed9d640 1 -- 172.21.15.43:0/3406681932 >> v1:172.21.15.43:6789/0 conn(0x7fcd2006c8d0 legacy=0x7fcd2006cd60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T15:08:40.229 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:40.222+0000 7fcd1ed9d640 1 -- 172.21.15.43:0/3406681932 --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcd2006b830 con 0x7fcd20143c20 2023-08-09T15:08:40.229 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:40.224+0000 7fcd1ed9d640 1 --2- 172.21.15.43:0/3406681932 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fcd20143c20 0x7fcd2006f540 secure :-1 s=READY pgs=314 cs=0 l=1 rev1=1 crypto rx=0x7fcd100097a0 tx=0x7fcd1002b820 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c6dd9fce64da39ab server_cookie=0 in_seq=0 out_seq=0 2023-08-09T15:08:40.230 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:40.224+0000 7fcd1e59c640 1 -- 172.21.15.43:0/3406681932 <== mon.2 v2:172.21.15.203:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcd10031780 con 0x7fcd20143c20 2023-08-09T15:08:40.230 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:40.224+0000 7fcd1e59c640 1 -- 172.21.15.43:0/3406681932 <== mon.2 v2:172.21.15.203:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcd100318e0 con 0x7fcd20143c20 2023-08-09T15:08:40.231 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:40.224+0000 7fcd1e59c640 1 -- 172.21.15.43:0/3406681932 <== mon.2 v2:172.21.15.203:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcd10031c70 con 0x7fcd20143c20 2023-08-09T15:08:40.231 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:40.225+0000 7fcd2515b640 1 -- 172.21.15.43:0/3406681932 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fcd20143c20 msgr2=0x7fcd2006f540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T15:08:40.231 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:40.225+0000 7fcd2515b640 1 --2- 172.21.15.43:0/3406681932 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fcd20143c20 0x7fcd2006f540 secure :-1 s=READY pgs=314 cs=0 l=1 rev1=1 crypto rx=0x7fcd100097a0 tx=0x7fcd1002b820 comp rx=0 tx=0).stop 2023-08-09T15:08:40.231 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:40.225+0000 7fcd2515b640 1 -- 172.21.15.43:0/3406681932 shutdown_connections 2023-08-09T15:08:40.232 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:40.225+0000 7fcd2515b640 1 --2- 172.21.15.43:0/3406681932 >> v2:172.21.15.203:3300/0 conn(0x7fcd200c86f0 0x7fcd200c8b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T15:08:40.232 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:40.225+0000 7fcd2515b640 1 --2- 172.21.15.43:0/3406681932 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fcd20143c20 0x7fcd2006f540 unknown :-1 s=CLOSED pgs=314 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T15:08:40.232 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:40.225+0000 7fcd2515b640 1 -- 172.21.15.43:0/3406681932 >> 172.21.15.43:0/3406681932 conn(0x7fcd200ce0a0 msgr2=0x7fcd200ce480 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-09T15:08:40.232 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:40.225+0000 7fcd2515b640 1 -- 172.21.15.43:0/3406681932 shutdown_connections 2023-08-09T15:08:40.232 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:40.225+0000 7fcd2515b640 1 -- 172.21.15.43:0/3406681932 wait complete. 2023-08-09T15:08:40.236 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:40.227+0000 7fcd2515b640 1 Processor -- start 2023-08-09T15:08:40.236 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:40.227+0000 7fcd2515b640 1 -- start start 2023-08-09T15:08:40.237 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:40.228+0000 7fcd2515b640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fcd2006c8d0 0x7fcd2015bec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T15:08:40.237 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:40.228+0000 7fcd2515b640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fcd200c86f0 0x7fcd2015cc90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T15:08:40.237 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:40.228+0000 7fcd2515b640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fcd20143c20 0x7fcd2012b720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T15:08:40.237 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:40.228+0000 7fcd2515b640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcd2013f0d0 con 0x7fcd2006c8d0 2023-08-09T15:08:40.238 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:40.228+0000 7fcd2515b640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcd2015c4d0 con 0x7fcd20143c20 2023-08-09T15:08:40.238 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:40.228+0000 7fcd2515b640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcd200c5210 con 0x7fcd200c86f0 2023-08-09T15:08:40.238 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:40.228+0000 7fcd1ed9d640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fcd2006c8d0 0x7fcd2015bec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T15:08:40.238 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:40.228+0000 7fcd1ed9d640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fcd2006c8d0 0x7fcd2015bec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:39388/0 (socket says 172.21.15.43:39388) 2023-08-09T15:08:40.239 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:40.228+0000 7fcd1ed9d640 1 -- 172.21.15.43:0/3865504278 learned_addr learned my addr 172.21.15.43:0/3865504278 (peer_addr_for_me v2:172.21.15.43:0/0) 2023-08-09T15:08:40.239 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:40.228+0000 7fcd1f59e640 1 --2- 172.21.15.43:0/3865504278 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fcd20143c20 0x7fcd2012b720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T15:08:40.239 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:40.228+0000 7fcd17fff640 1 --2- 172.21.15.43:0/3865504278 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fcd200c86f0 0x7fcd2015cc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T15:08:40.239 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:40.229+0000 7fcd1ed9d640 1 -- 172.21.15.43:0/3865504278 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fcd200c86f0 msgr2=0x7fcd2015cc90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T15:08:40.240 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:40.229+0000 7fcd1ed9d640 1 --2- 172.21.15.43:0/3865504278 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fcd200c86f0 0x7fcd2015cc90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T15:08:40.240 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:40.229+0000 7fcd1ed9d640 1 -- 172.21.15.43:0/3865504278 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fcd20143c20 msgr2=0x7fcd2012b720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T15:08:40.240 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:40.229+0000 7fcd1ed9d640 1 --2- 172.21.15.43:0/3865504278 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fcd20143c20 0x7fcd2012b720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T15:08:40.241 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:40.229+0000 7fcd1ed9d640 1 -- 172.21.15.43:0/3865504278 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcd10009400 con 0x7fcd2006c8d0 2023-08-09T15:08:40.241 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:40.229+0000 7fcd1ed9d640 1 --2- 172.21.15.43:0/3865504278 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fcd2006c8d0 0x7fcd2015bec0 secure :-1 s=READY pgs=2189 cs=0 l=1 rev1=1 crypto rx=0x7fcd10034040 tx=0x7fcd100046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-09T15:08:40.241 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:40.229+0000 7fcd177fe640 1 -- 172.21.15.43:0/3865504278 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcd10032820 con 0x7fcd2006c8d0 2023-08-09T15:08:40.241 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:40.229+0000 7fcd177fe640 1 -- 172.21.15.43:0/3865504278 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcd10032980 con 0x7fcd2006c8d0 2023-08-09T15:08:40.242 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:40.229+0000 7fcd177fe640 1 -- 172.21.15.43:0/3865504278 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcd10032d30 con 0x7fcd2006c8d0 2023-08-09T15:08:40.242 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:40.229+0000 7fcd2515b640 1 -- 172.21.15.43:0/3865504278 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcd200c54f0 con 0x7fcd2006c8d0 2023-08-09T15:08:40.242 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:40.229+0000 7fcd2515b640 1 -- 172.21.15.43:0/3865504278 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcd200c57d0 con 0x7fcd2006c8d0 2023-08-09T15:08:40.242 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:40.231+0000 7fcd2515b640 1 -- 172.21.15.43:0/3865504278 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcd200739f0 con 0x7fcd2006c8d0 2023-08-09T15:08:40.243 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:40.233+0000 7fcd177fe640 1 -- 172.21.15.43:0/3865504278 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 106) v1 ==== 88443+0+0 (secure 0 0 0) 0x7fcd1004e440 con 0x7fcd2006c8d0 2023-08-09T15:08:40.243 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:40.233+0000 7fcd177fe640 1 --2- 172.21.15.43:0/3865504278 >> [v2:172.21.15.203:6832/3082732769,v1:172.21.15.203:6833/3082732769] conn(0x7fcd0006a170 0x7fcd0006c630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T15:08:40.243 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:40.233+0000 7fcd177fe640 1 -- 172.21.15.43:0/3865504278 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(197..197 src has 1..197) v4 ==== 6692+0+0 (secure 0 0 0) 0x7fcd100aaaf0 con 0x7fcd2006c8d0 2023-08-09T15:08:40.243 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:40.233+0000 7fcd17fff640 1 --2- 172.21.15.43:0/3865504278 >> [v2:172.21.15.203:6832/3082732769,v1:172.21.15.203:6833/3082732769] conn(0x7fcd0006a170 0x7fcd0006c630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T15:08:40.243 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:40.234+0000 7fcd17fff640 1 --2- 172.21.15.43:0/3865504278 >> [v2:172.21.15.203:6832/3082732769,v1:172.21.15.203:6833/3082732769] conn(0x7fcd0006a170 0x7fcd0006c630 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7fcd2015c400 tx=0x7fcd08008040 comp rx=0 tx=0).ready entity=mgr.8500 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-09T15:08:40.244 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:40.237+0000 7fcd177fe640 1 -- 172.21.15.43:0/3865504278 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178066 (secure 0 0 0) 0x7fcd10075ca0 con 0x7fcd2006c8d0 2023-08-09T15:08:40.447 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:40.445+0000 7fcd2515b640 1 -- 172.21.15.43:0/3865504278 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "osd blocklist clear"} v 0) v1 -- 0x7fcd200cec90 con 0x7fcd2006c8d0 2023-08-09T15:08:41.058 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:41.056+0000 7fcd177fe640 1 -- 172.21.15.43:0/3865504278 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist clear"}]=0 removed all blocklist entries v199) v1 ==== 97+0+0 (secure 0 0 0) 0x7fcd100792c0 con 0x7fcd2006c8d0 2023-08-09T15:08:41.058 INFO:teuthology.orchestra.run.smithi043.stderr: removed all blocklist entries 2023-08-09T15:08:41.061 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:41.060+0000 7fcd2515b640 1 -- 172.21.15.43:0/3865504278 >> [v2:172.21.15.203:6832/3082732769,v1:172.21.15.203:6833/3082732769] conn(0x7fcd0006a170 msgr2=0x7fcd0006c630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T15:08:41.062 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:41.060+0000 7fcd2515b640 1 --2- 172.21.15.43:0/3865504278 >> [v2:172.21.15.203:6832/3082732769,v1:172.21.15.203:6833/3082732769] conn(0x7fcd0006a170 0x7fcd0006c630 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7fcd2015c400 tx=0x7fcd08008040 comp rx=0 tx=0).stop 2023-08-09T15:08:41.062 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:41.060+0000 7fcd2515b640 1 -- 172.21.15.43:0/3865504278 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fcd2006c8d0 msgr2=0x7fcd2015bec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T15:08:41.062 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:41.060+0000 7fcd2515b640 1 --2- 172.21.15.43:0/3865504278 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fcd2006c8d0 0x7fcd2015bec0 secure :-1 s=READY pgs=2189 cs=0 l=1 rev1=1 crypto rx=0x7fcd10034040 tx=0x7fcd100046c0 comp rx=0 tx=0).stop 2023-08-09T15:08:41.062 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:41.060+0000 7fcd2515b640 1 -- 172.21.15.43:0/3865504278 shutdown_connections 2023-08-09T15:08:41.062 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:41.060+0000 7fcd2515b640 1 --2- 172.21.15.43:0/3865504278 >> [v2:172.21.15.203:6832/3082732769,v1:172.21.15.203:6833/3082732769] conn(0x7fcd0006a170 0x7fcd0006c630 unknown :-1 s=CLOSED pgs=47 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T15:08:41.062 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:41.060+0000 7fcd2515b640 1 --2- 172.21.15.43:0/3865504278 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fcd20143c20 0x7fcd2012b720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T15:08:41.063 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:41.060+0000 7fcd2515b640 1 --2- 172.21.15.43:0/3865504278 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fcd200c86f0 0x7fcd2015cc90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T15:08:41.063 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:41.060+0000 7fcd2515b640 1 --2- 172.21.15.43:0/3865504278 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fcd2006c8d0 0x7fcd2015bec0 unknown :-1 s=CLOSED pgs=2189 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T15:08:41.063 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:41.060+0000 7fcd2515b640 1 -- 172.21.15.43:0/3865504278 >> 172.21.15.43:0/3865504278 conn(0x7fcd200ce0a0 msgr2=0x7fcd200cab00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-09T15:08:41.063 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:41.061+0000 7fcd2515b640 1 -- 172.21.15.43:0/3865504278 shutdown_connections 2023-08-09T15:08:41.064 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:41.061+0000 7fcd2515b640 1 -- 172.21.15.43:0/3865504278 wait complete. 2023-08-09T15:08:41.084 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth ls --format=json-pretty 2023-08-09T15:08:41.309 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:41.309+0000 7f6dde52b640 1 Processor -- start 2023-08-09T15:08:41.310 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:41.309+0000 7f6dde52b640 1 -- start start 2023-08-09T15:08:41.311 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:41.311+0000 7f6dde52b640 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7f6dd8143c20 0x7f6dd806f540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T15:08:41.311 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:41.311+0000 7f6dde52b640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f6dd80c86f0 0x7f6dd80c8b50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T15:08:41.311 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:41.311+0000 7f6dde52b640 1 -- --> v1:172.21.15.43:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6dd80c9090 con 0x7f6dd806c8d0 2023-08-09T15:08:41.311 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:41.311+0000 7f6dde52b640 1 -- --> v2:172.21.15.43:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6dd8144000 con 0x7f6dd80c86f0 2023-08-09T15:08:41.312 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:41.311+0000 7f6dde52b640 1 -- --> v2:172.21.15.203:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6dd806c440 con 0x7f6dd8143c20 2023-08-09T15:08:41.313 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:41.311+0000 7f6ddcac8640 1 --1- >> v1:172.21.15.43:6789/0 conn(0x7f6dd806c8d0 0x7f6dd806cd00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.43:6789/0 says I am v1:172.21.15.43:59610/0 (socket says 172.21.15.43:59610) 2023-08-09T15:08:41.314 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:41.311+0000 7f6ddcac8640 1 -- 172.21.15.43:0/2011706241 learned_addr learned my addr 172.21.15.43:0/2011706241 (peer_addr_for_me v1:172.21.15.43:0/0) 2023-08-09T15:08:41.314 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:41.311+0000 7f6dcf7fe640 1 --2- 172.21.15.43:0/2011706241 >> v2:172.21.15.43:3300/0 conn(0x7f6dd80c86f0 0x7f6dd80c8b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T15:08:41.314 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:41.312+0000 7f6dd7fff640 1 --2- 172.21.15.43:0/2011706241 >> v2:172.21.15.203:3300/0 conn(0x7f6dd8143c20 0x7f6dd806f540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T15:08:41.314 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:41.312+0000 7f6dd77fe640 1 -- 172.21.15.43:0/2011706241 <== mon.0 v1:172.21.15.43:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2545184542 0 0) 0x7f6dd80c9090 con 0x7f6dd806c8d0 2023-08-09T15:08:41.314 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:41.312+0000 7f6dd77fe640 1 -- 172.21.15.43:0/2011706241 --> v1:172.21.15.43:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f6dc4003520 con 0x7f6dd806c8d0 2023-08-09T15:08:41.314 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:41.312+0000 7f6dcf7fe640 1 -- 172.21.15.43:0/2011706241 >> v2:172.21.15.203:3300/0 conn(0x7f6dd8143c20 msgr2=0x7f6dd806f540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-09T15:08:41.315 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:41.312+0000 7f6dcf7fe640 1 --2- 172.21.15.43:0/2011706241 >> v2:172.21.15.203:3300/0 conn(0x7f6dd8143c20 0x7f6dd806f540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T15:08:41.315 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:41.312+0000 7f6dcf7fe640 1 -- 172.21.15.43:0/2011706241 >> v1:172.21.15.43:6789/0 conn(0x7f6dd806c8d0 legacy=0x7f6dd806cd00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T15:08:41.315 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:41.312+0000 7f6dcf7fe640 1 -- 172.21.15.43:0/2011706241 --> v2:172.21.15.43:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6dd806b830 con 0x7f6dd80c86f0 2023-08-09T15:08:41.315 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:41.312+0000 7f6dcf7fe640 1 --2- 172.21.15.43:0/2011706241 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f6dd80c86f0 0x7f6dd80c8b50 secure :-1 s=READY pgs=2191 cs=0 l=1 rev1=1 crypto rx=0x7f6dd806e0f0 tx=0x7f6dc002bda0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f592fc97b3c299f9 server_cookie=0 in_seq=0 out_seq=0 2023-08-09T15:08:41.315 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:41.312+0000 7f6dd77fe640 1 -- 172.21.15.43:0/2011706241 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6dc002cb50 con 0x7f6dd80c86f0 2023-08-09T15:08:41.315 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:41.312+0000 7f6dd77fe640 1 -- 172.21.15.43:0/2011706241 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6dc002ccb0 con 0x7f6dd80c86f0 2023-08-09T15:08:41.315 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:41.312+0000 7f6dd77fe640 1 -- 172.21.15.43:0/2011706241 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6dc0034640 con 0x7f6dd80c86f0 2023-08-09T15:08:41.316 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:41.313+0000 7f6dde52b640 1 -- 172.21.15.43:0/2011706241 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f6dd80c86f0 msgr2=0x7f6dd80c8b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T15:08:41.316 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:41.313+0000 7f6dde52b640 1 --2- 172.21.15.43:0/2011706241 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f6dd80c86f0 0x7f6dd80c8b50 secure :-1 s=READY pgs=2191 cs=0 l=1 rev1=1 crypto rx=0x7f6dd806e0f0 tx=0x7f6dc002bda0 comp rx=0 tx=0).stop 2023-08-09T15:08:41.317 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:41.313+0000 7f6dde52b640 1 -- 172.21.15.43:0/2011706241 shutdown_connections 2023-08-09T15:08:41.317 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:41.313+0000 7f6dde52b640 1 --2- 172.21.15.43:0/2011706241 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f6dd80c86f0 0x7f6dd80c8b50 unknown :-1 s=CLOSED pgs=2191 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T15:08:41.317 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:41.313+0000 7f6dde52b640 1 --2- 172.21.15.43:0/2011706241 >> v2:172.21.15.203:3300/0 conn(0x7f6dd8143c20 0x7f6dd806f540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T15:08:41.317 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:41.313+0000 7f6dde52b640 1 -- 172.21.15.43:0/2011706241 >> 172.21.15.43:0/2011706241 conn(0x7f6dd80ce0a0 msgr2=0x7f6dd80ce480 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-09T15:08:41.317 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:41.313+0000 7f6dde52b640 1 -- 172.21.15.43:0/2011706241 shutdown_connections 2023-08-09T15:08:41.317 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:41.313+0000 7f6dde52b640 1 -- 172.21.15.43:0/2011706241 wait complete. 2023-08-09T15:08:41.317 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:41.315+0000 7f6dde52b640 1 Processor -- start 2023-08-09T15:08:41.318 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:41.315+0000 7f6dde52b640 1 -- start start 2023-08-09T15:08:41.319 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:41.316+0000 7f6dde52b640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f6dd806c8d0 0x7f6dd8159d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T15:08:41.319 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:41.316+0000 7f6dde52b640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f6dd80c86f0 0x7f6dd80ac940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T15:08:41.319 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:41.316+0000 7f6dde52b640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f6dd8143c20 0x7f6dd80ace80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T15:08:41.320 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:41.317+0000 7f6dde52b640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6dd80ac4b0 con 0x7f6dd806c8d0 2023-08-09T15:08:41.320 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:41.317+0000 7f6dde52b640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6dd815a280 con 0x7f6dd80c86f0 2023-08-09T15:08:41.320 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:41.317+0000 7f6dde52b640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6dd80ab8c0 con 0x7f6dd8143c20 2023-08-09T15:08:41.320 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:41.317+0000 7f6dd7fff640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f6dd806c8d0 0x7f6dd8159d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T15:08:41.320 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:41.317+0000 7f6dd7fff640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f6dd806c8d0 0x7f6dd8159d40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:39408/0 (socket says 172.21.15.43:39408) 2023-08-09T15:08:41.320 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:41.317+0000 7f6dd7fff640 1 -- 172.21.15.43:0/2417525845 learned_addr learned my addr 172.21.15.43:0/2417525845 (peer_addr_for_me v2:172.21.15.43:0/0) 2023-08-09T15:08:41.321 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:41.317+0000 7f6dcf7fe640 1 --2- 172.21.15.43:0/2417525845 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f6dd80c86f0 0x7f6dd80ac940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T15:08:41.321 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:41.317+0000 7f6dd7fff640 1 -- 172.21.15.43:0/2417525845 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f6dd8143c20 msgr2=0x7f6dd80ace80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T15:08:41.321 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:41.317+0000 7f6dd7fff640 1 --2- 172.21.15.43:0/2417525845 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f6dd8143c20 0x7f6dd80ace80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T15:08:41.321 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:41.317+0000 7f6dd7fff640 1 -- 172.21.15.43:0/2417525845 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f6dd80c86f0 msgr2=0x7f6dd80ac940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T15:08:41.321 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:41.317+0000 7f6dd7fff640 1 --2- 172.21.15.43:0/2417525845 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f6dd80c86f0 0x7f6dd80ac940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T15:08:41.321 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:41.317+0000 7f6dd7fff640 1 -- 172.21.15.43:0/2417525845 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6dc0009a70 con 0x7f6dd806c8d0 2023-08-09T15:08:41.321 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:41.318+0000 7f6dd7fff640 1 --2- 172.21.15.43:0/2417525845 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f6dd806c8d0 0x7f6dd8159d40 secure :-1 s=READY pgs=2192 cs=0 l=1 rev1=1 crypto rx=0x7f6dc800d800 tx=0x7f6dc800dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-09T15:08:41.321 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:41.318+0000 7f6dd57fa640 1 -- 172.21.15.43:0/2417525845 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6dc800b6c0 con 0x7f6dd806c8d0 2023-08-09T15:08:41.322 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:41.318+0000 7f6dd57fa640 1 -- 172.21.15.43:0/2417525845 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6dc800b820 con 0x7f6dd806c8d0 2023-08-09T15:08:41.322 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:41.318+0000 7f6dde52b640 1 -- 172.21.15.43:0/2417525845 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6dd80abb80 con 0x7f6dd806c8d0 2023-08-09T15:08:41.322 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:41.318+0000 7f6dd57fa640 1 -- 172.21.15.43:0/2417525845 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6dc800bbb0 con 0x7f6dd806c8d0 2023-08-09T15:08:41.322 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:41.318+0000 7f6dde52b640 1 -- 172.21.15.43:0/2417525845 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6dd80ac140 con 0x7f6dd806c8d0 2023-08-09T15:08:41.322 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:41.319+0000 7f6dde52b640 1 -- 172.21.15.43:0/2417525845 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6dd80739f0 con 0x7f6dd806c8d0 2023-08-09T15:08:41.323 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:41.320+0000 7f6dd57fa640 1 -- 172.21.15.43:0/2417525845 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 106) v1 ==== 88443+0+0 (secure 0 0 0) 0x7f6dc800bd10 con 0x7f6dd806c8d0 2023-08-09T15:08:41.323 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:41.323+0000 7f6dd57fa640 1 --2- 172.21.15.43:0/2417525845 >> [v2:172.21.15.203:6832/3082732769,v1:172.21.15.203:6833/3082732769] conn(0x7f6db806a1a0 0x7f6db806c660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T15:08:41.323 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:41.323+0000 7f6dd57fa640 1 -- 172.21.15.43:0/2417525845 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(201..201 src has 1..201) v4 ==== 5574+0+0 (secure 0 0 0) 0x7f6dc8088af0 con 0x7f6dd806c8d0 2023-08-09T15:08:41.324 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:41.323+0000 7f6dcf7fe640 1 --2- 172.21.15.43:0/2417525845 >> [v2:172.21.15.203:6832/3082732769,v1:172.21.15.203:6833/3082732769] conn(0x7f6db806a1a0 0x7f6db806c660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T15:08:41.325 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:41.324+0000 7f6dcf7fe640 1 --2- 172.21.15.43:0/2417525845 >> [v2:172.21.15.203:6832/3082732769,v1:172.21.15.203:6833/3082732769] conn(0x7f6db806a1a0 0x7f6db806c660 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7f6dc0036000 tx=0x7f6dc0030000 comp rx=0 tx=0).ready entity=mgr.8500 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-09T15:08:41.328 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:41.328+0000 7f6dd57fa640 1 -- 172.21.15.43:0/2417525845 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178066 (secure 0 0 0) 0x7f6dc8057ca0 con 0x7f6dd806c8d0 2023-08-09T15:08:41.582 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:41.581+0000 7f6dde52b640 1 -- 172.21.15.43:0/2417525845 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "auth ls", "format": "json-pretty"} v 0) v1 -- 0x7f6dd80cec90 con 0x7f6dd806c8d0 2023-08-09T15:08:41.583 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:41.582+0000 7f6dd57fa640 1 -- 172.21.15.43:0/2417525845 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "auth ls", "format": "json-pretty"}]=0 v45) v1 ==== 80+0+5304 (secure 0 0 0) 0x7f6dc8057830 con 0x7f6dd806c8d0 2023-08-09T15:08:41.583 INFO:teuthology.orchestra.run.smithi043.stdout: 2023-08-09T15:08:41.583 INFO:teuthology.orchestra.run.smithi043.stdout:{ 2023-08-09T15:08:41.584 INFO:teuthology.orchestra.run.smithi043.stdout: "auth_dump": [ 2023-08-09T15:08:41.584 INFO:teuthology.orchestra.run.smithi043.stdout: { 2023-08-09T15:08:41.584 INFO:teuthology.orchestra.run.smithi043.stdout: "entity": "mds.a", 2023-08-09T15:08:41.584 INFO:teuthology.orchestra.run.smithi043.stdout: "key": "AQCOldNkPu6GCRAAZecwctvMji0Zubb60KMlyQ==", 2023-08-09T15:08:41.584 INFO:teuthology.orchestra.run.smithi043.stdout: "caps": { 2023-08-09T15:08:41.584 INFO:teuthology.orchestra.run.smithi043.stdout: "mds": "allow", 2023-08-09T15:08:41.585 INFO:teuthology.orchestra.run.smithi043.stdout: "mgr": "allow *", 2023-08-09T15:08:41.585 INFO:teuthology.orchestra.run.smithi043.stdout: "mon": "allow *", 2023-08-09T15:08:41.585 INFO:teuthology.orchestra.run.smithi043.stdout: "osd": "allow *" 2023-08-09T15:08:41.585 INFO:teuthology.orchestra.run.smithi043.stdout: } 2023-08-09T15:08:41.585 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2023-08-09T15:08:41.585 INFO:teuthology.orchestra.run.smithi043.stdout: { 2023-08-09T15:08:41.585 INFO:teuthology.orchestra.run.smithi043.stdout: "entity": "mds.b", 2023-08-09T15:08:41.586 INFO:teuthology.orchestra.run.smithi043.stdout: "key": "AQCOldNkt2+5KxAAHcckB+S5NZHsF5z22StXmw==", 2023-08-09T15:08:41.586 INFO:teuthology.orchestra.run.smithi043.stdout: "caps": { 2023-08-09T15:08:41.586 INFO:teuthology.orchestra.run.smithi043.stdout: "mds": "allow", 2023-08-09T15:08:41.586 INFO:teuthology.orchestra.run.smithi043.stdout: "mgr": "allow *", 2023-08-09T15:08:41.586 INFO:teuthology.orchestra.run.smithi043.stdout: "mon": "allow *", 2023-08-09T15:08:41.586 INFO:teuthology.orchestra.run.smithi043.stdout: "osd": "allow *" 2023-08-09T15:08:41.586 INFO:teuthology.orchestra.run.smithi043.stdout: } 2023-08-09T15:08:41.587 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2023-08-09T15:08:41.587 INFO:teuthology.orchestra.run.smithi043.stdout: { 2023-08-09T15:08:41.587 INFO:teuthology.orchestra.run.smithi043.stdout: "entity": "mds.c", 2023-08-09T15:08:41.587 INFO:teuthology.orchestra.run.smithi043.stdout: "key": "AQCOldNk5V46GhAAQBhkvfjVSJCSHLS/YkhgCA==", 2023-08-09T15:08:41.587 INFO:teuthology.orchestra.run.smithi043.stdout: "caps": { 2023-08-09T15:08:41.587 INFO:teuthology.orchestra.run.smithi043.stdout: "mds": "allow", 2023-08-09T15:08:41.587 INFO:teuthology.orchestra.run.smithi043.stdout: "mgr": "allow *", 2023-08-09T15:08:41.588 INFO:teuthology.orchestra.run.smithi043.stdout: "mon": "allow *", 2023-08-09T15:08:41.588 INFO:teuthology.orchestra.run.smithi043.stdout: "osd": "allow *" 2023-08-09T15:08:41.588 INFO:teuthology.orchestra.run.smithi043.stdout: } 2023-08-09T15:08:41.588 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2023-08-09T15:08:41.588 INFO:teuthology.orchestra.run.smithi043.stdout: { 2023-08-09T15:08:41.588 INFO:teuthology.orchestra.run.smithi043.stdout: "entity": "mds.d", 2023-08-09T15:08:41.588 INFO:teuthology.orchestra.run.smithi043.stdout: "key": "AQCPldNkNBoTARAAMVW+rSyXQKaau6I90fUX9Q==", 2023-08-09T15:08:41.588 INFO:teuthology.orchestra.run.smithi043.stdout: "caps": { 2023-08-09T15:08:41.589 INFO:teuthology.orchestra.run.smithi043.stdout: "mds": "allow", 2023-08-09T15:08:41.589 INFO:teuthology.orchestra.run.smithi043.stdout: "mgr": "allow *", 2023-08-09T15:08:41.589 INFO:teuthology.orchestra.run.smithi043.stdout: "mon": "allow *", 2023-08-09T15:08:41.590 INFO:teuthology.orchestra.run.smithi043.stdout: "osd": "allow *" 2023-08-09T15:08:41.590 INFO:teuthology.orchestra.run.smithi043.stdout: } 2023-08-09T15:08:41.590 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2023-08-09T15:08:41.590 INFO:teuthology.orchestra.run.smithi043.stdout: { 2023-08-09T15:08:41.590 INFO:teuthology.orchestra.run.smithi043.stdout: "entity": "osd.0", 2023-08-09T15:08:41.591 INFO:teuthology.orchestra.run.smithi043.stdout: "key": "AQCRldNkltPlMhAAAt3rFvraCtI0aSqGvt6UxA==", 2023-08-09T15:08:41.591 INFO:teuthology.orchestra.run.smithi043.stdout: "caps": { 2023-08-09T15:08:41.591 INFO:teuthology.orchestra.run.smithi043.stdout: "mgr": "allow profile osd", 2023-08-09T15:08:41.591 INFO:teuthology.orchestra.run.smithi043.stdout: "mon": "allow profile osd", 2023-08-09T15:08:41.591 INFO:teuthology.orchestra.run.smithi043.stdout: "osd": "allow *" 2023-08-09T15:08:41.591 INFO:teuthology.orchestra.run.smithi043.stdout: } 2023-08-09T15:08:41.591 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2023-08-09T15:08:41.591 INFO:teuthology.orchestra.run.smithi043.stdout: { 2023-08-09T15:08:41.592 INFO:teuthology.orchestra.run.smithi043.stdout: "entity": "osd.1", 2023-08-09T15:08:41.592 INFO:teuthology.orchestra.run.smithi043.stdout: "key": "AQCVldNkzpKYEhAA+wen0kV/vUDLQztq3vhfOg==", 2023-08-09T15:08:41.592 INFO:teuthology.orchestra.run.smithi043.stdout: "caps": { 2023-08-09T15:08:41.592 INFO:teuthology.orchestra.run.smithi043.stdout: "mgr": "allow profile osd", 2023-08-09T15:08:41.592 INFO:teuthology.orchestra.run.smithi043.stdout: "mon": "allow profile osd", 2023-08-09T15:08:41.592 INFO:teuthology.orchestra.run.smithi043.stdout: "osd": "allow *" 2023-08-09T15:08:41.592 INFO:teuthology.orchestra.run.smithi043.stdout: } 2023-08-09T15:08:41.593 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2023-08-09T15:08:41.593 INFO:teuthology.orchestra.run.smithi043.stdout: { 2023-08-09T15:08:41.593 INFO:teuthology.orchestra.run.smithi043.stdout: "entity": "osd.2", 2023-08-09T15:08:41.593 INFO:teuthology.orchestra.run.smithi043.stdout: "key": "AQCYldNkYbwNHhAAQY8g9dldb1V9M2MM2SHKDg==", 2023-08-09T15:08:41.593 INFO:teuthology.orchestra.run.smithi043.stdout: "caps": { 2023-08-09T15:08:41.593 INFO:teuthology.orchestra.run.smithi043.stdout: "mgr": "allow profile osd", 2023-08-09T15:08:41.593 INFO:teuthology.orchestra.run.smithi043.stdout: "mon": "allow profile osd", 2023-08-09T15:08:41.593 INFO:teuthology.orchestra.run.smithi043.stdout: "osd": "allow *" 2023-08-09T15:08:41.594 INFO:teuthology.orchestra.run.smithi043.stdout: } 2023-08-09T15:08:41.594 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2023-08-09T15:08:41.594 INFO:teuthology.orchestra.run.smithi043.stdout: { 2023-08-09T15:08:41.594 INFO:teuthology.orchestra.run.smithi043.stdout: "entity": "osd.3", 2023-08-09T15:08:41.594 INFO:teuthology.orchestra.run.smithi043.stdout: "key": "AQCbldNkeyW6KRAAsd+kgqik2aD4U7PXwccWtA==", 2023-08-09T15:08:41.594 INFO:teuthology.orchestra.run.smithi043.stdout: "caps": { 2023-08-09T15:08:41.594 INFO:teuthology.orchestra.run.smithi043.stdout: "mgr": "allow profile osd", 2023-08-09T15:08:41.594 INFO:teuthology.orchestra.run.smithi043.stdout: "mon": "allow profile osd", 2023-08-09T15:08:41.595 INFO:teuthology.orchestra.run.smithi043.stdout: "osd": "allow *" 2023-08-09T15:08:41.595 INFO:teuthology.orchestra.run.smithi043.stdout: } 2023-08-09T15:08:41.595 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2023-08-09T15:08:41.595 INFO:teuthology.orchestra.run.smithi043.stdout: { 2023-08-09T15:08:41.595 INFO:teuthology.orchestra.run.smithi043.stdout: "entity": "osd.4", 2023-08-09T15:08:41.595 INFO:teuthology.orchestra.run.smithi043.stdout: "key": "AQCildNk80UkOBAAtX+4JGpOVcfZEHetJoeZKw==", 2023-08-09T15:08:41.595 INFO:teuthology.orchestra.run.smithi043.stdout: "caps": { 2023-08-09T15:08:41.596 INFO:teuthology.orchestra.run.smithi043.stdout: "mgr": "allow profile osd", 2023-08-09T15:08:41.596 INFO:teuthology.orchestra.run.smithi043.stdout: "mon": "allow profile osd", 2023-08-09T15:08:41.596 INFO:teuthology.orchestra.run.smithi043.stdout: "osd": "allow *" 2023-08-09T15:08:41.596 INFO:teuthology.orchestra.run.smithi043.stdout: } 2023-08-09T15:08:41.596 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2023-08-09T15:08:41.596 INFO:teuthology.orchestra.run.smithi043.stdout: { 2023-08-09T15:08:41.596 INFO:teuthology.orchestra.run.smithi043.stdout: "entity": "osd.5", 2023-08-09T15:08:41.596 INFO:teuthology.orchestra.run.smithi043.stdout: "key": "AQCmldNkCA9SBxAAZj4So6MRj2Eq1I9H9JzLDA==", 2023-08-09T15:08:41.597 INFO:teuthology.orchestra.run.smithi043.stdout: "caps": { 2023-08-09T15:08:41.597 INFO:teuthology.orchestra.run.smithi043.stdout: "mgr": "allow profile osd", 2023-08-09T15:08:41.597 INFO:teuthology.orchestra.run.smithi043.stdout: "mon": "allow profile osd", 2023-08-09T15:08:41.597 INFO:teuthology.orchestra.run.smithi043.stdout: "osd": "allow *" 2023-08-09T15:08:41.597 INFO:teuthology.orchestra.run.smithi043.stdout: } 2023-08-09T15:08:41.597 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2023-08-09T15:08:41.597 INFO:teuthology.orchestra.run.smithi043.stdout: { 2023-08-09T15:08:41.597 INFO:teuthology.orchestra.run.smithi043.stdout: "entity": "osd.6", 2023-08-09T15:08:41.598 INFO:teuthology.orchestra.run.smithi043.stdout: "key": "AQCpldNkaOwAExAAraV1tUwr9uU6GheZ+hvdqQ==", 2023-08-09T15:08:41.598 INFO:teuthology.orchestra.run.smithi043.stdout: "caps": { 2023-08-09T15:08:41.598 INFO:teuthology.orchestra.run.smithi043.stdout: "mgr": "allow profile osd", 2023-08-09T15:08:41.598 INFO:teuthology.orchestra.run.smithi043.stdout: "mon": "allow profile osd", 2023-08-09T15:08:41.598 INFO:teuthology.orchestra.run.smithi043.stdout: "osd": "allow *" 2023-08-09T15:08:41.598 INFO:teuthology.orchestra.run.smithi043.stdout: } 2023-08-09T15:08:41.598 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2023-08-09T15:08:41.598 INFO:teuthology.orchestra.run.smithi043.stdout: { 2023-08-09T15:08:41.599 INFO:teuthology.orchestra.run.smithi043.stdout: "entity": "osd.7", 2023-08-09T15:08:41.599 INFO:teuthology.orchestra.run.smithi043.stdout: "key": "AQCsldNk2/GwLBAAP42EXPivzLk44X8aMEVVCg==", 2023-08-09T15:08:41.599 INFO:teuthology.orchestra.run.smithi043.stdout: "caps": { 2023-08-09T15:08:41.599 INFO:teuthology.orchestra.run.smithi043.stdout: "mgr": "allow profile osd", 2023-08-09T15:08:41.599 INFO:teuthology.orchestra.run.smithi043.stdout: "mon": "allow profile osd", 2023-08-09T15:08:41.599 INFO:teuthology.orchestra.run.smithi043.stdout: "osd": "allow *" 2023-08-09T15:08:41.599 INFO:teuthology.orchestra.run.smithi043.stdout: } 2023-08-09T15:08:41.600 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2023-08-09T15:08:41.600 INFO:teuthology.orchestra.run.smithi043.stdout: { 2023-08-09T15:08:41.600 INFO:teuthology.orchestra.run.smithi043.stdout: "entity": "client.0", 2023-08-09T15:08:41.600 INFO:teuthology.orchestra.run.smithi043.stdout: "key": "AQCPldNkg8zrDxAAM/hnJaE6S9HU/fRwQeBV5w==", 2023-08-09T15:08:41.600 INFO:teuthology.orchestra.run.smithi043.stdout: "caps": { 2023-08-09T15:08:41.600 INFO:teuthology.orchestra.run.smithi043.stdout: "mds": "allow", 2023-08-09T15:08:41.600 INFO:teuthology.orchestra.run.smithi043.stdout: "mon": "allow r", 2023-08-09T15:08:41.600 INFO:teuthology.orchestra.run.smithi043.stdout: "osd": "allow rw tag cephfs data=cephfs" 2023-08-09T15:08:41.601 INFO:teuthology.orchestra.run.smithi043.stdout: } 2023-08-09T15:08:41.601 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2023-08-09T15:08:41.601 INFO:teuthology.orchestra.run.smithi043.stdout: { 2023-08-09T15:08:41.601 INFO:teuthology.orchestra.run.smithi043.stdout: "entity": "client.1", 2023-08-09T15:08:41.601 INFO:teuthology.orchestra.run.smithi043.stdout: "key": "AQCPldNklfFxJxAA19+L78rW6jU1uOyjey6WNA==", 2023-08-09T15:08:41.601 INFO:teuthology.orchestra.run.smithi043.stdout: "caps": { 2023-08-09T15:08:41.601 INFO:teuthology.orchestra.run.smithi043.stdout: "mds": "allow", 2023-08-09T15:08:41.601 INFO:teuthology.orchestra.run.smithi043.stdout: "mgr": "allow r", 2023-08-09T15:08:41.602 INFO:teuthology.orchestra.run.smithi043.stdout: "mon": "allow rw", 2023-08-09T15:08:41.602 INFO:teuthology.orchestra.run.smithi043.stdout: "osd": "allow rwx" 2023-08-09T15:08:41.602 INFO:teuthology.orchestra.run.smithi043.stdout: } 2023-08-09T15:08:41.602 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2023-08-09T15:08:41.602 INFO:teuthology.orchestra.run.smithi043.stdout: { 2023-08-09T15:08:41.602 INFO:teuthology.orchestra.run.smithi043.stdout: "entity": "client.2", 2023-08-09T15:08:41.602 INFO:teuthology.orchestra.run.smithi043.stdout: "key": "AQCPldNkC2lgGxAAZ8qZboOCUaQlcpkPL1ozHA==", 2023-08-09T15:08:41.603 INFO:teuthology.orchestra.run.smithi043.stdout: "caps": { 2023-08-09T15:08:41.603 INFO:teuthology.orchestra.run.smithi043.stdout: "mds": "allow", 2023-08-09T15:08:41.603 INFO:teuthology.orchestra.run.smithi043.stdout: "mgr": "allow r", 2023-08-09T15:08:41.603 INFO:teuthology.orchestra.run.smithi043.stdout: "mon": "allow rw", 2023-08-09T15:08:41.603 INFO:teuthology.orchestra.run.smithi043.stdout: "osd": "allow rwx" 2023-08-09T15:08:41.603 INFO:teuthology.orchestra.run.smithi043.stdout: } 2023-08-09T15:08:41.603 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2023-08-09T15:08:41.603 INFO:teuthology.orchestra.run.smithi043.stdout: { 2023-08-09T15:08:41.604 INFO:teuthology.orchestra.run.smithi043.stdout: "entity": "client.3", 2023-08-09T15:08:41.604 INFO:teuthology.orchestra.run.smithi043.stdout: "key": "AQCPldNkl+rXMhAA9rf3KdnSYbvngqmLYnIGGA==", 2023-08-09T15:08:41.604 INFO:teuthology.orchestra.run.smithi043.stdout: "caps": { 2023-08-09T15:08:41.604 INFO:teuthology.orchestra.run.smithi043.stdout: "mds": "allow", 2023-08-09T15:08:41.604 INFO:teuthology.orchestra.run.smithi043.stdout: "mgr": "allow r", 2023-08-09T15:08:41.604 INFO:teuthology.orchestra.run.smithi043.stdout: "mon": "allow rw", 2023-08-09T15:08:41.604 INFO:teuthology.orchestra.run.smithi043.stdout: "osd": "allow rwx" 2023-08-09T15:08:41.605 INFO:teuthology.orchestra.run.smithi043.stdout: } 2023-08-09T15:08:41.605 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2023-08-09T15:08:41.605 INFO:teuthology.orchestra.run.smithi043.stdout: { 2023-08-09T15:08:41.605 INFO:teuthology.orchestra.run.smithi043.stdout: "entity": "client.admin", 2023-08-09T15:08:41.605 INFO:teuthology.orchestra.run.smithi043.stdout: "key": "AQCMldNkUae6NRAA2FBTr0QPBdKNDoe1WK5zyg==", 2023-08-09T15:08:41.605 INFO:teuthology.orchestra.run.smithi043.stdout: "caps": { 2023-08-09T15:08:41.605 INFO:teuthology.orchestra.run.smithi043.stdout: "mds": "allow *", 2023-08-09T15:08:41.605 INFO:teuthology.orchestra.run.smithi043.stdout: "mgr": "allow *", 2023-08-09T15:08:41.606 INFO:teuthology.orchestra.run.smithi043.stdout: "mon": "allow *", 2023-08-09T15:08:41.606 INFO:teuthology.orchestra.run.smithi043.stdout: "osd": "allow *" 2023-08-09T15:08:41.606 INFO:teuthology.orchestra.run.smithi043.stdout: } 2023-08-09T15:08:41.606 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2023-08-09T15:08:41.606 INFO:teuthology.orchestra.run.smithi043.stdout: { 2023-08-09T15:08:41.606 INFO:teuthology.orchestra.run.smithi043.stdout: "entity": "mgr.x", 2023-08-09T15:08:41.606 INFO:teuthology.orchestra.run.smithi043.stdout: "key": "AQCNldNk2ufSNxAAU7msRkwLaoLZHTIsk0ck/g==", 2023-08-09T15:08:41.606 INFO:teuthology.orchestra.run.smithi043.stdout: "caps": { 2023-08-09T15:08:41.607 INFO:teuthology.orchestra.run.smithi043.stdout: "mds": "allow *", 2023-08-09T15:08:41.607 INFO:teuthology.orchestra.run.smithi043.stdout: "mon": "allow profile mgr", 2023-08-09T15:08:41.607 INFO:teuthology.orchestra.run.smithi043.stdout: "osd": "allow *" 2023-08-09T15:08:41.607 INFO:teuthology.orchestra.run.smithi043.stdout: } 2023-08-09T15:08:41.607 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2023-08-09T15:08:41.607 INFO:teuthology.orchestra.run.smithi043.stdout: { 2023-08-09T15:08:41.607 INFO:teuthology.orchestra.run.smithi043.stdout: "entity": "mgr.y", 2023-08-09T15:08:41.608 INFO:teuthology.orchestra.run.smithi043.stdout: "key": "AQCNldNk2YgzLBAAA11Q8uAMEgE/bUni+tXIKA==", 2023-08-09T15:08:41.608 INFO:teuthology.orchestra.run.smithi043.stdout: "caps": { 2023-08-09T15:08:41.608 INFO:teuthology.orchestra.run.smithi043.stdout: "mds": "allow *", 2023-08-09T15:08:41.608 INFO:teuthology.orchestra.run.smithi043.stdout: "mon": "allow profile mgr", 2023-08-09T15:08:41.608 INFO:teuthology.orchestra.run.smithi043.stdout: "osd": "allow *" 2023-08-09T15:08:41.608 INFO:teuthology.orchestra.run.smithi043.stdout: } 2023-08-09T15:08:41.608 INFO:teuthology.orchestra.run.smithi043.stdout: } 2023-08-09T15:08:41.608 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2023-08-09T15:08:41.609 INFO:teuthology.orchestra.run.smithi043.stdout:} 2023-08-09T15:08:41.610 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:41.585+0000 7f6dde52b640 1 -- 172.21.15.43:0/2417525845 >> [v2:172.21.15.203:6832/3082732769,v1:172.21.15.203:6833/3082732769] conn(0x7f6db806a1a0 msgr2=0x7f6db806c660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T15:08:41.610 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:41.585+0000 7f6dde52b640 1 --2- 172.21.15.43:0/2417525845 >> [v2:172.21.15.203:6832/3082732769,v1:172.21.15.203:6833/3082732769] conn(0x7f6db806a1a0 0x7f6db806c660 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7f6dc0036000 tx=0x7f6dc0030000 comp rx=0 tx=0).stop 2023-08-09T15:08:41.610 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:41.585+0000 7f6dde52b640 1 -- 172.21.15.43:0/2417525845 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f6dd806c8d0 msgr2=0x7f6dd8159d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T15:08:41.610 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:41.585+0000 7f6dde52b640 1 --2- 172.21.15.43:0/2417525845 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f6dd806c8d0 0x7f6dd8159d40 secure :-1 s=READY pgs=2192 cs=0 l=1 rev1=1 crypto rx=0x7f6dc800d800 tx=0x7f6dc800dcc0 comp rx=0 tx=0).stop 2023-08-09T15:08:41.610 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:41.586+0000 7f6dde52b640 1 -- 172.21.15.43:0/2417525845 shutdown_connections 2023-08-09T15:08:41.610 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:41.586+0000 7f6dde52b640 1 --2- 172.21.15.43:0/2417525845 >> [v2:172.21.15.203:6832/3082732769,v1:172.21.15.203:6833/3082732769] conn(0x7f6db806a1a0 0x7f6db806c660 unknown :-1 s=CLOSED pgs=49 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T15:08:41.611 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:41.586+0000 7f6dde52b640 1 --2- 172.21.15.43:0/2417525845 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f6dd8143c20 0x7f6dd80ace80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T15:08:41.611 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:41.586+0000 7f6dde52b640 1 --2- 172.21.15.43:0/2417525845 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f6dd80c86f0 0x7f6dd80ac940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T15:08:41.611 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:41.586+0000 7f6dde52b640 1 --2- 172.21.15.43:0/2417525845 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f6dd806c8d0 0x7f6dd8159d40 unknown :-1 s=CLOSED pgs=2192 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T15:08:41.611 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:41.586+0000 7f6dde52b640 1 -- 172.21.15.43:0/2417525845 >> 172.21.15.43:0/2417525845 conn(0x7f6dd80ce0a0 msgr2=0x7f6dd80adc00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-09T15:08:41.611 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:41.586+0000 7f6dde52b640 1 -- 172.21.15.43:0/2417525845 shutdown_connections 2023-08-09T15:08:41.611 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:41.586+0000 7f6dde52b640 1 -- 172.21.15.43:0/2417525845 wait complete. 2023-08-09T15:08:41.612 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-08-09T15:08:41.827 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:41.826+0000 7f2792287640 1 Processor -- start 2023-08-09T15:08:41.827 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:41.827+0000 7f2792287640 1 -- start start 2023-08-09T15:08:41.828 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:41.828+0000 7f2792287640 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7f278c0b2790 0x7f278c0d2ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T15:08:41.829 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:41.828+0000 7f2792287640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f278c0b3050 0x7f278c0b3f60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T15:08:41.829 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:41.828+0000 7f2792287640 1 -- --> v1:172.21.15.43:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f278c0ce090 con 0x7f278c0b2b70 2023-08-09T15:08:41.829 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:41.828+0000 7f2792287640 1 -- --> v2:172.21.15.43:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f278c0b3950 con 0x7f278c0b3050 2023-08-09T15:08:41.829 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:41.828+0000 7f2792287640 1 -- --> v2:172.21.15.203:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f278c0cd7f0 con 0x7f278c0b2790 2023-08-09T15:08:41.830 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:41.829+0000 7f278b7fe640 1 --1- >> v1:172.21.15.43:6789/0 conn(0x7f278c0b2b70 0x7f278c0b5330 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.43:6789/0 says I am v1:172.21.15.43:59622/0 (socket says 172.21.15.43:59622) 2023-08-09T15:08:41.830 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:41.829+0000 7f278b7fe640 1 -- 172.21.15.43:0/450197526 learned_addr learned my addr 172.21.15.43:0/450197526 (peer_addr_for_me v1:172.21.15.43:0/0) 2023-08-09T15:08:41.830 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:41.829+0000 7f2790824640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f278c0b3050 0x7f278c0b3f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T15:08:41.830 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:41.829+0000 7f278bfff640 1 --2- 172.21.15.43:0/450197526 >> v2:172.21.15.203:3300/0 conn(0x7f278c0b2790 0x7f278c0d2ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T15:08:41.831 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:41.829+0000 7f2790824640 1 -- 172.21.15.43:0/450197526 >> v2:172.21.15.203:3300/0 conn(0x7f278c0b2790 msgr2=0x7f278c0d2ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-09T15:08:41.831 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:41.829+0000 7f2790824640 1 --2- 172.21.15.43:0/450197526 >> v2:172.21.15.203:3300/0 conn(0x7f278c0b2790 0x7f278c0d2ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T15:08:41.831 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:41.829+0000 7f2790824640 1 -- 172.21.15.43:0/450197526 >> v1:172.21.15.43:6789/0 conn(0x7f278c0b2b70 legacy=0x7f278c0b5330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T15:08:41.831 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:41.829+0000 7f2790824640 1 -- 172.21.15.43:0/450197526 --> v2:172.21.15.43:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f278c0cdad0 con 0x7f278c0b3050 2023-08-09T15:08:41.831 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:41.829+0000 7f278affd640 1 -- 172.21.15.43:0/450197526 <== mon.0 v1:172.21.15.43:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2289707199 0 0) 0x7f278c0ce090 con 0x7f278c0b2b70 2023-08-09T15:08:41.832 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:41.830+0000 7f2790824640 1 --2- 172.21.15.43:0/450197526 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f278c0b3050 0x7f278c0b3f60 secure :-1 s=READY pgs=2194 cs=0 l=1 rev1=1 crypto rx=0x7f278c08a4a0 tx=0x7f278402bec0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2496ee0ccf71ddf1 server_cookie=0 in_seq=0 out_seq=0 2023-08-09T15:08:41.832 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:41.831+0000 7f278affd640 1 -- 172.21.15.43:0/450197526 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f278402cc70 con 0x7f278c0b3050 2023-08-09T15:08:41.832 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:41.831+0000 7f278affd640 1 -- 172.21.15.43:0/450197526 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f278402cdd0 con 0x7f278c0b3050 2023-08-09T15:08:41.833 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:41.831+0000 7f278affd640 1 -- 172.21.15.43:0/450197526 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2784034660 con 0x7f278c0b3050 2023-08-09T15:08:41.833 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:41.831+0000 7f2792287640 1 -- 172.21.15.43:0/450197526 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f278c0b3050 msgr2=0x7f278c0b3f60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T15:08:41.833 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:41.831+0000 7f2792287640 1 --2- 172.21.15.43:0/450197526 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f278c0b3050 0x7f278c0b3f60 secure :-1 s=READY pgs=2194 cs=0 l=1 rev1=1 crypto rx=0x7f278c08a4a0 tx=0x7f278402bec0 comp rx=0 tx=0).stop 2023-08-09T15:08:41.833 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:41.831+0000 7f2792287640 1 -- 172.21.15.43:0/450197526 shutdown_connections 2023-08-09T15:08:41.833 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:41.831+0000 7f2792287640 1 --2- 172.21.15.43:0/450197526 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f278c0b3050 0x7f278c0b3f60 unknown :-1 s=CLOSED pgs=2194 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T15:08:41.834 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:41.831+0000 7f2792287640 1 --2- 172.21.15.43:0/450197526 >> v2:172.21.15.203:3300/0 conn(0x7f278c0b2790 0x7f278c0d2ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T15:08:41.834 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:41.831+0000 7f2792287640 1 -- 172.21.15.43:0/450197526 >> 172.21.15.43:0/450197526 conn(0x7f278c09b7b0 msgr2=0x7f278c0d24e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-09T15:08:41.834 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:41.832+0000 7f2792287640 1 -- 172.21.15.43:0/450197526 shutdown_connections 2023-08-09T15:08:41.834 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:41.832+0000 7f2792287640 1 -- 172.21.15.43:0/450197526 wait complete. 2023-08-09T15:08:41.835 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:41.833+0000 7f2792287640 1 Processor -- start 2023-08-09T15:08:41.836 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:41.833+0000 7f2792287640 1 -- start start 2023-08-09T15:08:41.836 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:41.834+0000 7f2792287640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f278c0b2790 0x7f278c0b0bc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T15:08:41.836 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:41.834+0000 7f2792287640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f278c0b2b70 0x7f278c0c8790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T15:08:41.836 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:41.834+0000 7f2792287640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f278c0b3050 0x7f278c0b3f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T15:08:41.836 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:41.834+0000 7f2792287640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f278c0af240 con 0x7f278c0b2790 2023-08-09T15:08:41.836 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:41.834+0000 7f2792287640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f278c0af520 con 0x7f278c0b2b70 2023-08-09T15:08:41.836 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:41.834+0000 7f2792287640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f278c0af800 con 0x7f278c0b3050 2023-08-09T15:08:41.837 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:41.835+0000 7f278bfff640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f278c0b2790 0x7f278c0b0bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T15:08:41.837 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:41.835+0000 7f278bfff640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f278c0b2790 0x7f278c0b0bc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:39422/0 (socket says 172.21.15.43:39422) 2023-08-09T15:08:41.837 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:41.835+0000 7f278bfff640 1 -- 172.21.15.43:0/227981885 learned_addr learned my addr 172.21.15.43:0/227981885 (peer_addr_for_me v2:172.21.15.43:0/0) 2023-08-09T15:08:41.837 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:41.835+0000 7f278b7fe640 1 --2- 172.21.15.43:0/227981885 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f278c0b2b70 0x7f278c0c8790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T15:08:41.837 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:41.835+0000 7f278bfff640 1 -- 172.21.15.43:0/227981885 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f278c0b3050 msgr2=0x7f278c0b3f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T15:08:41.837 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:41.835+0000 7f2790824640 1 --2- 172.21.15.43:0/227981885 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f278c0b3050 0x7f278c0b3f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T15:08:41.837 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:41.835+0000 7f278bfff640 1 --2- 172.21.15.43:0/227981885 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f278c0b3050 0x7f278c0b3f60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T15:08:41.837 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:41.835+0000 7f278bfff640 1 -- 172.21.15.43:0/227981885 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f278c0b2b70 msgr2=0x7f278c0c8790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T15:08:41.838 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:41.835+0000 7f278bfff640 1 --2- 172.21.15.43:0/227981885 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f278c0b2b70 0x7f278c0c8790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T15:08:41.838 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:41.835+0000 7f278bfff640 1 -- 172.21.15.43:0/227981885 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2784009a70 con 0x7f278c0b2790 2023-08-09T15:08:41.838 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:41.835+0000 7f278b7fe640 1 --2- 172.21.15.43:0/227981885 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f278c0b2b70 0x7f278c0c8790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-08-09T15:08:41.838 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:41.835+0000 7f278bfff640 1 --2- 172.21.15.43:0/227981885 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f278c0b2790 0x7f278c0b0bc0 secure :-1 s=READY pgs=2195 cs=0 l=1 rev1=1 crypto rx=0x7f277c00d5d0 tx=0x7f277c00da90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-09T15:08:41.839 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:41.836+0000 7f2788ff9640 1 -- 172.21.15.43:0/227981885 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f277c0125f0 con 0x7f278c0b2790 2023-08-09T15:08:41.839 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:41.836+0000 7f2788ff9640 1 -- 172.21.15.43:0/227981885 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f277c012770 con 0x7f278c0b2790 2023-08-09T15:08:41.839 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:41.836+0000 7f2788ff9640 1 -- 172.21.15.43:0/227981885 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f277c012b20 con 0x7f278c0b2790 2023-08-09T15:08:41.839 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:41.836+0000 7f2792287640 1 -- 172.21.15.43:0/227981885 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f278c0afae0 con 0x7f278c0b2790 2023-08-09T15:08:41.839 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:41.836+0000 7f2792287640 1 -- 172.21.15.43:0/227981885 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f278c0b1100 con 0x7f278c0b2790 2023-08-09T15:08:41.840 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:41.837+0000 7f2792287640 1 -- 172.21.15.43:0/227981885 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f278c0739f0 con 0x7f278c0b2790 2023-08-09T15:08:41.840 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:41.838+0000 7f2788ff9640 1 -- 172.21.15.43:0/227981885 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 106) v1 ==== 88443+0+0 (secure 0 0 0) 0x7f277c012c80 con 0x7f278c0b2790 2023-08-09T15:08:41.840 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:41.839+0000 7f2788ff9640 1 --2- 172.21.15.43:0/227981885 >> [v2:172.21.15.203:6832/3082732769,v1:172.21.15.203:6833/3082732769] conn(0x7f277806a170 0x7f277806c630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T15:08:41.840 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:41.839+0000 7f2788ff9640 1 -- 172.21.15.43:0/227981885 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(201..201 src has 1..201) v4 ==== 5574+0+0 (secure 0 0 0) 0x7f277c088af0 con 0x7f278c0b2790 2023-08-09T15:08:41.840 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:41.840+0000 7f278b7fe640 1 --2- 172.21.15.43:0/227981885 >> [v2:172.21.15.203:6832/3082732769,v1:172.21.15.203:6833/3082732769] conn(0x7f277806a170 0x7f277806c630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T15:08:41.841 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:41.841+0000 7f278b7fe640 1 --2- 172.21.15.43:0/227981885 >> [v2:172.21.15.203:6832/3082732769,v1:172.21.15.203:6833/3082732769] conn(0x7f277806a170 0x7f277806c630 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7f278c06b4c0 tx=0x7f2780009040 comp rx=0 tx=0).ready entity=mgr.8500 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-09T15:08:41.843 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:41.843+0000 7f2788ff9640 1 -- 172.21.15.43:0/227981885 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178066 (secure 0 0 0) 0x7f277c054210 con 0x7f278c0b2790 2023-08-09T15:08:42.041 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:42.040+0000 7f2792287640 1 -- 172.21.15.43:0/227981885 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f278c089fe0 con 0x7f278c0b2790 2023-08-09T15:08:42.042 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:42.042+0000 7f2788ff9640 1 -- 172.21.15.43:0/227981885 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v201) v1 ==== 74+0+14854 (secure 0 0 0) 0x7f277c057830 con 0x7f278c0b2790 2023-08-09T15:08:42.043 INFO:teuthology.orchestra.run.smithi043.stdout: 2023-08-09T15:08:42.043 INFO:teuthology.orchestra.run.smithi043.stdout:{"epoch":201,"fsid":"daac87ee-1337-4250-a955-aea4e408a39b","created":"2023-08-09T13:33:41.701198+0000","modified":"2023-08-09T15:08:41.065444+0000","last_up_change":"2023-08-09T13:33:53.783453+0000","last_in_change":"2023-08-09T13:33:45.148904+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":23,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":"rbd","create_time":"2023-08-09T13:33:54.365137+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":".mgr","create_time":"2023-08-09T13:33:54.831803+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"cbd49ecd-e896-4fbd-864e-5c37a829b165","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":184,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6800","nonce":502667335},{"type":"v1","addr":"172.21.15.43:6801","nonce":502667335}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6802","nonce":502667335},{"type":"v1","addr":"172.21.15.43:6803","nonce":502667335}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6806","nonce":502667335},{"type":"v1","addr":"172.21.15.43:6807","nonce":502667335}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6804","nonce":502667335},{"type":"v1","addr":"172.21.15.43:6805","nonce":502667335}]},"public_addr":"172.21.15.43:6801/502667335","cluster_addr":"172.21.15.43:6803/502667335","heartbeat_back_addr":"172.21.15.43:6807/502667335","heartbeat_front_addr":"172.21.15.43:6805/502667335","state":["exists","up"]},{"osd":1,"uuid":"75c7e0fe-1ec7-4fe8-a2f2-3661a65b8500","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":184,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6808","nonce":3289134610},{"type":"v1","addr":"172.21.15.43:6809","nonce":3289134610}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6810","nonce":3289134610},{"type":"v1","addr":"172.21.15.43:6812","nonce":3289134610}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6821","nonce":3289134610},{"type":"v1","addr":"172.21.15.43:6824","nonce":3289134610}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6815","nonce":3289134610},{"type":"v1","addr":"172.21.15.43:6818","nonce":3289134610}]},"public_addr":"172.21.15.43:6809/3289134610","cluster_addr":"172.21.15.43:6812/3289134610","heartbeat_back_addr":"172.21.15.43:6824/3289134610","heartbeat_front_addr":"172.21.15.43:6818/3289134610","state":["exists","up"]},{"osd":2,"uuid":"bb5c41e9-7f5f-4125-9b01-3bb8bfc2ea83","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":184,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6811","nonce":4079709736},{"type":"v1","addr":"172.21.15.43:6814","nonce":4079709736}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6817","nonce":4079709736},{"type":"v1","addr":"172.21.15.43:6820","nonce":4079709736}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6829","nonce":4079709736},{"type":"v1","addr":"172.21.15.43:6831","nonce":4079709736}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6823","nonce":4079709736},{"type":"v1","addr":"172.21.15.43:6826","nonce":4079709736}]},"public_addr":"172.21.15.43:6814/4079709736","cluster_addr":"172.21.15.43:6820/4079709736","heartbeat_back_addr":"172.21.15.43:6831/4079709736","heartbeat_front_addr":"172.21.15.43:6826/4079709736","state":["exists","up"]},{"osd":3,"uuid":"bfe4c8b4-694a-465c-87e6-2b2aab852613","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":184,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6813","nonce":1709538759},{"type":"v1","addr":"172.21.15.43:6816","nonce":1709538759}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6819","nonce":1709538759},{"type":"v1","addr":"172.21.15.43:6822","nonce":1709538759}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6828","nonce":1709538759},{"type":"v1","addr":"172.21.15.43:6830","nonce":1709538759}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6825","nonce":1709538759},{"type":"v1","addr":"172.21.15.43:6827","nonce":1709538759}]},"public_addr":"172.21.15.43:6816/1709538759","cluster_addr":"172.21.15.43:6822/1709538759","heartbeat_back_addr":"172.21.15.43:6830/1709538759","heartbeat_front_addr":"172.21.15.43:6827/1709538759","state":["exists","up"]},{"osd":4,"uuid":"fa2e8a77-aba1-4756-853e-0d174a5bdeba","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":184,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6800","nonce":4196866481},{"type":"v1","addr":"172.21.15.203:6801","nonce":4196866481}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6802","nonce":4196866481},{"type":"v1","addr":"172.21.15.203:6803","nonce":4196866481}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6806","nonce":4196866481},{"type":"v1","addr":"172.21.15.203:6807","nonce":4196866481}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6804","nonce":4196866481},{"type":"v1","addr":"172.21.15.203:6805","nonce":4196866481}]},"public_addr":"172.21.15.203:6801/4196866481","cluster_addr":"172.21.15.203:6803/4196866481","heartbeat_back_addr":"172.21.15.203:6807/4196866481","heartbeat_front_addr":"172.21.15.203:6805/4196866481","state":["exists","up"]},{"osd":5,"uuid":"81811e4b-4d24-4f51-a82c-f1d700a2a00d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":184,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6816","nonce":3059290081},{"type":"v1","addr":"172.21.15.203:6817","nonce":3059290081}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6818","nonce":3059290081},{"type":"v1","addr":"172.21.15.203:6819","nonce":3059290081}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6822","nonce":3059290081},{"type":"v1","addr":"172.21.15.203:6823","nonce":3059290081}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6820","nonce":3059290081},{"type":"v1","addr":"172.21.15.203:6821","nonce":3059290081}]},"public_addr":"172.21.15.203:6817/3059290081","cluster_addr":"172.21.15.203:6819/3059290081","heartbeat_back_addr":"172.21.15.203:6823/3059290081","heartbeat_front_addr":"172.21.15.203:6821/3059290081","state":["exists","up"]},{"osd":6,"uuid":"307bac4d-6a5b-443d-af75-56d695d7674f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":184,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6824","nonce":512155569},{"type":"v1","addr":"172.21.15.203:6825","nonce":512155569}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6826","nonce":512155569},{"type":"v1","addr":"172.21.15.203:6827","nonce":512155569}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6830","nonce":512155569},{"type":"v1","addr":"172.21.15.203:6831","nonce":512155569}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6828","nonce":512155569},{"type":"v1","addr":"172.21.15.203:6829","nonce":512155569}]},"public_addr":"172.21.15.203:6825/512155569","cluster_addr":"172.21.15.203:6827/512155569","heartbeat_back_addr":"172.21.15.203:6831/512155569","heartbeat_front_addr":"172.21.15.203:6829/512155569","state":["exists","up"]},{"osd":7,"uuid":"a95dc7ca-9f1b-4295-af70-68a22a2d7d5e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":184,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6808","nonce":3331450438},{"type":"v1","addr":"172.21.15.203:6809","nonce":3331450438}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6810","nonce":3331450438},{"type":"v1","addr":"172.21.15.203:6811","nonce":3331450438}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6814","nonce":3331450438},{"type":"v1","addr":"172.21.15.203:6815","nonce":3331450438}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6812","nonce":3331450438},{"type":"v1","addr":"172.21.15.203:6813","nonce":3331450438}]},"public_addr":"172.21.15.203:6809/3331450438","cluster_addr":"172.21.15.203:6811/3331450438","heartbeat_back_addr":"172.21.15.203:6815/3331450438","heartbeat_front_addr":"172.21.15.203:6813/3331450438","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-09T13:33:50.547652+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-09T13:33:50.589469+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-09T13:33:50.561724+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-09T13:33:50.599650+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-09T13:33:50.411855+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-09T13:33:50.219681+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-09T13:33:50.504664+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-09T13:33:50.441248+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":7,"snaps":[{"begin":104,"length":1},{"begin":221,"length":1}]}],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-08-09T15:08:42.046 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:42.045+0000 7f2792287640 1 -- 172.21.15.43:0/227981885 >> [v2:172.21.15.203:6832/3082732769,v1:172.21.15.203:6833/3082732769] conn(0x7f277806a170 msgr2=0x7f277806c630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T15:08:42.046 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:42.045+0000 7f2792287640 1 --2- 172.21.15.43:0/227981885 >> [v2:172.21.15.203:6832/3082732769,v1:172.21.15.203:6833/3082732769] conn(0x7f277806a170 0x7f277806c630 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7f278c06b4c0 tx=0x7f2780009040 comp rx=0 tx=0).stop 2023-08-09T15:08:42.046 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:42.045+0000 7f2792287640 1 -- 172.21.15.43:0/227981885 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f278c0b2790 msgr2=0x7f278c0b0bc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T15:08:42.047 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:42.045+0000 7f2792287640 1 --2- 172.21.15.43:0/227981885 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f278c0b2790 0x7f278c0b0bc0 secure :-1 s=READY pgs=2195 cs=0 l=1 rev1=1 crypto rx=0x7f277c00d5d0 tx=0x7f277c00da90 comp rx=0 tx=0).stop 2023-08-09T15:08:42.047 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:42.045+0000 7f2792287640 1 -- 172.21.15.43:0/227981885 shutdown_connections 2023-08-09T15:08:42.047 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:42.045+0000 7f2792287640 1 --2- 172.21.15.43:0/227981885 >> [v2:172.21.15.203:6832/3082732769,v1:172.21.15.203:6833/3082732769] conn(0x7f277806a170 0x7f277806c630 unknown :-1 s=CLOSED pgs=50 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T15:08:42.047 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:42.045+0000 7f2792287640 1 --2- 172.21.15.43:0/227981885 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f278c0b3050 0x7f278c0b3f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T15:08:42.048 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:42.045+0000 7f2792287640 1 --2- 172.21.15.43:0/227981885 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f278c0b2b70 0x7f278c0c8790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T15:08:42.048 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:42.045+0000 7f2792287640 1 --2- 172.21.15.43:0/227981885 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f278c0b2790 0x7f278c0b0bc0 unknown :-1 s=CLOSED pgs=2195 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T15:08:42.048 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:42.045+0000 7f2792287640 1 -- 172.21.15.43:0/227981885 >> 172.21.15.43:0/227981885 conn(0x7f278c09b7b0 msgr2=0x7f278c0cf060 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-09T15:08:42.049 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:42.045+0000 7f2792287640 1 -- 172.21.15.43:0/227981885 shutdown_connections 2023-08-09T15:08:42.049 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:42.046+0000 7f2792287640 1 -- 172.21.15.43:0/227981885 wait complete. 2023-08-09T15:08:42.067 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': 'rbd', 'create_time': '2023-08-09T13:33:54.365137+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}, 'read_balance': {'score_acting': 2, 'score_stable': 2, 'optimal_score': 1, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': '.mgr', 'create_time': '2023-08-09T13:33:54.831803+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-08-09T15:08:42.068 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get rbd pg_num 2023-08-09T15:08:42.265 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:42.264+0000 7f394c95c640 1 Processor -- start 2023-08-09T15:08:42.265 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:42.265+0000 7f394c95c640 1 -- start start 2023-08-09T15:08:42.266 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:42.266+0000 7f394c95c640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f39480c90a0 0x7f394815af10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T15:08:42.267 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:42.266+0000 7f394c95c640 1 -- --> v1:172.21.15.43:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3948158280 con 0x7f39480c9ee0 2023-08-09T15:08:42.267 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:42.266+0000 7f394c95c640 1 -- --> v1:172.21.15.203:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f394815b560 con 0x7f3948157ea0 2023-08-09T15:08:42.267 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:42.266+0000 7f394c95c640 1 -- --> v2:172.21.15.43:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f394815b840 con 0x7f39480c90a0 2023-08-09T15:08:42.267 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:42.267+0000 7f394759e640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f39480c90a0 0x7f394815af10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T15:08:42.268 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:42.267+0000 7f394759e640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f39480c90a0 0x7f394815af10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:39424/0 (socket says 172.21.15.43:39424) 2023-08-09T15:08:42.268 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:42.267+0000 7f394759e640 1 -- 172.21.15.43:0/3401787896 learned_addr learned my addr 172.21.15.43:0/3401787896 (peer_addr_for_me v2:172.21.15.43:0/0) 2023-08-09T15:08:42.268 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:42.267+0000 7f393ffff640 1 --1- >> v1:172.21.15.43:6789/0 conn(0x7f39480c9ee0 0x7f394815a800 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.43:6789/0 says I am v1:172.21.15.43:59632/0 (socket says 172.21.15.43:59632) 2023-08-09T15:08:42.268 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:42.267+0000 7f394759e640 1 -- 172.21.15.43:0/3401787896 >> v1:172.21.15.203:6789/0 conn(0x7f3948157ea0 legacy=0x7f39480c97d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-09T15:08:42.268 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:42.267+0000 7f394759e640 1 -- 172.21.15.43:0/3401787896 >> v1:172.21.15.43:6789/0 conn(0x7f39480c9ee0 legacy=0x7f394815a800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T15:08:42.268 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:42.267+0000 7f394759e640 1 -- 172.21.15.43:0/3401787896 --> v2:172.21.15.43:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f39480c85c0 con 0x7f39480c90a0 2023-08-09T15:08:42.269 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:42.268+0000 7f394759e640 1 --2- 172.21.15.43:0/3401787896 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f39480c90a0 0x7f394815af10 secure :-1 s=READY pgs=2197 cs=0 l=1 rev1=1 crypto rx=0x7f3940009e50 tx=0x7f394002bf70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5978ff2ac78fafc server_cookie=0 in_seq=0 out_seq=0 2023-08-09T15:08:42.269 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:42.269+0000 7f394659c640 1 -- 172.21.15.43:0/3401787896 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f394002cb10 con 0x7f39480c90a0 2023-08-09T15:08:42.269 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:42.269+0000 7f394659c640 1 -- 172.21.15.43:0/3401787896 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f394002cc70 con 0x7f39480c90a0 2023-08-09T15:08:42.270 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:42.269+0000 7f394659c640 1 -- 172.21.15.43:0/3401787896 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f39400343f0 con 0x7f39480c90a0 2023-08-09T15:08:42.270 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:42.269+0000 7f394c95c640 1 -- 172.21.15.43:0/3401787896 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f39480c90a0 msgr2=0x7f394815af10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T15:08:42.270 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:42.269+0000 7f394c95c640 1 --2- 172.21.15.43:0/3401787896 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f39480c90a0 0x7f394815af10 secure :-1 s=READY pgs=2197 cs=0 l=1 rev1=1 crypto rx=0x7f3940009e50 tx=0x7f394002bf70 comp rx=0 tx=0).stop 2023-08-09T15:08:42.271 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:42.269+0000 7f394c95c640 1 -- 172.21.15.43:0/3401787896 shutdown_connections 2023-08-09T15:08:42.271 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:42.269+0000 7f394c95c640 1 --2- 172.21.15.43:0/3401787896 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f39480c90a0 0x7f394815af10 unknown :-1 s=CLOSED pgs=2197 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T15:08:42.271 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:42.270+0000 7f394c95c640 1 -- 172.21.15.43:0/3401787896 >> 172.21.15.43:0/3401787896 conn(0x7f39480cddd0 msgr2=0x7f39480ce1e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-09T15:08:42.271 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:42.270+0000 7f394c95c640 1 -- 172.21.15.43:0/3401787896 shutdown_connections 2023-08-09T15:08:42.271 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:42.270+0000 7f394c95c640 1 -- 172.21.15.43:0/3401787896 wait complete. 2023-08-09T15:08:42.272 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:42.271+0000 7f394c95c640 1 Processor -- start 2023-08-09T15:08:42.272 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:42.272+0000 7f394c95c640 1 -- start start 2023-08-09T15:08:42.273 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:42.273+0000 7f394c95c640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f39480c90a0 0x7f39480c4ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T15:08:42.273 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:42.273+0000 7f394c95c640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f39480c9ee0 0x7f39480c4150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T15:08:42.273 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:42.273+0000 7f394c95c640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f3948157ea0 0x7f39480c6e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T15:08:42.273 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:42.273+0000 7f394c95c640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3948155da0 con 0x7f39480c90a0 2023-08-09T15:08:42.273 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:42.273+0000 7f394c95c640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f39481efef0 con 0x7f3948157ea0 2023-08-09T15:08:42.274 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:42.273+0000 7f394c95c640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f39481f01d0 con 0x7f39480c9ee0 2023-08-09T15:08:42.274 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:42.273+0000 7f3946d9d640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f39480c90a0 0x7f39480c4ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T15:08:42.274 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:42.273+0000 7f3946d9d640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f39480c90a0 0x7f39480c4ea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:39430/0 (socket says 172.21.15.43:39430) 2023-08-09T15:08:42.274 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:42.273+0000 7f3946d9d640 1 -- 172.21.15.43:0/1947782549 learned_addr learned my addr 172.21.15.43:0/1947782549 (peer_addr_for_me v2:172.21.15.43:0/0) 2023-08-09T15:08:42.275 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:42.273+0000 7f3946d9d640 1 -- 172.21.15.43:0/1947782549 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f39480c9ee0 msgr2=0x7f39480c4150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T15:08:42.275 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:42.273+0000 7f3946d9d640 1 --2- 172.21.15.43:0/1947782549 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f39480c9ee0 0x7f39480c4150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T15:08:42.275 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:42.273+0000 7f3946d9d640 1 -- 172.21.15.43:0/1947782549 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f3948157ea0 msgr2=0x7f39480c6e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T15:08:42.276 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:42.273+0000 7f3946d9d640 1 --2- 172.21.15.43:0/1947782549 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f3948157ea0 0x7f39480c6e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T15:08:42.276 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:42.273+0000 7f3946d9d640 1 -- 172.21.15.43:0/1947782549 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3940009b30 con 0x7f39480c90a0 2023-08-09T15:08:42.276 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:42.274+0000 7f3946d9d640 1 --2- 172.21.15.43:0/1947782549 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f39480c90a0 0x7f39480c4ea0 secure :-1 s=READY pgs=2198 cs=0 l=1 rev1=1 crypto rx=0x7f393800b3f0 tx=0x7f393800b8b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-09T15:08:42.276 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:42.274+0000 7f393f7fe640 1 -- 172.21.15.43:0/1947782549 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3938011830 con 0x7f39480c90a0 2023-08-09T15:08:42.277 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:42.274+0000 7f393f7fe640 1 -- 172.21.15.43:0/1947782549 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3938011990 con 0x7f39480c90a0 2023-08-09T15:08:42.277 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:42.274+0000 7f393f7fe640 1 -- 172.21.15.43:0/1947782549 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3938011d40 con 0x7f39480c90a0 2023-08-09T15:08:42.277 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:42.274+0000 7f394c95c640 1 -- 172.21.15.43:0/1947782549 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f39481f04b0 con 0x7f39480c90a0 2023-08-09T15:08:42.277 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:42.274+0000 7f394c95c640 1 -- 172.21.15.43:0/1947782549 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f39480c53e0 con 0x7f39480c90a0 2023-08-09T15:08:42.278 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:42.275+0000 7f394c95c640 1 -- 172.21.15.43:0/1947782549 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f39481f30b0 con 0x7f39480c90a0 2023-08-09T15:08:42.279 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:42.276+0000 7f393f7fe640 1 -- 172.21.15.43:0/1947782549 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 106) v1 ==== 88443+0+0 (secure 0 0 0) 0x7f393801a400 con 0x7f39480c90a0 2023-08-09T15:08:42.279 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:42.277+0000 7f393f7fe640 1 --2- 172.21.15.43:0/1947782549 >> [v2:172.21.15.203:6832/3082732769,v1:172.21.15.203:6833/3082732769] conn(0x7f392806a170 0x7f392806c630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T15:08:42.279 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:42.277+0000 7f393f7fe640 1 -- 172.21.15.43:0/1947782549 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(201..201 src has 1..201) v4 ==== 5574+0+0 (secure 0 0 0) 0x7f3938088340 con 0x7f39480c90a0 2023-08-09T15:08:42.280 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:42.278+0000 7f393ffff640 1 --2- 172.21.15.43:0/1947782549 >> [v2:172.21.15.203:6832/3082732769,v1:172.21.15.203:6833/3082732769] conn(0x7f392806a170 0x7f392806c630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T15:08:42.280 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:42.279+0000 7f393ffff640 1 --2- 172.21.15.43:0/1947782549 >> [v2:172.21.15.203:6832/3082732769,v1:172.21.15.203:6833/3082732769] conn(0x7f392806a170 0x7f392806c630 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f39480c2980 tx=0x7f39300079c0 comp rx=0 tx=0).ready entity=mgr.8500 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-09T15:08:42.282 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:42.282+0000 7f393f7fe640 1 -- 172.21.15.43:0/1947782549 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178066 (secure 0 0 0) 0x7f3938053af0 con 0x7f39480c90a0 2023-08-09T15:08:42.475 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:42.475+0000 7f394c95c640 1 -- 172.21.15.43:0/1947782549 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"} v 0) v1 -- 0x7f394804e390 con 0x7f39480c90a0 2023-08-09T15:08:42.476 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:42.476+0000 7f393f7fe640 1 -- 172.21.15.43:0/1947782549 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"}]=0 v201) v1 ==== 92+0+10 (secure 0 0 0) 0x7f3938057110 con 0x7f39480c90a0 2023-08-09T15:08:42.477 INFO:teuthology.orchestra.run.smithi043.stdout:pg_num: 8 2023-08-09T15:08:42.479 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:42.478+0000 7f394c95c640 1 -- 172.21.15.43:0/1947782549 >> [v2:172.21.15.203:6832/3082732769,v1:172.21.15.203:6833/3082732769] conn(0x7f392806a170 msgr2=0x7f392806c630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T15:08:42.479 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:42.478+0000 7f394c95c640 1 --2- 172.21.15.43:0/1947782549 >> [v2:172.21.15.203:6832/3082732769,v1:172.21.15.203:6833/3082732769] conn(0x7f392806a170 0x7f392806c630 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f39480c2980 tx=0x7f39300079c0 comp rx=0 tx=0).stop 2023-08-09T15:08:42.480 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:42.478+0000 7f394c95c640 1 -- 172.21.15.43:0/1947782549 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f39480c90a0 msgr2=0x7f39480c4ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T15:08:42.480 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:42.479+0000 7f394c95c640 1 --2- 172.21.15.43:0/1947782549 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f39480c90a0 0x7f39480c4ea0 secure :-1 s=READY pgs=2198 cs=0 l=1 rev1=1 crypto rx=0x7f393800b3f0 tx=0x7f393800b8b0 comp rx=0 tx=0).stop 2023-08-09T15:08:42.480 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:42.479+0000 7f394c95c640 1 -- 172.21.15.43:0/1947782549 shutdown_connections 2023-08-09T15:08:42.480 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:42.479+0000 7f394c95c640 1 --2- 172.21.15.43:0/1947782549 >> [v2:172.21.15.203:6832/3082732769,v1:172.21.15.203:6833/3082732769] conn(0x7f392806a170 0x7f392806c630 unknown :-1 s=CLOSED pgs=54 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T15:08:42.480 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:42.479+0000 7f394c95c640 1 --2- 172.21.15.43:0/1947782549 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f3948157ea0 0x7f39480c6e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T15:08:42.481 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:42.479+0000 7f394c95c640 1 --2- 172.21.15.43:0/1947782549 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f39480c9ee0 0x7f39480c4150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T15:08:42.481 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:42.479+0000 7f394c95c640 1 --2- 172.21.15.43:0/1947782549 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f39480c90a0 0x7f39480c4ea0 unknown :-1 s=CLOSED pgs=2198 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T15:08:42.481 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:42.479+0000 7f394c95c640 1 -- 172.21.15.43:0/1947782549 >> 172.21.15.43:0/1947782549 conn(0x7f39480cddd0 msgr2=0x7f39480c0fb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-09T15:08:42.481 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:42.479+0000 7f394c95c640 1 -- 172.21.15.43:0/1947782549 shutdown_connections 2023-08-09T15:08:42.481 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:42.479+0000 7f394c95c640 1 -- 172.21.15.43:0/1947782549 wait complete. 2023-08-09T15:08:42.497 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-08-09T15:08:42.728 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:42.727+0000 7f6c34576640 1 Processor -- start 2023-08-09T15:08:42.728 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:42.728+0000 7f6c34576640 1 -- start start 2023-08-09T15:08:42.729 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:42.729+0000 7f6c34576640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f6c2c15c550 0x7f6c2c0cd6d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T15:08:42.730 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:42.729+0000 7f6c34576640 1 -- --> v1:172.21.15.43:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6c2c0b7bd0 con 0x7f6c2c0b45f0 2023-08-09T15:08:42.730 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:42.729+0000 7f6c34576640 1 -- --> v1:172.21.15.203:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6c2c0d1c00 con 0x7f6c2c0d04b0 2023-08-09T15:08:42.730 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:42.729+0000 7f6c34576640 1 -- --> v2:172.21.15.43:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6c2c15c930 con 0x7f6c2c15c550 2023-08-09T15:08:42.730 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:42.730+0000 7f6c32312640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f6c2c15c550 0x7f6c2c0cd6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T15:08:42.731 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:42.730+0000 7f6c32312640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f6c2c15c550 0x7f6c2c0cd6d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:39436/0 (socket says 172.21.15.43:39436) 2023-08-09T15:08:42.731 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:42.730+0000 7f6c32312640 1 -- 172.21.15.43:0/609659684 learned_addr learned my addr 172.21.15.43:0/609659684 (peer_addr_for_me v2:172.21.15.43:0/0) 2023-08-09T15:08:42.731 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:42.730+0000 7f6c32312640 1 -- 172.21.15.43:0/609659684 >> v1:172.21.15.203:6789/0 conn(0x7f6c2c0d04b0 legacy=0x7f6c2c0b3ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-09T15:08:42.731 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:42.730+0000 7f6c32312640 1 -- 172.21.15.43:0/609659684 >> v1:172.21.15.43:6789/0 conn(0x7f6c2c0b45f0 legacy=0x7f6c2c0b59d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T15:08:42.731 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:42.730+0000 7f6c32312640 1 -- 172.21.15.43:0/609659684 --> v2:172.21.15.43:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6c2c0b54f0 con 0x7f6c2c15c550 2023-08-09T15:08:42.732 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:42.732+0000 7f6c32312640 1 --2- 172.21.15.43:0/609659684 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f6c2c15c550 0x7f6c2c0cd6d0 secure :-1 s=READY pgs=2200 cs=0 l=1 rev1=1 crypto rx=0x7f6c1c009870 tx=0x7f6c1c02b910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=be95b1c1b13e78c3 server_cookie=0 in_seq=0 out_seq=0 2023-08-09T15:08:42.732 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:42.732+0000 7f6c31310640 1 -- 172.21.15.43:0/609659684 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6c1c031870 con 0x7f6c2c15c550 2023-08-09T15:08:42.732 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:42.732+0000 7f6c31310640 1 -- 172.21.15.43:0/609659684 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6c1c0319d0 con 0x7f6c2c15c550 2023-08-09T15:08:42.733 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:42.732+0000 7f6c31310640 1 -- 172.21.15.43:0/609659684 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6c1c031d60 con 0x7f6c2c15c550 2023-08-09T15:08:42.733 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:42.732+0000 7f6c34576640 1 -- 172.21.15.43:0/609659684 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f6c2c15c550 msgr2=0x7f6c2c0cd6d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T15:08:42.734 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:42.732+0000 7f6c34576640 1 --2- 172.21.15.43:0/609659684 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f6c2c15c550 0x7f6c2c0cd6d0 secure :-1 s=READY pgs=2200 cs=0 l=1 rev1=1 crypto rx=0x7f6c1c009870 tx=0x7f6c1c02b910 comp rx=0 tx=0).stop 2023-08-09T15:08:42.734 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:42.733+0000 7f6c34576640 1 -- 172.21.15.43:0/609659684 shutdown_connections 2023-08-09T15:08:42.734 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:42.733+0000 7f6c34576640 1 --2- 172.21.15.43:0/609659684 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f6c2c15c550 0x7f6c2c0cd6d0 unknown :-1 s=CLOSED pgs=2200 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T15:08:42.734 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:42.733+0000 7f6c34576640 1 -- 172.21.15.43:0/609659684 >> 172.21.15.43:0/609659684 conn(0x7f6c2c071520 msgr2=0x7f6c2c0ceee0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-09T15:08:42.734 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:42.733+0000 7f6c34576640 1 -- 172.21.15.43:0/609659684 shutdown_connections 2023-08-09T15:08:42.734 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:42.733+0000 7f6c34576640 1 -- 172.21.15.43:0/609659684 wait complete. 2023-08-09T15:08:42.735 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:42.734+0000 7f6c34576640 1 Processor -- start 2023-08-09T15:08:42.735 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:42.735+0000 7f6c34576640 1 -- start start 2023-08-09T15:08:42.736 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:42.736+0000 7f6c34576640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f6c2c0b45f0 0x7f6c2c1f5c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T15:08:42.736 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:42.736+0000 7f6c34576640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f6c2c0d04b0 0x7f6c2c14d640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T15:08:42.736 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:42.736+0000 7f6c34576640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f6c2c15c550 0x7f6c2c0aa630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T15:08:42.736 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:42.736+0000 7f6c34576640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6c2c1505a0 con 0x7f6c2c0d04b0 2023-08-09T15:08:42.736 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:42.736+0000 7f6c34576640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6c2c14e910 con 0x7f6c2c15c550 2023-08-09T15:08:42.737 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:42.736+0000 7f6c34576640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6c2c14d150 con 0x7f6c2c0b45f0 2023-08-09T15:08:42.738 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:42.736+0000 7f6c31b11640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f6c2c0d04b0 0x7f6c2c14d640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T15:08:42.738 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:42.736+0000 7f6c31b11640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f6c2c0d04b0 0x7f6c2c14d640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:39450/0 (socket says 172.21.15.43:39450) 2023-08-09T15:08:42.738 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:42.736+0000 7f6c31b11640 1 -- 172.21.15.43:0/2486129472 learned_addr learned my addr 172.21.15.43:0/2486129472 (peer_addr_for_me v2:172.21.15.43:0/0) 2023-08-09T15:08:42.738 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:42.736+0000 7f6c31b11640 1 -- 172.21.15.43:0/2486129472 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f6c2c0b45f0 msgr2=0x7f6c2c1f5c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T15:08:42.738 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:42.736+0000 7f6c31b11640 1 --2- 172.21.15.43:0/2486129472 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f6c2c0b45f0 0x7f6c2c1f5c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T15:08:42.738 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:42.736+0000 7f6c31b11640 1 -- 172.21.15.43:0/2486129472 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f6c2c15c550 msgr2=0x7f6c2c0aa630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T15:08:42.739 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:42.736+0000 7f6c31b11640 1 --2- 172.21.15.43:0/2486129472 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f6c2c15c550 0x7f6c2c0aa630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T15:08:42.739 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:42.736+0000 7f6c31b11640 1 -- 172.21.15.43:0/2486129472 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6c1c0094d0 con 0x7f6c2c0d04b0 2023-08-09T15:08:42.739 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:42.737+0000 7f6c31b11640 1 --2- 172.21.15.43:0/2486129472 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f6c2c0d04b0 0x7f6c2c14d640 secure :-1 s=READY pgs=2201 cs=0 l=1 rev1=1 crypto rx=0x7f6c2000c7c0 tx=0x7f6c2000cc80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-09T15:08:42.739 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:42.737+0000 7f6c2affd640 1 -- 172.21.15.43:0/2486129472 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6c2000aa50 con 0x7f6c2c0d04b0 2023-08-09T15:08:42.739 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:42.737+0000 7f6c2affd640 1 -- 172.21.15.43:0/2486129472 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6c2000abb0 con 0x7f6c2c0d04b0 2023-08-09T15:08:42.739 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:42.737+0000 7f6c2affd640 1 -- 172.21.15.43:0/2486129472 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6c20012420 con 0x7f6c2c0d04b0 2023-08-09T15:08:42.739 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:42.737+0000 7f6c34576640 1 -- 172.21.15.43:0/2486129472 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6c2c0aab70 con 0x7f6c2c0d04b0 2023-08-09T15:08:42.740 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:42.737+0000 7f6c34576640 1 -- 172.21.15.43:0/2486129472 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6c2c14dfe0 con 0x7f6c2c0d04b0 2023-08-09T15:08:42.740 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:42.738+0000 7f6c34576640 1 -- 172.21.15.43:0/2486129472 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6c2c0739f0 con 0x7f6c2c0d04b0 2023-08-09T15:08:42.740 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:42.739+0000 7f6c2affd640 1 -- 172.21.15.43:0/2486129472 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 106) v1 ==== 88443+0+0 (secure 0 0 0) 0x7f6c20012580 con 0x7f6c2c0d04b0 2023-08-09T15:08:42.741 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:42.740+0000 7f6c2affd640 1 --2- 172.21.15.43:0/2486129472 >> [v2:172.21.15.203:6832/3082732769,v1:172.21.15.203:6833/3082732769] conn(0x7f6c1806a170 0x7f6c1806c630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T15:08:42.741 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:42.740+0000 7f6c2affd640 1 -- 172.21.15.43:0/2486129472 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(201..201 src has 1..201) v4 ==== 5574+0+0 (secure 0 0 0) 0x7f6c20088820 con 0x7f6c2c0d04b0 2023-08-09T15:08:42.741 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:42.741+0000 7f6c32312640 1 --2- 172.21.15.43:0/2486129472 >> [v2:172.21.15.203:6832/3082732769,v1:172.21.15.203:6833/3082732769] conn(0x7f6c1806a170 0x7f6c1806c630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T15:08:42.742 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:42.742+0000 7f6c32312640 1 --2- 172.21.15.43:0/2486129472 >> [v2:172.21.15.203:6832/3082732769,v1:172.21.15.203:6833/3082732769] conn(0x7f6c1806a170 0x7f6c1806c630 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f6c1c036000 tx=0x7f6c1c030000 comp rx=0 tx=0).ready entity=mgr.8500 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-09T15:08:42.745 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:42.745+0000 7f6c2affd640 1 -- 172.21.15.43:0/2486129472 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178066 (secure 0 0 0) 0x7f6c20053f40 con 0x7f6c2c0d04b0 2023-08-09T15:08:42.936 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:42.935+0000 7f6c34576640 1 -- 172.21.15.43:0/2486129472 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f6c2c14ea50 con 0x7f6c2c0d04b0 2023-08-09T15:08:42.936 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:42.936+0000 7f6c2affd640 1 -- 172.21.15.43:0/2486129472 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v201) v1 ==== 93+0+10 (secure 0 0 0) 0x7f6c20057560 con 0x7f6c2c0d04b0 2023-08-09T15:08:42.937 INFO:teuthology.orchestra.run.smithi043.stdout:pg_num: 1 2023-08-09T15:08:42.939 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:42.939+0000 7f6c34576640 1 -- 172.21.15.43:0/2486129472 >> [v2:172.21.15.203:6832/3082732769,v1:172.21.15.203:6833/3082732769] conn(0x7f6c1806a170 msgr2=0x7f6c1806c630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T15:08:42.939 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:42.939+0000 7f6c34576640 1 --2- 172.21.15.43:0/2486129472 >> [v2:172.21.15.203:6832/3082732769,v1:172.21.15.203:6833/3082732769] conn(0x7f6c1806a170 0x7f6c1806c630 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f6c1c036000 tx=0x7f6c1c030000 comp rx=0 tx=0).stop 2023-08-09T15:08:42.940 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:42.939+0000 7f6c34576640 1 -- 172.21.15.43:0/2486129472 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f6c2c0d04b0 msgr2=0x7f6c2c14d640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T15:08:42.940 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:42.939+0000 7f6c34576640 1 --2- 172.21.15.43:0/2486129472 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f6c2c0d04b0 0x7f6c2c14d640 secure :-1 s=READY pgs=2201 cs=0 l=1 rev1=1 crypto rx=0x7f6c2000c7c0 tx=0x7f6c2000cc80 comp rx=0 tx=0).stop 2023-08-09T15:08:42.940 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:42.939+0000 7f6c34576640 1 -- 172.21.15.43:0/2486129472 shutdown_connections 2023-08-09T15:08:42.940 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:42.939+0000 7f6c34576640 1 --2- 172.21.15.43:0/2486129472 >> [v2:172.21.15.203:6832/3082732769,v1:172.21.15.203:6833/3082732769] conn(0x7f6c1806a170 0x7f6c1806c630 unknown :-1 s=CLOSED pgs=55 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T15:08:42.940 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:42.939+0000 7f6c34576640 1 --2- 172.21.15.43:0/2486129472 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f6c2c15c550 0x7f6c2c0aa630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T15:08:42.940 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:42.939+0000 7f6c34576640 1 --2- 172.21.15.43:0/2486129472 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f6c2c0d04b0 0x7f6c2c14d640 unknown :-1 s=CLOSED pgs=2201 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T15:08:42.941 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:42.939+0000 7f6c34576640 1 --2- 172.21.15.43:0/2486129472 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f6c2c0b45f0 0x7f6c2c1f5c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T15:08:42.941 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:42.939+0000 7f6c34576640 1 -- 172.21.15.43:0/2486129472 >> 172.21.15.43:0/2486129472 conn(0x7f6c2c071520 msgr2=0x7f6c2c0d2440 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-09T15:08:42.941 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:42.939+0000 7f6c34576640 1 -- 172.21.15.43:0/2486129472 shutdown_connections 2023-08-09T15:08:42.941 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:42.939+0000 7f6c34576640 1 -- 172.21.15.43:0/2486129472 wait complete. 2023-08-09T15:08:42.959 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --format=json-pretty osd lspools 2023-08-09T15:08:43.181 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:43.180+0000 7f4f646ba640 1 Processor -- start 2023-08-09T15:08:43.181 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:43.181+0000 7f4f646ba640 1 -- start start 2023-08-09T15:08:43.182 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:43.182+0000 7f4f646ba640 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7f4f5c0b4670 0x7f4f5c0d2ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T15:08:43.183 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:43.182+0000 7f4f646ba640 1 -- --> v1:172.21.15.43:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4f5c0ce090 con 0x7f4f5c0b2b70 2023-08-09T15:08:43.183 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:43.182+0000 7f4f646ba640 1 -- --> v1:172.21.15.203:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4f5c0cd7f0 con 0x7f4f5c0b2790 2023-08-09T15:08:43.183 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:43.182+0000 7f4f646ba640 1 -- --> v2:172.21.15.203:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4f5c0b3950 con 0x7f4f5c0b4670 2023-08-09T15:08:43.185 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:43.183+0000 7f4f61c55640 1 --1- >> v1:172.21.15.43:6789/0 conn(0x7f4f5c0b2b70 0x7f4f5c0b3f60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.43:6789/0 says I am v1:172.21.15.43:59648/0 (socket says 172.21.15.43:59648) 2023-08-09T15:08:43.185 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:43.183+0000 7f4f61c55640 1 -- 172.21.15.43:0/1240175596 learned_addr learned my addr 172.21.15.43:0/1240175596 (peer_addr_for_me v1:172.21.15.43:0/0) 2023-08-09T15:08:43.185 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:43.183+0000 7f4f62c57640 1 --2- 172.21.15.43:0/1240175596 >> v2:172.21.15.203:3300/0 conn(0x7f4f5c0b4670 0x7f4f5c0d2ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T15:08:43.185 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:43.183+0000 7f4f61454640 1 -- 172.21.15.43:0/1240175596 <== mon.0 v1:172.21.15.43:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3785637876 0 0) 0x7f4f5c0ce090 con 0x7f4f5c0b2b70 2023-08-09T15:08:43.185 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:43.183+0000 7f4f61454640 1 -- 172.21.15.43:0/1240175596 --> v1:172.21.15.43:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f4f48003520 con 0x7f4f5c0b2b70 2023-08-09T15:08:43.185 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:43.184+0000 7f4f61454640 1 -- 172.21.15.43:0/1240175596 <== mon.0 v1:172.21.15.43:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 286276581 0 0) 0x7f4f48003520 con 0x7f4f5c0b2b70 2023-08-09T15:08:43.186 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:43.184+0000 7f4f61454640 1 -- 172.21.15.43:0/1240175596 >> v2:172.21.15.203:3300/0 conn(0x7f4f5c0b4670 msgr2=0x7f4f5c0d2ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-09T15:08:43.186 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:43.184+0000 7f4f61454640 1 --2- 172.21.15.43:0/1240175596 >> v2:172.21.15.203:3300/0 conn(0x7f4f5c0b4670 0x7f4f5c0d2ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T15:08:43.188 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:43.184+0000 7f4f61454640 1 -- 172.21.15.43:0/1240175596 >> v1:172.21.15.203:6789/0 conn(0x7f4f5c0b2790 legacy=0x7f4f5c0b5330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T15:08:43.188 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:43.184+0000 7f4f61454640 1 -- 172.21.15.43:0/1240175596 --> v1:172.21.15.43:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4f5c0cdad0 con 0x7f4f5c0b2b70 2023-08-09T15:08:43.188 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:43.184+0000 7f4f61454640 1 -- 172.21.15.43:0/1240175596 <== mon.0 v1:172.21.15.43:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1285150740 0 0) 0x7f4f4c001670 con 0x7f4f5c0b2b70 2023-08-09T15:08:43.188 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:43.184+0000 7f4f61454640 1 -- 172.21.15.43:0/1240175596 >> v1:172.21.15.43:6789/0 conn(0x7f4f5c0b2b70 legacy=0x7f4f5c0b3f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T15:08:43.188 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:43.184+0000 7f4f61454640 1 --2- 172.21.15.43:0/1240175596 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f4f480039e0 0x7f4f48003e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T15:08:43.189 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:43.184+0000 7f4f61454640 1 --2- 172.21.15.43:0/1240175596 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f4f480047c0 0x7f4f48004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T15:08:43.189 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:43.184+0000 7f4f61454640 1 --2- 172.21.15.43:0/1240175596 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f4f48005180 0x7f4f4800b980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T15:08:43.189 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:43.184+0000 7f4f61454640 1 -- 172.21.15.43:0/1240175596 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4f5c0b3950 con 0x7f4f480047c0 2023-08-09T15:08:43.189 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:43.184+0000 7f4f61454640 1 -- 172.21.15.43:0/1240175596 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4f4800c0b0 con 0x7f4f48005180 2023-08-09T15:08:43.189 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:43.184+0000 7f4f61454640 1 -- 172.21.15.43:0/1240175596 --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4f4800c270 con 0x7f4f480039e0 2023-08-09T15:08:43.189 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:43.184+0000 7f4f61c55640 1 --2- 172.21.15.43:0/1240175596 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f4f480047c0 0x7f4f48004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T15:08:43.189 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:43.185+0000 7f4f61c55640 1 -- 172.21.15.43:0/1240175596 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f4f480039e0 msgr2=0x7f4f48003e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-09T15:08:43.189 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:43.185+0000 7f4f61c55640 1 --2- 172.21.15.43:0/1240175596 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f4f480039e0 0x7f4f48003e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T15:08:43.190 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:43.185+0000 7f4f61c55640 1 -- 172.21.15.43:0/1240175596 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f4f48005180 msgr2=0x7f4f4800b980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-09T15:08:43.190 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:43.185+0000 7f4f61c55640 1 --2- 172.21.15.43:0/1240175596 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f4f48005180 0x7f4f4800b980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T15:08:43.190 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:43.185+0000 7f4f61c55640 1 -- 172.21.15.43:0/1240175596 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4f48003520 con 0x7f4f480047c0 2023-08-09T15:08:43.190 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:43.185+0000 7f4f62c57640 1 -- 172.21.15.43:0/1240175596 reap_dead start 2023-08-09T15:08:43.192 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:43.186+0000 7f4f61c55640 1 --2- 172.21.15.43:0/1240175596 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f4f480047c0 0x7f4f48004c40 secure :-1 s=READY pgs=2203 cs=0 l=1 rev1=1 crypto rx=0x7f4f4c00c240 tx=0x7f4f480057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2796212cc4600844 server_cookie=0 in_seq=0 out_seq=0 2023-08-09T15:08:43.193 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:43.186+0000 7f4f61454640 1 -- 172.21.15.43:0/1240175596 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4f4c005ad0 con 0x7f4f480047c0 2023-08-09T15:08:43.193 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:43.186+0000 7f4f61454640 1 -- 172.21.15.43:0/1240175596 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4f4c005c50 con 0x7f4f480047c0 2023-08-09T15:08:43.193 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:43.186+0000 7f4f61454640 1 -- 172.21.15.43:0/1240175596 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4f4c02c8b0 con 0x7f4f480047c0 2023-08-09T15:08:43.193 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:43.186+0000 7f4f646ba640 1 -- 172.21.15.43:0/1240175596 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f4f480047c0 msgr2=0x7f4f48004c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T15:08:43.193 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:43.186+0000 7f4f646ba640 1 --2- 172.21.15.43:0/1240175596 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f4f480047c0 0x7f4f48004c40 secure :-1 s=READY pgs=2203 cs=0 l=1 rev1=1 crypto rx=0x7f4f4c00c240 tx=0x7f4f480057f0 comp rx=0 tx=0).stop 2023-08-09T15:08:43.193 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:43.187+0000 7f4f646ba640 1 -- 172.21.15.43:0/1240175596 shutdown_connections 2023-08-09T15:08:43.193 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:43.187+0000 7f4f646ba640 1 --2- 172.21.15.43:0/1240175596 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f4f480047c0 0x7f4f48004c40 unknown :-1 s=CLOSED pgs=2203 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T15:08:43.194 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:43.187+0000 7f4f646ba640 1 -- 172.21.15.43:0/1240175596 >> 172.21.15.43:0/1240175596 conn(0x7f4f5c09b7b0 msgr2=0x7f4f5c0d24e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-09T15:08:43.194 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:43.187+0000 7f4f646ba640 1 -- 172.21.15.43:0/1240175596 shutdown_connections 2023-08-09T15:08:43.194 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:43.187+0000 7f4f646ba640 1 -- 172.21.15.43:0/1240175596 wait complete. 2023-08-09T15:08:43.194 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:43.188+0000 7f4f646ba640 1 Processor -- start 2023-08-09T15:08:43.194 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:43.189+0000 7f4f646ba640 1 -- start start 2023-08-09T15:08:43.194 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:43.189+0000 7f4f646ba640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f4f480047c0 0x7f4f5c0b0f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T15:08:43.194 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:43.189+0000 7f4f646ba640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f4f5c0c95b0 0x7f4f5c15e710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T15:08:43.194 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:43.189+0000 7f4f646ba640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f4f5c1f2180 0x7f4f5c1f5730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T15:08:43.195 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:43.190+0000 7f4f646ba640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4f5c0c9a90 con 0x7f4f5c0c95b0 2023-08-09T15:08:43.195 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:43.190+0000 7f4f646ba640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4f5c0c9d70 con 0x7f4f480047c0 2023-08-09T15:08:43.195 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:43.190+0000 7f4f646ba640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4f5c0ca050 con 0x7f4f5c1f2180 2023-08-09T15:08:43.195 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:43.190+0000 7f4f61c55640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f4f5c0c95b0 0x7f4f5c15e710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T15:08:43.195 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:43.190+0000 7f4f61c55640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f4f5c0c95b0 0x7f4f5c15e710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:39460/0 (socket says 172.21.15.43:39460) 2023-08-09T15:08:43.195 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:43.190+0000 7f4f61c55640 1 -- 172.21.15.43:0/3279597086 learned_addr learned my addr 172.21.15.43:0/3279597086 (peer_addr_for_me v2:172.21.15.43:0/0) 2023-08-09T15:08:43.195 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:43.190+0000 7f4f62456640 1 --2- 172.21.15.43:0/3279597086 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f4f480047c0 0x7f4f5c0b0f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T15:08:43.196 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:43.190+0000 7f4f62c57640 1 --2- 172.21.15.43:0/3279597086 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f4f5c1f2180 0x7f4f5c1f5730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T15:08:43.196 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:43.190+0000 7f4f61c55640 1 -- 172.21.15.43:0/3279597086 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f4f5c1f2180 msgr2=0x7f4f5c1f5730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T15:08:43.196 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:43.190+0000 7f4f61c55640 1 --2- 172.21.15.43:0/3279597086 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f4f5c1f2180 0x7f4f5c1f5730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T15:08:43.196 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:43.190+0000 7f4f61c55640 1 -- 172.21.15.43:0/3279597086 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f4f480047c0 msgr2=0x7f4f5c0b0f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T15:08:43.196 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:43.190+0000 7f4f61c55640 1 --2- 172.21.15.43:0/3279597086 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f4f480047c0 0x7f4f5c0b0f60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T15:08:43.196 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:43.190+0000 7f4f61c55640 1 -- 172.21.15.43:0/3279597086 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4f5c0ca330 con 0x7f4f5c0c95b0 2023-08-09T15:08:43.196 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:43.191+0000 7f4f61c55640 1 --2- 172.21.15.43:0/3279597086 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f4f5c0c95b0 0x7f4f5c15e710 secure :-1 s=READY pgs=2204 cs=0 l=1 rev1=1 crypto rx=0x7f4f4c00c240 tx=0x7f4f4c02e070 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-09T15:08:43.196 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:43.191+0000 7f4f52ffd640 1 -- 172.21.15.43:0/3279597086 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4f4c03c040 con 0x7f4f5c0c95b0 2023-08-09T15:08:43.197 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:43.191+0000 7f4f52ffd640 1 -- 172.21.15.43:0/3279597086 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4f4c02cce0 con 0x7f4f5c0c95b0 2023-08-09T15:08:43.197 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:43.191+0000 7f4f52ffd640 1 -- 172.21.15.43:0/3279597086 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4f4c0072b0 con 0x7f4f5c0c95b0 2023-08-09T15:08:43.197 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:43.191+0000 7f4f646ba640 1 -- 172.21.15.43:0/3279597086 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4f5c0ca610 con 0x7f4f5c0c95b0 2023-08-09T15:08:43.197 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:43.191+0000 7f4f646ba640 1 -- 172.21.15.43:0/3279597086 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4f5c15efb0 con 0x7f4f5c0c95b0 2023-08-09T15:08:43.197 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:43.192+0000 7f4f646ba640 1 -- 172.21.15.43:0/3279597086 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4f5c0d05a0 con 0x7f4f5c0c95b0 2023-08-09T15:08:43.198 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:43.193+0000 7f4f52ffd640 1 -- 172.21.15.43:0/3279597086 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 106) v1 ==== 88443+0+0 (secure 0 0 0) 0x7f4f4c032070 con 0x7f4f5c0c95b0 2023-08-09T15:08:43.198 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:43.194+0000 7f4f52ffd640 1 --2- 172.21.15.43:0/3279597086 >> [v2:172.21.15.203:6832/3082732769,v1:172.21.15.203:6833/3082732769] conn(0x7f4f4006a170 0x7f4f4006c630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T15:08:43.198 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:43.194+0000 7f4f52ffd640 1 -- 172.21.15.43:0/3279597086 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(201..201 src has 1..201) v4 ==== 5574+0+0 (secure 0 0 0) 0x7f4f4c0a97f0 con 0x7f4f5c0c95b0 2023-08-09T15:08:43.198 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:43.195+0000 7f4f62456640 1 --2- 172.21.15.43:0/3279597086 >> [v2:172.21.15.203:6832/3082732769,v1:172.21.15.203:6833/3082732769] conn(0x7f4f4006a170 0x7f4f4006c630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T15:08:43.199 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:43.196+0000 7f4f62456640 1 --2- 172.21.15.43:0/3279597086 >> [v2:172.21.15.203:6832/3082732769,v1:172.21.15.203:6833/3082732769] conn(0x7f4f4006a170 0x7f4f4006c630 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f4f58007fd0 tx=0x7f4f58009040 comp rx=0 tx=0).ready entity=mgr.8500 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-09T15:08:43.199 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:43.199+0000 7f4f52ffd640 1 -- 172.21.15.43:0/3279597086 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178066 (secure 0 0 0) 0x7f4f4c0750e0 con 0x7f4f5c0c95b0 2023-08-09T15:08:43.392 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:43.391+0000 7f4f646ba640 1 -- 172.21.15.43:0/3279597086 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) v1 -- 0x7f4f5c0739f0 con 0x7f4f5c0c95b0 2023-08-09T15:08:43.392 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:43.392+0000 7f4f52ffd640 1 -- 172.21.15.43:0/3279597086 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v201) v1 ==== 84+0+126 (secure 0 0 0) 0x7f4f4c046090 con 0x7f4f5c0c95b0 2023-08-09T15:08:43.393 INFO:teuthology.orchestra.run.smithi043.stdout: 2023-08-09T15:08:43.393 INFO:teuthology.orchestra.run.smithi043.stdout:[ 2023-08-09T15:08:43.393 INFO:teuthology.orchestra.run.smithi043.stdout: { 2023-08-09T15:08:43.393 INFO:teuthology.orchestra.run.smithi043.stdout: "poolnum": 1, 2023-08-09T15:08:43.393 INFO:teuthology.orchestra.run.smithi043.stdout: "poolname": "rbd" 2023-08-09T15:08:43.393 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2023-08-09T15:08:43.394 INFO:teuthology.orchestra.run.smithi043.stdout: { 2023-08-09T15:08:43.394 INFO:teuthology.orchestra.run.smithi043.stdout: "poolnum": 2, 2023-08-09T15:08:43.394 INFO:teuthology.orchestra.run.smithi043.stdout: "poolname": ".mgr" 2023-08-09T15:08:43.394 INFO:teuthology.orchestra.run.smithi043.stdout: } 2023-08-09T15:08:43.394 INFO:teuthology.orchestra.run.smithi043.stdout:] 2023-08-09T15:08:43.396 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:43.395+0000 7f4f646ba640 1 -- 172.21.15.43:0/3279597086 >> [v2:172.21.15.203:6832/3082732769,v1:172.21.15.203:6833/3082732769] conn(0x7f4f4006a170 msgr2=0x7f4f4006c630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T15:08:43.396 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:43.395+0000 7f4f646ba640 1 --2- 172.21.15.43:0/3279597086 >> [v2:172.21.15.203:6832/3082732769,v1:172.21.15.203:6833/3082732769] conn(0x7f4f4006a170 0x7f4f4006c630 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f4f58007fd0 tx=0x7f4f58009040 comp rx=0 tx=0).stop 2023-08-09T15:08:43.396 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:43.395+0000 7f4f646ba640 1 -- 172.21.15.43:0/3279597086 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f4f5c0c95b0 msgr2=0x7f4f5c15e710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T15:08:43.396 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:43.395+0000 7f4f646ba640 1 --2- 172.21.15.43:0/3279597086 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f4f5c0c95b0 0x7f4f5c15e710 secure :-1 s=READY pgs=2204 cs=0 l=1 rev1=1 crypto rx=0x7f4f4c00c240 tx=0x7f4f4c02e070 comp rx=0 tx=0).stop 2023-08-09T15:08:43.396 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:43.395+0000 7f4f646ba640 1 -- 172.21.15.43:0/3279597086 shutdown_connections 2023-08-09T15:08:43.397 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:43.395+0000 7f4f646ba640 1 --2- 172.21.15.43:0/3279597086 >> [v2:172.21.15.203:6832/3082732769,v1:172.21.15.203:6833/3082732769] conn(0x7f4f4006a170 0x7f4f4006c630 unknown :-1 s=CLOSED pgs=56 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T15:08:43.397 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:43.395+0000 7f4f646ba640 1 --2- 172.21.15.43:0/3279597086 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f4f5c1f2180 0x7f4f5c1f5730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T15:08:43.397 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:43.395+0000 7f4f646ba640 1 --2- 172.21.15.43:0/3279597086 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f4f5c0c95b0 0x7f4f5c15e710 unknown :-1 s=CLOSED pgs=2204 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T15:08:43.397 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:43.395+0000 7f4f646ba640 1 --2- 172.21.15.43:0/3279597086 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f4f480047c0 0x7f4f5c0b0f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T15:08:43.397 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:43.395+0000 7f4f646ba640 1 -- 172.21.15.43:0/3279597086 >> 172.21.15.43:0/3279597086 conn(0x7f4f5c09b7b0 msgr2=0x7f4f5c15fb00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-09T15:08:43.397 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:43.395+0000 7f4f646ba640 1 -- 172.21.15.43:0/3279597086 shutdown_connections 2023-08-09T15:08:43.397 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:43.395+0000 7f4f646ba640 1 -- 172.21.15.43:0/3279597086 wait complete. 2023-08-09T15:08:43.414 DEBUG:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2023-08-09T15:08:43.415 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_metadata --pg_num_min 64 2023-08-09T15:08:43.645 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:43.644+0000 7fec3550e640 1 Processor -- start 2023-08-09T15:08:43.645 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:43.644+0000 7fec3550e640 1 -- start start 2023-08-09T15:08:43.646 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:43.646+0000 7fec3550e640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fec300ca370 0x7fec3015b2a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T15:08:43.646 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:43.646+0000 7fec3550e640 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7fec300ce4c0 0x7fec300ce950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T15:08:43.647 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:43.646+0000 7fec3550e640 1 -- --> v1:172.21.15.43:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fec3015ae10 con 0x7fec3006fb70 2023-08-09T15:08:43.647 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:43.646+0000 7fec3550e640 1 -- --> v2:172.21.15.203:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fec300cee90 con 0x7fec300ce4c0 2023-08-09T15:08:43.647 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:43.646+0000 7fec3550e640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fec30154e70 con 0x7fec300ca370 2023-08-09T15:08:43.647 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:43.646+0000 7fec2effd640 1 --1- >> v1:172.21.15.43:6789/0 conn(0x7fec3006fb70 0x7fec300caab0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.43:6789/0 says I am v1:172.21.15.43:59650/0 (socket says 172.21.15.43:59650) 2023-08-09T15:08:43.647 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:43.646+0000 7fec2effd640 1 -- 172.21.15.43:0/2119772417 learned_addr learned my addr 172.21.15.43:0/2119772417 (peer_addr_for_me v1:172.21.15.43:0/0) 2023-08-09T15:08:43.649 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:43.647+0000 7fec2f7fe640 1 --2- 172.21.15.43:0/2119772417 >> v2:172.21.15.203:3300/0 conn(0x7fec300ce4c0 0x7fec300ce950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T15:08:43.649 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:43.647+0000 7fec2e7fc640 1 --2- 172.21.15.43:0/2119772417 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fec300ca370 0x7fec3015b2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T15:08:43.649 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:43.647+0000 7fec2dffb640 1 -- 172.21.15.43:0/2119772417 <== mon.0 v1:172.21.15.43:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1101018983 0 0) 0x7fec3015ae10 con 0x7fec3006fb70 2023-08-09T15:08:43.649 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:43.647+0000 7fec2dffb640 1 -- 172.21.15.43:0/2119772417 --> v1:172.21.15.43:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fec18003520 con 0x7fec3006fb70 2023-08-09T15:08:43.650 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:43.647+0000 7fec2dffb640 1 -- 172.21.15.43:0/2119772417 <== mon.0 v1:172.21.15.43:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2199778213 0 0) 0x7fec18003520 con 0x7fec3006fb70 2023-08-09T15:08:43.650 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:43.647+0000 7fec2dffb640 1 -- 172.21.15.43:0/2119772417 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fec300ca370 msgr2=0x7fec3015b2a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-09T15:08:43.650 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:43.648+0000 7fec2dffb640 1 --2- 172.21.15.43:0/2119772417 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fec300ca370 0x7fec3015b2a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T15:08:43.651 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:43.648+0000 7fec2dffb640 1 -- 172.21.15.43:0/2119772417 >> v2:172.21.15.203:3300/0 conn(0x7fec300ce4c0 msgr2=0x7fec300ce950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-09T15:08:43.651 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:43.648+0000 7fec2dffb640 1 --2- 172.21.15.43:0/2119772417 >> v2:172.21.15.203:3300/0 conn(0x7fec300ce4c0 0x7fec300ce950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T15:08:43.652 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:43.648+0000 7fec2dffb640 1 -- 172.21.15.43:0/2119772417 --> v1:172.21.15.43:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fec3006ff50 con 0x7fec3006fb70 2023-08-09T15:08:43.652 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:43.648+0000 7fec2dffb640 1 -- 172.21.15.43:0/2119772417 <== mon.0 v1:172.21.15.43:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1285150740 0 0) 0x7fec20001670 con 0x7fec3006fb70 2023-08-09T15:08:43.652 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:43.648+0000 7fec2dffb640 1 -- 172.21.15.43:0/2119772417 >> v1:172.21.15.43:6789/0 conn(0x7fec3006fb70 legacy=0x7fec300caab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T15:08:43.652 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:43.648+0000 7fec2dffb640 1 --2- 172.21.15.43:0/2119772417 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fec180039e0 0x7fec18003e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T15:08:43.652 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:43.648+0000 7fec2dffb640 1 --2- 172.21.15.43:0/2119772417 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fec180047c0 0x7fec18004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T15:08:43.652 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:43.648+0000 7fec2dffb640 1 --2- 172.21.15.43:0/2119772417 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fec300ca370 0x7fec18005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T15:08:43.653 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:43.648+0000 7fec2dffb640 1 -- 172.21.15.43:0/2119772417 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fec300cee90 con 0x7fec180047c0 2023-08-09T15:08:43.653 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:43.648+0000 7fec2dffb640 1 -- 172.21.15.43:0/2119772417 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fec30154e70 con 0x7fec180039e0 2023-08-09T15:08:43.653 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:43.648+0000 7fec2dffb640 1 -- 172.21.15.43:0/2119772417 --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fec1800b980 con 0x7fec300ca370 2023-08-09T15:08:43.653 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:43.648+0000 7fec2e7fc640 1 --2- 172.21.15.43:0/2119772417 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fec180047c0 0x7fec18004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T15:08:43.653 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:43.648+0000 7fec2effd640 1 --2- 172.21.15.43:0/2119772417 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fec180039e0 0x7fec18003e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T15:08:43.654 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:43.648+0000 7fec2f7fe640 1 --2- 172.21.15.43:0/2119772417 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fec300ca370 0x7fec18005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T15:08:43.654 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:43.648+0000 7fec2e7fc640 1 -- 172.21.15.43:0/2119772417 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fec300ca370 msgr2=0x7fec18005180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-09T15:08:43.654 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:43.648+0000 7fec2e7fc640 1 --2- 172.21.15.43:0/2119772417 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fec300ca370 0x7fec18005180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T15:08:43.654 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:43.648+0000 7fec2e7fc640 1 -- 172.21.15.43:0/2119772417 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fec180039e0 msgr2=0x7fec18003e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-09T15:08:43.655 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:43.648+0000 7fec2e7fc640 1 --2- 172.21.15.43:0/2119772417 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fec180039e0 0x7fec18003e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T15:08:43.655 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:43.649+0000 7fec2e7fc640 1 -- 172.21.15.43:0/2119772417 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fec18003520 con 0x7fec180047c0 2023-08-09T15:08:43.657 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:43.650+0000 7fec2e7fc640 1 --2- 172.21.15.43:0/2119772417 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fec180047c0 0x7fec18004c40 secure :-1 s=READY pgs=2206 cs=0 l=1 rev1=1 crypto rx=0x7fec24009af0 tx=0x7fec180057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=27a99d4a4285015f server_cookie=0 in_seq=0 out_seq=0 2023-08-09T15:08:43.657 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:43.650+0000 7fec2dffb640 1 -- 172.21.15.43:0/2119772417 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fec2402c6b0 con 0x7fec180047c0 2023-08-09T15:08:43.658 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:43.650+0000 7fec2dffb640 1 -- 172.21.15.43:0/2119772417 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fec2402c810 con 0x7fec180047c0 2023-08-09T15:08:43.658 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:43.650+0000 7fec2dffb640 1 -- 172.21.15.43:0/2119772417 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fec2402cba0 con 0x7fec180047c0 2023-08-09T15:08:43.658 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:43.650+0000 7fec3550e640 1 -- 172.21.15.43:0/2119772417 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fec180047c0 msgr2=0x7fec18004c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T15:08:43.658 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:43.650+0000 7fec3550e640 1 --2- 172.21.15.43:0/2119772417 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fec180047c0 0x7fec18004c40 secure :-1 s=READY pgs=2206 cs=0 l=1 rev1=1 crypto rx=0x7fec24009af0 tx=0x7fec180057f0 comp rx=0 tx=0).stop 2023-08-09T15:08:43.658 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:43.650+0000 7fec2f7fe640 1 -- 172.21.15.43:0/2119772417 reap_dead start 2023-08-09T15:08:43.658 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:43.651+0000 7fec3550e640 1 -- 172.21.15.43:0/2119772417 shutdown_connections 2023-08-09T15:08:43.658 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:43.651+0000 7fec3550e640 1 -- 172.21.15.43:0/2119772417 >> 172.21.15.43:0/2119772417 conn(0x7fec3006b990 msgr2=0x7fec3006bda0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-09T15:08:43.659 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:43.651+0000 7fec3550e640 1 -- 172.21.15.43:0/2119772417 shutdown_connections 2023-08-09T15:08:43.659 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:43.651+0000 7fec3550e640 1 -- 172.21.15.43:0/2119772417 wait complete. 2023-08-09T15:08:43.659 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:43.652+0000 7fec3550e640 1 Processor -- start 2023-08-09T15:08:43.659 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:43.653+0000 7fec3550e640 1 -- start start 2023-08-09T15:08:43.659 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:43.654+0000 7fec3550e640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fec180047c0 0x7fec300c68e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T15:08:43.659 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:43.654+0000 7fec3550e640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fec3015ccc0 0x7fec300afe10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T15:08:43.659 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:43.654+0000 7fec3550e640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fec30159de0 0x7fec30153490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T15:08:43.659 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:43.654+0000 7fec3550e640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fec300c5520 con 0x7fec180047c0 2023-08-09T15:08:43.660 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:43.654+0000 7fec3550e640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fec300c6e20 con 0x7fec30159de0 2023-08-09T15:08:43.660 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:43.654+0000 7fec3550e640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fec300cd500 con 0x7fec3015ccc0 2023-08-09T15:08:43.660 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:43.654+0000 7fec2effd640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fec180047c0 0x7fec300c68e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T15:08:43.660 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:43.654+0000 7fec2effd640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fec180047c0 0x7fec300c68e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:39468/0 (socket says 172.21.15.43:39468) 2023-08-09T15:08:43.660 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:43.654+0000 7fec2effd640 1 -- 172.21.15.43:0/1757148680 learned_addr learned my addr 172.21.15.43:0/1757148680 (peer_addr_for_me v2:172.21.15.43:0/0) 2023-08-09T15:08:43.660 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:43.654+0000 7fec2e7fc640 1 --2- 172.21.15.43:0/1757148680 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fec3015ccc0 0x7fec300afe10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T15:08:43.660 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:43.654+0000 7fec2f7fe640 1 --2- 172.21.15.43:0/1757148680 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fec30159de0 0x7fec30153490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T15:08:43.661 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:43.654+0000 7fec2effd640 1 -- 172.21.15.43:0/1757148680 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fec3015ccc0 msgr2=0x7fec300afe10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T15:08:43.661 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:43.654+0000 7fec2effd640 1 --2- 172.21.15.43:0/1757148680 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fec3015ccc0 0x7fec300afe10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T15:08:43.661 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:43.654+0000 7fec2effd640 1 -- 172.21.15.43:0/1757148680 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fec30159de0 msgr2=0x7fec30153490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T15:08:43.661 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:43.654+0000 7fec2effd640 1 --2- 172.21.15.43:0/1757148680 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fec30159de0 0x7fec30153490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T15:08:43.662 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:43.654+0000 7fec2effd640 1 -- 172.21.15.43:0/1757148680 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fec30159a30 con 0x7fec180047c0 2023-08-09T15:08:43.662 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:43.654+0000 7fec2e7fc640 1 --2- 172.21.15.43:0/1757148680 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fec3015ccc0 0x7fec300afe10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-08-09T15:08:43.662 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:43.655+0000 7fec2effd640 1 --2- 172.21.15.43:0/1757148680 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fec180047c0 0x7fec300c68e0 secure :-1 s=READY pgs=2207 cs=0 l=1 rev1=1 crypto rx=0x7fec2000e780 tx=0x7fec2000ec40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-09T15:08:43.662 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:43.655+0000 7fec177fe640 1 -- 172.21.15.43:0/1757148680 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fec2000ee30 con 0x7fec180047c0 2023-08-09T15:08:43.662 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:43.655+0000 7fec177fe640 1 -- 172.21.15.43:0/1757148680 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fec2000c8d0 con 0x7fec180047c0 2023-08-09T15:08:43.662 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:43.655+0000 7fec177fe640 1 -- 172.21.15.43:0/1757148680 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fec2000cc80 con 0x7fec180047c0 2023-08-09T15:08:43.662 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:43.655+0000 7fec3550e640 1 -- 172.21.15.43:0/1757148680 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fec300cd710 con 0x7fec180047c0 2023-08-09T15:08:43.663 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:43.655+0000 7fec3550e640 1 -- 172.21.15.43:0/1757148680 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fec30159700 con 0x7fec180047c0 2023-08-09T15:08:43.663 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:43.656+0000 7fec3550e640 1 -- 172.21.15.43:0/1757148680 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fec300739f0 con 0x7fec180047c0 2023-08-09T15:08:43.664 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:43.657+0000 7fec177fe640 1 -- 172.21.15.43:0/1757148680 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 106) v1 ==== 88443+0+0 (secure 0 0 0) 0x7fec20013420 con 0x7fec180047c0 2023-08-09T15:08:43.664 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:43.659+0000 7fec177fe640 1 --2- 172.21.15.43:0/1757148680 >> [v2:172.21.15.203:6832/3082732769,v1:172.21.15.203:6833/3082732769] conn(0x7fec1c06a1a0 0x7fec1c06c660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T15:08:43.665 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:43.660+0000 7fec177fe640 1 -- 172.21.15.43:0/1757148680 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(201..201 src has 1..201) v4 ==== 5574+0+0 (secure 0 0 0) 0x7fec20088140 con 0x7fec180047c0 2023-08-09T15:08:43.665 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:43.660+0000 7fec2e7fc640 1 --2- 172.21.15.43:0/1757148680 >> [v2:172.21.15.203:6832/3082732769,v1:172.21.15.203:6833/3082732769] conn(0x7fec1c06a1a0 0x7fec1c06c660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T15:08:43.665 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:43.661+0000 7fec2e7fc640 1 --2- 172.21.15.43:0/1757148680 >> [v2:172.21.15.203:6832/3082732769,v1:172.21.15.203:6833/3082732769] conn(0x7fec1c06a1a0 0x7fec1c06c660 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7fec24036000 tx=0x7fec24030000 comp rx=0 tx=0).ready entity=mgr.8500 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-09T15:08:43.665 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:43.664+0000 7fec177fe640 1 -- 172.21.15.43:0/1757148680 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178066 (secure 0 0 0) 0x7fec200538f0 con 0x7fec180047c0 2023-08-09T15:08:43.860 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:43.860+0000 7fec3550e640 1 -- 172.21.15.43:0/1757148680 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64} v 0) v1 -- 0x7fec300cb8d0 con 0x7fec180047c0 2023-08-09T15:08:44.056 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:44.056+0000 7fec177fe640 1 -- 172.21.15.43:0/1757148680 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]=0 pool 'cephfs_metadata' created v202) v1 ==== 138+0+0 (secure 0 0 0) 0x7fec20056f10 con 0x7fec180047c0 2023-08-09T15:08:44.057 INFO:teuthology.orchestra.run.smithi043.stderr:pool 'cephfs_metadata' created 2023-08-09T15:08:44.063 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:44.062+0000 7fec3550e640 1 -- 172.21.15.43:0/1757148680 >> [v2:172.21.15.203:6832/3082732769,v1:172.21.15.203:6833/3082732769] conn(0x7fec1c06a1a0 msgr2=0x7fec1c06c660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T15:08:44.063 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:44.062+0000 7fec3550e640 1 --2- 172.21.15.43:0/1757148680 >> [v2:172.21.15.203:6832/3082732769,v1:172.21.15.203:6833/3082732769] conn(0x7fec1c06a1a0 0x7fec1c06c660 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7fec24036000 tx=0x7fec24030000 comp rx=0 tx=0).stop 2023-08-09T15:08:44.064 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:44.062+0000 7fec3550e640 1 -- 172.21.15.43:0/1757148680 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fec180047c0 msgr2=0x7fec300c68e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T15:08:44.064 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:44.062+0000 7fec3550e640 1 --2- 172.21.15.43:0/1757148680 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fec180047c0 0x7fec300c68e0 secure :-1 s=READY pgs=2207 cs=0 l=1 rev1=1 crypto rx=0x7fec2000e780 tx=0x7fec2000ec40 comp rx=0 tx=0).stop 2023-08-09T15:08:44.065 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:44.063+0000 7fec3550e640 1 -- 172.21.15.43:0/1757148680 shutdown_connections 2023-08-09T15:08:44.065 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:44.063+0000 7fec3550e640 1 --2- 172.21.15.43:0/1757148680 >> [v2:172.21.15.203:6832/3082732769,v1:172.21.15.203:6833/3082732769] conn(0x7fec1c06a1a0 0x7fec1c06c660 unknown :-1 s=CLOSED pgs=57 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T15:08:44.066 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:44.063+0000 7fec3550e640 1 --2- 172.21.15.43:0/1757148680 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fec30159de0 0x7fec30153490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T15:08:44.066 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:44.063+0000 7fec3550e640 1 --2- 172.21.15.43:0/1757148680 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fec3015ccc0 0x7fec300afe10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T15:08:44.066 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:44.063+0000 7fec3550e640 1 --2- 172.21.15.43:0/1757148680 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fec180047c0 0x7fec300c68e0 unknown :-1 s=CLOSED pgs=2207 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T15:08:44.066 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:44.063+0000 7fec3550e640 1 -- 172.21.15.43:0/1757148680 >> 172.21.15.43:0/1757148680 conn(0x7fec3006b990 msgr2=0x7fec3006bda0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-09T15:08:44.067 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:44.063+0000 7fec3550e640 1 -- 172.21.15.43:0/1757148680 shutdown_connections 2023-08-09T15:08:44.067 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:44.066+0000 7fec3550e640 1 -- 172.21.15.43:0/1757148680 wait complete. 2023-08-09T15:08:44.089 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data 64 --pg_num_min 64 --target_size_ratio 0.9 2023-08-09T15:08:44.317 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:44.316+0000 7f06f4e8a640 1 Processor -- start 2023-08-09T15:08:44.317 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:44.317+0000 7f06f4e8a640 1 -- start start 2023-08-09T15:08:44.319 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:44.319+0000 7f06f4e8a640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f06f00cb9a0 0x7f06f00cbde0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T15:08:44.319 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:44.319+0000 7f06f4e8a640 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7f06f006f310 0x7f06f006f760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T15:08:44.319 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:44.319+0000 7f06f4e8a640 1 -- --> v1:172.21.15.203:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f06f00acfa0 con 0x7f06f0070ef0 2023-08-09T15:08:44.319 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:44.319+0000 7f06f4e8a640 1 -- --> v2:172.21.15.43:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f06f00cc320 con 0x7f06f00cb9a0 2023-08-09T15:08:44.319 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:44.319+0000 7f06f4e8a640 1 -- --> v2:172.21.15.203:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f06f015c940 con 0x7f06f006f310 2023-08-09T15:08:44.320 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:44.319+0000 7f06eeffd640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f06f00cb9a0 0x7f06f00cbde0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T15:08:44.320 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:44.319+0000 7f06eeffd640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f06f00cb9a0 0x7f06f00cbde0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:39478/0 (socket says 172.21.15.43:39478) 2023-08-09T15:08:44.320 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:44.319+0000 7f06eeffd640 1 -- 172.21.15.43:0/4273672197 learned_addr learned my addr 172.21.15.43:0/4273672197 (peer_addr_for_me v2:172.21.15.43:0/0) 2023-08-09T15:08:44.321 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:44.319+0000 7f06effff640 1 --2- 172.21.15.43:0/4273672197 >> v2:172.21.15.203:3300/0 conn(0x7f06f006f310 0x7f06f006f760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T15:08:44.321 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:44.319+0000 7f06eeffd640 1 -- 172.21.15.43:0/4273672197 >> v2:172.21.15.203:3300/0 conn(0x7f06f006f310 msgr2=0x7f06f006f760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-09T15:08:44.321 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:44.320+0000 7f06eeffd640 1 --2- 172.21.15.43:0/4273672197 >> v2:172.21.15.203:3300/0 conn(0x7f06f006f310 0x7f06f006f760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T15:08:44.321 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:44.320+0000 7f06eeffd640 1 -- 172.21.15.43:0/4273672197 >> v1:172.21.15.203:6789/0 conn(0x7f06f0070ef0 legacy=0x7f06f00ab8d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-09T15:08:44.321 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:44.320+0000 7f06eeffd640 1 -- 172.21.15.43:0/4273672197 --> v2:172.21.15.43:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f06f00ac390 con 0x7f06f00cb9a0 2023-08-09T15:08:44.321 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:44.320+0000 7f06effff640 1 --2- 172.21.15.43:0/4273672197 >> v2:172.21.15.203:3300/0 conn(0x7f06f006f310 0x7f06f006f760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-08-09T15:08:44.322 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:44.321+0000 7f06eeffd640 1 --2- 172.21.15.43:0/4273672197 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f06f00cb9a0 0x7f06f00cbde0 secure :-1 s=READY pgs=2208 cs=0 l=1 rev1=1 crypto rx=0x7f06f0146380 tx=0x7f06e402b9a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3a855a0ca7a16e16 server_cookie=0 in_seq=0 out_seq=0 2023-08-09T15:08:44.322 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:44.321+0000 7f06ee7fc640 1 -- 172.21.15.43:0/4273672197 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f06e4030860 con 0x7f06f00cb9a0 2023-08-09T15:08:44.322 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:44.321+0000 7f06ee7fc640 1 -- 172.21.15.43:0/4273672197 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f06e40309c0 con 0x7f06f00cb9a0 2023-08-09T15:08:44.323 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:44.321+0000 7f06ee7fc640 1 -- 172.21.15.43:0/4273672197 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f06e4030d50 con 0x7f06f00cb9a0 2023-08-09T15:08:44.323 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:44.322+0000 7f06f4e8a640 1 -- 172.21.15.43:0/4273672197 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f06f00cb9a0 msgr2=0x7f06f00cbde0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T15:08:44.323 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:44.322+0000 7f06f4e8a640 1 --2- 172.21.15.43:0/4273672197 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f06f00cb9a0 0x7f06f00cbde0 secure :-1 s=READY pgs=2208 cs=0 l=1 rev1=1 crypto rx=0x7f06f0146380 tx=0x7f06e402b9a0 comp rx=0 tx=0).stop 2023-08-09T15:08:44.323 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:44.322+0000 7f06f4e8a640 1 -- 172.21.15.43:0/4273672197 shutdown_connections 2023-08-09T15:08:44.323 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:44.322+0000 7f06f4e8a640 1 --2- 172.21.15.43:0/4273672197 >> v2:172.21.15.203:3300/0 conn(0x7f06f006f310 0x7f06f006f760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T15:08:44.323 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:44.322+0000 7f06f4e8a640 1 --2- 172.21.15.43:0/4273672197 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f06f00cb9a0 0x7f06f00cbde0 unknown :-1 s=CLOSED pgs=2208 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T15:08:44.323 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:44.322+0000 7f06f4e8a640 1 -- 172.21.15.43:0/4273672197 >> 172.21.15.43:0/4273672197 conn(0x7f06f00cddd0 msgr2=0x7f06f00ce1e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-09T15:08:44.324 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:44.322+0000 7f06f4e8a640 1 -- 172.21.15.43:0/4273672197 shutdown_connections 2023-08-09T15:08:44.324 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:44.322+0000 7f06f4e8a640 1 -- 172.21.15.43:0/4273672197 wait complete. 2023-08-09T15:08:44.325 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:44.325+0000 7f06f4e8a640 1 Processor -- start 2023-08-09T15:08:44.325 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:44.325+0000 7f06f4e8a640 1 -- start start 2023-08-09T15:08:44.328 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:44.326+0000 7f06f4e8a640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f06f006f310 0x7f06f015b970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T15:08:44.328 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:44.326+0000 7f06f4e8a640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f06f0070ef0 0x7f06f00ab110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T15:08:44.328 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:44.326+0000 7f06f4e8a640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f06f00cb9a0 0x7f06f01f9820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T15:08:44.329 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:44.326+0000 7f06f4e8a640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f06f00aad50 con 0x7f06f006f310 2023-08-09T15:08:44.329 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:44.326+0000 7f06f4e8a640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f06f015fb70 con 0x7f06f00cb9a0 2023-08-09T15:08:44.329 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:44.326+0000 7f06f4e8a640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f06f00aa8c0 con 0x7f06f0070ef0 2023-08-09T15:08:44.329 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:44.327+0000 7f06ef7fe640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f06f006f310 0x7f06f015b970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T15:08:44.329 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:44.327+0000 7f06ef7fe640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f06f006f310 0x7f06f015b970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:39484/0 (socket says 172.21.15.43:39484) 2023-08-09T15:08:44.329 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:44.327+0000 7f06ef7fe640 1 -- 172.21.15.43:0/698572006 learned_addr learned my addr 172.21.15.43:0/698572006 (peer_addr_for_me v2:172.21.15.43:0/0) 2023-08-09T15:08:44.329 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:44.327+0000 7f06ef7fe640 1 -- 172.21.15.43:0/698572006 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f06f0070ef0 msgr2=0x7f06f00ab110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T15:08:44.330 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:44.327+0000 7f06ef7fe640 1 --2- 172.21.15.43:0/698572006 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f06f0070ef0 0x7f06f00ab110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T15:08:44.330 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:44.327+0000 7f06effff640 1 --2- 172.21.15.43:0/698572006 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f06f00cb9a0 0x7f06f01f9820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T15:08:44.330 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:44.327+0000 7f06ef7fe640 1 -- 172.21.15.43:0/698572006 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f06f00cb9a0 msgr2=0x7f06f01f9820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T15:08:44.330 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:44.327+0000 7f06ef7fe640 1 --2- 172.21.15.43:0/698572006 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f06f00cb9a0 0x7f06f01f9820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T15:08:44.330 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:44.327+0000 7f06ef7fe640 1 -- 172.21.15.43:0/698572006 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f06e40094d0 con 0x7f06f006f310 2023-08-09T15:08:44.330 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:44.328+0000 7f06ef7fe640 1 --2- 172.21.15.43:0/698572006 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f06f006f310 0x7f06f015b970 secure :-1 s=READY pgs=2209 cs=0 l=1 rev1=1 crypto rx=0x7f06e000c440 tx=0x7f06e000c900 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-09T15:08:44.330 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:44.328+0000 7f06d7fff640 1 -- 172.21.15.43:0/698572006 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f06e0005480 con 0x7f06f006f310 2023-08-09T15:08:44.331 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:44.328+0000 7f06d7fff640 1 -- 172.21.15.43:0/698572006 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f06e00035f0 con 0x7f06f006f310 2023-08-09T15:08:44.331 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:44.328+0000 7f06d7fff640 1 -- 172.21.15.43:0/698572006 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f06e00039c0 con 0x7f06f006f310 2023-08-09T15:08:44.331 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:44.328+0000 7f06f4e8a640 1 -- 172.21.15.43:0/698572006 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f06f00a8430 con 0x7f06f006f310 2023-08-09T15:08:44.331 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:44.328+0000 7f06f4e8a640 1 -- 172.21.15.43:0/698572006 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f06f00a7e70 con 0x7f06f006f310 2023-08-09T15:08:44.331 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:44.329+0000 7f06f4e8a640 1 -- 172.21.15.43:0/698572006 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f06f0145110 con 0x7f06f006f310 2023-08-09T15:08:44.332 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:44.330+0000 7f06d7fff640 1 -- 172.21.15.43:0/698572006 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 106) v1 ==== 88443+0+0 (secure 0 0 0) 0x7f06e0003b20 con 0x7f06f006f310 2023-08-09T15:08:44.332 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:44.332+0000 7f06d7fff640 1 --2- 172.21.15.43:0/698572006 >> [v2:172.21.15.203:6832/3082732769,v1:172.21.15.203:6833/3082732769] conn(0x7f06dc06a170 0x7f06dc06c630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T15:08:44.332 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:44.332+0000 7f06d7fff640 1 -- 172.21.15.43:0/698572006 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(202..202 src has 1..202) v4 ==== 5944+0+0 (secure 0 0 0) 0x7f06e0087970 con 0x7f06f006f310 2023-08-09T15:08:44.333 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:44.333+0000 7f06eeffd640 1 --2- 172.21.15.43:0/698572006 >> [v2:172.21.15.203:6832/3082732769,v1:172.21.15.203:6833/3082732769] conn(0x7f06dc06a170 0x7f06dc06c630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T15:08:44.334 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:44.334+0000 7f06eeffd640 1 --2- 172.21.15.43:0/698572006 >> [v2:172.21.15.203:6832/3082732769,v1:172.21.15.203:6833/3082732769] conn(0x7f06dc06a170 0x7f06dc06c630 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f06e4036000 tx=0x7f06e403b040 comp rx=0 tx=0).ready entity=mgr.8500 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-09T15:08:44.337 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:44.337+0000 7f06d7fff640 1 -- 172.21.15.43:0/698572006 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178066 (secure 0 0 0) 0x7f06e0056bc0 con 0x7f06f006f310 2023-08-09T15:08:44.535 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:44.534+0000 7f06f4e8a640 1 -- 172.21.15.43:0/698572006 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9} v 0) v1 -- 0x7f06f004e390 con 0x7f06f006f310 2023-08-09T15:08:45.068 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:45.068+0000 7f06d7fff640 1 -- 172.21.15.43:0/698572006 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]=0 pool 'cephfs_data' created v203) v1 ==== 170+0+0 (secure 0 0 0) 0x7f06e0056750 con 0x7f06f006f310 2023-08-09T15:08:45.069 INFO:teuthology.orchestra.run.smithi043.stderr:pool 'cephfs_data' created 2023-08-09T15:08:45.076 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:45.076+0000 7f06f4e8a640 1 -- 172.21.15.43:0/698572006 >> [v2:172.21.15.203:6832/3082732769,v1:172.21.15.203:6833/3082732769] conn(0x7f06dc06a170 msgr2=0x7f06dc06c630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T15:08:45.077 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:45.076+0000 7f06f4e8a640 1 --2- 172.21.15.43:0/698572006 >> [v2:172.21.15.203:6832/3082732769,v1:172.21.15.203:6833/3082732769] conn(0x7f06dc06a170 0x7f06dc06c630 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f06e4036000 tx=0x7f06e403b040 comp rx=0 tx=0).stop 2023-08-09T15:08:45.077 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:45.076+0000 7f06f4e8a640 1 -- 172.21.15.43:0/698572006 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f06f006f310 msgr2=0x7f06f015b970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T15:08:45.077 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:45.076+0000 7f06f4e8a640 1 --2- 172.21.15.43:0/698572006 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f06f006f310 0x7f06f015b970 secure :-1 s=READY pgs=2209 cs=0 l=1 rev1=1 crypto rx=0x7f06e000c440 tx=0x7f06e000c900 comp rx=0 tx=0).stop 2023-08-09T15:08:45.079 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:45.078+0000 7f06f4e8a640 1 -- 172.21.15.43:0/698572006 shutdown_connections 2023-08-09T15:08:45.079 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:45.078+0000 7f06f4e8a640 1 --2- 172.21.15.43:0/698572006 >> [v2:172.21.15.203:6832/3082732769,v1:172.21.15.203:6833/3082732769] conn(0x7f06dc06a170 0x7f06dc06c630 unknown :-1 s=CLOSED pgs=58 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T15:08:45.079 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:45.078+0000 7f06f4e8a640 1 --2- 172.21.15.43:0/698572006 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f06f00cb9a0 0x7f06f01f9820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T15:08:45.079 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:45.078+0000 7f06f4e8a640 1 --2- 172.21.15.43:0/698572006 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f06f0070ef0 0x7f06f00ab110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T15:08:45.079 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:45.078+0000 7f06f4e8a640 1 --2- 172.21.15.43:0/698572006 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f06f006f310 0x7f06f015b970 unknown :-1 s=CLOSED pgs=2209 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T15:08:45.080 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:45.078+0000 7f06f4e8a640 1 -- 172.21.15.43:0/698572006 >> 172.21.15.43:0/698572006 conn(0x7f06f00cddd0 msgr2=0x7f06f006bb50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-09T15:08:45.082 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:45.081+0000 7f06f4e8a640 1 -- 172.21.15.43:0/698572006 shutdown_connections 2023-08-09T15:08:45.082 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:45.082+0000 7f06f4e8a640 1 -- 172.21.15.43:0/698572006 wait complete. 2023-08-09T15:08:45.115 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs new cephfs cephfs_metadata cephfs_data 2023-08-09T15:08:45.312 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:45.311+0000 7f58a8712640 1 Processor -- start 2023-08-09T15:08:45.312 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:45.312+0000 7f58a8712640 1 -- start start 2023-08-09T15:08:45.313 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:45.313+0000 7f58a8712640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f58a015c550 0x7f58a00cd6d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T15:08:45.314 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:45.313+0000 7f58a8712640 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7f58a00d04b0 0x7f58a00b3ee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T15:08:45.314 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:45.313+0000 7f58a8712640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f58a00b4420 0x7f58a006a760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T15:08:45.314 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:45.313+0000 7f58a8712640 1 -- --> v2:172.21.15.43:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f58a00b4870 con 0x7f58a015c550 2023-08-09T15:08:45.314 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:45.313+0000 7f58a8712640 1 -- --> v2:172.21.15.203:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f58a00b7bd0 con 0x7f58a00d04b0 2023-08-09T15:08:45.314 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:45.313+0000 7f58a8712640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f58a00d1c00 con 0x7f58a00b4420 2023-08-09T15:08:45.315 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:45.314+0000 7f58a64ae640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f58a015c550 0x7f58a00cd6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T15:08:45.315 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:45.314+0000 7f58a64ae640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f58a015c550 0x7f58a00cd6d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:39752/0 (socket says 172.21.15.43:39752) 2023-08-09T15:08:45.316 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:45.314+0000 7f58a64ae640 1 -- 172.21.15.43:0/2497903653 learned_addr learned my addr 172.21.15.43:0/2497903653 (peer_addr_for_me v2:172.21.15.43:0/0) 2023-08-09T15:08:45.316 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:45.314+0000 7f58a5cad640 1 --2- 172.21.15.43:0/2497903653 >> v2:172.21.15.203:3300/0 conn(0x7f58a00d04b0 0x7f58a00b3ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T15:08:45.316 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:45.314+0000 7f58a6caf640 1 --2- 172.21.15.43:0/2497903653 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f58a00b4420 0x7f58a006a760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T15:08:45.316 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:45.314+0000 7f58a64ae640 1 -- 172.21.15.43:0/2497903653 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f58a00b4420 msgr2=0x7f58a006a760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-09T15:08:45.316 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:45.314+0000 7f58a64ae640 1 --2- 172.21.15.43:0/2497903653 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f58a00b4420 0x7f58a006a760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T15:08:45.316 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:45.314+0000 7f58a64ae640 1 -- 172.21.15.43:0/2497903653 >> v2:172.21.15.203:3300/0 conn(0x7f58a00d04b0 msgr2=0x7f58a00b3ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-09T15:08:45.317 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:45.314+0000 7f58a64ae640 1 --2- 172.21.15.43:0/2497903653 >> v2:172.21.15.203:3300/0 conn(0x7f58a00d04b0 0x7f58a00b3ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T15:08:45.317 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:45.314+0000 7f58a64ae640 1 -- 172.21.15.43:0/2497903653 --> v2:172.21.15.43:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f58a015c930 con 0x7f58a015c550 2023-08-09T15:08:45.318 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:45.316+0000 7f58a64ae640 1 --2- 172.21.15.43:0/2497903653 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f58a015c550 0x7f58a00cd6d0 secure :-1 s=READY pgs=2210 cs=0 l=1 rev1=1 crypto rx=0x7f58a00b5050 tx=0x7f589c02b880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7da6d5c1b19dd936 server_cookie=0 in_seq=0 out_seq=0 2023-08-09T15:08:45.318 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:45.316+0000 7f58a54ac640 1 -- 172.21.15.43:0/2497903653 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f589c0317e0 con 0x7f58a015c550 2023-08-09T15:08:45.318 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:45.316+0000 7f58a54ac640 1 -- 172.21.15.43:0/2497903653 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f589c031940 con 0x7f58a015c550 2023-08-09T15:08:45.318 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:45.316+0000 7f58a54ac640 1 -- 172.21.15.43:0/2497903653 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f589c031cd0 con 0x7f58a015c550 2023-08-09T15:08:45.318 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:45.317+0000 7f58a8712640 1 -- 172.21.15.43:0/2497903653 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f58a015c550 msgr2=0x7f58a00cd6d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T15:08:45.318 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:45.317+0000 7f58a8712640 1 --2- 172.21.15.43:0/2497903653 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f58a015c550 0x7f58a00cd6d0 secure :-1 s=READY pgs=2210 cs=0 l=1 rev1=1 crypto rx=0x7f58a00b5050 tx=0x7f589c02b880 comp rx=0 tx=0).stop 2023-08-09T15:08:45.319 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:45.317+0000 7f58a8712640 1 -- 172.21.15.43:0/2497903653 shutdown_connections 2023-08-09T15:08:45.319 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:45.317+0000 7f58a8712640 1 --2- 172.21.15.43:0/2497903653 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f58a00b4420 0x7f58a006a760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T15:08:45.319 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:45.317+0000 7f58a8712640 1 --2- 172.21.15.43:0/2497903653 >> v2:172.21.15.203:3300/0 conn(0x7f58a00d04b0 0x7f58a00b3ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T15:08:45.319 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:45.317+0000 7f58a8712640 1 --2- 172.21.15.43:0/2497903653 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f58a015c550 0x7f58a00cd6d0 unknown :-1 s=CLOSED pgs=2210 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T15:08:45.319 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:45.317+0000 7f58a8712640 1 -- 172.21.15.43:0/2497903653 >> 172.21.15.43:0/2497903653 conn(0x7f58a0071520 msgr2=0x7f58a00ceee0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-09T15:08:45.319 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:45.317+0000 7f58a8712640 1 -- 172.21.15.43:0/2497903653 shutdown_connections 2023-08-09T15:08:45.319 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:45.317+0000 7f58a8712640 1 -- 172.21.15.43:0/2497903653 wait complete. 2023-08-09T15:08:45.321 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:45.319+0000 7f58a8712640 1 Processor -- start 2023-08-09T15:08:45.321 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:45.319+0000 7f58a8712640 1 -- start start 2023-08-09T15:08:45.321 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:45.320+0000 7f58a8712640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f58a00b4420 0x7f58a01f12b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T15:08:45.321 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:45.320+0000 7f58a8712640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f58a00d04b0 0x7f58a00caf60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T15:08:45.321 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:45.320+0000 7f58a8712640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f58a015c550 0x7f58a00ab280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T15:08:45.322 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:45.320+0000 7f58a8712640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f58a01f17f0 con 0x7f58a00b4420 2023-08-09T15:08:45.322 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:45.320+0000 7f58a8712640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f58a00cb970 con 0x7f58a00d04b0 2023-08-09T15:08:45.322 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:45.320+0000 7f58a8712640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f58a00cbc50 con 0x7f58a015c550 2023-08-09T15:08:45.322 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:45.320+0000 7f58a64ae640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f58a00b4420 0x7f58a01f12b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T15:08:45.322 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:45.320+0000 7f58a64ae640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f58a00b4420 0x7f58a01f12b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:39758/0 (socket says 172.21.15.43:39758) 2023-08-09T15:08:45.322 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:45.320+0000 7f58a64ae640 1 -- 172.21.15.43:0/484379833 learned_addr learned my addr 172.21.15.43:0/484379833 (peer_addr_for_me v2:172.21.15.43:0/0) 2023-08-09T15:08:45.323 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:45.321+0000 7f58a6caf640 1 --2- 172.21.15.43:0/484379833 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f58a015c550 0x7f58a00ab280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T15:08:45.323 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:45.321+0000 7f58a5cad640 1 --2- 172.21.15.43:0/484379833 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f58a00d04b0 0x7f58a00caf60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T15:08:45.323 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:45.321+0000 7f58a64ae640 1 -- 172.21.15.43:0/484379833 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f58a015c550 msgr2=0x7f58a00ab280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T15:08:45.323 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:45.321+0000 7f58a64ae640 1 --2- 172.21.15.43:0/484379833 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f58a015c550 0x7f58a00ab280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T15:08:45.323 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:45.321+0000 7f58a64ae640 1 -- 172.21.15.43:0/484379833 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f58a00d04b0 msgr2=0x7f58a00caf60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T15:08:45.323 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:45.321+0000 7f58a64ae640 1 --2- 172.21.15.43:0/484379833 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f58a00d04b0 0x7f58a00caf60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T15:08:45.323 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:45.321+0000 7f58a64ae640 1 -- 172.21.15.43:0/484379833 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f589c0094d0 con 0x7f58a00b4420 2023-08-09T15:08:45.324 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:45.321+0000 7f58a64ae640 1 --2- 172.21.15.43:0/484379833 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f58a00b4420 0x7f58a01f12b0 secure :-1 s=READY pgs=2211 cs=0 l=1 rev1=1 crypto rx=0x7f589c02b8c0 tx=0x7f589c0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-09T15:08:45.324 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:45.321+0000 7f5896ffd640 1 -- 172.21.15.43:0/484379833 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f589c032880 con 0x7f58a00b4420 2023-08-09T15:08:45.325 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:45.322+0000 7f5896ffd640 1 -- 172.21.15.43:0/484379833 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f589c032a00 con 0x7f58a00b4420 2023-08-09T15:08:45.325 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:45.322+0000 7f5896ffd640 1 -- 172.21.15.43:0/484379833 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f589c032db0 con 0x7f58a00b4420 2023-08-09T15:08:45.325 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:45.322+0000 7f58a8712640 1 -- 172.21.15.43:0/484379833 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f58a00cbf30 con 0x7f58a00b4420 2023-08-09T15:08:45.325 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:45.322+0000 7f58a8712640 1 -- 172.21.15.43:0/484379833 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f58a00a9140 con 0x7f58a00b4420 2023-08-09T15:08:45.325 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:45.323+0000 7f58a8712640 1 -- 172.21.15.43:0/484379833 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f58a00739f0 con 0x7f58a00b4420 2023-08-09T15:08:45.326 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:45.324+0000 7f5896ffd640 1 -- 172.21.15.43:0/484379833 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 106) v1 ==== 88443+0+0 (secure 0 0 0) 0x7f589c04d3f0 con 0x7f58a00b4420 2023-08-09T15:08:45.327 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:45.325+0000 7f5896ffd640 1 --2- 172.21.15.43:0/484379833 >> [v2:172.21.15.203:6832/3082732769,v1:172.21.15.203:6833/3082732769] conn(0x7f588406a170 0x7f588406c630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T15:08:45.327 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:45.325+0000 7f5896ffd640 1 -- 172.21.15.43:0/484379833 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(203..203 src has 1..203) v4 ==== 6326+0+0 (secure 0 0 0) 0x7f589c0aa980 con 0x7f58a00b4420 2023-08-09T15:08:45.327 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:45.326+0000 7f58a5cad640 1 --2- 172.21.15.43:0/484379833 >> [v2:172.21.15.203:6832/3082732769,v1:172.21.15.203:6833/3082732769] conn(0x7f588406a170 0x7f588406c630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T15:08:45.328 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:45.327+0000 7f58a5cad640 1 --2- 172.21.15.43:0/484379833 >> [v2:172.21.15.203:6832/3082732769,v1:172.21.15.203:6833/3082732769] conn(0x7f588406a170 0x7f588406c630 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f58a015e250 tx=0x7f5890008040 comp rx=0 tx=0).ready entity=mgr.8500 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-09T15:08:45.330 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:45.330+0000 7f5896ffd640 1 -- 172.21.15.43:0/484379833 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178066 (secure 0 0 0) 0x7f589c075d20 con 0x7f58a00b4420 2023-08-09T15:08:45.561 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:45.560+0000 7f58a8712640 1 -- 172.21.15.43:0/484379833 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"} v 0) v1 -- 0x7f58a00cbd90 con 0x7f58a00b4420 2023-08-09T15:08:46.131 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:46.130+0000 7f5896ffd640 1 -- 172.21.15.43:0/484379833 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]=0 new fs with metadata pool 24 and data pool 25 v92) v1 ==== 174+0+0 (secure 0 0 0) 0x7f58a00cbd90 con 0x7f58a00b4420 2023-08-09T15:08:46.132 INFO:teuthology.orchestra.run.smithi043.stderr:new fs with metadata pool 24 and data pool 25 2023-08-09T15:08:46.150 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:46.150+0000 7f58a8712640 1 -- 172.21.15.43:0/484379833 >> [v2:172.21.15.203:6832/3082732769,v1:172.21.15.203:6833/3082732769] conn(0x7f588406a170 msgr2=0x7f588406c630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T15:08:46.151 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:46.151+0000 7f58a8712640 1 --2- 172.21.15.43:0/484379833 >> [v2:172.21.15.203:6832/3082732769,v1:172.21.15.203:6833/3082732769] conn(0x7f588406a170 0x7f588406c630 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f58a015e250 tx=0x7f5890008040 comp rx=0 tx=0).stop 2023-08-09T15:08:46.153 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:46.153+0000 7f58a8712640 1 -- 172.21.15.43:0/484379833 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f58a00b4420 msgr2=0x7f58a01f12b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T15:08:46.153 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:46.153+0000 7f58a8712640 1 --2- 172.21.15.43:0/484379833 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f58a00b4420 0x7f58a01f12b0 secure :-1 s=READY pgs=2211 cs=0 l=1 rev1=1 crypto rx=0x7f589c02b8c0 tx=0x7f589c0046c0 comp rx=0 tx=0).stop 2023-08-09T15:08:46.154 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:46.154+0000 7f58a8712640 1 -- 172.21.15.43:0/484379833 shutdown_connections 2023-08-09T15:08:46.155 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:46.154+0000 7f58a8712640 1 --2- 172.21.15.43:0/484379833 >> [v2:172.21.15.203:6832/3082732769,v1:172.21.15.203:6833/3082732769] conn(0x7f588406a170 0x7f588406c630 unknown :-1 s=CLOSED pgs=59 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T15:08:46.155 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:46.154+0000 7f58a8712640 1 --2- 172.21.15.43:0/484379833 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f58a015c550 0x7f58a00ab280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T15:08:46.155 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:46.154+0000 7f58a8712640 1 --2- 172.21.15.43:0/484379833 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f58a00d04b0 0x7f58a00caf60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T15:08:46.155 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:46.154+0000 7f58a8712640 1 --2- 172.21.15.43:0/484379833 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f58a00b4420 0x7f58a01f12b0 unknown :-1 s=CLOSED pgs=2211 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T15:08:46.155 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:46.154+0000 7f58a8712640 1 -- 172.21.15.43:0/484379833 >> 172.21.15.43:0/484379833 conn(0x7f58a0071520 msgr2=0x7f58a015cac0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-09T15:08:46.156 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:46.155+0000 7f58a8712640 1 -- 172.21.15.43:0/484379833 shutdown_connections 2023-08-09T15:08:46.157 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:46.157+0000 7f58a8712640 1 -- 172.21.15.43:0/484379833 wait complete. 2023-08-09T15:08:46.175 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-08-09T15:08:46.357 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:46.357+0000 7f88ef0ec640 1 Processor -- start 2023-08-09T15:08:46.357 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:46.357+0000 7f88ef0ec640 1 -- start start 2023-08-09T15:08:46.359 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:46.358+0000 7f88ef0ec640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f88e80af6f0 0x7f88e806a8d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T15:08:46.359 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:46.358+0000 7f88ef0ec640 1 -- --> v1:172.21.15.43:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f88e80af1a0 con 0x7f88e806b140 2023-08-09T15:08:46.359 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:46.358+0000 7f88ef0ec640 1 -- --> v1:172.21.15.203:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f88e80cdc20 con 0x7f88e80cf760 2023-08-09T15:08:46.359 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:46.358+0000 7f88ef0ec640 1 -- --> v2:172.21.15.43:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f88e806be10 con 0x7f88e80af6f0 2023-08-09T15:08:46.359 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:46.359+0000 7f88dffff640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f88e80af6f0 0x7f88e806a8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T15:08:46.360 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:46.359+0000 7f88dffff640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f88e80af6f0 0x7f88e806a8d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:39762/0 (socket says 172.21.15.43:39762) 2023-08-09T15:08:46.360 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:46.359+0000 7f88dffff640 1 -- 172.21.15.43:0/153921918 learned_addr learned my addr 172.21.15.43:0/153921918 (peer_addr_for_me v2:172.21.15.43:0/0) 2023-08-09T15:08:46.361 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:46.359+0000 7f88dffff640 1 -- 172.21.15.43:0/153921918 >> v1:172.21.15.203:6789/0 conn(0x7f88e80cf760 legacy=0x7f88e806dcb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-09T15:08:46.361 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:46.359+0000 7f88dffff640 1 -- 172.21.15.43:0/153921918 >> v1:172.21.15.43:6789/0 conn(0x7f88e806b140 legacy=0x7f88e806b520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T15:08:46.361 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:46.359+0000 7f88dffff640 1 -- 172.21.15.43:0/153921918 --> v2:172.21.15.43:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f88e8160100 con 0x7f88e80af6f0 2023-08-09T15:08:46.361 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:46.359+0000 7f88df7fe640 1 -- 172.21.15.43:0/153921918 <== mon.0 v1:172.21.15.43:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1765274496 0 0) 0x7f88e80af1a0 con 0x7f88e806b140 2023-08-09T15:08:46.361 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:46.360+0000 7f88dffff640 1 --2- 172.21.15.43:0/153921918 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f88e80af6f0 0x7f88e806a8d0 secure :-1 s=READY pgs=2213 cs=0 l=1 rev1=1 crypto rx=0x7f88d8009870 tx=0x7f88d802b990 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ea1bcfdf676d3761 server_cookie=0 in_seq=0 out_seq=0 2023-08-09T15:08:46.362 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:46.361+0000 7f88df7fe640 1 -- 172.21.15.43:0/153921918 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f88d8030850 con 0x7f88e80af6f0 2023-08-09T15:08:46.362 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:46.361+0000 7f88df7fe640 1 -- 172.21.15.43:0/153921918 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f88d80309b0 con 0x7f88e80af6f0 2023-08-09T15:08:46.362 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:46.361+0000 7f88df7fe640 1 -- 172.21.15.43:0/153921918 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f88d8030d40 con 0x7f88e80af6f0 2023-08-09T15:08:46.363 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:46.361+0000 7f88ef0ec640 1 -- 172.21.15.43:0/153921918 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f88e80af6f0 msgr2=0x7f88e806a8d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T15:08:46.363 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:46.361+0000 7f88ef0ec640 1 --2- 172.21.15.43:0/153921918 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f88e80af6f0 0x7f88e806a8d0 secure :-1 s=READY pgs=2213 cs=0 l=1 rev1=1 crypto rx=0x7f88d8009870 tx=0x7f88d802b990 comp rx=0 tx=0).stop 2023-08-09T15:08:46.363 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:46.361+0000 7f88ef0ec640 1 -- 172.21.15.43:0/153921918 shutdown_connections 2023-08-09T15:08:46.363 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:46.361+0000 7f88ef0ec640 1 --2- 172.21.15.43:0/153921918 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f88e80af6f0 0x7f88e806a8d0 unknown :-1 s=CLOSED pgs=2213 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T15:08:46.363 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:46.362+0000 7f88ef0ec640 1 -- 172.21.15.43:0/153921918 >> 172.21.15.43:0/153921918 conn(0x7f88e80ccd80 msgr2=0x7f88e80cd190 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-09T15:08:46.363 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:46.362+0000 7f88ef0ec640 1 -- 172.21.15.43:0/153921918 shutdown_connections 2023-08-09T15:08:46.364 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:46.362+0000 7f88ef0ec640 1 -- 172.21.15.43:0/153921918 wait complete. 2023-08-09T15:08:46.364 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:46.363+0000 7f88ef0ec640 1 Processor -- start 2023-08-09T15:08:46.364 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:46.363+0000 7f88ef0ec640 1 -- start start 2023-08-09T15:08:46.365 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:46.364+0000 7f88ef0ec640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f88e806b140 0x7f88e8152020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T15:08:46.366 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:46.364+0000 7f88ef0ec640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f88e80af6f0 0x7f88e80ce610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T15:08:46.366 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:46.364+0000 7f88ef0ec640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f88e80cf760 0x7f88e80ceb50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T15:08:46.366 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:46.364+0000 7f88ef0ec640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f88e8152560 con 0x7f88e80cf760 2023-08-09T15:08:46.366 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:46.364+0000 7f88ef0ec640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f88e81f4fa0 con 0x7f88e80af6f0 2023-08-09T15:08:46.366 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:46.364+0000 7f88ef0ec640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f88e81f51f0 con 0x7f88e806b140 2023-08-09T15:08:46.366 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:46.365+0000 7f88ed689640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f88e80cf760 0x7f88e80ceb50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T15:08:46.367 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:46.365+0000 7f88ed689640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f88e80cf760 0x7f88e80ceb50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:39766/0 (socket says 172.21.15.43:39766) 2023-08-09T15:08:46.367 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:46.365+0000 7f88ed689640 1 -- 172.21.15.43:0/3479778844 learned_addr learned my addr 172.21.15.43:0/3479778844 (peer_addr_for_me v2:172.21.15.43:0/0) 2023-08-09T15:08:46.367 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:46.365+0000 7f88ece88640 1 --2- 172.21.15.43:0/3479778844 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f88e806b140 0x7f88e8152020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T15:08:46.367 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:46.365+0000 7f88dffff640 1 --2- 172.21.15.43:0/3479778844 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f88e80af6f0 0x7f88e80ce610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T15:08:46.367 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:46.365+0000 7f88ed689640 1 -- 172.21.15.43:0/3479778844 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f88e806b140 msgr2=0x7f88e8152020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T15:08:46.368 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:46.365+0000 7f88ed689640 1 --2- 172.21.15.43:0/3479778844 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f88e806b140 0x7f88e8152020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T15:08:46.368 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:46.365+0000 7f88ed689640 1 -- 172.21.15.43:0/3479778844 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f88e80af6f0 msgr2=0x7f88e80ce610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T15:08:46.368 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:46.365+0000 7f88ed689640 1 --2- 172.21.15.43:0/3479778844 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f88e80af6f0 0x7f88e80ce610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T15:08:46.368 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:46.365+0000 7f88ed689640 1 -- 172.21.15.43:0/3479778844 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f88d80094d0 con 0x7f88e80cf760 2023-08-09T15:08:46.368 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:46.365+0000 7f88dffff640 1 --2- 172.21.15.43:0/3479778844 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f88e80af6f0 0x7f88e80ce610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-08-09T15:08:46.369 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:46.366+0000 7f88ed689640 1 --2- 172.21.15.43:0/3479778844 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f88e80cf760 0x7f88e80ceb50 secure :-1 s=READY pgs=2214 cs=0 l=1 rev1=1 crypto rx=0x7f88e000c700 tx=0x7f88e000cbc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-09T15:08:46.369 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:46.366+0000 7f88dd7fa640 1 -- 172.21.15.43:0/3479778844 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f88e000aa50 con 0x7f88e80cf760 2023-08-09T15:08:46.369 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:46.366+0000 7f88dd7fa640 1 -- 172.21.15.43:0/3479778844 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f88e000abb0 con 0x7f88e80cf760 2023-08-09T15:08:46.370 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:46.366+0000 7f88dd7fa640 1 -- 172.21.15.43:0/3479778844 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f88e000ce10 con 0x7f88e80cf760 2023-08-09T15:08:46.370 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:46.366+0000 7f88ef0ec640 1 -- 172.21.15.43:0/3479778844 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f88e81f54d0 con 0x7f88e80cf760 2023-08-09T15:08:46.370 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:46.366+0000 7f88ef0ec640 1 -- 172.21.15.43:0/3479778844 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f88e81f5a90 con 0x7f88e80cf760 2023-08-09T15:08:46.370 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:46.367+0000 7f88ef0ec640 1 -- 172.21.15.43:0/3479778844 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f88e81f5c80 con 0x7f88e80cf760 2023-08-09T15:08:46.370 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:46.368+0000 7f88dd7fa640 1 -- 172.21.15.43:0/3479778844 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 106) v1 ==== 88443+0+0 (secure 0 0 0) 0x7f88e0012420 con 0x7f88e80cf760 2023-08-09T15:08:46.371 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:46.370+0000 7f88dd7fa640 1 --2- 172.21.15.43:0/3479778844 >> [v2:172.21.15.203:6832/3082732769,v1:172.21.15.203:6833/3082732769] conn(0x7f88d406a1a0 0x7f88d406c660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T15:08:46.371 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:46.370+0000 7f88dd7fa640 1 -- 172.21.15.43:0/3479778844 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(204..204 src has 1..204) v4 ==== 6426+0+0 (secure 0 0 0) 0x7f88e0088de0 con 0x7f88e80cf760 2023-08-09T15:08:46.371 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:46.370+0000 7f88ece88640 1 --2- 172.21.15.43:0/3479778844 >> [v2:172.21.15.203:6832/3082732769,v1:172.21.15.203:6833/3082732769] conn(0x7f88d406a1a0 0x7f88d406c660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T15:08:46.372 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:46.372+0000 7f88ece88640 1 --2- 172.21.15.43:0/3479778844 >> [v2:172.21.15.203:6832/3082732769,v1:172.21.15.203:6833/3082732769] conn(0x7f88d406a1a0 0x7f88d406c660 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f88e80b1190 tx=0x7f88e4009040 comp rx=0 tx=0).ready entity=mgr.8500 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-09T15:08:46.374 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:46.374+0000 7f88dd7fa640 1 -- 172.21.15.43:0/3479778844 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178066 (secure 0 0 0) 0x7f88e0057c40 con 0x7f88e80cf760 2023-08-09T15:08:46.561 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:46.561+0000 7f88ef0ec640 1 -- 172.21.15.43:0/3479778844 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f88e80739f0 con 0x7f88e80cf760 2023-08-09T15:08:46.564 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:46.564+0000 7f88dd7fa640 1 -- 172.21.15.43:0/3479778844 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v204) v1 ==== 74+0+18579 (secure 0 0 0) 0x7f88e0016090 con 0x7f88e80cf760 2023-08-09T15:08:46.565 INFO:teuthology.orchestra.run.smithi043.stdout: 2023-08-09T15:08:46.565 INFO:teuthology.orchestra.run.smithi043.stdout:{"epoch":204,"fsid":"daac87ee-1337-4250-a955-aea4e408a39b","created":"2023-08-09T13:33:41.701198+0000","modified":"2023-08-09T15:08:45.562226+0000","last_up_change":"2023-08-09T13:33:53.783453+0000","last_in_change":"2023-08-09T13:33:45.148904+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":25,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":"rbd","create_time":"2023-08-09T13:33:54.365137+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":".mgr","create_time":"2023-08-09T13:33:54.831803+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":24,"pool_name":"cephfs_metadata","create_time":"2023-08-09T15:08:43.920337+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"204","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":25,"pool_name":"cephfs_data","create_time":"2023-08-09T15:08:44.594489+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"204","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"cbd49ecd-e896-4fbd-864e-5c37a829b165","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":203,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6800","nonce":502667335},{"type":"v1","addr":"172.21.15.43:6801","nonce":502667335}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6802","nonce":502667335},{"type":"v1","addr":"172.21.15.43:6803","nonce":502667335}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6806","nonce":502667335},{"type":"v1","addr":"172.21.15.43:6807","nonce":502667335}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6804","nonce":502667335},{"type":"v1","addr":"172.21.15.43:6805","nonce":502667335}]},"public_addr":"172.21.15.43:6801/502667335","cluster_addr":"172.21.15.43:6803/502667335","heartbeat_back_addr":"172.21.15.43:6807/502667335","heartbeat_front_addr":"172.21.15.43:6805/502667335","state":["exists","up"]},{"osd":1,"uuid":"75c7e0fe-1ec7-4fe8-a2f2-3661a65b8500","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":203,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6808","nonce":3289134610},{"type":"v1","addr":"172.21.15.43:6809","nonce":3289134610}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6810","nonce":3289134610},{"type":"v1","addr":"172.21.15.43:6812","nonce":3289134610}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6821","nonce":3289134610},{"type":"v1","addr":"172.21.15.43:6824","nonce":3289134610}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6815","nonce":3289134610},{"type":"v1","addr":"172.21.15.43:6818","nonce":3289134610}]},"public_addr":"172.21.15.43:6809/3289134610","cluster_addr":"172.21.15.43:6812/3289134610","heartbeat_back_addr":"172.21.15.43:6824/3289134610","heartbeat_front_addr":"172.21.15.43:6818/3289134610","state":["exists","up"]},{"osd":2,"uuid":"bb5c41e9-7f5f-4125-9b01-3bb8bfc2ea83","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":203,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6811","nonce":4079709736},{"type":"v1","addr":"172.21.15.43:6814","nonce":4079709736}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6817","nonce":4079709736},{"type":"v1","addr":"172.21.15.43:6820","nonce":4079709736}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6829","nonce":4079709736},{"type":"v1","addr":"172.21.15.43:6831","nonce":4079709736}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6823","nonce":4079709736},{"type":"v1","addr":"172.21.15.43:6826","nonce":4079709736}]},"public_addr":"172.21.15.43:6814/4079709736","cluster_addr":"172.21.15.43:6820/4079709736","heartbeat_back_addr":"172.21.15.43:6831/4079709736","heartbeat_front_addr":"172.21.15.43:6826/4079709736","state":["exists","up"]},{"osd":3,"uuid":"bfe4c8b4-694a-465c-87e6-2b2aab852613","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":203,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6813","nonce":1709538759},{"type":"v1","addr":"172.21.15.43:6816","nonce":1709538759}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6819","nonce":1709538759},{"type":"v1","addr":"172.21.15.43:6822","nonce":1709538759}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6828","nonce":1709538759},{"type":"v1","addr":"172.21.15.43:6830","nonce":1709538759}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6825","nonce":1709538759},{"type":"v1","addr":"172.21.15.43:6827","nonce":1709538759}]},"public_addr":"172.21.15.43:6816/1709538759","cluster_addr":"172.21.15.43:6822/1709538759","heartbeat_back_addr":"172.21.15.43:6830/1709538759","heartbeat_front_addr":"172.21.15.43:6827/1709538759","state":["exists","up"]},{"osd":4,"uuid":"fa2e8a77-aba1-4756-853e-0d174a5bdeba","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":203,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6800","nonce":4196866481},{"type":"v1","addr":"172.21.15.203:6801","nonce":4196866481}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6802","nonce":4196866481},{"type":"v1","addr":"172.21.15.203:6803","nonce":4196866481}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6806","nonce":4196866481},{"type":"v1","addr":"172.21.15.203:6807","nonce":4196866481}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6804","nonce":4196866481},{"type":"v1","addr":"172.21.15.203:6805","nonce":4196866481}]},"public_addr":"172.21.15.203:6801/4196866481","cluster_addr":"172.21.15.203:6803/4196866481","heartbeat_back_addr":"172.21.15.203:6807/4196866481","heartbeat_front_addr":"172.21.15.203:6805/4196866481","state":["exists","up"]},{"osd":5,"uuid":"81811e4b-4d24-4f51-a82c-f1d700a2a00d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":203,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6816","nonce":3059290081},{"type":"v1","addr":"172.21.15.203:6817","nonce":3059290081}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6818","nonce":3059290081},{"type":"v1","addr":"172.21.15.203:6819","nonce":3059290081}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6822","nonce":3059290081},{"type":"v1","addr":"172.21.15.203:6823","nonce":3059290081}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6820","nonce":3059290081},{"type":"v1","addr":"172.21.15.203:6821","nonce":3059290081}]},"public_addr":"172.21.15.203:6817/3059290081","cluster_addr":"172.21.15.203:6819/3059290081","heartbeat_back_addr":"172.21.15.203:6823/3059290081","heartbeat_front_addr":"172.21.15.203:6821/3059290081","state":["exists","up"]},{"osd":6,"uuid":"307bac4d-6a5b-443d-af75-56d695d7674f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":203,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6824","nonce":512155569},{"type":"v1","addr":"172.21.15.203:6825","nonce":512155569}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6826","nonce":512155569},{"type":"v1","addr":"172.21.15.203:6827","nonce":512155569}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6830","nonce":512155569},{"type":"v1","addr":"172.21.15.203:6831","nonce":512155569}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6828","nonce":512155569},{"type":"v1","addr":"172.21.15.203:6829","nonce":512155569}]},"public_addr":"172.21.15.203:6825/512155569","cluster_addr":"172.21.15.203:6827/512155569","heartbeat_back_addr":"172.21.15.203:6831/512155569","heartbeat_front_addr":"172.21.15.203:6829/512155569","state":["exists","up"]},{"osd":7,"uuid":"a95dc7ca-9f1b-4295-af70-68a22a2d7d5e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":203,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6808","nonce":3331450438},{"type":"v1","addr":"172.21.15.203:6809","nonce":3331450438}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6810","nonce":3331450438},{"type":"v1","addr":"172.21.15.203:6811","nonce":3331450438}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6814","nonce":3331450438},{"type":"v1","addr":"172.21.15.203:6815","nonce":3331450438}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6812","nonce":3331450438},{"type":"v1","addr":"172.21.15.203:6813","nonce":3331450438}]},"public_addr":"172.21.15.203:6809/3331450438","cluster_addr":"172.21.15.203:6811/3331450438","heartbeat_back_addr":"172.21.15.203:6815/3331450438","heartbeat_front_addr":"172.21.15.203:6813/3331450438","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-09T13:33:50.547652+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-09T13:33:50.589469+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-09T13:33:50.561724+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-09T13:33:50.599650+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-09T13:33:50.411855+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-09T13:33:50.219681+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-09T13:33:50.504664+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-09T13:33:50.441248+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":7,"snaps":[{"begin":104,"length":1},{"begin":221,"length":1}]}],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-08-09T15:08:46.567 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:46.566+0000 7f88ef0ec640 1 -- 172.21.15.43:0/3479778844 >> [v2:172.21.15.203:6832/3082732769,v1:172.21.15.203:6833/3082732769] conn(0x7f88d406a1a0 msgr2=0x7f88d406c660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T15:08:46.567 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:46.566+0000 7f88ef0ec640 1 --2- 172.21.15.43:0/3479778844 >> [v2:172.21.15.203:6832/3082732769,v1:172.21.15.203:6833/3082732769] conn(0x7f88d406a1a0 0x7f88d406c660 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f88e80b1190 tx=0x7f88e4009040 comp rx=0 tx=0).stop 2023-08-09T15:08:46.567 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:46.566+0000 7f88ef0ec640 1 -- 172.21.15.43:0/3479778844 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f88e80cf760 msgr2=0x7f88e80ceb50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T15:08:46.567 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:46.566+0000 7f88ef0ec640 1 --2- 172.21.15.43:0/3479778844 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f88e80cf760 0x7f88e80ceb50 secure :-1 s=READY pgs=2214 cs=0 l=1 rev1=1 crypto rx=0x7f88e000c700 tx=0x7f88e000cbc0 comp rx=0 tx=0).stop 2023-08-09T15:08:46.567 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:46.566+0000 7f88ef0ec640 1 -- 172.21.15.43:0/3479778844 shutdown_connections 2023-08-09T15:08:46.568 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:46.566+0000 7f88ef0ec640 1 --2- 172.21.15.43:0/3479778844 >> [v2:172.21.15.203:6832/3082732769,v1:172.21.15.203:6833/3082732769] conn(0x7f88d406a1a0 0x7f88d406c660 unknown :-1 s=CLOSED pgs=60 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T15:08:46.568 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:46.566+0000 7f88ef0ec640 1 --2- 172.21.15.43:0/3479778844 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f88e80cf760 0x7f88e80ceb50 unknown :-1 s=CLOSED pgs=2214 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T15:08:46.568 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:46.566+0000 7f88ef0ec640 1 --2- 172.21.15.43:0/3479778844 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f88e80af6f0 0x7f88e80ce610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T15:08:46.568 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:46.567+0000 7f88ef0ec640 1 --2- 172.21.15.43:0/3479778844 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f88e806b140 0x7f88e8152020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T15:08:46.568 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:46.567+0000 7f88ef0ec640 1 -- 172.21.15.43:0/3479778844 >> 172.21.15.43:0/3479778844 conn(0x7f88e80ccd80 msgr2=0x7f88e80b11e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-09T15:08:46.568 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:46.567+0000 7f88ef0ec640 1 -- 172.21.15.43:0/3479778844 shutdown_connections 2023-08-09T15:08:46.569 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:46.567+0000 7f88ef0ec640 1 -- 172.21.15.43:0/3479778844 wait complete. 2023-08-09T15:08:46.585 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-08-09T15:08:46.808 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:46.807+0000 7f635400d640 1 Processor -- start 2023-08-09T15:08:46.808 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:46.808+0000 7f635400d640 1 -- start start 2023-08-09T15:08:46.809 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:46.809+0000 7f635400d640 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7f634c142ca0 0x7f634c0c96e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T15:08:46.809 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:46.809+0000 7f635400d640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f634c15ce40 0x7f634c0cce00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T15:08:46.810 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:46.809+0000 7f635400d640 1 -- --> v1:172.21.15.203:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f634c143080 con 0x7f634c0cd340 2023-08-09T15:08:46.810 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:46.809+0000 7f635400d640 1 -- --> v2:172.21.15.43:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f634c0cd750 con 0x7f634c15ce40 2023-08-09T15:08:46.810 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:46.809+0000 7f635400d640 1 -- --> v2:172.21.15.203:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f634c06d400 con 0x7f634c142ca0 2023-08-09T15:08:46.811 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:46.809+0000 7f63515a8640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f634c15ce40 0x7f634c0cce00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T15:08:46.811 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:46.810+0000 7f63515a8640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f634c15ce40 0x7f634c0cce00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:39772/0 (socket says 172.21.15.43:39772) 2023-08-09T15:08:46.812 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:46.810+0000 7f63515a8640 1 -- 172.21.15.43:0/320510582 learned_addr learned my addr 172.21.15.43:0/320510582 (peer_addr_for_me v2:172.21.15.43:0/0) 2023-08-09T15:08:46.812 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:46.810+0000 7f6351da9640 1 --2- 172.21.15.43:0/320510582 >> v2:172.21.15.203:3300/0 conn(0x7f634c142ca0 0x7f634c0c96e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T15:08:46.812 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:46.810+0000 7f63515a8640 1 -- 172.21.15.43:0/320510582 >> v2:172.21.15.203:3300/0 conn(0x7f634c142ca0 msgr2=0x7f634c0c96e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-09T15:08:46.812 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:46.810+0000 7f63515a8640 1 --2- 172.21.15.43:0/320510582 >> v2:172.21.15.203:3300/0 conn(0x7f634c142ca0 0x7f634c0c96e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T15:08:46.812 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:46.810+0000 7f63515a8640 1 -- 172.21.15.43:0/320510582 >> v1:172.21.15.203:6789/0 conn(0x7f634c0cd340 legacy=0x7f634c0cbc70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-09T15:08:46.812 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:46.810+0000 7f63515a8640 1 -- 172.21.15.43:0/320510582 --> v2:172.21.15.43:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f634c0c9d90 con 0x7f634c15ce40 2023-08-09T15:08:46.814 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:46.811+0000 7f63515a8640 1 --2- 172.21.15.43:0/320510582 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f634c15ce40 0x7f634c0cce00 secure :-1 s=READY pgs=2215 cs=0 l=1 rev1=1 crypto rx=0x7f634c0c5120 tx=0x7f633c02b910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a55074fe7522c3d4 server_cookie=0 in_seq=0 out_seq=0 2023-08-09T15:08:46.814 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:46.812+0000 7f6350da7640 1 -- 172.21.15.43:0/320510582 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f633c031870 con 0x7f634c15ce40 2023-08-09T15:08:46.814 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:46.812+0000 7f6350da7640 1 -- 172.21.15.43:0/320510582 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f633c0319d0 con 0x7f634c15ce40 2023-08-09T15:08:46.814 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:46.812+0000 7f6350da7640 1 -- 172.21.15.43:0/320510582 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f633c031d60 con 0x7f634c15ce40 2023-08-09T15:08:46.814 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:46.813+0000 7f635400d640 1 -- 172.21.15.43:0/320510582 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f634c15ce40 msgr2=0x7f634c0cce00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T15:08:46.814 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:46.813+0000 7f635400d640 1 --2- 172.21.15.43:0/320510582 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f634c15ce40 0x7f634c0cce00 secure :-1 s=READY pgs=2215 cs=0 l=1 rev1=1 crypto rx=0x7f634c0c5120 tx=0x7f633c02b910 comp rx=0 tx=0).stop 2023-08-09T15:08:46.815 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:46.813+0000 7f635400d640 1 -- 172.21.15.43:0/320510582 shutdown_connections 2023-08-09T15:08:46.815 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:46.813+0000 7f635400d640 1 --2- 172.21.15.43:0/320510582 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f634c15ce40 0x7f634c0cce00 unknown :-1 s=CLOSED pgs=2215 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T15:08:46.815 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:46.813+0000 7f635400d640 1 --2- 172.21.15.43:0/320510582 >> v2:172.21.15.203:3300/0 conn(0x7f634c142ca0 0x7f634c0c96e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T15:08:46.815 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:46.813+0000 7f635400d640 1 -- 172.21.15.43:0/320510582 >> 172.21.15.43:0/320510582 conn(0x7f634c0ce0a0 msgr2=0x7f634c0ce480 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-09T15:08:46.815 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:46.813+0000 7f635400d640 1 -- 172.21.15.43:0/320510582 shutdown_connections 2023-08-09T15:08:46.815 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:46.813+0000 7f635400d640 1 -- 172.21.15.43:0/320510582 wait complete. 2023-08-09T15:08:46.816 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:46.815+0000 7f635400d640 1 Processor -- start 2023-08-09T15:08:46.816 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:46.815+0000 7f635400d640 1 -- start start 2023-08-09T15:08:46.817 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:46.816+0000 7f635400d640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f634c0cd340 0x7f634c13bff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T15:08:46.817 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:46.816+0000 7f635400d640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f634c142ca0 0x7f634c070710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T15:08:46.817 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:46.816+0000 7f635400d640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f634c15ce40 0x7f634c14a490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T15:08:46.817 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:46.816+0000 7f635400d640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f634c0c52f0 con 0x7f634c0cd340 2023-08-09T15:08:46.817 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:46.816+0000 7f635400d640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f634c0c55d0 con 0x7f634c15ce40 2023-08-09T15:08:46.817 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:46.816+0000 7f635400d640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f634c06ff10 con 0x7f634c142ca0 2023-08-09T15:08:46.817 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:46.816+0000 7f6351da9640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f634c0cd340 0x7f634c13bff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T15:08:46.818 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:46.816+0000 7f6351da9640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f634c0cd340 0x7f634c13bff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:39782/0 (socket says 172.21.15.43:39782) 2023-08-09T15:08:46.818 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:46.816+0000 7f6351da9640 1 -- 172.21.15.43:0/145635845 learned_addr learned my addr 172.21.15.43:0/145635845 (peer_addr_for_me v2:172.21.15.43:0/0) 2023-08-09T15:08:46.819 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:46.816+0000 7f6351da9640 1 -- 172.21.15.43:0/145635845 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f634c142ca0 msgr2=0x7f634c070710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T15:08:46.819 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:46.817+0000 7f6351da9640 1 --2- 172.21.15.43:0/145635845 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f634c142ca0 0x7f634c070710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T15:08:46.819 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:46.817+0000 7f6351da9640 1 -- 172.21.15.43:0/145635845 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f634c15ce40 msgr2=0x7f634c14a490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T15:08:46.819 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:46.817+0000 7f6351da9640 1 --2- 172.21.15.43:0/145635845 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f634c15ce40 0x7f634c14a490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T15:08:46.819 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:46.817+0000 7f6351da9640 1 -- 172.21.15.43:0/145635845 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f633c0094d0 con 0x7f634c0cd340 2023-08-09T15:08:46.820 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:46.817+0000 7f6351da9640 1 --2- 172.21.15.43:0/145635845 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f634c0cd340 0x7f634c13bff0 secure :-1 s=READY pgs=2216 cs=0 l=1 rev1=1 crypto rx=0x7f6348007ef0 tx=0x7f634800e4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-09T15:08:46.820 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:46.817+0000 7f63427fc640 1 -- 172.21.15.43:0/145635845 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6348014620 con 0x7f634c0cd340 2023-08-09T15:08:46.820 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:46.817+0000 7f63427fc640 1 -- 172.21.15.43:0/145635845 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6348014780 con 0x7f634c0cd340 2023-08-09T15:08:46.820 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:46.817+0000 7f63427fc640 1 -- 172.21.15.43:0/145635845 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6348014b30 con 0x7f634c0cd340 2023-08-09T15:08:46.820 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:46.817+0000 7f635400d640 1 -- 172.21.15.43:0/145635845 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f634c0701f0 con 0x7f634c0cd340 2023-08-09T15:08:46.820 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:46.817+0000 7f635400d640 1 -- 172.21.15.43:0/145635845 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f634c070c50 con 0x7f634c0cd340 2023-08-09T15:08:46.820 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:46.819+0000 7f635400d640 1 -- 172.21.15.43:0/145635845 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f634c0739f0 con 0x7f634c0cd340 2023-08-09T15:08:46.822 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:46.820+0000 7f63427fc640 1 -- 172.21.15.43:0/145635845 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 106) v1 ==== 88443+0+0 (secure 0 0 0) 0x7f6348014c90 con 0x7f634c0cd340 2023-08-09T15:08:46.822 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:46.821+0000 7f63427fc640 1 --2- 172.21.15.43:0/145635845 >> [v2:172.21.15.203:6832/3082732769,v1:172.21.15.203:6833/3082732769] conn(0x7f633006a170 0x7f633006c630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T15:08:46.822 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:46.821+0000 7f63427fc640 1 -- 172.21.15.43:0/145635845 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(204..204 src has 1..204) v4 ==== 6426+0+0 (secure 0 0 0) 0x7f6348089340 con 0x7f634c0cd340 2023-08-09T15:08:46.823 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:46.822+0000 7f63515a8640 1 --2- 172.21.15.43:0/145635845 >> [v2:172.21.15.203:6832/3082732769,v1:172.21.15.203:6833/3082732769] conn(0x7f633006a170 0x7f633006c630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T15:08:46.823 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:46.823+0000 7f63515a8640 1 --2- 172.21.15.43:0/145635845 >> [v2:172.21.15.203:6832/3082732769,v1:172.21.15.203:6833/3082732769] conn(0x7f633006a170 0x7f633006c630 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f633c036000 tx=0x7f633c030000 comp rx=0 tx=0).ready entity=mgr.8500 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-09T15:08:46.826 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:46.826+0000 7f63427fc640 1 -- 172.21.15.43:0/145635845 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178066 (secure 0 0 0) 0x7f6348054720 con 0x7f634c0cd340 2023-08-09T15:08:47.018 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:47.018+0000 7f635400d640 1 -- 172.21.15.43:0/145635845 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f634c0c5430 con 0x7f634c0cd340 2023-08-09T15:08:47.020 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:47.020+0000 7f63427fc640 1 -- 172.21.15.43:0/145635845 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v204) v1 ==== 74+0+18579 (secure 0 0 0) 0x7f634c0c5430 con 0x7f634c0cd340 2023-08-09T15:08:47.021 INFO:teuthology.orchestra.run.smithi043.stdout: 2023-08-09T15:08:47.021 INFO:teuthology.orchestra.run.smithi043.stdout:{"epoch":204,"fsid":"daac87ee-1337-4250-a955-aea4e408a39b","created":"2023-08-09T13:33:41.701198+0000","modified":"2023-08-09T15:08:45.562226+0000","last_up_change":"2023-08-09T13:33:53.783453+0000","last_in_change":"2023-08-09T13:33:45.148904+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":25,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":"rbd","create_time":"2023-08-09T13:33:54.365137+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":".mgr","create_time":"2023-08-09T13:33:54.831803+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":24,"pool_name":"cephfs_metadata","create_time":"2023-08-09T15:08:43.920337+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"204","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":25,"pool_name":"cephfs_data","create_time":"2023-08-09T15:08:44.594489+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"204","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"cbd49ecd-e896-4fbd-864e-5c37a829b165","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":203,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6800","nonce":502667335},{"type":"v1","addr":"172.21.15.43:6801","nonce":502667335}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6802","nonce":502667335},{"type":"v1","addr":"172.21.15.43:6803","nonce":502667335}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6806","nonce":502667335},{"type":"v1","addr":"172.21.15.43:6807","nonce":502667335}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6804","nonce":502667335},{"type":"v1","addr":"172.21.15.43:6805","nonce":502667335}]},"public_addr":"172.21.15.43:6801/502667335","cluster_addr":"172.21.15.43:6803/502667335","heartbeat_back_addr":"172.21.15.43:6807/502667335","heartbeat_front_addr":"172.21.15.43:6805/502667335","state":["exists","up"]},{"osd":1,"uuid":"75c7e0fe-1ec7-4fe8-a2f2-3661a65b8500","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":203,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6808","nonce":3289134610},{"type":"v1","addr":"172.21.15.43:6809","nonce":3289134610}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6810","nonce":3289134610},{"type":"v1","addr":"172.21.15.43:6812","nonce":3289134610}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6821","nonce":3289134610},{"type":"v1","addr":"172.21.15.43:6824","nonce":3289134610}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6815","nonce":3289134610},{"type":"v1","addr":"172.21.15.43:6818","nonce":3289134610}]},"public_addr":"172.21.15.43:6809/3289134610","cluster_addr":"172.21.15.43:6812/3289134610","heartbeat_back_addr":"172.21.15.43:6824/3289134610","heartbeat_front_addr":"172.21.15.43:6818/3289134610","state":["exists","up"]},{"osd":2,"uuid":"bb5c41e9-7f5f-4125-9b01-3bb8bfc2ea83","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":203,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6811","nonce":4079709736},{"type":"v1","addr":"172.21.15.43:6814","nonce":4079709736}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6817","nonce":4079709736},{"type":"v1","addr":"172.21.15.43:6820","nonce":4079709736}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6829","nonce":4079709736},{"type":"v1","addr":"172.21.15.43:6831","nonce":4079709736}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6823","nonce":4079709736},{"type":"v1","addr":"172.21.15.43:6826","nonce":4079709736}]},"public_addr":"172.21.15.43:6814/4079709736","cluster_addr":"172.21.15.43:6820/4079709736","heartbeat_back_addr":"172.21.15.43:6831/4079709736","heartbeat_front_addr":"172.21.15.43:6826/4079709736","state":["exists","up"]},{"osd":3,"uuid":"bfe4c8b4-694a-465c-87e6-2b2aab852613","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":203,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6813","nonce":1709538759},{"type":"v1","addr":"172.21.15.43:6816","nonce":1709538759}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6819","nonce":1709538759},{"type":"v1","addr":"172.21.15.43:6822","nonce":1709538759}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6828","nonce":1709538759},{"type":"v1","addr":"172.21.15.43:6830","nonce":1709538759}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6825","nonce":1709538759},{"type":"v1","addr":"172.21.15.43:6827","nonce":1709538759}]},"public_addr":"172.21.15.43:6816/1709538759","cluster_addr":"172.21.15.43:6822/1709538759","heartbeat_back_addr":"172.21.15.43:6830/1709538759","heartbeat_front_addr":"172.21.15.43:6827/1709538759","state":["exists","up"]},{"osd":4,"uuid":"fa2e8a77-aba1-4756-853e-0d174a5bdeba","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":203,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6800","nonce":4196866481},{"type":"v1","addr":"172.21.15.203:6801","nonce":4196866481}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6802","nonce":4196866481},{"type":"v1","addr":"172.21.15.203:6803","nonce":4196866481}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6806","nonce":4196866481},{"type":"v1","addr":"172.21.15.203:6807","nonce":4196866481}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6804","nonce":4196866481},{"type":"v1","addr":"172.21.15.203:6805","nonce":4196866481}]},"public_addr":"172.21.15.203:6801/4196866481","cluster_addr":"172.21.15.203:6803/4196866481","heartbeat_back_addr":"172.21.15.203:6807/4196866481","heartbeat_front_addr":"172.21.15.203:6805/4196866481","state":["exists","up"]},{"osd":5,"uuid":"81811e4b-4d24-4f51-a82c-f1d700a2a00d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":203,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6816","nonce":3059290081},{"type":"v1","addr":"172.21.15.203:6817","nonce":3059290081}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6818","nonce":3059290081},{"type":"v1","addr":"172.21.15.203:6819","nonce":3059290081}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6822","nonce":3059290081},{"type":"v1","addr":"172.21.15.203:6823","nonce":3059290081}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6820","nonce":3059290081},{"type":"v1","addr":"172.21.15.203:6821","nonce":3059290081}]},"public_addr":"172.21.15.203:6817/3059290081","cluster_addr":"172.21.15.203:6819/3059290081","heartbeat_back_addr":"172.21.15.203:6823/3059290081","heartbeat_front_addr":"172.21.15.203:6821/3059290081","state":["exists","up"]},{"osd":6,"uuid":"307bac4d-6a5b-443d-af75-56d695d7674f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":203,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6824","nonce":512155569},{"type":"v1","addr":"172.21.15.203:6825","nonce":512155569}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6826","nonce":512155569},{"type":"v1","addr":"172.21.15.203:6827","nonce":512155569}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6830","nonce":512155569},{"type":"v1","addr":"172.21.15.203:6831","nonce":512155569}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6828","nonce":512155569},{"type":"v1","addr":"172.21.15.203:6829","nonce":512155569}]},"public_addr":"172.21.15.203:6825/512155569","cluster_addr":"172.21.15.203:6827/512155569","heartbeat_back_addr":"172.21.15.203:6831/512155569","heartbeat_front_addr":"172.21.15.203:6829/512155569","state":["exists","up"]},{"osd":7,"uuid":"a95dc7ca-9f1b-4295-af70-68a22a2d7d5e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":203,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6808","nonce":3331450438},{"type":"v1","addr":"172.21.15.203:6809","nonce":3331450438}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6810","nonce":3331450438},{"type":"v1","addr":"172.21.15.203:6811","nonce":3331450438}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6814","nonce":3331450438},{"type":"v1","addr":"172.21.15.203:6815","nonce":3331450438}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6812","nonce":3331450438},{"type":"v1","addr":"172.21.15.203:6813","nonce":3331450438}]},"public_addr":"172.21.15.203:6809/3331450438","cluster_addr":"172.21.15.203:6811/3331450438","heartbeat_back_addr":"172.21.15.203:6815/3331450438","heartbeat_front_addr":"172.21.15.203:6813/3331450438","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-09T13:33:50.547652+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-09T13:33:50.589469+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-09T13:33:50.561724+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-09T13:33:50.599650+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-09T13:33:50.411855+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-09T13:33:50.219681+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-09T13:33:50.504664+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-09T13:33:50.441248+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":7,"snaps":[{"begin":104,"length":1},{"begin":221,"length":1}]}],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-08-09T15:08:47.023 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:47.023+0000 7f635400d640 1 -- 172.21.15.43:0/145635845 >> [v2:172.21.15.203:6832/3082732769,v1:172.21.15.203:6833/3082732769] conn(0x7f633006a170 msgr2=0x7f633006c630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T15:08:47.023 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:47.023+0000 7f635400d640 1 --2- 172.21.15.43:0/145635845 >> [v2:172.21.15.203:6832/3082732769,v1:172.21.15.203:6833/3082732769] conn(0x7f633006a170 0x7f633006c630 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f633c036000 tx=0x7f633c030000 comp rx=0 tx=0).stop 2023-08-09T15:08:47.023 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:47.023+0000 7f635400d640 1 -- 172.21.15.43:0/145635845 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f634c0cd340 msgr2=0x7f634c13bff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T15:08:47.024 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:47.023+0000 7f635400d640 1 --2- 172.21.15.43:0/145635845 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f634c0cd340 0x7f634c13bff0 secure :-1 s=READY pgs=2216 cs=0 l=1 rev1=1 crypto rx=0x7f6348007ef0 tx=0x7f634800e4b0 comp rx=0 tx=0).stop 2023-08-09T15:08:47.024 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:47.023+0000 7f635400d640 1 -- 172.21.15.43:0/145635845 shutdown_connections 2023-08-09T15:08:47.024 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:47.023+0000 7f635400d640 1 --2- 172.21.15.43:0/145635845 >> [v2:172.21.15.203:6832/3082732769,v1:172.21.15.203:6833/3082732769] conn(0x7f633006a170 0x7f633006c630 unknown :-1 s=CLOSED pgs=61 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T15:08:47.024 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:47.023+0000 7f635400d640 1 --2- 172.21.15.43:0/145635845 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f634c15ce40 0x7f634c14a490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T15:08:47.024 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:47.023+0000 7f635400d640 1 --2- 172.21.15.43:0/145635845 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f634c142ca0 0x7f634c070710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T15:08:47.024 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:47.023+0000 7f635400d640 1 --2- 172.21.15.43:0/145635845 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f634c0cd340 0x7f634c13bff0 unknown :-1 s=CLOSED pgs=2216 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T15:08:47.024 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:47.023+0000 7f635400d640 1 -- 172.21.15.43:0/145635845 >> 172.21.15.43:0/145635845 conn(0x7f634c0ce0a0 msgr2=0x7f634c13caf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-09T15:08:47.025 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:47.023+0000 7f635400d640 1 -- 172.21.15.43:0/145635845 shutdown_connections 2023-08-09T15:08:47.025 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:47.023+0000 7f635400d640 1 -- 172.21.15.43:0/145635845 wait complete. 2023-08-09T15:08:47.043 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs standby_count_wanted 0 2023-08-09T15:08:47.276 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:47.276+0000 7f325f899640 1 Processor -- start 2023-08-09T15:08:47.276 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:47.276+0000 7f325f899640 1 -- start start 2023-08-09T15:08:47.278 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:47.278+0000 7f325f899640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f32580d0a70 0x7f32580cd7f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T15:08:47.278 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:47.278+0000 7f325f899640 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7f32580cdd30 0x7f32580b3fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T15:08:47.278 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:47.278+0000 7f325f899640 1 -- --> v1:172.21.15.43:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f32580d0f70 con 0x7f325815c680 2023-08-09T15:08:47.278 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:47.278+0000 7f325f899640 1 -- --> v2:172.21.15.43:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f32580ccd10 con 0x7f32580d0a70 2023-08-09T15:08:47.278 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:47.278+0000 7f325f899640 1 -- --> v2:172.21.15.203:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f32580ccff0 con 0x7f32580cdd30 2023-08-09T15:08:47.279 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:47.278+0000 7f325d635640 1 --1- >> v1:172.21.15.43:6789/0 conn(0x7f325815c680 0x7f32580ce760 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.43:6789/0 says I am v1:172.21.15.43:42132/0 (socket says 172.21.15.43:42132) 2023-08-09T15:08:47.279 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:47.278+0000 7f325d635640 1 -- 172.21.15.43:0/3560413637 learned_addr learned my addr 172.21.15.43:0/3560413637 (peer_addr_for_me v1:172.21.15.43:0/0) 2023-08-09T15:08:47.279 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:47.278+0000 7f325ce34640 1 --2- 172.21.15.43:0/3560413637 >> v2:172.21.15.43:3300/0 conn(0x7f32580d0a70 0x7f32580cd7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T15:08:47.280 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:47.278+0000 7f325de36640 1 --2- 172.21.15.43:0/3560413637 >> v2:172.21.15.203:3300/0 conn(0x7f32580cdd30 0x7f32580b3fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T15:08:47.280 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:47.279+0000 7f325ce34640 1 -- 172.21.15.43:0/3560413637 >> v2:172.21.15.203:3300/0 conn(0x7f32580cdd30 msgr2=0x7f32580b3fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-09T15:08:47.280 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:47.279+0000 7f325ce34640 1 --2- 172.21.15.43:0/3560413637 >> v2:172.21.15.203:3300/0 conn(0x7f32580cdd30 0x7f32580b3fa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T15:08:47.280 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:47.279+0000 7f325ce34640 1 -- 172.21.15.43:0/3560413637 >> v1:172.21.15.43:6789/0 conn(0x7f325815c680 legacy=0x7f32580ce760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T15:08:47.280 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:47.279+0000 7f325ce34640 1 -- 172.21.15.43:0/3560413637 --> v2:172.21.15.43:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f32580cd2d0 con 0x7f32580d0a70 2023-08-09T15:08:47.280 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:47.279+0000 7f324ffff640 1 -- 172.21.15.43:0/3560413637 <== mon.0 v1:172.21.15.43:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3657734104 0 0) 0x7f32580d0f70 con 0x7f325815c680 2023-08-09T15:08:47.281 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:47.280+0000 7f325ce34640 1 --2- 172.21.15.43:0/3560413637 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f32580d0a70 0x7f32580cd7f0 secure :-1 s=READY pgs=2218 cs=0 l=1 rev1=1 crypto rx=0x7f325806b9f0 tx=0x7f324802b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=45788f31c62106aa server_cookie=0 in_seq=0 out_seq=0 2023-08-09T15:08:47.281 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:47.280+0000 7f324ffff640 1 -- 172.21.15.43:0/3560413637 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3248031830 con 0x7f32580d0a70 2023-08-09T15:08:47.282 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:47.280+0000 7f324ffff640 1 -- 172.21.15.43:0/3560413637 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3248031990 con 0x7f32580d0a70 2023-08-09T15:08:47.282 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:47.280+0000 7f324ffff640 1 -- 172.21.15.43:0/3560413637 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3248031d20 con 0x7f32580d0a70 2023-08-09T15:08:47.282 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:47.281+0000 7f325f899640 1 -- 172.21.15.43:0/3560413637 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f32580d0a70 msgr2=0x7f32580cd7f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T15:08:47.282 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:47.281+0000 7f325f899640 1 --2- 172.21.15.43:0/3560413637 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f32580d0a70 0x7f32580cd7f0 secure :-1 s=READY pgs=2218 cs=0 l=1 rev1=1 crypto rx=0x7f325806b9f0 tx=0x7f324802b8d0 comp rx=0 tx=0).stop 2023-08-09T15:08:47.282 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:47.281+0000 7f325f899640 1 -- 172.21.15.43:0/3560413637 shutdown_connections 2023-08-09T15:08:47.282 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:47.281+0000 7f325f899640 1 --2- 172.21.15.43:0/3560413637 >> v2:172.21.15.203:3300/0 conn(0x7f32580cdd30 0x7f32580b3fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T15:08:47.282 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:47.281+0000 7f325f899640 1 --2- 172.21.15.43:0/3560413637 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f32580d0a70 0x7f32580cd7f0 unknown :-1 s=CLOSED pgs=2218 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T15:08:47.283 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:47.281+0000 7f325f899640 1 -- 172.21.15.43:0/3560413637 >> 172.21.15.43:0/3560413637 conn(0x7f3258071520 msgr2=0x7f32580d2520 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-09T15:08:47.283 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:47.281+0000 7f325f899640 1 -- 172.21.15.43:0/3560413637 shutdown_connections 2023-08-09T15:08:47.283 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:47.281+0000 7f325f899640 1 -- 172.21.15.43:0/3560413637 wait complete. 2023-08-09T15:08:47.283 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:47.283+0000 7f325f899640 1 Processor -- start 2023-08-09T15:08:47.283 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:47.283+0000 7f325f899640 1 -- start start 2023-08-09T15:08:47.284 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:47.284+0000 7f325f899640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f32580d0a70 0x7f32581f2700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T15:08:47.284 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:47.284+0000 7f325f899640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f325815c680 0x7f32580cba30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T15:08:47.285 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:47.284+0000 7f325f899640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f32581f4700 0x7f32581f4bb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T15:08:47.285 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:47.284+0000 7f325f899640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f32581f50f0 con 0x7f32581f4700 2023-08-09T15:08:47.285 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:47.284+0000 7f325f899640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f325815d450 con 0x7f325815c680 2023-08-09T15:08:47.285 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:47.284+0000 7f325f899640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f325815da10 con 0x7f32580d0a70 2023-08-09T15:08:47.285 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:47.284+0000 7f325de36640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f32581f4700 0x7f32581f4bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T15:08:47.285 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:47.284+0000 7f325de36640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f32581f4700 0x7f32581f4bb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:39806/0 (socket says 172.21.15.43:39806) 2023-08-09T15:08:47.285 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:47.284+0000 7f325de36640 1 -- 172.21.15.43:0/2928243008 learned_addr learned my addr 172.21.15.43:0/2928243008 (peer_addr_for_me v2:172.21.15.43:0/0) 2023-08-09T15:08:47.286 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:47.285+0000 7f325ce34640 1 --2- 172.21.15.43:0/2928243008 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f325815c680 0x7f32580cba30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T15:08:47.287 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:47.285+0000 7f325d635640 1 --2- 172.21.15.43:0/2928243008 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f32580d0a70 0x7f32581f2700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T15:08:47.287 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:47.285+0000 7f325de36640 1 -- 172.21.15.43:0/2928243008 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f32580d0a70 msgr2=0x7f32581f2700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T15:08:47.287 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:47.285+0000 7f325de36640 1 --2- 172.21.15.43:0/2928243008 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f32580d0a70 0x7f32581f2700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T15:08:47.287 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:47.285+0000 7f325de36640 1 -- 172.21.15.43:0/2928243008 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f325815c680 msgr2=0x7f32580cba30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T15:08:47.287 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:47.285+0000 7f325de36640 1 --2- 172.21.15.43:0/2928243008 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f325815c680 0x7f32580cba30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T15:08:47.287 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:47.285+0000 7f325de36640 1 -- 172.21.15.43:0/2928243008 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3248009400 con 0x7f32581f4700 2023-08-09T15:08:47.287 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:47.285+0000 7f325ce34640 1 --2- 172.21.15.43:0/2928243008 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f325815c680 0x7f32580cba30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-08-09T15:08:47.287 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:47.285+0000 7f325de36640 1 --2- 172.21.15.43:0/2928243008 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f32581f4700 0x7f32581f4bb0 secure :-1 s=READY pgs=2219 cs=0 l=1 rev1=1 crypto rx=0x7f3250007e60 tx=0x7f325000e4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-09T15:08:47.288 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:47.285+0000 7f324dffb640 1 -- 172.21.15.43:0/2928243008 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3250014620 con 0x7f32581f4700 2023-08-09T15:08:47.288 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:47.285+0000 7f324dffb640 1 -- 172.21.15.43:0/2928243008 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3250014780 con 0x7f32581f4700 2023-08-09T15:08:47.288 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:47.285+0000 7f324dffb640 1 -- 172.21.15.43:0/2928243008 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3250014b30 con 0x7f32581f4700 2023-08-09T15:08:47.288 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:47.285+0000 7f325f899640 1 -- 172.21.15.43:0/2928243008 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f32581f2c40 con 0x7f32581f4700 2023-08-09T15:08:47.288 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:47.286+0000 7f325f899640 1 -- 172.21.15.43:0/2928243008 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f325815f860 con 0x7f32581f4700 2023-08-09T15:08:47.289 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:47.287+0000 7f325f899640 1 -- 172.21.15.43:0/2928243008 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f325815dba0 con 0x7f32581f4700 2023-08-09T15:08:47.289 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:47.288+0000 7f324dffb640 1 -- 172.21.15.43:0/2928243008 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 106) v1 ==== 88443+0+0 (secure 0 0 0) 0x7f3250014c90 con 0x7f32581f4700 2023-08-09T15:08:47.289 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:47.289+0000 7f324dffb640 1 --2- 172.21.15.43:0/2928243008 >> [v2:172.21.15.203:6832/3082732769,v1:172.21.15.203:6833/3082732769] conn(0x7f323c06a170 0x7f323c06c630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T15:08:47.289 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:47.289+0000 7f324dffb640 1 -- 172.21.15.43:0/2928243008 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(205..205 src has 1..205) v4 ==== 6426+0+0 (secure 0 0 0) 0x7f3250089270 con 0x7f32581f4700 2023-08-09T15:08:47.290 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:47.289+0000 7f325d635640 1 --2- 172.21.15.43:0/2928243008 >> [v2:172.21.15.203:6832/3082732769,v1:172.21.15.203:6833/3082732769] conn(0x7f323c06a170 0x7f323c06c630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T15:08:47.291 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:47.291+0000 7f325d635640 1 --2- 172.21.15.43:0/2928243008 >> [v2:172.21.15.203:6832/3082732769,v1:172.21.15.203:6833/3082732769] conn(0x7f323c06a170 0x7f323c06c630 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f3254000f60 tx=0x7f3254009040 comp rx=0 tx=0).ready entity=mgr.8500 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-09T15:08:47.293 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:47.294+0000 7f324dffb640 1 -- 172.21.15.43:0/2928243008 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178066 (secure 0 0 0) 0x7f32500546d0 con 0x7f32581f4700 2023-08-09T15:08:47.523 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:47.522+0000 7f325f899640 1 -- 172.21.15.43:0/2928243008 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} v 0) v1 -- 0x7f32580739f0 con 0x7f32581f4700 2023-08-09T15:08:48.145 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:48.144+0000 7f324dffb640 1 -- 172.21.15.43:0/2928243008 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]=0 v95) v1 ==== 118+0+0 (secure 0 0 0) 0x7f3250057cf0 con 0x7f32581f4700 2023-08-09T15:08:48.149 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:48.148+0000 7f325f899640 1 -- 172.21.15.43:0/2928243008 >> [v2:172.21.15.203:6832/3082732769,v1:172.21.15.203:6833/3082732769] conn(0x7f323c06a170 msgr2=0x7f323c06c630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T15:08:48.149 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:48.148+0000 7f325f899640 1 --2- 172.21.15.43:0/2928243008 >> [v2:172.21.15.203:6832/3082732769,v1:172.21.15.203:6833/3082732769] conn(0x7f323c06a170 0x7f323c06c630 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f3254000f60 tx=0x7f3254009040 comp rx=0 tx=0).stop 2023-08-09T15:08:48.149 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:48.148+0000 7f325f899640 1 -- 172.21.15.43:0/2928243008 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f32581f4700 msgr2=0x7f32581f4bb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T15:08:48.150 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:48.148+0000 7f325f899640 1 --2- 172.21.15.43:0/2928243008 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f32581f4700 0x7f32581f4bb0 secure :-1 s=READY pgs=2219 cs=0 l=1 rev1=1 crypto rx=0x7f3250007e60 tx=0x7f325000e4b0 comp rx=0 tx=0).stop 2023-08-09T15:08:48.150 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:48.148+0000 7f325f899640 1 -- 172.21.15.43:0/2928243008 shutdown_connections 2023-08-09T15:08:48.150 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:48.148+0000 7f325f899640 1 --2- 172.21.15.43:0/2928243008 >> [v2:172.21.15.203:6832/3082732769,v1:172.21.15.203:6833/3082732769] conn(0x7f323c06a170 0x7f323c06c630 unknown :-1 s=CLOSED pgs=62 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T15:08:48.150 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:48.148+0000 7f325f899640 1 --2- 172.21.15.43:0/2928243008 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f32581f4700 0x7f32581f4bb0 unknown :-1 s=CLOSED pgs=2219 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T15:08:48.150 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:48.148+0000 7f325f899640 1 --2- 172.21.15.43:0/2928243008 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f325815c680 0x7f32580cba30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T15:08:48.150 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:48.148+0000 7f325f899640 1 --2- 172.21.15.43:0/2928243008 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f32580d0a70 0x7f32581f2700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T15:08:48.150 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:48.148+0000 7f325f899640 1 -- 172.21.15.43:0/2928243008 >> 172.21.15.43:0/2928243008 conn(0x7f3258071520 msgr2=0x7f32580ce350 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-09T15:08:48.150 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:48.148+0000 7f325f899640 1 -- 172.21.15.43:0/2928243008 shutdown_connections 2023-08-09T15:08:48.151 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:48.149+0000 7f325f899640 1 -- 172.21.15.43:0/2928243008 wait complete. 2023-08-09T15:08:48.172 DEBUG:tasks.cephfs.filesystem:fs_config: {} 2023-08-09T15:08:48.172 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs allow_standby_replay false 2023-08-09T15:08:48.407 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:48.407+0000 7fd4f184d640 1 Processor -- start 2023-08-09T15:08:48.407 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:48.407+0000 7fd4f184d640 1 -- start start 2023-08-09T15:08:48.409 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:48.409+0000 7fd4f184d640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7fd4ec0c86f0 0x7fd4ec0c8b50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T15:08:48.409 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:48.409+0000 7fd4f184d640 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7fd4ec06c8d0 0x7fd4ec06cd00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T15:08:48.409 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:48.409+0000 7fd4f184d640 1 -- --> v1:172.21.15.43:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd4ec0c9090 con 0x7fd4ec143c20 2023-08-09T15:08:48.409 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:48.409+0000 7fd4f184d640 1 -- --> v2:172.21.15.43:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd4ec144000 con 0x7fd4ec0c86f0 2023-08-09T15:08:48.409 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:48.409+0000 7fd4f184d640 1 -- --> v2:172.21.15.203:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd4ec06c440 con 0x7fd4ec06c8d0 2023-08-09T15:08:48.410 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:48.409+0000 7fd4ea7fc640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7fd4ec0c86f0 0x7fd4ec0c8b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T15:08:48.410 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:48.409+0000 7fd4eaffd640 1 --1- >> v1:172.21.15.43:6789/0 conn(0x7fd4ec143c20 0x7fd4ec06f540 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.43:6789/0 says I am v1:172.21.15.43:42142/0 (socket says 172.21.15.43:42142) 2023-08-09T15:08:48.411 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:48.409+0000 7fd4eaffd640 1 -- 172.21.15.43:0/3245563168 learned_addr learned my addr 172.21.15.43:0/3245563168 (peer_addr_for_me v1:172.21.15.43:0/0) 2023-08-09T15:08:48.411 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:48.410+0000 7fd4eb7fe640 1 --2- 172.21.15.43:0/3245563168 >> v2:172.21.15.203:3300/0 conn(0x7fd4ec06c8d0 0x7fd4ec06cd00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T15:08:48.411 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:48.410+0000 7fd4ea7fc640 1 -- 172.21.15.43:0/3245563168 >> v2:172.21.15.203:3300/0 conn(0x7fd4ec06c8d0 msgr2=0x7fd4ec06cd00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-09T15:08:48.411 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:48.410+0000 7fd4ea7fc640 1 --2- 172.21.15.43:0/3245563168 >> v2:172.21.15.203:3300/0 conn(0x7fd4ec06c8d0 0x7fd4ec06cd00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T15:08:48.411 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:48.410+0000 7fd4ea7fc640 1 -- 172.21.15.43:0/3245563168 >> v1:172.21.15.43:6789/0 conn(0x7fd4ec143c20 legacy=0x7fd4ec06f540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T15:08:48.411 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:48.410+0000 7fd4ea7fc640 1 -- 172.21.15.43:0/3245563168 --> v2:172.21.15.43:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd4ec06fc50 con 0x7fd4ec0c86f0 2023-08-09T15:08:48.411 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:48.410+0000 7fd4e9ffb640 1 -- 172.21.15.43:0/3245563168 <== mon.0 v1:172.21.15.43:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1415764805 0 0) 0x7fd4ec0c9090 con 0x7fd4ec143c20 2023-08-09T15:08:48.412 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:48.411+0000 7fd4ea7fc640 1 --2- 172.21.15.43:0/3245563168 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fd4ec0c86f0 0x7fd4ec0c8b50 secure :-1 s=READY pgs=2221 cs=0 l=1 rev1=1 crypto rx=0x7fd4ec0caad0 tx=0x7fd4e002b840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=487ea147c1238620 server_cookie=0 in_seq=0 out_seq=0 2023-08-09T15:08:48.413 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:48.411+0000 7fd4e9ffb640 1 -- 172.21.15.43:0/3245563168 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd4e002e070 con 0x7fd4ec0c86f0 2023-08-09T15:08:48.413 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:48.411+0000 7fd4e9ffb640 1 -- 172.21.15.43:0/3245563168 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd4e00315a0 con 0x7fd4ec0c86f0 2023-08-09T15:08:48.413 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:48.411+0000 7fd4e9ffb640 1 -- 172.21.15.43:0/3245563168 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd4e0031930 con 0x7fd4ec0c86f0 2023-08-09T15:08:48.413 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:48.412+0000 7fd4f184d640 1 -- 172.21.15.43:0/3245563168 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fd4ec0c86f0 msgr2=0x7fd4ec0c8b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T15:08:48.413 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:48.412+0000 7fd4f184d640 1 --2- 172.21.15.43:0/3245563168 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fd4ec0c86f0 0x7fd4ec0c8b50 secure :-1 s=READY pgs=2221 cs=0 l=1 rev1=1 crypto rx=0x7fd4ec0caad0 tx=0x7fd4e002b840 comp rx=0 tx=0).stop 2023-08-09T15:08:48.413 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:48.412+0000 7fd4f184d640 1 -- 172.21.15.43:0/3245563168 shutdown_connections 2023-08-09T15:08:48.413 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:48.412+0000 7fd4f184d640 1 --2- 172.21.15.43:0/3245563168 >> v2:172.21.15.203:3300/0 conn(0x7fd4ec06c8d0 0x7fd4ec06cd00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T15:08:48.414 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:48.412+0000 7fd4f184d640 1 --2- 172.21.15.43:0/3245563168 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fd4ec0c86f0 0x7fd4ec0c8b50 unknown :-1 s=CLOSED pgs=2221 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T15:08:48.414 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:48.412+0000 7fd4f184d640 1 -- 172.21.15.43:0/3245563168 >> 172.21.15.43:0/3245563168 conn(0x7fd4ec0ce0a0 msgr2=0x7fd4ec0ce480 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-09T15:08:48.414 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:48.412+0000 7fd4f184d640 1 -- 172.21.15.43:0/3245563168 shutdown_connections 2023-08-09T15:08:48.414 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:48.412+0000 7fd4f184d640 1 -- 172.21.15.43:0/3245563168 wait complete. 2023-08-09T15:08:48.415 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:48.414+0000 7fd4f184d640 1 Processor -- start 2023-08-09T15:08:48.415 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:48.414+0000 7fd4f184d640 1 -- start start 2023-08-09T15:08:48.415 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:48.415+0000 7fd4f184d640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fd4ec06c8d0 0x7fd4ec0b31e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T15:08:48.415 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:48.415+0000 7fd4f184d640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fd4ec0c86f0 0x7fd4ec0b0d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T15:08:48.415 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:48.415+0000 7fd4f184d640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fd4ec143c20 0x7fd4ec0c6140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T15:08:48.415 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:48.415+0000 7fd4f184d640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd4ec0c5170 con 0x7fd4ec143c20 2023-08-09T15:08:48.416 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:48.415+0000 7fd4f184d640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd4ec0c5380 con 0x7fd4ec06c8d0 2023-08-09T15:08:48.416 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:48.415+0000 7fd4f184d640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd4ec0c5660 con 0x7fd4ec0c86f0 2023-08-09T15:08:48.417 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:48.415+0000 7fd4eb7fe640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fd4ec143c20 0x7fd4ec0c6140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T15:08:48.417 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:48.415+0000 7fd4eb7fe640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fd4ec143c20 0x7fd4ec0c6140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:39822/0 (socket says 172.21.15.43:39822) 2023-08-09T15:08:48.417 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:48.415+0000 7fd4eb7fe640 1 -- 172.21.15.43:0/2134935459 learned_addr learned my addr 172.21.15.43:0/2134935459 (peer_addr_for_me v2:172.21.15.43:0/0) 2023-08-09T15:08:48.417 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:48.415+0000 7fd4eaffd640 1 --2- 172.21.15.43:0/2134935459 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fd4ec06c8d0 0x7fd4ec0b31e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T15:08:48.418 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:48.415+0000 7fd4ea7fc640 1 --2- 172.21.15.43:0/2134935459 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fd4ec0c86f0 0x7fd4ec0b0d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T15:08:48.418 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:48.415+0000 7fd4eb7fe640 1 -- 172.21.15.43:0/2134935459 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fd4ec0c86f0 msgr2=0x7fd4ec0b0d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T15:08:48.418 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:48.415+0000 7fd4eb7fe640 1 --2- 172.21.15.43:0/2134935459 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fd4ec0c86f0 0x7fd4ec0b0d80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T15:08:48.418 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:48.415+0000 7fd4eb7fe640 1 -- 172.21.15.43:0/2134935459 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fd4ec06c8d0 msgr2=0x7fd4ec0b31e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T15:08:48.418 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:48.415+0000 7fd4eb7fe640 1 --2- 172.21.15.43:0/2134935459 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fd4ec06c8d0 0x7fd4ec0b31e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T15:08:48.418 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:48.416+0000 7fd4eb7fe640 1 -- 172.21.15.43:0/2134935459 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd4e0009400 con 0x7fd4ec143c20 2023-08-09T15:08:48.418 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:48.416+0000 7fd4ea7fc640 1 --2- 172.21.15.43:0/2134935459 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fd4ec0c86f0 0x7fd4ec0b0d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-08-09T15:08:48.419 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:48.416+0000 7fd4eb7fe640 1 --2- 172.21.15.43:0/2134935459 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fd4ec143c20 0x7fd4ec0c6140 secure :-1 s=READY pgs=2222 cs=0 l=1 rev1=1 crypto rx=0x7fd4e400d490 tx=0x7fd4e400d950 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-09T15:08:48.419 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:48.416+0000 7fd4d3fff640 1 -- 172.21.15.43:0/2134935459 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd4e40125f0 con 0x7fd4ec143c20 2023-08-09T15:08:48.419 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:48.416+0000 7fd4d3fff640 1 -- 172.21.15.43:0/2134935459 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd4e4012770 con 0x7fd4ec143c20 2023-08-09T15:08:48.419 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:48.416+0000 7fd4f184d640 1 -- 172.21.15.43:0/2134935459 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd4ec0c5940 con 0x7fd4ec143c20 2023-08-09T15:08:48.419 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:48.416+0000 7fd4d3fff640 1 -- 172.21.15.43:0/2134935459 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd4e4012b20 con 0x7fd4ec143c20 2023-08-09T15:08:48.419 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:48.416+0000 7fd4f184d640 1 -- 172.21.15.43:0/2134935459 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd4ec0b3720 con 0x7fd4ec143c20 2023-08-09T15:08:48.420 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:48.417+0000 7fd4f184d640 1 -- 172.21.15.43:0/2134935459 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd4ec0c5e60 con 0x7fd4ec143c20 2023-08-09T15:08:48.420 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:48.418+0000 7fd4d3fff640 1 -- 172.21.15.43:0/2134935459 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 106) v1 ==== 88443+0+0 (secure 0 0 0) 0x7fd4e4012c80 con 0x7fd4ec143c20 2023-08-09T15:08:48.420 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:48.420+0000 7fd4d3fff640 1 --2- 172.21.15.43:0/2134935459 >> [v2:172.21.15.203:6832/3082732769,v1:172.21.15.203:6833/3082732769] conn(0x7fd4d806a170 0x7fd4d806c630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T15:08:48.420 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:48.420+0000 7fd4d3fff640 1 -- 172.21.15.43:0/2134935459 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(205..205 src has 1..205) v4 ==== 6426+0+0 (secure 0 0 0) 0x7fd4e4088d50 con 0x7fd4ec143c20 2023-08-09T15:08:48.421 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:48.420+0000 7fd4eaffd640 1 --2- 172.21.15.43:0/2134935459 >> [v2:172.21.15.203:6832/3082732769,v1:172.21.15.203:6833/3082732769] conn(0x7fd4d806a170 0x7fd4d806c630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T15:08:48.422 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:48.422+0000 7fd4eaffd640 1 --2- 172.21.15.43:0/2134935459 >> [v2:172.21.15.203:6832/3082732769,v1:172.21.15.203:6833/3082732769] conn(0x7fd4d806a170 0x7fd4d806c630 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7fd4dc0012e0 tx=0x7fd4dc007ed0 comp rx=0 tx=0).ready entity=mgr.8500 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-09T15:08:48.424 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:48.424+0000 7fd4d3fff640 1 -- 172.21.15.43:0/2134935459 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178066 (secure 0 0 0) 0x7fd4ec0c5e60 con 0x7fd4ec143c20 2023-08-09T15:08:48.651 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:48.651+0000 7fd4f184d640 1 -- 172.21.15.43:0/2134935459 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"} v 0) v1 -- 0x7fd4ec0c5e60 con 0x7fd4ec143c20 2023-08-09T15:08:49.158 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:49.157+0000 7fd4d3fff640 1 -- 172.21.15.43:0/2134935459 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"}]=0 v96) v1 ==== 122+0+0 (secure 0 0 0) 0x7fd4ec0c5e60 con 0x7fd4ec143c20 2023-08-09T15:08:49.162 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:49.161+0000 7fd4f184d640 1 -- 172.21.15.43:0/2134935459 >> [v2:172.21.15.203:6832/3082732769,v1:172.21.15.203:6833/3082732769] conn(0x7fd4d806a170 msgr2=0x7fd4d806c630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T15:08:49.162 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:49.161+0000 7fd4f184d640 1 --2- 172.21.15.43:0/2134935459 >> [v2:172.21.15.203:6832/3082732769,v1:172.21.15.203:6833/3082732769] conn(0x7fd4d806a170 0x7fd4d806c630 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7fd4dc0012e0 tx=0x7fd4dc007ed0 comp rx=0 tx=0).stop 2023-08-09T15:08:49.163 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:49.161+0000 7fd4f184d640 1 -- 172.21.15.43:0/2134935459 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fd4ec143c20 msgr2=0x7fd4ec0c6140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T15:08:49.163 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:49.161+0000 7fd4f184d640 1 --2- 172.21.15.43:0/2134935459 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fd4ec143c20 0x7fd4ec0c6140 secure :-1 s=READY pgs=2222 cs=0 l=1 rev1=1 crypto rx=0x7fd4e400d490 tx=0x7fd4e400d950 comp rx=0 tx=0).stop 2023-08-09T15:08:49.163 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:49.161+0000 7fd4f184d640 1 -- 172.21.15.43:0/2134935459 shutdown_connections 2023-08-09T15:08:49.163 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:49.161+0000 7fd4f184d640 1 --2- 172.21.15.43:0/2134935459 >> [v2:172.21.15.203:6832/3082732769,v1:172.21.15.203:6833/3082732769] conn(0x7fd4d806a170 0x7fd4d806c630 unknown :-1 s=CLOSED pgs=63 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T15:08:49.163 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:49.161+0000 7fd4f184d640 1 --2- 172.21.15.43:0/2134935459 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fd4ec143c20 0x7fd4ec0c6140 unknown :-1 s=CLOSED pgs=2222 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T15:08:49.164 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:49.161+0000 7fd4f184d640 1 --2- 172.21.15.43:0/2134935459 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fd4ec0c86f0 0x7fd4ec0b0d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T15:08:49.164 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:49.161+0000 7fd4f184d640 1 --2- 172.21.15.43:0/2134935459 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fd4ec06c8d0 0x7fd4ec0b31e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T15:08:49.164 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:49.161+0000 7fd4f184d640 1 -- 172.21.15.43:0/2134935459 >> 172.21.15.43:0/2134935459 conn(0x7fd4ec0ce0a0 msgr2=0x7fd4ec0aefc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-09T15:08:49.164 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:49.162+0000 7fd4f184d640 1 -- 172.21.15.43:0/2134935459 shutdown_connections 2023-08-09T15:08:49.165 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:49.162+0000 7fd4f184d640 1 -- 172.21.15.43:0/2134935459 wait complete. 2023-08-09T15:08:49.188 DEBUG:tasks.cephfs.filesystem:Not Creating any subvols for filesystem 'cephfs' 2023-08-09T15:08:49.188 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-08-09T15:08:49.418 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:49.418+0000 7f2d2dd01640 1 Processor -- start 2023-08-09T15:08:49.418 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:49.418+0000 7f2d2dd01640 1 -- start start 2023-08-09T15:08:49.420 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:49.419+0000 7f2d2dd01640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f2d280b2b70 0x7f2d280d2ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T15:08:49.420 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:49.419+0000 7f2d2dd01640 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7f2d280b3050 0x7f2d280b3f60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T15:08:49.420 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:49.420+0000 7f2d2dd01640 1 -- --> v1:172.21.15.203:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2d280ce090 con 0x7f2d280b2790 2023-08-09T15:08:49.420 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:49.420+0000 7f2d2dd01640 1 -- --> v2:172.21.15.43:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2d280b3950 con 0x7f2d280b2b70 2023-08-09T15:08:49.421 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:49.420+0000 7f2d2dd01640 1 -- --> v2:172.21.15.203:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2d280cd7f0 con 0x7f2d280b3050 2023-08-09T15:08:49.421 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:49.420+0000 7f2d1effd640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f2d280b2b70 0x7f2d280d2ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T15:08:49.422 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:49.420+0000 7f2d1effd640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f2d280b2b70 0x7f2d280d2ed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:39834/0 (socket says 172.21.15.43:39834) 2023-08-09T15:08:49.422 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:49.420+0000 7f2d1effd640 1 -- 172.21.15.43:0/149338425 learned_addr learned my addr 172.21.15.43:0/149338425 (peer_addr_for_me v2:172.21.15.43:0/0) 2023-08-09T15:08:49.422 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:49.420+0000 7f2d27fff640 1 --2- 172.21.15.43:0/149338425 >> v2:172.21.15.203:3300/0 conn(0x7f2d280b3050 0x7f2d280b3f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T15:08:49.422 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:49.420+0000 7f2d1effd640 1 -- 172.21.15.43:0/149338425 >> v2:172.21.15.203:3300/0 conn(0x7f2d280b3050 msgr2=0x7f2d280b3f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-09T15:08:49.422 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:49.420+0000 7f2d1effd640 1 --2- 172.21.15.43:0/149338425 >> v2:172.21.15.203:3300/0 conn(0x7f2d280b3050 0x7f2d280b3f60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T15:08:49.422 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:49.420+0000 7f2d1effd640 1 -- 172.21.15.43:0/149338425 >> v1:172.21.15.203:6789/0 conn(0x7f2d280b2790 legacy=0x7f2d280b5330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-09T15:08:49.423 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:49.420+0000 7f2d1effd640 1 -- 172.21.15.43:0/149338425 --> v2:172.21.15.43:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2d280cdad0 con 0x7f2d280b2b70 2023-08-09T15:08:49.423 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:49.422+0000 7f2d1effd640 1 --2- 172.21.15.43:0/149338425 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f2d280b2b70 0x7f2d280d2ed0 secure :-1 s=READY pgs=2223 cs=0 l=1 rev1=1 crypto rx=0x7f2d2808a270 tx=0x7f2d1002b910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=21cb85f46a613a7e server_cookie=0 in_seq=0 out_seq=0 2023-08-09T15:08:49.423 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:49.422+0000 7f2d26ffd640 1 -- 172.21.15.43:0/149338425 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2d10031870 con 0x7f2d280b2b70 2023-08-09T15:08:49.424 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:49.422+0000 7f2d26ffd640 1 -- 172.21.15.43:0/149338425 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2d100319d0 con 0x7f2d280b2b70 2023-08-09T15:08:49.424 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:49.422+0000 7f2d26ffd640 1 -- 172.21.15.43:0/149338425 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2d10031d60 con 0x7f2d280b2b70 2023-08-09T15:08:49.424 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:49.423+0000 7f2d2dd01640 1 -- 172.21.15.43:0/149338425 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f2d280b2b70 msgr2=0x7f2d280d2ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T15:08:49.424 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:49.423+0000 7f2d2dd01640 1 --2- 172.21.15.43:0/149338425 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f2d280b2b70 0x7f2d280d2ed0 secure :-1 s=READY pgs=2223 cs=0 l=1 rev1=1 crypto rx=0x7f2d2808a270 tx=0x7f2d1002b910 comp rx=0 tx=0).stop 2023-08-09T15:08:49.424 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:49.423+0000 7f2d2dd01640 1 -- 172.21.15.43:0/149338425 shutdown_connections 2023-08-09T15:08:49.424 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:49.423+0000 7f2d2dd01640 1 --2- 172.21.15.43:0/149338425 >> v2:172.21.15.203:3300/0 conn(0x7f2d280b3050 0x7f2d280b3f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T15:08:49.424 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:49.423+0000 7f2d2dd01640 1 --2- 172.21.15.43:0/149338425 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f2d280b2b70 0x7f2d280d2ed0 unknown :-1 s=CLOSED pgs=2223 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T15:08:49.425 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:49.423+0000 7f2d2dd01640 1 -- 172.21.15.43:0/149338425 >> 172.21.15.43:0/149338425 conn(0x7f2d2809b7b0 msgr2=0x7f2d280d24e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-09T15:08:49.425 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:49.423+0000 7f2d2dd01640 1 -- 172.21.15.43:0/149338425 shutdown_connections 2023-08-09T15:08:49.425 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:49.423+0000 7f2d2dd01640 1 -- 172.21.15.43:0/149338425 wait complete. 2023-08-09T15:08:49.425 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:49.425+0000 7f2d2dd01640 1 Processor -- start 2023-08-09T15:08:49.426 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:49.425+0000 7f2d2dd01640 1 -- start start 2023-08-09T15:08:49.427 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:49.426+0000 7f2d2dd01640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f2d280b2790 0x7f2d28154c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T15:08:49.427 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:49.426+0000 7f2d2dd01640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f2d280b2b70 0x7f2d280c8520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T15:08:49.427 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:49.426+0000 7f2d2dd01640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f2d280b3050 0x7f2d280c8a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T15:08:49.427 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:49.426+0000 7f2d2dd01640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2d280b0730 con 0x7f2d280b2b70 2023-08-09T15:08:49.428 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:49.426+0000 7f2d2dd01640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2d28155150 con 0x7f2d280b2790 2023-08-09T15:08:49.428 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:49.426+0000 7f2d2dd01640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2d280c6c40 con 0x7f2d280b3050 2023-08-09T15:08:49.428 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:49.426+0000 7f2d1effd640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f2d280b2b70 0x7f2d280c8520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T15:08:49.428 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:49.426+0000 7f2d1effd640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f2d280b2b70 0x7f2d280c8520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:39844/0 (socket says 172.21.15.43:39844) 2023-08-09T15:08:49.428 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:49.426+0000 7f2d1effd640 1 -- 172.21.15.43:0/3141550063 learned_addr learned my addr 172.21.15.43:0/3141550063 (peer_addr_for_me v2:172.21.15.43:0/0) 2023-08-09T15:08:49.428 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:49.426+0000 7f2d277fe640 1 --2- 172.21.15.43:0/3141550063 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f2d280b2790 0x7f2d28154c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T15:08:49.428 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:49.426+0000 7f2d1effd640 1 -- 172.21.15.43:0/3141550063 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f2d280b3050 msgr2=0x7f2d280c8a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T15:08:49.429 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:49.426+0000 7f2d1effd640 1 --2- 172.21.15.43:0/3141550063 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f2d280b3050 0x7f2d280c8a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T15:08:49.429 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:49.426+0000 7f2d1effd640 1 -- 172.21.15.43:0/3141550063 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f2d280b2790 msgr2=0x7f2d28154c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T15:08:49.429 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:49.426+0000 7f2d1effd640 1 --2- 172.21.15.43:0/3141550063 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f2d280b2790 0x7f2d28154c10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T15:08:49.429 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:49.427+0000 7f2d1effd640 1 -- 172.21.15.43:0/3141550063 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2d100094d0 con 0x7f2d280b2b70 2023-08-09T15:08:49.429 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:49.427+0000 7f2d1effd640 1 --2- 172.21.15.43:0/3141550063 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f2d280b2b70 0x7f2d280c8520 secure :-1 s=READY pgs=2224 cs=0 l=1 rev1=1 crypto rx=0x7f2d1002b950 tx=0x7f2d100046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-09T15:08:49.430 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:49.427+0000 7f2d24ff9640 1 -- 172.21.15.43:0/3141550063 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2d1003ad30 con 0x7f2d280b2b70 2023-08-09T15:08:49.430 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:49.427+0000 7f2d24ff9640 1 -- 172.21.15.43:0/3141550063 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2d100443f0 con 0x7f2d280b2b70 2023-08-09T15:08:49.430 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:49.427+0000 7f2d2dd01640 1 -- 172.21.15.43:0/3141550063 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2d280c6f20 con 0x7f2d280b2b70 2023-08-09T15:08:49.430 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:49.427+0000 7f2d24ff9640 1 -- 172.21.15.43:0/3141550063 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2d100447a0 con 0x7f2d280b2b70 2023-08-09T15:08:49.430 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:49.427+0000 7f2d2dd01640 1 -- 172.21.15.43:0/3141550063 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2d280c74e0 con 0x7f2d280b2b70 2023-08-09T15:08:49.430 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:49.429+0000 7f2d2dd01640 1 -- 172.21.15.43:0/3141550063 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2d280739f0 con 0x7f2d280b2b70 2023-08-09T15:08:49.431 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:49.430+0000 7f2d24ff9640 1 -- 172.21.15.43:0/3141550063 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 106) v1 ==== 88443+0+0 (secure 0 0 0) 0x7f2d1004d450 con 0x7f2d280b2b70 2023-08-09T15:08:49.432 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:49.431+0000 7f2d24ff9640 1 --2- 172.21.15.43:0/3141550063 >> [v2:172.21.15.203:6832/3082732769,v1:172.21.15.203:6833/3082732769] conn(0x7f2d0806a170 0x7f2d0806c630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T15:08:49.432 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:49.431+0000 7f2d24ff9640 1 -- 172.21.15.43:0/3141550063 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(205..205 src has 1..205) v4 ==== 6426+0+0 (secure 0 0 0) 0x7f2d1002d040 con 0x7f2d280b2b70 2023-08-09T15:08:49.432 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:49.432+0000 7f2d277fe640 1 --2- 172.21.15.43:0/3141550063 >> [v2:172.21.15.203:6832/3082732769,v1:172.21.15.203:6833/3082732769] conn(0x7f2d0806a170 0x7f2d0806c630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T15:08:49.433 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:49.433+0000 7f2d277fe640 1 --2- 172.21.15.43:0/3141550063 >> [v2:172.21.15.203:6832/3082732769,v1:172.21.15.203:6833/3082732769] conn(0x7f2d0806a170 0x7f2d0806c630 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f2d1800a730 tx=0x7f2d18008040 comp rx=0 tx=0).ready entity=mgr.8500 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-09T15:08:49.434 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:49.434+0000 7f2d24ff9640 1 -- 172.21.15.43:0/3141550063 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178066 (secure 0 0 0) 0x7f2d10076740 con 0x7f2d280b2b70 2023-08-09T15:08:49.654 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:49.654+0000 7f2d2dd01640 1 -- 172.21.15.43:0/3141550063 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f2d280b4b00 con 0x7f2d280b2b70 2023-08-09T15:08:49.655 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:49.655+0000 7f2d24ff9640 1 -- 172.21.15.43:0/3141550063 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 96 v96) v1 ==== 94+0+4883 (secure 0 0 0) 0x7f2d10079d60 con 0x7f2d280b2b70 2023-08-09T15:08:49.656 INFO:teuthology.orchestra.run.smithi043.stdout: 2023-08-09T15:08:49.656 INFO:teuthology.orchestra.run.smithi043.stdout:{"epoch":96,"default_fscid":11,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":7703,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.203:6836/1774059384","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6835","nonce":1774059384},{"type":"v1","addr":"172.21.15.203:6836","nonce":1774059384}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":91},{"gid":8665,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.203:6837/682905899","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6834","nonce":682905899},{"type":"v1","addr":"172.21.15.203:6837","nonce":682905899}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":91},{"gid":10752,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.43:6834/4168463642","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6832","nonce":4168463642},{"type":"v1","addr":"172.21.15.43:6834","nonce":4168463642}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":90}],"filesystems":[{"mdsmap":{"epoch":96,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-08-09T15:08:45.563041+0000","modified":"2023-08-09T15:08:49.150213+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":10755},"failed":[],"damaged":[],"stopped":[],"info":{"gid_10755":{"gid":10755,"name":"a","rank":0,"incarnation":92,"state":"up:active","state_seq":3,"addr":"172.21.15.43:6835/349688227","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6833","nonce":349688227},{"type":"v1","addr":"172.21.15.43:6835","nonce":349688227}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[25],"metadata_pool":24,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":11}]} 2023-08-09T15:08:49.657 INFO:teuthology.orchestra.run.smithi043.stderr:dumped fsmap epoch 96 2023-08-09T15:08:49.658 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:49.657+0000 7f2d2dd01640 1 -- 172.21.15.43:0/3141550063 >> [v2:172.21.15.203:6832/3082732769,v1:172.21.15.203:6833/3082732769] conn(0x7f2d0806a170 msgr2=0x7f2d0806c630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T15:08:49.658 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:49.657+0000 7f2d2dd01640 1 --2- 172.21.15.43:0/3141550063 >> [v2:172.21.15.203:6832/3082732769,v1:172.21.15.203:6833/3082732769] conn(0x7f2d0806a170 0x7f2d0806c630 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f2d1800a730 tx=0x7f2d18008040 comp rx=0 tx=0).stop 2023-08-09T15:08:49.659 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:49.657+0000 7f2d2dd01640 1 -- 172.21.15.43:0/3141550063 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f2d280b2b70 msgr2=0x7f2d280c8520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T15:08:49.659 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:49.657+0000 7f2d2dd01640 1 --2- 172.21.15.43:0/3141550063 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f2d280b2b70 0x7f2d280c8520 secure :-1 s=READY pgs=2224 cs=0 l=1 rev1=1 crypto rx=0x7f2d1002b950 tx=0x7f2d100046c0 comp rx=0 tx=0).stop 2023-08-09T15:08:49.659 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:49.658+0000 7f2d2dd01640 1 -- 172.21.15.43:0/3141550063 shutdown_connections 2023-08-09T15:08:49.660 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:49.658+0000 7f2d2dd01640 1 --2- 172.21.15.43:0/3141550063 >> [v2:172.21.15.203:6832/3082732769,v1:172.21.15.203:6833/3082732769] conn(0x7f2d0806a170 0x7f2d0806c630 unknown :-1 s=CLOSED pgs=64 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T15:08:49.660 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:49.658+0000 7f2d2dd01640 1 --2- 172.21.15.43:0/3141550063 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f2d280b3050 0x7f2d280c8a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T15:08:49.660 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:49.658+0000 7f2d2dd01640 1 --2- 172.21.15.43:0/3141550063 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f2d280b2b70 0x7f2d280c8520 unknown :-1 s=CLOSED pgs=2224 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T15:08:49.660 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:49.658+0000 7f2d2dd01640 1 --2- 172.21.15.43:0/3141550063 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f2d280b2790 0x7f2d28154c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T15:08:49.660 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:49.658+0000 7f2d2dd01640 1 -- 172.21.15.43:0/3141550063 >> 172.21.15.43:0/3141550063 conn(0x7f2d2809b7b0 msgr2=0x7f2d280ceda0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-09T15:08:49.660 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:49.658+0000 7f2d2dd01640 1 -- 172.21.15.43:0/3141550063 shutdown_connections 2023-08-09T15:08:49.661 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:49.658+0000 7f2d2dd01640 1 -- 172.21.15.43:0/3141550063 wait complete. 2023-08-09T15:08:49.677 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-08-09T15:08:49.891 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:49.890+0000 7f4b9fac5640 1 Processor -- start 2023-08-09T15:08:49.891 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:49.890+0000 7f4b9fac5640 1 -- start start 2023-08-09T15:08:49.893 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:49.892+0000 7f4b9fac5640 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7f4b980b4670 0x7f4b980d2ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T15:08:49.894 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:49.892+0000 7f4b9fac5640 1 -- --> v1:172.21.15.43:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4b980ce090 con 0x7f4b980b2790 2023-08-09T15:08:49.894 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:49.892+0000 7f4b9fac5640 1 -- --> v1:172.21.15.203:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4b980cd7f0 con 0x7f4b980b2b70 2023-08-09T15:08:49.894 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:49.892+0000 7f4b9fac5640 1 -- --> v2:172.21.15.203:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4b980b3950 con 0x7f4b980b4670 2023-08-09T15:08:49.894 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:49.892+0000 7f4b9d861640 1 --1- >> v1:172.21.15.43:6789/0 conn(0x7f4b980b2790 0x7f4b980b5330 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.43:6789/0 says I am v1:172.21.15.43:42146/0 (socket says 172.21.15.43:42146) 2023-08-09T15:08:49.895 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:49.892+0000 7f4b9d861640 1 -- 172.21.15.43:0/1722044066 learned_addr learned my addr 172.21.15.43:0/1722044066 (peer_addr_for_me v1:172.21.15.43:0/0) 2023-08-09T15:08:49.895 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:49.892+0000 7f4b9e062640 1 --2- 172.21.15.43:0/1722044066 >> v2:172.21.15.203:3300/0 conn(0x7f4b980b4670 0x7f4b980d2ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T15:08:49.895 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:49.893+0000 7f4b9c85f640 1 -- 172.21.15.43:0/1722044066 <== mon.0 v1:172.21.15.43:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3989953712 0 0) 0x7f4b980ce090 con 0x7f4b980b2790 2023-08-09T15:08:49.895 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:49.893+0000 7f4b9c85f640 1 -- 172.21.15.43:0/1722044066 --> v1:172.21.15.43:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f4b84003520 con 0x7f4b980b2790 2023-08-09T15:08:49.901 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:49.893+0000 7f4b9c85f640 1 -- 172.21.15.43:0/1722044066 <== mon.0 v1:172.21.15.43:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3793841813 0 0) 0x7f4b84003520 con 0x7f4b980b2790 2023-08-09T15:08:49.901 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:49.893+0000 7f4b9c85f640 1 -- 172.21.15.43:0/1722044066 >> v2:172.21.15.203:3300/0 conn(0x7f4b980b4670 msgr2=0x7f4b980d2ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-09T15:08:49.902 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:49.893+0000 7f4b9c85f640 1 --2- 172.21.15.43:0/1722044066 >> v2:172.21.15.203:3300/0 conn(0x7f4b980b4670 0x7f4b980d2ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T15:08:49.902 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:49.893+0000 7f4b9c85f640 1 -- 172.21.15.43:0/1722044066 >> v1:172.21.15.203:6789/0 conn(0x7f4b980b2b70 legacy=0x7f4b980b3f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-09T15:08:49.902 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:49.893+0000 7f4b9c85f640 1 -- 172.21.15.43:0/1722044066 --> v1:172.21.15.43:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4b980cdad0 con 0x7f4b980b2790 2023-08-09T15:08:49.902 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:49.893+0000 7f4b9c85f640 1 -- 172.21.15.43:0/1722044066 <== mon.0 v1:172.21.15.43:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1285150740 0 0) 0x7f4b94001670 con 0x7f4b980b2790 2023-08-09T15:08:49.902 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:49.893+0000 7f4b9c85f640 1 -- 172.21.15.43:0/1722044066 >> v1:172.21.15.43:6789/0 conn(0x7f4b980b2790 legacy=0x7f4b980b5330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T15:08:49.902 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:49.893+0000 7f4b9c85f640 1 --2- 172.21.15.43:0/1722044066 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f4b840039e0 0x7f4b84003de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T15:08:49.902 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:49.893+0000 7f4b9c85f640 1 --2- 172.21.15.43:0/1722044066 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f4b840047c0 0x7f4b84004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T15:08:49.903 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:49.893+0000 7f4b9c85f640 1 --2- 172.21.15.43:0/1722044066 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f4b840051d0 0x7f4b8400b980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T15:08:49.903 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:49.893+0000 7f4b9c85f640 1 -- 172.21.15.43:0/1722044066 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4b980b3950 con 0x7f4b840047c0 2023-08-09T15:08:49.903 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:49.893+0000 7f4b9c85f640 1 -- 172.21.15.43:0/1722044066 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4b8400c120 con 0x7f4b840051d0 2023-08-09T15:08:49.903 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:49.893+0000 7f4b9c85f640 1 -- 172.21.15.43:0/1722044066 --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4b8400c2e0 con 0x7f4b840039e0 2023-08-09T15:08:49.903 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:49.893+0000 7f4b9d060640 1 --2- 172.21.15.43:0/1722044066 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f4b840047c0 0x7f4b84004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T15:08:49.904 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:49.894+0000 7f4b9d861640 1 --2- 172.21.15.43:0/1722044066 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f4b840039e0 0x7f4b84003de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T15:08:49.904 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:49.894+0000 7f4b9e062640 1 --2- 172.21.15.43:0/1722044066 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f4b840051d0 0x7f4b8400b980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T15:08:49.904 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:49.894+0000 7f4b9d060640 1 -- 172.21.15.43:0/1722044066 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f4b840039e0 msgr2=0x7f4b84003de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-09T15:08:49.904 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:49.894+0000 7f4b9d060640 1 --2- 172.21.15.43:0/1722044066 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f4b840039e0 0x7f4b84003de0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T15:08:49.904 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:49.894+0000 7f4b9d060640 1 -- 172.21.15.43:0/1722044066 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f4b840051d0 msgr2=0x7f4b8400b980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-09T15:08:49.904 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:49.894+0000 7f4b9d060640 1 --2- 172.21.15.43:0/1722044066 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f4b840051d0 0x7f4b8400b980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T15:08:49.904 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:49.894+0000 7f4b9d060640 1 -- 172.21.15.43:0/1722044066 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4b84003520 con 0x7f4b840047c0 2023-08-09T15:08:49.905 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:49.894+0000 7f4b9e062640 1 --2- 172.21.15.43:0/1722044066 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f4b840051d0 0x7f4b8400b980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-08-09T15:08:49.905 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:49.894+0000 7f4b9e062640 1 -- 172.21.15.43:0/1722044066 reap_dead start 2023-08-09T15:08:49.905 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:49.895+0000 7f4b9d060640 1 --2- 172.21.15.43:0/1722044066 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f4b840047c0 0x7f4b84004c40 secure :-1 s=READY pgs=2226 cs=0 l=1 rev1=1 crypto rx=0x7f4b88003db0 tx=0x7f4b840057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fe8910ecb74c02bc server_cookie=0 in_seq=0 out_seq=0 2023-08-09T15:08:49.905 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:49.895+0000 7f4b9c85f640 1 -- 172.21.15.43:0/1722044066 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4b88032600 con 0x7f4b840047c0 2023-08-09T15:08:49.905 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:49.895+0000 7f4b9c85f640 1 -- 172.21.15.43:0/1722044066 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4b8802e050 con 0x7f4b840047c0 2023-08-09T15:08:49.905 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:49.895+0000 7f4b9c85f640 1 -- 172.21.15.43:0/1722044066 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4b880329e0 con 0x7f4b840047c0 2023-08-09T15:08:49.905 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:49.896+0000 7f4b9fac5640 1 -- 172.21.15.43:0/1722044066 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f4b840047c0 msgr2=0x7f4b84004c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T15:08:49.906 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:49.896+0000 7f4b9fac5640 1 --2- 172.21.15.43:0/1722044066 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f4b840047c0 0x7f4b84004c40 secure :-1 s=READY pgs=2226 cs=0 l=1 rev1=1 crypto rx=0x7f4b88003db0 tx=0x7f4b840057f0 comp rx=0 tx=0).stop 2023-08-09T15:08:49.906 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:49.896+0000 7f4b9fac5640 1 -- 172.21.15.43:0/1722044066 shutdown_connections 2023-08-09T15:08:49.906 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:49.896+0000 7f4b9fac5640 1 --2- 172.21.15.43:0/1722044066 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f4b840047c0 0x7f4b84004c40 unknown :-1 s=CLOSED pgs=2226 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T15:08:49.906 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:49.896+0000 7f4b9fac5640 1 -- 172.21.15.43:0/1722044066 >> 172.21.15.43:0/1722044066 conn(0x7f4b9809b7b0 msgr2=0x7f4b980d24e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-09T15:08:49.906 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:49.896+0000 7f4b9fac5640 1 -- 172.21.15.43:0/1722044066 shutdown_connections 2023-08-09T15:08:49.906 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:49.896+0000 7f4b9fac5640 1 -- 172.21.15.43:0/1722044066 wait complete. 2023-08-09T15:08:49.906 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:49.898+0000 7f4b9fac5640 1 Processor -- start 2023-08-09T15:08:49.906 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:49.898+0000 7f4b9fac5640 1 -- start start 2023-08-09T15:08:49.907 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:49.899+0000 7f4b9fac5640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f4b840047c0 0x7f4b980afab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T15:08:49.907 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:49.899+0000 7f4b9fac5640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f4b980c8dd0 0x7f4b9815cbb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T15:08:49.907 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:49.899+0000 7f4b9fac5640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f4b981f1970 0x7f4b980cac60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T15:08:49.907 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:49.899+0000 7f4b9fac5640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4b980b0140 con 0x7f4b840047c0 2023-08-09T15:08:49.907 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:49.899+0000 7f4b9fac5640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4b980c9250 con 0x7f4b981f1970 2023-08-09T15:08:49.907 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:49.899+0000 7f4b9fac5640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4b9815dbb0 con 0x7f4b980c8dd0 2023-08-09T15:08:49.907 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:49.899+0000 7f4b9d861640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f4b840047c0 0x7f4b980afab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T15:08:49.908 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:49.899+0000 7f4b9d861640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f4b840047c0 0x7f4b980afab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:39864/0 (socket says 172.21.15.43:39864) 2023-08-09T15:08:49.908 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:49.899+0000 7f4b9d861640 1 -- 172.21.15.43:0/3892881949 learned_addr learned my addr 172.21.15.43:0/3892881949 (peer_addr_for_me v2:172.21.15.43:0/0) 2023-08-09T15:08:49.908 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:49.900+0000 7f4b9e062640 1 --2- 172.21.15.43:0/3892881949 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f4b981f1970 0x7f4b980cac60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T15:08:49.908 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:49.900+0000 7f4b9d060640 1 --2- 172.21.15.43:0/3892881949 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f4b980c8dd0 0x7f4b9815cbb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T15:08:49.908 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:49.900+0000 7f4b9d861640 1 -- 172.21.15.43:0/3892881949 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f4b980c8dd0 msgr2=0x7f4b9815cbb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T15:08:49.908 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:49.900+0000 7f4b9d861640 1 --2- 172.21.15.43:0/3892881949 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f4b980c8dd0 0x7f4b9815cbb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T15:08:49.909 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:49.900+0000 7f4b9d861640 1 -- 172.21.15.43:0/3892881949 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f4b981f1970 msgr2=0x7f4b980cac60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T15:08:49.909 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:49.900+0000 7f4b9d861640 1 --2- 172.21.15.43:0/3892881949 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f4b981f1970 0x7f4b980cac60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T15:08:49.909 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:49.900+0000 7f4b9d861640 1 -- 172.21.15.43:0/3892881949 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4b980b0670 con 0x7f4b840047c0 2023-08-09T15:08:49.909 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:49.900+0000 7f4b9e062640 1 --2- 172.21.15.43:0/3892881949 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f4b981f1970 0x7f4b980cac60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-08-09T15:08:49.909 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:49.900+0000 7f4b9d060640 1 --2- 172.21.15.43:0/3892881949 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f4b980c8dd0 0x7f4b9815cbb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-08-09T15:08:49.909 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:49.900+0000 7f4b9d861640 1 --2- 172.21.15.43:0/3892881949 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f4b840047c0 0x7f4b980afab0 secure :-1 s=READY pgs=2227 cs=0 l=1 rev1=1 crypto rx=0x7f4b94000f60 tx=0x7f4b9400dd40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-09T15:08:49.909 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:49.900+0000 7f4b8e7fc640 1 -- 172.21.15.43:0/3892881949 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4b940074e0 con 0x7f4b840047c0 2023-08-09T15:08:49.910 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:49.901+0000 7f4b8e7fc640 1 -- 172.21.15.43:0/3892881949 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4b94007660 con 0x7f4b840047c0 2023-08-09T15:08:49.910 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:49.901+0000 7f4b9fac5640 1 -- 172.21.15.43:0/3892881949 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4b980b0950 con 0x7f4b840047c0 2023-08-09T15:08:49.910 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:49.901+0000 7f4b8e7fc640 1 -- 172.21.15.43:0/3892881949 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4b94004a20 con 0x7f4b840047c0 2023-08-09T15:08:49.910 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:49.901+0000 7f4b9fac5640 1 -- 172.21.15.43:0/3892881949 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4b980b0f10 con 0x7f4b840047c0 2023-08-09T15:08:49.911 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:49.902+0000 7f4b9fac5640 1 -- 172.21.15.43:0/3892881949 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4b980739f0 con 0x7f4b840047c0 2023-08-09T15:08:49.911 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:49.903+0000 7f4b8e7fc640 1 -- 172.21.15.43:0/3892881949 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 106) v1 ==== 88443+0+0 (secure 0 0 0) 0x7f4b94004b80 con 0x7f4b840047c0 2023-08-09T15:08:49.911 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:49.904+0000 7f4b8e7fc640 1 --2- 172.21.15.43:0/3892881949 >> [v2:172.21.15.203:6832/3082732769,v1:172.21.15.203:6833/3082732769] conn(0x7f4b7c06a170 0x7f4b7c06c630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T15:08:49.911 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:49.904+0000 7f4b8e7fc640 1 -- 172.21.15.43:0/3892881949 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(205..205 src has 1..205) v4 ==== 6426+0+0 (secure 0 0 0) 0x7f4b94088760 con 0x7f4b840047c0 2023-08-09T15:08:49.912 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:49.904+0000 7f4b9d060640 1 --2- 172.21.15.43:0/3892881949 >> [v2:172.21.15.203:6832/3082732769,v1:172.21.15.203:6833/3082732769] conn(0x7f4b7c06a170 0x7f4b7c06c630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T15:08:49.912 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:49.906+0000 7f4b9d060640 1 --2- 172.21.15.43:0/3892881949 >> [v2:172.21.15.203:6832/3082732769,v1:172.21.15.203:6833/3082732769] conn(0x7f4b7c06a170 0x7f4b7c06c630 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f4b88034000 tx=0x7f4b88009030 comp rx=0 tx=0).ready entity=mgr.8500 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-09T15:08:49.912 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:49.908+0000 7f4b8e7fc640 1 -- 172.21.15.43:0/3892881949 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178066 (secure 0 0 0) 0x7f4b94057570 con 0x7f4b840047c0 2023-08-09T15:08:50.106 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:50.106+0000 7f4b9fac5640 1 -- 172.21.15.43:0/3892881949 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f4b980b4b00 con 0x7f4b840047c0 2023-08-09T15:08:50.108 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:50.108+0000 7f4b8e7fc640 1 -- 172.21.15.43:0/3892881949 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v205) v1 ==== 74+0+18566 (secure 0 0 0) 0x7f4b94057100 con 0x7f4b840047c0 2023-08-09T15:08:50.109 INFO:teuthology.orchestra.run.smithi043.stdout: 2023-08-09T15:08:50.109 INFO:teuthology.orchestra.run.smithi043.stdout:{"epoch":205,"fsid":"daac87ee-1337-4250-a955-aea4e408a39b","created":"2023-08-09T13:33:41.701198+0000","modified":"2023-08-09T15:08:47.125724+0000","last_up_change":"2023-08-09T13:33:53.783453+0000","last_in_change":"2023-08-09T13:33:45.148904+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":25,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":"rbd","create_time":"2023-08-09T13:33:54.365137+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":".mgr","create_time":"2023-08-09T13:33:54.831803+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":24,"pool_name":"cephfs_metadata","create_time":"2023-08-09T15:08:43.920337+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"204","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":25,"pool_name":"cephfs_data","create_time":"2023-08-09T15:08:44.594489+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"205","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"cbd49ecd-e896-4fbd-864e-5c37a829b165","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":203,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6800","nonce":502667335},{"type":"v1","addr":"172.21.15.43:6801","nonce":502667335}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6802","nonce":502667335},{"type":"v1","addr":"172.21.15.43:6803","nonce":502667335}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6806","nonce":502667335},{"type":"v1","addr":"172.21.15.43:6807","nonce":502667335}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6804","nonce":502667335},{"type":"v1","addr":"172.21.15.43:6805","nonce":502667335}]},"public_addr":"172.21.15.43:6801/502667335","cluster_addr":"172.21.15.43:6803/502667335","heartbeat_back_addr":"172.21.15.43:6807/502667335","heartbeat_front_addr":"172.21.15.43:6805/502667335","state":["exists","up"]},{"osd":1,"uuid":"75c7e0fe-1ec7-4fe8-a2f2-3661a65b8500","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":203,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6808","nonce":3289134610},{"type":"v1","addr":"172.21.15.43:6809","nonce":3289134610}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6810","nonce":3289134610},{"type":"v1","addr":"172.21.15.43:6812","nonce":3289134610}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6821","nonce":3289134610},{"type":"v1","addr":"172.21.15.43:6824","nonce":3289134610}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6815","nonce":3289134610},{"type":"v1","addr":"172.21.15.43:6818","nonce":3289134610}]},"public_addr":"172.21.15.43:6809/3289134610","cluster_addr":"172.21.15.43:6812/3289134610","heartbeat_back_addr":"172.21.15.43:6824/3289134610","heartbeat_front_addr":"172.21.15.43:6818/3289134610","state":["exists","up"]},{"osd":2,"uuid":"bb5c41e9-7f5f-4125-9b01-3bb8bfc2ea83","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":203,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6811","nonce":4079709736},{"type":"v1","addr":"172.21.15.43:6814","nonce":4079709736}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6817","nonce":4079709736},{"type":"v1","addr":"172.21.15.43:6820","nonce":4079709736}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6829","nonce":4079709736},{"type":"v1","addr":"172.21.15.43:6831","nonce":4079709736}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6823","nonce":4079709736},{"type":"v1","addr":"172.21.15.43:6826","nonce":4079709736}]},"public_addr":"172.21.15.43:6814/4079709736","cluster_addr":"172.21.15.43:6820/4079709736","heartbeat_back_addr":"172.21.15.43:6831/4079709736","heartbeat_front_addr":"172.21.15.43:6826/4079709736","state":["exists","up"]},{"osd":3,"uuid":"bfe4c8b4-694a-465c-87e6-2b2aab852613","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":203,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6813","nonce":1709538759},{"type":"v1","addr":"172.21.15.43:6816","nonce":1709538759}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6819","nonce":1709538759},{"type":"v1","addr":"172.21.15.43:6822","nonce":1709538759}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6828","nonce":1709538759},{"type":"v1","addr":"172.21.15.43:6830","nonce":1709538759}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6825","nonce":1709538759},{"type":"v1","addr":"172.21.15.43:6827","nonce":1709538759}]},"public_addr":"172.21.15.43:6816/1709538759","cluster_addr":"172.21.15.43:6822/1709538759","heartbeat_back_addr":"172.21.15.43:6830/1709538759","heartbeat_front_addr":"172.21.15.43:6827/1709538759","state":["exists","up"]},{"osd":4,"uuid":"fa2e8a77-aba1-4756-853e-0d174a5bdeba","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":203,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6800","nonce":4196866481},{"type":"v1","addr":"172.21.15.203:6801","nonce":4196866481}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6802","nonce":4196866481},{"type":"v1","addr":"172.21.15.203:6803","nonce":4196866481}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6806","nonce":4196866481},{"type":"v1","addr":"172.21.15.203:6807","nonce":4196866481}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6804","nonce":4196866481},{"type":"v1","addr":"172.21.15.203:6805","nonce":4196866481}]},"public_addr":"172.21.15.203:6801/4196866481","cluster_addr":"172.21.15.203:6803/4196866481","heartbeat_back_addr":"172.21.15.203:6807/4196866481","heartbeat_front_addr":"172.21.15.203:6805/4196866481","state":["exists","up"]},{"osd":5,"uuid":"81811e4b-4d24-4f51-a82c-f1d700a2a00d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":203,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6816","nonce":3059290081},{"type":"v1","addr":"172.21.15.203:6817","nonce":3059290081}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6818","nonce":3059290081},{"type":"v1","addr":"172.21.15.203:6819","nonce":3059290081}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6822","nonce":3059290081},{"type":"v1","addr":"172.21.15.203:6823","nonce":3059290081}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6820","nonce":3059290081},{"type":"v1","addr":"172.21.15.203:6821","nonce":3059290081}]},"public_addr":"172.21.15.203:6817/3059290081","cluster_addr":"172.21.15.203:6819/3059290081","heartbeat_back_addr":"172.21.15.203:6823/3059290081","heartbeat_front_addr":"172.21.15.203:6821/3059290081","state":["exists","up"]},{"osd":6,"uuid":"307bac4d-6a5b-443d-af75-56d695d7674f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":203,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6824","nonce":512155569},{"type":"v1","addr":"172.21.15.203:6825","nonce":512155569}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6826","nonce":512155569},{"type":"v1","addr":"172.21.15.203:6827","nonce":512155569}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6830","nonce":512155569},{"type":"v1","addr":"172.21.15.203:6831","nonce":512155569}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6828","nonce":512155569},{"type":"v1","addr":"172.21.15.203:6829","nonce":512155569}]},"public_addr":"172.21.15.203:6825/512155569","cluster_addr":"172.21.15.203:6827/512155569","heartbeat_back_addr":"172.21.15.203:6831/512155569","heartbeat_front_addr":"172.21.15.203:6829/512155569","state":["exists","up"]},{"osd":7,"uuid":"a95dc7ca-9f1b-4295-af70-68a22a2d7d5e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":203,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6808","nonce":3331450438},{"type":"v1","addr":"172.21.15.203:6809","nonce":3331450438}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6810","nonce":3331450438},{"type":"v1","addr":"172.21.15.203:6811","nonce":3331450438}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6814","nonce":3331450438},{"type":"v1","addr":"172.21.15.203:6815","nonce":3331450438}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6812","nonce":3331450438},{"type":"v1","addr":"172.21.15.203:6813","nonce":3331450438}]},"public_addr":"172.21.15.203:6809/3331450438","cluster_addr":"172.21.15.203:6811/3331450438","heartbeat_back_addr":"172.21.15.203:6815/3331450438","heartbeat_front_addr":"172.21.15.203:6813/3331450438","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-09T13:33:50.547652+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-09T13:33:50.589469+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-09T13:33:50.561724+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-09T13:33:50.599650+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-09T13:33:50.411855+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-09T13:33:50.219681+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-09T13:33:50.504664+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-09T13:33:50.441248+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":7,"snaps":[{"begin":104,"length":1},{"begin":221,"length":1}]}],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-08-09T15:08:50.112 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:50.111+0000 7f4b9fac5640 1 -- 172.21.15.43:0/3892881949 >> [v2:172.21.15.203:6832/3082732769,v1:172.21.15.203:6833/3082732769] conn(0x7f4b7c06a170 msgr2=0x7f4b7c06c630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T15:08:50.112 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:50.111+0000 7f4b9fac5640 1 --2- 172.21.15.43:0/3892881949 >> [v2:172.21.15.203:6832/3082732769,v1:172.21.15.203:6833/3082732769] conn(0x7f4b7c06a170 0x7f4b7c06c630 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f4b88034000 tx=0x7f4b88009030 comp rx=0 tx=0).stop 2023-08-09T15:08:50.112 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:50.111+0000 7f4b9fac5640 1 -- 172.21.15.43:0/3892881949 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f4b840047c0 msgr2=0x7f4b980afab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T15:08:50.112 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:50.111+0000 7f4b9fac5640 1 --2- 172.21.15.43:0/3892881949 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f4b840047c0 0x7f4b980afab0 secure :-1 s=READY pgs=2227 cs=0 l=1 rev1=1 crypto rx=0x7f4b94000f60 tx=0x7f4b9400dd40 comp rx=0 tx=0).stop 2023-08-09T15:08:50.112 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:50.111+0000 7f4b9fac5640 1 -- 172.21.15.43:0/3892881949 shutdown_connections 2023-08-09T15:08:50.112 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:50.111+0000 7f4b9fac5640 1 --2- 172.21.15.43:0/3892881949 >> [v2:172.21.15.203:6832/3082732769,v1:172.21.15.203:6833/3082732769] conn(0x7f4b7c06a170 0x7f4b7c06c630 unknown :-1 s=CLOSED pgs=65 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T15:08:50.112 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:50.111+0000 7f4b9fac5640 1 --2- 172.21.15.43:0/3892881949 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f4b981f1970 0x7f4b980cac60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T15:08:50.113 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:50.111+0000 7f4b9fac5640 1 --2- 172.21.15.43:0/3892881949 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f4b980c8dd0 0x7f4b9815cbb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T15:08:50.113 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:50.111+0000 7f4b9fac5640 1 --2- 172.21.15.43:0/3892881949 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f4b840047c0 0x7f4b980afab0 unknown :-1 s=CLOSED pgs=2227 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T15:08:50.113 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:50.111+0000 7f4b9fac5640 1 -- 172.21.15.43:0/3892881949 >> 172.21.15.43:0/3892881949 conn(0x7f4b9809b7b0 msgr2=0x7f4b980ceda0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-09T15:08:50.113 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:50.111+0000 7f4b9fac5640 1 -- 172.21.15.43:0/3892881949 shutdown_connections 2023-08-09T15:08:50.113 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:50.111+0000 7f4b9fac5640 1 -- 172.21.15.43:0/3892881949 wait complete. 2023-08-09T15:08:50.132 INFO:tasks.cephfs.filesystem.ceph_manager:waiting for clean 2023-08-09T15:08:50.133 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-08-09T15:08:50.358 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:50.357+0000 7ff2decd7640 1 Processor -- start 2023-08-09T15:08:50.358 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:50.358+0000 7ff2decd7640 1 -- start start 2023-08-09T15:08:50.359 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:50.359+0000 7ff2decd7640 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7ff2d806b190 0x7ff2d80ccce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T15:08:50.359 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:50.359+0000 7ff2decd7640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7ff2d806c740 0x7ff2d806cbf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T15:08:50.359 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:50.359+0000 7ff2decd7640 1 -- --> v1:172.21.15.203:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff2d806c2b0 con 0x7ff2d80c87d0 2023-08-09T15:08:50.360 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:50.359+0000 7ff2decd7640 1 -- --> v2:172.21.15.43:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff2d8158eb0 con 0x7ff2d806c740 2023-08-09T15:08:50.360 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:50.359+0000 7ff2decd7640 1 -- --> v2:172.21.15.203:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff2d806b6a0 con 0x7ff2d806b190 2023-08-09T15:08:50.363 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:50.359+0000 7ff2de4d6640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7ff2d806c740 0x7ff2d806cbf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T15:08:50.364 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:50.359+0000 7ff2de4d6640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7ff2d806c740 0x7ff2d806cbf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:39876/0 (socket says 172.21.15.43:39876) 2023-08-09T15:08:50.364 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:50.359+0000 7ff2de4d6640 1 -- 172.21.15.43:0/379211455 learned_addr learned my addr 172.21.15.43:0/379211455 (peer_addr_for_me v2:172.21.15.43:0/0) 2023-08-09T15:08:50.364 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:50.360+0000 7ff2dd4d4640 1 --2- 172.21.15.43:0/379211455 >> v2:172.21.15.203:3300/0 conn(0x7ff2d806b190 0x7ff2d80ccce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T15:08:50.364 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:50.360+0000 7ff2de4d6640 1 -- 172.21.15.43:0/379211455 >> v2:172.21.15.203:3300/0 conn(0x7ff2d806b190 msgr2=0x7ff2d80ccce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-09T15:08:50.364 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:50.360+0000 7ff2de4d6640 1 --2- 172.21.15.43:0/379211455 >> v2:172.21.15.203:3300/0 conn(0x7ff2d806b190 0x7ff2d80ccce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T15:08:50.364 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:50.360+0000 7ff2de4d6640 1 -- 172.21.15.43:0/379211455 >> v1:172.21.15.203:6789/0 conn(0x7ff2d80c87d0 legacy=0x7ff2d80c8bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-09T15:08:50.364 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:50.360+0000 7ff2de4d6640 1 -- 172.21.15.43:0/379211455 --> v2:172.21.15.43:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff2d806b980 con 0x7ff2d806c740 2023-08-09T15:08:50.365 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:50.361+0000 7ff2de4d6640 1 --2- 172.21.15.43:0/379211455 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff2d806c740 0x7ff2d806cbf0 secure :-1 s=READY pgs=2228 cs=0 l=1 rev1=1 crypto rx=0x7ff2d80cb9a0 tx=0x7ff2d002be50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ad9c111a8be0406e server_cookie=0 in_seq=0 out_seq=0 2023-08-09T15:08:50.365 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:50.362+0000 7ff2dccd3640 1 -- 172.21.15.43:0/379211455 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff2d002c9f0 con 0x7ff2d806c740 2023-08-09T15:08:50.365 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:50.362+0000 7ff2dccd3640 1 -- 172.21.15.43:0/379211455 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff2d002cb50 con 0x7ff2d806c740 2023-08-09T15:08:50.365 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:50.362+0000 7ff2dccd3640 1 -- 172.21.15.43:0/379211455 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff2d00343f0 con 0x7ff2d806c740 2023-08-09T15:08:50.365 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:50.362+0000 7ff2decd7640 1 -- 172.21.15.43:0/379211455 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff2d806c740 msgr2=0x7ff2d806cbf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T15:08:50.365 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:50.362+0000 7ff2decd7640 1 --2- 172.21.15.43:0/379211455 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff2d806c740 0x7ff2d806cbf0 secure :-1 s=READY pgs=2228 cs=0 l=1 rev1=1 crypto rx=0x7ff2d80cb9a0 tx=0x7ff2d002be50 comp rx=0 tx=0).stop 2023-08-09T15:08:50.365 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:50.362+0000 7ff2decd7640 1 -- 172.21.15.43:0/379211455 shutdown_connections 2023-08-09T15:08:50.366 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:50.362+0000 7ff2decd7640 1 --2- 172.21.15.43:0/379211455 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff2d806c740 0x7ff2d806cbf0 unknown :-1 s=CLOSED pgs=2228 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T15:08:50.366 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:50.362+0000 7ff2decd7640 1 --2- 172.21.15.43:0/379211455 >> v2:172.21.15.203:3300/0 conn(0x7ff2d806b190 0x7ff2d80ccce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T15:08:50.366 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:50.362+0000 7ff2decd7640 1 -- 172.21.15.43:0/379211455 >> 172.21.15.43:0/379211455 conn(0x7ff2d80cde90 msgr2=0x7ff2d80ce2a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-09T15:08:50.366 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:50.363+0000 7ff2decd7640 1 -- 172.21.15.43:0/379211455 shutdown_connections 2023-08-09T15:08:50.366 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:50.363+0000 7ff2decd7640 1 -- 172.21.15.43:0/379211455 wait complete. 2023-08-09T15:08:50.368 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:50.364+0000 7ff2decd7640 1 Processor -- start 2023-08-09T15:08:50.368 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:50.364+0000 7ff2decd7640 1 -- start start 2023-08-09T15:08:50.368 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:50.365+0000 7ff2decd7640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff2d806b190 0x7ff2d81f4ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T15:08:50.368 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:50.365+0000 7ff2decd7640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7ff2d806c740 0x7ff2d815aa90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T15:08:50.368 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:50.365+0000 7ff2decd7640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7ff2d80c87d0 0x7ff2d80c4f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T15:08:50.368 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:50.365+0000 7ff2decd7640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff2d815b400 con 0x7ff2d806b190 2023-08-09T15:08:50.369 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:50.365+0000 7ff2decd7640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff2d815b6e0 con 0x7ff2d806c740 2023-08-09T15:08:50.369 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:50.365+0000 7ff2decd7640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff2d80997f0 con 0x7ff2d80c87d0 2023-08-09T15:08:50.369 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:50.366+0000 7ff2ddcd5640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff2d806b190 0x7ff2d81f4ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T15:08:50.369 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:50.366+0000 7ff2ddcd5640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff2d806b190 0x7ff2d81f4ff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:39882/0 (socket says 172.21.15.43:39882) 2023-08-09T15:08:50.369 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:50.366+0000 7ff2ddcd5640 1 -- 172.21.15.43:0/1792879230 learned_addr learned my addr 172.21.15.43:0/1792879230 (peer_addr_for_me v2:172.21.15.43:0/0) 2023-08-09T15:08:50.369 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:50.366+0000 7ff2de4d6640 1 --2- 172.21.15.43:0/1792879230 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7ff2d80c87d0 0x7ff2d80c4f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T15:08:50.369 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:50.366+0000 7ff2dd4d4640 1 --2- 172.21.15.43:0/1792879230 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7ff2d806c740 0x7ff2d815aa90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T15:08:50.370 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:50.366+0000 7ff2ddcd5640 1 -- 172.21.15.43:0/1792879230 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7ff2d80c87d0 msgr2=0x7ff2d80c4f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T15:08:50.370 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:50.366+0000 7ff2ddcd5640 1 --2- 172.21.15.43:0/1792879230 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7ff2d80c87d0 0x7ff2d80c4f60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T15:08:50.370 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:50.366+0000 7ff2ddcd5640 1 -- 172.21.15.43:0/1792879230 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7ff2d806c740 msgr2=0x7ff2d815aa90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T15:08:50.370 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:50.366+0000 7ff2ddcd5640 1 --2- 172.21.15.43:0/1792879230 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7ff2d806c740 0x7ff2d815aa90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T15:08:50.370 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:50.366+0000 7ff2ddcd5640 1 -- 172.21.15.43:0/1792879230 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff2d0009b30 con 0x7ff2d806b190 2023-08-09T15:08:50.370 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:50.366+0000 7ff2dd4d4640 1 --2- 172.21.15.43:0/1792879230 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7ff2d806c740 0x7ff2d815aa90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-08-09T15:08:50.370 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:50.366+0000 7ff2ddcd5640 1 --2- 172.21.15.43:0/1792879230 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff2d806b190 0x7ff2d81f4ff0 secure :-1 s=READY pgs=2229 cs=0 l=1 rev1=1 crypto rx=0x7ff2d400c430 tx=0x7ff2d400c8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-09T15:08:50.371 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:50.367+0000 7ff2ce7fc640 1 -- 172.21.15.43:0/1792879230 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff2d4011790 con 0x7ff2d806b190 2023-08-09T15:08:50.371 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:50.367+0000 7ff2ce7fc640 1 -- 172.21.15.43:0/1792879230 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff2d40118f0 con 0x7ff2d806b190 2023-08-09T15:08:50.371 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:50.367+0000 7ff2ce7fc640 1 -- 172.21.15.43:0/1792879230 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff2d4011ce0 con 0x7ff2d806b190 2023-08-09T15:08:50.371 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:50.367+0000 7ff2decd7640 1 -- 172.21.15.43:0/1792879230 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff2d8099ad0 con 0x7ff2d806b190 2023-08-09T15:08:50.371 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:50.367+0000 7ff2decd7640 1 -- 172.21.15.43:0/1792879230 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff2d815afd0 con 0x7ff2d806b190 2023-08-09T15:08:50.372 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:50.368+0000 7ff2decd7640 1 -- 172.21.15.43:0/1792879230 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff2d80739f0 con 0x7ff2d806b190 2023-08-09T15:08:50.372 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:50.369+0000 7ff2ce7fc640 1 -- 172.21.15.43:0/1792879230 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 106) v1 ==== 88443+0+0 (secure 0 0 0) 0x7ff2d401c460 con 0x7ff2d806b190 2023-08-09T15:08:50.372 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:50.370+0000 7ff2ce7fc640 1 --2- 172.21.15.43:0/1792879230 >> [v2:172.21.15.203:6832/3082732769,v1:172.21.15.203:6833/3082732769] conn(0x7ff2bc06a170 0x7ff2bc06c630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T15:08:50.372 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:50.370+0000 7ff2ce7fc640 1 -- 172.21.15.43:0/1792879230 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(205..205 src has 1..205) v4 ==== 6426+0+0 (secure 0 0 0) 0x7ff2d40888a0 con 0x7ff2d806b190 2023-08-09T15:08:50.372 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:50.371+0000 7ff2dd4d4640 1 --2- 172.21.15.43:0/1792879230 >> [v2:172.21.15.203:6832/3082732769,v1:172.21.15.203:6833/3082732769] conn(0x7ff2bc06a170 0x7ff2bc06c630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T15:08:50.373 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:50.372+0000 7ff2dd4d4640 1 --2- 172.21.15.43:0/1792879230 >> [v2:172.21.15.203:6832/3082732769,v1:172.21.15.203:6833/3082732769] conn(0x7ff2bc06a170 0x7ff2bc06c630 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7ff2c8006fd0 tx=0x7ff2c8008040 comp rx=0 tx=0).ready entity=mgr.8500 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-09T15:08:50.375 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:50.375+0000 7ff2ce7fc640 1 -- 172.21.15.43:0/1792879230 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178066 (secure 0 0 0) 0x7ff2d4053c70 con 0x7ff2d806b190 2023-08-09T15:08:50.566 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:50.566+0000 7ff2decd7640 1 -- 172.21.15.43:0/1792879230 --> [v2:172.21.15.203:6832/3082732769,v1:172.21.15.203:6833/3082732769] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7ff2d815b1f0 con 0x7ff2bc06a170 2023-08-09T15:08:50.597 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:50.596+0000 7ff2ce7fc640 1 -- 172.21.15.43:0/1792879230 <== mgr.8500 v2:172.21.15.203:6832/3082732769 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+308709 (secure 0 0 0) 0x7ff2d815b1f0 con 0x7ff2bc06a170 2023-08-09T15:08:50.600 INFO:teuthology.orchestra.run.smithi043.stdout: 2023-08-09T15:08:50.602 INFO:teuthology.orchestra.run.smithi043.stderr:dumped all 2023-08-09T15:08:50.603 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:50.602+0000 7ff2decd7640 1 -- 172.21.15.43:0/1792879230 >> [v2:172.21.15.203:6832/3082732769,v1:172.21.15.203:6833/3082732769] conn(0x7ff2bc06a170 msgr2=0x7ff2bc06c630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T15:08:50.604 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:50.602+0000 7ff2decd7640 1 --2- 172.21.15.43:0/1792879230 >> [v2:172.21.15.203:6832/3082732769,v1:172.21.15.203:6833/3082732769] conn(0x7ff2bc06a170 0x7ff2bc06c630 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7ff2c8006fd0 tx=0x7ff2c8008040 comp rx=0 tx=0).stop 2023-08-09T15:08:50.604 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:50.602+0000 7ff2decd7640 1 -- 172.21.15.43:0/1792879230 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff2d806b190 msgr2=0x7ff2d81f4ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T15:08:50.604 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:50.602+0000 7ff2decd7640 1 --2- 172.21.15.43:0/1792879230 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff2d806b190 0x7ff2d81f4ff0 secure :-1 s=READY pgs=2229 cs=0 l=1 rev1=1 crypto rx=0x7ff2d400c430 tx=0x7ff2d400c8f0 comp rx=0 tx=0).stop 2023-08-09T15:08:50.604 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:50.602+0000 7ff2decd7640 1 -- 172.21.15.43:0/1792879230 shutdown_connections 2023-08-09T15:08:50.604 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:50.602+0000 7ff2decd7640 1 --2- 172.21.15.43:0/1792879230 >> [v2:172.21.15.203:6832/3082732769,v1:172.21.15.203:6833/3082732769] conn(0x7ff2bc06a170 0x7ff2bc06c630 unknown :-1 s=CLOSED pgs=66 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T15:08:50.604 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:50.602+0000 7ff2decd7640 1 --2- 172.21.15.43:0/1792879230 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7ff2d80c87d0 0x7ff2d80c4f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T15:08:50.605 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:50.602+0000 7ff2decd7640 1 --2- 172.21.15.43:0/1792879230 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7ff2d806c740 0x7ff2d815aa90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T15:08:50.605 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:50.602+0000 7ff2decd7640 1 --2- 172.21.15.43:0/1792879230 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff2d806b190 0x7ff2d81f4ff0 unknown :-1 s=CLOSED pgs=2229 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T15:08:50.605 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:50.603+0000 7ff2decd7640 1 -- 172.21.15.43:0/1792879230 >> 172.21.15.43:0/1792879230 conn(0x7ff2d80cde90 msgr2=0x7ff2d80ae1d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-09T15:08:50.605 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:50.603+0000 7ff2decd7640 1 -- 172.21.15.43:0/1792879230 shutdown_connections 2023-08-09T15:08:50.605 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:50.603+0000 7ff2decd7640 1 -- 172.21.15.43:0/1792879230 wait complete. 2023-08-09T15:08:50.624 INFO:teuthology.orchestra.run.smithi043.stdout:{"pg_ready":true,"pg_map":{"version":27,"stamp":"2023-08-09T15:08:49.621614+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":23,"num_object_clones":0,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":0,"num_read":1771,"num_read_kb":3684,"num_write":547,"num_write_kb":2430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":345,"ondisk_log_size":345,"up":210,"acting":210,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":99,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":398776,"kb_used_data":21068,"kb_used_omap":13,"kb_used_meta":377586,"kb_avail":754575944,"statfs":{"total":773094113280,"available":772685766656,"internally_reserved":0,"allocated":21573632,"data_stored":12257246,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":14248,"internal_metadata":386648152},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":30,"apply_latency_ms":30,"commit_latency_ns":30000000,"apply_latency_ns":30000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"5.568343"},"pg_stats":[{"pgid":"25.3f","version":"0'0","reported_seq":9,"reported_epoch":204,"state":"active+clean","last_fresh":"2023-08-09T15:08:46.151623+0000","last_change":"2023-08-09T15:08:46.150997+0000","last_active":"2023-08-09T15:08:46.151623+0000","last_peered":"2023-08-09T15:08:46.151623+0000","last_clean":"2023-08-09T15:08:46.151623+0000","last_became_active":"2023-08-09T15:08:46.150707+0000","last_became_peered":"2023-08-09T15:08:46.150707+0000","last_unstale":"2023-08-09T15:08:46.151623+0000","last_undegraded":"2023-08-09T15:08:46.151623+0000","last_fullsized":"2023-08-09T15:08:46.151623+0000","mapping_epoch":203,"log_start":"0'0","ondisk_log_start":"0'0","created":203,"last_epoch_clean":204,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T15:08:45.049361+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T15:08:45.049361+0000","last_clean_scrub_stamp":"2023-08-09T15:08:45.049361+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T23:34:12.228290+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"25.3e","version":"0'0","reported_seq":9,"reported_epoch":204,"state":"active+clean","last_fresh":"2023-08-09T15:08:46.140267+0000","last_change":"2023-08-09T15:08:46.139233+0000","last_active":"2023-08-09T15:08:46.140267+0000","last_peered":"2023-08-09T15:08:46.140267+0000","last_clean":"2023-08-09T15:08:46.140267+0000","last_became_active":"2023-08-09T15:08:46.138915+0000","last_became_peered":"2023-08-09T15:08:46.138915+0000","last_unstale":"2023-08-09T15:08:46.140267+0000","last_undegraded":"2023-08-09T15:08:46.140267+0000","last_fullsized":"2023-08-09T15:08:46.140267+0000","mapping_epoch":203,"log_start":"0'0","ondisk_log_start":"0'0","created":203,"last_epoch_clean":204,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T15:08:45.049361+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T15:08:45.049361+0000","last_clean_scrub_stamp":"2023-08-09T15:08:45.049361+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T15:13:34.907877+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"25.3d","version":"0'0","reported_seq":9,"reported_epoch":204,"state":"active+clean","last_fresh":"2023-08-09T15:08:46.142938+0000","last_change":"2023-08-09T15:08:46.141564+0000","last_active":"2023-08-09T15:08:46.142938+0000","last_peered":"2023-08-09T15:08:46.142938+0000","last_clean":"2023-08-09T15:08:46.142938+0000","last_became_active":"2023-08-09T15:08:46.139612+0000","last_became_peered":"2023-08-09T15:08:46.139612+0000","last_unstale":"2023-08-09T15:08:46.142938+0000","last_undegraded":"2023-08-09T15:08:46.142938+0000","last_fullsized":"2023-08-09T15:08:46.142938+0000","mapping_epoch":203,"log_start":"0'0","ondisk_log_start":"0'0","created":203,"last_epoch_clean":204,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T15:08:45.049361+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T15:08:45.049361+0000","last_clean_scrub_stamp":"2023-08-09T15:08:45.049361+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T15:23:43.221053+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"25.3c","version":"0'0","reported_seq":9,"reported_epoch":204,"state":"active+clean","last_fresh":"2023-08-09T15:08:46.139835+0000","last_change":"2023-08-09T15:08:46.138550+0000","last_active":"2023-08-09T15:08:46.139835+0000","last_peered":"2023-08-09T15:08:46.139835+0000","last_clean":"2023-08-09T15:08:46.139835+0000","last_became_active":"2023-08-09T15:08:46.138191+0000","last_became_peered":"2023-08-09T15:08:46.138191+0000","last_unstale":"2023-08-09T15:08:46.139835+0000","last_undegraded":"2023-08-09T15:08:46.139835+0000","last_fullsized":"2023-08-09T15:08:46.139835+0000","mapping_epoch":203,"log_start":"0'0","ondisk_log_start":"0'0","created":203,"last_epoch_clean":204,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T15:08:45.049361+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T15:08:45.049361+0000","last_clean_scrub_stamp":"2023-08-09T15:08:45.049361+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T22:35:34.627875+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"25.3b","version":"0'0","reported_seq":9,"reported_epoch":204,"state":"active+clean","last_fresh":"2023-08-09T15:08:46.149607+0000","last_change":"2023-08-09T15:08:46.139068+0000","last_active":"2023-08-09T15:08:46.149607+0000","last_peered":"2023-08-09T15:08:46.149607+0000","last_clean":"2023-08-09T15:08:46.149607+0000","last_became_active":"2023-08-09T15:08:46.138724+0000","last_became_peered":"2023-08-09T15:08:46.138724+0000","last_unstale":"2023-08-09T15:08:46.149607+0000","last_undegraded":"2023-08-09T15:08:46.149607+0000","last_fullsized":"2023-08-09T15:08:46.149607+0000","mapping_epoch":203,"log_start":"0'0","ondisk_log_start":"0'0","created":203,"last_epoch_clean":204,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T15:08:45.049361+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T15:08:45.049361+0000","last_clean_scrub_stamp":"2023-08-09T15:08:45.049361+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T22:31:17.210241+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"25.3a","version":"0'0","reported_seq":9,"reported_epoch":204,"state":"active+clean","last_fresh":"2023-08-09T15:08:46.142865+0000","last_change":"2023-08-09T15:08:46.133362+0000","last_active":"2023-08-09T15:08:46.142865+0000","last_peered":"2023-08-09T15:08:46.142865+0000","last_clean":"2023-08-09T15:08:46.142865+0000","last_became_active":"2023-08-09T15:08:46.133068+0000","last_became_peered":"2023-08-09T15:08:46.133068+0000","last_unstale":"2023-08-09T15:08:46.142865+0000","last_undegraded":"2023-08-09T15:08:46.142865+0000","last_fullsized":"2023-08-09T15:08:46.142865+0000","mapping_epoch":203,"log_start":"0'0","ondisk_log_start":"0'0","created":203,"last_epoch_clean":204,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T15:08:45.049361+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T15:08:45.049361+0000","last_clean_scrub_stamp":"2023-08-09T15:08:45.049361+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T22:33:34.990604+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"25.39","version":"0'0","reported_seq":9,"reported_epoch":204,"state":"active+clean","last_fresh":"2023-08-09T15:08:46.132517+0000","last_change":"2023-08-09T15:08:46.126835+0000","last_active":"2023-08-09T15:08:46.132517+0000","last_peered":"2023-08-09T15:08:46.132517+0000","last_clean":"2023-08-09T15:08:46.132517+0000","last_became_active":"2023-08-09T15:08:46.126531+0000","last_became_peered":"2023-08-09T15:08:46.126531+0000","last_unstale":"2023-08-09T15:08:46.132517+0000","last_undegraded":"2023-08-09T15:08:46.132517+0000","last_fullsized":"2023-08-09T15:08:46.132517+0000","mapping_epoch":203,"log_start":"0'0","ondisk_log_start":"0'0","created":203,"last_epoch_clean":204,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T15:08:45.049361+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T15:08:45.049361+0000","last_clean_scrub_stamp":"2023-08-09T15:08:45.049361+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T17:53:57.019159+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"25.38","version":"0'0","reported_seq":9,"reported_epoch":204,"state":"active+clean","last_fresh":"2023-08-09T15:08:46.135716+0000","last_change":"2023-08-09T15:08:46.122273+0000","last_active":"2023-08-09T15:08:46.135716+0000","last_peered":"2023-08-09T15:08:46.135716+0000","last_clean":"2023-08-09T15:08:46.135716+0000","last_became_active":"2023-08-09T15:08:46.121879+0000","last_became_peered":"2023-08-09T15:08:46.121879+0000","last_unstale":"2023-08-09T15:08:46.135716+0000","last_undegraded":"2023-08-09T15:08:46.135716+0000","last_fullsized":"2023-08-09T15:08:46.135716+0000","mapping_epoch":203,"log_start":"0'0","ondisk_log_start":"0'0","created":203,"last_epoch_clean":204,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T15:08:45.049361+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T15:08:45.049361+0000","last_clean_scrub_stamp":"2023-08-09T15:08:45.049361+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T17:56:00.843251+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"25.37","version":"0'0","reported_seq":9,"reported_epoch":204,"state":"active+clean","last_fresh":"2023-08-09T15:08:46.130710+0000","last_change":"2023-08-09T15:08:46.129914+0000","last_active":"2023-08-09T15:08:46.130710+0000","last_peered":"2023-08-09T15:08:46.130710+0000","last_clean":"2023-08-09T15:08:46.130710+0000","last_became_active":"2023-08-09T15:08:46.129632+0000","last_became_peered":"2023-08-09T15:08:46.129632+0000","last_unstale":"2023-08-09T15:08:46.130710+0000","last_undegraded":"2023-08-09T15:08:46.130710+0000","last_fullsized":"2023-08-09T15:08:46.130710+0000","mapping_epoch":203,"log_start":"0'0","ondisk_log_start":"0'0","created":203,"last_epoch_clean":204,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T15:08:45.049361+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T15:08:45.049361+0000","last_clean_scrub_stamp":"2023-08-09T15:08:45.049361+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T17:38:53.025570+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"25.36","version":"0'0","reported_seq":9,"reported_epoch":204,"state":"active+clean","last_fresh":"2023-08-09T15:08:46.128715+0000","last_change":"2023-08-09T15:08:46.125176+0000","last_active":"2023-08-09T15:08:46.128715+0000","last_peered":"2023-08-09T15:08:46.128715+0000","last_clean":"2023-08-09T15:08:46.128715+0000","last_became_active":"2023-08-09T15:08:46.124622+0000","last_became_peered":"2023-08-09T15:08:46.124622+0000","last_unstale":"2023-08-09T15:08:46.128715+0000","last_undegraded":"2023-08-09T15:08:46.128715+0000","last_fullsized":"2023-08-09T15:08:46.128715+0000","mapping_epoch":203,"log_start":"0'0","ondisk_log_start":"0'0","created":203,"last_epoch_clean":204,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T15:08:45.049361+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T15:08:45.049361+0000","last_clean_scrub_stamp":"2023-08-09T15:08:45.049361+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-11T00:10:28.556439+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"25.35","version":"0'0","reported_seq":9,"reported_epoch":204,"state":"active+clean","last_fresh":"2023-08-09T15:08:46.139319+0000","last_change":"2023-08-09T15:08:46.137191+0000","last_active":"2023-08-09T15:08:46.139319+0000","last_peered":"2023-08-09T15:08:46.139319+0000","last_clean":"2023-08-09T15:08:46.139319+0000","last_became_active":"2023-08-09T15:08:46.136655+0000","last_became_peered":"2023-08-09T15:08:46.136655+0000","last_unstale":"2023-08-09T15:08:46.139319+0000","last_undegraded":"2023-08-09T15:08:46.139319+0000","last_fullsized":"2023-08-09T15:08:46.139319+0000","mapping_epoch":203,"log_start":"0'0","ondisk_log_start":"0'0","created":203,"last_epoch_clean":204,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T15:08:45.049361+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T15:08:45.049361+0000","last_clean_scrub_stamp":"2023-08-09T15:08:45.049361+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-11T02:09:17.233487+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"25.34","version":"0'0","reported_seq":9,"reported_epoch":204,"state":"active+clean","last_fresh":"2023-08-09T15:08:46.135115+0000","last_change":"2023-08-09T15:08:46.133622+0000","last_active":"2023-08-09T15:08:46.135115+0000","last_peered":"2023-08-09T15:08:46.135115+0000","last_clean":"2023-08-09T15:08:46.135115+0000","last_became_active":"2023-08-09T15:08:46.133275+0000","last_became_peered":"2023-08-09T15:08:46.133275+0000","last_unstale":"2023-08-09T15:08:46.135115+0000","last_undegraded":"2023-08-09T15:08:46.135115+0000","last_fullsized":"2023-08-09T15:08:46.135115+0000","mapping_epoch":203,"log_start":"0'0","ondisk_log_start":"0'0","created":203,"last_epoch_clean":204,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T15:08:45.049361+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T15:08:45.049361+0000","last_clean_scrub_stamp":"2023-08-09T15:08:45.049361+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-11T02:31:31.905005+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"25.33","version":"0'0","reported_seq":9,"reported_epoch":204,"state":"active+clean","last_fresh":"2023-08-09T15:08:46.135012+0000","last_change":"2023-08-09T15:08:46.131224+0000","last_active":"2023-08-09T15:08:46.135012+0000","last_peered":"2023-08-09T15:08:46.135012+0000","last_clean":"2023-08-09T15:08:46.135012+0000","last_became_active":"2023-08-09T15:08:46.130861+0000","last_became_peered":"2023-08-09T15:08:46.130861+0000","last_unstale":"2023-08-09T15:08:46.135012+0000","last_undegraded":"2023-08-09T15:08:46.135012+0000","last_fullsized":"2023-08-09T15:08:46.135012+0000","mapping_epoch":203,"log_start":"0'0","ondisk_log_start":"0'0","created":203,"last_epoch_clean":204,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T15:08:45.049361+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T15:08:45.049361+0000","last_clean_scrub_stamp":"2023-08-09T15:08:45.049361+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-11T00:12:28.783608+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"25.32","version":"0'0","reported_seq":9,"reported_epoch":204,"state":"active+clean","last_fresh":"2023-08-09T15:08:46.135540+0000","last_change":"2023-08-09T15:08:46.132431+0000","last_active":"2023-08-09T15:08:46.135540+0000","last_peered":"2023-08-09T15:08:46.135540+0000","last_clean":"2023-08-09T15:08:46.135540+0000","last_became_active":"2023-08-09T15:08:46.132176+0000","last_became_peered":"2023-08-09T15:08:46.132176+0000","last_unstale":"2023-08-09T15:08:46.135540+0000","last_undegraded":"2023-08-09T15:08:46.135540+0000","last_fullsized":"2023-08-09T15:08:46.135540+0000","mapping_epoch":203,"log_start":"0'0","ondisk_log_start":"0'0","created":203,"last_epoch_clean":204,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T15:08:45.049361+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T15:08:45.049361+0000","last_clean_scrub_stamp":"2023-08-09T15:08:45.049361+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T19:53:04.905618+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"25.31","version":"0'0","reported_seq":9,"reported_epoch":204,"state":"active+clean","last_fresh":"2023-08-09T15:08:46.135898+0000","last_change":"2023-08-09T15:08:46.134472+0000","last_active":"2023-08-09T15:08:46.135898+0000","last_peered":"2023-08-09T15:08:46.135898+0000","last_clean":"2023-08-09T15:08:46.135898+0000","last_became_active":"2023-08-09T15:08:46.134023+0000","last_became_peered":"2023-08-09T15:08:46.134023+0000","last_unstale":"2023-08-09T15:08:46.135898+0000","last_undegraded":"2023-08-09T15:08:46.135898+0000","last_fullsized":"2023-08-09T15:08:46.135898+0000","mapping_epoch":203,"log_start":"0'0","ondisk_log_start":"0'0","created":203,"last_epoch_clean":204,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T15:08:45.049361+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T15:08:45.049361+0000","last_clean_scrub_stamp":"2023-08-09T15:08:45.049361+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T23:06:50.796980+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"25.30","version":"0'0","reported_seq":9,"reported_epoch":204,"state":"active+clean","last_fresh":"2023-08-09T15:08:46.136595+0000","last_change":"2023-08-09T15:08:46.132274+0000","last_active":"2023-08-09T15:08:46.136595+0000","last_peered":"2023-08-09T15:08:46.136595+0000","last_clean":"2023-08-09T15:08:46.136595+0000","last_became_active":"2023-08-09T15:08:46.131966+0000","last_became_peered":"2023-08-09T15:08:46.131966+0000","last_unstale":"2023-08-09T15:08:46.136595+0000","last_undegraded":"2023-08-09T15:08:46.136595+0000","last_fullsized":"2023-08-09T15:08:46.136595+0000","mapping_epoch":203,"log_start":"0'0","ondisk_log_start":"0'0","created":203,"last_epoch_clean":204,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T15:08:45.049361+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T15:08:45.049361+0000","last_clean_scrub_stamp":"2023-08-09T15:08:45.049361+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T18:39:53.638768+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"25.2f","version":"0'0","reported_seq":9,"reported_epoch":204,"state":"active+clean","last_fresh":"2023-08-09T15:08:46.148093+0000","last_change":"2023-08-09T15:08:46.139165+0000","last_active":"2023-08-09T15:08:46.148093+0000","last_peered":"2023-08-09T15:08:46.148093+0000","last_clean":"2023-08-09T15:08:46.148093+0000","last_became_active":"2023-08-09T15:08:46.138796+0000","last_became_peered":"2023-08-09T15:08:46.138796+0000","last_unstale":"2023-08-09T15:08:46.148093+0000","last_undegraded":"2023-08-09T15:08:46.148093+0000","last_fullsized":"2023-08-09T15:08:46.148093+0000","mapping_epoch":203,"log_start":"0'0","ondisk_log_start":"0'0","created":203,"last_epoch_clean":204,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T15:08:45.049361+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T15:08:45.049361+0000","last_clean_scrub_stamp":"2023-08-09T15:08:45.049361+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-11T03:04:07.233708+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"25.2e","version":"0'0","reported_seq":9,"reported_epoch":204,"state":"active+clean","last_fresh":"2023-08-09T15:08:46.140492+0000","last_change":"2023-08-09T15:08:46.138485+0000","last_active":"2023-08-09T15:08:46.140492+0000","last_peered":"2023-08-09T15:08:46.140492+0000","last_clean":"2023-08-09T15:08:46.140492+0000","last_became_active":"2023-08-09T15:08:46.138217+0000","last_became_peered":"2023-08-09T15:08:46.138217+0000","last_unstale":"2023-08-09T15:08:46.140492+0000","last_undegraded":"2023-08-09T15:08:46.140492+0000","last_fullsized":"2023-08-09T15:08:46.140492+0000","mapping_epoch":203,"log_start":"0'0","ondisk_log_start":"0'0","created":203,"last_epoch_clean":204,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T15:08:45.049361+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T15:08:45.049361+0000","last_clean_scrub_stamp":"2023-08-09T15:08:45.049361+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T16:48:46.531516+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"25.2d","version":"0'0","reported_seq":9,"reported_epoch":204,"state":"active+clean","last_fresh":"2023-08-09T15:08:46.149742+0000","last_change":"2023-08-09T15:08:46.141111+0000","last_active":"2023-08-09T15:08:46.149742+0000","last_peered":"2023-08-09T15:08:46.149742+0000","last_clean":"2023-08-09T15:08:46.149742+0000","last_became_active":"2023-08-09T15:08:46.137325+0000","last_became_peered":"2023-08-09T15:08:46.137325+0000","last_unstale":"2023-08-09T15:08:46.149742+0000","last_undegraded":"2023-08-09T15:08:46.149742+0000","last_fullsized":"2023-08-09T15:08:46.149742+0000","mapping_epoch":203,"log_start":"0'0","ondisk_log_start":"0'0","created":203,"last_epoch_clean":204,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T15:08:45.049361+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T15:08:45.049361+0000","last_clean_scrub_stamp":"2023-08-09T15:08:45.049361+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T16:08:16.971247+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"25.2c","version":"0'0","reported_seq":9,"reported_epoch":204,"state":"active+clean","last_fresh":"2023-08-09T15:08:46.147332+0000","last_change":"2023-08-09T15:08:46.138291+0000","last_active":"2023-08-09T15:08:46.147332+0000","last_peered":"2023-08-09T15:08:46.147332+0000","last_clean":"2023-08-09T15:08:46.147332+0000","last_became_active":"2023-08-09T15:08:46.133273+0000","last_became_peered":"2023-08-09T15:08:46.133273+0000","last_unstale":"2023-08-09T15:08:46.147332+0000","last_undegraded":"2023-08-09T15:08:46.147332+0000","last_fullsized":"2023-08-09T15:08:46.147332+0000","mapping_epoch":203,"log_start":"0'0","ondisk_log_start":"0'0","created":203,"last_epoch_clean":204,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T15:08:45.049361+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T15:08:45.049361+0000","last_clean_scrub_stamp":"2023-08-09T15:08:45.049361+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T23:54:29.242692+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"25.2b","version":"0'0","reported_seq":9,"reported_epoch":204,"state":"active+clean","last_fresh":"2023-08-09T15:08:46.145468+0000","last_change":"2023-08-09T15:08:46.135809+0000","last_active":"2023-08-09T15:08:46.145468+0000","last_peered":"2023-08-09T15:08:46.145468+0000","last_clean":"2023-08-09T15:08:46.145468+0000","last_became_active":"2023-08-09T15:08:46.135429+0000","last_became_peered":"2023-08-09T15:08:46.135429+0000","last_unstale":"2023-08-09T15:08:46.145468+0000","last_undegraded":"2023-08-09T15:08:46.145468+0000","last_fullsized":"2023-08-09T15:08:46.145468+0000","mapping_epoch":203,"log_start":"0'0","ondisk_log_start":"0'0","created":203,"last_epoch_clean":204,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T15:08:45.049361+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T15:08:45.049361+0000","last_clean_scrub_stamp":"2023-08-09T15:08:45.049361+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-11T01:46:34.819975+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"25.2a","version":"0'0","reported_seq":9,"reported_epoch":204,"state":"active+clean","last_fresh":"2023-08-09T15:08:46.149972+0000","last_change":"2023-08-09T15:08:46.144130+0000","last_active":"2023-08-09T15:08:46.149972+0000","last_peered":"2023-08-09T15:08:46.149972+0000","last_clean":"2023-08-09T15:08:46.149972+0000","last_became_active":"2023-08-09T15:08:46.142345+0000","last_became_peered":"2023-08-09T15:08:46.142345+0000","last_unstale":"2023-08-09T15:08:46.149972+0000","last_undegraded":"2023-08-09T15:08:46.149972+0000","last_fullsized":"2023-08-09T15:08:46.149972+0000","mapping_epoch":203,"log_start":"0'0","ondisk_log_start":"0'0","created":203,"last_epoch_clean":204,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T15:08:45.049361+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T15:08:45.049361+0000","last_clean_scrub_stamp":"2023-08-09T15:08:45.049361+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T21:32:52.411688+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"25.29","version":"0'0","reported_seq":9,"reported_epoch":204,"state":"active+clean","last_fresh":"2023-08-09T15:08:46.136326+0000","last_change":"2023-08-09T15:08:46.125725+0000","last_active":"2023-08-09T15:08:46.136326+0000","last_peered":"2023-08-09T15:08:46.136326+0000","last_clean":"2023-08-09T15:08:46.136326+0000","last_became_active":"2023-08-09T15:08:46.125456+0000","last_became_peered":"2023-08-09T15:08:46.125456+0000","last_unstale":"2023-08-09T15:08:46.136326+0000","last_undegraded":"2023-08-09T15:08:46.136326+0000","last_fullsized":"2023-08-09T15:08:46.136326+0000","mapping_epoch":203,"log_start":"0'0","ondisk_log_start":"0'0","created":203,"last_epoch_clean":204,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T15:08:45.049361+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T15:08:45.049361+0000","last_clean_scrub_stamp":"2023-08-09T15:08:45.049361+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T19:10:37.660853+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"25.28","version":"0'0","reported_seq":9,"reported_epoch":204,"state":"active+clean","last_fresh":"2023-08-09T15:08:46.136571+0000","last_change":"2023-08-09T15:08:46.129604+0000","last_active":"2023-08-09T15:08:46.136571+0000","last_peered":"2023-08-09T15:08:46.136571+0000","last_clean":"2023-08-09T15:08:46.136571+0000","last_became_active":"2023-08-09T15:08:46.129287+0000","last_became_peered":"2023-08-09T15:08:46.129287+0000","last_unstale":"2023-08-09T15:08:46.136571+0000","last_undegraded":"2023-08-09T15:08:46.136571+0000","last_fullsized":"2023-08-09T15:08:46.136571+0000","mapping_epoch":203,"log_start":"0'0","ondisk_log_start":"0'0","created":203,"last_epoch_clean":204,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T15:08:45.049361+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T15:08:45.049361+0000","last_clean_scrub_stamp":"2023-08-09T15:08:45.049361+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T15:43:17.545121+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"25.27","version":"0'0","reported_seq":9,"reported_epoch":204,"state":"active+clean","last_fresh":"2023-08-09T15:08:46.142873+0000","last_change":"2023-08-09T15:08:46.141818+0000","last_active":"2023-08-09T15:08:46.142873+0000","last_peered":"2023-08-09T15:08:46.142873+0000","last_clean":"2023-08-09T15:08:46.142873+0000","last_became_active":"2023-08-09T15:08:46.141512+0000","last_became_peered":"2023-08-09T15:08:46.141512+0000","last_unstale":"2023-08-09T15:08:46.142873+0000","last_undegraded":"2023-08-09T15:08:46.142873+0000","last_fullsized":"2023-08-09T15:08:46.142873+0000","mapping_epoch":203,"log_start":"0'0","ondisk_log_start":"0'0","created":203,"last_epoch_clean":204,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T15:08:45.049361+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T15:08:45.049361+0000","last_clean_scrub_stamp":"2023-08-09T15:08:45.049361+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-11T00:17:53.301522+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"25.26","version":"0'0","reported_seq":9,"reported_epoch":204,"state":"active+clean","last_fresh":"2023-08-09T15:08:46.129717+0000","last_change":"2023-08-09T15:08:46.123803+0000","last_active":"2023-08-09T15:08:46.129717+0000","last_peered":"2023-08-09T15:08:46.129717+0000","last_clean":"2023-08-09T15:08:46.129717+0000","last_became_active":"2023-08-09T15:08:46.123453+0000","last_became_peered":"2023-08-09T15:08:46.123453+0000","last_unstale":"2023-08-09T15:08:46.129717+0000","last_undegraded":"2023-08-09T15:08:46.129717+0000","last_fullsized":"2023-08-09T15:08:46.129717+0000","mapping_epoch":203,"log_start":"0'0","ondisk_log_start":"0'0","created":203,"last_epoch_clean":204,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T15:08:45.049361+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T15:08:45.049361+0000","last_clean_scrub_stamp":"2023-08-09T15:08:45.049361+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T22:44:05.779076+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"25.25","version":"0'0","reported_seq":9,"reported_epoch":204,"state":"active+clean","last_fresh":"2023-08-09T15:08:46.142920+0000","last_change":"2023-08-09T15:08:46.141330+0000","last_active":"2023-08-09T15:08:46.142920+0000","last_peered":"2023-08-09T15:08:46.142920+0000","last_clean":"2023-08-09T15:08:46.142920+0000","last_became_active":"2023-08-09T15:08:46.141065+0000","last_became_peered":"2023-08-09T15:08:46.141065+0000","last_unstale":"2023-08-09T15:08:46.142920+0000","last_undegraded":"2023-08-09T15:08:46.142920+0000","last_fullsized":"2023-08-09T15:08:46.142920+0000","mapping_epoch":203,"log_start":"0'0","ondisk_log_start":"0'0","created":203,"last_epoch_clean":204,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T15:08:45.049361+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T15:08:45.049361+0000","last_clean_scrub_stamp":"2023-08-09T15:08:45.049361+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-11T02:56:50.712559+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"25.24","version":"0'0","reported_seq":9,"reported_epoch":204,"state":"active+clean","last_fresh":"2023-08-09T15:08:46.148941+0000","last_change":"2023-08-09T15:08:46.140614+0000","last_active":"2023-08-09T15:08:46.148941+0000","last_peered":"2023-08-09T15:08:46.148941+0000","last_clean":"2023-08-09T15:08:46.148941+0000","last_became_active":"2023-08-09T15:08:46.138328+0000","last_became_peered":"2023-08-09T15:08:46.138328+0000","last_unstale":"2023-08-09T15:08:46.148941+0000","last_undegraded":"2023-08-09T15:08:46.148941+0000","last_fullsized":"2023-08-09T15:08:46.148941+0000","mapping_epoch":203,"log_start":"0'0","ondisk_log_start":"0'0","created":203,"last_epoch_clean":204,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T15:08:45.049361+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T15:08:45.049361+0000","last_clean_scrub_stamp":"2023-08-09T15:08:45.049361+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T22:39:17.933987+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"25.23","version":"0'0","reported_seq":9,"reported_epoch":204,"state":"active+clean","last_fresh":"2023-08-09T15:08:46.135127+0000","last_change":"2023-08-09T15:08:46.133633+0000","last_active":"2023-08-09T15:08:46.135127+0000","last_peered":"2023-08-09T15:08:46.135127+0000","last_clean":"2023-08-09T15:08:46.135127+0000","last_became_active":"2023-08-09T15:08:46.133330+0000","last_became_peered":"2023-08-09T15:08:46.133330+0000","last_unstale":"2023-08-09T15:08:46.135127+0000","last_undegraded":"2023-08-09T15:08:46.135127+0000","last_fullsized":"2023-08-09T15:08:46.135127+0000","mapping_epoch":203,"log_start":"0'0","ondisk_log_start":"0'0","created":203,"last_epoch_clean":204,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T15:08:45.049361+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T15:08:45.049361+0000","last_clean_scrub_stamp":"2023-08-09T15:08:45.049361+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T20:38:09.810823+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"25.22","version":"0'0","reported_seq":9,"reported_epoch":204,"state":"active+clean","last_fresh":"2023-08-09T15:08:46.135520+0000","last_change":"2023-08-09T15:08:46.131940+0000","last_active":"2023-08-09T15:08:46.135520+0000","last_peered":"2023-08-09T15:08:46.135520+0000","last_clean":"2023-08-09T15:08:46.135520+0000","last_became_active":"2023-08-09T15:08:46.131576+0000","last_became_peered":"2023-08-09T15:08:46.131576+0000","last_unstale":"2023-08-09T15:08:46.135520+0000","last_undegraded":"2023-08-09T15:08:46.135520+0000","last_fullsized":"2023-08-09T15:08:46.135520+0000","mapping_epoch":203,"log_start":"0'0","ondisk_log_start":"0'0","created":203,"last_epoch_clean":204,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T15:08:45.049361+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T15:08:45.049361+0000","last_clean_scrub_stamp":"2023-08-09T15:08:45.049361+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-11T01:07:15.792938+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"25.21","version":"0'0","reported_seq":9,"reported_epoch":204,"state":"active+clean","last_fresh":"2023-08-09T15:08:46.136279+0000","last_change":"2023-08-09T15:08:46.125173+0000","last_active":"2023-08-09T15:08:46.136279+0000","last_peered":"2023-08-09T15:08:46.136279+0000","last_clean":"2023-08-09T15:08:46.136279+0000","last_became_active":"2023-08-09T15:08:46.124880+0000","last_became_peered":"2023-08-09T15:08:46.124880+0000","last_unstale":"2023-08-09T15:08:46.136279+0000","last_undegraded":"2023-08-09T15:08:46.136279+0000","last_fullsized":"2023-08-09T15:08:46.136279+0000","mapping_epoch":203,"log_start":"0'0","ondisk_log_start":"0'0","created":203,"last_epoch_clean":204,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T15:08:45.049361+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T15:08:45.049361+0000","last_clean_scrub_stamp":"2023-08-09T15:08:45.049361+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T20:32:20.754257+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"25.20","version":"0'0","reported_seq":9,"reported_epoch":204,"state":"active+clean","last_fresh":"2023-08-09T15:08:46.144251+0000","last_change":"2023-08-09T15:08:46.134105+0000","last_active":"2023-08-09T15:08:46.144251+0000","last_peered":"2023-08-09T15:08:46.144251+0000","last_clean":"2023-08-09T15:08:46.144251+0000","last_became_active":"2023-08-09T15:08:46.133781+0000","last_became_peered":"2023-08-09T15:08:46.133781+0000","last_unstale":"2023-08-09T15:08:46.144251+0000","last_undegraded":"2023-08-09T15:08:46.144251+0000","last_fullsized":"2023-08-09T15:08:46.144251+0000","mapping_epoch":203,"log_start":"0'0","ondisk_log_start":"0'0","created":203,"last_epoch_clean":204,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T15:08:45.049361+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T15:08:45.049361+0000","last_clean_scrub_stamp":"2023-08-09T15:08:45.049361+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T21:54:42.038300+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"25.16","version":"0'0","reported_seq":9,"reported_epoch":204,"state":"active+clean","last_fresh":"2023-08-09T15:08:46.136443+0000","last_change":"2023-08-09T15:08:46.128258+0000","last_active":"2023-08-09T15:08:46.136443+0000","last_peered":"2023-08-09T15:08:46.136443+0000","last_clean":"2023-08-09T15:08:46.136443+0000","last_became_active":"2023-08-09T15:08:46.127605+0000","last_became_peered":"2023-08-09T15:08:46.127605+0000","last_unstale":"2023-08-09T15:08:46.136443+0000","last_undegraded":"2023-08-09T15:08:46.136443+0000","last_fullsized":"2023-08-09T15:08:46.136443+0000","mapping_epoch":203,"log_start":"0'0","ondisk_log_start":"0'0","created":203,"last_epoch_clean":204,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T15:08:45.049361+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T15:08:45.049361+0000","last_clean_scrub_stamp":"2023-08-09T15:08:45.049361+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-11T02:45:10.800604+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"24.17","version":"0'0","reported_seq":11,"reported_epoch":204,"state":"active+clean","last_fresh":"2023-08-09T15:08:46.104628+0000","last_change":"2023-08-09T15:08:45.088142+0000","last_active":"2023-08-09T15:08:46.104628+0000","last_peered":"2023-08-09T15:08:46.104628+0000","last_clean":"2023-08-09T15:08:46.104628+0000","last_became_active":"2023-08-09T15:08:45.087867+0000","last_became_peered":"2023-08-09T15:08:45.087867+0000","last_unstale":"2023-08-09T15:08:46.104628+0000","last_undegraded":"2023-08-09T15:08:46.104628+0000","last_fullsized":"2023-08-09T15:08:46.104628+0000","mapping_epoch":202,"log_start":"0'0","ondisk_log_start":"0'0","created":202,"last_epoch_clean":203,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T15:08:44.039630+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T15:08:44.039630+0000","last_clean_scrub_stamp":"2023-08-09T15:08:44.039630+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T17:42:15.287480+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"25.17","version":"0'0","reported_seq":9,"reported_epoch":204,"state":"active+clean","last_fresh":"2023-08-09T15:08:46.135975+0000","last_change":"2023-08-09T15:08:46.128585+0000","last_active":"2023-08-09T15:08:46.135975+0000","last_peered":"2023-08-09T15:08:46.135975+0000","last_clean":"2023-08-09T15:08:46.135975+0000","last_became_active":"2023-08-09T15:08:46.128272+0000","last_became_peered":"2023-08-09T15:08:46.128272+0000","last_unstale":"2023-08-09T15:08:46.135975+0000","last_undegraded":"2023-08-09T15:08:46.135975+0000","last_fullsized":"2023-08-09T15:08:46.135975+0000","mapping_epoch":203,"log_start":"0'0","ondisk_log_start":"0'0","created":203,"last_epoch_clean":204,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T15:08:45.049361+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T15:08:45.049361+0000","last_clean_scrub_stamp":"2023-08-09T15:08:45.049361+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-11T01:10:22.933001+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"24.16","version":"0'0","reported_seq":11,"reported_epoch":204,"state":"active+clean","last_fresh":"2023-08-09T15:08:46.117412+0000","last_change":"2023-08-09T15:08:45.093806+0000","last_active":"2023-08-09T15:08:46.117412+0000","last_peered":"2023-08-09T15:08:46.117412+0000","last_clean":"2023-08-09T15:08:46.117412+0000","last_became_active":"2023-08-09T15:08:45.093473+0000","last_became_peered":"2023-08-09T15:08:45.093473+0000","last_unstale":"2023-08-09T15:08:46.117412+0000","last_undegraded":"2023-08-09T15:08:46.117412+0000","last_fullsized":"2023-08-09T15:08:46.117412+0000","mapping_epoch":202,"log_start":"0'0","ondisk_log_start":"0'0","created":202,"last_epoch_clean":203,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T15:08:44.039630+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T15:08:44.039630+0000","last_clean_scrub_stamp":"2023-08-09T15:08:44.039630+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T20:43:13.822110+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"25.14","version":"0'0","reported_seq":9,"reported_epoch":204,"state":"active+clean","last_fresh":"2023-08-09T15:08:46.130621+0000","last_change":"2023-08-09T15:08:46.126770+0000","last_active":"2023-08-09T15:08:46.130621+0000","last_peered":"2023-08-09T15:08:46.130621+0000","last_clean":"2023-08-09T15:08:46.130621+0000","last_became_active":"2023-08-09T15:08:46.126465+0000","last_became_peered":"2023-08-09T15:08:46.126465+0000","last_unstale":"2023-08-09T15:08:46.130621+0000","last_undegraded":"2023-08-09T15:08:46.130621+0000","last_fullsized":"2023-08-09T15:08:46.130621+0000","mapping_epoch":203,"log_start":"0'0","ondisk_log_start":"0'0","created":203,"last_epoch_clean":204,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T15:08:45.049361+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T15:08:45.049361+0000","last_clean_scrub_stamp":"2023-08-09T15:08:45.049361+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T21:17:05.616400+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"24.15","version":"204'3","reported_seq":14,"reported_epoch":204,"state":"active+clean","last_fresh":"2023-08-09T15:08:46.155929+0000","last_change":"2023-08-09T15:08:45.095626+0000","last_active":"2023-08-09T15:08:46.155929+0000","last_peered":"2023-08-09T15:08:46.155929+0000","last_clean":"2023-08-09T15:08:46.155929+0000","last_became_active":"2023-08-09T15:08:45.095242+0000","last_became_peered":"2023-08-09T15:08:45.095242+0000","last_unstale":"2023-08-09T15:08:46.155929+0000","last_undegraded":"2023-08-09T15:08:46.155929+0000","last_fullsized":"2023-08-09T15:08:46.155929+0000","mapping_epoch":202,"log_start":"0'0","ondisk_log_start":"0'0","created":202,"last_epoch_clean":203,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T15:08:44.039630+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T15:08:44.039630+0000","last_clean_scrub_stamp":"2023-08-09T15:08:44.039630+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-11T00:50:08.710962+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"25.15","version":"0'0","reported_seq":9,"reported_epoch":204,"state":"active+clean","last_fresh":"2023-08-09T15:08:46.149763+0000","last_change":"2023-08-09T15:08:46.141740+0000","last_active":"2023-08-09T15:08:46.149763+0000","last_peered":"2023-08-09T15:08:46.149763+0000","last_clean":"2023-08-09T15:08:46.149763+0000","last_became_active":"2023-08-09T15:08:46.141428+0000","last_became_peered":"2023-08-09T15:08:46.141428+0000","last_unstale":"2023-08-09T15:08:46.149763+0000","last_undegraded":"2023-08-09T15:08:46.149763+0000","last_fullsized":"2023-08-09T15:08:46.149763+0000","mapping_epoch":203,"log_start":"0'0","ondisk_log_start":"0'0","created":203,"last_epoch_clean":204,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T15:08:45.049361+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T15:08:45.049361+0000","last_clean_scrub_stamp":"2023-08-09T15:08:45.049361+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T17:25:01.501655+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"24.14","version":"204'6","reported_seq":17,"reported_epoch":204,"state":"active+clean","last_fresh":"2023-08-09T15:08:46.157247+0000","last_change":"2023-08-09T15:08:45.089903+0000","last_active":"2023-08-09T15:08:46.157247+0000","last_peered":"2023-08-09T15:08:46.157247+0000","last_clean":"2023-08-09T15:08:46.157247+0000","last_became_active":"2023-08-09T15:08:45.089610+0000","last_became_peered":"2023-08-09T15:08:45.089610+0000","last_unstale":"2023-08-09T15:08:46.157247+0000","last_undegraded":"2023-08-09T15:08:46.157247+0000","last_fullsized":"2023-08-09T15:08:46.157247+0000","mapping_epoch":202,"log_start":"0'0","ondisk_log_start":"0'0","created":202,"last_epoch_clean":203,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T15:08:44.039630+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T15:08:44.039630+0000","last_clean_scrub_stamp":"2023-08-09T15:08:44.039630+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T17:16:30.418309+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"25.12","version":"0'0","reported_seq":9,"reported_epoch":204,"state":"active+clean","last_fresh":"2023-08-09T15:08:46.137120+0000","last_change":"2023-08-09T15:08:46.127062+0000","last_active":"2023-08-09T15:08:46.137120+0000","last_peered":"2023-08-09T15:08:46.137120+0000","last_clean":"2023-08-09T15:08:46.137120+0000","last_became_active":"2023-08-09T15:08:46.126722+0000","last_became_peered":"2023-08-09T15:08:46.126722+0000","last_unstale":"2023-08-09T15:08:46.137120+0000","last_undegraded":"2023-08-09T15:08:46.137120+0000","last_fullsized":"2023-08-09T15:08:46.137120+0000","mapping_epoch":203,"log_start":"0'0","ondisk_log_start":"0'0","created":203,"last_epoch_clean":204,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T15:08:45.049361+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T15:08:45.049361+0000","last_clean_scrub_stamp":"2023-08-09T15:08:45.049361+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-11T01:16:19.078625+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"24.13","version":"204'2","reported_seq":13,"reported_epoch":204,"state":"active+clean","last_fresh":"2023-08-09T15:08:46.156865+0000","last_change":"2023-08-09T15:08:45.093138+0000","last_active":"2023-08-09T15:08:46.156865+0000","last_peered":"2023-08-09T15:08:46.156865+0000","last_clean":"2023-08-09T15:08:46.156865+0000","last_became_active":"2023-08-09T15:08:45.092792+0000","last_became_peered":"2023-08-09T15:08:45.092792+0000","last_unstale":"2023-08-09T15:08:46.156865+0000","last_undegraded":"2023-08-09T15:08:46.156865+0000","last_fullsized":"2023-08-09T15:08:46.156865+0000","mapping_epoch":202,"log_start":"0'0","ondisk_log_start":"0'0","created":202,"last_epoch_clean":203,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T15:08:44.039630+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T15:08:44.039630+0000","last_clean_scrub_stamp":"2023-08-09T15:08:44.039630+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T15:11:33.035314+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"25.13","version":"0'0","reported_seq":9,"reported_epoch":204,"state":"active+clean","last_fresh":"2023-08-09T15:08:46.132112+0000","last_change":"2023-08-09T15:08:46.125769+0000","last_active":"2023-08-09T15:08:46.132112+0000","last_peered":"2023-08-09T15:08:46.132112+0000","last_clean":"2023-08-09T15:08:46.132112+0000","last_became_active":"2023-08-09T15:08:46.125483+0000","last_became_peered":"2023-08-09T15:08:46.125483+0000","last_unstale":"2023-08-09T15:08:46.132112+0000","last_undegraded":"2023-08-09T15:08:46.132112+0000","last_fullsized":"2023-08-09T15:08:46.132112+0000","mapping_epoch":203,"log_start":"0'0","ondisk_log_start":"0'0","created":203,"last_epoch_clean":204,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T15:08:45.049361+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T15:08:45.049361+0000","last_clean_scrub_stamp":"2023-08-09T15:08:45.049361+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-11T01:40:22.044849+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"24.12","version":"0'0","reported_seq":11,"reported_epoch":204,"state":"active+clean","last_fresh":"2023-08-09T15:08:46.120634+0000","last_change":"2023-08-09T15:08:45.095964+0000","last_active":"2023-08-09T15:08:46.120634+0000","last_peered":"2023-08-09T15:08:46.120634+0000","last_clean":"2023-08-09T15:08:46.120634+0000","last_became_active":"2023-08-09T15:08:45.095675+0000","last_became_peered":"2023-08-09T15:08:45.095675+0000","last_unstale":"2023-08-09T15:08:46.120634+0000","last_undegraded":"2023-08-09T15:08:46.120634+0000","last_fullsized":"2023-08-09T15:08:46.120634+0000","mapping_epoch":202,"log_start":"0'0","ondisk_log_start":"0'0","created":202,"last_epoch_clean":203,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T15:08:44.039630+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T15:08:44.039630+0000","last_clean_scrub_stamp":"2023-08-09T15:08:44.039630+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-11T03:01:48.069708+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"25.10","version":"0'0","reported_seq":9,"reported_epoch":204,"state":"active+clean","last_fresh":"2023-08-09T15:08:46.130212+0000","last_change":"2023-08-09T15:08:46.129212+0000","last_active":"2023-08-09T15:08:46.130212+0000","last_peered":"2023-08-09T15:08:46.130212+0000","last_clean":"2023-08-09T15:08:46.130212+0000","last_became_active":"2023-08-09T15:08:46.128930+0000","last_became_peered":"2023-08-09T15:08:46.128930+0000","last_unstale":"2023-08-09T15:08:46.130212+0000","last_undegraded":"2023-08-09T15:08:46.130212+0000","last_fullsized":"2023-08-09T15:08:46.130212+0000","mapping_epoch":203,"log_start":"0'0","ondisk_log_start":"0'0","created":203,"last_epoch_clean":204,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T15:08:45.049361+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T15:08:45.049361+0000","last_clean_scrub_stamp":"2023-08-09T15:08:45.049361+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T16:30:28.142764+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"24.11","version":"0'0","reported_seq":11,"reported_epoch":204,"state":"active+clean","last_fresh":"2023-08-09T15:08:46.105202+0000","last_change":"2023-08-09T15:08:45.091703+0000","last_active":"2023-08-09T15:08:46.105202+0000","last_peered":"2023-08-09T15:08:46.105202+0000","last_clean":"2023-08-09T15:08:46.105202+0000","last_became_active":"2023-08-09T15:08:45.091437+0000","last_became_peered":"2023-08-09T15:08:45.091437+0000","last_unstale":"2023-08-09T15:08:46.105202+0000","last_undegraded":"2023-08-09T15:08:46.105202+0000","last_fullsized":"2023-08-09T15:08:46.105202+0000","mapping_epoch":202,"log_start":"0'0","ondisk_log_start":"0'0","created":202,"last_epoch_clean":203,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T15:08:44.039630+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T15:08:44.039630+0000","last_clean_scrub_stamp":"2023-08-09T15:08:44.039630+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-11T00:38:14.601055+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"25.11","version":"0'0","reported_seq":9,"reported_epoch":204,"state":"active+clean","last_fresh":"2023-08-09T15:08:46.147109+0000","last_change":"2023-08-09T15:08:46.135169+0000","last_active":"2023-08-09T15:08:46.147109+0000","last_peered":"2023-08-09T15:08:46.147109+0000","last_clean":"2023-08-09T15:08:46.147109+0000","last_became_active":"2023-08-09T15:08:46.134688+0000","last_became_peered":"2023-08-09T15:08:46.134688+0000","last_unstale":"2023-08-09T15:08:46.147109+0000","last_undegraded":"2023-08-09T15:08:46.147109+0000","last_fullsized":"2023-08-09T15:08:46.147109+0000","mapping_epoch":203,"log_start":"0'0","ondisk_log_start":"0'0","created":203,"last_epoch_clean":204,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T15:08:45.049361+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T15:08:45.049361+0000","last_clean_scrub_stamp":"2023-08-09T15:08:45.049361+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-11T00:19:57.173009+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"24.10","version":"204'8","reported_seq":19,"reported_epoch":204,"state":"active+clean","last_fresh":"2023-08-09T15:08:46.156956+0000","last_change":"2023-08-09T15:08:45.086979+0000","last_active":"2023-08-09T15:08:46.156956+0000","last_peered":"2023-08-09T15:08:46.156956+0000","last_clean":"2023-08-09T15:08:46.156956+0000","last_became_active":"2023-08-09T15:08:45.086321+0000","last_became_peered":"2023-08-09T15:08:45.086321+0000","last_unstale":"2023-08-09T15:08:46.156956+0000","last_undegraded":"2023-08-09T15:08:46.156956+0000","last_fullsized":"2023-08-09T15:08:46.156956+0000","mapping_epoch":202,"log_start":"0'0","ondisk_log_start":"0'0","created":202,"last_epoch_clean":203,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T15:08:44.039630+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T15:08:44.039630+0000","last_clean_scrub_stamp":"2023-08-09T15:08:44.039630+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-11T01:50:54.696216+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"25.e","version":"0'0","reported_seq":9,"reported_epoch":204,"state":"active+clean","last_fresh":"2023-08-09T15:08:46.146607+0000","last_change":"2023-08-09T15:08:46.134264+0000","last_active":"2023-08-09T15:08:46.146607+0000","last_peered":"2023-08-09T15:08:46.146607+0000","last_clean":"2023-08-09T15:08:46.146607+0000","last_became_active":"2023-08-09T15:08:46.133953+0000","last_became_peered":"2023-08-09T15:08:46.133953+0000","last_unstale":"2023-08-09T15:08:46.146607+0000","last_undegraded":"2023-08-09T15:08:46.146607+0000","last_fullsized":"2023-08-09T15:08:46.146607+0000","mapping_epoch":203,"log_start":"0'0","ondisk_log_start":"0'0","created":203,"last_epoch_clean":204,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T15:08:45.049361+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T15:08:45.049361+0000","last_clean_scrub_stamp":"2023-08-09T15:08:45.049361+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-11T02:04:24.281163+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"24.f","version":"204'1","reported_seq":12,"reported_epoch":204,"state":"active+clean","last_fresh":"2023-08-09T15:08:46.152505+0000","last_change":"2023-08-09T15:08:45.098360+0000","last_active":"2023-08-09T15:08:46.152505+0000","last_peered":"2023-08-09T15:08:46.152505+0000","last_clean":"2023-08-09T15:08:46.152505+0000","last_became_active":"2023-08-09T15:08:45.098110+0000","last_became_peered":"2023-08-09T15:08:45.098110+0000","last_unstale":"2023-08-09T15:08:46.152505+0000","last_undegraded":"2023-08-09T15:08:46.152505+0000","last_fullsized":"2023-08-09T15:08:46.152505+0000","mapping_epoch":202,"log_start":"0'0","ondisk_log_start":"0'0","created":202,"last_epoch_clean":203,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T15:08:44.039630+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T15:08:44.039630+0000","last_clean_scrub_stamp":"2023-08-09T15:08:44.039630+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-11T02:46:55.641275+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"25.f","version":"0'0","reported_seq":9,"reported_epoch":204,"state":"active+clean","last_fresh":"2023-08-09T15:08:46.141110+0000","last_change":"2023-08-09T15:08:46.137534+0000","last_active":"2023-08-09T15:08:46.141110+0000","last_peered":"2023-08-09T15:08:46.141110+0000","last_clean":"2023-08-09T15:08:46.141110+0000","last_became_active":"2023-08-09T15:08:46.137099+0000","last_became_peered":"2023-08-09T15:08:46.137099+0000","last_unstale":"2023-08-09T15:08:46.141110+0000","last_undegraded":"2023-08-09T15:08:46.141110+0000","last_fullsized":"2023-08-09T15:08:46.141110+0000","mapping_epoch":203,"log_start":"0'0","ondisk_log_start":"0'0","created":203,"last_epoch_clean":204,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T15:08:45.049361+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T15:08:45.049361+0000","last_clean_scrub_stamp":"2023-08-09T15:08:45.049361+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T16:49:16.683294+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"24.e","version":"204'2","reported_seq":13,"reported_epoch":204,"state":"active+clean","last_fresh":"2023-08-09T15:08:46.156808+0000","last_change":"2023-08-09T15:08:45.089498+0000","last_active":"2023-08-09T15:08:46.156808+0000","last_peered":"2023-08-09T15:08:46.156808+0000","last_clean":"2023-08-09T15:08:46.156808+0000","last_became_active":"2023-08-09T15:08:45.089176+0000","last_became_peered":"2023-08-09T15:08:45.089176+0000","last_unstale":"2023-08-09T15:08:46.156808+0000","last_undegraded":"2023-08-09T15:08:46.156808+0000","last_fullsized":"2023-08-09T15:08:46.156808+0000","mapping_epoch":202,"log_start":"0'0","ondisk_log_start":"0'0","created":202,"last_epoch_clean":203,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T15:08:44.039630+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T15:08:44.039630+0000","last_clean_scrub_stamp":"2023-08-09T15:08:44.039630+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T22:07:02.953997+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"25.c","version":"0'0","reported_seq":9,"reported_epoch":204,"state":"active+clean","last_fresh":"2023-08-09T15:08:46.131124+0000","last_change":"2023-08-09T15:08:46.128985+0000","last_active":"2023-08-09T15:08:46.131124+0000","last_peered":"2023-08-09T15:08:46.131124+0000","last_clean":"2023-08-09T15:08:46.131124+0000","last_became_active":"2023-08-09T15:08:46.128680+0000","last_became_peered":"2023-08-09T15:08:46.128680+0000","last_unstale":"2023-08-09T15:08:46.131124+0000","last_undegraded":"2023-08-09T15:08:46.131124+0000","last_fullsized":"2023-08-09T15:08:46.131124+0000","mapping_epoch":203,"log_start":"0'0","ondisk_log_start":"0'0","created":203,"last_epoch_clean":204,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T15:08:45.049361+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T15:08:45.049361+0000","last_clean_scrub_stamp":"2023-08-09T15:08:45.049361+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-11T01:49:59.788507+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"24.d","version":"204'1","reported_seq":12,"reported_epoch":204,"state":"active+clean","last_fresh":"2023-08-09T15:08:46.154048+0000","last_change":"2023-08-09T15:08:45.088107+0000","last_active":"2023-08-09T15:08:46.154048+0000","last_peered":"2023-08-09T15:08:46.154048+0000","last_clean":"2023-08-09T15:08:46.154048+0000","last_became_active":"2023-08-09T15:08:45.087782+0000","last_became_peered":"2023-08-09T15:08:45.087782+0000","last_unstale":"2023-08-09T15:08:46.154048+0000","last_undegraded":"2023-08-09T15:08:46.154048+0000","last_fullsized":"2023-08-09T15:08:46.154048+0000","mapping_epoch":202,"log_start":"0'0","ondisk_log_start":"0'0","created":202,"last_epoch_clean":203,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T15:08:44.039630+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T15:08:44.039630+0000","last_clean_scrub_stamp":"2023-08-09T15:08:44.039630+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T15:39:55.302767+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"25.d","version":"0'0","reported_seq":9,"reported_epoch":204,"state":"active+clean","last_fresh":"2023-08-09T15:08:46.139803+0000","last_change":"2023-08-09T15:08:46.138571+0000","last_active":"2023-08-09T15:08:46.139803+0000","last_peered":"2023-08-09T15:08:46.139803+0000","last_clean":"2023-08-09T15:08:46.139803+0000","last_became_active":"2023-08-09T15:08:46.138271+0000","last_became_peered":"2023-08-09T15:08:46.138271+0000","last_unstale":"2023-08-09T15:08:46.139803+0000","last_undegraded":"2023-08-09T15:08:46.139803+0000","last_fullsized":"2023-08-09T15:08:46.139803+0000","mapping_epoch":203,"log_start":"0'0","ondisk_log_start":"0'0","created":203,"last_epoch_clean":204,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T15:08:45.049361+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T15:08:45.049361+0000","last_clean_scrub_stamp":"2023-08-09T15:08:45.049361+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T16:24:39.791163+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"24.c","version":"0'0","reported_seq":11,"reported_epoch":204,"state":"active+clean","last_fresh":"2023-08-09T15:08:46.106493+0000","last_change":"2023-08-09T15:08:45.089852+0000","last_active":"2023-08-09T15:08:46.106493+0000","last_peered":"2023-08-09T15:08:46.106493+0000","last_clean":"2023-08-09T15:08:46.106493+0000","last_became_active":"2023-08-09T15:08:45.088837+0000","last_became_peered":"2023-08-09T15:08:45.088837+0000","last_unstale":"2023-08-09T15:08:46.106493+0000","last_undegraded":"2023-08-09T15:08:46.106493+0000","last_fullsized":"2023-08-09T15:08:46.106493+0000","mapping_epoch":202,"log_start":"0'0","ondisk_log_start":"0'0","created":202,"last_epoch_clean":203,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T15:08:44.039630+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T15:08:44.039630+0000","last_clean_scrub_stamp":"2023-08-09T15:08:44.039630+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T22:11:33.824164+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"25.a","version":"0'0","reported_seq":9,"reported_epoch":204,"state":"active+clean","last_fresh":"2023-08-09T15:08:46.149947+0000","last_change":"2023-08-09T15:08:46.143670+0000","last_active":"2023-08-09T15:08:46.149947+0000","last_peered":"2023-08-09T15:08:46.149947+0000","last_clean":"2023-08-09T15:08:46.149947+0000","last_became_active":"2023-08-09T15:08:46.143371+0000","last_became_peered":"2023-08-09T15:08:46.143371+0000","last_unstale":"2023-08-09T15:08:46.149947+0000","last_undegraded":"2023-08-09T15:08:46.149947+0000","last_fullsized":"2023-08-09T15:08:46.149947+0000","mapping_epoch":203,"log_start":"0'0","ondisk_log_start":"0'0","created":203,"last_epoch_clean":204,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T15:08:45.049361+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T15:08:45.049361+0000","last_clean_scrub_stamp":"2023-08-09T15:08:45.049361+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T23:25:10.934767+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"24.b","version":"204'2","reported_seq":13,"reported_epoch":204,"state":"active+clean","last_fresh":"2023-08-09T15:08:46.153823+0000","last_change":"2023-08-09T15:08:45.099141+0000","last_active":"2023-08-09T15:08:46.153823+0000","last_peered":"2023-08-09T15:08:46.153823+0000","last_clean":"2023-08-09T15:08:46.153823+0000","last_became_active":"2023-08-09T15:08:45.098823+0000","last_became_peered":"2023-08-09T15:08:45.098823+0000","last_unstale":"2023-08-09T15:08:46.153823+0000","last_undegraded":"2023-08-09T15:08:46.153823+0000","last_fullsized":"2023-08-09T15:08:46.153823+0000","mapping_epoch":202,"log_start":"0'0","ondisk_log_start":"0'0","created":202,"last_epoch_clean":203,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T15:08:44.039630+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T15:08:44.039630+0000","last_clean_scrub_stamp":"2023-08-09T15:08:44.039630+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T23:52:19.241728+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"25.b","version":"0'0","reported_seq":9,"reported_epoch":204,"state":"active+clean","last_fresh":"2023-08-09T15:08:46.132090+0000","last_change":"2023-08-09T15:08:46.124313+0000","last_active":"2023-08-09T15:08:46.132090+0000","last_peered":"2023-08-09T15:08:46.132090+0000","last_clean":"2023-08-09T15:08:46.132090+0000","last_became_active":"2023-08-09T15:08:46.124031+0000","last_became_peered":"2023-08-09T15:08:46.124031+0000","last_unstale":"2023-08-09T15:08:46.132090+0000","last_undegraded":"2023-08-09T15:08:46.132090+0000","last_fullsized":"2023-08-09T15:08:46.132090+0000","mapping_epoch":203,"log_start":"0'0","ondisk_log_start":"0'0","created":203,"last_epoch_clean":204,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T15:08:45.049361+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T15:08:45.049361+0000","last_clean_scrub_stamp":"2023-08-09T15:08:45.049361+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-11T02:17:00.563221+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"24.a","version":"0'0","reported_seq":11,"reported_epoch":204,"state":"active+clean","last_fresh":"2023-08-09T15:08:46.118441+0000","last_change":"2023-08-09T15:08:45.093370+0000","last_active":"2023-08-09T15:08:46.118441+0000","last_peered":"2023-08-09T15:08:46.118441+0000","last_clean":"2023-08-09T15:08:46.118441+0000","last_became_active":"2023-08-09T15:08:45.092965+0000","last_became_peered":"2023-08-09T15:08:45.092965+0000","last_unstale":"2023-08-09T15:08:46.118441+0000","last_undegraded":"2023-08-09T15:08:46.118441+0000","last_fullsized":"2023-08-09T15:08:46.118441+0000","mapping_epoch":202,"log_start":"0'0","ondisk_log_start":"0'0","created":202,"last_epoch_clean":203,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T15:08:44.039630+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T15:08:44.039630+0000","last_clean_scrub_stamp":"2023-08-09T15:08:44.039630+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T22:05:12.350996+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"25.8","version":"0'0","reported_seq":9,"reported_epoch":204,"state":"active+clean","last_fresh":"2023-08-09T15:08:46.134447+0000","last_change":"2023-08-09T15:08:46.127835+0000","last_active":"2023-08-09T15:08:46.134447+0000","last_peered":"2023-08-09T15:08:46.134447+0000","last_clean":"2023-08-09T15:08:46.134447+0000","last_became_active":"2023-08-09T15:08:46.127529+0000","last_became_peered":"2023-08-09T15:08:46.127529+0000","last_unstale":"2023-08-09T15:08:46.134447+0000","last_undegraded":"2023-08-09T15:08:46.134447+0000","last_fullsized":"2023-08-09T15:08:46.134447+0000","mapping_epoch":203,"log_start":"0'0","ondisk_log_start":"0'0","created":203,"last_epoch_clean":204,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T15:08:45.049361+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T15:08:45.049361+0000","last_clean_scrub_stamp":"2023-08-09T15:08:45.049361+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T18:45:41.505319+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"24.9","version":"0'0","reported_seq":11,"reported_epoch":204,"state":"active+clean","last_fresh":"2023-08-09T15:08:46.102736+0000","last_change":"2023-08-09T15:08:45.097893+0000","last_active":"2023-08-09T15:08:46.102736+0000","last_peered":"2023-08-09T15:08:46.102736+0000","last_clean":"2023-08-09T15:08:46.102736+0000","last_became_active":"2023-08-09T15:08:45.097618+0000","last_became_peered":"2023-08-09T15:08:45.097618+0000","last_unstale":"2023-08-09T15:08:46.102736+0000","last_undegraded":"2023-08-09T15:08:46.102736+0000","last_fullsized":"2023-08-09T15:08:46.102736+0000","mapping_epoch":202,"log_start":"0'0","ondisk_log_start":"0'0","created":202,"last_epoch_clean":203,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T15:08:44.039630+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T15:08:44.039630+0000","last_clean_scrub_stamp":"2023-08-09T15:08:44.039630+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T15:17:53.837111+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"25.9","version":"0'0","reported_seq":9,"reported_epoch":204,"state":"active+clean","last_fresh":"2023-08-09T15:08:46.140972+0000","last_change":"2023-08-09T15:08:46.133100+0000","last_active":"2023-08-09T15:08:46.140972+0000","last_peered":"2023-08-09T15:08:46.140972+0000","last_clean":"2023-08-09T15:08:46.140972+0000","last_became_active":"2023-08-09T15:08:46.132718+0000","last_became_peered":"2023-08-09T15:08:46.132718+0000","last_unstale":"2023-08-09T15:08:46.140972+0000","last_undegraded":"2023-08-09T15:08:46.140972+0000","last_fullsized":"2023-08-09T15:08:46.140972+0000","mapping_epoch":203,"log_start":"0'0","ondisk_log_start":"0'0","created":203,"last_epoch_clean":204,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T15:08:45.049361+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T15:08:45.049361+0000","last_clean_scrub_stamp":"2023-08-09T15:08:45.049361+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T16:51:47.226654+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"24.8","version":"0'0","reported_seq":11,"reported_epoch":204,"state":"active+clean","last_fresh":"2023-08-09T15:08:46.116342+0000","last_change":"2023-08-09T15:08:45.089228+0000","last_active":"2023-08-09T15:08:46.116342+0000","last_peered":"2023-08-09T15:08:46.116342+0000","last_clean":"2023-08-09T15:08:46.116342+0000","last_became_active":"2023-08-09T15:08:45.088776+0000","last_became_peered":"2023-08-09T15:08:45.088776+0000","last_unstale":"2023-08-09T15:08:46.116342+0000","last_undegraded":"2023-08-09T15:08:46.116342+0000","last_fullsized":"2023-08-09T15:08:46.116342+0000","mapping_epoch":202,"log_start":"0'0","ondisk_log_start":"0'0","created":202,"last_epoch_clean":203,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T15:08:44.039630+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T15:08:44.039630+0000","last_clean_scrub_stamp":"2023-08-09T15:08:44.039630+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T19:52:42.363999+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"25.6","version":"0'0","reported_seq":9,"reported_epoch":204,"state":"active+clean","last_fresh":"2023-08-09T15:08:46.142930+0000","last_change":"2023-08-09T15:08:46.133149+0000","last_active":"2023-08-09T15:08:46.142930+0000","last_peered":"2023-08-09T15:08:46.142930+0000","last_clean":"2023-08-09T15:08:46.142930+0000","last_became_active":"2023-08-09T15:08:46.132736+0000","last_became_peered":"2023-08-09T15:08:46.132736+0000","last_unstale":"2023-08-09T15:08:46.142930+0000","last_undegraded":"2023-08-09T15:08:46.142930+0000","last_fullsized":"2023-08-09T15:08:46.142930+0000","mapping_epoch":203,"log_start":"0'0","ondisk_log_start":"0'0","created":203,"last_epoch_clean":204,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T15:08:45.049361+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T15:08:45.049361+0000","last_clean_scrub_stamp":"2023-08-09T15:08:45.049361+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T15:33:36.716547+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"24.7","version":"204'2","reported_seq":13,"reported_epoch":204,"state":"active+clean","last_fresh":"2023-08-09T15:08:46.154036+0000","last_change":"2023-08-09T15:08:45.095073+0000","last_active":"2023-08-09T15:08:46.154036+0000","last_peered":"2023-08-09T15:08:46.154036+0000","last_clean":"2023-08-09T15:08:46.154036+0000","last_became_active":"2023-08-09T15:08:45.094734+0000","last_became_peered":"2023-08-09T15:08:45.094734+0000","last_unstale":"2023-08-09T15:08:46.154036+0000","last_undegraded":"2023-08-09T15:08:46.154036+0000","last_fullsized":"2023-08-09T15:08:46.154036+0000","mapping_epoch":202,"log_start":"0'0","ondisk_log_start":"0'0","created":202,"last_epoch_clean":203,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T15:08:44.039630+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T15:08:44.039630+0000","last_clean_scrub_stamp":"2023-08-09T15:08:44.039630+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-11T02:53:18.098726+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"25.7","version":"0'0","reported_seq":9,"reported_epoch":204,"state":"active+clean","last_fresh":"2023-08-09T15:08:46.135768+0000","last_change":"2023-08-09T15:08:46.130455+0000","last_active":"2023-08-09T15:08:46.135768+0000","last_peered":"2023-08-09T15:08:46.135768+0000","last_clean":"2023-08-09T15:08:46.135768+0000","last_became_active":"2023-08-09T15:08:46.130168+0000","last_became_peered":"2023-08-09T15:08:46.130168+0000","last_unstale":"2023-08-09T15:08:46.135768+0000","last_undegraded":"2023-08-09T15:08:46.135768+0000","last_fullsized":"2023-08-09T15:08:46.135768+0000","mapping_epoch":203,"log_start":"0'0","ondisk_log_start":"0'0","created":203,"last_epoch_clean":204,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T15:08:45.049361+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T15:08:45.049361+0000","last_clean_scrub_stamp":"2023-08-09T15:08:45.049361+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T15:58:04.851493+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"24.6","version":"0'0","reported_seq":11,"reported_epoch":204,"state":"active+clean","last_fresh":"2023-08-09T15:08:46.122642+0000","last_change":"2023-08-09T15:08:45.089606+0000","last_active":"2023-08-09T15:08:46.122642+0000","last_peered":"2023-08-09T15:08:46.122642+0000","last_clean":"2023-08-09T15:08:46.122642+0000","last_became_active":"2023-08-09T15:08:45.089344+0000","last_became_peered":"2023-08-09T15:08:45.089344+0000","last_unstale":"2023-08-09T15:08:46.122642+0000","last_undegraded":"2023-08-09T15:08:46.122642+0000","last_fullsized":"2023-08-09T15:08:46.122642+0000","mapping_epoch":202,"log_start":"0'0","ondisk_log_start":"0'0","created":202,"last_epoch_clean":203,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T15:08:44.039630+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T15:08:44.039630+0000","last_clean_scrub_stamp":"2023-08-09T15:08:44.039630+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T23:53:45.920055+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"25.4","version":"0'0","reported_seq":9,"reported_epoch":204,"state":"active+clean","last_fresh":"2023-08-09T15:08:46.136029+0000","last_change":"2023-08-09T15:08:46.127889+0000","last_active":"2023-08-09T15:08:46.136029+0000","last_peered":"2023-08-09T15:08:46.136029+0000","last_clean":"2023-08-09T15:08:46.136029+0000","last_became_active":"2023-08-09T15:08:46.127609+0000","last_became_peered":"2023-08-09T15:08:46.127609+0000","last_unstale":"2023-08-09T15:08:46.136029+0000","last_undegraded":"2023-08-09T15:08:46.136029+0000","last_fullsized":"2023-08-09T15:08:46.136029+0000","mapping_epoch":203,"log_start":"0'0","ondisk_log_start":"0'0","created":203,"last_epoch_clean":204,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T15:08:45.049361+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T15:08:45.049361+0000","last_clean_scrub_stamp":"2023-08-09T15:08:45.049361+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T22:16:43.576858+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"24.5","version":"0'0","reported_seq":11,"reported_epoch":204,"state":"active+clean","last_fresh":"2023-08-09T15:08:46.120333+0000","last_change":"2023-08-09T15:08:45.089605+0000","last_active":"2023-08-09T15:08:46.120333+0000","last_peered":"2023-08-09T15:08:46.120333+0000","last_clean":"2023-08-09T15:08:46.120333+0000","last_became_active":"2023-08-09T15:08:45.089267+0000","last_became_peered":"2023-08-09T15:08:45.089267+0000","last_unstale":"2023-08-09T15:08:46.120333+0000","last_undegraded":"2023-08-09T15:08:46.120333+0000","last_fullsized":"2023-08-09T15:08:46.120333+0000","mapping_epoch":202,"log_start":"0'0","ondisk_log_start":"0'0","created":202,"last_epoch_clean":203,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T15:08:44.039630+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T15:08:44.039630+0000","last_clean_scrub_stamp":"2023-08-09T15:08:44.039630+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T15:14:23.464887+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"25.5","version":"0'0","reported_seq":9,"reported_epoch":204,"state":"active+clean","last_fresh":"2023-08-09T15:08:46.144821+0000","last_change":"2023-08-09T15:08:46.140247+0000","last_active":"2023-08-09T15:08:46.144821+0000","last_peered":"2023-08-09T15:08:46.144821+0000","last_clean":"2023-08-09T15:08:46.144821+0000","last_became_active":"2023-08-09T15:08:46.139940+0000","last_became_peered":"2023-08-09T15:08:46.139940+0000","last_unstale":"2023-08-09T15:08:46.144821+0000","last_undegraded":"2023-08-09T15:08:46.144821+0000","last_fullsized":"2023-08-09T15:08:46.144821+0000","mapping_epoch":203,"log_start":"0'0","ondisk_log_start":"0'0","created":203,"last_epoch_clean":204,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T15:08:45.049361+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T15:08:45.049361+0000","last_clean_scrub_stamp":"2023-08-09T15:08:45.049361+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T20:28:03.166356+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"24.4","version":"0'0","reported_seq":11,"reported_epoch":204,"state":"active+clean","last_fresh":"2023-08-09T15:08:46.106030+0000","last_change":"2023-08-09T15:08:45.092363+0000","last_active":"2023-08-09T15:08:46.106030+0000","last_peered":"2023-08-09T15:08:46.106030+0000","last_clean":"2023-08-09T15:08:46.106030+0000","last_became_active":"2023-08-09T15:08:45.092079+0000","last_became_peered":"2023-08-09T15:08:45.092079+0000","last_unstale":"2023-08-09T15:08:46.106030+0000","last_undegraded":"2023-08-09T15:08:46.106030+0000","last_fullsized":"2023-08-09T15:08:46.106030+0000","mapping_epoch":202,"log_start":"0'0","ondisk_log_start":"0'0","created":202,"last_epoch_clean":203,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T15:08:44.039630+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T15:08:44.039630+0000","last_clean_scrub_stamp":"2023-08-09T15:08:44.039630+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T19:07:11.609326+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"25.2","version":"0'0","reported_seq":9,"reported_epoch":204,"state":"active+clean","last_fresh":"2023-08-09T15:08:46.141342+0000","last_change":"2023-08-09T15:08:46.139532+0000","last_active":"2023-08-09T15:08:46.141342+0000","last_peered":"2023-08-09T15:08:46.141342+0000","last_clean":"2023-08-09T15:08:46.141342+0000","last_became_active":"2023-08-09T15:08:46.139060+0000","last_became_peered":"2023-08-09T15:08:46.139060+0000","last_unstale":"2023-08-09T15:08:46.141342+0000","last_undegraded":"2023-08-09T15:08:46.141342+0000","last_fullsized":"2023-08-09T15:08:46.141342+0000","mapping_epoch":203,"log_start":"0'0","ondisk_log_start":"0'0","created":203,"last_epoch_clean":204,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T15:08:45.049361+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T15:08:45.049361+0000","last_clean_scrub_stamp":"2023-08-09T15:08:45.049361+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-11T00:56:55.799238+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"24.3","version":"0'0","reported_seq":11,"reported_epoch":204,"state":"active+clean","last_fresh":"2023-08-09T15:08:46.122226+0000","last_change":"2023-08-09T15:08:45.086089+0000","last_active":"2023-08-09T15:08:46.122226+0000","last_peered":"2023-08-09T15:08:46.122226+0000","last_clean":"2023-08-09T15:08:46.122226+0000","last_became_active":"2023-08-09T15:08:45.085719+0000","last_became_peered":"2023-08-09T15:08:45.085719+0000","last_unstale":"2023-08-09T15:08:46.122226+0000","last_undegraded":"2023-08-09T15:08:46.122226+0000","last_fullsized":"2023-08-09T15:08:46.122226+0000","mapping_epoch":202,"log_start":"0'0","ondisk_log_start":"0'0","created":202,"last_epoch_clean":203,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T15:08:44.039630+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T15:08:44.039630+0000","last_clean_scrub_stamp":"2023-08-09T15:08:44.039630+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T23:31:38.750789+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"25.3","version":"0'0","reported_seq":9,"reported_epoch":204,"state":"active+clean","last_fresh":"2023-08-09T15:08:46.135798+0000","last_change":"2023-08-09T15:08:46.133084+0000","last_active":"2023-08-09T15:08:46.135798+0000","last_peered":"2023-08-09T15:08:46.135798+0000","last_clean":"2023-08-09T15:08:46.135798+0000","last_became_active":"2023-08-09T15:08:46.132801+0000","last_became_peered":"2023-08-09T15:08:46.132801+0000","last_unstale":"2023-08-09T15:08:46.135798+0000","last_undegraded":"2023-08-09T15:08:46.135798+0000","last_fullsized":"2023-08-09T15:08:46.135798+0000","mapping_epoch":203,"log_start":"0'0","ondisk_log_start":"0'0","created":203,"last_epoch_clean":204,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T15:08:45.049361+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T15:08:45.049361+0000","last_clean_scrub_stamp":"2023-08-09T15:08:45.049361+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T22:36:09.290713+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"24.2","version":"0'0","reported_seq":11,"reported_epoch":204,"state":"active+clean","last_fresh":"2023-08-09T15:08:46.116671+0000","last_change":"2023-08-09T15:08:45.086746+0000","last_active":"2023-08-09T15:08:46.116671+0000","last_peered":"2023-08-09T15:08:46.116671+0000","last_clean":"2023-08-09T15:08:46.116671+0000","last_became_active":"2023-08-09T15:08:45.086092+0000","last_became_peered":"2023-08-09T15:08:45.086092+0000","last_unstale":"2023-08-09T15:08:46.116671+0000","last_undegraded":"2023-08-09T15:08:46.116671+0000","last_fullsized":"2023-08-09T15:08:46.116671+0000","mapping_epoch":202,"log_start":"0'0","ondisk_log_start":"0'0","created":202,"last_epoch_clean":203,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T15:08:44.039630+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T15:08:44.039630+0000","last_clean_scrub_stamp":"2023-08-09T15:08:44.039630+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-11T01:15:55.832665+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"25.0","version":"0'0","reported_seq":9,"reported_epoch":204,"state":"active+clean","last_fresh":"2023-08-09T15:08:46.134427+0000","last_change":"2023-08-09T15:08:46.127664+0000","last_active":"2023-08-09T15:08:46.134427+0000","last_peered":"2023-08-09T15:08:46.134427+0000","last_clean":"2023-08-09T15:08:46.134427+0000","last_became_active":"2023-08-09T15:08:46.127206+0000","last_became_peered":"2023-08-09T15:08:46.127206+0000","last_unstale":"2023-08-09T15:08:46.134427+0000","last_undegraded":"2023-08-09T15:08:46.134427+0000","last_fullsized":"2023-08-09T15:08:46.134427+0000","mapping_epoch":203,"log_start":"0'0","ondisk_log_start":"0'0","created":203,"last_epoch_clean":204,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T15:08:45.049361+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T15:08:45.049361+0000","last_clean_scrub_stamp":"2023-08-09T15:08:45.049361+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-11T01:04:02.365522+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"24.1","version":"0'0","reported_seq":11,"reported_epoch":204,"state":"active+clean","last_fresh":"2023-08-09T15:08:46.116754+0000","last_change":"2023-08-09T15:08:45.086500+0000","last_active":"2023-08-09T15:08:46.116754+0000","last_peered":"2023-08-09T15:08:46.116754+0000","last_clean":"2023-08-09T15:08:46.116754+0000","last_became_active":"2023-08-09T15:08:45.086062+0000","last_became_peered":"2023-08-09T15:08:45.086062+0000","last_unstale":"2023-08-09T15:08:46.116754+0000","last_undegraded":"2023-08-09T15:08:46.116754+0000","last_fullsized":"2023-08-09T15:08:46.116754+0000","mapping_epoch":202,"log_start":"0'0","ondisk_log_start":"0'0","created":202,"last_epoch_clean":203,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T15:08:44.039630+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T15:08:44.039630+0000","last_clean_scrub_stamp":"2023-08-09T15:08:44.039630+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T19:26:21.854345+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"25.1","version":"0'0","reported_seq":9,"reported_epoch":204,"state":"active+clean","last_fresh":"2023-08-09T15:08:46.145278+0000","last_change":"2023-08-09T15:08:46.133327+0000","last_active":"2023-08-09T15:08:46.145278+0000","last_peered":"2023-08-09T15:08:46.145278+0000","last_clean":"2023-08-09T15:08:46.145278+0000","last_became_active":"2023-08-09T15:08:46.127486+0000","last_became_peered":"2023-08-09T15:08:46.127486+0000","last_unstale":"2023-08-09T15:08:46.145278+0000","last_undegraded":"2023-08-09T15:08:46.145278+0000","last_fullsized":"2023-08-09T15:08:46.145278+0000","mapping_epoch":203,"log_start":"0'0","ondisk_log_start":"0'0","created":203,"last_epoch_clean":204,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T15:08:45.049361+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T15:08:45.049361+0000","last_clean_scrub_stamp":"2023-08-09T15:08:45.049361+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T19:43:08.286766+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"24.0","version":"0'0","reported_seq":11,"reported_epoch":204,"state":"active+clean","last_fresh":"2023-08-09T15:08:46.101554+0000","last_change":"2023-08-09T15:08:45.096555+0000","last_active":"2023-08-09T15:08:46.101554+0000","last_peered":"2023-08-09T15:08:46.101554+0000","last_clean":"2023-08-09T15:08:46.101554+0000","last_became_active":"2023-08-09T15:08:45.096249+0000","last_became_peered":"2023-08-09T15:08:45.096249+0000","last_unstale":"2023-08-09T15:08:46.101554+0000","last_undegraded":"2023-08-09T15:08:46.101554+0000","last_fullsized":"2023-08-09T15:08:46.101554+0000","mapping_epoch":202,"log_start":"0'0","ondisk_log_start":"0'0","created":202,"last_epoch_clean":203,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T15:08:44.039630+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T15:08:44.039630+0000","last_clean_scrub_stamp":"2023-08-09T15:08:44.039630+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T22:26:19.857459+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"25.18","version":"0'0","reported_seq":9,"reported_epoch":204,"state":"active+clean","last_fresh":"2023-08-09T15:08:46.134973+0000","last_change":"2023-08-09T15:08:46.129858+0000","last_active":"2023-08-09T15:08:46.134973+0000","last_peered":"2023-08-09T15:08:46.134973+0000","last_clean":"2023-08-09T15:08:46.134973+0000","last_became_active":"2023-08-09T15:08:46.129528+0000","last_became_peered":"2023-08-09T15:08:46.129528+0000","last_unstale":"2023-08-09T15:08:46.134973+0000","last_undegraded":"2023-08-09T15:08:46.134973+0000","last_fullsized":"2023-08-09T15:08:46.134973+0000","mapping_epoch":203,"log_start":"0'0","ondisk_log_start":"0'0","created":203,"last_epoch_clean":204,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T15:08:45.049361+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T15:08:45.049361+0000","last_clean_scrub_stamp":"2023-08-09T15:08:45.049361+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T18:26:49.778606+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"24.19","version":"0'0","reported_seq":11,"reported_epoch":204,"state":"active+clean","last_fresh":"2023-08-09T15:08:46.126776+0000","last_change":"2023-08-09T15:08:45.087792+0000","last_active":"2023-08-09T15:08:46.126776+0000","last_peered":"2023-08-09T15:08:46.126776+0000","last_clean":"2023-08-09T15:08:46.126776+0000","last_became_active":"2023-08-09T15:08:45.087519+0000","last_became_peered":"2023-08-09T15:08:45.087519+0000","last_unstale":"2023-08-09T15:08:46.126776+0000","last_undegraded":"2023-08-09T15:08:46.126776+0000","last_fullsized":"2023-08-09T15:08:46.126776+0000","mapping_epoch":202,"log_start":"0'0","ondisk_log_start":"0'0","created":202,"last_epoch_clean":203,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T15:08:44.039630+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T15:08:44.039630+0000","last_clean_scrub_stamp":"2023-08-09T15:08:44.039630+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T23:30:42.203379+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":378,"reported_epoch":204,"state":"active+clean","last_fresh":"2023-08-09T15:08:46.119414+0000","last_change":"2023-08-09T13:33:56.401406+0000","last_active":"2023-08-09T15:08:46.119414+0000","last_peered":"2023-08-09T15:08:46.119414+0000","last_clean":"2023-08-09T15:08:46.119414+0000","last_became_active":"2023-08-09T13:33:56.400487+0000","last_became_peered":"2023-08-09T13:33:56.400487+0000","last_unstale":"2023-08-09T15:08:46.119414+0000","last_undegraded":"2023-08-09T15:08:46.119414+0000","last_fullsized":"2023-08-09T15:08:46.119414+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:33:54.845869+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:33:54.845869+0000","last_clean_scrub_stamp":"2023-08-09T13:33:54.845869+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T23:09:59.457761+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"25.19","version":"0'0","reported_seq":9,"reported_epoch":204,"state":"active+clean","last_fresh":"2023-08-09T15:08:46.132535+0000","last_change":"2023-08-09T15:08:46.130530+0000","last_active":"2023-08-09T15:08:46.132535+0000","last_peered":"2023-08-09T15:08:46.132535+0000","last_clean":"2023-08-09T15:08:46.132535+0000","last_became_active":"2023-08-09T15:08:46.130226+0000","last_became_peered":"2023-08-09T15:08:46.130226+0000","last_unstale":"2023-08-09T15:08:46.132535+0000","last_undegraded":"2023-08-09T15:08:46.132535+0000","last_fullsized":"2023-08-09T15:08:46.132535+0000","mapping_epoch":203,"log_start":"0'0","ondisk_log_start":"0'0","created":203,"last_epoch_clean":204,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T15:08:45.049361+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T15:08:45.049361+0000","last_clean_scrub_stamp":"2023-08-09T15:08:45.049361+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T23:21:48.300259+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"24.18","version":"0'0","reported_seq":11,"reported_epoch":204,"state":"active+clean","last_fresh":"2023-08-09T15:08:46.113596+0000","last_change":"2023-08-09T15:08:45.087159+0000","last_active":"2023-08-09T15:08:46.113596+0000","last_peered":"2023-08-09T15:08:46.113596+0000","last_clean":"2023-08-09T15:08:46.113596+0000","last_became_active":"2023-08-09T15:08:45.086705+0000","last_became_peered":"2023-08-09T15:08:45.086705+0000","last_unstale":"2023-08-09T15:08:46.113596+0000","last_undegraded":"2023-08-09T15:08:46.113596+0000","last_fullsized":"2023-08-09T15:08:46.113596+0000","mapping_epoch":202,"log_start":"0'0","ondisk_log_start":"0'0","created":202,"last_epoch_clean":203,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T15:08:44.039630+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T15:08:44.039630+0000","last_clean_scrub_stamp":"2023-08-09T15:08:44.039630+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T15:37:22.384686+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":388,"reported_epoch":204,"state":"active+clean","last_fresh":"2023-08-09T15:08:46.118306+0000","last_change":"2023-08-09T13:33:56.401123+0000","last_active":"2023-08-09T15:08:46.118306+0000","last_peered":"2023-08-09T15:08:46.118306+0000","last_clean":"2023-08-09T15:08:46.118306+0000","last_became_active":"2023-08-09T13:33:56.399933+0000","last_became_peered":"2023-08-09T13:33:56.399933+0000","last_unstale":"2023-08-09T15:08:46.118306+0000","last_undegraded":"2023-08-09T15:08:46.118306+0000","last_fullsized":"2023-08-09T15:08:46.118306+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:33:54.845869+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:33:54.845869+0000","last_clean_scrub_stamp":"2023-08-09T13:33:54.845869+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-11T01:01:10.799631+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"25.1a","version":"0'0","reported_seq":9,"reported_epoch":204,"state":"active+clean","last_fresh":"2023-08-09T15:08:46.149920+0000","last_change":"2023-08-09T15:08:46.143064+0000","last_active":"2023-08-09T15:08:46.149920+0000","last_peered":"2023-08-09T15:08:46.149920+0000","last_clean":"2023-08-09T15:08:46.149920+0000","last_became_active":"2023-08-09T15:08:46.141541+0000","last_became_peered":"2023-08-09T15:08:46.141541+0000","last_unstale":"2023-08-09T15:08:46.149920+0000","last_undegraded":"2023-08-09T15:08:46.149920+0000","last_fullsized":"2023-08-09T15:08:46.149920+0000","mapping_epoch":203,"log_start":"0'0","ondisk_log_start":"0'0","created":203,"last_epoch_clean":204,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T15:08:45.049361+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T15:08:45.049361+0000","last_clean_scrub_stamp":"2023-08-09T15:08:45.049361+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-11T00:34:17.523462+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"24.1b","version":"204'1","reported_seq":12,"reported_epoch":204,"state":"active+clean","last_fresh":"2023-08-09T15:08:46.152409+0000","last_change":"2023-08-09T15:08:45.099228+0000","last_active":"2023-08-09T15:08:46.152409+0000","last_peered":"2023-08-09T15:08:46.152409+0000","last_clean":"2023-08-09T15:08:46.152409+0000","last_became_active":"2023-08-09T15:08:45.098602+0000","last_became_peered":"2023-08-09T15:08:45.098602+0000","last_unstale":"2023-08-09T15:08:46.152409+0000","last_undegraded":"2023-08-09T15:08:46.152409+0000","last_fullsized":"2023-08-09T15:08:46.152409+0000","mapping_epoch":202,"log_start":"0'0","ondisk_log_start":"0'0","created":202,"last_epoch_clean":203,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T15:08:44.039630+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T15:08:44.039630+0000","last_clean_scrub_stamp":"2023-08-09T15:08:44.039630+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T19:47:48.622160+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.2","version":"0'0","reported_seq":376,"reported_epoch":204,"state":"active+clean","last_fresh":"2023-08-09T15:08:46.127610+0000","last_change":"2023-08-09T13:33:56.402794+0000","last_active":"2023-08-09T15:08:46.127610+0000","last_peered":"2023-08-09T15:08:46.127610+0000","last_clean":"2023-08-09T15:08:46.127610+0000","last_became_active":"2023-08-09T13:33:56.402038+0000","last_became_peered":"2023-08-09T13:33:56.402038+0000","last_unstale":"2023-08-09T15:08:46.127610+0000","last_undegraded":"2023-08-09T15:08:46.127610+0000","last_fullsized":"2023-08-09T15:08:46.127610+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:33:54.845869+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:33:54.845869+0000","last_clean_scrub_stamp":"2023-08-09T13:33:54.845869+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T21:22:56.846648+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"25.1b","version":"0'0","reported_seq":9,"reported_epoch":204,"state":"active+clean","last_fresh":"2023-08-09T15:08:46.136567+0000","last_change":"2023-08-09T15:08:46.132150+0000","last_active":"2023-08-09T15:08:46.136567+0000","last_peered":"2023-08-09T15:08:46.136567+0000","last_clean":"2023-08-09T15:08:46.136567+0000","last_became_active":"2023-08-09T15:08:46.131829+0000","last_became_peered":"2023-08-09T15:08:46.131829+0000","last_unstale":"2023-08-09T15:08:46.136567+0000","last_undegraded":"2023-08-09T15:08:46.136567+0000","last_fullsized":"2023-08-09T15:08:46.136567+0000","mapping_epoch":203,"log_start":"0'0","ondisk_log_start":"0'0","created":203,"last_epoch_clean":204,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T15:08:45.049361+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T15:08:45.049361+0000","last_clean_scrub_stamp":"2023-08-09T15:08:45.049361+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T22:10:49.551259+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"24.1a","version":"204'1","reported_seq":12,"reported_epoch":204,"state":"active+clean","last_fresh":"2023-08-09T15:08:46.156000+0000","last_change":"2023-08-09T15:08:45.087523+0000","last_active":"2023-08-09T15:08:46.156000+0000","last_peered":"2023-08-09T15:08:46.156000+0000","last_clean":"2023-08-09T15:08:46.156000+0000","last_became_active":"2023-08-09T15:08:45.087207+0000","last_became_peered":"2023-08-09T15:08:45.087207+0000","last_unstale":"2023-08-09T15:08:46.156000+0000","last_undegraded":"2023-08-09T15:08:46.156000+0000","last_fullsized":"2023-08-09T15:08:46.156000+0000","mapping_epoch":202,"log_start":"0'0","ondisk_log_start":"0'0","created":202,"last_epoch_clean":203,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T15:08:44.039630+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T15:08:44.039630+0000","last_clean_scrub_stamp":"2023-08-09T15:08:44.039630+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T22:07:46.924207+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.3","version":"0'0","reported_seq":392,"reported_epoch":204,"state":"active+clean","last_fresh":"2023-08-09T15:08:46.102085+0000","last_change":"2023-08-09T13:33:55.886749+0000","last_active":"2023-08-09T15:08:46.102085+0000","last_peered":"2023-08-09T15:08:46.102085+0000","last_clean":"2023-08-09T15:08:46.102085+0000","last_became_active":"2023-08-09T13:33:55.886116+0000","last_became_peered":"2023-08-09T13:33:55.886116+0000","last_unstale":"2023-08-09T15:08:46.102085+0000","last_undegraded":"2023-08-09T15:08:46.102085+0000","last_fullsized":"2023-08-09T15:08:46.102085+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:33:54.845869+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:33:54.845869+0000","last_clean_scrub_stamp":"2023-08-09T13:33:54.845869+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T17:43:10.480990+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"193'307","reported_seq":1201,"reported_epoch":204,"state":"active+clean","last_fresh":"2023-08-09T15:08:46.119082+0000","last_change":"2023-08-09T13:33:55.886345+0000","last_active":"2023-08-09T15:08:46.119082+0000","last_peered":"2023-08-09T15:08:46.119082+0000","last_clean":"2023-08-09T15:08:46.119082+0000","last_became_active":"2023-08-09T13:33:55.885781+0000","last_became_peered":"2023-08-09T13:33:55.885781+0000","last_unstale":"2023-08-09T15:08:46.119082+0000","last_undegraded":"2023-08-09T15:08:46.119082+0000","last_fullsized":"2023-08-09T15:08:46.119082+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:33:54.845869+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:33:54.845869+0000","last_clean_scrub_stamp":"2023-08-09T13:33:54.845869+0000","objects_scrubbed":0,"log_size":307,"log_dups_size":0,"ondisk_log_size":307,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T17:27:08.086934+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1771,"num_read_kb":3684,"num_write":503,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"25.1c","version":"0'0","reported_seq":9,"reported_epoch":204,"state":"active+clean","last_fresh":"2023-08-09T15:08:46.137767+0000","last_change":"2023-08-09T15:08:46.136662+0000","last_active":"2023-08-09T15:08:46.137767+0000","last_peered":"2023-08-09T15:08:46.137767+0000","last_clean":"2023-08-09T15:08:46.137767+0000","last_became_active":"2023-08-09T15:08:46.136372+0000","last_became_peered":"2023-08-09T15:08:46.136372+0000","last_unstale":"2023-08-09T15:08:46.137767+0000","last_undegraded":"2023-08-09T15:08:46.137767+0000","last_fullsized":"2023-08-09T15:08:46.137767+0000","mapping_epoch":203,"log_start":"0'0","ondisk_log_start":"0'0","created":203,"last_epoch_clean":204,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T15:08:45.049361+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T15:08:45.049361+0000","last_clean_scrub_stamp":"2023-08-09T15:08:45.049361+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-11T02:46:30.377863+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"24.1d","version":"204'2","reported_seq":13,"reported_epoch":204,"state":"active+clean","last_fresh":"2023-08-09T15:08:46.150662+0000","last_change":"2023-08-09T15:08:45.087713+0000","last_active":"2023-08-09T15:08:46.150662+0000","last_peered":"2023-08-09T15:08:46.150662+0000","last_clean":"2023-08-09T15:08:46.150662+0000","last_became_active":"2023-08-09T15:08:45.087305+0000","last_became_peered":"2023-08-09T15:08:45.087305+0000","last_unstale":"2023-08-09T15:08:46.150662+0000","last_undegraded":"2023-08-09T15:08:46.150662+0000","last_fullsized":"2023-08-09T15:08:46.150662+0000","mapping_epoch":202,"log_start":"0'0","ondisk_log_start":"0'0","created":202,"last_epoch_clean":203,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T15:08:44.039630+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T15:08:44.039630+0000","last_clean_scrub_stamp":"2023-08-09T15:08:44.039630+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T17:53:50.187553+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":394,"reported_epoch":204,"state":"active+clean","last_fresh":"2023-08-09T15:08:46.106272+0000","last_change":"2023-08-09T13:33:55.889565+0000","last_active":"2023-08-09T15:08:46.106272+0000","last_peered":"2023-08-09T15:08:46.106272+0000","last_clean":"2023-08-09T15:08:46.106272+0000","last_became_active":"2023-08-09T13:33:55.889164+0000","last_became_peered":"2023-08-09T13:33:55.889164+0000","last_unstale":"2023-08-09T15:08:46.106272+0000","last_undegraded":"2023-08-09T15:08:46.106272+0000","last_fullsized":"2023-08-09T15:08:46.106272+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:33:54.845869+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:33:54.845869+0000","last_clean_scrub_stamp":"2023-08-09T13:33:54.845869+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T15:22:20.360173+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"25.1d","version":"0'0","reported_seq":9,"reported_epoch":204,"state":"active+clean","last_fresh":"2023-08-09T15:08:46.137153+0000","last_change":"2023-08-09T15:08:46.133494+0000","last_active":"2023-08-09T15:08:46.137153+0000","last_peered":"2023-08-09T15:08:46.137153+0000","last_clean":"2023-08-09T15:08:46.137153+0000","last_became_active":"2023-08-09T15:08:46.133145+0000","last_became_peered":"2023-08-09T15:08:46.133145+0000","last_unstale":"2023-08-09T15:08:46.137153+0000","last_undegraded":"2023-08-09T15:08:46.137153+0000","last_fullsized":"2023-08-09T15:08:46.137153+0000","mapping_epoch":203,"log_start":"0'0","ondisk_log_start":"0'0","created":203,"last_epoch_clean":204,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T15:08:45.049361+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T15:08:45.049361+0000","last_clean_scrub_stamp":"2023-08-09T15:08:45.049361+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T22:17:05.642057+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"24.1c","version":"204'2","reported_seq":13,"reported_epoch":204,"state":"active+clean","last_fresh":"2023-08-09T15:08:46.148091+0000","last_change":"2023-08-09T15:08:45.089908+0000","last_active":"2023-08-09T15:08:46.148091+0000","last_peered":"2023-08-09T15:08:46.148091+0000","last_clean":"2023-08-09T15:08:46.148091+0000","last_became_active":"2023-08-09T15:08:45.089612+0000","last_became_peered":"2023-08-09T15:08:45.089612+0000","last_unstale":"2023-08-09T15:08:46.148091+0000","last_undegraded":"2023-08-09T15:08:46.148091+0000","last_fullsized":"2023-08-09T15:08:46.148091+0000","mapping_epoch":202,"log_start":"0'0","ondisk_log_start":"0'0","created":202,"last_epoch_clean":203,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T15:08:44.039630+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T15:08:44.039630+0000","last_clean_scrub_stamp":"2023-08-09T15:08:44.039630+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T20:53:18.888010+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":386,"reported_epoch":204,"state":"active+clean","last_fresh":"2023-08-09T15:08:46.107478+0000","last_change":"2023-08-09T13:33:56.404236+0000","last_active":"2023-08-09T15:08:46.107478+0000","last_peered":"2023-08-09T15:08:46.107478+0000","last_clean":"2023-08-09T15:08:46.107478+0000","last_became_active":"2023-08-09T13:33:56.402485+0000","last_became_peered":"2023-08-09T13:33:56.402485+0000","last_unstale":"2023-08-09T15:08:46.107478+0000","last_undegraded":"2023-08-09T15:08:46.107478+0000","last_fullsized":"2023-08-09T15:08:46.107478+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:33:54.845869+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:33:54.845869+0000","last_clean_scrub_stamp":"2023-08-09T13:33:54.845869+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T16:03:34.208897+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"25.1e","version":"0'0","reported_seq":9,"reported_epoch":204,"state":"active+clean","last_fresh":"2023-08-09T15:08:46.136637+0000","last_change":"2023-08-09T15:08:46.131911+0000","last_active":"2023-08-09T15:08:46.136637+0000","last_peered":"2023-08-09T15:08:46.136637+0000","last_clean":"2023-08-09T15:08:46.136637+0000","last_became_active":"2023-08-09T15:08:46.130158+0000","last_became_peered":"2023-08-09T15:08:46.130158+0000","last_unstale":"2023-08-09T15:08:46.136637+0000","last_undegraded":"2023-08-09T15:08:46.136637+0000","last_fullsized":"2023-08-09T15:08:46.136637+0000","mapping_epoch":203,"log_start":"0'0","ondisk_log_start":"0'0","created":203,"last_epoch_clean":204,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T15:08:45.049361+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T15:08:45.049361+0000","last_clean_scrub_stamp":"2023-08-09T15:08:45.049361+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T20:44:17.973099+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"24.1f","version":"204'4","reported_seq":15,"reported_epoch":204,"state":"active+clean","last_fresh":"2023-08-09T15:08:46.152032+0000","last_change":"2023-08-09T15:08:45.097633+0000","last_active":"2023-08-09T15:08:46.152032+0000","last_peered":"2023-08-09T15:08:46.152032+0000","last_clean":"2023-08-09T15:08:46.152032+0000","last_became_active":"2023-08-09T15:08:45.097388+0000","last_became_peered":"2023-08-09T15:08:45.097388+0000","last_unstale":"2023-08-09T15:08:46.152032+0000","last_undegraded":"2023-08-09T15:08:46.152032+0000","last_fullsized":"2023-08-09T15:08:46.152032+0000","mapping_epoch":202,"log_start":"0'0","ondisk_log_start":"0'0","created":202,"last_epoch_clean":203,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T15:08:44.039630+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T15:08:44.039630+0000","last_clean_scrub_stamp":"2023-08-09T15:08:44.039630+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T19:55:17.731855+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":374,"reported_epoch":204,"state":"active+clean","last_fresh":"2023-08-09T15:08:46.118852+0000","last_change":"2023-08-09T13:33:56.401781+0000","last_active":"2023-08-09T15:08:46.118852+0000","last_peered":"2023-08-09T15:08:46.118852+0000","last_clean":"2023-08-09T15:08:46.118852+0000","last_became_active":"2023-08-09T13:33:56.400336+0000","last_became_peered":"2023-08-09T13:33:56.400336+0000","last_unstale":"2023-08-09T15:08:46.118852+0000","last_undegraded":"2023-08-09T15:08:46.118852+0000","last_fullsized":"2023-08-09T15:08:46.118852+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:33:54.845869+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:33:54.845869+0000","last_clean_scrub_stamp":"2023-08-09T13:33:54.845869+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T21:34:46.146543+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"25.1f","version":"0'0","reported_seq":9,"reported_epoch":204,"state":"active+clean","last_fresh":"2023-08-09T15:08:46.150362+0000","last_change":"2023-08-09T15:08:46.143687+0000","last_active":"2023-08-09T15:08:46.150362+0000","last_peered":"2023-08-09T15:08:46.150362+0000","last_clean":"2023-08-09T15:08:46.150362+0000","last_became_active":"2023-08-09T15:08:46.143398+0000","last_became_peered":"2023-08-09T15:08:46.143398+0000","last_unstale":"2023-08-09T15:08:46.150362+0000","last_undegraded":"2023-08-09T15:08:46.150362+0000","last_fullsized":"2023-08-09T15:08:46.150362+0000","mapping_epoch":203,"log_start":"0'0","ondisk_log_start":"0'0","created":203,"last_epoch_clean":204,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T15:08:45.049361+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T15:08:45.049361+0000","last_clean_scrub_stamp":"2023-08-09T15:08:45.049361+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T17:29:59.824781+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"24.1e","version":"204'1","reported_seq":12,"reported_epoch":204,"state":"active+clean","last_fresh":"2023-08-09T15:08:46.146842+0000","last_change":"2023-08-09T15:08:45.081144+0000","last_active":"2023-08-09T15:08:46.146842+0000","last_peered":"2023-08-09T15:08:46.146842+0000","last_clean":"2023-08-09T15:08:46.146842+0000","last_became_active":"2023-08-09T15:08:45.080850+0000","last_became_peered":"2023-08-09T15:08:45.080850+0000","last_unstale":"2023-08-09T15:08:46.146842+0000","last_undegraded":"2023-08-09T15:08:46.146842+0000","last_fullsized":"2023-08-09T15:08:46.146842+0000","mapping_epoch":202,"log_start":"0'0","ondisk_log_start":"0'0","created":202,"last_epoch_clean":203,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T15:08:44.039630+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T15:08:44.039630+0000","last_clean_scrub_stamp":"2023-08-09T15:08:44.039630+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T15:49:06.727195+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":374,"reported_epoch":204,"state":"active+clean","last_fresh":"2023-08-09T15:08:46.119593+0000","last_change":"2023-08-09T13:33:55.886389+0000","last_active":"2023-08-09T15:08:46.119593+0000","last_peered":"2023-08-09T15:08:46.119593+0000","last_clean":"2023-08-09T15:08:46.119593+0000","last_became_active":"2023-08-09T13:33:55.885638+0000","last_became_peered":"2023-08-09T13:33:55.885638+0000","last_unstale":"2023-08-09T15:08:46.119593+0000","last_undegraded":"2023-08-09T15:08:46.119593+0000","last_fullsized":"2023-08-09T15:08:46.119593+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:33:54.845869+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:33:54.845869+0000","last_clean_scrub_stamp":"2023-08-09T13:33:54.845869+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T16:56:49.433631+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]}],"pool_stats":[{"poolid":25,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":24,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":38,"ondisk_log_size":38,"up":64,"acting":64,"num_store_stats":8},{"poolid":2,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1771,"num_read_kb":3684,"num_write":503,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":307,"ondisk_log_size":307,"up":2,"acting":2,"num_store_stats":2},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":8}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834575981,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":52224,"kb_used_data":3060,"kb_used_omap":1,"kb_used_meta":49150,"kb_avail":94319616,"statfs":{"total":96636764160,"available":96583286784,"internally_reserved":0,"allocated":3133440,"data_stored":1971235,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1781,"internal_metadata":50329867},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Aug 9 15:08:46 2023","interfaces":[{"interface":"back","average":{"1min":1.2789999999999999,"5min":1.246,"15min":1.2350000000000001},"min":{"1min":0.92300000000000004,"5min":0.77000000000000002,"15min":0.76100000000000001},"max":{"1min":2.056,"5min":2.056,"15min":2.056},"last":0.92300000000000004},{"interface":"front","average":{"1min":1.234,"5min":1.2709999999999999,"15min":1.2689999999999999},"min":{"1min":0.91200000000000003,"5min":0.84699999999999998,"15min":0.81200000000000006},"max":{"1min":1.7769999999999999,"5min":1.863,"15min":1.863},"last":0.95299999999999996}]},{"osd":1,"last update":"Wed Aug 9 15:08:46 2023","interfaces":[{"interface":"back","average":{"1min":1.212,"5min":1.2909999999999999,"15min":1.3129999999999999},"min":{"1min":0.88,"5min":0.88,"15min":0.83399999999999996},"max":{"1min":1.762,"5min":1.762,"15min":1.855},"last":1.113},{"interface":"front","average":{"1min":1.3100000000000001,"5min":1.335,"15min":1.3140000000000001},"min":{"1min":0.879,"5min":0.879,"15min":0.879},"max":{"1min":2.1520000000000001,"5min":2.1520000000000001,"15min":2.1520000000000001},"last":1.173}]},{"osd":2,"last update":"Wed Aug 9 15:08:46 2023","interfaces":[{"interface":"back","average":{"1min":1.1639999999999999,"5min":1.333,"15min":1.3080000000000001},"min":{"1min":0.98699999999999999,"5min":0.93300000000000005,"15min":0.86799999999999999},"max":{"1min":1.4199999999999999,"5min":1.7729999999999999,"15min":1.7829999999999999},"last":1.016},{"interface":"front","average":{"1min":1.337,"5min":1.4259999999999999,"15min":1.4059999999999999},"min":{"1min":1.056,"5min":1.056,"15min":0.88500000000000001},"max":{"1min":1.8340000000000001,"5min":1.8340000000000001,"15min":1.8400000000000001},"last":1.216}]},{"osd":3,"last update":"Wed Aug 9 15:08:46 2023","interfaces":[{"interface":"back","average":{"1min":1.3029999999999999,"5min":1.365,"15min":1.359},"min":{"1min":0.80800000000000005,"5min":0.80800000000000005,"15min":0.80800000000000005},"max":{"1min":1.657,"5min":1.9259999999999999,"15min":1.9259999999999999},"last":1.2569999999999999},{"interface":"front","average":{"1min":1.409,"5min":1.4950000000000001,"15min":1.4810000000000001},"min":{"1min":1.02,"5min":0.94499999999999995,"15min":0.94499999999999995},"max":{"1min":1.669,"5min":1.901,"15min":1.9530000000000001},"last":1.2989999999999999}]},{"osd":4,"last update":"Wed Aug 9 15:08:46 2023","interfaces":[{"interface":"back","average":{"1min":1.3360000000000001,"5min":1.3009999999999999,"15min":1.3049999999999999},"min":{"1min":0.86799999999999999,"5min":0.86799999999999999,"15min":0.69599999999999995},"max":{"1min":2.2810000000000001,"5min":2.2810000000000001,"15min":2.2810000000000001},"last":1.341},{"interface":"front","average":{"1min":1.3680000000000001,"5min":1.379,"15min":1.3799999999999999},"min":{"1min":0.94399999999999995,"5min":0.92300000000000004,"15min":0.66000000000000003},"max":{"1min":1.782,"5min":2.0830000000000002,"15min":2.0830000000000002},"last":1.782}]},{"osd":5,"last update":"Wed Aug 9 15:08:46 2023","interfaces":[{"interface":"back","average":{"1min":1.462,"5min":1.4450000000000001,"15min":1.4550000000000001},"min":{"1min":0.89000000000000001,"5min":0.86099999999999999,"15min":0.86099999999999999},"max":{"1min":1.9550000000000001,"5min":1.9550000000000001,"15min":1.9550000000000001},"last":1.4610000000000001},{"interface":"front","average":{"1min":1.5289999999999999,"5min":1.4950000000000001,"15min":1.4910000000000001},"min":{"1min":1.0049999999999999,"5min":0.98399999999999999,"15min":0.95099999999999996},"max":{"1min":2.5310000000000001,"5min":2.5310000000000001,"15min":2.5310000000000001},"last":1.5029999999999999}]},{"osd":6,"last update":"Wed Aug 9 15:08:46 2023","interfaces":[{"interface":"back","average":{"1min":1.6970000000000001,"5min":1.6399999999999999,"15min":1.605},"min":{"1min":1.4139999999999999,"5min":1.268,"15min":1.101},"max":{"1min":2.8959999999999999,"5min":2.8959999999999999,"15min":2.8959999999999999},"last":1.5800000000000001},{"interface":"front","average":{"1min":1.6200000000000001,"5min":1.5820000000000001,"15min":1.55},"min":{"1min":1.2809999999999999,"5min":1.1519999999999999,"15min":0.90800000000000003},"max":{"1min":2.2130000000000001,"5min":2.222,"15min":2.222},"last":1.919}]}]},{"osd":6,"up_from":13,"seq":55834575981,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":59844,"kb_used_data":2488,"kb_used_omap":1,"kb_used_meta":57342,"kb_avail":94311996,"statfs":{"total":96636764160,"available":96575483904,"internally_reserved":0,"allocated":2547712,"data_stored":1386998,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1781,"internal_metadata":58718475},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Aug 9 15:07:53 2023","interfaces":[{"interface":"back","average":{"1min":1.2829999999999999,"5min":1.2170000000000001,"15min":1.2230000000000001},"min":{"1min":0.94299999999999995,"5min":0.90000000000000002,"15min":0.64100000000000001},"max":{"1min":1.772,"5min":1.772,"15min":1.8859999999999999},"last":1.181},{"interface":"front","average":{"1min":1.25,"5min":1.2649999999999999,"15min":1.25},"min":{"1min":0.92100000000000004,"5min":0.83099999999999996,"15min":0.71399999999999997},"max":{"1min":1.8680000000000001,"5min":1.8680000000000001,"15min":1.917},"last":1.0009999999999999}]},{"osd":1,"last update":"Wed Aug 9 15:07:53 2023","interfaces":[{"interface":"back","average":{"1min":1.236,"5min":1.194,"15min":1.2010000000000001},"min":{"1min":1.022,"5min":0.90200000000000002,"15min":0.75700000000000001},"max":{"1min":1.5600000000000001,"5min":1.7529999999999999,"15min":1.7529999999999999},"last":0.91900000000000004},{"interface":"front","average":{"1min":1.3540000000000001,"5min":1.3580000000000001,"15min":1.343},"min":{"1min":0.99399999999999999,"5min":0.91100000000000003,"15min":0.84499999999999997},"max":{"1min":1.9410000000000001,"5min":1.9410000000000001,"15min":1.9410000000000001},"last":1.266}]},{"osd":2,"last update":"Wed Aug 9 15:07:53 2023","interfaces":[{"interface":"back","average":{"1min":1.3460000000000001,"5min":1.3500000000000001,"15min":1.343},"min":{"1min":1.0009999999999999,"5min":0.82999999999999996,"15min":0.752},"max":{"1min":1.623,"5min":1.9259999999999999,"15min":2.081},"last":0.82099999999999995},{"interface":"front","average":{"1min":1.3660000000000001,"5min":1.349,"15min":1.3540000000000001},"min":{"1min":1.0589999999999999,"5min":0.90600000000000003,"15min":0.88900000000000001},"max":{"1min":1.6990000000000001,"5min":1.8700000000000001,"15min":1.8700000000000001},"last":1.2889999999999999}]},{"osd":3,"last update":"Wed Aug 9 15:07:53 2023","interfaces":[{"interface":"back","average":{"1min":1.3919999999999999,"5min":1.401,"15min":1.395},"min":{"1min":1.018,"5min":0.98499999999999999,"15min":0.85499999999999998},"max":{"1min":1.831,"5min":1.831,"15min":1.9450000000000001},"last":1.413},{"interface":"front","average":{"1min":1.3480000000000001,"5min":1.3440000000000001,"15min":1.353},"min":{"1min":1.103,"5min":0.97599999999999998,"15min":0.95299999999999996},"max":{"1min":1.6319999999999999,"5min":1.8660000000000001,"15min":1.8660000000000001},"last":1.0449999999999999}]},{"osd":4,"last update":"Wed Aug 9 15:07:53 2023","interfaces":[{"interface":"back","average":{"1min":1.351,"5min":1.3640000000000001,"15min":1.321},"min":{"1min":0.78800000000000003,"5min":0.77200000000000002,"15min":0.48599999999999999},"max":{"1min":2.0470000000000002,"5min":2.0470000000000002,"15min":2.0470000000000002},"last":1.3540000000000001},{"interface":"front","average":{"1min":1.4330000000000001,"5min":1.466,"15min":1.413},"min":{"1min":0.82099999999999995,"5min":0.82099999999999995,"15min":0.52000000000000002},"max":{"1min":1.8480000000000001,"5min":1.9970000000000001,"15min":2.2280000000000002},"last":1.133}]},{"osd":5,"last update":"Wed Aug 9 15:07:53 2023","interfaces":[{"interface":"back","average":{"1min":1.405,"5min":1.3919999999999999,"15min":1.3600000000000001},"min":{"1min":0.94299999999999995,"5min":0.93899999999999995,"15min":0.72599999999999998},"max":{"1min":1.754,"5min":2.6059999999999999,"15min":2.6059999999999999},"last":1.234},{"interface":"front","average":{"1min":1.633,"5min":1.6299999999999999,"15min":1.607},"min":{"1min":1.1180000000000001,"5min":0.85599999999999998,"15min":0.68500000000000005},"max":{"1min":2.1200000000000001,"5min":2.1200000000000001,"15min":2.1899999999999999},"last":1.6579999999999999}]},{"osd":7,"last update":"Wed Aug 9 15:07:53 2023","interfaces":[{"interface":"back","average":{"1min":1.6359999999999999,"5min":1.645,"15min":1.5980000000000001},"min":{"1min":1.2929999999999999,"5min":0.84699999999999998,"15min":0.624},"max":{"1min":1.972,"5min":2.4340000000000002,"15min":2.4340000000000002},"last":1.5580000000000001},{"interface":"front","average":{"1min":1.589,"5min":1.599,"15min":1.5860000000000001},"min":{"1min":1.282,"5min":1.143,"15min":0.83899999999999997},"max":{"1min":1.9019999999999999,"5min":2.7400000000000002,"15min":2.7400000000000002},"last":1.6060000000000001}]}]},{"osd":5,"up_from":13,"seq":55834575981,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":47552,"kb_used_data":2480,"kb_used_omap":1,"kb_used_meta":45054,"kb_avail":94324288,"statfs":{"total":96636764160,"available":96588070912,"internally_reserved":0,"allocated":2539520,"data_stored":1380867,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1781,"internal_metadata":46135563},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Aug 9 15:08:42 2023","interfaces":[{"interface":"back","average":{"1min":1.2210000000000001,"5min":1.1859999999999999,"15min":1.202},"min":{"1min":0.59399999999999997,"5min":0.59399999999999997,"15min":0.59399999999999997},"max":{"1min":1.7370000000000001,"5min":1.754,"15min":1.754},"last":0.96899999999999997},{"interface":"front","average":{"1min":1.1559999999999999,"5min":1.2430000000000001,"15min":1.228},"min":{"1min":0.67000000000000004,"5min":0.67000000000000004,"15min":0.67000000000000004},"max":{"1min":1.6180000000000001,"5min":1.865,"15min":1.976},"last":0.76700000000000002}]},{"osd":1,"last update":"Wed Aug 9 15:08:42 2023","interfaces":[{"interface":"back","average":{"1min":1.256,"5min":1.2450000000000001,"15min":1.258},"min":{"1min":0.65200000000000002,"5min":0.65200000000000002,"15min":0.65200000000000002},"max":{"1min":1.823,"5min":2.016,"15min":2.016},"last":1.4419999999999999},{"interface":"front","average":{"1min":1.196,"5min":1.3460000000000001,"15min":1.359},"min":{"1min":0.751,"5min":0.751,"15min":0.751},"max":{"1min":1.728,"5min":1.8959999999999999,"15min":1.8979999999999999},"last":1.1619999999999999}]},{"osd":2,"last update":"Wed Aug 9 15:08:42 2023","interfaces":[{"interface":"back","average":{"1min":1.351,"5min":1.325,"15min":1.321},"min":{"1min":0.89700000000000002,"5min":0.89700000000000002,"15min":0.82499999999999996},"max":{"1min":1.855,"5min":2.032,"15min":2.032},"last":0.89700000000000002},{"interface":"front","average":{"1min":1.3089999999999999,"5min":1.3839999999999999,"15min":1.3839999999999999},"min":{"1min":0.81100000000000005,"5min":0.81100000000000005,"15min":0.81100000000000005},"max":{"1min":1.956,"5min":1.964,"15min":1.964},"last":1.6830000000000001}]},{"osd":3,"last update":"Wed Aug 9 15:08:42 2023","interfaces":[{"interface":"back","average":{"1min":1.355,"5min":1.4099999999999999,"15min":1.387},"min":{"1min":1.111,"5min":0.97399999999999998,"15min":0.93799999999999994},"max":{"1min":1.486,"5min":1.867,"15min":1.899},"last":1.286},{"interface":"front","average":{"1min":1.3620000000000001,"5min":1.4630000000000001,"15min":1.4610000000000001},"min":{"1min":0.85999999999999999,"5min":0.85999999999999999,"15min":0.85999999999999999},"max":{"1min":1.7070000000000001,"5min":2.0379999999999998,"15min":2.0379999999999998},"last":1.5009999999999999}]},{"osd":4,"last update":"Wed Aug 9 15:08:42 2023","interfaces":[{"interface":"back","average":{"1min":1.3720000000000001,"5min":1.3320000000000001,"15min":1.3320000000000001},"min":{"1min":0.96699999999999997,"5min":0.79000000000000004,"15min":0.62},"max":{"1min":2.0299999999999998,"5min":2.0299999999999998,"15min":2.0299999999999998},"last":1.226},{"interface":"front","average":{"1min":1.4810000000000001,"5min":1.421,"15min":1.417},"min":{"1min":0.97899999999999998,"5min":0.85899999999999999,"15min":0.64200000000000002},"max":{"1min":1.9239999999999999,"5min":2.0110000000000001,"15min":2.0390000000000001},"last":1.548}]},{"osd":6,"last update":"Wed Aug 9 15:08:42 2023","interfaces":[{"interface":"back","average":{"1min":1.391,"5min":1.4159999999999999,"15min":1.4279999999999999},"min":{"1min":0.98399999999999999,"5min":0.96099999999999997,"15min":0.67400000000000004},"max":{"1min":1.8799999999999999,"5min":2.1000000000000001,"15min":2.1000000000000001},"last":1.325},{"interface":"front","average":{"1min":1.5189999999999999,"5min":1.524,"15min":1.5229999999999999},"min":{"1min":1.1879999999999999,"5min":0.91200000000000003,"15min":0.79900000000000004},"max":{"1min":2.1349999999999998,"5min":2.1349999999999998,"15min":2.3410000000000002},"last":1.577}]},{"osd":7,"last update":"Wed Aug 9 15:08:42 2023","interfaces":[{"interface":"back","average":{"1min":1.5980000000000001,"5min":1.5820000000000001,"15min":1.595},"min":{"1min":1.286,"5min":1.0569999999999999,"15min":0.86099999999999999},"max":{"1min":1.929,"5min":1.956,"15min":2.4689999999999999},"last":1.3999999999999999},{"interface":"front","average":{"1min":1.6399999999999999,"5min":1.6459999999999999,"15min":1.657},"min":{"1min":1.2829999999999999,"5min":1.044,"15min":0.83399999999999996},"max":{"1min":1.889,"5min":2.1539999999999999,"15min":2.4239999999999999},"last":1.6279999999999999}]}]},{"osd":4,"up_from":13,"seq":55834575981,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":47552,"kb_used_data":2480,"kb_used_omap":1,"kb_used_meta":45054,"kb_avail":94324288,"statfs":{"total":96636764160,"available":96588070912,"internally_reserved":0,"allocated":2539520,"data_stored":1380867,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1781,"internal_metadata":46135563},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Aug 9 15:08:09 2023","interfaces":[{"interface":"back","average":{"1min":1.1859999999999999,"5min":1.2310000000000001,"15min":1.232},"min":{"1min":0.92100000000000004,"5min":0.63500000000000001,"15min":0.63500000000000001},"max":{"1min":1.6919999999999999,"5min":1.8360000000000001,"15min":2.2959999999999998},"last":0.77000000000000002},{"interface":"front","average":{"1min":1.3859999999999999,"5min":1.363,"15min":1.3300000000000001},"min":{"1min":1.0720000000000001,"5min":0.92800000000000005,"15min":0.85099999999999998},"max":{"1min":1.7529999999999999,"5min":1.7529999999999999,"15min":2.2709999999999999},"last":0.72699999999999998}]},{"osd":1,"last update":"Wed Aug 9 15:08:09 2023","interfaces":[{"interface":"back","average":{"1min":1.28,"5min":1.371,"15min":1.3500000000000001},"min":{"1min":0.94099999999999995,"5min":0.89700000000000002,"15min":0.84699999999999998},"max":{"1min":1.756,"5min":1.8460000000000001,"15min":1.9650000000000001},"last":1.052},{"interface":"front","average":{"1min":1.2729999999999999,"5min":1.26,"15min":1.278},"min":{"1min":0.89900000000000002,"5min":0.83399999999999996,"15min":0.83399999999999996},"max":{"1min":1.8600000000000001,"5min":1.8600000000000001,"15min":2.117},"last":0.98099999999999998}]},{"osd":2,"last update":"Wed Aug 9 15:08:09 2023","interfaces":[{"interface":"back","average":{"1min":1.28,"5min":1.3029999999999999,"15min":1.3100000000000001},"min":{"1min":0.91000000000000003,"5min":0.90700000000000003,"15min":0.84599999999999997},"max":{"1min":1.633,"5min":1.7989999999999999,"15min":1.883},"last":1.3819999999999999},{"interface":"front","average":{"1min":1.4019999999999999,"5min":1.403,"15min":1.411},"min":{"1min":1.21,"5min":0.79200000000000004,"15min":0.79200000000000004},"max":{"1min":1.667,"5min":1.8240000000000001,"15min":2.117},"last":1.4550000000000001}]},{"osd":3,"last update":"Wed Aug 9 15:08:09 2023","interfaces":[{"interface":"back","average":{"1min":1.345,"5min":1.347,"15min":1.3500000000000001},"min":{"1min":1.0089999999999999,"5min":0.88700000000000001,"15min":0.88700000000000001},"max":{"1min":1.591,"5min":1.931,"15min":2.4870000000000001},"last":1.292},{"interface":"front","average":{"1min":1.444,"5min":1.458,"15min":1.478},"min":{"1min":1.1879999999999999,"5min":1.0249999999999999,"15min":1.0249999999999999},"max":{"1min":1.909,"5min":2.5459999999999998,"15min":2.5459999999999998},"last":1.5289999999999999}]},{"osd":5,"last update":"Wed Aug 9 15:08:09 2023","interfaces":[{"interface":"back","average":{"1min":1.425,"5min":1.385,"15min":1.4159999999999999},"min":{"1min":1.0089999999999999,"5min":0.82999999999999996,"15min":0.82199999999999995},"max":{"1min":1.8700000000000001,"5min":1.893,"15min":2.4430000000000001},"last":1.22},{"interface":"front","average":{"1min":1.389,"5min":1.3799999999999999,"15min":1.397},"min":{"1min":0.91000000000000003,"5min":0.70899999999999996,"15min":0.70899999999999996},"max":{"1min":1.7130000000000001,"5min":2.1150000000000002,"15min":2.4249999999999998},"last":1.7050000000000001}]},{"osd":6,"last update":"Wed Aug 9 15:08:09 2023","interfaces":[{"interface":"back","average":{"1min":1.448,"5min":1.3899999999999999,"15min":1.419},"min":{"1min":0.95399999999999996,"5min":0.86299999999999999,"15min":0.80600000000000005},"max":{"1min":2.0049999999999999,"5min":2.9119999999999999,"15min":2.9119999999999999},"last":1.659},{"interface":"front","average":{"1min":1.5740000000000001,"5min":1.5700000000000001,"15min":1.5840000000000001},"min":{"1min":1.0329999999999999,"5min":0.86399999999999999,"15min":0.86399999999999999},"max":{"1min":1.8160000000000001,"5min":2.488,"15min":2.5219999999999998},"last":1.5760000000000001}]},{"osd":7,"last update":"Wed Aug 9 15:08:09 2023","interfaces":[{"interface":"back","average":{"1min":1.484,"5min":1.498,"15min":1.5269999999999999},"min":{"1min":1.0049999999999999,"5min":0.94599999999999995,"15min":0.94599999999999995},"max":{"1min":1.9610000000000001,"5min":2.355,"15min":2.5910000000000002},"last":1.772},{"interface":"front","average":{"1min":1.611,"5min":1.5840000000000001,"15min":1.6160000000000001},"min":{"1min":1.1259999999999999,"5min":0.90300000000000002,"15min":0.90300000000000002},"max":{"1min":2.1720000000000002,"5min":3.028,"15min":3.028},"last":1.748}]}]},{"osd":3,"up_from":13,"seq":55834575982,"num_pgs":19,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":63936,"kb_used_data":2480,"kb_used_omap":1,"kb_used_meta":61438,"kb_avail":94307904,"statfs":{"total":96636764160,"available":96571293696,"internally_reserved":0,"allocated":2539520,"data_stored":1380867,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1781,"internal_metadata":62912779},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Aug 9 15:07:55 2023","interfaces":[{"interface":"back","average":{"1min":1.036,"5min":0.97899999999999998,"15min":0.97399999999999998},"min":{"1min":0.745,"5min":0.66400000000000003,"15min":0.45200000000000001},"max":{"1min":1.5900000000000001,"5min":2.2090000000000001,"15min":2.2090000000000001},"last":1.577},{"interface":"front","average":{"1min":1.1599999999999999,"5min":1.175,"15min":1.179},"min":{"1min":0.78300000000000003,"5min":0.77900000000000003,"15min":0.56100000000000005},"max":{"1min":1.8089999999999999,"5min":1.8720000000000001,"15min":1.8720000000000001},"last":1.0760000000000001}]},{"osd":1,"last update":"Wed Aug 9 15:07:55 2023","interfaces":[{"interface":"back","average":{"1min":1.04,"5min":1.0860000000000001,"15min":1.0800000000000001},"min":{"1min":0.79600000000000004,"5min":0.748,"15min":0.51500000000000001},"max":{"1min":1.476,"5min":2.27,"15min":2.27},"last":1.0009999999999999},{"interface":"front","average":{"1min":1.288,"5min":1.292,"15min":1.264},"min":{"1min":0.93500000000000005,"5min":0.86199999999999999,"15min":0.623},"max":{"1min":1.772,"5min":1.9730000000000001,"15min":1.9730000000000001},"last":1.1319999999999999}]},{"osd":2,"last update":"Wed Aug 9 15:07:55 2023","interfaces":[{"interface":"back","average":{"1min":1.304,"5min":1.3360000000000001,"15min":1.339},"min":{"1min":0.998,"5min":0.879,"15min":0.55300000000000005},"max":{"1min":1.675,"5min":2.4079999999999999,"15min":2.4079999999999999},"last":1.3080000000000001},{"interface":"front","average":{"1min":1.486,"5min":1.4630000000000001,"15min":1.4390000000000001},"min":{"1min":0.99299999999999999,"5min":0.90000000000000002,"15min":0.72999999999999998},"max":{"1min":1.837,"5min":1.915,"15min":2.1259999999999999},"last":1.3779999999999999}]},{"osd":4,"last update":"Wed Aug 9 15:07:55 2023","interfaces":[{"interface":"back","average":{"1min":1.429,"5min":1.4339999999999999,"15min":1.421},"min":{"1min":1.0469999999999999,"5min":0.96999999999999997,"15min":0.72599999999999998},"max":{"1min":1.9179999999999999,"5min":2.2069999999999999,"15min":2.238},"last":1.4419999999999999},{"interface":"front","average":{"1min":1.4390000000000001,"5min":1.452,"15min":1.474},"min":{"1min":1.083,"5min":1.042,"15min":0.89500000000000002},"max":{"1min":1.851,"5min":2.0019999999999998,"15min":2.3460000000000001},"last":1.246}]},{"osd":5,"last update":"Wed Aug 9 15:07:55 2023","interfaces":[{"interface":"back","average":{"1min":1.5589999999999999,"5min":1.546,"15min":1.554},"min":{"1min":1.2050000000000001,"5min":1.2050000000000001,"15min":0.88800000000000001},"max":{"1min":1.895,"5min":1.9850000000000001,"15min":2.5579999999999998},"last":1.4219999999999999},{"interface":"front","average":{"1min":1.639,"5min":1.6100000000000001,"15min":1.6100000000000001},"min":{"1min":1.2150000000000001,"5min":1.0680000000000001,"15min":0.95599999999999996},"max":{"1min":2.2469999999999999,"5min":2.3109999999999999,"15min":2.4049999999999998},"last":1.653}]},{"osd":6,"last update":"Wed Aug 9 15:07:55 2023","interfaces":[{"interface":"back","average":{"1min":1.5509999999999999,"5min":1.591,"15min":1.601},"min":{"1min":1.2150000000000001,"5min":1.0860000000000001,"15min":0.96899999999999997},"max":{"1min":1.972,"5min":2.1669999999999998,"15min":2.4590000000000001},"last":1.7549999999999999},{"interface":"front","average":{"1min":1.6279999999999999,"5min":1.6639999999999999,"15min":1.6779999999999999},"min":{"1min":1.222,"5min":1.222,"15min":0.93500000000000005},"max":{"1min":2.032,"5min":2.1429999999999998,"15min":2.6509999999999998},"last":1.671}]},{"osd":7,"last update":"Wed Aug 9 15:07:55 2023","interfaces":[{"interface":"back","average":{"1min":1.8080000000000001,"5min":1.7729999999999999,"15min":1.758},"min":{"1min":1.5329999999999999,"5min":1.2250000000000001,"15min":1.0840000000000001},"max":{"1min":2.1699999999999999,"5min":2.2909999999999999,"15min":2.516},"last":2.008},{"interface":"front","average":{"1min":1.7170000000000001,"5min":1.706,"15min":1.726},"min":{"1min":1.3819999999999999,"5min":1.169,"15min":1.0369999999999999},"max":{"1min":2.0619999999999998,"5min":2.1200000000000001,"15min":2.9380000000000002},"last":1.8280000000000001}]}]},{"osd":2,"up_from":13,"seq":55834575981,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":41812,"kb_used_data":2504,"kb_used_omap":1,"kb_used_meta":39294,"kb_avail":94330028,"statfs":{"total":96636764160,"available":96593948672,"internally_reserved":0,"allocated":2564096,"data_stored":1389053,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1781,"internal_metadata":40237323},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Aug 9 15:07:45 2023","interfaces":[{"interface":"back","average":{"1min":1.034,"5min":0.96799999999999997,"15min":0.95099999999999996},"min":{"1min":0.76400000000000001,"5min":0.70099999999999996,"15min":0.44400000000000001},"max":{"1min":1.482,"5min":1.587,"15min":1.625},"last":0.44800000000000001},{"interface":"front","average":{"1min":1.133,"5min":1.115,"15min":1.0900000000000001},"min":{"1min":0.72699999999999998,"5min":0.70699999999999996,"15min":0.503},"max":{"1min":1.764,"5min":2.052,"15min":2.052},"last":0.59499999999999997}]},{"osd":1,"last update":"Wed Aug 9 15:07:45 2023","interfaces":[{"interface":"back","average":{"1min":1.3620000000000001,"5min":1.266,"15min":1.232},"min":{"1min":0.90700000000000003,"5min":0.83799999999999997,"15min":0.59399999999999997},"max":{"1min":1.7150000000000001,"5min":1.8149999999999999,"15min":1.825},"last":0.72399999999999998},{"interface":"front","average":{"1min":1.2490000000000001,"5min":1.387,"15min":1.4079999999999999},"min":{"1min":0.84399999999999997,"5min":0.84399999999999997,"15min":0.58299999999999996},"max":{"1min":1.6579999999999999,"5min":1.9490000000000001,"15min":1.9490000000000001},"last":0.77200000000000002}]},{"osd":3,"last update":"Wed Aug 9 15:07:45 2023","interfaces":[{"interface":"back","average":{"1min":1.143,"5min":1.1200000000000001,"15min":1.1060000000000001},"min":{"1min":0.96399999999999997,"5min":0.753,"15min":0.54800000000000004},"max":{"1min":1.496,"5min":1.7050000000000001,"15min":2.0070000000000001},"last":0.83199999999999996},{"interface":"front","average":{"1min":1.425,"5min":1.3899999999999999,"15min":1.369},"min":{"1min":0.93600000000000005,"5min":0.93600000000000005,"15min":0.76000000000000001},"max":{"1min":1.921,"5min":2.0470000000000002,"15min":2.0470000000000002},"last":0.64800000000000002}]},{"osd":4,"last update":"Wed Aug 9 15:07:45 2023","interfaces":[{"interface":"back","average":{"1min":1.6439999999999999,"5min":1.6100000000000001,"15min":1.542},"min":{"1min":1.3200000000000001,"5min":1.252,"15min":1.014},"max":{"1min":1.994,"5min":2.464,"15min":2.464},"last":0.90000000000000002},{"interface":"front","average":{"1min":1.4810000000000001,"5min":1.512,"15min":1.474},"min":{"1min":1.048,"5min":1.048,"15min":0.82399999999999995},"max":{"1min":1.9219999999999999,"5min":2.0720000000000001,"15min":2.2759999999999998},"last":1.0249999999999999}]},{"osd":5,"last update":"Wed Aug 9 15:07:45 2023","interfaces":[{"interface":"back","average":{"1min":1.5720000000000001,"5min":1.5669999999999999,"15min":1.5349999999999999},"min":{"1min":1.327,"5min":1.1220000000000001,"15min":0.92200000000000004},"max":{"1min":2.0369999999999999,"5min":2.1139999999999999,"15min":2.423},"last":0.93400000000000005},{"interface":"front","average":{"1min":1.5069999999999999,"5min":1.5089999999999999,"15min":1.5029999999999999},"min":{"1min":1.117,"5min":1.117,"15min":1.03},"max":{"1min":1.827,"5min":1.931,"15min":2.214},"last":1.2629999999999999}]},{"osd":6,"last update":"Wed Aug 9 15:07:45 2023","interfaces":[{"interface":"back","average":{"1min":1.571,"5min":1.548,"15min":1.526},"min":{"1min":1.133,"5min":1.0269999999999999,"15min":0.88},"max":{"1min":1.9450000000000001,"5min":2.0819999999999999,"15min":2.4550000000000001},"last":1.117},{"interface":"front","average":{"1min":1.7270000000000001,"5min":1.659,"15min":1.6319999999999999},"min":{"1min":1.3380000000000001,"5min":1.171,"15min":1.0549999999999999},"max":{"1min":2.077,"5min":2.351,"15min":2.3839999999999999},"last":1.0649999999999999}]},{"osd":7,"last update":"Wed Aug 9 15:07:45 2023","interfaces":[{"interface":"back","average":{"1min":1.6779999999999999,"5min":1.6739999999999999,"15min":1.6599999999999999},"min":{"1min":1.171,"5min":1.171,"15min":0.97699999999999998},"max":{"1min":2.1749999999999998,"5min":2.302,"15min":2.5099999999999998},"last":1.167},{"interface":"front","average":{"1min":1.655,"5min":1.661,"15min":1.6379999999999999},"min":{"1min":1.2330000000000001,"5min":1.224,"15min":0.84799999999999998},"max":{"1min":2.0230000000000001,"5min":2.3809999999999998,"15min":2.3809999999999998},"last":1.3300000000000001}]}]},{"osd":1,"up_from":13,"seq":55834575981,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":56344,"kb_used_data":3084,"kb_used_omap":1,"kb_used_meta":53246,"kb_avail":94315496,"statfs":{"total":96636764160,"available":96579067904,"internally_reserved":0,"allocated":3158016,"data_stored":1980339,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1781,"internal_metadata":54524171},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Aug 9 15:07:53 2023","interfaces":[{"interface":"back","average":{"1min":1.0620000000000001,"5min":1.0389999999999999,"15min":1.016},"min":{"1min":0.74299999999999999,"5min":0.59099999999999997,"15min":0.42399999999999999},"max":{"1min":1.6890000000000001,"5min":1.8129999999999999,"15min":2.0939999999999999},"last":1.2450000000000001},{"interface":"front","average":{"1min":1.149,"5min":1.1539999999999999,"15min":1.147},"min":{"1min":0.83499999999999996,"5min":0.68100000000000005,"15min":0.57799999999999996},"max":{"1min":1.78,"5min":1.8680000000000001,"15min":2.0190000000000001},"last":1.0940000000000001}]},{"osd":2,"last update":"Wed Aug 9 15:07:53 2023","interfaces":[{"interface":"back","average":{"1min":1.0640000000000001,"5min":1.0900000000000001,"15min":1.034},"min":{"1min":0.84999999999999998,"5min":0.64100000000000001,"15min":0.60099999999999998},"max":{"1min":1.589,"5min":2.3940000000000001,"15min":2.3940000000000001},"last":0.90100000000000002},{"interface":"front","average":{"1min":1.2350000000000001,"5min":1.2969999999999999,"15min":1.276},"min":{"1min":0.96099999999999997,"5min":0.871,"15min":0.67900000000000005},"max":{"1min":1.6499999999999999,"5min":1.8899999999999999,"15min":2},"last":0.82199999999999995}]},{"osd":3,"last update":"Wed Aug 9 15:07:53 2023","interfaces":[{"interface":"back","average":{"1min":1.4330000000000001,"5min":1.401,"15min":1.411},"min":{"1min":1.1100000000000001,"5min":1.0089999999999999,"15min":0.629},"max":{"1min":1.7829999999999999,"5min":2.109,"15min":2.109},"last":1.444},{"interface":"front","average":{"1min":1.302,"5min":1.3380000000000001,"15min":1.327},"min":{"1min":0.97199999999999998,"5min":0.97199999999999998,"15min":0.54100000000000004},"max":{"1min":1.5780000000000001,"5min":2.0259999999999998,"15min":2.0259999999999998},"last":1.024}]},{"osd":4,"last update":"Wed Aug 9 15:07:53 2023","interfaces":[{"interface":"back","average":{"1min":1.4890000000000001,"5min":1.5329999999999999,"15min":1.496},"min":{"1min":1.181,"5min":0.97599999999999998,"15min":0.751},"max":{"1min":1.8799999999999999,"5min":2.214,"15min":2.214},"last":1.1619999999999999},{"interface":"front","average":{"1min":1.425,"5min":1.452,"15min":1.4219999999999999},"min":{"1min":1.046,"5min":0.90600000000000003,"15min":0.90600000000000003},"max":{"1min":1.9630000000000001,"5min":2.1190000000000002,"15min":2.1190000000000002},"last":1.9730000000000001}]},{"osd":5,"last update":"Wed Aug 9 15:07:53 2023","interfaces":[{"interface":"back","average":{"1min":1.446,"5min":1.462,"15min":1.456},"min":{"1min":1.236,"5min":1.0389999999999999,"15min":0.86099999999999999},"max":{"1min":1.7829999999999999,"5min":2.508,"15min":2.508},"last":1.6870000000000001},{"interface":"front","average":{"1min":1.595,"5min":1.661,"15min":1.609},"min":{"1min":1.294,"5min":1.1339999999999999,"15min":0.78700000000000003},"max":{"1min":1.9870000000000001,"5min":2.2480000000000002,"15min":2.2480000000000002},"last":1.821}]},{"osd":6,"last update":"Wed Aug 9 15:07:53 2023","interfaces":[{"interface":"back","average":{"1min":1.6479999999999999,"5min":1.6779999999999999,"15min":1.631},"min":{"1min":1.2929999999999999,"5min":1.282,"15min":0.997},"max":{"1min":2.1400000000000001,"5min":2.294,"15min":2.294},"last":1.3029999999999999},{"interface":"front","average":{"1min":1.542,"5min":1.599,"15min":1.589},"min":{"1min":1.155,"5min":0.98999999999999999,"15min":0.91000000000000003},"max":{"1min":1.8220000000000001,"5min":2.278,"15min":2.5270000000000001},"last":1.8420000000000001}]},{"osd":7,"last update":"Wed Aug 9 15:07:53 2023","interfaces":[{"interface":"back","average":{"1min":1.75,"5min":1.782,"15min":1.72},"min":{"1min":1.399,"5min":1.26,"15min":0.92700000000000005},"max":{"1min":2.165,"5min":2.3050000000000002,"15min":2.488},"last":1.625},{"interface":"front","average":{"1min":1.645,"5min":1.776,"15min":1.714},"min":{"1min":1.254,"5min":1.254,"15min":1.0469999999999999},"max":{"1min":2.1040000000000001,"5min":2.4249999999999998,"15min":2.4249999999999998},"last":1.429}]}]},{"osd":0,"up_from":13,"seq":55834575981,"num_pgs":7,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":29512,"kb_used_data":2492,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94342328,"statfs":{"total":96636764160,"available":96606543872,"internally_reserved":0,"allocated":2551808,"data_stored":1387020,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1781,"internal_metadata":27654411},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Wed Aug 9 15:07:52 2023","interfaces":[{"interface":"back","average":{"1min":0.88100000000000001,"5min":0.93000000000000005,"15min":0.97299999999999998},"min":{"1min":0.69099999999999995,"5min":0.68799999999999994,"15min":0.51100000000000001},"max":{"1min":1.47,"5min":1.569,"15min":1.7230000000000001},"last":1.1859999999999999},{"interface":"front","average":{"1min":1.1890000000000001,"5min":1.175,"15min":1.1579999999999999},"min":{"1min":0.86299999999999999,"5min":0.76000000000000001,"15min":0.44},"max":{"1min":1.46,"5min":1.885,"15min":1.9410000000000001},"last":0.76100000000000001}]},{"osd":2,"last update":"Wed Aug 9 15:07:52 2023","interfaces":[{"interface":"back","average":{"1min":1.141,"5min":1.2090000000000001,"15min":1.198},"min":{"1min":0.81999999999999995,"5min":0.81999999999999995,"15min":0.61599999999999999},"max":{"1min":1.9039999999999999,"5min":1.9039999999999999,"15min":1.9039999999999999},"last":1.3360000000000001},{"interface":"front","average":{"1min":1.167,"5min":1.26,"15min":1.254},"min":{"1min":0.93999999999999995,"5min":0.79000000000000004,"15min":0.59699999999999998},"max":{"1min":1.458,"5min":1.9470000000000001,"15min":1.9470000000000001},"last":0.70899999999999996}]},{"osd":3,"last update":"Wed Aug 9 15:07:52 2023","interfaces":[{"interface":"back","average":{"1min":1.198,"5min":1.2290000000000001,"15min":1.224},"min":{"1min":0.93999999999999995,"5min":0.82499999999999996,"15min":0.46200000000000002},"max":{"1min":1.6200000000000001,"5min":1.9079999999999999,"15min":1.9079999999999999},"last":1.01},{"interface":"front","average":{"1min":1.3839999999999999,"5min":1.419,"15min":1.407},"min":{"1min":1.0129999999999999,"5min":1.0129999999999999,"15min":0.56599999999999995},"max":{"1min":1.8009999999999999,"5min":1.893,"15min":2.2469999999999999},"last":1.5}]},{"osd":4,"last update":"Wed Aug 9 15:07:52 2023","interfaces":[{"interface":"back","average":{"1min":1.4159999999999999,"5min":1.452,"15min":1.4139999999999999},"min":{"1min":1.1040000000000001,"5min":1.1040000000000001,"15min":0.61599999999999999},"max":{"1min":1.8049999999999999,"5min":1.966,"15min":1.966},"last":1.3759999999999999},{"interface":"front","average":{"1min":1.472,"5min":1.5509999999999999,"15min":1.5309999999999999},"min":{"1min":1.321,"5min":1.022,"15min":0.69499999999999995},"max":{"1min":1.748,"5min":2.0369999999999999,"15min":2.278},"last":1.613}]},{"osd":5,"last update":"Wed Aug 9 15:07:52 2023","interfaces":[{"interface":"back","average":{"1min":1.5800000000000001,"5min":1.54,"15min":1.538},"min":{"1min":1.276,"5min":0.98299999999999998,"15min":0.73499999999999999},"max":{"1min":1.9710000000000001,"5min":1.9950000000000001,"15min":2.3180000000000001},"last":1.264},{"interface":"front","average":{"1min":1.5169999999999999,"5min":1.5449999999999999,"15min":1.512},"min":{"1min":1.3480000000000001,"5min":1.0860000000000001,"15min":0.67100000000000004},"max":{"1min":1.73,"5min":2.3740000000000001,"15min":2.3740000000000001},"last":1.302}]},{"osd":6,"last update":"Wed Aug 9 15:07:52 2023","interfaces":[{"interface":"back","average":{"1min":1.579,"5min":1.6579999999999999,"15min":1.647},"min":{"1min":1.413,"5min":1.2989999999999999,"15min":0.82199999999999995},"max":{"1min":1.7749999999999999,"5min":2.1320000000000001,"15min":2.3239999999999998},"last":1.71},{"interface":"front","average":{"1min":1.6799999999999999,"5min":1.667,"15min":1.633},"min":{"1min":1.4550000000000001,"5min":1.159,"15min":0.79600000000000004},"max":{"1min":1.8500000000000001,"5min":2.1859999999999999,"15min":2.2839999999999998},"last":1.3999999999999999}]},{"osd":7,"last update":"Wed Aug 9 15:07:52 2023","interfaces":[{"interface":"back","average":{"1min":1.704,"5min":1.712,"15min":1.6990000000000001},"min":{"1min":1.4590000000000001,"5min":1.244,"15min":0.89900000000000002},"max":{"1min":1.98,"5min":2.319,"15min":2.4140000000000001},"last":1.4570000000000001},{"interface":"front","average":{"1min":1.635,"5min":1.6739999999999999,"15min":1.631},"min":{"1min":1.288,"5min":1.131,"15min":0.71699999999999997},"max":{"1min":2.1139999999999999,"5min":2.3039999999999998,"15min":2.3759999999999999},"last":1.4330000000000001}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":24,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":22,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":24,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":24,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":694,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":24,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":194,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":24,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":2194,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":24,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":24,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":24,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":104,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":25,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":25,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":25,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":25,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":25,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":25,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":25,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":25,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-08-09T15:08:50.637 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-08-09T15:08:50.898 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:50.897+0000 7fcb6e171640 1 Processor -- start 2023-08-09T15:08:50.898 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:50.897+0000 7fcb6e171640 1 -- start start 2023-08-09T15:08:50.899 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:50.899+0000 7fcb6e171640 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7fcb6806b200 0x7fcb6806b690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T15:08:50.899 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:50.899+0000 7fcb6e171640 1 -- --> v1:172.21.15.43:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fcb6815b7a0 con 0x7fcb6813e190 2023-08-09T15:08:50.900 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:50.899+0000 7fcb6e171640 1 -- --> v1:172.21.15.203:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fcb6813dd00 con 0x7fcb6815e7d0 2023-08-09T15:08:50.900 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:50.899+0000 7fcb6e171640 1 -- --> v2:172.21.15.203:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fcb6806bbd0 con 0x7fcb6806b200 2023-08-09T15:08:50.900 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:50.899+0000 7fcb67fff640 1 --1- >> v1:172.21.15.43:6789/0 conn(0x7fcb6813e190 0x7fcb6813e5c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.43:6789/0 says I am v1:172.21.15.43:42160/0 (socket says 172.21.15.43:42160) 2023-08-09T15:08:50.900 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:50.899+0000 7fcb67fff640 1 -- 172.21.15.43:0/3741915938 learned_addr learned my addr 172.21.15.43:0/3741915938 (peer_addr_for_me v1:172.21.15.43:0/0) 2023-08-09T15:08:50.901 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:50.899+0000 7fcb5effd640 1 --2- 172.21.15.43:0/3741915938 >> v2:172.21.15.203:3300/0 conn(0x7fcb6806b200 0x7fcb6806b690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T15:08:50.901 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:50.900+0000 7fcb66ffd640 1 -- 172.21.15.43:0/3741915938 <== mon.0 v1:172.21.15.43:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2299195240 0 0) 0x7fcb6815b7a0 con 0x7fcb6813e190 2023-08-09T15:08:50.902 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:50.900+0000 7fcb66ffd640 1 -- 172.21.15.43:0/3741915938 --> v1:172.21.15.43:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fcb54003520 con 0x7fcb6813e190 2023-08-09T15:08:50.902 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:50.900+0000 7fcb66ffd640 1 -- 172.21.15.43:0/3741915938 <== mon.0 v1:172.21.15.43:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1529952689 0 0) 0x7fcb54003520 con 0x7fcb6813e190 2023-08-09T15:08:50.902 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:50.900+0000 7fcb66ffd640 1 -- 172.21.15.43:0/3741915938 >> v2:172.21.15.203:3300/0 conn(0x7fcb6806b200 msgr2=0x7fcb6806b690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-09T15:08:50.902 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:50.900+0000 7fcb66ffd640 1 --2- 172.21.15.43:0/3741915938 >> v2:172.21.15.203:3300/0 conn(0x7fcb6806b200 0x7fcb6806b690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T15:08:50.902 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:50.900+0000 7fcb66ffd640 1 -- 172.21.15.43:0/3741915938 >> v1:172.21.15.203:6789/0 conn(0x7fcb6815e7d0 legacy=0x7fcb6815a7b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T15:08:50.902 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:50.900+0000 7fcb66ffd640 1 -- 172.21.15.43:0/3741915938 --> v1:172.21.15.43:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcb6815b1e0 con 0x7fcb6813e190 2023-08-09T15:08:50.902 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:50.900+0000 7fcb66ffd640 1 -- 172.21.15.43:0/3741915938 <== mon.0 v1:172.21.15.43:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1285150740 0 0) 0x7fcb60001670 con 0x7fcb6813e190 2023-08-09T15:08:50.903 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:50.900+0000 7fcb66ffd640 1 -- 172.21.15.43:0/3741915938 >> v1:172.21.15.43:6789/0 conn(0x7fcb6813e190 legacy=0x7fcb6813e5c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T15:08:50.903 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:50.900+0000 7fcb66ffd640 1 --2- 172.21.15.43:0/3741915938 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fcb54003970 0x7fcb54003d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T15:08:50.903 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:50.900+0000 7fcb66ffd640 1 --2- 172.21.15.43:0/3741915938 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fcb540047c0 0x7fcb54004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T15:08:50.903 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:50.900+0000 7fcb66ffd640 1 --2- 172.21.15.43:0/3741915938 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fcb54005180 0x7fcb5400b980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T15:08:50.903 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:50.900+0000 7fcb66ffd640 1 -- 172.21.15.43:0/3741915938 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcb6806bbd0 con 0x7fcb540047c0 2023-08-09T15:08:50.903 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:50.900+0000 7fcb66ffd640 1 -- 172.21.15.43:0/3741915938 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcb5400c120 con 0x7fcb54005180 2023-08-09T15:08:50.903 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:50.900+0000 7fcb66ffd640 1 -- 172.21.15.43:0/3741915938 --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcb5400c2e0 con 0x7fcb54003970 2023-08-09T15:08:50.904 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:50.901+0000 7fcb5effd640 1 --2- 172.21.15.43:0/3741915938 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fcb540047c0 0x7fcb54004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T15:08:50.904 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:50.901+0000 7fcb5effd640 1 -- 172.21.15.43:0/3741915938 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fcb54003970 msgr2=0x7fcb54003d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-09T15:08:50.904 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:50.901+0000 7fcb5effd640 1 --2- 172.21.15.43:0/3741915938 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fcb54003970 0x7fcb54003d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T15:08:50.904 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:50.901+0000 7fcb5effd640 1 -- 172.21.15.43:0/3741915938 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fcb54005180 msgr2=0x7fcb5400b980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-09T15:08:50.904 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:50.901+0000 7fcb5effd640 1 --2- 172.21.15.43:0/3741915938 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fcb54005180 0x7fcb5400b980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T15:08:50.904 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:50.901+0000 7fcb5effd640 1 -- 172.21.15.43:0/3741915938 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcb54003520 con 0x7fcb540047c0 2023-08-09T15:08:50.904 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:50.901+0000 7fcb67fff640 1 -- 172.21.15.43:0/3741915938 reap_dead start 2023-08-09T15:08:50.905 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:50.902+0000 7fcb5effd640 1 --2- 172.21.15.43:0/3741915938 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fcb540047c0 0x7fcb54004c40 secure :-1 s=READY pgs=2231 cs=0 l=1 rev1=1 crypto rx=0x7fcb50009fc0 tx=0x7fcb540057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6f88f6cbd3e3759f server_cookie=0 in_seq=0 out_seq=0 2023-08-09T15:08:50.906 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:50.902+0000 7fcb66ffd640 1 -- 172.21.15.43:0/3741915938 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcb5002cc20 con 0x7fcb540047c0 2023-08-09T15:08:50.906 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:50.902+0000 7fcb66ffd640 1 -- 172.21.15.43:0/3741915938 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcb5000f040 con 0x7fcb540047c0 2023-08-09T15:08:50.906 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:50.902+0000 7fcb66ffd640 1 -- 172.21.15.43:0/3741915938 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcb50034440 con 0x7fcb540047c0 2023-08-09T15:08:50.906 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:50.903+0000 7fcb6e171640 1 -- 172.21.15.43:0/3741915938 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fcb540047c0 msgr2=0x7fcb54004c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T15:08:50.906 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:50.903+0000 7fcb6e171640 1 --2- 172.21.15.43:0/3741915938 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fcb540047c0 0x7fcb54004c40 secure :-1 s=READY pgs=2231 cs=0 l=1 rev1=1 crypto rx=0x7fcb50009fc0 tx=0x7fcb540057f0 comp rx=0 tx=0).stop 2023-08-09T15:08:50.906 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:50.903+0000 7fcb6e171640 1 -- 172.21.15.43:0/3741915938 shutdown_connections 2023-08-09T15:08:50.906 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:50.903+0000 7fcb6e171640 1 --2- 172.21.15.43:0/3741915938 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fcb540047c0 0x7fcb54004c40 unknown :-1 s=CLOSED pgs=2231 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T15:08:50.907 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:50.903+0000 7fcb6e171640 1 -- 172.21.15.43:0/3741915938 >> 172.21.15.43:0/3741915938 conn(0x7fcb6806cd70 msgr2=0x7fcb680697c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-09T15:08:50.907 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:50.903+0000 7fcb6e171640 1 -- 172.21.15.43:0/3741915938 shutdown_connections 2023-08-09T15:08:50.907 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:50.903+0000 7fcb6e171640 1 -- 172.21.15.43:0/3741915938 wait complete. 2023-08-09T15:08:50.907 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:50.905+0000 7fcb6e171640 1 Processor -- start 2023-08-09T15:08:50.907 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:50.905+0000 7fcb6e171640 1 -- start start 2023-08-09T15:08:50.909 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:50.906+0000 7fcb6e171640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fcb540047c0 0x7fcb680a7e90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T15:08:50.909 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:50.906+0000 7fcb6e171640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fcb680cdd70 0x7fcb6814fd70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T15:08:50.909 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:50.906+0000 7fcb6e171640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fcb680ce1f0 0x7fcb681502b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T15:08:50.909 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:50.906+0000 7fcb6e171640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcb680a9540 con 0x7fcb680cdd70 2023-08-09T15:08:50.909 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:50.906+0000 7fcb6e171640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcb680a84a0 con 0x7fcb540047c0 2023-08-09T15:08:50.909 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:50.906+0000 7fcb6e171640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcb680a8780 con 0x7fcb680ce1f0 2023-08-09T15:08:50.909 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:50.907+0000 7fcb5effd640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fcb680cdd70 0x7fcb6814fd70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T15:08:50.910 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:50.907+0000 7fcb5effd640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fcb680cdd70 0x7fcb6814fd70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:39902/0 (socket says 172.21.15.43:39902) 2023-08-09T15:08:50.910 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:50.907+0000 7fcb5effd640 1 -- 172.21.15.43:0/1390627598 learned_addr learned my addr 172.21.15.43:0/1390627598 (peer_addr_for_me v2:172.21.15.43:0/0) 2023-08-09T15:08:50.910 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:50.907+0000 7fcb67fff640 1 --2- 172.21.15.43:0/1390627598 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fcb680ce1f0 0x7fcb681502b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T15:08:50.910 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:50.907+0000 7fcb677fe640 1 --2- 172.21.15.43:0/1390627598 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fcb540047c0 0x7fcb680a7e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T15:08:50.910 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:50.907+0000 7fcb5effd640 1 -- 172.21.15.43:0/1390627598 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fcb680ce1f0 msgr2=0x7fcb681502b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T15:08:50.910 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:50.907+0000 7fcb5effd640 1 --2- 172.21.15.43:0/1390627598 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fcb680ce1f0 0x7fcb681502b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T15:08:50.910 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:50.907+0000 7fcb5effd640 1 -- 172.21.15.43:0/1390627598 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fcb540047c0 msgr2=0x7fcb680a7e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T15:08:50.910 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:50.907+0000 7fcb5effd640 1 --2- 172.21.15.43:0/1390627598 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fcb540047c0 0x7fcb680a7e90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T15:08:50.911 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:50.907+0000 7fcb5effd640 1 -- 172.21.15.43:0/1390627598 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcb680a8a60 con 0x7fcb680cdd70 2023-08-09T15:08:50.911 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:50.907+0000 7fcb677fe640 1 --2- 172.21.15.43:0/1390627598 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fcb540047c0 0x7fcb680a7e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-08-09T15:08:50.911 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:50.907+0000 7fcb5effd640 1 --2- 172.21.15.43:0/1390627598 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fcb680cdd70 0x7fcb6814fd70 secure :-1 s=READY pgs=2232 cs=0 l=1 rev1=1 crypto rx=0x7fcb5000e230 tx=0x7fcb50009750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-09T15:08:50.911 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:50.908+0000 7fcb64ff9640 1 -- 172.21.15.43:0/1390627598 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcb50034c50 con 0x7fcb680cdd70 2023-08-09T15:08:50.911 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:50.908+0000 7fcb64ff9640 1 -- 172.21.15.43:0/1390627598 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcb50034db0 con 0x7fcb680cdd70 2023-08-09T15:08:50.911 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:50.908+0000 7fcb6e171640 1 -- 172.21.15.43:0/1390627598 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcb680a8d40 con 0x7fcb680cdd70 2023-08-09T15:08:50.911 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:50.908+0000 7fcb64ff9640 1 -- 172.21.15.43:0/1390627598 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcb50003bb0 con 0x7fcb680cdd70 2023-08-09T15:08:50.912 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:50.908+0000 7fcb6e171640 1 -- 172.21.15.43:0/1390627598 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcb680aa980 con 0x7fcb680cdd70 2023-08-09T15:08:50.912 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:50.909+0000 7fcb6e171640 1 -- 172.21.15.43:0/1390627598 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcb680739f0 con 0x7fcb680cdd70 2023-08-09T15:08:50.913 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:50.910+0000 7fcb64ff9640 1 -- 172.21.15.43:0/1390627598 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 106) v1 ==== 88443+0+0 (secure 0 0 0) 0x7fcb50040450 con 0x7fcb680cdd70 2023-08-09T15:08:50.913 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:50.911+0000 7fcb64ff9640 1 --2- 172.21.15.43:0/1390627598 >> [v2:172.21.15.203:6832/3082732769,v1:172.21.15.203:6833/3082732769] conn(0x7fcb4806a170 0x7fcb4806c630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T15:08:50.913 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:50.911+0000 7fcb64ff9640 1 -- 172.21.15.43:0/1390627598 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(205..205 src has 1..205) v4 ==== 6426+0+0 (secure 0 0 0) 0x7fcb500ab1c0 con 0x7fcb680cdd70 2023-08-09T15:08:50.913 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:50.912+0000 7fcb677fe640 1 --2- 172.21.15.43:0/1390627598 >> [v2:172.21.15.203:6832/3082732769,v1:172.21.15.203:6833/3082732769] conn(0x7fcb4806a170 0x7fcb4806c630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T15:08:50.913 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:50.913+0000 7fcb677fe640 1 --2- 172.21.15.43:0/1390627598 >> [v2:172.21.15.203:6832/3082732769,v1:172.21.15.203:6833/3082732769] conn(0x7fcb4806a170 0x7fcb4806c630 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7fcb6804f440 tx=0x7fcb58009040 comp rx=0 tx=0).ready entity=mgr.8500 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-09T15:08:50.915 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:50.915+0000 7fcb64ff9640 1 -- 172.21.15.43:0/1390627598 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178066 (secure 0 0 0) 0x7fcb50076590 con 0x7fcb680cdd70 2023-08-09T15:08:51.105 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:51.105+0000 7fcb6e171640 1 -- 172.21.15.43:0/1390627598 --> [v2:172.21.15.203:6832/3082732769,v1:172.21.15.203:6833/3082732769] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7fcb6813d990 con 0x7fcb4806a170 2023-08-09T15:08:51.129 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:51.129+0000 7fcb64ff9640 1 -- 172.21.15.43:0/1390627598 <== mgr.8500 v2:172.21.15.203:6832/3082732769 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+308709 (secure 0 0 0) 0x7fcb58001c60 con 0x7fcb4806a170 2023-08-09T15:08:51.131 INFO:teuthology.orchestra.run.smithi043.stdout: 2023-08-09T15:08:51.135 INFO:teuthology.orchestra.run.smithi043.stderr:dumped all 2023-08-09T15:08:51.136 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:51.136+0000 7fcb6e171640 1 -- 172.21.15.43:0/1390627598 >> [v2:172.21.15.203:6832/3082732769,v1:172.21.15.203:6833/3082732769] conn(0x7fcb4806a170 msgr2=0x7fcb4806c630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T15:08:51.136 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:51.136+0000 7fcb6e171640 1 --2- 172.21.15.43:0/1390627598 >> [v2:172.21.15.203:6832/3082732769,v1:172.21.15.203:6833/3082732769] conn(0x7fcb4806a170 0x7fcb4806c630 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7fcb6804f440 tx=0x7fcb58009040 comp rx=0 tx=0).stop 2023-08-09T15:08:51.137 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:51.136+0000 7fcb6e171640 1 -- 172.21.15.43:0/1390627598 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fcb680cdd70 msgr2=0x7fcb6814fd70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T15:08:51.137 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:51.136+0000 7fcb6e171640 1 --2- 172.21.15.43:0/1390627598 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fcb680cdd70 0x7fcb6814fd70 secure :-1 s=READY pgs=2232 cs=0 l=1 rev1=1 crypto rx=0x7fcb5000e230 tx=0x7fcb50009750 comp rx=0 tx=0).stop 2023-08-09T15:08:51.137 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:51.136+0000 7fcb6e171640 1 -- 172.21.15.43:0/1390627598 shutdown_connections 2023-08-09T15:08:51.137 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:51.136+0000 7fcb6e171640 1 --2- 172.21.15.43:0/1390627598 >> [v2:172.21.15.203:6832/3082732769,v1:172.21.15.203:6833/3082732769] conn(0x7fcb4806a170 0x7fcb4806c630 unknown :-1 s=CLOSED pgs=67 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T15:08:51.138 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:51.136+0000 7fcb6e171640 1 --2- 172.21.15.43:0/1390627598 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fcb680ce1f0 0x7fcb681502b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T15:08:51.138 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:51.136+0000 7fcb6e171640 1 --2- 172.21.15.43:0/1390627598 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fcb680cdd70 0x7fcb6814fd70 unknown :-1 s=CLOSED pgs=2232 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T15:08:51.138 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:51.136+0000 7fcb6e171640 1 --2- 172.21.15.43:0/1390627598 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fcb540047c0 0x7fcb680a7e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T15:08:51.139 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:51.136+0000 7fcb6e171640 1 -- 172.21.15.43:0/1390627598 >> 172.21.15.43:0/1390627598 conn(0x7fcb6806cd70 msgr2=0x7fcb68150e50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-09T15:08:51.139 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:51.137+0000 7fcb6e171640 1 -- 172.21.15.43:0/1390627598 shutdown_connections 2023-08-09T15:08:51.139 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:51.137+0000 7fcb6e171640 1 -- 172.21.15.43:0/1390627598 wait complete. 2023-08-09T15:08:51.157 INFO:teuthology.orchestra.run.smithi043.stdout:{"pg_ready":true,"pg_map":{"version":27,"stamp":"2023-08-09T15:08:49.621614+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":23,"num_object_clones":0,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":0,"num_read":1771,"num_read_kb":3684,"num_write":547,"num_write_kb":2430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":345,"ondisk_log_size":345,"up":210,"acting":210,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":99,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":398776,"kb_used_data":21068,"kb_used_omap":13,"kb_used_meta":377586,"kb_avail":754575944,"statfs":{"total":773094113280,"available":772685766656,"internally_reserved":0,"allocated":21573632,"data_stored":12257246,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":14248,"internal_metadata":386648152},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":30,"apply_latency_ms":30,"commit_latency_ns":30000000,"apply_latency_ns":30000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"5.568343"},"pg_stats":[{"pgid":"25.3f","version":"0'0","reported_seq":9,"reported_epoch":204,"state":"active+clean","last_fresh":"2023-08-09T15:08:46.151623+0000","last_change":"2023-08-09T15:08:46.150997+0000","last_active":"2023-08-09T15:08:46.151623+0000","last_peered":"2023-08-09T15:08:46.151623+0000","last_clean":"2023-08-09T15:08:46.151623+0000","last_became_active":"2023-08-09T15:08:46.150707+0000","last_became_peered":"2023-08-09T15:08:46.150707+0000","last_unstale":"2023-08-09T15:08:46.151623+0000","last_undegraded":"2023-08-09T15:08:46.151623+0000","last_fullsized":"2023-08-09T15:08:46.151623+0000","mapping_epoch":203,"log_start":"0'0","ondisk_log_start":"0'0","created":203,"last_epoch_clean":204,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T15:08:45.049361+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T15:08:45.049361+0000","last_clean_scrub_stamp":"2023-08-09T15:08:45.049361+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T23:34:12.228290+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"25.3e","version":"0'0","reported_seq":9,"reported_epoch":204,"state":"active+clean","last_fresh":"2023-08-09T15:08:46.140267+0000","last_change":"2023-08-09T15:08:46.139233+0000","last_active":"2023-08-09T15:08:46.140267+0000","last_peered":"2023-08-09T15:08:46.140267+0000","last_clean":"2023-08-09T15:08:46.140267+0000","last_became_active":"2023-08-09T15:08:46.138915+0000","last_became_peered":"2023-08-09T15:08:46.138915+0000","last_unstale":"2023-08-09T15:08:46.140267+0000","last_undegraded":"2023-08-09T15:08:46.140267+0000","last_fullsized":"2023-08-09T15:08:46.140267+0000","mapping_epoch":203,"log_start":"0'0","ondisk_log_start":"0'0","created":203,"last_epoch_clean":204,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T15:08:45.049361+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T15:08:45.049361+0000","last_clean_scrub_stamp":"2023-08-09T15:08:45.049361+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T15:13:34.907877+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"25.3d","version":"0'0","reported_seq":9,"reported_epoch":204,"state":"active+clean","last_fresh":"2023-08-09T15:08:46.142938+0000","last_change":"2023-08-09T15:08:46.141564+0000","last_active":"2023-08-09T15:08:46.142938+0000","last_peered":"2023-08-09T15:08:46.142938+0000","last_clean":"2023-08-09T15:08:46.142938+0000","last_became_active":"2023-08-09T15:08:46.139612+0000","last_became_peered":"2023-08-09T15:08:46.139612+0000","last_unstale":"2023-08-09T15:08:46.142938+0000","last_undegraded":"2023-08-09T15:08:46.142938+0000","last_fullsized":"2023-08-09T15:08:46.142938+0000","mapping_epoch":203,"log_start":"0'0","ondisk_log_start":"0'0","created":203,"last_epoch_clean":204,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T15:08:45.049361+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T15:08:45.049361+0000","last_clean_scrub_stamp":"2023-08-09T15:08:45.049361+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T15:23:43.221053+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"25.3c","version":"0'0","reported_seq":9,"reported_epoch":204,"state":"active+clean","last_fresh":"2023-08-09T15:08:46.139835+0000","last_change":"2023-08-09T15:08:46.138550+0000","last_active":"2023-08-09T15:08:46.139835+0000","last_peered":"2023-08-09T15:08:46.139835+0000","last_clean":"2023-08-09T15:08:46.139835+0000","last_became_active":"2023-08-09T15:08:46.138191+0000","last_became_peered":"2023-08-09T15:08:46.138191+0000","last_unstale":"2023-08-09T15:08:46.139835+0000","last_undegraded":"2023-08-09T15:08:46.139835+0000","last_fullsized":"2023-08-09T15:08:46.139835+0000","mapping_epoch":203,"log_start":"0'0","ondisk_log_start":"0'0","created":203,"last_epoch_clean":204,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T15:08:45.049361+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T15:08:45.049361+0000","last_clean_scrub_stamp":"2023-08-09T15:08:45.049361+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T22:35:34.627875+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"25.3b","version":"0'0","reported_seq":9,"reported_epoch":204,"state":"active+clean","last_fresh":"2023-08-09T15:08:46.149607+0000","last_change":"2023-08-09T15:08:46.139068+0000","last_active":"2023-08-09T15:08:46.149607+0000","last_peered":"2023-08-09T15:08:46.149607+0000","last_clean":"2023-08-09T15:08:46.149607+0000","last_became_active":"2023-08-09T15:08:46.138724+0000","last_became_peered":"2023-08-09T15:08:46.138724+0000","last_unstale":"2023-08-09T15:08:46.149607+0000","last_undegraded":"2023-08-09T15:08:46.149607+0000","last_fullsized":"2023-08-09T15:08:46.149607+0000","mapping_epoch":203,"log_start":"0'0","ondisk_log_start":"0'0","created":203,"last_epoch_clean":204,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T15:08:45.049361+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T15:08:45.049361+0000","last_clean_scrub_stamp":"2023-08-09T15:08:45.049361+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T22:31:17.210241+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"25.3a","version":"0'0","reported_seq":9,"reported_epoch":204,"state":"active+clean","last_fresh":"2023-08-09T15:08:46.142865+0000","last_change":"2023-08-09T15:08:46.133362+0000","last_active":"2023-08-09T15:08:46.142865+0000","last_peered":"2023-08-09T15:08:46.142865+0000","last_clean":"2023-08-09T15:08:46.142865+0000","last_became_active":"2023-08-09T15:08:46.133068+0000","last_became_peered":"2023-08-09T15:08:46.133068+0000","last_unstale":"2023-08-09T15:08:46.142865+0000","last_undegraded":"2023-08-09T15:08:46.142865+0000","last_fullsized":"2023-08-09T15:08:46.142865+0000","mapping_epoch":203,"log_start":"0'0","ondisk_log_start":"0'0","created":203,"last_epoch_clean":204,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T15:08:45.049361+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T15:08:45.049361+0000","last_clean_scrub_stamp":"2023-08-09T15:08:45.049361+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T22:33:34.990604+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"25.39","version":"0'0","reported_seq":9,"reported_epoch":204,"state":"active+clean","last_fresh":"2023-08-09T15:08:46.132517+0000","last_change":"2023-08-09T15:08:46.126835+0000","last_active":"2023-08-09T15:08:46.132517+0000","last_peered":"2023-08-09T15:08:46.132517+0000","last_clean":"2023-08-09T15:08:46.132517+0000","last_became_active":"2023-08-09T15:08:46.126531+0000","last_became_peered":"2023-08-09T15:08:46.126531+0000","last_unstale":"2023-08-09T15:08:46.132517+0000","last_undegraded":"2023-08-09T15:08:46.132517+0000","last_fullsized":"2023-08-09T15:08:46.132517+0000","mapping_epoch":203,"log_start":"0'0","ondisk_log_start":"0'0","created":203,"last_epoch_clean":204,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T15:08:45.049361+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T15:08:45.049361+0000","last_clean_scrub_stamp":"2023-08-09T15:08:45.049361+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T17:53:57.019159+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"25.38","version":"0'0","reported_seq":9,"reported_epoch":204,"state":"active+clean","last_fresh":"2023-08-09T15:08:46.135716+0000","last_change":"2023-08-09T15:08:46.122273+0000","last_active":"2023-08-09T15:08:46.135716+0000","last_peered":"2023-08-09T15:08:46.135716+0000","last_clean":"2023-08-09T15:08:46.135716+0000","last_became_active":"2023-08-09T15:08:46.121879+0000","last_became_peered":"2023-08-09T15:08:46.121879+0000","last_unstale":"2023-08-09T15:08:46.135716+0000","last_undegraded":"2023-08-09T15:08:46.135716+0000","last_fullsized":"2023-08-09T15:08:46.135716+0000","mapping_epoch":203,"log_start":"0'0","ondisk_log_start":"0'0","created":203,"last_epoch_clean":204,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T15:08:45.049361+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T15:08:45.049361+0000","last_clean_scrub_stamp":"2023-08-09T15:08:45.049361+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T17:56:00.843251+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"25.37","version":"0'0","reported_seq":9,"reported_epoch":204,"state":"active+clean","last_fresh":"2023-08-09T15:08:46.130710+0000","last_change":"2023-08-09T15:08:46.129914+0000","last_active":"2023-08-09T15:08:46.130710+0000","last_peered":"2023-08-09T15:08:46.130710+0000","last_clean":"2023-08-09T15:08:46.130710+0000","last_became_active":"2023-08-09T15:08:46.129632+0000","last_became_peered":"2023-08-09T15:08:46.129632+0000","last_unstale":"2023-08-09T15:08:46.130710+0000","last_undegraded":"2023-08-09T15:08:46.130710+0000","last_fullsized":"2023-08-09T15:08:46.130710+0000","mapping_epoch":203,"log_start":"0'0","ondisk_log_start":"0'0","created":203,"last_epoch_clean":204,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T15:08:45.049361+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T15:08:45.049361+0000","last_clean_scrub_stamp":"2023-08-09T15:08:45.049361+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T17:38:53.025570+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"25.36","version":"0'0","reported_seq":9,"reported_epoch":204,"state":"active+clean","last_fresh":"2023-08-09T15:08:46.128715+0000","last_change":"2023-08-09T15:08:46.125176+0000","last_active":"2023-08-09T15:08:46.128715+0000","last_peered":"2023-08-09T15:08:46.128715+0000","last_clean":"2023-08-09T15:08:46.128715+0000","last_became_active":"2023-08-09T15:08:46.124622+0000","last_became_peered":"2023-08-09T15:08:46.124622+0000","last_unstale":"2023-08-09T15:08:46.128715+0000","last_undegraded":"2023-08-09T15:08:46.128715+0000","last_fullsized":"2023-08-09T15:08:46.128715+0000","mapping_epoch":203,"log_start":"0'0","ondisk_log_start":"0'0","created":203,"last_epoch_clean":204,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T15:08:45.049361+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T15:08:45.049361+0000","last_clean_scrub_stamp":"2023-08-09T15:08:45.049361+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-11T00:10:28.556439+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"25.35","version":"0'0","reported_seq":9,"reported_epoch":204,"state":"active+clean","last_fresh":"2023-08-09T15:08:46.139319+0000","last_change":"2023-08-09T15:08:46.137191+0000","last_active":"2023-08-09T15:08:46.139319+0000","last_peered":"2023-08-09T15:08:46.139319+0000","last_clean":"2023-08-09T15:08:46.139319+0000","last_became_active":"2023-08-09T15:08:46.136655+0000","last_became_peered":"2023-08-09T15:08:46.136655+0000","last_unstale":"2023-08-09T15:08:46.139319+0000","last_undegraded":"2023-08-09T15:08:46.139319+0000","last_fullsized":"2023-08-09T15:08:46.139319+0000","mapping_epoch":203,"log_start":"0'0","ondisk_log_start":"0'0","created":203,"last_epoch_clean":204,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T15:08:45.049361+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T15:08:45.049361+0000","last_clean_scrub_stamp":"2023-08-09T15:08:45.049361+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-11T02:09:17.233487+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"25.34","version":"0'0","reported_seq":9,"reported_epoch":204,"state":"active+clean","last_fresh":"2023-08-09T15:08:46.135115+0000","last_change":"2023-08-09T15:08:46.133622+0000","last_active":"2023-08-09T15:08:46.135115+0000","last_peered":"2023-08-09T15:08:46.135115+0000","last_clean":"2023-08-09T15:08:46.135115+0000","last_became_active":"2023-08-09T15:08:46.133275+0000","last_became_peered":"2023-08-09T15:08:46.133275+0000","last_unstale":"2023-08-09T15:08:46.135115+0000","last_undegraded":"2023-08-09T15:08:46.135115+0000","last_fullsized":"2023-08-09T15:08:46.135115+0000","mapping_epoch":203,"log_start":"0'0","ondisk_log_start":"0'0","created":203,"last_epoch_clean":204,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T15:08:45.049361+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T15:08:45.049361+0000","last_clean_scrub_stamp":"2023-08-09T15:08:45.049361+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-11T02:31:31.905005+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"25.33","version":"0'0","reported_seq":9,"reported_epoch":204,"state":"active+clean","last_fresh":"2023-08-09T15:08:46.135012+0000","last_change":"2023-08-09T15:08:46.131224+0000","last_active":"2023-08-09T15:08:46.135012+0000","last_peered":"2023-08-09T15:08:46.135012+0000","last_clean":"2023-08-09T15:08:46.135012+0000","last_became_active":"2023-08-09T15:08:46.130861+0000","last_became_peered":"2023-08-09T15:08:46.130861+0000","last_unstale":"2023-08-09T15:08:46.135012+0000","last_undegraded":"2023-08-09T15:08:46.135012+0000","last_fullsized":"2023-08-09T15:08:46.135012+0000","mapping_epoch":203,"log_start":"0'0","ondisk_log_start":"0'0","created":203,"last_epoch_clean":204,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T15:08:45.049361+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T15:08:45.049361+0000","last_clean_scrub_stamp":"2023-08-09T15:08:45.049361+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-11T00:12:28.783608+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"25.32","version":"0'0","reported_seq":9,"reported_epoch":204,"state":"active+clean","last_fresh":"2023-08-09T15:08:46.135540+0000","last_change":"2023-08-09T15:08:46.132431+0000","last_active":"2023-08-09T15:08:46.135540+0000","last_peered":"2023-08-09T15:08:46.135540+0000","last_clean":"2023-08-09T15:08:46.135540+0000","last_became_active":"2023-08-09T15:08:46.132176+0000","last_became_peered":"2023-08-09T15:08:46.132176+0000","last_unstale":"2023-08-09T15:08:46.135540+0000","last_undegraded":"2023-08-09T15:08:46.135540+0000","last_fullsized":"2023-08-09T15:08:46.135540+0000","mapping_epoch":203,"log_start":"0'0","ondisk_log_start":"0'0","created":203,"last_epoch_clean":204,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T15:08:45.049361+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T15:08:45.049361+0000","last_clean_scrub_stamp":"2023-08-09T15:08:45.049361+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T19:53:04.905618+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"25.31","version":"0'0","reported_seq":9,"reported_epoch":204,"state":"active+clean","last_fresh":"2023-08-09T15:08:46.135898+0000","last_change":"2023-08-09T15:08:46.134472+0000","last_active":"2023-08-09T15:08:46.135898+0000","last_peered":"2023-08-09T15:08:46.135898+0000","last_clean":"2023-08-09T15:08:46.135898+0000","last_became_active":"2023-08-09T15:08:46.134023+0000","last_became_peered":"2023-08-09T15:08:46.134023+0000","last_unstale":"2023-08-09T15:08:46.135898+0000","last_undegraded":"2023-08-09T15:08:46.135898+0000","last_fullsized":"2023-08-09T15:08:46.135898+0000","mapping_epoch":203,"log_start":"0'0","ondisk_log_start":"0'0","created":203,"last_epoch_clean":204,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T15:08:45.049361+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T15:08:45.049361+0000","last_clean_scrub_stamp":"2023-08-09T15:08:45.049361+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T23:06:50.796980+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"25.30","version":"0'0","reported_seq":9,"reported_epoch":204,"state":"active+clean","last_fresh":"2023-08-09T15:08:46.136595+0000","last_change":"2023-08-09T15:08:46.132274+0000","last_active":"2023-08-09T15:08:46.136595+0000","last_peered":"2023-08-09T15:08:46.136595+0000","last_clean":"2023-08-09T15:08:46.136595+0000","last_became_active":"2023-08-09T15:08:46.131966+0000","last_became_peered":"2023-08-09T15:08:46.131966+0000","last_unstale":"2023-08-09T15:08:46.136595+0000","last_undegraded":"2023-08-09T15:08:46.136595+0000","last_fullsized":"2023-08-09T15:08:46.136595+0000","mapping_epoch":203,"log_start":"0'0","ondisk_log_start":"0'0","created":203,"last_epoch_clean":204,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T15:08:45.049361+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T15:08:45.049361+0000","last_clean_scrub_stamp":"2023-08-09T15:08:45.049361+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T18:39:53.638768+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"25.2f","version":"0'0","reported_seq":9,"reported_epoch":204,"state":"active+clean","last_fresh":"2023-08-09T15:08:46.148093+0000","last_change":"2023-08-09T15:08:46.139165+0000","last_active":"2023-08-09T15:08:46.148093+0000","last_peered":"2023-08-09T15:08:46.148093+0000","last_clean":"2023-08-09T15:08:46.148093+0000","last_became_active":"2023-08-09T15:08:46.138796+0000","last_became_peered":"2023-08-09T15:08:46.138796+0000","last_unstale":"2023-08-09T15:08:46.148093+0000","last_undegraded":"2023-08-09T15:08:46.148093+0000","last_fullsized":"2023-08-09T15:08:46.148093+0000","mapping_epoch":203,"log_start":"0'0","ondisk_log_start":"0'0","created":203,"last_epoch_clean":204,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T15:08:45.049361+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T15:08:45.049361+0000","last_clean_scrub_stamp":"2023-08-09T15:08:45.049361+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-11T03:04:07.233708+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"25.2e","version":"0'0","reported_seq":9,"reported_epoch":204,"state":"active+clean","last_fresh":"2023-08-09T15:08:46.140492+0000","last_change":"2023-08-09T15:08:46.138485+0000","last_active":"2023-08-09T15:08:46.140492+0000","last_peered":"2023-08-09T15:08:46.140492+0000","last_clean":"2023-08-09T15:08:46.140492+0000","last_became_active":"2023-08-09T15:08:46.138217+0000","last_became_peered":"2023-08-09T15:08:46.138217+0000","last_unstale":"2023-08-09T15:08:46.140492+0000","last_undegraded":"2023-08-09T15:08:46.140492+0000","last_fullsized":"2023-08-09T15:08:46.140492+0000","mapping_epoch":203,"log_start":"0'0","ondisk_log_start":"0'0","created":203,"last_epoch_clean":204,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T15:08:45.049361+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T15:08:45.049361+0000","last_clean_scrub_stamp":"2023-08-09T15:08:45.049361+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T16:48:46.531516+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"25.2d","version":"0'0","reported_seq":9,"reported_epoch":204,"state":"active+clean","last_fresh":"2023-08-09T15:08:46.149742+0000","last_change":"2023-08-09T15:08:46.141111+0000","last_active":"2023-08-09T15:08:46.149742+0000","last_peered":"2023-08-09T15:08:46.149742+0000","last_clean":"2023-08-09T15:08:46.149742+0000","last_became_active":"2023-08-09T15:08:46.137325+0000","last_became_peered":"2023-08-09T15:08:46.137325+0000","last_unstale":"2023-08-09T15:08:46.149742+0000","last_undegraded":"2023-08-09T15:08:46.149742+0000","last_fullsized":"2023-08-09T15:08:46.149742+0000","mapping_epoch":203,"log_start":"0'0","ondisk_log_start":"0'0","created":203,"last_epoch_clean":204,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T15:08:45.049361+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T15:08:45.049361+0000","last_clean_scrub_stamp":"2023-08-09T15:08:45.049361+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T16:08:16.971247+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"25.2c","version":"0'0","reported_seq":9,"reported_epoch":204,"state":"active+clean","last_fresh":"2023-08-09T15:08:46.147332+0000","last_change":"2023-08-09T15:08:46.138291+0000","last_active":"2023-08-09T15:08:46.147332+0000","last_peered":"2023-08-09T15:08:46.147332+0000","last_clean":"2023-08-09T15:08:46.147332+0000","last_became_active":"2023-08-09T15:08:46.133273+0000","last_became_peered":"2023-08-09T15:08:46.133273+0000","last_unstale":"2023-08-09T15:08:46.147332+0000","last_undegraded":"2023-08-09T15:08:46.147332+0000","last_fullsized":"2023-08-09T15:08:46.147332+0000","mapping_epoch":203,"log_start":"0'0","ondisk_log_start":"0'0","created":203,"last_epoch_clean":204,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T15:08:45.049361+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T15:08:45.049361+0000","last_clean_scrub_stamp":"2023-08-09T15:08:45.049361+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T23:54:29.242692+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"25.2b","version":"0'0","reported_seq":9,"reported_epoch":204,"state":"active+clean","last_fresh":"2023-08-09T15:08:46.145468+0000","last_change":"2023-08-09T15:08:46.135809+0000","last_active":"2023-08-09T15:08:46.145468+0000","last_peered":"2023-08-09T15:08:46.145468+0000","last_clean":"2023-08-09T15:08:46.145468+0000","last_became_active":"2023-08-09T15:08:46.135429+0000","last_became_peered":"2023-08-09T15:08:46.135429+0000","last_unstale":"2023-08-09T15:08:46.145468+0000","last_undegraded":"2023-08-09T15:08:46.145468+0000","last_fullsized":"2023-08-09T15:08:46.145468+0000","mapping_epoch":203,"log_start":"0'0","ondisk_log_start":"0'0","created":203,"last_epoch_clean":204,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T15:08:45.049361+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T15:08:45.049361+0000","last_clean_scrub_stamp":"2023-08-09T15:08:45.049361+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-11T01:46:34.819975+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"25.2a","version":"0'0","reported_seq":9,"reported_epoch":204,"state":"active+clean","last_fresh":"2023-08-09T15:08:46.149972+0000","last_change":"2023-08-09T15:08:46.144130+0000","last_active":"2023-08-09T15:08:46.149972+0000","last_peered":"2023-08-09T15:08:46.149972+0000","last_clean":"2023-08-09T15:08:46.149972+0000","last_became_active":"2023-08-09T15:08:46.142345+0000","last_became_peered":"2023-08-09T15:08:46.142345+0000","last_unstale":"2023-08-09T15:08:46.149972+0000","last_undegraded":"2023-08-09T15:08:46.149972+0000","last_fullsized":"2023-08-09T15:08:46.149972+0000","mapping_epoch":203,"log_start":"0'0","ondisk_log_start":"0'0","created":203,"last_epoch_clean":204,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T15:08:45.049361+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T15:08:45.049361+0000","last_clean_scrub_stamp":"2023-08-09T15:08:45.049361+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T21:32:52.411688+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"25.29","version":"0'0","reported_seq":9,"reported_epoch":204,"state":"active+clean","last_fresh":"2023-08-09T15:08:46.136326+0000","last_change":"2023-08-09T15:08:46.125725+0000","last_active":"2023-08-09T15:08:46.136326+0000","last_peered":"2023-08-09T15:08:46.136326+0000","last_clean":"2023-08-09T15:08:46.136326+0000","last_became_active":"2023-08-09T15:08:46.125456+0000","last_became_peered":"2023-08-09T15:08:46.125456+0000","last_unstale":"2023-08-09T15:08:46.136326+0000","last_undegraded":"2023-08-09T15:08:46.136326+0000","last_fullsized":"2023-08-09T15:08:46.136326+0000","mapping_epoch":203,"log_start":"0'0","ondisk_log_start":"0'0","created":203,"last_epoch_clean":204,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T15:08:45.049361+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T15:08:45.049361+0000","last_clean_scrub_stamp":"2023-08-09T15:08:45.049361+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T19:10:37.660853+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"25.28","version":"0'0","reported_seq":9,"reported_epoch":204,"state":"active+clean","last_fresh":"2023-08-09T15:08:46.136571+0000","last_change":"2023-08-09T15:08:46.129604+0000","last_active":"2023-08-09T15:08:46.136571+0000","last_peered":"2023-08-09T15:08:46.136571+0000","last_clean":"2023-08-09T15:08:46.136571+0000","last_became_active":"2023-08-09T15:08:46.129287+0000","last_became_peered":"2023-08-09T15:08:46.129287+0000","last_unstale":"2023-08-09T15:08:46.136571+0000","last_undegraded":"2023-08-09T15:08:46.136571+0000","last_fullsized":"2023-08-09T15:08:46.136571+0000","mapping_epoch":203,"log_start":"0'0","ondisk_log_start":"0'0","created":203,"last_epoch_clean":204,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T15:08:45.049361+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T15:08:45.049361+0000","last_clean_scrub_stamp":"2023-08-09T15:08:45.049361+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T15:43:17.545121+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"25.27","version":"0'0","reported_seq":9,"reported_epoch":204,"state":"active+clean","last_fresh":"2023-08-09T15:08:46.142873+0000","last_change":"2023-08-09T15:08:46.141818+0000","last_active":"2023-08-09T15:08:46.142873+0000","last_peered":"2023-08-09T15:08:46.142873+0000","last_clean":"2023-08-09T15:08:46.142873+0000","last_became_active":"2023-08-09T15:08:46.141512+0000","last_became_peered":"2023-08-09T15:08:46.141512+0000","last_unstale":"2023-08-09T15:08:46.142873+0000","last_undegraded":"2023-08-09T15:08:46.142873+0000","last_fullsized":"2023-08-09T15:08:46.142873+0000","mapping_epoch":203,"log_start":"0'0","ondisk_log_start":"0'0","created":203,"last_epoch_clean":204,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T15:08:45.049361+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T15:08:45.049361+0000","last_clean_scrub_stamp":"2023-08-09T15:08:45.049361+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-11T00:17:53.301522+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"25.26","version":"0'0","reported_seq":9,"reported_epoch":204,"state":"active+clean","last_fresh":"2023-08-09T15:08:46.129717+0000","last_change":"2023-08-09T15:08:46.123803+0000","last_active":"2023-08-09T15:08:46.129717+0000","last_peered":"2023-08-09T15:08:46.129717+0000","last_clean":"2023-08-09T15:08:46.129717+0000","last_became_active":"2023-08-09T15:08:46.123453+0000","last_became_peered":"2023-08-09T15:08:46.123453+0000","last_unstale":"2023-08-09T15:08:46.129717+0000","last_undegraded":"2023-08-09T15:08:46.129717+0000","last_fullsized":"2023-08-09T15:08:46.129717+0000","mapping_epoch":203,"log_start":"0'0","ondisk_log_start":"0'0","created":203,"last_epoch_clean":204,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T15:08:45.049361+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T15:08:45.049361+0000","last_clean_scrub_stamp":"2023-08-09T15:08:45.049361+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T22:44:05.779076+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"25.25","version":"0'0","reported_seq":9,"reported_epoch":204,"state":"active+clean","last_fresh":"2023-08-09T15:08:46.142920+0000","last_change":"2023-08-09T15:08:46.141330+0000","last_active":"2023-08-09T15:08:46.142920+0000","last_peered":"2023-08-09T15:08:46.142920+0000","last_clean":"2023-08-09T15:08:46.142920+0000","last_became_active":"2023-08-09T15:08:46.141065+0000","last_became_peered":"2023-08-09T15:08:46.141065+0000","last_unstale":"2023-08-09T15:08:46.142920+0000","last_undegraded":"2023-08-09T15:08:46.142920+0000","last_fullsized":"2023-08-09T15:08:46.142920+0000","mapping_epoch":203,"log_start":"0'0","ondisk_log_start":"0'0","created":203,"last_epoch_clean":204,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T15:08:45.049361+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T15:08:45.049361+0000","last_clean_scrub_stamp":"2023-08-09T15:08:45.049361+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-11T02:56:50.712559+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"25.24","version":"0'0","reported_seq":9,"reported_epoch":204,"state":"active+clean","last_fresh":"2023-08-09T15:08:46.148941+0000","last_change":"2023-08-09T15:08:46.140614+0000","last_active":"2023-08-09T15:08:46.148941+0000","last_peered":"2023-08-09T15:08:46.148941+0000","last_clean":"2023-08-09T15:08:46.148941+0000","last_became_active":"2023-08-09T15:08:46.138328+0000","last_became_peered":"2023-08-09T15:08:46.138328+0000","last_unstale":"2023-08-09T15:08:46.148941+0000","last_undegraded":"2023-08-09T15:08:46.148941+0000","last_fullsized":"2023-08-09T15:08:46.148941+0000","mapping_epoch":203,"log_start":"0'0","ondisk_log_start":"0'0","created":203,"last_epoch_clean":204,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T15:08:45.049361+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T15:08:45.049361+0000","last_clean_scrub_stamp":"2023-08-09T15:08:45.049361+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T22:39:17.933987+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"25.23","version":"0'0","reported_seq":9,"reported_epoch":204,"state":"active+clean","last_fresh":"2023-08-09T15:08:46.135127+0000","last_change":"2023-08-09T15:08:46.133633+0000","last_active":"2023-08-09T15:08:46.135127+0000","last_peered":"2023-08-09T15:08:46.135127+0000","last_clean":"2023-08-09T15:08:46.135127+0000","last_became_active":"2023-08-09T15:08:46.133330+0000","last_became_peered":"2023-08-09T15:08:46.133330+0000","last_unstale":"2023-08-09T15:08:46.135127+0000","last_undegraded":"2023-08-09T15:08:46.135127+0000","last_fullsized":"2023-08-09T15:08:46.135127+0000","mapping_epoch":203,"log_start":"0'0","ondisk_log_start":"0'0","created":203,"last_epoch_clean":204,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T15:08:45.049361+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T15:08:45.049361+0000","last_clean_scrub_stamp":"2023-08-09T15:08:45.049361+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T20:38:09.810823+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"25.22","version":"0'0","reported_seq":9,"reported_epoch":204,"state":"active+clean","last_fresh":"2023-08-09T15:08:46.135520+0000","last_change":"2023-08-09T15:08:46.131940+0000","last_active":"2023-08-09T15:08:46.135520+0000","last_peered":"2023-08-09T15:08:46.135520+0000","last_clean":"2023-08-09T15:08:46.135520+0000","last_became_active":"2023-08-09T15:08:46.131576+0000","last_became_peered":"2023-08-09T15:08:46.131576+0000","last_unstale":"2023-08-09T15:08:46.135520+0000","last_undegraded":"2023-08-09T15:08:46.135520+0000","last_fullsized":"2023-08-09T15:08:46.135520+0000","mapping_epoch":203,"log_start":"0'0","ondisk_log_start":"0'0","created":203,"last_epoch_clean":204,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T15:08:45.049361+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T15:08:45.049361+0000","last_clean_scrub_stamp":"2023-08-09T15:08:45.049361+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-11T01:07:15.792938+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"25.21","version":"0'0","reported_seq":9,"reported_epoch":204,"state":"active+clean","last_fresh":"2023-08-09T15:08:46.136279+0000","last_change":"2023-08-09T15:08:46.125173+0000","last_active":"2023-08-09T15:08:46.136279+0000","last_peered":"2023-08-09T15:08:46.136279+0000","last_clean":"2023-08-09T15:08:46.136279+0000","last_became_active":"2023-08-09T15:08:46.124880+0000","last_became_peered":"2023-08-09T15:08:46.124880+0000","last_unstale":"2023-08-09T15:08:46.136279+0000","last_undegraded":"2023-08-09T15:08:46.136279+0000","last_fullsized":"2023-08-09T15:08:46.136279+0000","mapping_epoch":203,"log_start":"0'0","ondisk_log_start":"0'0","created":203,"last_epoch_clean":204,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T15:08:45.049361+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T15:08:45.049361+0000","last_clean_scrub_stamp":"2023-08-09T15:08:45.049361+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T20:32:20.754257+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"25.20","version":"0'0","reported_seq":9,"reported_epoch":204,"state":"active+clean","last_fresh":"2023-08-09T15:08:46.144251+0000","last_change":"2023-08-09T15:08:46.134105+0000","last_active":"2023-08-09T15:08:46.144251+0000","last_peered":"2023-08-09T15:08:46.144251+0000","last_clean":"2023-08-09T15:08:46.144251+0000","last_became_active":"2023-08-09T15:08:46.133781+0000","last_became_peered":"2023-08-09T15:08:46.133781+0000","last_unstale":"2023-08-09T15:08:46.144251+0000","last_undegraded":"2023-08-09T15:08:46.144251+0000","last_fullsized":"2023-08-09T15:08:46.144251+0000","mapping_epoch":203,"log_start":"0'0","ondisk_log_start":"0'0","created":203,"last_epoch_clean":204,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T15:08:45.049361+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T15:08:45.049361+0000","last_clean_scrub_stamp":"2023-08-09T15:08:45.049361+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T21:54:42.038300+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"25.16","version":"0'0","reported_seq":9,"reported_epoch":204,"state":"active+clean","last_fresh":"2023-08-09T15:08:46.136443+0000","last_change":"2023-08-09T15:08:46.128258+0000","last_active":"2023-08-09T15:08:46.136443+0000","last_peered":"2023-08-09T15:08:46.136443+0000","last_clean":"2023-08-09T15:08:46.136443+0000","last_became_active":"2023-08-09T15:08:46.127605+0000","last_became_peered":"2023-08-09T15:08:46.127605+0000","last_unstale":"2023-08-09T15:08:46.136443+0000","last_undegraded":"2023-08-09T15:08:46.136443+0000","last_fullsized":"2023-08-09T15:08:46.136443+0000","mapping_epoch":203,"log_start":"0'0","ondisk_log_start":"0'0","created":203,"last_epoch_clean":204,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T15:08:45.049361+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T15:08:45.049361+0000","last_clean_scrub_stamp":"2023-08-09T15:08:45.049361+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-11T02:45:10.800604+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"24.17","version":"0'0","reported_seq":11,"reported_epoch":204,"state":"active+clean","last_fresh":"2023-08-09T15:08:46.104628+0000","last_change":"2023-08-09T15:08:45.088142+0000","last_active":"2023-08-09T15:08:46.104628+0000","last_peered":"2023-08-09T15:08:46.104628+0000","last_clean":"2023-08-09T15:08:46.104628+0000","last_became_active":"2023-08-09T15:08:45.087867+0000","last_became_peered":"2023-08-09T15:08:45.087867+0000","last_unstale":"2023-08-09T15:08:46.104628+0000","last_undegraded":"2023-08-09T15:08:46.104628+0000","last_fullsized":"2023-08-09T15:08:46.104628+0000","mapping_epoch":202,"log_start":"0'0","ondisk_log_start":"0'0","created":202,"last_epoch_clean":203,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T15:08:44.039630+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T15:08:44.039630+0000","last_clean_scrub_stamp":"2023-08-09T15:08:44.039630+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T17:42:15.287480+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"25.17","version":"0'0","reported_seq":9,"reported_epoch":204,"state":"active+clean","last_fresh":"2023-08-09T15:08:46.135975+0000","last_change":"2023-08-09T15:08:46.128585+0000","last_active":"2023-08-09T15:08:46.135975+0000","last_peered":"2023-08-09T15:08:46.135975+0000","last_clean":"2023-08-09T15:08:46.135975+0000","last_became_active":"2023-08-09T15:08:46.128272+0000","last_became_peered":"2023-08-09T15:08:46.128272+0000","last_unstale":"2023-08-09T15:08:46.135975+0000","last_undegraded":"2023-08-09T15:08:46.135975+0000","last_fullsized":"2023-08-09T15:08:46.135975+0000","mapping_epoch":203,"log_start":"0'0","ondisk_log_start":"0'0","created":203,"last_epoch_clean":204,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T15:08:45.049361+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T15:08:45.049361+0000","last_clean_scrub_stamp":"2023-08-09T15:08:45.049361+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-11T01:10:22.933001+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"24.16","version":"0'0","reported_seq":11,"reported_epoch":204,"state":"active+clean","last_fresh":"2023-08-09T15:08:46.117412+0000","last_change":"2023-08-09T15:08:45.093806+0000","last_active":"2023-08-09T15:08:46.117412+0000","last_peered":"2023-08-09T15:08:46.117412+0000","last_clean":"2023-08-09T15:08:46.117412+0000","last_became_active":"2023-08-09T15:08:45.093473+0000","last_became_peered":"2023-08-09T15:08:45.093473+0000","last_unstale":"2023-08-09T15:08:46.117412+0000","last_undegraded":"2023-08-09T15:08:46.117412+0000","last_fullsized":"2023-08-09T15:08:46.117412+0000","mapping_epoch":202,"log_start":"0'0","ondisk_log_start":"0'0","created":202,"last_epoch_clean":203,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T15:08:44.039630+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T15:08:44.039630+0000","last_clean_scrub_stamp":"2023-08-09T15:08:44.039630+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T20:43:13.822110+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"25.14","version":"0'0","reported_seq":9,"reported_epoch":204,"state":"active+clean","last_fresh":"2023-08-09T15:08:46.130621+0000","last_change":"2023-08-09T15:08:46.126770+0000","last_active":"2023-08-09T15:08:46.130621+0000","last_peered":"2023-08-09T15:08:46.130621+0000","last_clean":"2023-08-09T15:08:46.130621+0000","last_became_active":"2023-08-09T15:08:46.126465+0000","last_became_peered":"2023-08-09T15:08:46.126465+0000","last_unstale":"2023-08-09T15:08:46.130621+0000","last_undegraded":"2023-08-09T15:08:46.130621+0000","last_fullsized":"2023-08-09T15:08:46.130621+0000","mapping_epoch":203,"log_start":"0'0","ondisk_log_start":"0'0","created":203,"last_epoch_clean":204,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T15:08:45.049361+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T15:08:45.049361+0000","last_clean_scrub_stamp":"2023-08-09T15:08:45.049361+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T21:17:05.616400+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"24.15","version":"204'3","reported_seq":14,"reported_epoch":204,"state":"active+clean","last_fresh":"2023-08-09T15:08:46.155929+0000","last_change":"2023-08-09T15:08:45.095626+0000","last_active":"2023-08-09T15:08:46.155929+0000","last_peered":"2023-08-09T15:08:46.155929+0000","last_clean":"2023-08-09T15:08:46.155929+0000","last_became_active":"2023-08-09T15:08:45.095242+0000","last_became_peered":"2023-08-09T15:08:45.095242+0000","last_unstale":"2023-08-09T15:08:46.155929+0000","last_undegraded":"2023-08-09T15:08:46.155929+0000","last_fullsized":"2023-08-09T15:08:46.155929+0000","mapping_epoch":202,"log_start":"0'0","ondisk_log_start":"0'0","created":202,"last_epoch_clean":203,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T15:08:44.039630+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T15:08:44.039630+0000","last_clean_scrub_stamp":"2023-08-09T15:08:44.039630+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-11T00:50:08.710962+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"25.15","version":"0'0","reported_seq":9,"reported_epoch":204,"state":"active+clean","last_fresh":"2023-08-09T15:08:46.149763+0000","last_change":"2023-08-09T15:08:46.141740+0000","last_active":"2023-08-09T15:08:46.149763+0000","last_peered":"2023-08-09T15:08:46.149763+0000","last_clean":"2023-08-09T15:08:46.149763+0000","last_became_active":"2023-08-09T15:08:46.141428+0000","last_became_peered":"2023-08-09T15:08:46.141428+0000","last_unstale":"2023-08-09T15:08:46.149763+0000","last_undegraded":"2023-08-09T15:08:46.149763+0000","last_fullsized":"2023-08-09T15:08:46.149763+0000","mapping_epoch":203,"log_start":"0'0","ondisk_log_start":"0'0","created":203,"last_epoch_clean":204,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T15:08:45.049361+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T15:08:45.049361+0000","last_clean_scrub_stamp":"2023-08-09T15:08:45.049361+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T17:25:01.501655+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"24.14","version":"204'6","reported_seq":17,"reported_epoch":204,"state":"active+clean","last_fresh":"2023-08-09T15:08:46.157247+0000","last_change":"2023-08-09T15:08:45.089903+0000","last_active":"2023-08-09T15:08:46.157247+0000","last_peered":"2023-08-09T15:08:46.157247+0000","last_clean":"2023-08-09T15:08:46.157247+0000","last_became_active":"2023-08-09T15:08:45.089610+0000","last_became_peered":"2023-08-09T15:08:45.089610+0000","last_unstale":"2023-08-09T15:08:46.157247+0000","last_undegraded":"2023-08-09T15:08:46.157247+0000","last_fullsized":"2023-08-09T15:08:46.157247+0000","mapping_epoch":202,"log_start":"0'0","ondisk_log_start":"0'0","created":202,"last_epoch_clean":203,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T15:08:44.039630+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T15:08:44.039630+0000","last_clean_scrub_stamp":"2023-08-09T15:08:44.039630+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T17:16:30.418309+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"25.12","version":"0'0","reported_seq":9,"reported_epoch":204,"state":"active+clean","last_fresh":"2023-08-09T15:08:46.137120+0000","last_change":"2023-08-09T15:08:46.127062+0000","last_active":"2023-08-09T15:08:46.137120+0000","last_peered":"2023-08-09T15:08:46.137120+0000","last_clean":"2023-08-09T15:08:46.137120+0000","last_became_active":"2023-08-09T15:08:46.126722+0000","last_became_peered":"2023-08-09T15:08:46.126722+0000","last_unstale":"2023-08-09T15:08:46.137120+0000","last_undegraded":"2023-08-09T15:08:46.137120+0000","last_fullsized":"2023-08-09T15:08:46.137120+0000","mapping_epoch":203,"log_start":"0'0","ondisk_log_start":"0'0","created":203,"last_epoch_clean":204,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T15:08:45.049361+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T15:08:45.049361+0000","last_clean_scrub_stamp":"2023-08-09T15:08:45.049361+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-11T01:16:19.078625+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"24.13","version":"204'2","reported_seq":13,"reported_epoch":204,"state":"active+clean","last_fresh":"2023-08-09T15:08:46.156865+0000","last_change":"2023-08-09T15:08:45.093138+0000","last_active":"2023-08-09T15:08:46.156865+0000","last_peered":"2023-08-09T15:08:46.156865+0000","last_clean":"2023-08-09T15:08:46.156865+0000","last_became_active":"2023-08-09T15:08:45.092792+0000","last_became_peered":"2023-08-09T15:08:45.092792+0000","last_unstale":"2023-08-09T15:08:46.156865+0000","last_undegraded":"2023-08-09T15:08:46.156865+0000","last_fullsized":"2023-08-09T15:08:46.156865+0000","mapping_epoch":202,"log_start":"0'0","ondisk_log_start":"0'0","created":202,"last_epoch_clean":203,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T15:08:44.039630+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T15:08:44.039630+0000","last_clean_scrub_stamp":"2023-08-09T15:08:44.039630+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T15:11:33.035314+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"25.13","version":"0'0","reported_seq":9,"reported_epoch":204,"state":"active+clean","last_fresh":"2023-08-09T15:08:46.132112+0000","last_change":"2023-08-09T15:08:46.125769+0000","last_active":"2023-08-09T15:08:46.132112+0000","last_peered":"2023-08-09T15:08:46.132112+0000","last_clean":"2023-08-09T15:08:46.132112+0000","last_became_active":"2023-08-09T15:08:46.125483+0000","last_became_peered":"2023-08-09T15:08:46.125483+0000","last_unstale":"2023-08-09T15:08:46.132112+0000","last_undegraded":"2023-08-09T15:08:46.132112+0000","last_fullsized":"2023-08-09T15:08:46.132112+0000","mapping_epoch":203,"log_start":"0'0","ondisk_log_start":"0'0","created":203,"last_epoch_clean":204,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T15:08:45.049361+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T15:08:45.049361+0000","last_clean_scrub_stamp":"2023-08-09T15:08:45.049361+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-11T01:40:22.044849+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"24.12","version":"0'0","reported_seq":11,"reported_epoch":204,"state":"active+clean","last_fresh":"2023-08-09T15:08:46.120634+0000","last_change":"2023-08-09T15:08:45.095964+0000","last_active":"2023-08-09T15:08:46.120634+0000","last_peered":"2023-08-09T15:08:46.120634+0000","last_clean":"2023-08-09T15:08:46.120634+0000","last_became_active":"2023-08-09T15:08:45.095675+0000","last_became_peered":"2023-08-09T15:08:45.095675+0000","last_unstale":"2023-08-09T15:08:46.120634+0000","last_undegraded":"2023-08-09T15:08:46.120634+0000","last_fullsized":"2023-08-09T15:08:46.120634+0000","mapping_epoch":202,"log_start":"0'0","ondisk_log_start":"0'0","created":202,"last_epoch_clean":203,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T15:08:44.039630+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T15:08:44.039630+0000","last_clean_scrub_stamp":"2023-08-09T15:08:44.039630+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-11T03:01:48.069708+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"25.10","version":"0'0","reported_seq":9,"reported_epoch":204,"state":"active+clean","last_fresh":"2023-08-09T15:08:46.130212+0000","last_change":"2023-08-09T15:08:46.129212+0000","last_active":"2023-08-09T15:08:46.130212+0000","last_peered":"2023-08-09T15:08:46.130212+0000","last_clean":"2023-08-09T15:08:46.130212+0000","last_became_active":"2023-08-09T15:08:46.128930+0000","last_became_peered":"2023-08-09T15:08:46.128930+0000","last_unstale":"2023-08-09T15:08:46.130212+0000","last_undegraded":"2023-08-09T15:08:46.130212+0000","last_fullsized":"2023-08-09T15:08:46.130212+0000","mapping_epoch":203,"log_start":"0'0","ondisk_log_start":"0'0","created":203,"last_epoch_clean":204,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T15:08:45.049361+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T15:08:45.049361+0000","last_clean_scrub_stamp":"2023-08-09T15:08:45.049361+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T16:30:28.142764+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"24.11","version":"0'0","reported_seq":11,"reported_epoch":204,"state":"active+clean","last_fresh":"2023-08-09T15:08:46.105202+0000","last_change":"2023-08-09T15:08:45.091703+0000","last_active":"2023-08-09T15:08:46.105202+0000","last_peered":"2023-08-09T15:08:46.105202+0000","last_clean":"2023-08-09T15:08:46.105202+0000","last_became_active":"2023-08-09T15:08:45.091437+0000","last_became_peered":"2023-08-09T15:08:45.091437+0000","last_unstale":"2023-08-09T15:08:46.105202+0000","last_undegraded":"2023-08-09T15:08:46.105202+0000","last_fullsized":"2023-08-09T15:08:46.105202+0000","mapping_epoch":202,"log_start":"0'0","ondisk_log_start":"0'0","created":202,"last_epoch_clean":203,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T15:08:44.039630+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T15:08:44.039630+0000","last_clean_scrub_stamp":"2023-08-09T15:08:44.039630+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-11T00:38:14.601055+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"25.11","version":"0'0","reported_seq":9,"reported_epoch":204,"state":"active+clean","last_fresh":"2023-08-09T15:08:46.147109+0000","last_change":"2023-08-09T15:08:46.135169+0000","last_active":"2023-08-09T15:08:46.147109+0000","last_peered":"2023-08-09T15:08:46.147109+0000","last_clean":"2023-08-09T15:08:46.147109+0000","last_became_active":"2023-08-09T15:08:46.134688+0000","last_became_peered":"2023-08-09T15:08:46.134688+0000","last_unstale":"2023-08-09T15:08:46.147109+0000","last_undegraded":"2023-08-09T15:08:46.147109+0000","last_fullsized":"2023-08-09T15:08:46.147109+0000","mapping_epoch":203,"log_start":"0'0","ondisk_log_start":"0'0","created":203,"last_epoch_clean":204,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T15:08:45.049361+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T15:08:45.049361+0000","last_clean_scrub_stamp":"2023-08-09T15:08:45.049361+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-11T00:19:57.173009+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"24.10","version":"204'8","reported_seq":19,"reported_epoch":204,"state":"active+clean","last_fresh":"2023-08-09T15:08:46.156956+0000","last_change":"2023-08-09T15:08:45.086979+0000","last_active":"2023-08-09T15:08:46.156956+0000","last_peered":"2023-08-09T15:08:46.156956+0000","last_clean":"2023-08-09T15:08:46.156956+0000","last_became_active":"2023-08-09T15:08:45.086321+0000","last_became_peered":"2023-08-09T15:08:45.086321+0000","last_unstale":"2023-08-09T15:08:46.156956+0000","last_undegraded":"2023-08-09T15:08:46.156956+0000","last_fullsized":"2023-08-09T15:08:46.156956+0000","mapping_epoch":202,"log_start":"0'0","ondisk_log_start":"0'0","created":202,"last_epoch_clean":203,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T15:08:44.039630+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T15:08:44.039630+0000","last_clean_scrub_stamp":"2023-08-09T15:08:44.039630+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-11T01:50:54.696216+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"25.e","version":"0'0","reported_seq":9,"reported_epoch":204,"state":"active+clean","last_fresh":"2023-08-09T15:08:46.146607+0000","last_change":"2023-08-09T15:08:46.134264+0000","last_active":"2023-08-09T15:08:46.146607+0000","last_peered":"2023-08-09T15:08:46.146607+0000","last_clean":"2023-08-09T15:08:46.146607+0000","last_became_active":"2023-08-09T15:08:46.133953+0000","last_became_peered":"2023-08-09T15:08:46.133953+0000","last_unstale":"2023-08-09T15:08:46.146607+0000","last_undegraded":"2023-08-09T15:08:46.146607+0000","last_fullsized":"2023-08-09T15:08:46.146607+0000","mapping_epoch":203,"log_start":"0'0","ondisk_log_start":"0'0","created":203,"last_epoch_clean":204,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T15:08:45.049361+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T15:08:45.049361+0000","last_clean_scrub_stamp":"2023-08-09T15:08:45.049361+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-11T02:04:24.281163+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"24.f","version":"204'1","reported_seq":12,"reported_epoch":204,"state":"active+clean","last_fresh":"2023-08-09T15:08:46.152505+0000","last_change":"2023-08-09T15:08:45.098360+0000","last_active":"2023-08-09T15:08:46.152505+0000","last_peered":"2023-08-09T15:08:46.152505+0000","last_clean":"2023-08-09T15:08:46.152505+0000","last_became_active":"2023-08-09T15:08:45.098110+0000","last_became_peered":"2023-08-09T15:08:45.098110+0000","last_unstale":"2023-08-09T15:08:46.152505+0000","last_undegraded":"2023-08-09T15:08:46.152505+0000","last_fullsized":"2023-08-09T15:08:46.152505+0000","mapping_epoch":202,"log_start":"0'0","ondisk_log_start":"0'0","created":202,"last_epoch_clean":203,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T15:08:44.039630+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T15:08:44.039630+0000","last_clean_scrub_stamp":"2023-08-09T15:08:44.039630+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-11T02:46:55.641275+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"25.f","version":"0'0","reported_seq":9,"reported_epoch":204,"state":"active+clean","last_fresh":"2023-08-09T15:08:46.141110+0000","last_change":"2023-08-09T15:08:46.137534+0000","last_active":"2023-08-09T15:08:46.141110+0000","last_peered":"2023-08-09T15:08:46.141110+0000","last_clean":"2023-08-09T15:08:46.141110+0000","last_became_active":"2023-08-09T15:08:46.137099+0000","last_became_peered":"2023-08-09T15:08:46.137099+0000","last_unstale":"2023-08-09T15:08:46.141110+0000","last_undegraded":"2023-08-09T15:08:46.141110+0000","last_fullsized":"2023-08-09T15:08:46.141110+0000","mapping_epoch":203,"log_start":"0'0","ondisk_log_start":"0'0","created":203,"last_epoch_clean":204,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T15:08:45.049361+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T15:08:45.049361+0000","last_clean_scrub_stamp":"2023-08-09T15:08:45.049361+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T16:49:16.683294+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"24.e","version":"204'2","reported_seq":13,"reported_epoch":204,"state":"active+clean","last_fresh":"2023-08-09T15:08:46.156808+0000","last_change":"2023-08-09T15:08:45.089498+0000","last_active":"2023-08-09T15:08:46.156808+0000","last_peered":"2023-08-09T15:08:46.156808+0000","last_clean":"2023-08-09T15:08:46.156808+0000","last_became_active":"2023-08-09T15:08:45.089176+0000","last_became_peered":"2023-08-09T15:08:45.089176+0000","last_unstale":"2023-08-09T15:08:46.156808+0000","last_undegraded":"2023-08-09T15:08:46.156808+0000","last_fullsized":"2023-08-09T15:08:46.156808+0000","mapping_epoch":202,"log_start":"0'0","ondisk_log_start":"0'0","created":202,"last_epoch_clean":203,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T15:08:44.039630+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T15:08:44.039630+0000","last_clean_scrub_stamp":"2023-08-09T15:08:44.039630+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T22:07:02.953997+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"25.c","version":"0'0","reported_seq":9,"reported_epoch":204,"state":"active+clean","last_fresh":"2023-08-09T15:08:46.131124+0000","last_change":"2023-08-09T15:08:46.128985+0000","last_active":"2023-08-09T15:08:46.131124+0000","last_peered":"2023-08-09T15:08:46.131124+0000","last_clean":"2023-08-09T15:08:46.131124+0000","last_became_active":"2023-08-09T15:08:46.128680+0000","last_became_peered":"2023-08-09T15:08:46.128680+0000","last_unstale":"2023-08-09T15:08:46.131124+0000","last_undegraded":"2023-08-09T15:08:46.131124+0000","last_fullsized":"2023-08-09T15:08:46.131124+0000","mapping_epoch":203,"log_start":"0'0","ondisk_log_start":"0'0","created":203,"last_epoch_clean":204,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T15:08:45.049361+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T15:08:45.049361+0000","last_clean_scrub_stamp":"2023-08-09T15:08:45.049361+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-11T01:49:59.788507+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"24.d","version":"204'1","reported_seq":12,"reported_epoch":204,"state":"active+clean","last_fresh":"2023-08-09T15:08:46.154048+0000","last_change":"2023-08-09T15:08:45.088107+0000","last_active":"2023-08-09T15:08:46.154048+0000","last_peered":"2023-08-09T15:08:46.154048+0000","last_clean":"2023-08-09T15:08:46.154048+0000","last_became_active":"2023-08-09T15:08:45.087782+0000","last_became_peered":"2023-08-09T15:08:45.087782+0000","last_unstale":"2023-08-09T15:08:46.154048+0000","last_undegraded":"2023-08-09T15:08:46.154048+0000","last_fullsized":"2023-08-09T15:08:46.154048+0000","mapping_epoch":202,"log_start":"0'0","ondisk_log_start":"0'0","created":202,"last_epoch_clean":203,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T15:08:44.039630+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T15:08:44.039630+0000","last_clean_scrub_stamp":"2023-08-09T15:08:44.039630+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T15:39:55.302767+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"25.d","version":"0'0","reported_seq":9,"reported_epoch":204,"state":"active+clean","last_fresh":"2023-08-09T15:08:46.139803+0000","last_change":"2023-08-09T15:08:46.138571+0000","last_active":"2023-08-09T15:08:46.139803+0000","last_peered":"2023-08-09T15:08:46.139803+0000","last_clean":"2023-08-09T15:08:46.139803+0000","last_became_active":"2023-08-09T15:08:46.138271+0000","last_became_peered":"2023-08-09T15:08:46.138271+0000","last_unstale":"2023-08-09T15:08:46.139803+0000","last_undegraded":"2023-08-09T15:08:46.139803+0000","last_fullsized":"2023-08-09T15:08:46.139803+0000","mapping_epoch":203,"log_start":"0'0","ondisk_log_start":"0'0","created":203,"last_epoch_clean":204,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T15:08:45.049361+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T15:08:45.049361+0000","last_clean_scrub_stamp":"2023-08-09T15:08:45.049361+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T16:24:39.791163+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"24.c","version":"0'0","reported_seq":11,"reported_epoch":204,"state":"active+clean","last_fresh":"2023-08-09T15:08:46.106493+0000","last_change":"2023-08-09T15:08:45.089852+0000","last_active":"2023-08-09T15:08:46.106493+0000","last_peered":"2023-08-09T15:08:46.106493+0000","last_clean":"2023-08-09T15:08:46.106493+0000","last_became_active":"2023-08-09T15:08:45.088837+0000","last_became_peered":"2023-08-09T15:08:45.088837+0000","last_unstale":"2023-08-09T15:08:46.106493+0000","last_undegraded":"2023-08-09T15:08:46.106493+0000","last_fullsized":"2023-08-09T15:08:46.106493+0000","mapping_epoch":202,"log_start":"0'0","ondisk_log_start":"0'0","created":202,"last_epoch_clean":203,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T15:08:44.039630+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T15:08:44.039630+0000","last_clean_scrub_stamp":"2023-08-09T15:08:44.039630+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T22:11:33.824164+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"25.a","version":"0'0","reported_seq":9,"reported_epoch":204,"state":"active+clean","last_fresh":"2023-08-09T15:08:46.149947+0000","last_change":"2023-08-09T15:08:46.143670+0000","last_active":"2023-08-09T15:08:46.149947+0000","last_peered":"2023-08-09T15:08:46.149947+0000","last_clean":"2023-08-09T15:08:46.149947+0000","last_became_active":"2023-08-09T15:08:46.143371+0000","last_became_peered":"2023-08-09T15:08:46.143371+0000","last_unstale":"2023-08-09T15:08:46.149947+0000","last_undegraded":"2023-08-09T15:08:46.149947+0000","last_fullsized":"2023-08-09T15:08:46.149947+0000","mapping_epoch":203,"log_start":"0'0","ondisk_log_start":"0'0","created":203,"last_epoch_clean":204,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T15:08:45.049361+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T15:08:45.049361+0000","last_clean_scrub_stamp":"2023-08-09T15:08:45.049361+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T23:25:10.934767+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"24.b","version":"204'2","reported_seq":13,"reported_epoch":204,"state":"active+clean","last_fresh":"2023-08-09T15:08:46.153823+0000","last_change":"2023-08-09T15:08:45.099141+0000","last_active":"2023-08-09T15:08:46.153823+0000","last_peered":"2023-08-09T15:08:46.153823+0000","last_clean":"2023-08-09T15:08:46.153823+0000","last_became_active":"2023-08-09T15:08:45.098823+0000","last_became_peered":"2023-08-09T15:08:45.098823+0000","last_unstale":"2023-08-09T15:08:46.153823+0000","last_undegraded":"2023-08-09T15:08:46.153823+0000","last_fullsized":"2023-08-09T15:08:46.153823+0000","mapping_epoch":202,"log_start":"0'0","ondisk_log_start":"0'0","created":202,"last_epoch_clean":203,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T15:08:44.039630+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T15:08:44.039630+0000","last_clean_scrub_stamp":"2023-08-09T15:08:44.039630+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T23:52:19.241728+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"25.b","version":"0'0","reported_seq":9,"reported_epoch":204,"state":"active+clean","last_fresh":"2023-08-09T15:08:46.132090+0000","last_change":"2023-08-09T15:08:46.124313+0000","last_active":"2023-08-09T15:08:46.132090+0000","last_peered":"2023-08-09T15:08:46.132090+0000","last_clean":"2023-08-09T15:08:46.132090+0000","last_became_active":"2023-08-09T15:08:46.124031+0000","last_became_peered":"2023-08-09T15:08:46.124031+0000","last_unstale":"2023-08-09T15:08:46.132090+0000","last_undegraded":"2023-08-09T15:08:46.132090+0000","last_fullsized":"2023-08-09T15:08:46.132090+0000","mapping_epoch":203,"log_start":"0'0","ondisk_log_start":"0'0","created":203,"last_epoch_clean":204,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T15:08:45.049361+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T15:08:45.049361+0000","last_clean_scrub_stamp":"2023-08-09T15:08:45.049361+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-11T02:17:00.563221+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"24.a","version":"0'0","reported_seq":11,"reported_epoch":204,"state":"active+clean","last_fresh":"2023-08-09T15:08:46.118441+0000","last_change":"2023-08-09T15:08:45.093370+0000","last_active":"2023-08-09T15:08:46.118441+0000","last_peered":"2023-08-09T15:08:46.118441+0000","last_clean":"2023-08-09T15:08:46.118441+0000","last_became_active":"2023-08-09T15:08:45.092965+0000","last_became_peered":"2023-08-09T15:08:45.092965+0000","last_unstale":"2023-08-09T15:08:46.118441+0000","last_undegraded":"2023-08-09T15:08:46.118441+0000","last_fullsized":"2023-08-09T15:08:46.118441+0000","mapping_epoch":202,"log_start":"0'0","ondisk_log_start":"0'0","created":202,"last_epoch_clean":203,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T15:08:44.039630+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T15:08:44.039630+0000","last_clean_scrub_stamp":"2023-08-09T15:08:44.039630+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T22:05:12.350996+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"25.8","version":"0'0","reported_seq":9,"reported_epoch":204,"state":"active+clean","last_fresh":"2023-08-09T15:08:46.134447+0000","last_change":"2023-08-09T15:08:46.127835+0000","last_active":"2023-08-09T15:08:46.134447+0000","last_peered":"2023-08-09T15:08:46.134447+0000","last_clean":"2023-08-09T15:08:46.134447+0000","last_became_active":"2023-08-09T15:08:46.127529+0000","last_became_peered":"2023-08-09T15:08:46.127529+0000","last_unstale":"2023-08-09T15:08:46.134447+0000","last_undegraded":"2023-08-09T15:08:46.134447+0000","last_fullsized":"2023-08-09T15:08:46.134447+0000","mapping_epoch":203,"log_start":"0'0","ondisk_log_start":"0'0","created":203,"last_epoch_clean":204,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T15:08:45.049361+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T15:08:45.049361+0000","last_clean_scrub_stamp":"2023-08-09T15:08:45.049361+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T18:45:41.505319+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"24.9","version":"0'0","reported_seq":11,"reported_epoch":204,"state":"active+clean","last_fresh":"2023-08-09T15:08:46.102736+0000","last_change":"2023-08-09T15:08:45.097893+0000","last_active":"2023-08-09T15:08:46.102736+0000","last_peered":"2023-08-09T15:08:46.102736+0000","last_clean":"2023-08-09T15:08:46.102736+0000","last_became_active":"2023-08-09T15:08:45.097618+0000","last_became_peered":"2023-08-09T15:08:45.097618+0000","last_unstale":"2023-08-09T15:08:46.102736+0000","last_undegraded":"2023-08-09T15:08:46.102736+0000","last_fullsized":"2023-08-09T15:08:46.102736+0000","mapping_epoch":202,"log_start":"0'0","ondisk_log_start":"0'0","created":202,"last_epoch_clean":203,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T15:08:44.039630+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T15:08:44.039630+0000","last_clean_scrub_stamp":"2023-08-09T15:08:44.039630+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T15:17:53.837111+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"25.9","version":"0'0","reported_seq":9,"reported_epoch":204,"state":"active+clean","last_fresh":"2023-08-09T15:08:46.140972+0000","last_change":"2023-08-09T15:08:46.133100+0000","last_active":"2023-08-09T15:08:46.140972+0000","last_peered":"2023-08-09T15:08:46.140972+0000","last_clean":"2023-08-09T15:08:46.140972+0000","last_became_active":"2023-08-09T15:08:46.132718+0000","last_became_peered":"2023-08-09T15:08:46.132718+0000","last_unstale":"2023-08-09T15:08:46.140972+0000","last_undegraded":"2023-08-09T15:08:46.140972+0000","last_fullsized":"2023-08-09T15:08:46.140972+0000","mapping_epoch":203,"log_start":"0'0","ondisk_log_start":"0'0","created":203,"last_epoch_clean":204,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T15:08:45.049361+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T15:08:45.049361+0000","last_clean_scrub_stamp":"2023-08-09T15:08:45.049361+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T16:51:47.226654+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"24.8","version":"0'0","reported_seq":11,"reported_epoch":204,"state":"active+clean","last_fresh":"2023-08-09T15:08:46.116342+0000","last_change":"2023-08-09T15:08:45.089228+0000","last_active":"2023-08-09T15:08:46.116342+0000","last_peered":"2023-08-09T15:08:46.116342+0000","last_clean":"2023-08-09T15:08:46.116342+0000","last_became_active":"2023-08-09T15:08:45.088776+0000","last_became_peered":"2023-08-09T15:08:45.088776+0000","last_unstale":"2023-08-09T15:08:46.116342+0000","last_undegraded":"2023-08-09T15:08:46.116342+0000","last_fullsized":"2023-08-09T15:08:46.116342+0000","mapping_epoch":202,"log_start":"0'0","ondisk_log_start":"0'0","created":202,"last_epoch_clean":203,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T15:08:44.039630+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T15:08:44.039630+0000","last_clean_scrub_stamp":"2023-08-09T15:08:44.039630+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T19:52:42.363999+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"25.6","version":"0'0","reported_seq":9,"reported_epoch":204,"state":"active+clean","last_fresh":"2023-08-09T15:08:46.142930+0000","last_change":"2023-08-09T15:08:46.133149+0000","last_active":"2023-08-09T15:08:46.142930+0000","last_peered":"2023-08-09T15:08:46.142930+0000","last_clean":"2023-08-09T15:08:46.142930+0000","last_became_active":"2023-08-09T15:08:46.132736+0000","last_became_peered":"2023-08-09T15:08:46.132736+0000","last_unstale":"2023-08-09T15:08:46.142930+0000","last_undegraded":"2023-08-09T15:08:46.142930+0000","last_fullsized":"2023-08-09T15:08:46.142930+0000","mapping_epoch":203,"log_start":"0'0","ondisk_log_start":"0'0","created":203,"last_epoch_clean":204,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T15:08:45.049361+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T15:08:45.049361+0000","last_clean_scrub_stamp":"2023-08-09T15:08:45.049361+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T15:33:36.716547+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"24.7","version":"204'2","reported_seq":13,"reported_epoch":204,"state":"active+clean","last_fresh":"2023-08-09T15:08:46.154036+0000","last_change":"2023-08-09T15:08:45.095073+0000","last_active":"2023-08-09T15:08:46.154036+0000","last_peered":"2023-08-09T15:08:46.154036+0000","last_clean":"2023-08-09T15:08:46.154036+0000","last_became_active":"2023-08-09T15:08:45.094734+0000","last_became_peered":"2023-08-09T15:08:45.094734+0000","last_unstale":"2023-08-09T15:08:46.154036+0000","last_undegraded":"2023-08-09T15:08:46.154036+0000","last_fullsized":"2023-08-09T15:08:46.154036+0000","mapping_epoch":202,"log_start":"0'0","ondisk_log_start":"0'0","created":202,"last_epoch_clean":203,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T15:08:44.039630+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T15:08:44.039630+0000","last_clean_scrub_stamp":"2023-08-09T15:08:44.039630+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-11T02:53:18.098726+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"25.7","version":"0'0","reported_seq":9,"reported_epoch":204,"state":"active+clean","last_fresh":"2023-08-09T15:08:46.135768+0000","last_change":"2023-08-09T15:08:46.130455+0000","last_active":"2023-08-09T15:08:46.135768+0000","last_peered":"2023-08-09T15:08:46.135768+0000","last_clean":"2023-08-09T15:08:46.135768+0000","last_became_active":"2023-08-09T15:08:46.130168+0000","last_became_peered":"2023-08-09T15:08:46.130168+0000","last_unstale":"2023-08-09T15:08:46.135768+0000","last_undegraded":"2023-08-09T15:08:46.135768+0000","last_fullsized":"2023-08-09T15:08:46.135768+0000","mapping_epoch":203,"log_start":"0'0","ondisk_log_start":"0'0","created":203,"last_epoch_clean":204,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T15:08:45.049361+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T15:08:45.049361+0000","last_clean_scrub_stamp":"2023-08-09T15:08:45.049361+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T15:58:04.851493+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"24.6","version":"0'0","reported_seq":11,"reported_epoch":204,"state":"active+clean","last_fresh":"2023-08-09T15:08:46.122642+0000","last_change":"2023-08-09T15:08:45.089606+0000","last_active":"2023-08-09T15:08:46.122642+0000","last_peered":"2023-08-09T15:08:46.122642+0000","last_clean":"2023-08-09T15:08:46.122642+0000","last_became_active":"2023-08-09T15:08:45.089344+0000","last_became_peered":"2023-08-09T15:08:45.089344+0000","last_unstale":"2023-08-09T15:08:46.122642+0000","last_undegraded":"2023-08-09T15:08:46.122642+0000","last_fullsized":"2023-08-09T15:08:46.122642+0000","mapping_epoch":202,"log_start":"0'0","ondisk_log_start":"0'0","created":202,"last_epoch_clean":203,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T15:08:44.039630+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T15:08:44.039630+0000","last_clean_scrub_stamp":"2023-08-09T15:08:44.039630+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T23:53:45.920055+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"25.4","version":"0'0","reported_seq":9,"reported_epoch":204,"state":"active+clean","last_fresh":"2023-08-09T15:08:46.136029+0000","last_change":"2023-08-09T15:08:46.127889+0000","last_active":"2023-08-09T15:08:46.136029+0000","last_peered":"2023-08-09T15:08:46.136029+0000","last_clean":"2023-08-09T15:08:46.136029+0000","last_became_active":"2023-08-09T15:08:46.127609+0000","last_became_peered":"2023-08-09T15:08:46.127609+0000","last_unstale":"2023-08-09T15:08:46.136029+0000","last_undegraded":"2023-08-09T15:08:46.136029+0000","last_fullsized":"2023-08-09T15:08:46.136029+0000","mapping_epoch":203,"log_start":"0'0","ondisk_log_start":"0'0","created":203,"last_epoch_clean":204,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T15:08:45.049361+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T15:08:45.049361+0000","last_clean_scrub_stamp":"2023-08-09T15:08:45.049361+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T22:16:43.576858+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"24.5","version":"0'0","reported_seq":11,"reported_epoch":204,"state":"active+clean","last_fresh":"2023-08-09T15:08:46.120333+0000","last_change":"2023-08-09T15:08:45.089605+0000","last_active":"2023-08-09T15:08:46.120333+0000","last_peered":"2023-08-09T15:08:46.120333+0000","last_clean":"2023-08-09T15:08:46.120333+0000","last_became_active":"2023-08-09T15:08:45.089267+0000","last_became_peered":"2023-08-09T15:08:45.089267+0000","last_unstale":"2023-08-09T15:08:46.120333+0000","last_undegraded":"2023-08-09T15:08:46.120333+0000","last_fullsized":"2023-08-09T15:08:46.120333+0000","mapping_epoch":202,"log_start":"0'0","ondisk_log_start":"0'0","created":202,"last_epoch_clean":203,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T15:08:44.039630+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T15:08:44.039630+0000","last_clean_scrub_stamp":"2023-08-09T15:08:44.039630+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T15:14:23.464887+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"25.5","version":"0'0","reported_seq":9,"reported_epoch":204,"state":"active+clean","last_fresh":"2023-08-09T15:08:46.144821+0000","last_change":"2023-08-09T15:08:46.140247+0000","last_active":"2023-08-09T15:08:46.144821+0000","last_peered":"2023-08-09T15:08:46.144821+0000","last_clean":"2023-08-09T15:08:46.144821+0000","last_became_active":"2023-08-09T15:08:46.139940+0000","last_became_peered":"2023-08-09T15:08:46.139940+0000","last_unstale":"2023-08-09T15:08:46.144821+0000","last_undegraded":"2023-08-09T15:08:46.144821+0000","last_fullsized":"2023-08-09T15:08:46.144821+0000","mapping_epoch":203,"log_start":"0'0","ondisk_log_start":"0'0","created":203,"last_epoch_clean":204,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T15:08:45.049361+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T15:08:45.049361+0000","last_clean_scrub_stamp":"2023-08-09T15:08:45.049361+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T20:28:03.166356+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"24.4","version":"0'0","reported_seq":11,"reported_epoch":204,"state":"active+clean","last_fresh":"2023-08-09T15:08:46.106030+0000","last_change":"2023-08-09T15:08:45.092363+0000","last_active":"2023-08-09T15:08:46.106030+0000","last_peered":"2023-08-09T15:08:46.106030+0000","last_clean":"2023-08-09T15:08:46.106030+0000","last_became_active":"2023-08-09T15:08:45.092079+0000","last_became_peered":"2023-08-09T15:08:45.092079+0000","last_unstale":"2023-08-09T15:08:46.106030+0000","last_undegraded":"2023-08-09T15:08:46.106030+0000","last_fullsized":"2023-08-09T15:08:46.106030+0000","mapping_epoch":202,"log_start":"0'0","ondisk_log_start":"0'0","created":202,"last_epoch_clean":203,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T15:08:44.039630+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T15:08:44.039630+0000","last_clean_scrub_stamp":"2023-08-09T15:08:44.039630+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T19:07:11.609326+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"25.2","version":"0'0","reported_seq":9,"reported_epoch":204,"state":"active+clean","last_fresh":"2023-08-09T15:08:46.141342+0000","last_change":"2023-08-09T15:08:46.139532+0000","last_active":"2023-08-09T15:08:46.141342+0000","last_peered":"2023-08-09T15:08:46.141342+0000","last_clean":"2023-08-09T15:08:46.141342+0000","last_became_active":"2023-08-09T15:08:46.139060+0000","last_became_peered":"2023-08-09T15:08:46.139060+0000","last_unstale":"2023-08-09T15:08:46.141342+0000","last_undegraded":"2023-08-09T15:08:46.141342+0000","last_fullsized":"2023-08-09T15:08:46.141342+0000","mapping_epoch":203,"log_start":"0'0","ondisk_log_start":"0'0","created":203,"last_epoch_clean":204,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T15:08:45.049361+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T15:08:45.049361+0000","last_clean_scrub_stamp":"2023-08-09T15:08:45.049361+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-11T00:56:55.799238+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"24.3","version":"0'0","reported_seq":11,"reported_epoch":204,"state":"active+clean","last_fresh":"2023-08-09T15:08:46.122226+0000","last_change":"2023-08-09T15:08:45.086089+0000","last_active":"2023-08-09T15:08:46.122226+0000","last_peered":"2023-08-09T15:08:46.122226+0000","last_clean":"2023-08-09T15:08:46.122226+0000","last_became_active":"2023-08-09T15:08:45.085719+0000","last_became_peered":"2023-08-09T15:08:45.085719+0000","last_unstale":"2023-08-09T15:08:46.122226+0000","last_undegraded":"2023-08-09T15:08:46.122226+0000","last_fullsized":"2023-08-09T15:08:46.122226+0000","mapping_epoch":202,"log_start":"0'0","ondisk_log_start":"0'0","created":202,"last_epoch_clean":203,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T15:08:44.039630+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T15:08:44.039630+0000","last_clean_scrub_stamp":"2023-08-09T15:08:44.039630+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T23:31:38.750789+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"25.3","version":"0'0","reported_seq":9,"reported_epoch":204,"state":"active+clean","last_fresh":"2023-08-09T15:08:46.135798+0000","last_change":"2023-08-09T15:08:46.133084+0000","last_active":"2023-08-09T15:08:46.135798+0000","last_peered":"2023-08-09T15:08:46.135798+0000","last_clean":"2023-08-09T15:08:46.135798+0000","last_became_active":"2023-08-09T15:08:46.132801+0000","last_became_peered":"2023-08-09T15:08:46.132801+0000","last_unstale":"2023-08-09T15:08:46.135798+0000","last_undegraded":"2023-08-09T15:08:46.135798+0000","last_fullsized":"2023-08-09T15:08:46.135798+0000","mapping_epoch":203,"log_start":"0'0","ondisk_log_start":"0'0","created":203,"last_epoch_clean":204,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T15:08:45.049361+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T15:08:45.049361+0000","last_clean_scrub_stamp":"2023-08-09T15:08:45.049361+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T22:36:09.290713+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"24.2","version":"0'0","reported_seq":11,"reported_epoch":204,"state":"active+clean","last_fresh":"2023-08-09T15:08:46.116671+0000","last_change":"2023-08-09T15:08:45.086746+0000","last_active":"2023-08-09T15:08:46.116671+0000","last_peered":"2023-08-09T15:08:46.116671+0000","last_clean":"2023-08-09T15:08:46.116671+0000","last_became_active":"2023-08-09T15:08:45.086092+0000","last_became_peered":"2023-08-09T15:08:45.086092+0000","last_unstale":"2023-08-09T15:08:46.116671+0000","last_undegraded":"2023-08-09T15:08:46.116671+0000","last_fullsized":"2023-08-09T15:08:46.116671+0000","mapping_epoch":202,"log_start":"0'0","ondisk_log_start":"0'0","created":202,"last_epoch_clean":203,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T15:08:44.039630+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T15:08:44.039630+0000","last_clean_scrub_stamp":"2023-08-09T15:08:44.039630+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-11T01:15:55.832665+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"25.0","version":"0'0","reported_seq":9,"reported_epoch":204,"state":"active+clean","last_fresh":"2023-08-09T15:08:46.134427+0000","last_change":"2023-08-09T15:08:46.127664+0000","last_active":"2023-08-09T15:08:46.134427+0000","last_peered":"2023-08-09T15:08:46.134427+0000","last_clean":"2023-08-09T15:08:46.134427+0000","last_became_active":"2023-08-09T15:08:46.127206+0000","last_became_peered":"2023-08-09T15:08:46.127206+0000","last_unstale":"2023-08-09T15:08:46.134427+0000","last_undegraded":"2023-08-09T15:08:46.134427+0000","last_fullsized":"2023-08-09T15:08:46.134427+0000","mapping_epoch":203,"log_start":"0'0","ondisk_log_start":"0'0","created":203,"last_epoch_clean":204,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T15:08:45.049361+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T15:08:45.049361+0000","last_clean_scrub_stamp":"2023-08-09T15:08:45.049361+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-11T01:04:02.365522+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"24.1","version":"0'0","reported_seq":11,"reported_epoch":204,"state":"active+clean","last_fresh":"2023-08-09T15:08:46.116754+0000","last_change":"2023-08-09T15:08:45.086500+0000","last_active":"2023-08-09T15:08:46.116754+0000","last_peered":"2023-08-09T15:08:46.116754+0000","last_clean":"2023-08-09T15:08:46.116754+0000","last_became_active":"2023-08-09T15:08:45.086062+0000","last_became_peered":"2023-08-09T15:08:45.086062+0000","last_unstale":"2023-08-09T15:08:46.116754+0000","last_undegraded":"2023-08-09T15:08:46.116754+0000","last_fullsized":"2023-08-09T15:08:46.116754+0000","mapping_epoch":202,"log_start":"0'0","ondisk_log_start":"0'0","created":202,"last_epoch_clean":203,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T15:08:44.039630+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T15:08:44.039630+0000","last_clean_scrub_stamp":"2023-08-09T15:08:44.039630+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T19:26:21.854345+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"25.1","version":"0'0","reported_seq":9,"reported_epoch":204,"state":"active+clean","last_fresh":"2023-08-09T15:08:46.145278+0000","last_change":"2023-08-09T15:08:46.133327+0000","last_active":"2023-08-09T15:08:46.145278+0000","last_peered":"2023-08-09T15:08:46.145278+0000","last_clean":"2023-08-09T15:08:46.145278+0000","last_became_active":"2023-08-09T15:08:46.127486+0000","last_became_peered":"2023-08-09T15:08:46.127486+0000","last_unstale":"2023-08-09T15:08:46.145278+0000","last_undegraded":"2023-08-09T15:08:46.145278+0000","last_fullsized":"2023-08-09T15:08:46.145278+0000","mapping_epoch":203,"log_start":"0'0","ondisk_log_start":"0'0","created":203,"last_epoch_clean":204,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T15:08:45.049361+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T15:08:45.049361+0000","last_clean_scrub_stamp":"2023-08-09T15:08:45.049361+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T19:43:08.286766+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"24.0","version":"0'0","reported_seq":11,"reported_epoch":204,"state":"active+clean","last_fresh":"2023-08-09T15:08:46.101554+0000","last_change":"2023-08-09T15:08:45.096555+0000","last_active":"2023-08-09T15:08:46.101554+0000","last_peered":"2023-08-09T15:08:46.101554+0000","last_clean":"2023-08-09T15:08:46.101554+0000","last_became_active":"2023-08-09T15:08:45.096249+0000","last_became_peered":"2023-08-09T15:08:45.096249+0000","last_unstale":"2023-08-09T15:08:46.101554+0000","last_undegraded":"2023-08-09T15:08:46.101554+0000","last_fullsized":"2023-08-09T15:08:46.101554+0000","mapping_epoch":202,"log_start":"0'0","ondisk_log_start":"0'0","created":202,"last_epoch_clean":203,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T15:08:44.039630+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T15:08:44.039630+0000","last_clean_scrub_stamp":"2023-08-09T15:08:44.039630+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T22:26:19.857459+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"25.18","version":"0'0","reported_seq":9,"reported_epoch":204,"state":"active+clean","last_fresh":"2023-08-09T15:08:46.134973+0000","last_change":"2023-08-09T15:08:46.129858+0000","last_active":"2023-08-09T15:08:46.134973+0000","last_peered":"2023-08-09T15:08:46.134973+0000","last_clean":"2023-08-09T15:08:46.134973+0000","last_became_active":"2023-08-09T15:08:46.129528+0000","last_became_peered":"2023-08-09T15:08:46.129528+0000","last_unstale":"2023-08-09T15:08:46.134973+0000","last_undegraded":"2023-08-09T15:08:46.134973+0000","last_fullsized":"2023-08-09T15:08:46.134973+0000","mapping_epoch":203,"log_start":"0'0","ondisk_log_start":"0'0","created":203,"last_epoch_clean":204,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T15:08:45.049361+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T15:08:45.049361+0000","last_clean_scrub_stamp":"2023-08-09T15:08:45.049361+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T18:26:49.778606+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"24.19","version":"0'0","reported_seq":11,"reported_epoch":204,"state":"active+clean","last_fresh":"2023-08-09T15:08:46.126776+0000","last_change":"2023-08-09T15:08:45.087792+0000","last_active":"2023-08-09T15:08:46.126776+0000","last_peered":"2023-08-09T15:08:46.126776+0000","last_clean":"2023-08-09T15:08:46.126776+0000","last_became_active":"2023-08-09T15:08:45.087519+0000","last_became_peered":"2023-08-09T15:08:45.087519+0000","last_unstale":"2023-08-09T15:08:46.126776+0000","last_undegraded":"2023-08-09T15:08:46.126776+0000","last_fullsized":"2023-08-09T15:08:46.126776+0000","mapping_epoch":202,"log_start":"0'0","ondisk_log_start":"0'0","created":202,"last_epoch_clean":203,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T15:08:44.039630+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T15:08:44.039630+0000","last_clean_scrub_stamp":"2023-08-09T15:08:44.039630+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T23:30:42.203379+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":378,"reported_epoch":204,"state":"active+clean","last_fresh":"2023-08-09T15:08:46.119414+0000","last_change":"2023-08-09T13:33:56.401406+0000","last_active":"2023-08-09T15:08:46.119414+0000","last_peered":"2023-08-09T15:08:46.119414+0000","last_clean":"2023-08-09T15:08:46.119414+0000","last_became_active":"2023-08-09T13:33:56.400487+0000","last_became_peered":"2023-08-09T13:33:56.400487+0000","last_unstale":"2023-08-09T15:08:46.119414+0000","last_undegraded":"2023-08-09T15:08:46.119414+0000","last_fullsized":"2023-08-09T15:08:46.119414+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:33:54.845869+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:33:54.845869+0000","last_clean_scrub_stamp":"2023-08-09T13:33:54.845869+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T23:09:59.457761+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"25.19","version":"0'0","reported_seq":9,"reported_epoch":204,"state":"active+clean","last_fresh":"2023-08-09T15:08:46.132535+0000","last_change":"2023-08-09T15:08:46.130530+0000","last_active":"2023-08-09T15:08:46.132535+0000","last_peered":"2023-08-09T15:08:46.132535+0000","last_clean":"2023-08-09T15:08:46.132535+0000","last_became_active":"2023-08-09T15:08:46.130226+0000","last_became_peered":"2023-08-09T15:08:46.130226+0000","last_unstale":"2023-08-09T15:08:46.132535+0000","last_undegraded":"2023-08-09T15:08:46.132535+0000","last_fullsized":"2023-08-09T15:08:46.132535+0000","mapping_epoch":203,"log_start":"0'0","ondisk_log_start":"0'0","created":203,"last_epoch_clean":204,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T15:08:45.049361+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T15:08:45.049361+0000","last_clean_scrub_stamp":"2023-08-09T15:08:45.049361+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T23:21:48.300259+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"24.18","version":"0'0","reported_seq":11,"reported_epoch":204,"state":"active+clean","last_fresh":"2023-08-09T15:08:46.113596+0000","last_change":"2023-08-09T15:08:45.087159+0000","last_active":"2023-08-09T15:08:46.113596+0000","last_peered":"2023-08-09T15:08:46.113596+0000","last_clean":"2023-08-09T15:08:46.113596+0000","last_became_active":"2023-08-09T15:08:45.086705+0000","last_became_peered":"2023-08-09T15:08:45.086705+0000","last_unstale":"2023-08-09T15:08:46.113596+0000","last_undegraded":"2023-08-09T15:08:46.113596+0000","last_fullsized":"2023-08-09T15:08:46.113596+0000","mapping_epoch":202,"log_start":"0'0","ondisk_log_start":"0'0","created":202,"last_epoch_clean":203,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T15:08:44.039630+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T15:08:44.039630+0000","last_clean_scrub_stamp":"2023-08-09T15:08:44.039630+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T15:37:22.384686+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":388,"reported_epoch":204,"state":"active+clean","last_fresh":"2023-08-09T15:08:46.118306+0000","last_change":"2023-08-09T13:33:56.401123+0000","last_active":"2023-08-09T15:08:46.118306+0000","last_peered":"2023-08-09T15:08:46.118306+0000","last_clean":"2023-08-09T15:08:46.118306+0000","last_became_active":"2023-08-09T13:33:56.399933+0000","last_became_peered":"2023-08-09T13:33:56.399933+0000","last_unstale":"2023-08-09T15:08:46.118306+0000","last_undegraded":"2023-08-09T15:08:46.118306+0000","last_fullsized":"2023-08-09T15:08:46.118306+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:33:54.845869+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:33:54.845869+0000","last_clean_scrub_stamp":"2023-08-09T13:33:54.845869+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-11T01:01:10.799631+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"25.1a","version":"0'0","reported_seq":9,"reported_epoch":204,"state":"active+clean","last_fresh":"2023-08-09T15:08:46.149920+0000","last_change":"2023-08-09T15:08:46.143064+0000","last_active":"2023-08-09T15:08:46.149920+0000","last_peered":"2023-08-09T15:08:46.149920+0000","last_clean":"2023-08-09T15:08:46.149920+0000","last_became_active":"2023-08-09T15:08:46.141541+0000","last_became_peered":"2023-08-09T15:08:46.141541+0000","last_unstale":"2023-08-09T15:08:46.149920+0000","last_undegraded":"2023-08-09T15:08:46.149920+0000","last_fullsized":"2023-08-09T15:08:46.149920+0000","mapping_epoch":203,"log_start":"0'0","ondisk_log_start":"0'0","created":203,"last_epoch_clean":204,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T15:08:45.049361+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T15:08:45.049361+0000","last_clean_scrub_stamp":"2023-08-09T15:08:45.049361+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-11T00:34:17.523462+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"24.1b","version":"204'1","reported_seq":12,"reported_epoch":204,"state":"active+clean","last_fresh":"2023-08-09T15:08:46.152409+0000","last_change":"2023-08-09T15:08:45.099228+0000","last_active":"2023-08-09T15:08:46.152409+0000","last_peered":"2023-08-09T15:08:46.152409+0000","last_clean":"2023-08-09T15:08:46.152409+0000","last_became_active":"2023-08-09T15:08:45.098602+0000","last_became_peered":"2023-08-09T15:08:45.098602+0000","last_unstale":"2023-08-09T15:08:46.152409+0000","last_undegraded":"2023-08-09T15:08:46.152409+0000","last_fullsized":"2023-08-09T15:08:46.152409+0000","mapping_epoch":202,"log_start":"0'0","ondisk_log_start":"0'0","created":202,"last_epoch_clean":203,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T15:08:44.039630+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T15:08:44.039630+0000","last_clean_scrub_stamp":"2023-08-09T15:08:44.039630+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T19:47:48.622160+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.2","version":"0'0","reported_seq":376,"reported_epoch":204,"state":"active+clean","last_fresh":"2023-08-09T15:08:46.127610+0000","last_change":"2023-08-09T13:33:56.402794+0000","last_active":"2023-08-09T15:08:46.127610+0000","last_peered":"2023-08-09T15:08:46.127610+0000","last_clean":"2023-08-09T15:08:46.127610+0000","last_became_active":"2023-08-09T13:33:56.402038+0000","last_became_peered":"2023-08-09T13:33:56.402038+0000","last_unstale":"2023-08-09T15:08:46.127610+0000","last_undegraded":"2023-08-09T15:08:46.127610+0000","last_fullsized":"2023-08-09T15:08:46.127610+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:33:54.845869+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:33:54.845869+0000","last_clean_scrub_stamp":"2023-08-09T13:33:54.845869+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T21:22:56.846648+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"25.1b","version":"0'0","reported_seq":9,"reported_epoch":204,"state":"active+clean","last_fresh":"2023-08-09T15:08:46.136567+0000","last_change":"2023-08-09T15:08:46.132150+0000","last_active":"2023-08-09T15:08:46.136567+0000","last_peered":"2023-08-09T15:08:46.136567+0000","last_clean":"2023-08-09T15:08:46.136567+0000","last_became_active":"2023-08-09T15:08:46.131829+0000","last_became_peered":"2023-08-09T15:08:46.131829+0000","last_unstale":"2023-08-09T15:08:46.136567+0000","last_undegraded":"2023-08-09T15:08:46.136567+0000","last_fullsized":"2023-08-09T15:08:46.136567+0000","mapping_epoch":203,"log_start":"0'0","ondisk_log_start":"0'0","created":203,"last_epoch_clean":204,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T15:08:45.049361+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T15:08:45.049361+0000","last_clean_scrub_stamp":"2023-08-09T15:08:45.049361+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T22:10:49.551259+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"24.1a","version":"204'1","reported_seq":12,"reported_epoch":204,"state":"active+clean","last_fresh":"2023-08-09T15:08:46.156000+0000","last_change":"2023-08-09T15:08:45.087523+0000","last_active":"2023-08-09T15:08:46.156000+0000","last_peered":"2023-08-09T15:08:46.156000+0000","last_clean":"2023-08-09T15:08:46.156000+0000","last_became_active":"2023-08-09T15:08:45.087207+0000","last_became_peered":"2023-08-09T15:08:45.087207+0000","last_unstale":"2023-08-09T15:08:46.156000+0000","last_undegraded":"2023-08-09T15:08:46.156000+0000","last_fullsized":"2023-08-09T15:08:46.156000+0000","mapping_epoch":202,"log_start":"0'0","ondisk_log_start":"0'0","created":202,"last_epoch_clean":203,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T15:08:44.039630+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T15:08:44.039630+0000","last_clean_scrub_stamp":"2023-08-09T15:08:44.039630+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T22:07:46.924207+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.3","version":"0'0","reported_seq":392,"reported_epoch":204,"state":"active+clean","last_fresh":"2023-08-09T15:08:46.102085+0000","last_change":"2023-08-09T13:33:55.886749+0000","last_active":"2023-08-09T15:08:46.102085+0000","last_peered":"2023-08-09T15:08:46.102085+0000","last_clean":"2023-08-09T15:08:46.102085+0000","last_became_active":"2023-08-09T13:33:55.886116+0000","last_became_peered":"2023-08-09T13:33:55.886116+0000","last_unstale":"2023-08-09T15:08:46.102085+0000","last_undegraded":"2023-08-09T15:08:46.102085+0000","last_fullsized":"2023-08-09T15:08:46.102085+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:33:54.845869+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:33:54.845869+0000","last_clean_scrub_stamp":"2023-08-09T13:33:54.845869+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T17:43:10.480990+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"193'307","reported_seq":1201,"reported_epoch":204,"state":"active+clean","last_fresh":"2023-08-09T15:08:46.119082+0000","last_change":"2023-08-09T13:33:55.886345+0000","last_active":"2023-08-09T15:08:46.119082+0000","last_peered":"2023-08-09T15:08:46.119082+0000","last_clean":"2023-08-09T15:08:46.119082+0000","last_became_active":"2023-08-09T13:33:55.885781+0000","last_became_peered":"2023-08-09T13:33:55.885781+0000","last_unstale":"2023-08-09T15:08:46.119082+0000","last_undegraded":"2023-08-09T15:08:46.119082+0000","last_fullsized":"2023-08-09T15:08:46.119082+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:33:54.845869+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:33:54.845869+0000","last_clean_scrub_stamp":"2023-08-09T13:33:54.845869+0000","objects_scrubbed":0,"log_size":307,"log_dups_size":0,"ondisk_log_size":307,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T17:27:08.086934+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1771,"num_read_kb":3684,"num_write":503,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"25.1c","version":"0'0","reported_seq":9,"reported_epoch":204,"state":"active+clean","last_fresh":"2023-08-09T15:08:46.137767+0000","last_change":"2023-08-09T15:08:46.136662+0000","last_active":"2023-08-09T15:08:46.137767+0000","last_peered":"2023-08-09T15:08:46.137767+0000","last_clean":"2023-08-09T15:08:46.137767+0000","last_became_active":"2023-08-09T15:08:46.136372+0000","last_became_peered":"2023-08-09T15:08:46.136372+0000","last_unstale":"2023-08-09T15:08:46.137767+0000","last_undegraded":"2023-08-09T15:08:46.137767+0000","last_fullsized":"2023-08-09T15:08:46.137767+0000","mapping_epoch":203,"log_start":"0'0","ondisk_log_start":"0'0","created":203,"last_epoch_clean":204,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T15:08:45.049361+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T15:08:45.049361+0000","last_clean_scrub_stamp":"2023-08-09T15:08:45.049361+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-11T02:46:30.377863+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"24.1d","version":"204'2","reported_seq":13,"reported_epoch":204,"state":"active+clean","last_fresh":"2023-08-09T15:08:46.150662+0000","last_change":"2023-08-09T15:08:45.087713+0000","last_active":"2023-08-09T15:08:46.150662+0000","last_peered":"2023-08-09T15:08:46.150662+0000","last_clean":"2023-08-09T15:08:46.150662+0000","last_became_active":"2023-08-09T15:08:45.087305+0000","last_became_peered":"2023-08-09T15:08:45.087305+0000","last_unstale":"2023-08-09T15:08:46.150662+0000","last_undegraded":"2023-08-09T15:08:46.150662+0000","last_fullsized":"2023-08-09T15:08:46.150662+0000","mapping_epoch":202,"log_start":"0'0","ondisk_log_start":"0'0","created":202,"last_epoch_clean":203,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T15:08:44.039630+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T15:08:44.039630+0000","last_clean_scrub_stamp":"2023-08-09T15:08:44.039630+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T17:53:50.187553+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":394,"reported_epoch":204,"state":"active+clean","last_fresh":"2023-08-09T15:08:46.106272+0000","last_change":"2023-08-09T13:33:55.889565+0000","last_active":"2023-08-09T15:08:46.106272+0000","last_peered":"2023-08-09T15:08:46.106272+0000","last_clean":"2023-08-09T15:08:46.106272+0000","last_became_active":"2023-08-09T13:33:55.889164+0000","last_became_peered":"2023-08-09T13:33:55.889164+0000","last_unstale":"2023-08-09T15:08:46.106272+0000","last_undegraded":"2023-08-09T15:08:46.106272+0000","last_fullsized":"2023-08-09T15:08:46.106272+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:33:54.845869+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:33:54.845869+0000","last_clean_scrub_stamp":"2023-08-09T13:33:54.845869+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T15:22:20.360173+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"25.1d","version":"0'0","reported_seq":9,"reported_epoch":204,"state":"active+clean","last_fresh":"2023-08-09T15:08:46.137153+0000","last_change":"2023-08-09T15:08:46.133494+0000","last_active":"2023-08-09T15:08:46.137153+0000","last_peered":"2023-08-09T15:08:46.137153+0000","last_clean":"2023-08-09T15:08:46.137153+0000","last_became_active":"2023-08-09T15:08:46.133145+0000","last_became_peered":"2023-08-09T15:08:46.133145+0000","last_unstale":"2023-08-09T15:08:46.137153+0000","last_undegraded":"2023-08-09T15:08:46.137153+0000","last_fullsized":"2023-08-09T15:08:46.137153+0000","mapping_epoch":203,"log_start":"0'0","ondisk_log_start":"0'0","created":203,"last_epoch_clean":204,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T15:08:45.049361+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T15:08:45.049361+0000","last_clean_scrub_stamp":"2023-08-09T15:08:45.049361+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T22:17:05.642057+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"24.1c","version":"204'2","reported_seq":13,"reported_epoch":204,"state":"active+clean","last_fresh":"2023-08-09T15:08:46.148091+0000","last_change":"2023-08-09T15:08:45.089908+0000","last_active":"2023-08-09T15:08:46.148091+0000","last_peered":"2023-08-09T15:08:46.148091+0000","last_clean":"2023-08-09T15:08:46.148091+0000","last_became_active":"2023-08-09T15:08:45.089612+0000","last_became_peered":"2023-08-09T15:08:45.089612+0000","last_unstale":"2023-08-09T15:08:46.148091+0000","last_undegraded":"2023-08-09T15:08:46.148091+0000","last_fullsized":"2023-08-09T15:08:46.148091+0000","mapping_epoch":202,"log_start":"0'0","ondisk_log_start":"0'0","created":202,"last_epoch_clean":203,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T15:08:44.039630+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T15:08:44.039630+0000","last_clean_scrub_stamp":"2023-08-09T15:08:44.039630+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T20:53:18.888010+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":386,"reported_epoch":204,"state":"active+clean","last_fresh":"2023-08-09T15:08:46.107478+0000","last_change":"2023-08-09T13:33:56.404236+0000","last_active":"2023-08-09T15:08:46.107478+0000","last_peered":"2023-08-09T15:08:46.107478+0000","last_clean":"2023-08-09T15:08:46.107478+0000","last_became_active":"2023-08-09T13:33:56.402485+0000","last_became_peered":"2023-08-09T13:33:56.402485+0000","last_unstale":"2023-08-09T15:08:46.107478+0000","last_undegraded":"2023-08-09T15:08:46.107478+0000","last_fullsized":"2023-08-09T15:08:46.107478+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:33:54.845869+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:33:54.845869+0000","last_clean_scrub_stamp":"2023-08-09T13:33:54.845869+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T16:03:34.208897+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"25.1e","version":"0'0","reported_seq":9,"reported_epoch":204,"state":"active+clean","last_fresh":"2023-08-09T15:08:46.136637+0000","last_change":"2023-08-09T15:08:46.131911+0000","last_active":"2023-08-09T15:08:46.136637+0000","last_peered":"2023-08-09T15:08:46.136637+0000","last_clean":"2023-08-09T15:08:46.136637+0000","last_became_active":"2023-08-09T15:08:46.130158+0000","last_became_peered":"2023-08-09T15:08:46.130158+0000","last_unstale":"2023-08-09T15:08:46.136637+0000","last_undegraded":"2023-08-09T15:08:46.136637+0000","last_fullsized":"2023-08-09T15:08:46.136637+0000","mapping_epoch":203,"log_start":"0'0","ondisk_log_start":"0'0","created":203,"last_epoch_clean":204,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T15:08:45.049361+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T15:08:45.049361+0000","last_clean_scrub_stamp":"2023-08-09T15:08:45.049361+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T20:44:17.973099+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"24.1f","version":"204'4","reported_seq":15,"reported_epoch":204,"state":"active+clean","last_fresh":"2023-08-09T15:08:46.152032+0000","last_change":"2023-08-09T15:08:45.097633+0000","last_active":"2023-08-09T15:08:46.152032+0000","last_peered":"2023-08-09T15:08:46.152032+0000","last_clean":"2023-08-09T15:08:46.152032+0000","last_became_active":"2023-08-09T15:08:45.097388+0000","last_became_peered":"2023-08-09T15:08:45.097388+0000","last_unstale":"2023-08-09T15:08:46.152032+0000","last_undegraded":"2023-08-09T15:08:46.152032+0000","last_fullsized":"2023-08-09T15:08:46.152032+0000","mapping_epoch":202,"log_start":"0'0","ondisk_log_start":"0'0","created":202,"last_epoch_clean":203,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T15:08:44.039630+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T15:08:44.039630+0000","last_clean_scrub_stamp":"2023-08-09T15:08:44.039630+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T19:55:17.731855+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":374,"reported_epoch":204,"state":"active+clean","last_fresh":"2023-08-09T15:08:46.118852+0000","last_change":"2023-08-09T13:33:56.401781+0000","last_active":"2023-08-09T15:08:46.118852+0000","last_peered":"2023-08-09T15:08:46.118852+0000","last_clean":"2023-08-09T15:08:46.118852+0000","last_became_active":"2023-08-09T13:33:56.400336+0000","last_became_peered":"2023-08-09T13:33:56.400336+0000","last_unstale":"2023-08-09T15:08:46.118852+0000","last_undegraded":"2023-08-09T15:08:46.118852+0000","last_fullsized":"2023-08-09T15:08:46.118852+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:33:54.845869+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:33:54.845869+0000","last_clean_scrub_stamp":"2023-08-09T13:33:54.845869+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T21:34:46.146543+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"25.1f","version":"0'0","reported_seq":9,"reported_epoch":204,"state":"active+clean","last_fresh":"2023-08-09T15:08:46.150362+0000","last_change":"2023-08-09T15:08:46.143687+0000","last_active":"2023-08-09T15:08:46.150362+0000","last_peered":"2023-08-09T15:08:46.150362+0000","last_clean":"2023-08-09T15:08:46.150362+0000","last_became_active":"2023-08-09T15:08:46.143398+0000","last_became_peered":"2023-08-09T15:08:46.143398+0000","last_unstale":"2023-08-09T15:08:46.150362+0000","last_undegraded":"2023-08-09T15:08:46.150362+0000","last_fullsized":"2023-08-09T15:08:46.150362+0000","mapping_epoch":203,"log_start":"0'0","ondisk_log_start":"0'0","created":203,"last_epoch_clean":204,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T15:08:45.049361+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T15:08:45.049361+0000","last_clean_scrub_stamp":"2023-08-09T15:08:45.049361+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T17:29:59.824781+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"24.1e","version":"204'1","reported_seq":12,"reported_epoch":204,"state":"active+clean","last_fresh":"2023-08-09T15:08:46.146842+0000","last_change":"2023-08-09T15:08:45.081144+0000","last_active":"2023-08-09T15:08:46.146842+0000","last_peered":"2023-08-09T15:08:46.146842+0000","last_clean":"2023-08-09T15:08:46.146842+0000","last_became_active":"2023-08-09T15:08:45.080850+0000","last_became_peered":"2023-08-09T15:08:45.080850+0000","last_unstale":"2023-08-09T15:08:46.146842+0000","last_undegraded":"2023-08-09T15:08:46.146842+0000","last_fullsized":"2023-08-09T15:08:46.146842+0000","mapping_epoch":202,"log_start":"0'0","ondisk_log_start":"0'0","created":202,"last_epoch_clean":203,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T15:08:44.039630+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T15:08:44.039630+0000","last_clean_scrub_stamp":"2023-08-09T15:08:44.039630+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T15:49:06.727195+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":374,"reported_epoch":204,"state":"active+clean","last_fresh":"2023-08-09T15:08:46.119593+0000","last_change":"2023-08-09T13:33:55.886389+0000","last_active":"2023-08-09T15:08:46.119593+0000","last_peered":"2023-08-09T15:08:46.119593+0000","last_clean":"2023-08-09T15:08:46.119593+0000","last_became_active":"2023-08-09T13:33:55.885638+0000","last_became_peered":"2023-08-09T13:33:55.885638+0000","last_unstale":"2023-08-09T15:08:46.119593+0000","last_undegraded":"2023-08-09T15:08:46.119593+0000","last_fullsized":"2023-08-09T15:08:46.119593+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:33:54.845869+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:33:54.845869+0000","last_clean_scrub_stamp":"2023-08-09T13:33:54.845869+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T16:56:49.433631+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]}],"pool_stats":[{"poolid":25,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":24,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":38,"ondisk_log_size":38,"up":64,"acting":64,"num_store_stats":8},{"poolid":2,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1771,"num_read_kb":3684,"num_write":503,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":307,"ondisk_log_size":307,"up":2,"acting":2,"num_store_stats":2},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":8}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834575981,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":52224,"kb_used_data":3060,"kb_used_omap":1,"kb_used_meta":49150,"kb_avail":94319616,"statfs":{"total":96636764160,"available":96583286784,"internally_reserved":0,"allocated":3133440,"data_stored":1971235,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1781,"internal_metadata":50329867},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Aug 9 15:08:46 2023","interfaces":[{"interface":"back","average":{"1min":1.2789999999999999,"5min":1.246,"15min":1.2350000000000001},"min":{"1min":0.92300000000000004,"5min":0.77000000000000002,"15min":0.76100000000000001},"max":{"1min":2.056,"5min":2.056,"15min":2.056},"last":0.92300000000000004},{"interface":"front","average":{"1min":1.234,"5min":1.2709999999999999,"15min":1.2689999999999999},"min":{"1min":0.91200000000000003,"5min":0.84699999999999998,"15min":0.81200000000000006},"max":{"1min":1.7769999999999999,"5min":1.863,"15min":1.863},"last":0.95299999999999996}]},{"osd":1,"last update":"Wed Aug 9 15:08:46 2023","interfaces":[{"interface":"back","average":{"1min":1.212,"5min":1.2909999999999999,"15min":1.3129999999999999},"min":{"1min":0.88,"5min":0.88,"15min":0.83399999999999996},"max":{"1min":1.762,"5min":1.762,"15min":1.855},"last":1.113},{"interface":"front","average":{"1min":1.3100000000000001,"5min":1.335,"15min":1.3140000000000001},"min":{"1min":0.879,"5min":0.879,"15min":0.879},"max":{"1min":2.1520000000000001,"5min":2.1520000000000001,"15min":2.1520000000000001},"last":1.173}]},{"osd":2,"last update":"Wed Aug 9 15:08:46 2023","interfaces":[{"interface":"back","average":{"1min":1.1639999999999999,"5min":1.333,"15min":1.3080000000000001},"min":{"1min":0.98699999999999999,"5min":0.93300000000000005,"15min":0.86799999999999999},"max":{"1min":1.4199999999999999,"5min":1.7729999999999999,"15min":1.7829999999999999},"last":1.016},{"interface":"front","average":{"1min":1.337,"5min":1.4259999999999999,"15min":1.4059999999999999},"min":{"1min":1.056,"5min":1.056,"15min":0.88500000000000001},"max":{"1min":1.8340000000000001,"5min":1.8340000000000001,"15min":1.8400000000000001},"last":1.216}]},{"osd":3,"last update":"Wed Aug 9 15:08:46 2023","interfaces":[{"interface":"back","average":{"1min":1.3029999999999999,"5min":1.365,"15min":1.359},"min":{"1min":0.80800000000000005,"5min":0.80800000000000005,"15min":0.80800000000000005},"max":{"1min":1.657,"5min":1.9259999999999999,"15min":1.9259999999999999},"last":1.2569999999999999},{"interface":"front","average":{"1min":1.409,"5min":1.4950000000000001,"15min":1.4810000000000001},"min":{"1min":1.02,"5min":0.94499999999999995,"15min":0.94499999999999995},"max":{"1min":1.669,"5min":1.901,"15min":1.9530000000000001},"last":1.2989999999999999}]},{"osd":4,"last update":"Wed Aug 9 15:08:46 2023","interfaces":[{"interface":"back","average":{"1min":1.3360000000000001,"5min":1.3009999999999999,"15min":1.3049999999999999},"min":{"1min":0.86799999999999999,"5min":0.86799999999999999,"15min":0.69599999999999995},"max":{"1min":2.2810000000000001,"5min":2.2810000000000001,"15min":2.2810000000000001},"last":1.341},{"interface":"front","average":{"1min":1.3680000000000001,"5min":1.379,"15min":1.3799999999999999},"min":{"1min":0.94399999999999995,"5min":0.92300000000000004,"15min":0.66000000000000003},"max":{"1min":1.782,"5min":2.0830000000000002,"15min":2.0830000000000002},"last":1.782}]},{"osd":5,"last update":"Wed Aug 9 15:08:46 2023","interfaces":[{"interface":"back","average":{"1min":1.462,"5min":1.4450000000000001,"15min":1.4550000000000001},"min":{"1min":0.89000000000000001,"5min":0.86099999999999999,"15min":0.86099999999999999},"max":{"1min":1.9550000000000001,"5min":1.9550000000000001,"15min":1.9550000000000001},"last":1.4610000000000001},{"interface":"front","average":{"1min":1.5289999999999999,"5min":1.4950000000000001,"15min":1.4910000000000001},"min":{"1min":1.0049999999999999,"5min":0.98399999999999999,"15min":0.95099999999999996},"max":{"1min":2.5310000000000001,"5min":2.5310000000000001,"15min":2.5310000000000001},"last":1.5029999999999999}]},{"osd":6,"last update":"Wed Aug 9 15:08:46 2023","interfaces":[{"interface":"back","average":{"1min":1.6970000000000001,"5min":1.6399999999999999,"15min":1.605},"min":{"1min":1.4139999999999999,"5min":1.268,"15min":1.101},"max":{"1min":2.8959999999999999,"5min":2.8959999999999999,"15min":2.8959999999999999},"last":1.5800000000000001},{"interface":"front","average":{"1min":1.6200000000000001,"5min":1.5820000000000001,"15min":1.55},"min":{"1min":1.2809999999999999,"5min":1.1519999999999999,"15min":0.90800000000000003},"max":{"1min":2.2130000000000001,"5min":2.222,"15min":2.222},"last":1.919}]}]},{"osd":6,"up_from":13,"seq":55834575981,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":59844,"kb_used_data":2488,"kb_used_omap":1,"kb_used_meta":57342,"kb_avail":94311996,"statfs":{"total":96636764160,"available":96575483904,"internally_reserved":0,"allocated":2547712,"data_stored":1386998,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1781,"internal_metadata":58718475},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Aug 9 15:07:53 2023","interfaces":[{"interface":"back","average":{"1min":1.2829999999999999,"5min":1.2170000000000001,"15min":1.2230000000000001},"min":{"1min":0.94299999999999995,"5min":0.90000000000000002,"15min":0.64100000000000001},"max":{"1min":1.772,"5min":1.772,"15min":1.8859999999999999},"last":1.181},{"interface":"front","average":{"1min":1.25,"5min":1.2649999999999999,"15min":1.25},"min":{"1min":0.92100000000000004,"5min":0.83099999999999996,"15min":0.71399999999999997},"max":{"1min":1.8680000000000001,"5min":1.8680000000000001,"15min":1.917},"last":1.0009999999999999}]},{"osd":1,"last update":"Wed Aug 9 15:07:53 2023","interfaces":[{"interface":"back","average":{"1min":1.236,"5min":1.194,"15min":1.2010000000000001},"min":{"1min":1.022,"5min":0.90200000000000002,"15min":0.75700000000000001},"max":{"1min":1.5600000000000001,"5min":1.7529999999999999,"15min":1.7529999999999999},"last":0.91900000000000004},{"interface":"front","average":{"1min":1.3540000000000001,"5min":1.3580000000000001,"15min":1.343},"min":{"1min":0.99399999999999999,"5min":0.91100000000000003,"15min":0.84499999999999997},"max":{"1min":1.9410000000000001,"5min":1.9410000000000001,"15min":1.9410000000000001},"last":1.266}]},{"osd":2,"last update":"Wed Aug 9 15:07:53 2023","interfaces":[{"interface":"back","average":{"1min":1.3460000000000001,"5min":1.3500000000000001,"15min":1.343},"min":{"1min":1.0009999999999999,"5min":0.82999999999999996,"15min":0.752},"max":{"1min":1.623,"5min":1.9259999999999999,"15min":2.081},"last":0.82099999999999995},{"interface":"front","average":{"1min":1.3660000000000001,"5min":1.349,"15min":1.3540000000000001},"min":{"1min":1.0589999999999999,"5min":0.90600000000000003,"15min":0.88900000000000001},"max":{"1min":1.6990000000000001,"5min":1.8700000000000001,"15min":1.8700000000000001},"last":1.2889999999999999}]},{"osd":3,"last update":"Wed Aug 9 15:07:53 2023","interfaces":[{"interface":"back","average":{"1min":1.3919999999999999,"5min":1.401,"15min":1.395},"min":{"1min":1.018,"5min":0.98499999999999999,"15min":0.85499999999999998},"max":{"1min":1.831,"5min":1.831,"15min":1.9450000000000001},"last":1.413},{"interface":"front","average":{"1min":1.3480000000000001,"5min":1.3440000000000001,"15min":1.353},"min":{"1min":1.103,"5min":0.97599999999999998,"15min":0.95299999999999996},"max":{"1min":1.6319999999999999,"5min":1.8660000000000001,"15min":1.8660000000000001},"last":1.0449999999999999}]},{"osd":4,"last update":"Wed Aug 9 15:07:53 2023","interfaces":[{"interface":"back","average":{"1min":1.351,"5min":1.3640000000000001,"15min":1.321},"min":{"1min":0.78800000000000003,"5min":0.77200000000000002,"15min":0.48599999999999999},"max":{"1min":2.0470000000000002,"5min":2.0470000000000002,"15min":2.0470000000000002},"last":1.3540000000000001},{"interface":"front","average":{"1min":1.4330000000000001,"5min":1.466,"15min":1.413},"min":{"1min":0.82099999999999995,"5min":0.82099999999999995,"15min":0.52000000000000002},"max":{"1min":1.8480000000000001,"5min":1.9970000000000001,"15min":2.2280000000000002},"last":1.133}]},{"osd":5,"last update":"Wed Aug 9 15:07:53 2023","interfaces":[{"interface":"back","average":{"1min":1.405,"5min":1.3919999999999999,"15min":1.3600000000000001},"min":{"1min":0.94299999999999995,"5min":0.93899999999999995,"15min":0.72599999999999998},"max":{"1min":1.754,"5min":2.6059999999999999,"15min":2.6059999999999999},"last":1.234},{"interface":"front","average":{"1min":1.633,"5min":1.6299999999999999,"15min":1.607},"min":{"1min":1.1180000000000001,"5min":0.85599999999999998,"15min":0.68500000000000005},"max":{"1min":2.1200000000000001,"5min":2.1200000000000001,"15min":2.1899999999999999},"last":1.6579999999999999}]},{"osd":7,"last update":"Wed Aug 9 15:07:53 2023","interfaces":[{"interface":"back","average":{"1min":1.6359999999999999,"5min":1.645,"15min":1.5980000000000001},"min":{"1min":1.2929999999999999,"5min":0.84699999999999998,"15min":0.624},"max":{"1min":1.972,"5min":2.4340000000000002,"15min":2.4340000000000002},"last":1.5580000000000001},{"interface":"front","average":{"1min":1.589,"5min":1.599,"15min":1.5860000000000001},"min":{"1min":1.282,"5min":1.143,"15min":0.83899999999999997},"max":{"1min":1.9019999999999999,"5min":2.7400000000000002,"15min":2.7400000000000002},"last":1.6060000000000001}]}]},{"osd":5,"up_from":13,"seq":55834575981,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":47552,"kb_used_data":2480,"kb_used_omap":1,"kb_used_meta":45054,"kb_avail":94324288,"statfs":{"total":96636764160,"available":96588070912,"internally_reserved":0,"allocated":2539520,"data_stored":1380867,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1781,"internal_metadata":46135563},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Aug 9 15:08:42 2023","interfaces":[{"interface":"back","average":{"1min":1.2210000000000001,"5min":1.1859999999999999,"15min":1.202},"min":{"1min":0.59399999999999997,"5min":0.59399999999999997,"15min":0.59399999999999997},"max":{"1min":1.7370000000000001,"5min":1.754,"15min":1.754},"last":0.96899999999999997},{"interface":"front","average":{"1min":1.1559999999999999,"5min":1.2430000000000001,"15min":1.228},"min":{"1min":0.67000000000000004,"5min":0.67000000000000004,"15min":0.67000000000000004},"max":{"1min":1.6180000000000001,"5min":1.865,"15min":1.976},"last":0.76700000000000002}]},{"osd":1,"last update":"Wed Aug 9 15:08:42 2023","interfaces":[{"interface":"back","average":{"1min":1.256,"5min":1.2450000000000001,"15min":1.258},"min":{"1min":0.65200000000000002,"5min":0.65200000000000002,"15min":0.65200000000000002},"max":{"1min":1.823,"5min":2.016,"15min":2.016},"last":1.4419999999999999},{"interface":"front","average":{"1min":1.196,"5min":1.3460000000000001,"15min":1.359},"min":{"1min":0.751,"5min":0.751,"15min":0.751},"max":{"1min":1.728,"5min":1.8959999999999999,"15min":1.8979999999999999},"last":1.1619999999999999}]},{"osd":2,"last update":"Wed Aug 9 15:08:42 2023","interfaces":[{"interface":"back","average":{"1min":1.351,"5min":1.325,"15min":1.321},"min":{"1min":0.89700000000000002,"5min":0.89700000000000002,"15min":0.82499999999999996},"max":{"1min":1.855,"5min":2.032,"15min":2.032},"last":0.89700000000000002},{"interface":"front","average":{"1min":1.3089999999999999,"5min":1.3839999999999999,"15min":1.3839999999999999},"min":{"1min":0.81100000000000005,"5min":0.81100000000000005,"15min":0.81100000000000005},"max":{"1min":1.956,"5min":1.964,"15min":1.964},"last":1.6830000000000001}]},{"osd":3,"last update":"Wed Aug 9 15:08:42 2023","interfaces":[{"interface":"back","average":{"1min":1.355,"5min":1.4099999999999999,"15min":1.387},"min":{"1min":1.111,"5min":0.97399999999999998,"15min":0.93799999999999994},"max":{"1min":1.486,"5min":1.867,"15min":1.899},"last":1.286},{"interface":"front","average":{"1min":1.3620000000000001,"5min":1.4630000000000001,"15min":1.4610000000000001},"min":{"1min":0.85999999999999999,"5min":0.85999999999999999,"15min":0.85999999999999999},"max":{"1min":1.7070000000000001,"5min":2.0379999999999998,"15min":2.0379999999999998},"last":1.5009999999999999}]},{"osd":4,"last update":"Wed Aug 9 15:08:42 2023","interfaces":[{"interface":"back","average":{"1min":1.3720000000000001,"5min":1.3320000000000001,"15min":1.3320000000000001},"min":{"1min":0.96699999999999997,"5min":0.79000000000000004,"15min":0.62},"max":{"1min":2.0299999999999998,"5min":2.0299999999999998,"15min":2.0299999999999998},"last":1.226},{"interface":"front","average":{"1min":1.4810000000000001,"5min":1.421,"15min":1.417},"min":{"1min":0.97899999999999998,"5min":0.85899999999999999,"15min":0.64200000000000002},"max":{"1min":1.9239999999999999,"5min":2.0110000000000001,"15min":2.0390000000000001},"last":1.548}]},{"osd":6,"last update":"Wed Aug 9 15:08:42 2023","interfaces":[{"interface":"back","average":{"1min":1.391,"5min":1.4159999999999999,"15min":1.4279999999999999},"min":{"1min":0.98399999999999999,"5min":0.96099999999999997,"15min":0.67400000000000004},"max":{"1min":1.8799999999999999,"5min":2.1000000000000001,"15min":2.1000000000000001},"last":1.325},{"interface":"front","average":{"1min":1.5189999999999999,"5min":1.524,"15min":1.5229999999999999},"min":{"1min":1.1879999999999999,"5min":0.91200000000000003,"15min":0.79900000000000004},"max":{"1min":2.1349999999999998,"5min":2.1349999999999998,"15min":2.3410000000000002},"last":1.577}]},{"osd":7,"last update":"Wed Aug 9 15:08:42 2023","interfaces":[{"interface":"back","average":{"1min":1.5980000000000001,"5min":1.5820000000000001,"15min":1.595},"min":{"1min":1.286,"5min":1.0569999999999999,"15min":0.86099999999999999},"max":{"1min":1.929,"5min":1.956,"15min":2.4689999999999999},"last":1.3999999999999999},{"interface":"front","average":{"1min":1.6399999999999999,"5min":1.6459999999999999,"15min":1.657},"min":{"1min":1.2829999999999999,"5min":1.044,"15min":0.83399999999999996},"max":{"1min":1.889,"5min":2.1539999999999999,"15min":2.4239999999999999},"last":1.6279999999999999}]}]},{"osd":4,"up_from":13,"seq":55834575981,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":47552,"kb_used_data":2480,"kb_used_omap":1,"kb_used_meta":45054,"kb_avail":94324288,"statfs":{"total":96636764160,"available":96588070912,"internally_reserved":0,"allocated":2539520,"data_stored":1380867,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1781,"internal_metadata":46135563},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Aug 9 15:08:09 2023","interfaces":[{"interface":"back","average":{"1min":1.1859999999999999,"5min":1.2310000000000001,"15min":1.232},"min":{"1min":0.92100000000000004,"5min":0.63500000000000001,"15min":0.63500000000000001},"max":{"1min":1.6919999999999999,"5min":1.8360000000000001,"15min":2.2959999999999998},"last":0.77000000000000002},{"interface":"front","average":{"1min":1.3859999999999999,"5min":1.363,"15min":1.3300000000000001},"min":{"1min":1.0720000000000001,"5min":0.92800000000000005,"15min":0.85099999999999998},"max":{"1min":1.7529999999999999,"5min":1.7529999999999999,"15min":2.2709999999999999},"last":0.72699999999999998}]},{"osd":1,"last update":"Wed Aug 9 15:08:09 2023","interfaces":[{"interface":"back","average":{"1min":1.28,"5min":1.371,"15min":1.3500000000000001},"min":{"1min":0.94099999999999995,"5min":0.89700000000000002,"15min":0.84699999999999998},"max":{"1min":1.756,"5min":1.8460000000000001,"15min":1.9650000000000001},"last":1.052},{"interface":"front","average":{"1min":1.2729999999999999,"5min":1.26,"15min":1.278},"min":{"1min":0.89900000000000002,"5min":0.83399999999999996,"15min":0.83399999999999996},"max":{"1min":1.8600000000000001,"5min":1.8600000000000001,"15min":2.117},"last":0.98099999999999998}]},{"osd":2,"last update":"Wed Aug 9 15:08:09 2023","interfaces":[{"interface":"back","average":{"1min":1.28,"5min":1.3029999999999999,"15min":1.3100000000000001},"min":{"1min":0.91000000000000003,"5min":0.90700000000000003,"15min":0.84599999999999997},"max":{"1min":1.633,"5min":1.7989999999999999,"15min":1.883},"last":1.3819999999999999},{"interface":"front","average":{"1min":1.4019999999999999,"5min":1.403,"15min":1.411},"min":{"1min":1.21,"5min":0.79200000000000004,"15min":0.79200000000000004},"max":{"1min":1.667,"5min":1.8240000000000001,"15min":2.117},"last":1.4550000000000001}]},{"osd":3,"last update":"Wed Aug 9 15:08:09 2023","interfaces":[{"interface":"back","average":{"1min":1.345,"5min":1.347,"15min":1.3500000000000001},"min":{"1min":1.0089999999999999,"5min":0.88700000000000001,"15min":0.88700000000000001},"max":{"1min":1.591,"5min":1.931,"15min":2.4870000000000001},"last":1.292},{"interface":"front","average":{"1min":1.444,"5min":1.458,"15min":1.478},"min":{"1min":1.1879999999999999,"5min":1.0249999999999999,"15min":1.0249999999999999},"max":{"1min":1.909,"5min":2.5459999999999998,"15min":2.5459999999999998},"last":1.5289999999999999}]},{"osd":5,"last update":"Wed Aug 9 15:08:09 2023","interfaces":[{"interface":"back","average":{"1min":1.425,"5min":1.385,"15min":1.4159999999999999},"min":{"1min":1.0089999999999999,"5min":0.82999999999999996,"15min":0.82199999999999995},"max":{"1min":1.8700000000000001,"5min":1.893,"15min":2.4430000000000001},"last":1.22},{"interface":"front","average":{"1min":1.389,"5min":1.3799999999999999,"15min":1.397},"min":{"1min":0.91000000000000003,"5min":0.70899999999999996,"15min":0.70899999999999996},"max":{"1min":1.7130000000000001,"5min":2.1150000000000002,"15min":2.4249999999999998},"last":1.7050000000000001}]},{"osd":6,"last update":"Wed Aug 9 15:08:09 2023","interfaces":[{"interface":"back","average":{"1min":1.448,"5min":1.3899999999999999,"15min":1.419},"min":{"1min":0.95399999999999996,"5min":0.86299999999999999,"15min":0.80600000000000005},"max":{"1min":2.0049999999999999,"5min":2.9119999999999999,"15min":2.9119999999999999},"last":1.659},{"interface":"front","average":{"1min":1.5740000000000001,"5min":1.5700000000000001,"15min":1.5840000000000001},"min":{"1min":1.0329999999999999,"5min":0.86399999999999999,"15min":0.86399999999999999},"max":{"1min":1.8160000000000001,"5min":2.488,"15min":2.5219999999999998},"last":1.5760000000000001}]},{"osd":7,"last update":"Wed Aug 9 15:08:09 2023","interfaces":[{"interface":"back","average":{"1min":1.484,"5min":1.498,"15min":1.5269999999999999},"min":{"1min":1.0049999999999999,"5min":0.94599999999999995,"15min":0.94599999999999995},"max":{"1min":1.9610000000000001,"5min":2.355,"15min":2.5910000000000002},"last":1.772},{"interface":"front","average":{"1min":1.611,"5min":1.5840000000000001,"15min":1.6160000000000001},"min":{"1min":1.1259999999999999,"5min":0.90300000000000002,"15min":0.90300000000000002},"max":{"1min":2.1720000000000002,"5min":3.028,"15min":3.028},"last":1.748}]}]},{"osd":3,"up_from":13,"seq":55834575982,"num_pgs":19,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":63936,"kb_used_data":2480,"kb_used_omap":1,"kb_used_meta":61438,"kb_avail":94307904,"statfs":{"total":96636764160,"available":96571293696,"internally_reserved":0,"allocated":2539520,"data_stored":1380867,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1781,"internal_metadata":62912779},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Aug 9 15:07:55 2023","interfaces":[{"interface":"back","average":{"1min":1.036,"5min":0.97899999999999998,"15min":0.97399999999999998},"min":{"1min":0.745,"5min":0.66400000000000003,"15min":0.45200000000000001},"max":{"1min":1.5900000000000001,"5min":2.2090000000000001,"15min":2.2090000000000001},"last":1.577},{"interface":"front","average":{"1min":1.1599999999999999,"5min":1.175,"15min":1.179},"min":{"1min":0.78300000000000003,"5min":0.77900000000000003,"15min":0.56100000000000005},"max":{"1min":1.8089999999999999,"5min":1.8720000000000001,"15min":1.8720000000000001},"last":1.0760000000000001}]},{"osd":1,"last update":"Wed Aug 9 15:07:55 2023","interfaces":[{"interface":"back","average":{"1min":1.04,"5min":1.0860000000000001,"15min":1.0800000000000001},"min":{"1min":0.79600000000000004,"5min":0.748,"15min":0.51500000000000001},"max":{"1min":1.476,"5min":2.27,"15min":2.27},"last":1.0009999999999999},{"interface":"front","average":{"1min":1.288,"5min":1.292,"15min":1.264},"min":{"1min":0.93500000000000005,"5min":0.86199999999999999,"15min":0.623},"max":{"1min":1.772,"5min":1.9730000000000001,"15min":1.9730000000000001},"last":1.1319999999999999}]},{"osd":2,"last update":"Wed Aug 9 15:07:55 2023","interfaces":[{"interface":"back","average":{"1min":1.304,"5min":1.3360000000000001,"15min":1.339},"min":{"1min":0.998,"5min":0.879,"15min":0.55300000000000005},"max":{"1min":1.675,"5min":2.4079999999999999,"15min":2.4079999999999999},"last":1.3080000000000001},{"interface":"front","average":{"1min":1.486,"5min":1.4630000000000001,"15min":1.4390000000000001},"min":{"1min":0.99299999999999999,"5min":0.90000000000000002,"15min":0.72999999999999998},"max":{"1min":1.837,"5min":1.915,"15min":2.1259999999999999},"last":1.3779999999999999}]},{"osd":4,"last update":"Wed Aug 9 15:07:55 2023","interfaces":[{"interface":"back","average":{"1min":1.429,"5min":1.4339999999999999,"15min":1.421},"min":{"1min":1.0469999999999999,"5min":0.96999999999999997,"15min":0.72599999999999998},"max":{"1min":1.9179999999999999,"5min":2.2069999999999999,"15min":2.238},"last":1.4419999999999999},{"interface":"front","average":{"1min":1.4390000000000001,"5min":1.452,"15min":1.474},"min":{"1min":1.083,"5min":1.042,"15min":0.89500000000000002},"max":{"1min":1.851,"5min":2.0019999999999998,"15min":2.3460000000000001},"last":1.246}]},{"osd":5,"last update":"Wed Aug 9 15:07:55 2023","interfaces":[{"interface":"back","average":{"1min":1.5589999999999999,"5min":1.546,"15min":1.554},"min":{"1min":1.2050000000000001,"5min":1.2050000000000001,"15min":0.88800000000000001},"max":{"1min":1.895,"5min":1.9850000000000001,"15min":2.5579999999999998},"last":1.4219999999999999},{"interface":"front","average":{"1min":1.639,"5min":1.6100000000000001,"15min":1.6100000000000001},"min":{"1min":1.2150000000000001,"5min":1.0680000000000001,"15min":0.95599999999999996},"max":{"1min":2.2469999999999999,"5min":2.3109999999999999,"15min":2.4049999999999998},"last":1.653}]},{"osd":6,"last update":"Wed Aug 9 15:07:55 2023","interfaces":[{"interface":"back","average":{"1min":1.5509999999999999,"5min":1.591,"15min":1.601},"min":{"1min":1.2150000000000001,"5min":1.0860000000000001,"15min":0.96899999999999997},"max":{"1min":1.972,"5min":2.1669999999999998,"15min":2.4590000000000001},"last":1.7549999999999999},{"interface":"front","average":{"1min":1.6279999999999999,"5min":1.6639999999999999,"15min":1.6779999999999999},"min":{"1min":1.222,"5min":1.222,"15min":0.93500000000000005},"max":{"1min":2.032,"5min":2.1429999999999998,"15min":2.6509999999999998},"last":1.671}]},{"osd":7,"last update":"Wed Aug 9 15:07:55 2023","interfaces":[{"interface":"back","average":{"1min":1.8080000000000001,"5min":1.7729999999999999,"15min":1.758},"min":{"1min":1.5329999999999999,"5min":1.2250000000000001,"15min":1.0840000000000001},"max":{"1min":2.1699999999999999,"5min":2.2909999999999999,"15min":2.516},"last":2.008},{"interface":"front","average":{"1min":1.7170000000000001,"5min":1.706,"15min":1.726},"min":{"1min":1.3819999999999999,"5min":1.169,"15min":1.0369999999999999},"max":{"1min":2.0619999999999998,"5min":2.1200000000000001,"15min":2.9380000000000002},"last":1.8280000000000001}]}]},{"osd":2,"up_from":13,"seq":55834575981,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":41812,"kb_used_data":2504,"kb_used_omap":1,"kb_used_meta":39294,"kb_avail":94330028,"statfs":{"total":96636764160,"available":96593948672,"internally_reserved":0,"allocated":2564096,"data_stored":1389053,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1781,"internal_metadata":40237323},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Aug 9 15:07:45 2023","interfaces":[{"interface":"back","average":{"1min":1.034,"5min":0.96799999999999997,"15min":0.95099999999999996},"min":{"1min":0.76400000000000001,"5min":0.70099999999999996,"15min":0.44400000000000001},"max":{"1min":1.482,"5min":1.587,"15min":1.625},"last":0.44800000000000001},{"interface":"front","average":{"1min":1.133,"5min":1.115,"15min":1.0900000000000001},"min":{"1min":0.72699999999999998,"5min":0.70699999999999996,"15min":0.503},"max":{"1min":1.764,"5min":2.052,"15min":2.052},"last":0.59499999999999997}]},{"osd":1,"last update":"Wed Aug 9 15:07:45 2023","interfaces":[{"interface":"back","average":{"1min":1.3620000000000001,"5min":1.266,"15min":1.232},"min":{"1min":0.90700000000000003,"5min":0.83799999999999997,"15min":0.59399999999999997},"max":{"1min":1.7150000000000001,"5min":1.8149999999999999,"15min":1.825},"last":0.72399999999999998},{"interface":"front","average":{"1min":1.2490000000000001,"5min":1.387,"15min":1.4079999999999999},"min":{"1min":0.84399999999999997,"5min":0.84399999999999997,"15min":0.58299999999999996},"max":{"1min":1.6579999999999999,"5min":1.9490000000000001,"15min":1.9490000000000001},"last":0.77200000000000002}]},{"osd":3,"last update":"Wed Aug 9 15:07:45 2023","interfaces":[{"interface":"back","average":{"1min":1.143,"5min":1.1200000000000001,"15min":1.1060000000000001},"min":{"1min":0.96399999999999997,"5min":0.753,"15min":0.54800000000000004},"max":{"1min":1.496,"5min":1.7050000000000001,"15min":2.0070000000000001},"last":0.83199999999999996},{"interface":"front","average":{"1min":1.425,"5min":1.3899999999999999,"15min":1.369},"min":{"1min":0.93600000000000005,"5min":0.93600000000000005,"15min":0.76000000000000001},"max":{"1min":1.921,"5min":2.0470000000000002,"15min":2.0470000000000002},"last":0.64800000000000002}]},{"osd":4,"last update":"Wed Aug 9 15:07:45 2023","interfaces":[{"interface":"back","average":{"1min":1.6439999999999999,"5min":1.6100000000000001,"15min":1.542},"min":{"1min":1.3200000000000001,"5min":1.252,"15min":1.014},"max":{"1min":1.994,"5min":2.464,"15min":2.464},"last":0.90000000000000002},{"interface":"front","average":{"1min":1.4810000000000001,"5min":1.512,"15min":1.474},"min":{"1min":1.048,"5min":1.048,"15min":0.82399999999999995},"max":{"1min":1.9219999999999999,"5min":2.0720000000000001,"15min":2.2759999999999998},"last":1.0249999999999999}]},{"osd":5,"last update":"Wed Aug 9 15:07:45 2023","interfaces":[{"interface":"back","average":{"1min":1.5720000000000001,"5min":1.5669999999999999,"15min":1.5349999999999999},"min":{"1min":1.327,"5min":1.1220000000000001,"15min":0.92200000000000004},"max":{"1min":2.0369999999999999,"5min":2.1139999999999999,"15min":2.423},"last":0.93400000000000005},{"interface":"front","average":{"1min":1.5069999999999999,"5min":1.5089999999999999,"15min":1.5029999999999999},"min":{"1min":1.117,"5min":1.117,"15min":1.03},"max":{"1min":1.827,"5min":1.931,"15min":2.214},"last":1.2629999999999999}]},{"osd":6,"last update":"Wed Aug 9 15:07:45 2023","interfaces":[{"interface":"back","average":{"1min":1.571,"5min":1.548,"15min":1.526},"min":{"1min":1.133,"5min":1.0269999999999999,"15min":0.88},"max":{"1min":1.9450000000000001,"5min":2.0819999999999999,"15min":2.4550000000000001},"last":1.117},{"interface":"front","average":{"1min":1.7270000000000001,"5min":1.659,"15min":1.6319999999999999},"min":{"1min":1.3380000000000001,"5min":1.171,"15min":1.0549999999999999},"max":{"1min":2.077,"5min":2.351,"15min":2.3839999999999999},"last":1.0649999999999999}]},{"osd":7,"last update":"Wed Aug 9 15:07:45 2023","interfaces":[{"interface":"back","average":{"1min":1.6779999999999999,"5min":1.6739999999999999,"15min":1.6599999999999999},"min":{"1min":1.171,"5min":1.171,"15min":0.97699999999999998},"max":{"1min":2.1749999999999998,"5min":2.302,"15min":2.5099999999999998},"last":1.167},{"interface":"front","average":{"1min":1.655,"5min":1.661,"15min":1.6379999999999999},"min":{"1min":1.2330000000000001,"5min":1.224,"15min":0.84799999999999998},"max":{"1min":2.0230000000000001,"5min":2.3809999999999998,"15min":2.3809999999999998},"last":1.3300000000000001}]}]},{"osd":1,"up_from":13,"seq":55834575981,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":56344,"kb_used_data":3084,"kb_used_omap":1,"kb_used_meta":53246,"kb_avail":94315496,"statfs":{"total":96636764160,"available":96579067904,"internally_reserved":0,"allocated":3158016,"data_stored":1980339,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1781,"internal_metadata":54524171},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Aug 9 15:07:53 2023","interfaces":[{"interface":"back","average":{"1min":1.0620000000000001,"5min":1.0389999999999999,"15min":1.016},"min":{"1min":0.74299999999999999,"5min":0.59099999999999997,"15min":0.42399999999999999},"max":{"1min":1.6890000000000001,"5min":1.8129999999999999,"15min":2.0939999999999999},"last":1.2450000000000001},{"interface":"front","average":{"1min":1.149,"5min":1.1539999999999999,"15min":1.147},"min":{"1min":0.83499999999999996,"5min":0.68100000000000005,"15min":0.57799999999999996},"max":{"1min":1.78,"5min":1.8680000000000001,"15min":2.0190000000000001},"last":1.0940000000000001}]},{"osd":2,"last update":"Wed Aug 9 15:07:53 2023","interfaces":[{"interface":"back","average":{"1min":1.0640000000000001,"5min":1.0900000000000001,"15min":1.034},"min":{"1min":0.84999999999999998,"5min":0.64100000000000001,"15min":0.60099999999999998},"max":{"1min":1.589,"5min":2.3940000000000001,"15min":2.3940000000000001},"last":0.90100000000000002},{"interface":"front","average":{"1min":1.2350000000000001,"5min":1.2969999999999999,"15min":1.276},"min":{"1min":0.96099999999999997,"5min":0.871,"15min":0.67900000000000005},"max":{"1min":1.6499999999999999,"5min":1.8899999999999999,"15min":2},"last":0.82199999999999995}]},{"osd":3,"last update":"Wed Aug 9 15:07:53 2023","interfaces":[{"interface":"back","average":{"1min":1.4330000000000001,"5min":1.401,"15min":1.411},"min":{"1min":1.1100000000000001,"5min":1.0089999999999999,"15min":0.629},"max":{"1min":1.7829999999999999,"5min":2.109,"15min":2.109},"last":1.444},{"interface":"front","average":{"1min":1.302,"5min":1.3380000000000001,"15min":1.327},"min":{"1min":0.97199999999999998,"5min":0.97199999999999998,"15min":0.54100000000000004},"max":{"1min":1.5780000000000001,"5min":2.0259999999999998,"15min":2.0259999999999998},"last":1.024}]},{"osd":4,"last update":"Wed Aug 9 15:07:53 2023","interfaces":[{"interface":"back","average":{"1min":1.4890000000000001,"5min":1.5329999999999999,"15min":1.496},"min":{"1min":1.181,"5min":0.97599999999999998,"15min":0.751},"max":{"1min":1.8799999999999999,"5min":2.214,"15min":2.214},"last":1.1619999999999999},{"interface":"front","average":{"1min":1.425,"5min":1.452,"15min":1.4219999999999999},"min":{"1min":1.046,"5min":0.90600000000000003,"15min":0.90600000000000003},"max":{"1min":1.9630000000000001,"5min":2.1190000000000002,"15min":2.1190000000000002},"last":1.9730000000000001}]},{"osd":5,"last update":"Wed Aug 9 15:07:53 2023","interfaces":[{"interface":"back","average":{"1min":1.446,"5min":1.462,"15min":1.456},"min":{"1min":1.236,"5min":1.0389999999999999,"15min":0.86099999999999999},"max":{"1min":1.7829999999999999,"5min":2.508,"15min":2.508},"last":1.6870000000000001},{"interface":"front","average":{"1min":1.595,"5min":1.661,"15min":1.609},"min":{"1min":1.294,"5min":1.1339999999999999,"15min":0.78700000000000003},"max":{"1min":1.9870000000000001,"5min":2.2480000000000002,"15min":2.2480000000000002},"last":1.821}]},{"osd":6,"last update":"Wed Aug 9 15:07:53 2023","interfaces":[{"interface":"back","average":{"1min":1.6479999999999999,"5min":1.6779999999999999,"15min":1.631},"min":{"1min":1.2929999999999999,"5min":1.282,"15min":0.997},"max":{"1min":2.1400000000000001,"5min":2.294,"15min":2.294},"last":1.3029999999999999},{"interface":"front","average":{"1min":1.542,"5min":1.599,"15min":1.589},"min":{"1min":1.155,"5min":0.98999999999999999,"15min":0.91000000000000003},"max":{"1min":1.8220000000000001,"5min":2.278,"15min":2.5270000000000001},"last":1.8420000000000001}]},{"osd":7,"last update":"Wed Aug 9 15:07:53 2023","interfaces":[{"interface":"back","average":{"1min":1.75,"5min":1.782,"15min":1.72},"min":{"1min":1.399,"5min":1.26,"15min":0.92700000000000005},"max":{"1min":2.165,"5min":2.3050000000000002,"15min":2.488},"last":1.625},{"interface":"front","average":{"1min":1.645,"5min":1.776,"15min":1.714},"min":{"1min":1.254,"5min":1.254,"15min":1.0469999999999999},"max":{"1min":2.1040000000000001,"5min":2.4249999999999998,"15min":2.4249999999999998},"last":1.429}]}]},{"osd":0,"up_from":13,"seq":55834575981,"num_pgs":7,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":29512,"kb_used_data":2492,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94342328,"statfs":{"total":96636764160,"available":96606543872,"internally_reserved":0,"allocated":2551808,"data_stored":1387020,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1781,"internal_metadata":27654411},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Wed Aug 9 15:07:52 2023","interfaces":[{"interface":"back","average":{"1min":0.88100000000000001,"5min":0.93000000000000005,"15min":0.97299999999999998},"min":{"1min":0.69099999999999995,"5min":0.68799999999999994,"15min":0.51100000000000001},"max":{"1min":1.47,"5min":1.569,"15min":1.7230000000000001},"last":1.1859999999999999},{"interface":"front","average":{"1min":1.1890000000000001,"5min":1.175,"15min":1.1579999999999999},"min":{"1min":0.86299999999999999,"5min":0.76000000000000001,"15min":0.44},"max":{"1min":1.46,"5min":1.885,"15min":1.9410000000000001},"last":0.76100000000000001}]},{"osd":2,"last update":"Wed Aug 9 15:07:52 2023","interfaces":[{"interface":"back","average":{"1min":1.141,"5min":1.2090000000000001,"15min":1.198},"min":{"1min":0.81999999999999995,"5min":0.81999999999999995,"15min":0.61599999999999999},"max":{"1min":1.9039999999999999,"5min":1.9039999999999999,"15min":1.9039999999999999},"last":1.3360000000000001},{"interface":"front","average":{"1min":1.167,"5min":1.26,"15min":1.254},"min":{"1min":0.93999999999999995,"5min":0.79000000000000004,"15min":0.59699999999999998},"max":{"1min":1.458,"5min":1.9470000000000001,"15min":1.9470000000000001},"last":0.70899999999999996}]},{"osd":3,"last update":"Wed Aug 9 15:07:52 2023","interfaces":[{"interface":"back","average":{"1min":1.198,"5min":1.2290000000000001,"15min":1.224},"min":{"1min":0.93999999999999995,"5min":0.82499999999999996,"15min":0.46200000000000002},"max":{"1min":1.6200000000000001,"5min":1.9079999999999999,"15min":1.9079999999999999},"last":1.01},{"interface":"front","average":{"1min":1.3839999999999999,"5min":1.419,"15min":1.407},"min":{"1min":1.0129999999999999,"5min":1.0129999999999999,"15min":0.56599999999999995},"max":{"1min":1.8009999999999999,"5min":1.893,"15min":2.2469999999999999},"last":1.5}]},{"osd":4,"last update":"Wed Aug 9 15:07:52 2023","interfaces":[{"interface":"back","average":{"1min":1.4159999999999999,"5min":1.452,"15min":1.4139999999999999},"min":{"1min":1.1040000000000001,"5min":1.1040000000000001,"15min":0.61599999999999999},"max":{"1min":1.8049999999999999,"5min":1.966,"15min":1.966},"last":1.3759999999999999},{"interface":"front","average":{"1min":1.472,"5min":1.5509999999999999,"15min":1.5309999999999999},"min":{"1min":1.321,"5min":1.022,"15min":0.69499999999999995},"max":{"1min":1.748,"5min":2.0369999999999999,"15min":2.278},"last":1.613}]},{"osd":5,"last update":"Wed Aug 9 15:07:52 2023","interfaces":[{"interface":"back","average":{"1min":1.5800000000000001,"5min":1.54,"15min":1.538},"min":{"1min":1.276,"5min":0.98299999999999998,"15min":0.73499999999999999},"max":{"1min":1.9710000000000001,"5min":1.9950000000000001,"15min":2.3180000000000001},"last":1.264},{"interface":"front","average":{"1min":1.5169999999999999,"5min":1.5449999999999999,"15min":1.512},"min":{"1min":1.3480000000000001,"5min":1.0860000000000001,"15min":0.67100000000000004},"max":{"1min":1.73,"5min":2.3740000000000001,"15min":2.3740000000000001},"last":1.302}]},{"osd":6,"last update":"Wed Aug 9 15:07:52 2023","interfaces":[{"interface":"back","average":{"1min":1.579,"5min":1.6579999999999999,"15min":1.647},"min":{"1min":1.413,"5min":1.2989999999999999,"15min":0.82199999999999995},"max":{"1min":1.7749999999999999,"5min":2.1320000000000001,"15min":2.3239999999999998},"last":1.71},{"interface":"front","average":{"1min":1.6799999999999999,"5min":1.667,"15min":1.633},"min":{"1min":1.4550000000000001,"5min":1.159,"15min":0.79600000000000004},"max":{"1min":1.8500000000000001,"5min":2.1859999999999999,"15min":2.2839999999999998},"last":1.3999999999999999}]},{"osd":7,"last update":"Wed Aug 9 15:07:52 2023","interfaces":[{"interface":"back","average":{"1min":1.704,"5min":1.712,"15min":1.6990000000000001},"min":{"1min":1.4590000000000001,"5min":1.244,"15min":0.89900000000000002},"max":{"1min":1.98,"5min":2.319,"15min":2.4140000000000001},"last":1.4570000000000001},{"interface":"front","average":{"1min":1.635,"5min":1.6739999999999999,"15min":1.631},"min":{"1min":1.288,"5min":1.131,"15min":0.71699999999999997},"max":{"1min":2.1139999999999999,"5min":2.3039999999999998,"15min":2.3759999999999999},"last":1.4330000000000001}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":24,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":22,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":24,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":24,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":694,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":24,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":194,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":24,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":2194,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":24,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":24,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":24,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":104,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":25,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":25,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":25,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":25,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":25,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":25,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":25,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":25,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-08-09T15:08:51.165 INFO:tasks.cephfs.filesystem.ceph_manager:clean! 2023-08-09T15:08:51.166 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth caps client.0 mon 'allow r' osd 'allow rw tag cephfs data=cephfs' mds allow 2023-08-09T15:08:51.445 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:51.445+0000 7f88d5dc8640 1 Processor -- start 2023-08-09T15:08:51.446 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:51.445+0000 7f88d5dc8640 1 -- start start 2023-08-09T15:08:51.447 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:51.447+0000 7f88d5dc8640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f88d00cb2d0 0x7f88d015b380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T15:08:51.447 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:51.447+0000 7f88d5dc8640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f88d00ce8d0 0x7f88d00c9f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T15:08:51.448 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:51.447+0000 7f88d5dc8640 1 -- --> v1:172.21.15.43:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f88d015aef0 con 0x7f88d015df60 2023-08-09T15:08:51.448 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:51.447+0000 7f88d5dc8640 1 -- --> v2:172.21.15.43:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f88d015a970 con 0x7f88d00ce8d0 2023-08-09T15:08:51.448 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:51.447+0000 7f88d5dc8640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f88d015b8c0 con 0x7f88d00cb2d0 2023-08-09T15:08:51.448 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:51.448+0000 7f88ceffd640 1 --1- >> v1:172.21.15.43:6789/0 conn(0x7f88d015df60 0x7f88d00ce1c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.43:6789/0 says I am v1:172.21.15.43:42168/0 (socket says 172.21.15.43:42168) 2023-08-09T15:08:51.449 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:51.448+0000 7f88ceffd640 1 -- 172.21.15.43:0/1135210034 learned_addr learned my addr 172.21.15.43:0/1135210034 (peer_addr_for_me v1:172.21.15.43:0/0) 2023-08-09T15:08:51.449 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:51.448+0000 7f88cffff640 1 --2- 172.21.15.43:0/1135210034 >> v2:172.21.15.43:3300/0 conn(0x7f88d00ce8d0 0x7f88d00c9f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T15:08:51.449 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:51.448+0000 7f88cf7fe640 1 --2- 172.21.15.43:0/1135210034 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f88d00cb2d0 0x7f88d015b380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T15:08:51.449 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:51.448+0000 7f88cffff640 1 -- 172.21.15.43:0/1135210034 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f88d00cb2d0 msgr2=0x7f88d015b380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-09T15:08:51.449 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:51.448+0000 7f88cffff640 1 --2- 172.21.15.43:0/1135210034 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f88d00cb2d0 0x7f88d015b380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T15:08:51.449 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:51.448+0000 7f88cffff640 1 -- 172.21.15.43:0/1135210034 >> v1:172.21.15.43:6789/0 conn(0x7f88d015df60 legacy=0x7f88d00ce1c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T15:08:51.449 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:51.448+0000 7f88ce7fc640 1 -- 172.21.15.43:0/1135210034 <== mon.0 v1:172.21.15.43:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2520796736 0 0) 0x7f88d015aef0 con 0x7f88d015df60 2023-08-09T15:08:51.450 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:51.448+0000 7f88cffff640 1 -- 172.21.15.43:0/1135210034 --> v2:172.21.15.43:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f88d015e940 con 0x7f88d00ce8d0 2023-08-09T15:08:51.450 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:51.450+0000 7f88cffff640 1 --2- 172.21.15.43:0/1135210034 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f88d00ce8d0 0x7f88d00c9f90 secure :-1 s=READY pgs=2234 cs=0 l=1 rev1=1 crypto rx=0x7f88c8009d90 tx=0x7f88c802beb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=809cf8b980a92197 server_cookie=0 in_seq=0 out_seq=0 2023-08-09T15:08:51.450 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:51.450+0000 7f88ce7fc640 1 -- 172.21.15.43:0/1135210034 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f88c802cc60 con 0x7f88d00ce8d0 2023-08-09T15:08:51.451 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:51.450+0000 7f88ce7fc640 1 -- 172.21.15.43:0/1135210034 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f88c802cdc0 con 0x7f88d00ce8d0 2023-08-09T15:08:51.451 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:51.450+0000 7f88ce7fc640 1 -- 172.21.15.43:0/1135210034 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f88c8034660 con 0x7f88d00ce8d0 2023-08-09T15:08:51.451 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:51.450+0000 7f88d5dc8640 1 -- 172.21.15.43:0/1135210034 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f88d00ce8d0 msgr2=0x7f88d00c9f90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T15:08:51.451 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:51.450+0000 7f88d5dc8640 1 --2- 172.21.15.43:0/1135210034 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f88d00ce8d0 0x7f88d00c9f90 secure :-1 s=READY pgs=2234 cs=0 l=1 rev1=1 crypto rx=0x7f88c8009d90 tx=0x7f88c802beb0 comp rx=0 tx=0).stop 2023-08-09T15:08:51.452 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:51.451+0000 7f88d5dc8640 1 -- 172.21.15.43:0/1135210034 shutdown_connections 2023-08-09T15:08:51.452 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:51.451+0000 7f88d5dc8640 1 --2- 172.21.15.43:0/1135210034 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f88d00ce8d0 0x7f88d00c9f90 unknown :-1 s=CLOSED pgs=2234 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T15:08:51.452 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:51.451+0000 7f88d5dc8640 1 --2- 172.21.15.43:0/1135210034 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f88d00cb2d0 0x7f88d015b380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T15:08:51.452 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:51.451+0000 7f88d5dc8640 1 -- 172.21.15.43:0/1135210034 >> 172.21.15.43:0/1135210034 conn(0x7f88d00aff50 msgr2=0x7f88d00b0360 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-09T15:08:51.452 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:51.451+0000 7f88d5dc8640 1 -- 172.21.15.43:0/1135210034 shutdown_connections 2023-08-09T15:08:51.452 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:51.451+0000 7f88d5dc8640 1 -- 172.21.15.43:0/1135210034 wait complete. 2023-08-09T15:08:51.453 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:51.452+0000 7f88d5dc8640 1 Processor -- start 2023-08-09T15:08:51.453 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:51.453+0000 7f88d5dc8640 1 -- start start 2023-08-09T15:08:51.454 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:51.454+0000 7f88d5dc8640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f88d00cb2d0 0x7f88d00b3470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T15:08:51.454 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:51.454+0000 7f88d5dc8640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f88d00ce8d0 0x7f88d0151ba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T15:08:51.454 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:51.454+0000 7f88d5dc8640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f88d015df60 0x7f88d01f48d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T15:08:51.454 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:51.454+0000 7f88d5dc8640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f88d00b1a70 con 0x7f88d00ce8d0 2023-08-09T15:08:51.455 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:51.454+0000 7f88d5dc8640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f88d00ca260 con 0x7f88d015df60 2023-08-09T15:08:51.455 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:51.454+0000 7f88d5dc8640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f88d00b0df0 con 0x7f88d00cb2d0 2023-08-09T15:08:51.455 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:51.454+0000 7f88ceffd640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f88d00ce8d0 0x7f88d0151ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T15:08:51.455 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:51.454+0000 7f88ceffd640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f88d00ce8d0 0x7f88d0151ba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:39920/0 (socket says 172.21.15.43:39920) 2023-08-09T15:08:51.455 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:51.454+0000 7f88ceffd640 1 -- 172.21.15.43:0/355461796 learned_addr learned my addr 172.21.15.43:0/355461796 (peer_addr_for_me v2:172.21.15.43:0/0) 2023-08-09T15:08:51.456 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:51.454+0000 7f88cf7fe640 1 --2- 172.21.15.43:0/355461796 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f88d00cb2d0 0x7f88d00b3470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T15:08:51.456 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:51.454+0000 7f88cffff640 1 --2- 172.21.15.43:0/355461796 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f88d015df60 0x7f88d01f48d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T15:08:51.456 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:51.454+0000 7f88ceffd640 1 -- 172.21.15.43:0/355461796 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f88d00cb2d0 msgr2=0x7f88d00b3470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T15:08:51.457 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:51.454+0000 7f88ceffd640 1 --2- 172.21.15.43:0/355461796 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f88d00cb2d0 0x7f88d00b3470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T15:08:51.457 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:51.454+0000 7f88ceffd640 1 -- 172.21.15.43:0/355461796 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f88d015df60 msgr2=0x7f88d01f48d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T15:08:51.457 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:51.454+0000 7f88ceffd640 1 --2- 172.21.15.43:0/355461796 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f88d015df60 0x7f88d01f48d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T15:08:51.457 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:51.454+0000 7f88ceffd640 1 -- 172.21.15.43:0/355461796 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f88c8009a70 con 0x7f88d00ce8d0 2023-08-09T15:08:51.457 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:51.455+0000 7f88ceffd640 1 --2- 172.21.15.43:0/355461796 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f88d00ce8d0 0x7f88d0151ba0 secure :-1 s=READY pgs=2235 cs=0 l=1 rev1=1 crypto rx=0x7f88c400c460 tx=0x7f88c400c920 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-09T15:08:51.457 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:51.455+0000 7f88b7fff640 1 -- 172.21.15.43:0/355461796 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f88c40074e0 con 0x7f88d00ce8d0 2023-08-09T15:08:51.457 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:51.455+0000 7f88b7fff640 1 -- 172.21.15.43:0/355461796 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f88c4007660 con 0x7f88d00ce8d0 2023-08-09T15:08:51.458 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:51.455+0000 7f88d5dc8640 1 -- 172.21.15.43:0/355461796 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f88d00b10d0 con 0x7f88d00ce8d0 2023-08-09T15:08:51.458 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:51.455+0000 7f88b7fff640 1 -- 172.21.15.43:0/355461796 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f88c4004a20 con 0x7f88d00ce8d0 2023-08-09T15:08:51.458 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:51.455+0000 7f88d5dc8640 1 -- 172.21.15.43:0/355461796 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f88d00b1690 con 0x7f88d00ce8d0 2023-08-09T15:08:51.458 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:51.456+0000 7f88d5dc8640 1 -- 172.21.15.43:0/355461796 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f88d00b1bb0 con 0x7f88d00ce8d0 2023-08-09T15:08:51.459 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:51.457+0000 7f88b7fff640 1 -- 172.21.15.43:0/355461796 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 106) v1 ==== 88443+0+0 (secure 0 0 0) 0x7f88c4004b80 con 0x7f88d00ce8d0 2023-08-09T15:08:51.459 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:51.458+0000 7f88b7fff640 1 --2- 172.21.15.43:0/355461796 >> [v2:172.21.15.203:6832/3082732769,v1:172.21.15.203:6833/3082732769] conn(0x7f88bc06a170 0x7f88bc06c630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T15:08:51.459 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:51.459+0000 7f88b7fff640 1 -- 172.21.15.43:0/355461796 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(205..205 src has 1..205) v4 ==== 6426+0+0 (secure 0 0 0) 0x7f88c4087d60 con 0x7f88d00ce8d0 2023-08-09T15:08:51.459 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:51.459+0000 7f88cf7fe640 1 --2- 172.21.15.43:0/355461796 >> [v2:172.21.15.203:6832/3082732769,v1:172.21.15.203:6833/3082732769] conn(0x7f88bc06a170 0x7f88bc06c630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T15:08:51.460 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:51.460+0000 7f88cf7fe640 1 --2- 172.21.15.43:0/355461796 >> [v2:172.21.15.203:6832/3082732769,v1:172.21.15.203:6833/3082732769] conn(0x7f88bc06a170 0x7f88bc06c630 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f88c0009ad0 tx=0x7f88c0008040 comp rx=0 tx=0).ready entity=mgr.8500 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-09T15:08:51.463 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:51.463+0000 7f88b7fff640 1 -- 172.21.15.43:0/355461796 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178066 (secure 0 0 0) 0x7f88d00b1bb0 con 0x7f88d00ce8d0 2023-08-09T15:08:51.728 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:51.728+0000 7f88d5dc8640 1 -- 172.21.15.43:0/355461796 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]} v 0) v1 -- 0x7f88d00b1bb0 con 0x7f88d00ce8d0 2023-08-09T15:08:51.735 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:51.733+0000 7f88b7fff640 1 -- 172.21.15.43:0/355461796 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]}]=0 updated caps for client.0 v46) v1 ==== 190+0+147 (secure 0 0 0) 0x7f88d00b1bb0 con 0x7f88d00ce8d0 2023-08-09T15:08:51.736 INFO:teuthology.orchestra.run.smithi043.stdout:[client.0] 2023-08-09T15:08:51.736 INFO:teuthology.orchestra.run.smithi043.stdout: key = AQCPldNkg8zrDxAAM/hnJaE6S9HU/fRwQeBV5w== 2023-08-09T15:08:51.736 INFO:teuthology.orchestra.run.smithi043.stdout: caps mds = "allow" 2023-08-09T15:08:51.737 INFO:teuthology.orchestra.run.smithi043.stdout: caps mon = "allow r" 2023-08-09T15:08:51.737 INFO:teuthology.orchestra.run.smithi043.stdout: caps osd = "allow rw tag cephfs data=cephfs" 2023-08-09T15:08:51.738 INFO:teuthology.orchestra.run.smithi043.stderr:updated caps for client.0 2023-08-09T15:08:51.738 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:51.736+0000 7f88d5dc8640 1 -- 172.21.15.43:0/355461796 >> [v2:172.21.15.203:6832/3082732769,v1:172.21.15.203:6833/3082732769] conn(0x7f88bc06a170 msgr2=0x7f88bc06c630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T15:08:51.739 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:51.736+0000 7f88d5dc8640 1 --2- 172.21.15.43:0/355461796 >> [v2:172.21.15.203:6832/3082732769,v1:172.21.15.203:6833/3082732769] conn(0x7f88bc06a170 0x7f88bc06c630 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f88c0009ad0 tx=0x7f88c0008040 comp rx=0 tx=0).stop 2023-08-09T15:08:51.739 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:51.736+0000 7f88d5dc8640 1 -- 172.21.15.43:0/355461796 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f88d00ce8d0 msgr2=0x7f88d0151ba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T15:08:51.739 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:51.736+0000 7f88d5dc8640 1 --2- 172.21.15.43:0/355461796 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f88d00ce8d0 0x7f88d0151ba0 secure :-1 s=READY pgs=2235 cs=0 l=1 rev1=1 crypto rx=0x7f88c400c460 tx=0x7f88c400c920 comp rx=0 tx=0).stop 2023-08-09T15:08:51.739 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:51.737+0000 7f88d5dc8640 1 -- 172.21.15.43:0/355461796 shutdown_connections 2023-08-09T15:08:51.739 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:51.737+0000 7f88d5dc8640 1 --2- 172.21.15.43:0/355461796 >> [v2:172.21.15.203:6832/3082732769,v1:172.21.15.203:6833/3082732769] conn(0x7f88bc06a170 0x7f88bc06c630 unknown :-1 s=CLOSED pgs=68 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T15:08:51.739 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:51.737+0000 7f88d5dc8640 1 --2- 172.21.15.43:0/355461796 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f88d015df60 0x7f88d01f48d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T15:08:51.739 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:51.737+0000 7f88d5dc8640 1 --2- 172.21.15.43:0/355461796 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f88d00ce8d0 0x7f88d0151ba0 unknown :-1 s=CLOSED pgs=2235 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T15:08:51.739 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:51.737+0000 7f88d5dc8640 1 --2- 172.21.15.43:0/355461796 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f88d00cb2d0 0x7f88d00b3470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T15:08:51.740 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:51.737+0000 7f88d5dc8640 1 -- 172.21.15.43:0/355461796 >> 172.21.15.43:0/355461796 conn(0x7f88d00aff50 msgr2=0x7f88d00ce190 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-09T15:08:51.740 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:51.737+0000 7f88d5dc8640 1 -- 172.21.15.43:0/355461796 shutdown_connections 2023-08-09T15:08:51.740 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:51.737+0000 7f88d5dc8640 1 -- 172.21.15.43:0/355461796 wait complete. 2023-08-09T15:08:51.757 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-08-09T15:08:51.980 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:51.980+0000 7fa4c7fff640 1 Processor -- start 2023-08-09T15:08:51.981 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:51.980+0000 7fa4c7fff640 1 -- start start 2023-08-09T15:08:51.982 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:51.982+0000 7fa4c7fff640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fa4c815a6a0 0x7fa4c80713b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T15:08:51.982 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:51.982+0000 7fa4c7fff640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7fa4c80c96c0 0x7fa4c815ad20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T15:08:51.983 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:51.982+0000 7fa4c7fff640 1 -- --> v1:172.21.15.43:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa4c80cdc70 con 0x7fa4c80cbd90 2023-08-09T15:08:51.983 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:51.982+0000 7fa4c7fff640 1 -- --> v2:172.21.15.43:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa4c8070910 con 0x7fa4c80c96c0 2023-08-09T15:08:51.983 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:51.982+0000 7fa4c7fff640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa4c80c88c0 con 0x7fa4c815a6a0 2023-08-09T15:08:51.983 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:51.982+0000 7fa4c67fc640 1 --1- >> v1:172.21.15.43:6789/0 conn(0x7fa4c80cbd90 0x7fa4c80c8fb0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.43:6789/0 says I am v1:172.21.15.43:42180/0 (socket says 172.21.15.43:42180) 2023-08-09T15:08:51.983 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:51.982+0000 7fa4c67fc640 1 -- 172.21.15.43:0/550244128 learned_addr learned my addr 172.21.15.43:0/550244128 (peer_addr_for_me v1:172.21.15.43:0/0) 2023-08-09T15:08:51.984 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:51.982+0000 7fa4c77fe640 1 --2- 172.21.15.43:0/550244128 >> v2:172.21.15.43:3300/0 conn(0x7fa4c80c96c0 0x7fa4c815ad20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T15:08:51.985 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:51.983+0000 7fa4c6ffd640 1 --2- 172.21.15.43:0/550244128 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fa4c815a6a0 0x7fa4c80713b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T15:08:51.985 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:51.983+0000 7fa4c5ffb640 1 -- 172.21.15.43:0/550244128 <== mon.0 v1:172.21.15.43:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3447431761 0 0) 0x7fa4c80cdc70 con 0x7fa4c80cbd90 2023-08-09T15:08:51.985 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:51.983+0000 7fa4c5ffb640 1 -- 172.21.15.43:0/550244128 --> v1:172.21.15.43:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fa4b0003520 con 0x7fa4c80cbd90 2023-08-09T15:08:51.985 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:51.983+0000 7fa4c77fe640 1 -- 172.21.15.43:0/550244128 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fa4c815a6a0 msgr2=0x7fa4c80713b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-09T15:08:51.985 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:51.983+0000 7fa4c77fe640 1 --2- 172.21.15.43:0/550244128 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fa4c815a6a0 0x7fa4c80713b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T15:08:51.986 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:51.983+0000 7fa4c77fe640 1 -- 172.21.15.43:0/550244128 >> v1:172.21.15.43:6789/0 conn(0x7fa4c80cbd90 legacy=0x7fa4c80c8fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T15:08:51.986 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:51.983+0000 7fa4c77fe640 1 -- 172.21.15.43:0/550244128 --> v2:172.21.15.43:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa4c8070bb0 con 0x7fa4c80c96c0 2023-08-09T15:08:51.986 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:51.984+0000 7fa4c77fe640 1 --2- 172.21.15.43:0/550244128 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa4c80c96c0 0x7fa4c815ad20 secure :-1 s=READY pgs=2237 cs=0 l=1 rev1=1 crypto rx=0x7fa4c0009d90 tx=0x7fa4c002be10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4b58891e95c39233 server_cookie=0 in_seq=0 out_seq=0 2023-08-09T15:08:51.986 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:51.984+0000 7fa4c5ffb640 1 -- 172.21.15.43:0/550244128 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa4c002cbc0 con 0x7fa4c80c96c0 2023-08-09T15:08:51.986 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:51.984+0000 7fa4c5ffb640 1 -- 172.21.15.43:0/550244128 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa4c002cd20 con 0x7fa4c80c96c0 2023-08-09T15:08:51.986 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:51.984+0000 7fa4c5ffb640 1 -- 172.21.15.43:0/550244128 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa4c0034640 con 0x7fa4c80c96c0 2023-08-09T15:08:51.990 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:51.985+0000 7fa4c7fff640 1 -- 172.21.15.43:0/550244128 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa4c80c96c0 msgr2=0x7fa4c815ad20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T15:08:51.990 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:51.985+0000 7fa4c7fff640 1 --2- 172.21.15.43:0/550244128 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa4c80c96c0 0x7fa4c815ad20 secure :-1 s=READY pgs=2237 cs=0 l=1 rev1=1 crypto rx=0x7fa4c0009d90 tx=0x7fa4c002be10 comp rx=0 tx=0).stop 2023-08-09T15:08:51.990 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:51.985+0000 7fa4c7fff640 1 -- 172.21.15.43:0/550244128 shutdown_connections 2023-08-09T15:08:51.990 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:51.985+0000 7fa4c7fff640 1 --2- 172.21.15.43:0/550244128 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa4c80c96c0 0x7fa4c815ad20 unknown :-1 s=CLOSED pgs=2237 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T15:08:51.990 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:51.985+0000 7fa4c7fff640 1 --2- 172.21.15.43:0/550244128 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fa4c815a6a0 0x7fa4c80713b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T15:08:51.991 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:51.985+0000 7fa4c7fff640 1 -- 172.21.15.43:0/550244128 >> 172.21.15.43:0/550244128 conn(0x7fa4c80cde90 msgr2=0x7fa4c80ce2a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-09T15:08:51.991 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:51.985+0000 7fa4c7fff640 1 -- 172.21.15.43:0/550244128 shutdown_connections 2023-08-09T15:08:51.991 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:51.985+0000 7fa4c7fff640 1 -- 172.21.15.43:0/550244128 wait complete. 2023-08-09T15:08:51.991 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:51.987+0000 7fa4c7fff640 1 Processor -- start 2023-08-09T15:08:51.991 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:51.987+0000 7fa4c7fff640 1 -- start start 2023-08-09T15:08:51.991 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:51.988+0000 7fa4c7fff640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fa4c80c96c0 0x7fa4c814e170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T15:08:51.991 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:51.988+0000 7fa4c7fff640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fa4c80cbd90 0x7fa4c80c6900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T15:08:51.992 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:51.988+0000 7fa4c7fff640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa4c815a6a0 0x7fa4c80c4f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T15:08:51.992 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:51.988+0000 7fa4c7fff640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa4c80c6470 con 0x7fa4c815a6a0 2023-08-09T15:08:51.992 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:51.988+0000 7fa4c7fff640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa4c80c5860 con 0x7fa4c80cbd90 2023-08-09T15:08:51.992 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:51.988+0000 7fa4c7fff640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa4c80c5b40 con 0x7fa4c80c96c0 2023-08-09T15:08:51.992 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:51.988+0000 7fa4c77fe640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa4c815a6a0 0x7fa4c80c4f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T15:08:51.993 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:51.988+0000 7fa4c77fe640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa4c815a6a0 0x7fa4c80c4f60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:39952/0 (socket says 172.21.15.43:39952) 2023-08-09T15:08:51.993 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:51.988+0000 7fa4c77fe640 1 -- 172.21.15.43:0/1394051161 learned_addr learned my addr 172.21.15.43:0/1394051161 (peer_addr_for_me v2:172.21.15.43:0/0) 2023-08-09T15:08:51.993 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:51.988+0000 7fa4c6ffd640 1 --2- 172.21.15.43:0/1394051161 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fa4c80c96c0 0x7fa4c814e170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T15:08:51.993 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:51.988+0000 7fa4c77fe640 1 -- 172.21.15.43:0/1394051161 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fa4c80c96c0 msgr2=0x7fa4c814e170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T15:08:51.994 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:51.988+0000 7fa4c77fe640 1 --2- 172.21.15.43:0/1394051161 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fa4c80c96c0 0x7fa4c814e170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T15:08:51.994 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:51.988+0000 7fa4c77fe640 1 -- 172.21.15.43:0/1394051161 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fa4c80cbd90 msgr2=0x7fa4c80c6900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T15:08:51.994 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:51.988+0000 7fa4c77fe640 1 --2- 172.21.15.43:0/1394051161 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fa4c80cbd90 0x7fa4c80c6900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T15:08:51.994 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:51.988+0000 7fa4c77fe640 1 -- 172.21.15.43:0/1394051161 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa4c0009a70 con 0x7fa4c815a6a0 2023-08-09T15:08:51.994 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:51.989+0000 7fa4c77fe640 1 --2- 172.21.15.43:0/1394051161 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa4c815a6a0 0x7fa4c80c4f60 secure :-1 s=READY pgs=2238 cs=0 l=1 rev1=1 crypto rx=0x7fa4c0037040 tx=0x7fa4c0004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-09T15:08:51.994 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:51.989+0000 7fa4af7fe640 1 -- 172.21.15.43:0/1394051161 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa4c0004800 con 0x7fa4c815a6a0 2023-08-09T15:08:51.994 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:51.989+0000 7fa4af7fe640 1 -- 172.21.15.43:0/1394051161 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa4c002ce90 con 0x7fa4c815a6a0 2023-08-09T15:08:51.995 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:51.989+0000 7fa4af7fe640 1 -- 172.21.15.43:0/1394051161 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa4c0034680 con 0x7fa4c815a6a0 2023-08-09T15:08:51.995 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:51.989+0000 7fa4c7fff640 1 -- 172.21.15.43:0/1394051161 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa4c80c5e20 con 0x7fa4c815a6a0 2023-08-09T15:08:51.995 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:51.989+0000 7fa4c7fff640 1 -- 172.21.15.43:0/1394051161 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa4c814e6b0 con 0x7fa4c815a6a0 2023-08-09T15:08:51.996 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:51.990+0000 7fa4c7fff640 1 -- 172.21.15.43:0/1394051161 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa4c80c65e0 con 0x7fa4c815a6a0 2023-08-09T15:08:51.996 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:51.991+0000 7fa4af7fe640 1 -- 172.21.15.43:0/1394051161 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 106) v1 ==== 88443+0+0 (secure 0 0 0) 0x7fa4c00347e0 con 0x7fa4c815a6a0 2023-08-09T15:08:51.996 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:51.992+0000 7fa4af7fe640 1 --2- 172.21.15.43:0/1394051161 >> [v2:172.21.15.203:6832/3082732769,v1:172.21.15.203:6833/3082732769] conn(0x7fa4b406a170 0x7fa4b406c630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T15:08:51.996 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:51.992+0000 7fa4af7fe640 1 -- 172.21.15.43:0/1394051161 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(205..205 src has 1..205) v4 ==== 6426+0+0 (secure 0 0 0) 0x7fa4c00aac20 con 0x7fa4c815a6a0 2023-08-09T15:08:51.996 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:51.993+0000 7fa4c6ffd640 1 --2- 172.21.15.43:0/1394051161 >> [v2:172.21.15.203:6832/3082732769,v1:172.21.15.203:6833/3082732769] conn(0x7fa4b406a170 0x7fa4b406c630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T15:08:51.997 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:51.994+0000 7fa4c6ffd640 1 --2- 172.21.15.43:0/1394051161 >> [v2:172.21.15.203:6832/3082732769,v1:172.21.15.203:6833/3082732769] conn(0x7fa4b406a170 0x7fa4b406c630 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7fa4b8006fd0 tx=0x7fa4b8008040 comp rx=0 tx=0).ready entity=mgr.8500 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-09T15:08:51.997 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:51.996+0000 7fa4af7fe640 1 -- 172.21.15.43:0/1394051161 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178066 (secure 0 0 0) 0x7fa4c0075ff0 con 0x7fa4c815a6a0 2023-08-09T15:08:52.055 INFO:tasks.ceph.mds.a.smithi043.stderr:2023-08-09T15:08:52.055+0000 7f2ad61b0640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2023-08-09T15:08:52.231 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:52.230+0000 7fa4c7fff640 1 -- 172.21.15.43:0/1394051161 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fa4c80739f0 con 0x7fa4c815a6a0 2023-08-09T15:08:52.232 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:52.231+0000 7fa4af7fe640 1 -- 172.21.15.43:0/1394051161 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 96 v96) v1 ==== 94+0+4883 (secure 0 0 0) 0x7fa4c0009450 con 0x7fa4c815a6a0 2023-08-09T15:08:52.232 INFO:teuthology.orchestra.run.smithi043.stdout: 2023-08-09T15:08:52.232 INFO:teuthology.orchestra.run.smithi043.stdout:{"epoch":96,"default_fscid":11,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":7703,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.203:6836/1774059384","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6835","nonce":1774059384},{"type":"v1","addr":"172.21.15.203:6836","nonce":1774059384}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":91},{"gid":8665,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.203:6837/682905899","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6834","nonce":682905899},{"type":"v1","addr":"172.21.15.203:6837","nonce":682905899}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":91},{"gid":10752,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.43:6834/4168463642","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6832","nonce":4168463642},{"type":"v1","addr":"172.21.15.43:6834","nonce":4168463642}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":90}],"filesystems":[{"mdsmap":{"epoch":96,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-08-09T15:08:45.563041+0000","modified":"2023-08-09T15:08:49.150213+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":10755},"failed":[],"damaged":[],"stopped":[],"info":{"gid_10755":{"gid":10755,"name":"a","rank":0,"incarnation":92,"state":"up:active","state_seq":3,"addr":"172.21.15.43:6835/349688227","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6833","nonce":349688227},{"type":"v1","addr":"172.21.15.43:6835","nonce":349688227}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[25],"metadata_pool":24,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":11}]} 2023-08-09T15:08:52.233 INFO:teuthology.orchestra.run.smithi043.stderr:dumped fsmap epoch 96 2023-08-09T15:08:52.235 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:52.234+0000 7fa4c7fff640 1 -- 172.21.15.43:0/1394051161 >> [v2:172.21.15.203:6832/3082732769,v1:172.21.15.203:6833/3082732769] conn(0x7fa4b406a170 msgr2=0x7fa4b406c630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T15:08:52.235 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:52.234+0000 7fa4c7fff640 1 --2- 172.21.15.43:0/1394051161 >> [v2:172.21.15.203:6832/3082732769,v1:172.21.15.203:6833/3082732769] conn(0x7fa4b406a170 0x7fa4b406c630 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7fa4b8006fd0 tx=0x7fa4b8008040 comp rx=0 tx=0).stop 2023-08-09T15:08:52.235 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:52.234+0000 7fa4c7fff640 1 -- 172.21.15.43:0/1394051161 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa4c815a6a0 msgr2=0x7fa4c80c4f60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T15:08:52.235 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:52.234+0000 7fa4c7fff640 1 --2- 172.21.15.43:0/1394051161 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa4c815a6a0 0x7fa4c80c4f60 secure :-1 s=READY pgs=2238 cs=0 l=1 rev1=1 crypto rx=0x7fa4c0037040 tx=0x7fa4c0004a40 comp rx=0 tx=0).stop 2023-08-09T15:08:52.236 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:52.235+0000 7fa4c7fff640 1 -- 172.21.15.43:0/1394051161 shutdown_connections 2023-08-09T15:08:52.236 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:52.235+0000 7fa4c7fff640 1 --2- 172.21.15.43:0/1394051161 >> [v2:172.21.15.203:6832/3082732769,v1:172.21.15.203:6833/3082732769] conn(0x7fa4b406a170 0x7fa4b406c630 unknown :-1 s=CLOSED pgs=69 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T15:08:52.236 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:52.235+0000 7fa4c7fff640 1 --2- 172.21.15.43:0/1394051161 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa4c815a6a0 0x7fa4c80c4f60 unknown :-1 s=CLOSED pgs=2238 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T15:08:52.236 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:52.235+0000 7fa4c7fff640 1 --2- 172.21.15.43:0/1394051161 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fa4c80cbd90 0x7fa4c80c6900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T15:08:52.236 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:52.235+0000 7fa4c7fff640 1 --2- 172.21.15.43:0/1394051161 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fa4c80c96c0 0x7fa4c814e170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T15:08:52.236 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:52.235+0000 7fa4c7fff640 1 -- 172.21.15.43:0/1394051161 >> 172.21.15.43:0/1394051161 conn(0x7fa4c80cde90 msgr2=0x7fa4c80acfb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-09T15:08:52.236 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:52.235+0000 7fa4c7fff640 1 -- 172.21.15.43:0/1394051161 shutdown_connections 2023-08-09T15:08:52.237 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:52.235+0000 7fa4c7fff640 1 -- 172.21.15.43:0/1394051161 wait complete. 2023-08-09T15:08:52.659 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 96, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2023-08-09T15:08:45.563041+0000', 'modified': '2023-08-09T15:08:49.150213+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 10755}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_10755': {'gid': 10755, 'name': 'a', 'rank': 0, 'incarnation': 92, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.43:6835/349688227', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.43:6833', 'nonce': 349688227}, {'type': 'v1', 'addr': '172.21.15.43:6835', 'nonce': 349688227}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138322906710015, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}}, 'data_pools': [25], 'metadata_pool': 24, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0} 2023-08-09T15:08:52.660 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2023-08-09T15:08:52.660 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.a status 2023-08-09T15:08:52.904 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:52.904+0000 7f2918401640 1 Processor -- start 2023-08-09T15:08:52.904 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:52.904+0000 7f2918401640 1 -- start start 2023-08-09T15:08:52.906 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:52.906+0000 7f2918401640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f291015c550 0x7f29100d1a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T15:08:52.906 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:52.906+0000 7f2918401640 1 -- --> v1:172.21.15.43:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f29100d0700 con 0x7f29100cf550 2023-08-09T15:08:52.906 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:52.906+0000 7f2918401640 1 -- --> v1:172.21.15.203:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f29100d09e0 con 0x7f291015c930 2023-08-09T15:08:52.906 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:52.906+0000 7f2918401640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f29100d0cc0 con 0x7f291015c550 2023-08-09T15:08:52.907 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:52.906+0000 7f291699e640 1 --1- >> v1:172.21.15.43:6789/0 conn(0x7f29100cf550 0x7f29100b5460 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.43:6789/0 says I am v1:172.21.15.43:42186/0 (socket says 172.21.15.43:42186) 2023-08-09T15:08:52.907 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:52.906+0000 7f291699e640 1 -- 172.21.15.43:0/751959874 learned_addr learned my addr 172.21.15.43:0/751959874 (peer_addr_for_me v1:172.21.15.43:0/0) 2023-08-09T15:08:52.907 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:52.906+0000 7f291619d640 1 --2- 172.21.15.43:0/751959874 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f291015c550 0x7f29100d1a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T15:08:52.907 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:52.907+0000 7f291519b640 1 -- 172.21.15.43:0/751959874 <== mon.0 v1:172.21.15.43:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3897686622 0 0) 0x7f29100d0700 con 0x7f29100cf550 2023-08-09T15:08:52.907 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:52.907+0000 7f291519b640 1 -- 172.21.15.43:0/751959874 --> v1:172.21.15.43:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f28fc003520 con 0x7f29100cf550 2023-08-09T15:08:52.909 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:52.907+0000 7f291519b640 1 -- 172.21.15.43:0/751959874 <== mon.0 v1:172.21.15.43:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1719159811 0 0) 0x7f28fc003520 con 0x7f29100cf550 2023-08-09T15:08:52.909 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:52.907+0000 7f291519b640 1 -- 172.21.15.43:0/751959874 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f291015c550 msgr2=0x7f29100d1a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-09T15:08:52.909 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:52.907+0000 7f291519b640 1 --2- 172.21.15.43:0/751959874 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f291015c550 0x7f29100d1a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T15:08:52.909 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:52.907+0000 7f291519b640 1 -- 172.21.15.43:0/751959874 >> v1:172.21.15.203:6789/0 conn(0x7f291015c930 legacy=0x7f29100cee40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T15:08:52.909 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:52.907+0000 7f291519b640 1 -- 172.21.15.43:0/751959874 --> v1:172.21.15.43:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f29100d0fa0 con 0x7f29100cf550 2023-08-09T15:08:52.909 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:52.907+0000 7f291519b640 1 -- 172.21.15.43:0/751959874 <== mon.0 v1:172.21.15.43:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1285150740 0 0) 0x7f2908001670 con 0x7f29100cf550 2023-08-09T15:08:52.910 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:52.907+0000 7f291519b640 1 -- 172.21.15.43:0/751959874 >> v1:172.21.15.43:6789/0 conn(0x7f29100cf550 legacy=0x7f29100b5460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T15:08:52.910 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:52.907+0000 7f291519b640 1 --2- 172.21.15.43:0/751959874 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f28fc003a10 0x7f28fc003e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T15:08:52.910 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:52.907+0000 7f291519b640 1 --2- 172.21.15.43:0/751959874 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f28fc0047c0 0x7f28fc004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T15:08:52.910 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:52.907+0000 7f291519b640 1 --2- 172.21.15.43:0/751959874 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f291015c550 0x7f28fc005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T15:08:52.910 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:52.907+0000 7f291519b640 1 -- 172.21.15.43:0/751959874 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f29100d0cc0 con 0x7f28fc0047c0 2023-08-09T15:08:52.910 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:52.907+0000 7f291519b640 1 -- 172.21.15.43:0/751959874 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f28fc00bb10 con 0x7f28fc003a10 2023-08-09T15:08:52.910 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:52.907+0000 7f291519b640 1 -- 172.21.15.43:0/751959874 --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f28fc00bcd0 con 0x7f291015c550 2023-08-09T15:08:52.910 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:52.908+0000 7f291599c640 1 --2- 172.21.15.43:0/751959874 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f28fc0047c0 0x7f28fc004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T15:08:52.911 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:52.908+0000 7f291619d640 1 --2- 172.21.15.43:0/751959874 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f28fc003a10 0x7f28fc003e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T15:08:52.911 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:52.908+0000 7f291699e640 1 --2- 172.21.15.43:0/751959874 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f291015c550 0x7f28fc005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T15:08:52.911 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:52.908+0000 7f291599c640 1 -- 172.21.15.43:0/751959874 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f291015c550 msgr2=0x7f28fc005180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-09T15:08:52.911 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:52.908+0000 7f291599c640 1 --2- 172.21.15.43:0/751959874 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f291015c550 0x7f28fc005180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T15:08:52.911 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:52.908+0000 7f291599c640 1 -- 172.21.15.43:0/751959874 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f28fc003a10 msgr2=0x7f28fc003e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-09T15:08:52.912 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:52.908+0000 7f291599c640 1 --2- 172.21.15.43:0/751959874 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f28fc003a10 0x7f28fc003e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T15:08:52.912 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:52.908+0000 7f291599c640 1 -- 172.21.15.43:0/751959874 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f28fc003520 con 0x7f28fc0047c0 2023-08-09T15:08:52.913 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:52.909+0000 7f291599c640 1 --2- 172.21.15.43:0/751959874 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f28fc0047c0 0x7f28fc004c40 secure :-1 s=READY pgs=2240 cs=0 l=1 rev1=1 crypto rx=0x7f2900001720 tx=0x7f28fc0057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=67fda08958783583 server_cookie=0 in_seq=0 out_seq=0 2023-08-09T15:08:52.914 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:52.910+0000 7f291519b640 1 -- 172.21.15.43:0/751959874 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f29000052d0 con 0x7f28fc0047c0 2023-08-09T15:08:52.914 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:52.910+0000 7f291519b640 1 -- 172.21.15.43:0/751959874 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f290002e050 con 0x7f28fc0047c0 2023-08-09T15:08:52.914 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:52.910+0000 7f291519b640 1 -- 172.21.15.43:0/751959874 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f29000035f0 con 0x7f28fc0047c0 2023-08-09T15:08:52.914 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:52.910+0000 7f2918401640 1 -- 172.21.15.43:0/751959874 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f28fc0047c0 msgr2=0x7f28fc004c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T15:08:52.914 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:52.910+0000 7f2918401640 1 --2- 172.21.15.43:0/751959874 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f28fc0047c0 0x7f28fc004c40 secure :-1 s=READY pgs=2240 cs=0 l=1 rev1=1 crypto rx=0x7f2900001720 tx=0x7f28fc0057f0 comp rx=0 tx=0).stop 2023-08-09T15:08:52.914 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:52.910+0000 7f291699e640 1 -- 172.21.15.43:0/751959874 reap_dead start 2023-08-09T15:08:52.914 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:52.910+0000 7f2918401640 1 -- 172.21.15.43:0/751959874 shutdown_connections 2023-08-09T15:08:52.915 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:52.910+0000 7f2918401640 1 -- 172.21.15.43:0/751959874 >> 172.21.15.43:0/751959874 conn(0x7f291006a4e0 msgr2=0x7f291006a8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-09T15:08:52.915 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:52.910+0000 7f2918401640 1 -- 172.21.15.43:0/751959874 shutdown_connections 2023-08-09T15:08:52.915 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:52.910+0000 7f2918401640 1 -- 172.21.15.43:0/751959874 wait complete. 2023-08-09T15:08:52.916 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:52.912+0000 7f2918401640 1 Processor -- start 2023-08-09T15:08:52.916 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:52.912+0000 7f2918401640 1 -- start start 2023-08-09T15:08:52.916 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:52.913+0000 7f2918401640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f28fc0047c0 0x7f29100c5f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T15:08:52.916 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:52.913+0000 7f2918401640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f29100c8280 0x7f29100b1120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T15:08:52.917 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:52.913+0000 7f2918401640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f29100d2660 0x7f29100d2ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T15:08:52.917 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:52.913+0000 7f2918401640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f29101f1750 con 0x7f28fc0047c0 2023-08-09T15:08:52.917 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:52.913+0000 7f2918401640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f29100d3020 con 0x7f29100c8280 2023-08-09T15:08:52.917 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:52.913+0000 7f2918401640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f29100c8700 con 0x7f29100d2660 2023-08-09T15:08:52.920 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:52.914+0000 7f291619d640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f28fc0047c0 0x7f29100c5f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T15:08:52.920 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:52.914+0000 7f291619d640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f28fc0047c0 0x7f29100c5f90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:39970/0 (socket says 172.21.15.43:39970) 2023-08-09T15:08:52.921 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:52.914+0000 7f291619d640 1 -- 172.21.15.43:0/1114929334 learned_addr learned my addr 172.21.15.43:0/1114929334 (peer_addr_for_me v2:172.21.15.43:0/0) 2023-08-09T15:08:52.921 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:52.914+0000 7f291599c640 1 --2- 172.21.15.43:0/1114929334 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f29100c8280 0x7f29100b1120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T15:08:52.921 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:52.914+0000 7f291699e640 1 --2- 172.21.15.43:0/1114929334 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f29100d2660 0x7f29100d2ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T15:08:52.921 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:52.914+0000 7f291619d640 1 -- 172.21.15.43:0/1114929334 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f29100d2660 msgr2=0x7f29100d2ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T15:08:52.921 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:52.914+0000 7f291619d640 1 --2- 172.21.15.43:0/1114929334 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f29100d2660 0x7f29100d2ae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T15:08:52.921 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:52.914+0000 7f291619d640 1 -- 172.21.15.43:0/1114929334 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f29100c8280 msgr2=0x7f29100b1120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T15:08:52.921 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:52.914+0000 7f291619d640 1 --2- 172.21.15.43:0/1114929334 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f29100c8280 0x7f29100b1120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T15:08:52.922 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:52.914+0000 7f291619d640 1 -- 172.21.15.43:0/1114929334 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f29100c6620 con 0x7f28fc0047c0 2023-08-09T15:08:52.922 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:52.915+0000 7f291619d640 1 --2- 172.21.15.43:0/1114929334 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f28fc0047c0 0x7f29100c5f90 secure :-1 s=READY pgs=2241 cs=0 l=1 rev1=1 crypto rx=0x7f290c007f30 tx=0x7f290c011660 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-09T15:08:52.922 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:52.915+0000 7f2906ffd640 1 -- 172.21.15.43:0/1114929334 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f290c00e030 con 0x7f28fc0047c0 2023-08-09T15:08:52.922 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:52.915+0000 7f2906ffd640 1 -- 172.21.15.43:0/1114929334 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f290c016470 con 0x7f28fc0047c0 2023-08-09T15:08:52.922 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:52.915+0000 7f2918401640 1 -- 172.21.15.43:0/1114929334 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f29100c6900 con 0x7f28fc0047c0 2023-08-09T15:08:52.923 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:52.915+0000 7f2906ffd640 1 -- 172.21.15.43:0/1114929334 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f290c016820 con 0x7f28fc0047c0 2023-08-09T15:08:52.923 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:52.915+0000 7f2918401640 1 -- 172.21.15.43:0/1114929334 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f29100c6ec0 con 0x7f28fc0047c0 2023-08-09T15:08:52.923 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:52.917+0000 7f2906ffd640 1 -- 172.21.15.43:0/1114929334 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 106) v1 ==== 88443+0+0 (secure 0 0 0) 0x7f290c016980 con 0x7f28fc0047c0 2023-08-09T15:08:52.923 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:52.919+0000 7f2906ffd640 1 --2- 172.21.15.43:0/1114929334 >> [v2:172.21.15.203:6832/3082732769,v1:172.21.15.203:6833/3082732769] conn(0x7f28f406a170 0x7f28f406c630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T15:08:52.923 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:52.919+0000 7f2906ffd640 1 -- 172.21.15.43:0/1114929334 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(205..205 src has 1..205) v4 ==== 6426+0+0 (secure 0 0 0) 0x7f290c00f070 con 0x7f28fc0047c0 2023-08-09T15:08:52.924 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:52.920+0000 7f291af76740 1 Processor -- start 2023-08-09T15:08:52.924 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:52.920+0000 7f291599c640 1 --2- 172.21.15.43:0/1114929334 >> [v2:172.21.15.203:6832/3082732769,v1:172.21.15.203:6833/3082732769] conn(0x7f28f406a170 0x7f28f406c630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T15:08:52.924 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:52.920+0000 7f291af76740 1 -- start start 2023-08-09T15:08:52.926 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:52.921+0000 7f291af76740 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x559c55d2fc50 0x559c55d30030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T15:08:52.926 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:52.921+0000 7f291af76740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x559c55cffbd0 0x559c55d2cec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T15:08:52.926 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:52.921+0000 7f291af76740 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x559c55d037d0 0x559c55d2d400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T15:08:52.926 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:52.921+0000 7f291af76740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x559c55c718f0 con 0x559c55cffbd0 2023-08-09T15:08:52.926 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:52.921+0000 7f291af76740 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x559c55c6efe0 con 0x559c55d2fc50 2023-08-09T15:08:52.926 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:52.921+0000 7f291af76740 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x559c55c6a8d0 con 0x559c55d037d0 2023-08-09T15:08:52.926 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:52.921+0000 7f291619d640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x559c55d2fc50 0x559c55d30030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T15:08:52.927 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:52.922+0000 7f291699e640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x559c55d037d0 0x559c55d2d400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T15:08:52.927 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:52.922+0000 7f291599c640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x559c55cffbd0 0x559c55d2cec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T15:08:52.927 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:52.922+0000 7f291619d640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x559c55d2fc50 0x559c55d30030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.203:3300/0 says I am v2:172.21.15.43:49510/0 (socket says 172.21.15.43:49510) 2023-08-09T15:08:52.927 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:52.922+0000 7f291619d640 1 -- 172.21.15.43:0/4114364409 learned_addr learned my addr 172.21.15.43:0/4114364409 (peer_addr_for_me v2:172.21.15.43:0/0) 2023-08-09T15:08:52.927 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:52.922+0000 7f291599c640 1 --2- 172.21.15.43:0/1114929334 >> [v2:172.21.15.203:6832/3082732769,v1:172.21.15.203:6833/3082732769] conn(0x7f28f406a170 0x7f28f406c630 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f2900033000 tx=0x7f2900009030 comp rx=0 tx=0).ready entity=mgr.8500 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-09T15:08:52.927 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:52.922+0000 7f291599c640 1 -- 172.21.15.43:0/4114364409 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x559c55d037d0 msgr2=0x559c55d2d400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-09T15:08:52.927 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:52.922+0000 7f291599c640 1 --2- 172.21.15.43:0/4114364409 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x559c55d037d0 0x559c55d2d400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T15:08:52.928 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:52.922+0000 7f291599c640 1 -- 172.21.15.43:0/4114364409 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x559c55d2fc50 msgr2=0x559c55d30030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-09T15:08:52.928 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:52.922+0000 7f291599c640 1 --2- 172.21.15.43:0/4114364409 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x559c55d2fc50 0x559c55d30030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T15:08:52.928 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:52.922+0000 7f291599c640 1 -- 172.21.15.43:0/4114364409 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x559c55c23ac0 con 0x559c55cffbd0 2023-08-09T15:08:52.928 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:52.923+0000 7f291599c640 1 --2- 172.21.15.43:0/4114364409 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x559c55cffbd0 0x559c55d2cec0 secure :-1 s=READY pgs=2242 cs=0 l=1 rev1=1 crypto rx=0x7f2900004000 tx=0x7f2900030000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=94e644b7dfbc6977 server_cookie=0 in_seq=0 out_seq=0 2023-08-09T15:08:52.928 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:52.923+0000 7f28f3fff640 1 -- 172.21.15.43:0/4114364409 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2900007650 con 0x559c55cffbd0 2023-08-09T15:08:52.928 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:52.923+0000 7f28f3fff640 1 -- 172.21.15.43:0/4114364409 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f29000067d0 con 0x559c55cffbd0 2023-08-09T15:08:52.929 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:52.923+0000 7f28f3fff640 1 -- 172.21.15.43:0/4114364409 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2900006b80 con 0x559c55cffbd0 2023-08-09T15:08:52.929 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:52.923+0000 7f291af76740 1 -- 172.21.15.43:0/4114364409 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x559c55cffbd0 msgr2=0x559c55d2cec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T15:08:52.929 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:52.923+0000 7f291af76740 1 --2- 172.21.15.43:0/4114364409 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x559c55cffbd0 0x559c55d2cec0 secure :-1 s=READY pgs=2242 cs=0 l=1 rev1=1 crypto rx=0x7f2900004000 tx=0x7f2900030000 comp rx=0 tx=0).stop 2023-08-09T15:08:52.929 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:52.924+0000 7f291af76740 1 -- 172.21.15.43:0/4114364409 shutdown_connections 2023-08-09T15:08:52.929 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:52.924+0000 7f291af76740 1 --2- 172.21.15.43:0/4114364409 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x559c55d037d0 0x559c55d2d400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T15:08:52.929 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:52.924+0000 7f291af76740 1 --2- 172.21.15.43:0/4114364409 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x559c55cffbd0 0x559c55d2cec0 unknown :-1 s=CLOSED pgs=2242 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T15:08:52.929 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:52.924+0000 7f291af76740 1 --2- 172.21.15.43:0/4114364409 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x559c55d2fc50 0x559c55d30030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T15:08:52.930 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:52.924+0000 7f291af76740 1 -- 172.21.15.43:0/4114364409 >> 172.21.15.43:0/4114364409 conn(0x559c55d01ee0 msgr2=0x559c55d4aa60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-09T15:08:52.930 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:52.924+0000 7f291af76740 1 -- 172.21.15.43:0/4114364409 shutdown_connections 2023-08-09T15:08:52.930 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:52.924+0000 7f291af76740 1 -- 172.21.15.43:0/4114364409 wait complete. 2023-08-09T15:08:52.930 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:52.924+0000 7f291af76740 1 -- start start 2023-08-09T15:08:52.931 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:52.925+0000 7f291af76740 1 Processor -- start 2023-08-09T15:08:52.933 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:52.926+0000 7f291af76740 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x559c55d037d0 0x559c55d28da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T15:08:52.934 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:52.926+0000 7f291af76740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x559c55d2fc50 0x559c55d31fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T15:08:52.934 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:52.926+0000 7f291af76740 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x559c55d324f0 0x559c55d27500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T15:08:52.934 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:52.926+0000 7f291af76740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x559c55c0ad40 con 0x559c55d2fc50 2023-08-09T15:08:52.934 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:52.926+0000 7f291af76740 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x559c55c0a9e0 con 0x559c55d324f0 2023-08-09T15:08:52.934 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:52.926+0000 7f291af76740 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x559c559d0f20 con 0x559c55d037d0 2023-08-09T15:08:52.934 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:52.926+0000 7f291af76740 10 client.0 _ll_register_callbacks cb 0x559c55c23160 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-08-09T15:08:52.934 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:52.926+0000 7f291599c640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x559c55d2fc50 0x559c55d31fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T15:08:52.935 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:52.926+0000 7f291599c640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x559c55d2fc50 0x559c55d31fb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:39992/0 (socket says 172.21.15.43:39992) 2023-08-09T15:08:52.935 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:52.926+0000 7f291599c640 1 -- 172.21.15.43:0/1290605920 learned_addr learned my addr 172.21.15.43:0/1290605920 (peer_addr_for_me v2:172.21.15.43:0/0) 2023-08-09T15:08:52.935 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:52.927+0000 7f291599c640 1 -- 172.21.15.43:0/1290605920 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x559c55d037d0 msgr2=0x559c55d28da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-09T15:08:52.935 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:52.927+0000 7f291599c640 1 --2- 172.21.15.43:0/1290605920 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x559c55d037d0 0x559c55d28da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T15:08:52.935 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:52.927+0000 7f291599c640 1 -- 172.21.15.43:0/1290605920 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x559c55d324f0 msgr2=0x559c55d27500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-09T15:08:52.935 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:52.927+0000 7f291599c640 1 --2- 172.21.15.43:0/1290605920 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x559c55d324f0 0x559c55d27500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T15:08:52.935 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:52.927+0000 7f291599c640 1 -- 172.21.15.43:0/1290605920 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f28fc003520 con 0x559c55d2fc50 2023-08-09T15:08:52.935 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:52.927+0000 7f291599c640 1 --2- 172.21.15.43:0/1290605920 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x559c55d2fc50 0x559c55d31fb0 secure :-1 s=READY pgs=2243 cs=0 l=1 rev1=1 crypto rx=0x7f2900034040 tx=0x7f2900001190 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b2b16f48f33f4a96 server_cookie=0 in_seq=0 out_seq=0 2023-08-09T15:08:52.936 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:52.927+0000 7f28f17fa640 10 client.0 ms_handle_connect on v2:172.21.15.43:3300/0 2023-08-09T15:08:52.936 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:52.928+0000 7f28f17fa640 1 -- 172.21.15.43:0/1290605920 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f29000037f0 con 0x559c55d2fc50 2023-08-09T15:08:52.936 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:52.928+0000 7f291af76740 1 -- 172.21.15.43:0/1290605920 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x559c55c42500 con 0x559c55d2fc50 2023-08-09T15:08:52.936 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:52.928+0000 7f28f17fa640 1 -- 172.21.15.43:0/1290605920 <== mon.0 v2:172.21.15.43:3300/0 2 ==== osd_map(205..205 src has 1..205) v4 ==== 6426+0+0 (secure 0 0 0) 0x7f2900007650 con 0x559c55d2fc50 2023-08-09T15:08:52.936 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:52.928+0000 7f28f17fa640 1 -- 172.21.15.43:0/1290605920 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_get_version_reply(handle=1 version=96) v2 ==== 24+0+0 (secure 0 0 0) 0x7f2900006ed0 con 0x559c55d2fc50 2023-08-09T15:08:52.936 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:52.928+0000 7f291af76740 10 client.10926 fetch_fsmap learned FSMap version 96 2023-08-09T15:08:52.936 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:52.928+0000 7f291af76740 1 -- 172.21.15.43:0/1290605920 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,fsmap=96,monmap=2+}) v3 -- 0x559c55c68f50 con 0x559c55d2fc50 2023-08-09T15:08:52.937 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:52.928+0000 7f28f17fa640 1 -- 172.21.15.43:0/1290605920 <== mon.0 v2:172.21.15.43:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x559c55c42500 con 0x559c55d2fc50 2023-08-09T15:08:52.937 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:52.928+0000 7f28f17fa640 1 -- 172.21.15.43:0/1290605920 <== mon.0 v2:172.21.15.43:3300/0 5 ==== fsmap(e 96) v1 ==== 3294+0+0 (secure 0 0 0) 0x7f2900003950 con 0x559c55d2fc50 2023-08-09T15:08:52.937 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:52.928+0000 7f291af76740 10 client.10926 fetch_fsmap finished waiting for FSMap version 96 2023-08-09T15:08:52.937 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:52.928+0000 7f291af76740 10 client.10926 resolve_mds: resolved name 'a' to daemon mds.a 2023-08-09T15:08:52.937 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:52.928+0000 7f291af76740 20 client.10926 populate_metadata read hostname 'smithi043' 2023-08-09T15:08:52.937 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:52.928+0000 7f291af76740 1 --2- 172.21.15.43:0/1290605920 >> [v2:172.21.15.43:6833/349688227,v1:172.21.15.43:6835/349688227] conn(0x559c55d664c0 0x559c55d688b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T15:08:52.937 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:52.928+0000 7f291af76740 4 client.10926 mds_command: new command op to 10755 tid=0[{"prefix": "get_command_descriptions"}] 2023-08-09T15:08:52.937 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:52.928+0000 7f291af76740 1 -- 172.21.15.43:0/1290605920 --> [v2:172.21.15.43:6833/349688227,v1:172.21.15.43:6835/349688227] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x559c55c3fbc0 con 0x559c55d664c0 2023-08-09T15:08:52.938 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:52.929+0000 7f291619d640 1 --2- 172.21.15.43:0/1290605920 >> [v2:172.21.15.43:6833/349688227,v1:172.21.15.43:6835/349688227] conn(0x559c55d664c0 0x559c55d688b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T15:08:52.938 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:52.929+0000 7f291619d640 1 --2- 172.21.15.43:0/1290605920 >> [v2:172.21.15.43:6833/349688227,v1:172.21.15.43:6835/349688227] conn(0x559c55d664c0 0x559c55d688b0 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=3bcc8348f04d0ebd server_cookie=ddb171b1e5b5b in_seq=0 out_seq=0 2023-08-09T15:08:52.938 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:52.929+0000 7f28f17fa640 10 client.10926 ms_handle_connect on v2:172.21.15.43:6833/349688227 2023-08-09T15:08:52.938 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:52.930+0000 7f28f17fa640 1 -- 172.21.15.43:0/1290605920 <== mds.0 v2:172.21.15.43:6833/349688227 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18017 (crc 0 0 0) 0x559c55c3fbc0 con 0x559c55d664c0 2023-08-09T15:08:52.938 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:52.930+0000 7f28f17fa640 10 client.10926 handle_command_reply: tid=0 2023-08-09T15:08:52.938 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:52.930+0000 7f291af76740 1 client.10926 shutdown 2023-08-09T15:08:52.939 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:52.930+0000 7f291af76740 1 -- 172.21.15.43:0/1290605920 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x559c55d2fc50 msgr2=0x559c55d31fb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T15:08:52.939 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:52.930+0000 7f291af76740 1 --2- 172.21.15.43:0/1290605920 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x559c55d2fc50 0x559c55d31fb0 secure :-1 s=READY pgs=2243 cs=0 l=1 rev1=1 crypto rx=0x7f2900034040 tx=0x7f2900001190 comp rx=0 tx=0).stop 2023-08-09T15:08:52.939 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:52.930+0000 7f291af76740 1 -- 172.21.15.43:0/1290605920 shutdown_connections 2023-08-09T15:08:52.939 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:52.930+0000 7f291af76740 1 --2- 172.21.15.43:0/1290605920 >> [v2:172.21.15.43:6833/349688227,v1:172.21.15.43:6835/349688227] conn(0x559c55d664c0 0x559c55d688b0 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T15:08:52.939 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:52.931+0000 7f291af76740 1 --2- 172.21.15.43:0/1290605920 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x559c55d324f0 0x559c55d27500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T15:08:52.939 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:52.931+0000 7f291af76740 1 --2- 172.21.15.43:0/1290605920 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x559c55d2fc50 0x559c55d31fb0 unknown :-1 s=CLOSED pgs=2243 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T15:08:52.940 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:52.931+0000 7f291af76740 1 --2- 172.21.15.43:0/1290605920 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x559c55d037d0 0x559c55d28da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T15:08:52.940 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:52.931+0000 7f291af76740 1 -- 172.21.15.43:0/1290605920 >> 172.21.15.43:0/1290605920 conn(0x559c55cffbd0 msgr2=0x559c55d03f80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-09T15:08:52.940 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:52.931+0000 7f28f17fa640 0 client.10926 ms_handle_reset on v2:172.21.15.43:6833/349688227 2023-08-09T15:08:52.940 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:52.931+0000 7f291af76740 1 -- 172.21.15.43:0/1290605920 shutdown_connections 2023-08-09T15:08:52.940 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:52.931+0000 7f291af76740 1 -- 172.21.15.43:0/1290605920 wait complete. 2023-08-09T15:08:52.940 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:52.931+0000 7f291af76740 20 client.10926 trim_cache size 0 max 16384 2023-08-09T15:08:52.952 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:52.951+0000 7f291af76740 1 Processor -- start 2023-08-09T15:08:52.952 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:52.952+0000 7f291af76740 1 -- start start 2023-08-09T15:08:52.954 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:52.952+0000 7f291af76740 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x559c55d037d0 0x559c55d6ce20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T15:08:52.954 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:52.952+0000 7f291af76740 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x559c55d2fc50 0x559c55d067b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T15:08:52.954 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:52.952+0000 7f291af76740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x559c55d324f0 0x559c55d4df70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T15:08:52.955 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:52.952+0000 7f291af76740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x559c55d282f0 con 0x559c55d324f0 2023-08-09T15:08:52.955 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:52.952+0000 7f291af76740 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x559c55d27630 con 0x559c55d037d0 2023-08-09T15:08:52.955 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:52.953+0000 7f291af76740 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x559c55d5cf90 con 0x559c55d2fc50 2023-08-09T15:08:52.955 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:52.953+0000 7f291699e640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x559c55d324f0 0x559c55d4df70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T15:08:52.955 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:52.953+0000 7f291699e640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x559c55d324f0 0x559c55d4df70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:39998/0 (socket says 172.21.15.43:39998) 2023-08-09T15:08:52.955 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:52.953+0000 7f291699e640 1 -- 172.21.15.43:0/3691910992 learned_addr learned my addr 172.21.15.43:0/3691910992 (peer_addr_for_me v2:172.21.15.43:0/0) 2023-08-09T15:08:52.955 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:52.953+0000 7f291699e640 1 -- 172.21.15.43:0/3691910992 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x559c55d2fc50 msgr2=0x559c55d067b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-09T15:08:52.956 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:52.953+0000 7f291699e640 1 --2- 172.21.15.43:0/3691910992 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x559c55d2fc50 0x559c55d067b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T15:08:52.956 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:52.953+0000 7f291699e640 1 -- 172.21.15.43:0/3691910992 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x559c55d037d0 msgr2=0x559c55d6ce20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-09T15:08:52.956 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:52.953+0000 7f291699e640 1 --2- 172.21.15.43:0/3691910992 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x559c55d037d0 0x559c55d6ce20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T15:08:52.956 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:52.953+0000 7f291699e640 1 -- 172.21.15.43:0/3691910992 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x559c55c9c300 con 0x559c55d324f0 2023-08-09T15:08:52.956 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:52.954+0000 7f291699e640 1 --2- 172.21.15.43:0/3691910992 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x559c55d324f0 0x559c55d4df70 secure :-1 s=READY pgs=2244 cs=0 l=1 rev1=1 crypto rx=0x7f291006b920 tx=0x7f2908006c90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1a69606f8ecf4b79 server_cookie=0 in_seq=0 out_seq=0 2023-08-09T15:08:52.956 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:52.954+0000 7f2904ff9640 1 -- 172.21.15.43:0/3691910992 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f29080060d0 con 0x559c55d324f0 2023-08-09T15:08:52.957 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:52.954+0000 7f2904ff9640 1 -- 172.21.15.43:0/3691910992 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2908006270 con 0x559c55d324f0 2023-08-09T15:08:52.957 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:52.954+0000 7f2904ff9640 1 -- 172.21.15.43:0/3691910992 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2908006620 con 0x559c55d324f0 2023-08-09T15:08:52.962 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:52.955+0000 7f291af76740 1 -- 172.21.15.43:0/3691910992 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x559c55d324f0 msgr2=0x559c55d4df70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T15:08:52.963 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:52.955+0000 7f291af76740 1 --2- 172.21.15.43:0/3691910992 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x559c55d324f0 0x559c55d4df70 secure :-1 s=READY pgs=2244 cs=0 l=1 rev1=1 crypto rx=0x7f291006b920 tx=0x7f2908006c90 comp rx=0 tx=0).stop 2023-08-09T15:08:52.963 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:52.955+0000 7f291af76740 1 -- 172.21.15.43:0/3691910992 shutdown_connections 2023-08-09T15:08:52.963 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:52.955+0000 7f291af76740 1 --2- 172.21.15.43:0/3691910992 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x559c55d324f0 0x559c55d4df70 unknown :-1 s=CLOSED pgs=2244 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T15:08:52.963 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:52.955+0000 7f291af76740 1 --2- 172.21.15.43:0/3691910992 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x559c55d2fc50 0x559c55d067b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T15:08:52.964 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:52.955+0000 7f291af76740 1 --2- 172.21.15.43:0/3691910992 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x559c55d037d0 0x559c55d6ce20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T15:08:52.964 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:52.955+0000 7f291af76740 1 -- 172.21.15.43:0/3691910992 >> 172.21.15.43:0/3691910992 conn(0x559c55cffbd0 msgr2=0x559c55d31de0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-09T15:08:52.964 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:52.955+0000 7f291af76740 1 -- 172.21.15.43:0/3691910992 shutdown_connections 2023-08-09T15:08:52.964 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:52.955+0000 7f291af76740 1 -- 172.21.15.43:0/3691910992 wait complete. 2023-08-09T15:08:52.965 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:52.956+0000 7f291af76740 1 -- start start 2023-08-09T15:08:52.965 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:52.956+0000 7f291af76740 1 Processor -- start 2023-08-09T15:08:52.965 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:52.957+0000 7f291af76740 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x559c55d037d0 0x559c55d31b60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T15:08:52.965 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:52.957+0000 7f291af76740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x559c55d2fc50 0x559c55d61720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T15:08:52.966 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:52.957+0000 7f291af76740 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x559c55d324f0 0x559c55d61c60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T15:08:52.966 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:52.957+0000 7f291af76740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x559c55d19b20 con 0x559c55d2fc50 2023-08-09T15:08:52.966 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:52.957+0000 7f291af76740 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x559c55d62950 con 0x559c55d037d0 2023-08-09T15:08:52.966 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:52.957+0000 7f291af76740 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x559c55d4c810 con 0x559c55d324f0 2023-08-09T15:08:52.966 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:52.957+0000 7f291af76740 10 client.0 _ll_register_callbacks cb 0x559c55d304c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-08-09T15:08:52.967 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:52.957+0000 7f291599c640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x559c55d2fc50 0x559c55d61720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T15:08:52.967 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:52.957+0000 7f291599c640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x559c55d2fc50 0x559c55d61720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:40004/0 (socket says 172.21.15.43:40004) 2023-08-09T15:08:52.967 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:52.957+0000 7f291599c640 1 -- 172.21.15.43:0/3394532047 learned_addr learned my addr 172.21.15.43:0/3394532047 (peer_addr_for_me v2:172.21.15.43:0/0) 2023-08-09T15:08:52.967 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:52.957+0000 7f291699e640 1 --2- 172.21.15.43:0/3394532047 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x559c55d324f0 0x559c55d61c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T15:08:52.968 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:52.957+0000 7f291619d640 1 --2- 172.21.15.43:0/3394532047 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x559c55d037d0 0x559c55d31b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T15:08:52.968 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:52.958+0000 7f291599c640 1 -- 172.21.15.43:0/3394532047 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x559c55d324f0 msgr2=0x559c55d61c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-09T15:08:52.968 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:52.958+0000 7f291599c640 1 --2- 172.21.15.43:0/3394532047 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x559c55d324f0 0x559c55d61c60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T15:08:52.968 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:52.958+0000 7f291599c640 1 -- 172.21.15.43:0/3394532047 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x559c55d037d0 msgr2=0x559c55d31b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-09T15:08:52.969 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:52.958+0000 7f291599c640 1 --2- 172.21.15.43:0/3394532047 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x559c55d037d0 0x559c55d31b60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T15:08:52.969 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:52.958+0000 7f291599c640 1 -- 172.21.15.43:0/3394532047 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f290800f040 con 0x559c55d2fc50 2023-08-09T15:08:52.969 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:52.958+0000 7f291599c640 1 --2- 172.21.15.43:0/3394532047 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x559c55d2fc50 0x559c55d61720 secure :-1 s=READY pgs=2245 cs=0 l=1 rev1=1 crypto rx=0x559c55d1a920 tx=0x7f2900001120 comp rx=0 tx=0).ready entity=mon.0 client_cookie=44759cf8f95417cc server_cookie=0 in_seq=0 out_seq=0 2023-08-09T15:08:52.969 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:52.958+0000 7f28f17fa640 10 client.0 ms_handle_connect on v2:172.21.15.43:3300/0 2023-08-09T15:08:52.970 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:52.958+0000 7f28f17fa640 1 -- 172.21.15.43:0/3394532047 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f290005d670 con 0x559c55d2fc50 2023-08-09T15:08:52.970 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:52.958+0000 7f28f17fa640 1 -- 172.21.15.43:0/3394532047 <== mon.0 v2:172.21.15.43:3300/0 2 ==== osd_map(205..205 src has 1..205) v4 ==== 6426+0+0 (secure 0 0 0) 0x7f2900006ae0 con 0x559c55d2fc50 2023-08-09T15:08:52.970 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:52.958+0000 7f291af76740 1 -- 172.21.15.43:0/3394532047 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x559c55c92800 con 0x559c55d2fc50 2023-08-09T15:08:52.971 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:52.959+0000 7f28f17fa640 1 -- 172.21.15.43:0/3394532047 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_get_version_reply(handle=1 version=96) v2 ==== 24+0+0 (secure 0 0 0) 0x7f2900006ed0 con 0x559c55d2fc50 2023-08-09T15:08:52.971 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:52.959+0000 7f291af76740 10 client.10932 fetch_fsmap learned FSMap version 96 2023-08-09T15:08:52.971 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:52.959+0000 7f291af76740 1 -- 172.21.15.43:0/3394532047 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,fsmap=96,monmap=2+}) v3 -- 0x559c55c09a10 con 0x559c55d2fc50 2023-08-09T15:08:52.971 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:52.959+0000 7f28f17fa640 1 -- 172.21.15.43:0/3394532047 <== mon.0 v2:172.21.15.43:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x559c55c92800 con 0x559c55d2fc50 2023-08-09T15:08:52.972 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:52.959+0000 7f28f17fa640 1 -- 172.21.15.43:0/3394532047 <== mon.0 v2:172.21.15.43:3300/0 5 ==== fsmap(e 96) v1 ==== 3294+0+0 (secure 0 0 0) 0x7f290006e070 con 0x559c55d2fc50 2023-08-09T15:08:52.972 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:52.959+0000 7f291af76740 10 client.10932 fetch_fsmap finished waiting for FSMap version 96 2023-08-09T15:08:52.972 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:52.959+0000 7f291af76740 10 client.10932 resolve_mds: resolved name 'a' to daemon mds.a 2023-08-09T15:08:52.972 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:52.959+0000 7f291af76740 20 client.10932 populate_metadata read hostname 'smithi043' 2023-08-09T15:08:52.973 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:52.959+0000 7f291af76740 1 --2- 172.21.15.43:0/3394532047 >> [v2:172.21.15.43:6833/349688227,v1:172.21.15.43:6835/349688227] conn(0x559c55d32db0 0x559c55d33190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T15:08:52.973 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:52.959+0000 7f291af76740 4 client.10932 mds_command: new command op to 10755 tid=0[{"prefix": "status"}] 2023-08-09T15:08:52.973 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:52.959+0000 7f291af76740 1 -- 172.21.15.43:0/3394532047 --> [v2:172.21.15.43:6833/349688227,v1:172.21.15.43:6835/349688227] -- command(tid 0: {"prefix": "status"}) v1 -- 0x559c55c638b0 con 0x559c55d32db0 2023-08-09T15:08:52.974 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:52.960+0000 7f291619d640 1 --2- 172.21.15.43:0/3394532047 >> [v2:172.21.15.43:6833/349688227,v1:172.21.15.43:6835/349688227] conn(0x559c55d32db0 0x559c55d33190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T15:08:52.974 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:52.960+0000 7f291619d640 1 --2- 172.21.15.43:0/3394532047 >> [v2:172.21.15.43:6833/349688227,v1:172.21.15.43:6835/349688227] conn(0x559c55d32db0 0x559c55d33190 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=1fa63354b24fb7b0 server_cookie=48c9791ca337e8ff in_seq=0 out_seq=0 2023-08-09T15:08:52.974 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:52.960+0000 7f28f17fa640 10 client.10932 ms_handle_connect on v2:172.21.15.43:6833/349688227 2023-08-09T15:08:52.974 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:52.960+0000 7f28f17fa640 1 -- 172.21.15.43:0/3394532047 <== mds.0 v2:172.21.15.43:6833/349688227 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+328 (crc 0 0 0) 0x559c55c638b0 con 0x559c55d32db0 2023-08-09T15:08:52.975 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:52.960+0000 7f28f17fa640 10 client.10932 handle_command_reply: tid=0 2023-08-09T15:08:52.975 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:52.960+0000 7f291af76740 1 client.10932 shutdown 2023-08-09T15:08:52.975 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:52.961+0000 7f291af76740 1 -- 172.21.15.43:0/3394532047 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x559c55d2fc50 msgr2=0x559c55d61720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T15:08:52.975 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:52.961+0000 7f291af76740 1 --2- 172.21.15.43:0/3394532047 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x559c55d2fc50 0x559c55d61720 secure :-1 s=READY pgs=2245 cs=0 l=1 rev1=1 crypto rx=0x559c55d1a920 tx=0x7f2900001120 comp rx=0 tx=0).stop 2023-08-09T15:08:52.976 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:52.961+0000 7f291af76740 1 -- 172.21.15.43:0/3394532047 shutdown_connections 2023-08-09T15:08:52.976 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:52.961+0000 7f291af76740 1 --2- 172.21.15.43:0/3394532047 >> [v2:172.21.15.43:6833/349688227,v1:172.21.15.43:6835/349688227] conn(0x559c55d32db0 0x559c55d33190 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T15:08:52.976 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:52.961+0000 7f291af76740 1 --2- 172.21.15.43:0/3394532047 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x559c55d324f0 0x559c55d61c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T15:08:52.976 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:52.961+0000 7f291af76740 1 --2- 172.21.15.43:0/3394532047 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x559c55d2fc50 0x559c55d61720 unknown :-1 s=CLOSED pgs=2245 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T15:08:52.977 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:52.961+0000 7f291af76740 1 --2- 172.21.15.43:0/3394532047 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x559c55d037d0 0x559c55d31b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T15:08:52.977 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:52.961+0000 7f28f17fa640 0 client.10932 ms_handle_reset on v2:172.21.15.43:6833/349688227 2023-08-09T15:08:52.977 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:52.961+0000 7f291af76740 1 -- 172.21.15.43:0/3394532047 >> 172.21.15.43:0/3394532047 conn(0x559c55cffbd0 msgr2=0x559c55d4df70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-09T15:08:52.978 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:52.961+0000 7f291af76740 1 -- 172.21.15.43:0/3394532047 shutdown_connections 2023-08-09T15:08:52.978 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:52.961+0000 7f291af76740 1 -- 172.21.15.43:0/3394532047 wait complete. 2023-08-09T15:08:52.978 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:52.961+0000 7f291af76740 20 client.10932 trim_cache size 0 max 16384 2023-08-09T15:08:52.979 INFO:teuthology.orchestra.run.smithi043.stdout:{ 2023-08-09T15:08:52.979 INFO:teuthology.orchestra.run.smithi043.stdout: "cluster_fsid": "daac87ee-1337-4250-a955-aea4e408a39b", 2023-08-09T15:08:52.979 INFO:teuthology.orchestra.run.smithi043.stdout: "whoami": 0, 2023-08-09T15:08:52.979 INFO:teuthology.orchestra.run.smithi043.stdout: "id": 10755, 2023-08-09T15:08:52.979 INFO:teuthology.orchestra.run.smithi043.stdout: "want_state": "up:active", 2023-08-09T15:08:52.979 INFO:teuthology.orchestra.run.smithi043.stdout: "state": "up:active", 2023-08-09T15:08:52.980 INFO:teuthology.orchestra.run.smithi043.stdout: "fs_name": "cephfs", 2023-08-09T15:08:52.980 INFO:teuthology.orchestra.run.smithi043.stdout: "rank_uptime": 6.8315890000000001, 2023-08-09T15:08:52.980 INFO:teuthology.orchestra.run.smithi043.stdout: "mdsmap_epoch": 96, 2023-08-09T15:08:52.980 INFO:teuthology.orchestra.run.smithi043.stdout: "osdmap_epoch": 205, 2023-08-09T15:08:52.980 INFO:teuthology.orchestra.run.smithi043.stdout: "osdmap_epoch_barrier": 205, 2023-08-09T15:08:52.980 INFO:teuthology.orchestra.run.smithi043.stdout: "uptime": 12.783432458 2023-08-09T15:08:52.981 INFO:teuthology.orchestra.run.smithi043.stdout:} 2023-08-09T15:08:52.983 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:52.962+0000 7f2918401640 1 -- 172.21.15.43:0/1114929334 >> [v2:172.21.15.203:6832/3082732769,v1:172.21.15.203:6833/3082732769] conn(0x7f28f406a170 msgr2=0x7f28f406c630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T15:08:52.983 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:52.962+0000 7f2918401640 1 --2- 172.21.15.43:0/1114929334 >> [v2:172.21.15.203:6832/3082732769,v1:172.21.15.203:6833/3082732769] conn(0x7f28f406a170 0x7f28f406c630 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f2900033000 tx=0x7f2900009030 comp rx=0 tx=0).stop 2023-08-09T15:08:52.983 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:52.962+0000 7f2918401640 1 -- 172.21.15.43:0/1114929334 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f28fc0047c0 msgr2=0x7f29100c5f90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T15:08:52.984 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:52.962+0000 7f2918401640 1 --2- 172.21.15.43:0/1114929334 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f28fc0047c0 0x7f29100c5f90 secure :-1 s=READY pgs=2241 cs=0 l=1 rev1=1 crypto rx=0x7f290c007f30 tx=0x7f290c011660 comp rx=0 tx=0).stop 2023-08-09T15:08:52.984 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:52.962+0000 7f2918401640 1 -- 172.21.15.43:0/1114929334 shutdown_connections 2023-08-09T15:08:52.984 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:52.962+0000 7f2918401640 1 --2- 172.21.15.43:0/1114929334 >> [v2:172.21.15.203:6832/3082732769,v1:172.21.15.203:6833/3082732769] conn(0x7f28f406a170 0x7f28f406c630 unknown :-1 s=CLOSED pgs=70 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T15:08:52.984 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:52.962+0000 7f2918401640 1 --2- 172.21.15.43:0/1114929334 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f29100d2660 0x7f29100d2ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T15:08:52.984 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:52.962+0000 7f2918401640 1 --2- 172.21.15.43:0/1114929334 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f29100c8280 0x7f29100b1120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T15:08:52.984 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:52.962+0000 7f2918401640 1 --2- 172.21.15.43:0/1114929334 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f28fc0047c0 0x7f29100c5f90 unknown :-1 s=CLOSED pgs=2241 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T15:08:52.984 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:52.962+0000 7f2918401640 1 -- 172.21.15.43:0/1114929334 >> 172.21.15.43:0/1114929334 conn(0x7f291006a4e0 msgr2=0x7f291015bab0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-09T15:08:52.985 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:52.962+0000 7f2918401640 1 -- 172.21.15.43:0/1114929334 shutdown_connections 2023-08-09T15:08:52.985 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:52.962+0000 7f2918401640 1 -- 172.21.15.43:0/1114929334 wait complete. 2023-08-09T15:08:52.985 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2023-08-09T15:08:53.221 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:53.221+0000 7f56f8512640 1 Processor -- start 2023-08-09T15:08:53.222 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:53.221+0000 7f56f8512640 1 -- start start 2023-08-09T15:08:53.223 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:53.223+0000 7f56f8512640 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7f56f00c86f0 0x7f56f00c8b50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T15:08:53.224 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:53.223+0000 7f56f8512640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f56f006c8d0 0x7f56f006cd40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T15:08:53.224 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:53.223+0000 7f56f8512640 1 -- --> v1:172.21.15.203:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f56f00c9090 con 0x7f56f0143c20 2023-08-09T15:08:53.224 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:53.223+0000 7f56f8512640 1 -- --> v2:172.21.15.203:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f56f0144000 con 0x7f56f00c86f0 2023-08-09T15:08:53.224 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:53.223+0000 7f56f8512640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f56f006c440 con 0x7f56f006c8d0 2023-08-09T15:08:53.225 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:53.224+0000 7f56f5aad640 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7f56f00c86f0 0x7f56f00c8b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T15:08:53.225 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:53.224+0000 7f56f5aad640 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7f56f00c86f0 0x7f56f00c8b50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.203:3300/0 says I am v2:172.21.15.43:49564/0 (socket says 172.21.15.43:49564) 2023-08-09T15:08:53.225 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:53.224+0000 7f56f5aad640 1 -- 172.21.15.43:0/305608746 learned_addr learned my addr 172.21.15.43:0/305608746 (peer_addr_for_me v2:172.21.15.43:0/0) 2023-08-09T15:08:53.226 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:53.224+0000 7f56f6aaf640 1 --2- 172.21.15.43:0/305608746 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f56f006c8d0 0x7f56f006cd40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T15:08:53.226 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:53.225+0000 7f56f5aad640 1 -- 172.21.15.43:0/305608746 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f56f006c8d0 msgr2=0x7f56f006cd40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-09T15:08:53.226 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:53.225+0000 7f56f5aad640 1 --2- 172.21.15.43:0/305608746 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f56f006c8d0 0x7f56f006cd40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T15:08:53.226 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:53.225+0000 7f56f5aad640 1 -- 172.21.15.43:0/305608746 >> v1:172.21.15.203:6789/0 conn(0x7f56f0143c20 legacy=0x7f56f006f540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T15:08:53.226 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:53.225+0000 7f56f5aad640 1 -- 172.21.15.43:0/305608746 --> v2:172.21.15.203:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f56f006fc50 con 0x7f56f00c86f0 2023-08-09T15:08:53.228 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:53.226+0000 7f56f5aad640 1 --2- 172.21.15.43:0/305608746 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f56f00c86f0 0x7f56f00c8b50 secure :-1 s=READY pgs=818 cs=0 l=1 rev1=1 crypto rx=0x7f56e0009870 tx=0x7f56e002b8f0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=381e8d615b2e7f8c server_cookie=0 in_seq=0 out_seq=0 2023-08-09T15:08:53.228 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:53.227+0000 7f56f52ac640 1 -- 172.21.15.43:0/305608746 <== mon.1 v2:172.21.15.203:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f56e0031640 con 0x7f56f00c86f0 2023-08-09T15:08:53.228 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:53.227+0000 7f56f52ac640 1 -- 172.21.15.43:0/305608746 <== mon.1 v2:172.21.15.203:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f56e00317a0 con 0x7f56f00c86f0 2023-08-09T15:08:53.229 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:53.227+0000 7f56f52ac640 1 -- 172.21.15.43:0/305608746 <== mon.1 v2:172.21.15.203:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f56e0031b30 con 0x7f56f00c86f0 2023-08-09T15:08:53.229 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:53.227+0000 7f56f8512640 1 -- 172.21.15.43:0/305608746 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f56f00c86f0 msgr2=0x7f56f00c8b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T15:08:53.229 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:53.227+0000 7f56f8512640 1 --2- 172.21.15.43:0/305608746 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f56f00c86f0 0x7f56f00c8b50 secure :-1 s=READY pgs=818 cs=0 l=1 rev1=1 crypto rx=0x7f56e0009870 tx=0x7f56e002b8f0 comp rx=0 tx=0).stop 2023-08-09T15:08:53.229 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:53.228+0000 7f56f8512640 1 -- 172.21.15.43:0/305608746 shutdown_connections 2023-08-09T15:08:53.229 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:53.228+0000 7f56f8512640 1 --2- 172.21.15.43:0/305608746 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f56f006c8d0 0x7f56f006cd40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T15:08:53.229 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:53.228+0000 7f56f8512640 1 --2- 172.21.15.43:0/305608746 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f56f00c86f0 0x7f56f00c8b50 unknown :-1 s=CLOSED pgs=818 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T15:08:53.229 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:53.228+0000 7f56f8512640 1 -- 172.21.15.43:0/305608746 >> 172.21.15.43:0/305608746 conn(0x7f56f00ce0a0 msgr2=0x7f56f00ce480 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-09T15:08:53.230 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:53.228+0000 7f56f8512640 1 -- 172.21.15.43:0/305608746 shutdown_connections 2023-08-09T15:08:53.230 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:53.228+0000 7f56f8512640 1 -- 172.21.15.43:0/305608746 wait complete. 2023-08-09T15:08:53.230 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:53.230+0000 7f56f8512640 1 Processor -- start 2023-08-09T15:08:53.230 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:53.230+0000 7f56f8512640 1 -- start start 2023-08-09T15:08:53.231 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:53.231+0000 7f56f8512640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f56f006c8d0 0x7f56f013a2a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T15:08:53.231 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:53.231+0000 7f56f8512640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f56f00c86f0 0x7f56f013be40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T15:08:53.232 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:53.231+0000 7f56f8512640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f56f0143c20 0x7f56f013c380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T15:08:53.232 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:53.231+0000 7f56f8512640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f56f013c8c0 con 0x7f56f006c8d0 2023-08-09T15:08:53.232 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:53.231+0000 7f56f8512640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f56f015aa70 con 0x7f56f0143c20 2023-08-09T15:08:53.232 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:53.231+0000 7f56f8512640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f56f015ad10 con 0x7f56f00c86f0 2023-08-09T15:08:53.232 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:53.231+0000 7f56f62ae640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f56f006c8d0 0x7f56f013a2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T15:08:53.232 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:53.231+0000 7f56f62ae640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f56f006c8d0 0x7f56f013a2a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:40008/0 (socket says 172.21.15.43:40008) 2023-08-09T15:08:53.232 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:53.231+0000 7f56f62ae640 1 -- 172.21.15.43:0/973493300 learned_addr learned my addr 172.21.15.43:0/973493300 (peer_addr_for_me v2:172.21.15.43:0/0) 2023-08-09T15:08:53.233 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:53.232+0000 7f56f62ae640 1 -- 172.21.15.43:0/973493300 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f56f00c86f0 msgr2=0x7f56f013be40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T15:08:53.233 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:53.232+0000 7f56f62ae640 1 --2- 172.21.15.43:0/973493300 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f56f00c86f0 0x7f56f013be40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T15:08:53.234 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:53.232+0000 7f56f62ae640 1 -- 172.21.15.43:0/973493300 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f56f0143c20 msgr2=0x7f56f013c380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T15:08:53.234 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:53.232+0000 7f56f62ae640 1 --2- 172.21.15.43:0/973493300 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f56f0143c20 0x7f56f013c380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T15:08:53.234 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:53.232+0000 7f56f62ae640 1 -- 172.21.15.43:0/973493300 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f56e00094d0 con 0x7f56f006c8d0 2023-08-09T15:08:53.234 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:53.232+0000 7f56f62ae640 1 --2- 172.21.15.43:0/973493300 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f56f006c8d0 0x7f56f013a2a0 secure :-1 s=READY pgs=2246 cs=0 l=1 rev1=1 crypto rx=0x7f56ec00c450 tx=0x7f56ec00c910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-09T15:08:53.234 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:53.232+0000 7f56e6ffd640 1 -- 172.21.15.43:0/973493300 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f56ec0052d0 con 0x7f56f006c8d0 2023-08-09T15:08:53.234 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:53.232+0000 7f56e6ffd640 1 -- 172.21.15.43:0/973493300 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f56ec005430 con 0x7f56f006c8d0 2023-08-09T15:08:53.234 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:53.232+0000 7f56f8512640 1 -- 172.21.15.43:0/973493300 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f56f015aff0 con 0x7f56f006c8d0 2023-08-09T15:08:53.235 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:53.232+0000 7f56e6ffd640 1 -- 172.21.15.43:0/973493300 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f56ec003830 con 0x7f56f006c8d0 2023-08-09T15:08:53.235 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:53.233+0000 7f56f8512640 1 -- 172.21.15.43:0/973493300 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f56f015b380 con 0x7f56f006c8d0 2023-08-09T15:08:53.235 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:53.234+0000 7f56f8512640 1 -- 172.21.15.43:0/973493300 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f56f015b510 con 0x7f56f006c8d0 2023-08-09T15:08:53.235 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:53.235+0000 7f56e6ffd640 1 -- 172.21.15.43:0/973493300 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 106) v1 ==== 88443+0+0 (secure 0 0 0) 0x7f56ec003990 con 0x7f56f006c8d0 2023-08-09T15:08:53.236 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:53.235+0000 7f56e6ffd640 1 --2- 172.21.15.43:0/973493300 >> [v2:172.21.15.203:6832/3082732769,v1:172.21.15.203:6833/3082732769] conn(0x7f56d406a170 0x7f56d406c630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T15:08:53.236 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:53.236+0000 7f56e6ffd640 1 -- 172.21.15.43:0/973493300 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(205..205 src has 1..205) v4 ==== 6426+0+0 (secure 0 0 0) 0x7f56ec087dc0 con 0x7f56f006c8d0 2023-08-09T15:08:53.236 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:53.236+0000 7f56f5aad640 1 --2- 172.21.15.43:0/973493300 >> [v2:172.21.15.203:6832/3082732769,v1:172.21.15.203:6833/3082732769] conn(0x7f56d406a170 0x7f56d406c630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T15:08:53.238 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:53.237+0000 7f56f5aad640 1 --2- 172.21.15.43:0/973493300 >> [v2:172.21.15.203:6832/3082732769,v1:172.21.15.203:6833/3082732769] conn(0x7f56d406a170 0x7f56d406c630 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f56e0036000 tx=0x7f56e0030000 comp rx=0 tx=0).ready entity=mgr.8500 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-09T15:08:53.240 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:53.240+0000 7f56e6ffd640 1 -- 172.21.15.43:0/973493300 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178066 (secure 0 0 0) 0x7f56ec0532c0 con 0x7f56f006c8d0 2023-08-09T15:08:53.473 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:53.473+0000 7f56f8512640 1 -- 172.21.15.43:0/973493300 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7f56f0066140 con 0x7f56f006c8d0 2023-08-09T15:08:53.474 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:53.474+0000 7f56e6ffd640 1 -- 172.21.15.43:0/973493300 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v96) v1 ==== 53+0+73 (secure 0 0 0) 0x7f56ec017060 con 0x7f56f006c8d0 2023-08-09T15:08:53.474 INFO:teuthology.orchestra.run.smithi043.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2023-08-09T15:08:53.478 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:53.477+0000 7f56f8512640 1 -- 172.21.15.43:0/973493300 >> [v2:172.21.15.203:6832/3082732769,v1:172.21.15.203:6833/3082732769] conn(0x7f56d406a170 msgr2=0x7f56d406c630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T15:08:53.478 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:53.477+0000 7f56f8512640 1 --2- 172.21.15.43:0/973493300 >> [v2:172.21.15.203:6832/3082732769,v1:172.21.15.203:6833/3082732769] conn(0x7f56d406a170 0x7f56d406c630 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f56e0036000 tx=0x7f56e0030000 comp rx=0 tx=0).stop 2023-08-09T15:08:53.478 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:53.477+0000 7f56f8512640 1 -- 172.21.15.43:0/973493300 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f56f006c8d0 msgr2=0x7f56f013a2a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T15:08:53.478 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:53.477+0000 7f56f8512640 1 --2- 172.21.15.43:0/973493300 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f56f006c8d0 0x7f56f013a2a0 secure :-1 s=READY pgs=2246 cs=0 l=1 rev1=1 crypto rx=0x7f56ec00c450 tx=0x7f56ec00c910 comp rx=0 tx=0).stop 2023-08-09T15:08:53.478 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:53.477+0000 7f56f8512640 1 -- 172.21.15.43:0/973493300 shutdown_connections 2023-08-09T15:08:53.478 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:53.477+0000 7f56f8512640 1 --2- 172.21.15.43:0/973493300 >> [v2:172.21.15.203:6832/3082732769,v1:172.21.15.203:6833/3082732769] conn(0x7f56d406a170 0x7f56d406c630 unknown :-1 s=CLOSED pgs=71 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T15:08:53.478 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:53.477+0000 7f56f8512640 1 --2- 172.21.15.43:0/973493300 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f56f0143c20 0x7f56f013c380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T15:08:53.479 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:53.477+0000 7f56f8512640 1 --2- 172.21.15.43:0/973493300 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f56f00c86f0 0x7f56f013be40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T15:08:53.479 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:53.477+0000 7f56f8512640 1 --2- 172.21.15.43:0/973493300 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f56f006c8d0 0x7f56f013a2a0 unknown :-1 s=CLOSED pgs=2246 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T15:08:53.479 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:53.477+0000 7f56f8512640 1 -- 172.21.15.43:0/973493300 >> 172.21.15.43:0/973493300 conn(0x7f56f00ce0a0 msgr2=0x7f56f00aef10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-09T15:08:53.479 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:53.477+0000 7f56f8512640 1 -- 172.21.15.43:0/973493300 shutdown_connections 2023-08-09T15:08:53.479 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:53.477+0000 7f56f8512640 1 -- 172.21.15.43:0/973493300 wait complete. 2023-08-09T15:08:53.497 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2023-08-09T15:08:53.497 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2023-08-09T15:08:53.497 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi043.front.sepia.ceph.com 2023-08-09T15:08:53.498 INFO:tasks.cephfs.mount:self.client.name = client.0 2023-08-09T15:08:53.498 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2023-08-09T15:08:53.498 INFO:tasks.cephfs.mount:self.cephfs_name = None 2023-08-09T15:08:53.498 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2023-08-09T15:08:53.499 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2023-08-09T15:08:53.499 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2023-08-09T15:08:53.499 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-09T15:08:53.500 DEBUG:teuthology.orchestra.run.smithi043:> ip addr 2023-08-09T15:08:53.554 INFO:teuthology.orchestra.run.smithi043.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2023-08-09T15:08:53.554 INFO:teuthology.orchestra.run.smithi043.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2023-08-09T15:08:53.554 INFO:teuthology.orchestra.run.smithi043.stdout: inet 127.0.0.1/8 scope host lo 2023-08-09T15:08:53.554 INFO:teuthology.orchestra.run.smithi043.stdout: valid_lft forever preferred_lft forever 2023-08-09T15:08:53.554 INFO:teuthology.orchestra.run.smithi043.stdout: inet6 ::1/128 scope host 2023-08-09T15:08:53.555 INFO:teuthology.orchestra.run.smithi043.stdout: valid_lft forever preferred_lft forever 2023-08-09T15:08:53.555 INFO:teuthology.orchestra.run.smithi043.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-08-09T15:08:53.555 INFO:teuthology.orchestra.run.smithi043.stdout: link/ether 0c:c4:7a:6b:fd:9c brd ff:ff:ff:ff:ff:ff 2023-08-09T15:08:53.555 INFO:teuthology.orchestra.run.smithi043.stdout: altname enp7s0f0 2023-08-09T15:08:53.555 INFO:teuthology.orchestra.run.smithi043.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-08-09T15:08:53.555 INFO:teuthology.orchestra.run.smithi043.stdout: link/ether 0c:c4:7a:6b:fd:9d brd ff:ff:ff:ff:ff:ff 2023-08-09T15:08:53.555 INFO:teuthology.orchestra.run.smithi043.stdout: altname enp7s0f1 2023-08-09T15:08:53.556 INFO:teuthology.orchestra.run.smithi043.stdout:4: ens1f0: mtu 1500 qdisc mq state UP group default qlen 1000 2023-08-09T15:08:53.556 INFO:teuthology.orchestra.run.smithi043.stdout: link/ether 0c:c4:7a:bd:13:58 brd ff:ff:ff:ff:ff:ff 2023-08-09T15:08:53.556 INFO:teuthology.orchestra.run.smithi043.stdout: altname enp1s0f0 2023-08-09T15:08:53.556 INFO:teuthology.orchestra.run.smithi043.stdout: inet 172.21.15.43/20 brd 172.21.15.255 scope global dynamic noprefixroute ens1f0 2023-08-09T15:08:53.556 INFO:teuthology.orchestra.run.smithi043.stdout: valid_lft 36570sec preferred_lft 36570sec 2023-08-09T15:08:53.556 INFO:teuthology.orchestra.run.smithi043.stdout: inet6 fe80::ec4:7aff:febd:1358/64 scope link 2023-08-09T15:08:53.556 INFO:teuthology.orchestra.run.smithi043.stdout: valid_lft forever preferred_lft forever 2023-08-09T15:08:53.556 INFO:teuthology.orchestra.run.smithi043.stdout:5: ens1f1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-08-09T15:08:53.557 INFO:teuthology.orchestra.run.smithi043.stdout: link/ether 0c:c4:7a:bd:13:59 brd ff:ff:ff:ff:ff:ff 2023-08-09T15:08:53.557 INFO:teuthology.orchestra.run.smithi043.stdout: altname enp1s0f1 2023-08-09T15:08:53.557 INFO:teuthology.orchestra.run.smithi043.stdout:8: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2023-08-09T15:08:53.557 INFO:teuthology.orchestra.run.smithi043.stdout: link/ether 56:43:50:19:f4:0c brd ff:ff:ff:ff:ff:ff 2023-08-09T15:08:53.557 INFO:teuthology.orchestra.run.smithi043.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2023-08-09T15:08:53.557 INFO:teuthology.orchestra.run.smithi043.stdout: valid_lft forever preferred_lft forever 2023-08-09T15:08:53.557 INFO:teuthology.orchestra.run.smithi043.stdout: inet6 fe80::e494:65ff:fe92:a182/64 scope link 2023-08-09T15:08:53.558 INFO:teuthology.orchestra.run.smithi043.stdout: valid_lft forever preferred_lft forever 2023-08-09T15:08:53.558 INFO:teuthology.orchestra.run.smithi043.stdout:9: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2023-08-09T15:08:53.558 INFO:teuthology.orchestra.run.smithi043.stdout: link/ether 7e:93:26:fa:07:8f brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.0 2023-08-09T15:08:53.558 INFO:teuthology.orchestra.run.smithi043.stdout: inet6 fe80::7c93:26ff:fefa:78f/64 scope link 2023-08-09T15:08:53.558 INFO:teuthology.orchestra.run.smithi043.stdout: valid_lft forever preferred_lft forever 2023-08-09T15:08:53.558 INFO:teuthology.orchestra.run.smithi043.stdout:10: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2023-08-09T15:08:53.558 INFO:teuthology.orchestra.run.smithi043.stdout: link/ether 56:43:50:19:f4:0c brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.2 2023-08-09T15:08:53.558 INFO:teuthology.orchestra.run.smithi043.stdout: inet6 fe80::5443:50ff:fe19:f40c/64 scope link 2023-08-09T15:08:53.559 INFO:teuthology.orchestra.run.smithi043.stdout: valid_lft forever preferred_lft forever 2023-08-09T15:08:53.560 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-09T15:08:53.560 DEBUG:teuthology.orchestra.run.smithi043:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2023-08-09T15:08:53.646 INFO:teuthology.orchestra.run.smithi043.stdout:1 2023-08-09T15:08:53.650 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-09T15:08:53.651 DEBUG:teuthology.orchestra.run.smithi043:> route 2023-08-09T15:08:53.707 INFO:teuthology.orchestra.run.smithi043.stdout:Kernel IP routing table 2023-08-09T15:08:53.707 INFO:teuthology.orchestra.run.smithi043.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2023-08-09T15:08:53.708 INFO:teuthology.orchestra.run.smithi043.stdout:default _gateway 0.0.0.0 UG 100 0 0 ens1f0 2023-08-09T15:08:53.708 INFO:teuthology.orchestra.run.smithi043.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 ens1f0 2023-08-09T15:08:53.708 INFO:teuthology.orchestra.run.smithi043.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2023-08-09T15:08:53.708 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-09T15:08:53.709 DEBUG:teuthology.orchestra.run.smithi043:> (cd / && exec bash -c ' 2023-08-09T15:08:53.709 DEBUG:teuthology.orchestra.run.smithi043:> set -e 2023-08-09T15:08:53.709 DEBUG:teuthology.orchestra.run.smithi043:> sudo iptables -A FORWARD -o ens1f0 -i ceph-brx -j ACCEPT 2023-08-09T15:08:53.710 DEBUG:teuthology.orchestra.run.smithi043:> sudo iptables -A FORWARD -i ens1f0 -o ceph-brx -j ACCEPT 2023-08-09T15:08:53.710 DEBUG:teuthology.orchestra.run.smithi043:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o ens1f0 -j MASQUERADE 2023-08-09T15:08:53.710 DEBUG:teuthology.orchestra.run.smithi043:> ') 2023-08-09T15:08:53.909 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-09T15:08:53.910 DEBUG:teuthology.orchestra.run.smithi043:> ip netns list 2023-08-09T15:08:53.966 INFO:teuthology.orchestra.run.smithi043.stdout:ceph-ns--home-ubuntu-cephtest-mnt.2 (id: 1) 2023-08-09T15:08:53.966 INFO:teuthology.orchestra.run.smithi043.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2023-08-09T15:08:53.967 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2023-08-09T15:08:53.968 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-09T15:08:53.968 DEBUG:teuthology.orchestra.run.smithi043:> sudo ip link set brx.0 up 2023-08-09T15:08:54.071 INFO:tasks.cephfs.fuse_mount:Client client.0 config is {} 2023-08-09T15:08:54.071 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-08-09T15:08:54.072 DEBUG:teuthology.orchestra.run.smithi043:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2023-08-09T15:08:54.129 INFO:teuthology.orchestra.run.smithi043.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2023-08-09T15:08:54.130 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-08-09T15:08:54.130 DEBUG:teuthology.orchestra.run.smithi043:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2023-08-09T15:08:54.188 DEBUG:teuthology.orchestra.run.smithi043:> sudo modprobe fuse 2023-08-09T15:08:54.295 INFO:teuthology.orchestra.run:Running command with timeout 30 2023-08-09T15:08:54.296 DEBUG:teuthology.orchestra.run.smithi043:> sudo mount -t fusectl /sys/fs/fuse/connections /sys/fs/fuse/connections 2023-08-09T15:08:54.400 INFO:teuthology.orchestra.run.smithi043.stderr:mount: /sys/fs/fuse/connections: /sys/fs/fuse/connections already mounted or mount point busy. 2023-08-09T15:08:54.404 DEBUG:teuthology.orchestra.run:got remote process result: 32 2023-08-09T15:08:54.406 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-09T15:08:54.407 DEBUG:teuthology.orchestra.run.smithi043:> ls /sys/fs/fuse/connections 2023-08-09T15:08:54.467 INFO:tasks.cephfs.fuse_mount:Pre-mount connections: [] 2023-08-09T15:08:54.468 DEBUG:teuthology.orchestra.run.smithi043:> (cd /home/ubuntu/cephtest && exec sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-fuse -f --admin-socket '/var/run/ceph/$cluster-$name.$pid.asok' /home/ubuntu/cephtest/mnt.0 --id 0) 2023-08-09T15:08:54.475 DEBUG:teuthology.orchestra.run.smithi043:> sudo modprobe fuse 2023-08-09T15:08:54.565 INFO:teuthology.orchestra.run:Running command with timeout 30 2023-08-09T15:08:54.566 DEBUG:teuthology.orchestra.run.smithi043:> sudo mount -t fusectl /sys/fs/fuse/connections /sys/fs/fuse/connections 2023-08-09T15:08:54.650 INFO:teuthology.orchestra.run.smithi043.stderr:mount: /sys/fs/fuse/connections: /sys/fs/fuse/connections already mounted or mount point busy. 2023-08-09T15:08:54.655 DEBUG:teuthology.orchestra.run:got remote process result: 32 2023-08-09T15:08:54.655 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-09T15:08:54.656 DEBUG:teuthology.orchestra.run.smithi043:> ls /sys/fs/fuse/connections 2023-08-09T15:08:54.684 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi043.stderr:2023-08-09T15:08:54.684+0000 7fb8a8960480 -1 init, newargv = 0x5561e89077e0 newargc=15 2023-08-09T15:08:54.684 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi043.stderr:ceph-fuse[148280]: starting ceph client 2023-08-09T15:08:54.694 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi043.stderr:ceph-fuse[148280]: starting fuse 2023-08-09T15:08:54.705 INFO:teuthology.orchestra.run.smithi043.stdout:40 2023-08-09T15:08:54.705 INFO:tasks.cephfs.fuse_mount:Post-mount connections: [40] 2023-08-09T15:08:54.706 DEBUG:teuthology.orchestra.run.smithi043:> sudo stdin-killer -- python3 -c ' 2023-08-09T15:08:54.706 DEBUG:teuthology.orchestra.run.smithi043:> import glob 2023-08-09T15:08:54.707 DEBUG:teuthology.orchestra.run.smithi043:> import re 2023-08-09T15:08:54.707 DEBUG:teuthology.orchestra.run.smithi043:> import os 2023-08-09T15:08:54.707 DEBUG:teuthology.orchestra.run.smithi043:> import subprocess 2023-08-09T15:08:54.707 DEBUG:teuthology.orchestra.run.smithi043:> 2023-08-09T15:08:54.708 DEBUG:teuthology.orchestra.run.smithi043:> def _find_admin_socket(client_name): 2023-08-09T15:08:54.708 DEBUG:teuthology.orchestra.run.smithi043:> asok_path = "/var/run/ceph/ceph-client.0.*.asok" 2023-08-09T15:08:54.708 DEBUG:teuthology.orchestra.run.smithi043:> files = glob.glob(asok_path) 2023-08-09T15:08:54.709 DEBUG:teuthology.orchestra.run.smithi043:> mountpoint = "/home/ubuntu/cephtest/mnt.0" 2023-08-09T15:08:54.709 DEBUG:teuthology.orchestra.run.smithi043:> 2023-08-09T15:08:54.709 DEBUG:teuthology.orchestra.run.smithi043:> # Given a non-glob path, it better be there 2023-08-09T15:08:54.710 DEBUG:teuthology.orchestra.run.smithi043:> if "*" not in asok_path: 2023-08-09T15:08:54.710 DEBUG:teuthology.orchestra.run.smithi043:> assert(len(files) == 1) 2023-08-09T15:08:54.710 DEBUG:teuthology.orchestra.run.smithi043:> return files[0] 2023-08-09T15:08:54.711 DEBUG:teuthology.orchestra.run.smithi043:> 2023-08-09T15:08:54.711 DEBUG:teuthology.orchestra.run.smithi043:> for f in files: 2023-08-09T15:08:54.711 DEBUG:teuthology.orchestra.run.smithi043:> pid = re.match(".*\.(\d+)\.asok$", f).group(1) 2023-08-09T15:08:54.712 DEBUG:teuthology.orchestra.run.smithi043:> if os.path.exists("/proc/{0}".format(pid)): 2023-08-09T15:08:54.712 DEBUG:teuthology.orchestra.run.smithi043:> with open("/proc/{0}/cmdline".format(pid), '"'"'r'"'"') as proc_f: 2023-08-09T15:08:54.712 DEBUG:teuthology.orchestra.run.smithi043:> contents = proc_f.read() 2023-08-09T15:08:54.713 DEBUG:teuthology.orchestra.run.smithi043:> if mountpoint in contents: 2023-08-09T15:08:54.713 DEBUG:teuthology.orchestra.run.smithi043:> return f 2023-08-09T15:08:54.713 DEBUG:teuthology.orchestra.run.smithi043:> raise RuntimeError("Client socket {0} not found".format(client_name)) 2023-08-09T15:08:54.713 DEBUG:teuthology.orchestra.run.smithi043:> 2023-08-09T15:08:54.714 DEBUG:teuthology.orchestra.run.smithi043:> print(_find_admin_socket("client.0")) 2023-08-09T15:08:54.714 DEBUG:teuthology.orchestra.run.smithi043:> ' 2023-08-09T15:08:54.925 INFO:teuthology.orchestra.run.smithi043.stdout:/var/run/ceph/ceph-client.0.148280.asok 2023-08-09T15:08:54.930 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:54 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2023-08-09T15:08:54.944 INFO:tasks.cephfs.fuse_mount:Found client admin socket at /var/run/ceph/ceph-client.0.148280.asok 2023-08-09T15:08:54.944 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-09T15:08:54.945 DEBUG:teuthology.orchestra.run.smithi043:> sudo ceph --admin-daemon /var/run/ceph/ceph-client.0.148280.asok status 2023-08-09T15:08:55.198 INFO:teuthology.orchestra.run.smithi043.stdout:{ 2023-08-09T15:08:55.198 INFO:teuthology.orchestra.run.smithi043.stdout: "metadata": { 2023-08-09T15:08:55.199 INFO:teuthology.orchestra.run.smithi043.stdout: "ceph_sha1": "668a86f232c9454b2ce5928d1432a29a72964d85", 2023-08-09T15:08:55.199 INFO:teuthology.orchestra.run.smithi043.stdout: "ceph_version": "ceph version 18.0.0-5389-g668a86f2 (668a86f232c9454b2ce5928d1432a29a72964d85) reef (dev)", 2023-08-09T15:08:55.199 INFO:teuthology.orchestra.run.smithi043.stdout: "entity_id": "0", 2023-08-09T15:08:55.199 INFO:teuthology.orchestra.run.smithi043.stdout: "hostname": "smithi043", 2023-08-09T15:08:55.199 INFO:teuthology.orchestra.run.smithi043.stdout: "mount_point": "/home/ubuntu/cephtest/mnt.0", 2023-08-09T15:08:55.199 INFO:teuthology.orchestra.run.smithi043.stdout: "pid": "148280", 2023-08-09T15:08:55.200 INFO:teuthology.orchestra.run.smithi043.stdout: "root": "/" 2023-08-09T15:08:55.200 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2023-08-09T15:08:55.200 INFO:teuthology.orchestra.run.smithi043.stdout: "dentry_count": 0, 2023-08-09T15:08:55.200 INFO:teuthology.orchestra.run.smithi043.stdout: "dentry_pinned_count": 0, 2023-08-09T15:08:55.201 INFO:teuthology.orchestra.run.smithi043.stdout: "id": 10941, 2023-08-09T15:08:55.201 INFO:teuthology.orchestra.run.smithi043.stdout: "inst": { 2023-08-09T15:08:55.201 INFO:teuthology.orchestra.run.smithi043.stdout: "name": { 2023-08-09T15:08:55.201 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "client", 2023-08-09T15:08:55.201 INFO:teuthology.orchestra.run.smithi043.stdout: "num": 10941 2023-08-09T15:08:55.201 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2023-08-09T15:08:55.201 INFO:teuthology.orchestra.run.smithi043.stdout: "addr": { 2023-08-09T15:08:55.202 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "v1", 2023-08-09T15:08:55.202 INFO:teuthology.orchestra.run.smithi043.stdout: "addr": "192.168.0.1:0", 2023-08-09T15:08:55.202 INFO:teuthology.orchestra.run.smithi043.stdout: "nonce": 1124489053 2023-08-09T15:08:55.202 INFO:teuthology.orchestra.run.smithi043.stdout: } 2023-08-09T15:08:55.202 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2023-08-09T15:08:55.202 INFO:teuthology.orchestra.run.smithi043.stdout: "addr": { 2023-08-09T15:08:55.202 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "v1", 2023-08-09T15:08:55.202 INFO:teuthology.orchestra.run.smithi043.stdout: "addr": "192.168.0.1:0", 2023-08-09T15:08:55.203 INFO:teuthology.orchestra.run.smithi043.stdout: "nonce": 1124489053 2023-08-09T15:08:55.203 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2023-08-09T15:08:55.203 INFO:teuthology.orchestra.run.smithi043.stdout: "inst_str": "client.10941 192.168.0.1:0/1124489053", 2023-08-09T15:08:55.203 INFO:teuthology.orchestra.run.smithi043.stdout: "addr_str": "192.168.0.1:0/1124489053", 2023-08-09T15:08:55.203 INFO:teuthology.orchestra.run.smithi043.stdout: "inode_count": 1, 2023-08-09T15:08:55.203 INFO:teuthology.orchestra.run.smithi043.stdout: "mds_epoch": 96, 2023-08-09T15:08:55.203 INFO:teuthology.orchestra.run.smithi043.stdout: "osd_epoch": 205, 2023-08-09T15:08:55.204 INFO:teuthology.orchestra.run.smithi043.stdout: "osd_epoch_barrier": 0, 2023-08-09T15:08:55.204 INFO:teuthology.orchestra.run.smithi043.stdout: "blocklisted": false, 2023-08-09T15:08:55.204 INFO:teuthology.orchestra.run.smithi043.stdout: "fs_name": "cephfs" 2023-08-09T15:08:55.204 INFO:teuthology.orchestra.run.smithi043.stdout:} 2023-08-09T15:08:55.214 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-09T15:08:55.215 DEBUG:teuthology.orchestra.run.smithi043:> stat --file-system '--printf=%T 2023-08-09T15:08:55.215 DEBUG:teuthology.orchestra.run.smithi043:> ' -- /home/ubuntu/cephtest/mnt.0 2023-08-09T15:08:55.276 INFO:teuthology.orchestra.run.smithi043.stdout:fuseblk 2023-08-09T15:08:55.277 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.0 2023-08-09T15:08:55.277 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-09T15:08:55.278 DEBUG:teuthology.orchestra.run.smithi043:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2023-08-09T15:08:55.384 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs volume ls 2023-08-09T15:08:55.616 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:55.615+0000 7fbc81d37640 1 Processor -- start 2023-08-09T15:08:55.616 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:55.616+0000 7fbc81d37640 1 -- start start 2023-08-09T15:08:55.618 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:55.618+0000 7fbc81d37640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fbc7c0c8670 0x7fbc7c15c8e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T15:08:55.618 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:55.618+0000 7fbc81d37640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7fbc7c0abbb0 0x7fbc7c0ac040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T15:08:55.618 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:55.618+0000 7fbc81d37640 1 -- --> v1:172.21.15.43:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbc7c070730 con 0x7fbc7c06f3d0 2023-08-09T15:08:55.618 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:55.618+0000 7fbc81d37640 1 -- --> v2:172.21.15.43:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbc7c156e60 con 0x7fbc7c0abbb0 2023-08-09T15:08:55.618 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:55.618+0000 7fbc81d37640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbc7c071620 con 0x7fbc7c0c8670 2023-08-09T15:08:55.619 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:55.618+0000 7fbc81536640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7fbc7c0abbb0 0x7fbc7c0ac040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T15:08:55.619 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:55.618+0000 7fbc81536640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7fbc7c0abbb0 0x7fbc7c0ac040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:41230/0 (socket says 172.21.15.43:41230) 2023-08-09T15:08:55.619 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:55.618+0000 7fbc81536640 1 -- 172.21.15.43:0/3604319599 learned_addr learned my addr 172.21.15.43:0/3604319599 (peer_addr_for_me v2:172.21.15.43:0/0) 2023-08-09T15:08:55.620 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:55.618+0000 7fbc80d35640 1 --2- 172.21.15.43:0/3604319599 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fbc7c0c8670 0x7fbc7c15c8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T15:08:55.620 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:55.619+0000 7fbc81536640 1 -- 172.21.15.43:0/3604319599 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fbc7c0c8670 msgr2=0x7fbc7c15c8e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-09T15:08:55.620 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:55.619+0000 7fbc81536640 1 --2- 172.21.15.43:0/3604319599 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fbc7c0c8670 0x7fbc7c15c8e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T15:08:55.620 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:55.619+0000 7fbc81536640 1 -- 172.21.15.43:0/3604319599 >> v1:172.21.15.43:6789/0 conn(0x7fbc7c06f3d0 legacy=0x7fbc7c0c8a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T15:08:55.620 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:55.619+0000 7fbc81536640 1 -- 172.21.15.43:0/3604319599 --> v2:172.21.15.43:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbc7c06fb20 con 0x7fbc7c0abbb0 2023-08-09T15:08:55.621 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:55.620+0000 7fbc81536640 1 --2- 172.21.15.43:0/3604319599 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fbc7c0abbb0 0x7fbc7c0ac040 secure :-1 s=READY pgs=2251 cs=0 l=1 rev1=1 crypto rx=0x7fbc74009e50 tx=0x7fbc7402be50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=45004b45989b9ff7 server_cookie=0 in_seq=0 out_seq=0 2023-08-09T15:08:55.621 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:55.620+0000 7fbc7b7fe640 1 -- 172.21.15.43:0/3604319599 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbc7402c9f0 con 0x7fbc7c0abbb0 2023-08-09T15:08:55.621 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:55.620+0000 7fbc7b7fe640 1 -- 172.21.15.43:0/3604319599 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbc7402cb50 con 0x7fbc7c0abbb0 2023-08-09T15:08:55.621 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:55.620+0000 7fbc7b7fe640 1 -- 172.21.15.43:0/3604319599 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbc740343f0 con 0x7fbc7c0abbb0 2023-08-09T15:08:55.622 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:55.621+0000 7fbc81d37640 1 -- 172.21.15.43:0/3604319599 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fbc7c0abbb0 msgr2=0x7fbc7c0ac040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T15:08:55.622 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:55.621+0000 7fbc81d37640 1 --2- 172.21.15.43:0/3604319599 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fbc7c0abbb0 0x7fbc7c0ac040 secure :-1 s=READY pgs=2251 cs=0 l=1 rev1=1 crypto rx=0x7fbc74009e50 tx=0x7fbc7402be50 comp rx=0 tx=0).stop 2023-08-09T15:08:55.622 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:55.621+0000 7fbc81d37640 1 -- 172.21.15.43:0/3604319599 shutdown_connections 2023-08-09T15:08:55.622 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:55.621+0000 7fbc81d37640 1 --2- 172.21.15.43:0/3604319599 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fbc7c0abbb0 0x7fbc7c0ac040 unknown :-1 s=CLOSED pgs=2251 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T15:08:55.622 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:55.621+0000 7fbc81d37640 1 --2- 172.21.15.43:0/3604319599 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fbc7c0c8670 0x7fbc7c15c8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T15:08:55.622 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:55.621+0000 7fbc81d37640 1 -- 172.21.15.43:0/3604319599 >> 172.21.15.43:0/3604319599 conn(0x7fbc7c0af950 msgr2=0x7fbc7c0afd60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-09T15:08:55.622 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:55.621+0000 7fbc81d37640 1 -- 172.21.15.43:0/3604319599 shutdown_connections 2023-08-09T15:08:55.623 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:55.621+0000 7fbc81d37640 1 -- 172.21.15.43:0/3604319599 wait complete. 2023-08-09T15:08:55.623 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:55.623+0000 7fbc81d37640 1 Processor -- start 2023-08-09T15:08:55.623 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:55.623+0000 7fbc81d37640 1 -- start start 2023-08-09T15:08:55.624 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:55.624+0000 7fbc81d37640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fbc7c06f3d0 0x7fbc7c154460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T15:08:55.624 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:55.624+0000 7fbc81d37640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fbc7c0abbb0 0x7fbc7c1f26f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T15:08:55.624 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:55.624+0000 7fbc81d37640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fbc7c0c8670 0x7fbc7c0b2cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T15:08:55.624 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:55.624+0000 7fbc81d37640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbc7c1f2c30 con 0x7fbc7c06f3d0 2023-08-09T15:08:55.625 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:55.624+0000 7fbc81d37640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbc7c1f6010 con 0x7fbc7c0c8670 2023-08-09T15:08:55.625 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:55.624+0000 7fbc81d37640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbc7c1f5400 con 0x7fbc7c0abbb0 2023-08-09T15:08:55.625 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:55.624+0000 7fbc80d35640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fbc7c06f3d0 0x7fbc7c154460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T15:08:55.626 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:55.624+0000 7fbc80d35640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fbc7c06f3d0 0x7fbc7c154460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:41242/0 (socket says 172.21.15.43:41242) 2023-08-09T15:08:55.626 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:55.624+0000 7fbc80d35640 1 -- 172.21.15.43:0/3451072168 learned_addr learned my addr 172.21.15.43:0/3451072168 (peer_addr_for_me v2:172.21.15.43:0/0) 2023-08-09T15:08:55.626 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:55.625+0000 7fbc80d35640 1 -- 172.21.15.43:0/3451072168 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fbc7c0abbb0 msgr2=0x7fbc7c1f26f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T15:08:55.626 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:55.625+0000 7fbc81536640 1 --2- 172.21.15.43:0/3451072168 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fbc7c0c8670 0x7fbc7c0b2cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T15:08:55.626 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:55.625+0000 7fbc80d35640 1 --2- 172.21.15.43:0/3451072168 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fbc7c0abbb0 0x7fbc7c1f26f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T15:08:55.626 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:55.625+0000 7fbc80d35640 1 -- 172.21.15.43:0/3451072168 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fbc7c0c8670 msgr2=0x7fbc7c0b2cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T15:08:55.626 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:55.625+0000 7fbc80d35640 1 --2- 172.21.15.43:0/3451072168 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fbc7c0c8670 0x7fbc7c0b2cf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T15:08:55.627 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:55.625+0000 7fbc80d35640 1 -- 172.21.15.43:0/3451072168 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbc74009b30 con 0x7fbc7c06f3d0 2023-08-09T15:08:55.627 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:55.625+0000 7fbc80d35640 1 --2- 172.21.15.43:0/3451072168 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fbc7c06f3d0 0x7fbc7c154460 secure :-1 s=READY pgs=2252 cs=0 l=1 rev1=1 crypto rx=0x7fbc7000d800 tx=0x7fbc7000dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-09T15:08:55.627 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:55.626+0000 7fbc797fa640 1 -- 172.21.15.43:0/3451072168 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbc7000b6c0 con 0x7fbc7c06f3d0 2023-08-09T15:08:55.627 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:55.626+0000 7fbc797fa640 1 -- 172.21.15.43:0/3451072168 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbc7000b820 con 0x7fbc7c06f3d0 2023-08-09T15:08:55.628 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:55.626+0000 7fbc797fa640 1 -- 172.21.15.43:0/3451072168 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbc7000bbb0 con 0x7fbc7c06f3d0 2023-08-09T15:08:55.628 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:55.626+0000 7fbc81d37640 1 -- 172.21.15.43:0/3451072168 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbc7c1f56e0 con 0x7fbc7c06f3d0 2023-08-09T15:08:55.628 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:55.626+0000 7fbc81d37640 1 -- 172.21.15.43:0/3451072168 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbc7c1f5ca0 con 0x7fbc7c06f3d0 2023-08-09T15:08:55.628 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:55.627+0000 7fbc81d37640 1 -- 172.21.15.43:0/3451072168 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbc7c0739f0 con 0x7fbc7c06f3d0 2023-08-09T15:08:55.628 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:55.628+0000 7fbc797fa640 1 -- 172.21.15.43:0/3451072168 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 106) v1 ==== 88443+0+0 (secure 0 0 0) 0x7fbc7000bd10 con 0x7fbc7c06f3d0 2023-08-09T15:08:55.629 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:55.629+0000 7fbc797fa640 1 --2- 172.21.15.43:0/3451072168 >> [v2:172.21.15.203:6832/3082732769,v1:172.21.15.203:6833/3082732769] conn(0x7fbc6006a170 0x7fbc6006c630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T15:08:55.629 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:55.629+0000 7fbc797fa640 1 -- 172.21.15.43:0/3451072168 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(205..205 src has 1..205) v4 ==== 6426+0+0 (secure 0 0 0) 0x7fbc70088e90 con 0x7fbc7c06f3d0 2023-08-09T15:08:55.630 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:55.630+0000 7fbc7bfff640 1 --2- 172.21.15.43:0/3451072168 >> [v2:172.21.15.203:6832/3082732769,v1:172.21.15.203:6833/3082732769] conn(0x7fbc6006a170 0x7fbc6006c630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T15:08:55.631 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:55.631+0000 7fbc7bfff640 1 --2- 172.21.15.43:0/3451072168 >> [v2:172.21.15.203:6832/3082732769,v1:172.21.15.203:6833/3082732769] conn(0x7fbc6006a170 0x7fbc6006c630 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7fbc68000f60 tx=0x7fbc680079b0 comp rx=0 tx=0).ready entity=mgr.8500 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-09T15:08:55.634 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:55.634+0000 7fbc797fa640 1 -- 172.21.15.43:0/3451072168 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178066 (secure 0 0 0) 0x7fbc70054260 con 0x7fbc7c06f3d0 2023-08-09T15:08:55.849 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:55.849+0000 7fbc81d37640 1 -- 172.21.15.43:0/3451072168 --> [v2:172.21.15.203:6832/3082732769,v1:172.21.15.203:6833/3082732769] -- mgr_command(tid 0: {"prefix": "fs volume ls", "target": ["mon-mgr", ""]}) v1 -- 0x7fbc7c0700e0 con 0x7fbc6006a170 2023-08-09T15:08:55.853 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:55.853+0000 7fbc797fa640 1 -- 172.21.15.43:0/3451072168 <== mgr.8500 v2:172.21.15.203:6832/3082732769 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+41 (secure 0 0 0) 0x7fbc68005080 con 0x7fbc6006a170 2023-08-09T15:08:55.854 INFO:teuthology.orchestra.run.smithi043.stdout:[ 2023-08-09T15:08:55.855 INFO:teuthology.orchestra.run.smithi043.stdout: { 2023-08-09T15:08:55.855 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "cephfs" 2023-08-09T15:08:55.855 INFO:teuthology.orchestra.run.smithi043.stdout: } 2023-08-09T15:08:55.855 INFO:teuthology.orchestra.run.smithi043.stdout:] 2023-08-09T15:08:55.857 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:55.855+0000 7fbc81d37640 1 -- 172.21.15.43:0/3451072168 >> [v2:172.21.15.203:6832/3082732769,v1:172.21.15.203:6833/3082732769] conn(0x7fbc6006a170 msgr2=0x7fbc6006c630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T15:08:55.857 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:55.855+0000 7fbc81d37640 1 --2- 172.21.15.43:0/3451072168 >> [v2:172.21.15.203:6832/3082732769,v1:172.21.15.203:6833/3082732769] conn(0x7fbc6006a170 0x7fbc6006c630 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7fbc68000f60 tx=0x7fbc680079b0 comp rx=0 tx=0).stop 2023-08-09T15:08:55.858 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:55.855+0000 7fbc81d37640 1 -- 172.21.15.43:0/3451072168 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fbc7c06f3d0 msgr2=0x7fbc7c154460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T15:08:55.858 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:55.855+0000 7fbc81d37640 1 --2- 172.21.15.43:0/3451072168 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fbc7c06f3d0 0x7fbc7c154460 secure :-1 s=READY pgs=2252 cs=0 l=1 rev1=1 crypto rx=0x7fbc7000d800 tx=0x7fbc7000dcc0 comp rx=0 tx=0).stop 2023-08-09T15:08:55.858 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:55.855+0000 7fbc81d37640 1 -- 172.21.15.43:0/3451072168 shutdown_connections 2023-08-09T15:08:55.859 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:55.855+0000 7fbc81d37640 1 --2- 172.21.15.43:0/3451072168 >> [v2:172.21.15.203:6832/3082732769,v1:172.21.15.203:6833/3082732769] conn(0x7fbc6006a170 0x7fbc6006c630 unknown :-1 s=CLOSED pgs=72 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T15:08:55.859 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:55.855+0000 7fbc81d37640 1 --2- 172.21.15.43:0/3451072168 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fbc7c0c8670 0x7fbc7c0b2cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T15:08:55.859 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:55.855+0000 7fbc81d37640 1 --2- 172.21.15.43:0/3451072168 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fbc7c0abbb0 0x7fbc7c1f26f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T15:08:55.860 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:55.855+0000 7fbc81d37640 1 --2- 172.21.15.43:0/3451072168 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fbc7c06f3d0 0x7fbc7c154460 unknown :-1 s=CLOSED pgs=2252 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T15:08:55.860 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:55.855+0000 7fbc81d37640 1 -- 172.21.15.43:0/3451072168 >> 172.21.15.43:0/3451072168 conn(0x7fbc7c0af950 msgr2=0x7fbc7c0ad780 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-09T15:08:55.860 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:55.855+0000 7fbc81d37640 1 -- 172.21.15.43:0/3451072168 shutdown_connections 2023-08-09T15:08:55.861 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:55.855+0000 7fbc81d37640 1 -- 172.21.15.43:0/3451072168 wait complete. 2023-08-09T15:08:55.876 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph mgr module enable snap_schedule 2023-08-09T15:08:56.097 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:56.097+0000 7f89593ea640 1 Processor -- start 2023-08-09T15:08:56.097 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:56.097+0000 7f89593ea640 1 -- start start 2023-08-09T15:08:56.098 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:56.098+0000 7f89593ea640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f89540b16b0 0x7f895415ae10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T15:08:56.099 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:56.098+0000 7f89593ea640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f895415b350 0x7f89540b38a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T15:08:56.099 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:56.098+0000 7f89593ea640 1 -- --> v1:172.21.15.43:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8954154b80 con 0x7f89541547a0 2023-08-09T15:08:56.099 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:56.098+0000 7f89593ea640 1 -- --> v2:172.21.15.43:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f89540b1e50 con 0x7f895415b350 2023-08-09T15:08:56.099 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:56.098+0000 7f89593ea640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8954070760 con 0x7f89540b16b0 2023-08-09T15:08:56.100 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:56.099+0000 7f8953fff640 1 --1- >> v1:172.21.15.43:6789/0 conn(0x7f89541547a0 0x7f895415a700 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.43:6789/0 says I am v1:172.21.15.43:49700/0 (socket says 172.21.15.43:49700) 2023-08-09T15:08:56.100 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:56.099+0000 7f8953fff640 1 -- 172.21.15.43:0/3397255095 learned_addr learned my addr 172.21.15.43:0/3397255095 (peer_addr_for_me v1:172.21.15.43:0/0) 2023-08-09T15:08:56.100 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:56.099+0000 7f8958be9640 1 --2- 172.21.15.43:0/3397255095 >> v2:172.21.15.43:3300/0 conn(0x7f895415b350 0x7f89540b38a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T15:08:56.100 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:56.099+0000 7f894b7fe640 1 --2- 172.21.15.43:0/3397255095 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f89540b16b0 0x7f895415ae10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T15:08:56.101 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:56.099+0000 7f8958be9640 1 -- 172.21.15.43:0/3397255095 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f89540b16b0 msgr2=0x7f895415ae10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-09T15:08:56.101 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:56.099+0000 7f89537fe640 1 -- 172.21.15.43:0/3397255095 <== mon.0 v1:172.21.15.43:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 34718723 0 0) 0x7f8954154b80 con 0x7f89541547a0 2023-08-09T15:08:56.101 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:56.099+0000 7f8958be9640 1 --2- 172.21.15.43:0/3397255095 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f89540b16b0 0x7f895415ae10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T15:08:56.101 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:56.099+0000 7f8958be9640 1 -- 172.21.15.43:0/3397255095 >> v1:172.21.15.43:6789/0 conn(0x7f89541547a0 legacy=0x7f895415a700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T15:08:56.101 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:56.099+0000 7f8958be9640 1 -- 172.21.15.43:0/3397255095 --> v2:172.21.15.43:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8954070a40 con 0x7f895415b350 2023-08-09T15:08:56.101 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:56.099+0000 7f894b7fe640 1 --2- 172.21.15.43:0/3397255095 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f89540b16b0 0x7f895415ae10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-08-09T15:08:56.102 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:56.100+0000 7f8958be9640 1 --2- 172.21.15.43:0/3397255095 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f895415b350 0x7f89540b38a0 secure :-1 s=READY pgs=2254 cs=0 l=1 rev1=1 crypto rx=0x7f894c0097a0 tx=0x7f894c02b8b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7874c90614a1f189 server_cookie=0 in_seq=0 out_seq=0 2023-08-09T15:08:56.102 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:56.101+0000 7f89537fe640 1 -- 172.21.15.43:0/3397255095 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f894c02d040 con 0x7f895415b350 2023-08-09T15:08:56.102 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:56.101+0000 7f89537fe640 1 -- 172.21.15.43:0/3397255095 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f894c031810 con 0x7f895415b350 2023-08-09T15:08:56.103 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:56.101+0000 7f89537fe640 1 -- 172.21.15.43:0/3397255095 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f894c031ba0 con 0x7f895415b350 2023-08-09T15:08:56.103 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:56.101+0000 7f89593ea640 1 -- 172.21.15.43:0/3397255095 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f895415b350 msgr2=0x7f89540b38a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T15:08:56.103 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:56.101+0000 7f89593ea640 1 --2- 172.21.15.43:0/3397255095 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f895415b350 0x7f89540b38a0 secure :-1 s=READY pgs=2254 cs=0 l=1 rev1=1 crypto rx=0x7f894c0097a0 tx=0x7f894c02b8b0 comp rx=0 tx=0).stop 2023-08-09T15:08:56.103 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:56.102+0000 7f89593ea640 1 -- 172.21.15.43:0/3397255095 shutdown_connections 2023-08-09T15:08:56.103 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:56.102+0000 7f89593ea640 1 --2- 172.21.15.43:0/3397255095 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f895415b350 0x7f89540b38a0 unknown :-1 s=CLOSED pgs=2254 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T15:08:56.103 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:56.102+0000 7f89593ea640 1 --2- 172.21.15.43:0/3397255095 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f89540b16b0 0x7f895415ae10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T15:08:56.103 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:56.102+0000 7f89593ea640 1 -- 172.21.15.43:0/3397255095 >> 172.21.15.43:0/3397255095 conn(0x7f89540ce640 msgr2=0x7f89540cea50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-09T15:08:56.103 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:56.102+0000 7f89593ea640 1 -- 172.21.15.43:0/3397255095 shutdown_connections 2023-08-09T15:08:56.104 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:56.102+0000 7f89593ea640 1 -- 172.21.15.43:0/3397255095 wait complete. 2023-08-09T15:08:56.104 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:56.103+0000 7f89593ea640 1 Processor -- start 2023-08-09T15:08:56.104 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:56.104+0000 7f89593ea640 1 -- start start 2023-08-09T15:08:56.105 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:56.105+0000 7f89593ea640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f89540b16b0 0x7f89541471f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T15:08:56.105 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:56.105+0000 7f89593ea640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f89541547a0 0x7f89540cc210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T15:08:56.105 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:56.105+0000 7f89593ea640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f895415b350 0x7f895406cb90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T15:08:56.105 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:56.105+0000 7f89593ea640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8954147730 con 0x7f895415b350 2023-08-09T15:08:56.106 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:56.105+0000 7f89593ea640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f89540cccd0 con 0x7f89541547a0 2023-08-09T15:08:56.106 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:56.105+0000 7f89593ea640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f89540ccfb0 con 0x7f89540b16b0 2023-08-09T15:08:56.106 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:56.105+0000 7f8958be9640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f895415b350 0x7f895406cb90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T15:08:56.106 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:56.105+0000 7f8958be9640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f895415b350 0x7f895406cb90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:41246/0 (socket says 172.21.15.43:41246) 2023-08-09T15:08:56.106 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:56.105+0000 7f8958be9640 1 -- 172.21.15.43:0/2468027339 learned_addr learned my addr 172.21.15.43:0/2468027339 (peer_addr_for_me v2:172.21.15.43:0/0) 2023-08-09T15:08:56.107 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:56.105+0000 7f894b7fe640 1 --2- 172.21.15.43:0/2468027339 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f89541547a0 0x7f89540cc210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T15:08:56.107 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:56.105+0000 7f8953fff640 1 --2- 172.21.15.43:0/2468027339 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f89540b16b0 0x7f89541471f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T15:08:56.107 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:56.105+0000 7f8958be9640 1 -- 172.21.15.43:0/2468027339 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f89540b16b0 msgr2=0x7f89541471f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T15:08:56.108 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:56.105+0000 7f8958be9640 1 --2- 172.21.15.43:0/2468027339 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f89540b16b0 0x7f89541471f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T15:08:56.108 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:56.105+0000 7f8958be9640 1 -- 172.21.15.43:0/2468027339 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f89541547a0 msgr2=0x7f89540cc210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T15:08:56.108 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:56.105+0000 7f8958be9640 1 --2- 172.21.15.43:0/2468027339 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f89541547a0 0x7f89540cc210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T15:08:56.108 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:56.105+0000 7f8958be9640 1 -- 172.21.15.43:0/2468027339 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f894c009400 con 0x7f895415b350 2023-08-09T15:08:56.108 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:56.106+0000 7f894b7fe640 1 --2- 172.21.15.43:0/2468027339 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f89541547a0 0x7f89540cc210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-08-09T15:08:56.108 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:56.106+0000 7f8958be9640 1 --2- 172.21.15.43:0/2468027339 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f895415b350 0x7f895406cb90 secure :-1 s=READY pgs=2255 cs=0 l=1 rev1=1 crypto rx=0x7f894c034040 tx=0x7f894c0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-09T15:08:56.108 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:56.106+0000 7f89517fa640 1 -- 172.21.15.43:0/2468027339 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f894c03aad0 con 0x7f895415b350 2023-08-09T15:08:56.109 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:56.106+0000 7f89517fa640 1 -- 172.21.15.43:0/2468027339 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f894c03ac30 con 0x7f895415b350 2023-08-09T15:08:56.109 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:56.106+0000 7f89517fa640 1 -- 172.21.15.43:0/2468027339 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f894c004030 con 0x7f895415b350 2023-08-09T15:08:56.109 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:56.106+0000 7f89593ea640 1 -- 172.21.15.43:0/2468027339 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f89540cd290 con 0x7f895415b350 2023-08-09T15:08:56.109 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:56.106+0000 7f89593ea640 1 -- 172.21.15.43:0/2468027339 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f89540cd850 con 0x7f895415b350 2023-08-09T15:08:56.109 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:56.108+0000 7f89593ea640 1 -- 172.21.15.43:0/2468027339 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f89540cda40 con 0x7f895415b350 2023-08-09T15:08:56.110 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:56.108+0000 7f89517fa640 1 -- 172.21.15.43:0/2468027339 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 106) v1 ==== 88443+0+0 (secure 0 0 0) 0x7f894c042070 con 0x7f895415b350 2023-08-09T15:08:56.110 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:56.110+0000 7f89517fa640 1 --2- 172.21.15.43:0/2468027339 >> [v2:172.21.15.203:6832/3082732769,v1:172.21.15.203:6833/3082732769] conn(0x7f893406a170 0x7f893406c630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T15:08:56.110 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:56.110+0000 7f89517fa640 1 -- 172.21.15.43:0/2468027339 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(205..205 src has 1..205) v4 ==== 6426+0+0 (secure 0 0 0) 0x7f894c0aa860 con 0x7f895415b350 2023-08-09T15:08:56.111 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:56.111+0000 7f8953fff640 1 --2- 172.21.15.43:0/2468027339 >> [v2:172.21.15.203:6832/3082732769,v1:172.21.15.203:6833/3082732769] conn(0x7f893406a170 0x7f893406c630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T15:08:56.112 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:56.112+0000 7f8953fff640 1 --2- 172.21.15.43:0/2468027339 >> [v2:172.21.15.203:6832/3082732769,v1:172.21.15.203:6833/3082732769] conn(0x7f893406a170 0x7f893406c630 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f89440012e0 tx=0x7f8944009040 comp rx=0 tx=0).ready entity=mgr.8500 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-09T15:08:56.114 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:56.115+0000 7f89517fa640 1 -- 172.21.15.43:0/2468027339 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178066 (secure 0 0 0) 0x7f894c075b20 con 0x7f895415b350 2023-08-09T15:08:56.359 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:56.359+0000 7f89593ea640 1 -- 172.21.15.43:0/2468027339 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "mgr module enable", "module": "snap_schedule"} v 0) v1 -- 0x7f895404e390 con 0x7f895415b350 2023-08-09T15:08:57.221 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:57.220+0000 7f89517fa640 1 -- 172.21.15.43:0/2468027339 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mgrmap(e 107) v1 ==== 88460+0+0 (secure 0 0 0) 0x7f894c075ef0 con 0x7f895415b350 2023-08-09T15:08:57.222 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:57.222+0000 7f89517fa640 1 -- 172.21.15.43:0/2468027339 <== mon.0 v2:172.21.15.43:3300/0 8 ==== mon_command_ack([{"prefix": "mgr module enable", "module": "snap_schedule"}]=0 v107) v1 ==== 92+0+0 (secure 0 0 0) 0x7f894c075b20 con 0x7f895415b350 2023-08-09T15:08:57.227 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:57.226+0000 7f89593ea640 1 -- 172.21.15.43:0/2468027339 >> [v2:172.21.15.203:6832/3082732769,v1:172.21.15.203:6833/3082732769] conn(0x7f893406a170 msgr2=0x7f893406c630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T15:08:57.227 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:57.226+0000 7f89593ea640 1 --2- 172.21.15.43:0/2468027339 >> [v2:172.21.15.203:6832/3082732769,v1:172.21.15.203:6833/3082732769] conn(0x7f893406a170 0x7f893406c630 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f89440012e0 tx=0x7f8944009040 comp rx=0 tx=0).stop 2023-08-09T15:08:57.228 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:57.226+0000 7f89593ea640 1 -- 172.21.15.43:0/2468027339 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f895415b350 msgr2=0x7f895406cb90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T15:08:57.228 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:57.226+0000 7f89593ea640 1 --2- 172.21.15.43:0/2468027339 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f895415b350 0x7f895406cb90 secure :-1 s=READY pgs=2255 cs=0 l=1 rev1=1 crypto rx=0x7f894c034040 tx=0x7f894c0046c0 comp rx=0 tx=0).stop 2023-08-09T15:08:57.229 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:57.226+0000 7f89593ea640 1 -- 172.21.15.43:0/2468027339 shutdown_connections 2023-08-09T15:08:57.229 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:57.226+0000 7f89593ea640 1 --2- 172.21.15.43:0/2468027339 >> [v2:172.21.15.203:6832/3082732769,v1:172.21.15.203:6833/3082732769] conn(0x7f893406a170 0x7f893406c630 unknown :-1 s=CLOSED pgs=73 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T15:08:57.229 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:57.226+0000 7f89593ea640 1 --2- 172.21.15.43:0/2468027339 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f895415b350 0x7f895406cb90 unknown :-1 s=CLOSED pgs=2255 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T15:08:57.229 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:57.226+0000 7f89593ea640 1 --2- 172.21.15.43:0/2468027339 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f89541547a0 0x7f89540cc210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T15:08:57.229 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:57.226+0000 7f89593ea640 1 --2- 172.21.15.43:0/2468027339 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f89540b16b0 0x7f89541471f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T15:08:57.230 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:57.226+0000 7f89593ea640 1 -- 172.21.15.43:0/2468027339 >> 172.21.15.43:0/2468027339 conn(0x7f89540ce640 msgr2=0x7f895415a4e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-09T15:08:57.230 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:57.226+0000 7f89593ea640 1 -- 172.21.15.43:0/2468027339 shutdown_connections 2023-08-09T15:08:57.230 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:57.226+0000 7f89593ea640 1 -- 172.21.15.43:0/2468027339 wait complete. 2023-08-09T15:08:57.251 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mgr mgr/snap_schedule/allow_m_granularity True 2023-08-09T15:08:57.441 INFO:tasks.ceph.mgr.x.smithi203.stderr:2023-08-09T15:08:57.440+0000 7fdd22e2b040 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2023-08-09T15:08:57.468 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:57.467+0000 7f294204b640 1 Processor -- start 2023-08-09T15:08:57.468 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:57.467+0000 7f294204b640 1 -- start start 2023-08-09T15:08:57.469 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:57.469+0000 7f294204b640 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7f293c15c2d0 0x7f293c0b7df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T15:08:57.469 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:57.469+0000 7f294204b640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f293c0cebf0 0x7f293c0cf040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T15:08:57.469 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:57.469+0000 7f294204b640 1 -- --> v1:172.21.15.203:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f293c0cf580 con 0x7f293c0d1b30 2023-08-09T15:08:57.470 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:57.469+0000 7f294204b640 1 -- --> v2:172.21.15.43:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f293c0d16a0 con 0x7f293c0cebf0 2023-08-09T15:08:57.470 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:57.469+0000 7f294204b640 1 -- --> v2:172.21.15.203:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f293c0d1fa0 con 0x7f293c15c2d0 2023-08-09T15:08:57.471 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:57.469+0000 7f294184a640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f293c0cebf0 0x7f293c0cf040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T15:08:57.471 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:57.470+0000 7f294184a640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f293c0cebf0 0x7f293c0cf040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:41266/0 (socket says 172.21.15.43:41266) 2023-08-09T15:08:57.471 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:57.470+0000 7f294184a640 1 -- 172.21.15.43:0/3425950513 learned_addr learned my addr 172.21.15.43:0/3425950513 (peer_addr_for_me v2:172.21.15.43:0/0) 2023-08-09T15:08:57.471 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:57.470+0000 7f2941049640 1 --2- 172.21.15.43:0/3425950513 >> v2:172.21.15.203:3300/0 conn(0x7f293c15c2d0 0x7f293c0b7df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T15:08:57.471 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:57.470+0000 7f294184a640 1 -- 172.21.15.43:0/3425950513 >> v2:172.21.15.203:3300/0 conn(0x7f293c15c2d0 msgr2=0x7f293c0b7df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-09T15:08:57.471 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:57.470+0000 7f294184a640 1 --2- 172.21.15.43:0/3425950513 >> v2:172.21.15.203:3300/0 conn(0x7f293c15c2d0 0x7f293c0b7df0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T15:08:57.471 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:57.470+0000 7f294184a640 1 -- 172.21.15.43:0/3425950513 >> v1:172.21.15.203:6789/0 conn(0x7f293c0d1b30 legacy=0x7f293c0cd020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T15:08:57.472 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:57.470+0000 7f294184a640 1 -- 172.21.15.43:0/3425950513 --> v2:172.21.15.43:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f293c0d0d70 con 0x7f293c0cebf0 2023-08-09T15:08:57.472 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:57.471+0000 7f294184a640 1 --2- 172.21.15.43:0/3425950513 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f293c0cebf0 0x7f293c0cf040 secure :-1 s=READY pgs=2259 cs=0 l=1 rev1=1 crypto rx=0x7f293c06ae00 tx=0x7f293402bf80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dc3c0570e025d80f server_cookie=0 in_seq=0 out_seq=0 2023-08-09T15:08:57.472 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:57.472+0000 7f2933fff640 1 -- 172.21.15.43:0/3425950513 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f293402cb20 con 0x7f293c0cebf0 2023-08-09T15:08:57.472 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:57.472+0000 7f2933fff640 1 -- 172.21.15.43:0/3425950513 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f293402cc80 con 0x7f293c0cebf0 2023-08-09T15:08:57.473 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:57.472+0000 7f2933fff640 1 -- 172.21.15.43:0/3425950513 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f29340343f0 con 0x7f293c0cebf0 2023-08-09T15:08:57.473 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:57.472+0000 7f294204b640 1 -- 172.21.15.43:0/3425950513 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f293c0cebf0 msgr2=0x7f293c0cf040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T15:08:57.474 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:57.472+0000 7f294204b640 1 --2- 172.21.15.43:0/3425950513 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f293c0cebf0 0x7f293c0cf040 secure :-1 s=READY pgs=2259 cs=0 l=1 rev1=1 crypto rx=0x7f293c06ae00 tx=0x7f293402bf80 comp rx=0 tx=0).stop 2023-08-09T15:08:57.474 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:57.473+0000 7f294204b640 1 -- 172.21.15.43:0/3425950513 shutdown_connections 2023-08-09T15:08:57.474 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:57.473+0000 7f294204b640 1 --2- 172.21.15.43:0/3425950513 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f293c0cebf0 0x7f293c0cf040 unknown :-1 s=CLOSED pgs=2259 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T15:08:57.474 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:57.473+0000 7f294204b640 1 --2- 172.21.15.43:0/3425950513 >> v2:172.21.15.203:3300/0 conn(0x7f293c15c2d0 0x7f293c0b7df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T15:08:57.474 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:57.473+0000 7f294204b640 1 -- 172.21.15.43:0/3425950513 >> 172.21.15.43:0/3425950513 conn(0x7f293c06a230 msgr2=0x7f293c06a640 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-09T15:08:57.475 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:57.473+0000 7f294204b640 1 -- 172.21.15.43:0/3425950513 shutdown_connections 2023-08-09T15:08:57.475 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:57.473+0000 7f294204b640 1 -- 172.21.15.43:0/3425950513 wait complete. 2023-08-09T15:08:57.478 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:57.474+0000 7f294204b640 1 Processor -- start 2023-08-09T15:08:57.478 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:57.475+0000 7f294204b640 1 -- start start 2023-08-09T15:08:57.479 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:57.475+0000 7f294204b640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f293c0cebf0 0x7f293c14c290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T15:08:57.479 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:57.475+0000 7f294204b640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f293c0d1b30 0x7f293c0cb140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T15:08:57.479 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:57.476+0000 7f294204b640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f293c15c2d0 0x7f293c0cb680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T15:08:57.480 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:57.476+0000 7f294204b640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f293c15d490 con 0x7f293c15c2d0 2023-08-09T15:08:57.480 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:57.476+0000 7f294204b640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f293c15d770 con 0x7f293c0cebf0 2023-08-09T15:08:57.480 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:57.476+0000 7f294204b640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f293c15da50 con 0x7f293c0d1b30 2023-08-09T15:08:57.480 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:57.476+0000 7f294184a640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f293c15c2d0 0x7f293c0cb680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T15:08:57.481 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:57.476+0000 7f294184a640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f293c15c2d0 0x7f293c0cb680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:41270/0 (socket says 172.21.15.43:41270) 2023-08-09T15:08:57.481 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:57.476+0000 7f294184a640 1 -- 172.21.15.43:0/1906849149 learned_addr learned my addr 172.21.15.43:0/1906849149 (peer_addr_for_me v2:172.21.15.43:0/0) 2023-08-09T15:08:57.481 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:57.476+0000 7f2941049640 1 --2- 172.21.15.43:0/1906849149 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f293c0cebf0 0x7f293c14c290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T15:08:57.481 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:57.476+0000 7f2940848640 1 --2- 172.21.15.43:0/1906849149 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f293c0d1b30 0x7f293c0cb140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T15:08:57.481 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:57.476+0000 7f294184a640 1 -- 172.21.15.43:0/1906849149 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f293c0d1b30 msgr2=0x7f293c0cb140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T15:08:57.481 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:57.476+0000 7f294184a640 1 --2- 172.21.15.43:0/1906849149 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f293c0d1b30 0x7f293c0cb140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T15:08:57.482 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:57.476+0000 7f294184a640 1 -- 172.21.15.43:0/1906849149 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f293c0cebf0 msgr2=0x7f293c14c290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T15:08:57.482 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:57.476+0000 7f294184a640 1 --2- 172.21.15.43:0/1906849149 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f293c0cebf0 0x7f293c14c290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T15:08:57.482 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:57.476+0000 7f294184a640 1 -- 172.21.15.43:0/1906849149 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2934009b30 con 0x7f293c15c2d0 2023-08-09T15:08:57.482 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:57.476+0000 7f2940848640 1 --2- 172.21.15.43:0/1906849149 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f293c0d1b30 0x7f293c0cb140 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-08-09T15:08:57.482 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:57.477+0000 7f294184a640 1 --2- 172.21.15.43:0/1906849149 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f293c15c2d0 0x7f293c0cb680 secure :-1 s=READY pgs=2260 cs=0 l=1 rev1=1 crypto rx=0x7f2934036040 tx=0x7f2934004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-09T15:08:57.482 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:57.477+0000 7f2931ffb640 1 -- 172.21.15.43:0/1906849149 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2934009510 con 0x7f293c15c2d0 2023-08-09T15:08:57.482 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:57.477+0000 7f2931ffb640 1 -- 172.21.15.43:0/1906849149 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f293402c920 con 0x7f293c15c2d0 2023-08-09T15:08:57.482 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:57.477+0000 7f2931ffb640 1 -- 172.21.15.43:0/1906849149 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f293403e690 con 0x7f293c15c2d0 2023-08-09T15:08:57.483 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:57.477+0000 7f294204b640 1 -- 172.21.15.43:0/1906849149 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f293c15dd30 con 0x7f293c15c2d0 2023-08-09T15:08:57.483 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:57.477+0000 7f294204b640 1 -- 172.21.15.43:0/1906849149 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f293c14c7d0 con 0x7f293c15c2d0 2023-08-09T15:08:57.483 INFO:tasks.ceph.mgr.y.smithi043.stderr:2023-08-09T15:08:57.478+0000 7ff5386d4040 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2023-08-09T15:08:57.484 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:57.479+0000 7f294204b640 1 -- 172.21.15.43:0/1906849149 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f293c15e1f0 con 0x7f293c15c2d0 2023-08-09T15:08:57.484 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:57.479+0000 7f2931ffb640 1 -- 172.21.15.43:0/1906849149 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 107) v1 ==== 88460+0+0 (secure 0 0 0) 0x7f293403e7f0 con 0x7f293c15c2d0 2023-08-09T15:08:57.484 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:57.481+0000 7f2931ffb640 1 --2- 172.21.15.43:0/1906849149 >> [v2:172.21.15.203:6832/3082732769,v1:172.21.15.203:6833/3082732769] conn(0x7f292006a1c0 0x7f292006c680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T15:08:57.484 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:57.481+0000 7f2931ffb640 1 -- 172.21.15.43:0/1906849149 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(205..205 src has 1..205) v4 ==== 6426+0+0 (secure 0 0 0) 0x7f29340ab4b0 con 0x7f293c15c2d0 2023-08-09T15:08:57.484 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:57.481+0000 7f2941049640 1 -- 172.21.15.43:0/1906849149 >> [v2:172.21.15.203:6832/3082732769,v1:172.21.15.203:6833/3082732769] conn(0x7f292006a1c0 msgr2=0x7f292006c680 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.203:6832/3082732769 2023-08-09T15:08:57.485 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:57.481+0000 7f2941049640 1 --2- 172.21.15.43:0/1906849149 >> [v2:172.21.15.203:6832/3082732769,v1:172.21.15.203:6833/3082732769] conn(0x7f292006a1c0 0x7f292006c680 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-08-09T15:08:57.485 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:57.485+0000 7f2931ffb640 1 -- 172.21.15.43:0/1906849149 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178066 (secure 0 0 0) 0x7f2934076880 con 0x7f293c15c2d0 2023-08-09T15:08:57.682 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:57.682+0000 7f294204b640 1 -- 172.21.15.43:0/1906849149 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command([{prefix=config set, name=mgr/snap_schedule/allow_m_granularity}] v 0) v1 -- 0x7f293c04e390 con 0x7f293c15c2d0 2023-08-09T15:08:57.682 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:57.682+0000 7f2941049640 1 -- 172.21.15.43:0/1906849149 >> [v2:172.21.15.203:6832/3082732769,v1:172.21.15.203:6833/3082732769] conn(0x7f292006a1c0 msgr2=0x7f292006c680 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.203:6832/3082732769 2023-08-09T15:08:57.683 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:57.682+0000 7f2941049640 1 --2- 172.21.15.43:0/1906849149 >> [v2:172.21.15.203:6832/3082732769,v1:172.21.15.203:6833/3082732769] conn(0x7f292006a1c0 0x7f292006c680 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2023-08-09T15:08:57.687 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:57.687+0000 7f2931ffb640 1 -- 172.21.15.43:0/1906849149 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mgr/snap_schedule/allow_m_granularity}]=0 v65)=0 v65) v1 ==== 138+0+0 (secure 0 0 0) 0x7f2934079ea0 con 0x7f293c15c2d0 2023-08-09T15:08:57.690 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:57.690+0000 7f294204b640 1 -- 172.21.15.43:0/1906849149 >> [v2:172.21.15.203:6832/3082732769,v1:172.21.15.203:6833/3082732769] conn(0x7f292006a1c0 msgr2=0x7f292006c680 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-08-09T15:08:57.690 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:57.690+0000 7f294204b640 1 --2- 172.21.15.43:0/1906849149 >> [v2:172.21.15.203:6832/3082732769,v1:172.21.15.203:6833/3082732769] conn(0x7f292006a1c0 0x7f292006c680 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T15:08:57.691 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:57.690+0000 7f294204b640 1 -- 172.21.15.43:0/1906849149 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f293c15c2d0 msgr2=0x7f293c0cb680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T15:08:57.691 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:57.690+0000 7f294204b640 1 --2- 172.21.15.43:0/1906849149 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f293c15c2d0 0x7f293c0cb680 secure :-1 s=READY pgs=2260 cs=0 l=1 rev1=1 crypto rx=0x7f2934036040 tx=0x7f2934004a40 comp rx=0 tx=0).stop 2023-08-09T15:08:57.691 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:57.690+0000 7f294204b640 1 -- 172.21.15.43:0/1906849149 shutdown_connections 2023-08-09T15:08:57.691 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:57.690+0000 7f294204b640 1 --2- 172.21.15.43:0/1906849149 >> [v2:172.21.15.203:6832/3082732769,v1:172.21.15.203:6833/3082732769] conn(0x7f292006a1c0 0x7f292006c680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T15:08:57.691 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:57.690+0000 7f294204b640 1 --2- 172.21.15.43:0/1906849149 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f293c15c2d0 0x7f293c0cb680 unknown :-1 s=CLOSED pgs=2260 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T15:08:57.691 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:57.690+0000 7f294204b640 1 --2- 172.21.15.43:0/1906849149 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f293c0d1b30 0x7f293c0cb140 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T15:08:57.691 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:57.690+0000 7f294204b640 1 --2- 172.21.15.43:0/1906849149 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f293c0cebf0 0x7f293c14c290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T15:08:57.691 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:57.690+0000 7f294204b640 1 -- 172.21.15.43:0/1906849149 >> 172.21.15.43:0/1906849149 conn(0x7f293c06a230 msgr2=0x7f293c15a610 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-09T15:08:57.692 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:57.690+0000 7f294204b640 1 -- 172.21.15.43:0/1906849149 shutdown_connections 2023-08-09T15:08:57.692 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:57.690+0000 7f294204b640 1 -- 172.21.15.43:0/1906849149 wait complete. 2023-08-09T15:08:57.709 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mgr mgr/snap_schedule/dump_on_update True 2023-08-09T15:08:57.934 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:57.933+0000 7f05ff369640 1 Processor -- start 2023-08-09T15:08:57.934 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:57.934+0000 7f05ff369640 1 -- start start 2023-08-09T15:08:57.935 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:57.935+0000 7f05ff369640 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7f05f8157ea0 0x7f05f80c97d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T15:08:57.936 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:57.935+0000 7f05ff369640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f05f815af10 0x7f05f80c9d10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T15:08:57.936 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:57.935+0000 7f05ff369640 1 -- --> v1:172.21.15.203:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f05f8158280 con 0x7f05f80c90a0 2023-08-09T15:08:57.936 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:57.935+0000 7f05ff369640 1 -- --> v2:172.21.15.203:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f05f815b840 con 0x7f05f8157ea0 2023-08-09T15:08:57.936 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:57.935+0000 7f05ff369640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f05f80b3810 con 0x7f05f815af10 2023-08-09T15:08:57.937 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:57.936+0000 7f05feb68640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f05f815af10 0x7f05f80c9d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T15:08:57.937 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:57.936+0000 7f05fe367640 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7f05f8157ea0 0x7f05f80c97d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T15:08:57.937 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:57.936+0000 7f05fe367640 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7f05f8157ea0 0x7f05f80c97d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.203:3300/0 says I am v2:172.21.15.43:43226/0 (socket says 172.21.15.43:43226) 2023-08-09T15:08:57.938 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:57.936+0000 7f05fe367640 1 -- 172.21.15.43:0/3605974512 learned_addr learned my addr 172.21.15.43:0/3605974512 (peer_addr_for_me v2:172.21.15.43:0/0) 2023-08-09T15:08:57.938 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:57.937+0000 7f05fd365640 1 -- 172.21.15.43:0/3605974512 <== mon.1 v1:172.21.15.203:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2339018295 0 0) 0x7f05e8001110 con 0x7f05f80c90a0 2023-08-09T15:08:57.938 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:57.937+0000 7f05fd365640 1 -- 172.21.15.43:0/3605974512 --> v1:172.21.15.203:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f05e4003520 con 0x7f05f80c90a0 2023-08-09T15:08:57.938 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:57.937+0000 7f05fe367640 1 -- 172.21.15.43:0/3605974512 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f05f815af10 msgr2=0x7f05f80c9d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-09T15:08:57.938 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:57.937+0000 7f05fe367640 1 --2- 172.21.15.43:0/3605974512 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f05f815af10 0x7f05f80c9d10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T15:08:57.938 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:57.937+0000 7f05fe367640 1 -- 172.21.15.43:0/3605974512 >> v1:172.21.15.203:6789/0 conn(0x7f05f80c90a0 legacy=0x7f05f815a800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T15:08:57.938 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:57.937+0000 7f05fe367640 1 -- 172.21.15.43:0/3605974512 --> v2:172.21.15.203:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f05f815b560 con 0x7f05f8157ea0 2023-08-09T15:08:57.939 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:57.939+0000 7f05fe367640 1 --2- 172.21.15.43:0/3605974512 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f05f8157ea0 0x7f05f80c97d0 secure :-1 s=READY pgs=823 cs=0 l=1 rev1=1 crypto rx=0x7f05f4009e80 tx=0x7f05f402bec0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=75e3316a55f78974 server_cookie=0 in_seq=0 out_seq=0 2023-08-09T15:08:57.939 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:57.939+0000 7f05fd365640 1 -- 172.21.15.43:0/3605974512 <== mon.1 v2:172.21.15.203:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f05f402ca60 con 0x7f05f8157ea0 2023-08-09T15:08:57.939 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:57.939+0000 7f05fd365640 1 -- 172.21.15.43:0/3605974512 <== mon.1 v2:172.21.15.203:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f05f402cbc0 con 0x7f05f8157ea0 2023-08-09T15:08:57.940 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:57.939+0000 7f05fd365640 1 -- 172.21.15.43:0/3605974512 <== mon.1 v2:172.21.15.203:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f05f40343f0 con 0x7f05f8157ea0 2023-08-09T15:08:57.940 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:57.940+0000 7f05ff369640 1 -- 172.21.15.43:0/3605974512 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f05f8157ea0 msgr2=0x7f05f80c97d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T15:08:57.941 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:57.940+0000 7f05ff369640 1 --2- 172.21.15.43:0/3605974512 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f05f8157ea0 0x7f05f80c97d0 secure :-1 s=READY pgs=823 cs=0 l=1 rev1=1 crypto rx=0x7f05f4009e80 tx=0x7f05f402bec0 comp rx=0 tx=0).stop 2023-08-09T15:08:57.941 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:57.940+0000 7f05ff369640 1 -- 172.21.15.43:0/3605974512 shutdown_connections 2023-08-09T15:08:57.941 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:57.940+0000 7f05ff369640 1 --2- 172.21.15.43:0/3605974512 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f05f815af10 0x7f05f80c9d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T15:08:57.941 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:57.940+0000 7f05ff369640 1 --2- 172.21.15.43:0/3605974512 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f05f8157ea0 0x7f05f80c97d0 unknown :-1 s=CLOSED pgs=823 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T15:08:57.941 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:57.940+0000 7f05ff369640 1 -- 172.21.15.43:0/3605974512 >> 172.21.15.43:0/3605974512 conn(0x7f05f80cddd0 msgr2=0x7f05f80ce1e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-09T15:08:57.941 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:57.940+0000 7f05ff369640 1 -- 172.21.15.43:0/3605974512 shutdown_connections 2023-08-09T15:08:57.941 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:57.940+0000 7f05ff369640 1 -- 172.21.15.43:0/3605974512 wait complete. 2023-08-09T15:08:57.942 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:57.942+0000 7f05ff369640 1 Processor -- start 2023-08-09T15:08:57.942 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:57.942+0000 7f05ff369640 1 -- start start 2023-08-09T15:08:57.943 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:57.943+0000 7f05ff369640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f05f80c90a0 0x7f05f80c2820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T15:08:57.944 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:57.943+0000 7f05ff369640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f05f8157ea0 0x7f05f81573e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T15:08:57.944 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:57.943+0000 7f05ff369640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f05f815af10 0x7f05f80c7590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T15:08:57.944 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:57.943+0000 7f05ff369640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f05f81f1cf0 con 0x7f05f815af10 2023-08-09T15:08:57.944 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:57.943+0000 7f05ff369640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f05f80c5140 con 0x7f05f8157ea0 2023-08-09T15:08:57.944 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:57.943+0000 7f05ff369640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f05f81f1fb0 con 0x7f05f80c90a0 2023-08-09T15:08:57.945 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:57.943+0000 7f05feb68640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f05f815af10 0x7f05f80c7590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T15:08:57.945 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:57.943+0000 7f05feb68640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f05f815af10 0x7f05f80c7590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:41278/0 (socket says 172.21.15.43:41278) 2023-08-09T15:08:57.945 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:57.943+0000 7f05feb68640 1 -- 172.21.15.43:0/1883670778 learned_addr learned my addr 172.21.15.43:0/1883670778 (peer_addr_for_me v2:172.21.15.43:0/0) 2023-08-09T15:08:57.945 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:57.943+0000 7f05fe367640 1 --2- 172.21.15.43:0/1883670778 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f05f80c90a0 0x7f05f80c2820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T15:08:57.945 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:57.943+0000 7f05fdb66640 1 --2- 172.21.15.43:0/1883670778 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f05f8157ea0 0x7f05f81573e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T15:08:57.946 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:57.944+0000 7f05feb68640 1 -- 172.21.15.43:0/1883670778 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f05f80c90a0 msgr2=0x7f05f80c2820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T15:08:57.946 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:57.944+0000 7f05feb68640 1 --2- 172.21.15.43:0/1883670778 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f05f80c90a0 0x7f05f80c2820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T15:08:57.947 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:57.944+0000 7f05feb68640 1 -- 172.21.15.43:0/1883670778 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f05f8157ea0 msgr2=0x7f05f81573e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T15:08:57.947 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:57.944+0000 7f05feb68640 1 --2- 172.21.15.43:0/1883670778 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f05f8157ea0 0x7f05f81573e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T15:08:57.947 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:57.944+0000 7f05feb68640 1 -- 172.21.15.43:0/1883670778 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f05f4009b30 con 0x7f05f815af10 2023-08-09T15:08:57.947 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:57.944+0000 7f05feb68640 1 --2- 172.21.15.43:0/1883670778 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f05f815af10 0x7f05f80c7590 secure :-1 s=READY pgs=2261 cs=0 l=1 rev1=1 crypto rx=0x7f05f000e7f0 tx=0x7f05f000ebf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-09T15:08:57.947 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:57.944+0000 7f05eeffd640 1 -- 172.21.15.43:0/1883670778 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f05f0009510 con 0x7f05f815af10 2023-08-09T15:08:57.947 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:57.945+0000 7f05eeffd640 1 -- 172.21.15.43:0/1883670778 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f05f000ee40 con 0x7f05f815af10 2023-08-09T15:08:57.947 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:57.945+0000 7f05eeffd640 1 -- 172.21.15.43:0/1883670778 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f05f000cb80 con 0x7f05f815af10 2023-08-09T15:08:57.948 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:57.945+0000 7f05ff369640 1 -- 172.21.15.43:0/1883670778 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f05f81f2290 con 0x7f05f815af10 2023-08-09T15:08:57.948 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:57.945+0000 7f05ff369640 1 -- 172.21.15.43:0/1883670778 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f05f81f2850 con 0x7f05f815af10 2023-08-09T15:08:57.948 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:57.946+0000 7f05ff369640 1 -- 172.21.15.43:0/1883670778 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f05f81f52c0 con 0x7f05f815af10 2023-08-09T15:08:57.949 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:57.947+0000 7f05eeffd640 1 -- 172.21.15.43:0/1883670778 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 107) v1 ==== 88460+0+0 (secure 0 0 0) 0x7f05f000cce0 con 0x7f05f815af10 2023-08-09T15:08:57.949 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:57.948+0000 7f05eeffd640 1 --2- 172.21.15.43:0/1883670778 >> [v2:172.21.15.203:6832/3082732769,v1:172.21.15.203:6833/3082732769] conn(0x7f05dc06a1f0 0x7f05dc06c6b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T15:08:57.949 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:57.948+0000 7f05eeffd640 1 -- 172.21.15.43:0/1883670778 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(205..205 src has 1..205) v4 ==== 6426+0+0 (secure 0 0 0) 0x7f05f00888b0 con 0x7f05f815af10 2023-08-09T15:08:57.949 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:57.948+0000 7f05fe367640 1 -- 172.21.15.43:0/1883670778 >> [v2:172.21.15.203:6832/3082732769,v1:172.21.15.203:6833/3082732769] conn(0x7f05dc06a1f0 msgr2=0x7f05dc06c6b0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.203:6832/3082732769 2023-08-09T15:08:57.949 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:57.948+0000 7f05fe367640 1 --2- 172.21.15.43:0/1883670778 >> [v2:172.21.15.203:6832/3082732769,v1:172.21.15.203:6833/3082732769] conn(0x7f05dc06a1f0 0x7f05dc06c6b0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-08-09T15:08:57.952 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:57.952+0000 7f05eeffd640 1 -- 172.21.15.43:0/1883670778 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178066 (secure 0 0 0) 0x7f05f0053d30 con 0x7f05f815af10 2023-08-09T15:08:58.147 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:58.146+0000 7f05ff369640 1 -- 172.21.15.43:0/1883670778 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command([{prefix=config set, name=mgr/snap_schedule/dump_on_update}] v 0) v1 -- 0x7f05f804e390 con 0x7f05f815af10 2023-08-09T15:08:58.150 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:58.149+0000 7f05fe367640 1 -- 172.21.15.43:0/1883670778 >> [v2:172.21.15.203:6832/3082732769,v1:172.21.15.203:6833/3082732769] conn(0x7f05dc06a1f0 msgr2=0x7f05dc06c6b0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.203:6832/3082732769 2023-08-09T15:08:58.150 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:58.149+0000 7f05fe367640 1 --2- 172.21.15.43:0/1883670778 >> [v2:172.21.15.203:6832/3082732769,v1:172.21.15.203:6833/3082732769] conn(0x7f05dc06a1f0 0x7f05dc06c6b0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2023-08-09T15:08:58.152 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:58.151+0000 7f05eeffd640 1 -- 172.21.15.43:0/1883670778 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mgr/snap_schedule/dump_on_update}]=0 v66)=0 v66) v1 ==== 133+0+0 (secure 0 0 0) 0x7f05f0057350 con 0x7f05f815af10 2023-08-09T15:08:58.154 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:58.154+0000 7f05ff369640 1 -- 172.21.15.43:0/1883670778 >> [v2:172.21.15.203:6832/3082732769,v1:172.21.15.203:6833/3082732769] conn(0x7f05dc06a1f0 msgr2=0x7f05dc06c6b0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-08-09T15:08:58.155 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:58.154+0000 7f05ff369640 1 --2- 172.21.15.43:0/1883670778 >> [v2:172.21.15.203:6832/3082732769,v1:172.21.15.203:6833/3082732769] conn(0x7f05dc06a1f0 0x7f05dc06c6b0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T15:08:58.155 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:58.154+0000 7f05ff369640 1 -- 172.21.15.43:0/1883670778 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f05f815af10 msgr2=0x7f05f80c7590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T15:08:58.155 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:58.154+0000 7f05ff369640 1 --2- 172.21.15.43:0/1883670778 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f05f815af10 0x7f05f80c7590 secure :-1 s=READY pgs=2261 cs=0 l=1 rev1=1 crypto rx=0x7f05f000e7f0 tx=0x7f05f000ebf0 comp rx=0 tx=0).stop 2023-08-09T15:08:58.155 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:58.154+0000 7f05ff369640 1 -- 172.21.15.43:0/1883670778 shutdown_connections 2023-08-09T15:08:58.155 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:58.154+0000 7f05ff369640 1 --2- 172.21.15.43:0/1883670778 >> [v2:172.21.15.203:6832/3082732769,v1:172.21.15.203:6833/3082732769] conn(0x7f05dc06a1f0 0x7f05dc06c6b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T15:08:58.155 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:58.154+0000 7f05ff369640 1 --2- 172.21.15.43:0/1883670778 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f05f815af10 0x7f05f80c7590 unknown :-1 s=CLOSED pgs=2261 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T15:08:58.156 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:58.154+0000 7f05ff369640 1 --2- 172.21.15.43:0/1883670778 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f05f8157ea0 0x7f05f81573e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T15:08:58.156 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:58.154+0000 7f05ff369640 1 --2- 172.21.15.43:0/1883670778 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f05f80c90a0 0x7f05f80c2820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T15:08:58.156 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:58.154+0000 7f05ff369640 1 -- 172.21.15.43:0/1883670778 >> 172.21.15.43:0/1883670778 conn(0x7f05f80cddd0 msgr2=0x7f05f80c95c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-09T15:08:58.156 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:58.154+0000 7f05ff369640 1 -- 172.21.15.43:0/1883670778 shutdown_connections 2023-08-09T15:08:58.156 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:58.154+0000 7f05ff369640 1 -- 172.21.15.43:0/1883670778 wait complete. 2023-08-09T15:08:58.157 INFO:tasks.ceph.mgr.x.smithi203.stderr:2023-08-09T15:08:58.154+0000 7fdd22e2b040 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2023-08-09T15:08:58.172 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-09T15:08:58.173 DEBUG:teuthology.orchestra.run.smithi043:> (cd /home/ubuntu/cephtest/mnt.0 && exec mkdir -p snap_test_dir1) 2023-08-09T15:08:58.235 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs snap-schedule add --fs cephfs snap_test_dir1 1M 2023-08-09T15:08:58.258 INFO:tasks.ceph.mgr.y.smithi043.stderr:2023-08-09T15:08:58.258+0000 7ff5386d4040 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2023-08-09T15:08:58.277 INFO:tasks.ceph.mgr.x.smithi203.stderr:2023-08-09T15:08:58.275+0000 7fdd22e2b040 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2023-08-09T15:08:58.332 INFO:tasks.ceph.mgr.x.smithi203.stderr:2023-08-09T15:08:58.331+0000 7fdd22e2b040 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2023-08-09T15:08:58.382 INFO:tasks.ceph.mgr.x.smithi203.stderr:2023-08-09T15:08:58.380+0000 7fdd22e2b040 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2023-08-09T15:08:58.386 INFO:tasks.ceph.mgr.y.smithi043.stderr:2023-08-09T15:08:58.386+0000 7ff5386d4040 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2023-08-09T15:08:58.447 INFO:tasks.ceph.mgr.y.smithi043.stderr:2023-08-09T15:08:58.446+0000 7ff5386d4040 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2023-08-09T15:08:58.458 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:58.458+0000 7f855a9e1640 1 Processor -- start 2023-08-09T15:08:58.458 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:58.458+0000 7f855a9e1640 1 -- start start 2023-08-09T15:08:58.459 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:58.459+0000 7f855a9e1640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f855415a510 0x7f8554160fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T15:08:58.460 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:58.459+0000 7f855a9e1640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f855415d5e0 0x7f85540b3ec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T15:08:58.460 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:58.459+0000 7f855a9e1640 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7f855415da60 0x7f85540b4400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T15:08:58.460 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:58.459+0000 7f855a9e1640 1 -- --> v2:172.21.15.43:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f855415a8f0 con 0x7f855415a510 2023-08-09T15:08:58.460 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:58.459+0000 7f855a9e1640 1 -- --> v2:172.21.15.203:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f85540ccd30 con 0x7f855415da60 2023-08-09T15:08:58.460 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:58.459+0000 7f855a9e1640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f85540cdd40 con 0x7f855415d5e0 2023-08-09T15:08:58.461 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:58.460+0000 7f8553fff640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f855415a510 0x7f8554160fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T15:08:58.462 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:58.460+0000 7f8553fff640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f855415a510 0x7f8554160fe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:41292/0 (socket says 172.21.15.43:41292) 2023-08-09T15:08:58.462 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:58.460+0000 7f8553fff640 1 -- 172.21.15.43:0/2723783545 learned_addr learned my addr 172.21.15.43:0/2723783545 (peer_addr_for_me v2:172.21.15.43:0/0) 2023-08-09T15:08:58.462 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:58.460+0000 7f85537fe640 1 --2- 172.21.15.43:0/2723783545 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f855415d5e0 0x7f85540b3ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T15:08:58.462 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:58.460+0000 7f8558f7e640 1 --2- 172.21.15.43:0/2723783545 >> v2:172.21.15.203:3300/0 conn(0x7f855415da60 0x7f85540b4400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T15:08:58.462 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:58.460+0000 7f8553fff640 1 -- 172.21.15.43:0/2723783545 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f855415d5e0 msgr2=0x7f85540b3ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-09T15:08:58.462 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:58.460+0000 7f8553fff640 1 --2- 172.21.15.43:0/2723783545 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f855415d5e0 0x7f85540b3ec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T15:08:58.462 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:58.460+0000 7f8553fff640 1 -- 172.21.15.43:0/2723783545 >> v2:172.21.15.203:3300/0 conn(0x7f855415da60 msgr2=0x7f85540b4400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-09T15:08:58.462 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:58.460+0000 7f8553fff640 1 --2- 172.21.15.43:0/2723783545 >> v2:172.21.15.203:3300/0 conn(0x7f855415da60 0x7f85540b4400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T15:08:58.463 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:58.460+0000 7f8553fff640 1 -- 172.21.15.43:0/2723783545 --> v2:172.21.15.43:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f85540ccf80 con 0x7f855415a510 2023-08-09T15:08:58.463 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:58.461+0000 7f8553fff640 1 --2- 172.21.15.43:0/2723783545 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f855415a510 0x7f8554160fe0 secure :-1 s=READY pgs=2262 cs=0 l=1 rev1=1 crypto rx=0x7f85540d1c30 tx=0x7f854402b900 comp rx=0 tx=0).ready entity=mon.0 client_cookie=59c08b2578ddcc00 server_cookie=0 in_seq=0 out_seq=0 2023-08-09T15:08:58.464 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:58.462+0000 7f8552ffd640 1 -- 172.21.15.43:0/2723783545 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8544031860 con 0x7f855415a510 2023-08-09T15:08:58.464 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:58.462+0000 7f8552ffd640 1 -- 172.21.15.43:0/2723783545 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f85440319c0 con 0x7f855415a510 2023-08-09T15:08:58.464 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:58.462+0000 7f8552ffd640 1 -- 172.21.15.43:0/2723783545 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8544031d50 con 0x7f855415a510 2023-08-09T15:08:58.465 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:58.462+0000 7f855a9e1640 1 -- 172.21.15.43:0/2723783545 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f855415a510 msgr2=0x7f8554160fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T15:08:58.465 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:58.462+0000 7f855a9e1640 1 --2- 172.21.15.43:0/2723783545 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f855415a510 0x7f8554160fe0 secure :-1 s=READY pgs=2262 cs=0 l=1 rev1=1 crypto rx=0x7f85540d1c30 tx=0x7f854402b900 comp rx=0 tx=0).stop 2023-08-09T15:08:58.466 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:58.462+0000 7f855a9e1640 1 -- 172.21.15.43:0/2723783545 shutdown_connections 2023-08-09T15:08:58.466 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:58.462+0000 7f855a9e1640 1 --2- 172.21.15.43:0/2723783545 >> v2:172.21.15.203:3300/0 conn(0x7f855415da60 0x7f85540b4400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T15:08:58.466 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:58.462+0000 7f855a9e1640 1 --2- 172.21.15.43:0/2723783545 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f855415d5e0 0x7f85540b3ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T15:08:58.466 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:58.462+0000 7f855a9e1640 1 --2- 172.21.15.43:0/2723783545 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f855415a510 0x7f8554160fe0 unknown :-1 s=CLOSED pgs=2262 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T15:08:58.466 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:58.462+0000 7f855a9e1640 1 -- 172.21.15.43:0/2723783545 >> 172.21.15.43:0/2723783545 conn(0x7f85540ceec0 msgr2=0x7f85540cf2d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-09T15:08:58.467 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:58.463+0000 7f855a9e1640 1 -- 172.21.15.43:0/2723783545 shutdown_connections 2023-08-09T15:08:58.467 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:58.463+0000 7f855a9e1640 1 -- 172.21.15.43:0/2723783545 wait complete. 2023-08-09T15:08:58.470 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:58.465+0000 7f855a9e1640 1 Processor -- start 2023-08-09T15:08:58.471 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:58.465+0000 7f855a9e1640 1 -- start start 2023-08-09T15:08:58.471 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:58.466+0000 7f855a9e1640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f855415a510 0x7f85540c7d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T15:08:58.471 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:58.466+0000 7f855a9e1640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f855415d5e0 0x7f85540c6960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T15:08:58.472 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:58.466+0000 7f855a9e1640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f855415da60 0x7f85540c6ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T15:08:58.472 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:58.466+0000 7f855a9e1640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8554148800 con 0x7f855415a510 2023-08-09T15:08:58.472 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:58.466+0000 7f855a9e1640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8554148a70 con 0x7f855415da60 2023-08-09T15:08:58.473 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:58.466+0000 7f855a9e1640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8554148d50 con 0x7f855415d5e0 2023-08-09T15:08:58.473 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:58.466+0000 7f8553fff640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f855415a510 0x7f85540c7d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T15:08:58.473 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:58.467+0000 7f8553fff640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f855415a510 0x7f85540c7d30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:41300/0 (socket says 172.21.15.43:41300) 2023-08-09T15:08:58.473 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:58.467+0000 7f8553fff640 1 -- 172.21.15.43:0/3431773671 learned_addr learned my addr 172.21.15.43:0/3431773671 (peer_addr_for_me v2:172.21.15.43:0/0) 2023-08-09T15:08:58.474 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:58.467+0000 7f85537fe640 1 --2- 172.21.15.43:0/3431773671 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f855415d5e0 0x7f85540c6960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T15:08:58.474 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:58.467+0000 7f8558f7e640 1 --2- 172.21.15.43:0/3431773671 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f855415da60 0x7f85540c6ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T15:08:58.474 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:58.467+0000 7f8553fff640 1 -- 172.21.15.43:0/3431773671 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f855415d5e0 msgr2=0x7f85540c6960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T15:08:58.474 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:58.467+0000 7f8553fff640 1 --2- 172.21.15.43:0/3431773671 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f855415d5e0 0x7f85540c6960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T15:08:58.475 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:58.467+0000 7f8553fff640 1 -- 172.21.15.43:0/3431773671 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f855415da60 msgr2=0x7f85540c6ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T15:08:58.475 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:58.467+0000 7f8553fff640 1 --2- 172.21.15.43:0/3431773671 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f855415da60 0x7f85540c6ea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T15:08:58.475 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:58.467+0000 7f8553fff640 1 -- 172.21.15.43:0/3431773671 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f85440094d0 con 0x7f855415a510 2023-08-09T15:08:58.475 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:58.467+0000 7f8558f7e640 1 --2- 172.21.15.43:0/3431773671 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f855415da60 0x7f85540c6ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-08-09T15:08:58.476 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:58.467+0000 7f8553fff640 1 --2- 172.21.15.43:0/3431773671 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f855415a510 0x7f85540c7d30 secure :-1 s=READY pgs=2263 cs=0 l=1 rev1=1 crypto rx=0x7f854402b940 tx=0x7f85440046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-09T15:08:58.476 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:58.468+0000 7f8550ff9640 1 -- 172.21.15.43:0/3431773671 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f854403ad30 con 0x7f855415a510 2023-08-09T15:08:58.476 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:58.468+0000 7f8550ff9640 1 -- 172.21.15.43:0/3431773671 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f85440443f0 con 0x7f855415a510 2023-08-09T15:08:58.477 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:58.468+0000 7f855a9e1640 1 -- 172.21.15.43:0/3431773671 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8554149030 con 0x7f855415a510 2023-08-09T15:08:58.477 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:58.468+0000 7f8550ff9640 1 -- 172.21.15.43:0/3431773671 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f85440447a0 con 0x7f855415a510 2023-08-09T15:08:58.477 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:58.468+0000 7f855a9e1640 1 -- 172.21.15.43:0/3431773671 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f85540cc970 con 0x7f855415a510 2023-08-09T15:08:58.477 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:58.469+0000 7f855a9e1640 1 -- 172.21.15.43:0/3431773671 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8554149270 con 0x7f855415a510 2023-08-09T15:08:58.478 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:58.470+0000 7f8550ff9640 1 -- 172.21.15.43:0/3431773671 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 107) v1 ==== 88460+0+0 (secure 0 0 0) 0x7f854404d450 con 0x7f855415a510 2023-08-09T15:08:58.479 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:58.471+0000 7f8550ff9640 1 --2- 172.21.15.43:0/3431773671 >> [v2:172.21.15.203:6832/3082732769,v1:172.21.15.203:6833/3082732769] conn(0x7f854006a1c0 0x7f854006c680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T15:08:58.479 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:58.472+0000 7f8550ff9640 1 -- 172.21.15.43:0/3431773671 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(205..205 src has 1..205) v4 ==== 6426+0+0 (secure 0 0 0) 0x7f85440ab100 con 0x7f855415a510 2023-08-09T15:08:58.480 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:58.472+0000 7f85537fe640 1 -- 172.21.15.43:0/3431773671 >> [v2:172.21.15.203:6832/3082732769,v1:172.21.15.203:6833/3082732769] conn(0x7f854006a1c0 msgr2=0x7f854006c680 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.203:6832/3082732769 2023-08-09T15:08:58.480 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:58.472+0000 7f85537fe640 1 --2- 172.21.15.43:0/3431773671 >> [v2:172.21.15.203:6832/3082732769,v1:172.21.15.203:6833/3082732769] conn(0x7f854006a1c0 0x7f854006c680 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-08-09T15:08:58.480 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:58.476+0000 7f8550ff9640 1 -- 172.21.15.43:0/3431773671 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178066 (secure 0 0 0) 0x7f8554149270 con 0x7f855415a510 2023-08-09T15:08:58.501 INFO:tasks.ceph.mgr.y.smithi043.stderr:2023-08-09T15:08:58.501+0000 7ff5386d4040 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2023-08-09T15:08:58.676 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:58.673+0000 7f85537fe640 1 -- 172.21.15.43:0/3431773671 >> [v2:172.21.15.203:6832/3082732769,v1:172.21.15.203:6833/3082732769] conn(0x7f854006a1c0 msgr2=0x7f854006c680 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.203:6832/3082732769 2023-08-09T15:08:58.677 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:58.673+0000 7f85537fe640 1 --2- 172.21.15.43:0/3431773671 >> [v2:172.21.15.203:6832/3082732769,v1:172.21.15.203:6833/3082732769] conn(0x7f854006a1c0 0x7f854006c680 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2023-08-09T15:08:58.688 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:58.688+0000 7f855a9e1640 1 -- 172.21.15.43:0/3431773671 --> [v2:172.21.15.203:6832/3082732769,v1:172.21.15.203:6833/3082732769] -- mgr_command(tid 0: {"prefix": "fs snap-schedule add", "fs": "cephfs", "path": "snap_test_dir1", "snap_schedule": "1M", "target": ["mon-mgr", ""]}) v1 -- 0x7f8554148be0 con 0x7f854006a1c0 2023-08-09T15:08:59.075 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:59.074+0000 7f85537fe640 1 -- 172.21.15.43:0/3431773671 >> [v2:172.21.15.203:6832/3082732769,v1:172.21.15.203:6833/3082732769] conn(0x7f854006a1c0 msgr2=0x7f854006c680 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.203:6832/3082732769 2023-08-09T15:08:59.075 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:59.074+0000 7f85537fe640 1 --2- 172.21.15.43:0/3431773671 >> [v2:172.21.15.203:6832/3082732769,v1:172.21.15.203:6833/3082732769] conn(0x7f854006a1c0 0x7f854006c680 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2023-08-09T15:08:59.099 INFO:tasks.ceph.mgr.x.smithi203.stderr:2023-08-09T15:08:59.098+0000 7fdd22e2b040 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2023-08-09T15:08:59.166 INFO:tasks.ceph.mgr.x.smithi203.stderr:2023-08-09T15:08:59.165+0000 7fdd22e2b040 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2023-08-09T15:08:59.220 INFO:tasks.ceph.mgr.y.smithi043.stderr:2023-08-09T15:08:59.220+0000 7ff5386d4040 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2023-08-09T15:08:59.221 INFO:tasks.ceph.mgr.x.smithi203.stderr:2023-08-09T15:08:59.219+0000 7fdd22e2b040 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2023-08-09T15:08:59.284 INFO:tasks.ceph.mgr.y.smithi043.stderr:2023-08-09T15:08:59.283+0000 7ff5386d4040 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2023-08-09T15:08:59.335 INFO:tasks.ceph.mgr.y.smithi043.stderr:2023-08-09T15:08:59.335+0000 7ff5386d4040 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2023-08-09T15:08:59.344 INFO:tasks.ceph.mgr.x.smithi203.stderr:2023-08-09T15:08:59.342+0000 7fdd22e2b040 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2023-08-09T15:08:59.458 INFO:tasks.ceph.mgr.y.smithi043.stderr:2023-08-09T15:08:59.458+0000 7ff5386d4040 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2023-08-09T15:08:59.500 INFO:tasks.ceph.mgr.x.smithi203.stderr:2023-08-09T15:08:59.498+0000 7fdd22e2b040 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2023-08-09T15:08:59.606 INFO:tasks.ceph.mgr.x.smithi203.stderr:2023-08-09T15:08:59.605+0000 7fdd22e2b040 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2023-08-09T15:08:59.622 INFO:tasks.ceph.mgr.y.smithi043.stderr:2023-08-09T15:08:59.621+0000 7ff5386d4040 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2023-08-09T15:08:59.654 INFO:tasks.ceph.mgr.x.smithi203.stderr:2023-08-09T15:08:59.653+0000 7fdd22e2b040 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2023-08-09T15:08:59.734 INFO:tasks.ceph.mgr.y.smithi043.stderr:2023-08-09T15:08:59.734+0000 7ff5386d4040 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2023-08-09T15:08:59.786 INFO:tasks.ceph.mgr.y.smithi043.stderr:2023-08-09T15:08:59.785+0000 7ff5386d4040 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2023-08-09T15:08:59.812 INFO:tasks.ceph.mgr.x.smithi203.stderr:2023-08-09T15:08:59.811+0000 7fdd22e2b040 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2023-08-09T15:08:59.876 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:59.875+0000 7f85537fe640 1 -- 172.21.15.43:0/3431773671 >> [v2:172.21.15.203:6832/3082732769,v1:172.21.15.203:6833/3082732769] conn(0x7f854006a1c0 msgr2=0x7f854006c680 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.203:6832/3082732769 2023-08-09T15:08:59.876 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:08:59.875+0000 7f85537fe640 1 --2- 172.21.15.43:0/3431773671 >> [v2:172.21.15.203:6832/3082732769,v1:172.21.15.203:6833/3082732769] conn(0x7f854006a1c0 0x7f854006c680 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2023-08-09T15:08:59.953 INFO:tasks.ceph.mgr.y.smithi043.stderr:2023-08-09T15:08:59.952+0000 7ff5386d4040 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2023-08-09T15:09:00.189 INFO:tasks.ceph.mgr.x.smithi203.stderr:2023-08-09T15:09:00.188+0000 7fdd22e2b040 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2023-08-09T15:09:00.357 INFO:tasks.ceph.mgr.y.smithi043.stderr:2023-08-09T15:09:00.356+0000 7ff5386d4040 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2023-08-09T15:09:00.409 INFO:tasks.ceph.mgr.x.smithi203.stderr:2023-08-09T15:09:00.408+0000 7fdd22e2b040 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2023-08-09T15:09:00.465 INFO:tasks.ceph.mgr.x.smithi203.stderr:2023-08-09T15:09:00.464+0000 7fdd22e2b040 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2023-08-09T15:09:00.515 INFO:tasks.ceph.mgr.x.smithi203.stderr:2023-08-09T15:09:00.514+0000 7fdd22e2b040 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2023-08-09T15:09:00.566 INFO:tasks.ceph.mgr.x.smithi203.stderr:2023-08-09T15:09:00.565+0000 7fdd22e2b040 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2023-08-09T15:09:00.587 INFO:tasks.ceph.mgr.y.smithi043.stderr:2023-08-09T15:09:00.587+0000 7ff5386d4040 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2023-08-09T15:09:00.647 INFO:tasks.ceph.mgr.y.smithi043.stderr:2023-08-09T15:09:00.646+0000 7ff5386d4040 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2023-08-09T15:09:00.699 INFO:tasks.ceph.mgr.y.smithi043.stderr:2023-08-09T15:09:00.698+0000 7ff5386d4040 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2023-08-09T15:09:00.754 INFO:tasks.ceph.mgr.y.smithi043.stderr:2023-08-09T15:09:00.753+0000 7ff5386d4040 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2023-08-09T15:09:00.858 INFO:tasks.ceph.mgr.x.smithi203.stderr:2023-08-09T15:09:00.857+0000 7fdd22e2b040 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2023-08-09T15:09:00.915 INFO:tasks.ceph.mgr.x.smithi203.stderr:2023-08-09T15:09:00.914+0000 7fdd22e2b040 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2023-08-09T15:09:00.966 INFO:tasks.ceph.mgr.x.smithi203.stderr:2023-08-09T15:09:00.965+0000 7fdd22e2b040 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2023-08-09T15:09:01.063 INFO:tasks.ceph.mgr.y.smithi043.stderr:2023-08-09T15:09:01.063+0000 7ff5386d4040 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2023-08-09T15:09:01.126 INFO:tasks.ceph.mgr.x.smithi203.stderr:2023-08-09T15:09:01.125+0000 7fdd22e2b040 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2023-08-09T15:09:01.126 INFO:tasks.ceph.mgr.y.smithi043.stderr:2023-08-09T15:09:01.126+0000 7ff5386d4040 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2023-08-09T15:09:01.175 INFO:tasks.ceph.mgr.x.smithi203.stderr:2023-08-09T15:09:01.174+0000 7fdd22e2b040 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2023-08-09T15:09:01.180 INFO:tasks.ceph.mgr.y.smithi043.stderr:2023-08-09T15:09:01.180+0000 7ff5386d4040 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2023-08-09T15:09:01.183 DEBUG:teuthology.orchestra.run.smithi043:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-08-09T15:09:01.190 DEBUG:teuthology.orchestra.run.smithi203:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-08-09T15:09:01.229 INFO:tasks.ceph.mgr.x.smithi203.stderr:2023-08-09T15:09:01.228+0000 7fdd22e2b040 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2023-08-09T15:09:01.283 INFO:tasks.ceph.mgr.x.smithi203.stderr:2023-08-09T15:09:01.281+0000 7fdd22e2b040 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2023-08-09T15:09:01.300 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:09:01.297+0000 7f8550ff9640 1 -- 172.21.15.43:0/3431773671 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mgrmap(e 108) v1 ==== 88069+0+0 (secure 0 0 0) 0x7f8544076680 con 0x7f855415a510 2023-08-09T15:09:01.301 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:09:01.298+0000 7f8550ff9640 1 -- 172.21.15.43:0/3431773671 >> [v2:172.21.15.203:6832/3082732769,v1:172.21.15.203:6833/3082732769] conn(0x7f854006a1c0 msgr2=0x7f854006c680 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-08-09T15:09:01.301 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:09:01.298+0000 7f8550ff9640 1 --2- 172.21.15.43:0/3431773671 >> [v2:172.21.15.203:6832/3082732769,v1:172.21.15.203:6833/3082732769] conn(0x7f854006a1c0 0x7f854006c680 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T15:09:01.365 INFO:tasks.ceph.mgr.y.smithi043.stderr:2023-08-09T15:09:01.365+0000 7ff5386d4040 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2023-08-09T15:09:01.423 INFO:tasks.ceph.mgr.y.smithi043.stderr:2023-08-09T15:09:01.423+0000 7ff5386d4040 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2023-08-09T15:09:01.481 INFO:tasks.ceph.mgr.x.smithi203.stderr:2023-08-09T15:09:01.479+0000 7fdca1929640 -1 client.0 error registering admin socket command: (17) File exists 2023-08-09T15:09:01.481 INFO:tasks.ceph.mgr.x.smithi203.stderr:2023-08-09T15:09:01.479+0000 7fdca1929640 -1 client.0 error registering admin socket command: (17) File exists 2023-08-09T15:09:01.481 INFO:tasks.ceph.mgr.x.smithi203.stderr:2023-08-09T15:09:01.479+0000 7fdca1929640 -1 client.0 error registering admin socket command: (17) File exists 2023-08-09T15:09:01.481 INFO:tasks.ceph.mgr.x.smithi203.stderr:2023-08-09T15:09:01.479+0000 7fdca1929640 -1 client.0 error registering admin socket command: (17) File exists 2023-08-09T15:09:01.481 INFO:tasks.ceph.mgr.x.smithi203.stderr:2023-08-09T15:09:01.479+0000 7fdca1929640 -1 client.0 error registering admin socket command: (17) File exists 2023-08-09T15:09:01.481 INFO:tasks.ceph.mgr.x.smithi203.stderr:2023-08-09T15:09:01.479+0000 7fdc9f124640 -1 client.0 error registering admin socket command: (17) File exists 2023-08-09T15:09:01.482 INFO:tasks.ceph.mgr.x.smithi203.stderr:2023-08-09T15:09:01.479+0000 7fdc9f124640 -1 client.0 error registering admin socket command: (17) File exists 2023-08-09T15:09:01.482 INFO:tasks.ceph.mgr.x.smithi203.stderr:2023-08-09T15:09:01.479+0000 7fdc9f124640 -1 client.0 error registering admin socket command: (17) File exists 2023-08-09T15:09:01.482 INFO:tasks.ceph.mgr.x.smithi203.stderr:2023-08-09T15:09:01.479+0000 7fdc9f124640 -1 client.0 error registering admin socket command: (17) File exists 2023-08-09T15:09:01.482 INFO:tasks.ceph.mgr.x.smithi203.stderr:2023-08-09T15:09:01.479+0000 7fdc9f124640 -1 client.0 error registering admin socket command: (17) File exists 2023-08-09T15:09:01.486 INFO:tasks.ceph.mgr.y.smithi043.stderr:2023-08-09T15:09:01.486+0000 7ff5386d4040 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2023-08-09T15:09:01.540 INFO:tasks.ceph.mgr.y.smithi043.stderr:2023-08-09T15:09:01.540+0000 7ff5386d4040 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2023-08-09T15:09:02.311 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:09:02.311+0000 7f8550ff9640 1 -- 172.21.15.43:0/3431773671 <== mon.0 v2:172.21.15.43:3300/0 8 ==== mgrmap(e 109) v1 ==== 88196+0+0 (secure 0 0 0) 0x7f85440a92b0 con 0x7f855415a510 2023-08-09T15:09:02.312 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:09:02.312+0000 7f8550ff9640 1 --2- 172.21.15.43:0/3431773671 >> [v2:172.21.15.203:6832/4012896436,v1:172.21.15.203:6833/4012896436] conn(0x7f8540072c30 0x7f8540075020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T15:09:02.312 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:09:02.312+0000 7f8550ff9640 1 -- 172.21.15.43:0/3431773671 --> [v2:172.21.15.203:6832/4012896436,v1:172.21.15.203:6833/4012896436] -- mgr_command(tid 0: {"prefix": "fs snap-schedule add", "fs": "cephfs", "path": "snap_test_dir1", "snap_schedule": "1M", "target": ["mon-mgr", ""]}) v1 -- 0x7f8554148be0 con 0x7f8540072c30 2023-08-09T15:09:02.314 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:09:02.314+0000 7f85537fe640 1 --2- 172.21.15.43:0/3431773671 >> [v2:172.21.15.203:6832/4012896436,v1:172.21.15.203:6833/4012896436] conn(0x7f8540072c30 0x7f8540075020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T15:09:02.333 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:09:02.333+0000 7f85537fe640 1 --2- 172.21.15.43:0/3431773671 >> [v2:172.21.15.203:6832/4012896436,v1:172.21.15.203:6833/4012896436] conn(0x7f8540072c30 0x7f8540075020 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7f85540aa3f0 tx=0x7f8548008040 comp rx=0 tx=0).ready entity=mgr.7763 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-09T15:09:02.491 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:09:02.491+0000 7f8550ff9640 1 -- 172.21.15.43:0/3431773671 <== mgr.7763 v2:172.21.15.203:6832/4012896436 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+37 (secure 0 0 0) 0x7f8554148be0 con 0x7f8540072c30 2023-08-09T15:09:02.492 INFO:teuthology.orchestra.run.smithi043.stdout:Schedule set for path snap_test_dir1 2023-08-09T15:09:02.495 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:09:02.494+0000 7f855a9e1640 1 -- 172.21.15.43:0/3431773671 >> [v2:172.21.15.203:6832/4012896436,v1:172.21.15.203:6833/4012896436] conn(0x7f8540072c30 msgr2=0x7f8540075020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T15:09:02.495 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:09:02.494+0000 7f855a9e1640 1 --2- 172.21.15.43:0/3431773671 >> [v2:172.21.15.203:6832/4012896436,v1:172.21.15.203:6833/4012896436] conn(0x7f8540072c30 0x7f8540075020 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7f85540aa3f0 tx=0x7f8548008040 comp rx=0 tx=0).stop 2023-08-09T15:09:02.496 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:09:02.494+0000 7f855a9e1640 1 -- 172.21.15.43:0/3431773671 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f855415a510 msgr2=0x7f85540c7d30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T15:09:02.496 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:09:02.494+0000 7f855a9e1640 1 --2- 172.21.15.43:0/3431773671 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f855415a510 0x7f85540c7d30 secure :-1 s=READY pgs=2263 cs=0 l=1 rev1=1 crypto rx=0x7f854402b940 tx=0x7f85440046c0 comp rx=0 tx=0).stop 2023-08-09T15:09:02.496 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:09:02.494+0000 7f8558f7e640 1 -- 172.21.15.43:0/3431773671 reap_dead start 2023-08-09T15:09:02.496 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:09:02.495+0000 7f855a9e1640 1 -- 172.21.15.43:0/3431773671 shutdown_connections 2023-08-09T15:09:02.496 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:09:02.495+0000 7f855a9e1640 1 -- 172.21.15.43:0/3431773671 >> 172.21.15.43:0/3431773671 conn(0x7f85540ceec0 msgr2=0x7f855415ac30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-09T15:09:02.496 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:09:02.495+0000 7f855a9e1640 1 -- 172.21.15.43:0/3431773671 shutdown_connections 2023-08-09T15:09:02.497 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:09:02.495+0000 7f855a9e1640 1 -- 172.21.15.43:0/3431773671 wait complete. 2023-08-09T15:09:02.517 DEBUG:tasks.cephfs.test_snap_schedules:res=Schedule set for path snap_test_dir1 2023-08-09T15:09:02.518 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs snap-schedule retention add --fs cephfs snap_test_dir1 1M 2023-08-09T15:09:02.762 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:09:02.762+0000 7f2d7f6bf640 1 Processor -- start 2023-08-09T15:09:02.773 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:09:02.762+0000 7f2d7f6bf640 1 -- start start 2023-08-09T15:09:02.776 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:09:02.764+0000 7f2d7f6bf640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f2d7815df60 0x7f2d780ce1c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T15:09:02.777 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:09:02.764+0000 7f2d7f6bf640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f2d7815e3a0 0x7f2d780ce700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T15:09:02.777 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:09:02.764+0000 7f2d7f6bf640 1 -- --> v1:172.21.15.43:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2d7815aef0 con 0x7f2d780cb2d0 2023-08-09T15:09:02.777 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:09:02.764+0000 7f2d7f6bf640 1 -- --> v2:172.21.15.43:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2d7815a970 con 0x7f2d7815e3a0 2023-08-09T15:09:02.777 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:09:02.764+0000 7f2d7f6bf640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2d7815e940 con 0x7f2d7815df60 2023-08-09T15:09:02.777 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:09:02.764+0000 7f2d7d45b640 1 --1- >> v1:172.21.15.43:6789/0 conn(0x7f2d780cb2d0 0x7f2d7815b380 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.43:6789/0 says I am v1:172.21.15.43:49714/0 (socket says 172.21.15.43:49714) 2023-08-09T15:09:02.777 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:09:02.764+0000 7f2d7d45b640 1 -- 172.21.15.43:0/4279971974 learned_addr learned my addr 172.21.15.43:0/4279971974 (peer_addr_for_me v1:172.21.15.43:0/0) 2023-08-09T15:09:02.778 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:09:02.764+0000 7f2d7dc5c640 1 --2- 172.21.15.43:0/4279971974 >> v2:172.21.15.43:3300/0 conn(0x7f2d7815e3a0 0x7f2d780ce700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T15:09:02.778 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:09:02.765+0000 7f2d7cc5a640 1 --2- 172.21.15.43:0/4279971974 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f2d7815df60 0x7f2d780ce1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T15:09:02.778 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:09:02.765+0000 7f2d6ffff640 1 -- 172.21.15.43:0/4279971974 <== mon.0 v1:172.21.15.43:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2239323938 0 0) 0x7f2d7815aef0 con 0x7f2d780cb2d0 2023-08-09T15:09:02.778 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:09:02.765+0000 7f2d7dc5c640 1 -- 172.21.15.43:0/4279971974 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f2d7815df60 msgr2=0x7f2d780ce1c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-09T15:09:02.778 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:09:02.765+0000 7f2d7dc5c640 1 --2- 172.21.15.43:0/4279971974 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f2d7815df60 0x7f2d780ce1c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T15:09:02.778 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:09:02.765+0000 7f2d7dc5c640 1 -- 172.21.15.43:0/4279971974 >> v1:172.21.15.43:6789/0 conn(0x7f2d780cb2d0 legacy=0x7f2d7815b380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T15:09:02.778 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:09:02.765+0000 7f2d7dc5c640 1 -- 172.21.15.43:0/4279971974 --> v2:172.21.15.43:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2d7815ec20 con 0x7f2d7815e3a0 2023-08-09T15:09:02.778 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:09:02.766+0000 7f2d7dc5c640 1 --2- 172.21.15.43:0/4279971974 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f2d7815e3a0 0x7f2d780ce700 secure :-1 s=READY pgs=2265 cs=0 l=1 rev1=1 crypto rx=0x7f2d70009d90 tx=0x7f2d7002beb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=80b99546ad0a8976 server_cookie=0 in_seq=0 out_seq=0 2023-08-09T15:09:02.779 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:09:02.766+0000 7f2d6ffff640 1 -- 172.21.15.43:0/4279971974 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2d7002cc60 con 0x7f2d7815e3a0 2023-08-09T15:09:02.779 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:09:02.766+0000 7f2d6ffff640 1 -- 172.21.15.43:0/4279971974 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2d7002cdc0 con 0x7f2d7815e3a0 2023-08-09T15:09:02.779 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:09:02.766+0000 7f2d6ffff640 1 -- 172.21.15.43:0/4279971974 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2d70034660 con 0x7f2d7815e3a0 2023-08-09T15:09:02.779 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:09:02.767+0000 7f2d7f6bf640 1 -- 172.21.15.43:0/4279971974 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f2d7815e3a0 msgr2=0x7f2d780ce700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T15:09:02.779 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:09:02.767+0000 7f2d7f6bf640 1 --2- 172.21.15.43:0/4279971974 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f2d7815e3a0 0x7f2d780ce700 secure :-1 s=READY pgs=2265 cs=0 l=1 rev1=1 crypto rx=0x7f2d70009d90 tx=0x7f2d7002beb0 comp rx=0 tx=0).stop 2023-08-09T15:09:02.779 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:09:02.767+0000 7f2d7f6bf640 1 -- 172.21.15.43:0/4279971974 shutdown_connections 2023-08-09T15:09:02.779 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:09:02.767+0000 7f2d7f6bf640 1 --2- 172.21.15.43:0/4279971974 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f2d7815e3a0 0x7f2d780ce700 unknown :-1 s=CLOSED pgs=2265 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T15:09:02.780 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:09:02.767+0000 7f2d7f6bf640 1 --2- 172.21.15.43:0/4279971974 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f2d7815df60 0x7f2d780ce1c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T15:09:02.780 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:09:02.767+0000 7f2d7f6bf640 1 -- 172.21.15.43:0/4279971974 >> 172.21.15.43:0/4279971974 conn(0x7f2d780aff50 msgr2=0x7f2d780b0360 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-09T15:09:02.780 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:09:02.767+0000 7f2d7f6bf640 1 -- 172.21.15.43:0/4279971974 shutdown_connections 2023-08-09T15:09:02.780 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:09:02.767+0000 7f2d7f6bf640 1 -- 172.21.15.43:0/4279971974 wait complete. 2023-08-09T15:09:02.780 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:09:02.769+0000 7f2d7f6bf640 1 Processor -- start 2023-08-09T15:09:02.780 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:09:02.769+0000 7f2d7f6bf640 1 -- start start 2023-08-09T15:09:02.780 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:09:02.770+0000 7f2d7f6bf640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f2d780cb2d0 0x7f2d780c4330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T15:09:02.780 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:09:02.770+0000 7f2d7f6bf640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f2d7815df60 0x7f2d780b3380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T15:09:02.781 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:09:02.770+0000 7f2d7f6bf640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f2d7815e3a0 0x7f2d780b38c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T15:09:02.781 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:09:02.770+0000 7f2d7f6bf640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2d781459d0 con 0x7f2d7815df60 2023-08-09T15:09:02.781 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:09:02.770+0000 7f2d7f6bf640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2d78144d50 con 0x7f2d780cb2d0 2023-08-09T15:09:02.781 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:09:02.770+0000 7f2d7f6bf640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2d78145030 con 0x7f2d7815e3a0 2023-08-09T15:09:02.781 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:09:02.770+0000 7f2d7cc5a640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f2d7815df60 0x7f2d780b3380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T15:09:02.781 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:09:02.770+0000 7f2d7cc5a640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f2d7815df60 0x7f2d780b3380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:41328/0 (socket says 172.21.15.43:41328) 2023-08-09T15:09:02.781 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:09:02.770+0000 7f2d7cc5a640 1 -- 172.21.15.43:0/237823337 learned_addr learned my addr 172.21.15.43:0/237823337 (peer_addr_for_me v2:172.21.15.43:0/0) 2023-08-09T15:09:02.781 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:09:02.771+0000 7f2d7d45b640 1 --2- 172.21.15.43:0/237823337 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f2d780cb2d0 0x7f2d780c4330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T15:09:02.782 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:09:02.771+0000 7f2d7dc5c640 1 --2- 172.21.15.43:0/237823337 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f2d7815e3a0 0x7f2d780b38c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T15:09:02.782 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:09:02.771+0000 7f2d7cc5a640 1 -- 172.21.15.43:0/237823337 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f2d7815e3a0 msgr2=0x7f2d780b38c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T15:09:02.782 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:09:02.771+0000 7f2d7cc5a640 1 --2- 172.21.15.43:0/237823337 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f2d7815e3a0 0x7f2d780b38c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T15:09:02.782 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:09:02.771+0000 7f2d7cc5a640 1 -- 172.21.15.43:0/237823337 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f2d780cb2d0 msgr2=0x7f2d780c4330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T15:09:02.782 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:09:02.771+0000 7f2d7cc5a640 1 --2- 172.21.15.43:0/237823337 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f2d780cb2d0 0x7f2d780c4330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T15:09:02.782 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:09:02.771+0000 7f2d7cc5a640 1 -- 172.21.15.43:0/237823337 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2d70009a70 con 0x7f2d7815df60 2023-08-09T15:09:02.782 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:09:02.771+0000 7f2d7dc5c640 1 --2- 172.21.15.43:0/237823337 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f2d7815e3a0 0x7f2d780b38c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-08-09T15:09:02.783 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:09:02.771+0000 7f2d7cc5a640 1 --2- 172.21.15.43:0/237823337 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f2d7815df60 0x7f2d780b3380 secure :-1 s=READY pgs=2266 cs=0 l=1 rev1=1 crypto rx=0x7f2d6800d800 tx=0x7f2d6800dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-09T15:09:02.783 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:09:02.772+0000 7f2d6dffb640 1 -- 172.21.15.43:0/237823337 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2d6800b6c0 con 0x7f2d7815df60 2023-08-09T15:09:02.783 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:09:02.772+0000 7f2d6dffb640 1 -- 172.21.15.43:0/237823337 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2d6800b820 con 0x7f2d7815df60 2023-08-09T15:09:02.783 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:09:02.772+0000 7f2d7f6bf640 1 -- 172.21.15.43:0/237823337 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2d78145310 con 0x7f2d7815df60 2023-08-09T15:09:02.783 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:09:02.772+0000 7f2d6dffb640 1 -- 172.21.15.43:0/237823337 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2d6800bbb0 con 0x7f2d7815df60 2023-08-09T15:09:02.783 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:09:02.772+0000 7f2d7f6bf640 1 -- 172.21.15.43:0/237823337 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2d780c4870 con 0x7f2d7815df60 2023-08-09T15:09:02.783 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:09:02.773+0000 7f2d7f6bf640 1 -- 172.21.15.43:0/237823337 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2d780739f0 con 0x7f2d7815df60 2023-08-09T15:09:02.783 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:09:02.774+0000 7f2d6dffb640 1 -- 172.21.15.43:0/237823337 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 109) v1 ==== 88196+0+0 (secure 0 0 0) 0x7f2d6800bd10 con 0x7f2d7815df60 2023-08-09T15:09:02.784 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:09:02.775+0000 7f2d6dffb640 1 --2- 172.21.15.43:0/237823337 >> [v2:172.21.15.203:6832/4012896436,v1:172.21.15.203:6833/4012896436] conn(0x7f2d5c069f20 0x7f2d5c06c3e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T15:09:02.784 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:09:02.775+0000 7f2d6dffb640 1 -- 172.21.15.43:0/237823337 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(206..206 src has 1..206) v4 ==== 6770+0+0 (secure 0 0 0) 0x7f2d68088a10 con 0x7f2d7815df60 2023-08-09T15:09:02.784 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:09:02.776+0000 7f2d7d45b640 1 --2- 172.21.15.43:0/237823337 >> [v2:172.21.15.203:6832/4012896436,v1:172.21.15.203:6833/4012896436] conn(0x7f2d5c069f20 0x7f2d5c06c3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T15:09:02.784 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:09:02.777+0000 7f2d7d45b640 1 --2- 172.21.15.43:0/237823337 >> [v2:172.21.15.203:6832/4012896436,v1:172.21.15.203:6833/4012896436] conn(0x7f2d5c069f20 0x7f2d5c06c3e0 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7f2d74000f60 tx=0x7f2d74007b70 comp rx=0 tx=0).ready entity=mgr.7763 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-09T15:09:02.785 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:09:02.780+0000 7f2d6dffb640 1 -- 172.21.15.43:0/237823337 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178066 (secure 0 0 0) 0x7f2d68057c00 con 0x7f2d7815df60 2023-08-09T15:09:02.997 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:09:02.997+0000 7f2d7f6bf640 1 -- 172.21.15.43:0/237823337 --> [v2:172.21.15.203:6832/4012896436,v1:172.21.15.203:6833/4012896436] -- mgr_command(tid 0: {"prefix": "fs snap-schedule retention add", "fs": "cephfs", "path": "snap_test_dir1", "retention_spec_or_period": "1M", "target": ["mon-mgr", ""]}) v1 -- 0x7f2d7815ab80 con 0x7f2d5c069f20 2023-08-09T15:09:03.141 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:09:03.141+0000 7f2d6dffb640 1 -- 172.21.15.43:0/237823337 <== mgr.7763 v2:172.21.15.203:6832/4012896436 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+39 (secure 0 0 0) 0x7f2d740071f0 con 0x7f2d5c069f20 2023-08-09T15:09:03.142 INFO:teuthology.orchestra.run.smithi043.stdout:Retention added to path snap_test_dir1 2023-08-09T15:09:03.146 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:09:03.146+0000 7f2d7f6bf640 1 -- 172.21.15.43:0/237823337 >> [v2:172.21.15.203:6832/4012896436,v1:172.21.15.203:6833/4012896436] conn(0x7f2d5c069f20 msgr2=0x7f2d5c06c3e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T15:09:03.146 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:09:03.146+0000 7f2d7f6bf640 1 --2- 172.21.15.43:0/237823337 >> [v2:172.21.15.203:6832/4012896436,v1:172.21.15.203:6833/4012896436] conn(0x7f2d5c069f20 0x7f2d5c06c3e0 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7f2d74000f60 tx=0x7f2d74007b70 comp rx=0 tx=0).stop 2023-08-09T15:09:03.146 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:09:03.146+0000 7f2d7f6bf640 1 -- 172.21.15.43:0/237823337 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f2d7815df60 msgr2=0x7f2d780b3380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T15:09:03.147 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:09:03.146+0000 7f2d7f6bf640 1 --2- 172.21.15.43:0/237823337 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f2d7815df60 0x7f2d780b3380 secure :-1 s=READY pgs=2266 cs=0 l=1 rev1=1 crypto rx=0x7f2d6800d800 tx=0x7f2d6800dcc0 comp rx=0 tx=0).stop 2023-08-09T15:09:03.147 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:09:03.146+0000 7f2d7f6bf640 1 -- 172.21.15.43:0/237823337 shutdown_connections 2023-08-09T15:09:03.147 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:09:03.146+0000 7f2d7f6bf640 1 --2- 172.21.15.43:0/237823337 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f2d7815e3a0 0x7f2d780b38c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T15:09:03.147 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:09:03.146+0000 7f2d7f6bf640 1 --2- 172.21.15.43:0/237823337 >> [v2:172.21.15.203:6832/4012896436,v1:172.21.15.203:6833/4012896436] conn(0x7f2d5c069f20 0x7f2d5c06c3e0 unknown :-1 s=CLOSED pgs=23 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T15:09:03.147 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:09:03.146+0000 7f2d7f6bf640 1 --2- 172.21.15.43:0/237823337 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f2d7815df60 0x7f2d780b3380 unknown :-1 s=CLOSED pgs=2266 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T15:09:03.147 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:09:03.146+0000 7f2d7f6bf640 1 --2- 172.21.15.43:0/237823337 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f2d780cb2d0 0x7f2d780c4330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T15:09:03.147 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:09:03.146+0000 7f2d7f6bf640 1 -- 172.21.15.43:0/237823337 >> 172.21.15.43:0/237823337 conn(0x7f2d780aff50 msgr2=0x7f2d7815b380 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-09T15:09:03.147 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:09:03.146+0000 7f2d7f6bf640 1 -- 172.21.15.43:0/237823337 shutdown_connections 2023-08-09T15:09:03.148 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:09:03.146+0000 7f2d7f6bf640 1 -- 172.21.15.43:0/237823337 wait complete. 2023-08-09T15:09:03.169 DEBUG:tasks.cephfs.test_snap_schedules:res=Retention added to path snap_test_dir1 2023-08-09T15:09:03.170 DEBUG:tasks.cephfs.test_snap_schedules:expecting snap snap_test_dir1/.snap/scheduled-2023-08-09-15_10_00 in ~57s... 2023-08-09T15:09:03.170 DEBUG:tasks.cephfs.test_snap_schedules:expected_schedule: ['1M'], expected_retention: [{'M': 1}] 2023-08-09T15:09:03.170 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs snap-schedule list --fs cephfs --format json snap_test_dir1 2023-08-09T15:09:03.407 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:09:03.407+0000 7f8733342640 1 Processor -- start 2023-08-09T15:09:03.407 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:09:03.407+0000 7f8733342640 1 -- start start 2023-08-09T15:09:03.409 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:09:03.409+0000 7f8733342640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f872c0b8040 0x7f872c0d1d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T15:09:03.409 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:09:03.409+0000 7f8733342640 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7f872c0b3ee0 0x7f872c0b4370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T15:09:03.410 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:09:03.409+0000 7f8733342640 1 -- --> v1:172.21.15.43:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f872c15cb30 con 0x7f872c15c750 2023-08-09T15:09:03.410 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:09:03.409+0000 7f8733342640 1 -- --> v2:172.21.15.203:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f872c0b8460 con 0x7f872c0b3ee0 2023-08-09T15:09:03.410 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:09:03.409+0000 7f8733342640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f872c0b48b0 con 0x7f872c0b8040 2023-08-09T15:09:03.410 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:09:03.409+0000 7f87310de640 1 --1- >> v1:172.21.15.43:6789/0 conn(0x7f872c15c750 0x7f872c0d04c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.43:6789/0 says I am v1:172.21.15.43:49718/0 (socket says 172.21.15.43:49718) 2023-08-09T15:09:03.410 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:09:03.409+0000 7f87310de640 1 -- 172.21.15.43:0/2208875604 learned_addr learned my addr 172.21.15.43:0/2208875604 (peer_addr_for_me v1:172.21.15.43:0/0) 2023-08-09T15:09:03.411 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:09:03.409+0000 7f87318df640 1 --2- 172.21.15.43:0/2208875604 >> v2:172.21.15.203:3300/0 conn(0x7f872c0b3ee0 0x7f872c0b4370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T15:09:03.411 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:09:03.410+0000 7f87308dd640 1 --2- 172.21.15.43:0/2208875604 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f872c0b8040 0x7f872c0d1d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T15:09:03.411 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:09:03.410+0000 7f871ffff640 1 -- 172.21.15.43:0/2208875604 <== mon.0 v1:172.21.15.43:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1332478239 0 0) 0x7f8728001110 con 0x7f872c15c750 2023-08-09T15:09:03.411 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:09:03.410+0000 7f871ffff640 1 -- 172.21.15.43:0/2208875604 --> v1:172.21.15.43:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f8718003520 con 0x7f872c15c750 2023-08-09T15:09:03.412 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:09:03.410+0000 7f87318df640 1 -- 172.21.15.43:0/2208875604 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f872c0b8040 msgr2=0x7f872c0d1d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-09T15:09:03.412 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:09:03.410+0000 7f87318df640 1 --2- 172.21.15.43:0/2208875604 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f872c0b8040 0x7f872c0d1d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T15:09:03.412 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:09:03.410+0000 7f87318df640 1 -- 172.21.15.43:0/2208875604 >> v1:172.21.15.43:6789/0 conn(0x7f872c15c750 legacy=0x7f872c0d04c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T15:09:03.412 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:09:03.410+0000 7f87318df640 1 -- 172.21.15.43:0/2208875604 --> v2:172.21.15.203:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f872c0d0cf0 con 0x7f872c0b3ee0 2023-08-09T15:09:03.413 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:09:03.411+0000 7f87318df640 1 --2- 172.21.15.43:0/2208875604 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f872c0b3ee0 0x7f872c0b4370 secure :-1 s=READY pgs=829 cs=0 l=1 rev1=1 crypto rx=0x7f8724009d90 tx=0x7f872402bf30 comp rx=0 tx=0).ready entity=mon.1 client_cookie=1b4098076e782646 server_cookie=0 in_seq=0 out_seq=0 2023-08-09T15:09:03.413 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:09:03.412+0000 7f871ffff640 1 -- 172.21.15.43:0/2208875604 <== mon.1 v2:172.21.15.203:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f872402cce0 con 0x7f872c0b3ee0 2023-08-09T15:09:03.413 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:09:03.412+0000 7f871ffff640 1 -- 172.21.15.43:0/2208875604 <== mon.1 v2:172.21.15.203:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f872402ce40 con 0x7f872c0b3ee0 2023-08-09T15:09:03.413 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:09:03.412+0000 7f871ffff640 1 -- 172.21.15.43:0/2208875604 <== mon.1 v2:172.21.15.203:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8724034660 con 0x7f872c0b3ee0 2023-08-09T15:09:03.413 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:09:03.412+0000 7f8733342640 1 -- 172.21.15.43:0/2208875604 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f872c0b3ee0 msgr2=0x7f872c0b4370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T15:09:03.414 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:09:03.412+0000 7f8733342640 1 --2- 172.21.15.43:0/2208875604 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f872c0b3ee0 0x7f872c0b4370 secure :-1 s=READY pgs=829 cs=0 l=1 rev1=1 crypto rx=0x7f8724009d90 tx=0x7f872402bf30 comp rx=0 tx=0).stop 2023-08-09T15:09:03.414 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:09:03.412+0000 7f8733342640 1 -- 172.21.15.43:0/2208875604 shutdown_connections 2023-08-09T15:09:03.414 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:09:03.412+0000 7f8733342640 1 --2- 172.21.15.43:0/2208875604 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f872c0b3ee0 0x7f872c0b4370 unknown :-1 s=CLOSED pgs=829 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T15:09:03.414 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:09:03.412+0000 7f8733342640 1 --2- 172.21.15.43:0/2208875604 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f872c0b8040 0x7f872c0d1d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T15:09:03.414 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:09:03.412+0000 7f8733342640 1 -- 172.21.15.43:0/2208875604 >> 172.21.15.43:0/2208875604 conn(0x7f872c0ceee0 msgr2=0x7f872c0cf2f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-09T15:09:03.414 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:09:03.412+0000 7f8733342640 1 -- 172.21.15.43:0/2208875604 shutdown_connections 2023-08-09T15:09:03.414 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:09:03.413+0000 7f8733342640 1 -- 172.21.15.43:0/2208875604 wait complete. 2023-08-09T15:09:03.415 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:09:03.414+0000 7f8733342640 1 Processor -- start 2023-08-09T15:09:03.415 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:09:03.414+0000 7f8733342640 1 -- start start 2023-08-09T15:09:03.421 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:09:03.415+0000 7f8733342640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f872c0b3ee0 0x7f872c15e740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T15:09:03.421 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:09:03.415+0000 7f8733342640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f872c0b8040 0x7f872c1f27e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T15:09:03.422 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:09:03.415+0000 7f8733342640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f872c15c750 0x7f872c0cb0b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T15:09:03.422 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:09:03.415+0000 7f8733342640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f872c14b380 con 0x7f872c0b8040 2023-08-09T15:09:03.422 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:09:03.415+0000 7f8733342640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f872c0c9530 con 0x7f872c15c750 2023-08-09T15:09:03.422 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:09:03.415+0000 7f8733342640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f872c1f2d20 con 0x7f872c0b3ee0 2023-08-09T15:09:03.422 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:09:03.416+0000 7f87308dd640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f872c0b8040 0x7f872c1f27e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T15:09:03.422 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:09:03.416+0000 7f87308dd640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f872c0b8040 0x7f872c1f27e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:41340/0 (socket says 172.21.15.43:41340) 2023-08-09T15:09:03.422 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:09:03.416+0000 7f87308dd640 1 -- 172.21.15.43:0/2442784312 learned_addr learned my addr 172.21.15.43:0/2442784312 (peer_addr_for_me v2:172.21.15.43:0/0) 2023-08-09T15:09:03.423 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:09:03.416+0000 7f87318df640 1 --2- 172.21.15.43:0/2442784312 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f872c15c750 0x7f872c0cb0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T15:09:03.423 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:09:03.416+0000 7f87310de640 1 --2- 172.21.15.43:0/2442784312 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f872c0b3ee0 0x7f872c15e740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T15:09:03.423 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:09:03.416+0000 7f87308dd640 1 -- 172.21.15.43:0/2442784312 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f872c0b3ee0 msgr2=0x7f872c15e740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T15:09:03.423 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:09:03.416+0000 7f87308dd640 1 --2- 172.21.15.43:0/2442784312 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f872c0b3ee0 0x7f872c15e740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T15:09:03.423 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:09:03.416+0000 7f87308dd640 1 -- 172.21.15.43:0/2442784312 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f872c15c750 msgr2=0x7f872c0cb0b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T15:09:03.423 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:09:03.416+0000 7f87308dd640 1 --2- 172.21.15.43:0/2442784312 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f872c15c750 0x7f872c0cb0b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T15:09:03.423 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:09:03.416+0000 7f87308dd640 1 -- 172.21.15.43:0/2442784312 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8724009a70 con 0x7f872c0b8040 2023-08-09T15:09:03.424 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:09:03.416+0000 7f87310de640 1 --2- 172.21.15.43:0/2442784312 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f872c0b3ee0 0x7f872c15e740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-08-09T15:09:03.424 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:09:03.417+0000 7f87308dd640 1 --2- 172.21.15.43:0/2442784312 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f872c0b8040 0x7f872c1f27e0 secure :-1 s=READY pgs=2269 cs=0 l=1 rev1=1 crypto rx=0x7f872000e9b0 tx=0x7f872000ee70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-09T15:09:03.424 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:09:03.417+0000 7f871dffb640 1 -- 172.21.15.43:0/2442784312 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8720009670 con 0x7f872c0b8040 2023-08-09T15:09:03.424 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:09:03.417+0000 7f871dffb640 1 -- 172.21.15.43:0/2442784312 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f872000c9a0 con 0x7f872c0b8040 2023-08-09T15:09:03.424 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:09:03.417+0000 7f871dffb640 1 -- 172.21.15.43:0/2442784312 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f872000cd50 con 0x7f872c0b8040 2023-08-09T15:09:03.424 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:09:03.417+0000 7f8733342640 1 -- 172.21.15.43:0/2442784312 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f872c0c9780 con 0x7f872c0b8040 2023-08-09T15:09:03.424 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:09:03.417+0000 7f8733342640 1 -- 172.21.15.43:0/2442784312 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f872c0c9d40 con 0x7f872c0b8040 2023-08-09T15:09:03.425 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:09:03.418+0000 7f8733342640 1 -- 172.21.15.43:0/2442784312 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f872c0739f0 con 0x7f872c0b8040 2023-08-09T15:09:03.425 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:09:03.419+0000 7f871dffb640 1 -- 172.21.15.43:0/2442784312 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 110) v1 ==== 88461+0+0 (secure 0 0 0) 0x7f8720013470 con 0x7f872c0b8040 2023-08-09T15:09:03.425 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:09:03.420+0000 7f871dffb640 1 --2- 172.21.15.43:0/2442784312 >> [v2:172.21.15.203:6832/4012896436,v1:172.21.15.203:6833/4012896436] conn(0x7f871006a1c0 0x7f871006c680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T15:09:03.425 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:09:03.420+0000 7f871dffb640 1 -- 172.21.15.43:0/2442784312 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(206..206 src has 1..206) v4 ==== 6770+0+0 (secure 0 0 0) 0x7f8720088c50 con 0x7f872c0b8040 2023-08-09T15:09:03.425 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:09:03.421+0000 7f87310de640 1 --2- 172.21.15.43:0/2442784312 >> [v2:172.21.15.203:6832/4012896436,v1:172.21.15.203:6833/4012896436] conn(0x7f871006a1c0 0x7f871006c680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T15:09:03.426 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:09:03.422+0000 7f87310de640 1 --2- 172.21.15.43:0/2442784312 >> [v2:172.21.15.203:6832/4012896436,v1:172.21.15.203:6833/4012896436] conn(0x7f871006a1c0 0x7f871006c680 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7f8728001270 tx=0x7f8728009040 comp rx=0 tx=0).ready entity=mgr.7763 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-09T15:09:03.426 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:09:03.425+0000 7f871dffb640 1 -- 172.21.15.43:0/2442784312 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178066 (secure 0 0 0) 0x7f8720053f70 con 0x7f872c0b8040 2023-08-09T15:09:03.655 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:09:03.655+0000 7f8733342640 1 -- 172.21.15.43:0/2442784312 --> [v2:172.21.15.203:6832/4012896436,v1:172.21.15.203:6833/4012896436] -- mgr_command(tid 0: {"prefix": "fs snap-schedule list", "fs": "cephfs", "path": "snap_test_dir1", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f872c0d0fd0 con 0x7f871006a1c0 2023-08-09T15:09:03.671 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:09:03.670+0000 7f871dffb640 1 -- 172.21.15.43:0/2442784312 <== mgr.7763 v2:172.21.15.203:6832/4012896436 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+72 (secure 0 0 0) 0x7f8728001500 con 0x7f871006a1c0 2023-08-09T15:09:03.671 INFO:teuthology.orchestra.run.smithi043.stdout: 2023-08-09T15:09:03.671 INFO:teuthology.orchestra.run.smithi043.stdout:{"path": "snap_test_dir1", "schedule": ["1M"], "retention": [{"M": 1}]} 2023-08-09T15:09:03.674 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:09:03.673+0000 7f8733342640 1 -- 172.21.15.43:0/2442784312 >> [v2:172.21.15.203:6832/4012896436,v1:172.21.15.203:6833/4012896436] conn(0x7f871006a1c0 msgr2=0x7f871006c680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T15:09:03.674 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:09:03.673+0000 7f8733342640 1 --2- 172.21.15.43:0/2442784312 >> [v2:172.21.15.203:6832/4012896436,v1:172.21.15.203:6833/4012896436] conn(0x7f871006a1c0 0x7f871006c680 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7f8728001270 tx=0x7f8728009040 comp rx=0 tx=0).stop 2023-08-09T15:09:03.674 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:09:03.673+0000 7f8733342640 1 -- 172.21.15.43:0/2442784312 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f872c0b8040 msgr2=0x7f872c1f27e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T15:09:03.674 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:09:03.673+0000 7f8733342640 1 --2- 172.21.15.43:0/2442784312 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f872c0b8040 0x7f872c1f27e0 secure :-1 s=READY pgs=2269 cs=0 l=1 rev1=1 crypto rx=0x7f872000e9b0 tx=0x7f872000ee70 comp rx=0 tx=0).stop 2023-08-09T15:09:03.675 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:09:03.673+0000 7f8733342640 1 -- 172.21.15.43:0/2442784312 shutdown_connections 2023-08-09T15:09:03.675 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:09:03.673+0000 7f8733342640 1 --2- 172.21.15.43:0/2442784312 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f872c15c750 0x7f872c0cb0b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T15:09:03.675 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:09:03.673+0000 7f8733342640 1 --2- 172.21.15.43:0/2442784312 >> [v2:172.21.15.203:6832/4012896436,v1:172.21.15.203:6833/4012896436] conn(0x7f871006a1c0 0x7f871006c680 unknown :-1 s=CLOSED pgs=25 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T15:09:03.675 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:09:03.673+0000 7f8733342640 1 --2- 172.21.15.43:0/2442784312 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f872c0b8040 0x7f872c1f27e0 unknown :-1 s=CLOSED pgs=2269 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T15:09:03.675 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:09:03.673+0000 7f8733342640 1 --2- 172.21.15.43:0/2442784312 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f872c0b3ee0 0x7f872c15e740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T15:09:03.675 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:09:03.673+0000 7f8733342640 1 -- 172.21.15.43:0/2442784312 >> 172.21.15.43:0/2442784312 conn(0x7f872c0ceee0 msgr2=0x7f872c157860 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-09T15:09:03.676 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:09:03.674+0000 7f8733342640 1 -- 172.21.15.43:0/2442784312 shutdown_connections 2023-08-09T15:09:03.676 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:09:03.674+0000 7f8733342640 1 -- 172.21.15.43:0/2442784312 wait complete. 2023-08-09T15:09:03.694 DEBUG:tasks.cephfs.test_snap_schedules:res= {"path": "snap_test_dir1", "schedule": ["1M"], "retention": [{"M": 1}]} 2023-08-09T15:09:03.694 DEBUG:tasks.cephfs.test_snap_schedules:json_res: {'path': 'snap_test_dir1', 'schedule': ['1M'], 'retention': [{'M': 1}]} 2023-08-09T15:09:03.695 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-09T15:09:03.695 DEBUG:teuthology.orchestra.run.smithi043:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-08-09T15:09:03.762 INFO:tasks.cephfs.test_snap_schedules:snapshots: set() 2023-08-09T15:09:03.763 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-08-09T15:09:03.763 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-08-09T15:09:04.764 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-09T15:09:04.764 DEBUG:teuthology.orchestra.run.smithi043:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-08-09T15:09:04.841 INFO:tasks.cephfs.test_snap_schedules:snapshots: set() 2023-08-09T15:09:04.842 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-08-09T15:09:04.842 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-08-09T15:09:05.843 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-09T15:09:05.843 DEBUG:teuthology.orchestra.run.smithi043:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-08-09T15:09:05.920 INFO:tasks.cephfs.test_snap_schedules:snapshots: set() 2023-08-09T15:09:05.921 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-08-09T15:09:05.921 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-08-09T15:09:06.922 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-09T15:09:06.923 DEBUG:teuthology.orchestra.run.smithi043:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-08-09T15:09:06.999 INFO:tasks.cephfs.test_snap_schedules:snapshots: set() 2023-08-09T15:09:07.000 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-08-09T15:09:07.000 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-08-09T15:09:08.001 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-09T15:09:08.001 DEBUG:teuthology.orchestra.run.smithi043:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-08-09T15:09:08.076 INFO:tasks.cephfs.test_snap_schedules:snapshots: set() 2023-08-09T15:09:08.076 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-08-09T15:09:08.077 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-08-09T15:09:09.077 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-09T15:09:09.078 DEBUG:teuthology.orchestra.run.smithi043:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-08-09T15:09:09.150 INFO:tasks.cephfs.test_snap_schedules:snapshots: set() 2023-08-09T15:09:09.151 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-08-09T15:09:09.151 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-08-09T15:09:10.152 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-09T15:09:10.153 DEBUG:teuthology.orchestra.run.smithi043:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-08-09T15:09:10.226 INFO:tasks.cephfs.test_snap_schedules:snapshots: set() 2023-08-09T15:09:10.226 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-08-09T15:09:10.227 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-08-09T15:09:11.228 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-09T15:09:11.228 DEBUG:teuthology.orchestra.run.smithi043:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-08-09T15:09:11.301 INFO:tasks.cephfs.test_snap_schedules:snapshots: set() 2023-08-09T15:09:11.302 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-08-09T15:09:11.302 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-08-09T15:09:12.303 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-09T15:09:12.304 DEBUG:teuthology.orchestra.run.smithi043:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-08-09T15:09:12.381 INFO:tasks.cephfs.test_snap_schedules:snapshots: set() 2023-08-09T15:09:12.381 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-08-09T15:09:12.382 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-08-09T15:09:13.383 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-09T15:09:13.383 DEBUG:teuthology.orchestra.run.smithi043:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-08-09T15:09:13.461 INFO:tasks.cephfs.test_snap_schedules:snapshots: set() 2023-08-09T15:09:13.461 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-08-09T15:09:13.461 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-08-09T15:09:14.463 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-09T15:09:14.464 DEBUG:teuthology.orchestra.run.smithi043:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-08-09T15:09:14.542 INFO:tasks.cephfs.test_snap_schedules:snapshots: set() 2023-08-09T15:09:14.542 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-08-09T15:09:14.542 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-08-09T15:09:15.544 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-09T15:09:15.544 DEBUG:teuthology.orchestra.run.smithi043:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-08-09T15:09:15.621 INFO:tasks.cephfs.test_snap_schedules:snapshots: set() 2023-08-09T15:09:15.621 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-08-09T15:09:15.621 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-08-09T15:09:16.622 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-09T15:09:16.623 DEBUG:teuthology.orchestra.run.smithi043:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-08-09T15:09:16.700 INFO:tasks.cephfs.test_snap_schedules:snapshots: set() 2023-08-09T15:09:16.701 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-08-09T15:09:16.701 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-08-09T15:09:17.702 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-09T15:09:17.703 DEBUG:teuthology.orchestra.run.smithi043:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-08-09T15:09:17.779 INFO:tasks.cephfs.test_snap_schedules:snapshots: set() 2023-08-09T15:09:17.779 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-08-09T15:09:17.780 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-08-09T15:09:18.780 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-09T15:09:18.781 DEBUG:teuthology.orchestra.run.smithi043:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-08-09T15:09:18.855 INFO:tasks.cephfs.test_snap_schedules:snapshots: set() 2023-08-09T15:09:18.855 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-08-09T15:09:18.856 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-08-09T15:09:19.857 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-09T15:09:19.858 DEBUG:teuthology.orchestra.run.smithi043:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-08-09T15:09:19.932 INFO:tasks.cephfs.test_snap_schedules:snapshots: set() 2023-08-09T15:09:19.932 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-08-09T15:09:19.933 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-08-09T15:09:20.934 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-09T15:09:20.934 DEBUG:teuthology.orchestra.run.smithi043:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-08-09T15:09:21.009 INFO:tasks.cephfs.test_snap_schedules:snapshots: set() 2023-08-09T15:09:21.010 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-08-09T15:09:21.010 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-08-09T15:09:22.011 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-09T15:09:22.012 DEBUG:teuthology.orchestra.run.smithi043:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-08-09T15:09:22.090 INFO:tasks.cephfs.test_snap_schedules:snapshots: set() 2023-08-09T15:09:22.090 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-08-09T15:09:22.091 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-08-09T15:09:23.091 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-09T15:09:23.092 DEBUG:teuthology.orchestra.run.smithi043:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-08-09T15:09:23.167 INFO:tasks.cephfs.test_snap_schedules:snapshots: set() 2023-08-09T15:09:23.167 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-08-09T15:09:23.167 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-08-09T15:09:24.169 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-09T15:09:24.169 DEBUG:teuthology.orchestra.run.smithi043:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-08-09T15:09:24.244 INFO:tasks.cephfs.test_snap_schedules:snapshots: set() 2023-08-09T15:09:24.244 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-08-09T15:09:24.245 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-08-09T15:09:25.245 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-09T15:09:25.246 DEBUG:teuthology.orchestra.run.smithi043:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-08-09T15:09:25.323 INFO:tasks.cephfs.test_snap_schedules:snapshots: set() 2023-08-09T15:09:25.323 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-08-09T15:09:25.324 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-08-09T15:09:26.324 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-09T15:09:26.325 DEBUG:teuthology.orchestra.run.smithi043:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-08-09T15:09:26.401 INFO:tasks.cephfs.test_snap_schedules:snapshots: set() 2023-08-09T15:09:26.402 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-08-09T15:09:26.402 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-08-09T15:09:27.403 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-09T15:09:27.403 DEBUG:teuthology.orchestra.run.smithi043:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-08-09T15:09:27.477 INFO:tasks.cephfs.test_snap_schedules:snapshots: set() 2023-08-09T15:09:27.478 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-08-09T15:09:27.478 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-08-09T15:09:28.479 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-09T15:09:28.480 DEBUG:teuthology.orchestra.run.smithi043:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-08-09T15:09:28.551 INFO:tasks.cephfs.test_snap_schedules:snapshots: set() 2023-08-09T15:09:28.552 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-08-09T15:09:28.552 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-08-09T15:09:29.553 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-09T15:09:29.553 DEBUG:teuthology.orchestra.run.smithi043:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-08-09T15:09:29.629 INFO:tasks.cephfs.test_snap_schedules:snapshots: set() 2023-08-09T15:09:29.629 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-08-09T15:09:29.629 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-08-09T15:09:30.630 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-09T15:09:30.631 DEBUG:teuthology.orchestra.run.smithi043:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-08-09T15:09:30.707 INFO:tasks.cephfs.test_snap_schedules:snapshots: set() 2023-08-09T15:09:30.708 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-08-09T15:09:30.708 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-08-09T15:09:31.404 DEBUG:teuthology.orchestra.run.smithi043:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-08-09T15:09:31.413 DEBUG:teuthology.orchestra.run.smithi203:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-08-09T15:09:31.710 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-09T15:09:31.711 DEBUG:teuthology.orchestra.run.smithi043:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-08-09T15:09:31.784 INFO:tasks.cephfs.test_snap_schedules:snapshots: set() 2023-08-09T15:09:31.785 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-08-09T15:09:31.785 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-08-09T15:09:32.787 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-09T15:09:32.787 DEBUG:teuthology.orchestra.run.smithi043:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-08-09T15:09:32.861 INFO:tasks.cephfs.test_snap_schedules:snapshots: set() 2023-08-09T15:09:32.862 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-08-09T15:09:32.862 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-08-09T15:09:33.863 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-09T15:09:33.864 DEBUG:teuthology.orchestra.run.smithi043:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-08-09T15:09:33.939 INFO:tasks.cephfs.test_snap_schedules:snapshots: set() 2023-08-09T15:09:33.939 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-08-09T15:09:33.940 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-08-09T15:09:34.941 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-09T15:09:34.941 DEBUG:teuthology.orchestra.run.smithi043:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-08-09T15:09:35.016 INFO:tasks.cephfs.test_snap_schedules:snapshots: set() 2023-08-09T15:09:35.016 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-08-09T15:09:35.017 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-08-09T15:09:36.018 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-09T15:09:36.019 DEBUG:teuthology.orchestra.run.smithi043:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-08-09T15:09:36.094 INFO:tasks.cephfs.test_snap_schedules:snapshots: set() 2023-08-09T15:09:36.094 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-08-09T15:09:36.095 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-08-09T15:09:37.096 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-09T15:09:37.096 DEBUG:teuthology.orchestra.run.smithi043:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-08-09T15:09:37.170 INFO:tasks.cephfs.test_snap_schedules:snapshots: set() 2023-08-09T15:09:37.170 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-08-09T15:09:37.170 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-08-09T15:09:38.171 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-09T15:09:38.172 DEBUG:teuthology.orchestra.run.smithi043:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-08-09T15:09:38.248 INFO:tasks.cephfs.test_snap_schedules:snapshots: set() 2023-08-09T15:09:38.249 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-08-09T15:09:38.249 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-08-09T15:09:39.250 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-09T15:09:39.250 DEBUG:teuthology.orchestra.run.smithi043:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-08-09T15:09:39.325 INFO:tasks.cephfs.test_snap_schedules:snapshots: set() 2023-08-09T15:09:39.326 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-08-09T15:09:39.326 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-08-09T15:09:40.327 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-09T15:09:40.327 DEBUG:teuthology.orchestra.run.smithi043:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-08-09T15:09:40.400 INFO:tasks.cephfs.test_snap_schedules:snapshots: set() 2023-08-09T15:09:40.401 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-08-09T15:09:40.401 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-08-09T15:09:41.402 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-09T15:09:41.403 DEBUG:teuthology.orchestra.run.smithi043:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-08-09T15:09:41.478 INFO:tasks.cephfs.test_snap_schedules:snapshots: set() 2023-08-09T15:09:41.479 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-08-09T15:09:41.479 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-08-09T15:09:42.479 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-09T15:09:42.480 DEBUG:teuthology.orchestra.run.smithi043:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-08-09T15:09:42.556 INFO:tasks.cephfs.test_snap_schedules:snapshots: set() 2023-08-09T15:09:42.556 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-08-09T15:09:42.557 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-08-09T15:09:43.558 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-09T15:09:43.559 DEBUG:teuthology.orchestra.run.smithi043:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-08-09T15:09:43.633 INFO:tasks.cephfs.test_snap_schedules:snapshots: set() 2023-08-09T15:09:43.634 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-08-09T15:09:43.634 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-08-09T15:09:44.635 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-09T15:09:44.636 DEBUG:teuthology.orchestra.run.smithi043:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-08-09T15:09:44.713 INFO:tasks.cephfs.test_snap_schedules:snapshots: set() 2023-08-09T15:09:44.714 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-08-09T15:09:44.714 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-08-09T15:09:45.715 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-09T15:09:45.716 DEBUG:teuthology.orchestra.run.smithi043:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-08-09T15:09:45.792 INFO:tasks.cephfs.test_snap_schedules:snapshots: set() 2023-08-09T15:09:45.793 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-08-09T15:09:45.793 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-08-09T15:09:46.794 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-09T15:09:46.794 DEBUG:teuthology.orchestra.run.smithi043:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-08-09T15:09:46.872 INFO:tasks.cephfs.test_snap_schedules:snapshots: set() 2023-08-09T15:09:46.872 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-08-09T15:09:46.873 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-08-09T15:09:47.874 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-09T15:09:47.874 DEBUG:teuthology.orchestra.run.smithi043:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-08-09T15:09:47.947 INFO:tasks.cephfs.test_snap_schedules:snapshots: set() 2023-08-09T15:09:47.948 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-08-09T15:09:47.948 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-08-09T15:09:48.949 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-09T15:09:48.949 DEBUG:teuthology.orchestra.run.smithi043:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-08-09T15:09:49.025 INFO:tasks.cephfs.test_snap_schedules:snapshots: set() 2023-08-09T15:09:49.025 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-08-09T15:09:49.025 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-08-09T15:09:50.026 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-09T15:09:50.027 DEBUG:teuthology.orchestra.run.smithi043:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-08-09T15:09:50.098 INFO:tasks.cephfs.test_snap_schedules:snapshots: set() 2023-08-09T15:09:50.099 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-08-09T15:09:50.099 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-08-09T15:09:51.100 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-09T15:09:51.100 DEBUG:teuthology.orchestra.run.smithi043:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-08-09T15:09:51.178 INFO:tasks.cephfs.test_snap_schedules:snapshots: set() 2023-08-09T15:09:51.178 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-08-09T15:09:51.179 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-08-09T15:09:52.179 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-09T15:09:52.180 DEBUG:teuthology.orchestra.run.smithi043:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-08-09T15:09:52.257 INFO:tasks.cephfs.test_snap_schedules:snapshots: set() 2023-08-09T15:09:52.257 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-08-09T15:09:52.258 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-08-09T15:09:53.259 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-09T15:09:53.259 DEBUG:teuthology.orchestra.run.smithi043:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-08-09T15:09:53.336 INFO:tasks.cephfs.test_snap_schedules:snapshots: set() 2023-08-09T15:09:53.336 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-08-09T15:09:53.337 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-08-09T15:09:54.338 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-09T15:09:54.339 DEBUG:teuthology.orchestra.run.smithi043:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-08-09T15:09:54.418 INFO:tasks.cephfs.test_snap_schedules:snapshots: set() 2023-08-09T15:09:54.418 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-08-09T15:09:54.418 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-08-09T15:09:55.420 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-09T15:09:55.421 DEBUG:teuthology.orchestra.run.smithi043:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-08-09T15:09:55.499 INFO:tasks.cephfs.test_snap_schedules:snapshots: set() 2023-08-09T15:09:55.499 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-08-09T15:09:55.499 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-08-09T15:09:56.500 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-09T15:09:56.501 DEBUG:teuthology.orchestra.run.smithi043:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-08-09T15:09:56.576 INFO:tasks.cephfs.test_snap_schedules:snapshots: set() 2023-08-09T15:09:56.576 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-08-09T15:09:56.576 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-08-09T15:09:57.578 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-09T15:09:57.578 DEBUG:teuthology.orchestra.run.smithi043:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-08-09T15:09:57.655 INFO:tasks.cephfs.test_snap_schedules:snapshots: set() 2023-08-09T15:09:57.655 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-08-09T15:09:57.656 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-08-09T15:09:58.656 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-09T15:09:58.657 DEBUG:teuthology.orchestra.run.smithi043:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-08-09T15:09:58.734 INFO:tasks.cephfs.test_snap_schedules:snapshots: set() 2023-08-09T15:09:58.735 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-08-09T15:09:58.735 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-08-09T15:09:59.736 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-09T15:09:59.737 DEBUG:teuthology.orchestra.run.smithi043:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-08-09T15:09:59.816 INFO:tasks.cephfs.test_snap_schedules:snapshots: set() 2023-08-09T15:09:59.817 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-08-09T15:09:59.817 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-08-09T15:10:00.164 INFO:tasks.ceph.mgr.x.smithi203.stderr:2023-08-09T15:10:00.163+0000 7fdc91108640 -1 client.0 error registering admin socket command: (17) File exists 2023-08-09T15:10:00.165 INFO:tasks.ceph.mgr.x.smithi203.stderr:2023-08-09T15:10:00.163+0000 7fdc91108640 -1 client.0 error registering admin socket command: (17) File exists 2023-08-09T15:10:00.165 INFO:tasks.ceph.mgr.x.smithi203.stderr:2023-08-09T15:10:00.163+0000 7fdc91108640 -1 client.0 error registering admin socket command: (17) File exists 2023-08-09T15:10:00.165 INFO:tasks.ceph.mgr.x.smithi203.stderr:2023-08-09T15:10:00.163+0000 7fdc91108640 -1 client.0 error registering admin socket command: (17) File exists 2023-08-09T15:10:00.165 INFO:tasks.ceph.mgr.x.smithi203.stderr:2023-08-09T15:10:00.163+0000 7fdc91108640 -1 client.0 error registering admin socket command: (17) File exists 2023-08-09T15:10:00.818 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-09T15:10:00.819 DEBUG:teuthology.orchestra.run.smithi043:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-08-09T15:10:00.894 INFO:teuthology.orchestra.run.smithi043.stdout:scheduled-2023-08-09-15_10_00_UTC 2023-08-09T15:10:00.896 INFO:tasks.cephfs.test_snap_schedules:snapshots: {'scheduled-2023-08-09-15_10_00_UTC'} 2023-08-09T15:10:00.896 INFO:tasks.cephfs.test_snap_schedules:added: {'scheduled-2023-08-09-15_10_00_UTC'} 2023-08-09T15:10:00.896 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-08-09T15:10:00.896 DEBUG:tasks.cephfs.test_snap_schedules:snapshots added=['scheduled-2023-08-09-15_10_00_UTC'] 2023-08-09T15:10:00.897 DEBUG:tasks.cephfs.test_snap_schedules:exec=1691593743.1699634, now = 1691593800.897262, timo = 57 2023-08-09T15:10:01.555 DEBUG:teuthology.orchestra.run.smithi043:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-08-09T15:10:01.564 DEBUG:teuthology.orchestra.run.smithi203:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-08-09T15:10:01.898 DEBUG:tasks.cephfs.test_snap_schedules:expecting removal of snap snap_test_dir1/.snap/scheduled-2023-08-09-15_10_00 in ~117s... 2023-08-09T15:10:01.898 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-09T15:10:01.899 DEBUG:teuthology.orchestra.run.smithi043:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-08-09T15:10:01.972 INFO:teuthology.orchestra.run.smithi043.stdout:scheduled-2023-08-09-15_10_00_UTC 2023-08-09T15:10:01.973 INFO:tasks.cephfs.test_snap_schedules:snapshots: {'scheduled-2023-08-09-15_10_00_UTC'} 2023-08-09T15:10:01.973 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-08-09T15:10:01.974 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-08-09T15:10:02.975 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-09T15:10:02.976 DEBUG:teuthology.orchestra.run.smithi043:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-08-09T15:10:03.049 INFO:teuthology.orchestra.run.smithi043.stdout:scheduled-2023-08-09-15_10_00_UTC 2023-08-09T15:10:03.050 INFO:tasks.cephfs.test_snap_schedules:snapshots: {'scheduled-2023-08-09-15_10_00_UTC'} 2023-08-09T15:10:03.051 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-08-09T15:10:03.051 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-08-09T15:10:04.052 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-09T15:10:04.053 DEBUG:teuthology.orchestra.run.smithi043:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-08-09T15:10:04.124 INFO:teuthology.orchestra.run.smithi043.stdout:scheduled-2023-08-09-15_10_00_UTC 2023-08-09T15:10:04.126 INFO:tasks.cephfs.test_snap_schedules:snapshots: {'scheduled-2023-08-09-15_10_00_UTC'} 2023-08-09T15:10:04.126 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-08-09T15:10:04.126 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-08-09T15:10:05.128 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-09T15:10:05.129 DEBUG:teuthology.orchestra.run.smithi043:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-08-09T15:10:05.205 INFO:teuthology.orchestra.run.smithi043.stdout:scheduled-2023-08-09-15_10_00_UTC 2023-08-09T15:10:05.206 INFO:tasks.cephfs.test_snap_schedules:snapshots: {'scheduled-2023-08-09-15_10_00_UTC'} 2023-08-09T15:10:05.206 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-08-09T15:10:05.207 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-08-09T15:10:06.208 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-09T15:10:06.209 DEBUG:teuthology.orchestra.run.smithi043:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-08-09T15:10:06.285 INFO:teuthology.orchestra.run.smithi043.stdout:scheduled-2023-08-09-15_10_00_UTC 2023-08-09T15:10:06.285 INFO:tasks.cephfs.test_snap_schedules:snapshots: {'scheduled-2023-08-09-15_10_00_UTC'} 2023-08-09T15:10:06.285 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-08-09T15:10:06.286 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-08-09T15:10:07.286 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-09T15:10:07.287 DEBUG:teuthology.orchestra.run.smithi043:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-08-09T15:10:07.362 INFO:teuthology.orchestra.run.smithi043.stdout:scheduled-2023-08-09-15_10_00_UTC 2023-08-09T15:10:07.363 INFO:tasks.cephfs.test_snap_schedules:snapshots: {'scheduled-2023-08-09-15_10_00_UTC'} 2023-08-09T15:10:07.364 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-08-09T15:10:07.364 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-08-09T15:10:08.365 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-09T15:10:08.366 DEBUG:teuthology.orchestra.run.smithi043:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-08-09T15:10:08.443 INFO:teuthology.orchestra.run.smithi043.stdout:scheduled-2023-08-09-15_10_00_UTC 2023-08-09T15:10:08.444 INFO:tasks.cephfs.test_snap_schedules:snapshots: {'scheduled-2023-08-09-15_10_00_UTC'} 2023-08-09T15:10:08.445 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-08-09T15:10:08.445 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-08-09T15:10:09.446 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-09T15:10:09.446 DEBUG:teuthology.orchestra.run.smithi043:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-08-09T15:10:09.521 INFO:teuthology.orchestra.run.smithi043.stdout:scheduled-2023-08-09-15_10_00_UTC 2023-08-09T15:10:09.523 INFO:tasks.cephfs.test_snap_schedules:snapshots: {'scheduled-2023-08-09-15_10_00_UTC'} 2023-08-09T15:10:09.523 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-08-09T15:10:09.524 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-08-09T15:10:10.525 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-09T15:10:10.526 DEBUG:teuthology.orchestra.run.smithi043:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-08-09T15:10:10.600 INFO:teuthology.orchestra.run.smithi043.stdout:scheduled-2023-08-09-15_10_00_UTC 2023-08-09T15:10:10.602 INFO:tasks.cephfs.test_snap_schedules:snapshots: {'scheduled-2023-08-09-15_10_00_UTC'} 2023-08-09T15:10:10.602 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-08-09T15:10:10.603 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-08-09T15:10:11.603 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-09T15:10:11.604 DEBUG:teuthology.orchestra.run.smithi043:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-08-09T15:10:11.678 INFO:teuthology.orchestra.run.smithi043.stdout:scheduled-2023-08-09-15_10_00_UTC 2023-08-09T15:10:11.679 INFO:tasks.cephfs.test_snap_schedules:snapshots: {'scheduled-2023-08-09-15_10_00_UTC'} 2023-08-09T15:10:11.679 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-08-09T15:10:11.679 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-08-09T15:10:12.680 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-09T15:10:12.681 DEBUG:teuthology.orchestra.run.smithi043:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-08-09T15:10:12.757 INFO:teuthology.orchestra.run.smithi043.stdout:scheduled-2023-08-09-15_10_00_UTC 2023-08-09T15:10:12.758 INFO:tasks.cephfs.test_snap_schedules:snapshots: {'scheduled-2023-08-09-15_10_00_UTC'} 2023-08-09T15:10:12.759 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-08-09T15:10:12.759 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-08-09T15:10:13.760 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-09T15:10:13.760 DEBUG:teuthology.orchestra.run.smithi043:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-08-09T15:10:13.835 INFO:teuthology.orchestra.run.smithi043.stdout:scheduled-2023-08-09-15_10_00_UTC 2023-08-09T15:10:13.836 INFO:tasks.cephfs.test_snap_schedules:snapshots: {'scheduled-2023-08-09-15_10_00_UTC'} 2023-08-09T15:10:13.837 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-08-09T15:10:13.837 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-08-09T15:10:14.838 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-09T15:10:14.839 DEBUG:teuthology.orchestra.run.smithi043:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-08-09T15:10:14.913 INFO:teuthology.orchestra.run.smithi043.stdout:scheduled-2023-08-09-15_10_00_UTC 2023-08-09T15:10:14.914 INFO:tasks.cephfs.test_snap_schedules:snapshots: {'scheduled-2023-08-09-15_10_00_UTC'} 2023-08-09T15:10:14.915 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-08-09T15:10:14.915 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-08-09T15:10:15.916 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-09T15:10:15.916 DEBUG:teuthology.orchestra.run.smithi043:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-08-09T15:10:15.989 INFO:teuthology.orchestra.run.smithi043.stdout:scheduled-2023-08-09-15_10_00_UTC 2023-08-09T15:10:15.990 INFO:tasks.cephfs.test_snap_schedules:snapshots: {'scheduled-2023-08-09-15_10_00_UTC'} 2023-08-09T15:10:15.990 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-08-09T15:10:15.991 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-08-09T15:10:16.991 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-09T15:10:16.992 DEBUG:teuthology.orchestra.run.smithi043:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-08-09T15:10:17.066 INFO:teuthology.orchestra.run.smithi043.stdout:scheduled-2023-08-09-15_10_00_UTC 2023-08-09T15:10:17.067 INFO:tasks.cephfs.test_snap_schedules:snapshots: {'scheduled-2023-08-09-15_10_00_UTC'} 2023-08-09T15:10:17.068 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-08-09T15:10:17.068 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-08-09T15:10:18.070 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-09T15:10:18.070 DEBUG:teuthology.orchestra.run.smithi043:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-08-09T15:10:18.144 INFO:teuthology.orchestra.run.smithi043.stdout:scheduled-2023-08-09-15_10_00_UTC 2023-08-09T15:10:18.145 INFO:tasks.cephfs.test_snap_schedules:snapshots: {'scheduled-2023-08-09-15_10_00_UTC'} 2023-08-09T15:10:18.146 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-08-09T15:10:18.146 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-08-09T15:10:19.147 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-09T15:10:19.147 DEBUG:teuthology.orchestra.run.smithi043:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-08-09T15:10:19.220 INFO:teuthology.orchestra.run.smithi043.stdout:scheduled-2023-08-09-15_10_00_UTC 2023-08-09T15:10:19.222 INFO:tasks.cephfs.test_snap_schedules:snapshots: {'scheduled-2023-08-09-15_10_00_UTC'} 2023-08-09T15:10:19.222 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-08-09T15:10:19.222 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-08-09T15:10:20.223 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-09T15:10:20.224 DEBUG:teuthology.orchestra.run.smithi043:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-08-09T15:10:20.299 INFO:teuthology.orchestra.run.smithi043.stdout:scheduled-2023-08-09-15_10_00_UTC 2023-08-09T15:10:20.300 INFO:tasks.cephfs.test_snap_schedules:snapshots: {'scheduled-2023-08-09-15_10_00_UTC'} 2023-08-09T15:10:20.301 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-08-09T15:10:20.301 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-08-09T15:10:21.302 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-09T15:10:21.302 DEBUG:teuthology.orchestra.run.smithi043:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-08-09T15:10:21.377 INFO:teuthology.orchestra.run.smithi043.stdout:scheduled-2023-08-09-15_10_00_UTC 2023-08-09T15:10:21.378 INFO:tasks.cephfs.test_snap_schedules:snapshots: {'scheduled-2023-08-09-15_10_00_UTC'} 2023-08-09T15:10:21.378 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-08-09T15:10:21.379 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-08-09T15:10:22.380 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-09T15:10:22.381 DEBUG:teuthology.orchestra.run.smithi043:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-08-09T15:10:22.454 INFO:teuthology.orchestra.run.smithi043.stdout:scheduled-2023-08-09-15_10_00_UTC 2023-08-09T15:10:22.455 INFO:tasks.cephfs.test_snap_schedules:snapshots: {'scheduled-2023-08-09-15_10_00_UTC'} 2023-08-09T15:10:22.455 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-08-09T15:10:22.455 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-08-09T15:10:23.456 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-09T15:10:23.457 DEBUG:teuthology.orchestra.run.smithi043:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-08-09T15:10:23.529 INFO:teuthology.orchestra.run.smithi043.stdout:scheduled-2023-08-09-15_10_00_UTC 2023-08-09T15:10:23.530 INFO:tasks.cephfs.test_snap_schedules:snapshots: {'scheduled-2023-08-09-15_10_00_UTC'} 2023-08-09T15:10:23.530 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-08-09T15:10:23.530 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-08-09T15:10:24.531 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-09T15:10:24.532 DEBUG:teuthology.orchestra.run.smithi043:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-08-09T15:10:24.605 INFO:teuthology.orchestra.run.smithi043.stdout:scheduled-2023-08-09-15_10_00_UTC 2023-08-09T15:10:24.606 INFO:tasks.cephfs.test_snap_schedules:snapshots: {'scheduled-2023-08-09-15_10_00_UTC'} 2023-08-09T15:10:24.606 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-08-09T15:10:24.607 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-08-09T15:10:25.608 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-09T15:10:25.608 DEBUG:teuthology.orchestra.run.smithi043:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-08-09T15:10:25.682 INFO:teuthology.orchestra.run.smithi043.stdout:scheduled-2023-08-09-15_10_00_UTC 2023-08-09T15:10:25.683 INFO:tasks.cephfs.test_snap_schedules:snapshots: {'scheduled-2023-08-09-15_10_00_UTC'} 2023-08-09T15:10:25.683 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-08-09T15:10:25.684 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-08-09T15:10:26.685 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-09T15:10:26.686 DEBUG:teuthology.orchestra.run.smithi043:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-08-09T15:10:26.757 INFO:teuthology.orchestra.run.smithi043.stdout:scheduled-2023-08-09-15_10_00_UTC 2023-08-09T15:10:26.758 INFO:tasks.cephfs.test_snap_schedules:snapshots: {'scheduled-2023-08-09-15_10_00_UTC'} 2023-08-09T15:10:26.759 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-08-09T15:10:26.759 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-08-09T15:10:27.760 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-09T15:10:27.761 DEBUG:teuthology.orchestra.run.smithi043:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-08-09T15:10:27.834 INFO:teuthology.orchestra.run.smithi043.stdout:scheduled-2023-08-09-15_10_00_UTC 2023-08-09T15:10:27.835 INFO:tasks.cephfs.test_snap_schedules:snapshots: {'scheduled-2023-08-09-15_10_00_UTC'} 2023-08-09T15:10:27.836 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-08-09T15:10:27.836 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-08-09T15:10:28.837 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-09T15:10:28.838 DEBUG:teuthology.orchestra.run.smithi043:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-08-09T15:10:28.912 INFO:teuthology.orchestra.run.smithi043.stdout:scheduled-2023-08-09-15_10_00_UTC 2023-08-09T15:10:28.913 INFO:tasks.cephfs.test_snap_schedules:snapshots: {'scheduled-2023-08-09-15_10_00_UTC'} 2023-08-09T15:10:28.914 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-08-09T15:10:28.914 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-08-09T15:10:29.915 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-09T15:10:29.915 DEBUG:teuthology.orchestra.run.smithi043:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-08-09T15:10:29.990 INFO:teuthology.orchestra.run.smithi043.stdout:scheduled-2023-08-09-15_10_00_UTC 2023-08-09T15:10:29.992 INFO:tasks.cephfs.test_snap_schedules:snapshots: {'scheduled-2023-08-09-15_10_00_UTC'} 2023-08-09T15:10:29.992 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-08-09T15:10:29.992 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-08-09T15:10:30.993 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-09T15:10:30.994 DEBUG:teuthology.orchestra.run.smithi043:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-08-09T15:10:31.070 INFO:teuthology.orchestra.run.smithi043.stdout:scheduled-2023-08-09-15_10_00_UTC 2023-08-09T15:10:31.072 INFO:tasks.cephfs.test_snap_schedules:snapshots: {'scheduled-2023-08-09-15_10_00_UTC'} 2023-08-09T15:10:31.072 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-08-09T15:10:31.073 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-08-09T15:10:31.820 DEBUG:teuthology.orchestra.run.smithi043:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-08-09T15:10:31.830 DEBUG:teuthology.orchestra.run.smithi203:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-08-09T15:10:32.073 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-09T15:10:32.074 DEBUG:teuthology.orchestra.run.smithi043:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-08-09T15:10:32.145 INFO:teuthology.orchestra.run.smithi043.stdout:scheduled-2023-08-09-15_10_00_UTC 2023-08-09T15:10:32.146 INFO:tasks.cephfs.test_snap_schedules:snapshots: {'scheduled-2023-08-09-15_10_00_UTC'} 2023-08-09T15:10:32.146 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-08-09T15:10:32.147 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-08-09T15:10:33.147 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-09T15:10:33.148 DEBUG:teuthology.orchestra.run.smithi043:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-08-09T15:10:33.219 INFO:teuthology.orchestra.run.smithi043.stdout:scheduled-2023-08-09-15_10_00_UTC 2023-08-09T15:10:33.221 INFO:tasks.cephfs.test_snap_schedules:snapshots: {'scheduled-2023-08-09-15_10_00_UTC'} 2023-08-09T15:10:33.221 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-08-09T15:10:33.221 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-08-09T15:10:34.222 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-09T15:10:34.223 DEBUG:teuthology.orchestra.run.smithi043:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-08-09T15:10:34.297 INFO:teuthology.orchestra.run.smithi043.stdout:scheduled-2023-08-09-15_10_00_UTC 2023-08-09T15:10:34.298 INFO:tasks.cephfs.test_snap_schedules:snapshots: {'scheduled-2023-08-09-15_10_00_UTC'} 2023-08-09T15:10:34.298 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-08-09T15:10:34.299 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-08-09T15:10:35.299 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-09T15:10:35.300 DEBUG:teuthology.orchestra.run.smithi043:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-08-09T15:10:35.371 INFO:teuthology.orchestra.run.smithi043.stdout:scheduled-2023-08-09-15_10_00_UTC 2023-08-09T15:10:35.372 INFO:tasks.cephfs.test_snap_schedules:snapshots: {'scheduled-2023-08-09-15_10_00_UTC'} 2023-08-09T15:10:35.373 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-08-09T15:10:35.373 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-08-09T15:10:36.374 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-09T15:10:36.375 DEBUG:teuthology.orchestra.run.smithi043:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-08-09T15:10:36.452 INFO:teuthology.orchestra.run.smithi043.stdout:scheduled-2023-08-09-15_10_00_UTC 2023-08-09T15:10:36.453 INFO:tasks.cephfs.test_snap_schedules:snapshots: {'scheduled-2023-08-09-15_10_00_UTC'} 2023-08-09T15:10:36.453 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-08-09T15:10:36.454 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-08-09T15:10:37.455 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-09T15:10:37.455 DEBUG:teuthology.orchestra.run.smithi043:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-08-09T15:10:37.527 INFO:teuthology.orchestra.run.smithi043.stdout:scheduled-2023-08-09-15_10_00_UTC 2023-08-09T15:10:37.528 INFO:tasks.cephfs.test_snap_schedules:snapshots: {'scheduled-2023-08-09-15_10_00_UTC'} 2023-08-09T15:10:37.529 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-08-09T15:10:37.529 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-08-09T15:10:38.530 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-09T15:10:38.531 DEBUG:teuthology.orchestra.run.smithi043:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-08-09T15:10:38.605 INFO:teuthology.orchestra.run.smithi043.stdout:scheduled-2023-08-09-15_10_00_UTC 2023-08-09T15:10:38.606 INFO:tasks.cephfs.test_snap_schedules:snapshots: {'scheduled-2023-08-09-15_10_00_UTC'} 2023-08-09T15:10:38.606 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-08-09T15:10:38.607 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-08-09T15:10:39.608 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-09T15:10:39.609 DEBUG:teuthology.orchestra.run.smithi043:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-08-09T15:10:39.680 INFO:teuthology.orchestra.run.smithi043.stdout:scheduled-2023-08-09-15_10_00_UTC 2023-08-09T15:10:39.681 INFO:tasks.cephfs.test_snap_schedules:snapshots: {'scheduled-2023-08-09-15_10_00_UTC'} 2023-08-09T15:10:39.681 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-08-09T15:10:39.682 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-08-09T15:10:40.683 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-09T15:10:40.684 DEBUG:teuthology.orchestra.run.smithi043:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-08-09T15:10:40.757 INFO:teuthology.orchestra.run.smithi043.stdout:scheduled-2023-08-09-15_10_00_UTC 2023-08-09T15:10:40.758 INFO:tasks.cephfs.test_snap_schedules:snapshots: {'scheduled-2023-08-09-15_10_00_UTC'} 2023-08-09T15:10:40.759 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-08-09T15:10:40.759 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-08-09T15:10:41.759 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-09T15:10:41.760 DEBUG:teuthology.orchestra.run.smithi043:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-08-09T15:10:41.833 INFO:teuthology.orchestra.run.smithi043.stdout:scheduled-2023-08-09-15_10_00_UTC 2023-08-09T15:10:41.834 INFO:tasks.cephfs.test_snap_schedules:snapshots: {'scheduled-2023-08-09-15_10_00_UTC'} 2023-08-09T15:10:41.835 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-08-09T15:10:41.835 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-08-09T15:10:42.836 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-09T15:10:42.836 DEBUG:teuthology.orchestra.run.smithi043:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-08-09T15:10:42.908 INFO:teuthology.orchestra.run.smithi043.stdout:scheduled-2023-08-09-15_10_00_UTC 2023-08-09T15:10:42.909 INFO:tasks.cephfs.test_snap_schedules:snapshots: {'scheduled-2023-08-09-15_10_00_UTC'} 2023-08-09T15:10:42.910 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-08-09T15:10:42.910 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-08-09T15:10:43.911 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-09T15:10:43.912 DEBUG:teuthology.orchestra.run.smithi043:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-08-09T15:10:43.986 INFO:teuthology.orchestra.run.smithi043.stdout:scheduled-2023-08-09-15_10_00_UTC 2023-08-09T15:10:43.988 INFO:tasks.cephfs.test_snap_schedules:snapshots: {'scheduled-2023-08-09-15_10_00_UTC'} 2023-08-09T15:10:43.988 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-08-09T15:10:43.989 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-08-09T15:10:44.990 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-09T15:10:44.991 DEBUG:teuthology.orchestra.run.smithi043:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-08-09T15:10:45.063 INFO:teuthology.orchestra.run.smithi043.stdout:scheduled-2023-08-09-15_10_00_UTC 2023-08-09T15:10:45.064 INFO:tasks.cephfs.test_snap_schedules:snapshots: {'scheduled-2023-08-09-15_10_00_UTC'} 2023-08-09T15:10:45.064 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-08-09T15:10:45.065 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-08-09T15:10:46.066 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-09T15:10:46.066 DEBUG:teuthology.orchestra.run.smithi043:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-08-09T15:10:46.145 INFO:teuthology.orchestra.run.smithi043.stdout:scheduled-2023-08-09-15_10_00_UTC 2023-08-09T15:10:46.146 INFO:tasks.cephfs.test_snap_schedules:snapshots: {'scheduled-2023-08-09-15_10_00_UTC'} 2023-08-09T15:10:46.147 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-08-09T15:10:46.147 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-08-09T15:10:47.148 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-09T15:10:47.149 DEBUG:teuthology.orchestra.run.smithi043:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-08-09T15:10:47.225 INFO:teuthology.orchestra.run.smithi043.stdout:scheduled-2023-08-09-15_10_00_UTC 2023-08-09T15:10:47.227 INFO:tasks.cephfs.test_snap_schedules:snapshots: {'scheduled-2023-08-09-15_10_00_UTC'} 2023-08-09T15:10:47.227 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-08-09T15:10:47.227 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-08-09T15:10:48.228 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-09T15:10:48.229 DEBUG:teuthology.orchestra.run.smithi043:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-08-09T15:10:48.302 INFO:teuthology.orchestra.run.smithi043.stdout:scheduled-2023-08-09-15_10_00_UTC 2023-08-09T15:10:48.304 INFO:tasks.cephfs.test_snap_schedules:snapshots: {'scheduled-2023-08-09-15_10_00_UTC'} 2023-08-09T15:10:48.304 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-08-09T15:10:48.304 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-08-09T15:10:49.305 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-09T15:10:49.306 DEBUG:teuthology.orchestra.run.smithi043:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-08-09T15:10:49.379 INFO:teuthology.orchestra.run.smithi043.stdout:scheduled-2023-08-09-15_10_00_UTC 2023-08-09T15:10:49.380 INFO:tasks.cephfs.test_snap_schedules:snapshots: {'scheduled-2023-08-09-15_10_00_UTC'} 2023-08-09T15:10:49.381 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-08-09T15:10:49.381 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-08-09T15:10:50.382 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-09T15:10:50.383 DEBUG:teuthology.orchestra.run.smithi043:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-08-09T15:10:50.456 INFO:teuthology.orchestra.run.smithi043.stdout:scheduled-2023-08-09-15_10_00_UTC 2023-08-09T15:10:50.458 INFO:tasks.cephfs.test_snap_schedules:snapshots: {'scheduled-2023-08-09-15_10_00_UTC'} 2023-08-09T15:10:50.458 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-08-09T15:10:50.458 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-08-09T15:10:51.459 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-09T15:10:51.460 DEBUG:teuthology.orchestra.run.smithi043:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-08-09T15:10:51.535 INFO:teuthology.orchestra.run.smithi043.stdout:scheduled-2023-08-09-15_10_00_UTC 2023-08-09T15:10:51.536 INFO:tasks.cephfs.test_snap_schedules:snapshots: {'scheduled-2023-08-09-15_10_00_UTC'} 2023-08-09T15:10:51.537 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-08-09T15:10:51.537 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-08-09T15:10:52.537 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-09T15:10:52.538 DEBUG:teuthology.orchestra.run.smithi043:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-08-09T15:10:52.614 INFO:teuthology.orchestra.run.smithi043.stdout:scheduled-2023-08-09-15_10_00_UTC 2023-08-09T15:10:52.615 INFO:tasks.cephfs.test_snap_schedules:snapshots: {'scheduled-2023-08-09-15_10_00_UTC'} 2023-08-09T15:10:52.615 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-08-09T15:10:52.616 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-08-09T15:10:53.616 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-09T15:10:53.617 DEBUG:teuthology.orchestra.run.smithi043:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-08-09T15:10:53.690 INFO:teuthology.orchestra.run.smithi043.stdout:scheduled-2023-08-09-15_10_00_UTC 2023-08-09T15:10:53.692 INFO:tasks.cephfs.test_snap_schedules:snapshots: {'scheduled-2023-08-09-15_10_00_UTC'} 2023-08-09T15:10:53.692 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-08-09T15:10:53.693 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-08-09T15:10:54.693 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-09T15:10:54.694 DEBUG:teuthology.orchestra.run.smithi043:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-08-09T15:10:54.766 INFO:teuthology.orchestra.run.smithi043.stdout:scheduled-2023-08-09-15_10_00_UTC 2023-08-09T15:10:54.768 INFO:tasks.cephfs.test_snap_schedules:snapshots: {'scheduled-2023-08-09-15_10_00_UTC'} 2023-08-09T15:10:54.768 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-08-09T15:10:54.768 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-08-09T15:10:55.770 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-09T15:10:55.770 DEBUG:teuthology.orchestra.run.smithi043:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-08-09T15:10:55.841 INFO:teuthology.orchestra.run.smithi043.stdout:scheduled-2023-08-09-15_10_00_UTC 2023-08-09T15:10:55.842 INFO:tasks.cephfs.test_snap_schedules:snapshots: {'scheduled-2023-08-09-15_10_00_UTC'} 2023-08-09T15:10:55.843 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-08-09T15:10:55.843 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-08-09T15:10:56.843 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-09T15:10:56.844 DEBUG:teuthology.orchestra.run.smithi043:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-08-09T15:10:56.914 INFO:teuthology.orchestra.run.smithi043.stdout:scheduled-2023-08-09-15_10_00_UTC 2023-08-09T15:10:56.915 INFO:tasks.cephfs.test_snap_schedules:snapshots: {'scheduled-2023-08-09-15_10_00_UTC'} 2023-08-09T15:10:56.916 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-08-09T15:10:56.916 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-08-09T15:10:57.917 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-09T15:10:57.918 DEBUG:teuthology.orchestra.run.smithi043:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-08-09T15:10:57.992 INFO:teuthology.orchestra.run.smithi043.stdout:scheduled-2023-08-09-15_10_00_UTC 2023-08-09T15:10:57.993 INFO:tasks.cephfs.test_snap_schedules:snapshots: {'scheduled-2023-08-09-15_10_00_UTC'} 2023-08-09T15:10:57.994 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-08-09T15:10:57.994 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-08-09T15:10:58.994 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-09T15:10:58.995 DEBUG:teuthology.orchestra.run.smithi043:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-08-09T15:10:59.069 INFO:teuthology.orchestra.run.smithi043.stdout:scheduled-2023-08-09-15_10_00_UTC 2023-08-09T15:10:59.071 INFO:tasks.cephfs.test_snap_schedules:snapshots: {'scheduled-2023-08-09-15_10_00_UTC'} 2023-08-09T15:10:59.071 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-08-09T15:10:59.071 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-08-09T15:11:00.073 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-09T15:11:00.073 DEBUG:teuthology.orchestra.run.smithi043:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-08-09T15:11:00.148 INFO:teuthology.orchestra.run.smithi043.stdout:scheduled-2023-08-09-15_10_00_UTC 2023-08-09T15:11:00.149 INFO:tasks.cephfs.test_snap_schedules:snapshots: {'scheduled-2023-08-09-15_10_00_UTC'} 2023-08-09T15:11:00.149 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-08-09T15:11:00.150 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-08-09T15:11:01.151 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-09T15:11:01.151 DEBUG:teuthology.orchestra.run.smithi043:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-08-09T15:11:01.226 INFO:teuthology.orchestra.run.smithi043.stdout:scheduled-2023-08-09-15_11_00_UTC 2023-08-09T15:11:01.227 INFO:tasks.cephfs.test_snap_schedules:snapshots: {'scheduled-2023-08-09-15_11_00_UTC'} 2023-08-09T15:11:01.228 INFO:tasks.cephfs.test_snap_schedules:added: {'scheduled-2023-08-09-15_11_00_UTC'} 2023-08-09T15:11:01.228 INFO:tasks.cephfs.test_snap_schedules:removed: {'scheduled-2023-08-09-15_10_00_UTC'} 2023-08-09T15:11:01.228 DEBUG:tasks.cephfs.test_snap_schedules:snapshots removed=['scheduled-2023-08-09-15_10_00_UTC'] 2023-08-09T15:11:01.229 DEBUG:tasks.cephfs.test_snap_schedules:exec=1691593743.1699634, now = 1691593861.229018, timo = 117 2023-08-09T15:11:01.975 DEBUG:teuthology.orchestra.run.smithi043:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-08-09T15:11:01.984 DEBUG:teuthology.orchestra.run.smithi203:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-08-09T15:11:02.229 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs snap-schedule remove --fs cephfs snap_test_dir1 2023-08-09T15:11:02.475 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:02.473+0000 7f9897547640 1 Processor -- start 2023-08-09T15:11:02.475 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:02.474+0000 7f9897547640 1 -- start start 2023-08-09T15:11:02.476 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:02.475+0000 7f9897547640 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7f989006fb70 0x7f98900caab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T15:11:02.477 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:02.475+0000 7f9897547640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f98900ce4c0 0x7f98900ce950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T15:11:02.477 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:02.475+0000 7f9897547640 1 -- --> v1:172.21.15.203:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f989015b9b0 con 0x7f98900ca370 2023-08-09T15:11:02.477 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:02.475+0000 7f9897547640 1 -- --> v2:172.21.15.43:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f98900cee90 con 0x7f98900ce4c0 2023-08-09T15:11:02.477 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:02.475+0000 7f9897547640 1 -- --> v2:172.21.15.203:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f989015ae10 con 0x7f989006fb70 2023-08-09T15:11:02.478 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:02.476+0000 7f9895ae4640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f98900ce4c0 0x7f98900ce950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T15:11:02.478 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:02.476+0000 7f9895ae4640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f98900ce4c0 0x7f98900ce950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:53956/0 (socket says 172.21.15.43:53956) 2023-08-09T15:11:02.478 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:02.476+0000 7f9895ae4640 1 -- 172.21.15.43:0/543283824 learned_addr learned my addr 172.21.15.43:0/543283824 (peer_addr_for_me v2:172.21.15.43:0/0) 2023-08-09T15:11:02.478 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:02.476+0000 7f98952e3640 1 --2- 172.21.15.43:0/543283824 >> v2:172.21.15.203:3300/0 conn(0x7f989006fb70 0x7f98900caab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T15:11:02.479 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:02.476+0000 7f9895ae4640 1 -- 172.21.15.43:0/543283824 >> v2:172.21.15.203:3300/0 conn(0x7f989006fb70 msgr2=0x7f98900caab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-09T15:11:02.479 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:02.476+0000 7f9895ae4640 1 --2- 172.21.15.43:0/543283824 >> v2:172.21.15.203:3300/0 conn(0x7f989006fb70 0x7f98900caab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T15:11:02.479 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:02.476+0000 7f9895ae4640 1 -- 172.21.15.43:0/543283824 >> v1:172.21.15.203:6789/0 conn(0x7f98900ca370 legacy=0x7f989015b2a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-09T15:11:02.479 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:02.476+0000 7f9895ae4640 1 -- 172.21.15.43:0/543283824 --> v2:172.21.15.43:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9890154e70 con 0x7f98900ce4c0 2023-08-09T15:11:02.480 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:02.477+0000 7f9895ae4640 1 --2- 172.21.15.43:0/543283824 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f98900ce4c0 0x7f98900ce950 secure :-1 s=READY pgs=2270 cs=0 l=1 rev1=1 crypto rx=0x7f98900cbb80 tx=0x7f988802bee0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ea16299981b5551f server_cookie=0 in_seq=0 out_seq=0 2023-08-09T15:11:02.480 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:02.478+0000 7f9887fff640 1 -- 172.21.15.43:0/543283824 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f988802ca80 con 0x7f98900ce4c0 2023-08-09T15:11:02.480 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:02.478+0000 7f9887fff640 1 -- 172.21.15.43:0/543283824 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f988802cbe0 con 0x7f98900ce4c0 2023-08-09T15:11:02.480 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:02.478+0000 7f9887fff640 1 -- 172.21.15.43:0/543283824 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f98880343f0 con 0x7f98900ce4c0 2023-08-09T15:11:02.480 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:02.479+0000 7f9897547640 1 -- 172.21.15.43:0/543283824 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f98900ce4c0 msgr2=0x7f98900ce950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T15:11:02.480 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:02.479+0000 7f9897547640 1 --2- 172.21.15.43:0/543283824 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f98900ce4c0 0x7f98900ce950 secure :-1 s=READY pgs=2270 cs=0 l=1 rev1=1 crypto rx=0x7f98900cbb80 tx=0x7f988802bee0 comp rx=0 tx=0).stop 2023-08-09T15:11:02.481 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:02.479+0000 7f9897547640 1 -- 172.21.15.43:0/543283824 shutdown_connections 2023-08-09T15:11:02.481 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:02.479+0000 7f9897547640 1 --2- 172.21.15.43:0/543283824 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f98900ce4c0 0x7f98900ce950 unknown :-1 s=CLOSED pgs=2270 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T15:11:02.481 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:02.479+0000 7f9897547640 1 --2- 172.21.15.43:0/543283824 >> v2:172.21.15.203:3300/0 conn(0x7f989006fb70 0x7f98900caab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T15:11:02.481 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:02.479+0000 7f9897547640 1 -- 172.21.15.43:0/543283824 >> 172.21.15.43:0/543283824 conn(0x7f989006b990 msgr2=0x7f989006bda0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-09T15:11:02.482 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:02.479+0000 7f9897547640 1 -- 172.21.15.43:0/543283824 shutdown_connections 2023-08-09T15:11:02.482 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:02.479+0000 7f9897547640 1 -- 172.21.15.43:0/543283824 wait complete. 2023-08-09T15:11:02.482 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:02.481+0000 7f9897547640 1 Processor -- start 2023-08-09T15:11:02.482 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:02.481+0000 7f9897547640 1 -- start start 2023-08-09T15:11:02.484 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:02.482+0000 7f9897547640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f989006fb70 0x7f9890156630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T15:11:02.484 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:02.482+0000 7f9897547640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f98900ca370 0x7f9890156b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T15:11:02.484 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:02.482+0000 7f9897547640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f98900ce4c0 0x7f98901ed490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T15:11:02.484 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:02.483+0000 7f9897547640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f98901eec20 con 0x7f989006fb70 2023-08-09T15:11:02.484 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:02.483+0000 7f9897547640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f98901ee0a0 con 0x7f98900ca370 2023-08-09T15:11:02.485 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:02.483+0000 7f9897547640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f98901ee380 con 0x7f98900ce4c0 2023-08-09T15:11:02.485 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:02.483+0000 7f98952e3640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f989006fb70 0x7f9890156630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T15:11:02.485 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:02.483+0000 7f98952e3640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f989006fb70 0x7f9890156630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:53962/0 (socket says 172.21.15.43:53962) 2023-08-09T15:11:02.485 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:02.483+0000 7f98952e3640 1 -- 172.21.15.43:0/2399222578 learned_addr learned my addr 172.21.15.43:0/2399222578 (peer_addr_for_me v2:172.21.15.43:0/0) 2023-08-09T15:11:02.486 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:02.483+0000 7f9894ae2640 1 --2- 172.21.15.43:0/2399222578 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f98900ca370 0x7f9890156b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T15:11:02.486 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:02.483+0000 7f98952e3640 1 -- 172.21.15.43:0/2399222578 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f98900ce4c0 msgr2=0x7f98901ed490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T15:11:02.486 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:02.483+0000 7f98952e3640 1 --2- 172.21.15.43:0/2399222578 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f98900ce4c0 0x7f98901ed490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T15:11:02.486 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:02.483+0000 7f98952e3640 1 -- 172.21.15.43:0/2399222578 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f98900ca370 msgr2=0x7f9890156b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T15:11:02.487 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:02.483+0000 7f98952e3640 1 --2- 172.21.15.43:0/2399222578 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f98900ca370 0x7f9890156b70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T15:11:02.487 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:02.483+0000 7f98952e3640 1 -- 172.21.15.43:0/2399222578 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9888009b30 con 0x7f989006fb70 2023-08-09T15:11:02.487 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:02.484+0000 7f98952e3640 1 --2- 172.21.15.43:0/2399222578 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f989006fb70 0x7f9890156630 secure :-1 s=READY pgs=2271 cs=0 l=1 rev1=1 crypto rx=0x7f988c00d800 tx=0x7f988c00dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-09T15:11:02.487 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:02.484+0000 7f9885ffb640 1 -- 172.21.15.43:0/2399222578 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f988c00b6c0 con 0x7f989006fb70 2023-08-09T15:11:02.487 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:02.484+0000 7f9885ffb640 1 -- 172.21.15.43:0/2399222578 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f988c00b820 con 0x7f989006fb70 2023-08-09T15:11:02.487 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:02.484+0000 7f9885ffb640 1 -- 172.21.15.43:0/2399222578 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f988c00bbb0 con 0x7f989006fb70 2023-08-09T15:11:02.487 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:02.484+0000 7f9897547640 1 -- 172.21.15.43:0/2399222578 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f98901ee660 con 0x7f989006fb70 2023-08-09T15:11:02.487 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:02.484+0000 7f9897547640 1 -- 172.21.15.43:0/2399222578 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f98900b39f0 con 0x7f989006fb70 2023-08-09T15:11:02.488 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:02.485+0000 7f9897547640 1 -- 172.21.15.43:0/2399222578 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9890070470 con 0x7f989006fb70 2023-08-09T15:11:02.488 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:02.486+0000 7f9885ffb640 1 -- 172.21.15.43:0/2399222578 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 113) v1 ==== 88472+0+0 (secure 0 0 0) 0x7f988c00bd10 con 0x7f989006fb70 2023-08-09T15:11:02.489 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:02.487+0000 7f9885ffb640 1 --2- 172.21.15.43:0/2399222578 >> [v2:172.21.15.203:6832/4012896436,v1:172.21.15.203:6833/4012896436] conn(0x7f987406a1c0 0x7f987406c680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T15:11:02.489 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:02.487+0000 7f9885ffb640 1 -- 172.21.15.43:0/2399222578 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(208..208 src has 1..208) v4 ==== 6828+0+0 (secure 0 0 0) 0x7f988c088ff0 con 0x7f989006fb70 2023-08-09T15:11:02.489 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:02.488+0000 7f9894ae2640 1 --2- 172.21.15.43:0/2399222578 >> [v2:172.21.15.203:6832/4012896436,v1:172.21.15.203:6833/4012896436] conn(0x7f987406a1c0 0x7f987406c680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T15:11:02.490 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:02.489+0000 7f9894ae2640 1 --2- 172.21.15.43:0/2399222578 >> [v2:172.21.15.203:6832/4012896436,v1:172.21.15.203:6833/4012896436] conn(0x7f987406a1c0 0x7f987406c680 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7f98901564d0 tx=0x7f9880009040 comp rx=0 tx=0).ready entity=mgr.7763 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-09T15:11:02.493 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:02.492+0000 7f9885ffb640 1 -- 172.21.15.43:0/2399222578 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178066 (secure 0 0 0) 0x7f988c0bb7b0 con 0x7f989006fb70 2023-08-09T15:11:02.711 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:02.710+0000 7f9897547640 1 -- 172.21.15.43:0/2399222578 --> [v2:172.21.15.203:6832/4012896436,v1:172.21.15.203:6833/4012896436] -- mgr_command(tid 0: {"prefix": "fs snap-schedule remove", "fs": "cephfs", "path": "snap_test_dir1", "target": ["mon-mgr", ""]}) v1 -- 0x7f98900cb320 con 0x7f987406a1c0 2023-08-09T15:11:02.880 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:02.879+0000 7f9885ffb640 1 -- 172.21.15.43:0/2399222578 <== mgr.7763 v2:172.21.15.203:6832/4012896436 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+41 (secure 0 0 0) 0x7f9880003980 con 0x7f987406a1c0 2023-08-09T15:11:02.881 INFO:teuthology.orchestra.run.smithi043.stdout:Schedule removed for path snap_test_dir1 2023-08-09T15:11:02.884 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:02.882+0000 7f9897547640 1 -- 172.21.15.43:0/2399222578 >> [v2:172.21.15.203:6832/4012896436,v1:172.21.15.203:6833/4012896436] conn(0x7f987406a1c0 msgr2=0x7f987406c680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T15:11:02.884 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:02.882+0000 7f9897547640 1 --2- 172.21.15.43:0/2399222578 >> [v2:172.21.15.203:6832/4012896436,v1:172.21.15.203:6833/4012896436] conn(0x7f987406a1c0 0x7f987406c680 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7f98901564d0 tx=0x7f9880009040 comp rx=0 tx=0).stop 2023-08-09T15:11:02.884 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:02.882+0000 7f9897547640 1 -- 172.21.15.43:0/2399222578 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f989006fb70 msgr2=0x7f9890156630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T15:11:02.884 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:02.882+0000 7f9897547640 1 --2- 172.21.15.43:0/2399222578 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f989006fb70 0x7f9890156630 secure :-1 s=READY pgs=2271 cs=0 l=1 rev1=1 crypto rx=0x7f988c00d800 tx=0x7f988c00dcc0 comp rx=0 tx=0).stop 2023-08-09T15:11:02.885 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:02.883+0000 7f9897547640 1 -- 172.21.15.43:0/2399222578 shutdown_connections 2023-08-09T15:11:02.885 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:02.883+0000 7f9897547640 1 --2- 172.21.15.43:0/2399222578 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f98900ce4c0 0x7f98901ed490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T15:11:02.885 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:02.883+0000 7f9897547640 1 --2- 172.21.15.43:0/2399222578 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f98900ca370 0x7f9890156b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T15:11:02.885 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:02.883+0000 7f9897547640 1 --2- 172.21.15.43:0/2399222578 >> [v2:172.21.15.203:6832/4012896436,v1:172.21.15.203:6833/4012896436] conn(0x7f987406a1c0 0x7f987406c680 unknown :-1 s=CLOSED pgs=26 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T15:11:02.885 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:02.883+0000 7f9897547640 1 --2- 172.21.15.43:0/2399222578 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f989006fb70 0x7f9890156630 unknown :-1 s=CLOSED pgs=2271 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T15:11:02.886 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:02.883+0000 7f9897547640 1 -- 172.21.15.43:0/2399222578 >> 172.21.15.43:0/2399222578 conn(0x7f989006b990 msgr2=0x7f989006bd70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-09T15:11:02.886 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:02.883+0000 7f9897547640 1 -- 172.21.15.43:0/2399222578 shutdown_connections 2023-08-09T15:11:02.886 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:02.883+0000 7f9897547640 1 -- 172.21.15.43:0/2399222578 wait complete. 2023-08-09T15:11:02.906 DEBUG:tasks.cephfs.test_snap_schedules:res=Schedule removed for path snap_test_dir1 2023-08-09T15:11:02.907 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-09T15:11:02.908 DEBUG:teuthology.orchestra.run.smithi043:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-08-09T15:11:02.970 INFO:teuthology.orchestra.run.smithi043.stdout:scheduled-2023-08-09-15_11_00_UTC 2023-08-09T15:11:02.971 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/.snap/scheduled-2023-08-09-15_11_00_UTC 2023-08-09T15:11:02.972 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-09T15:11:02.973 DEBUG:teuthology.orchestra.run.smithi043:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/.snap/scheduled-2023-08-09-15_11_00_UTC) 2023-08-09T15:11:03.044 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-09T15:11:03.045 DEBUG:teuthology.orchestra.run.smithi043:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1) 2023-08-09T15:11:03.105 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph mgr module disable snap_schedule 2023-08-09T15:11:03.329 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:03.328+0000 7f74971e1640 1 Processor -- start 2023-08-09T15:11:03.330 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:03.328+0000 7f74971e1640 1 -- start start 2023-08-09T15:11:03.331 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:03.330+0000 7f74971e1640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f7490142ca0 0x7f74900c96e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T15:11:03.331 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:03.330+0000 7f74971e1640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f74900cd510 0x7f74900c3290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T15:11:03.332 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:03.330+0000 7f74971e1640 1 -- --> v1:172.21.15.43:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7490143080 con 0x7f749015ce40 2023-08-09T15:11:03.332 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:03.330+0000 7f74971e1640 1 -- --> v2:172.21.15.43:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f749006d400 con 0x7f74900cd510 2023-08-09T15:11:03.332 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:03.330+0000 7f74971e1640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f74900c9d90 con 0x7f7490142ca0 2023-08-09T15:11:03.333 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:03.330+0000 7f749577e640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f74900cd510 0x7f74900c3290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T15:11:03.333 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:03.330+0000 7f749577e640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f74900cd510 0x7f74900c3290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:53968/0 (socket says 172.21.15.43:53968) 2023-08-09T15:11:03.333 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:03.330+0000 7f749577e640 1 -- 172.21.15.43:0/1759046612 learned_addr learned my addr 172.21.15.43:0/1759046612 (peer_addr_for_me v2:172.21.15.43:0/0) 2023-08-09T15:11:03.333 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:03.330+0000 7f7487fff640 1 --1- 172.21.15.43:0/1759046612 >> v1:172.21.15.43:6789/0 conn(0x7f749015ce40 0x7f74900cce00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.43:6789/0 says I am v1:172.21.15.43:42824/0 (socket says 172.21.15.43:42824) 2023-08-09T15:11:03.333 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:03.331+0000 7f7494f7d640 1 --2- 172.21.15.43:0/1759046612 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f7490142ca0 0x7f74900c96e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T15:11:03.333 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:03.331+0000 7f749577e640 1 -- 172.21.15.43:0/1759046612 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f7490142ca0 msgr2=0x7f74900c96e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-09T15:11:03.334 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:03.331+0000 7f749577e640 1 --2- 172.21.15.43:0/1759046612 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f7490142ca0 0x7f74900c96e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T15:11:03.334 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:03.331+0000 7f749577e640 1 -- 172.21.15.43:0/1759046612 >> v1:172.21.15.43:6789/0 conn(0x7f749015ce40 legacy=0x7f74900cce00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T15:11:03.334 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:03.331+0000 7f749577e640 1 -- 172.21.15.43:0/1759046612 --> v2:172.21.15.43:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f74900ca070 con 0x7f74900cd510 2023-08-09T15:11:03.334 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:03.331+0000 7f74877fe640 1 -- 172.21.15.43:0/1759046612 <== mon.0 v1:172.21.15.43:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3115076472 0 0) 0x7f7490143080 con 0x7f749015ce40 2023-08-09T15:11:03.335 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:03.332+0000 7f749577e640 1 --2- 172.21.15.43:0/1759046612 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f74900cd510 0x7f74900c3290 secure :-1 s=READY pgs=2273 cs=0 l=1 rev1=1 crypto rx=0x7f7488009870 tx=0x7f748802b980 comp rx=0 tx=0).ready entity=mon.0 client_cookie=867f41c27f71df3a server_cookie=0 in_seq=0 out_seq=0 2023-08-09T15:11:03.335 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:03.332+0000 7f74877fe640 1 -- 172.21.15.43:0/1759046612 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7488030840 con 0x7f74900cd510 2023-08-09T15:11:03.335 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:03.332+0000 7f74877fe640 1 -- 172.21.15.43:0/1759046612 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f74880309a0 con 0x7f74900cd510 2023-08-09T15:11:03.335 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:03.332+0000 7f74877fe640 1 -- 172.21.15.43:0/1759046612 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7488030d30 con 0x7f74900cd510 2023-08-09T15:11:03.336 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:03.333+0000 7f74971e1640 1 -- 172.21.15.43:0/1759046612 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f74900cd510 msgr2=0x7f74900c3290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T15:11:03.336 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:03.333+0000 7f74971e1640 1 --2- 172.21.15.43:0/1759046612 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f74900cd510 0x7f74900c3290 secure :-1 s=READY pgs=2273 cs=0 l=1 rev1=1 crypto rx=0x7f7488009870 tx=0x7f748802b980 comp rx=0 tx=0).stop 2023-08-09T15:11:03.336 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:03.333+0000 7f74971e1640 1 -- 172.21.15.43:0/1759046612 shutdown_connections 2023-08-09T15:11:03.336 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:03.333+0000 7f74971e1640 1 --2- 172.21.15.43:0/1759046612 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f74900cd510 0x7f74900c3290 unknown :-1 s=CLOSED pgs=2273 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T15:11:03.336 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:03.333+0000 7f74971e1640 1 --2- 172.21.15.43:0/1759046612 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f7490142ca0 0x7f74900c96e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T15:11:03.336 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:03.333+0000 7f74971e1640 1 -- 172.21.15.43:0/1759046612 >> 172.21.15.43:0/1759046612 conn(0x7f74900ce0a0 msgr2=0x7f74900ce480 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-09T15:11:03.336 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:03.333+0000 7f74971e1640 1 -- 172.21.15.43:0/1759046612 shutdown_connections 2023-08-09T15:11:03.336 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:03.333+0000 7f74971e1640 1 -- 172.21.15.43:0/1759046612 wait complete. 2023-08-09T15:11:03.337 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:03.335+0000 7f74971e1640 1 Processor -- start 2023-08-09T15:11:03.338 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:03.335+0000 7f74971e1640 1 -- start start 2023-08-09T15:11:03.338 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:03.336+0000 7f74971e1640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f74900cd510 0x7f7490158a90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T15:11:03.338 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:03.336+0000 7f74971e1640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f7490142ca0 0x7f7490159900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T15:11:03.338 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:03.336+0000 7f74971e1640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f749015ce40 0x7f7490159e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T15:11:03.338 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:03.336+0000 7f74971e1640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f74901420c0 con 0x7f74900cd510 2023-08-09T15:11:03.338 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:03.336+0000 7f74971e1640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f74901423a0 con 0x7f7490142ca0 2023-08-09T15:11:03.338 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:03.336+0000 7f74971e1640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f749015a710 con 0x7f749015ce40 2023-08-09T15:11:03.338 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:03.336+0000 7f7494f7d640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f74900cd510 0x7f7490158a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T15:11:03.339 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:03.336+0000 7f7494f7d640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f74900cd510 0x7f7490158a90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:53976/0 (socket says 172.21.15.43:53976) 2023-08-09T15:11:03.339 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:03.336+0000 7f7494f7d640 1 -- 172.21.15.43:0/3757077694 learned_addr learned my addr 172.21.15.43:0/3757077694 (peer_addr_for_me v2:172.21.15.43:0/0) 2023-08-09T15:11:03.340 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:03.337+0000 7f7487fff640 1 --2- 172.21.15.43:0/3757077694 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f7490142ca0 0x7f7490159900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T15:11:03.340 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:03.337+0000 7f7494f7d640 1 -- 172.21.15.43:0/3757077694 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f749015ce40 msgr2=0x7f7490159e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T15:11:03.340 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:03.337+0000 7f7494f7d640 1 --2- 172.21.15.43:0/3757077694 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f749015ce40 0x7f7490159e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T15:11:03.340 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:03.337+0000 7f7494f7d640 1 -- 172.21.15.43:0/3757077694 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f7490142ca0 msgr2=0x7f7490159900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T15:11:03.340 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:03.337+0000 7f7494f7d640 1 --2- 172.21.15.43:0/3757077694 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f7490142ca0 0x7f7490159900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T15:11:03.340 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:03.337+0000 7f7494f7d640 1 -- 172.21.15.43:0/3757077694 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f74880094d0 con 0x7f74900cd510 2023-08-09T15:11:03.341 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:03.337+0000 7f7494f7d640 1 --2- 172.21.15.43:0/3757077694 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f74900cd510 0x7f7490158a90 secure :-1 s=READY pgs=2274 cs=0 l=1 rev1=1 crypto rx=0x7f748c007ef0 tx=0x7f748c00e4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-09T15:11:03.341 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:03.337+0000 7f74857fa640 1 -- 172.21.15.43:0/3757077694 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f748c014620 con 0x7f74900cd510 2023-08-09T15:11:03.341 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:03.337+0000 7f74857fa640 1 -- 172.21.15.43:0/3757077694 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f748c014780 con 0x7f74900cd510 2023-08-09T15:11:03.341 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:03.337+0000 7f74857fa640 1 -- 172.21.15.43:0/3757077694 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f748c014b30 con 0x7f74900cd510 2023-08-09T15:11:03.341 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:03.337+0000 7f74971e1640 1 -- 172.21.15.43:0/3757077694 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f749015a9f0 con 0x7f74900cd510 2023-08-09T15:11:03.341 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:03.338+0000 7f74971e1640 1 -- 172.21.15.43:0/3757077694 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7490158fd0 con 0x7f74900cd510 2023-08-09T15:11:03.341 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:03.339+0000 7f74971e1640 1 -- 172.21.15.43:0/3757077694 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f749015a560 con 0x7f74900cd510 2023-08-09T15:11:03.342 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:03.340+0000 7f74857fa640 1 -- 172.21.15.43:0/3757077694 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 113) v1 ==== 88472+0+0 (secure 0 0 0) 0x7f748c014c90 con 0x7f74900cd510 2023-08-09T15:11:03.342 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:03.341+0000 7f74857fa640 1 --2- 172.21.15.43:0/3757077694 >> [v2:172.21.15.203:6832/4012896436,v1:172.21.15.203:6833/4012896436] conn(0x7f747c06a1c0 0x7f747c06c680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T15:11:03.342 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:03.341+0000 7f74857fa640 1 -- 172.21.15.43:0/3757077694 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(208..208 src has 1..208) v4 ==== 6828+0+0 (secure 0 0 0) 0x7f748c0894f0 con 0x7f74900cd510 2023-08-09T15:11:03.343 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:03.342+0000 7f7487fff640 1 --2- 172.21.15.43:0/3757077694 >> [v2:172.21.15.203:6832/4012896436,v1:172.21.15.203:6833/4012896436] conn(0x7f747c06a1c0 0x7f747c06c680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T15:11:03.344 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:03.343+0000 7f7487fff640 1 --2- 172.21.15.43:0/3757077694 >> [v2:172.21.15.203:6832/4012896436,v1:172.21.15.203:6833/4012896436] conn(0x7f747c06a1c0 0x7f747c06c680 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7f74900afc80 tx=0x7f7480009040 comp rx=0 tx=0).ready entity=mgr.7763 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-09T15:11:03.347 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:03.346+0000 7f74857fa640 1 -- 172.21.15.43:0/3757077694 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178066 (secure 0 0 0) 0x7f748c0547d0 con 0x7f74900cd510 2023-08-09T15:11:03.597 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:03.596+0000 7f74971e1640 1 -- 172.21.15.43:0/3757077694 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "mgr module disable", "module": "snap_schedule"} v 0) v1 -- 0x7f74900cb0b0 con 0x7f74900cd510 2023-08-09T15:11:04.177 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:04.175+0000 7f74857fa640 1 -- 172.21.15.43:0/3757077694 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mgrmap(e 114) v1 ==== 88455+0+0 (secure 0 0 0) 0x7f748c0875b0 con 0x7f74900cd510 2023-08-09T15:11:04.179 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:04.177+0000 7f74857fa640 1 -- 172.21.15.43:0/3757077694 <== mon.0 v2:172.21.15.43:3300/0 8 ==== mon_command_ack([{"prefix": "mgr module disable", "module": "snap_schedule"}]=0 v114) v1 ==== 93+0+0 (secure 0 0 0) 0x7f748c06f820 con 0x7f74900cd510 2023-08-09T15:11:04.183 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:04.181+0000 7f74971e1640 1 -- 172.21.15.43:0/3757077694 >> [v2:172.21.15.203:6832/4012896436,v1:172.21.15.203:6833/4012896436] conn(0x7f747c06a1c0 msgr2=0x7f747c06c680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T15:11:04.183 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:04.181+0000 7f74971e1640 1 --2- 172.21.15.43:0/3757077694 >> [v2:172.21.15.203:6832/4012896436,v1:172.21.15.203:6833/4012896436] conn(0x7f747c06a1c0 0x7f747c06c680 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7f74900afc80 tx=0x7f7480009040 comp rx=0 tx=0).stop 2023-08-09T15:11:04.183 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:04.181+0000 7f74971e1640 1 -- 172.21.15.43:0/3757077694 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f74900cd510 msgr2=0x7f7490158a90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T15:11:04.183 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:04.181+0000 7f74971e1640 1 --2- 172.21.15.43:0/3757077694 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f74900cd510 0x7f7490158a90 secure :-1 s=READY pgs=2274 cs=0 l=1 rev1=1 crypto rx=0x7f748c007ef0 tx=0x7f748c00e4b0 comp rx=0 tx=0).stop 2023-08-09T15:11:04.183 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:04.181+0000 7f74971e1640 1 -- 172.21.15.43:0/3757077694 shutdown_connections 2023-08-09T15:11:04.184 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:04.181+0000 7f74971e1640 1 --2- 172.21.15.43:0/3757077694 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f749015ce40 0x7f7490159e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T15:11:04.184 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:04.181+0000 7f74971e1640 1 --2- 172.21.15.43:0/3757077694 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f7490142ca0 0x7f7490159900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T15:11:04.184 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:04.181+0000 7f74971e1640 1 --2- 172.21.15.43:0/3757077694 >> [v2:172.21.15.203:6832/4012896436,v1:172.21.15.203:6833/4012896436] conn(0x7f747c06a1c0 0x7f747c06c680 unknown :-1 s=CLOSED pgs=27 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T15:11:04.184 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:04.181+0000 7f74971e1640 1 --2- 172.21.15.43:0/3757077694 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f74900cd510 0x7f7490158a90 unknown :-1 s=CLOSED pgs=2274 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T15:11:04.184 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:04.181+0000 7f74971e1640 1 -- 172.21.15.43:0/3757077694 >> 172.21.15.43:0/3757077694 conn(0x7f74900ce0a0 msgr2=0x7f74900ca9a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-09T15:11:04.184 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:04.181+0000 7f74971e1640 1 -- 172.21.15.43:0/3757077694 shutdown_connections 2023-08-09T15:11:04.184 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:04.181+0000 7f74971e1640 1 -- 172.21.15.43:0/3757077694 wait complete. 2023-08-09T15:11:04.202 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2023-08-09T15:11:04.203 DEBUG:teuthology.orchestra.run.smithi043:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2023-08-09T15:11:04.313 DEBUG:teuthology.orchestra.run.smithi203:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2023-08-09T15:11:04.405 INFO:tasks.ceph.mgr.x.smithi203.stderr:2023-08-09T15:11:04.405+0000 7fc2cf013040 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2023-08-09T15:11:04.409 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2023-08-09T15:11:04.410 DEBUG:teuthology.orchestra.run.smithi043:> set -ex 2023-08-09T15:11:04.410 DEBUG:teuthology.orchestra.run.smithi043:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-09T15:11:04.423 INFO:tasks.ceph.mgr.y.smithi043.stderr:2023-08-09T15:11:04.422+0000 7fb51ee0f040 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2023-08-09T15:11:04.469 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-08-09T15:11:04.731 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:04.730+0000 7ff53ea2f640 1 Processor -- start 2023-08-09T15:11:04.732 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:04.730+0000 7ff53ea2f640 1 -- start start 2023-08-09T15:11:04.733 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:04.731+0000 7ff53ea2f640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7ff5380ce720 0x7ff5380ceb00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T15:11:04.733 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:04.731+0000 7ff53ea2f640 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7ff53815ecc0 0x7ff53806d2a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T15:11:04.733 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:04.731+0000 7ff53ea2f640 1 -- --> v1:172.21.15.43:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff5380cd8a0 con 0x7ff5380ce080 2023-08-09T15:11:04.733 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:04.731+0000 7ff53ea2f640 1 -- --> v2:172.21.15.43:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff5380cf350 con 0x7ff5380ce720 2023-08-09T15:11:04.733 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:04.731+0000 7ff53ea2f640 1 -- --> v2:172.21.15.203:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff5380cf040 con 0x7ff53815ecc0 2023-08-09T15:11:04.734 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:04.732+0000 7ff537fff640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7ff5380ce720 0x7ff5380ceb00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T15:11:04.734 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:04.732+0000 7ff5377fe640 1 --1- >> v1:172.21.15.43:6789/0 conn(0x7ff5380ce080 0x7ff53815e5b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.43:6789/0 says I am v1:172.21.15.43:42834/0 (socket says 172.21.15.43:42834) 2023-08-09T15:11:04.734 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:04.732+0000 7ff5377fe640 1 -- 172.21.15.43:0/3362186464 learned_addr learned my addr 172.21.15.43:0/3362186464 (peer_addr_for_me v1:172.21.15.43:0/0) 2023-08-09T15:11:04.735 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:04.732+0000 7ff53cfcc640 1 --2- 172.21.15.43:0/3362186464 >> v2:172.21.15.203:3300/0 conn(0x7ff53815ecc0 0x7ff53806d2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T15:11:04.735 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:04.732+0000 7ff537fff640 1 -- 172.21.15.43:0/3362186464 >> v2:172.21.15.203:3300/0 conn(0x7ff53815ecc0 msgr2=0x7ff53806d2a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-09T15:11:04.735 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:04.732+0000 7ff537fff640 1 --2- 172.21.15.43:0/3362186464 >> v2:172.21.15.203:3300/0 conn(0x7ff53815ecc0 0x7ff53806d2a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T15:11:04.735 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:04.732+0000 7ff537fff640 1 -- 172.21.15.43:0/3362186464 >> v1:172.21.15.43:6789/0 conn(0x7ff5380ce080 legacy=0x7ff53815e5b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T15:11:04.735 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:04.732+0000 7ff537fff640 1 -- 172.21.15.43:0/3362186464 --> v2:172.21.15.43:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff538071a70 con 0x7ff5380ce720 2023-08-09T15:11:04.735 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:04.732+0000 7ff536ffd640 1 -- 172.21.15.43:0/3362186464 <== mon.0 v1:172.21.15.43:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3276950569 0 0) 0x7ff5380cd8a0 con 0x7ff5380ce080 2023-08-09T15:11:04.735 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:04.732+0000 7ff53cfcc640 1 --2- 172.21.15.43:0/3362186464 >> v2:172.21.15.203:3300/0 conn(0x7ff53815ecc0 0x7ff53806d2a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-08-09T15:11:04.736 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:04.733+0000 7ff537fff640 1 --2- 172.21.15.43:0/3362186464 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff5380ce720 0x7ff5380ceb00 secure :-1 s=READY pgs=2279 cs=0 l=1 rev1=1 crypto rx=0x7ff5380cc010 tx=0x7ff52802b840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2be7965432e0aa9 server_cookie=0 in_seq=0 out_seq=0 2023-08-09T15:11:04.737 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:04.734+0000 7ff536ffd640 1 -- 172.21.15.43:0/3362186464 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff52802e070 con 0x7ff5380ce720 2023-08-09T15:11:04.737 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:04.734+0000 7ff536ffd640 1 -- 172.21.15.43:0/3362186464 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff5280315a0 con 0x7ff5380ce720 2023-08-09T15:11:04.737 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:04.734+0000 7ff536ffd640 1 -- 172.21.15.43:0/3362186464 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff528031930 con 0x7ff5380ce720 2023-08-09T15:11:04.737 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:04.734+0000 7ff53ea2f640 1 -- 172.21.15.43:0/3362186464 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff5380ce720 msgr2=0x7ff5380ceb00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T15:11:04.737 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:04.734+0000 7ff53ea2f640 1 --2- 172.21.15.43:0/3362186464 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff5380ce720 0x7ff5380ceb00 secure :-1 s=READY pgs=2279 cs=0 l=1 rev1=1 crypto rx=0x7ff5380cc010 tx=0x7ff52802b840 comp rx=0 tx=0).stop 2023-08-09T15:11:04.737 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:04.735+0000 7ff53ea2f640 1 -- 172.21.15.43:0/3362186464 shutdown_connections 2023-08-09T15:11:04.737 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:04.735+0000 7ff53ea2f640 1 --2- 172.21.15.43:0/3362186464 >> v2:172.21.15.203:3300/0 conn(0x7ff53815ecc0 0x7ff53806d2a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T15:11:04.738 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:04.735+0000 7ff53ea2f640 1 --2- 172.21.15.43:0/3362186464 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff5380ce720 0x7ff5380ceb00 unknown :-1 s=CLOSED pgs=2279 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T15:11:04.738 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:04.735+0000 7ff53ea2f640 1 -- 172.21.15.43:0/3362186464 >> 172.21.15.43:0/3362186464 conn(0x7ff5380ccd80 msgr2=0x7ff5380cd190 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-09T15:11:04.738 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:04.735+0000 7ff53ea2f640 1 -- 172.21.15.43:0/3362186464 shutdown_connections 2023-08-09T15:11:04.738 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:04.735+0000 7ff53ea2f640 1 -- 172.21.15.43:0/3362186464 wait complete. 2023-08-09T15:11:04.738 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:04.737+0000 7ff53ea2f640 1 Processor -- start 2023-08-09T15:11:04.739 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:04.737+0000 7ff53ea2f640 1 -- start start 2023-08-09T15:11:04.739 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:04.738+0000 7ff53ea2f640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7ff5380ce080 0x7ff53806c490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T15:11:04.740 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:04.738+0000 7ff53ea2f640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff5380ce720 0x7ff538158a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T15:11:04.740 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:04.738+0000 7ff53ea2f640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7ff53815ecc0 0x7ff5380b1f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T15:11:04.740 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:04.738+0000 7ff53ea2f640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff5380b4a70 con 0x7ff5380ce720 2023-08-09T15:11:04.740 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:04.738+0000 7ff53ea2f640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff5380b4d50 con 0x7ff5380ce080 2023-08-09T15:11:04.740 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:04.738+0000 7ff53ea2f640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff5380b5030 con 0x7ff53815ecc0 2023-08-09T15:11:04.740 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:04.738+0000 7ff5377fe640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff5380ce720 0x7ff538158a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T15:11:04.740 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:04.738+0000 7ff5377fe640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff5380ce720 0x7ff538158a40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:54004/0 (socket says 172.21.15.43:54004) 2023-08-09T15:11:04.740 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:04.738+0000 7ff5377fe640 1 -- 172.21.15.43:0/2884576531 learned_addr learned my addr 172.21.15.43:0/2884576531 (peer_addr_for_me v2:172.21.15.43:0/0) 2023-08-09T15:11:04.742 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:04.738+0000 7ff537fff640 1 --2- 172.21.15.43:0/2884576531 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7ff5380ce080 0x7ff53806c490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T15:11:04.742 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:04.739+0000 7ff53cfcc640 1 --2- 172.21.15.43:0/2884576531 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7ff53815ecc0 0x7ff5380b1f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T15:11:04.742 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:04.739+0000 7ff5377fe640 1 -- 172.21.15.43:0/2884576531 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7ff53815ecc0 msgr2=0x7ff5380b1f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T15:11:04.742 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:04.739+0000 7ff5377fe640 1 --2- 172.21.15.43:0/2884576531 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7ff53815ecc0 0x7ff5380b1f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T15:11:04.742 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:04.739+0000 7ff5377fe640 1 -- 172.21.15.43:0/2884576531 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7ff5380ce080 msgr2=0x7ff53806c490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T15:11:04.742 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:04.739+0000 7ff5377fe640 1 --2- 172.21.15.43:0/2884576531 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7ff5380ce080 0x7ff53806c490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T15:11:04.742 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:04.739+0000 7ff5377fe640 1 -- 172.21.15.43:0/2884576531 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff5280093a0 con 0x7ff5380ce720 2023-08-09T15:11:04.743 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:04.739+0000 7ff5377fe640 1 --2- 172.21.15.43:0/2884576531 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff5380ce720 0x7ff538158a40 secure :-1 s=READY pgs=2280 cs=0 l=1 rev1=1 crypto rx=0x7ff52c00c410 tx=0x7ff52c00c8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-09T15:11:04.743 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:04.739+0000 7ff534ff9640 1 -- 172.21.15.43:0/2884576531 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff52c0074e0 con 0x7ff5380ce720 2023-08-09T15:11:04.743 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:04.739+0000 7ff534ff9640 1 -- 172.21.15.43:0/2884576531 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff52c007660 con 0x7ff5380ce720 2023-08-09T15:11:04.743 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:04.739+0000 7ff534ff9640 1 -- 172.21.15.43:0/2884576531 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff52c004a20 con 0x7ff5380ce720 2023-08-09T15:11:04.743 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:04.739+0000 7ff53ea2f640 1 -- 172.21.15.43:0/2884576531 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff5380b5310 con 0x7ff5380ce720 2023-08-09T15:11:04.743 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:04.740+0000 7ff53ea2f640 1 -- 172.21.15.43:0/2884576531 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff538158f80 con 0x7ff5380ce720 2023-08-09T15:11:04.744 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:04.741+0000 7ff53ea2f640 1 -- 172.21.15.43:0/2884576531 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff5380739f0 con 0x7ff5380ce720 2023-08-09T15:11:04.744 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:04.742+0000 7ff534ff9640 1 -- 172.21.15.43:0/2884576531 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 114) v1 ==== 88455+0+0 (secure 0 0 0) 0x7ff52c004b80 con 0x7ff5380ce720 2023-08-09T15:11:04.744 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:04.743+0000 7ff534ff9640 1 --2- 172.21.15.43:0/2884576531 >> [v2:172.21.15.203:6832/4012896436,v1:172.21.15.203:6833/4012896436] conn(0x7ff52406a170 0x7ff52406c630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T15:11:04.745 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:04.743+0000 7ff534ff9640 1 -- 172.21.15.43:0/2884576531 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(208..208 src has 1..208) v4 ==== 6828+0+0 (secure 0 0 0) 0x7ff52c088890 con 0x7ff5380ce720 2023-08-09T15:11:04.745 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:04.743+0000 7ff537fff640 1 -- 172.21.15.43:0/2884576531 >> [v2:172.21.15.203:6832/4012896436,v1:172.21.15.203:6833/4012896436] conn(0x7ff52406a170 msgr2=0x7ff52406c630 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.203:6832/4012896436 2023-08-09T15:11:04.745 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:04.743+0000 7ff537fff640 1 --2- 172.21.15.43:0/2884576531 >> [v2:172.21.15.203:6832/4012896436,v1:172.21.15.203:6833/4012896436] conn(0x7ff52406a170 0x7ff52406c630 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-08-09T15:11:04.748 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:04.747+0000 7ff534ff9640 1 -- 172.21.15.43:0/2884576531 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178066 (secure 0 0 0) 0x7ff52c0575f0 con 0x7ff5380ce720 2023-08-09T15:11:04.939 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:04.937+0000 7ff53ea2f640 1 -- 172.21.15.43:0/2884576531 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7ff53806eee0 con 0x7ff5380ce720 2023-08-09T15:11:04.940 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:04.939+0000 7ff534ff9640 1 -- 172.21.15.43:0/2884576531 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v208) v1 ==== 74+0+19170 (secure 0 0 0) 0x7ff52c057180 con 0x7ff5380ce720 2023-08-09T15:11:04.941 INFO:teuthology.orchestra.run.smithi043.stdout: 2023-08-09T15:11:04.941 INFO:teuthology.orchestra.run.smithi043.stdout:{"epoch":208,"fsid":"daac87ee-1337-4250-a955-aea4e408a39b","created":"2023-08-09T13:33:41.701198+0000","modified":"2023-08-09T15:10:02.420083+0000","last_up_change":"2023-08-09T13:33:53.783453+0000","last_in_change":"2023-08-09T13:33:45.148904+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":25,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":"rbd","create_time":"2023-08-09T13:33:54.365137+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":".mgr","create_time":"2023-08-09T13:33:54.831803+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":24,"pool_name":"cephfs_metadata","create_time":"2023-08-09T15:08:43.920337+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"204","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":25,"pool_name":"cephfs_data","create_time":"2023-08-09T15:08:44.594489+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"205","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"cbd49ecd-e896-4fbd-864e-5c37a829b165","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":203,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6800","nonce":502667335},{"type":"v1","addr":"172.21.15.43:6801","nonce":502667335}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6802","nonce":502667335},{"type":"v1","addr":"172.21.15.43:6803","nonce":502667335}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6806","nonce":502667335},{"type":"v1","addr":"172.21.15.43:6807","nonce":502667335}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6804","nonce":502667335},{"type":"v1","addr":"172.21.15.43:6805","nonce":502667335}]},"public_addr":"172.21.15.43:6801/502667335","cluster_addr":"172.21.15.43:6803/502667335","heartbeat_back_addr":"172.21.15.43:6807/502667335","heartbeat_front_addr":"172.21.15.43:6805/502667335","state":["exists","up"]},{"osd":1,"uuid":"75c7e0fe-1ec7-4fe8-a2f2-3661a65b8500","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":203,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6808","nonce":3289134610},{"type":"v1","addr":"172.21.15.43:6809","nonce":3289134610}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6810","nonce":3289134610},{"type":"v1","addr":"172.21.15.43:6812","nonce":3289134610}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6821","nonce":3289134610},{"type":"v1","addr":"172.21.15.43:6824","nonce":3289134610}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6815","nonce":3289134610},{"type":"v1","addr":"172.21.15.43:6818","nonce":3289134610}]},"public_addr":"172.21.15.43:6809/3289134610","cluster_addr":"172.21.15.43:6812/3289134610","heartbeat_back_addr":"172.21.15.43:6824/3289134610","heartbeat_front_addr":"172.21.15.43:6818/3289134610","state":["exists","up"]},{"osd":2,"uuid":"bb5c41e9-7f5f-4125-9b01-3bb8bfc2ea83","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":203,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6811","nonce":4079709736},{"type":"v1","addr":"172.21.15.43:6814","nonce":4079709736}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6817","nonce":4079709736},{"type":"v1","addr":"172.21.15.43:6820","nonce":4079709736}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6829","nonce":4079709736},{"type":"v1","addr":"172.21.15.43:6831","nonce":4079709736}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6823","nonce":4079709736},{"type":"v1","addr":"172.21.15.43:6826","nonce":4079709736}]},"public_addr":"172.21.15.43:6814/4079709736","cluster_addr":"172.21.15.43:6820/4079709736","heartbeat_back_addr":"172.21.15.43:6831/4079709736","heartbeat_front_addr":"172.21.15.43:6826/4079709736","state":["exists","up"]},{"osd":3,"uuid":"bfe4c8b4-694a-465c-87e6-2b2aab852613","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":203,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6813","nonce":1709538759},{"type":"v1","addr":"172.21.15.43:6816","nonce":1709538759}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6819","nonce":1709538759},{"type":"v1","addr":"172.21.15.43:6822","nonce":1709538759}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6828","nonce":1709538759},{"type":"v1","addr":"172.21.15.43:6830","nonce":1709538759}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6825","nonce":1709538759},{"type":"v1","addr":"172.21.15.43:6827","nonce":1709538759}]},"public_addr":"172.21.15.43:6816/1709538759","cluster_addr":"172.21.15.43:6822/1709538759","heartbeat_back_addr":"172.21.15.43:6830/1709538759","heartbeat_front_addr":"172.21.15.43:6827/1709538759","state":["exists","up"]},{"osd":4,"uuid":"fa2e8a77-aba1-4756-853e-0d174a5bdeba","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":203,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6800","nonce":4196866481},{"type":"v1","addr":"172.21.15.203:6801","nonce":4196866481}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6802","nonce":4196866481},{"type":"v1","addr":"172.21.15.203:6803","nonce":4196866481}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6806","nonce":4196866481},{"type":"v1","addr":"172.21.15.203:6807","nonce":4196866481}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6804","nonce":4196866481},{"type":"v1","addr":"172.21.15.203:6805","nonce":4196866481}]},"public_addr":"172.21.15.203:6801/4196866481","cluster_addr":"172.21.15.203:6803/4196866481","heartbeat_back_addr":"172.21.15.203:6807/4196866481","heartbeat_front_addr":"172.21.15.203:6805/4196866481","state":["exists","up"]},{"osd":5,"uuid":"81811e4b-4d24-4f51-a82c-f1d700a2a00d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":207,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6816","nonce":3059290081},{"type":"v1","addr":"172.21.15.203:6817","nonce":3059290081}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6818","nonce":3059290081},{"type":"v1","addr":"172.21.15.203:6819","nonce":3059290081}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6822","nonce":3059290081},{"type":"v1","addr":"172.21.15.203:6823","nonce":3059290081}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6820","nonce":3059290081},{"type":"v1","addr":"172.21.15.203:6821","nonce":3059290081}]},"public_addr":"172.21.15.203:6817/3059290081","cluster_addr":"172.21.15.203:6819/3059290081","heartbeat_back_addr":"172.21.15.203:6823/3059290081","heartbeat_front_addr":"172.21.15.203:6821/3059290081","state":["exists","up"]},{"osd":6,"uuid":"307bac4d-6a5b-443d-af75-56d695d7674f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":203,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6824","nonce":512155569},{"type":"v1","addr":"172.21.15.203:6825","nonce":512155569}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6826","nonce":512155569},{"type":"v1","addr":"172.21.15.203:6827","nonce":512155569}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6830","nonce":512155569},{"type":"v1","addr":"172.21.15.203:6831","nonce":512155569}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6828","nonce":512155569},{"type":"v1","addr":"172.21.15.203:6829","nonce":512155569}]},"public_addr":"172.21.15.203:6825/512155569","cluster_addr":"172.21.15.203:6827/512155569","heartbeat_back_addr":"172.21.15.203:6831/512155569","heartbeat_front_addr":"172.21.15.203:6829/512155569","state":["exists","up"]},{"osd":7,"uuid":"a95dc7ca-9f1b-4295-af70-68a22a2d7d5e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":207,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6808","nonce":3331450438},{"type":"v1","addr":"172.21.15.203:6809","nonce":3331450438}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6810","nonce":3331450438},{"type":"v1","addr":"172.21.15.203:6811","nonce":3331450438}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6814","nonce":3331450438},{"type":"v1","addr":"172.21.15.203:6815","nonce":3331450438}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6812","nonce":3331450438},{"type":"v1","addr":"172.21.15.203:6813","nonce":3331450438}]},"public_addr":"172.21.15.203:6809/3331450438","cluster_addr":"172.21.15.203:6811/3331450438","heartbeat_back_addr":"172.21.15.203:6815/3331450438","heartbeat_front_addr":"172.21.15.203:6813/3331450438","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-09T13:33:50.547652+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-09T13:33:50.589469+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-09T13:33:50.561724+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-09T13:33:50.599650+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-09T13:33:50.411855+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-09T13:33:50.219681+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-09T13:33:50.504664+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-09T13:33:50.441248+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"25.f","mappings":[{"from":0,"to":1}]},{"pgid":"25.3f","mappings":[{"from":0,"to":2}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.203:6833/3082732769":"2023-08-10T15:09:01.285901+0000","172.21.15.203:6832/3082732769":"2023-08-10T15:09:01.285901+0000","172.21.15.203:0/1756781921":"2023-08-10T15:09:01.285901+0000","172.21.15.203:0/454071895":"2023-08-10T15:09:01.285901+0000","172.21.15.203:0/2854569557":"2023-08-10T15:09:01.285901+0000","172.21.15.203:0/1159944241":"2023-08-10T15:09:01.285901+0000","172.21.15.203:0/1645632393":"2023-08-10T15:09:01.285901+0000","172.21.15.203:0/840449585":"2023-08-10T15:09:01.285901+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":7,"snaps":[{"begin":104,"length":1},{"begin":221,"length":1}]}],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-08-09T15:11:04.943 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:04.941+0000 7ff53ea2f640 1 -- 172.21.15.43:0/2884576531 >> [v2:172.21.15.203:6832/4012896436,v1:172.21.15.203:6833/4012896436] conn(0x7ff52406a170 msgr2=0x7ff52406c630 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-08-09T15:11:04.944 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:04.942+0000 7ff53ea2f640 1 --2- 172.21.15.43:0/2884576531 >> [v2:172.21.15.203:6832/4012896436,v1:172.21.15.203:6833/4012896436] conn(0x7ff52406a170 0x7ff52406c630 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T15:11:04.944 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:04.942+0000 7ff53ea2f640 1 -- 172.21.15.43:0/2884576531 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff5380ce720 msgr2=0x7ff538158a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T15:11:04.944 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:04.942+0000 7ff53ea2f640 1 --2- 172.21.15.43:0/2884576531 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff5380ce720 0x7ff538158a40 secure :-1 s=READY pgs=2280 cs=0 l=1 rev1=1 crypto rx=0x7ff52c00c410 tx=0x7ff52c00c8d0 comp rx=0 tx=0).stop 2023-08-09T15:11:04.945 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:04.942+0000 7ff53ea2f640 1 -- 172.21.15.43:0/2884576531 shutdown_connections 2023-08-09T15:11:04.945 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:04.942+0000 7ff53ea2f640 1 --2- 172.21.15.43:0/2884576531 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7ff53815ecc0 0x7ff5380b1f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T15:11:04.945 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:04.942+0000 7ff53ea2f640 1 --2- 172.21.15.43:0/2884576531 >> [v2:172.21.15.203:6832/4012896436,v1:172.21.15.203:6833/4012896436] conn(0x7ff52406a170 0x7ff52406c630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T15:11:04.946 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:04.942+0000 7ff53ea2f640 1 --2- 172.21.15.43:0/2884576531 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff5380ce720 0x7ff538158a40 unknown :-1 s=CLOSED pgs=2280 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T15:11:04.946 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:04.942+0000 7ff53ea2f640 1 --2- 172.21.15.43:0/2884576531 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7ff5380ce080 0x7ff53806c490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T15:11:04.946 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:04.942+0000 7ff53ea2f640 1 -- 172.21.15.43:0/2884576531 >> 172.21.15.43:0/2884576531 conn(0x7ff5380ccd80 msgr2=0x7ff5381598e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-09T15:11:04.946 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:04.942+0000 7ff53ea2f640 1 -- 172.21.15.43:0/2884576531 shutdown_connections 2023-08-09T15:11:04.947 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:04.942+0000 7ff53ea2f640 1 -- 172.21.15.43:0/2884576531 wait complete. 2023-08-09T15:11:04.962 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': 'rbd', 'create_time': '2023-08-09T13:33:54.365137+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}, 'read_balance': {'score_acting': 2, 'score_stable': 2, 'optimal_score': 1, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': '.mgr', 'create_time': '2023-08-09T13:33:54.831803+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 24, 'pool_name': 'cephfs_metadata', 'create_time': '2023-08-09T15:08:43.920337+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '204', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 25, 'pool_name': 'cephfs_data', 'create_time': '2023-08-09T15:08:44.594489+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '205', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.3799999952316284, 'score_stable': 1.3799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.3799999952316284, 'raw_score_stable': 1.3799999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-08-09T15:11:04.963 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get rbd pg_num 2023-08-09T15:11:05.120 INFO:tasks.ceph.mgr.x.smithi203.stderr:2023-08-09T15:11:05.120+0000 7fc2cf013040 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2023-08-09T15:11:05.182 INFO:tasks.ceph.mgr.y.smithi043.stderr:2023-08-09T15:11:05.181+0000 7fb51ee0f040 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2023-08-09T15:11:05.184 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:05.183+0000 7f68a0234640 1 Processor -- start 2023-08-09T15:11:05.184 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:05.183+0000 7f68a0234640 1 -- start start 2023-08-09T15:11:05.186 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:05.185+0000 7f68a0234640 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7f689806e920 0x7f6898164ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T15:11:05.186 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:05.185+0000 7f68a0234640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f68980cfd90 0x7f68980ce140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T15:11:05.186 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:05.185+0000 7f68a0234640 1 -- --> v1:172.21.15.43:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f68981655d0 con 0x7f6898160c00 2023-08-09T15:11:05.187 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:05.185+0000 7f68a0234640 1 -- --> v2:172.21.15.43:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f689806db60 con 0x7f68980cfd90 2023-08-09T15:11:05.187 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:05.185+0000 7f68a0234640 1 -- --> v2:172.21.15.203:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f689806de40 con 0x7f689806e920 2023-08-09T15:11:05.187 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:05.185+0000 7f689dfd0640 1 --1- >> v1:172.21.15.43:6789/0 conn(0x7f6898160c00 0x7f68981647c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.43:6789/0 says I am v1:172.21.15.43:45524/0 (socket says 172.21.15.43:45524) 2023-08-09T15:11:05.188 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:05.185+0000 7f689dfd0640 1 -- 172.21.15.43:0/2759719838 learned_addr learned my addr 172.21.15.43:0/2759719838 (peer_addr_for_me v1:172.21.15.43:0/0) 2023-08-09T15:11:05.188 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:05.185+0000 7f689e7d1640 1 --2- 172.21.15.43:0/2759719838 >> v2:172.21.15.43:3300/0 conn(0x7f68980cfd90 0x7f68980ce140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T15:11:05.188 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:05.186+0000 7f689d7cf640 1 --2- 172.21.15.43:0/2759719838 >> v2:172.21.15.203:3300/0 conn(0x7f689806e920 0x7f6898164ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T15:11:05.188 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:05.186+0000 7f689e7d1640 1 -- 172.21.15.43:0/2759719838 >> v2:172.21.15.203:3300/0 conn(0x7f689806e920 msgr2=0x7f6898164ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-09T15:11:05.188 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:05.186+0000 7f689e7d1640 1 --2- 172.21.15.43:0/2759719838 >> v2:172.21.15.203:3300/0 conn(0x7f689806e920 0x7f6898164ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T15:11:05.188 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:05.186+0000 7f689e7d1640 1 -- 172.21.15.43:0/2759719838 >> v1:172.21.15.43:6789/0 conn(0x7f6898160c00 legacy=0x7f68981647c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T15:11:05.188 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:05.186+0000 7f689e7d1640 1 -- 172.21.15.43:0/2759719838 --> v2:172.21.15.43:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f689806e120 con 0x7f68980cfd90 2023-08-09T15:11:05.189 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:05.187+0000 7f689e7d1640 1 --2- 172.21.15.43:0/2759719838 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f68980cfd90 0x7f68980ce140 secure :-1 s=READY pgs=2282 cs=0 l=1 rev1=1 crypto rx=0x7f689806d310 tx=0x7f689002bec0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=84a72126de489c3d server_cookie=0 in_seq=0 out_seq=0 2023-08-09T15:11:05.189 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:05.188+0000 7f689cfce640 1 -- 172.21.15.43:0/2759719838 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f689002cc70 con 0x7f68980cfd90 2023-08-09T15:11:05.189 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:05.188+0000 7f689cfce640 1 -- 172.21.15.43:0/2759719838 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f689002cdd0 con 0x7f68980cfd90 2023-08-09T15:11:05.190 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:05.188+0000 7f689cfce640 1 -- 172.21.15.43:0/2759719838 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6890034660 con 0x7f68980cfd90 2023-08-09T15:11:05.190 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:05.188+0000 7f68a0234640 1 -- 172.21.15.43:0/2759719838 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f68980cfd90 msgr2=0x7f68980ce140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T15:11:05.191 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:05.188+0000 7f68a0234640 1 --2- 172.21.15.43:0/2759719838 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f68980cfd90 0x7f68980ce140 secure :-1 s=READY pgs=2282 cs=0 l=1 rev1=1 crypto rx=0x7f689806d310 tx=0x7f689002bec0 comp rx=0 tx=0).stop 2023-08-09T15:11:05.191 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:05.189+0000 7f68a0234640 1 -- 172.21.15.43:0/2759719838 shutdown_connections 2023-08-09T15:11:05.191 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:05.189+0000 7f68a0234640 1 --2- 172.21.15.43:0/2759719838 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f68980cfd90 0x7f68980ce140 unknown :-1 s=CLOSED pgs=2282 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T15:11:05.191 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:05.189+0000 7f68a0234640 1 --2- 172.21.15.43:0/2759719838 >> v2:172.21.15.203:3300/0 conn(0x7f689806e920 0x7f6898164ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T15:11:05.191 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:05.189+0000 7f68a0234640 1 -- 172.21.15.43:0/2759719838 >> 172.21.15.43:0/2759719838 conn(0x7f68980b1dd0 msgr2=0x7f68980b21e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-09T15:11:05.191 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:05.189+0000 7f68a0234640 1 -- 172.21.15.43:0/2759719838 shutdown_connections 2023-08-09T15:11:05.191 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:05.189+0000 7f68a0234640 1 -- 172.21.15.43:0/2759719838 wait complete. 2023-08-09T15:11:05.192 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:05.190+0000 7f68a0234640 1 Processor -- start 2023-08-09T15:11:05.192 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:05.191+0000 7f68a0234640 1 -- start start 2023-08-09T15:11:05.193 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:05.191+0000 7f68a0234640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f689806e920 0x7f68980c4780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T15:11:05.193 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:05.191+0000 7f68a0234640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f68980cfd90 0x7f6898158b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T15:11:05.193 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:05.191+0000 7f68a0234640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f6898160c00 0x7f68980c6010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T15:11:05.193 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:05.191+0000 7f68a0234640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f68980b5ea0 con 0x7f6898160c00 2023-08-09T15:11:05.193 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:05.192+0000 7f68a0234640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f68980c4cc0 con 0x7f689806e920 2023-08-09T15:11:05.193 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:05.192+0000 7f68a0234640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f68980c7310 con 0x7f68980cfd90 2023-08-09T15:11:05.195 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:05.192+0000 7f689e7d1640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f6898160c00 0x7f68980c6010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T15:11:05.195 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:05.192+0000 7f689e7d1640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f6898160c00 0x7f68980c6010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:46396/0 (socket says 172.21.15.43:46396) 2023-08-09T15:11:05.195 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:05.192+0000 7f689e7d1640 1 -- 172.21.15.43:0/3432845783 learned_addr learned my addr 172.21.15.43:0/3432845783 (peer_addr_for_me v2:172.21.15.43:0/0) 2023-08-09T15:11:05.195 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:05.192+0000 7f689d7cf640 1 --2- 172.21.15.43:0/3432845783 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f68980cfd90 0x7f6898158b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T15:11:05.195 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:05.192+0000 7f689dfd0640 1 --2- 172.21.15.43:0/3432845783 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f689806e920 0x7f68980c4780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T15:11:05.195 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:05.192+0000 7f689e7d1640 1 -- 172.21.15.43:0/3432845783 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f68980cfd90 msgr2=0x7f6898158b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T15:11:05.196 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:05.192+0000 7f689e7d1640 1 --2- 172.21.15.43:0/3432845783 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f68980cfd90 0x7f6898158b60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T15:11:05.196 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:05.192+0000 7f689e7d1640 1 -- 172.21.15.43:0/3432845783 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f689806e920 msgr2=0x7f68980c4780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T15:11:05.196 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:05.192+0000 7f689e7d1640 1 --2- 172.21.15.43:0/3432845783 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f689806e920 0x7f68980c4780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T15:11:05.196 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:05.192+0000 7f689e7d1640 1 -- 172.21.15.43:0/3432845783 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6890009a70 con 0x7f6898160c00 2023-08-09T15:11:05.196 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:05.193+0000 7f689e7d1640 1 --2- 172.21.15.43:0/3432845783 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f6898160c00 0x7f68980c6010 secure :-1 s=READY pgs=2283 cs=0 l=1 rev1=1 crypto rx=0x7f689002c4d0 tx=0x7f6890004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-09T15:11:05.196 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:05.193+0000 7f688e7fc640 1 -- 172.21.15.43:0/3432845783 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6890040650 con 0x7f6898160c00 2023-08-09T15:11:05.196 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:05.193+0000 7f688e7fc640 1 -- 172.21.15.43:0/3432845783 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f68900407d0 con 0x7f6898160c00 2023-08-09T15:11:05.197 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:05.193+0000 7f68a0234640 1 -- 172.21.15.43:0/3432845783 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f68980c54b0 con 0x7f6898160c00 2023-08-09T15:11:05.197 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:05.193+0000 7f688e7fc640 1 -- 172.21.15.43:0/3432845783 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6890040b80 con 0x7f6898160c00 2023-08-09T15:11:05.197 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:05.193+0000 7f68a0234640 1 -- 172.21.15.43:0/3432845783 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f68980c4ef0 con 0x7f6898160c00 2023-08-09T15:11:05.197 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:05.195+0000 7f68a0234640 1 -- 172.21.15.43:0/3432845783 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f68980739f0 con 0x7f6898160c00 2023-08-09T15:11:05.197 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:05.195+0000 7f688e7fc640 1 -- 172.21.15.43:0/3432845783 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 114) v1 ==== 88455+0+0 (secure 0 0 0) 0x7f6890040ce0 con 0x7f6898160c00 2023-08-09T15:11:05.198 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:05.197+0000 7f688e7fc640 1 --2- 172.21.15.43:0/3432845783 >> [v2:172.21.15.203:6832/4012896436,v1:172.21.15.203:6833/4012896436] conn(0x7f687c06a170 0x7f687c06c630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T15:11:05.198 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:05.197+0000 7f688e7fc640 1 -- 172.21.15.43:0/3432845783 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(208..208 src has 1..208) v4 ==== 6828+0+0 (secure 0 0 0) 0x7f68900ab7b0 con 0x7f6898160c00 2023-08-09T15:11:05.198 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:05.197+0000 7f689dfd0640 1 -- 172.21.15.43:0/3432845783 >> [v2:172.21.15.203:6832/4012896436,v1:172.21.15.203:6833/4012896436] conn(0x7f687c06a170 msgr2=0x7f687c06c630 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.203:6832/4012896436 2023-08-09T15:11:05.199 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:05.197+0000 7f689dfd0640 1 --2- 172.21.15.43:0/3432845783 >> [v2:172.21.15.203:6832/4012896436,v1:172.21.15.203:6833/4012896436] conn(0x7f687c06a170 0x7f687c06c630 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-08-09T15:11:05.201 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:05.200+0000 7f688e7fc640 1 -- 172.21.15.43:0/3432845783 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178066 (secure 0 0 0) 0x7f6890076a80 con 0x7f6898160c00 2023-08-09T15:11:05.244 INFO:tasks.ceph.mgr.x.smithi203.stderr:2023-08-09T15:11:05.244+0000 7fc2cf013040 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2023-08-09T15:11:05.301 INFO:tasks.ceph.mgr.x.smithi203.stderr:2023-08-09T15:11:05.301+0000 7fc2cf013040 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2023-08-09T15:11:05.333 INFO:tasks.ceph.mgr.y.smithi043.stderr:2023-08-09T15:11:05.331+0000 7fb51ee0f040 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2023-08-09T15:11:05.351 INFO:tasks.ceph.mgr.x.smithi203.stderr:2023-08-09T15:11:05.350+0000 7fc2cf013040 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2023-08-09T15:11:05.394 INFO:tasks.ceph.mgr.y.smithi043.stderr:2023-08-09T15:11:05.392+0000 7fb51ee0f040 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2023-08-09T15:11:05.399 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:05.398+0000 7f689dfd0640 1 -- 172.21.15.43:0/3432845783 >> [v2:172.21.15.203:6832/4012896436,v1:172.21.15.203:6833/4012896436] conn(0x7f687c06a170 msgr2=0x7f687c06c630 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.203:6832/4012896436 2023-08-09T15:11:05.400 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:05.398+0000 7f689dfd0640 1 --2- 172.21.15.43:0/3432845783 >> [v2:172.21.15.203:6832/4012896436,v1:172.21.15.203:6833/4012896436] conn(0x7f687c06a170 0x7f687c06c630 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2023-08-09T15:11:05.409 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:05.408+0000 7f68a0234640 1 -- 172.21.15.43:0/3432845783 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"} v 0) v1 -- 0x7f68980b29c0 con 0x7f6898160c00 2023-08-09T15:11:05.410 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:05.409+0000 7f688e7fc640 1 -- 172.21.15.43:0/3432845783 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"}]=0 v208) v1 ==== 92+0+10 (secure 0 0 0) 0x7f689007a0a0 con 0x7f6898160c00 2023-08-09T15:11:05.410 INFO:teuthology.orchestra.run.smithi043.stdout:pg_num: 8 2023-08-09T15:11:05.414 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:05.412+0000 7f68a0234640 1 -- 172.21.15.43:0/3432845783 >> [v2:172.21.15.203:6832/4012896436,v1:172.21.15.203:6833/4012896436] conn(0x7f687c06a170 msgr2=0x7f687c06c630 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-08-09T15:11:05.414 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:05.412+0000 7f68a0234640 1 --2- 172.21.15.43:0/3432845783 >> [v2:172.21.15.203:6832/4012896436,v1:172.21.15.203:6833/4012896436] conn(0x7f687c06a170 0x7f687c06c630 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T15:11:05.414 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:05.412+0000 7f68a0234640 1 -- 172.21.15.43:0/3432845783 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f6898160c00 msgr2=0x7f68980c6010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T15:11:05.415 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:05.412+0000 7f68a0234640 1 --2- 172.21.15.43:0/3432845783 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f6898160c00 0x7f68980c6010 secure :-1 s=READY pgs=2283 cs=0 l=1 rev1=1 crypto rx=0x7f689002c4d0 tx=0x7f6890004a40 comp rx=0 tx=0).stop 2023-08-09T15:11:05.415 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:05.413+0000 7f68a0234640 1 -- 172.21.15.43:0/3432845783 shutdown_connections 2023-08-09T15:11:05.415 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:05.413+0000 7f68a0234640 1 --2- 172.21.15.43:0/3432845783 >> [v2:172.21.15.203:6832/4012896436,v1:172.21.15.203:6833/4012896436] conn(0x7f687c06a170 0x7f687c06c630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T15:11:05.415 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:05.413+0000 7f68a0234640 1 --2- 172.21.15.43:0/3432845783 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f6898160c00 0x7f68980c6010 unknown :-1 s=CLOSED pgs=2283 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T15:11:05.415 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:05.413+0000 7f68a0234640 1 --2- 172.21.15.43:0/3432845783 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f68980cfd90 0x7f6898158b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T15:11:05.415 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:05.413+0000 7f68a0234640 1 --2- 172.21.15.43:0/3432845783 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f689806e920 0x7f68980c4780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T15:11:05.415 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:05.413+0000 7f68a0234640 1 -- 172.21.15.43:0/3432845783 >> 172.21.15.43:0/3432845783 conn(0x7f68980b1dd0 msgr2=0x7f68980cd5a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-09T15:11:05.416 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:05.413+0000 7f68a0234640 1 -- 172.21.15.43:0/3432845783 shutdown_connections 2023-08-09T15:11:05.416 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:05.413+0000 7f68a0234640 1 -- 172.21.15.43:0/3432845783 wait complete. 2023-08-09T15:11:05.432 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-08-09T15:11:05.451 INFO:tasks.ceph.mgr.y.smithi043.stderr:2023-08-09T15:11:05.450+0000 7fb51ee0f040 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2023-08-09T15:11:05.636 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:05.634+0000 7ff533fff640 1 Processor -- start 2023-08-09T15:11:05.636 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:05.635+0000 7ff533fff640 1 -- start start 2023-08-09T15:11:05.638 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:05.636+0000 7ff533fff640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7ff53415a1b0 0x7ff5340b1ad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T15:11:05.638 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:05.636+0000 7ff533fff640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7ff53415e9c0 0x7ff5340b2010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T15:11:05.638 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:05.636+0000 7ff533fff640 1 -- --> v1:172.21.15.43:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff5340cc6c0 con 0x7ff5340cd630 2023-08-09T15:11:05.638 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:05.636+0000 7ff533fff640 1 -- --> v2:172.21.15.43:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff53415dee0 con 0x7ff53415a1b0 2023-08-09T15:11:05.638 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:05.636+0000 7ff533fff640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff53415e1c0 con 0x7ff53415e9c0 2023-08-09T15:11:05.639 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:05.637+0000 7ff532ffd640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7ff53415a1b0 0x7ff5340b1ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T15:11:05.639 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:05.637+0000 7ff532ffd640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7ff53415a1b0 0x7ff5340b1ad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:46406/0 (socket says 172.21.15.43:46406) 2023-08-09T15:11:05.639 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:05.637+0000 7ff532ffd640 1 -- 172.21.15.43:0/593578181 learned_addr learned my addr 172.21.15.43:0/593578181 (peer_addr_for_me v2:172.21.15.43:0/0) 2023-08-09T15:11:05.639 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:05.637+0000 7ff5337fe640 1 --2- 172.21.15.43:0/593578181 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7ff53415e9c0 0x7ff5340b2010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T15:11:05.640 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:05.637+0000 7ff532ffd640 1 -- 172.21.15.43:0/593578181 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7ff53415e9c0 msgr2=0x7ff5340b2010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-09T15:11:05.640 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:05.637+0000 7ff532ffd640 1 --2- 172.21.15.43:0/593578181 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7ff53415e9c0 0x7ff5340b2010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T15:11:05.640 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:05.637+0000 7ff532ffd640 1 -- 172.21.15.43:0/593578181 >> v1:172.21.15.43:6789/0 conn(0x7ff5340cd630 legacy=0x7ff5340cdc20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T15:11:05.640 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:05.637+0000 7ff532ffd640 1 -- 172.21.15.43:0/593578181 --> v2:172.21.15.43:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff53415e4a0 con 0x7ff53415a1b0 2023-08-09T15:11:05.640 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:05.637+0000 7ff531ffb640 1 -- 172.21.15.43:0/593578181 <== mon.0 v1:172.21.15.43:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1788363795 0 0) 0x7ff5340cc6c0 con 0x7ff5340cd630 2023-08-09T15:11:05.641 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:05.638+0000 7ff532ffd640 1 --2- 172.21.15.43:0/593578181 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff53415a1b0 0x7ff5340b1ad0 secure :-1 s=READY pgs=2285 cs=0 l=1 rev1=1 crypto rx=0x7ff524009870 tx=0x7ff52402b980 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f2224b560a7a935d server_cookie=0 in_seq=0 out_seq=0 2023-08-09T15:11:05.641 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:05.639+0000 7ff531ffb640 1 -- 172.21.15.43:0/593578181 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff524030840 con 0x7ff53415a1b0 2023-08-09T15:11:05.641 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:05.639+0000 7ff531ffb640 1 -- 172.21.15.43:0/593578181 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff5240309a0 con 0x7ff53415a1b0 2023-08-09T15:11:05.642 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:05.639+0000 7ff531ffb640 1 -- 172.21.15.43:0/593578181 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff524030d30 con 0x7ff53415a1b0 2023-08-09T15:11:05.642 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:05.640+0000 7ff533fff640 1 -- 172.21.15.43:0/593578181 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff53415a1b0 msgr2=0x7ff5340b1ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T15:11:05.642 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:05.640+0000 7ff533fff640 1 --2- 172.21.15.43:0/593578181 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff53415a1b0 0x7ff5340b1ad0 secure :-1 s=READY pgs=2285 cs=0 l=1 rev1=1 crypto rx=0x7ff524009870 tx=0x7ff52402b980 comp rx=0 tx=0).stop 2023-08-09T15:11:05.642 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:05.640+0000 7ff533fff640 1 -- 172.21.15.43:0/593578181 shutdown_connections 2023-08-09T15:11:05.642 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:05.640+0000 7ff533fff640 1 --2- 172.21.15.43:0/593578181 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7ff53415e9c0 0x7ff5340b2010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T15:11:05.642 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:05.640+0000 7ff533fff640 1 --2- 172.21.15.43:0/593578181 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff53415a1b0 0x7ff5340b1ad0 unknown :-1 s=CLOSED pgs=2285 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T15:11:05.642 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:05.640+0000 7ff533fff640 1 -- 172.21.15.43:0/593578181 >> 172.21.15.43:0/593578181 conn(0x7ff5340ccad0 msgr2=0x7ff5340cceb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-09T15:11:05.643 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:05.640+0000 7ff533fff640 1 -- 172.21.15.43:0/593578181 shutdown_connections 2023-08-09T15:11:05.643 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:05.640+0000 7ff533fff640 1 -- 172.21.15.43:0/593578181 wait complete. 2023-08-09T15:11:05.643 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:05.641+0000 7ff533fff640 1 Processor -- start 2023-08-09T15:11:05.643 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:05.642+0000 7ff533fff640 1 -- start start 2023-08-09T15:11:05.644 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:05.643+0000 7ff533fff640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7ff5340cd630 0x7ff53406b2b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T15:11:05.644 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:05.643+0000 7ff533fff640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff53415a1b0 0x7ff53414f450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T15:11:05.644 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:05.643+0000 7ff533fff640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7ff53415e9c0 0x7ff5341550b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T15:11:05.645 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:05.643+0000 7ff533fff640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff5340c4a10 con 0x7ff53415a1b0 2023-08-09T15:11:05.645 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:05.643+0000 7ff533fff640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff5340c3b60 con 0x7ff53415e9c0 2023-08-09T15:11:05.645 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:05.643+0000 7ff533fff640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff53406b7f0 con 0x7ff5340cd630 2023-08-09T15:11:05.645 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:05.643+0000 7ff5327fc640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff53415a1b0 0x7ff53414f450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T15:11:05.645 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:05.643+0000 7ff5327fc640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff53415a1b0 0x7ff53414f450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:46410/0 (socket says 172.21.15.43:46410) 2023-08-09T15:11:05.645 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:05.643+0000 7ff5327fc640 1 -- 172.21.15.43:0/2181144103 learned_addr learned my addr 172.21.15.43:0/2181144103 (peer_addr_for_me v2:172.21.15.43:0/0) 2023-08-09T15:11:05.645 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:05.643+0000 7ff532ffd640 1 --2- 172.21.15.43:0/2181144103 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7ff5340cd630 0x7ff53406b2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T15:11:05.646 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:05.643+0000 7ff5337fe640 1 --2- 172.21.15.43:0/2181144103 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7ff53415e9c0 0x7ff5341550b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T15:11:05.647 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:05.643+0000 7ff5327fc640 1 -- 172.21.15.43:0/2181144103 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7ff5340cd630 msgr2=0x7ff53406b2b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T15:11:05.647 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:05.643+0000 7ff5327fc640 1 --2- 172.21.15.43:0/2181144103 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7ff5340cd630 0x7ff53406b2b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T15:11:05.647 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:05.643+0000 7ff5327fc640 1 -- 172.21.15.43:0/2181144103 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7ff53415e9c0 msgr2=0x7ff5341550b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T15:11:05.647 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:05.643+0000 7ff5327fc640 1 --2- 172.21.15.43:0/2181144103 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7ff53415e9c0 0x7ff5341550b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T15:11:05.647 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:05.643+0000 7ff5327fc640 1 -- 172.21.15.43:0/2181144103 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff5240094d0 con 0x7ff53415a1b0 2023-08-09T15:11:05.647 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:05.644+0000 7ff5327fc640 1 --2- 172.21.15.43:0/2181144103 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff53415a1b0 0x7ff53414f450 secure :-1 s=READY pgs=2286 cs=0 l=1 rev1=1 crypto rx=0x7ff52800c4b0 tx=0x7ff52800c970 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-09T15:11:05.647 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:05.644+0000 7ff51b7fe640 1 -- 172.21.15.43:0/2181144103 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff528005480 con 0x7ff53415a1b0 2023-08-09T15:11:05.648 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:05.644+0000 7ff51b7fe640 1 -- 172.21.15.43:0/2181144103 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff5280035f0 con 0x7ff53415a1b0 2023-08-09T15:11:05.648 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:05.644+0000 7ff51b7fe640 1 -- 172.21.15.43:0/2181144103 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff5280039e0 con 0x7ff53415a1b0 2023-08-09T15:11:05.648 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:05.644+0000 7ff533fff640 1 -- 172.21.15.43:0/2181144103 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff5340c3e00 con 0x7ff53415a1b0 2023-08-09T15:11:05.648 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:05.644+0000 7ff533fff640 1 -- 172.21.15.43:0/2181144103 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff5340c43c0 con 0x7ff53415a1b0 2023-08-09T15:11:05.648 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:05.645+0000 7ff533fff640 1 -- 172.21.15.43:0/2181144103 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff5341f3170 con 0x7ff53415a1b0 2023-08-09T15:11:05.649 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:05.646+0000 7ff51b7fe640 1 -- 172.21.15.43:0/2181144103 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 114) v1 ==== 88455+0+0 (secure 0 0 0) 0x7ff528003b40 con 0x7ff53415a1b0 2023-08-09T15:11:05.649 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:05.647+0000 7ff51b7fe640 1 --2- 172.21.15.43:0/2181144103 >> [v2:172.21.15.203:6832/4012896436,v1:172.21.15.203:6833/4012896436] conn(0x7ff52006a1a0 0x7ff52006c660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T15:11:05.649 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:05.647+0000 7ff51b7fe640 1 -- 172.21.15.43:0/2181144103 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(208..208 src has 1..208) v4 ==== 6828+0+0 (secure 0 0 0) 0x7ff528088070 con 0x7ff53415a1b0 2023-08-09T15:11:05.649 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:05.647+0000 7ff532ffd640 1 -- 172.21.15.43:0/2181144103 >> [v2:172.21.15.203:6832/4012896436,v1:172.21.15.203:6833/4012896436] conn(0x7ff52006a1a0 msgr2=0x7ff52006c660 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.203:6832/4012896436 2023-08-09T15:11:05.649 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:05.647+0000 7ff532ffd640 1 --2- 172.21.15.43:0/2181144103 >> [v2:172.21.15.203:6832/4012896436,v1:172.21.15.203:6833/4012896436] conn(0x7ff52006a1a0 0x7ff52006c660 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-08-09T15:11:05.653 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:05.652+0000 7ff51b7fe640 1 -- 172.21.15.43:0/2181144103 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178066 (secure 0 0 0) 0x7ff5280ba7b0 con 0x7ff53415a1b0 2023-08-09T15:11:05.849 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:05.848+0000 7ff533fff640 1 -- 172.21.15.43:0/2181144103 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7ff53404e390 con 0x7ff53415a1b0 2023-08-09T15:11:05.850 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:05.848+0000 7ff532ffd640 1 -- 172.21.15.43:0/2181144103 >> [v2:172.21.15.203:6832/4012896436,v1:172.21.15.203:6833/4012896436] conn(0x7ff52006a1a0 msgr2=0x7ff52006c660 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.203:6832/4012896436 2023-08-09T15:11:05.850 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:05.848+0000 7ff532ffd640 1 --2- 172.21.15.43:0/2181144103 >> [v2:172.21.15.203:6832/4012896436,v1:172.21.15.203:6833/4012896436] conn(0x7ff52006a1a0 0x7ff52006c660 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2023-08-09T15:11:05.851 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:05.849+0000 7ff51b7fe640 1 -- 172.21.15.43:0/2181144103 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v208) v1 ==== 93+0+10 (secure 0 0 0) 0x7ff528017020 con 0x7ff53415a1b0 2023-08-09T15:11:05.851 INFO:teuthology.orchestra.run.smithi043.stdout:pg_num: 1 2023-08-09T15:11:05.853 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:05.851+0000 7ff533fff640 1 -- 172.21.15.43:0/2181144103 >> [v2:172.21.15.203:6832/4012896436,v1:172.21.15.203:6833/4012896436] conn(0x7ff52006a1a0 msgr2=0x7ff52006c660 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-08-09T15:11:05.853 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:05.851+0000 7ff533fff640 1 --2- 172.21.15.43:0/2181144103 >> [v2:172.21.15.203:6832/4012896436,v1:172.21.15.203:6833/4012896436] conn(0x7ff52006a1a0 0x7ff52006c660 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T15:11:05.853 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:05.851+0000 7ff533fff640 1 -- 172.21.15.43:0/2181144103 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff53415a1b0 msgr2=0x7ff53414f450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T15:11:05.853 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:05.851+0000 7ff533fff640 1 --2- 172.21.15.43:0/2181144103 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff53415a1b0 0x7ff53414f450 secure :-1 s=READY pgs=2286 cs=0 l=1 rev1=1 crypto rx=0x7ff52800c4b0 tx=0x7ff52800c970 comp rx=0 tx=0).stop 2023-08-09T15:11:05.853 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:05.851+0000 7ff533fff640 1 -- 172.21.15.43:0/2181144103 shutdown_connections 2023-08-09T15:11:05.853 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:05.851+0000 7ff533fff640 1 --2- 172.21.15.43:0/2181144103 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7ff53415e9c0 0x7ff5341550b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T15:11:05.853 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:05.851+0000 7ff533fff640 1 --2- 172.21.15.43:0/2181144103 >> [v2:172.21.15.203:6832/4012896436,v1:172.21.15.203:6833/4012896436] conn(0x7ff52006a1a0 0x7ff52006c660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T15:11:05.854 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:05.851+0000 7ff533fff640 1 --2- 172.21.15.43:0/2181144103 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff53415a1b0 0x7ff53414f450 unknown :-1 s=CLOSED pgs=2286 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T15:11:05.854 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:05.851+0000 7ff533fff640 1 --2- 172.21.15.43:0/2181144103 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7ff5340cd630 0x7ff53406b2b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T15:11:05.854 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:05.851+0000 7ff533fff640 1 -- 172.21.15.43:0/2181144103 >> 172.21.15.43:0/2181144103 conn(0x7ff5340ccad0 msgr2=0x7ff5340af980 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-09T15:11:05.854 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:05.852+0000 7ff533fff640 1 -- 172.21.15.43:0/2181144103 shutdown_connections 2023-08-09T15:11:05.854 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:05.852+0000 7ff533fff640 1 -- 172.21.15.43:0/2181144103 wait complete. 2023-08-09T15:11:05.871 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-08-09T15:11:06.061 INFO:tasks.ceph.mgr.x.smithi203.stderr:2023-08-09T15:11:06.061+0000 7fc2cf013040 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2023-08-09T15:11:06.109 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:06.108+0000 7ff407fff640 1 Processor -- start 2023-08-09T15:11:06.110 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:06.108+0000 7ff407fff640 1 -- start start 2023-08-09T15:11:06.111 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:06.110+0000 7ff407fff640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7ff40806d760 0x7ff40806dbe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T15:11:06.111 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:06.110+0000 7ff407fff640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7ff4080ce3d0 0x7ff4080ce860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T15:11:06.111 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:06.110+0000 7ff407fff640 1 -- --> v1:172.21.15.203:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff4080cdf40 con 0x7ff40813d250 2023-08-09T15:11:06.112 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:06.110+0000 7ff407fff640 1 -- --> v2:172.21.15.43:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff40806e120 con 0x7ff4080ce3d0 2023-08-09T15:11:06.112 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:06.110+0000 7ff407fff640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff4080ceda0 con 0x7ff40806d760 2023-08-09T15:11:06.112 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:06.110+0000 7ff4077fe640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7ff4080ce3d0 0x7ff4080ce860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T15:11:06.113 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:06.110+0000 7ff4077fe640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7ff4080ce3d0 0x7ff4080ce860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:46420/0 (socket says 172.21.15.43:46420) 2023-08-09T15:11:06.113 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:06.110+0000 7ff4077fe640 1 -- 172.21.15.43:0/52513392 learned_addr learned my addr 172.21.15.43:0/52513392 (peer_addr_for_me v2:172.21.15.43:0/0) 2023-08-09T15:11:06.113 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:06.111+0000 7ff3fffff640 1 --2- 172.21.15.43:0/52513392 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7ff40806d760 0x7ff40806dbe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T15:11:06.113 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:06.111+0000 7ff4077fe640 1 -- 172.21.15.43:0/52513392 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7ff40806d760 msgr2=0x7ff40806dbe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-09T15:11:06.113 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:06.111+0000 7ff4077fe640 1 --2- 172.21.15.43:0/52513392 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7ff40806d760 0x7ff40806dbe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T15:11:06.113 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:06.111+0000 7ff4077fe640 1 -- 172.21.15.43:0/52513392 >> v1:172.21.15.203:6789/0 conn(0x7ff40813d250 legacy=0x7ff40813d770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-09T15:11:06.114 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:06.111+0000 7ff4077fe640 1 -- 172.21.15.43:0/52513392 --> v2:172.21.15.43:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff40813de80 con 0x7ff4080ce3d0 2023-08-09T15:11:06.114 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:06.112+0000 7ff4077fe640 1 --2- 172.21.15.43:0/52513392 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff4080ce3d0 0x7ff4080ce860 secure :-1 s=READY pgs=2287 cs=0 l=1 rev1=1 crypto rx=0x7ff400009e50 tx=0x7ff40002bed0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=786497c946c1c07c server_cookie=0 in_seq=0 out_seq=0 2023-08-09T15:11:06.115 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:06.112+0000 7ff4067fc640 1 -- 172.21.15.43:0/52513392 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff40002ca70 con 0x7ff4080ce3d0 2023-08-09T15:11:06.115 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:06.112+0000 7ff4067fc640 1 -- 172.21.15.43:0/52513392 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff40002cbd0 con 0x7ff4080ce3d0 2023-08-09T15:11:06.115 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:06.112+0000 7ff4067fc640 1 -- 172.21.15.43:0/52513392 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff4000343f0 con 0x7ff4080ce3d0 2023-08-09T15:11:06.115 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:06.113+0000 7ff407fff640 1 -- 172.21.15.43:0/52513392 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff4080ce3d0 msgr2=0x7ff4080ce860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T15:11:06.115 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:06.113+0000 7ff407fff640 1 --2- 172.21.15.43:0/52513392 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff4080ce3d0 0x7ff4080ce860 secure :-1 s=READY pgs=2287 cs=0 l=1 rev1=1 crypto rx=0x7ff400009e50 tx=0x7ff40002bed0 comp rx=0 tx=0).stop 2023-08-09T15:11:06.116 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:06.113+0000 7ff407fff640 1 -- 172.21.15.43:0/52513392 shutdown_connections 2023-08-09T15:11:06.116 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:06.113+0000 7ff407fff640 1 --2- 172.21.15.43:0/52513392 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff4080ce3d0 0x7ff4080ce860 unknown :-1 s=CLOSED pgs=2287 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T15:11:06.116 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:06.113+0000 7ff407fff640 1 --2- 172.21.15.43:0/52513392 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7ff40806d760 0x7ff40806dbe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T15:11:06.116 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:06.113+0000 7ff407fff640 1 -- 172.21.15.43:0/52513392 >> 172.21.15.43:0/52513392 conn(0x7ff40806cde0 msgr2=0x7ff4080cef70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-09T15:11:06.116 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:06.113+0000 7ff407fff640 1 -- 172.21.15.43:0/52513392 shutdown_connections 2023-08-09T15:11:06.116 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:06.113+0000 7ff407fff640 1 -- 172.21.15.43:0/52513392 wait complete. 2023-08-09T15:11:06.117 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:06.115+0000 7ff407fff640 1 Processor -- start 2023-08-09T15:11:06.117 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:06.115+0000 7ff407fff640 1 -- start start 2023-08-09T15:11:06.118 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:06.116+0000 7ff407fff640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7ff40806d760 0x7ff40813ae10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T15:11:06.118 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:06.116+0000 7ff407fff640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7ff4080ce3d0 0x7ff408069b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T15:11:06.118 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:06.116+0000 7ff407fff640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff40813d250 0x7ff40806a070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T15:11:06.118 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:06.116+0000 7ff407fff640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff4081f1620 con 0x7ff40813d250 2023-08-09T15:11:06.118 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:06.116+0000 7ff407fff640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff408154d60 con 0x7ff4080ce3d0 2023-08-09T15:11:06.119 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:06.116+0000 7ff407fff640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff408155000 con 0x7ff40806d760 2023-08-09T15:11:06.119 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:06.117+0000 7ff4077fe640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff40813d250 0x7ff40806a070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T15:11:06.119 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:06.117+0000 7ff4077fe640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff40813d250 0x7ff40806a070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:46432/0 (socket says 172.21.15.43:46432) 2023-08-09T15:11:06.119 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:06.117+0000 7ff4077fe640 1 -- 172.21.15.43:0/1463007989 learned_addr learned my addr 172.21.15.43:0/1463007989 (peer_addr_for_me v2:172.21.15.43:0/0) 2023-08-09T15:11:06.120 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:06.117+0000 7ff406ffd640 1 --2- 172.21.15.43:0/1463007989 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7ff40806d760 0x7ff40813ae10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T15:11:06.120 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:06.117+0000 7ff3fffff640 1 --2- 172.21.15.43:0/1463007989 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7ff4080ce3d0 0x7ff408069b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T15:11:06.120 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:06.117+0000 7ff4077fe640 1 -- 172.21.15.43:0/1463007989 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7ff40806d760 msgr2=0x7ff40813ae10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T15:11:06.120 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:06.117+0000 7ff4077fe640 1 --2- 172.21.15.43:0/1463007989 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7ff40806d760 0x7ff40813ae10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T15:11:06.121 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:06.117+0000 7ff4077fe640 1 -- 172.21.15.43:0/1463007989 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7ff4080ce3d0 msgr2=0x7ff408069b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T15:11:06.121 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:06.117+0000 7ff4077fe640 1 --2- 172.21.15.43:0/1463007989 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7ff4080ce3d0 0x7ff408069b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T15:11:06.121 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:06.117+0000 7ff4077fe640 1 -- 172.21.15.43:0/1463007989 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff400009b30 con 0x7ff40813d250 2023-08-09T15:11:06.121 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:06.117+0000 7ff3fffff640 1 --2- 172.21.15.43:0/1463007989 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7ff4080ce3d0 0x7ff408069b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-08-09T15:11:06.121 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:06.118+0000 7ff4077fe640 1 --2- 172.21.15.43:0/1463007989 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff40813d250 0x7ff40806a070 secure :-1 s=READY pgs=2288 cs=0 l=1 rev1=1 crypto rx=0x7ff400036040 tx=0x7ff400004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-09T15:11:06.121 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:06.118+0000 7ff404ff9640 1 -- 172.21.15.43:0/1463007989 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff400009510 con 0x7ff40813d250 2023-08-09T15:11:06.121 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:06.118+0000 7ff404ff9640 1 -- 172.21.15.43:0/1463007989 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff40003e450 con 0x7ff40813d250 2023-08-09T15:11:06.122 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:06.118+0000 7ff404ff9640 1 -- 172.21.15.43:0/1463007989 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff40003e7e0 con 0x7ff40813d250 2023-08-09T15:11:06.122 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:06.118+0000 7ff407fff640 1 -- 172.21.15.43:0/1463007989 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff4081552e0 con 0x7ff40813d250 2023-08-09T15:11:06.122 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:06.118+0000 7ff407fff640 1 -- 172.21.15.43:0/1463007989 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff408157a90 con 0x7ff40813d250 2023-08-09T15:11:06.122 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:06.119+0000 7ff407fff640 1 -- 172.21.15.43:0/1463007989 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff40804e390 con 0x7ff40813d250 2023-08-09T15:11:06.123 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:06.120+0000 7ff404ff9640 1 -- 172.21.15.43:0/1463007989 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 114) v1 ==== 88455+0+0 (secure 0 0 0) 0x7ff40003e940 con 0x7ff40813d250 2023-08-09T15:11:06.123 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:06.121+0000 7ff404ff9640 1 --2- 172.21.15.43:0/1463007989 >> [v2:172.21.15.203:6832/4012896436,v1:172.21.15.203:6833/4012896436] conn(0x7ff3e806a1a0 0x7ff3e806c660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T15:11:06.123 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:06.121+0000 7ff404ff9640 1 -- 172.21.15.43:0/1463007989 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(208..208 src has 1..208) v4 ==== 6828+0+0 (secure 0 0 0) 0x7ff4000ab6a0 con 0x7ff40813d250 2023-08-09T15:11:06.123 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:06.122+0000 7ff406ffd640 1 -- 172.21.15.43:0/1463007989 >> [v2:172.21.15.203:6832/4012896436,v1:172.21.15.203:6833/4012896436] conn(0x7ff3e806a1a0 msgr2=0x7ff3e806c660 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.203:6832/4012896436 2023-08-09T15:11:06.123 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:06.122+0000 7ff406ffd640 1 --2- 172.21.15.43:0/1463007989 >> [v2:172.21.15.203:6832/4012896436,v1:172.21.15.203:6833/4012896436] conn(0x7ff3e806a1a0 0x7ff3e806c660 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-08-09T15:11:06.124 INFO:tasks.ceph.mgr.x.smithi203.stderr:2023-08-09T15:11:06.123+0000 7fc2cf013040 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2023-08-09T15:11:06.127 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:06.126+0000 7ff404ff9640 1 -- 172.21.15.43:0/1463007989 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178066 (secure 0 0 0) 0x7ff400076a10 con 0x7ff40813d250 2023-08-09T15:11:06.175 INFO:tasks.ceph.mgr.x.smithi203.stderr:2023-08-09T15:11:06.174+0000 7fc2cf013040 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2023-08-09T15:11:06.199 INFO:tasks.ceph.mgr.y.smithi043.stderr:2023-08-09T15:11:06.198+0000 7fb51ee0f040 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2023-08-09T15:11:06.265 INFO:tasks.ceph.mgr.y.smithi043.stderr:2023-08-09T15:11:06.264+0000 7fb51ee0f040 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2023-08-09T15:11:06.300 INFO:tasks.ceph.mgr.x.smithi203.stderr:2023-08-09T15:11:06.299+0000 7fc2cf013040 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2023-08-09T15:11:06.318 INFO:tasks.ceph.mgr.y.smithi043.stderr:2023-08-09T15:11:06.317+0000 7fb51ee0f040 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2023-08-09T15:11:06.320 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:06.319+0000 7ff407fff640 1 -- 172.21.15.43:0/1463007989 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7ff4080afd70 con 0x7ff40813d250 2023-08-09T15:11:06.321 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:06.320+0000 7ff404ff9640 1 -- 172.21.15.43:0/1463007989 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v209) v1 ==== 104+0+11 (secure 0 0 0) 0x7ff40007a030 con 0x7ff40813d250 2023-08-09T15:11:06.321 INFO:teuthology.orchestra.run.smithi043.stdout:pg_num: 32 2023-08-09T15:11:06.323 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:06.322+0000 7ff407fff640 1 -- 172.21.15.43:0/1463007989 >> [v2:172.21.15.203:6832/4012896436,v1:172.21.15.203:6833/4012896436] conn(0x7ff3e806a1a0 msgr2=0x7ff3e806c660 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-08-09T15:11:06.324 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:06.322+0000 7ff407fff640 1 --2- 172.21.15.43:0/1463007989 >> [v2:172.21.15.203:6832/4012896436,v1:172.21.15.203:6833/4012896436] conn(0x7ff3e806a1a0 0x7ff3e806c660 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T15:11:06.324 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:06.322+0000 7ff407fff640 1 -- 172.21.15.43:0/1463007989 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff40813d250 msgr2=0x7ff40806a070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T15:11:06.324 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:06.322+0000 7ff407fff640 1 --2- 172.21.15.43:0/1463007989 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff40813d250 0x7ff40806a070 secure :-1 s=READY pgs=2288 cs=0 l=1 rev1=1 crypto rx=0x7ff400036040 tx=0x7ff400004a40 comp rx=0 tx=0).stop 2023-08-09T15:11:06.324 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:06.322+0000 7ff407fff640 1 -- 172.21.15.43:0/1463007989 shutdown_connections 2023-08-09T15:11:06.324 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:06.322+0000 7ff407fff640 1 --2- 172.21.15.43:0/1463007989 >> [v2:172.21.15.203:6832/4012896436,v1:172.21.15.203:6833/4012896436] conn(0x7ff3e806a1a0 0x7ff3e806c660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T15:11:06.324 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:06.322+0000 7ff407fff640 1 --2- 172.21.15.43:0/1463007989 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff40813d250 0x7ff40806a070 unknown :-1 s=CLOSED pgs=2288 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T15:11:06.325 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:06.322+0000 7ff407fff640 1 --2- 172.21.15.43:0/1463007989 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7ff4080ce3d0 0x7ff408069b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T15:11:06.325 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:06.322+0000 7ff407fff640 1 --2- 172.21.15.43:0/1463007989 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7ff40806d760 0x7ff40813ae10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T15:11:06.325 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:06.322+0000 7ff407fff640 1 -- 172.21.15.43:0/1463007989 >> 172.21.15.43:0/1463007989 conn(0x7ff40806cde0 msgr2=0x7ff4080b0290 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-09T15:11:06.325 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:06.322+0000 7ff407fff640 1 -- 172.21.15.43:0/1463007989 shutdown_connections 2023-08-09T15:11:06.325 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:06.322+0000 7ff407fff640 1 -- 172.21.15.43:0/1463007989 wait complete. 2023-08-09T15:11:06.339 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-08-09T15:11:06.450 INFO:tasks.ceph.mgr.y.smithi043.stderr:2023-08-09T15:11:06.448+0000 7fb51ee0f040 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2023-08-09T15:11:06.468 INFO:tasks.ceph.mgr.x.smithi203.stderr:2023-08-09T15:11:06.468+0000 7fc2cf013040 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2023-08-09T15:11:06.548 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:06.546+0000 7f47c1c0d640 1 Processor -- start 2023-08-09T15:11:06.548 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:06.547+0000 7f47c1c0d640 1 -- start start 2023-08-09T15:11:06.549 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:06.548+0000 7f47c1c0d640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f47bc15c550 0x7f47bc0cd6d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T15:11:06.550 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:06.548+0000 7f47c1c0d640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f47bc0b45f0 0x7f47bc0b59d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T15:11:06.550 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:06.548+0000 7f47c1c0d640 1 -- --> v1:172.21.15.203:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f47bc0b7bd0 con 0x7f47bc0d04b0 2023-08-09T15:11:06.550 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:06.548+0000 7f47c1c0d640 1 -- --> v2:172.21.15.43:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f47bc0d1c00 con 0x7f47bc15c550 2023-08-09T15:11:06.550 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:06.548+0000 7f47c1c0d640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f47bc15c930 con 0x7f47bc0b45f0 2023-08-09T15:11:06.551 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:06.548+0000 7f47bb7fe640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f47bc15c550 0x7f47bc0cd6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T15:11:06.551 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:06.549+0000 7f47bb7fe640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f47bc15c550 0x7f47bc0cd6d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:46436/0 (socket says 172.21.15.43:46436) 2023-08-09T15:11:06.551 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:06.549+0000 7f47bb7fe640 1 -- 172.21.15.43:0/2533577635 learned_addr learned my addr 172.21.15.43:0/2533577635 (peer_addr_for_me v2:172.21.15.43:0/0) 2023-08-09T15:11:06.551 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:06.549+0000 7f47bbfff640 1 --2- 172.21.15.43:0/2533577635 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f47bc0b45f0 0x7f47bc0b59d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T15:11:06.551 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:06.549+0000 7f47bb7fe640 1 -- 172.21.15.43:0/2533577635 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f47bc0b45f0 msgr2=0x7f47bc0b59d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-09T15:11:06.551 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:06.549+0000 7f47bb7fe640 1 --2- 172.21.15.43:0/2533577635 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f47bc0b45f0 0x7f47bc0b59d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T15:11:06.552 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:06.549+0000 7f47bb7fe640 1 -- 172.21.15.43:0/2533577635 >> v1:172.21.15.203:6789/0 conn(0x7f47bc0d04b0 legacy=0x7f47bc0b3ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T15:11:06.552 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:06.549+0000 7f47bb7fe640 1 -- 172.21.15.43:0/2533577635 --> v2:172.21.15.43:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f47bc0b54f0 con 0x7f47bc15c550 2023-08-09T15:11:06.552 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:06.549+0000 7f47bbfff640 1 --2- 172.21.15.43:0/2533577635 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f47bc0b45f0 0x7f47bc0b59d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-08-09T15:11:06.553 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:06.550+0000 7f47bb7fe640 1 --2- 172.21.15.43:0/2533577635 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f47bc15c550 0x7f47bc0cd6d0 secure :-1 s=READY pgs=2289 cs=0 l=1 rev1=1 crypto rx=0x7f47ac009870 tx=0x7f47ac02b8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7fc492f8060970a3 server_cookie=0 in_seq=0 out_seq=0 2023-08-09T15:11:06.553 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:06.550+0000 7f47baffd640 1 -- 172.21.15.43:0/2533577635 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f47ac031640 con 0x7f47bc15c550 2023-08-09T15:11:06.553 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:06.551+0000 7f47baffd640 1 -- 172.21.15.43:0/2533577635 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f47ac0317a0 con 0x7f47bc15c550 2023-08-09T15:11:06.553 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:06.551+0000 7f47baffd640 1 -- 172.21.15.43:0/2533577635 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f47ac031b30 con 0x7f47bc15c550 2023-08-09T15:11:06.554 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:06.551+0000 7f47c1c0d640 1 -- 172.21.15.43:0/2533577635 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f47bc15c550 msgr2=0x7f47bc0cd6d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T15:11:06.554 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:06.551+0000 7f47c1c0d640 1 --2- 172.21.15.43:0/2533577635 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f47bc15c550 0x7f47bc0cd6d0 secure :-1 s=READY pgs=2289 cs=0 l=1 rev1=1 crypto rx=0x7f47ac009870 tx=0x7f47ac02b8f0 comp rx=0 tx=0).stop 2023-08-09T15:11:06.554 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:06.551+0000 7f47c1c0d640 1 -- 172.21.15.43:0/2533577635 shutdown_connections 2023-08-09T15:11:06.554 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:06.551+0000 7f47c1c0d640 1 --2- 172.21.15.43:0/2533577635 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f47bc0b45f0 0x7f47bc0b59d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T15:11:06.554 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:06.551+0000 7f47c1c0d640 1 --2- 172.21.15.43:0/2533577635 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f47bc15c550 0x7f47bc0cd6d0 unknown :-1 s=CLOSED pgs=2289 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T15:11:06.554 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:06.551+0000 7f47c1c0d640 1 -- 172.21.15.43:0/2533577635 >> 172.21.15.43:0/2533577635 conn(0x7f47bc071520 msgr2=0x7f47bc0ceee0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-09T15:11:06.554 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:06.551+0000 7f47c1c0d640 1 -- 172.21.15.43:0/2533577635 shutdown_connections 2023-08-09T15:11:06.555 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:06.551+0000 7f47c1c0d640 1 -- 172.21.15.43:0/2533577635 wait complete. 2023-08-09T15:11:06.555 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:06.553+0000 7f47c1c0d640 1 Processor -- start 2023-08-09T15:11:06.555 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:06.553+0000 7f47c1c0d640 1 -- start start 2023-08-09T15:11:06.556 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:06.554+0000 7f47c1c0d640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f47bc0b45f0 0x7f47bc153df0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T15:11:06.557 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:06.554+0000 7f47c1c0d640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f47bc0d04b0 0x7f47bc1f18d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T15:11:06.557 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:06.554+0000 7f47c1c0d640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f47bc15c550 0x7f47bc0cab20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T15:11:06.557 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:06.554+0000 7f47c1c0d640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f47bc0c70a0 con 0x7f47bc0d04b0 2023-08-09T15:11:06.557 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:06.554+0000 7f47c1c0d640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f47bc154330 con 0x7f47bc0b45f0 2023-08-09T15:11:06.557 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:06.554+0000 7f47c1c0d640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f47bc0c74f0 con 0x7f47bc15c550 2023-08-09T15:11:06.557 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:06.555+0000 7f47b3fff640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f47bc0d04b0 0x7f47bc1f18d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T15:11:06.557 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:06.555+0000 7f47b3fff640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f47bc0d04b0 0x7f47bc1f18d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:46438/0 (socket says 172.21.15.43:46438) 2023-08-09T15:11:06.558 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:06.555+0000 7f47b3fff640 1 -- 172.21.15.43:0/1431670142 learned_addr learned my addr 172.21.15.43:0/1431670142 (peer_addr_for_me v2:172.21.15.43:0/0) 2023-08-09T15:11:06.558 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:06.555+0000 7f47bbfff640 1 --2- 172.21.15.43:0/1431670142 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f47bc15c550 0x7f47bc0cab20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T15:11:06.558 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:06.555+0000 7f47bb7fe640 1 --2- 172.21.15.43:0/1431670142 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f47bc0b45f0 0x7f47bc153df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T15:11:06.558 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:06.555+0000 7f47b3fff640 1 -- 172.21.15.43:0/1431670142 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f47bc15c550 msgr2=0x7f47bc0cab20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T15:11:06.558 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:06.555+0000 7f47b3fff640 1 --2- 172.21.15.43:0/1431670142 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f47bc15c550 0x7f47bc0cab20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T15:11:06.558 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:06.555+0000 7f47b3fff640 1 -- 172.21.15.43:0/1431670142 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f47bc0b45f0 msgr2=0x7f47bc153df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T15:11:06.559 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:06.555+0000 7f47b3fff640 1 --2- 172.21.15.43:0/1431670142 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f47bc0b45f0 0x7f47bc153df0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T15:11:06.559 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:06.555+0000 7f47b3fff640 1 -- 172.21.15.43:0/1431670142 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f47ac0094d0 con 0x7f47bc0d04b0 2023-08-09T15:11:06.560 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:06.555+0000 7f47b3fff640 1 --2- 172.21.15.43:0/1431670142 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f47bc0d04b0 0x7f47bc1f18d0 secure :-1 s=READY pgs=2290 cs=0 l=1 rev1=1 crypto rx=0x7f47a400c450 tx=0x7f47a400c910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-09T15:11:06.560 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:06.556+0000 7f47b8ff9640 1 -- 172.21.15.43:0/1431670142 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f47a40052d0 con 0x7f47bc0d04b0 2023-08-09T15:11:06.560 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:06.556+0000 7f47b8ff9640 1 -- 172.21.15.43:0/1431670142 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f47a4005430 con 0x7f47bc0d04b0 2023-08-09T15:11:06.560 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:06.556+0000 7f47b8ff9640 1 -- 172.21.15.43:0/1431670142 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f47a4012870 con 0x7f47bc0d04b0 2023-08-09T15:11:06.560 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:06.556+0000 7f47c1c0d640 1 -- 172.21.15.43:0/1431670142 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f47bc0c77d0 con 0x7f47bc0d04b0 2023-08-09T15:11:06.560 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:06.556+0000 7f47c1c0d640 1 -- 172.21.15.43:0/1431670142 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f47bc0c7d90 con 0x7f47bc0d04b0 2023-08-09T15:11:06.560 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:06.557+0000 7f47c1c0d640 1 -- 172.21.15.43:0/1431670142 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f47bc0c7fb0 con 0x7f47bc0d04b0 2023-08-09T15:11:06.561 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:06.558+0000 7f47b8ff9640 1 -- 172.21.15.43:0/1431670142 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 114) v1 ==== 88455+0+0 (secure 0 0 0) 0x7f47a40129d0 con 0x7f47bc0d04b0 2023-08-09T15:11:06.561 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:06.559+0000 7f47b8ff9640 1 --2- 172.21.15.43:0/1431670142 >> [v2:172.21.15.203:6832/4012896436,v1:172.21.15.203:6833/4012896436] conn(0x7f479c06a170 0x7f479c06c630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T15:11:06.561 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:06.559+0000 7f47b8ff9640 1 -- 172.21.15.43:0/1431670142 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(209..209 src has 1..209) v4 ==== 6916+0+0 (secure 0 0 0) 0x7f47a40885d0 con 0x7f47bc0d04b0 2023-08-09T15:11:06.562 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:06.559+0000 7f47bb7fe640 1 -- 172.21.15.43:0/1431670142 >> [v2:172.21.15.203:6832/4012896436,v1:172.21.15.203:6833/4012896436] conn(0x7f479c06a170 msgr2=0x7f479c06c630 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.203:6832/4012896436 2023-08-09T15:11:06.562 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:06.559+0000 7f47bb7fe640 1 --2- 172.21.15.43:0/1431670142 >> [v2:172.21.15.203:6832/4012896436,v1:172.21.15.203:6833/4012896436] conn(0x7f479c06a170 0x7f479c06c630 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-08-09T15:11:06.564 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:06.563+0000 7f47b8ff9640 1 -- 172.21.15.43:0/1431670142 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178066 (secure 0 0 0) 0x7f47a4053840 con 0x7f47bc0d04b0 2023-08-09T15:11:06.583 INFO:tasks.ceph.mgr.x.smithi203.stderr:2023-08-09T15:11:06.583+0000 7fc2cf013040 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2023-08-09T15:11:06.621 INFO:tasks.ceph.mgr.y.smithi043.stderr:2023-08-09T15:11:06.620+0000 7fb51ee0f040 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2023-08-09T15:11:06.633 INFO:tasks.ceph.mgr.x.smithi203.stderr:2023-08-09T15:11:06.633+0000 7fc2cf013040 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2023-08-09T15:11:06.735 INFO:tasks.ceph.mgr.y.smithi043.stderr:2023-08-09T15:11:06.733+0000 7fb51ee0f040 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2023-08-09T15:11:06.755 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:06.754+0000 7f47c1c0d640 1 -- 172.21.15.43:0/1431670142 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f47bc0739f0 con 0x7f47bc0d04b0 2023-08-09T15:11:06.756 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:06.754+0000 7f47b8ff9640 1 -- 172.21.15.43:0/1431670142 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v209) v1 ==== 100+0+11 (secure 0 0 0) 0x7f47a4056e60 con 0x7f47bc0d04b0 2023-08-09T15:11:06.756 INFO:teuthology.orchestra.run.smithi043.stdout:pg_num: 64 2023-08-09T15:11:06.758 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:06.757+0000 7f47c1c0d640 1 -- 172.21.15.43:0/1431670142 >> [v2:172.21.15.203:6832/4012896436,v1:172.21.15.203:6833/4012896436] conn(0x7f479c06a170 msgr2=0x7f479c06c630 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-08-09T15:11:06.759 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:06.757+0000 7f47c1c0d640 1 --2- 172.21.15.43:0/1431670142 >> [v2:172.21.15.203:6832/4012896436,v1:172.21.15.203:6833/4012896436] conn(0x7f479c06a170 0x7f479c06c630 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T15:11:06.759 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:06.757+0000 7f47c1c0d640 1 -- 172.21.15.43:0/1431670142 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f47bc0d04b0 msgr2=0x7f47bc1f18d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T15:11:06.759 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:06.757+0000 7f47c1c0d640 1 --2- 172.21.15.43:0/1431670142 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f47bc0d04b0 0x7f47bc1f18d0 secure :-1 s=READY pgs=2290 cs=0 l=1 rev1=1 crypto rx=0x7f47a400c450 tx=0x7f47a400c910 comp rx=0 tx=0).stop 2023-08-09T15:11:06.759 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:06.757+0000 7f47c1c0d640 1 -- 172.21.15.43:0/1431670142 shutdown_connections 2023-08-09T15:11:06.759 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:06.757+0000 7f47c1c0d640 1 --2- 172.21.15.43:0/1431670142 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f47bc15c550 0x7f47bc0cab20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T15:11:06.759 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:06.757+0000 7f47c1c0d640 1 --2- 172.21.15.43:0/1431670142 >> [v2:172.21.15.203:6832/4012896436,v1:172.21.15.203:6833/4012896436] conn(0x7f479c06a170 0x7f479c06c630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T15:11:06.759 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:06.757+0000 7f47c1c0d640 1 --2- 172.21.15.43:0/1431670142 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f47bc0d04b0 0x7f47bc1f18d0 unknown :-1 s=CLOSED pgs=2290 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T15:11:06.760 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:06.757+0000 7f47c1c0d640 1 --2- 172.21.15.43:0/1431670142 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f47bc0b45f0 0x7f47bc153df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T15:11:06.760 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:06.757+0000 7f47c1c0d640 1 -- 172.21.15.43:0/1431670142 >> 172.21.15.43:0/1431670142 conn(0x7f47bc071520 msgr2=0x7f47bc15f4a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-09T15:11:06.760 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:06.757+0000 7f47c1c0d640 1 -- 172.21.15.43:0/1431670142 shutdown_connections 2023-08-09T15:11:06.760 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:06.757+0000 7f47c1c0d640 1 -- 172.21.15.43:0/1431670142 wait complete. 2023-08-09T15:11:06.776 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2023-08-09T15:11:06.787 INFO:tasks.ceph.mgr.y.smithi043.stderr:2023-08-09T15:11:06.786+0000 7fb51ee0f040 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2023-08-09T15:11:06.795 INFO:tasks.ceph.mgr.x.smithi203.stderr:2023-08-09T15:11:06.795+0000 7fc2cf013040 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2023-08-09T15:11:06.966 INFO:tasks.ceph.mgr.y.smithi043.stderr:2023-08-09T15:11:06.964+0000 7fb51ee0f040 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2023-08-09T15:11:06.982 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:06.980+0000 7f16cf84c640 1 Processor -- start 2023-08-09T15:11:06.982 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:06.980+0000 7f16cf84c640 1 -- start start 2023-08-09T15:11:06.983 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:06.982+0000 7f16cf84c640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f16c815c680 0x7f16c80ce760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T15:11:06.983 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:06.982+0000 7f16cf84c640 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7f16c80d0a70 0x7f16c80cd7f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T15:11:06.984 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:06.982+0000 7f16cf84c640 1 -- --> v1:172.21.15.203:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f16c80d0f70 con 0x7f16c80cdd30 2023-08-09T15:11:06.984 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:06.982+0000 7f16cf84c640 1 -- --> v2:172.21.15.203:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f16c80ccd10 con 0x7f16c80d0a70 2023-08-09T15:11:06.984 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:06.982+0000 7f16cf84c640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f16c80ccff0 con 0x7f16c815c680 2023-08-09T15:11:06.985 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:06.983+0000 7f16cd5e8640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f16c815c680 0x7f16c80ce760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T15:11:06.985 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:06.983+0000 7f16ccde7640 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7f16c80d0a70 0x7f16c80cd7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T15:11:06.985 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:06.983+0000 7f16cd5e8640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f16c815c680 0x7f16c80ce760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.203:3301/0 says I am v2:172.21.15.43:36114/0 (socket says 172.21.15.43:36114) 2023-08-09T15:11:06.985 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:06.983+0000 7f16cd5e8640 1 -- 172.21.15.43:0/4218578048 learned_addr learned my addr 172.21.15.43:0/4218578048 (peer_addr_for_me v2:172.21.15.43:0/0) 2023-08-09T15:11:06.986 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:06.984+0000 7f16ccde7640 1 -- 172.21.15.43:0/4218578048 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f16c815c680 msgr2=0x7f16c80ce760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-09T15:11:06.986 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:06.984+0000 7f16ccde7640 1 --2- 172.21.15.43:0/4218578048 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f16c815c680 0x7f16c80ce760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T15:11:06.986 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:06.984+0000 7f16ccde7640 1 -- 172.21.15.43:0/4218578048 >> v1:172.21.15.203:6789/0 conn(0x7f16c80cdd30 legacy=0x7f16c80b3fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T15:11:06.986 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:06.984+0000 7f16ccde7640 1 -- 172.21.15.43:0/4218578048 --> v2:172.21.15.203:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f16c80cd2d0 con 0x7f16c80d0a70 2023-08-09T15:11:06.986 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:06.984+0000 7f16cd5e8640 1 --2- 172.21.15.43:0/4218578048 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f16c815c680 0x7f16c80ce760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-08-09T15:11:06.987 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:06.985+0000 7f16ccde7640 1 --2- 172.21.15.43:0/4218578048 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f16c80d0a70 0x7f16c80cd7f0 secure :-1 s=READY pgs=839 cs=0 l=1 rev1=1 crypto rx=0x7f16b80097a0 tx=0x7f16b802b8b0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=d26928862225b10d server_cookie=0 in_seq=0 out_seq=0 2023-08-09T15:11:06.987 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:06.986+0000 7f16bffff640 1 -- 172.21.15.43:0/4218578048 <== mon.1 v2:172.21.15.203:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f16b802d040 con 0x7f16c80d0a70 2023-08-09T15:11:06.987 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:06.986+0000 7f16bffff640 1 -- 172.21.15.43:0/4218578048 <== mon.1 v2:172.21.15.203:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f16b8031810 con 0x7f16c80d0a70 2023-08-09T15:11:06.987 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:06.986+0000 7f16bffff640 1 -- 172.21.15.43:0/4218578048 <== mon.1 v2:172.21.15.203:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f16b8031ba0 con 0x7f16c80d0a70 2023-08-09T15:11:06.988 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:06.986+0000 7f16cf84c640 1 -- 172.21.15.43:0/4218578048 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f16c80d0a70 msgr2=0x7f16c80cd7f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T15:11:06.988 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:06.986+0000 7f16cf84c640 1 --2- 172.21.15.43:0/4218578048 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f16c80d0a70 0x7f16c80cd7f0 secure :-1 s=READY pgs=839 cs=0 l=1 rev1=1 crypto rx=0x7f16b80097a0 tx=0x7f16b802b8b0 comp rx=0 tx=0).stop 2023-08-09T15:11:06.988 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:06.986+0000 7f16cf84c640 1 -- 172.21.15.43:0/4218578048 shutdown_connections 2023-08-09T15:11:06.989 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:06.986+0000 7f16cf84c640 1 --2- 172.21.15.43:0/4218578048 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f16c80d0a70 0x7f16c80cd7f0 unknown :-1 s=CLOSED pgs=839 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T15:11:06.989 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:06.986+0000 7f16cf84c640 1 --2- 172.21.15.43:0/4218578048 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f16c815c680 0x7f16c80ce760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T15:11:06.989 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:06.986+0000 7f16cf84c640 1 -- 172.21.15.43:0/4218578048 >> 172.21.15.43:0/4218578048 conn(0x7f16c8071520 msgr2=0x7f16c80d2520 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-09T15:11:06.989 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:06.987+0000 7f16cf84c640 1 -- 172.21.15.43:0/4218578048 shutdown_connections 2023-08-09T15:11:06.989 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:06.987+0000 7f16cf84c640 1 -- 172.21.15.43:0/4218578048 wait complete. 2023-08-09T15:11:06.990 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:06.988+0000 7f16cf84c640 1 Processor -- start 2023-08-09T15:11:06.990 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:06.988+0000 7f16cf84c640 1 -- start start 2023-08-09T15:11:06.991 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:06.989+0000 7f16cf84c640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f16c80d0a70 0x7f16c80caa60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T15:11:06.991 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:06.989+0000 7f16cf84c640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f16c815c680 0x7f16c815d540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T15:11:06.992 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:06.989+0000 7f16cf84c640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f16c815e260 0x7f16c815e6f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T15:11:06.992 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:06.989+0000 7f16cf84c640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f16c815ec30 con 0x7f16c80d0a70 2023-08-09T15:11:06.992 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:06.989+0000 7f16cf84c640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f16c815d0b0 con 0x7f16c815c680 2023-08-09T15:11:06.992 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:06.989+0000 7f16cf84c640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f16c81f56c0 con 0x7f16c815e260 2023-08-09T15:11:06.992 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:06.990+0000 7f16cd5e8640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f16c80d0a70 0x7f16c80caa60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T15:11:06.992 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:06.990+0000 7f16cd5e8640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f16c80d0a70 0x7f16c80caa60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:46448/0 (socket says 172.21.15.43:46448) 2023-08-09T15:11:06.992 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:06.990+0000 7f16cd5e8640 1 -- 172.21.15.43:0/4025881362 learned_addr learned my addr 172.21.15.43:0/4025881362 (peer_addr_for_me v2:172.21.15.43:0/0) 2023-08-09T15:11:06.993 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:06.990+0000 7f16ccde7640 1 --2- 172.21.15.43:0/4025881362 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f16c815c680 0x7f16c815d540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T15:11:06.993 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:06.990+0000 7f16cdde9640 1 --2- 172.21.15.43:0/4025881362 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f16c815e260 0x7f16c815e6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T15:11:06.994 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:06.990+0000 7f16cd5e8640 1 -- 172.21.15.43:0/4025881362 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f16c815e260 msgr2=0x7f16c815e6f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T15:11:06.994 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:06.990+0000 7f16cd5e8640 1 --2- 172.21.15.43:0/4025881362 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f16c815e260 0x7f16c815e6f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T15:11:06.994 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:06.990+0000 7f16cd5e8640 1 -- 172.21.15.43:0/4025881362 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f16c815c680 msgr2=0x7f16c815d540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T15:11:06.994 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:06.990+0000 7f16cd5e8640 1 --2- 172.21.15.43:0/4025881362 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f16c815c680 0x7f16c815d540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T15:11:06.994 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:06.990+0000 7f16cd5e8640 1 -- 172.21.15.43:0/4025881362 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f16c40094d0 con 0x7f16c80d0a70 2023-08-09T15:11:06.994 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:06.991+0000 7f16cd5e8640 1 --2- 172.21.15.43:0/4025881362 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f16c80d0a70 0x7f16c80caa60 secure :-1 s=READY pgs=2291 cs=0 l=1 rev1=1 crypto rx=0x7f16c400ea00 tx=0x7f16c400eec0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-09T15:11:06.994 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:06.991+0000 7f16bdffb640 1 -- 172.21.15.43:0/4025881362 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f16c400ca60 con 0x7f16c80d0a70 2023-08-09T15:11:06.995 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:06.991+0000 7f16bdffb640 1 -- 172.21.15.43:0/4025881362 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f16c400cbc0 con 0x7f16c80d0a70 2023-08-09T15:11:06.995 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:06.991+0000 7f16bdffb640 1 -- 172.21.15.43:0/4025881362 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f16c4013470 con 0x7f16c80d0a70 2023-08-09T15:11:06.995 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:06.991+0000 7f16cf84c640 1 -- 172.21.15.43:0/4025881362 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f16b8009400 con 0x7f16c80d0a70 2023-08-09T15:11:06.995 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:06.991+0000 7f16cf84c640 1 -- 172.21.15.43:0/4025881362 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f16c80cc330 con 0x7f16c80d0a70 2023-08-09T15:11:06.995 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:06.992+0000 7f16cf84c640 1 -- 172.21.15.43:0/4025881362 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f16c80739f0 con 0x7f16c80d0a70 2023-08-09T15:11:06.996 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:06.993+0000 7f16bdffb640 1 -- 172.21.15.43:0/4025881362 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 114) v1 ==== 88455+0+0 (secure 0 0 0) 0x7f16c40135d0 con 0x7f16c80d0a70 2023-08-09T15:11:06.996 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:06.994+0000 7f16bdffb640 1 --2- 172.21.15.43:0/4025881362 >> [v2:172.21.15.203:6832/4012896436,v1:172.21.15.203:6833/4012896436] conn(0x7f16ac06a170 0x7f16ac06c630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T15:11:06.996 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:06.994+0000 7f16bdffb640 1 -- 172.21.15.43:0/4025881362 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(209..209 src has 1..209) v4 ==== 6916+0+0 (secure 0 0 0) 0x7f16c4088df0 con 0x7f16c80d0a70 2023-08-09T15:11:06.996 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:06.994+0000 7f16ccde7640 1 -- 172.21.15.43:0/4025881362 >> [v2:172.21.15.203:6832/4012896436,v1:172.21.15.203:6833/4012896436] conn(0x7f16ac06a170 msgr2=0x7f16ac06c630 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.203:6832/4012896436 2023-08-09T15:11:06.997 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:06.994+0000 7f16ccde7640 1 --2- 172.21.15.43:0/4025881362 >> [v2:172.21.15.203:6832/4012896436,v1:172.21.15.203:6833/4012896436] conn(0x7f16ac06a170 0x7f16ac06c630 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-08-09T15:11:06.999 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:06.998+0000 7f16bdffb640 1 -- 172.21.15.43:0/4025881362 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178066 (secure 0 0 0) 0x7f16c4053fd0 con 0x7f16c80d0a70 2023-08-09T15:11:07.196 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:07.195+0000 7f16ccde7640 1 -- 172.21.15.43:0/4025881362 >> [v2:172.21.15.203:6832/4012896436,v1:172.21.15.203:6833/4012896436] conn(0x7f16ac06a170 msgr2=0x7f16ac06c630 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.203:6832/4012896436 2023-08-09T15:11:07.197 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:07.195+0000 7f16ccde7640 1 --2- 172.21.15.43:0/4025881362 >> [v2:172.21.15.203:6832/4012896436,v1:172.21.15.203:6833/4012896436] conn(0x7f16ac06a170 0x7f16ac06c630 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2023-08-09T15:11:07.198 INFO:tasks.ceph.mgr.x.smithi203.stderr:2023-08-09T15:11:07.198+0000 7fc2cf013040 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2023-08-09T15:11:07.215 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:07.214+0000 7f16cf84c640 1 -- 172.21.15.43:0/4025881362 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7f16c815fe60 con 0x7f16c80d0a70 2023-08-09T15:11:07.216 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:07.215+0000 7f16bdffb640 1 -- 172.21.15.43:0/4025881362 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 8 entries v210) v1 ==== 80+0+476 (secure 0 0 0) 0x7f16c40171f0 con 0x7f16c80d0a70 2023-08-09T15:11:07.216 INFO:teuthology.orchestra.run.smithi043.stdout:172.21.15.203:6833/3082732769 2023-08-10T15:09:01.285901+0000 2023-08-09T15:11:07.216 INFO:teuthology.orchestra.run.smithi043.stdout:172.21.15.203:6832/3082732769 2023-08-10T15:09:01.285901+0000 2023-08-09T15:11:07.217 INFO:teuthology.orchestra.run.smithi043.stdout:172.21.15.203:0/1756781921 2023-08-10T15:09:01.285901+0000 2023-08-09T15:11:07.217 INFO:teuthology.orchestra.run.smithi043.stdout:172.21.15.203:0/454071895 2023-08-10T15:09:01.285901+0000 2023-08-09T15:11:07.217 INFO:teuthology.orchestra.run.smithi043.stdout:172.21.15.203:0/2854569557 2023-08-10T15:09:01.285901+0000 2023-08-09T15:11:07.217 INFO:teuthology.orchestra.run.smithi043.stdout:172.21.15.203:0/1159944241 2023-08-10T15:09:01.285901+0000 2023-08-09T15:11:07.217 INFO:teuthology.orchestra.run.smithi043.stdout:172.21.15.203:0/1645632393 2023-08-10T15:09:01.285901+0000 2023-08-09T15:11:07.217 INFO:teuthology.orchestra.run.smithi043.stdout:172.21.15.203:0/840449585 2023-08-10T15:09:01.285901+0000 2023-08-09T15:11:07.218 INFO:teuthology.orchestra.run.smithi043.stderr:listed 8 entries 2023-08-09T15:11:07.219 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:07.218+0000 7f16cf84c640 1 -- 172.21.15.43:0/4025881362 >> [v2:172.21.15.203:6832/4012896436,v1:172.21.15.203:6833/4012896436] conn(0x7f16ac06a170 msgr2=0x7f16ac06c630 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-08-09T15:11:07.219 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:07.218+0000 7f16cf84c640 1 --2- 172.21.15.43:0/4025881362 >> [v2:172.21.15.203:6832/4012896436,v1:172.21.15.203:6833/4012896436] conn(0x7f16ac06a170 0x7f16ac06c630 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T15:11:07.220 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:07.218+0000 7f16cf84c640 1 -- 172.21.15.43:0/4025881362 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f16c80d0a70 msgr2=0x7f16c80caa60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T15:11:07.220 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:07.218+0000 7f16cf84c640 1 --2- 172.21.15.43:0/4025881362 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f16c80d0a70 0x7f16c80caa60 secure :-1 s=READY pgs=2291 cs=0 l=1 rev1=1 crypto rx=0x7f16c400ea00 tx=0x7f16c400eec0 comp rx=0 tx=0).stop 2023-08-09T15:11:07.220 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:07.218+0000 7f16cf84c640 1 -- 172.21.15.43:0/4025881362 shutdown_connections 2023-08-09T15:11:07.221 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:07.218+0000 7f16cf84c640 1 --2- 172.21.15.43:0/4025881362 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f16c815e260 0x7f16c815e6f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T15:11:07.221 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:07.218+0000 7f16cf84c640 1 --2- 172.21.15.43:0/4025881362 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f16c815c680 0x7f16c815d540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T15:11:07.221 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:07.218+0000 7f16cf84c640 1 --2- 172.21.15.43:0/4025881362 >> [v2:172.21.15.203:6832/4012896436,v1:172.21.15.203:6833/4012896436] conn(0x7f16ac06a170 0x7f16ac06c630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T15:11:07.221 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:07.218+0000 7f16cf84c640 1 --2- 172.21.15.43:0/4025881362 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f16c80d0a70 0x7f16c80caa60 unknown :-1 s=CLOSED pgs=2291 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T15:11:07.221 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:07.218+0000 7f16cf84c640 1 -- 172.21.15.43:0/4025881362 >> 172.21.15.43:0/4025881362 conn(0x7f16c8071520 msgr2=0x7f16c80ce120 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-09T15:11:07.221 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:07.219+0000 7f16cf84c640 1 -- 172.21.15.43:0/4025881362 shutdown_connections 2023-08-09T15:11:07.222 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:07.219+0000 7f16cf84c640 1 -- 172.21.15.43:0/4025881362 wait complete. 2023-08-09T15:11:07.239 INFO:tasks.cephfs.fuse_mount:Running fusermount -u on ubuntu@smithi043.front.sepia.ceph.com... 2023-08-09T15:11:07.240 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-09T15:11:07.240 DEBUG:teuthology.orchestra.run.smithi043:> sudo fusermount -u /home/ubuntu/cephtest/mnt.0 2023-08-09T15:11:07.367 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi043.stderr:ceph-fuse[148280]: fuse finished with error 0 and tester_r 0 2023-08-09T15:11:07.371 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi043.front.sepia.ceph.com 2023-08-09T15:11:07.372 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-09T15:11:07.372 DEBUG:teuthology.orchestra.run.smithi043:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2023-08-09T15:11:07.396 INFO:tasks.ceph.mgr.y.smithi043.stderr:2023-08-09T15:11:07.395+0000 7fb51ee0f040 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2023-08-09T15:11:07.436 INFO:tasks.ceph.mgr.x.smithi203.stderr:2023-08-09T15:11:07.436+0000 7fc2cf013040 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2023-08-09T15:11:07.470 DEBUG:teuthology.orchestra.run.smithi203:> set -ex 2023-08-09T15:11:07.471 DEBUG:teuthology.orchestra.run.smithi203:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-09T15:11:07.495 INFO:tasks.ceph.mgr.x.smithi203.stderr:2023-08-09T15:11:07.495+0000 7fc2cf013040 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2023-08-09T15:11:07.526 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4053648,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497576k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=27,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248784k,nr_inodes=812196,mode=700,uid=1000,gid=1275 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497576k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2023-08-09T15:11:07.526 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi203.front.sepia.ceph.com 2023-08-09T15:11:07.527 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-09T15:11:07.528 DEBUG:teuthology.orchestra.run.smithi203:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.1) 2023-08-09T15:11:07.552 INFO:tasks.ceph.mgr.x.smithi203.stderr:2023-08-09T15:11:07.552+0000 7fc2cf013040 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2023-08-09T15:11:07.599 INFO:teuthology.orchestra.run.smithi203.stderr:rmdir: failed to remove '/home/ubuntu/cephtest/mnt.1': No such file or directory 2023-08-09T15:11:07.600 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-08-09T15:11:07.600 DEBUG:teuthology.orchestra.run.smithi043:> set -ex 2023-08-09T15:11:07.601 DEBUG:teuthology.orchestra.run.smithi043:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-09T15:11:07.609 INFO:tasks.ceph.mgr.x.smithi203.stderr:2023-08-09T15:11:07.608+0000 7fc2cf013040 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2023-08-09T15:11:07.631 INFO:tasks.ceph.mgr.y.smithi043.stderr:2023-08-09T15:11:07.630+0000 7fb51ee0f040 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2023-08-09T15:11:07.657 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4054441,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6498844k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=27,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3249420k,nr_inodes=812355,mode=700,uid=1000,gid=1275 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6498844k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2023-08-09T15:11:07.658 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi043.front.sepia.ceph.com 2023-08-09T15:11:07.659 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-09T15:11:07.659 DEBUG:teuthology.orchestra.run.smithi043:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.2) 2023-08-09T15:11:07.692 INFO:tasks.ceph.mgr.y.smithi043.stderr:2023-08-09T15:11:07.690+0000 7fb51ee0f040 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2023-08-09T15:11:07.744 INFO:tasks.ceph.mgr.y.smithi043.stderr:2023-08-09T15:11:07.743+0000 7fb51ee0f040 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2023-08-09T15:11:07.750 INFO:teuthology.orchestra.run.smithi043.stderr:rmdir: failed to remove '/home/ubuntu/cephtest/mnt.2': No such file or directory 2023-08-09T15:11:07.751 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-08-09T15:11:07.752 DEBUG:teuthology.orchestra.run.smithi203:> set -ex 2023-08-09T15:11:07.752 DEBUG:teuthology.orchestra.run.smithi203:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-09T15:11:07.799 INFO:tasks.ceph.mgr.y.smithi043.stderr:2023-08-09T15:11:07.798+0000 7fb51ee0f040 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2023-08-09T15:11:07.809 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4053648,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497576k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=27,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248784k,nr_inodes=812196,mode=700,uid=1000,gid=1275 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497576k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2023-08-09T15:11:07.809 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi203.front.sepia.ceph.com 2023-08-09T15:11:07.810 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-09T15:11:07.810 DEBUG:teuthology.orchestra.run.smithi203:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.3) 2023-08-09T15:11:07.857 INFO:teuthology.orchestra.run.smithi203.stderr:rmdir: failed to remove '/home/ubuntu/cephtest/mnt.3': No such file or directory 2023-08-09T15:11:07.858 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-08-09T15:11:07.859 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-08-09T15:11:07.916 INFO:tasks.ceph.mgr.x.smithi203.stderr:2023-08-09T15:11:07.915+0000 7fc2cf013040 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2023-08-09T15:11:07.975 INFO:tasks.ceph.mgr.x.smithi203.stderr:2023-08-09T15:11:07.973+0000 7fc2cf013040 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2023-08-09T15:11:08.027 INFO:tasks.ceph.mgr.x.smithi203.stderr:2023-08-09T15:11:08.026+0000 7fc2cf013040 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2023-08-09T15:11:08.080 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:08.078+0000 7f3f63301640 1 Processor -- start 2023-08-09T15:11:08.080 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:08.078+0000 7f3f63301640 1 -- start start 2023-08-09T15:11:08.082 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:08.080+0000 7f3f63301640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f3f5c0ce720 0x7f3f5c0ceb00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T15:11:08.082 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:08.080+0000 7f3f63301640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f3f5c0ce080 0x7f3f5c15e5b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T15:11:08.082 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:08.080+0000 7f3f63301640 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7f3f5c15eaf0 0x7f3f5c15efa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T15:11:08.082 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:08.080+0000 7f3f63301640 1 -- --> v2:172.21.15.43:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3f5c0cd8a0 con 0x7f3f5c0ce720 2023-08-09T15:11:08.082 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:08.080+0000 7f3f63301640 1 -- --> v2:172.21.15.203:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3f5c0cf350 con 0x7f3f5c15eaf0 2023-08-09T15:11:08.082 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:08.080+0000 7f3f63301640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3f5c0cf040 con 0x7f3f5c0ce080 2023-08-09T15:11:08.083 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:08.080+0000 7f3f6109d640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f3f5c0ce720 0x7f3f5c0ceb00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T15:11:08.083 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:08.081+0000 7f3f6109d640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f3f5c0ce720 0x7f3f5c0ceb00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:46454/0 (socket says 172.21.15.43:46454) 2023-08-09T15:11:08.083 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:08.081+0000 7f3f6109d640 1 -- 172.21.15.43:0/900382401 learned_addr learned my addr 172.21.15.43:0/900382401 (peer_addr_for_me v2:172.21.15.43:0/0) 2023-08-09T15:11:08.084 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:08.081+0000 7f3f6089c640 1 --2- 172.21.15.43:0/900382401 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f3f5c0ce080 0x7f3f5c15e5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T15:11:08.084 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:08.081+0000 7f3f6189e640 1 --2- 172.21.15.43:0/900382401 >> v2:172.21.15.203:3300/0 conn(0x7f3f5c15eaf0 0x7f3f5c15efa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T15:11:08.084 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:08.081+0000 7f3f6109d640 1 -- 172.21.15.43:0/900382401 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f3f5c0ce080 msgr2=0x7f3f5c15e5b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-09T15:11:08.084 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:08.081+0000 7f3f6109d640 1 --2- 172.21.15.43:0/900382401 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f3f5c0ce080 0x7f3f5c15e5b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T15:11:08.084 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:08.081+0000 7f3f6109d640 1 -- 172.21.15.43:0/900382401 >> v2:172.21.15.203:3300/0 conn(0x7f3f5c15eaf0 msgr2=0x7f3f5c15efa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-09T15:11:08.085 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:08.081+0000 7f3f6109d640 1 --2- 172.21.15.43:0/900382401 >> v2:172.21.15.203:3300/0 conn(0x7f3f5c15eaf0 0x7f3f5c15efa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T15:11:08.085 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:08.081+0000 7f3f6109d640 1 -- 172.21.15.43:0/900382401 --> v2:172.21.15.43:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3f5c071a70 con 0x7f3f5c0ce720 2023-08-09T15:11:08.085 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:08.081+0000 7f3f6189e640 1 --2- 172.21.15.43:0/900382401 >> v2:172.21.15.203:3300/0 conn(0x7f3f5c15eaf0 0x7f3f5c15efa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-08-09T15:11:08.085 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:08.081+0000 7f3f6089c640 1 --2- 172.21.15.43:0/900382401 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f3f5c0ce080 0x7f3f5c15e5b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-08-09T15:11:08.085 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:08.082+0000 7f3f6109d640 1 --2- 172.21.15.43:0/900382401 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f3f5c0ce720 0x7f3f5c0ceb00 secure :-1 s=READY pgs=2292 cs=0 l=1 rev1=1 crypto rx=0x7f3f5c0ca370 tx=0x7f3f5802b860 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1b3e1b27a7d9af29 server_cookie=0 in_seq=0 out_seq=0 2023-08-09T15:11:08.085 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:08.082+0000 7f3f53fff640 1 -- 172.21.15.43:0/900382401 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3f58004230 con 0x7f3f5c0ce720 2023-08-09T15:11:08.085 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:08.082+0000 7f3f53fff640 1 -- 172.21.15.43:0/900382401 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3f58004390 con 0x7f3f5c0ce720 2023-08-09T15:11:08.086 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:08.083+0000 7f3f53fff640 1 -- 172.21.15.43:0/900382401 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3f58003810 con 0x7f3f5c0ce720 2023-08-09T15:11:08.086 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:08.083+0000 7f3f63301640 1 -- 172.21.15.43:0/900382401 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f3f5c0ce720 msgr2=0x7f3f5c0ceb00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T15:11:08.086 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:08.083+0000 7f3f63301640 1 --2- 172.21.15.43:0/900382401 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f3f5c0ce720 0x7f3f5c0ceb00 secure :-1 s=READY pgs=2292 cs=0 l=1 rev1=1 crypto rx=0x7f3f5c0ca370 tx=0x7f3f5802b860 comp rx=0 tx=0).stop 2023-08-09T15:11:08.087 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:08.083+0000 7f3f63301640 1 -- 172.21.15.43:0/900382401 shutdown_connections 2023-08-09T15:11:08.087 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:08.083+0000 7f3f63301640 1 --2- 172.21.15.43:0/900382401 >> v2:172.21.15.203:3300/0 conn(0x7f3f5c15eaf0 0x7f3f5c15efa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T15:11:08.087 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:08.083+0000 7f3f63301640 1 --2- 172.21.15.43:0/900382401 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f3f5c0ce080 0x7f3f5c15e5b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T15:11:08.087 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:08.083+0000 7f3f63301640 1 --2- 172.21.15.43:0/900382401 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f3f5c0ce720 0x7f3f5c0ceb00 unknown :-1 s=CLOSED pgs=2292 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T15:11:08.087 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:08.083+0000 7f3f63301640 1 -- 172.21.15.43:0/900382401 >> 172.21.15.43:0/900382401 conn(0x7f3f5c0ccd80 msgr2=0x7f3f5c0cd190 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-09T15:11:08.087 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:08.083+0000 7f3f63301640 1 -- 172.21.15.43:0/900382401 shutdown_connections 2023-08-09T15:11:08.087 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:08.083+0000 7f3f63301640 1 -- 172.21.15.43:0/900382401 wait complete. 2023-08-09T15:11:08.088 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:08.085+0000 7f3f63301640 1 Processor -- start 2023-08-09T15:11:08.088 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:08.085+0000 7f3f63301640 1 -- start start 2023-08-09T15:11:08.089 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:08.086+0000 7f3f63301640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f3f5c0ce080 0x7f3f5c0c3d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T15:11:08.089 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:08.086+0000 7f3f63301640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f3f5c0ce720 0x7f3f5c149600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T15:11:08.089 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:08.086+0000 7f3f63301640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f3f5c15eaf0 0x7f3f5c0b1f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T15:11:08.090 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:08.086+0000 7f3f63301640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3f5c0c4630 con 0x7f3f5c15eaf0 2023-08-09T15:11:08.090 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:08.086+0000 7f3f63301640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3f5c0c4910 con 0x7f3f5c0ce720 2023-08-09T15:11:08.090 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:08.086+0000 7f3f63301640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3f5c0c4bf0 con 0x7f3f5c0ce080 2023-08-09T15:11:08.090 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:08.086+0000 7f3f6189e640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f3f5c15eaf0 0x7f3f5c0b1f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T15:11:08.090 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:08.087+0000 7f3f6189e640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f3f5c15eaf0 0x7f3f5c0b1f30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:46456/0 (socket says 172.21.15.43:46456) 2023-08-09T15:11:08.090 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:08.087+0000 7f3f6189e640 1 -- 172.21.15.43:0/3592118831 learned_addr learned my addr 172.21.15.43:0/3592118831 (peer_addr_for_me v2:172.21.15.43:0/0) 2023-08-09T15:11:08.090 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:08.087+0000 7f3f6089c640 1 --2- 172.21.15.43:0/3592118831 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f3f5c0ce720 0x7f3f5c149600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T15:11:08.090 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:08.087+0000 7f3f6109d640 1 --2- 172.21.15.43:0/3592118831 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f3f5c0ce080 0x7f3f5c0c3d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T15:11:08.091 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:08.087+0000 7f3f6189e640 1 -- 172.21.15.43:0/3592118831 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f3f5c0ce080 msgr2=0x7f3f5c0c3d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T15:11:08.091 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:08.087+0000 7f3f6189e640 1 --2- 172.21.15.43:0/3592118831 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f3f5c0ce080 0x7f3f5c0c3d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T15:11:08.091 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:08.087+0000 7f3f6189e640 1 -- 172.21.15.43:0/3592118831 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f3f5c0ce720 msgr2=0x7f3f5c149600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T15:11:08.091 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:08.087+0000 7f3f6189e640 1 --2- 172.21.15.43:0/3592118831 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f3f5c0ce720 0x7f3f5c149600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T15:11:08.091 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:08.087+0000 7f3f6189e640 1 -- 172.21.15.43:0/3592118831 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3f58009460 con 0x7f3f5c15eaf0 2023-08-09T15:11:08.091 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:08.087+0000 7f3f6189e640 1 --2- 172.21.15.43:0/3592118831 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f3f5c15eaf0 0x7f3f5c0b1f30 secure :-1 s=READY pgs=2293 cs=0 l=1 rev1=1 crypto rx=0x7f3f5400cef0 tx=0x7f3f5400a710 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-09T15:11:08.091 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:08.088+0000 7f3f51ffb640 1 -- 172.21.15.43:0/3592118831 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3f5400d840 con 0x7f3f5c15eaf0 2023-08-09T15:11:08.092 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:08.088+0000 7f3f51ffb640 1 -- 172.21.15.43:0/3592118831 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3f5400d9c0 con 0x7f3f5c15eaf0 2023-08-09T15:11:08.092 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:08.088+0000 7f3f63301640 1 -- 172.21.15.43:0/3592118831 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3f5c0c4ed0 con 0x7f3f5c15eaf0 2023-08-09T15:11:08.092 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:08.088+0000 7f3f51ffb640 1 -- 172.21.15.43:0/3592118831 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3f5400dd70 con 0x7f3f5c15eaf0 2023-08-09T15:11:08.092 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:08.088+0000 7f3f63301640 1 -- 172.21.15.43:0/3592118831 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3f5c0c4290 con 0x7f3f5c15eaf0 2023-08-09T15:11:08.093 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:08.089+0000 7f3f63301640 1 -- 172.21.15.43:0/3592118831 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3f5c0c53f0 con 0x7f3f5c15eaf0 2023-08-09T15:11:08.093 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:08.090+0000 7f3f51ffb640 1 -- 172.21.15.43:0/3592118831 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 114) v1 ==== 88455+0+0 (secure 0 0 0) 0x7f3f5401e3b0 con 0x7f3f5c15eaf0 2023-08-09T15:11:08.093 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:08.091+0000 7f3f51ffb640 1 --2- 172.21.15.43:0/3592118831 >> [v2:172.21.15.203:6832/4012896436,v1:172.21.15.203:6833/4012896436] conn(0x7f3f4006a170 0x7f3f4006c630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T15:11:08.093 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:08.091+0000 7f3f51ffb640 1 -- 172.21.15.43:0/3592118831 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(210..210 src has 1..210) v4 ==== 6900+0+0 (secure 0 0 0) 0x7f3f54089600 con 0x7f3f5c15eaf0 2023-08-09T15:11:08.093 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:08.091+0000 7f3f6109d640 1 -- 172.21.15.43:0/3592118831 >> [v2:172.21.15.203:6832/4012896436,v1:172.21.15.203:6833/4012896436] conn(0x7f3f4006a170 msgr2=0x7f3f4006c630 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.203:6832/4012896436 2023-08-09T15:11:08.093 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:08.091+0000 7f3f6109d640 1 --2- 172.21.15.43:0/3592118831 >> [v2:172.21.15.203:6832/4012896436,v1:172.21.15.203:6833/4012896436] conn(0x7f3f4006a170 0x7f3f4006c630 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-08-09T15:11:08.096 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:08.095+0000 7f3f51ffb640 1 -- 172.21.15.43:0/3592118831 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178066 (secure 0 0 0) 0x7f3f5c0c53f0 con 0x7f3f5c15eaf0 2023-08-09T15:11:08.112 INFO:tasks.ceph.mgr.y.smithi043.stderr:2023-08-09T15:11:08.110+0000 7fb51ee0f040 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2023-08-09T15:11:08.173 INFO:tasks.ceph.mgr.y.smithi043.stderr:2023-08-09T15:11:08.172+0000 7fb51ee0f040 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2023-08-09T15:11:08.188 INFO:tasks.ceph.mgr.x.smithi203.stderr:2023-08-09T15:11:08.187+0000 7fc2cf013040 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2023-08-09T15:11:08.229 INFO:tasks.ceph.mgr.y.smithi043.stderr:2023-08-09T15:11:08.227+0000 7fb51ee0f040 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2023-08-09T15:11:08.240 INFO:tasks.ceph.mgr.x.smithi203.stderr:2023-08-09T15:11:08.239+0000 7fc2cf013040 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2023-08-09T15:11:08.294 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:08.292+0000 7f3f6109d640 1 -- 172.21.15.43:0/3592118831 >> [v2:172.21.15.203:6832/4012896436,v1:172.21.15.203:6833/4012896436] conn(0x7f3f4006a170 msgr2=0x7f3f4006c630 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.203:6832/4012896436 2023-08-09T15:11:08.294 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:08.292+0000 7f3f6109d640 1 --2- 172.21.15.43:0/3592118831 >> [v2:172.21.15.203:6832/4012896436,v1:172.21.15.203:6833/4012896436] conn(0x7f3f4006a170 0x7f3f4006c630 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2023-08-09T15:11:08.294 INFO:tasks.ceph.mgr.x.smithi203.stderr:2023-08-09T15:11:08.293+0000 7fc2cf013040 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2023-08-09T15:11:08.327 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:08.326+0000 7f3f63301640 1 -- 172.21.15.43:0/3592118831 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f3f5c0c53f0 con 0x7f3f5c15eaf0 2023-08-09T15:11:08.327 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:08.326+0000 7f3f51ffb640 1 -- 172.21.15.43:0/3592118831 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 96 v96) v1 ==== 94+0+4883 (secure 0 0 0) 0x7f3f5c0c53f0 con 0x7f3f5c15eaf0 2023-08-09T15:11:08.328 INFO:teuthology.orchestra.run.smithi043.stdout: 2023-08-09T15:11:08.328 INFO:teuthology.orchestra.run.smithi043.stdout:{"epoch":96,"default_fscid":11,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":7703,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.203:6836/1774059384","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6835","nonce":1774059384},{"type":"v1","addr":"172.21.15.203:6836","nonce":1774059384}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":91},{"gid":8665,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.203:6837/682905899","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6834","nonce":682905899},{"type":"v1","addr":"172.21.15.203:6837","nonce":682905899}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":91},{"gid":10752,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.43:6834/4168463642","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6832","nonce":4168463642},{"type":"v1","addr":"172.21.15.43:6834","nonce":4168463642}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":90}],"filesystems":[{"mdsmap":{"epoch":96,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-08-09T15:08:45.563041+0000","modified":"2023-08-09T15:08:49.150213+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":10755},"failed":[],"damaged":[],"stopped":[],"info":{"gid_10755":{"gid":10755,"name":"a","rank":0,"incarnation":92,"state":"up:active","state_seq":3,"addr":"172.21.15.43:6835/349688227","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6833","nonce":349688227},{"type":"v1","addr":"172.21.15.43:6835","nonce":349688227}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[25],"metadata_pool":24,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":11}]} 2023-08-09T15:11:08.328 INFO:teuthology.orchestra.run.smithi043.stderr:dumped fsmap epoch 96 2023-08-09T15:11:08.331 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:08.329+0000 7f3f63301640 1 -- 172.21.15.43:0/3592118831 >> [v2:172.21.15.203:6832/4012896436,v1:172.21.15.203:6833/4012896436] conn(0x7f3f4006a170 msgr2=0x7f3f4006c630 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-08-09T15:11:08.331 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:08.329+0000 7f3f63301640 1 --2- 172.21.15.43:0/3592118831 >> [v2:172.21.15.203:6832/4012896436,v1:172.21.15.203:6833/4012896436] conn(0x7f3f4006a170 0x7f3f4006c630 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T15:11:08.331 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:08.329+0000 7f3f63301640 1 -- 172.21.15.43:0/3592118831 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f3f5c15eaf0 msgr2=0x7f3f5c0b1f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T15:11:08.331 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:08.329+0000 7f3f63301640 1 --2- 172.21.15.43:0/3592118831 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f3f5c15eaf0 0x7f3f5c0b1f30 secure :-1 s=READY pgs=2293 cs=0 l=1 rev1=1 crypto rx=0x7f3f5400cef0 tx=0x7f3f5400a710 comp rx=0 tx=0).stop 2023-08-09T15:11:08.331 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:08.329+0000 7f3f63301640 1 -- 172.21.15.43:0/3592118831 shutdown_connections 2023-08-09T15:11:08.331 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:08.329+0000 7f3f63301640 1 --2- 172.21.15.43:0/3592118831 >> [v2:172.21.15.203:6832/4012896436,v1:172.21.15.203:6833/4012896436] conn(0x7f3f4006a170 0x7f3f4006c630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T15:11:08.332 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:08.329+0000 7f3f63301640 1 --2- 172.21.15.43:0/3592118831 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f3f5c15eaf0 0x7f3f5c0b1f30 unknown :-1 s=CLOSED pgs=2293 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T15:11:08.332 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:08.329+0000 7f3f63301640 1 --2- 172.21.15.43:0/3592118831 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f3f5c0ce720 0x7f3f5c149600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T15:11:08.332 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:08.329+0000 7f3f63301640 1 --2- 172.21.15.43:0/3592118831 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f3f5c0ce080 0x7f3f5c0c3d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T15:11:08.332 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:08.329+0000 7f3f63301640 1 -- 172.21.15.43:0/3592118831 >> 172.21.15.43:0/3592118831 conn(0x7f3f5c0ccd80 msgr2=0x7f3f5c159950 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-09T15:11:08.332 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:08.329+0000 7f3f63301640 1 -- 172.21.15.43:0/3592118831 shutdown_connections 2023-08-09T15:11:08.332 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:08.330+0000 7f3f63301640 1 -- 172.21.15.43:0/3592118831 wait complete. 2023-08-09T15:11:08.346 INFO:tasks.ceph.mgr.x.smithi203.stderr:2023-08-09T15:11:08.344+0000 7fc2cf013040 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2023-08-09T15:11:08.348 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-08-09T15:11:08.405 INFO:tasks.ceph.mgr.y.smithi043.stderr:2023-08-09T15:11:08.403+0000 7fb51ee0f040 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2023-08-09T15:11:08.432 INFO:tasks.ceph.mgr.x.smithi203.stderr:2023-08-09T15:11:08.430+0000 7fc254fec640 -1 client.0 error registering admin socket command: (17) File exists 2023-08-09T15:11:08.432 INFO:tasks.ceph.mgr.x.smithi203.stderr:2023-08-09T15:11:08.430+0000 7fc254fec640 -1 client.0 error registering admin socket command: (17) File exists 2023-08-09T15:11:08.432 INFO:tasks.ceph.mgr.x.smithi203.stderr:2023-08-09T15:11:08.430+0000 7fc254fec640 -1 client.0 error registering admin socket command: (17) File exists 2023-08-09T15:11:08.433 INFO:tasks.ceph.mgr.x.smithi203.stderr:2023-08-09T15:11:08.430+0000 7fc254fec640 -1 client.0 error registering admin socket command: (17) File exists 2023-08-09T15:11:08.433 INFO:tasks.ceph.mgr.x.smithi203.stderr:2023-08-09T15:11:08.430+0000 7fc254fec640 -1 client.0 error registering admin socket command: (17) File exists 2023-08-09T15:11:08.433 INFO:tasks.ceph.mgr.x.smithi203.stderr:2023-08-09T15:11:08.431+0000 7fc2517e5640 -1 client.0 error registering admin socket command: (17) File exists 2023-08-09T15:11:08.433 INFO:tasks.ceph.mgr.x.smithi203.stderr:2023-08-09T15:11:08.431+0000 7fc2517e5640 -1 client.0 error registering admin socket command: (17) File exists 2023-08-09T15:11:08.434 INFO:tasks.ceph.mgr.x.smithi203.stderr:2023-08-09T15:11:08.431+0000 7fc2517e5640 -1 client.0 error registering admin socket command: (17) File exists 2023-08-09T15:11:08.434 INFO:tasks.ceph.mgr.x.smithi203.stderr:2023-08-09T15:11:08.431+0000 7fc2517e5640 -1 client.0 error registering admin socket command: (17) File exists 2023-08-09T15:11:08.434 INFO:tasks.ceph.mgr.x.smithi203.stderr:2023-08-09T15:11:08.431+0000 7fc2517e5640 -1 client.0 error registering admin socket command: (17) File exists 2023-08-09T15:11:08.460 INFO:tasks.ceph.mgr.y.smithi043.stderr:2023-08-09T15:11:08.458+0000 7fb51ee0f040 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2023-08-09T15:11:08.516 INFO:tasks.ceph.mgr.y.smithi043.stderr:2023-08-09T15:11:08.515+0000 7fb51ee0f040 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2023-08-09T15:11:08.550 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:08.548+0000 7f6fbd5d5640 1 Processor -- start 2023-08-09T15:11:08.550 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:08.549+0000 7f6fbd5d5640 1 -- start start 2023-08-09T15:11:08.552 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:08.550+0000 7f6fbd5d5640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f6fb8069ed0 0x7f6fb806dd30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T15:11:08.552 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:08.550+0000 7f6fbd5d5640 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7f6fb815a630 0x7f6fb806e270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T15:11:08.552 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:08.550+0000 7f6fbd5d5640 1 -- --> v1:172.21.15.43:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6fb80716b0 con 0x7f6fb815a250 2023-08-09T15:11:08.553 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:08.551+0000 7f6fbd5d5640 1 -- --> v2:172.21.15.203:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6fb8070aa0 con 0x7f6fb815a630 2023-08-09T15:11:08.553 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:08.551+0000 7f6fbd5d5640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6fb8070d80 con 0x7f6fb8069ed0 2023-08-09T15:11:08.553 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:08.551+0000 7f6fb6ffd640 1 --1- >> v1:172.21.15.43:6789/0 conn(0x7f6fb815a250 0x7f6fb80697c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.43:6789/0 says I am v1:172.21.15.43:45534/0 (socket says 172.21.15.43:45534) 2023-08-09T15:11:08.553 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:08.551+0000 7f6fb6ffd640 1 -- 172.21.15.43:0/2301289338 learned_addr learned my addr 172.21.15.43:0/2301289338 (peer_addr_for_me v1:172.21.15.43:0/0) 2023-08-09T15:11:08.555 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:08.551+0000 7f6fb67fc640 1 --2- 172.21.15.43:0/2301289338 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f6fb8069ed0 0x7f6fb806dd30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T15:11:08.555 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:08.551+0000 7f6fb77fe640 1 --2- 172.21.15.43:0/2301289338 >> v2:172.21.15.203:3300/0 conn(0x7f6fb815a630 0x7f6fb806e270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T15:11:08.555 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:08.551+0000 7f6fb5ffb640 1 -- 172.21.15.43:0/2301289338 <== mon.0 v1:172.21.15.43:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 413264987 0 0) 0x7f6fb80716b0 con 0x7f6fb815a250 2023-08-09T15:11:08.555 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:08.551+0000 7f6fb5ffb640 1 -- 172.21.15.43:0/2301289338 --> v1:172.21.15.43:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f6fa0003520 con 0x7f6fb815a250 2023-08-09T15:11:08.555 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:08.552+0000 7f6fb5ffb640 1 -- 172.21.15.43:0/2301289338 <== mon.0 v1:172.21.15.43:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 834495585 0 0) 0x7f6fa0003520 con 0x7f6fb815a250 2023-08-09T15:11:08.555 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:08.552+0000 7f6fb5ffb640 1 -- 172.21.15.43:0/2301289338 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f6fb8069ed0 msgr2=0x7f6fb806dd30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-09T15:11:08.556 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:08.552+0000 7f6fb5ffb640 1 --2- 172.21.15.43:0/2301289338 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f6fb8069ed0 0x7f6fb806dd30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T15:11:08.556 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:08.552+0000 7f6fb5ffb640 1 -- 172.21.15.43:0/2301289338 >> v2:172.21.15.203:3300/0 conn(0x7f6fb815a630 msgr2=0x7f6fb806e270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-09T15:11:08.556 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:08.552+0000 7f6fb5ffb640 1 --2- 172.21.15.43:0/2301289338 >> v2:172.21.15.203:3300/0 conn(0x7f6fb815a630 0x7f6fb806e270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T15:11:08.556 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:08.552+0000 7f6fb5ffb640 1 -- 172.21.15.43:0/2301289338 --> v1:172.21.15.43:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6fb8071060 con 0x7f6fb815a250 2023-08-09T15:11:08.556 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:08.552+0000 7f6fb5ffb640 1 -- 172.21.15.43:0/2301289338 <== mon.0 v1:172.21.15.43:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1285150740 0 0) 0x7f6fa8001670 con 0x7f6fb815a250 2023-08-09T15:11:08.556 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:08.552+0000 7f6fb5ffb640 1 -- 172.21.15.43:0/2301289338 >> v1:172.21.15.43:6789/0 conn(0x7f6fb815a250 legacy=0x7f6fb80697c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T15:11:08.557 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:08.552+0000 7f6fb5ffb640 1 --2- 172.21.15.43:0/2301289338 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f6fa00039e0 0x7f6fa0003e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T15:11:08.557 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:08.552+0000 7f6fb5ffb640 1 --2- 172.21.15.43:0/2301289338 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f6fa00047c0 0x7f6fa0004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T15:11:08.557 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:08.552+0000 7f6fb5ffb640 1 --2- 172.21.15.43:0/2301289338 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f6fb8069ed0 0x7f6fa0005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T15:11:08.557 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:08.552+0000 7f6fb5ffb640 1 -- 172.21.15.43:0/2301289338 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6fb8070aa0 con 0x7f6fa00047c0 2023-08-09T15:11:08.557 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:08.552+0000 7f6fb5ffb640 1 -- 172.21.15.43:0/2301289338 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6fb8070d80 con 0x7f6fa00039e0 2023-08-09T15:11:08.557 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:08.552+0000 7f6fb5ffb640 1 -- 172.21.15.43:0/2301289338 --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6fa000b980 con 0x7f6fb8069ed0 2023-08-09T15:11:08.557 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:08.552+0000 7f6fb67fc640 1 --2- 172.21.15.43:0/2301289338 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f6fa00047c0 0x7f6fa0004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T15:11:08.557 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:08.553+0000 7f6fb77fe640 1 --2- 172.21.15.43:0/2301289338 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f6fb8069ed0 0x7f6fa0005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T15:11:08.558 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:08.553+0000 7f6fb6ffd640 1 --2- 172.21.15.43:0/2301289338 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f6fa00039e0 0x7f6fa0003e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T15:11:08.558 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:08.553+0000 7f6fb67fc640 1 -- 172.21.15.43:0/2301289338 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f6fb8069ed0 msgr2=0x7f6fa0005180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-09T15:11:08.558 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:08.553+0000 7f6fb67fc640 1 --2- 172.21.15.43:0/2301289338 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f6fb8069ed0 0x7f6fa0005180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T15:11:08.558 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:08.553+0000 7f6fb67fc640 1 -- 172.21.15.43:0/2301289338 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f6fa00039e0 msgr2=0x7f6fa0003e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-09T15:11:08.558 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:08.553+0000 7f6fb67fc640 1 --2- 172.21.15.43:0/2301289338 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f6fa00039e0 0x7f6fa0003e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T15:11:08.558 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:08.553+0000 7f6fb67fc640 1 -- 172.21.15.43:0/2301289338 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6fa0003520 con 0x7f6fa00047c0 2023-08-09T15:11:08.558 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:08.553+0000 7f6fb6ffd640 1 --2- 172.21.15.43:0/2301289338 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f6fa00039e0 0x7f6fa0003e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-08-09T15:11:08.558 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:08.553+0000 7f6fb77fe640 1 --2- 172.21.15.43:0/2301289338 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f6fb8069ed0 0x7f6fa0005180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-08-09T15:11:08.559 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:08.554+0000 7f6fb67fc640 1 --2- 172.21.15.43:0/2301289338 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f6fa00047c0 0x7f6fa0004c40 secure :-1 s=READY pgs=2298 cs=0 l=1 rev1=1 crypto rx=0x7f6fac009af0 tx=0x7f6fa00057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4fb2655e5372012a server_cookie=0 in_seq=0 out_seq=0 2023-08-09T15:11:08.560 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:08.554+0000 7f6fb5ffb640 1 -- 172.21.15.43:0/2301289338 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6fac02c6b0 con 0x7f6fa00047c0 2023-08-09T15:11:08.560 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:08.554+0000 7f6fb5ffb640 1 -- 172.21.15.43:0/2301289338 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6fac02c810 con 0x7f6fa00047c0 2023-08-09T15:11:08.560 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:08.554+0000 7f6fb5ffb640 1 -- 172.21.15.43:0/2301289338 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6fac02cba0 con 0x7f6fa00047c0 2023-08-09T15:11:08.560 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:08.555+0000 7f6fbd5d5640 1 -- 172.21.15.43:0/2301289338 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f6fa00047c0 msgr2=0x7f6fa0004c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T15:11:08.560 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:08.555+0000 7f6fbd5d5640 1 --2- 172.21.15.43:0/2301289338 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f6fa00047c0 0x7f6fa0004c40 secure :-1 s=READY pgs=2298 cs=0 l=1 rev1=1 crypto rx=0x7f6fac009af0 tx=0x7f6fa00057f0 comp rx=0 tx=0).stop 2023-08-09T15:11:08.560 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:08.555+0000 7f6fb77fe640 1 -- 172.21.15.43:0/2301289338 reap_dead start 2023-08-09T15:11:08.561 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:08.555+0000 7f6fbd5d5640 1 -- 172.21.15.43:0/2301289338 shutdown_connections 2023-08-09T15:11:08.561 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:08.555+0000 7f6fbd5d5640 1 -- 172.21.15.43:0/2301289338 >> 172.21.15.43:0/2301289338 conn(0x7f6fb80cbc80 msgr2=0x7f6fb80cc090 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-09T15:11:08.561 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:08.555+0000 7f6fbd5d5640 1 -- 172.21.15.43:0/2301289338 shutdown_connections 2023-08-09T15:11:08.561 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:08.555+0000 7f6fbd5d5640 1 -- 172.21.15.43:0/2301289338 wait complete. 2023-08-09T15:11:08.561 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:08.558+0000 7f6fbd5d5640 1 Processor -- start 2023-08-09T15:11:08.561 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:08.558+0000 7f6fbd5d5640 1 -- start start 2023-08-09T15:11:08.563 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:08.560+0000 7f6fbd5d5640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f6fa00047c0 0x7f6fb8156340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T15:11:08.563 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:08.560+0000 7f6fbd5d5640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f6fb80ccdd0 0x7f6fb80c6d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T15:11:08.563 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:08.560+0000 7f6fbd5d5640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f6fb80cd250 0x7f6fb80c7290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T15:11:08.563 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:08.560+0000 7f6fbd5d5640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6fb8156880 con 0x7f6fb80cd250 2023-08-09T15:11:08.563 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:08.560+0000 7f6fbd5d5640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6fb80cd7b0 con 0x7f6fb80ccdd0 2023-08-09T15:11:08.563 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:08.560+0000 7f6fbd5d5640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6fb80cda90 con 0x7f6fa00047c0 2023-08-09T15:11:08.564 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:08.560+0000 7f6fb77fe640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f6fb80cd250 0x7f6fb80c7290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T15:11:08.564 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:08.560+0000 7f6fb77fe640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f6fb80cd250 0x7f6fb80c7290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:46470/0 (socket says 172.21.15.43:46470) 2023-08-09T15:11:08.564 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:08.560+0000 7f6fb77fe640 1 -- 172.21.15.43:0/1414423250 learned_addr learned my addr 172.21.15.43:0/1414423250 (peer_addr_for_me v2:172.21.15.43:0/0) 2023-08-09T15:11:08.564 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:08.560+0000 7f6fb67fc640 1 --2- 172.21.15.43:0/1414423250 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f6fb80ccdd0 0x7f6fb80c6d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T15:11:08.564 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:08.560+0000 7f6fb6ffd640 1 --2- 172.21.15.43:0/1414423250 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f6fa00047c0 0x7f6fb8156340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T15:11:08.565 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:08.560+0000 7f6fb77fe640 1 -- 172.21.15.43:0/1414423250 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f6fa00047c0 msgr2=0x7f6fb8156340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T15:11:08.565 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:08.560+0000 7f6fb77fe640 1 --2- 172.21.15.43:0/1414423250 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f6fa00047c0 0x7f6fb8156340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T15:11:08.565 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:08.560+0000 7f6fb77fe640 1 -- 172.21.15.43:0/1414423250 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f6fb80ccdd0 msgr2=0x7f6fb80c6d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T15:11:08.565 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:08.560+0000 7f6fb77fe640 1 --2- 172.21.15.43:0/1414423250 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f6fb80ccdd0 0x7f6fb80c6d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T15:11:08.565 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:08.560+0000 7f6fb77fe640 1 -- 172.21.15.43:0/1414423250 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6fb80cdd70 con 0x7f6fb80cd250 2023-08-09T15:11:08.565 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:08.560+0000 7f6fb6ffd640 1 --2- 172.21.15.43:0/1414423250 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f6fa00047c0 0x7f6fb8156340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-08-09T15:11:08.565 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:08.561+0000 7f6fb77fe640 1 --2- 172.21.15.43:0/1414423250 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f6fb80cd250 0x7f6fb80c7290 secure :-1 s=READY pgs=2299 cs=0 l=1 rev1=1 crypto rx=0x7f6fb806a790 tx=0x7f6fb000f520 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-09T15:11:08.565 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:08.561+0000 7f6f9f7fe640 1 -- 172.21.15.43:0/1414423250 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6fb0016620 con 0x7f6fb80cd250 2023-08-09T15:11:08.566 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:08.561+0000 7f6f9f7fe640 1 -- 172.21.15.43:0/1414423250 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6fb0016780 con 0x7f6fb80cd250 2023-08-09T15:11:08.566 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:08.561+0000 7f6f9f7fe640 1 -- 172.21.15.43:0/1414423250 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6fb0016b10 con 0x7f6fb80cd250 2023-08-09T15:11:08.566 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:08.561+0000 7f6fbd5d5640 1 -- 172.21.15.43:0/1414423250 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6fb80ce050 con 0x7f6fb80cd250 2023-08-09T15:11:08.566 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:08.561+0000 7f6fbd5d5640 1 -- 172.21.15.43:0/1414423250 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6fb8145f40 con 0x7f6fb80cd250 2023-08-09T15:11:08.567 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:08.563+0000 7f6fbd5d5640 1 -- 172.21.15.43:0/1414423250 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6fb80ce570 con 0x7f6fb80cd250 2023-08-09T15:11:08.567 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:08.563+0000 7f6f9f7fe640 1 -- 172.21.15.43:0/1414423250 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 115) v1 ==== 88052+0+0 (secure 0 0 0) 0x7f6fb0016c70 con 0x7f6fb80cd250 2023-08-09T15:11:08.567 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:08.564+0000 7f6f9f7fe640 1 -- 172.21.15.43:0/1414423250 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(211..211 src has 1..211) v4 ==== 7200+0+0 (secure 0 0 0) 0x7f6fb0089250 con 0x7f6fb80cd250 2023-08-09T15:11:08.570 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:08.569+0000 7f6f9f7fe640 1 -- 172.21.15.43:0/1414423250 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178066 (secure 0 0 0) 0x7f6fb80ce570 con 0x7f6fb80cd250 2023-08-09T15:11:08.573 INFO:tasks.ceph.mgr.y.smithi043.stderr:2023-08-09T15:11:08.572+0000 7fb51ee0f040 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2023-08-09T15:11:08.764 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:08.763+0000 7f6fbd5d5640 1 -- 172.21.15.43:0/1414423250 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f6fb80ce570 con 0x7f6fb80cd250 2023-08-09T15:11:08.766 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:08.765+0000 7f6f9f7fe640 1 -- 172.21.15.43:0/1414423250 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v211) v1 ==== 74+0+19745 (secure 0 0 0) 0x7f6fb80ce570 con 0x7f6fb80cd250 2023-08-09T15:11:08.767 INFO:teuthology.orchestra.run.smithi043.stdout: 2023-08-09T15:11:08.767 INFO:teuthology.orchestra.run.smithi043.stdout:{"epoch":211,"fsid":"daac87ee-1337-4250-a955-aea4e408a39b","created":"2023-08-09T13:33:41.701198+0000","modified":"2023-08-09T15:11:08.350101+0000","last_up_change":"2023-08-09T13:33:53.783453+0000","last_in_change":"2023-08-09T13:33:45.148904+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":25,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":"rbd","create_time":"2023-08-09T13:33:54.365137+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":".mgr","create_time":"2023-08-09T13:33:54.831803+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":24,"pool_name":"cephfs_metadata","create_time":"2023-08-09T15:08:43.920337+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"204","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":25,"pool_name":"cephfs_data","create_time":"2023-08-09T15:08:44.594489+0000","flags":8193,"flags_names":"hashpspool,selfmanaged_snaps","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"210","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":5,"snap_epoch":210,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"cbd49ecd-e896-4fbd-864e-5c37a829b165","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":203,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6800","nonce":502667335},{"type":"v1","addr":"172.21.15.43:6801","nonce":502667335}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6802","nonce":502667335},{"type":"v1","addr":"172.21.15.43:6803","nonce":502667335}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6806","nonce":502667335},{"type":"v1","addr":"172.21.15.43:6807","nonce":502667335}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6804","nonce":502667335},{"type":"v1","addr":"172.21.15.43:6805","nonce":502667335}]},"public_addr":"172.21.15.43:6801/502667335","cluster_addr":"172.21.15.43:6803/502667335","heartbeat_back_addr":"172.21.15.43:6807/502667335","heartbeat_front_addr":"172.21.15.43:6805/502667335","state":["exists","up"]},{"osd":1,"uuid":"75c7e0fe-1ec7-4fe8-a2f2-3661a65b8500","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":203,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6808","nonce":3289134610},{"type":"v1","addr":"172.21.15.43:6809","nonce":3289134610}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6810","nonce":3289134610},{"type":"v1","addr":"172.21.15.43:6812","nonce":3289134610}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6821","nonce":3289134610},{"type":"v1","addr":"172.21.15.43:6824","nonce":3289134610}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6815","nonce":3289134610},{"type":"v1","addr":"172.21.15.43:6818","nonce":3289134610}]},"public_addr":"172.21.15.43:6809/3289134610","cluster_addr":"172.21.15.43:6812/3289134610","heartbeat_back_addr":"172.21.15.43:6824/3289134610","heartbeat_front_addr":"172.21.15.43:6818/3289134610","state":["exists","up"]},{"osd":2,"uuid":"bb5c41e9-7f5f-4125-9b01-3bb8bfc2ea83","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":203,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6811","nonce":4079709736},{"type":"v1","addr":"172.21.15.43:6814","nonce":4079709736}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6817","nonce":4079709736},{"type":"v1","addr":"172.21.15.43:6820","nonce":4079709736}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6829","nonce":4079709736},{"type":"v1","addr":"172.21.15.43:6831","nonce":4079709736}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6823","nonce":4079709736},{"type":"v1","addr":"172.21.15.43:6826","nonce":4079709736}]},"public_addr":"172.21.15.43:6814/4079709736","cluster_addr":"172.21.15.43:6820/4079709736","heartbeat_back_addr":"172.21.15.43:6831/4079709736","heartbeat_front_addr":"172.21.15.43:6826/4079709736","state":["exists","up"]},{"osd":3,"uuid":"bfe4c8b4-694a-465c-87e6-2b2aab852613","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":203,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6813","nonce":1709538759},{"type":"v1","addr":"172.21.15.43:6816","nonce":1709538759}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6819","nonce":1709538759},{"type":"v1","addr":"172.21.15.43:6822","nonce":1709538759}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6828","nonce":1709538759},{"type":"v1","addr":"172.21.15.43:6830","nonce":1709538759}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6825","nonce":1709538759},{"type":"v1","addr":"172.21.15.43:6827","nonce":1709538759}]},"public_addr":"172.21.15.43:6816/1709538759","cluster_addr":"172.21.15.43:6822/1709538759","heartbeat_back_addr":"172.21.15.43:6830/1709538759","heartbeat_front_addr":"172.21.15.43:6827/1709538759","state":["exists","up"]},{"osd":4,"uuid":"fa2e8a77-aba1-4756-853e-0d174a5bdeba","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":203,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6800","nonce":4196866481},{"type":"v1","addr":"172.21.15.203:6801","nonce":4196866481}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6802","nonce":4196866481},{"type":"v1","addr":"172.21.15.203:6803","nonce":4196866481}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6806","nonce":4196866481},{"type":"v1","addr":"172.21.15.203:6807","nonce":4196866481}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6804","nonce":4196866481},{"type":"v1","addr":"172.21.15.203:6805","nonce":4196866481}]},"public_addr":"172.21.15.203:6801/4196866481","cluster_addr":"172.21.15.203:6803/4196866481","heartbeat_back_addr":"172.21.15.203:6807/4196866481","heartbeat_front_addr":"172.21.15.203:6805/4196866481","state":["exists","up"]},{"osd":5,"uuid":"81811e4b-4d24-4f51-a82c-f1d700a2a00d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":207,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6816","nonce":3059290081},{"type":"v1","addr":"172.21.15.203:6817","nonce":3059290081}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6818","nonce":3059290081},{"type":"v1","addr":"172.21.15.203:6819","nonce":3059290081}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6822","nonce":3059290081},{"type":"v1","addr":"172.21.15.203:6823","nonce":3059290081}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6820","nonce":3059290081},{"type":"v1","addr":"172.21.15.203:6821","nonce":3059290081}]},"public_addr":"172.21.15.203:6817/3059290081","cluster_addr":"172.21.15.203:6819/3059290081","heartbeat_back_addr":"172.21.15.203:6823/3059290081","heartbeat_front_addr":"172.21.15.203:6821/3059290081","state":["exists","up"]},{"osd":6,"uuid":"307bac4d-6a5b-443d-af75-56d695d7674f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":203,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6824","nonce":512155569},{"type":"v1","addr":"172.21.15.203:6825","nonce":512155569}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6826","nonce":512155569},{"type":"v1","addr":"172.21.15.203:6827","nonce":512155569}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6830","nonce":512155569},{"type":"v1","addr":"172.21.15.203:6831","nonce":512155569}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6828","nonce":512155569},{"type":"v1","addr":"172.21.15.203:6829","nonce":512155569}]},"public_addr":"172.21.15.203:6825/512155569","cluster_addr":"172.21.15.203:6827/512155569","heartbeat_back_addr":"172.21.15.203:6831/512155569","heartbeat_front_addr":"172.21.15.203:6829/512155569","state":["exists","up"]},{"osd":7,"uuid":"a95dc7ca-9f1b-4295-af70-68a22a2d7d5e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":207,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6808","nonce":3331450438},{"type":"v1","addr":"172.21.15.203:6809","nonce":3331450438}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6810","nonce":3331450438},{"type":"v1","addr":"172.21.15.203:6811","nonce":3331450438}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6814","nonce":3331450438},{"type":"v1","addr":"172.21.15.203:6815","nonce":3331450438}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6812","nonce":3331450438},{"type":"v1","addr":"172.21.15.203:6813","nonce":3331450438}]},"public_addr":"172.21.15.203:6809/3331450438","cluster_addr":"172.21.15.203:6811/3331450438","heartbeat_back_addr":"172.21.15.203:6815/3331450438","heartbeat_front_addr":"172.21.15.203:6813/3331450438","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-09T13:33:50.547652+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-09T13:33:50.589469+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-09T13:33:50.561724+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-09T13:33:50.599650+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-09T13:33:50.411855+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-09T13:33:50.219681+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-09T13:33:50.504664+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-09T13:33:50.441248+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"25.f","mappings":[{"from":0,"to":1}]},{"pgid":"25.3f","mappings":[{"from":0,"to":2}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.203:0/822076630":"2023-08-10T15:11:08.350045+0000","172.21.15.203:0/79247817":"2023-08-10T15:11:08.350045+0000","172.21.15.203:6833/4012896436":"2023-08-10T15:11:08.350045+0000","172.21.15.203:6832/4012896436":"2023-08-10T15:11:08.350045+0000","172.21.15.203:0/3345703784":"2023-08-10T15:11:08.350045+0000","172.21.15.203:0/1987920954":"2023-08-10T15:11:08.350045+0000","172.21.15.203:0/4253505338":"2023-08-10T15:11:08.350045+0000","172.21.15.203:6833/3082732769":"2023-08-10T15:09:01.285901+0000","172.21.15.203:6832/3082732769":"2023-08-10T15:09:01.285901+0000","172.21.15.203:0/1756781921":"2023-08-10T15:09:01.285901+0000","172.21.15.203:0/454071895":"2023-08-10T15:09:01.285901+0000","172.21.15.203:0/2854569557":"2023-08-10T15:09:01.285901+0000","172.21.15.203:0/2821325802":"2023-08-10T15:11:08.350045+0000","172.21.15.203:0/1159944241":"2023-08-10T15:09:01.285901+0000","172.21.15.203:0/1645632393":"2023-08-10T15:09:01.285901+0000","172.21.15.203:0/840449585":"2023-08-10T15:09:01.285901+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":7,"snaps":[{"begin":104,"length":1},{"begin":221,"length":1}]},{"pool":25,"snaps":[{"begin":2,"length":4}]}],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-08-09T15:11:08.769 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:08.768+0000 7f6fbd5d5640 1 -- 172.21.15.43:0/1414423250 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f6fb80cd250 msgr2=0x7f6fb80c7290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T15:11:08.769 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:08.768+0000 7f6fbd5d5640 1 --2- 172.21.15.43:0/1414423250 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f6fb80cd250 0x7f6fb80c7290 secure :-1 s=READY pgs=2299 cs=0 l=1 rev1=1 crypto rx=0x7f6fb806a790 tx=0x7f6fb000f520 comp rx=0 tx=0).stop 2023-08-09T15:11:08.770 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:08.768+0000 7f6fbd5d5640 1 -- 172.21.15.43:0/1414423250 shutdown_connections 2023-08-09T15:11:08.770 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:08.768+0000 7f6fbd5d5640 1 --2- 172.21.15.43:0/1414423250 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f6fb80cd250 0x7f6fb80c7290 unknown :-1 s=CLOSED pgs=2299 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T15:11:08.770 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:08.768+0000 7f6fbd5d5640 1 --2- 172.21.15.43:0/1414423250 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f6fb80ccdd0 0x7f6fb80c6d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T15:11:08.770 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:08.768+0000 7f6fbd5d5640 1 --2- 172.21.15.43:0/1414423250 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f6fa00047c0 0x7f6fb8156340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T15:11:08.770 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:08.768+0000 7f6fbd5d5640 1 -- 172.21.15.43:0/1414423250 >> 172.21.15.43:0/1414423250 conn(0x7f6fb80cbc80 msgr2=0x7f6fb806ea10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-09T15:11:08.771 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:08.768+0000 7f6fbd5d5640 1 -- 172.21.15.43:0/1414423250 shutdown_connections 2023-08-09T15:11:08.771 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:08.768+0000 7f6fbd5d5640 1 -- 172.21.15.43:0/1414423250 wait complete. 2023-08-09T15:11:08.788 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': 'rbd', 'create_time': '2023-08-09T13:33:54.365137+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}, 'read_balance': {'score_acting': 2, 'score_stable': 2, 'optimal_score': 1, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': '.mgr', 'create_time': '2023-08-09T13:33:54.831803+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 24, 'pool_name': 'cephfs_metadata', 'create_time': '2023-08-09T15:08:43.920337+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '204', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 25, 'pool_name': 'cephfs_data', 'create_time': '2023-08-09T15:08:44.594489+0000', 'flags': 8193, 'flags_names': 'hashpspool,selfmanaged_snaps', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '210', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 5, 'snap_epoch': 210, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.3799999952316284, 'score_stable': 1.3799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.3799999952316284, 'raw_score_stable': 1.3799999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-08-09T15:11:08.789 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get rbd pg_num 2023-08-09T15:11:09.015 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:09.014+0000 7fa30a370640 1 Processor -- start 2023-08-09T15:11:09.015 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:09.014+0000 7fa30a370640 1 -- start start 2023-08-09T15:11:09.016 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:09.015+0000 7fa30a370640 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7fa30415a1b0 0x7fa3040b1ad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T15:11:09.017 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:09.015+0000 7fa30a370640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7fa30415e9c0 0x7fa3040b2010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T15:11:09.017 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:09.015+0000 7fa30a370640 1 -- --> v1:172.21.15.43:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa3040cc6c0 con 0x7fa3040cd630 2023-08-09T15:11:09.017 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:09.015+0000 7fa30a370640 1 -- --> v2:172.21.15.43:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa30415dee0 con 0x7fa30415e9c0 2023-08-09T15:11:09.017 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:09.015+0000 7fa30a370640 1 -- --> v2:172.21.15.203:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa30415e1c0 con 0x7fa30415a1b0 2023-08-09T15:11:09.018 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:09.016+0000 7fa309b6f640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7fa30415e9c0 0x7fa3040b2010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T15:11:09.018 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:09.016+0000 7fa308b6d640 1 --1- >> v1:172.21.15.43:6789/0 conn(0x7fa3040cd630 0x7fa3040cdc20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.43:6789/0 says I am v1:172.21.15.43:45542/0 (socket says 172.21.15.43:45542) 2023-08-09T15:11:09.018 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:09.016+0000 7fa308b6d640 1 -- 172.21.15.43:0/806895435 learned_addr learned my addr 172.21.15.43:0/806895435 (peer_addr_for_me v1:172.21.15.43:0/0) 2023-08-09T15:11:09.018 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:09.016+0000 7fa30936e640 1 --2- 172.21.15.43:0/806895435 >> v2:172.21.15.203:3300/0 conn(0x7fa30415a1b0 0x7fa3040b1ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T15:11:09.018 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:09.016+0000 7fa309b6f640 1 -- 172.21.15.43:0/806895435 >> v2:172.21.15.203:3300/0 conn(0x7fa30415a1b0 msgr2=0x7fa3040b1ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-09T15:11:09.019 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:09.016+0000 7fa309b6f640 1 --2- 172.21.15.43:0/806895435 >> v2:172.21.15.203:3300/0 conn(0x7fa30415a1b0 0x7fa3040b1ad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T15:11:09.019 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:09.016+0000 7fa309b6f640 1 -- 172.21.15.43:0/806895435 >> v1:172.21.15.43:6789/0 conn(0x7fa3040cd630 legacy=0x7fa3040cdc20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T15:11:09.019 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:09.016+0000 7fa309b6f640 1 -- 172.21.15.43:0/806895435 --> v2:172.21.15.43:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa30415e4a0 con 0x7fa30415e9c0 2023-08-09T15:11:09.019 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:09.016+0000 7fa30936e640 1 --2- 172.21.15.43:0/806895435 >> v2:172.21.15.203:3300/0 conn(0x7fa30415a1b0 0x7fa3040b1ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-08-09T15:11:09.020 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:09.018+0000 7fa309b6f640 1 --2- 172.21.15.43:0/806895435 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa30415e9c0 0x7fa3040b2010 secure :-1 s=READY pgs=2301 cs=0 l=1 rev1=1 crypto rx=0x7fa30406a7c0 tx=0x7fa2fc02bec0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3762bf490a6e6236 server_cookie=0 in_seq=0 out_seq=0 2023-08-09T15:11:09.020 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:09.018+0000 7fa303fff640 1 -- 172.21.15.43:0/806895435 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa2fc02cc70 con 0x7fa30415e9c0 2023-08-09T15:11:09.021 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:09.018+0000 7fa303fff640 1 -- 172.21.15.43:0/806895435 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa2fc02cdd0 con 0x7fa30415e9c0 2023-08-09T15:11:09.021 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:09.018+0000 7fa303fff640 1 -- 172.21.15.43:0/806895435 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa2fc034660 con 0x7fa30415e9c0 2023-08-09T15:11:09.021 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:09.019+0000 7fa30a370640 1 -- 172.21.15.43:0/806895435 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa30415e9c0 msgr2=0x7fa3040b2010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T15:11:09.021 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:09.019+0000 7fa30a370640 1 --2- 172.21.15.43:0/806895435 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa30415e9c0 0x7fa3040b2010 secure :-1 s=READY pgs=2301 cs=0 l=1 rev1=1 crypto rx=0x7fa30406a7c0 tx=0x7fa2fc02bec0 comp rx=0 tx=0).stop 2023-08-09T15:11:09.021 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:09.019+0000 7fa30a370640 1 -- 172.21.15.43:0/806895435 shutdown_connections 2023-08-09T15:11:09.021 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:09.019+0000 7fa30a370640 1 --2- 172.21.15.43:0/806895435 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa30415e9c0 0x7fa3040b2010 unknown :-1 s=CLOSED pgs=2301 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T15:11:09.021 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:09.019+0000 7fa30a370640 1 --2- 172.21.15.43:0/806895435 >> v2:172.21.15.203:3300/0 conn(0x7fa30415a1b0 0x7fa3040b1ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T15:11:09.021 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:09.019+0000 7fa30a370640 1 -- 172.21.15.43:0/806895435 >> 172.21.15.43:0/806895435 conn(0x7fa3040ccad0 msgr2=0x7fa3040cceb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-09T15:11:09.022 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:09.019+0000 7fa30a370640 1 -- 172.21.15.43:0/806895435 shutdown_connections 2023-08-09T15:11:09.022 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:09.019+0000 7fa30a370640 1 -- 172.21.15.43:0/806895435 wait complete. 2023-08-09T15:11:09.022 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:09.020+0000 7fa30a370640 1 Processor -- start 2023-08-09T15:11:09.022 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:09.021+0000 7fa30a370640 1 -- start start 2023-08-09T15:11:09.023 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:09.022+0000 7fa30a370640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fa3040cd630 0x7fa3040aefc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T15:11:09.023 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:09.022+0000 7fa30a370640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa30415a1b0 0x7fa30415cd90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T15:11:09.024 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:09.022+0000 7fa30a370640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fa30415e9c0 0x7fa30406a8d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T15:11:09.024 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:09.022+0000 7fa30a370640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa3040c5790 con 0x7fa30415a1b0 2023-08-09T15:11:09.024 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:09.022+0000 7fa30a370640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa3040af500 con 0x7fa30415e9c0 2023-08-09T15:11:09.024 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:09.022+0000 7fa30a370640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa3040c4b80 con 0x7fa3040cd630 2023-08-09T15:11:09.024 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:09.022+0000 7fa308b6d640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa30415a1b0 0x7fa30415cd90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T15:11:09.024 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:09.022+0000 7fa308b6d640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa30415a1b0 0x7fa30415cd90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:46482/0 (socket says 172.21.15.43:46482) 2023-08-09T15:11:09.025 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:09.022+0000 7fa308b6d640 1 -- 172.21.15.43:0/3499680771 learned_addr learned my addr 172.21.15.43:0/3499680771 (peer_addr_for_me v2:172.21.15.43:0/0) 2023-08-09T15:11:09.025 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:09.022+0000 7fa30936e640 1 --2- 172.21.15.43:0/3499680771 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fa3040cd630 0x7fa3040aefc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T15:11:09.026 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:09.022+0000 7fa308b6d640 1 -- 172.21.15.43:0/3499680771 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fa3040cd630 msgr2=0x7fa3040aefc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T15:11:09.026 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:09.022+0000 7fa308b6d640 1 --2- 172.21.15.43:0/3499680771 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fa3040cd630 0x7fa3040aefc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T15:11:09.026 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:09.022+0000 7fa308b6d640 1 -- 172.21.15.43:0/3499680771 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fa30415e9c0 msgr2=0x7fa30406a8d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T15:11:09.026 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:09.022+0000 7fa308b6d640 1 --2- 172.21.15.43:0/3499680771 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fa30415e9c0 0x7fa30406a8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T15:11:09.026 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:09.022+0000 7fa308b6d640 1 -- 172.21.15.43:0/3499680771 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa2fc009a70 con 0x7fa30415a1b0 2023-08-09T15:11:09.026 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:09.023+0000 7fa308b6d640 1 --2- 172.21.15.43:0/3499680771 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa30415a1b0 0x7fa30415cd90 secure :-1 s=READY pgs=2302 cs=0 l=1 rev1=1 crypto rx=0x7fa2f800c4a0 tx=0x7fa2f800c960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-09T15:11:09.027 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:09.023+0000 7fa301ffb640 1 -- 172.21.15.43:0/3499680771 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa2f8005ce0 con 0x7fa30415a1b0 2023-08-09T15:11:09.027 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:09.023+0000 7fa301ffb640 1 -- 172.21.15.43:0/3499680771 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa2f8005e40 con 0x7fa30415a1b0 2023-08-09T15:11:09.027 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:09.023+0000 7fa301ffb640 1 -- 172.21.15.43:0/3499680771 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa2f8007270 con 0x7fa30415a1b0 2023-08-09T15:11:09.027 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:09.023+0000 7fa30a370640 1 -- 172.21.15.43:0/3499680771 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa3040c4e60 con 0x7fa30415a1b0 2023-08-09T15:11:09.027 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:09.023+0000 7fa30a370640 1 -- 172.21.15.43:0/3499680771 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa3040c5420 con 0x7fa30415a1b0 2023-08-09T15:11:09.027 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:09.024+0000 7fa30a370640 1 -- 172.21.15.43:0/3499680771 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa3040c5900 con 0x7fa30415a1b0 2023-08-09T15:11:09.028 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:09.026+0000 7fa301ffb640 1 -- 172.21.15.43:0/3499680771 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 115) v1 ==== 88052+0+0 (secure 0 0 0) 0x7fa2f8007490 con 0x7fa30415a1b0 2023-08-09T15:11:09.028 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:09.027+0000 7fa301ffb640 1 -- 172.21.15.43:0/3499680771 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(211..211 src has 1..211) v4 ==== 7200+0+0 (secure 0 0 0) 0x7fa2f8087950 con 0x7fa30415a1b0 2023-08-09T15:11:09.032 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:09.031+0000 7fa301ffb640 1 -- 172.21.15.43:0/3499680771 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178066 (secure 0 0 0) 0x7fa2f8053040 con 0x7fa30415a1b0 2023-08-09T15:11:09.231 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:09.230+0000 7fa30a370640 1 -- 172.21.15.43:0/3499680771 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"} v 0) v1 -- 0x7fa30404e390 con 0x7fa30415a1b0 2023-08-09T15:11:09.232 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:09.231+0000 7fa301ffb640 1 -- 172.21.15.43:0/3499680771 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"}]=0 v211) v1 ==== 92+0+10 (secure 0 0 0) 0x7fa2f8017020 con 0x7fa30415a1b0 2023-08-09T15:11:09.232 INFO:teuthology.orchestra.run.smithi043.stdout:pg_num: 8 2023-08-09T15:11:09.234 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:09.233+0000 7fa30a370640 1 -- 172.21.15.43:0/3499680771 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa30415a1b0 msgr2=0x7fa30415cd90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T15:11:09.235 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:09.233+0000 7fa30a370640 1 --2- 172.21.15.43:0/3499680771 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa30415a1b0 0x7fa30415cd90 secure :-1 s=READY pgs=2302 cs=0 l=1 rev1=1 crypto rx=0x7fa2f800c4a0 tx=0x7fa2f800c960 comp rx=0 tx=0).stop 2023-08-09T15:11:09.235 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:09.233+0000 7fa30a370640 1 -- 172.21.15.43:0/3499680771 shutdown_connections 2023-08-09T15:11:09.235 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:09.233+0000 7fa30a370640 1 --2- 172.21.15.43:0/3499680771 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fa30415e9c0 0x7fa30406a8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T15:11:09.235 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:09.233+0000 7fa30a370640 1 --2- 172.21.15.43:0/3499680771 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa30415a1b0 0x7fa30415cd90 unknown :-1 s=CLOSED pgs=2302 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T15:11:09.235 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:09.233+0000 7fa30a370640 1 --2- 172.21.15.43:0/3499680771 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fa3040cd630 0x7fa3040aefc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T15:11:09.235 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:09.233+0000 7fa30a370640 1 -- 172.21.15.43:0/3499680771 >> 172.21.15.43:0/3499680771 conn(0x7fa3040ccad0 msgr2=0x7fa3040b1990 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-09T15:11:09.236 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:09.233+0000 7fa30a370640 1 -- 172.21.15.43:0/3499680771 shutdown_connections 2023-08-09T15:11:09.236 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:09.233+0000 7fa30a370640 1 -- 172.21.15.43:0/3499680771 wait complete. 2023-08-09T15:11:09.254 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-08-09T15:11:09.481 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:09.480+0000 7f51e3fff640 1 Processor -- start 2023-08-09T15:11:09.482 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:09.480+0000 7f51e3fff640 1 -- start start 2023-08-09T15:11:09.483 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:09.482+0000 7f51e3fff640 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7f51e40c9ee0 0x7f51e415a800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T15:11:09.483 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:09.482+0000 7f51e3fff640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f51e40c90a0 0x7f51e415ad40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T15:11:09.483 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:09.482+0000 7f51e3fff640 1 -- --> v1:172.21.15.43:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f51e4158280 con 0x7f51e4157ea0 2023-08-09T15:11:09.483 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:09.482+0000 7f51e3fff640 1 -- --> v2:172.21.15.203:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f51e415b840 con 0x7f51e40c9ee0 2023-08-09T15:11:09.484 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:09.482+0000 7f51e3fff640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f51e40b3810 con 0x7f51e40c90a0 2023-08-09T15:11:09.484 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:09.482+0000 7f51e2ffd640 1 --1- >> v1:172.21.15.43:6789/0 conn(0x7f51e4157ea0 0x7f51e40c97d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.43:6789/0 says I am v1:172.21.15.43:45544/0 (socket says 172.21.15.43:45544) 2023-08-09T15:11:09.484 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:09.482+0000 7f51e2ffd640 1 -- 172.21.15.43:0/4142410943 learned_addr learned my addr 172.21.15.43:0/4142410943 (peer_addr_for_me v1:172.21.15.43:0/0) 2023-08-09T15:11:09.485 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:09.482+0000 7f51e27fc640 1 -- 172.21.15.43:0/4142410943 <== mon.0 v1:172.21.15.43:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2495771532 0 0) 0x7f51d4001110 con 0x7f51e4157ea0 2023-08-09T15:11:09.485 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:09.483+0000 7f51dbfff640 1 --2- 172.21.15.43:0/4142410943 >> v2:172.21.15.203:3300/0 conn(0x7f51e40c9ee0 0x7f51e415a800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T15:11:09.486 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:09.483+0000 7f51e37fe640 1 --2- 172.21.15.43:0/4142410943 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f51e40c90a0 0x7f51e415ad40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T15:11:09.486 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:09.483+0000 7f51e27fc640 1 -- 172.21.15.43:0/4142410943 --> v1:172.21.15.43:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f51d0003520 con 0x7f51e4157ea0 2023-08-09T15:11:09.486 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:09.483+0000 7f51e27fc640 1 -- 172.21.15.43:0/4142410943 <== mon.0 v1:172.21.15.43:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 553745189 0 0) 0x7f51d0003520 con 0x7f51e4157ea0 2023-08-09T15:11:09.486 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:09.483+0000 7f51e27fc640 1 -- 172.21.15.43:0/4142410943 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f51e40c90a0 msgr2=0x7f51e415ad40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-09T15:11:09.486 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:09.483+0000 7f51e27fc640 1 --2- 172.21.15.43:0/4142410943 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f51e40c90a0 0x7f51e415ad40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T15:11:09.486 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:09.483+0000 7f51e27fc640 1 -- 172.21.15.43:0/4142410943 >> v2:172.21.15.203:3300/0 conn(0x7f51e40c9ee0 msgr2=0x7f51e415a800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-09T15:11:09.486 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:09.483+0000 7f51e27fc640 1 --2- 172.21.15.43:0/4142410943 >> v2:172.21.15.203:3300/0 conn(0x7f51e40c9ee0 0x7f51e415a800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T15:11:09.487 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:09.483+0000 7f51e27fc640 1 -- 172.21.15.43:0/4142410943 --> v1:172.21.15.43:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f51e415b560 con 0x7f51e4157ea0 2023-08-09T15:11:09.487 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:09.483+0000 7f51e27fc640 1 -- 172.21.15.43:0/4142410943 <== mon.0 v1:172.21.15.43:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1285150740 0 0) 0x7f51d4001800 con 0x7f51e4157ea0 2023-08-09T15:11:09.487 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:09.483+0000 7f51e27fc640 1 -- 172.21.15.43:0/4142410943 >> v1:172.21.15.43:6789/0 conn(0x7f51e4157ea0 legacy=0x7f51e40c97d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T15:11:09.487 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:09.483+0000 7f51e27fc640 1 --2- 172.21.15.43:0/4142410943 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f51d00039a0 0x7f51d0003dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T15:11:09.487 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:09.483+0000 7f51e27fc640 1 --2- 172.21.15.43:0/4142410943 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f51d00047c0 0x7f51d0004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T15:11:09.487 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:09.483+0000 7f51e27fc640 1 --2- 172.21.15.43:0/4142410943 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f51e40c90a0 0x7f51d0005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T15:11:09.487 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:09.483+0000 7f51e27fc640 1 -- 172.21.15.43:0/4142410943 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f51e415b840 con 0x7f51d00047c0 2023-08-09T15:11:09.487 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:09.483+0000 7f51e27fc640 1 -- 172.21.15.43:0/4142410943 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f51e40b3810 con 0x7f51d00039a0 2023-08-09T15:11:09.488 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:09.483+0000 7f51e27fc640 1 -- 172.21.15.43:0/4142410943 --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f51d000bb00 con 0x7f51e40c90a0 2023-08-09T15:11:09.488 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:09.483+0000 7f51dbfff640 1 --2- 172.21.15.43:0/4142410943 >> v2:172.21.15.203:3300/0 conn(0x7f51e40c9ee0 0x7f51e415a800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-08-09T15:11:09.488 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:09.483+0000 7f51dbfff640 1 --2- 172.21.15.43:0/4142410943 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f51d00047c0 0x7f51d0004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T15:11:09.488 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:09.484+0000 7f51e2ffd640 1 --2- 172.21.15.43:0/4142410943 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f51d00039a0 0x7f51d0003dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T15:11:09.488 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:09.484+0000 7f51e37fe640 1 --2- 172.21.15.43:0/4142410943 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f51e40c90a0 0x7f51d0005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T15:11:09.488 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:09.484+0000 7f51dbfff640 1 -- 172.21.15.43:0/4142410943 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f51e40c90a0 msgr2=0x7f51d0005180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-09T15:11:09.488 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:09.484+0000 7f51dbfff640 1 --2- 172.21.15.43:0/4142410943 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f51e40c90a0 0x7f51d0005180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T15:11:09.489 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:09.484+0000 7f51dbfff640 1 -- 172.21.15.43:0/4142410943 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f51d00039a0 msgr2=0x7f51d0003dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-09T15:11:09.489 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:09.484+0000 7f51dbfff640 1 --2- 172.21.15.43:0/4142410943 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f51d00039a0 0x7f51d0003dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T15:11:09.489 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:09.484+0000 7f51dbfff640 1 -- 172.21.15.43:0/4142410943 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f51d0003520 con 0x7f51d00047c0 2023-08-09T15:11:09.490 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:09.485+0000 7f51dbfff640 1 --2- 172.21.15.43:0/4142410943 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f51d00047c0 0x7f51d0004c40 secure :-1 s=READY pgs=2304 cs=0 l=1 rev1=1 crypto rx=0x7f51e4071780 tx=0x7f51d00057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b006b962d2d2015e server_cookie=0 in_seq=0 out_seq=0 2023-08-09T15:11:09.490 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:09.486+0000 7f51e27fc640 1 -- 172.21.15.43:0/4142410943 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f51cc030660 con 0x7f51d00047c0 2023-08-09T15:11:09.490 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:09.486+0000 7f51e27fc640 1 -- 172.21.15.43:0/4142410943 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f51cc0307c0 con 0x7f51d00047c0 2023-08-09T15:11:09.490 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:09.486+0000 7f51e27fc640 1 -- 172.21.15.43:0/4142410943 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f51cc030b30 con 0x7f51d00047c0 2023-08-09T15:11:09.490 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:09.486+0000 7f51e3fff640 1 -- 172.21.15.43:0/4142410943 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f51d00047c0 msgr2=0x7f51d0004c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T15:11:09.491 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:09.486+0000 7f51e3fff640 1 --2- 172.21.15.43:0/4142410943 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f51d00047c0 0x7f51d0004c40 secure :-1 s=READY pgs=2304 cs=0 l=1 rev1=1 crypto rx=0x7f51e4071780 tx=0x7f51d00057f0 comp rx=0 tx=0).stop 2023-08-09T15:11:09.491 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:09.486+0000 7f51e37fe640 1 -- 172.21.15.43:0/4142410943 reap_dead start 2023-08-09T15:11:09.491 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:09.486+0000 7f51e3fff640 1 -- 172.21.15.43:0/4142410943 shutdown_connections 2023-08-09T15:11:09.491 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:09.486+0000 7f51e3fff640 1 -- 172.21.15.43:0/4142410943 >> 172.21.15.43:0/4142410943 conn(0x7f51e40cddd0 msgr2=0x7f51e40ce1e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-09T15:11:09.491 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:09.486+0000 7f51e3fff640 1 -- 172.21.15.43:0/4142410943 shutdown_connections 2023-08-09T15:11:09.491 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:09.486+0000 7f51e3fff640 1 -- 172.21.15.43:0/4142410943 wait complete. 2023-08-09T15:11:09.491 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:09.488+0000 7f51e3fff640 1 Processor -- start 2023-08-09T15:11:09.491 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:09.488+0000 7f51e3fff640 1 -- start start 2023-08-09T15:11:09.493 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:09.489+0000 7f51e3fff640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f51d00047c0 0x7f51e406be30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T15:11:09.493 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:09.489+0000 7f51e3fff640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f51e40c68d0 0x7f51e40c6d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T15:11:09.493 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:09.489+0000 7f51e3fff640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f51e4153fb0 0x7f51e4154430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T15:11:09.493 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:09.489+0000 7f51e3fff640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f51e40c5cc0 con 0x7f51e4153fb0 2023-08-09T15:11:09.494 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:09.489+0000 7f51e3fff640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f51e40c7290 con 0x7f51e40c68d0 2023-08-09T15:11:09.494 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:09.489+0000 7f51e3fff640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f51e4154970 con 0x7f51d00047c0 2023-08-09T15:11:09.494 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:09.490+0000 7f51e37fe640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f51e4153fb0 0x7f51e4154430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T15:11:09.494 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:09.490+0000 7f51e37fe640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f51e4153fb0 0x7f51e4154430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:46494/0 (socket says 172.21.15.43:46494) 2023-08-09T15:11:09.494 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:09.490+0000 7f51e37fe640 1 -- 172.21.15.43:0/4210161380 learned_addr learned my addr 172.21.15.43:0/4210161380 (peer_addr_for_me v2:172.21.15.43:0/0) 2023-08-09T15:11:09.494 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:09.490+0000 7f51e2ffd640 1 --2- 172.21.15.43:0/4210161380 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f51d00047c0 0x7f51e406be30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T15:11:09.494 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:09.490+0000 7f51dbfff640 1 --2- 172.21.15.43:0/4210161380 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f51e40c68d0 0x7f51e40c6d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T15:11:09.494 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:09.490+0000 7f51e37fe640 1 -- 172.21.15.43:0/4210161380 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f51d00047c0 msgr2=0x7f51e406be30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T15:11:09.495 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:09.490+0000 7f51e37fe640 1 --2- 172.21.15.43:0/4210161380 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f51d00047c0 0x7f51e406be30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T15:11:09.495 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:09.490+0000 7f51e37fe640 1 -- 172.21.15.43:0/4210161380 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f51e40c68d0 msgr2=0x7f51e40c6d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T15:11:09.495 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:09.490+0000 7f51e37fe640 1 --2- 172.21.15.43:0/4210161380 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f51e40c68d0 0x7f51e40c6d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T15:11:09.495 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:09.490+0000 7f51e37fe640 1 -- 172.21.15.43:0/4210161380 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f51e40c5040 con 0x7f51e4153fb0 2023-08-09T15:11:09.495 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:09.490+0000 7f51dbfff640 1 --2- 172.21.15.43:0/4210161380 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f51e40c68d0 0x7f51e40c6d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-08-09T15:11:09.495 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:09.490+0000 7f51e37fe640 1 --2- 172.21.15.43:0/4210161380 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f51e4153fb0 0x7f51e4154430 secure :-1 s=READY pgs=2305 cs=0 l=1 rev1=1 crypto rx=0x7f51e40ca790 tx=0x7f51dc009cd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-09T15:11:09.495 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:09.491+0000 7f51e0ff9640 1 -- 172.21.15.43:0/4210161380 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f51dc007cc0 con 0x7f51e4153fb0 2023-08-09T15:11:09.496 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:09.491+0000 7f51e0ff9640 1 -- 172.21.15.43:0/4210161380 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f51dc00f050 con 0x7f51e4153fb0 2023-08-09T15:11:09.496 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:09.491+0000 7f51e0ff9640 1 -- 172.21.15.43:0/4210161380 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f51dc014650 con 0x7f51e4153fb0 2023-08-09T15:11:09.496 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:09.491+0000 7f51e3fff640 1 -- 172.21.15.43:0/4210161380 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f51e40c5320 con 0x7f51e4153fb0 2023-08-09T15:11:09.496 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:09.491+0000 7f51e3fff640 1 -- 172.21.15.43:0/4210161380 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f51e40c58e0 con 0x7f51e4153fb0 2023-08-09T15:11:09.497 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:09.492+0000 7f51e3fff640 1 -- 172.21.15.43:0/4210161380 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f51e40c5e00 con 0x7f51e4153fb0 2023-08-09T15:11:09.497 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:09.493+0000 7f51e0ff9640 1 -- 172.21.15.43:0/4210161380 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 116) v1 ==== 88179+0+0 (secure 0 0 0) 0x7f51dc0147b0 con 0x7f51e4153fb0 2023-08-09T15:11:09.497 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:09.494+0000 7f51e0ff9640 1 --2- 172.21.15.43:0/4210161380 >> [v2:172.21.15.203:6832/4034093268,v1:172.21.15.203:6833/4034093268] conn(0x7f51c4069ea0 0x7f51c406c360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T15:11:09.497 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:09.494+0000 7f51e0ff9640 1 -- 172.21.15.43:0/4210161380 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(211..211 src has 1..211) v4 ==== 7200+0+0 (secure 0 0 0) 0x7f51dc0889a0 con 0x7f51e4153fb0 2023-08-09T15:11:09.497 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:09.495+0000 7f51e2ffd640 1 --2- 172.21.15.43:0/4210161380 >> [v2:172.21.15.203:6832/4034093268,v1:172.21.15.203:6833/4034093268] conn(0x7f51c4069ea0 0x7f51c406c360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T15:11:09.497 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:09.496+0000 7f51e2ffd640 1 --2- 172.21.15.43:0/4210161380 >> [v2:172.21.15.203:6832/4034093268,v1:172.21.15.203:6833/4034093268] conn(0x7f51c4069ea0 0x7f51c406c360 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7f51d4001050 tx=0x7f51d4009040 comp rx=0 tx=0).ready entity=mgr.8833 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-09T15:11:09.500 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:09.498+0000 7f51e0ff9640 1 -- 172.21.15.43:0/4210161380 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178066 (secure 0 0 0) 0x7f51e40c5e00 con 0x7f51e4153fb0 2023-08-09T15:11:09.697 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:09.696+0000 7f51e3fff640 1 -- 172.21.15.43:0/4210161380 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f51e41ef860 con 0x7f51e4153fb0 2023-08-09T15:11:09.698 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:09.697+0000 7f51e0ff9640 1 -- 172.21.15.43:0/4210161380 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v211) v1 ==== 93+0+10 (secure 0 0 0) 0x7f51e41ef860 con 0x7f51e4153fb0 2023-08-09T15:11:09.699 INFO:teuthology.orchestra.run.smithi043.stdout:pg_num: 1 2023-08-09T15:11:09.701 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:09.699+0000 7f51e3fff640 1 -- 172.21.15.43:0/4210161380 >> [v2:172.21.15.203:6832/4034093268,v1:172.21.15.203:6833/4034093268] conn(0x7f51c4069ea0 msgr2=0x7f51c406c360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T15:11:09.701 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:09.699+0000 7f51e3fff640 1 --2- 172.21.15.43:0/4210161380 >> [v2:172.21.15.203:6832/4034093268,v1:172.21.15.203:6833/4034093268] conn(0x7f51c4069ea0 0x7f51c406c360 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7f51d4001050 tx=0x7f51d4009040 comp rx=0 tx=0).stop 2023-08-09T15:11:09.701 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:09.699+0000 7f51e3fff640 1 -- 172.21.15.43:0/4210161380 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f51e4153fb0 msgr2=0x7f51e4154430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T15:11:09.701 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:09.699+0000 7f51e3fff640 1 --2- 172.21.15.43:0/4210161380 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f51e4153fb0 0x7f51e4154430 secure :-1 s=READY pgs=2305 cs=0 l=1 rev1=1 crypto rx=0x7f51e40ca790 tx=0x7f51dc009cd0 comp rx=0 tx=0).stop 2023-08-09T15:11:09.701 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:09.699+0000 7f51e3fff640 1 -- 172.21.15.43:0/4210161380 shutdown_connections 2023-08-09T15:11:09.701 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:09.699+0000 7f51e3fff640 1 --2- 172.21.15.43:0/4210161380 >> [v2:172.21.15.203:6832/4034093268,v1:172.21.15.203:6833/4034093268] conn(0x7f51c4069ea0 0x7f51c406c360 unknown :-1 s=CLOSED pgs=20 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T15:11:09.701 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:09.699+0000 7f51e3fff640 1 --2- 172.21.15.43:0/4210161380 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f51e4153fb0 0x7f51e4154430 unknown :-1 s=CLOSED pgs=2305 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T15:11:09.702 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:09.699+0000 7f51e3fff640 1 --2- 172.21.15.43:0/4210161380 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f51e40c68d0 0x7f51e40c6d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T15:11:09.702 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:09.699+0000 7f51e3fff640 1 --2- 172.21.15.43:0/4210161380 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f51d00047c0 0x7f51e406be30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T15:11:09.702 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:09.699+0000 7f51e3fff640 1 -- 172.21.15.43:0/4210161380 >> 172.21.15.43:0/4210161380 conn(0x7f51e40cddd0 msgr2=0x7f51e40ce1e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-09T15:11:09.702 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:09.699+0000 7f51e3fff640 1 -- 172.21.15.43:0/4210161380 shutdown_connections 2023-08-09T15:11:09.702 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:09.699+0000 7f51e3fff640 1 -- 172.21.15.43:0/4210161380 wait complete. 2023-08-09T15:11:09.721 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-08-09T15:11:09.954 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:09.952+0000 7faf5612f640 1 Processor -- start 2023-08-09T15:11:09.954 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:09.953+0000 7faf5612f640 1 -- start start 2023-08-09T15:11:09.956 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:09.954+0000 7faf5612f640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7faf500d0a70 0x7faf500cd7f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T15:11:09.956 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:09.954+0000 7faf5612f640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7faf500cdd30 0x7faf500b3fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T15:11:09.956 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:09.954+0000 7faf5612f640 1 -- --> v1:172.21.15.43:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7faf500d0f70 con 0x7faf5015c680 2023-08-09T15:11:09.956 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:09.954+0000 7faf5612f640 1 -- --> v2:172.21.15.43:3300/0 -- mon_getmap magic: 0 v1 -- 0x7faf500ccd10 con 0x7faf500cdd30 2023-08-09T15:11:09.956 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:09.954+0000 7faf5612f640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7faf500ccff0 con 0x7faf500d0a70 2023-08-09T15:11:09.957 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:09.955+0000 7faf4ffff640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7faf500cdd30 0x7faf500b3fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T15:11:09.957 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:09.955+0000 7faf4f7fe640 1 --1- >> v1:172.21.15.43:6789/0 conn(0x7faf5015c680 0x7faf500ce760 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.43:6789/0 says I am v1:172.21.15.43:45558/0 (socket says 172.21.15.43:45558) 2023-08-09T15:11:09.957 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:09.955+0000 7faf4f7fe640 1 -- 172.21.15.43:0/436557573 learned_addr learned my addr 172.21.15.43:0/436557573 (peer_addr_for_me v1:172.21.15.43:0/0) 2023-08-09T15:11:09.958 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:09.955+0000 7faf4ffff640 1 -- 172.21.15.43:0/436557573 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7faf500d0a70 msgr2=0x7faf500cd7f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-09T15:11:09.958 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:09.955+0000 7faf4ffff640 1 --2- 172.21.15.43:0/436557573 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7faf500d0a70 0x7faf500cd7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T15:11:09.958 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:09.955+0000 7faf4ffff640 1 -- 172.21.15.43:0/436557573 >> v1:172.21.15.43:6789/0 conn(0x7faf5015c680 legacy=0x7faf500ce760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T15:11:09.958 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:09.955+0000 7faf4ffff640 1 -- 172.21.15.43:0/436557573 --> v2:172.21.15.43:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faf500cd2d0 con 0x7faf500cdd30 2023-08-09T15:11:09.958 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:09.956+0000 7faf4ffff640 1 --2- 172.21.15.43:0/436557573 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7faf500cdd30 0x7faf500b3fa0 secure :-1 s=READY pgs=2307 cs=0 l=1 rev1=1 crypto rx=0x7faf48009d90 tx=0x7faf4802beb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=11742ac0fd757258 server_cookie=0 in_seq=0 out_seq=0 2023-08-09T15:11:09.959 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:09.957+0000 7faf4effd640 1 -- 172.21.15.43:0/436557573 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faf4802cc60 con 0x7faf500cdd30 2023-08-09T15:11:09.959 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:09.957+0000 7faf4effd640 1 -- 172.21.15.43:0/436557573 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7faf4802cdc0 con 0x7faf500cdd30 2023-08-09T15:11:09.959 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:09.957+0000 7faf4effd640 1 -- 172.21.15.43:0/436557573 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faf48034660 con 0x7faf500cdd30 2023-08-09T15:11:09.960 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:09.957+0000 7faf5612f640 1 -- 172.21.15.43:0/436557573 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7faf500cdd30 msgr2=0x7faf500b3fa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T15:11:09.960 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:09.957+0000 7faf5612f640 1 --2- 172.21.15.43:0/436557573 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7faf500cdd30 0x7faf500b3fa0 secure :-1 s=READY pgs=2307 cs=0 l=1 rev1=1 crypto rx=0x7faf48009d90 tx=0x7faf4802beb0 comp rx=0 tx=0).stop 2023-08-09T15:11:09.960 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:09.957+0000 7faf5612f640 1 -- 172.21.15.43:0/436557573 shutdown_connections 2023-08-09T15:11:09.960 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:09.957+0000 7faf5612f640 1 --2- 172.21.15.43:0/436557573 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7faf500cdd30 0x7faf500b3fa0 unknown :-1 s=CLOSED pgs=2307 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T15:11:09.961 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:09.957+0000 7faf5612f640 1 --2- 172.21.15.43:0/436557573 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7faf500d0a70 0x7faf500cd7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T15:11:09.961 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:09.957+0000 7faf5612f640 1 -- 172.21.15.43:0/436557573 >> 172.21.15.43:0/436557573 conn(0x7faf50071520 msgr2=0x7faf500d2520 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-09T15:11:09.961 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:09.957+0000 7faf5612f640 1 -- 172.21.15.43:0/436557573 shutdown_connections 2023-08-09T15:11:09.961 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:09.958+0000 7faf5612f640 1 -- 172.21.15.43:0/436557573 wait complete. 2023-08-09T15:11:09.962 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:09.959+0000 7faf5612f640 1 Processor -- start 2023-08-09T15:11:09.962 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:09.959+0000 7faf5612f640 1 -- start start 2023-08-09T15:11:09.962 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:09.960+0000 7faf5612f640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7faf500d0a70 0x7faf50156c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T15:11:09.962 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:09.960+0000 7faf5612f640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7faf5015c680 0x7faf50159940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T15:11:09.963 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:09.960+0000 7faf5612f640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7faf50159e80 0x7faf501546a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T15:11:09.963 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:09.960+0000 7faf5612f640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faf50146350 con 0x7faf500d0a70 2023-08-09T15:11:09.963 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:09.960+0000 7faf5612f640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faf5015a310 con 0x7faf50159e80 2023-08-09T15:11:09.963 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:09.960+0000 7faf5612f640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7faf501456f0 con 0x7faf5015c680 2023-08-09T15:11:09.963 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:09.961+0000 7faf4f7fe640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7faf500d0a70 0x7faf50156c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T15:11:09.963 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:09.961+0000 7faf4f7fe640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7faf500d0a70 0x7faf50156c60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:46520/0 (socket says 172.21.15.43:46520) 2023-08-09T15:11:09.963 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:09.961+0000 7faf4f7fe640 1 -- 172.21.15.43:0/3026235547 learned_addr learned my addr 172.21.15.43:0/3026235547 (peer_addr_for_me v2:172.21.15.43:0/0) 2023-08-09T15:11:09.965 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:09.961+0000 7faf46ffd640 1 --2- 172.21.15.43:0/3026235547 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7faf5015c680 0x7faf50159940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T15:11:09.965 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:09.961+0000 7faf4f7fe640 1 -- 172.21.15.43:0/3026235547 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7faf5015c680 msgr2=0x7faf50159940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T15:11:09.965 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:09.961+0000 7faf4ffff640 1 --2- 172.21.15.43:0/3026235547 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7faf50159e80 0x7faf501546a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T15:11:09.965 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:09.961+0000 7faf4f7fe640 1 --2- 172.21.15.43:0/3026235547 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7faf5015c680 0x7faf50159940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T15:11:09.965 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:09.961+0000 7faf4f7fe640 1 -- 172.21.15.43:0/3026235547 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7faf50159e80 msgr2=0x7faf501546a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T15:11:09.965 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:09.961+0000 7faf4f7fe640 1 --2- 172.21.15.43:0/3026235547 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7faf50159e80 0x7faf501546a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T15:11:09.966 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:09.961+0000 7faf4f7fe640 1 -- 172.21.15.43:0/3026235547 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faf48009a70 con 0x7faf500d0a70 2023-08-09T15:11:09.966 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:09.961+0000 7faf4ffff640 1 --2- 172.21.15.43:0/3026235547 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7faf50159e80 0x7faf501546a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-08-09T15:11:09.966 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:09.961+0000 7faf46ffd640 1 --2- 172.21.15.43:0/3026235547 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7faf5015c680 0x7faf50159940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-08-09T15:11:09.966 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:09.962+0000 7faf4f7fe640 1 --2- 172.21.15.43:0/3026235547 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7faf500d0a70 0x7faf50156c60 secure :-1 s=READY pgs=2308 cs=0 l=1 rev1=1 crypto rx=0x7faf4000c480 tx=0x7faf4000c940 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-09T15:11:09.966 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:09.962+0000 7faf4cff9640 1 -- 172.21.15.43:0/3026235547 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faf40005ce0 con 0x7faf500d0a70 2023-08-09T15:11:09.966 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:09.962+0000 7faf4cff9640 1 -- 172.21.15.43:0/3026235547 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7faf40005e40 con 0x7faf500d0a70 2023-08-09T15:11:09.966 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:09.962+0000 7faf4cff9640 1 -- 172.21.15.43:0/3026235547 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faf400072f0 con 0x7faf500d0a70 2023-08-09T15:11:09.967 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:09.962+0000 7faf5612f640 1 -- 172.21.15.43:0/3026235547 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7faf501459b0 con 0x7faf500d0a70 2023-08-09T15:11:09.967 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:09.962+0000 7faf5612f640 1 -- 172.21.15.43:0/3026235547 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7faf50145f70 con 0x7faf500d0a70 2023-08-09T15:11:09.967 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:09.964+0000 7faf5612f640 1 -- 172.21.15.43:0/3026235547 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7faf50146490 con 0x7faf500d0a70 2023-08-09T15:11:09.967 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:09.965+0000 7faf4cff9640 1 -- 172.21.15.43:0/3026235547 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 116) v1 ==== 88179+0+0 (secure 0 0 0) 0x7faf40007510 con 0x7faf500d0a70 2023-08-09T15:11:09.968 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:09.966+0000 7faf4cff9640 1 --2- 172.21.15.43:0/3026235547 >> [v2:172.21.15.203:6832/4034093268,v1:172.21.15.203:6833/4034093268] conn(0x7faf30069ea0 0x7faf3006c360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T15:11:09.968 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:09.966+0000 7faf4cff9640 1 -- 172.21.15.43:0/3026235547 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(211..211 src has 1..211) v4 ==== 7200+0+0 (secure 0 0 0) 0x7faf40087bb0 con 0x7faf500d0a70 2023-08-09T15:11:09.968 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:09.966+0000 7faf46ffd640 1 --2- 172.21.15.43:0/3026235547 >> [v2:172.21.15.203:6832/4034093268,v1:172.21.15.203:6833/4034093268] conn(0x7faf30069ea0 0x7faf3006c360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T15:11:09.969 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:09.968+0000 7faf46ffd640 1 --2- 172.21.15.43:0/3026235547 >> [v2:172.21.15.203:6832/4034093268,v1:172.21.15.203:6833/4034093268] conn(0x7faf30069ea0 0x7faf3006c360 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7faf38006fd0 tx=0x7faf38008040 comp rx=0 tx=0).ready entity=mgr.8833 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-09T15:11:09.971 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:09.970+0000 7faf4cff9640 1 -- 172.21.15.43:0/3026235547 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178066 (secure 0 0 0) 0x7faf50146490 con 0x7faf500d0a70 2023-08-09T15:11:10.171 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:10.170+0000 7faf5612f640 1 -- 172.21.15.43:0/3026235547 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7faf50146490 con 0x7faf500d0a70 2023-08-09T15:11:10.171 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:10.170+0000 7faf4cff9640 1 -- 172.21.15.43:0/3026235547 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v211) v1 ==== 104+0+11 (secure 0 0 0) 0x7faf50146490 con 0x7faf500d0a70 2023-08-09T15:11:10.172 INFO:teuthology.orchestra.run.smithi043.stdout:pg_num: 32 2023-08-09T15:11:10.174 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:10.173+0000 7faf5612f640 1 -- 172.21.15.43:0/3026235547 >> [v2:172.21.15.203:6832/4034093268,v1:172.21.15.203:6833/4034093268] conn(0x7faf30069ea0 msgr2=0x7faf3006c360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T15:11:10.175 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:10.173+0000 7faf5612f640 1 --2- 172.21.15.43:0/3026235547 >> [v2:172.21.15.203:6832/4034093268,v1:172.21.15.203:6833/4034093268] conn(0x7faf30069ea0 0x7faf3006c360 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7faf38006fd0 tx=0x7faf38008040 comp rx=0 tx=0).stop 2023-08-09T15:11:10.175 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:10.173+0000 7faf5612f640 1 -- 172.21.15.43:0/3026235547 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7faf500d0a70 msgr2=0x7faf50156c60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T15:11:10.175 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:10.173+0000 7faf5612f640 1 --2- 172.21.15.43:0/3026235547 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7faf500d0a70 0x7faf50156c60 secure :-1 s=READY pgs=2308 cs=0 l=1 rev1=1 crypto rx=0x7faf4000c480 tx=0x7faf4000c940 comp rx=0 tx=0).stop 2023-08-09T15:11:10.175 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:10.173+0000 7faf5612f640 1 -- 172.21.15.43:0/3026235547 shutdown_connections 2023-08-09T15:11:10.175 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:10.173+0000 7faf5612f640 1 --2- 172.21.15.43:0/3026235547 >> [v2:172.21.15.203:6832/4034093268,v1:172.21.15.203:6833/4034093268] conn(0x7faf30069ea0 0x7faf3006c360 unknown :-1 s=CLOSED pgs=22 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T15:11:10.175 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:10.173+0000 7faf5612f640 1 --2- 172.21.15.43:0/3026235547 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7faf50159e80 0x7faf501546a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T15:11:10.176 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:10.173+0000 7faf5612f640 1 --2- 172.21.15.43:0/3026235547 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7faf5015c680 0x7faf50159940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T15:11:10.176 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:10.173+0000 7faf5612f640 1 --2- 172.21.15.43:0/3026235547 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7faf500d0a70 0x7faf50156c60 unknown :-1 s=CLOSED pgs=2308 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T15:11:10.176 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:10.173+0000 7faf5612f640 1 -- 172.21.15.43:0/3026235547 >> 172.21.15.43:0/3026235547 conn(0x7faf50071520 msgr2=0x7faf50157d10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-09T15:11:10.176 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:10.173+0000 7faf5612f640 1 -- 172.21.15.43:0/3026235547 shutdown_connections 2023-08-09T15:11:10.176 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:10.173+0000 7faf5612f640 1 -- 172.21.15.43:0/3026235547 wait complete. 2023-08-09T15:11:10.194 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-08-09T15:11:10.408 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:10.407+0000 7facb72ad640 1 Processor -- start 2023-08-09T15:11:10.408 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:10.407+0000 7facb72ad640 1 -- start start 2023-08-09T15:11:10.410 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:10.409+0000 7facb72ad640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7facb00cb2d0 0x7facb015b380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T15:11:10.410 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:10.409+0000 7facb72ad640 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7facb015df60 0x7facb00ce1c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T15:11:10.411 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:10.409+0000 7facb72ad640 1 -- --> v1:172.21.15.203:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7facb015aef0 con 0x7facb015e3c0 2023-08-09T15:11:10.411 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:10.409+0000 7facb72ad640 1 -- --> v2:172.21.15.203:3300/0 -- mon_getmap magic: 0 v1 -- 0x7facb015a970 con 0x7facb015df60 2023-08-09T15:11:10.411 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:10.409+0000 7facb72ad640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7facb015b8c0 con 0x7facb00cb2d0 2023-08-09T15:11:10.411 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:10.410+0000 7facb5049640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7facb00cb2d0 0x7facb015b380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T15:11:10.411 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:10.410+0000 7facb4848640 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7facb015df60 0x7facb00ce1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T15:11:10.412 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:10.410+0000 7facb5049640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7facb00cb2d0 0x7facb015b380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.203:3301/0 says I am v2:172.21.15.43:36220/0 (socket says 172.21.15.43:36220) 2023-08-09T15:11:10.412 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:10.410+0000 7facb5049640 1 -- 172.21.15.43:0/1462791946 learned_addr learned my addr 172.21.15.43:0/1462791946 (peer_addr_for_me v2:172.21.15.43:0/0) 2023-08-09T15:11:10.412 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:10.411+0000 7facb4848640 1 -- 172.21.15.43:0/1462791946 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7facb00cb2d0 msgr2=0x7facb015b380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-09T15:11:10.412 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:10.411+0000 7facb4848640 1 --2- 172.21.15.43:0/1462791946 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7facb00cb2d0 0x7facb015b380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T15:11:10.413 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:10.411+0000 7facb4848640 1 -- 172.21.15.43:0/1462791946 >> v1:172.21.15.203:6789/0 conn(0x7facb015e3c0 legacy=0x7facb00c9f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T15:11:10.413 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:10.411+0000 7facb4848640 1 -- 172.21.15.43:0/1462791946 --> v2:172.21.15.203:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7facb015e940 con 0x7facb015df60 2023-08-09T15:11:10.413 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:10.411+0000 7facb5049640 1 --2- 172.21.15.43:0/1462791946 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7facb00cb2d0 0x7facb015b380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-08-09T15:11:10.413 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:10.412+0000 7facb4848640 1 --2- 172.21.15.43:0/1462791946 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7facb015df60 0x7facb00ce1c0 secure :-1 s=READY pgs=847 cs=0 l=1 rev1=1 crypto rx=0x7faca00097a0 tx=0x7faca002b8b0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=b01883df72968dd server_cookie=0 in_seq=0 out_seq=0 2023-08-09T15:11:10.414 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:10.413+0000 7faca7fff640 1 -- 172.21.15.43:0/1462791946 <== mon.1 v2:172.21.15.203:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faca002d040 con 0x7facb015df60 2023-08-09T15:11:10.414 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:10.413+0000 7faca7fff640 1 -- 172.21.15.43:0/1462791946 <== mon.1 v2:172.21.15.203:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7faca0031810 con 0x7facb015df60 2023-08-09T15:11:10.414 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:10.413+0000 7faca7fff640 1 -- 172.21.15.43:0/1462791946 <== mon.1 v2:172.21.15.203:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faca0031ba0 con 0x7facb015df60 2023-08-09T15:11:10.415 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:10.413+0000 7facb72ad640 1 -- 172.21.15.43:0/1462791946 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7facb015df60 msgr2=0x7facb00ce1c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T15:11:10.415 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:10.413+0000 7facb72ad640 1 --2- 172.21.15.43:0/1462791946 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7facb015df60 0x7facb00ce1c0 secure :-1 s=READY pgs=847 cs=0 l=1 rev1=1 crypto rx=0x7faca00097a0 tx=0x7faca002b8b0 comp rx=0 tx=0).stop 2023-08-09T15:11:10.415 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:10.414+0000 7facb72ad640 1 -- 172.21.15.43:0/1462791946 shutdown_connections 2023-08-09T15:11:10.415 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:10.414+0000 7facb72ad640 1 --2- 172.21.15.43:0/1462791946 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7facb015df60 0x7facb00ce1c0 unknown :-1 s=CLOSED pgs=847 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T15:11:10.415 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:10.414+0000 7facb72ad640 1 --2- 172.21.15.43:0/1462791946 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7facb00cb2d0 0x7facb015b380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T15:11:10.416 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:10.414+0000 7facb72ad640 1 -- 172.21.15.43:0/1462791946 >> 172.21.15.43:0/1462791946 conn(0x7facb00aff50 msgr2=0x7facb00b0360 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-09T15:11:10.416 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:10.414+0000 7facb72ad640 1 -- 172.21.15.43:0/1462791946 shutdown_connections 2023-08-09T15:11:10.416 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:10.414+0000 7facb72ad640 1 -- 172.21.15.43:0/1462791946 wait complete. 2023-08-09T15:11:10.417 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:10.416+0000 7facb72ad640 1 Processor -- start 2023-08-09T15:11:10.417 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:10.416+0000 7facb72ad640 1 -- start start 2023-08-09T15:11:10.419 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:10.417+0000 7facb72ad640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7facb00cb2d0 0x7facb0148ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T15:11:10.419 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:10.417+0000 7facb72ad640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7facb015df60 0x7facb00ce400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T15:11:10.419 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:10.417+0000 7facb72ad640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7facb015e3c0 0x7facb0148280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T15:11:10.419 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:10.417+0000 7facb72ad640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7facb0149960 con 0x7facb00cb2d0 2023-08-09T15:11:10.419 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:10.417+0000 7facb72ad640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7facb00b33e0 con 0x7facb015df60 2023-08-09T15:11:10.419 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:10.417+0000 7facb72ad640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7facb00c6720 con 0x7facb015e3c0 2023-08-09T15:11:10.420 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:10.418+0000 7facb5049640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7facb00cb2d0 0x7facb0148ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T15:11:10.420 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:10.418+0000 7facb5049640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7facb00cb2d0 0x7facb0148ea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:46532/0 (socket says 172.21.15.43:46532) 2023-08-09T15:11:10.420 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:10.418+0000 7facb5049640 1 -- 172.21.15.43:0/3350240860 learned_addr learned my addr 172.21.15.43:0/3350240860 (peer_addr_for_me v2:172.21.15.43:0/0) 2023-08-09T15:11:10.421 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:10.418+0000 7facb584a640 1 --2- 172.21.15.43:0/3350240860 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7facb015e3c0 0x7facb0148280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T15:11:10.421 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:10.418+0000 7facb4848640 1 --2- 172.21.15.43:0/3350240860 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7facb015df60 0x7facb00ce400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T15:11:10.421 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:10.418+0000 7facb5049640 1 -- 172.21.15.43:0/3350240860 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7facb015e3c0 msgr2=0x7facb0148280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T15:11:10.421 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:10.418+0000 7facb5049640 1 --2- 172.21.15.43:0/3350240860 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7facb015e3c0 0x7facb0148280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T15:11:10.422 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:10.418+0000 7facb5049640 1 -- 172.21.15.43:0/3350240860 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7facb015df60 msgr2=0x7facb00ce400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T15:11:10.422 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:10.418+0000 7facb5049640 1 --2- 172.21.15.43:0/3350240860 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7facb015df60 0x7facb00ce400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T15:11:10.422 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:10.418+0000 7facb5049640 1 -- 172.21.15.43:0/3350240860 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7facac0094d0 con 0x7facb00cb2d0 2023-08-09T15:11:10.422 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:10.418+0000 7facb4848640 1 --2- 172.21.15.43:0/3350240860 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7facb015df60 0x7facb00ce400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-08-09T15:11:10.422 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:10.419+0000 7facb5049640 1 --2- 172.21.15.43:0/3350240860 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7facb00cb2d0 0x7facb0148ea0 secure :-1 s=READY pgs=2309 cs=0 l=1 rev1=1 crypto rx=0x7facac00e9d0 tx=0x7facac00ee90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-09T15:11:10.422 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:10.419+0000 7faca5ffb640 1 -- 172.21.15.43:0/3350240860 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7facac00ca00 con 0x7facb00cb2d0 2023-08-09T15:11:10.422 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:10.419+0000 7faca5ffb640 1 -- 172.21.15.43:0/3350240860 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7facac00cb60 con 0x7facb00cb2d0 2023-08-09T15:11:10.423 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:10.419+0000 7faca5ffb640 1 -- 172.21.15.43:0/3350240860 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7facac013470 con 0x7facb00cb2d0 2023-08-09T15:11:10.423 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:10.419+0000 7facb72ad640 1 -- 172.21.15.43:0/3350240860 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7faca0009400 con 0x7facb00cb2d0 2023-08-09T15:11:10.423 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:10.419+0000 7facb72ad640 1 -- 172.21.15.43:0/3350240860 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7facb0149440 con 0x7facb00cb2d0 2023-08-09T15:11:10.423 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:10.421+0000 7facb72ad640 1 -- 172.21.15.43:0/3350240860 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7facb00739f0 con 0x7facb00cb2d0 2023-08-09T15:11:10.423 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:10.421+0000 7faca5ffb640 1 -- 172.21.15.43:0/3350240860 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 116) v1 ==== 88179+0+0 (secure 0 0 0) 0x7facac0135d0 con 0x7facb00cb2d0 2023-08-09T15:11:10.424 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:10.422+0000 7faca5ffb640 1 --2- 172.21.15.43:0/3350240860 >> [v2:172.21.15.203:6832/4034093268,v1:172.21.15.203:6833/4034093268] conn(0x7fac94069ea0 0x7fac9406c360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T15:11:10.424 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:10.423+0000 7faca5ffb640 1 -- 172.21.15.43:0/3350240860 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(211..211 src has 1..211) v4 ==== 7200+0+0 (secure 0 0 0) 0x7facac0889d0 con 0x7facb00cb2d0 2023-08-09T15:11:10.424 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:10.423+0000 7facb4848640 1 --2- 172.21.15.43:0/3350240860 >> [v2:172.21.15.203:6832/4034093268,v1:172.21.15.203:6833/4034093268] conn(0x7fac94069ea0 0x7fac9406c360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T15:11:10.426 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:10.425+0000 7facb4848640 1 --2- 172.21.15.43:0/3350240860 >> [v2:172.21.15.203:6832/4034093268,v1:172.21.15.203:6833/4034093268] conn(0x7fac94069ea0 0x7fac9406c360 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7faca0036000 tx=0x7faca003c040 comp rx=0 tx=0).ready entity=mgr.8833 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-09T15:11:10.428 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:10.427+0000 7faca5ffb640 1 -- 172.21.15.43:0/3350240860 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178066 (secure 0 0 0) 0x7facac053ee0 con 0x7facb00cb2d0 2023-08-09T15:11:10.629 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:10.628+0000 7facb72ad640 1 -- 172.21.15.43:0/3350240860 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7facb00ae5e0 con 0x7facb00cb2d0 2023-08-09T15:11:10.630 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:10.629+0000 7faca5ffb640 1 -- 172.21.15.43:0/3350240860 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v211) v1 ==== 100+0+11 (secure 0 0 0) 0x7facac057500 con 0x7facb00cb2d0 2023-08-09T15:11:10.631 INFO:teuthology.orchestra.run.smithi043.stdout:pg_num: 64 2023-08-09T15:11:10.633 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:10.632+0000 7facb72ad640 1 -- 172.21.15.43:0/3350240860 >> [v2:172.21.15.203:6832/4034093268,v1:172.21.15.203:6833/4034093268] conn(0x7fac94069ea0 msgr2=0x7fac9406c360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T15:11:10.633 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:10.632+0000 7facb72ad640 1 --2- 172.21.15.43:0/3350240860 >> [v2:172.21.15.203:6832/4034093268,v1:172.21.15.203:6833/4034093268] conn(0x7fac94069ea0 0x7fac9406c360 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7faca0036000 tx=0x7faca003c040 comp rx=0 tx=0).stop 2023-08-09T15:11:10.633 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:10.632+0000 7facb72ad640 1 -- 172.21.15.43:0/3350240860 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7facb00cb2d0 msgr2=0x7facb0148ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T15:11:10.634 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:10.632+0000 7facb72ad640 1 --2- 172.21.15.43:0/3350240860 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7facb00cb2d0 0x7facb0148ea0 secure :-1 s=READY pgs=2309 cs=0 l=1 rev1=1 crypto rx=0x7facac00e9d0 tx=0x7facac00ee90 comp rx=0 tx=0).stop 2023-08-09T15:11:10.634 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:10.632+0000 7facb72ad640 1 -- 172.21.15.43:0/3350240860 shutdown_connections 2023-08-09T15:11:10.634 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:10.632+0000 7facb72ad640 1 --2- 172.21.15.43:0/3350240860 >> [v2:172.21.15.203:6832/4034093268,v1:172.21.15.203:6833/4034093268] conn(0x7fac94069ea0 0x7fac9406c360 unknown :-1 s=CLOSED pgs=23 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T15:11:10.634 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:10.632+0000 7facb72ad640 1 --2- 172.21.15.43:0/3350240860 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7facb015e3c0 0x7facb0148280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T15:11:10.634 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:10.632+0000 7facb72ad640 1 --2- 172.21.15.43:0/3350240860 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7facb015df60 0x7facb00ce400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T15:11:10.634 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:10.632+0000 7facb72ad640 1 --2- 172.21.15.43:0/3350240860 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7facb00cb2d0 0x7facb0148ea0 unknown :-1 s=CLOSED pgs=2309 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T15:11:10.634 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:10.632+0000 7facb72ad640 1 -- 172.21.15.43:0/3350240860 >> 172.21.15.43:0/3350240860 conn(0x7facb00aff50 msgr2=0x7facb00b0330 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-09T15:11:10.635 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:10.632+0000 7facb72ad640 1 -- 172.21.15.43:0/3350240860 shutdown_connections 2023-08-09T15:11:10.635 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:10.632+0000 7facb72ad640 1 -- 172.21.15.43:0/3350240860 wait complete. 2023-08-09T15:11:10.652 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-08-09T15:11:10.870 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:10.869+0000 7fa170c4c640 1 Processor -- start 2023-08-09T15:11:10.871 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:10.869+0000 7fa170c4c640 1 -- start start 2023-08-09T15:11:10.872 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:10.871+0000 7fa170c4c640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fa16c0b2b70 0x7fa16c0d2ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T15:11:10.872 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:10.871+0000 7fa170c4c640 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7fa16c0b3050 0x7fa16c0b3f60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T15:11:10.872 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:10.871+0000 7fa170c4c640 1 -- --> v1:172.21.15.43:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa16c0ce090 con 0x7fa16c0b2790 2023-08-09T15:11:10.873 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:10.871+0000 7fa170c4c640 1 -- --> v2:172.21.15.203:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa16c0b3950 con 0x7fa16c0b3050 2023-08-09T15:11:10.873 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:10.871+0000 7fa170c4c640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa16c0cd7f0 con 0x7fa16c0b2b70 2023-08-09T15:11:10.873 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:10.871+0000 7fa16a59c640 1 --1- >> v1:172.21.15.43:6789/0 conn(0x7fa16c0b2790 0x7fa16c0b5330 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.43:6789/0 says I am v1:172.21.15.43:45564/0 (socket says 172.21.15.43:45564) 2023-08-09T15:11:10.873 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:10.871+0000 7fa16a59c640 1 -- 172.21.15.43:0/4035632247 learned_addr learned my addr 172.21.15.43:0/4035632247 (peer_addr_for_me v1:172.21.15.43:0/0) 2023-08-09T15:11:10.874 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:10.872+0000 7fa16ad9d640 1 --2- 172.21.15.43:0/4035632247 >> v2:172.21.15.203:3300/0 conn(0x7fa16c0b3050 0x7fa16c0b3f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T15:11:10.875 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:10.872+0000 7fa169d9b640 1 --2- 172.21.15.43:0/4035632247 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fa16c0b2b70 0x7fa16c0d2ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T15:11:10.875 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:10.872+0000 7fa16959a640 1 -- 172.21.15.43:0/4035632247 <== mon.0 v1:172.21.15.43:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3533409071 0 0) 0x7fa16c0ce090 con 0x7fa16c0b2790 2023-08-09T15:11:10.875 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:10.872+0000 7fa16959a640 1 -- 172.21.15.43:0/4035632247 --> v1:172.21.15.43:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fa158003520 con 0x7fa16c0b2790 2023-08-09T15:11:10.875 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:10.872+0000 7fa16959a640 1 -- 172.21.15.43:0/4035632247 <== mon.0 v1:172.21.15.43:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2810804590 0 0) 0x7fa158003520 con 0x7fa16c0b2790 2023-08-09T15:11:10.875 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:10.872+0000 7fa16959a640 1 -- 172.21.15.43:0/4035632247 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fa16c0b2b70 msgr2=0x7fa16c0d2ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-09T15:11:10.875 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:10.872+0000 7fa16959a640 1 --2- 172.21.15.43:0/4035632247 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fa16c0b2b70 0x7fa16c0d2ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T15:11:10.875 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:10.872+0000 7fa16959a640 1 -- 172.21.15.43:0/4035632247 >> v2:172.21.15.203:3300/0 conn(0x7fa16c0b3050 msgr2=0x7fa16c0b3f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-09T15:11:10.876 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:10.872+0000 7fa16959a640 1 --2- 172.21.15.43:0/4035632247 >> v2:172.21.15.203:3300/0 conn(0x7fa16c0b3050 0x7fa16c0b3f60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T15:11:10.876 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:10.872+0000 7fa16959a640 1 -- 172.21.15.43:0/4035632247 --> v1:172.21.15.43:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa16c0cdad0 con 0x7fa16c0b2790 2023-08-09T15:11:10.876 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:10.872+0000 7fa16959a640 1 -- 172.21.15.43:0/4035632247 <== mon.0 v1:172.21.15.43:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1285150740 0 0) 0x7fa15c0019f0 con 0x7fa16c0b2790 2023-08-09T15:11:10.876 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:10.872+0000 7fa16959a640 1 -- 172.21.15.43:0/4035632247 >> v1:172.21.15.43:6789/0 conn(0x7fa16c0b2790 legacy=0x7fa16c0b5330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T15:11:10.876 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:10.872+0000 7fa16959a640 1 --2- 172.21.15.43:0/4035632247 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fa1580039e0 0x7fa158003e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T15:11:10.876 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:10.872+0000 7fa16959a640 1 --2- 172.21.15.43:0/4035632247 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fa16c0b2b70 0x7fa1580047c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T15:11:10.876 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:10.872+0000 7fa16959a640 1 --2- 172.21.15.43:0/4035632247 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa158004d00 0x7fa158005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T15:11:10.876 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:10.872+0000 7fa16959a640 1 -- 172.21.15.43:0/4035632247 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa16c0b3950 con 0x7fa158004d00 2023-08-09T15:11:10.877 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:10.872+0000 7fa16959a640 1 -- 172.21.15.43:0/4035632247 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa16c0cd7f0 con 0x7fa1580039e0 2023-08-09T15:11:10.877 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:10.872+0000 7fa16959a640 1 -- 172.21.15.43:0/4035632247 --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa15800ba00 con 0x7fa16c0b2b70 2023-08-09T15:11:10.877 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:10.872+0000 7fa16ad9d640 1 --2- 172.21.15.43:0/4035632247 >> v2:172.21.15.203:3300/0 conn(0x7fa16c0b3050 0x7fa16c0b3f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-08-09T15:11:10.877 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:10.873+0000 7fa16ad9d640 1 --2- 172.21.15.43:0/4035632247 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa158004d00 0x7fa158005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T15:11:10.877 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:10.873+0000 7fa169d9b640 1 --2- 172.21.15.43:0/4035632247 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fa16c0b2b70 0x7fa1580047c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T15:11:10.877 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:10.873+0000 7fa16a59c640 1 --2- 172.21.15.43:0/4035632247 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fa1580039e0 0x7fa158003e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T15:11:10.877 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:10.873+0000 7fa16ad9d640 1 -- 172.21.15.43:0/4035632247 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fa16c0b2b70 msgr2=0x7fa1580047c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-09T15:11:10.878 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:10.873+0000 7fa16ad9d640 1 --2- 172.21.15.43:0/4035632247 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fa16c0b2b70 0x7fa1580047c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T15:11:10.878 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:10.873+0000 7fa16ad9d640 1 -- 172.21.15.43:0/4035632247 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fa1580039e0 msgr2=0x7fa158003e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-09T15:11:10.878 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:10.873+0000 7fa16ad9d640 1 --2- 172.21.15.43:0/4035632247 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fa1580039e0 0x7fa158003e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T15:11:10.878 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:10.873+0000 7fa16ad9d640 1 -- 172.21.15.43:0/4035632247 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa158003520 con 0x7fa158004d00 2023-08-09T15:11:10.879 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:10.874+0000 7fa16ad9d640 1 --2- 172.21.15.43:0/4035632247 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa158004d00 0x7fa158005180 secure :-1 s=READY pgs=2311 cs=0 l=1 rev1=1 crypto rx=0x7fa1640098a0 tx=0x7fa1580057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9e0ab6402a5e1dcd server_cookie=0 in_seq=0 out_seq=0 2023-08-09T15:11:10.879 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:10.875+0000 7fa16959a640 1 -- 172.21.15.43:0/4035632247 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa164037600 con 0x7fa158004d00 2023-08-09T15:11:10.879 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:10.875+0000 7fa16959a640 1 -- 172.21.15.43:0/4035632247 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa164037760 con 0x7fa158004d00 2023-08-09T15:11:10.879 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:10.875+0000 7fa16959a640 1 -- 172.21.15.43:0/4035632247 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa164037ad0 con 0x7fa158004d00 2023-08-09T15:11:10.879 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:10.875+0000 7fa170c4c640 1 -- 172.21.15.43:0/4035632247 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa158004d00 msgr2=0x7fa158005180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T15:11:10.880 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:10.875+0000 7fa170c4c640 1 --2- 172.21.15.43:0/4035632247 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa158004d00 0x7fa158005180 secure :-1 s=READY pgs=2311 cs=0 l=1 rev1=1 crypto rx=0x7fa1640098a0 tx=0x7fa1580057f0 comp rx=0 tx=0).stop 2023-08-09T15:11:10.880 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:10.875+0000 7fa16ad9d640 1 -- 172.21.15.43:0/4035632247 reap_dead start 2023-08-09T15:11:10.880 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:10.875+0000 7fa170c4c640 1 -- 172.21.15.43:0/4035632247 shutdown_connections 2023-08-09T15:11:10.880 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:10.875+0000 7fa170c4c640 1 -- 172.21.15.43:0/4035632247 >> 172.21.15.43:0/4035632247 conn(0x7fa16c09b7b0 msgr2=0x7fa16c0d24e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-09T15:11:10.880 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:10.875+0000 7fa170c4c640 1 -- 172.21.15.43:0/4035632247 shutdown_connections 2023-08-09T15:11:10.880 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:10.875+0000 7fa170c4c640 1 -- 172.21.15.43:0/4035632247 wait complete. 2023-08-09T15:11:10.880 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:10.877+0000 7fa170c4c640 1 Processor -- start 2023-08-09T15:11:10.881 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:10.878+0000 7fa170c4c640 1 -- start start 2023-08-09T15:11:10.882 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:10.878+0000 7fa170c4c640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa158004d00 0x7fa16c0b1a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T15:11:10.882 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:10.879+0000 7fa170c4c640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fa16c1f60f0 0x7fa16c0caa60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T15:11:10.882 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:10.879+0000 7fa170c4c640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fa16c0c8b50 0x7fa16c1f5530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T15:11:10.882 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:10.879+0000 7fa170c4c640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa16c0b1fc0 con 0x7fa158004d00 2023-08-09T15:11:10.882 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:10.879+0000 7fa170c4c640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa16c0cafa0 con 0x7fa16c1f60f0 2023-08-09T15:11:10.882 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:10.879+0000 7fa170c4c640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa16c0c7ab0 con 0x7fa16c0c8b50 2023-08-09T15:11:10.883 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:10.879+0000 7fa16a59c640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa158004d00 0x7fa16c0b1a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T15:11:10.883 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:10.879+0000 7fa16a59c640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa158004d00 0x7fa16c0b1a80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:46546/0 (socket says 172.21.15.43:46546) 2023-08-09T15:11:10.883 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:10.879+0000 7fa16a59c640 1 -- 172.21.15.43:0/2122325772 learned_addr learned my addr 172.21.15.43:0/2122325772 (peer_addr_for_me v2:172.21.15.43:0/0) 2023-08-09T15:11:10.883 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:10.879+0000 7fa169d9b640 1 --2- 172.21.15.43:0/2122325772 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fa16c1f60f0 0x7fa16c0caa60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T15:11:10.883 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:10.879+0000 7fa16ad9d640 1 --2- 172.21.15.43:0/2122325772 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fa16c0c8b50 0x7fa16c1f5530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T15:11:10.883 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:10.879+0000 7fa16a59c640 1 -- 172.21.15.43:0/2122325772 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fa16c0c8b50 msgr2=0x7fa16c1f5530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T15:11:10.883 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:10.879+0000 7fa16a59c640 1 --2- 172.21.15.43:0/2122325772 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fa16c0c8b50 0x7fa16c1f5530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T15:11:10.884 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:10.879+0000 7fa16a59c640 1 -- 172.21.15.43:0/2122325772 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fa16c1f60f0 msgr2=0x7fa16c0caa60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T15:11:10.884 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:10.879+0000 7fa16a59c640 1 --2- 172.21.15.43:0/2122325772 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fa16c1f60f0 0x7fa16c0caa60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T15:11:10.884 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:10.879+0000 7fa16a59c640 1 -- 172.21.15.43:0/2122325772 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa16c0c7d90 con 0x7fa158004d00 2023-08-09T15:11:10.884 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:10.879+0000 7fa169d9b640 1 --2- 172.21.15.43:0/2122325772 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fa16c1f60f0 0x7fa16c0caa60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-08-09T15:11:10.884 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:10.880+0000 7fa16a59c640 1 --2- 172.21.15.43:0/2122325772 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa158004d00 0x7fa16c0b1a80 secure :-1 s=READY pgs=2312 cs=0 l=1 rev1=1 crypto rx=0x7fa15c007960 tx=0x7fa15c00e530 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-09T15:11:10.884 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:10.880+0000 7fa162ffd640 1 -- 172.21.15.43:0/2122325772 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa15c00ee50 con 0x7fa158004d00 2023-08-09T15:11:10.884 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:10.880+0000 7fa162ffd640 1 -- 172.21.15.43:0/2122325772 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa15c013420 con 0x7fa158004d00 2023-08-09T15:11:10.884 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:10.880+0000 7fa162ffd640 1 -- 172.21.15.43:0/2122325772 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa15c013790 con 0x7fa158004d00 2023-08-09T15:11:10.885 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:10.880+0000 7fa170c4c640 1 -- 172.21.15.43:0/2122325772 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa16c0c8070 con 0x7fa158004d00 2023-08-09T15:11:10.885 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:10.880+0000 7fa170c4c640 1 -- 172.21.15.43:0/2122325772 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa16c0c8630 con 0x7fa158004d00 2023-08-09T15:11:10.885 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:10.881+0000 7fa170c4c640 1 -- 172.21.15.43:0/2122325772 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa16c0739f0 con 0x7fa158004d00 2023-08-09T15:11:10.886 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:10.882+0000 7fa162ffd640 1 -- 172.21.15.43:0/2122325772 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 116) v1 ==== 88179+0+0 (secure 0 0 0) 0x7fa15c0138f0 con 0x7fa158004d00 2023-08-09T15:11:10.886 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:10.883+0000 7fa162ffd640 1 --2- 172.21.15.43:0/2122325772 >> [v2:172.21.15.203:6832/4034093268,v1:172.21.15.203:6833/4034093268] conn(0x7fa14c069ed0 0x7fa14c06c390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T15:11:10.886 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:10.883+0000 7fa162ffd640 1 -- 172.21.15.43:0/2122325772 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(211..211 src has 1..211) v4 ==== 7200+0+0 (secure 0 0 0) 0x7fa15c088d70 con 0x7fa158004d00 2023-08-09T15:11:10.886 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:10.884+0000 7fa169d9b640 1 --2- 172.21.15.43:0/2122325772 >> [v2:172.21.15.203:6832/4034093268,v1:172.21.15.203:6833/4034093268] conn(0x7fa14c069ed0 0x7fa14c06c390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T15:11:10.886 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:10.885+0000 7fa169d9b640 1 --2- 172.21.15.43:0/2122325772 >> [v2:172.21.15.203:6832/4034093268,v1:172.21.15.203:6833/4034093268] conn(0x7fa14c069ed0 0x7fa14c06c390 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7fa16c0b4ad0 tx=0x7fa154008040 comp rx=0 tx=0).ready entity=mgr.8833 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-09T15:11:10.889 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:10.887+0000 7fa162ffd640 1 -- 172.21.15.43:0/2122325772 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178066 (secure 0 0 0) 0x7fa15c054310 con 0x7fa158004d00 2023-08-09T15:11:11.120 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:11.119+0000 7fa170c4c640 1 -- 172.21.15.43:0/2122325772 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fa16c0739f0 con 0x7fa158004d00 2023-08-09T15:11:11.121 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:11.120+0000 7fa162ffd640 1 -- 172.21.15.43:0/2122325772 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 96 v96) v1 ==== 94+0+4883 (secure 0 0 0) 0x7fa15c057930 con 0x7fa158004d00 2023-08-09T15:11:11.121 INFO:teuthology.orchestra.run.smithi043.stdout: 2023-08-09T15:11:11.121 INFO:teuthology.orchestra.run.smithi043.stdout:{"epoch":96,"default_fscid":11,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":7703,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.203:6836/1774059384","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6835","nonce":1774059384},{"type":"v1","addr":"172.21.15.203:6836","nonce":1774059384}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":91},{"gid":8665,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.203:6837/682905899","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6834","nonce":682905899},{"type":"v1","addr":"172.21.15.203:6837","nonce":682905899}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":91},{"gid":10752,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.43:6834/4168463642","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6832","nonce":4168463642},{"type":"v1","addr":"172.21.15.43:6834","nonce":4168463642}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":90}],"filesystems":[{"mdsmap":{"epoch":96,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-08-09T15:08:45.563041+0000","modified":"2023-08-09T15:08:49.150213+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":10755},"failed":[],"damaged":[],"stopped":[],"info":{"gid_10755":{"gid":10755,"name":"a","rank":0,"incarnation":92,"state":"up:active","state_seq":3,"addr":"172.21.15.43:6835/349688227","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6833","nonce":349688227},{"type":"v1","addr":"172.21.15.43:6835","nonce":349688227}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[25],"metadata_pool":24,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":11}]} 2023-08-09T15:11:11.122 INFO:teuthology.orchestra.run.smithi043.stderr:dumped fsmap epoch 96 2023-08-09T15:11:11.123 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:11.122+0000 7fa170c4c640 1 -- 172.21.15.43:0/2122325772 >> [v2:172.21.15.203:6832/4034093268,v1:172.21.15.203:6833/4034093268] conn(0x7fa14c069ed0 msgr2=0x7fa14c06c390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T15:11:11.124 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:11.122+0000 7fa170c4c640 1 --2- 172.21.15.43:0/2122325772 >> [v2:172.21.15.203:6832/4034093268,v1:172.21.15.203:6833/4034093268] conn(0x7fa14c069ed0 0x7fa14c06c390 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7fa16c0b4ad0 tx=0x7fa154008040 comp rx=0 tx=0).stop 2023-08-09T15:11:11.124 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:11.122+0000 7fa170c4c640 1 -- 172.21.15.43:0/2122325772 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa158004d00 msgr2=0x7fa16c0b1a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T15:11:11.124 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:11.122+0000 7fa170c4c640 1 --2- 172.21.15.43:0/2122325772 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa158004d00 0x7fa16c0b1a80 secure :-1 s=READY pgs=2312 cs=0 l=1 rev1=1 crypto rx=0x7fa15c007960 tx=0x7fa15c00e530 comp rx=0 tx=0).stop 2023-08-09T15:11:11.125 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:11.122+0000 7fa170c4c640 1 -- 172.21.15.43:0/2122325772 shutdown_connections 2023-08-09T15:11:11.125 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:11.123+0000 7fa170c4c640 1 --2- 172.21.15.43:0/2122325772 >> [v2:172.21.15.203:6832/4034093268,v1:172.21.15.203:6833/4034093268] conn(0x7fa14c069ed0 0x7fa14c06c390 unknown :-1 s=CLOSED pgs=25 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T15:11:11.125 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:11.123+0000 7fa170c4c640 1 --2- 172.21.15.43:0/2122325772 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fa16c0c8b50 0x7fa16c1f5530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T15:11:11.125 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:11.123+0000 7fa170c4c640 1 --2- 172.21.15.43:0/2122325772 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fa16c1f60f0 0x7fa16c0caa60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T15:11:11.125 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:11.123+0000 7fa170c4c640 1 --2- 172.21.15.43:0/2122325772 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa158004d00 0x7fa16c0b1a80 unknown :-1 s=CLOSED pgs=2312 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T15:11:11.125 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:11.123+0000 7fa170c4c640 1 -- 172.21.15.43:0/2122325772 >> 172.21.15.43:0/2122325772 conn(0x7fa16c09b7b0 msgr2=0x7fa16c15c980 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-09T15:11:11.125 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:11.123+0000 7fa170c4c640 1 -- 172.21.15.43:0/2122325772 shutdown_connections 2023-08-09T15:11:11.126 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:11.123+0000 7fa170c4c640 1 -- 172.21.15.43:0/2122325772 wait complete. 2023-08-09T15:11:11.144 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-08-09T15:11:11.376 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:11.375+0000 7f6e2359e640 1 Processor -- start 2023-08-09T15:11:11.376 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:11.375+0000 7f6e2359e640 1 -- start start 2023-08-09T15:11:11.378 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:11.376+0000 7f6e2359e640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f6e2415c4c0 0x7f6e240afd20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T15:11:11.378 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:11.376+0000 7f6e2359e640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f6e2406ab70 0x7f6e2406b250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T15:11:11.378 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:11.376+0000 7f6e2359e640 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7f6e2406b790 0x7f6e240cc930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T15:11:11.378 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:11.376+0000 7f6e2359e640 1 -- --> v2:172.21.15.43:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6e2406bc20 con 0x7f6e2406ab70 2023-08-09T15:11:11.378 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:11.376+0000 7f6e2359e640 1 -- --> v2:172.21.15.203:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6e240b0fc0 con 0x7f6e2406b790 2023-08-09T15:11:11.378 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:11.376+0000 7f6e2359e640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6e240af8e0 con 0x7f6e2415c4c0 2023-08-09T15:11:11.378 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:11.376+0000 7f6e21d9b640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f6e2406ab70 0x7f6e2406b250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T15:11:11.379 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:11.377+0000 7f6e21d9b640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f6e2406ab70 0x7f6e2406b250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:46558/0 (socket says 172.21.15.43:46558) 2023-08-09T15:11:11.379 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:11.377+0000 7f6e21d9b640 1 -- 172.21.15.43:0/2382712987 learned_addr learned my addr 172.21.15.43:0/2382712987 (peer_addr_for_me v2:172.21.15.43:0/0) 2023-08-09T15:11:11.380 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:11.377+0000 7f6e2259c640 1 --2- 172.21.15.43:0/2382712987 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f6e2415c4c0 0x7f6e240afd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T15:11:11.380 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:11.377+0000 7f6e21d9b640 1 -- 172.21.15.43:0/2382712987 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f6e2415c4c0 msgr2=0x7f6e240afd20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-09T15:11:11.380 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:11.377+0000 7f6e21d9b640 1 --2- 172.21.15.43:0/2382712987 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f6e2415c4c0 0x7f6e240afd20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T15:11:11.380 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:11.377+0000 7f6e21d9b640 1 -- 172.21.15.43:0/2382712987 >> v2:172.21.15.203:3300/0 conn(0x7f6e2406b790 msgr2=0x7f6e240cc930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-09T15:11:11.380 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:11.377+0000 7f6e21d9b640 1 --2- 172.21.15.43:0/2382712987 >> v2:172.21.15.203:3300/0 conn(0x7f6e2406b790 0x7f6e240cc930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T15:11:11.380 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:11.377+0000 7f6e21d9b640 1 -- 172.21.15.43:0/2382712987 --> v2:172.21.15.43:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6e240b03b0 con 0x7f6e2406ab70 2023-08-09T15:11:11.381 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:11.378+0000 7f6e21d9b640 1 --2- 172.21.15.43:0/2382712987 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f6e2406ab70 0x7f6e2406b250 secure :-1 s=READY pgs=2313 cs=0 l=1 rev1=1 crypto rx=0x7f6e240af600 tx=0x7f6e1802b910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ad050d24993238b7 server_cookie=0 in_seq=0 out_seq=0 2023-08-09T15:11:11.381 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:11.378+0000 7f6e2159a640 1 -- 172.21.15.43:0/2382712987 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6e18031870 con 0x7f6e2406ab70 2023-08-09T15:11:11.381 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:11.378+0000 7f6e2159a640 1 -- 172.21.15.43:0/2382712987 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6e180319d0 con 0x7f6e2406ab70 2023-08-09T15:11:11.381 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:11.378+0000 7f6e2159a640 1 -- 172.21.15.43:0/2382712987 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6e18031d60 con 0x7f6e2406ab70 2023-08-09T15:11:11.382 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:11.379+0000 7f6e2359e640 1 -- 172.21.15.43:0/2382712987 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f6e2406ab70 msgr2=0x7f6e2406b250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T15:11:11.382 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:11.379+0000 7f6e2359e640 1 --2- 172.21.15.43:0/2382712987 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f6e2406ab70 0x7f6e2406b250 secure :-1 s=READY pgs=2313 cs=0 l=1 rev1=1 crypto rx=0x7f6e240af600 tx=0x7f6e1802b910 comp rx=0 tx=0).stop 2023-08-09T15:11:11.382 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:11.379+0000 7f6e2359e640 1 -- 172.21.15.43:0/2382712987 shutdown_connections 2023-08-09T15:11:11.382 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:11.379+0000 7f6e2359e640 1 --2- 172.21.15.43:0/2382712987 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f6e2406ab70 0x7f6e2406b250 unknown :-1 s=CLOSED pgs=2313 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T15:11:11.382 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:11.379+0000 7f6e2359e640 1 --2- 172.21.15.43:0/2382712987 >> v2:172.21.15.203:3300/0 conn(0x7f6e2406b790 0x7f6e240cc930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T15:11:11.382 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:11.379+0000 7f6e2359e640 1 --2- 172.21.15.43:0/2382712987 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f6e2415c4c0 0x7f6e240afd20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T15:11:11.383 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:11.379+0000 7f6e2359e640 1 -- 172.21.15.43:0/2382712987 >> 172.21.15.43:0/2382712987 conn(0x7f6e24089fe0 msgr2=0x7f6e24099450 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-09T15:11:11.383 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:11.379+0000 7f6e2359e640 1 -- 172.21.15.43:0/2382712987 shutdown_connections 2023-08-09T15:11:11.383 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:11.379+0000 7f6e2359e640 1 -- 172.21.15.43:0/2382712987 wait complete. 2023-08-09T15:11:11.385 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:11.381+0000 7f6e2359e640 1 Processor -- start 2023-08-09T15:11:11.386 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:11.381+0000 7f6e2359e640 1 -- start start 2023-08-09T15:11:11.386 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:11.382+0000 7f6e2359e640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f6e2406ab70 0x7f6e24152480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T15:11:11.386 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:11.382+0000 7f6e2359e640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f6e2406b790 0x7f6e240ca6e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T15:11:11.387 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:11.382+0000 7f6e2359e640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f6e2415c4c0 0x7f6e24151d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T15:11:11.387 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:11.382+0000 7f6e2359e640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6e24152280 con 0x7f6e2406b790 2023-08-09T15:11:11.387 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:11.382+0000 7f6e2359e640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6e240c7ec0 con 0x7f6e2415c4c0 2023-08-09T15:11:11.387 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:11.382+0000 7f6e2359e640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6e240c7be0 con 0x7f6e2406ab70 2023-08-09T15:11:11.388 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:11.382+0000 7f6e21d9b640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f6e2406b790 0x7f6e240ca6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T15:11:11.388 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:11.382+0000 7f6e21d9b640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f6e2406b790 0x7f6e240ca6e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:46574/0 (socket says 172.21.15.43:46574) 2023-08-09T15:11:11.388 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:11.382+0000 7f6e21d9b640 1 -- 172.21.15.43:0/2090809521 learned_addr learned my addr 172.21.15.43:0/2090809521 (peer_addr_for_me v2:172.21.15.43:0/0) 2023-08-09T15:11:11.388 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:11.383+0000 7f6e2259c640 1 --2- 172.21.15.43:0/2090809521 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f6e2406ab70 0x7f6e24152480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T15:11:11.389 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:11.383+0000 7f6e21d9b640 1 -- 172.21.15.43:0/2090809521 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f6e2406ab70 msgr2=0x7f6e24152480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T15:11:11.389 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:11.383+0000 7f6e21d9b640 1 --2- 172.21.15.43:0/2090809521 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f6e2406ab70 0x7f6e24152480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T15:11:11.389 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:11.383+0000 7f6e21d9b640 1 -- 172.21.15.43:0/2090809521 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f6e2415c4c0 msgr2=0x7f6e24151d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T15:11:11.389 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:11.383+0000 7f6e21d9b640 1 --2- 172.21.15.43:0/2090809521 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f6e2415c4c0 0x7f6e24151d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T15:11:11.390 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:11.383+0000 7f6e21d9b640 1 -- 172.21.15.43:0/2090809521 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6e180094d0 con 0x7f6e2406b790 2023-08-09T15:11:11.390 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:11.383+0000 7f6e21d9b640 1 --2- 172.21.15.43:0/2090809521 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f6e2406b790 0x7f6e240ca6e0 secure :-1 s=READY pgs=2314 cs=0 l=1 rev1=1 crypto rx=0x7f6e18000c00 tx=0x7f6e180046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-09T15:11:11.390 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:11.384+0000 7f6e12ffd640 1 -- 172.21.15.43:0/2090809521 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6e1803ad10 con 0x7f6e2406b790 2023-08-09T15:11:11.391 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:11.384+0000 7f6e12ffd640 1 -- 172.21.15.43:0/2090809521 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6e1803ae70 con 0x7f6e2406b790 2023-08-09T15:11:11.391 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:11.384+0000 7f6e12ffd640 1 -- 172.21.15.43:0/2090809521 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6e18044610 con 0x7f6e2406b790 2023-08-09T15:11:11.391 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:11.384+0000 7f6e2359e640 1 -- 172.21.15.43:0/2090809521 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6e240c52a0 con 0x7f6e2406b790 2023-08-09T15:11:11.391 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:11.384+0000 7f6e2359e640 1 -- 172.21.15.43:0/2090809521 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6e240c5860 con 0x7f6e2406b790 2023-08-09T15:11:11.393 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:11.385+0000 7f6e2359e640 1 -- 172.21.15.43:0/2090809521 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6e240c5a50 con 0x7f6e2406b790 2023-08-09T15:11:11.393 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:11.386+0000 7f6e12ffd640 1 -- 172.21.15.43:0/2090809521 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 117) v1 ==== 88387+0+0 (secure 0 0 0) 0x7f6e1804d450 con 0x7f6e2406b790 2023-08-09T15:11:11.394 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:11.387+0000 7f6e12ffd640 1 --2- 172.21.15.43:0/2090809521 >> [v2:172.21.15.203:6832/4034093268,v1:172.21.15.203:6833/4034093268] conn(0x7f6e0406a0e0 0x7f6e0406c5a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T15:11:11.394 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:11.387+0000 7f6e12ffd640 1 -- 172.21.15.43:0/2090809521 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(211..211 src has 1..211) v4 ==== 7200+0+0 (secure 0 0 0) 0x7f6e1802d040 con 0x7f6e2406b790 2023-08-09T15:11:11.394 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:11.388+0000 7f6e2259c640 1 --2- 172.21.15.43:0/2090809521 >> [v2:172.21.15.203:6832/4034093268,v1:172.21.15.203:6833/4034093268] conn(0x7f6e0406a0e0 0x7f6e0406c5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T15:11:11.394 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:11.389+0000 7f6e2259c640 1 --2- 172.21.15.43:0/2090809521 >> [v2:172.21.15.203:6832/4034093268,v1:172.21.15.203:6833/4034093268] conn(0x7f6e0406a0e0 0x7f6e0406c5a0 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7f6e240cd140 tx=0x7f6e140084b0 comp rx=0 tx=0).ready entity=mgr.8833 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-09T15:11:11.395 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:11.391+0000 7f6e12ffd640 1 -- 172.21.15.43:0/2090809521 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178066 (secure 0 0 0) 0x7f6e180763b0 con 0x7f6e2406b790 2023-08-09T15:11:11.587 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:11.585+0000 7f6e2359e640 1 -- 172.21.15.43:0/2090809521 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f6e240b0c20 con 0x7f6e2406b790 2023-08-09T15:11:11.590 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:11.588+0000 7f6e12ffd640 1 -- 172.21.15.43:0/2090809521 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v211) v1 ==== 74+0+19745 (secure 0 0 0) 0x7f6e180799d0 con 0x7f6e2406b790 2023-08-09T15:11:11.590 INFO:teuthology.orchestra.run.smithi043.stdout: 2023-08-09T15:11:11.591 INFO:teuthology.orchestra.run.smithi043.stdout:{"epoch":211,"fsid":"daac87ee-1337-4250-a955-aea4e408a39b","created":"2023-08-09T13:33:41.701198+0000","modified":"2023-08-09T15:11:08.350101+0000","last_up_change":"2023-08-09T13:33:53.783453+0000","last_in_change":"2023-08-09T13:33:45.148904+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":25,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":"rbd","create_time":"2023-08-09T13:33:54.365137+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":".mgr","create_time":"2023-08-09T13:33:54.831803+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":24,"pool_name":"cephfs_metadata","create_time":"2023-08-09T15:08:43.920337+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"204","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":25,"pool_name":"cephfs_data","create_time":"2023-08-09T15:08:44.594489+0000","flags":8193,"flags_names":"hashpspool,selfmanaged_snaps","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"210","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":5,"snap_epoch":210,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"cbd49ecd-e896-4fbd-864e-5c37a829b165","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":203,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6800","nonce":502667335},{"type":"v1","addr":"172.21.15.43:6801","nonce":502667335}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6802","nonce":502667335},{"type":"v1","addr":"172.21.15.43:6803","nonce":502667335}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6806","nonce":502667335},{"type":"v1","addr":"172.21.15.43:6807","nonce":502667335}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6804","nonce":502667335},{"type":"v1","addr":"172.21.15.43:6805","nonce":502667335}]},"public_addr":"172.21.15.43:6801/502667335","cluster_addr":"172.21.15.43:6803/502667335","heartbeat_back_addr":"172.21.15.43:6807/502667335","heartbeat_front_addr":"172.21.15.43:6805/502667335","state":["exists","up"]},{"osd":1,"uuid":"75c7e0fe-1ec7-4fe8-a2f2-3661a65b8500","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":203,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6808","nonce":3289134610},{"type":"v1","addr":"172.21.15.43:6809","nonce":3289134610}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6810","nonce":3289134610},{"type":"v1","addr":"172.21.15.43:6812","nonce":3289134610}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6821","nonce":3289134610},{"type":"v1","addr":"172.21.15.43:6824","nonce":3289134610}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6815","nonce":3289134610},{"type":"v1","addr":"172.21.15.43:6818","nonce":3289134610}]},"public_addr":"172.21.15.43:6809/3289134610","cluster_addr":"172.21.15.43:6812/3289134610","heartbeat_back_addr":"172.21.15.43:6824/3289134610","heartbeat_front_addr":"172.21.15.43:6818/3289134610","state":["exists","up"]},{"osd":2,"uuid":"bb5c41e9-7f5f-4125-9b01-3bb8bfc2ea83","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":203,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6811","nonce":4079709736},{"type":"v1","addr":"172.21.15.43:6814","nonce":4079709736}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6817","nonce":4079709736},{"type":"v1","addr":"172.21.15.43:6820","nonce":4079709736}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6829","nonce":4079709736},{"type":"v1","addr":"172.21.15.43:6831","nonce":4079709736}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6823","nonce":4079709736},{"type":"v1","addr":"172.21.15.43:6826","nonce":4079709736}]},"public_addr":"172.21.15.43:6814/4079709736","cluster_addr":"172.21.15.43:6820/4079709736","heartbeat_back_addr":"172.21.15.43:6831/4079709736","heartbeat_front_addr":"172.21.15.43:6826/4079709736","state":["exists","up"]},{"osd":3,"uuid":"bfe4c8b4-694a-465c-87e6-2b2aab852613","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":203,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6813","nonce":1709538759},{"type":"v1","addr":"172.21.15.43:6816","nonce":1709538759}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6819","nonce":1709538759},{"type":"v1","addr":"172.21.15.43:6822","nonce":1709538759}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6828","nonce":1709538759},{"type":"v1","addr":"172.21.15.43:6830","nonce":1709538759}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6825","nonce":1709538759},{"type":"v1","addr":"172.21.15.43:6827","nonce":1709538759}]},"public_addr":"172.21.15.43:6816/1709538759","cluster_addr":"172.21.15.43:6822/1709538759","heartbeat_back_addr":"172.21.15.43:6830/1709538759","heartbeat_front_addr":"172.21.15.43:6827/1709538759","state":["exists","up"]},{"osd":4,"uuid":"fa2e8a77-aba1-4756-853e-0d174a5bdeba","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":203,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6800","nonce":4196866481},{"type":"v1","addr":"172.21.15.203:6801","nonce":4196866481}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6802","nonce":4196866481},{"type":"v1","addr":"172.21.15.203:6803","nonce":4196866481}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6806","nonce":4196866481},{"type":"v1","addr":"172.21.15.203:6807","nonce":4196866481}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6804","nonce":4196866481},{"type":"v1","addr":"172.21.15.203:6805","nonce":4196866481}]},"public_addr":"172.21.15.203:6801/4196866481","cluster_addr":"172.21.15.203:6803/4196866481","heartbeat_back_addr":"172.21.15.203:6807/4196866481","heartbeat_front_addr":"172.21.15.203:6805/4196866481","state":["exists","up"]},{"osd":5,"uuid":"81811e4b-4d24-4f51-a82c-f1d700a2a00d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":207,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6816","nonce":3059290081},{"type":"v1","addr":"172.21.15.203:6817","nonce":3059290081}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6818","nonce":3059290081},{"type":"v1","addr":"172.21.15.203:6819","nonce":3059290081}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6822","nonce":3059290081},{"type":"v1","addr":"172.21.15.203:6823","nonce":3059290081}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6820","nonce":3059290081},{"type":"v1","addr":"172.21.15.203:6821","nonce":3059290081}]},"public_addr":"172.21.15.203:6817/3059290081","cluster_addr":"172.21.15.203:6819/3059290081","heartbeat_back_addr":"172.21.15.203:6823/3059290081","heartbeat_front_addr":"172.21.15.203:6821/3059290081","state":["exists","up"]},{"osd":6,"uuid":"307bac4d-6a5b-443d-af75-56d695d7674f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":203,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6824","nonce":512155569},{"type":"v1","addr":"172.21.15.203:6825","nonce":512155569}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6826","nonce":512155569},{"type":"v1","addr":"172.21.15.203:6827","nonce":512155569}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6830","nonce":512155569},{"type":"v1","addr":"172.21.15.203:6831","nonce":512155569}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6828","nonce":512155569},{"type":"v1","addr":"172.21.15.203:6829","nonce":512155569}]},"public_addr":"172.21.15.203:6825/512155569","cluster_addr":"172.21.15.203:6827/512155569","heartbeat_back_addr":"172.21.15.203:6831/512155569","heartbeat_front_addr":"172.21.15.203:6829/512155569","state":["exists","up"]},{"osd":7,"uuid":"a95dc7ca-9f1b-4295-af70-68a22a2d7d5e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":207,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6808","nonce":3331450438},{"type":"v1","addr":"172.21.15.203:6809","nonce":3331450438}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6810","nonce":3331450438},{"type":"v1","addr":"172.21.15.203:6811","nonce":3331450438}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6814","nonce":3331450438},{"type":"v1","addr":"172.21.15.203:6815","nonce":3331450438}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6812","nonce":3331450438},{"type":"v1","addr":"172.21.15.203:6813","nonce":3331450438}]},"public_addr":"172.21.15.203:6809/3331450438","cluster_addr":"172.21.15.203:6811/3331450438","heartbeat_back_addr":"172.21.15.203:6815/3331450438","heartbeat_front_addr":"172.21.15.203:6813/3331450438","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-09T13:33:50.547652+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-09T13:33:50.589469+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-09T13:33:50.561724+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-09T13:33:50.599650+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-09T13:33:50.411855+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-09T13:33:50.219681+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-09T13:33:50.504664+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-09T13:33:50.441248+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"25.f","mappings":[{"from":0,"to":1}]},{"pgid":"25.3f","mappings":[{"from":0,"to":2}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.203:0/822076630":"2023-08-10T15:11:08.350045+0000","172.21.15.203:0/79247817":"2023-08-10T15:11:08.350045+0000","172.21.15.203:6833/4012896436":"2023-08-10T15:11:08.350045+0000","172.21.15.203:6832/4012896436":"2023-08-10T15:11:08.350045+0000","172.21.15.203:0/3345703784":"2023-08-10T15:11:08.350045+0000","172.21.15.203:0/1987920954":"2023-08-10T15:11:08.350045+0000","172.21.15.203:0/4253505338":"2023-08-10T15:11:08.350045+0000","172.21.15.203:6833/3082732769":"2023-08-10T15:09:01.285901+0000","172.21.15.203:6832/3082732769":"2023-08-10T15:09:01.285901+0000","172.21.15.203:0/1756781921":"2023-08-10T15:09:01.285901+0000","172.21.15.203:0/454071895":"2023-08-10T15:09:01.285901+0000","172.21.15.203:0/2854569557":"2023-08-10T15:09:01.285901+0000","172.21.15.203:0/2821325802":"2023-08-10T15:11:08.350045+0000","172.21.15.203:0/1159944241":"2023-08-10T15:09:01.285901+0000","172.21.15.203:0/1645632393":"2023-08-10T15:09:01.285901+0000","172.21.15.203:0/840449585":"2023-08-10T15:09:01.285901+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":7,"snaps":[{"begin":104,"length":1},{"begin":221,"length":1}]},{"pool":25,"snaps":[{"begin":2,"length":4}]}],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-08-09T15:11:11.593 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:11.592+0000 7f6e2359e640 1 -- 172.21.15.43:0/2090809521 >> [v2:172.21.15.203:6832/4034093268,v1:172.21.15.203:6833/4034093268] conn(0x7f6e0406a0e0 msgr2=0x7f6e0406c5a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T15:11:11.593 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:11.592+0000 7f6e2359e640 1 --2- 172.21.15.43:0/2090809521 >> [v2:172.21.15.203:6832/4034093268,v1:172.21.15.203:6833/4034093268] conn(0x7f6e0406a0e0 0x7f6e0406c5a0 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7f6e240cd140 tx=0x7f6e140084b0 comp rx=0 tx=0).stop 2023-08-09T15:11:11.594 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:11.592+0000 7f6e2359e640 1 -- 172.21.15.43:0/2090809521 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f6e2406b790 msgr2=0x7f6e240ca6e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T15:11:11.594 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:11.592+0000 7f6e2359e640 1 --2- 172.21.15.43:0/2090809521 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f6e2406b790 0x7f6e240ca6e0 secure :-1 s=READY pgs=2314 cs=0 l=1 rev1=1 crypto rx=0x7f6e18000c00 tx=0x7f6e180046c0 comp rx=0 tx=0).stop 2023-08-09T15:11:11.594 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:11.592+0000 7f6e2359e640 1 -- 172.21.15.43:0/2090809521 shutdown_connections 2023-08-09T15:11:11.594 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:11.592+0000 7f6e2359e640 1 --2- 172.21.15.43:0/2090809521 >> [v2:172.21.15.203:6832/4034093268,v1:172.21.15.203:6833/4034093268] conn(0x7f6e0406a0e0 0x7f6e0406c5a0 unknown :-1 s=CLOSED pgs=26 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T15:11:11.594 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:11.592+0000 7f6e2359e640 1 --2- 172.21.15.43:0/2090809521 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f6e2415c4c0 0x7f6e24151d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T15:11:11.594 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:11.592+0000 7f6e2359e640 1 --2- 172.21.15.43:0/2090809521 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f6e2406b790 0x7f6e240ca6e0 unknown :-1 s=CLOSED pgs=2314 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T15:11:11.594 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:11.592+0000 7f6e2359e640 1 --2- 172.21.15.43:0/2090809521 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f6e2406ab70 0x7f6e24152480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T15:11:11.594 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:11.592+0000 7f6e2359e640 1 -- 172.21.15.43:0/2090809521 >> 172.21.15.43:0/2090809521 conn(0x7f6e24089fe0 msgr2=0x7f6e240aec50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-09T15:11:11.595 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:11.592+0000 7f6e2359e640 1 -- 172.21.15.43:0/2090809521 shutdown_connections 2023-08-09T15:11:11.595 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:11.592+0000 7f6e2359e640 1 -- 172.21.15.43:0/2090809521 wait complete. 2023-08-09T15:11:11.613 INFO:tasks.cephfs.filesystem:Destroying file system cephfs and related pools 2023-08-09T15:11:11.614 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-08-09T15:11:11.842 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:11.841+0000 7f32a1174640 1 Processor -- start 2023-08-09T15:11:11.842 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:11.841+0000 7f32a1174640 1 -- start start 2023-08-09T15:11:11.844 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:11.843+0000 7f32a1174640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f329c15c620 0x7f329c0cfe70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T15:11:11.845 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:11.843+0000 7f32a1174640 1 -- --> v1:172.21.15.43:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f329c0b2330 con 0x7f329c0cf4f0 2023-08-09T15:11:11.845 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:11.843+0000 7f32a1174640 1 -- --> v1:172.21.15.203:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f329c0d2ec0 con 0x7f329c0d0ef0 2023-08-09T15:11:11.845 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:11.843+0000 7f32a1174640 1 -- --> v2:172.21.15.43:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f329c0d0500 con 0x7f329c15c620 2023-08-09T15:11:11.845 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:11.843+0000 7f329bfff640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f329c15c620 0x7f329c0cfe70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T15:11:11.845 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:11.843+0000 7f329bfff640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f329c15c620 0x7f329c0cfe70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:46578/0 (socket says 172.21.15.43:46578) 2023-08-09T15:11:11.845 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:11.843+0000 7f329bfff640 1 -- 172.21.15.43:0/3065105197 learned_addr learned my addr 172.21.15.43:0/3065105197 (peer_addr_for_me v2:172.21.15.43:0/0) 2023-08-09T15:11:11.845 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:11.843+0000 7f329b7fe640 1 --1- 172.21.15.43:0/3065105197 >> v1:172.21.15.43:6789/0 conn(0x7f329c0cf4f0 0x7f329c0d07e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.43:6789/0 says I am v1:172.21.15.43:45568/0 (socket says 172.21.15.43:45568) 2023-08-09T15:11:11.846 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:11.844+0000 7f329bfff640 1 -- 172.21.15.43:0/3065105197 >> v1:172.21.15.203:6789/0 conn(0x7f329c0d0ef0 legacy=0x7f329c0b27c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-09T15:11:11.846 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:11.844+0000 7f329bfff640 1 -- 172.21.15.43:0/3065105197 >> v1:172.21.15.43:6789/0 conn(0x7f329c0cf4f0 legacy=0x7f329c0d07e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T15:11:11.846 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:11.844+0000 7f329bfff640 1 -- 172.21.15.43:0/3065105197 --> v2:172.21.15.43:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f329c0b1a00 con 0x7f329c15c620 2023-08-09T15:11:11.846 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:11.844+0000 7f329affd640 1 -- 172.21.15.43:0/3065105197 <== mon.0 v1:172.21.15.43:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3266301267 0 0) 0x7f329c0b2330 con 0x7f329c0cf4f0 2023-08-09T15:11:11.847 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:11.845+0000 7f329bfff640 1 --2- 172.21.15.43:0/3065105197 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f329c15c620 0x7f329c0cfe70 secure :-1 s=READY pgs=2316 cs=0 l=1 rev1=1 crypto rx=0x7f328c009e40 tx=0x7f328c02be40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1674cf9cc401b312 server_cookie=0 in_seq=0 out_seq=0 2023-08-09T15:11:11.847 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:11.845+0000 7f329affd640 1 -- 172.21.15.43:0/3065105197 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f328c02c9e0 con 0x7f329c15c620 2023-08-09T15:11:11.848 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:11.845+0000 7f329affd640 1 -- 172.21.15.43:0/3065105197 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f328c02cb40 con 0x7f329c15c620 2023-08-09T15:11:11.848 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:11.845+0000 7f329affd640 1 -- 172.21.15.43:0/3065105197 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f328c0343f0 con 0x7f329c15c620 2023-08-09T15:11:11.848 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:11.846+0000 7f32a1174640 1 -- 172.21.15.43:0/3065105197 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f329c15c620 msgr2=0x7f329c0cfe70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T15:11:11.848 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:11.846+0000 7f32a1174640 1 --2- 172.21.15.43:0/3065105197 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f329c15c620 0x7f329c0cfe70 secure :-1 s=READY pgs=2316 cs=0 l=1 rev1=1 crypto rx=0x7f328c009e40 tx=0x7f328c02be40 comp rx=0 tx=0).stop 2023-08-09T15:11:11.848 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:11.846+0000 7f32a1174640 1 -- 172.21.15.43:0/3065105197 shutdown_connections 2023-08-09T15:11:11.848 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:11.846+0000 7f32a1174640 1 --2- 172.21.15.43:0/3065105197 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f329c15c620 0x7f329c0cfe70 unknown :-1 s=CLOSED pgs=2316 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T15:11:11.848 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:11.846+0000 7f32a1174640 1 -- 172.21.15.43:0/3065105197 >> 172.21.15.43:0/3065105197 conn(0x7f329c09b650 msgr2=0x7f329c0d22a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-09T15:11:11.849 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:11.846+0000 7f32a1174640 1 -- 172.21.15.43:0/3065105197 shutdown_connections 2023-08-09T15:11:11.849 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:11.846+0000 7f32a1174640 1 -- 172.21.15.43:0/3065105197 wait complete. 2023-08-09T15:11:11.850 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:11.848+0000 7f32a1174640 1 Processor -- start 2023-08-09T15:11:11.850 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:11.849+0000 7f32a1174640 1 -- start start 2023-08-09T15:11:11.851 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:11.850+0000 7f32a1174640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f329c0cf4f0 0x7f329c148dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T15:11:11.851 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:11.850+0000 7f32a1174640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f329c0d0ef0 0x7f329c1f1d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T15:11:11.852 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:11.850+0000 7f32a1174640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f329c15c620 0x7f329c0c6e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T15:11:11.852 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:11.850+0000 7f32a1174640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f329c15ee50 con 0x7f329c15c620 2023-08-09T15:11:11.852 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:11.850+0000 7f32a1174640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f329c1f2700 con 0x7f329c0d0ef0 2023-08-09T15:11:11.852 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:11.850+0000 7f32a1174640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f329c1f29e0 con 0x7f329c0cf4f0 2023-08-09T15:11:11.853 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:11.850+0000 7f32a0973640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f329c15c620 0x7f329c0c6e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T15:11:11.853 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:11.850+0000 7f32a0973640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f329c15c620 0x7f329c0c6e20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:46594/0 (socket says 172.21.15.43:46594) 2023-08-09T15:11:11.854 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:11.850+0000 7f32a0973640 1 -- 172.21.15.43:0/2925691533 learned_addr learned my addr 172.21.15.43:0/2925691533 (peer_addr_for_me v2:172.21.15.43:0/0) 2023-08-09T15:11:11.854 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:11.851+0000 7f329bfff640 1 --2- 172.21.15.43:0/2925691533 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f329c0cf4f0 0x7f329c148dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T15:11:11.854 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:11.851+0000 7f329b7fe640 1 --2- 172.21.15.43:0/2925691533 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f329c0d0ef0 0x7f329c1f1d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T15:11:11.854 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:11.851+0000 7f32a0973640 1 -- 172.21.15.43:0/2925691533 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f329c0cf4f0 msgr2=0x7f329c148dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T15:11:11.854 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:11.851+0000 7f32a0973640 1 --2- 172.21.15.43:0/2925691533 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f329c0cf4f0 0x7f329c148dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T15:11:11.854 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:11.851+0000 7f32a0973640 1 -- 172.21.15.43:0/2925691533 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f329c0d0ef0 msgr2=0x7f329c1f1d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T15:11:11.854 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:11.851+0000 7f32a0973640 1 --2- 172.21.15.43:0/2925691533 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f329c0d0ef0 0x7f329c1f1d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T15:11:11.854 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:11.851+0000 7f32a0973640 1 -- 172.21.15.43:0/2925691533 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f328c009ad0 con 0x7f329c15c620 2023-08-09T15:11:11.855 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:11.851+0000 7f32a0973640 1 --2- 172.21.15.43:0/2925691533 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f329c15c620 0x7f329c0c6e20 secure :-1 s=READY pgs=2317 cs=0 l=1 rev1=1 crypto rx=0x7f329400c410 tx=0x7f329400c8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-09T15:11:11.855 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:11.852+0000 7f3298ff9640 1 -- 172.21.15.43:0/2925691533 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3294011790 con 0x7f329c15c620 2023-08-09T15:11:11.855 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:11.852+0000 7f3298ff9640 1 -- 172.21.15.43:0/2925691533 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f32940118f0 con 0x7f329c15c620 2023-08-09T15:11:11.855 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:11.852+0000 7f3298ff9640 1 -- 172.21.15.43:0/2925691533 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3294011ce0 con 0x7f329c15c620 2023-08-09T15:11:11.855 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:11.852+0000 7f32a1174640 1 -- 172.21.15.43:0/2925691533 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f329c1f2cc0 con 0x7f329c15c620 2023-08-09T15:11:11.855 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:11.852+0000 7f32a1174640 1 -- 172.21.15.43:0/2925691533 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f329c149300 con 0x7f329c15c620 2023-08-09T15:11:11.856 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:11.853+0000 7f32a1174640 1 -- 172.21.15.43:0/2925691533 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f329c15e940 con 0x7f329c15c620 2023-08-09T15:11:11.856 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:11.854+0000 7f3298ff9640 1 -- 172.21.15.43:0/2925691533 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 117) v1 ==== 88387+0+0 (secure 0 0 0) 0x7f329401c460 con 0x7f329c15c620 2023-08-09T15:11:11.857 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:11.856+0000 7f3298ff9640 1 --2- 172.21.15.43:0/2925691533 >> [v2:172.21.15.203:6832/4034093268,v1:172.21.15.203:6833/4034093268] conn(0x7f328806a130 0x7f328806c5f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T15:11:11.857 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:11.856+0000 7f3298ff9640 1 -- 172.21.15.43:0/2925691533 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(211..211 src has 1..211) v4 ==== 7200+0+0 (secure 0 0 0) 0x7f3294088740 con 0x7f329c15c620 2023-08-09T15:11:11.857 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:11.856+0000 7f329bfff640 1 --2- 172.21.15.43:0/2925691533 >> [v2:172.21.15.203:6832/4034093268,v1:172.21.15.203:6833/4034093268] conn(0x7f328806a130 0x7f328806c5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T15:11:11.858 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:11.857+0000 7f329bfff640 1 --2- 172.21.15.43:0/2925691533 >> [v2:172.21.15.203:6832/4034093268,v1:172.21.15.203:6833/4034093268] conn(0x7f328806a130 0x7f328806c5f0 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7f328c035000 tx=0x7f328c030000 comp rx=0 tx=0).ready entity=mgr.8833 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-09T15:11:11.861 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:11.860+0000 7f3298ff9640 1 -- 172.21.15.43:0/2925691533 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178066 (secure 0 0 0) 0x7f329c15e940 con 0x7f329c15c620 2023-08-09T15:11:12.092 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:12.090+0000 7f32a1174640 1 -- 172.21.15.43:0/2925691533 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f329c15e940 con 0x7f329c15c620 2023-08-09T15:11:12.093 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:12.092+0000 7f3298ff9640 1 -- 172.21.15.43:0/2925691533 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 96 v96) v1 ==== 94+0+4883 (secure 0 0 0) 0x7f329c15e940 con 0x7f329c15c620 2023-08-09T15:11:12.093 INFO:teuthology.orchestra.run.smithi043.stdout: 2023-08-09T15:11:12.093 INFO:teuthology.orchestra.run.smithi043.stdout:{"epoch":96,"default_fscid":11,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":7703,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.203:6836/1774059384","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6835","nonce":1774059384},{"type":"v1","addr":"172.21.15.203:6836","nonce":1774059384}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":91},{"gid":8665,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.203:6837/682905899","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6834","nonce":682905899},{"type":"v1","addr":"172.21.15.203:6837","nonce":682905899}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":91},{"gid":10752,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.43:6834/4168463642","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6832","nonce":4168463642},{"type":"v1","addr":"172.21.15.43:6834","nonce":4168463642}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":90}],"filesystems":[{"mdsmap":{"epoch":96,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-08-09T15:08:45.563041+0000","modified":"2023-08-09T15:08:49.150213+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":10755},"failed":[],"damaged":[],"stopped":[],"info":{"gid_10755":{"gid":10755,"name":"a","rank":0,"incarnation":92,"state":"up:active","state_seq":3,"addr":"172.21.15.43:6835/349688227","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6833","nonce":349688227},{"type":"v1","addr":"172.21.15.43:6835","nonce":349688227}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[25],"metadata_pool":24,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":11}]} 2023-08-09T15:11:12.094 INFO:teuthology.orchestra.run.smithi043.stderr:dumped fsmap epoch 96 2023-08-09T15:11:12.095 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:12.094+0000 7f32a1174640 1 -- 172.21.15.43:0/2925691533 >> [v2:172.21.15.203:6832/4034093268,v1:172.21.15.203:6833/4034093268] conn(0x7f328806a130 msgr2=0x7f328806c5f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T15:11:12.096 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:12.094+0000 7f32a1174640 1 --2- 172.21.15.43:0/2925691533 >> [v2:172.21.15.203:6832/4034093268,v1:172.21.15.203:6833/4034093268] conn(0x7f328806a130 0x7f328806c5f0 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7f328c035000 tx=0x7f328c030000 comp rx=0 tx=0).stop 2023-08-09T15:11:12.096 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:12.094+0000 7f32a1174640 1 -- 172.21.15.43:0/2925691533 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f329c15c620 msgr2=0x7f329c0c6e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T15:11:12.096 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:12.094+0000 7f32a1174640 1 --2- 172.21.15.43:0/2925691533 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f329c15c620 0x7f329c0c6e20 secure :-1 s=READY pgs=2317 cs=0 l=1 rev1=1 crypto rx=0x7f329400c410 tx=0x7f329400c8d0 comp rx=0 tx=0).stop 2023-08-09T15:11:12.096 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:12.094+0000 7f32a1174640 1 -- 172.21.15.43:0/2925691533 shutdown_connections 2023-08-09T15:11:12.096 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:12.094+0000 7f32a1174640 1 --2- 172.21.15.43:0/2925691533 >> [v2:172.21.15.203:6832/4034093268,v1:172.21.15.203:6833/4034093268] conn(0x7f328806a130 0x7f328806c5f0 unknown :-1 s=CLOSED pgs=27 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T15:11:12.096 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:12.094+0000 7f32a1174640 1 --2- 172.21.15.43:0/2925691533 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f329c15c620 0x7f329c0c6e20 unknown :-1 s=CLOSED pgs=2317 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T15:11:12.097 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:12.094+0000 7f32a1174640 1 --2- 172.21.15.43:0/2925691533 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f329c0d0ef0 0x7f329c1f1d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T15:11:12.097 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:12.094+0000 7f32a1174640 1 --2- 172.21.15.43:0/2925691533 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f329c0cf4f0 0x7f329c148dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T15:11:12.097 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:12.094+0000 7f32a1174640 1 -- 172.21.15.43:0/2925691533 >> 172.21.15.43:0/2925691533 conn(0x7f329c09b650 msgr2=0x7f329c0b0fc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-09T15:11:12.097 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:12.094+0000 7f32a1174640 1 -- 172.21.15.43:0/2925691533 shutdown_connections 2023-08-09T15:11:12.097 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:12.095+0000 7f32a1174640 1 -- 172.21.15.43:0/2925691533 wait complete. 2023-08-09T15:11:12.115 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-08-09T15:11:12.337 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:12.336+0000 7fc2c3fff640 1 Processor -- start 2023-08-09T15:11:12.337 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:12.336+0000 7fc2c3fff640 1 -- start start 2023-08-09T15:11:12.339 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:12.337+0000 7fc2c3fff640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7fc2c415f7b0 0x7fc2c40b4760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T15:11:12.339 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:12.337+0000 7fc2c3fff640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fc2c40b1050 0x7fc2c40ccfa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T15:11:12.339 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:12.337+0000 7fc2c3fff640 1 -- --> v1:172.21.15.203:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc2c40cd930 con 0x7fc2c40cd4e0 2023-08-09T15:11:12.339 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:12.337+0000 7fc2c3fff640 1 -- --> v2:172.21.15.43:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc2c40b7bf0 con 0x7fc2c415f7b0 2023-08-09T15:11:12.339 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:12.337+0000 7fc2c3fff640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc2c40b7ed0 con 0x7fc2c40b1050 2023-08-09T15:11:12.340 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:12.338+0000 7fc2c2ffd640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7fc2c415f7b0 0x7fc2c40b4760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T15:11:12.340 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:12.338+0000 7fc2c2ffd640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7fc2c415f7b0 0x7fc2c40b4760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:46610/0 (socket says 172.21.15.43:46610) 2023-08-09T15:11:12.340 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:12.338+0000 7fc2c2ffd640 1 -- 172.21.15.43:0/4097840054 learned_addr learned my addr 172.21.15.43:0/4097840054 (peer_addr_for_me v2:172.21.15.43:0/0) 2023-08-09T15:11:12.340 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:12.338+0000 7fc2c27fc640 1 --2- 172.21.15.43:0/4097840054 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fc2c40b1050 0x7fc2c40ccfa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T15:11:12.341 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:12.338+0000 7fc2c2ffd640 1 -- 172.21.15.43:0/4097840054 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fc2c40b1050 msgr2=0x7fc2c40ccfa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-09T15:11:12.341 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:12.338+0000 7fc2c2ffd640 1 --2- 172.21.15.43:0/4097840054 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fc2c40b1050 0x7fc2c40ccfa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T15:11:12.341 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:12.338+0000 7fc2c2ffd640 1 -- 172.21.15.43:0/4097840054 >> v1:172.21.15.203:6789/0 conn(0x7fc2c40cd4e0 legacy=0x7fc2c40ceae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-09T15:11:12.341 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:12.338+0000 7fc2c2ffd640 1 -- 172.21.15.43:0/4097840054 --> v2:172.21.15.43:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc2c40d0370 con 0x7fc2c415f7b0 2023-08-09T15:11:12.342 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:12.339+0000 7fc2c2ffd640 1 --2- 172.21.15.43:0/4097840054 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc2c415f7b0 0x7fc2c40b4760 secure :-1 s=READY pgs=2318 cs=0 l=1 rev1=1 crypto rx=0x7fc2b4009e50 tx=0x7fc2b402bed0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e1d1393956c66f3e server_cookie=0 in_seq=0 out_seq=0 2023-08-09T15:11:12.342 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:12.340+0000 7fc2c1ffb640 1 -- 172.21.15.43:0/4097840054 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc2b402ca70 con 0x7fc2c415f7b0 2023-08-09T15:11:12.342 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:12.340+0000 7fc2c1ffb640 1 -- 172.21.15.43:0/4097840054 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc2b402cbd0 con 0x7fc2c415f7b0 2023-08-09T15:11:12.342 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:12.340+0000 7fc2c1ffb640 1 -- 172.21.15.43:0/4097840054 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc2b40343f0 con 0x7fc2c415f7b0 2023-08-09T15:11:12.343 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:12.340+0000 7fc2c3fff640 1 -- 172.21.15.43:0/4097840054 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc2c415f7b0 msgr2=0x7fc2c40b4760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T15:11:12.343 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:12.340+0000 7fc2c3fff640 1 --2- 172.21.15.43:0/4097840054 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc2c415f7b0 0x7fc2c40b4760 secure :-1 s=READY pgs=2318 cs=0 l=1 rev1=1 crypto rx=0x7fc2b4009e50 tx=0x7fc2b402bed0 comp rx=0 tx=0).stop 2023-08-09T15:11:12.343 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:12.340+0000 7fc2c3fff640 1 -- 172.21.15.43:0/4097840054 shutdown_connections 2023-08-09T15:11:12.343 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:12.340+0000 7fc2c3fff640 1 --2- 172.21.15.43:0/4097840054 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fc2c40b1050 0x7fc2c40ccfa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T15:11:12.343 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:12.340+0000 7fc2c3fff640 1 --2- 172.21.15.43:0/4097840054 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc2c415f7b0 0x7fc2c40b4760 unknown :-1 s=CLOSED pgs=2318 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T15:11:12.343 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:12.340+0000 7fc2c3fff640 1 -- 172.21.15.43:0/4097840054 >> 172.21.15.43:0/4097840054 conn(0x7fc2c406a2b0 msgr2=0x7fc2c406a690 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-09T15:11:12.343 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:12.341+0000 7fc2c3fff640 1 -- 172.21.15.43:0/4097840054 shutdown_connections 2023-08-09T15:11:12.344 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:12.341+0000 7fc2c3fff640 1 -- 172.21.15.43:0/4097840054 wait complete. 2023-08-09T15:11:12.344 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:12.342+0000 7fc2c3fff640 1 Processor -- start 2023-08-09T15:11:12.344 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:12.343+0000 7fc2c3fff640 1 -- start start 2023-08-09T15:11:12.345 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:12.344+0000 7fc2c3fff640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc2c40b1050 0x7fc2c40b1e90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T15:11:12.345 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:12.344+0000 7fc2c3fff640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fc2c40cd4e0 0x7fc2c4142690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T15:11:12.345 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:12.344+0000 7fc2c3fff640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fc2c415f7b0 0x7fc2c41446b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T15:11:12.345 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:12.344+0000 7fc2c3fff640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc2c40ca3f0 con 0x7fc2c40b1050 2023-08-09T15:11:12.345 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:12.344+0000 7fc2c3fff640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc2c40cac90 con 0x7fc2c415f7b0 2023-08-09T15:11:12.346 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:12.344+0000 7fc2c3fff640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc2c4142bd0 con 0x7fc2c40cd4e0 2023-08-09T15:11:12.347 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:12.344+0000 7fc2c2ffd640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc2c40b1050 0x7fc2c40b1e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T15:11:12.347 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:12.344+0000 7fc2c2ffd640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc2c40b1050 0x7fc2c40b1e90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:46622/0 (socket says 172.21.15.43:46622) 2023-08-09T15:11:12.347 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:12.344+0000 7fc2c2ffd640 1 -- 172.21.15.43:0/561652415 learned_addr learned my addr 172.21.15.43:0/561652415 (peer_addr_for_me v2:172.21.15.43:0/0) 2023-08-09T15:11:12.347 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:12.344+0000 7fc2c27fc640 1 --2- 172.21.15.43:0/561652415 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fc2c40cd4e0 0x7fc2c4142690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T15:11:12.347 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:12.344+0000 7fc2c2ffd640 1 -- 172.21.15.43:0/561652415 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fc2c40cd4e0 msgr2=0x7fc2c4142690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T15:11:12.347 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:12.344+0000 7fc2c37fe640 1 --2- 172.21.15.43:0/561652415 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fc2c415f7b0 0x7fc2c41446b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T15:11:12.347 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:12.344+0000 7fc2c2ffd640 1 --2- 172.21.15.43:0/561652415 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fc2c40cd4e0 0x7fc2c4142690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T15:11:12.348 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:12.344+0000 7fc2c2ffd640 1 -- 172.21.15.43:0/561652415 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fc2c415f7b0 msgr2=0x7fc2c41446b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T15:11:12.348 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:12.345+0000 7fc2c2ffd640 1 --2- 172.21.15.43:0/561652415 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fc2c415f7b0 0x7fc2c41446b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T15:11:12.348 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:12.345+0000 7fc2c2ffd640 1 -- 172.21.15.43:0/561652415 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc2b4009b30 con 0x7fc2c40b1050 2023-08-09T15:11:12.348 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:12.345+0000 7fc2c37fe640 1 --2- 172.21.15.43:0/561652415 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fc2c415f7b0 0x7fc2c41446b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-08-09T15:11:12.348 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:12.345+0000 7fc2c2ffd640 1 --2- 172.21.15.43:0/561652415 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc2c40b1050 0x7fc2c40b1e90 secure :-1 s=READY pgs=2319 cs=0 l=1 rev1=1 crypto rx=0x7fc2b4036040 tx=0x7fc2b4004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-09T15:11:12.348 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:12.345+0000 7fc2bb7fe640 1 -- 172.21.15.43:0/561652415 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc2b4009510 con 0x7fc2c40b1050 2023-08-09T15:11:12.348 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:12.345+0000 7fc2bb7fe640 1 -- 172.21.15.43:0/561652415 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc2b403e450 con 0x7fc2c40b1050 2023-08-09T15:11:12.349 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:12.345+0000 7fc2bb7fe640 1 -- 172.21.15.43:0/561652415 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc2b403e800 con 0x7fc2c40b1050 2023-08-09T15:11:12.349 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:12.345+0000 7fc2c3fff640 1 -- 172.21.15.43:0/561652415 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc2c40b4a50 con 0x7fc2c40b1050 2023-08-09T15:11:12.349 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:12.346+0000 7fc2c3fff640 1 -- 172.21.15.43:0/561652415 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc2c40ca920 con 0x7fc2c40b1050 2023-08-09T15:11:12.349 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:12.347+0000 7fc2c3fff640 1 -- 172.21.15.43:0/561652415 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc2c40739f0 con 0x7fc2c40b1050 2023-08-09T15:11:12.349 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:12.348+0000 7fc2bb7fe640 1 -- 172.21.15.43:0/561652415 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 117) v1 ==== 88387+0+0 (secure 0 0 0) 0x7fc2b403e960 con 0x7fc2c40b1050 2023-08-09T15:11:12.350 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:12.348+0000 7fc2bb7fe640 1 --2- 172.21.15.43:0/561652415 >> [v2:172.21.15.203:6832/4034093268,v1:172.21.15.203:6833/4034093268] conn(0x7fc2a406a0e0 0x7fc2a406c5a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T15:11:12.350 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:12.349+0000 7fc2bb7fe640 1 -- 172.21.15.43:0/561652415 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(211..211 src has 1..211) v4 ==== 7200+0+0 (secure 0 0 0) 0x7fc2b40ab6f0 con 0x7fc2c40b1050 2023-08-09T15:11:12.350 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:12.349+0000 7fc2c27fc640 1 --2- 172.21.15.43:0/561652415 >> [v2:172.21.15.203:6832/4034093268,v1:172.21.15.203:6833/4034093268] conn(0x7fc2a406a0e0 0x7fc2a406c5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T15:11:12.352 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:12.350+0000 7fc2c27fc640 1 --2- 172.21.15.43:0/561652415 >> [v2:172.21.15.203:6832/4034093268,v1:172.21.15.203:6833/4034093268] conn(0x7fc2a406a0e0 0x7fc2a406c5a0 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7fc2c40d1f80 tx=0x7fc2ac008040 comp rx=0 tx=0).ready entity=mgr.8833 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-09T15:11:12.354 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:12.353+0000 7fc2bb7fe640 1 -- 172.21.15.43:0/561652415 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178066 (secure 0 0 0) 0x7fc2b4076930 con 0x7fc2c40b1050 2023-08-09T15:11:12.589 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:12.587+0000 7fc2c3fff640 1 -- 172.21.15.43:0/561652415 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fc2c40cd930 con 0x7fc2c40b1050 2023-08-09T15:11:12.590 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:12.589+0000 7fc2bb7fe640 1 -- 172.21.15.43:0/561652415 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 96 v96) v1 ==== 94+0+4883 (secure 0 0 0) 0x7fc2c40cd930 con 0x7fc2c40b1050 2023-08-09T15:11:12.590 INFO:teuthology.orchestra.run.smithi043.stdout: 2023-08-09T15:11:12.591 INFO:teuthology.orchestra.run.smithi043.stdout:{"epoch":96,"default_fscid":11,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":7703,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.203:6836/1774059384","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6835","nonce":1774059384},{"type":"v1","addr":"172.21.15.203:6836","nonce":1774059384}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":91},{"gid":8665,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.203:6837/682905899","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6834","nonce":682905899},{"type":"v1","addr":"172.21.15.203:6837","nonce":682905899}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":91},{"gid":10752,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.43:6834/4168463642","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6832","nonce":4168463642},{"type":"v1","addr":"172.21.15.43:6834","nonce":4168463642}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":90}],"filesystems":[{"mdsmap":{"epoch":96,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-08-09T15:08:45.563041+0000","modified":"2023-08-09T15:08:49.150213+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":10755},"failed":[],"damaged":[],"stopped":[],"info":{"gid_10755":{"gid":10755,"name":"a","rank":0,"incarnation":92,"state":"up:active","state_seq":3,"addr":"172.21.15.43:6835/349688227","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6833","nonce":349688227},{"type":"v1","addr":"172.21.15.43:6835","nonce":349688227}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[25],"metadata_pool":24,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":11}]} 2023-08-09T15:11:12.591 INFO:teuthology.orchestra.run.smithi043.stderr:dumped fsmap epoch 96 2023-08-09T15:11:12.593 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:12.591+0000 7fc2c3fff640 1 -- 172.21.15.43:0/561652415 >> [v2:172.21.15.203:6832/4034093268,v1:172.21.15.203:6833/4034093268] conn(0x7fc2a406a0e0 msgr2=0x7fc2a406c5a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T15:11:12.593 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:12.591+0000 7fc2c3fff640 1 --2- 172.21.15.43:0/561652415 >> [v2:172.21.15.203:6832/4034093268,v1:172.21.15.203:6833/4034093268] conn(0x7fc2a406a0e0 0x7fc2a406c5a0 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7fc2c40d1f80 tx=0x7fc2ac008040 comp rx=0 tx=0).stop 2023-08-09T15:11:12.593 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:12.591+0000 7fc2c3fff640 1 -- 172.21.15.43:0/561652415 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc2c40b1050 msgr2=0x7fc2c40b1e90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T15:11:12.593 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:12.591+0000 7fc2c3fff640 1 --2- 172.21.15.43:0/561652415 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc2c40b1050 0x7fc2c40b1e90 secure :-1 s=READY pgs=2319 cs=0 l=1 rev1=1 crypto rx=0x7fc2b4036040 tx=0x7fc2b4004a40 comp rx=0 tx=0).stop 2023-08-09T15:11:12.593 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:12.592+0000 7fc2c3fff640 1 -- 172.21.15.43:0/561652415 shutdown_connections 2023-08-09T15:11:12.594 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:12.592+0000 7fc2c3fff640 1 --2- 172.21.15.43:0/561652415 >> [v2:172.21.15.203:6832/4034093268,v1:172.21.15.203:6833/4034093268] conn(0x7fc2a406a0e0 0x7fc2a406c5a0 unknown :-1 s=CLOSED pgs=28 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T15:11:12.594 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:12.592+0000 7fc2c3fff640 1 --2- 172.21.15.43:0/561652415 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fc2c415f7b0 0x7fc2c41446b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T15:11:12.594 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:12.592+0000 7fc2c3fff640 1 --2- 172.21.15.43:0/561652415 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fc2c40cd4e0 0x7fc2c4142690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T15:11:12.594 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:12.592+0000 7fc2c3fff640 1 --2- 172.21.15.43:0/561652415 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc2c40b1050 0x7fc2c40b1e90 unknown :-1 s=CLOSED pgs=2319 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T15:11:12.595 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:12.592+0000 7fc2c3fff640 1 -- 172.21.15.43:0/561652415 >> 172.21.15.43:0/561652415 conn(0x7fc2c406a2b0 msgr2=0x7fc2c415e5a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-09T15:11:12.595 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:12.592+0000 7fc2c3fff640 1 -- 172.21.15.43:0/561652415 shutdown_connections 2023-08-09T15:11:12.595 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:12.592+0000 7fc2c3fff640 1 -- 172.21.15.43:0/561652415 wait complete. 2023-08-09T15:11:12.613 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-08-09T15:11:12.840 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:12.839+0000 7fe6d5015640 1 Processor -- start 2023-08-09T15:11:12.840 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:12.839+0000 7fe6d5015640 1 -- start start 2023-08-09T15:11:12.842 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:12.841+0000 7fe6d5015640 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7fe6d00acdf0 0x7fe6d00ad340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T15:11:12.842 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:12.841+0000 7fe6d5015640 1 -- --> v1:172.21.15.43:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe6d00cbd90 con 0x7fe6d00c8b30 2023-08-09T15:11:12.842 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:12.841+0000 7fe6d5015640 1 -- --> v1:172.21.15.203:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe6d00ca2b0 con 0x7fe6d00c8750 2023-08-09T15:11:12.842 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:12.841+0000 7fe6d5015640 1 -- --> v2:172.21.15.203:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe6d015e810 con 0x7fe6d00acdf0 2023-08-09T15:11:12.843 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:12.841+0000 7fe6cf7fe640 1 --1- >> v1:172.21.15.43:6789/0 conn(0x7fe6d00c8b30 0x7fe6d0070f00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.43:6789/0 says I am v1:172.21.15.43:45576/0 (socket says 172.21.15.43:45576) 2023-08-09T15:11:12.843 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:12.841+0000 7fe6cf7fe640 1 -- 172.21.15.43:0/4046992132 learned_addr learned my addr 172.21.15.43:0/4046992132 (peer_addr_for_me v1:172.21.15.43:0/0) 2023-08-09T15:11:12.844 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:12.841+0000 7fe6ceffd640 1 -- 172.21.15.43:0/4046992132 <== mon.0 v1:172.21.15.43:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4020765789 0 0) 0x7fe6d00cbd90 con 0x7fe6d00c8b30 2023-08-09T15:11:12.844 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:12.842+0000 7fe6d4814640 1 --2- 172.21.15.43:0/4046992132 >> v2:172.21.15.203:3300/0 conn(0x7fe6d00acdf0 0x7fe6d00ad340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T15:11:12.844 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:12.842+0000 7fe6ceffd640 1 -- 172.21.15.43:0/4046992132 --> v1:172.21.15.43:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fe6b8003520 con 0x7fe6d00c8b30 2023-08-09T15:11:12.844 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:12.842+0000 7fe6ceffd640 1 -- 172.21.15.43:0/4046992132 <== mon.0 v1:172.21.15.43:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 133567160 0 0) 0x7fe6b8003520 con 0x7fe6d00c8b30 2023-08-09T15:11:12.845 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:12.842+0000 7fe6ceffd640 1 -- 172.21.15.43:0/4046992132 >> v2:172.21.15.203:3300/0 conn(0x7fe6d00acdf0 msgr2=0x7fe6d00ad340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-09T15:11:12.845 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:12.842+0000 7fe6ceffd640 1 --2- 172.21.15.43:0/4046992132 >> v2:172.21.15.203:3300/0 conn(0x7fe6d00acdf0 0x7fe6d00ad340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T15:11:12.845 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:12.842+0000 7fe6ceffd640 1 -- 172.21.15.43:0/4046992132 >> v1:172.21.15.203:6789/0 conn(0x7fe6d00c8750 legacy=0x7fe6d006f410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T15:11:12.845 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:12.842+0000 7fe6ceffd640 1 -- 172.21.15.43:0/4046992132 --> v1:172.21.15.43:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe6d00c9a20 con 0x7fe6d00c8b30 2023-08-09T15:11:12.845 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:12.842+0000 7fe6ceffd640 1 -- 172.21.15.43:0/4046992132 <== mon.0 v1:172.21.15.43:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1285150740 0 0) 0x7fe6c4001670 con 0x7fe6d00c8b30 2023-08-09T15:11:12.845 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:12.842+0000 7fe6ceffd640 1 -- 172.21.15.43:0/4046992132 >> v1:172.21.15.43:6789/0 conn(0x7fe6d00c8b30 legacy=0x7fe6d0070f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T15:11:12.846 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:12.842+0000 7fe6ceffd640 1 --2- 172.21.15.43:0/4046992132 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fe6b8003970 0x7fe6b8003d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T15:11:12.846 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:12.842+0000 7fe6ceffd640 1 --2- 172.21.15.43:0/4046992132 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fe6b80047c0 0x7fe6b8004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T15:11:12.846 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:12.842+0000 7fe6ceffd640 1 --2- 172.21.15.43:0/4046992132 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fe6b8005180 0x7fe6b800b980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T15:11:12.846 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:12.842+0000 7fe6ceffd640 1 -- 172.21.15.43:0/4046992132 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe6d015e810 con 0x7fe6b8003970 2023-08-09T15:11:12.846 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:12.842+0000 7fe6ceffd640 1 -- 172.21.15.43:0/4046992132 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe6b800c120 con 0x7fe6b8005180 2023-08-09T15:11:12.846 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:12.842+0000 7fe6ceffd640 1 -- 172.21.15.43:0/4046992132 --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe6b800c2e0 con 0x7fe6b80047c0 2023-08-09T15:11:12.846 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:12.843+0000 7fe6cffff640 1 --2- 172.21.15.43:0/4046992132 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fe6b8003970 0x7fe6b8003d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T15:11:12.846 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:12.843+0000 7fe6cf7fe640 1 --2- 172.21.15.43:0/4046992132 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fe6b80047c0 0x7fe6b8004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T15:11:12.847 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:12.843+0000 7fe6d4814640 1 --2- 172.21.15.43:0/4046992132 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fe6b8005180 0x7fe6b800b980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T15:11:12.847 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:12.843+0000 7fe6cffff640 1 -- 172.21.15.43:0/4046992132 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fe6b80047c0 msgr2=0x7fe6b8004c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-09T15:11:12.848 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:12.843+0000 7fe6cffff640 1 --2- 172.21.15.43:0/4046992132 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fe6b80047c0 0x7fe6b8004c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T15:11:12.848 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:12.843+0000 7fe6cffff640 1 -- 172.21.15.43:0/4046992132 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fe6b8005180 msgr2=0x7fe6b800b980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-09T15:11:12.848 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:12.843+0000 7fe6cffff640 1 --2- 172.21.15.43:0/4046992132 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fe6b8005180 0x7fe6b800b980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T15:11:12.848 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:12.843+0000 7fe6d4814640 1 -- 172.21.15.43:0/4046992132 reap_dead start 2023-08-09T15:11:12.848 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:12.843+0000 7fe6cffff640 1 -- 172.21.15.43:0/4046992132 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe6b8003520 con 0x7fe6b8003970 2023-08-09T15:11:12.848 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:12.845+0000 7fe6cffff640 1 --2- 172.21.15.43:0/4046992132 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fe6b8003970 0x7fe6b8003d90 secure :-1 s=READY pgs=2321 cs=0 l=1 rev1=1 crypto rx=0x7fe6c0001750 tx=0x7fe6b80057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=12729757f8921c03 server_cookie=0 in_seq=0 out_seq=0 2023-08-09T15:11:12.848 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:12.846+0000 7fe6ceffd640 1 -- 172.21.15.43:0/4046992132 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe6c00052d0 con 0x7fe6b8003970 2023-08-09T15:11:12.849 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:12.846+0000 7fe6ceffd640 1 -- 172.21.15.43:0/4046992132 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe6c002e050 con 0x7fe6b8003970 2023-08-09T15:11:12.849 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:12.846+0000 7fe6ceffd640 1 -- 172.21.15.43:0/4046992132 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe6c0003830 con 0x7fe6b8003970 2023-08-09T15:11:12.849 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:12.846+0000 7fe6d5015640 1 -- 172.21.15.43:0/4046992132 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fe6b8003970 msgr2=0x7fe6b8003d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T15:11:12.849 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:12.846+0000 7fe6d5015640 1 --2- 172.21.15.43:0/4046992132 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fe6b8003970 0x7fe6b8003d90 secure :-1 s=READY pgs=2321 cs=0 l=1 rev1=1 crypto rx=0x7fe6c0001750 tx=0x7fe6b80057f0 comp rx=0 tx=0).stop 2023-08-09T15:11:12.850 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:12.846+0000 7fe6d5015640 1 -- 172.21.15.43:0/4046992132 shutdown_connections 2023-08-09T15:11:12.850 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:12.846+0000 7fe6d5015640 1 --2- 172.21.15.43:0/4046992132 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fe6b8003970 0x7fe6b8003d90 unknown :-1 s=CLOSED pgs=2321 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T15:11:12.850 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:12.846+0000 7fe6d5015640 1 -- 172.21.15.43:0/4046992132 >> 172.21.15.43:0/4046992132 conn(0x7fe6d00cde90 msgr2=0x7fe6d00ce2a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-09T15:11:12.850 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:12.847+0000 7fe6d5015640 1 -- 172.21.15.43:0/4046992132 shutdown_connections 2023-08-09T15:11:12.850 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:12.847+0000 7fe6d5015640 1 -- 172.21.15.43:0/4046992132 wait complete. 2023-08-09T15:11:12.850 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:12.848+0000 7fe6d5015640 1 Processor -- start 2023-08-09T15:11:12.851 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:12.849+0000 7fe6d5015640 1 -- start start 2023-08-09T15:11:12.852 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:12.850+0000 7fe6d5015640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fe6b8003970 0x7fe6d01588b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T15:11:12.852 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:12.850+0000 7fe6d5015640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fe6d0158f80 0x7fe6d00aeba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T15:11:12.852 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:12.850+0000 7fe6d5015640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fe6d00af0e0 0x7fe6d01e4b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T15:11:12.852 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:12.850+0000 7fe6d5015640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe6d00c9f70 con 0x7fe6d0158f80 2023-08-09T15:11:12.852 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:12.850+0000 7fe6d5015640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe6d01e4720 con 0x7fe6d00af0e0 2023-08-09T15:11:12.853 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:12.850+0000 7fe6d5015640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe6d0158df0 con 0x7fe6b8003970 2023-08-09T15:11:12.853 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:12.850+0000 7fe6cf7fe640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fe6d0158f80 0x7fe6d00aeba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T15:11:12.853 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:12.850+0000 7fe6cf7fe640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fe6d0158f80 0x7fe6d00aeba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:46634/0 (socket says 172.21.15.43:46634) 2023-08-09T15:11:12.853 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:12.850+0000 7fe6cf7fe640 1 -- 172.21.15.43:0/3192330245 learned_addr learned my addr 172.21.15.43:0/3192330245 (peer_addr_for_me v2:172.21.15.43:0/0) 2023-08-09T15:11:12.853 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:12.850+0000 7fe6cffff640 1 --2- 172.21.15.43:0/3192330245 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fe6b8003970 0x7fe6d01588b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T15:11:12.853 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:12.850+0000 7fe6d4814640 1 --2- 172.21.15.43:0/3192330245 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fe6d00af0e0 0x7fe6d01e4b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T15:11:12.853 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:12.850+0000 7fe6cf7fe640 1 -- 172.21.15.43:0/3192330245 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fe6b8003970 msgr2=0x7fe6d01588b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T15:11:12.853 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:12.850+0000 7fe6cf7fe640 1 --2- 172.21.15.43:0/3192330245 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fe6b8003970 0x7fe6d01588b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T15:11:12.854 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:12.850+0000 7fe6cf7fe640 1 -- 172.21.15.43:0/3192330245 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fe6d00af0e0 msgr2=0x7fe6d01e4b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T15:11:12.854 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:12.850+0000 7fe6cf7fe640 1 --2- 172.21.15.43:0/3192330245 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fe6d00af0e0 0x7fe6d01e4b40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T15:11:12.854 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:12.850+0000 7fe6cf7fe640 1 -- 172.21.15.43:0/3192330245 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe6d01e3b30 con 0x7fe6d0158f80 2023-08-09T15:11:12.854 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:12.851+0000 7fe6cf7fe640 1 --2- 172.21.15.43:0/3192330245 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fe6d0158f80 0x7fe6d00aeba0 secure :-1 s=READY pgs=2322 cs=0 l=1 rev1=1 crypto rx=0x7fe6c4007290 tx=0x7fe6c4007750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-09T15:11:12.855 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:12.851+0000 7fe6ccff9640 1 -- 172.21.15.43:0/3192330245 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe6c4001650 con 0x7fe6d0158f80 2023-08-09T15:11:12.855 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:12.851+0000 7fe6ccff9640 1 -- 172.21.15.43:0/3192330245 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe6c400f9c0 con 0x7fe6d0158f80 2023-08-09T15:11:12.855 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:12.851+0000 7fe6d5015640 1 -- 172.21.15.43:0/3192330245 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe6d01e3d80 con 0x7fe6d0158f80 2023-08-09T15:11:12.855 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:12.851+0000 7fe6ccff9640 1 -- 172.21.15.43:0/3192330245 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe6c400fd50 con 0x7fe6d0158f80 2023-08-09T15:11:12.855 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:12.851+0000 7fe6d5015640 1 -- 172.21.15.43:0/3192330245 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe6d01e4340 con 0x7fe6d0158f80 2023-08-09T15:11:12.855 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:12.852+0000 7fe6d5015640 1 -- 172.21.15.43:0/3192330245 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe6d01e4860 con 0x7fe6d0158f80 2023-08-09T15:11:12.856 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:12.853+0000 7fe6ccff9640 1 -- 172.21.15.43:0/3192330245 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 117) v1 ==== 88387+0+0 (secure 0 0 0) 0x7fe6c401d460 con 0x7fe6d0158f80 2023-08-09T15:11:12.856 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:12.855+0000 7fe6ccff9640 1 --2- 172.21.15.43:0/3192330245 >> [v2:172.21.15.203:6832/4034093268,v1:172.21.15.203:6833/4034093268] conn(0x7fe6bc06a0e0 0x7fe6bc06c5a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T15:11:12.856 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:12.855+0000 7fe6ccff9640 1 -- 172.21.15.43:0/3192330245 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(211..211 src has 1..211) v4 ==== 7200+0+0 (secure 0 0 0) 0x7fe6c4088c80 con 0x7fe6d0158f80 2023-08-09T15:11:12.857 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:12.855+0000 7fe6cffff640 1 --2- 172.21.15.43:0/3192330245 >> [v2:172.21.15.203:6832/4034093268,v1:172.21.15.203:6833/4034093268] conn(0x7fe6bc06a0e0 0x7fe6bc06c5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T15:11:12.858 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:12.857+0000 7fe6cffff640 1 --2- 172.21.15.43:0/3192330245 >> [v2:172.21.15.203:6832/4034093268,v1:172.21.15.203:6833/4034093268] conn(0x7fe6bc06a0e0 0x7fe6bc06c5a0 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7fe6c0033000 tx=0x7fe6c0009030 comp rx=0 tx=0).ready entity=mgr.8833 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-09T15:11:12.860 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:12.859+0000 7fe6ccff9640 1 -- 172.21.15.43:0/3192330245 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178066 (secure 0 0 0) 0x7fe6d01e4860 con 0x7fe6d0158f80 2023-08-09T15:11:13.053 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:13.052+0000 7fe6d5015640 1 -- 172.21.15.43:0/3192330245 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fe6d01e4860 con 0x7fe6d0158f80 2023-08-09T15:11:13.056 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:13.054+0000 7fe6ccff9640 1 -- 172.21.15.43:0/3192330245 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v211) v1 ==== 74+0+19745 (secure 0 0 0) 0x7fe6d01e4860 con 0x7fe6d0158f80 2023-08-09T15:11:13.056 INFO:teuthology.orchestra.run.smithi043.stdout: 2023-08-09T15:11:13.056 INFO:teuthology.orchestra.run.smithi043.stdout:{"epoch":211,"fsid":"daac87ee-1337-4250-a955-aea4e408a39b","created":"2023-08-09T13:33:41.701198+0000","modified":"2023-08-09T15:11:08.350101+0000","last_up_change":"2023-08-09T13:33:53.783453+0000","last_in_change":"2023-08-09T13:33:45.148904+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":25,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":"rbd","create_time":"2023-08-09T13:33:54.365137+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":".mgr","create_time":"2023-08-09T13:33:54.831803+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":24,"pool_name":"cephfs_metadata","create_time":"2023-08-09T15:08:43.920337+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"204","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":25,"pool_name":"cephfs_data","create_time":"2023-08-09T15:08:44.594489+0000","flags":8193,"flags_names":"hashpspool,selfmanaged_snaps","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"210","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":5,"snap_epoch":210,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"cbd49ecd-e896-4fbd-864e-5c37a829b165","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":203,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6800","nonce":502667335},{"type":"v1","addr":"172.21.15.43:6801","nonce":502667335}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6802","nonce":502667335},{"type":"v1","addr":"172.21.15.43:6803","nonce":502667335}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6806","nonce":502667335},{"type":"v1","addr":"172.21.15.43:6807","nonce":502667335}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6804","nonce":502667335},{"type":"v1","addr":"172.21.15.43:6805","nonce":502667335}]},"public_addr":"172.21.15.43:6801/502667335","cluster_addr":"172.21.15.43:6803/502667335","heartbeat_back_addr":"172.21.15.43:6807/502667335","heartbeat_front_addr":"172.21.15.43:6805/502667335","state":["exists","up"]},{"osd":1,"uuid":"75c7e0fe-1ec7-4fe8-a2f2-3661a65b8500","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":203,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6808","nonce":3289134610},{"type":"v1","addr":"172.21.15.43:6809","nonce":3289134610}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6810","nonce":3289134610},{"type":"v1","addr":"172.21.15.43:6812","nonce":3289134610}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6821","nonce":3289134610},{"type":"v1","addr":"172.21.15.43:6824","nonce":3289134610}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6815","nonce":3289134610},{"type":"v1","addr":"172.21.15.43:6818","nonce":3289134610}]},"public_addr":"172.21.15.43:6809/3289134610","cluster_addr":"172.21.15.43:6812/3289134610","heartbeat_back_addr":"172.21.15.43:6824/3289134610","heartbeat_front_addr":"172.21.15.43:6818/3289134610","state":["exists","up"]},{"osd":2,"uuid":"bb5c41e9-7f5f-4125-9b01-3bb8bfc2ea83","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":203,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6811","nonce":4079709736},{"type":"v1","addr":"172.21.15.43:6814","nonce":4079709736}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6817","nonce":4079709736},{"type":"v1","addr":"172.21.15.43:6820","nonce":4079709736}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6829","nonce":4079709736},{"type":"v1","addr":"172.21.15.43:6831","nonce":4079709736}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6823","nonce":4079709736},{"type":"v1","addr":"172.21.15.43:6826","nonce":4079709736}]},"public_addr":"172.21.15.43:6814/4079709736","cluster_addr":"172.21.15.43:6820/4079709736","heartbeat_back_addr":"172.21.15.43:6831/4079709736","heartbeat_front_addr":"172.21.15.43:6826/4079709736","state":["exists","up"]},{"osd":3,"uuid":"bfe4c8b4-694a-465c-87e6-2b2aab852613","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":203,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6813","nonce":1709538759},{"type":"v1","addr":"172.21.15.43:6816","nonce":1709538759}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6819","nonce":1709538759},{"type":"v1","addr":"172.21.15.43:6822","nonce":1709538759}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6828","nonce":1709538759},{"type":"v1","addr":"172.21.15.43:6830","nonce":1709538759}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6825","nonce":1709538759},{"type":"v1","addr":"172.21.15.43:6827","nonce":1709538759}]},"public_addr":"172.21.15.43:6816/1709538759","cluster_addr":"172.21.15.43:6822/1709538759","heartbeat_back_addr":"172.21.15.43:6830/1709538759","heartbeat_front_addr":"172.21.15.43:6827/1709538759","state":["exists","up"]},{"osd":4,"uuid":"fa2e8a77-aba1-4756-853e-0d174a5bdeba","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":203,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6800","nonce":4196866481},{"type":"v1","addr":"172.21.15.203:6801","nonce":4196866481}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6802","nonce":4196866481},{"type":"v1","addr":"172.21.15.203:6803","nonce":4196866481}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6806","nonce":4196866481},{"type":"v1","addr":"172.21.15.203:6807","nonce":4196866481}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6804","nonce":4196866481},{"type":"v1","addr":"172.21.15.203:6805","nonce":4196866481}]},"public_addr":"172.21.15.203:6801/4196866481","cluster_addr":"172.21.15.203:6803/4196866481","heartbeat_back_addr":"172.21.15.203:6807/4196866481","heartbeat_front_addr":"172.21.15.203:6805/4196866481","state":["exists","up"]},{"osd":5,"uuid":"81811e4b-4d24-4f51-a82c-f1d700a2a00d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":207,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6816","nonce":3059290081},{"type":"v1","addr":"172.21.15.203:6817","nonce":3059290081}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6818","nonce":3059290081},{"type":"v1","addr":"172.21.15.203:6819","nonce":3059290081}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6822","nonce":3059290081},{"type":"v1","addr":"172.21.15.203:6823","nonce":3059290081}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6820","nonce":3059290081},{"type":"v1","addr":"172.21.15.203:6821","nonce":3059290081}]},"public_addr":"172.21.15.203:6817/3059290081","cluster_addr":"172.21.15.203:6819/3059290081","heartbeat_back_addr":"172.21.15.203:6823/3059290081","heartbeat_front_addr":"172.21.15.203:6821/3059290081","state":["exists","up"]},{"osd":6,"uuid":"307bac4d-6a5b-443d-af75-56d695d7674f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":203,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6824","nonce":512155569},{"type":"v1","addr":"172.21.15.203:6825","nonce":512155569}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6826","nonce":512155569},{"type":"v1","addr":"172.21.15.203:6827","nonce":512155569}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6830","nonce":512155569},{"type":"v1","addr":"172.21.15.203:6831","nonce":512155569}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6828","nonce":512155569},{"type":"v1","addr":"172.21.15.203:6829","nonce":512155569}]},"public_addr":"172.21.15.203:6825/512155569","cluster_addr":"172.21.15.203:6827/512155569","heartbeat_back_addr":"172.21.15.203:6831/512155569","heartbeat_front_addr":"172.21.15.203:6829/512155569","state":["exists","up"]},{"osd":7,"uuid":"a95dc7ca-9f1b-4295-af70-68a22a2d7d5e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":207,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6808","nonce":3331450438},{"type":"v1","addr":"172.21.15.203:6809","nonce":3331450438}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6810","nonce":3331450438},{"type":"v1","addr":"172.21.15.203:6811","nonce":3331450438}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6814","nonce":3331450438},{"type":"v1","addr":"172.21.15.203:6815","nonce":3331450438}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6812","nonce":3331450438},{"type":"v1","addr":"172.21.15.203:6813","nonce":3331450438}]},"public_addr":"172.21.15.203:6809/3331450438","cluster_addr":"172.21.15.203:6811/3331450438","heartbeat_back_addr":"172.21.15.203:6815/3331450438","heartbeat_front_addr":"172.21.15.203:6813/3331450438","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-09T13:33:50.547652+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-09T13:33:50.589469+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-09T13:33:50.561724+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-09T13:33:50.599650+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-09T13:33:50.411855+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-09T13:33:50.219681+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-09T13:33:50.504664+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-09T13:33:50.441248+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"25.f","mappings":[{"from":0,"to":1}]},{"pgid":"25.3f","mappings":[{"from":0,"to":2}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.203:0/822076630":"2023-08-10T15:11:08.350045+0000","172.21.15.203:0/79247817":"2023-08-10T15:11:08.350045+0000","172.21.15.203:6833/4012896436":"2023-08-10T15:11:08.350045+0000","172.21.15.203:6832/4012896436":"2023-08-10T15:11:08.350045+0000","172.21.15.203:0/3345703784":"2023-08-10T15:11:08.350045+0000","172.21.15.203:0/1987920954":"2023-08-10T15:11:08.350045+0000","172.21.15.203:0/4253505338":"2023-08-10T15:11:08.350045+0000","172.21.15.203:6833/3082732769":"2023-08-10T15:09:01.285901+0000","172.21.15.203:6832/3082732769":"2023-08-10T15:09:01.285901+0000","172.21.15.203:0/1756781921":"2023-08-10T15:09:01.285901+0000","172.21.15.203:0/454071895":"2023-08-10T15:09:01.285901+0000","172.21.15.203:0/2854569557":"2023-08-10T15:09:01.285901+0000","172.21.15.203:0/2821325802":"2023-08-10T15:11:08.350045+0000","172.21.15.203:0/1159944241":"2023-08-10T15:09:01.285901+0000","172.21.15.203:0/1645632393":"2023-08-10T15:09:01.285901+0000","172.21.15.203:0/840449585":"2023-08-10T15:09:01.285901+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":7,"snaps":[{"begin":104,"length":1},{"begin":221,"length":1}]},{"pool":25,"snaps":[{"begin":2,"length":4}]}],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-08-09T15:11:13.059 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:13.057+0000 7fe6d5015640 1 -- 172.21.15.43:0/3192330245 >> [v2:172.21.15.203:6832/4034093268,v1:172.21.15.203:6833/4034093268] conn(0x7fe6bc06a0e0 msgr2=0x7fe6bc06c5a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T15:11:13.059 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:13.057+0000 7fe6d5015640 1 --2- 172.21.15.43:0/3192330245 >> [v2:172.21.15.203:6832/4034093268,v1:172.21.15.203:6833/4034093268] conn(0x7fe6bc06a0e0 0x7fe6bc06c5a0 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7fe6c0033000 tx=0x7fe6c0009030 comp rx=0 tx=0).stop 2023-08-09T15:11:13.059 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:13.057+0000 7fe6d5015640 1 -- 172.21.15.43:0/3192330245 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fe6d0158f80 msgr2=0x7fe6d00aeba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T15:11:13.059 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:13.057+0000 7fe6d5015640 1 --2- 172.21.15.43:0/3192330245 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fe6d0158f80 0x7fe6d00aeba0 secure :-1 s=READY pgs=2322 cs=0 l=1 rev1=1 crypto rx=0x7fe6c4007290 tx=0x7fe6c4007750 comp rx=0 tx=0).stop 2023-08-09T15:11:13.059 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:13.057+0000 7fe6d5015640 1 -- 172.21.15.43:0/3192330245 shutdown_connections 2023-08-09T15:11:13.059 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:13.057+0000 7fe6d5015640 1 --2- 172.21.15.43:0/3192330245 >> [v2:172.21.15.203:6832/4034093268,v1:172.21.15.203:6833/4034093268] conn(0x7fe6bc06a0e0 0x7fe6bc06c5a0 unknown :-1 s=CLOSED pgs=29 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T15:11:13.059 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:13.057+0000 7fe6d5015640 1 --2- 172.21.15.43:0/3192330245 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fe6d00af0e0 0x7fe6d01e4b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T15:11:13.060 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:13.057+0000 7fe6d5015640 1 --2- 172.21.15.43:0/3192330245 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fe6d0158f80 0x7fe6d00aeba0 unknown :-1 s=CLOSED pgs=2322 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T15:11:13.060 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:13.057+0000 7fe6d5015640 1 --2- 172.21.15.43:0/3192330245 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fe6b8003970 0x7fe6d01588b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T15:11:13.060 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:13.057+0000 7fe6d5015640 1 -- 172.21.15.43:0/3192330245 >> 172.21.15.43:0/3192330245 conn(0x7fe6d00cde90 msgr2=0x7fe6d00ac250 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-09T15:11:13.060 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:13.058+0000 7fe6d5015640 1 -- 172.21.15.43:0/3192330245 shutdown_connections 2023-08-09T15:11:13.061 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:13.058+0000 7fe6d5015640 1 -- 172.21.15.43:0/3192330245 wait complete. 2023-08-09T15:11:13.079 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2023-08-09T15:11:13.314 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:13.313+0000 7fe3e2902640 1 Processor -- start 2023-08-09T15:11:13.314 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:13.313+0000 7fe3e2902640 1 -- start start 2023-08-09T15:11:13.316 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:13.315+0000 7fe3e2902640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fe3dc0b1050 0x7fe3dc0b4760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T15:11:13.316 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:13.315+0000 7fe3e2902640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7fe3dc0cd6b0 0x7fe3dc0d0e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T15:11:13.316 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:13.315+0000 7fe3e2902640 1 -- --> v1:172.21.15.203:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe3dc0b7bf0 con 0x7fe3dc15f7b0 2023-08-09T15:11:13.316 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:13.315+0000 7fe3e2902640 1 -- --> v2:172.21.15.43:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe3dc0b7ed0 con 0x7fe3dc0cd6b0 2023-08-09T15:11:13.316 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:13.315+0000 7fe3e2902640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe3dc0d0370 con 0x7fe3dc0b1050 2023-08-09T15:11:13.317 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:13.315+0000 7fe3e2101640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7fe3dc0cd6b0 0x7fe3dc0d0e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T15:11:13.317 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:13.315+0000 7fe3e2101640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7fe3dc0cd6b0 0x7fe3dc0d0e50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:46648/0 (socket says 172.21.15.43:46648) 2023-08-09T15:11:13.317 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:13.315+0000 7fe3e2101640 1 -- 172.21.15.43:0/1805907253 learned_addr learned my addr 172.21.15.43:0/1805907253 (peer_addr_for_me v2:172.21.15.43:0/0) 2023-08-09T15:11:13.318 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:13.315+0000 7fe3e10ff640 1 --2- 172.21.15.43:0/1805907253 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fe3dc0b1050 0x7fe3dc0b4760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T15:11:13.318 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:13.315+0000 7fe3e2101640 1 -- 172.21.15.43:0/1805907253 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fe3dc0b1050 msgr2=0x7fe3dc0b4760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-09T15:11:13.318 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:13.315+0000 7fe3e2101640 1 --2- 172.21.15.43:0/1805907253 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fe3dc0b1050 0x7fe3dc0b4760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T15:11:13.318 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:13.316+0000 7fe3e2101640 1 -- 172.21.15.43:0/1805907253 >> v1:172.21.15.203:6789/0 conn(0x7fe3dc15f7b0 legacy=0x7fe3dc0ccfa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T15:11:13.318 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:13.316+0000 7fe3e2101640 1 -- 172.21.15.43:0/1805907253 --> v2:172.21.15.43:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe3dc0d0650 con 0x7fe3dc0cd6b0 2023-08-09T15:11:13.319 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:13.317+0000 7fe3e2101640 1 --2- 172.21.15.43:0/1805907253 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fe3dc0cd6b0 0x7fe3dc0d0e50 secure :-1 s=READY pgs=2323 cs=0 l=1 rev1=1 crypto rx=0x7fe3d4009870 tx=0x7fe3d402b980 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fa5aafdf463a080b server_cookie=0 in_seq=0 out_seq=0 2023-08-09T15:11:13.319 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:13.317+0000 7fe3e08fe640 1 -- 172.21.15.43:0/1805907253 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe3d4030840 con 0x7fe3dc0cd6b0 2023-08-09T15:11:13.319 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:13.317+0000 7fe3e08fe640 1 -- 172.21.15.43:0/1805907253 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe3d40309a0 con 0x7fe3dc0cd6b0 2023-08-09T15:11:13.319 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:13.317+0000 7fe3e08fe640 1 -- 172.21.15.43:0/1805907253 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe3d4030d30 con 0x7fe3dc0cd6b0 2023-08-09T15:11:13.319 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:13.319+0000 7fe3e2902640 1 -- 172.21.15.43:0/1805907253 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fe3dc0cd6b0 msgr2=0x7fe3dc0d0e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T15:11:13.319 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:13.319+0000 7fe3e2902640 1 --2- 172.21.15.43:0/1805907253 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fe3dc0cd6b0 0x7fe3dc0d0e50 secure :-1 s=READY pgs=2323 cs=0 l=1 rev1=1 crypto rx=0x7fe3d4009870 tx=0x7fe3d402b980 comp rx=0 tx=0).stop 2023-08-09T15:11:13.320 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:13.319+0000 7fe3e2902640 1 -- 172.21.15.43:0/1805907253 shutdown_connections 2023-08-09T15:11:13.320 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:13.319+0000 7fe3e2902640 1 --2- 172.21.15.43:0/1805907253 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fe3dc0cd6b0 0x7fe3dc0d0e50 unknown :-1 s=CLOSED pgs=2323 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T15:11:13.320 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:13.319+0000 7fe3e2902640 1 --2- 172.21.15.43:0/1805907253 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fe3dc0b1050 0x7fe3dc0b4760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T15:11:13.320 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:13.319+0000 7fe3e2902640 1 -- 172.21.15.43:0/1805907253 >> 172.21.15.43:0/1805907253 conn(0x7fe3dc06a2b0 msgr2=0x7fe3dc06a690 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-09T15:11:13.320 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:13.319+0000 7fe3e2902640 1 -- 172.21.15.43:0/1805907253 shutdown_connections 2023-08-09T15:11:13.321 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:13.319+0000 7fe3e2902640 1 -- 172.21.15.43:0/1805907253 wait complete. 2023-08-09T15:11:13.321 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:13.320+0000 7fe3e2902640 1 Processor -- start 2023-08-09T15:11:13.321 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:13.320+0000 7fe3e2902640 1 -- start start 2023-08-09T15:11:13.322 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:13.321+0000 7fe3e2902640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fe3dc0b1050 0x7fe3dc14c710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T15:11:13.322 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:13.321+0000 7fe3e2902640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fe3dc0cd6b0 0x7fe3dc0d1760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T15:11:13.322 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:13.321+0000 7fe3e2902640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fe3dc15f7b0 0x7fe3dc155590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T15:11:13.323 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:13.321+0000 7fe3e2902640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe3dc0c86b0 con 0x7fe3dc0cd6b0 2023-08-09T15:11:13.323 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:13.321+0000 7fe3e2902640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe3dc0c7aa0 con 0x7fe3dc15f7b0 2023-08-09T15:11:13.323 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:13.321+0000 7fe3e2902640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe3dc0c7d80 con 0x7fe3dc0b1050 2023-08-09T15:11:13.324 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:13.321+0000 7fe3e10ff640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fe3dc0cd6b0 0x7fe3dc0d1760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T15:11:13.324 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:13.321+0000 7fe3e10ff640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fe3dc0cd6b0 0x7fe3dc0d1760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:46652/0 (socket says 172.21.15.43:46652) 2023-08-09T15:11:13.324 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:13.321+0000 7fe3e10ff640 1 -- 172.21.15.43:0/2531749818 learned_addr learned my addr 172.21.15.43:0/2531749818 (peer_addr_for_me v2:172.21.15.43:0/0) 2023-08-09T15:11:13.325 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:13.322+0000 7fe3e10ff640 1 -- 172.21.15.43:0/2531749818 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fe3dc0b1050 msgr2=0x7fe3dc14c710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T15:11:13.325 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:13.322+0000 7fe3e2101640 1 --2- 172.21.15.43:0/2531749818 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fe3dc15f7b0 0x7fe3dc155590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T15:11:13.325 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:13.322+0000 7fe3e1900640 1 --2- 172.21.15.43:0/2531749818 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fe3dc0b1050 0x7fe3dc14c710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T15:11:13.325 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:13.322+0000 7fe3e10ff640 1 --2- 172.21.15.43:0/2531749818 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fe3dc0b1050 0x7fe3dc14c710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T15:11:13.325 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:13.322+0000 7fe3e10ff640 1 -- 172.21.15.43:0/2531749818 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fe3dc15f7b0 msgr2=0x7fe3dc155590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T15:11:13.325 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:13.322+0000 7fe3e10ff640 1 --2- 172.21.15.43:0/2531749818 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fe3dc15f7b0 0x7fe3dc155590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T15:11:13.325 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:13.322+0000 7fe3e10ff640 1 -- 172.21.15.43:0/2531749818 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe3d40094d0 con 0x7fe3dc0cd6b0 2023-08-09T15:11:13.325 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:13.322+0000 7fe3e10ff640 1 --2- 172.21.15.43:0/2531749818 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fe3dc0cd6b0 0x7fe3dc0d1760 secure :-1 s=READY pgs=2324 cs=0 l=1 rev1=1 crypto rx=0x7fe3cc00d7d0 tx=0x7fe3cc00dc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-09T15:11:13.326 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:13.322+0000 7fe3d27fc640 1 -- 172.21.15.43:0/2531749818 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe3cc00b6c0 con 0x7fe3dc0cd6b0 2023-08-09T15:11:13.326 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:13.323+0000 7fe3d27fc640 1 -- 172.21.15.43:0/2531749818 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe3cc00b840 con 0x7fe3dc0cd6b0 2023-08-09T15:11:13.326 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:13.323+0000 7fe3d27fc640 1 -- 172.21.15.43:0/2531749818 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe3cc00bbf0 con 0x7fe3dc0cd6b0 2023-08-09T15:11:13.326 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:13.323+0000 7fe3e2902640 1 -- 172.21.15.43:0/2531749818 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe3dc0c8060 con 0x7fe3dc0cd6b0 2023-08-09T15:11:13.326 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:13.323+0000 7fe3e2902640 1 -- 172.21.15.43:0/2531749818 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe3dc14cc50 con 0x7fe3dc0cd6b0 2023-08-09T15:11:13.327 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:13.324+0000 7fe3e2902640 1 -- 172.21.15.43:0/2531749818 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe3dc0c8860 con 0x7fe3dc0cd6b0 2023-08-09T15:11:13.327 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:13.326+0000 7fe3d27fc640 1 -- 172.21.15.43:0/2531749818 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 118) v1 ==== 88443+0+0 (secure 0 0 0) 0x7fe3cc00bd50 con 0x7fe3dc0cd6b0 2023-08-09T15:11:13.327 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:13.327+0000 7fe3d27fc640 1 --2- 172.21.15.43:0/2531749818 >> [v2:172.21.15.203:6832/4034093268,v1:172.21.15.203:6833/4034093268] conn(0x7fe3c006a170 0x7fe3c006c630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T15:11:13.328 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:13.327+0000 7fe3d27fc640 1 -- 172.21.15.43:0/2531749818 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(211..211 src has 1..211) v4 ==== 7200+0+0 (secure 0 0 0) 0x7fe3cc089170 con 0x7fe3dc0cd6b0 2023-08-09T15:11:13.328 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:13.327+0000 7fe3e1900640 1 --2- 172.21.15.43:0/2531749818 >> [v2:172.21.15.203:6832/4034093268,v1:172.21.15.203:6833/4034093268] conn(0x7fe3c006a170 0x7fe3c006c630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T15:11:13.329 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:13.329+0000 7fe3e1900640 1 --2- 172.21.15.43:0/2531749818 >> [v2:172.21.15.203:6832/4034093268,v1:172.21.15.203:6833/4034093268] conn(0x7fe3c006a170 0x7fe3c006c630 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0x7fe3d8007fd0 tx=0x7fe3d8009040 comp rx=0 tx=0).ready entity=mgr.8833 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-09T15:11:13.332 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:13.331+0000 7fe3d27fc640 1 -- 172.21.15.43:0/2531749818 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178066 (secure 0 0 0) 0x7fe3cc057cf0 con 0x7fe3dc0cd6b0 2023-08-09T15:11:13.565 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:13.563+0000 7fe3e2902640 1 -- 172.21.15.43:0/2531749818 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) v1 -- 0x7fe3dc0739f0 con 0x7fe3dc0cd6b0 2023-08-09T15:11:13.565 INFO:tasks.ceph.mon.a.smithi043.stderr:2023-08-09T15:11:13.564+0000 7f32867f3640 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2023-08-09T15:11:13.575 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:13.572+0000 7fe3d27fc640 1 -- 172.21.15.43:0/2531749818 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v97) v1 ==== 161+0+0 (secure 0 0 0) 0x7fe3cc057880 con 0x7fe3dc0cd6b0 2023-08-09T15:11:13.575 INFO:teuthology.orchestra.run.smithi043.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2023-08-09T15:11:13.578 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:13.577+0000 7fe3e2902640 1 -- 172.21.15.43:0/2531749818 >> [v2:172.21.15.203:6832/4034093268,v1:172.21.15.203:6833/4034093268] conn(0x7fe3c006a170 msgr2=0x7fe3c006c630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T15:11:13.578 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:13.577+0000 7fe3e2902640 1 --2- 172.21.15.43:0/2531749818 >> [v2:172.21.15.203:6832/4034093268,v1:172.21.15.203:6833/4034093268] conn(0x7fe3c006a170 0x7fe3c006c630 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0x7fe3d8007fd0 tx=0x7fe3d8009040 comp rx=0 tx=0).stop 2023-08-09T15:11:13.578 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:13.577+0000 7fe3e2902640 1 -- 172.21.15.43:0/2531749818 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fe3dc0cd6b0 msgr2=0x7fe3dc0d1760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T15:11:13.579 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:13.577+0000 7fe3e2902640 1 --2- 172.21.15.43:0/2531749818 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fe3dc0cd6b0 0x7fe3dc0d1760 secure :-1 s=READY pgs=2324 cs=0 l=1 rev1=1 crypto rx=0x7fe3cc00d7d0 tx=0x7fe3cc00dc90 comp rx=0 tx=0).stop 2023-08-09T15:11:13.579 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:13.577+0000 7fe3e2902640 1 -- 172.21.15.43:0/2531749818 shutdown_connections 2023-08-09T15:11:13.579 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:13.577+0000 7fe3e2902640 1 --2- 172.21.15.43:0/2531749818 >> [v2:172.21.15.203:6832/4034093268,v1:172.21.15.203:6833/4034093268] conn(0x7fe3c006a170 0x7fe3c006c630 unknown :-1 s=CLOSED pgs=30 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T15:11:13.579 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:13.577+0000 7fe3e2902640 1 --2- 172.21.15.43:0/2531749818 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fe3dc15f7b0 0x7fe3dc155590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T15:11:13.579 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:13.577+0000 7fe3e2902640 1 --2- 172.21.15.43:0/2531749818 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fe3dc0cd6b0 0x7fe3dc0d1760 unknown :-1 s=CLOSED pgs=2324 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T15:11:13.579 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:13.577+0000 7fe3e2902640 1 --2- 172.21.15.43:0/2531749818 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fe3dc0b1050 0x7fe3dc14c710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T15:11:13.579 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:13.577+0000 7fe3e2902640 1 -- 172.21.15.43:0/2531749818 >> 172.21.15.43:0/2531749818 conn(0x7fe3dc06a2b0 msgr2=0x7fe3dc15ddb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-09T15:11:13.580 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:13.577+0000 7fe3e2902640 1 -- 172.21.15.43:0/2531749818 shutdown_connections 2023-08-09T15:11:13.580 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:13.577+0000 7fe3e2902640 1 -- 172.21.15.43:0/2531749818 wait complete. 2023-08-09T15:11:13.596 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs rm cephfs --yes-i-really-mean-it 2023-08-09T15:11:13.642 INFO:tasks.ceph.mds.a.smithi043.stdout:starting mds.a at 2023-08-09T15:11:13.791 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:13.791+0000 7fc5ed122640 1 Processor -- start 2023-08-09T15:11:13.792 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:13.791+0000 7fc5ed122640 1 -- start start 2023-08-09T15:11:13.793 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:13.791+0000 7fc5ed122640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fc5e80cb2d0 0x7fc5e815b380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T15:11:13.793 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:13.791+0000 7fc5ed122640 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7fc5e815df60 0x7fc5e80ce1c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T15:11:13.793 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:13.791+0000 7fc5ed122640 1 -- --> v1:172.21.15.43:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc5e815aef0 con 0x7fc5e815e3c0 2023-08-09T15:11:13.793 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:13.791+0000 7fc5ed122640 1 -- --> v2:172.21.15.203:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc5e815a970 con 0x7fc5e815df60 2023-08-09T15:11:13.793 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:13.791+0000 7fc5ed122640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc5e815b8c0 con 0x7fc5e80cb2d0 2023-08-09T15:11:13.794 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:13.792+0000 7fc5e759e640 1 --1- >> v1:172.21.15.43:6789/0 conn(0x7fc5e815e3c0 0x7fc5e80c9f90 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.43:6789/0 says I am v1:172.21.15.43:45578/0 (socket says 172.21.15.43:45578) 2023-08-09T15:11:13.794 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:13.792+0000 7fc5e759e640 1 -- 172.21.15.43:0/1924695638 learned_addr learned my addr 172.21.15.43:0/1924695638 (peer_addr_for_me v1:172.21.15.43:0/0) 2023-08-09T15:11:13.795 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:13.792+0000 7fc5e6d9d640 1 --2- 172.21.15.43:0/1924695638 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fc5e80cb2d0 0x7fc5e815b380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T15:11:13.795 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:13.792+0000 7fc5e659c640 1 --2- 172.21.15.43:0/1924695638 >> v2:172.21.15.203:3300/0 conn(0x7fc5e815df60 0x7fc5e80ce1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T15:11:13.795 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:13.792+0000 7fc5e5d9b640 1 -- 172.21.15.43:0/1924695638 <== mon.0 v1:172.21.15.43:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 276784242 0 0) 0x7fc5e815aef0 con 0x7fc5e815e3c0 2023-08-09T15:11:13.796 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:13.792+0000 7fc5e5d9b640 1 -- 172.21.15.43:0/1924695638 --> v1:172.21.15.43:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fc5d0003520 con 0x7fc5e815e3c0 2023-08-09T15:11:13.796 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:13.792+0000 7fc5e5d9b640 1 -- 172.21.15.43:0/1924695638 <== mon.0 v1:172.21.15.43:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3010747045 0 0) 0x7fc5d0003520 con 0x7fc5e815e3c0 2023-08-09T15:11:13.796 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:13.792+0000 7fc5e5d9b640 1 -- 172.21.15.43:0/1924695638 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fc5e80cb2d0 msgr2=0x7fc5e815b380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-09T15:11:13.796 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:13.792+0000 7fc5e5d9b640 1 --2- 172.21.15.43:0/1924695638 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fc5e80cb2d0 0x7fc5e815b380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T15:11:13.796 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:13.793+0000 7fc5e5d9b640 1 -- 172.21.15.43:0/1924695638 >> v2:172.21.15.203:3300/0 conn(0x7fc5e815df60 msgr2=0x7fc5e80ce1c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-09T15:11:13.796 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:13.793+0000 7fc5e5d9b640 1 --2- 172.21.15.43:0/1924695638 >> v2:172.21.15.203:3300/0 conn(0x7fc5e815df60 0x7fc5e80ce1c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T15:11:13.796 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:13.793+0000 7fc5e5d9b640 1 -- 172.21.15.43:0/1924695638 --> v1:172.21.15.43:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc5e815e940 con 0x7fc5e815e3c0 2023-08-09T15:11:13.797 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:13.793+0000 7fc5e5d9b640 1 -- 172.21.15.43:0/1924695638 <== mon.0 v1:172.21.15.43:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1285150740 0 0) 0x7fc5e00019f0 con 0x7fc5e815e3c0 2023-08-09T15:11:13.797 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:13.793+0000 7fc5e659c640 1 --2- 172.21.15.43:0/1924695638 >> v2:172.21.15.203:3300/0 conn(0x7fc5e815df60 0x7fc5e80ce1c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-08-09T15:11:13.797 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:13.793+0000 7fc5e5d9b640 1 -- 172.21.15.43:0/1924695638 >> v1:172.21.15.43:6789/0 conn(0x7fc5e815e3c0 legacy=0x7fc5e80c9f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T15:11:13.797 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:13.793+0000 7fc5e5d9b640 1 --2- 172.21.15.43:0/1924695638 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc5d0003970 0x7fc5d0003d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T15:11:13.797 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:13.793+0000 7fc5e5d9b640 1 --2- 172.21.15.43:0/1924695638 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fc5d00047c0 0x7fc5d0004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T15:11:13.797 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:13.793+0000 7fc5e5d9b640 1 --2- 172.21.15.43:0/1924695638 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fc5e80cb2d0 0x7fc5d0005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T15:11:13.797 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:13.793+0000 7fc5e5d9b640 1 -- 172.21.15.43:0/1924695638 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc5e815a970 con 0x7fc5d0003970 2023-08-09T15:11:13.797 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:13.793+0000 7fc5e5d9b640 1 -- 172.21.15.43:0/1924695638 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc5e815b8c0 con 0x7fc5d00047c0 2023-08-09T15:11:13.798 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:13.793+0000 7fc5e5d9b640 1 -- 172.21.15.43:0/1924695638 --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc5d000ba30 con 0x7fc5e80cb2d0 2023-08-09T15:11:13.798 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:13.793+0000 7fc5e6d9d640 1 --2- 172.21.15.43:0/1924695638 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc5d0003970 0x7fc5d0003d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T15:11:13.798 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:13.793+0000 7fc5e6d9d640 1 -- 172.21.15.43:0/1924695638 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fc5e80cb2d0 msgr2=0x7fc5d0005180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-09T15:11:13.798 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:13.793+0000 7fc5e6d9d640 1 --2- 172.21.15.43:0/1924695638 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fc5e80cb2d0 0x7fc5d0005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T15:11:13.798 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:13.793+0000 7fc5e6d9d640 1 -- 172.21.15.43:0/1924695638 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fc5d00047c0 msgr2=0x7fc5d0004c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-09T15:11:13.798 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:13.793+0000 7fc5e659c640 1 --2- 172.21.15.43:0/1924695638 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fc5d00047c0 0x7fc5d0004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T15:11:13.798 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:13.793+0000 7fc5e6d9d640 1 --2- 172.21.15.43:0/1924695638 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fc5d00047c0 0x7fc5d0004c40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T15:11:13.799 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:13.793+0000 7fc5e6d9d640 1 -- 172.21.15.43:0/1924695638 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc5d0003520 con 0x7fc5d0003970 2023-08-09T15:11:13.800 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:13.794+0000 7fc5e6d9d640 1 --2- 172.21.15.43:0/1924695638 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc5d0003970 0x7fc5d0003d90 secure :-1 s=READY pgs=2328 cs=0 l=1 rev1=1 crypto rx=0x7fc5d800ba00 tx=0x7fc5d00057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9e32ca19b81d6fa server_cookie=0 in_seq=0 out_seq=0 2023-08-09T15:11:13.800 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:13.795+0000 7fc5e5d9b640 1 -- 172.21.15.43:0/1924695638 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc5d8032630 con 0x7fc5d0003970 2023-08-09T15:11:13.801 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:13.795+0000 7fc5e5d9b640 1 -- 172.21.15.43:0/1924695638 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc5d8032790 con 0x7fc5d0003970 2023-08-09T15:11:13.801 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:13.795+0000 7fc5e5d9b640 1 -- 172.21.15.43:0/1924695638 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc5d8032b20 con 0x7fc5d0003970 2023-08-09T15:11:13.801 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:13.795+0000 7fc5ed122640 1 -- 172.21.15.43:0/1924695638 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc5d0003970 msgr2=0x7fc5d0003d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T15:11:13.801 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:13.795+0000 7fc5ed122640 1 --2- 172.21.15.43:0/1924695638 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc5d0003970 0x7fc5d0003d90 secure :-1 s=READY pgs=2328 cs=0 l=1 rev1=1 crypto rx=0x7fc5d800ba00 tx=0x7fc5d00057f0 comp rx=0 tx=0).stop 2023-08-09T15:11:13.801 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:13.795+0000 7fc5e759e640 1 -- 172.21.15.43:0/1924695638 reap_dead start 2023-08-09T15:11:13.801 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:13.795+0000 7fc5ed122640 1 -- 172.21.15.43:0/1924695638 shutdown_connections 2023-08-09T15:11:13.801 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:13.795+0000 7fc5ed122640 1 -- 172.21.15.43:0/1924695638 >> 172.21.15.43:0/1924695638 conn(0x7fc5e80aff50 msgr2=0x7fc5e80b0360 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-09T15:11:13.802 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:13.795+0000 7fc5ed122640 1 -- 172.21.15.43:0/1924695638 shutdown_connections 2023-08-09T15:11:13.802 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:13.795+0000 7fc5ed122640 1 -- 172.21.15.43:0/1924695638 wait complete. 2023-08-09T15:11:13.802 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:13.798+0000 7fc5ed122640 1 Processor -- start 2023-08-09T15:11:13.802 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:13.798+0000 7fc5ed122640 1 -- start start 2023-08-09T15:11:13.802 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:13.798+0000 7fc5ed122640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc5d0003970 0x7fc5e80ce730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T15:11:13.802 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:13.798+0000 7fc5ed122640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fc5e80c6ae0 0x7fc5e80cd4d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T15:11:13.802 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:13.798+0000 7fc5ed122640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fc5e8159440 0x7fc5e81598c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T15:11:13.802 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:13.798+0000 7fc5ed122640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc5e81498c0 con 0x7fc5d0003970 2023-08-09T15:11:13.803 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:13.798+0000 7fc5ed122640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc5e80c6f60 con 0x7fc5e80c6ae0 2023-08-09T15:11:13.803 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:13.798+0000 7fc5ed122640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc5e8159e00 con 0x7fc5e8159440 2023-08-09T15:11:13.803 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:13.799+0000 7fc5e6d9d640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc5d0003970 0x7fc5e80ce730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T15:11:13.803 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:13.799+0000 7fc5e6d9d640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc5d0003970 0x7fc5e80ce730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:46696/0 (socket says 172.21.15.43:46696) 2023-08-09T15:11:13.803 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:13.799+0000 7fc5e6d9d640 1 -- 172.21.15.43:0/3151926943 learned_addr learned my addr 172.21.15.43:0/3151926943 (peer_addr_for_me v2:172.21.15.43:0/0) 2023-08-09T15:11:13.803 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:13.799+0000 7fc5e759e640 1 --2- 172.21.15.43:0/3151926943 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fc5e8159440 0x7fc5e81598c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T15:11:13.803 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:13.799+0000 7fc5e6d9d640 1 -- 172.21.15.43:0/3151926943 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fc5e8159440 msgr2=0x7fc5e81598c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T15:11:13.803 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:13.799+0000 7fc5e6d9d640 1 --2- 172.21.15.43:0/3151926943 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fc5e8159440 0x7fc5e81598c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T15:11:13.804 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:13.799+0000 7fc5e6d9d640 1 -- 172.21.15.43:0/3151926943 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fc5e80c6ae0 msgr2=0x7fc5e80cd4d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T15:11:13.804 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:13.799+0000 7fc5e6d9d640 1 --2- 172.21.15.43:0/3151926943 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fc5e80c6ae0 0x7fc5e80cd4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T15:11:13.804 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:13.799+0000 7fc5e6d9d640 1 -- 172.21.15.43:0/3151926943 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc5e80c7170 con 0x7fc5d0003970 2023-08-09T15:11:13.804 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:13.799+0000 7fc5e6d9d640 1 --2- 172.21.15.43:0/3151926943 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc5d0003970 0x7fc5e80ce730 secure :-1 s=READY pgs=2329 cs=0 l=1 rev1=1 crypto rx=0x7fc5d8006b10 tx=0x7fc5d8004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-09T15:11:13.805 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:13.800+0000 7fc5cf7fe640 1 -- 172.21.15.43:0/3151926943 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc5d8004220 con 0x7fc5d0003970 2023-08-09T15:11:13.805 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:13.800+0000 7fc5cf7fe640 1 -- 172.21.15.43:0/3151926943 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc5d8004380 con 0x7fc5d0003970 2023-08-09T15:11:13.805 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:13.800+0000 7fc5ed122640 1 -- 172.21.15.43:0/3151926943 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc5e80c7450 con 0x7fc5d0003970 2023-08-09T15:11:13.805 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:13.800+0000 7fc5cf7fe640 1 -- 172.21.15.43:0/3151926943 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc5d8003830 con 0x7fc5d0003970 2023-08-09T15:11:13.806 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:13.800+0000 7fc5ed122640 1 -- 172.21.15.43:0/3151926943 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc5e80b24d0 con 0x7fc5d0003970 2023-08-09T15:11:13.806 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:13.801+0000 7fc5ed122640 1 -- 172.21.15.43:0/3151926943 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc5e80b2050 con 0x7fc5d0003970 2023-08-09T15:11:13.806 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:13.802+0000 7fc5cf7fe640 1 -- 172.21.15.43:0/3151926943 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 118) v1 ==== 88443+0+0 (secure 0 0 0) 0x7fc5d8043070 con 0x7fc5d0003970 2023-08-09T15:11:13.806 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:13.802+0000 7fc5cf7fe640 1 --2- 172.21.15.43:0/3151926943 >> [v2:172.21.15.203:6832/4034093268,v1:172.21.15.203:6833/4034093268] conn(0x7fc5d406a170 0x7fc5d406c630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T15:11:13.806 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:13.803+0000 7fc5cf7fe640 1 -- 172.21.15.43:0/3151926943 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(212..212 src has 1..212) v4 ==== 7286+0+0 (secure 0 0 0) 0x7fc5d80ab220 con 0x7fc5d0003970 2023-08-09T15:11:13.806 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:13.803+0000 7fc5e659c640 1 --2- 172.21.15.43:0/3151926943 >> [v2:172.21.15.203:6832/4034093268,v1:172.21.15.203:6833/4034093268] conn(0x7fc5d406a170 0x7fc5d406c630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T15:11:13.807 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:13.804+0000 7fc5e659c640 1 --2- 172.21.15.43:0/3151926943 >> [v2:172.21.15.203:6832/4034093268,v1:172.21.15.203:6833/4034093268] conn(0x7fc5d406a170 0x7fc5d406c630 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0x7fc5dc006e80 tx=0x7fc5dc006e10 comp rx=0 tx=0).ready entity=mgr.8833 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-09T15:11:13.807 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:13.806+0000 7fc5cf7fe640 1 -- 172.21.15.43:0/3151926943 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178066 (secure 0 0 0) 0x7fc5e80b2050 con 0x7fc5d0003970 2023-08-09T15:11:14.027 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:14.025+0000 7fc5ed122640 1 -- 172.21.15.43:0/3151926943 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} v 0) v1 -- 0x7fc5e80b2050 con 0x7fc5d0003970 2023-08-09T15:11:14.577 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:14.577+0000 7fc5cf7fe640 1 -- 172.21.15.43:0/3151926943 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]=0 v98) v1 ==== 104+0+0 (secure 0 0 0) 0x7fc5e80b2050 con 0x7fc5d0003970 2023-08-09T15:11:14.581 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:14.581+0000 7fc5ed122640 1 -- 172.21.15.43:0/3151926943 >> [v2:172.21.15.203:6832/4034093268,v1:172.21.15.203:6833/4034093268] conn(0x7fc5d406a170 msgr2=0x7fc5d406c630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T15:11:14.581 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:14.581+0000 7fc5ed122640 1 --2- 172.21.15.43:0/3151926943 >> [v2:172.21.15.203:6832/4034093268,v1:172.21.15.203:6833/4034093268] conn(0x7fc5d406a170 0x7fc5d406c630 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0x7fc5dc006e80 tx=0x7fc5dc006e10 comp rx=0 tx=0).stop 2023-08-09T15:11:14.581 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:14.581+0000 7fc5ed122640 1 -- 172.21.15.43:0/3151926943 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc5d0003970 msgr2=0x7fc5e80ce730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T15:11:14.581 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:14.581+0000 7fc5ed122640 1 --2- 172.21.15.43:0/3151926943 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc5d0003970 0x7fc5e80ce730 secure :-1 s=READY pgs=2329 cs=0 l=1 rev1=1 crypto rx=0x7fc5d8006b10 tx=0x7fc5d8004730 comp rx=0 tx=0).stop 2023-08-09T15:11:14.582 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:14.581+0000 7fc5ed122640 1 -- 172.21.15.43:0/3151926943 shutdown_connections 2023-08-09T15:11:14.582 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:14.581+0000 7fc5ed122640 1 --2- 172.21.15.43:0/3151926943 >> [v2:172.21.15.203:6832/4034093268,v1:172.21.15.203:6833/4034093268] conn(0x7fc5d406a170 0x7fc5d406c630 unknown :-1 s=CLOSED pgs=31 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T15:11:14.582 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:14.581+0000 7fc5ed122640 1 --2- 172.21.15.43:0/3151926943 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fc5e8159440 0x7fc5e81598c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T15:11:14.582 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:14.581+0000 7fc5ed122640 1 --2- 172.21.15.43:0/3151926943 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fc5e80c6ae0 0x7fc5e80cd4d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T15:11:14.582 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:14.581+0000 7fc5ed122640 1 --2- 172.21.15.43:0/3151926943 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc5d0003970 0x7fc5e80ce730 unknown :-1 s=CLOSED pgs=2329 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T15:11:14.582 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:14.581+0000 7fc5ed122640 1 -- 172.21.15.43:0/3151926943 >> 172.21.15.43:0/3151926943 conn(0x7fc5e80aff50 msgr2=0x7fc5e815c590 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-09T15:11:14.583 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:14.581+0000 7fc5ed122640 1 -- 172.21.15.43:0/3151926943 shutdown_connections 2023-08-09T15:11:14.583 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:14.581+0000 7fc5ed122640 1 -- 172.21.15.43:0/3151926943 wait complete. 2023-08-09T15:11:14.603 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_metadata cephfs_metadata --yes-i-really-really-mean-it 2023-08-09T15:11:14.889 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:14.888+0000 7f6eca7a2640 1 Processor -- start 2023-08-09T15:11:14.889 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:14.888+0000 7f6eca7a2640 1 -- start start 2023-08-09T15:11:14.890 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:14.890+0000 7f6eca7a2640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f6ec415a330 0x7f6ec40c9780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T15:11:14.890 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:14.890+0000 7f6eca7a2640 1 -- --> v1:172.21.15.43:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6ec415dd30 con 0x7f6ec415ea60 2023-08-09T15:11:14.891 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:14.890+0000 7f6eca7a2640 1 -- --> v1:172.21.15.203:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6ec4071490 con 0x7f6ec40aff80 2023-08-09T15:11:14.891 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:14.890+0000 7f6eca7a2640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6ec40c9cc0 con 0x7f6ec415a330 2023-08-09T15:11:14.891 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:14.891+0000 7f6ec9fa1640 1 --1- >> v1:172.21.15.203:6789/0 conn(0x7f6ec40aff80 0x7f6ec406b400 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.203:6789/0 says I am v1:172.21.15.43:53654/0 (socket says 172.21.15.43:53654) 2023-08-09T15:11:14.891 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:14.891+0000 7f6ec9fa1640 1 -- 172.21.15.43:0/4197758471 learned_addr learned my addr 172.21.15.43:0/4197758471 (peer_addr_for_me v1:172.21.15.43:0/0) 2023-08-09T15:11:14.891 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:14.891+0000 7f6ec97a0640 1 --2- 172.21.15.43:0/4197758471 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f6ec415a330 0x7f6ec40c9780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T15:11:14.892 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:14.891+0000 7f6ec8f9f640 1 --1- 172.21.15.43:0/4197758471 >> v1:172.21.15.43:6789/0 conn(0x7f6ec415ea60 0x7f6ec40af870 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.43:6789/0 says I am v1:172.21.15.43:45594/0 (socket says 172.21.15.43:45594) 2023-08-09T15:11:14.892 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:14.891+0000 7f6ebbfff640 1 -- 172.21.15.43:0/4197758471 <== mon.1 v1:172.21.15.203:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3057849641 0 0) 0x7f6ec4071490 con 0x7f6ec40aff80 2023-08-09T15:11:14.892 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:14.892+0000 7f6ebbfff640 1 -- 172.21.15.43:0/4197758471 --> v1:172.21.15.203:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f6eb0003520 con 0x7f6ec40aff80 2023-08-09T15:11:14.893 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:14.892+0000 7f6ebbfff640 1 -- 172.21.15.43:0/4197758471 <== mon.0 v1:172.21.15.43:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2165850760 0 0) 0x7f6ec415dd30 con 0x7f6ec415ea60 2023-08-09T15:11:14.893 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:14.892+0000 7f6ebbfff640 1 -- 172.21.15.43:0/4197758471 --> v1:172.21.15.43:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f6ec4071490 con 0x7f6ec415ea60 2023-08-09T15:11:14.893 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:14.892+0000 7f6ec97a0640 1 -- 172.21.15.43:0/4197758471 >> v1:172.21.15.203:6789/0 conn(0x7f6ec40aff80 legacy=0x7f6ec406b400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T15:11:14.893 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:14.892+0000 7f6ec97a0640 1 -- 172.21.15.43:0/4197758471 >> v1:172.21.15.43:6789/0 conn(0x7f6ec415ea60 legacy=0x7f6ec40af870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T15:11:14.893 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:14.892+0000 7f6ec97a0640 1 -- 172.21.15.43:0/4197758471 --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6ec415df80 con 0x7f6ec415a330 2023-08-09T15:11:14.894 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:14.893+0000 7f6ec97a0640 1 --2- 172.21.15.43:0/4197758471 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f6ec415a330 0x7f6ec40c9780 secure :-1 s=READY pgs=330 cs=0 l=1 rev1=1 crypto rx=0x7f6ec00097d0 tx=0x7f6ec002b940 comp rx=0 tx=0).ready entity=mon.2 client_cookie=cf019ce6a9c4b2f4 server_cookie=0 in_seq=0 out_seq=0 2023-08-09T15:11:14.894 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:14.894+0000 7f6ebbfff640 1 -- 172.21.15.43:0/4197758471 <== mon.2 v2:172.21.15.203:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6ec0030800 con 0x7f6ec415a330 2023-08-09T15:11:14.894 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:14.894+0000 7f6ebbfff640 1 -- 172.21.15.43:0/4197758471 <== mon.2 v2:172.21.15.203:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6ec0030960 con 0x7f6ec415a330 2023-08-09T15:11:14.895 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:14.894+0000 7f6ebbfff640 1 -- 172.21.15.43:0/4197758471 <== mon.2 v2:172.21.15.203:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6ec0030cd0 con 0x7f6ec415a330 2023-08-09T15:11:14.895 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:14.895+0000 7f6eca7a2640 1 -- 172.21.15.43:0/4197758471 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f6ec415a330 msgr2=0x7f6ec40c9780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T15:11:14.895 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:14.895+0000 7f6eca7a2640 1 --2- 172.21.15.43:0/4197758471 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f6ec415a330 0x7f6ec40c9780 secure :-1 s=READY pgs=330 cs=0 l=1 rev1=1 crypto rx=0x7f6ec00097d0 tx=0x7f6ec002b940 comp rx=0 tx=0).stop 2023-08-09T15:11:14.896 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:14.895+0000 7f6eca7a2640 1 -- 172.21.15.43:0/4197758471 shutdown_connections 2023-08-09T15:11:14.896 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:14.895+0000 7f6eca7a2640 1 --2- 172.21.15.43:0/4197758471 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f6ec415a330 0x7f6ec40c9780 unknown :-1 s=CLOSED pgs=330 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T15:11:14.896 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:14.895+0000 7f6eca7a2640 1 -- 172.21.15.43:0/4197758471 >> 172.21.15.43:0/4197758471 conn(0x7f6ec40ca870 msgr2=0x7f6ec40cac50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-09T15:11:14.896 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:14.895+0000 7f6eca7a2640 1 -- 172.21.15.43:0/4197758471 shutdown_connections 2023-08-09T15:11:14.896 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:14.895+0000 7f6eca7a2640 1 -- 172.21.15.43:0/4197758471 wait complete. 2023-08-09T15:11:14.897 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:14.897+0000 7f6eca7a2640 1 Processor -- start 2023-08-09T15:11:14.897 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:14.897+0000 7f6eca7a2640 1 -- start start 2023-08-09T15:11:14.899 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:14.898+0000 7f6eca7a2640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f6ec40aff80 0x7f6ec40b2fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T15:11:14.899 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:14.898+0000 7f6eca7a2640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f6ec415a330 0x7f6ec41f0580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T15:11:14.899 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:14.898+0000 7f6eca7a2640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f6ec415ea60 0x7f6ec41547b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T15:11:14.899 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:14.898+0000 7f6eca7a2640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6ec415d2c0 con 0x7f6ec415ea60 2023-08-09T15:11:14.899 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:14.898+0000 7f6eca7a2640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6ec41f0ac0 con 0x7f6ec40aff80 2023-08-09T15:11:14.899 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:14.898+0000 7f6eca7a2640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6ec40b1f20 con 0x7f6ec415a330 2023-08-09T15:11:14.900 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:14.899+0000 7f6ec9fa1640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f6ec415ea60 0x7f6ec41547b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T15:11:14.900 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:14.899+0000 7f6ec9fa1640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f6ec415ea60 0x7f6ec41547b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:46708/0 (socket says 172.21.15.43:46708) 2023-08-09T15:11:14.900 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:14.899+0000 7f6ec9fa1640 1 -- 172.21.15.43:0/2993432579 learned_addr learned my addr 172.21.15.43:0/2993432579 (peer_addr_for_me v2:172.21.15.43:0/0) 2023-08-09T15:11:14.901 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:14.899+0000 7f6ec97a0640 1 --2- 172.21.15.43:0/2993432579 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f6ec40aff80 0x7f6ec40b2fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T15:11:14.901 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:14.899+0000 7f6ec8f9f640 1 --2- 172.21.15.43:0/2993432579 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f6ec415a330 0x7f6ec41f0580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T15:11:14.901 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:14.899+0000 7f6ec9fa1640 1 -- 172.21.15.43:0/2993432579 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f6ec415a330 msgr2=0x7f6ec41f0580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T15:11:14.901 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:14.899+0000 7f6ec9fa1640 1 --2- 172.21.15.43:0/2993432579 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f6ec415a330 0x7f6ec41f0580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T15:11:14.902 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:14.899+0000 7f6ec9fa1640 1 -- 172.21.15.43:0/2993432579 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f6ec40aff80 msgr2=0x7f6ec40b2fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T15:11:14.902 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:14.899+0000 7f6ec9fa1640 1 --2- 172.21.15.43:0/2993432579 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f6ec40aff80 0x7f6ec40b2fc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T15:11:14.902 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:14.899+0000 7f6ec9fa1640 1 -- 172.21.15.43:0/2993432579 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6ec0009400 con 0x7f6ec415ea60 2023-08-09T15:11:14.902 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:14.899+0000 7f6ec9fa1640 1 --2- 172.21.15.43:0/2993432579 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f6ec415ea60 0x7f6ec41547b0 secure :-1 s=READY pgs=2331 cs=0 l=1 rev1=1 crypto rx=0x7f6ebc00cbc0 tx=0x7f6ebc00a430 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-09T15:11:14.902 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:14.900+0000 7f6eb9ffb640 1 -- 172.21.15.43:0/2993432579 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6ebc00ace0 con 0x7f6ec415ea60 2023-08-09T15:11:14.902 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:14.900+0000 7f6eb9ffb640 1 -- 172.21.15.43:0/2993432579 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6ebc00ae40 con 0x7f6ec415ea60 2023-08-09T15:11:14.902 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:14.900+0000 7f6eb9ffb640 1 -- 172.21.15.43:0/2993432579 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6ebc012600 con 0x7f6ec415ea60 2023-08-09T15:11:14.902 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:14.900+0000 7f6eca7a2640 1 -- 172.21.15.43:0/2993432579 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6ec40b2200 con 0x7f6ec415ea60 2023-08-09T15:11:14.903 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:14.900+0000 7f6eca7a2640 1 -- 172.21.15.43:0/2993432579 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6ec40b27c0 con 0x7f6ec415ea60 2023-08-09T15:11:14.903 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:14.901+0000 7f6eca7a2640 1 -- 172.21.15.43:0/2993432579 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6ec40739f0 con 0x7f6ec415ea60 2023-08-09T15:11:14.903 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:14.902+0000 7f6eb9ffb640 1 -- 172.21.15.43:0/2993432579 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 118) v1 ==== 88443+0+0 (secure 0 0 0) 0x7f6ebc007070 con 0x7f6ec415ea60 2023-08-09T15:11:14.903 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:14.903+0000 7f6eb9ffb640 1 --2- 172.21.15.43:0/2993432579 >> [v2:172.21.15.203:6832/4034093268,v1:172.21.15.203:6833/4034093268] conn(0x7f6ea806a170 0x7f6ea806c630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T15:11:14.904 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:14.903+0000 7f6eb9ffb640 1 -- 172.21.15.43:0/2993432579 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(213..213 src has 1..213) v4 ==== 7286+0+0 (secure 0 0 0) 0x7f6ebc088b30 con 0x7f6ec415ea60 2023-08-09T15:11:14.904 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:14.904+0000 7f6ec97a0640 1 --2- 172.21.15.43:0/2993432579 >> [v2:172.21.15.203:6832/4034093268,v1:172.21.15.203:6833/4034093268] conn(0x7f6ea806a170 0x7f6ea806c630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T15:11:14.905 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:14.905+0000 7f6ec97a0640 1 --2- 172.21.15.43:0/2993432579 >> [v2:172.21.15.203:6832/4034093268,v1:172.21.15.203:6833/4034093268] conn(0x7f6ea806a170 0x7f6ea806c630 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0x7f6ec0036000 tx=0x7f6ec0008030 comp rx=0 tx=0).ready entity=mgr.8833 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-09T15:11:14.908 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:14.908+0000 7f6eb9ffb640 1 -- 172.21.15.43:0/2993432579 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178066 (secure 0 0 0) 0x7f6ebc053c50 con 0x7f6ec415ea60 2023-08-09T15:11:15.108 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:15.107+0000 7f6eca7a2640 1 -- 172.21.15.43:0/2993432579 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7f6ec406c550 con 0x7f6ec415ea60 2023-08-09T15:11:15.584 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:15.583+0000 7f6eb9ffb640 1 -- 172.21.15.43:0/2993432579 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_metadata' removed v214) v1 ==== 181+0+0 (secure 0 0 0) 0x7f6ebc057270 con 0x7f6ec415ea60 2023-08-09T15:11:15.584 INFO:teuthology.orchestra.run.smithi043.stderr:pool 'cephfs_metadata' removed 2023-08-09T15:11:15.588 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:15.588+0000 7f6eca7a2640 1 -- 172.21.15.43:0/2993432579 >> [v2:172.21.15.203:6832/4034093268,v1:172.21.15.203:6833/4034093268] conn(0x7f6ea806a170 msgr2=0x7f6ea806c630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T15:11:15.588 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:15.588+0000 7f6eca7a2640 1 --2- 172.21.15.43:0/2993432579 >> [v2:172.21.15.203:6832/4034093268,v1:172.21.15.203:6833/4034093268] conn(0x7f6ea806a170 0x7f6ea806c630 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0x7f6ec0036000 tx=0x7f6ec0008030 comp rx=0 tx=0).stop 2023-08-09T15:11:15.588 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:15.588+0000 7f6eca7a2640 1 -- 172.21.15.43:0/2993432579 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f6ec415ea60 msgr2=0x7f6ec41547b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T15:11:15.588 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:15.588+0000 7f6eca7a2640 1 --2- 172.21.15.43:0/2993432579 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f6ec415ea60 0x7f6ec41547b0 secure :-1 s=READY pgs=2331 cs=0 l=1 rev1=1 crypto rx=0x7f6ebc00cbc0 tx=0x7f6ebc00a430 comp rx=0 tx=0).stop 2023-08-09T15:11:15.589 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:15.588+0000 7f6eca7a2640 1 -- 172.21.15.43:0/2993432579 shutdown_connections 2023-08-09T15:11:15.589 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:15.588+0000 7f6eca7a2640 1 --2- 172.21.15.43:0/2993432579 >> [v2:172.21.15.203:6832/4034093268,v1:172.21.15.203:6833/4034093268] conn(0x7f6ea806a170 0x7f6ea806c630 unknown :-1 s=CLOSED pgs=33 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T15:11:15.589 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:15.588+0000 7f6eca7a2640 1 --2- 172.21.15.43:0/2993432579 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f6ec415ea60 0x7f6ec41547b0 unknown :-1 s=CLOSED pgs=2331 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T15:11:15.589 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:15.588+0000 7f6eca7a2640 1 --2- 172.21.15.43:0/2993432579 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f6ec415a330 0x7f6ec41f0580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T15:11:15.590 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:15.588+0000 7f6eca7a2640 1 --2- 172.21.15.43:0/2993432579 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f6ec40aff80 0x7f6ec40b2fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T15:11:15.590 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:15.588+0000 7f6eca7a2640 1 -- 172.21.15.43:0/2993432579 >> 172.21.15.43:0/2993432579 conn(0x7f6ec40ca870 msgr2=0x7f6ec406b8a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-09T15:11:15.590 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:15.588+0000 7f6eca7a2640 1 -- 172.21.15.43:0/2993432579 shutdown_connections 2023-08-09T15:11:15.590 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:15.588+0000 7f6eca7a2640 1 -- 172.21.15.43:0/2993432579 wait complete. 2023-08-09T15:11:15.610 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_data cephfs_data --yes-i-really-really-mean-it 2023-08-09T15:11:15.843 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:15.843+0000 7f11dc898640 1 Processor -- start 2023-08-09T15:11:15.844 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:15.843+0000 7f11dc898640 1 -- start start 2023-08-09T15:11:15.845 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:15.845+0000 7f11dc898640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f11d815c4f0 0x7f11d806b400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T15:11:15.845 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:15.845+0000 7f11dc898640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f11d806cae0 0x7f11d8160590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T15:11:15.846 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:15.845+0000 7f11dc898640 1 -- --> v1:172.21.15.43:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f11d80ad260 con 0x7f11d8160ad0 2023-08-09T15:11:15.846 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:15.845+0000 7f11dc898640 1 -- --> v2:172.21.15.43:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f11d806c280 con 0x7f11d815c4f0 2023-08-09T15:11:15.846 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:15.845+0000 7f11dc898640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f11d806b9e0 con 0x7f11d806cae0 2023-08-09T15:11:15.847 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:15.845+0000 7f11d6d9d640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f11d815c4f0 0x7f11d806b400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T15:11:15.847 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:15.845+0000 7f11d759e640 1 --1- >> v1:172.21.15.43:6789/0 conn(0x7f11d8160ad0 0x7f11d815c8e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.43:6789/0 says I am v1:172.21.15.43:45522/0 (socket says 172.21.15.43:45522) 2023-08-09T15:11:15.847 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:15.845+0000 7f11d759e640 1 -- 172.21.15.43:0/3568553760 learned_addr learned my addr 172.21.15.43:0/3568553760 (peer_addr_for_me v1:172.21.15.43:0/0) 2023-08-09T15:11:15.847 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:15.845+0000 7f11d6d9d640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f11d815c4f0 0x7f11d806b400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:37690/0 (socket says 172.21.15.43:37690) 2023-08-09T15:11:15.847 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:15.846+0000 7f11d659c640 1 --2- 172.21.15.43:0/3568553760 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f11d806cae0 0x7f11d8160590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T15:11:15.847 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:15.846+0000 7f11d6d9d640 1 -- 172.21.15.43:0/3568553760 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f11d806cae0 msgr2=0x7f11d8160590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-09T15:11:15.848 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:15.846+0000 7f11d6d9d640 1 --2- 172.21.15.43:0/3568553760 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f11d806cae0 0x7f11d8160590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T15:11:15.848 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:15.846+0000 7f11d6d9d640 1 -- 172.21.15.43:0/3568553760 >> v1:172.21.15.43:6789/0 conn(0x7f11d8160ad0 legacy=0x7f11d815c8e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T15:11:15.848 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:15.846+0000 7f11d5d9b640 1 -- 172.21.15.43:0/3568553760 <== mon.0 v1:172.21.15.43:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1363795714 0 0) 0x7f11d80ad260 con 0x7f11d8160ad0 2023-08-09T15:11:15.848 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:15.846+0000 7f11d6d9d640 1 -- 172.21.15.43:0/3568553760 --> v2:172.21.15.43:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f11d806bcc0 con 0x7f11d815c4f0 2023-08-09T15:11:15.849 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:15.847+0000 7f11d6d9d640 1 --2- 172.21.15.43:0/3568553760 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f11d815c4f0 0x7f11d806b400 secure :-1 s=READY pgs=2333 cs=0 l=1 rev1=1 crypto rx=0x7f11c80097a0 tx=0x7f11c802b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=29605cc9c5569c7c server_cookie=0 in_seq=0 out_seq=0 2023-08-09T15:11:15.849 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:15.847+0000 7f11d5d9b640 1 -- 172.21.15.43:0/3568553760 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f11c8031830 con 0x7f11d815c4f0 2023-08-09T15:11:15.849 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:15.847+0000 7f11d5d9b640 1 -- 172.21.15.43:0/3568553760 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f11c8031990 con 0x7f11d815c4f0 2023-08-09T15:11:15.850 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:15.847+0000 7f11d5d9b640 1 -- 172.21.15.43:0/3568553760 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f11c8031d20 con 0x7f11d815c4f0 2023-08-09T15:11:15.850 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:15.848+0000 7f11dc898640 1 -- 172.21.15.43:0/3568553760 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f11d815c4f0 msgr2=0x7f11d806b400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T15:11:15.850 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:15.848+0000 7f11dc898640 1 --2- 172.21.15.43:0/3568553760 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f11d815c4f0 0x7f11d806b400 secure :-1 s=READY pgs=2333 cs=0 l=1 rev1=1 crypto rx=0x7f11c80097a0 tx=0x7f11c802b8d0 comp rx=0 tx=0).stop 2023-08-09T15:11:15.850 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:15.848+0000 7f11dc898640 1 -- 172.21.15.43:0/3568553760 shutdown_connections 2023-08-09T15:11:15.850 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:15.848+0000 7f11dc898640 1 --2- 172.21.15.43:0/3568553760 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f11d806cae0 0x7f11d8160590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T15:11:15.850 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:15.848+0000 7f11dc898640 1 --2- 172.21.15.43:0/3568553760 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f11d815c4f0 0x7f11d806b400 unknown :-1 s=CLOSED pgs=2333 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T15:11:15.850 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:15.848+0000 7f11dc898640 1 -- 172.21.15.43:0/3568553760 >> 172.21.15.43:0/3568553760 conn(0x7f11d80ca870 msgr2=0x7f11d80cac50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-09T15:11:15.851 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:15.848+0000 7f11dc898640 1 -- 172.21.15.43:0/3568553760 shutdown_connections 2023-08-09T15:11:15.851 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:15.848+0000 7f11dc898640 1 -- 172.21.15.43:0/3568553760 wait complete. 2023-08-09T15:11:15.853 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:15.850+0000 7f11dc898640 1 Processor -- start 2023-08-09T15:11:15.853 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:15.850+0000 7f11dc898640 1 -- start start 2023-08-09T15:11:15.853 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:15.851+0000 7f11dc898640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f11d806cae0 0x7f11d81505e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T15:11:15.854 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:15.851+0000 7f11dc898640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f11d8160ad0 0x7f11d80c1ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T15:11:15.854 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:15.851+0000 7f11dc898640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f11d80c2530 0x7f11d80c9b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T15:11:15.854 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:15.851+0000 7f11dc898640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f11d80c7c10 con 0x7f11d8160ad0 2023-08-09T15:11:15.854 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:15.851+0000 7f11dc898640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f11d81f2840 con 0x7f11d806cae0 2023-08-09T15:11:15.854 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:15.851+0000 7f11dc898640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f11d81f1c30 con 0x7f11d80c2530 2023-08-09T15:11:15.854 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:15.851+0000 7f11d659c640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f11d8160ad0 0x7f11d80c1ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T15:11:15.855 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:15.851+0000 7f11d659c640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f11d8160ad0 0x7f11d80c1ff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:37706/0 (socket says 172.21.15.43:37706) 2023-08-09T15:11:15.855 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:15.852+0000 7f11d659c640 1 -- 172.21.15.43:0/1042695009 learned_addr learned my addr 172.21.15.43:0/1042695009 (peer_addr_for_me v2:172.21.15.43:0/0) 2023-08-09T15:11:15.855 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:15.852+0000 7f11d6d9d640 1 --2- 172.21.15.43:0/1042695009 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f11d806cae0 0x7f11d81505e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T15:11:15.855 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:15.852+0000 7f11d759e640 1 --2- 172.21.15.43:0/1042695009 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f11d80c2530 0x7f11d80c9b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T15:11:15.855 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:15.852+0000 7f11d659c640 1 -- 172.21.15.43:0/1042695009 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f11d80c2530 msgr2=0x7f11d80c9b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T15:11:15.855 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:15.852+0000 7f11d659c640 1 --2- 172.21.15.43:0/1042695009 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f11d80c2530 0x7f11d80c9b10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T15:11:15.855 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:15.852+0000 7f11d659c640 1 -- 172.21.15.43:0/1042695009 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f11d806cae0 msgr2=0x7f11d81505e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T15:11:15.856 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:15.852+0000 7f11d659c640 1 --2- 172.21.15.43:0/1042695009 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f11d806cae0 0x7f11d81505e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T15:11:15.856 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:15.852+0000 7f11d659c640 1 -- 172.21.15.43:0/1042695009 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f11c8009400 con 0x7f11d8160ad0 2023-08-09T15:11:15.856 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:15.852+0000 7f11d659c640 1 --2- 172.21.15.43:0/1042695009 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f11d8160ad0 0x7f11d80c1ff0 secure :-1 s=READY pgs=2334 cs=0 l=1 rev1=1 crypto rx=0x7f11cc00d800 tx=0x7f11cc00dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-09T15:11:15.856 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:15.853+0000 7f11bf7fe640 1 -- 172.21.15.43:0/1042695009 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f11cc00b6c0 con 0x7f11d8160ad0 2023-08-09T15:11:15.856 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:15.853+0000 7f11bf7fe640 1 -- 172.21.15.43:0/1042695009 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f11cc00b820 con 0x7f11d8160ad0 2023-08-09T15:11:15.856 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:15.853+0000 7f11dc898640 1 -- 172.21.15.43:0/1042695009 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f11d81f1f10 con 0x7f11d8160ad0 2023-08-09T15:11:15.857 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:15.853+0000 7f11bf7fe640 1 -- 172.21.15.43:0/1042695009 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f11cc00bbb0 con 0x7f11d8160ad0 2023-08-09T15:11:15.857 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:15.853+0000 7f11dc898640 1 -- 172.21.15.43:0/1042695009 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f11d81f24d0 con 0x7f11d8160ad0 2023-08-09T15:11:15.857 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:15.854+0000 7f11dc898640 1 -- 172.21.15.43:0/1042695009 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f11d80c5900 con 0x7f11d8160ad0 2023-08-09T15:11:15.858 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:15.855+0000 7f11bf7fe640 1 -- 172.21.15.43:0/1042695009 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 118) v1 ==== 88443+0+0 (secure 0 0 0) 0x7f11cc00bd10 con 0x7f11d8160ad0 2023-08-09T15:11:15.858 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:15.856+0000 7f11bf7fe640 1 --2- 172.21.15.43:0/1042695009 >> [v2:172.21.15.203:6832/4034093268,v1:172.21.15.203:6833/4034093268] conn(0x7f11c406a170 0x7f11c406c630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T15:11:15.858 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:15.856+0000 7f11bf7fe640 1 -- 172.21.15.43:0/1042695009 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(214..214 src has 1..214) v4 ==== 6820+0+0 (secure 0 0 0) 0x7f11cc088ff0 con 0x7f11d8160ad0 2023-08-09T15:11:15.858 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:15.857+0000 7f11d6d9d640 1 --2- 172.21.15.43:0/1042695009 >> [v2:172.21.15.203:6832/4034093268,v1:172.21.15.203:6833/4034093268] conn(0x7f11c406a170 0x7f11c406c630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T15:11:15.858 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:15.858+0000 7f11d6d9d640 1 --2- 172.21.15.43:0/1042695009 >> [v2:172.21.15.203:6832/4034093268,v1:172.21.15.203:6833/4034093268] conn(0x7f11c406a170 0x7f11c406c630 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0x7f11c8036000 tx=0x7f11c8008030 comp rx=0 tx=0).ready entity=mgr.8833 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-09T15:11:15.861 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:15.861+0000 7f11bf7fe640 1 -- 172.21.15.43:0/1042695009 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178066 (secure 0 0 0) 0x7f11cc0bb7b0 con 0x7f11d8160ad0 2023-08-09T15:11:16.077 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:16.076+0000 7f11dc898640 1 -- 172.21.15.43:0/1042695009 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7f11d806bf70 con 0x7f11d8160ad0 2023-08-09T15:11:16.588 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:16.588+0000 7f11bf7fe640 1 -- 172.21.15.43:0/1042695009 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data' removed v215) v1 ==== 169+0+0 (secure 0 0 0) 0x7f11d80c5900 con 0x7f11d8160ad0 2023-08-09T15:11:16.589 INFO:teuthology.orchestra.run.smithi043.stderr:pool 'cephfs_data' removed 2023-08-09T15:11:16.595 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:16.594+0000 7f11dc898640 1 -- 172.21.15.43:0/1042695009 >> [v2:172.21.15.203:6832/4034093268,v1:172.21.15.203:6833/4034093268] conn(0x7f11c406a170 msgr2=0x7f11c406c630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T15:11:16.595 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:16.594+0000 7f11dc898640 1 --2- 172.21.15.43:0/1042695009 >> [v2:172.21.15.203:6832/4034093268,v1:172.21.15.203:6833/4034093268] conn(0x7f11c406a170 0x7f11c406c630 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0x7f11c8036000 tx=0x7f11c8008030 comp rx=0 tx=0).stop 2023-08-09T15:11:16.595 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:16.594+0000 7f11dc898640 1 -- 172.21.15.43:0/1042695009 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f11d8160ad0 msgr2=0x7f11d80c1ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T15:11:16.595 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:16.594+0000 7f11dc898640 1 --2- 172.21.15.43:0/1042695009 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f11d8160ad0 0x7f11d80c1ff0 secure :-1 s=READY pgs=2334 cs=0 l=1 rev1=1 crypto rx=0x7f11cc00d800 tx=0x7f11cc00dcc0 comp rx=0 tx=0).stop 2023-08-09T15:11:16.599 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:16.599+0000 7f11dc898640 1 -- 172.21.15.43:0/1042695009 shutdown_connections 2023-08-09T15:11:16.600 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:16.599+0000 7f11dc898640 1 --2- 172.21.15.43:0/1042695009 >> [v2:172.21.15.203:6832/4034093268,v1:172.21.15.203:6833/4034093268] conn(0x7f11c406a170 0x7f11c406c630 unknown :-1 s=CLOSED pgs=34 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T15:11:16.600 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:16.599+0000 7f11dc898640 1 --2- 172.21.15.43:0/1042695009 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f11d80c2530 0x7f11d80c9b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T15:11:16.600 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:16.599+0000 7f11dc898640 1 --2- 172.21.15.43:0/1042695009 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f11d8160ad0 0x7f11d80c1ff0 unknown :-1 s=CLOSED pgs=2334 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T15:11:16.600 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:16.599+0000 7f11dc898640 1 --2- 172.21.15.43:0/1042695009 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f11d806cae0 0x7f11d81505e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T15:11:16.600 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:16.599+0000 7f11dc898640 1 -- 172.21.15.43:0/1042695009 >> 172.21.15.43:0/1042695009 conn(0x7f11d80ca870 msgr2=0x7f11d80ab8d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-09T15:11:16.601 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:16.600+0000 7f11dc898640 1 -- 172.21.15.43:0/1042695009 shutdown_connections 2023-08-09T15:11:16.601 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:16.600+0000 7f11dc898640 1 -- 172.21.15.43:0/1042695009 wait complete. 2023-08-09T15:11:16.623 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config rm mgr mgr/snap_schedule/dump_on_update 2023-08-09T15:11:16.845 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:16.845+0000 7f1f8f083640 1 Processor -- start 2023-08-09T15:11:16.846 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:16.845+0000 7f1f8f083640 1 -- start start 2023-08-09T15:11:16.847 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:16.847+0000 7f1f8f083640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f1f880cb2d0 0x7f1f8815b380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T15:11:16.847 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:16.847+0000 7f1f8f083640 1 -- --> v1:172.21.15.43:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1f8815aef0 con 0x7f1f8815df60 2023-08-09T15:11:16.847 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:16.847+0000 7f1f8f083640 1 -- --> v1:172.21.15.203:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1f8815b8c0 con 0x7f1f880ce8d0 2023-08-09T15:11:16.847 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:16.847+0000 7f1f8f083640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1f8815a970 con 0x7f1f880cb2d0 2023-08-09T15:11:16.849 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:16.847+0000 7f1f7ffff640 1 --1- >> v1:172.21.15.43:6789/0 conn(0x7f1f8815df60 0x7f1f880ce1c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.43:6789/0 says I am v1:172.21.15.43:45530/0 (socket says 172.21.15.43:45530) 2023-08-09T15:11:16.849 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:16.847+0000 7f1f7ffff640 1 -- 172.21.15.43:0/802749911 learned_addr learned my addr 172.21.15.43:0/802749911 (peer_addr_for_me v1:172.21.15.43:0/0) 2023-08-09T15:11:16.849 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:16.848+0000 7f1f8ce1f640 1 --2- 172.21.15.43:0/802749911 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f1f880cb2d0 0x7f1f8815b380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T15:11:16.849 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:16.848+0000 7f1f7f7fe640 1 -- 172.21.15.43:0/802749911 <== mon.0 v1:172.21.15.43:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2614246617 0 0) 0x7f1f8815aef0 con 0x7f1f8815df60 2023-08-09T15:11:16.850 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:16.848+0000 7f1f7f7fe640 1 -- 172.21.15.43:0/802749911 --> v1:172.21.15.43:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f1f70003520 con 0x7f1f8815df60 2023-08-09T15:11:16.850 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:16.848+0000 7f1f7f7fe640 1 -- 172.21.15.43:0/802749911 <== mon.0 v1:172.21.15.43:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 4284224764 0 0) 0x7f1f70003520 con 0x7f1f8815df60 2023-08-09T15:11:16.850 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:16.848+0000 7f1f7f7fe640 1 -- 172.21.15.43:0/802749911 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f1f880cb2d0 msgr2=0x7f1f8815b380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-09T15:11:16.850 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:16.848+0000 7f1f7f7fe640 1 --2- 172.21.15.43:0/802749911 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f1f880cb2d0 0x7f1f8815b380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T15:11:16.850 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:16.848+0000 7f1f7f7fe640 1 -- 172.21.15.43:0/802749911 >> v1:172.21.15.203:6789/0 conn(0x7f1f880ce8d0 legacy=0x7f1f880c9f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T15:11:16.850 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:16.848+0000 7f1f7f7fe640 1 -- 172.21.15.43:0/802749911 --> v1:172.21.15.43:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1f8815e940 con 0x7f1f8815df60 2023-08-09T15:11:16.850 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:16.848+0000 7f1f7f7fe640 1 -- 172.21.15.43:0/802749911 <== mon.0 v1:172.21.15.43:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1285150740 0 0) 0x7f1f78001670 con 0x7f1f8815df60 2023-08-09T15:11:16.850 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:16.848+0000 7f1f7f7fe640 1 -- 172.21.15.43:0/802749911 >> v1:172.21.15.43:6789/0 conn(0x7f1f8815df60 legacy=0x7f1f880ce1c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T15:11:16.851 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:16.848+0000 7f1f7f7fe640 1 --2- 172.21.15.43:0/802749911 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f1f700039a0 0x7f1f70003dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T15:11:16.851 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:16.848+0000 7f1f7f7fe640 1 --2- 172.21.15.43:0/802749911 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f1f700047c0 0x7f1f70004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T15:11:16.851 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:16.848+0000 7f1f7f7fe640 1 --2- 172.21.15.43:0/802749911 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f1f880cb2d0 0x7f1f70005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T15:11:16.851 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:16.848+0000 7f1f7f7fe640 1 -- 172.21.15.43:0/802749911 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1f8815a970 con 0x7f1f700039a0 2023-08-09T15:11:16.851 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:16.848+0000 7f1f7f7fe640 1 -- 172.21.15.43:0/802749911 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1f7000fe40 con 0x7f1f700047c0 2023-08-09T15:11:16.851 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:16.848+0000 7f1f7f7fe640 1 -- 172.21.15.43:0/802749911 --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1f70010000 con 0x7f1f880cb2d0 2023-08-09T15:11:16.851 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:16.849+0000 7f1f8ce1f640 1 --2- 172.21.15.43:0/802749911 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f1f700039a0 0x7f1f70003dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T15:11:16.852 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:16.849+0000 7f1f7ffff640 1 --2- 172.21.15.43:0/802749911 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f1f700047c0 0x7f1f70004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T15:11:16.853 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:16.849+0000 7f1f8d620640 1 --2- 172.21.15.43:0/802749911 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f1f880cb2d0 0x7f1f70005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T15:11:16.853 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:16.849+0000 7f1f8ce1f640 1 -- 172.21.15.43:0/802749911 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f1f880cb2d0 msgr2=0x7f1f70005180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-09T15:11:16.853 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:16.849+0000 7f1f8ce1f640 1 --2- 172.21.15.43:0/802749911 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f1f880cb2d0 0x7f1f70005180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T15:11:16.853 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:16.849+0000 7f1f8ce1f640 1 -- 172.21.15.43:0/802749911 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f1f700047c0 msgr2=0x7f1f70004c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-09T15:11:16.853 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:16.849+0000 7f1f8ce1f640 1 --2- 172.21.15.43:0/802749911 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f1f700047c0 0x7f1f70004c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T15:11:16.853 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:16.849+0000 7f1f8ce1f640 1 -- 172.21.15.43:0/802749911 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1f70003520 con 0x7f1f700039a0 2023-08-09T15:11:16.853 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:16.849+0000 7f1f7ffff640 1 --2- 172.21.15.43:0/802749911 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f1f700047c0 0x7f1f70004c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-08-09T15:11:16.854 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:16.850+0000 7f1f8ce1f640 1 --2- 172.21.15.43:0/802749911 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f1f700039a0 0x7f1f70003dc0 secure :-1 s=READY pgs=2336 cs=0 l=1 rev1=1 crypto rx=0x7f1f84009a90 tx=0x7f1f700057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1a166d650476aa9b server_cookie=0 in_seq=0 out_seq=0 2023-08-09T15:11:16.854 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:16.851+0000 7f1f7f7fe640 1 -- 172.21.15.43:0/802749911 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1f8402c660 con 0x7f1f700039a0 2023-08-09T15:11:16.854 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:16.851+0000 7f1f7f7fe640 1 -- 172.21.15.43:0/802749911 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1f8402c7c0 con 0x7f1f700039a0 2023-08-09T15:11:16.854 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:16.851+0000 7f1f7f7fe640 1 -- 172.21.15.43:0/802749911 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1f8402cb50 con 0x7f1f700039a0 2023-08-09T15:11:16.854 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:16.851+0000 7f1f8f083640 1 -- 172.21.15.43:0/802749911 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f1f700039a0 msgr2=0x7f1f70003dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T15:11:16.854 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:16.851+0000 7f1f8f083640 1 --2- 172.21.15.43:0/802749911 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f1f700039a0 0x7f1f70003dc0 secure :-1 s=READY pgs=2336 cs=0 l=1 rev1=1 crypto rx=0x7f1f84009a90 tx=0x7f1f700057f0 comp rx=0 tx=0).stop 2023-08-09T15:11:16.854 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:16.851+0000 7f1f8d620640 1 -- 172.21.15.43:0/802749911 reap_dead start 2023-08-09T15:11:16.855 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:16.851+0000 7f1f8f083640 1 -- 172.21.15.43:0/802749911 shutdown_connections 2023-08-09T15:11:16.855 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:16.851+0000 7f1f8f083640 1 -- 172.21.15.43:0/802749911 >> 172.21.15.43:0/802749911 conn(0x7f1f880aff50 msgr2=0x7f1f880b0360 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-09T15:11:16.855 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:16.851+0000 7f1f8f083640 1 -- 172.21.15.43:0/802749911 shutdown_connections 2023-08-09T15:11:16.855 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:16.851+0000 7f1f8f083640 1 -- 172.21.15.43:0/802749911 wait complete. 2023-08-09T15:11:16.856 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:16.853+0000 7f1f8f083640 1 Processor -- start 2023-08-09T15:11:16.857 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:16.853+0000 7f1f8f083640 1 -- start start 2023-08-09T15:11:16.857 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:16.854+0000 7f1f8f083640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f1f700039a0 0x7f1f8814eb70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T15:11:16.857 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:16.854+0000 7f1f8f083640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f1f880c5fa0 0x7f1f880c7820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T15:11:16.857 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:16.854+0000 7f1f8f083640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f1f88152570 0x7f1f881529f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T15:11:16.857 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:16.854+0000 7f1f8f083640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1f88152f30 con 0x7f1f700039a0 2023-08-09T15:11:16.857 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:16.854+0000 7f1f8f083640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1f880c81b0 con 0x7f1f88152570 2023-08-09T15:11:16.857 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:16.854+0000 7f1f8f083640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1f880b0c50 con 0x7f1f880c5fa0 2023-08-09T15:11:16.858 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:16.854+0000 7f1f8ce1f640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f1f700039a0 0x7f1f8814eb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T15:11:16.858 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:16.855+0000 7f1f8ce1f640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f1f700039a0 0x7f1f8814eb70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:37724/0 (socket says 172.21.15.43:37724) 2023-08-09T15:11:16.858 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:16.855+0000 7f1f8ce1f640 1 -- 172.21.15.43:0/736031455 learned_addr learned my addr 172.21.15.43:0/736031455 (peer_addr_for_me v2:172.21.15.43:0/0) 2023-08-09T15:11:16.858 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:16.855+0000 7f1f8ce1f640 1 -- 172.21.15.43:0/736031455 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f1f880c5fa0 msgr2=0x7f1f880c7820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T15:11:16.858 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:16.855+0000 7f1f8ce1f640 1 --2- 172.21.15.43:0/736031455 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f1f880c5fa0 0x7f1f880c7820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T15:11:16.858 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:16.855+0000 7f1f8ce1f640 1 -- 172.21.15.43:0/736031455 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f1f88152570 msgr2=0x7f1f881529f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T15:11:16.858 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:16.855+0000 7f1f8d620640 1 --2- 172.21.15.43:0/736031455 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f1f88152570 0x7f1f881529f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T15:11:16.858 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:16.855+0000 7f1f8ce1f640 1 --2- 172.21.15.43:0/736031455 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f1f88152570 0x7f1f881529f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T15:11:16.859 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:16.855+0000 7f1f8ce1f640 1 -- 172.21.15.43:0/736031455 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1f880b0f30 con 0x7f1f700039a0 2023-08-09T15:11:16.859 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:16.855+0000 7f1f8ce1f640 1 --2- 172.21.15.43:0/736031455 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f1f700039a0 0x7f1f8814eb70 secure :-1 s=READY pgs=2337 cs=0 l=1 rev1=1 crypto rx=0x7f1f84009a90 tx=0x7f1f84004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-09T15:11:16.859 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:16.856+0000 7f1f7d7fa640 1 -- 172.21.15.43:0/736031455 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1f84003810 con 0x7f1f700039a0 2023-08-09T15:11:16.859 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:16.856+0000 7f1f7d7fa640 1 -- 172.21.15.43:0/736031455 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1f84003970 con 0x7f1f700039a0 2023-08-09T15:11:16.859 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:16.856+0000 7f1f7d7fa640 1 -- 172.21.15.43:0/736031455 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1f84003d00 con 0x7f1f700039a0 2023-08-09T15:11:16.859 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:16.856+0000 7f1f8f083640 1 -- 172.21.15.43:0/736031455 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1f880b1210 con 0x7f1f700039a0 2023-08-09T15:11:16.859 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:16.856+0000 7f1f8f083640 1 -- 172.21.15.43:0/736031455 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1f8814f0b0 con 0x7f1f700039a0 2023-08-09T15:11:16.860 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:16.858+0000 7f1f8f083640 1 -- 172.21.15.43:0/736031455 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1f880739f0 con 0x7f1f700039a0 2023-08-09T15:11:16.860 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:16.859+0000 7f1f7d7fa640 1 -- 172.21.15.43:0/736031455 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 118) v1 ==== 88443+0+0 (secure 0 0 0) 0x7f1f84040450 con 0x7f1f700039a0 2023-08-09T15:11:16.861 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:16.861+0000 7f1f7d7fa640 1 --2- 172.21.15.43:0/736031455 >> [v2:172.21.15.203:6832/4034093268,v1:172.21.15.203:6833/4034093268] conn(0x7f1f7406a1a0 0x7f1f7406c660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T15:11:16.861 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:16.861+0000 7f1f7d7fa640 1 -- 172.21.15.43:0/736031455 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(215..215 src has 1..215) v4 ==== 6348+0+0 (secure 0 0 0) 0x7f1f840aa740 con 0x7f1f700039a0 2023-08-09T15:11:16.862 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:16.862+0000 7f1f7ffff640 1 --2- 172.21.15.43:0/736031455 >> [v2:172.21.15.203:6832/4034093268,v1:172.21.15.203:6833/4034093268] conn(0x7f1f7406a1a0 0x7f1f7406c660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T15:11:16.863 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:16.863+0000 7f1f7ffff640 1 --2- 172.21.15.43:0/736031455 >> [v2:172.21.15.203:6832/4034093268,v1:172.21.15.203:6833/4034093268] conn(0x7f1f7406a1a0 0x7f1f7406c660 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7f1f880ae0f0 tx=0x7f1f78009040 comp rx=0 tx=0).ready entity=mgr.8833 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-09T15:11:16.868 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:16.868+0000 7f1f7d7fa640 1 -- 172.21.15.43:0/736031455 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178066 (secure 0 0 0) 0x7f1f84075b60 con 0x7f1f700039a0 2023-08-09T15:11:17.064 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:17.064+0000 7f1f8f083640 1 -- 172.21.15.43:0/736031455 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "config rm", "who": "mgr", "name": "mgr/snap_schedule/dump_on_update"} v 0) v1 -- 0x7f1f880ade40 con 0x7f1f700039a0 2023-08-09T15:11:17.073 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:17.073+0000 7f1f7d7fa640 1 -- 172.21.15.43:0/736031455 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "config rm", "who": "mgr", "name": "mgr/snap_schedule/dump_on_update"}]=0 v67) v1 ==== 115+0+0 (secure 0 0 0) 0x7f1f840044e0 con 0x7f1f700039a0 2023-08-09T15:11:17.076 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:17.075+0000 7f1f8f083640 1 -- 172.21.15.43:0/736031455 >> [v2:172.21.15.203:6832/4034093268,v1:172.21.15.203:6833/4034093268] conn(0x7f1f7406a1a0 msgr2=0x7f1f7406c660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T15:11:17.076 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:17.075+0000 7f1f8f083640 1 --2- 172.21.15.43:0/736031455 >> [v2:172.21.15.203:6832/4034093268,v1:172.21.15.203:6833/4034093268] conn(0x7f1f7406a1a0 0x7f1f7406c660 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7f1f880ae0f0 tx=0x7f1f78009040 comp rx=0 tx=0).stop 2023-08-09T15:11:17.077 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:17.075+0000 7f1f8f083640 1 -- 172.21.15.43:0/736031455 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f1f700039a0 msgr2=0x7f1f8814eb70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T15:11:17.077 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:17.075+0000 7f1f8f083640 1 --2- 172.21.15.43:0/736031455 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f1f700039a0 0x7f1f8814eb70 secure :-1 s=READY pgs=2337 cs=0 l=1 rev1=1 crypto rx=0x7f1f84009a90 tx=0x7f1f84004730 comp rx=0 tx=0).stop 2023-08-09T15:11:17.077 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:17.076+0000 7f1f8f083640 1 -- 172.21.15.43:0/736031455 shutdown_connections 2023-08-09T15:11:17.077 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:17.076+0000 7f1f8f083640 1 --2- 172.21.15.43:0/736031455 >> [v2:172.21.15.203:6832/4034093268,v1:172.21.15.203:6833/4034093268] conn(0x7f1f7406a1a0 0x7f1f7406c660 unknown :-1 s=CLOSED pgs=35 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T15:11:17.077 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:17.076+0000 7f1f8f083640 1 --2- 172.21.15.43:0/736031455 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f1f88152570 0x7f1f881529f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T15:11:17.077 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:17.076+0000 7f1f8f083640 1 --2- 172.21.15.43:0/736031455 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f1f880c5fa0 0x7f1f880c7820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T15:11:17.077 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:17.076+0000 7f1f8f083640 1 --2- 172.21.15.43:0/736031455 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f1f700039a0 0x7f1f8814eb70 unknown :-1 s=CLOSED pgs=2337 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T15:11:17.077 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:17.076+0000 7f1f8f083640 1 -- 172.21.15.43:0/736031455 >> 172.21.15.43:0/736031455 conn(0x7f1f880aff50 msgr2=0x7f1f880c85e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-09T15:11:17.078 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:17.076+0000 7f1f8f083640 1 -- 172.21.15.43:0/736031455 shutdown_connections 2023-08-09T15:11:17.078 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:17.076+0000 7f1f8f083640 1 -- 172.21.15.43:0/736031455 wait complete. 2023-08-09T15:11:17.095 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config rm mgr mgr/snap_schedule/allow_m_granularity 2023-08-09T15:11:17.331 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:17.331+0000 7fe453fff640 1 Processor -- start 2023-08-09T15:11:17.332 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:17.331+0000 7fe453fff640 1 -- start start 2023-08-09T15:11:17.333 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:17.333+0000 7fe453fff640 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7fe45415c1a0 0x7fe45406db30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T15:11:17.334 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:17.333+0000 7fe453fff640 1 -- --> v1:172.21.15.43:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe454160080 con 0x7fe45415c580 2023-08-09T15:11:17.334 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:17.333+0000 7fe453fff640 1 -- --> v1:172.21.15.203:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe454160360 con 0x7fe454069ed0 2023-08-09T15:11:17.334 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:17.333+0000 7fe453fff640 1 -- --> v2:172.21.15.203:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe4540cb810 con 0x7fe45415c1a0 2023-08-09T15:11:17.334 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:17.333+0000 7fe4527fc640 1 --1- >> v1:172.21.15.43:6789/0 conn(0x7fe45415c580 0x7fe4540697c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.43:6789/0 says I am v1:172.21.15.43:45540/0 (socket says 172.21.15.43:45540) 2023-08-09T15:11:17.334 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:17.333+0000 7fe4527fc640 1 -- 172.21.15.43:0/4129498939 learned_addr learned my addr 172.21.15.43:0/4129498939 (peer_addr_for_me v1:172.21.15.43:0/0) 2023-08-09T15:11:17.335 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:17.334+0000 7fe451ffb640 1 -- 172.21.15.43:0/4129498939 <== mon.0 v1:172.21.15.43:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2572218146 0 0) 0x7fe454160080 con 0x7fe45415c580 2023-08-09T15:11:17.335 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:17.334+0000 7fe451ffb640 1 -- 172.21.15.43:0/4129498939 --> v1:172.21.15.43:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fe440003520 con 0x7fe45415c580 2023-08-09T15:11:17.336 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:17.334+0000 7fe452ffd640 1 --2- 172.21.15.43:0/4129498939 >> v2:172.21.15.203:3300/0 conn(0x7fe45415c1a0 0x7fe45406db30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T15:11:17.336 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:17.334+0000 7fe451ffb640 1 -- 172.21.15.43:0/4129498939 <== mon.0 v1:172.21.15.43:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1122669714 0 0) 0x7fe440003520 con 0x7fe45415c580 2023-08-09T15:11:17.336 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:17.334+0000 7fe451ffb640 1 -- 172.21.15.43:0/4129498939 >> v2:172.21.15.203:3300/0 conn(0x7fe45415c1a0 msgr2=0x7fe45406db30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-09T15:11:17.337 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:17.334+0000 7fe451ffb640 1 --2- 172.21.15.43:0/4129498939 >> v2:172.21.15.203:3300/0 conn(0x7fe45415c1a0 0x7fe45406db30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T15:11:17.337 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:17.334+0000 7fe451ffb640 1 -- 172.21.15.43:0/4129498939 >> v1:172.21.15.203:6789/0 conn(0x7fe454069ed0 legacy=0x7fe4540ca5c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-09T15:11:17.337 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:17.334+0000 7fe451ffb640 1 -- 172.21.15.43:0/4129498939 --> v1:172.21.15.43:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe454160640 con 0x7fe45415c580 2023-08-09T15:11:17.337 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:17.334+0000 7fe451ffb640 1 -- 172.21.15.43:0/4129498939 <== mon.0 v1:172.21.15.43:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1285150740 0 0) 0x7fe43c0019f0 con 0x7fe45415c580 2023-08-09T15:11:17.337 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:17.334+0000 7fe451ffb640 1 -- 172.21.15.43:0/4129498939 >> v1:172.21.15.43:6789/0 conn(0x7fe45415c580 legacy=0x7fe4540697c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T15:11:17.337 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:17.334+0000 7fe451ffb640 1 --2- 172.21.15.43:0/4129498939 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fe440003970 0x7fe440003d70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T15:11:17.337 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:17.334+0000 7fe451ffb640 1 --2- 172.21.15.43:0/4129498939 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fe4400047c0 0x7fe440004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T15:11:17.337 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:17.334+0000 7fe451ffb640 1 --2- 172.21.15.43:0/4129498939 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fe4400052a0 0x7fe44000b980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T15:11:17.338 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:17.334+0000 7fe451ffb640 1 -- 172.21.15.43:0/4129498939 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe4540cb810 con 0x7fe4400052a0 2023-08-09T15:11:17.338 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:17.334+0000 7fe451ffb640 1 -- 172.21.15.43:0/4129498939 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe44000c1f0 con 0x7fe4400047c0 2023-08-09T15:11:17.338 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:17.334+0000 7fe451ffb640 1 -- 172.21.15.43:0/4129498939 --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe44000c3b0 con 0x7fe440003970 2023-08-09T15:11:17.338 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:17.335+0000 7fe4537fe640 1 --2- 172.21.15.43:0/4129498939 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fe4400052a0 0x7fe44000b980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T15:11:17.338 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:17.335+0000 7fe4527fc640 1 --2- 172.21.15.43:0/4129498939 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fe4400047c0 0x7fe440004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T15:11:17.338 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:17.335+0000 7fe4537fe640 1 -- 172.21.15.43:0/4129498939 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fe440003970 msgr2=0x7fe440003d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-09T15:11:17.338 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:17.335+0000 7fe4537fe640 1 --2- 172.21.15.43:0/4129498939 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fe440003970 0x7fe440003d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T15:11:17.339 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:17.335+0000 7fe4537fe640 1 -- 172.21.15.43:0/4129498939 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fe4400047c0 msgr2=0x7fe440004c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-09T15:11:17.339 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:17.335+0000 7fe4537fe640 1 --2- 172.21.15.43:0/4129498939 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fe4400047c0 0x7fe440004c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T15:11:17.339 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:17.335+0000 7fe4537fe640 1 -- 172.21.15.43:0/4129498939 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe440003520 con 0x7fe4400052a0 2023-08-09T15:11:17.340 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:17.336+0000 7fe4537fe640 1 -- 172.21.15.43:0/4129498939 reap_dead start 2023-08-09T15:11:17.340 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:17.336+0000 7fe4537fe640 1 --2- 172.21.15.43:0/4129498939 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fe4400052a0 0x7fe44000b980 secure :-1 s=READY pgs=2339 cs=0 l=1 rev1=1 crypto rx=0x7fe44c003d80 tx=0x7fe4400057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6767deff0b24a506 server_cookie=0 in_seq=0 out_seq=0 2023-08-09T15:11:17.340 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:17.337+0000 7fe451ffb640 1 -- 172.21.15.43:0/4129498939 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe44c007220 con 0x7fe4400052a0 2023-08-09T15:11:17.341 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:17.337+0000 7fe451ffb640 1 -- 172.21.15.43:0/4129498939 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe44c02e050 con 0x7fe4400052a0 2023-08-09T15:11:17.341 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:17.337+0000 7fe451ffb640 1 -- 172.21.15.43:0/4129498939 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe44c0075e0 con 0x7fe4400052a0 2023-08-09T15:11:17.341 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:17.337+0000 7fe453fff640 1 -- 172.21.15.43:0/4129498939 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fe4400052a0 msgr2=0x7fe44000b980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T15:11:17.341 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:17.337+0000 7fe453fff640 1 --2- 172.21.15.43:0/4129498939 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fe4400052a0 0x7fe44000b980 secure :-1 s=READY pgs=2339 cs=0 l=1 rev1=1 crypto rx=0x7fe44c003d80 tx=0x7fe4400057f0 comp rx=0 tx=0).stop 2023-08-09T15:11:17.341 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:17.338+0000 7fe453fff640 1 -- 172.21.15.43:0/4129498939 shutdown_connections 2023-08-09T15:11:17.341 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:17.338+0000 7fe453fff640 1 --2- 172.21.15.43:0/4129498939 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fe4400052a0 0x7fe44000b980 unknown :-1 s=CLOSED pgs=2339 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T15:11:17.341 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:17.338+0000 7fe453fff640 1 -- 172.21.15.43:0/4129498939 >> 172.21.15.43:0/4129498939 conn(0x7fe4540cb970 msgr2=0x7fe4540cbd50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-09T15:11:17.342 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:17.338+0000 7fe453fff640 1 -- 172.21.15.43:0/4129498939 shutdown_connections 2023-08-09T15:11:17.342 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:17.338+0000 7fe453fff640 1 -- 172.21.15.43:0/4129498939 wait complete. 2023-08-09T15:11:17.342 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:17.340+0000 7fe453fff640 1 Processor -- start 2023-08-09T15:11:17.342 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:17.340+0000 7fe453fff640 1 -- start start 2023-08-09T15:11:17.343 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:17.341+0000 7fe453fff640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fe4400052a0 0x7fe4541f8a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T15:11:17.344 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:17.341+0000 7fe453fff640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fe4541f68b0 0x7fe45415e2d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T15:11:17.344 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:17.341+0000 7fe453fff640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fe45415e810 0x7fe4540ca9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T15:11:17.344 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:17.341+0000 7fe453fff640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe45415f930 con 0x7fe45415e810 2023-08-09T15:11:17.344 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:17.341+0000 7fe453fff640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe454152ec0 con 0x7fe4400052a0 2023-08-09T15:11:17.344 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:17.341+0000 7fe453fff640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe4541f8fc0 con 0x7fe4541f68b0 2023-08-09T15:11:17.344 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:17.341+0000 7fe4537fe640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fe45415e810 0x7fe4540ca9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T15:11:17.344 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:17.341+0000 7fe4537fe640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fe45415e810 0x7fe4540ca9f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:37740/0 (socket says 172.21.15.43:37740) 2023-08-09T15:11:17.345 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:17.341+0000 7fe4537fe640 1 -- 172.21.15.43:0/2089636958 learned_addr learned my addr 172.21.15.43:0/2089636958 (peer_addr_for_me v2:172.21.15.43:0/0) 2023-08-09T15:11:17.345 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:17.341+0000 7fe452ffd640 1 --2- 172.21.15.43:0/2089636958 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fe4400052a0 0x7fe4541f8a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T15:11:17.345 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:17.342+0000 7fe4537fe640 1 -- 172.21.15.43:0/2089636958 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fe4541f68b0 msgr2=0x7fe45415e2d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T15:11:17.345 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:17.342+0000 7fe4537fe640 1 --2- 172.21.15.43:0/2089636958 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fe4541f68b0 0x7fe45415e2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T15:11:17.345 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:17.342+0000 7fe4537fe640 1 -- 172.21.15.43:0/2089636958 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fe4400052a0 msgr2=0x7fe4541f8a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T15:11:17.345 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:17.342+0000 7fe4537fe640 1 --2- 172.21.15.43:0/2089636958 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fe4400052a0 0x7fe4541f8a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T15:11:17.346 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:17.342+0000 7fe4537fe640 1 -- 172.21.15.43:0/2089636958 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe45415ecb0 con 0x7fe45415e810 2023-08-09T15:11:17.346 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:17.342+0000 7fe452ffd640 1 --2- 172.21.15.43:0/2089636958 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fe4400052a0 0x7fe4541f8a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-08-09T15:11:17.346 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:17.342+0000 7fe4537fe640 1 --2- 172.21.15.43:0/2089636958 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fe45415e810 0x7fe4540ca9f0 secure :-1 s=READY pgs=2340 cs=0 l=1 rev1=1 crypto rx=0x7fe44c003ea0 tx=0x7fe44c0036f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-09T15:11:17.346 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:17.342+0000 7fe44b7fe640 1 -- 172.21.15.43:0/2089636958 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe44c006ea0 con 0x7fe45415e810 2023-08-09T15:11:17.346 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:17.342+0000 7fe44b7fe640 1 -- 172.21.15.43:0/2089636958 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe44c03e470 con 0x7fe45415e810 2023-08-09T15:11:17.346 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:17.342+0000 7fe44b7fe640 1 -- 172.21.15.43:0/2089636958 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe44c03e820 con 0x7fe45415e810 2023-08-09T15:11:17.346 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:17.342+0000 7fe453fff640 1 -- 172.21.15.43:0/2089636958 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe45415ef90 con 0x7fe45415e810 2023-08-09T15:11:17.346 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:17.342+0000 7fe453fff640 1 -- 172.21.15.43:0/2089636958 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe45415f550 con 0x7fe45415e810 2023-08-09T15:11:17.347 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:17.344+0000 7fe453fff640 1 -- 172.21.15.43:0/2089636958 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe45404e390 con 0x7fe45415e810 2023-08-09T15:11:17.347 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:17.345+0000 7fe44b7fe640 1 -- 172.21.15.43:0/2089636958 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 118) v1 ==== 88443+0+0 (secure 0 0 0) 0x7fe44c03e980 con 0x7fe45415e810 2023-08-09T15:11:17.347 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:17.346+0000 7fe44b7fe640 1 --2- 172.21.15.43:0/2089636958 >> [v2:172.21.15.203:6832/4034093268,v1:172.21.15.203:6833/4034093268] conn(0x7fe43406a170 0x7fe43406c630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T15:11:17.348 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:17.346+0000 7fe44b7fe640 1 -- 172.21.15.43:0/2089636958 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(215..215 src has 1..215) v4 ==== 6348+0+0 (secure 0 0 0) 0x7fe44c0aa700 con 0x7fe45415e810 2023-08-09T15:11:17.348 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:17.347+0000 7fe452ffd640 1 --2- 172.21.15.43:0/2089636958 >> [v2:172.21.15.203:6832/4034093268,v1:172.21.15.203:6833/4034093268] conn(0x7fe43406a170 0x7fe43406c630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T15:11:17.348 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:17.347+0000 7fe452ffd640 1 --2- 172.21.15.43:0/2089636958 >> [v2:172.21.15.203:6832/4034093268,v1:172.21.15.203:6833/4034093268] conn(0x7fe43406a170 0x7fe43406c630 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7fe444006a90 tx=0x7fe444008040 comp rx=0 tx=0).ready entity=mgr.8833 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-09T15:11:17.351 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:17.351+0000 7fe44b7fe640 1 -- 172.21.15.43:0/2089636958 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178066 (secure 0 0 0) 0x7fe44c075b30 con 0x7fe45415e810 2023-08-09T15:11:17.545 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:17.544+0000 7fe453fff640 1 -- 172.21.15.43:0/2089636958 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "config rm", "who": "mgr", "name": "mgr/snap_schedule/allow_m_granularity"} v 0) v1 -- 0x7fe454070950 con 0x7fe45415e810 2023-08-09T15:11:17.551 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:17.551+0000 7fe44b7fe640 1 -- 172.21.15.43:0/2089636958 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "config rm", "who": "mgr", "name": "mgr/snap_schedule/allow_m_granularity"}]=0 v68) v1 ==== 120+0+0 (secure 0 0 0) 0x7fe44c079150 con 0x7fe45415e810 2023-08-09T15:11:17.555 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:17.554+0000 7fe453fff640 1 -- 172.21.15.43:0/2089636958 >> [v2:172.21.15.203:6832/4034093268,v1:172.21.15.203:6833/4034093268] conn(0x7fe43406a170 msgr2=0x7fe43406c630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T15:11:17.555 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:17.554+0000 7fe453fff640 1 --2- 172.21.15.43:0/2089636958 >> [v2:172.21.15.203:6832/4034093268,v1:172.21.15.203:6833/4034093268] conn(0x7fe43406a170 0x7fe43406c630 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7fe444006a90 tx=0x7fe444008040 comp rx=0 tx=0).stop 2023-08-09T15:11:17.555 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:17.554+0000 7fe453fff640 1 -- 172.21.15.43:0/2089636958 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fe45415e810 msgr2=0x7fe4540ca9f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T15:11:17.555 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:17.554+0000 7fe453fff640 1 --2- 172.21.15.43:0/2089636958 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fe45415e810 0x7fe4540ca9f0 secure :-1 s=READY pgs=2340 cs=0 l=1 rev1=1 crypto rx=0x7fe44c003ea0 tx=0x7fe44c0036f0 comp rx=0 tx=0).stop 2023-08-09T15:11:17.556 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:17.555+0000 7fe453fff640 1 -- 172.21.15.43:0/2089636958 shutdown_connections 2023-08-09T15:11:17.556 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:17.555+0000 7fe453fff640 1 --2- 172.21.15.43:0/2089636958 >> [v2:172.21.15.203:6832/4034093268,v1:172.21.15.203:6833/4034093268] conn(0x7fe43406a170 0x7fe43406c630 unknown :-1 s=CLOSED pgs=36 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T15:11:17.556 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:17.555+0000 7fe453fff640 1 --2- 172.21.15.43:0/2089636958 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fe45415e810 0x7fe4540ca9f0 unknown :-1 s=CLOSED pgs=2340 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T15:11:17.556 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:17.555+0000 7fe453fff640 1 --2- 172.21.15.43:0/2089636958 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fe4541f68b0 0x7fe45415e2d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T15:11:17.556 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:17.555+0000 7fe453fff640 1 --2- 172.21.15.43:0/2089636958 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fe4400052a0 0x7fe4541f8a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T15:11:17.556 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:17.555+0000 7fe453fff640 1 -- 172.21.15.43:0/2089636958 >> 172.21.15.43:0/2089636958 conn(0x7fe4540cb970 msgr2=0x7fe4540ce3c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-09T15:11:17.557 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:17.555+0000 7fe453fff640 1 -- 172.21.15.43:0/2089636958 shutdown_connections 2023-08-09T15:11:17.557 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:17.555+0000 7fe453fff640 1 -- 172.21.15.43:0/2089636958 wait complete. 2023-08-09T15:11:17.573 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config rm mon mon_allow_pool_delete 2023-08-09T15:11:17.803 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:17.803+0000 7f043bfff640 1 Processor -- start 2023-08-09T15:11:17.804 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:17.803+0000 7f043bfff640 1 -- start start 2023-08-09T15:11:17.805 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:17.805+0000 7f043bfff640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f043c0c9ee0 0x7f043c15a800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T15:11:17.805 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:17.805+0000 7f043bfff640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f043c0c90a0 0x7f043c15ad40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T15:11:17.806 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:17.805+0000 7f043bfff640 1 -- --> v1:172.21.15.43:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f043c158280 con 0x7f043c157ea0 2023-08-09T15:11:17.806 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:17.805+0000 7f043bfff640 1 -- --> v2:172.21.15.43:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f043c15b840 con 0x7f043c0c90a0 2023-08-09T15:11:17.806 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:17.805+0000 7f043bfff640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f043c0b3810 con 0x7f043c0c9ee0 2023-08-09T15:11:17.807 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:17.805+0000 7f043b7fe640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f043c0c90a0 0x7f043c15ad40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T15:11:17.807 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:17.806+0000 7f043b7fe640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f043c0c90a0 0x7f043c15ad40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:37750/0 (socket says 172.21.15.43:37750) 2023-08-09T15:11:17.807 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:17.806+0000 7f043b7fe640 1 -- 172.21.15.43:0/3991806170 learned_addr learned my addr 172.21.15.43:0/3991806170 (peer_addr_for_me v2:172.21.15.43:0/0) 2023-08-09T15:11:17.807 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:17.806+0000 7f043affd640 1 --1- 172.21.15.43:0/3991806170 >> v1:172.21.15.43:6789/0 conn(0x7f043c157ea0 0x7f043c0c97d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.43:6789/0 says I am v1:172.21.15.43:45556/0 (socket says 172.21.15.43:45556) 2023-08-09T15:11:17.807 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:17.806+0000 7f043a7fc640 1 --2- 172.21.15.43:0/3991806170 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f043c0c9ee0 0x7f043c15a800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T15:11:17.807 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:17.806+0000 7f043b7fe640 1 -- 172.21.15.43:0/3991806170 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f043c0c9ee0 msgr2=0x7f043c15a800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-09T15:11:17.808 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:17.806+0000 7f043b7fe640 1 --2- 172.21.15.43:0/3991806170 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f043c0c9ee0 0x7f043c15a800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T15:11:17.808 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:17.806+0000 7f043b7fe640 1 -- 172.21.15.43:0/3991806170 >> v1:172.21.15.43:6789/0 conn(0x7f043c157ea0 legacy=0x7f043c0c97d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T15:11:17.808 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:17.806+0000 7f043b7fe640 1 -- 172.21.15.43:0/3991806170 --> v2:172.21.15.43:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f043c15b560 con 0x7f043c0c90a0 2023-08-09T15:11:17.808 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:17.806+0000 7f043a7fc640 1 --2- 172.21.15.43:0/3991806170 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f043c0c9ee0 0x7f043c15a800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-08-09T15:11:17.809 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:17.808+0000 7f043b7fe640 1 --2- 172.21.15.43:0/3991806170 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f043c0c90a0 0x7f043c15ad40 secure :-1 s=READY pgs=2342 cs=0 l=1 rev1=1 crypto rx=0x7f0434009870 tx=0x7f043402b980 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6b10d807916bfebc server_cookie=0 in_seq=0 out_seq=0 2023-08-09T15:11:17.809 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:17.808+0000 7f0439ffb640 1 -- 172.21.15.43:0/3991806170 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0434030840 con 0x7f043c0c90a0 2023-08-09T15:11:17.810 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:17.808+0000 7f0439ffb640 1 -- 172.21.15.43:0/3991806170 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f04340309a0 con 0x7f043c0c90a0 2023-08-09T15:11:17.810 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:17.808+0000 7f0439ffb640 1 -- 172.21.15.43:0/3991806170 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0434030d30 con 0x7f043c0c90a0 2023-08-09T15:11:17.810 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:17.809+0000 7f043bfff640 1 -- 172.21.15.43:0/3991806170 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f043c0c90a0 msgr2=0x7f043c15ad40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T15:11:17.810 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:17.809+0000 7f043bfff640 1 --2- 172.21.15.43:0/3991806170 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f043c0c90a0 0x7f043c15ad40 secure :-1 s=READY pgs=2342 cs=0 l=1 rev1=1 crypto rx=0x7f0434009870 tx=0x7f043402b980 comp rx=0 tx=0).stop 2023-08-09T15:11:17.810 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:17.809+0000 7f043bfff640 1 -- 172.21.15.43:0/3991806170 shutdown_connections 2023-08-09T15:11:17.810 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:17.809+0000 7f043bfff640 1 --2- 172.21.15.43:0/3991806170 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f043c0c90a0 0x7f043c15ad40 unknown :-1 s=CLOSED pgs=2342 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T15:11:17.810 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:17.809+0000 7f043bfff640 1 --2- 172.21.15.43:0/3991806170 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f043c0c9ee0 0x7f043c15a800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T15:11:17.811 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:17.809+0000 7f043bfff640 1 -- 172.21.15.43:0/3991806170 >> 172.21.15.43:0/3991806170 conn(0x7f043c0cddd0 msgr2=0x7f043c0ce1e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-09T15:11:17.811 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:17.809+0000 7f043bfff640 1 -- 172.21.15.43:0/3991806170 shutdown_connections 2023-08-09T15:11:17.811 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:17.809+0000 7f043bfff640 1 -- 172.21.15.43:0/3991806170 wait complete. 2023-08-09T15:11:17.811 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:17.811+0000 7f043bfff640 1 Processor -- start 2023-08-09T15:11:17.811 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:17.811+0000 7f043bfff640 1 -- start start 2023-08-09T15:11:17.812 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:17.812+0000 7f043bfff640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f043c0c90a0 0x7f043c150410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T15:11:17.812 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:17.812+0000 7f043bfff640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f043c0c9ee0 0x7f043c0c2fa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T15:11:17.813 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:17.812+0000 7f043bfff640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f043c157ea0 0x7f043c0c2140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T15:11:17.813 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:17.812+0000 7f043bfff640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f043c0c1980 con 0x7f043c0c90a0 2023-08-09T15:11:17.813 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:17.812+0000 7f043bfff640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f043c0c3940 con 0x7f043c157ea0 2023-08-09T15:11:17.813 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:17.812+0000 7f043bfff640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f043c0c3c20 con 0x7f043c0c9ee0 2023-08-09T15:11:17.814 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:17.812+0000 7f043affd640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f043c0c90a0 0x7f043c150410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T15:11:17.814 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:17.812+0000 7f043affd640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f043c0c90a0 0x7f043c150410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:37752/0 (socket says 172.21.15.43:37752) 2023-08-09T15:11:17.814 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:17.812+0000 7f043affd640 1 -- 172.21.15.43:0/2481863426 learned_addr learned my addr 172.21.15.43:0/2481863426 (peer_addr_for_me v2:172.21.15.43:0/0) 2023-08-09T15:11:17.815 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:17.813+0000 7f043a7fc640 1 --2- 172.21.15.43:0/2481863426 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f043c0c9ee0 0x7f043c0c2fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T15:11:17.815 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:17.813+0000 7f043b7fe640 1 --2- 172.21.15.43:0/2481863426 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f043c157ea0 0x7f043c0c2140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T15:11:17.815 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:17.813+0000 7f043affd640 1 -- 172.21.15.43:0/2481863426 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f043c0c9ee0 msgr2=0x7f043c0c2fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T15:11:17.816 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:17.813+0000 7f043affd640 1 --2- 172.21.15.43:0/2481863426 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f043c0c9ee0 0x7f043c0c2fa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T15:11:17.816 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:17.813+0000 7f043affd640 1 -- 172.21.15.43:0/2481863426 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f043c157ea0 msgr2=0x7f043c0c2140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T15:11:17.816 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:17.813+0000 7f043affd640 1 --2- 172.21.15.43:0/2481863426 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f043c157ea0 0x7f043c0c2140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T15:11:17.816 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:17.813+0000 7f043affd640 1 -- 172.21.15.43:0/2481863426 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f04340094d0 con 0x7f043c0c90a0 2023-08-09T15:11:17.816 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:17.813+0000 7f043b7fe640 1 --2- 172.21.15.43:0/2481863426 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f043c157ea0 0x7f043c0c2140 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-08-09T15:11:17.816 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:17.813+0000 7f043a7fc640 1 --2- 172.21.15.43:0/2481863426 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f043c0c9ee0 0x7f043c0c2fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-08-09T15:11:17.817 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:17.813+0000 7f043affd640 1 --2- 172.21.15.43:0/2481863426 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f043c0c90a0 0x7f043c150410 secure :-1 s=READY pgs=2343 cs=0 l=1 rev1=1 crypto rx=0x7f042c00c7a0 tx=0x7f042c00cc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-09T15:11:17.817 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:17.813+0000 7f0433fff640 1 -- 172.21.15.43:0/2481863426 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f042c00a640 con 0x7f043c0c90a0 2023-08-09T15:11:17.817 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:17.813+0000 7f0433fff640 1 -- 172.21.15.43:0/2481863426 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f042c00a7a0 con 0x7f043c0c90a0 2023-08-09T15:11:17.817 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:17.813+0000 7f0433fff640 1 -- 172.21.15.43:0/2481863426 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f042c00ab10 con 0x7f043c0c90a0 2023-08-09T15:11:17.817 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:17.813+0000 7f043bfff640 1 -- 172.21.15.43:0/2481863426 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f043c0c1c20 con 0x7f043c0c90a0 2023-08-09T15:11:17.817 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:17.814+0000 7f043bfff640 1 -- 172.21.15.43:0/2481863426 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f043c150950 con 0x7f043c0c90a0 2023-08-09T15:11:17.817 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:17.815+0000 7f043bfff640 1 -- 172.21.15.43:0/2481863426 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f043c0c1e60 con 0x7f043c0c90a0 2023-08-09T15:11:17.818 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:17.816+0000 7f0433fff640 1 -- 172.21.15.43:0/2481863426 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 118) v1 ==== 88443+0+0 (secure 0 0 0) 0x7f042c00ac70 con 0x7f043c0c90a0 2023-08-09T15:11:17.818 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:17.817+0000 7f0433fff640 1 --2- 172.21.15.43:0/2481863426 >> [v2:172.21.15.203:6832/4034093268,v1:172.21.15.203:6833/4034093268] conn(0x7f041c06a170 0x7f041c06c630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T15:11:17.818 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:17.817+0000 7f0433fff640 1 -- 172.21.15.43:0/2481863426 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(215..215 src has 1..215) v4 ==== 6348+0+0 (secure 0 0 0) 0x7f042c087e90 con 0x7f043c0c90a0 2023-08-09T15:11:17.818 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:17.818+0000 7f043a7fc640 1 --2- 172.21.15.43:0/2481863426 >> [v2:172.21.15.203:6832/4034093268,v1:172.21.15.203:6833/4034093268] conn(0x7f041c06a170 0x7f041c06c630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T15:11:17.819 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:17.819+0000 7f043a7fc640 1 --2- 172.21.15.43:0/2481863426 >> [v2:172.21.15.203:6832/4034093268,v1:172.21.15.203:6833/4034093268] conn(0x7f041c06a170 0x7f041c06c630 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7f0424006fd0 tx=0x7f0424008040 comp rx=0 tx=0).ready entity=mgr.8833 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-09T15:11:17.821 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:17.821+0000 7f0433fff640 1 -- 172.21.15.43:0/2481863426 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178066 (secure 0 0 0) 0x7f043c0c1e60 con 0x7f043c0c90a0 2023-08-09T15:11:18.019 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:18.018+0000 7f043bfff640 1 -- 172.21.15.43:0/2481863426 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"} v 0) v1 -- 0x7f043c158280 con 0x7f043c0c90a0 2023-08-09T15:11:18.025 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:18.025+0000 7f0433fff640 1 -- 172.21.15.43:0/2481863426 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"}]=0 v69) v1 ==== 104+0+0 (secure 0 0 0) 0x7f043c158280 con 0x7f043c0c90a0 2023-08-09T15:11:18.028 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:18.028+0000 7f043bfff640 1 -- 172.21.15.43:0/2481863426 >> [v2:172.21.15.203:6832/4034093268,v1:172.21.15.203:6833/4034093268] conn(0x7f041c06a170 msgr2=0x7f041c06c630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T15:11:18.029 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:18.028+0000 7f043bfff640 1 --2- 172.21.15.43:0/2481863426 >> [v2:172.21.15.203:6832/4034093268,v1:172.21.15.203:6833/4034093268] conn(0x7f041c06a170 0x7f041c06c630 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7f0424006fd0 tx=0x7f0424008040 comp rx=0 tx=0).stop 2023-08-09T15:11:18.029 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:18.028+0000 7f043bfff640 1 -- 172.21.15.43:0/2481863426 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f043c0c90a0 msgr2=0x7f043c150410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T15:11:18.029 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:18.028+0000 7f043bfff640 1 --2- 172.21.15.43:0/2481863426 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f043c0c90a0 0x7f043c150410 secure :-1 s=READY pgs=2343 cs=0 l=1 rev1=1 crypto rx=0x7f042c00c7a0 tx=0x7f042c00cc60 comp rx=0 tx=0).stop 2023-08-09T15:11:18.029 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:18.028+0000 7f043bfff640 1 -- 172.21.15.43:0/2481863426 shutdown_connections 2023-08-09T15:11:18.029 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:18.028+0000 7f043bfff640 1 --2- 172.21.15.43:0/2481863426 >> [v2:172.21.15.203:6832/4034093268,v1:172.21.15.203:6833/4034093268] conn(0x7f041c06a170 0x7f041c06c630 unknown :-1 s=CLOSED pgs=37 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T15:11:18.030 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:18.028+0000 7f043bfff640 1 --2- 172.21.15.43:0/2481863426 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f043c157ea0 0x7f043c0c2140 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T15:11:18.030 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:18.028+0000 7f043bfff640 1 --2- 172.21.15.43:0/2481863426 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f043c0c9ee0 0x7f043c0c2fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T15:11:18.030 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:18.028+0000 7f043bfff640 1 --2- 172.21.15.43:0/2481863426 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f043c0c90a0 0x7f043c150410 unknown :-1 s=CLOSED pgs=2343 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T15:11:18.030 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:18.028+0000 7f043bfff640 1 -- 172.21.15.43:0/2481863426 >> 172.21.15.43:0/2481863426 conn(0x7f043c0cddd0 msgr2=0x7f043c06cb10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-09T15:11:18.030 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:18.028+0000 7f043bfff640 1 -- 172.21.15.43:0/2481863426 shutdown_connections 2023-08-09T15:11:18.031 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:18.028+0000 7f043bfff640 1 -- 172.21.15.43:0/2481863426 wait complete. 2023-08-09T15:11:18.050 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Ended test tasks.cephfs.test_snap_schedules.TestSnapSchedules.test_snap_schedule_with_retention' 2023-08-09T15:11:18.280 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:18.279+0000 7f05b25b7640 1 Processor -- start 2023-08-09T15:11:18.280 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:18.279+0000 7f05b25b7640 1 -- start start 2023-08-09T15:11:18.281 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:18.281+0000 7f05b25b7640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f05ac142ca0 0x7f05ac0c96e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T15:11:18.282 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:18.281+0000 7f05b25b7640 1 -- --> v1:172.21.15.43:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f05ac143080 con 0x7f05ac15ce40 2023-08-09T15:11:18.282 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:18.281+0000 7f05b25b7640 1 -- --> v1:172.21.15.203:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f05ac06d400 con 0x7f05ac0cd510 2023-08-09T15:11:18.282 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:18.281+0000 7f05b25b7640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f05ac0c9d90 con 0x7f05ac142ca0 2023-08-09T15:11:18.283 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:18.282+0000 7f05a37fe640 1 --1- >> v1:172.21.15.43:6789/0 conn(0x7f05ac15ce40 0x7f05ac0cce00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.43:6789/0 says I am v1:172.21.15.43:45560/0 (socket says 172.21.15.43:45560) 2023-08-09T15:11:18.283 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:18.282+0000 7f05a37fe640 1 -- 172.21.15.43:0/3712834723 learned_addr learned my addr 172.21.15.43:0/3712834723 (peer_addr_for_me v1:172.21.15.43:0/0) 2023-08-09T15:11:18.284 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:18.282+0000 7f05abfff640 1 --2- 172.21.15.43:0/3712834723 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f05ac142ca0 0x7f05ac0c96e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T15:11:18.284 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:18.282+0000 7f05ab7fe640 1 -- 172.21.15.43:0/3712834723 <== mon.0 v1:172.21.15.43:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3480895156 0 0) 0x7f05ac143080 con 0x7f05ac15ce40 2023-08-09T15:11:18.284 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:18.282+0000 7f05ab7fe640 1 -- 172.21.15.43:0/3712834723 --> v1:172.21.15.43:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f0598003520 con 0x7f05ac15ce40 2023-08-09T15:11:18.284 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:18.282+0000 7f05ab7fe640 1 -- 172.21.15.43:0/3712834723 <== mon.0 v1:172.21.15.43:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 647172585 0 0) 0x7f0598003520 con 0x7f05ac15ce40 2023-08-09T15:11:18.284 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:18.282+0000 7f05ab7fe640 1 -- 172.21.15.43:0/3712834723 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f05ac142ca0 msgr2=0x7f05ac0c96e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-09T15:11:18.284 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:18.282+0000 7f05ab7fe640 1 --2- 172.21.15.43:0/3712834723 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f05ac142ca0 0x7f05ac0c96e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T15:11:18.284 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:18.283+0000 7f05ab7fe640 1 -- 172.21.15.43:0/3712834723 >> v1:172.21.15.203:6789/0 conn(0x7f05ac0cd510 legacy=0x7f05ac0c3290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T15:11:18.285 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:18.283+0000 7f05ab7fe640 1 -- 172.21.15.43:0/3712834723 --> v1:172.21.15.43:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f05ac0ca070 con 0x7f05ac15ce40 2023-08-09T15:11:18.285 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:18.283+0000 7f05ab7fe640 1 -- 172.21.15.43:0/3712834723 <== mon.0 v1:172.21.15.43:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1285150740 0 0) 0x7f0594001670 con 0x7f05ac15ce40 2023-08-09T15:11:18.285 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:18.283+0000 7f05ab7fe640 1 -- 172.21.15.43:0/3712834723 >> v1:172.21.15.43:6789/0 conn(0x7f05ac15ce40 legacy=0x7f05ac0cce00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T15:11:18.285 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:18.283+0000 7f05ab7fe640 1 --2- 172.21.15.43:0/3712834723 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f0598003a10 0x7f0598003e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T15:11:18.285 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:18.283+0000 7f05ab7fe640 1 --2- 172.21.15.43:0/3712834723 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f05980047c0 0x7f0598004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T15:11:18.285 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:18.283+0000 7f05ab7fe640 1 --2- 172.21.15.43:0/3712834723 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f05ac142ca0 0x7f0598005230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T15:11:18.285 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:18.283+0000 7f05ab7fe640 1 -- 172.21.15.43:0/3712834723 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f05ac0c9d90 con 0x7f05980047c0 2023-08-09T15:11:18.286 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:18.283+0000 7f05ab7fe640 1 -- 172.21.15.43:0/3712834723 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f059800bb40 con 0x7f0598003a10 2023-08-09T15:11:18.286 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:18.283+0000 7f05ab7fe640 1 -- 172.21.15.43:0/3712834723 --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f059800bd00 con 0x7f05ac142ca0 2023-08-09T15:11:18.286 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:18.283+0000 7f05a37fe640 1 --2- 172.21.15.43:0/3712834723 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f05980047c0 0x7f0598004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T15:11:18.288 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:18.283+0000 7f05abfff640 1 --2- 172.21.15.43:0/3712834723 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f0598003a10 0x7f0598003e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T15:11:18.288 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:18.283+0000 7f05b0b54640 1 --2- 172.21.15.43:0/3712834723 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f05ac142ca0 0x7f0598005230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T15:11:18.288 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:18.283+0000 7f05a37fe640 1 -- 172.21.15.43:0/3712834723 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f05ac142ca0 msgr2=0x7f0598005230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-09T15:11:18.288 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:18.284+0000 7f05a37fe640 1 --2- 172.21.15.43:0/3712834723 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f05ac142ca0 0x7f0598005230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T15:11:18.288 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:18.284+0000 7f05a37fe640 1 -- 172.21.15.43:0/3712834723 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f0598003a10 msgr2=0x7f0598003e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-09T15:11:18.289 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:18.284+0000 7f05a37fe640 1 --2- 172.21.15.43:0/3712834723 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f0598003a10 0x7f0598003e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T15:11:18.289 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:18.284+0000 7f05a37fe640 1 -- 172.21.15.43:0/3712834723 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0598003520 con 0x7f05980047c0 2023-08-09T15:11:18.289 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:18.284+0000 7f05abfff640 1 --2- 172.21.15.43:0/3712834723 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f0598003a10 0x7f0598003e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-08-09T15:11:18.289 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:18.285+0000 7f05a37fe640 1 --2- 172.21.15.43:0/3712834723 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f05980047c0 0x7f0598004c40 secure :-1 s=READY pgs=2345 cs=0 l=1 rev1=1 crypto rx=0x7f059400c240 tx=0x7f05980057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=63af7a6b9c601412 server_cookie=0 in_seq=0 out_seq=0 2023-08-09T15:11:18.289 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:18.285+0000 7f05ab7fe640 1 -- 172.21.15.43:0/3712834723 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0594005ad0 con 0x7f05980047c0 2023-08-09T15:11:18.289 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:18.285+0000 7f05ab7fe640 1 -- 172.21.15.43:0/3712834723 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0594005c30 con 0x7f05980047c0 2023-08-09T15:11:18.289 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:18.285+0000 7f05ab7fe640 1 -- 172.21.15.43:0/3712834723 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f059402c8b0 con 0x7f05980047c0 2023-08-09T15:11:18.290 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:18.286+0000 7f05b25b7640 1 -- 172.21.15.43:0/3712834723 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f05980047c0 msgr2=0x7f0598004c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T15:11:18.290 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:18.286+0000 7f05b25b7640 1 --2- 172.21.15.43:0/3712834723 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f05980047c0 0x7f0598004c40 secure :-1 s=READY pgs=2345 cs=0 l=1 rev1=1 crypto rx=0x7f059400c240 tx=0x7f05980057f0 comp rx=0 tx=0).stop 2023-08-09T15:11:18.290 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:18.286+0000 7f05b0b54640 1 -- 172.21.15.43:0/3712834723 reap_dead start 2023-08-09T15:11:18.290 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:18.286+0000 7f05b25b7640 1 -- 172.21.15.43:0/3712834723 shutdown_connections 2023-08-09T15:11:18.290 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:18.286+0000 7f05b25b7640 1 -- 172.21.15.43:0/3712834723 >> 172.21.15.43:0/3712834723 conn(0x7f05ac0ce0a0 msgr2=0x7f05ac0ce480 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-09T15:11:18.290 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:18.286+0000 7f05b25b7640 1 -- 172.21.15.43:0/3712834723 shutdown_connections 2023-08-09T15:11:18.290 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:18.286+0000 7f05b25b7640 1 -- 172.21.15.43:0/3712834723 wait complete. 2023-08-09T15:11:18.291 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:18.288+0000 7f05b25b7640 1 Processor -- start 2023-08-09T15:11:18.291 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:18.288+0000 7f05b25b7640 1 -- start start 2023-08-09T15:11:18.293 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:18.289+0000 7f05b25b7640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f05980047c0 0x7f05ac0c6b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T15:11:18.293 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:18.289+0000 7f05b25b7640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f05ac158be0 0x7f05ac1668e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T15:11:18.293 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:18.289+0000 7f05b25b7640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f05ac0caf70 0x7f05ac155a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T15:11:18.294 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:18.289+0000 7f05b25b7640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f05ac166450 con 0x7f05ac158be0 2023-08-09T15:11:18.294 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:18.289+0000 7f05b25b7640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f05ac0c7490 con 0x7f05980047c0 2023-08-09T15:11:18.294 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:18.289+0000 7f05b25b7640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f05ac165b20 con 0x7f05ac0caf70 2023-08-09T15:11:18.295 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:18.289+0000 7f05a37fe640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f05ac158be0 0x7f05ac1668e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T15:11:18.295 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:18.289+0000 7f05a37fe640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f05ac158be0 0x7f05ac1668e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:37776/0 (socket says 172.21.15.43:37776) 2023-08-09T15:11:18.295 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:18.289+0000 7f05a37fe640 1 -- 172.21.15.43:0/2377051059 learned_addr learned my addr 172.21.15.43:0/2377051059 (peer_addr_for_me v2:172.21.15.43:0/0) 2023-08-09T15:11:18.295 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:18.290+0000 7f05a37fe640 1 -- 172.21.15.43:0/2377051059 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f05ac0caf70 msgr2=0x7f05ac155a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T15:11:18.296 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:18.290+0000 7f05b0b54640 1 --2- 172.21.15.43:0/2377051059 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f05ac0caf70 0x7f05ac155a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T15:11:18.296 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:18.290+0000 7f05abfff640 1 --2- 172.21.15.43:0/2377051059 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f05980047c0 0x7f05ac0c6b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T15:11:18.296 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:18.290+0000 7f05a37fe640 1 --2- 172.21.15.43:0/2377051059 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f05ac0caf70 0x7f05ac155a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T15:11:18.296 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:18.290+0000 7f05a37fe640 1 -- 172.21.15.43:0/2377051059 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f05980047c0 msgr2=0x7f05ac0c6b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T15:11:18.297 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:18.290+0000 7f05a37fe640 1 --2- 172.21.15.43:0/2377051059 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f05980047c0 0x7f05ac0c6b90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T15:11:18.297 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:18.290+0000 7f05a37fe640 1 -- 172.21.15.43:0/2377051059 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f05ac165e00 con 0x7f05ac158be0 2023-08-09T15:11:18.297 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:18.290+0000 7f05b0b54640 1 --2- 172.21.15.43:0/2377051059 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f05ac0caf70 0x7f05ac155a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-08-09T15:11:18.297 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:18.290+0000 7f05abfff640 1 --2- 172.21.15.43:0/2377051059 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f05980047c0 0x7f05ac0c6b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-08-09T15:11:18.297 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:18.290+0000 7f05a37fe640 1 --2- 172.21.15.43:0/2377051059 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f05ac158be0 0x7f05ac1668e0 secure :-1 s=READY pgs=2346 cs=0 l=1 rev1=1 crypto rx=0x7f059402c450 tx=0x7f0594001a00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-09T15:11:18.297 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:18.290+0000 7f05a97fa640 1 -- 172.21.15.43:0/2377051059 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0594039070 con 0x7f05ac158be0 2023-08-09T15:11:18.297 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:18.290+0000 7f05a97fa640 1 -- 172.21.15.43:0/2377051059 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f059402dd50 con 0x7f05ac158be0 2023-08-09T15:11:18.298 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:18.290+0000 7f05a97fa640 1 -- 172.21.15.43:0/2377051059 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f059403c040 con 0x7f05ac158be0 2023-08-09T15:11:18.298 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:18.290+0000 7f05b25b7640 1 -- 172.21.15.43:0/2377051059 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f05ac1660e0 con 0x7f05ac158be0 2023-08-09T15:11:18.298 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:18.291+0000 7f05b25b7640 1 -- 172.21.15.43:0/2377051059 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f05ac166e20 con 0x7f05ac158be0 2023-08-09T15:11:18.298 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:18.292+0000 7f05b25b7640 1 -- 172.21.15.43:0/2377051059 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f05ac167010 con 0x7f05ac158be0 2023-08-09T15:11:18.298 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:18.293+0000 7f05a97fa640 1 -- 172.21.15.43:0/2377051059 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 118) v1 ==== 88443+0+0 (secure 0 0 0) 0x7f0594032070 con 0x7f05ac158be0 2023-08-09T15:11:18.299 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:18.294+0000 7f05a97fa640 1 --2- 172.21.15.43:0/2377051059 >> [v2:172.21.15.203:6832/4034093268,v1:172.21.15.203:6833/4034093268] conn(0x7f058c06a1a0 0x7f058c06c660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T15:11:18.299 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:18.294+0000 7f05a97fa640 1 -- 172.21.15.43:0/2377051059 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(215..215 src has 1..215) v4 ==== 6348+0+0 (secure 0 0 0) 0x7f05940a9a20 con 0x7f05ac158be0 2023-08-09T15:11:18.299 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:18.294+0000 7f05abfff640 1 --2- 172.21.15.43:0/2377051059 >> [v2:172.21.15.203:6832/4034093268,v1:172.21.15.203:6833/4034093268] conn(0x7f058c06a1a0 0x7f058c06c660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T15:11:18.299 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:18.296+0000 7f05abfff640 1 --2- 172.21.15.43:0/2377051059 >> [v2:172.21.15.203:6832/4034093268,v1:172.21.15.203:6833/4034093268] conn(0x7f058c06a1a0 0x7f058c06c660 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7f059c009540 tx=0x7f059c00b040 comp rx=0 tx=0).ready entity=mgr.8833 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-09T15:11:18.300 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:18.298+0000 7f05a97fa640 1 -- 172.21.15.43:0/2377051059 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178066 (secure 0 0 0) 0x7f0594074ff0 con 0x7f05ac158be0 2023-08-09T15:11:18.569 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:18.568+0000 7f05b25b7640 1 -- 172.21.15.43:0/2377051059 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_snap_schedules.TestSnapSchedules.test_snap_schedule_with_retention"]} v 0) v1 -- 0x7f05ac0739f0 con 0x7f05ac158be0 2023-08-09T15:11:18.589 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:18.588+0000 7f05a97fa640 1 -- 172.21.15.43:0/2377051059 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_snap_schedules.TestSnapSchedules.test_snap_schedule_with_retention"]}]=0 v3201) v1 ==== 163+0+0 (secure 0 0 0) 0x7f0594046020 con 0x7f05ac158be0 2023-08-09T15:11:18.591 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:18.591+0000 7f05b25b7640 1 -- 172.21.15.43:0/2377051059 >> [v2:172.21.15.203:6832/4034093268,v1:172.21.15.203:6833/4034093268] conn(0x7f058c06a1a0 msgr2=0x7f058c06c660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T15:11:18.592 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:18.591+0000 7f05b25b7640 1 --2- 172.21.15.43:0/2377051059 >> [v2:172.21.15.203:6832/4034093268,v1:172.21.15.203:6833/4034093268] conn(0x7f058c06a1a0 0x7f058c06c660 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7f059c009540 tx=0x7f059c00b040 comp rx=0 tx=0).stop 2023-08-09T15:11:18.592 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:18.591+0000 7f05b25b7640 1 -- 172.21.15.43:0/2377051059 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f05ac158be0 msgr2=0x7f05ac1668e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T15:11:18.592 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:18.591+0000 7f05b25b7640 1 --2- 172.21.15.43:0/2377051059 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f05ac158be0 0x7f05ac1668e0 secure :-1 s=READY pgs=2346 cs=0 l=1 rev1=1 crypto rx=0x7f059402c450 tx=0x7f0594001a00 comp rx=0 tx=0).stop 2023-08-09T15:11:18.592 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:18.591+0000 7f05b25b7640 1 -- 172.21.15.43:0/2377051059 shutdown_connections 2023-08-09T15:11:18.592 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:18.591+0000 7f05b25b7640 1 --2- 172.21.15.43:0/2377051059 >> [v2:172.21.15.203:6832/4034093268,v1:172.21.15.203:6833/4034093268] conn(0x7f058c06a1a0 0x7f058c06c660 unknown :-1 s=CLOSED pgs=38 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T15:11:18.592 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:18.591+0000 7f05b25b7640 1 --2- 172.21.15.43:0/2377051059 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f05ac0caf70 0x7f05ac155a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T15:11:18.592 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:18.591+0000 7f05b25b7640 1 --2- 172.21.15.43:0/2377051059 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f05ac158be0 0x7f05ac1668e0 unknown :-1 s=CLOSED pgs=2346 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T15:11:18.593 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:18.591+0000 7f05b25b7640 1 --2- 172.21.15.43:0/2377051059 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f05980047c0 0x7f05ac0c6b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T15:11:18.593 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:18.591+0000 7f05b25b7640 1 -- 172.21.15.43:0/2377051059 >> 172.21.15.43:0/2377051059 conn(0x7f05ac0ce0a0 msgr2=0x7f05ac0c3a50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-09T15:11:18.593 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:18.591+0000 7f05b25b7640 1 -- 172.21.15.43:0/2377051059 shutdown_connections 2023-08-09T15:11:18.593 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:18.592+0000 7f05b25b7640 1 -- 172.21.15.43:0/2377051059 wait complete. 2023-08-09T15:11:18.612 INFO:tasks.cephfs_test_runner:Test scheduled snapshots along with rentention policy ... ok 2023-08-09T15:11:18.613 INFO:tasks.cephfs_test_runner:Starting test: test_snap_dir_name (tasks.cephfs.test_snap_schedules.TestSnapSchedulesSnapdir) Test the correctness of snap directory name 2023-08-09T15:11:18.613 INFO:tasks.cephfs_test_runner:test_snap_dir_name (tasks.cephfs.test_snap_schedules.TestSnapSchedulesSnapdir) 2023-08-09T15:11:18.613 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Starting test tasks.cephfs.test_snap_schedules.TestSnapSchedulesSnapdir.test_snap_dir_name' 2023-08-09T15:11:18.842 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:18.842+0000 7efe2095f640 1 Processor -- start 2023-08-09T15:11:18.843 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:18.842+0000 7efe2095f640 1 -- start start 2023-08-09T15:11:18.844 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:18.844+0000 7efe2095f640 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7efe1c06cfe0 0x7efe1c0c8b50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T15:11:18.844 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:18.844+0000 7efe2095f640 1 -- --> v1:172.21.15.43:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7efe1c0c9090 con 0x7efe1c143c20 2023-08-09T15:11:18.844 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:18.844+0000 7efe2095f640 1 -- --> v1:172.21.15.203:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7efe1c06c440 con 0x7efe1c0c86f0 2023-08-09T15:11:18.845 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:18.844+0000 7efe2095f640 1 -- --> v2:172.21.15.203:3300/0 -- mon_getmap magic: 0 v1 -- 0x7efe1c144000 con 0x7efe1c06cfe0 2023-08-09T15:11:18.845 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:18.844+0000 7efe1a59c640 1 --1- >> v1:172.21.15.43:6789/0 conn(0x7efe1c143c20 0x7efe1c06f540 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.43:6789/0 says I am v1:172.21.15.43:45570/0 (socket says 172.21.15.43:45570) 2023-08-09T15:11:18.845 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:18.844+0000 7efe1a59c640 1 -- 172.21.15.43:0/2761934298 learned_addr learned my addr 172.21.15.43:0/2761934298 (peer_addr_for_me v1:172.21.15.43:0/0) 2023-08-09T15:11:18.846 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:18.845+0000 7efe1ad9d640 1 --2- 172.21.15.43:0/2761934298 >> v2:172.21.15.203:3300/0 conn(0x7efe1c06cfe0 0x7efe1c0c8b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T15:11:18.846 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:18.845+0000 7efe19d9b640 1 -- 172.21.15.43:0/2761934298 <== mon.0 v1:172.21.15.43:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 593825366 0 0) 0x7efe1c0c9090 con 0x7efe1c143c20 2023-08-09T15:11:18.846 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:18.845+0000 7efe19d9b640 1 -- 172.21.15.43:0/2761934298 --> v1:172.21.15.43:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7efe08003520 con 0x7efe1c143c20 2023-08-09T15:11:18.847 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:18.845+0000 7efe19d9b640 1 -- 172.21.15.43:0/2761934298 <== mon.0 v1:172.21.15.43:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 421278422 0 0) 0x7efe08003520 con 0x7efe1c143c20 2023-08-09T15:11:18.847 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:18.845+0000 7efe19d9b640 1 -- 172.21.15.43:0/2761934298 >> v2:172.21.15.203:3300/0 conn(0x7efe1c06cfe0 msgr2=0x7efe1c0c8b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-09T15:11:18.847 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:18.845+0000 7efe19d9b640 1 --2- 172.21.15.43:0/2761934298 >> v2:172.21.15.203:3300/0 conn(0x7efe1c06cfe0 0x7efe1c0c8b50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T15:11:18.847 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:18.846+0000 7efe19d9b640 1 -- 172.21.15.43:0/2761934298 >> v1:172.21.15.203:6789/0 conn(0x7efe1c0c86f0 legacy=0x7efe1c06c8d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T15:11:18.848 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:18.846+0000 7efe19d9b640 1 -- 172.21.15.43:0/2761934298 --> v1:172.21.15.43:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efe1c06fc50 con 0x7efe1c143c20 2023-08-09T15:11:18.848 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:18.846+0000 7efe19d9b640 1 -- 172.21.15.43:0/2761934298 <== mon.0 v1:172.21.15.43:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1285150740 0 0) 0x7efe0c001670 con 0x7efe1c143c20 2023-08-09T15:11:18.848 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:18.846+0000 7efe19d9b640 1 -- 172.21.15.43:0/2761934298 >> v1:172.21.15.43:6789/0 conn(0x7efe1c143c20 legacy=0x7efe1c06f540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T15:11:18.848 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:18.846+0000 7efe19d9b640 1 --2- 172.21.15.43:0/2761934298 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7efe080039e0 0x7efe08003e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T15:11:18.848 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:18.846+0000 7efe19d9b640 1 --2- 172.21.15.43:0/2761934298 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7efe080047c0 0x7efe08004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T15:11:18.848 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:18.846+0000 7efe19d9b640 1 --2- 172.21.15.43:0/2761934298 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7efe08005280 0x7efe0800b980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T15:11:18.848 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:18.846+0000 7efe19d9b640 1 -- 172.21.15.43:0/2761934298 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efe1c144000 con 0x7efe080039e0 2023-08-09T15:11:18.849 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:18.846+0000 7efe19d9b640 1 -- 172.21.15.43:0/2761934298 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efe0800c1c0 con 0x7efe08005280 2023-08-09T15:11:18.849 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:18.846+0000 7efe19d9b640 1 -- 172.21.15.43:0/2761934298 --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7efe0800c380 con 0x7efe080047c0 2023-08-09T15:11:18.849 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:18.846+0000 7efe1a59c640 1 --2- 172.21.15.43:0/2761934298 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7efe080039e0 0x7efe08003e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T15:11:18.849 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:18.846+0000 7efe1a59c640 1 -- 172.21.15.43:0/2761934298 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7efe080047c0 msgr2=0x7efe08004c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-09T15:11:18.849 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:18.846+0000 7efe1a59c640 1 --2- 172.21.15.43:0/2761934298 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7efe080047c0 0x7efe08004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T15:11:18.849 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:18.846+0000 7efe1a59c640 1 -- 172.21.15.43:0/2761934298 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7efe08005280 msgr2=0x7efe0800b980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-09T15:11:18.849 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:18.846+0000 7efe1a59c640 1 --2- 172.21.15.43:0/2761934298 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7efe08005280 0x7efe0800b980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T15:11:18.849 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:18.846+0000 7efe1a59c640 1 -- 172.21.15.43:0/2761934298 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efe08003520 con 0x7efe080039e0 2023-08-09T15:11:18.850 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:18.846+0000 7efe1ad9d640 1 -- 172.21.15.43:0/2761934298 reap_dead start 2023-08-09T15:11:18.850 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:18.848+0000 7efe1a59c640 1 --2- 172.21.15.43:0/2761934298 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7efe080039e0 0x7efe08003e00 secure :-1 s=READY pgs=2348 cs=0 l=1 rev1=1 crypto rx=0x7efe0c00c240 tx=0x7efe080057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=501f38d5a31d06ff server_cookie=0 in_seq=0 out_seq=0 2023-08-09T15:11:18.851 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:18.848+0000 7efe19d9b640 1 -- 172.21.15.43:0/2761934298 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efe0c005ad0 con 0x7efe080039e0 2023-08-09T15:11:18.851 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:18.848+0000 7efe19d9b640 1 -- 172.21.15.43:0/2761934298 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efe0c005c30 con 0x7efe080039e0 2023-08-09T15:11:18.851 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:18.848+0000 7efe19d9b640 1 -- 172.21.15.43:0/2761934298 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efe0c02caf0 con 0x7efe080039e0 2023-08-09T15:11:18.851 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:18.848+0000 7efe2095f640 1 -- 172.21.15.43:0/2761934298 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7efe080039e0 msgr2=0x7efe08003e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T15:11:18.851 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:18.848+0000 7efe2095f640 1 --2- 172.21.15.43:0/2761934298 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7efe080039e0 0x7efe08003e00 secure :-1 s=READY pgs=2348 cs=0 l=1 rev1=1 crypto rx=0x7efe0c00c240 tx=0x7efe080057f0 comp rx=0 tx=0).stop 2023-08-09T15:11:18.851 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:18.849+0000 7efe2095f640 1 -- 172.21.15.43:0/2761934298 shutdown_connections 2023-08-09T15:11:18.852 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:18.849+0000 7efe2095f640 1 --2- 172.21.15.43:0/2761934298 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7efe080039e0 0x7efe08003e00 unknown :-1 s=CLOSED pgs=2348 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T15:11:18.852 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:18.849+0000 7efe2095f640 1 -- 172.21.15.43:0/2761934298 >> 172.21.15.43:0/2761934298 conn(0x7efe1c0ce0a0 msgr2=0x7efe1c0ce480 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-09T15:11:18.852 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:18.849+0000 7efe2095f640 1 -- 172.21.15.43:0/2761934298 shutdown_connections 2023-08-09T15:11:18.852 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:18.849+0000 7efe2095f640 1 -- 172.21.15.43:0/2761934298 wait complete. 2023-08-09T15:11:18.852 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:18.850+0000 7efe2095f640 1 Processor -- start 2023-08-09T15:11:18.854 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:18.851+0000 7efe2095f640 1 -- start start 2023-08-09T15:11:18.854 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:18.852+0000 7efe2095f640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7efe080039e0 0x7efe1c0b2000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T15:11:18.854 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:18.852+0000 7efe2095f640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7efe1c0b0ea0 0x7efe1c159d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T15:11:18.854 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:18.852+0000 7efe2095f640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7efe1c15ba20 0x7efe1c0b3190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T15:11:18.854 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:18.852+0000 7efe2095f640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efe1c15a2c0 con 0x7efe1c0b0ea0 2023-08-09T15:11:18.854 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:18.852+0000 7efe2095f640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efe1c0b2610 con 0x7efe080039e0 2023-08-09T15:11:18.854 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:18.852+0000 7efe2095f640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7efe1c0c6ad0 con 0x7efe1c15ba20 2023-08-09T15:11:18.855 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:18.852+0000 7efe13fff640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7efe1c0b0ea0 0x7efe1c159d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T15:11:18.855 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:18.852+0000 7efe13fff640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7efe1c0b0ea0 0x7efe1c159d80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:37800/0 (socket says 172.21.15.43:37800) 2023-08-09T15:11:18.855 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:18.852+0000 7efe13fff640 1 -- 172.21.15.43:0/4011221084 learned_addr learned my addr 172.21.15.43:0/4011221084 (peer_addr_for_me v2:172.21.15.43:0/0) 2023-08-09T15:11:18.855 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:18.852+0000 7efe1a59c640 1 --2- 172.21.15.43:0/4011221084 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7efe080039e0 0x7efe1c0b2000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T15:11:18.855 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:18.852+0000 7efe1ad9d640 1 --2- 172.21.15.43:0/4011221084 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7efe1c15ba20 0x7efe1c0b3190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T15:11:18.855 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:18.852+0000 7efe13fff640 1 -- 172.21.15.43:0/4011221084 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7efe1c15ba20 msgr2=0x7efe1c0b3190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T15:11:18.855 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:18.852+0000 7efe13fff640 1 --2- 172.21.15.43:0/4011221084 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7efe1c15ba20 0x7efe1c0b3190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T15:11:18.856 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:18.852+0000 7efe13fff640 1 -- 172.21.15.43:0/4011221084 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7efe080039e0 msgr2=0x7efe1c0b2000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T15:11:18.856 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:18.852+0000 7efe13fff640 1 --2- 172.21.15.43:0/4011221084 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7efe080039e0 0x7efe1c0b2000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T15:11:18.856 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:18.852+0000 7efe13fff640 1 -- 172.21.15.43:0/4011221084 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efe1c0c6db0 con 0x7efe1c0b0ea0 2023-08-09T15:11:18.856 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:18.853+0000 7efe13fff640 1 --2- 172.21.15.43:0/4011221084 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7efe1c0b0ea0 0x7efe1c159d80 secure :-1 s=READY pgs=2349 cs=0 l=1 rev1=1 crypto rx=0x7efe0400c390 tx=0x7efe0400c850 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-09T15:11:18.856 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:18.853+0000 7efe12ffd640 1 -- 172.21.15.43:0/4011221084 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efe040052d0 con 0x7efe1c0b0ea0 2023-08-09T15:11:18.856 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:18.853+0000 7efe12ffd640 1 -- 172.21.15.43:0/4011221084 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efe04005450 con 0x7efe1c0b0ea0 2023-08-09T15:11:18.856 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:18.853+0000 7efe12ffd640 1 -- 172.21.15.43:0/4011221084 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efe04007260 con 0x7efe1c0b0ea0 2023-08-09T15:11:18.857 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:18.853+0000 7efe2095f640 1 -- 172.21.15.43:0/4011221084 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7efe1c0c7090 con 0x7efe1c0b0ea0 2023-08-09T15:11:18.857 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:18.853+0000 7efe2095f640 1 -- 172.21.15.43:0/4011221084 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7efe1c0c5370 con 0x7efe1c0b0ea0 2023-08-09T15:11:18.857 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:18.854+0000 7efe2095f640 1 -- 172.21.15.43:0/4011221084 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7efe1c0c5530 con 0x7efe1c0b0ea0 2023-08-09T15:11:18.858 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:18.855+0000 7efe12ffd640 1 -- 172.21.15.43:0/4011221084 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 118) v1 ==== 88443+0+0 (secure 0 0 0) 0x7efe040073c0 con 0x7efe1c0b0ea0 2023-08-09T15:11:18.858 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:18.856+0000 7efe12ffd640 1 --2- 172.21.15.43:0/4011221084 >> [v2:172.21.15.203:6832/4034093268,v1:172.21.15.203:6833/4034093268] conn(0x7efdfc06a1a0 0x7efdfc06c660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T15:11:18.858 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:18.856+0000 7efe12ffd640 1 -- 172.21.15.43:0/4011221084 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(215..215 src has 1..215) v4 ==== 6348+0+0 (secure 0 0 0) 0x7efe04087c40 con 0x7efe1c0b0ea0 2023-08-09T15:11:18.858 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:18.857+0000 7efe1a59c640 1 --2- 172.21.15.43:0/4011221084 >> [v2:172.21.15.203:6832/4034093268,v1:172.21.15.203:6833/4034093268] conn(0x7efdfc06a1a0 0x7efdfc06c660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T15:11:18.859 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:18.859+0000 7efe1a59c640 1 --2- 172.21.15.43:0/4011221084 >> [v2:172.21.15.203:6832/4034093268,v1:172.21.15.203:6833/4034093268] conn(0x7efdfc06a1a0 0x7efdfc06c660 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7efe0c034000 tx=0x7efe0c009030 comp rx=0 tx=0).ready entity=mgr.8833 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-09T15:11:18.861 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:18.861+0000 7efe12ffd640 1 -- 172.21.15.43:0/4011221084 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178066 (secure 0 0 0) 0x7efe1c0c5530 con 0x7efe1c0b0ea0 2023-08-09T15:11:19.124 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:19.124+0000 7efe2095f640 1 -- 172.21.15.43:0/4011221084 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_snap_schedules.TestSnapSchedulesSnapdir.test_snap_dir_name"]} v 0) v1 -- 0x7efe1c0c5500 con 0x7efe1c0b0ea0 2023-08-09T15:11:19.595 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:19.594+0000 7efe12ffd640 1 -- 172.21.15.43:0/4011221084 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_snap_schedules.TestSnapSchedulesSnapdir.test_snap_dir_name"]}]=0 v3202) v1 ==== 158+0+0 (secure 0 0 0) 0x7efe04053060 con 0x7efe1c0b0ea0 2023-08-09T15:11:19.599 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:19.598+0000 7efe2095f640 1 -- 172.21.15.43:0/4011221084 >> [v2:172.21.15.203:6832/4034093268,v1:172.21.15.203:6833/4034093268] conn(0x7efdfc06a1a0 msgr2=0x7efdfc06c660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T15:11:19.599 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:19.598+0000 7efe2095f640 1 --2- 172.21.15.43:0/4011221084 >> [v2:172.21.15.203:6832/4034093268,v1:172.21.15.203:6833/4034093268] conn(0x7efdfc06a1a0 0x7efdfc06c660 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7efe0c034000 tx=0x7efe0c009030 comp rx=0 tx=0).stop 2023-08-09T15:11:19.599 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:19.598+0000 7efe2095f640 1 -- 172.21.15.43:0/4011221084 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7efe1c0b0ea0 msgr2=0x7efe1c159d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T15:11:19.599 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:19.598+0000 7efe2095f640 1 --2- 172.21.15.43:0/4011221084 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7efe1c0b0ea0 0x7efe1c159d80 secure :-1 s=READY pgs=2349 cs=0 l=1 rev1=1 crypto rx=0x7efe0400c390 tx=0x7efe0400c850 comp rx=0 tx=0).stop 2023-08-09T15:11:19.600 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:19.598+0000 7efe2095f640 1 -- 172.21.15.43:0/4011221084 shutdown_connections 2023-08-09T15:11:19.600 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:19.598+0000 7efe2095f640 1 --2- 172.21.15.43:0/4011221084 >> [v2:172.21.15.203:6832/4034093268,v1:172.21.15.203:6833/4034093268] conn(0x7efdfc06a1a0 0x7efdfc06c660 unknown :-1 s=CLOSED pgs=39 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T15:11:19.600 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:19.598+0000 7efe2095f640 1 --2- 172.21.15.43:0/4011221084 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7efe1c15ba20 0x7efe1c0b3190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T15:11:19.600 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:19.599+0000 7efe2095f640 1 --2- 172.21.15.43:0/4011221084 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7efe1c0b0ea0 0x7efe1c159d80 unknown :-1 s=CLOSED pgs=2349 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T15:11:19.600 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:19.599+0000 7efe2095f640 1 --2- 172.21.15.43:0/4011221084 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7efe080039e0 0x7efe1c0b2000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T15:11:19.600 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:19.599+0000 7efe2095f640 1 -- 172.21.15.43:0/4011221084 >> 172.21.15.43:0/4011221084 conn(0x7efe1c0ce0a0 msgr2=0x7efe1c070f10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-09T15:11:19.601 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:19.599+0000 7efe2095f640 1 -- 172.21.15.43:0/4011221084 shutdown_connections 2023-08-09T15:11:19.601 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:19.599+0000 7efe2095f640 1 -- 172.21.15.43:0/4011221084 wait complete. 2023-08-09T15:11:19.621 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-08-09T15:11:19.856 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:19.856+0000 7f1a61574640 1 Processor -- start 2023-08-09T15:11:19.856 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:19.856+0000 7f1a61574640 1 -- start start 2023-08-09T15:11:19.858 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:19.858+0000 7f1a61574640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f1a5c0d0ef0 0x7f1a5c0cfe70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T15:11:19.858 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:19.858+0000 7f1a61574640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f1a5c0cf4f0 0x7f1a5c0b27c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T15:11:19.858 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:19.858+0000 7f1a61574640 1 -- --> v1:172.21.15.203:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1a5c0b2330 con 0x7f1a5c15c620 2023-08-09T15:11:19.858 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:19.858+0000 7f1a61574640 1 -- --> v2:172.21.15.43:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1a5c0d2ec0 con 0x7f1a5c0cf4f0 2023-08-09T15:11:19.859 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:19.858+0000 7f1a61574640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1a5c0d0500 con 0x7f1a5c0d0ef0 2023-08-09T15:11:19.859 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:19.858+0000 7f1a60d73640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f1a5c0cf4f0 0x7f1a5c0b27c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T15:11:19.860 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:19.858+0000 7f1a60d73640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f1a5c0cf4f0 0x7f1a5c0b27c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:37812/0 (socket says 172.21.15.43:37812) 2023-08-09T15:11:19.860 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:19.858+0000 7f1a60d73640 1 -- 172.21.15.43:0/1189583212 learned_addr learned my addr 172.21.15.43:0/1189583212 (peer_addr_for_me v2:172.21.15.43:0/0) 2023-08-09T15:11:19.860 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:19.859+0000 7f1a5b7fe640 1 --2- 172.21.15.43:0/1189583212 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f1a5c0d0ef0 0x7f1a5c0cfe70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T15:11:19.860 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:19.859+0000 7f1a60d73640 1 -- 172.21.15.43:0/1189583212 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f1a5c0d0ef0 msgr2=0x7f1a5c0cfe70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-09T15:11:19.860 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:19.859+0000 7f1a60d73640 1 --2- 172.21.15.43:0/1189583212 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f1a5c0d0ef0 0x7f1a5c0cfe70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T15:11:19.860 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:19.859+0000 7f1a60d73640 1 -- 172.21.15.43:0/1189583212 >> v1:172.21.15.203:6789/0 conn(0x7f1a5c15c620 legacy=0x7f1a5c0d07e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-09T15:11:19.860 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:19.859+0000 7f1a60d73640 1 -- 172.21.15.43:0/1189583212 --> v2:172.21.15.43:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1a5c0b1a00 con 0x7f1a5c0cf4f0 2023-08-09T15:11:19.861 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:19.859+0000 7f1a5b7fe640 1 --2- 172.21.15.43:0/1189583212 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f1a5c0d0ef0 0x7f1a5c0cfe70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-08-09T15:11:19.861 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:19.860+0000 7f1a60d73640 1 --2- 172.21.15.43:0/1189583212 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f1a5c0cf4f0 0x7f1a5c0b27c0 secure :-1 s=READY pgs=2350 cs=0 l=1 rev1=1 crypto rx=0x7f1a54009870 tx=0x7f1a5402b860 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2d56ba7bfc800da6 server_cookie=0 in_seq=0 out_seq=0 2023-08-09T15:11:19.862 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:19.860+0000 7f1a5affd640 1 -- 172.21.15.43:0/1189583212 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1a540317c0 con 0x7f1a5c0cf4f0 2023-08-09T15:11:19.862 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:19.860+0000 7f1a5affd640 1 -- 172.21.15.43:0/1189583212 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1a54031920 con 0x7f1a5c0cf4f0 2023-08-09T15:11:19.862 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:19.860+0000 7f1a5affd640 1 -- 172.21.15.43:0/1189583212 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1a54031cb0 con 0x7f1a5c0cf4f0 2023-08-09T15:11:19.862 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:19.861+0000 7f1a61574640 1 -- 172.21.15.43:0/1189583212 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f1a5c0cf4f0 msgr2=0x7f1a5c0b27c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T15:11:19.862 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:19.861+0000 7f1a61574640 1 --2- 172.21.15.43:0/1189583212 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f1a5c0cf4f0 0x7f1a5c0b27c0 secure :-1 s=READY pgs=2350 cs=0 l=1 rev1=1 crypto rx=0x7f1a54009870 tx=0x7f1a5402b860 comp rx=0 tx=0).stop 2023-08-09T15:11:19.862 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:19.861+0000 7f1a61574640 1 -- 172.21.15.43:0/1189583212 shutdown_connections 2023-08-09T15:11:19.862 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:19.861+0000 7f1a61574640 1 --2- 172.21.15.43:0/1189583212 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f1a5c0cf4f0 0x7f1a5c0b27c0 unknown :-1 s=CLOSED pgs=2350 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T15:11:19.863 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:19.861+0000 7f1a61574640 1 --2- 172.21.15.43:0/1189583212 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f1a5c0d0ef0 0x7f1a5c0cfe70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T15:11:19.863 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:19.861+0000 7f1a61574640 1 -- 172.21.15.43:0/1189583212 >> 172.21.15.43:0/1189583212 conn(0x7f1a5c09b650 msgr2=0x7f1a5c0d22a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-09T15:11:19.863 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:19.861+0000 7f1a61574640 1 -- 172.21.15.43:0/1189583212 shutdown_connections 2023-08-09T15:11:19.863 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:19.861+0000 7f1a61574640 1 -- 172.21.15.43:0/1189583212 wait complete. 2023-08-09T15:11:19.863 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:19.863+0000 7f1a61574640 1 Processor -- start 2023-08-09T15:11:19.864 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:19.863+0000 7f1a61574640 1 -- start start 2023-08-09T15:11:19.865 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:19.864+0000 7f1a61574640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f1a5c0cf4f0 0x7f1a5c0cbe00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T15:11:19.865 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:19.864+0000 7f1a61574640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f1a5c0d0ef0 0x7f1a5c0caf20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T15:11:19.865 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:19.864+0000 7f1a61574640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f1a5c15c620 0x7f1a5c0ca590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T15:11:19.865 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:19.864+0000 7f1a61574640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1a5c0cb460 con 0x7f1a5c15c620 2023-08-09T15:11:19.865 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:19.864+0000 7f1a61574640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1a5c0caad0 con 0x7f1a5c0cf4f0 2023-08-09T15:11:19.865 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:19.864+0000 7f1a61574640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1a5c1f1d00 con 0x7f1a5c0d0ef0 2023-08-09T15:11:19.867 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:19.865+0000 7f1a60d73640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f1a5c15c620 0x7f1a5c0ca590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T15:11:19.867 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:19.865+0000 7f1a60d73640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f1a5c15c620 0x7f1a5c0ca590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:37814/0 (socket says 172.21.15.43:37814) 2023-08-09T15:11:19.867 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:19.865+0000 7f1a60d73640 1 -- 172.21.15.43:0/1963329526 learned_addr learned my addr 172.21.15.43:0/1963329526 (peer_addr_for_me v2:172.21.15.43:0/0) 2023-08-09T15:11:19.867 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:19.865+0000 7f1a5bfff640 1 --2- 172.21.15.43:0/1963329526 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f1a5c0cf4f0 0x7f1a5c0cbe00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T15:11:19.867 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:19.865+0000 7f1a5b7fe640 1 --2- 172.21.15.43:0/1963329526 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f1a5c0d0ef0 0x7f1a5c0caf20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T15:11:19.867 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:19.865+0000 7f1a60d73640 1 -- 172.21.15.43:0/1963329526 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f1a5c0d0ef0 msgr2=0x7f1a5c0caf20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T15:11:19.867 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:19.865+0000 7f1a60d73640 1 --2- 172.21.15.43:0/1963329526 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f1a5c0d0ef0 0x7f1a5c0caf20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T15:11:19.868 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:19.865+0000 7f1a60d73640 1 -- 172.21.15.43:0/1963329526 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f1a5c0cf4f0 msgr2=0x7f1a5c0cbe00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T15:11:19.868 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:19.865+0000 7f1a60d73640 1 --2- 172.21.15.43:0/1963329526 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f1a5c0cf4f0 0x7f1a5c0cbe00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T15:11:19.868 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:19.865+0000 7f1a60d73640 1 -- 172.21.15.43:0/1963329526 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1a540094d0 con 0x7f1a5c15c620 2023-08-09T15:11:19.868 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:19.866+0000 7f1a60d73640 1 --2- 172.21.15.43:0/1963329526 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f1a5c15c620 0x7f1a5c0ca590 secure :-1 s=READY pgs=2351 cs=0 l=1 rev1=1 crypto rx=0x7f1a5402b8a0 tx=0x7f1a540046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-09T15:11:19.868 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:19.866+0000 7f1a58ff9640 1 -- 172.21.15.43:0/1963329526 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1a54032920 con 0x7f1a5c15c620 2023-08-09T15:11:19.868 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:19.866+0000 7f1a58ff9640 1 -- 172.21.15.43:0/1963329526 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1a54032a80 con 0x7f1a5c15c620 2023-08-09T15:11:19.868 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:19.866+0000 7f1a58ff9640 1 -- 172.21.15.43:0/1963329526 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1a54032e30 con 0x7f1a5c15c620 2023-08-09T15:11:19.869 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:19.866+0000 7f1a61574640 1 -- 172.21.15.43:0/1963329526 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1a5c1f1fe0 con 0x7f1a5c15c620 2023-08-09T15:11:19.869 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:19.867+0000 7f1a61574640 1 -- 172.21.15.43:0/1963329526 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1a5c1f25a0 con 0x7f1a5c15c620 2023-08-09T15:11:19.869 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:19.868+0000 7f1a61574640 1 -- 172.21.15.43:0/1963329526 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1a5c1f4f80 con 0x7f1a5c15c620 2023-08-09T15:11:19.869 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:19.869+0000 7f1a58ff9640 1 -- 172.21.15.43:0/1963329526 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 118) v1 ==== 88443+0+0 (secure 0 0 0) 0x7f1a5404d3f0 con 0x7f1a5c15c620 2023-08-09T15:11:19.870 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:19.870+0000 7f1a58ff9640 1 --2- 172.21.15.43:0/1963329526 >> [v2:172.21.15.203:6832/4034093268,v1:172.21.15.203:6833/4034093268] conn(0x7f1a4806a170 0x7f1a4806c630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T15:11:19.870 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:19.870+0000 7f1a58ff9640 1 -- 172.21.15.43:0/1963329526 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(215..215 src has 1..215) v4 ==== 6348+0+0 (secure 0 0 0) 0x7f1a540aa7d0 con 0x7f1a5c15c620 2023-08-09T15:11:19.870 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:19.870+0000 7f1a5bfff640 1 --2- 172.21.15.43:0/1963329526 >> [v2:172.21.15.203:6832/4034093268,v1:172.21.15.203:6833/4034093268] conn(0x7f1a4806a170 0x7f1a4806c630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T15:11:19.872 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:19.872+0000 7f1a5bfff640 1 --2- 172.21.15.43:0/1963329526 >> [v2:172.21.15.203:6832/4034093268,v1:172.21.15.203:6833/4034093268] conn(0x7f1a4806a170 0x7f1a4806c630 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7f1a4c00b450 tx=0x7f1a4c00d040 comp rx=0 tx=0).ready entity=mgr.8833 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-09T15:11:19.875 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:19.874+0000 7f1a58ff9640 1 -- 172.21.15.43:0/1963329526 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178066 (secure 0 0 0) 0x7f1a5c1f4f80 con 0x7f1a5c15c620 2023-08-09T15:11:20.069 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:20.068+0000 7f1a61574640 1 -- 172.21.15.43:0/1963329526 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f1a5c1f4f80 con 0x7f1a5c15c620 2023-08-09T15:11:20.071 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:20.070+0000 7f1a58ff9640 1 -- 172.21.15.43:0/1963329526 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v215) v1 ==== 74+0+15996 (secure 0 0 0) 0x7f1a5c1f4f80 con 0x7f1a5c15c620 2023-08-09T15:11:20.072 INFO:teuthology.orchestra.run.smithi043.stdout: 2023-08-09T15:11:20.072 INFO:teuthology.orchestra.run.smithi043.stdout:{"epoch":215,"fsid":"daac87ee-1337-4250-a955-aea4e408a39b","created":"2023-08-09T13:33:41.701198+0000","modified":"2023-08-09T15:11:16.576464+0000","last_up_change":"2023-08-09T13:33:53.783453+0000","last_in_change":"2023-08-09T13:33:45.148904+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":25,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":"rbd","create_time":"2023-08-09T13:33:54.365137+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":".mgr","create_time":"2023-08-09T13:33:54.831803+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"cbd49ecd-e896-4fbd-864e-5c37a829b165","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":203,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6800","nonce":502667335},{"type":"v1","addr":"172.21.15.43:6801","nonce":502667335}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6802","nonce":502667335},{"type":"v1","addr":"172.21.15.43:6803","nonce":502667335}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6806","nonce":502667335},{"type":"v1","addr":"172.21.15.43:6807","nonce":502667335}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6804","nonce":502667335},{"type":"v1","addr":"172.21.15.43:6805","nonce":502667335}]},"public_addr":"172.21.15.43:6801/502667335","cluster_addr":"172.21.15.43:6803/502667335","heartbeat_back_addr":"172.21.15.43:6807/502667335","heartbeat_front_addr":"172.21.15.43:6805/502667335","state":["exists","up"]},{"osd":1,"uuid":"75c7e0fe-1ec7-4fe8-a2f2-3661a65b8500","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":203,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6808","nonce":3289134610},{"type":"v1","addr":"172.21.15.43:6809","nonce":3289134610}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6810","nonce":3289134610},{"type":"v1","addr":"172.21.15.43:6812","nonce":3289134610}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6821","nonce":3289134610},{"type":"v1","addr":"172.21.15.43:6824","nonce":3289134610}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6815","nonce":3289134610},{"type":"v1","addr":"172.21.15.43:6818","nonce":3289134610}]},"public_addr":"172.21.15.43:6809/3289134610","cluster_addr":"172.21.15.43:6812/3289134610","heartbeat_back_addr":"172.21.15.43:6824/3289134610","heartbeat_front_addr":"172.21.15.43:6818/3289134610","state":["exists","up"]},{"osd":2,"uuid":"bb5c41e9-7f5f-4125-9b01-3bb8bfc2ea83","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":203,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6811","nonce":4079709736},{"type":"v1","addr":"172.21.15.43:6814","nonce":4079709736}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6817","nonce":4079709736},{"type":"v1","addr":"172.21.15.43:6820","nonce":4079709736}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6829","nonce":4079709736},{"type":"v1","addr":"172.21.15.43:6831","nonce":4079709736}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6823","nonce":4079709736},{"type":"v1","addr":"172.21.15.43:6826","nonce":4079709736}]},"public_addr":"172.21.15.43:6814/4079709736","cluster_addr":"172.21.15.43:6820/4079709736","heartbeat_back_addr":"172.21.15.43:6831/4079709736","heartbeat_front_addr":"172.21.15.43:6826/4079709736","state":["exists","up"]},{"osd":3,"uuid":"bfe4c8b4-694a-465c-87e6-2b2aab852613","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":203,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6813","nonce":1709538759},{"type":"v1","addr":"172.21.15.43:6816","nonce":1709538759}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6819","nonce":1709538759},{"type":"v1","addr":"172.21.15.43:6822","nonce":1709538759}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6828","nonce":1709538759},{"type":"v1","addr":"172.21.15.43:6830","nonce":1709538759}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6825","nonce":1709538759},{"type":"v1","addr":"172.21.15.43:6827","nonce":1709538759}]},"public_addr":"172.21.15.43:6816/1709538759","cluster_addr":"172.21.15.43:6822/1709538759","heartbeat_back_addr":"172.21.15.43:6830/1709538759","heartbeat_front_addr":"172.21.15.43:6827/1709538759","state":["exists","up"]},{"osd":4,"uuid":"fa2e8a77-aba1-4756-853e-0d174a5bdeba","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":203,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6800","nonce":4196866481},{"type":"v1","addr":"172.21.15.203:6801","nonce":4196866481}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6802","nonce":4196866481},{"type":"v1","addr":"172.21.15.203:6803","nonce":4196866481}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6806","nonce":4196866481},{"type":"v1","addr":"172.21.15.203:6807","nonce":4196866481}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6804","nonce":4196866481},{"type":"v1","addr":"172.21.15.203:6805","nonce":4196866481}]},"public_addr":"172.21.15.203:6801/4196866481","cluster_addr":"172.21.15.203:6803/4196866481","heartbeat_back_addr":"172.21.15.203:6807/4196866481","heartbeat_front_addr":"172.21.15.203:6805/4196866481","state":["exists","up"]},{"osd":5,"uuid":"81811e4b-4d24-4f51-a82c-f1d700a2a00d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":207,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6816","nonce":3059290081},{"type":"v1","addr":"172.21.15.203:6817","nonce":3059290081}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6818","nonce":3059290081},{"type":"v1","addr":"172.21.15.203:6819","nonce":3059290081}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6822","nonce":3059290081},{"type":"v1","addr":"172.21.15.203:6823","nonce":3059290081}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6820","nonce":3059290081},{"type":"v1","addr":"172.21.15.203:6821","nonce":3059290081}]},"public_addr":"172.21.15.203:6817/3059290081","cluster_addr":"172.21.15.203:6819/3059290081","heartbeat_back_addr":"172.21.15.203:6823/3059290081","heartbeat_front_addr":"172.21.15.203:6821/3059290081","state":["exists","up"]},{"osd":6,"uuid":"307bac4d-6a5b-443d-af75-56d695d7674f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":203,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6824","nonce":512155569},{"type":"v1","addr":"172.21.15.203:6825","nonce":512155569}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6826","nonce":512155569},{"type":"v1","addr":"172.21.15.203:6827","nonce":512155569}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6830","nonce":512155569},{"type":"v1","addr":"172.21.15.203:6831","nonce":512155569}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6828","nonce":512155569},{"type":"v1","addr":"172.21.15.203:6829","nonce":512155569}]},"public_addr":"172.21.15.203:6825/512155569","cluster_addr":"172.21.15.203:6827/512155569","heartbeat_back_addr":"172.21.15.203:6831/512155569","heartbeat_front_addr":"172.21.15.203:6829/512155569","state":["exists","up"]},{"osd":7,"uuid":"a95dc7ca-9f1b-4295-af70-68a22a2d7d5e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":207,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6808","nonce":3331450438},{"type":"v1","addr":"172.21.15.203:6809","nonce":3331450438}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6810","nonce":3331450438},{"type":"v1","addr":"172.21.15.203:6811","nonce":3331450438}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6814","nonce":3331450438},{"type":"v1","addr":"172.21.15.203:6815","nonce":3331450438}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6812","nonce":3331450438},{"type":"v1","addr":"172.21.15.203:6813","nonce":3331450438}]},"public_addr":"172.21.15.203:6809/3331450438","cluster_addr":"172.21.15.203:6811/3331450438","heartbeat_back_addr":"172.21.15.203:6815/3331450438","heartbeat_front_addr":"172.21.15.203:6813/3331450438","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-09T13:33:50.547652+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-09T13:33:50.589469+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-09T13:33:50.561724+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-09T13:33:50.599650+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-09T13:33:50.411855+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-09T13:33:50.219681+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-09T13:33:50.504664+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-09T13:33:50.441248+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.203:0/822076630":"2023-08-10T15:11:08.350045+0000","172.21.15.203:0/79247817":"2023-08-10T15:11:08.350045+0000","172.21.15.203:6833/4012896436":"2023-08-10T15:11:08.350045+0000","172.21.15.43:6835/349688227":"2023-08-10T15:11:13.565832+0000","172.21.15.203:6832/4012896436":"2023-08-10T15:11:08.350045+0000","172.21.15.203:0/3345703784":"2023-08-10T15:11:08.350045+0000","172.21.15.203:0/1987920954":"2023-08-10T15:11:08.350045+0000","172.21.15.203:0/4253505338":"2023-08-10T15:11:08.350045+0000","172.21.15.203:6833/3082732769":"2023-08-10T15:09:01.285901+0000","172.21.15.203:6832/3082732769":"2023-08-10T15:09:01.285901+0000","172.21.15.203:0/1756781921":"2023-08-10T15:09:01.285901+0000","172.21.15.203:0/454071895":"2023-08-10T15:09:01.285901+0000","172.21.15.203:0/2854569557":"2023-08-10T15:09:01.285901+0000","172.21.15.203:0/2821325802":"2023-08-10T15:11:08.350045+0000","172.21.15.203:0/1159944241":"2023-08-10T15:09:01.285901+0000","172.21.15.43:6833/349688227":"2023-08-10T15:11:13.565832+0000","172.21.15.203:0/1645632393":"2023-08-10T15:09:01.285901+0000","172.21.15.203:0/840449585":"2023-08-10T15:09:01.285901+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":7,"snaps":[{"begin":104,"length":1},{"begin":221,"length":1}]}],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-08-09T15:11:20.074 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:20.073+0000 7f1a61574640 1 -- 172.21.15.43:0/1963329526 >> [v2:172.21.15.203:6832/4034093268,v1:172.21.15.203:6833/4034093268] conn(0x7f1a4806a170 msgr2=0x7f1a4806c630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T15:11:20.074 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:20.073+0000 7f1a61574640 1 --2- 172.21.15.43:0/1963329526 >> [v2:172.21.15.203:6832/4034093268,v1:172.21.15.203:6833/4034093268] conn(0x7f1a4806a170 0x7f1a4806c630 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7f1a4c00b450 tx=0x7f1a4c00d040 comp rx=0 tx=0).stop 2023-08-09T15:11:20.075 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:20.073+0000 7f1a61574640 1 -- 172.21.15.43:0/1963329526 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f1a5c15c620 msgr2=0x7f1a5c0ca590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T15:11:20.075 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:20.073+0000 7f1a61574640 1 --2- 172.21.15.43:0/1963329526 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f1a5c15c620 0x7f1a5c0ca590 secure :-1 s=READY pgs=2351 cs=0 l=1 rev1=1 crypto rx=0x7f1a5402b8a0 tx=0x7f1a540046c0 comp rx=0 tx=0).stop 2023-08-09T15:11:20.075 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:20.073+0000 7f1a61574640 1 -- 172.21.15.43:0/1963329526 shutdown_connections 2023-08-09T15:11:20.076 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:20.073+0000 7f1a61574640 1 --2- 172.21.15.43:0/1963329526 >> [v2:172.21.15.203:6832/4034093268,v1:172.21.15.203:6833/4034093268] conn(0x7f1a4806a170 0x7f1a4806c630 unknown :-1 s=CLOSED pgs=40 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T15:11:20.076 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:20.073+0000 7f1a61574640 1 --2- 172.21.15.43:0/1963329526 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f1a5c15c620 0x7f1a5c0ca590 unknown :-1 s=CLOSED pgs=2351 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T15:11:20.076 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:20.073+0000 7f1a61574640 1 --2- 172.21.15.43:0/1963329526 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f1a5c0d0ef0 0x7f1a5c0caf20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T15:11:20.077 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:20.073+0000 7f1a61574640 1 --2- 172.21.15.43:0/1963329526 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f1a5c0cf4f0 0x7f1a5c0cbe00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T15:11:20.077 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:20.073+0000 7f1a61574640 1 -- 172.21.15.43:0/1963329526 >> 172.21.15.43:0/1963329526 conn(0x7f1a5c09b650 msgr2=0x7f1a5c0ce8d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-09T15:11:20.077 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:20.073+0000 7f1a61574640 1 -- 172.21.15.43:0/1963329526 shutdown_connections 2023-08-09T15:11:20.077 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:20.074+0000 7f1a61574640 1 -- 172.21.15.43:0/1963329526 wait complete. 2023-08-09T15:11:20.095 INFO:tasks.cephfs.cephfs_test_case.ceph_manager:[{'pool': 1, 'pool_name': 'rbd', 'create_time': '2023-08-09T13:33:54.365137+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}, 'read_balance': {'score_acting': 2, 'score_stable': 2, 'optimal_score': 1, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': '.mgr', 'create_time': '2023-08-09T13:33:54.831803+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-08-09T15:11:20.095 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get rbd pg_num 2023-08-09T15:11:20.331 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:20.331+0000 7f108c1e8640 1 Processor -- start 2023-08-09T15:11:20.332 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:20.331+0000 7f108c1e8640 1 -- start start 2023-08-09T15:11:20.333 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:20.332+0000 7f108c1e8640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f10840ce8d0 0x7f10840c9f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T15:11:20.333 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:20.332+0000 7f108c1e8640 1 -- --> v1:172.21.15.43:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f108415aef0 con 0x7f10840cb2d0 2023-08-09T15:11:20.333 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:20.333+0000 7f108c1e8640 1 -- --> v1:172.21.15.203:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f108415e940 con 0x7f108415df60 2023-08-09T15:11:20.333 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:20.333+0000 7f108c1e8640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f108415a970 con 0x7f10840ce8d0 2023-08-09T15:11:20.335 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:20.333+0000 7f1089f84640 1 --1- >> v1:172.21.15.43:6789/0 conn(0x7f10840cb2d0 0x7f108415b380 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.43:6789/0 says I am v1:172.21.15.43:45574/0 (socket says 172.21.15.43:45574) 2023-08-09T15:11:20.335 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:20.333+0000 7f1089f84640 1 -- 172.21.15.43:0/324425907 learned_addr learned my addr 172.21.15.43:0/324425907 (peer_addr_for_me v1:172.21.15.43:0/0) 2023-08-09T15:11:20.335 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:20.333+0000 7f108a785640 1 --2- 172.21.15.43:0/324425907 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f10840ce8d0 0x7f10840c9f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T15:11:20.335 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:20.334+0000 7f1088f82640 1 -- 172.21.15.43:0/324425907 <== mon.0 v1:172.21.15.43:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3389861112 0 0) 0x7f108415aef0 con 0x7f10840cb2d0 2023-08-09T15:11:20.335 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:20.334+0000 7f1088f82640 1 -- 172.21.15.43:0/324425907 --> v1:172.21.15.43:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f1070003520 con 0x7f10840cb2d0 2023-08-09T15:11:20.335 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:20.334+0000 7f1088f82640 1 -- 172.21.15.43:0/324425907 <== mon.1 v1:172.21.15.203:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3294806950 0 0) 0x7f108415e940 con 0x7f108415df60 2023-08-09T15:11:20.336 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:20.334+0000 7f1088f82640 1 -- 172.21.15.43:0/324425907 --> v1:172.21.15.203:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f108415aef0 con 0x7f108415df60 2023-08-09T15:11:20.336 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:20.334+0000 7f1088f82640 1 -- 172.21.15.43:0/324425907 <== mon.0 v1:172.21.15.43:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3639091993 0 0) 0x7f1070003520 con 0x7f10840cb2d0 2023-08-09T15:11:20.336 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:20.334+0000 7f1088f82640 1 -- 172.21.15.43:0/324425907 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f10840ce8d0 msgr2=0x7f10840c9f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-09T15:11:20.336 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:20.334+0000 7f1088f82640 1 --2- 172.21.15.43:0/324425907 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f10840ce8d0 0x7f10840c9f90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T15:11:20.336 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:20.334+0000 7f1088f82640 1 -- 172.21.15.43:0/324425907 >> v1:172.21.15.203:6789/0 conn(0x7f108415df60 legacy=0x7f10840ce1c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T15:11:20.336 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:20.334+0000 7f1088f82640 1 -- 172.21.15.43:0/324425907 --> v1:172.21.15.43:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f108415ec20 con 0x7f10840cb2d0 2023-08-09T15:11:20.336 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:20.334+0000 7f1088f82640 1 -- 172.21.15.43:0/324425907 <== mon.0 v1:172.21.15.43:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1285150740 0 0) 0x7f1080001670 con 0x7f10840cb2d0 2023-08-09T15:11:20.337 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:20.334+0000 7f1088f82640 1 -- 172.21.15.43:0/324425907 >> v1:172.21.15.43:6789/0 conn(0x7f10840cb2d0 legacy=0x7f108415b380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T15:11:20.337 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:20.334+0000 7f1088f82640 1 --2- 172.21.15.43:0/324425907 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f1070003b30 0x7f1070003f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T15:11:20.337 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:20.334+0000 7f1088f82640 1 --2- 172.21.15.43:0/324425907 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f10700047c0 0x7f1070004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T15:11:20.337 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:20.334+0000 7f1088f82640 1 --2- 172.21.15.43:0/324425907 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f1070005180 0x7f107000b980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T15:11:20.337 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:20.334+0000 7f1088f82640 1 -- 172.21.15.43:0/324425907 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f108415a970 con 0x7f1070005180 2023-08-09T15:11:20.337 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:20.334+0000 7f1088f82640 1 -- 172.21.15.43:0/324425907 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f107000bf70 con 0x7f1070003b30 2023-08-09T15:11:20.337 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:20.334+0000 7f1088f82640 1 -- 172.21.15.43:0/324425907 --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f107000c130 con 0x7f10700047c0 2023-08-09T15:11:20.338 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:20.335+0000 7f108a785640 1 --2- 172.21.15.43:0/324425907 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f1070005180 0x7f107000b980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T15:11:20.339 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:20.335+0000 7f1089783640 1 --2- 172.21.15.43:0/324425907 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f10700047c0 0x7f1070004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T15:11:20.339 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:20.335+0000 7f1089f84640 1 --2- 172.21.15.43:0/324425907 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f1070003b30 0x7f1070003f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T15:11:20.339 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:20.335+0000 7f108a785640 1 -- 172.21.15.43:0/324425907 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f10700047c0 msgr2=0x7f1070004c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-09T15:11:20.339 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:20.335+0000 7f108a785640 1 --2- 172.21.15.43:0/324425907 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f10700047c0 0x7f1070004c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T15:11:20.339 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:20.335+0000 7f108a785640 1 -- 172.21.15.43:0/324425907 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f1070003b30 msgr2=0x7f1070003f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-09T15:11:20.340 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:20.335+0000 7f108a785640 1 --2- 172.21.15.43:0/324425907 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f1070003b30 0x7f1070003f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T15:11:20.340 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:20.335+0000 7f108a785640 1 -- 172.21.15.43:0/324425907 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1070003520 con 0x7f1070005180 2023-08-09T15:11:20.340 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:20.335+0000 7f1089f84640 1 --2- 172.21.15.43:0/324425907 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f1070003b30 0x7f1070003f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-08-09T15:11:20.340 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:20.336+0000 7f108a785640 1 --2- 172.21.15.43:0/324425907 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f1070005180 0x7f107000b980 secure :-1 s=READY pgs=2353 cs=0 l=1 rev1=1 crypto rx=0x7f1084069ad0 tx=0x7f10700057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3ba3ab467df5565a server_cookie=0 in_seq=0 out_seq=0 2023-08-09T15:11:20.340 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:20.337+0000 7f1088f82640 1 -- 172.21.15.43:0/324425907 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f107c02cb80 con 0x7f1070005180 2023-08-09T15:11:20.340 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:20.337+0000 7f1088f82640 1 -- 172.21.15.43:0/324425907 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f107c00f040 con 0x7f1070005180 2023-08-09T15:11:20.341 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:20.337+0000 7f1088f82640 1 -- 172.21.15.43:0/324425907 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f107c034440 con 0x7f1070005180 2023-08-09T15:11:20.341 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:20.337+0000 7f108c1e8640 1 -- 172.21.15.43:0/324425907 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f1070005180 msgr2=0x7f107000b980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T15:11:20.341 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:20.337+0000 7f108c1e8640 1 --2- 172.21.15.43:0/324425907 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f1070005180 0x7f107000b980 secure :-1 s=READY pgs=2353 cs=0 l=1 rev1=1 crypto rx=0x7f1084069ad0 tx=0x7f10700057f0 comp rx=0 tx=0).stop 2023-08-09T15:11:20.341 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:20.337+0000 7f108a785640 1 -- 172.21.15.43:0/324425907 reap_dead start 2023-08-09T15:11:20.341 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:20.338+0000 7f108c1e8640 1 -- 172.21.15.43:0/324425907 shutdown_connections 2023-08-09T15:11:20.341 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:20.338+0000 7f108c1e8640 1 -- 172.21.15.43:0/324425907 >> 172.21.15.43:0/324425907 conn(0x7f10840aff50 msgr2=0x7f10840b0360 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-09T15:11:20.342 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:20.338+0000 7f108c1e8640 1 -- 172.21.15.43:0/324425907 shutdown_connections 2023-08-09T15:11:20.342 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:20.338+0000 7f108c1e8640 1 -- 172.21.15.43:0/324425907 wait complete. 2023-08-09T15:11:20.343 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:20.340+0000 7f108c1e8640 1 Processor -- start 2023-08-09T15:11:20.344 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:20.340+0000 7f108c1e8640 1 -- start start 2023-08-09T15:11:20.344 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:20.341+0000 7f108c1e8640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f1070005180 0x7f10840cd840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T15:11:20.344 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:20.341+0000 7f108c1e8640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f10840b0c50 0x7f10840c7ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T15:11:20.344 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:20.341+0000 7f108c1e8640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f1084159440 0x7f10841598c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T15:11:20.344 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:20.341+0000 7f108c1e8640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1084159e00 con 0x7f1084159440 2023-08-09T15:11:20.344 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:20.341+0000 7f108c1e8640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f10840c60d0 con 0x7f1070005180 2023-08-09T15:11:20.344 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:20.341+0000 7f108c1e8640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f10840c8020 con 0x7f10840b0c50 2023-08-09T15:11:20.345 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:20.341+0000 7f108a785640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f1084159440 0x7f10841598c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T15:11:20.345 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:20.341+0000 7f108a785640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f1084159440 0x7f10841598c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:37830/0 (socket says 172.21.15.43:37830) 2023-08-09T15:11:20.345 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:20.341+0000 7f108a785640 1 -- 172.21.15.43:0/973924539 learned_addr learned my addr 172.21.15.43:0/973924539 (peer_addr_for_me v2:172.21.15.43:0/0) 2023-08-09T15:11:20.345 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:20.341+0000 7f1089f84640 1 --2- 172.21.15.43:0/973924539 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f1070005180 0x7f10840cd840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T15:11:20.345 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:20.341+0000 7f1089783640 1 --2- 172.21.15.43:0/973924539 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f10840b0c50 0x7f10840c7ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T15:11:20.345 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:20.342+0000 7f108a785640 1 -- 172.21.15.43:0/973924539 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f10840b0c50 msgr2=0x7f10840c7ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T15:11:20.345 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:20.342+0000 7f108a785640 1 --2- 172.21.15.43:0/973924539 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f10840b0c50 0x7f10840c7ae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T15:11:20.346 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:20.342+0000 7f108a785640 1 -- 172.21.15.43:0/973924539 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f1070005180 msgr2=0x7f10840cd840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T15:11:20.346 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:20.342+0000 7f108a785640 1 --2- 172.21.15.43:0/973924539 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f1070005180 0x7f10840cd840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T15:11:20.346 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:20.342+0000 7f108a785640 1 -- 172.21.15.43:0/973924539 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f10840b1210 con 0x7f1084159440 2023-08-09T15:11:20.346 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:20.342+0000 7f108a785640 1 --2- 172.21.15.43:0/973924539 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f1084159440 0x7f10841598c0 secure :-1 s=READY pgs=2354 cs=0 l=1 rev1=1 crypto rx=0x7f107c037040 tx=0x7f107c009750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-09T15:11:20.346 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:20.342+0000 7f107a7fc640 1 -- 172.21.15.43:0/973924539 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f107c009450 con 0x7f1084159440 2023-08-09T15:11:20.346 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:20.342+0000 7f107a7fc640 1 -- 172.21.15.43:0/973924539 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f107c034bd0 con 0x7f1084159440 2023-08-09T15:11:20.346 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:20.342+0000 7f107a7fc640 1 -- 172.21.15.43:0/973924539 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f107c03e410 con 0x7f1084159440 2023-08-09T15:11:20.346 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:20.342+0000 7f108c1e8640 1 -- 172.21.15.43:0/973924539 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f10840c6370 con 0x7f1084159440 2023-08-09T15:11:20.347 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:20.343+0000 7f108c1e8640 1 -- 172.21.15.43:0/973924539 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f10840c6930 con 0x7f1084159440 2023-08-09T15:11:20.347 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:20.344+0000 7f108c1e8640 1 -- 172.21.15.43:0/973924539 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f10840739f0 con 0x7f1084159440 2023-08-09T15:11:20.348 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:20.345+0000 7f107a7fc640 1 -- 172.21.15.43:0/973924539 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 118) v1 ==== 88443+0+0 (secure 0 0 0) 0x7f107c03b050 con 0x7f1084159440 2023-08-09T15:11:20.348 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:20.346+0000 7f107a7fc640 1 --2- 172.21.15.43:0/973924539 >> [v2:172.21.15.203:6832/4034093268,v1:172.21.15.203:6833/4034093268] conn(0x7f106806a1a0 0x7f106806c660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T15:11:20.348 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:20.346+0000 7f107a7fc640 1 -- 172.21.15.43:0/973924539 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(215..215 src has 1..215) v4 ==== 6348+0+0 (secure 0 0 0) 0x7f107c0a9d90 con 0x7f1084159440 2023-08-09T15:11:20.348 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:20.346+0000 7f1089f84640 1 --2- 172.21.15.43:0/973924539 >> [v2:172.21.15.203:6832/4034093268,v1:172.21.15.203:6833/4034093268] conn(0x7f106806a1a0 0x7f106806c660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T15:11:20.349 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:20.348+0000 7f1089f84640 1 --2- 172.21.15.43:0/973924539 >> [v2:172.21.15.203:6832/4034093268,v1:172.21.15.203:6833/4034093268] conn(0x7f106806a1a0 0x7f106806c660 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7f1080000c00 tx=0x7f1080009040 comp rx=0 tx=0).ready entity=mgr.8833 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-09T15:11:20.350 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:20.350+0000 7f107a7fc640 1 -- 172.21.15.43:0/973924539 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178066 (secure 0 0 0) 0x7f107c075260 con 0x7f1084159440 2023-08-09T15:11:20.551 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:20.550+0000 7f108c1e8640 1 -- 172.21.15.43:0/973924539 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"} v 0) v1 -- 0x7f108415a210 con 0x7f1084159440 2023-08-09T15:11:20.552 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:20.551+0000 7f107a7fc640 1 -- 172.21.15.43:0/973924539 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"}]=0 v215) v1 ==== 92+0+10 (secure 0 0 0) 0x7f107c048020 con 0x7f1084159440 2023-08-09T15:11:20.552 INFO:teuthology.orchestra.run.smithi043.stdout:pg_num: 8 2023-08-09T15:11:20.554 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:20.554+0000 7f108c1e8640 1 -- 172.21.15.43:0/973924539 >> [v2:172.21.15.203:6832/4034093268,v1:172.21.15.203:6833/4034093268] conn(0x7f106806a1a0 msgr2=0x7f106806c660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T15:11:20.555 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:20.554+0000 7f108c1e8640 1 --2- 172.21.15.43:0/973924539 >> [v2:172.21.15.203:6832/4034093268,v1:172.21.15.203:6833/4034093268] conn(0x7f106806a1a0 0x7f106806c660 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7f1080000c00 tx=0x7f1080009040 comp rx=0 tx=0).stop 2023-08-09T15:11:20.555 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:20.554+0000 7f108c1e8640 1 -- 172.21.15.43:0/973924539 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f1084159440 msgr2=0x7f10841598c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T15:11:20.555 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:20.554+0000 7f108c1e8640 1 --2- 172.21.15.43:0/973924539 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f1084159440 0x7f10841598c0 secure :-1 s=READY pgs=2354 cs=0 l=1 rev1=1 crypto rx=0x7f107c037040 tx=0x7f107c009750 comp rx=0 tx=0).stop 2023-08-09T15:11:20.556 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:20.554+0000 7f108c1e8640 1 -- 172.21.15.43:0/973924539 shutdown_connections 2023-08-09T15:11:20.556 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:20.554+0000 7f108c1e8640 1 --2- 172.21.15.43:0/973924539 >> [v2:172.21.15.203:6832/4034093268,v1:172.21.15.203:6833/4034093268] conn(0x7f106806a1a0 0x7f106806c660 unknown :-1 s=CLOSED pgs=41 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T15:11:20.556 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:20.554+0000 7f108c1e8640 1 --2- 172.21.15.43:0/973924539 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f1084159440 0x7f10841598c0 unknown :-1 s=CLOSED pgs=2354 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T15:11:20.556 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:20.554+0000 7f108c1e8640 1 --2- 172.21.15.43:0/973924539 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f10840b0c50 0x7f10840c7ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T15:11:20.556 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:20.554+0000 7f108c1e8640 1 --2- 172.21.15.43:0/973924539 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f1070005180 0x7f10840cd840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T15:11:20.556 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:20.554+0000 7f108c1e8640 1 -- 172.21.15.43:0/973924539 >> 172.21.15.43:0/973924539 conn(0x7f10840aff50 msgr2=0x7f108415c620 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-09T15:11:20.556 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:20.555+0000 7f108c1e8640 1 -- 172.21.15.43:0/973924539 shutdown_connections 2023-08-09T15:11:20.556 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:20.555+0000 7f108c1e8640 1 -- 172.21.15.43:0/973924539 wait complete. 2023-08-09T15:11:20.575 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-08-09T15:11:20.799 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:20.798+0000 7fa022b42640 1 Processor -- start 2023-08-09T15:11:20.799 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:20.799+0000 7fa022b42640 1 -- start start 2023-08-09T15:11:20.801 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:20.800+0000 7fa022b42640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fa01c0d04b0 0x7fa01c0cd6d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T15:11:20.801 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:20.800+0000 7fa022b42640 1 -- --> v1:172.21.15.43:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa01c0b7bd0 con 0x7fa01c15c550 2023-08-09T15:11:20.801 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:20.801+0000 7fa022b42640 1 -- --> v1:172.21.15.203:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa01c0d1c00 con 0x7fa01c0b45f0 2023-08-09T15:11:20.801 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:20.801+0000 7fa022b42640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa01c15c930 con 0x7fa01c0d04b0 2023-08-09T15:11:20.802 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:20.801+0000 7fa0208de640 1 --1- >> v1:172.21.15.43:6789/0 conn(0x7fa01c15c550 0x7fa01c0b3ee0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.43:6789/0 says I am v1:172.21.15.43:45580/0 (socket says 172.21.15.43:45580) 2023-08-09T15:11:20.802 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:20.801+0000 7fa0208de640 1 -- 172.21.15.43:0/3847712202 learned_addr learned my addr 172.21.15.43:0/3847712202 (peer_addr_for_me v1:172.21.15.43:0/0) 2023-08-09T15:11:20.803 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:20.801+0000 7fa0210df640 1 --2- 172.21.15.43:0/3847712202 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fa01c0d04b0 0x7fa01c0cd6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T15:11:20.803 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:20.801+0000 7fa01b7fe640 1 -- 172.21.15.43:0/3847712202 <== mon.0 v1:172.21.15.43:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4194249593 0 0) 0x7fa01c0b7bd0 con 0x7fa01c15c550 2023-08-09T15:11:20.803 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:20.801+0000 7fa01b7fe640 1 -- 172.21.15.43:0/3847712202 --> v1:172.21.15.43:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fa008003520 con 0x7fa01c15c550 2023-08-09T15:11:20.804 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:20.802+0000 7fa01b7fe640 1 -- 172.21.15.43:0/3847712202 <== mon.0 v1:172.21.15.43:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 4159646575 0 0) 0x7fa008003520 con 0x7fa01c15c550 2023-08-09T15:11:20.804 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:20.802+0000 7fa01b7fe640 1 -- 172.21.15.43:0/3847712202 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fa01c0d04b0 msgr2=0x7fa01c0cd6d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-09T15:11:20.804 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:20.802+0000 7fa01b7fe640 1 --2- 172.21.15.43:0/3847712202 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fa01c0d04b0 0x7fa01c0cd6d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T15:11:20.804 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:20.802+0000 7fa01b7fe640 1 -- 172.21.15.43:0/3847712202 >> v1:172.21.15.203:6789/0 conn(0x7fa01c0b45f0 legacy=0x7fa01c0b59d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T15:11:20.804 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:20.802+0000 7fa01b7fe640 1 -- 172.21.15.43:0/3847712202 --> v1:172.21.15.43:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa01c0b54f0 con 0x7fa01c15c550 2023-08-09T15:11:20.804 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:20.802+0000 7fa01b7fe640 1 -- 172.21.15.43:0/3847712202 <== mon.0 v1:172.21.15.43:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1285150740 0 0) 0x7fa004001670 con 0x7fa01c15c550 2023-08-09T15:11:20.805 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:20.802+0000 7fa01b7fe640 1 -- 172.21.15.43:0/3847712202 >> v1:172.21.15.43:6789/0 conn(0x7fa01c15c550 legacy=0x7fa01c0b3ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T15:11:20.805 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:20.802+0000 7fa01b7fe640 1 --2- 172.21.15.43:0/3847712202 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fa01c0d04b0 0x7fa0080039a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T15:11:20.805 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:20.802+0000 7fa01b7fe640 1 --2- 172.21.15.43:0/3847712202 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa008003f20 0x7fa0080047c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T15:11:20.805 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:20.802+0000 7fa01b7fe640 1 --2- 172.21.15.43:0/3847712202 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fa008004d00 0x7fa008005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T15:11:20.805 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:20.802+0000 7fa01b7fe640 1 -- 172.21.15.43:0/3847712202 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa01c15c930 con 0x7fa008003f20 2023-08-09T15:11:20.805 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:20.802+0000 7fa01b7fe640 1 -- 172.21.15.43:0/3847712202 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa00800ba60 con 0x7fa008004d00 2023-08-09T15:11:20.805 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:20.802+0000 7fa01b7fe640 1 -- 172.21.15.43:0/3847712202 --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa00800bc20 con 0x7fa01c0d04b0 2023-08-09T15:11:20.806 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:20.802+0000 7fa01bfff640 1 --2- 172.21.15.43:0/3847712202 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa008003f20 0x7fa0080047c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T15:11:20.806 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:20.803+0000 7fa0208de640 1 --2- 172.21.15.43:0/3847712202 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fa01c0d04b0 0x7fa0080039a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T15:11:20.806 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:20.803+0000 7fa0210df640 1 --2- 172.21.15.43:0/3847712202 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fa008004d00 0x7fa008005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T15:11:20.806 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:20.803+0000 7fa01bfff640 1 -- 172.21.15.43:0/3847712202 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fa01c0d04b0 msgr2=0x7fa0080039a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-09T15:11:20.807 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:20.803+0000 7fa01bfff640 1 --2- 172.21.15.43:0/3847712202 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fa01c0d04b0 0x7fa0080039a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T15:11:20.807 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:20.803+0000 7fa01bfff640 1 -- 172.21.15.43:0/3847712202 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fa008004d00 msgr2=0x7fa008005180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-09T15:11:20.807 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:20.803+0000 7fa01bfff640 1 --2- 172.21.15.43:0/3847712202 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fa008004d00 0x7fa008005180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T15:11:20.807 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:20.803+0000 7fa01bfff640 1 -- 172.21.15.43:0/3847712202 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa008003520 con 0x7fa008003f20 2023-08-09T15:11:20.808 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:20.804+0000 7fa01bfff640 1 --2- 172.21.15.43:0/3847712202 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa008003f20 0x7fa0080047c0 secure :-1 s=READY pgs=2356 cs=0 l=1 rev1=1 crypto rx=0x7fa00c001750 tx=0x7fa0080057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2cbd93647139b9be server_cookie=0 in_seq=0 out_seq=0 2023-08-09T15:11:20.808 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:20.804+0000 7fa01b7fe640 1 -- 172.21.15.43:0/3847712202 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa00c0052d0 con 0x7fa008003f20 2023-08-09T15:11:20.809 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:20.804+0000 7fa01b7fe640 1 -- 172.21.15.43:0/3847712202 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa00c02e050 con 0x7fa008003f20 2023-08-09T15:11:20.809 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:20.804+0000 7fa01b7fe640 1 -- 172.21.15.43:0/3847712202 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa00c003830 con 0x7fa008003f20 2023-08-09T15:11:20.809 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:20.805+0000 7fa022b42640 1 -- 172.21.15.43:0/3847712202 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa008003f20 msgr2=0x7fa0080047c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T15:11:20.809 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:20.805+0000 7fa022b42640 1 --2- 172.21.15.43:0/3847712202 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa008003f20 0x7fa0080047c0 secure :-1 s=READY pgs=2356 cs=0 l=1 rev1=1 crypto rx=0x7fa00c001750 tx=0x7fa0080057f0 comp rx=0 tx=0).stop 2023-08-09T15:11:20.809 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:20.805+0000 7fa0210df640 1 -- 172.21.15.43:0/3847712202 reap_dead start 2023-08-09T15:11:20.809 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:20.805+0000 7fa022b42640 1 -- 172.21.15.43:0/3847712202 shutdown_connections 2023-08-09T15:11:20.809 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:20.805+0000 7fa022b42640 1 -- 172.21.15.43:0/3847712202 >> 172.21.15.43:0/3847712202 conn(0x7fa01c071520 msgr2=0x7fa01c0ceee0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-09T15:11:20.810 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:20.805+0000 7fa022b42640 1 -- 172.21.15.43:0/3847712202 shutdown_connections 2023-08-09T15:11:20.810 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:20.805+0000 7fa022b42640 1 -- 172.21.15.43:0/3847712202 wait complete. 2023-08-09T15:11:20.810 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:20.807+0000 7fa022b42640 1 Processor -- start 2023-08-09T15:11:20.811 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:20.807+0000 7fa022b42640 1 -- start start 2023-08-09T15:11:20.811 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:20.808+0000 7fa022b42640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fa008003f20 0x7fa01c14cf70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T15:11:20.811 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:20.808+0000 7fa022b42640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa01c15e4f0 0x7fa01c0aa7b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T15:11:20.811 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:20.808+0000 7fa022b42640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fa01c15e970 0x7fa01c0aacf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T15:11:20.811 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:20.808+0000 7fa022b42640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa01c0cc0f0 con 0x7fa01c15e4f0 2023-08-09T15:11:20.811 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:20.808+0000 7fa022b42640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa01c0cc3d0 con 0x7fa008003f20 2023-08-09T15:11:20.811 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:20.808+0000 7fa022b42640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa01c0cc6b0 con 0x7fa01c15e970 2023-08-09T15:11:20.813 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:20.808+0000 7fa01bfff640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa01c15e4f0 0x7fa01c0aa7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T15:11:20.813 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:20.808+0000 7fa01bfff640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa01c15e4f0 0x7fa01c0aa7b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:37850/0 (socket says 172.21.15.43:37850) 2023-08-09T15:11:20.813 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:20.809+0000 7fa01bfff640 1 -- 172.21.15.43:0/1327859374 learned_addr learned my addr 172.21.15.43:0/1327859374 (peer_addr_for_me v2:172.21.15.43:0/0) 2023-08-09T15:11:20.813 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:20.809+0000 7fa0208de640 1 --2- 172.21.15.43:0/1327859374 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fa008003f20 0x7fa01c14cf70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T15:11:20.813 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:20.809+0000 7fa0210df640 1 --2- 172.21.15.43:0/1327859374 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fa01c15e970 0x7fa01c0aacf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T15:11:20.813 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:20.809+0000 7fa01bfff640 1 -- 172.21.15.43:0/1327859374 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fa01c15e970 msgr2=0x7fa01c0aacf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T15:11:20.814 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:20.809+0000 7fa01bfff640 1 --2- 172.21.15.43:0/1327859374 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fa01c15e970 0x7fa01c0aacf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T15:11:20.814 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:20.809+0000 7fa01bfff640 1 -- 172.21.15.43:0/1327859374 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fa008003f20 msgr2=0x7fa01c14cf70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T15:11:20.814 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:20.809+0000 7fa01bfff640 1 --2- 172.21.15.43:0/1327859374 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fa008003f20 0x7fa01c14cf70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T15:11:20.814 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:20.809+0000 7fa01bfff640 1 -- 172.21.15.43:0/1327859374 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa01c0cc990 con 0x7fa01c15e4f0 2023-08-09T15:11:20.814 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:20.809+0000 7fa01bfff640 1 --2- 172.21.15.43:0/1327859374 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa01c15e4f0 0x7fa01c0aa7b0 secure :-1 s=READY pgs=2357 cs=0 l=1 rev1=1 crypto rx=0x7fa00c008fd0 tx=0x7fa00c001120 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-09T15:11:20.814 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:20.810+0000 7fa0197fa640 1 -- 172.21.15.43:0/1327859374 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa00c0076c0 con 0x7fa01c15e4f0 2023-08-09T15:11:20.814 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:20.810+0000 7fa0197fa640 1 -- 172.21.15.43:0/1327859374 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa00c007a30 con 0x7fa01c15e4f0 2023-08-09T15:11:20.815 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:20.810+0000 7fa0197fa640 1 -- 172.21.15.43:0/1327859374 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa00c006be0 con 0x7fa01c15e4f0 2023-08-09T15:11:20.815 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:20.810+0000 7fa022b42640 1 -- 172.21.15.43:0/1327859374 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa01c0ccc70 con 0x7fa01c15e4f0 2023-08-09T15:11:20.815 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:20.810+0000 7fa022b42640 1 -- 172.21.15.43:0/1327859374 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa01c0a8dc0 con 0x7fa01c15e4f0 2023-08-09T15:11:20.815 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:20.811+0000 7fa022b42640 1 -- 172.21.15.43:0/1327859374 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa01c0cc7f0 con 0x7fa01c15e4f0 2023-08-09T15:11:20.815 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:20.812+0000 7fa0197fa640 1 -- 172.21.15.43:0/1327859374 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 118) v1 ==== 88443+0+0 (secure 0 0 0) 0x7fa00c006d40 con 0x7fa01c15e4f0 2023-08-09T15:11:20.816 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:20.813+0000 7fa0197fa640 1 --2- 172.21.15.43:0/1327859374 >> [v2:172.21.15.203:6832/4034093268,v1:172.21.15.203:6833/4034093268] conn(0x7f9ffc06a170 0x7f9ffc06c630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T15:11:20.816 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:20.814+0000 7fa0197fa640 1 -- 172.21.15.43:0/1327859374 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(215..215 src has 1..215) v4 ==== 6348+0+0 (secure 0 0 0) 0x7fa00c0aaa00 con 0x7fa01c15e4f0 2023-08-09T15:11:20.816 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:20.814+0000 7fa0208de640 1 --2- 172.21.15.43:0/1327859374 >> [v2:172.21.15.203:6832/4034093268,v1:172.21.15.203:6833/4034093268] conn(0x7f9ffc06a170 0x7f9ffc06c630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T15:11:20.816 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:20.815+0000 7fa0208de640 1 --2- 172.21.15.43:0/1327859374 >> [v2:172.21.15.203:6832/4034093268,v1:172.21.15.203:6833/4034093268] conn(0x7f9ffc06a170 0x7f9ffc06c630 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7fa004001eb0 tx=0x7fa004009040 comp rx=0 tx=0).ready entity=mgr.8833 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-09T15:11:20.818 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:20.818+0000 7fa0197fa640 1 -- 172.21.15.43:0/1327859374 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178066 (secure 0 0 0) 0x7fa01c0cc7f0 con 0x7fa01c15e4f0 2023-08-09T15:11:21.010 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:21.010+0000 7fa022b42640 1 -- 172.21.15.43:0/1327859374 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7fa01c0cc7f0 con 0x7fa01c15e4f0 2023-08-09T15:11:21.011 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:21.010+0000 7fa0197fa640 1 -- 172.21.15.43:0/1327859374 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v215) v1 ==== 93+0+10 (secure 0 0 0) 0x7fa01c0cc7f0 con 0x7fa01c15e4f0 2023-08-09T15:11:21.011 INFO:teuthology.orchestra.run.smithi043.stdout:pg_num: 1 2023-08-09T15:11:21.013 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:21.013+0000 7fa022b42640 1 -- 172.21.15.43:0/1327859374 >> [v2:172.21.15.203:6832/4034093268,v1:172.21.15.203:6833/4034093268] conn(0x7f9ffc06a170 msgr2=0x7f9ffc06c630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T15:11:21.013 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:21.013+0000 7fa022b42640 1 --2- 172.21.15.43:0/1327859374 >> [v2:172.21.15.203:6832/4034093268,v1:172.21.15.203:6833/4034093268] conn(0x7f9ffc06a170 0x7f9ffc06c630 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7fa004001eb0 tx=0x7fa004009040 comp rx=0 tx=0).stop 2023-08-09T15:11:21.013 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:21.013+0000 7fa022b42640 1 -- 172.21.15.43:0/1327859374 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa01c15e4f0 msgr2=0x7fa01c0aa7b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T15:11:21.014 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:21.013+0000 7fa022b42640 1 --2- 172.21.15.43:0/1327859374 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa01c15e4f0 0x7fa01c0aa7b0 secure :-1 s=READY pgs=2357 cs=0 l=1 rev1=1 crypto rx=0x7fa00c008fd0 tx=0x7fa00c001120 comp rx=0 tx=0).stop 2023-08-09T15:11:21.014 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:21.013+0000 7fa022b42640 1 -- 172.21.15.43:0/1327859374 shutdown_connections 2023-08-09T15:11:21.014 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:21.013+0000 7fa022b42640 1 --2- 172.21.15.43:0/1327859374 >> [v2:172.21.15.203:6832/4034093268,v1:172.21.15.203:6833/4034093268] conn(0x7f9ffc06a170 0x7f9ffc06c630 unknown :-1 s=CLOSED pgs=42 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T15:11:21.014 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:21.013+0000 7fa022b42640 1 --2- 172.21.15.43:0/1327859374 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fa01c15e970 0x7fa01c0aacf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T15:11:21.015 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:21.013+0000 7fa022b42640 1 --2- 172.21.15.43:0/1327859374 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa01c15e4f0 0x7fa01c0aa7b0 unknown :-1 s=CLOSED pgs=2357 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T15:11:21.015 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:21.013+0000 7fa022b42640 1 --2- 172.21.15.43:0/1327859374 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fa008003f20 0x7fa01c14cf70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T15:11:21.015 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:21.013+0000 7fa022b42640 1 -- 172.21.15.43:0/1327859374 >> 172.21.15.43:0/1327859374 conn(0x7fa01c071520 msgr2=0x7fa01c15a770 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-09T15:11:21.015 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:21.013+0000 7fa022b42640 1 -- 172.21.15.43:0/1327859374 shutdown_connections 2023-08-09T15:11:21.015 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:21.013+0000 7fa022b42640 1 -- 172.21.15.43:0/1327859374 wait complete. 2023-08-09T15:11:21.036 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mon mon_allow_pool_delete True 2023-08-09T15:11:21.253 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:21.253+0000 7f6080d2f640 1 Processor -- start 2023-08-09T15:11:21.254 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:21.253+0000 7f6080d2f640 1 -- start start 2023-08-09T15:11:21.255 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:21.254+0000 7f6080d2f640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f607c0d0a70 0x7f607c0cd7f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T15:11:21.255 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:21.254+0000 7f6080d2f640 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7f607c0cdd30 0x7f607c0b3fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T15:11:21.255 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:21.254+0000 7f6080d2f640 1 -- --> v1:172.21.15.203:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f607c0d0f70 con 0x7f607c15c680 2023-08-09T15:11:21.255 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:21.254+0000 7f6080d2f640 1 -- --> v2:172.21.15.203:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f607c0ccd10 con 0x7f607c0cdd30 2023-08-09T15:11:21.255 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:21.254+0000 7f6080d2f640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f607c0ccff0 con 0x7f607c0d0a70 2023-08-09T15:11:21.256 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:21.255+0000 7f6079d9b640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f607c0d0a70 0x7f607c0cd7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T15:11:21.256 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:21.255+0000 7f6079d9b640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f607c0d0a70 0x7f607c0cd7f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.203:3301/0 says I am v2:172.21.15.43:48276/0 (socket says 172.21.15.43:48276) 2023-08-09T15:11:21.256 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:21.255+0000 7f6079d9b640 1 -- 172.21.15.43:0/3747366629 learned_addr learned my addr 172.21.15.43:0/3747366629 (peer_addr_for_me v2:172.21.15.43:0/0) 2023-08-09T15:11:21.257 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:21.255+0000 7f607ad9d640 1 --2- 172.21.15.43:0/3747366629 >> v2:172.21.15.203:3300/0 conn(0x7f607c0cdd30 0x7f607c0b3fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T15:11:21.257 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:21.256+0000 7f6079d9b640 1 -- 172.21.15.43:0/3747366629 >> v2:172.21.15.203:3300/0 conn(0x7f607c0cdd30 msgr2=0x7f607c0b3fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-09T15:11:21.257 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:21.256+0000 7f6079d9b640 1 --2- 172.21.15.43:0/3747366629 >> v2:172.21.15.203:3300/0 conn(0x7f607c0cdd30 0x7f607c0b3fa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T15:11:21.257 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:21.256+0000 7f6079d9b640 1 -- 172.21.15.43:0/3747366629 >> v1:172.21.15.203:6789/0 conn(0x7f607c15c680 legacy=0x7f607c0ce760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T15:11:21.257 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:21.256+0000 7f6079d9b640 1 -- 172.21.15.43:0/3747366629 --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f607c0cd2d0 con 0x7f607c0d0a70 2023-08-09T15:11:21.257 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:21.257+0000 7f6079d9b640 1 --2- 172.21.15.43:0/3747366629 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f607c0d0a70 0x7f607c0cd7f0 secure :-1 s=READY pgs=331 cs=0 l=1 rev1=1 crypto rx=0x7f6070009870 tx=0x7f607002b980 comp rx=0 tx=0).ready entity=mon.2 client_cookie=1b31eee469ae5919 server_cookie=0 in_seq=0 out_seq=0 2023-08-09T15:11:21.258 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:21.258+0000 7f607959a640 1 -- 172.21.15.43:0/3747366629 <== mon.2 v2:172.21.15.203:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6070030840 con 0x7f607c0d0a70 2023-08-09T15:11:21.258 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:21.258+0000 7f607959a640 1 -- 172.21.15.43:0/3747366629 <== mon.2 v2:172.21.15.203:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f60700309a0 con 0x7f607c0d0a70 2023-08-09T15:11:21.258 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:21.258+0000 7f607959a640 1 -- 172.21.15.43:0/3747366629 <== mon.2 v2:172.21.15.203:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6070030d30 con 0x7f607c0d0a70 2023-08-09T15:11:21.259 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:21.258+0000 7f6080d2f640 1 -- 172.21.15.43:0/3747366629 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f607c0d0a70 msgr2=0x7f607c0cd7f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T15:11:21.259 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:21.258+0000 7f6080d2f640 1 --2- 172.21.15.43:0/3747366629 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f607c0d0a70 0x7f607c0cd7f0 secure :-1 s=READY pgs=331 cs=0 l=1 rev1=1 crypto rx=0x7f6070009870 tx=0x7f607002b980 comp rx=0 tx=0).stop 2023-08-09T15:11:21.259 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:21.259+0000 7f6080d2f640 1 -- 172.21.15.43:0/3747366629 shutdown_connections 2023-08-09T15:11:21.259 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:21.259+0000 7f6080d2f640 1 --2- 172.21.15.43:0/3747366629 >> v2:172.21.15.203:3300/0 conn(0x7f607c0cdd30 0x7f607c0b3fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T15:11:21.260 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:21.259+0000 7f6080d2f640 1 --2- 172.21.15.43:0/3747366629 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f607c0d0a70 0x7f607c0cd7f0 unknown :-1 s=CLOSED pgs=331 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T15:11:21.260 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:21.259+0000 7f6080d2f640 1 -- 172.21.15.43:0/3747366629 >> 172.21.15.43:0/3747366629 conn(0x7f607c071520 msgr2=0x7f607c0d2520 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-09T15:11:21.260 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:21.259+0000 7f6080d2f640 1 -- 172.21.15.43:0/3747366629 shutdown_connections 2023-08-09T15:11:21.260 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:21.259+0000 7f6080d2f640 1 -- 172.21.15.43:0/3747366629 wait complete. 2023-08-09T15:11:21.260 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:21.260+0000 7f6080d2f640 1 Processor -- start 2023-08-09T15:11:21.261 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:21.261+0000 7f6080d2f640 1 -- start start 2023-08-09T15:11:21.262 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:21.262+0000 7f6080d2f640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f607c0d0a70 0x7f607c152600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T15:11:21.262 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:21.262+0000 7f6080d2f640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f607c15c680 0x7f607c159ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T15:11:21.262 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:21.262+0000 7f6080d2f640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f607c0b3e80 0x7f607c15a530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T15:11:21.262 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:21.262+0000 7f6080d2f640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f607c15ce20 con 0x7f607c0b3e80 2023-08-09T15:11:21.263 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:21.262+0000 7f6080d2f640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f607c15d100 con 0x7f607c0d0a70 2023-08-09T15:11:21.263 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:21.262+0000 7f6080d2f640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f607c15d3e0 con 0x7f607c15c680 2023-08-09T15:11:21.263 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:21.262+0000 7f607ad9d640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f607c0b3e80 0x7f607c15a530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T15:11:21.263 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:21.262+0000 7f607ad9d640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f607c0b3e80 0x7f607c15a530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:37852/0 (socket says 172.21.15.43:37852) 2023-08-09T15:11:21.263 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:21.262+0000 7f607ad9d640 1 -- 172.21.15.43:0/1661105649 learned_addr learned my addr 172.21.15.43:0/1661105649 (peer_addr_for_me v2:172.21.15.43:0/0) 2023-08-09T15:11:21.264 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:21.262+0000 7f6079d9b640 1 --2- 172.21.15.43:0/1661105649 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f607c15c680 0x7f607c159ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T15:11:21.264 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:21.262+0000 7f607a59c640 1 --2- 172.21.15.43:0/1661105649 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f607c0d0a70 0x7f607c152600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T15:11:21.264 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:21.262+0000 7f607ad9d640 1 -- 172.21.15.43:0/1661105649 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f607c15c680 msgr2=0x7f607c159ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T15:11:21.265 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:21.262+0000 7f607ad9d640 1 --2- 172.21.15.43:0/1661105649 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f607c15c680 0x7f607c159ff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T15:11:21.265 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:21.262+0000 7f607ad9d640 1 -- 172.21.15.43:0/1661105649 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f607c0d0a70 msgr2=0x7f607c152600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T15:11:21.265 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:21.262+0000 7f607ad9d640 1 --2- 172.21.15.43:0/1661105649 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f607c0d0a70 0x7f607c152600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T15:11:21.265 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:21.262+0000 7f607ad9d640 1 -- 172.21.15.43:0/1661105649 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f60700094d0 con 0x7f607c0b3e80 2023-08-09T15:11:21.265 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:21.263+0000 7f607ad9d640 1 --2- 172.21.15.43:0/1661105649 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f607c0b3e80 0x7f607c15a530 secure :-1 s=READY pgs=2358 cs=0 l=1 rev1=1 crypto rx=0x7f607400e930 tx=0x7f607400edf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-09T15:11:21.265 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:21.263+0000 7f606affd640 1 -- 172.21.15.43:0/1661105649 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6074009690 con 0x7f607c0b3e80 2023-08-09T15:11:21.265 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:21.263+0000 7f606affd640 1 -- 172.21.15.43:0/1661105649 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f607400c980 con 0x7f607c0b3e80 2023-08-09T15:11:21.266 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:21.263+0000 7f606affd640 1 -- 172.21.15.43:0/1661105649 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f607400cd30 con 0x7f607c0b3e80 2023-08-09T15:11:21.266 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:21.263+0000 7f6080d2f640 1 -- 172.21.15.43:0/1661105649 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f607c15d6c0 con 0x7f607c0b3e80 2023-08-09T15:11:21.266 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:21.263+0000 7f6080d2f640 1 -- 172.21.15.43:0/1661105649 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f607c152b40 con 0x7f607c0b3e80 2023-08-09T15:11:21.266 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:21.264+0000 7f6080d2f640 1 -- 172.21.15.43:0/1661105649 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f607c15dbe0 con 0x7f607c0b3e80 2023-08-09T15:11:21.266 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:21.265+0000 7f606affd640 1 -- 172.21.15.43:0/1661105649 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 118) v1 ==== 88443+0+0 (secure 0 0 0) 0x7f6074013470 con 0x7f607c0b3e80 2023-08-09T15:11:21.267 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:21.266+0000 7f606affd640 1 --2- 172.21.15.43:0/1661105649 >> [v2:172.21.15.203:6832/4034093268,v1:172.21.15.203:6833/4034093268] conn(0x7f605c06a170 0x7f605c06c630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T15:11:21.267 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:21.267+0000 7f606affd640 1 -- 172.21.15.43:0/1661105649 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(215..215 src has 1..215) v4 ==== 6348+0+0 (secure 0 0 0) 0x7f6074088af0 con 0x7f607c0b3e80 2023-08-09T15:11:21.267 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:21.267+0000 7f607a59c640 1 --2- 172.21.15.43:0/1661105649 >> [v2:172.21.15.203:6832/4034093268,v1:172.21.15.203:6833/4034093268] conn(0x7f605c06a170 0x7f605c06c630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T15:11:21.268 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:21.268+0000 7f607a59c640 1 --2- 172.21.15.43:0/1661105649 >> [v2:172.21.15.203:6832/4034093268,v1:172.21.15.203:6833/4034093268] conn(0x7f605c06a170 0x7f605c06c630 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7f606c0012e0 tx=0x7f606c009040 comp rx=0 tx=0).ready entity=mgr.8833 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-09T15:11:21.271 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:21.271+0000 7f606affd640 1 -- 172.21.15.43:0/1661105649 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178066 (secure 0 0 0) 0x7f6074057a60 con 0x7f607c0b3e80 2023-08-09T15:11:21.469 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:21.469+0000 7f6080d2f640 1 -- 172.21.15.43:0/1661105649 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command([{prefix=config set, name=mon_allow_pool_delete}] v 0) v1 -- 0x7f607c0739f0 con 0x7f607c0b3e80 2023-08-09T15:11:21.477 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:21.477+0000 7f606affd640 1 -- 172.21.15.43:0/1661105649 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_allow_pool_delete}]=0 v70)=0 v70) v1 ==== 122+0+0 (secure 0 0 0) 0x7f60740575f0 con 0x7f607c0b3e80 2023-08-09T15:11:21.480 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:21.480+0000 7f6080d2f640 1 -- 172.21.15.43:0/1661105649 >> [v2:172.21.15.203:6832/4034093268,v1:172.21.15.203:6833/4034093268] conn(0x7f605c06a170 msgr2=0x7f605c06c630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T15:11:21.481 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:21.480+0000 7f6080d2f640 1 --2- 172.21.15.43:0/1661105649 >> [v2:172.21.15.203:6832/4034093268,v1:172.21.15.203:6833/4034093268] conn(0x7f605c06a170 0x7f605c06c630 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7f606c0012e0 tx=0x7f606c009040 comp rx=0 tx=0).stop 2023-08-09T15:11:21.481 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:21.480+0000 7f6080d2f640 1 -- 172.21.15.43:0/1661105649 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f607c0b3e80 msgr2=0x7f607c15a530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T15:11:21.481 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:21.480+0000 7f6080d2f640 1 --2- 172.21.15.43:0/1661105649 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f607c0b3e80 0x7f607c15a530 secure :-1 s=READY pgs=2358 cs=0 l=1 rev1=1 crypto rx=0x7f607400e930 tx=0x7f607400edf0 comp rx=0 tx=0).stop 2023-08-09T15:11:21.481 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:21.480+0000 7f6080d2f640 1 -- 172.21.15.43:0/1661105649 shutdown_connections 2023-08-09T15:11:21.481 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:21.480+0000 7f6080d2f640 1 --2- 172.21.15.43:0/1661105649 >> [v2:172.21.15.203:6832/4034093268,v1:172.21.15.203:6833/4034093268] conn(0x7f605c06a170 0x7f605c06c630 unknown :-1 s=CLOSED pgs=43 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T15:11:21.481 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:21.480+0000 7f6080d2f640 1 --2- 172.21.15.43:0/1661105649 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f607c0b3e80 0x7f607c15a530 unknown :-1 s=CLOSED pgs=2358 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T15:11:21.481 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:21.480+0000 7f6080d2f640 1 --2- 172.21.15.43:0/1661105649 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f607c15c680 0x7f607c159ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T15:11:21.482 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:21.480+0000 7f6080d2f640 1 --2- 172.21.15.43:0/1661105649 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f607c0d0a70 0x7f607c152600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T15:11:21.482 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:21.480+0000 7f6080d2f640 1 -- 172.21.15.43:0/1661105649 >> 172.21.15.43:0/1661105649 conn(0x7f607c071520 msgr2=0x7f607c0cf460 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-09T15:11:21.482 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:21.480+0000 7f6080d2f640 1 -- 172.21.15.43:0/1661105649 shutdown_connections 2023-08-09T15:11:21.482 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:21.480+0000 7f6080d2f640 1 -- 172.21.15.43:0/1661105649 wait complete. 2023-08-09T15:11:21.501 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2023-08-09T15:11:21.501 DEBUG:teuthology.orchestra.run.smithi043:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2023-08-09T15:11:21.628 DEBUG:teuthology.orchestra.run.smithi203:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2023-08-09T15:11:21.755 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2023-08-09T15:11:21.755 DEBUG:teuthology.orchestra.run.smithi043:> set -ex 2023-08-09T15:11:21.756 DEBUG:teuthology.orchestra.run.smithi043:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-09T15:11:21.819 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4054441,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6498844k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=27,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3249420k,nr_inodes=812355,mode=700,uid=1000,gid=1275 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6498844k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2023-08-09T15:11:21.819 DEBUG:teuthology.orchestra.run.smithi203:> set -ex 2023-08-09T15:11:21.820 DEBUG:teuthology.orchestra.run.smithi203:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-09T15:11:21.881 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4053648,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497576k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=27,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248784k,nr_inodes=812196,mode=700,uid=1000,gid=1275 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497576k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2023-08-09T15:11:21.881 DEBUG:teuthology.orchestra.run.smithi043:> set -ex 2023-08-09T15:11:21.882 DEBUG:teuthology.orchestra.run.smithi043:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-09T15:11:21.944 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4054441,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6498844k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=27,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3249420k,nr_inodes=812355,mode=700,uid=1000,gid=1275 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6498844k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2023-08-09T15:11:21.945 DEBUG:teuthology.orchestra.run.smithi203:> set -ex 2023-08-09T15:11:21.945 DEBUG:teuthology.orchestra.run.smithi203:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-09T15:11:22.003 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4053648,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497576k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=27,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248784k,nr_inodes=812196,mode=700,uid=1000,gid=1275 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497576k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2023-08-09T15:11:22.003 INFO:tasks.cephfs.cephfs_test_case:[, , , ] 2023-08-09T15:11:22.003 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-08-09T15:11:22.244 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:22.244+0000 7f388258b640 1 Processor -- start 2023-08-09T15:11:22.245 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:22.244+0000 7f388258b640 1 -- start start 2023-08-09T15:11:22.246 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:22.245+0000 7f388258b640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f387c0b2790 0x7f387c0d2ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T15:11:22.246 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:22.245+0000 7f388258b640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f387c0b2b70 0x7f387c0b5330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T15:11:22.246 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:22.245+0000 7f388258b640 1 -- --> v1:172.21.15.43:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f387c0ce090 con 0x7f387c0b3050 2023-08-09T15:11:22.246 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:22.245+0000 7f388258b640 1 -- --> v2:172.21.15.43:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f387c0b3950 con 0x7f387c0b2790 2023-08-09T15:11:22.246 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:22.245+0000 7f388258b640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f387c0cd7f0 con 0x7f387c0b2b70 2023-08-09T15:11:22.247 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:22.246+0000 7f387bfff640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f387c0b2790 0x7f387c0d2ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T15:11:22.247 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:22.246+0000 7f387bfff640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f387c0b2790 0x7f387c0d2ed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:37854/0 (socket says 172.21.15.43:37854) 2023-08-09T15:11:22.247 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:22.246+0000 7f387bfff640 1 -- 172.21.15.43:0/2327327480 learned_addr learned my addr 172.21.15.43:0/2327327480 (peer_addr_for_me v2:172.21.15.43:0/0) 2023-08-09T15:11:22.248 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:22.246+0000 7f3880b28640 1 --1- 172.21.15.43:0/2327327480 >> v1:172.21.15.43:6789/0 conn(0x7f387c0b3050 0x7f387c0b3f60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.43:6789/0 says I am v1:172.21.15.43:45590/0 (socket says 172.21.15.43:45590) 2023-08-09T15:11:22.248 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:22.246+0000 7f387b7fe640 1 --2- 172.21.15.43:0/2327327480 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f387c0b2b70 0x7f387c0b5330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T15:11:22.248 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:22.246+0000 7f387bfff640 1 -- 172.21.15.43:0/2327327480 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f387c0b2b70 msgr2=0x7f387c0b5330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-09T15:11:22.248 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:22.246+0000 7f387bfff640 1 --2- 172.21.15.43:0/2327327480 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f387c0b2b70 0x7f387c0b5330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T15:11:22.248 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:22.246+0000 7f387bfff640 1 -- 172.21.15.43:0/2327327480 >> v1:172.21.15.43:6789/0 conn(0x7f387c0b3050 legacy=0x7f387c0b3f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T15:11:22.248 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:22.246+0000 7f387bfff640 1 -- 172.21.15.43:0/2327327480 --> v2:172.21.15.43:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f387c0cdad0 con 0x7f387c0b2790 2023-08-09T15:11:22.249 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:22.248+0000 7f387bfff640 1 --2- 172.21.15.43:0/2327327480 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f387c0b2790 0x7f387c0d2ed0 secure :-1 s=READY pgs=2360 cs=0 l=1 rev1=1 crypto rx=0x7f386c009e50 tx=0x7f386c02bf60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c9e015b105e337ba server_cookie=0 in_seq=0 out_seq=0 2023-08-09T15:11:22.249 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:22.248+0000 7f387affd640 1 -- 172.21.15.43:0/2327327480 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f386c02cb00 con 0x7f387c0b2790 2023-08-09T15:11:22.250 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:22.248+0000 7f387affd640 1 -- 172.21.15.43:0/2327327480 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f386c02cc60 con 0x7f387c0b2790 2023-08-09T15:11:22.250 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:22.248+0000 7f387affd640 1 -- 172.21.15.43:0/2327327480 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f386c0343f0 con 0x7f387c0b2790 2023-08-09T15:11:22.250 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:22.248+0000 7f388258b640 1 -- 172.21.15.43:0/2327327480 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f387c0b2790 msgr2=0x7f387c0d2ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T15:11:22.250 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:22.248+0000 7f388258b640 1 --2- 172.21.15.43:0/2327327480 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f387c0b2790 0x7f387c0d2ed0 secure :-1 s=READY pgs=2360 cs=0 l=1 rev1=1 crypto rx=0x7f386c009e50 tx=0x7f386c02bf60 comp rx=0 tx=0).stop 2023-08-09T15:11:22.250 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:22.249+0000 7f388258b640 1 -- 172.21.15.43:0/2327327480 shutdown_connections 2023-08-09T15:11:22.250 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:22.249+0000 7f388258b640 1 --2- 172.21.15.43:0/2327327480 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f387c0b2b70 0x7f387c0b5330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T15:11:22.250 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:22.249+0000 7f388258b640 1 --2- 172.21.15.43:0/2327327480 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f387c0b2790 0x7f387c0d2ed0 unknown :-1 s=CLOSED pgs=2360 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T15:11:22.250 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:22.249+0000 7f388258b640 1 -- 172.21.15.43:0/2327327480 >> 172.21.15.43:0/2327327480 conn(0x7f387c09b7b0 msgr2=0x7f387c0d24e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-09T15:11:22.251 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:22.249+0000 7f388258b640 1 -- 172.21.15.43:0/2327327480 shutdown_connections 2023-08-09T15:11:22.251 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:22.249+0000 7f388258b640 1 -- 172.21.15.43:0/2327327480 wait complete. 2023-08-09T15:11:22.251 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:22.250+0000 7f388258b640 1 Processor -- start 2023-08-09T15:11:22.251 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:22.251+0000 7f388258b640 1 -- start start 2023-08-09T15:11:22.252 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:22.252+0000 7f388258b640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f387c0b2790 0x7f387c147510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T15:11:22.252 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:22.252+0000 7f388258b640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f387c0b2b70 0x7f387c06ab90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T15:11:22.253 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:22.252+0000 7f388258b640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f387c0b3050 0x7f387c06b0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T15:11:22.253 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:22.252+0000 7f388258b640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f387c0c8e60 con 0x7f387c0b2b70 2023-08-09T15:11:22.253 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:22.252+0000 7f388258b640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f387c147a50 con 0x7f387c0b2790 2023-08-09T15:11:22.253 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:22.252+0000 7f388258b640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f387c0c8250 con 0x7f387c0b3050 2023-08-09T15:11:22.253 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:22.252+0000 7f387b7fe640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f387c0b2b70 0x7f387c06ab90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T15:11:22.253 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:22.252+0000 7f387b7fe640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f387c0b2b70 0x7f387c06ab90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:37856/0 (socket says 172.21.15.43:37856) 2023-08-09T15:11:22.253 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:22.252+0000 7f387b7fe640 1 -- 172.21.15.43:0/2604008409 learned_addr learned my addr 172.21.15.43:0/2604008409 (peer_addr_for_me v2:172.21.15.43:0/0) 2023-08-09T15:11:22.253 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:22.252+0000 7f387bfff640 1 --2- 172.21.15.43:0/2604008409 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f387c0b2790 0x7f387c147510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T15:11:22.255 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:22.252+0000 7f387b7fe640 1 -- 172.21.15.43:0/2604008409 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f387c0b3050 msgr2=0x7f387c06b0d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T15:11:22.256 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:22.252+0000 7f3880b28640 1 --2- 172.21.15.43:0/2604008409 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f387c0b3050 0x7f387c06b0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T15:11:22.256 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:22.252+0000 7f387b7fe640 1 --2- 172.21.15.43:0/2604008409 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f387c0b3050 0x7f387c06b0d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T15:11:22.256 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:22.253+0000 7f387b7fe640 1 -- 172.21.15.43:0/2604008409 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f387c0b2790 msgr2=0x7f387c147510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T15:11:22.256 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:22.253+0000 7f387b7fe640 1 --2- 172.21.15.43:0/2604008409 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f387c0b2790 0x7f387c147510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T15:11:22.257 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:22.253+0000 7f387b7fe640 1 -- 172.21.15.43:0/2604008409 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f386c009b30 con 0x7f387c0b2b70 2023-08-09T15:11:22.257 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:22.253+0000 7f387b7fe640 1 --2- 172.21.15.43:0/2604008409 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f387c0b2b70 0x7f387c06ab90 secure :-1 s=READY pgs=2361 cs=0 l=1 rev1=1 crypto rx=0x7f386400d5f0 tx=0x7f386400dab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-09T15:11:22.257 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:22.253+0000 7f3878ff9640 1 -- 172.21.15.43:0/2604008409 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f38640125f0 con 0x7f387c0b2b70 2023-08-09T15:11:22.258 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:22.253+0000 7f3878ff9640 1 -- 172.21.15.43:0/2604008409 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3864012750 con 0x7f387c0b2b70 2023-08-09T15:11:22.258 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:22.253+0000 7f3878ff9640 1 -- 172.21.15.43:0/2604008409 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3864012b00 con 0x7f387c0b2b70 2023-08-09T15:11:22.258 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:22.253+0000 7f388258b640 1 -- 172.21.15.43:0/2604008409 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f387c0c8530 con 0x7f387c0b2b70 2023-08-09T15:11:22.259 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:22.253+0000 7f388258b640 1 -- 172.21.15.43:0/2604008409 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f387c0c8af0 con 0x7f387c0b2b70 2023-08-09T15:11:22.259 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:22.255+0000 7f388258b640 1 -- 172.21.15.43:0/2604008409 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f387c0c8fd0 con 0x7f387c0b2b70 2023-08-09T15:11:22.260 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:22.256+0000 7f3878ff9640 1 -- 172.21.15.43:0/2604008409 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 118) v1 ==== 88443+0+0 (secure 0 0 0) 0x7f3864012c60 con 0x7f387c0b2b70 2023-08-09T15:11:22.260 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:22.257+0000 7f3878ff9640 1 --2- 172.21.15.43:0/2604008409 >> [v2:172.21.15.203:6832/4034093268,v1:172.21.15.203:6833/4034093268] conn(0x7f385c06a170 0x7f385c06c630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T15:11:22.260 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:22.257+0000 7f3878ff9640 1 -- 172.21.15.43:0/2604008409 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(215..215 src has 1..215) v4 ==== 6348+0+0 (secure 0 0 0) 0x7f3864088d40 con 0x7f387c0b2b70 2023-08-09T15:11:22.260 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:22.258+0000 7f387bfff640 1 --2- 172.21.15.43:0/2604008409 >> [v2:172.21.15.203:6832/4034093268,v1:172.21.15.203:6833/4034093268] conn(0x7f385c06a170 0x7f385c06c630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T15:11:22.260 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:22.259+0000 7f387bfff640 1 --2- 172.21.15.43:0/2604008409 >> [v2:172.21.15.203:6832/4034093268,v1:172.21.15.203:6833/4034093268] conn(0x7f385c06a170 0x7f385c06c630 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7f386c035000 tx=0x7f386c030000 comp rx=0 tx=0).ready entity=mgr.8833 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-09T15:11:22.262 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:22.262+0000 7f3878ff9640 1 -- 172.21.15.43:0/2604008409 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178066 (secure 0 0 0) 0x7f3864054160 con 0x7f387c0b2b70 2023-08-09T15:11:22.493 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:22.493+0000 7f388258b640 1 -- 172.21.15.43:0/2604008409 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f387c0739f0 con 0x7f387c0b2b70 2023-08-09T15:11:22.494 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:22.494+0000 7f3878ff9640 1 -- 172.21.15.43:0/2604008409 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 98 v98) v1 ==== 94+0+3704 (secure 0 0 0) 0x7f3864057780 con 0x7f387c0b2b70 2023-08-09T15:11:22.495 INFO:teuthology.orchestra.run.smithi043.stdout: 2023-08-09T15:11:22.495 INFO:teuthology.orchestra.run.smithi043.stdout:{"epoch":98,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":7703,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.203:6836/1774059384","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6835","nonce":1774059384},{"type":"v1","addr":"172.21.15.203:6836","nonce":1774059384}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":91},{"gid":8665,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.203:6837/682905899","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6834","nonce":682905899},{"type":"v1","addr":"172.21.15.203:6837","nonce":682905899}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":91},{"gid":10752,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.43:6834/4168463642","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6832","nonce":4168463642},{"type":"v1","addr":"172.21.15.43:6834","nonce":4168463642}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":90},{"gid":11232,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.43:6835/3231245356","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6833","nonce":3231245356},{"type":"v1","addr":"172.21.15.43:6835","nonce":3231245356}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":98}],"filesystems":[]} 2023-08-09T15:11:22.495 INFO:teuthology.orchestra.run.smithi043.stderr:dumped fsmap epoch 98 2023-08-09T15:11:22.497 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:22.497+0000 7f388258b640 1 -- 172.21.15.43:0/2604008409 >> [v2:172.21.15.203:6832/4034093268,v1:172.21.15.203:6833/4034093268] conn(0x7f385c06a170 msgr2=0x7f385c06c630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T15:11:22.497 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:22.497+0000 7f388258b640 1 --2- 172.21.15.43:0/2604008409 >> [v2:172.21.15.203:6832/4034093268,v1:172.21.15.203:6833/4034093268] conn(0x7f385c06a170 0x7f385c06c630 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7f386c035000 tx=0x7f386c030000 comp rx=0 tx=0).stop 2023-08-09T15:11:22.498 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:22.497+0000 7f388258b640 1 -- 172.21.15.43:0/2604008409 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f387c0b2b70 msgr2=0x7f387c06ab90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T15:11:22.498 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:22.497+0000 7f388258b640 1 --2- 172.21.15.43:0/2604008409 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f387c0b2b70 0x7f387c06ab90 secure :-1 s=READY pgs=2361 cs=0 l=1 rev1=1 crypto rx=0x7f386400d5f0 tx=0x7f386400dab0 comp rx=0 tx=0).stop 2023-08-09T15:11:22.498 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:22.497+0000 7f388258b640 1 -- 172.21.15.43:0/2604008409 shutdown_connections 2023-08-09T15:11:22.498 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:22.497+0000 7f388258b640 1 --2- 172.21.15.43:0/2604008409 >> [v2:172.21.15.203:6832/4034093268,v1:172.21.15.203:6833/4034093268] conn(0x7f385c06a170 0x7f385c06c630 unknown :-1 s=CLOSED pgs=44 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T15:11:22.499 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:22.497+0000 7f388258b640 1 --2- 172.21.15.43:0/2604008409 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f387c0b3050 0x7f387c06b0d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T15:11:22.499 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:22.497+0000 7f388258b640 1 --2- 172.21.15.43:0/2604008409 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f387c0b2b70 0x7f387c06ab90 unknown :-1 s=CLOSED pgs=2361 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T15:11:22.499 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:22.497+0000 7f388258b640 1 --2- 172.21.15.43:0/2604008409 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f387c0b2790 0x7f387c147510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T15:11:22.499 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:22.497+0000 7f388258b640 1 -- 172.21.15.43:0/2604008409 >> 172.21.15.43:0/2604008409 conn(0x7f387c09b7b0 msgr2=0x7f387c0ceac0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-09T15:11:22.499 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:22.497+0000 7f388258b640 1 -- 172.21.15.43:0/2604008409 shutdown_connections 2023-08-09T15:11:22.500 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:22.497+0000 7f388258b640 1 -- 172.21.15.43:0/2604008409 wait complete. 2023-08-09T15:11:22.516 INFO:tasks.ceph.mds.a:Restarting daemon 2023-08-09T15:11:22.516 INFO:tasks.ceph.mds.a:Stopping old one... 2023-08-09T15:11:22.516 DEBUG:tasks.ceph.mds.a:waiting for process to exit 2023-08-09T15:11:22.517 INFO:teuthology.orchestra.run:waiting for 300 2023-08-09T15:11:22.517 INFO:tasks.ceph.mds.c:Restarting daemon 2023-08-09T15:11:22.517 INFO:tasks.ceph.mds.c:Stopping old one... 2023-08-09T15:11:22.517 DEBUG:tasks.ceph.mds.c:waiting for process to exit 2023-08-09T15:11:22.517 INFO:teuthology.orchestra.run:waiting for 300 2023-08-09T15:11:22.518 INFO:tasks.ceph.mds.b:Restarting daemon 2023-08-09T15:11:22.518 INFO:tasks.ceph.mds.b:Stopping old one... 2023-08-09T15:11:22.518 DEBUG:tasks.ceph.mds.b:waiting for process to exit 2023-08-09T15:11:22.518 INFO:teuthology.orchestra.run:waiting for 300 2023-08-09T15:11:22.519 INFO:tasks.ceph.mds.d:Restarting daemon 2023-08-09T15:11:22.519 INFO:tasks.ceph.mds.d:Stopping old one... 2023-08-09T15:11:22.519 DEBUG:tasks.ceph.mds.d:waiting for process to exit 2023-08-09T15:11:22.519 INFO:teuthology.orchestra.run:waiting for 300 2023-08-09T15:11:22.535 INFO:tasks.ceph.mds.a:Stopped 2023-08-09T15:11:22.535 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i a 2023-08-09T15:11:22.537 INFO:tasks.ceph.mds.c:Stopped 2023-08-09T15:11:22.537 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i c 2023-08-09T15:11:22.544 INFO:tasks.ceph.mds.d:Stopped 2023-08-09T15:11:22.545 DEBUG:teuthology.orchestra.run.smithi203:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i d 2023-08-09T15:11:22.553 INFO:tasks.ceph.mds.b:Stopped 2023-08-09T15:11:22.553 DEBUG:teuthology.orchestra.run.smithi203:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i b 2023-08-09T15:11:22.585 INFO:tasks.ceph.mds.a:Started 2023-08-09T15:11:22.586 DEBUG:teuthology.parallel:result is None 2023-08-09T15:11:22.593 INFO:tasks.ceph.mds.c:Started 2023-08-09T15:11:22.593 DEBUG:teuthology.parallel:result is None 2023-08-09T15:11:22.601 INFO:tasks.ceph.mds.d:Started 2023-08-09T15:11:22.602 DEBUG:teuthology.parallel:result is None 2023-08-09T15:11:22.609 INFO:tasks.ceph.mds.b:Started 2023-08-09T15:11:22.609 DEBUG:teuthology.parallel:result is None 2023-08-09T15:11:22.610 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist clear 2023-08-09T15:11:22.757 INFO:tasks.ceph.mds.a.smithi043.stdout:starting mds.a at 2023-08-09T15:11:22.761 INFO:tasks.ceph.mds.c.smithi043.stdout:starting mds.c at 2023-08-09T15:11:22.768 INFO:tasks.ceph.mds.d.smithi203.stdout:starting mds.d at 2023-08-09T15:11:22.774 INFO:tasks.ceph.mds.b.smithi203.stdout:starting mds.b at 2023-08-09T15:11:22.810 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:22.810+0000 7feb9a4d7640 1 Processor -- start 2023-08-09T15:11:22.810 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:22.810+0000 7feb9a4d7640 1 -- start start 2023-08-09T15:11:22.811 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:22.811+0000 7feb9a4d7640 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7feb941546f0 0x7feb9415e640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T15:11:22.812 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:22.811+0000 7feb9a4d7640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7feb9415b460 0x7feb9406b3d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T15:11:22.812 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:22.811+0000 7feb9a4d7640 1 -- --> v1:172.21.15.43:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7feb9406d770 con 0x7feb94070ab0 2023-08-09T15:11:22.813 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:22.811+0000 7feb9a4d7640 1 -- --> v2:172.21.15.43:3300/0 -- mon_getmap magic: 0 v1 -- 0x7feb9406cf30 con 0x7feb9415b460 2023-08-09T15:11:22.813 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:22.811+0000 7feb9a4d7640 1 -- --> v2:172.21.15.203:3300/0 -- mon_getmap magic: 0 v1 -- 0x7feb9406d1b0 con 0x7feb941546f0 2023-08-09T15:11:22.813 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:22.811+0000 7feb98cd4640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7feb9415b460 0x7feb9406b3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T15:11:22.813 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:22.811+0000 7feb99cd6640 1 --1- >> v1:172.21.15.43:6789/0 conn(0x7feb94070ab0 0x7feb940c84b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.43:6789/0 says I am v1:172.21.15.43:45618/0 (socket says 172.21.15.43:45618) 2023-08-09T15:11:22.813 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:22.811+0000 7feb98cd4640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7feb9415b460 0x7feb9406b3d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:37880/0 (socket says 172.21.15.43:37880) 2023-08-09T15:11:22.813 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:22.811+0000 7feb99cd6640 1 -- 172.21.15.43:0/1025859320 learned_addr learned my addr 172.21.15.43:0/1025859320 (peer_addr_for_me v1:172.21.15.43:0/0) 2023-08-09T15:11:22.813 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:22.811+0000 7feb994d5640 1 --2- 172.21.15.43:0/1025859320 >> v2:172.21.15.203:3300/0 conn(0x7feb941546f0 0x7feb9415e640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T15:11:22.815 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:22.812+0000 7feb98cd4640 1 -- 172.21.15.43:0/1025859320 >> v2:172.21.15.203:3300/0 conn(0x7feb941546f0 msgr2=0x7feb9415e640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-09T15:11:22.815 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:22.812+0000 7feb98cd4640 1 --2- 172.21.15.43:0/1025859320 >> v2:172.21.15.203:3300/0 conn(0x7feb941546f0 0x7feb9415e640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T15:11:22.815 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:22.812+0000 7feb98cd4640 1 -- 172.21.15.43:0/1025859320 >> v1:172.21.15.43:6789/0 conn(0x7feb94070ab0 legacy=0x7feb940c84b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T15:11:22.815 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:22.812+0000 7feb98cd4640 1 -- 172.21.15.43:0/1025859320 --> v2:172.21.15.43:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7feb9406d490 con 0x7feb9415b460 2023-08-09T15:11:22.815 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:22.812+0000 7feb8bfff640 1 -- 172.21.15.43:0/1025859320 <== mon.0 v1:172.21.15.43:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4154099719 0 0) 0x7feb8c001110 con 0x7feb94070ab0 2023-08-09T15:11:22.816 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:22.813+0000 7feb98cd4640 1 --2- 172.21.15.43:0/1025859320 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7feb9415b460 0x7feb9406b3d0 secure :-1 s=READY pgs=2368 cs=0 l=1 rev1=1 crypto rx=0x7feb840097a0 tx=0x7feb8402b7b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1e4f5f5460ba2cab server_cookie=0 in_seq=0 out_seq=0 2023-08-09T15:11:22.816 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:22.813+0000 7feb8bfff640 1 -- 172.21.15.43:0/1025859320 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feb84031710 con 0x7feb9415b460 2023-08-09T15:11:22.816 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:22.813+0000 7feb8bfff640 1 -- 172.21.15.43:0/1025859320 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7feb8402f050 con 0x7feb9415b460 2023-08-09T15:11:22.816 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:22.813+0000 7feb8bfff640 1 -- 172.21.15.43:0/1025859320 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feb84031a90 con 0x7feb9415b460 2023-08-09T15:11:22.816 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:22.814+0000 7feb9a4d7640 1 -- 172.21.15.43:0/1025859320 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7feb9415b460 msgr2=0x7feb9406b3d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T15:11:22.816 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:22.814+0000 7feb9a4d7640 1 --2- 172.21.15.43:0/1025859320 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7feb9415b460 0x7feb9406b3d0 secure :-1 s=READY pgs=2368 cs=0 l=1 rev1=1 crypto rx=0x7feb840097a0 tx=0x7feb8402b7b0 comp rx=0 tx=0).stop 2023-08-09T15:11:22.816 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:22.814+0000 7feb9a4d7640 1 -- 172.21.15.43:0/1025859320 shutdown_connections 2023-08-09T15:11:22.816 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:22.814+0000 7feb9a4d7640 1 --2- 172.21.15.43:0/1025859320 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7feb9415b460 0x7feb9406b3d0 unknown :-1 s=CLOSED pgs=2368 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T15:11:22.817 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:22.814+0000 7feb9a4d7640 1 --2- 172.21.15.43:0/1025859320 >> v2:172.21.15.203:3300/0 conn(0x7feb941546f0 0x7feb9415e640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T15:11:22.817 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:22.814+0000 7feb9a4d7640 1 -- 172.21.15.43:0/1025859320 >> 172.21.15.43:0/1025859320 conn(0x7feb940af950 msgr2=0x7feb940afd60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-09T15:11:22.817 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:22.814+0000 7feb9a4d7640 1 -- 172.21.15.43:0/1025859320 shutdown_connections 2023-08-09T15:11:22.817 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:22.814+0000 7feb9a4d7640 1 -- 172.21.15.43:0/1025859320 wait complete. 2023-08-09T15:11:22.819 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:22.816+0000 7feb9a4d7640 1 Processor -- start 2023-08-09T15:11:22.819 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:22.816+0000 7feb9a4d7640 1 -- start start 2023-08-09T15:11:22.819 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:22.817+0000 7feb9a4d7640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7feb94070ab0 0x7feb940b1930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T15:11:22.819 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:22.817+0000 7feb9a4d7640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7feb941546f0 0x7feb9414c3e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T15:11:22.819 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:22.817+0000 7feb9a4d7640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7feb940b3330 0x7feb940c4350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T15:11:22.819 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:22.817+0000 7feb9a4d7640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feb940b1510 con 0x7feb941546f0 2023-08-09T15:11:22.819 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:22.817+0000 7feb9a4d7640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feb9414c920 con 0x7feb94070ab0 2023-08-09T15:11:22.820 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:22.817+0000 7feb9a4d7640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7feb940c4890 con 0x7feb940b3330 2023-08-09T15:11:22.820 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:22.817+0000 7feb98cd4640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7feb941546f0 0x7feb9414c3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T15:11:22.820 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:22.817+0000 7feb98cd4640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7feb941546f0 0x7feb9414c3e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:37896/0 (socket says 172.21.15.43:37896) 2023-08-09T15:11:22.820 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:22.817+0000 7feb98cd4640 1 -- 172.21.15.43:0/3133875453 learned_addr learned my addr 172.21.15.43:0/3133875453 (peer_addr_for_me v2:172.21.15.43:0/0) 2023-08-09T15:11:22.820 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:22.818+0000 7feb994d5640 1 --2- 172.21.15.43:0/3133875453 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7feb94070ab0 0x7feb940b1930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T15:11:22.820 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:22.818+0000 7feb99cd6640 1 --2- 172.21.15.43:0/3133875453 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7feb940b3330 0x7feb940c4350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T15:11:22.820 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:22.818+0000 7feb98cd4640 1 -- 172.21.15.43:0/3133875453 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7feb940b3330 msgr2=0x7feb940c4350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T15:11:22.821 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:22.818+0000 7feb98cd4640 1 --2- 172.21.15.43:0/3133875453 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7feb940b3330 0x7feb940c4350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T15:11:22.821 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:22.818+0000 7feb98cd4640 1 -- 172.21.15.43:0/3133875453 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7feb94070ab0 msgr2=0x7feb940b1930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T15:11:22.821 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:22.818+0000 7feb98cd4640 1 --2- 172.21.15.43:0/3133875453 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7feb94070ab0 0x7feb940b1930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T15:11:22.821 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:22.818+0000 7feb98cd4640 1 -- 172.21.15.43:0/3133875453 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7feb84009400 con 0x7feb941546f0 2023-08-09T15:11:22.821 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:22.818+0000 7feb99cd6640 1 --2- 172.21.15.43:0/3133875453 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7feb940b3330 0x7feb940c4350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-08-09T15:11:22.821 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:22.818+0000 7feb98cd4640 1 --2- 172.21.15.43:0/3133875453 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7feb941546f0 0x7feb9414c3e0 secure :-1 s=READY pgs=2369 cs=0 l=1 rev1=1 crypto rx=0x7feb84000c00 tx=0x7feb840046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-09T15:11:22.821 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:22.818+0000 7feb89ffb640 1 -- 172.21.15.43:0/3133875453 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feb84004230 con 0x7feb941546f0 2023-08-09T15:11:22.822 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:22.819+0000 7feb89ffb640 1 -- 172.21.15.43:0/3133875453 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7feb84004390 con 0x7feb941546f0 2023-08-09T15:11:22.822 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:22.819+0000 7feb89ffb640 1 -- 172.21.15.43:0/3133875453 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feb8403d840 con 0x7feb941546f0 2023-08-09T15:11:22.822 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:22.819+0000 7feb9a4d7640 1 -- 172.21.15.43:0/3133875453 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7feb940b0bb0 con 0x7feb941546f0 2023-08-09T15:11:22.822 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:22.819+0000 7feb9a4d7640 1 -- 172.21.15.43:0/3133875453 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7feb940b1130 con 0x7feb941546f0 2023-08-09T15:11:22.823 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:22.820+0000 7feb9a4d7640 1 -- 172.21.15.43:0/3133875453 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7feb940b1650 con 0x7feb941546f0 2023-08-09T15:11:22.823 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:22.821+0000 7feb89ffb640 1 -- 172.21.15.43:0/3133875453 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 118) v1 ==== 88443+0+0 (secure 0 0 0) 0x7feb8403a050 con 0x7feb941546f0 2023-08-09T15:11:22.823 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:22.822+0000 7feb89ffb640 1 --2- 172.21.15.43:0/3133875453 >> [v2:172.21.15.203:6832/4034093268,v1:172.21.15.203:6833/4034093268] conn(0x7feb7806a170 0x7feb7806c630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T15:11:22.823 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:22.822+0000 7feb89ffb640 1 -- 172.21.15.43:0/3133875453 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(215..215 src has 1..215) v4 ==== 6348+0+0 (secure 0 0 0) 0x7feb840aa3e0 con 0x7feb941546f0 2023-08-09T15:11:22.823 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:22.822+0000 7feb994d5640 1 --2- 172.21.15.43:0/3133875453 >> [v2:172.21.15.203:6832/4034093268,v1:172.21.15.203:6833/4034093268] conn(0x7feb7806a170 0x7feb7806c630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T15:11:22.824 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:22.823+0000 7feb994d5640 1 --2- 172.21.15.43:0/3133875453 >> [v2:172.21.15.203:6832/4034093268,v1:172.21.15.203:6833/4034093268] conn(0x7feb7806a170 0x7feb7806c630 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7feb900074a0 tx=0x7feb9000b040 comp rx=0 tx=0).ready entity=mgr.8833 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-09T15:11:22.826 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:22.826+0000 7feb89ffb640 1 -- 172.21.15.43:0/3133875453 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178066 (secure 0 0 0) 0x7feb940b1650 con 0x7feb941546f0 2023-08-09T15:11:23.034 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:23.034+0000 7feb9a4d7640 1 -- 172.21.15.43:0/3133875453 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "osd blocklist clear"} v 0) v1 -- 0x7feb940b1650 con 0x7feb941546f0 2023-08-09T15:11:23.647 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:23.647+0000 7feb89ffb640 1 -- 172.21.15.43:0/3133875453 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist clear"}]=0 removed all blocklist entries v217) v1 ==== 97+0+0 (secure 0 0 0) 0x7feb940b1650 con 0x7feb941546f0 2023-08-09T15:11:23.647 INFO:teuthology.orchestra.run.smithi043.stderr: removed all blocklist entries 2023-08-09T15:11:23.651 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:23.650+0000 7feb9a4d7640 1 -- 172.21.15.43:0/3133875453 >> [v2:172.21.15.203:6832/4034093268,v1:172.21.15.203:6833/4034093268] conn(0x7feb7806a170 msgr2=0x7feb7806c630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T15:11:23.651 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:23.650+0000 7feb9a4d7640 1 --2- 172.21.15.43:0/3133875453 >> [v2:172.21.15.203:6832/4034093268,v1:172.21.15.203:6833/4034093268] conn(0x7feb7806a170 0x7feb7806c630 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7feb900074a0 tx=0x7feb9000b040 comp rx=0 tx=0).stop 2023-08-09T15:11:23.651 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:23.651+0000 7feb9a4d7640 1 -- 172.21.15.43:0/3133875453 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7feb941546f0 msgr2=0x7feb9414c3e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T15:11:23.651 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:23.651+0000 7feb9a4d7640 1 --2- 172.21.15.43:0/3133875453 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7feb941546f0 0x7feb9414c3e0 secure :-1 s=READY pgs=2369 cs=0 l=1 rev1=1 crypto rx=0x7feb84000c00 tx=0x7feb840046c0 comp rx=0 tx=0).stop 2023-08-09T15:11:23.652 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:23.651+0000 7feb9a4d7640 1 -- 172.21.15.43:0/3133875453 shutdown_connections 2023-08-09T15:11:23.652 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:23.651+0000 7feb9a4d7640 1 --2- 172.21.15.43:0/3133875453 >> [v2:172.21.15.203:6832/4034093268,v1:172.21.15.203:6833/4034093268] conn(0x7feb7806a170 0x7feb7806c630 unknown :-1 s=CLOSED pgs=45 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T15:11:23.652 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:23.651+0000 7feb9a4d7640 1 --2- 172.21.15.43:0/3133875453 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7feb940b3330 0x7feb940c4350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T15:11:23.652 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:23.651+0000 7feb9a4d7640 1 --2- 172.21.15.43:0/3133875453 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7feb941546f0 0x7feb9414c3e0 unknown :-1 s=CLOSED pgs=2369 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T15:11:23.652 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:23.651+0000 7feb9a4d7640 1 --2- 172.21.15.43:0/3133875453 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7feb94070ab0 0x7feb940b1930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T15:11:23.653 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:23.651+0000 7feb9a4d7640 1 -- 172.21.15.43:0/3133875453 >> 172.21.15.43:0/3133875453 conn(0x7feb940af950 msgr2=0x7feb940c9340 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-09T15:11:23.653 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:23.651+0000 7feb9a4d7640 1 -- 172.21.15.43:0/3133875453 shutdown_connections 2023-08-09T15:11:23.653 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:23.651+0000 7feb9a4d7640 1 -- 172.21.15.43:0/3133875453 wait complete. 2023-08-09T15:11:23.674 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth ls --format=json-pretty 2023-08-09T15:11:23.903 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:23.903+0000 7f22d210c640 1 Processor -- start 2023-08-09T15:11:23.903 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:23.903+0000 7f22d210c640 1 -- start start 2023-08-09T15:11:23.905 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:23.904+0000 7f22d210c640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f22cc15c630 0x7f22cc0abef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T15:11:23.905 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:23.905+0000 7f22d210c640 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7f22cc0cdc60 0x7f22cc15ca10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T15:11:23.905 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:23.905+0000 7f22d210c640 1 -- --> v1:172.21.15.43:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f22cc0cca90 con 0x7f22cc0ccf20 2023-08-09T15:11:23.906 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:23.905+0000 7f22d210c640 1 -- --> v2:172.21.15.43:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f22cc0cbc70 con 0x7f22cc15c630 2023-08-09T15:11:23.906 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:23.905+0000 7f22d210c640 1 -- --> v2:172.21.15.203:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f22cc0cbe80 con 0x7f22cc0cdc60 2023-08-09T15:11:23.906 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:23.905+0000 7f22cb7fe640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f22cc15c630 0x7f22cc0abef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T15:11:23.906 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:23.905+0000 7f22c2ffd640 1 --1- >> v1:172.21.15.43:6789/0 conn(0x7f22cc0ccf20 0x7f22cc0cd550 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.43:6789/0 says I am v1:172.21.15.43:45620/0 (socket says 172.21.15.43:45620) 2023-08-09T15:11:23.906 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:23.905+0000 7f22c2ffd640 1 -- 172.21.15.43:0/3864653137 learned_addr learned my addr 172.21.15.43:0/3864653137 (peer_addr_for_me v1:172.21.15.43:0/0) 2023-08-09T15:11:23.906 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:23.905+0000 7f22cb7fe640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f22cc15c630 0x7f22cc0abef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:37902/0 (socket says 172.21.15.43:37902) 2023-08-09T15:11:23.907 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:23.905+0000 7f22cbfff640 1 --2- 172.21.15.43:0/3864653137 >> v2:172.21.15.203:3300/0 conn(0x7f22cc0cdc60 0x7f22cc15ca10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T15:11:23.907 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:23.905+0000 7f22cb7fe640 1 -- 172.21.15.43:0/3864653137 >> v2:172.21.15.203:3300/0 conn(0x7f22cc0cdc60 msgr2=0x7f22cc15ca10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-09T15:11:23.908 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:23.905+0000 7f22cb7fe640 1 --2- 172.21.15.43:0/3864653137 >> v2:172.21.15.203:3300/0 conn(0x7f22cc0cdc60 0x7f22cc15ca10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T15:11:23.908 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:23.906+0000 7f22cb7fe640 1 -- 172.21.15.43:0/3864653137 >> v1:172.21.15.43:6789/0 conn(0x7f22cc0ccf20 legacy=0x7f22cc0cd550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T15:11:23.908 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:23.906+0000 7f22cb7fe640 1 -- 172.21.15.43:0/3864653137 --> v2:172.21.15.43:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f22cc0cc160 con 0x7f22cc15c630 2023-08-09T15:11:23.908 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:23.906+0000 7f22caffd640 1 -- 172.21.15.43:0/3864653137 <== mon.0 v1:172.21.15.43:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1804066507 0 0) 0x7f22cc0cca90 con 0x7f22cc0ccf20 2023-08-09T15:11:23.908 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:23.907+0000 7f22cb7fe640 1 --2- 172.21.15.43:0/3864653137 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f22cc15c630 0x7f22cc0abef0 secure :-1 s=READY pgs=2371 cs=0 l=1 rev1=1 crypto rx=0x7f22cc0ca9b0 tx=0x7f22bc02b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5afe67bbfca09447 server_cookie=0 in_seq=0 out_seq=0 2023-08-09T15:11:23.908 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:23.907+0000 7f22caffd640 1 -- 172.21.15.43:0/3864653137 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f22bc031830 con 0x7f22cc15c630 2023-08-09T15:11:23.909 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:23.907+0000 7f22caffd640 1 -- 172.21.15.43:0/3864653137 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f22bc031990 con 0x7f22cc15c630 2023-08-09T15:11:23.909 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:23.907+0000 7f22caffd640 1 -- 172.21.15.43:0/3864653137 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f22bc031d20 con 0x7f22cc15c630 2023-08-09T15:11:23.909 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:23.908+0000 7f22d210c640 1 -- 172.21.15.43:0/3864653137 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f22cc15c630 msgr2=0x7f22cc0abef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T15:11:23.909 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:23.908+0000 7f22d210c640 1 --2- 172.21.15.43:0/3864653137 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f22cc15c630 0x7f22cc0abef0 secure :-1 s=READY pgs=2371 cs=0 l=1 rev1=1 crypto rx=0x7f22cc0ca9b0 tx=0x7f22bc02b8d0 comp rx=0 tx=0).stop 2023-08-09T15:11:23.910 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:23.908+0000 7f22d210c640 1 -- 172.21.15.43:0/3864653137 shutdown_connections 2023-08-09T15:11:23.910 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:23.908+0000 7f22d210c640 1 --2- 172.21.15.43:0/3864653137 >> v2:172.21.15.203:3300/0 conn(0x7f22cc0cdc60 0x7f22cc15ca10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T15:11:23.910 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:23.908+0000 7f22d210c640 1 --2- 172.21.15.43:0/3864653137 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f22cc15c630 0x7f22cc0abef0 unknown :-1 s=CLOSED pgs=2371 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T15:11:23.910 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:23.908+0000 7f22d210c640 1 -- 172.21.15.43:0/3864653137 >> 172.21.15.43:0/3864653137 conn(0x7f22cc0ce0a0 msgr2=0x7f22cc0ce480 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-09T15:11:23.910 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:23.908+0000 7f22d210c640 1 -- 172.21.15.43:0/3864653137 shutdown_connections 2023-08-09T15:11:23.910 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:23.908+0000 7f22d210c640 1 -- 172.21.15.43:0/3864653137 wait complete. 2023-08-09T15:11:23.911 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:23.909+0000 7f22d210c640 1 Processor -- start 2023-08-09T15:11:23.912 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:23.910+0000 7f22d210c640 1 -- start start 2023-08-09T15:11:23.912 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:23.911+0000 7f22d210c640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f22cc0ccf20 0x7f22cc1584f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T15:11:23.912 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:23.911+0000 7f22d210c640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f22cc0cdc60 0x7f22cc150350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T15:11:23.912 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:23.911+0000 7f22d210c640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f22cc15c630 0x7f22cc0c6300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T15:11:23.912 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:23.911+0000 7f22d210c640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f22cc0c7e10 con 0x7f22cc15c630 2023-08-09T15:11:23.912 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:23.911+0000 7f22d210c640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f22cc159650 con 0x7f22cc0cdc60 2023-08-09T15:11:23.913 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:23.911+0000 7f22d210c640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f22cc158ad0 con 0x7f22cc0ccf20 2023-08-09T15:11:23.913 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:23.911+0000 7f22cbfff640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f22cc15c630 0x7f22cc0c6300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T15:11:23.913 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:23.911+0000 7f22cbfff640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f22cc15c630 0x7f22cc0c6300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:37914/0 (socket says 172.21.15.43:37914) 2023-08-09T15:11:23.913 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:23.911+0000 7f22cbfff640 1 -- 172.21.15.43:0/951920110 learned_addr learned my addr 172.21.15.43:0/951920110 (peer_addr_for_me v2:172.21.15.43:0/0) 2023-08-09T15:11:23.913 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:23.911+0000 7f22cb7fe640 1 --2- 172.21.15.43:0/951920110 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f22cc0ccf20 0x7f22cc1584f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T15:11:23.913 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:23.911+0000 7f22c2ffd640 1 --2- 172.21.15.43:0/951920110 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f22cc0cdc60 0x7f22cc150350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T15:11:23.913 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:23.911+0000 7f22cbfff640 1 -- 172.21.15.43:0/951920110 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f22cc0ccf20 msgr2=0x7f22cc1584f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T15:11:23.914 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:23.911+0000 7f22cbfff640 1 --2- 172.21.15.43:0/951920110 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f22cc0ccf20 0x7f22cc1584f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T15:11:23.914 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:23.911+0000 7f22cbfff640 1 -- 172.21.15.43:0/951920110 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f22cc0cdc60 msgr2=0x7f22cc150350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T15:11:23.914 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:23.911+0000 7f22cbfff640 1 --2- 172.21.15.43:0/951920110 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f22cc0cdc60 0x7f22cc150350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T15:11:23.914 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:23.911+0000 7f22cbfff640 1 -- 172.21.15.43:0/951920110 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f22bc009400 con 0x7f22cc15c630 2023-08-09T15:11:23.915 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:23.912+0000 7f22cbfff640 1 --2- 172.21.15.43:0/951920110 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f22cc15c630 0x7f22cc0c6300 secure :-1 s=READY pgs=2372 cs=0 l=1 rev1=1 crypto rx=0x7f22c400cd20 tx=0x7f22c400a660 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-09T15:11:23.915 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:23.912+0000 7f22c8ff9640 1 -- 172.21.15.43:0/951920110 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f22c4003b90 con 0x7f22cc15c630 2023-08-09T15:11:23.915 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:23.912+0000 7f22c8ff9640 1 -- 172.21.15.43:0/951920110 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f22c4003cf0 con 0x7f22cc15c630 2023-08-09T15:11:23.915 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:23.912+0000 7f22c8ff9640 1 -- 172.21.15.43:0/951920110 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f22c4004240 con 0x7f22cc15c630 2023-08-09T15:11:23.915 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:23.912+0000 7f22d210c640 1 -- 172.21.15.43:0/951920110 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f22cc158db0 con 0x7f22cc15c630 2023-08-09T15:11:23.915 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:23.912+0000 7f22d210c640 1 -- 172.21.15.43:0/951920110 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f22cc159370 con 0x7f22cc15c630 2023-08-09T15:11:23.916 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:23.913+0000 7f22d210c640 1 -- 172.21.15.43:0/951920110 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f22cc15ca60 con 0x7f22cc15c630 2023-08-09T15:11:23.916 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:23.914+0000 7f22c8ff9640 1 -- 172.21.15.43:0/951920110 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 118) v1 ==== 88443+0+0 (secure 0 0 0) 0x7f22c40043a0 con 0x7f22cc15c630 2023-08-09T15:11:23.916 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:23.915+0000 7f22c8ff9640 1 --2- 172.21.15.43:0/951920110 >> [v2:172.21.15.203:6832/4034093268,v1:172.21.15.203:6833/4034093268] conn(0x7f22ac06a170 0x7f22ac06c630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T15:11:23.917 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:23.915+0000 7f22c8ff9640 1 -- 172.21.15.43:0/951920110 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(219..219 src has 1..219) v4 ==== 5574+0+0 (secure 0 0 0) 0x7f22c4012070 con 0x7f22cc15c630 2023-08-09T15:11:23.917 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:23.916+0000 7f22cb7fe640 1 --2- 172.21.15.43:0/951920110 >> [v2:172.21.15.203:6832/4034093268,v1:172.21.15.203:6833/4034093268] conn(0x7f22ac06a170 0x7f22ac06c630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T15:11:23.917 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:23.917+0000 7f22cb7fe640 1 --2- 172.21.15.43:0/951920110 >> [v2:172.21.15.203:6832/4034093268,v1:172.21.15.203:6833/4034093268] conn(0x7f22ac06a170 0x7f22ac06c630 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7f22bc036000 tx=0x7f22bc008030 comp rx=0 tx=0).ready entity=mgr.8833 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-09T15:11:23.920 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:23.920+0000 7f22c8ff9640 1 -- 172.21.15.43:0/951920110 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178066 (secure 0 0 0) 0x7f22c4054650 con 0x7f22cc15c630 2023-08-09T15:11:24.173 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:24.172+0000 7f22d210c640 1 -- 172.21.15.43:0/951920110 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "auth ls", "format": "json-pretty"} v 0) v1 -- 0x7f22cc0739f0 con 0x7f22cc15c630 2023-08-09T15:11:24.173 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:24.173+0000 7f22c8ff9640 1 -- 172.21.15.43:0/951920110 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "auth ls", "format": "json-pretty"}]=0 v47) v1 ==== 80+0+5304 (secure 0 0 0) 0x7f22c4057c70 con 0x7f22cc15c630 2023-08-09T15:11:24.174 INFO:teuthology.orchestra.run.smithi043.stdout: 2023-08-09T15:11:24.174 INFO:teuthology.orchestra.run.smithi043.stdout:{ 2023-08-09T15:11:24.174 INFO:teuthology.orchestra.run.smithi043.stdout: "auth_dump": [ 2023-08-09T15:11:24.175 INFO:teuthology.orchestra.run.smithi043.stdout: { 2023-08-09T15:11:24.175 INFO:teuthology.orchestra.run.smithi043.stdout: "entity": "mds.a", 2023-08-09T15:11:24.175 INFO:teuthology.orchestra.run.smithi043.stdout: "key": "AQCOldNkPu6GCRAAZecwctvMji0Zubb60KMlyQ==", 2023-08-09T15:11:24.175 INFO:teuthology.orchestra.run.smithi043.stdout: "caps": { 2023-08-09T15:11:24.176 INFO:teuthology.orchestra.run.smithi043.stdout: "mds": "allow", 2023-08-09T15:11:24.176 INFO:teuthology.orchestra.run.smithi043.stdout: "mgr": "allow *", 2023-08-09T15:11:24.176 INFO:teuthology.orchestra.run.smithi043.stdout: "mon": "allow *", 2023-08-09T15:11:24.177 INFO:teuthology.orchestra.run.smithi043.stdout: "osd": "allow *" 2023-08-09T15:11:24.177 INFO:teuthology.orchestra.run.smithi043.stdout: } 2023-08-09T15:11:24.177 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2023-08-09T15:11:24.177 INFO:teuthology.orchestra.run.smithi043.stdout: { 2023-08-09T15:11:24.178 INFO:teuthology.orchestra.run.smithi043.stdout: "entity": "mds.b", 2023-08-09T15:11:24.178 INFO:teuthology.orchestra.run.smithi043.stdout: "key": "AQCOldNkt2+5KxAAHcckB+S5NZHsF5z22StXmw==", 2023-08-09T15:11:24.178 INFO:teuthology.orchestra.run.smithi043.stdout: "caps": { 2023-08-09T15:11:24.178 INFO:teuthology.orchestra.run.smithi043.stdout: "mds": "allow", 2023-08-09T15:11:24.179 INFO:teuthology.orchestra.run.smithi043.stdout: "mgr": "allow *", 2023-08-09T15:11:24.179 INFO:teuthology.orchestra.run.smithi043.stdout: "mon": "allow *", 2023-08-09T15:11:24.179 INFO:teuthology.orchestra.run.smithi043.stdout: "osd": "allow *" 2023-08-09T15:11:24.180 INFO:teuthology.orchestra.run.smithi043.stdout: } 2023-08-09T15:11:24.180 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2023-08-09T15:11:24.180 INFO:teuthology.orchestra.run.smithi043.stdout: { 2023-08-09T15:11:24.180 INFO:teuthology.orchestra.run.smithi043.stdout: "entity": "mds.c", 2023-08-09T15:11:24.180 INFO:teuthology.orchestra.run.smithi043.stdout: "key": "AQCOldNk5V46GhAAQBhkvfjVSJCSHLS/YkhgCA==", 2023-08-09T15:11:24.180 INFO:teuthology.orchestra.run.smithi043.stdout: "caps": { 2023-08-09T15:11:24.180 INFO:teuthology.orchestra.run.smithi043.stdout: "mds": "allow", 2023-08-09T15:11:24.181 INFO:teuthology.orchestra.run.smithi043.stdout: "mgr": "allow *", 2023-08-09T15:11:24.181 INFO:teuthology.orchestra.run.smithi043.stdout: "mon": "allow *", 2023-08-09T15:11:24.181 INFO:teuthology.orchestra.run.smithi043.stdout: "osd": "allow *" 2023-08-09T15:11:24.181 INFO:teuthology.orchestra.run.smithi043.stdout: } 2023-08-09T15:11:24.181 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2023-08-09T15:11:24.181 INFO:teuthology.orchestra.run.smithi043.stdout: { 2023-08-09T15:11:24.181 INFO:teuthology.orchestra.run.smithi043.stdout: "entity": "mds.d", 2023-08-09T15:11:24.181 INFO:teuthology.orchestra.run.smithi043.stdout: "key": "AQCPldNkNBoTARAAMVW+rSyXQKaau6I90fUX9Q==", 2023-08-09T15:11:24.182 INFO:teuthology.orchestra.run.smithi043.stdout: "caps": { 2023-08-09T15:11:24.182 INFO:teuthology.orchestra.run.smithi043.stdout: "mds": "allow", 2023-08-09T15:11:24.182 INFO:teuthology.orchestra.run.smithi043.stdout: "mgr": "allow *", 2023-08-09T15:11:24.182 INFO:teuthology.orchestra.run.smithi043.stdout: "mon": "allow *", 2023-08-09T15:11:24.182 INFO:teuthology.orchestra.run.smithi043.stdout: "osd": "allow *" 2023-08-09T15:11:24.182 INFO:teuthology.orchestra.run.smithi043.stdout: } 2023-08-09T15:11:24.182 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2023-08-09T15:11:24.183 INFO:teuthology.orchestra.run.smithi043.stdout: { 2023-08-09T15:11:24.183 INFO:teuthology.orchestra.run.smithi043.stdout: "entity": "osd.0", 2023-08-09T15:11:24.183 INFO:teuthology.orchestra.run.smithi043.stdout: "key": "AQCRldNkltPlMhAAAt3rFvraCtI0aSqGvt6UxA==", 2023-08-09T15:11:24.183 INFO:teuthology.orchestra.run.smithi043.stdout: "caps": { 2023-08-09T15:11:24.183 INFO:teuthology.orchestra.run.smithi043.stdout: "mgr": "allow profile osd", 2023-08-09T15:11:24.183 INFO:teuthology.orchestra.run.smithi043.stdout: "mon": "allow profile osd", 2023-08-09T15:11:24.183 INFO:teuthology.orchestra.run.smithi043.stdout: "osd": "allow *" 2023-08-09T15:11:24.183 INFO:teuthology.orchestra.run.smithi043.stdout: } 2023-08-09T15:11:24.184 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2023-08-09T15:11:24.184 INFO:teuthology.orchestra.run.smithi043.stdout: { 2023-08-09T15:11:24.184 INFO:teuthology.orchestra.run.smithi043.stdout: "entity": "osd.1", 2023-08-09T15:11:24.184 INFO:teuthology.orchestra.run.smithi043.stdout: "key": "AQCVldNkzpKYEhAA+wen0kV/vUDLQztq3vhfOg==", 2023-08-09T15:11:24.184 INFO:teuthology.orchestra.run.smithi043.stdout: "caps": { 2023-08-09T15:11:24.184 INFO:teuthology.orchestra.run.smithi043.stdout: "mgr": "allow profile osd", 2023-08-09T15:11:24.184 INFO:teuthology.orchestra.run.smithi043.stdout: "mon": "allow profile osd", 2023-08-09T15:11:24.185 INFO:teuthology.orchestra.run.smithi043.stdout: "osd": "allow *" 2023-08-09T15:11:24.185 INFO:teuthology.orchestra.run.smithi043.stdout: } 2023-08-09T15:11:24.185 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2023-08-09T15:11:24.185 INFO:teuthology.orchestra.run.smithi043.stdout: { 2023-08-09T15:11:24.185 INFO:teuthology.orchestra.run.smithi043.stdout: "entity": "osd.2", 2023-08-09T15:11:24.185 INFO:teuthology.orchestra.run.smithi043.stdout: "key": "AQCYldNkYbwNHhAAQY8g9dldb1V9M2MM2SHKDg==", 2023-08-09T15:11:24.185 INFO:teuthology.orchestra.run.smithi043.stdout: "caps": { 2023-08-09T15:11:24.186 INFO:teuthology.orchestra.run.smithi043.stdout: "mgr": "allow profile osd", 2023-08-09T15:11:24.186 INFO:teuthology.orchestra.run.smithi043.stdout: "mon": "allow profile osd", 2023-08-09T15:11:24.186 INFO:teuthology.orchestra.run.smithi043.stdout: "osd": "allow *" 2023-08-09T15:11:24.186 INFO:teuthology.orchestra.run.smithi043.stdout: } 2023-08-09T15:11:24.186 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2023-08-09T15:11:24.186 INFO:teuthology.orchestra.run.smithi043.stdout: { 2023-08-09T15:11:24.186 INFO:teuthology.orchestra.run.smithi043.stdout: "entity": "osd.3", 2023-08-09T15:11:24.186 INFO:teuthology.orchestra.run.smithi043.stdout: "key": "AQCbldNkeyW6KRAAsd+kgqik2aD4U7PXwccWtA==", 2023-08-09T15:11:24.187 INFO:teuthology.orchestra.run.smithi043.stdout: "caps": { 2023-08-09T15:11:24.187 INFO:teuthology.orchestra.run.smithi043.stdout: "mgr": "allow profile osd", 2023-08-09T15:11:24.187 INFO:teuthology.orchestra.run.smithi043.stdout: "mon": "allow profile osd", 2023-08-09T15:11:24.187 INFO:teuthology.orchestra.run.smithi043.stdout: "osd": "allow *" 2023-08-09T15:11:24.187 INFO:teuthology.orchestra.run.smithi043.stdout: } 2023-08-09T15:11:24.187 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2023-08-09T15:11:24.187 INFO:teuthology.orchestra.run.smithi043.stdout: { 2023-08-09T15:11:24.187 INFO:teuthology.orchestra.run.smithi043.stdout: "entity": "osd.4", 2023-08-09T15:11:24.188 INFO:teuthology.orchestra.run.smithi043.stdout: "key": "AQCildNk80UkOBAAtX+4JGpOVcfZEHetJoeZKw==", 2023-08-09T15:11:24.188 INFO:teuthology.orchestra.run.smithi043.stdout: "caps": { 2023-08-09T15:11:24.188 INFO:teuthology.orchestra.run.smithi043.stdout: "mgr": "allow profile osd", 2023-08-09T15:11:24.188 INFO:teuthology.orchestra.run.smithi043.stdout: "mon": "allow profile osd", 2023-08-09T15:11:24.188 INFO:teuthology.orchestra.run.smithi043.stdout: "osd": "allow *" 2023-08-09T15:11:24.188 INFO:teuthology.orchestra.run.smithi043.stdout: } 2023-08-09T15:11:24.188 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2023-08-09T15:11:24.189 INFO:teuthology.orchestra.run.smithi043.stdout: { 2023-08-09T15:11:24.189 INFO:teuthology.orchestra.run.smithi043.stdout: "entity": "osd.5", 2023-08-09T15:11:24.189 INFO:teuthology.orchestra.run.smithi043.stdout: "key": "AQCmldNkCA9SBxAAZj4So6MRj2Eq1I9H9JzLDA==", 2023-08-09T15:11:24.189 INFO:teuthology.orchestra.run.smithi043.stdout: "caps": { 2023-08-09T15:11:24.189 INFO:teuthology.orchestra.run.smithi043.stdout: "mgr": "allow profile osd", 2023-08-09T15:11:24.189 INFO:teuthology.orchestra.run.smithi043.stdout: "mon": "allow profile osd", 2023-08-09T15:11:24.189 INFO:teuthology.orchestra.run.smithi043.stdout: "osd": "allow *" 2023-08-09T15:11:24.189 INFO:teuthology.orchestra.run.smithi043.stdout: } 2023-08-09T15:11:24.190 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2023-08-09T15:11:24.190 INFO:teuthology.orchestra.run.smithi043.stdout: { 2023-08-09T15:11:24.190 INFO:teuthology.orchestra.run.smithi043.stdout: "entity": "osd.6", 2023-08-09T15:11:24.190 INFO:teuthology.orchestra.run.smithi043.stdout: "key": "AQCpldNkaOwAExAAraV1tUwr9uU6GheZ+hvdqQ==", 2023-08-09T15:11:24.190 INFO:teuthology.orchestra.run.smithi043.stdout: "caps": { 2023-08-09T15:11:24.190 INFO:teuthology.orchestra.run.smithi043.stdout: "mgr": "allow profile osd", 2023-08-09T15:11:24.190 INFO:teuthology.orchestra.run.smithi043.stdout: "mon": "allow profile osd", 2023-08-09T15:11:24.191 INFO:teuthology.orchestra.run.smithi043.stdout: "osd": "allow *" 2023-08-09T15:11:24.191 INFO:teuthology.orchestra.run.smithi043.stdout: } 2023-08-09T15:11:24.191 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2023-08-09T15:11:24.191 INFO:teuthology.orchestra.run.smithi043.stdout: { 2023-08-09T15:11:24.191 INFO:teuthology.orchestra.run.smithi043.stdout: "entity": "osd.7", 2023-08-09T15:11:24.191 INFO:teuthology.orchestra.run.smithi043.stdout: "key": "AQCsldNk2/GwLBAAP42EXPivzLk44X8aMEVVCg==", 2023-08-09T15:11:24.191 INFO:teuthology.orchestra.run.smithi043.stdout: "caps": { 2023-08-09T15:11:24.191 INFO:teuthology.orchestra.run.smithi043.stdout: "mgr": "allow profile osd", 2023-08-09T15:11:24.192 INFO:teuthology.orchestra.run.smithi043.stdout: "mon": "allow profile osd", 2023-08-09T15:11:24.192 INFO:teuthology.orchestra.run.smithi043.stdout: "osd": "allow *" 2023-08-09T15:11:24.192 INFO:teuthology.orchestra.run.smithi043.stdout: } 2023-08-09T15:11:24.192 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2023-08-09T15:11:24.192 INFO:teuthology.orchestra.run.smithi043.stdout: { 2023-08-09T15:11:24.192 INFO:teuthology.orchestra.run.smithi043.stdout: "entity": "client.0", 2023-08-09T15:11:24.192 INFO:teuthology.orchestra.run.smithi043.stdout: "key": "AQCPldNkg8zrDxAAM/hnJaE6S9HU/fRwQeBV5w==", 2023-08-09T15:11:24.193 INFO:teuthology.orchestra.run.smithi043.stdout: "caps": { 2023-08-09T15:11:24.193 INFO:teuthology.orchestra.run.smithi043.stdout: "mds": "allow", 2023-08-09T15:11:24.193 INFO:teuthology.orchestra.run.smithi043.stdout: "mon": "allow r", 2023-08-09T15:11:24.193 INFO:teuthology.orchestra.run.smithi043.stdout: "osd": "allow rw tag cephfs data=cephfs" 2023-08-09T15:11:24.193 INFO:teuthology.orchestra.run.smithi043.stdout: } 2023-08-09T15:11:24.193 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2023-08-09T15:11:24.193 INFO:teuthology.orchestra.run.smithi043.stdout: { 2023-08-09T15:11:24.193 INFO:teuthology.orchestra.run.smithi043.stdout: "entity": "client.1", 2023-08-09T15:11:24.194 INFO:teuthology.orchestra.run.smithi043.stdout: "key": "AQCPldNklfFxJxAA19+L78rW6jU1uOyjey6WNA==", 2023-08-09T15:11:24.194 INFO:teuthology.orchestra.run.smithi043.stdout: "caps": { 2023-08-09T15:11:24.194 INFO:teuthology.orchestra.run.smithi043.stdout: "mds": "allow", 2023-08-09T15:11:24.194 INFO:teuthology.orchestra.run.smithi043.stdout: "mgr": "allow r", 2023-08-09T15:11:24.194 INFO:teuthology.orchestra.run.smithi043.stdout: "mon": "allow rw", 2023-08-09T15:11:24.194 INFO:teuthology.orchestra.run.smithi043.stdout: "osd": "allow rwx" 2023-08-09T15:11:24.194 INFO:teuthology.orchestra.run.smithi043.stdout: } 2023-08-09T15:11:24.194 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2023-08-09T15:11:24.195 INFO:teuthology.orchestra.run.smithi043.stdout: { 2023-08-09T15:11:24.195 INFO:teuthology.orchestra.run.smithi043.stdout: "entity": "client.2", 2023-08-09T15:11:24.195 INFO:teuthology.orchestra.run.smithi043.stdout: "key": "AQCPldNkC2lgGxAAZ8qZboOCUaQlcpkPL1ozHA==", 2023-08-09T15:11:24.195 INFO:teuthology.orchestra.run.smithi043.stdout: "caps": { 2023-08-09T15:11:24.195 INFO:teuthology.orchestra.run.smithi043.stdout: "mds": "allow", 2023-08-09T15:11:24.195 INFO:teuthology.orchestra.run.smithi043.stdout: "mgr": "allow r", 2023-08-09T15:11:24.195 INFO:teuthology.orchestra.run.smithi043.stdout: "mon": "allow rw", 2023-08-09T15:11:24.195 INFO:teuthology.orchestra.run.smithi043.stdout: "osd": "allow rwx" 2023-08-09T15:11:24.196 INFO:teuthology.orchestra.run.smithi043.stdout: } 2023-08-09T15:11:24.196 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2023-08-09T15:11:24.196 INFO:teuthology.orchestra.run.smithi043.stdout: { 2023-08-09T15:11:24.196 INFO:teuthology.orchestra.run.smithi043.stdout: "entity": "client.3", 2023-08-09T15:11:24.196 INFO:teuthology.orchestra.run.smithi043.stdout: "key": "AQCPldNkl+rXMhAA9rf3KdnSYbvngqmLYnIGGA==", 2023-08-09T15:11:24.197 INFO:teuthology.orchestra.run.smithi043.stdout: "caps": { 2023-08-09T15:11:24.197 INFO:teuthology.orchestra.run.smithi043.stdout: "mds": "allow", 2023-08-09T15:11:24.197 INFO:teuthology.orchestra.run.smithi043.stdout: "mgr": "allow r", 2023-08-09T15:11:24.197 INFO:teuthology.orchestra.run.smithi043.stdout: "mon": "allow rw", 2023-08-09T15:11:24.197 INFO:teuthology.orchestra.run.smithi043.stdout: "osd": "allow rwx" 2023-08-09T15:11:24.198 INFO:teuthology.orchestra.run.smithi043.stdout: } 2023-08-09T15:11:24.198 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2023-08-09T15:11:24.198 INFO:teuthology.orchestra.run.smithi043.stdout: { 2023-08-09T15:11:24.198 INFO:teuthology.orchestra.run.smithi043.stdout: "entity": "client.admin", 2023-08-09T15:11:24.198 INFO:teuthology.orchestra.run.smithi043.stdout: "key": "AQCMldNkUae6NRAA2FBTr0QPBdKNDoe1WK5zyg==", 2023-08-09T15:11:24.198 INFO:teuthology.orchestra.run.smithi043.stdout: "caps": { 2023-08-09T15:11:24.198 INFO:teuthology.orchestra.run.smithi043.stdout: "mds": "allow *", 2023-08-09T15:11:24.198 INFO:teuthology.orchestra.run.smithi043.stdout: "mgr": "allow *", 2023-08-09T15:11:24.199 INFO:teuthology.orchestra.run.smithi043.stdout: "mon": "allow *", 2023-08-09T15:11:24.199 INFO:teuthology.orchestra.run.smithi043.stdout: "osd": "allow *" 2023-08-09T15:11:24.199 INFO:teuthology.orchestra.run.smithi043.stdout: } 2023-08-09T15:11:24.199 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2023-08-09T15:11:24.199 INFO:teuthology.orchestra.run.smithi043.stdout: { 2023-08-09T15:11:24.199 INFO:teuthology.orchestra.run.smithi043.stdout: "entity": "mgr.x", 2023-08-09T15:11:24.199 INFO:teuthology.orchestra.run.smithi043.stdout: "key": "AQCNldNk2ufSNxAAU7msRkwLaoLZHTIsk0ck/g==", 2023-08-09T15:11:24.200 INFO:teuthology.orchestra.run.smithi043.stdout: "caps": { 2023-08-09T15:11:24.200 INFO:teuthology.orchestra.run.smithi043.stdout: "mds": "allow *", 2023-08-09T15:11:24.200 INFO:teuthology.orchestra.run.smithi043.stdout: "mon": "allow profile mgr", 2023-08-09T15:11:24.200 INFO:teuthology.orchestra.run.smithi043.stdout: "osd": "allow *" 2023-08-09T15:11:24.200 INFO:teuthology.orchestra.run.smithi043.stdout: } 2023-08-09T15:11:24.200 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2023-08-09T15:11:24.200 INFO:teuthology.orchestra.run.smithi043.stdout: { 2023-08-09T15:11:24.201 INFO:teuthology.orchestra.run.smithi043.stdout: "entity": "mgr.y", 2023-08-09T15:11:24.201 INFO:teuthology.orchestra.run.smithi043.stdout: "key": "AQCNldNk2YgzLBAAA11Q8uAMEgE/bUni+tXIKA==", 2023-08-09T15:11:24.201 INFO:teuthology.orchestra.run.smithi043.stdout: "caps": { 2023-08-09T15:11:24.201 INFO:teuthology.orchestra.run.smithi043.stdout: "mds": "allow *", 2023-08-09T15:11:24.201 INFO:teuthology.orchestra.run.smithi043.stdout: "mon": "allow profile mgr", 2023-08-09T15:11:24.201 INFO:teuthology.orchestra.run.smithi043.stdout: "osd": "allow *" 2023-08-09T15:11:24.201 INFO:teuthology.orchestra.run.smithi043.stdout: } 2023-08-09T15:11:24.202 INFO:teuthology.orchestra.run.smithi043.stdout: } 2023-08-09T15:11:24.202 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2023-08-09T15:11:24.202 INFO:teuthology.orchestra.run.smithi043.stdout:} 2023-08-09T15:11:24.203 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:24.176+0000 7f22d210c640 1 -- 172.21.15.43:0/951920110 >> [v2:172.21.15.203:6832/4034093268,v1:172.21.15.203:6833/4034093268] conn(0x7f22ac06a170 msgr2=0x7f22ac06c630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T15:11:24.203 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:24.177+0000 7f22d210c640 1 --2- 172.21.15.43:0/951920110 >> [v2:172.21.15.203:6832/4034093268,v1:172.21.15.203:6833/4034093268] conn(0x7f22ac06a170 0x7f22ac06c630 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7f22bc036000 tx=0x7f22bc008030 comp rx=0 tx=0).stop 2023-08-09T15:11:24.203 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:24.177+0000 7f22d210c640 1 -- 172.21.15.43:0/951920110 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f22cc15c630 msgr2=0x7f22cc0c6300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T15:11:24.204 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:24.177+0000 7f22d210c640 1 --2- 172.21.15.43:0/951920110 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f22cc15c630 0x7f22cc0c6300 secure :-1 s=READY pgs=2372 cs=0 l=1 rev1=1 crypto rx=0x7f22c400cd20 tx=0x7f22c400a660 comp rx=0 tx=0).stop 2023-08-09T15:11:24.204 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:24.177+0000 7f22d210c640 1 -- 172.21.15.43:0/951920110 shutdown_connections 2023-08-09T15:11:24.204 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:24.177+0000 7f22d210c640 1 --2- 172.21.15.43:0/951920110 >> [v2:172.21.15.203:6832/4034093268,v1:172.21.15.203:6833/4034093268] conn(0x7f22ac06a170 0x7f22ac06c630 unknown :-1 s=CLOSED pgs=47 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T15:11:24.204 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:24.177+0000 7f22d210c640 1 --2- 172.21.15.43:0/951920110 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f22cc15c630 0x7f22cc0c6300 unknown :-1 s=CLOSED pgs=2372 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T15:11:24.204 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:24.177+0000 7f22d210c640 1 --2- 172.21.15.43:0/951920110 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f22cc0cdc60 0x7f22cc150350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T15:11:24.204 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:24.177+0000 7f22d210c640 1 --2- 172.21.15.43:0/951920110 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f22cc0ccf20 0x7f22cc1584f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T15:11:24.204 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:24.177+0000 7f22d210c640 1 -- 172.21.15.43:0/951920110 >> 172.21.15.43:0/951920110 conn(0x7f22cc0ce0a0 msgr2=0x7f22cc0c3fd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-09T15:11:24.205 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:24.177+0000 7f22d210c640 1 -- 172.21.15.43:0/951920110 shutdown_connections 2023-08-09T15:11:24.205 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:24.177+0000 7f22d210c640 1 -- 172.21.15.43:0/951920110 wait complete. 2023-08-09T15:11:24.205 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-08-09T15:11:24.431 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:24.430+0000 7f1ac48ac640 1 Processor -- start 2023-08-09T15:11:24.432 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:24.431+0000 7f1ac48ac640 1 -- start start 2023-08-09T15:11:24.433 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:24.432+0000 7f1ac48ac640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f1ac00c8b30 0x7f1ac00ad340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T15:11:24.433 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:24.432+0000 7f1ac48ac640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f1ac00acdf0 0x7f1ac015b2c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T15:11:24.434 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:24.432+0000 7f1ac48ac640 1 -- --> v1:172.21.15.43:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1ac00cbd90 con 0x7f1ac00c8750 2023-08-09T15:11:24.434 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:24.432+0000 7f1ac48ac640 1 -- --> v2:172.21.15.43:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1ac015e810 con 0x7f1ac00c8b30 2023-08-09T15:11:24.434 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:24.432+0000 7f1ac48ac640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1ac00c9f70 con 0x7f1ac00acdf0 2023-08-09T15:11:24.434 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:24.433+0000 7f1abed9d640 1 --1- >> v1:172.21.15.43:6789/0 conn(0x7f1ac00c8750 0x7f1ac006f410 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.43:6789/0 says I am v1:172.21.15.43:45628/0 (socket says 172.21.15.43:45628) 2023-08-09T15:11:24.435 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:24.433+0000 7f1abed9d640 1 -- 172.21.15.43:0/3631289170 learned_addr learned my addr 172.21.15.43:0/3631289170 (peer_addr_for_me v1:172.21.15.43:0/0) 2023-08-09T15:11:24.435 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:24.433+0000 7f1ab7fff640 1 --2- 172.21.15.43:0/3631289170 >> v2:172.21.15.43:3300/0 conn(0x7f1ac00c8b30 0x7f1ac00ad340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T15:11:24.437 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:24.433+0000 7f1abf59e640 1 --2- 172.21.15.43:0/3631289170 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f1ac00acdf0 0x7f1ac015b2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T15:11:24.437 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:24.433+0000 7f1abe59c640 1 -- 172.21.15.43:0/3631289170 <== mon.0 v1:172.21.15.43:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1074768771 0 0) 0x7f1ac00cbd90 con 0x7f1ac00c8750 2023-08-09T15:11:24.437 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:24.433+0000 7f1abe59c640 1 -- 172.21.15.43:0/3631289170 --> v1:172.21.15.43:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f1aac003520 con 0x7f1ac00c8750 2023-08-09T15:11:24.437 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:24.434+0000 7f1ab7fff640 1 -- 172.21.15.43:0/3631289170 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f1ac00acdf0 msgr2=0x7f1ac015b2c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-09T15:11:24.437 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:24.434+0000 7f1ab7fff640 1 --2- 172.21.15.43:0/3631289170 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f1ac00acdf0 0x7f1ac015b2c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T15:11:24.438 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:24.434+0000 7f1ab7fff640 1 -- 172.21.15.43:0/3631289170 >> v1:172.21.15.43:6789/0 conn(0x7f1ac00c8750 legacy=0x7f1ac006f410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T15:11:24.438 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:24.434+0000 7f1ab7fff640 1 -- 172.21.15.43:0/3631289170 --> v2:172.21.15.43:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1ac00ca2b0 con 0x7f1ac00c8b30 2023-08-09T15:11:24.438 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:24.435+0000 7f1ab7fff640 1 --2- 172.21.15.43:0/3631289170 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f1ac00c8b30 0x7f1ac00ad340 secure :-1 s=READY pgs=2374 cs=0 l=1 rev1=1 crypto rx=0x7f1aa80097a0 tx=0x7f1aa802b820 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1cd3b100a8c8620 server_cookie=0 in_seq=0 out_seq=0 2023-08-09T15:11:24.438 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:24.435+0000 7f1abe59c640 1 -- 172.21.15.43:0/3631289170 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1aa8031780 con 0x7f1ac00c8b30 2023-08-09T15:11:24.438 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:24.435+0000 7f1abe59c640 1 -- 172.21.15.43:0/3631289170 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1aa80318e0 con 0x7f1ac00c8b30 2023-08-09T15:11:24.438 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:24.435+0000 7f1abe59c640 1 -- 172.21.15.43:0/3631289170 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1aa8031c50 con 0x7f1ac00c8b30 2023-08-09T15:11:24.438 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:24.436+0000 7f1ac48ac640 1 -- 172.21.15.43:0/3631289170 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f1ac00c8b30 msgr2=0x7f1ac00ad340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T15:11:24.439 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:24.436+0000 7f1ac48ac640 1 --2- 172.21.15.43:0/3631289170 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f1ac00c8b30 0x7f1ac00ad340 secure :-1 s=READY pgs=2374 cs=0 l=1 rev1=1 crypto rx=0x7f1aa80097a0 tx=0x7f1aa802b820 comp rx=0 tx=0).stop 2023-08-09T15:11:24.439 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:24.436+0000 7f1ac48ac640 1 -- 172.21.15.43:0/3631289170 shutdown_connections 2023-08-09T15:11:24.439 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:24.436+0000 7f1ac48ac640 1 --2- 172.21.15.43:0/3631289170 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f1ac00acdf0 0x7f1ac015b2c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T15:11:24.439 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:24.436+0000 7f1ac48ac640 1 --2- 172.21.15.43:0/3631289170 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f1ac00c8b30 0x7f1ac00ad340 unknown :-1 s=CLOSED pgs=2374 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T15:11:24.439 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:24.436+0000 7f1ac48ac640 1 -- 172.21.15.43:0/3631289170 >> 172.21.15.43:0/3631289170 conn(0x7f1ac00cde90 msgr2=0x7f1ac00ce2a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-09T15:11:24.439 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:24.436+0000 7f1ac48ac640 1 -- 172.21.15.43:0/3631289170 shutdown_connections 2023-08-09T15:11:24.439 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:24.436+0000 7f1ac48ac640 1 -- 172.21.15.43:0/3631289170 wait complete. 2023-08-09T15:11:24.440 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:24.438+0000 7f1ac48ac640 1 Processor -- start 2023-08-09T15:11:24.441 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:24.438+0000 7f1ac48ac640 1 -- start start 2023-08-09T15:11:24.441 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:24.439+0000 7f1ac48ac640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f1ac00acdf0 0x7f1ac0158f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T15:11:24.441 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:24.439+0000 7f1ac48ac640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f1ac00c8750 0x7f1ac00b3150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T15:11:24.441 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:24.439+0000 7f1ac48ac640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f1ac00c8b30 0x7f1ac00b3690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T15:11:24.441 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:24.439+0000 7f1ac48ac640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1ac01594d0 con 0x7f1ac00c8750 2023-08-09T15:11:24.442 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:24.439+0000 7f1ac48ac640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1ac00b0ab0 con 0x7f1ac00acdf0 2023-08-09T15:11:24.442 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:24.439+0000 7f1ac48ac640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1ac00b0d90 con 0x7f1ac00c8b30 2023-08-09T15:11:24.442 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:24.440+0000 7f1ab7fff640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f1ac00c8750 0x7f1ac00b3150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T15:11:24.442 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:24.440+0000 7f1ab7fff640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f1ac00c8750 0x7f1ac00b3150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:37932/0 (socket says 172.21.15.43:37932) 2023-08-09T15:11:24.442 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:24.440+0000 7f1ab7fff640 1 -- 172.21.15.43:0/1763842400 learned_addr learned my addr 172.21.15.43:0/1763842400 (peer_addr_for_me v2:172.21.15.43:0/0) 2023-08-09T15:11:24.442 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:24.440+0000 7f1abf59e640 1 --2- 172.21.15.43:0/1763842400 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f1ac00c8b30 0x7f1ac00b3690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T15:11:24.442 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:24.440+0000 7f1abed9d640 1 --2- 172.21.15.43:0/1763842400 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f1ac00acdf0 0x7f1ac0158f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T15:11:24.442 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:24.440+0000 7f1ab7fff640 1 -- 172.21.15.43:0/1763842400 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f1ac00c8b30 msgr2=0x7f1ac00b3690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T15:11:24.443 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:24.440+0000 7f1ab7fff640 1 --2- 172.21.15.43:0/1763842400 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f1ac00c8b30 0x7f1ac00b3690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T15:11:24.443 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:24.440+0000 7f1ab7fff640 1 -- 172.21.15.43:0/1763842400 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f1ac00acdf0 msgr2=0x7f1ac0158f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T15:11:24.443 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:24.440+0000 7f1ab7fff640 1 --2- 172.21.15.43:0/1763842400 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f1ac00acdf0 0x7f1ac0158f90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T15:11:24.443 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:24.440+0000 7f1ab7fff640 1 -- 172.21.15.43:0/1763842400 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1aa8009400 con 0x7f1ac00c8750 2023-08-09T15:11:24.444 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:24.441+0000 7f1ab7fff640 1 --2- 172.21.15.43:0/1763842400 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f1ac00c8750 0x7f1ac00b3150 secure :-1 s=READY pgs=2375 cs=0 l=1 rev1=1 crypto rx=0x7f1aa8034040 tx=0x7f1aa80046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-09T15:11:24.444 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:24.441+0000 7f1ab77fe640 1 -- 172.21.15.43:0/1763842400 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1aa8032820 con 0x7f1ac00c8750 2023-08-09T15:11:24.444 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:24.441+0000 7f1ab77fe640 1 -- 172.21.15.43:0/1763842400 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1aa8032980 con 0x7f1ac00c8750 2023-08-09T15:11:24.444 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:24.441+0000 7f1ab77fe640 1 -- 172.21.15.43:0/1763842400 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1aa8032cf0 con 0x7f1ac00c8750 2023-08-09T15:11:24.444 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:24.441+0000 7f1ac48ac640 1 -- 172.21.15.43:0/1763842400 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1ac00b1070 con 0x7f1ac00c8750 2023-08-09T15:11:24.445 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:24.441+0000 7f1ac48ac640 1 -- 172.21.15.43:0/1763842400 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1ac00b1630 con 0x7f1ac00c8750 2023-08-09T15:11:24.445 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:24.442+0000 7f1ac48ac640 1 -- 172.21.15.43:0/1763842400 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1ac00b17c0 con 0x7f1ac00c8750 2023-08-09T15:11:24.445 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:24.443+0000 7f1ab77fe640 1 -- 172.21.15.43:0/1763842400 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 118) v1 ==== 88443+0+0 (secure 0 0 0) 0x7f1aa804e440 con 0x7f1ac00c8750 2023-08-09T15:11:24.445 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:24.444+0000 7f1ab77fe640 1 --2- 172.21.15.43:0/1763842400 >> [v2:172.21.15.203:6832/4034093268,v1:172.21.15.203:6833/4034093268] conn(0x7f1aa006a170 0x7f1aa006c630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T15:11:24.446 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:24.445+0000 7f1ab77fe640 1 -- 172.21.15.43:0/1763842400 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(219..219 src has 1..219) v4 ==== 5574+0+0 (secure 0 0 0) 0x7f1aa80aa680 con 0x7f1ac00c8750 2023-08-09T15:11:24.446 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:24.445+0000 7f1abed9d640 1 --2- 172.21.15.43:0/1763842400 >> [v2:172.21.15.203:6832/4034093268,v1:172.21.15.203:6833/4034093268] conn(0x7f1aa006a170 0x7f1aa006c630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T15:11:24.446 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:24.446+0000 7f1abed9d640 1 --2- 172.21.15.43:0/1763842400 >> [v2:172.21.15.203:6832/4034093268,v1:172.21.15.203:6833/4034093268] conn(0x7f1aa006a170 0x7f1aa006c630 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7f1ab0005880 tx=0x7f1ab0009040 comp rx=0 tx=0).ready entity=mgr.8833 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-09T15:11:24.449 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:24.449+0000 7f1ab77fe640 1 -- 172.21.15.43:0/1763842400 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178066 (secure 0 0 0) 0x7f1aa8075dc0 con 0x7f1ac00c8750 2023-08-09T15:11:24.649 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:24.648+0000 7f1ac48ac640 1 -- 172.21.15.43:0/1763842400 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f1ac00c9230 con 0x7f1ac00c8750 2023-08-09T15:11:24.650 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:24.650+0000 7f1ab77fe640 1 -- 172.21.15.43:0/1763842400 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v219) v1 ==== 74+0+14854 (secure 0 0 0) 0x7f1aa80793e0 con 0x7f1ac00c8750 2023-08-09T15:11:24.651 INFO:teuthology.orchestra.run.smithi043.stdout: 2023-08-09T15:11:24.651 INFO:teuthology.orchestra.run.smithi043.stdout:{"epoch":219,"fsid":"daac87ee-1337-4250-a955-aea4e408a39b","created":"2023-08-09T13:33:41.701198+0000","modified":"2023-08-09T15:11:23.654598+0000","last_up_change":"2023-08-09T13:33:53.783453+0000","last_in_change":"2023-08-09T13:33:45.148904+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":25,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":"rbd","create_time":"2023-08-09T13:33:54.365137+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":".mgr","create_time":"2023-08-09T13:33:54.831803+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"cbd49ecd-e896-4fbd-864e-5c37a829b165","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":203,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6800","nonce":502667335},{"type":"v1","addr":"172.21.15.43:6801","nonce":502667335}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6802","nonce":502667335},{"type":"v1","addr":"172.21.15.43:6803","nonce":502667335}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6806","nonce":502667335},{"type":"v1","addr":"172.21.15.43:6807","nonce":502667335}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6804","nonce":502667335},{"type":"v1","addr":"172.21.15.43:6805","nonce":502667335}]},"public_addr":"172.21.15.43:6801/502667335","cluster_addr":"172.21.15.43:6803/502667335","heartbeat_back_addr":"172.21.15.43:6807/502667335","heartbeat_front_addr":"172.21.15.43:6805/502667335","state":["exists","up"]},{"osd":1,"uuid":"75c7e0fe-1ec7-4fe8-a2f2-3661a65b8500","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":203,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6808","nonce":3289134610},{"type":"v1","addr":"172.21.15.43:6809","nonce":3289134610}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6810","nonce":3289134610},{"type":"v1","addr":"172.21.15.43:6812","nonce":3289134610}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6821","nonce":3289134610},{"type":"v1","addr":"172.21.15.43:6824","nonce":3289134610}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6815","nonce":3289134610},{"type":"v1","addr":"172.21.15.43:6818","nonce":3289134610}]},"public_addr":"172.21.15.43:6809/3289134610","cluster_addr":"172.21.15.43:6812/3289134610","heartbeat_back_addr":"172.21.15.43:6824/3289134610","heartbeat_front_addr":"172.21.15.43:6818/3289134610","state":["exists","up"]},{"osd":2,"uuid":"bb5c41e9-7f5f-4125-9b01-3bb8bfc2ea83","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":203,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6811","nonce":4079709736},{"type":"v1","addr":"172.21.15.43:6814","nonce":4079709736}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6817","nonce":4079709736},{"type":"v1","addr":"172.21.15.43:6820","nonce":4079709736}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6829","nonce":4079709736},{"type":"v1","addr":"172.21.15.43:6831","nonce":4079709736}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6823","nonce":4079709736},{"type":"v1","addr":"172.21.15.43:6826","nonce":4079709736}]},"public_addr":"172.21.15.43:6814/4079709736","cluster_addr":"172.21.15.43:6820/4079709736","heartbeat_back_addr":"172.21.15.43:6831/4079709736","heartbeat_front_addr":"172.21.15.43:6826/4079709736","state":["exists","up"]},{"osd":3,"uuid":"bfe4c8b4-694a-465c-87e6-2b2aab852613","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":203,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6813","nonce":1709538759},{"type":"v1","addr":"172.21.15.43:6816","nonce":1709538759}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6819","nonce":1709538759},{"type":"v1","addr":"172.21.15.43:6822","nonce":1709538759}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6828","nonce":1709538759},{"type":"v1","addr":"172.21.15.43:6830","nonce":1709538759}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6825","nonce":1709538759},{"type":"v1","addr":"172.21.15.43:6827","nonce":1709538759}]},"public_addr":"172.21.15.43:6816/1709538759","cluster_addr":"172.21.15.43:6822/1709538759","heartbeat_back_addr":"172.21.15.43:6830/1709538759","heartbeat_front_addr":"172.21.15.43:6827/1709538759","state":["exists","up"]},{"osd":4,"uuid":"fa2e8a77-aba1-4756-853e-0d174a5bdeba","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":203,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6800","nonce":4196866481},{"type":"v1","addr":"172.21.15.203:6801","nonce":4196866481}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6802","nonce":4196866481},{"type":"v1","addr":"172.21.15.203:6803","nonce":4196866481}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6806","nonce":4196866481},{"type":"v1","addr":"172.21.15.203:6807","nonce":4196866481}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6804","nonce":4196866481},{"type":"v1","addr":"172.21.15.203:6805","nonce":4196866481}]},"public_addr":"172.21.15.203:6801/4196866481","cluster_addr":"172.21.15.203:6803/4196866481","heartbeat_back_addr":"172.21.15.203:6807/4196866481","heartbeat_front_addr":"172.21.15.203:6805/4196866481","state":["exists","up"]},{"osd":5,"uuid":"81811e4b-4d24-4f51-a82c-f1d700a2a00d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":207,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6816","nonce":3059290081},{"type":"v1","addr":"172.21.15.203:6817","nonce":3059290081}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6818","nonce":3059290081},{"type":"v1","addr":"172.21.15.203:6819","nonce":3059290081}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6822","nonce":3059290081},{"type":"v1","addr":"172.21.15.203:6823","nonce":3059290081}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6820","nonce":3059290081},{"type":"v1","addr":"172.21.15.203:6821","nonce":3059290081}]},"public_addr":"172.21.15.203:6817/3059290081","cluster_addr":"172.21.15.203:6819/3059290081","heartbeat_back_addr":"172.21.15.203:6823/3059290081","heartbeat_front_addr":"172.21.15.203:6821/3059290081","state":["exists","up"]},{"osd":6,"uuid":"307bac4d-6a5b-443d-af75-56d695d7674f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":203,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6824","nonce":512155569},{"type":"v1","addr":"172.21.15.203:6825","nonce":512155569}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6826","nonce":512155569},{"type":"v1","addr":"172.21.15.203:6827","nonce":512155569}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6830","nonce":512155569},{"type":"v1","addr":"172.21.15.203:6831","nonce":512155569}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6828","nonce":512155569},{"type":"v1","addr":"172.21.15.203:6829","nonce":512155569}]},"public_addr":"172.21.15.203:6825/512155569","cluster_addr":"172.21.15.203:6827/512155569","heartbeat_back_addr":"172.21.15.203:6831/512155569","heartbeat_front_addr":"172.21.15.203:6829/512155569","state":["exists","up"]},{"osd":7,"uuid":"a95dc7ca-9f1b-4295-af70-68a22a2d7d5e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":207,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6808","nonce":3331450438},{"type":"v1","addr":"172.21.15.203:6809","nonce":3331450438}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6810","nonce":3331450438},{"type":"v1","addr":"172.21.15.203:6811","nonce":3331450438}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6814","nonce":3331450438},{"type":"v1","addr":"172.21.15.203:6815","nonce":3331450438}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6812","nonce":3331450438},{"type":"v1","addr":"172.21.15.203:6813","nonce":3331450438}]},"public_addr":"172.21.15.203:6809/3331450438","cluster_addr":"172.21.15.203:6811/3331450438","heartbeat_back_addr":"172.21.15.203:6815/3331450438","heartbeat_front_addr":"172.21.15.203:6813/3331450438","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-09T13:33:50.547652+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-09T13:33:50.589469+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-09T13:33:50.561724+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-09T13:33:50.599650+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-09T13:33:50.411855+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-09T13:33:50.219681+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-09T13:33:50.504664+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-09T13:33:50.441248+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":7,"snaps":[{"begin":104,"length":1},{"begin":221,"length":1}]}],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-08-09T15:11:24.653 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:24.653+0000 7f1ac48ac640 1 -- 172.21.15.43:0/1763842400 >> [v2:172.21.15.203:6832/4034093268,v1:172.21.15.203:6833/4034093268] conn(0x7f1aa006a170 msgr2=0x7f1aa006c630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T15:11:24.653 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:24.653+0000 7f1ac48ac640 1 --2- 172.21.15.43:0/1763842400 >> [v2:172.21.15.203:6832/4034093268,v1:172.21.15.203:6833/4034093268] conn(0x7f1aa006a170 0x7f1aa006c630 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7f1ab0005880 tx=0x7f1ab0009040 comp rx=0 tx=0).stop 2023-08-09T15:11:24.653 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:24.653+0000 7f1ac48ac640 1 -- 172.21.15.43:0/1763842400 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f1ac00c8750 msgr2=0x7f1ac00b3150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T15:11:24.653 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:24.653+0000 7f1ac48ac640 1 --2- 172.21.15.43:0/1763842400 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f1ac00c8750 0x7f1ac00b3150 secure :-1 s=READY pgs=2375 cs=0 l=1 rev1=1 crypto rx=0x7f1aa8034040 tx=0x7f1aa80046c0 comp rx=0 tx=0).stop 2023-08-09T15:11:24.654 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:24.653+0000 7f1ac48ac640 1 -- 172.21.15.43:0/1763842400 shutdown_connections 2023-08-09T15:11:24.654 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:24.653+0000 7f1ac48ac640 1 --2- 172.21.15.43:0/1763842400 >> [v2:172.21.15.203:6832/4034093268,v1:172.21.15.203:6833/4034093268] conn(0x7f1aa006a170 0x7f1aa006c630 unknown :-1 s=CLOSED pgs=48 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T15:11:24.654 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:24.653+0000 7f1ac48ac640 1 --2- 172.21.15.43:0/1763842400 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f1ac00c8b30 0x7f1ac00b3690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T15:11:24.654 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:24.653+0000 7f1ac48ac640 1 --2- 172.21.15.43:0/1763842400 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f1ac00c8750 0x7f1ac00b3150 unknown :-1 s=CLOSED pgs=2375 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T15:11:24.655 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:24.653+0000 7f1ac48ac640 1 --2- 172.21.15.43:0/1763842400 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f1ac00acdf0 0x7f1ac0158f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T15:11:24.655 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:24.653+0000 7f1ac48ac640 1 -- 172.21.15.43:0/1763842400 >> 172.21.15.43:0/1763842400 conn(0x7f1ac00cde90 msgr2=0x7f1ac00c06e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-09T15:11:24.655 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:24.653+0000 7f1ac48ac640 1 -- 172.21.15.43:0/1763842400 shutdown_connections 2023-08-09T15:11:24.655 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:24.653+0000 7f1ac48ac640 1 -- 172.21.15.43:0/1763842400 wait complete. 2023-08-09T15:11:24.674 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': 'rbd', 'create_time': '2023-08-09T13:33:54.365137+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}, 'read_balance': {'score_acting': 2, 'score_stable': 2, 'optimal_score': 1, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': '.mgr', 'create_time': '2023-08-09T13:33:54.831803+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-08-09T15:11:24.675 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get rbd pg_num 2023-08-09T15:11:24.896 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:24.895+0000 7ff669d14640 1 Processor -- start 2023-08-09T15:11:24.896 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:24.896+0000 7ff669d14640 1 -- start start 2023-08-09T15:11:24.898 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:24.897+0000 7ff669d14640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7ff6640b7df0 0x7ff6640d1b30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T15:11:24.898 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:24.897+0000 7ff669d14640 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7ff6640cebf0 0x7ff6640cf060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T15:11:24.898 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:24.897+0000 7ff669d14640 1 -- --> v1:172.21.15.43:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff6640d16a0 con 0x7ff66415c2d0 2023-08-09T15:11:24.898 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:24.897+0000 7ff669d14640 1 -- --> v2:172.21.15.203:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff6640b8230 con 0x7ff6640cebf0 2023-08-09T15:11:24.898 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:24.897+0000 7ff669d14640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff6640cf5a0 con 0x7ff6640b7df0 2023-08-09T15:11:24.899 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:24.898+0000 7ff668d12640 1 --1- >> v1:172.21.15.43:6789/0 conn(0x7ff66415c2d0 0x7ff6640cd020 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.43:6789/0 says I am v1:172.21.15.43:45640/0 (socket says 172.21.15.43:45640) 2023-08-09T15:11:24.899 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:24.898+0000 7ff668d12640 1 -- 172.21.15.43:0/2224861568 learned_addr learned my addr 172.21.15.43:0/2224861568 (peer_addr_for_me v1:172.21.15.43:0/0) 2023-08-09T15:11:24.900 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:24.898+0000 7ff669513640 1 --2- 172.21.15.43:0/2224861568 >> v2:172.21.15.203:3300/0 conn(0x7ff6640cebf0 0x7ff6640cf060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T15:11:24.900 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:24.898+0000 7ff65bfff640 1 --2- 172.21.15.43:0/2224861568 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7ff6640b7df0 0x7ff6640d1b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T15:11:24.901 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:24.898+0000 7ff65b7fe640 1 -- 172.21.15.43:0/2224861568 <== mon.0 v1:172.21.15.43:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1636765452 0 0) 0x7ff6640d16a0 con 0x7ff66415c2d0 2023-08-09T15:11:24.901 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:24.898+0000 7ff65b7fe640 1 -- 172.21.15.43:0/2224861568 --> v1:172.21.15.43:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ff64c003520 con 0x7ff66415c2d0 2023-08-09T15:11:24.901 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:24.899+0000 7ff65b7fe640 1 -- 172.21.15.43:0/2224861568 <== mon.0 v1:172.21.15.43:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2694156331 0 0) 0x7ff64c003520 con 0x7ff66415c2d0 2023-08-09T15:11:24.901 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:24.899+0000 7ff65b7fe640 1 -- 172.21.15.43:0/2224861568 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7ff6640b7df0 msgr2=0x7ff6640d1b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-09T15:11:24.901 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:24.899+0000 7ff65b7fe640 1 --2- 172.21.15.43:0/2224861568 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7ff6640b7df0 0x7ff6640d1b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T15:11:24.901 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:24.899+0000 7ff65b7fe640 1 -- 172.21.15.43:0/2224861568 >> v2:172.21.15.203:3300/0 conn(0x7ff6640cebf0 msgr2=0x7ff6640cf060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-09T15:11:24.901 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:24.899+0000 7ff65b7fe640 1 --2- 172.21.15.43:0/2224861568 >> v2:172.21.15.203:3300/0 conn(0x7ff6640cebf0 0x7ff6640cf060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T15:11:24.902 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:24.899+0000 7ff65b7fe640 1 -- 172.21.15.43:0/2224861568 --> v1:172.21.15.43:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff6640d0d70 con 0x7ff66415c2d0 2023-08-09T15:11:24.902 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:24.899+0000 7ff65b7fe640 1 -- 172.21.15.43:0/2224861568 <== mon.0 v1:172.21.15.43:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1285150740 0 0) 0x7ff660001670 con 0x7ff66415c2d0 2023-08-09T15:11:24.902 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:24.899+0000 7ff65b7fe640 1 -- 172.21.15.43:0/2224861568 >> v1:172.21.15.43:6789/0 conn(0x7ff66415c2d0 legacy=0x7ff6640cd020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T15:11:24.902 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:24.899+0000 7ff65b7fe640 1 --2- 172.21.15.43:0/2224861568 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff64c003970 0x7ff64c003d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T15:11:24.902 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:24.899+0000 7ff65b7fe640 1 --2- 172.21.15.43:0/2224861568 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7ff6640b7df0 0x7ff64c0047c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T15:11:24.902 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:24.899+0000 7ff65b7fe640 1 --2- 172.21.15.43:0/2224861568 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7ff64c004d00 0x7ff64c005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T15:11:24.902 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:24.899+0000 7ff65b7fe640 1 -- 172.21.15.43:0/2224861568 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff6640cf5a0 con 0x7ff64c003970 2023-08-09T15:11:24.903 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:24.899+0000 7ff65b7fe640 1 -- 172.21.15.43:0/2224861568 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff64c00b980 con 0x7ff64c004d00 2023-08-09T15:11:24.903 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:24.899+0000 7ff65b7fe640 1 -- 172.21.15.43:0/2224861568 --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff64c00bac0 con 0x7ff6640b7df0 2023-08-09T15:11:24.903 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:24.899+0000 7ff668d12640 1 --2- 172.21.15.43:0/2224861568 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff64c003970 0x7ff64c003d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T15:11:24.903 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:24.900+0000 7ff65bfff640 1 --2- 172.21.15.43:0/2224861568 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7ff6640b7df0 0x7ff64c0047c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T15:11:24.903 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:24.900+0000 7ff668d12640 1 -- 172.21.15.43:0/2224861568 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7ff6640b7df0 msgr2=0x7ff64c0047c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-09T15:11:24.903 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:24.900+0000 7ff668d12640 1 --2- 172.21.15.43:0/2224861568 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7ff6640b7df0 0x7ff64c0047c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T15:11:24.903 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:24.900+0000 7ff668d12640 1 -- 172.21.15.43:0/2224861568 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7ff64c004d00 msgr2=0x7ff64c005180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-09T15:11:24.903 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:24.900+0000 7ff669513640 1 --2- 172.21.15.43:0/2224861568 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7ff64c004d00 0x7ff64c005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T15:11:24.904 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:24.900+0000 7ff668d12640 1 --2- 172.21.15.43:0/2224861568 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7ff64c004d00 0x7ff64c005180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T15:11:24.904 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:24.900+0000 7ff668d12640 1 -- 172.21.15.43:0/2224861568 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff64c003520 con 0x7ff64c003970 2023-08-09T15:11:24.905 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:24.901+0000 7ff668d12640 1 --2- 172.21.15.43:0/2224861568 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff64c003970 0x7ff64c003d90 secure :-1 s=READY pgs=2377 cs=0 l=1 rev1=1 crypto rx=0x7ff66000c240 tx=0x7ff64c0057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=41ac5b47ae894321 server_cookie=0 in_seq=0 out_seq=0 2023-08-09T15:11:24.905 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:24.901+0000 7ff65b7fe640 1 -- 172.21.15.43:0/2224861568 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff660005ad0 con 0x7ff64c003970 2023-08-09T15:11:24.905 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:24.901+0000 7ff65b7fe640 1 -- 172.21.15.43:0/2224861568 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff660005c50 con 0x7ff64c003970 2023-08-09T15:11:24.905 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:24.901+0000 7ff65b7fe640 1 -- 172.21.15.43:0/2224861568 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff66002c8d0 con 0x7ff64c003970 2023-08-09T15:11:24.905 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:24.902+0000 7ff669d14640 1 -- 172.21.15.43:0/2224861568 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff64c003970 msgr2=0x7ff64c003d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T15:11:24.906 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:24.902+0000 7ff669d14640 1 --2- 172.21.15.43:0/2224861568 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff64c003970 0x7ff64c003d90 secure :-1 s=READY pgs=2377 cs=0 l=1 rev1=1 crypto rx=0x7ff66000c240 tx=0x7ff64c0057f0 comp rx=0 tx=0).stop 2023-08-09T15:11:24.906 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:24.902+0000 7ff669513640 1 -- 172.21.15.43:0/2224861568 reap_dead start 2023-08-09T15:11:24.906 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:24.902+0000 7ff669d14640 1 -- 172.21.15.43:0/2224861568 shutdown_connections 2023-08-09T15:11:24.906 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:24.902+0000 7ff669d14640 1 -- 172.21.15.43:0/2224861568 >> 172.21.15.43:0/2224861568 conn(0x7ff66406a230 msgr2=0x7ff66406a640 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-09T15:11:24.906 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:24.902+0000 7ff669d14640 1 -- 172.21.15.43:0/2224861568 shutdown_connections 2023-08-09T15:11:24.906 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:24.902+0000 7ff669d14640 1 -- 172.21.15.43:0/2224861568 wait complete. 2023-08-09T15:11:24.906 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:24.904+0000 7ff669d14640 1 Processor -- start 2023-08-09T15:11:24.906 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:24.904+0000 7ff669d14640 1 -- start start 2023-08-09T15:11:24.908 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:24.905+0000 7ff669d14640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff64c003970 0x7ff6641490c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T15:11:24.908 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:24.905+0000 7ff669d14640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7ff6640c7cb0 0x7ff66415f290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T15:11:24.908 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:24.905+0000 7ff669d14640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7ff66414a7f0 0x7ff6640a8b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T15:11:24.909 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:24.905+0000 7ff669d14640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff6640c8130 con 0x7ff64c003970 2023-08-09T15:11:24.909 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:24.905+0000 7ff669d14640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff66415f7d0 con 0x7ff66414a7f0 2023-08-09T15:11:24.909 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:24.905+0000 7ff669d14640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff664149750 con 0x7ff6640c7cb0 2023-08-09T15:11:24.909 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:24.905+0000 7ff668d12640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff64c003970 0x7ff6641490c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T15:11:24.909 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:24.905+0000 7ff668d12640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff64c003970 0x7ff6641490c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:37944/0 (socket says 172.21.15.43:37944) 2023-08-09T15:11:24.909 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:24.905+0000 7ff668d12640 1 -- 172.21.15.43:0/464626380 learned_addr learned my addr 172.21.15.43:0/464626380 (peer_addr_for_me v2:172.21.15.43:0/0) 2023-08-09T15:11:24.909 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:24.906+0000 7ff668d12640 1 -- 172.21.15.43:0/464626380 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7ff6640c7cb0 msgr2=0x7ff66415f290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T15:11:24.910 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:24.906+0000 7ff65bfff640 1 --2- 172.21.15.43:0/464626380 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7ff6640c7cb0 0x7ff66415f290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T15:11:24.910 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:24.906+0000 7ff669513640 1 --2- 172.21.15.43:0/464626380 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7ff66414a7f0 0x7ff6640a8b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T15:11:24.910 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:24.906+0000 7ff668d12640 1 --2- 172.21.15.43:0/464626380 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7ff6640c7cb0 0x7ff66415f290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T15:11:24.910 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:24.906+0000 7ff668d12640 1 -- 172.21.15.43:0/464626380 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7ff66414a7f0 msgr2=0x7ff6640a8b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T15:11:24.910 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:24.906+0000 7ff668d12640 1 --2- 172.21.15.43:0/464626380 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7ff66414a7f0 0x7ff6640a8b90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T15:11:24.910 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:24.906+0000 7ff668d12640 1 -- 172.21.15.43:0/464626380 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff664149a30 con 0x7ff64c003970 2023-08-09T15:11:24.910 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:24.906+0000 7ff668d12640 1 --2- 172.21.15.43:0/464626380 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff64c003970 0x7ff6641490c0 secure :-1 s=READY pgs=2378 cs=0 l=1 rev1=1 crypto rx=0x7ff660001050 tx=0x7ff660000fe0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-09T15:11:24.911 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:24.907+0000 7ff6597fa640 1 -- 172.21.15.43:0/464626380 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff66002cad0 con 0x7ff64c003970 2023-08-09T15:11:24.911 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:24.907+0000 7ff6597fa640 1 -- 172.21.15.43:0/464626380 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff66002cc30 con 0x7ff64c003970 2023-08-09T15:11:24.911 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:24.907+0000 7ff6597fa640 1 -- 172.21.15.43:0/464626380 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff660007070 con 0x7ff64c003970 2023-08-09T15:11:24.911 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:24.907+0000 7ff669d14640 1 -- 172.21.15.43:0/464626380 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff664149d10 con 0x7ff64c003970 2023-08-09T15:11:24.911 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:24.907+0000 7ff669d14640 1 -- 172.21.15.43:0/464626380 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff66414a2d0 con 0x7ff64c003970 2023-08-09T15:11:24.911 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:24.908+0000 7ff669d14640 1 -- 172.21.15.43:0/464626380 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff66414a490 con 0x7ff64c003970 2023-08-09T15:11:24.912 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:24.909+0000 7ff6597fa640 1 -- 172.21.15.43:0/464626380 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 118) v1 ==== 88443+0+0 (secure 0 0 0) 0x7ff660032070 con 0x7ff64c003970 2023-08-09T15:11:24.912 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:24.910+0000 7ff6597fa640 1 --2- 172.21.15.43:0/464626380 >> [v2:172.21.15.203:6832/4034093268,v1:172.21.15.203:6833/4034093268] conn(0x7ff65006a170 0x7ff65006c630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T15:11:24.912 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:24.910+0000 7ff6597fa640 1 -- 172.21.15.43:0/464626380 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(219..219 src has 1..219) v4 ==== 5574+0+0 (secure 0 0 0) 0x7ff6600a95f0 con 0x7ff64c003970 2023-08-09T15:11:24.912 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:24.911+0000 7ff65bfff640 1 --2- 172.21.15.43:0/464626380 >> [v2:172.21.15.203:6832/4034093268,v1:172.21.15.203:6833/4034093268] conn(0x7ff65006a170 0x7ff65006c630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T15:11:24.913 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:24.912+0000 7ff65bfff640 1 --2- 172.21.15.43:0/464626380 >> [v2:172.21.15.203:6832/4034093268,v1:172.21.15.203:6833/4034093268] conn(0x7ff65006a170 0x7ff65006c630 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7ff664148f90 tx=0x7ff654008040 comp rx=0 tx=0).ready entity=mgr.8833 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-09T15:11:24.915 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:24.915+0000 7ff6597fa640 1 -- 172.21.15.43:0/464626380 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178066 (secure 0 0 0) 0x7ff660074d10 con 0x7ff64c003970 2023-08-09T15:11:25.108 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:25.108+0000 7ff669d14640 1 -- 172.21.15.43:0/464626380 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"} v 0) v1 -- 0x7ff66404e390 con 0x7ff64c003970 2023-08-09T15:11:25.109 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:25.109+0000 7ff6597fa640 1 -- 172.21.15.43:0/464626380 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"}]=0 v219) v1 ==== 92+0+10 (secure 0 0 0) 0x7ff660078330 con 0x7ff64c003970 2023-08-09T15:11:25.110 INFO:teuthology.orchestra.run.smithi043.stdout:pg_num: 8 2023-08-09T15:11:25.113 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:25.112+0000 7ff669d14640 1 -- 172.21.15.43:0/464626380 >> [v2:172.21.15.203:6832/4034093268,v1:172.21.15.203:6833/4034093268] conn(0x7ff65006a170 msgr2=0x7ff65006c630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T15:11:25.113 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:25.112+0000 7ff669d14640 1 --2- 172.21.15.43:0/464626380 >> [v2:172.21.15.203:6832/4034093268,v1:172.21.15.203:6833/4034093268] conn(0x7ff65006a170 0x7ff65006c630 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7ff664148f90 tx=0x7ff654008040 comp rx=0 tx=0).stop 2023-08-09T15:11:25.114 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:25.112+0000 7ff669d14640 1 -- 172.21.15.43:0/464626380 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff64c003970 msgr2=0x7ff6641490c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T15:11:25.114 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:25.112+0000 7ff669d14640 1 --2- 172.21.15.43:0/464626380 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff64c003970 0x7ff6641490c0 secure :-1 s=READY pgs=2378 cs=0 l=1 rev1=1 crypto rx=0x7ff660001050 tx=0x7ff660000fe0 comp rx=0 tx=0).stop 2023-08-09T15:11:25.114 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:25.112+0000 7ff669d14640 1 -- 172.21.15.43:0/464626380 shutdown_connections 2023-08-09T15:11:25.115 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:25.112+0000 7ff669d14640 1 --2- 172.21.15.43:0/464626380 >> [v2:172.21.15.203:6832/4034093268,v1:172.21.15.203:6833/4034093268] conn(0x7ff65006a170 0x7ff65006c630 unknown :-1 s=CLOSED pgs=52 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T15:11:25.115 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:25.112+0000 7ff669d14640 1 --2- 172.21.15.43:0/464626380 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7ff66414a7f0 0x7ff6640a8b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T15:11:25.115 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:25.112+0000 7ff669d14640 1 --2- 172.21.15.43:0/464626380 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7ff6640c7cb0 0x7ff66415f290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T15:11:25.115 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:25.112+0000 7ff669d14640 1 --2- 172.21.15.43:0/464626380 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff64c003970 0x7ff6641490c0 unknown :-1 s=CLOSED pgs=2378 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T15:11:25.116 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:25.112+0000 7ff669d14640 1 -- 172.21.15.43:0/464626380 >> 172.21.15.43:0/464626380 conn(0x7ff66406a230 msgr2=0x7ff6640cccc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-09T15:11:25.116 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:25.112+0000 7ff669d14640 1 -- 172.21.15.43:0/464626380 shutdown_connections 2023-08-09T15:11:25.116 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:25.112+0000 7ff669d14640 1 -- 172.21.15.43:0/464626380 wait complete. 2023-08-09T15:11:25.131 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-08-09T15:11:25.362 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:25.362+0000 7fa2dcf7f640 1 Processor -- start 2023-08-09T15:11:25.363 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:25.362+0000 7fa2dcf7f640 1 -- start start 2023-08-09T15:11:25.364 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:25.364+0000 7fa2dcf7f640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fa2d8157ea0 0x7fa2d80c97d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T15:11:25.364 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:25.364+0000 7fa2dcf7f640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7fa2d80c90a0 0x7fa2d80c9d10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T15:11:25.365 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:25.364+0000 7fa2dcf7f640 1 -- --> v1:172.21.15.43:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa2d8158280 con 0x7fa2d815a800 2023-08-09T15:11:25.365 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:25.364+0000 7fa2dcf7f640 1 -- --> v2:172.21.15.43:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa2d815b840 con 0x7fa2d80c90a0 2023-08-09T15:11:25.365 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:25.364+0000 7fa2dcf7f640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa2d815b350 con 0x7fa2d8157ea0 2023-08-09T15:11:25.365 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:25.364+0000 7fa2d6ffd640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7fa2d80c90a0 0x7fa2d80c9d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T15:11:25.365 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:25.364+0000 7fa2d6ffd640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7fa2d80c90a0 0x7fa2d80c9d10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:55940/0 (socket says 172.21.15.43:55940) 2023-08-09T15:11:25.365 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:25.364+0000 7fa2d6ffd640 1 -- 172.21.15.43:0/1163603606 learned_addr learned my addr 172.21.15.43:0/1163603606 (peer_addr_for_me v2:172.21.15.43:0/0) 2023-08-09T15:11:25.365 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:25.364+0000 7fa2d7fff640 1 --1- 172.21.15.43:0/1163603606 >> v1:172.21.15.43:6789/0 conn(0x7fa2d815a800 0x7fa2d815ac40 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.43:6789/0 says I am v1:172.21.15.43:42856/0 (socket says 172.21.15.43:42856) 2023-08-09T15:11:25.366 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:25.365+0000 7fa2d77fe640 1 --2- 172.21.15.43:0/1163603606 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fa2d8157ea0 0x7fa2d80c97d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T15:11:25.366 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:25.365+0000 7fa2d6ffd640 1 -- 172.21.15.43:0/1163603606 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fa2d8157ea0 msgr2=0x7fa2d80c97d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-09T15:11:25.367 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:25.365+0000 7fa2d6ffd640 1 --2- 172.21.15.43:0/1163603606 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fa2d8157ea0 0x7fa2d80c97d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T15:11:25.367 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:25.365+0000 7fa2d6ffd640 1 -- 172.21.15.43:0/1163603606 >> v1:172.21.15.43:6789/0 conn(0x7fa2d815a800 legacy=0x7fa2d815ac40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T15:11:25.367 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:25.365+0000 7fa2d6ffd640 1 -- 172.21.15.43:0/1163603606 --> v2:172.21.15.43:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa2d815b560 con 0x7fa2d80c90a0 2023-08-09T15:11:25.367 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:25.366+0000 7fa2d6ffd640 1 --2- 172.21.15.43:0/1163603606 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa2d80c90a0 0x7fa2d80c9d10 secure :-1 s=READY pgs=2380 cs=0 l=1 rev1=1 crypto rx=0x7fa2cc009870 tx=0x7fa2cc02b980 comp rx=0 tx=0).ready entity=mon.0 client_cookie=78fc460d90c6ecbf server_cookie=0 in_seq=0 out_seq=0 2023-08-09T15:11:25.367 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:25.366+0000 7fa2d67fc640 1 -- 172.21.15.43:0/1163603606 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa2cc030840 con 0x7fa2d80c90a0 2023-08-09T15:11:25.367 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:25.366+0000 7fa2d67fc640 1 -- 172.21.15.43:0/1163603606 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa2cc0309a0 con 0x7fa2d80c90a0 2023-08-09T15:11:25.367 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:25.366+0000 7fa2d67fc640 1 -- 172.21.15.43:0/1163603606 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa2cc030d30 con 0x7fa2d80c90a0 2023-08-09T15:11:25.368 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:25.367+0000 7fa2dcf7f640 1 -- 172.21.15.43:0/1163603606 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa2d80c90a0 msgr2=0x7fa2d80c9d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T15:11:25.368 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:25.367+0000 7fa2dcf7f640 1 --2- 172.21.15.43:0/1163603606 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa2d80c90a0 0x7fa2d80c9d10 secure :-1 s=READY pgs=2380 cs=0 l=1 rev1=1 crypto rx=0x7fa2cc009870 tx=0x7fa2cc02b980 comp rx=0 tx=0).stop 2023-08-09T15:11:25.368 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:25.367+0000 7fa2dcf7f640 1 -- 172.21.15.43:0/1163603606 shutdown_connections 2023-08-09T15:11:25.369 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:25.367+0000 7fa2dcf7f640 1 --2- 172.21.15.43:0/1163603606 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa2d80c90a0 0x7fa2d80c9d10 unknown :-1 s=CLOSED pgs=2380 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T15:11:25.369 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:25.367+0000 7fa2dcf7f640 1 --2- 172.21.15.43:0/1163603606 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fa2d8157ea0 0x7fa2d80c97d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T15:11:25.369 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:25.367+0000 7fa2dcf7f640 1 -- 172.21.15.43:0/1163603606 >> 172.21.15.43:0/1163603606 conn(0x7fa2d80cddd0 msgr2=0x7fa2d80ce1e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-09T15:11:25.369 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:25.367+0000 7fa2dcf7f640 1 -- 172.21.15.43:0/1163603606 shutdown_connections 2023-08-09T15:11:25.369 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:25.367+0000 7fa2dcf7f640 1 -- 172.21.15.43:0/1163603606 wait complete. 2023-08-09T15:11:25.370 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:25.369+0000 7fa2dcf7f640 1 Processor -- start 2023-08-09T15:11:25.370 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:25.369+0000 7fa2dcf7f640 1 -- start start 2023-08-09T15:11:25.371 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:25.370+0000 7fa2dcf7f640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fa2d80c90a0 0x7fa2d81505c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T15:11:25.371 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:25.370+0000 7fa2dcf7f640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fa2d8157ea0 0x7fa2d815c380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T15:11:25.371 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:25.370+0000 7fa2dcf7f640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa2d815a800 0x7fa2d815c8c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T15:11:25.372 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:25.370+0000 7fa2dcf7f640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa2d815dbf0 con 0x7fa2d815a800 2023-08-09T15:11:25.372 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:25.370+0000 7fa2dcf7f640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa2d80c1900 con 0x7fa2d8157ea0 2023-08-09T15:11:25.372 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:25.370+0000 7fa2dcf7f640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa2d8154920 con 0x7fa2d80c90a0 2023-08-09T15:11:25.372 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:25.370+0000 7fa2d7fff640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa2d815a800 0x7fa2d815c8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T15:11:25.372 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:25.370+0000 7fa2d7fff640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa2d815a800 0x7fa2d815c8c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:55948/0 (socket says 172.21.15.43:55948) 2023-08-09T15:11:25.372 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:25.370+0000 7fa2d7fff640 1 -- 172.21.15.43:0/609425451 learned_addr learned my addr 172.21.15.43:0/609425451 (peer_addr_for_me v2:172.21.15.43:0/0) 2023-08-09T15:11:25.372 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:25.371+0000 7fa2d6ffd640 1 --2- 172.21.15.43:0/609425451 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fa2d8157ea0 0x7fa2d815c380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T15:11:25.373 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:25.371+0000 7fa2d77fe640 1 --2- 172.21.15.43:0/609425451 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fa2d80c90a0 0x7fa2d81505c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T15:11:25.373 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:25.371+0000 7fa2d7fff640 1 -- 172.21.15.43:0/609425451 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fa2d80c90a0 msgr2=0x7fa2d81505c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T15:11:25.373 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:25.371+0000 7fa2d7fff640 1 --2- 172.21.15.43:0/609425451 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fa2d80c90a0 0x7fa2d81505c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T15:11:25.373 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:25.371+0000 7fa2d7fff640 1 -- 172.21.15.43:0/609425451 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fa2d8157ea0 msgr2=0x7fa2d815c380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T15:11:25.373 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:25.371+0000 7fa2d7fff640 1 --2- 172.21.15.43:0/609425451 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fa2d8157ea0 0x7fa2d815c380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T15:11:25.373 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:25.371+0000 7fa2d7fff640 1 -- 172.21.15.43:0/609425451 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa2cc0094d0 con 0x7fa2d815a800 2023-08-09T15:11:25.374 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:25.371+0000 7fa2d6ffd640 1 --2- 172.21.15.43:0/609425451 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fa2d8157ea0 0x7fa2d815c380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-08-09T15:11:25.374 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:25.371+0000 7fa2d7fff640 1 --2- 172.21.15.43:0/609425451 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa2d815a800 0x7fa2d815c8c0 secure :-1 s=READY pgs=2381 cs=0 l=1 rev1=1 crypto rx=0x7fa2d000c760 tx=0x7fa2d000cc20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-09T15:11:25.374 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:25.371+0000 7fa2bffff640 1 -- 172.21.15.43:0/609425451 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa2d000ce70 con 0x7fa2d815a800 2023-08-09T15:11:25.375 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:25.371+0000 7fa2bffff640 1 -- 172.21.15.43:0/609425451 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa2d000a640 con 0x7fa2d815a800 2023-08-09T15:11:25.375 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:25.371+0000 7fa2bffff640 1 -- 172.21.15.43:0/609425451 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa2d000a9b0 con 0x7fa2d815a800 2023-08-09T15:11:25.375 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:25.371+0000 7fa2dcf7f640 1 -- 172.21.15.43:0/609425451 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa2d80c1b50 con 0x7fa2d815a800 2023-08-09T15:11:25.375 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:25.372+0000 7fa2dcf7f640 1 -- 172.21.15.43:0/609425451 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa2d80c2110 con 0x7fa2d815a800 2023-08-09T15:11:25.375 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:25.373+0000 7fa2dcf7f640 1 -- 172.21.15.43:0/609425451 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa2d804e390 con 0x7fa2d815a800 2023-08-09T15:11:25.375 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:25.374+0000 7fa2bffff640 1 -- 172.21.15.43:0/609425451 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 118) v1 ==== 88443+0+0 (secure 0 0 0) 0x7fa2d000ab10 con 0x7fa2d815a800 2023-08-09T15:11:25.376 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:25.375+0000 7fa2bffff640 1 --2- 172.21.15.43:0/609425451 >> [v2:172.21.15.203:6832/4034093268,v1:172.21.15.203:6833/4034093268] conn(0x7fa2c406a1a0 0x7fa2c406c660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T15:11:25.376 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:25.375+0000 7fa2bffff640 1 -- 172.21.15.43:0/609425451 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(219..219 src has 1..219) v4 ==== 5574+0+0 (secure 0 0 0) 0x7fa2d0087ac0 con 0x7fa2d815a800 2023-08-09T15:11:25.376 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:25.376+0000 7fa2d77fe640 1 --2- 172.21.15.43:0/609425451 >> [v2:172.21.15.203:6832/4034093268,v1:172.21.15.203:6833/4034093268] conn(0x7fa2c406a1a0 0x7fa2c406c660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T15:11:25.377 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:25.377+0000 7fa2d77fe640 1 --2- 172.21.15.43:0/609425451 >> [v2:172.21.15.203:6832/4034093268,v1:172.21.15.203:6833/4034093268] conn(0x7fa2c406a1a0 0x7fa2c406c660 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7fa2c8006fd0 tx=0x7fa2c8008040 comp rx=0 tx=0).ready entity=mgr.8833 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-09T15:11:25.380 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:25.380+0000 7fa2bffff640 1 -- 172.21.15.43:0/609425451 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178066 (secure 0 0 0) 0x7fa2d0056d20 con 0x7fa2d815a800 2023-08-09T15:11:25.572 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:25.571+0000 7fa2dcf7f640 1 -- 172.21.15.43:0/609425451 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7fa2d8159400 con 0x7fa2d815a800 2023-08-09T15:11:25.572 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:25.572+0000 7fa2bffff640 1 -- 172.21.15.43:0/609425451 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v219) v1 ==== 93+0+10 (secure 0 0 0) 0x7fa2d8159400 con 0x7fa2d815a800 2023-08-09T15:11:25.573 INFO:teuthology.orchestra.run.smithi043.stdout:pg_num: 1 2023-08-09T15:11:25.575 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:25.575+0000 7fa2dcf7f640 1 -- 172.21.15.43:0/609425451 >> [v2:172.21.15.203:6832/4034093268,v1:172.21.15.203:6833/4034093268] conn(0x7fa2c406a1a0 msgr2=0x7fa2c406c660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T15:11:25.575 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:25.575+0000 7fa2dcf7f640 1 --2- 172.21.15.43:0/609425451 >> [v2:172.21.15.203:6832/4034093268,v1:172.21.15.203:6833/4034093268] conn(0x7fa2c406a1a0 0x7fa2c406c660 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7fa2c8006fd0 tx=0x7fa2c8008040 comp rx=0 tx=0).stop 2023-08-09T15:11:25.575 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:25.575+0000 7fa2dcf7f640 1 -- 172.21.15.43:0/609425451 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa2d815a800 msgr2=0x7fa2d815c8c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T15:11:25.576 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:25.575+0000 7fa2dcf7f640 1 --2- 172.21.15.43:0/609425451 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa2d815a800 0x7fa2d815c8c0 secure :-1 s=READY pgs=2381 cs=0 l=1 rev1=1 crypto rx=0x7fa2d000c760 tx=0x7fa2d000cc20 comp rx=0 tx=0).stop 2023-08-09T15:11:25.576 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:25.575+0000 7fa2dcf7f640 1 -- 172.21.15.43:0/609425451 shutdown_connections 2023-08-09T15:11:25.577 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:25.575+0000 7fa2dcf7f640 1 --2- 172.21.15.43:0/609425451 >> [v2:172.21.15.203:6832/4034093268,v1:172.21.15.203:6833/4034093268] conn(0x7fa2c406a1a0 0x7fa2c406c660 unknown :-1 s=CLOSED pgs=53 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T15:11:25.577 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:25.575+0000 7fa2dcf7f640 1 --2- 172.21.15.43:0/609425451 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa2d815a800 0x7fa2d815c8c0 unknown :-1 s=CLOSED pgs=2381 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T15:11:25.577 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:25.575+0000 7fa2dcf7f640 1 --2- 172.21.15.43:0/609425451 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fa2d8157ea0 0x7fa2d815c380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T15:11:25.577 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:25.575+0000 7fa2dcf7f640 1 --2- 172.21.15.43:0/609425451 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fa2d80c90a0 0x7fa2d81505c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T15:11:25.577 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:25.575+0000 7fa2dcf7f640 1 -- 172.21.15.43:0/609425451 >> 172.21.15.43:0/609425451 conn(0x7fa2d80cddd0 msgr2=0x7fa2d80c95c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-09T15:11:25.577 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:25.575+0000 7fa2dcf7f640 1 -- 172.21.15.43:0/609425451 shutdown_connections 2023-08-09T15:11:25.577 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:25.576+0000 7fa2dcf7f640 1 -- 172.21.15.43:0/609425451 wait complete. 2023-08-09T15:11:25.594 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --format=json-pretty osd lspools 2023-08-09T15:11:25.812 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:25.812+0000 7fe2706cc640 1 Processor -- start 2023-08-09T15:11:25.812 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:25.812+0000 7fe2706cc640 1 -- start start 2023-08-09T15:11:25.814 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:25.813+0000 7fe2706cc640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fe26806b140 0x7fe26806b520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T15:11:25.814 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:25.814+0000 7fe2706cc640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7fe2680cf760 0x7fe26806dcb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T15:11:25.814 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:25.814+0000 7fe2706cc640 1 -- --> v1:172.21.15.43:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe2680af1a0 con 0x7fe2680af6f0 2023-08-09T15:11:25.814 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:25.814+0000 7fe2706cc640 1 -- --> v2:172.21.15.43:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe26806be10 con 0x7fe2680cf760 2023-08-09T15:11:25.814 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:25.814+0000 7fe2706cc640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe26806afe0 con 0x7fe26806b140 2023-08-09T15:11:25.815 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:25.814+0000 7fe26ec69640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7fe2680cf760 0x7fe26806dcb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T15:11:25.815 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:25.814+0000 7fe26dc67640 1 --1- >> v1:172.21.15.43:6789/0 conn(0x7fe2680af6f0 0x7fe26806a8d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.43:6789/0 says I am v1:172.21.15.43:42864/0 (socket says 172.21.15.43:42864) 2023-08-09T15:11:25.816 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:25.814+0000 7fe26ec69640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7fe2680cf760 0x7fe26806dcb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:55954/0 (socket says 172.21.15.43:55954) 2023-08-09T15:11:25.816 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:25.814+0000 7fe26dc67640 1 -- 172.21.15.43:0/4239084355 learned_addr learned my addr 172.21.15.43:0/4239084355 (peer_addr_for_me v1:172.21.15.43:0/0) 2023-08-09T15:11:25.816 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:25.814+0000 7fe26e468640 1 --2- 172.21.15.43:0/4239084355 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fe26806b140 0x7fe26806b520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T15:11:25.816 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:25.815+0000 7fe26ec69640 1 -- 172.21.15.43:0/4239084355 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fe26806b140 msgr2=0x7fe26806b520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-09T15:11:25.816 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:25.815+0000 7fe26ec69640 1 --2- 172.21.15.43:0/4239084355 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fe26806b140 0x7fe26806b520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T15:11:25.816 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:25.815+0000 7fe26ec69640 1 -- 172.21.15.43:0/4239084355 >> v1:172.21.15.43:6789/0 conn(0x7fe2680af6f0 legacy=0x7fe26806a8d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T15:11:25.817 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:25.815+0000 7fe26ec69640 1 -- 172.21.15.43:0/4239084355 --> v2:172.21.15.43:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe2680cdc20 con 0x7fe2680cf760 2023-08-09T15:11:25.817 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:25.815+0000 7fe26d466640 1 -- 172.21.15.43:0/4239084355 <== mon.0 v1:172.21.15.43:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 346292566 0 0) 0x7fe2680af1a0 con 0x7fe2680af6f0 2023-08-09T15:11:25.818 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:25.816+0000 7fe26ec69640 1 --2- 172.21.15.43:0/4239084355 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fe2680cf760 0x7fe26806dcb0 secure :-1 s=READY pgs=2383 cs=0 l=1 rev1=1 crypto rx=0x7fe260009d90 tx=0x7fe26002beb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1d832df2286ebf86 server_cookie=0 in_seq=0 out_seq=0 2023-08-09T15:11:25.818 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:25.816+0000 7fe26d466640 1 -- 172.21.15.43:0/4239084355 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe26002cc60 con 0x7fe2680cf760 2023-08-09T15:11:25.818 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:25.816+0000 7fe26d466640 1 -- 172.21.15.43:0/4239084355 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe26002cdc0 con 0x7fe2680cf760 2023-08-09T15:11:25.818 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:25.816+0000 7fe26d466640 1 -- 172.21.15.43:0/4239084355 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe260034660 con 0x7fe2680cf760 2023-08-09T15:11:25.818 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:25.817+0000 7fe2706cc640 1 -- 172.21.15.43:0/4239084355 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fe2680cf760 msgr2=0x7fe26806dcb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T15:11:25.818 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:25.817+0000 7fe2706cc640 1 --2- 172.21.15.43:0/4239084355 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fe2680cf760 0x7fe26806dcb0 secure :-1 s=READY pgs=2383 cs=0 l=1 rev1=1 crypto rx=0x7fe260009d90 tx=0x7fe26002beb0 comp rx=0 tx=0).stop 2023-08-09T15:11:25.819 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:25.817+0000 7fe2706cc640 1 -- 172.21.15.43:0/4239084355 shutdown_connections 2023-08-09T15:11:25.819 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:25.817+0000 7fe2706cc640 1 --2- 172.21.15.43:0/4239084355 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fe2680cf760 0x7fe26806dcb0 unknown :-1 s=CLOSED pgs=2383 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T15:11:25.819 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:25.817+0000 7fe2706cc640 1 --2- 172.21.15.43:0/4239084355 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fe26806b140 0x7fe26806b520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T15:11:25.819 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:25.817+0000 7fe2706cc640 1 -- 172.21.15.43:0/4239084355 >> 172.21.15.43:0/4239084355 conn(0x7fe2680ccd80 msgr2=0x7fe2680cd190 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-09T15:11:25.819 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:25.817+0000 7fe2706cc640 1 -- 172.21.15.43:0/4239084355 shutdown_connections 2023-08-09T15:11:25.819 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:25.817+0000 7fe2706cc640 1 -- 172.21.15.43:0/4239084355 wait complete. 2023-08-09T15:11:25.820 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:25.819+0000 7fe2706cc640 1 Processor -- start 2023-08-09T15:11:25.820 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:25.819+0000 7fe2706cc640 1 -- start start 2023-08-09T15:11:25.822 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:25.820+0000 7fe2706cc640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fe26806b140 0x7fe2681540d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T15:11:25.822 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:25.820+0000 7fe2706cc640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fe2680af6f0 0x7fe26809d9b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T15:11:25.822 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:25.820+0000 7fe2706cc640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fe2680cf760 0x7fe2680adf40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T15:11:25.822 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:25.820+0000 7fe2706cc640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe26809acb0 con 0x7fe2680cf760 2023-08-09T15:11:25.822 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:25.820+0000 7fe2706cc640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe26809a030 con 0x7fe2680af6f0 2023-08-09T15:11:25.823 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:25.820+0000 7fe2706cc640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe26809a310 con 0x7fe26806b140 2023-08-09T15:11:25.823 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:25.820+0000 7fe26ec69640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fe2680cf760 0x7fe2680adf40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T15:11:25.823 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:25.820+0000 7fe26ec69640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fe2680cf760 0x7fe2680adf40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:55964/0 (socket says 172.21.15.43:55964) 2023-08-09T15:11:25.823 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:25.820+0000 7fe26ec69640 1 -- 172.21.15.43:0/3398312036 learned_addr learned my addr 172.21.15.43:0/3398312036 (peer_addr_for_me v2:172.21.15.43:0/0) 2023-08-09T15:11:25.823 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:25.821+0000 7fe26e468640 1 --2- 172.21.15.43:0/3398312036 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fe26806b140 0x7fe2681540d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T15:11:25.823 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:25.821+0000 7fe26dc67640 1 --2- 172.21.15.43:0/3398312036 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fe2680af6f0 0x7fe26809d9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T15:11:25.823 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:25.821+0000 7fe26ec69640 1 -- 172.21.15.43:0/3398312036 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fe26806b140 msgr2=0x7fe2681540d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T15:11:25.824 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:25.821+0000 7fe26ec69640 1 --2- 172.21.15.43:0/3398312036 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fe26806b140 0x7fe2681540d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T15:11:25.824 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:25.821+0000 7fe26ec69640 1 -- 172.21.15.43:0/3398312036 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fe2680af6f0 msgr2=0x7fe26809d9b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T15:11:25.824 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:25.821+0000 7fe26ec69640 1 --2- 172.21.15.43:0/3398312036 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fe2680af6f0 0x7fe26809d9b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T15:11:25.824 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:25.821+0000 7fe26ec69640 1 -- 172.21.15.43:0/3398312036 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe260009a70 con 0x7fe2680cf760 2023-08-09T15:11:25.824 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:25.821+0000 7fe26dc67640 1 --2- 172.21.15.43:0/3398312036 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fe2680af6f0 0x7fe26809d9b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-08-09T15:11:25.824 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:25.821+0000 7fe26e468640 1 --2- 172.21.15.43:0/3398312036 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fe26806b140 0x7fe2681540d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-08-09T15:11:25.825 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:25.821+0000 7fe26ec69640 1 --2- 172.21.15.43:0/3398312036 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fe2680cf760 0x7fe2680adf40 secure :-1 s=READY pgs=2384 cs=0 l=1 rev1=1 crypto rx=0x7fe260009eb0 tx=0x7fe260004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-09T15:11:25.825 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:25.822+0000 7fe25effd640 1 -- 172.21.15.43:0/3398312036 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe26003e610 con 0x7fe2680cf760 2023-08-09T15:11:25.825 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:25.822+0000 7fe25effd640 1 -- 172.21.15.43:0/3398312036 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe26003e770 con 0x7fe2680cf760 2023-08-09T15:11:25.825 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:25.822+0000 7fe25effd640 1 -- 172.21.15.43:0/3398312036 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe26003eb00 con 0x7fe2680cf760 2023-08-09T15:11:25.826 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:25.822+0000 7fe2706cc640 1 -- 172.21.15.43:0/3398312036 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe26809a5f0 con 0x7fe2680cf760 2023-08-09T15:11:25.826 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:25.822+0000 7fe2706cc640 1 -- 172.21.15.43:0/3398312036 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe26809def0 con 0x7fe2680cf760 2023-08-09T15:11:25.827 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:25.823+0000 7fe2706cc640 1 -- 172.21.15.43:0/3398312036 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe26809adf0 con 0x7fe2680cf760 2023-08-09T15:11:25.828 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:25.824+0000 7fe25effd640 1 -- 172.21.15.43:0/3398312036 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 118) v1 ==== 88443+0+0 (secure 0 0 0) 0x7fe26003b050 con 0x7fe2680cf760 2023-08-09T15:11:25.828 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:25.825+0000 7fe25effd640 1 --2- 172.21.15.43:0/3398312036 >> [v2:172.21.15.203:6832/4034093268,v1:172.21.15.203:6833/4034093268] conn(0x7fe24c06a140 0x7fe24c06c600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T15:11:25.828 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:25.825+0000 7fe25effd640 1 -- 172.21.15.43:0/3398312036 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(219..219 src has 1..219) v4 ==== 5574+0+0 (secure 0 0 0) 0x7fe2600aa970 con 0x7fe2680cf760 2023-08-09T15:11:25.829 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:25.826+0000 7fe26e468640 1 --2- 172.21.15.43:0/3398312036 >> [v2:172.21.15.203:6832/4034093268,v1:172.21.15.203:6833/4034093268] conn(0x7fe24c06a140 0x7fe24c06c600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T15:11:25.829 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:25.827+0000 7fe26e468640 1 --2- 172.21.15.43:0/3398312036 >> [v2:172.21.15.203:6832/4034093268,v1:172.21.15.203:6833/4034093268] conn(0x7fe24c06a140 0x7fe24c06c600 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7fe264009780 tx=0x7fe264008040 comp rx=0 tx=0).ready entity=mgr.8833 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-09T15:11:25.830 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:25.829+0000 7fe25effd640 1 -- 172.21.15.43:0/3398312036 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178066 (secure 0 0 0) 0x7fe26809adf0 con 0x7fe2680cf760 2023-08-09T15:11:26.027 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:26.027+0000 7fe2706cc640 1 -- 172.21.15.43:0/3398312036 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) v1 -- 0x7fe26809adf0 con 0x7fe2680cf760 2023-08-09T15:11:26.028 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:26.028+0000 7fe25effd640 1 -- 172.21.15.43:0/3398312036 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v219) v1 ==== 84+0+126 (secure 0 0 0) 0x7fe26809adf0 con 0x7fe2680cf760 2023-08-09T15:11:26.028 INFO:teuthology.orchestra.run.smithi043.stdout: 2023-08-09T15:11:26.029 INFO:teuthology.orchestra.run.smithi043.stdout:[ 2023-08-09T15:11:26.029 INFO:teuthology.orchestra.run.smithi043.stdout: { 2023-08-09T15:11:26.029 INFO:teuthology.orchestra.run.smithi043.stdout: "poolnum": 1, 2023-08-09T15:11:26.029 INFO:teuthology.orchestra.run.smithi043.stdout: "poolname": "rbd" 2023-08-09T15:11:26.029 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2023-08-09T15:11:26.029 INFO:teuthology.orchestra.run.smithi043.stdout: { 2023-08-09T15:11:26.029 INFO:teuthology.orchestra.run.smithi043.stdout: "poolnum": 2, 2023-08-09T15:11:26.030 INFO:teuthology.orchestra.run.smithi043.stdout: "poolname": ".mgr" 2023-08-09T15:11:26.030 INFO:teuthology.orchestra.run.smithi043.stdout: } 2023-08-09T15:11:26.030 INFO:teuthology.orchestra.run.smithi043.stdout:] 2023-08-09T15:11:26.030 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:26.030+0000 7fe2706cc640 1 -- 172.21.15.43:0/3398312036 >> [v2:172.21.15.203:6832/4034093268,v1:172.21.15.203:6833/4034093268] conn(0x7fe24c06a140 msgr2=0x7fe24c06c600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T15:11:26.031 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:26.030+0000 7fe2706cc640 1 --2- 172.21.15.43:0/3398312036 >> [v2:172.21.15.203:6832/4034093268,v1:172.21.15.203:6833/4034093268] conn(0x7fe24c06a140 0x7fe24c06c600 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7fe264009780 tx=0x7fe264008040 comp rx=0 tx=0).stop 2023-08-09T15:11:26.031 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:26.030+0000 7fe2706cc640 1 -- 172.21.15.43:0/3398312036 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fe2680cf760 msgr2=0x7fe2680adf40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T15:11:26.031 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:26.030+0000 7fe2706cc640 1 --2- 172.21.15.43:0/3398312036 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fe2680cf760 0x7fe2680adf40 secure :-1 s=READY pgs=2384 cs=0 l=1 rev1=1 crypto rx=0x7fe260009eb0 tx=0x7fe260004a40 comp rx=0 tx=0).stop 2023-08-09T15:11:26.032 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:26.030+0000 7fe2706cc640 1 -- 172.21.15.43:0/3398312036 shutdown_connections 2023-08-09T15:11:26.032 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:26.031+0000 7fe2706cc640 1 --2- 172.21.15.43:0/3398312036 >> [v2:172.21.15.203:6832/4034093268,v1:172.21.15.203:6833/4034093268] conn(0x7fe24c06a140 0x7fe24c06c600 unknown :-1 s=CLOSED pgs=54 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T15:11:26.032 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:26.031+0000 7fe2706cc640 1 --2- 172.21.15.43:0/3398312036 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fe2680cf760 0x7fe2680adf40 unknown :-1 s=CLOSED pgs=2384 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T15:11:26.032 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:26.031+0000 7fe2706cc640 1 --2- 172.21.15.43:0/3398312036 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fe2680af6f0 0x7fe26809d9b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T15:11:26.032 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:26.031+0000 7fe2706cc640 1 --2- 172.21.15.43:0/3398312036 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fe26806b140 0x7fe2681540d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T15:11:26.032 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:26.031+0000 7fe2706cc640 1 -- 172.21.15.43:0/3398312036 >> 172.21.15.43:0/3398312036 conn(0x7fe2680ccd80 msgr2=0x7fe2680cd160 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-09T15:11:26.033 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:26.031+0000 7fe2706cc640 1 -- 172.21.15.43:0/3398312036 shutdown_connections 2023-08-09T15:11:26.033 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:26.031+0000 7fe2706cc640 1 -- 172.21.15.43:0/3398312036 wait complete. 2023-08-09T15:11:26.048 DEBUG:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2023-08-09T15:11:26.049 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_metadata --pg_num_min 64 2023-08-09T15:11:26.270 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:26.270+0000 7f112359e640 1 Processor -- start 2023-08-09T15:11:26.270 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:26.270+0000 7f112359e640 1 -- start start 2023-08-09T15:11:26.272 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:26.271+0000 7f112359e640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f11240cd3c0 0x7f11240cec70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T15:11:26.272 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:26.271+0000 7f112359e640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f11240cf1b0 0x7f11240b5880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T15:11:26.272 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:26.271+0000 7f112359e640 1 -- --> v1:172.21.15.43:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f11240cf5e0 con 0x7f112415c400 2023-08-09T15:11:26.272 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:26.271+0000 7f112359e640 1 -- --> v2:172.21.15.43:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f112415c7e0 con 0x7f11240cd3c0 2023-08-09T15:11:26.273 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:26.271+0000 7f112359e640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f11240d13e0 con 0x7f11240cf1b0 2023-08-09T15:11:26.273 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:26.272+0000 7f1119d9b640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f11240cd3c0 0x7f11240cec70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T15:11:26.273 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:26.272+0000 7f112259c640 1 --1- >> v1:172.21.15.43:6789/0 conn(0x7f112415c400 0x7f11240d0cd0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.43:6789/0 says I am v1:172.21.15.43:42868/0 (socket says 172.21.15.43:42868) 2023-08-09T15:11:26.273 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:26.272+0000 7f112259c640 1 -- 172.21.15.43:0/4260388761 learned_addr learned my addr 172.21.15.43:0/4260388761 (peer_addr_for_me v1:172.21.15.43:0/0) 2023-08-09T15:11:26.274 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:26.272+0000 7f1122d9d640 1 --2- 172.21.15.43:0/4260388761 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f11240cf1b0 0x7f11240b5880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T15:11:26.274 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:26.272+0000 7f1119d9b640 1 -- 172.21.15.43:0/4260388761 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f11240cf1b0 msgr2=0x7f11240b5880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-09T15:11:26.274 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:26.272+0000 7f1119d9b640 1 --2- 172.21.15.43:0/4260388761 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f11240cf1b0 0x7f11240b5880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T15:11:26.274 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:26.272+0000 7f1119d9b640 1 -- 172.21.15.43:0/4260388761 >> v1:172.21.15.43:6789/0 conn(0x7f112415c400 legacy=0x7f11240d0cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T15:11:26.275 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:26.272+0000 7f1119d9b640 1 -- 172.21.15.43:0/4260388761 --> v2:172.21.15.43:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f11240d1690 con 0x7f11240cd3c0 2023-08-09T15:11:26.275 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:26.272+0000 7f1121d9b640 1 -- 172.21.15.43:0/4260388761 <== mon.0 v1:172.21.15.43:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2358870971 0 0) 0x7f11240cf5e0 con 0x7f112415c400 2023-08-09T15:11:26.275 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:26.272+0000 7f1122d9d640 1 --2- 172.21.15.43:0/4260388761 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f11240cf1b0 0x7f11240b5880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-08-09T15:11:26.275 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:26.274+0000 7f1119d9b640 1 --2- 172.21.15.43:0/4260388761 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f11240cd3c0 0x7f11240cec70 secure :-1 s=READY pgs=2386 cs=0 l=1 rev1=1 crypto rx=0x7f110c0097a0 tx=0x7f110c02b940 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6cbe3b39474d0143 server_cookie=0 in_seq=0 out_seq=0 2023-08-09T15:11:26.275 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:26.274+0000 7f1121d9b640 1 -- 172.21.15.43:0/4260388761 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f110c030800 con 0x7f11240cd3c0 2023-08-09T15:11:26.275 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:26.274+0000 7f1121d9b640 1 -- 172.21.15.43:0/4260388761 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f110c030960 con 0x7f11240cd3c0 2023-08-09T15:11:26.275 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:26.274+0000 7f1121d9b640 1 -- 172.21.15.43:0/4260388761 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f110c030cf0 con 0x7f11240cd3c0 2023-08-09T15:11:26.276 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:26.274+0000 7f112359e640 1 -- 172.21.15.43:0/4260388761 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f11240cd3c0 msgr2=0x7f11240cec70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T15:11:26.276 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:26.274+0000 7f112359e640 1 --2- 172.21.15.43:0/4260388761 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f11240cd3c0 0x7f11240cec70 secure :-1 s=READY pgs=2386 cs=0 l=1 rev1=1 crypto rx=0x7f110c0097a0 tx=0x7f110c02b940 comp rx=0 tx=0).stop 2023-08-09T15:11:26.276 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:26.274+0000 7f112359e640 1 -- 172.21.15.43:0/4260388761 shutdown_connections 2023-08-09T15:11:26.277 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:26.274+0000 7f112359e640 1 --2- 172.21.15.43:0/4260388761 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f11240cf1b0 0x7f11240b5880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T15:11:26.277 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:26.274+0000 7f112359e640 1 --2- 172.21.15.43:0/4260388761 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f11240cd3c0 0x7f11240cec70 unknown :-1 s=CLOSED pgs=2386 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T15:11:26.277 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:26.274+0000 7f112359e640 1 -- 172.21.15.43:0/4260388761 >> 172.21.15.43:0/4260388761 conn(0x7f112406a320 msgr2=0x7f112406a730 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-09T15:11:26.277 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:26.275+0000 7f112359e640 1 -- 172.21.15.43:0/4260388761 shutdown_connections 2023-08-09T15:11:26.277 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:26.275+0000 7f112359e640 1 -- 172.21.15.43:0/4260388761 wait complete. 2023-08-09T15:11:26.278 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:26.276+0000 7f112359e640 1 Processor -- start 2023-08-09T15:11:26.278 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:26.276+0000 7f112359e640 1 -- start start 2023-08-09T15:11:26.278 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:26.277+0000 7f112359e640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f11240cd3c0 0x7f1124151450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T15:11:26.278 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:26.277+0000 7f112359e640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f11240cf1b0 0x7f11240c9560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T15:11:26.279 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:26.277+0000 7f112359e640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f112415c400 0x7f11240b67d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T15:11:26.279 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:26.277+0000 7f112359e640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f112415ebc0 con 0x7f112415c400 2023-08-09T15:11:26.279 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:26.277+0000 7f112359e640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1124151990 con 0x7f11240cf1b0 2023-08-09T15:11:26.279 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:26.277+0000 7f112359e640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f112415dfb0 con 0x7f11240cd3c0 2023-08-09T15:11:26.279 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:26.278+0000 7f1122d9d640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f112415c400 0x7f11240b67d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T15:11:26.279 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:26.278+0000 7f1122d9d640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f112415c400 0x7f11240b67d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:55988/0 (socket says 172.21.15.43:55988) 2023-08-09T15:11:26.279 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:26.278+0000 7f1122d9d640 1 -- 172.21.15.43:0/2094974825 learned_addr learned my addr 172.21.15.43:0/2094974825 (peer_addr_for_me v2:172.21.15.43:0/0) 2023-08-09T15:11:26.281 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:26.278+0000 7f1119d9b640 1 --2- 172.21.15.43:0/2094974825 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f11240cf1b0 0x7f11240c9560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T15:11:26.281 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:26.278+0000 7f112259c640 1 --2- 172.21.15.43:0/2094974825 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f11240cd3c0 0x7f1124151450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T15:11:26.281 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:26.278+0000 7f1122d9d640 1 -- 172.21.15.43:0/2094974825 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f11240cd3c0 msgr2=0x7f1124151450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T15:11:26.281 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:26.278+0000 7f1122d9d640 1 --2- 172.21.15.43:0/2094974825 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f11240cd3c0 0x7f1124151450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T15:11:26.282 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:26.278+0000 7f1122d9d640 1 -- 172.21.15.43:0/2094974825 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f11240cf1b0 msgr2=0x7f11240c9560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T15:11:26.282 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:26.278+0000 7f1122d9d640 1 --2- 172.21.15.43:0/2094974825 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f11240cf1b0 0x7f11240c9560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T15:11:26.282 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:26.278+0000 7f1122d9d640 1 -- 172.21.15.43:0/2094974825 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f110c009400 con 0x7f112415c400 2023-08-09T15:11:26.282 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:26.278+0000 7f1119d9b640 1 --2- 172.21.15.43:0/2094974825 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f11240cf1b0 0x7f11240c9560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-08-09T15:11:26.282 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:26.278+0000 7f1122d9d640 1 --2- 172.21.15.43:0/2094974825 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f112415c400 0x7f11240b67d0 secure :-1 s=READY pgs=2387 cs=0 l=1 rev1=1 crypto rx=0x7f111c00c7f0 tx=0x7f111c00ccb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-09T15:11:26.282 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:26.279+0000 7f111b7fe640 1 -- 172.21.15.43:0/2094974825 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f111c007820 con 0x7f112415c400 2023-08-09T15:11:26.282 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:26.279+0000 7f111b7fe640 1 -- 172.21.15.43:0/2094974825 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f111c007980 con 0x7f112415c400 2023-08-09T15:11:26.283 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:26.279+0000 7f112359e640 1 -- 172.21.15.43:0/2094974825 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f112415e290 con 0x7f112415c400 2023-08-09T15:11:26.283 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:26.279+0000 7f111b7fe640 1 -- 172.21.15.43:0/2094974825 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f111c007d70 con 0x7f112415c400 2023-08-09T15:11:26.283 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:26.279+0000 7f112359e640 1 -- 172.21.15.43:0/2094974825 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f112415e850 con 0x7f112415c400 2023-08-09T15:11:26.283 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:26.280+0000 7f112359e640 1 -- 172.21.15.43:0/2094974825 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1124066200 con 0x7f112415c400 2023-08-09T15:11:26.284 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:26.281+0000 7f111b7fe640 1 -- 172.21.15.43:0/2094974825 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 118) v1 ==== 88443+0+0 (secure 0 0 0) 0x7f111c0113f0 con 0x7f112415c400 2023-08-09T15:11:26.284 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:26.282+0000 7f111b7fe640 1 --2- 172.21.15.43:0/2094974825 >> [v2:172.21.15.203:6832/4034093268,v1:172.21.15.203:6833/4034093268] conn(0x7f110406a170 0x7f110406c630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T15:11:26.284 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:26.282+0000 7f111b7fe640 1 -- 172.21.15.43:0/2094974825 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(219..219 src has 1..219) v4 ==== 5574+0+0 (secure 0 0 0) 0x7f111c088a80 con 0x7f112415c400 2023-08-09T15:11:26.284 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:26.283+0000 7f112259c640 1 --2- 172.21.15.43:0/2094974825 >> [v2:172.21.15.203:6832/4034093268,v1:172.21.15.203:6833/4034093268] conn(0x7f110406a170 0x7f110406c630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T15:11:26.284 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:26.284+0000 7f112259c640 1 --2- 172.21.15.43:0/2094974825 >> [v2:172.21.15.203:6832/4034093268,v1:172.21.15.203:6833/4034093268] conn(0x7f110406a170 0x7f110406c630 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f11140012e0 tx=0x7f1114009040 comp rx=0 tx=0).ready entity=mgr.8833 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-09T15:11:26.286 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:26.286+0000 7f111b7fe640 1 -- 172.21.15.43:0/2094974825 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178066 (secure 0 0 0) 0x7f1124066200 con 0x7f112415c400 2023-08-09T15:11:26.479 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:26.479+0000 7f112359e640 1 -- 172.21.15.43:0/2094974825 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64} v 0) v1 -- 0x7f1124066200 con 0x7f112415c400 2023-08-09T15:11:26.660 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:26.660+0000 7f111b7fe640 1 -- 172.21.15.43:0/2094974825 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]=0 pool 'cephfs_metadata' created v220) v1 ==== 138+0+0 (secure 0 0 0) 0x7f1124066200 con 0x7f112415c400 2023-08-09T15:11:26.661 INFO:teuthology.orchestra.run.smithi043.stderr:pool 'cephfs_metadata' created 2023-08-09T15:11:26.668 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:26.667+0000 7f112359e640 1 -- 172.21.15.43:0/2094974825 >> [v2:172.21.15.203:6832/4034093268,v1:172.21.15.203:6833/4034093268] conn(0x7f110406a170 msgr2=0x7f110406c630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T15:11:26.668 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:26.667+0000 7f112359e640 1 --2- 172.21.15.43:0/2094974825 >> [v2:172.21.15.203:6832/4034093268,v1:172.21.15.203:6833/4034093268] conn(0x7f110406a170 0x7f110406c630 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f11140012e0 tx=0x7f1114009040 comp rx=0 tx=0).stop 2023-08-09T15:11:26.669 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:26.667+0000 7f112359e640 1 -- 172.21.15.43:0/2094974825 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f112415c400 msgr2=0x7f11240b67d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T15:11:26.669 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:26.667+0000 7f112359e640 1 --2- 172.21.15.43:0/2094974825 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f112415c400 0x7f11240b67d0 secure :-1 s=READY pgs=2387 cs=0 l=1 rev1=1 crypto rx=0x7f111c00c7f0 tx=0x7f111c00ccb0 comp rx=0 tx=0).stop 2023-08-09T15:11:26.670 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:26.668+0000 7f112359e640 1 -- 172.21.15.43:0/2094974825 shutdown_connections 2023-08-09T15:11:26.670 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:26.668+0000 7f112359e640 1 --2- 172.21.15.43:0/2094974825 >> [v2:172.21.15.203:6832/4034093268,v1:172.21.15.203:6833/4034093268] conn(0x7f110406a170 0x7f110406c630 unknown :-1 s=CLOSED pgs=55 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T15:11:26.671 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:26.668+0000 7f112359e640 1 --2- 172.21.15.43:0/2094974825 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f112415c400 0x7f11240b67d0 unknown :-1 s=CLOSED pgs=2387 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T15:11:26.671 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:26.668+0000 7f112359e640 1 --2- 172.21.15.43:0/2094974825 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f11240cf1b0 0x7f11240c9560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T15:11:26.671 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:26.668+0000 7f112359e640 1 --2- 172.21.15.43:0/2094974825 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f11240cd3c0 0x7f1124151450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T15:11:26.672 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:26.668+0000 7f112359e640 1 -- 172.21.15.43:0/2094974825 >> 172.21.15.43:0/2094974825 conn(0x7f112406a320 msgr2=0x7f11240c69c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-09T15:11:26.672 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:26.669+0000 7f112359e640 1 -- 172.21.15.43:0/2094974825 shutdown_connections 2023-08-09T15:11:26.672 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:26.669+0000 7f112359e640 1 -- 172.21.15.43:0/2094974825 wait complete. 2023-08-09T15:11:26.691 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data 64 --pg_num_min 64 --target_size_ratio 0.9 2023-08-09T15:11:26.915 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:26.915+0000 7f028f59e640 1 Processor -- start 2023-08-09T15:11:26.916 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:26.915+0000 7f028f59e640 1 -- start start 2023-08-09T15:11:26.917 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:26.917+0000 7f028f59e640 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7f029015c1a0 0x7f029006db30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T15:11:26.917 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:26.917+0000 7f028f59e640 1 -- --> v1:172.21.15.43:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0290160080 con 0x7f029015c580 2023-08-09T15:11:26.918 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:26.917+0000 7f028f59e640 1 -- --> v1:172.21.15.203:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0290160360 con 0x7f0290069ed0 2023-08-09T15:11:26.918 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:26.917+0000 7f028f59e640 1 -- --> v2:172.21.15.203:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f02900cb810 con 0x7f029015c1a0 2023-08-09T15:11:26.919 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:26.917+0000 7f028dd9b640 1 --1- >> v1:172.21.15.43:6789/0 conn(0x7f029015c580 0x7f02900697c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.43:6789/0 says I am v1:172.21.15.43:42872/0 (socket says 172.21.15.43:42872) 2023-08-09T15:11:26.919 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:26.917+0000 7f028dd9b640 1 -- 172.21.15.43:0/2828886889 learned_addr learned my addr 172.21.15.43:0/2828886889 (peer_addr_for_me v1:172.21.15.43:0/0) 2023-08-09T15:11:26.919 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:26.918+0000 7f028e59c640 1 --2- 172.21.15.43:0/2828886889 >> v2:172.21.15.203:3300/0 conn(0x7f029015c1a0 0x7f029006db30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T15:11:26.919 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:26.918+0000 7f028d59a640 1 -- 172.21.15.43:0/2828886889 <== mon.0 v1:172.21.15.43:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3557453750 0 0) 0x7f0290160080 con 0x7f029015c580 2023-08-09T15:11:26.919 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:26.918+0000 7f028d59a640 1 -- 172.21.15.43:0/2828886889 --> v1:172.21.15.43:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f0278003520 con 0x7f029015c580 2023-08-09T15:11:26.920 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:26.918+0000 7f028d59a640 1 -- 172.21.15.43:0/2828886889 <== mon.0 v1:172.21.15.43:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 4078242712 0 0) 0x7f0278003520 con 0x7f029015c580 2023-08-09T15:11:26.920 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:26.918+0000 7f028d59a640 1 -- 172.21.15.43:0/2828886889 >> v2:172.21.15.203:3300/0 conn(0x7f029015c1a0 msgr2=0x7f029006db30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-09T15:11:26.920 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:26.918+0000 7f028d59a640 1 --2- 172.21.15.43:0/2828886889 >> v2:172.21.15.203:3300/0 conn(0x7f029015c1a0 0x7f029006db30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T15:11:26.920 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:26.918+0000 7f028d59a640 1 -- 172.21.15.43:0/2828886889 >> v1:172.21.15.203:6789/0 conn(0x7f0290069ed0 legacy=0x7f02900ca5c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T15:11:26.920 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:26.918+0000 7f028d59a640 1 -- 172.21.15.43:0/2828886889 --> v1:172.21.15.43:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0290160640 con 0x7f029015c580 2023-08-09T15:11:26.920 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:26.918+0000 7f028d59a640 1 -- 172.21.15.43:0/2828886889 <== mon.0 v1:172.21.15.43:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1285150740 0 0) 0x7f0284001670 con 0x7f029015c580 2023-08-09T15:11:26.921 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:26.918+0000 7f028d59a640 1 -- 172.21.15.43:0/2828886889 >> v1:172.21.15.43:6789/0 conn(0x7f029015c580 legacy=0x7f02900697c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T15:11:26.921 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:26.918+0000 7f028d59a640 1 --2- 172.21.15.43:0/2828886889 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f0278003970 0x7f0278003d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T15:11:26.921 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:26.918+0000 7f028d59a640 1 --2- 172.21.15.43:0/2828886889 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f02780047c0 0x7f0278004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T15:11:26.921 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:26.918+0000 7f028d59a640 1 --2- 172.21.15.43:0/2828886889 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f02780052a0 0x7f027800b980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T15:11:26.921 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:26.918+0000 7f028d59a640 1 -- 172.21.15.43:0/2828886889 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f02900cb810 con 0x7f02780052a0 2023-08-09T15:11:26.921 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:26.918+0000 7f028d59a640 1 -- 172.21.15.43:0/2828886889 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f027800c170 con 0x7f02780047c0 2023-08-09T15:11:26.921 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:26.918+0000 7f028d59a640 1 -- 172.21.15.43:0/2828886889 --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f027800c330 con 0x7f0278003970 2023-08-09T15:11:26.921 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:26.919+0000 7f028ed9d640 1 --2- 172.21.15.43:0/2828886889 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f02780052a0 0x7f027800b980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T15:11:26.923 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:26.919+0000 7f028e59c640 1 --2- 172.21.15.43:0/2828886889 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f0278003970 0x7f0278003d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T15:11:26.923 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:26.919+0000 7f028dd9b640 1 --2- 172.21.15.43:0/2828886889 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f02780047c0 0x7f0278004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T15:11:26.923 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:26.919+0000 7f028ed9d640 1 -- 172.21.15.43:0/2828886889 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f0278003970 msgr2=0x7f0278003d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-09T15:11:26.923 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:26.919+0000 7f028ed9d640 1 --2- 172.21.15.43:0/2828886889 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f0278003970 0x7f0278003d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T15:11:26.923 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:26.919+0000 7f028ed9d640 1 -- 172.21.15.43:0/2828886889 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f02780047c0 msgr2=0x7f0278004c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-09T15:11:26.923 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:26.919+0000 7f028ed9d640 1 --2- 172.21.15.43:0/2828886889 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f02780047c0 0x7f0278004c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T15:11:26.924 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:26.919+0000 7f028ed9d640 1 -- 172.21.15.43:0/2828886889 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0278003520 con 0x7f02780052a0 2023-08-09T15:11:26.924 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:26.921+0000 7f028ed9d640 1 -- 172.21.15.43:0/2828886889 reap_dead start 2023-08-09T15:11:26.924 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:26.921+0000 7f028ed9d640 1 --2- 172.21.15.43:0/2828886889 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f02780052a0 0x7f027800b980 secure :-1 s=READY pgs=2389 cs=0 l=1 rev1=1 crypto rx=0x7f0288001720 tx=0x7f02780057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ca02731fa5a1c54e server_cookie=0 in_seq=0 out_seq=0 2023-08-09T15:11:26.924 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:26.921+0000 7f028d59a640 1 -- 172.21.15.43:0/2828886889 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f02880052d0 con 0x7f02780052a0 2023-08-09T15:11:26.924 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:26.921+0000 7f028d59a640 1 -- 172.21.15.43:0/2828886889 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f028802e050 con 0x7f02780052a0 2023-08-09T15:11:26.924 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:26.921+0000 7f028d59a640 1 -- 172.21.15.43:0/2828886889 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f02880035f0 con 0x7f02780052a0 2023-08-09T15:11:26.924 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:26.922+0000 7f028f59e640 1 -- 172.21.15.43:0/2828886889 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f02780052a0 msgr2=0x7f027800b980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T15:11:26.925 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:26.922+0000 7f028f59e640 1 --2- 172.21.15.43:0/2828886889 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f02780052a0 0x7f027800b980 secure :-1 s=READY pgs=2389 cs=0 l=1 rev1=1 crypto rx=0x7f0288001720 tx=0x7f02780057f0 comp rx=0 tx=0).stop 2023-08-09T15:11:26.925 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:26.922+0000 7f028f59e640 1 -- 172.21.15.43:0/2828886889 shutdown_connections 2023-08-09T15:11:26.925 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:26.922+0000 7f028f59e640 1 --2- 172.21.15.43:0/2828886889 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f02780052a0 0x7f027800b980 unknown :-1 s=CLOSED pgs=2389 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T15:11:26.925 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:26.922+0000 7f028f59e640 1 -- 172.21.15.43:0/2828886889 >> 172.21.15.43:0/2828886889 conn(0x7f02900cb970 msgr2=0x7f02900cbd50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-09T15:11:26.925 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:26.922+0000 7f028f59e640 1 -- 172.21.15.43:0/2828886889 shutdown_connections 2023-08-09T15:11:26.925 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:26.922+0000 7f028f59e640 1 -- 172.21.15.43:0/2828886889 wait complete. 2023-08-09T15:11:26.926 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:26.924+0000 7f028f59e640 1 Processor -- start 2023-08-09T15:11:26.927 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:26.924+0000 7f028f59e640 1 -- start start 2023-08-09T15:11:26.927 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:26.925+0000 7f028f59e640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f02780052a0 0x7f029006e000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T15:11:26.927 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:26.925+0000 7f028f59e640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f02900b2490 0x7f0290158180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T15:11:26.927 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:26.925+0000 7f028f59e640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f029015ea60 0x7f029015ef10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T15:11:26.927 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:26.925+0000 7f028f59e640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f029015f450 con 0x7f02780052a0 2023-08-09T15:11:26.927 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:26.925+0000 7f028f59e640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f02900c3480 con 0x7f029015ea60 2023-08-09T15:11:26.927 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:26.925+0000 7f028f59e640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f02900c2b50 con 0x7f02900b2490 2023-08-09T15:11:26.928 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:26.926+0000 7f028e59c640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f02780052a0 0x7f029006e000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T15:11:26.928 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:26.926+0000 7f028e59c640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f02780052a0 0x7f029006e000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:56002/0 (socket says 172.21.15.43:56002) 2023-08-09T15:11:26.928 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:26.926+0000 7f028e59c640 1 -- 172.21.15.43:0/1513846176 learned_addr learned my addr 172.21.15.43:0/1513846176 (peer_addr_for_me v2:172.21.15.43:0/0) 2023-08-09T15:11:26.928 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:26.926+0000 7f028dd9b640 1 --2- 172.21.15.43:0/1513846176 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f02900b2490 0x7f0290158180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T15:11:26.928 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:26.926+0000 7f028e59c640 1 -- 172.21.15.43:0/1513846176 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f02900b2490 msgr2=0x7f0290158180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T15:11:26.929 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:26.926+0000 7f028e59c640 1 --2- 172.21.15.43:0/1513846176 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f02900b2490 0x7f0290158180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T15:11:26.929 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:26.926+0000 7f028e59c640 1 -- 172.21.15.43:0/1513846176 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f029015ea60 msgr2=0x7f029015ef10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T15:11:26.929 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:26.926+0000 7f028e59c640 1 --2- 172.21.15.43:0/1513846176 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f029015ea60 0x7f029015ef10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T15:11:26.929 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:26.926+0000 7f028e59c640 1 -- 172.21.15.43:0/1513846176 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f02900c2e30 con 0x7f02780052a0 2023-08-09T15:11:26.929 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:26.926+0000 7f028e59c640 1 --2- 172.21.15.43:0/1513846176 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f02780052a0 0x7f029006e000 secure :-1 s=READY pgs=2390 cs=0 l=1 rev1=1 crypto rx=0x7f0280009fb0 tx=0x7f02800115a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-09T15:11:26.930 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:26.927+0000 7f027effd640 1 -- 172.21.15.43:0/1513846176 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f028000e030 con 0x7f02780052a0 2023-08-09T15:11:26.930 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:26.927+0000 7f027effd640 1 -- 172.21.15.43:0/1513846176 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0280016470 con 0x7f02780052a0 2023-08-09T15:11:26.930 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:26.927+0000 7f027effd640 1 -- 172.21.15.43:0/1513846176 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0280016800 con 0x7f02780052a0 2023-08-09T15:11:26.930 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:26.927+0000 7f028f59e640 1 -- 172.21.15.43:0/1513846176 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f02900c3110 con 0x7f02780052a0 2023-08-09T15:11:26.931 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:26.927+0000 7f028f59e640 1 -- 172.21.15.43:0/1513846176 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0290158940 con 0x7f02780052a0 2023-08-09T15:11:26.931 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:26.928+0000 7f028f59e640 1 -- 172.21.15.43:0/1513846176 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f02900c32d0 con 0x7f02780052a0 2023-08-09T15:11:26.931 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:26.929+0000 7f027effd640 1 -- 172.21.15.43:0/1513846176 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 118) v1 ==== 88443+0+0 (secure 0 0 0) 0x7f0280016960 con 0x7f02780052a0 2023-08-09T15:11:26.931 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:26.930+0000 7f027effd640 1 --2- 172.21.15.43:0/1513846176 >> [v2:172.21.15.203:6832/4034093268,v1:172.21.15.203:6833/4034093268] conn(0x7f027006a1a0 0x7f027006c660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T15:11:26.932 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:26.931+0000 7f027effd640 1 -- 172.21.15.43:0/1513846176 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(220..220 src has 1..220) v4 ==== 5944+0+0 (secure 0 0 0) 0x7f028000f070 con 0x7f02780052a0 2023-08-09T15:11:26.932 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:26.931+0000 7f028dd9b640 1 --2- 172.21.15.43:0/1513846176 >> [v2:172.21.15.203:6832/4034093268,v1:172.21.15.203:6833/4034093268] conn(0x7f027006a1a0 0x7f027006c660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T15:11:26.933 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:26.933+0000 7f028dd9b640 1 --2- 172.21.15.43:0/1513846176 >> [v2:172.21.15.203:6832/4034093268,v1:172.21.15.203:6833/4034093268] conn(0x7f027006a1a0 0x7f027006c660 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f0284001200 tx=0x7f0284000f30 comp rx=0 tx=0).ready entity=mgr.8833 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-09T15:11:26.936 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:26.936+0000 7f027effd640 1 -- 172.21.15.43:0/1513846176 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178066 (secure 0 0 0) 0x7f0280054cd0 con 0x7f02780052a0 2023-08-09T15:11:27.131 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:27.131+0000 7f028f59e640 1 -- 172.21.15.43:0/1513846176 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9} v 0) v1 -- 0x7f029004e390 con 0x7f02780052a0 2023-08-09T15:11:27.676 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:27.674+0000 7f027effd640 1 -- 172.21.15.43:0/1513846176 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]=0 pool 'cephfs_data' created v221) v1 ==== 170+0+0 (secure 0 0 0) 0x7f02800582f0 con 0x7f02780052a0 2023-08-09T15:11:27.682 INFO:teuthology.orchestra.run.smithi043.stderr:pool 'cephfs_data' created 2023-08-09T15:11:27.691 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:27.691+0000 7f028f59e640 1 -- 172.21.15.43:0/1513846176 >> [v2:172.21.15.203:6832/4034093268,v1:172.21.15.203:6833/4034093268] conn(0x7f027006a1a0 msgr2=0x7f027006c660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T15:11:27.692 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:27.691+0000 7f028f59e640 1 --2- 172.21.15.43:0/1513846176 >> [v2:172.21.15.203:6832/4034093268,v1:172.21.15.203:6833/4034093268] conn(0x7f027006a1a0 0x7f027006c660 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f0284001200 tx=0x7f0284000f30 comp rx=0 tx=0).stop 2023-08-09T15:11:27.692 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:27.691+0000 7f028f59e640 1 -- 172.21.15.43:0/1513846176 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f02780052a0 msgr2=0x7f029006e000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T15:11:27.693 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:27.693+0000 7f028f59e640 1 --2- 172.21.15.43:0/1513846176 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f02780052a0 0x7f029006e000 secure :-1 s=READY pgs=2390 cs=0 l=1 rev1=1 crypto rx=0x7f0280009fb0 tx=0x7f02800115a0 comp rx=0 tx=0).stop 2023-08-09T15:11:27.696 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:27.696+0000 7f028f59e640 1 -- 172.21.15.43:0/1513846176 shutdown_connections 2023-08-09T15:11:27.696 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:27.696+0000 7f028f59e640 1 --2- 172.21.15.43:0/1513846176 >> [v2:172.21.15.203:6832/4034093268,v1:172.21.15.203:6833/4034093268] conn(0x7f027006a1a0 0x7f027006c660 unknown :-1 s=CLOSED pgs=56 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T15:11:27.696 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:27.696+0000 7f028f59e640 1 --2- 172.21.15.43:0/1513846176 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f029015ea60 0x7f029015ef10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T15:11:27.696 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:27.696+0000 7f028f59e640 1 --2- 172.21.15.43:0/1513846176 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f02900b2490 0x7f0290158180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T15:11:27.697 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:27.696+0000 7f028f59e640 1 --2- 172.21.15.43:0/1513846176 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f02780052a0 0x7f029006e000 unknown :-1 s=CLOSED pgs=2390 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T15:11:27.697 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:27.696+0000 7f028f59e640 1 -- 172.21.15.43:0/1513846176 >> 172.21.15.43:0/1513846176 conn(0x7f02900cb970 msgr2=0x7f02900ca7b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-09T15:11:27.697 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:27.696+0000 7f028f59e640 1 -- 172.21.15.43:0/1513846176 shutdown_connections 2023-08-09T15:11:27.697 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:27.697+0000 7f028f59e640 1 -- 172.21.15.43:0/1513846176 wait complete. 2023-08-09T15:11:27.722 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs new cephfs cephfs_metadata cephfs_data 2023-08-09T15:11:27.917 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:27.917+0000 7fddbe8c5640 1 Processor -- start 2023-08-09T15:11:27.918 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:27.917+0000 7fddbe8c5640 1 -- start start 2023-08-09T15:11:27.919 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:27.918+0000 7fddbe8c5640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7fddb80b0230 0x7fddb80cd060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T15:11:27.919 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:27.918+0000 7fddbe8c5640 1 -- --> v1:172.21.15.43:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fddb80cb240 con 0x7fddb80cee90 2023-08-09T15:11:27.919 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:27.918+0000 7fddbe8c5640 1 -- --> v1:172.21.15.203:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fddb80cf270 con 0x7fddb80ccbb0 2023-08-09T15:11:27.920 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:27.918+0000 7fddbe8c5640 1 -- --> v2:172.21.15.43:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fddb80cd5a0 con 0x7fddb80b0230 2023-08-09T15:11:27.920 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:27.918+0000 7fddb7fff640 1 --1- >> v1:172.21.15.43:6789/0 conn(0x7fddb80cee90 0x7fddb80cdd90 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.43:6789/0 says I am v1:172.21.15.43:42876/0 (socket says 172.21.15.43:42876) 2023-08-09T15:11:27.920 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:27.919+0000 7fddb7fff640 1 -- 172.21.15.43:0/2942306499 learned_addr learned my addr 172.21.15.43:0/2942306499 (peer_addr_for_me v1:172.21.15.43:0/0) 2023-08-09T15:11:27.920 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:27.919+0000 7fddbce62640 1 --2- 172.21.15.43:0/2942306499 >> v2:172.21.15.43:3300/0 conn(0x7fddb80b0230 0x7fddb80cd060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T15:11:27.921 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:27.919+0000 7fddbce62640 1 -- 172.21.15.43:0/2942306499 >> v1:172.21.15.203:6789/0 conn(0x7fddb80ccbb0 legacy=0x7fddb80afb20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-09T15:11:27.921 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:27.919+0000 7fddbce62640 1 -- 172.21.15.43:0/2942306499 >> v1:172.21.15.43:6789/0 conn(0x7fddb80cee90 legacy=0x7fddb80cdd90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T15:11:27.921 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:27.919+0000 7fddbce62640 1 -- 172.21.15.43:0/2942306499 --> v2:172.21.15.43:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fddb80b0770 con 0x7fddb80b0230 2023-08-09T15:11:27.921 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:27.920+0000 7fddbce62640 1 --2- 172.21.15.43:0/2942306499 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fddb80b0230 0x7fddb80cd060 secure :-1 s=READY pgs=2392 cs=0 l=1 rev1=1 crypto rx=0x7fddb00097a0 tx=0x7fddb002b840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=22f045ee0d3da7ef server_cookie=0 in_seq=0 out_seq=0 2023-08-09T15:11:27.921 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:27.921+0000 7fddb6ffd640 1 -- 172.21.15.43:0/2942306499 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fddb002e070 con 0x7fddb80b0230 2023-08-09T15:11:27.921 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:27.921+0000 7fddb6ffd640 1 -- 172.21.15.43:0/2942306499 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fddb00315a0 con 0x7fddb80b0230 2023-08-09T15:11:27.922 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:27.921+0000 7fddb6ffd640 1 -- 172.21.15.43:0/2942306499 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fddb0031930 con 0x7fddb80b0230 2023-08-09T15:11:27.922 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:27.921+0000 7fddbe8c5640 1 -- 172.21.15.43:0/2942306499 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fddb80b0230 msgr2=0x7fddb80cd060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T15:11:27.923 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:27.921+0000 7fddbe8c5640 1 --2- 172.21.15.43:0/2942306499 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fddb80b0230 0x7fddb80cd060 secure :-1 s=READY pgs=2392 cs=0 l=1 rev1=1 crypto rx=0x7fddb00097a0 tx=0x7fddb002b840 comp rx=0 tx=0).stop 2023-08-09T15:11:27.923 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:27.922+0000 7fddbe8c5640 1 -- 172.21.15.43:0/2942306499 shutdown_connections 2023-08-09T15:11:27.923 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:27.922+0000 7fddbe8c5640 1 --2- 172.21.15.43:0/2942306499 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fddb80b0230 0x7fddb80cd060 unknown :-1 s=CLOSED pgs=2392 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T15:11:27.923 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:27.922+0000 7fddbe8c5640 1 -- 172.21.15.43:0/2942306499 >> 172.21.15.43:0/2942306499 conn(0x7fddb8089fe0 msgr2=0x7fddb80d01c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-09T15:11:27.923 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:27.922+0000 7fddbe8c5640 1 -- 172.21.15.43:0/2942306499 shutdown_connections 2023-08-09T15:11:27.924 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:27.922+0000 7fddbe8c5640 1 -- 172.21.15.43:0/2942306499 wait complete. 2023-08-09T15:11:27.924 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:27.923+0000 7fddbe8c5640 1 Processor -- start 2023-08-09T15:11:27.924 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:27.924+0000 7fddbe8c5640 1 -- start start 2023-08-09T15:11:27.926 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:27.925+0000 7fddbe8c5640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fddb80b0230 0x7fddb815c640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T15:11:27.926 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:27.925+0000 7fddbe8c5640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fddb80ccbb0 0x7fddb80c8aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T15:11:27.926 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:27.925+0000 7fddbe8c5640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fddb80cee90 0x7fddb80c8fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T15:11:27.926 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:27.925+0000 7fddbe8c5640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fddb80c9520 con 0x7fddb80ccbb0 2023-08-09T15:11:27.927 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:27.925+0000 7fddbe8c5640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fddb80c7290 con 0x7fddb80b0230 2023-08-09T15:11:27.927 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:27.925+0000 7fddbe8c5640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fddb80c7550 con 0x7fddb80cee90 2023-08-09T15:11:27.927 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:27.925+0000 7fddb77fe640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fddb80ccbb0 0x7fddb80c8aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T15:11:27.928 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:27.925+0000 7fddb77fe640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fddb80ccbb0 0x7fddb80c8aa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:56020/0 (socket says 172.21.15.43:56020) 2023-08-09T15:11:27.928 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:27.925+0000 7fddb77fe640 1 -- 172.21.15.43:0/1898800654 learned_addr learned my addr 172.21.15.43:0/1898800654 (peer_addr_for_me v2:172.21.15.43:0/0) 2023-08-09T15:11:27.928 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:27.925+0000 7fddb7fff640 1 --2- 172.21.15.43:0/1898800654 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fddb80b0230 0x7fddb815c640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T15:11:27.928 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:27.925+0000 7fddbce62640 1 --2- 172.21.15.43:0/1898800654 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fddb80cee90 0x7fddb80c8fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T15:11:27.929 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:27.926+0000 7fddb77fe640 1 -- 172.21.15.43:0/1898800654 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fddb80cee90 msgr2=0x7fddb80c8fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T15:11:27.929 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:27.926+0000 7fddb77fe640 1 --2- 172.21.15.43:0/1898800654 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fddb80cee90 0x7fddb80c8fe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T15:11:27.929 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:27.926+0000 7fddb77fe640 1 -- 172.21.15.43:0/1898800654 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fddb80b0230 msgr2=0x7fddb815c640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T15:11:27.930 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:27.926+0000 7fddb77fe640 1 --2- 172.21.15.43:0/1898800654 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fddb80b0230 0x7fddb815c640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T15:11:27.930 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:27.926+0000 7fddb77fe640 1 -- 172.21.15.43:0/1898800654 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fddb0009400 con 0x7fddb80ccbb0 2023-08-09T15:11:27.931 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:27.926+0000 7fddb77fe640 1 --2- 172.21.15.43:0/1898800654 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fddb80ccbb0 0x7fddb80c8aa0 secure :-1 s=READY pgs=2393 cs=0 l=1 rev1=1 crypto rx=0x7fdda000c430 tx=0x7fdda000c8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-09T15:11:27.932 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:27.926+0000 7fddb4ff9640 1 -- 172.21.15.43:0/1898800654 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdda0011790 con 0x7fddb80ccbb0 2023-08-09T15:11:27.932 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:27.926+0000 7fddb4ff9640 1 -- 172.21.15.43:0/1898800654 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdda00118f0 con 0x7fddb80ccbb0 2023-08-09T15:11:27.932 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:27.926+0000 7fddbe8c5640 1 -- 172.21.15.43:0/1898800654 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fddb80c7830 con 0x7fddb80ccbb0 2023-08-09T15:11:27.933 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:27.926+0000 7fddb4ff9640 1 -- 172.21.15.43:0/1898800654 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdda0011ce0 con 0x7fddb80ccbb0 2023-08-09T15:11:27.933 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:27.926+0000 7fddbe8c5640 1 -- 172.21.15.43:0/1898800654 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fddb80c7df0 con 0x7fddb80ccbb0 2023-08-09T15:11:27.933 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:27.927+0000 7fddbe8c5640 1 -- 172.21.15.43:0/1898800654 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fddb815dfc0 con 0x7fddb80ccbb0 2023-08-09T15:11:27.933 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:27.929+0000 7fddb4ff9640 1 -- 172.21.15.43:0/1898800654 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 118) v1 ==== 88443+0+0 (secure 0 0 0) 0x7fdda001c460 con 0x7fddb80ccbb0 2023-08-09T15:11:27.934 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:27.930+0000 7fddb4ff9640 1 --2- 172.21.15.43:0/1898800654 >> [v2:172.21.15.203:6832/4034093268,v1:172.21.15.203:6833/4034093268] conn(0x7fdd9806a170 0x7fdd9806c630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T15:11:27.934 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:27.930+0000 7fddb4ff9640 1 -- 172.21.15.43:0/1898800654 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(221..221 src has 1..221) v4 ==== 6326+0+0 (secure 0 0 0) 0x7fdda00890b0 con 0x7fddb80ccbb0 2023-08-09T15:11:27.934 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:27.931+0000 7fddb7fff640 1 --2- 172.21.15.43:0/1898800654 >> [v2:172.21.15.203:6832/4034093268,v1:172.21.15.203:6833/4034093268] conn(0x7fdd9806a170 0x7fdd9806c630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T15:11:27.934 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:27.932+0000 7fddb7fff640 1 --2- 172.21.15.43:0/1898800654 >> [v2:172.21.15.203:6832/4034093268,v1:172.21.15.203:6833/4034093268] conn(0x7fdd9806a170 0x7fdd9806c630 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7fdda8007fd0 tx=0x7fdda8009040 comp rx=0 tx=0).ready entity=mgr.8833 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-09T15:11:27.934 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:27.934+0000 7fddb4ff9640 1 -- 172.21.15.43:0/1898800654 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178066 (secure 0 0 0) 0x7fdda00545f0 con 0x7fddb80ccbb0 2023-08-09T15:11:28.167 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:28.167+0000 7fddbe8c5640 1 -- 172.21.15.43:0/1898800654 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"} v 0) v1 -- 0x7fddb815dfc0 con 0x7fddb80ccbb0 2023-08-09T15:11:28.703 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:28.702+0000 7fddb4ff9640 1 -- 172.21.15.43:0/1898800654 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]=0 new fs with metadata pool 26 and data pool 27 v101) v1 ==== 174+0+0 (secure 0 0 0) 0x7fddb815dfc0 con 0x7fddb80ccbb0 2023-08-09T15:11:28.706 INFO:teuthology.orchestra.run.smithi043.stderr:new fs with metadata pool 26 and data pool 27 2023-08-09T15:11:28.727 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:28.727+0000 7fddbe8c5640 1 -- 172.21.15.43:0/1898800654 >> [v2:172.21.15.203:6832/4034093268,v1:172.21.15.203:6833/4034093268] conn(0x7fdd9806a170 msgr2=0x7fdd9806c630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T15:11:28.728 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:28.727+0000 7fddbe8c5640 1 --2- 172.21.15.43:0/1898800654 >> [v2:172.21.15.203:6832/4034093268,v1:172.21.15.203:6833/4034093268] conn(0x7fdd9806a170 0x7fdd9806c630 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7fdda8007fd0 tx=0x7fdda8009040 comp rx=0 tx=0).stop 2023-08-09T15:11:28.728 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:28.727+0000 7fddbe8c5640 1 -- 172.21.15.43:0/1898800654 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fddb80ccbb0 msgr2=0x7fddb80c8aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T15:11:28.728 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:28.727+0000 7fddbe8c5640 1 --2- 172.21.15.43:0/1898800654 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fddb80ccbb0 0x7fddb80c8aa0 secure :-1 s=READY pgs=2393 cs=0 l=1 rev1=1 crypto rx=0x7fdda000c430 tx=0x7fdda000c8f0 comp rx=0 tx=0).stop 2023-08-09T15:11:28.728 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:28.727+0000 7fddbe8c5640 1 -- 172.21.15.43:0/1898800654 shutdown_connections 2023-08-09T15:11:28.728 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:28.727+0000 7fddbe8c5640 1 --2- 172.21.15.43:0/1898800654 >> [v2:172.21.15.203:6832/4034093268,v1:172.21.15.203:6833/4034093268] conn(0x7fdd9806a170 0x7fdd9806c630 unknown :-1 s=CLOSED pgs=57 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T15:11:28.729 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:28.727+0000 7fddbe8c5640 1 --2- 172.21.15.43:0/1898800654 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fddb80cee90 0x7fddb80c8fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T15:11:28.729 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:28.727+0000 7fddbe8c5640 1 --2- 172.21.15.43:0/1898800654 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fddb80ccbb0 0x7fddb80c8aa0 secure :-1 s=CLOSED pgs=2393 cs=0 l=1 rev1=1 crypto rx=0x7fdda000c430 tx=0x7fdda000c8f0 comp rx=0 tx=0).stop 2023-08-09T15:11:28.729 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:28.727+0000 7fddbe8c5640 1 --2- 172.21.15.43:0/1898800654 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fddb80b0230 0x7fddb815c640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T15:11:28.729 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:28.727+0000 7fddbe8c5640 1 -- 172.21.15.43:0/1898800654 >> 172.21.15.43:0/1898800654 conn(0x7fddb8089fe0 msgr2=0x7fddb806eb40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-09T15:11:28.730 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:28.730+0000 7fddbe8c5640 1 -- 172.21.15.43:0/1898800654 shutdown_connections 2023-08-09T15:11:28.731 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:28.731+0000 7fddbe8c5640 1 -- 172.21.15.43:0/1898800654 wait complete. 2023-08-09T15:11:28.756 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-08-09T15:11:28.947 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:28.946+0000 7f9dacbc8640 1 Processor -- start 2023-08-09T15:11:28.947 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:28.947+0000 7f9dacbc8640 1 -- start start 2023-08-09T15:11:28.949 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:28.948+0000 7f9dacbc8640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f9da80b2790 0x7f9da80d2ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T15:11:28.949 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:28.948+0000 7f9dacbc8640 1 -- --> v1:172.21.15.43:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9da80ce090 con 0x7f9da80b3050 2023-08-09T15:11:28.949 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:28.948+0000 7f9dacbc8640 1 -- --> v1:172.21.15.203:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9da80cd7f0 con 0x7f9da80b2b70 2023-08-09T15:11:28.949 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:28.948+0000 7f9dacbc8640 1 -- --> v2:172.21.15.43:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9da80b3950 con 0x7f9da80b2790 2023-08-09T15:11:28.950 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:28.949+0000 7f9da659c640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f9da80b2790 0x7f9da80d2ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T15:11:28.950 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:28.949+0000 7f9da659c640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f9da80b2790 0x7f9da80d2ed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:56030/0 (socket says 172.21.15.43:56030) 2023-08-09T15:11:28.950 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:28.949+0000 7f9da659c640 1 -- 172.21.15.43:0/119143884 learned_addr learned my addr 172.21.15.43:0/119143884 (peer_addr_for_me v2:172.21.15.43:0/0) 2023-08-09T15:11:28.951 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:28.949+0000 7f9da659c640 1 -- 172.21.15.43:0/119143884 >> v1:172.21.15.203:6789/0 conn(0x7f9da80b2b70 legacy=0x7f9da80b5330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-09T15:11:28.951 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:28.949+0000 7f9da659c640 1 -- 172.21.15.43:0/119143884 >> v1:172.21.15.43:6789/0 conn(0x7f9da80b3050 legacy=0x7f9da80b3f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T15:11:28.951 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:28.949+0000 7f9da659c640 1 -- 172.21.15.43:0/119143884 --> v2:172.21.15.43:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9da80cdad0 con 0x7f9da80b2790 2023-08-09T15:11:28.951 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:28.951+0000 7f9da659c640 1 --2- 172.21.15.43:0/119143884 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f9da80b2790 0x7f9da80d2ed0 secure :-1 s=READY pgs=2395 cs=0 l=1 rev1=1 crypto rx=0x7f9d98009870 tx=0x7f9d9802b9a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f0b9a2da14026a13 server_cookie=0 in_seq=0 out_seq=0 2023-08-09T15:11:28.952 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:28.951+0000 7f9da5d9b640 1 -- 172.21.15.43:0/119143884 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9d98030860 con 0x7f9da80b2790 2023-08-09T15:11:28.952 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:28.951+0000 7f9da5d9b640 1 -- 172.21.15.43:0/119143884 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9d980309c0 con 0x7f9da80b2790 2023-08-09T15:11:28.952 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:28.951+0000 7f9da5d9b640 1 -- 172.21.15.43:0/119143884 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9d98030d50 con 0x7f9da80b2790 2023-08-09T15:11:28.953 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:28.951+0000 7f9dacbc8640 1 -- 172.21.15.43:0/119143884 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f9da80b2790 msgr2=0x7f9da80d2ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T15:11:28.953 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:28.951+0000 7f9dacbc8640 1 --2- 172.21.15.43:0/119143884 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f9da80b2790 0x7f9da80d2ed0 secure :-1 s=READY pgs=2395 cs=0 l=1 rev1=1 crypto rx=0x7f9d98009870 tx=0x7f9d9802b9a0 comp rx=0 tx=0).stop 2023-08-09T15:11:28.953 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:28.952+0000 7f9dacbc8640 1 -- 172.21.15.43:0/119143884 shutdown_connections 2023-08-09T15:11:28.953 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:28.952+0000 7f9dacbc8640 1 --2- 172.21.15.43:0/119143884 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f9da80b2790 0x7f9da80d2ed0 unknown :-1 s=CLOSED pgs=2395 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T15:11:28.954 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:28.952+0000 7f9dacbc8640 1 -- 172.21.15.43:0/119143884 >> 172.21.15.43:0/119143884 conn(0x7f9da809b7b0 msgr2=0x7f9da80d24e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-09T15:11:28.954 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:28.952+0000 7f9dacbc8640 1 -- 172.21.15.43:0/119143884 shutdown_connections 2023-08-09T15:11:28.954 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:28.952+0000 7f9dacbc8640 1 -- 172.21.15.43:0/119143884 wait complete. 2023-08-09T15:11:28.954 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:28.954+0000 7f9dacbc8640 1 Processor -- start 2023-08-09T15:11:28.954 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:28.954+0000 7f9dacbc8640 1 -- start start 2023-08-09T15:11:28.955 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:28.955+0000 7f9dacbc8640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f9da80b2790 0x7f9da8151970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T15:11:28.955 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:28.955+0000 7f9dacbc8640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f9da80b2b70 0x7f9da80b02e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T15:11:28.955 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:28.955+0000 7f9dacbc8640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f9da80b3050 0x7f9da80c6010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T15:11:28.955 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:28.955+0000 7f9dacbc8640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9da80afec0 con 0x7f9da80b2b70 2023-08-09T15:11:28.956 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:28.955+0000 7f9dacbc8640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9da80cbff0 con 0x7f9da80b2790 2023-08-09T15:11:28.956 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:28.955+0000 7f9dacbc8640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9da80af240 con 0x7f9da80b3050 2023-08-09T15:11:28.957 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:28.955+0000 7f9d9ffff640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f9da80b2b70 0x7f9da80b02e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T15:11:28.957 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:28.955+0000 7f9d9ffff640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f9da80b2b70 0x7f9da80b02e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:56032/0 (socket says 172.21.15.43:56032) 2023-08-09T15:11:28.957 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:28.955+0000 7f9d9ffff640 1 -- 172.21.15.43:0/218575928 learned_addr learned my addr 172.21.15.43:0/218575928 (peer_addr_for_me v2:172.21.15.43:0/0) 2023-08-09T15:11:28.958 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:28.956+0000 7f9da659c640 1 --2- 172.21.15.43:0/218575928 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f9da80b2790 0x7f9da8151970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T15:11:28.958 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:28.956+0000 7f9da6d9d640 1 --2- 172.21.15.43:0/218575928 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f9da80b3050 0x7f9da80c6010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T15:11:28.958 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:28.956+0000 7f9d9ffff640 1 -- 172.21.15.43:0/218575928 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f9da80b3050 msgr2=0x7f9da80c6010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T15:11:28.958 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:28.956+0000 7f9d9ffff640 1 --2- 172.21.15.43:0/218575928 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f9da80b3050 0x7f9da80c6010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T15:11:28.958 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:28.956+0000 7f9d9ffff640 1 -- 172.21.15.43:0/218575928 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f9da80b2790 msgr2=0x7f9da8151970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T15:11:28.958 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:28.956+0000 7f9d9ffff640 1 --2- 172.21.15.43:0/218575928 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f9da80b2790 0x7f9da8151970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T15:11:28.959 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:28.956+0000 7f9d9ffff640 1 -- 172.21.15.43:0/218575928 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9d980094d0 con 0x7f9da80b2b70 2023-08-09T15:11:28.959 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:28.956+0000 7f9da6d9d640 1 --2- 172.21.15.43:0/218575928 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f9da80b3050 0x7f9da80c6010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-08-09T15:11:28.959 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:28.956+0000 7f9da659c640 1 --2- 172.21.15.43:0/218575928 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f9da80b2790 0x7f9da8151970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-08-09T15:11:28.959 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:28.956+0000 7f9d9ffff640 1 --2- 172.21.15.43:0/218575928 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f9da80b2b70 0x7f9da80b02e0 secure :-1 s=READY pgs=2396 cs=0 l=1 rev1=1 crypto rx=0x7f9d9000b3f0 tx=0x7f9d9000b8b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-09T15:11:28.959 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:28.956+0000 7f9d9effd640 1 -- 172.21.15.43:0/218575928 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9d90011830 con 0x7f9da80b2b70 2023-08-09T15:11:28.960 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:28.956+0000 7f9d9effd640 1 -- 172.21.15.43:0/218575928 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9d90011990 con 0x7f9da80b2b70 2023-08-09T15:11:28.960 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:28.956+0000 7f9d9effd640 1 -- 172.21.15.43:0/218575928 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9d90011d40 con 0x7f9da80b2b70 2023-08-09T15:11:28.960 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:28.956+0000 7f9dacbc8640 1 -- 172.21.15.43:0/218575928 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9da80af520 con 0x7f9da80b2b70 2023-08-09T15:11:28.960 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:28.957+0000 7f9dacbc8640 1 -- 172.21.15.43:0/218575928 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9da80afae0 con 0x7f9da80b2b70 2023-08-09T15:11:28.961 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:28.958+0000 7f9dacbc8640 1 -- 172.21.15.43:0/218575928 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9da80b0000 con 0x7f9da80b2b70 2023-08-09T15:11:28.961 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:28.959+0000 7f9d9effd640 1 -- 172.21.15.43:0/218575928 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 118) v1 ==== 88443+0+0 (secure 0 0 0) 0x7f9d9001a400 con 0x7f9da80b2b70 2023-08-09T15:11:28.961 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:28.960+0000 7f9d9effd640 1 --2- 172.21.15.43:0/218575928 >> [v2:172.21.15.203:6832/4034093268,v1:172.21.15.203:6833/4034093268] conn(0x7f9d8806a170 0x7f9d8806c630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T15:11:28.961 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:28.960+0000 7f9d9effd640 1 -- 172.21.15.43:0/218575928 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(222..222 src has 1..222) v4 ==== 6426+0+0 (secure 0 0 0) 0x7f9d90088730 con 0x7f9da80b2b70 2023-08-09T15:11:28.961 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:28.960+0000 7f9da659c640 1 --2- 172.21.15.43:0/218575928 >> [v2:172.21.15.203:6832/4034093268,v1:172.21.15.203:6833/4034093268] conn(0x7f9d8806a170 0x7f9d8806c630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T15:11:28.962 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:28.962+0000 7f9da659c640 1 --2- 172.21.15.43:0/218575928 >> [v2:172.21.15.203:6832/4034093268,v1:172.21.15.203:6833/4034093268] conn(0x7f9d8806a170 0x7f9d8806c630 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f9d98036000 tx=0x7f9d9803b040 comp rx=0 tx=0).ready entity=mgr.8833 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-09T15:11:28.964 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:28.964+0000 7f9d9effd640 1 -- 172.21.15.43:0/218575928 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178066 (secure 0 0 0) 0x7f9da80b0000 con 0x7f9da80b2b70 2023-08-09T15:11:29.162 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:29.162+0000 7f9dacbc8640 1 -- 172.21.15.43:0/218575928 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f9da80b0000 con 0x7f9da80b2b70 2023-08-09T15:11:29.164 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:29.164+0000 7f9d9effd640 1 -- 172.21.15.43:0/218575928 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v222) v1 ==== 74+0+18583 (secure 0 0 0) 0x7f9da80b0000 con 0x7f9da80b2b70 2023-08-09T15:11:29.164 INFO:teuthology.orchestra.run.smithi043.stdout: 2023-08-09T15:11:29.165 INFO:teuthology.orchestra.run.smithi043.stdout:{"epoch":222,"fsid":"daac87ee-1337-4250-a955-aea4e408a39b","created":"2023-08-09T13:33:41.701198+0000","modified":"2023-08-09T15:11:28.168297+0000","last_up_change":"2023-08-09T13:33:53.783453+0000","last_in_change":"2023-08-09T13:33:45.148904+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":27,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":"rbd","create_time":"2023-08-09T13:33:54.365137+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":".mgr","create_time":"2023-08-09T13:33:54.831803+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":26,"pool_name":"cephfs_metadata","create_time":"2023-08-09T15:11:26.544757+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"222","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":27,"pool_name":"cephfs_data","create_time":"2023-08-09T15:11:27.194686+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"222","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.6299999952316284,"score_stable":1.6299999952316284,"optimal_score":1,"raw_score_acting":1.6299999952316284,"raw_score_stable":1.6299999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"cbd49ecd-e896-4fbd-864e-5c37a829b165","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":221,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6800","nonce":502667335},{"type":"v1","addr":"172.21.15.43:6801","nonce":502667335}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6802","nonce":502667335},{"type":"v1","addr":"172.21.15.43:6803","nonce":502667335}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6806","nonce":502667335},{"type":"v1","addr":"172.21.15.43:6807","nonce":502667335}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6804","nonce":502667335},{"type":"v1","addr":"172.21.15.43:6805","nonce":502667335}]},"public_addr":"172.21.15.43:6801/502667335","cluster_addr":"172.21.15.43:6803/502667335","heartbeat_back_addr":"172.21.15.43:6807/502667335","heartbeat_front_addr":"172.21.15.43:6805/502667335","state":["exists","up"]},{"osd":1,"uuid":"75c7e0fe-1ec7-4fe8-a2f2-3661a65b8500","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":221,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6808","nonce":3289134610},{"type":"v1","addr":"172.21.15.43:6809","nonce":3289134610}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6810","nonce":3289134610},{"type":"v1","addr":"172.21.15.43:6812","nonce":3289134610}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6821","nonce":3289134610},{"type":"v1","addr":"172.21.15.43:6824","nonce":3289134610}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6815","nonce":3289134610},{"type":"v1","addr":"172.21.15.43:6818","nonce":3289134610}]},"public_addr":"172.21.15.43:6809/3289134610","cluster_addr":"172.21.15.43:6812/3289134610","heartbeat_back_addr":"172.21.15.43:6824/3289134610","heartbeat_front_addr":"172.21.15.43:6818/3289134610","state":["exists","up"]},{"osd":2,"uuid":"bb5c41e9-7f5f-4125-9b01-3bb8bfc2ea83","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":221,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6811","nonce":4079709736},{"type":"v1","addr":"172.21.15.43:6814","nonce":4079709736}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6817","nonce":4079709736},{"type":"v1","addr":"172.21.15.43:6820","nonce":4079709736}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6829","nonce":4079709736},{"type":"v1","addr":"172.21.15.43:6831","nonce":4079709736}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6823","nonce":4079709736},{"type":"v1","addr":"172.21.15.43:6826","nonce":4079709736}]},"public_addr":"172.21.15.43:6814/4079709736","cluster_addr":"172.21.15.43:6820/4079709736","heartbeat_back_addr":"172.21.15.43:6831/4079709736","heartbeat_front_addr":"172.21.15.43:6826/4079709736","state":["exists","up"]},{"osd":3,"uuid":"bfe4c8b4-694a-465c-87e6-2b2aab852613","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":221,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6813","nonce":1709538759},{"type":"v1","addr":"172.21.15.43:6816","nonce":1709538759}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6819","nonce":1709538759},{"type":"v1","addr":"172.21.15.43:6822","nonce":1709538759}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6828","nonce":1709538759},{"type":"v1","addr":"172.21.15.43:6830","nonce":1709538759}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6825","nonce":1709538759},{"type":"v1","addr":"172.21.15.43:6827","nonce":1709538759}]},"public_addr":"172.21.15.43:6816/1709538759","cluster_addr":"172.21.15.43:6822/1709538759","heartbeat_back_addr":"172.21.15.43:6830/1709538759","heartbeat_front_addr":"172.21.15.43:6827/1709538759","state":["exists","up"]},{"osd":4,"uuid":"fa2e8a77-aba1-4756-853e-0d174a5bdeba","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":221,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6800","nonce":4196866481},{"type":"v1","addr":"172.21.15.203:6801","nonce":4196866481}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6802","nonce":4196866481},{"type":"v1","addr":"172.21.15.203:6803","nonce":4196866481}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6806","nonce":4196866481},{"type":"v1","addr":"172.21.15.203:6807","nonce":4196866481}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6804","nonce":4196866481},{"type":"v1","addr":"172.21.15.203:6805","nonce":4196866481}]},"public_addr":"172.21.15.203:6801/4196866481","cluster_addr":"172.21.15.203:6803/4196866481","heartbeat_back_addr":"172.21.15.203:6807/4196866481","heartbeat_front_addr":"172.21.15.203:6805/4196866481","state":["exists","up"]},{"osd":5,"uuid":"81811e4b-4d24-4f51-a82c-f1d700a2a00d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":221,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6816","nonce":3059290081},{"type":"v1","addr":"172.21.15.203:6817","nonce":3059290081}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6818","nonce":3059290081},{"type":"v1","addr":"172.21.15.203:6819","nonce":3059290081}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6822","nonce":3059290081},{"type":"v1","addr":"172.21.15.203:6823","nonce":3059290081}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6820","nonce":3059290081},{"type":"v1","addr":"172.21.15.203:6821","nonce":3059290081}]},"public_addr":"172.21.15.203:6817/3059290081","cluster_addr":"172.21.15.203:6819/3059290081","heartbeat_back_addr":"172.21.15.203:6823/3059290081","heartbeat_front_addr":"172.21.15.203:6821/3059290081","state":["exists","up"]},{"osd":6,"uuid":"307bac4d-6a5b-443d-af75-56d695d7674f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":221,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6824","nonce":512155569},{"type":"v1","addr":"172.21.15.203:6825","nonce":512155569}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6826","nonce":512155569},{"type":"v1","addr":"172.21.15.203:6827","nonce":512155569}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6830","nonce":512155569},{"type":"v1","addr":"172.21.15.203:6831","nonce":512155569}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6828","nonce":512155569},{"type":"v1","addr":"172.21.15.203:6829","nonce":512155569}]},"public_addr":"172.21.15.203:6825/512155569","cluster_addr":"172.21.15.203:6827/512155569","heartbeat_back_addr":"172.21.15.203:6831/512155569","heartbeat_front_addr":"172.21.15.203:6829/512155569","state":["exists","up"]},{"osd":7,"uuid":"a95dc7ca-9f1b-4295-af70-68a22a2d7d5e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":221,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6808","nonce":3331450438},{"type":"v1","addr":"172.21.15.203:6809","nonce":3331450438}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6810","nonce":3331450438},{"type":"v1","addr":"172.21.15.203:6811","nonce":3331450438}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6814","nonce":3331450438},{"type":"v1","addr":"172.21.15.203:6815","nonce":3331450438}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6812","nonce":3331450438},{"type":"v1","addr":"172.21.15.203:6813","nonce":3331450438}]},"public_addr":"172.21.15.203:6809/3331450438","cluster_addr":"172.21.15.203:6811/3331450438","heartbeat_back_addr":"172.21.15.203:6815/3331450438","heartbeat_front_addr":"172.21.15.203:6813/3331450438","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-09T13:33:50.547652+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-09T13:33:50.589469+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-09T13:33:50.561724+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-09T13:33:50.599650+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-09T13:33:50.411855+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-09T13:33:50.219681+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-09T13:33:50.504664+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-09T13:33:50.441248+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":7,"snaps":[{"begin":104,"length":1},{"begin":221,"length":1}]}],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-08-09T15:11:29.167 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:29.166+0000 7f9dacbc8640 1 -- 172.21.15.43:0/218575928 >> [v2:172.21.15.203:6832/4034093268,v1:172.21.15.203:6833/4034093268] conn(0x7f9d8806a170 msgr2=0x7f9d8806c630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T15:11:29.167 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:29.166+0000 7f9dacbc8640 1 --2- 172.21.15.43:0/218575928 >> [v2:172.21.15.203:6832/4034093268,v1:172.21.15.203:6833/4034093268] conn(0x7f9d8806a170 0x7f9d8806c630 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f9d98036000 tx=0x7f9d9803b040 comp rx=0 tx=0).stop 2023-08-09T15:11:29.168 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:29.166+0000 7f9dacbc8640 1 -- 172.21.15.43:0/218575928 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f9da80b2b70 msgr2=0x7f9da80b02e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T15:11:29.168 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:29.166+0000 7f9dacbc8640 1 --2- 172.21.15.43:0/218575928 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f9da80b2b70 0x7f9da80b02e0 secure :-1 s=READY pgs=2396 cs=0 l=1 rev1=1 crypto rx=0x7f9d9000b3f0 tx=0x7f9d9000b8b0 comp rx=0 tx=0).stop 2023-08-09T15:11:29.168 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:29.167+0000 7f9dacbc8640 1 -- 172.21.15.43:0/218575928 shutdown_connections 2023-08-09T15:11:29.168 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:29.167+0000 7f9dacbc8640 1 --2- 172.21.15.43:0/218575928 >> [v2:172.21.15.203:6832/4034093268,v1:172.21.15.203:6833/4034093268] conn(0x7f9d8806a170 0x7f9d8806c630 unknown :-1 s=CLOSED pgs=58 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T15:11:29.168 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:29.167+0000 7f9dacbc8640 1 --2- 172.21.15.43:0/218575928 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f9da80b3050 0x7f9da80c6010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T15:11:29.169 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:29.167+0000 7f9dacbc8640 1 --2- 172.21.15.43:0/218575928 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f9da80b2b70 0x7f9da80b02e0 unknown :-1 s=CLOSED pgs=2396 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T15:11:29.169 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:29.167+0000 7f9dacbc8640 1 --2- 172.21.15.43:0/218575928 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f9da80b2790 0x7f9da8151970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T15:11:29.169 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:29.167+0000 7f9dacbc8640 1 -- 172.21.15.43:0/218575928 >> 172.21.15.43:0/218575928 conn(0x7f9da809b7b0 msgr2=0x7f9da80cf660 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-09T15:11:29.169 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:29.167+0000 7f9dacbc8640 1 -- 172.21.15.43:0/218575928 shutdown_connections 2023-08-09T15:11:29.169 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:29.167+0000 7f9dacbc8640 1 -- 172.21.15.43:0/218575928 wait complete. 2023-08-09T15:11:29.185 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-08-09T15:11:29.392 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:29.392+0000 7f7c380f3640 1 Processor -- start 2023-08-09T15:11:29.393 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:29.392+0000 7f7c380f3640 1 -- start start 2023-08-09T15:11:29.394 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:29.394+0000 7f7c380f3640 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7f7c300ccf20 0x7f7c300cd550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T15:11:29.394 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:29.394+0000 7f7c380f3640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f7c300cda90 0x7f7c3015ca10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T15:11:29.395 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:29.394+0000 7f7c380f3640 1 -- --> v1:172.21.15.203:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7c300cca90 con 0x7f7c3015c630 2023-08-09T15:11:29.395 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:29.394+0000 7f7c380f3640 1 -- --> v2:172.21.15.43:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7c300ac600 con 0x7f7c300cda90 2023-08-09T15:11:29.395 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:29.394+0000 7f7c380f3640 1 -- --> v2:172.21.15.203:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7c300cbc70 con 0x7f7c300ccf20 2023-08-09T15:11:29.396 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:29.394+0000 7f7c36690640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f7c300cda90 0x7f7c3015ca10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T15:11:29.396 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:29.394+0000 7f7c36690640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f7c300cda90 0x7f7c3015ca10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:56044/0 (socket says 172.21.15.43:56044) 2023-08-09T15:11:29.396 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:29.394+0000 7f7c36690640 1 -- 172.21.15.43:0/2543421793 learned_addr learned my addr 172.21.15.43:0/2543421793 (peer_addr_for_me v2:172.21.15.43:0/0) 2023-08-09T15:11:29.396 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:29.395+0000 7f7c3568e640 1 --2- 172.21.15.43:0/2543421793 >> v2:172.21.15.203:3300/0 conn(0x7f7c300ccf20 0x7f7c300cd550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T15:11:29.396 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:29.395+0000 7f7c36690640 1 -- 172.21.15.43:0/2543421793 >> v2:172.21.15.203:3300/0 conn(0x7f7c300ccf20 msgr2=0x7f7c300cd550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-09T15:11:29.396 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:29.395+0000 7f7c36690640 1 --2- 172.21.15.43:0/2543421793 >> v2:172.21.15.203:3300/0 conn(0x7f7c300ccf20 0x7f7c300cd550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T15:11:29.396 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:29.395+0000 7f7c36690640 1 -- 172.21.15.43:0/2543421793 >> v1:172.21.15.203:6789/0 conn(0x7f7c3015c630 legacy=0x7f7c300abef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-09T15:11:29.397 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:29.395+0000 7f7c36690640 1 -- 172.21.15.43:0/2543421793 --> v2:172.21.15.43:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7c300cdea0 con 0x7f7c300cda90 2023-08-09T15:11:29.398 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:29.396+0000 7f7c36690640 1 --2- 172.21.15.43:0/2543421793 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f7c300cda90 0x7f7c3015ca10 secure :-1 s=READY pgs=2397 cs=0 l=1 rev1=1 crypto rx=0x7f7c3006a660 tx=0x7f7c2802bee0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c5d0037e1fedd7d1 server_cookie=0 in_seq=0 out_seq=0 2023-08-09T15:11:29.398 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:29.396+0000 7f7c34e8d640 1 -- 172.21.15.43:0/2543421793 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7c2802ca80 con 0x7f7c300cda90 2023-08-09T15:11:29.398 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:29.396+0000 7f7c34e8d640 1 -- 172.21.15.43:0/2543421793 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7c2802cbe0 con 0x7f7c300cda90 2023-08-09T15:11:29.398 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:29.396+0000 7f7c34e8d640 1 -- 172.21.15.43:0/2543421793 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7c280343f0 con 0x7f7c300cda90 2023-08-09T15:11:29.398 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:29.397+0000 7f7c380f3640 1 -- 172.21.15.43:0/2543421793 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f7c300cda90 msgr2=0x7f7c3015ca10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T15:11:29.398 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:29.397+0000 7f7c380f3640 1 --2- 172.21.15.43:0/2543421793 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f7c300cda90 0x7f7c3015ca10 secure :-1 s=READY pgs=2397 cs=0 l=1 rev1=1 crypto rx=0x7f7c3006a660 tx=0x7f7c2802bee0 comp rx=0 tx=0).stop 2023-08-09T15:11:29.398 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:29.397+0000 7f7c380f3640 1 -- 172.21.15.43:0/2543421793 shutdown_connections 2023-08-09T15:11:29.399 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:29.397+0000 7f7c380f3640 1 --2- 172.21.15.43:0/2543421793 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f7c300cda90 0x7f7c3015ca10 unknown :-1 s=CLOSED pgs=2397 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T15:11:29.399 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:29.397+0000 7f7c380f3640 1 --2- 172.21.15.43:0/2543421793 >> v2:172.21.15.203:3300/0 conn(0x7f7c300ccf20 0x7f7c300cd550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T15:11:29.399 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:29.397+0000 7f7c380f3640 1 -- 172.21.15.43:0/2543421793 >> 172.21.15.43:0/2543421793 conn(0x7f7c300ce0a0 msgr2=0x7f7c300ce480 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-09T15:11:29.399 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:29.397+0000 7f7c380f3640 1 -- 172.21.15.43:0/2543421793 shutdown_connections 2023-08-09T15:11:29.399 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:29.397+0000 7f7c380f3640 1 -- 172.21.15.43:0/2543421793 wait complete. 2023-08-09T15:11:29.400 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:29.399+0000 7f7c380f3640 1 Processor -- start 2023-08-09T15:11:29.400 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:29.399+0000 7f7c380f3640 1 -- start start 2023-08-09T15:11:29.400 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:29.400+0000 7f7c380f3640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f7c300ccf20 0x7f7c3015f860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T15:11:29.400 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:29.400+0000 7f7c380f3640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f7c300cda90 0x7f7c300caa80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T15:11:29.401 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:29.400+0000 7f7c380f3640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f7c3015c630 0x7f7c300b3460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T15:11:29.401 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:29.400+0000 7f7c380f3640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7c300cafc0 con 0x7f7c300ccf20 2023-08-09T15:11:29.401 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:29.400+0000 7f7c380f3640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7c300cb260 con 0x7f7c3015c630 2023-08-09T15:11:29.401 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:29.400+0000 7f7c380f3640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7c300cb540 con 0x7f7c300cda90 2023-08-09T15:11:29.402 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:29.401+0000 7f7c35e8f640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f7c300ccf20 0x7f7c3015f860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T15:11:29.403 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:29.401+0000 7f7c35e8f640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f7c300ccf20 0x7f7c3015f860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:56052/0 (socket says 172.21.15.43:56052) 2023-08-09T15:11:29.403 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:29.401+0000 7f7c35e8f640 1 -- 172.21.15.43:0/2289165091 learned_addr learned my addr 172.21.15.43:0/2289165091 (peer_addr_for_me v2:172.21.15.43:0/0) 2023-08-09T15:11:29.403 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:29.401+0000 7f7c3568e640 1 --2- 172.21.15.43:0/2289165091 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f7c300cda90 0x7f7c300caa80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T15:11:29.403 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:29.401+0000 7f7c36690640 1 --2- 172.21.15.43:0/2289165091 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f7c3015c630 0x7f7c300b3460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T15:11:29.403 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:29.401+0000 7f7c35e8f640 1 -- 172.21.15.43:0/2289165091 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f7c300cda90 msgr2=0x7f7c300caa80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T15:11:29.403 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:29.401+0000 7f7c35e8f640 1 --2- 172.21.15.43:0/2289165091 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f7c300cda90 0x7f7c300caa80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T15:11:29.403 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:29.401+0000 7f7c35e8f640 1 -- 172.21.15.43:0/2289165091 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f7c3015c630 msgr2=0x7f7c300b3460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T15:11:29.404 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:29.401+0000 7f7c35e8f640 1 --2- 172.21.15.43:0/2289165091 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f7c3015c630 0x7f7c300b3460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T15:11:29.404 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:29.401+0000 7f7c35e8f640 1 -- 172.21.15.43:0/2289165091 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7c28009b30 con 0x7f7c300ccf20 2023-08-09T15:11:29.404 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:29.401+0000 7f7c36690640 1 --2- 172.21.15.43:0/2289165091 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f7c3015c630 0x7f7c300b3460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-08-09T15:11:29.404 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:29.401+0000 7f7c35e8f640 1 --2- 172.21.15.43:0/2289165091 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f7c300ccf20 0x7f7c3015f860 secure :-1 s=READY pgs=2398 cs=0 l=1 rev1=1 crypto rx=0x7f7c2c00b3e0 tx=0x7f7c2c00b8a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-09T15:11:29.404 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:29.402+0000 7f7c267fc640 1 -- 172.21.15.43:0/2289165091 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7c2c011830 con 0x7f7c300ccf20 2023-08-09T15:11:29.404 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:29.402+0000 7f7c267fc640 1 -- 172.21.15.43:0/2289165091 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7c2c0119b0 con 0x7f7c300ccf20 2023-08-09T15:11:29.404 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:29.402+0000 7f7c267fc640 1 -- 172.21.15.43:0/2289165091 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7c2c011d60 con 0x7f7c300ccf20 2023-08-09T15:11:29.405 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:29.402+0000 7f7c380f3640 1 -- 172.21.15.43:0/2289165091 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7c300cb820 con 0x7f7c300ccf20 2023-08-09T15:11:29.405 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:29.402+0000 7f7c380f3640 1 -- 172.21.15.43:0/2289165091 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7c300c6540 con 0x7f7c300ccf20 2023-08-09T15:11:29.405 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:29.403+0000 7f7c380f3640 1 -- 172.21.15.43:0/2289165091 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7c300739f0 con 0x7f7c300ccf20 2023-08-09T15:11:29.405 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:29.404+0000 7f7c267fc640 1 -- 172.21.15.43:0/2289165091 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 118) v1 ==== 88443+0+0 (secure 0 0 0) 0x7f7c2c01a400 con 0x7f7c300ccf20 2023-08-09T15:11:29.406 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:29.405+0000 7f7c267fc640 1 --2- 172.21.15.43:0/2289165091 >> [v2:172.21.15.203:6832/4034093268,v1:172.21.15.203:6833/4034093268] conn(0x7f7c1406a1c0 0x7f7c1406c680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T15:11:29.406 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:29.406+0000 7f7c267fc640 1 -- 172.21.15.43:0/2289165091 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(222..222 src has 1..222) v4 ==== 6426+0+0 (secure 0 0 0) 0x7f7c2c088a60 con 0x7f7c300ccf20 2023-08-09T15:11:29.406 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:29.406+0000 7f7c3568e640 1 --2- 172.21.15.43:0/2289165091 >> [v2:172.21.15.203:6832/4034093268,v1:172.21.15.203:6833/4034093268] conn(0x7f7c1406a1c0 0x7f7c1406c680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T15:11:29.407 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:29.407+0000 7f7c3568e640 1 --2- 172.21.15.43:0/2289165091 >> [v2:172.21.15.203:6832/4034093268,v1:172.21.15.203:6833/4034093268] conn(0x7f7c1406a1c0 0x7f7c1406c680 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f7c300c8130 tx=0x7f7c20008040 comp rx=0 tx=0).ready entity=mgr.8833 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-09T15:11:29.410 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:29.410+0000 7f7c267fc640 1 -- 172.21.15.43:0/2289165091 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178066 (secure 0 0 0) 0x7f7c2c053eb0 con 0x7f7c300ccf20 2023-08-09T15:11:29.605 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:29.604+0000 7f7c380f3640 1 -- 172.21.15.43:0/2289165091 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f7c300cec90 con 0x7f7c300ccf20 2023-08-09T15:11:29.607 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:29.607+0000 7f7c267fc640 1 -- 172.21.15.43:0/2289165091 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v222) v1 ==== 74+0+18583 (secure 0 0 0) 0x7f7c2c0574d0 con 0x7f7c300ccf20 2023-08-09T15:11:29.608 INFO:teuthology.orchestra.run.smithi043.stdout: 2023-08-09T15:11:29.608 INFO:teuthology.orchestra.run.smithi043.stdout:{"epoch":222,"fsid":"daac87ee-1337-4250-a955-aea4e408a39b","created":"2023-08-09T13:33:41.701198+0000","modified":"2023-08-09T15:11:28.168297+0000","last_up_change":"2023-08-09T13:33:53.783453+0000","last_in_change":"2023-08-09T13:33:45.148904+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":27,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":"rbd","create_time":"2023-08-09T13:33:54.365137+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":".mgr","create_time":"2023-08-09T13:33:54.831803+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":26,"pool_name":"cephfs_metadata","create_time":"2023-08-09T15:11:26.544757+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"222","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":27,"pool_name":"cephfs_data","create_time":"2023-08-09T15:11:27.194686+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"222","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.6299999952316284,"score_stable":1.6299999952316284,"optimal_score":1,"raw_score_acting":1.6299999952316284,"raw_score_stable":1.6299999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"cbd49ecd-e896-4fbd-864e-5c37a829b165","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":221,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6800","nonce":502667335},{"type":"v1","addr":"172.21.15.43:6801","nonce":502667335}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6802","nonce":502667335},{"type":"v1","addr":"172.21.15.43:6803","nonce":502667335}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6806","nonce":502667335},{"type":"v1","addr":"172.21.15.43:6807","nonce":502667335}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6804","nonce":502667335},{"type":"v1","addr":"172.21.15.43:6805","nonce":502667335}]},"public_addr":"172.21.15.43:6801/502667335","cluster_addr":"172.21.15.43:6803/502667335","heartbeat_back_addr":"172.21.15.43:6807/502667335","heartbeat_front_addr":"172.21.15.43:6805/502667335","state":["exists","up"]},{"osd":1,"uuid":"75c7e0fe-1ec7-4fe8-a2f2-3661a65b8500","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":221,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6808","nonce":3289134610},{"type":"v1","addr":"172.21.15.43:6809","nonce":3289134610}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6810","nonce":3289134610},{"type":"v1","addr":"172.21.15.43:6812","nonce":3289134610}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6821","nonce":3289134610},{"type":"v1","addr":"172.21.15.43:6824","nonce":3289134610}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6815","nonce":3289134610},{"type":"v1","addr":"172.21.15.43:6818","nonce":3289134610}]},"public_addr":"172.21.15.43:6809/3289134610","cluster_addr":"172.21.15.43:6812/3289134610","heartbeat_back_addr":"172.21.15.43:6824/3289134610","heartbeat_front_addr":"172.21.15.43:6818/3289134610","state":["exists","up"]},{"osd":2,"uuid":"bb5c41e9-7f5f-4125-9b01-3bb8bfc2ea83","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":221,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6811","nonce":4079709736},{"type":"v1","addr":"172.21.15.43:6814","nonce":4079709736}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6817","nonce":4079709736},{"type":"v1","addr":"172.21.15.43:6820","nonce":4079709736}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6829","nonce":4079709736},{"type":"v1","addr":"172.21.15.43:6831","nonce":4079709736}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6823","nonce":4079709736},{"type":"v1","addr":"172.21.15.43:6826","nonce":4079709736}]},"public_addr":"172.21.15.43:6814/4079709736","cluster_addr":"172.21.15.43:6820/4079709736","heartbeat_back_addr":"172.21.15.43:6831/4079709736","heartbeat_front_addr":"172.21.15.43:6826/4079709736","state":["exists","up"]},{"osd":3,"uuid":"bfe4c8b4-694a-465c-87e6-2b2aab852613","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":221,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6813","nonce":1709538759},{"type":"v1","addr":"172.21.15.43:6816","nonce":1709538759}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6819","nonce":1709538759},{"type":"v1","addr":"172.21.15.43:6822","nonce":1709538759}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6828","nonce":1709538759},{"type":"v1","addr":"172.21.15.43:6830","nonce":1709538759}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6825","nonce":1709538759},{"type":"v1","addr":"172.21.15.43:6827","nonce":1709538759}]},"public_addr":"172.21.15.43:6816/1709538759","cluster_addr":"172.21.15.43:6822/1709538759","heartbeat_back_addr":"172.21.15.43:6830/1709538759","heartbeat_front_addr":"172.21.15.43:6827/1709538759","state":["exists","up"]},{"osd":4,"uuid":"fa2e8a77-aba1-4756-853e-0d174a5bdeba","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":221,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6800","nonce":4196866481},{"type":"v1","addr":"172.21.15.203:6801","nonce":4196866481}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6802","nonce":4196866481},{"type":"v1","addr":"172.21.15.203:6803","nonce":4196866481}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6806","nonce":4196866481},{"type":"v1","addr":"172.21.15.203:6807","nonce":4196866481}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6804","nonce":4196866481},{"type":"v1","addr":"172.21.15.203:6805","nonce":4196866481}]},"public_addr":"172.21.15.203:6801/4196866481","cluster_addr":"172.21.15.203:6803/4196866481","heartbeat_back_addr":"172.21.15.203:6807/4196866481","heartbeat_front_addr":"172.21.15.203:6805/4196866481","state":["exists","up"]},{"osd":5,"uuid":"81811e4b-4d24-4f51-a82c-f1d700a2a00d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":221,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6816","nonce":3059290081},{"type":"v1","addr":"172.21.15.203:6817","nonce":3059290081}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6818","nonce":3059290081},{"type":"v1","addr":"172.21.15.203:6819","nonce":3059290081}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6822","nonce":3059290081},{"type":"v1","addr":"172.21.15.203:6823","nonce":3059290081}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6820","nonce":3059290081},{"type":"v1","addr":"172.21.15.203:6821","nonce":3059290081}]},"public_addr":"172.21.15.203:6817/3059290081","cluster_addr":"172.21.15.203:6819/3059290081","heartbeat_back_addr":"172.21.15.203:6823/3059290081","heartbeat_front_addr":"172.21.15.203:6821/3059290081","state":["exists","up"]},{"osd":6,"uuid":"307bac4d-6a5b-443d-af75-56d695d7674f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":221,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6824","nonce":512155569},{"type":"v1","addr":"172.21.15.203:6825","nonce":512155569}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6826","nonce":512155569},{"type":"v1","addr":"172.21.15.203:6827","nonce":512155569}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6830","nonce":512155569},{"type":"v1","addr":"172.21.15.203:6831","nonce":512155569}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6828","nonce":512155569},{"type":"v1","addr":"172.21.15.203:6829","nonce":512155569}]},"public_addr":"172.21.15.203:6825/512155569","cluster_addr":"172.21.15.203:6827/512155569","heartbeat_back_addr":"172.21.15.203:6831/512155569","heartbeat_front_addr":"172.21.15.203:6829/512155569","state":["exists","up"]},{"osd":7,"uuid":"a95dc7ca-9f1b-4295-af70-68a22a2d7d5e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":221,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6808","nonce":3331450438},{"type":"v1","addr":"172.21.15.203:6809","nonce":3331450438}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6810","nonce":3331450438},{"type":"v1","addr":"172.21.15.203:6811","nonce":3331450438}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6814","nonce":3331450438},{"type":"v1","addr":"172.21.15.203:6815","nonce":3331450438}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6812","nonce":3331450438},{"type":"v1","addr":"172.21.15.203:6813","nonce":3331450438}]},"public_addr":"172.21.15.203:6809/3331450438","cluster_addr":"172.21.15.203:6811/3331450438","heartbeat_back_addr":"172.21.15.203:6815/3331450438","heartbeat_front_addr":"172.21.15.203:6813/3331450438","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-09T13:33:50.547652+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-09T13:33:50.589469+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-09T13:33:50.561724+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-09T13:33:50.599650+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-09T13:33:50.411855+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-09T13:33:50.219681+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-09T13:33:50.504664+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-09T13:33:50.441248+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":7,"snaps":[{"begin":104,"length":1},{"begin":221,"length":1}]}],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-08-09T15:11:29.610 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:29.610+0000 7f7c380f3640 1 -- 172.21.15.43:0/2289165091 >> [v2:172.21.15.203:6832/4034093268,v1:172.21.15.203:6833/4034093268] conn(0x7f7c1406a1c0 msgr2=0x7f7c1406c680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T15:11:29.610 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:29.610+0000 7f7c380f3640 1 --2- 172.21.15.43:0/2289165091 >> [v2:172.21.15.203:6832/4034093268,v1:172.21.15.203:6833/4034093268] conn(0x7f7c1406a1c0 0x7f7c1406c680 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f7c300c8130 tx=0x7f7c20008040 comp rx=0 tx=0).stop 2023-08-09T15:11:29.610 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:29.610+0000 7f7c380f3640 1 -- 172.21.15.43:0/2289165091 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f7c300ccf20 msgr2=0x7f7c3015f860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T15:11:29.611 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:29.610+0000 7f7c380f3640 1 --2- 172.21.15.43:0/2289165091 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f7c300ccf20 0x7f7c3015f860 secure :-1 s=READY pgs=2398 cs=0 l=1 rev1=1 crypto rx=0x7f7c2c00b3e0 tx=0x7f7c2c00b8a0 comp rx=0 tx=0).stop 2023-08-09T15:11:29.611 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:29.610+0000 7f7c380f3640 1 -- 172.21.15.43:0/2289165091 shutdown_connections 2023-08-09T15:11:29.611 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:29.610+0000 7f7c380f3640 1 --2- 172.21.15.43:0/2289165091 >> [v2:172.21.15.203:6832/4034093268,v1:172.21.15.203:6833/4034093268] conn(0x7f7c1406a1c0 0x7f7c1406c680 unknown :-1 s=CLOSED pgs=59 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T15:11:29.612 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:29.610+0000 7f7c380f3640 1 --2- 172.21.15.43:0/2289165091 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f7c3015c630 0x7f7c300b3460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T15:11:29.612 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:29.610+0000 7f7c380f3640 1 --2- 172.21.15.43:0/2289165091 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f7c300cda90 0x7f7c300caa80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T15:11:29.612 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:29.610+0000 7f7c380f3640 1 --2- 172.21.15.43:0/2289165091 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f7c300ccf20 0x7f7c3015f860 unknown :-1 s=CLOSED pgs=2398 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T15:11:29.612 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:29.610+0000 7f7c380f3640 1 -- 172.21.15.43:0/2289165091 >> 172.21.15.43:0/2289165091 conn(0x7f7c300ce0a0 msgr2=0x7f7c3015bbc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-09T15:11:29.612 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:29.610+0000 7f7c380f3640 1 -- 172.21.15.43:0/2289165091 shutdown_connections 2023-08-09T15:11:29.612 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:29.611+0000 7f7c380f3640 1 -- 172.21.15.43:0/2289165091 wait complete. 2023-08-09T15:11:29.630 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs standby_count_wanted 0 2023-08-09T15:11:29.860 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:29.859+0000 7ff0908f2640 1 Processor -- start 2023-08-09T15:11:29.860 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:29.859+0000 7ff0908f2640 1 -- start start 2023-08-09T15:11:29.861 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:29.861+0000 7ff0908f2640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7ff08c1590b0 0x7ff08c0cafa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T15:11:29.862 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:29.861+0000 7ff0908f2640 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7ff08c0c96a0 0x7ff08c0cb9f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T15:11:29.862 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:29.861+0000 7ff0908f2640 1 -- --> v1:172.21.15.203:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff08c0b07e0 con 0x7ff08c06fd40 2023-08-09T15:11:29.862 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:29.861+0000 7ff0908f2640 1 -- --> v2:172.21.15.203:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff08c0701a0 con 0x7ff08c0c96a0 2023-08-09T15:11:29.863 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:29.861+0000 7ff0908f2640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff08c070420 con 0x7ff08c1590b0 2023-08-09T15:11:29.864 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:29.862+0000 7ff08b59e640 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7ff08c0c96a0 0x7ff08c0cb9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T15:11:29.864 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:29.862+0000 7ff08b59e640 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7ff08c0c96a0 0x7ff08c0cb9f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.203:3300/0 says I am v2:172.21.15.43:57726/0 (socket says 172.21.15.43:57726) 2023-08-09T15:11:29.865 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:29.862+0000 7ff08b59e640 1 -- 172.21.15.43:0/1989337590 learned_addr learned my addr 172.21.15.43:0/1989337590 (peer_addr_for_me v2:172.21.15.43:0/0) 2023-08-09T15:11:29.865 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:29.862+0000 7ff08ad9d640 1 --2- 172.21.15.43:0/1989337590 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7ff08c1590b0 0x7ff08c0cafa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T15:11:29.865 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:29.862+0000 7ff08ad9d640 1 -- 172.21.15.43:0/1989337590 >> v2:172.21.15.203:3300/0 conn(0x7ff08c0c96a0 msgr2=0x7ff08c0cb9f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-09T15:11:29.865 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:29.862+0000 7ff08ad9d640 1 --2- 172.21.15.43:0/1989337590 >> v2:172.21.15.203:3300/0 conn(0x7ff08c0c96a0 0x7ff08c0cb9f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T15:11:29.866 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:29.862+0000 7ff08ad9d640 1 -- 172.21.15.43:0/1989337590 >> v1:172.21.15.203:6789/0 conn(0x7ff08c06fd40 legacy=0x7ff08c0c8f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T15:11:29.866 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:29.862+0000 7ff08ad9d640 1 -- 172.21.15.43:0/1989337590 --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff08c070700 con 0x7ff08c1590b0 2023-08-09T15:11:29.866 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:29.864+0000 7ff08ad9d640 1 --2- 172.21.15.43:0/1989337590 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7ff08c1590b0 0x7ff08c0cafa0 secure :-1 s=READY pgs=337 cs=0 l=1 rev1=1 crypto rx=0x7ff07c0097a0 tx=0x7ff07c02b8b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=74e41d4e04f122b1 server_cookie=0 in_seq=0 out_seq=0 2023-08-09T15:11:29.870 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:29.864+0000 7ff089d9b640 1 -- 172.21.15.43:0/1989337590 <== mon.2 v2:172.21.15.203:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff07c02d040 con 0x7ff08c1590b0 2023-08-09T15:11:29.870 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:29.864+0000 7ff089d9b640 1 -- 172.21.15.43:0/1989337590 <== mon.2 v2:172.21.15.203:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff07c031810 con 0x7ff08c1590b0 2023-08-09T15:11:29.871 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:29.864+0000 7ff089d9b640 1 -- 172.21.15.43:0/1989337590 <== mon.2 v2:172.21.15.203:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff07c031ba0 con 0x7ff08c1590b0 2023-08-09T15:11:29.871 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:29.865+0000 7ff0908f2640 1 -- 172.21.15.43:0/1989337590 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7ff08c1590b0 msgr2=0x7ff08c0cafa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T15:11:29.871 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:29.865+0000 7ff0908f2640 1 --2- 172.21.15.43:0/1989337590 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7ff08c1590b0 0x7ff08c0cafa0 secure :-1 s=READY pgs=337 cs=0 l=1 rev1=1 crypto rx=0x7ff07c0097a0 tx=0x7ff07c02b8b0 comp rx=0 tx=0).stop 2023-08-09T15:11:29.871 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:29.865+0000 7ff0908f2640 1 -- 172.21.15.43:0/1989337590 shutdown_connections 2023-08-09T15:11:29.872 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:29.865+0000 7ff0908f2640 1 --2- 172.21.15.43:0/1989337590 >> v2:172.21.15.203:3300/0 conn(0x7ff08c0c96a0 0x7ff08c0cb9f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T15:11:29.872 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:29.865+0000 7ff0908f2640 1 --2- 172.21.15.43:0/1989337590 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7ff08c1590b0 0x7ff08c0cafa0 unknown :-1 s=CLOSED pgs=337 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T15:11:29.872 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:29.865+0000 7ff0908f2640 1 -- 172.21.15.43:0/1989337590 >> 172.21.15.43:0/1989337590 conn(0x7ff08c0af9f0 msgr2=0x7ff08c0afdd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-09T15:11:29.873 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:29.865+0000 7ff0908f2640 1 -- 172.21.15.43:0/1989337590 shutdown_connections 2023-08-09T15:11:29.873 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:29.865+0000 7ff0908f2640 1 -- 172.21.15.43:0/1989337590 wait complete. 2023-08-09T15:11:29.873 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:29.867+0000 7ff0908f2640 1 Processor -- start 2023-08-09T15:11:29.874 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:29.867+0000 7ff0908f2640 1 -- start start 2023-08-09T15:11:29.874 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:29.868+0000 7ff0908f2640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7ff08c06fd40 0x7ff08c158000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T15:11:29.874 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:29.868+0000 7ff0908f2640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff08c0c96a0 0x7ff08c0aecc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T15:11:29.874 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:29.868+0000 7ff0908f2640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7ff08c1590b0 0x7ff08c0adc40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T15:11:29.874 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:29.868+0000 7ff0908f2640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff08c0af5c0 con 0x7ff08c0c96a0 2023-08-09T15:11:29.874 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:29.868+0000 7ff0908f2640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff08c157520 con 0x7ff08c1590b0 2023-08-09T15:11:29.875 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:29.868+0000 7ff0908f2640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff08c157800 con 0x7ff08c06fd40 2023-08-09T15:11:29.875 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:29.868+0000 7ff08a59c640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff08c0c96a0 0x7ff08c0aecc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T15:11:29.875 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:29.868+0000 7ff08a59c640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff08c0c96a0 0x7ff08c0aecc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:56068/0 (socket says 172.21.15.43:56068) 2023-08-09T15:11:29.875 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:29.868+0000 7ff08a59c640 1 -- 172.21.15.43:0/290834290 learned_addr learned my addr 172.21.15.43:0/290834290 (peer_addr_for_me v2:172.21.15.43:0/0) 2023-08-09T15:11:29.875 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:29.868+0000 7ff08b59e640 1 --2- 172.21.15.43:0/290834290 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7ff08c1590b0 0x7ff08c0adc40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T15:11:29.875 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:29.868+0000 7ff08a59c640 1 -- 172.21.15.43:0/290834290 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7ff08c06fd40 msgr2=0x7ff08c158000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T15:11:29.875 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:29.868+0000 7ff08a59c640 1 --2- 172.21.15.43:0/290834290 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7ff08c06fd40 0x7ff08c158000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T15:11:29.876 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:29.869+0000 7ff08a59c640 1 -- 172.21.15.43:0/290834290 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7ff08c1590b0 msgr2=0x7ff08c0adc40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T15:11:29.876 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:29.869+0000 7ff08a59c640 1 --2- 172.21.15.43:0/290834290 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7ff08c1590b0 0x7ff08c0adc40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T15:11:29.876 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:29.869+0000 7ff08a59c640 1 -- 172.21.15.43:0/290834290 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff07c009400 con 0x7ff08c0c96a0 2023-08-09T15:11:29.876 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:29.869+0000 7ff08a59c640 1 --2- 172.21.15.43:0/290834290 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff08c0c96a0 0x7ff08c0aecc0 secure :-1 s=READY pgs=2399 cs=0 l=1 rev1=1 crypto rx=0x7ff08000c400 tx=0x7ff08000c8c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-09T15:11:29.876 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:29.869+0000 7ff0737fe640 1 -- 172.21.15.43:0/290834290 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff0800052d0 con 0x7ff08c0c96a0 2023-08-09T15:11:29.876 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:29.869+0000 7ff0737fe640 1 -- 172.21.15.43:0/290834290 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff080005430 con 0x7ff08c0c96a0 2023-08-09T15:11:29.876 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:29.869+0000 7ff0908f2640 1 -- 172.21.15.43:0/290834290 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff08c157ae0 con 0x7ff08c0c96a0 2023-08-09T15:11:29.877 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:29.869+0000 7ff0737fe640 1 -- 172.21.15.43:0/290834290 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff080003830 con 0x7ff08c0c96a0 2023-08-09T15:11:29.877 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:29.869+0000 7ff0908f2640 1 -- 172.21.15.43:0/290834290 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff08c0af200 con 0x7ff08c0c96a0 2023-08-09T15:11:29.881 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:29.871+0000 7ff0908f2640 1 -- 172.21.15.43:0/290834290 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff08c157380 con 0x7ff08c0c96a0 2023-08-09T15:11:29.881 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:29.872+0000 7ff0737fe640 1 -- 172.21.15.43:0/290834290 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 118) v1 ==== 88443+0+0 (secure 0 0 0) 0x7ff080003990 con 0x7ff08c0c96a0 2023-08-09T15:11:29.881 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:29.873+0000 7ff0737fe640 1 --2- 172.21.15.43:0/290834290 >> [v2:172.21.15.203:6832/4034093268,v1:172.21.15.203:6833/4034093268] conn(0x7ff07806a170 0x7ff07806c630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T15:11:29.881 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:29.873+0000 7ff0737fe640 1 -- 172.21.15.43:0/290834290 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(223..223 src has 1..223) v4 ==== 6426+0+0 (secure 0 0 0) 0x7ff080087ea0 con 0x7ff08c0c96a0 2023-08-09T15:11:29.881 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:29.873+0000 7ff08ad9d640 1 --2- 172.21.15.43:0/290834290 >> [v2:172.21.15.203:6832/4034093268,v1:172.21.15.203:6833/4034093268] conn(0x7ff07806a170 0x7ff07806c630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T15:11:29.882 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:29.875+0000 7ff08ad9d640 1 --2- 172.21.15.43:0/290834290 >> [v2:172.21.15.203:6832/4034093268,v1:172.21.15.203:6833/4034093268] conn(0x7ff07806a170 0x7ff07806c630 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7ff07c036000 tx=0x7ff07c030000 comp rx=0 tx=0).ready entity=mgr.8833 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-09T15:11:29.882 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:29.877+0000 7ff0737fe640 1 -- 172.21.15.43:0/290834290 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178066 (secure 0 0 0) 0x7ff08c157380 con 0x7ff08c0c96a0 2023-08-09T15:11:30.114 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:30.114+0000 7ff0908f2640 1 -- 172.21.15.43:0/290834290 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} v 0) v1 -- 0x7ff08c157380 con 0x7ff08c0c96a0 2023-08-09T15:11:30.743 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:30.742+0000 7ff0737fe640 1 -- 172.21.15.43:0/290834290 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]=0 v103) v1 ==== 118+0+0 (secure 0 0 0) 0x7ff08c157380 con 0x7ff08c0c96a0 2023-08-09T15:11:30.746 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:30.745+0000 7ff0908f2640 1 -- 172.21.15.43:0/290834290 >> [v2:172.21.15.203:6832/4034093268,v1:172.21.15.203:6833/4034093268] conn(0x7ff07806a170 msgr2=0x7ff07806c630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T15:11:30.746 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:30.745+0000 7ff0908f2640 1 --2- 172.21.15.43:0/290834290 >> [v2:172.21.15.203:6832/4034093268,v1:172.21.15.203:6833/4034093268] conn(0x7ff07806a170 0x7ff07806c630 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7ff07c036000 tx=0x7ff07c030000 comp rx=0 tx=0).stop 2023-08-09T15:11:30.747 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:30.746+0000 7ff0908f2640 1 -- 172.21.15.43:0/290834290 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff08c0c96a0 msgr2=0x7ff08c0aecc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T15:11:30.747 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:30.746+0000 7ff0908f2640 1 --2- 172.21.15.43:0/290834290 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff08c0c96a0 0x7ff08c0aecc0 secure :-1 s=READY pgs=2399 cs=0 l=1 rev1=1 crypto rx=0x7ff08000c400 tx=0x7ff08000c8c0 comp rx=0 tx=0).stop 2023-08-09T15:11:30.747 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:30.746+0000 7ff0908f2640 1 -- 172.21.15.43:0/290834290 shutdown_connections 2023-08-09T15:11:30.747 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:30.746+0000 7ff0908f2640 1 --2- 172.21.15.43:0/290834290 >> [v2:172.21.15.203:6832/4034093268,v1:172.21.15.203:6833/4034093268] conn(0x7ff07806a170 0x7ff07806c630 unknown :-1 s=CLOSED pgs=60 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T15:11:30.747 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:30.746+0000 7ff0908f2640 1 --2- 172.21.15.43:0/290834290 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7ff08c1590b0 0x7ff08c0adc40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T15:11:30.747 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:30.746+0000 7ff0908f2640 1 --2- 172.21.15.43:0/290834290 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff08c0c96a0 0x7ff08c0aecc0 unknown :-1 s=CLOSED pgs=2399 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T15:11:30.747 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:30.746+0000 7ff0908f2640 1 --2- 172.21.15.43:0/290834290 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7ff08c06fd40 0x7ff08c158000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T15:11:30.748 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:30.746+0000 7ff0908f2640 1 -- 172.21.15.43:0/290834290 >> 172.21.15.43:0/290834290 conn(0x7ff08c0af9f0 msgr2=0x7ff08c0cb960 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-09T15:11:30.748 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:30.746+0000 7ff0908f2640 1 -- 172.21.15.43:0/290834290 shutdown_connections 2023-08-09T15:11:30.748 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:30.746+0000 7ff0908f2640 1 -- 172.21.15.43:0/290834290 wait complete. 2023-08-09T15:11:30.767 DEBUG:tasks.cephfs.filesystem:fs_config: {} 2023-08-09T15:11:30.768 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs allow_standby_replay false 2023-08-09T15:11:31.001 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:31.000+0000 7ffa0ad86640 1 Processor -- start 2023-08-09T15:11:31.001 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:31.000+0000 7ffa0ad86640 1 -- start start 2023-08-09T15:11:31.002 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:31.002+0000 7ffa0ad86640 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7ffa04157ce0 0x7ffa0415b0f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T15:11:31.003 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:31.002+0000 7ffa0ad86640 1 -- --> v1:172.21.15.43:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ffa0406b480 con 0x7ffa040c8470 2023-08-09T15:11:31.003 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:31.002+0000 7ffa0ad86640 1 -- --> v1:172.21.15.203:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ffa04071580 con 0x7ffa0406f2c0 2023-08-09T15:11:31.003 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:31.002+0000 7ffa0ad86640 1 -- --> v2:172.21.15.203:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ffa041580c0 con 0x7ffa04157ce0 2023-08-09T15:11:31.007 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:31.002+0000 7ffa0a585640 1 --1- >> v1:172.21.15.43:6789/0 conn(0x7ffa040c8470 0x7ffa040c8880 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.43:6789/0 says I am v1:172.21.15.43:42898/0 (socket says 172.21.15.43:42898) 2023-08-09T15:11:31.007 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:31.002+0000 7ffa0a585640 1 -- 172.21.15.43:0/2546648074 learned_addr learned my addr 172.21.15.43:0/2546648074 (peer_addr_for_me v1:172.21.15.43:0/0) 2023-08-09T15:11:31.008 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:31.003+0000 7ffa09d84640 1 --2- 172.21.15.43:0/2546648074 >> v2:172.21.15.203:3300/0 conn(0x7ffa04157ce0 0x7ffa0415b0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T15:11:31.008 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:31.003+0000 7ffa08d82640 1 -- 172.21.15.43:0/2546648074 <== mon.0 v1:172.21.15.43:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3002245480 0 0) 0x7ffa0406b480 con 0x7ffa040c8470 2023-08-09T15:11:31.008 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:31.003+0000 7ffa08d82640 1 -- 172.21.15.43:0/2546648074 --> v1:172.21.15.43:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ff9f0003520 con 0x7ffa040c8470 2023-08-09T15:11:31.008 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:31.003+0000 7ffa08d82640 1 -- 172.21.15.43:0/2546648074 <== mon.0 v1:172.21.15.43:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3844381290 0 0) 0x7ff9f0003520 con 0x7ffa040c8470 2023-08-09T15:11:31.009 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:31.003+0000 7ffa08d82640 1 -- 172.21.15.43:0/2546648074 >> v2:172.21.15.203:3300/0 conn(0x7ffa04157ce0 msgr2=0x7ffa0415b0f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-09T15:11:31.009 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:31.003+0000 7ffa08d82640 1 --2- 172.21.15.43:0/2546648074 >> v2:172.21.15.203:3300/0 conn(0x7ffa04157ce0 0x7ffa0415b0f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T15:11:31.009 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:31.003+0000 7ffa08d82640 1 -- 172.21.15.43:0/2546648074 >> v1:172.21.15.203:6789/0 conn(0x7ffa0406f2c0 legacy=0x7ffa0406bc40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T15:11:31.009 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:31.003+0000 7ffa08d82640 1 -- 172.21.15.43:0/2546648074 --> v1:172.21.15.43:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ffa0406c350 con 0x7ffa040c8470 2023-08-09T15:11:31.010 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:31.003+0000 7ffa08d82640 1 -- 172.21.15.43:0/2546648074 <== mon.0 v1:172.21.15.43:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1285150740 0 0) 0x7ff9fc0019f0 con 0x7ffa040c8470 2023-08-09T15:11:31.010 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:31.003+0000 7ffa08d82640 1 -- 172.21.15.43:0/2546648074 >> v1:172.21.15.43:6789/0 conn(0x7ffa040c8470 legacy=0x7ffa040c8880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T15:11:31.010 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:31.004+0000 7ffa08d82640 1 --2- 172.21.15.43:0/2546648074 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff9f0003970 0x7ff9f0003db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T15:11:31.011 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:31.004+0000 7ffa08d82640 1 --2- 172.21.15.43:0/2546648074 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7ff9f00047c0 0x7ff9f0004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T15:11:31.011 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:31.004+0000 7ffa08d82640 1 --2- 172.21.15.43:0/2546648074 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7ff9f00051d0 0x7ff9f000b980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T15:11:31.011 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:31.004+0000 7ffa08d82640 1 -- 172.21.15.43:0/2546648074 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffa041580c0 con 0x7ff9f0003970 2023-08-09T15:11:31.011 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:31.004+0000 7ffa08d82640 1 -- 172.21.15.43:0/2546648074 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff9f000c100 con 0x7ff9f00047c0 2023-08-09T15:11:31.012 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:31.004+0000 7ffa08d82640 1 -- 172.21.15.43:0/2546648074 --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff9f000c2c0 con 0x7ff9f00051d0 2023-08-09T15:11:31.012 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:31.004+0000 7ffa09d84640 1 --2- 172.21.15.43:0/2546648074 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff9f0003970 0x7ff9f0003db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T15:11:31.012 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:31.004+0000 7ffa0a585640 1 --2- 172.21.15.43:0/2546648074 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7ff9f00051d0 0x7ff9f000b980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T15:11:31.013 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:31.004+0000 7ffa09d84640 1 -- 172.21.15.43:0/2546648074 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7ff9f00051d0 msgr2=0x7ff9f000b980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-09T15:11:31.013 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:31.004+0000 7ffa09d84640 1 --2- 172.21.15.43:0/2546648074 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7ff9f00051d0 0x7ff9f000b980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T15:11:31.013 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:31.004+0000 7ffa09d84640 1 -- 172.21.15.43:0/2546648074 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7ff9f00047c0 msgr2=0x7ff9f0004c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-09T15:11:31.013 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:31.004+0000 7ffa09d84640 1 --2- 172.21.15.43:0/2546648074 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7ff9f00047c0 0x7ff9f0004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T15:11:31.014 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:31.004+0000 7ffa09d84640 1 -- 172.21.15.43:0/2546648074 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff9f0003520 con 0x7ff9f0003970 2023-08-09T15:11:31.014 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:31.004+0000 7ffa0a585640 1 -- 172.21.15.43:0/2546648074 reap_dead start 2023-08-09T15:11:31.014 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:31.005+0000 7ffa09d84640 1 --2- 172.21.15.43:0/2546648074 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff9f0003970 0x7ff9f0003db0 secure :-1 s=READY pgs=2401 cs=0 l=1 rev1=1 crypto rx=0x7ffa00009ac0 tx=0x7ff9f00057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8b7bce992c845dd8 server_cookie=0 in_seq=0 out_seq=0 2023-08-09T15:11:31.015 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:31.006+0000 7ffa08d82640 1 -- 172.21.15.43:0/2546648074 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffa0002c6d0 con 0x7ff9f0003970 2023-08-09T15:11:31.015 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:31.006+0000 7ffa08d82640 1 -- 172.21.15.43:0/2546648074 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ffa0002c830 con 0x7ff9f0003970 2023-08-09T15:11:31.015 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:31.006+0000 7ffa08d82640 1 -- 172.21.15.43:0/2546648074 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffa0002cbc0 con 0x7ff9f0003970 2023-08-09T15:11:31.015 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:31.006+0000 7ffa0ad86640 1 -- 172.21.15.43:0/2546648074 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff9f0003970 msgr2=0x7ff9f0003db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T15:11:31.016 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:31.006+0000 7ffa0ad86640 1 --2- 172.21.15.43:0/2546648074 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff9f0003970 0x7ff9f0003db0 secure :-1 s=READY pgs=2401 cs=0 l=1 rev1=1 crypto rx=0x7ffa00009ac0 tx=0x7ff9f00057f0 comp rx=0 tx=0).stop 2023-08-09T15:11:31.016 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:31.007+0000 7ffa0ad86640 1 -- 172.21.15.43:0/2546648074 shutdown_connections 2023-08-09T15:11:31.016 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:31.007+0000 7ffa0ad86640 1 --2- 172.21.15.43:0/2546648074 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff9f0003970 0x7ff9f0003db0 unknown :-1 s=CLOSED pgs=2401 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T15:11:31.017 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:31.007+0000 7ffa0ad86640 1 -- 172.21.15.43:0/2546648074 >> 172.21.15.43:0/2546648074 conn(0x7ffa040af9f0 msgr2=0x7ffa040afdd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-09T15:11:31.017 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:31.007+0000 7ffa0ad86640 1 -- 172.21.15.43:0/2546648074 shutdown_connections 2023-08-09T15:11:31.017 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:31.007+0000 7ffa0ad86640 1 -- 172.21.15.43:0/2546648074 wait complete. 2023-08-09T15:11:31.021 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:31.008+0000 7ffa0ad86640 1 Processor -- start 2023-08-09T15:11:31.021 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:31.009+0000 7ffa0ad86640 1 -- start start 2023-08-09T15:11:31.021 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:31.009+0000 7ffa0ad86640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7ff9f0003970 0x7ffa040cc740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T15:11:31.022 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:31.010+0000 7ffa0ad86640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7ffa040c4530 0x7ffa04154a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T15:11:31.022 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:31.010+0000 7ffa0ad86640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ffa040af370 0x7ffa04154f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T15:11:31.022 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:31.010+0000 7ffa0ad86640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffa041f2780 con 0x7ffa040af370 2023-08-09T15:11:31.022 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:31.010+0000 7ffa0ad86640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffa041554a0 con 0x7ffa040c4530 2023-08-09T15:11:31.023 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:31.010+0000 7ffa0ad86640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ffa041f1bd0 con 0x7ff9f0003970 2023-08-09T15:11:31.023 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:31.010+0000 7ffa0a585640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ffa040af370 0x7ffa04154f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T15:11:31.023 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:31.010+0000 7ffa0a585640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ffa040af370 0x7ffa04154f60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:56096/0 (socket says 172.21.15.43:56096) 2023-08-09T15:11:31.023 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:31.010+0000 7ffa0a585640 1 -- 172.21.15.43:0/2063560375 learned_addr learned my addr 172.21.15.43:0/2063560375 (peer_addr_for_me v2:172.21.15.43:0/0) 2023-08-09T15:11:31.024 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:31.010+0000 7ffa09d84640 1 --2- 172.21.15.43:0/2063560375 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7ff9f0003970 0x7ffa040cc740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T15:11:31.024 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:31.010+0000 7ffa09583640 1 --2- 172.21.15.43:0/2063560375 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7ffa040c4530 0x7ffa04154a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T15:11:31.024 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:31.011+0000 7ffa0a585640 1 -- 172.21.15.43:0/2063560375 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7ff9f0003970 msgr2=0x7ffa040cc740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T15:11:31.024 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:31.011+0000 7ffa0a585640 1 --2- 172.21.15.43:0/2063560375 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7ff9f0003970 0x7ffa040cc740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T15:11:31.024 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:31.011+0000 7ffa0a585640 1 -- 172.21.15.43:0/2063560375 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7ffa040c4530 msgr2=0x7ffa04154a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T15:11:31.025 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:31.011+0000 7ffa0a585640 1 --2- 172.21.15.43:0/2063560375 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7ffa040c4530 0x7ffa04154a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T15:11:31.025 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:31.011+0000 7ffa0a585640 1 -- 172.21.15.43:0/2063560375 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ffa040c7ec0 con 0x7ffa040af370 2023-08-09T15:11:31.025 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:31.011+0000 7ffa0a585640 1 --2- 172.21.15.43:0/2063560375 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ffa040af370 0x7ffa04154f60 secure :-1 s=READY pgs=2402 cs=0 l=1 rev1=1 crypto rx=0x7ff9fc001580 tx=0x7ff9fc00b0d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-09T15:11:31.025 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:31.011+0000 7ff9fa7fc640 1 -- 172.21.15.43:0/2063560375 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff9fc00b8d0 con 0x7ffa040af370 2023-08-09T15:11:31.025 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:31.011+0000 7ff9fa7fc640 1 -- 172.21.15.43:0/2063560375 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff9fc00ba50 con 0x7ffa040af370 2023-08-09T15:11:31.025 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:31.011+0000 7ff9fa7fc640 1 -- 172.21.15.43:0/2063560375 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff9fc00be00 con 0x7ffa040af370 2023-08-09T15:11:31.025 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:31.011+0000 7ffa0ad86640 1 -- 172.21.15.43:0/2063560375 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ffa040c4ef0 con 0x7ffa040af370 2023-08-09T15:11:31.026 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:31.011+0000 7ffa0ad86640 1 -- 172.21.15.43:0/2063560375 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ffa041f20c0 con 0x7ffa040af370 2023-08-09T15:11:31.026 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:31.013+0000 7ffa0ad86640 1 -- 172.21.15.43:0/2063560375 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ffa041f28c0 con 0x7ffa040af370 2023-08-09T15:11:31.026 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:31.013+0000 7ff9fa7fc640 1 -- 172.21.15.43:0/2063560375 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 118) v1 ==== 88443+0+0 (secure 0 0 0) 0x7ff9fc007070 con 0x7ffa040af370 2023-08-09T15:11:31.026 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:31.014+0000 7ff9fa7fc640 1 --2- 172.21.15.43:0/2063560375 >> [v2:172.21.15.203:6832/4034093268,v1:172.21.15.203:6833/4034093268] conn(0x7ff9e806a170 0x7ff9e806c630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T15:11:31.026 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:31.014+0000 7ff9fa7fc640 1 -- 172.21.15.43:0/2063560375 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(223..223 src has 1..223) v4 ==== 6426+0+0 (secure 0 0 0) 0x7ff9fc088bf0 con 0x7ffa040af370 2023-08-09T15:11:31.027 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:31.015+0000 7ffa09d84640 1 --2- 172.21.15.43:0/2063560375 >> [v2:172.21.15.203:6832/4034093268,v1:172.21.15.203:6833/4034093268] conn(0x7ff9e806a170 0x7ff9e806c630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T15:11:31.027 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:31.016+0000 7ffa09d84640 1 --2- 172.21.15.43:0/2063560375 >> [v2:172.21.15.203:6832/4034093268,v1:172.21.15.203:6833/4034093268] conn(0x7ff9e806a170 0x7ff9e806c630 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7ffa00036000 tx=0x7ffa00030000 comp rx=0 tx=0).ready entity=mgr.8833 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-09T15:11:31.027 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:31.019+0000 7ff9fa7fc640 1 -- 172.21.15.43:0/2063560375 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178066 (secure 0 0 0) 0x7ff9fc053f40 con 0x7ffa040af370 2023-08-09T15:11:31.249 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:31.249+0000 7ffa0ad86640 1 -- 172.21.15.43:0/2063560375 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"} v 0) v1 -- 0x7ffa0404e390 con 0x7ffa040af370 2023-08-09T15:11:31.753 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:31.752+0000 7ff9fa7fc640 1 -- 172.21.15.43:0/2063560375 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"}]=0 v104) v1 ==== 122+0+0 (secure 0 0 0) 0x7ff9fc057560 con 0x7ffa040af370 2023-08-09T15:11:31.756 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:31.756+0000 7ffa0ad86640 1 -- 172.21.15.43:0/2063560375 >> [v2:172.21.15.203:6832/4034093268,v1:172.21.15.203:6833/4034093268] conn(0x7ff9e806a170 msgr2=0x7ff9e806c630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T15:11:31.756 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:31.756+0000 7ffa0ad86640 1 --2- 172.21.15.43:0/2063560375 >> [v2:172.21.15.203:6832/4034093268,v1:172.21.15.203:6833/4034093268] conn(0x7ff9e806a170 0x7ff9e806c630 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7ffa00036000 tx=0x7ffa00030000 comp rx=0 tx=0).stop 2023-08-09T15:11:31.757 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:31.756+0000 7ffa0ad86640 1 -- 172.21.15.43:0/2063560375 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ffa040af370 msgr2=0x7ffa04154f60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T15:11:31.757 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:31.756+0000 7ffa0ad86640 1 --2- 172.21.15.43:0/2063560375 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ffa040af370 0x7ffa04154f60 secure :-1 s=READY pgs=2402 cs=0 l=1 rev1=1 crypto rx=0x7ff9fc001580 tx=0x7ff9fc00b0d0 comp rx=0 tx=0).stop 2023-08-09T15:11:31.757 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:31.756+0000 7ffa0ad86640 1 -- 172.21.15.43:0/2063560375 shutdown_connections 2023-08-09T15:11:31.758 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:31.756+0000 7ffa0ad86640 1 --2- 172.21.15.43:0/2063560375 >> [v2:172.21.15.203:6832/4034093268,v1:172.21.15.203:6833/4034093268] conn(0x7ff9e806a170 0x7ff9e806c630 unknown :-1 s=CLOSED pgs=61 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T15:11:31.758 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:31.756+0000 7ffa0ad86640 1 --2- 172.21.15.43:0/2063560375 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ffa040af370 0x7ffa04154f60 unknown :-1 s=CLOSED pgs=2402 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T15:11:31.758 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:31.756+0000 7ffa0ad86640 1 --2- 172.21.15.43:0/2063560375 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7ffa040c4530 0x7ffa04154a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T15:11:31.758 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:31.756+0000 7ffa0ad86640 1 --2- 172.21.15.43:0/2063560375 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7ff9f0003970 0x7ffa040cc740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T15:11:31.758 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:31.756+0000 7ffa0ad86640 1 -- 172.21.15.43:0/2063560375 >> 172.21.15.43:0/2063560375 conn(0x7ffa040af9f0 msgr2=0x7ffa04155c70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-09T15:11:31.758 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:31.757+0000 7ffa0ad86640 1 -- 172.21.15.43:0/2063560375 shutdown_connections 2023-08-09T15:11:31.758 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:31.757+0000 7ffa0ad86640 1 -- 172.21.15.43:0/2063560375 wait complete. 2023-08-09T15:11:31.778 DEBUG:tasks.cephfs.filesystem:Not Creating any subvols for filesystem 'cephfs' 2023-08-09T15:11:31.779 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-08-09T15:11:32.007 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:32.006+0000 7fba84988640 1 Processor -- start 2023-08-09T15:11:32.007 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:32.006+0000 7fba84988640 1 -- start start 2023-08-09T15:11:32.008 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:32.008+0000 7fba84988640 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7fba8015a6a0 0x7fba800713b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T15:11:32.008 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:32.008+0000 7fba84988640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7fba800c96c0 0x7fba8015ad20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T15:11:32.009 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:32.008+0000 7fba84988640 1 -- --> v1:172.21.15.43:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fba800cdc70 con 0x7fba800cbd90 2023-08-09T15:11:32.009 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:32.008+0000 7fba84988640 1 -- --> v2:172.21.15.43:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fba80070910 con 0x7fba800c96c0 2023-08-09T15:11:32.009 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:32.008+0000 7fba84988640 1 -- --> v2:172.21.15.203:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fba800c88c0 con 0x7fba8015a6a0 2023-08-09T15:11:32.010 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:32.008+0000 7fba7ffff640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7fba800c96c0 0x7fba8015ad20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T15:11:32.010 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:32.008+0000 7fba7ffff640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7fba800c96c0 0x7fba8015ad20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:56112/0 (socket says 172.21.15.43:56112) 2023-08-09T15:11:32.010 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:32.008+0000 7fba7ffff640 1 -- 172.21.15.43:0/4198486442 learned_addr learned my addr 172.21.15.43:0/4198486442 (peer_addr_for_me v2:172.21.15.43:0/0) 2023-08-09T15:11:32.010 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:32.009+0000 7fba7f7fe640 1 --2- 172.21.15.43:0/4198486442 >> v2:172.21.15.203:3300/0 conn(0x7fba8015a6a0 0x7fba800713b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T15:11:32.010 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:32.009+0000 7fba7ffff640 1 -- 172.21.15.43:0/4198486442 >> v2:172.21.15.203:3300/0 conn(0x7fba8015a6a0 msgr2=0x7fba800713b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-09T15:11:32.010 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:32.009+0000 7fba7ffff640 1 --2- 172.21.15.43:0/4198486442 >> v2:172.21.15.203:3300/0 conn(0x7fba8015a6a0 0x7fba800713b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T15:11:32.011 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:32.009+0000 7fba7ffff640 1 -- 172.21.15.43:0/4198486442 >> v1:172.21.15.43:6789/0 conn(0x7fba800cbd90 legacy=0x7fba800c8fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T15:11:32.011 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:32.009+0000 7fba7ffff640 1 -- 172.21.15.43:0/4198486442 --> v2:172.21.15.43:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fba80070bb0 con 0x7fba800c96c0 2023-08-09T15:11:32.011 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:32.010+0000 7fba7ffff640 1 --2- 172.21.15.43:0/4198486442 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fba800c96c0 0x7fba8015ad20 secure :-1 s=READY pgs=2404 cs=0 l=1 rev1=1 crypto rx=0x7fba8006d920 tx=0x7fba7802bee0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b191561cb3163d11 server_cookie=0 in_seq=0 out_seq=0 2023-08-09T15:11:32.012 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:32.010+0000 7fba7e7fc640 1 -- 172.21.15.43:0/4198486442 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fba7802ca80 con 0x7fba800c96c0 2023-08-09T15:11:32.012 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:32.010+0000 7fba7e7fc640 1 -- 172.21.15.43:0/4198486442 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fba7802cbe0 con 0x7fba800c96c0 2023-08-09T15:11:32.012 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:32.010+0000 7fba7e7fc640 1 -- 172.21.15.43:0/4198486442 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fba780343f0 con 0x7fba800c96c0 2023-08-09T15:11:32.012 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:32.011+0000 7fba84988640 1 -- 172.21.15.43:0/4198486442 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fba800c96c0 msgr2=0x7fba8015ad20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T15:11:32.012 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:32.011+0000 7fba84988640 1 --2- 172.21.15.43:0/4198486442 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fba800c96c0 0x7fba8015ad20 secure :-1 s=READY pgs=2404 cs=0 l=1 rev1=1 crypto rx=0x7fba8006d920 tx=0x7fba7802bee0 comp rx=0 tx=0).stop 2023-08-09T15:11:32.012 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:32.011+0000 7fba84988640 1 -- 172.21.15.43:0/4198486442 shutdown_connections 2023-08-09T15:11:32.013 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:32.011+0000 7fba84988640 1 --2- 172.21.15.43:0/4198486442 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fba800c96c0 0x7fba8015ad20 unknown :-1 s=CLOSED pgs=2404 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T15:11:32.013 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:32.011+0000 7fba84988640 1 --2- 172.21.15.43:0/4198486442 >> v2:172.21.15.203:3300/0 conn(0x7fba8015a6a0 0x7fba800713b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T15:11:32.013 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:32.011+0000 7fba84988640 1 -- 172.21.15.43:0/4198486442 >> 172.21.15.43:0/4198486442 conn(0x7fba800cde90 msgr2=0x7fba800ce2a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-09T15:11:32.013 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:32.011+0000 7fba84988640 1 -- 172.21.15.43:0/4198486442 shutdown_connections 2023-08-09T15:11:32.013 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:32.011+0000 7fba84988640 1 -- 172.21.15.43:0/4198486442 wait complete. 2023-08-09T15:11:32.014 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:32.013+0000 7fba84988640 1 Processor -- start 2023-08-09T15:11:32.014 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:32.013+0000 7fba84988640 1 -- start start 2023-08-09T15:11:32.015 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:32.014+0000 7fba84988640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fba800c96c0 0x7fba80151c30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T15:11:32.015 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:32.014+0000 7fba84988640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fba800cbd90 0x7fba801527b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T15:11:32.015 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:32.014+0000 7fba84988640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fba8015a6a0 0x7fba8006c310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T15:11:32.015 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:32.014+0000 7fba84988640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fba8013a040 con 0x7fba800c96c0 2023-08-09T15:11:32.015 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:32.014+0000 7fba84988640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fba8013a320 con 0x7fba800cbd90 2023-08-09T15:11:32.015 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:32.014+0000 7fba84988640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fba8013a600 con 0x7fba8015a6a0 2023-08-09T15:11:32.016 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:32.014+0000 7fba7f7fe640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fba800c96c0 0x7fba80151c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T15:11:32.016 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:32.014+0000 7fba7f7fe640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fba800c96c0 0x7fba80151c30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:56116/0 (socket says 172.21.15.43:56116) 2023-08-09T15:11:32.016 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:32.014+0000 7fba7f7fe640 1 -- 172.21.15.43:0/1442303328 learned_addr learned my addr 172.21.15.43:0/1442303328 (peer_addr_for_me v2:172.21.15.43:0/0) 2023-08-09T15:11:32.016 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:32.015+0000 7fba7f7fe640 1 -- 172.21.15.43:0/1442303328 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fba8015a6a0 msgr2=0x7fba8006c310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T15:11:32.016 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:32.015+0000 7fba7f7fe640 1 --2- 172.21.15.43:0/1442303328 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fba8015a6a0 0x7fba8006c310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T15:11:32.016 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:32.015+0000 7fba7f7fe640 1 -- 172.21.15.43:0/1442303328 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fba800cbd90 msgr2=0x7fba801527b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T15:11:32.016 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:32.015+0000 7fba7f7fe640 1 --2- 172.21.15.43:0/1442303328 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fba800cbd90 0x7fba801527b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T15:11:32.017 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:32.015+0000 7fba7f7fe640 1 -- 172.21.15.43:0/1442303328 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fba78009b30 con 0x7fba800c96c0 2023-08-09T15:11:32.018 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:32.015+0000 7fba7f7fe640 1 --2- 172.21.15.43:0/1442303328 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fba800c96c0 0x7fba80151c30 secure :-1 s=READY pgs=2405 cs=0 l=1 rev1=1 crypto rx=0x7fba7000d5d0 tx=0x7fba7000da90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-09T15:11:32.018 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:32.015+0000 7fba67fff640 1 -- 172.21.15.43:0/1442303328 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fba700125f0 con 0x7fba800c96c0 2023-08-09T15:11:32.018 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:32.015+0000 7fba67fff640 1 -- 172.21.15.43:0/1442303328 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fba70012750 con 0x7fba800c96c0 2023-08-09T15:11:32.018 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:32.015+0000 7fba67fff640 1 -- 172.21.15.43:0/1442303328 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fba70012b00 con 0x7fba800c96c0 2023-08-09T15:11:32.018 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:32.015+0000 7fba84988640 1 -- 172.21.15.43:0/1442303328 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fba8013a8e0 con 0x7fba800c96c0 2023-08-09T15:11:32.018 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:32.015+0000 7fba84988640 1 -- 172.21.15.43:0/1442303328 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fba8014e320 con 0x7fba800c96c0 2023-08-09T15:11:32.019 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:32.017+0000 7fba84988640 1 -- 172.21.15.43:0/1442303328 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fba800739f0 con 0x7fba800c96c0 2023-08-09T15:11:32.019 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:32.017+0000 7fba67fff640 1 -- 172.21.15.43:0/1442303328 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 118) v1 ==== 88443+0+0 (secure 0 0 0) 0x7fba70012c60 con 0x7fba800c96c0 2023-08-09T15:11:32.019 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:32.018+0000 7fba67fff640 1 --2- 172.21.15.43:0/1442303328 >> [v2:172.21.15.203:6832/4034093268,v1:172.21.15.203:6833/4034093268] conn(0x7fba6c06a170 0x7fba6c06c630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T15:11:32.019 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:32.018+0000 7fba67fff640 1 -- 172.21.15.43:0/1442303328 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(223..223 src has 1..223) v4 ==== 6426+0+0 (secure 0 0 0) 0x7fba70088db0 con 0x7fba800c96c0 2023-08-09T15:11:32.020 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:32.019+0000 7fba7effd640 1 --2- 172.21.15.43:0/1442303328 >> [v2:172.21.15.203:6832/4034093268,v1:172.21.15.203:6833/4034093268] conn(0x7fba6c06a170 0x7fba6c06c630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T15:11:32.020 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:32.020+0000 7fba7effd640 1 --2- 172.21.15.43:0/1442303328 >> [v2:172.21.15.203:6832/4034093268,v1:172.21.15.203:6833/4034093268] conn(0x7fba6c06a170 0x7fba6c06c630 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7fba74000f60 tx=0x7fba740079b0 comp rx=0 tx=0).ready entity=mgr.8833 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-09T15:11:32.022 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:32.022+0000 7fba67fff640 1 -- 172.21.15.43:0/1442303328 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178066 (secure 0 0 0) 0x7fba7008d070 con 0x7fba800c96c0 2023-08-09T15:11:32.142 DEBUG:teuthology.orchestra.run.smithi043:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-08-09T15:11:32.151 DEBUG:teuthology.orchestra.run.smithi203:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-08-09T15:11:32.259 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:32.259+0000 7fba84988640 1 -- 172.21.15.43:0/1442303328 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fba800cec10 con 0x7fba800c96c0 2023-08-09T15:11:32.260 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:32.260+0000 7fba67fff640 1 -- 172.21.15.43:0/1442303328 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 104 v104) v1 ==== 95+0+4894 (secure 0 0 0) 0x7fba70054210 con 0x7fba800c96c0 2023-08-09T15:11:32.261 INFO:teuthology.orchestra.run.smithi043.stdout: 2023-08-09T15:11:32.261 INFO:teuthology.orchestra.run.smithi043.stdout:{"epoch":104,"default_fscid":12,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":8012,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.203:6837/2464832767","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6836","nonce":2464832767},{"type":"v1","addr":"172.21.15.203:6837","nonce":2464832767}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":100},{"gid":9010,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.43:6835/3989386049","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6834","nonce":3989386049},{"type":"v1","addr":"172.21.15.43:6835","nonce":3989386049}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":100},{"gid":9016,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.203:6835/2213455074","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6834","nonce":2213455074},{"type":"v1","addr":"172.21.15.203:6835","nonce":2213455074}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":100}],"filesystems":[{"mdsmap":{"epoch":104,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-08-09T15:11:28.168916+0000","modified":"2023-08-09T15:11:31.739094+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":11325},"failed":[],"damaged":[],"stopped":[],"info":{"gid_11325":{"gid":11325,"name":"a","rank":0,"incarnation":101,"state":"up:active","state_seq":3,"addr":"172.21.15.43:6833/1425989175","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6832","nonce":1425989175},{"type":"v1","addr":"172.21.15.43:6833","nonce":1425989175}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[27],"metadata_pool":26,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":12}]} 2023-08-09T15:11:32.262 INFO:teuthology.orchestra.run.smithi043.stderr:dumped fsmap epoch 104 2023-08-09T15:11:32.263 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:32.263+0000 7fba84988640 1 -- 172.21.15.43:0/1442303328 >> [v2:172.21.15.203:6832/4034093268,v1:172.21.15.203:6833/4034093268] conn(0x7fba6c06a170 msgr2=0x7fba6c06c630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T15:11:32.264 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:32.263+0000 7fba84988640 1 --2- 172.21.15.43:0/1442303328 >> [v2:172.21.15.203:6832/4034093268,v1:172.21.15.203:6833/4034093268] conn(0x7fba6c06a170 0x7fba6c06c630 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7fba74000f60 tx=0x7fba740079b0 comp rx=0 tx=0).stop 2023-08-09T15:11:32.264 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:32.263+0000 7fba84988640 1 -- 172.21.15.43:0/1442303328 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fba800c96c0 msgr2=0x7fba80151c30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T15:11:32.264 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:32.263+0000 7fba84988640 1 --2- 172.21.15.43:0/1442303328 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fba800c96c0 0x7fba80151c30 secure :-1 s=READY pgs=2405 cs=0 l=1 rev1=1 crypto rx=0x7fba7000d5d0 tx=0x7fba7000da90 comp rx=0 tx=0).stop 2023-08-09T15:11:32.264 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:32.263+0000 7fba84988640 1 -- 172.21.15.43:0/1442303328 shutdown_connections 2023-08-09T15:11:32.264 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:32.263+0000 7fba84988640 1 --2- 172.21.15.43:0/1442303328 >> [v2:172.21.15.203:6832/4034093268,v1:172.21.15.203:6833/4034093268] conn(0x7fba6c06a170 0x7fba6c06c630 unknown :-1 s=CLOSED pgs=62 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T15:11:32.264 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:32.263+0000 7fba84988640 1 --2- 172.21.15.43:0/1442303328 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fba8015a6a0 0x7fba8006c310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T15:11:32.265 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:32.263+0000 7fba84988640 1 --2- 172.21.15.43:0/1442303328 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fba800cbd90 0x7fba801527b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T15:11:32.265 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:32.263+0000 7fba84988640 1 --2- 172.21.15.43:0/1442303328 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fba800c96c0 0x7fba80151c30 unknown :-1 s=CLOSED pgs=2405 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T15:11:32.265 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:32.263+0000 7fba84988640 1 -- 172.21.15.43:0/1442303328 >> 172.21.15.43:0/1442303328 conn(0x7fba800cde90 msgr2=0x7fba8015cdb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-09T15:11:32.265 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:32.264+0000 7fba84988640 1 -- 172.21.15.43:0/1442303328 shutdown_connections 2023-08-09T15:11:32.266 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:32.264+0000 7fba84988640 1 -- 172.21.15.43:0/1442303328 wait complete. 2023-08-09T15:11:32.283 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-08-09T15:11:32.565 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:32.565+0000 7f94d2b96640 1 Processor -- start 2023-08-09T15:11:32.566 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:32.565+0000 7f94d2b96640 1 -- start start 2023-08-09T15:11:32.567 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:32.567+0000 7f94d2b96640 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7f94cc0b2790 0x7f94cc0d2ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T15:11:32.567 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:32.567+0000 7f94d2b96640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f94cc0b2b70 0x7f94cc0b5330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T15:11:32.567 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:32.567+0000 7f94d2b96640 1 -- --> v1:172.21.15.43:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f94cc0ce090 con 0x7f94cc0b3050 2023-08-09T15:11:32.568 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:32.567+0000 7f94d2b96640 1 -- --> v2:172.21.15.203:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f94cc0b3950 con 0x7f94cc0b2790 2023-08-09T15:11:32.568 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:32.567+0000 7f94d2b96640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f94cc0cd7f0 con 0x7f94cc0b2b70 2023-08-09T15:11:32.569 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:32.567+0000 7f94d1133640 1 --1- >> v1:172.21.15.43:6789/0 conn(0x7f94cc0b3050 0x7f94cc0b3f60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.43:6789/0 says I am v1:172.21.15.43:42914/0 (socket says 172.21.15.43:42914) 2023-08-09T15:11:32.569 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:32.567+0000 7f94d1133640 1 -- 172.21.15.43:0/395441388 learned_addr learned my addr 172.21.15.43:0/395441388 (peer_addr_for_me v1:172.21.15.43:0/0) 2023-08-09T15:11:32.570 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:32.568+0000 7f94d0932640 1 --2- 172.21.15.43:0/395441388 >> v2:172.21.15.203:3300/0 conn(0x7f94cc0b2790 0x7f94cc0d2ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T15:11:32.570 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:32.568+0000 7f94cbfff640 1 --2- 172.21.15.43:0/395441388 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f94cc0b2b70 0x7f94cc0b5330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T15:11:32.570 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:32.568+0000 7f94cb7fe640 1 -- 172.21.15.43:0/395441388 <== mon.0 v1:172.21.15.43:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3104972421 0 0) 0x7f94cc0ce090 con 0x7f94cc0b3050 2023-08-09T15:11:32.570 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:32.568+0000 7f94cb7fe640 1 -- 172.21.15.43:0/395441388 --> v1:172.21.15.43:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f94bc003520 con 0x7f94cc0b3050 2023-08-09T15:11:32.570 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:32.568+0000 7f94cb7fe640 1 -- 172.21.15.43:0/395441388 <== mon.0 v1:172.21.15.43:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2572699248 0 0) 0x7f94bc003520 con 0x7f94cc0b3050 2023-08-09T15:11:32.570 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:32.568+0000 7f94cb7fe640 1 -- 172.21.15.43:0/395441388 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f94cc0b2b70 msgr2=0x7f94cc0b5330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-09T15:11:32.570 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:32.568+0000 7f94cb7fe640 1 --2- 172.21.15.43:0/395441388 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f94cc0b2b70 0x7f94cc0b5330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T15:11:32.571 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:32.568+0000 7f94cb7fe640 1 -- 172.21.15.43:0/395441388 >> v2:172.21.15.203:3300/0 conn(0x7f94cc0b2790 msgr2=0x7f94cc0d2ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-09T15:11:32.571 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:32.568+0000 7f94cb7fe640 1 --2- 172.21.15.43:0/395441388 >> v2:172.21.15.203:3300/0 conn(0x7f94cc0b2790 0x7f94cc0d2ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T15:11:32.571 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:32.568+0000 7f94cb7fe640 1 -- 172.21.15.43:0/395441388 --> v1:172.21.15.43:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f94cc0cdad0 con 0x7f94cc0b3050 2023-08-09T15:11:32.571 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:32.568+0000 7f94cb7fe640 1 -- 172.21.15.43:0/395441388 <== mon.0 v1:172.21.15.43:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1285150740 0 0) 0x7f94c40019f0 con 0x7f94cc0b3050 2023-08-09T15:11:32.571 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:32.568+0000 7f94cb7fe640 1 -- 172.21.15.43:0/395441388 >> v1:172.21.15.43:6789/0 conn(0x7f94cc0b3050 legacy=0x7f94cc0b3f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T15:11:32.571 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:32.568+0000 7f94cb7fe640 1 --2- 172.21.15.43:0/395441388 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f94cc0b2b70 0x7f94bc0039e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T15:11:32.571 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:32.568+0000 7f94cb7fe640 1 --2- 172.21.15.43:0/395441388 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f94bc003f60 0x7f94bc0047c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T15:11:32.571 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:32.568+0000 7f94cb7fe640 1 --2- 172.21.15.43:0/395441388 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f94bc004d00 0x7f94bc005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T15:11:32.572 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:32.568+0000 7f94cb7fe640 1 -- 172.21.15.43:0/395441388 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f94cc0b3950 con 0x7f94bc003f60 2023-08-09T15:11:32.572 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:32.568+0000 7f94cb7fe640 1 -- 172.21.15.43:0/395441388 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f94cc0cd7f0 con 0x7f94bc004d00 2023-08-09T15:11:32.572 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:32.568+0000 7f94cb7fe640 1 -- 172.21.15.43:0/395441388 --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f94bc00b9b0 con 0x7f94cc0b2b70 2023-08-09T15:11:32.572 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:32.569+0000 7f94cbfff640 1 --2- 172.21.15.43:0/395441388 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f94bc003f60 0x7f94bc0047c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T15:11:32.572 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:32.569+0000 7f94d1133640 1 --2- 172.21.15.43:0/395441388 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f94bc004d00 0x7f94bc005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T15:11:32.572 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:32.569+0000 7f94d0932640 1 --2- 172.21.15.43:0/395441388 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f94cc0b2b70 0x7f94bc0039e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T15:11:32.573 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:32.569+0000 7f94cbfff640 1 -- 172.21.15.43:0/395441388 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f94cc0b2b70 msgr2=0x7f94bc0039e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-09T15:11:32.574 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:32.569+0000 7f94cbfff640 1 --2- 172.21.15.43:0/395441388 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f94cc0b2b70 0x7f94bc0039e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T15:11:32.574 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:32.569+0000 7f94cbfff640 1 -- 172.21.15.43:0/395441388 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f94bc004d00 msgr2=0x7f94bc005180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-09T15:11:32.574 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:32.569+0000 7f94cbfff640 1 --2- 172.21.15.43:0/395441388 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f94bc004d00 0x7f94bc005180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T15:11:32.574 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:32.569+0000 7f94cbfff640 1 -- 172.21.15.43:0/395441388 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f94bc003520 con 0x7f94bc003f60 2023-08-09T15:11:32.574 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:32.569+0000 7f94d1133640 1 --2- 172.21.15.43:0/395441388 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f94bc004d00 0x7f94bc005180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-08-09T15:11:32.574 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:32.571+0000 7f94cbfff640 1 --2- 172.21.15.43:0/395441388 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f94bc003f60 0x7f94bc0047c0 secure :-1 s=READY pgs=2407 cs=0 l=1 rev1=1 crypto rx=0x7f94b8009ac0 tx=0x7f94bc0057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e77aac3caf9c7e90 server_cookie=0 in_seq=0 out_seq=0 2023-08-09T15:11:32.574 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:32.571+0000 7f94cb7fe640 1 -- 172.21.15.43:0/395441388 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f94b802c6b0 con 0x7f94bc003f60 2023-08-09T15:11:32.575 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:32.571+0000 7f94cb7fe640 1 -- 172.21.15.43:0/395441388 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f94b802c810 con 0x7f94bc003f60 2023-08-09T15:11:32.575 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:32.571+0000 7f94cb7fe640 1 -- 172.21.15.43:0/395441388 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f94b802cba0 con 0x7f94bc003f60 2023-08-09T15:11:32.575 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:32.572+0000 7f94d2b96640 1 -- 172.21.15.43:0/395441388 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f94bc003f60 msgr2=0x7f94bc0047c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T15:11:32.575 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:32.572+0000 7f94d2b96640 1 --2- 172.21.15.43:0/395441388 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f94bc003f60 0x7f94bc0047c0 secure :-1 s=READY pgs=2407 cs=0 l=1 rev1=1 crypto rx=0x7f94b8009ac0 tx=0x7f94bc0057f0 comp rx=0 tx=0).stop 2023-08-09T15:11:32.575 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:32.572+0000 7f94d1133640 1 -- 172.21.15.43:0/395441388 reap_dead start 2023-08-09T15:11:32.575 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:32.572+0000 7f94d2b96640 1 -- 172.21.15.43:0/395441388 shutdown_connections 2023-08-09T15:11:32.575 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:32.572+0000 7f94d2b96640 1 -- 172.21.15.43:0/395441388 >> 172.21.15.43:0/395441388 conn(0x7f94cc09b7b0 msgr2=0x7f94cc0d24e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-09T15:11:32.576 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:32.572+0000 7f94d2b96640 1 -- 172.21.15.43:0/395441388 shutdown_connections 2023-08-09T15:11:32.576 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:32.572+0000 7f94d2b96640 1 -- 172.21.15.43:0/395441388 wait complete. 2023-08-09T15:11:32.577 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:32.574+0000 7f94d2b96640 1 Processor -- start 2023-08-09T15:11:32.577 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:32.574+0000 7f94d2b96640 1 -- start start 2023-08-09T15:11:32.577 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:32.575+0000 7f94d2b96640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f94bc003f60 0x7f94cc0b14f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T15:11:32.578 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:32.575+0000 7f94d2b96640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f94cc0afae0 0x7f94cc0cf220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T15:11:32.578 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:32.575+0000 7f94d2b96640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f94cc1f47c0 0x7f94cc1f4c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T15:11:32.578 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:32.575+0000 7f94d2b96640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f94cc0b1ee0 con 0x7f94cc1f47c0 2023-08-09T15:11:32.578 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:32.575+0000 7f94d2b96640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f94cc0c6800 con 0x7f94bc003f60 2023-08-09T15:11:32.578 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:32.575+0000 7f94d2b96640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f94cc0b1c00 con 0x7f94cc0afae0 2023-08-09T15:11:32.578 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:32.575+0000 7f94d1133640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f94cc1f47c0 0x7f94cc1f4c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T15:11:32.578 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:32.575+0000 7f94d1133640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f94cc1f47c0 0x7f94cc1f4c20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:56144/0 (socket says 172.21.15.43:56144) 2023-08-09T15:11:32.579 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:32.575+0000 7f94d1133640 1 -- 172.21.15.43:0/1788720573 learned_addr learned my addr 172.21.15.43:0/1788720573 (peer_addr_for_me v2:172.21.15.43:0/0) 2023-08-09T15:11:32.579 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:32.576+0000 7f94d0932640 1 --2- 172.21.15.43:0/1788720573 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f94bc003f60 0x7f94cc0b14f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T15:11:32.579 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:32.576+0000 7f94cbfff640 1 --2- 172.21.15.43:0/1788720573 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f94cc0afae0 0x7f94cc0cf220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T15:11:32.579 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:32.576+0000 7f94d1133640 1 -- 172.21.15.43:0/1788720573 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f94cc0afae0 msgr2=0x7f94cc0cf220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T15:11:32.579 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:32.576+0000 7f94d1133640 1 --2- 172.21.15.43:0/1788720573 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f94cc0afae0 0x7f94cc0cf220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T15:11:32.579 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:32.576+0000 7f94d1133640 1 -- 172.21.15.43:0/1788720573 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f94bc003f60 msgr2=0x7f94cc0b14f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T15:11:32.579 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:32.576+0000 7f94d1133640 1 --2- 172.21.15.43:0/1788720573 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f94bc003f60 0x7f94cc0b14f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T15:11:32.579 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:32.576+0000 7f94d1133640 1 -- 172.21.15.43:0/1788720573 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f94cc0c61b0 con 0x7f94cc1f47c0 2023-08-09T15:11:32.580 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:32.576+0000 7f94d1133640 1 --2- 172.21.15.43:0/1788720573 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f94cc1f47c0 0x7f94cc1f4c20 secure :-1 s=READY pgs=2408 cs=0 l=1 rev1=1 crypto rx=0x7f94cc072d30 tx=0x7f94c400e530 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-09T15:11:32.580 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:32.577+0000 7f94c97fa640 1 -- 172.21.15.43:0/1788720573 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f94c4007ae0 con 0x7f94cc1f47c0 2023-08-09T15:11:32.580 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:32.577+0000 7f94c97fa640 1 -- 172.21.15.43:0/1788720573 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f94c400ed40 con 0x7f94cc1f47c0 2023-08-09T15:11:32.580 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:32.577+0000 7f94c97fa640 1 -- 172.21.15.43:0/1788720573 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f94c4013660 con 0x7f94cc1f47c0 2023-08-09T15:11:32.580 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:32.577+0000 7f94d2b96640 1 -- 172.21.15.43:0/1788720573 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f94cc0c6490 con 0x7f94cc1f47c0 2023-08-09T15:11:32.580 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:32.577+0000 7f94d2b96640 1 -- 172.21.15.43:0/1788720573 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f94cc1f5160 con 0x7f94cc1f47c0 2023-08-09T15:11:32.581 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:32.578+0000 7f94d2b96640 1 -- 172.21.15.43:0/1788720573 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f94cc0c6970 con 0x7f94cc1f47c0 2023-08-09T15:11:32.581 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:32.579+0000 7f94c97fa640 1 -- 172.21.15.43:0/1788720573 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 118) v1 ==== 88443+0+0 (secure 0 0 0) 0x7f94c40137c0 con 0x7f94cc1f47c0 2023-08-09T15:11:32.581 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:32.580+0000 7f94c97fa640 1 --2- 172.21.15.43:0/1788720573 >> [v2:172.21.15.203:6832/4034093268,v1:172.21.15.203:6833/4034093268] conn(0x7f94b006a1a0 0x7f94b006c660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T15:11:32.581 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:32.580+0000 7f94c97fa640 1 -- 172.21.15.43:0/1788720573 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(223..223 src has 1..223) v4 ==== 6426+0+0 (secure 0 0 0) 0x7f94c4088d00 con 0x7f94cc1f47c0 2023-08-09T15:11:32.582 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:32.581+0000 7f94d0932640 1 --2- 172.21.15.43:0/1788720573 >> [v2:172.21.15.203:6832/4034093268,v1:172.21.15.203:6833/4034093268] conn(0x7f94b006a1a0 0x7f94b006c660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T15:11:32.582 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:32.582+0000 7f94d0932640 1 --2- 172.21.15.43:0/1788720573 >> [v2:172.21.15.203:6832/4034093268,v1:172.21.15.203:6833/4034093268] conn(0x7f94b006a1a0 0x7f94b006c660 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f94c0009a90 tx=0x7f94c0008040 comp rx=0 tx=0).ready entity=mgr.8833 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-09T15:11:32.584 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:32.584+0000 7f94c97fa640 1 -- 172.21.15.43:0/1788720573 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178066 (secure 0 0 0) 0x7f94c4054170 con 0x7f94cc1f47c0 2023-08-09T15:11:32.782 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:32.781+0000 7f94d2b96640 1 -- 172.21.15.43:0/1788720573 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f94cc0c5fb0 con 0x7f94cc1f47c0 2023-08-09T15:11:32.785 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:32.784+0000 7f94c97fa640 1 -- 172.21.15.43:0/1788720573 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v223) v1 ==== 74+0+18570 (secure 0 0 0) 0x7f94c4057790 con 0x7f94cc1f47c0 2023-08-09T15:11:32.785 INFO:teuthology.orchestra.run.smithi043.stdout: 2023-08-09T15:11:32.786 INFO:teuthology.orchestra.run.smithi043.stdout:{"epoch":223,"fsid":"daac87ee-1337-4250-a955-aea4e408a39b","created":"2023-08-09T13:33:41.701198+0000","modified":"2023-08-09T15:11:29.689664+0000","last_up_change":"2023-08-09T13:33:53.783453+0000","last_in_change":"2023-08-09T13:33:45.148904+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":27,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":"rbd","create_time":"2023-08-09T13:33:54.365137+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":".mgr","create_time":"2023-08-09T13:33:54.831803+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":26,"pool_name":"cephfs_metadata","create_time":"2023-08-09T15:11:26.544757+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"222","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":27,"pool_name":"cephfs_data","create_time":"2023-08-09T15:11:27.194686+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"223","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.6299999952316284,"score_stable":1.6299999952316284,"optimal_score":1,"raw_score_acting":1.6299999952316284,"raw_score_stable":1.6299999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"cbd49ecd-e896-4fbd-864e-5c37a829b165","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":221,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6800","nonce":502667335},{"type":"v1","addr":"172.21.15.43:6801","nonce":502667335}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6802","nonce":502667335},{"type":"v1","addr":"172.21.15.43:6803","nonce":502667335}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6806","nonce":502667335},{"type":"v1","addr":"172.21.15.43:6807","nonce":502667335}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6804","nonce":502667335},{"type":"v1","addr":"172.21.15.43:6805","nonce":502667335}]},"public_addr":"172.21.15.43:6801/502667335","cluster_addr":"172.21.15.43:6803/502667335","heartbeat_back_addr":"172.21.15.43:6807/502667335","heartbeat_front_addr":"172.21.15.43:6805/502667335","state":["exists","up"]},{"osd":1,"uuid":"75c7e0fe-1ec7-4fe8-a2f2-3661a65b8500","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":221,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6808","nonce":3289134610},{"type":"v1","addr":"172.21.15.43:6809","nonce":3289134610}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6810","nonce":3289134610},{"type":"v1","addr":"172.21.15.43:6812","nonce":3289134610}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6821","nonce":3289134610},{"type":"v1","addr":"172.21.15.43:6824","nonce":3289134610}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6815","nonce":3289134610},{"type":"v1","addr":"172.21.15.43:6818","nonce":3289134610}]},"public_addr":"172.21.15.43:6809/3289134610","cluster_addr":"172.21.15.43:6812/3289134610","heartbeat_back_addr":"172.21.15.43:6824/3289134610","heartbeat_front_addr":"172.21.15.43:6818/3289134610","state":["exists","up"]},{"osd":2,"uuid":"bb5c41e9-7f5f-4125-9b01-3bb8bfc2ea83","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":221,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6811","nonce":4079709736},{"type":"v1","addr":"172.21.15.43:6814","nonce":4079709736}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6817","nonce":4079709736},{"type":"v1","addr":"172.21.15.43:6820","nonce":4079709736}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6829","nonce":4079709736},{"type":"v1","addr":"172.21.15.43:6831","nonce":4079709736}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6823","nonce":4079709736},{"type":"v1","addr":"172.21.15.43:6826","nonce":4079709736}]},"public_addr":"172.21.15.43:6814/4079709736","cluster_addr":"172.21.15.43:6820/4079709736","heartbeat_back_addr":"172.21.15.43:6831/4079709736","heartbeat_front_addr":"172.21.15.43:6826/4079709736","state":["exists","up"]},{"osd":3,"uuid":"bfe4c8b4-694a-465c-87e6-2b2aab852613","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":221,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6813","nonce":1709538759},{"type":"v1","addr":"172.21.15.43:6816","nonce":1709538759}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6819","nonce":1709538759},{"type":"v1","addr":"172.21.15.43:6822","nonce":1709538759}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6828","nonce":1709538759},{"type":"v1","addr":"172.21.15.43:6830","nonce":1709538759}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6825","nonce":1709538759},{"type":"v1","addr":"172.21.15.43:6827","nonce":1709538759}]},"public_addr":"172.21.15.43:6816/1709538759","cluster_addr":"172.21.15.43:6822/1709538759","heartbeat_back_addr":"172.21.15.43:6830/1709538759","heartbeat_front_addr":"172.21.15.43:6827/1709538759","state":["exists","up"]},{"osd":4,"uuid":"fa2e8a77-aba1-4756-853e-0d174a5bdeba","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":221,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6800","nonce":4196866481},{"type":"v1","addr":"172.21.15.203:6801","nonce":4196866481}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6802","nonce":4196866481},{"type":"v1","addr":"172.21.15.203:6803","nonce":4196866481}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6806","nonce":4196866481},{"type":"v1","addr":"172.21.15.203:6807","nonce":4196866481}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6804","nonce":4196866481},{"type":"v1","addr":"172.21.15.203:6805","nonce":4196866481}]},"public_addr":"172.21.15.203:6801/4196866481","cluster_addr":"172.21.15.203:6803/4196866481","heartbeat_back_addr":"172.21.15.203:6807/4196866481","heartbeat_front_addr":"172.21.15.203:6805/4196866481","state":["exists","up"]},{"osd":5,"uuid":"81811e4b-4d24-4f51-a82c-f1d700a2a00d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":221,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6816","nonce":3059290081},{"type":"v1","addr":"172.21.15.203:6817","nonce":3059290081}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6818","nonce":3059290081},{"type":"v1","addr":"172.21.15.203:6819","nonce":3059290081}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6822","nonce":3059290081},{"type":"v1","addr":"172.21.15.203:6823","nonce":3059290081}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6820","nonce":3059290081},{"type":"v1","addr":"172.21.15.203:6821","nonce":3059290081}]},"public_addr":"172.21.15.203:6817/3059290081","cluster_addr":"172.21.15.203:6819/3059290081","heartbeat_back_addr":"172.21.15.203:6823/3059290081","heartbeat_front_addr":"172.21.15.203:6821/3059290081","state":["exists","up"]},{"osd":6,"uuid":"307bac4d-6a5b-443d-af75-56d695d7674f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":221,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6824","nonce":512155569},{"type":"v1","addr":"172.21.15.203:6825","nonce":512155569}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6826","nonce":512155569},{"type":"v1","addr":"172.21.15.203:6827","nonce":512155569}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6830","nonce":512155569},{"type":"v1","addr":"172.21.15.203:6831","nonce":512155569}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6828","nonce":512155569},{"type":"v1","addr":"172.21.15.203:6829","nonce":512155569}]},"public_addr":"172.21.15.203:6825/512155569","cluster_addr":"172.21.15.203:6827/512155569","heartbeat_back_addr":"172.21.15.203:6831/512155569","heartbeat_front_addr":"172.21.15.203:6829/512155569","state":["exists","up"]},{"osd":7,"uuid":"a95dc7ca-9f1b-4295-af70-68a22a2d7d5e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":221,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6808","nonce":3331450438},{"type":"v1","addr":"172.21.15.203:6809","nonce":3331450438}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6810","nonce":3331450438},{"type":"v1","addr":"172.21.15.203:6811","nonce":3331450438}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6814","nonce":3331450438},{"type":"v1","addr":"172.21.15.203:6815","nonce":3331450438}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6812","nonce":3331450438},{"type":"v1","addr":"172.21.15.203:6813","nonce":3331450438}]},"public_addr":"172.21.15.203:6809/3331450438","cluster_addr":"172.21.15.203:6811/3331450438","heartbeat_back_addr":"172.21.15.203:6815/3331450438","heartbeat_front_addr":"172.21.15.203:6813/3331450438","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-09T13:33:50.547652+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-09T13:33:50.589469+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-09T13:33:50.561724+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-09T13:33:50.599650+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-09T13:33:50.411855+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-09T13:33:50.219681+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-09T13:33:50.504664+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-09T13:33:50.441248+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":7,"snaps":[{"begin":104,"length":1},{"begin":221,"length":1}]}],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-08-09T15:11:32.787 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:32.787+0000 7f94d2b96640 1 -- 172.21.15.43:0/1788720573 >> [v2:172.21.15.203:6832/4034093268,v1:172.21.15.203:6833/4034093268] conn(0x7f94b006a1a0 msgr2=0x7f94b006c660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T15:11:32.788 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:32.787+0000 7f94d2b96640 1 --2- 172.21.15.43:0/1788720573 >> [v2:172.21.15.203:6832/4034093268,v1:172.21.15.203:6833/4034093268] conn(0x7f94b006a1a0 0x7f94b006c660 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f94c0009a90 tx=0x7f94c0008040 comp rx=0 tx=0).stop 2023-08-09T15:11:32.788 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:32.787+0000 7f94d2b96640 1 -- 172.21.15.43:0/1788720573 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f94cc1f47c0 msgr2=0x7f94cc1f4c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T15:11:32.788 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:32.787+0000 7f94d2b96640 1 --2- 172.21.15.43:0/1788720573 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f94cc1f47c0 0x7f94cc1f4c20 secure :-1 s=READY pgs=2408 cs=0 l=1 rev1=1 crypto rx=0x7f94cc072d30 tx=0x7f94c400e530 comp rx=0 tx=0).stop 2023-08-09T15:11:32.788 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:32.787+0000 7f94d2b96640 1 -- 172.21.15.43:0/1788720573 shutdown_connections 2023-08-09T15:11:32.788 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:32.787+0000 7f94d2b96640 1 --2- 172.21.15.43:0/1788720573 >> [v2:172.21.15.203:6832/4034093268,v1:172.21.15.203:6833/4034093268] conn(0x7f94b006a1a0 0x7f94b006c660 unknown :-1 s=CLOSED pgs=63 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T15:11:32.788 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:32.787+0000 7f94d2b96640 1 --2- 172.21.15.43:0/1788720573 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f94cc1f47c0 0x7f94cc1f4c20 unknown :-1 s=CLOSED pgs=2408 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T15:11:32.789 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:32.787+0000 7f94d2b96640 1 --2- 172.21.15.43:0/1788720573 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f94cc0afae0 0x7f94cc0cf220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T15:11:32.789 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:32.787+0000 7f94d2b96640 1 --2- 172.21.15.43:0/1788720573 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f94bc003f60 0x7f94cc0b14f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T15:11:32.789 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:32.787+0000 7f94d2b96640 1 -- 172.21.15.43:0/1788720573 >> 172.21.15.43:0/1788720573 conn(0x7f94cc09b7b0 msgr2=0x7f94cc154b50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-09T15:11:32.789 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:32.787+0000 7f94d2b96640 1 -- 172.21.15.43:0/1788720573 shutdown_connections 2023-08-09T15:11:32.789 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:32.787+0000 7f94d2b96640 1 -- 172.21.15.43:0/1788720573 wait complete. 2023-08-09T15:11:32.808 INFO:tasks.cephfs.filesystem.ceph_manager:waiting for clean 2023-08-09T15:11:32.809 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-08-09T15:11:33.042 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:33.041+0000 7f8c73bab640 1 Processor -- start 2023-08-09T15:11:33.042 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:33.042+0000 7f8c73bab640 1 -- start start 2023-08-09T15:11:33.043 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:33.043+0000 7f8c73bab640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f8c6c143c20 0x7f8c6c06f540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T15:11:33.044 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:33.043+0000 7f8c73bab640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f8c6c0c86f0 0x7f8c6c0c8b50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T15:11:33.044 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:33.043+0000 7f8c73bab640 1 -- --> v1:172.21.15.203:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8c6c0c9090 con 0x7f8c6c06c8d0 2023-08-09T15:11:33.044 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:33.043+0000 7f8c73bab640 1 -- --> v2:172.21.15.43:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8c6c144000 con 0x7f8c6c0c86f0 2023-08-09T15:11:33.044 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:33.043+0000 7f8c73bab640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8c6c06c440 con 0x7f8c6c143c20 2023-08-09T15:11:33.045 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:33.043+0000 7f8c71146640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f8c6c0c86f0 0x7f8c6c0c8b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T15:11:33.046 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:33.044+0000 7f8c71146640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f8c6c0c86f0 0x7f8c6c0c8b50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:56148/0 (socket says 172.21.15.43:56148) 2023-08-09T15:11:33.046 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:33.044+0000 7f8c71146640 1 -- 172.21.15.43:0/3629297965 learned_addr learned my addr 172.21.15.43:0/3629297965 (peer_addr_for_me v2:172.21.15.43:0/0) 2023-08-09T15:11:33.046 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:33.044+0000 7f8c71146640 1 -- 172.21.15.43:0/3629297965 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f8c6c143c20 msgr2=0x7f8c6c06f540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-09T15:11:33.046 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:33.044+0000 7f8c71947640 1 --2- 172.21.15.43:0/3629297965 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f8c6c143c20 0x7f8c6c06f540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T15:11:33.046 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:33.044+0000 7f8c71146640 1 --2- 172.21.15.43:0/3629297965 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f8c6c143c20 0x7f8c6c06f540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T15:11:33.046 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:33.044+0000 7f8c71146640 1 -- 172.21.15.43:0/3629297965 >> v1:172.21.15.203:6789/0 conn(0x7f8c6c06c8d0 legacy=0x7f8c6c06cd60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-09T15:11:33.047 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:33.044+0000 7f8c71146640 1 -- 172.21.15.43:0/3629297965 --> v2:172.21.15.43:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8c6c06b830 con 0x7f8c6c0c86f0 2023-08-09T15:11:33.047 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:33.044+0000 7f8c71947640 1 --2- 172.21.15.43:0/3629297965 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f8c6c143c20 0x7f8c6c06f540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-08-09T15:11:33.047 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:33.045+0000 7f8c71146640 1 --2- 172.21.15.43:0/3629297965 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f8c6c0c86f0 0x7f8c6c0c8b50 secure :-1 s=READY pgs=2409 cs=0 l=1 rev1=1 crypto rx=0x7f8c68009870 tx=0x7f8c6802b860 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d1d953f4decc0b90 server_cookie=0 in_seq=0 out_seq=0 2023-08-09T15:11:33.048 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:33.046+0000 7f8c70945640 1 -- 172.21.15.43:0/3629297965 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8c680317c0 con 0x7f8c6c0c86f0 2023-08-09T15:11:33.048 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:33.046+0000 7f8c70945640 1 -- 172.21.15.43:0/3629297965 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8c68031920 con 0x7f8c6c0c86f0 2023-08-09T15:11:33.048 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:33.046+0000 7f8c70945640 1 -- 172.21.15.43:0/3629297965 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8c68031cb0 con 0x7f8c6c0c86f0 2023-08-09T15:11:33.048 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:33.046+0000 7f8c73bab640 1 -- 172.21.15.43:0/3629297965 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f8c6c0c86f0 msgr2=0x7f8c6c0c8b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T15:11:33.048 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:33.046+0000 7f8c73bab640 1 --2- 172.21.15.43:0/3629297965 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f8c6c0c86f0 0x7f8c6c0c8b50 secure :-1 s=READY pgs=2409 cs=0 l=1 rev1=1 crypto rx=0x7f8c68009870 tx=0x7f8c6802b860 comp rx=0 tx=0).stop 2023-08-09T15:11:33.049 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:33.046+0000 7f8c73bab640 1 -- 172.21.15.43:0/3629297965 shutdown_connections 2023-08-09T15:11:33.049 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:33.046+0000 7f8c73bab640 1 --2- 172.21.15.43:0/3629297965 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f8c6c0c86f0 0x7f8c6c0c8b50 unknown :-1 s=CLOSED pgs=2409 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T15:11:33.049 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:33.046+0000 7f8c73bab640 1 --2- 172.21.15.43:0/3629297965 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f8c6c143c20 0x7f8c6c06f540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T15:11:33.049 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:33.046+0000 7f8c73bab640 1 -- 172.21.15.43:0/3629297965 >> 172.21.15.43:0/3629297965 conn(0x7f8c6c0ce0a0 msgr2=0x7f8c6c0ce480 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-09T15:11:33.049 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:33.047+0000 7f8c73bab640 1 -- 172.21.15.43:0/3629297965 shutdown_connections 2023-08-09T15:11:33.049 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:33.047+0000 7f8c73bab640 1 -- 172.21.15.43:0/3629297965 wait complete. 2023-08-09T15:11:33.051 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:33.048+0000 7f8c73bab640 1 Processor -- start 2023-08-09T15:11:33.051 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:33.048+0000 7f8c73bab640 1 -- start start 2023-08-09T15:11:33.051 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:33.049+0000 7f8c73bab640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f8c6c06c8d0 0x7f8c6c0cad90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T15:11:33.051 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:33.049+0000 7f8c73bab640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f8c6c0c86f0 0x7f8c6c0c7800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T15:11:33.051 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:33.049+0000 7f8c73bab640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f8c6c143c20 0x7f8c6c159a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T15:11:33.051 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:33.049+0000 7f8c73bab640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8c6c0c82e0 con 0x7f8c6c143c20 2023-08-09T15:11:33.052 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:33.049+0000 7f8c73bab640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8c6c0c6270 con 0x7f8c6c0c86f0 2023-08-09T15:11:33.052 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:33.049+0000 7f8c73bab640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8c6c0c6510 con 0x7f8c6c06c8d0 2023-08-09T15:11:33.052 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:33.050+0000 7f8c72148640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f8c6c143c20 0x7f8c6c159a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T15:11:33.052 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:33.050+0000 7f8c72148640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f8c6c143c20 0x7f8c6c159a60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:56158/0 (socket says 172.21.15.43:56158) 2023-08-09T15:11:33.052 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:33.050+0000 7f8c72148640 1 -- 172.21.15.43:0/2784254906 learned_addr learned my addr 172.21.15.43:0/2784254906 (peer_addr_for_me v2:172.21.15.43:0/0) 2023-08-09T15:11:33.052 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:33.050+0000 7f8c71146640 1 --2- 172.21.15.43:0/2784254906 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f8c6c0c86f0 0x7f8c6c0c7800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T15:11:33.052 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:33.050+0000 7f8c71947640 1 --2- 172.21.15.43:0/2784254906 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f8c6c06c8d0 0x7f8c6c0cad90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T15:11:33.053 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:33.050+0000 7f8c72148640 1 -- 172.21.15.43:0/2784254906 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f8c6c06c8d0 msgr2=0x7f8c6c0cad90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T15:11:33.053 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:33.050+0000 7f8c72148640 1 --2- 172.21.15.43:0/2784254906 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f8c6c06c8d0 0x7f8c6c0cad90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T15:11:33.053 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:33.050+0000 7f8c72148640 1 -- 172.21.15.43:0/2784254906 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f8c6c0c86f0 msgr2=0x7f8c6c0c7800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T15:11:33.053 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:33.050+0000 7f8c72148640 1 --2- 172.21.15.43:0/2784254906 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f8c6c0c86f0 0x7f8c6c0c7800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T15:11:33.053 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:33.050+0000 7f8c72148640 1 -- 172.21.15.43:0/2784254906 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8c680094d0 con 0x7f8c6c143c20 2023-08-09T15:11:33.054 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:33.051+0000 7f8c72148640 1 --2- 172.21.15.43:0/2784254906 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f8c6c143c20 0x7f8c6c159a60 secure :-1 s=READY pgs=2410 cs=0 l=1 rev1=1 crypto rx=0x7f8c6400b9e0 tx=0x7f8c6400bea0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-09T15:11:33.054 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:33.051+0000 7f8c5e7fc640 1 -- 172.21.15.43:0/2784254906 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8c6400caa0 con 0x7f8c6c143c20 2023-08-09T15:11:33.054 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:33.051+0000 7f8c5e7fc640 1 -- 172.21.15.43:0/2784254906 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8c6400cc00 con 0x7f8c6c143c20 2023-08-09T15:11:33.055 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:33.051+0000 7f8c5e7fc640 1 -- 172.21.15.43:0/2784254906 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8c64014440 con 0x7f8c6c143c20 2023-08-09T15:11:33.055 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:33.051+0000 7f8c73bab640 1 -- 172.21.15.43:0/2784254906 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8c6c0c67f0 con 0x7f8c6c143c20 2023-08-09T15:11:33.055 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:33.051+0000 7f8c73bab640 1 -- 172.21.15.43:0/2784254906 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8c6c0c5460 con 0x7f8c6c143c20 2023-08-09T15:11:33.055 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:33.052+0000 7f8c73bab640 1 -- 172.21.15.43:0/2784254906 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8c6c140110 con 0x7f8c6c143c20 2023-08-09T15:11:33.055 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:33.054+0000 7f8c5e7fc640 1 -- 172.21.15.43:0/2784254906 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 118) v1 ==== 88443+0+0 (secure 0 0 0) 0x7f8c6401d3b0 con 0x7f8c6c143c20 2023-08-09T15:11:33.056 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:33.055+0000 7f8c5e7fc640 1 --2- 172.21.15.43:0/2784254906 >> [v2:172.21.15.203:6832/4034093268,v1:172.21.15.203:6833/4034093268] conn(0x7f8c5006a170 0x7f8c5006c630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T15:11:33.056 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:33.055+0000 7f8c5e7fc640 1 -- 172.21.15.43:0/2784254906 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(223..223 src has 1..223) v4 ==== 6426+0+0 (secure 0 0 0) 0x7f8c64088580 con 0x7f8c6c143c20 2023-08-09T15:11:33.057 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:33.056+0000 7f8c71947640 1 --2- 172.21.15.43:0/2784254906 >> [v2:172.21.15.203:6832/4034093268,v1:172.21.15.203:6833/4034093268] conn(0x7f8c5006a170 0x7f8c5006c630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T15:11:33.057 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:33.057+0000 7f8c71947640 1 --2- 172.21.15.43:0/2784254906 >> [v2:172.21.15.203:6832/4034093268,v1:172.21.15.203:6833/4034093268] conn(0x7f8c5006a170 0x7f8c5006c630 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f8c6000a990 tx=0x7f8c60008040 comp rx=0 tx=0).ready entity=mgr.8833 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-09T15:11:33.060 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:33.060+0000 7f8c5e7fc640 1 -- 172.21.15.43:0/2784254906 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178066 (secure 0 0 0) 0x7f8c640539e0 con 0x7f8c6c143c20 2023-08-09T15:11:33.252 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:33.252+0000 7f8c73bab640 1 -- 172.21.15.43:0/2784254906 --> [v2:172.21.15.203:6832/4034093268,v1:172.21.15.203:6833/4034093268] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f8c6c06bdf0 con 0x7f8c5006a170 2023-08-09T15:11:33.282 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:33.282+0000 7f8c5e7fc640 1 -- 172.21.15.43:0/2784254906 <== mgr.8833 v2:172.21.15.203:6832/4034093268 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+308958 (secure 0 0 0) 0x7f8c6000a710 con 0x7f8c5006a170 2023-08-09T15:11:33.287 INFO:teuthology.orchestra.run.smithi043.stdout: 2023-08-09T15:11:33.289 INFO:teuthology.orchestra.run.smithi043.stderr:dumped all 2023-08-09T15:11:33.291 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:33.286+0000 7f8c73bab640 1 -- 172.21.15.43:0/2784254906 >> [v2:172.21.15.203:6832/4034093268,v1:172.21.15.203:6833/4034093268] conn(0x7f8c5006a170 msgr2=0x7f8c5006c630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T15:11:33.292 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:33.286+0000 7f8c73bab640 1 --2- 172.21.15.43:0/2784254906 >> [v2:172.21.15.203:6832/4034093268,v1:172.21.15.203:6833/4034093268] conn(0x7f8c5006a170 0x7f8c5006c630 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f8c6000a990 tx=0x7f8c60008040 comp rx=0 tx=0).stop 2023-08-09T15:11:33.292 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:33.286+0000 7f8c73bab640 1 -- 172.21.15.43:0/2784254906 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f8c6c143c20 msgr2=0x7f8c6c159a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T15:11:33.292 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:33.287+0000 7f8c73bab640 1 --2- 172.21.15.43:0/2784254906 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f8c6c143c20 0x7f8c6c159a60 secure :-1 s=READY pgs=2410 cs=0 l=1 rev1=1 crypto rx=0x7f8c6400b9e0 tx=0x7f8c6400bea0 comp rx=0 tx=0).stop 2023-08-09T15:11:33.293 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:33.287+0000 7f8c73bab640 1 -- 172.21.15.43:0/2784254906 shutdown_connections 2023-08-09T15:11:33.293 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:33.287+0000 7f8c73bab640 1 --2- 172.21.15.43:0/2784254906 >> [v2:172.21.15.203:6832/4034093268,v1:172.21.15.203:6833/4034093268] conn(0x7f8c5006a170 0x7f8c5006c630 unknown :-1 s=CLOSED pgs=64 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T15:11:33.293 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:33.287+0000 7f8c73bab640 1 --2- 172.21.15.43:0/2784254906 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f8c6c143c20 0x7f8c6c159a60 unknown :-1 s=CLOSED pgs=2410 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T15:11:33.294 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:33.287+0000 7f8c73bab640 1 --2- 172.21.15.43:0/2784254906 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f8c6c0c86f0 0x7f8c6c0c7800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T15:11:33.294 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:33.287+0000 7f8c73bab640 1 --2- 172.21.15.43:0/2784254906 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f8c6c06c8d0 0x7f8c6c0cad90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T15:11:33.294 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:33.287+0000 7f8c73bab640 1 -- 172.21.15.43:0/2784254906 >> 172.21.15.43:0/2784254906 conn(0x7f8c6c0ce0a0 msgr2=0x7f8c6c0af020 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-09T15:11:33.294 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:33.287+0000 7f8c73bab640 1 -- 172.21.15.43:0/2784254906 shutdown_connections 2023-08-09T15:11:33.295 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:33.287+0000 7f8c73bab640 1 -- 172.21.15.43:0/2784254906 wait complete. 2023-08-09T15:11:33.306 INFO:teuthology.orchestra.run.smithi043.stdout:{"pg_ready":true,"pg_map":{"version":27,"stamp":"2023-08-09T15:11:32.387382+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":23,"num_object_clones":0,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":0,"num_read":1963,"num_read_kb":4096,"num_write":587,"num_write_kb":2430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":367,"ondisk_log_size":367,"up":210,"acting":210,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":105,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":384252,"kb_used_data":22928,"kb_used_omap":13,"kb_used_meta":361202,"kb_avail":754590468,"statfs":{"total":773094113280,"available":772700639232,"internally_reserved":0,"allocated":23478272,"data_stored":13295800,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":14248,"internal_metadata":369870936},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":27,"apply_latency_ms":27,"commit_latency_ns":27000000,"apply_latency_ns":27000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"5.734891"},"pg_stats":[{"pgid":"27.3f","version":"0'0","reported_seq":9,"reported_epoch":222,"state":"active+clean","last_fresh":"2023-08-09T15:11:28.730392+0000","last_change":"2023-08-09T15:11:28.728482+0000","last_active":"2023-08-09T15:11:28.730392+0000","last_peered":"2023-08-09T15:11:28.730392+0000","last_clean":"2023-08-09T15:11:28.730392+0000","last_became_active":"2023-08-09T15:11:28.728034+0000","last_became_peered":"2023-08-09T15:11:28.728034+0000","last_unstale":"2023-08-09T15:11:28.730392+0000","last_undegraded":"2023-08-09T15:11:28.730392+0000","last_fullsized":"2023-08-09T15:11:28.730392+0000","mapping_epoch":221,"log_start":"0'0","ondisk_log_start":"0'0","created":221,"last_epoch_clean":222,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T15:11:27.650831+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T15:11:27.650831+0000","last_clean_scrub_stamp":"2023-08-09T15:11:27.650831+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-11T01:35:23.418645+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"27.3e","version":"0'0","reported_seq":9,"reported_epoch":222,"state":"active+clean","last_fresh":"2023-08-09T15:11:28.738466+0000","last_change":"2023-08-09T15:11:28.737777+0000","last_active":"2023-08-09T15:11:28.738466+0000","last_peered":"2023-08-09T15:11:28.738466+0000","last_clean":"2023-08-09T15:11:28.738466+0000","last_became_active":"2023-08-09T15:11:28.737460+0000","last_became_peered":"2023-08-09T15:11:28.737460+0000","last_unstale":"2023-08-09T15:11:28.738466+0000","last_undegraded":"2023-08-09T15:11:28.738466+0000","last_fullsized":"2023-08-09T15:11:28.738466+0000","mapping_epoch":221,"log_start":"0'0","ondisk_log_start":"0'0","created":221,"last_epoch_clean":222,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T15:11:27.650831+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T15:11:27.650831+0000","last_clean_scrub_stamp":"2023-08-09T15:11:27.650831+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T20:46:57.262034+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"27.3d","version":"0'0","reported_seq":9,"reported_epoch":222,"state":"active+clean","last_fresh":"2023-08-09T15:11:28.736836+0000","last_change":"2023-08-09T15:11:28.730121+0000","last_active":"2023-08-09T15:11:28.736836+0000","last_peered":"2023-08-09T15:11:28.736836+0000","last_clean":"2023-08-09T15:11:28.736836+0000","last_became_active":"2023-08-09T15:11:28.729853+0000","last_became_peered":"2023-08-09T15:11:28.729853+0000","last_unstale":"2023-08-09T15:11:28.736836+0000","last_undegraded":"2023-08-09T15:11:28.736836+0000","last_fullsized":"2023-08-09T15:11:28.736836+0000","mapping_epoch":221,"log_start":"0'0","ondisk_log_start":"0'0","created":221,"last_epoch_clean":222,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T15:11:27.650831+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T15:11:27.650831+0000","last_clean_scrub_stamp":"2023-08-09T15:11:27.650831+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T23:05:56.110390+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"27.3c","version":"0'0","reported_seq":9,"reported_epoch":222,"state":"active+clean","last_fresh":"2023-08-09T15:11:28.741424+0000","last_change":"2023-08-09T15:11:28.739877+0000","last_active":"2023-08-09T15:11:28.741424+0000","last_peered":"2023-08-09T15:11:28.741424+0000","last_clean":"2023-08-09T15:11:28.741424+0000","last_became_active":"2023-08-09T15:11:28.739598+0000","last_became_peered":"2023-08-09T15:11:28.739598+0000","last_unstale":"2023-08-09T15:11:28.741424+0000","last_undegraded":"2023-08-09T15:11:28.741424+0000","last_fullsized":"2023-08-09T15:11:28.741424+0000","mapping_epoch":221,"log_start":"0'0","ondisk_log_start":"0'0","created":221,"last_epoch_clean":222,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T15:11:27.650831+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T15:11:27.650831+0000","last_clean_scrub_stamp":"2023-08-09T15:11:27.650831+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T23:13:39.900734+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"27.3b","version":"0'0","reported_seq":9,"reported_epoch":222,"state":"active+clean","last_fresh":"2023-08-09T15:11:28.722482+0000","last_change":"2023-08-09T15:11:28.718442+0000","last_active":"2023-08-09T15:11:28.722482+0000","last_peered":"2023-08-09T15:11:28.722482+0000","last_clean":"2023-08-09T15:11:28.722482+0000","last_became_active":"2023-08-09T15:11:28.718118+0000","last_became_peered":"2023-08-09T15:11:28.718118+0000","last_unstale":"2023-08-09T15:11:28.722482+0000","last_undegraded":"2023-08-09T15:11:28.722482+0000","last_fullsized":"2023-08-09T15:11:28.722482+0000","mapping_epoch":221,"log_start":"0'0","ondisk_log_start":"0'0","created":221,"last_epoch_clean":222,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T15:11:27.650831+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T15:11:27.650831+0000","last_clean_scrub_stamp":"2023-08-09T15:11:27.650831+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T22:19:26.178328+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"27.3a","version":"0'0","reported_seq":9,"reported_epoch":222,"state":"active+clean","last_fresh":"2023-08-09T15:11:28.733344+0000","last_change":"2023-08-09T15:11:28.731519+0000","last_active":"2023-08-09T15:11:28.733344+0000","last_peered":"2023-08-09T15:11:28.733344+0000","last_clean":"2023-08-09T15:11:28.733344+0000","last_became_active":"2023-08-09T15:11:28.731160+0000","last_became_peered":"2023-08-09T15:11:28.731160+0000","last_unstale":"2023-08-09T15:11:28.733344+0000","last_undegraded":"2023-08-09T15:11:28.733344+0000","last_fullsized":"2023-08-09T15:11:28.733344+0000","mapping_epoch":221,"log_start":"0'0","ondisk_log_start":"0'0","created":221,"last_epoch_clean":222,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T15:11:27.650831+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T15:11:27.650831+0000","last_clean_scrub_stamp":"2023-08-09T15:11:27.650831+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T20:47:00.574569+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"27.39","version":"0'0","reported_seq":9,"reported_epoch":222,"state":"active+clean","last_fresh":"2023-08-09T15:11:28.740429+0000","last_change":"2023-08-09T15:11:28.739099+0000","last_active":"2023-08-09T15:11:28.740429+0000","last_peered":"2023-08-09T15:11:28.740429+0000","last_clean":"2023-08-09T15:11:28.740429+0000","last_became_active":"2023-08-09T15:11:28.738829+0000","last_became_peered":"2023-08-09T15:11:28.738829+0000","last_unstale":"2023-08-09T15:11:28.740429+0000","last_undegraded":"2023-08-09T15:11:28.740429+0000","last_fullsized":"2023-08-09T15:11:28.740429+0000","mapping_epoch":221,"log_start":"0'0","ondisk_log_start":"0'0","created":221,"last_epoch_clean":222,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T15:11:27.650831+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T15:11:27.650831+0000","last_clean_scrub_stamp":"2023-08-09T15:11:27.650831+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T23:49:16.381965+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"27.38","version":"0'0","reported_seq":9,"reported_epoch":222,"state":"active+clean","last_fresh":"2023-08-09T15:11:28.722731+0000","last_change":"2023-08-09T15:11:28.719958+0000","last_active":"2023-08-09T15:11:28.722731+0000","last_peered":"2023-08-09T15:11:28.722731+0000","last_clean":"2023-08-09T15:11:28.722731+0000","last_became_active":"2023-08-09T15:11:28.719703+0000","last_became_peered":"2023-08-09T15:11:28.719703+0000","last_unstale":"2023-08-09T15:11:28.722731+0000","last_undegraded":"2023-08-09T15:11:28.722731+0000","last_fullsized":"2023-08-09T15:11:28.722731+0000","mapping_epoch":221,"log_start":"0'0","ondisk_log_start":"0'0","created":221,"last_epoch_clean":222,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T15:11:27.650831+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T15:11:27.650831+0000","last_clean_scrub_stamp":"2023-08-09T15:11:27.650831+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T16:59:53.165135+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"27.37","version":"0'0","reported_seq":9,"reported_epoch":222,"state":"active+clean","last_fresh":"2023-08-09T15:11:28.731327+0000","last_change":"2023-08-09T15:11:28.730386+0000","last_active":"2023-08-09T15:11:28.731327+0000","last_peered":"2023-08-09T15:11:28.731327+0000","last_clean":"2023-08-09T15:11:28.731327+0000","last_became_active":"2023-08-09T15:11:28.730045+0000","last_became_peered":"2023-08-09T15:11:28.730045+0000","last_unstale":"2023-08-09T15:11:28.731327+0000","last_undegraded":"2023-08-09T15:11:28.731327+0000","last_fullsized":"2023-08-09T15:11:28.731327+0000","mapping_epoch":221,"log_start":"0'0","ondisk_log_start":"0'0","created":221,"last_epoch_clean":222,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T15:11:27.650831+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T15:11:27.650831+0000","last_clean_scrub_stamp":"2023-08-09T15:11:27.650831+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T16:44:13.514166+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"27.36","version":"0'0","reported_seq":9,"reported_epoch":222,"state":"active+clean","last_fresh":"2023-08-09T15:11:28.721194+0000","last_change":"2023-08-09T15:11:28.709161+0000","last_active":"2023-08-09T15:11:28.721194+0000","last_peered":"2023-08-09T15:11:28.721194+0000","last_clean":"2023-08-09T15:11:28.721194+0000","last_became_active":"2023-08-09T15:11:28.708761+0000","last_became_peered":"2023-08-09T15:11:28.708761+0000","last_unstale":"2023-08-09T15:11:28.721194+0000","last_undegraded":"2023-08-09T15:11:28.721194+0000","last_fullsized":"2023-08-09T15:11:28.721194+0000","mapping_epoch":221,"log_start":"0'0","ondisk_log_start":"0'0","created":221,"last_epoch_clean":222,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T15:11:27.650831+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T15:11:27.650831+0000","last_clean_scrub_stamp":"2023-08-09T15:11:27.650831+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T17:22:22.512775+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"27.35","version":"0'0","reported_seq":9,"reported_epoch":222,"state":"active+clean","last_fresh":"2023-08-09T15:11:28.738359+0000","last_change":"2023-08-09T15:11:28.736745+0000","last_active":"2023-08-09T15:11:28.738359+0000","last_peered":"2023-08-09T15:11:28.738359+0000","last_clean":"2023-08-09T15:11:28.738359+0000","last_became_active":"2023-08-09T15:11:28.736295+0000","last_became_peered":"2023-08-09T15:11:28.736295+0000","last_unstale":"2023-08-09T15:11:28.738359+0000","last_undegraded":"2023-08-09T15:11:28.738359+0000","last_fullsized":"2023-08-09T15:11:28.738359+0000","mapping_epoch":221,"log_start":"0'0","ondisk_log_start":"0'0","created":221,"last_epoch_clean":222,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T15:11:27.650831+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T15:11:27.650831+0000","last_clean_scrub_stamp":"2023-08-09T15:11:27.650831+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T16:40:16.167053+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"27.34","version":"0'0","reported_seq":9,"reported_epoch":222,"state":"active+clean","last_fresh":"2023-08-09T15:11:28.721496+0000","last_change":"2023-08-09T15:11:28.720577+0000","last_active":"2023-08-09T15:11:28.721496+0000","last_peered":"2023-08-09T15:11:28.721496+0000","last_clean":"2023-08-09T15:11:28.721496+0000","last_became_active":"2023-08-09T15:11:28.720287+0000","last_became_peered":"2023-08-09T15:11:28.720287+0000","last_unstale":"2023-08-09T15:11:28.721496+0000","last_undegraded":"2023-08-09T15:11:28.721496+0000","last_fullsized":"2023-08-09T15:11:28.721496+0000","mapping_epoch":221,"log_start":"0'0","ondisk_log_start":"0'0","created":221,"last_epoch_clean":222,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T15:11:27.650831+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T15:11:27.650831+0000","last_clean_scrub_stamp":"2023-08-09T15:11:27.650831+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T23:50:31.613421+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"27.33","version":"0'0","reported_seq":9,"reported_epoch":222,"state":"active+clean","last_fresh":"2023-08-09T15:11:28.735057+0000","last_change":"2023-08-09T15:11:28.733219+0000","last_active":"2023-08-09T15:11:28.735057+0000","last_peered":"2023-08-09T15:11:28.735057+0000","last_clean":"2023-08-09T15:11:28.735057+0000","last_became_active":"2023-08-09T15:11:28.732900+0000","last_became_peered":"2023-08-09T15:11:28.732900+0000","last_unstale":"2023-08-09T15:11:28.735057+0000","last_undegraded":"2023-08-09T15:11:28.735057+0000","last_fullsized":"2023-08-09T15:11:28.735057+0000","mapping_epoch":221,"log_start":"0'0","ondisk_log_start":"0'0","created":221,"last_epoch_clean":222,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T15:11:27.650831+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T15:11:27.650831+0000","last_clean_scrub_stamp":"2023-08-09T15:11:27.650831+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T23:08:13.925314+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"27.32","version":"0'0","reported_seq":9,"reported_epoch":222,"state":"active+clean","last_fresh":"2023-08-09T15:11:28.721116+0000","last_change":"2023-08-09T15:11:28.718885+0000","last_active":"2023-08-09T15:11:28.721116+0000","last_peered":"2023-08-09T15:11:28.721116+0000","last_clean":"2023-08-09T15:11:28.721116+0000","last_became_active":"2023-08-09T15:11:28.718534+0000","last_became_peered":"2023-08-09T15:11:28.718534+0000","last_unstale":"2023-08-09T15:11:28.721116+0000","last_undegraded":"2023-08-09T15:11:28.721116+0000","last_fullsized":"2023-08-09T15:11:28.721116+0000","mapping_epoch":221,"log_start":"0'0","ondisk_log_start":"0'0","created":221,"last_epoch_clean":222,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T15:11:27.650831+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T15:11:27.650831+0000","last_clean_scrub_stamp":"2023-08-09T15:11:27.650831+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T22:16:22.217326+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"27.31","version":"0'0","reported_seq":9,"reported_epoch":222,"state":"active+clean","last_fresh":"2023-08-09T15:11:28.729261+0000","last_change":"2023-08-09T15:11:28.726825+0000","last_active":"2023-08-09T15:11:28.729261+0000","last_peered":"2023-08-09T15:11:28.729261+0000","last_clean":"2023-08-09T15:11:28.729261+0000","last_became_active":"2023-08-09T15:11:28.726513+0000","last_became_peered":"2023-08-09T15:11:28.726513+0000","last_unstale":"2023-08-09T15:11:28.729261+0000","last_undegraded":"2023-08-09T15:11:28.729261+0000","last_fullsized":"2023-08-09T15:11:28.729261+0000","mapping_epoch":221,"log_start":"0'0","ondisk_log_start":"0'0","created":221,"last_epoch_clean":222,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T15:11:27.650831+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T15:11:27.650831+0000","last_clean_scrub_stamp":"2023-08-09T15:11:27.650831+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T23:43:15.374226+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"27.30","version":"0'0","reported_seq":9,"reported_epoch":222,"state":"active+clean","last_fresh":"2023-08-09T15:11:28.722268+0000","last_change":"2023-08-09T15:11:28.719171+0000","last_active":"2023-08-09T15:11:28.722268+0000","last_peered":"2023-08-09T15:11:28.722268+0000","last_clean":"2023-08-09T15:11:28.722268+0000","last_became_active":"2023-08-09T15:11:28.715862+0000","last_became_peered":"2023-08-09T15:11:28.715862+0000","last_unstale":"2023-08-09T15:11:28.722268+0000","last_undegraded":"2023-08-09T15:11:28.722268+0000","last_fullsized":"2023-08-09T15:11:28.722268+0000","mapping_epoch":221,"log_start":"0'0","ondisk_log_start":"0'0","created":221,"last_epoch_clean":222,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T15:11:27.650831+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T15:11:27.650831+0000","last_clean_scrub_stamp":"2023-08-09T15:11:27.650831+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-11T00:39:07.832326+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"27.2f","version":"0'0","reported_seq":9,"reported_epoch":222,"state":"active+clean","last_fresh":"2023-08-09T15:11:28.730362+0000","last_change":"2023-08-09T15:11:28.728297+0000","last_active":"2023-08-09T15:11:28.730362+0000","last_peered":"2023-08-09T15:11:28.730362+0000","last_clean":"2023-08-09T15:11:28.730362+0000","last_became_active":"2023-08-09T15:11:28.727641+0000","last_became_peered":"2023-08-09T15:11:28.727641+0000","last_unstale":"2023-08-09T15:11:28.730362+0000","last_undegraded":"2023-08-09T15:11:28.730362+0000","last_fullsized":"2023-08-09T15:11:28.730362+0000","mapping_epoch":221,"log_start":"0'0","ondisk_log_start":"0'0","created":221,"last_epoch_clean":222,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T15:11:27.650831+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T15:11:27.650831+0000","last_clean_scrub_stamp":"2023-08-09T15:11:27.650831+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T19:53:18.094118+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"27.2e","version":"0'0","reported_seq":9,"reported_epoch":222,"state":"active+clean","last_fresh":"2023-08-09T15:11:28.729832+0000","last_change":"2023-08-09T15:11:28.726570+0000","last_active":"2023-08-09T15:11:28.729832+0000","last_peered":"2023-08-09T15:11:28.729832+0000","last_clean":"2023-08-09T15:11:28.729832+0000","last_became_active":"2023-08-09T15:11:28.726283+0000","last_became_peered":"2023-08-09T15:11:28.726283+0000","last_unstale":"2023-08-09T15:11:28.729832+0000","last_undegraded":"2023-08-09T15:11:28.729832+0000","last_fullsized":"2023-08-09T15:11:28.729832+0000","mapping_epoch":221,"log_start":"0'0","ondisk_log_start":"0'0","created":221,"last_epoch_clean":222,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T15:11:27.650831+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T15:11:27.650831+0000","last_clean_scrub_stamp":"2023-08-09T15:11:27.650831+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-11T02:28:06.430438+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"27.2d","version":"0'0","reported_seq":9,"reported_epoch":222,"state":"active+clean","last_fresh":"2023-08-09T15:11:28.704837+0000","last_change":"2023-08-09T15:11:28.702907+0000","last_active":"2023-08-09T15:11:28.704837+0000","last_peered":"2023-08-09T15:11:28.704837+0000","last_clean":"2023-08-09T15:11:28.704837+0000","last_became_active":"2023-08-09T15:11:28.702602+0000","last_became_peered":"2023-08-09T15:11:28.702602+0000","last_unstale":"2023-08-09T15:11:28.704837+0000","last_undegraded":"2023-08-09T15:11:28.704837+0000","last_fullsized":"2023-08-09T15:11:28.704837+0000","mapping_epoch":221,"log_start":"0'0","ondisk_log_start":"0'0","created":221,"last_epoch_clean":222,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T15:11:27.650831+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T15:11:27.650831+0000","last_clean_scrub_stamp":"2023-08-09T15:11:27.650831+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T20:03:38.044588+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"27.2c","version":"0'0","reported_seq":9,"reported_epoch":222,"state":"active+clean","last_fresh":"2023-08-09T15:11:28.725409+0000","last_change":"2023-08-09T15:11:28.723326+0000","last_active":"2023-08-09T15:11:28.725409+0000","last_peered":"2023-08-09T15:11:28.725409+0000","last_clean":"2023-08-09T15:11:28.725409+0000","last_became_active":"2023-08-09T15:11:28.722852+0000","last_became_peered":"2023-08-09T15:11:28.722852+0000","last_unstale":"2023-08-09T15:11:28.725409+0000","last_undegraded":"2023-08-09T15:11:28.725409+0000","last_fullsized":"2023-08-09T15:11:28.725409+0000","mapping_epoch":221,"log_start":"0'0","ondisk_log_start":"0'0","created":221,"last_epoch_clean":222,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T15:11:27.650831+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T15:11:27.650831+0000","last_clean_scrub_stamp":"2023-08-09T15:11:27.650831+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T15:46:00.146591+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"27.2b","version":"0'0","reported_seq":9,"reported_epoch":222,"state":"active+clean","last_fresh":"2023-08-09T15:11:28.729328+0000","last_change":"2023-08-09T15:11:28.727762+0000","last_active":"2023-08-09T15:11:28.729328+0000","last_peered":"2023-08-09T15:11:28.729328+0000","last_clean":"2023-08-09T15:11:28.729328+0000","last_became_active":"2023-08-09T15:11:28.727489+0000","last_became_peered":"2023-08-09T15:11:28.727489+0000","last_unstale":"2023-08-09T15:11:28.729328+0000","last_undegraded":"2023-08-09T15:11:28.729328+0000","last_fullsized":"2023-08-09T15:11:28.729328+0000","mapping_epoch":221,"log_start":"0'0","ondisk_log_start":"0'0","created":221,"last_epoch_clean":222,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T15:11:27.650831+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T15:11:27.650831+0000","last_clean_scrub_stamp":"2023-08-09T15:11:27.650831+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-11T02:07:42.345848+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"27.2a","version":"0'0","reported_seq":9,"reported_epoch":222,"state":"active+clean","last_fresh":"2023-08-09T15:11:28.736601+0000","last_change":"2023-08-09T15:11:28.729939+0000","last_active":"2023-08-09T15:11:28.736601+0000","last_peered":"2023-08-09T15:11:28.736601+0000","last_clean":"2023-08-09T15:11:28.736601+0000","last_became_active":"2023-08-09T15:11:28.729668+0000","last_became_peered":"2023-08-09T15:11:28.729668+0000","last_unstale":"2023-08-09T15:11:28.736601+0000","last_undegraded":"2023-08-09T15:11:28.736601+0000","last_fullsized":"2023-08-09T15:11:28.736601+0000","mapping_epoch":221,"log_start":"0'0","ondisk_log_start":"0'0","created":221,"last_epoch_clean":222,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T15:11:27.650831+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T15:11:27.650831+0000","last_clean_scrub_stamp":"2023-08-09T15:11:27.650831+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T17:35:23.899405+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"27.29","version":"0'0","reported_seq":9,"reported_epoch":222,"state":"active+clean","last_fresh":"2023-08-09T15:11:28.746494+0000","last_change":"2023-08-09T15:11:28.744022+0000","last_active":"2023-08-09T15:11:28.746494+0000","last_peered":"2023-08-09T15:11:28.746494+0000","last_clean":"2023-08-09T15:11:28.746494+0000","last_became_active":"2023-08-09T15:11:28.741741+0000","last_became_peered":"2023-08-09T15:11:28.741741+0000","last_unstale":"2023-08-09T15:11:28.746494+0000","last_undegraded":"2023-08-09T15:11:28.746494+0000","last_fullsized":"2023-08-09T15:11:28.746494+0000","mapping_epoch":221,"log_start":"0'0","ondisk_log_start":"0'0","created":221,"last_epoch_clean":222,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T15:11:27.650831+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T15:11:27.650831+0000","last_clean_scrub_stamp":"2023-08-09T15:11:27.650831+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T15:19:42.337949+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"27.28","version":"0'0","reported_seq":9,"reported_epoch":222,"state":"active+clean","last_fresh":"2023-08-09T15:11:28.742052+0000","last_change":"2023-08-09T15:11:28.740832+0000","last_active":"2023-08-09T15:11:28.742052+0000","last_peered":"2023-08-09T15:11:28.742052+0000","last_clean":"2023-08-09T15:11:28.742052+0000","last_became_active":"2023-08-09T15:11:28.740545+0000","last_became_peered":"2023-08-09T15:11:28.740545+0000","last_unstale":"2023-08-09T15:11:28.742052+0000","last_undegraded":"2023-08-09T15:11:28.742052+0000","last_fullsized":"2023-08-09T15:11:28.742052+0000","mapping_epoch":221,"log_start":"0'0","ondisk_log_start":"0'0","created":221,"last_epoch_clean":222,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T15:11:27.650831+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T15:11:27.650831+0000","last_clean_scrub_stamp":"2023-08-09T15:11:27.650831+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T20:56:29.070500+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"27.27","version":"0'0","reported_seq":9,"reported_epoch":222,"state":"active+clean","last_fresh":"2023-08-09T15:11:28.729516+0000","last_change":"2023-08-09T15:11:28.727940+0000","last_active":"2023-08-09T15:11:28.729516+0000","last_peered":"2023-08-09T15:11:28.729516+0000","last_clean":"2023-08-09T15:11:28.729516+0000","last_became_active":"2023-08-09T15:11:28.727521+0000","last_became_peered":"2023-08-09T15:11:28.727521+0000","last_unstale":"2023-08-09T15:11:28.729516+0000","last_undegraded":"2023-08-09T15:11:28.729516+0000","last_fullsized":"2023-08-09T15:11:28.729516+0000","mapping_epoch":221,"log_start":"0'0","ondisk_log_start":"0'0","created":221,"last_epoch_clean":222,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T15:11:27.650831+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T15:11:27.650831+0000","last_clean_scrub_stamp":"2023-08-09T15:11:27.650831+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T19:51:20.128071+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"27.26","version":"0'0","reported_seq":9,"reported_epoch":222,"state":"active+clean","last_fresh":"2023-08-09T15:11:28.739982+0000","last_change":"2023-08-09T15:11:28.738901+0000","last_active":"2023-08-09T15:11:28.739982+0000","last_peered":"2023-08-09T15:11:28.739982+0000","last_clean":"2023-08-09T15:11:28.739982+0000","last_became_active":"2023-08-09T15:11:28.738620+0000","last_became_peered":"2023-08-09T15:11:28.738620+0000","last_unstale":"2023-08-09T15:11:28.739982+0000","last_undegraded":"2023-08-09T15:11:28.739982+0000","last_fullsized":"2023-08-09T15:11:28.739982+0000","mapping_epoch":221,"log_start":"0'0","ondisk_log_start":"0'0","created":221,"last_epoch_clean":222,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T15:11:27.650831+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T15:11:27.650831+0000","last_clean_scrub_stamp":"2023-08-09T15:11:27.650831+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T17:35:23.899405+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"27.25","version":"0'0","reported_seq":9,"reported_epoch":222,"state":"active+clean","last_fresh":"2023-08-09T15:11:28.718080+0000","last_change":"2023-08-09T15:11:28.717076+0000","last_active":"2023-08-09T15:11:28.718080+0000","last_peered":"2023-08-09T15:11:28.718080+0000","last_clean":"2023-08-09T15:11:28.718080+0000","last_became_active":"2023-08-09T15:11:28.716512+0000","last_became_peered":"2023-08-09T15:11:28.716512+0000","last_unstale":"2023-08-09T15:11:28.718080+0000","last_undegraded":"2023-08-09T15:11:28.718080+0000","last_fullsized":"2023-08-09T15:11:28.718080+0000","mapping_epoch":221,"log_start":"0'0","ondisk_log_start":"0'0","created":221,"last_epoch_clean":222,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T15:11:27.650831+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T15:11:27.650831+0000","last_clean_scrub_stamp":"2023-08-09T15:11:27.650831+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T19:55:13.844084+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"27.24","version":"0'0","reported_seq":9,"reported_epoch":222,"state":"active+clean","last_fresh":"2023-08-09T15:11:28.737389+0000","last_change":"2023-08-09T15:11:28.725049+0000","last_active":"2023-08-09T15:11:28.737389+0000","last_peered":"2023-08-09T15:11:28.737389+0000","last_clean":"2023-08-09T15:11:28.737389+0000","last_became_active":"2023-08-09T15:11:28.724712+0000","last_became_peered":"2023-08-09T15:11:28.724712+0000","last_unstale":"2023-08-09T15:11:28.737389+0000","last_undegraded":"2023-08-09T15:11:28.737389+0000","last_fullsized":"2023-08-09T15:11:28.737389+0000","mapping_epoch":221,"log_start":"0'0","ondisk_log_start":"0'0","created":221,"last_epoch_clean":222,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T15:11:27.650831+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T15:11:27.650831+0000","last_clean_scrub_stamp":"2023-08-09T15:11:27.650831+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T17:39:58.055854+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"27.23","version":"0'0","reported_seq":9,"reported_epoch":222,"state":"active+clean","last_fresh":"2023-08-09T15:11:28.723557+0000","last_change":"2023-08-09T15:11:28.721852+0000","last_active":"2023-08-09T15:11:28.723557+0000","last_peered":"2023-08-09T15:11:28.723557+0000","last_clean":"2023-08-09T15:11:28.723557+0000","last_became_active":"2023-08-09T15:11:28.721573+0000","last_became_peered":"2023-08-09T15:11:28.721573+0000","last_unstale":"2023-08-09T15:11:28.723557+0000","last_undegraded":"2023-08-09T15:11:28.723557+0000","last_fullsized":"2023-08-09T15:11:28.723557+0000","mapping_epoch":221,"log_start":"0'0","ondisk_log_start":"0'0","created":221,"last_epoch_clean":222,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T15:11:27.650831+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T15:11:27.650831+0000","last_clean_scrub_stamp":"2023-08-09T15:11:27.650831+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-11T01:56:21.129627+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"27.22","version":"0'0","reported_seq":9,"reported_epoch":222,"state":"active+clean","last_fresh":"2023-08-09T15:11:28.735269+0000","last_change":"2023-08-09T15:11:28.731453+0000","last_active":"2023-08-09T15:11:28.735269+0000","last_peered":"2023-08-09T15:11:28.735269+0000","last_clean":"2023-08-09T15:11:28.735269+0000","last_became_active":"2023-08-09T15:11:28.730958+0000","last_became_peered":"2023-08-09T15:11:28.730958+0000","last_unstale":"2023-08-09T15:11:28.735269+0000","last_undegraded":"2023-08-09T15:11:28.735269+0000","last_fullsized":"2023-08-09T15:11:28.735269+0000","mapping_epoch":221,"log_start":"0'0","ondisk_log_start":"0'0","created":221,"last_epoch_clean":222,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T15:11:27.650831+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T15:11:27.650831+0000","last_clean_scrub_stamp":"2023-08-09T15:11:27.650831+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-11T02:15:51.962022+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"27.21","version":"0'0","reported_seq":9,"reported_epoch":222,"state":"active+clean","last_fresh":"2023-08-09T15:11:28.730834+0000","last_change":"2023-08-09T15:11:28.727691+0000","last_active":"2023-08-09T15:11:28.730834+0000","last_peered":"2023-08-09T15:11:28.730834+0000","last_clean":"2023-08-09T15:11:28.730834+0000","last_became_active":"2023-08-09T15:11:28.727355+0000","last_became_peered":"2023-08-09T15:11:28.727355+0000","last_unstale":"2023-08-09T15:11:28.730834+0000","last_undegraded":"2023-08-09T15:11:28.730834+0000","last_fullsized":"2023-08-09T15:11:28.730834+0000","mapping_epoch":221,"log_start":"0'0","ondisk_log_start":"0'0","created":221,"last_epoch_clean":222,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T15:11:27.650831+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T15:11:27.650831+0000","last_clean_scrub_stamp":"2023-08-09T15:11:27.650831+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T15:35:58.727081+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"27.20","version":"0'0","reported_seq":9,"reported_epoch":222,"state":"active+clean","last_fresh":"2023-08-09T15:11:28.718939+0000","last_change":"2023-08-09T15:11:28.714950+0000","last_active":"2023-08-09T15:11:28.718939+0000","last_peered":"2023-08-09T15:11:28.718939+0000","last_clean":"2023-08-09T15:11:28.718939+0000","last_became_active":"2023-08-09T15:11:28.714526+0000","last_became_peered":"2023-08-09T15:11:28.714526+0000","last_unstale":"2023-08-09T15:11:28.718939+0000","last_undegraded":"2023-08-09T15:11:28.718939+0000","last_fullsized":"2023-08-09T15:11:28.718939+0000","mapping_epoch":221,"log_start":"0'0","ondisk_log_start":"0'0","created":221,"last_epoch_clean":222,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T15:11:27.650831+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T15:11:27.650831+0000","last_clean_scrub_stamp":"2023-08-09T15:11:27.650831+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T17:43:27.363007+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"27.16","version":"0'0","reported_seq":9,"reported_epoch":222,"state":"active+clean","last_fresh":"2023-08-09T15:11:28.734747+0000","last_change":"2023-08-09T15:11:28.732566+0000","last_active":"2023-08-09T15:11:28.734747+0000","last_peered":"2023-08-09T15:11:28.734747+0000","last_clean":"2023-08-09T15:11:28.734747+0000","last_became_active":"2023-08-09T15:11:28.731793+0000","last_became_peered":"2023-08-09T15:11:28.731793+0000","last_unstale":"2023-08-09T15:11:28.734747+0000","last_undegraded":"2023-08-09T15:11:28.734747+0000","last_fullsized":"2023-08-09T15:11:28.734747+0000","mapping_epoch":221,"log_start":"0'0","ondisk_log_start":"0'0","created":221,"last_epoch_clean":222,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T15:11:27.650831+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T15:11:27.650831+0000","last_clean_scrub_stamp":"2023-08-09T15:11:27.650831+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T15:48:40.328019+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"26.17","version":"0'0","reported_seq":11,"reported_epoch":222,"state":"active+clean","last_fresh":"2023-08-09T15:11:28.707232+0000","last_change":"2023-08-09T15:11:27.698829+0000","last_active":"2023-08-09T15:11:28.707232+0000","last_peered":"2023-08-09T15:11:28.707232+0000","last_clean":"2023-08-09T15:11:28.707232+0000","last_became_active":"2023-08-09T15:11:27.698559+0000","last_became_peered":"2023-08-09T15:11:27.698559+0000","last_unstale":"2023-08-09T15:11:28.707232+0000","last_undegraded":"2023-08-09T15:11:28.707232+0000","last_fullsized":"2023-08-09T15:11:28.707232+0000","mapping_epoch":220,"log_start":"0'0","ondisk_log_start":"0'0","created":220,"last_epoch_clean":221,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T15:11:26.638086+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T15:11:26.638086+0000","last_clean_scrub_stamp":"2023-08-09T15:11:26.638086+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T19:09:54.207782+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"27.17","version":"0'0","reported_seq":9,"reported_epoch":222,"state":"active+clean","last_fresh":"2023-08-09T15:11:28.735232+0000","last_change":"2023-08-09T15:11:28.723837+0000","last_active":"2023-08-09T15:11:28.735232+0000","last_peered":"2023-08-09T15:11:28.735232+0000","last_clean":"2023-08-09T15:11:28.735232+0000","last_became_active":"2023-08-09T15:11:28.723559+0000","last_became_peered":"2023-08-09T15:11:28.723559+0000","last_unstale":"2023-08-09T15:11:28.735232+0000","last_undegraded":"2023-08-09T15:11:28.735232+0000","last_fullsized":"2023-08-09T15:11:28.735232+0000","mapping_epoch":221,"log_start":"0'0","ondisk_log_start":"0'0","created":221,"last_epoch_clean":222,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T15:11:27.650831+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T15:11:27.650831+0000","last_clean_scrub_stamp":"2023-08-09T15:11:27.650831+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T18:52:30.153594+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"26.16","version":"0'0","reported_seq":11,"reported_epoch":222,"state":"active+clean","last_fresh":"2023-08-09T15:11:28.724239+0000","last_change":"2023-08-09T15:11:27.701082+0000","last_active":"2023-08-09T15:11:28.724239+0000","last_peered":"2023-08-09T15:11:28.724239+0000","last_clean":"2023-08-09T15:11:28.724239+0000","last_became_active":"2023-08-09T15:11:27.700749+0000","last_became_peered":"2023-08-09T15:11:27.700749+0000","last_unstale":"2023-08-09T15:11:28.724239+0000","last_undegraded":"2023-08-09T15:11:28.724239+0000","last_fullsized":"2023-08-09T15:11:28.724239+0000","mapping_epoch":220,"log_start":"0'0","ondisk_log_start":"0'0","created":220,"last_epoch_clean":221,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T15:11:26.638086+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T15:11:26.638086+0000","last_clean_scrub_stamp":"2023-08-09T15:11:26.638086+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T20:02:44.106120+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"27.14","version":"0'0","reported_seq":9,"reported_epoch":222,"state":"active+clean","last_fresh":"2023-08-09T15:11:28.729536+0000","last_change":"2023-08-09T15:11:28.728579+0000","last_active":"2023-08-09T15:11:28.729536+0000","last_peered":"2023-08-09T15:11:28.729536+0000","last_clean":"2023-08-09T15:11:28.729536+0000","last_became_active":"2023-08-09T15:11:28.728303+0000","last_became_peered":"2023-08-09T15:11:28.728303+0000","last_unstale":"2023-08-09T15:11:28.729536+0000","last_undegraded":"2023-08-09T15:11:28.729536+0000","last_fullsized":"2023-08-09T15:11:28.729536+0000","mapping_epoch":221,"log_start":"0'0","ondisk_log_start":"0'0","created":221,"last_epoch_clean":222,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T15:11:27.650831+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T15:11:27.650831+0000","last_clean_scrub_stamp":"2023-08-09T15:11:27.650831+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-11T00:01:24.436012+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"26.15","version":"222'3","reported_seq":14,"reported_epoch":222,"state":"active+clean","last_fresh":"2023-08-09T15:11:28.724700+0000","last_change":"2023-08-09T15:11:27.691311+0000","last_active":"2023-08-09T15:11:28.724700+0000","last_peered":"2023-08-09T15:11:28.724700+0000","last_clean":"2023-08-09T15:11:28.724700+0000","last_became_active":"2023-08-09T15:11:27.691027+0000","last_became_peered":"2023-08-09T15:11:27.691027+0000","last_unstale":"2023-08-09T15:11:28.724700+0000","last_undegraded":"2023-08-09T15:11:28.724700+0000","last_fullsized":"2023-08-09T15:11:28.724700+0000","mapping_epoch":220,"log_start":"0'0","ondisk_log_start":"0'0","created":220,"last_epoch_clean":221,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T15:11:26.638086+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T15:11:26.638086+0000","last_clean_scrub_stamp":"2023-08-09T15:11:26.638086+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T16:10:22.748982+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"27.15","version":"0'0","reported_seq":9,"reported_epoch":222,"state":"active+clean","last_fresh":"2023-08-09T15:11:28.728936+0000","last_change":"2023-08-09T15:11:28.725915+0000","last_active":"2023-08-09T15:11:28.728936+0000","last_peered":"2023-08-09T15:11:28.728936+0000","last_clean":"2023-08-09T15:11:28.728936+0000","last_became_active":"2023-08-09T15:11:28.725656+0000","last_became_peered":"2023-08-09T15:11:28.725656+0000","last_unstale":"2023-08-09T15:11:28.728936+0000","last_undegraded":"2023-08-09T15:11:28.728936+0000","last_fullsized":"2023-08-09T15:11:28.728936+0000","mapping_epoch":221,"log_start":"0'0","ondisk_log_start":"0'0","created":221,"last_epoch_clean":222,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T15:11:27.650831+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T15:11:27.650831+0000","last_clean_scrub_stamp":"2023-08-09T15:11:27.650831+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T20:03:51.144236+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"26.14","version":"222'6","reported_seq":17,"reported_epoch":222,"state":"active+clean","last_fresh":"2023-08-09T15:11:28.748422+0000","last_change":"2023-08-09T15:11:27.702153+0000","last_active":"2023-08-09T15:11:28.748422+0000","last_peered":"2023-08-09T15:11:28.748422+0000","last_clean":"2023-08-09T15:11:28.748422+0000","last_became_active":"2023-08-09T15:11:27.701805+0000","last_became_peered":"2023-08-09T15:11:27.701805+0000","last_unstale":"2023-08-09T15:11:28.748422+0000","last_undegraded":"2023-08-09T15:11:28.748422+0000","last_fullsized":"2023-08-09T15:11:28.748422+0000","mapping_epoch":220,"log_start":"0'0","ondisk_log_start":"0'0","created":220,"last_epoch_clean":221,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T15:11:26.638086+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T15:11:26.638086+0000","last_clean_scrub_stamp":"2023-08-09T15:11:26.638086+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-11T02:52:55.736990+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"27.12","version":"0'0","reported_seq":9,"reported_epoch":222,"state":"active+clean","last_fresh":"2023-08-09T15:11:28.722465+0000","last_change":"2023-08-09T15:11:28.717067+0000","last_active":"2023-08-09T15:11:28.722465+0000","last_peered":"2023-08-09T15:11:28.722465+0000","last_clean":"2023-08-09T15:11:28.722465+0000","last_became_active":"2023-08-09T15:11:28.716732+0000","last_became_peered":"2023-08-09T15:11:28.716732+0000","last_unstale":"2023-08-09T15:11:28.722465+0000","last_undegraded":"2023-08-09T15:11:28.722465+0000","last_fullsized":"2023-08-09T15:11:28.722465+0000","mapping_epoch":221,"log_start":"0'0","ondisk_log_start":"0'0","created":221,"last_epoch_clean":222,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T15:11:27.650831+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T15:11:27.650831+0000","last_clean_scrub_stamp":"2023-08-09T15:11:27.650831+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T22:09:46.565197+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"26.13","version":"222'2","reported_seq":13,"reported_epoch":222,"state":"active+clean","last_fresh":"2023-08-09T15:11:28.746371+0000","last_change":"2023-08-09T15:11:27.703741+0000","last_active":"2023-08-09T15:11:28.746371+0000","last_peered":"2023-08-09T15:11:28.746371+0000","last_clean":"2023-08-09T15:11:28.746371+0000","last_became_active":"2023-08-09T15:11:27.703366+0000","last_became_peered":"2023-08-09T15:11:27.703366+0000","last_unstale":"2023-08-09T15:11:28.746371+0000","last_undegraded":"2023-08-09T15:11:28.746371+0000","last_fullsized":"2023-08-09T15:11:28.746371+0000","mapping_epoch":220,"log_start":"0'0","ondisk_log_start":"0'0","created":220,"last_epoch_clean":221,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T15:11:26.638086+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T15:11:26.638086+0000","last_clean_scrub_stamp":"2023-08-09T15:11:26.638086+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T20:00:55.907532+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"27.13","version":"0'0","reported_seq":9,"reported_epoch":222,"state":"active+clean","last_fresh":"2023-08-09T15:11:28.720827+0000","last_change":"2023-08-09T15:11:28.717409+0000","last_active":"2023-08-09T15:11:28.720827+0000","last_peered":"2023-08-09T15:11:28.720827+0000","last_clean":"2023-08-09T15:11:28.720827+0000","last_became_active":"2023-08-09T15:11:28.715486+0000","last_became_peered":"2023-08-09T15:11:28.715486+0000","last_unstale":"2023-08-09T15:11:28.720827+0000","last_undegraded":"2023-08-09T15:11:28.720827+0000","last_fullsized":"2023-08-09T15:11:28.720827+0000","mapping_epoch":221,"log_start":"0'0","ondisk_log_start":"0'0","created":221,"last_epoch_clean":222,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T15:11:27.650831+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T15:11:27.650831+0000","last_clean_scrub_stamp":"2023-08-09T15:11:27.650831+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-11T02:40:53.891575+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"26.12","version":"0'0","reported_seq":11,"reported_epoch":222,"state":"active+clean","last_fresh":"2023-08-09T15:11:28.724826+0000","last_change":"2023-08-09T15:11:27.703826+0000","last_active":"2023-08-09T15:11:28.724826+0000","last_peered":"2023-08-09T15:11:28.724826+0000","last_clean":"2023-08-09T15:11:28.724826+0000","last_became_active":"2023-08-09T15:11:27.703447+0000","last_became_peered":"2023-08-09T15:11:27.703447+0000","last_unstale":"2023-08-09T15:11:28.724826+0000","last_undegraded":"2023-08-09T15:11:28.724826+0000","last_fullsized":"2023-08-09T15:11:28.724826+0000","mapping_epoch":220,"log_start":"0'0","ondisk_log_start":"0'0","created":220,"last_epoch_clean":221,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T15:11:26.638086+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T15:11:26.638086+0000","last_clean_scrub_stamp":"2023-08-09T15:11:26.638086+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-11T02:28:37.016859+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"27.10","version":"0'0","reported_seq":9,"reported_epoch":222,"state":"active+clean","last_fresh":"2023-08-09T15:11:28.736631+0000","last_change":"2023-08-09T15:11:28.725193+0000","last_active":"2023-08-09T15:11:28.736631+0000","last_peered":"2023-08-09T15:11:28.736631+0000","last_clean":"2023-08-09T15:11:28.736631+0000","last_became_active":"2023-08-09T15:11:28.724918+0000","last_became_peered":"2023-08-09T15:11:28.724918+0000","last_unstale":"2023-08-09T15:11:28.736631+0000","last_undegraded":"2023-08-09T15:11:28.736631+0000","last_fullsized":"2023-08-09T15:11:28.736631+0000","mapping_epoch":221,"log_start":"0'0","ondisk_log_start":"0'0","created":221,"last_epoch_clean":222,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T15:11:27.650831+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T15:11:27.650831+0000","last_clean_scrub_stamp":"2023-08-09T15:11:27.650831+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T16:16:24.924181+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"26.11","version":"0'0","reported_seq":11,"reported_epoch":222,"state":"active+clean","last_fresh":"2023-08-09T15:11:28.700891+0000","last_change":"2023-08-09T15:11:27.686506+0000","last_active":"2023-08-09T15:11:28.700891+0000","last_peered":"2023-08-09T15:11:28.700891+0000","last_clean":"2023-08-09T15:11:28.700891+0000","last_became_active":"2023-08-09T15:11:27.686193+0000","last_became_peered":"2023-08-09T15:11:27.686193+0000","last_unstale":"2023-08-09T15:11:28.700891+0000","last_undegraded":"2023-08-09T15:11:28.700891+0000","last_fullsized":"2023-08-09T15:11:28.700891+0000","mapping_epoch":220,"log_start":"0'0","ondisk_log_start":"0'0","created":220,"last_epoch_clean":221,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T15:11:26.638086+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T15:11:26.638086+0000","last_clean_scrub_stamp":"2023-08-09T15:11:26.638086+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T19:12:04.370705+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"27.11","version":"0'0","reported_seq":9,"reported_epoch":222,"state":"active+clean","last_fresh":"2023-08-09T15:11:28.721342+0000","last_change":"2023-08-09T15:11:28.717262+0000","last_active":"2023-08-09T15:11:28.721342+0000","last_peered":"2023-08-09T15:11:28.721342+0000","last_clean":"2023-08-09T15:11:28.721342+0000","last_became_active":"2023-08-09T15:11:28.716984+0000","last_became_peered":"2023-08-09T15:11:28.716984+0000","last_unstale":"2023-08-09T15:11:28.721342+0000","last_undegraded":"2023-08-09T15:11:28.721342+0000","last_fullsized":"2023-08-09T15:11:28.721342+0000","mapping_epoch":221,"log_start":"0'0","ondisk_log_start":"0'0","created":221,"last_epoch_clean":222,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T15:11:27.650831+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T15:11:27.650831+0000","last_clean_scrub_stamp":"2023-08-09T15:11:27.650831+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T21:46:15.531410+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"26.10","version":"222'8","reported_seq":19,"reported_epoch":222,"state":"active+clean","last_fresh":"2023-08-09T15:11:28.737844+0000","last_change":"2023-08-09T15:11:27.690582+0000","last_active":"2023-08-09T15:11:28.737844+0000","last_peered":"2023-08-09T15:11:28.737844+0000","last_clean":"2023-08-09T15:11:28.737844+0000","last_became_active":"2023-08-09T15:11:27.690233+0000","last_became_peered":"2023-08-09T15:11:27.690233+0000","last_unstale":"2023-08-09T15:11:28.737844+0000","last_undegraded":"2023-08-09T15:11:28.737844+0000","last_fullsized":"2023-08-09T15:11:28.737844+0000","mapping_epoch":220,"log_start":"0'0","ondisk_log_start":"0'0","created":220,"last_epoch_clean":221,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T15:11:26.638086+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T15:11:26.638086+0000","last_clean_scrub_stamp":"2023-08-09T15:11:26.638086+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T21:03:24.709891+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"27.e","version":"0'0","reported_seq":9,"reported_epoch":222,"state":"active+clean","last_fresh":"2023-08-09T15:11:28.735325+0000","last_change":"2023-08-09T15:11:28.730634+0000","last_active":"2023-08-09T15:11:28.735325+0000","last_peered":"2023-08-09T15:11:28.735325+0000","last_clean":"2023-08-09T15:11:28.735325+0000","last_became_active":"2023-08-09T15:11:28.730191+0000","last_became_peered":"2023-08-09T15:11:28.730191+0000","last_unstale":"2023-08-09T15:11:28.735325+0000","last_undegraded":"2023-08-09T15:11:28.735325+0000","last_fullsized":"2023-08-09T15:11:28.735325+0000","mapping_epoch":221,"log_start":"0'0","ondisk_log_start":"0'0","created":221,"last_epoch_clean":222,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T15:11:27.650831+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T15:11:27.650831+0000","last_clean_scrub_stamp":"2023-08-09T15:11:27.650831+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-11T01:17:45.162466+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"26.f","version":"222'1","reported_seq":12,"reported_epoch":222,"state":"active+clean","last_fresh":"2023-08-09T15:11:28.722856+0000","last_change":"2023-08-09T15:11:27.695677+0000","last_active":"2023-08-09T15:11:28.722856+0000","last_peered":"2023-08-09T15:11:28.722856+0000","last_clean":"2023-08-09T15:11:28.722856+0000","last_became_active":"2023-08-09T15:11:27.695371+0000","last_became_peered":"2023-08-09T15:11:27.695371+0000","last_unstale":"2023-08-09T15:11:28.722856+0000","last_undegraded":"2023-08-09T15:11:28.722856+0000","last_fullsized":"2023-08-09T15:11:28.722856+0000","mapping_epoch":220,"log_start":"0'0","ondisk_log_start":"0'0","created":220,"last_epoch_clean":221,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T15:11:26.638086+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T15:11:26.638086+0000","last_clean_scrub_stamp":"2023-08-09T15:11:26.638086+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T22:13:31.139984+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"27.f","version":"0'0","reported_seq":9,"reported_epoch":222,"state":"active+clean","last_fresh":"2023-08-09T15:11:28.729457+0000","last_change":"2023-08-09T15:11:28.727784+0000","last_active":"2023-08-09T15:11:28.729457+0000","last_peered":"2023-08-09T15:11:28.729457+0000","last_clean":"2023-08-09T15:11:28.729457+0000","last_became_active":"2023-08-09T15:11:28.727118+0000","last_became_peered":"2023-08-09T15:11:28.727118+0000","last_unstale":"2023-08-09T15:11:28.729457+0000","last_undegraded":"2023-08-09T15:11:28.729457+0000","last_fullsized":"2023-08-09T15:11:28.729457+0000","mapping_epoch":221,"log_start":"0'0","ondisk_log_start":"0'0","created":221,"last_epoch_clean":222,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T15:11:27.650831+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T15:11:27.650831+0000","last_clean_scrub_stamp":"2023-08-09T15:11:27.650831+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T22:31:02.501508+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"26.e","version":"222'2","reported_seq":13,"reported_epoch":222,"state":"active+clean","last_fresh":"2023-08-09T15:11:28.734137+0000","last_change":"2023-08-09T15:11:27.693614+0000","last_active":"2023-08-09T15:11:28.734137+0000","last_peered":"2023-08-09T15:11:28.734137+0000","last_clean":"2023-08-09T15:11:28.734137+0000","last_became_active":"2023-08-09T15:11:27.692947+0000","last_became_peered":"2023-08-09T15:11:27.692947+0000","last_unstale":"2023-08-09T15:11:28.734137+0000","last_undegraded":"2023-08-09T15:11:28.734137+0000","last_fullsized":"2023-08-09T15:11:28.734137+0000","mapping_epoch":220,"log_start":"0'0","ondisk_log_start":"0'0","created":220,"last_epoch_clean":221,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T15:11:26.638086+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T15:11:26.638086+0000","last_clean_scrub_stamp":"2023-08-09T15:11:26.638086+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T21:40:23.646378+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"27.c","version":"0'0","reported_seq":9,"reported_epoch":222,"state":"active+clean","last_fresh":"2023-08-09T15:11:28.737408+0000","last_change":"2023-08-09T15:11:28.733143+0000","last_active":"2023-08-09T15:11:28.737408+0000","last_peered":"2023-08-09T15:11:28.737408+0000","last_clean":"2023-08-09T15:11:28.737408+0000","last_became_active":"2023-08-09T15:11:28.732814+0000","last_became_peered":"2023-08-09T15:11:28.732814+0000","last_unstale":"2023-08-09T15:11:28.737408+0000","last_undegraded":"2023-08-09T15:11:28.737408+0000","last_fullsized":"2023-08-09T15:11:28.737408+0000","mapping_epoch":221,"log_start":"0'0","ondisk_log_start":"0'0","created":221,"last_epoch_clean":222,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T15:11:27.650831+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T15:11:27.650831+0000","last_clean_scrub_stamp":"2023-08-09T15:11:27.650831+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-11T01:30:42.299691+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"26.d","version":"222'1","reported_seq":12,"reported_epoch":222,"state":"active+clean","last_fresh":"2023-08-09T15:11:28.737171+0000","last_change":"2023-08-09T15:11:27.701548+0000","last_active":"2023-08-09T15:11:28.737171+0000","last_peered":"2023-08-09T15:11:28.737171+0000","last_clean":"2023-08-09T15:11:28.737171+0000","last_became_active":"2023-08-09T15:11:27.701234+0000","last_became_peered":"2023-08-09T15:11:27.701234+0000","last_unstale":"2023-08-09T15:11:28.737171+0000","last_undegraded":"2023-08-09T15:11:28.737171+0000","last_fullsized":"2023-08-09T15:11:28.737171+0000","mapping_epoch":220,"log_start":"0'0","ondisk_log_start":"0'0","created":220,"last_epoch_clean":221,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T15:11:26.638086+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T15:11:26.638086+0000","last_clean_scrub_stamp":"2023-08-09T15:11:26.638086+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T16:42:13.581769+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"27.d","version":"0'0","reported_seq":9,"reported_epoch":222,"state":"active+clean","last_fresh":"2023-08-09T15:11:28.733006+0000","last_change":"2023-08-09T15:11:28.731334+0000","last_active":"2023-08-09T15:11:28.733006+0000","last_peered":"2023-08-09T15:11:28.733006+0000","last_clean":"2023-08-09T15:11:28.733006+0000","last_became_active":"2023-08-09T15:11:28.730843+0000","last_became_peered":"2023-08-09T15:11:28.730843+0000","last_unstale":"2023-08-09T15:11:28.733006+0000","last_undegraded":"2023-08-09T15:11:28.733006+0000","last_fullsized":"2023-08-09T15:11:28.733006+0000","mapping_epoch":221,"log_start":"0'0","ondisk_log_start":"0'0","created":221,"last_epoch_clean":222,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T15:11:27.650831+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T15:11:27.650831+0000","last_clean_scrub_stamp":"2023-08-09T15:11:27.650831+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T15:50:57.838660+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"26.c","version":"0'0","reported_seq":11,"reported_epoch":222,"state":"active+clean","last_fresh":"2023-08-09T15:11:28.704245+0000","last_change":"2023-08-09T15:11:27.693369+0000","last_active":"2023-08-09T15:11:28.704245+0000","last_peered":"2023-08-09T15:11:28.704245+0000","last_clean":"2023-08-09T15:11:28.704245+0000","last_became_active":"2023-08-09T15:11:27.692360+0000","last_became_peered":"2023-08-09T15:11:27.692360+0000","last_unstale":"2023-08-09T15:11:28.704245+0000","last_undegraded":"2023-08-09T15:11:28.704245+0000","last_fullsized":"2023-08-09T15:11:28.704245+0000","mapping_epoch":220,"log_start":"0'0","ondisk_log_start":"0'0","created":220,"last_epoch_clean":221,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T15:11:26.638086+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T15:11:26.638086+0000","last_clean_scrub_stamp":"2023-08-09T15:11:26.638086+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T18:17:59.659054+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"27.a","version":"0'0","reported_seq":9,"reported_epoch":222,"state":"active+clean","last_fresh":"2023-08-09T15:11:28.719326+0000","last_change":"2023-08-09T15:11:28.714046+0000","last_active":"2023-08-09T15:11:28.719326+0000","last_peered":"2023-08-09T15:11:28.719326+0000","last_clean":"2023-08-09T15:11:28.719326+0000","last_became_active":"2023-08-09T15:11:28.713722+0000","last_became_peered":"2023-08-09T15:11:28.713722+0000","last_unstale":"2023-08-09T15:11:28.719326+0000","last_undegraded":"2023-08-09T15:11:28.719326+0000","last_fullsized":"2023-08-09T15:11:28.719326+0000","mapping_epoch":221,"log_start":"0'0","ondisk_log_start":"0'0","created":221,"last_epoch_clean":222,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T15:11:27.650831+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T15:11:27.650831+0000","last_clean_scrub_stamp":"2023-08-09T15:11:27.650831+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T20:21:40.252700+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"26.b","version":"222'2","reported_seq":13,"reported_epoch":222,"state":"active+clean","last_fresh":"2023-08-09T15:11:28.725371+0000","last_change":"2023-08-09T15:11:27.685327+0000","last_active":"2023-08-09T15:11:28.725371+0000","last_peered":"2023-08-09T15:11:28.725371+0000","last_clean":"2023-08-09T15:11:28.725371+0000","last_became_active":"2023-08-09T15:11:27.685042+0000","last_became_peered":"2023-08-09T15:11:27.685042+0000","last_unstale":"2023-08-09T15:11:28.725371+0000","last_undegraded":"2023-08-09T15:11:28.725371+0000","last_fullsized":"2023-08-09T15:11:28.725371+0000","mapping_epoch":220,"log_start":"0'0","ondisk_log_start":"0'0","created":220,"last_epoch_clean":221,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T15:11:26.638086+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T15:11:26.638086+0000","last_clean_scrub_stamp":"2023-08-09T15:11:26.638086+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T16:44:31.400957+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"27.b","version":"0'0","reported_seq":9,"reported_epoch":222,"state":"active+clean","last_fresh":"2023-08-09T15:11:28.724202+0000","last_change":"2023-08-09T15:11:28.717915+0000","last_active":"2023-08-09T15:11:28.724202+0000","last_peered":"2023-08-09T15:11:28.724202+0000","last_clean":"2023-08-09T15:11:28.724202+0000","last_became_active":"2023-08-09T15:11:28.717606+0000","last_became_peered":"2023-08-09T15:11:28.717606+0000","last_unstale":"2023-08-09T15:11:28.724202+0000","last_undegraded":"2023-08-09T15:11:28.724202+0000","last_fullsized":"2023-08-09T15:11:28.724202+0000","mapping_epoch":221,"log_start":"0'0","ondisk_log_start":"0'0","created":221,"last_epoch_clean":222,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T15:11:27.650831+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T15:11:27.650831+0000","last_clean_scrub_stamp":"2023-08-09T15:11:27.650831+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T23:43:01.111359+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"26.a","version":"0'0","reported_seq":11,"reported_epoch":222,"state":"active+clean","last_fresh":"2023-08-09T15:11:28.697813+0000","last_change":"2023-08-09T15:11:27.693720+0000","last_active":"2023-08-09T15:11:28.697813+0000","last_peered":"2023-08-09T15:11:28.697813+0000","last_clean":"2023-08-09T15:11:28.697813+0000","last_became_active":"2023-08-09T15:11:27.693401+0000","last_became_peered":"2023-08-09T15:11:27.693401+0000","last_unstale":"2023-08-09T15:11:28.697813+0000","last_undegraded":"2023-08-09T15:11:28.697813+0000","last_fullsized":"2023-08-09T15:11:28.697813+0000","mapping_epoch":220,"log_start":"0'0","ondisk_log_start":"0'0","created":220,"last_epoch_clean":221,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T15:11:26.638086+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T15:11:26.638086+0000","last_clean_scrub_stamp":"2023-08-09T15:11:26.638086+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T23:01:04.684315+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"27.8","version":"0'0","reported_seq":9,"reported_epoch":222,"state":"active+clean","last_fresh":"2023-08-09T15:11:28.738386+0000","last_change":"2023-08-09T15:11:28.737655+0000","last_active":"2023-08-09T15:11:28.738386+0000","last_peered":"2023-08-09T15:11:28.738386+0000","last_clean":"2023-08-09T15:11:28.738386+0000","last_became_active":"2023-08-09T15:11:28.737354+0000","last_became_peered":"2023-08-09T15:11:28.737354+0000","last_unstale":"2023-08-09T15:11:28.738386+0000","last_undegraded":"2023-08-09T15:11:28.738386+0000","last_fullsized":"2023-08-09T15:11:28.738386+0000","mapping_epoch":221,"log_start":"0'0","ondisk_log_start":"0'0","created":221,"last_epoch_clean":222,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T15:11:27.650831+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T15:11:27.650831+0000","last_clean_scrub_stamp":"2023-08-09T15:11:27.650831+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T17:40:36.100186+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"26.9","version":"0'0","reported_seq":11,"reported_epoch":222,"state":"active+clean","last_fresh":"2023-08-09T15:11:28.696909+0000","last_change":"2023-08-09T15:11:27.701663+0000","last_active":"2023-08-09T15:11:28.696909+0000","last_peered":"2023-08-09T15:11:28.696909+0000","last_clean":"2023-08-09T15:11:28.696909+0000","last_became_active":"2023-08-09T15:11:27.701336+0000","last_became_peered":"2023-08-09T15:11:27.701336+0000","last_unstale":"2023-08-09T15:11:28.696909+0000","last_undegraded":"2023-08-09T15:11:28.696909+0000","last_fullsized":"2023-08-09T15:11:28.696909+0000","mapping_epoch":220,"log_start":"0'0","ondisk_log_start":"0'0","created":220,"last_epoch_clean":221,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T15:11:26.638086+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T15:11:26.638086+0000","last_clean_scrub_stamp":"2023-08-09T15:11:26.638086+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T19:53:59.772090+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"27.9","version":"0'0","reported_seq":9,"reported_epoch":222,"state":"active+clean","last_fresh":"2023-08-09T15:11:28.745475+0000","last_change":"2023-08-09T15:11:28.741869+0000","last_active":"2023-08-09T15:11:28.745475+0000","last_peered":"2023-08-09T15:11:28.745475+0000","last_clean":"2023-08-09T15:11:28.745475+0000","last_became_active":"2023-08-09T15:11:28.737075+0000","last_became_peered":"2023-08-09T15:11:28.737075+0000","last_unstale":"2023-08-09T15:11:28.745475+0000","last_undegraded":"2023-08-09T15:11:28.745475+0000","last_fullsized":"2023-08-09T15:11:28.745475+0000","mapping_epoch":221,"log_start":"0'0","ondisk_log_start":"0'0","created":221,"last_epoch_clean":222,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T15:11:27.650831+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T15:11:27.650831+0000","last_clean_scrub_stamp":"2023-08-09T15:11:27.650831+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T15:37:59.332944+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"26.8","version":"0'0","reported_seq":11,"reported_epoch":222,"state":"active+clean","last_fresh":"2023-08-09T15:11:28.708652+0000","last_change":"2023-08-09T15:11:27.696329+0000","last_active":"2023-08-09T15:11:28.708652+0000","last_peered":"2023-08-09T15:11:28.708652+0000","last_clean":"2023-08-09T15:11:28.708652+0000","last_became_active":"2023-08-09T15:11:27.696043+0000","last_became_peered":"2023-08-09T15:11:27.696043+0000","last_unstale":"2023-08-09T15:11:28.708652+0000","last_undegraded":"2023-08-09T15:11:28.708652+0000","last_fullsized":"2023-08-09T15:11:28.708652+0000","mapping_epoch":220,"log_start":"0'0","ondisk_log_start":"0'0","created":220,"last_epoch_clean":221,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T15:11:26.638086+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T15:11:26.638086+0000","last_clean_scrub_stamp":"2023-08-09T15:11:26.638086+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T20:30:44.755082+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"27.6","version":"0'0","reported_seq":9,"reported_epoch":222,"state":"active+clean","last_fresh":"2023-08-09T15:11:28.723604+0000","last_change":"2023-08-09T15:11:28.721153+0000","last_active":"2023-08-09T15:11:28.723604+0000","last_peered":"2023-08-09T15:11:28.723604+0000","last_clean":"2023-08-09T15:11:28.723604+0000","last_became_active":"2023-08-09T15:11:28.720875+0000","last_became_peered":"2023-08-09T15:11:28.720875+0000","last_unstale":"2023-08-09T15:11:28.723604+0000","last_undegraded":"2023-08-09T15:11:28.723604+0000","last_fullsized":"2023-08-09T15:11:28.723604+0000","mapping_epoch":221,"log_start":"0'0","ondisk_log_start":"0'0","created":221,"last_epoch_clean":222,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T15:11:27.650831+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T15:11:27.650831+0000","last_clean_scrub_stamp":"2023-08-09T15:11:27.650831+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-11T00:24:33.663742+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"26.7","version":"222'2","reported_seq":13,"reported_epoch":222,"state":"active+clean","last_fresh":"2023-08-09T15:11:28.737068+0000","last_change":"2023-08-09T15:11:27.695871+0000","last_active":"2023-08-09T15:11:28.737068+0000","last_peered":"2023-08-09T15:11:28.737068+0000","last_clean":"2023-08-09T15:11:28.737068+0000","last_became_active":"2023-08-09T15:11:27.695549+0000","last_became_peered":"2023-08-09T15:11:27.695549+0000","last_unstale":"2023-08-09T15:11:28.737068+0000","last_undegraded":"2023-08-09T15:11:28.737068+0000","last_fullsized":"2023-08-09T15:11:28.737068+0000","mapping_epoch":220,"log_start":"0'0","ondisk_log_start":"0'0","created":220,"last_epoch_clean":221,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T15:11:26.638086+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T15:11:26.638086+0000","last_clean_scrub_stamp":"2023-08-09T15:11:26.638086+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-11T01:51:31.087330+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"27.7","version":"0'0","reported_seq":9,"reported_epoch":222,"state":"active+clean","last_fresh":"2023-08-09T15:11:28.721151+0000","last_change":"2023-08-09T15:11:28.716986+0000","last_active":"2023-08-09T15:11:28.721151+0000","last_peered":"2023-08-09T15:11:28.721151+0000","last_clean":"2023-08-09T15:11:28.721151+0000","last_became_active":"2023-08-09T15:11:28.716684+0000","last_became_peered":"2023-08-09T15:11:28.716684+0000","last_unstale":"2023-08-09T15:11:28.721151+0000","last_undegraded":"2023-08-09T15:11:28.721151+0000","last_fullsized":"2023-08-09T15:11:28.721151+0000","mapping_epoch":221,"log_start":"0'0","ondisk_log_start":"0'0","created":221,"last_epoch_clean":222,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T15:11:27.650831+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T15:11:27.650831+0000","last_clean_scrub_stamp":"2023-08-09T15:11:27.650831+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-11T00:59:54.696173+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"26.6","version":"0'0","reported_seq":11,"reported_epoch":222,"state":"active+clean","last_fresh":"2023-08-09T15:11:28.722837+0000","last_change":"2023-08-09T15:11:27.688523+0000","last_active":"2023-08-09T15:11:28.722837+0000","last_peered":"2023-08-09T15:11:28.722837+0000","last_clean":"2023-08-09T15:11:28.722837+0000","last_became_active":"2023-08-09T15:11:27.688249+0000","last_became_peered":"2023-08-09T15:11:27.688249+0000","last_unstale":"2023-08-09T15:11:28.722837+0000","last_undegraded":"2023-08-09T15:11:28.722837+0000","last_fullsized":"2023-08-09T15:11:28.722837+0000","mapping_epoch":220,"log_start":"0'0","ondisk_log_start":"0'0","created":220,"last_epoch_clean":221,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T15:11:26.638086+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T15:11:26.638086+0000","last_clean_scrub_stamp":"2023-08-09T15:11:26.638086+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T16:31:57.538963+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"27.4","version":"0'0","reported_seq":9,"reported_epoch":222,"state":"active+clean","last_fresh":"2023-08-09T15:11:28.729248+0000","last_change":"2023-08-09T15:11:28.726566+0000","last_active":"2023-08-09T15:11:28.729248+0000","last_peered":"2023-08-09T15:11:28.729248+0000","last_clean":"2023-08-09T15:11:28.729248+0000","last_became_active":"2023-08-09T15:11:28.726268+0000","last_became_peered":"2023-08-09T15:11:28.726268+0000","last_unstale":"2023-08-09T15:11:28.729248+0000","last_undegraded":"2023-08-09T15:11:28.729248+0000","last_fullsized":"2023-08-09T15:11:28.729248+0000","mapping_epoch":221,"log_start":"0'0","ondisk_log_start":"0'0","created":221,"last_epoch_clean":222,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T15:11:27.650831+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T15:11:27.650831+0000","last_clean_scrub_stamp":"2023-08-09T15:11:27.650831+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T17:59:31.034062+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"26.5","version":"0'0","reported_seq":11,"reported_epoch":222,"state":"active+clean","last_fresh":"2023-08-09T15:11:28.707003+0000","last_change":"2023-08-09T15:11:27.697255+0000","last_active":"2023-08-09T15:11:28.707003+0000","last_peered":"2023-08-09T15:11:28.707003+0000","last_clean":"2023-08-09T15:11:28.707003+0000","last_became_active":"2023-08-09T15:11:27.696966+0000","last_became_peered":"2023-08-09T15:11:27.696966+0000","last_unstale":"2023-08-09T15:11:28.707003+0000","last_undegraded":"2023-08-09T15:11:28.707003+0000","last_fullsized":"2023-08-09T15:11:28.707003+0000","mapping_epoch":220,"log_start":"0'0","ondisk_log_start":"0'0","created":220,"last_epoch_clean":221,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T15:11:26.638086+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T15:11:26.638086+0000","last_clean_scrub_stamp":"2023-08-09T15:11:26.638086+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T18:42:35.227494+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"27.5","version":"0'0","reported_seq":9,"reported_epoch":222,"state":"active+clean","last_fresh":"2023-08-09T15:11:28.745387+0000","last_change":"2023-08-09T15:11:28.741728+0000","last_active":"2023-08-09T15:11:28.745387+0000","last_peered":"2023-08-09T15:11:28.745387+0000","last_clean":"2023-08-09T15:11:28.745387+0000","last_became_active":"2023-08-09T15:11:28.737109+0000","last_became_peered":"2023-08-09T15:11:28.737109+0000","last_unstale":"2023-08-09T15:11:28.745387+0000","last_undegraded":"2023-08-09T15:11:28.745387+0000","last_fullsized":"2023-08-09T15:11:28.745387+0000","mapping_epoch":221,"log_start":"0'0","ondisk_log_start":"0'0","created":221,"last_epoch_clean":222,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T15:11:27.650831+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T15:11:27.650831+0000","last_clean_scrub_stamp":"2023-08-09T15:11:27.650831+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-11T02:53:02.242201+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"26.4","version":"0'0","reported_seq":11,"reported_epoch":222,"state":"active+clean","last_fresh":"2023-08-09T15:11:28.704395+0000","last_change":"2023-08-09T15:11:27.696570+0000","last_active":"2023-08-09T15:11:28.704395+0000","last_peered":"2023-08-09T15:11:28.704395+0000","last_clean":"2023-08-09T15:11:28.704395+0000","last_became_active":"2023-08-09T15:11:27.696288+0000","last_became_peered":"2023-08-09T15:11:27.696288+0000","last_unstale":"2023-08-09T15:11:28.704395+0000","last_undegraded":"2023-08-09T15:11:28.704395+0000","last_fullsized":"2023-08-09T15:11:28.704395+0000","mapping_epoch":220,"log_start":"0'0","ondisk_log_start":"0'0","created":220,"last_epoch_clean":221,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T15:11:26.638086+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T15:11:26.638086+0000","last_clean_scrub_stamp":"2023-08-09T15:11:26.638086+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T15:39:52.536660+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"27.2","version":"0'0","reported_seq":9,"reported_epoch":222,"state":"active+clean","last_fresh":"2023-08-09T15:11:28.740533+0000","last_change":"2023-08-09T15:11:28.739535+0000","last_active":"2023-08-09T15:11:28.740533+0000","last_peered":"2023-08-09T15:11:28.740533+0000","last_clean":"2023-08-09T15:11:28.740533+0000","last_became_active":"2023-08-09T15:11:28.739236+0000","last_became_peered":"2023-08-09T15:11:28.739236+0000","last_unstale":"2023-08-09T15:11:28.740533+0000","last_undegraded":"2023-08-09T15:11:28.740533+0000","last_fullsized":"2023-08-09T15:11:28.740533+0000","mapping_epoch":221,"log_start":"0'0","ondisk_log_start":"0'0","created":221,"last_epoch_clean":222,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T15:11:27.650831+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T15:11:27.650831+0000","last_clean_scrub_stamp":"2023-08-09T15:11:27.650831+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T17:17:17.167017+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"26.3","version":"0'0","reported_seq":11,"reported_epoch":222,"state":"active+clean","last_fresh":"2023-08-09T15:11:28.719799+0000","last_change":"2023-08-09T15:11:27.694233+0000","last_active":"2023-08-09T15:11:28.719799+0000","last_peered":"2023-08-09T15:11:28.719799+0000","last_clean":"2023-08-09T15:11:28.719799+0000","last_became_active":"2023-08-09T15:11:27.693874+0000","last_became_peered":"2023-08-09T15:11:27.693874+0000","last_unstale":"2023-08-09T15:11:28.719799+0000","last_undegraded":"2023-08-09T15:11:28.719799+0000","last_fullsized":"2023-08-09T15:11:28.719799+0000","mapping_epoch":220,"log_start":"0'0","ondisk_log_start":"0'0","created":220,"last_epoch_clean":221,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T15:11:26.638086+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T15:11:26.638086+0000","last_clean_scrub_stamp":"2023-08-09T15:11:26.638086+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T16:21:38.029689+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"27.3","version":"0'0","reported_seq":9,"reported_epoch":222,"state":"active+clean","last_fresh":"2023-08-09T15:11:28.734034+0000","last_change":"2023-08-09T15:11:28.731848+0000","last_active":"2023-08-09T15:11:28.734034+0000","last_peered":"2023-08-09T15:11:28.734034+0000","last_clean":"2023-08-09T15:11:28.734034+0000","last_became_active":"2023-08-09T15:11:28.731560+0000","last_became_peered":"2023-08-09T15:11:28.731560+0000","last_unstale":"2023-08-09T15:11:28.734034+0000","last_undegraded":"2023-08-09T15:11:28.734034+0000","last_fullsized":"2023-08-09T15:11:28.734034+0000","mapping_epoch":221,"log_start":"0'0","ondisk_log_start":"0'0","created":221,"last_epoch_clean":222,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T15:11:27.650831+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T15:11:27.650831+0000","last_clean_scrub_stamp":"2023-08-09T15:11:27.650831+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T17:43:40.889407+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"26.2","version":"0'0","reported_seq":11,"reported_epoch":222,"state":"active+clean","last_fresh":"2023-08-09T15:11:28.709953+0000","last_change":"2023-08-09T15:11:27.696309+0000","last_active":"2023-08-09T15:11:28.709953+0000","last_peered":"2023-08-09T15:11:28.709953+0000","last_clean":"2023-08-09T15:11:28.709953+0000","last_became_active":"2023-08-09T15:11:27.696020+0000","last_became_peered":"2023-08-09T15:11:27.696020+0000","last_unstale":"2023-08-09T15:11:28.709953+0000","last_undegraded":"2023-08-09T15:11:28.709953+0000","last_fullsized":"2023-08-09T15:11:28.709953+0000","mapping_epoch":220,"log_start":"0'0","ondisk_log_start":"0'0","created":220,"last_epoch_clean":221,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T15:11:26.638086+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T15:11:26.638086+0000","last_clean_scrub_stamp":"2023-08-09T15:11:26.638086+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T16:42:30.861840+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"27.0","version":"0'0","reported_seq":9,"reported_epoch":222,"state":"active+clean","last_fresh":"2023-08-09T15:11:28.738646+0000","last_change":"2023-08-09T15:11:28.736640+0000","last_active":"2023-08-09T15:11:28.738646+0000","last_peered":"2023-08-09T15:11:28.738646+0000","last_clean":"2023-08-09T15:11:28.738646+0000","last_became_active":"2023-08-09T15:11:28.736266+0000","last_became_peered":"2023-08-09T15:11:28.736266+0000","last_unstale":"2023-08-09T15:11:28.738646+0000","last_undegraded":"2023-08-09T15:11:28.738646+0000","last_fullsized":"2023-08-09T15:11:28.738646+0000","mapping_epoch":221,"log_start":"0'0","ondisk_log_start":"0'0","created":221,"last_epoch_clean":222,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T15:11:27.650831+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T15:11:27.650831+0000","last_clean_scrub_stamp":"2023-08-09T15:11:27.650831+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T20:23:24.264548+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"26.1","version":"0'0","reported_seq":11,"reported_epoch":222,"state":"active+clean","last_fresh":"2023-08-09T15:11:28.724556+0000","last_change":"2023-08-09T15:11:27.700679+0000","last_active":"2023-08-09T15:11:28.724556+0000","last_peered":"2023-08-09T15:11:28.724556+0000","last_clean":"2023-08-09T15:11:28.724556+0000","last_became_active":"2023-08-09T15:11:27.700284+0000","last_became_peered":"2023-08-09T15:11:27.700284+0000","last_unstale":"2023-08-09T15:11:28.724556+0000","last_undegraded":"2023-08-09T15:11:28.724556+0000","last_fullsized":"2023-08-09T15:11:28.724556+0000","mapping_epoch":220,"log_start":"0'0","ondisk_log_start":"0'0","created":220,"last_epoch_clean":221,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T15:11:26.638086+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T15:11:26.638086+0000","last_clean_scrub_stamp":"2023-08-09T15:11:26.638086+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T20:11:14.995122+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"27.1","version":"0'0","reported_seq":9,"reported_epoch":222,"state":"active+clean","last_fresh":"2023-08-09T15:11:28.735285+0000","last_change":"2023-08-09T15:11:28.731250+0000","last_active":"2023-08-09T15:11:28.735285+0000","last_peered":"2023-08-09T15:11:28.735285+0000","last_clean":"2023-08-09T15:11:28.735285+0000","last_became_active":"2023-08-09T15:11:28.730955+0000","last_became_peered":"2023-08-09T15:11:28.730955+0000","last_unstale":"2023-08-09T15:11:28.735285+0000","last_undegraded":"2023-08-09T15:11:28.735285+0000","last_fullsized":"2023-08-09T15:11:28.735285+0000","mapping_epoch":221,"log_start":"0'0","ondisk_log_start":"0'0","created":221,"last_epoch_clean":222,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T15:11:27.650831+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T15:11:27.650831+0000","last_clean_scrub_stamp":"2023-08-09T15:11:27.650831+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T17:34:00.028606+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"26.0","version":"0'0","reported_seq":11,"reported_epoch":222,"state":"active+clean","last_fresh":"2023-08-09T15:11:28.713270+0000","last_change":"2023-08-09T15:11:27.691790+0000","last_active":"2023-08-09T15:11:28.713270+0000","last_peered":"2023-08-09T15:11:28.713270+0000","last_clean":"2023-08-09T15:11:28.713270+0000","last_became_active":"2023-08-09T15:11:27.691442+0000","last_became_peered":"2023-08-09T15:11:27.691442+0000","last_unstale":"2023-08-09T15:11:28.713270+0000","last_undegraded":"2023-08-09T15:11:28.713270+0000","last_fullsized":"2023-08-09T15:11:28.713270+0000","mapping_epoch":220,"log_start":"0'0","ondisk_log_start":"0'0","created":220,"last_epoch_clean":221,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T15:11:26.638086+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T15:11:26.638086+0000","last_clean_scrub_stamp":"2023-08-09T15:11:26.638086+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T21:45:58.163091+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"27.1a","version":"0'0","reported_seq":9,"reported_epoch":222,"state":"active+clean","last_fresh":"2023-08-09T15:11:28.729738+0000","last_change":"2023-08-09T15:11:28.726755+0000","last_active":"2023-08-09T15:11:28.729738+0000","last_peered":"2023-08-09T15:11:28.729738+0000","last_clean":"2023-08-09T15:11:28.729738+0000","last_became_active":"2023-08-09T15:11:28.726364+0000","last_became_peered":"2023-08-09T15:11:28.726364+0000","last_unstale":"2023-08-09T15:11:28.729738+0000","last_undegraded":"2023-08-09T15:11:28.729738+0000","last_fullsized":"2023-08-09T15:11:28.729738+0000","mapping_epoch":221,"log_start":"0'0","ondisk_log_start":"0'0","created":221,"last_epoch_clean":222,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T15:11:27.650831+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T15:11:27.650831+0000","last_clean_scrub_stamp":"2023-08-09T15:11:27.650831+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T16:27:47.389259+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"26.1b","version":"222'1","reported_seq":12,"reported_epoch":222,"state":"active+clean","last_fresh":"2023-08-09T15:11:28.740859+0000","last_change":"2023-08-09T15:11:27.688736+0000","last_active":"2023-08-09T15:11:28.740859+0000","last_peered":"2023-08-09T15:11:28.740859+0000","last_clean":"2023-08-09T15:11:28.740859+0000","last_became_active":"2023-08-09T15:11:27.688431+0000","last_became_peered":"2023-08-09T15:11:27.688431+0000","last_unstale":"2023-08-09T15:11:28.740859+0000","last_undegraded":"2023-08-09T15:11:28.740859+0000","last_fullsized":"2023-08-09T15:11:28.740859+0000","mapping_epoch":220,"log_start":"0'0","ondisk_log_start":"0'0","created":220,"last_epoch_clean":221,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T15:11:26.638086+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T15:11:26.638086+0000","last_clean_scrub_stamp":"2023-08-09T15:11:26.638086+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-11T00:48:20.135672+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":411,"reported_epoch":222,"state":"active+clean","last_fresh":"2023-08-09T15:11:28.723778+0000","last_change":"2023-08-09T13:33:56.401406+0000","last_active":"2023-08-09T15:11:28.723778+0000","last_peered":"2023-08-09T15:11:28.723778+0000","last_clean":"2023-08-09T15:11:28.723778+0000","last_became_active":"2023-08-09T13:33:56.400487+0000","last_became_peered":"2023-08-09T13:33:56.400487+0000","last_unstale":"2023-08-09T15:11:28.723778+0000","last_undegraded":"2023-08-09T15:11:28.723778+0000","last_fullsized":"2023-08-09T15:11:28.723778+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:33:54.845869+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:33:54.845869+0000","last_clean_scrub_stamp":"2023-08-09T13:33:54.845869+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T20:59:28.425902+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"27.1b","version":"0'0","reported_seq":9,"reported_epoch":222,"state":"active+clean","last_fresh":"2023-08-09T15:11:28.745857+0000","last_change":"2023-08-09T15:11:28.745303+0000","last_active":"2023-08-09T15:11:28.745857+0000","last_peered":"2023-08-09T15:11:28.745857+0000","last_clean":"2023-08-09T15:11:28.745857+0000","last_became_active":"2023-08-09T15:11:28.744982+0000","last_became_peered":"2023-08-09T15:11:28.744982+0000","last_unstale":"2023-08-09T15:11:28.745857+0000","last_undegraded":"2023-08-09T15:11:28.745857+0000","last_fullsized":"2023-08-09T15:11:28.745857+0000","mapping_epoch":221,"log_start":"0'0","ondisk_log_start":"0'0","created":221,"last_epoch_clean":222,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T15:11:27.650831+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T15:11:27.650831+0000","last_clean_scrub_stamp":"2023-08-09T15:11:27.650831+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T19:21:28.600270+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"26.1a","version":"222'1","reported_seq":12,"reported_epoch":222,"state":"active+clean","last_fresh":"2023-08-09T15:11:28.724136+0000","last_change":"2023-08-09T15:11:27.701924+0000","last_active":"2023-08-09T15:11:28.724136+0000","last_peered":"2023-08-09T15:11:28.724136+0000","last_clean":"2023-08-09T15:11:28.724136+0000","last_became_active":"2023-08-09T15:11:27.701594+0000","last_became_peered":"2023-08-09T15:11:27.701594+0000","last_unstale":"2023-08-09T15:11:28.724136+0000","last_undegraded":"2023-08-09T15:11:28.724136+0000","last_fullsized":"2023-08-09T15:11:28.724136+0000","mapping_epoch":220,"log_start":"0'0","ondisk_log_start":"0'0","created":220,"last_epoch_clean":221,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T15:11:26.638086+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T15:11:26.638086+0000","last_clean_scrub_stamp":"2023-08-09T15:11:26.638086+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-11T01:13:04.521727+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":417,"reported_epoch":222,"state":"active+clean","last_fresh":"2023-08-09T15:11:28.707730+0000","last_change":"2023-08-09T13:33:56.401123+0000","last_active":"2023-08-09T15:11:28.707730+0000","last_peered":"2023-08-09T15:11:28.707730+0000","last_clean":"2023-08-09T15:11:28.707730+0000","last_became_active":"2023-08-09T13:33:56.399933+0000","last_became_peered":"2023-08-09T13:33:56.399933+0000","last_unstale":"2023-08-09T15:11:28.707730+0000","last_undegraded":"2023-08-09T15:11:28.707730+0000","last_fullsized":"2023-08-09T15:11:28.707730+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:33:54.845869+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:33:54.845869+0000","last_clean_scrub_stamp":"2023-08-09T13:33:54.845869+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-11T00:12:40.731648+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"27.18","version":"0'0","reported_seq":9,"reported_epoch":222,"state":"active+clean","last_fresh":"2023-08-09T15:11:28.730329+0000","last_change":"2023-08-09T15:11:28.729102+0000","last_active":"2023-08-09T15:11:28.730329+0000","last_peered":"2023-08-09T15:11:28.730329+0000","last_clean":"2023-08-09T15:11:28.730329+0000","last_became_active":"2023-08-09T15:11:28.728641+0000","last_became_peered":"2023-08-09T15:11:28.728641+0000","last_unstale":"2023-08-09T15:11:28.730329+0000","last_undegraded":"2023-08-09T15:11:28.730329+0000","last_fullsized":"2023-08-09T15:11:28.730329+0000","mapping_epoch":221,"log_start":"0'0","ondisk_log_start":"0'0","created":221,"last_epoch_clean":222,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T15:11:27.650831+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T15:11:27.650831+0000","last_clean_scrub_stamp":"2023-08-09T15:11:27.650831+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T17:58:43.444721+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"26.19","version":"0'0","reported_seq":11,"reported_epoch":222,"state":"active+clean","last_fresh":"2023-08-09T15:11:28.705420+0000","last_change":"2023-08-09T15:11:27.697411+0000","last_active":"2023-08-09T15:11:28.705420+0000","last_peered":"2023-08-09T15:11:28.705420+0000","last_clean":"2023-08-09T15:11:28.705420+0000","last_became_active":"2023-08-09T15:11:27.697016+0000","last_became_peered":"2023-08-09T15:11:27.697016+0000","last_unstale":"2023-08-09T15:11:28.705420+0000","last_undegraded":"2023-08-09T15:11:28.705420+0000","last_fullsized":"2023-08-09T15:11:28.705420+0000","mapping_epoch":220,"log_start":"0'0","ondisk_log_start":"0'0","created":220,"last_epoch_clean":221,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T15:11:26.638086+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T15:11:26.638086+0000","last_clean_scrub_stamp":"2023-08-09T15:11:26.638086+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T22:10:29.522663+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.2","version":"0'0","reported_seq":409,"reported_epoch":222,"state":"active+clean","last_fresh":"2023-08-09T15:11:28.720422+0000","last_change":"2023-08-09T13:33:56.402794+0000","last_active":"2023-08-09T15:11:28.720422+0000","last_peered":"2023-08-09T15:11:28.720422+0000","last_clean":"2023-08-09T15:11:28.720422+0000","last_became_active":"2023-08-09T13:33:56.402038+0000","last_became_peered":"2023-08-09T13:33:56.402038+0000","last_unstale":"2023-08-09T15:11:28.720422+0000","last_undegraded":"2023-08-09T15:11:28.720422+0000","last_fullsized":"2023-08-09T15:11:28.720422+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:33:54.845869+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:33:54.845869+0000","last_clean_scrub_stamp":"2023-08-09T13:33:54.845869+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T15:07:55.893669+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"27.19","version":"0'0","reported_seq":9,"reported_epoch":222,"state":"active+clean","last_fresh":"2023-08-09T15:11:28.739936+0000","last_change":"2023-08-09T15:11:28.739182+0000","last_active":"2023-08-09T15:11:28.739936+0000","last_peered":"2023-08-09T15:11:28.739936+0000","last_clean":"2023-08-09T15:11:28.739936+0000","last_became_active":"2023-08-09T15:11:28.738892+0000","last_became_peered":"2023-08-09T15:11:28.738892+0000","last_unstale":"2023-08-09T15:11:28.739936+0000","last_undegraded":"2023-08-09T15:11:28.739936+0000","last_fullsized":"2023-08-09T15:11:28.739936+0000","mapping_epoch":221,"log_start":"0'0","ondisk_log_start":"0'0","created":221,"last_epoch_clean":222,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T15:11:27.650831+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T15:11:27.650831+0000","last_clean_scrub_stamp":"2023-08-09T15:11:27.650831+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T23:40:48.203972+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"26.18","version":"0'0","reported_seq":11,"reported_epoch":222,"state":"active+clean","last_fresh":"2023-08-09T15:11:28.722942+0000","last_change":"2023-08-09T15:11:27.700112+0000","last_active":"2023-08-09T15:11:28.722942+0000","last_peered":"2023-08-09T15:11:28.722942+0000","last_clean":"2023-08-09T15:11:28.722942+0000","last_became_active":"2023-08-09T15:11:27.699806+0000","last_became_peered":"2023-08-09T15:11:27.699806+0000","last_unstale":"2023-08-09T15:11:28.722942+0000","last_undegraded":"2023-08-09T15:11:28.722942+0000","last_fullsized":"2023-08-09T15:11:28.722942+0000","mapping_epoch":220,"log_start":"0'0","ondisk_log_start":"0'0","created":220,"last_epoch_clean":221,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T15:11:26.638086+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T15:11:26.638086+0000","last_clean_scrub_stamp":"2023-08-09T15:11:26.638086+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T18:46:32.279659+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.3","version":"0'0","reported_seq":423,"reported_epoch":222,"state":"active+clean","last_fresh":"2023-08-09T15:11:28.706215+0000","last_change":"2023-08-09T13:33:55.886749+0000","last_active":"2023-08-09T15:11:28.706215+0000","last_peered":"2023-08-09T15:11:28.706215+0000","last_clean":"2023-08-09T15:11:28.706215+0000","last_became_active":"2023-08-09T13:33:55.886116+0000","last_became_peered":"2023-08-09T13:33:55.886116+0000","last_unstale":"2023-08-09T15:11:28.706215+0000","last_undegraded":"2023-08-09T15:11:28.706215+0000","last_fullsized":"2023-08-09T15:11:28.706215+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:33:54.845869+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:33:54.845869+0000","last_clean_scrub_stamp":"2023-08-09T13:33:54.845869+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T23:03:25.407293+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"211'329","reported_seq":1312,"reported_epoch":222,"state":"active+clean","last_fresh":"2023-08-09T15:11:28.723653+0000","last_change":"2023-08-09T13:33:55.886345+0000","last_active":"2023-08-09T15:11:28.723653+0000","last_peered":"2023-08-09T15:11:28.723653+0000","last_clean":"2023-08-09T15:11:28.723653+0000","last_became_active":"2023-08-09T13:33:55.885781+0000","last_became_peered":"2023-08-09T13:33:55.885781+0000","last_unstale":"2023-08-09T15:11:28.723653+0000","last_undegraded":"2023-08-09T15:11:28.723653+0000","last_fullsized":"2023-08-09T15:11:28.723653+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:33:54.845869+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:33:54.845869+0000","last_clean_scrub_stamp":"2023-08-09T13:33:54.845869+0000","objects_scrubbed":0,"log_size":329,"log_dups_size":0,"ondisk_log_size":329,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T14:52:28.626982+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1963,"num_read_kb":4096,"num_write":543,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"27.1e","version":"0'0","reported_seq":9,"reported_epoch":222,"state":"active+clean","last_fresh":"2023-08-09T15:11:28.735347+0000","last_change":"2023-08-09T15:11:28.731087+0000","last_active":"2023-08-09T15:11:28.735347+0000","last_peered":"2023-08-09T15:11:28.735347+0000","last_clean":"2023-08-09T15:11:28.735347+0000","last_became_active":"2023-08-09T15:11:28.730778+0000","last_became_peered":"2023-08-09T15:11:28.730778+0000","last_unstale":"2023-08-09T15:11:28.735347+0000","last_undegraded":"2023-08-09T15:11:28.735347+0000","last_fullsized":"2023-08-09T15:11:28.735347+0000","mapping_epoch":221,"log_start":"0'0","ondisk_log_start":"0'0","created":221,"last_epoch_clean":222,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T15:11:27.650831+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T15:11:27.650831+0000","last_clean_scrub_stamp":"2023-08-09T15:11:27.650831+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T23:09:43.428168+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"26.1f","version":"222'4","reported_seq":15,"reported_epoch":222,"state":"active+clean","last_fresh":"2023-08-09T15:11:28.743494+0000","last_change":"2023-08-09T15:11:27.695330+0000","last_active":"2023-08-09T15:11:28.743494+0000","last_peered":"2023-08-09T15:11:28.743494+0000","last_clean":"2023-08-09T15:11:28.743494+0000","last_became_active":"2023-08-09T15:11:27.695008+0000","last_became_peered":"2023-08-09T15:11:27.695008+0000","last_unstale":"2023-08-09T15:11:28.743494+0000","last_undegraded":"2023-08-09T15:11:28.743494+0000","last_fullsized":"2023-08-09T15:11:28.743494+0000","mapping_epoch":220,"log_start":"0'0","ondisk_log_start":"0'0","created":220,"last_epoch_clean":221,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T15:11:26.638086+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T15:11:26.638086+0000","last_clean_scrub_stamp":"2023-08-09T15:11:26.638086+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-11T02:38:21.656496+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":427,"reported_epoch":222,"state":"active+clean","last_fresh":"2023-08-09T15:11:28.699926+0000","last_change":"2023-08-09T13:33:55.889565+0000","last_active":"2023-08-09T15:11:28.699926+0000","last_peered":"2023-08-09T15:11:28.699926+0000","last_clean":"2023-08-09T15:11:28.699926+0000","last_became_active":"2023-08-09T13:33:55.889164+0000","last_became_peered":"2023-08-09T13:33:55.889164+0000","last_unstale":"2023-08-09T15:11:28.699926+0000","last_undegraded":"2023-08-09T15:11:28.699926+0000","last_fullsized":"2023-08-09T15:11:28.699926+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:33:54.845869+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:33:54.845869+0000","last_clean_scrub_stamp":"2023-08-09T13:33:54.845869+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T20:53:51.649656+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"27.1f","version":"0'0","reported_seq":9,"reported_epoch":222,"state":"active+clean","last_fresh":"2023-08-09T15:11:28.745479+0000","last_change":"2023-08-09T15:11:28.732705+0000","last_active":"2023-08-09T15:11:28.745479+0000","last_peered":"2023-08-09T15:11:28.745479+0000","last_clean":"2023-08-09T15:11:28.745479+0000","last_became_active":"2023-08-09T15:11:28.732431+0000","last_became_peered":"2023-08-09T15:11:28.732431+0000","last_unstale":"2023-08-09T15:11:28.745479+0000","last_undegraded":"2023-08-09T15:11:28.745479+0000","last_fullsized":"2023-08-09T15:11:28.745479+0000","mapping_epoch":221,"log_start":"0'0","ondisk_log_start":"0'0","created":221,"last_epoch_clean":222,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T15:11:27.650831+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T15:11:27.650831+0000","last_clean_scrub_stamp":"2023-08-09T15:11:27.650831+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T20:46:32.635231+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"26.1e","version":"222'1","reported_seq":12,"reported_epoch":222,"state":"active+clean","last_fresh":"2023-08-09T15:11:28.730475+0000","last_change":"2023-08-09T15:11:27.694291+0000","last_active":"2023-08-09T15:11:28.730475+0000","last_peered":"2023-08-09T15:11:28.730475+0000","last_clean":"2023-08-09T15:11:28.730475+0000","last_became_active":"2023-08-09T15:11:27.693664+0000","last_became_peered":"2023-08-09T15:11:27.693664+0000","last_unstale":"2023-08-09T15:11:28.730475+0000","last_undegraded":"2023-08-09T15:11:28.730475+0000","last_fullsized":"2023-08-09T15:11:28.730475+0000","mapping_epoch":220,"log_start":"0'0","ondisk_log_start":"0'0","created":220,"last_epoch_clean":221,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T15:11:26.638086+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T15:11:26.638086+0000","last_clean_scrub_stamp":"2023-08-09T15:11:26.638086+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T23:34:36.700017+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":415,"reported_epoch":222,"state":"active+clean","last_fresh":"2023-08-09T15:11:28.728714+0000","last_change":"2023-08-09T13:33:56.404236+0000","last_active":"2023-08-09T15:11:28.728714+0000","last_peered":"2023-08-09T15:11:28.728714+0000","last_clean":"2023-08-09T15:11:28.728714+0000","last_became_active":"2023-08-09T13:33:56.402485+0000","last_became_peered":"2023-08-09T13:33:56.402485+0000","last_unstale":"2023-08-09T15:11:28.728714+0000","last_undegraded":"2023-08-09T15:11:28.728714+0000","last_fullsized":"2023-08-09T15:11:28.728714+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:33:54.845869+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:33:54.845869+0000","last_clean_scrub_stamp":"2023-08-09T13:33:54.845869+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T13:56:39.809056+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"27.1c","version":"0'0","reported_seq":9,"reported_epoch":222,"state":"active+clean","last_fresh":"2023-08-09T15:11:28.740530+0000","last_change":"2023-08-09T15:11:28.739252+0000","last_active":"2023-08-09T15:11:28.740530+0000","last_peered":"2023-08-09T15:11:28.740530+0000","last_clean":"2023-08-09T15:11:28.740530+0000","last_became_active":"2023-08-09T15:11:28.738158+0000","last_became_peered":"2023-08-09T15:11:28.738158+0000","last_unstale":"2023-08-09T15:11:28.740530+0000","last_undegraded":"2023-08-09T15:11:28.740530+0000","last_fullsized":"2023-08-09T15:11:28.740530+0000","mapping_epoch":221,"log_start":"0'0","ondisk_log_start":"0'0","created":221,"last_epoch_clean":222,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T15:11:27.650831+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T15:11:27.650831+0000","last_clean_scrub_stamp":"2023-08-09T15:11:27.650831+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T15:35:15.135484+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"26.1d","version":"222'2","reported_seq":13,"reported_epoch":222,"state":"active+clean","last_fresh":"2023-08-09T15:11:28.729546+0000","last_change":"2023-08-09T15:11:27.690620+0000","last_active":"2023-08-09T15:11:28.729546+0000","last_peered":"2023-08-09T15:11:28.729546+0000","last_clean":"2023-08-09T15:11:28.729546+0000","last_became_active":"2023-08-09T15:11:27.690312+0000","last_became_peered":"2023-08-09T15:11:27.690312+0000","last_unstale":"2023-08-09T15:11:28.729546+0000","last_undegraded":"2023-08-09T15:11:28.729546+0000","last_fullsized":"2023-08-09T15:11:28.729546+0000","mapping_epoch":220,"log_start":"0'0","ondisk_log_start":"0'0","created":220,"last_epoch_clean":221,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T15:11:26.638086+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T15:11:26.638086+0000","last_clean_scrub_stamp":"2023-08-09T15:11:26.638086+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T18:13:52.359530+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":408,"reported_epoch":222,"state":"active+clean","last_fresh":"2023-08-09T15:11:28.716347+0000","last_change":"2023-08-09T13:33:56.401781+0000","last_active":"2023-08-09T15:11:28.716347+0000","last_peered":"2023-08-09T15:11:28.716347+0000","last_clean":"2023-08-09T15:11:28.716347+0000","last_became_active":"2023-08-09T13:33:56.400336+0000","last_became_peered":"2023-08-09T13:33:56.400336+0000","last_unstale":"2023-08-09T15:11:28.716347+0000","last_undegraded":"2023-08-09T15:11:28.716347+0000","last_fullsized":"2023-08-09T15:11:28.716347+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:33:54.845869+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:33:54.845869+0000","last_clean_scrub_stamp":"2023-08-09T13:33:54.845869+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-11T00:43:56.815579+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"27.1d","version":"0'0","reported_seq":9,"reported_epoch":222,"state":"active+clean","last_fresh":"2023-08-09T15:11:28.729148+0000","last_change":"2023-08-09T15:11:28.725941+0000","last_active":"2023-08-09T15:11:28.729148+0000","last_peered":"2023-08-09T15:11:28.729148+0000","last_clean":"2023-08-09T15:11:28.729148+0000","last_became_active":"2023-08-09T15:11:28.725586+0000","last_became_peered":"2023-08-09T15:11:28.725586+0000","last_unstale":"2023-08-09T15:11:28.729148+0000","last_undegraded":"2023-08-09T15:11:28.729148+0000","last_fullsized":"2023-08-09T15:11:28.729148+0000","mapping_epoch":221,"log_start":"0'0","ondisk_log_start":"0'0","created":221,"last_epoch_clean":222,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T15:11:27.650831+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T15:11:27.650831+0000","last_clean_scrub_stamp":"2023-08-09T15:11:27.650831+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T20:53:21.300084+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"26.1c","version":"222'2","reported_seq":13,"reported_epoch":222,"state":"active+clean","last_fresh":"2023-08-09T15:11:28.742373+0000","last_change":"2023-08-09T15:11:27.697916+0000","last_active":"2023-08-09T15:11:28.742373+0000","last_peered":"2023-08-09T15:11:28.742373+0000","last_clean":"2023-08-09T15:11:28.742373+0000","last_became_active":"2023-08-09T15:11:27.697637+0000","last_became_peered":"2023-08-09T15:11:27.697637+0000","last_unstale":"2023-08-09T15:11:28.742373+0000","last_undegraded":"2023-08-09T15:11:28.742373+0000","last_fullsized":"2023-08-09T15:11:28.742373+0000","mapping_epoch":220,"log_start":"0'0","ondisk_log_start":"0'0","created":220,"last_epoch_clean":221,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T15:11:26.638086+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T15:11:26.638086+0000","last_clean_scrub_stamp":"2023-08-09T15:11:26.638086+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T20:54:00.982392+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":408,"reported_epoch":222,"state":"active+clean","last_fresh":"2023-08-09T15:11:28.725867+0000","last_change":"2023-08-09T13:33:55.886389+0000","last_active":"2023-08-09T15:11:28.725867+0000","last_peered":"2023-08-09T15:11:28.725867+0000","last_clean":"2023-08-09T15:11:28.725867+0000","last_became_active":"2023-08-09T13:33:55.885638+0000","last_became_peered":"2023-08-09T13:33:55.885638+0000","last_unstale":"2023-08-09T15:11:28.725867+0000","last_undegraded":"2023-08-09T15:11:28.725867+0000","last_fullsized":"2023-08-09T15:11:28.725867+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:33:54.845869+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:33:54.845869+0000","last_clean_scrub_stamp":"2023-08-09T13:33:54.845869+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T13:34:44.592008+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]}],"pool_stats":[{"poolid":26,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":38,"ondisk_log_size":38,"up":64,"acting":64,"num_store_stats":8},{"poolid":2,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1963,"num_read_kb":4096,"num_write":543,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":329,"ondisk_log_size":329,"up":2,"acting":2,"num_store_stats":2},{"poolid":27,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":8}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834576013,"num_pgs":17,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":52476,"kb_used_data":3308,"kb_used_omap":1,"kb_used_meta":49150,"kb_avail":94319364,"statfs":{"total":96636764160,"available":96583028736,"internally_reserved":0,"allocated":3387392,"data_stored":2107110,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1781,"internal_metadata":50329867},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Aug 9 15:10:49 2023","interfaces":[{"interface":"back","average":{"1min":1.2330000000000001,"5min":1.2569999999999999,"15min":1.234},"min":{"1min":0.81599999999999995,"5min":0.76700000000000002,"15min":0.76100000000000001},"max":{"1min":1.6739999999999999,"5min":2.056,"15min":2.056},"last":1.3049999999999999},{"interface":"front","average":{"1min":1.2689999999999999,"5min":1.264,"15min":1.2649999999999999},"min":{"1min":0.72899999999999998,"5min":0.72899999999999998,"15min":0.72899999999999998},"max":{"1min":1.732,"5min":1.863,"15min":1.863},"last":1.5169999999999999}]},{"osd":1,"last update":"Wed Aug 9 15:10:49 2023","interfaces":[{"interface":"back","average":{"1min":1.337,"5min":1.294,"15min":1.3160000000000001},"min":{"1min":0.95399999999999996,"5min":0.63500000000000001,"15min":0.63500000000000001},"max":{"1min":1.7090000000000001,"5min":1.853,"15min":1.853},"last":1.1950000000000001},{"interface":"front","average":{"1min":1.3,"5min":1.3380000000000001,"15min":1.3149999999999999},"min":{"1min":0.96499999999999997,"5min":0.879,"15min":0.879},"max":{"1min":1.6519999999999999,"5min":2.1520000000000001,"15min":2.1520000000000001},"last":1.1499999999999999}]},{"osd":2,"last update":"Wed Aug 9 15:10:49 2023","interfaces":[{"interface":"back","average":{"1min":1.3500000000000001,"5min":1.3160000000000001,"15min":1.3069999999999999},"min":{"1min":0.95299999999999996,"5min":0.81299999999999994,"15min":0.81299999999999994},"max":{"1min":1.861,"5min":1.861,"15min":1.861},"last":1.3340000000000001},{"interface":"front","average":{"1min":1.3480000000000001,"5min":1.397,"15min":1.4019999999999999},"min":{"1min":1.0189999999999999,"5min":1.0189999999999999,"15min":0.88500000000000001},"max":{"1min":1.603,"5min":1.9370000000000001,"15min":1.9370000000000001},"last":1.1020000000000001}]},{"osd":3,"last update":"Wed Aug 9 15:10:49 2023","interfaces":[{"interface":"back","average":{"1min":1.387,"5min":1.3500000000000001,"15min":1.3580000000000001},"min":{"1min":1.024,"5min":0.80800000000000005,"15min":0.80800000000000005},"max":{"1min":1.7370000000000001,"5min":1.881,"15min":1.9259999999999999},"last":0.98099999999999998},{"interface":"front","average":{"1min":1.446,"5min":1.466,"15min":1.48},"min":{"1min":1.127,"5min":0.92700000000000005,"15min":0.92700000000000005},"max":{"1min":1.9239999999999999,"5min":1.9239999999999999,"15min":1.9239999999999999},"last":1.603}]},{"osd":4,"last update":"Wed Aug 9 15:10:49 2023","interfaces":[{"interface":"back","average":{"1min":1.3320000000000001,"5min":1.298,"15min":1.3009999999999999},"min":{"1min":0.90300000000000002,"5min":0.86799999999999999,"15min":0.69599999999999995},"max":{"1min":1.8300000000000001,"5min":2.2810000000000001,"15min":2.2810000000000001},"last":1.2749999999999999},{"interface":"front","average":{"1min":1.4019999999999999,"5min":1.3839999999999999,"15min":1.3879999999999999},"min":{"1min":0.97499999999999998,"5min":0.85099999999999998,"15min":0.66000000000000003},"max":{"1min":1.875,"5min":1.875,"15min":2.0830000000000002},"last":1.474}]},{"osd":5,"last update":"Wed Aug 9 15:10:49 2023","interfaces":[{"interface":"back","average":{"1min":1.395,"5min":1.431,"15min":1.4510000000000001},"min":{"1min":0.89900000000000002,"5min":0.86099999999999999,"15min":0.86099999999999999},"max":{"1min":1.7589999999999999,"5min":1.9550000000000001,"15min":1.9550000000000001},"last":1.4259999999999999},{"interface":"front","average":{"1min":1.405,"5min":1.4670000000000001,"15min":1.4830000000000001},"min":{"1min":0.89000000000000001,"5min":0.89000000000000001,"15min":0.89000000000000001},"max":{"1min":1.825,"5min":2.5310000000000001,"15min":2.5310000000000001},"last":1.724}]},{"osd":6,"last update":"Wed Aug 9 15:10:49 2023","interfaces":[{"interface":"back","average":{"1min":1.55,"5min":1.617,"15min":1.5980000000000001},"min":{"1min":1.1819999999999999,"5min":1.121,"15min":1.101},"max":{"1min":2.1850000000000001,"5min":2.8959999999999999,"15min":2.8959999999999999},"last":1.6499999999999999},{"interface":"front","average":{"1min":1.522,"5min":1.556,"15min":1.546},"min":{"1min":1.026,"5min":1.026,"15min":0.90800000000000003},"max":{"1min":1.992,"5min":2.214,"15min":2.222},"last":1.5609999999999999}]}]},{"osd":6,"up_from":13,"seq":55834576013,"num_pgs":15,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":60072,"kb_used_data":2716,"kb_used_omap":1,"kb_used_meta":57342,"kb_avail":94311768,"statfs":{"total":96636764160,"available":96575250432,"internally_reserved":0,"allocated":2781184,"data_stored":1514351,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1781,"internal_metadata":58718475},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Aug 9 15:10:57 2023","interfaces":[{"interface":"back","average":{"1min":1.1830000000000001,"5min":1.228,"15min":1.22},"min":{"1min":0.75600000000000001,"5min":0.75600000000000001,"15min":0.75600000000000001},"max":{"1min":1.734,"5min":1.772,"15min":1.8859999999999999},"last":1.468},{"interface":"front","average":{"1min":1.2589999999999999,"5min":1.2490000000000001,"15min":1.254},"min":{"1min":0.83799999999999997,"5min":0.83799999999999997,"15min":0.77400000000000002},"max":{"1min":1.639,"5min":2.0800000000000001,"15min":2.0800000000000001},"last":1.266}]},{"osd":1,"last update":"Wed Aug 9 15:10:57 2023","interfaces":[{"interface":"back","average":{"1min":1.319,"5min":1.2050000000000001,"15min":1.2010000000000001},"min":{"1min":0.90300000000000002,"5min":0.70999999999999996,"15min":0.70999999999999996},"max":{"1min":2.6760000000000002,"5min":2.6760000000000002,"15min":2.6760000000000002},"last":1.2430000000000001},{"interface":"front","average":{"1min":1.2969999999999999,"5min":1.3100000000000001,"15min":1.345},"min":{"1min":0.90300000000000002,"5min":0.82299999999999995,"15min":0.82299999999999995},"max":{"1min":1.998,"5min":1.998,"15min":1.998},"last":1.339}]},{"osd":2,"last update":"Wed Aug 9 15:10:57 2023","interfaces":[{"interface":"back","average":{"1min":1.351,"5min":1.3400000000000001,"15min":1.3480000000000001},"min":{"1min":0.91900000000000004,"5min":0.82099999999999995,"15min":0.82099999999999995},"max":{"1min":1.8149999999999999,"5min":2.0089999999999999,"15min":2.081},"last":1.5369999999999999},{"interface":"front","average":{"1min":1.4299999999999999,"5min":1.3400000000000001,"15min":1.3540000000000001},"min":{"1min":1.04,"5min":0.80700000000000005,"15min":0.80700000000000005},"max":{"1min":1.9219999999999999,"5min":1.9219999999999999,"15min":1.9219999999999999},"last":1.163}]},{"osd":3,"last update":"Wed Aug 9 15:10:57 2023","interfaces":[{"interface":"back","average":{"1min":1.4910000000000001,"5min":1.4390000000000001,"15min":1.4099999999999999},"min":{"1min":1.1359999999999999,"5min":0.79700000000000004,"15min":0.79700000000000004},"max":{"1min":1.859,"5min":2.0179999999999998,"15min":2.0179999999999998},"last":0.85299999999999998},{"interface":"front","average":{"1min":1.3340000000000001,"5min":1.3420000000000001,"15min":1.351},"min":{"1min":0.95899999999999996,"5min":0.95899999999999996,"15min":0.95899999999999996},"max":{"1min":1.71,"5min":2.0539999999999998,"15min":2.0539999999999998},"last":1.002}]},{"osd":4,"last update":"Wed Aug 9 15:10:57 2023","interfaces":[{"interface":"back","average":{"1min":1.4299999999999999,"5min":1.3560000000000001,"15min":1.3300000000000001},"min":{"1min":0.89700000000000002,"5min":0.53500000000000003,"15min":0.53500000000000003},"max":{"1min":2.0409999999999999,"5min":2.0470000000000002,"15min":2.0470000000000002},"last":1.593},{"interface":"front","average":{"1min":1.5600000000000001,"5min":1.4850000000000001,"15min":1.4350000000000001},"min":{"1min":0.96199999999999997,"5min":0.82099999999999995,"15min":0.82099999999999995},"max":{"1min":2.1139999999999999,"5min":2.1589999999999998,"15min":2.1589999999999998},"last":1.05}]},{"osd":5,"last update":"Wed Aug 9 15:10:57 2023","interfaces":[{"interface":"back","average":{"1min":1.492,"5min":1.4370000000000001,"15min":1.381},"min":{"1min":0.86599999999999999,"5min":0.86599999999999999,"15min":0.80600000000000005},"max":{"1min":1.881,"5min":2.1269999999999998,"15min":2.6059999999999999},"last":1.133},{"interface":"front","average":{"1min":1.7090000000000001,"5min":1.6599999999999999,"15min":1.625},"min":{"1min":1.3919999999999999,"5min":0.99399999999999999,"15min":0.85599999999999998},"max":{"1min":2.0609999999999999,"5min":2.125,"15min":2.1899999999999999},"last":1.6639999999999999}]},{"osd":7,"last update":"Wed Aug 9 15:10:57 2023","interfaces":[{"interface":"back","average":{"1min":1.7210000000000001,"5min":1.671,"15min":1.6160000000000001},"min":{"1min":1.2689999999999999,"5min":1.1240000000000001,"15min":0.84699999999999998},"max":{"1min":2.2229999999999999,"5min":2.2280000000000002,"15min":2.4340000000000002},"last":1.508},{"interface":"front","average":{"1min":1.6970000000000001,"5min":1.643,"15min":1.6020000000000001},"min":{"1min":1.2649999999999999,"5min":1.0940000000000001,"15min":1.0209999999999999},"max":{"1min":2.1349999999999998,"5min":2.3300000000000001,"15min":2.7400000000000002},"last":1.3109999999999999}]}]},{"osd":5,"up_from":13,"seq":55834576013,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":47776,"kb_used_data":2704,"kb_used_omap":1,"kb_used_meta":45054,"kb_avail":94324064,"statfs":{"total":96636764160,"available":96587841536,"internally_reserved":0,"allocated":2768896,"data_stored":1507638,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1781,"internal_metadata":46135563},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Aug 9 15:10:47 2023","interfaces":[{"interface":"back","average":{"1min":1.23,"5min":1.218,"15min":1.206},"min":{"1min":0.97699999999999998,"5min":0.59399999999999997,"15min":0.59399999999999997},"max":{"1min":1.6120000000000001,"5min":1.7370000000000001,"15min":1.754},"last":0.57699999999999996},{"interface":"front","average":{"1min":1.262,"5min":1.2230000000000001,"15min":1.224},"min":{"1min":0.81699999999999995,"5min":0.57899999999999996,"15min":0.57899999999999996},"max":{"1min":1.6870000000000001,"5min":1.865,"15min":1.976},"last":0.55600000000000005}]},{"osd":1,"last update":"Wed Aug 9 15:10:47 2023","interfaces":[{"interface":"back","average":{"1min":1.282,"5min":1.2769999999999999,"15min":1.254},"min":{"1min":0.89200000000000002,"5min":0.65200000000000002,"15min":0.65200000000000002},"max":{"1min":1.726,"5min":2.016,"15min":2.016},"last":0.67500000000000004},{"interface":"front","average":{"1min":1.3939999999999999,"5min":1.319,"15min":1.351},"min":{"1min":0.98999999999999999,"5min":0.74299999999999999,"15min":0.74299999999999999},"max":{"1min":1.7150000000000001,"5min":1.8959999999999999,"15min":1.8979999999999999},"last":0.65000000000000002}]},{"osd":2,"last update":"Wed Aug 9 15:10:47 2023","interfaces":[{"interface":"back","average":{"1min":1.3280000000000001,"5min":1.3220000000000001,"15min":1.321},"min":{"1min":0.88900000000000001,"5min":0.88700000000000001,"15min":0.82499999999999996},"max":{"1min":1.8859999999999999,"5min":1.952,"15min":2.032},"last":0.60199999999999998},{"interface":"front","average":{"1min":1.367,"5min":1.3700000000000001,"15min":1.3720000000000001},"min":{"1min":1.0760000000000001,"5min":0.81100000000000005,"15min":0.81100000000000005},"max":{"1min":1.6679999999999999,"5min":1.964,"15min":1.964},"last":0.61899999999999999}]},{"osd":3,"last update":"Wed Aug 9 15:10:47 2023","interfaces":[{"interface":"back","average":{"1min":1.4239999999999999,"5min":1.389,"15min":1.395},"min":{"1min":0.95999999999999996,"5min":0.95999999999999996,"15min":0.95099999999999996},"max":{"1min":1.796,"5min":1.798,"15min":1.899},"last":0.746},{"interface":"front","average":{"1min":1.393,"5min":1.4390000000000001,"15min":1.45},"min":{"1min":0.99099999999999999,"5min":0.85999999999999999,"15min":0.85999999999999999},"max":{"1min":1.7509999999999999,"5min":2.0379999999999998,"15min":2.0379999999999998},"last":0.66400000000000003}]},{"osd":4,"last update":"Wed Aug 9 15:10:47 2023","interfaces":[{"interface":"back","average":{"1min":1.266,"5min":1.3140000000000001,"15min":1.3220000000000001},"min":{"1min":0.84099999999999997,"5min":0.84099999999999997,"15min":0.62},"max":{"1min":1.7210000000000001,"5min":2.0299999999999998,"15min":2.0299999999999998},"last":0.53000000000000003},{"interface":"front","average":{"1min":1.3029999999999999,"5min":1.377,"15min":1.405},"min":{"1min":0.76200000000000001,"5min":0.76200000000000001,"15min":0.64200000000000002},"max":{"1min":1.6950000000000001,"5min":1.9239999999999999,"15min":2.0390000000000001},"last":0.69199999999999995}]},{"osd":6,"last update":"Wed Aug 9 15:10:47 2023","interfaces":[{"interface":"back","average":{"1min":1.4079999999999999,"5min":1.3919999999999999,"15min":1.421},"min":{"1min":0.97799999999999998,"5min":0.96099999999999997,"15min":0.67400000000000004},"max":{"1min":1.712,"5min":2.1000000000000001,"15min":2.1000000000000001},"last":0.501},{"interface":"front","average":{"1min":1.448,"5min":1.4970000000000001,"15min":1.506},"min":{"1min":1.0449999999999999,"5min":1.0449999999999999,"15min":0.79900000000000004},"max":{"1min":1.839,"5min":2.1349999999999998,"15min":2.1349999999999998},"last":0.70599999999999996}]},{"osd":7,"last update":"Wed Aug 9 15:10:47 2023","interfaces":[{"interface":"back","average":{"1min":1.577,"5min":1.548,"15min":1.587},"min":{"1min":1.1499999999999999,"5min":1.0569999999999999,"15min":0.86099999999999999},"max":{"1min":1.8280000000000001,"5min":1.9670000000000001,"15min":2.4689999999999999},"last":0.71999999999999997},{"interface":"front","average":{"1min":1.585,"5min":1.6060000000000001,"15min":1.6439999999999999},"min":{"1min":1.101,"5min":1.044,"15min":0.83399999999999996},"max":{"1min":1.8939999999999999,"5min":2.1539999999999999,"15min":2.4239999999999999},"last":0.63800000000000001}]}]},{"osd":4,"up_from":13,"seq":55834576013,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":47804,"kb_used_data":2736,"kb_used_omap":1,"kb_used_meta":45054,"kb_avail":94324036,"statfs":{"total":96636764160,"available":96587812864,"internally_reserved":0,"allocated":2801664,"data_stored":1516846,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1781,"internal_metadata":46135563},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Aug 9 15:11:11 2023","interfaces":[{"interface":"back","average":{"1min":1.3089999999999999,"5min":1.1970000000000001,"15min":1.226},"min":{"1min":0.81399999999999995,"5min":0.60399999999999998,"15min":0.60399999999999998},"max":{"1min":1.708,"5min":1.708,"15min":2.2959999999999998},"last":0.73999999999999999},{"interface":"front","average":{"1min":1.3540000000000001,"5min":1.3009999999999999,"15min":1.3200000000000001},"min":{"1min":0.85699999999999998,"5min":0.66800000000000004,"15min":0.66800000000000004},"max":{"1min":2.0779999999999998,"5min":2.0779999999999998,"15min":2.2709999999999999},"last":1.5489999999999999}]},{"osd":1,"last update":"Wed Aug 9 15:11:11 2023","interfaces":[{"interface":"back","average":{"1min":1.3480000000000001,"5min":1.3580000000000001,"15min":1.355},"min":{"1min":0.95899999999999996,"5min":0.74199999999999999,"15min":0.74199999999999999},"max":{"1min":1.911,"5min":1.931,"15min":1.931},"last":1.413},{"interface":"front","average":{"1min":1.306,"5min":1.258,"15min":1.282},"min":{"1min":0.74299999999999999,"5min":0.73899999999999999,"15min":0.73899999999999999},"max":{"1min":1.6890000000000001,"5min":1.8600000000000001,"15min":2.117},"last":1.117}]},{"osd":2,"last update":"Wed Aug 9 15:11:11 2023","interfaces":[{"interface":"back","average":{"1min":1.27,"5min":1.246,"15min":1.2929999999999999},"min":{"1min":0.63100000000000001,"5min":0.63100000000000001,"15min":0.63100000000000001},"max":{"1min":1.7709999999999999,"5min":1.7709999999999999,"15min":1.883},"last":0.875},{"interface":"front","average":{"1min":1.4239999999999999,"5min":1.415,"15min":1.409},"min":{"1min":0.88700000000000001,"5min":0.79200000000000004,"15min":0.79200000000000004},"max":{"1min":2.306,"5min":2.343,"15min":2.343},"last":1.492}]},{"osd":3,"last update":"Wed Aug 9 15:11:11 2023","interfaces":[{"interface":"back","average":{"1min":1.411,"5min":1.3580000000000001,"15min":1.357},"min":{"1min":0.93899999999999995,"5min":0.88700000000000001,"15min":0.88700000000000001},"max":{"1min":1.8009999999999999,"5min":1.8009999999999999,"15min":2.4870000000000001},"last":0.95299999999999996},{"interface":"front","average":{"1min":1.4279999999999999,"5min":1.4390000000000001,"15min":1.4690000000000001},"min":{"1min":0.79700000000000004,"5min":0.79700000000000004,"15min":0.79700000000000004},"max":{"1min":1.8660000000000001,"5min":1.909,"15min":2.5459999999999998},"last":1.173}]},{"osd":5,"last update":"Wed Aug 9 15:11:11 2023","interfaces":[{"interface":"back","average":{"1min":1.3120000000000001,"5min":1.3620000000000001,"15min":1.3999999999999999},"min":{"1min":0.82199999999999995,"5min":0.82199999999999995,"15min":0.82199999999999995},"max":{"1min":2.633,"5min":2.633,"15min":2.633},"last":1.2410000000000001},{"interface":"front","average":{"1min":1.355,"5min":1.4079999999999999,"15min":1.393},"min":{"1min":0.68100000000000005,"5min":0.68100000000000005,"15min":0.68100000000000005},"max":{"1min":1.9930000000000001,"5min":1.9930000000000001,"15min":2.4249999999999998},"last":1.3620000000000001}]},{"osd":6,"last update":"Wed Aug 9 15:11:11 2023","interfaces":[{"interface":"back","average":{"1min":1.4119999999999999,"5min":1.4099999999999999,"15min":1.4139999999999999},"min":{"1min":0.71299999999999997,"5min":0.71299999999999997,"15min":0.71299999999999997},"max":{"1min":1.927,"5min":2.0049999999999999,"15min":2.9119999999999999},"last":1.0509999999999999},{"interface":"front","average":{"1min":1.571,"5min":1.5840000000000001,"15min":1.5800000000000001},"min":{"1min":0.77000000000000002,"5min":0.77000000000000002,"15min":0.77000000000000002},"max":{"1min":2.9529999999999998,"5min":2.9529999999999998,"15min":2.9529999999999998},"last":1.607}]},{"osd":7,"last update":"Wed Aug 9 15:11:11 2023","interfaces":[{"interface":"back","average":{"1min":1.5,"5min":1.52,"15min":1.53},"min":{"1min":0.83899999999999997,"5min":0.83899999999999997,"15min":0.83899999999999997},"max":{"1min":2.1589999999999998,"5min":2.1589999999999998,"15min":2.5910000000000002},"last":1.2150000000000001},{"interface":"front","average":{"1min":1.6419999999999999,"5min":1.5800000000000001,"15min":1.609},"min":{"1min":0.90000000000000002,"5min":0.90000000000000002,"15min":0.90000000000000002},"max":{"1min":2.218,"5min":2.218,"15min":3.028},"last":1.298}]}]},{"osd":3,"up_from":13,"seq":55834576014,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":47800,"kb_used_data":2728,"kb_used_omap":1,"kb_used_meta":45054,"kb_avail":94324040,"statfs":{"total":96636764160,"available":96587816960,"internally_reserved":0,"allocated":2793472,"data_stored":1515312,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1781,"internal_metadata":46135563},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Aug 9 15:11:03 2023","interfaces":[{"interface":"back","average":{"1min":1.105,"5min":0.999,"15min":0.98499999999999999},"min":{"1min":0.77900000000000003,"5min":0.45200000000000001,"15min":0.45200000000000001},"max":{"1min":1.657,"5min":1.6699999999999999,"15min":2.2090000000000001},"last":0.47699999999999998},{"interface":"front","average":{"1min":1.208,"5min":1.137,"15min":1.167},"min":{"1min":0.872,"5min":0.45200000000000001,"15min":0.45200000000000001},"max":{"1min":1.6020000000000001,"5min":1.9630000000000001,"15min":1.9630000000000001},"last":0.58399999999999996}]},{"osd":1,"last update":"Wed Aug 9 15:11:03 2023","interfaces":[{"interface":"back","average":{"1min":1.0449999999999999,"5min":1.0469999999999999,"15min":1.0780000000000001},"min":{"1min":0.80300000000000005,"5min":0.53900000000000003,"15min":0.53900000000000003},"max":{"1min":1.446,"5min":1.956,"15min":2.27},"last":0.70999999999999996},{"interface":"front","average":{"1min":1.276,"5min":1.2529999999999999,"15min":1.266},"min":{"1min":0.98799999999999999,"5min":0.498,"15min":0.498},"max":{"1min":1.6419999999999999,"5min":1.9730000000000001,"15min":1.9730000000000001},"last":0.68000000000000005}]},{"osd":2,"last update":"Wed Aug 9 15:11:03 2023","interfaces":[{"interface":"back","average":{"1min":1.387,"5min":1.296,"15min":1.337},"min":{"1min":1.002,"5min":0.70799999999999996,"15min":0.70799999999999996},"max":{"1min":1.861,"5min":1.9650000000000001,"15min":2.4079999999999999},"last":1.1870000000000001},{"interface":"front","average":{"1min":1.4470000000000001,"5min":1.4279999999999999,"15min":1.4410000000000001},"min":{"1min":1.077,"5min":0.72499999999999998,"15min":0.72499999999999998},"max":{"1min":1.883,"5min":2.0510000000000002,"15min":2.1259999999999999},"last":0.80800000000000005}]},{"osd":4,"last update":"Wed Aug 9 15:11:03 2023","interfaces":[{"interface":"back","average":{"1min":1.3540000000000001,"5min":1.3720000000000001,"15min":1.4099999999999999},"min":{"1min":1.01,"5min":0.748,"15min":0.72599999999999998},"max":{"1min":1.708,"5min":1.9179999999999999,"15min":2.2069999999999999},"last":0.752},{"interface":"front","average":{"1min":1.5349999999999999,"5min":1.4510000000000001,"15min":1.4650000000000001},"min":{"1min":1.1919999999999999,"5min":0.90500000000000003,"15min":0.89500000000000002},"max":{"1min":1.863,"5min":1.968,"15min":2.056},"last":1.0109999999999999}]},{"osd":5,"last update":"Wed Aug 9 15:11:03 2023","interfaces":[{"interface":"back","average":{"1min":1.5620000000000001,"5min":1.518,"15min":1.548},"min":{"1min":1.3320000000000001,"5min":0.88700000000000001,"15min":0.88700000000000001},"max":{"1min":2.0870000000000002,"5min":2.0870000000000002,"15min":2.1669999999999998},"last":0.84299999999999997},{"interface":"front","average":{"1min":1.704,"5min":1.6040000000000001,"15min":1.6160000000000001},"min":{"1min":1.337,"5min":0.85799999999999998,"15min":0.85799999999999998},"max":{"1min":2.2290000000000001,"5min":2.2469999999999999,"15min":2.4049999999999998},"last":0.89300000000000002}]},{"osd":6,"last update":"Wed Aug 9 15:11:03 2023","interfaces":[{"interface":"back","average":{"1min":1.532,"5min":1.5269999999999999,"15min":1.595},"min":{"1min":1.0569999999999999,"5min":0.81200000000000006,"15min":0.81200000000000006},"max":{"1min":1.968,"5min":2.4860000000000002,"15min":2.4860000000000002},"last":0.69599999999999995},{"interface":"front","average":{"1min":1.673,"5min":1.6319999999999999,"15min":1.669},"min":{"1min":1.2270000000000001,"5min":0.95499999999999996,"15min":0.95299999999999996},"max":{"1min":2.214,"5min":2.4039999999999999,"15min":2.4039999999999999},"last":0.92200000000000004}]},{"osd":7,"last update":"Wed Aug 9 15:11:03 2023","interfaces":[{"interface":"back","average":{"1min":1.8069999999999999,"5min":1.738,"15min":1.762},"min":{"1min":1.4239999999999999,"5min":1.123,"15min":1.123},"max":{"1min":2.347,"5min":2.347,"15min":2.347},"last":1.1259999999999999},{"interface":"front","average":{"1min":1.635,"5min":1.6779999999999999,"15min":1.718},"min":{"1min":1.3620000000000001,"5min":0.84899999999999998,"15min":0.84899999999999998},"max":{"1min":1.8480000000000001,"5min":3.3220000000000001,"15min":3.3220000000000001},"last":1.1539999999999999}]}]},{"osd":2,"up_from":13,"seq":55834576013,"num_pgs":6,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":42024,"kb_used_data":2716,"kb_used_omap":1,"kb_used_meta":39294,"kb_avail":94329816,"statfs":{"total":96636764160,"available":96593731584,"internally_reserved":0,"allocated":2781184,"data_stored":1515130,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1781,"internal_metadata":40237323},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Aug 9 15:10:49 2023","interfaces":[{"interface":"back","average":{"1min":1.081,"5min":0.98699999999999999,"15min":0.96399999999999997},"min":{"1min":0.72399999999999998,"5min":0.44800000000000001,"15min":0.44800000000000001},"max":{"1min":1.556,"5min":1.556,"15min":1.587},"last":0.85599999999999998},{"interface":"front","average":{"1min":1.111,"5min":1.101,"15min":1.0940000000000001},"min":{"1min":0.82099999999999995,"5min":0.59499999999999997,"15min":0.59499999999999997},"max":{"1min":1.629,"5min":1.764,"15min":2.052},"last":1.141}]},{"osd":1,"last update":"Wed Aug 9 15:10:49 2023","interfaces":[{"interface":"back","average":{"1min":1.3740000000000001,"5min":1.288,"15min":1.2490000000000001},"min":{"1min":1.127,"5min":0.72399999999999998,"15min":0.72399999999999998},"max":{"1min":1.5449999999999999,"5min":2.0179999999999998,"15min":2.0179999999999998},"last":1.089},{"interface":"front","average":{"1min":1.452,"5min":1.3480000000000001,"15min":1.4059999999999999},"min":{"1min":1.1459999999999999,"5min":0.67900000000000005,"15min":0.67900000000000005},"max":{"1min":1.6870000000000001,"5min":1.95,"15min":1.95},"last":1.276}]},{"osd":3,"last update":"Wed Aug 9 15:10:49 2023","interfaces":[{"interface":"back","average":{"1min":1.3029999999999999,"5min":1.111,"15min":1.111},"min":{"1min":0.89100000000000001,"5min":0.52100000000000002,"15min":0.52100000000000002},"max":{"1min":1.802,"5min":1.871,"15min":1.871},"last":0.90200000000000002},{"interface":"front","average":{"1min":1.46,"5min":1.3939999999999999,"15min":1.385},"min":{"1min":0.94499999999999995,"5min":0.64800000000000002,"15min":0.64800000000000002},"max":{"1min":1.9299999999999999,"5min":1.9299999999999999,"15min":2.0470000000000002},"last":1.1990000000000001}]},{"osd":4,"last update":"Wed Aug 9 15:10:49 2023","interfaces":[{"interface":"back","average":{"1min":1.659,"5min":1.5720000000000001,"15min":1.5569999999999999},"min":{"1min":1.355,"5min":0.90000000000000002,"15min":0.90000000000000002},"max":{"1min":1.9790000000000001,"5min":2.0649999999999999,"15min":2.464},"last":1.6950000000000001},{"interface":"front","average":{"1min":1.4770000000000001,"5min":1.472,"15min":1.466},"min":{"1min":1.153,"5min":0.81799999999999995,"15min":0.81799999999999995},"max":{"1min":1.8400000000000001,"5min":2.0720000000000001,"15min":2.0720000000000001},"last":1.649}]},{"osd":5,"last update":"Wed Aug 9 15:10:49 2023","interfaces":[{"interface":"back","average":{"1min":1.496,"5min":1.538,"15min":1.528},"min":{"1min":1.177,"5min":0.82799999999999996,"15min":0.82799999999999996},"max":{"1min":1.879,"5min":2.0369999999999999,"15min":2.1139999999999999},"last":1.804},{"interface":"front","average":{"1min":1.5269999999999999,"5min":1.4530000000000001,"15min":1.496},"min":{"1min":1.2310000000000001,"5min":0.93999999999999995,"15min":0.93999999999999995},"max":{"1min":1.968,"5min":1.968,"15min":2.214},"last":1.732}]},{"osd":6,"last update":"Wed Aug 9 15:10:49 2023","interfaces":[{"interface":"back","average":{"1min":1.5349999999999999,"5min":1.5129999999999999,"15min":1.528},"min":{"1min":1.0820000000000001,"5min":1.0680000000000001,"15min":1.022},"max":{"1min":2.1110000000000002,"5min":2.1110000000000002,"15min":2.2999999999999998},"last":1.5629999999999999},{"interface":"front","average":{"1min":1.7709999999999999,"5min":1.677,"15min":1.645},"min":{"1min":1.28,"5min":1.0649999999999999,"15min":1.0649999999999999},"max":{"1min":2.077,"5min":2.177,"15min":2.351},"last":1.7709999999999999}]},{"osd":7,"last update":"Wed Aug 9 15:10:49 2023","interfaces":[{"interface":"back","average":{"1min":1.6870000000000001,"5min":1.6419999999999999,"15min":1.6439999999999999},"min":{"1min":1.3129999999999999,"5min":0.87,"15min":0.87},"max":{"1min":2.0840000000000001,"5min":2.302,"15min":2.302},"last":1.8999999999999999},{"interface":"front","average":{"1min":1.6910000000000001,"5min":1.6259999999999999,"15min":1.637},"min":{"1min":1.4139999999999999,"5min":1.0189999999999999,"15min":1.0189999999999999},"max":{"1min":2.242,"5min":2.3809999999999998,"15min":2.3809999999999998},"last":1.6040000000000001}]}]},{"osd":1,"up_from":13,"seq":55834576013,"num_pgs":9,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":56564,"kb_used_data":3304,"kb_used_omap":1,"kb_used_meta":53246,"kb_avail":94315276,"statfs":{"total":96636764160,"available":96578842624,"internally_reserved":0,"allocated":3383296,"data_stored":2105554,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1781,"internal_metadata":54524171},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Aug 9 15:11:02 2023","interfaces":[{"interface":"back","average":{"1min":1.0529999999999999,"5min":1.006,"15min":1.0129999999999999},"min":{"1min":0.77200000000000002,"5min":0.42299999999999999,"15min":0.42299999999999999},"max":{"1min":1.8799999999999999,"5min":1.8799999999999999,"15min":2.0939999999999999},"last":1.139},{"interface":"front","average":{"1min":1.2749999999999999,"5min":1.1479999999999999,"15min":1.147},"min":{"1min":0.86399999999999999,"5min":0.46500000000000002,"15min":0.46500000000000002},"max":{"1min":1.7549999999999999,"5min":1.78,"15min":1.8939999999999999},"last":1.212}]},{"osd":2,"last update":"Wed Aug 9 15:11:02 2023","interfaces":[{"interface":"back","average":{"1min":1.03,"5min":1.048,"15min":1.0469999999999999},"min":{"1min":0.77000000000000002,"5min":0.55800000000000005,"15min":0.55800000000000005},"max":{"1min":1.361,"5min":1.589,"15min":2.3940000000000001},"last":2.2240000000000002},{"interface":"front","average":{"1min":1.3839999999999999,"5min":1.2509999999999999,"15min":1.2789999999999999},"min":{"1min":0.94699999999999995,"5min":0.46700000000000003,"15min":0.46700000000000003},"max":{"1min":2.169,"5min":2.169,"15min":2.169},"last":1.2829999999999999}]},{"osd":3,"last update":"Wed Aug 9 15:11:02 2023","interfaces":[{"interface":"back","average":{"1min":1.5409999999999999,"5min":1.397,"15min":1.4039999999999999},"min":{"1min":1.0860000000000001,"5min":0.51000000000000001,"15min":0.51000000000000001},"max":{"1min":2.093,"5min":2.093,"15min":2.109},"last":1.569},{"interface":"front","average":{"1min":1.3879999999999999,"5min":1.329,"15min":1.331},"min":{"1min":1.0189999999999999,"5min":0.66000000000000003,"15min":0.66000000000000003},"max":{"1min":2.052,"5min":2.052,"15min":2.052},"last":1.8680000000000001}]},{"osd":4,"last update":"Wed Aug 9 15:11:02 2023","interfaces":[{"interface":"back","average":{"1min":1.53,"5min":1.4950000000000001,"15min":1.4890000000000001},"min":{"1min":1.145,"5min":0.753,"15min":0.753},"max":{"1min":2.0419999999999998,"5min":2.0419999999999998,"15min":2.214},"last":1.6180000000000001},{"interface":"front","average":{"1min":1.45,"5min":1.4299999999999999,"15min":1.423},"min":{"1min":0.997,"5min":0.69199999999999995,"15min":0.69199999999999995},"max":{"1min":2.0310000000000001,"5min":2.0310000000000001,"15min":2.1190000000000002},"last":1.496}]},{"osd":5,"last update":"Wed Aug 9 15:11:02 2023","interfaces":[{"interface":"back","average":{"1min":1.448,"5min":1.4550000000000001,"15min":1.46},"min":{"1min":1.071,"5min":0.77800000000000002,"15min":0.77800000000000002},"max":{"1min":1.994,"5min":1.994,"15min":2.508},"last":1.909},{"interface":"front","average":{"1min":1.663,"5min":1.603,"15min":1.607},"min":{"1min":1.238,"5min":0.79100000000000004,"15min":0.79100000000000004},"max":{"1min":2.3809999999999998,"5min":2.3809999999999998,"15min":2.3809999999999998},"last":1.99}]},{"osd":6,"last update":"Wed Aug 9 15:11:02 2023","interfaces":[{"interface":"back","average":{"1min":1.706,"5min":1.647,"15min":1.6299999999999999},"min":{"1min":1.298,"5min":0.85199999999999998,"15min":0.85199999999999998},"max":{"1min":2.347,"5min":2.347,"15min":2.347},"last":2.1669999999999998},{"interface":"front","average":{"1min":1.617,"5min":1.577,"15min":1.5880000000000001},"min":{"1min":1.147,"5min":0.73499999999999999,"15min":0.73499999999999999},"max":{"1min":2.218,"5min":2.218,"15min":2.278},"last":1.7030000000000001}]},{"osd":7,"last update":"Wed Aug 9 15:11:02 2023","interfaces":[{"interface":"back","average":{"1min":1.744,"5min":1.7210000000000001,"15min":1.714},"min":{"1min":1.446,"5min":0.86399999999999999,"15min":0.86399999999999999},"max":{"1min":2.3090000000000002,"5min":2.3090000000000002,"15min":2.488},"last":2.1299999999999999},{"interface":"front","average":{"1min":1.78,"5min":1.7190000000000001,"15min":1.7150000000000001},"min":{"1min":1.4019999999999999,"5min":0.93400000000000005,"15min":0.93400000000000005},"max":{"1min":2.4260000000000002,"5min":2.4260000000000002,"15min":2.4260000000000002},"last":2.0579999999999998}]}]},{"osd":0,"up_from":13,"seq":55834576013,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":29736,"kb_used_data":2716,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94342104,"statfs":{"total":96636764160,"available":96606314496,"internally_reserved":0,"allocated":2781184,"data_stored":1513859,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1781,"internal_metadata":27654411},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Wed Aug 9 15:10:59 2023","interfaces":[{"interface":"back","average":{"1min":0.95599999999999996,"5min":0.90800000000000003,"15min":0.95499999999999996},"min":{"1min":0.65300000000000002,"5min":0.40000000000000002,"15min":0.40000000000000002},"max":{"1min":1.5680000000000001,"5min":1.617,"15min":1.6299999999999999},"last":0.93200000000000005},{"interface":"front","average":{"1min":1.161,"5min":1.1220000000000001,"15min":1.153},"min":{"1min":0.61699999999999999,"5min":0.61699999999999999,"15min":0.61699999999999999},"max":{"1min":2.1560000000000001,"5min":2.1560000000000001,"15min":2.1560000000000001},"last":1.105}]},{"osd":2,"last update":"Wed Aug 9 15:10:59 2023","interfaces":[{"interface":"back","average":{"1min":1.163,"5min":1.1799999999999999,"15min":1.1970000000000001},"min":{"1min":0.73299999999999998,"5min":0.505,"15min":0.505},"max":{"1min":1.5960000000000001,"5min":2.4769999999999999,"15min":2.4769999999999999},"last":1.401},{"interface":"front","average":{"1min":1.238,"5min":1.1850000000000001,"15min":1.236},"min":{"1min":0.80000000000000004,"5min":0.53100000000000003,"15min":0.53100000000000003},"max":{"1min":1.8180000000000001,"5min":1.8180000000000001,"15min":1.9470000000000001},"last":1.26}]},{"osd":3,"last update":"Wed Aug 9 15:10:59 2023","interfaces":[{"interface":"back","average":{"1min":1.1359999999999999,"5min":1.173,"15min":1.2170000000000001},"min":{"1min":0.65800000000000003,"5min":0.56299999999999994,"15min":0.56299999999999994},"max":{"1min":1.492,"5min":2.1459999999999999,"15min":2.1459999999999999},"last":1.23},{"interface":"front","average":{"1min":1.407,"5min":1.347,"15min":1.4019999999999999},"min":{"1min":0.79700000000000004,"5min":0.59999999999999998,"15min":0.59999999999999998},"max":{"1min":1.867,"5min":1.867,"15min":2.2469999999999999},"last":1.3180000000000001}]},{"osd":4,"last update":"Wed Aug 9 15:10:59 2023","interfaces":[{"interface":"back","average":{"1min":1.339,"5min":1.371,"15min":1.417},"min":{"1min":1.1060000000000001,"5min":0.88200000000000001,"15min":0.88200000000000001},"max":{"1min":1.825,"5min":1.853,"15min":1.966},"last":1.044},{"interface":"front","average":{"1min":1.484,"5min":1.476,"15min":1.52},"min":{"1min":0.93999999999999995,"5min":0.79400000000000004,"15min":0.79400000000000004},"max":{"1min":1.972,"5min":2.0369999999999999,"15min":2.278},"last":1.5069999999999999}]},{"osd":5,"last update":"Wed Aug 9 15:10:59 2023","interfaces":[{"interface":"back","average":{"1min":1.5580000000000001,"5min":1.5269999999999999,"15min":1.54},"min":{"1min":1.2809999999999999,"5min":0.96299999999999997,"15min":0.96299999999999997},"max":{"1min":1.9139999999999999,"5min":2.355,"15min":2.355},"last":1.5589999999999999},{"interface":"front","average":{"1min":1.46,"5min":1.4810000000000001,"15min":1.506},"min":{"1min":0.93200000000000005,"5min":0.82899999999999996,"15min":0.82899999999999996},"max":{"1min":1.9019999999999999,"5min":2.0800000000000001,"15min":2.3740000000000001},"last":1.367}]},{"osd":6,"last update":"Wed Aug 9 15:10:59 2023","interfaces":[{"interface":"back","average":{"1min":1.5669999999999999,"5min":1.603,"15min":1.637},"min":{"1min":1.0740000000000001,"5min":1.0740000000000001,"15min":1.0740000000000001},"max":{"1min":2.0369999999999999,"5min":2.7490000000000001,"15min":2.7490000000000001},"last":1.599},{"interface":"front","average":{"1min":1.583,"5min":1.591,"15min":1.621},"min":{"1min":1.1319999999999999,"5min":0.84699999999999998,"15min":0.84699999999999998},"max":{"1min":2.077,"5min":2.274,"15min":2.2839999999999998},"last":1.9450000000000001}]},{"osd":7,"last update":"Wed Aug 9 15:10:59 2023","interfaces":[{"interface":"back","average":{"1min":1.6479999999999999,"5min":1.641,"15min":1.6859999999999999},"min":{"1min":0.94899999999999995,"5min":0.94899999999999995,"15min":0.94899999999999995},"max":{"1min":2.012,"5min":2.645,"15min":2.645},"last":1.8340000000000001},{"interface":"front","average":{"1min":1.6100000000000001,"5min":1.6120000000000001,"15min":1.6259999999999999},"min":{"1min":1.2110000000000001,"5min":0.89100000000000001,"15min":0.89100000000000001},"max":{"1min":1.9530000000000001,"5min":2.2000000000000002,"15min":2.3759999999999999},"last":1.4299999999999999}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":26,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":26,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":56,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":26,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":26,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":182,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":26,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":1716,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":26,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":26,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":26,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":27,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":27,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":27,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":27,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":27,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":27,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":27,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":27,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-08-09T15:11:33.314 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-08-09T15:11:33.568 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:33.568+0000 7f4e2bfe1640 1 Processor -- start 2023-08-09T15:11:33.569 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:33.568+0000 7f4e2bfe1640 1 -- start start 2023-08-09T15:11:33.570 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:33.570+0000 7f4e2bfe1640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f4e24143c20 0x7f4e2406f540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T15:11:33.571 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:33.570+0000 7f4e2bfe1640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f4e240c86f0 0x7f4e240c8b50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T15:11:33.571 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:33.570+0000 7f4e2bfe1640 1 -- --> v1:172.21.15.203:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4e240c9090 con 0x7f4e2406c8d0 2023-08-09T15:11:33.571 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:33.570+0000 7f4e2bfe1640 1 -- --> v2:172.21.15.43:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4e24144000 con 0x7f4e24143c20 2023-08-09T15:11:33.571 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:33.570+0000 7f4e2bfe1640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4e2406c440 con 0x7f4e240c86f0 2023-08-09T15:11:33.571 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:33.570+0000 7f4e29d7d640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f4e24143c20 0x7f4e2406f540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T15:11:33.571 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:33.570+0000 7f4e29d7d640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f4e24143c20 0x7f4e2406f540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:56160/0 (socket says 172.21.15.43:56160) 2023-08-09T15:11:33.572 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:33.570+0000 7f4e29d7d640 1 -- 172.21.15.43:0/1970525064 learned_addr learned my addr 172.21.15.43:0/1970525064 (peer_addr_for_me v2:172.21.15.43:0/0) 2023-08-09T15:11:33.573 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:33.571+0000 7f4e29d7d640 1 -- 172.21.15.43:0/1970525064 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f4e240c86f0 msgr2=0x7f4e240c8b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-09T15:11:33.573 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:33.571+0000 7f4e2957c640 1 --2- 172.21.15.43:0/1970525064 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f4e240c86f0 0x7f4e240c8b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T15:11:33.573 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:33.571+0000 7f4e29d7d640 1 --2- 172.21.15.43:0/1970525064 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f4e240c86f0 0x7f4e240c8b50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T15:11:33.573 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:33.571+0000 7f4e29d7d640 1 -- 172.21.15.43:0/1970525064 >> v1:172.21.15.203:6789/0 conn(0x7f4e2406c8d0 legacy=0x7f4e2406cd60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-09T15:11:33.573 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:33.571+0000 7f4e29d7d640 1 -- 172.21.15.43:0/1970525064 --> v2:172.21.15.43:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4e2406b830 con 0x7f4e24143c20 2023-08-09T15:11:33.573 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:33.571+0000 7f4e2957c640 1 --2- 172.21.15.43:0/1970525064 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f4e240c86f0 0x7f4e240c8b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-08-09T15:11:33.574 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:33.572+0000 7f4e29d7d640 1 --2- 172.21.15.43:0/1970525064 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f4e24143c20 0x7f4e2406f540 secure :-1 s=READY pgs=2411 cs=0 l=1 rev1=1 crypto rx=0x7f4e20009870 tx=0x7f4e2002b860 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fbe803280ab23617 server_cookie=0 in_seq=0 out_seq=0 2023-08-09T15:11:33.574 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:33.572+0000 7f4e28d7b640 1 -- 172.21.15.43:0/1970525064 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4e200317c0 con 0x7f4e24143c20 2023-08-09T15:11:33.574 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:33.572+0000 7f4e28d7b640 1 -- 172.21.15.43:0/1970525064 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4e20031920 con 0x7f4e24143c20 2023-08-09T15:11:33.574 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:33.572+0000 7f4e28d7b640 1 -- 172.21.15.43:0/1970525064 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4e20031cb0 con 0x7f4e24143c20 2023-08-09T15:11:33.575 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:33.573+0000 7f4e2bfe1640 1 -- 172.21.15.43:0/1970525064 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f4e24143c20 msgr2=0x7f4e2406f540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T15:11:33.575 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:33.573+0000 7f4e2bfe1640 1 --2- 172.21.15.43:0/1970525064 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f4e24143c20 0x7f4e2406f540 secure :-1 s=READY pgs=2411 cs=0 l=1 rev1=1 crypto rx=0x7f4e20009870 tx=0x7f4e2002b860 comp rx=0 tx=0).stop 2023-08-09T15:11:33.575 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:33.573+0000 7f4e2bfe1640 1 -- 172.21.15.43:0/1970525064 shutdown_connections 2023-08-09T15:11:33.575 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:33.573+0000 7f4e2bfe1640 1 --2- 172.21.15.43:0/1970525064 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f4e240c86f0 0x7f4e240c8b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T15:11:33.576 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:33.573+0000 7f4e2bfe1640 1 --2- 172.21.15.43:0/1970525064 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f4e24143c20 0x7f4e2406f540 unknown :-1 s=CLOSED pgs=2411 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T15:11:33.576 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:33.573+0000 7f4e2bfe1640 1 -- 172.21.15.43:0/1970525064 >> 172.21.15.43:0/1970525064 conn(0x7f4e240ce0a0 msgr2=0x7f4e240ce480 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-09T15:11:33.576 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:33.573+0000 7f4e2bfe1640 1 -- 172.21.15.43:0/1970525064 shutdown_connections 2023-08-09T15:11:33.576 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:33.573+0000 7f4e2bfe1640 1 -- 172.21.15.43:0/1970525064 wait complete. 2023-08-09T15:11:33.577 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:33.575+0000 7f4e2bfe1640 1 Processor -- start 2023-08-09T15:11:33.578 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:33.575+0000 7f4e2bfe1640 1 -- start start 2023-08-09T15:11:33.578 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:33.576+0000 7f4e2bfe1640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f4e2406c8d0 0x7f4e2412a440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T15:11:33.578 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:33.576+0000 7f4e2bfe1640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f4e240c86f0 0x7f4e240c75b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T15:11:33.578 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:33.576+0000 7f4e2bfe1640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f4e24143c20 0x7f4e240c7af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T15:11:33.578 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:33.576+0000 7f4e2bfe1640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4e240c7190 con 0x7f4e2406c8d0 2023-08-09T15:11:33.578 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:33.576+0000 7f4e2bfe1640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4e240c6510 con 0x7f4e240c86f0 2023-08-09T15:11:33.579 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:33.576+0000 7f4e2bfe1640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4e240c67f0 con 0x7f4e24143c20 2023-08-09T15:11:33.579 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:33.577+0000 7f4e29d7d640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f4e2406c8d0 0x7f4e2412a440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T15:11:33.579 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:33.577+0000 7f4e29d7d640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f4e2406c8d0 0x7f4e2412a440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:56166/0 (socket says 172.21.15.43:56166) 2023-08-09T15:11:33.579 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:33.577+0000 7f4e29d7d640 1 -- 172.21.15.43:0/4014392743 learned_addr learned my addr 172.21.15.43:0/4014392743 (peer_addr_for_me v2:172.21.15.43:0/0) 2023-08-09T15:11:33.579 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:33.577+0000 7f4e2957c640 1 --2- 172.21.15.43:0/4014392743 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f4e240c86f0 0x7f4e240c75b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T15:11:33.579 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:33.577+0000 7f4e29d7d640 1 -- 172.21.15.43:0/4014392743 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f4e24143c20 msgr2=0x7f4e240c7af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T15:11:33.579 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:33.577+0000 7f4e29d7d640 1 --2- 172.21.15.43:0/4014392743 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f4e24143c20 0x7f4e240c7af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T15:11:33.580 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:33.577+0000 7f4e29d7d640 1 -- 172.21.15.43:0/4014392743 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f4e240c86f0 msgr2=0x7f4e240c75b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T15:11:33.580 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:33.577+0000 7f4e29d7d640 1 --2- 172.21.15.43:0/4014392743 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f4e240c86f0 0x7f4e240c75b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T15:11:33.580 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:33.577+0000 7f4e29d7d640 1 -- 172.21.15.43:0/4014392743 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4e200094d0 con 0x7f4e2406c8d0 2023-08-09T15:11:33.580 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:33.577+0000 7f4e29d7d640 1 --2- 172.21.15.43:0/4014392743 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f4e2406c8d0 0x7f4e2412a440 secure :-1 s=READY pgs=2412 cs=0 l=1 rev1=1 crypto rx=0x7f4e2002b8a0 tx=0x7f4e200046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-09T15:11:33.581 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:33.578+0000 7f4e1a7fc640 1 -- 172.21.15.43:0/4014392743 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4e200329a0 con 0x7f4e2406c8d0 2023-08-09T15:11:33.581 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:33.578+0000 7f4e1a7fc640 1 -- 172.21.15.43:0/4014392743 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4e20032b40 con 0x7f4e2406c8d0 2023-08-09T15:11:33.582 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:33.578+0000 7f4e1a7fc640 1 -- 172.21.15.43:0/4014392743 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4e200453f0 con 0x7f4e2406c8d0 2023-08-09T15:11:33.582 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:33.578+0000 7f4e2bfe1640 1 -- 172.21.15.43:0/4014392743 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4e240c6ad0 con 0x7f4e2406c8d0 2023-08-09T15:11:33.582 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:33.578+0000 7f4e2bfe1640 1 -- 172.21.15.43:0/4014392743 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4e240c61c0 con 0x7f4e2406c8d0 2023-08-09T15:11:33.582 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:33.579+0000 7f4e2bfe1640 1 -- 172.21.15.43:0/4014392743 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4e240739f0 con 0x7f4e2406c8d0 2023-08-09T15:11:33.582 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:33.580+0000 7f4e1a7fc640 1 -- 172.21.15.43:0/4014392743 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 118) v1 ==== 88443+0+0 (secure 0 0 0) 0x7f4e2004e450 con 0x7f4e2406c8d0 2023-08-09T15:11:33.582 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:33.581+0000 7f4e1a7fc640 1 --2- 172.21.15.43:0/4014392743 >> [v2:172.21.15.203:6832/4034093268,v1:172.21.15.203:6833/4034093268] conn(0x7f4e0806a170 0x7f4e0806c630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T15:11:33.582 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:33.581+0000 7f4e1a7fc640 1 -- 172.21.15.43:0/4014392743 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(223..223 src has 1..223) v4 ==== 6426+0+0 (secure 0 0 0) 0x7f4e200aac20 con 0x7f4e2406c8d0 2023-08-09T15:11:33.583 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:33.582+0000 7f4e2957c640 1 --2- 172.21.15.43:0/4014392743 >> [v2:172.21.15.203:6832/4034093268,v1:172.21.15.203:6833/4034093268] conn(0x7f4e0806a170 0x7f4e0806c630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T15:11:33.583 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:33.583+0000 7f4e2957c640 1 --2- 172.21.15.43:0/4014392743 >> [v2:172.21.15.203:6832/4034093268,v1:172.21.15.203:6833/4034093268] conn(0x7f4e0806a170 0x7f4e0806c630 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f4e2412c4a0 tx=0x7f4e14008040 comp rx=0 tx=0).ready entity=mgr.8833 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-09T15:11:33.586 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:33.586+0000 7f4e1a7fc640 1 -- 172.21.15.43:0/4014392743 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178066 (secure 0 0 0) 0x7f4e20076080 con 0x7f4e2406c8d0 2023-08-09T15:11:33.651 INFO:tasks.ceph.mds.a.smithi043.stderr:2023-08-09T15:11:33.651+0000 7f97818b1640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2023-08-09T15:11:33.779 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:33.779+0000 7f4e2bfe1640 1 -- 172.21.15.43:0/4014392743 --> [v2:172.21.15.203:6832/4034093268,v1:172.21.15.203:6833/4034093268] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f4e2406bdf0 con 0x7f4e0806a170 2023-08-09T15:11:33.803 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:33.802+0000 7f4e1a7fc640 1 -- 172.21.15.43:0/4014392743 <== mgr.8833 v2:172.21.15.203:6832/4034093268 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+308958 (secure 0 0 0) 0x7f4e1400c040 con 0x7f4e0806a170 2023-08-09T15:11:33.805 INFO:teuthology.orchestra.run.smithi043.stdout: 2023-08-09T15:11:33.808 INFO:teuthology.orchestra.run.smithi043.stderr:dumped all 2023-08-09T15:11:33.810 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:33.809+0000 7f4e2bfe1640 1 -- 172.21.15.43:0/4014392743 >> [v2:172.21.15.203:6832/4034093268,v1:172.21.15.203:6833/4034093268] conn(0x7f4e0806a170 msgr2=0x7f4e0806c630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T15:11:33.810 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:33.809+0000 7f4e2bfe1640 1 --2- 172.21.15.43:0/4014392743 >> [v2:172.21.15.203:6832/4034093268,v1:172.21.15.203:6833/4034093268] conn(0x7f4e0806a170 0x7f4e0806c630 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f4e2412c4a0 tx=0x7f4e14008040 comp rx=0 tx=0).stop 2023-08-09T15:11:33.810 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:33.809+0000 7f4e2bfe1640 1 -- 172.21.15.43:0/4014392743 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f4e2406c8d0 msgr2=0x7f4e2412a440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T15:11:33.810 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:33.809+0000 7f4e2bfe1640 1 --2- 172.21.15.43:0/4014392743 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f4e2406c8d0 0x7f4e2412a440 secure :-1 s=READY pgs=2412 cs=0 l=1 rev1=1 crypto rx=0x7f4e2002b8a0 tx=0x7f4e200046c0 comp rx=0 tx=0).stop 2023-08-09T15:11:33.810 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:33.809+0000 7f4e2bfe1640 1 -- 172.21.15.43:0/4014392743 shutdown_connections 2023-08-09T15:11:33.811 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:33.809+0000 7f4e2bfe1640 1 --2- 172.21.15.43:0/4014392743 >> [v2:172.21.15.203:6832/4034093268,v1:172.21.15.203:6833/4034093268] conn(0x7f4e0806a170 0x7f4e0806c630 unknown :-1 s=CLOSED pgs=65 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T15:11:33.811 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:33.809+0000 7f4e2bfe1640 1 --2- 172.21.15.43:0/4014392743 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f4e24143c20 0x7f4e240c7af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T15:11:33.811 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:33.809+0000 7f4e2bfe1640 1 --2- 172.21.15.43:0/4014392743 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f4e240c86f0 0x7f4e240c75b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T15:11:33.811 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:33.809+0000 7f4e2bfe1640 1 --2- 172.21.15.43:0/4014392743 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f4e2406c8d0 0x7f4e2412a440 unknown :-1 s=CLOSED pgs=2412 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T15:11:33.811 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:33.809+0000 7f4e2bfe1640 1 -- 172.21.15.43:0/4014392743 >> 172.21.15.43:0/4014392743 conn(0x7f4e240ce0a0 msgr2=0x7f4e2415cb90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-09T15:11:33.811 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:33.809+0000 7f4e2bfe1640 1 -- 172.21.15.43:0/4014392743 shutdown_connections 2023-08-09T15:11:33.811 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:33.809+0000 7f4e2bfe1640 1 -- 172.21.15.43:0/4014392743 wait complete. 2023-08-09T15:11:33.830 INFO:teuthology.orchestra.run.smithi043.stdout:{"pg_ready":true,"pg_map":{"version":27,"stamp":"2023-08-09T15:11:32.387382+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":23,"num_object_clones":0,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":0,"num_read":1963,"num_read_kb":4096,"num_write":587,"num_write_kb":2430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":367,"ondisk_log_size":367,"up":210,"acting":210,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":105,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":384252,"kb_used_data":22928,"kb_used_omap":13,"kb_used_meta":361202,"kb_avail":754590468,"statfs":{"total":773094113280,"available":772700639232,"internally_reserved":0,"allocated":23478272,"data_stored":13295800,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":14248,"internal_metadata":369870936},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":27,"apply_latency_ms":27,"commit_latency_ns":27000000,"apply_latency_ns":27000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"5.734891"},"pg_stats":[{"pgid":"27.3f","version":"0'0","reported_seq":9,"reported_epoch":222,"state":"active+clean","last_fresh":"2023-08-09T15:11:28.730392+0000","last_change":"2023-08-09T15:11:28.728482+0000","last_active":"2023-08-09T15:11:28.730392+0000","last_peered":"2023-08-09T15:11:28.730392+0000","last_clean":"2023-08-09T15:11:28.730392+0000","last_became_active":"2023-08-09T15:11:28.728034+0000","last_became_peered":"2023-08-09T15:11:28.728034+0000","last_unstale":"2023-08-09T15:11:28.730392+0000","last_undegraded":"2023-08-09T15:11:28.730392+0000","last_fullsized":"2023-08-09T15:11:28.730392+0000","mapping_epoch":221,"log_start":"0'0","ondisk_log_start":"0'0","created":221,"last_epoch_clean":222,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T15:11:27.650831+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T15:11:27.650831+0000","last_clean_scrub_stamp":"2023-08-09T15:11:27.650831+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-11T01:35:23.418645+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"27.3e","version":"0'0","reported_seq":9,"reported_epoch":222,"state":"active+clean","last_fresh":"2023-08-09T15:11:28.738466+0000","last_change":"2023-08-09T15:11:28.737777+0000","last_active":"2023-08-09T15:11:28.738466+0000","last_peered":"2023-08-09T15:11:28.738466+0000","last_clean":"2023-08-09T15:11:28.738466+0000","last_became_active":"2023-08-09T15:11:28.737460+0000","last_became_peered":"2023-08-09T15:11:28.737460+0000","last_unstale":"2023-08-09T15:11:28.738466+0000","last_undegraded":"2023-08-09T15:11:28.738466+0000","last_fullsized":"2023-08-09T15:11:28.738466+0000","mapping_epoch":221,"log_start":"0'0","ondisk_log_start":"0'0","created":221,"last_epoch_clean":222,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T15:11:27.650831+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T15:11:27.650831+0000","last_clean_scrub_stamp":"2023-08-09T15:11:27.650831+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T20:46:57.262034+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"27.3d","version":"0'0","reported_seq":9,"reported_epoch":222,"state":"active+clean","last_fresh":"2023-08-09T15:11:28.736836+0000","last_change":"2023-08-09T15:11:28.730121+0000","last_active":"2023-08-09T15:11:28.736836+0000","last_peered":"2023-08-09T15:11:28.736836+0000","last_clean":"2023-08-09T15:11:28.736836+0000","last_became_active":"2023-08-09T15:11:28.729853+0000","last_became_peered":"2023-08-09T15:11:28.729853+0000","last_unstale":"2023-08-09T15:11:28.736836+0000","last_undegraded":"2023-08-09T15:11:28.736836+0000","last_fullsized":"2023-08-09T15:11:28.736836+0000","mapping_epoch":221,"log_start":"0'0","ondisk_log_start":"0'0","created":221,"last_epoch_clean":222,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T15:11:27.650831+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T15:11:27.650831+0000","last_clean_scrub_stamp":"2023-08-09T15:11:27.650831+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T23:05:56.110390+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"27.3c","version":"0'0","reported_seq":9,"reported_epoch":222,"state":"active+clean","last_fresh":"2023-08-09T15:11:28.741424+0000","last_change":"2023-08-09T15:11:28.739877+0000","last_active":"2023-08-09T15:11:28.741424+0000","last_peered":"2023-08-09T15:11:28.741424+0000","last_clean":"2023-08-09T15:11:28.741424+0000","last_became_active":"2023-08-09T15:11:28.739598+0000","last_became_peered":"2023-08-09T15:11:28.739598+0000","last_unstale":"2023-08-09T15:11:28.741424+0000","last_undegraded":"2023-08-09T15:11:28.741424+0000","last_fullsized":"2023-08-09T15:11:28.741424+0000","mapping_epoch":221,"log_start":"0'0","ondisk_log_start":"0'0","created":221,"last_epoch_clean":222,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T15:11:27.650831+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T15:11:27.650831+0000","last_clean_scrub_stamp":"2023-08-09T15:11:27.650831+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T23:13:39.900734+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"27.3b","version":"0'0","reported_seq":9,"reported_epoch":222,"state":"active+clean","last_fresh":"2023-08-09T15:11:28.722482+0000","last_change":"2023-08-09T15:11:28.718442+0000","last_active":"2023-08-09T15:11:28.722482+0000","last_peered":"2023-08-09T15:11:28.722482+0000","last_clean":"2023-08-09T15:11:28.722482+0000","last_became_active":"2023-08-09T15:11:28.718118+0000","last_became_peered":"2023-08-09T15:11:28.718118+0000","last_unstale":"2023-08-09T15:11:28.722482+0000","last_undegraded":"2023-08-09T15:11:28.722482+0000","last_fullsized":"2023-08-09T15:11:28.722482+0000","mapping_epoch":221,"log_start":"0'0","ondisk_log_start":"0'0","created":221,"last_epoch_clean":222,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T15:11:27.650831+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T15:11:27.650831+0000","last_clean_scrub_stamp":"2023-08-09T15:11:27.650831+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T22:19:26.178328+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"27.3a","version":"0'0","reported_seq":9,"reported_epoch":222,"state":"active+clean","last_fresh":"2023-08-09T15:11:28.733344+0000","last_change":"2023-08-09T15:11:28.731519+0000","last_active":"2023-08-09T15:11:28.733344+0000","last_peered":"2023-08-09T15:11:28.733344+0000","last_clean":"2023-08-09T15:11:28.733344+0000","last_became_active":"2023-08-09T15:11:28.731160+0000","last_became_peered":"2023-08-09T15:11:28.731160+0000","last_unstale":"2023-08-09T15:11:28.733344+0000","last_undegraded":"2023-08-09T15:11:28.733344+0000","last_fullsized":"2023-08-09T15:11:28.733344+0000","mapping_epoch":221,"log_start":"0'0","ondisk_log_start":"0'0","created":221,"last_epoch_clean":222,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T15:11:27.650831+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T15:11:27.650831+0000","last_clean_scrub_stamp":"2023-08-09T15:11:27.650831+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T20:47:00.574569+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"27.39","version":"0'0","reported_seq":9,"reported_epoch":222,"state":"active+clean","last_fresh":"2023-08-09T15:11:28.740429+0000","last_change":"2023-08-09T15:11:28.739099+0000","last_active":"2023-08-09T15:11:28.740429+0000","last_peered":"2023-08-09T15:11:28.740429+0000","last_clean":"2023-08-09T15:11:28.740429+0000","last_became_active":"2023-08-09T15:11:28.738829+0000","last_became_peered":"2023-08-09T15:11:28.738829+0000","last_unstale":"2023-08-09T15:11:28.740429+0000","last_undegraded":"2023-08-09T15:11:28.740429+0000","last_fullsized":"2023-08-09T15:11:28.740429+0000","mapping_epoch":221,"log_start":"0'0","ondisk_log_start":"0'0","created":221,"last_epoch_clean":222,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T15:11:27.650831+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T15:11:27.650831+0000","last_clean_scrub_stamp":"2023-08-09T15:11:27.650831+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T23:49:16.381965+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"27.38","version":"0'0","reported_seq":9,"reported_epoch":222,"state":"active+clean","last_fresh":"2023-08-09T15:11:28.722731+0000","last_change":"2023-08-09T15:11:28.719958+0000","last_active":"2023-08-09T15:11:28.722731+0000","last_peered":"2023-08-09T15:11:28.722731+0000","last_clean":"2023-08-09T15:11:28.722731+0000","last_became_active":"2023-08-09T15:11:28.719703+0000","last_became_peered":"2023-08-09T15:11:28.719703+0000","last_unstale":"2023-08-09T15:11:28.722731+0000","last_undegraded":"2023-08-09T15:11:28.722731+0000","last_fullsized":"2023-08-09T15:11:28.722731+0000","mapping_epoch":221,"log_start":"0'0","ondisk_log_start":"0'0","created":221,"last_epoch_clean":222,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T15:11:27.650831+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T15:11:27.650831+0000","last_clean_scrub_stamp":"2023-08-09T15:11:27.650831+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T16:59:53.165135+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"27.37","version":"0'0","reported_seq":9,"reported_epoch":222,"state":"active+clean","last_fresh":"2023-08-09T15:11:28.731327+0000","last_change":"2023-08-09T15:11:28.730386+0000","last_active":"2023-08-09T15:11:28.731327+0000","last_peered":"2023-08-09T15:11:28.731327+0000","last_clean":"2023-08-09T15:11:28.731327+0000","last_became_active":"2023-08-09T15:11:28.730045+0000","last_became_peered":"2023-08-09T15:11:28.730045+0000","last_unstale":"2023-08-09T15:11:28.731327+0000","last_undegraded":"2023-08-09T15:11:28.731327+0000","last_fullsized":"2023-08-09T15:11:28.731327+0000","mapping_epoch":221,"log_start":"0'0","ondisk_log_start":"0'0","created":221,"last_epoch_clean":222,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T15:11:27.650831+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T15:11:27.650831+0000","last_clean_scrub_stamp":"2023-08-09T15:11:27.650831+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T16:44:13.514166+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"27.36","version":"0'0","reported_seq":9,"reported_epoch":222,"state":"active+clean","last_fresh":"2023-08-09T15:11:28.721194+0000","last_change":"2023-08-09T15:11:28.709161+0000","last_active":"2023-08-09T15:11:28.721194+0000","last_peered":"2023-08-09T15:11:28.721194+0000","last_clean":"2023-08-09T15:11:28.721194+0000","last_became_active":"2023-08-09T15:11:28.708761+0000","last_became_peered":"2023-08-09T15:11:28.708761+0000","last_unstale":"2023-08-09T15:11:28.721194+0000","last_undegraded":"2023-08-09T15:11:28.721194+0000","last_fullsized":"2023-08-09T15:11:28.721194+0000","mapping_epoch":221,"log_start":"0'0","ondisk_log_start":"0'0","created":221,"last_epoch_clean":222,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T15:11:27.650831+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T15:11:27.650831+0000","last_clean_scrub_stamp":"2023-08-09T15:11:27.650831+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T17:22:22.512775+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"27.35","version":"0'0","reported_seq":9,"reported_epoch":222,"state":"active+clean","last_fresh":"2023-08-09T15:11:28.738359+0000","last_change":"2023-08-09T15:11:28.736745+0000","last_active":"2023-08-09T15:11:28.738359+0000","last_peered":"2023-08-09T15:11:28.738359+0000","last_clean":"2023-08-09T15:11:28.738359+0000","last_became_active":"2023-08-09T15:11:28.736295+0000","last_became_peered":"2023-08-09T15:11:28.736295+0000","last_unstale":"2023-08-09T15:11:28.738359+0000","last_undegraded":"2023-08-09T15:11:28.738359+0000","last_fullsized":"2023-08-09T15:11:28.738359+0000","mapping_epoch":221,"log_start":"0'0","ondisk_log_start":"0'0","created":221,"last_epoch_clean":222,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T15:11:27.650831+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T15:11:27.650831+0000","last_clean_scrub_stamp":"2023-08-09T15:11:27.650831+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T16:40:16.167053+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"27.34","version":"0'0","reported_seq":9,"reported_epoch":222,"state":"active+clean","last_fresh":"2023-08-09T15:11:28.721496+0000","last_change":"2023-08-09T15:11:28.720577+0000","last_active":"2023-08-09T15:11:28.721496+0000","last_peered":"2023-08-09T15:11:28.721496+0000","last_clean":"2023-08-09T15:11:28.721496+0000","last_became_active":"2023-08-09T15:11:28.720287+0000","last_became_peered":"2023-08-09T15:11:28.720287+0000","last_unstale":"2023-08-09T15:11:28.721496+0000","last_undegraded":"2023-08-09T15:11:28.721496+0000","last_fullsized":"2023-08-09T15:11:28.721496+0000","mapping_epoch":221,"log_start":"0'0","ondisk_log_start":"0'0","created":221,"last_epoch_clean":222,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T15:11:27.650831+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T15:11:27.650831+0000","last_clean_scrub_stamp":"2023-08-09T15:11:27.650831+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T23:50:31.613421+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"27.33","version":"0'0","reported_seq":9,"reported_epoch":222,"state":"active+clean","last_fresh":"2023-08-09T15:11:28.735057+0000","last_change":"2023-08-09T15:11:28.733219+0000","last_active":"2023-08-09T15:11:28.735057+0000","last_peered":"2023-08-09T15:11:28.735057+0000","last_clean":"2023-08-09T15:11:28.735057+0000","last_became_active":"2023-08-09T15:11:28.732900+0000","last_became_peered":"2023-08-09T15:11:28.732900+0000","last_unstale":"2023-08-09T15:11:28.735057+0000","last_undegraded":"2023-08-09T15:11:28.735057+0000","last_fullsized":"2023-08-09T15:11:28.735057+0000","mapping_epoch":221,"log_start":"0'0","ondisk_log_start":"0'0","created":221,"last_epoch_clean":222,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T15:11:27.650831+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T15:11:27.650831+0000","last_clean_scrub_stamp":"2023-08-09T15:11:27.650831+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T23:08:13.925314+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"27.32","version":"0'0","reported_seq":9,"reported_epoch":222,"state":"active+clean","last_fresh":"2023-08-09T15:11:28.721116+0000","last_change":"2023-08-09T15:11:28.718885+0000","last_active":"2023-08-09T15:11:28.721116+0000","last_peered":"2023-08-09T15:11:28.721116+0000","last_clean":"2023-08-09T15:11:28.721116+0000","last_became_active":"2023-08-09T15:11:28.718534+0000","last_became_peered":"2023-08-09T15:11:28.718534+0000","last_unstale":"2023-08-09T15:11:28.721116+0000","last_undegraded":"2023-08-09T15:11:28.721116+0000","last_fullsized":"2023-08-09T15:11:28.721116+0000","mapping_epoch":221,"log_start":"0'0","ondisk_log_start":"0'0","created":221,"last_epoch_clean":222,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T15:11:27.650831+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T15:11:27.650831+0000","last_clean_scrub_stamp":"2023-08-09T15:11:27.650831+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T22:16:22.217326+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"27.31","version":"0'0","reported_seq":9,"reported_epoch":222,"state":"active+clean","last_fresh":"2023-08-09T15:11:28.729261+0000","last_change":"2023-08-09T15:11:28.726825+0000","last_active":"2023-08-09T15:11:28.729261+0000","last_peered":"2023-08-09T15:11:28.729261+0000","last_clean":"2023-08-09T15:11:28.729261+0000","last_became_active":"2023-08-09T15:11:28.726513+0000","last_became_peered":"2023-08-09T15:11:28.726513+0000","last_unstale":"2023-08-09T15:11:28.729261+0000","last_undegraded":"2023-08-09T15:11:28.729261+0000","last_fullsized":"2023-08-09T15:11:28.729261+0000","mapping_epoch":221,"log_start":"0'0","ondisk_log_start":"0'0","created":221,"last_epoch_clean":222,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T15:11:27.650831+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T15:11:27.650831+0000","last_clean_scrub_stamp":"2023-08-09T15:11:27.650831+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T23:43:15.374226+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"27.30","version":"0'0","reported_seq":9,"reported_epoch":222,"state":"active+clean","last_fresh":"2023-08-09T15:11:28.722268+0000","last_change":"2023-08-09T15:11:28.719171+0000","last_active":"2023-08-09T15:11:28.722268+0000","last_peered":"2023-08-09T15:11:28.722268+0000","last_clean":"2023-08-09T15:11:28.722268+0000","last_became_active":"2023-08-09T15:11:28.715862+0000","last_became_peered":"2023-08-09T15:11:28.715862+0000","last_unstale":"2023-08-09T15:11:28.722268+0000","last_undegraded":"2023-08-09T15:11:28.722268+0000","last_fullsized":"2023-08-09T15:11:28.722268+0000","mapping_epoch":221,"log_start":"0'0","ondisk_log_start":"0'0","created":221,"last_epoch_clean":222,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T15:11:27.650831+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T15:11:27.650831+0000","last_clean_scrub_stamp":"2023-08-09T15:11:27.650831+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-11T00:39:07.832326+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"27.2f","version":"0'0","reported_seq":9,"reported_epoch":222,"state":"active+clean","last_fresh":"2023-08-09T15:11:28.730362+0000","last_change":"2023-08-09T15:11:28.728297+0000","last_active":"2023-08-09T15:11:28.730362+0000","last_peered":"2023-08-09T15:11:28.730362+0000","last_clean":"2023-08-09T15:11:28.730362+0000","last_became_active":"2023-08-09T15:11:28.727641+0000","last_became_peered":"2023-08-09T15:11:28.727641+0000","last_unstale":"2023-08-09T15:11:28.730362+0000","last_undegraded":"2023-08-09T15:11:28.730362+0000","last_fullsized":"2023-08-09T15:11:28.730362+0000","mapping_epoch":221,"log_start":"0'0","ondisk_log_start":"0'0","created":221,"last_epoch_clean":222,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T15:11:27.650831+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T15:11:27.650831+0000","last_clean_scrub_stamp":"2023-08-09T15:11:27.650831+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T19:53:18.094118+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"27.2e","version":"0'0","reported_seq":9,"reported_epoch":222,"state":"active+clean","last_fresh":"2023-08-09T15:11:28.729832+0000","last_change":"2023-08-09T15:11:28.726570+0000","last_active":"2023-08-09T15:11:28.729832+0000","last_peered":"2023-08-09T15:11:28.729832+0000","last_clean":"2023-08-09T15:11:28.729832+0000","last_became_active":"2023-08-09T15:11:28.726283+0000","last_became_peered":"2023-08-09T15:11:28.726283+0000","last_unstale":"2023-08-09T15:11:28.729832+0000","last_undegraded":"2023-08-09T15:11:28.729832+0000","last_fullsized":"2023-08-09T15:11:28.729832+0000","mapping_epoch":221,"log_start":"0'0","ondisk_log_start":"0'0","created":221,"last_epoch_clean":222,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T15:11:27.650831+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T15:11:27.650831+0000","last_clean_scrub_stamp":"2023-08-09T15:11:27.650831+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-11T02:28:06.430438+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"27.2d","version":"0'0","reported_seq":9,"reported_epoch":222,"state":"active+clean","last_fresh":"2023-08-09T15:11:28.704837+0000","last_change":"2023-08-09T15:11:28.702907+0000","last_active":"2023-08-09T15:11:28.704837+0000","last_peered":"2023-08-09T15:11:28.704837+0000","last_clean":"2023-08-09T15:11:28.704837+0000","last_became_active":"2023-08-09T15:11:28.702602+0000","last_became_peered":"2023-08-09T15:11:28.702602+0000","last_unstale":"2023-08-09T15:11:28.704837+0000","last_undegraded":"2023-08-09T15:11:28.704837+0000","last_fullsized":"2023-08-09T15:11:28.704837+0000","mapping_epoch":221,"log_start":"0'0","ondisk_log_start":"0'0","created":221,"last_epoch_clean":222,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T15:11:27.650831+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T15:11:27.650831+0000","last_clean_scrub_stamp":"2023-08-09T15:11:27.650831+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T20:03:38.044588+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"27.2c","version":"0'0","reported_seq":9,"reported_epoch":222,"state":"active+clean","last_fresh":"2023-08-09T15:11:28.725409+0000","last_change":"2023-08-09T15:11:28.723326+0000","last_active":"2023-08-09T15:11:28.725409+0000","last_peered":"2023-08-09T15:11:28.725409+0000","last_clean":"2023-08-09T15:11:28.725409+0000","last_became_active":"2023-08-09T15:11:28.722852+0000","last_became_peered":"2023-08-09T15:11:28.722852+0000","last_unstale":"2023-08-09T15:11:28.725409+0000","last_undegraded":"2023-08-09T15:11:28.725409+0000","last_fullsized":"2023-08-09T15:11:28.725409+0000","mapping_epoch":221,"log_start":"0'0","ondisk_log_start":"0'0","created":221,"last_epoch_clean":222,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T15:11:27.650831+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T15:11:27.650831+0000","last_clean_scrub_stamp":"2023-08-09T15:11:27.650831+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T15:46:00.146591+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"27.2b","version":"0'0","reported_seq":9,"reported_epoch":222,"state":"active+clean","last_fresh":"2023-08-09T15:11:28.729328+0000","last_change":"2023-08-09T15:11:28.727762+0000","last_active":"2023-08-09T15:11:28.729328+0000","last_peered":"2023-08-09T15:11:28.729328+0000","last_clean":"2023-08-09T15:11:28.729328+0000","last_became_active":"2023-08-09T15:11:28.727489+0000","last_became_peered":"2023-08-09T15:11:28.727489+0000","last_unstale":"2023-08-09T15:11:28.729328+0000","last_undegraded":"2023-08-09T15:11:28.729328+0000","last_fullsized":"2023-08-09T15:11:28.729328+0000","mapping_epoch":221,"log_start":"0'0","ondisk_log_start":"0'0","created":221,"last_epoch_clean":222,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T15:11:27.650831+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T15:11:27.650831+0000","last_clean_scrub_stamp":"2023-08-09T15:11:27.650831+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-11T02:07:42.345848+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"27.2a","version":"0'0","reported_seq":9,"reported_epoch":222,"state":"active+clean","last_fresh":"2023-08-09T15:11:28.736601+0000","last_change":"2023-08-09T15:11:28.729939+0000","last_active":"2023-08-09T15:11:28.736601+0000","last_peered":"2023-08-09T15:11:28.736601+0000","last_clean":"2023-08-09T15:11:28.736601+0000","last_became_active":"2023-08-09T15:11:28.729668+0000","last_became_peered":"2023-08-09T15:11:28.729668+0000","last_unstale":"2023-08-09T15:11:28.736601+0000","last_undegraded":"2023-08-09T15:11:28.736601+0000","last_fullsized":"2023-08-09T15:11:28.736601+0000","mapping_epoch":221,"log_start":"0'0","ondisk_log_start":"0'0","created":221,"last_epoch_clean":222,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T15:11:27.650831+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T15:11:27.650831+0000","last_clean_scrub_stamp":"2023-08-09T15:11:27.650831+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T17:35:23.899405+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"27.29","version":"0'0","reported_seq":9,"reported_epoch":222,"state":"active+clean","last_fresh":"2023-08-09T15:11:28.746494+0000","last_change":"2023-08-09T15:11:28.744022+0000","last_active":"2023-08-09T15:11:28.746494+0000","last_peered":"2023-08-09T15:11:28.746494+0000","last_clean":"2023-08-09T15:11:28.746494+0000","last_became_active":"2023-08-09T15:11:28.741741+0000","last_became_peered":"2023-08-09T15:11:28.741741+0000","last_unstale":"2023-08-09T15:11:28.746494+0000","last_undegraded":"2023-08-09T15:11:28.746494+0000","last_fullsized":"2023-08-09T15:11:28.746494+0000","mapping_epoch":221,"log_start":"0'0","ondisk_log_start":"0'0","created":221,"last_epoch_clean":222,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T15:11:27.650831+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T15:11:27.650831+0000","last_clean_scrub_stamp":"2023-08-09T15:11:27.650831+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T15:19:42.337949+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"27.28","version":"0'0","reported_seq":9,"reported_epoch":222,"state":"active+clean","last_fresh":"2023-08-09T15:11:28.742052+0000","last_change":"2023-08-09T15:11:28.740832+0000","last_active":"2023-08-09T15:11:28.742052+0000","last_peered":"2023-08-09T15:11:28.742052+0000","last_clean":"2023-08-09T15:11:28.742052+0000","last_became_active":"2023-08-09T15:11:28.740545+0000","last_became_peered":"2023-08-09T15:11:28.740545+0000","last_unstale":"2023-08-09T15:11:28.742052+0000","last_undegraded":"2023-08-09T15:11:28.742052+0000","last_fullsized":"2023-08-09T15:11:28.742052+0000","mapping_epoch":221,"log_start":"0'0","ondisk_log_start":"0'0","created":221,"last_epoch_clean":222,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T15:11:27.650831+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T15:11:27.650831+0000","last_clean_scrub_stamp":"2023-08-09T15:11:27.650831+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T20:56:29.070500+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"27.27","version":"0'0","reported_seq":9,"reported_epoch":222,"state":"active+clean","last_fresh":"2023-08-09T15:11:28.729516+0000","last_change":"2023-08-09T15:11:28.727940+0000","last_active":"2023-08-09T15:11:28.729516+0000","last_peered":"2023-08-09T15:11:28.729516+0000","last_clean":"2023-08-09T15:11:28.729516+0000","last_became_active":"2023-08-09T15:11:28.727521+0000","last_became_peered":"2023-08-09T15:11:28.727521+0000","last_unstale":"2023-08-09T15:11:28.729516+0000","last_undegraded":"2023-08-09T15:11:28.729516+0000","last_fullsized":"2023-08-09T15:11:28.729516+0000","mapping_epoch":221,"log_start":"0'0","ondisk_log_start":"0'0","created":221,"last_epoch_clean":222,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T15:11:27.650831+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T15:11:27.650831+0000","last_clean_scrub_stamp":"2023-08-09T15:11:27.650831+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T19:51:20.128071+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"27.26","version":"0'0","reported_seq":9,"reported_epoch":222,"state":"active+clean","last_fresh":"2023-08-09T15:11:28.739982+0000","last_change":"2023-08-09T15:11:28.738901+0000","last_active":"2023-08-09T15:11:28.739982+0000","last_peered":"2023-08-09T15:11:28.739982+0000","last_clean":"2023-08-09T15:11:28.739982+0000","last_became_active":"2023-08-09T15:11:28.738620+0000","last_became_peered":"2023-08-09T15:11:28.738620+0000","last_unstale":"2023-08-09T15:11:28.739982+0000","last_undegraded":"2023-08-09T15:11:28.739982+0000","last_fullsized":"2023-08-09T15:11:28.739982+0000","mapping_epoch":221,"log_start":"0'0","ondisk_log_start":"0'0","created":221,"last_epoch_clean":222,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T15:11:27.650831+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T15:11:27.650831+0000","last_clean_scrub_stamp":"2023-08-09T15:11:27.650831+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T17:35:23.899405+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"27.25","version":"0'0","reported_seq":9,"reported_epoch":222,"state":"active+clean","last_fresh":"2023-08-09T15:11:28.718080+0000","last_change":"2023-08-09T15:11:28.717076+0000","last_active":"2023-08-09T15:11:28.718080+0000","last_peered":"2023-08-09T15:11:28.718080+0000","last_clean":"2023-08-09T15:11:28.718080+0000","last_became_active":"2023-08-09T15:11:28.716512+0000","last_became_peered":"2023-08-09T15:11:28.716512+0000","last_unstale":"2023-08-09T15:11:28.718080+0000","last_undegraded":"2023-08-09T15:11:28.718080+0000","last_fullsized":"2023-08-09T15:11:28.718080+0000","mapping_epoch":221,"log_start":"0'0","ondisk_log_start":"0'0","created":221,"last_epoch_clean":222,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T15:11:27.650831+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T15:11:27.650831+0000","last_clean_scrub_stamp":"2023-08-09T15:11:27.650831+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T19:55:13.844084+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"27.24","version":"0'0","reported_seq":9,"reported_epoch":222,"state":"active+clean","last_fresh":"2023-08-09T15:11:28.737389+0000","last_change":"2023-08-09T15:11:28.725049+0000","last_active":"2023-08-09T15:11:28.737389+0000","last_peered":"2023-08-09T15:11:28.737389+0000","last_clean":"2023-08-09T15:11:28.737389+0000","last_became_active":"2023-08-09T15:11:28.724712+0000","last_became_peered":"2023-08-09T15:11:28.724712+0000","last_unstale":"2023-08-09T15:11:28.737389+0000","last_undegraded":"2023-08-09T15:11:28.737389+0000","last_fullsized":"2023-08-09T15:11:28.737389+0000","mapping_epoch":221,"log_start":"0'0","ondisk_log_start":"0'0","created":221,"last_epoch_clean":222,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T15:11:27.650831+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T15:11:27.650831+0000","last_clean_scrub_stamp":"2023-08-09T15:11:27.650831+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T17:39:58.055854+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"27.23","version":"0'0","reported_seq":9,"reported_epoch":222,"state":"active+clean","last_fresh":"2023-08-09T15:11:28.723557+0000","last_change":"2023-08-09T15:11:28.721852+0000","last_active":"2023-08-09T15:11:28.723557+0000","last_peered":"2023-08-09T15:11:28.723557+0000","last_clean":"2023-08-09T15:11:28.723557+0000","last_became_active":"2023-08-09T15:11:28.721573+0000","last_became_peered":"2023-08-09T15:11:28.721573+0000","last_unstale":"2023-08-09T15:11:28.723557+0000","last_undegraded":"2023-08-09T15:11:28.723557+0000","last_fullsized":"2023-08-09T15:11:28.723557+0000","mapping_epoch":221,"log_start":"0'0","ondisk_log_start":"0'0","created":221,"last_epoch_clean":222,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T15:11:27.650831+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T15:11:27.650831+0000","last_clean_scrub_stamp":"2023-08-09T15:11:27.650831+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-11T01:56:21.129627+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"27.22","version":"0'0","reported_seq":9,"reported_epoch":222,"state":"active+clean","last_fresh":"2023-08-09T15:11:28.735269+0000","last_change":"2023-08-09T15:11:28.731453+0000","last_active":"2023-08-09T15:11:28.735269+0000","last_peered":"2023-08-09T15:11:28.735269+0000","last_clean":"2023-08-09T15:11:28.735269+0000","last_became_active":"2023-08-09T15:11:28.730958+0000","last_became_peered":"2023-08-09T15:11:28.730958+0000","last_unstale":"2023-08-09T15:11:28.735269+0000","last_undegraded":"2023-08-09T15:11:28.735269+0000","last_fullsized":"2023-08-09T15:11:28.735269+0000","mapping_epoch":221,"log_start":"0'0","ondisk_log_start":"0'0","created":221,"last_epoch_clean":222,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T15:11:27.650831+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T15:11:27.650831+0000","last_clean_scrub_stamp":"2023-08-09T15:11:27.650831+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-11T02:15:51.962022+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"27.21","version":"0'0","reported_seq":9,"reported_epoch":222,"state":"active+clean","last_fresh":"2023-08-09T15:11:28.730834+0000","last_change":"2023-08-09T15:11:28.727691+0000","last_active":"2023-08-09T15:11:28.730834+0000","last_peered":"2023-08-09T15:11:28.730834+0000","last_clean":"2023-08-09T15:11:28.730834+0000","last_became_active":"2023-08-09T15:11:28.727355+0000","last_became_peered":"2023-08-09T15:11:28.727355+0000","last_unstale":"2023-08-09T15:11:28.730834+0000","last_undegraded":"2023-08-09T15:11:28.730834+0000","last_fullsized":"2023-08-09T15:11:28.730834+0000","mapping_epoch":221,"log_start":"0'0","ondisk_log_start":"0'0","created":221,"last_epoch_clean":222,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T15:11:27.650831+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T15:11:27.650831+0000","last_clean_scrub_stamp":"2023-08-09T15:11:27.650831+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T15:35:58.727081+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"27.20","version":"0'0","reported_seq":9,"reported_epoch":222,"state":"active+clean","last_fresh":"2023-08-09T15:11:28.718939+0000","last_change":"2023-08-09T15:11:28.714950+0000","last_active":"2023-08-09T15:11:28.718939+0000","last_peered":"2023-08-09T15:11:28.718939+0000","last_clean":"2023-08-09T15:11:28.718939+0000","last_became_active":"2023-08-09T15:11:28.714526+0000","last_became_peered":"2023-08-09T15:11:28.714526+0000","last_unstale":"2023-08-09T15:11:28.718939+0000","last_undegraded":"2023-08-09T15:11:28.718939+0000","last_fullsized":"2023-08-09T15:11:28.718939+0000","mapping_epoch":221,"log_start":"0'0","ondisk_log_start":"0'0","created":221,"last_epoch_clean":222,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T15:11:27.650831+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T15:11:27.650831+0000","last_clean_scrub_stamp":"2023-08-09T15:11:27.650831+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T17:43:27.363007+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"27.16","version":"0'0","reported_seq":9,"reported_epoch":222,"state":"active+clean","last_fresh":"2023-08-09T15:11:28.734747+0000","last_change":"2023-08-09T15:11:28.732566+0000","last_active":"2023-08-09T15:11:28.734747+0000","last_peered":"2023-08-09T15:11:28.734747+0000","last_clean":"2023-08-09T15:11:28.734747+0000","last_became_active":"2023-08-09T15:11:28.731793+0000","last_became_peered":"2023-08-09T15:11:28.731793+0000","last_unstale":"2023-08-09T15:11:28.734747+0000","last_undegraded":"2023-08-09T15:11:28.734747+0000","last_fullsized":"2023-08-09T15:11:28.734747+0000","mapping_epoch":221,"log_start":"0'0","ondisk_log_start":"0'0","created":221,"last_epoch_clean":222,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T15:11:27.650831+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T15:11:27.650831+0000","last_clean_scrub_stamp":"2023-08-09T15:11:27.650831+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T15:48:40.328019+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"26.17","version":"0'0","reported_seq":11,"reported_epoch":222,"state":"active+clean","last_fresh":"2023-08-09T15:11:28.707232+0000","last_change":"2023-08-09T15:11:27.698829+0000","last_active":"2023-08-09T15:11:28.707232+0000","last_peered":"2023-08-09T15:11:28.707232+0000","last_clean":"2023-08-09T15:11:28.707232+0000","last_became_active":"2023-08-09T15:11:27.698559+0000","last_became_peered":"2023-08-09T15:11:27.698559+0000","last_unstale":"2023-08-09T15:11:28.707232+0000","last_undegraded":"2023-08-09T15:11:28.707232+0000","last_fullsized":"2023-08-09T15:11:28.707232+0000","mapping_epoch":220,"log_start":"0'0","ondisk_log_start":"0'0","created":220,"last_epoch_clean":221,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T15:11:26.638086+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T15:11:26.638086+0000","last_clean_scrub_stamp":"2023-08-09T15:11:26.638086+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T19:09:54.207782+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"27.17","version":"0'0","reported_seq":9,"reported_epoch":222,"state":"active+clean","last_fresh":"2023-08-09T15:11:28.735232+0000","last_change":"2023-08-09T15:11:28.723837+0000","last_active":"2023-08-09T15:11:28.735232+0000","last_peered":"2023-08-09T15:11:28.735232+0000","last_clean":"2023-08-09T15:11:28.735232+0000","last_became_active":"2023-08-09T15:11:28.723559+0000","last_became_peered":"2023-08-09T15:11:28.723559+0000","last_unstale":"2023-08-09T15:11:28.735232+0000","last_undegraded":"2023-08-09T15:11:28.735232+0000","last_fullsized":"2023-08-09T15:11:28.735232+0000","mapping_epoch":221,"log_start":"0'0","ondisk_log_start":"0'0","created":221,"last_epoch_clean":222,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T15:11:27.650831+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T15:11:27.650831+0000","last_clean_scrub_stamp":"2023-08-09T15:11:27.650831+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T18:52:30.153594+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"26.16","version":"0'0","reported_seq":11,"reported_epoch":222,"state":"active+clean","last_fresh":"2023-08-09T15:11:28.724239+0000","last_change":"2023-08-09T15:11:27.701082+0000","last_active":"2023-08-09T15:11:28.724239+0000","last_peered":"2023-08-09T15:11:28.724239+0000","last_clean":"2023-08-09T15:11:28.724239+0000","last_became_active":"2023-08-09T15:11:27.700749+0000","last_became_peered":"2023-08-09T15:11:27.700749+0000","last_unstale":"2023-08-09T15:11:28.724239+0000","last_undegraded":"2023-08-09T15:11:28.724239+0000","last_fullsized":"2023-08-09T15:11:28.724239+0000","mapping_epoch":220,"log_start":"0'0","ondisk_log_start":"0'0","created":220,"last_epoch_clean":221,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T15:11:26.638086+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T15:11:26.638086+0000","last_clean_scrub_stamp":"2023-08-09T15:11:26.638086+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T20:02:44.106120+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"27.14","version":"0'0","reported_seq":9,"reported_epoch":222,"state":"active+clean","last_fresh":"2023-08-09T15:11:28.729536+0000","last_change":"2023-08-09T15:11:28.728579+0000","last_active":"2023-08-09T15:11:28.729536+0000","last_peered":"2023-08-09T15:11:28.729536+0000","last_clean":"2023-08-09T15:11:28.729536+0000","last_became_active":"2023-08-09T15:11:28.728303+0000","last_became_peered":"2023-08-09T15:11:28.728303+0000","last_unstale":"2023-08-09T15:11:28.729536+0000","last_undegraded":"2023-08-09T15:11:28.729536+0000","last_fullsized":"2023-08-09T15:11:28.729536+0000","mapping_epoch":221,"log_start":"0'0","ondisk_log_start":"0'0","created":221,"last_epoch_clean":222,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T15:11:27.650831+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T15:11:27.650831+0000","last_clean_scrub_stamp":"2023-08-09T15:11:27.650831+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-11T00:01:24.436012+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"26.15","version":"222'3","reported_seq":14,"reported_epoch":222,"state":"active+clean","last_fresh":"2023-08-09T15:11:28.724700+0000","last_change":"2023-08-09T15:11:27.691311+0000","last_active":"2023-08-09T15:11:28.724700+0000","last_peered":"2023-08-09T15:11:28.724700+0000","last_clean":"2023-08-09T15:11:28.724700+0000","last_became_active":"2023-08-09T15:11:27.691027+0000","last_became_peered":"2023-08-09T15:11:27.691027+0000","last_unstale":"2023-08-09T15:11:28.724700+0000","last_undegraded":"2023-08-09T15:11:28.724700+0000","last_fullsized":"2023-08-09T15:11:28.724700+0000","mapping_epoch":220,"log_start":"0'0","ondisk_log_start":"0'0","created":220,"last_epoch_clean":221,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T15:11:26.638086+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T15:11:26.638086+0000","last_clean_scrub_stamp":"2023-08-09T15:11:26.638086+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T16:10:22.748982+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"27.15","version":"0'0","reported_seq":9,"reported_epoch":222,"state":"active+clean","last_fresh":"2023-08-09T15:11:28.728936+0000","last_change":"2023-08-09T15:11:28.725915+0000","last_active":"2023-08-09T15:11:28.728936+0000","last_peered":"2023-08-09T15:11:28.728936+0000","last_clean":"2023-08-09T15:11:28.728936+0000","last_became_active":"2023-08-09T15:11:28.725656+0000","last_became_peered":"2023-08-09T15:11:28.725656+0000","last_unstale":"2023-08-09T15:11:28.728936+0000","last_undegraded":"2023-08-09T15:11:28.728936+0000","last_fullsized":"2023-08-09T15:11:28.728936+0000","mapping_epoch":221,"log_start":"0'0","ondisk_log_start":"0'0","created":221,"last_epoch_clean":222,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T15:11:27.650831+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T15:11:27.650831+0000","last_clean_scrub_stamp":"2023-08-09T15:11:27.650831+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T20:03:51.144236+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"26.14","version":"222'6","reported_seq":17,"reported_epoch":222,"state":"active+clean","last_fresh":"2023-08-09T15:11:28.748422+0000","last_change":"2023-08-09T15:11:27.702153+0000","last_active":"2023-08-09T15:11:28.748422+0000","last_peered":"2023-08-09T15:11:28.748422+0000","last_clean":"2023-08-09T15:11:28.748422+0000","last_became_active":"2023-08-09T15:11:27.701805+0000","last_became_peered":"2023-08-09T15:11:27.701805+0000","last_unstale":"2023-08-09T15:11:28.748422+0000","last_undegraded":"2023-08-09T15:11:28.748422+0000","last_fullsized":"2023-08-09T15:11:28.748422+0000","mapping_epoch":220,"log_start":"0'0","ondisk_log_start":"0'0","created":220,"last_epoch_clean":221,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T15:11:26.638086+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T15:11:26.638086+0000","last_clean_scrub_stamp":"2023-08-09T15:11:26.638086+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-11T02:52:55.736990+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"27.12","version":"0'0","reported_seq":9,"reported_epoch":222,"state":"active+clean","last_fresh":"2023-08-09T15:11:28.722465+0000","last_change":"2023-08-09T15:11:28.717067+0000","last_active":"2023-08-09T15:11:28.722465+0000","last_peered":"2023-08-09T15:11:28.722465+0000","last_clean":"2023-08-09T15:11:28.722465+0000","last_became_active":"2023-08-09T15:11:28.716732+0000","last_became_peered":"2023-08-09T15:11:28.716732+0000","last_unstale":"2023-08-09T15:11:28.722465+0000","last_undegraded":"2023-08-09T15:11:28.722465+0000","last_fullsized":"2023-08-09T15:11:28.722465+0000","mapping_epoch":221,"log_start":"0'0","ondisk_log_start":"0'0","created":221,"last_epoch_clean":222,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T15:11:27.650831+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T15:11:27.650831+0000","last_clean_scrub_stamp":"2023-08-09T15:11:27.650831+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T22:09:46.565197+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"26.13","version":"222'2","reported_seq":13,"reported_epoch":222,"state":"active+clean","last_fresh":"2023-08-09T15:11:28.746371+0000","last_change":"2023-08-09T15:11:27.703741+0000","last_active":"2023-08-09T15:11:28.746371+0000","last_peered":"2023-08-09T15:11:28.746371+0000","last_clean":"2023-08-09T15:11:28.746371+0000","last_became_active":"2023-08-09T15:11:27.703366+0000","last_became_peered":"2023-08-09T15:11:27.703366+0000","last_unstale":"2023-08-09T15:11:28.746371+0000","last_undegraded":"2023-08-09T15:11:28.746371+0000","last_fullsized":"2023-08-09T15:11:28.746371+0000","mapping_epoch":220,"log_start":"0'0","ondisk_log_start":"0'0","created":220,"last_epoch_clean":221,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T15:11:26.638086+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T15:11:26.638086+0000","last_clean_scrub_stamp":"2023-08-09T15:11:26.638086+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T20:00:55.907532+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"27.13","version":"0'0","reported_seq":9,"reported_epoch":222,"state":"active+clean","last_fresh":"2023-08-09T15:11:28.720827+0000","last_change":"2023-08-09T15:11:28.717409+0000","last_active":"2023-08-09T15:11:28.720827+0000","last_peered":"2023-08-09T15:11:28.720827+0000","last_clean":"2023-08-09T15:11:28.720827+0000","last_became_active":"2023-08-09T15:11:28.715486+0000","last_became_peered":"2023-08-09T15:11:28.715486+0000","last_unstale":"2023-08-09T15:11:28.720827+0000","last_undegraded":"2023-08-09T15:11:28.720827+0000","last_fullsized":"2023-08-09T15:11:28.720827+0000","mapping_epoch":221,"log_start":"0'0","ondisk_log_start":"0'0","created":221,"last_epoch_clean":222,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T15:11:27.650831+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T15:11:27.650831+0000","last_clean_scrub_stamp":"2023-08-09T15:11:27.650831+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-11T02:40:53.891575+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"26.12","version":"0'0","reported_seq":11,"reported_epoch":222,"state":"active+clean","last_fresh":"2023-08-09T15:11:28.724826+0000","last_change":"2023-08-09T15:11:27.703826+0000","last_active":"2023-08-09T15:11:28.724826+0000","last_peered":"2023-08-09T15:11:28.724826+0000","last_clean":"2023-08-09T15:11:28.724826+0000","last_became_active":"2023-08-09T15:11:27.703447+0000","last_became_peered":"2023-08-09T15:11:27.703447+0000","last_unstale":"2023-08-09T15:11:28.724826+0000","last_undegraded":"2023-08-09T15:11:28.724826+0000","last_fullsized":"2023-08-09T15:11:28.724826+0000","mapping_epoch":220,"log_start":"0'0","ondisk_log_start":"0'0","created":220,"last_epoch_clean":221,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T15:11:26.638086+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T15:11:26.638086+0000","last_clean_scrub_stamp":"2023-08-09T15:11:26.638086+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-11T02:28:37.016859+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"27.10","version":"0'0","reported_seq":9,"reported_epoch":222,"state":"active+clean","last_fresh":"2023-08-09T15:11:28.736631+0000","last_change":"2023-08-09T15:11:28.725193+0000","last_active":"2023-08-09T15:11:28.736631+0000","last_peered":"2023-08-09T15:11:28.736631+0000","last_clean":"2023-08-09T15:11:28.736631+0000","last_became_active":"2023-08-09T15:11:28.724918+0000","last_became_peered":"2023-08-09T15:11:28.724918+0000","last_unstale":"2023-08-09T15:11:28.736631+0000","last_undegraded":"2023-08-09T15:11:28.736631+0000","last_fullsized":"2023-08-09T15:11:28.736631+0000","mapping_epoch":221,"log_start":"0'0","ondisk_log_start":"0'0","created":221,"last_epoch_clean":222,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T15:11:27.650831+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T15:11:27.650831+0000","last_clean_scrub_stamp":"2023-08-09T15:11:27.650831+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T16:16:24.924181+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"26.11","version":"0'0","reported_seq":11,"reported_epoch":222,"state":"active+clean","last_fresh":"2023-08-09T15:11:28.700891+0000","last_change":"2023-08-09T15:11:27.686506+0000","last_active":"2023-08-09T15:11:28.700891+0000","last_peered":"2023-08-09T15:11:28.700891+0000","last_clean":"2023-08-09T15:11:28.700891+0000","last_became_active":"2023-08-09T15:11:27.686193+0000","last_became_peered":"2023-08-09T15:11:27.686193+0000","last_unstale":"2023-08-09T15:11:28.700891+0000","last_undegraded":"2023-08-09T15:11:28.700891+0000","last_fullsized":"2023-08-09T15:11:28.700891+0000","mapping_epoch":220,"log_start":"0'0","ondisk_log_start":"0'0","created":220,"last_epoch_clean":221,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T15:11:26.638086+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T15:11:26.638086+0000","last_clean_scrub_stamp":"2023-08-09T15:11:26.638086+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T19:12:04.370705+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"27.11","version":"0'0","reported_seq":9,"reported_epoch":222,"state":"active+clean","last_fresh":"2023-08-09T15:11:28.721342+0000","last_change":"2023-08-09T15:11:28.717262+0000","last_active":"2023-08-09T15:11:28.721342+0000","last_peered":"2023-08-09T15:11:28.721342+0000","last_clean":"2023-08-09T15:11:28.721342+0000","last_became_active":"2023-08-09T15:11:28.716984+0000","last_became_peered":"2023-08-09T15:11:28.716984+0000","last_unstale":"2023-08-09T15:11:28.721342+0000","last_undegraded":"2023-08-09T15:11:28.721342+0000","last_fullsized":"2023-08-09T15:11:28.721342+0000","mapping_epoch":221,"log_start":"0'0","ondisk_log_start":"0'0","created":221,"last_epoch_clean":222,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T15:11:27.650831+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T15:11:27.650831+0000","last_clean_scrub_stamp":"2023-08-09T15:11:27.650831+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T21:46:15.531410+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"26.10","version":"222'8","reported_seq":19,"reported_epoch":222,"state":"active+clean","last_fresh":"2023-08-09T15:11:28.737844+0000","last_change":"2023-08-09T15:11:27.690582+0000","last_active":"2023-08-09T15:11:28.737844+0000","last_peered":"2023-08-09T15:11:28.737844+0000","last_clean":"2023-08-09T15:11:28.737844+0000","last_became_active":"2023-08-09T15:11:27.690233+0000","last_became_peered":"2023-08-09T15:11:27.690233+0000","last_unstale":"2023-08-09T15:11:28.737844+0000","last_undegraded":"2023-08-09T15:11:28.737844+0000","last_fullsized":"2023-08-09T15:11:28.737844+0000","mapping_epoch":220,"log_start":"0'0","ondisk_log_start":"0'0","created":220,"last_epoch_clean":221,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T15:11:26.638086+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T15:11:26.638086+0000","last_clean_scrub_stamp":"2023-08-09T15:11:26.638086+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T21:03:24.709891+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"27.e","version":"0'0","reported_seq":9,"reported_epoch":222,"state":"active+clean","last_fresh":"2023-08-09T15:11:28.735325+0000","last_change":"2023-08-09T15:11:28.730634+0000","last_active":"2023-08-09T15:11:28.735325+0000","last_peered":"2023-08-09T15:11:28.735325+0000","last_clean":"2023-08-09T15:11:28.735325+0000","last_became_active":"2023-08-09T15:11:28.730191+0000","last_became_peered":"2023-08-09T15:11:28.730191+0000","last_unstale":"2023-08-09T15:11:28.735325+0000","last_undegraded":"2023-08-09T15:11:28.735325+0000","last_fullsized":"2023-08-09T15:11:28.735325+0000","mapping_epoch":221,"log_start":"0'0","ondisk_log_start":"0'0","created":221,"last_epoch_clean":222,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T15:11:27.650831+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T15:11:27.650831+0000","last_clean_scrub_stamp":"2023-08-09T15:11:27.650831+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-11T01:17:45.162466+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"26.f","version":"222'1","reported_seq":12,"reported_epoch":222,"state":"active+clean","last_fresh":"2023-08-09T15:11:28.722856+0000","last_change":"2023-08-09T15:11:27.695677+0000","last_active":"2023-08-09T15:11:28.722856+0000","last_peered":"2023-08-09T15:11:28.722856+0000","last_clean":"2023-08-09T15:11:28.722856+0000","last_became_active":"2023-08-09T15:11:27.695371+0000","last_became_peered":"2023-08-09T15:11:27.695371+0000","last_unstale":"2023-08-09T15:11:28.722856+0000","last_undegraded":"2023-08-09T15:11:28.722856+0000","last_fullsized":"2023-08-09T15:11:28.722856+0000","mapping_epoch":220,"log_start":"0'0","ondisk_log_start":"0'0","created":220,"last_epoch_clean":221,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T15:11:26.638086+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T15:11:26.638086+0000","last_clean_scrub_stamp":"2023-08-09T15:11:26.638086+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T22:13:31.139984+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"27.f","version":"0'0","reported_seq":9,"reported_epoch":222,"state":"active+clean","last_fresh":"2023-08-09T15:11:28.729457+0000","last_change":"2023-08-09T15:11:28.727784+0000","last_active":"2023-08-09T15:11:28.729457+0000","last_peered":"2023-08-09T15:11:28.729457+0000","last_clean":"2023-08-09T15:11:28.729457+0000","last_became_active":"2023-08-09T15:11:28.727118+0000","last_became_peered":"2023-08-09T15:11:28.727118+0000","last_unstale":"2023-08-09T15:11:28.729457+0000","last_undegraded":"2023-08-09T15:11:28.729457+0000","last_fullsized":"2023-08-09T15:11:28.729457+0000","mapping_epoch":221,"log_start":"0'0","ondisk_log_start":"0'0","created":221,"last_epoch_clean":222,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T15:11:27.650831+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T15:11:27.650831+0000","last_clean_scrub_stamp":"2023-08-09T15:11:27.650831+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T22:31:02.501508+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"26.e","version":"222'2","reported_seq":13,"reported_epoch":222,"state":"active+clean","last_fresh":"2023-08-09T15:11:28.734137+0000","last_change":"2023-08-09T15:11:27.693614+0000","last_active":"2023-08-09T15:11:28.734137+0000","last_peered":"2023-08-09T15:11:28.734137+0000","last_clean":"2023-08-09T15:11:28.734137+0000","last_became_active":"2023-08-09T15:11:27.692947+0000","last_became_peered":"2023-08-09T15:11:27.692947+0000","last_unstale":"2023-08-09T15:11:28.734137+0000","last_undegraded":"2023-08-09T15:11:28.734137+0000","last_fullsized":"2023-08-09T15:11:28.734137+0000","mapping_epoch":220,"log_start":"0'0","ondisk_log_start":"0'0","created":220,"last_epoch_clean":221,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T15:11:26.638086+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T15:11:26.638086+0000","last_clean_scrub_stamp":"2023-08-09T15:11:26.638086+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T21:40:23.646378+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"27.c","version":"0'0","reported_seq":9,"reported_epoch":222,"state":"active+clean","last_fresh":"2023-08-09T15:11:28.737408+0000","last_change":"2023-08-09T15:11:28.733143+0000","last_active":"2023-08-09T15:11:28.737408+0000","last_peered":"2023-08-09T15:11:28.737408+0000","last_clean":"2023-08-09T15:11:28.737408+0000","last_became_active":"2023-08-09T15:11:28.732814+0000","last_became_peered":"2023-08-09T15:11:28.732814+0000","last_unstale":"2023-08-09T15:11:28.737408+0000","last_undegraded":"2023-08-09T15:11:28.737408+0000","last_fullsized":"2023-08-09T15:11:28.737408+0000","mapping_epoch":221,"log_start":"0'0","ondisk_log_start":"0'0","created":221,"last_epoch_clean":222,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T15:11:27.650831+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T15:11:27.650831+0000","last_clean_scrub_stamp":"2023-08-09T15:11:27.650831+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-11T01:30:42.299691+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"26.d","version":"222'1","reported_seq":12,"reported_epoch":222,"state":"active+clean","last_fresh":"2023-08-09T15:11:28.737171+0000","last_change":"2023-08-09T15:11:27.701548+0000","last_active":"2023-08-09T15:11:28.737171+0000","last_peered":"2023-08-09T15:11:28.737171+0000","last_clean":"2023-08-09T15:11:28.737171+0000","last_became_active":"2023-08-09T15:11:27.701234+0000","last_became_peered":"2023-08-09T15:11:27.701234+0000","last_unstale":"2023-08-09T15:11:28.737171+0000","last_undegraded":"2023-08-09T15:11:28.737171+0000","last_fullsized":"2023-08-09T15:11:28.737171+0000","mapping_epoch":220,"log_start":"0'0","ondisk_log_start":"0'0","created":220,"last_epoch_clean":221,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T15:11:26.638086+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T15:11:26.638086+0000","last_clean_scrub_stamp":"2023-08-09T15:11:26.638086+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T16:42:13.581769+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"27.d","version":"0'0","reported_seq":9,"reported_epoch":222,"state":"active+clean","last_fresh":"2023-08-09T15:11:28.733006+0000","last_change":"2023-08-09T15:11:28.731334+0000","last_active":"2023-08-09T15:11:28.733006+0000","last_peered":"2023-08-09T15:11:28.733006+0000","last_clean":"2023-08-09T15:11:28.733006+0000","last_became_active":"2023-08-09T15:11:28.730843+0000","last_became_peered":"2023-08-09T15:11:28.730843+0000","last_unstale":"2023-08-09T15:11:28.733006+0000","last_undegraded":"2023-08-09T15:11:28.733006+0000","last_fullsized":"2023-08-09T15:11:28.733006+0000","mapping_epoch":221,"log_start":"0'0","ondisk_log_start":"0'0","created":221,"last_epoch_clean":222,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T15:11:27.650831+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T15:11:27.650831+0000","last_clean_scrub_stamp":"2023-08-09T15:11:27.650831+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T15:50:57.838660+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"26.c","version":"0'0","reported_seq":11,"reported_epoch":222,"state":"active+clean","last_fresh":"2023-08-09T15:11:28.704245+0000","last_change":"2023-08-09T15:11:27.693369+0000","last_active":"2023-08-09T15:11:28.704245+0000","last_peered":"2023-08-09T15:11:28.704245+0000","last_clean":"2023-08-09T15:11:28.704245+0000","last_became_active":"2023-08-09T15:11:27.692360+0000","last_became_peered":"2023-08-09T15:11:27.692360+0000","last_unstale":"2023-08-09T15:11:28.704245+0000","last_undegraded":"2023-08-09T15:11:28.704245+0000","last_fullsized":"2023-08-09T15:11:28.704245+0000","mapping_epoch":220,"log_start":"0'0","ondisk_log_start":"0'0","created":220,"last_epoch_clean":221,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T15:11:26.638086+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T15:11:26.638086+0000","last_clean_scrub_stamp":"2023-08-09T15:11:26.638086+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T18:17:59.659054+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"27.a","version":"0'0","reported_seq":9,"reported_epoch":222,"state":"active+clean","last_fresh":"2023-08-09T15:11:28.719326+0000","last_change":"2023-08-09T15:11:28.714046+0000","last_active":"2023-08-09T15:11:28.719326+0000","last_peered":"2023-08-09T15:11:28.719326+0000","last_clean":"2023-08-09T15:11:28.719326+0000","last_became_active":"2023-08-09T15:11:28.713722+0000","last_became_peered":"2023-08-09T15:11:28.713722+0000","last_unstale":"2023-08-09T15:11:28.719326+0000","last_undegraded":"2023-08-09T15:11:28.719326+0000","last_fullsized":"2023-08-09T15:11:28.719326+0000","mapping_epoch":221,"log_start":"0'0","ondisk_log_start":"0'0","created":221,"last_epoch_clean":222,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T15:11:27.650831+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T15:11:27.650831+0000","last_clean_scrub_stamp":"2023-08-09T15:11:27.650831+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T20:21:40.252700+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"26.b","version":"222'2","reported_seq":13,"reported_epoch":222,"state":"active+clean","last_fresh":"2023-08-09T15:11:28.725371+0000","last_change":"2023-08-09T15:11:27.685327+0000","last_active":"2023-08-09T15:11:28.725371+0000","last_peered":"2023-08-09T15:11:28.725371+0000","last_clean":"2023-08-09T15:11:28.725371+0000","last_became_active":"2023-08-09T15:11:27.685042+0000","last_became_peered":"2023-08-09T15:11:27.685042+0000","last_unstale":"2023-08-09T15:11:28.725371+0000","last_undegraded":"2023-08-09T15:11:28.725371+0000","last_fullsized":"2023-08-09T15:11:28.725371+0000","mapping_epoch":220,"log_start":"0'0","ondisk_log_start":"0'0","created":220,"last_epoch_clean":221,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T15:11:26.638086+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T15:11:26.638086+0000","last_clean_scrub_stamp":"2023-08-09T15:11:26.638086+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T16:44:31.400957+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"27.b","version":"0'0","reported_seq":9,"reported_epoch":222,"state":"active+clean","last_fresh":"2023-08-09T15:11:28.724202+0000","last_change":"2023-08-09T15:11:28.717915+0000","last_active":"2023-08-09T15:11:28.724202+0000","last_peered":"2023-08-09T15:11:28.724202+0000","last_clean":"2023-08-09T15:11:28.724202+0000","last_became_active":"2023-08-09T15:11:28.717606+0000","last_became_peered":"2023-08-09T15:11:28.717606+0000","last_unstale":"2023-08-09T15:11:28.724202+0000","last_undegraded":"2023-08-09T15:11:28.724202+0000","last_fullsized":"2023-08-09T15:11:28.724202+0000","mapping_epoch":221,"log_start":"0'0","ondisk_log_start":"0'0","created":221,"last_epoch_clean":222,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T15:11:27.650831+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T15:11:27.650831+0000","last_clean_scrub_stamp":"2023-08-09T15:11:27.650831+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T23:43:01.111359+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"26.a","version":"0'0","reported_seq":11,"reported_epoch":222,"state":"active+clean","last_fresh":"2023-08-09T15:11:28.697813+0000","last_change":"2023-08-09T15:11:27.693720+0000","last_active":"2023-08-09T15:11:28.697813+0000","last_peered":"2023-08-09T15:11:28.697813+0000","last_clean":"2023-08-09T15:11:28.697813+0000","last_became_active":"2023-08-09T15:11:27.693401+0000","last_became_peered":"2023-08-09T15:11:27.693401+0000","last_unstale":"2023-08-09T15:11:28.697813+0000","last_undegraded":"2023-08-09T15:11:28.697813+0000","last_fullsized":"2023-08-09T15:11:28.697813+0000","mapping_epoch":220,"log_start":"0'0","ondisk_log_start":"0'0","created":220,"last_epoch_clean":221,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T15:11:26.638086+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T15:11:26.638086+0000","last_clean_scrub_stamp":"2023-08-09T15:11:26.638086+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T23:01:04.684315+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"27.8","version":"0'0","reported_seq":9,"reported_epoch":222,"state":"active+clean","last_fresh":"2023-08-09T15:11:28.738386+0000","last_change":"2023-08-09T15:11:28.737655+0000","last_active":"2023-08-09T15:11:28.738386+0000","last_peered":"2023-08-09T15:11:28.738386+0000","last_clean":"2023-08-09T15:11:28.738386+0000","last_became_active":"2023-08-09T15:11:28.737354+0000","last_became_peered":"2023-08-09T15:11:28.737354+0000","last_unstale":"2023-08-09T15:11:28.738386+0000","last_undegraded":"2023-08-09T15:11:28.738386+0000","last_fullsized":"2023-08-09T15:11:28.738386+0000","mapping_epoch":221,"log_start":"0'0","ondisk_log_start":"0'0","created":221,"last_epoch_clean":222,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T15:11:27.650831+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T15:11:27.650831+0000","last_clean_scrub_stamp":"2023-08-09T15:11:27.650831+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T17:40:36.100186+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"26.9","version":"0'0","reported_seq":11,"reported_epoch":222,"state":"active+clean","last_fresh":"2023-08-09T15:11:28.696909+0000","last_change":"2023-08-09T15:11:27.701663+0000","last_active":"2023-08-09T15:11:28.696909+0000","last_peered":"2023-08-09T15:11:28.696909+0000","last_clean":"2023-08-09T15:11:28.696909+0000","last_became_active":"2023-08-09T15:11:27.701336+0000","last_became_peered":"2023-08-09T15:11:27.701336+0000","last_unstale":"2023-08-09T15:11:28.696909+0000","last_undegraded":"2023-08-09T15:11:28.696909+0000","last_fullsized":"2023-08-09T15:11:28.696909+0000","mapping_epoch":220,"log_start":"0'0","ondisk_log_start":"0'0","created":220,"last_epoch_clean":221,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T15:11:26.638086+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T15:11:26.638086+0000","last_clean_scrub_stamp":"2023-08-09T15:11:26.638086+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T19:53:59.772090+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"27.9","version":"0'0","reported_seq":9,"reported_epoch":222,"state":"active+clean","last_fresh":"2023-08-09T15:11:28.745475+0000","last_change":"2023-08-09T15:11:28.741869+0000","last_active":"2023-08-09T15:11:28.745475+0000","last_peered":"2023-08-09T15:11:28.745475+0000","last_clean":"2023-08-09T15:11:28.745475+0000","last_became_active":"2023-08-09T15:11:28.737075+0000","last_became_peered":"2023-08-09T15:11:28.737075+0000","last_unstale":"2023-08-09T15:11:28.745475+0000","last_undegraded":"2023-08-09T15:11:28.745475+0000","last_fullsized":"2023-08-09T15:11:28.745475+0000","mapping_epoch":221,"log_start":"0'0","ondisk_log_start":"0'0","created":221,"last_epoch_clean":222,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T15:11:27.650831+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T15:11:27.650831+0000","last_clean_scrub_stamp":"2023-08-09T15:11:27.650831+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T15:37:59.332944+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"26.8","version":"0'0","reported_seq":11,"reported_epoch":222,"state":"active+clean","last_fresh":"2023-08-09T15:11:28.708652+0000","last_change":"2023-08-09T15:11:27.696329+0000","last_active":"2023-08-09T15:11:28.708652+0000","last_peered":"2023-08-09T15:11:28.708652+0000","last_clean":"2023-08-09T15:11:28.708652+0000","last_became_active":"2023-08-09T15:11:27.696043+0000","last_became_peered":"2023-08-09T15:11:27.696043+0000","last_unstale":"2023-08-09T15:11:28.708652+0000","last_undegraded":"2023-08-09T15:11:28.708652+0000","last_fullsized":"2023-08-09T15:11:28.708652+0000","mapping_epoch":220,"log_start":"0'0","ondisk_log_start":"0'0","created":220,"last_epoch_clean":221,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T15:11:26.638086+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T15:11:26.638086+0000","last_clean_scrub_stamp":"2023-08-09T15:11:26.638086+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T20:30:44.755082+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"27.6","version":"0'0","reported_seq":9,"reported_epoch":222,"state":"active+clean","last_fresh":"2023-08-09T15:11:28.723604+0000","last_change":"2023-08-09T15:11:28.721153+0000","last_active":"2023-08-09T15:11:28.723604+0000","last_peered":"2023-08-09T15:11:28.723604+0000","last_clean":"2023-08-09T15:11:28.723604+0000","last_became_active":"2023-08-09T15:11:28.720875+0000","last_became_peered":"2023-08-09T15:11:28.720875+0000","last_unstale":"2023-08-09T15:11:28.723604+0000","last_undegraded":"2023-08-09T15:11:28.723604+0000","last_fullsized":"2023-08-09T15:11:28.723604+0000","mapping_epoch":221,"log_start":"0'0","ondisk_log_start":"0'0","created":221,"last_epoch_clean":222,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T15:11:27.650831+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T15:11:27.650831+0000","last_clean_scrub_stamp":"2023-08-09T15:11:27.650831+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-11T00:24:33.663742+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"26.7","version":"222'2","reported_seq":13,"reported_epoch":222,"state":"active+clean","last_fresh":"2023-08-09T15:11:28.737068+0000","last_change":"2023-08-09T15:11:27.695871+0000","last_active":"2023-08-09T15:11:28.737068+0000","last_peered":"2023-08-09T15:11:28.737068+0000","last_clean":"2023-08-09T15:11:28.737068+0000","last_became_active":"2023-08-09T15:11:27.695549+0000","last_became_peered":"2023-08-09T15:11:27.695549+0000","last_unstale":"2023-08-09T15:11:28.737068+0000","last_undegraded":"2023-08-09T15:11:28.737068+0000","last_fullsized":"2023-08-09T15:11:28.737068+0000","mapping_epoch":220,"log_start":"0'0","ondisk_log_start":"0'0","created":220,"last_epoch_clean":221,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T15:11:26.638086+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T15:11:26.638086+0000","last_clean_scrub_stamp":"2023-08-09T15:11:26.638086+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-11T01:51:31.087330+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"27.7","version":"0'0","reported_seq":9,"reported_epoch":222,"state":"active+clean","last_fresh":"2023-08-09T15:11:28.721151+0000","last_change":"2023-08-09T15:11:28.716986+0000","last_active":"2023-08-09T15:11:28.721151+0000","last_peered":"2023-08-09T15:11:28.721151+0000","last_clean":"2023-08-09T15:11:28.721151+0000","last_became_active":"2023-08-09T15:11:28.716684+0000","last_became_peered":"2023-08-09T15:11:28.716684+0000","last_unstale":"2023-08-09T15:11:28.721151+0000","last_undegraded":"2023-08-09T15:11:28.721151+0000","last_fullsized":"2023-08-09T15:11:28.721151+0000","mapping_epoch":221,"log_start":"0'0","ondisk_log_start":"0'0","created":221,"last_epoch_clean":222,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T15:11:27.650831+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T15:11:27.650831+0000","last_clean_scrub_stamp":"2023-08-09T15:11:27.650831+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-11T00:59:54.696173+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"26.6","version":"0'0","reported_seq":11,"reported_epoch":222,"state":"active+clean","last_fresh":"2023-08-09T15:11:28.722837+0000","last_change":"2023-08-09T15:11:27.688523+0000","last_active":"2023-08-09T15:11:28.722837+0000","last_peered":"2023-08-09T15:11:28.722837+0000","last_clean":"2023-08-09T15:11:28.722837+0000","last_became_active":"2023-08-09T15:11:27.688249+0000","last_became_peered":"2023-08-09T15:11:27.688249+0000","last_unstale":"2023-08-09T15:11:28.722837+0000","last_undegraded":"2023-08-09T15:11:28.722837+0000","last_fullsized":"2023-08-09T15:11:28.722837+0000","mapping_epoch":220,"log_start":"0'0","ondisk_log_start":"0'0","created":220,"last_epoch_clean":221,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T15:11:26.638086+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T15:11:26.638086+0000","last_clean_scrub_stamp":"2023-08-09T15:11:26.638086+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T16:31:57.538963+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"27.4","version":"0'0","reported_seq":9,"reported_epoch":222,"state":"active+clean","last_fresh":"2023-08-09T15:11:28.729248+0000","last_change":"2023-08-09T15:11:28.726566+0000","last_active":"2023-08-09T15:11:28.729248+0000","last_peered":"2023-08-09T15:11:28.729248+0000","last_clean":"2023-08-09T15:11:28.729248+0000","last_became_active":"2023-08-09T15:11:28.726268+0000","last_became_peered":"2023-08-09T15:11:28.726268+0000","last_unstale":"2023-08-09T15:11:28.729248+0000","last_undegraded":"2023-08-09T15:11:28.729248+0000","last_fullsized":"2023-08-09T15:11:28.729248+0000","mapping_epoch":221,"log_start":"0'0","ondisk_log_start":"0'0","created":221,"last_epoch_clean":222,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T15:11:27.650831+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T15:11:27.650831+0000","last_clean_scrub_stamp":"2023-08-09T15:11:27.650831+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T17:59:31.034062+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"26.5","version":"0'0","reported_seq":11,"reported_epoch":222,"state":"active+clean","last_fresh":"2023-08-09T15:11:28.707003+0000","last_change":"2023-08-09T15:11:27.697255+0000","last_active":"2023-08-09T15:11:28.707003+0000","last_peered":"2023-08-09T15:11:28.707003+0000","last_clean":"2023-08-09T15:11:28.707003+0000","last_became_active":"2023-08-09T15:11:27.696966+0000","last_became_peered":"2023-08-09T15:11:27.696966+0000","last_unstale":"2023-08-09T15:11:28.707003+0000","last_undegraded":"2023-08-09T15:11:28.707003+0000","last_fullsized":"2023-08-09T15:11:28.707003+0000","mapping_epoch":220,"log_start":"0'0","ondisk_log_start":"0'0","created":220,"last_epoch_clean":221,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T15:11:26.638086+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T15:11:26.638086+0000","last_clean_scrub_stamp":"2023-08-09T15:11:26.638086+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T18:42:35.227494+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"27.5","version":"0'0","reported_seq":9,"reported_epoch":222,"state":"active+clean","last_fresh":"2023-08-09T15:11:28.745387+0000","last_change":"2023-08-09T15:11:28.741728+0000","last_active":"2023-08-09T15:11:28.745387+0000","last_peered":"2023-08-09T15:11:28.745387+0000","last_clean":"2023-08-09T15:11:28.745387+0000","last_became_active":"2023-08-09T15:11:28.737109+0000","last_became_peered":"2023-08-09T15:11:28.737109+0000","last_unstale":"2023-08-09T15:11:28.745387+0000","last_undegraded":"2023-08-09T15:11:28.745387+0000","last_fullsized":"2023-08-09T15:11:28.745387+0000","mapping_epoch":221,"log_start":"0'0","ondisk_log_start":"0'0","created":221,"last_epoch_clean":222,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T15:11:27.650831+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T15:11:27.650831+0000","last_clean_scrub_stamp":"2023-08-09T15:11:27.650831+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-11T02:53:02.242201+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"26.4","version":"0'0","reported_seq":11,"reported_epoch":222,"state":"active+clean","last_fresh":"2023-08-09T15:11:28.704395+0000","last_change":"2023-08-09T15:11:27.696570+0000","last_active":"2023-08-09T15:11:28.704395+0000","last_peered":"2023-08-09T15:11:28.704395+0000","last_clean":"2023-08-09T15:11:28.704395+0000","last_became_active":"2023-08-09T15:11:27.696288+0000","last_became_peered":"2023-08-09T15:11:27.696288+0000","last_unstale":"2023-08-09T15:11:28.704395+0000","last_undegraded":"2023-08-09T15:11:28.704395+0000","last_fullsized":"2023-08-09T15:11:28.704395+0000","mapping_epoch":220,"log_start":"0'0","ondisk_log_start":"0'0","created":220,"last_epoch_clean":221,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T15:11:26.638086+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T15:11:26.638086+0000","last_clean_scrub_stamp":"2023-08-09T15:11:26.638086+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T15:39:52.536660+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"27.2","version":"0'0","reported_seq":9,"reported_epoch":222,"state":"active+clean","last_fresh":"2023-08-09T15:11:28.740533+0000","last_change":"2023-08-09T15:11:28.739535+0000","last_active":"2023-08-09T15:11:28.740533+0000","last_peered":"2023-08-09T15:11:28.740533+0000","last_clean":"2023-08-09T15:11:28.740533+0000","last_became_active":"2023-08-09T15:11:28.739236+0000","last_became_peered":"2023-08-09T15:11:28.739236+0000","last_unstale":"2023-08-09T15:11:28.740533+0000","last_undegraded":"2023-08-09T15:11:28.740533+0000","last_fullsized":"2023-08-09T15:11:28.740533+0000","mapping_epoch":221,"log_start":"0'0","ondisk_log_start":"0'0","created":221,"last_epoch_clean":222,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T15:11:27.650831+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T15:11:27.650831+0000","last_clean_scrub_stamp":"2023-08-09T15:11:27.650831+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T17:17:17.167017+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"26.3","version":"0'0","reported_seq":11,"reported_epoch":222,"state":"active+clean","last_fresh":"2023-08-09T15:11:28.719799+0000","last_change":"2023-08-09T15:11:27.694233+0000","last_active":"2023-08-09T15:11:28.719799+0000","last_peered":"2023-08-09T15:11:28.719799+0000","last_clean":"2023-08-09T15:11:28.719799+0000","last_became_active":"2023-08-09T15:11:27.693874+0000","last_became_peered":"2023-08-09T15:11:27.693874+0000","last_unstale":"2023-08-09T15:11:28.719799+0000","last_undegraded":"2023-08-09T15:11:28.719799+0000","last_fullsized":"2023-08-09T15:11:28.719799+0000","mapping_epoch":220,"log_start":"0'0","ondisk_log_start":"0'0","created":220,"last_epoch_clean":221,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T15:11:26.638086+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T15:11:26.638086+0000","last_clean_scrub_stamp":"2023-08-09T15:11:26.638086+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T16:21:38.029689+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"27.3","version":"0'0","reported_seq":9,"reported_epoch":222,"state":"active+clean","last_fresh":"2023-08-09T15:11:28.734034+0000","last_change":"2023-08-09T15:11:28.731848+0000","last_active":"2023-08-09T15:11:28.734034+0000","last_peered":"2023-08-09T15:11:28.734034+0000","last_clean":"2023-08-09T15:11:28.734034+0000","last_became_active":"2023-08-09T15:11:28.731560+0000","last_became_peered":"2023-08-09T15:11:28.731560+0000","last_unstale":"2023-08-09T15:11:28.734034+0000","last_undegraded":"2023-08-09T15:11:28.734034+0000","last_fullsized":"2023-08-09T15:11:28.734034+0000","mapping_epoch":221,"log_start":"0'0","ondisk_log_start":"0'0","created":221,"last_epoch_clean":222,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T15:11:27.650831+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T15:11:27.650831+0000","last_clean_scrub_stamp":"2023-08-09T15:11:27.650831+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T17:43:40.889407+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"26.2","version":"0'0","reported_seq":11,"reported_epoch":222,"state":"active+clean","last_fresh":"2023-08-09T15:11:28.709953+0000","last_change":"2023-08-09T15:11:27.696309+0000","last_active":"2023-08-09T15:11:28.709953+0000","last_peered":"2023-08-09T15:11:28.709953+0000","last_clean":"2023-08-09T15:11:28.709953+0000","last_became_active":"2023-08-09T15:11:27.696020+0000","last_became_peered":"2023-08-09T15:11:27.696020+0000","last_unstale":"2023-08-09T15:11:28.709953+0000","last_undegraded":"2023-08-09T15:11:28.709953+0000","last_fullsized":"2023-08-09T15:11:28.709953+0000","mapping_epoch":220,"log_start":"0'0","ondisk_log_start":"0'0","created":220,"last_epoch_clean":221,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T15:11:26.638086+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T15:11:26.638086+0000","last_clean_scrub_stamp":"2023-08-09T15:11:26.638086+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T16:42:30.861840+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"27.0","version":"0'0","reported_seq":9,"reported_epoch":222,"state":"active+clean","last_fresh":"2023-08-09T15:11:28.738646+0000","last_change":"2023-08-09T15:11:28.736640+0000","last_active":"2023-08-09T15:11:28.738646+0000","last_peered":"2023-08-09T15:11:28.738646+0000","last_clean":"2023-08-09T15:11:28.738646+0000","last_became_active":"2023-08-09T15:11:28.736266+0000","last_became_peered":"2023-08-09T15:11:28.736266+0000","last_unstale":"2023-08-09T15:11:28.738646+0000","last_undegraded":"2023-08-09T15:11:28.738646+0000","last_fullsized":"2023-08-09T15:11:28.738646+0000","mapping_epoch":221,"log_start":"0'0","ondisk_log_start":"0'0","created":221,"last_epoch_clean":222,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T15:11:27.650831+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T15:11:27.650831+0000","last_clean_scrub_stamp":"2023-08-09T15:11:27.650831+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T20:23:24.264548+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"26.1","version":"0'0","reported_seq":11,"reported_epoch":222,"state":"active+clean","last_fresh":"2023-08-09T15:11:28.724556+0000","last_change":"2023-08-09T15:11:27.700679+0000","last_active":"2023-08-09T15:11:28.724556+0000","last_peered":"2023-08-09T15:11:28.724556+0000","last_clean":"2023-08-09T15:11:28.724556+0000","last_became_active":"2023-08-09T15:11:27.700284+0000","last_became_peered":"2023-08-09T15:11:27.700284+0000","last_unstale":"2023-08-09T15:11:28.724556+0000","last_undegraded":"2023-08-09T15:11:28.724556+0000","last_fullsized":"2023-08-09T15:11:28.724556+0000","mapping_epoch":220,"log_start":"0'0","ondisk_log_start":"0'0","created":220,"last_epoch_clean":221,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T15:11:26.638086+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T15:11:26.638086+0000","last_clean_scrub_stamp":"2023-08-09T15:11:26.638086+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T20:11:14.995122+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"27.1","version":"0'0","reported_seq":9,"reported_epoch":222,"state":"active+clean","last_fresh":"2023-08-09T15:11:28.735285+0000","last_change":"2023-08-09T15:11:28.731250+0000","last_active":"2023-08-09T15:11:28.735285+0000","last_peered":"2023-08-09T15:11:28.735285+0000","last_clean":"2023-08-09T15:11:28.735285+0000","last_became_active":"2023-08-09T15:11:28.730955+0000","last_became_peered":"2023-08-09T15:11:28.730955+0000","last_unstale":"2023-08-09T15:11:28.735285+0000","last_undegraded":"2023-08-09T15:11:28.735285+0000","last_fullsized":"2023-08-09T15:11:28.735285+0000","mapping_epoch":221,"log_start":"0'0","ondisk_log_start":"0'0","created":221,"last_epoch_clean":222,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T15:11:27.650831+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T15:11:27.650831+0000","last_clean_scrub_stamp":"2023-08-09T15:11:27.650831+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T17:34:00.028606+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"26.0","version":"0'0","reported_seq":11,"reported_epoch":222,"state":"active+clean","last_fresh":"2023-08-09T15:11:28.713270+0000","last_change":"2023-08-09T15:11:27.691790+0000","last_active":"2023-08-09T15:11:28.713270+0000","last_peered":"2023-08-09T15:11:28.713270+0000","last_clean":"2023-08-09T15:11:28.713270+0000","last_became_active":"2023-08-09T15:11:27.691442+0000","last_became_peered":"2023-08-09T15:11:27.691442+0000","last_unstale":"2023-08-09T15:11:28.713270+0000","last_undegraded":"2023-08-09T15:11:28.713270+0000","last_fullsized":"2023-08-09T15:11:28.713270+0000","mapping_epoch":220,"log_start":"0'0","ondisk_log_start":"0'0","created":220,"last_epoch_clean":221,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T15:11:26.638086+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T15:11:26.638086+0000","last_clean_scrub_stamp":"2023-08-09T15:11:26.638086+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T21:45:58.163091+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"27.1a","version":"0'0","reported_seq":9,"reported_epoch":222,"state":"active+clean","last_fresh":"2023-08-09T15:11:28.729738+0000","last_change":"2023-08-09T15:11:28.726755+0000","last_active":"2023-08-09T15:11:28.729738+0000","last_peered":"2023-08-09T15:11:28.729738+0000","last_clean":"2023-08-09T15:11:28.729738+0000","last_became_active":"2023-08-09T15:11:28.726364+0000","last_became_peered":"2023-08-09T15:11:28.726364+0000","last_unstale":"2023-08-09T15:11:28.729738+0000","last_undegraded":"2023-08-09T15:11:28.729738+0000","last_fullsized":"2023-08-09T15:11:28.729738+0000","mapping_epoch":221,"log_start":"0'0","ondisk_log_start":"0'0","created":221,"last_epoch_clean":222,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T15:11:27.650831+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T15:11:27.650831+0000","last_clean_scrub_stamp":"2023-08-09T15:11:27.650831+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T16:27:47.389259+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"26.1b","version":"222'1","reported_seq":12,"reported_epoch":222,"state":"active+clean","last_fresh":"2023-08-09T15:11:28.740859+0000","last_change":"2023-08-09T15:11:27.688736+0000","last_active":"2023-08-09T15:11:28.740859+0000","last_peered":"2023-08-09T15:11:28.740859+0000","last_clean":"2023-08-09T15:11:28.740859+0000","last_became_active":"2023-08-09T15:11:27.688431+0000","last_became_peered":"2023-08-09T15:11:27.688431+0000","last_unstale":"2023-08-09T15:11:28.740859+0000","last_undegraded":"2023-08-09T15:11:28.740859+0000","last_fullsized":"2023-08-09T15:11:28.740859+0000","mapping_epoch":220,"log_start":"0'0","ondisk_log_start":"0'0","created":220,"last_epoch_clean":221,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T15:11:26.638086+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T15:11:26.638086+0000","last_clean_scrub_stamp":"2023-08-09T15:11:26.638086+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-11T00:48:20.135672+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":411,"reported_epoch":222,"state":"active+clean","last_fresh":"2023-08-09T15:11:28.723778+0000","last_change":"2023-08-09T13:33:56.401406+0000","last_active":"2023-08-09T15:11:28.723778+0000","last_peered":"2023-08-09T15:11:28.723778+0000","last_clean":"2023-08-09T15:11:28.723778+0000","last_became_active":"2023-08-09T13:33:56.400487+0000","last_became_peered":"2023-08-09T13:33:56.400487+0000","last_unstale":"2023-08-09T15:11:28.723778+0000","last_undegraded":"2023-08-09T15:11:28.723778+0000","last_fullsized":"2023-08-09T15:11:28.723778+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:33:54.845869+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:33:54.845869+0000","last_clean_scrub_stamp":"2023-08-09T13:33:54.845869+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T20:59:28.425902+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"27.1b","version":"0'0","reported_seq":9,"reported_epoch":222,"state":"active+clean","last_fresh":"2023-08-09T15:11:28.745857+0000","last_change":"2023-08-09T15:11:28.745303+0000","last_active":"2023-08-09T15:11:28.745857+0000","last_peered":"2023-08-09T15:11:28.745857+0000","last_clean":"2023-08-09T15:11:28.745857+0000","last_became_active":"2023-08-09T15:11:28.744982+0000","last_became_peered":"2023-08-09T15:11:28.744982+0000","last_unstale":"2023-08-09T15:11:28.745857+0000","last_undegraded":"2023-08-09T15:11:28.745857+0000","last_fullsized":"2023-08-09T15:11:28.745857+0000","mapping_epoch":221,"log_start":"0'0","ondisk_log_start":"0'0","created":221,"last_epoch_clean":222,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T15:11:27.650831+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T15:11:27.650831+0000","last_clean_scrub_stamp":"2023-08-09T15:11:27.650831+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T19:21:28.600270+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"26.1a","version":"222'1","reported_seq":12,"reported_epoch":222,"state":"active+clean","last_fresh":"2023-08-09T15:11:28.724136+0000","last_change":"2023-08-09T15:11:27.701924+0000","last_active":"2023-08-09T15:11:28.724136+0000","last_peered":"2023-08-09T15:11:28.724136+0000","last_clean":"2023-08-09T15:11:28.724136+0000","last_became_active":"2023-08-09T15:11:27.701594+0000","last_became_peered":"2023-08-09T15:11:27.701594+0000","last_unstale":"2023-08-09T15:11:28.724136+0000","last_undegraded":"2023-08-09T15:11:28.724136+0000","last_fullsized":"2023-08-09T15:11:28.724136+0000","mapping_epoch":220,"log_start":"0'0","ondisk_log_start":"0'0","created":220,"last_epoch_clean":221,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T15:11:26.638086+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T15:11:26.638086+0000","last_clean_scrub_stamp":"2023-08-09T15:11:26.638086+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-11T01:13:04.521727+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":417,"reported_epoch":222,"state":"active+clean","last_fresh":"2023-08-09T15:11:28.707730+0000","last_change":"2023-08-09T13:33:56.401123+0000","last_active":"2023-08-09T15:11:28.707730+0000","last_peered":"2023-08-09T15:11:28.707730+0000","last_clean":"2023-08-09T15:11:28.707730+0000","last_became_active":"2023-08-09T13:33:56.399933+0000","last_became_peered":"2023-08-09T13:33:56.399933+0000","last_unstale":"2023-08-09T15:11:28.707730+0000","last_undegraded":"2023-08-09T15:11:28.707730+0000","last_fullsized":"2023-08-09T15:11:28.707730+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:33:54.845869+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:33:54.845869+0000","last_clean_scrub_stamp":"2023-08-09T13:33:54.845869+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-11T00:12:40.731648+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"27.18","version":"0'0","reported_seq":9,"reported_epoch":222,"state":"active+clean","last_fresh":"2023-08-09T15:11:28.730329+0000","last_change":"2023-08-09T15:11:28.729102+0000","last_active":"2023-08-09T15:11:28.730329+0000","last_peered":"2023-08-09T15:11:28.730329+0000","last_clean":"2023-08-09T15:11:28.730329+0000","last_became_active":"2023-08-09T15:11:28.728641+0000","last_became_peered":"2023-08-09T15:11:28.728641+0000","last_unstale":"2023-08-09T15:11:28.730329+0000","last_undegraded":"2023-08-09T15:11:28.730329+0000","last_fullsized":"2023-08-09T15:11:28.730329+0000","mapping_epoch":221,"log_start":"0'0","ondisk_log_start":"0'0","created":221,"last_epoch_clean":222,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T15:11:27.650831+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T15:11:27.650831+0000","last_clean_scrub_stamp":"2023-08-09T15:11:27.650831+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T17:58:43.444721+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"26.19","version":"0'0","reported_seq":11,"reported_epoch":222,"state":"active+clean","last_fresh":"2023-08-09T15:11:28.705420+0000","last_change":"2023-08-09T15:11:27.697411+0000","last_active":"2023-08-09T15:11:28.705420+0000","last_peered":"2023-08-09T15:11:28.705420+0000","last_clean":"2023-08-09T15:11:28.705420+0000","last_became_active":"2023-08-09T15:11:27.697016+0000","last_became_peered":"2023-08-09T15:11:27.697016+0000","last_unstale":"2023-08-09T15:11:28.705420+0000","last_undegraded":"2023-08-09T15:11:28.705420+0000","last_fullsized":"2023-08-09T15:11:28.705420+0000","mapping_epoch":220,"log_start":"0'0","ondisk_log_start":"0'0","created":220,"last_epoch_clean":221,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T15:11:26.638086+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T15:11:26.638086+0000","last_clean_scrub_stamp":"2023-08-09T15:11:26.638086+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T22:10:29.522663+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.2","version":"0'0","reported_seq":409,"reported_epoch":222,"state":"active+clean","last_fresh":"2023-08-09T15:11:28.720422+0000","last_change":"2023-08-09T13:33:56.402794+0000","last_active":"2023-08-09T15:11:28.720422+0000","last_peered":"2023-08-09T15:11:28.720422+0000","last_clean":"2023-08-09T15:11:28.720422+0000","last_became_active":"2023-08-09T13:33:56.402038+0000","last_became_peered":"2023-08-09T13:33:56.402038+0000","last_unstale":"2023-08-09T15:11:28.720422+0000","last_undegraded":"2023-08-09T15:11:28.720422+0000","last_fullsized":"2023-08-09T15:11:28.720422+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:33:54.845869+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:33:54.845869+0000","last_clean_scrub_stamp":"2023-08-09T13:33:54.845869+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T15:07:55.893669+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"27.19","version":"0'0","reported_seq":9,"reported_epoch":222,"state":"active+clean","last_fresh":"2023-08-09T15:11:28.739936+0000","last_change":"2023-08-09T15:11:28.739182+0000","last_active":"2023-08-09T15:11:28.739936+0000","last_peered":"2023-08-09T15:11:28.739936+0000","last_clean":"2023-08-09T15:11:28.739936+0000","last_became_active":"2023-08-09T15:11:28.738892+0000","last_became_peered":"2023-08-09T15:11:28.738892+0000","last_unstale":"2023-08-09T15:11:28.739936+0000","last_undegraded":"2023-08-09T15:11:28.739936+0000","last_fullsized":"2023-08-09T15:11:28.739936+0000","mapping_epoch":221,"log_start":"0'0","ondisk_log_start":"0'0","created":221,"last_epoch_clean":222,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T15:11:27.650831+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T15:11:27.650831+0000","last_clean_scrub_stamp":"2023-08-09T15:11:27.650831+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T23:40:48.203972+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"26.18","version":"0'0","reported_seq":11,"reported_epoch":222,"state":"active+clean","last_fresh":"2023-08-09T15:11:28.722942+0000","last_change":"2023-08-09T15:11:27.700112+0000","last_active":"2023-08-09T15:11:28.722942+0000","last_peered":"2023-08-09T15:11:28.722942+0000","last_clean":"2023-08-09T15:11:28.722942+0000","last_became_active":"2023-08-09T15:11:27.699806+0000","last_became_peered":"2023-08-09T15:11:27.699806+0000","last_unstale":"2023-08-09T15:11:28.722942+0000","last_undegraded":"2023-08-09T15:11:28.722942+0000","last_fullsized":"2023-08-09T15:11:28.722942+0000","mapping_epoch":220,"log_start":"0'0","ondisk_log_start":"0'0","created":220,"last_epoch_clean":221,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T15:11:26.638086+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T15:11:26.638086+0000","last_clean_scrub_stamp":"2023-08-09T15:11:26.638086+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T18:46:32.279659+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.3","version":"0'0","reported_seq":423,"reported_epoch":222,"state":"active+clean","last_fresh":"2023-08-09T15:11:28.706215+0000","last_change":"2023-08-09T13:33:55.886749+0000","last_active":"2023-08-09T15:11:28.706215+0000","last_peered":"2023-08-09T15:11:28.706215+0000","last_clean":"2023-08-09T15:11:28.706215+0000","last_became_active":"2023-08-09T13:33:55.886116+0000","last_became_peered":"2023-08-09T13:33:55.886116+0000","last_unstale":"2023-08-09T15:11:28.706215+0000","last_undegraded":"2023-08-09T15:11:28.706215+0000","last_fullsized":"2023-08-09T15:11:28.706215+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:33:54.845869+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:33:54.845869+0000","last_clean_scrub_stamp":"2023-08-09T13:33:54.845869+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T23:03:25.407293+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"211'329","reported_seq":1312,"reported_epoch":222,"state":"active+clean","last_fresh":"2023-08-09T15:11:28.723653+0000","last_change":"2023-08-09T13:33:55.886345+0000","last_active":"2023-08-09T15:11:28.723653+0000","last_peered":"2023-08-09T15:11:28.723653+0000","last_clean":"2023-08-09T15:11:28.723653+0000","last_became_active":"2023-08-09T13:33:55.885781+0000","last_became_peered":"2023-08-09T13:33:55.885781+0000","last_unstale":"2023-08-09T15:11:28.723653+0000","last_undegraded":"2023-08-09T15:11:28.723653+0000","last_fullsized":"2023-08-09T15:11:28.723653+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:33:54.845869+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:33:54.845869+0000","last_clean_scrub_stamp":"2023-08-09T13:33:54.845869+0000","objects_scrubbed":0,"log_size":329,"log_dups_size":0,"ondisk_log_size":329,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T14:52:28.626982+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1963,"num_read_kb":4096,"num_write":543,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"27.1e","version":"0'0","reported_seq":9,"reported_epoch":222,"state":"active+clean","last_fresh":"2023-08-09T15:11:28.735347+0000","last_change":"2023-08-09T15:11:28.731087+0000","last_active":"2023-08-09T15:11:28.735347+0000","last_peered":"2023-08-09T15:11:28.735347+0000","last_clean":"2023-08-09T15:11:28.735347+0000","last_became_active":"2023-08-09T15:11:28.730778+0000","last_became_peered":"2023-08-09T15:11:28.730778+0000","last_unstale":"2023-08-09T15:11:28.735347+0000","last_undegraded":"2023-08-09T15:11:28.735347+0000","last_fullsized":"2023-08-09T15:11:28.735347+0000","mapping_epoch":221,"log_start":"0'0","ondisk_log_start":"0'0","created":221,"last_epoch_clean":222,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T15:11:27.650831+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T15:11:27.650831+0000","last_clean_scrub_stamp":"2023-08-09T15:11:27.650831+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T23:09:43.428168+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"26.1f","version":"222'4","reported_seq":15,"reported_epoch":222,"state":"active+clean","last_fresh":"2023-08-09T15:11:28.743494+0000","last_change":"2023-08-09T15:11:27.695330+0000","last_active":"2023-08-09T15:11:28.743494+0000","last_peered":"2023-08-09T15:11:28.743494+0000","last_clean":"2023-08-09T15:11:28.743494+0000","last_became_active":"2023-08-09T15:11:27.695008+0000","last_became_peered":"2023-08-09T15:11:27.695008+0000","last_unstale":"2023-08-09T15:11:28.743494+0000","last_undegraded":"2023-08-09T15:11:28.743494+0000","last_fullsized":"2023-08-09T15:11:28.743494+0000","mapping_epoch":220,"log_start":"0'0","ondisk_log_start":"0'0","created":220,"last_epoch_clean":221,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T15:11:26.638086+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T15:11:26.638086+0000","last_clean_scrub_stamp":"2023-08-09T15:11:26.638086+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-11T02:38:21.656496+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":427,"reported_epoch":222,"state":"active+clean","last_fresh":"2023-08-09T15:11:28.699926+0000","last_change":"2023-08-09T13:33:55.889565+0000","last_active":"2023-08-09T15:11:28.699926+0000","last_peered":"2023-08-09T15:11:28.699926+0000","last_clean":"2023-08-09T15:11:28.699926+0000","last_became_active":"2023-08-09T13:33:55.889164+0000","last_became_peered":"2023-08-09T13:33:55.889164+0000","last_unstale":"2023-08-09T15:11:28.699926+0000","last_undegraded":"2023-08-09T15:11:28.699926+0000","last_fullsized":"2023-08-09T15:11:28.699926+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:33:54.845869+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:33:54.845869+0000","last_clean_scrub_stamp":"2023-08-09T13:33:54.845869+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T20:53:51.649656+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"27.1f","version":"0'0","reported_seq":9,"reported_epoch":222,"state":"active+clean","last_fresh":"2023-08-09T15:11:28.745479+0000","last_change":"2023-08-09T15:11:28.732705+0000","last_active":"2023-08-09T15:11:28.745479+0000","last_peered":"2023-08-09T15:11:28.745479+0000","last_clean":"2023-08-09T15:11:28.745479+0000","last_became_active":"2023-08-09T15:11:28.732431+0000","last_became_peered":"2023-08-09T15:11:28.732431+0000","last_unstale":"2023-08-09T15:11:28.745479+0000","last_undegraded":"2023-08-09T15:11:28.745479+0000","last_fullsized":"2023-08-09T15:11:28.745479+0000","mapping_epoch":221,"log_start":"0'0","ondisk_log_start":"0'0","created":221,"last_epoch_clean":222,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T15:11:27.650831+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T15:11:27.650831+0000","last_clean_scrub_stamp":"2023-08-09T15:11:27.650831+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T20:46:32.635231+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"26.1e","version":"222'1","reported_seq":12,"reported_epoch":222,"state":"active+clean","last_fresh":"2023-08-09T15:11:28.730475+0000","last_change":"2023-08-09T15:11:27.694291+0000","last_active":"2023-08-09T15:11:28.730475+0000","last_peered":"2023-08-09T15:11:28.730475+0000","last_clean":"2023-08-09T15:11:28.730475+0000","last_became_active":"2023-08-09T15:11:27.693664+0000","last_became_peered":"2023-08-09T15:11:27.693664+0000","last_unstale":"2023-08-09T15:11:28.730475+0000","last_undegraded":"2023-08-09T15:11:28.730475+0000","last_fullsized":"2023-08-09T15:11:28.730475+0000","mapping_epoch":220,"log_start":"0'0","ondisk_log_start":"0'0","created":220,"last_epoch_clean":221,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T15:11:26.638086+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T15:11:26.638086+0000","last_clean_scrub_stamp":"2023-08-09T15:11:26.638086+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T23:34:36.700017+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":415,"reported_epoch":222,"state":"active+clean","last_fresh":"2023-08-09T15:11:28.728714+0000","last_change":"2023-08-09T13:33:56.404236+0000","last_active":"2023-08-09T15:11:28.728714+0000","last_peered":"2023-08-09T15:11:28.728714+0000","last_clean":"2023-08-09T15:11:28.728714+0000","last_became_active":"2023-08-09T13:33:56.402485+0000","last_became_peered":"2023-08-09T13:33:56.402485+0000","last_unstale":"2023-08-09T15:11:28.728714+0000","last_undegraded":"2023-08-09T15:11:28.728714+0000","last_fullsized":"2023-08-09T15:11:28.728714+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:33:54.845869+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:33:54.845869+0000","last_clean_scrub_stamp":"2023-08-09T13:33:54.845869+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T13:56:39.809056+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"27.1c","version":"0'0","reported_seq":9,"reported_epoch":222,"state":"active+clean","last_fresh":"2023-08-09T15:11:28.740530+0000","last_change":"2023-08-09T15:11:28.739252+0000","last_active":"2023-08-09T15:11:28.740530+0000","last_peered":"2023-08-09T15:11:28.740530+0000","last_clean":"2023-08-09T15:11:28.740530+0000","last_became_active":"2023-08-09T15:11:28.738158+0000","last_became_peered":"2023-08-09T15:11:28.738158+0000","last_unstale":"2023-08-09T15:11:28.740530+0000","last_undegraded":"2023-08-09T15:11:28.740530+0000","last_fullsized":"2023-08-09T15:11:28.740530+0000","mapping_epoch":221,"log_start":"0'0","ondisk_log_start":"0'0","created":221,"last_epoch_clean":222,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T15:11:27.650831+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T15:11:27.650831+0000","last_clean_scrub_stamp":"2023-08-09T15:11:27.650831+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T15:35:15.135484+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"26.1d","version":"222'2","reported_seq":13,"reported_epoch":222,"state":"active+clean","last_fresh":"2023-08-09T15:11:28.729546+0000","last_change":"2023-08-09T15:11:27.690620+0000","last_active":"2023-08-09T15:11:28.729546+0000","last_peered":"2023-08-09T15:11:28.729546+0000","last_clean":"2023-08-09T15:11:28.729546+0000","last_became_active":"2023-08-09T15:11:27.690312+0000","last_became_peered":"2023-08-09T15:11:27.690312+0000","last_unstale":"2023-08-09T15:11:28.729546+0000","last_undegraded":"2023-08-09T15:11:28.729546+0000","last_fullsized":"2023-08-09T15:11:28.729546+0000","mapping_epoch":220,"log_start":"0'0","ondisk_log_start":"0'0","created":220,"last_epoch_clean":221,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T15:11:26.638086+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T15:11:26.638086+0000","last_clean_scrub_stamp":"2023-08-09T15:11:26.638086+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T18:13:52.359530+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":408,"reported_epoch":222,"state":"active+clean","last_fresh":"2023-08-09T15:11:28.716347+0000","last_change":"2023-08-09T13:33:56.401781+0000","last_active":"2023-08-09T15:11:28.716347+0000","last_peered":"2023-08-09T15:11:28.716347+0000","last_clean":"2023-08-09T15:11:28.716347+0000","last_became_active":"2023-08-09T13:33:56.400336+0000","last_became_peered":"2023-08-09T13:33:56.400336+0000","last_unstale":"2023-08-09T15:11:28.716347+0000","last_undegraded":"2023-08-09T15:11:28.716347+0000","last_fullsized":"2023-08-09T15:11:28.716347+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:33:54.845869+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:33:54.845869+0000","last_clean_scrub_stamp":"2023-08-09T13:33:54.845869+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-11T00:43:56.815579+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"27.1d","version":"0'0","reported_seq":9,"reported_epoch":222,"state":"active+clean","last_fresh":"2023-08-09T15:11:28.729148+0000","last_change":"2023-08-09T15:11:28.725941+0000","last_active":"2023-08-09T15:11:28.729148+0000","last_peered":"2023-08-09T15:11:28.729148+0000","last_clean":"2023-08-09T15:11:28.729148+0000","last_became_active":"2023-08-09T15:11:28.725586+0000","last_became_peered":"2023-08-09T15:11:28.725586+0000","last_unstale":"2023-08-09T15:11:28.729148+0000","last_undegraded":"2023-08-09T15:11:28.729148+0000","last_fullsized":"2023-08-09T15:11:28.729148+0000","mapping_epoch":221,"log_start":"0'0","ondisk_log_start":"0'0","created":221,"last_epoch_clean":222,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T15:11:27.650831+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T15:11:27.650831+0000","last_clean_scrub_stamp":"2023-08-09T15:11:27.650831+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T20:53:21.300084+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"26.1c","version":"222'2","reported_seq":13,"reported_epoch":222,"state":"active+clean","last_fresh":"2023-08-09T15:11:28.742373+0000","last_change":"2023-08-09T15:11:27.697916+0000","last_active":"2023-08-09T15:11:28.742373+0000","last_peered":"2023-08-09T15:11:28.742373+0000","last_clean":"2023-08-09T15:11:28.742373+0000","last_became_active":"2023-08-09T15:11:27.697637+0000","last_became_peered":"2023-08-09T15:11:27.697637+0000","last_unstale":"2023-08-09T15:11:28.742373+0000","last_undegraded":"2023-08-09T15:11:28.742373+0000","last_fullsized":"2023-08-09T15:11:28.742373+0000","mapping_epoch":220,"log_start":"0'0","ondisk_log_start":"0'0","created":220,"last_epoch_clean":221,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T15:11:26.638086+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T15:11:26.638086+0000","last_clean_scrub_stamp":"2023-08-09T15:11:26.638086+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T20:54:00.982392+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":408,"reported_epoch":222,"state":"active+clean","last_fresh":"2023-08-09T15:11:28.725867+0000","last_change":"2023-08-09T13:33:55.886389+0000","last_active":"2023-08-09T15:11:28.725867+0000","last_peered":"2023-08-09T15:11:28.725867+0000","last_clean":"2023-08-09T15:11:28.725867+0000","last_became_active":"2023-08-09T13:33:55.885638+0000","last_became_peered":"2023-08-09T13:33:55.885638+0000","last_unstale":"2023-08-09T15:11:28.725867+0000","last_undegraded":"2023-08-09T15:11:28.725867+0000","last_fullsized":"2023-08-09T15:11:28.725867+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:33:54.845869+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:33:54.845869+0000","last_clean_scrub_stamp":"2023-08-09T13:33:54.845869+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T13:34:44.592008+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]}],"pool_stats":[{"poolid":26,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":38,"ondisk_log_size":38,"up":64,"acting":64,"num_store_stats":8},{"poolid":2,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1963,"num_read_kb":4096,"num_write":543,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":329,"ondisk_log_size":329,"up":2,"acting":2,"num_store_stats":2},{"poolid":27,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":8}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834576013,"num_pgs":17,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":52476,"kb_used_data":3308,"kb_used_omap":1,"kb_used_meta":49150,"kb_avail":94319364,"statfs":{"total":96636764160,"available":96583028736,"internally_reserved":0,"allocated":3387392,"data_stored":2107110,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1781,"internal_metadata":50329867},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Aug 9 15:10:49 2023","interfaces":[{"interface":"back","average":{"1min":1.2330000000000001,"5min":1.2569999999999999,"15min":1.234},"min":{"1min":0.81599999999999995,"5min":0.76700000000000002,"15min":0.76100000000000001},"max":{"1min":1.6739999999999999,"5min":2.056,"15min":2.056},"last":1.3049999999999999},{"interface":"front","average":{"1min":1.2689999999999999,"5min":1.264,"15min":1.2649999999999999},"min":{"1min":0.72899999999999998,"5min":0.72899999999999998,"15min":0.72899999999999998},"max":{"1min":1.732,"5min":1.863,"15min":1.863},"last":1.5169999999999999}]},{"osd":1,"last update":"Wed Aug 9 15:10:49 2023","interfaces":[{"interface":"back","average":{"1min":1.337,"5min":1.294,"15min":1.3160000000000001},"min":{"1min":0.95399999999999996,"5min":0.63500000000000001,"15min":0.63500000000000001},"max":{"1min":1.7090000000000001,"5min":1.853,"15min":1.853},"last":1.1950000000000001},{"interface":"front","average":{"1min":1.3,"5min":1.3380000000000001,"15min":1.3149999999999999},"min":{"1min":0.96499999999999997,"5min":0.879,"15min":0.879},"max":{"1min":1.6519999999999999,"5min":2.1520000000000001,"15min":2.1520000000000001},"last":1.1499999999999999}]},{"osd":2,"last update":"Wed Aug 9 15:10:49 2023","interfaces":[{"interface":"back","average":{"1min":1.3500000000000001,"5min":1.3160000000000001,"15min":1.3069999999999999},"min":{"1min":0.95299999999999996,"5min":0.81299999999999994,"15min":0.81299999999999994},"max":{"1min":1.861,"5min":1.861,"15min":1.861},"last":1.3340000000000001},{"interface":"front","average":{"1min":1.3480000000000001,"5min":1.397,"15min":1.4019999999999999},"min":{"1min":1.0189999999999999,"5min":1.0189999999999999,"15min":0.88500000000000001},"max":{"1min":1.603,"5min":1.9370000000000001,"15min":1.9370000000000001},"last":1.1020000000000001}]},{"osd":3,"last update":"Wed Aug 9 15:10:49 2023","interfaces":[{"interface":"back","average":{"1min":1.387,"5min":1.3500000000000001,"15min":1.3580000000000001},"min":{"1min":1.024,"5min":0.80800000000000005,"15min":0.80800000000000005},"max":{"1min":1.7370000000000001,"5min":1.881,"15min":1.9259999999999999},"last":0.98099999999999998},{"interface":"front","average":{"1min":1.446,"5min":1.466,"15min":1.48},"min":{"1min":1.127,"5min":0.92700000000000005,"15min":0.92700000000000005},"max":{"1min":1.9239999999999999,"5min":1.9239999999999999,"15min":1.9239999999999999},"last":1.603}]},{"osd":4,"last update":"Wed Aug 9 15:10:49 2023","interfaces":[{"interface":"back","average":{"1min":1.3320000000000001,"5min":1.298,"15min":1.3009999999999999},"min":{"1min":0.90300000000000002,"5min":0.86799999999999999,"15min":0.69599999999999995},"max":{"1min":1.8300000000000001,"5min":2.2810000000000001,"15min":2.2810000000000001},"last":1.2749999999999999},{"interface":"front","average":{"1min":1.4019999999999999,"5min":1.3839999999999999,"15min":1.3879999999999999},"min":{"1min":0.97499999999999998,"5min":0.85099999999999998,"15min":0.66000000000000003},"max":{"1min":1.875,"5min":1.875,"15min":2.0830000000000002},"last":1.474}]},{"osd":5,"last update":"Wed Aug 9 15:10:49 2023","interfaces":[{"interface":"back","average":{"1min":1.395,"5min":1.431,"15min":1.4510000000000001},"min":{"1min":0.89900000000000002,"5min":0.86099999999999999,"15min":0.86099999999999999},"max":{"1min":1.7589999999999999,"5min":1.9550000000000001,"15min":1.9550000000000001},"last":1.4259999999999999},{"interface":"front","average":{"1min":1.405,"5min":1.4670000000000001,"15min":1.4830000000000001},"min":{"1min":0.89000000000000001,"5min":0.89000000000000001,"15min":0.89000000000000001},"max":{"1min":1.825,"5min":2.5310000000000001,"15min":2.5310000000000001},"last":1.724}]},{"osd":6,"last update":"Wed Aug 9 15:10:49 2023","interfaces":[{"interface":"back","average":{"1min":1.55,"5min":1.617,"15min":1.5980000000000001},"min":{"1min":1.1819999999999999,"5min":1.121,"15min":1.101},"max":{"1min":2.1850000000000001,"5min":2.8959999999999999,"15min":2.8959999999999999},"last":1.6499999999999999},{"interface":"front","average":{"1min":1.522,"5min":1.556,"15min":1.546},"min":{"1min":1.026,"5min":1.026,"15min":0.90800000000000003},"max":{"1min":1.992,"5min":2.214,"15min":2.222},"last":1.5609999999999999}]}]},{"osd":6,"up_from":13,"seq":55834576013,"num_pgs":15,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":60072,"kb_used_data":2716,"kb_used_omap":1,"kb_used_meta":57342,"kb_avail":94311768,"statfs":{"total":96636764160,"available":96575250432,"internally_reserved":0,"allocated":2781184,"data_stored":1514351,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1781,"internal_metadata":58718475},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Aug 9 15:10:57 2023","interfaces":[{"interface":"back","average":{"1min":1.1830000000000001,"5min":1.228,"15min":1.22},"min":{"1min":0.75600000000000001,"5min":0.75600000000000001,"15min":0.75600000000000001},"max":{"1min":1.734,"5min":1.772,"15min":1.8859999999999999},"last":1.468},{"interface":"front","average":{"1min":1.2589999999999999,"5min":1.2490000000000001,"15min":1.254},"min":{"1min":0.83799999999999997,"5min":0.83799999999999997,"15min":0.77400000000000002},"max":{"1min":1.639,"5min":2.0800000000000001,"15min":2.0800000000000001},"last":1.266}]},{"osd":1,"last update":"Wed Aug 9 15:10:57 2023","interfaces":[{"interface":"back","average":{"1min":1.319,"5min":1.2050000000000001,"15min":1.2010000000000001},"min":{"1min":0.90300000000000002,"5min":0.70999999999999996,"15min":0.70999999999999996},"max":{"1min":2.6760000000000002,"5min":2.6760000000000002,"15min":2.6760000000000002},"last":1.2430000000000001},{"interface":"front","average":{"1min":1.2969999999999999,"5min":1.3100000000000001,"15min":1.345},"min":{"1min":0.90300000000000002,"5min":0.82299999999999995,"15min":0.82299999999999995},"max":{"1min":1.998,"5min":1.998,"15min":1.998},"last":1.339}]},{"osd":2,"last update":"Wed Aug 9 15:10:57 2023","interfaces":[{"interface":"back","average":{"1min":1.351,"5min":1.3400000000000001,"15min":1.3480000000000001},"min":{"1min":0.91900000000000004,"5min":0.82099999999999995,"15min":0.82099999999999995},"max":{"1min":1.8149999999999999,"5min":2.0089999999999999,"15min":2.081},"last":1.5369999999999999},{"interface":"front","average":{"1min":1.4299999999999999,"5min":1.3400000000000001,"15min":1.3540000000000001},"min":{"1min":1.04,"5min":0.80700000000000005,"15min":0.80700000000000005},"max":{"1min":1.9219999999999999,"5min":1.9219999999999999,"15min":1.9219999999999999},"last":1.163}]},{"osd":3,"last update":"Wed Aug 9 15:10:57 2023","interfaces":[{"interface":"back","average":{"1min":1.4910000000000001,"5min":1.4390000000000001,"15min":1.4099999999999999},"min":{"1min":1.1359999999999999,"5min":0.79700000000000004,"15min":0.79700000000000004},"max":{"1min":1.859,"5min":2.0179999999999998,"15min":2.0179999999999998},"last":0.85299999999999998},{"interface":"front","average":{"1min":1.3340000000000001,"5min":1.3420000000000001,"15min":1.351},"min":{"1min":0.95899999999999996,"5min":0.95899999999999996,"15min":0.95899999999999996},"max":{"1min":1.71,"5min":2.0539999999999998,"15min":2.0539999999999998},"last":1.002}]},{"osd":4,"last update":"Wed Aug 9 15:10:57 2023","interfaces":[{"interface":"back","average":{"1min":1.4299999999999999,"5min":1.3560000000000001,"15min":1.3300000000000001},"min":{"1min":0.89700000000000002,"5min":0.53500000000000003,"15min":0.53500000000000003},"max":{"1min":2.0409999999999999,"5min":2.0470000000000002,"15min":2.0470000000000002},"last":1.593},{"interface":"front","average":{"1min":1.5600000000000001,"5min":1.4850000000000001,"15min":1.4350000000000001},"min":{"1min":0.96199999999999997,"5min":0.82099999999999995,"15min":0.82099999999999995},"max":{"1min":2.1139999999999999,"5min":2.1589999999999998,"15min":2.1589999999999998},"last":1.05}]},{"osd":5,"last update":"Wed Aug 9 15:10:57 2023","interfaces":[{"interface":"back","average":{"1min":1.492,"5min":1.4370000000000001,"15min":1.381},"min":{"1min":0.86599999999999999,"5min":0.86599999999999999,"15min":0.80600000000000005},"max":{"1min":1.881,"5min":2.1269999999999998,"15min":2.6059999999999999},"last":1.133},{"interface":"front","average":{"1min":1.7090000000000001,"5min":1.6599999999999999,"15min":1.625},"min":{"1min":1.3919999999999999,"5min":0.99399999999999999,"15min":0.85599999999999998},"max":{"1min":2.0609999999999999,"5min":2.125,"15min":2.1899999999999999},"last":1.6639999999999999}]},{"osd":7,"last update":"Wed Aug 9 15:10:57 2023","interfaces":[{"interface":"back","average":{"1min":1.7210000000000001,"5min":1.671,"15min":1.6160000000000001},"min":{"1min":1.2689999999999999,"5min":1.1240000000000001,"15min":0.84699999999999998},"max":{"1min":2.2229999999999999,"5min":2.2280000000000002,"15min":2.4340000000000002},"last":1.508},{"interface":"front","average":{"1min":1.6970000000000001,"5min":1.643,"15min":1.6020000000000001},"min":{"1min":1.2649999999999999,"5min":1.0940000000000001,"15min":1.0209999999999999},"max":{"1min":2.1349999999999998,"5min":2.3300000000000001,"15min":2.7400000000000002},"last":1.3109999999999999}]}]},{"osd":5,"up_from":13,"seq":55834576013,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":47776,"kb_used_data":2704,"kb_used_omap":1,"kb_used_meta":45054,"kb_avail":94324064,"statfs":{"total":96636764160,"available":96587841536,"internally_reserved":0,"allocated":2768896,"data_stored":1507638,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1781,"internal_metadata":46135563},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Aug 9 15:10:47 2023","interfaces":[{"interface":"back","average":{"1min":1.23,"5min":1.218,"15min":1.206},"min":{"1min":0.97699999999999998,"5min":0.59399999999999997,"15min":0.59399999999999997},"max":{"1min":1.6120000000000001,"5min":1.7370000000000001,"15min":1.754},"last":0.57699999999999996},{"interface":"front","average":{"1min":1.262,"5min":1.2230000000000001,"15min":1.224},"min":{"1min":0.81699999999999995,"5min":0.57899999999999996,"15min":0.57899999999999996},"max":{"1min":1.6870000000000001,"5min":1.865,"15min":1.976},"last":0.55600000000000005}]},{"osd":1,"last update":"Wed Aug 9 15:10:47 2023","interfaces":[{"interface":"back","average":{"1min":1.282,"5min":1.2769999999999999,"15min":1.254},"min":{"1min":0.89200000000000002,"5min":0.65200000000000002,"15min":0.65200000000000002},"max":{"1min":1.726,"5min":2.016,"15min":2.016},"last":0.67500000000000004},{"interface":"front","average":{"1min":1.3939999999999999,"5min":1.319,"15min":1.351},"min":{"1min":0.98999999999999999,"5min":0.74299999999999999,"15min":0.74299999999999999},"max":{"1min":1.7150000000000001,"5min":1.8959999999999999,"15min":1.8979999999999999},"last":0.65000000000000002}]},{"osd":2,"last update":"Wed Aug 9 15:10:47 2023","interfaces":[{"interface":"back","average":{"1min":1.3280000000000001,"5min":1.3220000000000001,"15min":1.321},"min":{"1min":0.88900000000000001,"5min":0.88700000000000001,"15min":0.82499999999999996},"max":{"1min":1.8859999999999999,"5min":1.952,"15min":2.032},"last":0.60199999999999998},{"interface":"front","average":{"1min":1.367,"5min":1.3700000000000001,"15min":1.3720000000000001},"min":{"1min":1.0760000000000001,"5min":0.81100000000000005,"15min":0.81100000000000005},"max":{"1min":1.6679999999999999,"5min":1.964,"15min":1.964},"last":0.61899999999999999}]},{"osd":3,"last update":"Wed Aug 9 15:10:47 2023","interfaces":[{"interface":"back","average":{"1min":1.4239999999999999,"5min":1.389,"15min":1.395},"min":{"1min":0.95999999999999996,"5min":0.95999999999999996,"15min":0.95099999999999996},"max":{"1min":1.796,"5min":1.798,"15min":1.899},"last":0.746},{"interface":"front","average":{"1min":1.393,"5min":1.4390000000000001,"15min":1.45},"min":{"1min":0.99099999999999999,"5min":0.85999999999999999,"15min":0.85999999999999999},"max":{"1min":1.7509999999999999,"5min":2.0379999999999998,"15min":2.0379999999999998},"last":0.66400000000000003}]},{"osd":4,"last update":"Wed Aug 9 15:10:47 2023","interfaces":[{"interface":"back","average":{"1min":1.266,"5min":1.3140000000000001,"15min":1.3220000000000001},"min":{"1min":0.84099999999999997,"5min":0.84099999999999997,"15min":0.62},"max":{"1min":1.7210000000000001,"5min":2.0299999999999998,"15min":2.0299999999999998},"last":0.53000000000000003},{"interface":"front","average":{"1min":1.3029999999999999,"5min":1.377,"15min":1.405},"min":{"1min":0.76200000000000001,"5min":0.76200000000000001,"15min":0.64200000000000002},"max":{"1min":1.6950000000000001,"5min":1.9239999999999999,"15min":2.0390000000000001},"last":0.69199999999999995}]},{"osd":6,"last update":"Wed Aug 9 15:10:47 2023","interfaces":[{"interface":"back","average":{"1min":1.4079999999999999,"5min":1.3919999999999999,"15min":1.421},"min":{"1min":0.97799999999999998,"5min":0.96099999999999997,"15min":0.67400000000000004},"max":{"1min":1.712,"5min":2.1000000000000001,"15min":2.1000000000000001},"last":0.501},{"interface":"front","average":{"1min":1.448,"5min":1.4970000000000001,"15min":1.506},"min":{"1min":1.0449999999999999,"5min":1.0449999999999999,"15min":0.79900000000000004},"max":{"1min":1.839,"5min":2.1349999999999998,"15min":2.1349999999999998},"last":0.70599999999999996}]},{"osd":7,"last update":"Wed Aug 9 15:10:47 2023","interfaces":[{"interface":"back","average":{"1min":1.577,"5min":1.548,"15min":1.587},"min":{"1min":1.1499999999999999,"5min":1.0569999999999999,"15min":0.86099999999999999},"max":{"1min":1.8280000000000001,"5min":1.9670000000000001,"15min":2.4689999999999999},"last":0.71999999999999997},{"interface":"front","average":{"1min":1.585,"5min":1.6060000000000001,"15min":1.6439999999999999},"min":{"1min":1.101,"5min":1.044,"15min":0.83399999999999996},"max":{"1min":1.8939999999999999,"5min":2.1539999999999999,"15min":2.4239999999999999},"last":0.63800000000000001}]}]},{"osd":4,"up_from":13,"seq":55834576013,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":47804,"kb_used_data":2736,"kb_used_omap":1,"kb_used_meta":45054,"kb_avail":94324036,"statfs":{"total":96636764160,"available":96587812864,"internally_reserved":0,"allocated":2801664,"data_stored":1516846,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1781,"internal_metadata":46135563},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Aug 9 15:11:11 2023","interfaces":[{"interface":"back","average":{"1min":1.3089999999999999,"5min":1.1970000000000001,"15min":1.226},"min":{"1min":0.81399999999999995,"5min":0.60399999999999998,"15min":0.60399999999999998},"max":{"1min":1.708,"5min":1.708,"15min":2.2959999999999998},"last":0.73999999999999999},{"interface":"front","average":{"1min":1.3540000000000001,"5min":1.3009999999999999,"15min":1.3200000000000001},"min":{"1min":0.85699999999999998,"5min":0.66800000000000004,"15min":0.66800000000000004},"max":{"1min":2.0779999999999998,"5min":2.0779999999999998,"15min":2.2709999999999999},"last":1.5489999999999999}]},{"osd":1,"last update":"Wed Aug 9 15:11:11 2023","interfaces":[{"interface":"back","average":{"1min":1.3480000000000001,"5min":1.3580000000000001,"15min":1.355},"min":{"1min":0.95899999999999996,"5min":0.74199999999999999,"15min":0.74199999999999999},"max":{"1min":1.911,"5min":1.931,"15min":1.931},"last":1.413},{"interface":"front","average":{"1min":1.306,"5min":1.258,"15min":1.282},"min":{"1min":0.74299999999999999,"5min":0.73899999999999999,"15min":0.73899999999999999},"max":{"1min":1.6890000000000001,"5min":1.8600000000000001,"15min":2.117},"last":1.117}]},{"osd":2,"last update":"Wed Aug 9 15:11:11 2023","interfaces":[{"interface":"back","average":{"1min":1.27,"5min":1.246,"15min":1.2929999999999999},"min":{"1min":0.63100000000000001,"5min":0.63100000000000001,"15min":0.63100000000000001},"max":{"1min":1.7709999999999999,"5min":1.7709999999999999,"15min":1.883},"last":0.875},{"interface":"front","average":{"1min":1.4239999999999999,"5min":1.415,"15min":1.409},"min":{"1min":0.88700000000000001,"5min":0.79200000000000004,"15min":0.79200000000000004},"max":{"1min":2.306,"5min":2.343,"15min":2.343},"last":1.492}]},{"osd":3,"last update":"Wed Aug 9 15:11:11 2023","interfaces":[{"interface":"back","average":{"1min":1.411,"5min":1.3580000000000001,"15min":1.357},"min":{"1min":0.93899999999999995,"5min":0.88700000000000001,"15min":0.88700000000000001},"max":{"1min":1.8009999999999999,"5min":1.8009999999999999,"15min":2.4870000000000001},"last":0.95299999999999996},{"interface":"front","average":{"1min":1.4279999999999999,"5min":1.4390000000000001,"15min":1.4690000000000001},"min":{"1min":0.79700000000000004,"5min":0.79700000000000004,"15min":0.79700000000000004},"max":{"1min":1.8660000000000001,"5min":1.909,"15min":2.5459999999999998},"last":1.173}]},{"osd":5,"last update":"Wed Aug 9 15:11:11 2023","interfaces":[{"interface":"back","average":{"1min":1.3120000000000001,"5min":1.3620000000000001,"15min":1.3999999999999999},"min":{"1min":0.82199999999999995,"5min":0.82199999999999995,"15min":0.82199999999999995},"max":{"1min":2.633,"5min":2.633,"15min":2.633},"last":1.2410000000000001},{"interface":"front","average":{"1min":1.355,"5min":1.4079999999999999,"15min":1.393},"min":{"1min":0.68100000000000005,"5min":0.68100000000000005,"15min":0.68100000000000005},"max":{"1min":1.9930000000000001,"5min":1.9930000000000001,"15min":2.4249999999999998},"last":1.3620000000000001}]},{"osd":6,"last update":"Wed Aug 9 15:11:11 2023","interfaces":[{"interface":"back","average":{"1min":1.4119999999999999,"5min":1.4099999999999999,"15min":1.4139999999999999},"min":{"1min":0.71299999999999997,"5min":0.71299999999999997,"15min":0.71299999999999997},"max":{"1min":1.927,"5min":2.0049999999999999,"15min":2.9119999999999999},"last":1.0509999999999999},{"interface":"front","average":{"1min":1.571,"5min":1.5840000000000001,"15min":1.5800000000000001},"min":{"1min":0.77000000000000002,"5min":0.77000000000000002,"15min":0.77000000000000002},"max":{"1min":2.9529999999999998,"5min":2.9529999999999998,"15min":2.9529999999999998},"last":1.607}]},{"osd":7,"last update":"Wed Aug 9 15:11:11 2023","interfaces":[{"interface":"back","average":{"1min":1.5,"5min":1.52,"15min":1.53},"min":{"1min":0.83899999999999997,"5min":0.83899999999999997,"15min":0.83899999999999997},"max":{"1min":2.1589999999999998,"5min":2.1589999999999998,"15min":2.5910000000000002},"last":1.2150000000000001},{"interface":"front","average":{"1min":1.6419999999999999,"5min":1.5800000000000001,"15min":1.609},"min":{"1min":0.90000000000000002,"5min":0.90000000000000002,"15min":0.90000000000000002},"max":{"1min":2.218,"5min":2.218,"15min":3.028},"last":1.298}]}]},{"osd":3,"up_from":13,"seq":55834576014,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":47800,"kb_used_data":2728,"kb_used_omap":1,"kb_used_meta":45054,"kb_avail":94324040,"statfs":{"total":96636764160,"available":96587816960,"internally_reserved":0,"allocated":2793472,"data_stored":1515312,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1781,"internal_metadata":46135563},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Aug 9 15:11:03 2023","interfaces":[{"interface":"back","average":{"1min":1.105,"5min":0.999,"15min":0.98499999999999999},"min":{"1min":0.77900000000000003,"5min":0.45200000000000001,"15min":0.45200000000000001},"max":{"1min":1.657,"5min":1.6699999999999999,"15min":2.2090000000000001},"last":0.47699999999999998},{"interface":"front","average":{"1min":1.208,"5min":1.137,"15min":1.167},"min":{"1min":0.872,"5min":0.45200000000000001,"15min":0.45200000000000001},"max":{"1min":1.6020000000000001,"5min":1.9630000000000001,"15min":1.9630000000000001},"last":0.58399999999999996}]},{"osd":1,"last update":"Wed Aug 9 15:11:03 2023","interfaces":[{"interface":"back","average":{"1min":1.0449999999999999,"5min":1.0469999999999999,"15min":1.0780000000000001},"min":{"1min":0.80300000000000005,"5min":0.53900000000000003,"15min":0.53900000000000003},"max":{"1min":1.446,"5min":1.956,"15min":2.27},"last":0.70999999999999996},{"interface":"front","average":{"1min":1.276,"5min":1.2529999999999999,"15min":1.266},"min":{"1min":0.98799999999999999,"5min":0.498,"15min":0.498},"max":{"1min":1.6419999999999999,"5min":1.9730000000000001,"15min":1.9730000000000001},"last":0.68000000000000005}]},{"osd":2,"last update":"Wed Aug 9 15:11:03 2023","interfaces":[{"interface":"back","average":{"1min":1.387,"5min":1.296,"15min":1.337},"min":{"1min":1.002,"5min":0.70799999999999996,"15min":0.70799999999999996},"max":{"1min":1.861,"5min":1.9650000000000001,"15min":2.4079999999999999},"last":1.1870000000000001},{"interface":"front","average":{"1min":1.4470000000000001,"5min":1.4279999999999999,"15min":1.4410000000000001},"min":{"1min":1.077,"5min":0.72499999999999998,"15min":0.72499999999999998},"max":{"1min":1.883,"5min":2.0510000000000002,"15min":2.1259999999999999},"last":0.80800000000000005}]},{"osd":4,"last update":"Wed Aug 9 15:11:03 2023","interfaces":[{"interface":"back","average":{"1min":1.3540000000000001,"5min":1.3720000000000001,"15min":1.4099999999999999},"min":{"1min":1.01,"5min":0.748,"15min":0.72599999999999998},"max":{"1min":1.708,"5min":1.9179999999999999,"15min":2.2069999999999999},"last":0.752},{"interface":"front","average":{"1min":1.5349999999999999,"5min":1.4510000000000001,"15min":1.4650000000000001},"min":{"1min":1.1919999999999999,"5min":0.90500000000000003,"15min":0.89500000000000002},"max":{"1min":1.863,"5min":1.968,"15min":2.056},"last":1.0109999999999999}]},{"osd":5,"last update":"Wed Aug 9 15:11:03 2023","interfaces":[{"interface":"back","average":{"1min":1.5620000000000001,"5min":1.518,"15min":1.548},"min":{"1min":1.3320000000000001,"5min":0.88700000000000001,"15min":0.88700000000000001},"max":{"1min":2.0870000000000002,"5min":2.0870000000000002,"15min":2.1669999999999998},"last":0.84299999999999997},{"interface":"front","average":{"1min":1.704,"5min":1.6040000000000001,"15min":1.6160000000000001},"min":{"1min":1.337,"5min":0.85799999999999998,"15min":0.85799999999999998},"max":{"1min":2.2290000000000001,"5min":2.2469999999999999,"15min":2.4049999999999998},"last":0.89300000000000002}]},{"osd":6,"last update":"Wed Aug 9 15:11:03 2023","interfaces":[{"interface":"back","average":{"1min":1.532,"5min":1.5269999999999999,"15min":1.595},"min":{"1min":1.0569999999999999,"5min":0.81200000000000006,"15min":0.81200000000000006},"max":{"1min":1.968,"5min":2.4860000000000002,"15min":2.4860000000000002},"last":0.69599999999999995},{"interface":"front","average":{"1min":1.673,"5min":1.6319999999999999,"15min":1.669},"min":{"1min":1.2270000000000001,"5min":0.95499999999999996,"15min":0.95299999999999996},"max":{"1min":2.214,"5min":2.4039999999999999,"15min":2.4039999999999999},"last":0.92200000000000004}]},{"osd":7,"last update":"Wed Aug 9 15:11:03 2023","interfaces":[{"interface":"back","average":{"1min":1.8069999999999999,"5min":1.738,"15min":1.762},"min":{"1min":1.4239999999999999,"5min":1.123,"15min":1.123},"max":{"1min":2.347,"5min":2.347,"15min":2.347},"last":1.1259999999999999},{"interface":"front","average":{"1min":1.635,"5min":1.6779999999999999,"15min":1.718},"min":{"1min":1.3620000000000001,"5min":0.84899999999999998,"15min":0.84899999999999998},"max":{"1min":1.8480000000000001,"5min":3.3220000000000001,"15min":3.3220000000000001},"last":1.1539999999999999}]}]},{"osd":2,"up_from":13,"seq":55834576013,"num_pgs":6,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":42024,"kb_used_data":2716,"kb_used_omap":1,"kb_used_meta":39294,"kb_avail":94329816,"statfs":{"total":96636764160,"available":96593731584,"internally_reserved":0,"allocated":2781184,"data_stored":1515130,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1781,"internal_metadata":40237323},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Aug 9 15:10:49 2023","interfaces":[{"interface":"back","average":{"1min":1.081,"5min":0.98699999999999999,"15min":0.96399999999999997},"min":{"1min":0.72399999999999998,"5min":0.44800000000000001,"15min":0.44800000000000001},"max":{"1min":1.556,"5min":1.556,"15min":1.587},"last":0.85599999999999998},{"interface":"front","average":{"1min":1.111,"5min":1.101,"15min":1.0940000000000001},"min":{"1min":0.82099999999999995,"5min":0.59499999999999997,"15min":0.59499999999999997},"max":{"1min":1.629,"5min":1.764,"15min":2.052},"last":1.141}]},{"osd":1,"last update":"Wed Aug 9 15:10:49 2023","interfaces":[{"interface":"back","average":{"1min":1.3740000000000001,"5min":1.288,"15min":1.2490000000000001},"min":{"1min":1.127,"5min":0.72399999999999998,"15min":0.72399999999999998},"max":{"1min":1.5449999999999999,"5min":2.0179999999999998,"15min":2.0179999999999998},"last":1.089},{"interface":"front","average":{"1min":1.452,"5min":1.3480000000000001,"15min":1.4059999999999999},"min":{"1min":1.1459999999999999,"5min":0.67900000000000005,"15min":0.67900000000000005},"max":{"1min":1.6870000000000001,"5min":1.95,"15min":1.95},"last":1.276}]},{"osd":3,"last update":"Wed Aug 9 15:10:49 2023","interfaces":[{"interface":"back","average":{"1min":1.3029999999999999,"5min":1.111,"15min":1.111},"min":{"1min":0.89100000000000001,"5min":0.52100000000000002,"15min":0.52100000000000002},"max":{"1min":1.802,"5min":1.871,"15min":1.871},"last":0.90200000000000002},{"interface":"front","average":{"1min":1.46,"5min":1.3939999999999999,"15min":1.385},"min":{"1min":0.94499999999999995,"5min":0.64800000000000002,"15min":0.64800000000000002},"max":{"1min":1.9299999999999999,"5min":1.9299999999999999,"15min":2.0470000000000002},"last":1.1990000000000001}]},{"osd":4,"last update":"Wed Aug 9 15:10:49 2023","interfaces":[{"interface":"back","average":{"1min":1.659,"5min":1.5720000000000001,"15min":1.5569999999999999},"min":{"1min":1.355,"5min":0.90000000000000002,"15min":0.90000000000000002},"max":{"1min":1.9790000000000001,"5min":2.0649999999999999,"15min":2.464},"last":1.6950000000000001},{"interface":"front","average":{"1min":1.4770000000000001,"5min":1.472,"15min":1.466},"min":{"1min":1.153,"5min":0.81799999999999995,"15min":0.81799999999999995},"max":{"1min":1.8400000000000001,"5min":2.0720000000000001,"15min":2.0720000000000001},"last":1.649}]},{"osd":5,"last update":"Wed Aug 9 15:10:49 2023","interfaces":[{"interface":"back","average":{"1min":1.496,"5min":1.538,"15min":1.528},"min":{"1min":1.177,"5min":0.82799999999999996,"15min":0.82799999999999996},"max":{"1min":1.879,"5min":2.0369999999999999,"15min":2.1139999999999999},"last":1.804},{"interface":"front","average":{"1min":1.5269999999999999,"5min":1.4530000000000001,"15min":1.496},"min":{"1min":1.2310000000000001,"5min":0.93999999999999995,"15min":0.93999999999999995},"max":{"1min":1.968,"5min":1.968,"15min":2.214},"last":1.732}]},{"osd":6,"last update":"Wed Aug 9 15:10:49 2023","interfaces":[{"interface":"back","average":{"1min":1.5349999999999999,"5min":1.5129999999999999,"15min":1.528},"min":{"1min":1.0820000000000001,"5min":1.0680000000000001,"15min":1.022},"max":{"1min":2.1110000000000002,"5min":2.1110000000000002,"15min":2.2999999999999998},"last":1.5629999999999999},{"interface":"front","average":{"1min":1.7709999999999999,"5min":1.677,"15min":1.645},"min":{"1min":1.28,"5min":1.0649999999999999,"15min":1.0649999999999999},"max":{"1min":2.077,"5min":2.177,"15min":2.351},"last":1.7709999999999999}]},{"osd":7,"last update":"Wed Aug 9 15:10:49 2023","interfaces":[{"interface":"back","average":{"1min":1.6870000000000001,"5min":1.6419999999999999,"15min":1.6439999999999999},"min":{"1min":1.3129999999999999,"5min":0.87,"15min":0.87},"max":{"1min":2.0840000000000001,"5min":2.302,"15min":2.302},"last":1.8999999999999999},{"interface":"front","average":{"1min":1.6910000000000001,"5min":1.6259999999999999,"15min":1.637},"min":{"1min":1.4139999999999999,"5min":1.0189999999999999,"15min":1.0189999999999999},"max":{"1min":2.242,"5min":2.3809999999999998,"15min":2.3809999999999998},"last":1.6040000000000001}]}]},{"osd":1,"up_from":13,"seq":55834576013,"num_pgs":9,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":56564,"kb_used_data":3304,"kb_used_omap":1,"kb_used_meta":53246,"kb_avail":94315276,"statfs":{"total":96636764160,"available":96578842624,"internally_reserved":0,"allocated":3383296,"data_stored":2105554,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1781,"internal_metadata":54524171},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Aug 9 15:11:02 2023","interfaces":[{"interface":"back","average":{"1min":1.0529999999999999,"5min":1.006,"15min":1.0129999999999999},"min":{"1min":0.77200000000000002,"5min":0.42299999999999999,"15min":0.42299999999999999},"max":{"1min":1.8799999999999999,"5min":1.8799999999999999,"15min":2.0939999999999999},"last":1.139},{"interface":"front","average":{"1min":1.2749999999999999,"5min":1.1479999999999999,"15min":1.147},"min":{"1min":0.86399999999999999,"5min":0.46500000000000002,"15min":0.46500000000000002},"max":{"1min":1.7549999999999999,"5min":1.78,"15min":1.8939999999999999},"last":1.212}]},{"osd":2,"last update":"Wed Aug 9 15:11:02 2023","interfaces":[{"interface":"back","average":{"1min":1.03,"5min":1.048,"15min":1.0469999999999999},"min":{"1min":0.77000000000000002,"5min":0.55800000000000005,"15min":0.55800000000000005},"max":{"1min":1.361,"5min":1.589,"15min":2.3940000000000001},"last":2.2240000000000002},{"interface":"front","average":{"1min":1.3839999999999999,"5min":1.2509999999999999,"15min":1.2789999999999999},"min":{"1min":0.94699999999999995,"5min":0.46700000000000003,"15min":0.46700000000000003},"max":{"1min":2.169,"5min":2.169,"15min":2.169},"last":1.2829999999999999}]},{"osd":3,"last update":"Wed Aug 9 15:11:02 2023","interfaces":[{"interface":"back","average":{"1min":1.5409999999999999,"5min":1.397,"15min":1.4039999999999999},"min":{"1min":1.0860000000000001,"5min":0.51000000000000001,"15min":0.51000000000000001},"max":{"1min":2.093,"5min":2.093,"15min":2.109},"last":1.569},{"interface":"front","average":{"1min":1.3879999999999999,"5min":1.329,"15min":1.331},"min":{"1min":1.0189999999999999,"5min":0.66000000000000003,"15min":0.66000000000000003},"max":{"1min":2.052,"5min":2.052,"15min":2.052},"last":1.8680000000000001}]},{"osd":4,"last update":"Wed Aug 9 15:11:02 2023","interfaces":[{"interface":"back","average":{"1min":1.53,"5min":1.4950000000000001,"15min":1.4890000000000001},"min":{"1min":1.145,"5min":0.753,"15min":0.753},"max":{"1min":2.0419999999999998,"5min":2.0419999999999998,"15min":2.214},"last":1.6180000000000001},{"interface":"front","average":{"1min":1.45,"5min":1.4299999999999999,"15min":1.423},"min":{"1min":0.997,"5min":0.69199999999999995,"15min":0.69199999999999995},"max":{"1min":2.0310000000000001,"5min":2.0310000000000001,"15min":2.1190000000000002},"last":1.496}]},{"osd":5,"last update":"Wed Aug 9 15:11:02 2023","interfaces":[{"interface":"back","average":{"1min":1.448,"5min":1.4550000000000001,"15min":1.46},"min":{"1min":1.071,"5min":0.77800000000000002,"15min":0.77800000000000002},"max":{"1min":1.994,"5min":1.994,"15min":2.508},"last":1.909},{"interface":"front","average":{"1min":1.663,"5min":1.603,"15min":1.607},"min":{"1min":1.238,"5min":0.79100000000000004,"15min":0.79100000000000004},"max":{"1min":2.3809999999999998,"5min":2.3809999999999998,"15min":2.3809999999999998},"last":1.99}]},{"osd":6,"last update":"Wed Aug 9 15:11:02 2023","interfaces":[{"interface":"back","average":{"1min":1.706,"5min":1.647,"15min":1.6299999999999999},"min":{"1min":1.298,"5min":0.85199999999999998,"15min":0.85199999999999998},"max":{"1min":2.347,"5min":2.347,"15min":2.347},"last":2.1669999999999998},{"interface":"front","average":{"1min":1.617,"5min":1.577,"15min":1.5880000000000001},"min":{"1min":1.147,"5min":0.73499999999999999,"15min":0.73499999999999999},"max":{"1min":2.218,"5min":2.218,"15min":2.278},"last":1.7030000000000001}]},{"osd":7,"last update":"Wed Aug 9 15:11:02 2023","interfaces":[{"interface":"back","average":{"1min":1.744,"5min":1.7210000000000001,"15min":1.714},"min":{"1min":1.446,"5min":0.86399999999999999,"15min":0.86399999999999999},"max":{"1min":2.3090000000000002,"5min":2.3090000000000002,"15min":2.488},"last":2.1299999999999999},{"interface":"front","average":{"1min":1.78,"5min":1.7190000000000001,"15min":1.7150000000000001},"min":{"1min":1.4019999999999999,"5min":0.93400000000000005,"15min":0.93400000000000005},"max":{"1min":2.4260000000000002,"5min":2.4260000000000002,"15min":2.4260000000000002},"last":2.0579999999999998}]}]},{"osd":0,"up_from":13,"seq":55834576013,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":29736,"kb_used_data":2716,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94342104,"statfs":{"total":96636764160,"available":96606314496,"internally_reserved":0,"allocated":2781184,"data_stored":1513859,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1781,"internal_metadata":27654411},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Wed Aug 9 15:10:59 2023","interfaces":[{"interface":"back","average":{"1min":0.95599999999999996,"5min":0.90800000000000003,"15min":0.95499999999999996},"min":{"1min":0.65300000000000002,"5min":0.40000000000000002,"15min":0.40000000000000002},"max":{"1min":1.5680000000000001,"5min":1.617,"15min":1.6299999999999999},"last":0.93200000000000005},{"interface":"front","average":{"1min":1.161,"5min":1.1220000000000001,"15min":1.153},"min":{"1min":0.61699999999999999,"5min":0.61699999999999999,"15min":0.61699999999999999},"max":{"1min":2.1560000000000001,"5min":2.1560000000000001,"15min":2.1560000000000001},"last":1.105}]},{"osd":2,"last update":"Wed Aug 9 15:10:59 2023","interfaces":[{"interface":"back","average":{"1min":1.163,"5min":1.1799999999999999,"15min":1.1970000000000001},"min":{"1min":0.73299999999999998,"5min":0.505,"15min":0.505},"max":{"1min":1.5960000000000001,"5min":2.4769999999999999,"15min":2.4769999999999999},"last":1.401},{"interface":"front","average":{"1min":1.238,"5min":1.1850000000000001,"15min":1.236},"min":{"1min":0.80000000000000004,"5min":0.53100000000000003,"15min":0.53100000000000003},"max":{"1min":1.8180000000000001,"5min":1.8180000000000001,"15min":1.9470000000000001},"last":1.26}]},{"osd":3,"last update":"Wed Aug 9 15:10:59 2023","interfaces":[{"interface":"back","average":{"1min":1.1359999999999999,"5min":1.173,"15min":1.2170000000000001},"min":{"1min":0.65800000000000003,"5min":0.56299999999999994,"15min":0.56299999999999994},"max":{"1min":1.492,"5min":2.1459999999999999,"15min":2.1459999999999999},"last":1.23},{"interface":"front","average":{"1min":1.407,"5min":1.347,"15min":1.4019999999999999},"min":{"1min":0.79700000000000004,"5min":0.59999999999999998,"15min":0.59999999999999998},"max":{"1min":1.867,"5min":1.867,"15min":2.2469999999999999},"last":1.3180000000000001}]},{"osd":4,"last update":"Wed Aug 9 15:10:59 2023","interfaces":[{"interface":"back","average":{"1min":1.339,"5min":1.371,"15min":1.417},"min":{"1min":1.1060000000000001,"5min":0.88200000000000001,"15min":0.88200000000000001},"max":{"1min":1.825,"5min":1.853,"15min":1.966},"last":1.044},{"interface":"front","average":{"1min":1.484,"5min":1.476,"15min":1.52},"min":{"1min":0.93999999999999995,"5min":0.79400000000000004,"15min":0.79400000000000004},"max":{"1min":1.972,"5min":2.0369999999999999,"15min":2.278},"last":1.5069999999999999}]},{"osd":5,"last update":"Wed Aug 9 15:10:59 2023","interfaces":[{"interface":"back","average":{"1min":1.5580000000000001,"5min":1.5269999999999999,"15min":1.54},"min":{"1min":1.2809999999999999,"5min":0.96299999999999997,"15min":0.96299999999999997},"max":{"1min":1.9139999999999999,"5min":2.355,"15min":2.355},"last":1.5589999999999999},{"interface":"front","average":{"1min":1.46,"5min":1.4810000000000001,"15min":1.506},"min":{"1min":0.93200000000000005,"5min":0.82899999999999996,"15min":0.82899999999999996},"max":{"1min":1.9019999999999999,"5min":2.0800000000000001,"15min":2.3740000000000001},"last":1.367}]},{"osd":6,"last update":"Wed Aug 9 15:10:59 2023","interfaces":[{"interface":"back","average":{"1min":1.5669999999999999,"5min":1.603,"15min":1.637},"min":{"1min":1.0740000000000001,"5min":1.0740000000000001,"15min":1.0740000000000001},"max":{"1min":2.0369999999999999,"5min":2.7490000000000001,"15min":2.7490000000000001},"last":1.599},{"interface":"front","average":{"1min":1.583,"5min":1.591,"15min":1.621},"min":{"1min":1.1319999999999999,"5min":0.84699999999999998,"15min":0.84699999999999998},"max":{"1min":2.077,"5min":2.274,"15min":2.2839999999999998},"last":1.9450000000000001}]},{"osd":7,"last update":"Wed Aug 9 15:10:59 2023","interfaces":[{"interface":"back","average":{"1min":1.6479999999999999,"5min":1.641,"15min":1.6859999999999999},"min":{"1min":0.94899999999999995,"5min":0.94899999999999995,"15min":0.94899999999999995},"max":{"1min":2.012,"5min":2.645,"15min":2.645},"last":1.8340000000000001},{"interface":"front","average":{"1min":1.6100000000000001,"5min":1.6120000000000001,"15min":1.6259999999999999},"min":{"1min":1.2110000000000001,"5min":0.89100000000000001,"15min":0.89100000000000001},"max":{"1min":1.9530000000000001,"5min":2.2000000000000002,"15min":2.3759999999999999},"last":1.4299999999999999}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":26,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":26,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":56,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":26,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":26,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":182,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":26,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":1716,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":26,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":26,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":26,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":27,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":27,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":27,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":27,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":27,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":27,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":27,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":27,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-08-09T15:11:33.837 INFO:tasks.cephfs.filesystem.ceph_manager:clean! 2023-08-09T15:11:33.838 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth caps client.0 mon 'allow r' osd 'allow rw tag cephfs data=cephfs' mds allow 2023-08-09T15:11:34.115 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:34.115+0000 7f6826c3f640 1 Processor -- start 2023-08-09T15:11:34.115 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:34.115+0000 7f6826c3f640 1 -- start start 2023-08-09T15:11:34.117 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:34.117+0000 7f6826c3f640 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7f68200cb2d0 0x7f682015b380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T15:11:34.117 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:34.117+0000 7f6826c3f640 1 -- --> v1:172.21.15.43:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f682015aef0 con 0x7f68200ce8d0 2023-08-09T15:11:34.117 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:34.117+0000 7f6826c3f640 1 -- --> v1:172.21.15.203:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f682015b8c0 con 0x7f682015df60 2023-08-09T15:11:34.117 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:34.117+0000 7f6826c3f640 1 -- --> v2:172.21.15.203:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f682015a970 con 0x7f68200cb2d0 2023-08-09T15:11:34.118 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:34.117+0000 7f68251dc640 1 --1- >> v1:172.21.15.43:6789/0 conn(0x7f68200ce8d0 0x7f68200c9f90 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.43:6789/0 says I am v1:172.21.15.43:42920/0 (socket says 172.21.15.43:42920) 2023-08-09T15:11:34.119 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:34.117+0000 7f68251dc640 1 -- 172.21.15.43:0/130176193 learned_addr learned my addr 172.21.15.43:0/130176193 (peer_addr_for_me v1:172.21.15.43:0/0) 2023-08-09T15:11:34.119 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:34.117+0000 7f68249db640 1 --2- 172.21.15.43:0/130176193 >> v2:172.21.15.203:3300/0 conn(0x7f68200cb2d0 0x7f682015b380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T15:11:34.119 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:34.118+0000 7f68177fe640 1 -- 172.21.15.43:0/130176193 <== mon.0 v1:172.21.15.43:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1890102309 0 0) 0x7f682015aef0 con 0x7f68200ce8d0 2023-08-09T15:11:34.119 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:34.118+0000 7f68177fe640 1 -- 172.21.15.43:0/130176193 --> v1:172.21.15.43:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f6808003520 con 0x7f68200ce8d0 2023-08-09T15:11:34.119 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:34.118+0000 7f68177fe640 1 -- 172.21.15.43:0/130176193 <== mon.0 v1:172.21.15.43:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 4217092227 0 0) 0x7f6808003520 con 0x7f68200ce8d0 2023-08-09T15:11:34.119 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:34.118+0000 7f68177fe640 1 -- 172.21.15.43:0/130176193 >> v2:172.21.15.203:3300/0 conn(0x7f68200cb2d0 msgr2=0x7f682015b380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-09T15:11:34.120 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:34.118+0000 7f68177fe640 1 --2- 172.21.15.43:0/130176193 >> v2:172.21.15.203:3300/0 conn(0x7f68200cb2d0 0x7f682015b380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T15:11:34.120 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:34.118+0000 7f68177fe640 1 -- 172.21.15.43:0/130176193 >> v1:172.21.15.203:6789/0 conn(0x7f682015df60 legacy=0x7f68200ce1c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T15:11:34.120 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:34.118+0000 7f68177fe640 1 -- 172.21.15.43:0/130176193 --> v1:172.21.15.43:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f682015e940 con 0x7f68200ce8d0 2023-08-09T15:11:34.120 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:34.118+0000 7f68177fe640 1 -- 172.21.15.43:0/130176193 <== mon.0 v1:172.21.15.43:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1285150740 0 0) 0x7f68180019f0 con 0x7f68200ce8d0 2023-08-09T15:11:34.120 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:34.118+0000 7f68177fe640 1 -- 172.21.15.43:0/130176193 >> v1:172.21.15.43:6789/0 conn(0x7f68200ce8d0 legacy=0x7f68200c9f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T15:11:34.120 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:34.118+0000 7f68177fe640 1 --2- 172.21.15.43:0/130176193 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f6808003970 0x7f6808003d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T15:11:34.120 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:34.118+0000 7f68177fe640 1 --2- 172.21.15.43:0/130176193 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f68080047c0 0x7f6808004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T15:11:34.120 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:34.118+0000 7f68177fe640 1 --2- 172.21.15.43:0/130176193 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f6808005180 0x7f680800b980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T15:11:34.121 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:34.118+0000 7f68177fe640 1 -- 172.21.15.43:0/130176193 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f682015a970 con 0x7f6808003970 2023-08-09T15:11:34.121 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:34.118+0000 7f68177fe640 1 -- 172.21.15.43:0/130176193 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f680800c020 con 0x7f6808005180 2023-08-09T15:11:34.121 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:34.118+0000 7f68177fe640 1 -- 172.21.15.43:0/130176193 --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f680800c1e0 con 0x7f68080047c0 2023-08-09T15:11:34.121 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:34.118+0000 7f68249db640 1 --2- 172.21.15.43:0/130176193 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f6808003970 0x7f6808003d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T15:11:34.122 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:34.119+0000 7f68251dc640 1 --2- 172.21.15.43:0/130176193 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f6808005180 0x7f680800b980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T15:11:34.122 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:34.119+0000 7f68249db640 1 -- 172.21.15.43:0/130176193 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f68080047c0 msgr2=0x7f6808004c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-09T15:11:34.123 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:34.119+0000 7f68249db640 1 --2- 172.21.15.43:0/130176193 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f68080047c0 0x7f6808004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T15:11:34.123 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:34.119+0000 7f68249db640 1 -- 172.21.15.43:0/130176193 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f6808005180 msgr2=0x7f680800b980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-09T15:11:34.123 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:34.119+0000 7f68249db640 1 --2- 172.21.15.43:0/130176193 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f6808005180 0x7f680800b980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T15:11:34.123 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:34.119+0000 7f68249db640 1 -- 172.21.15.43:0/130176193 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6808003520 con 0x7f6808003970 2023-08-09T15:11:34.123 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:34.119+0000 7f68251dc640 1 --2- 172.21.15.43:0/130176193 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f6808005180 0x7f680800b980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-08-09T15:11:34.123 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:34.119+0000 7f68251dc640 1 -- 172.21.15.43:0/130176193 reap_dead start 2023-08-09T15:11:34.123 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:34.120+0000 7f68249db640 1 --2- 172.21.15.43:0/130176193 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f6808003970 0x7f6808003d90 secure :-1 s=READY pgs=2414 cs=0 l=1 rev1=1 crypto rx=0x7f681c009ac0 tx=0x7f68080057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d0699052c3619743 server_cookie=0 in_seq=0 out_seq=0 2023-08-09T15:11:34.123 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:34.121+0000 7f68177fe640 1 -- 172.21.15.43:0/130176193 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f681c02c6b0 con 0x7f6808003970 2023-08-09T15:11:34.124 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:34.121+0000 7f68177fe640 1 -- 172.21.15.43:0/130176193 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f681c02c810 con 0x7f6808003970 2023-08-09T15:11:34.124 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:34.121+0000 7f68177fe640 1 -- 172.21.15.43:0/130176193 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f681c02cba0 con 0x7f6808003970 2023-08-09T15:11:34.124 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:34.121+0000 7f6826c3f640 1 -- 172.21.15.43:0/130176193 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f6808003970 msgr2=0x7f6808003d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T15:11:34.124 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:34.121+0000 7f6826c3f640 1 --2- 172.21.15.43:0/130176193 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f6808003970 0x7f6808003d90 secure :-1 s=READY pgs=2414 cs=0 l=1 rev1=1 crypto rx=0x7f681c009ac0 tx=0x7f68080057f0 comp rx=0 tx=0).stop 2023-08-09T15:11:34.124 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:34.121+0000 7f6826c3f640 1 -- 172.21.15.43:0/130176193 shutdown_connections 2023-08-09T15:11:34.124 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:34.121+0000 7f6826c3f640 1 --2- 172.21.15.43:0/130176193 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f6808003970 0x7f6808003d90 unknown :-1 s=CLOSED pgs=2414 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T15:11:34.124 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:34.121+0000 7f6826c3f640 1 -- 172.21.15.43:0/130176193 >> 172.21.15.43:0/130176193 conn(0x7f68200aff50 msgr2=0x7f68200b0360 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-09T15:11:34.125 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:34.121+0000 7f6826c3f640 1 -- 172.21.15.43:0/130176193 shutdown_connections 2023-08-09T15:11:34.125 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:34.121+0000 7f6826c3f640 1 -- 172.21.15.43:0/130176193 wait complete. 2023-08-09T15:11:34.126 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:34.123+0000 7f6826c3f640 1 Processor -- start 2023-08-09T15:11:34.126 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:34.123+0000 7f6826c3f640 1 -- start start 2023-08-09T15:11:34.126 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:34.124+0000 7f6826c3f640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f6808003970 0x7f68200cbb80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T15:11:34.126 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:34.124+0000 7f6826c3f640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f6820150580 0x7f68200c9f10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T15:11:34.127 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:34.125+0000 7f6826c3f640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f6820149de0 0x7f682014a260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T15:11:34.127 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:34.125+0000 7f6826c3f640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f682014a7a0 con 0x7f6808003970 2023-08-09T15:11:34.127 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:34.125+0000 7f6826c3f640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f68200ac830 con 0x7f6820150580 2023-08-09T15:11:34.127 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:34.125+0000 7f6826c3f640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f68200cc0c0 con 0x7f6820149de0 2023-08-09T15:11:34.127 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:34.125+0000 7f68249db640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f6808003970 0x7f68200cbb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T15:11:34.127 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:34.125+0000 7f68249db640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f6808003970 0x7f68200cbb80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:56178/0 (socket says 172.21.15.43:56178) 2023-08-09T15:11:34.127 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:34.125+0000 7f68249db640 1 -- 172.21.15.43:0/1594887119 learned_addr learned my addr 172.21.15.43:0/1594887119 (peer_addr_for_me v2:172.21.15.43:0/0) 2023-08-09T15:11:34.127 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:34.125+0000 7f68251dc640 1 --2- 172.21.15.43:0/1594887119 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f6820149de0 0x7f682014a260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T15:11:34.128 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:34.125+0000 7f6817fff640 1 --2- 172.21.15.43:0/1594887119 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f6820150580 0x7f68200c9f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T15:11:34.128 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:34.125+0000 7f68249db640 1 -- 172.21.15.43:0/1594887119 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f6820149de0 msgr2=0x7f682014a260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T15:11:34.128 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:34.125+0000 7f68249db640 1 --2- 172.21.15.43:0/1594887119 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f6820149de0 0x7f682014a260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T15:11:34.128 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:34.125+0000 7f68249db640 1 -- 172.21.15.43:0/1594887119 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f6820150580 msgr2=0x7f68200c9f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T15:11:34.128 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:34.125+0000 7f68249db640 1 --2- 172.21.15.43:0/1594887119 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f6820150580 0x7f68200c9f10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T15:11:34.128 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:34.125+0000 7f68249db640 1 -- 172.21.15.43:0/1594887119 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f682014aa30 con 0x7f6808003970 2023-08-09T15:11:34.128 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:34.125+0000 7f6817fff640 1 --2- 172.21.15.43:0/1594887119 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f6820150580 0x7f68200c9f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-08-09T15:11:34.129 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:34.126+0000 7f68249db640 1 --2- 172.21.15.43:0/1594887119 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f6808003970 0x7f68200cbb80 secure :-1 s=READY pgs=2415 cs=0 l=1 rev1=1 crypto rx=0x7f681c009ac0 tx=0x7f681c004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-09T15:11:34.129 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:34.126+0000 7f68157fa640 1 -- 172.21.15.43:0/1594887119 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f681c003810 con 0x7f6808003970 2023-08-09T15:11:34.130 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:34.126+0000 7f68157fa640 1 -- 172.21.15.43:0/1594887119 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f681c003970 con 0x7f6808003970 2023-08-09T15:11:34.130 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:34.126+0000 7f6826c3f640 1 -- 172.21.15.43:0/1594887119 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f68200cc340 con 0x7f6808003970 2023-08-09T15:11:34.130 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:34.126+0000 7f68157fa640 1 -- 172.21.15.43:0/1594887119 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f681c003d00 con 0x7f6808003970 2023-08-09T15:11:34.130 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:34.126+0000 7f6826c3f640 1 -- 172.21.15.43:0/1594887119 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f68200abd50 con 0x7f6808003970 2023-08-09T15:11:34.130 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:34.128+0000 7f6826c3f640 1 -- 172.21.15.43:0/1594887119 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f68200739f0 con 0x7f6808003970 2023-08-09T15:11:34.130 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:34.128+0000 7f68157fa640 1 -- 172.21.15.43:0/1594887119 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 118) v1 ==== 88443+0+0 (secure 0 0 0) 0x7f681c040450 con 0x7f6808003970 2023-08-09T15:11:34.130 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:34.129+0000 7f68157fa640 1 --2- 172.21.15.43:0/1594887119 >> [v2:172.21.15.203:6832/4034093268,v1:172.21.15.203:6833/4034093268] conn(0x7f680c06a170 0x7f680c06c630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T15:11:34.131 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:34.129+0000 7f68157fa640 1 -- 172.21.15.43:0/1594887119 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(223..223 src has 1..223) v4 ==== 6426+0+0 (secure 0 0 0) 0x7f681c0aa7b0 con 0x7f6808003970 2023-08-09T15:11:34.131 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:34.130+0000 7f6817fff640 1 --2- 172.21.15.43:0/1594887119 >> [v2:172.21.15.203:6832/4034093268,v1:172.21.15.203:6833/4034093268] conn(0x7f680c06a170 0x7f680c06c630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T15:11:34.131 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:34.131+0000 7f6817fff640 1 --2- 172.21.15.43:0/1594887119 >> [v2:172.21.15.203:6832/4034093268,v1:172.21.15.203:6833/4034093268] conn(0x7f680c06a170 0x7f680c06c630 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f68200cd510 tx=0x7f6810009040 comp rx=0 tx=0).ready entity=mgr.8833 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-09T15:11:34.135 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:34.134+0000 7f68157fa640 1 -- 172.21.15.43:0/1594887119 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178066 (secure 0 0 0) 0x7f681c075b80 con 0x7f6808003970 2023-08-09T15:11:34.394 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:34.393+0000 7f6826c3f640 1 -- 172.21.15.43:0/1594887119 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]} v 0) v1 -- 0x7f68200aeb10 con 0x7f6808003970 2023-08-09T15:11:34.401 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:34.401+0000 7f68157fa640 1 -- 172.21.15.43:0/1594887119 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]}]=0 updated caps for client.0 v48) v1 ==== 190+0+147 (secure 0 0 0) 0x7f681c003e60 con 0x7f6808003970 2023-08-09T15:11:34.401 INFO:teuthology.orchestra.run.smithi043.stdout:[client.0] 2023-08-09T15:11:34.402 INFO:teuthology.orchestra.run.smithi043.stdout: key = AQCPldNkg8zrDxAAM/hnJaE6S9HU/fRwQeBV5w== 2023-08-09T15:11:34.402 INFO:teuthology.orchestra.run.smithi043.stdout: caps mds = "allow" 2023-08-09T15:11:34.402 INFO:teuthology.orchestra.run.smithi043.stdout: caps mon = "allow r" 2023-08-09T15:11:34.402 INFO:teuthology.orchestra.run.smithi043.stdout: caps osd = "allow rw tag cephfs data=cephfs" 2023-08-09T15:11:34.402 INFO:teuthology.orchestra.run.smithi043.stderr:updated caps for client.0 2023-08-09T15:11:34.404 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:34.403+0000 7f6826c3f640 1 -- 172.21.15.43:0/1594887119 >> [v2:172.21.15.203:6832/4034093268,v1:172.21.15.203:6833/4034093268] conn(0x7f680c06a170 msgr2=0x7f680c06c630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T15:11:34.404 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:34.403+0000 7f6826c3f640 1 --2- 172.21.15.43:0/1594887119 >> [v2:172.21.15.203:6832/4034093268,v1:172.21.15.203:6833/4034093268] conn(0x7f680c06a170 0x7f680c06c630 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f68200cd510 tx=0x7f6810009040 comp rx=0 tx=0).stop 2023-08-09T15:11:34.404 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:34.403+0000 7f6826c3f640 1 -- 172.21.15.43:0/1594887119 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f6808003970 msgr2=0x7f68200cbb80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T15:11:34.405 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:34.403+0000 7f6826c3f640 1 --2- 172.21.15.43:0/1594887119 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f6808003970 0x7f68200cbb80 secure :-1 s=READY pgs=2415 cs=0 l=1 rev1=1 crypto rx=0x7f681c009ac0 tx=0x7f681c004730 comp rx=0 tx=0).stop 2023-08-09T15:11:34.405 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:34.403+0000 7f6826c3f640 1 -- 172.21.15.43:0/1594887119 shutdown_connections 2023-08-09T15:11:34.405 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:34.403+0000 7f6826c3f640 1 --2- 172.21.15.43:0/1594887119 >> [v2:172.21.15.203:6832/4034093268,v1:172.21.15.203:6833/4034093268] conn(0x7f680c06a170 0x7f680c06c630 unknown :-1 s=CLOSED pgs=66 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T15:11:34.405 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:34.403+0000 7f6826c3f640 1 --2- 172.21.15.43:0/1594887119 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f6820149de0 0x7f682014a260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T15:11:34.405 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:34.403+0000 7f6826c3f640 1 --2- 172.21.15.43:0/1594887119 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f6820150580 0x7f68200c9f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T15:11:34.405 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:34.403+0000 7f6826c3f640 1 --2- 172.21.15.43:0/1594887119 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f6808003970 0x7f68200cbb80 unknown :-1 s=CLOSED pgs=2415 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T15:11:34.405 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:34.403+0000 7f6826c3f640 1 -- 172.21.15.43:0/1594887119 >> 172.21.15.43:0/1594887119 conn(0x7f68200aff50 msgr2=0x7f68200ce190 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-09T15:11:34.406 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:34.404+0000 7f6826c3f640 1 -- 172.21.15.43:0/1594887119 shutdown_connections 2023-08-09T15:11:34.406 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:34.404+0000 7f6826c3f640 1 -- 172.21.15.43:0/1594887119 wait complete. 2023-08-09T15:11:34.422 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-08-09T15:11:34.641 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:34.640+0000 7fce36542640 1 Processor -- start 2023-08-09T15:11:34.641 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:34.641+0000 7fce36542640 1 -- start start 2023-08-09T15:11:34.642 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:34.642+0000 7fce36542640 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7fce300b2b70 0x7fce300d2ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T15:11:34.643 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:34.642+0000 7fce36542640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7fce300b3050 0x7fce300b3f60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T15:11:34.643 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:34.642+0000 7fce36542640 1 -- --> v1:172.21.15.43:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fce300ce090 con 0x7fce300b2790 2023-08-09T15:11:34.643 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:34.642+0000 7fce36542640 1 -- --> v2:172.21.15.43:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fce300b3950 con 0x7fce300b3050 2023-08-09T15:11:34.643 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:34.642+0000 7fce36542640 1 -- --> v2:172.21.15.203:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fce300cd7f0 con 0x7fce300b2b70 2023-08-09T15:11:34.644 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:34.643+0000 7fce2ffff640 1 --1- >> v1:172.21.15.43:6789/0 conn(0x7fce300b2790 0x7fce300b5330 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.43:6789/0 says I am v1:172.21.15.43:42930/0 (socket says 172.21.15.43:42930) 2023-08-09T15:11:34.644 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:34.643+0000 7fce2ffff640 1 -- 172.21.15.43:0/1925376430 learned_addr learned my addr 172.21.15.43:0/1925376430 (peer_addr_for_me v1:172.21.15.43:0/0) 2023-08-09T15:11:34.644 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:34.643+0000 7fce34adf640 1 --2- 172.21.15.43:0/1925376430 >> v2:172.21.15.43:3300/0 conn(0x7fce300b3050 0x7fce300b3f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T15:11:34.644 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:34.643+0000 7fce2f7fe640 1 --2- 172.21.15.43:0/1925376430 >> v2:172.21.15.203:3300/0 conn(0x7fce300b2b70 0x7fce300d2ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T15:11:34.645 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:34.643+0000 7fce2effd640 1 -- 172.21.15.43:0/1925376430 <== mon.0 v1:172.21.15.43:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2299353889 0 0) 0x7fce300ce090 con 0x7fce300b2790 2023-08-09T15:11:34.645 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:34.643+0000 7fce2effd640 1 -- 172.21.15.43:0/1925376430 --> v1:172.21.15.43:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fce18003520 con 0x7fce300b2790 2023-08-09T15:11:34.645 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:34.643+0000 7fce34adf640 1 -- 172.21.15.43:0/1925376430 >> v2:172.21.15.203:3300/0 conn(0x7fce300b2b70 msgr2=0x7fce300d2ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-09T15:11:34.645 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:34.643+0000 7fce34adf640 1 --2- 172.21.15.43:0/1925376430 >> v2:172.21.15.203:3300/0 conn(0x7fce300b2b70 0x7fce300d2ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T15:11:34.645 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:34.643+0000 7fce34adf640 1 -- 172.21.15.43:0/1925376430 >> v1:172.21.15.43:6789/0 conn(0x7fce300b2790 legacy=0x7fce300b5330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T15:11:34.645 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:34.643+0000 7fce34adf640 1 -- 172.21.15.43:0/1925376430 --> v2:172.21.15.43:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fce300cdad0 con 0x7fce300b3050 2023-08-09T15:11:34.646 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:34.644+0000 7fce34adf640 1 --2- 172.21.15.43:0/1925376430 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fce300b3050 0x7fce300b3f60 secure :-1 s=READY pgs=2417 cs=0 l=1 rev1=1 crypto rx=0x7fce30066af0 tx=0x7fce2802b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=71f5e401c4c2abb8 server_cookie=0 in_seq=0 out_seq=0 2023-08-09T15:11:34.646 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:34.645+0000 7fce2effd640 1 -- 172.21.15.43:0/1925376430 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fce28031830 con 0x7fce300b3050 2023-08-09T15:11:34.646 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:34.645+0000 7fce2effd640 1 -- 172.21.15.43:0/1925376430 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fce28031990 con 0x7fce300b3050 2023-08-09T15:11:34.647 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:34.645+0000 7fce2effd640 1 -- 172.21.15.43:0/1925376430 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fce28031d00 con 0x7fce300b3050 2023-08-09T15:11:34.647 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:34.645+0000 7fce36542640 1 -- 172.21.15.43:0/1925376430 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fce300b3050 msgr2=0x7fce300b3f60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T15:11:34.647 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:34.645+0000 7fce36542640 1 --2- 172.21.15.43:0/1925376430 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fce300b3050 0x7fce300b3f60 secure :-1 s=READY pgs=2417 cs=0 l=1 rev1=1 crypto rx=0x7fce30066af0 tx=0x7fce2802b8d0 comp rx=0 tx=0).stop 2023-08-09T15:11:34.647 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:34.645+0000 7fce36542640 1 -- 172.21.15.43:0/1925376430 shutdown_connections 2023-08-09T15:11:34.647 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:34.645+0000 7fce36542640 1 --2- 172.21.15.43:0/1925376430 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fce300b3050 0x7fce300b3f60 unknown :-1 s=CLOSED pgs=2417 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T15:11:34.647 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:34.645+0000 7fce36542640 1 --2- 172.21.15.43:0/1925376430 >> v2:172.21.15.203:3300/0 conn(0x7fce300b2b70 0x7fce300d2ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T15:11:34.647 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:34.645+0000 7fce36542640 1 -- 172.21.15.43:0/1925376430 >> 172.21.15.43:0/1925376430 conn(0x7fce3009b7b0 msgr2=0x7fce300d24e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-09T15:11:34.648 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:34.646+0000 7fce36542640 1 -- 172.21.15.43:0/1925376430 shutdown_connections 2023-08-09T15:11:34.648 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:34.646+0000 7fce36542640 1 -- 172.21.15.43:0/1925376430 wait complete. 2023-08-09T15:11:34.648 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:34.647+0000 7fce36542640 1 Processor -- start 2023-08-09T15:11:34.648 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:34.647+0000 7fce36542640 1 -- start start 2023-08-09T15:11:34.649 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:34.648+0000 7fce36542640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fce300b2790 0x7fce300b06f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T15:11:34.649 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:34.648+0000 7fce36542640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fce300b2b70 0x7fce30153760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T15:11:34.650 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:34.648+0000 7fce36542640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fce300b3050 0x7fce300afc20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T15:11:34.650 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:34.648+0000 7fce36542640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fce300af790 con 0x7fce300b3050 2023-08-09T15:11:34.650 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:34.648+0000 7fce36542640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fce300b0c30 con 0x7fce300b2790 2023-08-09T15:11:34.650 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:34.648+0000 7fce36542640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fce300b0ed0 con 0x7fce300b2b70 2023-08-09T15:11:34.650 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:34.649+0000 7fce34adf640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fce300b3050 0x7fce300afc20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T15:11:34.650 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:34.649+0000 7fce34adf640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fce300b3050 0x7fce300afc20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:56194/0 (socket says 172.21.15.43:56194) 2023-08-09T15:11:34.650 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:34.649+0000 7fce34adf640 1 -- 172.21.15.43:0/2632089530 learned_addr learned my addr 172.21.15.43:0/2632089530 (peer_addr_for_me v2:172.21.15.43:0/0) 2023-08-09T15:11:34.651 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:34.649+0000 7fce2f7fe640 1 --2- 172.21.15.43:0/2632089530 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fce300b2b70 0x7fce30153760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T15:11:34.651 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:34.649+0000 7fce2ffff640 1 --2- 172.21.15.43:0/2632089530 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fce300b2790 0x7fce300b06f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T15:11:34.651 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:34.649+0000 7fce34adf640 1 -- 172.21.15.43:0/2632089530 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fce300b2b70 msgr2=0x7fce30153760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T15:11:34.651 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:34.649+0000 7fce34adf640 1 --2- 172.21.15.43:0/2632089530 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fce300b2b70 0x7fce30153760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T15:11:34.651 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:34.649+0000 7fce34adf640 1 -- 172.21.15.43:0/2632089530 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fce300b2790 msgr2=0x7fce300b06f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T15:11:34.651 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:34.649+0000 7fce34adf640 1 --2- 172.21.15.43:0/2632089530 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fce300b2790 0x7fce300b06f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T15:11:34.651 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:34.649+0000 7fce34adf640 1 -- 172.21.15.43:0/2632089530 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fce28009400 con 0x7fce300b3050 2023-08-09T15:11:34.652 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:34.649+0000 7fce2f7fe640 1 --2- 172.21.15.43:0/2632089530 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fce300b2b70 0x7fce30153760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-08-09T15:11:34.652 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:34.650+0000 7fce34adf640 1 --2- 172.21.15.43:0/2632089530 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fce300b3050 0x7fce300afc20 secure :-1 s=READY pgs=2418 cs=0 l=1 rev1=1 crypto rx=0x7fce28034040 tx=0x7fce280046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-09T15:11:34.653 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:34.650+0000 7fce2cff9640 1 -- 172.21.15.43:0/2632089530 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fce2803a830 con 0x7fce300b3050 2023-08-09T15:11:34.653 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:34.650+0000 7fce2cff9640 1 -- 172.21.15.43:0/2632089530 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fce2803a990 con 0x7fce300b3050 2023-08-09T15:11:34.653 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:34.650+0000 7fce2cff9640 1 -- 172.21.15.43:0/2632089530 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fce2803ad00 con 0x7fce300b3050 2023-08-09T15:11:34.653 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:34.650+0000 7fce36542640 1 -- 172.21.15.43:0/2632089530 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fce300af140 con 0x7fce300b3050 2023-08-09T15:11:34.653 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:34.650+0000 7fce36542640 1 -- 172.21.15.43:0/2632089530 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fce300b11b0 con 0x7fce300b3050 2023-08-09T15:11:34.653 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:34.651+0000 7fce36542640 1 -- 172.21.15.43:0/2632089530 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fce300af900 con 0x7fce300b3050 2023-08-09T15:11:34.653 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:34.652+0000 7fce2cff9640 1 -- 172.21.15.43:0/2632089530 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 118) v1 ==== 88443+0+0 (secure 0 0 0) 0x7fce280443f0 con 0x7fce300b3050 2023-08-09T15:11:34.654 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:34.653+0000 7fce2cff9640 1 --2- 172.21.15.43:0/2632089530 >> [v2:172.21.15.203:6832/4034093268,v1:172.21.15.203:6833/4034093268] conn(0x7fce1c06a1a0 0x7fce1c06c660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T15:11:34.654 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:34.653+0000 7fce2cff9640 1 -- 172.21.15.43:0/2632089530 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(223..223 src has 1..223) v4 ==== 6426+0+0 (secure 0 0 0) 0x7fce280aabc0 con 0x7fce300b3050 2023-08-09T15:11:34.654 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:34.654+0000 7fce2ffff640 1 --2- 172.21.15.43:0/2632089530 >> [v2:172.21.15.203:6832/4034093268,v1:172.21.15.203:6833/4034093268] conn(0x7fce1c06a1a0 0x7fce1c06c660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T15:11:34.655 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:34.655+0000 7fce2ffff640 1 --2- 172.21.15.43:0/2632089530 >> [v2:172.21.15.203:6832/4034093268,v1:172.21.15.203:6833/4034093268] conn(0x7fce1c06a1a0 0x7fce1c06c660 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7fce200012e0 tx=0x7fce20009040 comp rx=0 tx=0).ready entity=mgr.8833 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-09T15:11:34.658 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:34.657+0000 7fce2cff9640 1 -- 172.21.15.43:0/2632089530 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178066 (secure 0 0 0) 0x7fce28075fa0 con 0x7fce300b3050 2023-08-09T15:11:34.884 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:34.884+0000 7fce36542640 1 -- 172.21.15.43:0/2632089530 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fce300739f0 con 0x7fce300b3050 2023-08-09T15:11:34.885 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:34.885+0000 7fce2cff9640 1 -- 172.21.15.43:0/2632089530 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 104 v104) v1 ==== 95+0+4894 (secure 0 0 0) 0x7fce28035090 con 0x7fce300b3050 2023-08-09T15:11:34.886 INFO:teuthology.orchestra.run.smithi043.stdout: 2023-08-09T15:11:34.886 INFO:teuthology.orchestra.run.smithi043.stdout:{"epoch":104,"default_fscid":12,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":8012,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.203:6837/2464832767","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6836","nonce":2464832767},{"type":"v1","addr":"172.21.15.203:6837","nonce":2464832767}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":100},{"gid":9010,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.43:6835/3989386049","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6834","nonce":3989386049},{"type":"v1","addr":"172.21.15.43:6835","nonce":3989386049}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":100},{"gid":9016,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.203:6835/2213455074","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6834","nonce":2213455074},{"type":"v1","addr":"172.21.15.203:6835","nonce":2213455074}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":100}],"filesystems":[{"mdsmap":{"epoch":104,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-08-09T15:11:28.168916+0000","modified":"2023-08-09T15:11:31.739094+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":11325},"failed":[],"damaged":[],"stopped":[],"info":{"gid_11325":{"gid":11325,"name":"a","rank":0,"incarnation":101,"state":"up:active","state_seq":3,"addr":"172.21.15.43:6833/1425989175","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6832","nonce":1425989175},{"type":"v1","addr":"172.21.15.43:6833","nonce":1425989175}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[27],"metadata_pool":26,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":12}]} 2023-08-09T15:11:34.886 INFO:teuthology.orchestra.run.smithi043.stderr:dumped fsmap epoch 104 2023-08-09T15:11:34.888 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:34.887+0000 7fce36542640 1 -- 172.21.15.43:0/2632089530 >> [v2:172.21.15.203:6832/4034093268,v1:172.21.15.203:6833/4034093268] conn(0x7fce1c06a1a0 msgr2=0x7fce1c06c660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T15:11:34.888 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:34.887+0000 7fce36542640 1 --2- 172.21.15.43:0/2632089530 >> [v2:172.21.15.203:6832/4034093268,v1:172.21.15.203:6833/4034093268] conn(0x7fce1c06a1a0 0x7fce1c06c660 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7fce200012e0 tx=0x7fce20009040 comp rx=0 tx=0).stop 2023-08-09T15:11:34.888 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:34.888+0000 7fce36542640 1 -- 172.21.15.43:0/2632089530 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fce300b3050 msgr2=0x7fce300afc20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T15:11:34.889 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:34.888+0000 7fce36542640 1 --2- 172.21.15.43:0/2632089530 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fce300b3050 0x7fce300afc20 secure :-1 s=READY pgs=2418 cs=0 l=1 rev1=1 crypto rx=0x7fce28034040 tx=0x7fce280046c0 comp rx=0 tx=0).stop 2023-08-09T15:11:34.890 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:34.888+0000 7fce36542640 1 -- 172.21.15.43:0/2632089530 shutdown_connections 2023-08-09T15:11:34.890 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:34.888+0000 7fce36542640 1 --2- 172.21.15.43:0/2632089530 >> [v2:172.21.15.203:6832/4034093268,v1:172.21.15.203:6833/4034093268] conn(0x7fce1c06a1a0 0x7fce1c06c660 unknown :-1 s=CLOSED pgs=67 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T15:11:34.890 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:34.888+0000 7fce36542640 1 --2- 172.21.15.43:0/2632089530 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fce300b3050 0x7fce300afc20 unknown :-1 s=CLOSED pgs=2418 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T15:11:34.891 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:34.888+0000 7fce36542640 1 --2- 172.21.15.43:0/2632089530 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fce300b2b70 0x7fce30153760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T15:11:34.891 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:34.888+0000 7fce36542640 1 --2- 172.21.15.43:0/2632089530 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fce300b2790 0x7fce300b06f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T15:11:34.891 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:34.888+0000 7fce36542640 1 -- 172.21.15.43:0/2632089530 >> 172.21.15.43:0/2632089530 conn(0x7fce3009b7b0 msgr2=0x7fce3015a4d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-09T15:11:34.892 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:34.888+0000 7fce36542640 1 -- 172.21.15.43:0/2632089530 shutdown_connections 2023-08-09T15:11:34.892 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:34.888+0000 7fce36542640 1 -- 172.21.15.43:0/2632089530 wait complete. 2023-08-09T15:11:35.311 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 104, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2023-08-09T15:11:28.168916+0000', 'modified': '2023-08-09T15:11:31.739094+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 11325}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_11325': {'gid': 11325, 'name': 'a', 'rank': 0, 'incarnation': 101, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.43:6833/1425989175', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.43:6832', 'nonce': 1425989175}, {'type': 'v1', 'addr': '172.21.15.43:6833', 'nonce': 1425989175}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138322906710015, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}}, 'data_pools': [27], 'metadata_pool': 26, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0} 2023-08-09T15:11:35.311 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2023-08-09T15:11:35.312 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.a status 2023-08-09T15:11:35.551 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:35.551+0000 7efcb1ecd640 1 Processor -- start 2023-08-09T15:11:35.552 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:35.551+0000 7efcb1ecd640 1 -- start start 2023-08-09T15:11:35.553 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:35.552+0000 7efcb1ecd640 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7efcac15f7b0 0x7efcac0b4760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T15:11:35.553 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:35.552+0000 7efcb1ecd640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7efcac0b1050 0x7efcac0ccfa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T15:11:35.554 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:35.553+0000 7efcb1ecd640 1 -- --> v1:172.21.15.203:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7efcac0cd930 con 0x7efcac0cd4e0 2023-08-09T15:11:35.554 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:35.553+0000 7efcb1ecd640 1 -- --> v2:172.21.15.203:3300/0 -- mon_getmap magic: 0 v1 -- 0x7efcac0b7bf0 con 0x7efcac15f7b0 2023-08-09T15:11:35.554 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:35.553+0000 7efcb1ecd640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7efcac0b7ed0 con 0x7efcac0b1050 2023-08-09T15:11:35.554 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:35.553+0000 7efcb0ecb640 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7efcac15f7b0 0x7efcac0b4760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T15:11:35.555 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:35.553+0000 7efca3fff640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7efcac0b1050 0x7efcac0ccfa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T15:11:35.555 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:35.554+0000 7efcb0ecb640 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7efcac15f7b0 0x7efcac0b4760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.203:3300/0 says I am v2:172.21.15.43:36342/0 (socket says 172.21.15.43:36342) 2023-08-09T15:11:35.556 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:35.554+0000 7efca3fff640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7efcac0b1050 0x7efcac0ccfa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.203:3301/0 says I am v2:172.21.15.43:59200/0 (socket says 172.21.15.43:59200) 2023-08-09T15:11:35.556 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:35.554+0000 7efcb0ecb640 1 -- 172.21.15.43:0/2993314254 learned_addr learned my addr 172.21.15.43:0/2993314254 (peer_addr_for_me v2:172.21.15.43:0/0) 2023-08-09T15:11:35.556 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:35.554+0000 7efcb16cc640 1 --1- 172.21.15.43:0/2993314254 >> v1:172.21.15.203:6789/0 conn(0x7efcac0cd4e0 0x7efcac0ceae0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.203:6789/0 says I am v1:172.21.15.43:45378/0 (socket says 172.21.15.43:45378) 2023-08-09T15:11:35.557 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:35.554+0000 7efca37fe640 1 -- 172.21.15.43:0/2993314254 <== mon.1 v1:172.21.15.203:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3189295756 0 0) 0x7efca4001490 con 0x7efcac0cd4e0 2023-08-09T15:11:35.557 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:35.554+0000 7efca37fe640 1 -- 172.21.15.43:0/2993314254 --> v1:172.21.15.203:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7efc94003520 con 0x7efcac0cd4e0 2023-08-09T15:11:35.557 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:35.554+0000 7efca3fff640 1 -- 172.21.15.43:0/2993314254 >> v2:172.21.15.203:3300/0 conn(0x7efcac15f7b0 msgr2=0x7efcac0b4760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-09T15:11:35.558 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:35.554+0000 7efca3fff640 1 --2- 172.21.15.43:0/2993314254 >> v2:172.21.15.203:3300/0 conn(0x7efcac15f7b0 0x7efcac0b4760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T15:11:35.558 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:35.554+0000 7efca3fff640 1 -- 172.21.15.43:0/2993314254 >> v1:172.21.15.203:6789/0 conn(0x7efcac0cd4e0 legacy=0x7efcac0ceae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T15:11:35.558 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:35.554+0000 7efca3fff640 1 -- 172.21.15.43:0/2993314254 --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efcac0d0370 con 0x7efcac0b1050 2023-08-09T15:11:35.559 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:35.556+0000 7efca3fff640 1 --2- 172.21.15.43:0/2993314254 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7efcac0b1050 0x7efcac0ccfa0 secure :-1 s=READY pgs=338 cs=0 l=1 rev1=1 crypto rx=0x7efc9c0097a0 tx=0x7efc9c02b8d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=fa7f11ca6ed9971e server_cookie=0 in_seq=0 out_seq=0 2023-08-09T15:11:35.559 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:35.556+0000 7efca37fe640 1 -- 172.21.15.43:0/2993314254 <== mon.2 v2:172.21.15.203:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efc9c031830 con 0x7efcac0b1050 2023-08-09T15:11:35.559 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:35.556+0000 7efca37fe640 1 -- 172.21.15.43:0/2993314254 <== mon.2 v2:172.21.15.203:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efc9c031990 con 0x7efcac0b1050 2023-08-09T15:11:35.559 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:35.556+0000 7efca37fe640 1 -- 172.21.15.43:0/2993314254 <== mon.2 v2:172.21.15.203:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efc9c031d20 con 0x7efcac0b1050 2023-08-09T15:11:35.559 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:35.557+0000 7efcb1ecd640 1 -- 172.21.15.43:0/2993314254 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7efcac0b1050 msgr2=0x7efcac0ccfa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T15:11:35.559 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:35.557+0000 7efcb1ecd640 1 --2- 172.21.15.43:0/2993314254 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7efcac0b1050 0x7efcac0ccfa0 secure :-1 s=READY pgs=338 cs=0 l=1 rev1=1 crypto rx=0x7efc9c0097a0 tx=0x7efc9c02b8d0 comp rx=0 tx=0).stop 2023-08-09T15:11:35.560 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:35.557+0000 7efcb1ecd640 1 -- 172.21.15.43:0/2993314254 shutdown_connections 2023-08-09T15:11:35.561 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:35.557+0000 7efcb1ecd640 1 --2- 172.21.15.43:0/2993314254 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7efcac0b1050 0x7efcac0ccfa0 unknown :-1 s=CLOSED pgs=338 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T15:11:35.561 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:35.557+0000 7efcb1ecd640 1 --2- 172.21.15.43:0/2993314254 >> v2:172.21.15.203:3300/0 conn(0x7efcac15f7b0 0x7efcac0b4760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T15:11:35.561 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:35.557+0000 7efcb1ecd640 1 -- 172.21.15.43:0/2993314254 >> 172.21.15.43:0/2993314254 conn(0x7efcac06a2b0 msgr2=0x7efcac06a690 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-09T15:11:35.561 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:35.557+0000 7efcb1ecd640 1 -- 172.21.15.43:0/2993314254 shutdown_connections 2023-08-09T15:11:35.561 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:35.557+0000 7efcb1ecd640 1 -- 172.21.15.43:0/2993314254 wait complete. 2023-08-09T15:11:35.561 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:35.559+0000 7efcb1ecd640 1 Processor -- start 2023-08-09T15:11:35.562 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:35.559+0000 7efcb1ecd640 1 -- start start 2023-08-09T15:11:35.562 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:35.560+0000 7efcb1ecd640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7efcac0b1050 0x7efcac14fb60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T15:11:35.562 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:35.560+0000 7efcb1ecd640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7efcac0cd4e0 0x7efcac0d1da0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T15:11:35.562 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:35.560+0000 7efcb1ecd640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7efcac15f7b0 0x7efcac0d22e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T15:11:35.562 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:35.560+0000 7efcb1ecd640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efcac0c60e0 con 0x7efcac0cd4e0 2023-08-09T15:11:35.562 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:35.560+0000 7efcb1ecd640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efcac0b27e0 con 0x7efcac0b1050 2023-08-09T15:11:35.562 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:35.560+0000 7efcb1ecd640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7efcac150b30 con 0x7efcac15f7b0 2023-08-09T15:11:35.562 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:35.560+0000 7efca3fff640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7efcac0cd4e0 0x7efcac0d1da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T15:11:35.563 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:35.560+0000 7efca3fff640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7efcac0cd4e0 0x7efcac0d1da0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:43534/0 (socket says 172.21.15.43:43534) 2023-08-09T15:11:35.563 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:35.560+0000 7efca3fff640 1 -- 172.21.15.43:0/3798044542 learned_addr learned my addr 172.21.15.43:0/3798044542 (peer_addr_for_me v2:172.21.15.43:0/0) 2023-08-09T15:11:35.564 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:35.561+0000 7efcb0ecb640 1 --2- 172.21.15.43:0/3798044542 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7efcac0b1050 0x7efcac14fb60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T15:11:35.564 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:35.561+0000 7efca3fff640 1 -- 172.21.15.43:0/3798044542 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7efcac15f7b0 msgr2=0x7efcac0d22e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T15:11:35.564 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:35.561+0000 7efca3fff640 1 --2- 172.21.15.43:0/3798044542 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7efcac15f7b0 0x7efcac0d22e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T15:11:35.564 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:35.561+0000 7efca3fff640 1 -- 172.21.15.43:0/3798044542 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7efcac0b1050 msgr2=0x7efcac14fb60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T15:11:35.564 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:35.561+0000 7efca3fff640 1 --2- 172.21.15.43:0/3798044542 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7efcac0b1050 0x7efcac14fb60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T15:11:35.564 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:35.561+0000 7efca3fff640 1 -- 172.21.15.43:0/3798044542 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efc9c009400 con 0x7efcac0cd4e0 2023-08-09T15:11:35.565 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:35.561+0000 7efca3fff640 1 --2- 172.21.15.43:0/3798044542 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7efcac0cd4e0 0x7efcac0d1da0 secure :-1 s=READY pgs=2419 cs=0 l=1 rev1=1 crypto rx=0x7efc9c02bee0 tx=0x7efc9c0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-09T15:11:35.565 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:35.561+0000 7efca17fa640 1 -- 172.21.15.43:0/3798044542 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efc9c03a630 con 0x7efcac0cd4e0 2023-08-09T15:11:35.565 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:35.562+0000 7efca17fa640 1 -- 172.21.15.43:0/3798044542 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efc9c03a7b0 con 0x7efcac0cd4e0 2023-08-09T15:11:35.565 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:35.562+0000 7efca17fa640 1 -- 172.21.15.43:0/3798044542 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efc9c03ab60 con 0x7efcac0cd4e0 2023-08-09T15:11:35.566 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:35.562+0000 7efcb1ecd640 1 -- 172.21.15.43:0/3798044542 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7efcac150e10 con 0x7efcac0cd4e0 2023-08-09T15:11:35.566 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:35.562+0000 7efcb1ecd640 1 -- 172.21.15.43:0/3798044542 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7efcac0b2ac0 con 0x7efcac0cd4e0 2023-08-09T15:11:35.567 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:35.564+0000 7efca17fa640 1 -- 172.21.15.43:0/3798044542 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 118) v1 ==== 88443+0+0 (secure 0 0 0) 0x7efc9c03acc0 con 0x7efcac0cd4e0 2023-08-09T15:11:35.567 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:35.565+0000 7efca17fa640 1 --2- 172.21.15.43:0/3798044542 >> [v2:172.21.15.203:6832/4034093268,v1:172.21.15.203:6833/4034093268] conn(0x7efc9806a170 0x7efc9806c630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T15:11:35.567 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:35.565+0000 7efca17fa640 1 -- 172.21.15.43:0/3798044542 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(223..223 src has 1..223) v4 ==== 6426+0+0 (secure 0 0 0) 0x7efc9c0aa310 con 0x7efcac0cd4e0 2023-08-09T15:11:35.567 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:35.566+0000 7efcb0ecb640 1 --2- 172.21.15.43:0/3798044542 >> [v2:172.21.15.203:6832/4034093268,v1:172.21.15.203:6833/4034093268] conn(0x7efc9806a170 0x7efc9806c630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T15:11:35.567 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:35.566+0000 7efcb5d78740 1 Processor -- start 2023-08-09T15:11:35.567 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:35.566+0000 7efcb5d78740 1 -- start start 2023-08-09T15:11:35.569 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:35.567+0000 7efcb0ecb640 1 --2- 172.21.15.43:0/3798044542 >> [v2:172.21.15.203:6832/4034093268,v1:172.21.15.203:6833/4034093268] conn(0x7efc9806a170 0x7efc9806c630 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7efca8009a00 tx=0x7efca8008040 comp rx=0 tx=0).ready entity=mgr.8833 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-09T15:11:35.569 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:35.567+0000 7efcb5d78740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x558bafcaee30 0x558bafcaf210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T15:11:35.569 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:35.567+0000 7efcb5d78740 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x558bafc7edb0 0x558bafcac0a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T15:11:35.569 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:35.567+0000 7efcb5d78740 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x558bafc829b0 0x558bafcac5e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T15:11:35.570 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:35.567+0000 7efcb5d78740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x558bafbf08f0 con 0x558bafcaee30 2023-08-09T15:11:35.570 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:35.567+0000 7efcb5d78740 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x558bafbedfe0 con 0x558bafc829b0 2023-08-09T15:11:35.570 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:35.567+0000 7efcb5d78740 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x558bafbe98d0 con 0x558bafc7edb0 2023-08-09T15:11:35.570 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:35.568+0000 7efcb0ecb640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x558bafcaee30 0x558bafcaf210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T15:11:35.571 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:35.568+0000 7efcb0ecb640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x558bafcaee30 0x558bafcaf210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:43542/0 (socket says 172.21.15.43:43542) 2023-08-09T15:11:35.571 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:35.568+0000 7efcb0ecb640 1 -- 172.21.15.43:0/3068374532 learned_addr learned my addr 172.21.15.43:0/3068374532 (peer_addr_for_me v2:172.21.15.43:0/0) 2023-08-09T15:11:35.571 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:35.568+0000 7efca3fff640 1 --2- 172.21.15.43:0/3068374532 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x558bafc7edb0 0x558bafcac0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T15:11:35.571 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:35.568+0000 7efcb16cc640 1 --2- 172.21.15.43:0/3068374532 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x558bafc829b0 0x558bafcac5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T15:11:35.571 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:35.568+0000 7efcb0ecb640 1 -- 172.21.15.43:0/3068374532 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x558bafc7edb0 msgr2=0x558bafcac0a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-09T15:11:35.571 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:35.568+0000 7efcb0ecb640 1 --2- 172.21.15.43:0/3068374532 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x558bafc7edb0 0x558bafcac0a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T15:11:35.571 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:35.568+0000 7efcb0ecb640 1 -- 172.21.15.43:0/3068374532 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x558bafc829b0 msgr2=0x558bafcac5e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-09T15:11:35.572 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:35.568+0000 7efcb0ecb640 1 --2- 172.21.15.43:0/3068374532 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x558bafc829b0 0x558bafcac5e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T15:11:35.572 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:35.568+0000 7efcb0ecb640 1 -- 172.21.15.43:0/3068374532 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x558bafba2ac0 con 0x558bafcaee30 2023-08-09T15:11:35.572 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:35.568+0000 7efcb16cc640 1 --2- 172.21.15.43:0/3068374532 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x558bafc829b0 0x558bafcac5e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-08-09T15:11:35.573 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:35.568+0000 7efcb0ecb640 1 --2- 172.21.15.43:0/3068374532 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x558bafcaee30 0x558bafcaf210 secure :-1 s=READY pgs=2420 cs=0 l=1 rev1=1 crypto rx=0x7efca8010a20 tx=0x7efca8013000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=86a5e477cfdf86b3 server_cookie=0 in_seq=0 out_seq=0 2023-08-09T15:11:35.578 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:35.569+0000 7efc92ffd640 1 -- 172.21.15.43:0/3068374532 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efca8021440 con 0x558bafcaee30 2023-08-09T15:11:35.578 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:35.569+0000 7efc92ffd640 1 -- 172.21.15.43:0/3068374532 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efca801a070 con 0x558bafcaee30 2023-08-09T15:11:35.579 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:35.569+0000 7efc92ffd640 1 -- 172.21.15.43:0/3068374532 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efca80217e0 con 0x558bafcaee30 2023-08-09T15:11:35.579 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:35.570+0000 7efcb5d78740 1 -- 172.21.15.43:0/3068374532 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x558bafcaee30 msgr2=0x558bafcaf210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T15:11:35.579 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:35.570+0000 7efcb5d78740 1 --2- 172.21.15.43:0/3068374532 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x558bafcaee30 0x558bafcaf210 secure :-1 s=READY pgs=2420 cs=0 l=1 rev1=1 crypto rx=0x7efca8010a20 tx=0x7efca8013000 comp rx=0 tx=0).stop 2023-08-09T15:11:35.579 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:35.570+0000 7efcb5d78740 1 -- 172.21.15.43:0/3068374532 shutdown_connections 2023-08-09T15:11:35.579 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:35.570+0000 7efcb5d78740 1 --2- 172.21.15.43:0/3068374532 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x558bafc829b0 0x558bafcac5e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T15:11:35.579 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:35.570+0000 7efcb5d78740 1 --2- 172.21.15.43:0/3068374532 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x558bafc7edb0 0x558bafcac0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T15:11:35.579 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:35.570+0000 7efcb5d78740 1 --2- 172.21.15.43:0/3068374532 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x558bafcaee30 0x558bafcaf210 unknown :-1 s=CLOSED pgs=2420 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T15:11:35.580 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:35.570+0000 7efcb5d78740 1 -- 172.21.15.43:0/3068374532 >> 172.21.15.43:0/3068374532 conn(0x558bafc810c0 msgr2=0x558bafcc9c40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-09T15:11:35.580 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:35.570+0000 7efcb5d78740 1 -- 172.21.15.43:0/3068374532 shutdown_connections 2023-08-09T15:11:35.580 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:35.570+0000 7efcb5d78740 1 -- 172.21.15.43:0/3068374532 wait complete. 2023-08-09T15:11:35.580 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:35.571+0000 7efcb5d78740 1 -- start start 2023-08-09T15:11:35.580 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:35.571+0000 7efcb5d78740 1 Processor -- start 2023-08-09T15:11:35.580 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:35.572+0000 7efcb5d78740 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x558bafc829b0 0x558bafc95e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T15:11:35.580 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:35.572+0000 7efcb5d78740 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x558bafcaee30 0x558bafcaf400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T15:11:35.581 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:35.572+0000 7efcb5d78740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x558bafcaf940 0x558bafc98a50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T15:11:35.581 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:35.572+0000 7efcb5d78740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x558bafb89d40 con 0x558bafcaf940 2023-08-09T15:11:35.581 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:35.572+0000 7efcb5d78740 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x558bafb899e0 con 0x558bafc829b0 2023-08-09T15:11:35.581 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:35.572+0000 7efcb5d78740 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x558baf94ff20 con 0x558bafcaee30 2023-08-09T15:11:35.581 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:35.572+0000 7efcb5d78740 10 client.0 _ll_register_callbacks cb 0x558bafba2160 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-08-09T15:11:35.581 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:35.572+0000 7efcb16cc640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x558bafcaf940 0x558bafc98a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T15:11:35.581 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:35.572+0000 7efcb16cc640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x558bafcaf940 0x558bafc98a50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:43550/0 (socket says 172.21.15.43:43550) 2023-08-09T15:11:35.582 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:35.572+0000 7efcb16cc640 1 -- 172.21.15.43:0/3436142052 learned_addr learned my addr 172.21.15.43:0/3436142052 (peer_addr_for_me v2:172.21.15.43:0/0) 2023-08-09T15:11:35.582 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:35.573+0000 7efca3fff640 1 --2- 172.21.15.43:0/3436142052 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x558bafcaee30 0x558bafcaf400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T15:11:35.582 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:35.573+0000 7efcb0ecb640 1 --2- 172.21.15.43:0/3436142052 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x558bafc829b0 0x558bafc95e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T15:11:35.582 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:35.573+0000 7efcb16cc640 1 -- 172.21.15.43:0/3436142052 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x558bafcaee30 msgr2=0x558bafcaf400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-09T15:11:35.583 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:35.573+0000 7efcb16cc640 1 --2- 172.21.15.43:0/3436142052 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x558bafcaee30 0x558bafcaf400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T15:11:35.583 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:35.573+0000 7efcb16cc640 1 -- 172.21.15.43:0/3436142052 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x558bafc829b0 msgr2=0x558bafc95e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-09T15:11:35.583 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:35.573+0000 7efcb16cc640 1 --2- 172.21.15.43:0/3436142052 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x558bafc829b0 0x558bafc95e60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T15:11:35.583 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:35.573+0000 7efcb16cc640 1 -- 172.21.15.43:0/3436142052 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7efca800e050 con 0x558bafcaf940 2023-08-09T15:11:35.583 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:35.573+0000 7efca3fff640 1 --2- 172.21.15.43:0/3436142052 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x558bafcaee30 0x558bafcaf400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-08-09T15:11:35.583 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:35.573+0000 7efcb0ecb640 1 --2- 172.21.15.43:0/3436142052 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x558bafc829b0 0x558bafc95e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-08-09T15:11:35.584 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:35.573+0000 7efcb16cc640 1 --2- 172.21.15.43:0/3436142052 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x558bafcaf940 0x558bafc98a50 secure :-1 s=READY pgs=2421 cs=0 l=1 rev1=1 crypto rx=0x7efca400d460 tx=0x7efca400d920 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ce753fbaad843fac server_cookie=0 in_seq=0 out_seq=0 2023-08-09T15:11:35.584 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:35.573+0000 7efc8b7fe640 10 client.0 ms_handle_connect on v2:172.21.15.43:3300/0 2023-08-09T15:11:35.584 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:35.573+0000 7efc8b7fe640 1 -- 172.21.15.43:0/3436142052 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efca400b070 con 0x558bafcaf940 2023-08-09T15:11:35.584 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:35.573+0000 7efcb5d78740 1 -- 172.21.15.43:0/3436142052 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x558bafbc1500 con 0x558bafcaf940 2023-08-09T15:11:35.585 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:35.573+0000 7efc8b7fe640 1 -- 172.21.15.43:0/3436142052 <== mon.0 v2:172.21.15.43:3300/0 2 ==== osd_map(223..223 src has 1..223) v4 ==== 6426+0+0 (secure 0 0 0) 0x7efca40070d0 con 0x558bafcaf940 2023-08-09T15:11:35.585 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:35.574+0000 7efc8b7fe640 1 -- 172.21.15.43:0/3436142052 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_get_version_reply(handle=1 version=104) v2 ==== 24+0+0 (secure 0 0 0) 0x7efca4007290 con 0x558bafcaf940 2023-08-09T15:11:35.585 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:35.574+0000 7efcb5d78740 10 client.11490 fetch_fsmap learned FSMap version 104 2023-08-09T15:11:35.585 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:35.574+0000 7efcb5d78740 1 -- 172.21.15.43:0/3436142052 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,fsmap=104,monmap=2+}) v3 -- 0x558bafbe7f50 con 0x558bafcaf940 2023-08-09T15:11:35.585 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:35.574+0000 7efc8b7fe640 1 -- 172.21.15.43:0/3436142052 <== mon.0 v2:172.21.15.43:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x558bafbc1500 con 0x558bafcaf940 2023-08-09T15:11:35.586 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:35.574+0000 7efc8b7fe640 1 -- 172.21.15.43:0/3436142052 <== mon.0 v2:172.21.15.43:3300/0 5 ==== fsmap(e 104) v1 ==== 3294+0+0 (secure 0 0 0) 0x7efca40073f0 con 0x558bafcaf940 2023-08-09T15:11:35.586 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:35.574+0000 7efcb5d78740 10 client.11490 fetch_fsmap finished waiting for FSMap version 104 2023-08-09T15:11:35.586 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:35.574+0000 7efcb5d78740 10 client.11490 resolve_mds: resolved name 'a' to daemon mds.a 2023-08-09T15:11:35.586 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:35.574+0000 7efcb5d78740 20 client.11490 populate_metadata read hostname 'smithi043' 2023-08-09T15:11:35.586 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:35.574+0000 7efcb5d78740 1 --2- 172.21.15.43:0/3436142052 >> [v2:172.21.15.43:6832/1425989175,v1:172.21.15.43:6833/1425989175] conn(0x558bafca8000 0x558bafca83e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T15:11:35.586 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:35.574+0000 7efcb5d78740 4 client.11490 mds_command: new command op to 11325 tid=0[{"prefix": "get_command_descriptions"}] 2023-08-09T15:11:35.586 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:35.574+0000 7efcb5d78740 1 -- 172.21.15.43:0/3436142052 --> [v2:172.21.15.43:6832/1425989175,v1:172.21.15.43:6833/1425989175] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x558bafbbebc0 con 0x558bafca8000 2023-08-09T15:11:35.587 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:35.575+0000 7efcb0ecb640 1 --2- 172.21.15.43:0/3436142052 >> [v2:172.21.15.43:6832/1425989175,v1:172.21.15.43:6833/1425989175] conn(0x558bafca8000 0x558bafca83e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T15:11:35.587 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:35.575+0000 7efcb0ecb640 1 --2- 172.21.15.43:0/3436142052 >> [v2:172.21.15.43:6832/1425989175,v1:172.21.15.43:6833/1425989175] conn(0x558bafca8000 0x558bafca83e0 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=71cbf29c421bf3b6 server_cookie=9f0cf25e19c6d982 in_seq=0 out_seq=0 2023-08-09T15:11:35.587 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:35.575+0000 7efc8b7fe640 10 client.11490 ms_handle_connect on v2:172.21.15.43:6832/1425989175 2023-08-09T15:11:35.587 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:35.576+0000 7efc8b7fe640 1 -- 172.21.15.43:0/3436142052 <== mds.0 v2:172.21.15.43:6832/1425989175 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18017 (crc 0 0 0) 0x558bafbbebc0 con 0x558bafca8000 2023-08-09T15:11:35.587 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:35.576+0000 7efc8b7fe640 10 client.11490 handle_command_reply: tid=0 2023-08-09T15:11:35.587 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:35.576+0000 7efcb5d78740 1 client.11490 shutdown 2023-08-09T15:11:35.587 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:35.576+0000 7efcb5d78740 1 -- 172.21.15.43:0/3436142052 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x558bafcaf940 msgr2=0x558bafc98a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T15:11:35.587 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:35.576+0000 7efcb5d78740 1 --2- 172.21.15.43:0/3436142052 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x558bafcaf940 0x558bafc98a50 secure :-1 s=READY pgs=2421 cs=0 l=1 rev1=1 crypto rx=0x7efca400d460 tx=0x7efca400d920 comp rx=0 tx=0).stop 2023-08-09T15:11:35.588 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:35.576+0000 7efcb5d78740 1 -- 172.21.15.43:0/3436142052 shutdown_connections 2023-08-09T15:11:35.588 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:35.576+0000 7efcb5d78740 1 --2- 172.21.15.43:0/3436142052 >> [v2:172.21.15.43:6832/1425989175,v1:172.21.15.43:6833/1425989175] conn(0x558bafca8000 0x558bafca83e0 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T15:11:35.588 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:35.576+0000 7efcb5d78740 1 --2- 172.21.15.43:0/3436142052 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x558bafcaf940 0x558bafc98a50 unknown :-1 s=CLOSED pgs=2421 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T15:11:35.588 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:35.576+0000 7efcb5d78740 1 --2- 172.21.15.43:0/3436142052 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x558bafcaee30 0x558bafcaf400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T15:11:35.588 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:35.576+0000 7efcb5d78740 1 --2- 172.21.15.43:0/3436142052 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x558bafc829b0 0x558bafc95e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T15:11:35.588 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:35.576+0000 7efc8b7fe640 0 client.11490 ms_handle_reset on v2:172.21.15.43:6832/1425989175 2023-08-09T15:11:35.588 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:35.576+0000 7efcb5d78740 1 -- 172.21.15.43:0/3436142052 >> 172.21.15.43:0/3436142052 conn(0x558bafc7edb0 msgr2=0x558bafc83260 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-09T15:11:35.589 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:35.577+0000 7efcb5d78740 1 -- 172.21.15.43:0/3436142052 shutdown_connections 2023-08-09T15:11:35.589 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:35.577+0000 7efcb5d78740 1 -- 172.21.15.43:0/3436142052 wait complete. 2023-08-09T15:11:35.589 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:35.577+0000 7efcb5d78740 20 client.11490 trim_cache size 0 max 16384 2023-08-09T15:11:35.594 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:35.594+0000 7efcb5d78740 1 Processor -- start 2023-08-09T15:11:35.594 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:35.594+0000 7efcb5d78740 1 -- start start 2023-08-09T15:11:35.595 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:35.595+0000 7efcb5d78740 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x558bafc829b0 0x558bafc7a750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T15:11:35.595 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:35.595+0000 7efcb5d78740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x558bafcaee30 0x558bafc95090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T15:11:35.596 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:35.595+0000 7efcb5d78740 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x558bafcaf940 0x558bafc955d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T15:11:35.596 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:35.595+0000 7efcb5d78740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x558bafca4430 con 0x558bafcaee30 2023-08-09T15:11:35.596 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:35.595+0000 7efcb5d78740 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x558bafce5280 con 0x558bafc829b0 2023-08-09T15:11:35.596 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:35.595+0000 7efcb5d78740 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x558bafc7f570 con 0x558bafcaf940 2023-08-09T15:11:35.596 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:35.595+0000 7efca3fff640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x558bafcaee30 0x558bafc95090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T15:11:35.597 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:35.595+0000 7efca3fff640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x558bafcaee30 0x558bafc95090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:43562/0 (socket says 172.21.15.43:43562) 2023-08-09T15:11:35.597 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:35.595+0000 7efca3fff640 1 -- 172.21.15.43:0/471753772 learned_addr learned my addr 172.21.15.43:0/471753772 (peer_addr_for_me v2:172.21.15.43:0/0) 2023-08-09T15:11:35.600 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:35.595+0000 7efca3fff640 1 -- 172.21.15.43:0/471753772 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x558bafcaf940 msgr2=0x558bafc955d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-09T15:11:35.600 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:35.595+0000 7efca3fff640 1 --2- 172.21.15.43:0/471753772 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x558bafcaf940 0x558bafc955d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T15:11:35.600 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:35.595+0000 7efca3fff640 1 -- 172.21.15.43:0/471753772 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x558bafc829b0 msgr2=0x558bafc7a750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-09T15:11:35.600 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:35.595+0000 7efca3fff640 1 --2- 172.21.15.43:0/471753772 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x558bafc829b0 0x558bafc7a750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T15:11:35.600 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:35.595+0000 7efca3fff640 1 -- 172.21.15.43:0/471753772 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x558bafc1b300 con 0x558bafcaee30 2023-08-09T15:11:35.600 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:35.596+0000 7efca3fff640 1 --2- 172.21.15.43:0/471753772 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x558bafcaee30 0x558bafc95090 secure :-1 s=READY pgs=2422 cs=0 l=1 rev1=1 crypto rx=0x7efc9c075790 tx=0x7efc9c078cb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=65f6f0c6d698ddb6 server_cookie=0 in_seq=0 out_seq=0 2023-08-09T15:11:35.601 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:35.596+0000 7efc937fe640 1 -- 172.21.15.43:0/471753772 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efc9c080fa0 con 0x558bafcaee30 2023-08-09T15:11:35.601 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:35.596+0000 7efc937fe640 1 -- 172.21.15.43:0/471753772 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efc9c077630 con 0x558bafcaee30 2023-08-09T15:11:35.601 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:35.596+0000 7efc937fe640 1 -- 172.21.15.43:0/471753772 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efc9c0a84c0 con 0x558bafcaee30 2023-08-09T15:11:35.601 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:35.596+0000 7efcb5d78740 1 -- 172.21.15.43:0/471753772 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x558bafcaee30 msgr2=0x558bafc95090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T15:11:35.601 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:35.596+0000 7efcb5d78740 1 --2- 172.21.15.43:0/471753772 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x558bafcaee30 0x558bafc95090 secure :-1 s=READY pgs=2422 cs=0 l=1 rev1=1 crypto rx=0x7efc9c075790 tx=0x7efc9c078cb0 comp rx=0 tx=0).stop 2023-08-09T15:11:35.601 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:35.597+0000 7efcb5d78740 1 -- 172.21.15.43:0/471753772 shutdown_connections 2023-08-09T15:11:35.601 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:35.597+0000 7efcb5d78740 1 --2- 172.21.15.43:0/471753772 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x558bafcaf940 0x558bafc955d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T15:11:35.602 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:35.597+0000 7efcb5d78740 1 --2- 172.21.15.43:0/471753772 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x558bafcaee30 0x558bafc95090 unknown :-1 s=CLOSED pgs=2422 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T15:11:35.602 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:35.597+0000 7efcb5d78740 1 --2- 172.21.15.43:0/471753772 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x558bafc829b0 0x558bafc7a750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T15:11:35.602 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:35.597+0000 7efcb5d78740 1 -- 172.21.15.43:0/471753772 >> 172.21.15.43:0/471753772 conn(0x558bafc7edb0 msgr2=0x558bafcb0900 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-09T15:11:35.602 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:35.597+0000 7efcb5d78740 1 -- 172.21.15.43:0/471753772 shutdown_connections 2023-08-09T15:11:35.602 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:35.597+0000 7efcb5d78740 1 -- 172.21.15.43:0/471753772 wait complete. 2023-08-09T15:11:35.602 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:35.597+0000 7efcb5d78740 1 -- start start 2023-08-09T15:11:35.602 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:35.598+0000 7efcb5d78740 1 Processor -- start 2023-08-09T15:11:35.602 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:35.598+0000 7efcb5d78740 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x558bafc829b0 0x558bafc95090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T15:11:35.603 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:35.599+0000 7efcb5d78740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x558bafcaee30 0x558bafc955d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T15:11:35.603 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:35.599+0000 7efcb5d78740 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x558bafcaf940 0x558bafcd8160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T15:11:35.603 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:35.599+0000 7efcb5d78740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x558bafca8d70 con 0x558bafcaee30 2023-08-09T15:11:35.603 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:35.599+0000 7efcb5d78740 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x558bafd33b10 con 0x558bafc829b0 2023-08-09T15:11:35.603 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:35.599+0000 7efcb5d78740 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x558bafca8600 con 0x558bafcaf940 2023-08-09T15:11:35.603 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:35.599+0000 7efcb5d78740 10 client.0 _ll_register_callbacks cb 0x7efca4007dd0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-08-09T15:11:35.603 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:35.599+0000 7efca3fff640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x558bafcaee30 0x558bafc955d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T15:11:35.604 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:35.599+0000 7efca3fff640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x558bafcaee30 0x558bafc955d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:43576/0 (socket says 172.21.15.43:43576) 2023-08-09T15:11:35.604 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:35.599+0000 7efca3fff640 1 -- 172.21.15.43:0/666380797 learned_addr learned my addr 172.21.15.43:0/666380797 (peer_addr_for_me v2:172.21.15.43:0/0) 2023-08-09T15:11:35.604 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:35.599+0000 7efca3fff640 1 -- 172.21.15.43:0/666380797 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x558bafcaf940 msgr2=0x558bafcd8160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-09T15:11:35.604 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:35.599+0000 7efca3fff640 1 --2- 172.21.15.43:0/666380797 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x558bafcaf940 0x558bafcd8160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T15:11:35.605 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:35.599+0000 7efcb0ecb640 1 --2- 172.21.15.43:0/666380797 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x558bafc829b0 0x558bafc95090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T15:11:35.605 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:35.599+0000 7efca3fff640 1 -- 172.21.15.43:0/666380797 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x558bafc829b0 msgr2=0x558bafc95090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-09T15:11:35.605 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:35.599+0000 7efca3fff640 1 --2- 172.21.15.43:0/666380797 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x558bafc829b0 0x558bafc95090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T15:11:35.606 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:35.599+0000 7efca3fff640 1 -- 172.21.15.43:0/666380797 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7efcac0b2ac0 con 0x558bafcaee30 2023-08-09T15:11:35.606 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:35.600+0000 7efca3fff640 1 --2- 172.21.15.43:0/666380797 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x558bafcaee30 0x558bafc955d0 secure :-1 s=READY pgs=2423 cs=0 l=1 rev1=1 crypto rx=0x7efc9c079220 tx=0x7efc9c0aa040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c87f55ef9dcbc3e1 server_cookie=0 in_seq=0 out_seq=0 2023-08-09T15:11:35.606 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:35.600+0000 7efc8b7fe640 10 client.0 ms_handle_connect on v2:172.21.15.43:3300/0 2023-08-09T15:11:35.608 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:35.600+0000 7efc8b7fe640 1 -- 172.21.15.43:0/666380797 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efc9c0a86d0 con 0x558bafcaee30 2023-08-09T15:11:35.608 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:35.600+0000 7efc8b7fe640 1 -- 172.21.15.43:0/666380797 <== mon.0 v2:172.21.15.43:3300/0 2 ==== osd_map(223..223 src has 1..223) v4 ==== 6426+0+0 (secure 0 0 0) 0x7efc9c090180 con 0x558bafcaee30 2023-08-09T15:11:35.608 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:35.600+0000 7efcb5d78740 1 -- 172.21.15.43:0/666380797 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x558bafc11800 con 0x558bafcaee30 2023-08-09T15:11:35.608 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:35.600+0000 7efc8b7fe640 1 -- 172.21.15.43:0/666380797 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_get_version_reply(handle=1 version=104) v2 ==== 24+0+0 (secure 0 0 0) 0x7efc9c090340 con 0x558bafcaee30 2023-08-09T15:11:35.608 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:35.600+0000 7efcb5d78740 10 client.11496 fetch_fsmap learned FSMap version 104 2023-08-09T15:11:35.608 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:35.600+0000 7efcb5d78740 1 -- 172.21.15.43:0/666380797 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,fsmap=104,monmap=2+}) v3 -- 0x558bafb88a10 con 0x558bafcaee30 2023-08-09T15:11:35.609 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:35.600+0000 7efc8b7fe640 1 -- 172.21.15.43:0/666380797 <== mon.0 v2:172.21.15.43:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efc9c09c020 con 0x558bafcaee30 2023-08-09T15:11:35.609 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:35.601+0000 7efc8b7fe640 1 -- 172.21.15.43:0/666380797 <== mon.0 v2:172.21.15.43:3300/0 5 ==== fsmap(e 104) v1 ==== 3294+0+0 (secure 0 0 0) 0x7efc9c07e050 con 0x558bafcaee30 2023-08-09T15:11:35.609 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:35.601+0000 7efcb5d78740 10 client.11496 fetch_fsmap finished waiting for FSMap version 104 2023-08-09T15:11:35.609 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:35.601+0000 7efcb5d78740 10 client.11496 resolve_mds: resolved name 'a' to daemon mds.a 2023-08-09T15:11:35.609 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:35.601+0000 7efcb5d78740 20 client.11496 populate_metadata read hostname 'smithi043' 2023-08-09T15:11:35.610 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:35.601+0000 7efcb5d78740 1 --2- 172.21.15.43:0/666380797 >> [v2:172.21.15.43:6832/1425989175,v1:172.21.15.43:6833/1425989175] conn(0x558bafcd8710 0x558bafcd8af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T15:11:35.610 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:35.601+0000 7efcb5d78740 4 client.11496 mds_command: new command op to 11325 tid=0[{"prefix": "status"}] 2023-08-09T15:11:35.610 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:35.601+0000 7efcb5d78740 1 -- 172.21.15.43:0/666380797 --> [v2:172.21.15.43:6832/1425989175,v1:172.21.15.43:6833/1425989175] -- command(tid 0: {"prefix": "status"}) v1 -- 0x558bafbe28b0 con 0x558bafcd8710 2023-08-09T15:11:35.610 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:35.601+0000 7efcb0ecb640 1 --2- 172.21.15.43:0/666380797 >> [v2:172.21.15.43:6832/1425989175,v1:172.21.15.43:6833/1425989175] conn(0x558bafcd8710 0x558bafcd8af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T15:11:35.610 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:35.601+0000 7efcb0ecb640 1 --2- 172.21.15.43:0/666380797 >> [v2:172.21.15.43:6832/1425989175,v1:172.21.15.43:6833/1425989175] conn(0x558bafcd8710 0x558bafcd8af0 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a288bdf9f4c04a87 server_cookie=4755cb8105f1260f in_seq=0 out_seq=0 2023-08-09T15:11:35.610 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:35.601+0000 7efc8b7fe640 10 client.11496 ms_handle_connect on v2:172.21.15.43:6832/1425989175 2023-08-09T15:11:35.610 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:35.602+0000 7efc8b7fe640 1 -- 172.21.15.43:0/666380797 <== mds.0 v2:172.21.15.43:6832/1425989175 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+329 (crc 0 0 0) 0x558bafbe28b0 con 0x558bafcd8710 2023-08-09T15:11:35.611 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:35.602+0000 7efc8b7fe640 10 client.11496 handle_command_reply: tid=0 2023-08-09T15:11:35.611 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:35.602+0000 7efcb5d78740 1 client.11496 shutdown 2023-08-09T15:11:35.611 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:35.602+0000 7efcb5d78740 1 -- 172.21.15.43:0/666380797 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x558bafcaee30 msgr2=0x558bafc955d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T15:11:35.611 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:35.602+0000 7efcb5d78740 1 --2- 172.21.15.43:0/666380797 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x558bafcaee30 0x558bafc955d0 secure :-1 s=READY pgs=2423 cs=0 l=1 rev1=1 crypto rx=0x7efc9c079220 tx=0x7efc9c0aa040 comp rx=0 tx=0).stop 2023-08-09T15:11:35.611 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:35.602+0000 7efcb5d78740 1 -- 172.21.15.43:0/666380797 shutdown_connections 2023-08-09T15:11:35.611 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:35.602+0000 7efcb5d78740 1 --2- 172.21.15.43:0/666380797 >> [v2:172.21.15.43:6832/1425989175,v1:172.21.15.43:6833/1425989175] conn(0x558bafcd8710 0x558bafcd8af0 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T15:11:35.611 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:35.602+0000 7efcb5d78740 1 --2- 172.21.15.43:0/666380797 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x558bafcaf940 0x558bafcd8160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T15:11:35.611 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:35.602+0000 7efcb5d78740 1 --2- 172.21.15.43:0/666380797 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x558bafcaee30 0x558bafc955d0 unknown :-1 s=CLOSED pgs=2423 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T15:11:35.612 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:35.602+0000 7efcb5d78740 1 --2- 172.21.15.43:0/666380797 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x558bafc829b0 0x558bafc95090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T15:11:35.612 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:35.602+0000 7efcb5d78740 1 -- 172.21.15.43:0/666380797 >> 172.21.15.43:0/666380797 conn(0x558bafc7edb0 msgr2=0x558bafcb08c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-09T15:11:35.612 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:35.602+0000 7efc8b7fe640 0 client.11496 ms_handle_reset on v2:172.21.15.43:6832/1425989175 2023-08-09T15:11:35.612 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:35.602+0000 7efcb5d78740 1 -- 172.21.15.43:0/666380797 shutdown_connections 2023-08-09T15:11:35.612 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:35.602+0000 7efcb5d78740 1 -- 172.21.15.43:0/666380797 wait complete. 2023-08-09T15:11:35.612 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:35.602+0000 7efcb5d78740 20 client.11496 trim_cache size 0 max 16384 2023-08-09T15:11:35.613 INFO:teuthology.orchestra.run.smithi043.stdout:{ 2023-08-09T15:11:35.613 INFO:teuthology.orchestra.run.smithi043.stdout: "cluster_fsid": "daac87ee-1337-4250-a955-aea4e408a39b", 2023-08-09T15:11:35.613 INFO:teuthology.orchestra.run.smithi043.stdout: "whoami": 0, 2023-08-09T15:11:35.613 INFO:teuthology.orchestra.run.smithi043.stdout: "id": 11325, 2023-08-09T15:11:35.613 INFO:teuthology.orchestra.run.smithi043.stdout: "want_state": "up:active", 2023-08-09T15:11:35.613 INFO:teuthology.orchestra.run.smithi043.stdout: "state": "up:active", 2023-08-09T15:11:35.614 INFO:teuthology.orchestra.run.smithi043.stdout: "fs_name": "cephfs", 2023-08-09T15:11:35.614 INFO:teuthology.orchestra.run.smithi043.stdout: "rank_uptime": 6.9105968899999999, 2023-08-09T15:11:35.614 INFO:teuthology.orchestra.run.smithi043.stdout: "mdsmap_epoch": 104, 2023-08-09T15:11:35.614 INFO:teuthology.orchestra.run.smithi043.stdout: "osdmap_epoch": 223, 2023-08-09T15:11:35.614 INFO:teuthology.orchestra.run.smithi043.stdout: "osdmap_epoch_barrier": 223, 2023-08-09T15:11:35.614 INFO:teuthology.orchestra.run.smithi043.stdout: "uptime": 12.840550172 2023-08-09T15:11:35.615 INFO:teuthology.orchestra.run.smithi043.stdout:} 2023-08-09T15:11:35.615 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:35.603+0000 7efcb1ecd640 1 -- 172.21.15.43:0/3798044542 >> [v2:172.21.15.203:6832/4034093268,v1:172.21.15.203:6833/4034093268] conn(0x7efc9806a170 msgr2=0x7efc9806c630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T15:11:35.616 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:35.603+0000 7efcb1ecd640 1 --2- 172.21.15.43:0/3798044542 >> [v2:172.21.15.203:6832/4034093268,v1:172.21.15.203:6833/4034093268] conn(0x7efc9806a170 0x7efc9806c630 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7efca8009a00 tx=0x7efca8008040 comp rx=0 tx=0).stop 2023-08-09T15:11:35.616 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:35.603+0000 7efcb1ecd640 1 -- 172.21.15.43:0/3798044542 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7efcac0cd4e0 msgr2=0x7efcac0d1da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T15:11:35.616 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:35.603+0000 7efcb1ecd640 1 --2- 172.21.15.43:0/3798044542 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7efcac0cd4e0 0x7efcac0d1da0 secure :-1 s=READY pgs=2419 cs=0 l=1 rev1=1 crypto rx=0x7efc9c02bee0 tx=0x7efc9c0046c0 comp rx=0 tx=0).stop 2023-08-09T15:11:35.616 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:35.603+0000 7efcb1ecd640 1 -- 172.21.15.43:0/3798044542 shutdown_connections 2023-08-09T15:11:35.616 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:35.603+0000 7efcb1ecd640 1 --2- 172.21.15.43:0/3798044542 >> [v2:172.21.15.203:6832/4034093268,v1:172.21.15.203:6833/4034093268] conn(0x7efc9806a170 0x7efc9806c630 unknown :-1 s=CLOSED pgs=68 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T15:11:35.617 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:35.603+0000 7efcb1ecd640 1 --2- 172.21.15.43:0/3798044542 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7efcac15f7b0 0x7efcac0d22e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T15:11:35.617 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:35.603+0000 7efcb1ecd640 1 --2- 172.21.15.43:0/3798044542 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7efcac0cd4e0 0x7efcac0d1da0 unknown :-1 s=CLOSED pgs=2419 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T15:11:35.617 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:35.603+0000 7efcb1ecd640 1 --2- 172.21.15.43:0/3798044542 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7efcac0b1050 0x7efcac14fb60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T15:11:35.617 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:35.603+0000 7efcb1ecd640 1 -- 172.21.15.43:0/3798044542 >> 172.21.15.43:0/3798044542 conn(0x7efcac06a2b0 msgr2=0x7efcac15ddb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-09T15:11:35.617 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:35.603+0000 7efcb1ecd640 1 -- 172.21.15.43:0/3798044542 shutdown_connections 2023-08-09T15:11:35.617 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:35.603+0000 7efcb1ecd640 1 -- 172.21.15.43:0/3798044542 wait complete. 2023-08-09T15:11:35.620 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2023-08-09T15:11:35.834 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:35.833+0000 7fa536037640 1 Processor -- start 2023-08-09T15:11:35.834 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:35.834+0000 7fa536037640 1 -- start start 2023-08-09T15:11:35.835 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:35.835+0000 7fa536037640 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7fa5300c8750 0x7fa5300ad340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T15:11:35.836 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:35.835+0000 7fa536037640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fa5300c8b30 0x7fa53006f410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T15:11:35.836 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:35.835+0000 7fa536037640 1 -- --> v1:172.21.15.43:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa5300cbd90 con 0x7fa5300acdf0 2023-08-09T15:11:35.836 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:35.835+0000 7fa536037640 1 -- --> v2:172.21.15.203:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa53015e810 con 0x7fa5300c8750 2023-08-09T15:11:35.836 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:35.835+0000 7fa536037640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa5300c9f70 con 0x7fa5300c8b30 2023-08-09T15:11:35.838 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:35.836+0000 7fa535836640 1 --1- >> v1:172.21.15.43:6789/0 conn(0x7fa5300acdf0 0x7fa530070f00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.43:6789/0 says I am v1:172.21.15.43:59206/0 (socket says 172.21.15.43:59206) 2023-08-09T15:11:35.838 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:35.836+0000 7fa535836640 1 -- 172.21.15.43:0/3767976341 learned_addr learned my addr 172.21.15.43:0/3767976341 (peer_addr_for_me v1:172.21.15.43:0/0) 2023-08-09T15:11:35.838 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:35.836+0000 7fa535035640 1 --2- 172.21.15.43:0/3767976341 >> v2:172.21.15.203:3300/0 conn(0x7fa5300c8750 0x7fa5300ad340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T15:11:35.838 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:35.836+0000 7fa534834640 1 --2- 172.21.15.43:0/3767976341 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fa5300c8b30 0x7fa53006f410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T15:11:35.839 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:35.836+0000 7fa527fff640 1 -- 172.21.15.43:0/3767976341 <== mon.0 v1:172.21.15.43:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1855319993 0 0) 0x7fa5300cbd90 con 0x7fa5300acdf0 2023-08-09T15:11:35.839 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:35.836+0000 7fa527fff640 1 -- 172.21.15.43:0/3767976341 --> v1:172.21.15.43:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fa51c003520 con 0x7fa5300acdf0 2023-08-09T15:11:35.839 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:35.837+0000 7fa527fff640 1 -- 172.21.15.43:0/3767976341 <== mon.0 v1:172.21.15.43:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2726226434 0 0) 0x7fa51c003520 con 0x7fa5300acdf0 2023-08-09T15:11:35.839 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:35.837+0000 7fa527fff640 1 -- 172.21.15.43:0/3767976341 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fa5300c8b30 msgr2=0x7fa53006f410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-09T15:11:35.839 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:35.837+0000 7fa527fff640 1 --2- 172.21.15.43:0/3767976341 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fa5300c8b30 0x7fa53006f410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T15:11:35.839 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:35.837+0000 7fa527fff640 1 -- 172.21.15.43:0/3767976341 >> v2:172.21.15.203:3300/0 conn(0x7fa5300c8750 msgr2=0x7fa5300ad340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-09T15:11:35.839 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:35.837+0000 7fa527fff640 1 --2- 172.21.15.43:0/3767976341 >> v2:172.21.15.203:3300/0 conn(0x7fa5300c8750 0x7fa5300ad340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T15:11:35.839 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:35.837+0000 7fa527fff640 1 -- 172.21.15.43:0/3767976341 --> v1:172.21.15.43:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa5300ca2b0 con 0x7fa5300acdf0 2023-08-09T15:11:35.840 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:35.837+0000 7fa527fff640 1 -- 172.21.15.43:0/3767976341 <== mon.0 v1:172.21.15.43:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1285150740 0 0) 0x7fa5280019f0 con 0x7fa5300acdf0 2023-08-09T15:11:35.840 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:35.837+0000 7fa527fff640 1 -- 172.21.15.43:0/3767976341 >> v1:172.21.15.43:6789/0 conn(0x7fa5300acdf0 legacy=0x7fa530070f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T15:11:35.840 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:35.837+0000 7fa527fff640 1 --2- 172.21.15.43:0/3767976341 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa51c003970 0x7fa51c003d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T15:11:35.840 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:35.837+0000 7fa527fff640 1 --2- 172.21.15.43:0/3767976341 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fa51c0047c0 0x7fa51c004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T15:11:35.840 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:35.837+0000 7fa527fff640 1 --2- 172.21.15.43:0/3767976341 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fa5300c8b30 0x7fa51c005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T15:11:35.840 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:35.837+0000 7fa527fff640 1 -- 172.21.15.43:0/3767976341 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa53015e810 con 0x7fa51c003970 2023-08-09T15:11:35.840 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:35.837+0000 7fa527fff640 1 -- 172.21.15.43:0/3767976341 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa5300c9f70 con 0x7fa51c0047c0 2023-08-09T15:11:35.841 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:35.837+0000 7fa527fff640 1 -- 172.21.15.43:0/3767976341 --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa51c00b980 con 0x7fa5300c8b30 2023-08-09T15:11:35.841 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:35.837+0000 7fa535035640 1 --2- 172.21.15.43:0/3767976341 >> v2:172.21.15.203:3300/0 conn(0x7fa5300c8750 0x7fa5300ad340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-08-09T15:11:35.841 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:35.838+0000 7fa535035640 1 --2- 172.21.15.43:0/3767976341 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa51c003970 0x7fa51c003d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T15:11:35.842 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:35.838+0000 7fa534834640 1 --2- 172.21.15.43:0/3767976341 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fa51c0047c0 0x7fa51c004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T15:11:35.842 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:35.838+0000 7fa535836640 1 --2- 172.21.15.43:0/3767976341 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fa5300c8b30 0x7fa51c005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T15:11:35.842 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:35.838+0000 7fa535035640 1 -- 172.21.15.43:0/3767976341 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fa5300c8b30 msgr2=0x7fa51c005180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-09T15:11:35.842 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:35.838+0000 7fa535035640 1 --2- 172.21.15.43:0/3767976341 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fa5300c8b30 0x7fa51c005180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T15:11:35.843 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:35.838+0000 7fa535035640 1 -- 172.21.15.43:0/3767976341 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fa51c0047c0 msgr2=0x7fa51c004c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-09T15:11:35.843 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:35.838+0000 7fa535035640 1 --2- 172.21.15.43:0/3767976341 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fa51c0047c0 0x7fa51c004c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T15:11:35.843 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:35.838+0000 7fa535035640 1 -- 172.21.15.43:0/3767976341 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa51c003520 con 0x7fa51c003970 2023-08-09T15:11:35.843 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:35.838+0000 7fa535836640 1 --2- 172.21.15.43:0/3767976341 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fa5300c8b30 0x7fa51c005180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-08-09T15:11:35.843 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:35.839+0000 7fa535035640 1 --2- 172.21.15.43:0/3767976341 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa51c003970 0x7fa51c003d90 secure :-1 s=READY pgs=2425 cs=0 l=1 rev1=1 crypto rx=0x7fa52c009c70 tx=0x7fa51c0057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c2628314199077a9 server_cookie=0 in_seq=0 out_seq=0 2023-08-09T15:11:35.843 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:35.839+0000 7fa527fff640 1 -- 172.21.15.43:0/3767976341 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa52c004020 con 0x7fa51c003970 2023-08-09T15:11:35.843 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:35.839+0000 7fa527fff640 1 -- 172.21.15.43:0/3767976341 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa52c004180 con 0x7fa51c003970 2023-08-09T15:11:35.843 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:35.839+0000 7fa527fff640 1 -- 172.21.15.43:0/3767976341 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa52c02c960 con 0x7fa51c003970 2023-08-09T15:11:35.844 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:35.840+0000 7fa536037640 1 -- 172.21.15.43:0/3767976341 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa51c003970 msgr2=0x7fa51c003d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T15:11:35.844 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:35.840+0000 7fa536037640 1 --2- 172.21.15.43:0/3767976341 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa51c003970 0x7fa51c003d90 secure :-1 s=READY pgs=2425 cs=0 l=1 rev1=1 crypto rx=0x7fa52c009c70 tx=0x7fa51c0057f0 comp rx=0 tx=0).stop 2023-08-09T15:11:35.844 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:35.840+0000 7fa535836640 1 -- 172.21.15.43:0/3767976341 reap_dead start 2023-08-09T15:11:35.844 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:35.840+0000 7fa536037640 1 -- 172.21.15.43:0/3767976341 shutdown_connections 2023-08-09T15:11:35.844 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:35.840+0000 7fa536037640 1 -- 172.21.15.43:0/3767976341 >> 172.21.15.43:0/3767976341 conn(0x7fa5300cde90 msgr2=0x7fa5300ce2a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-09T15:11:35.844 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:35.840+0000 7fa536037640 1 -- 172.21.15.43:0/3767976341 shutdown_connections 2023-08-09T15:11:35.844 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:35.840+0000 7fa536037640 1 -- 172.21.15.43:0/3767976341 wait complete. 2023-08-09T15:11:35.846 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:35.842+0000 7fa536037640 1 Processor -- start 2023-08-09T15:11:35.846 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:35.842+0000 7fa536037640 1 -- start start 2023-08-09T15:11:35.846 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:35.843+0000 7fa536037640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa51c003970 0x7fa5300b3520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T15:11:35.847 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:35.843+0000 7fa536037640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fa5300c69e0 0x7fa530159120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T15:11:35.847 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:35.843+0000 7fa536037640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fa53015db50 0x7fa53015e000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T15:11:35.847 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:35.843+0000 7fa536037640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa53015e540 con 0x7fa51c003970 2023-08-09T15:11:35.847 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:35.843+0000 7fa536037640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa5300c52b0 con 0x7fa5300c69e0 2023-08-09T15:11:35.847 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:35.843+0000 7fa536037640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa530159660 con 0x7fa53015db50 2023-08-09T15:11:35.847 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:35.844+0000 7fa535035640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa51c003970 0x7fa5300b3520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T15:11:35.847 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:35.844+0000 7fa535035640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa51c003970 0x7fa5300b3520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:43604/0 (socket says 172.21.15.43:43604) 2023-08-09T15:11:35.847 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:35.844+0000 7fa535035640 1 -- 172.21.15.43:0/2801838873 learned_addr learned my addr 172.21.15.43:0/2801838873 (peer_addr_for_me v2:172.21.15.43:0/0) 2023-08-09T15:11:35.848 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:35.844+0000 7fa534834640 1 --2- 172.21.15.43:0/2801838873 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fa5300c69e0 0x7fa530159120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T15:11:35.848 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:35.844+0000 7fa535836640 1 --2- 172.21.15.43:0/2801838873 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fa53015db50 0x7fa53015e000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T15:11:35.848 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:35.844+0000 7fa535035640 1 -- 172.21.15.43:0/2801838873 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fa53015db50 msgr2=0x7fa53015e000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T15:11:35.848 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:35.844+0000 7fa535035640 1 --2- 172.21.15.43:0/2801838873 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fa53015db50 0x7fa53015e000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T15:11:35.848 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:35.844+0000 7fa535035640 1 -- 172.21.15.43:0/2801838873 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fa5300c69e0 msgr2=0x7fa530159120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T15:11:35.848 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:35.844+0000 7fa535035640 1 --2- 172.21.15.43:0/2801838873 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fa5300c69e0 0x7fa530159120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T15:11:35.848 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:35.844+0000 7fa535035640 1 -- 172.21.15.43:0/2801838873 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa5300c7150 con 0x7fa51c003970 2023-08-09T15:11:35.849 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:35.844+0000 7fa534834640 1 --2- 172.21.15.43:0/2801838873 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fa5300c69e0 0x7fa530159120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-08-09T15:11:35.849 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:35.845+0000 7fa535035640 1 --2- 172.21.15.43:0/2801838873 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa51c003970 0x7fa5300b3520 secure :-1 s=READY pgs=2426 cs=0 l=1 rev1=1 crypto rx=0x7fa52c009d90 tx=0x7fa52c004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-09T15:11:35.849 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:35.845+0000 7fa525ffb640 1 -- 172.21.15.43:0/2801838873 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa52c02cb70 con 0x7fa51c003970 2023-08-09T15:11:35.849 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:35.845+0000 7fa525ffb640 1 -- 172.21.15.43:0/2801838873 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa52c02ccd0 con 0x7fa51c003970 2023-08-09T15:11:35.849 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:35.845+0000 7fa525ffb640 1 -- 172.21.15.43:0/2801838873 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa52c006260 con 0x7fa51c003970 2023-08-09T15:11:35.849 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:35.845+0000 7fa536037640 1 -- 172.21.15.43:0/2801838873 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa5300c7430 con 0x7fa51c003970 2023-08-09T15:11:35.849 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:35.845+0000 7fa536037640 1 -- 172.21.15.43:0/2801838873 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa5300c79f0 con 0x7fa51c003970 2023-08-09T15:11:35.850 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:35.846+0000 7fa536037640 1 -- 172.21.15.43:0/2801838873 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa5300c9230 con 0x7fa51c003970 2023-08-09T15:11:35.850 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:35.847+0000 7fa525ffb640 1 -- 172.21.15.43:0/2801838873 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 118) v1 ==== 88443+0+0 (secure 0 0 0) 0x7fa52c0063c0 con 0x7fa51c003970 2023-08-09T15:11:35.850 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:35.848+0000 7fa525ffb640 1 --2- 172.21.15.43:0/2801838873 >> [v2:172.21.15.203:6832/4034093268,v1:172.21.15.203:6833/4034093268] conn(0x7fa51406a170 0x7fa51406c630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T15:11:35.850 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:35.848+0000 7fa525ffb640 1 -- 172.21.15.43:0/2801838873 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(223..223 src has 1..223) v4 ==== 6426+0+0 (secure 0 0 0) 0x7fa52c0aac20 con 0x7fa51c003970 2023-08-09T15:11:35.851 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:35.849+0000 7fa534834640 1 --2- 172.21.15.43:0/2801838873 >> [v2:172.21.15.203:6832/4034093268,v1:172.21.15.203:6833/4034093268] conn(0x7fa51406a170 0x7fa51406c630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T15:11:35.851 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:35.850+0000 7fa534834640 1 --2- 172.21.15.43:0/2801838873 >> [v2:172.21.15.203:6832/4034093268,v1:172.21.15.203:6833/4034093268] conn(0x7fa51406a170 0x7fa51406c630 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7fa5300c7f30 tx=0x7fa520008040 comp rx=0 tx=0).ready entity=mgr.8833 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-09T15:11:35.853 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:35.852+0000 7fa525ffb640 1 -- 172.21.15.43:0/2801838873 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178066 (secure 0 0 0) 0x7fa52c076090 con 0x7fa51c003970 2023-08-09T15:11:36.085 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:36.084+0000 7fa536037640 1 -- 172.21.15.43:0/2801838873 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7fa5300739f0 con 0x7fa51c003970 2023-08-09T15:11:36.085 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:36.085+0000 7fa525ffb640 1 -- 172.21.15.43:0/2801838873 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v104) v1 ==== 53+0+73 (secure 0 0 0) 0x7fa52c043060 con 0x7fa51c003970 2023-08-09T15:11:36.086 INFO:teuthology.orchestra.run.smithi043.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2023-08-09T15:11:36.088 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:36.087+0000 7fa536037640 1 -- 172.21.15.43:0/2801838873 >> [v2:172.21.15.203:6832/4034093268,v1:172.21.15.203:6833/4034093268] conn(0x7fa51406a170 msgr2=0x7fa51406c630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T15:11:36.088 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:36.087+0000 7fa536037640 1 --2- 172.21.15.43:0/2801838873 >> [v2:172.21.15.203:6832/4034093268,v1:172.21.15.203:6833/4034093268] conn(0x7fa51406a170 0x7fa51406c630 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7fa5300c7f30 tx=0x7fa520008040 comp rx=0 tx=0).stop 2023-08-09T15:11:36.089 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:36.088+0000 7fa536037640 1 -- 172.21.15.43:0/2801838873 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa51c003970 msgr2=0x7fa5300b3520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T15:11:36.089 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:36.088+0000 7fa536037640 1 --2- 172.21.15.43:0/2801838873 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa51c003970 0x7fa5300b3520 secure :-1 s=READY pgs=2426 cs=0 l=1 rev1=1 crypto rx=0x7fa52c009d90 tx=0x7fa52c004730 comp rx=0 tx=0).stop 2023-08-09T15:11:36.089 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:36.088+0000 7fa536037640 1 -- 172.21.15.43:0/2801838873 shutdown_connections 2023-08-09T15:11:36.089 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:36.088+0000 7fa536037640 1 --2- 172.21.15.43:0/2801838873 >> [v2:172.21.15.203:6832/4034093268,v1:172.21.15.203:6833/4034093268] conn(0x7fa51406a170 0x7fa51406c630 unknown :-1 s=CLOSED pgs=69 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T15:11:36.089 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:36.088+0000 7fa536037640 1 --2- 172.21.15.43:0/2801838873 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fa53015db50 0x7fa53015e000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T15:11:36.089 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:36.088+0000 7fa536037640 1 --2- 172.21.15.43:0/2801838873 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fa5300c69e0 0x7fa530159120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T15:11:36.089 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:36.088+0000 7fa536037640 1 --2- 172.21.15.43:0/2801838873 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa51c003970 0x7fa5300b3520 unknown :-1 s=CLOSED pgs=2426 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T15:11:36.089 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:36.088+0000 7fa536037640 1 -- 172.21.15.43:0/2801838873 >> 172.21.15.43:0/2801838873 conn(0x7fa5300cde90 msgr2=0x7fa5300ac670 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-09T15:11:36.090 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:36.088+0000 7fa536037640 1 -- 172.21.15.43:0/2801838873 shutdown_connections 2023-08-09T15:11:36.090 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:36.088+0000 7fa536037640 1 -- 172.21.15.43:0/2801838873 wait complete. 2023-08-09T15:11:36.106 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2023-08-09T15:11:36.106 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2023-08-09T15:11:36.107 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi043.front.sepia.ceph.com 2023-08-09T15:11:36.107 INFO:tasks.cephfs.mount:self.client.name = client.0 2023-08-09T15:11:36.107 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2023-08-09T15:11:36.107 INFO:tasks.cephfs.mount:self.cephfs_name = None 2023-08-09T15:11:36.108 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2023-08-09T15:11:36.108 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2023-08-09T15:11:36.108 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2023-08-09T15:11:36.109 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-09T15:11:36.109 DEBUG:teuthology.orchestra.run.smithi043:> ip addr 2023-08-09T15:11:36.167 INFO:teuthology.orchestra.run.smithi043.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2023-08-09T15:11:36.168 INFO:teuthology.orchestra.run.smithi043.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2023-08-09T15:11:36.168 INFO:teuthology.orchestra.run.smithi043.stdout: inet 127.0.0.1/8 scope host lo 2023-08-09T15:11:36.168 INFO:teuthology.orchestra.run.smithi043.stdout: valid_lft forever preferred_lft forever 2023-08-09T15:11:36.168 INFO:teuthology.orchestra.run.smithi043.stdout: inet6 ::1/128 scope host 2023-08-09T15:11:36.169 INFO:teuthology.orchestra.run.smithi043.stdout: valid_lft forever preferred_lft forever 2023-08-09T15:11:36.169 INFO:teuthology.orchestra.run.smithi043.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-08-09T15:11:36.169 INFO:teuthology.orchestra.run.smithi043.stdout: link/ether 0c:c4:7a:6b:fd:9c brd ff:ff:ff:ff:ff:ff 2023-08-09T15:11:36.169 INFO:teuthology.orchestra.run.smithi043.stdout: altname enp7s0f0 2023-08-09T15:11:36.169 INFO:teuthology.orchestra.run.smithi043.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-08-09T15:11:36.169 INFO:teuthology.orchestra.run.smithi043.stdout: link/ether 0c:c4:7a:6b:fd:9d brd ff:ff:ff:ff:ff:ff 2023-08-09T15:11:36.169 INFO:teuthology.orchestra.run.smithi043.stdout: altname enp7s0f1 2023-08-09T15:11:36.170 INFO:teuthology.orchestra.run.smithi043.stdout:4: ens1f0: mtu 1500 qdisc mq state UP group default qlen 1000 2023-08-09T15:11:36.170 INFO:teuthology.orchestra.run.smithi043.stdout: link/ether 0c:c4:7a:bd:13:58 brd ff:ff:ff:ff:ff:ff 2023-08-09T15:11:36.170 INFO:teuthology.orchestra.run.smithi043.stdout: altname enp1s0f0 2023-08-09T15:11:36.170 INFO:teuthology.orchestra.run.smithi043.stdout: inet 172.21.15.43/20 brd 172.21.15.255 scope global dynamic noprefixroute ens1f0 2023-08-09T15:11:36.170 INFO:teuthology.orchestra.run.smithi043.stdout: valid_lft 36407sec preferred_lft 36407sec 2023-08-09T15:11:36.170 INFO:teuthology.orchestra.run.smithi043.stdout: inet6 fe80::ec4:7aff:febd:1358/64 scope link 2023-08-09T15:11:36.170 INFO:teuthology.orchestra.run.smithi043.stdout: valid_lft forever preferred_lft forever 2023-08-09T15:11:36.171 INFO:teuthology.orchestra.run.smithi043.stdout:5: ens1f1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-08-09T15:11:36.171 INFO:teuthology.orchestra.run.smithi043.stdout: link/ether 0c:c4:7a:bd:13:59 brd ff:ff:ff:ff:ff:ff 2023-08-09T15:11:36.171 INFO:teuthology.orchestra.run.smithi043.stdout: altname enp1s0f1 2023-08-09T15:11:36.171 INFO:teuthology.orchestra.run.smithi043.stdout:8: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2023-08-09T15:11:36.171 INFO:teuthology.orchestra.run.smithi043.stdout: link/ether 56:43:50:19:f4:0c brd ff:ff:ff:ff:ff:ff 2023-08-09T15:11:36.171 INFO:teuthology.orchestra.run.smithi043.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2023-08-09T15:11:36.172 INFO:teuthology.orchestra.run.smithi043.stdout: valid_lft forever preferred_lft forever 2023-08-09T15:11:36.172 INFO:teuthology.orchestra.run.smithi043.stdout: inet6 fe80::e494:65ff:fe92:a182/64 scope link 2023-08-09T15:11:36.172 INFO:teuthology.orchestra.run.smithi043.stdout: valid_lft forever preferred_lft forever 2023-08-09T15:11:36.172 INFO:teuthology.orchestra.run.smithi043.stdout:9: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2023-08-09T15:11:36.172 INFO:teuthology.orchestra.run.smithi043.stdout: link/ether 7e:93:26:fa:07:8f brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.0 2023-08-09T15:11:36.172 INFO:teuthology.orchestra.run.smithi043.stdout: inet6 fe80::7c93:26ff:fefa:78f/64 scope link 2023-08-09T15:11:36.172 INFO:teuthology.orchestra.run.smithi043.stdout: valid_lft forever preferred_lft forever 2023-08-09T15:11:36.173 INFO:teuthology.orchestra.run.smithi043.stdout:10: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2023-08-09T15:11:36.173 INFO:teuthology.orchestra.run.smithi043.stdout: link/ether 56:43:50:19:f4:0c brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.2 2023-08-09T15:11:36.173 INFO:teuthology.orchestra.run.smithi043.stdout: inet6 fe80::5443:50ff:fe19:f40c/64 scope link 2023-08-09T15:11:36.173 INFO:teuthology.orchestra.run.smithi043.stdout: valid_lft forever preferred_lft forever 2023-08-09T15:11:36.174 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-09T15:11:36.174 DEBUG:teuthology.orchestra.run.smithi043:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2023-08-09T15:11:36.278 INFO:teuthology.orchestra.run.smithi043.stdout:1 2023-08-09T15:11:36.283 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-09T15:11:36.283 DEBUG:teuthology.orchestra.run.smithi043:> route 2023-08-09T15:11:36.343 INFO:teuthology.orchestra.run.smithi043.stdout:Kernel IP routing table 2023-08-09T15:11:36.369 INFO:teuthology.orchestra.run.smithi043.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2023-08-09T15:11:36.369 INFO:teuthology.orchestra.run.smithi043.stdout:default _gateway 0.0.0.0 UG 100 0 0 ens1f0 2023-08-09T15:11:36.370 INFO:teuthology.orchestra.run.smithi043.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 ens1f0 2023-08-09T15:11:36.370 INFO:teuthology.orchestra.run.smithi043.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2023-08-09T15:11:36.371 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-09T15:11:36.372 DEBUG:teuthology.orchestra.run.smithi043:> (cd / && exec bash -c ' 2023-08-09T15:11:36.372 DEBUG:teuthology.orchestra.run.smithi043:> set -e 2023-08-09T15:11:36.373 DEBUG:teuthology.orchestra.run.smithi043:> sudo iptables -A FORWARD -o ens1f0 -i ceph-brx -j ACCEPT 2023-08-09T15:11:36.373 DEBUG:teuthology.orchestra.run.smithi043:> sudo iptables -A FORWARD -i ens1f0 -o ceph-brx -j ACCEPT 2023-08-09T15:11:36.373 DEBUG:teuthology.orchestra.run.smithi043:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o ens1f0 -j MASQUERADE 2023-08-09T15:11:36.374 DEBUG:teuthology.orchestra.run.smithi043:> ') 2023-08-09T15:11:36.581 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-09T15:11:36.582 DEBUG:teuthology.orchestra.run.smithi043:> ip netns list 2023-08-09T15:11:36.640 INFO:teuthology.orchestra.run.smithi043.stdout:ceph-ns--home-ubuntu-cephtest-mnt.2 (id: 1) 2023-08-09T15:11:36.641 INFO:teuthology.orchestra.run.smithi043.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2023-08-09T15:11:36.642 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2023-08-09T15:11:36.642 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-09T15:11:36.642 DEBUG:teuthology.orchestra.run.smithi043:> sudo ip link set brx.0 up 2023-08-09T15:11:36.749 INFO:tasks.cephfs.fuse_mount:Client client.0 config is {} 2023-08-09T15:11:36.750 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-08-09T15:11:36.750 DEBUG:teuthology.orchestra.run.smithi043:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2023-08-09T15:11:36.807 INFO:teuthology.orchestra.run.smithi043.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2023-08-09T15:11:36.808 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-08-09T15:11:36.808 DEBUG:teuthology.orchestra.run.smithi043:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2023-08-09T15:11:36.861 DEBUG:teuthology.orchestra.run.smithi043:> sudo modprobe fuse 2023-08-09T15:11:36.963 INFO:teuthology.orchestra.run:Running command with timeout 30 2023-08-09T15:11:36.963 DEBUG:teuthology.orchestra.run.smithi043:> sudo mount -t fusectl /sys/fs/fuse/connections /sys/fs/fuse/connections 2023-08-09T15:11:37.070 INFO:teuthology.orchestra.run.smithi043.stderr:mount: /sys/fs/fuse/connections: /sys/fs/fuse/connections already mounted or mount point busy. 2023-08-09T15:11:37.074 DEBUG:teuthology.orchestra.run:got remote process result: 32 2023-08-09T15:11:37.075 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-09T15:11:37.075 DEBUG:teuthology.orchestra.run.smithi043:> ls /sys/fs/fuse/connections 2023-08-09T15:11:37.134 INFO:tasks.cephfs.fuse_mount:Pre-mount connections: [] 2023-08-09T15:11:37.134 DEBUG:teuthology.orchestra.run.smithi043:> (cd /home/ubuntu/cephtest && exec sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-fuse -f --admin-socket '/var/run/ceph/$cluster-$name.$pid.asok' /home/ubuntu/cephtest/mnt.0 --id 0) 2023-08-09T15:11:37.140 DEBUG:teuthology.orchestra.run.smithi043:> sudo modprobe fuse 2023-08-09T15:11:37.242 INFO:teuthology.orchestra.run:Running command with timeout 30 2023-08-09T15:11:37.243 DEBUG:teuthology.orchestra.run.smithi043:> sudo mount -t fusectl /sys/fs/fuse/connections /sys/fs/fuse/connections 2023-08-09T15:11:37.327 INFO:teuthology.orchestra.run.smithi043.stderr:mount: /sys/fs/fuse/connections: /sys/fs/fuse/connections already mounted or mount point busy. 2023-08-09T15:11:37.330 DEBUG:teuthology.orchestra.run:got remote process result: 32 2023-08-09T15:11:37.331 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-09T15:11:37.331 DEBUG:teuthology.orchestra.run.smithi043:> ls /sys/fs/fuse/connections 2023-08-09T15:11:37.359 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi043.stderr:2023-08-09T15:11:37.359+0000 7f78f0044480 -1 init, newargv = 0x561113edf460 newargc=15 2023-08-09T15:11:37.360 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi043.stderr:ceph-fuse[153632]: starting ceph client 2023-08-09T15:11:37.370 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi043.stderr:ceph-fuse[153632]: starting fuse 2023-08-09T15:11:37.380 INFO:teuthology.orchestra.run.smithi043.stdout:40 2023-08-09T15:11:37.381 INFO:tasks.cephfs.fuse_mount:Post-mount connections: [40] 2023-08-09T15:11:37.381 DEBUG:teuthology.orchestra.run.smithi043:> sudo stdin-killer -- python3 -c ' 2023-08-09T15:11:37.381 DEBUG:teuthology.orchestra.run.smithi043:> import glob 2023-08-09T15:11:37.382 DEBUG:teuthology.orchestra.run.smithi043:> import re 2023-08-09T15:11:37.382 DEBUG:teuthology.orchestra.run.smithi043:> import os 2023-08-09T15:11:37.383 DEBUG:teuthology.orchestra.run.smithi043:> import subprocess 2023-08-09T15:11:37.383 DEBUG:teuthology.orchestra.run.smithi043:> 2023-08-09T15:11:37.384 DEBUG:teuthology.orchestra.run.smithi043:> def _find_admin_socket(client_name): 2023-08-09T15:11:37.384 DEBUG:teuthology.orchestra.run.smithi043:> asok_path = "/var/run/ceph/ceph-client.0.*.asok" 2023-08-09T15:11:37.384 DEBUG:teuthology.orchestra.run.smithi043:> files = glob.glob(asok_path) 2023-08-09T15:11:37.385 DEBUG:teuthology.orchestra.run.smithi043:> mountpoint = "/home/ubuntu/cephtest/mnt.0" 2023-08-09T15:11:37.385 DEBUG:teuthology.orchestra.run.smithi043:> 2023-08-09T15:11:37.385 DEBUG:teuthology.orchestra.run.smithi043:> # Given a non-glob path, it better be there 2023-08-09T15:11:37.386 DEBUG:teuthology.orchestra.run.smithi043:> if "*" not in asok_path: 2023-08-09T15:11:37.386 DEBUG:teuthology.orchestra.run.smithi043:> assert(len(files) == 1) 2023-08-09T15:11:37.386 DEBUG:teuthology.orchestra.run.smithi043:> return files[0] 2023-08-09T15:11:37.387 DEBUG:teuthology.orchestra.run.smithi043:> 2023-08-09T15:11:37.387 DEBUG:teuthology.orchestra.run.smithi043:> for f in files: 2023-08-09T15:11:37.387 DEBUG:teuthology.orchestra.run.smithi043:> pid = re.match(".*\.(\d+)\.asok$", f).group(1) 2023-08-09T15:11:37.387 DEBUG:teuthology.orchestra.run.smithi043:> if os.path.exists("/proc/{0}".format(pid)): 2023-08-09T15:11:37.388 DEBUG:teuthology.orchestra.run.smithi043:> with open("/proc/{0}/cmdline".format(pid), '"'"'r'"'"') as proc_f: 2023-08-09T15:11:37.388 DEBUG:teuthology.orchestra.run.smithi043:> contents = proc_f.read() 2023-08-09T15:11:37.388 DEBUG:teuthology.orchestra.run.smithi043:> if mountpoint in contents: 2023-08-09T15:11:37.389 DEBUG:teuthology.orchestra.run.smithi043:> return f 2023-08-09T15:11:37.389 DEBUG:teuthology.orchestra.run.smithi043:> raise RuntimeError("Client socket {0} not found".format(client_name)) 2023-08-09T15:11:37.390 DEBUG:teuthology.orchestra.run.smithi043:> 2023-08-09T15:11:37.390 DEBUG:teuthology.orchestra.run.smithi043:> print(_find_admin_socket("client.0")) 2023-08-09T15:11:37.390 DEBUG:teuthology.orchestra.run.smithi043:> ' 2023-08-09T15:11:37.602 INFO:teuthology.orchestra.run.smithi043.stdout:/var/run/ceph/ceph-client.0.153632.asok 2023-08-09T15:11:37.607 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:37 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2023-08-09T15:11:37.621 INFO:tasks.cephfs.fuse_mount:Found client admin socket at /var/run/ceph/ceph-client.0.153632.asok 2023-08-09T15:11:37.622 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-09T15:11:37.622 DEBUG:teuthology.orchestra.run.smithi043:> sudo ceph --admin-daemon /var/run/ceph/ceph-client.0.153632.asok status 2023-08-09T15:11:37.872 INFO:teuthology.orchestra.run.smithi043.stdout:{ 2023-08-09T15:11:37.872 INFO:teuthology.orchestra.run.smithi043.stdout: "metadata": { 2023-08-09T15:11:37.872 INFO:teuthology.orchestra.run.smithi043.stdout: "ceph_sha1": "668a86f232c9454b2ce5928d1432a29a72964d85", 2023-08-09T15:11:37.873 INFO:teuthology.orchestra.run.smithi043.stdout: "ceph_version": "ceph version 18.0.0-5389-g668a86f2 (668a86f232c9454b2ce5928d1432a29a72964d85) reef (dev)", 2023-08-09T15:11:37.873 INFO:teuthology.orchestra.run.smithi043.stdout: "entity_id": "0", 2023-08-09T15:11:37.873 INFO:teuthology.orchestra.run.smithi043.stdout: "hostname": "smithi043", 2023-08-09T15:11:37.873 INFO:teuthology.orchestra.run.smithi043.stdout: "mount_point": "/home/ubuntu/cephtest/mnt.0", 2023-08-09T15:11:37.873 INFO:teuthology.orchestra.run.smithi043.stdout: "pid": "153632", 2023-08-09T15:11:37.873 INFO:teuthology.orchestra.run.smithi043.stdout: "root": "/" 2023-08-09T15:11:37.873 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2023-08-09T15:11:37.873 INFO:teuthology.orchestra.run.smithi043.stdout: "dentry_count": 0, 2023-08-09T15:11:37.874 INFO:teuthology.orchestra.run.smithi043.stdout: "dentry_pinned_count": 0, 2023-08-09T15:11:37.874 INFO:teuthology.orchestra.run.smithi043.stdout: "id": 11508, 2023-08-09T15:11:37.874 INFO:teuthology.orchestra.run.smithi043.stdout: "inst": { 2023-08-09T15:11:37.874 INFO:teuthology.orchestra.run.smithi043.stdout: "name": { 2023-08-09T15:11:37.874 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "client", 2023-08-09T15:11:37.874 INFO:teuthology.orchestra.run.smithi043.stdout: "num": 11508 2023-08-09T15:11:37.874 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2023-08-09T15:11:37.875 INFO:teuthology.orchestra.run.smithi043.stdout: "addr": { 2023-08-09T15:11:37.875 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "v1", 2023-08-09T15:11:37.875 INFO:teuthology.orchestra.run.smithi043.stdout: "addr": "192.168.0.1:0", 2023-08-09T15:11:37.875 INFO:teuthology.orchestra.run.smithi043.stdout: "nonce": 2148464280 2023-08-09T15:11:37.875 INFO:teuthology.orchestra.run.smithi043.stdout: } 2023-08-09T15:11:37.875 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2023-08-09T15:11:37.875 INFO:teuthology.orchestra.run.smithi043.stdout: "addr": { 2023-08-09T15:11:37.875 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "v1", 2023-08-09T15:11:37.876 INFO:teuthology.orchestra.run.smithi043.stdout: "addr": "192.168.0.1:0", 2023-08-09T15:11:37.876 INFO:teuthology.orchestra.run.smithi043.stdout: "nonce": 2148464280 2023-08-09T15:11:37.876 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2023-08-09T15:11:37.876 INFO:teuthology.orchestra.run.smithi043.stdout: "inst_str": "client.11508 192.168.0.1:0/2148464280", 2023-08-09T15:11:37.876 INFO:teuthology.orchestra.run.smithi043.stdout: "addr_str": "192.168.0.1:0/2148464280", 2023-08-09T15:11:37.876 INFO:teuthology.orchestra.run.smithi043.stdout: "inode_count": 1, 2023-08-09T15:11:37.876 INFO:teuthology.orchestra.run.smithi043.stdout: "mds_epoch": 104, 2023-08-09T15:11:37.877 INFO:teuthology.orchestra.run.smithi043.stdout: "osd_epoch": 223, 2023-08-09T15:11:37.877 INFO:teuthology.orchestra.run.smithi043.stdout: "osd_epoch_barrier": 0, 2023-08-09T15:11:37.877 INFO:teuthology.orchestra.run.smithi043.stdout: "blocklisted": false, 2023-08-09T15:11:37.877 INFO:teuthology.orchestra.run.smithi043.stdout: "fs_name": "cephfs" 2023-08-09T15:11:37.877 INFO:teuthology.orchestra.run.smithi043.stdout:} 2023-08-09T15:11:37.887 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-09T15:11:37.888 DEBUG:teuthology.orchestra.run.smithi043:> stat --file-system '--printf=%T 2023-08-09T15:11:37.888 DEBUG:teuthology.orchestra.run.smithi043:> ' -- /home/ubuntu/cephtest/mnt.0 2023-08-09T15:11:37.947 INFO:teuthology.orchestra.run.smithi043.stdout:fuseblk 2023-08-09T15:11:37.947 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.0 2023-08-09T15:11:37.947 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-09T15:11:37.948 DEBUG:teuthology.orchestra.run.smithi043:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2023-08-09T15:11:38.052 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs volume ls 2023-08-09T15:11:38.287 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:38.286+0000 7f787205e640 1 Processor -- start 2023-08-09T15:11:38.287 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:38.286+0000 7f787205e640 1 -- start start 2023-08-09T15:11:38.288 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:38.288+0000 7f787205e640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f786c143c20 0x7f786c06f540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T15:11:38.289 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:38.288+0000 7f787205e640 1 -- --> v1:172.21.15.43:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f786c06c440 con 0x7f786c0c86f0 2023-08-09T15:11:38.289 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:38.288+0000 7f787205e640 1 -- --> v1:172.21.15.203:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f786c06b830 con 0x7f786c06cfe0 2023-08-09T15:11:38.289 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:38.288+0000 7f787205e640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f786c144000 con 0x7f786c143c20 2023-08-09T15:11:38.290 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:38.288+0000 7f786affd640 1 --1- >> v1:172.21.15.43:6789/0 conn(0x7f786c0c86f0 0x7f786c06c8d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.43:6789/0 says I am v1:172.21.15.43:59210/0 (socket says 172.21.15.43:59210) 2023-08-09T15:11:38.290 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:38.288+0000 7f786affd640 1 -- 172.21.15.43:0/999867255 learned_addr learned my addr 172.21.15.43:0/999867255 (peer_addr_for_me v1:172.21.15.43:0/0) 2023-08-09T15:11:38.290 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:38.289+0000 7f786b7fe640 1 --2- 172.21.15.43:0/999867255 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f786c143c20 0x7f786c06f540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T15:11:38.291 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:38.289+0000 7f786a7fc640 1 -- 172.21.15.43:0/999867255 <== mon.0 v1:172.21.15.43:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1906351772 0 0) 0x7f786c06c440 con 0x7f786c0c86f0 2023-08-09T15:11:38.291 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:38.289+0000 7f786a7fc640 1 -- 172.21.15.43:0/999867255 --> v1:172.21.15.43:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f7854003520 con 0x7f786c0c86f0 2023-08-09T15:11:38.291 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:38.289+0000 7f786a7fc640 1 -- 172.21.15.43:0/999867255 <== mon.0 v1:172.21.15.43:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 4252074277 0 0) 0x7f7854003520 con 0x7f786c0c86f0 2023-08-09T15:11:38.291 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:38.289+0000 7f786a7fc640 1 -- 172.21.15.43:0/999867255 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f786c143c20 msgr2=0x7f786c06f540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-09T15:11:38.291 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:38.289+0000 7f786a7fc640 1 --2- 172.21.15.43:0/999867255 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f786c143c20 0x7f786c06f540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T15:11:38.291 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:38.289+0000 7f786a7fc640 1 -- 172.21.15.43:0/999867255 >> v1:172.21.15.203:6789/0 conn(0x7f786c06cfe0 legacy=0x7f786c0ccc00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T15:11:38.292 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:38.289+0000 7f786a7fc640 1 -- 172.21.15.43:0/999867255 --> v1:172.21.15.43:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f786c06bb10 con 0x7f786c0c86f0 2023-08-09T15:11:38.292 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:38.289+0000 7f786a7fc640 1 -- 172.21.15.43:0/999867255 <== mon.0 v1:172.21.15.43:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1285150740 0 0) 0x7f7860001670 con 0x7f786c0c86f0 2023-08-09T15:11:38.292 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:38.289+0000 7f786a7fc640 1 -- 172.21.15.43:0/999867255 >> v1:172.21.15.43:6789/0 conn(0x7f786c0c86f0 legacy=0x7f786c06c8d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T15:11:38.292 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:38.289+0000 7f786a7fc640 1 --2- 172.21.15.43:0/999867255 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f786c143c20 0x7f7854003a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T15:11:38.292 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:38.289+0000 7f786a7fc640 1 --2- 172.21.15.43:0/999867255 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f7854003f90 0x7f78540047c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T15:11:38.292 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:38.290+0000 7f786a7fc640 1 --2- 172.21.15.43:0/999867255 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f7854004d00 0x7f7854005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T15:11:38.292 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:38.290+0000 7f786a7fc640 1 -- 172.21.15.43:0/999867255 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f786c144000 con 0x7f7854003f90 2023-08-09T15:11:38.293 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:38.290+0000 7f786a7fc640 1 -- 172.21.15.43:0/999867255 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f785400bb70 con 0x7f7854004d00 2023-08-09T15:11:38.293 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:38.290+0000 7f786a7fc640 1 -- 172.21.15.43:0/999867255 --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f785400bd30 con 0x7f786c143c20 2023-08-09T15:11:38.293 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:38.290+0000 7f786affd640 1 --2- 172.21.15.43:0/999867255 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f7854003f90 0x7f78540047c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T15:11:38.294 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:38.290+0000 7f786b7fe640 1 --2- 172.21.15.43:0/999867255 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f786c143c20 0x7f7854003a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T15:11:38.294 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:38.290+0000 7f786affd640 1 -- 172.21.15.43:0/999867255 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f786c143c20 msgr2=0x7f7854003a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-09T15:11:38.295 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:38.290+0000 7f786bfff640 1 --2- 172.21.15.43:0/999867255 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f7854004d00 0x7f7854005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T15:11:38.295 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:38.290+0000 7f786affd640 1 --2- 172.21.15.43:0/999867255 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f786c143c20 0x7f7854003a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T15:11:38.295 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:38.290+0000 7f786affd640 1 -- 172.21.15.43:0/999867255 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f7854004d00 msgr2=0x7f7854005180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-09T15:11:38.295 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:38.290+0000 7f786affd640 1 --2- 172.21.15.43:0/999867255 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f7854004d00 0x7f7854005180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T15:11:38.295 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:38.290+0000 7f786affd640 1 -- 172.21.15.43:0/999867255 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7854003520 con 0x7f7854003f90 2023-08-09T15:11:38.295 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:38.290+0000 7f786b7fe640 1 --2- 172.21.15.43:0/999867255 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f786c143c20 0x7f7854003a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-08-09T15:11:38.295 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:38.291+0000 7f786affd640 1 --2- 172.21.15.43:0/999867255 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f7854003f90 0x7f78540047c0 secure :-1 s=READY pgs=2431 cs=0 l=1 rev1=1 crypto rx=0x7f786000c240 tx=0x7f78540057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ab6b259661379e07 server_cookie=0 in_seq=0 out_seq=0 2023-08-09T15:11:38.296 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:38.292+0000 7f786a7fc640 1 -- 172.21.15.43:0/999867255 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7860005ad0 con 0x7f7854003f90 2023-08-09T15:11:38.296 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:38.292+0000 7f786a7fc640 1 -- 172.21.15.43:0/999867255 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7860005c50 con 0x7f7854003f90 2023-08-09T15:11:38.296 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:38.292+0000 7f786a7fc640 1 -- 172.21.15.43:0/999867255 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f786002c8b0 con 0x7f7854003f90 2023-08-09T15:11:38.296 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:38.292+0000 7f787205e640 1 -- 172.21.15.43:0/999867255 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f7854003f90 msgr2=0x7f78540047c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T15:11:38.296 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:38.292+0000 7f787205e640 1 --2- 172.21.15.43:0/999867255 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f7854003f90 0x7f78540047c0 secure :-1 s=READY pgs=2431 cs=0 l=1 rev1=1 crypto rx=0x7f786000c240 tx=0x7f78540057f0 comp rx=0 tx=0).stop 2023-08-09T15:11:38.296 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:38.292+0000 7f786bfff640 1 -- 172.21.15.43:0/999867255 reap_dead start 2023-08-09T15:11:38.296 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:38.293+0000 7f787205e640 1 -- 172.21.15.43:0/999867255 shutdown_connections 2023-08-09T15:11:38.297 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:38.293+0000 7f787205e640 1 -- 172.21.15.43:0/999867255 >> 172.21.15.43:0/999867255 conn(0x7f786c0ce0a0 msgr2=0x7f786c0ce480 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-09T15:11:38.297 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:38.293+0000 7f787205e640 1 -- 172.21.15.43:0/999867255 shutdown_connections 2023-08-09T15:11:38.297 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:38.293+0000 7f787205e640 1 -- 172.21.15.43:0/999867255 wait complete. 2023-08-09T15:11:38.299 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:38.294+0000 7f787205e640 1 Processor -- start 2023-08-09T15:11:38.299 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:38.295+0000 7f787205e640 1 -- start start 2023-08-09T15:11:38.299 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:38.296+0000 7f787205e640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f7854003f90 0x7f786c15ca80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T15:11:38.299 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:38.296+0000 7f787205e640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f786c13d2d0 0x7f786c0b2990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T15:11:38.299 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:38.296+0000 7f787205e640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f786c141e40 0x7f786c1422c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T15:11:38.299 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:38.296+0000 7f787205e640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f786c13ff10 con 0x7f786c141e40 2023-08-09T15:11:38.299 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:38.296+0000 7f787205e640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f786c142800 con 0x7f786c13d2d0 2023-08-09T15:11:38.300 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:38.296+0000 7f787205e640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f786c15cfc0 con 0x7f7854003f90 2023-08-09T15:11:38.300 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:38.296+0000 7f786bfff640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f786c141e40 0x7f786c1422c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T15:11:38.300 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:38.296+0000 7f786bfff640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f786c141e40 0x7f786c1422c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:43624/0 (socket says 172.21.15.43:43624) 2023-08-09T15:11:38.300 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:38.296+0000 7f786bfff640 1 -- 172.21.15.43:0/3984286125 learned_addr learned my addr 172.21.15.43:0/3984286125 (peer_addr_for_me v2:172.21.15.43:0/0) 2023-08-09T15:11:38.300 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:38.296+0000 7f786b7fe640 1 --2- 172.21.15.43:0/3984286125 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f7854003f90 0x7f786c15ca80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T15:11:38.300 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:38.296+0000 7f786affd640 1 --2- 172.21.15.43:0/3984286125 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f786c13d2d0 0x7f786c0b2990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T15:11:38.300 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:38.296+0000 7f786bfff640 1 -- 172.21.15.43:0/3984286125 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f7854003f90 msgr2=0x7f786c15ca80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T15:11:38.301 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:38.296+0000 7f786bfff640 1 --2- 172.21.15.43:0/3984286125 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f7854003f90 0x7f786c15ca80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T15:11:38.301 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:38.296+0000 7f786bfff640 1 -- 172.21.15.43:0/3984286125 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f786c13d2d0 msgr2=0x7f786c0b2990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T15:11:38.301 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:38.296+0000 7f786bfff640 1 --2- 172.21.15.43:0/3984286125 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f786c13d2d0 0x7f786c0b2990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T15:11:38.301 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:38.296+0000 7f786bfff640 1 -- 172.21.15.43:0/3984286125 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f786c140160 con 0x7f786c141e40 2023-08-09T15:11:38.301 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:38.297+0000 7f786bfff640 1 --2- 172.21.15.43:0/3984286125 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f786c141e40 0x7f786c1422c0 secure :-1 s=READY pgs=2432 cs=0 l=1 rev1=1 crypto rx=0x7f786c06a750 tx=0x7f786400d9d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-09T15:11:38.301 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:38.297+0000 7f7853fff640 1 -- 172.21.15.43:0/3984286125 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7864003d80 con 0x7f786c141e40 2023-08-09T15:11:38.301 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:38.297+0000 7f7853fff640 1 -- 172.21.15.43:0/3984286125 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7864005070 con 0x7f786c141e40 2023-08-09T15:11:38.301 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:38.297+0000 7f7853fff640 1 -- 172.21.15.43:0/3984286125 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7864005440 con 0x7f786c141e40 2023-08-09T15:11:38.302 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:38.297+0000 7f787205e640 1 -- 172.21.15.43:0/3984286125 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f786c140440 con 0x7f786c141e40 2023-08-09T15:11:38.302 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:38.297+0000 7f787205e640 1 -- 172.21.15.43:0/3984286125 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f786c0c6000 con 0x7f786c141e40 2023-08-09T15:11:38.302 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:38.299+0000 7f787205e640 1 -- 172.21.15.43:0/3984286125 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f786c0c5b60 con 0x7f786c141e40 2023-08-09T15:11:38.303 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:38.300+0000 7f7853fff640 1 -- 172.21.15.43:0/3984286125 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 118) v1 ==== 88443+0+0 (secure 0 0 0) 0x7f78640055a0 con 0x7f786c141e40 2023-08-09T15:11:38.303 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:38.301+0000 7f7853fff640 1 --2- 172.21.15.43:0/3984286125 >> [v2:172.21.15.203:6832/4034093268,v1:172.21.15.203:6833/4034093268] conn(0x7f785806a170 0x7f785806c630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T15:11:38.303 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:38.301+0000 7f7853fff640 1 -- 172.21.15.43:0/3984286125 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(223..223 src has 1..223) v4 ==== 6426+0+0 (secure 0 0 0) 0x7f78640882a0 con 0x7f786c141e40 2023-08-09T15:11:38.303 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:38.301+0000 7f786b7fe640 1 --2- 172.21.15.43:0/3984286125 >> [v2:172.21.15.203:6832/4034093268,v1:172.21.15.203:6833/4034093268] conn(0x7f785806a170 0x7f785806c630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T15:11:38.303 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:38.303+0000 7f786b7fe640 1 --2- 172.21.15.43:0/3984286125 >> [v2:172.21.15.203:6832/4034093268,v1:172.21.15.203:6833/4034093268] conn(0x7f785806a170 0x7f785806c630 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f785c009aa0 tx=0x7f785c008040 comp rx=0 tx=0).ready entity=mgr.8833 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-09T15:11:38.305 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:38.305+0000 7f7853fff640 1 -- 172.21.15.43:0/3984286125 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178066 (secure 0 0 0) 0x7f7864053560 con 0x7f786c141e40 2023-08-09T15:11:38.526 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:38.525+0000 7f787205e640 1 -- 172.21.15.43:0/3984286125 --> [v2:172.21.15.203:6832/4034093268,v1:172.21.15.203:6833/4034093268] -- mgr_command(tid 0: {"prefix": "fs volume ls", "target": ["mon-mgr", ""]}) v1 -- 0x7f786c06c0d0 con 0x7f785806a170 2023-08-09T15:11:38.529 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:38.529+0000 7f7853fff640 1 -- 172.21.15.43:0/3984286125 <== mgr.8833 v2:172.21.15.203:6832/4034093268 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+41 (secure 0 0 0) 0x7f785c0097d0 con 0x7f785806a170 2023-08-09T15:11:38.530 INFO:teuthology.orchestra.run.smithi043.stdout:[ 2023-08-09T15:11:38.530 INFO:teuthology.orchestra.run.smithi043.stdout: { 2023-08-09T15:11:38.530 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "cephfs" 2023-08-09T15:11:38.530 INFO:teuthology.orchestra.run.smithi043.stdout: } 2023-08-09T15:11:38.530 INFO:teuthology.orchestra.run.smithi043.stdout:] 2023-08-09T15:11:38.532 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:38.532+0000 7f787205e640 1 -- 172.21.15.43:0/3984286125 >> [v2:172.21.15.203:6832/4034093268,v1:172.21.15.203:6833/4034093268] conn(0x7f785806a170 msgr2=0x7f785806c630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T15:11:38.533 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:38.532+0000 7f787205e640 1 --2- 172.21.15.43:0/3984286125 >> [v2:172.21.15.203:6832/4034093268,v1:172.21.15.203:6833/4034093268] conn(0x7f785806a170 0x7f785806c630 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f785c009aa0 tx=0x7f785c008040 comp rx=0 tx=0).stop 2023-08-09T15:11:38.533 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:38.532+0000 7f787205e640 1 -- 172.21.15.43:0/3984286125 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f786c141e40 msgr2=0x7f786c1422c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T15:11:38.533 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:38.532+0000 7f787205e640 1 --2- 172.21.15.43:0/3984286125 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f786c141e40 0x7f786c1422c0 secure :-1 s=READY pgs=2432 cs=0 l=1 rev1=1 crypto rx=0x7f786c06a750 tx=0x7f786400d9d0 comp rx=0 tx=0).stop 2023-08-09T15:11:38.533 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:38.532+0000 7f787205e640 1 -- 172.21.15.43:0/3984286125 shutdown_connections 2023-08-09T15:11:38.533 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:38.532+0000 7f787205e640 1 --2- 172.21.15.43:0/3984286125 >> [v2:172.21.15.203:6832/4034093268,v1:172.21.15.203:6833/4034093268] conn(0x7f785806a170 0x7f785806c630 unknown :-1 s=CLOSED pgs=70 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T15:11:38.533 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:38.532+0000 7f787205e640 1 --2- 172.21.15.43:0/3984286125 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f786c141e40 0x7f786c1422c0 unknown :-1 s=CLOSED pgs=2432 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T15:11:38.533 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:38.532+0000 7f787205e640 1 --2- 172.21.15.43:0/3984286125 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f786c13d2d0 0x7f786c0b2990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T15:11:38.534 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:38.532+0000 7f787205e640 1 --2- 172.21.15.43:0/3984286125 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f7854003f90 0x7f786c15ca80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T15:11:38.534 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:38.532+0000 7f787205e640 1 -- 172.21.15.43:0/3984286125 >> 172.21.15.43:0/3984286125 conn(0x7f786c0ce0a0 msgr2=0x7f786c0cc370 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-09T15:11:38.534 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:38.532+0000 7f787205e640 1 -- 172.21.15.43:0/3984286125 shutdown_connections 2023-08-09T15:11:38.534 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:38.532+0000 7f787205e640 1 -- 172.21.15.43:0/3984286125 wait complete. 2023-08-09T15:11:38.553 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph mgr module enable snap_schedule 2023-08-09T15:11:38.786 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:38.785+0000 7f8589e34640 1 Processor -- start 2023-08-09T15:11:38.786 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:38.786+0000 7f8589e34640 1 -- start start 2023-08-09T15:11:38.788 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:38.787+0000 7f8589e34640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f858415c680 0x7f85840ce760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T15:11:38.788 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:38.787+0000 7f8589e34640 1 -- --> v1:172.21.15.43:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f85840cdf00 con 0x7f85840d0a70 2023-08-09T15:11:38.788 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:38.787+0000 7f8589e34640 1 -- --> v1:172.21.15.203:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f85840d0f70 con 0x7f85840b3fa0 2023-08-09T15:11:38.788 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:38.787+0000 7f8589e34640 1 -- --> v2:172.21.15.43:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f85840ccd10 con 0x7f858415c680 2023-08-09T15:11:38.788 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:38.787+0000 7f85837fe640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f858415c680 0x7f85840ce760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T15:11:38.788 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:38.788+0000 7f8582ffd640 1 --1- >> v1:172.21.15.43:6789/0 conn(0x7f85840d0a70 0x7f85840cd7f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.43:6789/0 says I am v1:172.21.15.43:59222/0 (socket says 172.21.15.43:59222) 2023-08-09T15:11:38.789 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:38.788+0000 7f8582ffd640 1 -- 172.21.15.43:0/253390393 learned_addr learned my addr 172.21.15.43:0/253390393 (peer_addr_for_me v1:172.21.15.43:0/0) 2023-08-09T15:11:38.789 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:38.788+0000 7f85837fe640 1 --2- 172.21.15.43:0/253390393 >> v2:172.21.15.43:3300/0 conn(0x7f858415c680 0x7f85840ce760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:43632/0 (socket says 172.21.15.43:43632) 2023-08-09T15:11:38.789 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:38.788+0000 7f85837fe640 1 -- 172.21.15.43:0/253390393 >> v1:172.21.15.203:6789/0 conn(0x7f85840b3fa0 legacy=0x7f85840b43b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-09T15:11:38.789 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:38.788+0000 7f85837fe640 1 -- 172.21.15.43:0/253390393 >> v1:172.21.15.43:6789/0 conn(0x7f85840d0a70 legacy=0x7f85840cd7f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T15:11:38.790 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:38.788+0000 7f85837fe640 1 -- 172.21.15.43:0/253390393 --> v2:172.21.15.43:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f85840ccff0 con 0x7f858415c680 2023-08-09T15:11:38.790 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:38.789+0000 7f85837fe640 1 --2- 172.21.15.43:0/253390393 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f858415c680 0x7f85840ce760 secure :-1 s=READY pgs=2434 cs=0 l=1 rev1=1 crypto rx=0x7f85740097a0 tx=0x7f857402b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d79230849f194169 server_cookie=0 in_seq=0 out_seq=0 2023-08-09T15:11:38.791 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:38.789+0000 7f85827fc640 1 -- 172.21.15.43:0/253390393 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8574031830 con 0x7f858415c680 2023-08-09T15:11:38.791 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:38.789+0000 7f85827fc640 1 -- 172.21.15.43:0/253390393 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8574031990 con 0x7f858415c680 2023-08-09T15:11:38.791 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:38.789+0000 7f85827fc640 1 -- 172.21.15.43:0/253390393 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8574031d20 con 0x7f858415c680 2023-08-09T15:11:38.791 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:38.790+0000 7f8589e34640 1 -- 172.21.15.43:0/253390393 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f858415c680 msgr2=0x7f85840ce760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T15:11:38.791 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:38.790+0000 7f8589e34640 1 --2- 172.21.15.43:0/253390393 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f858415c680 0x7f85840ce760 secure :-1 s=READY pgs=2434 cs=0 l=1 rev1=1 crypto rx=0x7f85740097a0 tx=0x7f857402b8d0 comp rx=0 tx=0).stop 2023-08-09T15:11:38.792 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:38.790+0000 7f8589e34640 1 -- 172.21.15.43:0/253390393 shutdown_connections 2023-08-09T15:11:38.792 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:38.790+0000 7f8589e34640 1 --2- 172.21.15.43:0/253390393 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f858415c680 0x7f85840ce760 unknown :-1 s=CLOSED pgs=2434 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T15:11:38.792 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:38.790+0000 7f8589e34640 1 -- 172.21.15.43:0/253390393 >> 172.21.15.43:0/253390393 conn(0x7f8584071520 msgr2=0x7f85840d2520 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-09T15:11:38.792 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:38.790+0000 7f8589e34640 1 -- 172.21.15.43:0/253390393 shutdown_connections 2023-08-09T15:11:38.792 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:38.790+0000 7f8589e34640 1 -- 172.21.15.43:0/253390393 wait complete. 2023-08-09T15:11:38.793 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:38.793+0000 7f8589e34640 1 Processor -- start 2023-08-09T15:11:38.793 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:38.793+0000 7f8589e34640 1 -- start start 2023-08-09T15:11:38.795 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:38.794+0000 7f8589e34640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f85840b3fa0 0x7f8584154e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T15:11:38.795 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:38.794+0000 7f8589e34640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f85840d0a70 0x7f8584158aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T15:11:38.795 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:38.794+0000 7f8589e34640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f858415c680 0x7f85841f5310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T15:11:38.795 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:38.794+0000 7f8589e34640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8584158fe0 con 0x7f85840b3fa0 2023-08-09T15:11:38.796 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:38.794+0000 7f8589e34640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f85841591d0 con 0x7f85840d0a70 2023-08-09T15:11:38.796 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:38.794+0000 7f8589e34640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f858415a960 con 0x7f858415c680 2023-08-09T15:11:38.796 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:38.795+0000 7f85837fe640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f85840b3fa0 0x7f8584154e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T15:11:38.796 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:38.795+0000 7f85837fe640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f85840b3fa0 0x7f8584154e40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:43634/0 (socket says 172.21.15.43:43634) 2023-08-09T15:11:38.796 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:38.795+0000 7f85837fe640 1 -- 172.21.15.43:0/2121365678 learned_addr learned my addr 172.21.15.43:0/2121365678 (peer_addr_for_me v2:172.21.15.43:0/0) 2023-08-09T15:11:38.797 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:38.795+0000 7f8582ffd640 1 --2- 172.21.15.43:0/2121365678 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f85840d0a70 0x7f8584158aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T15:11:38.797 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:38.795+0000 7f85837fe640 1 -- 172.21.15.43:0/2121365678 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f858415c680 msgr2=0x7f85841f5310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T15:11:38.797 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:38.795+0000 7f85837fe640 1 --2- 172.21.15.43:0/2121365678 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f858415c680 0x7f85841f5310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T15:11:38.797 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:38.795+0000 7f85837fe640 1 -- 172.21.15.43:0/2121365678 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f85840d0a70 msgr2=0x7f8584158aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T15:11:38.798 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:38.795+0000 7f85837fe640 1 --2- 172.21.15.43:0/2121365678 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f85840d0a70 0x7f8584158aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T15:11:38.798 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:38.795+0000 7f85837fe640 1 -- 172.21.15.43:0/2121365678 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8574009400 con 0x7f85840b3fa0 2023-08-09T15:11:38.798 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:38.795+0000 7f8582ffd640 1 --2- 172.21.15.43:0/2121365678 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f85840d0a70 0x7f8584158aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-08-09T15:11:38.798 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:38.796+0000 7f85837fe640 1 --2- 172.21.15.43:0/2121365678 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f85840b3fa0 0x7f8584154e40 secure :-1 s=READY pgs=2435 cs=0 l=1 rev1=1 crypto rx=0x7f85740097a0 tx=0x7f85740046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-09T15:11:38.798 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:38.796+0000 7f856bfff640 1 -- 172.21.15.43:0/2121365678 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f857403aa60 con 0x7f85840b3fa0 2023-08-09T15:11:38.798 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:38.796+0000 7f856bfff640 1 -- 172.21.15.43:0/2121365678 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f857403abc0 con 0x7f85840b3fa0 2023-08-09T15:11:38.798 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:38.796+0000 7f856bfff640 1 -- 172.21.15.43:0/2121365678 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f85740443f0 con 0x7f85840b3fa0 2023-08-09T15:11:38.799 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:38.796+0000 7f8589e34640 1 -- 172.21.15.43:0/2121365678 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f858415ac40 con 0x7f85840b3fa0 2023-08-09T15:11:38.799 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:38.796+0000 7f8589e34640 1 -- 172.21.15.43:0/2121365678 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f858415b200 con 0x7f85840b3fa0 2023-08-09T15:11:38.799 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:38.797+0000 7f8589e34640 1 -- 172.21.15.43:0/2121365678 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8584065fd0 con 0x7f85840b3fa0 2023-08-09T15:11:38.799 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:38.798+0000 7f856bfff640 1 -- 172.21.15.43:0/2121365678 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 118) v1 ==== 88443+0+0 (secure 0 0 0) 0x7f857404d450 con 0x7f85840b3fa0 2023-08-09T15:11:38.800 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:38.800+0000 7f856bfff640 1 --2- 172.21.15.43:0/2121365678 >> [v2:172.21.15.203:6832/4034093268,v1:172.21.15.203:6833/4034093268] conn(0x7f857006a170 0x7f857006c630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T15:11:38.800 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:38.800+0000 7f856bfff640 1 -- 172.21.15.43:0/2121365678 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(223..223 src has 1..223) v4 ==== 6426+0+0 (secure 0 0 0) 0x7f85740aaca0 con 0x7f85840b3fa0 2023-08-09T15:11:38.801 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:38.801+0000 7f8582ffd640 1 --2- 172.21.15.43:0/2121365678 >> [v2:172.21.15.203:6832/4034093268,v1:172.21.15.203:6833/4034093268] conn(0x7f857006a170 0x7f857006c630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T15:11:38.802 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:38.802+0000 7f8582ffd640 1 --2- 172.21.15.43:0/2121365678 >> [v2:172.21.15.203:6832/4034093268,v1:172.21.15.203:6833/4034093268] conn(0x7f857006a170 0x7f857006c630 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f85840cb680 tx=0x7f8578009040 comp rx=0 tx=0).ready entity=mgr.8833 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-09T15:11:38.804 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:38.804+0000 7f856bfff640 1 -- 172.21.15.43:0/2121365678 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178066 (secure 0 0 0) 0x7f8574075f60 con 0x7f85840b3fa0 2023-08-09T15:11:39.048 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:39.047+0000 7f8589e34640 1 -- 172.21.15.43:0/2121365678 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "mgr module enable", "module": "snap_schedule"} v 0) v1 -- 0x7f858415b3f0 con 0x7f85840b3fa0 2023-08-09T15:11:39.143 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:39.142+0000 7f856bfff640 1 -- 172.21.15.43:0/2121365678 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mgrmap(e 119) v1 ==== 88460+0+0 (secure 0 0 0) 0x7f85740aa030 con 0x7f85840b3fa0 2023-08-09T15:11:39.145 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:39.145+0000 7f856bfff640 1 -- 172.21.15.43:0/2121365678 <== mon.0 v2:172.21.15.43:3300/0 8 ==== mon_command_ack([{"prefix": "mgr module enable", "module": "snap_schedule"}]=0 v119) v1 ==== 92+0+0 (secure 0 0 0) 0x7f85740a2d80 con 0x7f85840b3fa0 2023-08-09T15:11:39.149 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:39.148+0000 7f8589e34640 1 -- 172.21.15.43:0/2121365678 >> [v2:172.21.15.203:6832/4034093268,v1:172.21.15.203:6833/4034093268] conn(0x7f857006a170 msgr2=0x7f857006c630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T15:11:39.149 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:39.148+0000 7f8589e34640 1 --2- 172.21.15.43:0/2121365678 >> [v2:172.21.15.203:6832/4034093268,v1:172.21.15.203:6833/4034093268] conn(0x7f857006a170 0x7f857006c630 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f85840cb680 tx=0x7f8578009040 comp rx=0 tx=0).stop 2023-08-09T15:11:39.149 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:39.148+0000 7f8589e34640 1 -- 172.21.15.43:0/2121365678 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f85840b3fa0 msgr2=0x7f8584154e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T15:11:39.150 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:39.148+0000 7f8589e34640 1 --2- 172.21.15.43:0/2121365678 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f85840b3fa0 0x7f8584154e40 secure :-1 s=READY pgs=2435 cs=0 l=1 rev1=1 crypto rx=0x7f85740097a0 tx=0x7f85740046c0 comp rx=0 tx=0).stop 2023-08-09T15:11:39.150 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:39.149+0000 7f8589e34640 1 -- 172.21.15.43:0/2121365678 shutdown_connections 2023-08-09T15:11:39.150 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:39.149+0000 7f8589e34640 1 --2- 172.21.15.43:0/2121365678 >> [v2:172.21.15.203:6832/4034093268,v1:172.21.15.203:6833/4034093268] conn(0x7f857006a170 0x7f857006c630 unknown :-1 s=CLOSED pgs=71 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T15:11:39.150 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:39.149+0000 7f8589e34640 1 --2- 172.21.15.43:0/2121365678 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f858415c680 0x7f85841f5310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T15:11:39.150 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:39.149+0000 7f8589e34640 1 --2- 172.21.15.43:0/2121365678 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f85840d0a70 0x7f8584158aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T15:11:39.151 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:39.149+0000 7f8589e34640 1 --2- 172.21.15.43:0/2121365678 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f85840b3fa0 0x7f8584154e40 unknown :-1 s=CLOSED pgs=2435 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T15:11:39.151 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:39.149+0000 7f8589e34640 1 -- 172.21.15.43:0/2121365678 >> 172.21.15.43:0/2121365678 conn(0x7f8584071520 msgr2=0x7f85840cefa0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-09T15:11:39.152 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:39.149+0000 7f8589e34640 1 -- 172.21.15.43:0/2121365678 shutdown_connections 2023-08-09T15:11:39.152 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:39.149+0000 7f8589e34640 1 -- 172.21.15.43:0/2121365678 wait complete. 2023-08-09T15:11:39.171 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mgr mgr/snap_schedule/allow_m_granularity True 2023-08-09T15:11:39.357 INFO:tasks.ceph.mgr.x.smithi203.stderr:2023-08-09T15:11:39.355+0000 7fb918e11040 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2023-08-09T15:11:39.390 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:39.389+0000 7ff3f5d37640 1 Processor -- start 2023-08-09T15:11:39.390 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:39.390+0000 7ff3f5d37640 1 -- start start 2023-08-09T15:11:39.391 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:39.391+0000 7ff3f5d37640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7ff3f006cd00 0x7ff3f00c9600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T15:11:39.392 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:39.391+0000 7ff3f5d37640 1 -- --> v1:172.21.15.43:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff3f00cd580 con 0x7ff3f00697c0 2023-08-09T15:11:39.392 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:39.391+0000 7ff3f5d37640 1 -- --> v1:172.21.15.203:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff3f00ccfc0 con 0x7ff3f0070370 2023-08-09T15:11:39.392 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:39.391+0000 7ff3f5d37640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff3f0071050 con 0x7ff3f006cd00 2023-08-09T15:11:39.392 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:39.392+0000 7ff3f4d35640 1 --1- >> v1:172.21.15.43:6789/0 conn(0x7ff3f00697c0 0x7ff3f0069ba0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.43:6789/0 says I am v1:172.21.15.43:59234/0 (socket says 172.21.15.43:59234) 2023-08-09T15:11:39.392 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:39.392+0000 7ff3f4d35640 1 -- 172.21.15.43:0/273413667 learned_addr learned my addr 172.21.15.43:0/273413667 (peer_addr_for_me v1:172.21.15.43:0/0) 2023-08-09T15:11:39.393 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:39.392+0000 7ff3effff640 1 --2- 172.21.15.43:0/273413667 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7ff3f006cd00 0x7ff3f00c9600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T15:11:39.393 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:39.392+0000 7ff3ef7fe640 1 -- 172.21.15.43:0/273413667 <== mon.0 v1:172.21.15.43:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2793204840 0 0) 0x7ff3e4001110 con 0x7ff3f00697c0 2023-08-09T15:11:39.393 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:39.392+0000 7ff3ef7fe640 1 -- 172.21.15.43:0/273413667 --> v1:172.21.15.43:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ff3e0003520 con 0x7ff3f00697c0 2023-08-09T15:11:39.394 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:39.393+0000 7ff3effff640 1 -- 172.21.15.43:0/273413667 >> v1:172.21.15.203:6789/0 conn(0x7ff3f0070370 legacy=0x7ff3f01600c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T15:11:39.394 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:39.393+0000 7ff3effff640 1 -- 172.21.15.43:0/273413667 >> v1:172.21.15.43:6789/0 conn(0x7ff3f00697c0 legacy=0x7ff3f0069ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T15:11:39.394 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:39.393+0000 7ff3effff640 1 -- 172.21.15.43:0/273413667 --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff3f00cd2a0 con 0x7ff3f006cd00 2023-08-09T15:11:39.394 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:39.394+0000 7ff3effff640 1 --2- 172.21.15.43:0/273413667 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7ff3f006cd00 0x7ff3f00c9600 secure :-1 s=READY pgs=339 cs=0 l=1 rev1=1 crypto rx=0x7ff3dc0097d0 tx=0x7ff3dc02b940 comp rx=0 tx=0).ready entity=mon.2 client_cookie=6662c3ae22b04b90 server_cookie=0 in_seq=0 out_seq=0 2023-08-09T15:11:39.395 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:39.394+0000 7ff3ef7fe640 1 -- 172.21.15.43:0/273413667 <== mon.2 v2:172.21.15.203:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff3dc030800 con 0x7ff3f006cd00 2023-08-09T15:11:39.395 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:39.395+0000 7ff3ef7fe640 1 -- 172.21.15.43:0/273413667 <== mon.2 v2:172.21.15.203:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff3dc030960 con 0x7ff3f006cd00 2023-08-09T15:11:39.395 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:39.395+0000 7ff3ef7fe640 1 -- 172.21.15.43:0/273413667 <== mon.2 v2:172.21.15.203:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff3dc030cd0 con 0x7ff3f006cd00 2023-08-09T15:11:39.396 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:39.395+0000 7ff3f5d37640 1 -- 172.21.15.43:0/273413667 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7ff3f006cd00 msgr2=0x7ff3f00c9600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T15:11:39.396 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:39.395+0000 7ff3f5d37640 1 --2- 172.21.15.43:0/273413667 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7ff3f006cd00 0x7ff3f00c9600 secure :-1 s=READY pgs=339 cs=0 l=1 rev1=1 crypto rx=0x7ff3dc0097d0 tx=0x7ff3dc02b940 comp rx=0 tx=0).stop 2023-08-09T15:11:39.396 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:39.395+0000 7ff3f5d37640 1 -- 172.21.15.43:0/273413667 shutdown_connections 2023-08-09T15:11:39.396 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:39.395+0000 7ff3f5d37640 1 --2- 172.21.15.43:0/273413667 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7ff3f006cd00 0x7ff3f00c9600 unknown :-1 s=CLOSED pgs=339 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T15:11:39.396 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:39.395+0000 7ff3f5d37640 1 -- 172.21.15.43:0/273413667 >> 172.21.15.43:0/273413667 conn(0x7ff3f00cba30 msgr2=0x7ff3f00cbe40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-09T15:11:39.397 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:39.395+0000 7ff3f5d37640 1 -- 172.21.15.43:0/273413667 shutdown_connections 2023-08-09T15:11:39.397 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:39.396+0000 7ff3f5d37640 1 -- 172.21.15.43:0/273413667 wait complete. 2023-08-09T15:11:39.397 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:39.397+0000 7ff3f5d37640 1 Processor -- start 2023-08-09T15:11:39.398 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:39.397+0000 7ff3f5d37640 1 -- start start 2023-08-09T15:11:39.398 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:39.398+0000 7ff3f5d37640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff3f00697c0 0x7ff3f00c57c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T15:11:39.399 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:39.398+0000 7ff3f5d37640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7ff3f006cd00 0x7ff3f015ca50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T15:11:39.399 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:39.398+0000 7ff3f5d37640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7ff3f0070370 0x7ff3f013f1c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T15:11:39.399 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:39.398+0000 7ff3f5d37640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff3f015f5e0 con 0x7ff3f00697c0 2023-08-09T15:11:39.399 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:39.398+0000 7ff3f5d37640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff3f015e9d0 con 0x7ff3f0070370 2023-08-09T15:11:39.399 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:39.398+0000 7ff3f5d37640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff3f015ecb0 con 0x7ff3f006cd00 2023-08-09T15:11:39.401 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:39.399+0000 7ff3f4d35640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff3f00697c0 0x7ff3f00c57c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T15:11:39.401 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:39.399+0000 7ff3f4d35640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff3f00697c0 0x7ff3f00c57c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:43662/0 (socket says 172.21.15.43:43662) 2023-08-09T15:11:39.401 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:39.399+0000 7ff3f4d35640 1 -- 172.21.15.43:0/2556483092 learned_addr learned my addr 172.21.15.43:0/2556483092 (peer_addr_for_me v2:172.21.15.43:0/0) 2023-08-09T15:11:39.401 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:39.399+0000 7ff3f5536640 1 --2- 172.21.15.43:0/2556483092 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7ff3f0070370 0x7ff3f013f1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T15:11:39.401 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:39.399+0000 7ff3effff640 1 --2- 172.21.15.43:0/2556483092 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7ff3f006cd00 0x7ff3f015ca50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T15:11:39.401 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:39.399+0000 7ff3f4d35640 1 -- 172.21.15.43:0/2556483092 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7ff3f006cd00 msgr2=0x7ff3f015ca50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T15:11:39.402 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:39.399+0000 7ff3f4d35640 1 --2- 172.21.15.43:0/2556483092 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7ff3f006cd00 0x7ff3f015ca50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T15:11:39.402 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:39.399+0000 7ff3f4d35640 1 -- 172.21.15.43:0/2556483092 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7ff3f0070370 msgr2=0x7ff3f013f1c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T15:11:39.402 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:39.399+0000 7ff3f4d35640 1 --2- 172.21.15.43:0/2556483092 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7ff3f0070370 0x7ff3f013f1c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T15:11:39.402 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:39.399+0000 7ff3f4d35640 1 -- 172.21.15.43:0/2556483092 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff3dc009400 con 0x7ff3f00697c0 2023-08-09T15:11:39.402 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:39.400+0000 7ff3f4d35640 1 --2- 172.21.15.43:0/2556483092 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff3f00697c0 0x7ff3f00c57c0 secure :-1 s=READY pgs=2439 cs=0 l=1 rev1=1 crypto rx=0x7ff3e400c420 tx=0x7ff3e400c8e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-09T15:11:39.402 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:39.400+0000 7ff3ed7fa640 1 -- 172.21.15.43:0/2556483092 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff3e4005ce0 con 0x7ff3f00697c0 2023-08-09T15:11:39.402 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:39.400+0000 7ff3ed7fa640 1 -- 172.21.15.43:0/2556483092 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff3e4005e40 con 0x7ff3f00697c0 2023-08-09T15:11:39.402 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:39.400+0000 7ff3f5d37640 1 -- 172.21.15.43:0/2556483092 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff3f015ef90 con 0x7ff3f00697c0 2023-08-09T15:11:39.403 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:39.400+0000 7ff3ed7fa640 1 -- 172.21.15.43:0/2556483092 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff3e40072f0 con 0x7ff3f00697c0 2023-08-09T15:11:39.403 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:39.400+0000 7ff3f5d37640 1 -- 172.21.15.43:0/2556483092 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff3f00cb0d0 con 0x7ff3f00697c0 2023-08-09T15:11:39.403 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:39.402+0000 7ff3f5d37640 1 -- 172.21.15.43:0/2556483092 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff3f015f750 con 0x7ff3f00697c0 2023-08-09T15:11:39.404 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:39.403+0000 7ff3ed7fa640 1 -- 172.21.15.43:0/2556483092 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 119) v1 ==== 88460+0+0 (secure 0 0 0) 0x7ff3e4007510 con 0x7ff3f00697c0 2023-08-09T15:11:39.405 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:39.404+0000 7ff3ed7fa640 1 --2- 172.21.15.43:0/2556483092 >> [v2:172.21.15.203:6832/4034093268,v1:172.21.15.203:6833/4034093268] conn(0x7ff3d406a1c0 0x7ff3d406c680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T15:11:39.405 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:39.404+0000 7ff3ed7fa640 1 -- 172.21.15.43:0/2556483092 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(223..223 src has 1..223) v4 ==== 6426+0+0 (secure 0 0 0) 0x7ff3e4087cd0 con 0x7ff3f00697c0 2023-08-09T15:11:39.406 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:39.404+0000 7ff3effff640 1 -- 172.21.15.43:0/2556483092 >> [v2:172.21.15.203:6832/4034093268,v1:172.21.15.203:6833/4034093268] conn(0x7ff3d406a1c0 msgr2=0x7ff3d406c680 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.203:6832/4034093268 2023-08-09T15:11:39.406 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:39.404+0000 7ff3effff640 1 --2- 172.21.15.43:0/2556483092 >> [v2:172.21.15.203:6832/4034093268,v1:172.21.15.203:6833/4034093268] conn(0x7ff3d406a1c0 0x7ff3d406c680 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-08-09T15:11:39.407 INFO:tasks.ceph.mgr.y.smithi043.stderr:2023-08-09T15:11:39.407+0000 7f4daec47040 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2023-08-09T15:11:39.413 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:39.413+0000 7ff3ed7fa640 1 -- 172.21.15.43:0/2556483092 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178066 (secure 0 0 0) 0x7ff3e40531f0 con 0x7ff3f00697c0 2023-08-09T15:11:39.605 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:39.605+0000 7ff3effff640 1 -- 172.21.15.43:0/2556483092 >> [v2:172.21.15.203:6832/4034093268,v1:172.21.15.203:6833/4034093268] conn(0x7ff3d406a1c0 msgr2=0x7ff3d406c680 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.203:6832/4034093268 2023-08-09T15:11:39.606 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:39.605+0000 7ff3effff640 1 --2- 172.21.15.43:0/2556483092 >> [v2:172.21.15.203:6832/4034093268,v1:172.21.15.203:6833/4034093268] conn(0x7ff3d406a1c0 0x7ff3d406c680 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2023-08-09T15:11:39.610 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:39.610+0000 7ff3f5d37640 1 -- 172.21.15.43:0/2556483092 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command([{prefix=config set, name=mgr/snap_schedule/allow_m_granularity}] v 0) v1 -- 0x7ff3f00cd580 con 0x7ff3f00697c0 2023-08-09T15:11:39.616 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:39.616+0000 7ff3ed7fa640 1 -- 172.21.15.43:0/2556483092 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mgr/snap_schedule/allow_m_granularity}]=0 v71)=0 v71) v1 ==== 138+0+0 (secure 0 0 0) 0x7ff3f00cd580 con 0x7ff3f00697c0 2023-08-09T15:11:39.618 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:39.618+0000 7ff3f5d37640 1 -- 172.21.15.43:0/2556483092 >> [v2:172.21.15.203:6832/4034093268,v1:172.21.15.203:6833/4034093268] conn(0x7ff3d406a1c0 msgr2=0x7ff3d406c680 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-08-09T15:11:39.619 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:39.618+0000 7ff3f5d37640 1 --2- 172.21.15.43:0/2556483092 >> [v2:172.21.15.203:6832/4034093268,v1:172.21.15.203:6833/4034093268] conn(0x7ff3d406a1c0 0x7ff3d406c680 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T15:11:39.619 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:39.618+0000 7ff3f5d37640 1 -- 172.21.15.43:0/2556483092 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff3f00697c0 msgr2=0x7ff3f00c57c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T15:11:39.619 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:39.618+0000 7ff3f5d37640 1 --2- 172.21.15.43:0/2556483092 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff3f00697c0 0x7ff3f00c57c0 secure :-1 s=READY pgs=2439 cs=0 l=1 rev1=1 crypto rx=0x7ff3e400c420 tx=0x7ff3e400c8e0 comp rx=0 tx=0).stop 2023-08-09T15:11:39.620 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:39.618+0000 7ff3f5d37640 1 -- 172.21.15.43:0/2556483092 shutdown_connections 2023-08-09T15:11:39.620 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:39.618+0000 7ff3f5d37640 1 --2- 172.21.15.43:0/2556483092 >> [v2:172.21.15.203:6832/4034093268,v1:172.21.15.203:6833/4034093268] conn(0x7ff3d406a1c0 0x7ff3d406c680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T15:11:39.620 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:39.618+0000 7ff3f5d37640 1 --2- 172.21.15.43:0/2556483092 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7ff3f0070370 0x7ff3f013f1c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T15:11:39.620 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:39.618+0000 7ff3f5d37640 1 --2- 172.21.15.43:0/2556483092 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7ff3f006cd00 0x7ff3f015ca50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T15:11:39.620 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:39.618+0000 7ff3f5d37640 1 --2- 172.21.15.43:0/2556483092 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff3f00697c0 0x7ff3f00c57c0 unknown :-1 s=CLOSED pgs=2439 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T15:11:39.620 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:39.619+0000 7ff3f5d37640 1 -- 172.21.15.43:0/2556483092 >> 172.21.15.43:0/2556483092 conn(0x7ff3f00cba30 msgr2=0x7ff3f00c9c20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-09T15:11:39.621 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:39.619+0000 7ff3f5d37640 1 -- 172.21.15.43:0/2556483092 shutdown_connections 2023-08-09T15:11:39.621 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:39.619+0000 7ff3f5d37640 1 -- 172.21.15.43:0/2556483092 wait complete. 2023-08-09T15:11:39.639 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mgr mgr/snap_schedule/dump_on_update True 2023-08-09T15:11:39.860 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:39.859+0000 7f945a9ee640 1 Processor -- start 2023-08-09T15:11:39.860 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:39.859+0000 7f945a9ee640 1 -- start start 2023-08-09T15:11:39.861 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:39.861+0000 7f945a9ee640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f9454071460 0x7f94540cc4f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T15:11:39.862 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:39.861+0000 7f945a9ee640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f9454143730 0x7f9454070630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T15:11:39.862 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:39.861+0000 7f945a9ee640 1 -- --> v1:172.21.15.203:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f945415de90 con 0x7f9454071840 2023-08-09T15:11:39.862 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:39.861+0000 7f945a9ee640 1 -- --> v2:172.21.15.43:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f945415a740 con 0x7f9454143730 2023-08-09T15:11:39.862 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:39.861+0000 7f945a9ee640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f94540cb680 con 0x7f9454071460 2023-08-09T15:11:39.862 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:39.861+0000 7f944bfff640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f9454143730 0x7f9454070630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T15:11:39.862 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:39.861+0000 7f944bfff640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f9454143730 0x7f9454070630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:43670/0 (socket says 172.21.15.43:43670) 2023-08-09T15:11:39.863 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:39.861+0000 7f944bfff640 1 -- 172.21.15.43:0/3538315064 learned_addr learned my addr 172.21.15.43:0/3538315064 (peer_addr_for_me v2:172.21.15.43:0/0) 2023-08-09T15:11:39.863 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:39.862+0000 7f9453fff640 1 --2- 172.21.15.43:0/3538315064 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f9454071460 0x7f94540cc4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T15:11:39.864 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:39.862+0000 7f944bfff640 1 -- 172.21.15.43:0/3538315064 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f9454071460 msgr2=0x7f94540cc4f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-09T15:11:39.864 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:39.862+0000 7f944bfff640 1 --2- 172.21.15.43:0/3538315064 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f9454071460 0x7f94540cc4f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T15:11:39.864 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:39.862+0000 7f944bfff640 1 -- 172.21.15.43:0/3538315064 >> v1:172.21.15.203:6789/0 conn(0x7f9454071840 legacy=0x7f94540697c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-09T15:11:39.864 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:39.862+0000 7f944bfff640 1 -- 172.21.15.43:0/3538315064 --> v2:172.21.15.43:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f94540c98c0 con 0x7f9454143730 2023-08-09T15:11:39.864 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:39.863+0000 7f944bfff640 1 --2- 172.21.15.43:0/3538315064 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f9454143730 0x7f9454070630 secure :-1 s=READY pgs=2440 cs=0 l=1 rev1=1 crypto rx=0x7f943c009850 tx=0x7f943c02b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2a68ca9b61f8d240 server_cookie=0 in_seq=0 out_seq=0 2023-08-09T15:11:39.864 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:39.863+0000 7f94537fe640 1 -- 172.21.15.43:0/3538315064 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f943c004230 con 0x7f9454143730 2023-08-09T15:11:39.864 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:39.863+0000 7f94537fe640 1 -- 172.21.15.43:0/3538315064 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f943c004390 con 0x7f9454143730 2023-08-09T15:11:39.865 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:39.863+0000 7f94537fe640 1 -- 172.21.15.43:0/3538315064 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f943c031830 con 0x7f9454143730 2023-08-09T15:11:39.865 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:39.864+0000 7f945a9ee640 1 -- 172.21.15.43:0/3538315064 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f9454143730 msgr2=0x7f9454070630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T15:11:39.865 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:39.864+0000 7f945a9ee640 1 --2- 172.21.15.43:0/3538315064 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f9454143730 0x7f9454070630 secure :-1 s=READY pgs=2440 cs=0 l=1 rev1=1 crypto rx=0x7f943c009850 tx=0x7f943c02b8d0 comp rx=0 tx=0).stop 2023-08-09T15:11:39.866 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:39.864+0000 7f945a9ee640 1 -- 172.21.15.43:0/3538315064 shutdown_connections 2023-08-09T15:11:39.866 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:39.864+0000 7f945a9ee640 1 --2- 172.21.15.43:0/3538315064 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f9454143730 0x7f9454070630 unknown :-1 s=CLOSED pgs=2440 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T15:11:39.866 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:39.864+0000 7f945a9ee640 1 --2- 172.21.15.43:0/3538315064 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f9454071460 0x7f94540cc4f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T15:11:39.866 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:39.864+0000 7f945a9ee640 1 -- 172.21.15.43:0/3538315064 >> 172.21.15.43:0/3538315064 conn(0x7f94540b0cc0 msgr2=0x7f94540b10d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-09T15:11:39.866 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:39.864+0000 7f945a9ee640 1 -- 172.21.15.43:0/3538315064 shutdown_connections 2023-08-09T15:11:39.866 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:39.864+0000 7f945a9ee640 1 -- 172.21.15.43:0/3538315064 wait complete. 2023-08-09T15:11:39.867 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:39.866+0000 7f945a9ee640 1 Processor -- start 2023-08-09T15:11:39.867 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:39.867+0000 7f945a9ee640 1 -- start start 2023-08-09T15:11:39.868 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:39.868+0000 7f945a9ee640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f9454071460 0x7f9454158640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T15:11:39.868 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:39.868+0000 7f945a9ee640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f9454071840 0x7f94540c8f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T15:11:39.869 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:39.868+0000 7f945a9ee640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f9454143730 0x7f94540adbc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T15:11:39.869 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:39.868+0000 7f945a9ee640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f94540c8ae0 con 0x7f9454071460 2023-08-09T15:11:39.869 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:39.868+0000 7f945a9ee640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9454158b80 con 0x7f9454071840 2023-08-09T15:11:39.869 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:39.868+0000 7f945a9ee640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f94540c9440 con 0x7f9454143730 2023-08-09T15:11:39.869 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:39.868+0000 7f9453fff640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f9454071460 0x7f9454158640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T15:11:39.869 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:39.868+0000 7f9453fff640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f9454071460 0x7f9454158640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:43680/0 (socket says 172.21.15.43:43680) 2023-08-09T15:11:39.870 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:39.868+0000 7f9453fff640 1 -- 172.21.15.43:0/3542491324 learned_addr learned my addr 172.21.15.43:0/3542491324 (peer_addr_for_me v2:172.21.15.43:0/0) 2023-08-09T15:11:39.871 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:39.868+0000 7f944bfff640 1 --2- 172.21.15.43:0/3542491324 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f9454071840 0x7f94540c8f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T15:11:39.871 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:39.868+0000 7f9453fff640 1 -- 172.21.15.43:0/3542491324 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f9454143730 msgr2=0x7f94540adbc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T15:11:39.871 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:39.868+0000 7f9453fff640 1 --2- 172.21.15.43:0/3542491324 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f9454143730 0x7f94540adbc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T15:11:39.871 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:39.869+0000 7f9453fff640 1 -- 172.21.15.43:0/3542491324 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f9454071840 msgr2=0x7f94540c8f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T15:11:39.872 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:39.869+0000 7f9453fff640 1 --2- 172.21.15.43:0/3542491324 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f9454071840 0x7f94540c8f00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T15:11:39.872 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:39.869+0000 7f9453fff640 1 -- 172.21.15.43:0/3542491324 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f943c009460 con 0x7f9454071460 2023-08-09T15:11:39.872 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:39.869+0000 7f944bfff640 1 --2- 172.21.15.43:0/3542491324 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f9454071840 0x7f94540c8f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-08-09T15:11:39.872 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:39.869+0000 7f9453fff640 1 --2- 172.21.15.43:0/3542491324 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f9454071460 0x7f9454158640 secure :-1 s=READY pgs=2441 cs=0 l=1 rev1=1 crypto rx=0x7f944400d830 tx=0x7f944400dcf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-09T15:11:39.872 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:39.869+0000 7f94517fa640 1 -- 172.21.15.43:0/3542491324 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f944400b660 con 0x7f9454071460 2023-08-09T15:11:39.872 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:39.869+0000 7f94517fa640 1 -- 172.21.15.43:0/3542491324 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f944400b7c0 con 0x7f9454071460 2023-08-09T15:11:39.873 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:39.869+0000 7f94517fa640 1 -- 172.21.15.43:0/3542491324 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f944400bb50 con 0x7f9454071460 2023-08-09T15:11:39.873 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:39.869+0000 7f945a9ee640 1 -- 172.21.15.43:0/3542491324 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f94540c7e60 con 0x7f9454071460 2023-08-09T15:11:39.873 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:39.869+0000 7f945a9ee640 1 -- 172.21.15.43:0/3542491324 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f94540c8420 con 0x7f9454071460 2023-08-09T15:11:39.873 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:39.871+0000 7f945a9ee640 1 -- 172.21.15.43:0/3542491324 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f94540739f0 con 0x7f9454071460 2023-08-09T15:11:39.874 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:39.871+0000 7f94517fa640 1 -- 172.21.15.43:0/3542491324 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 119) v1 ==== 88460+0+0 (secure 0 0 0) 0x7f944401e460 con 0x7f9454071460 2023-08-09T15:11:39.874 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:39.873+0000 7f94517fa640 1 --2- 172.21.15.43:0/3542491324 >> [v2:172.21.15.203:6832/4034093268,v1:172.21.15.203:6833/4034093268] conn(0x7f943406a1c0 0x7f943406c680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T15:11:39.874 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:39.873+0000 7f94517fa640 1 -- 172.21.15.43:0/3542491324 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(223..223 src has 1..223) v4 ==== 6426+0+0 (secure 0 0 0) 0x7f9444088f50 con 0x7f9454071460 2023-08-09T15:11:39.874 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:39.873+0000 7f944bfff640 1 -- 172.21.15.43:0/3542491324 >> [v2:172.21.15.203:6832/4034093268,v1:172.21.15.203:6833/4034093268] conn(0x7f943406a1c0 msgr2=0x7f943406c680 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.203:6832/4034093268 2023-08-09T15:11:39.874 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:39.873+0000 7f944bfff640 1 --2- 172.21.15.43:0/3542491324 >> [v2:172.21.15.203:6832/4034093268,v1:172.21.15.203:6833/4034093268] conn(0x7f943406a1c0 0x7f943406c680 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-08-09T15:11:39.877 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:39.877+0000 7f94517fa640 1 -- 172.21.15.43:0/3542491324 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178066 (secure 0 0 0) 0x7f9444054450 con 0x7f9454071460 2023-08-09T15:11:40.050 INFO:tasks.ceph.mgr.x.smithi203.stderr:2023-08-09T15:11:40.049+0000 7fb918e11040 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2023-08-09T15:11:40.068 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:40.067+0000 7f945a9ee640 1 -- 172.21.15.43:0/3542491324 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command([{prefix=config set, name=mgr/snap_schedule/dump_on_update}] v 0) v1 -- 0x7f94540cab10 con 0x7f9454071460 2023-08-09T15:11:40.073 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:40.073+0000 7f94517fa640 1 -- 172.21.15.43:0/3542491324 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mgr/snap_schedule/dump_on_update}]=0 v72)=0 v72) v1 ==== 133+0+0 (secure 0 0 0) 0x7f9444057a70 con 0x7f9454071460 2023-08-09T15:11:40.074 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:40.074+0000 7f944bfff640 1 -- 172.21.15.43:0/3542491324 >> [v2:172.21.15.203:6832/4034093268,v1:172.21.15.203:6833/4034093268] conn(0x7f943406a1c0 msgr2=0x7f943406c680 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.203:6832/4034093268 2023-08-09T15:11:40.074 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:40.074+0000 7f944bfff640 1 --2- 172.21.15.43:0/3542491324 >> [v2:172.21.15.203:6832/4034093268,v1:172.21.15.203:6833/4034093268] conn(0x7f943406a1c0 0x7f943406c680 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2023-08-09T15:11:40.077 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:40.077+0000 7f945a9ee640 1 -- 172.21.15.43:0/3542491324 >> [v2:172.21.15.203:6832/4034093268,v1:172.21.15.203:6833/4034093268] conn(0x7f943406a1c0 msgr2=0x7f943406c680 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-08-09T15:11:40.078 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:40.077+0000 7f945a9ee640 1 --2- 172.21.15.43:0/3542491324 >> [v2:172.21.15.203:6832/4034093268,v1:172.21.15.203:6833/4034093268] conn(0x7f943406a1c0 0x7f943406c680 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T15:11:40.078 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:40.077+0000 7f945a9ee640 1 -- 172.21.15.43:0/3542491324 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f9454071460 msgr2=0x7f9454158640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T15:11:40.078 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:40.077+0000 7f945a9ee640 1 --2- 172.21.15.43:0/3542491324 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f9454071460 0x7f9454158640 secure :-1 s=READY pgs=2441 cs=0 l=1 rev1=1 crypto rx=0x7f944400d830 tx=0x7f944400dcf0 comp rx=0 tx=0).stop 2023-08-09T15:11:40.078 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:40.077+0000 7f945a9ee640 1 -- 172.21.15.43:0/3542491324 shutdown_connections 2023-08-09T15:11:40.078 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:40.077+0000 7f945a9ee640 1 --2- 172.21.15.43:0/3542491324 >> [v2:172.21.15.203:6832/4034093268,v1:172.21.15.203:6833/4034093268] conn(0x7f943406a1c0 0x7f943406c680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T15:11:40.078 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:40.077+0000 7f945a9ee640 1 --2- 172.21.15.43:0/3542491324 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f9454143730 0x7f94540adbc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T15:11:40.078 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:40.077+0000 7f945a9ee640 1 --2- 172.21.15.43:0/3542491324 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f9454071840 0x7f94540c8f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T15:11:40.079 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:40.077+0000 7f945a9ee640 1 --2- 172.21.15.43:0/3542491324 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f9454071460 0x7f9454158640 unknown :-1 s=CLOSED pgs=2441 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T15:11:40.079 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:40.077+0000 7f945a9ee640 1 -- 172.21.15.43:0/3542491324 >> 172.21.15.43:0/3542491324 conn(0x7f94540b0cc0 msgr2=0x7f9454159ae0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-09T15:11:40.079 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:40.077+0000 7f945a9ee640 1 -- 172.21.15.43:0/3542491324 shutdown_connections 2023-08-09T15:11:40.079 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:40.077+0000 7f945a9ee640 1 -- 172.21.15.43:0/3542491324 wait complete. 2023-08-09T15:11:40.103 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-09T15:11:40.103 DEBUG:teuthology.orchestra.run.smithi043:> (cd /home/ubuntu/cephtest/mnt.0 && exec mkdir -p snap_test_dir1) 2023-08-09T15:11:40.169 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs snap-schedule add --fs cephfs snap_test_dir1 1M 2023-08-09T15:11:40.172 INFO:tasks.ceph.mgr.x.smithi203.stderr:2023-08-09T15:11:40.171+0000 7fb918e11040 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2023-08-09T15:11:40.179 INFO:tasks.ceph.mgr.y.smithi043.stderr:2023-08-09T15:11:40.179+0000 7f4daec47040 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2023-08-09T15:11:40.229 INFO:tasks.ceph.mgr.x.smithi203.stderr:2023-08-09T15:11:40.228+0000 7fb918e11040 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2023-08-09T15:11:40.278 INFO:tasks.ceph.mgr.x.smithi203.stderr:2023-08-09T15:11:40.278+0000 7fb918e11040 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2023-08-09T15:11:40.306 INFO:tasks.ceph.mgr.y.smithi043.stderr:2023-08-09T15:11:40.305+0000 7f4daec47040 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2023-08-09T15:11:40.365 INFO:tasks.ceph.mgr.y.smithi043.stderr:2023-08-09T15:11:40.365+0000 7f4daec47040 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2023-08-09T15:11:40.374 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:40.373+0000 7faa7ebea640 1 Processor -- start 2023-08-09T15:11:40.374 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:40.374+0000 7faa7ebea640 1 -- start start 2023-08-09T15:11:40.375 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:40.375+0000 7faa7ebea640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7faa7815a510 0x7faa78160fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T15:11:40.376 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:40.375+0000 7faa7ebea640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7faa7815d5e0 0x7faa780b3ec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T15:11:40.376 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:40.375+0000 7faa7ebea640 1 -- --> v1:172.21.15.203:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7faa780ccd30 con 0x7faa7815da60 2023-08-09T15:11:40.376 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:40.375+0000 7faa7ebea640 1 -- --> v2:172.21.15.43:3300/0 -- mon_getmap magic: 0 v1 -- 0x7faa7815a8f0 con 0x7faa7815a510 2023-08-09T15:11:40.376 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:40.375+0000 7faa7ebea640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7faa780cdd40 con 0x7faa7815d5e0 2023-08-09T15:11:40.377 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:40.376+0000 7faa7c986640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7faa7815a510 0x7faa78160fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T15:11:40.377 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:40.376+0000 7faa7c986640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7faa7815a510 0x7faa78160fe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:43684/0 (socket says 172.21.15.43:43684) 2023-08-09T15:11:40.377 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:40.376+0000 7faa7c986640 1 -- 172.21.15.43:0/2424141560 learned_addr learned my addr 172.21.15.43:0/2424141560 (peer_addr_for_me v2:172.21.15.43:0/0) 2023-08-09T15:11:40.377 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:40.376+0000 7faa77fff640 1 --2- 172.21.15.43:0/2424141560 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7faa7815d5e0 0x7faa780b3ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T15:11:40.377 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:40.376+0000 7faa7c986640 1 -- 172.21.15.43:0/2424141560 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7faa7815d5e0 msgr2=0x7faa780b3ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-09T15:11:40.378 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:40.376+0000 7faa7c986640 1 --2- 172.21.15.43:0/2424141560 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7faa7815d5e0 0x7faa780b3ec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T15:11:40.378 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:40.376+0000 7faa7c986640 1 -- 172.21.15.43:0/2424141560 >> v1:172.21.15.203:6789/0 conn(0x7faa7815da60 legacy=0x7faa7806a1f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-09T15:11:40.378 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:40.376+0000 7faa7c986640 1 -- 172.21.15.43:0/2424141560 --> v2:172.21.15.43:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faa780ccf80 con 0x7faa7815a510 2023-08-09T15:11:40.379 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:40.377+0000 7faa7c986640 1 --2- 172.21.15.43:0/2424141560 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7faa7815a510 0x7faa78160fe0 secure :-1 s=READY pgs=2442 cs=0 l=1 rev1=1 crypto rx=0x7faa6c009870 tx=0x7faa6c02b8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b9468a0827090b32 server_cookie=0 in_seq=0 out_seq=0 2023-08-09T15:11:40.379 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:40.378+0000 7faa777fe640 1 -- 172.21.15.43:0/2424141560 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faa6c031640 con 0x7faa7815a510 2023-08-09T15:11:40.379 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:40.378+0000 7faa777fe640 1 -- 172.21.15.43:0/2424141560 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7faa6c0317a0 con 0x7faa7815a510 2023-08-09T15:11:40.379 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:40.378+0000 7faa777fe640 1 -- 172.21.15.43:0/2424141560 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faa6c031b30 con 0x7faa7815a510 2023-08-09T15:11:40.380 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:40.378+0000 7faa7ebea640 1 -- 172.21.15.43:0/2424141560 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7faa7815a510 msgr2=0x7faa78160fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T15:11:40.380 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:40.378+0000 7faa7ebea640 1 --2- 172.21.15.43:0/2424141560 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7faa7815a510 0x7faa78160fe0 secure :-1 s=READY pgs=2442 cs=0 l=1 rev1=1 crypto rx=0x7faa6c009870 tx=0x7faa6c02b8f0 comp rx=0 tx=0).stop 2023-08-09T15:11:40.380 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:40.378+0000 7faa7ebea640 1 -- 172.21.15.43:0/2424141560 shutdown_connections 2023-08-09T15:11:40.380 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:40.378+0000 7faa7ebea640 1 --2- 172.21.15.43:0/2424141560 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7faa7815d5e0 0x7faa780b3ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T15:11:40.380 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:40.379+0000 7faa7ebea640 1 --2- 172.21.15.43:0/2424141560 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7faa7815a510 0x7faa78160fe0 unknown :-1 s=CLOSED pgs=2442 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T15:11:40.380 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:40.379+0000 7faa7ebea640 1 -- 172.21.15.43:0/2424141560 >> 172.21.15.43:0/2424141560 conn(0x7faa780ceec0 msgr2=0x7faa780cf2d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-09T15:11:40.380 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:40.379+0000 7faa7ebea640 1 -- 172.21.15.43:0/2424141560 shutdown_connections 2023-08-09T15:11:40.381 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:40.379+0000 7faa7ebea640 1 -- 172.21.15.43:0/2424141560 wait complete. 2023-08-09T15:11:40.381 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:40.380+0000 7faa7ebea640 1 Processor -- start 2023-08-09T15:11:40.381 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:40.381+0000 7faa7ebea640 1 -- start start 2023-08-09T15:11:40.382 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:40.381+0000 7faa7ebea640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7faa7815a510 0x7faa781f5c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T15:11:40.382 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:40.381+0000 7faa7ebea640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7faa7815d5e0 0x7faa781463d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T15:11:40.382 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:40.382+0000 7faa7ebea640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7faa7815da60 0x7faa780d0520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T15:11:40.383 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:40.382+0000 7faa7ebea640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faa780c6f70 con 0x7faa7815da60 2023-08-09T15:11:40.383 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:40.382+0000 7faa7ebea640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faa7814d050 con 0x7faa7815a510 2023-08-09T15:11:40.383 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:40.382+0000 7faa7ebea640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7faa781f61c0 con 0x7faa7815d5e0 2023-08-09T15:11:40.383 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:40.382+0000 7faa7d187640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7faa7815da60 0x7faa780d0520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T15:11:40.383 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:40.382+0000 7faa7d187640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7faa7815da60 0x7faa780d0520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:43698/0 (socket says 172.21.15.43:43698) 2023-08-09T15:11:40.383 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:40.382+0000 7faa7d187640 1 -- 172.21.15.43:0/2102300234 learned_addr learned my addr 172.21.15.43:0/2102300234 (peer_addr_for_me v2:172.21.15.43:0/0) 2023-08-09T15:11:40.383 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:40.382+0000 7faa77fff640 1 --2- 172.21.15.43:0/2102300234 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7faa7815d5e0 0x7faa781463d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T15:11:40.383 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:40.382+0000 7faa7c986640 1 --2- 172.21.15.43:0/2102300234 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7faa7815a510 0x7faa781f5c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T15:11:40.384 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:40.382+0000 7faa7d187640 1 -- 172.21.15.43:0/2102300234 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7faa7815d5e0 msgr2=0x7faa781463d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T15:11:40.385 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:40.382+0000 7faa7d187640 1 --2- 172.21.15.43:0/2102300234 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7faa7815d5e0 0x7faa781463d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T15:11:40.385 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:40.382+0000 7faa7d187640 1 -- 172.21.15.43:0/2102300234 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7faa7815a510 msgr2=0x7faa781f5c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T15:11:40.385 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:40.382+0000 7faa7d187640 1 --2- 172.21.15.43:0/2102300234 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7faa7815a510 0x7faa781f5c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T15:11:40.385 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:40.382+0000 7faa7d187640 1 -- 172.21.15.43:0/2102300234 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faa6c0094d0 con 0x7faa7815da60 2023-08-09T15:11:40.385 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:40.383+0000 7faa7d187640 1 --2- 172.21.15.43:0/2102300234 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7faa7815da60 0x7faa780d0520 secure :-1 s=READY pgs=2443 cs=0 l=1 rev1=1 crypto rx=0x7faa7000c7b0 tx=0x7faa7000cc70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-09T15:11:40.385 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:40.383+0000 7faa757fa640 1 -- 172.21.15.43:0/2102300234 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faa70004b60 con 0x7faa7815da60 2023-08-09T15:11:40.386 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:40.383+0000 7faa757fa640 1 -- 172.21.15.43:0/2102300234 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7faa70004cc0 con 0x7faa7815da60 2023-08-09T15:11:40.386 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:40.383+0000 7faa7ebea640 1 -- 172.21.15.43:0/2102300234 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7faa78146c90 con 0x7faa7815da60 2023-08-09T15:11:40.386 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:40.383+0000 7faa757fa640 1 -- 172.21.15.43:0/2102300234 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faa7000aa90 con 0x7faa7815da60 2023-08-09T15:11:40.386 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:40.383+0000 7faa7ebea640 1 -- 172.21.15.43:0/2102300234 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7faa780c66d0 con 0x7faa7815da60 2023-08-09T15:11:40.386 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:40.384+0000 7faa7ebea640 1 -- 172.21.15.43:0/2102300234 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7faa780739f0 con 0x7faa7815da60 2023-08-09T15:11:40.387 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:40.385+0000 7faa757fa640 1 -- 172.21.15.43:0/2102300234 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 119) v1 ==== 88460+0+0 (secure 0 0 0) 0x7faa7000abf0 con 0x7faa7815da60 2023-08-09T15:11:40.387 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:40.386+0000 7faa757fa640 1 --2- 172.21.15.43:0/2102300234 >> [v2:172.21.15.203:6832/4034093268,v1:172.21.15.203:6833/4034093268] conn(0x7faa5c06a1c0 0x7faa5c06c680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T15:11:40.387 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:40.386+0000 7faa757fa640 1 -- 172.21.15.43:0/2102300234 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(223..223 src has 1..223) v4 ==== 6426+0+0 (secure 0 0 0) 0x7faa700889c0 con 0x7faa7815da60 2023-08-09T15:11:40.387 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:40.386+0000 7faa7c986640 1 -- 172.21.15.43:0/2102300234 >> [v2:172.21.15.203:6832/4034093268,v1:172.21.15.203:6833/4034093268] conn(0x7faa5c06a1c0 msgr2=0x7faa5c06c680 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.203:6832/4034093268 2023-08-09T15:11:40.387 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:40.386+0000 7faa7c986640 1 --2- 172.21.15.43:0/2102300234 >> [v2:172.21.15.203:6832/4034093268,v1:172.21.15.203:6833/4034093268] conn(0x7faa5c06a1c0 0x7faa5c06c680 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-08-09T15:11:40.390 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:40.390+0000 7faa757fa640 1 -- 172.21.15.43:0/2102300234 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178066 (secure 0 0 0) 0x7faa70053e20 con 0x7faa7815da60 2023-08-09T15:11:40.420 INFO:tasks.ceph.mgr.y.smithi043.stderr:2023-08-09T15:11:40.419+0000 7f4daec47040 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2023-08-09T15:11:40.587 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:40.587+0000 7faa7c986640 1 -- 172.21.15.43:0/2102300234 >> [v2:172.21.15.203:6832/4034093268,v1:172.21.15.203:6833/4034093268] conn(0x7faa5c06a1c0 msgr2=0x7faa5c06c680 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.203:6832/4034093268 2023-08-09T15:11:40.588 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:40.587+0000 7faa7c986640 1 --2- 172.21.15.43:0/2102300234 >> [v2:172.21.15.203:6832/4034093268,v1:172.21.15.203:6833/4034093268] conn(0x7faa5c06a1c0 0x7faa5c06c680 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2023-08-09T15:11:40.606 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:40.606+0000 7faa7ebea640 1 -- 172.21.15.43:0/2102300234 --> [v2:172.21.15.203:6832/4034093268,v1:172.21.15.203:6833/4034093268] -- mgr_command(tid 0: {"prefix": "fs snap-schedule add", "fs": "cephfs", "path": "snap_test_dir1", "snap_schedule": "1M", "target": ["mon-mgr", ""]}) v1 -- 0x7faa780cd540 con 0x7faa5c06a1c0 2023-08-09T15:11:40.967 INFO:tasks.ceph.mgr.x.smithi203.stderr:2023-08-09T15:11:40.967+0000 7fb918e11040 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2023-08-09T15:11:40.988 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:40.988+0000 7faa7c986640 1 -- 172.21.15.43:0/2102300234 >> [v2:172.21.15.203:6832/4034093268,v1:172.21.15.203:6833/4034093268] conn(0x7faa5c06a1c0 msgr2=0x7faa5c06c680 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.203:6832/4034093268 2023-08-09T15:11:40.989 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:40.988+0000 7faa7c986640 1 --2- 172.21.15.43:0/2102300234 >> [v2:172.21.15.203:6832/4034093268,v1:172.21.15.203:6833/4034093268] conn(0x7faa5c06a1c0 0x7faa5c06c680 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2023-08-09T15:11:41.027 INFO:tasks.ceph.mgr.x.smithi203.stderr:2023-08-09T15:11:41.027+0000 7fb918e11040 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2023-08-09T15:11:41.076 INFO:tasks.ceph.mgr.x.smithi203.stderr:2023-08-09T15:11:41.075+0000 7fb918e11040 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2023-08-09T15:11:41.173 INFO:tasks.ceph.mgr.y.smithi043.stderr:2023-08-09T15:11:41.172+0000 7f4daec47040 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2023-08-09T15:11:41.192 INFO:tasks.ceph.mgr.x.smithi203.stderr:2023-08-09T15:11:41.192+0000 7fb918e11040 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2023-08-09T15:11:41.236 INFO:tasks.ceph.mgr.y.smithi043.stderr:2023-08-09T15:11:41.236+0000 7f4daec47040 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2023-08-09T15:11:41.288 INFO:tasks.ceph.mgr.y.smithi043.stderr:2023-08-09T15:11:41.287+0000 7f4daec47040 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2023-08-09T15:11:41.346 INFO:tasks.ceph.mgr.x.smithi203.stderr:2023-08-09T15:11:41.346+0000 7fb918e11040 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2023-08-09T15:11:41.411 INFO:tasks.ceph.mgr.y.smithi043.stderr:2023-08-09T15:11:41.410+0000 7f4daec47040 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2023-08-09T15:11:41.453 INFO:tasks.ceph.mgr.x.smithi203.stderr:2023-08-09T15:11:41.453+0000 7fb918e11040 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2023-08-09T15:11:41.502 INFO:tasks.ceph.mgr.x.smithi203.stderr:2023-08-09T15:11:41.501+0000 7fb918e11040 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2023-08-09T15:11:41.574 INFO:tasks.ceph.mgr.y.smithi043.stderr:2023-08-09T15:11:41.573+0000 7f4daec47040 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2023-08-09T15:11:41.660 INFO:tasks.ceph.mgr.x.smithi203.stderr:2023-08-09T15:11:41.659+0000 7fb918e11040 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2023-08-09T15:11:41.686 INFO:tasks.ceph.mgr.y.smithi043.stderr:2023-08-09T15:11:41.686+0000 7f4daec47040 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2023-08-09T15:11:41.738 INFO:tasks.ceph.mgr.y.smithi043.stderr:2023-08-09T15:11:41.737+0000 7f4daec47040 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2023-08-09T15:11:41.790 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:41.789+0000 7faa7c986640 1 -- 172.21.15.43:0/2102300234 >> [v2:172.21.15.203:6832/4034093268,v1:172.21.15.203:6833/4034093268] conn(0x7faa5c06a1c0 msgr2=0x7faa5c06c680 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.203:6832/4034093268 2023-08-09T15:11:41.791 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:41.789+0000 7faa7c986640 1 --2- 172.21.15.43:0/2102300234 >> [v2:172.21.15.203:6832/4034093268,v1:172.21.15.203:6833/4034093268] conn(0x7faa5c06a1c0 0x7faa5c06c680 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2023-08-09T15:11:41.905 INFO:tasks.ceph.mgr.y.smithi043.stderr:2023-08-09T15:11:41.904+0000 7f4daec47040 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2023-08-09T15:11:42.037 INFO:tasks.ceph.mgr.x.smithi203.stderr:2023-08-09T15:11:42.036+0000 7fb918e11040 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2023-08-09T15:11:42.256 INFO:tasks.ceph.mgr.x.smithi203.stderr:2023-08-09T15:11:42.255+0000 7fb918e11040 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2023-08-09T15:11:42.302 INFO:tasks.ceph.mgr.y.smithi043.stderr:2023-08-09T15:11:42.302+0000 7f4daec47040 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2023-08-09T15:11:42.312 INFO:tasks.ceph.mgr.x.smithi203.stderr:2023-08-09T15:11:42.312+0000 7fb918e11040 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2023-08-09T15:11:42.361 INFO:tasks.ceph.mgr.x.smithi203.stderr:2023-08-09T15:11:42.361+0000 7fb918e11040 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2023-08-09T15:11:42.413 INFO:tasks.ceph.mgr.x.smithi203.stderr:2023-08-09T15:11:42.413+0000 7fb918e11040 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2023-08-09T15:11:42.531 INFO:tasks.ceph.mgr.y.smithi043.stderr:2023-08-09T15:11:42.531+0000 7f4daec47040 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2023-08-09T15:11:42.591 INFO:tasks.ceph.mgr.y.smithi043.stderr:2023-08-09T15:11:42.590+0000 7f4daec47040 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2023-08-09T15:11:42.643 INFO:tasks.ceph.mgr.y.smithi043.stderr:2023-08-09T15:11:42.642+0000 7f4daec47040 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2023-08-09T15:11:42.697 INFO:tasks.ceph.mgr.y.smithi043.stderr:2023-08-09T15:11:42.696+0000 7f4daec47040 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2023-08-09T15:11:42.706 INFO:tasks.ceph.mgr.x.smithi203.stderr:2023-08-09T15:11:42.706+0000 7fb918e11040 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2023-08-09T15:11:42.764 INFO:tasks.ceph.mgr.x.smithi203.stderr:2023-08-09T15:11:42.763+0000 7fb918e11040 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2023-08-09T15:11:42.817 INFO:tasks.ceph.mgr.x.smithi203.stderr:2023-08-09T15:11:42.816+0000 7fb918e11040 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2023-08-09T15:11:42.976 INFO:tasks.ceph.mgr.x.smithi203.stderr:2023-08-09T15:11:42.975+0000 7fb918e11040 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2023-08-09T15:11:43.004 INFO:tasks.ceph.mgr.y.smithi043.stderr:2023-08-09T15:11:43.003+0000 7f4daec47040 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2023-08-09T15:11:43.024 INFO:tasks.ceph.mgr.x.smithi203.stderr:2023-08-09T15:11:43.024+0000 7fb918e11040 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2023-08-09T15:11:43.064 INFO:tasks.ceph.mgr.y.smithi043.stderr:2023-08-09T15:11:43.063+0000 7f4daec47040 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2023-08-09T15:11:43.076 INFO:tasks.ceph.mgr.x.smithi203.stderr:2023-08-09T15:11:43.076+0000 7fb918e11040 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2023-08-09T15:11:43.118 INFO:tasks.ceph.mgr.y.smithi043.stderr:2023-08-09T15:11:43.117+0000 7f4daec47040 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2023-08-09T15:11:43.126 INFO:tasks.ceph.mgr.x.smithi203.stderr:2023-08-09T15:11:43.125+0000 7fb918e11040 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2023-08-09T15:11:43.147 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:43.146+0000 7faa757fa640 1 -- 172.21.15.43:0/2102300234 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mgrmap(e 120) v1 ==== 88069+0+0 (secure 0 0 0) 0x7faa700541f0 con 0x7faa7815da60 2023-08-09T15:11:43.148 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:43.147+0000 7faa757fa640 1 -- 172.21.15.43:0/2102300234 >> [v2:172.21.15.203:6832/4034093268,v1:172.21.15.203:6833/4034093268] conn(0x7faa5c06a1c0 msgr2=0x7faa5c06c680 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-08-09T15:11:43.148 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:43.147+0000 7faa757fa640 1 --2- 172.21.15.43:0/2102300234 >> [v2:172.21.15.203:6832/4034093268,v1:172.21.15.203:6833/4034093268] conn(0x7faa5c06a1c0 0x7faa5c06c680 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T15:11:43.306 INFO:tasks.ceph.mgr.x.smithi203.stderr:2023-08-09T15:11:43.305+0000 7fb8985a0640 -1 client.0 error registering admin socket command: (17) File exists 2023-08-09T15:11:43.306 INFO:tasks.ceph.mgr.x.smithi203.stderr:2023-08-09T15:11:43.305+0000 7fb8985a0640 -1 client.0 error registering admin socket command: (17) File exists 2023-08-09T15:11:43.306 INFO:tasks.ceph.mgr.x.smithi203.stderr:2023-08-09T15:11:43.305+0000 7fb8985a0640 -1 client.0 error registering admin socket command: (17) File exists 2023-08-09T15:11:43.306 INFO:tasks.ceph.mgr.x.smithi203.stderr:2023-08-09T15:11:43.305+0000 7fb8985a0640 -1 client.0 error registering admin socket command: (17) File exists 2023-08-09T15:11:43.306 INFO:tasks.ceph.mgr.x.smithi203.stderr:2023-08-09T15:11:43.305+0000 7fb8985a0640 -1 client.0 error registering admin socket command: (17) File exists 2023-08-09T15:11:43.307 INFO:tasks.ceph.mgr.x.smithi203.stderr:2023-08-09T15:11:43.305+0000 7fb894598640 -1 client.0 error registering admin socket command: (17) File exists 2023-08-09T15:11:43.307 INFO:tasks.ceph.mgr.x.smithi203.stderr:2023-08-09T15:11:43.305+0000 7fb894598640 -1 client.0 error registering admin socket command: (17) File exists 2023-08-09T15:11:43.307 INFO:tasks.ceph.mgr.x.smithi203.stderr:2023-08-09T15:11:43.305+0000 7fb894598640 -1 client.0 error registering admin socket command: (17) File exists 2023-08-09T15:11:43.307 INFO:tasks.ceph.mgr.x.smithi203.stderr:2023-08-09T15:11:43.305+0000 7fb894598640 -1 client.0 error registering admin socket command: (17) File exists 2023-08-09T15:11:43.307 INFO:tasks.ceph.mgr.x.smithi203.stderr:2023-08-09T15:11:43.305+0000 7fb894598640 -1 client.0 error registering admin socket command: (17) File exists 2023-08-09T15:11:43.307 INFO:tasks.ceph.mgr.y.smithi043.stderr:2023-08-09T15:11:43.306+0000 7f4daec47040 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2023-08-09T15:11:43.360 INFO:tasks.ceph.mgr.y.smithi043.stderr:2023-08-09T15:11:43.360+0000 7f4daec47040 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2023-08-09T15:11:43.416 INFO:tasks.ceph.mgr.y.smithi043.stderr:2023-08-09T15:11:43.415+0000 7f4daec47040 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2023-08-09T15:11:43.468 INFO:tasks.ceph.mgr.y.smithi043.stderr:2023-08-09T15:11:43.467+0000 7f4daec47040 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2023-08-09T15:11:44.215 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:44.214+0000 7faa757fa640 1 -- 172.21.15.43:0/2102300234 <== mon.0 v2:172.21.15.43:3300/0 8 ==== mgrmap(e 121) v1 ==== 88196+0+0 (secure 0 0 0) 0x7faa70075d60 con 0x7faa7815da60 2023-08-09T15:11:44.216 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:44.215+0000 7faa757fa640 1 --2- 172.21.15.43:0/2102300234 >> [v2:172.21.15.203:6832/2375771024,v1:172.21.15.203:6833/2375771024] conn(0x7faa5c072cb0 0x7faa5c0750a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T15:11:44.216 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:44.215+0000 7faa757fa640 1 -- 172.21.15.43:0/2102300234 --> [v2:172.21.15.203:6832/2375771024,v1:172.21.15.203:6833/2375771024] -- mgr_command(tid 0: {"prefix": "fs snap-schedule add", "fs": "cephfs", "path": "snap_test_dir1", "snap_schedule": "1M", "target": ["mon-mgr", ""]}) v1 -- 0x7faa5c036270 con 0x7faa5c072cb0 2023-08-09T15:11:44.222 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:44.221+0000 7faa7c986640 1 --2- 172.21.15.43:0/2102300234 >> [v2:172.21.15.203:6832/2375771024,v1:172.21.15.203:6833/2375771024] conn(0x7faa5c072cb0 0x7faa5c0750a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T15:11:44.237 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:44.235+0000 7faa7c986640 1 --2- 172.21.15.43:0/2102300234 >> [v2:172.21.15.203:6832/2375771024,v1:172.21.15.203:6833/2375771024] conn(0x7faa5c072cb0 0x7faa5c0750a0 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7faa6c036000 tx=0x7faa6c008030 comp rx=0 tx=0).ready entity=mgr.9103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-09T15:11:44.380 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:44.379+0000 7faa757fa640 1 -- 172.21.15.43:0/2102300234 <== mgr.9103 v2:172.21.15.203:6832/2375771024 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+37 (secure 0 0 0) 0x7faa6c0042f0 con 0x7faa5c072cb0 2023-08-09T15:11:44.381 INFO:teuthology.orchestra.run.smithi043.stdout:Schedule set for path snap_test_dir1 2023-08-09T15:11:44.384 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:44.383+0000 7faa7ebea640 1 -- 172.21.15.43:0/2102300234 >> [v2:172.21.15.203:6832/2375771024,v1:172.21.15.203:6833/2375771024] conn(0x7faa5c072cb0 msgr2=0x7faa5c0750a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T15:11:44.384 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:44.383+0000 7faa7ebea640 1 --2- 172.21.15.43:0/2102300234 >> [v2:172.21.15.203:6832/2375771024,v1:172.21.15.203:6833/2375771024] conn(0x7faa5c072cb0 0x7faa5c0750a0 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7faa6c036000 tx=0x7faa6c008030 comp rx=0 tx=0).stop 2023-08-09T15:11:44.384 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:44.383+0000 7faa7ebea640 1 -- 172.21.15.43:0/2102300234 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7faa7815da60 msgr2=0x7faa780d0520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T15:11:44.385 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:44.383+0000 7faa7ebea640 1 --2- 172.21.15.43:0/2102300234 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7faa7815da60 0x7faa780d0520 secure :-1 s=READY pgs=2443 cs=0 l=1 rev1=1 crypto rx=0x7faa7000c7b0 tx=0x7faa7000cc70 comp rx=0 tx=0).stop 2023-08-09T15:11:44.385 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:44.383+0000 7faa7d187640 1 -- 172.21.15.43:0/2102300234 reap_dead start 2023-08-09T15:11:44.385 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:44.383+0000 7faa7ebea640 1 -- 172.21.15.43:0/2102300234 shutdown_connections 2023-08-09T15:11:44.385 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:44.383+0000 7faa7ebea640 1 -- 172.21.15.43:0/2102300234 >> 172.21.15.43:0/2102300234 conn(0x7faa780ceec0 msgr2=0x7faa78160570 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-09T15:11:44.385 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:44.383+0000 7faa7ebea640 1 -- 172.21.15.43:0/2102300234 shutdown_connections 2023-08-09T15:11:44.386 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:44.383+0000 7faa7ebea640 1 -- 172.21.15.43:0/2102300234 wait complete. 2023-08-09T15:11:44.407 DEBUG:tasks.cephfs.test_snap_schedules:res=Schedule set for path snap_test_dir1 2023-08-09T15:11:44.407 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs snap-schedule retention add --fs cephfs snap_test_dir1 1M 2023-08-09T15:11:44.695 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:44.693+0000 7f630a724640 1 Processor -- start 2023-08-09T15:11:44.695 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:44.693+0000 7f630a724640 1 -- start start 2023-08-09T15:11:44.696 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:44.695+0000 7f630a724640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f63040cb9a0 0x7f63040cbde0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T15:11:44.697 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:44.695+0000 7f630a724640 1 -- --> v1:172.21.15.43:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f63040acfa0 con 0x7f630406f310 2023-08-09T15:11:44.697 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:44.695+0000 7f630a724640 1 -- --> v1:172.21.15.203:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f63040cc320 con 0x7f6304070ef0 2023-08-09T15:11:44.697 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:44.695+0000 7f630a724640 1 -- --> v2:172.21.15.43:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f630415c940 con 0x7f63040cb9a0 2023-08-09T15:11:44.698 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:44.696+0000 7f6308f21640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f63040cb9a0 0x7f63040cbde0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T15:11:44.698 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:44.696+0000 7f6308f21640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f63040cb9a0 0x7f63040cbde0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:43708/0 (socket says 172.21.15.43:43708) 2023-08-09T15:11:44.698 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:44.696+0000 7f6308f21640 1 -- 172.21.15.43:0/3239171596 learned_addr learned my addr 172.21.15.43:0/3239171596 (peer_addr_for_me v2:172.21.15.43:0/0) 2023-08-09T15:11:44.698 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:44.696+0000 7f6308f21640 1 -- 172.21.15.43:0/3239171596 >> v1:172.21.15.203:6789/0 conn(0x7f6304070ef0 legacy=0x7f63040ab8d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T15:11:44.698 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:44.696+0000 7f6308f21640 1 -- 172.21.15.43:0/3239171596 >> v1:172.21.15.43:6789/0 conn(0x7f630406f310 legacy=0x7f630406f760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T15:11:44.698 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:44.696+0000 7f6308f21640 1 -- 172.21.15.43:0/3239171596 --> v2:172.21.15.43:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f63040ac390 con 0x7f63040cb9a0 2023-08-09T15:11:44.699 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:44.698+0000 7f6308f21640 1 --2- 172.21.15.43:0/3239171596 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f63040cb9a0 0x7f63040cbde0 secure :-1 s=READY pgs=2445 cs=0 l=1 rev1=1 crypto rx=0x7f62f4009870 tx=0x7f62f402bac0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7d524442d2b7cd29 server_cookie=0 in_seq=0 out_seq=0 2023-08-09T15:11:44.699 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:44.698+0000 7f62fbfff640 1 -- 172.21.15.43:0/3239171596 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f62f402d880 con 0x7f63040cb9a0 2023-08-09T15:11:44.699 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:44.698+0000 7f62fbfff640 1 -- 172.21.15.43:0/3239171596 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f62f402d9e0 con 0x7f63040cb9a0 2023-08-09T15:11:44.700 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:44.698+0000 7f62fbfff640 1 -- 172.21.15.43:0/3239171596 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f62f402dd70 con 0x7f63040cb9a0 2023-08-09T15:11:44.700 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:44.699+0000 7f630a724640 1 -- 172.21.15.43:0/3239171596 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f63040cb9a0 msgr2=0x7f63040cbde0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T15:11:44.701 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:44.699+0000 7f630a724640 1 --2- 172.21.15.43:0/3239171596 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f63040cb9a0 0x7f63040cbde0 secure :-1 s=READY pgs=2445 cs=0 l=1 rev1=1 crypto rx=0x7f62f4009870 tx=0x7f62f402bac0 comp rx=0 tx=0).stop 2023-08-09T15:11:44.701 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:44.699+0000 7f630a724640 1 -- 172.21.15.43:0/3239171596 shutdown_connections 2023-08-09T15:11:44.701 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:44.699+0000 7f630a724640 1 --2- 172.21.15.43:0/3239171596 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f63040cb9a0 0x7f63040cbde0 unknown :-1 s=CLOSED pgs=2445 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T15:11:44.701 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:44.699+0000 7f630a724640 1 -- 172.21.15.43:0/3239171596 >> 172.21.15.43:0/3239171596 conn(0x7f63040cddd0 msgr2=0x7f63040ce1e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-09T15:11:44.701 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:44.699+0000 7f630a724640 1 -- 172.21.15.43:0/3239171596 shutdown_connections 2023-08-09T15:11:44.701 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:44.699+0000 7f630a724640 1 -- 172.21.15.43:0/3239171596 wait complete. 2023-08-09T15:11:44.702 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:44.700+0000 7f630a724640 1 Processor -- start 2023-08-09T15:11:44.702 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:44.701+0000 7f630a724640 1 -- start start 2023-08-09T15:11:44.703 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:44.702+0000 7f630a724640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f630406f310 0x7f63041452c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T15:11:44.703 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:44.702+0000 7f630a724640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f6304070ef0 0x7f6304145800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T15:11:44.703 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:44.702+0000 7f630a724640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f63040cb9a0 0x7f6304143100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T15:11:44.703 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:44.702+0000 7f630a724640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f63041596d0 con 0x7f6304070ef0 2023-08-09T15:11:44.704 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:44.702+0000 7f630a724640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f63041449f0 con 0x7f630406f310 2023-08-09T15:11:44.704 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:44.702+0000 7f630a724640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6304143e70 con 0x7f63040cb9a0 2023-08-09T15:11:44.704 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:44.702+0000 7f6308f21640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f6304070ef0 0x7f6304145800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T15:11:44.704 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:44.702+0000 7f6308f21640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f6304070ef0 0x7f6304145800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:43720/0 (socket says 172.21.15.43:43720) 2023-08-09T15:11:44.704 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:44.702+0000 7f6308f21640 1 -- 172.21.15.43:0/373155434 learned_addr learned my addr 172.21.15.43:0/373155434 (peer_addr_for_me v2:172.21.15.43:0/0) 2023-08-09T15:11:44.705 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:44.702+0000 7f6309722640 1 --2- 172.21.15.43:0/373155434 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f630406f310 0x7f63041452c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T15:11:44.705 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:44.702+0000 7f6308f21640 1 -- 172.21.15.43:0/373155434 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f63040cb9a0 msgr2=0x7f6304143100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T15:11:44.706 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:44.702+0000 7f6308f21640 1 --2- 172.21.15.43:0/373155434 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f63040cb9a0 0x7f6304143100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T15:11:44.706 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:44.702+0000 7f6308f21640 1 -- 172.21.15.43:0/373155434 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f630406f310 msgr2=0x7f63041452c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T15:11:44.706 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:44.702+0000 7f6308f21640 1 --2- 172.21.15.43:0/373155434 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f630406f310 0x7f63041452c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T15:11:44.706 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:44.702+0000 7f6308f21640 1 -- 172.21.15.43:0/373155434 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f62f40094d0 con 0x7f6304070ef0 2023-08-09T15:11:44.706 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:44.702+0000 7f6309722640 1 --2- 172.21.15.43:0/373155434 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f630406f310 0x7f63041452c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-08-09T15:11:44.706 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:44.703+0000 7f6308f21640 1 --2- 172.21.15.43:0/373155434 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f6304070ef0 0x7f6304145800 secure :-1 s=READY pgs=2446 cs=0 l=1 rev1=1 crypto rx=0x7f62f4009990 tx=0x7f62f4004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-09T15:11:44.706 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:44.703+0000 7f62f9ffb640 1 -- 172.21.15.43:0/373155434 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f62f403c040 con 0x7f6304070ef0 2023-08-09T15:11:44.707 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:44.703+0000 7f62f9ffb640 1 -- 172.21.15.43:0/373155434 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f62f4037c60 con 0x7f6304070ef0 2023-08-09T15:11:44.707 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:44.703+0000 7f62f9ffb640 1 -- 172.21.15.43:0/373155434 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f62f402d820 con 0x7f6304070ef0 2023-08-09T15:11:44.707 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:44.703+0000 7f630a724640 1 -- 172.21.15.43:0/373155434 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6304144150 con 0x7f6304070ef0 2023-08-09T15:11:44.707 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:44.703+0000 7f630a724640 1 -- 172.21.15.43:0/373155434 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6304144710 con 0x7f6304070ef0 2023-08-09T15:11:44.707 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:44.704+0000 7f630a724640 1 -- 172.21.15.43:0/373155434 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f63041f7470 con 0x7f6304070ef0 2023-08-09T15:11:44.708 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:44.705+0000 7f62f9ffb640 1 -- 172.21.15.43:0/373155434 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 121) v1 ==== 88196+0+0 (secure 0 0 0) 0x7f62f404b430 con 0x7f6304070ef0 2023-08-09T15:11:44.708 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:44.706+0000 7f62f9ffb640 1 --2- 172.21.15.43:0/373155434 >> [v2:172.21.15.203:6832/2375771024,v1:172.21.15.203:6833/2375771024] conn(0x7f62e8069ef0 0x7f62e806c3b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T15:11:44.708 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:44.706+0000 7f62f9ffb640 1 -- 172.21.15.43:0/373155434 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(224..224 src has 1..224) v4 ==== 6770+0+0 (secure 0 0 0) 0x7f62f4032070 con 0x7f6304070ef0 2023-08-09T15:11:44.708 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:44.707+0000 7f6309722640 1 --2- 172.21.15.43:0/373155434 >> [v2:172.21.15.203:6832/2375771024,v1:172.21.15.203:6833/2375771024] conn(0x7f62e8069ef0 0x7f62e806c3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T15:11:44.709 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:44.708+0000 7f6309722640 1 --2- 172.21.15.43:0/373155434 >> [v2:172.21.15.203:6832/2375771024,v1:172.21.15.203:6833/2375771024] conn(0x7f62e8069ef0 0x7f62e806c3b0 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7f6300000f60 tx=0x7f6300009040 comp rx=0 tx=0).ready entity=mgr.9103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-09T15:11:44.712 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:44.710+0000 7f62f9ffb640 1 -- 172.21.15.43:0/373155434 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178066 (secure 0 0 0) 0x7f62f40dc7b0 con 0x7f6304070ef0 2023-08-09T15:11:44.935 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:44.934+0000 7f630a724640 1 -- 172.21.15.43:0/373155434 --> [v2:172.21.15.203:6832/2375771024,v1:172.21.15.203:6833/2375771024] -- mgr_command(tid 0: {"prefix": "fs snap-schedule retention add", "fs": "cephfs", "path": "snap_test_dir1", "retention_spec_or_period": "1M", "target": ["mon-mgr", ""]}) v1 -- 0x7f63040ac950 con 0x7f62e8069ef0 2023-08-09T15:11:45.075 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:45.073+0000 7f62f9ffb640 1 -- 172.21.15.43:0/373155434 <== mgr.9103 v2:172.21.15.203:6832/2375771024 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+39 (secure 0 0 0) 0x7f6300001d30 con 0x7f62e8069ef0 2023-08-09T15:11:45.075 INFO:teuthology.orchestra.run.smithi043.stdout:Retention added to path snap_test_dir1 2023-08-09T15:11:45.078 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:45.076+0000 7f630a724640 1 -- 172.21.15.43:0/373155434 >> [v2:172.21.15.203:6832/2375771024,v1:172.21.15.203:6833/2375771024] conn(0x7f62e8069ef0 msgr2=0x7f62e806c3b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T15:11:45.078 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:45.076+0000 7f630a724640 1 --2- 172.21.15.43:0/373155434 >> [v2:172.21.15.203:6832/2375771024,v1:172.21.15.203:6833/2375771024] conn(0x7f62e8069ef0 0x7f62e806c3b0 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7f6300000f60 tx=0x7f6300009040 comp rx=0 tx=0).stop 2023-08-09T15:11:45.078 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:45.077+0000 7f630a724640 1 -- 172.21.15.43:0/373155434 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f6304070ef0 msgr2=0x7f6304145800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T15:11:45.078 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:45.077+0000 7f630a724640 1 --2- 172.21.15.43:0/373155434 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f6304070ef0 0x7f6304145800 secure :-1 s=READY pgs=2446 cs=0 l=1 rev1=1 crypto rx=0x7f62f4009990 tx=0x7f62f4004730 comp rx=0 tx=0).stop 2023-08-09T15:11:45.079 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:45.077+0000 7f630a724640 1 -- 172.21.15.43:0/373155434 shutdown_connections 2023-08-09T15:11:45.079 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:45.077+0000 7f630a724640 1 --2- 172.21.15.43:0/373155434 >> [v2:172.21.15.203:6832/2375771024,v1:172.21.15.203:6833/2375771024] conn(0x7f62e8069ef0 0x7f62e806c3b0 unknown :-1 s=CLOSED pgs=23 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T15:11:45.079 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:45.077+0000 7f630a724640 1 --2- 172.21.15.43:0/373155434 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f63040cb9a0 0x7f6304143100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T15:11:45.080 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:45.077+0000 7f630a724640 1 --2- 172.21.15.43:0/373155434 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f6304070ef0 0x7f6304145800 unknown :-1 s=CLOSED pgs=2446 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T15:11:45.080 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:45.077+0000 7f630a724640 1 --2- 172.21.15.43:0/373155434 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f630406f310 0x7f63041452c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T15:11:45.080 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:45.077+0000 7f630a724640 1 -- 172.21.15.43:0/373155434 >> 172.21.15.43:0/373155434 conn(0x7f63040cddd0 msgr2=0x7f63040aec90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-09T15:11:45.080 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:45.077+0000 7f630a724640 1 -- 172.21.15.43:0/373155434 shutdown_connections 2023-08-09T15:11:45.080 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:45.077+0000 7f630a724640 1 -- 172.21.15.43:0/373155434 wait complete. 2023-08-09T15:11:45.101 DEBUG:tasks.cephfs.test_snap_schedules:res=Retention added to path snap_test_dir1 2023-08-09T15:11:45.106 DEBUG:teuthology.orchestra.run.smithi043:> sudo mkdir -p /etc/ceph && sudo chmod 0755 /etc/ceph && sudo tee /etc/ceph/ceph.conf && sudo chmod 0644 /etc/ceph/ceph.conf > /dev/null 2023-08-09T15:11:45.111 DEBUG:teuthology.orchestra.run.smithi203:> sudo mkdir -p /etc/ceph && sudo chmod 0755 /etc/ceph && sudo tee /etc/ceph/ceph.conf && sudo chmod 0644 /etc/ceph/ceph.conf > /dev/null 2023-08-09T15:11:45.303 INFO:teuthology.orchestra.run.smithi203.stdout:[global] 2023-08-09T15:11:45.304 INFO:teuthology.orchestra.run.smithi203.stdout: chdir = "" 2023-08-09T15:11:45.304 INFO:teuthology.orchestra.run.smithi203.stdout: pid file = /var/run/ceph/$cluster-$name.pid 2023-08-09T15:11:45.304 INFO:teuthology.orchestra.run.smithi203.stdout: auth supported = cephx 2023-08-09T15:11:45.304 INFO:teuthology.orchestra.run.smithi203.stdout: 2023-08-09T15:11:45.304 INFO:teuthology.orchestra.run.smithi203.stdout: filestore xattr use omap = true 2023-08-09T15:11:45.305 INFO:teuthology.orchestra.run.smithi203.stdout: 2023-08-09T15:11:45.305 INFO:teuthology.orchestra.run.smithi203.stdout: mon clock drift allowed = 1.000 2023-08-09T15:11:45.305 INFO:teuthology.orchestra.run.smithi203.stdout: 2023-08-09T15:11:45.305 INFO:teuthology.orchestra.run.smithi203.stdout: osd crush chooseleaf type = 0 2023-08-09T15:11:45.305 INFO:teuthology.orchestra.run.smithi203.stdout: auth debug = true 2023-08-09T15:11:45.305 INFO:teuthology.orchestra.run.smithi203.stdout: 2023-08-09T15:11:45.305 INFO:teuthology.orchestra.run.smithi203.stdout: ms die on old message = true 2023-08-09T15:11:45.305 INFO:teuthology.orchestra.run.smithi203.stdout: ms die on bug = true 2023-08-09T15:11:45.306 INFO:teuthology.orchestra.run.smithi203.stdout: 2023-08-09T15:11:45.306 INFO:teuthology.orchestra.run.smithi203.stdout: mon max pg per osd = 10000 # >= luminous 2023-08-09T15:11:45.306 INFO:teuthology.orchestra.run.smithi203.stdout: mon pg warn max object skew = 0 2023-08-09T15:11:45.306 INFO:teuthology.orchestra.run.smithi203.stdout: 2023-08-09T15:11:45.306 INFO:teuthology.orchestra.run.smithi203.stdout: # disable pg_autoscaler by default for new pools 2023-08-09T15:11:45.306 INFO:teuthology.orchestra.run.smithi203.stdout: osd_pool_default_pg_autoscale_mode = off 2023-08-09T15:11:45.306 INFO:teuthology.orchestra.run.smithi203.stdout: 2023-08-09T15:11:45.306 INFO:teuthology.orchestra.run.smithi203.stdout: osd pool default size = 2 2023-08-09T15:11:45.307 INFO:teuthology.orchestra.run.smithi203.stdout: 2023-08-09T15:11:45.307 INFO:teuthology.orchestra.run.smithi203.stdout: mon osd allow primary affinity = true 2023-08-09T15:11:45.307 INFO:teuthology.orchestra.run.smithi203.stdout: mon osd allow pg remap = true 2023-08-09T15:11:45.307 INFO:teuthology.orchestra.run.smithi203.stdout: mon warn on legacy crush tunables = false 2023-08-09T15:11:45.307 INFO:teuthology.orchestra.run.smithi203.stdout: mon warn on crush straw calc version zero = false 2023-08-09T15:11:45.307 INFO:teuthology.orchestra.run.smithi203.stdout: mon warn on no sortbitwise = false 2023-08-09T15:11:45.307 INFO:teuthology.orchestra.run.smithi203.stdout: mon warn on osd down out interval zero = false 2023-08-09T15:11:45.307 INFO:teuthology.orchestra.run.smithi203.stdout: mon warn on too few osds = false 2023-08-09T15:11:45.308 INFO:teuthology.orchestra.run.smithi203.stdout: mon_warn_on_pool_pg_num_not_power_of_two = false 2023-08-09T15:11:45.308 INFO:teuthology.orchestra.run.smithi203.stdout: mon_warn_on_pool_no_redundancy = false 2023-08-09T15:11:45.308 INFO:teuthology.orchestra.run.smithi203.stdout: mon_allow_pool_size_one = true 2023-08-09T15:11:45.308 INFO:teuthology.orchestra.run.smithi203.stdout: 2023-08-09T15:11:45.308 INFO:teuthology.orchestra.run.smithi203.stdout: osd pool default erasure code profile = plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd 2023-08-09T15:11:45.309 INFO:teuthology.orchestra.run.smithi203.stdout: 2023-08-09T15:11:45.309 INFO:teuthology.orchestra.run.smithi203.stdout: osd default data pool replay window = 5 2023-08-09T15:11:45.309 INFO:teuthology.orchestra.run.smithi203.stdout: 2023-08-09T15:11:45.309 INFO:teuthology.orchestra.run.smithi203.stdout: mon allow pool delete = true 2023-08-09T15:11:45.309 INFO:teuthology.orchestra.run.smithi203.stdout: 2023-08-09T15:11:45.309 INFO:teuthology.orchestra.run.smithi203.stdout: mon cluster log file level = debug 2023-08-09T15:11:45.309 INFO:teuthology.orchestra.run.smithi203.stdout: debug asserts on shutdown = true 2023-08-09T15:11:45.310 INFO:teuthology.orchestra.run.smithi203.stdout: mon health detail to clog = false 2023-08-09T15:11:45.310 INFO:teuthology.orchestra.run.smithi203.stdout: mon host = "172.21.15.43,172.21.15.203,[v2:172.21.15.203:3301,v1:172.21.15.203:6790]" 2023-08-09T15:11:45.310 INFO:teuthology.orchestra.run.smithi203.stdout: fsid = daac87ee-1337-4250-a955-aea4e408a39b 2023-08-09T15:11:45.310 INFO:teuthology.orchestra.run.smithi203.stdout: 2023-08-09T15:11:45.310 INFO:teuthology.orchestra.run.smithi203.stdout:[osd] 2023-08-09T15:11:45.310 INFO:teuthology.orchestra.run.smithi203.stdout: osd journal size = 100 2023-08-09T15:11:45.310 INFO:teuthology.orchestra.run.smithi203.stdout: 2023-08-09T15:11:45.310 INFO:teuthology.orchestra.run.smithi203.stdout: osd scrub load threshold = 5.0 2023-08-09T15:11:45.311 INFO:teuthology.orchestra.run.smithi203.stdout: osd scrub max interval = 600 2023-08-09T15:11:45.311 INFO:teuthology.orchestra.run.smithi203.stdout: osd mclock profile = high_recovery_ops 2023-08-09T15:11:45.311 INFO:teuthology.orchestra.run.smithi203.stdout: 2023-08-09T15:11:45.311 INFO:teuthology.orchestra.run.smithi203.stdout: osd recover clone overlap = true 2023-08-09T15:11:45.311 INFO:teuthology.orchestra.run.smithi203.stdout: osd recovery max chunk = 1048576 2023-08-09T15:11:45.311 INFO:teuthology.orchestra.run.smithi203.stdout: 2023-08-09T15:11:45.311 INFO:teuthology.orchestra.run.smithi203.stdout: osd debug shutdown = true 2023-08-09T15:11:45.311 INFO:teuthology.orchestra.run.smithi203.stdout: osd debug op order = true 2023-08-09T15:11:45.312 INFO:teuthology.orchestra.run.smithi203.stdout: osd debug verify stray on activate = true 2023-08-09T15:11:45.312 INFO:teuthology.orchestra.run.smithi203.stdout: 2023-08-09T15:11:45.312 INFO:teuthology.orchestra.run.smithi203.stdout: osd open classes on start = true 2023-08-09T15:11:45.312 INFO:teuthology.orchestra.run.smithi203.stdout: osd debug pg log writeout = true 2023-08-09T15:11:45.312 INFO:teuthology.orchestra.run.smithi203.stdout: 2023-08-09T15:11:45.312 INFO:teuthology.orchestra.run.smithi203.stdout: osd deep scrub update digest min age = 30 2023-08-09T15:11:45.312 INFO:teuthology.orchestra.run.smithi203.stdout: 2023-08-09T15:11:45.312 INFO:teuthology.orchestra.run.smithi203.stdout: osd map max advance = 10 2023-08-09T15:11:45.313 INFO:teuthology.orchestra.run.smithi203.stdout: 2023-08-09T15:11:45.313 INFO:teuthology.orchestra.run.smithi203.stdout: journal zero on create = true 2023-08-09T15:11:45.313 INFO:teuthology.orchestra.run.smithi203.stdout: 2023-08-09T15:11:45.313 INFO:teuthology.orchestra.run.smithi203.stdout: filestore ondisk finisher threads = 3 2023-08-09T15:11:45.313 INFO:teuthology.orchestra.run.smithi203.stdout: filestore apply finisher threads = 3 2023-08-09T15:11:45.313 INFO:teuthology.orchestra.run.smithi203.stdout: 2023-08-09T15:11:45.313 INFO:teuthology.orchestra.run.smithi203.stdout: bdev debug aio = true 2023-08-09T15:11:45.313 INFO:teuthology.orchestra.run.smithi203.stdout: osd debug misdirected ops = true 2023-08-09T15:11:45.314 INFO:teuthology.orchestra.run.smithi203.stdout: bluestore block size = 96636764160 2023-08-09T15:11:45.314 INFO:teuthology.orchestra.run.smithi203.stdout: bluestore fsck on mount = True 2023-08-09T15:11:45.314 INFO:teuthology.orchestra.run.smithi203.stdout: debug bluefs = 20 2023-08-09T15:11:45.314 INFO:teuthology.orchestra.run.smithi203.stdout: debug bluestore = 20 2023-08-09T15:11:45.314 INFO:teuthology.orchestra.run.smithi203.stdout: debug ms = 1 2023-08-09T15:11:45.314 INFO:teuthology.orchestra.run.smithi203.stdout: debug osd = 20 2023-08-09T15:11:45.314 INFO:teuthology.orchestra.run.smithi203.stdout: debug rocksdb = 10 2023-08-09T15:11:45.314 INFO:teuthology.orchestra.run.smithi203.stdout: mon osd backfillfull_ratio = 0.85 2023-08-09T15:11:45.315 INFO:teuthology.orchestra.run.smithi203.stdout: mon osd full ratio = 0.9 2023-08-09T15:11:45.315 INFO:teuthology.orchestra.run.smithi203.stdout: mon osd nearfull ratio = 0.8 2023-08-09T15:11:45.315 INFO:teuthology.orchestra.run.smithi203.stdout: osd failsafe full ratio = 0.95 2023-08-09T15:11:45.315 INFO:teuthology.orchestra.run.smithi203.stdout: osd objectstore = bluestore 2023-08-09T15:11:45.315 INFO:teuthology.orchestra.run.smithi203.stdout: osd op complaint time = 180 2023-08-09T15:11:45.315 INFO:teuthology.orchestra.run.smithi203.stdout: 2023-08-09T15:11:45.315 INFO:teuthology.orchestra.run.smithi203.stdout:[mgr] 2023-08-09T15:11:45.315 INFO:teuthology.orchestra.run.smithi203.stdout: debug ms = 1 2023-08-09T15:11:45.316 INFO:teuthology.orchestra.run.smithi203.stdout: debug mgr = 20 2023-08-09T15:11:45.316 INFO:teuthology.orchestra.run.smithi203.stdout: debug mon = 20 2023-08-09T15:11:45.316 INFO:teuthology.orchestra.run.smithi203.stdout: debug auth = 20 2023-08-09T15:11:45.316 INFO:teuthology.orchestra.run.smithi203.stdout: mon reweight min pgs per osd = 4 2023-08-09T15:11:45.316 INFO:teuthology.orchestra.run.smithi203.stdout: mon reweight min bytes per osd = 10 2023-08-09T15:11:45.316 INFO:teuthology.orchestra.run.smithi203.stdout: mgr/telemetry/nag = false 2023-08-09T15:11:45.316 INFO:teuthology.orchestra.run.smithi203.stdout: debug client = 20 2023-08-09T15:11:45.317 INFO:teuthology.orchestra.run.smithi203.stdout: debug finisher = 20 2023-08-09T15:11:45.317 INFO:teuthology.orchestra.run.smithi203.stdout: 2023-08-09T15:11:45.317 INFO:teuthology.orchestra.run.smithi203.stdout:[mon] 2023-08-09T15:11:45.317 INFO:teuthology.orchestra.run.smithi203.stdout: debug ms = 1 2023-08-09T15:11:45.317 INFO:teuthology.orchestra.run.smithi203.stdout: debug mon = 20 2023-08-09T15:11:45.317 INFO:teuthology.orchestra.run.smithi203.stdout: debug paxos = 20 2023-08-09T15:11:45.317 INFO:teuthology.orchestra.run.smithi203.stdout: debug auth = 20 2023-08-09T15:11:45.317 INFO:teuthology.orchestra.run.smithi203.stdout: mon data avail warn = 5 2023-08-09T15:11:45.318 INFO:teuthology.orchestra.run.smithi203.stdout: mon mgr mkfs grace = 240 2023-08-09T15:11:45.318 INFO:teuthology.orchestra.run.smithi203.stdout: mon reweight min pgs per osd = 4 2023-08-09T15:11:45.318 INFO:teuthology.orchestra.run.smithi203.stdout: mon osd reporter subtree level = osd 2023-08-09T15:11:45.318 INFO:teuthology.orchestra.run.smithi203.stdout: mon osd prime pg temp = true 2023-08-09T15:11:45.318 INFO:teuthology.orchestra.run.smithi203.stdout: mon reweight min bytes per osd = 10 2023-08-09T15:11:45.318 INFO:teuthology.orchestra.run.smithi203.stdout: 2023-08-09T15:11:45.318 INFO:teuthology.orchestra.run.smithi203.stdout: # rotate auth tickets quickly to exercise renewal paths 2023-08-09T15:11:45.318 INFO:teuthology.orchestra.run.smithi203.stdout: auth mon ticket ttl = 660 # 11m 2023-08-09T15:11:45.319 INFO:teuthology.orchestra.run.smithi203.stdout: auth service ticket ttl = 240 # 4m 2023-08-09T15:11:45.319 INFO:teuthology.orchestra.run.smithi203.stdout: 2023-08-09T15:11:45.319 INFO:teuthology.orchestra.run.smithi203.stdout: # don't complain about insecure global_id in the test suite 2023-08-09T15:11:45.319 INFO:teuthology.orchestra.run.smithi203.stdout: mon_warn_on_insecure_global_id_reclaim = false 2023-08-09T15:11:45.319 INFO:teuthology.orchestra.run.smithi203.stdout: mon_warn_on_insecure_global_id_reclaim_allowed = false 2023-08-09T15:11:45.319 INFO:teuthology.orchestra.run.smithi203.stdout: 2023-08-09T15:11:45.319 INFO:teuthology.orchestra.run.smithi203.stdout: # 1m isn't quite enough 2023-08-09T15:11:45.319 INFO:teuthology.orchestra.run.smithi203.stdout: mon_down_mkfs_grace = 2m 2023-08-09T15:11:45.320 INFO:teuthology.orchestra.run.smithi203.stdout: 2023-08-09T15:11:45.320 INFO:teuthology.orchestra.run.smithi203.stdout: mon_warn_on_filestore_osds = false 2023-08-09T15:11:45.320 INFO:teuthology.orchestra.run.smithi203.stdout: mon op complaint time = 120 2023-08-09T15:11:45.320 INFO:teuthology.orchestra.run.smithi203.stdout: 2023-08-09T15:11:45.320 INFO:teuthology.orchestra.run.smithi203.stdout:[client] 2023-08-09T15:11:45.320 INFO:teuthology.orchestra.run.smithi203.stdout: rgw cache enabled = true 2023-08-09T15:11:45.320 INFO:teuthology.orchestra.run.smithi203.stdout: rgw enable ops log = true 2023-08-09T15:11:45.320 INFO:teuthology.orchestra.run.smithi203.stdout: rgw enable usage log = true 2023-08-09T15:11:45.321 INFO:teuthology.orchestra.run.smithi203.stdout: log file = /var/log/ceph/$cluster-$name.$pid.log 2023-08-09T15:11:45.321 INFO:teuthology.orchestra.run.smithi203.stdout: admin socket = /var/run/ceph/$cluster-$name.$pid.asok 2023-08-09T15:11:45.321 INFO:teuthology.orchestra.run.smithi203.stdout: client mount timeout = 600 2023-08-09T15:11:45.321 INFO:teuthology.orchestra.run.smithi203.stdout: debug client = 20 2023-08-09T15:11:45.321 INFO:teuthology.orchestra.run.smithi203.stdout: debug ms = 1 2023-08-09T15:11:45.321 INFO:teuthology.orchestra.run.smithi203.stdout: pid file = "" 2023-08-09T15:11:45.321 INFO:teuthology.orchestra.run.smithi203.stdout: rados mon op timeout = 900 2023-08-09T15:11:45.321 INFO:teuthology.orchestra.run.smithi203.stdout: rados osd op timeout = 900 2023-08-09T15:11:45.322 INFO:teuthology.orchestra.run.smithi203.stdout: client snapdir = .snap 2023-08-09T15:11:45.322 INFO:teuthology.orchestra.run.smithi203.stdout:[mon.a] 2023-08-09T15:11:45.322 INFO:teuthology.orchestra.run.smithi203.stdout:[mon.b] 2023-08-09T15:11:45.322 INFO:teuthology.orchestra.run.smithi203.stdout:[mon.c] 2023-08-09T15:11:45.322 INFO:teuthology.orchestra.run.smithi203.stdout:[mds.a] 2023-08-09T15:11:45.322 INFO:teuthology.orchestra.run.smithi203.stdout:[mds.c] 2023-08-09T15:11:45.322 INFO:teuthology.orchestra.run.smithi203.stdout:[mds.b] 2023-08-09T15:11:45.322 INFO:teuthology.orchestra.run.smithi203.stdout:[mds.d] 2023-08-09T15:11:45.323 INFO:teuthology.orchestra.run.smithi203.stdout:[mds] 2023-08-09T15:11:45.323 INFO:teuthology.orchestra.run.smithi203.stdout: debug mds = 20 2023-08-09T15:11:45.323 INFO:teuthology.orchestra.run.smithi203.stdout: debug mds balancer = 20 2023-08-09T15:11:45.323 INFO:teuthology.orchestra.run.smithi203.stdout: debug ms = 1 2023-08-09T15:11:45.323 INFO:teuthology.orchestra.run.smithi203.stdout: mds debug frag = True 2023-08-09T15:11:45.323 INFO:teuthology.orchestra.run.smithi203.stdout: mds debug scatterstat = True 2023-08-09T15:11:45.323 INFO:teuthology.orchestra.run.smithi203.stdout: mds op complaint time = 180 2023-08-09T15:11:45.323 INFO:teuthology.orchestra.run.smithi203.stdout: mds verify scatter = True 2023-08-09T15:11:45.324 INFO:teuthology.orchestra.run.smithi203.stdout: osd op complaint time = 180 2023-08-09T15:11:45.324 INFO:teuthology.orchestra.run.smithi203.stdout: rados mon op timeout = 900 2023-08-09T15:11:45.324 INFO:teuthology.orchestra.run.smithi203.stdout: rados osd op timeout = 900 2023-08-09T15:11:45.324 INFO:teuthology.orchestra.run.smithi043.stdout:[global] 2023-08-09T15:11:45.325 INFO:teuthology.orchestra.run.smithi043.stdout: chdir = "" 2023-08-09T15:11:45.325 INFO:teuthology.orchestra.run.smithi043.stdout: pid file = /var/run/ceph/$cluster-$name.pid 2023-08-09T15:11:45.325 INFO:teuthology.orchestra.run.smithi043.stdout: auth supported = cephx 2023-08-09T15:11:45.325 INFO:teuthology.orchestra.run.smithi043.stdout: 2023-08-09T15:11:45.325 INFO:teuthology.orchestra.run.smithi043.stdout: filestore xattr use omap = true 2023-08-09T15:11:45.325 INFO:teuthology.orchestra.run.smithi043.stdout: 2023-08-09T15:11:45.325 INFO:teuthology.orchestra.run.smithi043.stdout: mon clock drift allowed = 1.000 2023-08-09T15:11:45.326 INFO:teuthology.orchestra.run.smithi043.stdout: 2023-08-09T15:11:45.326 INFO:teuthology.orchestra.run.smithi043.stdout: osd crush chooseleaf type = 0 2023-08-09T15:11:45.326 INFO:teuthology.orchestra.run.smithi043.stdout: auth debug = true 2023-08-09T15:11:45.326 INFO:teuthology.orchestra.run.smithi043.stdout: 2023-08-09T15:11:45.326 INFO:teuthology.orchestra.run.smithi043.stdout: ms die on old message = true 2023-08-09T15:11:45.326 INFO:teuthology.orchestra.run.smithi043.stdout: ms die on bug = true 2023-08-09T15:11:45.326 INFO:teuthology.orchestra.run.smithi043.stdout: 2023-08-09T15:11:45.326 INFO:teuthology.orchestra.run.smithi043.stdout: mon max pg per osd = 10000 # >= luminous 2023-08-09T15:11:45.327 INFO:teuthology.orchestra.run.smithi043.stdout: mon pg warn max object skew = 0 2023-08-09T15:11:45.327 INFO:teuthology.orchestra.run.smithi043.stdout: 2023-08-09T15:11:45.327 INFO:teuthology.orchestra.run.smithi043.stdout: # disable pg_autoscaler by default for new pools 2023-08-09T15:11:45.327 INFO:teuthology.orchestra.run.smithi043.stdout: osd_pool_default_pg_autoscale_mode = off 2023-08-09T15:11:45.327 INFO:teuthology.orchestra.run.smithi043.stdout: 2023-08-09T15:11:45.327 INFO:teuthology.orchestra.run.smithi043.stdout: osd pool default size = 2 2023-08-09T15:11:45.327 INFO:teuthology.orchestra.run.smithi043.stdout: 2023-08-09T15:11:45.327 INFO:teuthology.orchestra.run.smithi043.stdout: mon osd allow primary affinity = true 2023-08-09T15:11:45.328 INFO:teuthology.orchestra.run.smithi043.stdout: mon osd allow pg remap = true 2023-08-09T15:11:45.328 INFO:teuthology.orchestra.run.smithi043.stdout: mon warn on legacy crush tunables = false 2023-08-09T15:11:45.328 INFO:teuthology.orchestra.run.smithi043.stdout: mon warn on crush straw calc version zero = false 2023-08-09T15:11:45.328 INFO:teuthology.orchestra.run.smithi043.stdout: mon warn on no sortbitwise = false 2023-08-09T15:11:45.328 INFO:teuthology.orchestra.run.smithi043.stdout: mon warn on osd down out interval zero = false 2023-08-09T15:11:45.328 INFO:teuthology.orchestra.run.smithi043.stdout: mon warn on too few osds = false 2023-08-09T15:11:45.328 INFO:teuthology.orchestra.run.smithi043.stdout: mon_warn_on_pool_pg_num_not_power_of_two = false 2023-08-09T15:11:45.328 INFO:teuthology.orchestra.run.smithi043.stdout: mon_warn_on_pool_no_redundancy = false 2023-08-09T15:11:45.329 INFO:teuthology.orchestra.run.smithi043.stdout: mon_allow_pool_size_one = true 2023-08-09T15:11:45.329 INFO:teuthology.orchestra.run.smithi043.stdout: 2023-08-09T15:11:45.329 INFO:teuthology.orchestra.run.smithi043.stdout: osd pool default erasure code profile = plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd 2023-08-09T15:11:45.329 INFO:teuthology.orchestra.run.smithi043.stdout: 2023-08-09T15:11:45.329 INFO:teuthology.orchestra.run.smithi043.stdout: osd default data pool replay window = 5 2023-08-09T15:11:45.329 INFO:teuthology.orchestra.run.smithi043.stdout: 2023-08-09T15:11:45.329 INFO:teuthology.orchestra.run.smithi043.stdout: mon allow pool delete = true 2023-08-09T15:11:45.329 INFO:teuthology.orchestra.run.smithi043.stdout: 2023-08-09T15:11:45.330 INFO:teuthology.orchestra.run.smithi043.stdout: mon cluster log file level = debug 2023-08-09T15:11:45.330 INFO:teuthology.orchestra.run.smithi043.stdout: debug asserts on shutdown = true 2023-08-09T15:11:45.330 INFO:teuthology.orchestra.run.smithi043.stdout: mon health detail to clog = false 2023-08-09T15:11:45.330 INFO:teuthology.orchestra.run.smithi043.stdout: mon host = "172.21.15.43,172.21.15.203,[v2:172.21.15.203:3301,v1:172.21.15.203:6790]" 2023-08-09T15:11:45.330 INFO:teuthology.orchestra.run.smithi043.stdout: fsid = daac87ee-1337-4250-a955-aea4e408a39b 2023-08-09T15:11:45.330 INFO:teuthology.orchestra.run.smithi043.stdout: 2023-08-09T15:11:45.330 INFO:teuthology.orchestra.run.smithi043.stdout:[osd] 2023-08-09T15:11:45.330 INFO:teuthology.orchestra.run.smithi043.stdout: osd journal size = 100 2023-08-09T15:11:45.331 INFO:teuthology.orchestra.run.smithi043.stdout: 2023-08-09T15:11:45.331 INFO:teuthology.orchestra.run.smithi043.stdout: osd scrub load threshold = 5.0 2023-08-09T15:11:45.331 INFO:teuthology.orchestra.run.smithi043.stdout: osd scrub max interval = 600 2023-08-09T15:11:45.331 INFO:teuthology.orchestra.run.smithi043.stdout: osd mclock profile = high_recovery_ops 2023-08-09T15:11:45.331 INFO:teuthology.orchestra.run.smithi043.stdout: 2023-08-09T15:11:45.331 INFO:teuthology.orchestra.run.smithi043.stdout: osd recover clone overlap = true 2023-08-09T15:11:45.331 INFO:teuthology.orchestra.run.smithi043.stdout: osd recovery max chunk = 1048576 2023-08-09T15:11:45.331 INFO:teuthology.orchestra.run.smithi043.stdout: 2023-08-09T15:11:45.332 INFO:teuthology.orchestra.run.smithi043.stdout: osd debug shutdown = true 2023-08-09T15:11:45.332 INFO:teuthology.orchestra.run.smithi043.stdout: osd debug op order = true 2023-08-09T15:11:45.332 INFO:teuthology.orchestra.run.smithi043.stdout: osd debug verify stray on activate = true 2023-08-09T15:11:45.332 INFO:teuthology.orchestra.run.smithi043.stdout: 2023-08-09T15:11:45.332 INFO:teuthology.orchestra.run.smithi043.stdout: osd open classes on start = true 2023-08-09T15:11:45.332 INFO:teuthology.orchestra.run.smithi043.stdout: osd debug pg log writeout = true 2023-08-09T15:11:45.332 INFO:teuthology.orchestra.run.smithi043.stdout: 2023-08-09T15:11:45.332 INFO:teuthology.orchestra.run.smithi043.stdout: osd deep scrub update digest min age = 30 2023-08-09T15:11:45.333 INFO:teuthology.orchestra.run.smithi043.stdout: 2023-08-09T15:11:45.333 INFO:teuthology.orchestra.run.smithi043.stdout: osd map max advance = 10 2023-08-09T15:11:45.333 INFO:teuthology.orchestra.run.smithi043.stdout: 2023-08-09T15:11:45.333 INFO:teuthology.orchestra.run.smithi043.stdout: journal zero on create = true 2023-08-09T15:11:45.333 INFO:teuthology.orchestra.run.smithi043.stdout: 2023-08-09T15:11:45.333 INFO:teuthology.orchestra.run.smithi043.stdout: filestore ondisk finisher threads = 3 2023-08-09T15:11:45.333 INFO:teuthology.orchestra.run.smithi043.stdout: filestore apply finisher threads = 3 2023-08-09T15:11:45.333 INFO:teuthology.orchestra.run.smithi043.stdout: 2023-08-09T15:11:45.334 INFO:teuthology.orchestra.run.smithi043.stdout: bdev debug aio = true 2023-08-09T15:11:45.334 INFO:teuthology.orchestra.run.smithi043.stdout: osd debug misdirected ops = true 2023-08-09T15:11:45.334 INFO:teuthology.orchestra.run.smithi043.stdout: bluestore block size = 96636764160 2023-08-09T15:11:45.334 INFO:teuthology.orchestra.run.smithi043.stdout: bluestore fsck on mount = True 2023-08-09T15:11:45.334 INFO:teuthology.orchestra.run.smithi043.stdout: debug bluefs = 20 2023-08-09T15:11:45.334 INFO:teuthology.orchestra.run.smithi043.stdout: debug bluestore = 20 2023-08-09T15:11:45.334 INFO:teuthology.orchestra.run.smithi043.stdout: debug ms = 1 2023-08-09T15:11:45.334 INFO:teuthology.orchestra.run.smithi043.stdout: debug osd = 20 2023-08-09T15:11:45.335 INFO:teuthology.orchestra.run.smithi043.stdout: debug rocksdb = 10 2023-08-09T15:11:45.335 INFO:teuthology.orchestra.run.smithi043.stdout: mon osd backfillfull_ratio = 0.85 2023-08-09T15:11:45.335 INFO:teuthology.orchestra.run.smithi043.stdout: mon osd full ratio = 0.9 2023-08-09T15:11:45.335 INFO:teuthology.orchestra.run.smithi043.stdout: mon osd nearfull ratio = 0.8 2023-08-09T15:11:45.335 INFO:teuthology.orchestra.run.smithi043.stdout: osd failsafe full ratio = 0.95 2023-08-09T15:11:45.335 INFO:teuthology.orchestra.run.smithi043.stdout: osd objectstore = bluestore 2023-08-09T15:11:45.335 INFO:teuthology.orchestra.run.smithi043.stdout: osd op complaint time = 180 2023-08-09T15:11:45.335 INFO:teuthology.orchestra.run.smithi043.stdout: 2023-08-09T15:11:45.336 INFO:teuthology.orchestra.run.smithi043.stdout:[mgr] 2023-08-09T15:11:45.336 INFO:teuthology.orchestra.run.smithi043.stdout: debug ms = 1 2023-08-09T15:11:45.336 INFO:teuthology.orchestra.run.smithi043.stdout: debug mgr = 20 2023-08-09T15:11:45.336 INFO:teuthology.orchestra.run.smithi043.stdout: debug mon = 20 2023-08-09T15:11:45.336 INFO:teuthology.orchestra.run.smithi043.stdout: debug auth = 20 2023-08-09T15:11:45.336 INFO:teuthology.orchestra.run.smithi043.stdout: mon reweight min pgs per osd = 4 2023-08-09T15:11:45.336 INFO:teuthology.orchestra.run.smithi043.stdout: mon reweight min bytes per osd = 10 2023-08-09T15:11:45.336 INFO:teuthology.orchestra.run.smithi043.stdout: mgr/telemetry/nag = false 2023-08-09T15:11:45.337 INFO:teuthology.orchestra.run.smithi043.stdout: debug client = 20 2023-08-09T15:11:45.337 INFO:teuthology.orchestra.run.smithi043.stdout: debug finisher = 20 2023-08-09T15:11:45.337 INFO:teuthology.orchestra.run.smithi043.stdout: 2023-08-09T15:11:45.337 INFO:teuthology.orchestra.run.smithi043.stdout:[mon] 2023-08-09T15:11:45.337 INFO:teuthology.orchestra.run.smithi043.stdout: debug ms = 1 2023-08-09T15:11:45.337 INFO:teuthology.orchestra.run.smithi043.stdout: debug mon = 20 2023-08-09T15:11:45.337 INFO:teuthology.orchestra.run.smithi043.stdout: debug paxos = 20 2023-08-09T15:11:45.338 INFO:teuthology.orchestra.run.smithi043.stdout: debug auth = 20 2023-08-09T15:11:45.338 INFO:teuthology.orchestra.run.smithi043.stdout: mon data avail warn = 5 2023-08-09T15:11:45.338 INFO:teuthology.orchestra.run.smithi043.stdout: mon mgr mkfs grace = 240 2023-08-09T15:11:45.338 INFO:teuthology.orchestra.run.smithi043.stdout: mon reweight min pgs per osd = 4 2023-08-09T15:11:45.338 INFO:teuthology.orchestra.run.smithi043.stdout: mon osd reporter subtree level = osd 2023-08-09T15:11:45.338 INFO:teuthology.orchestra.run.smithi043.stdout: mon osd prime pg temp = true 2023-08-09T15:11:45.338 INFO:teuthology.orchestra.run.smithi043.stdout: mon reweight min bytes per osd = 10 2023-08-09T15:11:45.338 INFO:teuthology.orchestra.run.smithi043.stdout: 2023-08-09T15:11:45.338 INFO:teuthology.orchestra.run.smithi043.stdout: # rotate auth tickets quickly to exercise renewal paths 2023-08-09T15:11:45.339 INFO:teuthology.orchestra.run.smithi043.stdout: auth mon ticket ttl = 660 # 11m 2023-08-09T15:11:45.339 INFO:teuthology.orchestra.run.smithi043.stdout: auth service ticket ttl = 240 # 4m 2023-08-09T15:11:45.339 INFO:teuthology.orchestra.run.smithi043.stdout: 2023-08-09T15:11:45.339 INFO:teuthology.orchestra.run.smithi043.stdout: # don't complain about insecure global_id in the test suite 2023-08-09T15:11:45.339 INFO:teuthology.orchestra.run.smithi043.stdout: mon_warn_on_insecure_global_id_reclaim = false 2023-08-09T15:11:45.339 INFO:teuthology.orchestra.run.smithi043.stdout: mon_warn_on_insecure_global_id_reclaim_allowed = false 2023-08-09T15:11:45.339 INFO:teuthology.orchestra.run.smithi043.stdout: 2023-08-09T15:11:45.339 INFO:teuthology.orchestra.run.smithi043.stdout: # 1m isn't quite enough 2023-08-09T15:11:45.340 INFO:teuthology.orchestra.run.smithi043.stdout: mon_down_mkfs_grace = 2m 2023-08-09T15:11:45.340 INFO:teuthology.orchestra.run.smithi043.stdout: 2023-08-09T15:11:45.340 INFO:teuthology.orchestra.run.smithi043.stdout: mon_warn_on_filestore_osds = false 2023-08-09T15:11:45.340 INFO:teuthology.orchestra.run.smithi043.stdout: mon op complaint time = 120 2023-08-09T15:11:45.340 INFO:teuthology.orchestra.run.smithi043.stdout: 2023-08-09T15:11:45.340 INFO:teuthology.orchestra.run.smithi043.stdout:[client] 2023-08-09T15:11:45.340 INFO:teuthology.orchestra.run.smithi043.stdout: rgw cache enabled = true 2023-08-09T15:11:45.341 INFO:teuthology.orchestra.run.smithi043.stdout: rgw enable ops log = true 2023-08-09T15:11:45.341 INFO:teuthology.orchestra.run.smithi043.stdout: rgw enable usage log = true 2023-08-09T15:11:45.341 INFO:teuthology.orchestra.run.smithi043.stdout: log file = /var/log/ceph/$cluster-$name.$pid.log 2023-08-09T15:11:45.341 INFO:teuthology.orchestra.run.smithi043.stdout: admin socket = /var/run/ceph/$cluster-$name.$pid.asok 2023-08-09T15:11:45.341 INFO:teuthology.orchestra.run.smithi043.stdout: client mount timeout = 600 2023-08-09T15:11:45.341 INFO:teuthology.orchestra.run.smithi043.stdout: debug client = 20 2023-08-09T15:11:45.341 INFO:teuthology.orchestra.run.smithi043.stdout: debug ms = 1 2023-08-09T15:11:45.341 INFO:teuthology.orchestra.run.smithi043.stdout: pid file = "" 2023-08-09T15:11:45.342 INFO:teuthology.orchestra.run.smithi043.stdout: rados mon op timeout = 900 2023-08-09T15:11:45.342 INFO:teuthology.orchestra.run.smithi043.stdout: rados osd op timeout = 900 2023-08-09T15:11:45.342 INFO:teuthology.orchestra.run.smithi043.stdout: client snapdir = .snap 2023-08-09T15:11:45.342 INFO:teuthology.orchestra.run.smithi043.stdout:[mon.a] 2023-08-09T15:11:45.342 INFO:teuthology.orchestra.run.smithi043.stdout:[mon.b] 2023-08-09T15:11:45.342 INFO:teuthology.orchestra.run.smithi043.stdout:[mon.c] 2023-08-09T15:11:45.342 INFO:teuthology.orchestra.run.smithi043.stdout:[mds.a] 2023-08-09T15:11:45.342 INFO:teuthology.orchestra.run.smithi043.stdout:[mds.c] 2023-08-09T15:11:45.343 INFO:teuthology.orchestra.run.smithi043.stdout:[mds.b] 2023-08-09T15:11:45.343 INFO:teuthology.orchestra.run.smithi043.stdout:[mds.d] 2023-08-09T15:11:45.343 INFO:teuthology.orchestra.run.smithi043.stdout:[mds] 2023-08-09T15:11:45.343 INFO:teuthology.orchestra.run.smithi043.stdout: debug mds = 20 2023-08-09T15:11:45.343 INFO:teuthology.orchestra.run.smithi043.stdout: debug mds balancer = 20 2023-08-09T15:11:45.343 INFO:teuthology.orchestra.run.smithi043.stdout: debug ms = 1 2023-08-09T15:11:45.343 INFO:teuthology.orchestra.run.smithi043.stdout: mds debug frag = True 2023-08-09T15:11:45.343 INFO:teuthology.orchestra.run.smithi043.stdout: mds debug scatterstat = True 2023-08-09T15:11:45.344 INFO:teuthology.orchestra.run.smithi043.stdout: mds op complaint time = 180 2023-08-09T15:11:45.344 INFO:teuthology.orchestra.run.smithi043.stdout: mds verify scatter = True 2023-08-09T15:11:45.344 INFO:teuthology.orchestra.run.smithi043.stdout: osd op complaint time = 180 2023-08-09T15:11:45.344 INFO:teuthology.orchestra.run.smithi043.stdout: rados mon op timeout = 900 2023-08-09T15:11:45.344 INFO:teuthology.orchestra.run.smithi043.stdout: rados osd op timeout = 900 2023-08-09T15:11:45.361 DEBUG:teuthology.orchestra.run.smithi043:> set -ex 2023-08-09T15:11:45.362 DEBUG:teuthology.orchestra.run.smithi043:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-09T15:11:45.421 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-08-09T15:11:45.653 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:45.651+0000 7f14c3687640 1 Processor -- start 2023-08-09T15:11:45.653 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:45.651+0000 7f14c3687640 1 -- start start 2023-08-09T15:11:45.655 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:45.653+0000 7f14c3687640 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7f14bc06fe60 0x7f14bc0cdbe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T15:11:45.655 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:45.653+0000 7f14c3687640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f14bc15b310 0x7f14bc15b720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T15:11:45.655 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:45.653+0000 7f14c3687640 1 -- --> v1:172.21.15.43:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f14bc06c2c0 con 0x7f14bc06b4b0 2023-08-09T15:11:45.655 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:45.653+0000 7f14c3687640 1 -- --> v2:172.21.15.43:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f14bc15bc60 con 0x7f14bc15b310 2023-08-09T15:11:45.655 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:45.653+0000 7f14c3687640 1 -- --> v2:172.21.15.203:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f14bc0cb690 con 0x7f14bc06fe60 2023-08-09T15:11:45.655 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:45.653+0000 7f14c1c24640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f14bc15b310 0x7f14bc15b720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T15:11:45.655 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:45.653+0000 7f14c1c24640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f14bc15b310 0x7f14bc15b720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:50084/0 (socket says 172.21.15.43:50084) 2023-08-09T15:11:45.656 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:45.653+0000 7f14c1c24640 1 -- 172.21.15.43:0/1801978409 learned_addr learned my addr 172.21.15.43:0/1801978409 (peer_addr_for_me v2:172.21.15.43:0/0) 2023-08-09T15:11:45.656 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:45.653+0000 7f14c0c22640 1 --1- 172.21.15.43:0/1801978409 >> v1:172.21.15.43:6789/0 conn(0x7f14bc06b4b0 0x7f14bc15be10 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.43:6789/0 says I am v1:172.21.15.43:56284/0 (socket says 172.21.15.43:56284) 2023-08-09T15:11:45.657 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:45.654+0000 7f14c1423640 1 --2- 172.21.15.43:0/1801978409 >> v2:172.21.15.203:3300/0 conn(0x7f14bc06fe60 0x7f14bc0cdbe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T15:11:45.657 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:45.654+0000 7f14c1c24640 1 -- 172.21.15.43:0/1801978409 >> v2:172.21.15.203:3300/0 conn(0x7f14bc06fe60 msgr2=0x7f14bc0cdbe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-09T15:11:45.657 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:45.654+0000 7f14c1c24640 1 --2- 172.21.15.43:0/1801978409 >> v2:172.21.15.203:3300/0 conn(0x7f14bc06fe60 0x7f14bc0cdbe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T15:11:45.657 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:45.654+0000 7f14c1c24640 1 -- 172.21.15.43:0/1801978409 >> v1:172.21.15.43:6789/0 conn(0x7f14bc06b4b0 legacy=0x7f14bc15be10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T15:11:45.657 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:45.654+0000 7f14c1c24640 1 -- 172.21.15.43:0/1801978409 --> v2:172.21.15.43:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f14bc06ba20 con 0x7f14bc15b310 2023-08-09T15:11:45.658 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:45.654+0000 7f14c1423640 1 --2- 172.21.15.43:0/1801978409 >> v2:172.21.15.203:3300/0 conn(0x7f14bc06fe60 0x7f14bc0cdbe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-08-09T15:11:45.658 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:45.655+0000 7f14c1c24640 1 --2- 172.21.15.43:0/1801978409 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f14bc15b310 0x7f14bc15b720 secure :-1 s=READY pgs=2448 cs=0 l=1 rev1=1 crypto rx=0x7f14bc0caac0 tx=0x7f14b402bf70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=776bfc3ec74ce9f4 server_cookie=0 in_seq=0 out_seq=0 2023-08-09T15:11:45.658 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:45.655+0000 7f14b3fff640 1 -- 172.21.15.43:0/1801978409 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f14b402cb10 con 0x7f14bc15b310 2023-08-09T15:11:45.658 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:45.655+0000 7f14b3fff640 1 -- 172.21.15.43:0/1801978409 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f14b402cc70 con 0x7f14bc15b310 2023-08-09T15:11:45.658 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:45.655+0000 7f14b3fff640 1 -- 172.21.15.43:0/1801978409 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f14b40343f0 con 0x7f14bc15b310 2023-08-09T15:11:45.659 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:45.656+0000 7f14c3687640 1 -- 172.21.15.43:0/1801978409 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f14bc15b310 msgr2=0x7f14bc15b720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T15:11:45.659 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:45.656+0000 7f14c3687640 1 --2- 172.21.15.43:0/1801978409 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f14bc15b310 0x7f14bc15b720 secure :-1 s=READY pgs=2448 cs=0 l=1 rev1=1 crypto rx=0x7f14bc0caac0 tx=0x7f14b402bf70 comp rx=0 tx=0).stop 2023-08-09T15:11:45.660 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:45.656+0000 7f14c3687640 1 -- 172.21.15.43:0/1801978409 shutdown_connections 2023-08-09T15:11:45.660 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:45.656+0000 7f14c3687640 1 --2- 172.21.15.43:0/1801978409 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f14bc15b310 0x7f14bc15b720 unknown :-1 s=CLOSED pgs=2448 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T15:11:45.660 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:45.656+0000 7f14c3687640 1 --2- 172.21.15.43:0/1801978409 >> v2:172.21.15.203:3300/0 conn(0x7f14bc06fe60 0x7f14bc0cdbe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T15:11:45.660 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:45.656+0000 7f14c3687640 1 -- 172.21.15.43:0/1801978409 >> 172.21.15.43:0/1801978409 conn(0x7f14bc069d70 msgr2=0x7f14bc0cac70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-09T15:11:45.660 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:45.656+0000 7f14c3687640 1 -- 172.21.15.43:0/1801978409 shutdown_connections 2023-08-09T15:11:45.660 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:45.656+0000 7f14c3687640 1 -- 172.21.15.43:0/1801978409 wait complete. 2023-08-09T15:11:45.660 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:45.658+0000 7f14c3687640 1 Processor -- start 2023-08-09T15:11:45.661 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:45.658+0000 7f14c3687640 1 -- start start 2023-08-09T15:11:45.663 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:45.659+0000 7f14c3687640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f14bc06b4b0 0x7f14bc0c76c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T15:11:45.663 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:45.659+0000 7f14c3687640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f14bc06fe60 0x7f14bc13d040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T15:11:45.664 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:45.659+0000 7f14c3687640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f14bc15b310 0x7f14bc14d5a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T15:11:45.664 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:45.659+0000 7f14c3687640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f14bc14ed40 con 0x7f14bc06fe60 2023-08-09T15:11:45.664 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:45.659+0000 7f14c3687640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f14bc14e4a0 con 0x7f14bc15b310 2023-08-09T15:11:45.665 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:45.659+0000 7f14c3687640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f14bc14e780 con 0x7f14bc06b4b0 2023-08-09T15:11:45.665 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:45.659+0000 7f14c0c22640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f14bc06fe60 0x7f14bc13d040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T15:11:45.665 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:45.660+0000 7f14c0c22640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f14bc06fe60 0x7f14bc13d040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:50096/0 (socket says 172.21.15.43:50096) 2023-08-09T15:11:45.665 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:45.660+0000 7f14c0c22640 1 -- 172.21.15.43:0/1594428990 learned_addr learned my addr 172.21.15.43:0/1594428990 (peer_addr_for_me v2:172.21.15.43:0/0) 2023-08-09T15:11:45.666 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:45.660+0000 7f14c0c22640 1 -- 172.21.15.43:0/1594428990 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f14bc06b4b0 msgr2=0x7f14bc0c76c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T15:11:45.666 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:45.660+0000 7f14c0c22640 1 --2- 172.21.15.43:0/1594428990 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f14bc06b4b0 0x7f14bc0c76c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T15:11:45.666 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:45.660+0000 7f14c0c22640 1 -- 172.21.15.43:0/1594428990 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f14bc15b310 msgr2=0x7f14bc14d5a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T15:11:45.666 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:45.660+0000 7f14c1c24640 1 --2- 172.21.15.43:0/1594428990 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f14bc15b310 0x7f14bc14d5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T15:11:45.667 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:45.660+0000 7f14c0c22640 1 --2- 172.21.15.43:0/1594428990 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f14bc15b310 0x7f14bc14d5a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T15:11:45.667 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:45.660+0000 7f14c0c22640 1 -- 172.21.15.43:0/1594428990 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f14b4009b30 con 0x7f14bc06fe60 2023-08-09T15:11:45.667 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:45.660+0000 7f14c0c22640 1 --2- 172.21.15.43:0/1594428990 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f14bc06fe60 0x7f14bc13d040 secure :-1 s=READY pgs=2449 cs=0 l=1 rev1=1 crypto rx=0x7f14ac00c4d0 tx=0x7f14ac00c990 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-09T15:11:45.668 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:45.661+0000 7f14b1ffb640 1 -- 172.21.15.43:0/1594428990 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f14ac007480 con 0x7f14bc06fe60 2023-08-09T15:11:45.668 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:45.661+0000 7f14b1ffb640 1 -- 172.21.15.43:0/1594428990 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f14ac0075e0 con 0x7f14bc06fe60 2023-08-09T15:11:45.668 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:45.661+0000 7f14b1ffb640 1 -- 172.21.15.43:0/1594428990 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f14ac0049e0 con 0x7f14bc06fe60 2023-08-09T15:11:45.669 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:45.661+0000 7f14c3687640 1 -- 172.21.15.43:0/1594428990 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f14bc14ea60 con 0x7f14bc06fe60 2023-08-09T15:11:45.669 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:45.661+0000 7f14c3687640 1 -- 172.21.15.43:0/1594428990 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f14bc0c71a0 con 0x7f14bc06fe60 2023-08-09T15:11:45.670 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:45.662+0000 7f14c3687640 1 -- 172.21.15.43:0/1594428990 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f14bc0c6d20 con 0x7f14bc06fe60 2023-08-09T15:11:45.671 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:45.663+0000 7f14b1ffb640 1 -- 172.21.15.43:0/1594428990 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 122) v1 ==== 88461+0+0 (secure 0 0 0) 0x7f14ac004b40 con 0x7f14bc06fe60 2023-08-09T15:11:45.671 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:45.664+0000 7f14b1ffb640 1 --2- 172.21.15.43:0/1594428990 >> [v2:172.21.15.203:6832/2375771024,v1:172.21.15.203:6833/2375771024] conn(0x7f14a006e5d0 0x7f14a0070a90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T15:11:45.671 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:45.664+0000 7f14b1ffb640 1 -- 172.21.15.43:0/1594428990 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(224..224 src has 1..224) v4 ==== 6770+0+0 (secure 0 0 0) 0x7f14ac087ba0 con 0x7f14bc06fe60 2023-08-09T15:11:45.672 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:45.665+0000 7f14c1423640 1 --2- 172.21.15.43:0/1594428990 >> [v2:172.21.15.203:6832/2375771024,v1:172.21.15.203:6833/2375771024] conn(0x7f14a006e5d0 0x7f14a0070a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T15:11:45.672 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:45.666+0000 7f14c1423640 1 --2- 172.21.15.43:0/1594428990 >> [v2:172.21.15.203:6832/2375771024,v1:172.21.15.203:6833/2375771024] conn(0x7f14a006e5d0 0x7f14a0070a90 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7f14b8006fd0 tx=0x7f14b8008040 comp rx=0 tx=0).ready entity=mgr.9103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-09T15:11:45.672 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:45.668+0000 7f14b1ffb640 1 -- 172.21.15.43:0/1594428990 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178066 (secure 0 0 0) 0x7f14bc0c6d20 con 0x7f14bc06fe60 2023-08-09T15:11:45.889 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:45.887+0000 7f14c3687640 1 -- 172.21.15.43:0/1594428990 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f14bc0c6d20 con 0x7f14bc06fe60 2023-08-09T15:11:45.892 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:45.891+0000 7f14b1ffb640 1 -- 172.21.15.43:0/1594428990 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v224) v1 ==== 74+0+19077 (secure 0 0 0) 0x7f14bc0c6d20 con 0x7f14bc06fe60 2023-08-09T15:11:45.893 INFO:teuthology.orchestra.run.smithi043.stdout: 2023-08-09T15:11:45.893 INFO:teuthology.orchestra.run.smithi043.stdout:{"epoch":224,"fsid":"daac87ee-1337-4250-a955-aea4e408a39b","created":"2023-08-09T13:33:41.701198+0000","modified":"2023-08-09T15:11:43.130258+0000","last_up_change":"2023-08-09T13:33:53.783453+0000","last_in_change":"2023-08-09T13:33:45.148904+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":27,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":"rbd","create_time":"2023-08-09T13:33:54.365137+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":".mgr","create_time":"2023-08-09T13:33:54.831803+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":26,"pool_name":"cephfs_metadata","create_time":"2023-08-09T15:11:26.544757+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"222","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":27,"pool_name":"cephfs_data","create_time":"2023-08-09T15:11:27.194686+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"223","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.6299999952316284,"score_stable":1.6299999952316284,"optimal_score":1,"raw_score_acting":1.6299999952316284,"raw_score_stable":1.6299999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"cbd49ecd-e896-4fbd-864e-5c37a829b165","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":221,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6800","nonce":502667335},{"type":"v1","addr":"172.21.15.43:6801","nonce":502667335}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6802","nonce":502667335},{"type":"v1","addr":"172.21.15.43:6803","nonce":502667335}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6806","nonce":502667335},{"type":"v1","addr":"172.21.15.43:6807","nonce":502667335}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6804","nonce":502667335},{"type":"v1","addr":"172.21.15.43:6805","nonce":502667335}]},"public_addr":"172.21.15.43:6801/502667335","cluster_addr":"172.21.15.43:6803/502667335","heartbeat_back_addr":"172.21.15.43:6807/502667335","heartbeat_front_addr":"172.21.15.43:6805/502667335","state":["exists","up"]},{"osd":1,"uuid":"75c7e0fe-1ec7-4fe8-a2f2-3661a65b8500","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":221,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6808","nonce":3289134610},{"type":"v1","addr":"172.21.15.43:6809","nonce":3289134610}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6810","nonce":3289134610},{"type":"v1","addr":"172.21.15.43:6812","nonce":3289134610}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6821","nonce":3289134610},{"type":"v1","addr":"172.21.15.43:6824","nonce":3289134610}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6815","nonce":3289134610},{"type":"v1","addr":"172.21.15.43:6818","nonce":3289134610}]},"public_addr":"172.21.15.43:6809/3289134610","cluster_addr":"172.21.15.43:6812/3289134610","heartbeat_back_addr":"172.21.15.43:6824/3289134610","heartbeat_front_addr":"172.21.15.43:6818/3289134610","state":["exists","up"]},{"osd":2,"uuid":"bb5c41e9-7f5f-4125-9b01-3bb8bfc2ea83","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":221,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6811","nonce":4079709736},{"type":"v1","addr":"172.21.15.43:6814","nonce":4079709736}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6817","nonce":4079709736},{"type":"v1","addr":"172.21.15.43:6820","nonce":4079709736}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6829","nonce":4079709736},{"type":"v1","addr":"172.21.15.43:6831","nonce":4079709736}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6823","nonce":4079709736},{"type":"v1","addr":"172.21.15.43:6826","nonce":4079709736}]},"public_addr":"172.21.15.43:6814/4079709736","cluster_addr":"172.21.15.43:6820/4079709736","heartbeat_back_addr":"172.21.15.43:6831/4079709736","heartbeat_front_addr":"172.21.15.43:6826/4079709736","state":["exists","up"]},{"osd":3,"uuid":"bfe4c8b4-694a-465c-87e6-2b2aab852613","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":221,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6813","nonce":1709538759},{"type":"v1","addr":"172.21.15.43:6816","nonce":1709538759}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6819","nonce":1709538759},{"type":"v1","addr":"172.21.15.43:6822","nonce":1709538759}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6828","nonce":1709538759},{"type":"v1","addr":"172.21.15.43:6830","nonce":1709538759}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6825","nonce":1709538759},{"type":"v1","addr":"172.21.15.43:6827","nonce":1709538759}]},"public_addr":"172.21.15.43:6816/1709538759","cluster_addr":"172.21.15.43:6822/1709538759","heartbeat_back_addr":"172.21.15.43:6830/1709538759","heartbeat_front_addr":"172.21.15.43:6827/1709538759","state":["exists","up"]},{"osd":4,"uuid":"fa2e8a77-aba1-4756-853e-0d174a5bdeba","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":221,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6800","nonce":4196866481},{"type":"v1","addr":"172.21.15.203:6801","nonce":4196866481}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6802","nonce":4196866481},{"type":"v1","addr":"172.21.15.203:6803","nonce":4196866481}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6806","nonce":4196866481},{"type":"v1","addr":"172.21.15.203:6807","nonce":4196866481}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6804","nonce":4196866481},{"type":"v1","addr":"172.21.15.203:6805","nonce":4196866481}]},"public_addr":"172.21.15.203:6801/4196866481","cluster_addr":"172.21.15.203:6803/4196866481","heartbeat_back_addr":"172.21.15.203:6807/4196866481","heartbeat_front_addr":"172.21.15.203:6805/4196866481","state":["exists","up"]},{"osd":5,"uuid":"81811e4b-4d24-4f51-a82c-f1d700a2a00d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":221,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6816","nonce":3059290081},{"type":"v1","addr":"172.21.15.203:6817","nonce":3059290081}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6818","nonce":3059290081},{"type":"v1","addr":"172.21.15.203:6819","nonce":3059290081}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6822","nonce":3059290081},{"type":"v1","addr":"172.21.15.203:6823","nonce":3059290081}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6820","nonce":3059290081},{"type":"v1","addr":"172.21.15.203:6821","nonce":3059290081}]},"public_addr":"172.21.15.203:6817/3059290081","cluster_addr":"172.21.15.203:6819/3059290081","heartbeat_back_addr":"172.21.15.203:6823/3059290081","heartbeat_front_addr":"172.21.15.203:6821/3059290081","state":["exists","up"]},{"osd":6,"uuid":"307bac4d-6a5b-443d-af75-56d695d7674f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":221,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6824","nonce":512155569},{"type":"v1","addr":"172.21.15.203:6825","nonce":512155569}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6826","nonce":512155569},{"type":"v1","addr":"172.21.15.203:6827","nonce":512155569}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6830","nonce":512155569},{"type":"v1","addr":"172.21.15.203:6831","nonce":512155569}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6828","nonce":512155569},{"type":"v1","addr":"172.21.15.203:6829","nonce":512155569}]},"public_addr":"172.21.15.203:6825/512155569","cluster_addr":"172.21.15.203:6827/512155569","heartbeat_back_addr":"172.21.15.203:6831/512155569","heartbeat_front_addr":"172.21.15.203:6829/512155569","state":["exists","up"]},{"osd":7,"uuid":"a95dc7ca-9f1b-4295-af70-68a22a2d7d5e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":221,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6808","nonce":3331450438},{"type":"v1","addr":"172.21.15.203:6809","nonce":3331450438}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6810","nonce":3331450438},{"type":"v1","addr":"172.21.15.203:6811","nonce":3331450438}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6814","nonce":3331450438},{"type":"v1","addr":"172.21.15.203:6815","nonce":3331450438}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6812","nonce":3331450438},{"type":"v1","addr":"172.21.15.203:6813","nonce":3331450438}]},"public_addr":"172.21.15.203:6809/3331450438","cluster_addr":"172.21.15.203:6811/3331450438","heartbeat_back_addr":"172.21.15.203:6815/3331450438","heartbeat_front_addr":"172.21.15.203:6813/3331450438","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-09T13:33:50.547652+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-09T13:33:50.589469+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-09T13:33:50.561724+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-09T13:33:50.599650+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-09T13:33:50.411855+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-09T13:33:50.219681+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-09T13:33:50.504664+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-09T13:33:50.441248+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.203:6832/4034093268":"2023-08-10T15:11:43.130145+0000","172.21.15.203:6833/4034093268":"2023-08-10T15:11:43.130145+0000","172.21.15.203:0/1654193324":"2023-08-10T15:11:43.130145+0000","172.21.15.203:0/4140323243":"2023-08-10T15:11:43.130145+0000","172.21.15.203:0/450863258":"2023-08-10T15:11:43.130145+0000","172.21.15.203:0/134877587":"2023-08-10T15:11:43.130145+0000","172.21.15.203:0/2330869600":"2023-08-10T15:11:43.130145+0000","172.21.15.203:0/2203818837":"2023-08-10T15:11:43.130145+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":7,"snaps":[{"begin":104,"length":1},{"begin":221,"length":1}]}],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-08-09T15:11:45.895 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:45.893+0000 7f14c3687640 1 -- 172.21.15.43:0/1594428990 >> [v2:172.21.15.203:6832/2375771024,v1:172.21.15.203:6833/2375771024] conn(0x7f14a006e5d0 msgr2=0x7f14a0070a90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T15:11:45.895 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:45.893+0000 7f14c3687640 1 --2- 172.21.15.43:0/1594428990 >> [v2:172.21.15.203:6832/2375771024,v1:172.21.15.203:6833/2375771024] conn(0x7f14a006e5d0 0x7f14a0070a90 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7f14b8006fd0 tx=0x7f14b8008040 comp rx=0 tx=0).stop 2023-08-09T15:11:45.895 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:45.893+0000 7f14c3687640 1 -- 172.21.15.43:0/1594428990 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f14bc06fe60 msgr2=0x7f14bc13d040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T15:11:45.895 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:45.893+0000 7f14c3687640 1 --2- 172.21.15.43:0/1594428990 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f14bc06fe60 0x7f14bc13d040 secure :-1 s=READY pgs=2449 cs=0 l=1 rev1=1 crypto rx=0x7f14ac00c4d0 tx=0x7f14ac00c990 comp rx=0 tx=0).stop 2023-08-09T15:11:45.896 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:45.894+0000 7f14c3687640 1 -- 172.21.15.43:0/1594428990 shutdown_connections 2023-08-09T15:11:45.896 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:45.894+0000 7f14c3687640 1 --2- 172.21.15.43:0/1594428990 >> [v2:172.21.15.203:6832/2375771024,v1:172.21.15.203:6833/2375771024] conn(0x7f14a006e5d0 0x7f14a0070a90 unknown :-1 s=CLOSED pgs=25 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T15:11:45.896 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:45.894+0000 7f14c3687640 1 --2- 172.21.15.43:0/1594428990 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f14bc15b310 0x7f14bc14d5a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T15:11:45.896 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:45.894+0000 7f14c3687640 1 --2- 172.21.15.43:0/1594428990 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f14bc06fe60 0x7f14bc13d040 unknown :-1 s=CLOSED pgs=2449 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T15:11:45.896 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:45.894+0000 7f14c3687640 1 --2- 172.21.15.43:0/1594428990 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f14bc06b4b0 0x7f14bc0c76c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T15:11:45.896 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:45.894+0000 7f14c3687640 1 -- 172.21.15.43:0/1594428990 >> 172.21.15.43:0/1594428990 conn(0x7f14bc069d70 msgr2=0x7f14bc0b14c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-09T15:11:45.896 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:45.894+0000 7f14c3687640 1 -- 172.21.15.43:0/1594428990 shutdown_connections 2023-08-09T15:11:45.897 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:45.894+0000 7f14c3687640 1 -- 172.21.15.43:0/1594428990 wait complete. 2023-08-09T15:11:45.915 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': 'rbd', 'create_time': '2023-08-09T13:33:54.365137+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}, 'read_balance': {'score_acting': 2, 'score_stable': 2, 'optimal_score': 1, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': '.mgr', 'create_time': '2023-08-09T13:33:54.831803+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 26, 'pool_name': 'cephfs_metadata', 'create_time': '2023-08-09T15:11:26.544757+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '222', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.75, 'score_stable': 1.75, 'optimal_score': 1, 'raw_score_acting': 1.75, 'raw_score_stable': 1.75, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 27, 'pool_name': 'cephfs_data', 'create_time': '2023-08-09T15:11:27.194686+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '223', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.6299999952316284, 'score_stable': 1.6299999952316284, 'optimal_score': 1, 'raw_score_acting': 1.6299999952316284, 'raw_score_stable': 1.6299999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-08-09T15:11:45.916 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get rbd pg_num 2023-08-09T15:11:46.146 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:46.144+0000 7fda9a134640 1 Processor -- start 2023-08-09T15:11:46.147 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:46.145+0000 7fda9a134640 1 -- start start 2023-08-09T15:11:46.148 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:46.147+0000 7fda9a134640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7fda940d1840 0x7fda94167580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T15:11:46.148 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:46.147+0000 7fda9a134640 1 -- --> v1:172.21.15.43:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fda94160e00 con 0x7fda940d1df0 2023-08-09T15:11:46.148 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:46.147+0000 7fda9a134640 1 -- --> v1:172.21.15.203:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fda941601f0 con 0x7fda9415c870 2023-08-09T15:11:46.148 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:46.147+0000 7fda9a134640 1 -- --> v2:172.21.15.43:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fda941604d0 con 0x7fda940d1840 2023-08-09T15:11:46.149 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:46.147+0000 7fda99933640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7fda940d1840 0x7fda94167580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T15:11:46.149 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:46.147+0000 7fda98931640 1 --1- >> v1:172.21.15.43:6789/0 conn(0x7fda940d1df0 0x7fda940cd1a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.43:6789/0 says I am v1:172.21.15.43:56296/0 (socket says 172.21.15.43:56296) 2023-08-09T15:11:46.149 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:46.147+0000 7fda98931640 1 -- 172.21.15.43:0/2273733156 learned_addr learned my addr 172.21.15.43:0/2273733156 (peer_addr_for_me v1:172.21.15.43:0/0) 2023-08-09T15:11:46.150 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:46.148+0000 7fda99933640 1 -- 172.21.15.43:0/2273733156 >> v1:172.21.15.203:6789/0 conn(0x7fda9415c870 legacy=0x7fda940b7d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T15:11:46.150 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:46.148+0000 7fda99933640 1 -- 172.21.15.43:0/2273733156 >> v1:172.21.15.43:6789/0 conn(0x7fda940d1df0 legacy=0x7fda940cd1a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T15:11:46.150 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:46.148+0000 7fda99933640 1 -- 172.21.15.43:0/2273733156 --> v2:172.21.15.43:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fda941607b0 con 0x7fda940d1840 2023-08-09T15:11:46.151 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:46.149+0000 7fda99933640 1 --2- 172.21.15.43:0/2273733156 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fda940d1840 0x7fda94167580 secure :-1 s=READY pgs=2451 cs=0 l=1 rev1=1 crypto rx=0x7fda8c009d90 tx=0x7fda8c02bf40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9449ad4b90fc1df9 server_cookie=0 in_seq=0 out_seq=0 2023-08-09T15:11:46.151 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:46.149+0000 7fda8bfff640 1 -- 172.21.15.43:0/2273733156 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fda8c02ccf0 con 0x7fda940d1840 2023-08-09T15:11:46.151 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:46.149+0000 7fda8bfff640 1 -- 172.21.15.43:0/2273733156 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fda8c02ce50 con 0x7fda940d1840 2023-08-09T15:11:46.151 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:46.149+0000 7fda8bfff640 1 -- 172.21.15.43:0/2273733156 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fda8c034660 con 0x7fda940d1840 2023-08-09T15:11:46.152 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:46.150+0000 7fda9a134640 1 -- 172.21.15.43:0/2273733156 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fda940d1840 msgr2=0x7fda94167580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T15:11:46.152 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:46.150+0000 7fda9a134640 1 --2- 172.21.15.43:0/2273733156 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fda940d1840 0x7fda94167580 secure :-1 s=READY pgs=2451 cs=0 l=1 rev1=1 crypto rx=0x7fda8c009d90 tx=0x7fda8c02bf40 comp rx=0 tx=0).stop 2023-08-09T15:11:46.152 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:46.150+0000 7fda9a134640 1 -- 172.21.15.43:0/2273733156 shutdown_connections 2023-08-09T15:11:46.152 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:46.150+0000 7fda9a134640 1 --2- 172.21.15.43:0/2273733156 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fda940d1840 0x7fda94167580 unknown :-1 s=CLOSED pgs=2451 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T15:11:46.152 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:46.150+0000 7fda9a134640 1 -- 172.21.15.43:0/2273733156 >> 172.21.15.43:0/2273733156 conn(0x7fda94071260 msgr2=0x7fda94071670 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-09T15:11:46.152 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:46.150+0000 7fda9a134640 1 -- 172.21.15.43:0/2273733156 shutdown_connections 2023-08-09T15:11:46.153 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:46.150+0000 7fda9a134640 1 -- 172.21.15.43:0/2273733156 wait complete. 2023-08-09T15:11:46.153 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:46.152+0000 7fda9a134640 1 Processor -- start 2023-08-09T15:11:46.153 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:46.152+0000 7fda9a134640 1 -- start start 2023-08-09T15:11:46.154 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:46.153+0000 7fda9a134640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fda940d1840 0x7fda94148650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T15:11:46.155 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:46.153+0000 7fda9a134640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fda940d1df0 0x7fda940cbe50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T15:11:46.155 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:46.153+0000 7fda9a134640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fda941f2d00 0x7fda940c9530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T15:11:46.155 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:46.153+0000 7fda9a134640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fda940cc9c0 con 0x7fda940d1df0 2023-08-09T15:11:46.155 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:46.153+0000 7fda9a134640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fda940cc6e0 con 0x7fda940d1840 2023-08-09T15:11:46.155 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:46.153+0000 7fda9a134640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fda940c7160 con 0x7fda941f2d00 2023-08-09T15:11:46.155 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:46.153+0000 7fda98931640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fda940d1df0 0x7fda940cbe50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T15:11:46.155 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:46.153+0000 7fda98931640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fda940d1df0 0x7fda940cbe50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:50114/0 (socket says 172.21.15.43:50114) 2023-08-09T15:11:46.156 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:46.153+0000 7fda98931640 1 -- 172.21.15.43:0/2987746360 learned_addr learned my addr 172.21.15.43:0/2987746360 (peer_addr_for_me v2:172.21.15.43:0/0) 2023-08-09T15:11:46.157 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:46.154+0000 7fda98931640 1 -- 172.21.15.43:0/2987746360 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fda941f2d00 msgr2=0x7fda940c9530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T15:11:46.157 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:46.154+0000 7fda98931640 1 --2- 172.21.15.43:0/2987746360 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fda941f2d00 0x7fda940c9530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T15:11:46.157 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:46.154+0000 7fda99132640 1 --2- 172.21.15.43:0/2987746360 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fda940d1840 0x7fda94148650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T15:11:46.157 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:46.154+0000 7fda98931640 1 -- 172.21.15.43:0/2987746360 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fda940d1840 msgr2=0x7fda94148650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T15:11:46.157 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:46.154+0000 7fda98931640 1 --2- 172.21.15.43:0/2987746360 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fda940d1840 0x7fda94148650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T15:11:46.157 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:46.154+0000 7fda98931640 1 -- 172.21.15.43:0/2987746360 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fda8c009a70 con 0x7fda940d1df0 2023-08-09T15:11:46.158 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:46.154+0000 7fda98931640 1 --2- 172.21.15.43:0/2987746360 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fda940d1df0 0x7fda940cbe50 secure :-1 s=READY pgs=2452 cs=0 l=1 rev1=1 crypto rx=0x7fda8400c480 tx=0x7fda8400c940 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-09T15:11:46.158 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:46.154+0000 7fda89ffb640 1 -- 172.21.15.43:0/2987746360 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fda84005ce0 con 0x7fda940d1df0 2023-08-09T15:11:46.158 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:46.154+0000 7fda89ffb640 1 -- 172.21.15.43:0/2987746360 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fda84005e40 con 0x7fda940d1df0 2023-08-09T15:11:46.158 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:46.154+0000 7fda89ffb640 1 -- 172.21.15.43:0/2987746360 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fda840072f0 con 0x7fda940d1df0 2023-08-09T15:11:46.158 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:46.154+0000 7fda9a134640 1 -- 172.21.15.43:0/2987746360 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fda940c7440 con 0x7fda940d1df0 2023-08-09T15:11:46.158 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:46.155+0000 7fda9a134640 1 -- 172.21.15.43:0/2987746360 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fda940cc390 con 0x7fda940d1df0 2023-08-09T15:11:46.159 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:46.156+0000 7fda9a134640 1 -- 172.21.15.43:0/2987746360 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fda9404e390 con 0x7fda940d1df0 2023-08-09T15:11:46.159 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:46.157+0000 7fda89ffb640 1 -- 172.21.15.43:0/2987746360 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 122) v1 ==== 88461+0+0 (secure 0 0 0) 0x7fda84007510 con 0x7fda940d1df0 2023-08-09T15:11:46.159 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:46.158+0000 7fda89ffb640 1 --2- 172.21.15.43:0/2987746360 >> [v2:172.21.15.203:6832/2375771024,v1:172.21.15.203:6833/2375771024] conn(0x7fda7806a1c0 0x7fda7806c680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T15:11:46.159 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:46.158+0000 7fda89ffb640 1 -- 172.21.15.43:0/2987746360 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(224..224 src has 1..224) v4 ==== 6770+0+0 (secure 0 0 0) 0x7fda84087ee0 con 0x7fda940d1df0 2023-08-09T15:11:46.160 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:46.159+0000 7fda99132640 1 --2- 172.21.15.43:0/2987746360 >> [v2:172.21.15.203:6832/2375771024,v1:172.21.15.203:6833/2375771024] conn(0x7fda7806a1c0 0x7fda7806c680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T15:11:46.161 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:46.160+0000 7fda99132640 1 --2- 172.21.15.43:0/2987746360 >> [v2:172.21.15.203:6832/2375771024,v1:172.21.15.203:6833/2375771024] conn(0x7fda7806a1c0 0x7fda7806c680 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7fda90000f60 tx=0x7fda90009040 comp rx=0 tx=0).ready entity=mgr.9103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-09T15:11:46.163 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:46.162+0000 7fda89ffb640 1 -- 172.21.15.43:0/2987746360 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178066 (secure 0 0 0) 0x7fda84053290 con 0x7fda940d1df0 2023-08-09T15:11:46.360 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:46.359+0000 7fda9a134640 1 -- 172.21.15.43:0/2987746360 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"} v 0) v1 -- 0x7fda940cfb00 con 0x7fda940d1df0 2023-08-09T15:11:46.362 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:46.360+0000 7fda89ffb640 1 -- 172.21.15.43:0/2987746360 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"}]=0 v224) v1 ==== 92+0+10 (secure 0 0 0) 0x7fda840170a0 con 0x7fda940d1df0 2023-08-09T15:11:46.363 INFO:teuthology.orchestra.run.smithi043.stdout:pg_num: 8 2023-08-09T15:11:46.363 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:46.362+0000 7fda9a134640 1 -- 172.21.15.43:0/2987746360 >> [v2:172.21.15.203:6832/2375771024,v1:172.21.15.203:6833/2375771024] conn(0x7fda7806a1c0 msgr2=0x7fda7806c680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T15:11:46.364 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:46.362+0000 7fda9a134640 1 --2- 172.21.15.43:0/2987746360 >> [v2:172.21.15.203:6832/2375771024,v1:172.21.15.203:6833/2375771024] conn(0x7fda7806a1c0 0x7fda7806c680 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7fda90000f60 tx=0x7fda90009040 comp rx=0 tx=0).stop 2023-08-09T15:11:46.364 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:46.362+0000 7fda9a134640 1 -- 172.21.15.43:0/2987746360 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fda940d1df0 msgr2=0x7fda940cbe50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T15:11:46.364 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:46.362+0000 7fda9a134640 1 --2- 172.21.15.43:0/2987746360 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fda940d1df0 0x7fda940cbe50 secure :-1 s=READY pgs=2452 cs=0 l=1 rev1=1 crypto rx=0x7fda8400c480 tx=0x7fda8400c940 comp rx=0 tx=0).stop 2023-08-09T15:11:46.364 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:46.362+0000 7fda9a134640 1 -- 172.21.15.43:0/2987746360 shutdown_connections 2023-08-09T15:11:46.365 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:46.362+0000 7fda9a134640 1 --2- 172.21.15.43:0/2987746360 >> [v2:172.21.15.203:6832/2375771024,v1:172.21.15.203:6833/2375771024] conn(0x7fda7806a1c0 0x7fda7806c680 unknown :-1 s=CLOSED pgs=26 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T15:11:46.365 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:46.362+0000 7fda9a134640 1 --2- 172.21.15.43:0/2987746360 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fda941f2d00 0x7fda940c9530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T15:11:46.365 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:46.362+0000 7fda9a134640 1 --2- 172.21.15.43:0/2987746360 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fda940d1df0 0x7fda940cbe50 unknown :-1 s=CLOSED pgs=2452 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T15:11:46.365 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:46.362+0000 7fda9a134640 1 --2- 172.21.15.43:0/2987746360 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fda940d1840 0x7fda94148650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T15:11:46.365 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:46.362+0000 7fda9a134640 1 -- 172.21.15.43:0/2987746360 >> 172.21.15.43:0/2987746360 conn(0x7fda94071260 msgr2=0x7fda940b5590 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-09T15:11:46.365 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:46.362+0000 7fda9a134640 1 -- 172.21.15.43:0/2987746360 shutdown_connections 2023-08-09T15:11:46.366 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:46.363+0000 7fda9a134640 1 -- 172.21.15.43:0/2987746360 wait complete. 2023-08-09T15:11:46.384 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-08-09T15:11:46.620 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:46.619+0000 7f654759e640 1 Processor -- start 2023-08-09T15:11:46.620 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:46.619+0000 7f654759e640 1 -- start start 2023-08-09T15:11:46.622 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:46.621+0000 7f654759e640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f65480d1df0 0x7f65480cd1a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T15:11:46.622 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:46.621+0000 7f654759e640 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7f65480cd6e0 0x7f65480d0830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T15:11:46.623 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:46.621+0000 7f654759e640 1 -- --> v1:172.21.15.43:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6548160e00 con 0x7f654815c870 2023-08-09T15:11:46.623 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:46.621+0000 7f654759e640 1 -- --> v2:172.21.15.203:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f65480cdb70 con 0x7f65480cd6e0 2023-08-09T15:11:46.623 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:46.621+0000 7f654759e640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f65481601f0 con 0x7f65480d1df0 2023-08-09T15:11:46.624 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:46.621+0000 7f654659c640 1 --1- >> v1:172.21.15.43:6789/0 conn(0x7f654815c870 0x7f65480b7d80 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.43:6789/0 says I am v1:172.21.15.43:56298/0 (socket says 172.21.15.43:56298) 2023-08-09T15:11:46.625 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:46.621+0000 7f654659c640 1 -- 172.21.15.43:0/178348391 learned_addr learned my addr 172.21.15.43:0/178348391 (peer_addr_for_me v1:172.21.15.43:0/0) 2023-08-09T15:11:46.625 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:46.622+0000 7f6546d9d640 1 --2- 172.21.15.43:0/178348391 >> v2:172.21.15.203:3300/0 conn(0x7f65480cd6e0 0x7f65480d0830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T15:11:46.625 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:46.622+0000 7f653dd9b640 1 --2- 172.21.15.43:0/178348391 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f65480d1df0 0x7f65480cd1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T15:11:46.625 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:46.622+0000 7f6545d9b640 1 -- 172.21.15.43:0/178348391 <== mon.0 v1:172.21.15.43:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2985618029 0 0) 0x7f6548160e00 con 0x7f654815c870 2023-08-09T15:11:46.625 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:46.622+0000 7f6545d9b640 1 -- 172.21.15.43:0/178348391 --> v1:172.21.15.43:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f6534003520 con 0x7f654815c870 2023-08-09T15:11:46.625 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:46.622+0000 7f6545d9b640 1 -- 172.21.15.43:0/178348391 <== mon.0 v1:172.21.15.43:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3757527802 0 0) 0x7f6534003520 con 0x7f654815c870 2023-08-09T15:11:46.625 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:46.622+0000 7f6545d9b640 1 -- 172.21.15.43:0/178348391 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f65480d1df0 msgr2=0x7f65480cd1a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-09T15:11:46.626 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:46.622+0000 7f6545d9b640 1 --2- 172.21.15.43:0/178348391 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f65480d1df0 0x7f65480cd1a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T15:11:46.626 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:46.622+0000 7f6545d9b640 1 -- 172.21.15.43:0/178348391 >> v2:172.21.15.203:3300/0 conn(0x7f65480cd6e0 msgr2=0x7f65480d0830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-09T15:11:46.626 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:46.622+0000 7f6545d9b640 1 --2- 172.21.15.43:0/178348391 >> v2:172.21.15.203:3300/0 conn(0x7f65480cd6e0 0x7f65480d0830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T15:11:46.626 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:46.622+0000 7f6545d9b640 1 -- 172.21.15.43:0/178348391 --> v1:172.21.15.43:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f65481604d0 con 0x7f654815c870 2023-08-09T15:11:46.626 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:46.622+0000 7f6545d9b640 1 -- 172.21.15.43:0/178348391 <== mon.0 v1:172.21.15.43:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1285150740 0 0) 0x7f6538001670 con 0x7f654815c870 2023-08-09T15:11:46.626 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:46.622+0000 7f6545d9b640 1 -- 172.21.15.43:0/178348391 >> v1:172.21.15.43:6789/0 conn(0x7f654815c870 legacy=0x7f65480b7d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T15:11:46.626 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:46.622+0000 7f6545d9b640 1 --2- 172.21.15.43:0/178348391 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f6534003970 0x7f6534003d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T15:11:46.626 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:46.622+0000 7f6545d9b640 1 --2- 172.21.15.43:0/178348391 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f65480d1df0 0x7f65340047c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T15:11:46.627 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:46.622+0000 7f6545d9b640 1 --2- 172.21.15.43:0/178348391 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f6534004d00 0x7f6534005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T15:11:46.627 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:46.622+0000 7f6545d9b640 1 -- 172.21.15.43:0/178348391 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f65480cdb70 con 0x7f6534003970 2023-08-09T15:11:46.627 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:46.622+0000 7f6545d9b640 1 -- 172.21.15.43:0/178348391 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f65481601f0 con 0x7f6534004d00 2023-08-09T15:11:46.627 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:46.622+0000 7f6545d9b640 1 -- 172.21.15.43:0/178348391 --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f653400b980 con 0x7f65480d1df0 2023-08-09T15:11:46.627 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:46.623+0000 7f654659c640 1 --2- 172.21.15.43:0/178348391 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f6534003970 0x7f6534003d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T15:11:46.627 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:46.623+0000 7f6546d9d640 1 --2- 172.21.15.43:0/178348391 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f6534004d00 0x7f6534005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T15:11:46.627 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:46.623+0000 7f654659c640 1 -- 172.21.15.43:0/178348391 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f65480d1df0 msgr2=0x7f65340047c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-09T15:11:46.628 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:46.623+0000 7f653dd9b640 1 --2- 172.21.15.43:0/178348391 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f65480d1df0 0x7f65340047c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T15:11:46.628 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:46.623+0000 7f654659c640 1 --2- 172.21.15.43:0/178348391 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f65480d1df0 0x7f65340047c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T15:11:46.628 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:46.623+0000 7f654659c640 1 -- 172.21.15.43:0/178348391 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f6534004d00 msgr2=0x7f6534005180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-09T15:11:46.628 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:46.623+0000 7f654659c640 1 --2- 172.21.15.43:0/178348391 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f6534004d00 0x7f6534005180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T15:11:46.628 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:46.623+0000 7f654659c640 1 -- 172.21.15.43:0/178348391 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6534003520 con 0x7f6534003970 2023-08-09T15:11:46.628 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:46.623+0000 7f6546d9d640 1 --2- 172.21.15.43:0/178348391 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f6534004d00 0x7f6534005180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-08-09T15:11:46.628 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:46.623+0000 7f653dd9b640 1 --2- 172.21.15.43:0/178348391 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f65480d1df0 0x7f65340047c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-08-09T15:11:46.630 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:46.624+0000 7f654659c640 1 --2- 172.21.15.43:0/178348391 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f6534003970 0x7f6534003d90 secure :-1 s=READY pgs=2454 cs=0 l=1 rev1=1 crypto rx=0x7f653800c270 tx=0x7f65340057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=21207e33b3f97aaa server_cookie=0 in_seq=0 out_seq=0 2023-08-09T15:11:46.630 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:46.625+0000 7f6545d9b640 1 -- 172.21.15.43:0/178348391 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f653802cae0 con 0x7f6534003970 2023-08-09T15:11:46.630 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:46.625+0000 7f6545d9b640 1 -- 172.21.15.43:0/178348391 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f653802cc40 con 0x7f6534003970 2023-08-09T15:11:46.630 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:46.625+0000 7f6545d9b640 1 -- 172.21.15.43:0/178348391 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f65380058d0 con 0x7f6534003970 2023-08-09T15:11:46.631 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:46.625+0000 7f654759e640 1 -- 172.21.15.43:0/178348391 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f6534003970 msgr2=0x7f6534003d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T15:11:46.631 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:46.625+0000 7f654759e640 1 --2- 172.21.15.43:0/178348391 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f6534003970 0x7f6534003d90 secure :-1 s=READY pgs=2454 cs=0 l=1 rev1=1 crypto rx=0x7f653800c270 tx=0x7f65340057f0 comp rx=0 tx=0).stop 2023-08-09T15:11:46.631 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:46.625+0000 7f6546d9d640 1 -- 172.21.15.43:0/178348391 reap_dead start 2023-08-09T15:11:46.631 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:46.625+0000 7f654759e640 1 -- 172.21.15.43:0/178348391 shutdown_connections 2023-08-09T15:11:46.631 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:46.625+0000 7f654759e640 1 -- 172.21.15.43:0/178348391 >> 172.21.15.43:0/178348391 conn(0x7f6548071260 msgr2=0x7f6548071670 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-09T15:11:46.631 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:46.625+0000 7f654759e640 1 -- 172.21.15.43:0/178348391 shutdown_connections 2023-08-09T15:11:46.631 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:46.625+0000 7f654759e640 1 -- 172.21.15.43:0/178348391 wait complete. 2023-08-09T15:11:46.631 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:46.627+0000 7f654759e640 1 Processor -- start 2023-08-09T15:11:46.632 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:46.627+0000 7f654759e640 1 -- start start 2023-08-09T15:11:46.632 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:46.628+0000 7f654759e640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f6534003970 0x7f65481595b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T15:11:46.632 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:46.628+0000 7f654759e640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f65480b13e0 0x7f65481f5130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T15:11:46.632 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:46.628+0000 7f654759e640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f65480d2460 0x7f65480d28e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T15:11:46.632 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:46.628+0000 7f654759e640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f65480d2e20 con 0x7f6534003970 2023-08-09T15:11:46.632 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:46.628+0000 7f654759e640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f654815e150 con 0x7f65480b13e0 2023-08-09T15:11:46.632 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:46.628+0000 7f654759e640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f65481f5a30 con 0x7f65480d2460 2023-08-09T15:11:46.633 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:46.629+0000 7f654659c640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f6534003970 0x7f65481595b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T15:11:46.633 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:46.629+0000 7f654659c640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f6534003970 0x7f65481595b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:50132/0 (socket says 172.21.15.43:50132) 2023-08-09T15:11:46.633 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:46.629+0000 7f654659c640 1 -- 172.21.15.43:0/3482470247 learned_addr learned my addr 172.21.15.43:0/3482470247 (peer_addr_for_me v2:172.21.15.43:0/0) 2023-08-09T15:11:46.634 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:46.629+0000 7f653dd9b640 1 --2- 172.21.15.43:0/3482470247 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f65480b13e0 0x7f65481f5130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T15:11:46.634 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:46.629+0000 7f6546d9d640 1 --2- 172.21.15.43:0/3482470247 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f65480d2460 0x7f65480d28e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T15:11:46.634 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:46.629+0000 7f654659c640 1 -- 172.21.15.43:0/3482470247 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f65480d2460 msgr2=0x7f65480d28e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T15:11:46.634 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:46.629+0000 7f654659c640 1 --2- 172.21.15.43:0/3482470247 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f65480d2460 0x7f65480d28e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T15:11:46.634 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:46.629+0000 7f654659c640 1 -- 172.21.15.43:0/3482470247 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f65480b13e0 msgr2=0x7f65481f5130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T15:11:46.635 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:46.629+0000 7f654659c640 1 --2- 172.21.15.43:0/3482470247 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f65480b13e0 0x7f65481f5130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T15:11:46.635 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:46.629+0000 7f654659c640 1 -- 172.21.15.43:0/3482470247 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f65481f5d10 con 0x7f6534003970 2023-08-09T15:11:46.635 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:46.630+0000 7f654659c640 1 --2- 172.21.15.43:0/3482470247 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f6534003970 0x7f65481595b0 secure :-1 s=READY pgs=2455 cs=0 l=1 rev1=1 crypto rx=0x7f653802c480 tx=0x7f6538001a00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-09T15:11:46.635 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:46.630+0000 7f653f7fe640 1 -- 172.21.15.43:0/3482470247 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6538039030 con 0x7f6534003970 2023-08-09T15:11:46.635 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:46.630+0000 7f653f7fe640 1 -- 172.21.15.43:0/3482470247 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6538004bf0 con 0x7f6534003970 2023-08-09T15:11:46.635 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:46.630+0000 7f653f7fe640 1 -- 172.21.15.43:0/3482470247 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f653803c040 con 0x7f6534003970 2023-08-09T15:11:46.635 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:46.630+0000 7f654759e640 1 -- 172.21.15.43:0/3482470247 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f654815dde0 con 0x7f6534003970 2023-08-09T15:11:46.635 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:46.630+0000 7f654759e640 1 -- 172.21.15.43:0/3482470247 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6548159af0 con 0x7f6534003970 2023-08-09T15:11:46.636 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:46.631+0000 7f654759e640 1 -- 172.21.15.43:0/3482470247 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f654804e390 con 0x7f6534003970 2023-08-09T15:11:46.636 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:46.633+0000 7f653f7fe640 1 -- 172.21.15.43:0/3482470247 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 122) v1 ==== 88461+0+0 (secure 0 0 0) 0x7f6538032070 con 0x7f6534003970 2023-08-09T15:11:46.636 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:46.633+0000 7f653f7fe640 1 --2- 172.21.15.43:0/3482470247 >> [v2:172.21.15.203:6832/2375771024,v1:172.21.15.203:6833/2375771024] conn(0x7f652806a1f0 0x7f652806c6b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T15:11:46.637 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:46.634+0000 7f653f7fe640 1 -- 172.21.15.43:0/3482470247 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(224..224 src has 1..224) v4 ==== 6770+0+0 (secure 0 0 0) 0x7f65380a9da0 con 0x7f6534003970 2023-08-09T15:11:46.637 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:46.634+0000 7f653dd9b640 1 --2- 172.21.15.43:0/3482470247 >> [v2:172.21.15.203:6832/2375771024,v1:172.21.15.203:6833/2375771024] conn(0x7f652806a1f0 0x7f652806c6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T15:11:46.637 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:46.635+0000 7f653dd9b640 1 --2- 172.21.15.43:0/3482470247 >> [v2:172.21.15.203:6832/2375771024,v1:172.21.15.203:6833/2375771024] conn(0x7f652806a1f0 0x7f652806c6b0 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7f654815a0d0 tx=0x7f6530008040 comp rx=0 tx=0).ready entity=mgr.9103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-09T15:11:46.641 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:46.640+0000 7f653f7fe640 1 -- 172.21.15.43:0/3482470247 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178066 (secure 0 0 0) 0x7f65380750a0 con 0x7f6534003970 2023-08-09T15:11:46.837 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:46.836+0000 7f654759e640 1 -- 172.21.15.43:0/3482470247 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f654804e390 con 0x7f6534003970 2023-08-09T15:11:46.838 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:46.837+0000 7f653f7fe640 1 -- 172.21.15.43:0/3482470247 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v224) v1 ==== 93+0+10 (secure 0 0 0) 0x7f65380786c0 con 0x7f6534003970 2023-08-09T15:11:46.839 INFO:teuthology.orchestra.run.smithi043.stdout:pg_num: 1 2023-08-09T15:11:46.841 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:46.840+0000 7f654759e640 1 -- 172.21.15.43:0/3482470247 >> [v2:172.21.15.203:6832/2375771024,v1:172.21.15.203:6833/2375771024] conn(0x7f652806a1f0 msgr2=0x7f652806c6b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T15:11:46.841 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:46.840+0000 7f654759e640 1 --2- 172.21.15.43:0/3482470247 >> [v2:172.21.15.203:6832/2375771024,v1:172.21.15.203:6833/2375771024] conn(0x7f652806a1f0 0x7f652806c6b0 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7f654815a0d0 tx=0x7f6530008040 comp rx=0 tx=0).stop 2023-08-09T15:11:46.841 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:46.840+0000 7f654759e640 1 -- 172.21.15.43:0/3482470247 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f6534003970 msgr2=0x7f65481595b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T15:11:46.842 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:46.840+0000 7f654759e640 1 --2- 172.21.15.43:0/3482470247 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f6534003970 0x7f65481595b0 secure :-1 s=READY pgs=2455 cs=0 l=1 rev1=1 crypto rx=0x7f653802c480 tx=0x7f6538001a00 comp rx=0 tx=0).stop 2023-08-09T15:11:46.842 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:46.840+0000 7f654759e640 1 -- 172.21.15.43:0/3482470247 shutdown_connections 2023-08-09T15:11:46.842 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:46.840+0000 7f654759e640 1 --2- 172.21.15.43:0/3482470247 >> [v2:172.21.15.203:6832/2375771024,v1:172.21.15.203:6833/2375771024] conn(0x7f652806a1f0 0x7f652806c6b0 unknown :-1 s=CLOSED pgs=27 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T15:11:46.842 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:46.840+0000 7f654759e640 1 --2- 172.21.15.43:0/3482470247 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f65480d2460 0x7f65480d28e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T15:11:46.842 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:46.840+0000 7f654759e640 1 --2- 172.21.15.43:0/3482470247 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f65480b13e0 0x7f65481f5130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T15:11:46.842 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:46.840+0000 7f654759e640 1 --2- 172.21.15.43:0/3482470247 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f6534003970 0x7f65481595b0 unknown :-1 s=CLOSED pgs=2455 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T15:11:46.842 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:46.840+0000 7f654759e640 1 -- 172.21.15.43:0/3482470247 >> 172.21.15.43:0/3482470247 conn(0x7f6548071260 msgr2=0x7f65480b6d90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-09T15:11:46.843 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:46.840+0000 7f654759e640 1 -- 172.21.15.43:0/3482470247 shutdown_connections 2023-08-09T15:11:46.843 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:46.840+0000 7f654759e640 1 -- 172.21.15.43:0/3482470247 wait complete. 2023-08-09T15:11:46.859 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-08-09T15:11:47.082 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:47.081+0000 7f807b2e5640 1 Processor -- start 2023-08-09T15:11:47.082 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:47.081+0000 7f807b2e5640 1 -- start start 2023-08-09T15:11:47.084 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:47.082+0000 7f807b2e5640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f807415a340 0x7f807406c6f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T15:11:47.084 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:47.083+0000 7f807b2e5640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f80740ae550 0x7f80740c9820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T15:11:47.084 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:47.083+0000 7f807b2e5640 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7f80740c9d60 0x7f807415df80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T15:11:47.084 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:47.083+0000 7f807b2e5640 1 -- --> v2:172.21.15.43:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f80740ca1c0 con 0x7f807415a340 2023-08-09T15:11:47.085 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:47.083+0000 7f807b2e5640 1 -- --> v2:172.21.15.203:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f80740aea50 con 0x7f80740c9d60 2023-08-09T15:11:47.085 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:47.083+0000 7f807b2e5640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f80740b1b40 con 0x7f80740ae550 2023-08-09T15:11:47.085 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:47.083+0000 7f807a2e3640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f807415a340 0x7f807406c6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T15:11:47.085 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:47.083+0000 7f807a2e3640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f807415a340 0x7f807406c6f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:50134/0 (socket says 172.21.15.43:50134) 2023-08-09T15:11:47.085 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:47.083+0000 7f807a2e3640 1 -- 172.21.15.43:0/1590596826 learned_addr learned my addr 172.21.15.43:0/1590596826 (peer_addr_for_me v2:172.21.15.43:0/0) 2023-08-09T15:11:47.086 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:47.083+0000 7f807aae4640 1 --2- 172.21.15.43:0/1590596826 >> v2:172.21.15.203:3300/0 conn(0x7f80740c9d60 0x7f807415df80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T15:11:47.086 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:47.083+0000 7f8079ae2640 1 --2- 172.21.15.43:0/1590596826 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f80740ae550 0x7f80740c9820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T15:11:47.086 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:47.083+0000 7f807a2e3640 1 -- 172.21.15.43:0/1590596826 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f80740ae550 msgr2=0x7f80740c9820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-09T15:11:47.086 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:47.083+0000 7f807a2e3640 1 --2- 172.21.15.43:0/1590596826 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f80740ae550 0x7f80740c9820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T15:11:47.087 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:47.083+0000 7f807a2e3640 1 -- 172.21.15.43:0/1590596826 >> v2:172.21.15.203:3300/0 conn(0x7f80740c9d60 msgr2=0x7f807415df80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-09T15:11:47.087 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:47.083+0000 7f807a2e3640 1 --2- 172.21.15.43:0/1590596826 >> v2:172.21.15.203:3300/0 conn(0x7f80740c9d60 0x7f807415df80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T15:11:47.087 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:47.084+0000 7f807a2e3640 1 -- 172.21.15.43:0/1590596826 --> v2:172.21.15.43:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f80740b0d80 con 0x7f807415a340 2023-08-09T15:11:47.087 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:47.084+0000 7f8079ae2640 1 --2- 172.21.15.43:0/1590596826 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f80740ae550 0x7f80740c9820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-08-09T15:11:47.087 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:47.085+0000 7f807a2e3640 1 --2- 172.21.15.43:0/1590596826 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f807415a340 0x7f807406c6f0 secure :-1 s=READY pgs=2456 cs=0 l=1 rev1=1 crypto rx=0x7f807406a4f0 tx=0x7f807002be50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dbf097b3b1306820 server_cookie=0 in_seq=0 out_seq=0 2023-08-09T15:11:47.092 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:47.085+0000 7f80792e1640 1 -- 172.21.15.43:0/1590596826 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f807002c9f0 con 0x7f807415a340 2023-08-09T15:11:47.092 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:47.085+0000 7f80792e1640 1 -- 172.21.15.43:0/1590596826 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f807002cb50 con 0x7f807415a340 2023-08-09T15:11:47.092 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:47.085+0000 7f80792e1640 1 -- 172.21.15.43:0/1590596826 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f80700343f0 con 0x7f807415a340 2023-08-09T15:11:47.092 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:47.086+0000 7f807b2e5640 1 -- 172.21.15.43:0/1590596826 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f807415a340 msgr2=0x7f807406c6f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T15:11:47.093 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:47.086+0000 7f807b2e5640 1 --2- 172.21.15.43:0/1590596826 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f807415a340 0x7f807406c6f0 secure :-1 s=READY pgs=2456 cs=0 l=1 rev1=1 crypto rx=0x7f807406a4f0 tx=0x7f807002be50 comp rx=0 tx=0).stop 2023-08-09T15:11:47.093 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:47.086+0000 7f807b2e5640 1 -- 172.21.15.43:0/1590596826 shutdown_connections 2023-08-09T15:11:47.093 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:47.086+0000 7f807b2e5640 1 --2- 172.21.15.43:0/1590596826 >> v2:172.21.15.203:3300/0 conn(0x7f80740c9d60 0x7f807415df80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T15:11:47.093 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:47.086+0000 7f807b2e5640 1 --2- 172.21.15.43:0/1590596826 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f80740ae550 0x7f80740c9820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T15:11:47.093 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:47.086+0000 7f807b2e5640 1 --2- 172.21.15.43:0/1590596826 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f807415a340 0x7f807406c6f0 unknown :-1 s=CLOSED pgs=2456 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T15:11:47.093 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:47.086+0000 7f807b2e5640 1 -- 172.21.15.43:0/1590596826 >> 172.21.15.43:0/1590596826 conn(0x7f80740cbc50 msgr2=0x7f80740cc030 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-09T15:11:47.094 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:47.086+0000 7f807b2e5640 1 -- 172.21.15.43:0/1590596826 shutdown_connections 2023-08-09T15:11:47.094 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:47.086+0000 7f807b2e5640 1 -- 172.21.15.43:0/1590596826 wait complete. 2023-08-09T15:11:47.094 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:47.088+0000 7f807b2e5640 1 Processor -- start 2023-08-09T15:11:47.094 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:47.088+0000 7f807b2e5640 1 -- start start 2023-08-09T15:11:47.094 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:47.089+0000 7f807b2e5640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f80740ae550 0x7f8074152ca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T15:11:47.094 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:47.089+0000 7f807b2e5640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f80740c9d60 0x7f80740c4a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T15:11:47.095 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:47.089+0000 7f807b2e5640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f807415a340 0x7f80741545b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T15:11:47.095 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:47.089+0000 7f807b2e5640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8074155b80 con 0x7f80740c9d60 2023-08-09T15:11:47.095 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:47.089+0000 7f807b2e5640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8074154f00 con 0x7f807415a340 2023-08-09T15:11:47.095 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:47.089+0000 7f807b2e5640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f80741551e0 con 0x7f80740ae550 2023-08-09T15:11:47.095 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:47.089+0000 7f8079ae2640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f80740c9d60 0x7f80740c4a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T15:11:47.095 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:47.089+0000 7f8079ae2640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f80740c9d60 0x7f80740c4a20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:50140/0 (socket says 172.21.15.43:50140) 2023-08-09T15:11:47.095 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:47.089+0000 7f8079ae2640 1 -- 172.21.15.43:0/1317809139 learned_addr learned my addr 172.21.15.43:0/1317809139 (peer_addr_for_me v2:172.21.15.43:0/0) 2023-08-09T15:11:47.096 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:47.089+0000 7f807aae4640 1 --2- 172.21.15.43:0/1317809139 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f807415a340 0x7f80741545b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T15:11:47.096 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:47.090+0000 7f807a2e3640 1 --2- 172.21.15.43:0/1317809139 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f80740ae550 0x7f8074152ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T15:11:47.096 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:47.090+0000 7f8079ae2640 1 -- 172.21.15.43:0/1317809139 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f80740ae550 msgr2=0x7f8074152ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T15:11:47.096 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:47.090+0000 7f8079ae2640 1 --2- 172.21.15.43:0/1317809139 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f80740ae550 0x7f8074152ca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T15:11:47.096 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:47.090+0000 7f8079ae2640 1 -- 172.21.15.43:0/1317809139 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f807415a340 msgr2=0x7f80741545b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T15:11:47.096 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:47.090+0000 7f8079ae2640 1 --2- 172.21.15.43:0/1317809139 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f807415a340 0x7f80741545b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T15:11:47.097 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:47.090+0000 7f8079ae2640 1 -- 172.21.15.43:0/1317809139 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8070009b30 con 0x7f80740c9d60 2023-08-09T15:11:47.097 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:47.090+0000 7f807aae4640 1 --2- 172.21.15.43:0/1317809139 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f807415a340 0x7f80741545b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-08-09T15:11:47.097 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:47.090+0000 7f807a2e3640 1 --2- 172.21.15.43:0/1317809139 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f80740ae550 0x7f8074152ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-08-09T15:11:47.097 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:47.090+0000 7f8079ae2640 1 --2- 172.21.15.43:0/1317809139 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f80740c9d60 0x7f80740c4a20 secure :-1 s=READY pgs=2457 cs=0 l=1 rev1=1 crypto rx=0x7f806400c7c0 tx=0x7f806400cc80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-09T15:11:47.097 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:47.090+0000 7f806affd640 1 -- 172.21.15.43:0/1317809139 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8064007a30 con 0x7f80740c9d60 2023-08-09T15:11:47.097 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:47.090+0000 7f806affd640 1 -- 172.21.15.43:0/1317809139 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8064007b90 con 0x7f80740c9d60 2023-08-09T15:11:47.097 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:47.090+0000 7f806affd640 1 -- 172.21.15.43:0/1317809139 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f806400ce70 con 0x7f80740c9d60 2023-08-09T15:11:47.098 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:47.090+0000 7f807b2e5640 1 -- 172.21.15.43:0/1317809139 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f80741554c0 con 0x7f80740c9d60 2023-08-09T15:11:47.098 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:47.090+0000 7f807b2e5640 1 -- 172.21.15.43:0/1317809139 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f80740c4f60 con 0x7f80740c9d60 2023-08-09T15:11:47.099 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:47.092+0000 7f807b2e5640 1 -- 172.21.15.43:0/1317809139 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f80740c5120 con 0x7f80740c9d60 2023-08-09T15:11:47.099 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:47.093+0000 7f806affd640 1 -- 172.21.15.43:0/1317809139 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 122) v1 ==== 88461+0+0 (secure 0 0 0) 0x7f80640113f0 con 0x7f80740c9d60 2023-08-09T15:11:47.099 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:47.094+0000 7f806affd640 1 --2- 172.21.15.43:0/1317809139 >> [v2:172.21.15.203:6832/2375771024,v1:172.21.15.203:6833/2375771024] conn(0x7f805806a1c0 0x7f805806c680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T15:11:47.099 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:47.094+0000 7f806affd640 1 -- 172.21.15.43:0/1317809139 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(224..224 src has 1..224) v4 ==== 6770+0+0 (secure 0 0 0) 0x7f8064088cf0 con 0x7f80740c9d60 2023-08-09T15:11:47.099 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:47.094+0000 7f807a2e3640 1 --2- 172.21.15.43:0/1317809139 >> [v2:172.21.15.203:6832/2375771024,v1:172.21.15.203:6833/2375771024] conn(0x7f805806a1c0 0x7f805806c680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T15:11:47.100 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:47.095+0000 7f807a2e3640 1 --2- 172.21.15.43:0/1317809139 >> [v2:172.21.15.203:6832/2375771024,v1:172.21.15.203:6833/2375771024] conn(0x7f805806a1c0 0x7f805806c680 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7f8070035000 tx=0x7f8070030000 comp rx=0 tx=0).ready entity=mgr.9103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-09T15:11:47.100 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:47.098+0000 7f806affd640 1 -- 172.21.15.43:0/1317809139 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178066 (secure 0 0 0) 0x7f80740c5120 con 0x7f80740c9d60 2023-08-09T15:11:47.244 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:47.243+0000 7f806affd640 1 -- 172.21.15.43:0/1317809139 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mgrmap(e 123) v1 ==== 88517+0+0 (secure 0 0 0) 0x7f8064054220 con 0x7f80740c9d60 2023-08-09T15:11:47.298 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:47.297+0000 7f807b2e5640 1 -- 172.21.15.43:0/1317809139 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f8074155cc0 con 0x7f80740c9d60 2023-08-09T15:11:47.299 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:47.297+0000 7f806affd640 1 -- 172.21.15.43:0/1317809139 <== mon.0 v2:172.21.15.43:3300/0 8 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v224) v1 ==== 104+0+11 (secure 0 0 0) 0x7f8074155cc0 con 0x7f80740c9d60 2023-08-09T15:11:47.299 INFO:teuthology.orchestra.run.smithi043.stdout:pg_num: 32 2023-08-09T15:11:47.302 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:47.300+0000 7f807b2e5640 1 -- 172.21.15.43:0/1317809139 >> [v2:172.21.15.203:6832/2375771024,v1:172.21.15.203:6833/2375771024] conn(0x7f805806a1c0 msgr2=0x7f805806c680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T15:11:47.302 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:47.300+0000 7f807b2e5640 1 --2- 172.21.15.43:0/1317809139 >> [v2:172.21.15.203:6832/2375771024,v1:172.21.15.203:6833/2375771024] conn(0x7f805806a1c0 0x7f805806c680 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7f8070035000 tx=0x7f8070030000 comp rx=0 tx=0).stop 2023-08-09T15:11:47.302 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:47.300+0000 7f807b2e5640 1 -- 172.21.15.43:0/1317809139 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f80740c9d60 msgr2=0x7f80740c4a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T15:11:47.302 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:47.300+0000 7f807b2e5640 1 --2- 172.21.15.43:0/1317809139 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f80740c9d60 0x7f80740c4a20 secure :-1 s=READY pgs=2457 cs=0 l=1 rev1=1 crypto rx=0x7f806400c7c0 tx=0x7f806400cc80 comp rx=0 tx=0).stop 2023-08-09T15:11:47.302 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:47.300+0000 7f807b2e5640 1 -- 172.21.15.43:0/1317809139 shutdown_connections 2023-08-09T15:11:47.303 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:47.300+0000 7f807b2e5640 1 --2- 172.21.15.43:0/1317809139 >> [v2:172.21.15.203:6832/2375771024,v1:172.21.15.203:6833/2375771024] conn(0x7f805806a1c0 0x7f805806c680 unknown :-1 s=CLOSED pgs=28 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T15:11:47.303 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:47.300+0000 7f807b2e5640 1 --2- 172.21.15.43:0/1317809139 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f807415a340 0x7f80741545b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T15:11:47.303 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:47.300+0000 7f807b2e5640 1 --2- 172.21.15.43:0/1317809139 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f80740c9d60 0x7f80740c4a20 unknown :-1 s=CLOSED pgs=2457 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T15:11:47.303 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:47.300+0000 7f807b2e5640 1 --2- 172.21.15.43:0/1317809139 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f80740ae550 0x7f8074152ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T15:11:47.303 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:47.300+0000 7f807b2e5640 1 -- 172.21.15.43:0/1317809139 >> 172.21.15.43:0/1317809139 conn(0x7f80740cbc50 msgr2=0x7f807415d090 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-09T15:11:47.303 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:47.300+0000 7f807b2e5640 1 -- 172.21.15.43:0/1317809139 shutdown_connections 2023-08-09T15:11:47.303 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:47.301+0000 7f807b2e5640 1 -- 172.21.15.43:0/1317809139 wait complete. 2023-08-09T15:11:47.322 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-08-09T15:11:47.558 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:47.557+0000 7faa07dbe640 1 Processor -- start 2023-08-09T15:11:47.558 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:47.557+0000 7faa07dbe640 1 -- start start 2023-08-09T15:11:47.560 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:47.558+0000 7faa07dbe640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7faa00160f20 0x7faa000cec80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T15:11:47.560 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:47.558+0000 7faa07dbe640 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7faa000cf1c0 0x7faa000d0a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T15:11:47.560 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:47.558+0000 7faa07dbe640 1 -- --> v1:172.21.15.43:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7faa0015cc60 con 0x7faa0015c880 2023-08-09T15:11:47.560 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:47.558+0000 7faa07dbe640 1 -- --> v2:172.21.15.203:3300/0 -- mon_getmap magic: 0 v1 -- 0x7faa000d0530 con 0x7faa000cf1c0 2023-08-09T15:11:47.560 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:47.558+0000 7faa07dbe640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7faa000cf620 con 0x7faa00160f20 2023-08-09T15:11:47.561 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:47.558+0000 7faa05b5a640 1 --1- >> v1:172.21.15.43:6789/0 conn(0x7faa0015c880 0x7faa000ccbc0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.43:6789/0 says I am v1:172.21.15.43:56304/0 (socket says 172.21.15.43:56304) 2023-08-09T15:11:47.561 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:47.558+0000 7faa05b5a640 1 -- 172.21.15.43:0/1087350053 learned_addr learned my addr 172.21.15.43:0/1087350053 (peer_addr_for_me v1:172.21.15.43:0/0) 2023-08-09T15:11:47.562 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:47.559+0000 7faa0635b640 1 --2- 172.21.15.43:0/1087350053 >> v2:172.21.15.203:3300/0 conn(0x7faa000cf1c0 0x7faa000d0a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T15:11:47.562 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:47.559+0000 7faa05359640 1 --2- 172.21.15.43:0/1087350053 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7faa00160f20 0x7faa000cec80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T15:11:47.563 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:47.559+0000 7faa04b58640 1 -- 172.21.15.43:0/1087350053 <== mon.0 v1:172.21.15.43:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1482935087 0 0) 0x7faa0015cc60 con 0x7faa0015c880 2023-08-09T15:11:47.563 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:47.559+0000 7faa04b58640 1 -- 172.21.15.43:0/1087350053 --> v1:172.21.15.43:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fa9ec003520 con 0x7faa0015c880 2023-08-09T15:11:47.563 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:47.559+0000 7faa04b58640 1 -- 172.21.15.43:0/1087350053 <== mon.0 v1:172.21.15.43:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1992950801 0 0) 0x7fa9ec003520 con 0x7faa0015c880 2023-08-09T15:11:47.563 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:47.559+0000 7faa04b58640 1 -- 172.21.15.43:0/1087350053 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7faa00160f20 msgr2=0x7faa000cec80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-09T15:11:47.563 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:47.559+0000 7faa04b58640 1 --2- 172.21.15.43:0/1087350053 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7faa00160f20 0x7faa000cec80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T15:11:47.564 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:47.559+0000 7faa04b58640 1 -- 172.21.15.43:0/1087350053 >> v2:172.21.15.203:3300/0 conn(0x7faa000cf1c0 msgr2=0x7faa000d0a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-09T15:11:47.564 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:47.559+0000 7faa04b58640 1 --2- 172.21.15.43:0/1087350053 >> v2:172.21.15.203:3300/0 conn(0x7faa000cf1c0 0x7faa000d0a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T15:11:47.564 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:47.559+0000 7faa04b58640 1 -- 172.21.15.43:0/1087350053 --> v1:172.21.15.43:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faa000cd410 con 0x7faa0015c880 2023-08-09T15:11:47.564 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:47.559+0000 7faa04b58640 1 -- 172.21.15.43:0/1087350053 <== mon.0 v1:172.21.15.43:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1285150740 0 0) 0x7fa9fc0017f0 con 0x7faa0015c880 2023-08-09T15:11:47.564 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:47.560+0000 7faa04b58640 1 -- 172.21.15.43:0/1087350053 >> v1:172.21.15.43:6789/0 conn(0x7faa0015c880 legacy=0x7faa000ccbc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T15:11:47.564 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:47.560+0000 7faa04b58640 1 --2- 172.21.15.43:0/1087350053 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fa9ec003970 0x7fa9ec003d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T15:11:47.564 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:47.560+0000 7faa04b58640 1 --2- 172.21.15.43:0/1087350053 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7faa00160f20 0x7fa9ec0047c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T15:11:47.564 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:47.560+0000 7faa04b58640 1 --2- 172.21.15.43:0/1087350053 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa9ec004d00 0x7fa9ec005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T15:11:47.565 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:47.560+0000 7faa04b58640 1 -- 172.21.15.43:0/1087350053 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faa000d0530 con 0x7fa9ec004d00 2023-08-09T15:11:47.565 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:47.560+0000 7faa04b58640 1 -- 172.21.15.43:0/1087350053 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faa000cf620 con 0x7fa9ec003970 2023-08-09T15:11:47.565 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:47.560+0000 7faa04b58640 1 -- 172.21.15.43:0/1087350053 --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa9ec00b980 con 0x7faa00160f20 2023-08-09T15:11:47.565 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:47.560+0000 7faa0635b640 1 --2- 172.21.15.43:0/1087350053 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa9ec004d00 0x7fa9ec005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T15:11:47.565 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:47.560+0000 7faa05b5a640 1 --2- 172.21.15.43:0/1087350053 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fa9ec003970 0x7fa9ec003d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T15:11:47.565 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:47.560+0000 7faa05359640 1 --2- 172.21.15.43:0/1087350053 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7faa00160f20 0x7fa9ec0047c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T15:11:47.565 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:47.560+0000 7faa0635b640 1 -- 172.21.15.43:0/1087350053 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7faa00160f20 msgr2=0x7fa9ec0047c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-09T15:11:47.565 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:47.560+0000 7faa0635b640 1 --2- 172.21.15.43:0/1087350053 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7faa00160f20 0x7fa9ec0047c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T15:11:47.566 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:47.560+0000 7faa0635b640 1 -- 172.21.15.43:0/1087350053 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fa9ec003970 msgr2=0x7fa9ec003d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-09T15:11:47.566 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:47.560+0000 7faa0635b640 1 --2- 172.21.15.43:0/1087350053 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fa9ec003970 0x7fa9ec003d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T15:11:47.566 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:47.560+0000 7faa0635b640 1 -- 172.21.15.43:0/1087350053 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa9ec003520 con 0x7fa9ec004d00 2023-08-09T15:11:47.568 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:47.562+0000 7faa0635b640 1 --2- 172.21.15.43:0/1087350053 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa9ec004d00 0x7fa9ec005180 secure :-1 s=READY pgs=2459 cs=0 l=1 rev1=1 crypto rx=0x7fa9f8009fc0 tx=0x7fa9ec0057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1951c1dba7d5f938 server_cookie=0 in_seq=0 out_seq=0 2023-08-09T15:11:47.568 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:47.562+0000 7faa04b58640 1 -- 172.21.15.43:0/1087350053 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa9f802cc60 con 0x7fa9ec004d00 2023-08-09T15:11:47.568 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:47.562+0000 7faa04b58640 1 -- 172.21.15.43:0/1087350053 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa9f800f040 con 0x7fa9ec004d00 2023-08-09T15:11:47.568 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:47.562+0000 7faa04b58640 1 -- 172.21.15.43:0/1087350053 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa9f8034440 con 0x7fa9ec004d00 2023-08-09T15:11:47.568 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:47.562+0000 7faa07dbe640 1 -- 172.21.15.43:0/1087350053 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa9ec004d00 msgr2=0x7fa9ec005180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T15:11:47.569 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:47.562+0000 7faa07dbe640 1 --2- 172.21.15.43:0/1087350053 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa9ec004d00 0x7fa9ec005180 secure :-1 s=READY pgs=2459 cs=0 l=1 rev1=1 crypto rx=0x7fa9f8009fc0 tx=0x7fa9ec0057f0 comp rx=0 tx=0).stop 2023-08-09T15:11:47.569 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:47.562+0000 7faa0635b640 1 -- 172.21.15.43:0/1087350053 reap_dead start 2023-08-09T15:11:47.569 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:47.563+0000 7faa07dbe640 1 -- 172.21.15.43:0/1087350053 shutdown_connections 2023-08-09T15:11:47.569 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:47.563+0000 7faa07dbe640 1 -- 172.21.15.43:0/1087350053 >> 172.21.15.43:0/1087350053 conn(0x7faa0008bfb0 msgr2=0x7faa0009b8d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-09T15:11:47.569 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:47.563+0000 7faa07dbe640 1 -- 172.21.15.43:0/1087350053 shutdown_connections 2023-08-09T15:11:47.569 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:47.563+0000 7faa07dbe640 1 -- 172.21.15.43:0/1087350053 wait complete. 2023-08-09T15:11:47.569 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:47.564+0000 7faa07dbe640 1 Processor -- start 2023-08-09T15:11:47.570 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:47.565+0000 7faa07dbe640 1 -- start start 2023-08-09T15:11:47.570 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:47.566+0000 7faa07dbe640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa9ec004d00 0x7faa001f1930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T15:11:47.570 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:47.566+0000 7faa07dbe640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7faa000ca4a0 0x7faa000cb430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T15:11:47.570 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:47.566+0000 7faa07dbe640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7faa000cb970 0x7faa0015f410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T15:11:47.570 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:47.566+0000 7faa07dbe640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faa000ca010 con 0x7fa9ec004d00 2023-08-09T15:11:47.570 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:47.566+0000 7faa07dbe640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faa001f1e70 con 0x7faa000cb970 2023-08-09T15:11:47.570 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:47.566+0000 7faa07dbe640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7faa000cbef0 con 0x7faa000ca4a0 2023-08-09T15:11:47.571 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:47.566+0000 7faa05b5a640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa9ec004d00 0x7faa001f1930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T15:11:47.571 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:47.566+0000 7faa05b5a640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa9ec004d00 0x7faa001f1930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:50162/0 (socket says 172.21.15.43:50162) 2023-08-09T15:11:47.571 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:47.566+0000 7faa05b5a640 1 -- 172.21.15.43:0/3965573813 learned_addr learned my addr 172.21.15.43:0/3965573813 (peer_addr_for_me v2:172.21.15.43:0/0) 2023-08-09T15:11:47.571 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:47.566+0000 7faa05b5a640 1 -- 172.21.15.43:0/3965573813 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7faa000ca4a0 msgr2=0x7faa000cb430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T15:11:47.571 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:47.566+0000 7faa05b5a640 1 --2- 172.21.15.43:0/3965573813 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7faa000ca4a0 0x7faa000cb430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T15:11:47.571 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:47.566+0000 7faa05b5a640 1 -- 172.21.15.43:0/3965573813 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7faa000cb970 msgr2=0x7faa0015f410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T15:11:47.571 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:47.566+0000 7faa05b5a640 1 --2- 172.21.15.43:0/3965573813 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7faa000cb970 0x7faa0015f410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T15:11:47.572 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:47.566+0000 7faa05b5a640 1 -- 172.21.15.43:0/3965573813 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faa000c96e0 con 0x7fa9ec004d00 2023-08-09T15:11:47.572 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:47.567+0000 7faa05b5a640 1 --2- 172.21.15.43:0/3965573813 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa9ec004d00 0x7faa001f1930 secure :-1 s=READY pgs=2460 cs=0 l=1 rev1=1 crypto rx=0x7fa9fc00e7e0 tx=0x7fa9fc00eca0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-09T15:11:47.573 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:47.567+0000 7fa9f67fc640 1 -- 172.21.15.43:0/3965573813 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa9fc0072e0 con 0x7fa9ec004d00 2023-08-09T15:11:47.573 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:47.567+0000 7fa9f67fc640 1 -- 172.21.15.43:0/3965573813 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa9fc007440 con 0x7fa9ec004d00 2023-08-09T15:11:47.573 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:47.567+0000 7fa9f67fc640 1 -- 172.21.15.43:0/3965573813 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa9fc0057a0 con 0x7fa9ec004d00 2023-08-09T15:11:47.573 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:47.567+0000 7faa07dbe640 1 -- 172.21.15.43:0/3965573813 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7faa000c99c0 con 0x7fa9ec004d00 2023-08-09T15:11:47.573 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:47.567+0000 7faa07dbe640 1 -- 172.21.15.43:0/3965573813 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7faa000ca920 con 0x7fa9ec004d00 2023-08-09T15:11:47.573 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:47.568+0000 7faa07dbe640 1 -- 172.21.15.43:0/3965573813 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7faa0004e390 con 0x7fa9ec004d00 2023-08-09T15:11:47.574 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:47.569+0000 7fa9f67fc640 1 -- 172.21.15.43:0/3965573813 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 123) v1 ==== 88517+0+0 (secure 0 0 0) 0x7fa9fc005900 con 0x7fa9ec004d00 2023-08-09T15:11:47.574 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:47.570+0000 7fa9f67fc640 1 --2- 172.21.15.43:0/3965573813 >> [v2:172.21.15.203:6832/2375771024,v1:172.21.15.203:6833/2375771024] conn(0x7fa9e406a250 0x7fa9e406c710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T15:11:47.574 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:47.570+0000 7fa9f67fc640 1 -- 172.21.15.43:0/3965573813 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(224..224 src has 1..224) v4 ==== 6770+0+0 (secure 0 0 0) 0x7fa9fc088c60 con 0x7fa9ec004d00 2023-08-09T15:11:47.574 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:47.571+0000 7faa05359640 1 --2- 172.21.15.43:0/3965573813 >> [v2:172.21.15.203:6832/2375771024,v1:172.21.15.203:6833/2375771024] conn(0x7fa9e406a250 0x7fa9e406c710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T15:11:47.574 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:47.572+0000 7faa05359640 1 --2- 172.21.15.43:0/3965573813 >> [v2:172.21.15.203:6832/2375771024,v1:172.21.15.203:6833/2375771024] conn(0x7fa9e406a250 0x7fa9e406c710 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7faa001f54a0 tx=0x7fa9f0008040 comp rx=0 tx=0).ready entity=mgr.9103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-09T15:11:47.576 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:47.574+0000 7fa9f67fc640 1 -- 172.21.15.43:0/3965573813 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178066 (secure 0 0 0) 0x7fa9fc053e00 con 0x7fa9ec004d00 2023-08-09T15:11:47.769 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:47.768+0000 7faa07dbe640 1 -- 172.21.15.43:0/3965573813 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7faa0006bb60 con 0x7fa9ec004d00 2023-08-09T15:11:47.770 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:47.769+0000 7fa9f67fc640 1 -- 172.21.15.43:0/3965573813 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v224) v1 ==== 100+0+11 (secure 0 0 0) 0x7fa9fc057420 con 0x7fa9ec004d00 2023-08-09T15:11:47.770 INFO:teuthology.orchestra.run.smithi043.stdout:pg_num: 64 2023-08-09T15:11:47.773 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:47.771+0000 7faa07dbe640 1 -- 172.21.15.43:0/3965573813 >> [v2:172.21.15.203:6832/2375771024,v1:172.21.15.203:6833/2375771024] conn(0x7fa9e406a250 msgr2=0x7fa9e406c710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T15:11:47.773 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:47.771+0000 7faa07dbe640 1 --2- 172.21.15.43:0/3965573813 >> [v2:172.21.15.203:6832/2375771024,v1:172.21.15.203:6833/2375771024] conn(0x7fa9e406a250 0x7fa9e406c710 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7faa001f54a0 tx=0x7fa9f0008040 comp rx=0 tx=0).stop 2023-08-09T15:11:47.773 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:47.771+0000 7faa07dbe640 1 -- 172.21.15.43:0/3965573813 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa9ec004d00 msgr2=0x7faa001f1930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T15:11:47.773 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:47.771+0000 7faa07dbe640 1 --2- 172.21.15.43:0/3965573813 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa9ec004d00 0x7faa001f1930 secure :-1 s=READY pgs=2460 cs=0 l=1 rev1=1 crypto rx=0x7fa9fc00e7e0 tx=0x7fa9fc00eca0 comp rx=0 tx=0).stop 2023-08-09T15:11:47.773 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:47.771+0000 7faa07dbe640 1 -- 172.21.15.43:0/3965573813 shutdown_connections 2023-08-09T15:11:47.774 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:47.771+0000 7faa07dbe640 1 --2- 172.21.15.43:0/3965573813 >> [v2:172.21.15.203:6832/2375771024,v1:172.21.15.203:6833/2375771024] conn(0x7fa9e406a250 0x7fa9e406c710 unknown :-1 s=CLOSED pgs=29 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T15:11:47.774 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:47.771+0000 7faa07dbe640 1 --2- 172.21.15.43:0/3965573813 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7faa000cb970 0x7faa0015f410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T15:11:47.774 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:47.771+0000 7faa07dbe640 1 --2- 172.21.15.43:0/3965573813 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7faa000ca4a0 0x7faa000cb430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T15:11:47.774 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:47.771+0000 7faa07dbe640 1 --2- 172.21.15.43:0/3965573813 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa9ec004d00 0x7faa001f1930 unknown :-1 s=CLOSED pgs=2460 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T15:11:47.774 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:47.771+0000 7faa07dbe640 1 -- 172.21.15.43:0/3965573813 >> 172.21.15.43:0/3965573813 conn(0x7faa0008bfb0 msgr2=0x7faa000d1870 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-09T15:11:47.774 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:47.772+0000 7faa07dbe640 1 -- 172.21.15.43:0/3965573813 shutdown_connections 2023-08-09T15:11:47.774 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:47.772+0000 7faa07dbe640 1 -- 172.21.15.43:0/3965573813 wait complete. 2023-08-09T15:11:47.796 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2023-08-09T15:11:48.027 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:48.026+0000 7fd458cec640 1 Processor -- start 2023-08-09T15:11:48.028 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:48.026+0000 7fd458cec640 1 -- start start 2023-08-09T15:11:48.029 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:48.028+0000 7fd458cec640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fd45406b4d0 0x7fd45406b950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T15:11:48.029 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:48.028+0000 7fd458cec640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7fd4540709b0 0x7fd454070e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T15:11:48.029 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:48.028+0000 7fd458cec640 1 -- --> v1:172.21.15.203:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd45415de60 con 0x7fd4541570b0 2023-08-09T15:11:48.030 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:48.028+0000 7fd458cec640 1 -- --> v2:172.21.15.43:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd454070580 con 0x7fd4540709b0 2023-08-09T15:11:48.030 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:48.028+0000 7fd458cec640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd45406be90 con 0x7fd45406b4d0 2023-08-09T15:11:48.031 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:48.028+0000 7fd452d9d640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7fd4540709b0 0x7fd454070e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T15:11:48.031 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:48.028+0000 7fd452d9d640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7fd4540709b0 0x7fd454070e40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:50170/0 (socket says 172.21.15.43:50170) 2023-08-09T15:11:48.031 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:48.028+0000 7fd452d9d640 1 -- 172.21.15.43:0/1894561199 learned_addr learned my addr 172.21.15.43:0/1894561199 (peer_addr_for_me v2:172.21.15.43:0/0) 2023-08-09T15:11:48.031 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:48.029+0000 7fd451d9b640 1 --2- 172.21.15.43:0/1894561199 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fd45406b4d0 0x7fd45406b950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T15:11:48.032 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:48.029+0000 7fd452d9d640 1 -- 172.21.15.43:0/1894561199 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fd45406b4d0 msgr2=0x7fd45406b950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-09T15:11:48.032 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:48.029+0000 7fd452d9d640 1 --2- 172.21.15.43:0/1894561199 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fd45406b4d0 0x7fd45406b950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T15:11:48.032 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:48.029+0000 7fd452d9d640 1 -- 172.21.15.43:0/1894561199 >> v1:172.21.15.203:6789/0 conn(0x7fd4541570b0 legacy=0x7fd4540cd950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-09T15:11:48.032 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:48.029+0000 7fd452d9d640 1 -- 172.21.15.43:0/1894561199 --> v2:172.21.15.43:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd45415d2e0 con 0x7fd4540709b0 2023-08-09T15:11:48.033 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:48.030+0000 7fd452d9d640 1 --2- 172.21.15.43:0/1894561199 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fd4540709b0 0x7fd454070e40 secure :-1 s=READY pgs=2461 cs=0 l=1 rev1=1 crypto rx=0x7fd44c009870 tx=0x7fd44c02b8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=baae0d45071cf19f server_cookie=0 in_seq=0 out_seq=0 2023-08-09T15:11:48.034 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:48.030+0000 7fd45159a640 1 -- 172.21.15.43:0/1894561199 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd44c031640 con 0x7fd4540709b0 2023-08-09T15:11:48.034 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:48.030+0000 7fd45159a640 1 -- 172.21.15.43:0/1894561199 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd44c0317a0 con 0x7fd4540709b0 2023-08-09T15:11:48.034 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:48.030+0000 7fd45159a640 1 -- 172.21.15.43:0/1894561199 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd44c031b30 con 0x7fd4540709b0 2023-08-09T15:11:48.034 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:48.031+0000 7fd458cec640 1 -- 172.21.15.43:0/1894561199 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fd4540709b0 msgr2=0x7fd454070e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T15:11:48.034 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:48.031+0000 7fd458cec640 1 --2- 172.21.15.43:0/1894561199 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fd4540709b0 0x7fd454070e40 secure :-1 s=READY pgs=2461 cs=0 l=1 rev1=1 crypto rx=0x7fd44c009870 tx=0x7fd44c02b8f0 comp rx=0 tx=0).stop 2023-08-09T15:11:48.034 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:48.031+0000 7fd458cec640 1 -- 172.21.15.43:0/1894561199 shutdown_connections 2023-08-09T15:11:48.035 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:48.031+0000 7fd458cec640 1 --2- 172.21.15.43:0/1894561199 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fd4540709b0 0x7fd454070e40 unknown :-1 s=CLOSED pgs=2461 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T15:11:48.035 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:48.031+0000 7fd458cec640 1 --2- 172.21.15.43:0/1894561199 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fd45406b4d0 0x7fd45406b950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T15:11:48.035 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:48.031+0000 7fd458cec640 1 -- 172.21.15.43:0/1894561199 >> 172.21.15.43:0/1894561199 conn(0x7fd454069dd0 msgr2=0x7fd4540cac80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-09T15:11:48.035 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:48.031+0000 7fd458cec640 1 -- 172.21.15.43:0/1894561199 shutdown_connections 2023-08-09T15:11:48.035 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:48.031+0000 7fd458cec640 1 -- 172.21.15.43:0/1894561199 wait complete. 2023-08-09T15:11:48.036 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:48.034+0000 7fd458cec640 1 Processor -- start 2023-08-09T15:11:48.036 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:48.034+0000 7fd458cec640 1 -- start start 2023-08-09T15:11:48.037 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:48.035+0000 7fd458cec640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fd45406b4d0 0x7fd45414a760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T15:11:48.037 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:48.035+0000 7fd458cec640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fd4540709b0 0x7fd4540acd90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T15:11:48.037 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:48.036+0000 7fd458cec640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fd4541570b0 0x7fd4540a9b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T15:11:48.038 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:48.036+0000 7fd458cec640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd4540a9710 con 0x7fd45406b4d0 2023-08-09T15:11:48.038 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:48.036+0000 7fd458cec640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd45414aca0 con 0x7fd4541570b0 2023-08-09T15:11:48.038 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:48.036+0000 7fd458cec640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd4540ad2d0 con 0x7fd4540709b0 2023-08-09T15:11:48.038 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:48.036+0000 7fd45259c640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fd45406b4d0 0x7fd45414a760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T15:11:48.038 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:48.036+0000 7fd45259c640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fd45406b4d0 0x7fd45414a760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:50184/0 (socket says 172.21.15.43:50184) 2023-08-09T15:11:48.038 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:48.036+0000 7fd45259c640 1 -- 172.21.15.43:0/1835075527 learned_addr learned my addr 172.21.15.43:0/1835075527 (peer_addr_for_me v2:172.21.15.43:0/0) 2023-08-09T15:11:48.039 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:48.036+0000 7fd452d9d640 1 --2- 172.21.15.43:0/1835075527 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fd4541570b0 0x7fd4540a9b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T15:11:48.040 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:48.036+0000 7fd451d9b640 1 --2- 172.21.15.43:0/1835075527 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fd4540709b0 0x7fd4540acd90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T15:11:48.040 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:48.036+0000 7fd45259c640 1 -- 172.21.15.43:0/1835075527 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fd4540709b0 msgr2=0x7fd4540acd90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T15:11:48.040 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:48.036+0000 7fd45259c640 1 --2- 172.21.15.43:0/1835075527 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fd4540709b0 0x7fd4540acd90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T15:11:48.040 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:48.036+0000 7fd45259c640 1 -- 172.21.15.43:0/1835075527 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fd4541570b0 msgr2=0x7fd4540a9b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T15:11:48.040 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:48.036+0000 7fd45259c640 1 --2- 172.21.15.43:0/1835075527 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fd4541570b0 0x7fd4540a9b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T15:11:48.040 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:48.036+0000 7fd45259c640 1 -- 172.21.15.43:0/1835075527 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd44c0094d0 con 0x7fd45406b4d0 2023-08-09T15:11:48.040 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:48.036+0000 7fd452d9d640 1 --2- 172.21.15.43:0/1835075527 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fd4541570b0 0x7fd4540a9b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-08-09T15:11:48.041 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:48.037+0000 7fd45259c640 1 --2- 172.21.15.43:0/1835075527 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fd45406b4d0 0x7fd45414a760 secure :-1 s=READY pgs=2462 cs=0 l=1 rev1=1 crypto rx=0x7fd44400ba50 tx=0x7fd44400bf10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-09T15:11:48.041 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:48.037+0000 7fd44affd640 1 -- 172.21.15.43:0/1835075527 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd44400cbe0 con 0x7fd45406b4d0 2023-08-09T15:11:48.041 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:48.037+0000 7fd44affd640 1 -- 172.21.15.43:0/1835075527 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd44400cd80 con 0x7fd45406b4d0 2023-08-09T15:11:48.041 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:48.037+0000 7fd458cec640 1 -- 172.21.15.43:0/1835075527 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd4540a8a90 con 0x7fd45406b4d0 2023-08-09T15:11:48.041 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:48.037+0000 7fd44affd640 1 -- 172.21.15.43:0/1835075527 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd444014680 con 0x7fd45406b4d0 2023-08-09T15:11:48.041 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:48.037+0000 7fd458cec640 1 -- 172.21.15.43:0/1835075527 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd4540a9050 con 0x7fd45406b4d0 2023-08-09T15:11:48.042 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:48.038+0000 7fd458cec640 1 -- 172.21.15.43:0/1835075527 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd45404db40 con 0x7fd45406b4d0 2023-08-09T15:11:48.042 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:48.039+0000 7fd44affd640 1 -- 172.21.15.43:0/1835075527 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 123) v1 ==== 88517+0+0 (secure 0 0 0) 0x7fd4440147e0 con 0x7fd45406b4d0 2023-08-09T15:11:48.043 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:48.040+0000 7fd44affd640 1 --2- 172.21.15.43:0/1835075527 >> [v2:172.21.15.203:6832/2375771024,v1:172.21.15.203:6833/2375771024] conn(0x7fd43406a250 0x7fd43406c710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T15:11:48.043 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:48.040+0000 7fd44affd640 1 -- 172.21.15.43:0/1835075527 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(224..224 src has 1..224) v4 ==== 6770+0+0 (secure 0 0 0) 0x7fd444088a80 con 0x7fd45406b4d0 2023-08-09T15:11:48.043 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:48.041+0000 7fd451d9b640 1 --2- 172.21.15.43:0/1835075527 >> [v2:172.21.15.203:6832/2375771024,v1:172.21.15.203:6833/2375771024] conn(0x7fd43406a250 0x7fd43406c710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T15:11:48.043 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:48.042+0000 7fd451d9b640 1 --2- 172.21.15.43:0/1835075527 >> [v2:172.21.15.203:6832/2375771024,v1:172.21.15.203:6833/2375771024] conn(0x7fd43406a250 0x7fd43406c710 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0x7fd43c006fd0 tx=0x7fd43c008040 comp rx=0 tx=0).ready entity=mgr.9103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-09T15:11:48.046 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:48.045+0000 7fd44affd640 1 -- 172.21.15.43:0/1835075527 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178066 (secure 0 0 0) 0x7fd444053cd0 con 0x7fd45406b4d0 2023-08-09T15:11:48.256 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:48.254+0000 7fd458cec640 1 -- 172.21.15.43:0/1835075527 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7fd454156950 con 0x7fd45406b4d0 2023-08-09T15:11:48.257 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:48.255+0000 7fd44affd640 1 -- 172.21.15.43:0/1835075527 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 8 entries v224) v1 ==== 80+0+476 (secure 0 0 0) 0x7fd4440191f0 con 0x7fd45406b4d0 2023-08-09T15:11:48.257 INFO:teuthology.orchestra.run.smithi043.stdout:172.21.15.203:6832/4034093268 2023-08-10T15:11:43.130145+0000 2023-08-09T15:11:48.258 INFO:teuthology.orchestra.run.smithi043.stdout:172.21.15.203:6833/4034093268 2023-08-10T15:11:43.130145+0000 2023-08-09T15:11:48.258 INFO:teuthology.orchestra.run.smithi043.stdout:172.21.15.203:0/1654193324 2023-08-10T15:11:43.130145+0000 2023-08-09T15:11:48.258 INFO:teuthology.orchestra.run.smithi043.stdout:172.21.15.203:0/4140323243 2023-08-10T15:11:43.130145+0000 2023-08-09T15:11:48.258 INFO:teuthology.orchestra.run.smithi043.stdout:172.21.15.203:0/450863258 2023-08-10T15:11:43.130145+0000 2023-08-09T15:11:48.258 INFO:teuthology.orchestra.run.smithi043.stdout:172.21.15.203:0/134877587 2023-08-10T15:11:43.130145+0000 2023-08-09T15:11:48.258 INFO:teuthology.orchestra.run.smithi043.stdout:172.21.15.203:0/2330869600 2023-08-10T15:11:43.130145+0000 2023-08-09T15:11:48.259 INFO:teuthology.orchestra.run.smithi043.stdout:172.21.15.203:0/2203818837 2023-08-10T15:11:43.130145+0000 2023-08-09T15:11:48.260 INFO:teuthology.orchestra.run.smithi043.stderr:listed 8 entries 2023-08-09T15:11:48.260 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:48.258+0000 7fd458cec640 1 -- 172.21.15.43:0/1835075527 >> [v2:172.21.15.203:6832/2375771024,v1:172.21.15.203:6833/2375771024] conn(0x7fd43406a250 msgr2=0x7fd43406c710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T15:11:48.260 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:48.258+0000 7fd458cec640 1 --2- 172.21.15.43:0/1835075527 >> [v2:172.21.15.203:6832/2375771024,v1:172.21.15.203:6833/2375771024] conn(0x7fd43406a250 0x7fd43406c710 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0x7fd43c006fd0 tx=0x7fd43c008040 comp rx=0 tx=0).stop 2023-08-09T15:11:48.260 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:48.258+0000 7fd458cec640 1 -- 172.21.15.43:0/1835075527 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fd45406b4d0 msgr2=0x7fd45414a760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T15:11:48.260 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:48.258+0000 7fd458cec640 1 --2- 172.21.15.43:0/1835075527 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fd45406b4d0 0x7fd45414a760 secure :-1 s=READY pgs=2462 cs=0 l=1 rev1=1 crypto rx=0x7fd44400ba50 tx=0x7fd44400bf10 comp rx=0 tx=0).stop 2023-08-09T15:11:48.261 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:48.258+0000 7fd458cec640 1 -- 172.21.15.43:0/1835075527 shutdown_connections 2023-08-09T15:11:48.261 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:48.258+0000 7fd458cec640 1 --2- 172.21.15.43:0/1835075527 >> [v2:172.21.15.203:6832/2375771024,v1:172.21.15.203:6833/2375771024] conn(0x7fd43406a250 0x7fd43406c710 unknown :-1 s=CLOSED pgs=30 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T15:11:48.261 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:48.258+0000 7fd458cec640 1 --2- 172.21.15.43:0/1835075527 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fd4541570b0 0x7fd4540a9b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T15:11:48.261 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:48.258+0000 7fd458cec640 1 --2- 172.21.15.43:0/1835075527 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fd4540709b0 0x7fd4540acd90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T15:11:48.261 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:48.258+0000 7fd458cec640 1 --2- 172.21.15.43:0/1835075527 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fd45406b4d0 0x7fd45414a760 unknown :-1 s=CLOSED pgs=2462 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T15:11:48.261 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:48.258+0000 7fd458cec640 1 -- 172.21.15.43:0/1835075527 >> 172.21.15.43:0/1835075527 conn(0x7fd454069dd0 msgr2=0x7fd4540c4a80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-09T15:11:48.262 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:48.258+0000 7fd458cec640 1 -- 172.21.15.43:0/1835075527 shutdown_connections 2023-08-09T15:11:48.262 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:48.258+0000 7fd458cec640 1 -- 172.21.15.43:0/1835075527 wait complete. 2023-08-09T15:11:48.280 DEBUG:teuthology.orchestra.run.smithi043:> set -ex 2023-08-09T15:11:48.280 DEBUG:teuthology.orchestra.run.smithi043:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-09T15:11:48.335 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-08-09T15:11:48.563 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:48.562+0000 7fd6c2722640 1 Processor -- start 2023-08-09T15:11:48.564 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:48.562+0000 7fd6c2722640 1 -- start start 2023-08-09T15:11:48.565 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:48.563+0000 7fd6c2722640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fd6bc0cf510 0x7fd6bc0cd270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T15:11:48.565 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:48.563+0000 7fd6c2722640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7fd6bc0d14c0 0x7fd6bc06b7f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T15:11:48.565 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:48.563+0000 7fd6c2722640 1 -- --> v1:172.21.15.203:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd6bc06a7e0 con 0x7fd6bc15fe60 2023-08-09T15:11:48.565 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:48.563+0000 7fd6c2722640 1 -- --> v2:172.21.15.43:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd6bc06b360 con 0x7fd6bc0d14c0 2023-08-09T15:11:48.565 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:48.563+0000 7fd6c2722640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd6bc06aa30 con 0x7fd6bc0cf510 2023-08-09T15:11:48.565 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:48.564+0000 7fd6c0cbf640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7fd6bc0d14c0 0x7fd6bc06b7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T15:11:48.566 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:48.564+0000 7fd6c0cbf640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7fd6bc0d14c0 0x7fd6bc06b7f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:50186/0 (socket says 172.21.15.43:50186) 2023-08-09T15:11:48.566 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:48.564+0000 7fd6c0cbf640 1 -- 172.21.15.43:0/2521270628 learned_addr learned my addr 172.21.15.43:0/2521270628 (peer_addr_for_me v2:172.21.15.43:0/0) 2023-08-09T15:11:48.566 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:48.564+0000 7fd6b37fe640 1 --2- 172.21.15.43:0/2521270628 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fd6bc0cf510 0x7fd6bc0cd270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T15:11:48.567 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:48.564+0000 7fd6c0cbf640 1 -- 172.21.15.43:0/2521270628 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fd6bc0cf510 msgr2=0x7fd6bc0cd270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-09T15:11:48.567 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:48.564+0000 7fd6c0cbf640 1 --2- 172.21.15.43:0/2521270628 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fd6bc0cf510 0x7fd6bc0cd270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T15:11:48.567 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:48.564+0000 7fd6c0cbf640 1 -- 172.21.15.43:0/2521270628 >> v1:172.21.15.203:6789/0 conn(0x7fd6bc15fe60 legacy=0x7fd6bc0cee00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-09T15:11:48.567 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:48.564+0000 7fd6c0cbf640 1 -- 172.21.15.43:0/2521270628 --> v2:172.21.15.43:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd6bc06ad10 con 0x7fd6bc0d14c0 2023-08-09T15:11:48.567 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:48.564+0000 7fd6b37fe640 1 --2- 172.21.15.43:0/2521270628 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fd6bc0cf510 0x7fd6bc0cd270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-08-09T15:11:48.567 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:48.565+0000 7fd6c0cbf640 1 --2- 172.21.15.43:0/2521270628 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fd6bc0d14c0 0x7fd6bc06b7f0 secure :-1 s=READY pgs=2463 cs=0 l=1 rev1=1 crypto rx=0x7fd6b4009e50 tx=0x7fd6b402bee0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=94d7705144c29047 server_cookie=0 in_seq=0 out_seq=0 2023-08-09T15:11:48.568 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:48.566+0000 7fd6bb7fe640 1 -- 172.21.15.43:0/2521270628 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd6b402ca80 con 0x7fd6bc0d14c0 2023-08-09T15:11:48.568 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:48.566+0000 7fd6bb7fe640 1 -- 172.21.15.43:0/2521270628 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd6b402cbe0 con 0x7fd6bc0d14c0 2023-08-09T15:11:48.569 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:48.566+0000 7fd6bb7fe640 1 -- 172.21.15.43:0/2521270628 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd6b40343f0 con 0x7fd6bc0d14c0 2023-08-09T15:11:48.569 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:48.566+0000 7fd6c2722640 1 -- 172.21.15.43:0/2521270628 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fd6bc0d14c0 msgr2=0x7fd6bc06b7f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T15:11:48.569 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:48.566+0000 7fd6c2722640 1 --2- 172.21.15.43:0/2521270628 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fd6bc0d14c0 0x7fd6bc06b7f0 secure :-1 s=READY pgs=2463 cs=0 l=1 rev1=1 crypto rx=0x7fd6b4009e50 tx=0x7fd6b402bee0 comp rx=0 tx=0).stop 2023-08-09T15:11:48.569 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:48.567+0000 7fd6c2722640 1 -- 172.21.15.43:0/2521270628 shutdown_connections 2023-08-09T15:11:48.569 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:48.567+0000 7fd6c2722640 1 --2- 172.21.15.43:0/2521270628 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fd6bc0d14c0 0x7fd6bc06b7f0 unknown :-1 s=CLOSED pgs=2463 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T15:11:48.569 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:48.567+0000 7fd6c2722640 1 --2- 172.21.15.43:0/2521270628 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fd6bc0cf510 0x7fd6bc0cd270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T15:11:48.569 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:48.567+0000 7fd6c2722640 1 -- 172.21.15.43:0/2521270628 >> 172.21.15.43:0/2521270628 conn(0x7fd6bc0d2660 msgr2=0x7fd6bc0d2a70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-09T15:11:48.569 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:48.567+0000 7fd6c2722640 1 -- 172.21.15.43:0/2521270628 shutdown_connections 2023-08-09T15:11:48.570 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:48.567+0000 7fd6c2722640 1 -- 172.21.15.43:0/2521270628 wait complete. 2023-08-09T15:11:48.570 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:48.568+0000 7fd6c2722640 1 Processor -- start 2023-08-09T15:11:48.570 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:48.569+0000 7fd6c2722640 1 -- start start 2023-08-09T15:11:48.571 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:48.570+0000 7fd6c2722640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fd6bc0cf510 0x7fd6bc0c9730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T15:11:48.571 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:48.570+0000 7fd6c2722640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fd6bc0d14c0 0x7fd6bc0c9c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T15:11:48.571 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:48.570+0000 7fd6c2722640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fd6bc15fe60 0x7fd6bc15aa50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T15:11:48.572 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:48.570+0000 7fd6c2722640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd6bc0cb3f0 con 0x7fd6bc15fe60 2023-08-09T15:11:48.572 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:48.570+0000 7fd6c2722640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd6bc0b6730 con 0x7fd6bc0d14c0 2023-08-09T15:11:48.572 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:48.570+0000 7fd6c2722640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd6bc0cbc00 con 0x7fd6bc0cf510 2023-08-09T15:11:48.572 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:48.570+0000 7fd6c0cbf640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fd6bc15fe60 0x7fd6bc15aa50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T15:11:48.572 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:48.570+0000 7fd6c0cbf640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fd6bc15fe60 0x7fd6bc15aa50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:50188/0 (socket says 172.21.15.43:50188) 2023-08-09T15:11:48.572 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:48.570+0000 7fd6c0cbf640 1 -- 172.21.15.43:0/1979070486 learned_addr learned my addr 172.21.15.43:0/1979070486 (peer_addr_for_me v2:172.21.15.43:0/0) 2023-08-09T15:11:48.573 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:48.570+0000 7fd6b37fe640 1 --2- 172.21.15.43:0/1979070486 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fd6bc0d14c0 0x7fd6bc0c9c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T15:11:48.573 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:48.570+0000 7fd6bbfff640 1 --2- 172.21.15.43:0/1979070486 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fd6bc0cf510 0x7fd6bc0c9730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T15:11:48.574 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:48.570+0000 7fd6c0cbf640 1 -- 172.21.15.43:0/1979070486 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fd6bc0cf510 msgr2=0x7fd6bc0c9730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T15:11:48.574 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:48.570+0000 7fd6c0cbf640 1 --2- 172.21.15.43:0/1979070486 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fd6bc0cf510 0x7fd6bc0c9730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T15:11:48.574 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:48.570+0000 7fd6c0cbf640 1 -- 172.21.15.43:0/1979070486 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fd6bc0d14c0 msgr2=0x7fd6bc0c9c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T15:11:48.574 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:48.570+0000 7fd6c0cbf640 1 --2- 172.21.15.43:0/1979070486 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fd6bc0d14c0 0x7fd6bc0c9c70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T15:11:48.574 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:48.570+0000 7fd6c0cbf640 1 -- 172.21.15.43:0/1979070486 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd6b4009b30 con 0x7fd6bc15fe60 2023-08-09T15:11:48.574 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:48.571+0000 7fd6c0cbf640 1 --2- 172.21.15.43:0/1979070486 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fd6bc15fe60 0x7fd6bc15aa50 secure :-1 s=READY pgs=2464 cs=0 l=1 rev1=1 crypto rx=0x7fd6b4036040 tx=0x7fd6b4004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-09T15:11:48.574 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:48.571+0000 7fd6b97fa640 1 -- 172.21.15.43:0/1979070486 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd6b403e450 con 0x7fd6bc15fe60 2023-08-09T15:11:48.575 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:48.571+0000 7fd6b97fa640 1 -- 172.21.15.43:0/1979070486 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd6b403e5d0 con 0x7fd6bc15fe60 2023-08-09T15:11:48.575 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:48.571+0000 7fd6b97fa640 1 -- 172.21.15.43:0/1979070486 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd6b403e980 con 0x7fd6bc15fe60 2023-08-09T15:11:48.575 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:48.571+0000 7fd6c2722640 1 -- 172.21.15.43:0/1979070486 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd6bc0cb640 con 0x7fd6bc15fe60 2023-08-09T15:11:48.575 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:48.571+0000 7fd6c2722640 1 -- 172.21.15.43:0/1979070486 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd6bc0b63c0 con 0x7fd6bc15fe60 2023-08-09T15:11:48.575 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:48.573+0000 7fd6c2722640 1 -- 172.21.15.43:0/1979070486 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd6bc0b68e0 con 0x7fd6bc15fe60 2023-08-09T15:11:48.576 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:48.574+0000 7fd6b97fa640 1 -- 172.21.15.43:0/1979070486 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 123) v1 ==== 88517+0+0 (secure 0 0 0) 0x7fd6b402cd50 con 0x7fd6bc15fe60 2023-08-09T15:11:48.576 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:48.575+0000 7fd6b97fa640 1 --2- 172.21.15.43:0/1979070486 >> [v2:172.21.15.203:6832/2375771024,v1:172.21.15.203:6833/2375771024] conn(0x7fd69c06a250 0x7fd69c06c710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T15:11:48.576 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:48.575+0000 7fd6b97fa640 1 -- 172.21.15.43:0/1979070486 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(224..224 src has 1..224) v4 ==== 6770+0+0 (secure 0 0 0) 0x7fd6b40ab800 con 0x7fd6bc15fe60 2023-08-09T15:11:48.577 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:48.575+0000 7fd6bbfff640 1 --2- 172.21.15.43:0/1979070486 >> [v2:172.21.15.203:6832/2375771024,v1:172.21.15.203:6833/2375771024] conn(0x7fd69c06a250 0x7fd69c06c710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T15:11:48.577 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:48.576+0000 7fd6bbfff640 1 --2- 172.21.15.43:0/1979070486 >> [v2:172.21.15.203:6832/2375771024,v1:172.21.15.203:6833/2375771024] conn(0x7fd69c06a250 0x7fd69c06c710 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0x7fd6ac006fd0 tx=0x7fd6ac008040 comp rx=0 tx=0).ready entity=mgr.9103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-09T15:11:48.580 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:48.579+0000 7fd6b97fa640 1 -- 172.21.15.43:0/1979070486 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178066 (secure 0 0 0) 0x7fd6bc0b68e0 con 0x7fd6bc15fe60 2023-08-09T15:11:48.773 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:48.772+0000 7fd6c2722640 1 -- 172.21.15.43:0/1979070486 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fd6bc0b68a0 con 0x7fd6bc15fe60 2023-08-09T15:11:48.775 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:48.774+0000 7fd6b97fa640 1 -- 172.21.15.43:0/1979070486 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v224) v1 ==== 74+0+19077 (secure 0 0 0) 0x7fd6b40769a0 con 0x7fd6bc15fe60 2023-08-09T15:11:48.776 INFO:teuthology.orchestra.run.smithi043.stdout: 2023-08-09T15:11:48.777 INFO:teuthology.orchestra.run.smithi043.stdout:{"epoch":224,"fsid":"daac87ee-1337-4250-a955-aea4e408a39b","created":"2023-08-09T13:33:41.701198+0000","modified":"2023-08-09T15:11:43.130258+0000","last_up_change":"2023-08-09T13:33:53.783453+0000","last_in_change":"2023-08-09T13:33:45.148904+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":27,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":"rbd","create_time":"2023-08-09T13:33:54.365137+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":".mgr","create_time":"2023-08-09T13:33:54.831803+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":26,"pool_name":"cephfs_metadata","create_time":"2023-08-09T15:11:26.544757+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"222","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":27,"pool_name":"cephfs_data","create_time":"2023-08-09T15:11:27.194686+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"223","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.6299999952316284,"score_stable":1.6299999952316284,"optimal_score":1,"raw_score_acting":1.6299999952316284,"raw_score_stable":1.6299999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"cbd49ecd-e896-4fbd-864e-5c37a829b165","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":221,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6800","nonce":502667335},{"type":"v1","addr":"172.21.15.43:6801","nonce":502667335}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6802","nonce":502667335},{"type":"v1","addr":"172.21.15.43:6803","nonce":502667335}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6806","nonce":502667335},{"type":"v1","addr":"172.21.15.43:6807","nonce":502667335}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6804","nonce":502667335},{"type":"v1","addr":"172.21.15.43:6805","nonce":502667335}]},"public_addr":"172.21.15.43:6801/502667335","cluster_addr":"172.21.15.43:6803/502667335","heartbeat_back_addr":"172.21.15.43:6807/502667335","heartbeat_front_addr":"172.21.15.43:6805/502667335","state":["exists","up"]},{"osd":1,"uuid":"75c7e0fe-1ec7-4fe8-a2f2-3661a65b8500","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":221,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6808","nonce":3289134610},{"type":"v1","addr":"172.21.15.43:6809","nonce":3289134610}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6810","nonce":3289134610},{"type":"v1","addr":"172.21.15.43:6812","nonce":3289134610}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6821","nonce":3289134610},{"type":"v1","addr":"172.21.15.43:6824","nonce":3289134610}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6815","nonce":3289134610},{"type":"v1","addr":"172.21.15.43:6818","nonce":3289134610}]},"public_addr":"172.21.15.43:6809/3289134610","cluster_addr":"172.21.15.43:6812/3289134610","heartbeat_back_addr":"172.21.15.43:6824/3289134610","heartbeat_front_addr":"172.21.15.43:6818/3289134610","state":["exists","up"]},{"osd":2,"uuid":"bb5c41e9-7f5f-4125-9b01-3bb8bfc2ea83","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":221,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6811","nonce":4079709736},{"type":"v1","addr":"172.21.15.43:6814","nonce":4079709736}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6817","nonce":4079709736},{"type":"v1","addr":"172.21.15.43:6820","nonce":4079709736}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6829","nonce":4079709736},{"type":"v1","addr":"172.21.15.43:6831","nonce":4079709736}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6823","nonce":4079709736},{"type":"v1","addr":"172.21.15.43:6826","nonce":4079709736}]},"public_addr":"172.21.15.43:6814/4079709736","cluster_addr":"172.21.15.43:6820/4079709736","heartbeat_back_addr":"172.21.15.43:6831/4079709736","heartbeat_front_addr":"172.21.15.43:6826/4079709736","state":["exists","up"]},{"osd":3,"uuid":"bfe4c8b4-694a-465c-87e6-2b2aab852613","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":221,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6813","nonce":1709538759},{"type":"v1","addr":"172.21.15.43:6816","nonce":1709538759}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6819","nonce":1709538759},{"type":"v1","addr":"172.21.15.43:6822","nonce":1709538759}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6828","nonce":1709538759},{"type":"v1","addr":"172.21.15.43:6830","nonce":1709538759}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6825","nonce":1709538759},{"type":"v1","addr":"172.21.15.43:6827","nonce":1709538759}]},"public_addr":"172.21.15.43:6816/1709538759","cluster_addr":"172.21.15.43:6822/1709538759","heartbeat_back_addr":"172.21.15.43:6830/1709538759","heartbeat_front_addr":"172.21.15.43:6827/1709538759","state":["exists","up"]},{"osd":4,"uuid":"fa2e8a77-aba1-4756-853e-0d174a5bdeba","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":221,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6800","nonce":4196866481},{"type":"v1","addr":"172.21.15.203:6801","nonce":4196866481}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6802","nonce":4196866481},{"type":"v1","addr":"172.21.15.203:6803","nonce":4196866481}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6806","nonce":4196866481},{"type":"v1","addr":"172.21.15.203:6807","nonce":4196866481}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6804","nonce":4196866481},{"type":"v1","addr":"172.21.15.203:6805","nonce":4196866481}]},"public_addr":"172.21.15.203:6801/4196866481","cluster_addr":"172.21.15.203:6803/4196866481","heartbeat_back_addr":"172.21.15.203:6807/4196866481","heartbeat_front_addr":"172.21.15.203:6805/4196866481","state":["exists","up"]},{"osd":5,"uuid":"81811e4b-4d24-4f51-a82c-f1d700a2a00d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":221,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6816","nonce":3059290081},{"type":"v1","addr":"172.21.15.203:6817","nonce":3059290081}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6818","nonce":3059290081},{"type":"v1","addr":"172.21.15.203:6819","nonce":3059290081}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6822","nonce":3059290081},{"type":"v1","addr":"172.21.15.203:6823","nonce":3059290081}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6820","nonce":3059290081},{"type":"v1","addr":"172.21.15.203:6821","nonce":3059290081}]},"public_addr":"172.21.15.203:6817/3059290081","cluster_addr":"172.21.15.203:6819/3059290081","heartbeat_back_addr":"172.21.15.203:6823/3059290081","heartbeat_front_addr":"172.21.15.203:6821/3059290081","state":["exists","up"]},{"osd":6,"uuid":"307bac4d-6a5b-443d-af75-56d695d7674f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":221,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6824","nonce":512155569},{"type":"v1","addr":"172.21.15.203:6825","nonce":512155569}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6826","nonce":512155569},{"type":"v1","addr":"172.21.15.203:6827","nonce":512155569}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6830","nonce":512155569},{"type":"v1","addr":"172.21.15.203:6831","nonce":512155569}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6828","nonce":512155569},{"type":"v1","addr":"172.21.15.203:6829","nonce":512155569}]},"public_addr":"172.21.15.203:6825/512155569","cluster_addr":"172.21.15.203:6827/512155569","heartbeat_back_addr":"172.21.15.203:6831/512155569","heartbeat_front_addr":"172.21.15.203:6829/512155569","state":["exists","up"]},{"osd":7,"uuid":"a95dc7ca-9f1b-4295-af70-68a22a2d7d5e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":221,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6808","nonce":3331450438},{"type":"v1","addr":"172.21.15.203:6809","nonce":3331450438}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6810","nonce":3331450438},{"type":"v1","addr":"172.21.15.203:6811","nonce":3331450438}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6814","nonce":3331450438},{"type":"v1","addr":"172.21.15.203:6815","nonce":3331450438}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6812","nonce":3331450438},{"type":"v1","addr":"172.21.15.203:6813","nonce":3331450438}]},"public_addr":"172.21.15.203:6809/3331450438","cluster_addr":"172.21.15.203:6811/3331450438","heartbeat_back_addr":"172.21.15.203:6815/3331450438","heartbeat_front_addr":"172.21.15.203:6813/3331450438","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-09T13:33:50.547652+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-09T13:33:50.589469+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-09T13:33:50.561724+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-09T13:33:50.599650+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-09T13:33:50.411855+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-09T13:33:50.219681+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-09T13:33:50.504664+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-09T13:33:50.441248+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.203:6832/4034093268":"2023-08-10T15:11:43.130145+0000","172.21.15.203:6833/4034093268":"2023-08-10T15:11:43.130145+0000","172.21.15.203:0/1654193324":"2023-08-10T15:11:43.130145+0000","172.21.15.203:0/4140323243":"2023-08-10T15:11:43.130145+0000","172.21.15.203:0/450863258":"2023-08-10T15:11:43.130145+0000","172.21.15.203:0/134877587":"2023-08-10T15:11:43.130145+0000","172.21.15.203:0/2330869600":"2023-08-10T15:11:43.130145+0000","172.21.15.203:0/2203818837":"2023-08-10T15:11:43.130145+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":7,"snaps":[{"begin":104,"length":1},{"begin":221,"length":1}]}],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-08-09T15:11:48.779 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:48.777+0000 7fd6c2722640 1 -- 172.21.15.43:0/1979070486 >> [v2:172.21.15.203:6832/2375771024,v1:172.21.15.203:6833/2375771024] conn(0x7fd69c06a250 msgr2=0x7fd69c06c710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T15:11:48.779 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:48.777+0000 7fd6c2722640 1 --2- 172.21.15.43:0/1979070486 >> [v2:172.21.15.203:6832/2375771024,v1:172.21.15.203:6833/2375771024] conn(0x7fd69c06a250 0x7fd69c06c710 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0x7fd6ac006fd0 tx=0x7fd6ac008040 comp rx=0 tx=0).stop 2023-08-09T15:11:48.779 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:48.777+0000 7fd6c2722640 1 -- 172.21.15.43:0/1979070486 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fd6bc15fe60 msgr2=0x7fd6bc15aa50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T15:11:48.779 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:48.777+0000 7fd6c2722640 1 --2- 172.21.15.43:0/1979070486 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fd6bc15fe60 0x7fd6bc15aa50 secure :-1 s=READY pgs=2464 cs=0 l=1 rev1=1 crypto rx=0x7fd6b4036040 tx=0x7fd6b4004a40 comp rx=0 tx=0).stop 2023-08-09T15:11:48.779 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:48.777+0000 7fd6c2722640 1 -- 172.21.15.43:0/1979070486 shutdown_connections 2023-08-09T15:11:48.779 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:48.777+0000 7fd6c2722640 1 --2- 172.21.15.43:0/1979070486 >> [v2:172.21.15.203:6832/2375771024,v1:172.21.15.203:6833/2375771024] conn(0x7fd69c06a250 0x7fd69c06c710 unknown :-1 s=CLOSED pgs=31 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T15:11:48.779 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:48.777+0000 7fd6c2722640 1 --2- 172.21.15.43:0/1979070486 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fd6bc15fe60 0x7fd6bc15aa50 unknown :-1 s=CLOSED pgs=2464 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T15:11:48.780 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:48.777+0000 7fd6c2722640 1 --2- 172.21.15.43:0/1979070486 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fd6bc0d14c0 0x7fd6bc0c9c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T15:11:48.780 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:48.777+0000 7fd6c2722640 1 --2- 172.21.15.43:0/1979070486 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fd6bc0cf510 0x7fd6bc0c9730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T15:11:48.780 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:48.777+0000 7fd6c2722640 1 -- 172.21.15.43:0/1979070486 >> 172.21.15.43:0/1979070486 conn(0x7fd6bc0d2660 msgr2=0x7fd6bc0b28a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-09T15:11:48.780 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:48.778+0000 7fd6c2722640 1 -- 172.21.15.43:0/1979070486 shutdown_connections 2023-08-09T15:11:48.780 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:48.778+0000 7fd6c2722640 1 -- 172.21.15.43:0/1979070486 wait complete. 2023-08-09T15:11:48.799 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': 'rbd', 'create_time': '2023-08-09T13:33:54.365137+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}, 'read_balance': {'score_acting': 2, 'score_stable': 2, 'optimal_score': 1, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': '.mgr', 'create_time': '2023-08-09T13:33:54.831803+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 26, 'pool_name': 'cephfs_metadata', 'create_time': '2023-08-09T15:11:26.544757+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '222', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.75, 'score_stable': 1.75, 'optimal_score': 1, 'raw_score_acting': 1.75, 'raw_score_stable': 1.75, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 27, 'pool_name': 'cephfs_data', 'create_time': '2023-08-09T15:11:27.194686+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '223', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.6299999952316284, 'score_stable': 1.6299999952316284, 'optimal_score': 1, 'raw_score_acting': 1.6299999952316284, 'raw_score_stable': 1.6299999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-08-09T15:11:48.800 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get rbd pg_num 2023-08-09T15:11:49.020 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:49.019+0000 7f9df5f4e640 1 Processor -- start 2023-08-09T15:11:49.020 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:49.019+0000 7f9df5f4e640 1 -- start start 2023-08-09T15:11:49.022 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:49.020+0000 7f9df5f4e640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f9df015c880 0x7f9df00ccbc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T15:11:49.022 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:49.021+0000 7f9df5f4e640 1 -- --> v1:172.21.15.43:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9df015cc60 con 0x7f9df00cf390 2023-08-09T15:11:49.022 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:49.021+0000 7f9df5f4e640 1 -- --> v1:172.21.15.203:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9df00d0530 con 0x7f9df0160f20 2023-08-09T15:11:49.022 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:49.021+0000 7f9df5f4e640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9df00cd410 con 0x7f9df015c880 2023-08-09T15:11:49.023 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:49.021+0000 7f9deffff640 1 --1- >> v1:172.21.15.43:6789/0 conn(0x7f9df00cf390 0x7f9df00d0a10 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.43:6789/0 says I am v1:172.21.15.43:56318/0 (socket says 172.21.15.43:56318) 2023-08-09T15:11:49.023 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:49.021+0000 7f9deffff640 1 -- 172.21.15.43:0/2744079476 learned_addr learned my addr 172.21.15.43:0/2744079476 (peer_addr_for_me v1:172.21.15.43:0/0) 2023-08-09T15:11:49.024 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:49.021+0000 7f9def7fe640 1 --2- 172.21.15.43:0/2744079476 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f9df015c880 0x7f9df00ccbc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T15:11:49.024 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:49.021+0000 7f9dee7fc640 1 -- 172.21.15.43:0/2744079476 <== mon.0 v1:172.21.15.43:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1933050372 0 0) 0x7f9df015cc60 con 0x7f9df00cf390 2023-08-09T15:11:49.024 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:49.021+0000 7f9dee7fc640 1 -- 172.21.15.43:0/2744079476 --> v1:172.21.15.43:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f9dd8003520 con 0x7f9df00cf390 2023-08-09T15:11:49.025 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:49.022+0000 7f9dee7fc640 1 -- 172.21.15.43:0/2744079476 <== mon.0 v1:172.21.15.43:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3351610248 0 0) 0x7f9dd8003520 con 0x7f9df00cf390 2023-08-09T15:11:49.025 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:49.022+0000 7f9dee7fc640 1 -- 172.21.15.43:0/2744079476 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f9df015c880 msgr2=0x7f9df00ccbc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-09T15:11:49.025 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:49.022+0000 7f9dee7fc640 1 --2- 172.21.15.43:0/2744079476 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f9df015c880 0x7f9df00ccbc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T15:11:49.025 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:49.022+0000 7f9dee7fc640 1 -- 172.21.15.43:0/2744079476 >> v1:172.21.15.203:6789/0 conn(0x7f9df0160f20 legacy=0x7f9df00cec80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T15:11:49.025 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:49.022+0000 7f9dee7fc640 1 -- 172.21.15.43:0/2744079476 --> v1:172.21.15.43:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9df00cd6f0 con 0x7f9df00cf390 2023-08-09T15:11:49.025 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:49.022+0000 7f9dee7fc640 1 -- 172.21.15.43:0/2744079476 <== mon.0 v1:172.21.15.43:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1285150740 0 0) 0x7f9de8001670 con 0x7f9df00cf390 2023-08-09T15:11:49.025 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:49.022+0000 7f9dee7fc640 1 -- 172.21.15.43:0/2744079476 >> v1:172.21.15.43:6789/0 conn(0x7f9df00cf390 legacy=0x7f9df00d0a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T15:11:49.026 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:49.022+0000 7f9dee7fc640 1 --2- 172.21.15.43:0/2744079476 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f9df015c880 0x7f9dd80039a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T15:11:49.026 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:49.022+0000 7f9dee7fc640 1 --2- 172.21.15.43:0/2744079476 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f9dd8003fd0 0x7f9dd80047c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T15:11:49.026 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:49.022+0000 7f9dee7fc640 1 --2- 172.21.15.43:0/2744079476 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f9dd8004d00 0x7f9dd8005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T15:11:49.026 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:49.022+0000 7f9dee7fc640 1 -- 172.21.15.43:0/2744079476 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9df00cd410 con 0x7f9dd8004d00 2023-08-09T15:11:49.026 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:49.022+0000 7f9dee7fc640 1 -- 172.21.15.43:0/2744079476 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9dd800bb40 con 0x7f9dd8003fd0 2023-08-09T15:11:49.026 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:49.022+0000 7f9dee7fc640 1 -- 172.21.15.43:0/2744079476 --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9dd800bd00 con 0x7f9df015c880 2023-08-09T15:11:49.026 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:49.022+0000 7f9deffff640 1 --2- 172.21.15.43:0/2744079476 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f9dd8004d00 0x7f9dd8005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T15:11:49.026 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:49.023+0000 7f9deeffd640 1 --2- 172.21.15.43:0/2744079476 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f9dd8003fd0 0x7f9dd80047c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T15:11:49.027 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:49.023+0000 7f9deffff640 1 -- 172.21.15.43:0/2744079476 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f9df015c880 msgr2=0x7f9dd80039a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-09T15:11:49.027 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:49.023+0000 7f9deffff640 1 --2- 172.21.15.43:0/2744079476 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f9df015c880 0x7f9dd80039a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T15:11:49.027 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:49.023+0000 7f9deffff640 1 -- 172.21.15.43:0/2744079476 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f9dd8003fd0 msgr2=0x7f9dd80047c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-09T15:11:49.027 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:49.023+0000 7f9deffff640 1 --2- 172.21.15.43:0/2744079476 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f9dd8003fd0 0x7f9dd80047c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T15:11:49.027 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:49.023+0000 7f9deffff640 1 -- 172.21.15.43:0/2744079476 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9dd8003520 con 0x7f9dd8004d00 2023-08-09T15:11:49.028 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:49.024+0000 7f9deffff640 1 --2- 172.21.15.43:0/2744079476 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f9dd8004d00 0x7f9dd8005180 secure :-1 s=READY pgs=2466 cs=0 l=1 rev1=1 crypto rx=0x7f9de800c240 tx=0x7f9dd80057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a7bba77482f1be6b server_cookie=0 in_seq=0 out_seq=0 2023-08-09T15:11:49.028 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:49.024+0000 7f9dee7fc640 1 -- 172.21.15.43:0/2744079476 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9de8005ad0 con 0x7f9dd8004d00 2023-08-09T15:11:49.028 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:49.024+0000 7f9dee7fc640 1 -- 172.21.15.43:0/2744079476 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9de8005c30 con 0x7f9dd8004d00 2023-08-09T15:11:49.029 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:49.024+0000 7f9dee7fc640 1 -- 172.21.15.43:0/2744079476 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9de802caf0 con 0x7f9dd8004d00 2023-08-09T15:11:49.029 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:49.025+0000 7f9df5f4e640 1 -- 172.21.15.43:0/2744079476 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f9dd8004d00 msgr2=0x7f9dd8005180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T15:11:49.029 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:49.025+0000 7f9df5f4e640 1 --2- 172.21.15.43:0/2744079476 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f9dd8004d00 0x7f9dd8005180 secure :-1 s=READY pgs=2466 cs=0 l=1 rev1=1 crypto rx=0x7f9de800c240 tx=0x7f9dd80057f0 comp rx=0 tx=0).stop 2023-08-09T15:11:49.029 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:49.025+0000 7f9deffff640 1 -- 172.21.15.43:0/2744079476 reap_dead start 2023-08-09T15:11:49.029 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:49.025+0000 7f9df5f4e640 1 -- 172.21.15.43:0/2744079476 shutdown_connections 2023-08-09T15:11:49.029 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:49.025+0000 7f9df5f4e640 1 -- 172.21.15.43:0/2744079476 >> 172.21.15.43:0/2744079476 conn(0x7f9df008bfb0 msgr2=0x7f9df009b8d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-09T15:11:49.029 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:49.025+0000 7f9df5f4e640 1 -- 172.21.15.43:0/2744079476 shutdown_connections 2023-08-09T15:11:49.030 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:49.025+0000 7f9df5f4e640 1 -- 172.21.15.43:0/2744079476 wait complete. 2023-08-09T15:11:49.030 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:49.027+0000 7f9df5f4e640 1 Processor -- start 2023-08-09T15:11:49.030 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:49.027+0000 7f9df5f4e640 1 -- start start 2023-08-09T15:11:49.031 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:49.028+0000 7f9df5f4e640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f9dd8004d00 0x7f9df015d010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T15:11:49.031 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:49.028+0000 7f9df5f4e640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f9df01505a0 0x7f9df00b4290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T15:11:49.031 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:49.028+0000 7f9df5f4e640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f9df00b47d0 0x7f9df00ca690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T15:11:49.032 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:49.028+0000 7f9df5f4e640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9df00b4c50 con 0x7f9df00b47d0 2023-08-09T15:11:49.032 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:49.028+0000 7f9df5f4e640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9df00b2ad0 con 0x7f9df01505a0 2023-08-09T15:11:49.032 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:49.028+0000 7f9df5f4e640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9df00b3650 con 0x7f9dd8004d00 2023-08-09T15:11:49.032 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:49.028+0000 7f9deffff640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f9df00b47d0 0x7f9df00ca690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T15:11:49.032 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:49.028+0000 7f9deffff640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f9df00b47d0 0x7f9df00ca690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:50204/0 (socket says 172.21.15.43:50204) 2023-08-09T15:11:49.032 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:49.028+0000 7f9deffff640 1 -- 172.21.15.43:0/3225676770 learned_addr learned my addr 172.21.15.43:0/3225676770 (peer_addr_for_me v2:172.21.15.43:0/0) 2023-08-09T15:11:49.032 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:49.029+0000 7f9def7fe640 1 --2- 172.21.15.43:0/3225676770 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f9dd8004d00 0x7f9df015d010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T15:11:49.033 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:49.029+0000 7f9deeffd640 1 --2- 172.21.15.43:0/3225676770 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f9df01505a0 0x7f9df00b4290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T15:11:49.033 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:49.029+0000 7f9deffff640 1 -- 172.21.15.43:0/3225676770 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f9dd8004d00 msgr2=0x7f9df015d010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T15:11:49.033 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:49.029+0000 7f9deffff640 1 --2- 172.21.15.43:0/3225676770 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f9dd8004d00 0x7f9df015d010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T15:11:49.033 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:49.029+0000 7f9deffff640 1 -- 172.21.15.43:0/3225676770 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f9df01505a0 msgr2=0x7f9df00b4290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T15:11:49.033 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:49.029+0000 7f9deffff640 1 --2- 172.21.15.43:0/3225676770 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f9df01505a0 0x7f9df00b4290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T15:11:49.033 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:49.029+0000 7f9deffff640 1 -- 172.21.15.43:0/3225676770 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9df01f1c70 con 0x7f9df00b47d0 2023-08-09T15:11:49.033 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:49.029+0000 7f9deffff640 1 --2- 172.21.15.43:0/3225676770 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f9df00b47d0 0x7f9df00ca690 secure :-1 s=READY pgs=2467 cs=0 l=1 rev1=1 crypto rx=0x7f9de800c360 tx=0x7f9de802e070 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-09T15:11:49.033 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:49.029+0000 7f9dd7fff640 1 -- 172.21.15.43:0/3225676770 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9de803c040 con 0x7f9df00b47d0 2023-08-09T15:11:49.034 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:49.029+0000 7f9dd7fff640 1 -- 172.21.15.43:0/3225676770 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9de802ccc0 con 0x7f9df00b47d0 2023-08-09T15:11:49.034 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:49.029+0000 7f9df5f4e640 1 -- 172.21.15.43:0/3225676770 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9df00b2d20 con 0x7f9df00b47d0 2023-08-09T15:11:49.034 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:49.029+0000 7f9dd7fff640 1 -- 172.21.15.43:0/3225676770 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9de8007270 con 0x7f9df00b47d0 2023-08-09T15:11:49.034 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:49.030+0000 7f9df5f4e640 1 -- 172.21.15.43:0/3225676770 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9df00b32e0 con 0x7f9df00b47d0 2023-08-09T15:11:49.035 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:49.031+0000 7f9df5f4e640 1 -- 172.21.15.43:0/3225676770 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9df004e390 con 0x7f9df00b47d0 2023-08-09T15:11:49.035 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:49.032+0000 7f9dd7fff640 1 -- 172.21.15.43:0/3225676770 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 123) v1 ==== 88517+0+0 (secure 0 0 0) 0x7f9de8032070 con 0x7f9df00b47d0 2023-08-09T15:11:49.035 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:49.033+0000 7f9dd7fff640 1 --2- 172.21.15.43:0/3225676770 >> [v2:172.21.15.203:6832/2375771024,v1:172.21.15.203:6833/2375771024] conn(0x7f9ddc06a250 0x7f9ddc06c710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T15:11:49.035 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:49.033+0000 7f9dd7fff640 1 -- 172.21.15.43:0/3225676770 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(224..224 src has 1..224) v4 ==== 6770+0+0 (secure 0 0 0) 0x7f9de80a9c40 con 0x7f9df00b47d0 2023-08-09T15:11:49.035 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:49.033+0000 7f9def7fe640 1 --2- 172.21.15.43:0/3225676770 >> [v2:172.21.15.203:6832/2375771024,v1:172.21.15.203:6833/2375771024] conn(0x7f9ddc06a250 0x7f9ddc06c710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T15:11:49.036 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:49.034+0000 7f9def7fe640 1 --2- 172.21.15.43:0/3225676770 >> [v2:172.21.15.203:6832/2375771024,v1:172.21.15.203:6833/2375771024] conn(0x7f9ddc06a250 0x7f9ddc06c710 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0x7f9de00070b0 tx=0x7f9de000b040 comp rx=0 tx=0).ready entity=mgr.9103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-09T15:11:49.038 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:49.037+0000 7f9dd7fff640 1 -- 172.21.15.43:0/3225676770 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178066 (secure 0 0 0) 0x7f9de8074d40 con 0x7f9df00b47d0 2023-08-09T15:11:49.232 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:49.231+0000 7f9df5f4e640 1 -- 172.21.15.43:0/3225676770 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"} v 0) v1 -- 0x7f9df00d1eb0 con 0x7f9df00b47d0 2023-08-09T15:11:49.233 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:49.232+0000 7f9dd7fff640 1 -- 172.21.15.43:0/3225676770 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"}]=0 v224) v1 ==== 92+0+10 (secure 0 0 0) 0x7f9de8078360 con 0x7f9df00b47d0 2023-08-09T15:11:49.233 INFO:teuthology.orchestra.run.smithi043.stdout:pg_num: 8 2023-08-09T15:11:49.235 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:49.234+0000 7f9df5f4e640 1 -- 172.21.15.43:0/3225676770 >> [v2:172.21.15.203:6832/2375771024,v1:172.21.15.203:6833/2375771024] conn(0x7f9ddc06a250 msgr2=0x7f9ddc06c710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T15:11:49.235 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:49.234+0000 7f9df5f4e640 1 --2- 172.21.15.43:0/3225676770 >> [v2:172.21.15.203:6832/2375771024,v1:172.21.15.203:6833/2375771024] conn(0x7f9ddc06a250 0x7f9ddc06c710 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0x7f9de00070b0 tx=0x7f9de000b040 comp rx=0 tx=0).stop 2023-08-09T15:11:49.236 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:49.234+0000 7f9df5f4e640 1 -- 172.21.15.43:0/3225676770 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f9df00b47d0 msgr2=0x7f9df00ca690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T15:11:49.236 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:49.234+0000 7f9df5f4e640 1 --2- 172.21.15.43:0/3225676770 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f9df00b47d0 0x7f9df00ca690 secure :-1 s=READY pgs=2467 cs=0 l=1 rev1=1 crypto rx=0x7f9de800c360 tx=0x7f9de802e070 comp rx=0 tx=0).stop 2023-08-09T15:11:49.236 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:49.234+0000 7f9df5f4e640 1 -- 172.21.15.43:0/3225676770 shutdown_connections 2023-08-09T15:11:49.236 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:49.234+0000 7f9df5f4e640 1 --2- 172.21.15.43:0/3225676770 >> [v2:172.21.15.203:6832/2375771024,v1:172.21.15.203:6833/2375771024] conn(0x7f9ddc06a250 0x7f9ddc06c710 unknown :-1 s=CLOSED pgs=32 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T15:11:49.237 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:49.234+0000 7f9df5f4e640 1 --2- 172.21.15.43:0/3225676770 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f9df00b47d0 0x7f9df00ca690 unknown :-1 s=CLOSED pgs=2467 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T15:11:49.237 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:49.234+0000 7f9df5f4e640 1 --2- 172.21.15.43:0/3225676770 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f9df01505a0 0x7f9df00b4290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T15:11:49.237 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:49.234+0000 7f9df5f4e640 1 --2- 172.21.15.43:0/3225676770 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f9dd8004d00 0x7f9df015d010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T15:11:49.237 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:49.234+0000 7f9df5f4e640 1 -- 172.21.15.43:0/3225676770 >> 172.21.15.43:0/3225676770 conn(0x7f9df008bfb0 msgr2=0x7f9df015a420 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-09T15:11:49.237 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:49.234+0000 7f9df5f4e640 1 -- 172.21.15.43:0/3225676770 shutdown_connections 2023-08-09T15:11:49.237 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:49.235+0000 7f9df5f4e640 1 -- 172.21.15.43:0/3225676770 wait complete. 2023-08-09T15:11:49.255 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-08-09T15:11:49.488 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:49.487+0000 7f89b192f640 1 Processor -- start 2023-08-09T15:11:49.488 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:49.487+0000 7f89b192f640 1 -- start start 2023-08-09T15:11:49.489 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:49.488+0000 7f89b192f640 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7f89ac15c880 0x7f89ac0ccbc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T15:11:49.489 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:49.488+0000 7f89b192f640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f89ac160f20 0x7f89ac0cec80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T15:11:49.490 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:49.488+0000 7f89b192f640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f89ac0cf1c0 0x7f89ac0b3ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T15:11:49.490 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:49.488+0000 7f89b192f640 1 -- --> v2:172.21.15.43:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f89ac15cc60 con 0x7f89ac0cf1c0 2023-08-09T15:11:49.490 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:49.488+0000 7f89b192f640 1 -- --> v2:172.21.15.203:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f89ac0d0530 con 0x7f89ac15c880 2023-08-09T15:11:49.490 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:49.488+0000 7f89b192f640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f89ac0cf620 con 0x7f89ac160f20 2023-08-09T15:11:49.491 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:49.488+0000 7f89ab7fe640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f89ac0cf1c0 0x7f89ac0b3ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T15:11:49.491 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:49.489+0000 7f89ab7fe640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f89ac0cf1c0 0x7f89ac0b3ff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:50218/0 (socket says 172.21.15.43:50218) 2023-08-09T15:11:49.491 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:49.489+0000 7f89ab7fe640 1 -- 172.21.15.43:0/501498 learned_addr learned my addr 172.21.15.43:0/501498 (peer_addr_for_me v2:172.21.15.43:0/0) 2023-08-09T15:11:49.491 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:49.489+0000 7f89ab7fe640 1 -- 172.21.15.43:0/501498 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f89ac160f20 msgr2=0x7f89ac0cec80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-09T15:11:49.491 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:49.489+0000 7f89ab7fe640 1 --2- 172.21.15.43:0/501498 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f89ac160f20 0x7f89ac0cec80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T15:11:49.491 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:49.489+0000 7f89ab7fe640 1 -- 172.21.15.43:0/501498 >> v2:172.21.15.203:3300/0 conn(0x7f89ac15c880 msgr2=0x7f89ac0ccbc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-09T15:11:49.492 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:49.489+0000 7f89ab7fe640 1 --2- 172.21.15.43:0/501498 >> v2:172.21.15.203:3300/0 conn(0x7f89ac15c880 0x7f89ac0ccbc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T15:11:49.492 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:49.489+0000 7f89ab7fe640 1 -- 172.21.15.43:0/501498 --> v2:172.21.15.43:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f89ac0cd410 con 0x7f89ac0cf1c0 2023-08-09T15:11:49.493 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:49.490+0000 7f89ab7fe640 1 --2- 172.21.15.43:0/501498 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f89ac0cf1c0 0x7f89ac0b3ff0 secure :-1 s=READY pgs=2468 cs=0 l=1 rev1=1 crypto rx=0x7f89ac0b5100 tx=0x7f89a402b920 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cb0049eed3edcb28 server_cookie=0 in_seq=0 out_seq=0 2023-08-09T15:11:49.493 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:49.491+0000 7f89a9ffb640 1 -- 172.21.15.43:0/501498 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f89a4031880 con 0x7f89ac0cf1c0 2023-08-09T15:11:49.493 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:49.491+0000 7f89a9ffb640 1 -- 172.21.15.43:0/501498 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f89a40319e0 con 0x7f89ac0cf1c0 2023-08-09T15:11:49.493 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:49.491+0000 7f89a9ffb640 1 -- 172.21.15.43:0/501498 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f89a4031d70 con 0x7f89ac0cf1c0 2023-08-09T15:11:49.493 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:49.491+0000 7f89b192f640 1 -- 172.21.15.43:0/501498 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f89ac0cf1c0 msgr2=0x7f89ac0b3ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T15:11:49.493 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:49.491+0000 7f89b192f640 1 --2- 172.21.15.43:0/501498 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f89ac0cf1c0 0x7f89ac0b3ff0 secure :-1 s=READY pgs=2468 cs=0 l=1 rev1=1 crypto rx=0x7f89ac0b5100 tx=0x7f89a402b920 comp rx=0 tx=0).stop 2023-08-09T15:11:49.494 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:49.491+0000 7f89b192f640 1 -- 172.21.15.43:0/501498 shutdown_connections 2023-08-09T15:11:49.494 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:49.491+0000 7f89b192f640 1 --2- 172.21.15.43:0/501498 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f89ac0cf1c0 0x7f89ac0b3ff0 unknown :-1 s=CLOSED pgs=2468 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T15:11:49.494 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:49.491+0000 7f89b192f640 1 --2- 172.21.15.43:0/501498 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f89ac160f20 0x7f89ac0cec80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T15:11:49.494 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:49.491+0000 7f89b192f640 1 --2- 172.21.15.43:0/501498 >> v2:172.21.15.203:3300/0 conn(0x7f89ac15c880 0x7f89ac0ccbc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T15:11:49.494 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:49.491+0000 7f89b192f640 1 -- 172.21.15.43:0/501498 >> 172.21.15.43:0/501498 conn(0x7f89ac08bfb0 msgr2=0x7f89ac09b8d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-09T15:11:49.494 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:49.491+0000 7f89b192f640 1 -- 172.21.15.43:0/501498 shutdown_connections 2023-08-09T15:11:49.494 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:49.491+0000 7f89b192f640 1 -- 172.21.15.43:0/501498 wait complete. 2023-08-09T15:11:49.495 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:49.493+0000 7f89b192f640 1 Processor -- start 2023-08-09T15:11:49.495 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:49.493+0000 7f89b192f640 1 -- start start 2023-08-09T15:11:49.496 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:49.494+0000 7f89b192f640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f89ac0cf1c0 0x7f89ac1571e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T15:11:49.496 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:49.494+0000 7f89b192f640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f89ac15c880 0x7f89ac15f820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T15:11:49.496 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:49.494+0000 7f89b192f640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f89ac160f20 0x7f89ac156ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T15:11:49.496 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:49.494+0000 7f89b192f640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f89ac0c63f0 con 0x7f89ac15c880 2023-08-09T15:11:49.496 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:49.494+0000 7f89b192f640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f89ac0c60a0 con 0x7f89ac160f20 2023-08-09T15:11:49.496 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:49.494+0000 7f89b192f640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f89ac157000 con 0x7f89ac0cf1c0 2023-08-09T15:11:49.497 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:49.495+0000 7f89aa7fc640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f89ac15c880 0x7f89ac15f820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T15:11:49.497 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:49.495+0000 7f89aa7fc640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f89ac15c880 0x7f89ac15f820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:50230/0 (socket says 172.21.15.43:50230) 2023-08-09T15:11:49.497 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:49.495+0000 7f89aa7fc640 1 -- 172.21.15.43:0/2151385516 learned_addr learned my addr 172.21.15.43:0/2151385516 (peer_addr_for_me v2:172.21.15.43:0/0) 2023-08-09T15:11:49.498 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:49.495+0000 7f89aaffd640 1 --2- 172.21.15.43:0/2151385516 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f89ac0cf1c0 0x7f89ac1571e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T15:11:49.498 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:49.495+0000 7f89ab7fe640 1 --2- 172.21.15.43:0/2151385516 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f89ac160f20 0x7f89ac156ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T15:11:49.498 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:49.495+0000 7f89aa7fc640 1 -- 172.21.15.43:0/2151385516 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f89ac0cf1c0 msgr2=0x7f89ac1571e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T15:11:49.498 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:49.495+0000 7f89aa7fc640 1 --2- 172.21.15.43:0/2151385516 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f89ac0cf1c0 0x7f89ac1571e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T15:11:49.499 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:49.495+0000 7f89aa7fc640 1 -- 172.21.15.43:0/2151385516 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f89ac160f20 msgr2=0x7f89ac156ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T15:11:49.499 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:49.495+0000 7f89aa7fc640 1 --2- 172.21.15.43:0/2151385516 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f89ac160f20 0x7f89ac156ac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T15:11:49.499 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:49.495+0000 7f89aa7fc640 1 -- 172.21.15.43:0/2151385516 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f89a40094d0 con 0x7f89ac15c880 2023-08-09T15:11:49.499 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:49.495+0000 7f89ab7fe640 1 --2- 172.21.15.43:0/2151385516 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f89ac160f20 0x7f89ac156ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-08-09T15:11:49.499 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:49.495+0000 7f89aa7fc640 1 --2- 172.21.15.43:0/2151385516 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f89ac15c880 0x7f89ac15f820 secure :-1 s=READY pgs=2469 cs=0 l=1 rev1=1 crypto rx=0x7f89a000b4d0 tx=0x7f89a000b990 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-09T15:11:49.499 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:49.496+0000 7f8993fff640 1 -- 172.21.15.43:0/2151385516 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f89a0010870 con 0x7f89ac15c880 2023-08-09T15:11:49.499 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:49.496+0000 7f8993fff640 1 -- 172.21.15.43:0/2151385516 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f89a00109d0 con 0x7f89ac15c880 2023-08-09T15:11:49.499 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:49.496+0000 7f8993fff640 1 -- 172.21.15.43:0/2151385516 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f89a0010d80 con 0x7f89ac15c880 2023-08-09T15:11:49.500 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:49.496+0000 7f89b192f640 1 -- 172.21.15.43:0/2151385516 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f89ac15ea80 con 0x7f89ac15c880 2023-08-09T15:11:49.500 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:49.496+0000 7f89b192f640 1 -- 172.21.15.43:0/2151385516 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f89ac0ca280 con 0x7f89ac15c880 2023-08-09T15:11:49.500 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:49.497+0000 7f89b192f640 1 -- 172.21.15.43:0/2151385516 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f89ac0c6530 con 0x7f89ac15c880 2023-08-09T15:11:49.500 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:49.498+0000 7f8993fff640 1 -- 172.21.15.43:0/2151385516 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 123) v1 ==== 88517+0+0 (secure 0 0 0) 0x7f89a000f3c0 con 0x7f89ac15c880 2023-08-09T15:11:49.501 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:49.499+0000 7f8993fff640 1 --2- 172.21.15.43:0/2151385516 >> [v2:172.21.15.203:6832/2375771024,v1:172.21.15.203:6833/2375771024] conn(0x7f899806a250 0x7f899806c710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T15:11:49.501 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:49.499+0000 7f8993fff640 1 -- 172.21.15.43:0/2151385516 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(224..224 src has 1..224) v4 ==== 6770+0+0 (secure 0 0 0) 0x7f89a0088b80 con 0x7f89ac15c880 2023-08-09T15:11:49.501 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:49.500+0000 7f89aaffd640 1 --2- 172.21.15.43:0/2151385516 >> [v2:172.21.15.203:6832/2375771024,v1:172.21.15.203:6833/2375771024] conn(0x7f899806a250 0x7f899806c710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T15:11:49.502 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:49.501+0000 7f89aaffd640 1 --2- 172.21.15.43:0/2151385516 >> [v2:172.21.15.203:6832/2375771024,v1:172.21.15.203:6833/2375771024] conn(0x7f899806a250 0x7f899806c710 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0x7f899c009f20 tx=0x7f899c0094f0 comp rx=0 tx=0).ready entity=mgr.9103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-09T15:11:49.505 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:49.503+0000 7f8993fff640 1 -- 172.21.15.43:0/2151385516 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178066 (secure 0 0 0) 0x7f89ac0c6530 con 0x7f89ac15c880 2023-08-09T15:11:49.700 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:49.698+0000 7f89b192f640 1 -- 172.21.15.43:0/2151385516 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f89ac0c6530 con 0x7f89ac15c880 2023-08-09T15:11:49.701 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:49.699+0000 7f8993fff640 1 -- 172.21.15.43:0/2151385516 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v224) v1 ==== 93+0+10 (secure 0 0 0) 0x7f89ac0c6530 con 0x7f89ac15c880 2023-08-09T15:11:49.701 INFO:teuthology.orchestra.run.smithi043.stdout:pg_num: 1 2023-08-09T15:11:49.703 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:49.702+0000 7f89b192f640 1 -- 172.21.15.43:0/2151385516 >> [v2:172.21.15.203:6832/2375771024,v1:172.21.15.203:6833/2375771024] conn(0x7f899806a250 msgr2=0x7f899806c710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T15:11:49.703 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:49.702+0000 7f89b192f640 1 --2- 172.21.15.43:0/2151385516 >> [v2:172.21.15.203:6832/2375771024,v1:172.21.15.203:6833/2375771024] conn(0x7f899806a250 0x7f899806c710 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0x7f899c009f20 tx=0x7f899c0094f0 comp rx=0 tx=0).stop 2023-08-09T15:11:49.703 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:49.702+0000 7f89b192f640 1 -- 172.21.15.43:0/2151385516 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f89ac15c880 msgr2=0x7f89ac15f820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T15:11:49.703 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:49.702+0000 7f89b192f640 1 --2- 172.21.15.43:0/2151385516 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f89ac15c880 0x7f89ac15f820 secure :-1 s=READY pgs=2469 cs=0 l=1 rev1=1 crypto rx=0x7f89a000b4d0 tx=0x7f89a000b990 comp rx=0 tx=0).stop 2023-08-09T15:11:49.704 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:49.702+0000 7f89b192f640 1 -- 172.21.15.43:0/2151385516 shutdown_connections 2023-08-09T15:11:49.704 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:49.702+0000 7f89b192f640 1 --2- 172.21.15.43:0/2151385516 >> [v2:172.21.15.203:6832/2375771024,v1:172.21.15.203:6833/2375771024] conn(0x7f899806a250 0x7f899806c710 unknown :-1 s=CLOSED pgs=33 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T15:11:49.704 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:49.702+0000 7f89b192f640 1 --2- 172.21.15.43:0/2151385516 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f89ac160f20 0x7f89ac156ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T15:11:49.705 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:49.702+0000 7f89b192f640 1 --2- 172.21.15.43:0/2151385516 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f89ac15c880 0x7f89ac15f820 unknown :-1 s=CLOSED pgs=2469 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T15:11:49.705 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:49.702+0000 7f89b192f640 1 --2- 172.21.15.43:0/2151385516 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f89ac0cf1c0 0x7f89ac1571e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T15:11:49.705 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:49.702+0000 7f89b192f640 1 -- 172.21.15.43:0/2151385516 >> 172.21.15.43:0/2151385516 conn(0x7f89ac08bfb0 msgr2=0x7f89ac0afcf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-09T15:11:49.705 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:49.702+0000 7f89b192f640 1 -- 172.21.15.43:0/2151385516 shutdown_connections 2023-08-09T15:11:49.706 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:49.702+0000 7f89b192f640 1 -- 172.21.15.43:0/2151385516 wait complete. 2023-08-09T15:11:49.724 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-08-09T15:11:49.944 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:49.943+0000 7f1b1ef86640 1 Processor -- start 2023-08-09T15:11:49.944 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:49.943+0000 7f1b1ef86640 1 -- start start 2023-08-09T15:11:49.946 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:49.944+0000 7f1b1ef86640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f1b180cb130 0x7f1b180700a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T15:11:49.946 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:49.944+0000 7f1b1ef86640 1 -- --> v1:172.21.15.43:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1b1815d8e0 con 0x7f1b18159110 2023-08-09T15:11:49.946 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:49.944+0000 7f1b1ef86640 1 -- --> v1:172.21.15.203:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1b1815dbc0 con 0x7f1b180ca310 2023-08-09T15:11:49.946 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:49.944+0000 7f1b1ef86640 1 -- --> v2:172.21.15.43:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1b180c9850 con 0x7f1b180cb130 2023-08-09T15:11:49.947 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:49.945+0000 7f1b1e785640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f1b180cb130 0x7f1b180700a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T15:11:49.947 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:49.945+0000 7f1b1e785640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f1b180cb130 0x7f1b180700a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:50234/0 (socket says 172.21.15.43:50234) 2023-08-09T15:11:49.947 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:49.945+0000 7f1b1e785640 1 -- 172.21.15.43:0/480263716 learned_addr learned my addr 172.21.15.43:0/480263716 (peer_addr_for_me v2:172.21.15.43:0/0) 2023-08-09T15:11:49.947 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:49.945+0000 7f1b1df84640 1 --1- 172.21.15.43:0/480263716 >> v1:172.21.15.43:6789/0 conn(0x7f1b18159110 0x7f1b1815cd70 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.43:6789/0 says I am v1:172.21.15.43:56332/0 (socket says 172.21.15.43:56332) 2023-08-09T15:11:49.948 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:49.945+0000 7f1b1e785640 1 -- 172.21.15.43:0/480263716 >> v1:172.21.15.203:6789/0 conn(0x7f1b180ca310 legacy=0x7f1b180caa20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-09T15:11:49.948 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:49.945+0000 7f1b1e785640 1 -- 172.21.15.43:0/480263716 >> v1:172.21.15.43:6789/0 conn(0x7f1b18159110 legacy=0x7f1b1815cd70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T15:11:49.948 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:49.945+0000 7f1b1e785640 1 -- 172.21.15.43:0/480263716 --> v2:172.21.15.43:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1b180c9b30 con 0x7f1b180cb130 2023-08-09T15:11:49.948 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:49.946+0000 7f1b1e785640 1 --2- 172.21.15.43:0/480263716 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f1b180cb130 0x7f1b180700a0 secure :-1 s=READY pgs=2471 cs=0 l=1 rev1=1 crypto rx=0x7f1b10009e50 tx=0x7f1b1002bf70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f8bdc44a8250ae51 server_cookie=0 in_seq=0 out_seq=0 2023-08-09T15:11:49.949 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:49.947+0000 7f1b1cf82640 1 -- 172.21.15.43:0/480263716 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1b1002cb10 con 0x7f1b180cb130 2023-08-09T15:11:49.949 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:49.947+0000 7f1b1cf82640 1 -- 172.21.15.43:0/480263716 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1b1002cc70 con 0x7f1b180cb130 2023-08-09T15:11:49.949 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:49.947+0000 7f1b1cf82640 1 -- 172.21.15.43:0/480263716 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1b100343f0 con 0x7f1b180cb130 2023-08-09T15:11:49.949 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:49.947+0000 7f1b1ef86640 1 -- 172.21.15.43:0/480263716 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f1b180cb130 msgr2=0x7f1b180700a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T15:11:49.950 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:49.947+0000 7f1b1ef86640 1 --2- 172.21.15.43:0/480263716 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f1b180cb130 0x7f1b180700a0 secure :-1 s=READY pgs=2471 cs=0 l=1 rev1=1 crypto rx=0x7f1b10009e50 tx=0x7f1b1002bf70 comp rx=0 tx=0).stop 2023-08-09T15:11:49.950 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:49.947+0000 7f1b1ef86640 1 -- 172.21.15.43:0/480263716 shutdown_connections 2023-08-09T15:11:49.950 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:49.947+0000 7f1b1ef86640 1 --2- 172.21.15.43:0/480263716 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f1b180cb130 0x7f1b180700a0 unknown :-1 s=CLOSED pgs=2471 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T15:11:49.950 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:49.947+0000 7f1b1ef86640 1 -- 172.21.15.43:0/480263716 >> 172.21.15.43:0/480263716 conn(0x7f1b180afb60 msgr2=0x7f1b180aff40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-09T15:11:49.950 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:49.948+0000 7f1b1ef86640 1 -- 172.21.15.43:0/480263716 shutdown_connections 2023-08-09T15:11:49.950 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:49.948+0000 7f1b1ef86640 1 -- 172.21.15.43:0/480263716 wait complete. 2023-08-09T15:11:49.951 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:49.949+0000 7f1b1ef86640 1 Processor -- start 2023-08-09T15:11:49.951 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:49.950+0000 7f1b1ef86640 1 -- start start 2023-08-09T15:11:49.952 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:49.951+0000 7f1b1ef86640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f1b180ca310 0x7f1b181540d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T15:11:49.952 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:49.951+0000 7f1b1ef86640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f1b180cb130 0x7f1b180c7580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T15:11:49.952 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:49.951+0000 7f1b1ef86640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f1b18159110 0x7f1b180c2410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T15:11:49.953 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:49.951+0000 7f1b1ef86640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1b181f3070 con 0x7f1b180cb130 2023-08-09T15:11:49.953 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:49.951+0000 7f1b1ef86640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1b18155bc0 con 0x7f1b180ca310 2023-08-09T15:11:49.953 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:49.951+0000 7f1b1ef86640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1b18155e60 con 0x7f1b18159110 2023-08-09T15:11:49.953 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:49.951+0000 7f1b1d783640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f1b180cb130 0x7f1b180c7580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T15:11:49.953 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:49.951+0000 7f1b1d783640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f1b180cb130 0x7f1b180c7580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:50248/0 (socket says 172.21.15.43:50248) 2023-08-09T15:11:49.954 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:49.951+0000 7f1b1d783640 1 -- 172.21.15.43:0/2766580473 learned_addr learned my addr 172.21.15.43:0/2766580473 (peer_addr_for_me v2:172.21.15.43:0/0) 2023-08-09T15:11:49.955 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:49.951+0000 7f1b1df84640 1 --2- 172.21.15.43:0/2766580473 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f1b180ca310 0x7f1b181540d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T15:11:49.955 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:49.951+0000 7f1b1e785640 1 --2- 172.21.15.43:0/2766580473 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f1b18159110 0x7f1b180c2410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T15:11:49.955 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:49.951+0000 7f1b1d783640 1 -- 172.21.15.43:0/2766580473 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f1b18159110 msgr2=0x7f1b180c2410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T15:11:49.955 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:49.951+0000 7f1b1d783640 1 --2- 172.21.15.43:0/2766580473 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f1b18159110 0x7f1b180c2410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T15:11:49.955 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:49.951+0000 7f1b1d783640 1 -- 172.21.15.43:0/2766580473 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f1b180ca310 msgr2=0x7f1b181540d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T15:11:49.955 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:49.951+0000 7f1b1d783640 1 --2- 172.21.15.43:0/2766580473 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f1b180ca310 0x7f1b181540d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T15:11:49.955 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:49.951+0000 7f1b1d783640 1 -- 172.21.15.43:0/2766580473 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1b10009b30 con 0x7f1b180cb130 2023-08-09T15:11:49.956 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:49.951+0000 7f1b1df84640 1 --2- 172.21.15.43:0/2766580473 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f1b180ca310 0x7f1b181540d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-08-09T15:11:49.956 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:49.952+0000 7f1b1d783640 1 --2- 172.21.15.43:0/2766580473 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f1b180cb130 0x7f1b180c7580 secure :-1 s=READY pgs=2472 cs=0 l=1 rev1=1 crypto rx=0x7f1b0800b3f0 tx=0x7f1b0800b8b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-09T15:11:49.956 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:49.952+0000 7f1b0e7fc640 1 -- 172.21.15.43:0/2766580473 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1b08011830 con 0x7f1b180cb130 2023-08-09T15:11:49.956 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:49.952+0000 7f1b0e7fc640 1 -- 172.21.15.43:0/2766580473 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1b08011990 con 0x7f1b180cb130 2023-08-09T15:11:49.957 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:49.952+0000 7f1b1ef86640 1 -- 172.21.15.43:0/2766580473 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1b18156140 con 0x7f1b180cb130 2023-08-09T15:11:49.957 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:49.952+0000 7f1b0e7fc640 1 -- 172.21.15.43:0/2766580473 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1b08011d40 con 0x7f1b180cb130 2023-08-09T15:11:49.957 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:49.952+0000 7f1b1ef86640 1 -- 172.21.15.43:0/2766580473 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1b18156700 con 0x7f1b180cb130 2023-08-09T15:11:49.957 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:49.953+0000 7f1b1ef86640 1 -- 172.21.15.43:0/2766580473 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1b181f3220 con 0x7f1b180cb130 2023-08-09T15:11:49.958 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:49.954+0000 7f1b0e7fc640 1 -- 172.21.15.43:0/2766580473 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 123) v1 ==== 88517+0+0 (secure 0 0 0) 0x7f1b0801a400 con 0x7f1b180cb130 2023-08-09T15:11:49.958 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:49.955+0000 7f1b0e7fc640 1 --2- 172.21.15.43:0/2766580473 >> [v2:172.21.15.203:6832/2375771024,v1:172.21.15.203:6833/2375771024] conn(0x7f1afc06a250 0x7f1afc06c710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T15:11:49.958 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:49.955+0000 7f1b0e7fc640 1 -- 172.21.15.43:0/2766580473 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(224..224 src has 1..224) v4 ==== 6770+0+0 (secure 0 0 0) 0x7f1b08088ab0 con 0x7f1b180cb130 2023-08-09T15:11:49.958 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:49.956+0000 7f1b1df84640 1 --2- 172.21.15.43:0/2766580473 >> [v2:172.21.15.203:6832/2375771024,v1:172.21.15.203:6833/2375771024] conn(0x7f1afc06a250 0x7f1afc06c710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T15:11:49.959 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:49.957+0000 7f1b1df84640 1 --2- 172.21.15.43:0/2766580473 >> [v2:172.21.15.203:6832/2375771024,v1:172.21.15.203:6833/2375771024] conn(0x7f1afc06a250 0x7f1afc06c710 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0x7f1b14000f60 tx=0x7f1b14009040 comp rx=0 tx=0).ready entity=mgr.9103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-09T15:11:49.960 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:49.959+0000 7f1b0e7fc640 1 -- 172.21.15.43:0/2766580473 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178066 (secure 0 0 0) 0x7f1b181f3220 con 0x7f1b180cb130 2023-08-09T15:11:50.144 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:50.143+0000 7f1b1ef86640 1 -- 172.21.15.43:0/2766580473 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f1b1804e390 con 0x7f1b180cb130 2023-08-09T15:11:50.145 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:50.143+0000 7f1b0e7fc640 1 -- 172.21.15.43:0/2766580473 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v224) v1 ==== 104+0+11 (secure 0 0 0) 0x7f1b08053c50 con 0x7f1b180cb130 2023-08-09T15:11:50.145 INFO:teuthology.orchestra.run.smithi043.stdout:pg_num: 32 2023-08-09T15:11:50.147 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:50.146+0000 7f1b1ef86640 1 -- 172.21.15.43:0/2766580473 >> [v2:172.21.15.203:6832/2375771024,v1:172.21.15.203:6833/2375771024] conn(0x7f1afc06a250 msgr2=0x7f1afc06c710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T15:11:50.147 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:50.146+0000 7f1b1ef86640 1 --2- 172.21.15.43:0/2766580473 >> [v2:172.21.15.203:6832/2375771024,v1:172.21.15.203:6833/2375771024] conn(0x7f1afc06a250 0x7f1afc06c710 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0x7f1b14000f60 tx=0x7f1b14009040 comp rx=0 tx=0).stop 2023-08-09T15:11:50.147 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:50.146+0000 7f1b1ef86640 1 -- 172.21.15.43:0/2766580473 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f1b180cb130 msgr2=0x7f1b180c7580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T15:11:50.147 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:50.146+0000 7f1b1ef86640 1 --2- 172.21.15.43:0/2766580473 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f1b180cb130 0x7f1b180c7580 secure :-1 s=READY pgs=2472 cs=0 l=1 rev1=1 crypto rx=0x7f1b0800b3f0 tx=0x7f1b0800b8b0 comp rx=0 tx=0).stop 2023-08-09T15:11:50.148 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:50.146+0000 7f1b1ef86640 1 -- 172.21.15.43:0/2766580473 shutdown_connections 2023-08-09T15:11:50.148 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:50.146+0000 7f1b1ef86640 1 --2- 172.21.15.43:0/2766580473 >> [v2:172.21.15.203:6832/2375771024,v1:172.21.15.203:6833/2375771024] conn(0x7f1afc06a250 0x7f1afc06c710 unknown :-1 s=CLOSED pgs=34 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T15:11:50.148 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:50.146+0000 7f1b1ef86640 1 --2- 172.21.15.43:0/2766580473 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f1b18159110 0x7f1b180c2410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T15:11:50.149 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:50.146+0000 7f1b1ef86640 1 --2- 172.21.15.43:0/2766580473 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f1b180cb130 0x7f1b180c7580 unknown :-1 s=CLOSED pgs=2472 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T15:11:50.149 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:50.146+0000 7f1b1ef86640 1 --2- 172.21.15.43:0/2766580473 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f1b180ca310 0x7f1b181540d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T15:11:50.149 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:50.146+0000 7f1b1ef86640 1 -- 172.21.15.43:0/2766580473 >> 172.21.15.43:0/2766580473 conn(0x7f1b180afb60 msgr2=0x7f1b180caa20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-09T15:11:50.149 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:50.146+0000 7f1b1ef86640 1 -- 172.21.15.43:0/2766580473 shutdown_connections 2023-08-09T15:11:50.149 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:50.146+0000 7f1b1ef86640 1 -- 172.21.15.43:0/2766580473 wait complete. 2023-08-09T15:11:50.165 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-08-09T15:11:50.385 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:50.382+0000 7fe313aec640 1 Processor -- start 2023-08-09T15:11:50.385 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:50.383+0000 7fe313aec640 1 -- start start 2023-08-09T15:11:50.387 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:50.384+0000 7fe313aec640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fe30c156820 0x7fe30c0ce570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T15:11:50.387 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:50.384+0000 7fe313aec640 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7fe30c06cda0 0x7fe30c15c760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T15:11:50.387 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:50.384+0000 7fe313aec640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7fe30c15cca0 0x7fe30c159110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T15:11:50.387 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:50.384+0000 7fe313aec640 1 -- --> v2:172.21.15.43:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe30c15d130 con 0x7fe30c15cca0 2023-08-09T15:11:50.388 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:50.384+0000 7fe313aec640 1 -- --> v2:172.21.15.203:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe30c156c00 con 0x7fe30c06cda0 2023-08-09T15:11:50.388 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:50.384+0000 7fe313aec640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe30c06d610 con 0x7fe30c156820 2023-08-09T15:11:50.388 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:50.385+0000 7fe312089640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7fe30c15cca0 0x7fe30c159110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T15:11:50.388 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:50.385+0000 7fe312089640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7fe30c15cca0 0x7fe30c159110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:50252/0 (socket says 172.21.15.43:50252) 2023-08-09T15:11:50.388 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:50.385+0000 7fe312089640 1 -- 172.21.15.43:0/1880063758 learned_addr learned my addr 172.21.15.43:0/1880063758 (peer_addr_for_me v2:172.21.15.43:0/0) 2023-08-09T15:11:50.388 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:50.385+0000 7fe311087640 1 --2- 172.21.15.43:0/1880063758 >> v2:172.21.15.203:3300/0 conn(0x7fe30c06cda0 0x7fe30c15c760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T15:11:50.388 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:50.385+0000 7fe311888640 1 --2- 172.21.15.43:0/1880063758 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fe30c156820 0x7fe30c0ce570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T15:11:50.389 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:50.385+0000 7fe312089640 1 -- 172.21.15.43:0/1880063758 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fe30c156820 msgr2=0x7fe30c0ce570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-09T15:11:50.389 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:50.385+0000 7fe312089640 1 --2- 172.21.15.43:0/1880063758 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fe30c156820 0x7fe30c0ce570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T15:11:50.389 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:50.385+0000 7fe312089640 1 -- 172.21.15.43:0/1880063758 >> v2:172.21.15.203:3300/0 conn(0x7fe30c06cda0 msgr2=0x7fe30c15c760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-09T15:11:50.389 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:50.385+0000 7fe312089640 1 --2- 172.21.15.43:0/1880063758 >> v2:172.21.15.203:3300/0 conn(0x7fe30c06cda0 0x7fe30c15c760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T15:11:50.389 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:50.385+0000 7fe312089640 1 -- 172.21.15.43:0/1880063758 --> v2:172.21.15.43:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe30c06d8f0 con 0x7fe30c15cca0 2023-08-09T15:11:50.389 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:50.385+0000 7fe311888640 1 --2- 172.21.15.43:0/1880063758 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fe30c156820 0x7fe30c0ce570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-08-09T15:11:50.390 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:50.386+0000 7fe312089640 1 --2- 172.21.15.43:0/1880063758 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fe30c15cca0 0x7fe30c159110 secure :-1 s=READY pgs=2473 cs=0 l=1 rev1=1 crypto rx=0x7fe30c0cfc30 tx=0x7fe30402be40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3131971698a6e5c3 server_cookie=0 in_seq=0 out_seq=0 2023-08-09T15:11:50.390 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:50.387+0000 7fe310886640 1 -- 172.21.15.43:0/1880063758 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe30402c9e0 con 0x7fe30c15cca0 2023-08-09T15:11:50.391 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:50.387+0000 7fe310886640 1 -- 172.21.15.43:0/1880063758 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe30402cb40 con 0x7fe30c15cca0 2023-08-09T15:11:50.391 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:50.387+0000 7fe310886640 1 -- 172.21.15.43:0/1880063758 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe3040343f0 con 0x7fe30c15cca0 2023-08-09T15:11:50.391 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:50.387+0000 7fe313aec640 1 -- 172.21.15.43:0/1880063758 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fe30c15cca0 msgr2=0x7fe30c159110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T15:11:50.391 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:50.387+0000 7fe313aec640 1 --2- 172.21.15.43:0/1880063758 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fe30c15cca0 0x7fe30c159110 secure :-1 s=READY pgs=2473 cs=0 l=1 rev1=1 crypto rx=0x7fe30c0cfc30 tx=0x7fe30402be40 comp rx=0 tx=0).stop 2023-08-09T15:11:50.391 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:50.388+0000 7fe313aec640 1 -- 172.21.15.43:0/1880063758 shutdown_connections 2023-08-09T15:11:50.391 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:50.388+0000 7fe313aec640 1 --2- 172.21.15.43:0/1880063758 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fe30c15cca0 0x7fe30c159110 unknown :-1 s=CLOSED pgs=2473 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T15:11:50.391 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:50.388+0000 7fe313aec640 1 --2- 172.21.15.43:0/1880063758 >> v2:172.21.15.203:3300/0 conn(0x7fe30c06cda0 0x7fe30c15c760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T15:11:50.391 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:50.388+0000 7fe313aec640 1 --2- 172.21.15.43:0/1880063758 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fe30c156820 0x7fe30c0ce570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T15:11:50.392 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:50.388+0000 7fe313aec640 1 -- 172.21.15.43:0/1880063758 >> 172.21.15.43:0/1880063758 conn(0x7fe30c0cb5b0 msgr2=0x7fe30c0cb990 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-09T15:11:50.392 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:50.388+0000 7fe313aec640 1 -- 172.21.15.43:0/1880063758 shutdown_connections 2023-08-09T15:11:50.392 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:50.388+0000 7fe313aec640 1 -- 172.21.15.43:0/1880063758 wait complete. 2023-08-09T15:11:50.393 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:50.389+0000 7fe313aec640 1 Processor -- start 2023-08-09T15:11:50.394 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:50.390+0000 7fe313aec640 1 -- start start 2023-08-09T15:11:50.394 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:50.391+0000 7fe313aec640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fe30c06cda0 0x7fe30c0c89b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T15:11:50.394 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:50.391+0000 7fe313aec640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fe30c156820 0x7fe30c14ff50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T15:11:50.394 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:50.391+0000 7fe313aec640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fe30c15cca0 0x7fe30c158da0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T15:11:50.394 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:50.391+0000 7fe313aec640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe30c0c2660 con 0x7fe30c156820 2023-08-09T15:11:50.394 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:50.391+0000 7fe313aec640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe30c0c8ef0 con 0x7fe30c06cda0 2023-08-09T15:11:50.394 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:50.391+0000 7fe313aec640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe30c0c7d50 con 0x7fe30c15cca0 2023-08-09T15:11:50.395 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:50.391+0000 7fe311087640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fe30c156820 0x7fe30c14ff50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T15:11:50.395 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:50.391+0000 7fe311087640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fe30c156820 0x7fe30c14ff50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:50260/0 (socket says 172.21.15.43:50260) 2023-08-09T15:11:50.395 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:50.391+0000 7fe311087640 1 -- 172.21.15.43:0/117557031 learned_addr learned my addr 172.21.15.43:0/117557031 (peer_addr_for_me v2:172.21.15.43:0/0) 2023-08-09T15:11:50.395 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:50.391+0000 7fe312089640 1 --2- 172.21.15.43:0/117557031 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fe30c15cca0 0x7fe30c158da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T15:11:50.395 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:50.391+0000 7fe311087640 1 -- 172.21.15.43:0/117557031 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fe30c15cca0 msgr2=0x7fe30c158da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T15:11:50.395 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:50.391+0000 7fe311087640 1 --2- 172.21.15.43:0/117557031 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fe30c15cca0 0x7fe30c158da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T15:11:50.395 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:50.391+0000 7fe311087640 1 -- 172.21.15.43:0/117557031 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fe30c06cda0 msgr2=0x7fe30c0c89b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T15:11:50.396 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:50.391+0000 7fe311888640 1 --2- 172.21.15.43:0/117557031 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fe30c06cda0 0x7fe30c0c89b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T15:11:50.396 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:50.391+0000 7fe311087640 1 --2- 172.21.15.43:0/117557031 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fe30c06cda0 0x7fe30c0c89b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T15:11:50.396 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:50.392+0000 7fe311087640 1 -- 172.21.15.43:0/117557031 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe304009b30 con 0x7fe30c156820 2023-08-09T15:11:50.396 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:50.392+0000 7fe312089640 1 --2- 172.21.15.43:0/117557031 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fe30c15cca0 0x7fe30c158da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-08-09T15:11:50.396 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:50.392+0000 7fe311087640 1 --2- 172.21.15.43:0/117557031 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fe30c156820 0x7fe30c14ff50 secure :-1 s=READY pgs=2474 cs=0 l=1 rev1=1 crypto rx=0x7fe2fc00d800 tx=0x7fe2fc00dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-09T15:11:50.396 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:50.392+0000 7fe3027fc640 1 -- 172.21.15.43:0/117557031 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe2fc0096a0 con 0x7fe30c156820 2023-08-09T15:11:50.396 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:50.392+0000 7fe3027fc640 1 -- 172.21.15.43:0/117557031 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe2fc009800 con 0x7fe30c156820 2023-08-09T15:11:50.397 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:50.392+0000 7fe313aec640 1 -- 172.21.15.43:0/117557031 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe30c0c8030 con 0x7fe30c156820 2023-08-09T15:11:50.397 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:50.392+0000 7fe3027fc640 1 -- 172.21.15.43:0/117557031 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe2fc009990 con 0x7fe30c156820 2023-08-09T15:11:50.397 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:50.392+0000 7fe313aec640 1 -- 172.21.15.43:0/117557031 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe30c0c2b50 con 0x7fe30c156820 2023-08-09T15:11:50.397 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:50.393+0000 7fe313aec640 1 -- 172.21.15.43:0/117557031 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe30c04e390 con 0x7fe30c156820 2023-08-09T15:11:50.398 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:50.394+0000 7fe3027fc640 1 -- 172.21.15.43:0/117557031 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 123) v1 ==== 88517+0+0 (secure 0 0 0) 0x7fe2fc009af0 con 0x7fe30c156820 2023-08-09T15:11:50.398 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:50.395+0000 7fe3027fc640 1 --2- 172.21.15.43:0/117557031 >> [v2:172.21.15.203:6832/2375771024,v1:172.21.15.203:6833/2375771024] conn(0x7fe2f006a250 0x7fe2f006c710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T15:11:50.398 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:50.395+0000 7fe3027fc640 1 -- 172.21.15.43:0/117557031 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(224..224 src has 1..224) v4 ==== 6770+0+0 (secure 0 0 0) 0x7fe2fc089080 con 0x7fe30c156820 2023-08-09T15:11:50.398 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:50.396+0000 7fe311888640 1 --2- 172.21.15.43:0/117557031 >> [v2:172.21.15.203:6832/2375771024,v1:172.21.15.203:6833/2375771024] conn(0x7fe2f006a250 0x7fe2f006c710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T15:11:50.398 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:50.397+0000 7fe311888640 1 --2- 172.21.15.43:0/117557031 >> [v2:172.21.15.203:6832/2375771024,v1:172.21.15.203:6833/2375771024] conn(0x7fe2f006a250 0x7fe2f006c710 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7fe30800b840 tx=0x7fe308008040 comp rx=0 tx=0).ready entity=mgr.9103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-09T15:11:50.401 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:50.400+0000 7fe3027fc640 1 -- 172.21.15.43:0/117557031 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178066 (secure 0 0 0) 0x7fe2fc0bb7b0 con 0x7fe30c156820 2023-08-09T15:11:50.590 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:50.588+0000 7fe313aec640 1 -- 172.21.15.43:0/117557031 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7fe30c06dba0 con 0x7fe30c156820 2023-08-09T15:11:50.591 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:50.589+0000 7fe3027fc640 1 -- 172.21.15.43:0/117557031 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v224) v1 ==== 100+0+11 (secure 0 0 0) 0x7fe2fc054210 con 0x7fe30c156820 2023-08-09T15:11:50.591 INFO:teuthology.orchestra.run.smithi043.stdout:pg_num: 64 2023-08-09T15:11:50.593 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:50.591+0000 7fe313aec640 1 -- 172.21.15.43:0/117557031 >> [v2:172.21.15.203:6832/2375771024,v1:172.21.15.203:6833/2375771024] conn(0x7fe2f006a250 msgr2=0x7fe2f006c710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T15:11:50.593 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:50.591+0000 7fe313aec640 1 --2- 172.21.15.43:0/117557031 >> [v2:172.21.15.203:6832/2375771024,v1:172.21.15.203:6833/2375771024] conn(0x7fe2f006a250 0x7fe2f006c710 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7fe30800b840 tx=0x7fe308008040 comp rx=0 tx=0).stop 2023-08-09T15:11:50.593 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:50.591+0000 7fe313aec640 1 -- 172.21.15.43:0/117557031 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fe30c156820 msgr2=0x7fe30c14ff50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T15:11:50.593 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:50.591+0000 7fe313aec640 1 --2- 172.21.15.43:0/117557031 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fe30c156820 0x7fe30c14ff50 secure :-1 s=READY pgs=2474 cs=0 l=1 rev1=1 crypto rx=0x7fe2fc00d800 tx=0x7fe2fc00dcc0 comp rx=0 tx=0).stop 2023-08-09T15:11:50.594 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:50.592+0000 7fe313aec640 1 -- 172.21.15.43:0/117557031 shutdown_connections 2023-08-09T15:11:50.594 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:50.592+0000 7fe313aec640 1 --2- 172.21.15.43:0/117557031 >> [v2:172.21.15.203:6832/2375771024,v1:172.21.15.203:6833/2375771024] conn(0x7fe2f006a250 0x7fe2f006c710 unknown :-1 s=CLOSED pgs=35 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T15:11:50.594 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:50.592+0000 7fe313aec640 1 --2- 172.21.15.43:0/117557031 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fe30c15cca0 0x7fe30c158da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T15:11:50.594 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:50.592+0000 7fe313aec640 1 --2- 172.21.15.43:0/117557031 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fe30c156820 0x7fe30c14ff50 unknown :-1 s=CLOSED pgs=2474 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T15:11:50.594 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:50.592+0000 7fe313aec640 1 --2- 172.21.15.43:0/117557031 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fe30c06cda0 0x7fe30c0c89b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T15:11:50.594 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:50.592+0000 7fe313aec640 1 -- 172.21.15.43:0/117557031 >> 172.21.15.43:0/117557031 conn(0x7fe30c0cb5b0 msgr2=0x7fe30c0ae710 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-09T15:11:50.594 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:50.592+0000 7fe313aec640 1 -- 172.21.15.43:0/117557031 shutdown_connections 2023-08-09T15:11:50.595 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:50.592+0000 7fe313aec640 1 -- 172.21.15.43:0/117557031 wait complete. 2023-08-09T15:11:50.610 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2023-08-09T15:11:50.825 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:50.823+0000 7fdc0aad9640 1 Processor -- start 2023-08-09T15:11:50.825 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:50.824+0000 7fdc0aad9640 1 -- start start 2023-08-09T15:11:50.826 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:50.825+0000 7fdc0aad9640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fdc0415c880 0x7fdc040ccbc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T15:11:50.827 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:50.825+0000 7fdc0aad9640 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7fdc040cf390 0x7fdc040d0a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T15:11:50.827 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:50.825+0000 7fdc0aad9640 1 -- --> v1:172.21.15.43:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fdc0415cc60 con 0x7fdc04160f20 2023-08-09T15:11:50.827 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:50.825+0000 7fdc0aad9640 1 -- --> v2:172.21.15.203:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fdc040d0530 con 0x7fdc040cf390 2023-08-09T15:11:50.827 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:50.825+0000 7fdc0aad9640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdc040cd410 con 0x7fdc0415c880 2023-08-09T15:11:50.828 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:50.826+0000 7fdc03fff640 1 --1- >> v1:172.21.15.43:6789/0 conn(0x7fdc04160f20 0x7fdc040cec80 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.43:6789/0 says I am v1:172.21.15.43:56344/0 (socket says 172.21.15.43:56344) 2023-08-09T15:11:50.829 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:50.826+0000 7fdc03fff640 1 -- 172.21.15.43:0/1065136477 learned_addr learned my addr 172.21.15.43:0/1065136477 (peer_addr_for_me v1:172.21.15.43:0/0) 2023-08-09T15:11:50.829 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:50.826+0000 7fdc08875640 1 --2- 172.21.15.43:0/1065136477 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fdc0415c880 0x7fdc040ccbc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T15:11:50.829 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:50.826+0000 7fdc09076640 1 --2- 172.21.15.43:0/1065136477 >> v2:172.21.15.203:3300/0 conn(0x7fdc040cf390 0x7fdc040d0a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T15:11:50.829 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:50.826+0000 7fdc037fe640 1 -- 172.21.15.43:0/1065136477 <== mon.0 v1:172.21.15.43:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 167785276 0 0) 0x7fdc0415cc60 con 0x7fdc04160f20 2023-08-09T15:11:50.829 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:50.826+0000 7fdc037fe640 1 -- 172.21.15.43:0/1065136477 --> v1:172.21.15.43:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fdbf4003520 con 0x7fdc04160f20 2023-08-09T15:11:50.829 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:50.827+0000 7fdc037fe640 1 -- 172.21.15.43:0/1065136477 <== mon.0 v1:172.21.15.43:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1217582945 0 0) 0x7fdbf4003520 con 0x7fdc04160f20 2023-08-09T15:11:50.830 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:50.827+0000 7fdc037fe640 1 -- 172.21.15.43:0/1065136477 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fdc0415c880 msgr2=0x7fdc040ccbc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-09T15:11:50.830 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:50.827+0000 7fdc037fe640 1 --2- 172.21.15.43:0/1065136477 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fdc0415c880 0x7fdc040ccbc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T15:11:50.830 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:50.827+0000 7fdc037fe640 1 -- 172.21.15.43:0/1065136477 >> v2:172.21.15.203:3300/0 conn(0x7fdc040cf390 msgr2=0x7fdc040d0a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-09T15:11:50.830 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:50.827+0000 7fdc037fe640 1 --2- 172.21.15.43:0/1065136477 >> v2:172.21.15.203:3300/0 conn(0x7fdc040cf390 0x7fdc040d0a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T15:11:50.830 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:50.827+0000 7fdc037fe640 1 -- 172.21.15.43:0/1065136477 --> v1:172.21.15.43:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdc040cd6f0 con 0x7fdc04160f20 2023-08-09T15:11:50.830 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:50.827+0000 7fdc037fe640 1 -- 172.21.15.43:0/1065136477 <== mon.0 v1:172.21.15.43:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1285150740 0 0) 0x7fdbf0001670 con 0x7fdc04160f20 2023-08-09T15:11:50.830 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:50.827+0000 7fdc037fe640 1 -- 172.21.15.43:0/1065136477 >> v1:172.21.15.43:6789/0 conn(0x7fdc04160f20 legacy=0x7fdc040cec80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T15:11:50.830 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:50.827+0000 7fdc037fe640 1 --2- 172.21.15.43:0/1065136477 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fdc0415c880 0x7fdbf4003970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T15:11:50.831 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:50.827+0000 7fdc037fe640 1 --2- 172.21.15.43:0/1065136477 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fdbf4003ef0 0x7fdbf40047c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T15:11:50.831 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:50.827+0000 7fdc037fe640 1 --2- 172.21.15.43:0/1065136477 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fdbf4004d00 0x7fdbf4005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T15:11:50.831 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:50.827+0000 7fdc037fe640 1 -- 172.21.15.43:0/1065136477 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdc040d0530 con 0x7fdbf4003ef0 2023-08-09T15:11:50.831 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:50.827+0000 7fdc037fe640 1 -- 172.21.15.43:0/1065136477 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdc040cd410 con 0x7fdbf4004d00 2023-08-09T15:11:50.831 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:50.827+0000 7fdc037fe640 1 -- 172.21.15.43:0/1065136477 --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdbf400b980 con 0x7fdc0415c880 2023-08-09T15:11:50.831 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:50.827+0000 7fdc03fff640 1 --2- 172.21.15.43:0/1065136477 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fdbf4003ef0 0x7fdbf40047c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T15:11:50.831 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:50.827+0000 7fdc08875640 1 --2- 172.21.15.43:0/1065136477 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fdc0415c880 0x7fdbf4003970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T15:11:50.832 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:50.827+0000 7fdc09076640 1 --2- 172.21.15.43:0/1065136477 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fdbf4004d00 0x7fdbf4005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T15:11:50.833 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:50.828+0000 7fdc03fff640 1 -- 172.21.15.43:0/1065136477 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fdc0415c880 msgr2=0x7fdbf4003970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-09T15:11:50.833 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:50.828+0000 7fdc03fff640 1 --2- 172.21.15.43:0/1065136477 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fdc0415c880 0x7fdbf4003970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T15:11:50.833 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:50.828+0000 7fdc03fff640 1 -- 172.21.15.43:0/1065136477 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fdbf4004d00 msgr2=0x7fdbf4005180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-09T15:11:50.833 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:50.828+0000 7fdc03fff640 1 --2- 172.21.15.43:0/1065136477 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fdbf4004d00 0x7fdbf4005180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T15:11:50.833 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:50.828+0000 7fdc03fff640 1 -- 172.21.15.43:0/1065136477 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdbf4003520 con 0x7fdbf4003ef0 2023-08-09T15:11:50.833 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:50.829+0000 7fdc03fff640 1 --2- 172.21.15.43:0/1065136477 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fdbf4003ef0 0x7fdbf40047c0 secure :-1 s=READY pgs=2476 cs=0 l=1 rev1=1 crypto rx=0x7fdbf000c240 tx=0x7fdbf40057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1b2a70ad11ed7686 server_cookie=0 in_seq=0 out_seq=0 2023-08-09T15:11:50.834 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:50.829+0000 7fdc037fe640 1 -- 172.21.15.43:0/1065136477 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdbf002cae0 con 0x7fdbf4003ef0 2023-08-09T15:11:50.834 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:50.829+0000 7fdc037fe640 1 -- 172.21.15.43:0/1065136477 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdbf002cc40 con 0x7fdbf4003ef0 2023-08-09T15:11:50.834 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:50.829+0000 7fdc037fe640 1 -- 172.21.15.43:0/1065136477 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdbf0005af0 con 0x7fdbf4003ef0 2023-08-09T15:11:50.834 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:50.830+0000 7fdc0aad9640 1 -- 172.21.15.43:0/1065136477 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fdbf4003ef0 msgr2=0x7fdbf40047c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T15:11:50.834 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:50.830+0000 7fdc0aad9640 1 --2- 172.21.15.43:0/1065136477 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fdbf4003ef0 0x7fdbf40047c0 secure :-1 s=READY pgs=2476 cs=0 l=1 rev1=1 crypto rx=0x7fdbf000c240 tx=0x7fdbf40057f0 comp rx=0 tx=0).stop 2023-08-09T15:11:50.834 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:50.830+0000 7fdc09076640 1 -- 172.21.15.43:0/1065136477 reap_dead start 2023-08-09T15:11:50.834 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:50.830+0000 7fdc0aad9640 1 -- 172.21.15.43:0/1065136477 shutdown_connections 2023-08-09T15:11:50.834 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:50.830+0000 7fdc0aad9640 1 -- 172.21.15.43:0/1065136477 >> 172.21.15.43:0/1065136477 conn(0x7fdc0408bfb0 msgr2=0x7fdc0409b8d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-09T15:11:50.835 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:50.830+0000 7fdc0aad9640 1 -- 172.21.15.43:0/1065136477 shutdown_connections 2023-08-09T15:11:50.835 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:50.830+0000 7fdc0aad9640 1 -- 172.21.15.43:0/1065136477 wait complete. 2023-08-09T15:11:50.836 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:50.832+0000 7fdc0aad9640 1 Processor -- start 2023-08-09T15:11:50.836 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:50.832+0000 7fdc0aad9640 1 -- start start 2023-08-09T15:11:50.837 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:50.833+0000 7fdc0aad9640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fdbf4003ef0 0x7fdc040b74f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T15:11:50.837 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:50.833+0000 7fdc0aad9640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fdc0415cfe0 0x7fdc04150a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T15:11:50.837 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:50.833+0000 7fdc0aad9640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fdc0415f8c0 0x7fdc040b3e90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T15:11:50.837 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:50.833+0000 7fdc0aad9640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdc0415e8d0 con 0x7fdbf4003ef0 2023-08-09T15:11:50.837 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:50.833+0000 7fdc0aad9640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdc040b7ad0 con 0x7fdc0415cfe0 2023-08-09T15:11:50.837 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:50.833+0000 7fdc0aad9640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdc040b7fd0 con 0x7fdc0415f8c0 2023-08-09T15:11:50.837 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:50.833+0000 7fdc08875640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fdbf4003ef0 0x7fdc040b74f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T15:11:50.838 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:50.833+0000 7fdc08875640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fdbf4003ef0 0x7fdc040b74f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:50278/0 (socket says 172.21.15.43:50278) 2023-08-09T15:11:50.838 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:50.834+0000 7fdc08875640 1 -- 172.21.15.43:0/2950764846 learned_addr learned my addr 172.21.15.43:0/2950764846 (peer_addr_for_me v2:172.21.15.43:0/0) 2023-08-09T15:11:50.838 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:50.834+0000 7fdc03fff640 1 --2- 172.21.15.43:0/2950764846 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fdc0415cfe0 0x7fdc04150a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T15:11:50.838 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:50.834+0000 7fdc09076640 1 --2- 172.21.15.43:0/2950764846 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fdc0415f8c0 0x7fdc040b3e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T15:11:50.838 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:50.834+0000 7fdc08875640 1 -- 172.21.15.43:0/2950764846 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fdc0415f8c0 msgr2=0x7fdc040b3e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T15:11:50.838 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:50.834+0000 7fdc08875640 1 --2- 172.21.15.43:0/2950764846 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fdc0415f8c0 0x7fdc040b3e90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T15:11:50.838 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:50.834+0000 7fdc08875640 1 -- 172.21.15.43:0/2950764846 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fdc0415cfe0 msgr2=0x7fdc04150a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T15:11:50.838 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:50.834+0000 7fdc08875640 1 --2- 172.21.15.43:0/2950764846 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fdc0415cfe0 0x7fdc04150a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T15:11:50.839 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:50.834+0000 7fdc08875640 1 -- 172.21.15.43:0/2950764846 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdc0415d460 con 0x7fdbf4003ef0 2023-08-09T15:11:50.839 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:50.834+0000 7fdc09076640 1 --2- 172.21.15.43:0/2950764846 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fdc0415f8c0 0x7fdc040b3e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-08-09T15:11:50.839 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:50.834+0000 7fdc08875640 1 --2- 172.21.15.43:0/2950764846 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fdbf4003ef0 0x7fdc040b74f0 secure :-1 s=READY pgs=2477 cs=0 l=1 rev1=1 crypto rx=0x7fdbf80115b0 tx=0x7fdbf8011a70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-09T15:11:50.839 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:50.834+0000 7fdc017fa640 1 -- 172.21.15.43:0/2950764846 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdbf8005da0 con 0x7fdbf4003ef0 2023-08-09T15:11:50.839 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:50.835+0000 7fdc017fa640 1 -- 172.21.15.43:0/2950764846 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdbf8006050 con 0x7fdbf4003ef0 2023-08-09T15:11:50.839 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:50.835+0000 7fdc017fa640 1 -- 172.21.15.43:0/2950764846 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdbf80063c0 con 0x7fdbf4003ef0 2023-08-09T15:11:50.839 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:50.835+0000 7fdc0aad9640 1 -- 172.21.15.43:0/2950764846 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdc0415eb00 con 0x7fdbf4003ef0 2023-08-09T15:11:50.839 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:50.835+0000 7fdc0aad9640 1 -- 172.21.15.43:0/2950764846 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdc0415f0c0 con 0x7fdbf4003ef0 2023-08-09T15:11:50.840 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:50.836+0000 7fdc0aad9640 1 -- 172.21.15.43:0/2950764846 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdc0404db40 con 0x7fdbf4003ef0 2023-08-09T15:11:50.840 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:50.837+0000 7fdc017fa640 1 -- 172.21.15.43:0/2950764846 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 123) v1 ==== 88517+0+0 (secure 0 0 0) 0x7fdbf8006520 con 0x7fdbf4003ef0 2023-08-09T15:11:50.840 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:50.838+0000 7fdc017fa640 1 --2- 172.21.15.43:0/2950764846 >> [v2:172.21.15.203:6832/2375771024,v1:172.21.15.203:6833/2375771024] conn(0x7fdbe806a280 0x7fdbe806c740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T15:11:50.841 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:50.838+0000 7fdc017fa640 1 -- 172.21.15.43:0/2950764846 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(224..224 src has 1..224) v4 ==== 6770+0+0 (secure 0 0 0) 0x7fdbf8089220 con 0x7fdbf4003ef0 2023-08-09T15:11:50.841 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:50.839+0000 7fdc03fff640 1 --2- 172.21.15.43:0/2950764846 >> [v2:172.21.15.203:6832/2375771024,v1:172.21.15.203:6833/2375771024] conn(0x7fdbe806a280 0x7fdbe806c740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T15:11:50.841 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:50.840+0000 7fdc03fff640 1 --2- 172.21.15.43:0/2950764846 >> [v2:172.21.15.203:6832/2375771024,v1:172.21.15.203:6833/2375771024] conn(0x7fdbe806a280 0x7fdbe806c740 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7fdbf0034000 tx=0x7fdbf0007000 comp rx=0 tx=0).ready entity=mgr.9103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-09T15:11:50.844 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:50.843+0000 7fdc017fa640 1 -- 172.21.15.43:0/2950764846 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178066 (secure 0 0 0) 0x7fdbf8054320 con 0x7fdbf4003ef0 2023-08-09T15:11:51.054 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:51.052+0000 7fdc0aad9640 1 -- 172.21.15.43:0/2950764846 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7fdc040d0dd0 con 0x7fdbf4003ef0 2023-08-09T15:11:51.054 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:51.053+0000 7fdc017fa640 1 -- 172.21.15.43:0/2950764846 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 8 entries v224) v1 ==== 80+0+476 (secure 0 0 0) 0x7fdbf8057940 con 0x7fdbf4003ef0 2023-08-09T15:11:51.055 INFO:teuthology.orchestra.run.smithi043.stdout:172.21.15.203:6832/4034093268 2023-08-10T15:11:43.130145+0000 2023-08-09T15:11:51.055 INFO:teuthology.orchestra.run.smithi043.stdout:172.21.15.203:6833/4034093268 2023-08-10T15:11:43.130145+0000 2023-08-09T15:11:51.055 INFO:teuthology.orchestra.run.smithi043.stdout:172.21.15.203:0/1654193324 2023-08-10T15:11:43.130145+0000 2023-08-09T15:11:51.055 INFO:teuthology.orchestra.run.smithi043.stdout:172.21.15.203:0/4140323243 2023-08-10T15:11:43.130145+0000 2023-08-09T15:11:51.055 INFO:teuthology.orchestra.run.smithi043.stdout:172.21.15.203:0/450863258 2023-08-10T15:11:43.130145+0000 2023-08-09T15:11:51.056 INFO:teuthology.orchestra.run.smithi043.stdout:172.21.15.203:0/134877587 2023-08-10T15:11:43.130145+0000 2023-08-09T15:11:51.056 INFO:teuthology.orchestra.run.smithi043.stdout:172.21.15.203:0/2330869600 2023-08-10T15:11:43.130145+0000 2023-08-09T15:11:51.056 INFO:teuthology.orchestra.run.smithi043.stdout:172.21.15.203:0/2203818837 2023-08-10T15:11:43.130145+0000 2023-08-09T15:11:51.056 INFO:teuthology.orchestra.run.smithi043.stderr:listed 8 entries 2023-08-09T15:11:51.058 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:51.056+0000 7fdc0aad9640 1 -- 172.21.15.43:0/2950764846 >> [v2:172.21.15.203:6832/2375771024,v1:172.21.15.203:6833/2375771024] conn(0x7fdbe806a280 msgr2=0x7fdbe806c740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T15:11:51.058 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:51.056+0000 7fdc0aad9640 1 --2- 172.21.15.43:0/2950764846 >> [v2:172.21.15.203:6832/2375771024,v1:172.21.15.203:6833/2375771024] conn(0x7fdbe806a280 0x7fdbe806c740 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7fdbf0034000 tx=0x7fdbf0007000 comp rx=0 tx=0).stop 2023-08-09T15:11:51.058 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:51.056+0000 7fdc0aad9640 1 -- 172.21.15.43:0/2950764846 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fdbf4003ef0 msgr2=0x7fdc040b74f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T15:11:51.058 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:51.056+0000 7fdc0aad9640 1 --2- 172.21.15.43:0/2950764846 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fdbf4003ef0 0x7fdc040b74f0 secure :-1 s=READY pgs=2477 cs=0 l=1 rev1=1 crypto rx=0x7fdbf80115b0 tx=0x7fdbf8011a70 comp rx=0 tx=0).stop 2023-08-09T15:11:51.059 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:51.056+0000 7fdc0aad9640 1 -- 172.21.15.43:0/2950764846 shutdown_connections 2023-08-09T15:11:51.059 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:51.056+0000 7fdc0aad9640 1 --2- 172.21.15.43:0/2950764846 >> [v2:172.21.15.203:6832/2375771024,v1:172.21.15.203:6833/2375771024] conn(0x7fdbe806a280 0x7fdbe806c740 unknown :-1 s=CLOSED pgs=36 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T15:11:51.059 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:51.056+0000 7fdc0aad9640 1 --2- 172.21.15.43:0/2950764846 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fdc0415f8c0 0x7fdc040b3e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T15:11:51.059 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:51.056+0000 7fdc0aad9640 1 --2- 172.21.15.43:0/2950764846 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fdc0415cfe0 0x7fdc04150a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T15:11:51.059 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:51.056+0000 7fdc0aad9640 1 --2- 172.21.15.43:0/2950764846 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fdbf4003ef0 0x7fdc040b74f0 unknown :-1 s=CLOSED pgs=2477 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T15:11:51.059 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:51.056+0000 7fdc0aad9640 1 -- 172.21.15.43:0/2950764846 >> 172.21.15.43:0/2950764846 conn(0x7fdc0408bfb0 msgr2=0x7fdc040d12f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-09T15:11:51.059 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:51.056+0000 7fdc0aad9640 1 -- 172.21.15.43:0/2950764846 shutdown_connections 2023-08-09T15:11:51.059 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:51.056+0000 7fdc0aad9640 1 -- 172.21.15.43:0/2950764846 wait complete. 2023-08-09T15:11:51.077 INFO:tasks.cephfs.fuse_mount:Running fusermount -u on ubuntu@smithi043.front.sepia.ceph.com... 2023-08-09T15:11:51.078 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-09T15:11:51.078 DEBUG:teuthology.orchestra.run.smithi043:> sudo fusermount -u /home/ubuntu/cephtest/mnt.0 2023-08-09T15:11:51.172 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi043.stderr:ceph-fuse[153632]: fuse finished with error 0 and tester_r 0 2023-08-09T15:11:51.176 INFO:teuthology.orchestra.run:waiting for 300 2023-08-09T15:11:57.278 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi043.front.sepia.ceph.com 2023-08-09T15:11:57.279 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-09T15:11:57.279 DEBUG:teuthology.orchestra.run.smithi043:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2023-08-09T15:11:57.345 DEBUG:teuthology.orchestra.run.smithi043:> set -ex 2023-08-09T15:11:57.345 DEBUG:teuthology.orchestra.run.smithi043:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-09T15:11:57.403 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4054441,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6498844k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=27,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3249420k,nr_inodes=812355,mode=700,uid=1000,gid=1275 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6498844k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2023-08-09T15:11:57.404 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2023-08-09T15:11:57.633 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:57.631+0000 7f5d7ffff640 1 Processor -- start 2023-08-09T15:11:57.633 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:57.632+0000 7f5d7ffff640 1 -- start start 2023-08-09T15:11:57.634 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:57.633+0000 7f5d7ffff640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f5d80158050 0x7f5d80070400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T15:11:57.635 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:57.633+0000 7f5d7ffff640 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7f5d8006b740 0x7f5d8015b320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T15:11:57.635 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:57.633+0000 7f5d7ffff640 1 -- --> v1:172.21.15.203:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5d80071420 con 0x7f5d80157c70 2023-08-09T15:11:57.635 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:57.633+0000 7f5d7ffff640 1 -- --> v2:172.21.15.203:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5d8006f710 con 0x7f5d8006b740 2023-08-09T15:11:57.635 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:57.633+0000 7f5d7ffff640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5d8015b860 con 0x7f5d80158050 2023-08-09T15:11:57.636 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:57.634+0000 7f5d77fff640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f5d80158050 0x7f5d80070400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T15:11:57.636 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:57.634+0000 7f5d7f7fe640 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7f5d8006b740 0x7f5d8015b320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T15:11:57.636 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:57.634+0000 7f5d77fff640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f5d80158050 0x7f5d80070400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.203:3301/0 says I am v2:172.21.15.43:38654/0 (socket says 172.21.15.43:38654) 2023-08-09T15:11:57.636 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:57.634+0000 7f5d77fff640 1 -- 172.21.15.43:0/1192374254 learned_addr learned my addr 172.21.15.43:0/1192374254 (peer_addr_for_me v2:172.21.15.43:0/0) 2023-08-09T15:11:57.637 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:57.635+0000 7f5d7e7fc640 1 -- 172.21.15.43:0/1192374254 <== mon.1 v1:172.21.15.203:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 580579982 0 0) 0x7f5d80071420 con 0x7f5d80157c70 2023-08-09T15:11:57.637 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:57.635+0000 7f5d7e7fc640 1 -- 172.21.15.43:0/1192374254 --> v1:172.21.15.203:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f5d6c003520 con 0x7f5d80157c70 2023-08-09T15:11:57.637 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:57.635+0000 7f5d77fff640 1 -- 172.21.15.43:0/1192374254 >> v2:172.21.15.203:3300/0 conn(0x7f5d8006b740 msgr2=0x7f5d8015b320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-09T15:11:57.637 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:57.635+0000 7f5d77fff640 1 --2- 172.21.15.43:0/1192374254 >> v2:172.21.15.203:3300/0 conn(0x7f5d8006b740 0x7f5d8015b320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T15:11:57.637 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:57.635+0000 7f5d77fff640 1 -- 172.21.15.43:0/1192374254 >> v1:172.21.15.203:6789/0 conn(0x7f5d80157c70 legacy=0x7f5d8006b030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T15:11:57.637 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:57.635+0000 7f5d77fff640 1 -- 172.21.15.43:0/1192374254 --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5d8006d650 con 0x7f5d80158050 2023-08-09T15:11:57.638 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:57.636+0000 7f5d77fff640 1 --2- 172.21.15.43:0/1192374254 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f5d80158050 0x7f5d80070400 secure :-1 s=READY pgs=349 cs=0 l=1 rev1=1 crypto rx=0x7f5d68009870 tx=0x7f5d6802b980 comp rx=0 tx=0).ready entity=mon.2 client_cookie=8331d0cc6a411115 server_cookie=0 in_seq=0 out_seq=0 2023-08-09T15:11:57.638 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:57.637+0000 7f5d7e7fc640 1 -- 172.21.15.43:0/1192374254 <== mon.2 v2:172.21.15.203:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5d68030840 con 0x7f5d80158050 2023-08-09T15:11:57.638 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:57.637+0000 7f5d7e7fc640 1 -- 172.21.15.43:0/1192374254 <== mon.2 v2:172.21.15.203:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5d680309a0 con 0x7f5d80158050 2023-08-09T15:11:57.639 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:57.637+0000 7f5d7e7fc640 1 -- 172.21.15.43:0/1192374254 <== mon.2 v2:172.21.15.203:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5d68030d30 con 0x7f5d80158050 2023-08-09T15:11:57.639 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:57.637+0000 7f5d7ffff640 1 -- 172.21.15.43:0/1192374254 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f5d80158050 msgr2=0x7f5d80070400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T15:11:57.640 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:57.637+0000 7f5d7ffff640 1 --2- 172.21.15.43:0/1192374254 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f5d80158050 0x7f5d80070400 secure :-1 s=READY pgs=349 cs=0 l=1 rev1=1 crypto rx=0x7f5d68009870 tx=0x7f5d6802b980 comp rx=0 tx=0).stop 2023-08-09T15:11:57.640 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:57.638+0000 7f5d7ffff640 1 -- 172.21.15.43:0/1192374254 shutdown_connections 2023-08-09T15:11:57.640 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:57.638+0000 7f5d7ffff640 1 --2- 172.21.15.43:0/1192374254 >> v2:172.21.15.203:3300/0 conn(0x7f5d8006b740 0x7f5d8015b320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T15:11:57.640 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:57.638+0000 7f5d7ffff640 1 --2- 172.21.15.43:0/1192374254 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f5d80158050 0x7f5d80070400 unknown :-1 s=CLOSED pgs=349 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T15:11:57.640 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:57.638+0000 7f5d7ffff640 1 -- 172.21.15.43:0/1192374254 >> 172.21.15.43:0/1192374254 conn(0x7f5d800ce020 msgr2=0x7f5d800ce430 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-09T15:11:57.640 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:57.638+0000 7f5d7ffff640 1 -- 172.21.15.43:0/1192374254 shutdown_connections 2023-08-09T15:11:57.640 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:57.638+0000 7f5d7ffff640 1 -- 172.21.15.43:0/1192374254 wait complete. 2023-08-09T15:11:57.641 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:57.639+0000 7f5d7ffff640 1 Processor -- start 2023-08-09T15:11:57.641 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:57.640+0000 7f5d7ffff640 1 -- start start 2023-08-09T15:11:57.642 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:57.641+0000 7f5d7ffff640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f5d8006b740 0x7f5d801520b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T15:11:57.642 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:57.641+0000 7f5d7ffff640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f5d80157c70 0x7f5d800c9aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T15:11:57.642 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:57.641+0000 7f5d7ffff640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f5d80158050 0x7f5d801f56c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T15:11:57.643 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:57.641+0000 7f5d7ffff640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5d80155050 con 0x7f5d8006b740 2023-08-09T15:11:57.643 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:57.641+0000 7f5d7ffff640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5d801f4fa0 con 0x7f5d80157c70 2023-08-09T15:11:57.643 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:57.641+0000 7f5d7ffff640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5d800af7a0 con 0x7f5d80158050 2023-08-09T15:11:57.643 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:57.641+0000 7f5d7effd640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f5d8006b740 0x7f5d801520b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T15:11:57.643 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:57.641+0000 7f5d7effd640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f5d8006b740 0x7f5d801520b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:41550/0 (socket says 172.21.15.43:41550) 2023-08-09T15:11:57.643 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:57.641+0000 7f5d7effd640 1 -- 172.21.15.43:0/3468458966 learned_addr learned my addr 172.21.15.43:0/3468458966 (peer_addr_for_me v2:172.21.15.43:0/0) 2023-08-09T15:11:57.644 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:57.641+0000 7f5d77fff640 1 --2- 172.21.15.43:0/3468458966 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f5d80157c70 0x7f5d800c9aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T15:11:57.644 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:57.641+0000 7f5d7effd640 1 -- 172.21.15.43:0/3468458966 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f5d80158050 msgr2=0x7f5d801f56c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T15:11:57.644 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:57.641+0000 7f5d7effd640 1 --2- 172.21.15.43:0/3468458966 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f5d80158050 0x7f5d801f56c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T15:11:57.645 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:57.641+0000 7f5d7effd640 1 -- 172.21.15.43:0/3468458966 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f5d80157c70 msgr2=0x7f5d800c9aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T15:11:57.645 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:57.641+0000 7f5d7effd640 1 --2- 172.21.15.43:0/3468458966 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f5d80157c70 0x7f5d800c9aa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T15:11:57.645 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:57.642+0000 7f5d7effd640 1 -- 172.21.15.43:0/3468458966 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5d680094d0 con 0x7f5d8006b740 2023-08-09T15:11:57.645 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:57.642+0000 7f5d7effd640 1 --2- 172.21.15.43:0/3468458966 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f5d8006b740 0x7f5d801520b0 secure :-1 s=READY pgs=2478 cs=0 l=1 rev1=1 crypto rx=0x7f5d7000c490 tx=0x7f5d7000c950 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-09T15:11:57.645 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:57.642+0000 7f5d7cff9640 1 -- 172.21.15.43:0/3468458966 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5d700052d0 con 0x7f5d8006b740 2023-08-09T15:11:57.645 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:57.642+0000 7f5d7cff9640 1 -- 172.21.15.43:0/3468458966 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5d70005430 con 0x7f5d8006b740 2023-08-09T15:11:57.645 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:57.642+0000 7f5d7ffff640 1 -- 172.21.15.43:0/3468458966 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5d801525f0 con 0x7f5d8006b740 2023-08-09T15:11:57.646 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:57.642+0000 7f5d7cff9640 1 -- 172.21.15.43:0/3468458966 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5d70003810 con 0x7f5d8006b740 2023-08-09T15:11:57.646 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:57.642+0000 7f5d7ffff640 1 -- 172.21.15.43:0/3468458966 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5d80154d70 con 0x7f5d8006b740 2023-08-09T15:11:57.646 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:57.643+0000 7f5d7ffff640 1 -- 172.21.15.43:0/3468458966 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5d8004e390 con 0x7f5d8006b740 2023-08-09T15:11:57.646 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:57.645+0000 7f5d7cff9640 1 -- 172.21.15.43:0/3468458966 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 123) v1 ==== 88517+0+0 (secure 0 0 0) 0x7f5d70003970 con 0x7f5d8006b740 2023-08-09T15:11:57.647 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:57.645+0000 7f5d7cff9640 1 --2- 172.21.15.43:0/3468458966 >> [v2:172.21.15.203:6832/2375771024,v1:172.21.15.203:6833/2375771024] conn(0x7f5d6006a250 0x7f5d6006c710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T15:11:57.647 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:57.646+0000 7f5d7cff9640 1 -- 172.21.15.43:0/3468458966 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(224..224 src has 1..224) v4 ==== 6770+0+0 (secure 0 0 0) 0x7f5d70088270 con 0x7f5d8006b740 2023-08-09T15:11:57.647 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:57.646+0000 7f5d77fff640 1 --2- 172.21.15.43:0/3468458966 >> [v2:172.21.15.203:6832/2375771024,v1:172.21.15.203:6833/2375771024] conn(0x7f5d6006a250 0x7f5d6006c710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T15:11:57.648 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:57.647+0000 7f5d77fff640 1 --2- 172.21.15.43:0/3468458966 >> [v2:172.21.15.203:6832/2375771024,v1:172.21.15.203:6833/2375771024] conn(0x7f5d6006a250 0x7f5d6006c710 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7f5d68036000 tx=0x7f5d6803b040 comp rx=0 tx=0).ready entity=mgr.9103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-09T15:11:57.652 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:57.650+0000 7f5d7cff9640 1 -- 172.21.15.43:0/3468458966 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178066 (secure 0 0 0) 0x7f5d70053410 con 0x7f5d8006b740 2023-08-09T15:11:57.886 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:57.885+0000 7f5d7ffff640 1 -- 172.21.15.43:0/3468458966 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7f5d800715c0 con 0x7f5d8006b740 2023-08-09T15:11:57.887 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:57.885+0000 7f5d7cff9640 1 -- 172.21.15.43:0/3468458966 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v104) v1 ==== 53+0+73 (secure 0 0 0) 0x7f5d70056a30 con 0x7f5d8006b740 2023-08-09T15:11:57.887 INFO:teuthology.orchestra.run.smithi043.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2023-08-09T15:11:57.889 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:57.888+0000 7f5d7ffff640 1 -- 172.21.15.43:0/3468458966 >> [v2:172.21.15.203:6832/2375771024,v1:172.21.15.203:6833/2375771024] conn(0x7f5d6006a250 msgr2=0x7f5d6006c710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T15:11:57.889 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:57.888+0000 7f5d7ffff640 1 --2- 172.21.15.43:0/3468458966 >> [v2:172.21.15.203:6832/2375771024,v1:172.21.15.203:6833/2375771024] conn(0x7f5d6006a250 0x7f5d6006c710 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7f5d68036000 tx=0x7f5d6803b040 comp rx=0 tx=0).stop 2023-08-09T15:11:57.889 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:57.888+0000 7f5d7ffff640 1 -- 172.21.15.43:0/3468458966 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f5d8006b740 msgr2=0x7f5d801520b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T15:11:57.890 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:57.888+0000 7f5d7ffff640 1 --2- 172.21.15.43:0/3468458966 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f5d8006b740 0x7f5d801520b0 secure :-1 s=READY pgs=2478 cs=0 l=1 rev1=1 crypto rx=0x7f5d7000c490 tx=0x7f5d7000c950 comp rx=0 tx=0).stop 2023-08-09T15:11:57.890 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:57.888+0000 7f5d7ffff640 1 -- 172.21.15.43:0/3468458966 shutdown_connections 2023-08-09T15:11:57.890 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:57.888+0000 7f5d7ffff640 1 --2- 172.21.15.43:0/3468458966 >> [v2:172.21.15.203:6832/2375771024,v1:172.21.15.203:6833/2375771024] conn(0x7f5d6006a250 0x7f5d6006c710 unknown :-1 s=CLOSED pgs=37 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T15:11:57.890 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:57.888+0000 7f5d7ffff640 1 --2- 172.21.15.43:0/3468458966 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f5d80158050 0x7f5d801f56c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T15:11:57.891 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:57.888+0000 7f5d7ffff640 1 --2- 172.21.15.43:0/3468458966 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f5d80157c70 0x7f5d800c9aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T15:11:57.891 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:57.888+0000 7f5d7ffff640 1 --2- 172.21.15.43:0/3468458966 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f5d8006b740 0x7f5d801520b0 unknown :-1 s=CLOSED pgs=2478 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T15:11:57.891 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:57.888+0000 7f5d7ffff640 1 -- 172.21.15.43:0/3468458966 >> 172.21.15.43:0/3468458966 conn(0x7f5d800ce020 msgr2=0x7f5d800ccdd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-09T15:11:57.891 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:57.888+0000 7f5d7ffff640 1 -- 172.21.15.43:0/3468458966 shutdown_connections 2023-08-09T15:11:57.891 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:11:57.888+0000 7f5d7ffff640 1 -- 172.21.15.43:0/3468458966 wait complete. 2023-08-09T15:11:57.908 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2023-08-09T15:11:57.908 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2023-08-09T15:11:57.909 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi043.front.sepia.ceph.com 2023-08-09T15:11:57.909 INFO:tasks.cephfs.mount:self.client.name = client.0 2023-08-09T15:11:57.909 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2023-08-09T15:11:57.909 INFO:tasks.cephfs.mount:self.cephfs_name = None 2023-08-09T15:11:57.910 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2023-08-09T15:11:57.910 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2023-08-09T15:11:57.910 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2023-08-09T15:11:57.910 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-09T15:11:57.911 DEBUG:teuthology.orchestra.run.smithi043:> ip addr 2023-08-09T15:11:57.967 INFO:teuthology.orchestra.run.smithi043.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2023-08-09T15:11:57.968 INFO:teuthology.orchestra.run.smithi043.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2023-08-09T15:11:57.968 INFO:teuthology.orchestra.run.smithi043.stdout: inet 127.0.0.1/8 scope host lo 2023-08-09T15:11:57.968 INFO:teuthology.orchestra.run.smithi043.stdout: valid_lft forever preferred_lft forever 2023-08-09T15:11:57.968 INFO:teuthology.orchestra.run.smithi043.stdout: inet6 ::1/128 scope host 2023-08-09T15:11:57.968 INFO:teuthology.orchestra.run.smithi043.stdout: valid_lft forever preferred_lft forever 2023-08-09T15:11:57.968 INFO:teuthology.orchestra.run.smithi043.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-08-09T15:11:57.969 INFO:teuthology.orchestra.run.smithi043.stdout: link/ether 0c:c4:7a:6b:fd:9c brd ff:ff:ff:ff:ff:ff 2023-08-09T15:11:57.969 INFO:teuthology.orchestra.run.smithi043.stdout: altname enp7s0f0 2023-08-09T15:11:57.969 INFO:teuthology.orchestra.run.smithi043.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-08-09T15:11:57.969 INFO:teuthology.orchestra.run.smithi043.stdout: link/ether 0c:c4:7a:6b:fd:9d brd ff:ff:ff:ff:ff:ff 2023-08-09T15:11:57.969 INFO:teuthology.orchestra.run.smithi043.stdout: altname enp7s0f1 2023-08-09T15:11:57.969 INFO:teuthology.orchestra.run.smithi043.stdout:4: ens1f0: mtu 1500 qdisc mq state UP group default qlen 1000 2023-08-09T15:11:57.969 INFO:teuthology.orchestra.run.smithi043.stdout: link/ether 0c:c4:7a:bd:13:58 brd ff:ff:ff:ff:ff:ff 2023-08-09T15:11:57.970 INFO:teuthology.orchestra.run.smithi043.stdout: altname enp1s0f0 2023-08-09T15:11:57.970 INFO:teuthology.orchestra.run.smithi043.stdout: inet 172.21.15.43/20 brd 172.21.15.255 scope global dynamic noprefixroute ens1f0 2023-08-09T15:11:57.970 INFO:teuthology.orchestra.run.smithi043.stdout: valid_lft 36386sec preferred_lft 36386sec 2023-08-09T15:11:57.970 INFO:teuthology.orchestra.run.smithi043.stdout: inet6 fe80::ec4:7aff:febd:1358/64 scope link 2023-08-09T15:11:57.970 INFO:teuthology.orchestra.run.smithi043.stdout: valid_lft forever preferred_lft forever 2023-08-09T15:11:57.970 INFO:teuthology.orchestra.run.smithi043.stdout:5: ens1f1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-08-09T15:11:57.970 INFO:teuthology.orchestra.run.smithi043.stdout: link/ether 0c:c4:7a:bd:13:59 brd ff:ff:ff:ff:ff:ff 2023-08-09T15:11:57.971 INFO:teuthology.orchestra.run.smithi043.stdout: altname enp1s0f1 2023-08-09T15:11:57.971 INFO:teuthology.orchestra.run.smithi043.stdout:8: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2023-08-09T15:11:57.971 INFO:teuthology.orchestra.run.smithi043.stdout: link/ether 56:43:50:19:f4:0c brd ff:ff:ff:ff:ff:ff 2023-08-09T15:11:57.971 INFO:teuthology.orchestra.run.smithi043.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2023-08-09T15:11:57.971 INFO:teuthology.orchestra.run.smithi043.stdout: valid_lft forever preferred_lft forever 2023-08-09T15:11:57.971 INFO:teuthology.orchestra.run.smithi043.stdout: inet6 fe80::e494:65ff:fe92:a182/64 scope link 2023-08-09T15:11:57.971 INFO:teuthology.orchestra.run.smithi043.stdout: valid_lft forever preferred_lft forever 2023-08-09T15:11:57.972 INFO:teuthology.orchestra.run.smithi043.stdout:9: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2023-08-09T15:11:57.972 INFO:teuthology.orchestra.run.smithi043.stdout: link/ether 7e:93:26:fa:07:8f brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.0 2023-08-09T15:11:57.972 INFO:teuthology.orchestra.run.smithi043.stdout: inet6 fe80::7c93:26ff:fefa:78f/64 scope link 2023-08-09T15:11:57.972 INFO:teuthology.orchestra.run.smithi043.stdout: valid_lft forever preferred_lft forever 2023-08-09T15:11:57.972 INFO:teuthology.orchestra.run.smithi043.stdout:10: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2023-08-09T15:11:57.972 INFO:teuthology.orchestra.run.smithi043.stdout: link/ether 56:43:50:19:f4:0c brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.2 2023-08-09T15:11:57.972 INFO:teuthology.orchestra.run.smithi043.stdout: inet6 fe80::5443:50ff:fe19:f40c/64 scope link 2023-08-09T15:11:57.973 INFO:teuthology.orchestra.run.smithi043.stdout: valid_lft forever preferred_lft forever 2023-08-09T15:11:57.973 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-09T15:11:57.974 DEBUG:teuthology.orchestra.run.smithi043:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2023-08-09T15:11:58.065 INFO:teuthology.orchestra.run.smithi043.stdout:1 2023-08-09T15:11:58.069 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-09T15:11:58.069 DEBUG:teuthology.orchestra.run.smithi043:> route 2023-08-09T15:11:58.124 INFO:teuthology.orchestra.run.smithi043.stdout:Kernel IP routing table 2023-08-09T15:11:58.124 INFO:teuthology.orchestra.run.smithi043.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2023-08-09T15:11:58.124 INFO:teuthology.orchestra.run.smithi043.stdout:default _gateway 0.0.0.0 UG 100 0 0 ens1f0 2023-08-09T15:11:58.124 INFO:teuthology.orchestra.run.smithi043.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 ens1f0 2023-08-09T15:11:58.125 INFO:teuthology.orchestra.run.smithi043.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2023-08-09T15:11:58.125 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-09T15:11:58.126 DEBUG:teuthology.orchestra.run.smithi043:> (cd / && exec bash -c ' 2023-08-09T15:11:58.126 DEBUG:teuthology.orchestra.run.smithi043:> set -e 2023-08-09T15:11:58.127 DEBUG:teuthology.orchestra.run.smithi043:> sudo iptables -A FORWARD -o ens1f0 -i ceph-brx -j ACCEPT 2023-08-09T15:11:58.127 DEBUG:teuthology.orchestra.run.smithi043:> sudo iptables -A FORWARD -i ens1f0 -o ceph-brx -j ACCEPT 2023-08-09T15:11:58.127 DEBUG:teuthology.orchestra.run.smithi043:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o ens1f0 -j MASQUERADE 2023-08-09T15:11:58.127 DEBUG:teuthology.orchestra.run.smithi043:> ') 2023-08-09T15:11:58.331 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-09T15:11:58.331 DEBUG:teuthology.orchestra.run.smithi043:> ip netns list 2023-08-09T15:11:58.387 INFO:teuthology.orchestra.run.smithi043.stdout:ceph-ns--home-ubuntu-cephtest-mnt.2 (id: 1) 2023-08-09T15:11:58.387 INFO:teuthology.orchestra.run.smithi043.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2023-08-09T15:11:58.388 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2023-08-09T15:11:58.389 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-09T15:11:58.389 DEBUG:teuthology.orchestra.run.smithi043:> sudo ip link set brx.0 up 2023-08-09T15:11:58.497 INFO:tasks.cephfs.fuse_mount:Client client.0 config is {} 2023-08-09T15:11:58.497 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-08-09T15:11:58.498 DEBUG:teuthology.orchestra.run.smithi043:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2023-08-09T15:11:58.558 INFO:teuthology.orchestra.run.smithi043.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2023-08-09T15:11:58.559 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-08-09T15:11:58.559 DEBUG:teuthology.orchestra.run.smithi043:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2023-08-09T15:11:58.616 DEBUG:teuthology.orchestra.run.smithi043:> sudo modprobe fuse 2023-08-09T15:11:58.714 INFO:teuthology.orchestra.run:Running command with timeout 30 2023-08-09T15:11:58.715 DEBUG:teuthology.orchestra.run.smithi043:> sudo mount -t fusectl /sys/fs/fuse/connections /sys/fs/fuse/connections 2023-08-09T15:11:58.810 INFO:teuthology.orchestra.run.smithi043.stderr:mount: /sys/fs/fuse/connections: /sys/fs/fuse/connections already mounted or mount point busy. 2023-08-09T15:11:58.813 DEBUG:teuthology.orchestra.run:got remote process result: 32 2023-08-09T15:11:58.814 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-09T15:11:58.815 DEBUG:teuthology.orchestra.run.smithi043:> ls /sys/fs/fuse/connections 2023-08-09T15:11:58.871 INFO:tasks.cephfs.fuse_mount:Pre-mount connections: [] 2023-08-09T15:11:58.872 DEBUG:teuthology.orchestra.run.smithi043:> (cd /home/ubuntu/cephtest && exec sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-fuse -f --admin-socket '/var/run/ceph/$cluster-$name.$pid.asok' /home/ubuntu/cephtest/mnt.0 --id 0) 2023-08-09T15:11:58.879 DEBUG:teuthology.orchestra.run.smithi043:> sudo modprobe fuse 2023-08-09T15:11:58.967 INFO:teuthology.orchestra.run:Running command with timeout 30 2023-08-09T15:11:58.967 DEBUG:teuthology.orchestra.run.smithi043:> sudo mount -t fusectl /sys/fs/fuse/connections /sys/fs/fuse/connections 2023-08-09T15:11:59.050 INFO:teuthology.orchestra.run.smithi043.stderr:mount: /sys/fs/fuse/connections: /sys/fs/fuse/connections already mounted or mount point busy. 2023-08-09T15:11:59.053 DEBUG:teuthology.orchestra.run:got remote process result: 32 2023-08-09T15:11:59.053 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-09T15:11:59.054 DEBUG:teuthology.orchestra.run.smithi043:> ls /sys/fs/fuse/connections 2023-08-09T15:11:59.094 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi043.stderr:2023-08-09T15:11:59.092+0000 7f79df493480 -1 init, newargv = 0x564eee545720 newargc=15 2023-08-09T15:11:59.094 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi043.stderr:ceph-fuse[154969]: starting ceph client 2023-08-09T15:11:59.103 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi043.stderr:ceph-fuse[154969]: starting fuse 2023-08-09T15:12:00.100 INFO:tasks.ceph.mgr.x.smithi203.stderr:2023-08-09T15:12:00.098+0000 7fb88757e640 -1 client.0 error registering admin socket command: (17) File exists 2023-08-09T15:12:00.100 INFO:tasks.ceph.mgr.x.smithi203.stderr:2023-08-09T15:12:00.098+0000 7fb88757e640 -1 client.0 error registering admin socket command: (17) File exists 2023-08-09T15:12:00.100 INFO:tasks.ceph.mgr.x.smithi203.stderr:2023-08-09T15:12:00.098+0000 7fb88757e640 -1 client.0 error registering admin socket command: (17) File exists 2023-08-09T15:12:00.100 INFO:tasks.ceph.mgr.x.smithi203.stderr:2023-08-09T15:12:00.098+0000 7fb88757e640 -1 client.0 error registering admin socket command: (17) File exists 2023-08-09T15:12:00.100 INFO:tasks.ceph.mgr.x.smithi203.stderr:2023-08-09T15:12:00.098+0000 7fb88757e640 -1 client.0 error registering admin socket command: (17) File exists 2023-08-09T15:12:00.201 DEBUG:teuthology.orchestra.run.smithi043:> sudo modprobe fuse 2023-08-09T15:12:00.316 INFO:teuthology.orchestra.run:Running command with timeout 30 2023-08-09T15:12:00.317 DEBUG:teuthology.orchestra.run.smithi043:> sudo mount -t fusectl /sys/fs/fuse/connections /sys/fs/fuse/connections 2023-08-09T15:12:00.424 INFO:teuthology.orchestra.run.smithi043.stderr:mount: /sys/fs/fuse/connections: /sys/fs/fuse/connections already mounted or mount point busy. 2023-08-09T15:12:00.428 DEBUG:teuthology.orchestra.run:got remote process result: 32 2023-08-09T15:12:00.429 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-09T15:12:00.429 DEBUG:teuthology.orchestra.run.smithi043:> ls /sys/fs/fuse/connections 2023-08-09T15:12:00.489 INFO:teuthology.orchestra.run.smithi043.stdout:40 2023-08-09T15:12:00.490 INFO:tasks.cephfs.fuse_mount:Post-mount connections: [40] 2023-08-09T15:12:00.490 DEBUG:teuthology.orchestra.run.smithi043:> sudo stdin-killer -- python3 -c ' 2023-08-09T15:12:00.490 DEBUG:teuthology.orchestra.run.smithi043:> import glob 2023-08-09T15:12:00.491 DEBUG:teuthology.orchestra.run.smithi043:> import re 2023-08-09T15:12:00.491 DEBUG:teuthology.orchestra.run.smithi043:> import os 2023-08-09T15:12:00.491 DEBUG:teuthology.orchestra.run.smithi043:> import subprocess 2023-08-09T15:12:00.492 DEBUG:teuthology.orchestra.run.smithi043:> 2023-08-09T15:12:00.492 DEBUG:teuthology.orchestra.run.smithi043:> def _find_admin_socket(client_name): 2023-08-09T15:12:00.492 DEBUG:teuthology.orchestra.run.smithi043:> asok_path = "/var/run/ceph/ceph-client.0.*.asok" 2023-08-09T15:12:00.493 DEBUG:teuthology.orchestra.run.smithi043:> files = glob.glob(asok_path) 2023-08-09T15:12:00.493 DEBUG:teuthology.orchestra.run.smithi043:> mountpoint = "/home/ubuntu/cephtest/mnt.0" 2023-08-09T15:12:00.493 DEBUG:teuthology.orchestra.run.smithi043:> 2023-08-09T15:12:00.493 DEBUG:teuthology.orchestra.run.smithi043:> # Given a non-glob path, it better be there 2023-08-09T15:12:00.494 DEBUG:teuthology.orchestra.run.smithi043:> if "*" not in asok_path: 2023-08-09T15:12:00.494 DEBUG:teuthology.orchestra.run.smithi043:> assert(len(files) == 1) 2023-08-09T15:12:00.494 DEBUG:teuthology.orchestra.run.smithi043:> return files[0] 2023-08-09T15:12:00.495 DEBUG:teuthology.orchestra.run.smithi043:> 2023-08-09T15:12:00.495 DEBUG:teuthology.orchestra.run.smithi043:> for f in files: 2023-08-09T15:12:00.495 DEBUG:teuthology.orchestra.run.smithi043:> pid = re.match(".*\.(\d+)\.asok$", f).group(1) 2023-08-09T15:12:00.495 DEBUG:teuthology.orchestra.run.smithi043:> if os.path.exists("/proc/{0}".format(pid)): 2023-08-09T15:12:00.496 DEBUG:teuthology.orchestra.run.smithi043:> with open("/proc/{0}/cmdline".format(pid), '"'"'r'"'"') as proc_f: 2023-08-09T15:12:00.496 DEBUG:teuthology.orchestra.run.smithi043:> contents = proc_f.read() 2023-08-09T15:12:00.496 DEBUG:teuthology.orchestra.run.smithi043:> if mountpoint in contents: 2023-08-09T15:12:00.497 DEBUG:teuthology.orchestra.run.smithi043:> return f 2023-08-09T15:12:00.497 DEBUG:teuthology.orchestra.run.smithi043:> raise RuntimeError("Client socket {0} not found".format(client_name)) 2023-08-09T15:12:00.497 DEBUG:teuthology.orchestra.run.smithi043:> 2023-08-09T15:12:00.497 DEBUG:teuthology.orchestra.run.smithi043:> print(_find_admin_socket("client.0")) 2023-08-09T15:12:00.498 DEBUG:teuthology.orchestra.run.smithi043:> ' 2023-08-09T15:12:00.691 INFO:teuthology.orchestra.run.smithi043.stdout:/var/run/ceph/ceph-client.0.154969.asok 2023-08-09T15:12:00.696 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:00 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2023-08-09T15:12:00.707 INFO:tasks.cephfs.fuse_mount:Found client admin socket at /var/run/ceph/ceph-client.0.154969.asok 2023-08-09T15:12:00.708 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-09T15:12:00.708 DEBUG:teuthology.orchestra.run.smithi043:> sudo ceph --admin-daemon /var/run/ceph/ceph-client.0.154969.asok status 2023-08-09T15:12:00.961 INFO:teuthology.orchestra.run.smithi043.stdout:{ 2023-08-09T15:12:00.962 INFO:teuthology.orchestra.run.smithi043.stdout: "metadata": { 2023-08-09T15:12:00.962 INFO:teuthology.orchestra.run.smithi043.stdout: "ceph_sha1": "668a86f232c9454b2ce5928d1432a29a72964d85", 2023-08-09T15:12:00.962 INFO:teuthology.orchestra.run.smithi043.stdout: "ceph_version": "ceph version 18.0.0-5389-g668a86f2 (668a86f232c9454b2ce5928d1432a29a72964d85) reef (dev)", 2023-08-09T15:12:00.962 INFO:teuthology.orchestra.run.smithi043.stdout: "entity_id": "0", 2023-08-09T15:12:00.962 INFO:teuthology.orchestra.run.smithi043.stdout: "hostname": "smithi043", 2023-08-09T15:12:00.962 INFO:teuthology.orchestra.run.smithi043.stdout: "mount_point": "/home/ubuntu/cephtest/mnt.0", 2023-08-09T15:12:00.963 INFO:teuthology.orchestra.run.smithi043.stdout: "pid": "154969", 2023-08-09T15:12:00.963 INFO:teuthology.orchestra.run.smithi043.stdout: "root": "/" 2023-08-09T15:12:00.963 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2023-08-09T15:12:00.963 INFO:teuthology.orchestra.run.smithi043.stdout: "dentry_count": 0, 2023-08-09T15:12:00.963 INFO:teuthology.orchestra.run.smithi043.stdout: "dentry_pinned_count": 0, 2023-08-09T15:12:00.963 INFO:teuthology.orchestra.run.smithi043.stdout: "id": 11670, 2023-08-09T15:12:00.963 INFO:teuthology.orchestra.run.smithi043.stdout: "inst": { 2023-08-09T15:12:00.964 INFO:teuthology.orchestra.run.smithi043.stdout: "name": { 2023-08-09T15:12:00.964 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "client", 2023-08-09T15:12:00.964 INFO:teuthology.orchestra.run.smithi043.stdout: "num": 11670 2023-08-09T15:12:00.964 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2023-08-09T15:12:00.964 INFO:teuthology.orchestra.run.smithi043.stdout: "addr": { 2023-08-09T15:12:00.964 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "v1", 2023-08-09T15:12:00.964 INFO:teuthology.orchestra.run.smithi043.stdout: "addr": "192.168.0.1:0", 2023-08-09T15:12:00.964 INFO:teuthology.orchestra.run.smithi043.stdout: "nonce": 1468181113 2023-08-09T15:12:00.965 INFO:teuthology.orchestra.run.smithi043.stdout: } 2023-08-09T15:12:00.965 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2023-08-09T15:12:00.965 INFO:teuthology.orchestra.run.smithi043.stdout: "addr": { 2023-08-09T15:12:00.965 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "v1", 2023-08-09T15:12:00.965 INFO:teuthology.orchestra.run.smithi043.stdout: "addr": "192.168.0.1:0", 2023-08-09T15:12:00.965 INFO:teuthology.orchestra.run.smithi043.stdout: "nonce": 1468181113 2023-08-09T15:12:00.965 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2023-08-09T15:12:00.965 INFO:teuthology.orchestra.run.smithi043.stdout: "inst_str": "client.11670 192.168.0.1:0/1468181113", 2023-08-09T15:12:00.966 INFO:teuthology.orchestra.run.smithi043.stdout: "addr_str": "192.168.0.1:0/1468181113", 2023-08-09T15:12:00.966 INFO:teuthology.orchestra.run.smithi043.stdout: "inode_count": 1, 2023-08-09T15:12:00.966 INFO:teuthology.orchestra.run.smithi043.stdout: "mds_epoch": 104, 2023-08-09T15:12:00.966 INFO:teuthology.orchestra.run.smithi043.stdout: "osd_epoch": 224, 2023-08-09T15:12:00.966 INFO:teuthology.orchestra.run.smithi043.stdout: "osd_epoch_barrier": 0, 2023-08-09T15:12:00.966 INFO:teuthology.orchestra.run.smithi043.stdout: "blocklisted": false, 2023-08-09T15:12:00.966 INFO:teuthology.orchestra.run.smithi043.stdout: "fs_name": "cephfs" 2023-08-09T15:12:00.966 INFO:teuthology.orchestra.run.smithi043.stdout:} 2023-08-09T15:12:00.977 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-09T15:12:00.978 DEBUG:teuthology.orchestra.run.smithi043:> stat --file-system '--printf=%T 2023-08-09T15:12:00.978 DEBUG:teuthology.orchestra.run.smithi043:> ' -- /home/ubuntu/cephtest/mnt.0 2023-08-09T15:12:01.038 INFO:teuthology.orchestra.run.smithi043.stdout:fuseblk 2023-08-09T15:12:01.038 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.0 2023-08-09T15:12:01.039 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-09T15:12:01.039 DEBUG:teuthology.orchestra.run.smithi043:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2023-08-09T15:12:01.149 INFO:tasks.cephfs.test_snap_schedules:expecting snap snap_test_dir1/.snap/scheduled-2023-08-09-15_12_00 in ~15s... 2023-08-09T15:12:01.149 DEBUG:tasks.cephfs.test_snap_schedules:expected_schedule: ['1M'], expected_retention: [{'M': 1}] 2023-08-09T15:12:01.150 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs snap-schedule list --fs cephfs --format json snap_test_dir1 2023-08-09T15:12:01.392 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:01.391+0000 7f70de64d640 1 Processor -- start 2023-08-09T15:12:01.393 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:01.391+0000 7f70de64d640 1 -- start start 2023-08-09T15:12:01.394 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:01.393+0000 7f70de64d640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f70d815a940 0x7f70d815c0a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T15:12:01.394 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:01.393+0000 7f70de64d640 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7f70d806d450 0x7f70d815b3d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T15:12:01.394 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:01.393+0000 7f70de64d640 1 -- --> v1:172.21.15.43:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f70d80cd220 con 0x7f70d8069740 2023-08-09T15:12:01.395 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:01.393+0000 7f70de64d640 1 -- --> v2:172.21.15.203:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f70d80cdaa0 con 0x7f70d806d450 2023-08-09T15:12:01.395 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:01.393+0000 7f70de64d640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f70d80cd4e0 con 0x7f70d815a940 2023-08-09T15:12:01.395 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:01.393+0000 7f70d7fff640 1 --1- >> v1:172.21.15.43:6789/0 conn(0x7f70d8069740 0x7f70d8069b20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.43:6789/0 says I am v1:172.21.15.43:34240/0 (socket says 172.21.15.43:34240) 2023-08-09T15:12:01.395 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:01.393+0000 7f70d7fff640 1 -- 172.21.15.43:0/58811982 learned_addr learned my addr 172.21.15.43:0/58811982 (peer_addr_for_me v1:172.21.15.43:0/0) 2023-08-09T15:12:01.396 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:01.393+0000 7f70d77fe640 1 -- 172.21.15.43:0/58811982 <== mon.0 v1:172.21.15.43:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1120162762 0 0) 0x7f70d80cd220 con 0x7f70d8069740 2023-08-09T15:12:01.397 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:01.394+0000 7f70d77fe640 1 -- 172.21.15.43:0/58811982 --> v1:172.21.15.43:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f70c4003520 con 0x7f70d8069740 2023-08-09T15:12:01.397 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:01.394+0000 7f70cf7fe640 1 --2- 172.21.15.43:0/58811982 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f70d815a940 0x7f70d815c0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T15:12:01.397 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:01.394+0000 7f70dcbea640 1 --2- 172.21.15.43:0/58811982 >> v2:172.21.15.203:3300/0 conn(0x7f70d806d450 0x7f70d815b3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T15:12:01.397 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:01.394+0000 7f70d77fe640 1 -- 172.21.15.43:0/58811982 <== mon.0 v1:172.21.15.43:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 365305169 0 0) 0x7f70c4003520 con 0x7f70d8069740 2023-08-09T15:12:01.397 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:01.394+0000 7f70d77fe640 1 -- 172.21.15.43:0/58811982 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f70d815a940 msgr2=0x7f70d815c0a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-09T15:12:01.397 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:01.394+0000 7f70d77fe640 1 --2- 172.21.15.43:0/58811982 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f70d815a940 0x7f70d815c0a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T15:12:01.398 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:01.394+0000 7f70d77fe640 1 -- 172.21.15.43:0/58811982 >> v2:172.21.15.203:3300/0 conn(0x7f70d806d450 msgr2=0x7f70d815b3d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-09T15:12:01.398 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:01.394+0000 7f70d77fe640 1 --2- 172.21.15.43:0/58811982 >> v2:172.21.15.203:3300/0 conn(0x7f70d806d450 0x7f70d815b3d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T15:12:01.398 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:01.394+0000 7f70d77fe640 1 -- 172.21.15.43:0/58811982 --> v1:172.21.15.43:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f70d80cd7c0 con 0x7f70d8069740 2023-08-09T15:12:01.398 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:01.394+0000 7f70d77fe640 1 -- 172.21.15.43:0/58811982 <== mon.0 v1:172.21.15.43:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1285150740 0 0) 0x7f70c8001670 con 0x7f70d8069740 2023-08-09T15:12:01.398 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:01.394+0000 7f70d77fe640 1 -- 172.21.15.43:0/58811982 >> v1:172.21.15.43:6789/0 conn(0x7f70d8069740 legacy=0x7f70d8069b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T15:12:01.398 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:01.394+0000 7f70d77fe640 1 --2- 172.21.15.43:0/58811982 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f70c4003970 0x7f70c4003d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T15:12:01.398 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:01.394+0000 7f70d77fe640 1 --2- 172.21.15.43:0/58811982 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f70d815a940 0x7f70c40047c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T15:12:01.398 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:01.394+0000 7f70d77fe640 1 --2- 172.21.15.43:0/58811982 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f70c4004d00 0x7f70c4005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T15:12:01.399 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:01.394+0000 7f70d77fe640 1 -- 172.21.15.43:0/58811982 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f70d80cdaa0 con 0x7f70c4003970 2023-08-09T15:12:01.399 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:01.394+0000 7f70d77fe640 1 -- 172.21.15.43:0/58811982 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f70d80cd4e0 con 0x7f70c4004d00 2023-08-09T15:12:01.399 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:01.394+0000 7f70d77fe640 1 -- 172.21.15.43:0/58811982 --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f70c400bae0 con 0x7f70d815a940 2023-08-09T15:12:01.399 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:01.394+0000 7f70d7fff640 1 --2- 172.21.15.43:0/58811982 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f70c4003970 0x7f70c4003d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T15:12:01.399 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:01.395+0000 7f70d7fff640 1 -- 172.21.15.43:0/58811982 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f70d815a940 msgr2=0x7f70c40047c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-09T15:12:01.399 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:01.395+0000 7f70d7fff640 1 --2- 172.21.15.43:0/58811982 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f70d815a940 0x7f70c40047c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T15:12:01.399 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:01.395+0000 7f70dcbea640 1 --2- 172.21.15.43:0/58811982 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f70c4004d00 0x7f70c4005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T15:12:01.399 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:01.395+0000 7f70d7fff640 1 -- 172.21.15.43:0/58811982 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f70c4004d00 msgr2=0x7f70c4005180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-09T15:12:01.400 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:01.395+0000 7f70d7fff640 1 --2- 172.21.15.43:0/58811982 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f70c4004d00 0x7f70c4005180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T15:12:01.400 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:01.395+0000 7f70d7fff640 1 -- 172.21.15.43:0/58811982 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f70c4003520 con 0x7f70c4003970 2023-08-09T15:12:01.401 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:01.396+0000 7f70d7fff640 1 --2- 172.21.15.43:0/58811982 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f70c4003970 0x7f70c4003d90 secure :-1 s=READY pgs=2483 cs=0 l=1 rev1=1 crypto rx=0x7f70c800c240 tx=0x7f70c40057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a523c3bb0981fba3 server_cookie=0 in_seq=0 out_seq=0 2023-08-09T15:12:01.401 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:01.396+0000 7f70d77fe640 1 -- 172.21.15.43:0/58811982 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f70c8005ad0 con 0x7f70c4003970 2023-08-09T15:12:01.401 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:01.396+0000 7f70d77fe640 1 -- 172.21.15.43:0/58811982 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f70c8005c30 con 0x7f70c4003970 2023-08-09T15:12:01.401 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:01.396+0000 7f70d77fe640 1 -- 172.21.15.43:0/58811982 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f70c802c8b0 con 0x7f70c4003970 2023-08-09T15:12:01.401 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:01.397+0000 7f70de64d640 1 -- 172.21.15.43:0/58811982 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f70c4003970 msgr2=0x7f70c4003d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T15:12:01.401 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:01.397+0000 7f70de64d640 1 --2- 172.21.15.43:0/58811982 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f70c4003970 0x7f70c4003d90 secure :-1 s=READY pgs=2483 cs=0 l=1 rev1=1 crypto rx=0x7f70c800c240 tx=0x7f70c40057f0 comp rx=0 tx=0).stop 2023-08-09T15:12:01.402 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:01.397+0000 7f70dcbea640 1 -- 172.21.15.43:0/58811982 reap_dead start 2023-08-09T15:12:01.402 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:01.397+0000 7f70de64d640 1 -- 172.21.15.43:0/58811982 shutdown_connections 2023-08-09T15:12:01.402 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:01.397+0000 7f70de64d640 1 -- 172.21.15.43:0/58811982 >> 172.21.15.43:0/58811982 conn(0x7f70d80b0ed0 msgr2=0x7f70d80b12e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-09T15:12:01.402 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:01.397+0000 7f70de64d640 1 -- 172.21.15.43:0/58811982 shutdown_connections 2023-08-09T15:12:01.402 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:01.397+0000 7f70de64d640 1 -- 172.21.15.43:0/58811982 wait complete. 2023-08-09T15:12:01.402 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:01.400+0000 7f70de64d640 1 Processor -- start 2023-08-09T15:12:01.404 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:01.400+0000 7f70de64d640 1 -- start start 2023-08-09T15:12:01.404 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:01.401+0000 7f70de64d640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f70c4003970 0x7f70d814a720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T15:12:01.404 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:01.401+0000 7f70de64d640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f70d80af620 0x7f70d815ce40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T15:12:01.404 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:01.401+0000 7f70de64d640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f70d81f5370 0x7f70d81f57f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T15:12:01.404 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:01.401+0000 7f70de64d640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f70d814e4a0 con 0x7f70c4003970 2023-08-09T15:12:01.404 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:01.401+0000 7f70de64d640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f70d814e6f0 con 0x7f70d80af620 2023-08-09T15:12:01.404 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:01.401+0000 7f70de64d640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f70d814e9d0 con 0x7f70d81f5370 2023-08-09T15:12:01.405 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:01.401+0000 7f70d7fff640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f70c4003970 0x7f70d814a720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T15:12:01.405 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:01.402+0000 7f70d7fff640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f70c4003970 0x7f70d814a720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:41560/0 (socket says 172.21.15.43:41560) 2023-08-09T15:12:01.405 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:01.402+0000 7f70d7fff640 1 -- 172.21.15.43:0/1686138255 learned_addr learned my addr 172.21.15.43:0/1686138255 (peer_addr_for_me v2:172.21.15.43:0/0) 2023-08-09T15:12:01.405 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:01.402+0000 7f70dcbea640 1 --2- 172.21.15.43:0/1686138255 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f70d81f5370 0x7f70d81f57f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T15:12:01.405 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:01.402+0000 7f70cf7fe640 1 --2- 172.21.15.43:0/1686138255 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f70d80af620 0x7f70d815ce40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T15:12:01.405 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:01.402+0000 7f70d7fff640 1 -- 172.21.15.43:0/1686138255 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f70d81f5370 msgr2=0x7f70d81f57f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T15:12:01.405 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:01.402+0000 7f70d7fff640 1 --2- 172.21.15.43:0/1686138255 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f70d81f5370 0x7f70d81f57f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T15:12:01.405 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:01.402+0000 7f70d7fff640 1 -- 172.21.15.43:0/1686138255 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f70d80af620 msgr2=0x7f70d815ce40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T15:12:01.406 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:01.402+0000 7f70d7fff640 1 --2- 172.21.15.43:0/1686138255 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f70d80af620 0x7f70d815ce40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T15:12:01.406 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:01.402+0000 7f70d7fff640 1 -- 172.21.15.43:0/1686138255 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f70d814ecb0 con 0x7f70c4003970 2023-08-09T15:12:01.406 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:01.402+0000 7f70d7fff640 1 --2- 172.21.15.43:0/1686138255 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f70c4003970 0x7f70d814a720 secure :-1 s=READY pgs=2484 cs=0 l=1 rev1=1 crypto rx=0x7f70c802c450 tx=0x7f70c8001a00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-09T15:12:01.407 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:01.403+0000 7f70d57fa640 1 -- 172.21.15.43:0/1686138255 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f70c802ccc0 con 0x7f70c4003970 2023-08-09T15:12:01.407 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:01.403+0000 7f70d57fa640 1 -- 172.21.15.43:0/1686138255 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f70c802ce20 con 0x7f70c4003970 2023-08-09T15:12:01.407 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:01.403+0000 7f70d57fa640 1 -- 172.21.15.43:0/1686138255 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f70c8007270 con 0x7f70c4003970 2023-08-09T15:12:01.407 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:01.403+0000 7f70de64d640 1 -- 172.21.15.43:0/1686138255 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f70d814ef90 con 0x7f70c4003970 2023-08-09T15:12:01.407 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:01.403+0000 7f70de64d640 1 -- 172.21.15.43:0/1686138255 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f70d814f790 con 0x7f70c4003970 2023-08-09T15:12:01.407 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:01.404+0000 7f70de64d640 1 -- 172.21.15.43:0/1686138255 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f70d804db40 con 0x7f70c4003970 2023-08-09T15:12:01.408 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:01.405+0000 7f70d57fa640 1 -- 172.21.15.43:0/1686138255 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 124) v1 ==== 88574+0+0 (secure 0 0 0) 0x7f70c8032070 con 0x7f70c4003970 2023-08-09T15:12:01.408 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:01.407+0000 7f70d57fa640 1 --2- 172.21.15.43:0/1686138255 >> [v2:172.21.15.203:6832/2375771024,v1:172.21.15.203:6833/2375771024] conn(0x7f70b806a310 0x7f70b806c7d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T15:12:01.408 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:01.407+0000 7f70d57fa640 1 -- 172.21.15.43:0/1686138255 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(224..224 src has 1..224) v4 ==== 6770+0+0 (secure 0 0 0) 0x7f70c80aa360 con 0x7f70c4003970 2023-08-09T15:12:01.409 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:01.407+0000 7f70cf7fe640 1 --2- 172.21.15.43:0/1686138255 >> [v2:172.21.15.203:6832/2375771024,v1:172.21.15.203:6833/2375771024] conn(0x7f70b806a310 0x7f70b806c7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T15:12:01.410 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:01.409+0000 7f70cf7fe640 1 --2- 172.21.15.43:0/1686138255 >> [v2:172.21.15.203:6832/2375771024,v1:172.21.15.203:6833/2375771024] conn(0x7f70b806a310 0x7f70b806c7d0 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7f70c0006a90 tx=0x7f70c0008040 comp rx=0 tx=0).ready entity=mgr.9103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-09T15:12:01.413 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:01.412+0000 7f70d57fa640 1 -- 172.21.15.43:0/1686138255 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178066 (secure 0 0 0) 0x7f70c8075410 con 0x7f70c4003970 2023-08-09T15:12:01.629 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:01.628+0000 7f70de64d640 1 -- 172.21.15.43:0/1686138255 --> [v2:172.21.15.203:6832/2375771024,v1:172.21.15.203:6833/2375771024] -- mgr_command(tid 0: {"prefix": "fs snap-schedule list", "fs": "cephfs", "path": "snap_test_dir1", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f70d815aeb0 con 0x7f70b806a310 2023-08-09T15:12:01.646 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:01.645+0000 7f70d57fa640 1 -- 172.21.15.43:0/1686138255 <== mgr.9103 v2:172.21.15.203:6832/2375771024 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+72 (secure 0 0 0) 0x7f70c000f8f0 con 0x7f70b806a310 2023-08-09T15:12:01.647 INFO:teuthology.orchestra.run.smithi043.stdout: 2023-08-09T15:12:01.647 INFO:teuthology.orchestra.run.smithi043.stdout:{"path": "snap_test_dir1", "schedule": ["1M"], "retention": [{"M": 1}]} 2023-08-09T15:12:01.649 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:01.647+0000 7f70de64d640 1 -- 172.21.15.43:0/1686138255 >> [v2:172.21.15.203:6832/2375771024,v1:172.21.15.203:6833/2375771024] conn(0x7f70b806a310 msgr2=0x7f70b806c7d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T15:12:01.649 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:01.647+0000 7f70de64d640 1 --2- 172.21.15.43:0/1686138255 >> [v2:172.21.15.203:6832/2375771024,v1:172.21.15.203:6833/2375771024] conn(0x7f70b806a310 0x7f70b806c7d0 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7f70c0006a90 tx=0x7f70c0008040 comp rx=0 tx=0).stop 2023-08-09T15:12:01.649 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:01.647+0000 7f70de64d640 1 -- 172.21.15.43:0/1686138255 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f70c4003970 msgr2=0x7f70d814a720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T15:12:01.649 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:01.647+0000 7f70de64d640 1 --2- 172.21.15.43:0/1686138255 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f70c4003970 0x7f70d814a720 secure :-1 s=READY pgs=2484 cs=0 l=1 rev1=1 crypto rx=0x7f70c802c450 tx=0x7f70c8001a00 comp rx=0 tx=0).stop 2023-08-09T15:12:01.650 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:01.648+0000 7f70de64d640 1 -- 172.21.15.43:0/1686138255 shutdown_connections 2023-08-09T15:12:01.650 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:01.648+0000 7f70de64d640 1 --2- 172.21.15.43:0/1686138255 >> [v2:172.21.15.203:6832/2375771024,v1:172.21.15.203:6833/2375771024] conn(0x7f70b806a310 0x7f70b806c7d0 unknown :-1 s=CLOSED pgs=38 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T15:12:01.650 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:01.648+0000 7f70de64d640 1 --2- 172.21.15.43:0/1686138255 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f70d81f5370 0x7f70d81f57f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T15:12:01.650 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:01.648+0000 7f70de64d640 1 --2- 172.21.15.43:0/1686138255 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f70d80af620 0x7f70d815ce40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T15:12:01.650 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:01.648+0000 7f70de64d640 1 --2- 172.21.15.43:0/1686138255 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f70c4003970 0x7f70d814a720 unknown :-1 s=CLOSED pgs=2484 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T15:12:01.650 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:01.648+0000 7f70de64d640 1 -- 172.21.15.43:0/1686138255 >> 172.21.15.43:0/1686138255 conn(0x7f70d80b0ed0 msgr2=0x7f70d815b2d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-09T15:12:01.650 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:01.648+0000 7f70de64d640 1 -- 172.21.15.43:0/1686138255 shutdown_connections 2023-08-09T15:12:01.651 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:01.648+0000 7f70de64d640 1 -- 172.21.15.43:0/1686138255 wait complete. 2023-08-09T15:12:01.667 DEBUG:tasks.cephfs.test_snap_schedules:res= {"path": "snap_test_dir1", "schedule": ["1M"], "retention": [{"M": 1}]} 2023-08-09T15:12:01.667 DEBUG:tasks.cephfs.test_snap_schedules:json_res: {'path': 'snap_test_dir1', 'schedule': ['1M'], 'retention': [{'M': 1}]} 2023-08-09T15:12:01.668 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs snap-schedule remove --fs cephfs snap_test_dir1 2023-08-09T15:12:01.907 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:01.906+0000 7f95f12e0640 1 Processor -- start 2023-08-09T15:12:01.908 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:01.906+0000 7f95f12e0640 1 -- start start 2023-08-09T15:12:01.909 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:01.907+0000 7f95f12e0640 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7f95ec0b5660 0x7f95ec0b3f80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T15:12:01.909 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:01.907+0000 7f95f12e0640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f95ec0b44c0 0x7f95ec0cdf00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T15:12:01.909 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:01.907+0000 7f95f12e0640 1 -- --> v1:172.21.15.203:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f95ec15ffc0 con 0x7f95ec15fbe0 2023-08-09T15:12:01.909 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:01.907+0000 7f95f12e0640 1 -- --> v2:172.21.15.43:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f95ec0d1f50 con 0x7f95ec0b44c0 2023-08-09T15:12:01.909 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:01.907+0000 7f95f12e0640 1 -- --> v2:172.21.15.203:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f95ec0b5ac0 con 0x7f95ec0b5660 2023-08-09T15:12:01.910 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:01.908+0000 7f95f0adf640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f95ec0b44c0 0x7f95ec0cdf00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T15:12:01.910 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:01.908+0000 7f95f0adf640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f95ec0b44c0 0x7f95ec0cdf00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:41572/0 (socket says 172.21.15.43:41572) 2023-08-09T15:12:01.910 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:01.908+0000 7f95f0adf640 1 -- 172.21.15.43:0/3060395749 learned_addr learned my addr 172.21.15.43:0/3060395749 (peer_addr_for_me v2:172.21.15.43:0/0) 2023-08-09T15:12:01.911 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:01.908+0000 7f95eb7fe640 1 --2- 172.21.15.43:0/3060395749 >> v2:172.21.15.203:3300/0 conn(0x7f95ec0b5660 0x7f95ec0b3f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T15:12:01.911 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:01.908+0000 7f95f0adf640 1 -- 172.21.15.43:0/3060395749 >> v2:172.21.15.203:3300/0 conn(0x7f95ec0b5660 msgr2=0x7f95ec0b3f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-09T15:12:01.911 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:01.908+0000 7f95f0adf640 1 --2- 172.21.15.43:0/3060395749 >> v2:172.21.15.203:3300/0 conn(0x7f95ec0b5660 0x7f95ec0b3f80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T15:12:01.911 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:01.908+0000 7f95f0adf640 1 -- 172.21.15.43:0/3060395749 >> v1:172.21.15.203:6789/0 conn(0x7f95ec15fbe0 legacy=0x7f95ec160be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-09T15:12:01.911 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:01.908+0000 7f95f0adf640 1 -- 172.21.15.43:0/3060395749 --> v2:172.21.15.43:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f95ec0cd460 con 0x7f95ec0b44c0 2023-08-09T15:12:01.912 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:01.910+0000 7f95f0adf640 1 --2- 172.21.15.43:0/3060395749 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f95ec0b44c0 0x7f95ec0cdf00 secure :-1 s=READY pgs=2485 cs=0 l=1 rev1=1 crypto rx=0x7f95ec071e50 tx=0x7f95e402b910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2ef8304594723811 server_cookie=0 in_seq=0 out_seq=0 2023-08-09T15:12:01.912 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:01.910+0000 7f95eaffd640 1 -- 172.21.15.43:0/3060395749 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f95e4031870 con 0x7f95ec0b44c0 2023-08-09T15:12:01.912 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:01.910+0000 7f95eaffd640 1 -- 172.21.15.43:0/3060395749 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f95e40319d0 con 0x7f95ec0b44c0 2023-08-09T15:12:01.913 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:01.910+0000 7f95eaffd640 1 -- 172.21.15.43:0/3060395749 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f95e4031d60 con 0x7f95ec0b44c0 2023-08-09T15:12:01.913 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:01.911+0000 7f95f12e0640 1 -- 172.21.15.43:0/3060395749 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f95ec0b44c0 msgr2=0x7f95ec0cdf00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T15:12:01.913 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:01.911+0000 7f95f12e0640 1 --2- 172.21.15.43:0/3060395749 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f95ec0b44c0 0x7f95ec0cdf00 secure :-1 s=READY pgs=2485 cs=0 l=1 rev1=1 crypto rx=0x7f95ec071e50 tx=0x7f95e402b910 comp rx=0 tx=0).stop 2023-08-09T15:12:01.913 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:01.911+0000 7f95f12e0640 1 -- 172.21.15.43:0/3060395749 shutdown_connections 2023-08-09T15:12:01.913 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:01.911+0000 7f95f12e0640 1 --2- 172.21.15.43:0/3060395749 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f95ec0b44c0 0x7f95ec0cdf00 unknown :-1 s=CLOSED pgs=2485 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T15:12:01.913 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:01.911+0000 7f95f12e0640 1 --2- 172.21.15.43:0/3060395749 >> v2:172.21.15.203:3300/0 conn(0x7f95ec0b5660 0x7f95ec0b3f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T15:12:01.913 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:01.911+0000 7f95f12e0640 1 -- 172.21.15.43:0/3060395749 >> 172.21.15.43:0/3060395749 conn(0x7f95ec0714a0 msgr2=0x7f95ec0d2500 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-09T15:12:01.914 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:01.911+0000 7f95f12e0640 1 -- 172.21.15.43:0/3060395749 shutdown_connections 2023-08-09T15:12:01.914 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:01.911+0000 7f95f12e0640 1 -- 172.21.15.43:0/3060395749 wait complete. 2023-08-09T15:12:01.914 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:01.913+0000 7f95f12e0640 1 Processor -- start 2023-08-09T15:12:01.914 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:01.913+0000 7f95f12e0640 1 -- start start 2023-08-09T15:12:01.915 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:01.914+0000 7f95f12e0640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f95ec0b44c0 0x7f95ec150070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T15:12:01.915 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:01.914+0000 7f95f12e0640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f95ec0b5660 0x7f95ec0c6cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T15:12:01.916 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:01.914+0000 7f95f12e0640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f95ec15fbe0 0x7f95ec159c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T15:12:01.916 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:01.914+0000 7f95f12e0640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f95ec0cc560 con 0x7f95ec0b44c0 2023-08-09T15:12:01.916 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:01.914+0000 7f95f12e0640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f95ec0c8590 con 0x7f95ec15fbe0 2023-08-09T15:12:01.916 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:01.914+0000 7f95f12e0640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f95ec0cbfa0 con 0x7f95ec0b5660 2023-08-09T15:12:01.917 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:01.915+0000 7f95ebfff640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f95ec0b44c0 0x7f95ec150070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T15:12:01.917 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:01.915+0000 7f95ebfff640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f95ec0b44c0 0x7f95ec150070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:41576/0 (socket says 172.21.15.43:41576) 2023-08-09T15:12:01.917 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:01.915+0000 7f95ebfff640 1 -- 172.21.15.43:0/2835834944 learned_addr learned my addr 172.21.15.43:0/2835834944 (peer_addr_for_me v2:172.21.15.43:0/0) 2023-08-09T15:12:01.917 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:01.915+0000 7f95f0adf640 1 --2- 172.21.15.43:0/2835834944 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f95ec15fbe0 0x7f95ec159c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T15:12:01.917 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:01.915+0000 7f95eb7fe640 1 --2- 172.21.15.43:0/2835834944 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f95ec0b5660 0x7f95ec0c6cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T15:12:01.918 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:01.915+0000 7f95ebfff640 1 -- 172.21.15.43:0/2835834944 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f95ec0b5660 msgr2=0x7f95ec0c6cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T15:12:01.918 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:01.915+0000 7f95ebfff640 1 --2- 172.21.15.43:0/2835834944 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f95ec0b5660 0x7f95ec0c6cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T15:12:01.918 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:01.915+0000 7f95ebfff640 1 -- 172.21.15.43:0/2835834944 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f95ec15fbe0 msgr2=0x7f95ec159c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T15:12:01.918 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:01.915+0000 7f95ebfff640 1 --2- 172.21.15.43:0/2835834944 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f95ec15fbe0 0x7f95ec159c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T15:12:01.918 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:01.915+0000 7f95ebfff640 1 -- 172.21.15.43:0/2835834944 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f95e40094d0 con 0x7f95ec0b44c0 2023-08-09T15:12:01.919 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:01.916+0000 7f95ebfff640 1 --2- 172.21.15.43:0/2835834944 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f95ec0b44c0 0x7f95ec150070 secure :-1 s=READY pgs=2486 cs=0 l=1 rev1=1 crypto rx=0x7f95dc00ba50 tx=0x7f95dc00bf10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-09T15:12:01.919 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:01.916+0000 7f95e8ff9640 1 -- 172.21.15.43:0/2835834944 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f95dc00cbe0 con 0x7f95ec0b44c0 2023-08-09T15:12:01.919 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:01.916+0000 7f95e8ff9640 1 -- 172.21.15.43:0/2835834944 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f95dc00cd80 con 0x7f95ec0b44c0 2023-08-09T15:12:01.919 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:01.916+0000 7f95f12e0640 1 -- 172.21.15.43:0/2835834944 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f95ec0cc280 con 0x7f95ec0b44c0 2023-08-09T15:12:01.919 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:01.916+0000 7f95e8ff9640 1 -- 172.21.15.43:0/2835834944 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f95dc014680 con 0x7f95ec0b44c0 2023-08-09T15:12:01.920 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:01.916+0000 7f95f12e0640 1 -- 172.21.15.43:0/2835834944 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f95ec0cbc50 con 0x7f95ec0b44c0 2023-08-09T15:12:01.920 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:01.917+0000 7f95f12e0640 1 -- 172.21.15.43:0/2835834944 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f95ec04e390 con 0x7f95ec0b44c0 2023-08-09T15:12:01.920 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:01.919+0000 7f95e8ff9640 1 -- 172.21.15.43:0/2835834944 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 124) v1 ==== 88574+0+0 (secure 0 0 0) 0x7f95dc0147e0 con 0x7f95ec0b44c0 2023-08-09T15:12:01.921 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:01.920+0000 7f95e8ff9640 1 --2- 172.21.15.43:0/2835834944 >> [v2:172.21.15.203:6832/2375771024,v1:172.21.15.203:6833/2375771024] conn(0x7f95cc06a2e0 0x7f95cc06c7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T15:12:01.922 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:01.921+0000 7f95e8ff9640 1 -- 172.21.15.43:0/2835834944 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(224..224 src has 1..224) v4 ==== 6770+0+0 (secure 0 0 0) 0x7f95dc088c10 con 0x7f95ec0b44c0 2023-08-09T15:12:01.922 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:01.921+0000 7f95eb7fe640 1 --2- 172.21.15.43:0/2835834944 >> [v2:172.21.15.203:6832/2375771024,v1:172.21.15.203:6833/2375771024] conn(0x7f95cc06a2e0 0x7f95cc06c7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T15:12:01.923 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:01.922+0000 7f95eb7fe640 1 --2- 172.21.15.43:0/2835834944 >> [v2:172.21.15.203:6832/2375771024,v1:172.21.15.203:6833/2375771024] conn(0x7f95cc06a2e0 0x7f95cc06c7a0 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7f95ec1f5b00 tx=0x7f95d4008040 comp rx=0 tx=0).ready entity=mgr.9103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-09T15:12:01.926 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:01.925+0000 7f95e8ff9640 1 -- 172.21.15.43:0/2835834944 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178066 (secure 0 0 0) 0x7f95dc053d00 con 0x7f95ec0b44c0 2023-08-09T15:12:02.146 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:02.144+0000 7f95f12e0640 1 -- 172.21.15.43:0/2835834944 --> [v2:172.21.15.203:6832/2375771024,v1:172.21.15.203:6833/2375771024] -- mgr_command(tid 0: {"prefix": "fs snap-schedule remove", "fs": "cephfs", "path": "snap_test_dir1", "target": ["mon-mgr", ""]}) v1 -- 0x7f95ec0d10f0 con 0x7f95cc06a2e0 2023-08-09T15:12:02.307 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:02.305+0000 7f95e8ff9640 1 -- 172.21.15.43:0/2835834944 <== mgr.9103 v2:172.21.15.203:6832/2375771024 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+41 (secure 0 0 0) 0x7f95d40076f0 con 0x7f95cc06a2e0 2023-08-09T15:12:02.307 INFO:teuthology.orchestra.run.smithi043.stdout:Schedule removed for path snap_test_dir1 2023-08-09T15:12:02.310 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:02.308+0000 7f95f12e0640 1 -- 172.21.15.43:0/2835834944 >> [v2:172.21.15.203:6832/2375771024,v1:172.21.15.203:6833/2375771024] conn(0x7f95cc06a2e0 msgr2=0x7f95cc06c7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T15:12:02.311 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:02.308+0000 7f95f12e0640 1 --2- 172.21.15.43:0/2835834944 >> [v2:172.21.15.203:6832/2375771024,v1:172.21.15.203:6833/2375771024] conn(0x7f95cc06a2e0 0x7f95cc06c7a0 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7f95ec1f5b00 tx=0x7f95d4008040 comp rx=0 tx=0).stop 2023-08-09T15:12:02.311 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:02.308+0000 7f95f12e0640 1 -- 172.21.15.43:0/2835834944 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f95ec0b44c0 msgr2=0x7f95ec150070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T15:12:02.311 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:02.308+0000 7f95f12e0640 1 --2- 172.21.15.43:0/2835834944 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f95ec0b44c0 0x7f95ec150070 secure :-1 s=READY pgs=2486 cs=0 l=1 rev1=1 crypto rx=0x7f95dc00ba50 tx=0x7f95dc00bf10 comp rx=0 tx=0).stop 2023-08-09T15:12:02.311 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:02.309+0000 7f95f12e0640 1 -- 172.21.15.43:0/2835834944 shutdown_connections 2023-08-09T15:12:02.311 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:02.309+0000 7f95f12e0640 1 --2- 172.21.15.43:0/2835834944 >> [v2:172.21.15.203:6832/2375771024,v1:172.21.15.203:6833/2375771024] conn(0x7f95cc06a2e0 0x7f95cc06c7a0 unknown :-1 s=CLOSED pgs=39 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T15:12:02.311 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:02.309+0000 7f95f12e0640 1 --2- 172.21.15.43:0/2835834944 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f95ec15fbe0 0x7f95ec159c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T15:12:02.311 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:02.309+0000 7f95f12e0640 1 --2- 172.21.15.43:0/2835834944 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f95ec0b5660 0x7f95ec0c6cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T15:12:02.311 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:02.309+0000 7f95f12e0640 1 --2- 172.21.15.43:0/2835834944 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f95ec0b44c0 0x7f95ec150070 unknown :-1 s=CLOSED pgs=2486 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T15:12:02.312 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:02.309+0000 7f95f12e0640 1 -- 172.21.15.43:0/2835834944 >> 172.21.15.43:0/2835834944 conn(0x7f95ec0714a0 msgr2=0x7f95ec0cf3b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-09T15:12:02.312 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:02.309+0000 7f95f12e0640 1 -- 172.21.15.43:0/2835834944 shutdown_connections 2023-08-09T15:12:02.312 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:02.309+0000 7f95f12e0640 1 -- 172.21.15.43:0/2835834944 wait complete. 2023-08-09T15:12:02.332 DEBUG:tasks.cephfs.test_snap_schedules:res=Schedule removed for path snap_test_dir1 2023-08-09T15:12:02.333 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-09T15:12:02.333 DEBUG:teuthology.orchestra.run.smithi043:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-08-09T15:12:02.354 DEBUG:teuthology.orchestra.run.smithi043:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-08-09T15:12:02.388 DEBUG:teuthology.orchestra.run.smithi203:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-08-09T15:12:02.399 INFO:teuthology.orchestra.run.smithi043.stdout:scheduled-2023-08-09-15_12_00_UTC 2023-08-09T15:12:02.400 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/.snap/scheduled-2023-08-09-15_12_00_UTC 2023-08-09T15:12:02.401 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-09T15:12:02.401 DEBUG:teuthology.orchestra.run.smithi043:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/.snap/scheduled-2023-08-09-15_12_00_UTC) 2023-08-09T15:12:02.504 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-09T15:12:02.505 DEBUG:teuthology.orchestra.run.smithi043:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1) 2023-08-09T15:12:02.564 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph mgr module disable snap_schedule 2023-08-09T15:12:02.848 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:02.846+0000 7fc7688ee640 1 Processor -- start 2023-08-09T15:12:02.849 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:02.847+0000 7fc7688ee640 1 -- start start 2023-08-09T15:12:02.850 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:02.848+0000 7fc7688ee640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7fc76415fe60 0x7fc7640cd270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T15:12:02.850 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:02.848+0000 7fc7688ee640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fc7640cf510 0x7fc76406b7f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T15:12:02.850 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:02.848+0000 7fc7688ee640 1 -- --> v1:172.21.15.43:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc76406a7e0 con 0x7fc7640d14c0 2023-08-09T15:12:02.850 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:02.848+0000 7fc7688ee640 1 -- --> v2:172.21.15.43:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc76406b360 con 0x7fc76415fe60 2023-08-09T15:12:02.850 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:02.848+0000 7fc7688ee640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc76406aa30 con 0x7fc7640cf510 2023-08-09T15:12:02.851 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:02.849+0000 7fc76259c640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7fc76415fe60 0x7fc7640cd270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T15:12:02.851 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:02.849+0000 7fc761d9b640 1 --1- >> v1:172.21.15.43:6789/0 conn(0x7fc7640d14c0 0x7fc7640cee00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.43:6789/0 says I am v1:172.21.15.43:34252/0 (socket says 172.21.15.43:34252) 2023-08-09T15:12:02.851 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:02.849+0000 7fc761d9b640 1 -- 172.21.15.43:0/1228418675 learned_addr learned my addr 172.21.15.43:0/1228418675 (peer_addr_for_me v1:172.21.15.43:0/0) 2023-08-09T15:12:02.851 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:02.849+0000 7fc762d9d640 1 --2- 172.21.15.43:0/1228418675 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fc7640cf510 0x7fc76406b7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T15:12:02.852 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:02.849+0000 7fc76259c640 1 -- 172.21.15.43:0/1228418675 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fc7640cf510 msgr2=0x7fc76406b7f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-09T15:12:02.852 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:02.849+0000 7fc76259c640 1 --2- 172.21.15.43:0/1228418675 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fc7640cf510 0x7fc76406b7f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T15:12:02.852 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:02.849+0000 7fc76259c640 1 -- 172.21.15.43:0/1228418675 >> v1:172.21.15.43:6789/0 conn(0x7fc7640d14c0 legacy=0x7fc7640cee00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T15:12:02.852 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:02.849+0000 7fc76259c640 1 -- 172.21.15.43:0/1228418675 --> v2:172.21.15.43:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc76406ad10 con 0x7fc76415fe60 2023-08-09T15:12:02.853 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:02.851+0000 7fc76259c640 1 --2- 172.21.15.43:0/1228418675 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc76415fe60 0x7fc7640cd270 secure :-1 s=READY pgs=2488 cs=0 l=1 rev1=1 crypto rx=0x7fc7540097d0 tx=0x7fc75402b8b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9646c51f27567f4c server_cookie=0 in_seq=0 out_seq=0 2023-08-09T15:12:02.853 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:02.851+0000 7fc76159a640 1 -- 172.21.15.43:0/1228418675 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc75402d040 con 0x7fc76415fe60 2023-08-09T15:12:02.853 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:02.851+0000 7fc76159a640 1 -- 172.21.15.43:0/1228418675 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc754031810 con 0x7fc76415fe60 2023-08-09T15:12:02.853 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:02.851+0000 7fc76159a640 1 -- 172.21.15.43:0/1228418675 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc754031ba0 con 0x7fc76415fe60 2023-08-09T15:12:02.854 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:02.852+0000 7fc7688ee640 1 -- 172.21.15.43:0/1228418675 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc76415fe60 msgr2=0x7fc7640cd270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T15:12:02.854 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:02.852+0000 7fc7688ee640 1 --2- 172.21.15.43:0/1228418675 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc76415fe60 0x7fc7640cd270 secure :-1 s=READY pgs=2488 cs=0 l=1 rev1=1 crypto rx=0x7fc7540097d0 tx=0x7fc75402b8b0 comp rx=0 tx=0).stop 2023-08-09T15:12:02.854 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:02.852+0000 7fc7688ee640 1 -- 172.21.15.43:0/1228418675 shutdown_connections 2023-08-09T15:12:02.854 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:02.852+0000 7fc7688ee640 1 --2- 172.21.15.43:0/1228418675 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fc7640cf510 0x7fc76406b7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T15:12:02.854 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:02.852+0000 7fc7688ee640 1 --2- 172.21.15.43:0/1228418675 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc76415fe60 0x7fc7640cd270 unknown :-1 s=CLOSED pgs=2488 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T15:12:02.854 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:02.852+0000 7fc7688ee640 1 -- 172.21.15.43:0/1228418675 >> 172.21.15.43:0/1228418675 conn(0x7fc7640d2660 msgr2=0x7fc7640d2a70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-09T15:12:02.854 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:02.852+0000 7fc7688ee640 1 -- 172.21.15.43:0/1228418675 shutdown_connections 2023-08-09T15:12:02.855 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:02.852+0000 7fc7688ee640 1 -- 172.21.15.43:0/1228418675 wait complete. 2023-08-09T15:12:02.855 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:02.854+0000 7fc7688ee640 1 Processor -- start 2023-08-09T15:12:02.855 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:02.854+0000 7fc7688ee640 1 -- start start 2023-08-09T15:12:02.856 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:02.855+0000 7fc7688ee640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fc7640cf510 0x7fc7640c7b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T15:12:02.856 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:02.855+0000 7fc7688ee640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc7640d14c0 0x7fc7640c89e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T15:12:02.856 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:02.855+0000 7fc7688ee640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fc76415fe60 0x7fc7641f5690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T15:12:02.856 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:02.855+0000 7fc7688ee640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc7640b5b90 con 0x7fc7640d14c0 2023-08-09T15:12:02.857 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:02.855+0000 7fc7688ee640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc7640b5db0 con 0x7fc7640cf510 2023-08-09T15:12:02.857 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:02.855+0000 7fc7688ee640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc7640b6090 con 0x7fc76415fe60 2023-08-09T15:12:02.857 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:02.855+0000 7fc761d9b640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc7640d14c0 0x7fc7640c89e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T15:12:02.858 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:02.855+0000 7fc761d9b640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc7640d14c0 0x7fc7640c89e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:41590/0 (socket says 172.21.15.43:41590) 2023-08-09T15:12:02.858 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:02.855+0000 7fc761d9b640 1 -- 172.21.15.43:0/3011013161 learned_addr learned my addr 172.21.15.43:0/3011013161 (peer_addr_for_me v2:172.21.15.43:0/0) 2023-08-09T15:12:02.858 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:02.855+0000 7fc762d9d640 1 --2- 172.21.15.43:0/3011013161 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fc76415fe60 0x7fc7641f5690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T15:12:02.858 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:02.855+0000 7fc76259c640 1 --2- 172.21.15.43:0/3011013161 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fc7640cf510 0x7fc7640c7b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T15:12:02.859 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:02.855+0000 7fc761d9b640 1 -- 172.21.15.43:0/3011013161 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fc76415fe60 msgr2=0x7fc7641f5690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T15:12:02.859 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:02.855+0000 7fc761d9b640 1 --2- 172.21.15.43:0/3011013161 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fc76415fe60 0x7fc7641f5690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T15:12:02.859 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:02.856+0000 7fc761d9b640 1 -- 172.21.15.43:0/3011013161 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fc7640cf510 msgr2=0x7fc7640c7b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T15:12:02.859 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:02.856+0000 7fc761d9b640 1 --2- 172.21.15.43:0/3011013161 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fc7640cf510 0x7fc7640c7b00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T15:12:02.859 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:02.856+0000 7fc761d9b640 1 -- 172.21.15.43:0/3011013161 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc754009400 con 0x7fc7640d14c0 2023-08-09T15:12:02.859 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:02.856+0000 7fc761d9b640 1 --2- 172.21.15.43:0/3011013161 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc7640d14c0 0x7fc7640c89e0 secure :-1 s=READY pgs=2489 cs=0 l=1 rev1=1 crypto rx=0x7fc75800cbc0 tx=0x7fc75800a430 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-09T15:12:02.859 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:02.856+0000 7fc752ffd640 1 -- 172.21.15.43:0/3011013161 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc75800ad40 con 0x7fc7640d14c0 2023-08-09T15:12:02.859 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:02.856+0000 7fc752ffd640 1 -- 172.21.15.43:0/3011013161 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc758012400 con 0x7fc7640d14c0 2023-08-09T15:12:02.860 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:02.856+0000 7fc7688ee640 1 -- 172.21.15.43:0/3011013161 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc7640b6370 con 0x7fc7640d14c0 2023-08-09T15:12:02.860 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:02.856+0000 7fc752ffd640 1 -- 172.21.15.43:0/3011013161 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc75800aea0 con 0x7fc7640d14c0 2023-08-09T15:12:02.860 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:02.856+0000 7fc7688ee640 1 -- 172.21.15.43:0/3011013161 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc7640b6930 con 0x7fc7640d14c0 2023-08-09T15:12:02.860 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:02.857+0000 7fc7688ee640 1 -- 172.21.15.43:0/3011013161 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc7640b6b50 con 0x7fc7640d14c0 2023-08-09T15:12:02.861 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:02.858+0000 7fc752ffd640 1 -- 172.21.15.43:0/3011013161 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 124) v1 ==== 88574+0+0 (secure 0 0 0) 0x7fc758012770 con 0x7fc7640d14c0 2023-08-09T15:12:02.861 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:02.859+0000 7fc752ffd640 1 --2- 172.21.15.43:0/3011013161 >> [v2:172.21.15.203:6832/2375771024,v1:172.21.15.203:6833/2375771024] conn(0x7fc74406a2e0 0x7fc74406c7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T15:12:02.861 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:02.859+0000 7fc752ffd640 1 -- 172.21.15.43:0/3011013161 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(224..224 src has 1..224) v4 ==== 6770+0+0 (secure 0 0 0) 0x7fc758088b80 con 0x7fc7640d14c0 2023-08-09T15:12:02.861 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:02.860+0000 7fc76259c640 1 --2- 172.21.15.43:0/3011013161 >> [v2:172.21.15.203:6832/2375771024,v1:172.21.15.203:6833/2375771024] conn(0x7fc74406a2e0 0x7fc74406c7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T15:12:02.862 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:02.861+0000 7fc76259c640 1 --2- 172.21.15.43:0/3011013161 >> [v2:172.21.15.203:6832/2375771024,v1:172.21.15.203:6833/2375771024] conn(0x7fc74406a2e0 0x7fc74406c7a0 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7fc754036000 tx=0x7fc75403c040 comp rx=0 tx=0).ready entity=mgr.9103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-09T15:12:02.865 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:02.864+0000 7fc752ffd640 1 -- 172.21.15.43:0/3011013161 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178066 (secure 0 0 0) 0x7fc758053ce0 con 0x7fc7640d14c0 2023-08-09T15:12:03.108 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:03.106+0000 7fc7688ee640 1 -- 172.21.15.43:0/3011013161 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "mgr module disable", "module": "snap_schedule"} v 0) v1 -- 0x7fc76404db40 con 0x7fc7640d14c0 2023-08-09T15:12:03.220 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:03.219+0000 7fc752ffd640 1 -- 172.21.15.43:0/3011013161 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mgrmap(e 125) v1 ==== 88557+0+0 (secure 0 0 0) 0x7fc7580540b0 con 0x7fc7640d14c0 2023-08-09T15:12:03.223 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:03.222+0000 7fc752ffd640 1 -- 172.21.15.43:0/3011013161 <== mon.0 v2:172.21.15.43:3300/0 8 ==== mon_command_ack([{"prefix": "mgr module disable", "module": "snap_schedule"}]=0 v125) v1 ==== 93+0+0 (secure 0 0 0) 0x7fc758053ce0 con 0x7fc7640d14c0 2023-08-09T15:12:03.227 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:03.225+0000 7fc7688ee640 1 -- 172.21.15.43:0/3011013161 >> [v2:172.21.15.203:6832/2375771024,v1:172.21.15.203:6833/2375771024] conn(0x7fc74406a2e0 msgr2=0x7fc74406c7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T15:12:03.227 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:03.225+0000 7fc7688ee640 1 --2- 172.21.15.43:0/3011013161 >> [v2:172.21.15.203:6832/2375771024,v1:172.21.15.203:6833/2375771024] conn(0x7fc74406a2e0 0x7fc74406c7a0 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7fc754036000 tx=0x7fc75403c040 comp rx=0 tx=0).stop 2023-08-09T15:12:03.227 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:03.225+0000 7fc7688ee640 1 -- 172.21.15.43:0/3011013161 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc7640d14c0 msgr2=0x7fc7640c89e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T15:12:03.227 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:03.225+0000 7fc7688ee640 1 --2- 172.21.15.43:0/3011013161 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc7640d14c0 0x7fc7640c89e0 secure :-1 s=READY pgs=2489 cs=0 l=1 rev1=1 crypto rx=0x7fc75800cbc0 tx=0x7fc75800a430 comp rx=0 tx=0).stop 2023-08-09T15:12:03.228 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:03.225+0000 7fc7688ee640 1 -- 172.21.15.43:0/3011013161 shutdown_connections 2023-08-09T15:12:03.228 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:03.225+0000 7fc7688ee640 1 --2- 172.21.15.43:0/3011013161 >> [v2:172.21.15.203:6832/2375771024,v1:172.21.15.203:6833/2375771024] conn(0x7fc74406a2e0 0x7fc74406c7a0 unknown :-1 s=CLOSED pgs=40 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T15:12:03.228 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:03.225+0000 7fc7688ee640 1 --2- 172.21.15.43:0/3011013161 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fc76415fe60 0x7fc7641f5690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T15:12:03.228 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:03.225+0000 7fc7688ee640 1 --2- 172.21.15.43:0/3011013161 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc7640d14c0 0x7fc7640c89e0 unknown :-1 s=CLOSED pgs=2489 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T15:12:03.228 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:03.225+0000 7fc7688ee640 1 --2- 172.21.15.43:0/3011013161 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fc7640cf510 0x7fc7640c7b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T15:12:03.228 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:03.226+0000 7fc7688ee640 1 -- 172.21.15.43:0/3011013161 >> 172.21.15.43:0/3011013161 conn(0x7fc7640d2660 msgr2=0x7fc76415e8a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-09T15:12:03.228 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:03.226+0000 7fc7688ee640 1 -- 172.21.15.43:0/3011013161 shutdown_connections 2023-08-09T15:12:03.229 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:03.226+0000 7fc7688ee640 1 -- 172.21.15.43:0/3011013161 wait complete. 2023-08-09T15:12:03.248 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2023-08-09T15:12:03.248 DEBUG:teuthology.orchestra.run.smithi043:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2023-08-09T15:12:03.367 DEBUG:teuthology.orchestra.run.smithi203:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2023-08-09T15:12:03.446 INFO:tasks.ceph.mgr.x.smithi203.stderr:2023-08-09T15:12:03.445+0000 7efcabc1b040 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2023-08-09T15:12:03.465 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2023-08-09T15:12:03.465 DEBUG:teuthology.orchestra.run.smithi043:> set -ex 2023-08-09T15:12:03.466 DEBUG:teuthology.orchestra.run.smithi043:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-09T15:12:03.487 INFO:tasks.ceph.mgr.y.smithi043.stderr:2023-08-09T15:12:03.486+0000 7ff1ea851040 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2023-08-09T15:12:03.522 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-08-09T15:12:03.786 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:03.784+0000 7f83344ec640 1 Processor -- start 2023-08-09T15:12:03.786 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:03.784+0000 7f83344ec640 1 -- start start 2023-08-09T15:12:03.788 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:03.786+0000 7f83344ec640 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7f832c0cb2c0 0x7f832c0ce380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T15:12:03.788 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:03.786+0000 7f83344ec640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f832c0cfae0 0x7f832c0b1f00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T15:12:03.789 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:03.786+0000 7f83344ec640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f832c0cff90 0x7f832c0b2440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T15:12:03.789 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:03.786+0000 7f83344ec640 1 -- --> v2:172.21.15.43:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f832c0cb6a0 con 0x7f832c0cfae0 2023-08-09T15:12:03.789 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:03.786+0000 7f83344ec640 1 -- --> v2:172.21.15.203:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f832c0ce8c0 con 0x7f832c0cb2c0 2023-08-09T15:12:03.789 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:03.786+0000 7f83344ec640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f832c15efa0 con 0x7f832c0cff90 2023-08-09T15:12:03.789 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:03.786+0000 7f8331a87640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f832c0cfae0 0x7f832c0b1f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T15:12:03.789 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:03.786+0000 7f8331a87640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f832c0cfae0 0x7f832c0b1f00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:41630/0 (socket says 172.21.15.43:41630) 2023-08-09T15:12:03.790 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:03.787+0000 7f8331a87640 1 -- 172.21.15.43:0/2484445071 learned_addr learned my addr 172.21.15.43:0/2484445071 (peer_addr_for_me v2:172.21.15.43:0/0) 2023-08-09T15:12:03.790 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:03.787+0000 7f8331a87640 1 -- 172.21.15.43:0/2484445071 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f832c0cff90 msgr2=0x7f832c0b2440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-09T15:12:03.790 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:03.787+0000 7f8331a87640 1 --2- 172.21.15.43:0/2484445071 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f832c0cff90 0x7f832c0b2440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T15:12:03.790 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:03.787+0000 7f8331a87640 1 -- 172.21.15.43:0/2484445071 >> v2:172.21.15.203:3300/0 conn(0x7f832c0cb2c0 msgr2=0x7f832c0ce380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-09T15:12:03.790 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:03.787+0000 7f8331a87640 1 --2- 172.21.15.43:0/2484445071 >> v2:172.21.15.203:3300/0 conn(0x7f832c0cb2c0 0x7f832c0ce380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T15:12:03.790 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:03.787+0000 7f8331a87640 1 -- 172.21.15.43:0/2484445071 --> v2:172.21.15.43:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f832c15e420 con 0x7f832c0cfae0 2023-08-09T15:12:03.793 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:03.788+0000 7f8331a87640 1 --2- 172.21.15.43:0/2484445071 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f832c0cfae0 0x7f832c0b1f00 secure :-1 s=READY pgs=2495 cs=0 l=1 rev1=1 crypto rx=0x7f832c06d5a0 tx=0x7f831c02b920 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e8b343557f49f80b server_cookie=0 in_seq=0 out_seq=0 2023-08-09T15:12:03.793 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:03.788+0000 7f8331286640 1 -- 172.21.15.43:0/2484445071 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f831c031880 con 0x7f832c0cfae0 2023-08-09T15:12:03.793 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:03.788+0000 7f8331286640 1 -- 172.21.15.43:0/2484445071 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f831c0319e0 con 0x7f832c0cfae0 2023-08-09T15:12:03.793 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:03.788+0000 7f8331286640 1 -- 172.21.15.43:0/2484445071 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f831c031d70 con 0x7f832c0cfae0 2023-08-09T15:12:03.793 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:03.789+0000 7f83344ec640 1 -- 172.21.15.43:0/2484445071 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f832c0cfae0 msgr2=0x7f832c0b1f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T15:12:03.794 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:03.789+0000 7f83344ec640 1 --2- 172.21.15.43:0/2484445071 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f832c0cfae0 0x7f832c0b1f00 secure :-1 s=READY pgs=2495 cs=0 l=1 rev1=1 crypto rx=0x7f832c06d5a0 tx=0x7f831c02b920 comp rx=0 tx=0).stop 2023-08-09T15:12:03.794 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:03.789+0000 7f83344ec640 1 -- 172.21.15.43:0/2484445071 shutdown_connections 2023-08-09T15:12:03.794 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:03.789+0000 7f83344ec640 1 --2- 172.21.15.43:0/2484445071 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f832c0cfae0 0x7f832c0b1f00 unknown :-1 s=CLOSED pgs=2495 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T15:12:03.794 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:03.789+0000 7f83344ec640 1 --2- 172.21.15.43:0/2484445071 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f832c0cff90 0x7f832c0b2440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T15:12:03.794 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:03.789+0000 7f83344ec640 1 --2- 172.21.15.43:0/2484445071 >> v2:172.21.15.203:3300/0 conn(0x7f832c0cb2c0 0x7f832c0ce380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T15:12:03.795 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:03.789+0000 7f83344ec640 1 -- 172.21.15.43:0/2484445071 >> 172.21.15.43:0/2484445071 conn(0x7f832c0ccf00 msgr2=0x7f832c0cd310 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-09T15:12:03.795 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:03.789+0000 7f83344ec640 1 -- 172.21.15.43:0/2484445071 shutdown_connections 2023-08-09T15:12:03.795 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:03.789+0000 7f83344ec640 1 -- 172.21.15.43:0/2484445071 wait complete. 2023-08-09T15:12:03.795 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:03.791+0000 7f83344ec640 1 Processor -- start 2023-08-09T15:12:03.795 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:03.791+0000 7f83344ec640 1 -- start start 2023-08-09T15:12:03.797 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:03.792+0000 7f83344ec640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f832c0cb2c0 0x7f832c1514e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T15:12:03.797 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:03.792+0000 7f83344ec640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f832c0cfae0 0x7f832c0b30a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T15:12:03.797 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:03.792+0000 7f83344ec640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f832c0cff90 0x7f832c0b5b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T15:12:03.797 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:03.792+0000 7f83344ec640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f832c06bc00 con 0x7f832c0cfae0 2023-08-09T15:12:03.797 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:03.792+0000 7f83344ec640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f832c0b60a0 con 0x7f832c0cb2c0 2023-08-09T15:12:03.797 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:03.792+0000 7f83344ec640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f832c0b37a0 con 0x7f832c0cff90 2023-08-09T15:12:03.797 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:03.792+0000 7f8331a87640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f832c0cfae0 0x7f832c0b30a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T15:12:03.798 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:03.792+0000 7f8331a87640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f832c0cfae0 0x7f832c0b30a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:41646/0 (socket says 172.21.15.43:41646) 2023-08-09T15:12:03.798 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:03.792+0000 7f8331a87640 1 -- 172.21.15.43:0/2227281263 learned_addr learned my addr 172.21.15.43:0/2227281263 (peer_addr_for_me v2:172.21.15.43:0/0) 2023-08-09T15:12:03.798 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:03.793+0000 7f8332288640 1 --2- 172.21.15.43:0/2227281263 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f832c0cb2c0 0x7f832c1514e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T15:12:03.798 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:03.793+0000 7f8331a87640 1 -- 172.21.15.43:0/2227281263 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f832c0cff90 msgr2=0x7f832c0b5b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T15:12:03.798 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:03.793+0000 7f8331a87640 1 --2- 172.21.15.43:0/2227281263 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f832c0cff90 0x7f832c0b5b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T15:12:03.798 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:03.793+0000 7f8331a87640 1 -- 172.21.15.43:0/2227281263 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f832c0cb2c0 msgr2=0x7f832c1514e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T15:12:03.799 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:03.793+0000 7f8331a87640 1 --2- 172.21.15.43:0/2227281263 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f832c0cb2c0 0x7f832c1514e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T15:12:03.799 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:03.793+0000 7f8331a87640 1 -- 172.21.15.43:0/2227281263 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f831c0094d0 con 0x7f832c0cfae0 2023-08-09T15:12:03.799 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:03.793+0000 7f8331a87640 1 --2- 172.21.15.43:0/2227281263 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f832c0cfae0 0x7f832c0b30a0 secure :-1 s=READY pgs=2496 cs=0 l=1 rev1=1 crypto rx=0x7f831c000c00 tx=0x7f831c0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-09T15:12:03.799 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:03.793+0000 7f8322ffd640 1 -- 172.21.15.43:0/2227281263 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f831c03ad30 con 0x7f832c0cfae0 2023-08-09T15:12:03.799 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:03.793+0000 7f8322ffd640 1 -- 172.21.15.43:0/2227281263 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f831c03ae90 con 0x7f832c0cfae0 2023-08-09T15:12:03.799 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:03.793+0000 7f83344ec640 1 -- 172.21.15.43:0/2227281263 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f832c06b260 con 0x7f832c0cfae0 2023-08-09T15:12:03.799 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:03.793+0000 7f8322ffd640 1 -- 172.21.15.43:0/2227281263 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f831c044630 con 0x7f832c0cfae0 2023-08-09T15:12:03.800 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:03.794+0000 7f83344ec640 1 -- 172.21.15.43:0/2227281263 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f832c06b820 con 0x7f832c0cfae0 2023-08-09T15:12:03.800 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:03.795+0000 7f83344ec640 1 -- 172.21.15.43:0/2227281263 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f832c06bd40 con 0x7f832c0cfae0 2023-08-09T15:12:03.800 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:03.795+0000 7f8322ffd640 1 -- 172.21.15.43:0/2227281263 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 125) v1 ==== 88557+0+0 (secure 0 0 0) 0x7f831c044790 con 0x7f832c0cfae0 2023-08-09T15:12:03.801 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:03.797+0000 7f8322ffd640 1 --2- 172.21.15.43:0/2227281263 >> [v2:172.21.15.203:6832/2375771024,v1:172.21.15.203:6833/2375771024] conn(0x7f831006a290 0x7f831006c750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T15:12:03.801 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:03.797+0000 7f8322ffd640 1 -- 172.21.15.43:0/2227281263 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(224..224 src has 1..224) v4 ==== 6770+0+0 (secure 0 0 0) 0x7f831c0ab500 con 0x7f832c0cfae0 2023-08-09T15:12:03.801 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:03.797+0000 7f8332288640 1 -- 172.21.15.43:0/2227281263 >> [v2:172.21.15.203:6832/2375771024,v1:172.21.15.203:6833/2375771024] conn(0x7f831006a290 msgr2=0x7f831006c750 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.203:6832/2375771024 2023-08-09T15:12:03.801 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:03.797+0000 7f8332288640 1 --2- 172.21.15.43:0/2227281263 >> [v2:172.21.15.203:6832/2375771024,v1:172.21.15.203:6833/2375771024] conn(0x7f831006a290 0x7f831006c750 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-08-09T15:12:03.803 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:03.801+0000 7f8322ffd640 1 -- 172.21.15.43:0/2227281263 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178066 (secure 0 0 0) 0x7f832c06bd40 con 0x7f832c0cfae0 2023-08-09T15:12:03.996 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:03.995+0000 7f83344ec640 1 -- 172.21.15.43:0/2227281263 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f832c06bd40 con 0x7f832c0cfae0 2023-08-09T15:12:03.998 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:03.997+0000 7f8322ffd640 1 -- 172.21.15.43:0/2227281263 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v224) v1 ==== 74+0+19077 (secure 0 0 0) 0x7f832c06bd40 con 0x7f832c0cfae0 2023-08-09T15:12:03.998 INFO:teuthology.orchestra.run.smithi043.stdout: 2023-08-09T15:12:03.999 INFO:teuthology.orchestra.run.smithi043.stdout:{"epoch":224,"fsid":"daac87ee-1337-4250-a955-aea4e408a39b","created":"2023-08-09T13:33:41.701198+0000","modified":"2023-08-09T15:11:43.130258+0000","last_up_change":"2023-08-09T13:33:53.783453+0000","last_in_change":"2023-08-09T13:33:45.148904+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":27,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":"rbd","create_time":"2023-08-09T13:33:54.365137+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":".mgr","create_time":"2023-08-09T13:33:54.831803+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":26,"pool_name":"cephfs_metadata","create_time":"2023-08-09T15:11:26.544757+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"222","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":27,"pool_name":"cephfs_data","create_time":"2023-08-09T15:11:27.194686+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"223","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.6299999952316284,"score_stable":1.6299999952316284,"optimal_score":1,"raw_score_acting":1.6299999952316284,"raw_score_stable":1.6299999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"cbd49ecd-e896-4fbd-864e-5c37a829b165","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":221,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6800","nonce":502667335},{"type":"v1","addr":"172.21.15.43:6801","nonce":502667335}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6802","nonce":502667335},{"type":"v1","addr":"172.21.15.43:6803","nonce":502667335}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6806","nonce":502667335},{"type":"v1","addr":"172.21.15.43:6807","nonce":502667335}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6804","nonce":502667335},{"type":"v1","addr":"172.21.15.43:6805","nonce":502667335}]},"public_addr":"172.21.15.43:6801/502667335","cluster_addr":"172.21.15.43:6803/502667335","heartbeat_back_addr":"172.21.15.43:6807/502667335","heartbeat_front_addr":"172.21.15.43:6805/502667335","state":["exists","up"]},{"osd":1,"uuid":"75c7e0fe-1ec7-4fe8-a2f2-3661a65b8500","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":221,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6808","nonce":3289134610},{"type":"v1","addr":"172.21.15.43:6809","nonce":3289134610}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6810","nonce":3289134610},{"type":"v1","addr":"172.21.15.43:6812","nonce":3289134610}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6821","nonce":3289134610},{"type":"v1","addr":"172.21.15.43:6824","nonce":3289134610}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6815","nonce":3289134610},{"type":"v1","addr":"172.21.15.43:6818","nonce":3289134610}]},"public_addr":"172.21.15.43:6809/3289134610","cluster_addr":"172.21.15.43:6812/3289134610","heartbeat_back_addr":"172.21.15.43:6824/3289134610","heartbeat_front_addr":"172.21.15.43:6818/3289134610","state":["exists","up"]},{"osd":2,"uuid":"bb5c41e9-7f5f-4125-9b01-3bb8bfc2ea83","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":221,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6811","nonce":4079709736},{"type":"v1","addr":"172.21.15.43:6814","nonce":4079709736}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6817","nonce":4079709736},{"type":"v1","addr":"172.21.15.43:6820","nonce":4079709736}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6829","nonce":4079709736},{"type":"v1","addr":"172.21.15.43:6831","nonce":4079709736}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6823","nonce":4079709736},{"type":"v1","addr":"172.21.15.43:6826","nonce":4079709736}]},"public_addr":"172.21.15.43:6814/4079709736","cluster_addr":"172.21.15.43:6820/4079709736","heartbeat_back_addr":"172.21.15.43:6831/4079709736","heartbeat_front_addr":"172.21.15.43:6826/4079709736","state":["exists","up"]},{"osd":3,"uuid":"bfe4c8b4-694a-465c-87e6-2b2aab852613","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":221,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6813","nonce":1709538759},{"type":"v1","addr":"172.21.15.43:6816","nonce":1709538759}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6819","nonce":1709538759},{"type":"v1","addr":"172.21.15.43:6822","nonce":1709538759}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6828","nonce":1709538759},{"type":"v1","addr":"172.21.15.43:6830","nonce":1709538759}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6825","nonce":1709538759},{"type":"v1","addr":"172.21.15.43:6827","nonce":1709538759}]},"public_addr":"172.21.15.43:6816/1709538759","cluster_addr":"172.21.15.43:6822/1709538759","heartbeat_back_addr":"172.21.15.43:6830/1709538759","heartbeat_front_addr":"172.21.15.43:6827/1709538759","state":["exists","up"]},{"osd":4,"uuid":"fa2e8a77-aba1-4756-853e-0d174a5bdeba","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":221,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6800","nonce":4196866481},{"type":"v1","addr":"172.21.15.203:6801","nonce":4196866481}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6802","nonce":4196866481},{"type":"v1","addr":"172.21.15.203:6803","nonce":4196866481}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6806","nonce":4196866481},{"type":"v1","addr":"172.21.15.203:6807","nonce":4196866481}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6804","nonce":4196866481},{"type":"v1","addr":"172.21.15.203:6805","nonce":4196866481}]},"public_addr":"172.21.15.203:6801/4196866481","cluster_addr":"172.21.15.203:6803/4196866481","heartbeat_back_addr":"172.21.15.203:6807/4196866481","heartbeat_front_addr":"172.21.15.203:6805/4196866481","state":["exists","up"]},{"osd":5,"uuid":"81811e4b-4d24-4f51-a82c-f1d700a2a00d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":221,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6816","nonce":3059290081},{"type":"v1","addr":"172.21.15.203:6817","nonce":3059290081}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6818","nonce":3059290081},{"type":"v1","addr":"172.21.15.203:6819","nonce":3059290081}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6822","nonce":3059290081},{"type":"v1","addr":"172.21.15.203:6823","nonce":3059290081}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6820","nonce":3059290081},{"type":"v1","addr":"172.21.15.203:6821","nonce":3059290081}]},"public_addr":"172.21.15.203:6817/3059290081","cluster_addr":"172.21.15.203:6819/3059290081","heartbeat_back_addr":"172.21.15.203:6823/3059290081","heartbeat_front_addr":"172.21.15.203:6821/3059290081","state":["exists","up"]},{"osd":6,"uuid":"307bac4d-6a5b-443d-af75-56d695d7674f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":221,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6824","nonce":512155569},{"type":"v1","addr":"172.21.15.203:6825","nonce":512155569}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6826","nonce":512155569},{"type":"v1","addr":"172.21.15.203:6827","nonce":512155569}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6830","nonce":512155569},{"type":"v1","addr":"172.21.15.203:6831","nonce":512155569}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6828","nonce":512155569},{"type":"v1","addr":"172.21.15.203:6829","nonce":512155569}]},"public_addr":"172.21.15.203:6825/512155569","cluster_addr":"172.21.15.203:6827/512155569","heartbeat_back_addr":"172.21.15.203:6831/512155569","heartbeat_front_addr":"172.21.15.203:6829/512155569","state":["exists","up"]},{"osd":7,"uuid":"a95dc7ca-9f1b-4295-af70-68a22a2d7d5e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":221,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6808","nonce":3331450438},{"type":"v1","addr":"172.21.15.203:6809","nonce":3331450438}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6810","nonce":3331450438},{"type":"v1","addr":"172.21.15.203:6811","nonce":3331450438}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6814","nonce":3331450438},{"type":"v1","addr":"172.21.15.203:6815","nonce":3331450438}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6812","nonce":3331450438},{"type":"v1","addr":"172.21.15.203:6813","nonce":3331450438}]},"public_addr":"172.21.15.203:6809/3331450438","cluster_addr":"172.21.15.203:6811/3331450438","heartbeat_back_addr":"172.21.15.203:6815/3331450438","heartbeat_front_addr":"172.21.15.203:6813/3331450438","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-09T13:33:50.547652+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-09T13:33:50.589469+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-09T13:33:50.561724+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-09T13:33:50.599650+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-09T13:33:50.411855+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-09T13:33:50.219681+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-09T13:33:50.504664+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-09T13:33:50.441248+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.203:6832/4034093268":"2023-08-10T15:11:43.130145+0000","172.21.15.203:6833/4034093268":"2023-08-10T15:11:43.130145+0000","172.21.15.203:0/1654193324":"2023-08-10T15:11:43.130145+0000","172.21.15.203:0/4140323243":"2023-08-10T15:11:43.130145+0000","172.21.15.203:0/450863258":"2023-08-10T15:11:43.130145+0000","172.21.15.203:0/134877587":"2023-08-10T15:11:43.130145+0000","172.21.15.203:0/2330869600":"2023-08-10T15:11:43.130145+0000","172.21.15.203:0/2203818837":"2023-08-10T15:11:43.130145+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":7,"snaps":[{"begin":104,"length":1},{"begin":221,"length":1}]}],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-08-09T15:12:03.999 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:03.998+0000 7f8332288640 1 -- 172.21.15.43:0/2227281263 >> [v2:172.21.15.203:6832/2375771024,v1:172.21.15.203:6833/2375771024] conn(0x7f831006a290 msgr2=0x7f831006c750 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.203:6832/2375771024 2023-08-09T15:12:04.000 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:03.998+0000 7f8332288640 1 --2- 172.21.15.43:0/2227281263 >> [v2:172.21.15.203:6832/2375771024,v1:172.21.15.203:6833/2375771024] conn(0x7f831006a290 0x7f831006c750 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2023-08-09T15:12:04.001 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:03.999+0000 7f83344ec640 1 -- 172.21.15.43:0/2227281263 >> [v2:172.21.15.203:6832/2375771024,v1:172.21.15.203:6833/2375771024] conn(0x7f831006a290 msgr2=0x7f831006c750 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-08-09T15:12:04.001 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:03.999+0000 7f83344ec640 1 --2- 172.21.15.43:0/2227281263 >> [v2:172.21.15.203:6832/2375771024,v1:172.21.15.203:6833/2375771024] conn(0x7f831006a290 0x7f831006c750 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T15:12:04.001 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:03.999+0000 7f83344ec640 1 -- 172.21.15.43:0/2227281263 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f832c0cfae0 msgr2=0x7f832c0b30a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T15:12:04.001 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:03.999+0000 7f83344ec640 1 --2- 172.21.15.43:0/2227281263 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f832c0cfae0 0x7f832c0b30a0 secure :-1 s=READY pgs=2496 cs=0 l=1 rev1=1 crypto rx=0x7f831c000c00 tx=0x7f831c0046c0 comp rx=0 tx=0).stop 2023-08-09T15:12:04.002 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:03.999+0000 7f83344ec640 1 -- 172.21.15.43:0/2227281263 shutdown_connections 2023-08-09T15:12:04.002 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:03.999+0000 7f83344ec640 1 --2- 172.21.15.43:0/2227281263 >> [v2:172.21.15.203:6832/2375771024,v1:172.21.15.203:6833/2375771024] conn(0x7f831006a290 0x7f831006c750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T15:12:04.002 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:03.999+0000 7f83344ec640 1 --2- 172.21.15.43:0/2227281263 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f832c0cff90 0x7f832c0b5b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T15:12:04.002 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:03.999+0000 7f83344ec640 1 --2- 172.21.15.43:0/2227281263 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f832c0cfae0 0x7f832c0b30a0 unknown :-1 s=CLOSED pgs=2496 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T15:12:04.002 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:03.999+0000 7f83344ec640 1 --2- 172.21.15.43:0/2227281263 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f832c0cb2c0 0x7f832c1514e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T15:12:04.002 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:03.999+0000 7f83344ec640 1 -- 172.21.15.43:0/2227281263 >> 172.21.15.43:0/2227281263 conn(0x7f832c0ccf00 msgr2=0x7f832c0b1a50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-09T15:12:04.002 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:04.000+0000 7f83344ec640 1 -- 172.21.15.43:0/2227281263 shutdown_connections 2023-08-09T15:12:04.002 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:04.000+0000 7f83344ec640 1 -- 172.21.15.43:0/2227281263 wait complete. 2023-08-09T15:12:04.019 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': 'rbd', 'create_time': '2023-08-09T13:33:54.365137+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}, 'read_balance': {'score_acting': 2, 'score_stable': 2, 'optimal_score': 1, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': '.mgr', 'create_time': '2023-08-09T13:33:54.831803+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 26, 'pool_name': 'cephfs_metadata', 'create_time': '2023-08-09T15:11:26.544757+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '222', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.75, 'score_stable': 1.75, 'optimal_score': 1, 'raw_score_acting': 1.75, 'raw_score_stable': 1.75, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 27, 'pool_name': 'cephfs_data', 'create_time': '2023-08-09T15:11:27.194686+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '223', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.6299999952316284, 'score_stable': 1.6299999952316284, 'optimal_score': 1, 'raw_score_acting': 1.6299999952316284, 'raw_score_stable': 1.6299999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-08-09T15:12:04.020 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get rbd pg_num 2023-08-09T15:12:04.144 INFO:tasks.ceph.mgr.x.smithi203.stderr:2023-08-09T15:12:04.143+0000 7efcabc1b040 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2023-08-09T15:12:04.233 INFO:tasks.ceph.mgr.y.smithi043.stderr:2023-08-09T15:12:04.231+0000 7ff1ea851040 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2023-08-09T15:12:04.233 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:04.232+0000 7f02290b0640 1 Processor -- start 2023-08-09T15:12:04.234 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:04.232+0000 7f02290b0640 1 -- start start 2023-08-09T15:12:04.235 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:04.234+0000 7f02290b0640 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7f022415c460 0x7f022406f570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T15:12:04.236 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:04.234+0000 7f02290b0640 1 -- --> v1:172.21.15.43:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f02240ce030 con 0x7f02240affc0 2023-08-09T15:12:04.236 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:04.234+0000 7f02290b0640 1 -- --> v1:172.21.15.203:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f02240ce2d0 con 0x7f022415c840 2023-08-09T15:12:04.236 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:04.234+0000 7f02290b0640 1 -- --> v2:172.21.15.203:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f02240ce5b0 con 0x7f022415c460 2023-08-09T15:12:04.237 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:04.235+0000 7f02288af640 1 --1- >> v1:172.21.15.43:6789/0 conn(0x7f02240affc0 0x7f02240b03d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.43:6789/0 says I am v1:172.21.15.43:34266/0 (socket says 172.21.15.43:34266) 2023-08-09T15:12:04.237 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:04.235+0000 7f02288af640 1 -- 172.21.15.43:0/1162076810 learned_addr learned my addr 172.21.15.43:0/1162076810 (peer_addr_for_me v1:172.21.15.43:0/0) 2023-08-09T15:12:04.237 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:04.235+0000 7f0223fff640 1 --2- 172.21.15.43:0/1162076810 >> v2:172.21.15.203:3300/0 conn(0x7f022415c460 0x7f022406f570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T15:12:04.239 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:04.235+0000 7f02237fe640 1 -- 172.21.15.43:0/1162076810 <== mon.0 v1:172.21.15.43:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1619017997 0 0) 0x7f02240ce030 con 0x7f02240affc0 2023-08-09T15:12:04.239 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:04.235+0000 7f02237fe640 1 -- 172.21.15.43:0/1162076810 --> v1:172.21.15.43:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f0210003520 con 0x7f02240affc0 2023-08-09T15:12:04.239 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:04.236+0000 7f0223fff640 1 -- 172.21.15.43:0/1162076810 >> v1:172.21.15.203:6789/0 conn(0x7f022415c840 legacy=0x7f02240cb4a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T15:12:04.239 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:04.236+0000 7f02237fe640 1 -- 172.21.15.43:0/1162076810 <== mon.0 v1:172.21.15.43:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 95701734 0 0) 0x7f0210003520 con 0x7f02240affc0 2023-08-09T15:12:04.239 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:04.236+0000 7f0223fff640 1 -- 172.21.15.43:0/1162076810 >> v1:172.21.15.43:6789/0 conn(0x7f02240affc0 legacy=0x7f02240b03d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T15:12:04.240 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:04.236+0000 7f0223fff640 1 -- 172.21.15.43:0/1162076810 --> v2:172.21.15.203:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f02240ce890 con 0x7f022415c460 2023-08-09T15:12:04.240 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:04.237+0000 7f0223fff640 1 --2- 172.21.15.43:0/1162076810 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f022415c460 0x7f022406f570 secure :-1 s=READY pgs=906 cs=0 l=1 rev1=1 crypto rx=0x7f02140097a0 tx=0x7f021402b820 comp rx=0 tx=0).ready entity=mon.1 client_cookie=ec7cfd50d1c57121 server_cookie=0 in_seq=0 out_seq=0 2023-08-09T15:12:04.240 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:04.238+0000 7f02237fe640 1 -- 172.21.15.43:0/1162076810 <== mon.1 v2:172.21.15.203:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f021402f050 con 0x7f022415c460 2023-08-09T15:12:04.240 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:04.238+0000 7f02237fe640 1 -- 172.21.15.43:0/1162076810 <== mon.1 v2:172.21.15.203:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0214031500 con 0x7f022415c460 2023-08-09T15:12:04.240 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:04.238+0000 7f02237fe640 1 -- 172.21.15.43:0/1162076810 <== mon.1 v2:172.21.15.203:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f021c0019f0 con 0x7f022415c460 2023-08-09T15:12:04.241 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:04.238+0000 7f02290b0640 1 -- 172.21.15.43:0/1162076810 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f022415c460 msgr2=0x7f022406f570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T15:12:04.241 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:04.238+0000 7f02290b0640 1 --2- 172.21.15.43:0/1162076810 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f022415c460 0x7f022406f570 secure :-1 s=READY pgs=906 cs=0 l=1 rev1=1 crypto rx=0x7f02140097a0 tx=0x7f021402b820 comp rx=0 tx=0).stop 2023-08-09T15:12:04.241 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:04.238+0000 7f02290b0640 1 -- 172.21.15.43:0/1162076810 shutdown_connections 2023-08-09T15:12:04.241 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:04.238+0000 7f02290b0640 1 --2- 172.21.15.43:0/1162076810 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f022415c460 0x7f022406f570 unknown :-1 s=CLOSED pgs=906 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T15:12:04.242 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:04.238+0000 7f02290b0640 1 -- 172.21.15.43:0/1162076810 >> 172.21.15.43:0/1162076810 conn(0x7f0224069b30 msgr2=0x7f0224069f40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-09T15:12:04.242 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:04.238+0000 7f02290b0640 1 -- 172.21.15.43:0/1162076810 shutdown_connections 2023-08-09T15:12:04.242 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:04.238+0000 7f02290b0640 1 -- 172.21.15.43:0/1162076810 wait complete. 2023-08-09T15:12:04.243 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:04.240+0000 7f02290b0640 1 Processor -- start 2023-08-09T15:12:04.243 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:04.240+0000 7f02290b0640 1 -- start start 2023-08-09T15:12:04.243 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:04.241+0000 7f02290b0640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f02240affc0 0x7f02240c5170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T15:12:04.243 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:04.241+0000 7f02290b0640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f022415c460 0x7f022415ef00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T15:12:04.244 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:04.241+0000 7f02290b0640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f022415c840 0x7f0224159cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T15:12:04.244 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:04.241+0000 7f02290b0640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f02241f19c0 con 0x7f022415c460 2023-08-09T15:12:04.244 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:04.241+0000 7f02290b0640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f02240b1000 con 0x7f02240affc0 2023-08-09T15:12:04.244 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:04.241+0000 7f02290b0640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f02240b12e0 con 0x7f022415c840 2023-08-09T15:12:04.244 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:04.242+0000 7f021bfff640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f022415c460 0x7f022415ef00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T15:12:04.244 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:04.242+0000 7f021bfff640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f022415c460 0x7f022415ef00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:41660/0 (socket says 172.21.15.43:41660) 2023-08-09T15:12:04.245 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:04.242+0000 7f021bfff640 1 -- 172.21.15.43:0/4035923201 learned_addr learned my addr 172.21.15.43:0/4035923201 (peer_addr_for_me v2:172.21.15.43:0/0) 2023-08-09T15:12:04.246 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:04.242+0000 7f0223fff640 1 --2- 172.21.15.43:0/4035923201 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f02240affc0 0x7f02240c5170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T15:12:04.246 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:04.242+0000 7f021bfff640 1 -- 172.21.15.43:0/4035923201 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f022415c840 msgr2=0x7f0224159cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T15:12:04.246 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:04.242+0000 7f02288af640 1 --2- 172.21.15.43:0/4035923201 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f022415c840 0x7f0224159cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T15:12:04.246 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:04.242+0000 7f021bfff640 1 --2- 172.21.15.43:0/4035923201 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f022415c840 0x7f0224159cf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T15:12:04.246 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:04.242+0000 7f021bfff640 1 -- 172.21.15.43:0/4035923201 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f02240affc0 msgr2=0x7f02240c5170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T15:12:04.247 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:04.242+0000 7f021bfff640 1 --2- 172.21.15.43:0/4035923201 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f02240affc0 0x7f02240c5170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T15:12:04.247 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:04.242+0000 7f021bfff640 1 -- 172.21.15.43:0/4035923201 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0214009400 con 0x7f022415c460 2023-08-09T15:12:04.247 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:04.242+0000 7f021bfff640 1 --2- 172.21.15.43:0/4035923201 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f022415c460 0x7f022415ef00 secure :-1 s=READY pgs=2498 cs=0 l=1 rev1=1 crypto rx=0x7f020c00c400 tx=0x7f020c00c8c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-09T15:12:04.247 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:04.243+0000 7f02217fa640 1 -- 172.21.15.43:0/4035923201 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f020c0052d0 con 0x7f022415c460 2023-08-09T15:12:04.247 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:04.243+0000 7f02217fa640 1 -- 172.21.15.43:0/4035923201 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f020c005430 con 0x7f022415c460 2023-08-09T15:12:04.247 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:04.243+0000 7f02290b0640 1 -- 172.21.15.43:0/4035923201 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f02240b15c0 con 0x7f022415c460 2023-08-09T15:12:04.247 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:04.243+0000 7f02217fa640 1 -- 172.21.15.43:0/4035923201 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f020c003830 con 0x7f022415c460 2023-08-09T15:12:04.247 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:04.243+0000 7f02290b0640 1 -- 172.21.15.43:0/4035923201 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f02240b1b80 con 0x7f022415c460 2023-08-09T15:12:04.248 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:04.244+0000 7f02290b0640 1 -- 172.21.15.43:0/4035923201 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f022404e390 con 0x7f022415c460 2023-08-09T15:12:04.248 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:04.245+0000 7f02217fa640 1 -- 172.21.15.43:0/4035923201 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 125) v1 ==== 88557+0+0 (secure 0 0 0) 0x7f020c003990 con 0x7f022415c460 2023-08-09T15:12:04.249 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:04.246+0000 7f02217fa640 1 --2- 172.21.15.43:0/4035923201 >> [v2:172.21.15.203:6832/2375771024,v1:172.21.15.203:6833/2375771024] conn(0x7f020406a290 0x7f020406c750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T15:12:04.249 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:04.246+0000 7f02217fa640 1 -- 172.21.15.43:0/4035923201 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(224..224 src has 1..224) v4 ==== 6770+0+0 (secure 0 0 0) 0x7f020c088230 con 0x7f022415c460 2023-08-09T15:12:04.249 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:04.246+0000 7f0223fff640 1 -- 172.21.15.43:0/4035923201 >> [v2:172.21.15.203:6832/2375771024,v1:172.21.15.203:6833/2375771024] conn(0x7f020406a290 msgr2=0x7f020406c750 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.203:6832/2375771024 2023-08-09T15:12:04.249 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:04.246+0000 7f0223fff640 1 --2- 172.21.15.43:0/4035923201 >> [v2:172.21.15.203:6832/2375771024,v1:172.21.15.203:6833/2375771024] conn(0x7f020406a290 0x7f020406c750 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-08-09T15:12:04.252 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:04.250+0000 7f02217fa640 1 -- 172.21.15.43:0/4035923201 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178066 (secure 0 0 0) 0x7f020c053330 con 0x7f022415c460 2023-08-09T15:12:04.266 INFO:tasks.ceph.mgr.x.smithi203.stderr:2023-08-09T15:12:04.265+0000 7efcabc1b040 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2023-08-09T15:12:04.322 INFO:tasks.ceph.mgr.x.smithi203.stderr:2023-08-09T15:12:04.320+0000 7efcabc1b040 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2023-08-09T15:12:04.361 INFO:tasks.ceph.mgr.y.smithi043.stderr:2023-08-09T15:12:04.359+0000 7ff1ea851040 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2023-08-09T15:12:04.371 INFO:tasks.ceph.mgr.x.smithi203.stderr:2023-08-09T15:12:04.369+0000 7efcabc1b040 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2023-08-09T15:12:04.420 INFO:tasks.ceph.mgr.y.smithi043.stderr:2023-08-09T15:12:04.418+0000 7ff1ea851040 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2023-08-09T15:12:04.448 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:04.447+0000 7f02290b0640 1 -- 172.21.15.43:0/4035923201 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"} v 0) v1 -- 0x7f02240caa40 con 0x7f022415c460 2023-08-09T15:12:04.448 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:04.447+0000 7f0223fff640 1 -- 172.21.15.43:0/4035923201 >> [v2:172.21.15.203:6832/2375771024,v1:172.21.15.203:6833/2375771024] conn(0x7f020406a290 msgr2=0x7f020406c750 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.203:6832/2375771024 2023-08-09T15:12:04.449 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:04.447+0000 7f0223fff640 1 --2- 172.21.15.43:0/4035923201 >> [v2:172.21.15.203:6832/2375771024,v1:172.21.15.203:6833/2375771024] conn(0x7f020406a290 0x7f020406c750 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2023-08-09T15:12:04.449 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:04.447+0000 7f02217fa640 1 -- 172.21.15.43:0/4035923201 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"}]=0 v224) v1 ==== 92+0+10 (secure 0 0 0) 0x7f020c056950 con 0x7f022415c460 2023-08-09T15:12:04.449 INFO:teuthology.orchestra.run.smithi043.stdout:pg_num: 8 2023-08-09T15:12:04.451 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:04.450+0000 7f02290b0640 1 -- 172.21.15.43:0/4035923201 >> [v2:172.21.15.203:6832/2375771024,v1:172.21.15.203:6833/2375771024] conn(0x7f020406a290 msgr2=0x7f020406c750 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-08-09T15:12:04.451 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:04.450+0000 7f02290b0640 1 --2- 172.21.15.43:0/4035923201 >> [v2:172.21.15.203:6832/2375771024,v1:172.21.15.203:6833/2375771024] conn(0x7f020406a290 0x7f020406c750 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T15:12:04.451 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:04.450+0000 7f02290b0640 1 -- 172.21.15.43:0/4035923201 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f022415c460 msgr2=0x7f022415ef00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T15:12:04.452 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:04.450+0000 7f02290b0640 1 --2- 172.21.15.43:0/4035923201 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f022415c460 0x7f022415ef00 secure :-1 s=READY pgs=2498 cs=0 l=1 rev1=1 crypto rx=0x7f020c00c400 tx=0x7f020c00c8c0 comp rx=0 tx=0).stop 2023-08-09T15:12:04.452 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:04.450+0000 7f02290b0640 1 -- 172.21.15.43:0/4035923201 shutdown_connections 2023-08-09T15:12:04.452 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:04.450+0000 7f02290b0640 1 --2- 172.21.15.43:0/4035923201 >> [v2:172.21.15.203:6832/2375771024,v1:172.21.15.203:6833/2375771024] conn(0x7f020406a290 0x7f020406c750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T15:12:04.453 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:04.450+0000 7f02290b0640 1 --2- 172.21.15.43:0/4035923201 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f022415c840 0x7f0224159cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T15:12:04.453 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:04.450+0000 7f02290b0640 1 --2- 172.21.15.43:0/4035923201 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f022415c460 0x7f022415ef00 unknown :-1 s=CLOSED pgs=2498 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T15:12:04.453 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:04.450+0000 7f02290b0640 1 --2- 172.21.15.43:0/4035923201 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f02240affc0 0x7f02240c5170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T15:12:04.453 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:04.450+0000 7f02290b0640 1 -- 172.21.15.43:0/4035923201 >> 172.21.15.43:0/4035923201 conn(0x7f0224069b30 msgr2=0x7f02240ade00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-09T15:12:04.453 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:04.450+0000 7f02290b0640 1 -- 172.21.15.43:0/4035923201 shutdown_connections 2023-08-09T15:12:04.453 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:04.450+0000 7f02290b0640 1 -- 172.21.15.43:0/4035923201 wait complete. 2023-08-09T15:12:04.468 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-08-09T15:12:04.474 INFO:tasks.ceph.mgr.y.smithi043.stderr:2023-08-09T15:12:04.471+0000 7ff1ea851040 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2023-08-09T15:12:04.678 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:04.677+0000 7f74ad883640 1 Processor -- start 2023-08-09T15:12:04.678 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:04.677+0000 7f74ad883640 1 -- start start 2023-08-09T15:12:04.680 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:04.679+0000 7f74ad883640 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7f74a815c880 0x7f74a80ccbc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T15:12:04.681 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:04.679+0000 7f74ad883640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f74a8160f20 0x7f74a80cec80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T15:12:04.681 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:04.679+0000 7f74ad883640 1 -- --> v1:172.21.15.203:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f74a815cc60 con 0x7f74a80cf1c0 2023-08-09T15:12:04.681 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:04.679+0000 7f74ad883640 1 -- --> v2:172.21.15.203:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f74a80d0530 con 0x7f74a815c880 2023-08-09T15:12:04.681 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:04.679+0000 7f74ad883640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f74a80d1a20 con 0x7f74a8160f20 2023-08-09T15:12:04.682 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:04.680+0000 7f74a6ffd640 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7f74a815c880 0x7f74a80ccbc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T15:12:04.682 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:04.680+0000 7f74a77fe640 1 --1- >> v1:172.21.15.203:6789/0 conn(0x7f74a80cf1c0 0x7f74a80b3ff0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.203:6789/0 says I am v1:172.21.15.43:33304/0 (socket says 172.21.15.43:33304) 2023-08-09T15:12:04.682 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:04.680+0000 7f74a77fe640 1 -- 172.21.15.43:0/408456544 learned_addr learned my addr 172.21.15.43:0/408456544 (peer_addr_for_me v1:172.21.15.43:0/0) 2023-08-09T15:12:04.682 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:04.680+0000 7f74a67fc640 1 --2- 172.21.15.43:0/408456544 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f74a8160f20 0x7f74a80cec80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T15:12:04.682 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:04.681+0000 7f74a67fc640 1 -- 172.21.15.43:0/408456544 >> v2:172.21.15.203:3300/0 conn(0x7f74a815c880 msgr2=0x7f74a80ccbc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-09T15:12:04.682 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:04.681+0000 7f74a67fc640 1 --2- 172.21.15.43:0/408456544 >> v2:172.21.15.203:3300/0 conn(0x7f74a815c880 0x7f74a80ccbc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T15:12:04.683 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:04.681+0000 7f74a67fc640 1 -- 172.21.15.43:0/408456544 >> v1:172.21.15.203:6789/0 conn(0x7f74a80cf1c0 legacy=0x7f74a80b3ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T15:12:04.683 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:04.681+0000 7f74a67fc640 1 -- 172.21.15.43:0/408456544 --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f74a80cf620 con 0x7f74a8160f20 2023-08-09T15:12:04.683 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:04.681+0000 7f74a6ffd640 1 --2- 172.21.15.43:0/408456544 >> v2:172.21.15.203:3300/0 conn(0x7f74a815c880 0x7f74a80ccbc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-08-09T15:12:04.683 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:04.682+0000 7f74a67fc640 1 --2- 172.21.15.43:0/408456544 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f74a8160f20 0x7f74a80cec80 secure :-1 s=READY pgs=351 cs=0 l=1 rev1=1 crypto rx=0x7f74900097a0 tx=0x7f749002b8b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=798e961f1a0e5c6 server_cookie=0 in_seq=0 out_seq=0 2023-08-09T15:12:04.684 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:04.682+0000 7f74a5ffb640 1 -- 172.21.15.43:0/408456544 <== mon.2 v2:172.21.15.203:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f749002d040 con 0x7f74a8160f20 2023-08-09T15:12:04.684 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:04.682+0000 7f74a5ffb640 1 -- 172.21.15.43:0/408456544 <== mon.2 v2:172.21.15.203:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7490031810 con 0x7f74a8160f20 2023-08-09T15:12:04.684 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:04.682+0000 7f74a5ffb640 1 -- 172.21.15.43:0/408456544 <== mon.2 v2:172.21.15.203:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7490031ba0 con 0x7f74a8160f20 2023-08-09T15:12:04.685 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:04.683+0000 7f74ad883640 1 -- 172.21.15.43:0/408456544 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f74a8160f20 msgr2=0x7f74a80cec80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T15:12:04.685 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:04.683+0000 7f74ad883640 1 --2- 172.21.15.43:0/408456544 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f74a8160f20 0x7f74a80cec80 secure :-1 s=READY pgs=351 cs=0 l=1 rev1=1 crypto rx=0x7f74900097a0 tx=0x7f749002b8b0 comp rx=0 tx=0).stop 2023-08-09T15:12:04.685 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:04.683+0000 7f74ad883640 1 -- 172.21.15.43:0/408456544 shutdown_connections 2023-08-09T15:12:04.685 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:04.683+0000 7f74ad883640 1 --2- 172.21.15.43:0/408456544 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f74a8160f20 0x7f74a80cec80 unknown :-1 s=CLOSED pgs=351 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T15:12:04.685 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:04.683+0000 7f74ad883640 1 --2- 172.21.15.43:0/408456544 >> v2:172.21.15.203:3300/0 conn(0x7f74a815c880 0x7f74a80ccbc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T15:12:04.686 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:04.683+0000 7f74ad883640 1 -- 172.21.15.43:0/408456544 >> 172.21.15.43:0/408456544 conn(0x7f74a808bfb0 msgr2=0x7f74a809b8d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-09T15:12:04.686 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:04.684+0000 7f74ad883640 1 -- 172.21.15.43:0/408456544 shutdown_connections 2023-08-09T15:12:04.686 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:04.684+0000 7f74ad883640 1 -- 172.21.15.43:0/408456544 wait complete. 2023-08-09T15:12:04.687 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:04.686+0000 7f74ad883640 1 Processor -- start 2023-08-09T15:12:04.688 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:04.687+0000 7f74ad883640 1 -- start start 2023-08-09T15:12:04.689 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:04.688+0000 7f74ad883640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f74a80cf1c0 0x7f74a81509f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T15:12:04.690 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:04.688+0000 7f74ad883640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f74a815c880 0x7f74a80c70f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T15:12:04.690 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:04.688+0000 7f74ad883640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f74a8160f20 0x7f74a80b1d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T15:12:04.690 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:04.688+0000 7f74ad883640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f74a80c8a70 con 0x7f74a80cf1c0 2023-08-09T15:12:04.690 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:04.688+0000 7f74ad883640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f74a80c79d0 con 0x7f74a815c880 2023-08-09T15:12:04.690 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:04.688+0000 7f74ad883640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f74a80c7cb0 con 0x7f74a8160f20 2023-08-09T15:12:04.691 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:04.689+0000 7f74a6ffd640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f74a80cf1c0 0x7f74a81509f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T15:12:04.691 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:04.689+0000 7f74a6ffd640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f74a80cf1c0 0x7f74a81509f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:41662/0 (socket says 172.21.15.43:41662) 2023-08-09T15:12:04.692 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:04.689+0000 7f74a6ffd640 1 -- 172.21.15.43:0/2095378286 learned_addr learned my addr 172.21.15.43:0/2095378286 (peer_addr_for_me v2:172.21.15.43:0/0) 2023-08-09T15:12:04.692 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:04.689+0000 7f74a67fc640 1 --2- 172.21.15.43:0/2095378286 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f74a815c880 0x7f74a80c70f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T15:12:04.692 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:04.689+0000 7f74a77fe640 1 --2- 172.21.15.43:0/2095378286 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f74a8160f20 0x7f74a80b1d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T15:12:04.692 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:04.689+0000 7f74a6ffd640 1 -- 172.21.15.43:0/2095378286 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f74a8160f20 msgr2=0x7f74a80b1d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T15:12:04.692 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:04.689+0000 7f74a6ffd640 1 --2- 172.21.15.43:0/2095378286 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f74a8160f20 0x7f74a80b1d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T15:12:04.692 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:04.689+0000 7f74a6ffd640 1 -- 172.21.15.43:0/2095378286 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f74a815c880 msgr2=0x7f74a80c70f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T15:12:04.692 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:04.689+0000 7f74a6ffd640 1 --2- 172.21.15.43:0/2095378286 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f74a815c880 0x7f74a80c70f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T15:12:04.693 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:04.689+0000 7f74a6ffd640 1 -- 172.21.15.43:0/2095378286 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7498009a70 con 0x7f74a80cf1c0 2023-08-09T15:12:04.693 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:04.689+0000 7f74a6ffd640 1 --2- 172.21.15.43:0/2095378286 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f74a80cf1c0 0x7f74a81509f0 secure :-1 s=READY pgs=2499 cs=0 l=1 rev1=1 crypto rx=0x7f749800efc0 tx=0x7f749800c550 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-09T15:12:04.693 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:04.690+0000 7f749ffff640 1 -- 172.21.15.43:0/2095378286 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7498014680 con 0x7f74a80cf1c0 2023-08-09T15:12:04.693 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:04.690+0000 7f749ffff640 1 -- 172.21.15.43:0/2095378286 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f749800f040 con 0x7f74a80cf1c0 2023-08-09T15:12:04.693 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:04.690+0000 7f749ffff640 1 -- 172.21.15.43:0/2095378286 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7498014a00 con 0x7f74a80cf1c0 2023-08-09T15:12:04.693 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:04.690+0000 7f74ad883640 1 -- 172.21.15.43:0/2095378286 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7490009400 con 0x7f74a80cf1c0 2023-08-09T15:12:04.693 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:04.690+0000 7f74ad883640 1 -- 172.21.15.43:0/2095378286 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f74a80c8270 con 0x7f74a80cf1c0 2023-08-09T15:12:04.694 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:04.691+0000 7f74ad883640 1 -- 172.21.15.43:0/2095378286 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f74a804e390 con 0x7f74a80cf1c0 2023-08-09T15:12:04.694 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:04.692+0000 7f749ffff640 1 -- 172.21.15.43:0/2095378286 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 125) v1 ==== 88557+0+0 (secure 0 0 0) 0x7f7498014b60 con 0x7f74a80cf1c0 2023-08-09T15:12:04.695 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:04.693+0000 7f749ffff640 1 --2- 172.21.15.43:0/2095378286 >> [v2:172.21.15.203:6832/2375771024,v1:172.21.15.203:6833/2375771024] conn(0x7f748806a290 0x7f748806c750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T15:12:04.695 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:04.693+0000 7f749ffff640 1 -- 172.21.15.43:0/2095378286 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(224..224 src has 1..224) v4 ==== 6770+0+0 (secure 0 0 0) 0x7f7498089ef0 con 0x7f74a80cf1c0 2023-08-09T15:12:04.695 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:04.693+0000 7f74a67fc640 1 -- 172.21.15.43:0/2095378286 >> [v2:172.21.15.203:6832/2375771024,v1:172.21.15.203:6833/2375771024] conn(0x7f748806a290 msgr2=0x7f748806c750 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.203:6832/2375771024 2023-08-09T15:12:04.695 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:04.693+0000 7f74a67fc640 1 --2- 172.21.15.43:0/2095378286 >> [v2:172.21.15.203:6832/2375771024,v1:172.21.15.203:6833/2375771024] conn(0x7f748806a290 0x7f748806c750 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-08-09T15:12:04.699 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:04.698+0000 7f749ffff640 1 -- 172.21.15.43:0/2095378286 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178066 (secure 0 0 0) 0x7f7498054e90 con 0x7f74a80cf1c0 2023-08-09T15:12:04.889 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:04.888+0000 7f74ad883640 1 -- 172.21.15.43:0/2095378286 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f74a815aef0 con 0x7f74a80cf1c0 2023-08-09T15:12:04.890 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:04.889+0000 7f749ffff640 1 -- 172.21.15.43:0/2095378286 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v224) v1 ==== 93+0+10 (secure 0 0 0) 0x7f74980584b0 con 0x7f74a80cf1c0 2023-08-09T15:12:04.890 INFO:teuthology.orchestra.run.smithi043.stdout:pg_num: 1 2023-08-09T15:12:04.893 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:04.891+0000 7f74ad883640 1 -- 172.21.15.43:0/2095378286 >> [v2:172.21.15.203:6832/2375771024,v1:172.21.15.203:6833/2375771024] conn(0x7f748806a290 msgr2=0x7f748806c750 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-08-09T15:12:04.893 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:04.891+0000 7f74ad883640 1 --2- 172.21.15.43:0/2095378286 >> [v2:172.21.15.203:6832/2375771024,v1:172.21.15.203:6833/2375771024] conn(0x7f748806a290 0x7f748806c750 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T15:12:04.893 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:04.891+0000 7f74ad883640 1 -- 172.21.15.43:0/2095378286 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f74a80cf1c0 msgr2=0x7f74a81509f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T15:12:04.893 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:04.891+0000 7f74ad883640 1 --2- 172.21.15.43:0/2095378286 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f74a80cf1c0 0x7f74a81509f0 secure :-1 s=READY pgs=2499 cs=0 l=1 rev1=1 crypto rx=0x7f749800efc0 tx=0x7f749800c550 comp rx=0 tx=0).stop 2023-08-09T15:12:04.893 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:04.891+0000 7f74ad883640 1 -- 172.21.15.43:0/2095378286 shutdown_connections 2023-08-09T15:12:04.893 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:04.891+0000 7f74ad883640 1 --2- 172.21.15.43:0/2095378286 >> [v2:172.21.15.203:6832/2375771024,v1:172.21.15.203:6833/2375771024] conn(0x7f748806a290 0x7f748806c750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T15:12:04.894 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:04.891+0000 7f74ad883640 1 --2- 172.21.15.43:0/2095378286 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f74a8160f20 0x7f74a80b1d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T15:12:04.894 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:04.891+0000 7f74ad883640 1 --2- 172.21.15.43:0/2095378286 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f74a815c880 0x7f74a80c70f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T15:12:04.894 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:04.891+0000 7f74ad883640 1 --2- 172.21.15.43:0/2095378286 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f74a80cf1c0 0x7f74a81509f0 unknown :-1 s=CLOSED pgs=2499 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T15:12:04.894 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:04.891+0000 7f74ad883640 1 -- 172.21.15.43:0/2095378286 >> 172.21.15.43:0/2095378286 conn(0x7f74a808bfb0 msgr2=0x7f74a81602e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-09T15:12:04.894 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:04.891+0000 7f74ad883640 1 -- 172.21.15.43:0/2095378286 shutdown_connections 2023-08-09T15:12:04.894 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:04.891+0000 7f74ad883640 1 -- 172.21.15.43:0/2095378286 wait complete. 2023-08-09T15:12:04.909 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-08-09T15:12:05.083 INFO:tasks.ceph.mgr.x.smithi203.stderr:2023-08-09T15:12:05.082+0000 7efcabc1b040 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2023-08-09T15:12:05.107 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:05.106+0000 7fd9a526a640 1 Processor -- start 2023-08-09T15:12:05.107 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:05.106+0000 7fd9a526a640 1 -- start start 2023-08-09T15:12:05.109 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:05.107+0000 7fd9a526a640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7fd9a015cdd0 0x7fd9a00ca7d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T15:12:05.109 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:05.108+0000 7fd9a526a640 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7fd9a00cad10 0x7fd9a0070780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T15:12:05.109 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:05.108+0000 7fd9a526a640 1 -- --> v1:172.21.15.203:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd9a00cdb80 con 0x7fd9a0159130 2023-08-09T15:12:05.109 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:05.108+0000 7fd9a526a640 1 -- --> v2:172.21.15.43:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd9a00702c0 con 0x7fd9a015cdd0 2023-08-09T15:12:05.109 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:05.108+0000 7fd9a526a640 1 -- --> v2:172.21.15.203:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd9a00cb1a0 con 0x7fd9a00cad10 2023-08-09T15:12:05.110 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:05.108+0000 7fd99f7fe640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7fd9a015cdd0 0x7fd9a00ca7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T15:12:05.110 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:05.108+0000 7fd99f7fe640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7fd9a015cdd0 0x7fd9a00ca7d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:33158/0 (socket says 172.21.15.43:33158) 2023-08-09T15:12:05.111 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:05.108+0000 7fd99f7fe640 1 -- 172.21.15.43:0/1602710983 learned_addr learned my addr 172.21.15.43:0/1602710983 (peer_addr_for_me v2:172.21.15.43:0/0) 2023-08-09T15:12:05.111 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:05.108+0000 7fd9a4a69640 1 --2- 172.21.15.43:0/1602710983 >> v2:172.21.15.203:3300/0 conn(0x7fd9a00cad10 0x7fd9a0070780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T15:12:05.111 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:05.108+0000 7fd99f7fe640 1 -- 172.21.15.43:0/1602710983 >> v2:172.21.15.203:3300/0 conn(0x7fd9a00cad10 msgr2=0x7fd9a0070780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-09T15:12:05.111 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:05.108+0000 7fd99f7fe640 1 --2- 172.21.15.43:0/1602710983 >> v2:172.21.15.203:3300/0 conn(0x7fd9a00cad10 0x7fd9a0070780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T15:12:05.111 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:05.109+0000 7fd99f7fe640 1 -- 172.21.15.43:0/1602710983 >> v1:172.21.15.203:6789/0 conn(0x7fd9a0159130 legacy=0x7fd9a015a260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-09T15:12:05.111 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:05.109+0000 7fd99f7fe640 1 -- 172.21.15.43:0/1602710983 --> v2:172.21.15.43:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd9a015d270 con 0x7fd9a015cdd0 2023-08-09T15:12:05.112 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:05.110+0000 7fd99f7fe640 1 --2- 172.21.15.43:0/1602710983 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fd9a015cdd0 0x7fd9a00ca7d0 secure :-1 s=READY pgs=2500 cs=0 l=1 rev1=1 crypto rx=0x7fd9a00ca520 tx=0x7fd99402b880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b0d36770cb0d7c3c server_cookie=0 in_seq=0 out_seq=0 2023-08-09T15:12:05.113 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:05.110+0000 7fd99effd640 1 -- 172.21.15.43:0/1602710983 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd9940317e0 con 0x7fd9a015cdd0 2023-08-09T15:12:05.113 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:05.110+0000 7fd99effd640 1 -- 172.21.15.43:0/1602710983 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd994031940 con 0x7fd9a015cdd0 2023-08-09T15:12:05.113 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:05.110+0000 7fd99effd640 1 -- 172.21.15.43:0/1602710983 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd994031cd0 con 0x7fd9a015cdd0 2023-08-09T15:12:05.113 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:05.111+0000 7fd9a526a640 1 -- 172.21.15.43:0/1602710983 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fd9a015cdd0 msgr2=0x7fd9a00ca7d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T15:12:05.113 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:05.111+0000 7fd9a526a640 1 --2- 172.21.15.43:0/1602710983 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fd9a015cdd0 0x7fd9a00ca7d0 secure :-1 s=READY pgs=2500 cs=0 l=1 rev1=1 crypto rx=0x7fd9a00ca520 tx=0x7fd99402b880 comp rx=0 tx=0).stop 2023-08-09T15:12:05.113 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:05.111+0000 7fd9a526a640 1 -- 172.21.15.43:0/1602710983 shutdown_connections 2023-08-09T15:12:05.113 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:05.111+0000 7fd9a526a640 1 --2- 172.21.15.43:0/1602710983 >> v2:172.21.15.203:3300/0 conn(0x7fd9a00cad10 0x7fd9a0070780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T15:12:05.114 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:05.111+0000 7fd9a526a640 1 --2- 172.21.15.43:0/1602710983 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fd9a015cdd0 0x7fd9a00ca7d0 unknown :-1 s=CLOSED pgs=2500 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T15:12:05.114 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:05.111+0000 7fd9a526a640 1 -- 172.21.15.43:0/1602710983 >> 172.21.15.43:0/1602710983 conn(0x7fd9a00afb60 msgr2=0x7fd9a00aff40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-09T15:12:05.114 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:05.111+0000 7fd9a526a640 1 -- 172.21.15.43:0/1602710983 shutdown_connections 2023-08-09T15:12:05.115 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:05.111+0000 7fd9a526a640 1 -- 172.21.15.43:0/1602710983 wait complete. 2023-08-09T15:12:05.117 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:05.112+0000 7fd9a526a640 1 Processor -- start 2023-08-09T15:12:05.117 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:05.113+0000 7fd9a526a640 1 -- start start 2023-08-09T15:12:05.117 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:05.114+0000 7fd9a526a640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fd9a00cad10 0x7fd9a00c6700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T15:12:05.118 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:05.114+0000 7fd9a526a640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fd9a0159130 0x7fd9a013a8a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T15:12:05.118 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:05.114+0000 7fd9a526a640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fd9a015cdd0 0x7fd9a014b6b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T15:12:05.118 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:05.114+0000 7fd9a526a640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd9a00c6da0 con 0x7fd9a0159130 2023-08-09T15:12:05.118 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:05.114+0000 7fd9a526a640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd9a00c7080 con 0x7fd9a015cdd0 2023-08-09T15:12:05.118 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:05.114+0000 7fd9a526a640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd9a00c7360 con 0x7fd9a00cad10 2023-08-09T15:12:05.118 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:05.114+0000 7fd99f7fe640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fd9a0159130 0x7fd9a013a8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T15:12:05.118 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:05.114+0000 7fd99f7fe640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fd9a0159130 0x7fd9a013a8a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:33170/0 (socket says 172.21.15.43:33170) 2023-08-09T15:12:05.118 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:05.114+0000 7fd99f7fe640 1 -- 172.21.15.43:0/347036559 learned_addr learned my addr 172.21.15.43:0/347036559 (peer_addr_for_me v2:172.21.15.43:0/0) 2023-08-09T15:12:05.119 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:05.114+0000 7fd99f7fe640 1 -- 172.21.15.43:0/347036559 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fd9a00cad10 msgr2=0x7fd9a00c6700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T15:12:05.119 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:05.114+0000 7fd99f7fe640 1 --2- 172.21.15.43:0/347036559 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fd9a00cad10 0x7fd9a00c6700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T15:12:05.119 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:05.114+0000 7fd9a4a69640 1 --2- 172.21.15.43:0/347036559 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fd9a015cdd0 0x7fd9a014b6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T15:12:05.119 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:05.114+0000 7fd99f7fe640 1 -- 172.21.15.43:0/347036559 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fd9a015cdd0 msgr2=0x7fd9a014b6b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T15:12:05.119 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:05.114+0000 7fd99f7fe640 1 --2- 172.21.15.43:0/347036559 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fd9a015cdd0 0x7fd9a014b6b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T15:12:05.119 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:05.114+0000 7fd99f7fe640 1 -- 172.21.15.43:0/347036559 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd9940094d0 con 0x7fd9a0159130 2023-08-09T15:12:05.119 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:05.115+0000 7fd99f7fe640 1 --2- 172.21.15.43:0/347036559 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fd9a0159130 0x7fd9a013a8a0 secure :-1 s=READY pgs=2501 cs=0 l=1 rev1=1 crypto rx=0x7fd99402b8c0 tx=0x7fd9940046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-09T15:12:05.120 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:05.115+0000 7fd99cff9640 1 -- 172.21.15.43:0/347036559 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd994032880 con 0x7fd9a0159130 2023-08-09T15:12:05.120 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:05.115+0000 7fd99cff9640 1 -- 172.21.15.43:0/347036559 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd9940329e0 con 0x7fd9a0159130 2023-08-09T15:12:05.120 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:05.115+0000 7fd99cff9640 1 -- 172.21.15.43:0/347036559 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd994032d90 con 0x7fd9a0159130 2023-08-09T15:12:05.120 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:05.115+0000 7fd9a526a640 1 -- 172.21.15.43:0/347036559 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd9a00c7640 con 0x7fd9a0159130 2023-08-09T15:12:05.120 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:05.115+0000 7fd9a526a640 1 -- 172.21.15.43:0/347036559 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd9a013ade0 con 0x7fd9a0159130 2023-08-09T15:12:05.121 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:05.116+0000 7fd9a526a640 1 -- 172.21.15.43:0/347036559 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd9a00c7b60 con 0x7fd9a0159130 2023-08-09T15:12:05.121 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:05.117+0000 7fd99cff9640 1 -- 172.21.15.43:0/347036559 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 125) v1 ==== 88557+0+0 (secure 0 0 0) 0x7fd99404e450 con 0x7fd9a0159130 2023-08-09T15:12:05.121 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:05.118+0000 7fd99cff9640 1 --2- 172.21.15.43:0/347036559 >> [v2:172.21.15.203:6832/2375771024,v1:172.21.15.203:6833/2375771024] conn(0x7fd98c06a290 0x7fd98c06c750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T15:12:05.121 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:05.118+0000 7fd99cff9640 1 -- 172.21.15.43:0/347036559 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(224..224 src has 1..224) v4 ==== 6770+0+0 (secure 0 0 0) 0x7fd9940aaf50 con 0x7fd9a0159130 2023-08-09T15:12:05.121 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:05.118+0000 7fd99ffff640 1 -- 172.21.15.43:0/347036559 >> [v2:172.21.15.203:6832/2375771024,v1:172.21.15.203:6833/2375771024] conn(0x7fd98c06a290 msgr2=0x7fd98c06c750 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.203:6832/2375771024 2023-08-09T15:12:05.122 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:05.118+0000 7fd99ffff640 1 --2- 172.21.15.43:0/347036559 >> [v2:172.21.15.203:6832/2375771024,v1:172.21.15.203:6833/2375771024] conn(0x7fd98c06a290 0x7fd98c06c750 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-08-09T15:12:05.123 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:05.122+0000 7fd99cff9640 1 -- 172.21.15.43:0/347036559 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178066 (secure 0 0 0) 0x7fd9a00c7b60 con 0x7fd9a0159130 2023-08-09T15:12:05.145 INFO:tasks.ceph.mgr.x.smithi203.stderr:2023-08-09T15:12:05.144+0000 7efcabc1b040 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2023-08-09T15:12:05.194 INFO:tasks.ceph.mgr.x.smithi203.stderr:2023-08-09T15:12:05.193+0000 7efcabc1b040 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2023-08-09T15:12:05.197 INFO:tasks.ceph.mgr.y.smithi043.stderr:2023-08-09T15:12:05.196+0000 7ff1ea851040 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2023-08-09T15:12:05.262 INFO:tasks.ceph.mgr.y.smithi043.stderr:2023-08-09T15:12:05.261+0000 7ff1ea851040 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2023-08-09T15:12:05.313 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:05.312+0000 7fd9a526a640 1 -- 172.21.15.43:0/347036559 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7fd9a00c7b60 con 0x7fd9a0159130 2023-08-09T15:12:05.314 INFO:tasks.ceph.mgr.x.smithi203.stderr:2023-08-09T15:12:05.312+0000 7efcabc1b040 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2023-08-09T15:12:05.314 INFO:tasks.ceph.mgr.y.smithi043.stderr:2023-08-09T15:12:05.313+0000 7ff1ea851040 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2023-08-09T15:12:05.315 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:05.313+0000 7fd99cff9640 1 -- 172.21.15.43:0/347036559 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v224) v1 ==== 104+0+11 (secure 0 0 0) 0x7fd9a00c7b60 con 0x7fd9a0159130 2023-08-09T15:12:05.315 INFO:teuthology.orchestra.run.smithi043.stdout:pg_num: 32 2023-08-09T15:12:05.317 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:05.315+0000 7fd9a526a640 1 -- 172.21.15.43:0/347036559 >> [v2:172.21.15.203:6832/2375771024,v1:172.21.15.203:6833/2375771024] conn(0x7fd98c06a290 msgr2=0x7fd98c06c750 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-08-09T15:12:05.317 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:05.315+0000 7fd9a526a640 1 --2- 172.21.15.43:0/347036559 >> [v2:172.21.15.203:6832/2375771024,v1:172.21.15.203:6833/2375771024] conn(0x7fd98c06a290 0x7fd98c06c750 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T15:12:05.317 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:05.315+0000 7fd9a526a640 1 -- 172.21.15.43:0/347036559 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fd9a0159130 msgr2=0x7fd9a013a8a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T15:12:05.317 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:05.315+0000 7fd9a526a640 1 --2- 172.21.15.43:0/347036559 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fd9a0159130 0x7fd9a013a8a0 secure :-1 s=READY pgs=2501 cs=0 l=1 rev1=1 crypto rx=0x7fd99402b8c0 tx=0x7fd9940046c0 comp rx=0 tx=0).stop 2023-08-09T15:12:05.318 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:05.316+0000 7fd9a526a640 1 -- 172.21.15.43:0/347036559 shutdown_connections 2023-08-09T15:12:05.318 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:05.316+0000 7fd9a526a640 1 --2- 172.21.15.43:0/347036559 >> [v2:172.21.15.203:6832/2375771024,v1:172.21.15.203:6833/2375771024] conn(0x7fd98c06a290 0x7fd98c06c750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T15:12:05.318 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:05.316+0000 7fd9a526a640 1 --2- 172.21.15.43:0/347036559 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fd9a015cdd0 0x7fd9a014b6b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T15:12:05.318 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:05.316+0000 7fd9a526a640 1 --2- 172.21.15.43:0/347036559 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fd9a0159130 0x7fd9a013a8a0 unknown :-1 s=CLOSED pgs=2501 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T15:12:05.318 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:05.316+0000 7fd9a526a640 1 --2- 172.21.15.43:0/347036559 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fd9a00cad10 0x7fd9a00c6700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T15:12:05.318 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:05.316+0000 7fd9a526a640 1 -- 172.21.15.43:0/347036559 >> 172.21.15.43:0/347036559 conn(0x7fd9a00afb60 msgr2=0x7fd9a0152390 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-09T15:12:05.318 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:05.316+0000 7fd9a526a640 1 -- 172.21.15.43:0/347036559 shutdown_connections 2023-08-09T15:12:05.318 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:05.316+0000 7fd9a526a640 1 -- 172.21.15.43:0/347036559 wait complete. 2023-08-09T15:12:05.335 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-08-09T15:12:05.440 INFO:tasks.ceph.mgr.y.smithi043.stderr:2023-08-09T15:12:05.439+0000 7ff1ea851040 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2023-08-09T15:12:05.471 INFO:tasks.ceph.mgr.x.smithi203.stderr:2023-08-09T15:12:05.469+0000 7efcabc1b040 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2023-08-09T15:12:05.541 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:05.540+0000 7f13d4a63640 1 Processor -- start 2023-08-09T15:12:05.542 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:05.540+0000 7f13d4a63640 1 -- start start 2023-08-09T15:12:05.543 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:05.542+0000 7f13d4a63640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f13d00d1df0 0x7f13d00cd1a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T15:12:05.543 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:05.542+0000 7f13d4a63640 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7f13d00cd6e0 0x7f13d00d0830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T15:12:05.544 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:05.542+0000 7f13d4a63640 1 -- --> v1:172.21.15.203:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f13d0160e00 con 0x7f13d015c870 2023-08-09T15:12:05.544 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:05.542+0000 7f13d4a63640 1 -- --> v2:172.21.15.203:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f13d00cdb70 con 0x7f13d00cd6e0 2023-08-09T15:12:05.544 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:05.542+0000 7f13d4a63640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f13d01601f0 con 0x7f13d00d1df0 2023-08-09T15:12:05.545 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:05.543+0000 7f13cffff640 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7f13d00cd6e0 0x7f13d00d0830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T15:12:05.545 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:05.543+0000 7f13ceffd640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f13d00d1df0 0x7f13d00cd1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T15:12:05.545 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:05.543+0000 7f13cffff640 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7f13d00cd6e0 0x7f13d00d0830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.203:3300/0 says I am v2:172.21.15.43:41202/0 (socket says 172.21.15.43:41202) 2023-08-09T15:12:05.545 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:05.543+0000 7f13ceffd640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f13d00d1df0 0x7f13d00cd1a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.203:3301/0 says I am v2:172.21.15.43:42822/0 (socket says 172.21.15.43:42822) 2023-08-09T15:12:05.545 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:05.543+0000 7f13cffff640 1 -- 172.21.15.43:0/3304310686 learned_addr learned my addr 172.21.15.43:0/3304310686 (peer_addr_for_me v2:172.21.15.43:0/0) 2023-08-09T15:12:05.546 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:05.543+0000 7f13cffff640 1 -- 172.21.15.43:0/3304310686 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f13d00d1df0 msgr2=0x7f13d00cd1a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-09T15:12:05.546 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:05.543+0000 7f13cffff640 1 --2- 172.21.15.43:0/3304310686 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f13d00d1df0 0x7f13d00cd1a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T15:12:05.546 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:05.543+0000 7f13cffff640 1 -- 172.21.15.43:0/3304310686 >> v1:172.21.15.203:6789/0 conn(0x7f13d015c870 legacy=0x7f13d00b7d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T15:12:05.546 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:05.543+0000 7f13cffff640 1 -- 172.21.15.43:0/3304310686 --> v2:172.21.15.203:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f13d01604d0 con 0x7f13d00cd6e0 2023-08-09T15:12:05.547 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:05.545+0000 7f13cffff640 1 --2- 172.21.15.43:0/3304310686 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f13d00cd6e0 0x7f13d00d0830 secure :-1 s=READY pgs=910 cs=0 l=1 rev1=1 crypto rx=0x7f13c8009e50 tx=0x7f13c802bf60 comp rx=0 tx=0).ready entity=mon.1 client_cookie=19c94b4ff5ba113a server_cookie=0 in_seq=0 out_seq=0 2023-08-09T15:12:05.547 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:05.545+0000 7f13ce7fc640 1 -- 172.21.15.43:0/3304310686 <== mon.1 v2:172.21.15.203:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f13c802cb00 con 0x7f13d00cd6e0 2023-08-09T15:12:05.547 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:05.545+0000 7f13ce7fc640 1 -- 172.21.15.43:0/3304310686 <== mon.1 v2:172.21.15.203:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f13c802cc60 con 0x7f13d00cd6e0 2023-08-09T15:12:05.547 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:05.545+0000 7f13ce7fc640 1 -- 172.21.15.43:0/3304310686 <== mon.1 v2:172.21.15.203:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f13c80343f0 con 0x7f13d00cd6e0 2023-08-09T15:12:05.548 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:05.546+0000 7f13d4a63640 1 -- 172.21.15.43:0/3304310686 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f13d00cd6e0 msgr2=0x7f13d00d0830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T15:12:05.548 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:05.546+0000 7f13d4a63640 1 --2- 172.21.15.43:0/3304310686 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f13d00cd6e0 0x7f13d00d0830 secure :-1 s=READY pgs=910 cs=0 l=1 rev1=1 crypto rx=0x7f13c8009e50 tx=0x7f13c802bf60 comp rx=0 tx=0).stop 2023-08-09T15:12:05.548 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:05.546+0000 7f13d4a63640 1 -- 172.21.15.43:0/3304310686 shutdown_connections 2023-08-09T15:12:05.549 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:05.546+0000 7f13d4a63640 1 --2- 172.21.15.43:0/3304310686 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f13d00cd6e0 0x7f13d00d0830 unknown :-1 s=CLOSED pgs=910 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T15:12:05.549 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:05.546+0000 7f13d4a63640 1 --2- 172.21.15.43:0/3304310686 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f13d00d1df0 0x7f13d00cd1a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T15:12:05.549 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:05.546+0000 7f13d4a63640 1 -- 172.21.15.43:0/3304310686 >> 172.21.15.43:0/3304310686 conn(0x7f13d0071260 msgr2=0x7f13d0071670 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-09T15:12:05.549 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:05.546+0000 7f13d4a63640 1 -- 172.21.15.43:0/3304310686 shutdown_connections 2023-08-09T15:12:05.549 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:05.546+0000 7f13d4a63640 1 -- 172.21.15.43:0/3304310686 wait complete. 2023-08-09T15:12:05.550 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:05.548+0000 7f13d4a63640 1 Processor -- start 2023-08-09T15:12:05.550 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:05.548+0000 7f13d4a63640 1 -- start start 2023-08-09T15:12:05.551 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:05.549+0000 7f13d4a63640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f13d00cd6e0 0x7f13d0158e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T15:12:05.551 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:05.549+0000 7f13d4a63640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f13d00d1df0 0x7f13d00ca6b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T15:12:05.552 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:05.549+0000 7f13d4a63640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f13d00cabf0 0x7f13d00d1170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T15:12:05.552 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:05.549+0000 7f13d4a63640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f13d0159380 con 0x7f13d00cd6e0 2023-08-09T15:12:05.552 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:05.549+0000 7f13d4a63640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f13d015f780 con 0x7f13d00d1df0 2023-08-09T15:12:05.552 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:05.549+0000 7f13d4a63640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f13d01f2cb0 con 0x7f13d00cabf0 2023-08-09T15:12:05.552 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:05.549+0000 7f13cf7fe640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f13d00cd6e0 0x7f13d0158e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T15:12:05.552 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:05.549+0000 7f13cf7fe640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f13d00cd6e0 0x7f13d0158e40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:33174/0 (socket says 172.21.15.43:33174) 2023-08-09T15:12:05.552 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:05.549+0000 7f13cf7fe640 1 -- 172.21.15.43:0/463561673 learned_addr learned my addr 172.21.15.43:0/463561673 (peer_addr_for_me v2:172.21.15.43:0/0) 2023-08-09T15:12:05.553 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:05.550+0000 7f13ceffd640 1 --2- 172.21.15.43:0/463561673 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f13d00d1df0 0x7f13d00ca6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T15:12:05.553 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:05.550+0000 7f13cffff640 1 --2- 172.21.15.43:0/463561673 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f13d00cabf0 0x7f13d00d1170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T15:12:05.553 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:05.550+0000 7f13cf7fe640 1 -- 172.21.15.43:0/463561673 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f13d00cabf0 msgr2=0x7f13d00d1170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T15:12:05.553 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:05.550+0000 7f13cf7fe640 1 --2- 172.21.15.43:0/463561673 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f13d00cabf0 0x7f13d00d1170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T15:12:05.553 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:05.550+0000 7f13cf7fe640 1 -- 172.21.15.43:0/463561673 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f13d00d1df0 msgr2=0x7f13d00ca6b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T15:12:05.553 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:05.550+0000 7f13cf7fe640 1 --2- 172.21.15.43:0/463561673 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f13d00d1df0 0x7f13d00ca6b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T15:12:05.553 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:05.550+0000 7f13cf7fe640 1 -- 172.21.15.43:0/463561673 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f13c8009b30 con 0x7f13d00cd6e0 2023-08-09T15:12:05.554 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:05.550+0000 7f13cffff640 1 --2- 172.21.15.43:0/463561673 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f13d00cabf0 0x7f13d00d1170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-08-09T15:12:05.554 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:05.550+0000 7f13cf7fe640 1 --2- 172.21.15.43:0/463561673 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f13d00cd6e0 0x7f13d0158e40 secure :-1 s=READY pgs=2502 cs=0 l=1 rev1=1 crypto rx=0x7f13c000c430 tx=0x7f13c000c8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-09T15:12:05.554 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:05.550+0000 7f13b7fff640 1 -- 172.21.15.43:0/463561673 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f13c00052d0 con 0x7f13d00cd6e0 2023-08-09T15:12:05.555 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:05.550+0000 7f13b7fff640 1 -- 172.21.15.43:0/463561673 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f13c0005430 con 0x7f13d00cd6e0 2023-08-09T15:12:05.555 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:05.550+0000 7f13b7fff640 1 -- 172.21.15.43:0/463561673 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f13c0003830 con 0x7f13d00cd6e0 2023-08-09T15:12:05.555 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:05.550+0000 7f13d4a63640 1 -- 172.21.15.43:0/463561673 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f13d01f2f90 con 0x7f13d00cd6e0 2023-08-09T15:12:05.555 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:05.551+0000 7f13d4a63640 1 -- 172.21.15.43:0/463561673 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f13d00c6310 con 0x7f13d00cd6e0 2023-08-09T15:12:05.555 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:05.552+0000 7f13d4a63640 1 -- 172.21.15.43:0/463561673 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f13d004e390 con 0x7f13d00cd6e0 2023-08-09T15:12:05.555 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:05.553+0000 7f13b7fff640 1 -- 172.21.15.43:0/463561673 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 125) v1 ==== 88557+0+0 (secure 0 0 0) 0x7f13c0003990 con 0x7f13d00cd6e0 2023-08-09T15:12:05.556 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:05.554+0000 7f13b7fff640 1 --2- 172.21.15.43:0/463561673 >> [v2:172.21.15.203:6832/2375771024,v1:172.21.15.203:6833/2375771024] conn(0x7f13bc06a290 0x7f13bc06c750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T15:12:05.556 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:05.554+0000 7f13b7fff640 1 -- 172.21.15.43:0/463561673 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(224..224 src has 1..224) v4 ==== 6770+0+0 (secure 0 0 0) 0x7f13c00880a0 con 0x7f13d00cd6e0 2023-08-09T15:12:05.557 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:05.554+0000 7f13ceffd640 1 -- 172.21.15.43:0/463561673 >> [v2:172.21.15.203:6832/2375771024,v1:172.21.15.203:6833/2375771024] conn(0x7f13bc06a290 msgr2=0x7f13bc06c750 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.203:6832/2375771024 2023-08-09T15:12:05.557 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:05.554+0000 7f13ceffd640 1 --2- 172.21.15.43:0/463561673 >> [v2:172.21.15.203:6832/2375771024,v1:172.21.15.203:6833/2375771024] conn(0x7f13bc06a290 0x7f13bc06c750 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-08-09T15:12:05.560 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:05.558+0000 7f13b7fff640 1 -- 172.21.15.43:0/463561673 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178066 (secure 0 0 0) 0x7f13c00532c0 con 0x7f13d00cd6e0 2023-08-09T15:12:05.580 INFO:tasks.ceph.mgr.x.smithi203.stderr:2023-08-09T15:12:05.578+0000 7efcabc1b040 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2023-08-09T15:12:05.608 INFO:tasks.ceph.mgr.y.smithi043.stderr:2023-08-09T15:12:05.606+0000 7ff1ea851040 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2023-08-09T15:12:05.629 INFO:tasks.ceph.mgr.x.smithi203.stderr:2023-08-09T15:12:05.627+0000 7efcabc1b040 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2023-08-09T15:12:05.721 INFO:tasks.ceph.mgr.y.smithi043.stderr:2023-08-09T15:12:05.720+0000 7ff1ea851040 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2023-08-09T15:12:05.750 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:05.749+0000 7f13d4a63640 1 -- 172.21.15.43:0/463561673 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f13d00cfd90 con 0x7f13d00cd6e0 2023-08-09T15:12:05.751 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:05.750+0000 7f13b7fff640 1 -- 172.21.15.43:0/463561673 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v224) v1 ==== 100+0+11 (secure 0 0 0) 0x7f13c00568e0 con 0x7f13d00cd6e0 2023-08-09T15:12:05.751 INFO:teuthology.orchestra.run.smithi043.stdout:pg_num: 64 2023-08-09T15:12:05.753 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:05.752+0000 7f13d4a63640 1 -- 172.21.15.43:0/463561673 >> [v2:172.21.15.203:6832/2375771024,v1:172.21.15.203:6833/2375771024] conn(0x7f13bc06a290 msgr2=0x7f13bc06c750 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-08-09T15:12:05.754 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:05.752+0000 7f13d4a63640 1 --2- 172.21.15.43:0/463561673 >> [v2:172.21.15.203:6832/2375771024,v1:172.21.15.203:6833/2375771024] conn(0x7f13bc06a290 0x7f13bc06c750 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T15:12:05.754 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:05.752+0000 7f13d4a63640 1 -- 172.21.15.43:0/463561673 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f13d00cd6e0 msgr2=0x7f13d0158e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T15:12:05.754 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:05.752+0000 7f13d4a63640 1 --2- 172.21.15.43:0/463561673 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f13d00cd6e0 0x7f13d0158e40 secure :-1 s=READY pgs=2502 cs=0 l=1 rev1=1 crypto rx=0x7f13c000c430 tx=0x7f13c000c8f0 comp rx=0 tx=0).stop 2023-08-09T15:12:05.754 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:05.752+0000 7f13d4a63640 1 -- 172.21.15.43:0/463561673 shutdown_connections 2023-08-09T15:12:05.754 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:05.752+0000 7f13d4a63640 1 --2- 172.21.15.43:0/463561673 >> [v2:172.21.15.203:6832/2375771024,v1:172.21.15.203:6833/2375771024] conn(0x7f13bc06a290 0x7f13bc06c750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T15:12:05.754 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:05.752+0000 7f13d4a63640 1 --2- 172.21.15.43:0/463561673 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f13d00cabf0 0x7f13d00d1170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T15:12:05.754 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:05.752+0000 7f13d4a63640 1 --2- 172.21.15.43:0/463561673 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f13d00d1df0 0x7f13d00ca6b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T15:12:05.755 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:05.752+0000 7f13d4a63640 1 --2- 172.21.15.43:0/463561673 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f13d00cd6e0 0x7f13d0158e40 unknown :-1 s=CLOSED pgs=2502 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T15:12:05.755 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:05.752+0000 7f13d4a63640 1 -- 172.21.15.43:0/463561673 >> 172.21.15.43:0/463561673 conn(0x7f13d0071260 msgr2=0x7f13d00b5bc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-09T15:12:05.755 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:05.752+0000 7f13d4a63640 1 -- 172.21.15.43:0/463561673 shutdown_connections 2023-08-09T15:12:05.755 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:05.753+0000 7f13d4a63640 1 -- 172.21.15.43:0/463561673 wait complete. 2023-08-09T15:12:05.772 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2023-08-09T15:12:05.777 INFO:tasks.ceph.mgr.y.smithi043.stderr:2023-08-09T15:12:05.773+0000 7ff1ea851040 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2023-08-09T15:12:05.787 INFO:tasks.ceph.mgr.x.smithi203.stderr:2023-08-09T15:12:05.786+0000 7efcabc1b040 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2023-08-09T15:12:05.945 INFO:tasks.ceph.mgr.y.smithi043.stderr:2023-08-09T15:12:05.943+0000 7ff1ea851040 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2023-08-09T15:12:05.986 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:05.984+0000 7fd6def6a640 1 Processor -- start 2023-08-09T15:12:05.987 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:05.985+0000 7fd6def6a640 1 -- start start 2023-08-09T15:12:05.988 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:05.986+0000 7fd6def6a640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7fd6d815a560 0x7fd6d806c470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T15:12:05.989 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:05.986+0000 7fd6def6a640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fd6d80b3ef0 0x7fd6d806c9b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T15:12:05.989 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:05.986+0000 7fd6def6a640 1 -- --> v1:172.21.15.203:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd6d80697d0 con 0x7fd6d80cb120 2023-08-09T15:12:05.989 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:05.986+0000 7fd6def6a640 1 -- --> v2:172.21.15.43:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd6d806d350 con 0x7fd6d815a560 2023-08-09T15:12:05.989 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:05.986+0000 7fd6def6a640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd6d80c9fe0 con 0x7fd6d80b3ef0 2023-08-09T15:12:05.989 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:05.987+0000 7fd6dcd06640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7fd6d815a560 0x7fd6d806c470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T15:12:05.990 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:05.987+0000 7fd6dcd06640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7fd6d815a560 0x7fd6d806c470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:33186/0 (socket says 172.21.15.43:33186) 2023-08-09T15:12:05.990 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:05.987+0000 7fd6dcd06640 1 -- 172.21.15.43:0/1241854499 learned_addr learned my addr 172.21.15.43:0/1241854499 (peer_addr_for_me v2:172.21.15.43:0/0) 2023-08-09T15:12:05.990 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:05.987+0000 7fd6dd507640 1 --2- 172.21.15.43:0/1241854499 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fd6d80b3ef0 0x7fd6d806c9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T15:12:05.990 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:05.987+0000 7fd6dcd06640 1 -- 172.21.15.43:0/1241854499 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fd6d80b3ef0 msgr2=0x7fd6d806c9b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-09T15:12:05.990 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:05.987+0000 7fd6dcd06640 1 --2- 172.21.15.43:0/1241854499 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fd6d80b3ef0 0x7fd6d806c9b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T15:12:05.991 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:05.987+0000 7fd6dcd06640 1 -- 172.21.15.43:0/1241854499 >> v1:172.21.15.203:6789/0 conn(0x7fd6d80cb120 legacy=0x7fd6d8069c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-09T15:12:05.991 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:05.987+0000 7fd6dcd06640 1 -- 172.21.15.43:0/1241854499 --> v2:172.21.15.43:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd6d815a940 con 0x7fd6d815a560 2023-08-09T15:12:05.992 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:05.988+0000 7fd6dcd06640 1 --2- 172.21.15.43:0/1241854499 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fd6d815a560 0x7fd6d806c470 secure :-1 s=READY pgs=2503 cs=0 l=1 rev1=1 crypto rx=0x7fd6c8009870 tx=0x7fd6c802b7e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f7fa8f81e5e0d214 server_cookie=0 in_seq=0 out_seq=0 2023-08-09T15:12:05.992 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:05.989+0000 7fd6d77fe640 1 -- 172.21.15.43:0/1241854499 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd6c8031740 con 0x7fd6d815a560 2023-08-09T15:12:05.992 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:05.989+0000 7fd6d77fe640 1 -- 172.21.15.43:0/1241854499 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd6c80318a0 con 0x7fd6d815a560 2023-08-09T15:12:05.992 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:05.989+0000 7fd6d77fe640 1 -- 172.21.15.43:0/1241854499 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd6c8031c30 con 0x7fd6d815a560 2023-08-09T15:12:05.993 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:05.989+0000 7fd6def6a640 1 -- 172.21.15.43:0/1241854499 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fd6d815a560 msgr2=0x7fd6d806c470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T15:12:05.993 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:05.989+0000 7fd6def6a640 1 --2- 172.21.15.43:0/1241854499 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fd6d815a560 0x7fd6d806c470 secure :-1 s=READY pgs=2503 cs=0 l=1 rev1=1 crypto rx=0x7fd6c8009870 tx=0x7fd6c802b7e0 comp rx=0 tx=0).stop 2023-08-09T15:12:05.993 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:05.989+0000 7fd6def6a640 1 -- 172.21.15.43:0/1241854499 shutdown_connections 2023-08-09T15:12:05.993 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:05.989+0000 7fd6def6a640 1 --2- 172.21.15.43:0/1241854499 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fd6d80b3ef0 0x7fd6d806c9b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T15:12:05.994 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:05.989+0000 7fd6def6a640 1 --2- 172.21.15.43:0/1241854499 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fd6d815a560 0x7fd6d806c470 unknown :-1 s=CLOSED pgs=2503 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T15:12:05.994 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:05.989+0000 7fd6def6a640 1 -- 172.21.15.43:0/1241854499 >> 172.21.15.43:0/1241854499 conn(0x7fd6d80cb5b0 msgr2=0x7fd6d80cb990 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-09T15:12:05.994 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:05.990+0000 7fd6def6a640 1 -- 172.21.15.43:0/1241854499 shutdown_connections 2023-08-09T15:12:05.994 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:05.990+0000 7fd6def6a640 1 -- 172.21.15.43:0/1241854499 wait complete. 2023-08-09T15:12:05.997 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:05.991+0000 7fd6def6a640 1 Processor -- start 2023-08-09T15:12:05.998 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:05.992+0000 7fd6def6a640 1 -- start start 2023-08-09T15:12:05.998 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:05.992+0000 7fd6def6a640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fd6d80b3ef0 0x7fd6d80c7a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T15:12:05.998 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:05.992+0000 7fd6def6a640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fd6d80cb120 0x7fd6d80c7f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T15:12:05.999 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:05.993+0000 7fd6def6a640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fd6d815a560 0x7fd6d80c5c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T15:12:05.999 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:05.993+0000 7fd6def6a640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd6d80c8ad0 con 0x7fd6d80b3ef0 2023-08-09T15:12:05.999 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:05.993+0000 7fd6def6a640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd6d81569e0 con 0x7fd6d815a560 2023-08-09T15:12:05.999 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:05.993+0000 7fd6def6a640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd6d8155350 con 0x7fd6d80cb120 2023-08-09T15:12:05.999 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:05.993+0000 7fd6dcd06640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fd6d80b3ef0 0x7fd6d80c7a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T15:12:06.000 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:05.993+0000 7fd6dcd06640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fd6d80b3ef0 0x7fd6d80c7a10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:33190/0 (socket says 172.21.15.43:33190) 2023-08-09T15:12:06.000 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:05.993+0000 7fd6dcd06640 1 -- 172.21.15.43:0/1174693566 learned_addr learned my addr 172.21.15.43:0/1174693566 (peer_addr_for_me v2:172.21.15.43:0/0) 2023-08-09T15:12:06.000 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:05.993+0000 7fd6d7fff640 1 --2- 172.21.15.43:0/1174693566 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fd6d80cb120 0x7fd6d80c7f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T15:12:06.000 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:05.993+0000 7fd6dd507640 1 --2- 172.21.15.43:0/1174693566 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fd6d815a560 0x7fd6d80c5c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T15:12:06.000 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:05.993+0000 7fd6dcd06640 1 -- 172.21.15.43:0/1174693566 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fd6d80cb120 msgr2=0x7fd6d80c7f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T15:12:06.001 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:05.993+0000 7fd6dcd06640 1 --2- 172.21.15.43:0/1174693566 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fd6d80cb120 0x7fd6d80c7f50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T15:12:06.001 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:05.993+0000 7fd6dcd06640 1 -- 172.21.15.43:0/1174693566 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fd6d815a560 msgr2=0x7fd6d80c5c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T15:12:06.001 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:05.993+0000 7fd6dcd06640 1 --2- 172.21.15.43:0/1174693566 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fd6d815a560 0x7fd6d80c5c00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T15:12:06.001 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:05.993+0000 7fd6dcd06640 1 -- 172.21.15.43:0/1174693566 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd6c80094d0 con 0x7fd6d80b3ef0 2023-08-09T15:12:06.002 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:05.993+0000 7fd6dd507640 1 --2- 172.21.15.43:0/1174693566 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fd6d815a560 0x7fd6d80c5c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-08-09T15:12:06.002 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:05.994+0000 7fd6dcd06640 1 --2- 172.21.15.43:0/1174693566 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fd6d80b3ef0 0x7fd6d80c7a10 secure :-1 s=READY pgs=2504 cs=0 l=1 rev1=1 crypto rx=0x7fd6c802b820 tx=0x7fd6c80046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-09T15:12:06.002 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:05.994+0000 7fd6d57fa640 1 -- 172.21.15.43:0/1174693566 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd6c8033c70 con 0x7fd6d80b3ef0 2023-08-09T15:12:06.002 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:05.994+0000 7fd6d57fa640 1 -- 172.21.15.43:0/1174693566 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd6c8033dd0 con 0x7fd6d80b3ef0 2023-08-09T15:12:06.002 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:05.994+0000 7fd6d57fa640 1 -- 172.21.15.43:0/1174693566 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd6c803c640 con 0x7fd6d80b3ef0 2023-08-09T15:12:06.003 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:05.994+0000 7fd6def6a640 1 -- 172.21.15.43:0/1174693566 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd6d80c87f0 con 0x7fd6d80b3ef0 2023-08-09T15:12:06.003 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:05.994+0000 7fd6def6a640 1 -- 172.21.15.43:0/1174693566 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd6d8156670 con 0x7fd6d80b3ef0 2023-08-09T15:12:06.003 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:05.995+0000 7fd6def6a640 1 -- 172.21.15.43:0/1174693566 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd6d8156800 con 0x7fd6d80b3ef0 2023-08-09T15:12:06.003 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:05.996+0000 7fd6d57fa640 1 -- 172.21.15.43:0/1174693566 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 125) v1 ==== 88557+0+0 (secure 0 0 0) 0x7fd6c803a040 con 0x7fd6d80b3ef0 2023-08-09T15:12:06.004 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:05.997+0000 7fd6d57fa640 1 --2- 172.21.15.43:0/1174693566 >> [v2:172.21.15.203:6832/2375771024,v1:172.21.15.203:6833/2375771024] conn(0x7fd6c406a290 0x7fd6c406c750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T15:12:06.005 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:05.997+0000 7fd6d57fa640 1 -- 172.21.15.43:0/1174693566 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(224..224 src has 1..224) v4 ==== 6770+0+0 (secure 0 0 0) 0x7fd6c80aa840 con 0x7fd6d80b3ef0 2023-08-09T15:12:06.005 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:05.997+0000 7fd6d7fff640 1 -- 172.21.15.43:0/1174693566 >> [v2:172.21.15.203:6832/2375771024,v1:172.21.15.203:6833/2375771024] conn(0x7fd6c406a290 msgr2=0x7fd6c406c750 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.203:6832/2375771024 2023-08-09T15:12:06.005 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:05.998+0000 7fd6d7fff640 1 --2- 172.21.15.43:0/1174693566 >> [v2:172.21.15.203:6832/2375771024,v1:172.21.15.203:6833/2375771024] conn(0x7fd6c406a290 0x7fd6c406c750 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-08-09T15:12:06.005 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:06.001+0000 7fd6d57fa640 1 -- 172.21.15.43:0/1174693566 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178066 (secure 0 0 0) 0x7fd6c80757e0 con 0x7fd6d80b3ef0 2023-08-09T15:12:06.169 INFO:tasks.ceph.mgr.x.smithi203.stderr:2023-08-09T15:12:06.166+0000 7efcabc1b040 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2023-08-09T15:12:06.200 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:06.199+0000 7fd6d7fff640 1 -- 172.21.15.43:0/1174693566 >> [v2:172.21.15.203:6832/2375771024,v1:172.21.15.203:6833/2375771024] conn(0x7fd6c406a290 msgr2=0x7fd6c406c750 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.203:6832/2375771024 2023-08-09T15:12:06.201 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:06.199+0000 7fd6d7fff640 1 --2- 172.21.15.43:0/1174693566 >> [v2:172.21.15.203:6832/2375771024,v1:172.21.15.203:6833/2375771024] conn(0x7fd6c406a290 0x7fd6c406c750 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2023-08-09T15:12:06.210 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:06.208+0000 7fd6def6a640 1 -- 172.21.15.43:0/1174693566 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7fd6d81561f0 con 0x7fd6d80b3ef0 2023-08-09T15:12:06.211 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:06.209+0000 7fd6d57fa640 1 -- 172.21.15.43:0/1174693566 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 8 entries v224) v1 ==== 80+0+476 (secure 0 0 0) 0x7fd6c802f1f0 con 0x7fd6d80b3ef0 2023-08-09T15:12:06.211 INFO:teuthology.orchestra.run.smithi043.stdout:172.21.15.203:6832/4034093268 2023-08-10T15:11:43.130145+0000 2023-08-09T15:12:06.211 INFO:teuthology.orchestra.run.smithi043.stdout:172.21.15.203:6833/4034093268 2023-08-10T15:11:43.130145+0000 2023-08-09T15:12:06.211 INFO:teuthology.orchestra.run.smithi043.stdout:172.21.15.203:0/1654193324 2023-08-10T15:11:43.130145+0000 2023-08-09T15:12:06.212 INFO:teuthology.orchestra.run.smithi043.stdout:172.21.15.203:0/4140323243 2023-08-10T15:11:43.130145+0000 2023-08-09T15:12:06.212 INFO:teuthology.orchestra.run.smithi043.stdout:172.21.15.203:0/450863258 2023-08-10T15:11:43.130145+0000 2023-08-09T15:12:06.212 INFO:teuthology.orchestra.run.smithi043.stdout:172.21.15.203:0/134877587 2023-08-10T15:11:43.130145+0000 2023-08-09T15:12:06.212 INFO:teuthology.orchestra.run.smithi043.stdout:172.21.15.203:0/2330869600 2023-08-10T15:11:43.130145+0000 2023-08-09T15:12:06.212 INFO:teuthology.orchestra.run.smithi043.stdout:172.21.15.203:0/2203818837 2023-08-10T15:11:43.130145+0000 2023-08-09T15:12:06.212 INFO:teuthology.orchestra.run.smithi043.stderr:listed 8 entries 2023-08-09T15:12:06.214 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:06.213+0000 7fd6def6a640 1 -- 172.21.15.43:0/1174693566 >> [v2:172.21.15.203:6832/2375771024,v1:172.21.15.203:6833/2375771024] conn(0x7fd6c406a290 msgr2=0x7fd6c406c750 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-08-09T15:12:06.215 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:06.213+0000 7fd6def6a640 1 --2- 172.21.15.43:0/1174693566 >> [v2:172.21.15.203:6832/2375771024,v1:172.21.15.203:6833/2375771024] conn(0x7fd6c406a290 0x7fd6c406c750 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T15:12:06.215 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:06.213+0000 7fd6def6a640 1 -- 172.21.15.43:0/1174693566 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fd6d80b3ef0 msgr2=0x7fd6d80c7a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T15:12:06.215 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:06.213+0000 7fd6def6a640 1 --2- 172.21.15.43:0/1174693566 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fd6d80b3ef0 0x7fd6d80c7a10 secure :-1 s=READY pgs=2504 cs=0 l=1 rev1=1 crypto rx=0x7fd6c802b820 tx=0x7fd6c80046c0 comp rx=0 tx=0).stop 2023-08-09T15:12:06.215 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:06.213+0000 7fd6def6a640 1 -- 172.21.15.43:0/1174693566 shutdown_connections 2023-08-09T15:12:06.215 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:06.213+0000 7fd6def6a640 1 --2- 172.21.15.43:0/1174693566 >> [v2:172.21.15.203:6832/2375771024,v1:172.21.15.203:6833/2375771024] conn(0x7fd6c406a290 0x7fd6c406c750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T15:12:06.215 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:06.213+0000 7fd6def6a640 1 --2- 172.21.15.43:0/1174693566 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fd6d815a560 0x7fd6d80c5c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T15:12:06.215 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:06.213+0000 7fd6def6a640 1 --2- 172.21.15.43:0/1174693566 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fd6d80cb120 0x7fd6d80c7f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T15:12:06.216 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:06.213+0000 7fd6def6a640 1 --2- 172.21.15.43:0/1174693566 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fd6d80b3ef0 0x7fd6d80c7a10 unknown :-1 s=CLOSED pgs=2504 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T15:12:06.216 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:06.213+0000 7fd6def6a640 1 -- 172.21.15.43:0/1174693566 >> 172.21.15.43:0/1174693566 conn(0x7fd6d80cb5b0 msgr2=0x7fd6d80b2cc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-09T15:12:06.216 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:06.213+0000 7fd6def6a640 1 -- 172.21.15.43:0/1174693566 shutdown_connections 2023-08-09T15:12:06.216 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:06.213+0000 7fd6def6a640 1 -- 172.21.15.43:0/1174693566 wait complete. 2023-08-09T15:12:06.244 INFO:tasks.cephfs.fuse_mount:Running fusermount -u on ubuntu@smithi043.front.sepia.ceph.com... 2023-08-09T15:12:06.244 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-09T15:12:06.244 DEBUG:teuthology.orchestra.run.smithi043:> sudo fusermount -u /home/ubuntu/cephtest/mnt.0 2023-08-09T15:12:06.333 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi043.stderr:ceph-fuse[154969]: fuse finished with error 0 and tester_r 0 2023-08-09T15:12:06.339 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi043.front.sepia.ceph.com 2023-08-09T15:12:06.340 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-09T15:12:06.340 DEBUG:teuthology.orchestra.run.smithi043:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2023-08-09T15:12:06.367 INFO:tasks.ceph.mgr.y.smithi043.stderr:2023-08-09T15:12:06.366+0000 7ff1ea851040 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2023-08-09T15:12:06.431 INFO:tasks.ceph.mgr.x.smithi203.stderr:2023-08-09T15:12:06.430+0000 7efcabc1b040 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2023-08-09T15:12:06.487 INFO:tasks.ceph.mgr.x.smithi203.stderr:2023-08-09T15:12:06.486+0000 7efcabc1b040 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2023-08-09T15:12:06.488 DEBUG:teuthology.orchestra.run.smithi203:> set -ex 2023-08-09T15:12:06.489 DEBUG:teuthology.orchestra.run.smithi203:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-09T15:12:06.541 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4053648,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497576k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=27,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248784k,nr_inodes=812196,mode=700,uid=1000,gid=1275 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497576k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2023-08-09T15:12:06.542 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi203.front.sepia.ceph.com 2023-08-09T15:12:06.542 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-09T15:12:06.542 DEBUG:teuthology.orchestra.run.smithi203:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.1) 2023-08-09T15:12:06.543 INFO:tasks.ceph.mgr.x.smithi203.stderr:2023-08-09T15:12:06.541+0000 7efcabc1b040 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2023-08-09T15:12:06.592 INFO:teuthology.orchestra.run.smithi203.stderr:rmdir: failed to remove '/home/ubuntu/cephtest/mnt.1': No such file or directory 2023-08-09T15:12:06.593 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-08-09T15:12:06.594 DEBUG:teuthology.orchestra.run.smithi043:> set -ex 2023-08-09T15:12:06.594 DEBUG:teuthology.orchestra.run.smithi043:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-09T15:12:06.601 INFO:tasks.ceph.mgr.x.smithi203.stderr:2023-08-09T15:12:06.600+0000 7efcabc1b040 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2023-08-09T15:12:06.604 INFO:tasks.ceph.mgr.y.smithi043.stderr:2023-08-09T15:12:06.599+0000 7ff1ea851040 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2023-08-09T15:12:06.652 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4054441,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6498844k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=27,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3249420k,nr_inodes=812355,mode=700,uid=1000,gid=1275 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6498844k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2023-08-09T15:12:06.653 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi043.front.sepia.ceph.com 2023-08-09T15:12:06.653 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-09T15:12:06.654 DEBUG:teuthology.orchestra.run.smithi043:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.2) 2023-08-09T15:12:06.660 INFO:tasks.ceph.mgr.y.smithi043.stderr:2023-08-09T15:12:06.659+0000 7ff1ea851040 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2023-08-09T15:12:06.706 INFO:teuthology.orchestra.run.smithi043.stderr:rmdir: failed to remove '/home/ubuntu/cephtest/mnt.2': No such file or directory 2023-08-09T15:12:06.707 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-08-09T15:12:06.708 DEBUG:teuthology.orchestra.run.smithi203:> set -ex 2023-08-09T15:12:06.708 DEBUG:teuthology.orchestra.run.smithi203:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-09T15:12:06.713 INFO:tasks.ceph.mgr.y.smithi043.stderr:2023-08-09T15:12:06.712+0000 7ff1ea851040 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2023-08-09T15:12:06.761 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4053648,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497576k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=27,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248784k,nr_inodes=812196,mode=700,uid=1000,gid=1275 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497576k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2023-08-09T15:12:06.762 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi203.front.sepia.ceph.com 2023-08-09T15:12:06.763 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-09T15:12:06.764 DEBUG:teuthology.orchestra.run.smithi203:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.3) 2023-08-09T15:12:06.768 INFO:tasks.ceph.mgr.y.smithi043.stderr:2023-08-09T15:12:06.766+0000 7ff1ea851040 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2023-08-09T15:12:06.813 INFO:teuthology.orchestra.run.smithi203.stderr:rmdir: failed to remove '/home/ubuntu/cephtest/mnt.3': No such file or directory 2023-08-09T15:12:06.813 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-08-09T15:12:06.814 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-08-09T15:12:06.904 INFO:tasks.ceph.mgr.x.smithi203.stderr:2023-08-09T15:12:06.902+0000 7efcabc1b040 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2023-08-09T15:12:06.961 INFO:tasks.ceph.mgr.x.smithi203.stderr:2023-08-09T15:12:06.960+0000 7efcabc1b040 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2023-08-09T15:12:07.012 INFO:tasks.ceph.mgr.x.smithi203.stderr:2023-08-09T15:12:07.011+0000 7efcabc1b040 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2023-08-09T15:12:07.038 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:07.036+0000 7ffb98a5d640 1 Processor -- start 2023-08-09T15:12:07.038 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:07.036+0000 7ffb98a5d640 1 -- start start 2023-08-09T15:12:07.040 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:07.038+0000 7ffb98a5d640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7ffb9415a4c0 0x7ffb940ca890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T15:12:07.040 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:07.038+0000 7ffb98a5d640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7ffb940cfc20 0x7ffb9415dec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T15:12:07.041 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:07.038+0000 7ffb98a5d640 1 -- --> v1:172.21.15.43:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ffb9415aa20 con 0x7ffb940cadd0 2023-08-09T15:12:07.041 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:07.038+0000 7ffb98a5d640 1 -- --> v2:172.21.15.43:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ffb9415b660 con 0x7ffb940cfc20 2023-08-09T15:12:07.042 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:07.038+0000 7ffb98a5d640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ffb9415acc0 con 0x7ffb9415a4c0 2023-08-09T15:12:07.042 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:07.038+0000 7ffb8bfff640 1 --1- >> v1:172.21.15.43:6789/0 conn(0x7ffb940cadd0 0x7ffb940cf510 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.43:6789/0 says I am v1:172.21.15.43:36664/0 (socket says 172.21.15.43:36664) 2023-08-09T15:12:07.042 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:07.038+0000 7ffb8bfff640 1 -- 172.21.15.43:0/2628018876 learned_addr learned my addr 172.21.15.43:0/2628018876 (peer_addr_for_me v1:172.21.15.43:0/0) 2023-08-09T15:12:07.043 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:07.038+0000 7ffb93fff640 1 --2- 172.21.15.43:0/2628018876 >> v2:172.21.15.43:3300/0 conn(0x7ffb940cfc20 0x7ffb9415dec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T15:12:07.043 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:07.039+0000 7ffb937fe640 1 --2- 172.21.15.43:0/2628018876 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7ffb9415a4c0 0x7ffb940ca890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T15:12:07.045 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:07.039+0000 7ffb93fff640 1 -- 172.21.15.43:0/2628018876 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7ffb9415a4c0 msgr2=0x7ffb940ca890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-09T15:12:07.045 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:07.039+0000 7ffb93fff640 1 --2- 172.21.15.43:0/2628018876 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7ffb9415a4c0 0x7ffb940ca890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T15:12:07.046 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:07.039+0000 7ffb93fff640 1 -- 172.21.15.43:0/2628018876 >> v1:172.21.15.43:6789/0 conn(0x7ffb940cadd0 legacy=0x7ffb940cf510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T15:12:07.046 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:07.039+0000 7ffb93fff640 1 -- 172.21.15.43:0/2628018876 --> v2:172.21.15.43:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ffb9415afa0 con 0x7ffb940cfc20 2023-08-09T15:12:07.046 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:07.039+0000 7ffb92ffd640 1 -- 172.21.15.43:0/2628018876 <== mon.0 v1:172.21.15.43:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2422639189 0 0) 0x7ffb9415aa20 con 0x7ffb940cadd0 2023-08-09T15:12:07.047 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:07.041+0000 7ffb93fff640 1 --2- 172.21.15.43:0/2628018876 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ffb940cfc20 0x7ffb9415dec0 secure :-1 s=READY pgs=2506 cs=0 l=1 rev1=1 crypto rx=0x7ffb8c009d90 tx=0x7ffb8c02bea0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cb40027281c4ad92 server_cookie=0 in_seq=0 out_seq=0 2023-08-09T15:12:07.047 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:07.041+0000 7ffb92ffd640 1 -- 172.21.15.43:0/2628018876 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffb8c02cc50 con 0x7ffb940cfc20 2023-08-09T15:12:07.047 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:07.041+0000 7ffb92ffd640 1 -- 172.21.15.43:0/2628018876 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ffb8c02cdb0 con 0x7ffb940cfc20 2023-08-09T15:12:07.047 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:07.041+0000 7ffb92ffd640 1 -- 172.21.15.43:0/2628018876 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffb8c034660 con 0x7ffb940cfc20 2023-08-09T15:12:07.048 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:07.042+0000 7ffb98a5d640 1 -- 172.21.15.43:0/2628018876 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ffb940cfc20 msgr2=0x7ffb9415dec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T15:12:07.048 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:07.042+0000 7ffb98a5d640 1 --2- 172.21.15.43:0/2628018876 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ffb940cfc20 0x7ffb9415dec0 secure :-1 s=READY pgs=2506 cs=0 l=1 rev1=1 crypto rx=0x7ffb8c009d90 tx=0x7ffb8c02bea0 comp rx=0 tx=0).stop 2023-08-09T15:12:07.048 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:07.042+0000 7ffb98a5d640 1 -- 172.21.15.43:0/2628018876 shutdown_connections 2023-08-09T15:12:07.049 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:07.042+0000 7ffb98a5d640 1 --2- 172.21.15.43:0/2628018876 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ffb940cfc20 0x7ffb9415dec0 unknown :-1 s=CLOSED pgs=2506 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T15:12:07.049 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:07.042+0000 7ffb98a5d640 1 --2- 172.21.15.43:0/2628018876 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7ffb9415a4c0 0x7ffb940ca890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T15:12:07.049 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:07.042+0000 7ffb98a5d640 1 -- 172.21.15.43:0/2628018876 >> 172.21.15.43:0/2628018876 conn(0x7ffb9406a190 msgr2=0x7ffb940cccc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-09T15:12:07.049 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:07.042+0000 7ffb98a5d640 1 -- 172.21.15.43:0/2628018876 shutdown_connections 2023-08-09T15:12:07.050 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:07.042+0000 7ffb98a5d640 1 -- 172.21.15.43:0/2628018876 wait complete. 2023-08-09T15:12:07.050 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:07.044+0000 7ffb98a5d640 1 Processor -- start 2023-08-09T15:12:07.050 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:07.044+0000 7ffb98a5d640 1 -- start start 2023-08-09T15:12:07.054 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:07.045+0000 7ffb98a5d640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7ffb940cadd0 0x7ffb940af180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T15:12:07.054 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:07.045+0000 7ffb98a5d640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7ffb940cfc20 0x7ffb940b13c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T15:12:07.054 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:07.045+0000 7ffb98a5d640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ffb9415a4c0 0x7ffb940b1900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T15:12:07.055 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:07.045+0000 7ffb98a5d640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffb9406bce0 con 0x7ffb9415a4c0 2023-08-09T15:12:07.055 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:07.045+0000 7ffb98a5d640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffb9406bfc0 con 0x7ffb940cfc20 2023-08-09T15:12:07.055 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:07.045+0000 7ffb98a5d640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ffb940b2e20 con 0x7ffb940cadd0 2023-08-09T15:12:07.055 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:07.045+0000 7ffb93fff640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ffb9415a4c0 0x7ffb940b1900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T15:12:07.055 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:07.045+0000 7ffb93fff640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ffb9415a4c0 0x7ffb940b1900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:33200/0 (socket says 172.21.15.43:33200) 2023-08-09T15:12:07.056 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:07.045+0000 7ffb93fff640 1 -- 172.21.15.43:0/2101432698 learned_addr learned my addr 172.21.15.43:0/2101432698 (peer_addr_for_me v2:172.21.15.43:0/0) 2023-08-09T15:12:07.056 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:07.046+0000 7ffb937fe640 1 --2- 172.21.15.43:0/2101432698 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7ffb940cadd0 0x7ffb940af180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T15:12:07.056 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:07.046+0000 7ffb8bfff640 1 --2- 172.21.15.43:0/2101432698 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7ffb940cfc20 0x7ffb940b13c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T15:12:07.056 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:07.046+0000 7ffb93fff640 1 -- 172.21.15.43:0/2101432698 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7ffb940cadd0 msgr2=0x7ffb940af180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T15:12:07.056 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:07.046+0000 7ffb93fff640 1 --2- 172.21.15.43:0/2101432698 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7ffb940cadd0 0x7ffb940af180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T15:12:07.056 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:07.046+0000 7ffb93fff640 1 -- 172.21.15.43:0/2101432698 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7ffb940cfc20 msgr2=0x7ffb940b13c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T15:12:07.057 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:07.046+0000 7ffb93fff640 1 --2- 172.21.15.43:0/2101432698 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7ffb940cfc20 0x7ffb940b13c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T15:12:07.057 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:07.046+0000 7ffb93fff640 1 -- 172.21.15.43:0/2101432698 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ffb8c009a70 con 0x7ffb9415a4c0 2023-08-09T15:12:07.057 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:07.046+0000 7ffb93fff640 1 --2- 172.21.15.43:0/2101432698 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ffb9415a4c0 0x7ffb940b1900 secure :-1 s=READY pgs=2507 cs=0 l=1 rev1=1 crypto rx=0x7ffb8c009d90 tx=0x7ffb8c004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-09T15:12:07.057 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:07.047+0000 7ffb90ff9640 1 -- 172.21.15.43:0/2101432698 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffb8c040650 con 0x7ffb9415a4c0 2023-08-09T15:12:07.057 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:07.047+0000 7ffb90ff9640 1 -- 172.21.15.43:0/2101432698 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ffb8c0407d0 con 0x7ffb9415a4c0 2023-08-09T15:12:07.057 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:07.047+0000 7ffb90ff9640 1 -- 172.21.15.43:0/2101432698 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffb8c040b80 con 0x7ffb9415a4c0 2023-08-09T15:12:07.057 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:07.047+0000 7ffb98a5d640 1 -- 172.21.15.43:0/2101432698 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ffb940b3100 con 0x7ffb9415a4c0 2023-08-09T15:12:07.058 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:07.047+0000 7ffb98a5d640 1 -- 172.21.15.43:0/2101432698 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ffb94156c80 con 0x7ffb9415a4c0 2023-08-09T15:12:07.058 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:07.048+0000 7ffb98a5d640 1 -- 172.21.15.43:0/2101432698 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ffb94156e10 con 0x7ffb9415a4c0 2023-08-09T15:12:07.058 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:07.050+0000 7ffb90ff9640 1 -- 172.21.15.43:0/2101432698 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 125) v1 ==== 88557+0+0 (secure 0 0 0) 0x7ffb8c040ce0 con 0x7ffb9415a4c0 2023-08-09T15:12:07.058 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:07.051+0000 7ffb90ff9640 1 --2- 172.21.15.43:0/2101432698 >> [v2:172.21.15.203:6832/2375771024,v1:172.21.15.203:6833/2375771024] conn(0x7ffb7406a2c0 0x7ffb7406c780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T15:12:07.058 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:07.051+0000 7ffb90ff9640 1 -- 172.21.15.43:0/2101432698 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(224..224 src has 1..224) v4 ==== 6770+0+0 (secure 0 0 0) 0x7ffb8c0aba80 con 0x7ffb9415a4c0 2023-08-09T15:12:07.058 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:07.052+0000 7ffb937fe640 1 -- 172.21.15.43:0/2101432698 >> [v2:172.21.15.203:6832/2375771024,v1:172.21.15.203:6833/2375771024] conn(0x7ffb7406a2c0 msgr2=0x7ffb7406c780 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.203:6832/2375771024 2023-08-09T15:12:07.058 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:07.052+0000 7ffb937fe640 1 --2- 172.21.15.43:0/2101432698 >> [v2:172.21.15.203:6832/2375771024,v1:172.21.15.203:6833/2375771024] conn(0x7ffb7406a2c0 0x7ffb7406c780 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-08-09T15:12:07.059 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:07.056+0000 7ffb90ff9640 1 -- 172.21.15.43:0/2101432698 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178066 (secure 0 0 0) 0x7ffb8c076b30 con 0x7ffb9415a4c0 2023-08-09T15:12:07.082 INFO:tasks.ceph.mgr.y.smithi043.stderr:2023-08-09T15:12:07.081+0000 7ff1ea851040 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2023-08-09T15:12:07.143 INFO:tasks.ceph.mgr.y.smithi043.stderr:2023-08-09T15:12:07.142+0000 7ff1ea851040 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2023-08-09T15:12:07.171 INFO:tasks.ceph.mgr.x.smithi203.stderr:2023-08-09T15:12:07.170+0000 7efcabc1b040 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2023-08-09T15:12:07.198 INFO:tasks.ceph.mgr.y.smithi043.stderr:2023-08-09T15:12:07.196+0000 7ff1ea851040 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2023-08-09T15:12:07.220 INFO:tasks.ceph.mgr.x.smithi203.stderr:2023-08-09T15:12:07.218+0000 7efcabc1b040 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2023-08-09T15:12:07.254 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:07.253+0000 7ffb937fe640 1 -- 172.21.15.43:0/2101432698 >> [v2:172.21.15.203:6832/2375771024,v1:172.21.15.203:6833/2375771024] conn(0x7ffb7406a2c0 msgr2=0x7ffb7406c780 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.203:6832/2375771024 2023-08-09T15:12:07.255 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:07.253+0000 7ffb937fe640 1 --2- 172.21.15.43:0/2101432698 >> [v2:172.21.15.203:6832/2375771024,v1:172.21.15.203:6833/2375771024] conn(0x7ffb7406a2c0 0x7ffb7406c780 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2023-08-09T15:12:07.272 INFO:tasks.ceph.mgr.x.smithi203.stderr:2023-08-09T15:12:07.270+0000 7efcabc1b040 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2023-08-09T15:12:07.286 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:07.284+0000 7ffb98a5d640 1 -- 172.21.15.43:0/2101432698 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7ffb940b35c0 con 0x7ffb9415a4c0 2023-08-09T15:12:07.286 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:07.285+0000 7ffb90ff9640 1 -- 172.21.15.43:0/2101432698 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 104 v104) v1 ==== 95+0+4894 (secure 0 0 0) 0x7ffb8c03e400 con 0x7ffb9415a4c0 2023-08-09T15:12:07.287 INFO:teuthology.orchestra.run.smithi043.stdout: 2023-08-09T15:12:07.287 INFO:teuthology.orchestra.run.smithi043.stdout:{"epoch":104,"default_fscid":12,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":8012,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.203:6837/2464832767","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6836","nonce":2464832767},{"type":"v1","addr":"172.21.15.203:6837","nonce":2464832767}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":100},{"gid":9010,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.43:6835/3989386049","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6834","nonce":3989386049},{"type":"v1","addr":"172.21.15.43:6835","nonce":3989386049}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":100},{"gid":9016,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.203:6835/2213455074","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6834","nonce":2213455074},{"type":"v1","addr":"172.21.15.203:6835","nonce":2213455074}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":100}],"filesystems":[{"mdsmap":{"epoch":104,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-08-09T15:11:28.168916+0000","modified":"2023-08-09T15:11:31.739094+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":11325},"failed":[],"damaged":[],"stopped":[],"info":{"gid_11325":{"gid":11325,"name":"a","rank":0,"incarnation":101,"state":"up:active","state_seq":3,"addr":"172.21.15.43:6833/1425989175","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6832","nonce":1425989175},{"type":"v1","addr":"172.21.15.43:6833","nonce":1425989175}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[27],"metadata_pool":26,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":12}]} 2023-08-09T15:12:07.287 INFO:teuthology.orchestra.run.smithi043.stderr:dumped fsmap epoch 104 2023-08-09T15:12:07.289 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:07.287+0000 7ffb98a5d640 1 -- 172.21.15.43:0/2101432698 >> [v2:172.21.15.203:6832/2375771024,v1:172.21.15.203:6833/2375771024] conn(0x7ffb7406a2c0 msgr2=0x7ffb7406c780 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-08-09T15:12:07.290 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:07.287+0000 7ffb98a5d640 1 --2- 172.21.15.43:0/2101432698 >> [v2:172.21.15.203:6832/2375771024,v1:172.21.15.203:6833/2375771024] conn(0x7ffb7406a2c0 0x7ffb7406c780 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T15:12:07.290 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:07.287+0000 7ffb98a5d640 1 -- 172.21.15.43:0/2101432698 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ffb9415a4c0 msgr2=0x7ffb940b1900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T15:12:07.290 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:07.287+0000 7ffb98a5d640 1 --2- 172.21.15.43:0/2101432698 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ffb9415a4c0 0x7ffb940b1900 secure :-1 s=READY pgs=2507 cs=0 l=1 rev1=1 crypto rx=0x7ffb8c009d90 tx=0x7ffb8c004a40 comp rx=0 tx=0).stop 2023-08-09T15:12:07.290 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:07.287+0000 7ffb98a5d640 1 -- 172.21.15.43:0/2101432698 shutdown_connections 2023-08-09T15:12:07.290 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:07.287+0000 7ffb98a5d640 1 --2- 172.21.15.43:0/2101432698 >> [v2:172.21.15.203:6832/2375771024,v1:172.21.15.203:6833/2375771024] conn(0x7ffb7406a2c0 0x7ffb7406c780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T15:12:07.290 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:07.287+0000 7ffb98a5d640 1 --2- 172.21.15.43:0/2101432698 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ffb9415a4c0 0x7ffb940b1900 unknown :-1 s=CLOSED pgs=2507 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T15:12:07.291 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:07.287+0000 7ffb98a5d640 1 --2- 172.21.15.43:0/2101432698 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7ffb940cfc20 0x7ffb940b13c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T15:12:07.291 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:07.287+0000 7ffb98a5d640 1 --2- 172.21.15.43:0/2101432698 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7ffb940cadd0 0x7ffb940af180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T15:12:07.291 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:07.287+0000 7ffb98a5d640 1 -- 172.21.15.43:0/2101432698 >> 172.21.15.43:0/2101432698 conn(0x7ffb9406a190 msgr2=0x7ffb9415d380 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-09T15:12:07.291 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:07.288+0000 7ffb98a5d640 1 -- 172.21.15.43:0/2101432698 shutdown_connections 2023-08-09T15:12:07.291 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:07.288+0000 7ffb98a5d640 1 -- 172.21.15.43:0/2101432698 wait complete. 2023-08-09T15:12:07.306 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-08-09T15:12:07.321 INFO:tasks.ceph.mgr.x.smithi203.stderr:2023-08-09T15:12:07.320+0000 7efcabc1b040 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2023-08-09T15:12:07.374 INFO:tasks.ceph.mgr.y.smithi043.stderr:2023-08-09T15:12:07.373+0000 7ff1ea851040 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2023-08-09T15:12:07.412 INFO:tasks.ceph.mgr.x.smithi203.stderr:2023-08-09T15:12:07.411+0000 7efc314cc640 -1 client.0 error registering admin socket command: (17) File exists 2023-08-09T15:12:07.412 INFO:tasks.ceph.mgr.x.smithi203.stderr:2023-08-09T15:12:07.411+0000 7efc314cc640 -1 client.0 error registering admin socket command: (17) File exists 2023-08-09T15:12:07.412 INFO:tasks.ceph.mgr.x.smithi203.stderr:2023-08-09T15:12:07.411+0000 7efc314cc640 -1 client.0 error registering admin socket command: (17) File exists 2023-08-09T15:12:07.413 INFO:tasks.ceph.mgr.x.smithi203.stderr:2023-08-09T15:12:07.411+0000 7efc314cc640 -1 client.0 error registering admin socket command: (17) File exists 2023-08-09T15:12:07.413 INFO:tasks.ceph.mgr.x.smithi203.stderr:2023-08-09T15:12:07.411+0000 7efc314cc640 -1 client.0 error registering admin socket command: (17) File exists 2023-08-09T15:12:07.413 INFO:tasks.ceph.mgr.x.smithi203.stderr:2023-08-09T15:12:07.412+0000 7efc2e4c6640 -1 client.0 error registering admin socket command: (17) File exists 2023-08-09T15:12:07.414 INFO:tasks.ceph.mgr.x.smithi203.stderr:2023-08-09T15:12:07.412+0000 7efc2e4c6640 -1 client.0 error registering admin socket command: (17) File exists 2023-08-09T15:12:07.414 INFO:tasks.ceph.mgr.x.smithi203.stderr:2023-08-09T15:12:07.412+0000 7efc2e4c6640 -1 client.0 error registering admin socket command: (17) File exists 2023-08-09T15:12:07.414 INFO:tasks.ceph.mgr.x.smithi203.stderr:2023-08-09T15:12:07.412+0000 7efc2e4c6640 -1 client.0 error registering admin socket command: (17) File exists 2023-08-09T15:12:07.414 INFO:tasks.ceph.mgr.x.smithi203.stderr:2023-08-09T15:12:07.412+0000 7efc2e4c6640 -1 client.0 error registering admin socket command: (17) File exists 2023-08-09T15:12:07.430 INFO:tasks.ceph.mgr.y.smithi043.stderr:2023-08-09T15:12:07.428+0000 7ff1ea851040 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2023-08-09T15:12:07.486 INFO:tasks.ceph.mgr.y.smithi043.stderr:2023-08-09T15:12:07.484+0000 7ff1ea851040 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2023-08-09T15:12:07.515 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:07.513+0000 7f78c9cfa640 1 Processor -- start 2023-08-09T15:12:07.515 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:07.513+0000 7f78c9cfa640 1 -- start start 2023-08-09T15:12:07.516 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:07.514+0000 7f78c9cfa640 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7f78c415fe60 0x7f78c40cd270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T15:12:07.516 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:07.515+0000 7f78c9cfa640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f78c40d14c0 0x7f78c40cee00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T15:12:07.516 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:07.515+0000 7f78c9cfa640 1 -- --> v1:172.21.15.43:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f78c406a7e0 con 0x7f78c40ce260 2023-08-09T15:12:07.517 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:07.515+0000 7f78c9cfa640 1 -- --> v2:172.21.15.203:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f78c406b360 con 0x7f78c415fe60 2023-08-09T15:12:07.517 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:07.515+0000 7f78c9cfa640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f78c406aa30 con 0x7f78c40d14c0 2023-08-09T15:12:07.519 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:07.515+0000 7f78c3fff640 1 --1- >> v1:172.21.15.43:6789/0 conn(0x7f78c40ce260 0x7f78c40d0440 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.43:6789/0 says I am v1:172.21.15.43:36674/0 (socket says 172.21.15.43:36674) 2023-08-09T15:12:07.519 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:07.515+0000 7f78c3fff640 1 -- 172.21.15.43:0/2471481359 learned_addr learned my addr 172.21.15.43:0/2471481359 (peer_addr_for_me v1:172.21.15.43:0/0) 2023-08-09T15:12:07.520 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:07.515+0000 7f78c37fe640 1 --2- 172.21.15.43:0/2471481359 >> v2:172.21.15.203:3300/0 conn(0x7f78c415fe60 0x7f78c40cd270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T15:12:07.520 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:07.515+0000 7f78c2ffd640 1 --2- 172.21.15.43:0/2471481359 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f78c40d14c0 0x7f78c40cee00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T15:12:07.520 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:07.515+0000 7f78c27fc640 1 -- 172.21.15.43:0/2471481359 <== mon.0 v1:172.21.15.43:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1016830845 0 0) 0x7f78c406a7e0 con 0x7f78c40ce260 2023-08-09T15:12:07.520 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:07.516+0000 7f78c27fc640 1 -- 172.21.15.43:0/2471481359 --> v1:172.21.15.43:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f78ac003520 con 0x7f78c40ce260 2023-08-09T15:12:07.520 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:07.516+0000 7f78c27fc640 1 -- 172.21.15.43:0/2471481359 <== mon.0 v1:172.21.15.43:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 21770195 0 0) 0x7f78ac003520 con 0x7f78c40ce260 2023-08-09T15:12:07.520 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:07.516+0000 7f78c27fc640 1 -- 172.21.15.43:0/2471481359 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f78c40d14c0 msgr2=0x7f78c40cee00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-09T15:12:07.521 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:07.516+0000 7f78c27fc640 1 --2- 172.21.15.43:0/2471481359 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f78c40d14c0 0x7f78c40cee00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T15:12:07.521 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:07.516+0000 7f78c27fc640 1 -- 172.21.15.43:0/2471481359 >> v2:172.21.15.203:3300/0 conn(0x7f78c415fe60 msgr2=0x7f78c40cd270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-09T15:12:07.521 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:07.516+0000 7f78c27fc640 1 --2- 172.21.15.43:0/2471481359 >> v2:172.21.15.203:3300/0 conn(0x7f78c415fe60 0x7f78c40cd270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T15:12:07.521 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:07.516+0000 7f78c27fc640 1 -- 172.21.15.43:0/2471481359 --> v1:172.21.15.43:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f78c406ad10 con 0x7f78c40ce260 2023-08-09T15:12:07.521 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:07.516+0000 7f78c27fc640 1 -- 172.21.15.43:0/2471481359 <== mon.0 v1:172.21.15.43:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1285150740 0 0) 0x7f78bc0019f0 con 0x7f78c40ce260 2023-08-09T15:12:07.521 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:07.516+0000 7f78c27fc640 1 -- 172.21.15.43:0/2471481359 >> v1:172.21.15.43:6789/0 conn(0x7f78c40ce260 legacy=0x7f78c40d0440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T15:12:07.521 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:07.516+0000 7f78c27fc640 1 --2- 172.21.15.43:0/2471481359 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f78c40d14c0 0x7f78ac003970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T15:12:07.522 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:07.516+0000 7f78c27fc640 1 --2- 172.21.15.43:0/2471481359 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f78ac003ef0 0x7f78ac0047c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T15:12:07.522 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:07.516+0000 7f78c27fc640 1 --2- 172.21.15.43:0/2471481359 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f78ac004d00 0x7f78ac005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T15:12:07.522 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:07.516+0000 7f78c27fc640 1 -- 172.21.15.43:0/2471481359 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f78c406b360 con 0x7f78ac003ef0 2023-08-09T15:12:07.522 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:07.516+0000 7f78c27fc640 1 -- 172.21.15.43:0/2471481359 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f78c406aa30 con 0x7f78ac004d00 2023-08-09T15:12:07.522 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:07.516+0000 7f78c27fc640 1 -- 172.21.15.43:0/2471481359 --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f78ac00b980 con 0x7f78c40d14c0 2023-08-09T15:12:07.522 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:07.516+0000 7f78c2ffd640 1 --2- 172.21.15.43:0/2471481359 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f78ac003ef0 0x7f78ac0047c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T15:12:07.522 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:07.517+0000 7f78c37fe640 1 --2- 172.21.15.43:0/2471481359 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f78c40d14c0 0x7f78ac003970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T15:12:07.522 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:07.517+0000 7f78c3fff640 1 --2- 172.21.15.43:0/2471481359 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f78ac004d00 0x7f78ac005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T15:12:07.523 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:07.517+0000 7f78c2ffd640 1 -- 172.21.15.43:0/2471481359 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f78c40d14c0 msgr2=0x7f78ac003970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-09T15:12:07.523 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:07.517+0000 7f78c2ffd640 1 --2- 172.21.15.43:0/2471481359 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f78c40d14c0 0x7f78ac003970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T15:12:07.523 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:07.517+0000 7f78c2ffd640 1 -- 172.21.15.43:0/2471481359 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f78ac004d00 msgr2=0x7f78ac005180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-09T15:12:07.523 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:07.517+0000 7f78c2ffd640 1 --2- 172.21.15.43:0/2471481359 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f78ac004d00 0x7f78ac005180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T15:12:07.523 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:07.517+0000 7f78c2ffd640 1 -- 172.21.15.43:0/2471481359 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f78ac003520 con 0x7f78ac003ef0 2023-08-09T15:12:07.523 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:07.517+0000 7f78c3fff640 1 --2- 172.21.15.43:0/2471481359 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f78ac004d00 0x7f78ac005180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-08-09T15:12:07.523 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:07.518+0000 7f78c2ffd640 1 --2- 172.21.15.43:0/2471481359 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f78ac003ef0 0x7f78ac0047c0 secure :-1 s=READY pgs=2509 cs=0 l=1 rev1=1 crypto rx=0x7f78b8009ac0 tx=0x7f78ac0057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=260476a045fed48b server_cookie=0 in_seq=0 out_seq=0 2023-08-09T15:12:07.523 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:07.518+0000 7f78c27fc640 1 -- 172.21.15.43:0/2471481359 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f78b802c6b0 con 0x7f78ac003ef0 2023-08-09T15:12:07.524 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:07.518+0000 7f78c27fc640 1 -- 172.21.15.43:0/2471481359 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f78b802c810 con 0x7f78ac003ef0 2023-08-09T15:12:07.524 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:07.518+0000 7f78c27fc640 1 -- 172.21.15.43:0/2471481359 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f78b802cba0 con 0x7f78ac003ef0 2023-08-09T15:12:07.526 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:07.519+0000 7f78c9cfa640 1 -- 172.21.15.43:0/2471481359 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f78ac003ef0 msgr2=0x7f78ac0047c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T15:12:07.527 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:07.519+0000 7f78c9cfa640 1 --2- 172.21.15.43:0/2471481359 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f78ac003ef0 0x7f78ac0047c0 secure :-1 s=READY pgs=2509 cs=0 l=1 rev1=1 crypto rx=0x7f78b8009ac0 tx=0x7f78ac0057f0 comp rx=0 tx=0).stop 2023-08-09T15:12:07.527 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:07.519+0000 7f78c3fff640 1 -- 172.21.15.43:0/2471481359 reap_dead start 2023-08-09T15:12:07.527 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:07.519+0000 7f78c9cfa640 1 -- 172.21.15.43:0/2471481359 shutdown_connections 2023-08-09T15:12:07.527 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:07.519+0000 7f78c9cfa640 1 -- 172.21.15.43:0/2471481359 >> 172.21.15.43:0/2471481359 conn(0x7f78c40d2660 msgr2=0x7f78c40d2a70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-09T15:12:07.528 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:07.519+0000 7f78c9cfa640 1 -- 172.21.15.43:0/2471481359 shutdown_connections 2023-08-09T15:12:07.528 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:07.519+0000 7f78c9cfa640 1 -- 172.21.15.43:0/2471481359 wait complete. 2023-08-09T15:12:07.528 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:07.521+0000 7f78c9cfa640 1 Processor -- start 2023-08-09T15:12:07.529 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:07.521+0000 7f78c9cfa640 1 -- start start 2023-08-09T15:12:07.529 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:07.522+0000 7f78c9cfa640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f78ac003ef0 0x7f78c415beb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T15:12:07.529 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:07.522+0000 7f78c9cfa640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f78c40c7e60 0x7f78c40b6310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T15:12:07.529 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:07.522+0000 7f78c9cfa640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f78c40c6b70 0x7f78c40b6850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T15:12:07.529 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:07.522+0000 7f78c9cfa640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f78c41f18e0 con 0x7f78ac003ef0 2023-08-09T15:12:07.529 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:07.522+0000 7f78c9cfa640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f78c40c7110 con 0x7f78c40c6b70 2023-08-09T15:12:07.530 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:07.522+0000 7f78c9cfa640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f78c40c73f0 con 0x7f78c40c7e60 2023-08-09T15:12:07.530 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:07.522+0000 7f78c37fe640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f78ac003ef0 0x7f78c415beb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T15:12:07.530 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:07.522+0000 7f78c37fe640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f78ac003ef0 0x7f78c415beb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:33210/0 (socket says 172.21.15.43:33210) 2023-08-09T15:12:07.530 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:07.522+0000 7f78c37fe640 1 -- 172.21.15.43:0/4268637157 learned_addr learned my addr 172.21.15.43:0/4268637157 (peer_addr_for_me v2:172.21.15.43:0/0) 2023-08-09T15:12:07.531 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:07.522+0000 7f78c37fe640 1 -- 172.21.15.43:0/4268637157 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f78c40c7e60 msgr2=0x7f78c40b6310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T15:12:07.531 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:07.522+0000 7f78c37fe640 1 --2- 172.21.15.43:0/4268637157 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f78c40c7e60 0x7f78c40b6310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T15:12:07.531 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:07.522+0000 7f78c37fe640 1 -- 172.21.15.43:0/4268637157 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f78c40c6b70 msgr2=0x7f78c40b6850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T15:12:07.531 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:07.522+0000 7f78c37fe640 1 --2- 172.21.15.43:0/4268637157 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f78c40c6b70 0x7f78c40b6850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T15:12:07.531 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:07.523+0000 7f78c37fe640 1 -- 172.21.15.43:0/4268637157 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f78c40c76d0 con 0x7f78ac003ef0 2023-08-09T15:12:07.531 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:07.523+0000 7f78c37fe640 1 --2- 172.21.15.43:0/4268637157 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f78ac003ef0 0x7f78c415beb0 secure :-1 s=READY pgs=2510 cs=0 l=1 rev1=1 crypto rx=0x7f78b40116d0 tx=0x7f78b4011b90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-09T15:12:07.531 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:07.523+0000 7f78abfff640 1 -- 172.21.15.43:0/4268637157 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f78b400f9c0 con 0x7f78ac003ef0 2023-08-09T15:12:07.532 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:07.523+0000 7f78abfff640 1 -- 172.21.15.43:0/4268637157 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f78b400fb20 con 0x7f78ac003ef0 2023-08-09T15:12:07.532 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:07.523+0000 7f78abfff640 1 -- 172.21.15.43:0/4268637157 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f78b400fe90 con 0x7f78ac003ef0 2023-08-09T15:12:07.532 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:07.523+0000 7f78c9cfa640 1 -- 172.21.15.43:0/4268637157 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f78c40c79b0 con 0x7f78ac003ef0 2023-08-09T15:12:07.532 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:07.523+0000 7f78c9cfa640 1 -- 172.21.15.43:0/4268637157 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f78c40b51f0 con 0x7f78ac003ef0 2023-08-09T15:12:07.532 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:07.525+0000 7f78c9cfa640 1 -- 172.21.15.43:0/4268637157 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f78c40c7530 con 0x7f78ac003ef0 2023-08-09T15:12:07.533 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:07.525+0000 7f78abfff640 1 -- 172.21.15.43:0/4268637157 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 126) v1 ==== 88052+0+0 (secure 0 0 0) 0x7f78b4015460 con 0x7f78ac003ef0 2023-08-09T15:12:07.533 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:07.526+0000 7f78abfff640 1 -- 172.21.15.43:0/4268637157 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(225..225 src has 1..225) v4 ==== 7200+0+0 (secure 0 0 0) 0x7f78b4088b00 con 0x7f78ac003ef0 2023-08-09T15:12:07.533 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:07.531+0000 7f78abfff640 1 -- 172.21.15.43:0/4268637157 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178066 (secure 0 0 0) 0x7f78c40c7530 con 0x7f78ac003ef0 2023-08-09T15:12:07.541 INFO:tasks.ceph.mgr.y.smithi043.stderr:2023-08-09T15:12:07.539+0000 7ff1ea851040 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2023-08-09T15:12:07.730 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:07.729+0000 7f78c9cfa640 1 -- 172.21.15.43:0/4268637157 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f78c40c7530 con 0x7f78ac003ef0 2023-08-09T15:12:07.734 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:07.732+0000 7f78abfff640 1 -- 172.21.15.43:0/4268637157 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v225) v1 ==== 74+0+19712 (secure 0 0 0) 0x7f78c40c7530 con 0x7f78ac003ef0 2023-08-09T15:12:07.735 INFO:teuthology.orchestra.run.smithi043.stdout: 2023-08-09T15:12:07.735 INFO:teuthology.orchestra.run.smithi043.stdout:{"epoch":225,"fsid":"daac87ee-1337-4250-a955-aea4e408a39b","created":"2023-08-09T13:33:41.701198+0000","modified":"2023-08-09T15:12:07.325033+0000","last_up_change":"2023-08-09T13:33:53.783453+0000","last_in_change":"2023-08-09T13:33:45.148904+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":27,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":"rbd","create_time":"2023-08-09T13:33:54.365137+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":".mgr","create_time":"2023-08-09T13:33:54.831803+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":26,"pool_name":"cephfs_metadata","create_time":"2023-08-09T15:11:26.544757+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"222","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":27,"pool_name":"cephfs_data","create_time":"2023-08-09T15:11:27.194686+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"223","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.6299999952316284,"score_stable":1.6299999952316284,"optimal_score":1,"raw_score_acting":1.6299999952316284,"raw_score_stable":1.6299999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"cbd49ecd-e896-4fbd-864e-5c37a829b165","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":221,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6800","nonce":502667335},{"type":"v1","addr":"172.21.15.43:6801","nonce":502667335}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6802","nonce":502667335},{"type":"v1","addr":"172.21.15.43:6803","nonce":502667335}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6806","nonce":502667335},{"type":"v1","addr":"172.21.15.43:6807","nonce":502667335}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6804","nonce":502667335},{"type":"v1","addr":"172.21.15.43:6805","nonce":502667335}]},"public_addr":"172.21.15.43:6801/502667335","cluster_addr":"172.21.15.43:6803/502667335","heartbeat_back_addr":"172.21.15.43:6807/502667335","heartbeat_front_addr":"172.21.15.43:6805/502667335","state":["exists","up"]},{"osd":1,"uuid":"75c7e0fe-1ec7-4fe8-a2f2-3661a65b8500","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":221,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6808","nonce":3289134610},{"type":"v1","addr":"172.21.15.43:6809","nonce":3289134610}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6810","nonce":3289134610},{"type":"v1","addr":"172.21.15.43:6812","nonce":3289134610}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6821","nonce":3289134610},{"type":"v1","addr":"172.21.15.43:6824","nonce":3289134610}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6815","nonce":3289134610},{"type":"v1","addr":"172.21.15.43:6818","nonce":3289134610}]},"public_addr":"172.21.15.43:6809/3289134610","cluster_addr":"172.21.15.43:6812/3289134610","heartbeat_back_addr":"172.21.15.43:6824/3289134610","heartbeat_front_addr":"172.21.15.43:6818/3289134610","state":["exists","up"]},{"osd":2,"uuid":"bb5c41e9-7f5f-4125-9b01-3bb8bfc2ea83","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":221,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6811","nonce":4079709736},{"type":"v1","addr":"172.21.15.43:6814","nonce":4079709736}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6817","nonce":4079709736},{"type":"v1","addr":"172.21.15.43:6820","nonce":4079709736}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6829","nonce":4079709736},{"type":"v1","addr":"172.21.15.43:6831","nonce":4079709736}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6823","nonce":4079709736},{"type":"v1","addr":"172.21.15.43:6826","nonce":4079709736}]},"public_addr":"172.21.15.43:6814/4079709736","cluster_addr":"172.21.15.43:6820/4079709736","heartbeat_back_addr":"172.21.15.43:6831/4079709736","heartbeat_front_addr":"172.21.15.43:6826/4079709736","state":["exists","up"]},{"osd":3,"uuid":"bfe4c8b4-694a-465c-87e6-2b2aab852613","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":221,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6813","nonce":1709538759},{"type":"v1","addr":"172.21.15.43:6816","nonce":1709538759}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6819","nonce":1709538759},{"type":"v1","addr":"172.21.15.43:6822","nonce":1709538759}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6828","nonce":1709538759},{"type":"v1","addr":"172.21.15.43:6830","nonce":1709538759}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6825","nonce":1709538759},{"type":"v1","addr":"172.21.15.43:6827","nonce":1709538759}]},"public_addr":"172.21.15.43:6816/1709538759","cluster_addr":"172.21.15.43:6822/1709538759","heartbeat_back_addr":"172.21.15.43:6830/1709538759","heartbeat_front_addr":"172.21.15.43:6827/1709538759","state":["exists","up"]},{"osd":4,"uuid":"fa2e8a77-aba1-4756-853e-0d174a5bdeba","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":221,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6800","nonce":4196866481},{"type":"v1","addr":"172.21.15.203:6801","nonce":4196866481}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6802","nonce":4196866481},{"type":"v1","addr":"172.21.15.203:6803","nonce":4196866481}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6806","nonce":4196866481},{"type":"v1","addr":"172.21.15.203:6807","nonce":4196866481}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6804","nonce":4196866481},{"type":"v1","addr":"172.21.15.203:6805","nonce":4196866481}]},"public_addr":"172.21.15.203:6801/4196866481","cluster_addr":"172.21.15.203:6803/4196866481","heartbeat_back_addr":"172.21.15.203:6807/4196866481","heartbeat_front_addr":"172.21.15.203:6805/4196866481","state":["exists","up"]},{"osd":5,"uuid":"81811e4b-4d24-4f51-a82c-f1d700a2a00d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":221,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6816","nonce":3059290081},{"type":"v1","addr":"172.21.15.203:6817","nonce":3059290081}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6818","nonce":3059290081},{"type":"v1","addr":"172.21.15.203:6819","nonce":3059290081}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6822","nonce":3059290081},{"type":"v1","addr":"172.21.15.203:6823","nonce":3059290081}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6820","nonce":3059290081},{"type":"v1","addr":"172.21.15.203:6821","nonce":3059290081}]},"public_addr":"172.21.15.203:6817/3059290081","cluster_addr":"172.21.15.203:6819/3059290081","heartbeat_back_addr":"172.21.15.203:6823/3059290081","heartbeat_front_addr":"172.21.15.203:6821/3059290081","state":["exists","up"]},{"osd":6,"uuid":"307bac4d-6a5b-443d-af75-56d695d7674f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":221,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6824","nonce":512155569},{"type":"v1","addr":"172.21.15.203:6825","nonce":512155569}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6826","nonce":512155569},{"type":"v1","addr":"172.21.15.203:6827","nonce":512155569}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6830","nonce":512155569},{"type":"v1","addr":"172.21.15.203:6831","nonce":512155569}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6828","nonce":512155569},{"type":"v1","addr":"172.21.15.203:6829","nonce":512155569}]},"public_addr":"172.21.15.203:6825/512155569","cluster_addr":"172.21.15.203:6827/512155569","heartbeat_back_addr":"172.21.15.203:6831/512155569","heartbeat_front_addr":"172.21.15.203:6829/512155569","state":["exists","up"]},{"osd":7,"uuid":"a95dc7ca-9f1b-4295-af70-68a22a2d7d5e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":221,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6808","nonce":3331450438},{"type":"v1","addr":"172.21.15.203:6809","nonce":3331450438}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6810","nonce":3331450438},{"type":"v1","addr":"172.21.15.203:6811","nonce":3331450438}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6814","nonce":3331450438},{"type":"v1","addr":"172.21.15.203:6815","nonce":3331450438}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6812","nonce":3331450438},{"type":"v1","addr":"172.21.15.203:6813","nonce":3331450438}]},"public_addr":"172.21.15.203:6809/3331450438","cluster_addr":"172.21.15.203:6811/3331450438","heartbeat_back_addr":"172.21.15.203:6815/3331450438","heartbeat_front_addr":"172.21.15.203:6813/3331450438","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-09T13:33:50.547652+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-09T13:33:50.589469+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-09T13:33:50.561724+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-09T13:33:50.599650+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-09T13:33:50.411855+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-09T13:33:50.219681+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-09T13:33:50.504664+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-09T13:33:50.441248+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.203:0/4004150203":"2023-08-10T15:12:07.324983+0000","172.21.15.203:6833/2375771024":"2023-08-10T15:12:07.324983+0000","172.21.15.203:6832/2375771024":"2023-08-10T15:12:07.324983+0000","172.21.15.203:0/1645319310":"2023-08-10T15:12:07.324983+0000","172.21.15.203:0/3324686399":"2023-08-10T15:12:07.324983+0000","172.21.15.203:0/286395424":"2023-08-10T15:12:07.324983+0000","172.21.15.203:0/1761748226":"2023-08-10T15:12:07.324983+0000","172.21.15.203:6832/4034093268":"2023-08-10T15:11:43.130145+0000","172.21.15.203:6833/4034093268":"2023-08-10T15:11:43.130145+0000","172.21.15.203:0/1654193324":"2023-08-10T15:11:43.130145+0000","172.21.15.203:0/3701038305":"2023-08-10T15:12:07.324983+0000","172.21.15.203:0/4140323243":"2023-08-10T15:11:43.130145+0000","172.21.15.203:0/450863258":"2023-08-10T15:11:43.130145+0000","172.21.15.203:0/134877587":"2023-08-10T15:11:43.130145+0000","172.21.15.203:0/3500453783":"2023-08-10T15:12:07.324983+0000","172.21.15.203:0/2330869600":"2023-08-10T15:11:43.130145+0000","172.21.15.203:0/1886393026":"2023-08-10T15:12:07.324983+0000","172.21.15.203:0/2203818837":"2023-08-10T15:11:43.130145+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":7,"snaps":[{"begin":104,"length":1},{"begin":221,"length":1}]}],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-08-09T15:12:07.737 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:07.736+0000 7f78c9cfa640 1 -- 172.21.15.43:0/4268637157 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f78ac003ef0 msgr2=0x7f78c415beb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T15:12:07.738 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:07.736+0000 7f78c9cfa640 1 --2- 172.21.15.43:0/4268637157 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f78ac003ef0 0x7f78c415beb0 secure :-1 s=READY pgs=2510 cs=0 l=1 rev1=1 crypto rx=0x7f78b40116d0 tx=0x7f78b4011b90 comp rx=0 tx=0).stop 2023-08-09T15:12:07.738 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:07.736+0000 7f78c9cfa640 1 -- 172.21.15.43:0/4268637157 shutdown_connections 2023-08-09T15:12:07.738 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:07.736+0000 7f78c9cfa640 1 --2- 172.21.15.43:0/4268637157 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f78c40c6b70 0x7f78c40b6850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T15:12:07.738 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:07.736+0000 7f78c9cfa640 1 --2- 172.21.15.43:0/4268637157 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f78c40c7e60 0x7f78c40b6310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T15:12:07.738 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:07.736+0000 7f78c9cfa640 1 --2- 172.21.15.43:0/4268637157 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f78ac003ef0 0x7f78c415beb0 unknown :-1 s=CLOSED pgs=2510 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T15:12:07.738 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:07.736+0000 7f78c9cfa640 1 -- 172.21.15.43:0/4268637157 >> 172.21.15.43:0/4268637157 conn(0x7f78c40d2660 msgr2=0x7f78c415ec60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-09T15:12:07.739 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:07.736+0000 7f78c9cfa640 1 -- 172.21.15.43:0/4268637157 shutdown_connections 2023-08-09T15:12:07.739 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:07.736+0000 7f78c9cfa640 1 -- 172.21.15.43:0/4268637157 wait complete. 2023-08-09T15:12:07.757 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': 'rbd', 'create_time': '2023-08-09T13:33:54.365137+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}, 'read_balance': {'score_acting': 2, 'score_stable': 2, 'optimal_score': 1, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': '.mgr', 'create_time': '2023-08-09T13:33:54.831803+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 26, 'pool_name': 'cephfs_metadata', 'create_time': '2023-08-09T15:11:26.544757+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '222', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.75, 'score_stable': 1.75, 'optimal_score': 1, 'raw_score_acting': 1.75, 'raw_score_stable': 1.75, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 27, 'pool_name': 'cephfs_data', 'create_time': '2023-08-09T15:11:27.194686+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '223', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.6299999952316284, 'score_stable': 1.6299999952316284, 'optimal_score': 1, 'raw_score_acting': 1.6299999952316284, 'raw_score_stable': 1.6299999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-08-09T15:12:07.758 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get rbd pg_num 2023-08-09T15:12:07.991 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:07.990+0000 7fd5e6621640 1 Processor -- start 2023-08-09T15:12:07.991 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:07.990+0000 7fd5e6621640 1 -- start start 2023-08-09T15:12:07.994 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:07.992+0000 7fd5e6621640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7fd5e015b1c0 0x7fd5e00c86f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T15:12:07.995 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:07.992+0000 7fd5e6621640 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7fd5e00c8c30 0x7fd5e00affc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T15:12:07.995 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:07.992+0000 7fd5e6621640 1 -- --> v1:172.21.15.43:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd5e0164ec0 con 0x7fd5e0161d10 2023-08-09T15:12:07.995 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:07.992+0000 7fd5e6621640 1 -- --> v2:172.21.15.43:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd5e00c1a90 con 0x7fd5e015b1c0 2023-08-09T15:12:07.995 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:07.992+0000 7fd5e6621640 1 -- --> v2:172.21.15.203:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd5e00ccc80 con 0x7fd5e00c8c30 2023-08-09T15:12:07.995 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:07.992+0000 7fd5e561f640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7fd5e015b1c0 0x7fd5e00c86f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T15:12:07.995 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:07.992+0000 7fd5e5e20640 1 --1- >> v1:172.21.15.43:6789/0 conn(0x7fd5e0161d10 0x7fd5e00ce000 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.43:6789/0 says I am v1:172.21.15.43:36684/0 (socket says 172.21.15.43:36684) 2023-08-09T15:12:07.995 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:07.992+0000 7fd5e5e20640 1 -- 172.21.15.43:0/3651932947 learned_addr learned my addr 172.21.15.43:0/3651932947 (peer_addr_for_me v1:172.21.15.43:0/0) 2023-08-09T15:12:07.996 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:07.992+0000 7fd5e4e1e640 1 --2- 172.21.15.43:0/3651932947 >> v2:172.21.15.203:3300/0 conn(0x7fd5e00c8c30 0x7fd5e00affc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T15:12:07.996 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:07.993+0000 7fd5e561f640 1 -- 172.21.15.43:0/3651932947 >> v2:172.21.15.203:3300/0 conn(0x7fd5e00c8c30 msgr2=0x7fd5e00affc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-09T15:12:07.996 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:07.993+0000 7fd5e561f640 1 --2- 172.21.15.43:0/3651932947 >> v2:172.21.15.203:3300/0 conn(0x7fd5e00c8c30 0x7fd5e00affc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T15:12:07.996 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:07.993+0000 7fd5e561f640 1 -- 172.21.15.43:0/3651932947 >> v1:172.21.15.43:6789/0 conn(0x7fd5e0161d10 legacy=0x7fd5e00ce000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T15:12:07.997 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:07.993+0000 7fd5e561f640 1 -- 172.21.15.43:0/3651932947 --> v2:172.21.15.43:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd5e00c90d0 con 0x7fd5e015b1c0 2023-08-09T15:12:07.997 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:07.993+0000 7fd5e4e1e640 1 --2- 172.21.15.43:0/3651932947 >> v2:172.21.15.203:3300/0 conn(0x7fd5e00c8c30 0x7fd5e00affc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-08-09T15:12:07.997 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:07.993+0000 7fd5d7fff640 1 -- 172.21.15.43:0/3651932947 <== mon.0 v1:172.21.15.43:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2139634464 0 0) 0x7fd5e0164ec0 con 0x7fd5e0161d10 2023-08-09T15:12:07.998 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:07.994+0000 7fd5e561f640 1 --2- 172.21.15.43:0/3651932947 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fd5e015b1c0 0x7fd5e00c86f0 secure :-1 s=READY pgs=2512 cs=0 l=1 rev1=1 crypto rx=0x7fd5e0071a20 tx=0x7fd5dc02b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d6b93a1cda4b1020 server_cookie=0 in_seq=0 out_seq=0 2023-08-09T15:12:07.998 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:07.994+0000 7fd5d7fff640 1 -- 172.21.15.43:0/3651932947 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd5dc031830 con 0x7fd5e015b1c0 2023-08-09T15:12:07.998 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:07.994+0000 7fd5d7fff640 1 -- 172.21.15.43:0/3651932947 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd5dc031990 con 0x7fd5e015b1c0 2023-08-09T15:12:07.998 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:07.994+0000 7fd5d7fff640 1 -- 172.21.15.43:0/3651932947 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd5dc031d20 con 0x7fd5e015b1c0 2023-08-09T15:12:07.999 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:07.995+0000 7fd5e6621640 1 -- 172.21.15.43:0/3651932947 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fd5e015b1c0 msgr2=0x7fd5e00c86f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T15:12:07.999 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:07.995+0000 7fd5e6621640 1 --2- 172.21.15.43:0/3651932947 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fd5e015b1c0 0x7fd5e00c86f0 secure :-1 s=READY pgs=2512 cs=0 l=1 rev1=1 crypto rx=0x7fd5e0071a20 tx=0x7fd5dc02b8d0 comp rx=0 tx=0).stop 2023-08-09T15:12:07.999 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:07.995+0000 7fd5e6621640 1 -- 172.21.15.43:0/3651932947 shutdown_connections 2023-08-09T15:12:07.999 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:07.995+0000 7fd5e6621640 1 --2- 172.21.15.43:0/3651932947 >> v2:172.21.15.203:3300/0 conn(0x7fd5e00c8c30 0x7fd5e00affc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T15:12:07.999 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:07.995+0000 7fd5e6621640 1 --2- 172.21.15.43:0/3651932947 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fd5e015b1c0 0x7fd5e00c86f0 unknown :-1 s=CLOSED pgs=2512 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T15:12:07.999 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:07.995+0000 7fd5e6621640 1 -- 172.21.15.43:0/3651932947 >> 172.21.15.43:0/3651932947 conn(0x7fd5e0069b30 msgr2=0x7fd5e0069f40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-09T15:12:07.999 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:07.995+0000 7fd5e6621640 1 -- 172.21.15.43:0/3651932947 shutdown_connections 2023-08-09T15:12:08.000 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:07.995+0000 7fd5e6621640 1 -- 172.21.15.43:0/3651932947 wait complete. 2023-08-09T15:12:08.000 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:07.996+0000 7fd5e6621640 1 Processor -- start 2023-08-09T15:12:08.000 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:07.997+0000 7fd5e6621640 1 -- start start 2023-08-09T15:12:08.001 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:07.998+0000 7fd5e6621640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fd5e00c8c30 0x7fd5e00b2350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T15:12:08.001 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:07.998+0000 7fd5e6621640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fd5e015b1c0 0x7fd5e00ae570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T15:12:08.001 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:07.998+0000 7fd5e6621640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fd5e0161d10 0x7fd5e014a160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T15:12:08.002 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:07.998+0000 7fd5e6621640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd5e015e3c0 con 0x7fd5e0161d10 2023-08-09T15:12:08.002 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:07.998+0000 7fd5e6621640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd5e015d320 con 0x7fd5e00c8c30 2023-08-09T15:12:08.002 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:07.998+0000 7fd5e6621640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd5e015d600 con 0x7fd5e015b1c0 2023-08-09T15:12:08.002 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:07.998+0000 7fd5e5e20640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fd5e0161d10 0x7fd5e014a160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T15:12:08.002 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:07.998+0000 7fd5e5e20640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fd5e0161d10 0x7fd5e014a160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:33236/0 (socket says 172.21.15.43:33236) 2023-08-09T15:12:08.002 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:07.998+0000 7fd5e5e20640 1 -- 172.21.15.43:0/1639292275 learned_addr learned my addr 172.21.15.43:0/1639292275 (peer_addr_for_me v2:172.21.15.43:0/0) 2023-08-09T15:12:08.002 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:07.998+0000 7fd5e561f640 1 --2- 172.21.15.43:0/1639292275 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fd5e00c8c30 0x7fd5e00b2350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T15:12:08.003 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:07.998+0000 7fd5e5e20640 1 -- 172.21.15.43:0/1639292275 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fd5e015b1c0 msgr2=0x7fd5e00ae570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T15:12:08.003 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:07.998+0000 7fd5e5e20640 1 --2- 172.21.15.43:0/1639292275 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fd5e015b1c0 0x7fd5e00ae570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T15:12:08.003 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:07.998+0000 7fd5e5e20640 1 -- 172.21.15.43:0/1639292275 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fd5e00c8c30 msgr2=0x7fd5e00b2350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T15:12:08.003 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:07.998+0000 7fd5e5e20640 1 --2- 172.21.15.43:0/1639292275 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fd5e00c8c30 0x7fd5e00b2350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T15:12:08.003 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:07.999+0000 7fd5e5e20640 1 -- 172.21.15.43:0/1639292275 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd5dc009400 con 0x7fd5e0161d10 2023-08-09T15:12:08.003 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:07.999+0000 7fd5e5e20640 1 --2- 172.21.15.43:0/1639292275 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fd5e0161d10 0x7fd5e014a160 secure :-1 s=READY pgs=2513 cs=0 l=1 rev1=1 crypto rx=0x7fd5d800c700 tx=0x7fd5d800cbc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-09T15:12:08.003 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:07.999+0000 7fd5d5ffb640 1 -- 172.21.15.43:0/1639292275 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd5d800ce10 con 0x7fd5e0161d10 2023-08-09T15:12:08.003 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:07.999+0000 7fd5d5ffb640 1 -- 172.21.15.43:0/1639292275 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd5d800a640 con 0x7fd5e0161d10 2023-08-09T15:12:08.004 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:07.999+0000 7fd5d5ffb640 1 -- 172.21.15.43:0/1639292275 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd5d800a9f0 con 0x7fd5e0161d10 2023-08-09T15:12:08.004 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:07.999+0000 7fd5e6621640 1 -- 172.21.15.43:0/1639292275 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd5e015d8e0 con 0x7fd5e0161d10 2023-08-09T15:12:08.004 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:07.999+0000 7fd5e6621640 1 -- 172.21.15.43:0/1639292275 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd5e015dea0 con 0x7fd5e0161d10 2023-08-09T15:12:08.004 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:08.001+0000 7fd5e6621640 1 -- 172.21.15.43:0/1639292275 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd5e015e0c0 con 0x7fd5e0161d10 2023-08-09T15:12:08.005 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:08.001+0000 7fd5d5ffb640 1 -- 172.21.15.43:0/1639292275 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 126) v1 ==== 88052+0+0 (secure 0 0 0) 0x7fd5d800ab50 con 0x7fd5e0161d10 2023-08-09T15:12:08.005 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:08.002+0000 7fd5d5ffb640 1 -- 172.21.15.43:0/1639292275 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(225..225 src has 1..225) v4 ==== 7200+0+0 (secure 0 0 0) 0x7fd5d8087980 con 0x7fd5e0161d10 2023-08-09T15:12:08.008 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:08.006+0000 7fd5d5ffb640 1 -- 172.21.15.43:0/1639292275 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178066 (secure 0 0 0) 0x7fd5d8052f80 con 0x7fd5e0161d10 2023-08-09T15:12:08.204 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:08.202+0000 7fd5e6621640 1 -- 172.21.15.43:0/1639292275 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"} v 0) v1 -- 0x7fd5e00b28f0 con 0x7fd5e0161d10 2023-08-09T15:12:08.205 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:08.203+0000 7fd5d5ffb640 1 -- 172.21.15.43:0/1639292275 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"}]=0 v225) v1 ==== 92+0+10 (secure 0 0 0) 0x7fd5d80565a0 con 0x7fd5e0161d10 2023-08-09T15:12:08.205 INFO:teuthology.orchestra.run.smithi043.stdout:pg_num: 8 2023-08-09T15:12:08.208 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:08.206+0000 7fd5e6621640 1 -- 172.21.15.43:0/1639292275 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fd5e0161d10 msgr2=0x7fd5e014a160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T15:12:08.208 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:08.206+0000 7fd5e6621640 1 --2- 172.21.15.43:0/1639292275 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fd5e0161d10 0x7fd5e014a160 secure :-1 s=READY pgs=2513 cs=0 l=1 rev1=1 crypto rx=0x7fd5d800c700 tx=0x7fd5d800cbc0 comp rx=0 tx=0).stop 2023-08-09T15:12:08.208 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:08.207+0000 7fd5e6621640 1 -- 172.21.15.43:0/1639292275 shutdown_connections 2023-08-09T15:12:08.208 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:08.207+0000 7fd5e6621640 1 --2- 172.21.15.43:0/1639292275 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fd5e0161d10 0x7fd5e014a160 unknown :-1 s=CLOSED pgs=2513 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T15:12:08.209 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:08.207+0000 7fd5e6621640 1 --2- 172.21.15.43:0/1639292275 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fd5e015b1c0 0x7fd5e00ae570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T15:12:08.209 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:08.207+0000 7fd5e6621640 1 --2- 172.21.15.43:0/1639292275 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fd5e00c8c30 0x7fd5e00b2350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T15:12:08.209 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:08.207+0000 7fd5e6621640 1 -- 172.21.15.43:0/1639292275 >> 172.21.15.43:0/1639292275 conn(0x7fd5e0069b30 msgr2=0x7fd5e006c050 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-09T15:12:08.209 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:08.207+0000 7fd5e6621640 1 -- 172.21.15.43:0/1639292275 shutdown_connections 2023-08-09T15:12:08.209 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:08.207+0000 7fd5e6621640 1 -- 172.21.15.43:0/1639292275 wait complete. 2023-08-09T15:12:08.227 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-08-09T15:12:08.473 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:08.472+0000 7f2d92596640 1 Processor -- start 2023-08-09T15:12:08.474 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:08.472+0000 7f2d92596640 1 -- start start 2023-08-09T15:12:08.475 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:08.473+0000 7f2d92596640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f2d8c160f20 0x7f2d8c0cec80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T15:12:08.475 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:08.473+0000 7f2d92596640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f2d8c0cf1c0 0x7f2d8c0d0a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T15:12:08.475 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:08.473+0000 7f2d92596640 1 -- --> v1:172.21.15.203:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2d8c15cc60 con 0x7f2d8c15c880 2023-08-09T15:12:08.475 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:08.473+0000 7f2d92596640 1 -- --> v2:172.21.15.43:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2d8c0d0530 con 0x7f2d8c0cf1c0 2023-08-09T15:12:08.475 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:08.473+0000 7f2d92596640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2d8c0cf620 con 0x7f2d8c160f20 2023-08-09T15:12:08.476 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:08.474+0000 7f2d90b33640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f2d8c0cf1c0 0x7f2d8c0d0a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T15:12:08.476 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:08.474+0000 7f2d90b33640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f2d8c0cf1c0 0x7f2d8c0d0a10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:33248/0 (socket says 172.21.15.43:33248) 2023-08-09T15:12:08.477 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:08.474+0000 7f2d90b33640 1 -- 172.21.15.43:0/61022786 learned_addr learned my addr 172.21.15.43:0/61022786 (peer_addr_for_me v2:172.21.15.43:0/0) 2023-08-09T15:12:08.477 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:08.474+0000 7f2d83fff640 1 --2- 172.21.15.43:0/61022786 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f2d8c160f20 0x7f2d8c0cec80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T15:12:08.477 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:08.474+0000 7f2d90b33640 1 -- 172.21.15.43:0/61022786 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f2d8c160f20 msgr2=0x7f2d8c0cec80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-09T15:12:08.477 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:08.474+0000 7f2d90b33640 1 --2- 172.21.15.43:0/61022786 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f2d8c160f20 0x7f2d8c0cec80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T15:12:08.477 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:08.474+0000 7f2d90b33640 1 -- 172.21.15.43:0/61022786 >> v1:172.21.15.203:6789/0 conn(0x7f2d8c15c880 legacy=0x7f2d8c0ccbc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-09T15:12:08.477 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:08.475+0000 7f2d90b33640 1 -- 172.21.15.43:0/61022786 --> v2:172.21.15.43:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2d8c0cd410 con 0x7f2d8c0cf1c0 2023-08-09T15:12:08.478 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:08.476+0000 7f2d90b33640 1 --2- 172.21.15.43:0/61022786 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f2d8c0cf1c0 0x7f2d8c0d0a10 secure :-1 s=READY pgs=2514 cs=0 l=1 rev1=1 crypto rx=0x7f2d84009e50 tx=0x7f2d8402bf60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fb103a77e0a0225a server_cookie=0 in_seq=0 out_seq=0 2023-08-09T15:12:08.479 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:08.476+0000 7f2d8b7fe640 1 -- 172.21.15.43:0/61022786 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2d8402cb00 con 0x7f2d8c0cf1c0 2023-08-09T15:12:08.479 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:08.476+0000 7f2d8b7fe640 1 -- 172.21.15.43:0/61022786 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2d8402cc60 con 0x7f2d8c0cf1c0 2023-08-09T15:12:08.479 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:08.476+0000 7f2d8b7fe640 1 -- 172.21.15.43:0/61022786 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2d840343f0 con 0x7f2d8c0cf1c0 2023-08-09T15:12:08.479 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:08.477+0000 7f2d92596640 1 -- 172.21.15.43:0/61022786 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f2d8c0cf1c0 msgr2=0x7f2d8c0d0a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T15:12:08.479 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:08.477+0000 7f2d92596640 1 --2- 172.21.15.43:0/61022786 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f2d8c0cf1c0 0x7f2d8c0d0a10 secure :-1 s=READY pgs=2514 cs=0 l=1 rev1=1 crypto rx=0x7f2d84009e50 tx=0x7f2d8402bf60 comp rx=0 tx=0).stop 2023-08-09T15:12:08.480 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:08.477+0000 7f2d92596640 1 -- 172.21.15.43:0/61022786 shutdown_connections 2023-08-09T15:12:08.480 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:08.477+0000 7f2d92596640 1 --2- 172.21.15.43:0/61022786 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f2d8c0cf1c0 0x7f2d8c0d0a10 unknown :-1 s=CLOSED pgs=2514 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T15:12:08.480 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:08.477+0000 7f2d92596640 1 --2- 172.21.15.43:0/61022786 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f2d8c160f20 0x7f2d8c0cec80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T15:12:08.480 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:08.477+0000 7f2d92596640 1 -- 172.21.15.43:0/61022786 >> 172.21.15.43:0/61022786 conn(0x7f2d8c08bfb0 msgr2=0x7f2d8c09b8d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-09T15:12:08.480 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:08.477+0000 7f2d92596640 1 -- 172.21.15.43:0/61022786 shutdown_connections 2023-08-09T15:12:08.480 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:08.477+0000 7f2d92596640 1 -- 172.21.15.43:0/61022786 wait complete. 2023-08-09T15:12:08.481 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:08.479+0000 7f2d92596640 1 Processor -- start 2023-08-09T15:12:08.481 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:08.479+0000 7f2d92596640 1 -- start start 2023-08-09T15:12:08.482 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:08.480+0000 7f2d92596640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f2d8c0cf1c0 0x7f2d8c15f570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T15:12:08.483 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:08.480+0000 7f2d92596640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f2d8c15c880 0x7f2d8c0b7630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T15:12:08.483 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:08.480+0000 7f2d92596640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f2d8c160f20 0x7f2d8c0b7b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T15:12:08.483 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:08.480+0000 7f2d92596640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2d8c0b4510 con 0x7f2d8c15c880 2023-08-09T15:12:08.483 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:08.480+0000 7f2d92596640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2d8c0b3940 con 0x7f2d8c0cf1c0 2023-08-09T15:12:08.483 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:08.480+0000 7f2d92596640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2d8c0b3be0 con 0x7f2d8c160f20 2023-08-09T15:12:08.484 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:08.481+0000 7f2d83fff640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f2d8c15c880 0x7f2d8c0b7630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T15:12:08.484 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:08.481+0000 7f2d83fff640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f2d8c15c880 0x7f2d8c0b7630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:33258/0 (socket says 172.21.15.43:33258) 2023-08-09T15:12:08.484 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:08.481+0000 7f2d83fff640 1 -- 172.21.15.43:0/163897389 learned_addr learned my addr 172.21.15.43:0/163897389 (peer_addr_for_me v2:172.21.15.43:0/0) 2023-08-09T15:12:08.484 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:08.481+0000 7f2d8bfff640 1 --2- 172.21.15.43:0/163897389 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f2d8c0cf1c0 0x7f2d8c15f570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T15:12:08.484 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:08.481+0000 7f2d90b33640 1 --2- 172.21.15.43:0/163897389 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f2d8c160f20 0x7f2d8c0b7b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T15:12:08.484 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:08.481+0000 7f2d83fff640 1 -- 172.21.15.43:0/163897389 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f2d8c160f20 msgr2=0x7f2d8c0b7b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T15:12:08.485 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:08.481+0000 7f2d83fff640 1 --2- 172.21.15.43:0/163897389 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f2d8c160f20 0x7f2d8c0b7b70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T15:12:08.485 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:08.481+0000 7f2d83fff640 1 -- 172.21.15.43:0/163897389 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f2d8c0cf1c0 msgr2=0x7f2d8c15f570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T15:12:08.485 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:08.481+0000 7f2d83fff640 1 --2- 172.21.15.43:0/163897389 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f2d8c0cf1c0 0x7f2d8c15f570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T15:12:08.485 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:08.481+0000 7f2d83fff640 1 -- 172.21.15.43:0/163897389 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2d84009b30 con 0x7f2d8c15c880 2023-08-09T15:12:08.485 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:08.481+0000 7f2d8bfff640 1 --2- 172.21.15.43:0/163897389 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f2d8c0cf1c0 0x7f2d8c15f570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-08-09T15:12:08.486 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:08.482+0000 7f2d83fff640 1 --2- 172.21.15.43:0/163897389 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f2d8c15c880 0x7f2d8c0b7630 secure :-1 s=READY pgs=2515 cs=0 l=1 rev1=1 crypto rx=0x7f2d7400d5f0 tx=0x7f2d7400dab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-09T15:12:08.486 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:08.482+0000 7f2d897fa640 1 -- 172.21.15.43:0/163897389 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2d740125f0 con 0x7f2d8c15c880 2023-08-09T15:12:08.486 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:08.482+0000 7f2d897fa640 1 -- 172.21.15.43:0/163897389 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2d74012750 con 0x7f2d8c15c880 2023-08-09T15:12:08.486 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:08.482+0000 7f2d897fa640 1 -- 172.21.15.43:0/163897389 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2d74012b00 con 0x7f2d8c15c880 2023-08-09T15:12:08.486 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:08.482+0000 7f2d92596640 1 -- 172.21.15.43:0/163897389 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2d8c0b3ec0 con 0x7f2d8c15c880 2023-08-09T15:12:08.487 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:08.482+0000 7f2d92596640 1 -- 172.21.15.43:0/163897389 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2d8c15fab0 con 0x7f2d8c15c880 2023-08-09T15:12:08.487 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:08.483+0000 7f2d92596640 1 -- 172.21.15.43:0/163897389 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2d8c04e390 con 0x7f2d8c15c880 2023-08-09T15:12:08.487 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:08.484+0000 7f2d897fa640 1 -- 172.21.15.43:0/163897389 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 127) v1 ==== 88179+0+0 (secure 0 0 0) 0x7f2d74012c60 con 0x7f2d8c15c880 2023-08-09T15:12:08.487 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:08.486+0000 7f2d897fa640 1 --2- 172.21.15.43:0/163897389 >> [v2:172.21.15.203:6832/1456699667,v1:172.21.15.203:6833/1456699667] conn(0x7f2d70069ea0 0x7f2d7006c360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T15:12:08.488 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:08.486+0000 7f2d897fa640 1 -- 172.21.15.43:0/163897389 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(226..226 src has 1..226) v4 ==== 7256+0+0 (secure 0 0 0) 0x7f2d74088bb0 con 0x7f2d8c15c880 2023-08-09T15:12:08.488 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:08.486+0000 7f2d8bfff640 1 --2- 172.21.15.43:0/163897389 >> [v2:172.21.15.203:6832/1456699667,v1:172.21.15.203:6833/1456699667] conn(0x7f2d70069ea0 0x7f2d7006c360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T15:12:08.489 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:08.488+0000 7f2d8bfff640 1 --2- 172.21.15.43:0/163897389 >> [v2:172.21.15.203:6832/1456699667,v1:172.21.15.203:6833/1456699667] conn(0x7f2d70069ea0 0x7f2d7006c360 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7f2d8c0cf190 tx=0x7f2d7c009040 comp rx=0 tx=0).ready entity=mgr.9232 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-09T15:12:08.491 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:08.490+0000 7f2d897fa640 1 -- 172.21.15.43:0/163897389 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178066 (secure 0 0 0) 0x7f2d74054060 con 0x7f2d8c15c880 2023-08-09T15:12:08.688 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:08.687+0000 7f2d92596640 1 -- 172.21.15.43:0/163897389 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f2d8c06bb60 con 0x7f2d8c15c880 2023-08-09T15:12:08.689 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:08.688+0000 7f2d897fa640 1 -- 172.21.15.43:0/163897389 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v226) v1 ==== 93+0+10 (secure 0 0 0) 0x7f2d74057680 con 0x7f2d8c15c880 2023-08-09T15:12:08.690 INFO:teuthology.orchestra.run.smithi043.stdout:pg_num: 1 2023-08-09T15:12:08.693 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:08.691+0000 7f2d92596640 1 -- 172.21.15.43:0/163897389 >> [v2:172.21.15.203:6832/1456699667,v1:172.21.15.203:6833/1456699667] conn(0x7f2d70069ea0 msgr2=0x7f2d7006c360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T15:12:08.693 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:08.691+0000 7f2d92596640 1 --2- 172.21.15.43:0/163897389 >> [v2:172.21.15.203:6832/1456699667,v1:172.21.15.203:6833/1456699667] conn(0x7f2d70069ea0 0x7f2d7006c360 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7f2d8c0cf190 tx=0x7f2d7c009040 comp rx=0 tx=0).stop 2023-08-09T15:12:08.693 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:08.691+0000 7f2d92596640 1 -- 172.21.15.43:0/163897389 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f2d8c15c880 msgr2=0x7f2d8c0b7630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T15:12:08.693 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:08.691+0000 7f2d92596640 1 --2- 172.21.15.43:0/163897389 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f2d8c15c880 0x7f2d8c0b7630 secure :-1 s=READY pgs=2515 cs=0 l=1 rev1=1 crypto rx=0x7f2d7400d5f0 tx=0x7f2d7400dab0 comp rx=0 tx=0).stop 2023-08-09T15:12:08.693 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:08.691+0000 7f2d92596640 1 -- 172.21.15.43:0/163897389 shutdown_connections 2023-08-09T15:12:08.693 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:08.691+0000 7f2d92596640 1 --2- 172.21.15.43:0/163897389 >> [v2:172.21.15.203:6832/1456699667,v1:172.21.15.203:6833/1456699667] conn(0x7f2d70069ea0 0x7f2d7006c360 unknown :-1 s=CLOSED pgs=20 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T15:12:08.694 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:08.691+0000 7f2d92596640 1 --2- 172.21.15.43:0/163897389 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f2d8c160f20 0x7f2d8c0b7b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T15:12:08.694 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:08.691+0000 7f2d92596640 1 --2- 172.21.15.43:0/163897389 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f2d8c15c880 0x7f2d8c0b7630 unknown :-1 s=CLOSED pgs=2515 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T15:12:08.694 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:08.691+0000 7f2d92596640 1 --2- 172.21.15.43:0/163897389 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f2d8c0cf1c0 0x7f2d8c15f570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T15:12:08.694 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:08.691+0000 7f2d92596640 1 -- 172.21.15.43:0/163897389 >> 172.21.15.43:0/163897389 conn(0x7f2d8c08bfb0 msgr2=0x7f2d8c1602e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-09T15:12:08.694 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:08.691+0000 7f2d92596640 1 -- 172.21.15.43:0/163897389 shutdown_connections 2023-08-09T15:12:08.694 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:08.691+0000 7f2d92596640 1 -- 172.21.15.43:0/163897389 wait complete. 2023-08-09T15:12:08.712 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-08-09T15:12:08.940 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:08.938+0000 7f2609ea6640 1 Processor -- start 2023-08-09T15:12:08.940 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:08.938+0000 7f2609ea6640 1 -- start start 2023-08-09T15:12:08.941 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:08.940+0000 7f2609ea6640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f26040c8880 0x7f26040c8ce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T15:12:08.942 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:08.940+0000 7f2609ea6640 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7f26040cbe00 0x7f26040cc250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T15:12:08.942 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:08.940+0000 7f2609ea6640 1 -- --> v1:172.21.15.203:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f26040c9220 con 0x7f260415c940 2023-08-09T15:12:08.942 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:08.940+0000 7f2609ea6640 1 -- --> v2:172.21.15.203:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f26040ca7b0 con 0x7f26040cbe00 2023-08-09T15:12:08.942 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:08.940+0000 7f2609ea6640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f26040cc790 con 0x7f26040c8880 2023-08-09T15:12:08.943 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:08.941+0000 7f2603fff640 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7f26040cbe00 0x7f26040cc250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T15:12:08.943 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:08.941+0000 7f2603fff640 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7f26040cbe00 0x7f26040cc250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.203:3300/0 says I am v2:172.21.15.43:41330/0 (socket says 172.21.15.43:41330) 2023-08-09T15:12:08.943 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:08.941+0000 7f2603fff640 1 -- 172.21.15.43:0/2566292662 learned_addr learned my addr 172.21.15.43:0/2566292662 (peer_addr_for_me v2:172.21.15.43:0/0) 2023-08-09T15:12:08.943 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:08.941+0000 7f2602ffd640 1 --2- 172.21.15.43:0/2566292662 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f26040c8880 0x7f26040c8ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T15:12:08.944 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:08.942+0000 7f2603fff640 1 -- 172.21.15.43:0/2566292662 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f26040c8880 msgr2=0x7f26040c8ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-09T15:12:08.944 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:08.942+0000 7f2603fff640 1 --2- 172.21.15.43:0/2566292662 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f26040c8880 0x7f26040c8ce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T15:12:08.944 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:08.942+0000 7f2603fff640 1 -- 172.21.15.43:0/2566292662 >> v1:172.21.15.203:6789/0 conn(0x7f260415c940 legacy=0x7f26040cb1c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T15:12:08.944 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:08.942+0000 7f2603fff640 1 -- 172.21.15.43:0/2566292662 --> v2:172.21.15.203:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f26040cad30 con 0x7f26040cbe00 2023-08-09T15:12:08.945 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:08.944+0000 7f2603fff640 1 --2- 172.21.15.43:0/2566292662 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f26040cbe00 0x7f26040cc250 secure :-1 s=READY pgs=918 cs=0 l=1 rev1=1 crypto rx=0x7f25fc009e50 tx=0x7f25fc02bf60 comp rx=0 tx=0).ready entity=mon.1 client_cookie=d30268f53a6b6066 server_cookie=0 in_seq=0 out_seq=0 2023-08-09T15:12:08.946 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:08.944+0000 7f26027fc640 1 -- 172.21.15.43:0/2566292662 <== mon.1 v2:172.21.15.203:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f25fc02cb00 con 0x7f26040cbe00 2023-08-09T15:12:08.946 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:08.944+0000 7f26027fc640 1 -- 172.21.15.43:0/2566292662 <== mon.1 v2:172.21.15.203:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f25fc02cc60 con 0x7f26040cbe00 2023-08-09T15:12:08.946 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:08.944+0000 7f26027fc640 1 -- 172.21.15.43:0/2566292662 <== mon.1 v2:172.21.15.203:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f25fc0343f0 con 0x7f26040cbe00 2023-08-09T15:12:08.947 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:08.945+0000 7f2609ea6640 1 -- 172.21.15.43:0/2566292662 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f26040cbe00 msgr2=0x7f26040cc250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T15:12:08.947 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:08.945+0000 7f2609ea6640 1 --2- 172.21.15.43:0/2566292662 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f26040cbe00 0x7f26040cc250 secure :-1 s=READY pgs=918 cs=0 l=1 rev1=1 crypto rx=0x7f25fc009e50 tx=0x7f25fc02bf60 comp rx=0 tx=0).stop 2023-08-09T15:12:08.947 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:08.945+0000 7f2609ea6640 1 -- 172.21.15.43:0/2566292662 shutdown_connections 2023-08-09T15:12:08.947 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:08.945+0000 7f2609ea6640 1 --2- 172.21.15.43:0/2566292662 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f26040cbe00 0x7f26040cc250 unknown :-1 s=CLOSED pgs=918 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T15:12:08.947 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:08.945+0000 7f2609ea6640 1 --2- 172.21.15.43:0/2566292662 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f26040c8880 0x7f26040c8ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T15:12:08.947 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:08.945+0000 7f2609ea6640 1 -- 172.21.15.43:0/2566292662 >> 172.21.15.43:0/2566292662 conn(0x7f2604069e60 msgr2=0x7f26040afd40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-09T15:12:08.948 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:08.945+0000 7f2609ea6640 1 -- 172.21.15.43:0/2566292662 shutdown_connections 2023-08-09T15:12:08.948 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:08.945+0000 7f2609ea6640 1 -- 172.21.15.43:0/2566292662 wait complete. 2023-08-09T15:12:08.948 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:08.947+0000 7f2609ea6640 1 Processor -- start 2023-08-09T15:12:08.948 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:08.947+0000 7f2609ea6640 1 -- start start 2023-08-09T15:12:08.949 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:08.948+0000 7f2609ea6640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f26040c8880 0x7f26040ac8c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T15:12:08.949 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:08.948+0000 7f2609ea6640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f26040cbe00 0x7f2604158f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T15:12:08.950 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:08.948+0000 7f2609ea6640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f260415c940 0x7f26040c4790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T15:12:08.950 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:08.948+0000 7f2609ea6640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f26040c4cd0 con 0x7f26040cbe00 2023-08-09T15:12:08.950 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:08.948+0000 7f2609ea6640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f26040c2ea0 con 0x7f260415c940 2023-08-09T15:12:08.950 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:08.948+0000 7f2609ea6640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f26040c4ec0 con 0x7f26040c8880 2023-08-09T15:12:08.951 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:08.949+0000 7f2602ffd640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f26040cbe00 0x7f2604158f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T15:12:08.951 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:08.949+0000 7f2602ffd640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f26040cbe00 0x7f2604158f20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:33264/0 (socket says 172.21.15.43:33264) 2023-08-09T15:12:08.952 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:08.949+0000 7f2602ffd640 1 -- 172.21.15.43:0/1948617651 learned_addr learned my addr 172.21.15.43:0/1948617651 (peer_addr_for_me v2:172.21.15.43:0/0) 2023-08-09T15:12:08.952 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:08.949+0000 7f26037fe640 1 --2- 172.21.15.43:0/1948617651 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f26040c8880 0x7f26040ac8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T15:12:08.952 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:08.949+0000 7f2603fff640 1 --2- 172.21.15.43:0/1948617651 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f260415c940 0x7f26040c4790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T15:12:08.952 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:08.949+0000 7f2602ffd640 1 -- 172.21.15.43:0/1948617651 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f26040c8880 msgr2=0x7f26040ac8c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T15:12:08.952 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:08.949+0000 7f2602ffd640 1 --2- 172.21.15.43:0/1948617651 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f26040c8880 0x7f26040ac8c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T15:12:08.952 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:08.949+0000 7f2602ffd640 1 -- 172.21.15.43:0/1948617651 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f260415c940 msgr2=0x7f26040c4790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T15:12:08.952 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:08.949+0000 7f2602ffd640 1 --2- 172.21.15.43:0/1948617651 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f260415c940 0x7f26040c4790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T15:12:08.953 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:08.949+0000 7f2602ffd640 1 -- 172.21.15.43:0/1948617651 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f25fc009b30 con 0x7f26040cbe00 2023-08-09T15:12:08.953 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:08.949+0000 7f2603fff640 1 --2- 172.21.15.43:0/1948617651 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f260415c940 0x7f26040c4790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-08-09T15:12:08.953 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:08.950+0000 7f2602ffd640 1 --2- 172.21.15.43:0/1948617651 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f26040cbe00 0x7f2604158f20 secure :-1 s=READY pgs=2516 cs=0 l=1 rev1=1 crypto rx=0x7f25f800e9b0 tx=0x7f25f800ee70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-09T15:12:08.953 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:08.950+0000 7f25ebfff640 1 -- 172.21.15.43:0/1948617651 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f25f8009670 con 0x7f26040cbe00 2023-08-09T15:12:08.953 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:08.950+0000 7f25ebfff640 1 -- 172.21.15.43:0/1948617651 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f25f800c9a0 con 0x7f26040cbe00 2023-08-09T15:12:08.953 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:08.950+0000 7f25ebfff640 1 -- 172.21.15.43:0/1948617651 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f25f800cd50 con 0x7f26040cbe00 2023-08-09T15:12:08.953 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:08.950+0000 7f2609ea6640 1 -- 172.21.15.43:0/1948617651 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f26040b0ea0 con 0x7f26040cbe00 2023-08-09T15:12:08.954 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:08.950+0000 7f2609ea6640 1 -- 172.21.15.43:0/1948617651 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f26040b1460 con 0x7f26040cbe00 2023-08-09T15:12:08.954 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:08.951+0000 7f2609ea6640 1 -- 172.21.15.43:0/1948617651 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f26040c3050 con 0x7f26040cbe00 2023-08-09T15:12:08.954 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:08.952+0000 7f25ebfff640 1 -- 172.21.15.43:0/1948617651 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 127) v1 ==== 88179+0+0 (secure 0 0 0) 0x7f25f8013470 con 0x7f26040cbe00 2023-08-09T15:12:08.955 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:08.953+0000 7f25ebfff640 1 --2- 172.21.15.43:0/1948617651 >> [v2:172.21.15.203:6832/1456699667,v1:172.21.15.203:6833/1456699667] conn(0x7f25f0069ea0 0x7f25f006c360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T15:12:08.955 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:08.953+0000 7f25ebfff640 1 -- 172.21.15.43:0/1948617651 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(226..226 src has 1..226) v4 ==== 7256+0+0 (secure 0 0 0) 0x7f25f80888b0 con 0x7f26040cbe00 2023-08-09T15:12:08.955 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:08.954+0000 7f26037fe640 1 --2- 172.21.15.43:0/1948617651 >> [v2:172.21.15.203:6832/1456699667,v1:172.21.15.203:6833/1456699667] conn(0x7f25f0069ea0 0x7f25f006c360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T15:12:08.956 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:08.955+0000 7f26037fe640 1 --2- 172.21.15.43:0/1948617651 >> [v2:172.21.15.203:6832/1456699667,v1:172.21.15.203:6833/1456699667] conn(0x7f25f0069ea0 0x7f25f006c360 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7f26040b3d30 tx=0x7f25f4009040 comp rx=0 tx=0).ready entity=mgr.9232 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-09T15:12:08.959 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:08.958+0000 7f25ebfff640 1 -- 172.21.15.43:0/1948617651 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178066 (secure 0 0 0) 0x7f26040c3050 con 0x7f26040cbe00 2023-08-09T15:12:09.163 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:09.162+0000 7f2609ea6640 1 -- 172.21.15.43:0/1948617651 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f26040c3010 con 0x7f26040cbe00 2023-08-09T15:12:09.164 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:09.163+0000 7f25ebfff640 1 -- 172.21.15.43:0/1948617651 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v226) v1 ==== 104+0+11 (secure 0 0 0) 0x7f25f8053e10 con 0x7f26040cbe00 2023-08-09T15:12:09.164 INFO:teuthology.orchestra.run.smithi043.stdout:pg_num: 32 2023-08-09T15:12:09.167 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:09.166+0000 7f2609ea6640 1 -- 172.21.15.43:0/1948617651 >> [v2:172.21.15.203:6832/1456699667,v1:172.21.15.203:6833/1456699667] conn(0x7f25f0069ea0 msgr2=0x7f25f006c360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T15:12:09.168 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:09.166+0000 7f2609ea6640 1 --2- 172.21.15.43:0/1948617651 >> [v2:172.21.15.203:6832/1456699667,v1:172.21.15.203:6833/1456699667] conn(0x7f25f0069ea0 0x7f25f006c360 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7f26040b3d30 tx=0x7f25f4009040 comp rx=0 tx=0).stop 2023-08-09T15:12:09.168 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:09.166+0000 7f2609ea6640 1 -- 172.21.15.43:0/1948617651 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f26040cbe00 msgr2=0x7f2604158f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T15:12:09.168 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:09.166+0000 7f2609ea6640 1 --2- 172.21.15.43:0/1948617651 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f26040cbe00 0x7f2604158f20 secure :-1 s=READY pgs=2516 cs=0 l=1 rev1=1 crypto rx=0x7f25f800e9b0 tx=0x7f25f800ee70 comp rx=0 tx=0).stop 2023-08-09T15:12:09.168 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:09.166+0000 7f2609ea6640 1 -- 172.21.15.43:0/1948617651 shutdown_connections 2023-08-09T15:12:09.168 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:09.166+0000 7f2609ea6640 1 --2- 172.21.15.43:0/1948617651 >> [v2:172.21.15.203:6832/1456699667,v1:172.21.15.203:6833/1456699667] conn(0x7f25f0069ea0 0x7f25f006c360 unknown :-1 s=CLOSED pgs=22 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T15:12:09.168 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:09.166+0000 7f2609ea6640 1 --2- 172.21.15.43:0/1948617651 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f260415c940 0x7f26040c4790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T15:12:09.168 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:09.166+0000 7f2609ea6640 1 --2- 172.21.15.43:0/1948617651 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f26040cbe00 0x7f2604158f20 unknown :-1 s=CLOSED pgs=2516 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T15:12:09.169 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:09.166+0000 7f2609ea6640 1 --2- 172.21.15.43:0/1948617651 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f26040c8880 0x7f26040ac8c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T15:12:09.169 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:09.166+0000 7f2609ea6640 1 -- 172.21.15.43:0/1948617651 >> 172.21.15.43:0/1948617651 conn(0x7f2604069e60 msgr2=0x7f26040cb0c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-09T15:12:09.169 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:09.166+0000 7f2609ea6640 1 -- 172.21.15.43:0/1948617651 shutdown_connections 2023-08-09T15:12:09.169 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:09.166+0000 7f2609ea6640 1 -- 172.21.15.43:0/1948617651 wait complete. 2023-08-09T15:12:09.186 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-08-09T15:12:09.424 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:09.423+0000 7fa6e0683640 1 Processor -- start 2023-08-09T15:12:09.424 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:09.423+0000 7fa6e0683640 1 -- start start 2023-08-09T15:12:09.426 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:09.424+0000 7fa6e0683640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fa6d815c980 0x7fa6d80afc80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T15:12:09.426 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:09.424+0000 7fa6e0683640 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7fa6d80ad570 0x7fa6d80b01c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T15:12:09.426 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:09.425+0000 7fa6e0683640 1 -- --> v1:172.21.15.43:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa6d81606a0 con 0x7fa6d80ab860 2023-08-09T15:12:09.427 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:09.425+0000 7fa6e0683640 1 -- --> v2:172.21.15.203:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa6d815cd70 con 0x7fa6d80ad570 2023-08-09T15:12:09.427 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:09.425+0000 7fa6e0683640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa6d8161220 con 0x7fa6d815c980 2023-08-09T15:12:09.427 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:09.425+0000 7fa6dec20640 1 --1- >> v1:172.21.15.43:6789/0 conn(0x7fa6d80ab860 0x7fa6d80abcf0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.43:6789/0 says I am v1:172.21.15.43:36688/0 (socket says 172.21.15.43:36688) 2023-08-09T15:12:09.427 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:09.425+0000 7fa6dec20640 1 -- 172.21.15.43:0/1412592027 learned_addr learned my addr 172.21.15.43:0/1412592027 (peer_addr_for_me v1:172.21.15.43:0/0) 2023-08-09T15:12:09.428 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:09.425+0000 7fa6ddc1e640 1 --2- 172.21.15.43:0/1412592027 >> v2:172.21.15.203:3300/0 conn(0x7fa6d80ad570 0x7fa6d80b01c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T15:12:09.428 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:09.425+0000 7fa6de41f640 1 --2- 172.21.15.43:0/1412592027 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fa6d815c980 0x7fa6d80afc80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T15:12:09.428 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:09.426+0000 7fa6dd41d640 1 -- 172.21.15.43:0/1412592027 <== mon.0 v1:172.21.15.43:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 739258118 0 0) 0x7fa6d81606a0 con 0x7fa6d80ab860 2023-08-09T15:12:09.428 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:09.426+0000 7fa6dd41d640 1 -- 172.21.15.43:0/1412592027 --> v1:172.21.15.43:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fa6c4003520 con 0x7fa6d80ab860 2023-08-09T15:12:09.428 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:09.426+0000 7fa6ddc1e640 1 -- 172.21.15.43:0/1412592027 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fa6d815c980 msgr2=0x7fa6d80afc80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-09T15:12:09.429 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:09.426+0000 7fa6ddc1e640 1 --2- 172.21.15.43:0/1412592027 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fa6d815c980 0x7fa6d80afc80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T15:12:09.429 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:09.426+0000 7fa6ddc1e640 1 -- 172.21.15.43:0/1412592027 >> v1:172.21.15.43:6789/0 conn(0x7fa6d80ab860 legacy=0x7fa6d80abcf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T15:12:09.429 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:09.426+0000 7fa6dd41d640 1 -- 172.21.15.43:0/1412592027 <== mon.0 v1:172.21.15.43:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3496407327 0 0) 0x7fa6c4003520 con 0x7fa6d80ab860 2023-08-09T15:12:09.429 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:09.426+0000 7fa6ddc1e640 1 -- 172.21.15.43:0/1412592027 --> v2:172.21.15.203:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa6d8160980 con 0x7fa6d80ad570 2023-08-09T15:12:09.430 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:09.428+0000 7fa6ddc1e640 1 --2- 172.21.15.43:0/1412592027 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fa6d80ad570 0x7fa6d80b01c0 secure :-1 s=READY pgs=919 cs=0 l=1 rev1=1 crypto rx=0x7fa6c80097a0 tx=0x7fa6c802b790 comp rx=0 tx=0).ready entity=mon.1 client_cookie=3a4ee1e3b865b704 server_cookie=0 in_seq=0 out_seq=0 2023-08-09T15:12:09.430 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:09.428+0000 7fa6dd41d640 1 -- 172.21.15.43:0/1412592027 <== mon.1 v2:172.21.15.203:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa6d00019f0 con 0x7fa6d80ad570 2023-08-09T15:12:09.430 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:09.428+0000 7fa6dd41d640 1 -- 172.21.15.43:0/1412592027 <== mon.1 v2:172.21.15.203:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa6c802e050 con 0x7fa6d80ad570 2023-08-09T15:12:09.430 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:09.428+0000 7fa6dd41d640 1 -- 172.21.15.43:0/1412592027 <== mon.1 v2:172.21.15.203:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa6c8030730 con 0x7fa6d80ad570 2023-08-09T15:12:09.431 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:09.429+0000 7fa6e0683640 1 -- 172.21.15.43:0/1412592027 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fa6d80ad570 msgr2=0x7fa6d80b01c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T15:12:09.431 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:09.429+0000 7fa6e0683640 1 --2- 172.21.15.43:0/1412592027 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fa6d80ad570 0x7fa6d80b01c0 secure :-1 s=READY pgs=919 cs=0 l=1 rev1=1 crypto rx=0x7fa6c80097a0 tx=0x7fa6c802b790 comp rx=0 tx=0).stop 2023-08-09T15:12:09.431 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:09.429+0000 7fa6e0683640 1 -- 172.21.15.43:0/1412592027 shutdown_connections 2023-08-09T15:12:09.431 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:09.429+0000 7fa6e0683640 1 --2- 172.21.15.43:0/1412592027 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fa6d80ad570 0x7fa6d80b01c0 unknown :-1 s=CLOSED pgs=919 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T15:12:09.431 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:09.429+0000 7fa6e0683640 1 --2- 172.21.15.43:0/1412592027 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fa6d815c980 0x7fa6d80afc80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T15:12:09.432 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:09.429+0000 7fa6e0683640 1 -- 172.21.15.43:0/1412592027 >> 172.21.15.43:0/1412592027 conn(0x7fa6d8069dd0 msgr2=0x7fa6d80cac80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-09T15:12:09.432 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:09.429+0000 7fa6e0683640 1 -- 172.21.15.43:0/1412592027 shutdown_connections 2023-08-09T15:12:09.432 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:09.429+0000 7fa6e0683640 1 -- 172.21.15.43:0/1412592027 wait complete. 2023-08-09T15:12:09.432 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:09.431+0000 7fa6e0683640 1 Processor -- start 2023-08-09T15:12:09.433 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:09.431+0000 7fa6e0683640 1 -- start start 2023-08-09T15:12:09.434 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:09.432+0000 7fa6e0683640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa6d80ab860 0x7fa6d81f15c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T15:12:09.434 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:09.432+0000 7fa6e0683640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fa6d80ad570 0x7fa6d80b0d70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T15:12:09.434 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:09.432+0000 7fa6e0683640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fa6d80b12b0 0x7fa6d80b2970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T15:12:09.434 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:09.432+0000 7fa6e0683640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa6d815dab0 con 0x7fa6d80ab860 2023-08-09T15:12:09.434 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:09.432+0000 7fa6e0683640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa6d815d4a0 con 0x7fa6d80b12b0 2023-08-09T15:12:09.434 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:09.432+0000 7fa6e0683640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa6d80b39f0 con 0x7fa6d80ad570 2023-08-09T15:12:09.435 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:09.433+0000 7fa6de41f640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa6d80ab860 0x7fa6d81f15c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T15:12:09.436 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:09.433+0000 7fa6de41f640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa6d80ab860 0x7fa6d81f15c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:33268/0 (socket says 172.21.15.43:33268) 2023-08-09T15:12:09.436 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:09.433+0000 7fa6de41f640 1 -- 172.21.15.43:0/815639067 learned_addr learned my addr 172.21.15.43:0/815639067 (peer_addr_for_me v2:172.21.15.43:0/0) 2023-08-09T15:12:09.436 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:09.433+0000 7fa6ddc1e640 1 --2- 172.21.15.43:0/815639067 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fa6d80ad570 0x7fa6d80b0d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T15:12:09.436 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:09.433+0000 7fa6dec20640 1 --2- 172.21.15.43:0/815639067 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fa6d80b12b0 0x7fa6d80b2970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T15:12:09.436 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:09.433+0000 7fa6de41f640 1 -- 172.21.15.43:0/815639067 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fa6d80ad570 msgr2=0x7fa6d80b0d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T15:12:09.436 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:09.433+0000 7fa6de41f640 1 --2- 172.21.15.43:0/815639067 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fa6d80ad570 0x7fa6d80b0d70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T15:12:09.436 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:09.433+0000 7fa6de41f640 1 -- 172.21.15.43:0/815639067 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fa6d80b12b0 msgr2=0x7fa6d80b2970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T15:12:09.437 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:09.433+0000 7fa6de41f640 1 --2- 172.21.15.43:0/815639067 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fa6d80b12b0 0x7fa6d80b2970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T15:12:09.437 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:09.433+0000 7fa6de41f640 1 -- 172.21.15.43:0/815639067 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa6c8009400 con 0x7fa6d80ab860 2023-08-09T15:12:09.437 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:09.433+0000 7fa6dec20640 1 --2- 172.21.15.43:0/815639067 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fa6d80b12b0 0x7fa6d80b2970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-08-09T15:12:09.437 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:09.434+0000 7fa6de41f640 1 --2- 172.21.15.43:0/815639067 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa6d80ab860 0x7fa6d81f15c0 secure :-1 s=READY pgs=2518 cs=0 l=1 rev1=1 crypto rx=0x7fa6d400e870 tx=0x7fa6d400ed30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-09T15:12:09.437 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:09.434+0000 7fa6ceffd640 1 -- 172.21.15.43:0/815639067 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa6d40094f0 con 0x7fa6d80ab860 2023-08-09T15:12:09.437 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:09.434+0000 7fa6ceffd640 1 -- 172.21.15.43:0/815639067 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa6d400c940 con 0x7fa6d80ab860 2023-08-09T15:12:09.437 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:09.434+0000 7fa6ceffd640 1 -- 172.21.15.43:0/815639067 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa6d400ccf0 con 0x7fa6d80ab860 2023-08-09T15:12:09.438 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:09.434+0000 7fa6e0683640 1 -- 172.21.15.43:0/815639067 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa6d815d740 con 0x7fa6d80ab860 2023-08-09T15:12:09.438 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:09.434+0000 7fa6e0683640 1 -- 172.21.15.43:0/815639067 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa6d81f1b00 con 0x7fa6d80ab860 2023-08-09T15:12:09.438 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:09.435+0000 7fa6e0683640 1 -- 172.21.15.43:0/815639067 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa6d804e390 con 0x7fa6d80ab860 2023-08-09T15:12:09.438 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:09.436+0000 7fa6ceffd640 1 -- 172.21.15.43:0/815639067 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 127) v1 ==== 88179+0+0 (secure 0 0 0) 0x7fa6d4013470 con 0x7fa6d80ab860 2023-08-09T15:12:09.439 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:09.437+0000 7fa6ceffd640 1 --2- 172.21.15.43:0/815639067 >> [v2:172.21.15.203:6832/1456699667,v1:172.21.15.203:6833/1456699667] conn(0x7fa6bc069ea0 0x7fa6bc06c360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T15:12:09.439 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:09.437+0000 7fa6ceffd640 1 -- 172.21.15.43:0/815639067 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(226..226 src has 1..226) v4 ==== 7256+0+0 (secure 0 0 0) 0x7fa6d4088810 con 0x7fa6d80ab860 2023-08-09T15:12:09.439 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:09.438+0000 7fa6ddc1e640 1 --2- 172.21.15.43:0/815639067 >> [v2:172.21.15.203:6832/1456699667,v1:172.21.15.203:6833/1456699667] conn(0x7fa6bc069ea0 0x7fa6bc06c360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T15:12:09.440 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:09.439+0000 7fa6ddc1e640 1 --2- 172.21.15.43:0/815639067 >> [v2:172.21.15.203:6832/1456699667,v1:172.21.15.203:6833/1456699667] conn(0x7fa6bc069ea0 0x7fa6bc06c360 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7fa6c8034000 tx=0x7fa6c802f000 comp rx=0 tx=0).ready entity=mgr.9232 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-09T15:12:09.443 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:09.442+0000 7fa6ceffd640 1 -- 172.21.15.43:0/815639067 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178066 (secure 0 0 0) 0x7fa6d4053cc0 con 0x7fa6d80ab860 2023-08-09T15:12:09.639 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:09.637+0000 7fa6e0683640 1 -- 172.21.15.43:0/815639067 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7fa6d80c9180 con 0x7fa6d80ab860 2023-08-09T15:12:09.639 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:09.638+0000 7fa6ceffd640 1 -- 172.21.15.43:0/815639067 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v226) v1 ==== 100+0+11 (secure 0 0 0) 0x7fa6d40572e0 con 0x7fa6d80ab860 2023-08-09T15:12:09.640 INFO:teuthology.orchestra.run.smithi043.stdout:pg_num: 64 2023-08-09T15:12:09.642 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:09.641+0000 7fa6e0683640 1 -- 172.21.15.43:0/815639067 >> [v2:172.21.15.203:6832/1456699667,v1:172.21.15.203:6833/1456699667] conn(0x7fa6bc069ea0 msgr2=0x7fa6bc06c360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T15:12:09.642 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:09.641+0000 7fa6e0683640 1 --2- 172.21.15.43:0/815639067 >> [v2:172.21.15.203:6832/1456699667,v1:172.21.15.203:6833/1456699667] conn(0x7fa6bc069ea0 0x7fa6bc06c360 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7fa6c8034000 tx=0x7fa6c802f000 comp rx=0 tx=0).stop 2023-08-09T15:12:09.642 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:09.641+0000 7fa6e0683640 1 -- 172.21.15.43:0/815639067 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa6d80ab860 msgr2=0x7fa6d81f15c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T15:12:09.643 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:09.641+0000 7fa6e0683640 1 --2- 172.21.15.43:0/815639067 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa6d80ab860 0x7fa6d81f15c0 secure :-1 s=READY pgs=2518 cs=0 l=1 rev1=1 crypto rx=0x7fa6d400e870 tx=0x7fa6d400ed30 comp rx=0 tx=0).stop 2023-08-09T15:12:09.643 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:09.641+0000 7fa6e0683640 1 -- 172.21.15.43:0/815639067 shutdown_connections 2023-08-09T15:12:09.643 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:09.641+0000 7fa6e0683640 1 --2- 172.21.15.43:0/815639067 >> [v2:172.21.15.203:6832/1456699667,v1:172.21.15.203:6833/1456699667] conn(0x7fa6bc069ea0 0x7fa6bc06c360 unknown :-1 s=CLOSED pgs=23 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T15:12:09.643 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:09.641+0000 7fa6e0683640 1 --2- 172.21.15.43:0/815639067 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fa6d80b12b0 0x7fa6d80b2970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T15:12:09.644 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:09.641+0000 7fa6e0683640 1 --2- 172.21.15.43:0/815639067 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fa6d80ad570 0x7fa6d80b0d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T15:12:09.644 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:09.641+0000 7fa6e0683640 1 --2- 172.21.15.43:0/815639067 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa6d80ab860 0x7fa6d81f15c0 unknown :-1 s=CLOSED pgs=2518 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T15:12:09.644 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:09.641+0000 7fa6e0683640 1 -- 172.21.15.43:0/815639067 >> 172.21.15.43:0/815639067 conn(0x7fa6d8069dd0 msgr2=0x7fa6d80ca220 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-09T15:12:09.644 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:09.641+0000 7fa6e0683640 1 -- 172.21.15.43:0/815639067 shutdown_connections 2023-08-09T15:12:09.645 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:09.641+0000 7fa6e0683640 1 -- 172.21.15.43:0/815639067 wait complete. 2023-08-09T15:12:09.661 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-08-09T15:12:09.889 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:09.888+0000 7f3d23984640 1 Processor -- start 2023-08-09T15:12:09.890 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:09.888+0000 7f3d23984640 1 -- start start 2023-08-09T15:12:09.891 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:09.890+0000 7f3d23984640 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7f3d1c15c900 0x7f3d1c0afc70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T15:12:09.891 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:09.890+0000 7f3d23984640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f3d1c160c00 0x7f3d1c0b01b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T15:12:09.891 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:09.890+0000 7f3d23984640 1 -- --> v1:172.21.15.203:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3d1c161220 con 0x7f3d1c15cce0 2023-08-09T15:12:09.891 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:09.890+0000 7f3d23984640 1 -- --> v2:172.21.15.203:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3d1c0cd100 con 0x7f3d1c15c900 2023-08-09T15:12:09.892 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:09.890+0000 7f3d23984640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3d1c15d1c0 con 0x7f3d1c160c00 2023-08-09T15:12:09.892 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:09.891+0000 7f3d20f1f640 1 --1- >> v1:172.21.15.203:6789/0 conn(0x7f3d1c15cce0 0x7f3d1c1604f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.203:6789/0 says I am v1:172.21.15.43:57984/0 (socket says 172.21.15.43:57984) 2023-08-09T15:12:09.893 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:09.891+0000 7f3d20f1f640 1 -- 172.21.15.43:0/1002700503 learned_addr learned my addr 172.21.15.43:0/1002700503 (peer_addr_for_me v1:172.21.15.43:0/0) 2023-08-09T15:12:09.893 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:09.891+0000 7f3d21720640 1 --2- 172.21.15.43:0/1002700503 >> v2:172.21.15.203:3300/0 conn(0x7f3d1c15c900 0x7f3d1c0afc70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T15:12:09.893 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:09.891+0000 7f3d21f21640 1 --2- 172.21.15.43:0/1002700503 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f3d1c160c00 0x7f3d1c0b01b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T15:12:09.893 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:09.891+0000 7f3d13fff640 1 -- 172.21.15.43:0/1002700503 <== mon.1 v1:172.21.15.203:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3104677214 0 0) 0x7f3d1c161220 con 0x7f3d1c15cce0 2023-08-09T15:12:09.893 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:09.891+0000 7f3d13fff640 1 -- 172.21.15.43:0/1002700503 --> v1:172.21.15.203:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f3d08003520 con 0x7f3d1c15cce0 2023-08-09T15:12:09.894 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:09.892+0000 7f3d21720640 1 -- 172.21.15.43:0/1002700503 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f3d1c160c00 msgr2=0x7f3d1c0b01b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-09T15:12:09.894 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:09.892+0000 7f3d21720640 1 --2- 172.21.15.43:0/1002700503 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f3d1c160c00 0x7f3d1c0b01b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T15:12:09.894 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:09.892+0000 7f3d21720640 1 -- 172.21.15.43:0/1002700503 >> v1:172.21.15.203:6789/0 conn(0x7f3d1c15cce0 legacy=0x7f3d1c1604f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T15:12:09.894 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:09.892+0000 7f3d21720640 1 -- 172.21.15.43:0/1002700503 --> v2:172.21.15.203:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3d1c15d4a0 con 0x7f3d1c15c900 2023-08-09T15:12:09.894 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:09.892+0000 7f3d21f21640 1 --2- 172.21.15.43:0/1002700503 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f3d1c160c00 0x7f3d1c0b01b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-08-09T15:12:09.895 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:09.893+0000 7f3d21720640 1 --2- 172.21.15.43:0/1002700503 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f3d1c15c900 0x7f3d1c0afc70 secure :-1 s=READY pgs=922 cs=0 l=1 rev1=1 crypto rx=0x7f3d180097a0 tx=0x7f3d1802b790 comp rx=0 tx=0).ready entity=mon.1 client_cookie=5232794796a80014 server_cookie=0 in_seq=0 out_seq=0 2023-08-09T15:12:09.896 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:09.893+0000 7f3d13fff640 1 -- 172.21.15.43:0/1002700503 <== mon.1 v2:172.21.15.203:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3d180316f0 con 0x7f3d1c15c900 2023-08-09T15:12:09.896 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:09.893+0000 7f3d13fff640 1 -- 172.21.15.43:0/1002700503 <== mon.1 v2:172.21.15.203:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3d18031850 con 0x7f3d1c15c900 2023-08-09T15:12:09.896 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:09.893+0000 7f3d13fff640 1 -- 172.21.15.43:0/1002700503 <== mon.1 v2:172.21.15.203:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3d18031be0 con 0x7f3d1c15c900 2023-08-09T15:12:09.896 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:09.894+0000 7f3d23984640 1 -- 172.21.15.43:0/1002700503 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f3d1c15c900 msgr2=0x7f3d1c0afc70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T15:12:09.896 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:09.894+0000 7f3d23984640 1 --2- 172.21.15.43:0/1002700503 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f3d1c15c900 0x7f3d1c0afc70 secure :-1 s=READY pgs=922 cs=0 l=1 rev1=1 crypto rx=0x7f3d180097a0 tx=0x7f3d1802b790 comp rx=0 tx=0).stop 2023-08-09T15:12:09.897 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:09.894+0000 7f3d23984640 1 -- 172.21.15.43:0/1002700503 shutdown_connections 2023-08-09T15:12:09.897 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:09.894+0000 7f3d23984640 1 --2- 172.21.15.43:0/1002700503 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f3d1c160c00 0x7f3d1c0b01b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T15:12:09.897 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:09.894+0000 7f3d23984640 1 --2- 172.21.15.43:0/1002700503 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f3d1c15c900 0x7f3d1c0afc70 unknown :-1 s=CLOSED pgs=922 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T15:12:09.897 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:09.894+0000 7f3d23984640 1 -- 172.21.15.43:0/1002700503 >> 172.21.15.43:0/1002700503 conn(0x7f3d1c069d70 msgr2=0x7f3d1c0cac70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-09T15:12:09.897 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:09.894+0000 7f3d23984640 1 -- 172.21.15.43:0/1002700503 shutdown_connections 2023-08-09T15:12:09.897 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:09.894+0000 7f3d23984640 1 -- 172.21.15.43:0/1002700503 wait complete. 2023-08-09T15:12:09.898 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:09.896+0000 7f3d23984640 1 Processor -- start 2023-08-09T15:12:09.898 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:09.896+0000 7f3d23984640 1 -- start start 2023-08-09T15:12:09.898 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:09.897+0000 7f3d23984640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f3d1c15c900 0x7f3d1c15fe90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T15:12:09.899 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:09.897+0000 7f3d23984640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f3d1c15cce0 0x7f3d1c15ac00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T15:12:09.899 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:09.897+0000 7f3d23984640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f3d1c160c00 0x7f3d1c0b21c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T15:12:09.899 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:09.897+0000 7f3d23984640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3d1c148b60 con 0x7f3d1c160c00 2023-08-09T15:12:09.899 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:09.897+0000 7f3d23984640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3d1c15b5a0 con 0x7f3d1c15cce0 2023-08-09T15:12:09.899 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:09.897+0000 7f3d23984640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3d1c148230 con 0x7f3d1c15c900 2023-08-09T15:12:09.899 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:09.897+0000 7f3d21f21640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f3d1c160c00 0x7f3d1c0b21c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T15:12:09.900 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:09.897+0000 7f3d21f21640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f3d1c160c00 0x7f3d1c0b21c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:33278/0 (socket says 172.21.15.43:33278) 2023-08-09T15:12:09.900 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:09.897+0000 7f3d21f21640 1 -- 172.21.15.43:0/2589289407 learned_addr learned my addr 172.21.15.43:0/2589289407 (peer_addr_for_me v2:172.21.15.43:0/0) 2023-08-09T15:12:09.901 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:09.898+0000 7f3d21720640 1 --2- 172.21.15.43:0/2589289407 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f3d1c15c900 0x7f3d1c15fe90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T15:12:09.901 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:09.898+0000 7f3d20f1f640 1 --2- 172.21.15.43:0/2589289407 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f3d1c15cce0 0x7f3d1c15ac00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T15:12:09.901 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:09.898+0000 7f3d21f21640 1 -- 172.21.15.43:0/2589289407 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f3d1c15c900 msgr2=0x7f3d1c15fe90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T15:12:09.901 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:09.898+0000 7f3d21f21640 1 --2- 172.21.15.43:0/2589289407 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f3d1c15c900 0x7f3d1c15fe90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T15:12:09.902 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:09.898+0000 7f3d21f21640 1 -- 172.21.15.43:0/2589289407 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f3d1c15cce0 msgr2=0x7f3d1c15ac00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T15:12:09.902 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:09.898+0000 7f3d21f21640 1 --2- 172.21.15.43:0/2589289407 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f3d1c15cce0 0x7f3d1c15ac00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T15:12:09.902 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:09.898+0000 7f3d21f21640 1 -- 172.21.15.43:0/2589289407 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3d14009a70 con 0x7f3d1c160c00 2023-08-09T15:12:09.902 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:09.898+0000 7f3d20f1f640 1 --2- 172.21.15.43:0/2589289407 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f3d1c15cce0 0x7f3d1c15ac00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-08-09T15:12:09.902 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:09.898+0000 7f3d21f21640 1 --2- 172.21.15.43:0/2589289407 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f3d1c160c00 0x7f3d1c0b21c0 secure :-1 s=READY pgs=2519 cs=0 l=1 rev1=1 crypto rx=0x7f3d1400efc0 tx=0x7f3d1400c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-09T15:12:09.902 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:09.898+0000 7f3d11ffb640 1 -- 172.21.15.43:0/2589289407 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3d14009cd0 con 0x7f3d1c160c00 2023-08-09T15:12:09.902 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:09.898+0000 7f3d11ffb640 1 -- 172.21.15.43:0/2589289407 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3d1400f040 con 0x7f3d1c160c00 2023-08-09T15:12:09.903 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:09.898+0000 7f3d11ffb640 1 -- 172.21.15.43:0/2589289407 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3d14003990 con 0x7f3d1c160c00 2023-08-09T15:12:09.903 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:09.898+0000 7f3d23984640 1 -- 172.21.15.43:0/2589289407 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3d18009400 con 0x7f3d1c160c00 2023-08-09T15:12:09.903 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:09.899+0000 7f3d23984640 1 -- 172.21.15.43:0/2589289407 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3d1c1487f0 con 0x7f3d1c160c00 2023-08-09T15:12:09.903 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:09.900+0000 7f3d23984640 1 -- 172.21.15.43:0/2589289407 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3d1c148d10 con 0x7f3d1c160c00 2023-08-09T15:12:09.904 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:09.901+0000 7f3d11ffb640 1 -- 172.21.15.43:0/2589289407 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 127) v1 ==== 88179+0+0 (secure 0 0 0) 0x7f3d14014480 con 0x7f3d1c160c00 2023-08-09T15:12:09.904 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:09.901+0000 7f3d11ffb640 1 --2- 172.21.15.43:0/2589289407 >> [v2:172.21.15.203:6832/1456699667,v1:172.21.15.203:6833/1456699667] conn(0x7f3d00069ea0 0x7f3d0006c360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T15:12:09.904 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:09.902+0000 7f3d11ffb640 1 -- 172.21.15.43:0/2589289407 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(226..226 src has 1..226) v4 ==== 7256+0+0 (secure 0 0 0) 0x7f3d14089330 con 0x7f3d1c160c00 2023-08-09T15:12:09.904 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:09.902+0000 7f3d21720640 1 --2- 172.21.15.43:0/2589289407 >> [v2:172.21.15.203:6832/1456699667,v1:172.21.15.203:6833/1456699667] conn(0x7f3d00069ea0 0x7f3d0006c360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T15:12:09.905 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:09.904+0000 7f3d21720640 1 --2- 172.21.15.43:0/2589289407 >> [v2:172.21.15.203:6832/1456699667,v1:172.21.15.203:6833/1456699667] conn(0x7f3d00069ea0 0x7f3d0006c360 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7f3d18036000 tx=0x7f3d18008030 comp rx=0 tx=0).ready entity=mgr.9232 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-09T15:12:09.908 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:09.907+0000 7f3d11ffb640 1 -- 172.21.15.43:0/2589289407 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178066 (secure 0 0 0) 0x7f3d1c148d10 con 0x7f3d1c160c00 2023-08-09T15:12:10.128 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:10.127+0000 7f3d11ffb640 1 -- 172.21.15.43:0/2589289407 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mgrmap(e 128) v1 ==== 88387+0+0 (secure 0 0 0) 0x7f3d14054bb0 con 0x7f3d1c160c00 2023-08-09T15:12:10.144 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:10.142+0000 7f3d23984640 1 -- 172.21.15.43:0/2589289407 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f3d1c148d10 con 0x7f3d1c160c00 2023-08-09T15:12:10.145 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:10.143+0000 7f3d11ffb640 1 -- 172.21.15.43:0/2589289407 <== mon.0 v2:172.21.15.43:3300/0 8 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 104 v104) v1 ==== 95+0+4894 (secure 0 0 0) 0x7f3d1c148d10 con 0x7f3d1c160c00 2023-08-09T15:12:10.145 INFO:teuthology.orchestra.run.smithi043.stdout: 2023-08-09T15:12:10.146 INFO:teuthology.orchestra.run.smithi043.stdout:{"epoch":104,"default_fscid":12,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":8012,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.203:6837/2464832767","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6836","nonce":2464832767},{"type":"v1","addr":"172.21.15.203:6837","nonce":2464832767}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":100},{"gid":9010,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.43:6835/3989386049","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6834","nonce":3989386049},{"type":"v1","addr":"172.21.15.43:6835","nonce":3989386049}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":100},{"gid":9016,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.203:6835/2213455074","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6834","nonce":2213455074},{"type":"v1","addr":"172.21.15.203:6835","nonce":2213455074}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":100}],"filesystems":[{"mdsmap":{"epoch":104,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-08-09T15:11:28.168916+0000","modified":"2023-08-09T15:11:31.739094+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":11325},"failed":[],"damaged":[],"stopped":[],"info":{"gid_11325":{"gid":11325,"name":"a","rank":0,"incarnation":101,"state":"up:active","state_seq":3,"addr":"172.21.15.43:6833/1425989175","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6832","nonce":1425989175},{"type":"v1","addr":"172.21.15.43:6833","nonce":1425989175}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[27],"metadata_pool":26,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":12}]} 2023-08-09T15:12:10.146 INFO:teuthology.orchestra.run.smithi043.stderr:dumped fsmap epoch 104 2023-08-09T15:12:10.148 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:10.146+0000 7f3d23984640 1 -- 172.21.15.43:0/2589289407 >> [v2:172.21.15.203:6832/1456699667,v1:172.21.15.203:6833/1456699667] conn(0x7f3d00069ea0 msgr2=0x7f3d0006c360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T15:12:10.148 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:10.146+0000 7f3d23984640 1 --2- 172.21.15.43:0/2589289407 >> [v2:172.21.15.203:6832/1456699667,v1:172.21.15.203:6833/1456699667] conn(0x7f3d00069ea0 0x7f3d0006c360 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7f3d18036000 tx=0x7f3d18008030 comp rx=0 tx=0).stop 2023-08-09T15:12:10.148 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:10.146+0000 7f3d23984640 1 -- 172.21.15.43:0/2589289407 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f3d1c160c00 msgr2=0x7f3d1c0b21c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T15:12:10.148 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:10.146+0000 7f3d23984640 1 --2- 172.21.15.43:0/2589289407 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f3d1c160c00 0x7f3d1c0b21c0 secure :-1 s=READY pgs=2519 cs=0 l=1 rev1=1 crypto rx=0x7f3d1400efc0 tx=0x7f3d1400c490 comp rx=0 tx=0).stop 2023-08-09T15:12:10.148 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:10.146+0000 7f3d23984640 1 -- 172.21.15.43:0/2589289407 shutdown_connections 2023-08-09T15:12:10.148 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:10.146+0000 7f3d23984640 1 --2- 172.21.15.43:0/2589289407 >> [v2:172.21.15.203:6832/1456699667,v1:172.21.15.203:6833/1456699667] conn(0x7f3d00069ea0 0x7f3d0006c360 unknown :-1 s=CLOSED pgs=25 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T15:12:10.149 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:10.146+0000 7f3d23984640 1 --2- 172.21.15.43:0/2589289407 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f3d1c160c00 0x7f3d1c0b21c0 unknown :-1 s=CLOSED pgs=2519 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T15:12:10.149 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:10.146+0000 7f3d23984640 1 --2- 172.21.15.43:0/2589289407 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f3d1c15cce0 0x7f3d1c15ac00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T15:12:10.149 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:10.146+0000 7f3d23984640 1 --2- 172.21.15.43:0/2589289407 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f3d1c15c900 0x7f3d1c15fe90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T15:12:10.149 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:10.146+0000 7f3d23984640 1 -- 172.21.15.43:0/2589289407 >> 172.21.15.43:0/2589289407 conn(0x7f3d1c069d70 msgr2=0x7f3d1c0c9b70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-09T15:12:10.149 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:10.146+0000 7f3d23984640 1 -- 172.21.15.43:0/2589289407 shutdown_connections 2023-08-09T15:12:10.149 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:10.146+0000 7f3d23984640 1 -- 172.21.15.43:0/2589289407 wait complete. 2023-08-09T15:12:10.168 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-08-09T15:12:10.398 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:10.396+0000 7fbc4f59e640 1 Processor -- start 2023-08-09T15:12:10.399 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:10.397+0000 7fbc4f59e640 1 -- start start 2023-08-09T15:12:10.400 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:10.398+0000 7fbc4f59e640 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7fbc5015c7e0 0x7fbc5006f500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T15:12:10.400 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:10.398+0000 7fbc4f59e640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7fbc5006fa40 0x7fbc500ab760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T15:12:10.401 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:10.398+0000 7fbc4f59e640 1 -- --> v1:172.21.15.43:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbc5006fe90 con 0x7fbc5015c400 2023-08-09T15:12:10.401 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:10.398+0000 7fbc4f59e640 1 -- --> v2:172.21.15.43:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbc500aad10 con 0x7fbc5006fa40 2023-08-09T15:12:10.401 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:10.398+0000 7fbc4f59e640 1 -- --> v2:172.21.15.203:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbc500ab2d0 con 0x7fbc5015c7e0 2023-08-09T15:12:10.402 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:10.398+0000 7fbc4e59c640 1 --1- >> v1:172.21.15.43:6789/0 conn(0x7fbc5015c400 0x7fbc500c9080 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.43:6789/0 says I am v1:172.21.15.43:36694/0 (socket says 172.21.15.43:36694) 2023-08-09T15:12:10.402 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:10.399+0000 7fbc4e59c640 1 -- 172.21.15.43:0/405269951 learned_addr learned my addr 172.21.15.43:0/405269951 (peer_addr_for_me v1:172.21.15.43:0/0) 2023-08-09T15:12:10.402 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:10.399+0000 7fbc4ed9d640 1 --2- 172.21.15.43:0/405269951 >> v2:172.21.15.43:3300/0 conn(0x7fbc5006fa40 0x7fbc500ab760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T15:12:10.402 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:10.399+0000 7fbc4dd9b640 1 --2- 172.21.15.43:0/405269951 >> v2:172.21.15.203:3300/0 conn(0x7fbc5015c7e0 0x7fbc5006f500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T15:12:10.405 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:10.399+0000 7fbc4d59a640 1 -- 172.21.15.43:0/405269951 <== mon.0 v1:172.21.15.43:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1252887951 0 0) 0x7fbc40001110 con 0x7fbc5015c400 2023-08-09T15:12:10.406 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:10.399+0000 7fbc4d59a640 1 -- 172.21.15.43:0/405269951 --> v1:172.21.15.43:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fbc38003520 con 0x7fbc5015c400 2023-08-09T15:12:10.406 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:10.399+0000 7fbc4ed9d640 1 -- 172.21.15.43:0/405269951 >> v2:172.21.15.203:3300/0 conn(0x7fbc5015c7e0 msgr2=0x7fbc5006f500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-09T15:12:10.406 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:10.399+0000 7fbc4ed9d640 1 --2- 172.21.15.43:0/405269951 >> v2:172.21.15.203:3300/0 conn(0x7fbc5015c7e0 0x7fbc5006f500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T15:12:10.406 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:10.399+0000 7fbc4ed9d640 1 -- 172.21.15.43:0/405269951 >> v1:172.21.15.43:6789/0 conn(0x7fbc5015c400 legacy=0x7fbc500c9080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T15:12:10.407 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:10.399+0000 7fbc4ed9d640 1 -- 172.21.15.43:0/405269951 --> v2:172.21.15.43:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbc500c9bf0 con 0x7fbc5006fa40 2023-08-09T15:12:10.407 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:10.399+0000 7fbc4dd9b640 1 --2- 172.21.15.43:0/405269951 >> v2:172.21.15.203:3300/0 conn(0x7fbc5015c7e0 0x7fbc5006f500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-08-09T15:12:10.407 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:10.400+0000 7fbc4ed9d640 1 --2- 172.21.15.43:0/405269951 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fbc5006fa40 0x7fbc500ab760 secure :-1 s=READY pgs=2521 cs=0 l=1 rev1=1 crypto rx=0x7fbc50160110 tx=0x7fbc4802bec0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=460b95fbad1a5ba5 server_cookie=0 in_seq=0 out_seq=0 2023-08-09T15:12:10.408 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:10.401+0000 7fbc4d59a640 1 -- 172.21.15.43:0/405269951 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbc4802cc70 con 0x7fbc5006fa40 2023-08-09T15:12:10.408 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:10.401+0000 7fbc4d59a640 1 -- 172.21.15.43:0/405269951 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbc4802cdd0 con 0x7fbc5006fa40 2023-08-09T15:12:10.408 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:10.401+0000 7fbc4d59a640 1 -- 172.21.15.43:0/405269951 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbc48034640 con 0x7fbc5006fa40 2023-08-09T15:12:10.408 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:10.401+0000 7fbc4f59e640 1 -- 172.21.15.43:0/405269951 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fbc5006fa40 msgr2=0x7fbc500ab760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T15:12:10.409 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:10.401+0000 7fbc4f59e640 1 --2- 172.21.15.43:0/405269951 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fbc5006fa40 0x7fbc500ab760 secure :-1 s=READY pgs=2521 cs=0 l=1 rev1=1 crypto rx=0x7fbc50160110 tx=0x7fbc4802bec0 comp rx=0 tx=0).stop 2023-08-09T15:12:10.409 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:10.402+0000 7fbc4f59e640 1 -- 172.21.15.43:0/405269951 shutdown_connections 2023-08-09T15:12:10.409 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:10.402+0000 7fbc4f59e640 1 --2- 172.21.15.43:0/405269951 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fbc5006fa40 0x7fbc500ab760 unknown :-1 s=CLOSED pgs=2521 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T15:12:10.410 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:10.402+0000 7fbc4f59e640 1 --2- 172.21.15.43:0/405269951 >> v2:172.21.15.203:3300/0 conn(0x7fbc5015c7e0 0x7fbc5006f500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T15:12:10.410 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:10.402+0000 7fbc4f59e640 1 -- 172.21.15.43:0/405269951 >> 172.21.15.43:0/405269951 conn(0x7fbc500ce020 msgr2=0x7fbc500ce430 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-09T15:12:10.410 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:10.402+0000 7fbc4f59e640 1 -- 172.21.15.43:0/405269951 shutdown_connections 2023-08-09T15:12:10.410 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:10.402+0000 7fbc4f59e640 1 -- 172.21.15.43:0/405269951 wait complete. 2023-08-09T15:12:10.411 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:10.403+0000 7fbc4f59e640 1 Processor -- start 2023-08-09T15:12:10.411 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:10.404+0000 7fbc4f59e640 1 -- start start 2023-08-09T15:12:10.414 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:10.405+0000 7fbc4f59e640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fbc5006fa40 0x7fbc501f1870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T15:12:10.414 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:10.405+0000 7fbc4f59e640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fbc5015c400 0x7fbc500b14c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T15:12:10.415 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:10.405+0000 7fbc4f59e640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fbc5015c7e0 0x7fbc50158f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T15:12:10.415 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:10.405+0000 7fbc4f59e640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbc500b2240 con 0x7fbc5015c7e0 2023-08-09T15:12:10.415 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:10.405+0000 7fbc4f59e640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbc50099240 con 0x7fbc5006fa40 2023-08-09T15:12:10.415 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:10.405+0000 7fbc4f59e640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbc500b1f60 con 0x7fbc5015c400 2023-08-09T15:12:10.416 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:10.405+0000 7fbc4ed9d640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fbc5015c7e0 0x7fbc50158f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T15:12:10.417 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:10.405+0000 7fbc4ed9d640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fbc5015c7e0 0x7fbc50158f70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:33302/0 (socket says 172.21.15.43:33302) 2023-08-09T15:12:10.417 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:10.405+0000 7fbc4ed9d640 1 -- 172.21.15.43:0/3740259100 learned_addr learned my addr 172.21.15.43:0/3740259100 (peer_addr_for_me v2:172.21.15.43:0/0) 2023-08-09T15:12:10.417 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:10.405+0000 7fbc4e59c640 1 --2- 172.21.15.43:0/3740259100 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fbc5006fa40 0x7fbc501f1870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T15:12:10.417 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:10.406+0000 7fbc4ed9d640 1 -- 172.21.15.43:0/3740259100 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fbc5015c400 msgr2=0x7fbc500b14c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T15:12:10.417 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:10.406+0000 7fbc4ed9d640 1 --2- 172.21.15.43:0/3740259100 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fbc5015c400 0x7fbc500b14c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T15:12:10.417 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:10.406+0000 7fbc4ed9d640 1 -- 172.21.15.43:0/3740259100 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fbc5006fa40 msgr2=0x7fbc501f1870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T15:12:10.418 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:10.406+0000 7fbc4ed9d640 1 --2- 172.21.15.43:0/3740259100 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fbc5006fa40 0x7fbc501f1870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T15:12:10.418 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:10.406+0000 7fbc4ed9d640 1 -- 172.21.15.43:0/3740259100 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbc48009a70 con 0x7fbc5015c7e0 2023-08-09T15:12:10.418 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:10.406+0000 7fbc4ed9d640 1 --2- 172.21.15.43:0/3740259100 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fbc5015c7e0 0x7fbc50158f70 secure :-1 s=READY pgs=2522 cs=0 l=1 rev1=1 crypto rx=0x7fbc4802c4d0 tx=0x7fbc48004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-09T15:12:10.418 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:10.406+0000 7fbc3effd640 1 -- 172.21.15.43:0/3740259100 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbc4803e610 con 0x7fbc5015c7e0 2023-08-09T15:12:10.418 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:10.406+0000 7fbc3effd640 1 -- 172.21.15.43:0/3740259100 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbc4803e770 con 0x7fbc5015c7e0 2023-08-09T15:12:10.418 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:10.406+0000 7fbc3effd640 1 -- 172.21.15.43:0/3740259100 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbc4803eb60 con 0x7fbc5015c7e0 2023-08-09T15:12:10.418 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:10.406+0000 7fbc4f59e640 1 -- 172.21.15.43:0/3740259100 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbc50098b80 con 0x7fbc5015c7e0 2023-08-09T15:12:10.419 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:10.407+0000 7fbc4f59e640 1 -- 172.21.15.43:0/3740259100 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbc501f1db0 con 0x7fbc5015c7e0 2023-08-09T15:12:10.419 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:10.408+0000 7fbc4f59e640 1 -- 172.21.15.43:0/3740259100 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbc500989e0 con 0x7fbc5015c7e0 2023-08-09T15:12:10.419 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:10.409+0000 7fbc3effd640 1 -- 172.21.15.43:0/3740259100 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 128) v1 ==== 88387+0+0 (secure 0 0 0) 0x7fbc4803b050 con 0x7fbc5015c7e0 2023-08-09T15:12:10.419 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:10.409+0000 7fbc3effd640 1 --2- 172.21.15.43:0/3740259100 >> [v2:172.21.15.203:6832/1456699667,v1:172.21.15.203:6833/1456699667] conn(0x7fbc3006a0e0 0x7fbc3006c5a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T15:12:10.420 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:10.410+0000 7fbc3effd640 1 -- 172.21.15.43:0/3740259100 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(226..226 src has 1..226) v4 ==== 7256+0+0 (secure 0 0 0) 0x7fbc480aaca0 con 0x7fbc5015c7e0 2023-08-09T15:12:10.420 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:10.410+0000 7fbc4e59c640 1 --2- 172.21.15.43:0/3740259100 >> [v2:172.21.15.203:6832/1456699667,v1:172.21.15.203:6833/1456699667] conn(0x7fbc3006a0e0 0x7fbc3006c5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T15:12:10.420 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:10.411+0000 7fbc4e59c640 1 --2- 172.21.15.43:0/3740259100 >> [v2:172.21.15.203:6832/1456699667,v1:172.21.15.203:6833/1456699667] conn(0x7fbc3006a0e0 0x7fbc3006c5a0 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7fbc40001270 tx=0x7fbc40009040 comp rx=0 tx=0).ready entity=mgr.9232 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-09T15:12:10.421 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:10.413+0000 7fbc3effd640 1 -- 172.21.15.43:0/3740259100 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178066 (secure 0 0 0) 0x7fbc500989e0 con 0x7fbc5015c7e0 2023-08-09T15:12:10.610 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:10.609+0000 7fbc4f59e640 1 -- 172.21.15.43:0/3740259100 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fbc500989e0 con 0x7fbc5015c7e0 2023-08-09T15:12:10.614 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:10.612+0000 7fbc3effd640 1 -- 172.21.15.43:0/3740259100 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v226) v1 ==== 74+0+19824 (secure 0 0 0) 0x7fbc500989e0 con 0x7fbc5015c7e0 2023-08-09T15:12:10.614 INFO:teuthology.orchestra.run.smithi043.stdout: 2023-08-09T15:12:10.615 INFO:teuthology.orchestra.run.smithi043.stdout:{"epoch":226,"fsid":"daac87ee-1337-4250-a955-aea4e408a39b","created":"2023-08-09T13:33:41.701198+0000","modified":"2023-08-09T15:12:08.329952+0000","last_up_change":"2023-08-09T13:33:53.783453+0000","last_in_change":"2023-08-09T13:33:45.148904+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":27,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":"rbd","create_time":"2023-08-09T13:33:54.365137+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":".mgr","create_time":"2023-08-09T13:33:54.831803+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":26,"pool_name":"cephfs_metadata","create_time":"2023-08-09T15:11:26.544757+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"222","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":27,"pool_name":"cephfs_data","create_time":"2023-08-09T15:11:27.194686+0000","flags":8193,"flags_names":"hashpspool,selfmanaged_snaps","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"226","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":3,"snap_epoch":226,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.6299999952316284,"score_stable":1.6299999952316284,"optimal_score":1,"raw_score_acting":1.6299999952316284,"raw_score_stable":1.6299999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"cbd49ecd-e896-4fbd-864e-5c37a829b165","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":221,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6800","nonce":502667335},{"type":"v1","addr":"172.21.15.43:6801","nonce":502667335}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6802","nonce":502667335},{"type":"v1","addr":"172.21.15.43:6803","nonce":502667335}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6806","nonce":502667335},{"type":"v1","addr":"172.21.15.43:6807","nonce":502667335}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6804","nonce":502667335},{"type":"v1","addr":"172.21.15.43:6805","nonce":502667335}]},"public_addr":"172.21.15.43:6801/502667335","cluster_addr":"172.21.15.43:6803/502667335","heartbeat_back_addr":"172.21.15.43:6807/502667335","heartbeat_front_addr":"172.21.15.43:6805/502667335","state":["exists","up"]},{"osd":1,"uuid":"75c7e0fe-1ec7-4fe8-a2f2-3661a65b8500","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":221,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6808","nonce":3289134610},{"type":"v1","addr":"172.21.15.43:6809","nonce":3289134610}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6810","nonce":3289134610},{"type":"v1","addr":"172.21.15.43:6812","nonce":3289134610}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6821","nonce":3289134610},{"type":"v1","addr":"172.21.15.43:6824","nonce":3289134610}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6815","nonce":3289134610},{"type":"v1","addr":"172.21.15.43:6818","nonce":3289134610}]},"public_addr":"172.21.15.43:6809/3289134610","cluster_addr":"172.21.15.43:6812/3289134610","heartbeat_back_addr":"172.21.15.43:6824/3289134610","heartbeat_front_addr":"172.21.15.43:6818/3289134610","state":["exists","up"]},{"osd":2,"uuid":"bb5c41e9-7f5f-4125-9b01-3bb8bfc2ea83","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":221,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6811","nonce":4079709736},{"type":"v1","addr":"172.21.15.43:6814","nonce":4079709736}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6817","nonce":4079709736},{"type":"v1","addr":"172.21.15.43:6820","nonce":4079709736}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6829","nonce":4079709736},{"type":"v1","addr":"172.21.15.43:6831","nonce":4079709736}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6823","nonce":4079709736},{"type":"v1","addr":"172.21.15.43:6826","nonce":4079709736}]},"public_addr":"172.21.15.43:6814/4079709736","cluster_addr":"172.21.15.43:6820/4079709736","heartbeat_back_addr":"172.21.15.43:6831/4079709736","heartbeat_front_addr":"172.21.15.43:6826/4079709736","state":["exists","up"]},{"osd":3,"uuid":"bfe4c8b4-694a-465c-87e6-2b2aab852613","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":221,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6813","nonce":1709538759},{"type":"v1","addr":"172.21.15.43:6816","nonce":1709538759}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6819","nonce":1709538759},{"type":"v1","addr":"172.21.15.43:6822","nonce":1709538759}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6828","nonce":1709538759},{"type":"v1","addr":"172.21.15.43:6830","nonce":1709538759}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6825","nonce":1709538759},{"type":"v1","addr":"172.21.15.43:6827","nonce":1709538759}]},"public_addr":"172.21.15.43:6816/1709538759","cluster_addr":"172.21.15.43:6822/1709538759","heartbeat_back_addr":"172.21.15.43:6830/1709538759","heartbeat_front_addr":"172.21.15.43:6827/1709538759","state":["exists","up"]},{"osd":4,"uuid":"fa2e8a77-aba1-4756-853e-0d174a5bdeba","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":221,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6800","nonce":4196866481},{"type":"v1","addr":"172.21.15.203:6801","nonce":4196866481}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6802","nonce":4196866481},{"type":"v1","addr":"172.21.15.203:6803","nonce":4196866481}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6806","nonce":4196866481},{"type":"v1","addr":"172.21.15.203:6807","nonce":4196866481}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6804","nonce":4196866481},{"type":"v1","addr":"172.21.15.203:6805","nonce":4196866481}]},"public_addr":"172.21.15.203:6801/4196866481","cluster_addr":"172.21.15.203:6803/4196866481","heartbeat_back_addr":"172.21.15.203:6807/4196866481","heartbeat_front_addr":"172.21.15.203:6805/4196866481","state":["exists","up"]},{"osd":5,"uuid":"81811e4b-4d24-4f51-a82c-f1d700a2a00d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":221,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6816","nonce":3059290081},{"type":"v1","addr":"172.21.15.203:6817","nonce":3059290081}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6818","nonce":3059290081},{"type":"v1","addr":"172.21.15.203:6819","nonce":3059290081}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6822","nonce":3059290081},{"type":"v1","addr":"172.21.15.203:6823","nonce":3059290081}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6820","nonce":3059290081},{"type":"v1","addr":"172.21.15.203:6821","nonce":3059290081}]},"public_addr":"172.21.15.203:6817/3059290081","cluster_addr":"172.21.15.203:6819/3059290081","heartbeat_back_addr":"172.21.15.203:6823/3059290081","heartbeat_front_addr":"172.21.15.203:6821/3059290081","state":["exists","up"]},{"osd":6,"uuid":"307bac4d-6a5b-443d-af75-56d695d7674f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":221,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6824","nonce":512155569},{"type":"v1","addr":"172.21.15.203:6825","nonce":512155569}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6826","nonce":512155569},{"type":"v1","addr":"172.21.15.203:6827","nonce":512155569}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6830","nonce":512155569},{"type":"v1","addr":"172.21.15.203:6831","nonce":512155569}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6828","nonce":512155569},{"type":"v1","addr":"172.21.15.203:6829","nonce":512155569}]},"public_addr":"172.21.15.203:6825/512155569","cluster_addr":"172.21.15.203:6827/512155569","heartbeat_back_addr":"172.21.15.203:6831/512155569","heartbeat_front_addr":"172.21.15.203:6829/512155569","state":["exists","up"]},{"osd":7,"uuid":"a95dc7ca-9f1b-4295-af70-68a22a2d7d5e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":221,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6808","nonce":3331450438},{"type":"v1","addr":"172.21.15.203:6809","nonce":3331450438}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6810","nonce":3331450438},{"type":"v1","addr":"172.21.15.203:6811","nonce":3331450438}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6814","nonce":3331450438},{"type":"v1","addr":"172.21.15.203:6815","nonce":3331450438}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6812","nonce":3331450438},{"type":"v1","addr":"172.21.15.203:6813","nonce":3331450438}]},"public_addr":"172.21.15.203:6809/3331450438","cluster_addr":"172.21.15.203:6811/3331450438","heartbeat_back_addr":"172.21.15.203:6815/3331450438","heartbeat_front_addr":"172.21.15.203:6813/3331450438","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-09T13:33:50.547652+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-09T13:33:50.589469+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-09T13:33:50.561724+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-09T13:33:50.599650+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-09T13:33:50.411855+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-09T13:33:50.219681+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-09T13:33:50.504664+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-09T13:33:50.441248+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.203:0/4004150203":"2023-08-10T15:12:07.324983+0000","172.21.15.203:6833/2375771024":"2023-08-10T15:12:07.324983+0000","172.21.15.203:6832/2375771024":"2023-08-10T15:12:07.324983+0000","172.21.15.203:0/1645319310":"2023-08-10T15:12:07.324983+0000","172.21.15.203:0/3324686399":"2023-08-10T15:12:07.324983+0000","172.21.15.203:0/286395424":"2023-08-10T15:12:07.324983+0000","172.21.15.203:0/1761748226":"2023-08-10T15:12:07.324983+0000","172.21.15.203:6832/4034093268":"2023-08-10T15:11:43.130145+0000","172.21.15.203:6833/4034093268":"2023-08-10T15:11:43.130145+0000","172.21.15.203:0/1654193324":"2023-08-10T15:11:43.130145+0000","172.21.15.203:0/3701038305":"2023-08-10T15:12:07.324983+0000","172.21.15.203:0/4140323243":"2023-08-10T15:11:43.130145+0000","172.21.15.203:0/450863258":"2023-08-10T15:11:43.130145+0000","172.21.15.203:0/134877587":"2023-08-10T15:11:43.130145+0000","172.21.15.203:0/3500453783":"2023-08-10T15:12:07.324983+0000","172.21.15.203:0/2330869600":"2023-08-10T15:11:43.130145+0000","172.21.15.203:0/1886393026":"2023-08-10T15:12:07.324983+0000","172.21.15.203:0/2203818837":"2023-08-10T15:11:43.130145+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":7,"snaps":[{"begin":104,"length":1},{"begin":221,"length":1}]},{"pool":27,"snaps":[{"begin":2,"length":2}]}],"new_removed_snaps":[{"pool":27,"snaps":[{"begin":2,"length":2}]}],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-08-09T15:12:10.617 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:10.615+0000 7fbc4f59e640 1 -- 172.21.15.43:0/3740259100 >> [v2:172.21.15.203:6832/1456699667,v1:172.21.15.203:6833/1456699667] conn(0x7fbc3006a0e0 msgr2=0x7fbc3006c5a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T15:12:10.618 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:10.615+0000 7fbc4f59e640 1 --2- 172.21.15.43:0/3740259100 >> [v2:172.21.15.203:6832/1456699667,v1:172.21.15.203:6833/1456699667] conn(0x7fbc3006a0e0 0x7fbc3006c5a0 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7fbc40001270 tx=0x7fbc40009040 comp rx=0 tx=0).stop 2023-08-09T15:12:10.618 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:10.615+0000 7fbc4f59e640 1 -- 172.21.15.43:0/3740259100 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fbc5015c7e0 msgr2=0x7fbc50158f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T15:12:10.619 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:10.615+0000 7fbc4f59e640 1 --2- 172.21.15.43:0/3740259100 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fbc5015c7e0 0x7fbc50158f70 secure :-1 s=READY pgs=2522 cs=0 l=1 rev1=1 crypto rx=0x7fbc4802c4d0 tx=0x7fbc48004a40 comp rx=0 tx=0).stop 2023-08-09T15:12:10.619 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:10.615+0000 7fbc4f59e640 1 -- 172.21.15.43:0/3740259100 shutdown_connections 2023-08-09T15:12:10.619 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:10.615+0000 7fbc4f59e640 1 --2- 172.21.15.43:0/3740259100 >> [v2:172.21.15.203:6832/1456699667,v1:172.21.15.203:6833/1456699667] conn(0x7fbc3006a0e0 0x7fbc3006c5a0 unknown :-1 s=CLOSED pgs=26 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T15:12:10.619 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:10.615+0000 7fbc4f59e640 1 --2- 172.21.15.43:0/3740259100 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fbc5015c7e0 0x7fbc50158f70 unknown :-1 s=CLOSED pgs=2522 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T15:12:10.620 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:10.615+0000 7fbc4f59e640 1 --2- 172.21.15.43:0/3740259100 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fbc5015c400 0x7fbc500b14c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T15:12:10.620 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:10.615+0000 7fbc4f59e640 1 --2- 172.21.15.43:0/3740259100 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fbc5006fa40 0x7fbc501f1870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T15:12:10.620 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:10.615+0000 7fbc4f59e640 1 -- 172.21.15.43:0/3740259100 >> 172.21.15.43:0/3740259100 conn(0x7fbc500ce020 msgr2=0x7fbc500cbbd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-09T15:12:10.621 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:10.616+0000 7fbc4f59e640 1 -- 172.21.15.43:0/3740259100 shutdown_connections 2023-08-09T15:12:10.621 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:10.616+0000 7fbc4f59e640 1 -- 172.21.15.43:0/3740259100 wait complete. 2023-08-09T15:12:10.636 INFO:tasks.cephfs.filesystem:Destroying file system cephfs and related pools 2023-08-09T15:12:10.636 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-08-09T15:12:10.863 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:10.862+0000 7f3bd8e8a640 1 Processor -- start 2023-08-09T15:12:10.864 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:10.862+0000 7f3bd8e8a640 1 -- start start 2023-08-09T15:12:10.866 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:10.863+0000 7f3bd8e8a640 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7f3bd40b3ef0 0x7f3bd40d1e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T15:12:10.866 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:10.863+0000 7f3bd8e8a640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f3bd40b8100 0x7f3bd40b42d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T15:12:10.866 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:10.863+0000 7f3bd8e8a640 1 -- --> v1:172.21.15.43:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3bd40b4810 con 0x7f3bd40d04b0 2023-08-09T15:12:10.866 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:10.863+0000 7f3bd8e8a640 1 -- --> v2:172.21.15.43:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3bd40d0d70 con 0x7f3bd40b8100 2023-08-09T15:12:10.867 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:10.863+0000 7f3bd8e8a640 1 -- --> v2:172.21.15.203:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3bd40d1050 con 0x7f3bd40b3ef0 2023-08-09T15:12:10.867 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:10.863+0000 7f3bd2ffd640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f3bd40b8100 0x7f3bd40b42d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T15:12:10.867 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:10.863+0000 7f3bd3fff640 1 --1- >> v1:172.21.15.43:6789/0 conn(0x7f3bd40d04b0 0x7f3bd40b54e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.43:6789/0 says I am v1:172.21.15.43:36698/0 (socket says 172.21.15.43:36698) 2023-08-09T15:12:10.867 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:10.863+0000 7f3bd3fff640 1 -- 172.21.15.43:0/3074283287 learned_addr learned my addr 172.21.15.43:0/3074283287 (peer_addr_for_me v1:172.21.15.43:0/0) 2023-08-09T15:12:10.867 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:10.864+0000 7f3bd37fe640 1 --2- 172.21.15.43:0/3074283287 >> v2:172.21.15.203:3300/0 conn(0x7f3bd40b3ef0 0x7f3bd40d1e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T15:12:10.867 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:10.864+0000 7f3bd2ffd640 1 -- 172.21.15.43:0/3074283287 >> v2:172.21.15.203:3300/0 conn(0x7f3bd40b3ef0 msgr2=0x7f3bd40d1e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-09T15:12:10.867 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:10.864+0000 7f3bd2ffd640 1 --2- 172.21.15.43:0/3074283287 >> v2:172.21.15.203:3300/0 conn(0x7f3bd40b3ef0 0x7f3bd40d1e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T15:12:10.868 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:10.864+0000 7f3bd2ffd640 1 -- 172.21.15.43:0/3074283287 >> v1:172.21.15.43:6789/0 conn(0x7f3bd40d04b0 legacy=0x7f3bd40b54e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T15:12:10.868 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:10.864+0000 7f3bd2ffd640 1 -- 172.21.15.43:0/3074283287 --> v2:172.21.15.43:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3bd40d1330 con 0x7f3bd40b8100 2023-08-09T15:12:10.869 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:10.865+0000 7f3bd2ffd640 1 --2- 172.21.15.43:0/3074283287 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f3bd40b8100 0x7f3bd40b42d0 secure :-1 s=READY pgs=2524 cs=0 l=1 rev1=1 crypto rx=0x7f3bd40d3510 tx=0x7f3bbc02b7a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5d3269853c933749 server_cookie=0 in_seq=0 out_seq=0 2023-08-09T15:12:10.869 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:10.865+0000 7f3bd27fc640 1 -- 172.21.15.43:0/3074283287 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3bbc033640 con 0x7f3bd40b8100 2023-08-09T15:12:10.869 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:10.866+0000 7f3bd27fc640 1 -- 172.21.15.43:0/3074283287 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3bbc0337a0 con 0x7f3bd40b8100 2023-08-09T15:12:10.869 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:10.866+0000 7f3bd27fc640 1 -- 172.21.15.43:0/3074283287 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3bbc033b30 con 0x7f3bd40b8100 2023-08-09T15:12:10.869 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:10.866+0000 7f3bd8e8a640 1 -- 172.21.15.43:0/3074283287 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f3bd40b8100 msgr2=0x7f3bd40b42d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T15:12:10.869 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:10.866+0000 7f3bd8e8a640 1 --2- 172.21.15.43:0/3074283287 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f3bd40b8100 0x7f3bd40b42d0 secure :-1 s=READY pgs=2524 cs=0 l=1 rev1=1 crypto rx=0x7f3bd40d3510 tx=0x7f3bbc02b7a0 comp rx=0 tx=0).stop 2023-08-09T15:12:10.870 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:10.866+0000 7f3bd8e8a640 1 -- 172.21.15.43:0/3074283287 shutdown_connections 2023-08-09T15:12:10.870 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:10.866+0000 7f3bd8e8a640 1 --2- 172.21.15.43:0/3074283287 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f3bd40b8100 0x7f3bd40b42d0 unknown :-1 s=CLOSED pgs=2524 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T15:12:10.870 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:10.866+0000 7f3bd8e8a640 1 --2- 172.21.15.43:0/3074283287 >> v2:172.21.15.203:3300/0 conn(0x7f3bd40b3ef0 0x7f3bd40d1e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T15:12:10.870 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:10.866+0000 7f3bd8e8a640 1 -- 172.21.15.43:0/3074283287 >> 172.21.15.43:0/3074283287 conn(0x7f3bd4071260 msgr2=0x7f3bd4071670 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-09T15:12:10.870 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:10.866+0000 7f3bd8e8a640 1 -- 172.21.15.43:0/3074283287 shutdown_connections 2023-08-09T15:12:10.870 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:10.866+0000 7f3bd8e8a640 1 -- 172.21.15.43:0/3074283287 wait complete. 2023-08-09T15:12:10.872 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:10.868+0000 7f3bd8e8a640 1 Processor -- start 2023-08-09T15:12:10.872 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:10.868+0000 7f3bd8e8a640 1 -- start start 2023-08-09T15:12:10.872 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:10.869+0000 7f3bd8e8a640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f3bd40b3ef0 0x7f3bd41f15f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T15:12:10.873 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:10.869+0000 7f3bd8e8a640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f3bd40b8100 0x7f3bd415d370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T15:12:10.873 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:10.869+0000 7f3bd8e8a640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f3bd40d04b0 0x7f3bd415d8b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T15:12:10.873 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:10.869+0000 7f3bd8e8a640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3bd415e520 con 0x7f3bd40b8100 2023-08-09T15:12:10.873 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:10.869+0000 7f3bd8e8a640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3bd40cb880 con 0x7f3bd40b3ef0 2023-08-09T15:12:10.873 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:10.869+0000 7f3bd8e8a640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3bd40cbb60 con 0x7f3bd40d04b0 2023-08-09T15:12:10.873 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:10.870+0000 7f3bd2ffd640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f3bd40b8100 0x7f3bd415d370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T15:12:10.873 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:10.870+0000 7f3bd2ffd640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f3bd40b8100 0x7f3bd415d370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:33322/0 (socket says 172.21.15.43:33322) 2023-08-09T15:12:10.874 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:10.870+0000 7f3bd2ffd640 1 -- 172.21.15.43:0/2702309615 learned_addr learned my addr 172.21.15.43:0/2702309615 (peer_addr_for_me v2:172.21.15.43:0/0) 2023-08-09T15:12:10.874 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:10.870+0000 7f3bd37fe640 1 --2- 172.21.15.43:0/2702309615 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f3bd40b3ef0 0x7f3bd41f15f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T15:12:10.874 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:10.870+0000 7f3bd3fff640 1 --2- 172.21.15.43:0/2702309615 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f3bd40d04b0 0x7f3bd415d8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T15:12:10.874 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:10.870+0000 7f3bd2ffd640 1 -- 172.21.15.43:0/2702309615 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f3bd40d04b0 msgr2=0x7f3bd415d8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T15:12:10.874 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:10.870+0000 7f3bd2ffd640 1 --2- 172.21.15.43:0/2702309615 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f3bd40d04b0 0x7f3bd415d8b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T15:12:10.874 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:10.870+0000 7f3bd2ffd640 1 -- 172.21.15.43:0/2702309615 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f3bd40b3ef0 msgr2=0x7f3bd41f15f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T15:12:10.874 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:10.870+0000 7f3bd2ffd640 1 --2- 172.21.15.43:0/2702309615 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f3bd40b3ef0 0x7f3bd41f15f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T15:12:10.875 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:10.870+0000 7f3bd2ffd640 1 -- 172.21.15.43:0/2702309615 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3bbc009400 con 0x7f3bd40b8100 2023-08-09T15:12:10.875 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:10.870+0000 7f3bd37fe640 1 --2- 172.21.15.43:0/2702309615 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f3bd40b3ef0 0x7f3bd41f15f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-08-09T15:12:10.875 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:10.870+0000 7f3bd3fff640 1 --2- 172.21.15.43:0/2702309615 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f3bd40d04b0 0x7f3bd415d8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-08-09T15:12:10.875 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:10.871+0000 7f3bd2ffd640 1 --2- 172.21.15.43:0/2702309615 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f3bd40b8100 0x7f3bd415d370 secure :-1 s=READY pgs=2525 cs=0 l=1 rev1=1 crypto rx=0x7f3bbc000c00 tx=0x7f3bbc0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-09T15:12:10.876 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:10.871+0000 7f3bcbfff640 1 -- 172.21.15.43:0/2702309615 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3bbc030cf0 con 0x7f3bd40b8100 2023-08-09T15:12:10.876 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:10.871+0000 7f3bcbfff640 1 -- 172.21.15.43:0/2702309615 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3bbc030e70 con 0x7f3bd40b8100 2023-08-09T15:12:10.877 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:10.871+0000 7f3bcbfff640 1 -- 172.21.15.43:0/2702309615 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3bbc03c640 con 0x7f3bd40b8100 2023-08-09T15:12:10.877 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:10.871+0000 7f3bd8e8a640 1 -- 172.21.15.43:0/2702309615 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3bd40cbe40 con 0x7f3bd40b8100 2023-08-09T15:12:10.877 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:10.871+0000 7f3bd8e8a640 1 -- 172.21.15.43:0/2702309615 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3bd41f1b30 con 0x7f3bd40b8100 2023-08-09T15:12:10.877 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:10.872+0000 7f3bd8e8a640 1 -- 172.21.15.43:0/2702309615 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3bd404e390 con 0x7f3bd40b8100 2023-08-09T15:12:10.877 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:10.873+0000 7f3bcbfff640 1 -- 172.21.15.43:0/2702309615 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 128) v1 ==== 88387+0+0 (secure 0 0 0) 0x7f3bbc03a040 con 0x7f3bd40b8100 2023-08-09T15:12:10.877 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:10.874+0000 7f3bcbfff640 1 --2- 172.21.15.43:0/2702309615 >> [v2:172.21.15.203:6832/1456699667,v1:172.21.15.203:6833/1456699667] conn(0x7f3bb406a0e0 0x7f3bb406c5a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T15:12:10.877 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:10.874+0000 7f3bcbfff640 1 -- 172.21.15.43:0/2702309615 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(226..226 src has 1..226) v4 ==== 7256+0+0 (secure 0 0 0) 0x7f3bbc033e40 con 0x7f3bd40b8100 2023-08-09T15:12:10.878 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:10.875+0000 7f3bd37fe640 1 --2- 172.21.15.43:0/2702309615 >> [v2:172.21.15.203:6832/1456699667,v1:172.21.15.203:6833/1456699667] conn(0x7f3bb406a0e0 0x7f3bb406c5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T15:12:10.878 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:10.876+0000 7f3bd37fe640 1 --2- 172.21.15.43:0/2702309615 >> [v2:172.21.15.203:6832/1456699667,v1:172.21.15.203:6833/1456699667] conn(0x7f3bb406a0e0 0x7f3bb406c5a0 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7f3bc4006fd0 tx=0x7f3bc4008040 comp rx=0 tx=0).ready entity=mgr.9232 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-09T15:12:10.879 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:10.878+0000 7f3bcbfff640 1 -- 172.21.15.43:0/2702309615 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178066 (secure 0 0 0) 0x7f3bbc075aa0 con 0x7f3bd40b8100 2023-08-09T15:12:11.114 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:11.112+0000 7f3bd8e8a640 1 -- 172.21.15.43:0/2702309615 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f3bd40d15e0 con 0x7f3bd40b8100 2023-08-09T15:12:11.115 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:11.114+0000 7f3bcbfff640 1 -- 172.21.15.43:0/2702309615 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 104 v104) v1 ==== 95+0+4894 (secure 0 0 0) 0x7f3bbc038090 con 0x7f3bd40b8100 2023-08-09T15:12:11.116 INFO:teuthology.orchestra.run.smithi043.stdout: 2023-08-09T15:12:11.116 INFO:teuthology.orchestra.run.smithi043.stdout:{"epoch":104,"default_fscid":12,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":8012,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.203:6837/2464832767","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6836","nonce":2464832767},{"type":"v1","addr":"172.21.15.203:6837","nonce":2464832767}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":100},{"gid":9010,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.43:6835/3989386049","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6834","nonce":3989386049},{"type":"v1","addr":"172.21.15.43:6835","nonce":3989386049}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":100},{"gid":9016,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.203:6835/2213455074","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6834","nonce":2213455074},{"type":"v1","addr":"172.21.15.203:6835","nonce":2213455074}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":100}],"filesystems":[{"mdsmap":{"epoch":104,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-08-09T15:11:28.168916+0000","modified":"2023-08-09T15:11:31.739094+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":11325},"failed":[],"damaged":[],"stopped":[],"info":{"gid_11325":{"gid":11325,"name":"a","rank":0,"incarnation":101,"state":"up:active","state_seq":3,"addr":"172.21.15.43:6833/1425989175","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6832","nonce":1425989175},{"type":"v1","addr":"172.21.15.43:6833","nonce":1425989175}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[27],"metadata_pool":26,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":12}]} 2023-08-09T15:12:11.117 INFO:teuthology.orchestra.run.smithi043.stderr:dumped fsmap epoch 104 2023-08-09T15:12:11.119 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:11.117+0000 7f3bd8e8a640 1 -- 172.21.15.43:0/2702309615 >> [v2:172.21.15.203:6832/1456699667,v1:172.21.15.203:6833/1456699667] conn(0x7f3bb406a0e0 msgr2=0x7f3bb406c5a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T15:12:11.119 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:11.117+0000 7f3bd8e8a640 1 --2- 172.21.15.43:0/2702309615 >> [v2:172.21.15.203:6832/1456699667,v1:172.21.15.203:6833/1456699667] conn(0x7f3bb406a0e0 0x7f3bb406c5a0 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7f3bc4006fd0 tx=0x7f3bc4008040 comp rx=0 tx=0).stop 2023-08-09T15:12:11.119 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:11.117+0000 7f3bd8e8a640 1 -- 172.21.15.43:0/2702309615 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f3bd40b8100 msgr2=0x7f3bd415d370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T15:12:11.120 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:11.117+0000 7f3bd8e8a640 1 --2- 172.21.15.43:0/2702309615 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f3bd40b8100 0x7f3bd415d370 secure :-1 s=READY pgs=2525 cs=0 l=1 rev1=1 crypto rx=0x7f3bbc000c00 tx=0x7f3bbc0046c0 comp rx=0 tx=0).stop 2023-08-09T15:12:11.120 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:11.117+0000 7f3bd8e8a640 1 -- 172.21.15.43:0/2702309615 shutdown_connections 2023-08-09T15:12:11.120 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:11.117+0000 7f3bd8e8a640 1 --2- 172.21.15.43:0/2702309615 >> [v2:172.21.15.203:6832/1456699667,v1:172.21.15.203:6833/1456699667] conn(0x7f3bb406a0e0 0x7f3bb406c5a0 unknown :-1 s=CLOSED pgs=27 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T15:12:11.120 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:11.117+0000 7f3bd8e8a640 1 --2- 172.21.15.43:0/2702309615 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f3bd40d04b0 0x7f3bd415d8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T15:12:11.120 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:11.117+0000 7f3bd8e8a640 1 --2- 172.21.15.43:0/2702309615 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f3bd40b8100 0x7f3bd415d370 unknown :-1 s=CLOSED pgs=2525 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T15:12:11.121 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:11.117+0000 7f3bd8e8a640 1 --2- 172.21.15.43:0/2702309615 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f3bd40b3ef0 0x7f3bd41f15f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T15:12:11.121 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:11.117+0000 7f3bd8e8a640 1 -- 172.21.15.43:0/2702309615 >> 172.21.15.43:0/2702309615 conn(0x7f3bd4071260 msgr2=0x7f3bd40b6440 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-09T15:12:11.121 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:11.118+0000 7f3bd8e8a640 1 -- 172.21.15.43:0/2702309615 shutdown_connections 2023-08-09T15:12:11.121 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:11.118+0000 7f3bd8e8a640 1 -- 172.21.15.43:0/2702309615 wait complete. 2023-08-09T15:12:11.138 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-08-09T15:12:11.367 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:11.366+0000 7f111c186640 1 Processor -- start 2023-08-09T15:12:11.367 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:11.366+0000 7f111c186640 1 -- start start 2023-08-09T15:12:11.369 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:11.367+0000 7f111c186640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f111415d060 0x7f11140c8840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T15:12:11.369 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:11.368+0000 7f111c186640 1 -- --> v1:172.21.15.43:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f111415d550 con 0x7f1114156e30 2023-08-09T15:12:11.369 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:11.368+0000 7f111c186640 1 -- --> v1:172.21.15.203:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f111415d830 con 0x7f1114157210 2023-08-09T15:12:11.369 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:11.368+0000 7f111c186640 1 -- --> v2:172.21.15.43:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f11140cd6b0 con 0x7f111415d060 2023-08-09T15:12:11.370 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:11.368+0000 7f1119f22640 1 --1- >> v1:172.21.15.43:6789/0 conn(0x7f1114156e30 0x7f111415a4f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.43:6789/0 says I am v1:172.21.15.43:36702/0 (socket says 172.21.15.43:36702) 2023-08-09T15:12:11.370 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:11.368+0000 7f1119f22640 1 -- 172.21.15.43:0/242323304 learned_addr learned my addr 172.21.15.43:0/242323304 (peer_addr_for_me v1:172.21.15.43:0/0) 2023-08-09T15:12:11.370 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:11.368+0000 7f1119721640 1 --2- 172.21.15.43:0/242323304 >> v2:172.21.15.43:3300/0 conn(0x7f111415d060 0x7f11140c8840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T15:12:11.370 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:11.368+0000 7f1119721640 1 -- 172.21.15.43:0/242323304 >> v1:172.21.15.203:6789/0 conn(0x7f1114157210 legacy=0x7f111406b4b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-09T15:12:11.371 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:11.368+0000 7f1119721640 1 -- 172.21.15.43:0/242323304 >> v1:172.21.15.43:6789/0 conn(0x7f1114156e30 legacy=0x7f111415a4f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T15:12:11.371 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:11.369+0000 7f1119721640 1 -- 172.21.15.43:0/242323304 --> v2:172.21.15.43:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f111415db10 con 0x7f111415d060 2023-08-09T15:12:11.371 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:11.369+0000 7f1118f20640 1 -- 172.21.15.43:0/242323304 <== mon.0 v1:172.21.15.43:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 469747545 0 0) 0x7f111415d550 con 0x7f1114156e30 2023-08-09T15:12:11.371 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:11.370+0000 7f1119721640 1 --2- 172.21.15.43:0/242323304 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f111415d060 0x7f11140c8840 secure :-1 s=READY pgs=2527 cs=0 l=1 rev1=1 crypto rx=0x7f11040097a0 tx=0x7f110402b840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=eb8222a32b5b2fd3 server_cookie=0 in_seq=0 out_seq=0 2023-08-09T15:12:11.372 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:11.370+0000 7f1118f20640 1 -- 172.21.15.43:0/242323304 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f110402e070 con 0x7f111415d060 2023-08-09T15:12:11.372 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:11.370+0000 7f1118f20640 1 -- 172.21.15.43:0/242323304 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f11040315a0 con 0x7f111415d060 2023-08-09T15:12:11.372 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:11.370+0000 7f1118f20640 1 -- 172.21.15.43:0/242323304 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1104031930 con 0x7f111415d060 2023-08-09T15:12:11.372 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:11.371+0000 7f111c186640 1 -- 172.21.15.43:0/242323304 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f111415d060 msgr2=0x7f11140c8840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T15:12:11.372 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:11.371+0000 7f111c186640 1 --2- 172.21.15.43:0/242323304 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f111415d060 0x7f11140c8840 secure :-1 s=READY pgs=2527 cs=0 l=1 rev1=1 crypto rx=0x7f11040097a0 tx=0x7f110402b840 comp rx=0 tx=0).stop 2023-08-09T15:12:11.373 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:11.371+0000 7f111c186640 1 -- 172.21.15.43:0/242323304 shutdown_connections 2023-08-09T15:12:11.373 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:11.371+0000 7f111c186640 1 --2- 172.21.15.43:0/242323304 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f111415d060 0x7f11140c8840 unknown :-1 s=CLOSED pgs=2527 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T15:12:11.373 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:11.371+0000 7f111c186640 1 -- 172.21.15.43:0/242323304 >> 172.21.15.43:0/242323304 conn(0x7f1114069d70 msgr2=0x7f11140cac70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-09T15:12:11.373 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:11.371+0000 7f111c186640 1 -- 172.21.15.43:0/242323304 shutdown_connections 2023-08-09T15:12:11.374 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:11.371+0000 7f111c186640 1 -- 172.21.15.43:0/242323304 wait complete. 2023-08-09T15:12:11.374 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:11.372+0000 7f111c186640 1 Processor -- start 2023-08-09T15:12:11.374 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:11.373+0000 7f111c186640 1 -- start start 2023-08-09T15:12:11.375 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:11.374+0000 7f111c186640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f1114156e30 0x7f11140c3290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T15:12:11.375 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:11.374+0000 7f111c186640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f1114157210 0x7f11140acd20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T15:12:11.375 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:11.374+0000 7f111c186640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f111415d060 0x7f111414b5f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T15:12:11.375 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:11.374+0000 7f111c186640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f11140ad260 con 0x7f1114157210 2023-08-09T15:12:11.375 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:11.374+0000 7f111c186640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f11140c38e0 con 0x7f1114156e30 2023-08-09T15:12:11.376 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:11.374+0000 7f111c186640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f11140c3bc0 con 0x7f111415d060 2023-08-09T15:12:11.377 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:11.374+0000 7f1119721640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f1114157210 0x7f11140acd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T15:12:11.377 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:11.374+0000 7f1119721640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f1114157210 0x7f11140acd20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:33348/0 (socket says 172.21.15.43:33348) 2023-08-09T15:12:11.377 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:11.374+0000 7f1119721640 1 -- 172.21.15.43:0/1466890988 learned_addr learned my addr 172.21.15.43:0/1466890988 (peer_addr_for_me v2:172.21.15.43:0/0) 2023-08-09T15:12:11.377 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:11.374+0000 7f111a723640 1 --2- 172.21.15.43:0/1466890988 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f111415d060 0x7f111414b5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T15:12:11.377 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:11.374+0000 7f1119f22640 1 --2- 172.21.15.43:0/1466890988 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f1114156e30 0x7f11140c3290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T15:12:11.378 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:11.374+0000 7f1119721640 1 -- 172.21.15.43:0/1466890988 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f111415d060 msgr2=0x7f111414b5f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T15:12:11.378 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:11.374+0000 7f1119721640 1 --2- 172.21.15.43:0/1466890988 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f111415d060 0x7f111414b5f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T15:12:11.378 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:11.374+0000 7f1119721640 1 -- 172.21.15.43:0/1466890988 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f1114156e30 msgr2=0x7f11140c3290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T15:12:11.378 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:11.374+0000 7f1119721640 1 --2- 172.21.15.43:0/1466890988 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f1114156e30 0x7f11140c3290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T15:12:11.378 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:11.374+0000 7f1119721640 1 -- 172.21.15.43:0/1466890988 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1104009400 con 0x7f1114157210 2023-08-09T15:12:11.378 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:11.375+0000 7f1119721640 1 --2- 172.21.15.43:0/1466890988 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f1114157210 0x7f11140acd20 secure :-1 s=READY pgs=2528 cs=0 l=1 rev1=1 crypto rx=0x7f1104034040 tx=0x7f11040046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-09T15:12:11.378 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:11.375+0000 7f110a7fc640 1 -- 172.21.15.43:0/1466890988 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1104032900 con 0x7f1114157210 2023-08-09T15:12:11.379 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:11.375+0000 7f110a7fc640 1 -- 172.21.15.43:0/1466890988 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1104032a60 con 0x7f1114157210 2023-08-09T15:12:11.379 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:11.375+0000 7f110a7fc640 1 -- 172.21.15.43:0/1466890988 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1104032e10 con 0x7f1114157210 2023-08-09T15:12:11.379 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:11.375+0000 7f111c186640 1 -- 172.21.15.43:0/1466890988 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f11140c3ea0 con 0x7f1114157210 2023-08-09T15:12:11.379 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:11.375+0000 7f111c186640 1 -- 172.21.15.43:0/1466890988 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f111414bb30 con 0x7f1114157210 2023-08-09T15:12:11.380 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:11.377+0000 7f111c186640 1 -- 172.21.15.43:0/1466890988 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f111404db40 con 0x7f1114157210 2023-08-09T15:12:11.380 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:11.378+0000 7f110a7fc640 1 -- 172.21.15.43:0/1466890988 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 128) v1 ==== 88387+0+0 (secure 0 0 0) 0x7f110403b3f0 con 0x7f1114157210 2023-08-09T15:12:11.380 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:11.378+0000 7f110a7fc640 1 --2- 172.21.15.43:0/1466890988 >> [v2:172.21.15.203:6832/1456699667,v1:172.21.15.203:6833/1456699667] conn(0x7f10f806a0e0 0x7f10f806c5a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T15:12:11.380 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:11.379+0000 7f110a7fc640 1 -- 172.21.15.43:0/1466890988 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(226..226 src has 1..226) v4 ==== 7256+0+0 (secure 0 0 0) 0x7f11040aad90 con 0x7f1114157210 2023-08-09T15:12:11.381 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:11.379+0000 7f1119f22640 1 --2- 172.21.15.43:0/1466890988 >> [v2:172.21.15.203:6832/1456699667,v1:172.21.15.203:6833/1456699667] conn(0x7f10f806a0e0 0x7f10f806c5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T15:12:11.381 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:11.380+0000 7f1119f22640 1 --2- 172.21.15.43:0/1466890988 >> [v2:172.21.15.203:6832/1456699667,v1:172.21.15.203:6833/1456699667] conn(0x7f10f806a0e0 0x7f10f806c5a0 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7f1110007fd0 tx=0x7f1110009040 comp rx=0 tx=0).ready entity=mgr.9232 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-09T15:12:11.384 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:11.383+0000 7f110a7fc640 1 -- 172.21.15.43:0/1466890988 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178066 (secure 0 0 0) 0x7f1104075dd0 con 0x7f1114157210 2023-08-09T15:12:11.419 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:11.418+0000 7f110a7fc640 1 -- 172.21.15.43:0/1466890988 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mgrmap(e 129) v1 ==== 88443+0+0 (secure 0 0 0) 0x7f110403bd30 con 0x7f1114157210 2023-08-09T15:12:11.630 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:11.629+0000 7f111c186640 1 -- 172.21.15.43:0/1466890988 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f11140cb8b0 con 0x7f1114157210 2023-08-09T15:12:11.631 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:11.630+0000 7f110a7fc640 1 -- 172.21.15.43:0/1466890988 <== mon.0 v2:172.21.15.43:3300/0 8 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 104 v104) v1 ==== 95+0+4894 (secure 0 0 0) 0x7f1104079860 con 0x7f1114157210 2023-08-09T15:12:11.632 INFO:teuthology.orchestra.run.smithi043.stdout: 2023-08-09T15:12:11.632 INFO:teuthology.orchestra.run.smithi043.stdout:{"epoch":104,"default_fscid":12,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":8012,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.203:6837/2464832767","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6836","nonce":2464832767},{"type":"v1","addr":"172.21.15.203:6837","nonce":2464832767}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":100},{"gid":9010,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.43:6835/3989386049","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6834","nonce":3989386049},{"type":"v1","addr":"172.21.15.43:6835","nonce":3989386049}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":100},{"gid":9016,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.203:6835/2213455074","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6834","nonce":2213455074},{"type":"v1","addr":"172.21.15.203:6835","nonce":2213455074}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":100}],"filesystems":[{"mdsmap":{"epoch":104,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-08-09T15:11:28.168916+0000","modified":"2023-08-09T15:11:31.739094+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":11325},"failed":[],"damaged":[],"stopped":[],"info":{"gid_11325":{"gid":11325,"name":"a","rank":0,"incarnation":101,"state":"up:active","state_seq":3,"addr":"172.21.15.43:6833/1425989175","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6832","nonce":1425989175},{"type":"v1","addr":"172.21.15.43:6833","nonce":1425989175}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[27],"metadata_pool":26,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":12}]} 2023-08-09T15:12:11.632 INFO:teuthology.orchestra.run.smithi043.stderr:dumped fsmap epoch 104 2023-08-09T15:12:11.634 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:11.633+0000 7f111c186640 1 -- 172.21.15.43:0/1466890988 >> [v2:172.21.15.203:6832/1456699667,v1:172.21.15.203:6833/1456699667] conn(0x7f10f806a0e0 msgr2=0x7f10f806c5a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T15:12:11.635 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:11.633+0000 7f111c186640 1 --2- 172.21.15.43:0/1466890988 >> [v2:172.21.15.203:6832/1456699667,v1:172.21.15.203:6833/1456699667] conn(0x7f10f806a0e0 0x7f10f806c5a0 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7f1110007fd0 tx=0x7f1110009040 comp rx=0 tx=0).stop 2023-08-09T15:12:11.635 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:11.633+0000 7f111c186640 1 -- 172.21.15.43:0/1466890988 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f1114157210 msgr2=0x7f11140acd20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T15:12:11.635 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:11.633+0000 7f111c186640 1 --2- 172.21.15.43:0/1466890988 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f1114157210 0x7f11140acd20 secure :-1 s=READY pgs=2528 cs=0 l=1 rev1=1 crypto rx=0x7f1104034040 tx=0x7f11040046c0 comp rx=0 tx=0).stop 2023-08-09T15:12:11.635 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:11.633+0000 7f111c186640 1 -- 172.21.15.43:0/1466890988 shutdown_connections 2023-08-09T15:12:11.636 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:11.633+0000 7f111c186640 1 --2- 172.21.15.43:0/1466890988 >> [v2:172.21.15.203:6832/1456699667,v1:172.21.15.203:6833/1456699667] conn(0x7f10f806a0e0 0x7f10f806c5a0 unknown :-1 s=CLOSED pgs=28 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T15:12:11.636 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:11.633+0000 7f111c186640 1 --2- 172.21.15.43:0/1466890988 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f111415d060 0x7f111414b5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T15:12:11.636 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:11.633+0000 7f111c186640 1 --2- 172.21.15.43:0/1466890988 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f1114157210 0x7f11140acd20 unknown :-1 s=CLOSED pgs=2528 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T15:12:11.636 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:11.633+0000 7f111c186640 1 --2- 172.21.15.43:0/1466890988 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f1114156e30 0x7f11140c3290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T15:12:11.636 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:11.633+0000 7f111c186640 1 -- 172.21.15.43:0/1466890988 >> 172.21.15.43:0/1466890988 conn(0x7f1114069d70 msgr2=0x7f11140c0fe0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-09T15:12:11.637 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:11.633+0000 7f111c186640 1 -- 172.21.15.43:0/1466890988 shutdown_connections 2023-08-09T15:12:11.637 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:11.634+0000 7f111c186640 1 -- 172.21.15.43:0/1466890988 wait complete. 2023-08-09T15:12:11.656 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-08-09T15:12:11.887 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:11.885+0000 7fd9b840d640 1 Processor -- start 2023-08-09T15:12:11.887 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:11.886+0000 7fd9b840d640 1 -- start start 2023-08-09T15:12:11.888 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:11.887+0000 7fd9b840d640 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7fd9b00cf510 0x7fd9b00cd270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T15:12:11.889 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:11.887+0000 7fd9b840d640 1 -- --> v1:172.21.15.43:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd9b006a7e0 con 0x7fd9b015fe60 2023-08-09T15:12:11.889 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:11.887+0000 7fd9b840d640 1 -- --> v1:172.21.15.203:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd9b006b360 con 0x7fd9b00d14c0 2023-08-09T15:12:11.889 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:11.887+0000 7fd9b840d640 1 -- --> v2:172.21.15.203:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd9b006aa30 con 0x7fd9b00cf510 2023-08-09T15:12:11.889 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:11.887+0000 7fd9b61a9640 1 --1- >> v1:172.21.15.43:6789/0 conn(0x7fd9b015fe60 0x7fd9b00cee00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.43:6789/0 says I am v1:172.21.15.43:36716/0 (socket says 172.21.15.43:36716) 2023-08-09T15:12:11.889 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:11.887+0000 7fd9b61a9640 1 -- 172.21.15.43:0/3988869319 learned_addr learned my addr 172.21.15.43:0/3988869319 (peer_addr_for_me v1:172.21.15.43:0/0) 2023-08-09T15:12:11.891 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:11.888+0000 7fd9b59a8640 1 --2- 172.21.15.43:0/3988869319 >> v2:172.21.15.203:3300/0 conn(0x7fd9b00cf510 0x7fd9b00cd270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T15:12:11.891 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:11.888+0000 7fd9b51a7640 1 -- 172.21.15.43:0/3988869319 <== mon.0 v1:172.21.15.43:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 434198348 0 0) 0x7fd9b006a7e0 con 0x7fd9b015fe60 2023-08-09T15:12:11.891 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:11.888+0000 7fd9b51a7640 1 -- 172.21.15.43:0/3988869319 --> v1:172.21.15.43:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fd99c003520 con 0x7fd9b015fe60 2023-08-09T15:12:11.892 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:11.888+0000 7fd9b51a7640 1 -- 172.21.15.43:0/3988869319 <== mon.0 v1:172.21.15.43:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3633269146 0 0) 0x7fd99c003520 con 0x7fd9b015fe60 2023-08-09T15:12:11.892 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:11.888+0000 7fd9b51a7640 1 -- 172.21.15.43:0/3988869319 >> v2:172.21.15.203:3300/0 conn(0x7fd9b00cf510 msgr2=0x7fd9b00cd270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-09T15:12:11.892 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:11.888+0000 7fd9b51a7640 1 --2- 172.21.15.43:0/3988869319 >> v2:172.21.15.203:3300/0 conn(0x7fd9b00cf510 0x7fd9b00cd270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T15:12:11.892 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:11.888+0000 7fd9b51a7640 1 -- 172.21.15.43:0/3988869319 >> v1:172.21.15.203:6789/0 conn(0x7fd9b00d14c0 legacy=0x7fd9b006b7f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T15:12:11.893 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:11.888+0000 7fd9b51a7640 1 -- 172.21.15.43:0/3988869319 --> v1:172.21.15.43:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd9b006ad10 con 0x7fd9b015fe60 2023-08-09T15:12:11.893 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:11.888+0000 7fd9b51a7640 1 -- 172.21.15.43:0/3988869319 <== mon.0 v1:172.21.15.43:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1285150740 0 0) 0x7fd9ac001670 con 0x7fd9b015fe60 2023-08-09T15:12:11.893 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:11.889+0000 7fd9b51a7640 1 -- 172.21.15.43:0/3988869319 >> v1:172.21.15.43:6789/0 conn(0x7fd9b015fe60 legacy=0x7fd9b00cee00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T15:12:11.893 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:11.889+0000 7fd9b51a7640 1 --2- 172.21.15.43:0/3988869319 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fd99c0039a0 0x7fd99c003dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T15:12:11.894 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:11.889+0000 7fd9b51a7640 1 --2- 172.21.15.43:0/3988869319 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fd99c0047c0 0x7fd99c004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T15:12:11.894 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:11.889+0000 7fd9b51a7640 1 --2- 172.21.15.43:0/3988869319 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fd99c005180 0x7fd99c00b980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T15:12:11.894 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:11.889+0000 7fd9b51a7640 1 -- 172.21.15.43:0/3988869319 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd9b006aa30 con 0x7fd99c005180 2023-08-09T15:12:11.895 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:11.889+0000 7fd9b51a7640 1 -- 172.21.15.43:0/3988869319 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd99c00c050 con 0x7fd99c0039a0 2023-08-09T15:12:11.895 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:11.889+0000 7fd9b51a7640 1 -- 172.21.15.43:0/3988869319 --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd99c00c210 con 0x7fd99c0047c0 2023-08-09T15:12:11.895 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:11.889+0000 7fd9b69aa640 1 --2- 172.21.15.43:0/3988869319 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fd99c005180 0x7fd99c00b980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T15:12:11.895 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:11.889+0000 7fd9b61a9640 1 --2- 172.21.15.43:0/3988869319 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fd99c0039a0 0x7fd99c003dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T15:12:11.895 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:11.889+0000 7fd9b59a8640 1 --2- 172.21.15.43:0/3988869319 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fd99c0047c0 0x7fd99c004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T15:12:11.895 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:11.889+0000 7fd9b69aa640 1 -- 172.21.15.43:0/3988869319 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fd99c0047c0 msgr2=0x7fd99c004c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-09T15:12:11.895 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:11.889+0000 7fd9b69aa640 1 --2- 172.21.15.43:0/3988869319 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fd99c0047c0 0x7fd99c004c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T15:12:11.896 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:11.889+0000 7fd9b69aa640 1 -- 172.21.15.43:0/3988869319 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fd99c0039a0 msgr2=0x7fd99c003dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-09T15:12:11.896 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:11.889+0000 7fd9b69aa640 1 --2- 172.21.15.43:0/3988869319 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fd99c0039a0 0x7fd99c003dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T15:12:11.896 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:11.889+0000 7fd9b69aa640 1 -- 172.21.15.43:0/3988869319 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd99c003520 con 0x7fd99c005180 2023-08-09T15:12:11.898 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:11.890+0000 7fd9b69aa640 1 -- 172.21.15.43:0/3988869319 reap_dead start 2023-08-09T15:12:11.898 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:11.891+0000 7fd9b69aa640 1 --2- 172.21.15.43:0/3988869319 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fd99c005180 0x7fd99c00b980 secure :-1 s=READY pgs=2530 cs=0 l=1 rev1=1 crypto rx=0x7fd9b00d18d0 tx=0x7fd99c0057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=772e24c7902a7aa9 server_cookie=0 in_seq=0 out_seq=0 2023-08-09T15:12:11.898 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:11.891+0000 7fd9b51a7640 1 -- 172.21.15.43:0/3988869319 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd9a8003b70 con 0x7fd99c005180 2023-08-09T15:12:11.898 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:11.891+0000 7fd9b51a7640 1 -- 172.21.15.43:0/3988869319 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd9a8003cd0 con 0x7fd99c005180 2023-08-09T15:12:11.898 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:11.891+0000 7fd9b51a7640 1 -- 172.21.15.43:0/3988869319 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd9a8005310 con 0x7fd99c005180 2023-08-09T15:12:11.899 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:11.891+0000 7fd9b840d640 1 -- 172.21.15.43:0/3988869319 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fd99c005180 msgr2=0x7fd99c00b980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T15:12:11.899 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:11.891+0000 7fd9b840d640 1 --2- 172.21.15.43:0/3988869319 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fd99c005180 0x7fd99c00b980 secure :-1 s=READY pgs=2530 cs=0 l=1 rev1=1 crypto rx=0x7fd9b00d18d0 tx=0x7fd99c0057f0 comp rx=0 tx=0).stop 2023-08-09T15:12:11.899 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:11.891+0000 7fd9b840d640 1 -- 172.21.15.43:0/3988869319 shutdown_connections 2023-08-09T15:12:11.899 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:11.891+0000 7fd9b840d640 1 --2- 172.21.15.43:0/3988869319 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fd99c005180 0x7fd99c00b980 unknown :-1 s=CLOSED pgs=2530 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T15:12:11.899 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:11.891+0000 7fd9b840d640 1 -- 172.21.15.43:0/3988869319 >> 172.21.15.43:0/3988869319 conn(0x7fd9b00d2660 msgr2=0x7fd9b00d2a70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-09T15:12:11.899 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:11.892+0000 7fd9b840d640 1 -- 172.21.15.43:0/3988869319 shutdown_connections 2023-08-09T15:12:11.899 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:11.892+0000 7fd9b840d640 1 -- 172.21.15.43:0/3988869319 wait complete. 2023-08-09T15:12:11.900 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:11.893+0000 7fd9b840d640 1 Processor -- start 2023-08-09T15:12:11.900 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:11.893+0000 7fd9b840d640 1 -- start start 2023-08-09T15:12:11.900 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:11.894+0000 7fd9b840d640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fd99c005180 0x7fd9b015a7c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T15:12:11.900 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:11.894+0000 7fd9b840d640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fd9b00cc2c0 0x7fd9b00cb0f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T15:12:11.900 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:11.894+0000 7fd9b840d640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fd9b00cb630 0x7fd9b00cbab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T15:12:11.900 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:11.894+0000 7fd9b840d640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd9b00caab0 con 0x7fd9b00cc2c0 2023-08-09T15:12:11.900 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:11.894+0000 7fd9b840d640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd9b00b72c0 con 0x7fd99c005180 2023-08-09T15:12:11.900 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:11.894+0000 7fd9b840d640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd9b00b75a0 con 0x7fd9b00cb630 2023-08-09T15:12:11.901 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:11.895+0000 7fd9b59a8640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fd9b00cc2c0 0x7fd9b00cb0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T15:12:11.901 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:11.895+0000 7fd9b59a8640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fd9b00cc2c0 0x7fd9b00cb0f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:33376/0 (socket says 172.21.15.43:33376) 2023-08-09T15:12:11.901 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:11.895+0000 7fd9b59a8640 1 -- 172.21.15.43:0/3527881717 learned_addr learned my addr 172.21.15.43:0/3527881717 (peer_addr_for_me v2:172.21.15.43:0/0) 2023-08-09T15:12:11.901 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:11.895+0000 7fd9b69aa640 1 --2- 172.21.15.43:0/3527881717 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fd9b00cb630 0x7fd9b00cbab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T15:12:11.901 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:11.895+0000 7fd9b61a9640 1 --2- 172.21.15.43:0/3527881717 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fd99c005180 0x7fd9b015a7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T15:12:11.901 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:11.895+0000 7fd9b59a8640 1 -- 172.21.15.43:0/3527881717 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fd9b00cb630 msgr2=0x7fd9b00cbab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T15:12:11.901 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:11.895+0000 7fd9b59a8640 1 --2- 172.21.15.43:0/3527881717 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fd9b00cb630 0x7fd9b00cbab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T15:12:11.902 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:11.895+0000 7fd9b59a8640 1 -- 172.21.15.43:0/3527881717 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fd99c005180 msgr2=0x7fd9b015a7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T15:12:11.902 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:11.895+0000 7fd9b59a8640 1 --2- 172.21.15.43:0/3527881717 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fd99c005180 0x7fd9b015a7c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T15:12:11.902 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:11.895+0000 7fd9b59a8640 1 -- 172.21.15.43:0/3527881717 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd9b00b7880 con 0x7fd9b00cc2c0 2023-08-09T15:12:11.902 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:11.896+0000 7fd9b59a8640 1 --2- 172.21.15.43:0/3527881717 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fd9b00cc2c0 0x7fd9b00cb0f0 secure :-1 s=READY pgs=2531 cs=0 l=1 rev1=1 crypto rx=0x7fd9a0011540 tx=0x7fd9a0011a00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-09T15:12:11.902 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:11.896+0000 7fd9a6ffd640 1 -- 172.21.15.43:0/3527881717 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd9a0015930 con 0x7fd9b00cc2c0 2023-08-09T15:12:11.902 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:11.896+0000 7fd9a6ffd640 1 -- 172.21.15.43:0/3527881717 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd9a0015a90 con 0x7fd9b00cc2c0 2023-08-09T15:12:11.902 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:11.896+0000 7fd9a6ffd640 1 -- 172.21.15.43:0/3527881717 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd9a0015e60 con 0x7fd9b00cc2c0 2023-08-09T15:12:11.902 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:11.896+0000 7fd9b840d640 1 -- 172.21.15.43:0/3527881717 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd9b00b7b60 con 0x7fd9b00cc2c0 2023-08-09T15:12:11.903 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:11.896+0000 7fd9b840d640 1 -- 172.21.15.43:0/3527881717 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd9b00c9910 con 0x7fd9b00cc2c0 2023-08-09T15:12:11.903 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:11.897+0000 7fd9b840d640 1 -- 172.21.15.43:0/3527881717 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd9b00b7400 con 0x7fd9b00cc2c0 2023-08-09T15:12:11.904 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:11.898+0000 7fd9a6ffd640 1 -- 172.21.15.43:0/3527881717 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 129) v1 ==== 88443+0+0 (secure 0 0 0) 0x7fd9a000f460 con 0x7fd9b00cc2c0 2023-08-09T15:12:11.904 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:11.899+0000 7fd9a6ffd640 1 --2- 172.21.15.43:0/3527881717 >> [v2:172.21.15.203:6832/1456699667,v1:172.21.15.203:6833/1456699667] conn(0x7fd99406a1a0 0x7fd99406c660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T15:12:11.904 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:11.899+0000 7fd9a6ffd640 1 -- 172.21.15.43:0/3527881717 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(226..226 src has 1..226) v4 ==== 7256+0+0 (secure 0 0 0) 0x7fd9a0089360 con 0x7fd9b00cc2c0 2023-08-09T15:12:11.904 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:11.900+0000 7fd9b61a9640 1 --2- 172.21.15.43:0/3527881717 >> [v2:172.21.15.203:6832/1456699667,v1:172.21.15.203:6833/1456699667] conn(0x7fd99406a1a0 0x7fd99406c660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T15:12:11.904 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:11.901+0000 7fd9b61a9640 1 --2- 172.21.15.43:0/3527881717 >> [v2:172.21.15.203:6832/1456699667,v1:172.21.15.203:6833/1456699667] conn(0x7fd99406a1a0 0x7fd99406c660 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7fd9ac001050 tx=0x7fd9ac009040 comp rx=0 tx=0).ready entity=mgr.9232 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-09T15:12:11.905 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:11.904+0000 7fd9a6ffd640 1 -- 172.21.15.43:0/3527881717 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178066 (secure 0 0 0) 0x7fd9b00b7400 con 0x7fd9b00cc2c0 2023-08-09T15:12:12.100 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:12.099+0000 7fd9b840d640 1 -- 172.21.15.43:0/3527881717 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fd9b00b7400 con 0x7fd9b00cc2c0 2023-08-09T15:12:12.101 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:12.100+0000 7fd9a6ffd640 1 -- 172.21.15.43:0/3527881717 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v226) v1 ==== 74+0+19824 (secure 0 0 0) 0x7fd9b00b7400 con 0x7fd9b00cc2c0 2023-08-09T15:12:12.102 INFO:teuthology.orchestra.run.smithi043.stdout: 2023-08-09T15:12:12.102 INFO:teuthology.orchestra.run.smithi043.stdout:{"epoch":226,"fsid":"daac87ee-1337-4250-a955-aea4e408a39b","created":"2023-08-09T13:33:41.701198+0000","modified":"2023-08-09T15:12:08.329952+0000","last_up_change":"2023-08-09T13:33:53.783453+0000","last_in_change":"2023-08-09T13:33:45.148904+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":27,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":"rbd","create_time":"2023-08-09T13:33:54.365137+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":".mgr","create_time":"2023-08-09T13:33:54.831803+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":26,"pool_name":"cephfs_metadata","create_time":"2023-08-09T15:11:26.544757+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"222","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":27,"pool_name":"cephfs_data","create_time":"2023-08-09T15:11:27.194686+0000","flags":8193,"flags_names":"hashpspool,selfmanaged_snaps","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"226","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":3,"snap_epoch":226,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.6299999952316284,"score_stable":1.6299999952316284,"optimal_score":1,"raw_score_acting":1.6299999952316284,"raw_score_stable":1.6299999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"cbd49ecd-e896-4fbd-864e-5c37a829b165","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":221,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6800","nonce":502667335},{"type":"v1","addr":"172.21.15.43:6801","nonce":502667335}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6802","nonce":502667335},{"type":"v1","addr":"172.21.15.43:6803","nonce":502667335}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6806","nonce":502667335},{"type":"v1","addr":"172.21.15.43:6807","nonce":502667335}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6804","nonce":502667335},{"type":"v1","addr":"172.21.15.43:6805","nonce":502667335}]},"public_addr":"172.21.15.43:6801/502667335","cluster_addr":"172.21.15.43:6803/502667335","heartbeat_back_addr":"172.21.15.43:6807/502667335","heartbeat_front_addr":"172.21.15.43:6805/502667335","state":["exists","up"]},{"osd":1,"uuid":"75c7e0fe-1ec7-4fe8-a2f2-3661a65b8500","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":221,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6808","nonce":3289134610},{"type":"v1","addr":"172.21.15.43:6809","nonce":3289134610}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6810","nonce":3289134610},{"type":"v1","addr":"172.21.15.43:6812","nonce":3289134610}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6821","nonce":3289134610},{"type":"v1","addr":"172.21.15.43:6824","nonce":3289134610}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6815","nonce":3289134610},{"type":"v1","addr":"172.21.15.43:6818","nonce":3289134610}]},"public_addr":"172.21.15.43:6809/3289134610","cluster_addr":"172.21.15.43:6812/3289134610","heartbeat_back_addr":"172.21.15.43:6824/3289134610","heartbeat_front_addr":"172.21.15.43:6818/3289134610","state":["exists","up"]},{"osd":2,"uuid":"bb5c41e9-7f5f-4125-9b01-3bb8bfc2ea83","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":221,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6811","nonce":4079709736},{"type":"v1","addr":"172.21.15.43:6814","nonce":4079709736}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6817","nonce":4079709736},{"type":"v1","addr":"172.21.15.43:6820","nonce":4079709736}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6829","nonce":4079709736},{"type":"v1","addr":"172.21.15.43:6831","nonce":4079709736}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6823","nonce":4079709736},{"type":"v1","addr":"172.21.15.43:6826","nonce":4079709736}]},"public_addr":"172.21.15.43:6814/4079709736","cluster_addr":"172.21.15.43:6820/4079709736","heartbeat_back_addr":"172.21.15.43:6831/4079709736","heartbeat_front_addr":"172.21.15.43:6826/4079709736","state":["exists","up"]},{"osd":3,"uuid":"bfe4c8b4-694a-465c-87e6-2b2aab852613","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":221,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6813","nonce":1709538759},{"type":"v1","addr":"172.21.15.43:6816","nonce":1709538759}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6819","nonce":1709538759},{"type":"v1","addr":"172.21.15.43:6822","nonce":1709538759}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6828","nonce":1709538759},{"type":"v1","addr":"172.21.15.43:6830","nonce":1709538759}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6825","nonce":1709538759},{"type":"v1","addr":"172.21.15.43:6827","nonce":1709538759}]},"public_addr":"172.21.15.43:6816/1709538759","cluster_addr":"172.21.15.43:6822/1709538759","heartbeat_back_addr":"172.21.15.43:6830/1709538759","heartbeat_front_addr":"172.21.15.43:6827/1709538759","state":["exists","up"]},{"osd":4,"uuid":"fa2e8a77-aba1-4756-853e-0d174a5bdeba","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":221,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6800","nonce":4196866481},{"type":"v1","addr":"172.21.15.203:6801","nonce":4196866481}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6802","nonce":4196866481},{"type":"v1","addr":"172.21.15.203:6803","nonce":4196866481}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6806","nonce":4196866481},{"type":"v1","addr":"172.21.15.203:6807","nonce":4196866481}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6804","nonce":4196866481},{"type":"v1","addr":"172.21.15.203:6805","nonce":4196866481}]},"public_addr":"172.21.15.203:6801/4196866481","cluster_addr":"172.21.15.203:6803/4196866481","heartbeat_back_addr":"172.21.15.203:6807/4196866481","heartbeat_front_addr":"172.21.15.203:6805/4196866481","state":["exists","up"]},{"osd":5,"uuid":"81811e4b-4d24-4f51-a82c-f1d700a2a00d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":221,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6816","nonce":3059290081},{"type":"v1","addr":"172.21.15.203:6817","nonce":3059290081}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6818","nonce":3059290081},{"type":"v1","addr":"172.21.15.203:6819","nonce":3059290081}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6822","nonce":3059290081},{"type":"v1","addr":"172.21.15.203:6823","nonce":3059290081}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6820","nonce":3059290081},{"type":"v1","addr":"172.21.15.203:6821","nonce":3059290081}]},"public_addr":"172.21.15.203:6817/3059290081","cluster_addr":"172.21.15.203:6819/3059290081","heartbeat_back_addr":"172.21.15.203:6823/3059290081","heartbeat_front_addr":"172.21.15.203:6821/3059290081","state":["exists","up"]},{"osd":6,"uuid":"307bac4d-6a5b-443d-af75-56d695d7674f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":221,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6824","nonce":512155569},{"type":"v1","addr":"172.21.15.203:6825","nonce":512155569}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6826","nonce":512155569},{"type":"v1","addr":"172.21.15.203:6827","nonce":512155569}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6830","nonce":512155569},{"type":"v1","addr":"172.21.15.203:6831","nonce":512155569}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6828","nonce":512155569},{"type":"v1","addr":"172.21.15.203:6829","nonce":512155569}]},"public_addr":"172.21.15.203:6825/512155569","cluster_addr":"172.21.15.203:6827/512155569","heartbeat_back_addr":"172.21.15.203:6831/512155569","heartbeat_front_addr":"172.21.15.203:6829/512155569","state":["exists","up"]},{"osd":7,"uuid":"a95dc7ca-9f1b-4295-af70-68a22a2d7d5e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":221,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6808","nonce":3331450438},{"type":"v1","addr":"172.21.15.203:6809","nonce":3331450438}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6810","nonce":3331450438},{"type":"v1","addr":"172.21.15.203:6811","nonce":3331450438}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6814","nonce":3331450438},{"type":"v1","addr":"172.21.15.203:6815","nonce":3331450438}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6812","nonce":3331450438},{"type":"v1","addr":"172.21.15.203:6813","nonce":3331450438}]},"public_addr":"172.21.15.203:6809/3331450438","cluster_addr":"172.21.15.203:6811/3331450438","heartbeat_back_addr":"172.21.15.203:6815/3331450438","heartbeat_front_addr":"172.21.15.203:6813/3331450438","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-09T13:33:50.547652+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-09T13:33:50.589469+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-09T13:33:50.561724+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-09T13:33:50.599650+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-09T13:33:50.411855+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-09T13:33:50.219681+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-09T13:33:50.504664+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-09T13:33:50.441248+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.203:0/4004150203":"2023-08-10T15:12:07.324983+0000","172.21.15.203:6833/2375771024":"2023-08-10T15:12:07.324983+0000","172.21.15.203:6832/2375771024":"2023-08-10T15:12:07.324983+0000","172.21.15.203:0/1645319310":"2023-08-10T15:12:07.324983+0000","172.21.15.203:0/3324686399":"2023-08-10T15:12:07.324983+0000","172.21.15.203:0/286395424":"2023-08-10T15:12:07.324983+0000","172.21.15.203:0/1761748226":"2023-08-10T15:12:07.324983+0000","172.21.15.203:6832/4034093268":"2023-08-10T15:11:43.130145+0000","172.21.15.203:6833/4034093268":"2023-08-10T15:11:43.130145+0000","172.21.15.203:0/1654193324":"2023-08-10T15:11:43.130145+0000","172.21.15.203:0/3701038305":"2023-08-10T15:12:07.324983+0000","172.21.15.203:0/4140323243":"2023-08-10T15:11:43.130145+0000","172.21.15.203:0/450863258":"2023-08-10T15:11:43.130145+0000","172.21.15.203:0/134877587":"2023-08-10T15:11:43.130145+0000","172.21.15.203:0/3500453783":"2023-08-10T15:12:07.324983+0000","172.21.15.203:0/2330869600":"2023-08-10T15:11:43.130145+0000","172.21.15.203:0/1886393026":"2023-08-10T15:12:07.324983+0000","172.21.15.203:0/2203818837":"2023-08-10T15:11:43.130145+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":7,"snaps":[{"begin":104,"length":1},{"begin":221,"length":1}]},{"pool":27,"snaps":[{"begin":2,"length":2}]}],"new_removed_snaps":[{"pool":27,"snaps":[{"begin":2,"length":2}]}],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-08-09T15:12:12.105 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:12.103+0000 7fd9b840d640 1 -- 172.21.15.43:0/3527881717 >> [v2:172.21.15.203:6832/1456699667,v1:172.21.15.203:6833/1456699667] conn(0x7fd99406a1a0 msgr2=0x7fd99406c660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T15:12:12.105 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:12.103+0000 7fd9b840d640 1 --2- 172.21.15.43:0/3527881717 >> [v2:172.21.15.203:6832/1456699667,v1:172.21.15.203:6833/1456699667] conn(0x7fd99406a1a0 0x7fd99406c660 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7fd9ac001050 tx=0x7fd9ac009040 comp rx=0 tx=0).stop 2023-08-09T15:12:12.105 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:12.103+0000 7fd9b840d640 1 -- 172.21.15.43:0/3527881717 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fd9b00cc2c0 msgr2=0x7fd9b00cb0f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T15:12:12.105 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:12.103+0000 7fd9b840d640 1 --2- 172.21.15.43:0/3527881717 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fd9b00cc2c0 0x7fd9b00cb0f0 secure :-1 s=READY pgs=2531 cs=0 l=1 rev1=1 crypto rx=0x7fd9a0011540 tx=0x7fd9a0011a00 comp rx=0 tx=0).stop 2023-08-09T15:12:12.105 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:12.103+0000 7fd9b840d640 1 -- 172.21.15.43:0/3527881717 shutdown_connections 2023-08-09T15:12:12.106 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:12.103+0000 7fd9b840d640 1 --2- 172.21.15.43:0/3527881717 >> [v2:172.21.15.203:6832/1456699667,v1:172.21.15.203:6833/1456699667] conn(0x7fd99406a1a0 0x7fd99406c660 unknown :-1 s=CLOSED pgs=29 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T15:12:12.106 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:12.103+0000 7fd9b840d640 1 --2- 172.21.15.43:0/3527881717 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fd9b00cb630 0x7fd9b00cbab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T15:12:12.106 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:12.103+0000 7fd9b840d640 1 --2- 172.21.15.43:0/3527881717 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fd9b00cc2c0 0x7fd9b00cb0f0 unknown :-1 s=CLOSED pgs=2531 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T15:12:12.106 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:12.103+0000 7fd9b840d640 1 --2- 172.21.15.43:0/3527881717 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fd99c005180 0x7fd9b015a7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T15:12:12.106 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:12.103+0000 7fd9b840d640 1 -- 172.21.15.43:0/3527881717 >> 172.21.15.43:0/3527881717 conn(0x7fd9b00d2660 msgr2=0x7fd9b015ca20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-09T15:12:12.106 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:12.104+0000 7fd9b840d640 1 -- 172.21.15.43:0/3527881717 shutdown_connections 2023-08-09T15:12:12.106 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:12.104+0000 7fd9b840d640 1 -- 172.21.15.43:0/3527881717 wait complete. 2023-08-09T15:12:12.124 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2023-08-09T15:12:12.362 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:12.361+0000 7fbe267b5640 1 Processor -- start 2023-08-09T15:12:12.363 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:12.361+0000 7fbe267b5640 1 -- start start 2023-08-09T15:12:12.364 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:12.363+0000 7fbe267b5640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fbe2015a530 0x7fbe2015eb20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T15:12:12.364 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:12.363+0000 7fbe267b5640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7fbe2006f670 0x7fbe200c9470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T15:12:12.364 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:12.363+0000 7fbe267b5640 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7fbe200c99b0 0x7fbe200afc80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T15:12:12.365 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:12.363+0000 7fbe267b5640 1 -- --> v2:172.21.15.43:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbe200701d0 con 0x7fbe2006f670 2023-08-09T15:12:12.365 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:12.363+0000 7fbe267b5640 1 -- --> v2:172.21.15.203:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbe2015f060 con 0x7fbe200c99b0 2023-08-09T15:12:12.365 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:12.363+0000 7fbe267b5640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbe2006fc10 con 0x7fbe2015a530 2023-08-09T15:12:12.366 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:12.363+0000 7fbe177fe640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7fbe2006f670 0x7fbe200c9470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T15:12:12.366 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:12.363+0000 7fbe177fe640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7fbe2006f670 0x7fbe200c9470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:33384/0 (socket says 172.21.15.43:33384) 2023-08-09T15:12:12.366 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:12.363+0000 7fbe177fe640 1 -- 172.21.15.43:0/1571776727 learned_addr learned my addr 172.21.15.43:0/1571776727 (peer_addr_for_me v2:172.21.15.43:0/0) 2023-08-09T15:12:12.366 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:12.364+0000 7fbe1ffff640 1 --2- 172.21.15.43:0/1571776727 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fbe2015a530 0x7fbe2015eb20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T15:12:12.366 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:12.364+0000 7fbe24d52640 1 --2- 172.21.15.43:0/1571776727 >> v2:172.21.15.203:3300/0 conn(0x7fbe200c99b0 0x7fbe200afc80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T15:12:12.366 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:12.364+0000 7fbe177fe640 1 -- 172.21.15.43:0/1571776727 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fbe2015a530 msgr2=0x7fbe2015eb20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-09T15:12:12.366 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:12.364+0000 7fbe177fe640 1 --2- 172.21.15.43:0/1571776727 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fbe2015a530 0x7fbe2015eb20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T15:12:12.367 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:12.364+0000 7fbe177fe640 1 -- 172.21.15.43:0/1571776727 >> v2:172.21.15.203:3300/0 conn(0x7fbe200c99b0 msgr2=0x7fbe200afc80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-09T15:12:12.367 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:12.364+0000 7fbe177fe640 1 --2- 172.21.15.43:0/1571776727 >> v2:172.21.15.203:3300/0 conn(0x7fbe200c99b0 0x7fbe200afc80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T15:12:12.367 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:12.364+0000 7fbe177fe640 1 -- 172.21.15.43:0/1571776727 --> v2:172.21.15.43:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbe2006fef0 con 0x7fbe2006f670 2023-08-09T15:12:12.368 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:12.365+0000 7fbe177fe640 1 --2- 172.21.15.43:0/1571776727 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fbe2006f670 0x7fbe200c9470 secure :-1 s=READY pgs=2532 cs=0 l=1 rev1=1 crypto rx=0x7fbe200c9d90 tx=0x7fbe0802b910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=74ce5dbadd9f891d server_cookie=0 in_seq=0 out_seq=0 2023-08-09T15:12:12.368 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:12.365+0000 7fbe1f7fe640 1 -- 172.21.15.43:0/1571776727 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbe08031870 con 0x7fbe2006f670 2023-08-09T15:12:12.368 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:12.365+0000 7fbe1f7fe640 1 -- 172.21.15.43:0/1571776727 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbe080319d0 con 0x7fbe2006f670 2023-08-09T15:12:12.368 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:12.365+0000 7fbe1f7fe640 1 -- 172.21.15.43:0/1571776727 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbe08031d60 con 0x7fbe2006f670 2023-08-09T15:12:12.369 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:12.366+0000 7fbe267b5640 1 -- 172.21.15.43:0/1571776727 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fbe2006f670 msgr2=0x7fbe200c9470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T15:12:12.369 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:12.366+0000 7fbe267b5640 1 --2- 172.21.15.43:0/1571776727 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fbe2006f670 0x7fbe200c9470 secure :-1 s=READY pgs=2532 cs=0 l=1 rev1=1 crypto rx=0x7fbe200c9d90 tx=0x7fbe0802b910 comp rx=0 tx=0).stop 2023-08-09T15:12:12.369 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:12.366+0000 7fbe267b5640 1 -- 172.21.15.43:0/1571776727 shutdown_connections 2023-08-09T15:12:12.369 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:12.366+0000 7fbe267b5640 1 --2- 172.21.15.43:0/1571776727 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fbe2006f670 0x7fbe200c9470 unknown :-1 s=CLOSED pgs=2532 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T15:12:12.369 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:12.366+0000 7fbe267b5640 1 --2- 172.21.15.43:0/1571776727 >> v2:172.21.15.203:3300/0 conn(0x7fbe200c99b0 0x7fbe200afc80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T15:12:12.369 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:12.366+0000 7fbe267b5640 1 --2- 172.21.15.43:0/1571776727 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fbe2015a530 0x7fbe2015eb20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T15:12:12.369 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:12.366+0000 7fbe267b5640 1 -- 172.21.15.43:0/1571776727 >> 172.21.15.43:0/1571776727 conn(0x7fbe20069dd0 msgr2=0x7fbe200cac80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-09T15:12:12.370 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:12.367+0000 7fbe267b5640 1 -- 172.21.15.43:0/1571776727 shutdown_connections 2023-08-09T15:12:12.370 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:12.367+0000 7fbe267b5640 1 -- 172.21.15.43:0/1571776727 wait complete. 2023-08-09T15:12:12.370 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:12.368+0000 7fbe267b5640 1 Processor -- start 2023-08-09T15:12:12.370 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:12.369+0000 7fbe267b5640 1 -- start start 2023-08-09T15:12:12.371 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:12.369+0000 7fbe267b5640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fbe2006f670 0x7fbe20149870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T15:12:12.371 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:12.369+0000 7fbe267b5640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fbe200c99b0 0x7fbe2015ab70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T15:12:12.372 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:12.370+0000 7fbe267b5640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fbe2015a530 0x7fbe200ccc10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T15:12:12.372 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:12.370+0000 7fbe267b5640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbe200c65b0 con 0x7fbe200c99b0 2023-08-09T15:12:12.372 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:12.370+0000 7fbe267b5640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbe2015bed0 con 0x7fbe2015a530 2023-08-09T15:12:12.372 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:12.370+0000 7fbe267b5640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbe2015b630 con 0x7fbe2006f670 2023-08-09T15:12:12.372 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:12.370+0000 7fbe177fe640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fbe200c99b0 0x7fbe2015ab70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T15:12:12.372 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:12.370+0000 7fbe177fe640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fbe200c99b0 0x7fbe2015ab70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:33392/0 (socket says 172.21.15.43:33392) 2023-08-09T15:12:12.372 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:12.370+0000 7fbe177fe640 1 -- 172.21.15.43:0/3532643403 learned_addr learned my addr 172.21.15.43:0/3532643403 (peer_addr_for_me v2:172.21.15.43:0/0) 2023-08-09T15:12:12.372 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:12.370+0000 7fbe24d52640 1 --2- 172.21.15.43:0/3532643403 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fbe2015a530 0x7fbe200ccc10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T15:12:12.373 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:12.370+0000 7fbe177fe640 1 -- 172.21.15.43:0/3532643403 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fbe2006f670 msgr2=0x7fbe20149870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T15:12:12.374 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:12.370+0000 7fbe177fe640 1 --2- 172.21.15.43:0/3532643403 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fbe2006f670 0x7fbe20149870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T15:12:12.374 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:12.370+0000 7fbe177fe640 1 -- 172.21.15.43:0/3532643403 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fbe2015a530 msgr2=0x7fbe200ccc10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T15:12:12.374 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:12.370+0000 7fbe177fe640 1 --2- 172.21.15.43:0/3532643403 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fbe2015a530 0x7fbe200ccc10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T15:12:12.374 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:12.370+0000 7fbe177fe640 1 -- 172.21.15.43:0/3532643403 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbe080094d0 con 0x7fbe200c99b0 2023-08-09T15:12:12.374 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:12.371+0000 7fbe177fe640 1 --2- 172.21.15.43:0/3532643403 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fbe200c99b0 0x7fbe2015ab70 secure :-1 s=READY pgs=2533 cs=0 l=1 rev1=1 crypto rx=0x7fbe08000c00 tx=0x7fbe080046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-09T15:12:12.374 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:12.371+0000 7fbe1d7fa640 1 -- 172.21.15.43:0/3532643403 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbe0803abe0 con 0x7fbe200c99b0 2023-08-09T15:12:12.374 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:12.371+0000 7fbe1d7fa640 1 -- 172.21.15.43:0/3532643403 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbe0803ad40 con 0x7fbe200c99b0 2023-08-09T15:12:12.375 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:12.371+0000 7fbe267b5640 1 -- 172.21.15.43:0/3532643403 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbe2015b910 con 0x7fbe200c99b0 2023-08-09T15:12:12.375 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:12.371+0000 7fbe1d7fa640 1 -- 172.21.15.43:0/3532643403 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbe08044610 con 0x7fbe200c99b0 2023-08-09T15:12:12.375 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:12.371+0000 7fbe267b5640 1 -- 172.21.15.43:0/3532643403 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbe20154f90 con 0x7fbe200c99b0 2023-08-09T15:12:12.375 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:12.372+0000 7fbe267b5640 1 -- 172.21.15.43:0/3532643403 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbe200c6310 con 0x7fbe200c99b0 2023-08-09T15:12:12.375 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:12.373+0000 7fbe1d7fa640 1 -- 172.21.15.43:0/3532643403 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 129) v1 ==== 88443+0+0 (secure 0 0 0) 0x7fbe08044770 con 0x7fbe200c99b0 2023-08-09T15:12:12.376 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:12.374+0000 7fbe1d7fa640 1 --2- 172.21.15.43:0/3532643403 >> [v2:172.21.15.203:6832/1456699667,v1:172.21.15.203:6833/1456699667] conn(0x7fbe0006a170 0x7fbe0006c630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T15:12:12.376 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:12.374+0000 7fbe1d7fa640 1 -- 172.21.15.43:0/3532643403 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(226..226 src has 1..226) v4 ==== 7256+0+0 (secure 0 0 0) 0x7fbe0802d040 con 0x7fbe200c99b0 2023-08-09T15:12:12.376 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:12.375+0000 7fbe1ffff640 1 --2- 172.21.15.43:0/3532643403 >> [v2:172.21.15.203:6832/1456699667,v1:172.21.15.203:6833/1456699667] conn(0x7fbe0006a170 0x7fbe0006c630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T15:12:12.377 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:12.376+0000 7fbe1ffff640 1 --2- 172.21.15.43:0/3532643403 >> [v2:172.21.15.203:6832/1456699667,v1:172.21.15.203:6833/1456699667] conn(0x7fbe0006a170 0x7fbe0006c630 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0x7fbe100070b0 tx=0x7fbe10007040 comp rx=0 tx=0).ready entity=mgr.9232 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-09T15:12:12.380 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:12.379+0000 7fbe1d7fa640 1 -- 172.21.15.43:0/3532643403 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178066 (secure 0 0 0) 0x7fbe200c6310 con 0x7fbe200c99b0 2023-08-09T15:12:12.612 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:12.611+0000 7fbe267b5640 1 -- 172.21.15.43:0/3532643403 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) v1 -- 0x7fbe200c66f0 con 0x7fbe200c99b0 2023-08-09T15:12:12.613 INFO:tasks.ceph.mon.a.smithi043.stderr:2023-08-09T15:12:12.612+0000 7f32867f3640 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2023-08-09T15:12:12.620 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:12.618+0000 7fbe1d7fa640 1 -- 172.21.15.43:0/3532643403 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v105) v1 ==== 161+0+0 (secure 0 0 0) 0x7fbe200c66f0 con 0x7fbe200c99b0 2023-08-09T15:12:12.620 INFO:teuthology.orchestra.run.smithi043.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2023-08-09T15:12:12.623 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:12.621+0000 7fbe267b5640 1 -- 172.21.15.43:0/3532643403 >> [v2:172.21.15.203:6832/1456699667,v1:172.21.15.203:6833/1456699667] conn(0x7fbe0006a170 msgr2=0x7fbe0006c630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T15:12:12.623 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:12.621+0000 7fbe267b5640 1 --2- 172.21.15.43:0/3532643403 >> [v2:172.21.15.203:6832/1456699667,v1:172.21.15.203:6833/1456699667] conn(0x7fbe0006a170 0x7fbe0006c630 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0x7fbe100070b0 tx=0x7fbe10007040 comp rx=0 tx=0).stop 2023-08-09T15:12:12.623 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:12.621+0000 7fbe267b5640 1 -- 172.21.15.43:0/3532643403 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fbe200c99b0 msgr2=0x7fbe2015ab70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T15:12:12.623 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:12.621+0000 7fbe267b5640 1 --2- 172.21.15.43:0/3532643403 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fbe200c99b0 0x7fbe2015ab70 secure :-1 s=READY pgs=2533 cs=0 l=1 rev1=1 crypto rx=0x7fbe08000c00 tx=0x7fbe080046c0 comp rx=0 tx=0).stop 2023-08-09T15:12:12.624 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:12.621+0000 7fbe267b5640 1 -- 172.21.15.43:0/3532643403 shutdown_connections 2023-08-09T15:12:12.624 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:12.621+0000 7fbe267b5640 1 --2- 172.21.15.43:0/3532643403 >> [v2:172.21.15.203:6832/1456699667,v1:172.21.15.203:6833/1456699667] conn(0x7fbe0006a170 0x7fbe0006c630 unknown :-1 s=CLOSED pgs=30 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T15:12:12.624 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:12.622+0000 7fbe267b5640 1 --2- 172.21.15.43:0/3532643403 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fbe2015a530 0x7fbe200ccc10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T15:12:12.624 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:12.622+0000 7fbe267b5640 1 --2- 172.21.15.43:0/3532643403 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fbe200c99b0 0x7fbe2015ab70 unknown :-1 s=CLOSED pgs=2533 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T15:12:12.624 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:12.622+0000 7fbe267b5640 1 --2- 172.21.15.43:0/3532643403 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fbe2006f670 0x7fbe20149870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T15:12:12.624 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:12.622+0000 7fbe267b5640 1 -- 172.21.15.43:0/3532643403 >> 172.21.15.43:0/3532643403 conn(0x7fbe20069dd0 msgr2=0x7fbe20159540 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-09T15:12:12.624 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:12.622+0000 7fbe267b5640 1 -- 172.21.15.43:0/3532643403 shutdown_connections 2023-08-09T15:12:12.625 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:12.622+0000 7fbe267b5640 1 -- 172.21.15.43:0/3532643403 wait complete. 2023-08-09T15:12:12.636 INFO:tasks.ceph.mds.a.smithi043.stderr: -5589> 2023-08-09T15:11:33.651+0000 7f97818b1640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2023-08-09T15:12:12.640 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs rm cephfs --yes-i-really-mean-it 2023-08-09T15:12:12.706 INFO:tasks.ceph.mds.a.smithi043.stdout:starting mds.a at 2023-08-09T15:12:12.847 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:12.846+0000 7fb79dd53640 1 Processor -- start 2023-08-09T15:12:12.848 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:12.846+0000 7fb79dd53640 1 -- start start 2023-08-09T15:12:12.850 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:12.848+0000 7fb79dd53640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7fb7980c1560 0x7fb7980c86d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T15:12:12.850 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:12.848+0000 7fb79dd53640 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7fb79806b7b0 0x7fb79806bbc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T15:12:12.850 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:12.848+0000 7fb79dd53640 1 -- --> v1:172.21.15.43:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb7980c8fd0 con 0x7fb798164f30 2023-08-09T15:12:12.850 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:12.848+0000 7fb79dd53640 1 -- --> v2:172.21.15.43:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb7980cc4f0 con 0x7fb7980c1560 2023-08-09T15:12:12.850 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:12.848+0000 7fb79dd53640 1 -- --> v2:172.21.15.203:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb7980c8cf0 con 0x7fb79806b7b0 2023-08-09T15:12:12.850 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:12.848+0000 7fb79cd51640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7fb7980c1560 0x7fb7980c86d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T15:12:12.850 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:12.848+0000 7fb78ffff640 1 --1- >> v1:172.21.15.43:6789/0 conn(0x7fb798164f30 0x7fb7980cc980 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.43:6789/0 says I am v1:172.21.15.43:36724/0 (socket says 172.21.15.43:36724) 2023-08-09T15:12:12.851 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:12.848+0000 7fb78ffff640 1 -- 172.21.15.43:0/180775726 learned_addr learned my addr 172.21.15.43:0/180775726 (peer_addr_for_me v1:172.21.15.43:0/0) 2023-08-09T15:12:12.852 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:12.849+0000 7fb79cd51640 1 -- 172.21.15.43:0/180775726 >> v2:172.21.15.203:3300/0 conn(0x7fb79806b7b0 msgr2=0x7fb79806bbc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-09T15:12:12.852 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:12.849+0000 7fb79d552640 1 --2- 172.21.15.43:0/180775726 >> v2:172.21.15.203:3300/0 conn(0x7fb79806b7b0 0x7fb79806bbc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T15:12:12.852 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:12.849+0000 7fb78f7fe640 1 -- 172.21.15.43:0/180775726 <== mon.0 v1:172.21.15.43:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2325086366 0 0) 0x7fb7980c8fd0 con 0x7fb798164f30 2023-08-09T15:12:12.852 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:12.849+0000 7fb79cd51640 1 --2- 172.21.15.43:0/180775726 >> v2:172.21.15.203:3300/0 conn(0x7fb79806b7b0 0x7fb79806bbc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T15:12:12.852 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:12.849+0000 7fb79cd51640 1 -- 172.21.15.43:0/180775726 >> v1:172.21.15.43:6789/0 conn(0x7fb798164f30 legacy=0x7fb7980cc980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T15:12:12.852 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:12.849+0000 7fb79cd51640 1 -- 172.21.15.43:0/180775726 --> v2:172.21.15.43:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb7980cbea0 con 0x7fb7980c1560 2023-08-09T15:12:12.852 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:12.849+0000 7fb79d552640 1 --2- 172.21.15.43:0/180775726 >> v2:172.21.15.203:3300/0 conn(0x7fb79806b7b0 0x7fb79806bbc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-08-09T15:12:12.853 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:12.850+0000 7fb79cd51640 1 --2- 172.21.15.43:0/180775726 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb7980c1560 0x7fb7980c86d0 secure :-1 s=READY pgs=2537 cs=0 l=1 rev1=1 crypto rx=0x7fb79806f520 tx=0x7fb79402b840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e8b274479519f3fe server_cookie=0 in_seq=0 out_seq=0 2023-08-09T15:12:12.853 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:12.850+0000 7fb78f7fe640 1 -- 172.21.15.43:0/180775726 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb79402e070 con 0x7fb7980c1560 2023-08-09T15:12:12.853 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:12.850+0000 7fb78f7fe640 1 -- 172.21.15.43:0/180775726 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb7940315a0 con 0x7fb7980c1560 2023-08-09T15:12:12.853 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:12.850+0000 7fb78f7fe640 1 -- 172.21.15.43:0/180775726 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb794031930 con 0x7fb7980c1560 2023-08-09T15:12:12.854 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:12.851+0000 7fb79dd53640 1 -- 172.21.15.43:0/180775726 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb7980c1560 msgr2=0x7fb7980c86d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T15:12:12.854 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:12.851+0000 7fb79dd53640 1 --2- 172.21.15.43:0/180775726 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb7980c1560 0x7fb7980c86d0 secure :-1 s=READY pgs=2537 cs=0 l=1 rev1=1 crypto rx=0x7fb79806f520 tx=0x7fb79402b840 comp rx=0 tx=0).stop 2023-08-09T15:12:12.854 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:12.851+0000 7fb79dd53640 1 -- 172.21.15.43:0/180775726 shutdown_connections 2023-08-09T15:12:12.854 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:12.851+0000 7fb79dd53640 1 --2- 172.21.15.43:0/180775726 >> v2:172.21.15.203:3300/0 conn(0x7fb79806b7b0 0x7fb79806bbc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T15:12:12.854 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:12.851+0000 7fb79dd53640 1 --2- 172.21.15.43:0/180775726 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb7980c1560 0x7fb7980c86d0 unknown :-1 s=CLOSED pgs=2537 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T15:12:12.855 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:12.851+0000 7fb79dd53640 1 -- 172.21.15.43:0/180775726 >> 172.21.15.43:0/180775726 conn(0x7fb7980afbd0 msgr2=0x7fb7980affb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-09T15:12:12.855 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:12.851+0000 7fb79dd53640 1 -- 172.21.15.43:0/180775726 shutdown_connections 2023-08-09T15:12:12.855 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:12.851+0000 7fb79dd53640 1 -- 172.21.15.43:0/180775726 wait complete. 2023-08-09T15:12:12.855 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:12.853+0000 7fb79dd53640 1 Processor -- start 2023-08-09T15:12:12.855 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:12.853+0000 7fb79dd53640 1 -- start start 2023-08-09T15:12:12.857 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:12.854+0000 7fb79dd53640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fb79806b7b0 0x7fb798155710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T15:12:12.857 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:12.854+0000 7fb79dd53640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fb7980c1560 0x7fb7980b11c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T15:12:12.857 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:12.854+0000 7fb79dd53640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb798164f30 0x7fb7980b1700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T15:12:12.857 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:12.854+0000 7fb79dd53640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb7980b1d30 con 0x7fb798164f30 2023-08-09T15:12:12.858 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:12.854+0000 7fb79dd53640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb7980b2010 con 0x7fb7980c1560 2023-08-09T15:12:12.858 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:12.854+0000 7fb79dd53640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb7980c64d0 con 0x7fb79806b7b0 2023-08-09T15:12:12.858 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:12.854+0000 7fb79d552640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb798164f30 0x7fb7980b1700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T15:12:12.858 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:12.854+0000 7fb79d552640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb798164f30 0x7fb7980b1700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:33422/0 (socket says 172.21.15.43:33422) 2023-08-09T15:12:12.858 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:12.854+0000 7fb79d552640 1 -- 172.21.15.43:0/1956882527 learned_addr learned my addr 172.21.15.43:0/1956882527 (peer_addr_for_me v2:172.21.15.43:0/0) 2023-08-09T15:12:12.858 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:12.854+0000 7fb78ffff640 1 --2- 172.21.15.43:0/1956882527 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fb7980c1560 0x7fb7980b11c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T15:12:12.858 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:12.854+0000 7fb79cd51640 1 --2- 172.21.15.43:0/1956882527 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fb79806b7b0 0x7fb798155710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T15:12:12.859 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:12.854+0000 7fb79d552640 1 -- 172.21.15.43:0/1956882527 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fb79806b7b0 msgr2=0x7fb798155710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T15:12:12.859 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:12.854+0000 7fb79d552640 1 --2- 172.21.15.43:0/1956882527 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fb79806b7b0 0x7fb798155710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T15:12:12.859 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:12.854+0000 7fb79d552640 1 -- 172.21.15.43:0/1956882527 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fb7980c1560 msgr2=0x7fb7980b11c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T15:12:12.859 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:12.854+0000 7fb79d552640 1 --2- 172.21.15.43:0/1956882527 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fb7980c1560 0x7fb7980b11c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T15:12:12.859 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:12.854+0000 7fb79d552640 1 -- 172.21.15.43:0/1956882527 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb794009400 con 0x7fb798164f30 2023-08-09T15:12:12.859 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:12.855+0000 7fb79d552640 1 --2- 172.21.15.43:0/1956882527 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb798164f30 0x7fb7980b1700 secure :-1 s=READY pgs=2538 cs=0 l=1 rev1=1 crypto rx=0x7fb79000cef0 tx=0x7fb79000a710 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-09T15:12:12.859 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:12.855+0000 7fb78d7fa640 1 -- 172.21.15.43:0/1956882527 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb79000d840 con 0x7fb798164f30 2023-08-09T15:12:12.859 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:12.855+0000 7fb78d7fa640 1 -- 172.21.15.43:0/1956882527 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb79000d9c0 con 0x7fb798164f30 2023-08-09T15:12:12.860 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:12.855+0000 7fb79dd53640 1 -- 172.21.15.43:0/1956882527 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb7980c67b0 con 0x7fb798164f30 2023-08-09T15:12:12.860 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:12.855+0000 7fb78d7fa640 1 -- 172.21.15.43:0/1956882527 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb79000dd70 con 0x7fb798164f30 2023-08-09T15:12:12.860 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:12.855+0000 7fb79dd53640 1 -- 172.21.15.43:0/1956882527 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb798155c50 con 0x7fb798164f30 2023-08-09T15:12:12.861 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:12.856+0000 7fb79dd53640 1 -- 172.21.15.43:0/1956882527 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb7980c69d0 con 0x7fb798164f30 2023-08-09T15:12:12.861 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:12.857+0000 7fb78d7fa640 1 -- 172.21.15.43:0/1956882527 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 129) v1 ==== 88443+0+0 (secure 0 0 0) 0x7fb79001e3b0 con 0x7fb798164f30 2023-08-09T15:12:12.861 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:12.858+0000 7fb78d7fa640 1 --2- 172.21.15.43:0/1956882527 >> [v2:172.21.15.203:6832/1456699667,v1:172.21.15.203:6833/1456699667] conn(0x7fb78406a170 0x7fb78406c630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T15:12:12.861 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:12.858+0000 7fb78d7fa640 1 -- 172.21.15.43:0/1956882527 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(227..227 src has 1..227) v4 ==== 7314+0+0 (secure 0 0 0) 0x7fb790089790 con 0x7fb798164f30 2023-08-09T15:12:12.861 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:12.859+0000 7fb79cd51640 1 --2- 172.21.15.43:0/1956882527 >> [v2:172.21.15.203:6832/1456699667,v1:172.21.15.203:6833/1456699667] conn(0x7fb78406a170 0x7fb78406c630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T15:12:12.862 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:12.860+0000 7fb79cd51640 1 --2- 172.21.15.43:0/1956882527 >> [v2:172.21.15.203:6832/1456699667,v1:172.21.15.203:6833/1456699667] conn(0x7fb78406a170 0x7fb78406c630 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0x7fb794036000 tx=0x7fb79403d040 comp rx=0 tx=0).ready entity=mgr.9232 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-09T15:12:12.864 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:12.863+0000 7fb78d7fa640 1 -- 172.21.15.43:0/1956882527 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178066 (secure 0 0 0) 0x7fb7900548a0 con 0x7fb798164f30 2023-08-09T15:12:13.101 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:13.099+0000 7fb79dd53640 1 -- 172.21.15.43:0/1956882527 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} v 0) v1 -- 0x7fb79804e390 con 0x7fb798164f30 2023-08-09T15:12:13.624 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:13.623+0000 7fb78d7fa640 1 -- 172.21.15.43:0/1956882527 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]=0 v106) v1 ==== 104+0+0 (secure 0 0 0) 0x7fb790057ec0 con 0x7fb798164f30 2023-08-09T15:12:13.628 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:13.626+0000 7fb79dd53640 1 -- 172.21.15.43:0/1956882527 >> [v2:172.21.15.203:6832/1456699667,v1:172.21.15.203:6833/1456699667] conn(0x7fb78406a170 msgr2=0x7fb78406c630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T15:12:13.628 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:13.626+0000 7fb79dd53640 1 --2- 172.21.15.43:0/1956882527 >> [v2:172.21.15.203:6832/1456699667,v1:172.21.15.203:6833/1456699667] conn(0x7fb78406a170 0x7fb78406c630 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0x7fb794036000 tx=0x7fb79403d040 comp rx=0 tx=0).stop 2023-08-09T15:12:13.629 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:13.627+0000 7fb79dd53640 1 -- 172.21.15.43:0/1956882527 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb798164f30 msgr2=0x7fb7980b1700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T15:12:13.629 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:13.627+0000 7fb79dd53640 1 --2- 172.21.15.43:0/1956882527 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb798164f30 0x7fb7980b1700 secure :-1 s=READY pgs=2538 cs=0 l=1 rev1=1 crypto rx=0x7fb79000cef0 tx=0x7fb79000a710 comp rx=0 tx=0).stop 2023-08-09T15:12:13.629 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:13.627+0000 7fb79dd53640 1 -- 172.21.15.43:0/1956882527 shutdown_connections 2023-08-09T15:12:13.629 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:13.627+0000 7fb79dd53640 1 --2- 172.21.15.43:0/1956882527 >> [v2:172.21.15.203:6832/1456699667,v1:172.21.15.203:6833/1456699667] conn(0x7fb78406a170 0x7fb78406c630 unknown :-1 s=CLOSED pgs=31 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T15:12:13.629 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:13.627+0000 7fb79dd53640 1 --2- 172.21.15.43:0/1956882527 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb798164f30 0x7fb7980b1700 unknown :-1 s=CLOSED pgs=2538 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T15:12:13.629 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:13.627+0000 7fb79dd53640 1 --2- 172.21.15.43:0/1956882527 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fb7980c1560 0x7fb7980b11c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T15:12:13.629 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:13.627+0000 7fb79dd53640 1 --2- 172.21.15.43:0/1956882527 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fb79806b7b0 0x7fb798155710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T15:12:13.630 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:13.627+0000 7fb79dd53640 1 -- 172.21.15.43:0/1956882527 >> 172.21.15.43:0/1956882527 conn(0x7fb7980afbd0 msgr2=0x7fb798162a00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-09T15:12:13.630 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:13.627+0000 7fb79dd53640 1 -- 172.21.15.43:0/1956882527 shutdown_connections 2023-08-09T15:12:13.630 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:13.627+0000 7fb79dd53640 1 -- 172.21.15.43:0/1956882527 wait complete. 2023-08-09T15:12:13.651 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_metadata cephfs_metadata --yes-i-really-really-mean-it 2023-08-09T15:12:13.876 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:13.875+0000 7fd306366640 1 Processor -- start 2023-08-09T15:12:13.877 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:13.875+0000 7fd306366640 1 -- start start 2023-08-09T15:12:13.878 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:13.877+0000 7fd306366640 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7fd3001430b0 0x7fd3000c8640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T15:12:13.878 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:13.877+0000 7fd306366640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fd30006cea0 0x7fd3000c3960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T15:12:13.878 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:13.877+0000 7fd306366640 1 -- --> v1:172.21.15.203:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd3000c8fe0 con 0x7fd3000c8b80 2023-08-09T15:12:13.878 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:13.877+0000 7fd306366640 1 -- --> v2:172.21.15.203:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd3000c4820 con 0x7fd3001430b0 2023-08-09T15:12:13.879 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:13.877+0000 7fd306366640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd30006ba70 con 0x7fd30006cea0 2023-08-09T15:12:13.879 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:13.878+0000 7fd305364640 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7fd3001430b0 0x7fd3000c8640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T15:12:13.879 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:13.878+0000 7fd305b65640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fd30006cea0 0x7fd3000c3960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T15:12:13.880 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:13.878+0000 7fd305364640 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7fd3001430b0 0x7fd3000c8640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.203:3300/0 says I am v2:172.21.15.43:41526/0 (socket says 172.21.15.43:41526) 2023-08-09T15:12:13.880 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:13.878+0000 7fd305364640 1 -- 172.21.15.43:0/1330554709 learned_addr learned my addr 172.21.15.43:0/1330554709 (peer_addr_for_me v2:172.21.15.43:0/0) 2023-08-09T15:12:13.880 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:13.878+0000 7fd305364640 1 -- 172.21.15.43:0/1330554709 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fd30006cea0 msgr2=0x7fd3000c3960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-09T15:12:13.881 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:13.878+0000 7fd305364640 1 --2- 172.21.15.43:0/1330554709 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fd30006cea0 0x7fd3000c3960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T15:12:13.881 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:13.878+0000 7fd305364640 1 -- 172.21.15.43:0/1330554709 >> v1:172.21.15.203:6789/0 conn(0x7fd3000c8b80 legacy=0x7fd3000c3250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T15:12:13.881 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:13.878+0000 7fd305364640 1 -- 172.21.15.43:0/1330554709 --> v2:172.21.15.203:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd300143490 con 0x7fd3001430b0 2023-08-09T15:12:13.881 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:13.880+0000 7fd305364640 1 --2- 172.21.15.43:0/1330554709 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fd3001430b0 0x7fd3000c8640 secure :-1 s=READY pgs=927 cs=0 l=1 rev1=1 crypto rx=0x7fd2fc009870 tx=0x7fd2fc02b980 comp rx=0 tx=0).ready entity=mon.1 client_cookie=2b743d6b0b0831cd server_cookie=0 in_seq=0 out_seq=0 2023-08-09T15:12:13.882 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:13.881+0000 7fd2f7fff640 1 -- 172.21.15.43:0/1330554709 <== mon.1 v2:172.21.15.203:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd2fc030840 con 0x7fd3001430b0 2023-08-09T15:12:13.882 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:13.881+0000 7fd2f7fff640 1 -- 172.21.15.43:0/1330554709 <== mon.1 v2:172.21.15.203:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd2fc0309a0 con 0x7fd3001430b0 2023-08-09T15:12:13.883 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:13.881+0000 7fd2f7fff640 1 -- 172.21.15.43:0/1330554709 <== mon.1 v2:172.21.15.203:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd2fc030d30 con 0x7fd3001430b0 2023-08-09T15:12:13.883 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:13.881+0000 7fd306366640 1 -- 172.21.15.43:0/1330554709 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fd3001430b0 msgr2=0x7fd3000c8640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T15:12:13.883 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:13.881+0000 7fd306366640 1 --2- 172.21.15.43:0/1330554709 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fd3001430b0 0x7fd3000c8640 secure :-1 s=READY pgs=927 cs=0 l=1 rev1=1 crypto rx=0x7fd2fc009870 tx=0x7fd2fc02b980 comp rx=0 tx=0).stop 2023-08-09T15:12:13.883 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:13.881+0000 7fd306366640 1 -- 172.21.15.43:0/1330554709 shutdown_connections 2023-08-09T15:12:13.884 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:13.881+0000 7fd306366640 1 --2- 172.21.15.43:0/1330554709 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fd30006cea0 0x7fd3000c3960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T15:12:13.884 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:13.881+0000 7fd306366640 1 --2- 172.21.15.43:0/1330554709 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fd3001430b0 0x7fd3000c8640 unknown :-1 s=CLOSED pgs=927 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T15:12:13.884 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:13.881+0000 7fd306366640 1 -- 172.21.15.43:0/1330554709 >> 172.21.15.43:0/1330554709 conn(0x7fd3000cdff0 msgr2=0x7fd3000ce3d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-09T15:12:13.884 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:13.881+0000 7fd306366640 1 -- 172.21.15.43:0/1330554709 shutdown_connections 2023-08-09T15:12:13.884 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:13.881+0000 7fd306366640 1 -- 172.21.15.43:0/1330554709 wait complete. 2023-08-09T15:12:13.885 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:13.883+0000 7fd306366640 1 Processor -- start 2023-08-09T15:12:13.885 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:13.883+0000 7fd306366640 1 -- start start 2023-08-09T15:12:13.886 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:13.884+0000 7fd306366640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fd30006cea0 0x7fd300159e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T15:12:13.886 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:13.884+0000 7fd306366640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fd3000c8b80 0x7fd30013c330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T15:12:13.886 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:13.884+0000 7fd306366640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fd3001430b0 0x7fd3000b1f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T15:12:13.886 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:13.884+0000 7fd306366640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd3001e2ac0 con 0x7fd3000c8b80 2023-08-09T15:12:13.886 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:13.884+0000 7fd306366640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd3001e2d30 con 0x7fd30006cea0 2023-08-09T15:12:13.887 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:13.884+0000 7fd306366640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd30015a3a0 con 0x7fd3001430b0 2023-08-09T15:12:13.888 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:13.885+0000 7fd304b63640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fd3000c8b80 0x7fd30013c330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T15:12:13.888 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:13.885+0000 7fd304b63640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fd3000c8b80 0x7fd30013c330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:33430/0 (socket says 172.21.15.43:33430) 2023-08-09T15:12:13.888 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:13.885+0000 7fd304b63640 1 -- 172.21.15.43:0/29182514 learned_addr learned my addr 172.21.15.43:0/29182514 (peer_addr_for_me v2:172.21.15.43:0/0) 2023-08-09T15:12:13.889 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:13.885+0000 7fd305364640 1 --2- 172.21.15.43:0/29182514 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fd30006cea0 0x7fd300159e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T15:12:13.889 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:13.885+0000 7fd305b65640 1 --2- 172.21.15.43:0/29182514 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fd3001430b0 0x7fd3000b1f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T15:12:13.889 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:13.885+0000 7fd304b63640 1 -- 172.21.15.43:0/29182514 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fd3001430b0 msgr2=0x7fd3000b1f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T15:12:13.889 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:13.885+0000 7fd304b63640 1 --2- 172.21.15.43:0/29182514 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fd3001430b0 0x7fd3000b1f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T15:12:13.889 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:13.885+0000 7fd304b63640 1 -- 172.21.15.43:0/29182514 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fd30006cea0 msgr2=0x7fd300159e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T15:12:13.890 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:13.885+0000 7fd304b63640 1 --2- 172.21.15.43:0/29182514 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fd30006cea0 0x7fd300159e60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T15:12:13.890 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:13.885+0000 7fd304b63640 1 -- 172.21.15.43:0/29182514 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd2fc0094d0 con 0x7fd3000c8b80 2023-08-09T15:12:13.890 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:13.885+0000 7fd305364640 1 --2- 172.21.15.43:0/29182514 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fd30006cea0 0x7fd300159e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-08-09T15:12:13.891 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:13.886+0000 7fd304b63640 1 --2- 172.21.15.43:0/29182514 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fd3000c8b80 0x7fd30013c330 secure :-1 s=READY pgs=2539 cs=0 l=1 rev1=1 crypto rx=0x7fd2f000c480 tx=0x7fd2f000c940 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-09T15:12:13.891 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:13.886+0000 7fd2f5ffb640 1 -- 172.21.15.43:0/29182514 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd2f00052d0 con 0x7fd3000c8b80 2023-08-09T15:12:13.891 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:13.886+0000 7fd2f5ffb640 1 -- 172.21.15.43:0/29182514 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd2f0005430 con 0x7fd3000c8b80 2023-08-09T15:12:13.891 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:13.886+0000 7fd2f5ffb640 1 -- 172.21.15.43:0/29182514 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd2f0012870 con 0x7fd3000c8b80 2023-08-09T15:12:13.892 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:13.886+0000 7fd306366640 1 -- 172.21.15.43:0/29182514 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd30013c870 con 0x7fd3000c8b80 2023-08-09T15:12:13.892 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:13.886+0000 7fd306366640 1 -- 172.21.15.43:0/29182514 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd30013cdf0 con 0x7fd3000c8b80 2023-08-09T15:12:13.893 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:13.887+0000 7fd306366640 1 -- 172.21.15.43:0/29182514 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd3001e2ee0 con 0x7fd3000c8b80 2023-08-09T15:12:13.893 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:13.889+0000 7fd2f5ffb640 1 -- 172.21.15.43:0/29182514 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 129) v1 ==== 88443+0+0 (secure 0 0 0) 0x7fd2f00129d0 con 0x7fd3000c8b80 2023-08-09T15:12:13.894 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:13.890+0000 7fd2f5ffb640 1 --2- 172.21.15.43:0/29182514 >> [v2:172.21.15.203:6832/1456699667,v1:172.21.15.203:6833/1456699667] conn(0x7fd2e406a170 0x7fd2e406c630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T15:12:13.894 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:13.890+0000 7fd2f5ffb640 1 -- 172.21.15.43:0/29182514 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(227..227 src has 1..227) v4 ==== 7314+0+0 (secure 0 0 0) 0x7fd2f00887d0 con 0x7fd3000c8b80 2023-08-09T15:12:13.894 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:13.891+0000 7fd305364640 1 --2- 172.21.15.43:0/29182514 >> [v2:172.21.15.203:6832/1456699667,v1:172.21.15.203:6833/1456699667] conn(0x7fd2e406a170 0x7fd2e406c630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T15:12:13.894 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:13.892+0000 7fd305364640 1 --2- 172.21.15.43:0/29182514 >> [v2:172.21.15.203:6832/1456699667,v1:172.21.15.203:6833/1456699667] conn(0x7fd2e406a170 0x7fd2e406c630 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0x7fd2fc036000 tx=0x7fd2fc03b040 comp rx=0 tx=0).ready entity=mgr.9232 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-09T15:12:13.895 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:13.894+0000 7fd2f5ffb640 1 -- 172.21.15.43:0/29182514 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178066 (secure 0 0 0) 0x7fd3001e2ee0 con 0x7fd3000c8b80 2023-08-09T15:12:14.090 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:14.089+0000 7fd306366640 1 -- 172.21.15.43:0/29182514 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7fd3001e2ee0 con 0x7fd3000c8b80 2023-08-09T15:12:14.636 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:14.634+0000 7fd2f5ffb640 1 -- 172.21.15.43:0/29182514 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_metadata' removed v228) v1 ==== 181+0+0 (secure 0 0 0) 0x7fd3001e2ee0 con 0x7fd3000c8b80 2023-08-09T15:12:14.637 INFO:teuthology.orchestra.run.smithi043.stderr:pool 'cephfs_metadata' removed 2023-08-09T15:12:14.642 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:14.639+0000 7fd306366640 1 -- 172.21.15.43:0/29182514 >> [v2:172.21.15.203:6832/1456699667,v1:172.21.15.203:6833/1456699667] conn(0x7fd2e406a170 msgr2=0x7fd2e406c630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T15:12:14.642 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:14.639+0000 7fd306366640 1 --2- 172.21.15.43:0/29182514 >> [v2:172.21.15.203:6832/1456699667,v1:172.21.15.203:6833/1456699667] conn(0x7fd2e406a170 0x7fd2e406c630 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0x7fd2fc036000 tx=0x7fd2fc03b040 comp rx=0 tx=0).stop 2023-08-09T15:12:14.642 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:14.639+0000 7fd306366640 1 -- 172.21.15.43:0/29182514 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fd3000c8b80 msgr2=0x7fd30013c330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T15:12:14.643 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:14.639+0000 7fd306366640 1 --2- 172.21.15.43:0/29182514 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fd3000c8b80 0x7fd30013c330 secure :-1 s=READY pgs=2539 cs=0 l=1 rev1=1 crypto rx=0x7fd2f000c480 tx=0x7fd2f000c940 comp rx=0 tx=0).stop 2023-08-09T15:12:14.643 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:14.640+0000 7fd306366640 1 -- 172.21.15.43:0/29182514 shutdown_connections 2023-08-09T15:12:14.644 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:14.640+0000 7fd306366640 1 --2- 172.21.15.43:0/29182514 >> [v2:172.21.15.203:6832/1456699667,v1:172.21.15.203:6833/1456699667] conn(0x7fd2e406a170 0x7fd2e406c630 unknown :-1 s=CLOSED pgs=33 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T15:12:14.644 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:14.640+0000 7fd306366640 1 --2- 172.21.15.43:0/29182514 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fd3001430b0 0x7fd3000b1f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T15:12:14.644 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:14.640+0000 7fd306366640 1 --2- 172.21.15.43:0/29182514 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fd3000c8b80 0x7fd30013c330 unknown :-1 s=CLOSED pgs=2539 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T15:12:14.645 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:14.640+0000 7fd306366640 1 --2- 172.21.15.43:0/29182514 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fd30006cea0 0x7fd300159e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T15:12:14.645 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:14.640+0000 7fd306366640 1 -- 172.21.15.43:0/29182514 >> 172.21.15.43:0/29182514 conn(0x7fd3000cdff0 msgr2=0x7fd30006fb60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-09T15:12:14.646 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:14.641+0000 7fd306366640 1 -- 172.21.15.43:0/29182514 shutdown_connections 2023-08-09T15:12:14.646 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:14.641+0000 7fd306366640 1 -- 172.21.15.43:0/29182514 wait complete. 2023-08-09T15:12:14.667 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_data cephfs_data --yes-i-really-really-mean-it 2023-08-09T15:12:14.897 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:14.896+0000 7f3d1fef3640 1 Processor -- start 2023-08-09T15:12:14.898 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:14.896+0000 7f3d1fef3640 1 -- start start 2023-08-09T15:12:14.899 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:14.898+0000 7f3d1fef3640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f3d180c9350 0x7f3d180c97c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T15:12:14.899 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:14.898+0000 7f3d1fef3640 1 -- --> v1:172.21.15.43:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3d180c8c00 con 0x7f3d180ca580 2023-08-09T15:12:14.899 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:14.898+0000 7f3d1fef3640 1 -- --> v1:172.21.15.203:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3d1815b2d0 con 0x7f3d1815a650 2023-08-09T15:12:14.900 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:14.898+0000 7f3d1fef3640 1 -- --> v2:172.21.15.43:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3d1815b5b0 con 0x7f3d180c9350 2023-08-09T15:12:14.901 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:14.898+0000 7f3d1d48e640 1 --1- >> v1:172.21.15.43:6789/0 conn(0x7f3d180ca580 0x7f3d1815bc40 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.43:6789/0 says I am v1:172.21.15.43:36732/0 (socket says 172.21.15.43:36732) 2023-08-09T15:12:14.901 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:14.898+0000 7f3d1d48e640 1 -- 172.21.15.43:0/2580698072 learned_addr learned my addr 172.21.15.43:0/2580698072 (peer_addr_for_me v1:172.21.15.43:0/0) 2023-08-09T15:12:14.902 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:14.898+0000 7f3d1e490640 1 --2- 172.21.15.43:0/2580698072 >> v2:172.21.15.43:3300/0 conn(0x7f3d180c9350 0x7f3d180c97c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T15:12:14.902 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:14.899+0000 7f3d1e490640 1 -- 172.21.15.43:0/2580698072 >> v1:172.21.15.203:6789/0 conn(0x7f3d1815a650 legacy=0x7f3d1815aa30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T15:12:14.902 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:14.899+0000 7f3d1cc8d640 1 -- 172.21.15.43:0/2580698072 <== mon.0 v1:172.21.15.43:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4284660538 0 0) 0x7f3d180c8c00 con 0x7f3d180ca580 2023-08-09T15:12:14.903 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:14.899+0000 7f3d1e490640 1 -- 172.21.15.43:0/2580698072 >> v1:172.21.15.43:6789/0 conn(0x7f3d180ca580 legacy=0x7f3d1815bc40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T15:12:14.903 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:14.899+0000 7f3d1e490640 1 -- 172.21.15.43:0/2580698072 --> v2:172.21.15.43:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3d1815b890 con 0x7f3d180c9350 2023-08-09T15:12:14.903 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:14.900+0000 7f3d1e490640 1 --2- 172.21.15.43:0/2580698072 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f3d180c9350 0x7f3d180c97c0 secure :-1 s=READY pgs=2541 cs=0 l=1 rev1=1 crypto rx=0x7f3d10009d90 tx=0x7f3d1002bf50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=21661be1e7c9193d server_cookie=0 in_seq=0 out_seq=0 2023-08-09T15:12:14.907 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:14.900+0000 7f3d1cc8d640 1 -- 172.21.15.43:0/2580698072 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3d1002cd00 con 0x7f3d180c9350 2023-08-09T15:12:14.907 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:14.900+0000 7f3d1cc8d640 1 -- 172.21.15.43:0/2580698072 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3d1002ce60 con 0x7f3d180c9350 2023-08-09T15:12:14.908 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:14.900+0000 7f3d1cc8d640 1 -- 172.21.15.43:0/2580698072 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3d10034660 con 0x7f3d180c9350 2023-08-09T15:12:14.908 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:14.901+0000 7f3d1fef3640 1 -- 172.21.15.43:0/2580698072 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f3d180c9350 msgr2=0x7f3d180c97c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T15:12:14.908 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:14.901+0000 7f3d1fef3640 1 --2- 172.21.15.43:0/2580698072 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f3d180c9350 0x7f3d180c97c0 secure :-1 s=READY pgs=2541 cs=0 l=1 rev1=1 crypto rx=0x7f3d10009d90 tx=0x7f3d1002bf50 comp rx=0 tx=0).stop 2023-08-09T15:12:14.908 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:14.901+0000 7f3d1fef3640 1 -- 172.21.15.43:0/2580698072 shutdown_connections 2023-08-09T15:12:14.908 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:14.901+0000 7f3d1fef3640 1 --2- 172.21.15.43:0/2580698072 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f3d180c9350 0x7f3d180c97c0 unknown :-1 s=CLOSED pgs=2541 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T15:12:14.909 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:14.901+0000 7f3d1fef3640 1 -- 172.21.15.43:0/2580698072 >> 172.21.15.43:0/2580698072 conn(0x7f3d180afdb0 msgr2=0x7f3d180b0190 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-09T15:12:14.909 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:14.901+0000 7f3d1fef3640 1 -- 172.21.15.43:0/2580698072 shutdown_connections 2023-08-09T15:12:14.909 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:14.901+0000 7f3d1fef3640 1 -- 172.21.15.43:0/2580698072 wait complete. 2023-08-09T15:12:14.909 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:14.903+0000 7f3d1fef3640 1 Processor -- start 2023-08-09T15:12:14.909 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:14.903+0000 7f3d1fef3640 1 -- start start 2023-08-09T15:12:14.909 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:14.904+0000 7f3d1fef3640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f3d180c9350 0x7f3d180c7cb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T15:12:14.909 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:14.904+0000 7f3d1fef3640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f3d180ca580 0x7f3d180c5770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T15:12:14.909 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:14.904+0000 7f3d1fef3640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f3d1815a650 0x7f3d180c5cb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T15:12:14.910 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:14.904+0000 7f3d1fef3640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3d1815d050 con 0x7f3d180c9350 2023-08-09T15:12:14.910 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:14.904+0000 7f3d1fef3640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3d180cc040 con 0x7f3d180ca580 2023-08-09T15:12:14.910 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:14.904+0000 7f3d1fef3640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3d180cc320 con 0x7f3d1815a650 2023-08-09T15:12:14.910 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:14.904+0000 7f3d1dc8f640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f3d180c9350 0x7f3d180c7cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T15:12:14.910 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:14.904+0000 7f3d1dc8f640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f3d180c9350 0x7f3d180c7cb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:33450/0 (socket says 172.21.15.43:33450) 2023-08-09T15:12:14.910 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:14.904+0000 7f3d1dc8f640 1 -- 172.21.15.43:0/3442816900 learned_addr learned my addr 172.21.15.43:0/3442816900 (peer_addr_for_me v2:172.21.15.43:0/0) 2023-08-09T15:12:14.910 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:14.905+0000 7f3d1d48e640 1 --2- 172.21.15.43:0/3442816900 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f3d180ca580 0x7f3d180c5770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T15:12:14.911 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:14.905+0000 7f3d1dc8f640 1 -- 172.21.15.43:0/3442816900 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f3d1815a650 msgr2=0x7f3d180c5cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T15:12:14.911 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:14.905+0000 7f3d1dc8f640 1 --2- 172.21.15.43:0/3442816900 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f3d1815a650 0x7f3d180c5cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T15:12:14.911 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:14.905+0000 7f3d1dc8f640 1 -- 172.21.15.43:0/3442816900 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f3d180ca580 msgr2=0x7f3d180c5770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T15:12:14.911 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:14.905+0000 7f3d1dc8f640 1 --2- 172.21.15.43:0/3442816900 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f3d180ca580 0x7f3d180c5770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T15:12:14.911 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:14.905+0000 7f3d1dc8f640 1 -- 172.21.15.43:0/3442816900 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3d10009a70 con 0x7f3d180c9350 2023-08-09T15:12:14.911 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:14.905+0000 7f3d1d48e640 1 --2- 172.21.15.43:0/3442816900 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f3d180ca580 0x7f3d180c5770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-08-09T15:12:14.911 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:14.905+0000 7f3d1dc8f640 1 --2- 172.21.15.43:0/3442816900 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f3d180c9350 0x7f3d180c7cb0 secure :-1 s=READY pgs=2542 cs=0 l=1 rev1=1 crypto rx=0x7f3d1400c400 tx=0x7f3d1400c8c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-09T15:12:14.911 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:14.905+0000 7f3d0e7fc640 1 -- 172.21.15.43:0/3442816900 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3d140052d0 con 0x7f3d180c9350 2023-08-09T15:12:14.912 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:14.905+0000 7f3d0e7fc640 1 -- 172.21.15.43:0/3442816900 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3d14005430 con 0x7f3d180c9350 2023-08-09T15:12:14.912 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:14.905+0000 7f3d0e7fc640 1 -- 172.21.15.43:0/3442816900 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3d14003830 con 0x7f3d180c9350 2023-08-09T15:12:14.912 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:14.905+0000 7f3d1fef3640 1 -- 172.21.15.43:0/3442816900 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3d180cc600 con 0x7f3d180c9350 2023-08-09T15:12:14.912 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:14.905+0000 7f3d1fef3640 1 -- 172.21.15.43:0/3442816900 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3d180c66c0 con 0x7f3d180c9350 2023-08-09T15:12:14.913 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:14.906+0000 7f3d1fef3640 1 -- 172.21.15.43:0/3442816900 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3d180cabb0 con 0x7f3d180c9350 2023-08-09T15:12:14.913 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:14.908+0000 7f3d0e7fc640 1 -- 172.21.15.43:0/3442816900 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 129) v1 ==== 88443+0+0 (secure 0 0 0) 0x7f3d14003990 con 0x7f3d180c9350 2023-08-09T15:12:14.913 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:14.909+0000 7f3d0e7fc640 1 --2- 172.21.15.43:0/3442816900 >> [v2:172.21.15.203:6832/1456699667,v1:172.21.15.203:6833/1456699667] conn(0x7f3cfc06a170 0x7f3cfc06c630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T15:12:14.914 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:14.909+0000 7f3d0e7fc640 1 -- 172.21.15.43:0/3442816900 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(228..228 src has 1..228) v4 ==== 6876+0+0 (secure 0 0 0) 0x7f3d14088060 con 0x7f3d180c9350 2023-08-09T15:12:14.914 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:14.909+0000 7f3d1d48e640 1 --2- 172.21.15.43:0/3442816900 >> [v2:172.21.15.203:6832/1456699667,v1:172.21.15.203:6833/1456699667] conn(0x7f3cfc06a170 0x7f3cfc06c630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T15:12:14.914 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:14.911+0000 7f3d1d48e640 1 --2- 172.21.15.43:0/3442816900 >> [v2:172.21.15.203:6832/1456699667,v1:172.21.15.203:6833/1456699667] conn(0x7f3cfc06a170 0x7f3cfc06c630 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0x7f3d18159740 tx=0x7f3d08009040 comp rx=0 tx=0).ready entity=mgr.9232 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-09T15:12:14.914 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:14.913+0000 7f3d0e7fc640 1 -- 172.21.15.43:0/3442816900 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178066 (secure 0 0 0) 0x7f3d140ba7b0 con 0x7f3d180c9350 2023-08-09T15:12:15.112 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:15.111+0000 7f3d1fef3640 1 -- 172.21.15.43:0/3442816900 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7f3d1804db40 con 0x7f3d180c9350 2023-08-09T15:12:15.643 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:15.642+0000 7f3d0e7fc640 1 -- 172.21.15.43:0/3442816900 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data' removed v229) v1 ==== 169+0+0 (secure 0 0 0) 0x7f3d14053270 con 0x7f3d180c9350 2023-08-09T15:12:15.645 INFO:teuthology.orchestra.run.smithi043.stderr:pool 'cephfs_data' removed 2023-08-09T15:12:15.650 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:15.649+0000 7f3d1fef3640 1 -- 172.21.15.43:0/3442816900 >> [v2:172.21.15.203:6832/1456699667,v1:172.21.15.203:6833/1456699667] conn(0x7f3cfc06a170 msgr2=0x7f3cfc06c630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T15:12:15.651 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:15.649+0000 7f3d1fef3640 1 --2- 172.21.15.43:0/3442816900 >> [v2:172.21.15.203:6832/1456699667,v1:172.21.15.203:6833/1456699667] conn(0x7f3cfc06a170 0x7f3cfc06c630 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0x7f3d18159740 tx=0x7f3d08009040 comp rx=0 tx=0).stop 2023-08-09T15:12:15.651 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:15.649+0000 7f3d1fef3640 1 -- 172.21.15.43:0/3442816900 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f3d180c9350 msgr2=0x7f3d180c7cb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T15:12:15.651 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:15.649+0000 7f3d1fef3640 1 --2- 172.21.15.43:0/3442816900 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f3d180c9350 0x7f3d180c7cb0 secure :-1 s=READY pgs=2542 cs=0 l=1 rev1=1 crypto rx=0x7f3d1400c400 tx=0x7f3d1400c8c0 comp rx=0 tx=0).stop 2023-08-09T15:12:15.651 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:15.649+0000 7f3d1fef3640 1 -- 172.21.15.43:0/3442816900 shutdown_connections 2023-08-09T15:12:15.651 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:15.649+0000 7f3d1fef3640 1 --2- 172.21.15.43:0/3442816900 >> [v2:172.21.15.203:6832/1456699667,v1:172.21.15.203:6833/1456699667] conn(0x7f3cfc06a170 0x7f3cfc06c630 unknown :-1 s=CLOSED pgs=34 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T15:12:15.652 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:15.649+0000 7f3d1fef3640 1 --2- 172.21.15.43:0/3442816900 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f3d1815a650 0x7f3d180c5cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T15:12:15.652 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:15.649+0000 7f3d1fef3640 1 --2- 172.21.15.43:0/3442816900 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f3d180ca580 0x7f3d180c5770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T15:12:15.652 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:15.649+0000 7f3d1fef3640 1 --2- 172.21.15.43:0/3442816900 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f3d180c9350 0x7f3d180c7cb0 unknown :-1 s=CLOSED pgs=2542 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T15:12:15.652 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:15.649+0000 7f3d1fef3640 1 -- 172.21.15.43:0/3442816900 >> 172.21.15.43:0/3442816900 conn(0x7f3d180afdb0 msgr2=0x7f3d18072cb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-09T15:12:15.652 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:15.650+0000 7f3d1fef3640 1 -- 172.21.15.43:0/3442816900 shutdown_connections 2023-08-09T15:12:15.653 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:15.651+0000 7f3d1fef3640 1 -- 172.21.15.43:0/3442816900 wait complete. 2023-08-09T15:12:15.680 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config rm mgr mgr/snap_schedule/dump_on_update 2023-08-09T15:12:15.895 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:15.894+0000 7fb25aaad640 1 Processor -- start 2023-08-09T15:12:15.896 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:15.894+0000 7fb25aaad640 1 -- start start 2023-08-09T15:12:15.897 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:15.896+0000 7fb25aaad640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fb25406f670 0x7fb2540c9470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T15:12:15.897 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:15.896+0000 7fb25aaad640 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7fb2540c99b0 0x7fb2540afc80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T15:12:15.897 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:15.896+0000 7fb25aaad640 1 -- --> v1:172.21.15.203:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb2540701d0 con 0x7fb25415a530 2023-08-09T15:12:15.897 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:15.896+0000 7fb25aaad640 1 -- --> v2:172.21.15.203:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb25406fc10 con 0x7fb2540c99b0 2023-08-09T15:12:15.898 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:15.896+0000 7fb25aaad640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb25406fef0 con 0x7fb25406f670 2023-08-09T15:12:15.898 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:15.896+0000 7fb24bfff640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fb25406f670 0x7fb2540c9470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T15:12:15.899 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:15.897+0000 7fb24bfff640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fb25406f670 0x7fb2540c9470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.203:3301/0 says I am v2:172.21.15.43:49592/0 (socket says 172.21.15.43:49592) 2023-08-09T15:12:15.899 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:15.897+0000 7fb24bfff640 1 -- 172.21.15.43:0/1396825574 learned_addr learned my addr 172.21.15.43:0/1396825574 (peer_addr_for_me v2:172.21.15.43:0/0) 2023-08-09T15:12:15.899 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:15.897+0000 7fb25904a640 1 --2- 172.21.15.43:0/1396825574 >> v2:172.21.15.203:3300/0 conn(0x7fb2540c99b0 0x7fb2540afc80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T15:12:15.899 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:15.897+0000 7fb24bfff640 1 -- 172.21.15.43:0/1396825574 >> v2:172.21.15.203:3300/0 conn(0x7fb2540c99b0 msgr2=0x7fb2540afc80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-09T15:12:15.899 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:15.897+0000 7fb24bfff640 1 --2- 172.21.15.43:0/1396825574 >> v2:172.21.15.203:3300/0 conn(0x7fb2540c99b0 0x7fb2540afc80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T15:12:15.899 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:15.897+0000 7fb24bfff640 1 -- 172.21.15.43:0/1396825574 >> v1:172.21.15.203:6789/0 conn(0x7fb25415a530 legacy=0x7fb25415eb20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-09T15:12:15.899 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:15.897+0000 7fb24bfff640 1 -- 172.21.15.43:0/1396825574 --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb2540c8c70 con 0x7fb25406f670 2023-08-09T15:12:15.900 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:15.899+0000 7fb24bfff640 1 --2- 172.21.15.43:0/1396825574 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fb25406f670 0x7fb2540c9470 secure :-1 s=READY pgs=360 cs=0 l=1 rev1=1 crypto rx=0x7fb244009870 tx=0x7fb24402b8f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=5f2bd93df2d8ee24 server_cookie=0 in_seq=0 out_seq=0 2023-08-09T15:12:15.901 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:15.899+0000 7fb24b7fe640 1 -- 172.21.15.43:0/1396825574 <== mon.2 v2:172.21.15.203:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb244031640 con 0x7fb25406f670 2023-08-09T15:12:15.901 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:15.899+0000 7fb24b7fe640 1 -- 172.21.15.43:0/1396825574 <== mon.2 v2:172.21.15.203:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb2440317a0 con 0x7fb25406f670 2023-08-09T15:12:15.901 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:15.899+0000 7fb24b7fe640 1 -- 172.21.15.43:0/1396825574 <== mon.2 v2:172.21.15.203:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb244031b30 con 0x7fb25406f670 2023-08-09T15:12:15.901 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:15.900+0000 7fb25aaad640 1 -- 172.21.15.43:0/1396825574 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fb25406f670 msgr2=0x7fb2540c9470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T15:12:15.902 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:15.900+0000 7fb25aaad640 1 --2- 172.21.15.43:0/1396825574 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fb25406f670 0x7fb2540c9470 secure :-1 s=READY pgs=360 cs=0 l=1 rev1=1 crypto rx=0x7fb244009870 tx=0x7fb24402b8f0 comp rx=0 tx=0).stop 2023-08-09T15:12:15.902 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:15.901+0000 7fb25aaad640 1 -- 172.21.15.43:0/1396825574 shutdown_connections 2023-08-09T15:12:15.902 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:15.901+0000 7fb25aaad640 1 --2- 172.21.15.43:0/1396825574 >> v2:172.21.15.203:3300/0 conn(0x7fb2540c99b0 0x7fb2540afc80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T15:12:15.903 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:15.901+0000 7fb25aaad640 1 --2- 172.21.15.43:0/1396825574 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fb25406f670 0x7fb2540c9470 unknown :-1 s=CLOSED pgs=360 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T15:12:15.903 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:15.901+0000 7fb25aaad640 1 -- 172.21.15.43:0/1396825574 >> 172.21.15.43:0/1396825574 conn(0x7fb254069dd0 msgr2=0x7fb2540cac80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-09T15:12:15.903 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:15.901+0000 7fb25aaad640 1 -- 172.21.15.43:0/1396825574 shutdown_connections 2023-08-09T15:12:15.903 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:15.901+0000 7fb25aaad640 1 -- 172.21.15.43:0/1396825574 wait complete. 2023-08-09T15:12:15.904 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:15.903+0000 7fb25aaad640 1 Processor -- start 2023-08-09T15:12:15.905 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:15.903+0000 7fb25aaad640 1 -- start start 2023-08-09T15:12:15.906 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:15.905+0000 7fb25aaad640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb25406f670 0x7fb254150270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T15:12:15.906 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:15.905+0000 7fb25aaad640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fb2540c99b0 0x7fb2540c5c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T15:12:15.906 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:15.905+0000 7fb25aaad640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fb25415a530 0x7fb2540b1e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T15:12:15.907 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:15.905+0000 7fb25aaad640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb2541507b0 con 0x7fb25406f670 2023-08-09T15:12:15.907 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:15.905+0000 7fb25aaad640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb2540c6530 con 0x7fb25415a530 2023-08-09T15:12:15.907 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:15.905+0000 7fb25aaad640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb2540c6810 con 0x7fb2540c99b0 2023-08-09T15:12:15.907 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:15.905+0000 7fb258849640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb25406f670 0x7fb254150270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T15:12:15.907 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:15.905+0000 7fb258849640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb25406f670 0x7fb254150270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:46190/0 (socket says 172.21.15.43:46190) 2023-08-09T15:12:15.907 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:15.905+0000 7fb258849640 1 -- 172.21.15.43:0/294048304 learned_addr learned my addr 172.21.15.43:0/294048304 (peer_addr_for_me v2:172.21.15.43:0/0) 2023-08-09T15:12:15.909 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:15.905+0000 7fb25904a640 1 --2- 172.21.15.43:0/294048304 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fb25415a530 0x7fb2540b1e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T15:12:15.909 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:15.905+0000 7fb24bfff640 1 --2- 172.21.15.43:0/294048304 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fb2540c99b0 0x7fb2540c5c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T15:12:15.909 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:15.905+0000 7fb258849640 1 -- 172.21.15.43:0/294048304 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fb2540c99b0 msgr2=0x7fb2540c5c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T15:12:15.909 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:15.905+0000 7fb258849640 1 --2- 172.21.15.43:0/294048304 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fb2540c99b0 0x7fb2540c5c00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T15:12:15.909 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:15.905+0000 7fb258849640 1 -- 172.21.15.43:0/294048304 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fb25415a530 msgr2=0x7fb2540b1e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T15:12:15.909 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:15.905+0000 7fb258849640 1 --2- 172.21.15.43:0/294048304 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fb25415a530 0x7fb2540b1e70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T15:12:15.910 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:15.905+0000 7fb258849640 1 -- 172.21.15.43:0/294048304 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb2440094d0 con 0x7fb25406f670 2023-08-09T15:12:15.910 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:15.906+0000 7fb258849640 1 --2- 172.21.15.43:0/294048304 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb25406f670 0x7fb254150270 secure :-1 s=READY pgs=2543 cs=0 l=1 rev1=1 crypto rx=0x7fb25000ba50 tx=0x7fb25000bf10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-09T15:12:15.910 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:15.906+0000 7fb2497fa640 1 -- 172.21.15.43:0/294048304 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb25000cbe0 con 0x7fb25406f670 2023-08-09T15:12:15.910 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:15.906+0000 7fb2497fa640 1 -- 172.21.15.43:0/294048304 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb25000cd80 con 0x7fb25406f670 2023-08-09T15:12:15.910 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:15.906+0000 7fb25aaad640 1 -- 172.21.15.43:0/294048304 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb2540c6af0 con 0x7fb25406f670 2023-08-09T15:12:15.910 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:15.906+0000 7fb2497fa640 1 -- 172.21.15.43:0/294048304 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb250014680 con 0x7fb25406f670 2023-08-09T15:12:15.910 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:15.906+0000 7fb25aaad640 1 -- 172.21.15.43:0/294048304 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb2540c70b0 con 0x7fb25406f670 2023-08-09T15:12:15.911 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:15.907+0000 7fb25aaad640 1 -- 172.21.15.43:0/294048304 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb2540c72a0 con 0x7fb25406f670 2023-08-09T15:12:15.911 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:15.908+0000 7fb2497fa640 1 -- 172.21.15.43:0/294048304 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 129) v1 ==== 88443+0+0 (secure 0 0 0) 0x7fb2500147e0 con 0x7fb25406f670 2023-08-09T15:12:15.911 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:15.909+0000 7fb2497fa640 1 --2- 172.21.15.43:0/294048304 >> [v2:172.21.15.203:6832/1456699667,v1:172.21.15.203:6833/1456699667] conn(0x7fb24006a170 0x7fb24006c630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T15:12:15.912 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:15.909+0000 7fb2497fa640 1 -- 172.21.15.43:0/294048304 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(229..229 src has 1..229) v4 ==== 6462+0+0 (secure 0 0 0) 0x7fb250088730 con 0x7fb25406f670 2023-08-09T15:12:15.912 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:15.910+0000 7fb24bfff640 1 --2- 172.21.15.43:0/294048304 >> [v2:172.21.15.203:6832/1456699667,v1:172.21.15.203:6833/1456699667] conn(0x7fb24006a170 0x7fb24006c630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T15:12:15.913 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:15.911+0000 7fb24bfff640 1 --2- 172.21.15.43:0/294048304 >> [v2:172.21.15.203:6832/1456699667,v1:172.21.15.203:6833/1456699667] conn(0x7fb24006a170 0x7fb24006c630 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7fb244036000 tx=0x7fb24403c040 comp rx=0 tx=0).ready entity=mgr.9232 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-09T15:12:15.915 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:15.914+0000 7fb2497fa640 1 -- 172.21.15.43:0/294048304 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178066 (secure 0 0 0) 0x7fb250053c30 con 0x7fb25406f670 2023-08-09T15:12:16.121 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:16.119+0000 7fb25aaad640 1 -- 172.21.15.43:0/294048304 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "config rm", "who": "mgr", "name": "mgr/snap_schedule/dump_on_update"} v 0) v1 -- 0x7fb25404db40 con 0x7fb25406f670 2023-08-09T15:12:16.129 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:16.128+0000 7fb2497fa640 1 -- 172.21.15.43:0/294048304 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "config rm", "who": "mgr", "name": "mgr/snap_schedule/dump_on_update"}]=0 v73) v1 ==== 115+0+0 (secure 0 0 0) 0x7fb250057250 con 0x7fb25406f670 2023-08-09T15:12:16.132 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:16.131+0000 7fb25aaad640 1 -- 172.21.15.43:0/294048304 >> [v2:172.21.15.203:6832/1456699667,v1:172.21.15.203:6833/1456699667] conn(0x7fb24006a170 msgr2=0x7fb24006c630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T15:12:16.132 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:16.131+0000 7fb25aaad640 1 --2- 172.21.15.43:0/294048304 >> [v2:172.21.15.203:6832/1456699667,v1:172.21.15.203:6833/1456699667] conn(0x7fb24006a170 0x7fb24006c630 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7fb244036000 tx=0x7fb24403c040 comp rx=0 tx=0).stop 2023-08-09T15:12:16.133 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:16.131+0000 7fb25aaad640 1 -- 172.21.15.43:0/294048304 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb25406f670 msgr2=0x7fb254150270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T15:12:16.133 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:16.131+0000 7fb25aaad640 1 --2- 172.21.15.43:0/294048304 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb25406f670 0x7fb254150270 secure :-1 s=READY pgs=2543 cs=0 l=1 rev1=1 crypto rx=0x7fb25000ba50 tx=0x7fb25000bf10 comp rx=0 tx=0).stop 2023-08-09T15:12:16.133 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:16.131+0000 7fb25aaad640 1 -- 172.21.15.43:0/294048304 shutdown_connections 2023-08-09T15:12:16.134 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:16.131+0000 7fb25aaad640 1 --2- 172.21.15.43:0/294048304 >> [v2:172.21.15.203:6832/1456699667,v1:172.21.15.203:6833/1456699667] conn(0x7fb24006a170 0x7fb24006c630 unknown :-1 s=CLOSED pgs=35 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T15:12:16.134 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:16.131+0000 7fb25aaad640 1 --2- 172.21.15.43:0/294048304 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fb25415a530 0x7fb2540b1e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T15:12:16.134 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:16.131+0000 7fb25aaad640 1 --2- 172.21.15.43:0/294048304 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fb2540c99b0 0x7fb2540c5c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T15:12:16.134 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:16.131+0000 7fb25aaad640 1 --2- 172.21.15.43:0/294048304 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb25406f670 0x7fb254150270 unknown :-1 s=CLOSED pgs=2543 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T15:12:16.134 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:16.131+0000 7fb25aaad640 1 -- 172.21.15.43:0/294048304 >> 172.21.15.43:0/294048304 conn(0x7fb254069dd0 msgr2=0x7fb2540b2ff0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-09T15:12:16.134 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:16.131+0000 7fb25aaad640 1 -- 172.21.15.43:0/294048304 shutdown_connections 2023-08-09T15:12:16.134 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:16.132+0000 7fb25aaad640 1 -- 172.21.15.43:0/294048304 wait complete. 2023-08-09T15:12:16.153 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config rm mgr mgr/snap_schedule/allow_m_granularity 2023-08-09T15:12:16.384 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:16.383+0000 7f2aa740d640 1 Processor -- start 2023-08-09T15:12:16.384 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:16.383+0000 7f2aa740d640 1 -- start start 2023-08-09T15:12:16.386 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:16.384+0000 7f2aa740d640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f2aa015c880 0x7f2aa00ccbc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T15:12:16.386 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:16.385+0000 7f2aa740d640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f2aa00cf390 0x7f2aa00d0a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T15:12:16.386 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:16.385+0000 7f2aa740d640 1 -- --> v1:172.21.15.203:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2aa015cc60 con 0x7f2aa0160f20 2023-08-09T15:12:16.386 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:16.385+0000 7f2aa740d640 1 -- --> v2:172.21.15.43:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2aa00d0530 con 0x7f2aa015c880 2023-08-09T15:12:16.387 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:16.385+0000 7f2aa740d640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2aa00cd410 con 0x7f2aa00cf390 2023-08-09T15:12:16.387 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:16.385+0000 7f2aa51a9640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f2aa015c880 0x7f2aa00ccbc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T15:12:16.388 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:16.385+0000 7f2aa51a9640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f2aa015c880 0x7f2aa00ccbc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:46196/0 (socket says 172.21.15.43:46196) 2023-08-09T15:12:16.388 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:16.385+0000 7f2aa51a9640 1 -- 172.21.15.43:0/271441438 learned_addr learned my addr 172.21.15.43:0/271441438 (peer_addr_for_me v2:172.21.15.43:0/0) 2023-08-09T15:12:16.388 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:16.385+0000 7f2aa59aa640 1 --2- 172.21.15.43:0/271441438 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f2aa00cf390 0x7f2aa00d0a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T15:12:16.388 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:16.386+0000 7f2aa51a9640 1 -- 172.21.15.43:0/271441438 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f2aa00cf390 msgr2=0x7f2aa00d0a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-09T15:12:16.388 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:16.386+0000 7f2aa51a9640 1 --2- 172.21.15.43:0/271441438 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f2aa00cf390 0x7f2aa00d0a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T15:12:16.388 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:16.386+0000 7f2aa51a9640 1 -- 172.21.15.43:0/271441438 >> v1:172.21.15.203:6789/0 conn(0x7f2aa0160f20 legacy=0x7f2aa00cec80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-09T15:12:16.388 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:16.386+0000 7f2aa51a9640 1 -- 172.21.15.43:0/271441438 --> v2:172.21.15.43:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2aa00cd6f0 con 0x7f2aa015c880 2023-08-09T15:12:16.389 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:16.386+0000 7f2aa59aa640 1 --2- 172.21.15.43:0/271441438 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f2aa00cf390 0x7f2aa00d0a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-08-09T15:12:16.389 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:16.387+0000 7f2aa51a9640 1 --2- 172.21.15.43:0/271441438 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f2aa015c880 0x7f2aa00ccbc0 secure :-1 s=READY pgs=2544 cs=0 l=1 rev1=1 crypto rx=0x7f2a9c009850 tx=0x7f2a9c02b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bcd5e25108a5ad4d server_cookie=0 in_seq=0 out_seq=0 2023-08-09T15:12:16.389 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:16.387+0000 7f2a97fff640 1 -- 172.21.15.43:0/271441438 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2a9c004230 con 0x7f2aa015c880 2023-08-09T15:12:16.390 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:16.388+0000 7f2a97fff640 1 -- 172.21.15.43:0/271441438 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2a9c004390 con 0x7f2aa015c880 2023-08-09T15:12:16.390 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:16.388+0000 7f2a97fff640 1 -- 172.21.15.43:0/271441438 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2a9c031830 con 0x7f2aa015c880 2023-08-09T15:12:16.390 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:16.388+0000 7f2aa740d640 1 -- 172.21.15.43:0/271441438 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f2aa015c880 msgr2=0x7f2aa00ccbc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T15:12:16.391 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:16.388+0000 7f2aa740d640 1 --2- 172.21.15.43:0/271441438 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f2aa015c880 0x7f2aa00ccbc0 secure :-1 s=READY pgs=2544 cs=0 l=1 rev1=1 crypto rx=0x7f2a9c009850 tx=0x7f2a9c02b8d0 comp rx=0 tx=0).stop 2023-08-09T15:12:16.391 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:16.388+0000 7f2aa740d640 1 -- 172.21.15.43:0/271441438 shutdown_connections 2023-08-09T15:12:16.391 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:16.388+0000 7f2aa740d640 1 --2- 172.21.15.43:0/271441438 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f2aa00cf390 0x7f2aa00d0a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T15:12:16.391 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:16.388+0000 7f2aa740d640 1 --2- 172.21.15.43:0/271441438 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f2aa015c880 0x7f2aa00ccbc0 unknown :-1 s=CLOSED pgs=2544 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T15:12:16.391 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:16.388+0000 7f2aa740d640 1 -- 172.21.15.43:0/271441438 >> 172.21.15.43:0/271441438 conn(0x7f2aa008bfb0 msgr2=0x7f2aa009b8d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-09T15:12:16.391 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:16.388+0000 7f2aa740d640 1 -- 172.21.15.43:0/271441438 shutdown_connections 2023-08-09T15:12:16.391 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:16.389+0000 7f2aa740d640 1 -- 172.21.15.43:0/271441438 wait complete. 2023-08-09T15:12:16.392 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:16.391+0000 7f2aa740d640 1 Processor -- start 2023-08-09T15:12:16.392 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:16.391+0000 7f2aa740d640 1 -- start start 2023-08-09T15:12:16.394 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:16.392+0000 7f2aa740d640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f2aa00cf390 0x7f2aa0157ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T15:12:16.394 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:16.392+0000 7f2aa740d640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f2aa015c880 0x7f2aa00c70f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T15:12:16.394 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:16.392+0000 7f2aa740d640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f2aa0160f20 0x7f2aa00c7630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T15:12:16.395 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:16.392+0000 7f2aa740d640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2aa0158010 con 0x7f2aa0160f20 2023-08-09T15:12:16.395 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:16.392+0000 7f2aa740d640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2aa00b1ee0 con 0x7f2aa015c880 2023-08-09T15:12:16.395 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:16.392+0000 7f2aa740d640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2aa00b2180 con 0x7f2aa00cf390 2023-08-09T15:12:16.395 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:16.393+0000 7f2aa59aa640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f2aa0160f20 0x7f2aa00c7630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T15:12:16.395 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:16.393+0000 7f2aa59aa640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f2aa0160f20 0x7f2aa00c7630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:46198/0 (socket says 172.21.15.43:46198) 2023-08-09T15:12:16.395 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:16.393+0000 7f2aa59aa640 1 -- 172.21.15.43:0/846546540 learned_addr learned my addr 172.21.15.43:0/846546540 (peer_addr_for_me v2:172.21.15.43:0/0) 2023-08-09T15:12:16.395 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:16.393+0000 7f2aa51a9640 1 --2- 172.21.15.43:0/846546540 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f2aa00cf390 0x7f2aa0157ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T15:12:16.396 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:16.393+0000 7f2aa49a8640 1 --2- 172.21.15.43:0/846546540 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f2aa015c880 0x7f2aa00c70f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T15:12:16.397 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:16.393+0000 7f2aa59aa640 1 -- 172.21.15.43:0/846546540 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f2aa00cf390 msgr2=0x7f2aa0157ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T15:12:16.397 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:16.393+0000 7f2aa59aa640 1 --2- 172.21.15.43:0/846546540 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f2aa00cf390 0x7f2aa0157ad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T15:12:16.397 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:16.393+0000 7f2aa59aa640 1 -- 172.21.15.43:0/846546540 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f2aa015c880 msgr2=0x7f2aa00c70f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T15:12:16.397 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:16.393+0000 7f2aa59aa640 1 --2- 172.21.15.43:0/846546540 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f2aa015c880 0x7f2aa00c70f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T15:12:16.397 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:16.393+0000 7f2aa59aa640 1 -- 172.21.15.43:0/846546540 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2a9c009460 con 0x7f2aa0160f20 2023-08-09T15:12:16.397 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:16.393+0000 7f2aa49a8640 1 --2- 172.21.15.43:0/846546540 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f2aa015c880 0x7f2aa00c70f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-08-09T15:12:16.397 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:16.394+0000 7f2aa59aa640 1 --2- 172.21.15.43:0/846546540 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f2aa0160f20 0x7f2aa00c7630 secure :-1 s=READY pgs=2545 cs=0 l=1 rev1=1 crypto rx=0x7f2a9800cd80 tx=0x7f2a9800a650 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-09T15:12:16.398 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:16.394+0000 7f2a95ffb640 1 -- 172.21.15.43:0/846546540 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2a98003b90 con 0x7f2aa0160f20 2023-08-09T15:12:16.398 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:16.394+0000 7f2a95ffb640 1 -- 172.21.15.43:0/846546540 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2a98003d10 con 0x7f2aa0160f20 2023-08-09T15:12:16.398 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:16.394+0000 7f2a95ffb640 1 -- 172.21.15.43:0/846546540 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2a98004260 con 0x7f2aa0160f20 2023-08-09T15:12:16.398 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:16.394+0000 7f2aa740d640 1 -- 172.21.15.43:0/846546540 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2aa00b2460 con 0x7f2aa0160f20 2023-08-09T15:12:16.398 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:16.394+0000 7f2aa740d640 1 -- 172.21.15.43:0/846546540 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2aa00b2a20 con 0x7f2aa0160f20 2023-08-09T15:12:16.399 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:16.396+0000 7f2aa740d640 1 -- 172.21.15.43:0/846546540 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2aa00b2c40 con 0x7f2aa0160f20 2023-08-09T15:12:16.399 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:16.397+0000 7f2a95ffb640 1 -- 172.21.15.43:0/846546540 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 129) v1 ==== 88443+0+0 (secure 0 0 0) 0x7f2a9800b040 con 0x7f2aa0160f20 2023-08-09T15:12:16.399 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:16.398+0000 7f2a95ffb640 1 --2- 172.21.15.43:0/846546540 >> [v2:172.21.15.203:6832/1456699667,v1:172.21.15.203:6833/1456699667] conn(0x7f2a8406a170 0x7f2a8406c630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T15:12:16.399 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:16.398+0000 7f2a95ffb640 1 -- 172.21.15.43:0/846546540 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(229..229 src has 1..229) v4 ==== 6462+0+0 (secure 0 0 0) 0x7f2a980890c0 con 0x7f2aa0160f20 2023-08-09T15:12:16.400 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:16.398+0000 7f2aa51a9640 1 --2- 172.21.15.43:0/846546540 >> [v2:172.21.15.203:6832/1456699667,v1:172.21.15.203:6833/1456699667] conn(0x7f2a8406a170 0x7f2a8406c630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T15:12:16.401 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:16.400+0000 7f2aa51a9640 1 --2- 172.21.15.43:0/846546540 >> [v2:172.21.15.203:6832/1456699667,v1:172.21.15.203:6833/1456699667] conn(0x7f2a8406a170 0x7f2a8406c630 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7f2a9c035000 tx=0x7f2a9c008030 comp rx=0 tx=0).ready entity=mgr.9232 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-09T15:12:16.403 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:16.402+0000 7f2a95ffb640 1 -- 172.21.15.43:0/846546540 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178066 (secure 0 0 0) 0x7f2a98054360 con 0x7f2aa0160f20 2023-08-09T15:12:16.595 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:16.593+0000 7f2aa740d640 1 -- 172.21.15.43:0/846546540 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "config rm", "who": "mgr", "name": "mgr/snap_schedule/allow_m_granularity"} v 0) v1 -- 0x7f2aa00cdef0 con 0x7f2aa0160f20 2023-08-09T15:12:16.604 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:16.603+0000 7f2a95ffb640 1 -- 172.21.15.43:0/846546540 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "config rm", "who": "mgr", "name": "mgr/snap_schedule/allow_m_granularity"}]=0 v74) v1 ==== 120+0+0 (secure 0 0 0) 0x7f2a98003e80 con 0x7f2aa0160f20 2023-08-09T15:12:16.607 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:16.606+0000 7f2aa740d640 1 -- 172.21.15.43:0/846546540 >> [v2:172.21.15.203:6832/1456699667,v1:172.21.15.203:6833/1456699667] conn(0x7f2a8406a170 msgr2=0x7f2a8406c630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T15:12:16.607 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:16.606+0000 7f2aa740d640 1 --2- 172.21.15.43:0/846546540 >> [v2:172.21.15.203:6832/1456699667,v1:172.21.15.203:6833/1456699667] conn(0x7f2a8406a170 0x7f2a8406c630 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7f2a9c035000 tx=0x7f2a9c008030 comp rx=0 tx=0).stop 2023-08-09T15:12:16.607 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:16.606+0000 7f2aa740d640 1 -- 172.21.15.43:0/846546540 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f2aa0160f20 msgr2=0x7f2aa00c7630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T15:12:16.608 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:16.606+0000 7f2aa740d640 1 --2- 172.21.15.43:0/846546540 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f2aa0160f20 0x7f2aa00c7630 secure :-1 s=READY pgs=2545 cs=0 l=1 rev1=1 crypto rx=0x7f2a9800cd80 tx=0x7f2a9800a650 comp rx=0 tx=0).stop 2023-08-09T15:12:16.608 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:16.606+0000 7f2aa740d640 1 -- 172.21.15.43:0/846546540 shutdown_connections 2023-08-09T15:12:16.609 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:16.606+0000 7f2aa740d640 1 --2- 172.21.15.43:0/846546540 >> [v2:172.21.15.203:6832/1456699667,v1:172.21.15.203:6833/1456699667] conn(0x7f2a8406a170 0x7f2a8406c630 unknown :-1 s=CLOSED pgs=36 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T15:12:16.609 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:16.606+0000 7f2aa740d640 1 --2- 172.21.15.43:0/846546540 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f2aa0160f20 0x7f2aa00c7630 unknown :-1 s=CLOSED pgs=2545 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T15:12:16.609 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:16.606+0000 7f2aa740d640 1 --2- 172.21.15.43:0/846546540 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f2aa015c880 0x7f2aa00c70f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T15:12:16.609 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:16.606+0000 7f2aa740d640 1 --2- 172.21.15.43:0/846546540 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f2aa00cf390 0x7f2aa0157ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T15:12:16.609 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:16.606+0000 7f2aa740d640 1 -- 172.21.15.43:0/846546540 >> 172.21.15.43:0/846546540 conn(0x7f2aa008bfb0 msgr2=0x7f2aa0160180 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-09T15:12:16.609 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:16.606+0000 7f2aa740d640 1 -- 172.21.15.43:0/846546540 shutdown_connections 2023-08-09T15:12:16.609 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:16.607+0000 7f2aa740d640 1 -- 172.21.15.43:0/846546540 wait complete. 2023-08-09T15:12:16.628 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config rm mon mon_allow_pool_delete 2023-08-09T15:12:16.866 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:16.864+0000 7f89af02c640 1 Processor -- start 2023-08-09T15:12:16.866 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:16.865+0000 7f89af02c640 1 -- start start 2023-08-09T15:12:16.868 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:16.866+0000 7f89af02c640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f89a80c3670 0x7f89a806eff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T15:12:16.868 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:16.867+0000 7f89af02c640 1 -- --> v1:172.21.15.43:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f89a806f840 con 0x7f89a806dae0 2023-08-09T15:12:16.868 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:16.867+0000 7f89af02c640 1 -- --> v1:172.21.15.203:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f89a806fb20 con 0x7f89a80cfc60 2023-08-09T15:12:16.868 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:16.867+0000 7f89af02c640 1 -- --> v2:172.21.15.43:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f89a806c5e0 con 0x7f89a80c3670 2023-08-09T15:12:16.869 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:16.867+0000 7f89ae02a640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f89a80c3670 0x7f89a806eff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T15:12:16.869 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:16.867+0000 7f89ae02a640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f89a80c3670 0x7f89a806eff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:46200/0 (socket says 172.21.15.43:46200) 2023-08-09T15:12:16.869 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:16.867+0000 7f89ae02a640 1 -- 172.21.15.43:0/851410894 learned_addr learned my addr 172.21.15.43:0/851410894 (peer_addr_for_me v2:172.21.15.43:0/0) 2023-08-09T15:12:16.869 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:16.867+0000 7f89ae82b640 1 --1- >> v1:172.21.15.43:6789/0 conn(0x7f89a806dae0 0x7f89a8164630 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.43:6789/0 says I am v1:172.21.15.43:60992/0 (socket says 172.21.15.43:60992) 2023-08-09T15:12:16.870 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:16.867+0000 7f89ae02a640 1 -- 172.21.15.43:0/851410894 >> v1:172.21.15.203:6789/0 conn(0x7f89a80cfc60 legacy=0x7f89a8161950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-09T15:12:16.870 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:16.867+0000 7f89ae02a640 1 -- 172.21.15.43:0/851410894 >> v1:172.21.15.43:6789/0 conn(0x7f89a806dae0 legacy=0x7f89a8164630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T15:12:16.870 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:16.867+0000 7f89ae02a640 1 -- 172.21.15.43:0/851410894 --> v2:172.21.15.43:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f89a806fe00 con 0x7f89a80c3670 2023-08-09T15:12:16.871 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:16.869+0000 7f89ae02a640 1 --2- 172.21.15.43:0/851410894 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f89a80c3670 0x7f89a806eff0 secure :-1 s=READY pgs=2547 cs=0 l=1 rev1=1 crypto rx=0x7f89a4009870 tx=0x7f89a402b910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=573ff3f6f258e1a server_cookie=0 in_seq=0 out_seq=0 2023-08-09T15:12:16.871 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:16.869+0000 7f89ad028640 1 -- 172.21.15.43:0/851410894 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f89a4031870 con 0x7f89a80c3670 2023-08-09T15:12:16.871 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:16.869+0000 7f89ad028640 1 -- 172.21.15.43:0/851410894 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f89a40319d0 con 0x7f89a80c3670 2023-08-09T15:12:16.871 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:16.869+0000 7f89ad028640 1 -- 172.21.15.43:0/851410894 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f89a4031d60 con 0x7f89a80c3670 2023-08-09T15:12:16.871 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:16.869+0000 7f89af02c640 1 -- 172.21.15.43:0/851410894 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f89a80c3670 msgr2=0x7f89a806eff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T15:12:16.871 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:16.870+0000 7f89af02c640 1 --2- 172.21.15.43:0/851410894 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f89a80c3670 0x7f89a806eff0 secure :-1 s=READY pgs=2547 cs=0 l=1 rev1=1 crypto rx=0x7f89a4009870 tx=0x7f89a402b910 comp rx=0 tx=0).stop 2023-08-09T15:12:16.872 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:16.870+0000 7f89af02c640 1 -- 172.21.15.43:0/851410894 shutdown_connections 2023-08-09T15:12:16.872 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:16.870+0000 7f89af02c640 1 --2- 172.21.15.43:0/851410894 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f89a80c3670 0x7f89a806eff0 unknown :-1 s=CLOSED pgs=2547 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T15:12:16.872 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:16.870+0000 7f89af02c640 1 -- 172.21.15.43:0/851410894 >> 172.21.15.43:0/851410894 conn(0x7f89a80d0210 msgr2=0x7f89a80d0620 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-09T15:12:16.872 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:16.870+0000 7f89af02c640 1 -- 172.21.15.43:0/851410894 shutdown_connections 2023-08-09T15:12:16.872 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:16.870+0000 7f89af02c640 1 -- 172.21.15.43:0/851410894 wait complete. 2023-08-09T15:12:16.873 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:16.871+0000 7f89af02c640 1 Processor -- start 2023-08-09T15:12:16.873 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:16.872+0000 7f89af02c640 1 -- start start 2023-08-09T15:12:16.874 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:16.873+0000 7f89af02c640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f89a806dae0 0x7f89a80c67e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T15:12:16.874 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:16.873+0000 7f89af02c640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f89a80c3670 0x7f89a815eeb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T15:12:16.875 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:16.873+0000 7f89af02c640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f89a80cfc60 0x7f89a80b50b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T15:12:16.875 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:16.873+0000 7f89af02c640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f89a80b4a90 con 0x7f89a806dae0 2023-08-09T15:12:16.875 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:16.873+0000 7f89af02c640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f89a80c5750 con 0x7f89a80c3670 2023-08-09T15:12:16.875 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:16.873+0000 7f89af02c640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f89a80c5a10 con 0x7f89a80cfc60 2023-08-09T15:12:16.875 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:16.873+0000 7f89ae02a640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f89a806dae0 0x7f89a80c67e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T15:12:16.875 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:16.873+0000 7f89ae02a640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f89a806dae0 0x7f89a80c67e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:46216/0 (socket says 172.21.15.43:46216) 2023-08-09T15:12:16.875 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:16.873+0000 7f89ae02a640 1 -- 172.21.15.43:0/746463932 learned_addr learned my addr 172.21.15.43:0/746463932 (peer_addr_for_me v2:172.21.15.43:0/0) 2023-08-09T15:12:16.876 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:16.873+0000 7f89ae02a640 1 -- 172.21.15.43:0/746463932 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f89a80cfc60 msgr2=0x7f89a80b50b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T15:12:16.877 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:16.873+0000 7f89ae02a640 1 --2- 172.21.15.43:0/746463932 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f89a80cfc60 0x7f89a80b50b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T15:12:16.877 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:16.873+0000 7f89ae02a640 1 -- 172.21.15.43:0/746463932 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f89a80c3670 msgr2=0x7f89a815eeb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T15:12:16.877 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:16.873+0000 7f89ae02a640 1 --2- 172.21.15.43:0/746463932 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f89a80c3670 0x7f89a815eeb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T15:12:16.877 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:16.874+0000 7f89ae02a640 1 -- 172.21.15.43:0/746463932 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f89a40094d0 con 0x7f89a806dae0 2023-08-09T15:12:16.877 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:16.874+0000 7f89ae02a640 1 --2- 172.21.15.43:0/746463932 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f89a806dae0 0x7f89a80c67e0 secure :-1 s=READY pgs=2548 cs=0 l=1 rev1=1 crypto rx=0x7f89a4009870 tx=0x7f89a40046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-09T15:12:16.878 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:16.874+0000 7f899effd640 1 -- 172.21.15.43:0/746463932 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f89a4031820 con 0x7f89a806dae0 2023-08-09T15:12:16.878 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:16.874+0000 7f899effd640 1 -- 172.21.15.43:0/746463932 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f89a40443f0 con 0x7f89a806dae0 2023-08-09T15:12:16.878 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:16.874+0000 7f89af02c640 1 -- 172.21.15.43:0/746463932 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f89a80c5cf0 con 0x7f89a806dae0 2023-08-09T15:12:16.878 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:16.874+0000 7f899effd640 1 -- 172.21.15.43:0/746463932 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f89a40447a0 con 0x7f89a806dae0 2023-08-09T15:12:16.878 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:16.874+0000 7f89af02c640 1 -- 172.21.15.43:0/746463932 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f89a80c7380 con 0x7f89a806dae0 2023-08-09T15:12:16.879 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:16.876+0000 7f89af02c640 1 -- 172.21.15.43:0/746463932 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f89a8162010 con 0x7f89a806dae0 2023-08-09T15:12:16.879 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:16.877+0000 7f899effd640 1 -- 172.21.15.43:0/746463932 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 129) v1 ==== 88443+0+0 (secure 0 0 0) 0x7f89a404d450 con 0x7f89a806dae0 2023-08-09T15:12:16.879 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:16.877+0000 7f899effd640 1 --2- 172.21.15.43:0/746463932 >> [v2:172.21.15.203:6832/1456699667,v1:172.21.15.203:6833/1456699667] conn(0x7f898c06a170 0x7f898c06c630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T15:12:16.879 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:16.878+0000 7f899effd640 1 -- 172.21.15.43:0/746463932 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(229..229 src has 1..229) v4 ==== 6462+0+0 (secure 0 0 0) 0x7f89a402d040 con 0x7f89a806dae0 2023-08-09T15:12:16.880 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:16.878+0000 7f89ad829640 1 --2- 172.21.15.43:0/746463932 >> [v2:172.21.15.203:6832/1456699667,v1:172.21.15.203:6833/1456699667] conn(0x7f898c06a170 0x7f898c06c630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T15:12:16.880 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:16.879+0000 7f89ad829640 1 --2- 172.21.15.43:0/746463932 >> [v2:172.21.15.203:6832/1456699667,v1:172.21.15.203:6833/1456699667] conn(0x7f898c06a170 0x7f898c06c630 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7f8998006fd0 tx=0x7f8998006bc0 comp rx=0 tx=0).ready entity=mgr.9232 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-09T15:12:16.883 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:16.882+0000 7f899effd640 1 -- 172.21.15.43:0/746463932 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178066 (secure 0 0 0) 0x7f89a40dd7b0 con 0x7f89a806dae0 2023-08-09T15:12:17.079 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:17.077+0000 7f89af02c640 1 -- 172.21.15.43:0/746463932 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"} v 0) v1 -- 0x7f89a80c7570 con 0x7f89a806dae0 2023-08-09T15:12:17.085 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:17.084+0000 7f899effd640 1 -- 172.21.15.43:0/746463932 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"}]=0 v75) v1 ==== 104+0+0 (secure 0 0 0) 0x7f89a40764d0 con 0x7f89a806dae0 2023-08-09T15:12:17.088 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:17.087+0000 7f89af02c640 1 -- 172.21.15.43:0/746463932 >> [v2:172.21.15.203:6832/1456699667,v1:172.21.15.203:6833/1456699667] conn(0x7f898c06a170 msgr2=0x7f898c06c630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T15:12:17.089 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:17.087+0000 7f89af02c640 1 --2- 172.21.15.43:0/746463932 >> [v2:172.21.15.203:6832/1456699667,v1:172.21.15.203:6833/1456699667] conn(0x7f898c06a170 0x7f898c06c630 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7f8998006fd0 tx=0x7f8998006bc0 comp rx=0 tx=0).stop 2023-08-09T15:12:17.089 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:17.087+0000 7f89af02c640 1 -- 172.21.15.43:0/746463932 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f89a806dae0 msgr2=0x7f89a80c67e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T15:12:17.089 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:17.087+0000 7f89af02c640 1 --2- 172.21.15.43:0/746463932 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f89a806dae0 0x7f89a80c67e0 secure :-1 s=READY pgs=2548 cs=0 l=1 rev1=1 crypto rx=0x7f89a4009870 tx=0x7f89a40046c0 comp rx=0 tx=0).stop 2023-08-09T15:12:17.089 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:17.087+0000 7f89af02c640 1 -- 172.21.15.43:0/746463932 shutdown_connections 2023-08-09T15:12:17.089 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:17.087+0000 7f89af02c640 1 --2- 172.21.15.43:0/746463932 >> [v2:172.21.15.203:6832/1456699667,v1:172.21.15.203:6833/1456699667] conn(0x7f898c06a170 0x7f898c06c630 unknown :-1 s=CLOSED pgs=37 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T15:12:17.089 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:17.087+0000 7f89af02c640 1 --2- 172.21.15.43:0/746463932 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f89a80cfc60 0x7f89a80b50b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T15:12:17.090 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:17.087+0000 7f89af02c640 1 --2- 172.21.15.43:0/746463932 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f89a80c3670 0x7f89a815eeb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T15:12:17.090 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:17.087+0000 7f89af02c640 1 --2- 172.21.15.43:0/746463932 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f89a806dae0 0x7f89a80c67e0 unknown :-1 s=CLOSED pgs=2548 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T15:12:17.090 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:17.087+0000 7f89af02c640 1 -- 172.21.15.43:0/746463932 >> 172.21.15.43:0/746463932 conn(0x7f89a80d0210 msgr2=0x7f89a80ce110 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-09T15:12:17.090 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:17.087+0000 7f89af02c640 1 -- 172.21.15.43:0/746463932 shutdown_connections 2023-08-09T15:12:17.090 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:17.087+0000 7f89af02c640 1 -- 172.21.15.43:0/746463932 wait complete. 2023-08-09T15:12:17.108 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Ended test tasks.cephfs.test_snap_schedules.TestSnapSchedulesSnapdir.test_snap_dir_name' 2023-08-09T15:12:17.333 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:17.332+0000 7febbdb86640 1 Processor -- start 2023-08-09T15:12:17.333 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:17.332+0000 7febbdb86640 1 -- start start 2023-08-09T15:12:17.335 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:17.333+0000 7febbdb86640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7febb80cfda0 0x7febb815cb10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T15:12:17.335 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:17.334+0000 7febbdb86640 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7febb806a850 0x7febb806ace0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T15:12:17.335 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:17.334+0000 7febbdb86640 1 -- --> v1:172.21.15.203:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7febb8070100 con 0x7febb80b1cc0 2023-08-09T15:12:17.335 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:17.334+0000 7febbdb86640 1 -- --> v2:172.21.15.203:3300/0 -- mon_getmap magic: 0 v1 -- 0x7febb806f2a0 con 0x7febb806a850 2023-08-09T15:12:17.335 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:17.334+0000 7febbdb86640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7febb806f580 con 0x7febb80cfda0 2023-08-09T15:12:17.336 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:17.335+0000 7febb7fff640 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7febb806a850 0x7febb806ace0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T15:12:17.336 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:17.335+0000 7febb7fff640 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7febb806a850 0x7febb806ace0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.203:3300/0 says I am v2:172.21.15.43:32778/0 (socket says 172.21.15.43:32778) 2023-08-09T15:12:17.336 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:17.335+0000 7febbcb84640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7febb80cfda0 0x7febb815cb10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T15:12:17.337 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:17.335+0000 7febb7fff640 1 -- 172.21.15.43:0/1027314842 learned_addr learned my addr 172.21.15.43:0/1027314842 (peer_addr_for_me v2:172.21.15.43:0/0) 2023-08-09T15:12:17.337 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:17.335+0000 7febb7fff640 1 -- 172.21.15.43:0/1027314842 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7febb80cfda0 msgr2=0x7febb815cb10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-09T15:12:17.337 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:17.335+0000 7febb7fff640 1 --2- 172.21.15.43:0/1027314842 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7febb80cfda0 0x7febb815cb10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T15:12:17.337 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:17.335+0000 7febb7fff640 1 -- 172.21.15.43:0/1027314842 >> v1:172.21.15.203:6789/0 conn(0x7febb80b1cc0 legacy=0x7febb80b20d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T15:12:17.338 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:17.335+0000 7febb7fff640 1 -- 172.21.15.43:0/1027314842 --> v2:172.21.15.203:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7febb806f860 con 0x7febb806a850 2023-08-09T15:12:17.338 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:17.337+0000 7febb7fff640 1 --2- 172.21.15.43:0/1027314842 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7febb806a850 0x7febb806ace0 secure :-1 s=READY pgs=933 cs=0 l=1 rev1=1 crypto rx=0x7feba8009870 tx=0x7feba802b8f0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=c0218089ae78d9d0 server_cookie=0 in_seq=0 out_seq=0 2023-08-09T15:12:17.340 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:17.337+0000 7febb77fe640 1 -- 172.21.15.43:0/1027314842 <== mon.1 v2:172.21.15.203:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feba8031640 con 0x7febb806a850 2023-08-09T15:12:17.340 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:17.337+0000 7febb77fe640 1 -- 172.21.15.43:0/1027314842 <== mon.1 v2:172.21.15.203:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7feba80317a0 con 0x7febb806a850 2023-08-09T15:12:17.341 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:17.337+0000 7febb77fe640 1 -- 172.21.15.43:0/1027314842 <== mon.1 v2:172.21.15.203:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feba8031b30 con 0x7febb806a850 2023-08-09T15:12:17.341 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:17.338+0000 7febbdb86640 1 -- 172.21.15.43:0/1027314842 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7febb806a850 msgr2=0x7febb806ace0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T15:12:17.341 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:17.338+0000 7febbdb86640 1 --2- 172.21.15.43:0/1027314842 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7febb806a850 0x7febb806ace0 secure :-1 s=READY pgs=933 cs=0 l=1 rev1=1 crypto rx=0x7feba8009870 tx=0x7feba802b8f0 comp rx=0 tx=0).stop 2023-08-09T15:12:17.341 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:17.338+0000 7febbdb86640 1 -- 172.21.15.43:0/1027314842 shutdown_connections 2023-08-09T15:12:17.342 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:17.338+0000 7febbdb86640 1 --2- 172.21.15.43:0/1027314842 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7febb806a850 0x7febb806ace0 unknown :-1 s=CLOSED pgs=933 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T15:12:17.342 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:17.338+0000 7febbdb86640 1 --2- 172.21.15.43:0/1027314842 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7febb80cfda0 0x7febb815cb10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T15:12:17.343 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:17.338+0000 7febbdb86640 1 -- 172.21.15.43:0/1027314842 >> 172.21.15.43:0/1027314842 conn(0x7febb806a190 msgr2=0x7febb80cccc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-09T15:12:17.343 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:17.338+0000 7febbdb86640 1 -- 172.21.15.43:0/1027314842 shutdown_connections 2023-08-09T15:12:17.343 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:17.338+0000 7febbdb86640 1 -- 172.21.15.43:0/1027314842 wait complete. 2023-08-09T15:12:17.346 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:17.340+0000 7febbdb86640 1 Processor -- start 2023-08-09T15:12:17.346 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:17.340+0000 7febbdb86640 1 -- start start 2023-08-09T15:12:17.346 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:17.341+0000 7febbdb86640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7febb806a850 0x7febb80b4140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T15:12:17.346 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:17.341+0000 7febbdb86640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7febb80b1cc0 0x7febb814a4f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T15:12:17.346 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:17.341+0000 7febbdb86640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7febb80cfda0 0x7febb80b58a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T15:12:17.347 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:17.341+0000 7febbdb86640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7febb80b5410 con 0x7febb80b1cc0 2023-08-09T15:12:17.347 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:17.341+0000 7febbdb86640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7febb814afa0 con 0x7febb80cfda0 2023-08-09T15:12:17.347 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:17.341+0000 7febbdb86640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7febb814aa30 con 0x7febb806a850 2023-08-09T15:12:17.347 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:17.341+0000 7febb7fff640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7febb80b1cc0 0x7febb814a4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T15:12:17.347 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:17.341+0000 7febb7fff640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7febb80b1cc0 0x7febb814a4f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:46222/0 (socket says 172.21.15.43:46222) 2023-08-09T15:12:17.347 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:17.341+0000 7febb7fff640 1 -- 172.21.15.43:0/922439427 learned_addr learned my addr 172.21.15.43:0/922439427 (peer_addr_for_me v2:172.21.15.43:0/0) 2023-08-09T15:12:17.347 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:17.342+0000 7febbcb84640 1 --2- 172.21.15.43:0/922439427 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7febb806a850 0x7febb80b4140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T15:12:17.348 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:17.342+0000 7febb7fff640 1 -- 172.21.15.43:0/922439427 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7febb806a850 msgr2=0x7febb80b4140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T15:12:17.348 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:17.342+0000 7febbd385640 1 --2- 172.21.15.43:0/922439427 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7febb80cfda0 0x7febb80b58a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T15:12:17.348 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:17.342+0000 7febb7fff640 1 --2- 172.21.15.43:0/922439427 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7febb806a850 0x7febb80b4140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T15:12:17.348 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:17.342+0000 7febb7fff640 1 -- 172.21.15.43:0/922439427 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7febb80cfda0 msgr2=0x7febb80b58a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T15:12:17.348 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:17.342+0000 7febb7fff640 1 --2- 172.21.15.43:0/922439427 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7febb80cfda0 0x7febb80b58a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T15:12:17.348 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:17.342+0000 7febb7fff640 1 -- 172.21.15.43:0/922439427 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7feba80094d0 con 0x7febb80b1cc0 2023-08-09T15:12:17.348 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:17.342+0000 7febbd385640 1 --2- 172.21.15.43:0/922439427 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7febb80cfda0 0x7febb80b58a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-08-09T15:12:17.349 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:17.342+0000 7febb7fff640 1 --2- 172.21.15.43:0/922439427 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7febb80b1cc0 0x7febb814a4f0 secure :-1 s=READY pgs=2549 cs=0 l=1 rev1=1 crypto rx=0x7feba8009990 tx=0x7feba80046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-09T15:12:17.349 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:17.343+0000 7febb57fa640 1 -- 172.21.15.43:0/922439427 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feba803a9f0 con 0x7febb80b1cc0 2023-08-09T15:12:17.349 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:17.343+0000 7febb57fa640 1 -- 172.21.15.43:0/922439427 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7feba803ab90 con 0x7febb80b1cc0 2023-08-09T15:12:17.349 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:17.343+0000 7febb57fa640 1 -- 172.21.15.43:0/922439427 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feba80443f0 con 0x7febb80b1cc0 2023-08-09T15:12:17.349 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:17.343+0000 7febbdb86640 1 -- 172.21.15.43:0/922439427 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7febb80b4800 con 0x7febb80b1cc0 2023-08-09T15:12:17.349 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:17.343+0000 7febbdb86640 1 -- 172.21.15.43:0/922439427 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7febb80b4dc0 con 0x7febb80b1cc0 2023-08-09T15:12:17.350 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:17.344+0000 7febbdb86640 1 -- 172.21.15.43:0/922439427 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7febb80b55c0 con 0x7febb80b1cc0 2023-08-09T15:12:17.350 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:17.346+0000 7febb57fa640 1 -- 172.21.15.43:0/922439427 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 129) v1 ==== 88443+0+0 (secure 0 0 0) 0x7feba802f050 con 0x7febb80b1cc0 2023-08-09T15:12:17.351 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:17.347+0000 7febb57fa640 1 --2- 172.21.15.43:0/922439427 >> [v2:172.21.15.203:6832/1456699667,v1:172.21.15.203:6833/1456699667] conn(0x7feba406a1a0 0x7feba406c660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T15:12:17.351 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:17.347+0000 7febb57fa640 1 -- 172.21.15.43:0/922439427 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(229..229 src has 1..229) v4 ==== 6462+0+0 (secure 0 0 0) 0x7feba80aa9b0 con 0x7febb80b1cc0 2023-08-09T15:12:17.351 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:17.347+0000 7febbcb84640 1 --2- 172.21.15.43:0/922439427 >> [v2:172.21.15.203:6832/1456699667,v1:172.21.15.203:6833/1456699667] conn(0x7feba406a1a0 0x7feba406c660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T15:12:17.351 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:17.349+0000 7febbcb84640 1 --2- 172.21.15.43:0/922439427 >> [v2:172.21.15.203:6832/1456699667,v1:172.21.15.203:6833/1456699667] conn(0x7feba406a1a0 0x7feba406c660 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7febac009970 tx=0x7febac008040 comp rx=0 tx=0).ready entity=mgr.9232 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-09T15:12:17.352 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:17.351+0000 7febb57fa640 1 -- 172.21.15.43:0/922439427 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178066 (secure 0 0 0) 0x7feba8075e10 con 0x7febb80b1cc0 2023-08-09T15:12:17.621 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:17.620+0000 7febbdb86640 1 -- 172.21.15.43:0/922439427 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_snap_schedules.TestSnapSchedulesSnapdir.test_snap_dir_name"]} v 0) v1 -- 0x7febb815d020 con 0x7febb80b1cc0 2023-08-09T15:12:17.659 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:17.657+0000 7febb57fa640 1 -- 172.21.15.43:0/922439427 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_snap_schedules.TestSnapSchedulesSnapdir.test_snap_dir_name"]}]=0 v3251) v1 ==== 155+0+0 (secure 0 0 0) 0x7feba8079430 con 0x7febb80b1cc0 2023-08-09T15:12:17.662 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:17.660+0000 7febbdb86640 1 -- 172.21.15.43:0/922439427 >> [v2:172.21.15.203:6832/1456699667,v1:172.21.15.203:6833/1456699667] conn(0x7feba406a1a0 msgr2=0x7feba406c660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T15:12:17.662 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:17.660+0000 7febbdb86640 1 --2- 172.21.15.43:0/922439427 >> [v2:172.21.15.203:6832/1456699667,v1:172.21.15.203:6833/1456699667] conn(0x7feba406a1a0 0x7feba406c660 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7febac009970 tx=0x7febac008040 comp rx=0 tx=0).stop 2023-08-09T15:12:17.662 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:17.660+0000 7febbdb86640 1 -- 172.21.15.43:0/922439427 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7febb80b1cc0 msgr2=0x7febb814a4f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T15:12:17.663 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:17.660+0000 7febbdb86640 1 --2- 172.21.15.43:0/922439427 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7febb80b1cc0 0x7febb814a4f0 secure :-1 s=READY pgs=2549 cs=0 l=1 rev1=1 crypto rx=0x7feba8009990 tx=0x7feba80046c0 comp rx=0 tx=0).stop 2023-08-09T15:12:17.663 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:17.661+0000 7febbdb86640 1 -- 172.21.15.43:0/922439427 shutdown_connections 2023-08-09T15:12:17.663 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:17.661+0000 7febbdb86640 1 --2- 172.21.15.43:0/922439427 >> [v2:172.21.15.203:6832/1456699667,v1:172.21.15.203:6833/1456699667] conn(0x7feba406a1a0 0x7feba406c660 unknown :-1 s=CLOSED pgs=38 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T15:12:17.663 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:17.661+0000 7febbdb86640 1 --2- 172.21.15.43:0/922439427 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7febb80cfda0 0x7febb80b58a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T15:12:17.663 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:17.661+0000 7febbdb86640 1 --2- 172.21.15.43:0/922439427 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7febb80b1cc0 0x7febb814a4f0 unknown :-1 s=CLOSED pgs=2549 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T15:12:17.663 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:17.661+0000 7febbdb86640 1 --2- 172.21.15.43:0/922439427 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7febb806a850 0x7febb80b4140 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T15:12:17.663 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:17.661+0000 7febbdb86640 1 -- 172.21.15.43:0/922439427 >> 172.21.15.43:0/922439427 conn(0x7febb806a190 msgr2=0x7febb815f570 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-09T15:12:17.664 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:17.661+0000 7febbdb86640 1 -- 172.21.15.43:0/922439427 shutdown_connections 2023-08-09T15:12:17.664 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:17.661+0000 7febbdb86640 1 -- 172.21.15.43:0/922439427 wait complete. 2023-08-09T15:12:17.683 INFO:tasks.cephfs_test_runner:Test the correctness of snap directory name ... ok 2023-08-09T15:12:17.683 INFO:tasks.cephfs_test_runner: 2023-08-09T15:12:17.683 INFO:tasks.cephfs_test_runner:---------------------------------------------------------------------- 2023-08-09T15:12:17.684 INFO:tasks.cephfs_test_runner:Ran 11 tests in 5820.019s 2023-08-09T15:12:17.684 INFO:tasks.cephfs_test_runner: 2023-08-09T15:12:17.684 INFO:tasks.cephfs_test_runner:OK 2023-08-09T15:12:17.684 DEBUG:teuthology.run_tasks:Unwinding manager cephfs_test_runner 2023-08-09T15:12:17.694 DEBUG:teuthology.run_tasks:Unwinding manager ceph-fuse 2023-08-09T15:12:17.711 INFO:tasks.ceph_fuse:Unmounting ceph-fuse clients... 2023-08-09T15:12:17.711 DEBUG:teuthology.orchestra.run.smithi043:> set -ex 2023-08-09T15:12:17.711 DEBUG:teuthology.orchestra.run.smithi043:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-09T15:12:17.773 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4054441,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6498844k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=27,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3249420k,nr_inodes=812355,mode=700,uid=1000,gid=1275 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6498844k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2023-08-09T15:12:17.773 DEBUG:teuthology.orchestra.run.smithi043:> set -ex 2023-08-09T15:12:17.774 DEBUG:teuthology.orchestra.run.smithi043:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-09T15:12:17.829 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4054441,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6498844k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=27,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3249420k,nr_inodes=812355,mode=700,uid=1000,gid=1275 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6498844k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2023-08-09T15:12:17.830 DEBUG:teuthology.orchestra.run.smithi203:> set -ex 2023-08-09T15:12:17.830 DEBUG:teuthology.orchestra.run.smithi203:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-09T15:12:17.889 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4053648,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497576k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=27,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248784k,nr_inodes=812196,mode=700,uid=1000,gid=1275 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497576k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2023-08-09T15:12:17.889 DEBUG:teuthology.orchestra.run.smithi203:> set -ex 2023-08-09T15:12:17.890 DEBUG:teuthology.orchestra.run.smithi203:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-09T15:12:17.943 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4053648,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497576k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=27,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248784k,nr_inodes=812196,mode=700,uid=1000,gid=1275 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497576k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2023-08-09T15:12:17.944 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-09T15:12:17.944 DEBUG:teuthology.orchestra.run.smithi043:> ip netns list 2023-08-09T15:12:18.006 INFO:teuthology.orchestra.run.smithi043.stdout:ceph-ns--home-ubuntu-cephtest-mnt.2 (id: 1) 2023-08-09T15:12:18.006 INFO:teuthology.orchestra.run.smithi043.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2023-08-09T15:12:18.007 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-09T15:12:18.007 DEBUG:teuthology.orchestra.run.smithi043:> sudo ip netns delete ceph-ns--home-ubuntu-cephtest-mnt.2 2023-08-09T15:12:18.111 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-09T15:12:18.112 DEBUG:teuthology.orchestra.run.smithi043:> sudo ip netns delete ceph-ns--home-ubuntu-cephtest-mnt.0 2023-08-09T15:12:18.207 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-09T15:12:18.207 DEBUG:teuthology.orchestra.run.smithi043:> sudo ip link delete ceph-brx 2023-08-09T15:12:18.321 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-09T15:12:18.321 DEBUG:teuthology.orchestra.run.smithi203:> ip netns list 2023-08-09T15:12:18.382 INFO:teuthology.orchestra.run.smithi203.stdout:ceph-ns--home-ubuntu-cephtest-mnt.3 (id: 1) 2023-08-09T15:12:18.382 INFO:teuthology.orchestra.run.smithi203.stdout:ceph-ns--home-ubuntu-cephtest-mnt.1 (id: 0) 2023-08-09T15:12:18.383 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-09T15:12:18.383 DEBUG:teuthology.orchestra.run.smithi203:> sudo ip netns delete ceph-ns--home-ubuntu-cephtest-mnt.3 2023-08-09T15:12:18.483 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-09T15:12:18.484 DEBUG:teuthology.orchestra.run.smithi203:> sudo ip netns delete ceph-ns--home-ubuntu-cephtest-mnt.1 2023-08-09T15:12:18.577 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-09T15:12:18.578 DEBUG:teuthology.orchestra.run.smithi203:> sudo ip link delete ceph-brx 2023-08-09T15:12:18.691 DEBUG:teuthology.run_tasks:Unwinding manager ceph 2023-08-09T15:12:18.700 INFO:tasks.ceph.ceph_manager.ceph:waiting for clean 2023-08-09T15:12:18.701 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-08-09T15:12:18.943 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:18.942+0000 7fcc7f59e640 1 Processor -- start 2023-08-09T15:12:18.944 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:18.942+0000 7fcc7f59e640 1 -- start start 2023-08-09T15:12:18.946 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:18.943+0000 7fcc7f59e640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7fcc8015a4c0 0x7fcc800ca890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T15:12:18.946 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:18.943+0000 7fcc7f59e640 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7fcc800cfc20 0x7fcc8015dec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T15:12:18.946 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:18.944+0000 7fcc7f59e640 1 -- --> v1:172.21.15.43:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fcc8015aa20 con 0x7fcc800cadd0 2023-08-09T15:12:18.946 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:18.944+0000 7fcc7f59e640 1 -- --> v2:172.21.15.43:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fcc8015b660 con 0x7fcc8015a4c0 2023-08-09T15:12:18.947 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:18.944+0000 7fcc7f59e640 1 -- --> v2:172.21.15.203:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fcc8015acc0 con 0x7fcc800cfc20 2023-08-09T15:12:18.947 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:18.944+0000 7fcc7e59c640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7fcc8015a4c0 0x7fcc800ca890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T15:12:18.947 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:18.944+0000 7fcc7e59c640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7fcc8015a4c0 0x7fcc800ca890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:46236/0 (socket says 172.21.15.43:46236) 2023-08-09T15:12:18.947 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:18.944+0000 7fcc7e59c640 1 -- 172.21.15.43:0/1334046644 learned_addr learned my addr 172.21.15.43:0/1334046644 (peer_addr_for_me v2:172.21.15.43:0/0) 2023-08-09T15:12:18.948 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:18.944+0000 7fcc7ed9d640 1 --2- 172.21.15.43:0/1334046644 >> v2:172.21.15.203:3300/0 conn(0x7fcc800cfc20 0x7fcc8015dec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T15:12:18.948 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:18.944+0000 7fcc7e59c640 1 -- 172.21.15.43:0/1334046644 >> v2:172.21.15.203:3300/0 conn(0x7fcc800cfc20 msgr2=0x7fcc8015dec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-09T15:12:18.948 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:18.944+0000 7fcc7e59c640 1 --2- 172.21.15.43:0/1334046644 >> v2:172.21.15.203:3300/0 conn(0x7fcc800cfc20 0x7fcc8015dec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T15:12:18.948 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:18.944+0000 7fcc7e59c640 1 -- 172.21.15.43:0/1334046644 >> v1:172.21.15.43:6789/0 conn(0x7fcc800cadd0 legacy=0x7fcc800cf510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T15:12:18.949 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:18.944+0000 7fcc7e59c640 1 -- 172.21.15.43:0/1334046644 --> v2:172.21.15.43:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcc8015afa0 con 0x7fcc8015a4c0 2023-08-09T15:12:18.949 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:18.944+0000 7fcc7ed9d640 1 --2- 172.21.15.43:0/1334046644 >> v2:172.21.15.203:3300/0 conn(0x7fcc800cfc20 0x7fcc8015dec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-08-09T15:12:18.953 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:18.946+0000 7fcc7e59c640 1 --2- 172.21.15.43:0/1334046644 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fcc8015a4c0 0x7fcc800ca890 secure :-1 s=READY pgs=2551 cs=0 l=1 rev1=1 crypto rx=0x7fcc800715c0 tx=0x7fcc7002b9b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7e8890967578990a server_cookie=0 in_seq=0 out_seq=0 2023-08-09T15:12:18.953 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:18.946+0000 7fcc7d59a640 1 -- 172.21.15.43:0/1334046644 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcc70030870 con 0x7fcc8015a4c0 2023-08-09T15:12:18.953 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:18.946+0000 7fcc7d59a640 1 -- 172.21.15.43:0/1334046644 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcc700309d0 con 0x7fcc8015a4c0 2023-08-09T15:12:18.954 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:18.946+0000 7fcc7d59a640 1 -- 172.21.15.43:0/1334046644 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcc70030d60 con 0x7fcc8015a4c0 2023-08-09T15:12:18.954 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:18.947+0000 7fcc7f59e640 1 -- 172.21.15.43:0/1334046644 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fcc8015a4c0 msgr2=0x7fcc800ca890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T15:12:18.954 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:18.947+0000 7fcc7f59e640 1 --2- 172.21.15.43:0/1334046644 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fcc8015a4c0 0x7fcc800ca890 secure :-1 s=READY pgs=2551 cs=0 l=1 rev1=1 crypto rx=0x7fcc800715c0 tx=0x7fcc7002b9b0 comp rx=0 tx=0).stop 2023-08-09T15:12:18.955 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:18.947+0000 7fcc7f59e640 1 -- 172.21.15.43:0/1334046644 shutdown_connections 2023-08-09T15:12:18.955 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:18.947+0000 7fcc7f59e640 1 --2- 172.21.15.43:0/1334046644 >> v2:172.21.15.203:3300/0 conn(0x7fcc800cfc20 0x7fcc8015dec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T15:12:18.955 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:18.947+0000 7fcc7f59e640 1 --2- 172.21.15.43:0/1334046644 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fcc8015a4c0 0x7fcc800ca890 unknown :-1 s=CLOSED pgs=2551 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T15:12:18.955 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:18.947+0000 7fcc7f59e640 1 -- 172.21.15.43:0/1334046644 >> 172.21.15.43:0/1334046644 conn(0x7fcc8006a190 msgr2=0x7fcc800cccc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-09T15:12:18.956 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:18.947+0000 7fcc7f59e640 1 -- 172.21.15.43:0/1334046644 shutdown_connections 2023-08-09T15:12:18.956 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:18.947+0000 7fcc7f59e640 1 -- 172.21.15.43:0/1334046644 wait complete. 2023-08-09T15:12:18.956 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:18.949+0000 7fcc7f59e640 1 Processor -- start 2023-08-09T15:12:18.956 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:18.949+0000 7fcc7f59e640 1 -- start start 2023-08-09T15:12:18.957 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:18.950+0000 7fcc7f59e640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fcc800cadd0 0x7fcc800c8f10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T15:12:18.957 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:18.950+0000 7fcc7f59e640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fcc800cfc20 0x7fcc801f13a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T15:12:18.957 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:18.950+0000 7fcc7f59e640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fcc8015a4c0 0x7fcc800b3830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T15:12:18.957 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:18.950+0000 7fcc7f59e640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcc801f18e0 con 0x7fcc8015a4c0 2023-08-09T15:12:18.958 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:18.950+0000 7fcc7f59e640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcc800b4060 con 0x7fcc800cadd0 2023-08-09T15:12:18.958 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:18.950+0000 7fcc7f59e640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcc800b4340 con 0x7fcc800cfc20 2023-08-09T15:12:18.958 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:18.950+0000 7fcc7ed9d640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fcc8015a4c0 0x7fcc800b3830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T15:12:18.959 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:18.950+0000 7fcc7ed9d640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fcc8015a4c0 0x7fcc800b3830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:46240/0 (socket says 172.21.15.43:46240) 2023-08-09T15:12:18.959 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:18.950+0000 7fcc7ed9d640 1 -- 172.21.15.43:0/3665259753 learned_addr learned my addr 172.21.15.43:0/3665259753 (peer_addr_for_me v2:172.21.15.43:0/0) 2023-08-09T15:12:18.959 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:18.950+0000 7fcc7e59c640 1 --2- 172.21.15.43:0/3665259753 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fcc800cadd0 0x7fcc800c8f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T15:12:18.959 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:18.950+0000 7fcc7dd9b640 1 --2- 172.21.15.43:0/3665259753 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fcc800cfc20 0x7fcc801f13a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T15:12:18.960 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:18.951+0000 7fcc7ed9d640 1 -- 172.21.15.43:0/3665259753 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fcc800cfc20 msgr2=0x7fcc801f13a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T15:12:18.960 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:18.951+0000 7fcc7ed9d640 1 --2- 172.21.15.43:0/3665259753 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fcc800cfc20 0x7fcc801f13a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T15:12:18.960 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:18.951+0000 7fcc7ed9d640 1 -- 172.21.15.43:0/3665259753 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fcc800cadd0 msgr2=0x7fcc800c8f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T15:12:18.960 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:18.951+0000 7fcc7ed9d640 1 --2- 172.21.15.43:0/3665259753 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fcc800cadd0 0x7fcc800c8f10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T15:12:18.961 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:18.951+0000 7fcc7ed9d640 1 -- 172.21.15.43:0/3665259753 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcc700094d0 con 0x7fcc8015a4c0 2023-08-09T15:12:18.961 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:18.951+0000 7fcc7ed9d640 1 --2- 172.21.15.43:0/3665259753 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fcc8015a4c0 0x7fcc800b3830 secure :-1 s=READY pgs=2552 cs=0 l=1 rev1=1 crypto rx=0x7fcc7800ce70 tx=0x7fcc7800a710 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-09T15:12:18.961 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:18.951+0000 7fcc76ffd640 1 -- 172.21.15.43:0/3665259753 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcc78003b90 con 0x7fcc8015a4c0 2023-08-09T15:12:18.962 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:18.951+0000 7fcc76ffd640 1 -- 172.21.15.43:0/3665259753 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcc78003d10 con 0x7fcc8015a4c0 2023-08-09T15:12:18.962 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:18.951+0000 7fcc76ffd640 1 -- 172.21.15.43:0/3665259753 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcc78004260 con 0x7fcc8015a4c0 2023-08-09T15:12:18.962 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:18.951+0000 7fcc7f59e640 1 -- 172.21.15.43:0/3665259753 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcc800b4620 con 0x7fcc8015a4c0 2023-08-09T15:12:18.962 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:18.952+0000 7fcc7f59e640 1 -- 172.21.15.43:0/3665259753 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcc800b4c70 con 0x7fcc8015a4c0 2023-08-09T15:12:18.963 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:18.953+0000 7fcc7f59e640 1 -- 172.21.15.43:0/3665259753 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcc8004e390 con 0x7fcc8015a4c0 2023-08-09T15:12:18.964 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:18.954+0000 7fcc76ffd640 1 -- 172.21.15.43:0/3665259753 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 129) v1 ==== 88443+0+0 (secure 0 0 0) 0x7fcc780043c0 con 0x7fcc8015a4c0 2023-08-09T15:12:18.964 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:18.954+0000 7fcc76ffd640 1 --2- 172.21.15.43:0/3665259753 >> [v2:172.21.15.203:6832/1456699667,v1:172.21.15.203:6833/1456699667] conn(0x7fcc6006a170 0x7fcc6006c630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T15:12:18.964 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:18.955+0000 7fcc76ffd640 1 -- 172.21.15.43:0/3665259753 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(229..229 src has 1..229) v4 ==== 6462+0+0 (secure 0 0 0) 0x7fcc780890b0 con 0x7fcc8015a4c0 2023-08-09T15:12:18.964 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:18.955+0000 7fcc7e59c640 1 --2- 172.21.15.43:0/3665259753 >> [v2:172.21.15.203:6832/1456699667,v1:172.21.15.203:6833/1456699667] conn(0x7fcc6006a170 0x7fcc6006c630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T15:12:18.964 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:18.956+0000 7fcc7e59c640 1 --2- 172.21.15.43:0/3665259753 >> [v2:172.21.15.203:6832/1456699667,v1:172.21.15.203:6833/1456699667] conn(0x7fcc6006a170 0x7fcc6006c630 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7fcc70036000 tx=0x7fcc7003b040 comp rx=0 tx=0).ready entity=mgr.9232 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-09T15:12:18.964 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:18.959+0000 7fcc76ffd640 1 -- 172.21.15.43:0/3665259753 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178066 (secure 0 0 0) 0x7fcc78054590 con 0x7fcc8015a4c0 2023-08-09T15:12:19.151 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:19.149+0000 7fcc7f59e640 1 -- 172.21.15.43:0/3665259753 --> [v2:172.21.15.203:6832/1456699667,v1:172.21.15.203:6833/1456699667] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7fcc8015b280 con 0x7fcc6006a170 2023-08-09T15:12:19.160 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:19.158+0000 7fcc76ffd640 1 -- 172.21.15.43:0/3665259753 <== mgr.9232 v2:172.21.15.203:6832/1456699667 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+70658 (secure 0 0 0) 0x7fcc70030810 con 0x7fcc6006a170 2023-08-09T15:12:19.161 INFO:teuthology.orchestra.run.smithi043.stdout: 2023-08-09T15:12:19.162 INFO:teuthology.orchestra.run.smithi043.stderr:dumped all 2023-08-09T15:12:19.163 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:19.161+0000 7fcc7f59e640 1 -- 172.21.15.43:0/3665259753 >> [v2:172.21.15.203:6832/1456699667,v1:172.21.15.203:6833/1456699667] conn(0x7fcc6006a170 msgr2=0x7fcc6006c630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T15:12:19.163 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:19.161+0000 7fcc7f59e640 1 --2- 172.21.15.43:0/3665259753 >> [v2:172.21.15.203:6832/1456699667,v1:172.21.15.203:6833/1456699667] conn(0x7fcc6006a170 0x7fcc6006c630 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7fcc70036000 tx=0x7fcc7003b040 comp rx=0 tx=0).stop 2023-08-09T15:12:19.163 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:19.161+0000 7fcc7f59e640 1 -- 172.21.15.43:0/3665259753 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fcc8015a4c0 msgr2=0x7fcc800b3830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T15:12:19.164 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:19.161+0000 7fcc7f59e640 1 --2- 172.21.15.43:0/3665259753 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fcc8015a4c0 0x7fcc800b3830 secure :-1 s=READY pgs=2552 cs=0 l=1 rev1=1 crypto rx=0x7fcc7800ce70 tx=0x7fcc7800a710 comp rx=0 tx=0).stop 2023-08-09T15:12:19.164 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:19.161+0000 7fcc7f59e640 1 -- 172.21.15.43:0/3665259753 shutdown_connections 2023-08-09T15:12:19.164 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:19.161+0000 7fcc7f59e640 1 --2- 172.21.15.43:0/3665259753 >> [v2:172.21.15.203:6832/1456699667,v1:172.21.15.203:6833/1456699667] conn(0x7fcc6006a170 0x7fcc6006c630 unknown :-1 s=CLOSED pgs=39 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T15:12:19.164 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:19.161+0000 7fcc7f59e640 1 --2- 172.21.15.43:0/3665259753 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fcc8015a4c0 0x7fcc800b3830 unknown :-1 s=CLOSED pgs=2552 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T15:12:19.164 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:19.161+0000 7fcc7f59e640 1 --2- 172.21.15.43:0/3665259753 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fcc800cfc20 0x7fcc801f13a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T15:12:19.164 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:19.161+0000 7fcc7f59e640 1 --2- 172.21.15.43:0/3665259753 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fcc800cadd0 0x7fcc800c8f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T15:12:19.165 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:19.161+0000 7fcc7f59e640 1 -- 172.21.15.43:0/3665259753 >> 172.21.15.43:0/3665259753 conn(0x7fcc8006a190 msgr2=0x7fcc800afd80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-09T15:12:19.165 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:19.161+0000 7fcc7f59e640 1 -- 172.21.15.43:0/3665259753 shutdown_connections 2023-08-09T15:12:19.165 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:19.162+0000 7fcc7f59e640 1 -- 172.21.15.43:0/3665259753 wait complete. 2023-08-09T15:12:19.182 INFO:teuthology.orchestra.run.smithi043.stdout:{"pg_ready":true,"pg_map":{"version":12,"stamp":"2023-08-09T15:12:17.363435+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2155,"num_read_kb":4508,"num_write":583,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":351,"ondisk_log_size":351,"up":18,"acting":18,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":210,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":394176,"kb_used_data":24660,"kb_used_omap":13,"kb_used_meta":369394,"kb_avail":754580544,"statfs":{"total":773094113280,"available":772690477056,"internally_reserved":0,"allocated":25251840,"data_stored":14900654,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":14248,"internal_metadata":378259544},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":12,"apply_latency_ms":12,"commit_latency_ns":12000000,"apply_latency_ns":12000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-676856,"num_objects":-24,"num_object_clones":0,"num_object_copies":-48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-24,"num_whiteouts":0,"num_read":-310,"num_read_kb":-155,"num_write":-319,"num_write_kb":-2330,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":-14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.001968"},"pg_stats":[{"pgid":"1.0","version":"0'0","reported_seq":419,"reported_epoch":226,"state":"active+clean","last_fresh":"2023-08-09T15:12:08.339437+0000","last_change":"2023-08-09T13:33:56.401406+0000","last_active":"2023-08-09T15:12:08.339437+0000","last_peered":"2023-08-09T15:12:08.339437+0000","last_clean":"2023-08-09T15:12:08.339437+0000","last_became_active":"2023-08-09T13:33:56.400487+0000","last_became_peered":"2023-08-09T13:33:56.400487+0000","last_unstale":"2023-08-09T15:12:08.339437+0000","last_undegraded":"2023-08-09T15:12:08.339437+0000","last_fullsized":"2023-08-09T15:12:08.339437+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:33:54.845869+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:33:54.845869+0000","last_clean_scrub_stamp":"2023-08-09T13:33:54.845869+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-11T01:31:30.716395+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":428,"reported_epoch":227,"state":"active+clean","last_fresh":"2023-08-09T15:12:13.396341+0000","last_change":"2023-08-09T13:33:56.401123+0000","last_active":"2023-08-09T15:12:13.396341+0000","last_peered":"2023-08-09T15:12:13.396341+0000","last_clean":"2023-08-09T15:12:13.396341+0000","last_became_active":"2023-08-09T13:33:56.399933+0000","last_became_peered":"2023-08-09T13:33:56.399933+0000","last_unstale":"2023-08-09T15:12:13.396341+0000","last_undegraded":"2023-08-09T15:12:13.396341+0000","last_fullsized":"2023-08-09T15:12:13.396341+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:33:54.845869+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:33:54.845869+0000","last_clean_scrub_stamp":"2023-08-09T13:33:54.845869+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T19:58:38.373560+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.2","version":"0'0","reported_seq":418,"reported_epoch":226,"state":"active+clean","last_fresh":"2023-08-09T15:12:08.338891+0000","last_change":"2023-08-09T13:33:56.402794+0000","last_active":"2023-08-09T15:12:08.338891+0000","last_peered":"2023-08-09T15:12:08.338891+0000","last_clean":"2023-08-09T15:12:08.338891+0000","last_became_active":"2023-08-09T13:33:56.402038+0000","last_became_peered":"2023-08-09T13:33:56.402038+0000","last_unstale":"2023-08-09T15:12:08.338891+0000","last_undegraded":"2023-08-09T15:12:08.338891+0000","last_fullsized":"2023-08-09T15:12:08.338891+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:33:54.845869+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:33:54.845869+0000","last_clean_scrub_stamp":"2023-08-09T13:33:54.845869+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T21:01:16.252596+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.3","version":"0'0","reported_seq":432,"reported_epoch":226,"state":"active+clean","last_fresh":"2023-08-09T15:12:08.336707+0000","last_change":"2023-08-09T13:33:55.886749+0000","last_active":"2023-08-09T15:12:08.336707+0000","last_peered":"2023-08-09T15:12:08.336707+0000","last_clean":"2023-08-09T15:12:08.336707+0000","last_became_active":"2023-08-09T13:33:55.886116+0000","last_became_peered":"2023-08-09T13:33:55.886116+0000","last_unstale":"2023-08-09T15:12:08.336707+0000","last_undegraded":"2023-08-09T15:12:08.336707+0000","last_fullsized":"2023-08-09T15:12:08.336707+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:33:54.845869+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:33:54.845869+0000","last_clean_scrub_stamp":"2023-08-09T13:33:54.845869+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T19:39:28.369478+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"226'351","reported_seq":1398,"reported_epoch":226,"state":"active+clean","last_fresh":"2023-08-09T15:12:09.468941+0000","last_change":"2023-08-09T13:33:55.886345+0000","last_active":"2023-08-09T15:12:09.468941+0000","last_peered":"2023-08-09T15:12:09.468941+0000","last_clean":"2023-08-09T15:12:09.468941+0000","last_became_active":"2023-08-09T13:33:55.885781+0000","last_became_peered":"2023-08-09T13:33:55.885781+0000","last_unstale":"2023-08-09T15:12:09.468941+0000","last_undegraded":"2023-08-09T15:12:09.468941+0000","last_fullsized":"2023-08-09T15:12:09.468941+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:33:54.845869+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:33:54.845869+0000","last_clean_scrub_stamp":"2023-08-09T13:33:54.845869+0000","objects_scrubbed":0,"log_size":351,"log_dups_size":0,"ondisk_log_size":351,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T14:05:51.715296+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2155,"num_read_kb":4508,"num_write":583,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":436,"reported_epoch":226,"state":"active+clean","last_fresh":"2023-08-09T15:12:09.014345+0000","last_change":"2023-08-09T13:33:55.889565+0000","last_active":"2023-08-09T15:12:09.014345+0000","last_peered":"2023-08-09T15:12:09.014345+0000","last_clean":"2023-08-09T15:12:09.014345+0000","last_became_active":"2023-08-09T13:33:55.889164+0000","last_became_peered":"2023-08-09T13:33:55.889164+0000","last_unstale":"2023-08-09T15:12:09.014345+0000","last_undegraded":"2023-08-09T15:12:09.014345+0000","last_fullsized":"2023-08-09T15:12:09.014345+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:33:54.845869+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:33:54.845869+0000","last_clean_scrub_stamp":"2023-08-09T13:33:54.845869+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T16:14:15.486288+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":425,"reported_epoch":227,"state":"active+clean","last_fresh":"2023-08-09T15:12:12.625132+0000","last_change":"2023-08-09T13:33:56.404236+0000","last_active":"2023-08-09T15:12:12.625132+0000","last_peered":"2023-08-09T15:12:12.625132+0000","last_clean":"2023-08-09T15:12:12.625132+0000","last_became_active":"2023-08-09T13:33:56.402485+0000","last_became_peered":"2023-08-09T13:33:56.402485+0000","last_unstale":"2023-08-09T15:12:12.625132+0000","last_undegraded":"2023-08-09T15:12:12.625132+0000","last_fullsized":"2023-08-09T15:12:12.625132+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:33:54.845869+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:33:54.845869+0000","last_clean_scrub_stamp":"2023-08-09T13:33:54.845869+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T19:12:54.152998+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":418,"reported_epoch":227,"state":"active+clean","last_fresh":"2023-08-09T15:12:12.626704+0000","last_change":"2023-08-09T13:33:56.401781+0000","last_active":"2023-08-09T15:12:12.626704+0000","last_peered":"2023-08-09T15:12:12.626704+0000","last_clean":"2023-08-09T15:12:12.626704+0000","last_became_active":"2023-08-09T13:33:56.400336+0000","last_became_peered":"2023-08-09T13:33:56.400336+0000","last_unstale":"2023-08-09T15:12:12.626704+0000","last_undegraded":"2023-08-09T15:12:12.626704+0000","last_fullsized":"2023-08-09T15:12:12.626704+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:33:54.845869+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:33:54.845869+0000","last_clean_scrub_stamp":"2023-08-09T13:33:54.845869+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T18:26:05.239627+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":418,"reported_epoch":227,"state":"active+clean","last_fresh":"2023-08-09T15:12:12.626964+0000","last_change":"2023-08-09T13:33:55.886389+0000","last_active":"2023-08-09T15:12:12.626964+0000","last_peered":"2023-08-09T15:12:12.626964+0000","last_clean":"2023-08-09T15:12:12.626964+0000","last_became_active":"2023-08-09T13:33:55.885638+0000","last_became_peered":"2023-08-09T13:33:55.885638+0000","last_unstale":"2023-08-09T15:12:12.626964+0000","last_undegraded":"2023-08-09T15:12:12.626964+0000","last_fullsized":"2023-08-09T15:12:12.626964+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:33:54.845869+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:33:54.845869+0000","last_clean_scrub_stamp":"2023-08-09T13:33:54.845869+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T19:15:48.495123+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2155,"num_read_kb":4508,"num_write":583,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":351,"ondisk_log_size":351,"up":2,"acting":2,"num_store_stats":2},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":8}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834576020,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":52852,"kb_used_data":3692,"kb_used_omap":1,"kb_used_meta":49150,"kb_avail":94318988,"statfs":{"total":96636764160,"available":96582643712,"internally_reserved":0,"allocated":3780608,"data_stored":2483360,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1781,"internal_metadata":50329867},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Aug 9 15:11:52 2023","interfaces":[{"interface":"back","average":{"1min":1.113,"5min":1.234,"15min":1.2290000000000001},"min":{"1min":0.52400000000000002,"5min":0.52400000000000002,"15min":0.52400000000000002},"max":{"1min":1.6080000000000001,"5min":2.056,"15min":2.056},"last":0.54800000000000004},{"interface":"front","average":{"1min":1.0900000000000001,"5min":1.2110000000000001,"15min":1.2529999999999999},"min":{"1min":0.55800000000000005,"5min":0.55800000000000005,"15min":0.55800000000000005},"max":{"1min":1.5960000000000001,"5min":1.7769999999999999,"15min":1.863},"last":0.76900000000000002}]},{"osd":1,"last update":"Wed Aug 9 15:11:52 2023","interfaces":[{"interface":"back","average":{"1min":1.1479999999999999,"5min":1.28,"15min":1.3049999999999999},"min":{"1min":0.52500000000000002,"5min":0.52500000000000002,"15min":0.52500000000000002},"max":{"1min":1.6619999999999999,"5min":1.853,"15min":1.853},"last":1.2549999999999999},{"interface":"front","average":{"1min":1.175,"5min":1.296,"15min":1.3100000000000001},"min":{"1min":0.748,"5min":0.748,"15min":0.748},"max":{"1min":1.605,"5min":2.1520000000000001,"15min":2.1520000000000001},"last":0.88800000000000001}]},{"osd":2,"last update":"Wed Aug 9 15:11:52 2023","interfaces":[{"interface":"back","average":{"1min":1.093,"5min":1.262,"15min":1.2969999999999999},"min":{"1min":0.70399999999999996,"5min":0.70399999999999996,"15min":0.70399999999999996},"max":{"1min":1.7050000000000001,"5min":1.861,"15min":1.861},"last":0.71299999999999997},{"interface":"front","average":{"1min":1.214,"5min":1.3480000000000001,"15min":1.393},"min":{"1min":0.58099999999999996,"5min":0.58099999999999996,"15min":0.58099999999999996},"max":{"1min":1.7889999999999999,"5min":1.9370000000000001,"15min":1.9370000000000001},"last":1.4470000000000001}]},{"osd":3,"last update":"Wed Aug 9 15:11:52 2023","interfaces":[{"interface":"back","average":{"1min":1.1950000000000001,"5min":1.3160000000000001,"15min":1.351},"min":{"1min":0.80700000000000005,"5min":0.80700000000000005,"15min":0.80700000000000005},"max":{"1min":1.798,"5min":1.798,"15min":1.9259999999999999},"last":0.83499999999999996},{"interface":"front","average":{"1min":1.335,"5min":1.4379999999999999,"15min":1.472},"min":{"1min":0.76200000000000001,"5min":0.76200000000000001,"15min":0.76200000000000001},"max":{"1min":1.9179999999999999,"5min":1.9239999999999999,"15min":1.9239999999999999},"last":1.53}]},{"osd":4,"last update":"Wed Aug 9 15:11:52 2023","interfaces":[{"interface":"back","average":{"1min":1.218,"5min":1.282,"15min":1.2989999999999999},"min":{"1min":0.56399999999999995,"5min":0.56399999999999995,"15min":0.56399999999999995},"max":{"1min":1.6080000000000001,"5min":2.2810000000000001,"15min":2.2810000000000001},"last":1.0489999999999999},{"interface":"front","average":{"1min":1.28,"5min":1.3560000000000001,"15min":1.385},"min":{"1min":0.622,"5min":0.622,"15min":0.622},"max":{"1min":2.0390000000000001,"5min":2.0390000000000001,"15min":2.0830000000000002},"last":1.3740000000000001}]},{"osd":5,"last update":"Wed Aug 9 15:11:52 2023","interfaces":[{"interface":"back","average":{"1min":1.3620000000000001,"5min":1.4219999999999999,"15min":1.4410000000000001},"min":{"1min":0.59299999999999997,"5min":0.59299999999999997,"15min":0.59299999999999997},"max":{"1min":2.0899999999999999,"5min":2.0899999999999999,"15min":2.0899999999999999},"last":1.583},{"interface":"front","average":{"1min":1.3959999999999999,"5min":1.448,"15min":1.4730000000000001},"min":{"1min":0.65500000000000003,"5min":0.65500000000000003,"15min":0.65500000000000003},"max":{"1min":2.206,"5min":2.5310000000000001,"15min":2.5310000000000001},"last":1.3220000000000001}]},{"osd":6,"last update":"Wed Aug 9 15:11:52 2023","interfaces":[{"interface":"back","average":{"1min":1.5069999999999999,"5min":1.587,"15min":1.589},"min":{"1min":0.69199999999999995,"5min":0.69199999999999995,"15min":0.69199999999999995},"max":{"1min":2.294,"5min":2.8959999999999999,"15min":2.8959999999999999},"last":1.4119999999999999},{"interface":"front","average":{"1min":1.359,"5min":1.5089999999999999,"15min":1.534},"min":{"1min":0.60899999999999999,"5min":0.60899999999999999,"15min":0.60899999999999999},"max":{"1min":2.161,"5min":2.214,"15min":2.222},"last":1.1299999999999999}]}]},{"osd":6,"up_from":13,"seq":55834576020,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":60444,"kb_used_data":3084,"kb_used_omap":1,"kb_used_meta":57342,"kb_avail":94311396,"statfs":{"total":96636764160,"available":96574869504,"internally_reserved":0,"allocated":3158016,"data_stored":1872067,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1781,"internal_metadata":58718475},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Aug 9 15:11:57 2023","interfaces":[{"interface":"back","average":{"1min":1.1639999999999999,"5min":1.2130000000000001,"15min":1.2190000000000001},"min":{"1min":0.76700000000000002,"5min":0.75600000000000001,"15min":0.75600000000000001},"max":{"1min":1.8169999999999999,"5min":1.8169999999999999,"15min":1.8859999999999999},"last":0.86599999999999999},{"interface":"front","average":{"1min":1.0720000000000001,"5min":1.21,"15min":1.234},"min":{"1min":0.64500000000000002,"5min":0.64500000000000002,"15min":0.64500000000000002},"max":{"1min":1.5109999999999999,"5min":2.0800000000000001,"15min":2.0800000000000001},"last":1.0189999999999999}]},{"osd":1,"last update":"Wed Aug 9 15:11:57 2023","interfaces":[{"interface":"back","average":{"1min":1.095,"5min":1.1879999999999999,"15min":1.1910000000000001},"min":{"1min":0.65900000000000003,"5min":0.65900000000000003,"15min":0.65900000000000003},"max":{"1min":1.776,"5min":2.6760000000000002,"15min":2.6760000000000002},"last":0.95399999999999996},{"interface":"front","average":{"1min":1.2869999999999999,"5min":1.3120000000000001,"15min":1.3460000000000001},"min":{"1min":0.72599999999999998,"5min":0.72599999999999998,"15min":0.72599999999999998},"max":{"1min":1.579,"5min":1.998,"15min":1.998},"last":1.198}]},{"osd":2,"last update":"Wed Aug 9 15:11:57 2023","interfaces":[{"interface":"back","average":{"1min":1.2629999999999999,"5min":1.3120000000000001,"15min":1.3400000000000001},"min":{"1min":0.79800000000000004,"5min":0.79800000000000004,"15min":0.79800000000000004},"max":{"1min":1.6659999999999999,"5min":2.0089999999999999,"15min":2.081},"last":1.3480000000000001},{"interface":"front","average":{"1min":1.3080000000000001,"5min":1.337,"15min":1.3480000000000001},"min":{"1min":0.73799999999999999,"5min":0.73799999999999999,"15min":0.73799999999999999},"max":{"1min":1.75,"5min":1.9219999999999999,"15min":1.9219999999999999},"last":1.1559999999999999}]},{"osd":3,"last update":"Wed Aug 9 15:11:57 2023","interfaces":[{"interface":"back","average":{"1min":1.302,"5min":1.4099999999999999,"15min":1.4019999999999999},"min":{"1min":0.70299999999999996,"5min":0.70299999999999996,"15min":0.70299999999999996},"max":{"1min":2.2669999999999999,"5min":2.2669999999999999,"15min":2.2669999999999999},"last":1.4059999999999999},{"interface":"front","average":{"1min":1.2270000000000001,"5min":1.319,"15min":1.3420000000000001},"min":{"1min":0.59699999999999998,"5min":0.59699999999999998,"15min":0.59699999999999998},"max":{"1min":1.855,"5min":2.0539999999999998,"15min":2.0539999999999998},"last":1.296}]},{"osd":4,"last update":"Wed Aug 9 15:11:57 2023","interfaces":[{"interface":"back","average":{"1min":1.339,"5min":1.3500000000000001,"15min":1.339},"min":{"1min":0.46500000000000002,"5min":0.46500000000000002,"15min":0.46500000000000002},"max":{"1min":1.7270000000000001,"5min":2.0470000000000002,"15min":2.0470000000000002},"last":1.508},{"interface":"front","average":{"1min":1.288,"5min":1.4470000000000001,"15min":1.4330000000000001},"min":{"1min":0.53000000000000003,"5min":0.53000000000000003,"15min":0.53000000000000003},"max":{"1min":1.6559999999999999,"5min":2.1589999999999998,"15min":2.1589999999999998},"last":1.5509999999999999}]},{"osd":5,"last update":"Wed Aug 9 15:11:57 2023","interfaces":[{"interface":"back","average":{"1min":1.3109999999999999,"5min":1.431,"15min":1.381},"min":{"1min":0.51100000000000001,"5min":0.51100000000000001,"15min":0.51100000000000001},"max":{"1min":1.9610000000000001,"5min":2.1269999999999998,"15min":2.6059999999999999},"last":1.859},{"interface":"front","average":{"1min":1.546,"5min":1.637,"15min":1.6299999999999999},"min":{"1min":0.67300000000000004,"5min":0.67300000000000004,"15min":0.67300000000000004},"max":{"1min":1.885,"5min":2.125,"15min":2.1899999999999999},"last":1.778}]},{"osd":7,"last update":"Wed Aug 9 15:11:57 2023","interfaces":[{"interface":"back","average":{"1min":1.506,"5min":1.639,"15min":1.6180000000000001},"min":{"1min":0.626,"5min":0.626,"15min":0.626},"max":{"1min":1.9319999999999999,"5min":2.2280000000000002,"15min":2.4340000000000002},"last":2.0249999999999999},{"interface":"front","average":{"1min":1.45,"5min":1.6180000000000001,"15min":1.6000000000000001},"min":{"1min":0.44600000000000001,"5min":0.44600000000000001,"15min":0.44600000000000001},"max":{"1min":2.0409999999999999,"5min":2.3300000000000001,"15min":2.7400000000000002},"last":1.7210000000000001}]}]},{"osd":5,"up_from":13,"seq":55834576020,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":47840,"kb_used_data":2768,"kb_used_omap":1,"kb_used_meta":45054,"kb_avail":94324000,"statfs":{"total":96636764160,"available":96587776000,"internally_reserved":0,"allocated":2834432,"data_stored":1545748,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1781,"internal_metadata":46135563},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Aug 9 15:11:47 2023","interfaces":[{"interface":"back","average":{"1min":1.0669999999999999,"5min":1.1919999999999999,"15min":1.194},"min":{"1min":0.57699999999999996,"5min":0.57699999999999996,"15min":0.57699999999999996},"max":{"1min":1.494,"5min":1.7370000000000001,"15min":1.754},"last":0.94999999999999996},{"interface":"front","average":{"1min":1.0389999999999999,"5min":1.167,"15min":1.208},"min":{"1min":0.55600000000000005,"5min":0.55600000000000005,"15min":0.55600000000000005},"max":{"1min":1.3480000000000001,"5min":1.6870000000000001,"15min":1.976},"last":1.04}]},{"osd":1,"last update":"Wed Aug 9 15:11:47 2023","interfaces":[{"interface":"back","average":{"1min":1.2490000000000001,"5min":1.28,"15min":1.2509999999999999},"min":{"1min":0.67500000000000004,"5min":0.65200000000000002,"15min":0.65200000000000002},"max":{"1min":1.9490000000000001,"5min":2.016,"15min":2.016},"last":1.153},{"interface":"front","average":{"1min":1.2150000000000001,"5min":1.288,"15min":1.343},"min":{"1min":0.65000000000000002,"5min":0.65000000000000002,"15min":0.65000000000000002},"max":{"1min":2.0169999999999999,"5min":2.0169999999999999,"15min":2.0169999999999999},"last":1.595}]},{"osd":2,"last update":"Wed Aug 9 15:11:47 2023","interfaces":[{"interface":"back","average":{"1min":1.0269999999999999,"5min":1.2629999999999999,"15min":1.3009999999999999},"min":{"1min":0.60199999999999998,"5min":0.60199999999999998,"15min":0.60199999999999998},"max":{"1min":1.48,"5min":1.8859999999999999,"15min":2.032},"last":1.4219999999999999},{"interface":"front","average":{"1min":1.3,"5min":1.339,"15min":1.3640000000000001},"min":{"1min":0.61899999999999999,"5min":0.61899999999999999,"15min":0.61899999999999999},"max":{"1min":1.7010000000000001,"5min":1.956,"15min":1.964},"last":1.52}]},{"osd":3,"last update":"Wed Aug 9 15:11:47 2023","interfaces":[{"interface":"back","average":{"1min":1.3,"5min":1.3740000000000001,"15min":1.3939999999999999},"min":{"1min":0.746,"5min":0.746,"15min":0.746},"max":{"1min":1.6639999999999999,"5min":1.798,"15min":1.899},"last":1.8540000000000001},{"interface":"front","average":{"1min":1.278,"5min":1.389,"15min":1.4370000000000001},"min":{"1min":0.66400000000000003,"5min":0.66400000000000003,"15min":0.66400000000000003},"max":{"1min":1.6559999999999999,"5min":1.8819999999999999,"15min":2.0379999999999998},"last":1.8049999999999999}]},{"osd":4,"last update":"Wed Aug 9 15:11:47 2023","interfaces":[{"interface":"back","average":{"1min":1.347,"5min":1.323,"15min":1.3220000000000001},"min":{"1min":0.53000000000000003,"5min":0.53000000000000003,"15min":0.53000000000000003},"max":{"1min":1.833,"5min":2.0299999999999998,"15min":2.0299999999999998},"last":1.6499999999999999},{"interface":"front","average":{"1min":1.4419999999999999,"5min":1.389,"15min":1.4059999999999999},"min":{"1min":0.69199999999999995,"5min":0.69199999999999995,"15min":0.69199999999999995},"max":{"1min":1.7709999999999999,"5min":1.9239999999999999,"15min":2.0390000000000001},"last":1.746}]},{"osd":6,"last update":"Wed Aug 9 15:11:47 2023","interfaces":[{"interface":"back","average":{"1min":1.298,"5min":1.3660000000000001,"15min":1.411},"min":{"1min":0.501,"5min":0.501,"15min":0.501},"max":{"1min":1.728,"5min":2.032,"15min":2.1000000000000001},"last":1.8839999999999999},{"interface":"front","average":{"1min":1.48,"5min":1.4850000000000001,"15min":1.5029999999999999},"min":{"1min":0.70599999999999996,"5min":0.70599999999999996,"15min":0.70599999999999996},"max":{"1min":2.125,"5min":2.1349999999999998,"15min":2.1349999999999998},"last":2.012}]},{"osd":7,"last update":"Wed Aug 9 15:11:47 2023","interfaces":[{"interface":"back","average":{"1min":1.5469999999999999,"5min":1.554,"15min":1.5800000000000001},"min":{"1min":0.71999999999999997,"5min":0.71999999999999997,"15min":0.71999999999999997},"max":{"1min":2.0760000000000001,"5min":2.0760000000000001,"15min":2.4689999999999999},"last":1.9610000000000001},{"interface":"front","average":{"1min":1.5309999999999999,"5min":1.595,"15min":1.637},"min":{"1min":0.63800000000000001,"5min":0.63800000000000001,"15min":0.63800000000000001},"max":{"1min":2.1709999999999998,"5min":2.1709999999999998,"15min":2.4239999999999999},"last":1.9370000000000001}]}]},{"osd":4,"up_from":13,"seq":55834576020,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":52292,"kb_used_data":3124,"kb_used_omap":1,"kb_used_meta":49150,"kb_avail":94319548,"statfs":{"total":96636764160,"available":96583217152,"internally_reserved":0,"allocated":3198976,"data_stored":1893648,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1781,"internal_metadata":50329867},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Aug 9 15:11:11 2023","interfaces":[{"interface":"back","average":{"1min":1.3089999999999999,"5min":1.1970000000000001,"15min":1.226},"min":{"1min":0.81399999999999995,"5min":0.60399999999999998,"15min":0.60399999999999998},"max":{"1min":1.708,"5min":1.708,"15min":2.2959999999999998},"last":1.131},{"interface":"front","average":{"1min":1.3540000000000001,"5min":1.3009999999999999,"15min":1.3200000000000001},"min":{"1min":0.85699999999999998,"5min":0.66800000000000004,"15min":0.66800000000000004},"max":{"1min":2.0779999999999998,"5min":2.0779999999999998,"15min":2.2709999999999999},"last":0.89800000000000002}]},{"osd":1,"last update":"Wed Aug 9 15:11:11 2023","interfaces":[{"interface":"back","average":{"1min":1.3480000000000001,"5min":1.3580000000000001,"15min":1.355},"min":{"1min":0.95899999999999996,"5min":0.74199999999999999,"15min":0.74199999999999999},"max":{"1min":1.911,"5min":1.931,"15min":1.931},"last":1.016},{"interface":"front","average":{"1min":1.306,"5min":1.258,"15min":1.282},"min":{"1min":0.74299999999999999,"5min":0.73899999999999999,"15min":0.73899999999999999},"max":{"1min":1.6890000000000001,"5min":1.8600000000000001,"15min":2.117},"last":0.82799999999999996}]},{"osd":2,"last update":"Wed Aug 9 15:11:11 2023","interfaces":[{"interface":"back","average":{"1min":1.27,"5min":1.246,"15min":1.2929999999999999},"min":{"1min":0.63100000000000001,"5min":0.63100000000000001,"15min":0.63100000000000001},"max":{"1min":1.7709999999999999,"5min":1.7709999999999999,"15min":1.883},"last":1.2849999999999999},{"interface":"front","average":{"1min":1.4239999999999999,"5min":1.415,"15min":1.409},"min":{"1min":0.88700000000000001,"5min":0.79200000000000004,"15min":0.79200000000000004},"max":{"1min":2.306,"5min":2.343,"15min":2.343},"last":1.2130000000000001}]},{"osd":3,"last update":"Wed Aug 9 15:11:11 2023","interfaces":[{"interface":"back","average":{"1min":1.411,"5min":1.3580000000000001,"15min":1.357},"min":{"1min":0.93899999999999995,"5min":0.88700000000000001,"15min":0.88700000000000001},"max":{"1min":1.8009999999999999,"5min":1.8009999999999999,"15min":2.4870000000000001},"last":0.96699999999999997},{"interface":"front","average":{"1min":1.4279999999999999,"5min":1.4390000000000001,"15min":1.4690000000000001},"min":{"1min":0.79700000000000004,"5min":0.79700000000000004,"15min":0.79700000000000004},"max":{"1min":1.8660000000000001,"5min":1.909,"15min":2.5459999999999998},"last":1.486}]},{"osd":5,"last update":"Wed Aug 9 15:11:11 2023","interfaces":[{"interface":"back","average":{"1min":1.3120000000000001,"5min":1.3620000000000001,"15min":1.3999999999999999},"min":{"1min":0.82199999999999995,"5min":0.82199999999999995,"15min":0.82199999999999995},"max":{"1min":2.633,"5min":2.633,"15min":2.633},"last":1.3260000000000001},{"interface":"front","average":{"1min":1.355,"5min":1.4079999999999999,"15min":1.393},"min":{"1min":0.68100000000000005,"5min":0.68100000000000005,"15min":0.68100000000000005},"max":{"1min":1.9930000000000001,"5min":1.9930000000000001,"15min":2.4249999999999998},"last":1.508}]},{"osd":6,"last update":"Wed Aug 9 15:11:11 2023","interfaces":[{"interface":"back","average":{"1min":1.4119999999999999,"5min":1.4099999999999999,"15min":1.4139999999999999},"min":{"1min":0.71299999999999997,"5min":0.71299999999999997,"15min":0.71299999999999997},"max":{"1min":1.927,"5min":2.0049999999999999,"15min":2.9119999999999999},"last":1.401},{"interface":"front","average":{"1min":1.571,"5min":1.5840000000000001,"15min":1.5800000000000001},"min":{"1min":0.77000000000000002,"5min":0.77000000000000002,"15min":0.77000000000000002},"max":{"1min":2.9529999999999998,"5min":2.9529999999999998,"15min":2.9529999999999998},"last":1.4510000000000001}]},{"osd":7,"last update":"Wed Aug 9 15:11:11 2023","interfaces":[{"interface":"back","average":{"1min":1.5,"5min":1.52,"15min":1.53},"min":{"1min":0.83899999999999997,"5min":0.83899999999999997,"15min":0.83899999999999997},"max":{"1min":2.1589999999999998,"5min":2.1589999999999998,"15min":2.5910000000000002},"last":1.5940000000000001},{"interface":"front","average":{"1min":1.6419999999999999,"5min":1.5800000000000001,"15min":1.609},"min":{"1min":0.90000000000000002,"5min":0.90000000000000002,"15min":0.90000000000000002},"max":{"1min":2.218,"5min":2.218,"15min":3.028},"last":1.5509999999999999}]}]},{"osd":3,"up_from":13,"seq":55834576021,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":48168,"kb_used_data":3100,"kb_used_omap":1,"kb_used_meta":45054,"kb_avail":94323672,"statfs":{"total":96636764160,"available":96587440128,"internally_reserved":0,"allocated":3174400,"data_stored":1873580,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1781,"internal_metadata":46135563},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Aug 9 15:12:06 2023","interfaces":[{"interface":"back","average":{"1min":0.75800000000000001,"5min":0.94699999999999995,"15min":0.96899999999999997},"min":{"1min":0.42799999999999999,"5min":0.42799999999999999,"15min":0.42799999999999999},"max":{"1min":1.24,"5min":1.6699999999999999,"15min":2.2090000000000001},"last":0.70199999999999996},{"interface":"front","average":{"1min":0.872,"5min":1.093,"15min":1.1439999999999999},"min":{"1min":0.434,"5min":0.434,"15min":0.434},"max":{"1min":1.4550000000000001,"5min":1.9630000000000001,"15min":1.9630000000000001},"last":0.75900000000000001}]},{"osd":1,"last update":"Wed Aug 9 15:12:06 2023","interfaces":[{"interface":"back","average":{"1min":0.90500000000000003,"5min":1.006,"15min":1.0660000000000001},"min":{"1min":0.437,"5min":0.437,"15min":0.437},"max":{"1min":1.595,"5min":1.956,"15min":2.27},"last":1.1850000000000001},{"interface":"front","average":{"1min":1.0960000000000001,"5min":1.2070000000000001,"15min":1.2569999999999999},"min":{"1min":0.63900000000000001,"5min":0.498,"15min":0.498},"max":{"1min":1.6140000000000001,"5min":1.78,"15min":1.9730000000000001},"last":1.0169999999999999}]},{"osd":2,"last update":"Wed Aug 9 15:12:06 2023","interfaces":[{"interface":"back","average":{"1min":1.024,"5min":1.236,"15min":1.3180000000000001},"min":{"1min":0.65400000000000003,"5min":0.65400000000000003,"15min":0.65400000000000003},"max":{"1min":1.3839999999999999,"5min":1.9650000000000001,"15min":2.4079999999999999},"last":0.83799999999999997},{"interface":"front","average":{"1min":1.2250000000000001,"5min":1.3720000000000001,"15min":1.4279999999999999},"min":{"1min":0.63400000000000001,"5min":0.63400000000000001,"15min":0.63400000000000001},"max":{"1min":1.7090000000000001,"5min":2.0510000000000002,"15min":2.1259999999999999},"last":1.3720000000000001}]},{"osd":4,"last update":"Wed Aug 9 15:12:06 2023","interfaces":[{"interface":"back","average":{"1min":1.2070000000000001,"5min":1.3260000000000001,"15min":1.3959999999999999},"min":{"1min":0.752,"5min":0.748,"15min":0.72599999999999998},"max":{"1min":1.744,"5min":1.9179999999999999,"15min":2.2069999999999999},"last":1.5529999999999999},{"interface":"front","average":{"1min":1.3220000000000001,"5min":1.4199999999999999,"15min":1.4570000000000001},"min":{"1min":0.77500000000000002,"5min":0.77500000000000002,"15min":0.77500000000000002},"max":{"1min":1.8939999999999999,"5min":1.9279999999999999,"15min":2.056},"last":1.319}]},{"osd":5,"last update":"Wed Aug 9 15:12:06 2023","interfaces":[{"interface":"back","average":{"1min":1.4039999999999999,"5min":1.502,"15min":1.538},"min":{"1min":0.77600000000000002,"5min":0.77600000000000002,"15min":0.77600000000000002},"max":{"1min":2.1890000000000001,"5min":2.1890000000000001,"15min":2.1890000000000001},"last":1.4379999999999999},{"interface":"front","average":{"1min":1.3740000000000001,"5min":1.5580000000000001,"15min":1.6000000000000001},"min":{"1min":0.88800000000000001,"5min":0.85799999999999998,"15min":0.85799999999999998},"max":{"1min":1.7330000000000001,"5min":2.2469999999999999,"15min":2.4049999999999998},"last":1.2470000000000001}]},{"osd":6,"last update":"Wed Aug 9 15:12:06 2023","interfaces":[{"interface":"back","average":{"1min":1.3560000000000001,"5min":1.4890000000000001,"15min":1.5780000000000001},"min":{"1min":0.69599999999999995,"5min":0.69599999999999995,"15min":0.69599999999999995},"max":{"1min":1.9670000000000001,"5min":2.4860000000000002,"15min":2.4860000000000002},"last":1.649},{"interface":"front","average":{"1min":1.458,"5min":1.5940000000000001,"15min":1.653},"min":{"1min":0.89700000000000002,"5min":0.89700000000000002,"15min":0.89700000000000002},"max":{"1min":2.04,"5min":2.4039999999999999,"15min":2.4039999999999999},"last":1.8169999999999999}]},{"osd":7,"last update":"Wed Aug 9 15:12:06 2023","interfaces":[{"interface":"back","average":{"1min":1.5620000000000001,"5min":1.696,"15min":1.7490000000000001},"min":{"1min":0.94799999999999995,"5min":0.94799999999999995,"15min":0.94799999999999995},"max":{"1min":1.8680000000000001,"5min":2.347,"15min":2.347},"last":1.4930000000000001},{"interface":"front","average":{"1min":1.4179999999999999,"5min":1.6220000000000001,"15min":1.694},"min":{"1min":0.81100000000000005,"5min":0.81100000000000005,"15min":0.81100000000000005},"max":{"1min":2.1309999999999998,"5min":3.3220000000000001,"15min":3.3220000000000001},"last":1.7689999999999999}]}]},{"osd":2,"up_from":13,"seq":55834576020,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":46172,"kb_used_data":2764,"kb_used_omap":1,"kb_used_meta":43390,"kb_avail":94325668,"statfs":{"total":96636764160,"available":96589484032,"internally_reserved":0,"allocated":2830336,"data_stored":1545166,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1781,"internal_metadata":44431627},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Aug 9 15:11:52 2023","interfaces":[{"interface":"back","average":{"1min":0.874,"5min":0.98499999999999999,"15min":0.95899999999999996},"min":{"1min":0.48199999999999998,"5min":0.44800000000000001,"15min":0.44800000000000001},"max":{"1min":1.591,"5min":1.591,"15min":1.591},"last":0.71799999999999997},{"interface":"front","average":{"1min":0.96399999999999997,"5min":1.0760000000000001,"15min":1.085},"min":{"1min":0.38900000000000001,"5min":0.38900000000000001,"15min":0.38900000000000001},"max":{"1min":1.5980000000000001,"5min":1.764,"15min":2.052},"last":0.91800000000000004}]},{"osd":1,"last update":"Wed Aug 9 15:11:52 2023","interfaces":[{"interface":"back","average":{"1min":0.98299999999999998,"5min":1.2330000000000001,"15min":1.2230000000000001},"min":{"1min":0.55400000000000005,"5min":0.55400000000000005,"15min":0.55400000000000005},"max":{"1min":1.403,"5min":2.0179999999999998,"15min":2.0179999999999998},"last":1.2450000000000001},{"interface":"front","average":{"1min":1.1359999999999999,"5min":1.292,"15min":1.381},"min":{"1min":0.51200000000000001,"5min":0.51200000000000001,"15min":0.51200000000000001},"max":{"1min":1.6639999999999999,"5min":1.95,"15min":1.95},"last":1.0920000000000001}]},{"osd":3,"last update":"Wed Aug 9 15:11:52 2023","interfaces":[{"interface":"back","average":{"1min":0.99099999999999999,"5min":1.089,"15min":1.1080000000000001},"min":{"1min":0.53500000000000003,"5min":0.52100000000000002,"15min":0.52100000000000002},"max":{"1min":1.5649999999999999,"5min":1.871,"15min":1.871},"last":0.876},{"interface":"front","average":{"1min":1.111,"5min":1.339,"15min":1.369},"min":{"1min":0.57099999999999995,"5min":0.57099999999999995,"15min":0.57099999999999995},"max":{"1min":1.6930000000000001,"5min":1.9299999999999999,"15min":2.0470000000000002},"last":1.028}]},{"osd":4,"last update":"Wed Aug 9 15:11:52 2023","interfaces":[{"interface":"back","average":{"1min":1.369,"5min":1.5229999999999999,"15min":1.5469999999999999},"min":{"1min":0.86399999999999999,"5min":0.86399999999999999,"15min":0.86399999999999999},"max":{"1min":1.7350000000000001,"5min":1.994,"15min":2.464},"last":1.534},{"interface":"front","average":{"1min":1.395,"5min":1.4399999999999999,"15min":1.464},"min":{"1min":0.93600000000000005,"5min":0.81799999999999995,"15min":0.81799999999999995},"max":{"1min":1.9370000000000001,"5min":1.98,"15min":2.0720000000000001},"last":1.2090000000000001}]},{"osd":5,"last update":"Wed Aug 9 15:11:52 2023","interfaces":[{"interface":"back","average":{"1min":1.4279999999999999,"5min":1.5009999999999999,"15min":1.5169999999999999},"min":{"1min":0.64800000000000002,"5min":0.64800000000000002,"15min":0.64800000000000002},"max":{"1min":1.913,"5min":2.0369999999999999,"15min":2.1139999999999999},"last":1.1619999999999999},{"interface":"front","average":{"1min":1.351,"5min":1.419,"15min":1.4870000000000001},"min":{"1min":0.83399999999999996,"5min":0.83399999999999996,"15min":0.83399999999999996},"max":{"1min":2.1520000000000001,"5min":2.1520000000000001,"15min":2.214},"last":1.296}]},{"osd":6,"last update":"Wed Aug 9 15:11:52 2023","interfaces":[{"interface":"back","average":{"1min":1.407,"5min":1.4890000000000001,"15min":1.5229999999999999},"min":{"1min":0.89500000000000002,"5min":0.89500000000000002,"15min":0.89500000000000002},"max":{"1min":2.0680000000000001,"5min":2.1110000000000002,"15min":2.2999999999999998},"last":1.371},{"interface":"front","average":{"1min":1.512,"5min":1.6479999999999999,"15min":1.6379999999999999},"min":{"1min":0.998,"5min":0.998,"15min":0.998},"max":{"1min":2.4140000000000001,"5min":2.4140000000000001,"15min":2.4140000000000001},"last":1.7609999999999999}]},{"osd":7,"last update":"Wed Aug 9 15:11:52 2023","interfaces":[{"interface":"back","average":{"1min":1.5249999999999999,"5min":1.605,"15min":1.6419999999999999},"min":{"1min":0.96299999999999997,"5min":0.87,"15min":0.87},"max":{"1min":2.1000000000000001,"5min":2.1749999999999998,"15min":2.302},"last":1.6699999999999999},{"interface":"front","average":{"1min":1.4650000000000001,"5min":1.5800000000000001,"15min":1.6299999999999999},"min":{"1min":0.71399999999999997,"5min":0.71399999999999997,"15min":0.71399999999999997},"max":{"1min":2.2360000000000002,"5min":2.242,"15min":2.3809999999999998},"last":1.341}]}]},{"osd":1,"up_from":13,"seq":55834576020,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":56612,"kb_used_data":3352,"kb_used_omap":1,"kb_used_meta":53246,"kb_avail":94315228,"statfs":{"total":96636764160,"available":96578793472,"internally_reserved":0,"allocated":3432448,"data_stored":2135590,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1781,"internal_metadata":54524171},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Aug 9 15:12:03 2023","interfaces":[{"interface":"back","average":{"1min":0.83399999999999996,"5min":0.97399999999999998,"15min":1},"min":{"1min":0.32100000000000001,"5min":0.32100000000000001,"15min":0.32100000000000001},"max":{"1min":1.153,"5min":1.8799999999999999,"15min":2.0939999999999999},"last":0.97899999999999998},{"interface":"front","average":{"1min":0.94299999999999995,"5min":1.1060000000000001,"15min":1.1399999999999999},"min":{"1min":0.40500000000000003,"5min":0.40500000000000003,"15min":0.40500000000000003},"max":{"1min":1.641,"5min":1.78,"15min":1.8939999999999999},"last":1.014}]},{"osd":2,"last update":"Wed Aug 9 15:12:03 2023","interfaces":[{"interface":"back","average":{"1min":0.93700000000000006,"5min":1.018,"15min":1.04},"min":{"1min":0.626,"5min":0.55800000000000005,"15min":0.55800000000000005},"max":{"1min":2.2240000000000002,"5min":2.2240000000000002,"15min":2.3940000000000001},"last":0.77700000000000002},{"interface":"front","average":{"1min":0.97399999999999998,"5min":1.1759999999999999,"15min":1.254},"min":{"1min":0.49299999999999999,"5min":0.46700000000000003,"15min":0.46700000000000003},"max":{"1min":1.5149999999999999,"5min":2.169,"15min":2.169},"last":1.425}]},{"osd":3,"last update":"Wed Aug 9 15:12:03 2023","interfaces":[{"interface":"back","average":{"1min":1.115,"5min":1.3340000000000001,"15min":1.381},"min":{"1min":0.54600000000000004,"5min":0.51000000000000001,"15min":0.51000000000000001},"max":{"1min":1.833,"5min":2.093,"15min":2.109},"last":1.502},{"interface":"front","average":{"1min":1.0409999999999999,"5min":1.256,"15min":1.3089999999999999},"min":{"1min":0.59099999999999997,"5min":0.59099999999999997,"15min":0.59099999999999997},"max":{"1min":1.8680000000000001,"5min":2.052,"15min":2.052},"last":1.1719999999999999}]},{"osd":4,"last update":"Wed Aug 9 15:12:03 2023","interfaces":[{"interface":"back","average":{"1min":1.254,"5min":1.4339999999999999,"15min":1.4730000000000001},"min":{"1min":0.82099999999999995,"5min":0.753,"15min":0.753},"max":{"1min":1.887,"5min":2.0419999999999998,"15min":2.214},"last":1.5489999999999999},{"interface":"front","average":{"1min":1.202,"5min":1.381,"15min":1.4119999999999999},"min":{"1min":0.84499999999999997,"5min":0.69199999999999995,"15min":0.69199999999999995},"max":{"1min":1.528,"5min":2.0310000000000001,"15min":2.1190000000000002},"last":1.2270000000000001}]},{"osd":5,"last update":"Wed Aug 9 15:12:03 2023","interfaces":[{"interface":"back","average":{"1min":1.2450000000000001,"5min":1.415,"15min":1.446},"min":{"1min":0.83399999999999996,"5min":0.77800000000000002,"15min":0.77800000000000002},"max":{"1min":1.909,"5min":1.994,"15min":2.508},"last":1.1100000000000001},{"interface":"front","average":{"1min":1.2829999999999999,"5min":1.53,"15min":1.5800000000000001},"min":{"1min":0.74299999999999999,"5min":0.74299999999999999,"15min":0.74299999999999999},"max":{"1min":1.99,"5min":2.3809999999999998,"15min":2.3809999999999998},"last":1.262}]},{"osd":6,"last update":"Wed Aug 9 15:12:03 2023","interfaces":[{"interface":"back","average":{"1min":1.321,"5min":1.575,"15min":1.613},"min":{"1min":0.80100000000000005,"5min":0.80100000000000005,"15min":0.80100000000000005},"max":{"1min":2.1669999999999998,"5min":2.347,"15min":2.347},"last":1.3089999999999999},{"interface":"front","average":{"1min":1.288,"5min":1.522,"15min":1.5720000000000001},"min":{"1min":0.84999999999999998,"5min":0.73499999999999999,"15min":0.73499999999999999},"max":{"1min":1.8140000000000001,"5min":2.218,"15min":2.278},"last":1.3360000000000001}]},{"osd":7,"last update":"Wed Aug 9 15:12:03 2023","interfaces":[{"interface":"back","average":{"1min":1.3959999999999999,"5min":1.6439999999999999,"15min":1.6910000000000001},"min":{"1min":0.90900000000000003,"5min":0.86399999999999999,"15min":0.86399999999999999},"max":{"1min":2.1299999999999999,"5min":2.3090000000000002,"15min":2.488},"last":1.5880000000000001},{"interface":"front","average":{"1min":1.385,"5min":1.633,"15min":1.698},"min":{"1min":0.75600000000000001,"5min":0.75600000000000001,"15min":0.75600000000000001},"max":{"1min":2.4980000000000002,"5min":2.4980000000000002,"15min":2.4980000000000002},"last":1.478}]}]},{"osd":0,"up_from":13,"seq":55834576020,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":29796,"kb_used_data":2776,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94342044,"statfs":{"total":96636764160,"available":96606253056,"internally_reserved":0,"allocated":2842624,"data_stored":1551495,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1781,"internal_metadata":27654411},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Wed Aug 9 15:12:00 2023","interfaces":[{"interface":"back","average":{"1min":0.85099999999999998,"5min":0.89600000000000002,"15min":0.95099999999999996},"min":{"1min":0.44900000000000001,"5min":0.40000000000000002,"15min":0.40000000000000002},"max":{"1min":1.3380000000000001,"5min":1.617,"15min":1.6299999999999999},"last":0.58499999999999996},{"interface":"front","average":{"1min":0.98799999999999999,"5min":1.091,"15min":1.135},"min":{"1min":0.56599999999999995,"5min":0.56599999999999995,"15min":0.56599999999999995},"max":{"1min":1.635,"5min":2.1560000000000001,"15min":2.1560000000000001},"last":0.86399999999999999}]},{"osd":2,"last update":"Wed Aug 9 15:12:00 2023","interfaces":[{"interface":"back","average":{"1min":0.873,"5min":1.097,"15min":1.1719999999999999},"min":{"1min":0.54000000000000004,"5min":0.505,"15min":0.505},"max":{"1min":1.401,"5min":2.4769999999999999,"15min":2.4769999999999999},"last":0.61699999999999999},{"interface":"front","average":{"1min":1.0069999999999999,"5min":1.125,"15min":1.2150000000000001},"min":{"1min":0.55500000000000005,"5min":0.53100000000000003,"15min":0.53100000000000003},"max":{"1min":1.468,"5min":1.8180000000000001,"15min":1.9470000000000001},"last":1.0820000000000001}]},{"osd":3,"last update":"Wed Aug 9 15:12:00 2023","interfaces":[{"interface":"back","average":{"1min":0.91200000000000003,"5min":1.1040000000000001,"15min":1.1870000000000001},"min":{"1min":0.45500000000000002,"5min":0.45500000000000002,"15min":0.45500000000000002},"max":{"1min":1.629,"5min":2.1459999999999999,"15min":2.1459999999999999},"last":1.0509999999999999},{"interface":"front","average":{"1min":1.079,"5min":1.284,"15min":1.3779999999999999},"min":{"1min":0.53100000000000003,"5min":0.53100000000000003,"15min":0.53100000000000003},"max":{"1min":1.5640000000000001,"5min":1.867,"15min":2.0760000000000001},"last":0.53500000000000003}]},{"osd":4,"last update":"Wed Aug 9 15:12:00 2023","interfaces":[{"interface":"back","average":{"1min":1.163,"5min":1.321,"15min":1.403},"min":{"1min":0.81799999999999995,"5min":0.81799999999999995,"15min":0.81799999999999995},"max":{"1min":1.5629999999999999,"5min":1.853,"15min":1.966},"last":1.2889999999999999},{"interface":"front","average":{"1min":1.296,"5min":1.421,"15min":1.5069999999999999},"min":{"1min":0.83099999999999996,"5min":0.79400000000000004,"15min":0.79400000000000004},"max":{"1min":1.734,"5min":1.972,"15min":2.278},"last":1.2529999999999999}]},{"osd":5,"last update":"Wed Aug 9 15:12:00 2023","interfaces":[{"interface":"back","average":{"1min":1.256,"5min":1.476,"15min":1.5129999999999999},"min":{"1min":0.84699999999999998,"5min":0.84699999999999998,"15min":0.84699999999999998},"max":{"1min":1.79,"5min":2.355,"15min":2.355},"last":1.3480000000000001},{"interface":"front","average":{"1min":1.2849999999999999,"5min":1.4299999999999999,"15min":1.49},"min":{"1min":0.84799999999999998,"5min":0.82899999999999996,"15min":0.82899999999999996},"max":{"1min":2.089,"5min":2.089,"15min":2.3740000000000001},"last":1.5029999999999999}]},{"osd":6,"last update":"Wed Aug 9 15:12:00 2023","interfaces":[{"interface":"back","average":{"1min":1.3129999999999999,"5min":1.5249999999999999,"15min":1.6160000000000001},"min":{"1min":0.79700000000000004,"5min":0.79700000000000004,"15min":0.79700000000000004},"max":{"1min":1.7889999999999999,"5min":2.7490000000000001,"15min":2.7490000000000001},"last":1.3160000000000001},{"interface":"front","average":{"1min":1.3839999999999999,"5min":1.534,"15min":1.597},"min":{"1min":0.89200000000000002,"5min":0.84699999999999998,"15min":0.84699999999999998},"max":{"1min":2.0840000000000001,"5min":2.274,"15min":2.2839999999999998},"last":1.4550000000000001}]},{"osd":7,"last update":"Wed Aug 9 15:12:00 2023","interfaces":[{"interface":"back","average":{"1min":1.4710000000000001,"5min":1.599,"15min":1.665},"min":{"1min":0.98599999999999999,"5min":0.94899999999999995,"15min":0.94899999999999995},"max":{"1min":2.2989999999999999,"5min":2.645,"15min":2.645},"last":1.5620000000000001},{"interface":"front","average":{"1min":1.4019999999999999,"5min":1.54,"15min":1.609},"min":{"1min":0.82599999999999996,"5min":0.82599999999999996,"15min":0.82599999999999996},"max":{"1min":2.1829999999999998,"5min":2.1829999999999998,"15min":2.3759999999999999},"last":1.395}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-08-09T15:12:19.185 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-08-09T15:12:19.412 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:19.411+0000 7f27e8776640 1 Processor -- start 2023-08-09T15:12:19.413 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:19.411+0000 7f27e8776640 1 -- start start 2023-08-09T15:12:19.414 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:19.412+0000 7f27e8776640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f27e006fe60 0x7f27e00cdbe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T15:12:19.414 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:19.412+0000 7f27e8776640 1 -- --> v1:172.21.15.43:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f27e006c2c0 con 0x7f27e015b310 2023-08-09T15:12:19.414 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:19.412+0000 7f27e8776640 1 -- --> v1:172.21.15.203:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f27e00cb690 con 0x7f27e006b4b0 2023-08-09T15:12:19.414 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:19.412+0000 7f27e8776640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f27e006ba20 con 0x7f27e006fe60 2023-08-09T15:12:19.415 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:19.413+0000 7f27e6d13640 1 --1- >> v1:172.21.15.43:6789/0 conn(0x7f27e015b310 0x7f27e00afc70 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.43:6789/0 says I am v1:172.21.15.43:32780/0 (socket says 172.21.15.43:32780) 2023-08-09T15:12:19.416 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:19.413+0000 7f27e6d13640 1 -- 172.21.15.43:0/1054526857 learned_addr learned my addr 172.21.15.43:0/1054526857 (peer_addr_for_me v1:172.21.15.43:0/0) 2023-08-09T15:12:19.416 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:19.413+0000 7f27e6512640 1 --2- 172.21.15.43:0/1054526857 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f27e006fe60 0x7f27e00cdbe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T15:12:19.416 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:19.413+0000 7f27e5510640 1 -- 172.21.15.43:0/1054526857 <== mon.0 v1:172.21.15.43:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2015763377 0 0) 0x7f27e006c2c0 con 0x7f27e015b310 2023-08-09T15:12:19.416 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:19.413+0000 7f27e5510640 1 -- 172.21.15.43:0/1054526857 --> v1:172.21.15.43:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f27cc003520 con 0x7f27e015b310 2023-08-09T15:12:19.416 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:19.414+0000 7f27e5510640 1 -- 172.21.15.43:0/1054526857 <== mon.0 v1:172.21.15.43:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3834401181 0 0) 0x7f27cc003520 con 0x7f27e015b310 2023-08-09T15:12:19.416 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:19.414+0000 7f27e5510640 1 -- 172.21.15.43:0/1054526857 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f27e006fe60 msgr2=0x7f27e00cdbe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-09T15:12:19.416 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:19.414+0000 7f27e5510640 1 --2- 172.21.15.43:0/1054526857 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f27e006fe60 0x7f27e00cdbe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T15:12:19.417 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:19.414+0000 7f27e5510640 1 -- 172.21.15.43:0/1054526857 >> v1:172.21.15.203:6789/0 conn(0x7f27e006b4b0 legacy=0x7f27e015be10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T15:12:19.417 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:19.414+0000 7f27e5510640 1 -- 172.21.15.43:0/1054526857 --> v1:172.21.15.43:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f27e006bd00 con 0x7f27e015b310 2023-08-09T15:12:19.417 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:19.414+0000 7f27e5510640 1 -- 172.21.15.43:0/1054526857 <== mon.0 v1:172.21.15.43:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1285150740 0 0) 0x7f27d8001670 con 0x7f27e015b310 2023-08-09T15:12:19.417 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:19.414+0000 7f27e5510640 1 -- 172.21.15.43:0/1054526857 >> v1:172.21.15.43:6789/0 conn(0x7f27e015b310 legacy=0x7f27e00afc70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T15:12:19.417 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:19.414+0000 7f27e5510640 1 --2- 172.21.15.43:0/1054526857 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f27cc0039a0 0x7f27cc003dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T15:12:19.417 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:19.414+0000 7f27e5510640 1 --2- 172.21.15.43:0/1054526857 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f27e006fe60 0x7f27cc0047c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T15:12:19.417 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:19.414+0000 7f27e5510640 1 --2- 172.21.15.43:0/1054526857 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f27cc004d00 0x7f27cc005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T15:12:19.417 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:19.414+0000 7f27e5510640 1 -- 172.21.15.43:0/1054526857 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f27e006ba20 con 0x7f27cc004d00 2023-08-09T15:12:19.418 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:19.414+0000 7f27e5510640 1 -- 172.21.15.43:0/1054526857 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f27cc00ba60 con 0x7f27cc0039a0 2023-08-09T15:12:19.418 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:19.414+0000 7f27e5510640 1 -- 172.21.15.43:0/1054526857 --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f27cc00bc20 con 0x7f27e006fe60 2023-08-09T15:12:19.418 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:19.414+0000 7f27e6d13640 1 --2- 172.21.15.43:0/1054526857 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f27cc004d00 0x7f27cc005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T15:12:19.419 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:19.415+0000 7f27e6512640 1 --2- 172.21.15.43:0/1054526857 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f27cc0039a0 0x7f27cc003dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T15:12:19.419 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:19.415+0000 7f27e5d11640 1 --2- 172.21.15.43:0/1054526857 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f27e006fe60 0x7f27cc0047c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T15:12:19.419 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:19.415+0000 7f27e6d13640 1 -- 172.21.15.43:0/1054526857 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f27e006fe60 msgr2=0x7f27cc0047c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-09T15:12:19.420 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:19.415+0000 7f27e6d13640 1 --2- 172.21.15.43:0/1054526857 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f27e006fe60 0x7f27cc0047c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T15:12:19.420 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:19.415+0000 7f27e6d13640 1 -- 172.21.15.43:0/1054526857 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f27cc0039a0 msgr2=0x7f27cc003dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-09T15:12:19.420 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:19.415+0000 7f27e6d13640 1 --2- 172.21.15.43:0/1054526857 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f27cc0039a0 0x7f27cc003dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T15:12:19.420 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:19.415+0000 7f27e6d13640 1 -- 172.21.15.43:0/1054526857 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f27cc003520 con 0x7f27cc004d00 2023-08-09T15:12:19.420 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:19.415+0000 7f27e6512640 1 --2- 172.21.15.43:0/1054526857 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f27cc0039a0 0x7f27cc003dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-08-09T15:12:19.420 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:19.416+0000 7f27e6d13640 1 --2- 172.21.15.43:0/1054526857 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f27cc004d00 0x7f27cc005180 secure :-1 s=READY pgs=2554 cs=0 l=1 rev1=1 crypto rx=0x7f27d8001eb0 tx=0x7f27cc0057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bb4c8c79d005d694 server_cookie=0 in_seq=0 out_seq=0 2023-08-09T15:12:19.420 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:19.416+0000 7f27e5510640 1 -- 172.21.15.43:0/1054526857 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f27d8005ad0 con 0x7f27cc004d00 2023-08-09T15:12:19.421 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:19.416+0000 7f27e5510640 1 -- 172.21.15.43:0/1054526857 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f27d8005c30 con 0x7f27cc004d00 2023-08-09T15:12:19.421 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:19.416+0000 7f27e5510640 1 -- 172.21.15.43:0/1054526857 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f27d802c7c0 con 0x7f27cc004d00 2023-08-09T15:12:19.421 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:19.416+0000 7f27e8776640 1 -- 172.21.15.43:0/1054526857 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f27cc004d00 msgr2=0x7f27cc005180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T15:12:19.421 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:19.416+0000 7f27e8776640 1 --2- 172.21.15.43:0/1054526857 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f27cc004d00 0x7f27cc005180 secure :-1 s=READY pgs=2554 cs=0 l=1 rev1=1 crypto rx=0x7f27d8001eb0 tx=0x7f27cc0057f0 comp rx=0 tx=0).stop 2023-08-09T15:12:19.421 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:19.416+0000 7f27e6d13640 1 -- 172.21.15.43:0/1054526857 reap_dead start 2023-08-09T15:12:19.421 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:19.417+0000 7f27e8776640 1 -- 172.21.15.43:0/1054526857 shutdown_connections 2023-08-09T15:12:19.422 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:19.417+0000 7f27e8776640 1 -- 172.21.15.43:0/1054526857 >> 172.21.15.43:0/1054526857 conn(0x7f27e0069d70 msgr2=0x7f27e00cac70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-09T15:12:19.422 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:19.417+0000 7f27e8776640 1 -- 172.21.15.43:0/1054526857 shutdown_connections 2023-08-09T15:12:19.422 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:19.417+0000 7f27e8776640 1 -- 172.21.15.43:0/1054526857 wait complete. 2023-08-09T15:12:19.423 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:19.418+0000 7f27e8776640 1 Processor -- start 2023-08-09T15:12:19.423 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:19.419+0000 7f27e8776640 1 -- start start 2023-08-09T15:12:19.424 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:19.419+0000 7f27e8776640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f27cc004d00 0x7f27e0153300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T15:12:19.424 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:19.419+0000 7f27e8776640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f27e0154350 0x7f27e01547d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T15:12:19.424 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:19.420+0000 7f27e8776640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f27e00c1ce0 0x7f27e00c2160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T15:12:19.424 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:19.420+0000 7f27e8776640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f27e0153840 con 0x7f27e0154350 2023-08-09T15:12:19.424 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:19.420+0000 7f27e8776640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f27e00b20e0 con 0x7f27e00c1ce0 2023-08-09T15:12:19.424 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:19.420+0000 7f27e8776640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f27e0154d10 con 0x7f27cc004d00 2023-08-09T15:12:19.424 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:19.420+0000 7f27e5d11640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f27e0154350 0x7f27e01547d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T15:12:19.425 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:19.420+0000 7f27e5d11640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f27e0154350 0x7f27e01547d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:46250/0 (socket says 172.21.15.43:46250) 2023-08-09T15:12:19.425 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:19.420+0000 7f27e5d11640 1 -- 172.21.15.43:0/4127648905 learned_addr learned my addr 172.21.15.43:0/4127648905 (peer_addr_for_me v2:172.21.15.43:0/0) 2023-08-09T15:12:19.425 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:19.420+0000 7f27e6d13640 1 --2- 172.21.15.43:0/4127648905 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f27e00c1ce0 0x7f27e00c2160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T15:12:19.425 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:19.420+0000 7f27e6512640 1 --2- 172.21.15.43:0/4127648905 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f27cc004d00 0x7f27e0153300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T15:12:19.425 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:19.420+0000 7f27e5d11640 1 -- 172.21.15.43:0/4127648905 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f27cc004d00 msgr2=0x7f27e0153300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T15:12:19.425 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:19.420+0000 7f27e5d11640 1 --2- 172.21.15.43:0/4127648905 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f27cc004d00 0x7f27e0153300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T15:12:19.425 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:19.420+0000 7f27e5d11640 1 -- 172.21.15.43:0/4127648905 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f27e00c1ce0 msgr2=0x7f27e00c2160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T15:12:19.426 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:19.420+0000 7f27e5d11640 1 --2- 172.21.15.43:0/4127648905 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f27e00c1ce0 0x7f27e00c2160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T15:12:19.426 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:19.420+0000 7f27e5d11640 1 -- 172.21.15.43:0/4127648905 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f27e00c26a0 con 0x7f27e0154350 2023-08-09T15:12:19.426 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:19.421+0000 7f27e5d11640 1 --2- 172.21.15.43:0/4127648905 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f27e0154350 0x7f27e01547d0 secure :-1 s=READY pgs=2555 cs=0 l=1 rev1=1 crypto rx=0x7f27d0001eb0 tx=0x7f27d000d9d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-09T15:12:19.426 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:19.421+0000 7f27d6ffd640 1 -- 172.21.15.43:0/4127648905 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f27d0005480 con 0x7f27e0154350 2023-08-09T15:12:19.426 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:19.421+0000 7f27d6ffd640 1 -- 172.21.15.43:0/4127648905 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f27d0003610 con 0x7f27e0154350 2023-08-09T15:12:19.426 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:19.421+0000 7f27d6ffd640 1 -- 172.21.15.43:0/4127648905 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f27d0003980 con 0x7f27e0154350 2023-08-09T15:12:19.426 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:19.421+0000 7f27e8776640 1 -- 172.21.15.43:0/4127648905 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f27e00c2930 con 0x7f27e0154350 2023-08-09T15:12:19.426 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:19.421+0000 7f27e8776640 1 -- 172.21.15.43:0/4127648905 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f27e00c2ef0 con 0x7f27e0154350 2023-08-09T15:12:19.427 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:19.422+0000 7f27e8776640 1 -- 172.21.15.43:0/4127648905 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f27e004db40 con 0x7f27e0154350 2023-08-09T15:12:19.427 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:19.423+0000 7f27d6ffd640 1 -- 172.21.15.43:0/4127648905 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 129) v1 ==== 88443+0+0 (secure 0 0 0) 0x7f27d0003ae0 con 0x7f27e0154350 2023-08-09T15:12:19.428 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:19.424+0000 7f27d6ffd640 1 --2- 172.21.15.43:0/4127648905 >> [v2:172.21.15.203:6832/1456699667,v1:172.21.15.203:6833/1456699667] conn(0x7f27c406a170 0x7f27c406c630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T15:12:19.428 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:19.424+0000 7f27d6ffd640 1 -- 172.21.15.43:0/4127648905 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(229..229 src has 1..229) v4 ==== 6462+0+0 (secure 0 0 0) 0x7f27d0088830 con 0x7f27e0154350 2023-08-09T15:12:19.428 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:19.425+0000 7f27e6512640 1 --2- 172.21.15.43:0/4127648905 >> [v2:172.21.15.203:6832/1456699667,v1:172.21.15.203:6833/1456699667] conn(0x7f27c406a170 0x7f27c406c630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T15:12:19.428 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:19.426+0000 7f27e6512640 1 --2- 172.21.15.43:0/4127648905 >> [v2:172.21.15.203:6832/1456699667,v1:172.21.15.203:6833/1456699667] conn(0x7f27c406a170 0x7f27c406c630 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7f27dc007440 tx=0x7f27dc00d040 comp rx=0 tx=0).ready entity=mgr.9232 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-09T15:12:19.430 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:19.429+0000 7f27d6ffd640 1 -- 172.21.15.43:0/4127648905 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178066 (secure 0 0 0) 0x7f27d0053c00 con 0x7f27e0154350 2023-08-09T15:12:19.617 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:19.616+0000 7f27e8776640 1 -- 172.21.15.43:0/4127648905 --> [v2:172.21.15.203:6832/1456699667,v1:172.21.15.203:6833/1456699667] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f27e015adf0 con 0x7f27c406a170 2023-08-09T15:12:19.625 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:19.624+0000 7f27d6ffd640 1 -- 172.21.15.43:0/4127648905 <== mgr.9232 v2:172.21.15.203:6832/1456699667 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+70673 (secure 0 0 0) 0x7f27dc011040 con 0x7f27c406a170 2023-08-09T15:12:19.626 INFO:teuthology.orchestra.run.smithi043.stdout: 2023-08-09T15:12:19.626 INFO:teuthology.orchestra.run.smithi043.stderr:dumped all 2023-08-09T15:12:19.627 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:19.626+0000 7f27e8776640 1 -- 172.21.15.43:0/4127648905 >> [v2:172.21.15.203:6832/1456699667,v1:172.21.15.203:6833/1456699667] conn(0x7f27c406a170 msgr2=0x7f27c406c630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T15:12:19.628 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:19.626+0000 7f27e8776640 1 --2- 172.21.15.43:0/4127648905 >> [v2:172.21.15.203:6832/1456699667,v1:172.21.15.203:6833/1456699667] conn(0x7f27c406a170 0x7f27c406c630 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7f27dc007440 tx=0x7f27dc00d040 comp rx=0 tx=0).stop 2023-08-09T15:12:19.628 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:19.626+0000 7f27e8776640 1 -- 172.21.15.43:0/4127648905 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f27e0154350 msgr2=0x7f27e01547d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T15:12:19.628 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:19.626+0000 7f27e8776640 1 --2- 172.21.15.43:0/4127648905 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f27e0154350 0x7f27e01547d0 secure :-1 s=READY pgs=2555 cs=0 l=1 rev1=1 crypto rx=0x7f27d0001eb0 tx=0x7f27d000d9d0 comp rx=0 tx=0).stop 2023-08-09T15:12:19.629 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:19.627+0000 7f27e8776640 1 -- 172.21.15.43:0/4127648905 shutdown_connections 2023-08-09T15:12:19.629 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:19.627+0000 7f27e8776640 1 --2- 172.21.15.43:0/4127648905 >> [v2:172.21.15.203:6832/1456699667,v1:172.21.15.203:6833/1456699667] conn(0x7f27c406a170 0x7f27c406c630 unknown :-1 s=CLOSED pgs=40 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T15:12:19.629 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:19.627+0000 7f27e8776640 1 --2- 172.21.15.43:0/4127648905 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f27e00c1ce0 0x7f27e00c2160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T15:12:19.629 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:19.627+0000 7f27e8776640 1 --2- 172.21.15.43:0/4127648905 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f27e0154350 0x7f27e01547d0 unknown :-1 s=CLOSED pgs=2555 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T15:12:19.629 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:19.627+0000 7f27e8776640 1 --2- 172.21.15.43:0/4127648905 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f27cc004d00 0x7f27e0153300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T15:12:19.629 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:19.627+0000 7f27e8776640 1 -- 172.21.15.43:0/4127648905 >> 172.21.15.43:0/4127648905 conn(0x7f27e0069d70 msgr2=0x7f27e0153ab0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-09T15:12:19.630 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:19.627+0000 7f27e8776640 1 -- 172.21.15.43:0/4127648905 shutdown_connections 2023-08-09T15:12:19.630 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:19.627+0000 7f27e8776640 1 -- 172.21.15.43:0/4127648905 wait complete. 2023-08-09T15:12:19.647 INFO:teuthology.orchestra.run.smithi043.stdout:{"pg_ready":true,"pg_map":{"version":13,"stamp":"2023-08-09T15:12:19.363975+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2155,"num_read_kb":4508,"num_write":583,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":351,"ondisk_log_size":351,"up":18,"acting":18,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":91,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":393136,"kb_used_data":23616,"kb_used_omap":13,"kb_used_meta":369394,"kb_avail":754581584,"statfs":{"total":773094113280,"available":772691542016,"internally_reserved":0,"allocated":24182784,"data_stored":13716990,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":14248,"internal_metadata":378259544},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":16,"apply_latency_ms":16,"commit_latency_ns":16000000,"apply_latency_ns":16000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-676856,"num_objects":-24,"num_object_clones":0,"num_object_copies":-48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-24,"num_whiteouts":0,"num_read":-310,"num_read_kb":-155,"num_write":-319,"num_write_kb":-2330,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":-14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.743755"},"pg_stats":[{"pgid":"1.0","version":"0'0","reported_seq":425,"reported_epoch":229,"state":"active+clean","last_fresh":"2023-08-09T15:12:15.642682+0000","last_change":"2023-08-09T13:33:56.401406+0000","last_active":"2023-08-09T15:12:15.642682+0000","last_peered":"2023-08-09T15:12:15.642682+0000","last_clean":"2023-08-09T15:12:15.642682+0000","last_became_active":"2023-08-09T13:33:56.400487+0000","last_became_peered":"2023-08-09T13:33:56.400487+0000","last_unstale":"2023-08-09T15:12:15.642682+0000","last_undegraded":"2023-08-09T15:12:15.642682+0000","last_fullsized":"2023-08-09T15:12:15.642682+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:33:54.845869+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:33:54.845869+0000","last_clean_scrub_stamp":"2023-08-09T13:33:54.845869+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T15:26:23.628880+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":432,"reported_epoch":229,"state":"active+clean","last_fresh":"2023-08-09T15:12:16.906007+0000","last_change":"2023-08-09T13:33:56.401123+0000","last_active":"2023-08-09T15:12:16.906007+0000","last_peered":"2023-08-09T15:12:16.906007+0000","last_clean":"2023-08-09T15:12:16.906007+0000","last_became_active":"2023-08-09T13:33:56.399933+0000","last_became_peered":"2023-08-09T13:33:56.399933+0000","last_unstale":"2023-08-09T15:12:16.906007+0000","last_undegraded":"2023-08-09T15:12:16.906007+0000","last_fullsized":"2023-08-09T15:12:16.906007+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:33:54.845869+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:33:54.845869+0000","last_clean_scrub_stamp":"2023-08-09T13:33:54.845869+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-11T00:04:51.245862+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.2","version":"0'0","reported_seq":424,"reported_epoch":229,"state":"active+clean","last_fresh":"2023-08-09T15:12:15.641337+0000","last_change":"2023-08-09T13:33:56.402794+0000","last_active":"2023-08-09T15:12:15.641337+0000","last_peered":"2023-08-09T15:12:15.641337+0000","last_clean":"2023-08-09T15:12:15.641337+0000","last_became_active":"2023-08-09T13:33:56.402038+0000","last_became_peered":"2023-08-09T13:33:56.402038+0000","last_unstale":"2023-08-09T15:12:15.641337+0000","last_undegraded":"2023-08-09T15:12:15.641337+0000","last_fullsized":"2023-08-09T15:12:15.641337+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:33:54.845869+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:33:54.845869+0000","last_clean_scrub_stamp":"2023-08-09T13:33:54.845869+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T20:39:01.972486+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.3","version":"0'0","reported_seq":438,"reported_epoch":229,"state":"active+clean","last_fresh":"2023-08-09T15:12:15.644104+0000","last_change":"2023-08-09T13:33:55.886749+0000","last_active":"2023-08-09T15:12:15.644104+0000","last_peered":"2023-08-09T15:12:15.644104+0000","last_clean":"2023-08-09T15:12:15.644104+0000","last_became_active":"2023-08-09T13:33:55.886116+0000","last_became_peered":"2023-08-09T13:33:55.886116+0000","last_unstale":"2023-08-09T15:12:15.644104+0000","last_undegraded":"2023-08-09T15:12:15.644104+0000","last_fullsized":"2023-08-09T15:12:15.644104+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:33:54.845869+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:33:54.845869+0000","last_clean_scrub_stamp":"2023-08-09T13:33:54.845869+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T19:17:22.620341+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"226'351","reported_seq":1404,"reported_epoch":229,"state":"active+clean","last_fresh":"2023-08-09T15:12:15.642688+0000","last_change":"2023-08-09T13:33:55.886345+0000","last_active":"2023-08-09T15:12:15.642688+0000","last_peered":"2023-08-09T15:12:15.642688+0000","last_clean":"2023-08-09T15:12:15.642688+0000","last_became_active":"2023-08-09T13:33:55.885781+0000","last_became_peered":"2023-08-09T13:33:55.885781+0000","last_unstale":"2023-08-09T15:12:15.642688+0000","last_undegraded":"2023-08-09T15:12:15.642688+0000","last_fullsized":"2023-08-09T15:12:15.642688+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:33:54.845869+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:33:54.845869+0000","last_clean_scrub_stamp":"2023-08-09T13:33:54.845869+0000","objects_scrubbed":0,"log_size":351,"log_dups_size":0,"ondisk_log_size":351,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T18:20:34.406519+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2155,"num_read_kb":4508,"num_write":583,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":442,"reported_epoch":229,"state":"active+clean","last_fresh":"2023-08-09T15:12:15.646193+0000","last_change":"2023-08-09T13:33:55.889565+0000","last_active":"2023-08-09T15:12:15.646193+0000","last_peered":"2023-08-09T15:12:15.646193+0000","last_clean":"2023-08-09T15:12:15.646193+0000","last_became_active":"2023-08-09T13:33:55.889164+0000","last_became_peered":"2023-08-09T13:33:55.889164+0000","last_unstale":"2023-08-09T15:12:15.646193+0000","last_undegraded":"2023-08-09T15:12:15.646193+0000","last_fullsized":"2023-08-09T15:12:15.646193+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:33:54.845869+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:33:54.845869+0000","last_clean_scrub_stamp":"2023-08-09T13:33:54.845869+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-11T01:05:24.824606+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":430,"reported_epoch":229,"state":"active+clean","last_fresh":"2023-08-09T15:12:17.176452+0000","last_change":"2023-08-09T13:33:56.404236+0000","last_active":"2023-08-09T15:12:17.176452+0000","last_peered":"2023-08-09T15:12:17.176452+0000","last_clean":"2023-08-09T15:12:17.176452+0000","last_became_active":"2023-08-09T13:33:56.402485+0000","last_became_peered":"2023-08-09T13:33:56.402485+0000","last_unstale":"2023-08-09T15:12:17.176452+0000","last_undegraded":"2023-08-09T15:12:17.176452+0000","last_fullsized":"2023-08-09T15:12:17.176452+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:33:54.845869+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:33:54.845869+0000","last_clean_scrub_stamp":"2023-08-09T13:33:54.845869+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T14:46:58.528793+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":422,"reported_epoch":229,"state":"active+clean","last_fresh":"2023-08-09T15:12:17.182618+0000","last_change":"2023-08-09T13:33:56.401781+0000","last_active":"2023-08-09T15:12:17.182618+0000","last_peered":"2023-08-09T15:12:17.182618+0000","last_clean":"2023-08-09T15:12:17.182618+0000","last_became_active":"2023-08-09T13:33:56.400336+0000","last_became_peered":"2023-08-09T13:33:56.400336+0000","last_unstale":"2023-08-09T15:12:17.182618+0000","last_undegraded":"2023-08-09T15:12:17.182618+0000","last_fullsized":"2023-08-09T15:12:17.182618+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:33:54.845869+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:33:54.845869+0000","last_clean_scrub_stamp":"2023-08-09T13:33:54.845869+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T19:20:37.616910+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":422,"reported_epoch":229,"state":"active+clean","last_fresh":"2023-08-09T15:12:17.186106+0000","last_change":"2023-08-09T13:33:55.886389+0000","last_active":"2023-08-09T15:12:17.186106+0000","last_peered":"2023-08-09T15:12:17.186106+0000","last_clean":"2023-08-09T15:12:17.186106+0000","last_became_active":"2023-08-09T13:33:55.885638+0000","last_became_peered":"2023-08-09T13:33:55.885638+0000","last_unstale":"2023-08-09T15:12:17.186106+0000","last_undegraded":"2023-08-09T15:12:17.186106+0000","last_fullsized":"2023-08-09T15:12:17.186106+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:33:54.845869+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:33:54.845869+0000","last_clean_scrub_stamp":"2023-08-09T13:33:54.845869+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T17:59:55.723013+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2155,"num_read_kb":4508,"num_write":583,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":351,"ondisk_log_size":351,"up":2,"acting":2,"num_store_stats":2},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":8}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834576021,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":52552,"kb_used_data":3388,"kb_used_omap":1,"kb_used_meta":49150,"kb_avail":94319288,"statfs":{"total":96636764160,"available":96582950912,"internally_reserved":0,"allocated":3469312,"data_stored":2157740,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1781,"internal_metadata":50329867},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Aug 9 15:11:52 2023","interfaces":[{"interface":"back","average":{"1min":1.113,"5min":1.234,"15min":1.2290000000000001},"min":{"1min":0.52400000000000002,"5min":0.52400000000000002,"15min":0.52400000000000002},"max":{"1min":1.6080000000000001,"5min":2.056,"15min":2.056},"last":0.99399999999999999},{"interface":"front","average":{"1min":1.0900000000000001,"5min":1.2110000000000001,"15min":1.2529999999999999},"min":{"1min":0.55800000000000005,"5min":0.55800000000000005,"15min":0.55800000000000005},"max":{"1min":1.5960000000000001,"5min":1.7769999999999999,"15min":1.863},"last":1.0189999999999999}]},{"osd":1,"last update":"Wed Aug 9 15:11:52 2023","interfaces":[{"interface":"back","average":{"1min":1.1479999999999999,"5min":1.28,"15min":1.3049999999999999},"min":{"1min":0.52500000000000002,"5min":0.52500000000000002,"15min":0.52500000000000002},"max":{"1min":1.6619999999999999,"5min":1.853,"15min":1.853},"last":1.177},{"interface":"front","average":{"1min":1.175,"5min":1.296,"15min":1.3100000000000001},"min":{"1min":0.748,"5min":0.748,"15min":0.748},"max":{"1min":1.605,"5min":2.1520000000000001,"15min":2.1520000000000001},"last":0.96599999999999997}]},{"osd":2,"last update":"Wed Aug 9 15:11:52 2023","interfaces":[{"interface":"back","average":{"1min":1.093,"5min":1.262,"15min":1.2969999999999999},"min":{"1min":0.70399999999999996,"5min":0.70399999999999996,"15min":0.70399999999999996},"max":{"1min":1.7050000000000001,"5min":1.861,"15min":1.861},"last":0.93200000000000005},{"interface":"front","average":{"1min":1.214,"5min":1.3480000000000001,"15min":1.393},"min":{"1min":0.58099999999999996,"5min":0.58099999999999996,"15min":0.58099999999999996},"max":{"1min":1.7889999999999999,"5min":1.9370000000000001,"15min":1.9370000000000001},"last":1.0620000000000001}]},{"osd":3,"last update":"Wed Aug 9 15:11:52 2023","interfaces":[{"interface":"back","average":{"1min":1.1950000000000001,"5min":1.3160000000000001,"15min":1.351},"min":{"1min":0.80700000000000005,"5min":0.80700000000000005,"15min":0.80700000000000005},"max":{"1min":1.798,"5min":1.798,"15min":1.9259999999999999},"last":1.0389999999999999},{"interface":"front","average":{"1min":1.335,"5min":1.4379999999999999,"15min":1.472},"min":{"1min":0.76200000000000001,"5min":0.76200000000000001,"15min":0.76200000000000001},"max":{"1min":1.9179999999999999,"5min":1.9239999999999999,"15min":1.9239999999999999},"last":1.125}]},{"osd":4,"last update":"Wed Aug 9 15:11:52 2023","interfaces":[{"interface":"back","average":{"1min":1.218,"5min":1.282,"15min":1.2989999999999999},"min":{"1min":0.56399999999999995,"5min":0.56399999999999995,"15min":0.56399999999999995},"max":{"1min":1.6080000000000001,"5min":2.2810000000000001,"15min":2.2810000000000001},"last":0.80000000000000004},{"interface":"front","average":{"1min":1.28,"5min":1.3560000000000001,"15min":1.385},"min":{"1min":0.622,"5min":0.622,"15min":0.622},"max":{"1min":2.0390000000000001,"5min":2.0390000000000001,"15min":2.0830000000000002},"last":0.73699999999999999}]},{"osd":5,"last update":"Wed Aug 9 15:11:52 2023","interfaces":[{"interface":"back","average":{"1min":1.3620000000000001,"5min":1.4219999999999999,"15min":1.4410000000000001},"min":{"1min":0.59299999999999997,"5min":0.59299999999999997,"15min":0.59299999999999997},"max":{"1min":2.0899999999999999,"5min":2.0899999999999999,"15min":2.0899999999999999},"last":1.2470000000000001},{"interface":"front","average":{"1min":1.3959999999999999,"5min":1.448,"15min":1.4730000000000001},"min":{"1min":0.65500000000000003,"5min":0.65500000000000003,"15min":0.65500000000000003},"max":{"1min":2.206,"5min":2.5310000000000001,"15min":2.5310000000000001},"last":1.1439999999999999}]},{"osd":6,"last update":"Wed Aug 9 15:11:52 2023","interfaces":[{"interface":"back","average":{"1min":1.5069999999999999,"5min":1.587,"15min":1.589},"min":{"1min":0.69199999999999995,"5min":0.69199999999999995,"15min":0.69199999999999995},"max":{"1min":2.294,"5min":2.8959999999999999,"15min":2.8959999999999999},"last":1.1020000000000001},{"interface":"front","average":{"1min":1.359,"5min":1.5089999999999999,"15min":1.534},"min":{"1min":0.60899999999999999,"5min":0.60899999999999999,"15min":0.60899999999999999},"max":{"1min":2.161,"5min":2.214,"15min":2.222},"last":1.085}]}]},{"osd":6,"up_from":13,"seq":55834576021,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":60160,"kb_used_data":2804,"kb_used_omap":1,"kb_used_meta":57342,"kb_avail":94311680,"statfs":{"total":96636764160,"available":96575160320,"internally_reserved":0,"allocated":2871296,"data_stored":1566011,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1781,"internal_metadata":58718475},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Aug 9 15:11:57 2023","interfaces":[{"interface":"back","average":{"1min":1.1639999999999999,"5min":1.2130000000000001,"15min":1.2190000000000001},"min":{"1min":0.76700000000000002,"5min":0.75600000000000001,"15min":0.75600000000000001},"max":{"1min":1.8169999999999999,"5min":1.8169999999999999,"15min":1.8859999999999999},"last":1.1859999999999999},{"interface":"front","average":{"1min":1.0720000000000001,"5min":1.21,"15min":1.234},"min":{"1min":0.64500000000000002,"5min":0.64500000000000002,"15min":0.64500000000000002},"max":{"1min":1.5109999999999999,"5min":2.0800000000000001,"15min":2.0800000000000001},"last":0.88700000000000001}]},{"osd":1,"last update":"Wed Aug 9 15:11:57 2023","interfaces":[{"interface":"back","average":{"1min":1.095,"5min":1.1879999999999999,"15min":1.1910000000000001},"min":{"1min":0.65900000000000003,"5min":0.65900000000000003,"15min":0.65900000000000003},"max":{"1min":1.776,"5min":2.6760000000000002,"15min":2.6760000000000002},"last":0.97499999999999998},{"interface":"front","average":{"1min":1.2869999999999999,"5min":1.3120000000000001,"15min":1.3460000000000001},"min":{"1min":0.72599999999999998,"5min":0.72599999999999998,"15min":0.72599999999999998},"max":{"1min":1.579,"5min":1.998,"15min":1.998},"last":1.093}]},{"osd":2,"last update":"Wed Aug 9 15:11:57 2023","interfaces":[{"interface":"back","average":{"1min":1.2629999999999999,"5min":1.3120000000000001,"15min":1.3400000000000001},"min":{"1min":0.79800000000000004,"5min":0.79800000000000004,"15min":0.79800000000000004},"max":{"1min":1.6659999999999999,"5min":2.0089999999999999,"15min":2.081},"last":1.25},{"interface":"front","average":{"1min":1.3080000000000001,"5min":1.337,"15min":1.3480000000000001},"min":{"1min":0.73799999999999999,"5min":0.73799999999999999,"15min":0.73799999999999999},"max":{"1min":1.75,"5min":1.9219999999999999,"15min":1.9219999999999999},"last":0.81799999999999995}]},{"osd":3,"last update":"Wed Aug 9 15:11:57 2023","interfaces":[{"interface":"back","average":{"1min":1.302,"5min":1.4099999999999999,"15min":1.4019999999999999},"min":{"1min":0.70299999999999996,"5min":0.70299999999999996,"15min":0.70299999999999996},"max":{"1min":2.2669999999999999,"5min":2.2669999999999999,"15min":2.2669999999999999},"last":0.73199999999999998},{"interface":"front","average":{"1min":1.2270000000000001,"5min":1.319,"15min":1.3420000000000001},"min":{"1min":0.59699999999999998,"5min":0.59699999999999998,"15min":0.59699999999999998},"max":{"1min":1.855,"5min":2.0539999999999998,"15min":2.0539999999999998},"last":0.92000000000000004}]},{"osd":4,"last update":"Wed Aug 9 15:11:57 2023","interfaces":[{"interface":"back","average":{"1min":1.339,"5min":1.3500000000000001,"15min":1.339},"min":{"1min":0.46500000000000002,"5min":0.46500000000000002,"15min":0.46500000000000002},"max":{"1min":1.7270000000000001,"5min":2.0470000000000002,"15min":2.0470000000000002},"last":1.3740000000000001},{"interface":"front","average":{"1min":1.288,"5min":1.4470000000000001,"15min":1.4330000000000001},"min":{"1min":0.53000000000000003,"5min":0.53000000000000003,"15min":0.53000000000000003},"max":{"1min":1.6559999999999999,"5min":2.1589999999999998,"15min":2.1589999999999998},"last":0.93999999999999995}]},{"osd":5,"last update":"Wed Aug 9 15:11:57 2023","interfaces":[{"interface":"back","average":{"1min":1.3109999999999999,"5min":1.431,"15min":1.381},"min":{"1min":0.51100000000000001,"5min":0.51100000000000001,"15min":0.51100000000000001},"max":{"1min":1.9610000000000001,"5min":2.1269999999999998,"15min":2.6059999999999999},"last":0.84899999999999998},{"interface":"front","average":{"1min":1.546,"5min":1.637,"15min":1.6299999999999999},"min":{"1min":0.67300000000000004,"5min":0.67300000000000004,"15min":0.67300000000000004},"max":{"1min":1.885,"5min":2.125,"15min":2.1899999999999999},"last":1.3100000000000001}]},{"osd":7,"last update":"Wed Aug 9 15:11:57 2023","interfaces":[{"interface":"back","average":{"1min":1.506,"5min":1.639,"15min":1.6180000000000001},"min":{"1min":0.626,"5min":0.626,"15min":0.626},"max":{"1min":1.9319999999999999,"5min":2.2280000000000002,"15min":2.4340000000000002},"last":0.99299999999999999},{"interface":"front","average":{"1min":1.45,"5min":1.6180000000000001,"15min":1.6000000000000001},"min":{"1min":0.44600000000000001,"5min":0.44600000000000001,"15min":0.44600000000000001},"max":{"1min":2.0409999999999999,"5min":2.3300000000000001,"15min":2.7400000000000002},"last":1.069}]}]},{"osd":5,"up_from":13,"seq":55834576021,"num_pgs":17,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":47876,"kb_used_data":2808,"kb_used_omap":1,"kb_used_meta":45054,"kb_avail":94323964,"statfs":{"total":96636764160,"available":96587739136,"internally_reserved":0,"allocated":2875392,"data_stored":1567372,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1781,"internal_metadata":46135563},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Aug 9 15:11:47 2023","interfaces":[{"interface":"back","average":{"1min":1.0669999999999999,"5min":1.1919999999999999,"15min":1.194},"min":{"1min":0.57699999999999996,"5min":0.57699999999999996,"15min":0.57699999999999996},"max":{"1min":1.494,"5min":1.7370000000000001,"15min":1.754},"last":0.76300000000000001},{"interface":"front","average":{"1min":1.0389999999999999,"5min":1.167,"15min":1.208},"min":{"1min":0.55600000000000005,"5min":0.55600000000000005,"15min":0.55600000000000005},"max":{"1min":1.3480000000000001,"5min":1.6870000000000001,"15min":1.976},"last":1.0680000000000001}]},{"osd":1,"last update":"Wed Aug 9 15:11:47 2023","interfaces":[{"interface":"back","average":{"1min":1.2490000000000001,"5min":1.28,"15min":1.2509999999999999},"min":{"1min":0.67500000000000004,"5min":0.65200000000000002,"15min":0.65200000000000002},"max":{"1min":1.9490000000000001,"5min":2.016,"15min":2.016},"last":0.69799999999999995},{"interface":"front","average":{"1min":1.2150000000000001,"5min":1.288,"15min":1.343},"min":{"1min":0.65000000000000002,"5min":0.65000000000000002,"15min":0.65000000000000002},"max":{"1min":2.0169999999999999,"5min":2.0169999999999999,"15min":2.0169999999999999},"last":1.6259999999999999}]},{"osd":2,"last update":"Wed Aug 9 15:11:47 2023","interfaces":[{"interface":"back","average":{"1min":1.0269999999999999,"5min":1.2629999999999999,"15min":1.3009999999999999},"min":{"1min":0.60199999999999998,"5min":0.60199999999999998,"15min":0.60199999999999998},"max":{"1min":1.48,"5min":1.8859999999999999,"15min":2.032},"last":1.3180000000000001},{"interface":"front","average":{"1min":1.3,"5min":1.339,"15min":1.3640000000000001},"min":{"1min":0.61899999999999999,"5min":0.61899999999999999,"15min":0.61899999999999999},"max":{"1min":1.7010000000000001,"5min":1.956,"15min":1.964},"last":1.4770000000000001}]},{"osd":3,"last update":"Wed Aug 9 15:11:47 2023","interfaces":[{"interface":"back","average":{"1min":1.3,"5min":1.3740000000000001,"15min":1.3939999999999999},"min":{"1min":0.746,"5min":0.746,"15min":0.746},"max":{"1min":1.6639999999999999,"5min":1.798,"15min":1.899},"last":1.373},{"interface":"front","average":{"1min":1.278,"5min":1.389,"15min":1.4370000000000001},"min":{"1min":0.66400000000000003,"5min":0.66400000000000003,"15min":0.66400000000000003},"max":{"1min":1.6559999999999999,"5min":1.8819999999999999,"15min":2.0379999999999998},"last":1.226}]},{"osd":4,"last update":"Wed Aug 9 15:11:47 2023","interfaces":[{"interface":"back","average":{"1min":1.347,"5min":1.323,"15min":1.3220000000000001},"min":{"1min":0.53000000000000003,"5min":0.53000000000000003,"15min":0.53000000000000003},"max":{"1min":1.833,"5min":2.0299999999999998,"15min":2.0299999999999998},"last":1.7210000000000001},{"interface":"front","average":{"1min":1.4419999999999999,"5min":1.389,"15min":1.4059999999999999},"min":{"1min":0.69199999999999995,"5min":0.69199999999999995,"15min":0.69199999999999995},"max":{"1min":1.7709999999999999,"5min":1.9239999999999999,"15min":2.0390000000000001},"last":1.742}]},{"osd":6,"last update":"Wed Aug 9 15:11:47 2023","interfaces":[{"interface":"back","average":{"1min":1.298,"5min":1.3660000000000001,"15min":1.411},"min":{"1min":0.501,"5min":0.501,"15min":0.501},"max":{"1min":1.728,"5min":2.032,"15min":2.1000000000000001},"last":1.675},{"interface":"front","average":{"1min":1.48,"5min":1.4850000000000001,"15min":1.5029999999999999},"min":{"1min":0.70599999999999996,"5min":0.70599999999999996,"15min":0.70599999999999996},"max":{"1min":2.125,"5min":2.1349999999999998,"15min":2.1349999999999998},"last":1.819}]},{"osd":7,"last update":"Wed Aug 9 15:11:47 2023","interfaces":[{"interface":"back","average":{"1min":1.5469999999999999,"5min":1.554,"15min":1.5800000000000001},"min":{"1min":0.71999999999999997,"5min":0.71999999999999997,"15min":0.71999999999999997},"max":{"1min":2.0760000000000001,"5min":2.0760000000000001,"15min":2.4689999999999999},"last":1.9370000000000001},{"interface":"front","average":{"1min":1.5309999999999999,"5min":1.595,"15min":1.637},"min":{"1min":0.63800000000000001,"5min":0.63800000000000001,"15min":0.63800000000000001},"max":{"1min":2.1709999999999998,"5min":2.1709999999999998,"15min":2.4239999999999999},"last":1.5609999999999999}]}]},{"osd":4,"up_from":13,"seq":55834576021,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":51972,"kb_used_data":2808,"kb_used_omap":1,"kb_used_meta":49150,"kb_avail":94319868,"statfs":{"total":96636764160,"available":96583544832,"internally_reserved":0,"allocated":2875392,"data_stored":1567372,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1781,"internal_metadata":50329867},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Aug 9 15:12:13 2023","interfaces":[{"interface":"back","average":{"1min":1.2370000000000001,"5min":1.196,"15min":1.222},"min":{"1min":0.64700000000000002,"5min":0.60399999999999998,"15min":0.60399999999999998},"max":{"1min":1.8169999999999999,"5min":1.8169999999999999,"15min":2.2959999999999998},"last":0.90500000000000003},{"interface":"front","average":{"1min":1.2270000000000001,"5min":1.2829999999999999,"15min":1.3129999999999999},"min":{"1min":0.77000000000000002,"5min":0.66800000000000004,"15min":0.66800000000000004},"max":{"1min":1.772,"5min":2.0779999999999998,"15min":2.2709999999999999},"last":1.454}]},{"osd":1,"last update":"Wed Aug 9 15:12:13 2023","interfaces":[{"interface":"back","average":{"1min":1.1439999999999999,"5min":1.288,"15min":1.3460000000000001},"min":{"1min":0.73199999999999998,"5min":0.73199999999999998,"15min":0.73199999999999998},"max":{"1min":1.9370000000000001,"5min":1.9370000000000001,"15min":1.9370000000000001},"last":0.97299999999999998},{"interface":"front","average":{"1min":1.2430000000000001,"5min":1.2609999999999999,"15min":1.278},"min":{"1min":0.76400000000000001,"5min":0.73899999999999999,"15min":0.73899999999999999},"max":{"1min":1.865,"5min":1.865,"15min":2.117},"last":1.379}]},{"osd":2,"last update":"Wed Aug 9 15:12:13 2023","interfaces":[{"interface":"back","average":{"1min":1.1859999999999999,"5min":1.2350000000000001,"15min":1.2829999999999999},"min":{"1min":0.70199999999999996,"5min":0.63100000000000001,"15min":0.63100000000000001},"max":{"1min":1.5469999999999999,"5min":1.7709999999999999,"15min":1.883},"last":1.026},{"interface":"front","average":{"1min":1.298,"5min":1.3819999999999999,"15min":1.401},"min":{"1min":0.91400000000000003,"5min":0.82699999999999996,"15min":0.79200000000000004},"max":{"1min":1.903,"5min":2.343,"15min":2.343},"last":1.127}]},{"osd":3,"last update":"Wed Aug 9 15:12:13 2023","interfaces":[{"interface":"back","average":{"1min":1.2789999999999999,"5min":1.3420000000000001,"15min":1.349},"min":{"1min":0.95299999999999996,"5min":0.90500000000000003,"15min":0.88700000000000001},"max":{"1min":1.6970000000000001,"5min":1.8009999999999999,"15min":2.4870000000000001},"last":1.6970000000000001},{"interface":"front","average":{"1min":1.3720000000000001,"5min":1.423,"15min":1.464},"min":{"1min":0.95299999999999996,"5min":0.79700000000000004,"15min":0.79700000000000004},"max":{"1min":1.8240000000000001,"5min":1.909,"15min":2.5459999999999998},"last":1.1990000000000001}]},{"osd":5,"last update":"Wed Aug 9 15:12:13 2023","interfaces":[{"interface":"back","average":{"1min":1.431,"5min":1.385,"15min":1.409},"min":{"1min":1.0029999999999999,"5min":0.82199999999999995,"15min":0.82199999999999995},"max":{"1min":2.004,"5min":2.633,"15min":2.633},"last":1.2470000000000001},{"interface":"front","average":{"1min":1.395,"5min":1.4059999999999999,"15min":1.393},"min":{"1min":1.048,"5min":0.68100000000000005,"15min":0.68100000000000005},"max":{"1min":1.8300000000000001,"5min":1.9930000000000001,"15min":2.4249999999999998},"last":1.5509999999999999}]},{"osd":6,"last update":"Wed Aug 9 15:12:13 2023","interfaces":[{"interface":"back","average":{"1min":1.3919999999999999,"5min":1.4330000000000001,"15min":1.4099999999999999},"min":{"1min":1.0509999999999999,"5min":0.71299999999999997,"15min":0.71299999999999997},"max":{"1min":1.73,"5min":2.0049999999999999,"15min":2.9119999999999999},"last":1.3149999999999999},{"interface":"front","average":{"1min":1.528,"5min":1.5680000000000001,"15min":1.579},"min":{"1min":1.1699999999999999,"5min":0.77000000000000002,"15min":0.77000000000000002},"max":{"1min":1.8680000000000001,"5min":2.9529999999999998,"15min":2.9529999999999998},"last":1.5309999999999999}]},{"osd":7,"last update":"Wed Aug 9 15:12:13 2023","interfaces":[{"interface":"back","average":{"1min":1.5029999999999999,"5min":1.5309999999999999,"15min":1.5229999999999999},"min":{"1min":1.1719999999999999,"5min":0.83899999999999997,"15min":0.83899999999999997},"max":{"1min":1.9370000000000001,"5min":2.1589999999999998,"15min":2.5910000000000002},"last":1.796},{"interface":"front","average":{"1min":1.599,"5min":1.6140000000000001,"15min":1.6060000000000001},"min":{"1min":1.206,"5min":0.90000000000000002,"15min":0.90000000000000002},"max":{"1min":1.8999999999999999,"5min":2.218,"15min":3.028},"last":1.502}]}]},{"osd":3,"up_from":13,"seq":55834576022,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":47876,"kb_used_data":2808,"kb_used_omap":1,"kb_used_meta":45054,"kb_avail":94323964,"statfs":{"total":96636764160,"available":96587739136,"internally_reserved":0,"allocated":2875392,"data_stored":1567372,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1781,"internal_metadata":46135563},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Aug 9 15:12:06 2023","interfaces":[{"interface":"back","average":{"1min":0.75800000000000001,"5min":0.94699999999999995,"15min":0.96899999999999997},"min":{"1min":0.42799999999999999,"5min":0.42799999999999999,"15min":0.42799999999999999},"max":{"1min":1.24,"5min":1.6699999999999999,"15min":2.2090000000000001},"last":0.64500000000000002},{"interface":"front","average":{"1min":0.872,"5min":1.093,"15min":1.1439999999999999},"min":{"1min":0.434,"5min":0.434,"15min":0.434},"max":{"1min":1.4550000000000001,"5min":1.9630000000000001,"15min":1.9630000000000001},"last":0.92000000000000004}]},{"osd":1,"last update":"Wed Aug 9 15:12:06 2023","interfaces":[{"interface":"back","average":{"1min":0.90500000000000003,"5min":1.006,"15min":1.0660000000000001},"min":{"1min":0.437,"5min":0.437,"15min":0.437},"max":{"1min":1.595,"5min":1.956,"15min":2.27},"last":0.77500000000000002},{"interface":"front","average":{"1min":1.0960000000000001,"5min":1.2070000000000001,"15min":1.2569999999999999},"min":{"1min":0.63900000000000001,"5min":0.498,"15min":0.498},"max":{"1min":1.6140000000000001,"5min":1.78,"15min":1.9730000000000001},"last":1.0620000000000001}]},{"osd":2,"last update":"Wed Aug 9 15:12:06 2023","interfaces":[{"interface":"back","average":{"1min":1.024,"5min":1.236,"15min":1.3180000000000001},"min":{"1min":0.65400000000000003,"5min":0.65400000000000003,"15min":0.65400000000000003},"max":{"1min":1.3839999999999999,"5min":1.9650000000000001,"15min":2.4079999999999999},"last":0.88500000000000001},{"interface":"front","average":{"1min":1.2250000000000001,"5min":1.3720000000000001,"15min":1.4279999999999999},"min":{"1min":0.63400000000000001,"5min":0.63400000000000001,"15min":0.63400000000000001},"max":{"1min":1.7090000000000001,"5min":2.0510000000000002,"15min":2.1259999999999999},"last":1.248}]},{"osd":4,"last update":"Wed Aug 9 15:12:06 2023","interfaces":[{"interface":"back","average":{"1min":1.2070000000000001,"5min":1.3260000000000001,"15min":1.3959999999999999},"min":{"1min":0.752,"5min":0.748,"15min":0.72599999999999998},"max":{"1min":1.744,"5min":1.9179999999999999,"15min":2.2069999999999999},"last":1.04},{"interface":"front","average":{"1min":1.3220000000000001,"5min":1.4199999999999999,"15min":1.4570000000000001},"min":{"1min":0.77500000000000002,"5min":0.77500000000000002,"15min":0.77500000000000002},"max":{"1min":1.8939999999999999,"5min":1.9279999999999999,"15min":2.056},"last":1.4419999999999999}]},{"osd":5,"last update":"Wed Aug 9 15:12:06 2023","interfaces":[{"interface":"back","average":{"1min":1.4039999999999999,"5min":1.502,"15min":1.538},"min":{"1min":0.77600000000000002,"5min":0.77600000000000002,"15min":0.77600000000000002},"max":{"1min":2.1890000000000001,"5min":2.1890000000000001,"15min":2.1890000000000001},"last":1.327},{"interface":"front","average":{"1min":1.3740000000000001,"5min":1.5580000000000001,"15min":1.6000000000000001},"min":{"1min":0.88800000000000001,"5min":0.85799999999999998,"15min":0.85799999999999998},"max":{"1min":1.7330000000000001,"5min":2.2469999999999999,"15min":2.4049999999999998},"last":1.49}]},{"osd":6,"last update":"Wed Aug 9 15:12:06 2023","interfaces":[{"interface":"back","average":{"1min":1.3560000000000001,"5min":1.4890000000000001,"15min":1.5780000000000001},"min":{"1min":0.69599999999999995,"5min":0.69599999999999995,"15min":0.69599999999999995},"max":{"1min":1.9670000000000001,"5min":2.4860000000000002,"15min":2.4860000000000002},"last":0.96799999999999997},{"interface":"front","average":{"1min":1.458,"5min":1.5940000000000001,"15min":1.653},"min":{"1min":0.89700000000000002,"5min":0.89700000000000002,"15min":0.89700000000000002},"max":{"1min":2.04,"5min":2.4039999999999999,"15min":2.4039999999999999},"last":1.1279999999999999}]},{"osd":7,"last update":"Wed Aug 9 15:12:06 2023","interfaces":[{"interface":"back","average":{"1min":1.5620000000000001,"5min":1.696,"15min":1.7490000000000001},"min":{"1min":0.94799999999999995,"5min":0.94799999999999995,"15min":0.94799999999999995},"max":{"1min":1.8680000000000001,"5min":2.347,"15min":2.347},"last":1.3620000000000001},{"interface":"front","average":{"1min":1.4179999999999999,"5min":1.6220000000000001,"15min":1.694},"min":{"1min":0.81100000000000005,"5min":0.81100000000000005,"15min":0.81100000000000005},"max":{"1min":2.1309999999999998,"5min":3.3220000000000001,"15min":3.3220000000000001},"last":0.998}]}]},{"osd":2,"up_from":13,"seq":55834576021,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":46220,"kb_used_data":2808,"kb_used_omap":1,"kb_used_meta":43390,"kb_avail":94325620,"statfs":{"total":96636764160,"available":96589434880,"internally_reserved":0,"allocated":2875392,"data_stored":1567372,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1781,"internal_metadata":44431627},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Aug 9 15:11:52 2023","interfaces":[{"interface":"back","average":{"1min":0.874,"5min":0.98499999999999999,"15min":0.95899999999999996},"min":{"1min":0.48199999999999998,"5min":0.44800000000000001,"15min":0.44800000000000001},"max":{"1min":1.591,"5min":1.591,"15min":1.591},"last":1.742},{"interface":"front","average":{"1min":0.96399999999999997,"5min":1.0760000000000001,"15min":1.085},"min":{"1min":0.38900000000000001,"5min":0.38900000000000001,"15min":0.38900000000000001},"max":{"1min":1.5980000000000001,"5min":1.764,"15min":2.052},"last":1.8149999999999999}]},{"osd":1,"last update":"Wed Aug 9 15:11:52 2023","interfaces":[{"interface":"back","average":{"1min":0.98299999999999998,"5min":1.2330000000000001,"15min":1.2230000000000001},"min":{"1min":0.55400000000000005,"5min":0.55400000000000005,"15min":0.55400000000000005},"max":{"1min":1.403,"5min":2.0179999999999998,"15min":2.0179999999999998},"last":1.9570000000000001},{"interface":"front","average":{"1min":1.1359999999999999,"5min":1.292,"15min":1.381},"min":{"1min":0.51200000000000001,"5min":0.51200000000000001,"15min":0.51200000000000001},"max":{"1min":1.6639999999999999,"5min":1.95,"15min":1.95},"last":1.927}]},{"osd":3,"last update":"Wed Aug 9 15:11:52 2023","interfaces":[{"interface":"back","average":{"1min":0.99099999999999999,"5min":1.089,"15min":1.1080000000000001},"min":{"1min":0.53500000000000003,"5min":0.52100000000000002,"15min":0.52100000000000002},"max":{"1min":1.5649999999999999,"5min":1.871,"15min":1.871},"last":0.71899999999999997},{"interface":"front","average":{"1min":1.111,"5min":1.339,"15min":1.369},"min":{"1min":0.57099999999999995,"5min":0.57099999999999995,"15min":0.57099999999999995},"max":{"1min":1.6930000000000001,"5min":1.9299999999999999,"15min":2.0470000000000002},"last":1.782}]},{"osd":4,"last update":"Wed Aug 9 15:11:52 2023","interfaces":[{"interface":"back","average":{"1min":1.369,"5min":1.5229999999999999,"15min":1.5469999999999999},"min":{"1min":0.86399999999999999,"5min":0.86399999999999999,"15min":0.86399999999999999},"max":{"1min":1.7350000000000001,"5min":1.994,"15min":2.464},"last":1.8600000000000001},{"interface":"front","average":{"1min":1.395,"5min":1.4399999999999999,"15min":1.464},"min":{"1min":0.93600000000000005,"5min":0.81799999999999995,"15min":0.81799999999999995},"max":{"1min":1.9370000000000001,"5min":1.98,"15min":2.0720000000000001},"last":1.4830000000000001}]},{"osd":5,"last update":"Wed Aug 9 15:11:52 2023","interfaces":[{"interface":"back","average":{"1min":1.4279999999999999,"5min":1.5009999999999999,"15min":1.5169999999999999},"min":{"1min":0.64800000000000002,"5min":0.64800000000000002,"15min":0.64800000000000002},"max":{"1min":1.913,"5min":2.0369999999999999,"15min":2.1139999999999999},"last":1.6000000000000001},{"interface":"front","average":{"1min":1.351,"5min":1.419,"15min":1.4870000000000001},"min":{"1min":0.83399999999999996,"5min":0.83399999999999996,"15min":0.83399999999999996},"max":{"1min":2.1520000000000001,"5min":2.1520000000000001,"15min":2.214},"last":1.677}]},{"osd":6,"last update":"Wed Aug 9 15:11:52 2023","interfaces":[{"interface":"back","average":{"1min":1.407,"5min":1.4890000000000001,"15min":1.5229999999999999},"min":{"1min":0.89500000000000002,"5min":0.89500000000000002,"15min":0.89500000000000002},"max":{"1min":2.0680000000000001,"5min":2.1110000000000002,"15min":2.2999999999999998},"last":1.5209999999999999},{"interface":"front","average":{"1min":1.512,"5min":1.6479999999999999,"15min":1.6379999999999999},"min":{"1min":0.998,"5min":0.998,"15min":0.998},"max":{"1min":2.4140000000000001,"5min":2.4140000000000001,"15min":2.4140000000000001},"last":1.9099999999999999}]},{"osd":7,"last update":"Wed Aug 9 15:11:52 2023","interfaces":[{"interface":"back","average":{"1min":1.5249999999999999,"5min":1.605,"15min":1.6419999999999999},"min":{"1min":0.96299999999999997,"5min":0.87,"15min":0.87},"max":{"1min":2.1000000000000001,"5min":2.1749999999999998,"15min":2.302},"last":1.698},{"interface":"front","average":{"1min":1.4650000000000001,"5min":1.5800000000000001,"15min":1.6299999999999999},"min":{"1min":0.71399999999999997,"5min":0.71399999999999997,"15min":0.71399999999999997},"max":{"1min":2.2360000000000002,"5min":2.242,"15min":2.3809999999999998},"last":1.466}]}]},{"osd":1,"up_from":13,"seq":55834576021,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":56656,"kb_used_data":3388,"kb_used_omap":1,"kb_used_meta":53246,"kb_avail":94315184,"statfs":{"total":96636764160,"available":96578748416,"internally_reserved":0,"allocated":3469312,"data_stored":2157740,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1781,"internal_metadata":54524171},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Aug 9 15:12:03 2023","interfaces":[{"interface":"back","average":{"1min":0.83399999999999996,"5min":0.97399999999999998,"15min":1},"min":{"1min":0.32100000000000001,"5min":0.32100000000000001,"15min":0.32100000000000001},"max":{"1min":1.153,"5min":1.8799999999999999,"15min":2.0939999999999999},"last":0.81499999999999995},{"interface":"front","average":{"1min":0.94299999999999995,"5min":1.1060000000000001,"15min":1.1399999999999999},"min":{"1min":0.40500000000000003,"5min":0.40500000000000003,"15min":0.40500000000000003},"max":{"1min":1.641,"5min":1.78,"15min":1.8939999999999999},"last":0.46800000000000003}]},{"osd":2,"last update":"Wed Aug 9 15:12:03 2023","interfaces":[{"interface":"back","average":{"1min":0.93700000000000006,"5min":1.018,"15min":1.04},"min":{"1min":0.626,"5min":0.55800000000000005,"15min":0.55800000000000005},"max":{"1min":2.2240000000000002,"5min":2.2240000000000002,"15min":2.3940000000000001},"last":1.206},{"interface":"front","average":{"1min":0.97399999999999998,"5min":1.1759999999999999,"15min":1.254},"min":{"1min":0.49299999999999999,"5min":0.46700000000000003,"15min":0.46700000000000003},"max":{"1min":1.5149999999999999,"5min":2.169,"15min":2.169},"last":0.86299999999999999}]},{"osd":3,"last update":"Wed Aug 9 15:12:03 2023","interfaces":[{"interface":"back","average":{"1min":1.115,"5min":1.3340000000000001,"15min":1.381},"min":{"1min":0.54600000000000004,"5min":0.51000000000000001,"15min":0.51000000000000001},"max":{"1min":1.833,"5min":2.093,"15min":2.109},"last":0.69099999999999995},{"interface":"front","average":{"1min":1.0409999999999999,"5min":1.256,"15min":1.3089999999999999},"min":{"1min":0.59099999999999997,"5min":0.59099999999999997,"15min":0.59099999999999997},"max":{"1min":1.8680000000000001,"5min":2.052,"15min":2.052},"last":0.66200000000000003}]},{"osd":4,"last update":"Wed Aug 9 15:12:03 2023","interfaces":[{"interface":"back","average":{"1min":1.254,"5min":1.4339999999999999,"15min":1.4730000000000001},"min":{"1min":0.82099999999999995,"5min":0.753,"15min":0.753},"max":{"1min":1.887,"5min":2.0419999999999998,"15min":2.214},"last":1.3600000000000001},{"interface":"front","average":{"1min":1.202,"5min":1.381,"15min":1.4119999999999999},"min":{"1min":0.84499999999999997,"5min":0.69199999999999995,"15min":0.69199999999999995},"max":{"1min":1.528,"5min":2.0310000000000001,"15min":2.1190000000000002},"last":0.91100000000000003}]},{"osd":5,"last update":"Wed Aug 9 15:12:03 2023","interfaces":[{"interface":"back","average":{"1min":1.2450000000000001,"5min":1.415,"15min":1.446},"min":{"1min":0.83399999999999996,"5min":0.77800000000000002,"15min":0.77800000000000002},"max":{"1min":1.909,"5min":1.994,"15min":2.508},"last":1.0189999999999999},{"interface":"front","average":{"1min":1.2829999999999999,"5min":1.53,"15min":1.5800000000000001},"min":{"1min":0.74299999999999999,"5min":0.74299999999999999,"15min":0.74299999999999999},"max":{"1min":1.99,"5min":2.3809999999999998,"15min":2.3809999999999998},"last":1.3009999999999999}]},{"osd":6,"last update":"Wed Aug 9 15:12:03 2023","interfaces":[{"interface":"back","average":{"1min":1.321,"5min":1.575,"15min":1.613},"min":{"1min":0.80100000000000005,"5min":0.80100000000000005,"15min":0.80100000000000005},"max":{"1min":2.1669999999999998,"5min":2.347,"15min":2.347},"last":1.165},{"interface":"front","average":{"1min":1.288,"5min":1.522,"15min":1.5720000000000001},"min":{"1min":0.84999999999999998,"5min":0.73499999999999999,"15min":0.73499999999999999},"max":{"1min":1.8140000000000001,"5min":2.218,"15min":2.278},"last":1.101}]},{"osd":7,"last update":"Wed Aug 9 15:12:03 2023","interfaces":[{"interface":"back","average":{"1min":1.3959999999999999,"5min":1.6439999999999999,"15min":1.6910000000000001},"min":{"1min":0.90900000000000003,"5min":0.86399999999999999,"15min":0.86399999999999999},"max":{"1min":2.1299999999999999,"5min":2.3090000000000002,"15min":2.488},"last":1.5129999999999999},{"interface":"front","average":{"1min":1.385,"5min":1.633,"15min":1.698},"min":{"1min":0.75600000000000001,"5min":0.75600000000000001,"15min":0.75600000000000001},"max":{"1min":2.4980000000000002,"5min":2.4980000000000002,"15min":2.4980000000000002},"last":1.2629999999999999}]}]},{"osd":0,"up_from":13,"seq":55834576021,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":29824,"kb_used_data":2804,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94342016,"statfs":{"total":96636764160,"available":96606224384,"internally_reserved":0,"allocated":2871296,"data_stored":1566011,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1781,"internal_metadata":27654411},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Wed Aug 9 15:12:00 2023","interfaces":[{"interface":"back","average":{"1min":0.85099999999999998,"5min":0.89600000000000002,"15min":0.95099999999999996},"min":{"1min":0.44900000000000001,"5min":0.40000000000000002,"15min":0.40000000000000002},"max":{"1min":1.3380000000000001,"5min":1.617,"15min":1.6299999999999999},"last":0.86599999999999999},{"interface":"front","average":{"1min":0.98799999999999999,"5min":1.091,"15min":1.135},"min":{"1min":0.56599999999999995,"5min":0.56599999999999995,"15min":0.56599999999999995},"max":{"1min":1.635,"5min":2.1560000000000001,"15min":2.1560000000000001},"last":0.95099999999999996}]},{"osd":2,"last update":"Wed Aug 9 15:12:00 2023","interfaces":[{"interface":"back","average":{"1min":0.873,"5min":1.097,"15min":1.1719999999999999},"min":{"1min":0.54000000000000004,"5min":0.505,"15min":0.505},"max":{"1min":1.401,"5min":2.4769999999999999,"15min":2.4769999999999999},"last":0.97799999999999998},{"interface":"front","average":{"1min":1.0069999999999999,"5min":1.125,"15min":1.2150000000000001},"min":{"1min":0.55500000000000005,"5min":0.53100000000000003,"15min":0.53100000000000003},"max":{"1min":1.468,"5min":1.8180000000000001,"15min":1.9470000000000001},"last":0.67300000000000004}]},{"osd":3,"last update":"Wed Aug 9 15:12:00 2023","interfaces":[{"interface":"back","average":{"1min":0.91200000000000003,"5min":1.1040000000000001,"15min":1.1870000000000001},"min":{"1min":0.45500000000000002,"5min":0.45500000000000002,"15min":0.45500000000000002},"max":{"1min":1.629,"5min":2.1459999999999999,"15min":2.1459999999999999},"last":1.002},{"interface":"front","average":{"1min":1.079,"5min":1.284,"15min":1.3779999999999999},"min":{"1min":0.53100000000000003,"5min":0.53100000000000003,"15min":0.53100000000000003},"max":{"1min":1.5640000000000001,"5min":1.867,"15min":2.0760000000000001},"last":1.1060000000000001}]},{"osd":4,"last update":"Wed Aug 9 15:12:00 2023","interfaces":[{"interface":"back","average":{"1min":1.163,"5min":1.321,"15min":1.403},"min":{"1min":0.81799999999999995,"5min":0.81799999999999995,"15min":0.81799999999999995},"max":{"1min":1.5629999999999999,"5min":1.853,"15min":1.966},"last":1.29},{"interface":"front","average":{"1min":1.296,"5min":1.421,"15min":1.5069999999999999},"min":{"1min":0.83099999999999996,"5min":0.79400000000000004,"15min":0.79400000000000004},"max":{"1min":1.734,"5min":1.972,"15min":2.278},"last":1.4299999999999999}]},{"osd":5,"last update":"Wed Aug 9 15:12:00 2023","interfaces":[{"interface":"back","average":{"1min":1.256,"5min":1.476,"15min":1.5129999999999999},"min":{"1min":0.84699999999999998,"5min":0.84699999999999998,"15min":0.84699999999999998},"max":{"1min":1.79,"5min":2.355,"15min":2.355},"last":0.79800000000000004},{"interface":"front","average":{"1min":1.2849999999999999,"5min":1.4299999999999999,"15min":1.49},"min":{"1min":0.84799999999999998,"5min":0.82899999999999996,"15min":0.82899999999999996},"max":{"1min":2.089,"5min":2.089,"15min":2.3740000000000001},"last":1.802}]},{"osd":6,"last update":"Wed Aug 9 15:12:00 2023","interfaces":[{"interface":"back","average":{"1min":1.3129999999999999,"5min":1.5249999999999999,"15min":1.6160000000000001},"min":{"1min":0.79700000000000004,"5min":0.79700000000000004,"15min":0.79700000000000004},"max":{"1min":1.7889999999999999,"5min":2.7490000000000001,"15min":2.7490000000000001},"last":1.343},{"interface":"front","average":{"1min":1.3839999999999999,"5min":1.534,"15min":1.597},"min":{"1min":0.89200000000000002,"5min":0.84699999999999998,"15min":0.84699999999999998},"max":{"1min":2.0840000000000001,"5min":2.274,"15min":2.2839999999999998},"last":1.5109999999999999}]},{"osd":7,"last update":"Wed Aug 9 15:12:00 2023","interfaces":[{"interface":"back","average":{"1min":1.4710000000000001,"5min":1.599,"15min":1.665},"min":{"1min":0.98599999999999999,"5min":0.94899999999999995,"15min":0.94899999999999995},"max":{"1min":2.2989999999999999,"5min":2.645,"15min":2.645},"last":1.7410000000000001},{"interface":"front","average":{"1min":1.4019999999999999,"5min":1.54,"15min":1.609},"min":{"1min":0.82599999999999996,"5min":0.82599999999999996,"15min":0.82599999999999996},"max":{"1min":2.1829999999999998,"5min":2.1829999999999998,"15min":2.3759999999999999},"last":1.6499999999999999}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-08-09T15:12:19.649 INFO:tasks.ceph.ceph_manager.ceph:clean! 2023-08-09T15:12:19.649 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-08-09T15:12:19.863 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:19.861+0000 7fa8f7f64640 1 Processor -- start 2023-08-09T15:12:19.863 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:19.862+0000 7fa8f7f64640 1 -- start start 2023-08-09T15:12:19.864 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:19.863+0000 7fa8f7f64640 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7fa8f0158ec0 0x7fa8f00afc70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T15:12:19.865 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:19.863+0000 7fa8f7f64640 1 -- --> v1:172.21.15.43:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa8f00b0590 con 0x7fa8f00b01b0 2023-08-09T15:12:19.865 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:19.863+0000 7fa8f7f64640 1 -- --> v1:172.21.15.203:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa8f015d790 con 0x7fa8f0158ae0 2023-08-09T15:12:19.865 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:19.863+0000 7fa8f7f64640 1 -- --> v2:172.21.15.203:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa8f015cc10 con 0x7fa8f0158ec0 2023-08-09T15:12:19.865 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:19.864+0000 7fa8f6501640 1 --1- >> v1:172.21.15.43:6789/0 conn(0x7fa8f00b01b0 0x7fa8f0070240 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.43:6789/0 says I am v1:172.21.15.43:32790/0 (socket says 172.21.15.43:32790) 2023-08-09T15:12:19.866 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:19.864+0000 7fa8f6501640 1 -- 172.21.15.43:0/409160546 learned_addr learned my addr 172.21.15.43:0/409160546 (peer_addr_for_me v1:172.21.15.43:0/0) 2023-08-09T15:12:19.866 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:19.864+0000 7fa8f54ff640 1 --2- 172.21.15.43:0/409160546 >> v2:172.21.15.203:3300/0 conn(0x7fa8f0158ec0 0x7fa8f00afc70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T15:12:19.866 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:19.864+0000 7fa8f4cfe640 1 -- 172.21.15.43:0/409160546 <== mon.0 v1:172.21.15.43:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2448863572 0 0) 0x7fa8f00b0590 con 0x7fa8f00b01b0 2023-08-09T15:12:19.866 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:19.864+0000 7fa8f4cfe640 1 -- 172.21.15.43:0/409160546 --> v1:172.21.15.43:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fa8dc003440 con 0x7fa8f00b01b0 2023-08-09T15:12:19.867 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:19.864+0000 7fa8f4cfe640 1 -- 172.21.15.43:0/409160546 <== mon.0 v1:172.21.15.43:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 490809837 0 0) 0x7fa8dc003440 con 0x7fa8f00b01b0 2023-08-09T15:12:19.868 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:19.865+0000 7fa8f4cfe640 1 -- 172.21.15.43:0/409160546 >> v2:172.21.15.203:3300/0 conn(0x7fa8f0158ec0 msgr2=0x7fa8f00afc70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-09T15:12:19.868 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:19.865+0000 7fa8f4cfe640 1 --2- 172.21.15.43:0/409160546 >> v2:172.21.15.203:3300/0 conn(0x7fa8f0158ec0 0x7fa8f00afc70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T15:12:19.868 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:19.865+0000 7fa8f4cfe640 1 -- 172.21.15.43:0/409160546 >> v1:172.21.15.203:6789/0 conn(0x7fa8f0158ae0 legacy=0x7fa8f00c9950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T15:12:19.868 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:19.865+0000 7fa8f4cfe640 1 -- 172.21.15.43:0/409160546 --> v1:172.21.15.43:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa8f015cef0 con 0x7fa8f00b01b0 2023-08-09T15:12:19.868 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:19.865+0000 7fa8f4cfe640 1 -- 172.21.15.43:0/409160546 <== mon.0 v1:172.21.15.43:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1285150740 0 0) 0x7fa8e80019f0 con 0x7fa8f00b01b0 2023-08-09T15:12:19.869 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:19.865+0000 7fa8f4cfe640 1 -- 172.21.15.43:0/409160546 >> v1:172.21.15.43:6789/0 conn(0x7fa8f00b01b0 legacy=0x7fa8f0070240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T15:12:19.869 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:19.865+0000 7fa8f4cfe640 1 --2- 172.21.15.43:0/409160546 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fa8dc003a60 0x7fa8dc003e80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T15:12:19.869 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:19.865+0000 7fa8f4cfe640 1 --2- 172.21.15.43:0/409160546 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fa8dc008850 0x7fa8dc00ac80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T15:12:19.869 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:19.865+0000 7fa8f4cfe640 1 --2- 172.21.15.43:0/409160546 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa8dc00b480 0x7fa8dc00d940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T15:12:19.869 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:19.865+0000 7fa8f4cfe640 1 -- 172.21.15.43:0/409160546 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa8f015cc10 con 0x7fa8dc00b480 2023-08-09T15:12:19.869 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:19.865+0000 7fa8f4cfe640 1 -- 172.21.15.43:0/409160546 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa8dc00e220 con 0x7fa8dc003a60 2023-08-09T15:12:19.869 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:19.865+0000 7fa8f4cfe640 1 -- 172.21.15.43:0/409160546 --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa8dc00e3e0 con 0x7fa8dc008850 2023-08-09T15:12:19.869 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:19.865+0000 7fa8f6501640 1 --2- 172.21.15.43:0/409160546 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa8dc00b480 0x7fa8dc00d940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T15:12:19.870 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:19.865+0000 7fa8f5d00640 1 --2- 172.21.15.43:0/409160546 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fa8dc003a60 0x7fa8dc003e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T15:12:19.870 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:19.866+0000 7fa8f54ff640 1 --2- 172.21.15.43:0/409160546 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fa8dc008850 0x7fa8dc00ac80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T15:12:19.870 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:19.866+0000 7fa8f6501640 1 -- 172.21.15.43:0/409160546 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fa8dc008850 msgr2=0x7fa8dc00ac80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-09T15:12:19.870 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:19.866+0000 7fa8f6501640 1 --2- 172.21.15.43:0/409160546 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fa8dc008850 0x7fa8dc00ac80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T15:12:19.870 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:19.866+0000 7fa8f6501640 1 -- 172.21.15.43:0/409160546 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fa8dc003a60 msgr2=0x7fa8dc003e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-09T15:12:19.870 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:19.866+0000 7fa8f6501640 1 --2- 172.21.15.43:0/409160546 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fa8dc003a60 0x7fa8dc003e80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T15:12:19.870 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:19.866+0000 7fa8f6501640 1 -- 172.21.15.43:0/409160546 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa8dc003440 con 0x7fa8dc00b480 2023-08-09T15:12:19.871 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:19.866+0000 7fa8f54ff640 1 --2- 172.21.15.43:0/409160546 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fa8dc008850 0x7fa8dc00ac80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-08-09T15:12:19.871 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:19.866+0000 7fa8f5d00640 1 --2- 172.21.15.43:0/409160546 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fa8dc003a60 0x7fa8dc003e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-08-09T15:12:19.872 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:19.867+0000 7fa8f6501640 1 -- 172.21.15.43:0/409160546 reap_dead start 2023-08-09T15:12:19.872 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:19.867+0000 7fa8f6501640 1 --2- 172.21.15.43:0/409160546 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa8dc00b480 0x7fa8dc00d940 secure :-1 s=READY pgs=2557 cs=0 l=1 rev1=1 crypto rx=0x7fa8e800a830 tx=0x7fa8dc0036f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e4e2c4a2342f79be server_cookie=0 in_seq=0 out_seq=0 2023-08-09T15:12:19.872 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:19.867+0000 7fa8f4cfe640 1 -- 172.21.15.43:0/409160546 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa8e80072d0 con 0x7fa8dc00b480 2023-08-09T15:12:19.872 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:19.867+0000 7fa8f4cfe640 1 -- 172.21.15.43:0/409160546 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa8e802f070 con 0x7fa8dc00b480 2023-08-09T15:12:19.872 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:19.867+0000 7fa8f4cfe640 1 -- 172.21.15.43:0/409160546 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa8e8007690 con 0x7fa8dc00b480 2023-08-09T15:12:19.873 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:19.868+0000 7fa8f7f64640 1 -- 172.21.15.43:0/409160546 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa8dc00b480 msgr2=0x7fa8dc00d940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T15:12:19.873 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:19.868+0000 7fa8f7f64640 1 --2- 172.21.15.43:0/409160546 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa8dc00b480 0x7fa8dc00d940 secure :-1 s=READY pgs=2557 cs=0 l=1 rev1=1 crypto rx=0x7fa8e800a830 tx=0x7fa8dc0036f0 comp rx=0 tx=0).stop 2023-08-09T15:12:19.873 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:19.868+0000 7fa8f7f64640 1 -- 172.21.15.43:0/409160546 shutdown_connections 2023-08-09T15:12:19.873 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:19.868+0000 7fa8f7f64640 1 --2- 172.21.15.43:0/409160546 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa8dc00b480 0x7fa8dc00d940 unknown :-1 s=CLOSED pgs=2557 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T15:12:19.873 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:19.868+0000 7fa8f7f64640 1 -- 172.21.15.43:0/409160546 >> 172.21.15.43:0/409160546 conn(0x7fa8f0069d70 msgr2=0x7fa8f00cac70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-09T15:12:19.873 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:19.868+0000 7fa8f7f64640 1 -- 172.21.15.43:0/409160546 shutdown_connections 2023-08-09T15:12:19.873 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:19.868+0000 7fa8f7f64640 1 -- 172.21.15.43:0/409160546 wait complete. 2023-08-09T15:12:19.874 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:19.870+0000 7fa8f7f64640 1 Processor -- start 2023-08-09T15:12:19.874 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:19.870+0000 7fa8f7f64640 1 -- start start 2023-08-09T15:12:19.875 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:19.871+0000 7fa8f7f64640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa8dc00b480 0x7fa8f0141ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T15:12:19.875 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:19.871+0000 7fa8f7f64640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fa8f01413a0 0x7fa8f0156350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T15:12:19.876 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:19.871+0000 7fa8f7f64640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fa8f0156890 0x7fa8f00b0f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T15:12:19.876 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:19.871+0000 7fa8f7f64640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa8f0156d10 con 0x7fa8dc00b480 2023-08-09T15:12:19.876 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:19.871+0000 7fa8f7f64640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa8f01f6680 con 0x7fa8f01413a0 2023-08-09T15:12:19.876 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:19.871+0000 7fa8f7f64640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa8f0141820 con 0x7fa8f0156890 2023-08-09T15:12:19.876 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:19.872+0000 7fa8f5d00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa8dc00b480 0x7fa8f0141ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T15:12:19.876 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:19.872+0000 7fa8f5d00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa8dc00b480 0x7fa8f0141ae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:46260/0 (socket says 172.21.15.43:46260) 2023-08-09T15:12:19.876 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:19.872+0000 7fa8f5d00640 1 -- 172.21.15.43:0/884331849 learned_addr learned my addr 172.21.15.43:0/884331849 (peer_addr_for_me v2:172.21.15.43:0/0) 2023-08-09T15:12:19.877 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:19.872+0000 7fa8f54ff640 1 --2- 172.21.15.43:0/884331849 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fa8f01413a0 0x7fa8f0156350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T15:12:19.877 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:19.872+0000 7fa8f5d00640 1 -- 172.21.15.43:0/884331849 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fa8f0156890 msgr2=0x7fa8f00b0f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T15:12:19.877 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:19.872+0000 7fa8f5d00640 1 --2- 172.21.15.43:0/884331849 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fa8f0156890 0x7fa8f00b0f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T15:12:19.877 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:19.872+0000 7fa8f5d00640 1 -- 172.21.15.43:0/884331849 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fa8f01413a0 msgr2=0x7fa8f0156350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T15:12:19.877 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:19.872+0000 7fa8f5d00640 1 --2- 172.21.15.43:0/884331849 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fa8f01413a0 0x7fa8f0156350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T15:12:19.877 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:19.872+0000 7fa8f5d00640 1 -- 172.21.15.43:0/884331849 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa8f01f5ab0 con 0x7fa8dc00b480 2023-08-09T15:12:19.877 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:19.872+0000 7fa8f54ff640 1 --2- 172.21.15.43:0/884331849 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fa8f01413a0 0x7fa8f0156350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-08-09T15:12:19.877 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:19.872+0000 7fa8f5d00640 1 --2- 172.21.15.43:0/884331849 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa8dc00b480 0x7fa8f0141ae0 secure :-1 s=READY pgs=2558 cs=0 l=1 rev1=1 crypto rx=0x7fa8ec00d770 tx=0x7fa8ec00dc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-09T15:12:19.878 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:19.873+0000 7fa8e67fc640 1 -- 172.21.15.43:0/884331849 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa8ec005270 con 0x7fa8dc00b480 2023-08-09T15:12:19.878 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:19.873+0000 7fa8e67fc640 1 -- 172.21.15.43:0/884331849 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa8ec0053d0 con 0x7fa8dc00b480 2023-08-09T15:12:19.878 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:19.873+0000 7fa8e67fc640 1 -- 172.21.15.43:0/884331849 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa8ec00b8f0 con 0x7fa8dc00b480 2023-08-09T15:12:19.878 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:19.873+0000 7fa8f7f64640 1 -- 172.21.15.43:0/884331849 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa8f01f5d50 con 0x7fa8dc00b480 2023-08-09T15:12:19.878 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:19.873+0000 7fa8f7f64640 1 -- 172.21.15.43:0/884331849 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa8f01f6310 con 0x7fa8dc00b480 2023-08-09T15:12:19.879 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:19.874+0000 7fa8f7f64640 1 -- 172.21.15.43:0/884331849 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa8f01f64d0 con 0x7fa8dc00b480 2023-08-09T15:12:19.879 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:19.875+0000 7fa8e67fc640 1 -- 172.21.15.43:0/884331849 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 129) v1 ==== 88443+0+0 (secure 0 0 0) 0x7fa8ec00ba50 con 0x7fa8dc00b480 2023-08-09T15:12:19.879 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:19.876+0000 7fa8e67fc640 1 --2- 172.21.15.43:0/884331849 >> [v2:172.21.15.203:6832/1456699667,v1:172.21.15.203:6833/1456699667] conn(0x7fa8d406a1a0 0x7fa8d406c660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T15:12:19.879 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:19.876+0000 7fa8e67fc640 1 -- 172.21.15.43:0/884331849 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(229..229 src has 1..229) v4 ==== 6462+0+0 (secure 0 0 0) 0x7fa8ec088900 con 0x7fa8dc00b480 2023-08-09T15:12:19.879 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:19.877+0000 7fa8f54ff640 1 --2- 172.21.15.43:0/884331849 >> [v2:172.21.15.203:6832/1456699667,v1:172.21.15.203:6833/1456699667] conn(0x7fa8d406a1a0 0x7fa8d406c660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T15:12:19.880 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:19.878+0000 7fa8f54ff640 1 --2- 172.21.15.43:0/884331849 >> [v2:172.21.15.203:6832/1456699667,v1:172.21.15.203:6833/1456699667] conn(0x7fa8d406a1a0 0x7fa8d406c660 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7fa8f015c4c0 tx=0x7fa8e0008040 comp rx=0 tx=0).ready entity=mgr.9232 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-09T15:12:19.882 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:19.880+0000 7fa8e67fc640 1 -- 172.21.15.43:0/884331849 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178066 (secure 0 0 0) 0x7fa8ec053c30 con 0x7fa8dc00b480 2023-08-09T15:12:20.071 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:20.070+0000 7fa8f7f64640 1 -- 172.21.15.43:0/884331849 --> [v2:172.21.15.203:6832/1456699667,v1:172.21.15.203:6833/1456699667] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7fa8f015d4b0 con 0x7fa8d406a1a0 2023-08-09T15:12:20.079 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:20.078+0000 7fa8e67fc640 1 -- 172.21.15.43:0/884331849 <== mgr.9232 v2:172.21.15.203:6832/1456699667 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+70673 (secure 0 0 0) 0x7fa8e00119a0 con 0x7fa8d406a1a0 2023-08-09T15:12:20.080 INFO:teuthology.orchestra.run.smithi043.stdout: 2023-08-09T15:12:20.080 INFO:teuthology.orchestra.run.smithi043.stderr:dumped all 2023-08-09T15:12:20.082 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:20.080+0000 7fa8f7f64640 1 -- 172.21.15.43:0/884331849 >> [v2:172.21.15.203:6832/1456699667,v1:172.21.15.203:6833/1456699667] conn(0x7fa8d406a1a0 msgr2=0x7fa8d406c660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T15:12:20.082 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:20.080+0000 7fa8f7f64640 1 --2- 172.21.15.43:0/884331849 >> [v2:172.21.15.203:6832/1456699667,v1:172.21.15.203:6833/1456699667] conn(0x7fa8d406a1a0 0x7fa8d406c660 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7fa8f015c4c0 tx=0x7fa8e0008040 comp rx=0 tx=0).stop 2023-08-09T15:12:20.082 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:20.080+0000 7fa8f7f64640 1 -- 172.21.15.43:0/884331849 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa8dc00b480 msgr2=0x7fa8f0141ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T15:12:20.082 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:20.080+0000 7fa8f7f64640 1 --2- 172.21.15.43:0/884331849 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa8dc00b480 0x7fa8f0141ae0 secure :-1 s=READY pgs=2558 cs=0 l=1 rev1=1 crypto rx=0x7fa8ec00d770 tx=0x7fa8ec00dc30 comp rx=0 tx=0).stop 2023-08-09T15:12:20.082 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:20.080+0000 7fa8f7f64640 1 -- 172.21.15.43:0/884331849 shutdown_connections 2023-08-09T15:12:20.083 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:20.080+0000 7fa8f7f64640 1 --2- 172.21.15.43:0/884331849 >> [v2:172.21.15.203:6832/1456699667,v1:172.21.15.203:6833/1456699667] conn(0x7fa8d406a1a0 0x7fa8d406c660 unknown :-1 s=CLOSED pgs=41 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T15:12:20.083 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:20.080+0000 7fa8f7f64640 1 --2- 172.21.15.43:0/884331849 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fa8f0156890 0x7fa8f00b0f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T15:12:20.083 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:20.080+0000 7fa8f7f64640 1 --2- 172.21.15.43:0/884331849 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fa8f01413a0 0x7fa8f0156350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T15:12:20.083 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:20.080+0000 7fa8f7f64640 1 --2- 172.21.15.43:0/884331849 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa8dc00b480 0x7fa8f0141ae0 unknown :-1 s=CLOSED pgs=2558 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T15:12:20.083 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:20.080+0000 7fa8f7f64640 1 -- 172.21.15.43:0/884331849 >> 172.21.15.43:0/884331849 conn(0x7fa8f0069d70 msgr2=0x7fa8f0070b30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-09T15:12:20.083 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:20.080+0000 7fa8f7f64640 1 -- 172.21.15.43:0/884331849 shutdown_connections 2023-08-09T15:12:20.084 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:20.080+0000 7fa8f7f64640 1 -- 172.21.15.43:0/884331849 wait complete. 2023-08-09T15:12:20.101 INFO:teuthology.orchestra.run.smithi043.stdout:{"pg_ready":true,"pg_map":{"version":13,"stamp":"2023-08-09T15:12:19.363975+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2155,"num_read_kb":4508,"num_write":583,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":351,"ondisk_log_size":351,"up":18,"acting":18,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":91,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":393136,"kb_used_data":23616,"kb_used_omap":13,"kb_used_meta":369394,"kb_avail":754581584,"statfs":{"total":773094113280,"available":772691542016,"internally_reserved":0,"allocated":24182784,"data_stored":13716990,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":14248,"internal_metadata":378259544},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":16,"apply_latency_ms":16,"commit_latency_ns":16000000,"apply_latency_ns":16000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-676856,"num_objects":-24,"num_object_clones":0,"num_object_copies":-48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-24,"num_whiteouts":0,"num_read":-310,"num_read_kb":-155,"num_write":-319,"num_write_kb":-2330,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":-14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.743755"},"pg_stats":[{"pgid":"1.0","version":"0'0","reported_seq":425,"reported_epoch":229,"state":"active+clean","last_fresh":"2023-08-09T15:12:15.642682+0000","last_change":"2023-08-09T13:33:56.401406+0000","last_active":"2023-08-09T15:12:15.642682+0000","last_peered":"2023-08-09T15:12:15.642682+0000","last_clean":"2023-08-09T15:12:15.642682+0000","last_became_active":"2023-08-09T13:33:56.400487+0000","last_became_peered":"2023-08-09T13:33:56.400487+0000","last_unstale":"2023-08-09T15:12:15.642682+0000","last_undegraded":"2023-08-09T15:12:15.642682+0000","last_fullsized":"2023-08-09T15:12:15.642682+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:33:54.845869+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:33:54.845869+0000","last_clean_scrub_stamp":"2023-08-09T13:33:54.845869+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T15:26:23.628880+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":432,"reported_epoch":229,"state":"active+clean","last_fresh":"2023-08-09T15:12:16.906007+0000","last_change":"2023-08-09T13:33:56.401123+0000","last_active":"2023-08-09T15:12:16.906007+0000","last_peered":"2023-08-09T15:12:16.906007+0000","last_clean":"2023-08-09T15:12:16.906007+0000","last_became_active":"2023-08-09T13:33:56.399933+0000","last_became_peered":"2023-08-09T13:33:56.399933+0000","last_unstale":"2023-08-09T15:12:16.906007+0000","last_undegraded":"2023-08-09T15:12:16.906007+0000","last_fullsized":"2023-08-09T15:12:16.906007+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:33:54.845869+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:33:54.845869+0000","last_clean_scrub_stamp":"2023-08-09T13:33:54.845869+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-11T00:04:51.245862+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.2","version":"0'0","reported_seq":424,"reported_epoch":229,"state":"active+clean","last_fresh":"2023-08-09T15:12:15.641337+0000","last_change":"2023-08-09T13:33:56.402794+0000","last_active":"2023-08-09T15:12:15.641337+0000","last_peered":"2023-08-09T15:12:15.641337+0000","last_clean":"2023-08-09T15:12:15.641337+0000","last_became_active":"2023-08-09T13:33:56.402038+0000","last_became_peered":"2023-08-09T13:33:56.402038+0000","last_unstale":"2023-08-09T15:12:15.641337+0000","last_undegraded":"2023-08-09T15:12:15.641337+0000","last_fullsized":"2023-08-09T15:12:15.641337+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:33:54.845869+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:33:54.845869+0000","last_clean_scrub_stamp":"2023-08-09T13:33:54.845869+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T20:39:01.972486+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.3","version":"0'0","reported_seq":438,"reported_epoch":229,"state":"active+clean","last_fresh":"2023-08-09T15:12:15.644104+0000","last_change":"2023-08-09T13:33:55.886749+0000","last_active":"2023-08-09T15:12:15.644104+0000","last_peered":"2023-08-09T15:12:15.644104+0000","last_clean":"2023-08-09T15:12:15.644104+0000","last_became_active":"2023-08-09T13:33:55.886116+0000","last_became_peered":"2023-08-09T13:33:55.886116+0000","last_unstale":"2023-08-09T15:12:15.644104+0000","last_undegraded":"2023-08-09T15:12:15.644104+0000","last_fullsized":"2023-08-09T15:12:15.644104+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:33:54.845869+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:33:54.845869+0000","last_clean_scrub_stamp":"2023-08-09T13:33:54.845869+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T19:17:22.620341+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"226'351","reported_seq":1404,"reported_epoch":229,"state":"active+clean","last_fresh":"2023-08-09T15:12:15.642688+0000","last_change":"2023-08-09T13:33:55.886345+0000","last_active":"2023-08-09T15:12:15.642688+0000","last_peered":"2023-08-09T15:12:15.642688+0000","last_clean":"2023-08-09T15:12:15.642688+0000","last_became_active":"2023-08-09T13:33:55.885781+0000","last_became_peered":"2023-08-09T13:33:55.885781+0000","last_unstale":"2023-08-09T15:12:15.642688+0000","last_undegraded":"2023-08-09T15:12:15.642688+0000","last_fullsized":"2023-08-09T15:12:15.642688+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:33:54.845869+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:33:54.845869+0000","last_clean_scrub_stamp":"2023-08-09T13:33:54.845869+0000","objects_scrubbed":0,"log_size":351,"log_dups_size":0,"ondisk_log_size":351,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T18:20:34.406519+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2155,"num_read_kb":4508,"num_write":583,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":442,"reported_epoch":229,"state":"active+clean","last_fresh":"2023-08-09T15:12:15.646193+0000","last_change":"2023-08-09T13:33:55.889565+0000","last_active":"2023-08-09T15:12:15.646193+0000","last_peered":"2023-08-09T15:12:15.646193+0000","last_clean":"2023-08-09T15:12:15.646193+0000","last_became_active":"2023-08-09T13:33:55.889164+0000","last_became_peered":"2023-08-09T13:33:55.889164+0000","last_unstale":"2023-08-09T15:12:15.646193+0000","last_undegraded":"2023-08-09T15:12:15.646193+0000","last_fullsized":"2023-08-09T15:12:15.646193+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:33:54.845869+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:33:54.845869+0000","last_clean_scrub_stamp":"2023-08-09T13:33:54.845869+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-11T01:05:24.824606+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":430,"reported_epoch":229,"state":"active+clean","last_fresh":"2023-08-09T15:12:17.176452+0000","last_change":"2023-08-09T13:33:56.404236+0000","last_active":"2023-08-09T15:12:17.176452+0000","last_peered":"2023-08-09T15:12:17.176452+0000","last_clean":"2023-08-09T15:12:17.176452+0000","last_became_active":"2023-08-09T13:33:56.402485+0000","last_became_peered":"2023-08-09T13:33:56.402485+0000","last_unstale":"2023-08-09T15:12:17.176452+0000","last_undegraded":"2023-08-09T15:12:17.176452+0000","last_fullsized":"2023-08-09T15:12:17.176452+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:33:54.845869+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:33:54.845869+0000","last_clean_scrub_stamp":"2023-08-09T13:33:54.845869+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T14:46:58.528793+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":422,"reported_epoch":229,"state":"active+clean","last_fresh":"2023-08-09T15:12:17.182618+0000","last_change":"2023-08-09T13:33:56.401781+0000","last_active":"2023-08-09T15:12:17.182618+0000","last_peered":"2023-08-09T15:12:17.182618+0000","last_clean":"2023-08-09T15:12:17.182618+0000","last_became_active":"2023-08-09T13:33:56.400336+0000","last_became_peered":"2023-08-09T13:33:56.400336+0000","last_unstale":"2023-08-09T15:12:17.182618+0000","last_undegraded":"2023-08-09T15:12:17.182618+0000","last_fullsized":"2023-08-09T15:12:17.182618+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:33:54.845869+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:33:54.845869+0000","last_clean_scrub_stamp":"2023-08-09T13:33:54.845869+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T19:20:37.616910+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":422,"reported_epoch":229,"state":"active+clean","last_fresh":"2023-08-09T15:12:17.186106+0000","last_change":"2023-08-09T13:33:55.886389+0000","last_active":"2023-08-09T15:12:17.186106+0000","last_peered":"2023-08-09T15:12:17.186106+0000","last_clean":"2023-08-09T15:12:17.186106+0000","last_became_active":"2023-08-09T13:33:55.885638+0000","last_became_peered":"2023-08-09T13:33:55.885638+0000","last_unstale":"2023-08-09T15:12:17.186106+0000","last_undegraded":"2023-08-09T15:12:17.186106+0000","last_fullsized":"2023-08-09T15:12:17.186106+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:33:54.845869+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:33:54.845869+0000","last_clean_scrub_stamp":"2023-08-09T13:33:54.845869+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T17:59:55.723013+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2155,"num_read_kb":4508,"num_write":583,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":351,"ondisk_log_size":351,"up":2,"acting":2,"num_store_stats":2},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":8}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834576021,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":52552,"kb_used_data":3388,"kb_used_omap":1,"kb_used_meta":49150,"kb_avail":94319288,"statfs":{"total":96636764160,"available":96582950912,"internally_reserved":0,"allocated":3469312,"data_stored":2157740,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1781,"internal_metadata":50329867},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Aug 9 15:11:52 2023","interfaces":[{"interface":"back","average":{"1min":1.113,"5min":1.234,"15min":1.2290000000000001},"min":{"1min":0.52400000000000002,"5min":0.52400000000000002,"15min":0.52400000000000002},"max":{"1min":1.6080000000000001,"5min":2.056,"15min":2.056},"last":0.99399999999999999},{"interface":"front","average":{"1min":1.0900000000000001,"5min":1.2110000000000001,"15min":1.2529999999999999},"min":{"1min":0.55800000000000005,"5min":0.55800000000000005,"15min":0.55800000000000005},"max":{"1min":1.5960000000000001,"5min":1.7769999999999999,"15min":1.863},"last":1.0189999999999999}]},{"osd":1,"last update":"Wed Aug 9 15:11:52 2023","interfaces":[{"interface":"back","average":{"1min":1.1479999999999999,"5min":1.28,"15min":1.3049999999999999},"min":{"1min":0.52500000000000002,"5min":0.52500000000000002,"15min":0.52500000000000002},"max":{"1min":1.6619999999999999,"5min":1.853,"15min":1.853},"last":1.177},{"interface":"front","average":{"1min":1.175,"5min":1.296,"15min":1.3100000000000001},"min":{"1min":0.748,"5min":0.748,"15min":0.748},"max":{"1min":1.605,"5min":2.1520000000000001,"15min":2.1520000000000001},"last":0.96599999999999997}]},{"osd":2,"last update":"Wed Aug 9 15:11:52 2023","interfaces":[{"interface":"back","average":{"1min":1.093,"5min":1.262,"15min":1.2969999999999999},"min":{"1min":0.70399999999999996,"5min":0.70399999999999996,"15min":0.70399999999999996},"max":{"1min":1.7050000000000001,"5min":1.861,"15min":1.861},"last":0.93200000000000005},{"interface":"front","average":{"1min":1.214,"5min":1.3480000000000001,"15min":1.393},"min":{"1min":0.58099999999999996,"5min":0.58099999999999996,"15min":0.58099999999999996},"max":{"1min":1.7889999999999999,"5min":1.9370000000000001,"15min":1.9370000000000001},"last":1.0620000000000001}]},{"osd":3,"last update":"Wed Aug 9 15:11:52 2023","interfaces":[{"interface":"back","average":{"1min":1.1950000000000001,"5min":1.3160000000000001,"15min":1.351},"min":{"1min":0.80700000000000005,"5min":0.80700000000000005,"15min":0.80700000000000005},"max":{"1min":1.798,"5min":1.798,"15min":1.9259999999999999},"last":1.0389999999999999},{"interface":"front","average":{"1min":1.335,"5min":1.4379999999999999,"15min":1.472},"min":{"1min":0.76200000000000001,"5min":0.76200000000000001,"15min":0.76200000000000001},"max":{"1min":1.9179999999999999,"5min":1.9239999999999999,"15min":1.9239999999999999},"last":1.125}]},{"osd":4,"last update":"Wed Aug 9 15:11:52 2023","interfaces":[{"interface":"back","average":{"1min":1.218,"5min":1.282,"15min":1.2989999999999999},"min":{"1min":0.56399999999999995,"5min":0.56399999999999995,"15min":0.56399999999999995},"max":{"1min":1.6080000000000001,"5min":2.2810000000000001,"15min":2.2810000000000001},"last":0.80000000000000004},{"interface":"front","average":{"1min":1.28,"5min":1.3560000000000001,"15min":1.385},"min":{"1min":0.622,"5min":0.622,"15min":0.622},"max":{"1min":2.0390000000000001,"5min":2.0390000000000001,"15min":2.0830000000000002},"last":0.73699999999999999}]},{"osd":5,"last update":"Wed Aug 9 15:11:52 2023","interfaces":[{"interface":"back","average":{"1min":1.3620000000000001,"5min":1.4219999999999999,"15min":1.4410000000000001},"min":{"1min":0.59299999999999997,"5min":0.59299999999999997,"15min":0.59299999999999997},"max":{"1min":2.0899999999999999,"5min":2.0899999999999999,"15min":2.0899999999999999},"last":1.2470000000000001},{"interface":"front","average":{"1min":1.3959999999999999,"5min":1.448,"15min":1.4730000000000001},"min":{"1min":0.65500000000000003,"5min":0.65500000000000003,"15min":0.65500000000000003},"max":{"1min":2.206,"5min":2.5310000000000001,"15min":2.5310000000000001},"last":1.1439999999999999}]},{"osd":6,"last update":"Wed Aug 9 15:11:52 2023","interfaces":[{"interface":"back","average":{"1min":1.5069999999999999,"5min":1.587,"15min":1.589},"min":{"1min":0.69199999999999995,"5min":0.69199999999999995,"15min":0.69199999999999995},"max":{"1min":2.294,"5min":2.8959999999999999,"15min":2.8959999999999999},"last":1.1020000000000001},{"interface":"front","average":{"1min":1.359,"5min":1.5089999999999999,"15min":1.534},"min":{"1min":0.60899999999999999,"5min":0.60899999999999999,"15min":0.60899999999999999},"max":{"1min":2.161,"5min":2.214,"15min":2.222},"last":1.085}]}]},{"osd":6,"up_from":13,"seq":55834576021,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":60160,"kb_used_data":2804,"kb_used_omap":1,"kb_used_meta":57342,"kb_avail":94311680,"statfs":{"total":96636764160,"available":96575160320,"internally_reserved":0,"allocated":2871296,"data_stored":1566011,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1781,"internal_metadata":58718475},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Aug 9 15:11:57 2023","interfaces":[{"interface":"back","average":{"1min":1.1639999999999999,"5min":1.2130000000000001,"15min":1.2190000000000001},"min":{"1min":0.76700000000000002,"5min":0.75600000000000001,"15min":0.75600000000000001},"max":{"1min":1.8169999999999999,"5min":1.8169999999999999,"15min":1.8859999999999999},"last":1.1859999999999999},{"interface":"front","average":{"1min":1.0720000000000001,"5min":1.21,"15min":1.234},"min":{"1min":0.64500000000000002,"5min":0.64500000000000002,"15min":0.64500000000000002},"max":{"1min":1.5109999999999999,"5min":2.0800000000000001,"15min":2.0800000000000001},"last":0.88700000000000001}]},{"osd":1,"last update":"Wed Aug 9 15:11:57 2023","interfaces":[{"interface":"back","average":{"1min":1.095,"5min":1.1879999999999999,"15min":1.1910000000000001},"min":{"1min":0.65900000000000003,"5min":0.65900000000000003,"15min":0.65900000000000003},"max":{"1min":1.776,"5min":2.6760000000000002,"15min":2.6760000000000002},"last":0.97499999999999998},{"interface":"front","average":{"1min":1.2869999999999999,"5min":1.3120000000000001,"15min":1.3460000000000001},"min":{"1min":0.72599999999999998,"5min":0.72599999999999998,"15min":0.72599999999999998},"max":{"1min":1.579,"5min":1.998,"15min":1.998},"last":1.093}]},{"osd":2,"last update":"Wed Aug 9 15:11:57 2023","interfaces":[{"interface":"back","average":{"1min":1.2629999999999999,"5min":1.3120000000000001,"15min":1.3400000000000001},"min":{"1min":0.79800000000000004,"5min":0.79800000000000004,"15min":0.79800000000000004},"max":{"1min":1.6659999999999999,"5min":2.0089999999999999,"15min":2.081},"last":1.25},{"interface":"front","average":{"1min":1.3080000000000001,"5min":1.337,"15min":1.3480000000000001},"min":{"1min":0.73799999999999999,"5min":0.73799999999999999,"15min":0.73799999999999999},"max":{"1min":1.75,"5min":1.9219999999999999,"15min":1.9219999999999999},"last":0.81799999999999995}]},{"osd":3,"last update":"Wed Aug 9 15:11:57 2023","interfaces":[{"interface":"back","average":{"1min":1.302,"5min":1.4099999999999999,"15min":1.4019999999999999},"min":{"1min":0.70299999999999996,"5min":0.70299999999999996,"15min":0.70299999999999996},"max":{"1min":2.2669999999999999,"5min":2.2669999999999999,"15min":2.2669999999999999},"last":0.73199999999999998},{"interface":"front","average":{"1min":1.2270000000000001,"5min":1.319,"15min":1.3420000000000001},"min":{"1min":0.59699999999999998,"5min":0.59699999999999998,"15min":0.59699999999999998},"max":{"1min":1.855,"5min":2.0539999999999998,"15min":2.0539999999999998},"last":0.92000000000000004}]},{"osd":4,"last update":"Wed Aug 9 15:11:57 2023","interfaces":[{"interface":"back","average":{"1min":1.339,"5min":1.3500000000000001,"15min":1.339},"min":{"1min":0.46500000000000002,"5min":0.46500000000000002,"15min":0.46500000000000002},"max":{"1min":1.7270000000000001,"5min":2.0470000000000002,"15min":2.0470000000000002},"last":1.3740000000000001},{"interface":"front","average":{"1min":1.288,"5min":1.4470000000000001,"15min":1.4330000000000001},"min":{"1min":0.53000000000000003,"5min":0.53000000000000003,"15min":0.53000000000000003},"max":{"1min":1.6559999999999999,"5min":2.1589999999999998,"15min":2.1589999999999998},"last":0.93999999999999995}]},{"osd":5,"last update":"Wed Aug 9 15:11:57 2023","interfaces":[{"interface":"back","average":{"1min":1.3109999999999999,"5min":1.431,"15min":1.381},"min":{"1min":0.51100000000000001,"5min":0.51100000000000001,"15min":0.51100000000000001},"max":{"1min":1.9610000000000001,"5min":2.1269999999999998,"15min":2.6059999999999999},"last":0.84899999999999998},{"interface":"front","average":{"1min":1.546,"5min":1.637,"15min":1.6299999999999999},"min":{"1min":0.67300000000000004,"5min":0.67300000000000004,"15min":0.67300000000000004},"max":{"1min":1.885,"5min":2.125,"15min":2.1899999999999999},"last":1.3100000000000001}]},{"osd":7,"last update":"Wed Aug 9 15:11:57 2023","interfaces":[{"interface":"back","average":{"1min":1.506,"5min":1.639,"15min":1.6180000000000001},"min":{"1min":0.626,"5min":0.626,"15min":0.626},"max":{"1min":1.9319999999999999,"5min":2.2280000000000002,"15min":2.4340000000000002},"last":0.99299999999999999},{"interface":"front","average":{"1min":1.45,"5min":1.6180000000000001,"15min":1.6000000000000001},"min":{"1min":0.44600000000000001,"5min":0.44600000000000001,"15min":0.44600000000000001},"max":{"1min":2.0409999999999999,"5min":2.3300000000000001,"15min":2.7400000000000002},"last":1.069}]}]},{"osd":5,"up_from":13,"seq":55834576021,"num_pgs":17,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":47876,"kb_used_data":2808,"kb_used_omap":1,"kb_used_meta":45054,"kb_avail":94323964,"statfs":{"total":96636764160,"available":96587739136,"internally_reserved":0,"allocated":2875392,"data_stored":1567372,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1781,"internal_metadata":46135563},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Aug 9 15:11:47 2023","interfaces":[{"interface":"back","average":{"1min":1.0669999999999999,"5min":1.1919999999999999,"15min":1.194},"min":{"1min":0.57699999999999996,"5min":0.57699999999999996,"15min":0.57699999999999996},"max":{"1min":1.494,"5min":1.7370000000000001,"15min":1.754},"last":0.76300000000000001},{"interface":"front","average":{"1min":1.0389999999999999,"5min":1.167,"15min":1.208},"min":{"1min":0.55600000000000005,"5min":0.55600000000000005,"15min":0.55600000000000005},"max":{"1min":1.3480000000000001,"5min":1.6870000000000001,"15min":1.976},"last":1.0680000000000001}]},{"osd":1,"last update":"Wed Aug 9 15:11:47 2023","interfaces":[{"interface":"back","average":{"1min":1.2490000000000001,"5min":1.28,"15min":1.2509999999999999},"min":{"1min":0.67500000000000004,"5min":0.65200000000000002,"15min":0.65200000000000002},"max":{"1min":1.9490000000000001,"5min":2.016,"15min":2.016},"last":0.69799999999999995},{"interface":"front","average":{"1min":1.2150000000000001,"5min":1.288,"15min":1.343},"min":{"1min":0.65000000000000002,"5min":0.65000000000000002,"15min":0.65000000000000002},"max":{"1min":2.0169999999999999,"5min":2.0169999999999999,"15min":2.0169999999999999},"last":1.6259999999999999}]},{"osd":2,"last update":"Wed Aug 9 15:11:47 2023","interfaces":[{"interface":"back","average":{"1min":1.0269999999999999,"5min":1.2629999999999999,"15min":1.3009999999999999},"min":{"1min":0.60199999999999998,"5min":0.60199999999999998,"15min":0.60199999999999998},"max":{"1min":1.48,"5min":1.8859999999999999,"15min":2.032},"last":1.3180000000000001},{"interface":"front","average":{"1min":1.3,"5min":1.339,"15min":1.3640000000000001},"min":{"1min":0.61899999999999999,"5min":0.61899999999999999,"15min":0.61899999999999999},"max":{"1min":1.7010000000000001,"5min":1.956,"15min":1.964},"last":1.4770000000000001}]},{"osd":3,"last update":"Wed Aug 9 15:11:47 2023","interfaces":[{"interface":"back","average":{"1min":1.3,"5min":1.3740000000000001,"15min":1.3939999999999999},"min":{"1min":0.746,"5min":0.746,"15min":0.746},"max":{"1min":1.6639999999999999,"5min":1.798,"15min":1.899},"last":1.373},{"interface":"front","average":{"1min":1.278,"5min":1.389,"15min":1.4370000000000001},"min":{"1min":0.66400000000000003,"5min":0.66400000000000003,"15min":0.66400000000000003},"max":{"1min":1.6559999999999999,"5min":1.8819999999999999,"15min":2.0379999999999998},"last":1.226}]},{"osd":4,"last update":"Wed Aug 9 15:11:47 2023","interfaces":[{"interface":"back","average":{"1min":1.347,"5min":1.323,"15min":1.3220000000000001},"min":{"1min":0.53000000000000003,"5min":0.53000000000000003,"15min":0.53000000000000003},"max":{"1min":1.833,"5min":2.0299999999999998,"15min":2.0299999999999998},"last":1.7210000000000001},{"interface":"front","average":{"1min":1.4419999999999999,"5min":1.389,"15min":1.4059999999999999},"min":{"1min":0.69199999999999995,"5min":0.69199999999999995,"15min":0.69199999999999995},"max":{"1min":1.7709999999999999,"5min":1.9239999999999999,"15min":2.0390000000000001},"last":1.742}]},{"osd":6,"last update":"Wed Aug 9 15:11:47 2023","interfaces":[{"interface":"back","average":{"1min":1.298,"5min":1.3660000000000001,"15min":1.411},"min":{"1min":0.501,"5min":0.501,"15min":0.501},"max":{"1min":1.728,"5min":2.032,"15min":2.1000000000000001},"last":1.675},{"interface":"front","average":{"1min":1.48,"5min":1.4850000000000001,"15min":1.5029999999999999},"min":{"1min":0.70599999999999996,"5min":0.70599999999999996,"15min":0.70599999999999996},"max":{"1min":2.125,"5min":2.1349999999999998,"15min":2.1349999999999998},"last":1.819}]},{"osd":7,"last update":"Wed Aug 9 15:11:47 2023","interfaces":[{"interface":"back","average":{"1min":1.5469999999999999,"5min":1.554,"15min":1.5800000000000001},"min":{"1min":0.71999999999999997,"5min":0.71999999999999997,"15min":0.71999999999999997},"max":{"1min":2.0760000000000001,"5min":2.0760000000000001,"15min":2.4689999999999999},"last":1.9370000000000001},{"interface":"front","average":{"1min":1.5309999999999999,"5min":1.595,"15min":1.637},"min":{"1min":0.63800000000000001,"5min":0.63800000000000001,"15min":0.63800000000000001},"max":{"1min":2.1709999999999998,"5min":2.1709999999999998,"15min":2.4239999999999999},"last":1.5609999999999999}]}]},{"osd":4,"up_from":13,"seq":55834576021,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":51972,"kb_used_data":2808,"kb_used_omap":1,"kb_used_meta":49150,"kb_avail":94319868,"statfs":{"total":96636764160,"available":96583544832,"internally_reserved":0,"allocated":2875392,"data_stored":1567372,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1781,"internal_metadata":50329867},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Aug 9 15:12:13 2023","interfaces":[{"interface":"back","average":{"1min":1.2370000000000001,"5min":1.196,"15min":1.222},"min":{"1min":0.64700000000000002,"5min":0.60399999999999998,"15min":0.60399999999999998},"max":{"1min":1.8169999999999999,"5min":1.8169999999999999,"15min":2.2959999999999998},"last":0.90500000000000003},{"interface":"front","average":{"1min":1.2270000000000001,"5min":1.2829999999999999,"15min":1.3129999999999999},"min":{"1min":0.77000000000000002,"5min":0.66800000000000004,"15min":0.66800000000000004},"max":{"1min":1.772,"5min":2.0779999999999998,"15min":2.2709999999999999},"last":1.454}]},{"osd":1,"last update":"Wed Aug 9 15:12:13 2023","interfaces":[{"interface":"back","average":{"1min":1.1439999999999999,"5min":1.288,"15min":1.3460000000000001},"min":{"1min":0.73199999999999998,"5min":0.73199999999999998,"15min":0.73199999999999998},"max":{"1min":1.9370000000000001,"5min":1.9370000000000001,"15min":1.9370000000000001},"last":0.97299999999999998},{"interface":"front","average":{"1min":1.2430000000000001,"5min":1.2609999999999999,"15min":1.278},"min":{"1min":0.76400000000000001,"5min":0.73899999999999999,"15min":0.73899999999999999},"max":{"1min":1.865,"5min":1.865,"15min":2.117},"last":1.379}]},{"osd":2,"last update":"Wed Aug 9 15:12:13 2023","interfaces":[{"interface":"back","average":{"1min":1.1859999999999999,"5min":1.2350000000000001,"15min":1.2829999999999999},"min":{"1min":0.70199999999999996,"5min":0.63100000000000001,"15min":0.63100000000000001},"max":{"1min":1.5469999999999999,"5min":1.7709999999999999,"15min":1.883},"last":1.026},{"interface":"front","average":{"1min":1.298,"5min":1.3819999999999999,"15min":1.401},"min":{"1min":0.91400000000000003,"5min":0.82699999999999996,"15min":0.79200000000000004},"max":{"1min":1.903,"5min":2.343,"15min":2.343},"last":1.127}]},{"osd":3,"last update":"Wed Aug 9 15:12:13 2023","interfaces":[{"interface":"back","average":{"1min":1.2789999999999999,"5min":1.3420000000000001,"15min":1.349},"min":{"1min":0.95299999999999996,"5min":0.90500000000000003,"15min":0.88700000000000001},"max":{"1min":1.6970000000000001,"5min":1.8009999999999999,"15min":2.4870000000000001},"last":1.6970000000000001},{"interface":"front","average":{"1min":1.3720000000000001,"5min":1.423,"15min":1.464},"min":{"1min":0.95299999999999996,"5min":0.79700000000000004,"15min":0.79700000000000004},"max":{"1min":1.8240000000000001,"5min":1.909,"15min":2.5459999999999998},"last":1.1990000000000001}]},{"osd":5,"last update":"Wed Aug 9 15:12:13 2023","interfaces":[{"interface":"back","average":{"1min":1.431,"5min":1.385,"15min":1.409},"min":{"1min":1.0029999999999999,"5min":0.82199999999999995,"15min":0.82199999999999995},"max":{"1min":2.004,"5min":2.633,"15min":2.633},"last":1.2470000000000001},{"interface":"front","average":{"1min":1.395,"5min":1.4059999999999999,"15min":1.393},"min":{"1min":1.048,"5min":0.68100000000000005,"15min":0.68100000000000005},"max":{"1min":1.8300000000000001,"5min":1.9930000000000001,"15min":2.4249999999999998},"last":1.5509999999999999}]},{"osd":6,"last update":"Wed Aug 9 15:12:13 2023","interfaces":[{"interface":"back","average":{"1min":1.3919999999999999,"5min":1.4330000000000001,"15min":1.4099999999999999},"min":{"1min":1.0509999999999999,"5min":0.71299999999999997,"15min":0.71299999999999997},"max":{"1min":1.73,"5min":2.0049999999999999,"15min":2.9119999999999999},"last":1.3149999999999999},{"interface":"front","average":{"1min":1.528,"5min":1.5680000000000001,"15min":1.579},"min":{"1min":1.1699999999999999,"5min":0.77000000000000002,"15min":0.77000000000000002},"max":{"1min":1.8680000000000001,"5min":2.9529999999999998,"15min":2.9529999999999998},"last":1.5309999999999999}]},{"osd":7,"last update":"Wed Aug 9 15:12:13 2023","interfaces":[{"interface":"back","average":{"1min":1.5029999999999999,"5min":1.5309999999999999,"15min":1.5229999999999999},"min":{"1min":1.1719999999999999,"5min":0.83899999999999997,"15min":0.83899999999999997},"max":{"1min":1.9370000000000001,"5min":2.1589999999999998,"15min":2.5910000000000002},"last":1.796},{"interface":"front","average":{"1min":1.599,"5min":1.6140000000000001,"15min":1.6060000000000001},"min":{"1min":1.206,"5min":0.90000000000000002,"15min":0.90000000000000002},"max":{"1min":1.8999999999999999,"5min":2.218,"15min":3.028},"last":1.502}]}]},{"osd":3,"up_from":13,"seq":55834576022,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":47876,"kb_used_data":2808,"kb_used_omap":1,"kb_used_meta":45054,"kb_avail":94323964,"statfs":{"total":96636764160,"available":96587739136,"internally_reserved":0,"allocated":2875392,"data_stored":1567372,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1781,"internal_metadata":46135563},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Aug 9 15:12:06 2023","interfaces":[{"interface":"back","average":{"1min":0.75800000000000001,"5min":0.94699999999999995,"15min":0.96899999999999997},"min":{"1min":0.42799999999999999,"5min":0.42799999999999999,"15min":0.42799999999999999},"max":{"1min":1.24,"5min":1.6699999999999999,"15min":2.2090000000000001},"last":0.64500000000000002},{"interface":"front","average":{"1min":0.872,"5min":1.093,"15min":1.1439999999999999},"min":{"1min":0.434,"5min":0.434,"15min":0.434},"max":{"1min":1.4550000000000001,"5min":1.9630000000000001,"15min":1.9630000000000001},"last":0.92000000000000004}]},{"osd":1,"last update":"Wed Aug 9 15:12:06 2023","interfaces":[{"interface":"back","average":{"1min":0.90500000000000003,"5min":1.006,"15min":1.0660000000000001},"min":{"1min":0.437,"5min":0.437,"15min":0.437},"max":{"1min":1.595,"5min":1.956,"15min":2.27},"last":0.77500000000000002},{"interface":"front","average":{"1min":1.0960000000000001,"5min":1.2070000000000001,"15min":1.2569999999999999},"min":{"1min":0.63900000000000001,"5min":0.498,"15min":0.498},"max":{"1min":1.6140000000000001,"5min":1.78,"15min":1.9730000000000001},"last":1.0620000000000001}]},{"osd":2,"last update":"Wed Aug 9 15:12:06 2023","interfaces":[{"interface":"back","average":{"1min":1.024,"5min":1.236,"15min":1.3180000000000001},"min":{"1min":0.65400000000000003,"5min":0.65400000000000003,"15min":0.65400000000000003},"max":{"1min":1.3839999999999999,"5min":1.9650000000000001,"15min":2.4079999999999999},"last":0.88500000000000001},{"interface":"front","average":{"1min":1.2250000000000001,"5min":1.3720000000000001,"15min":1.4279999999999999},"min":{"1min":0.63400000000000001,"5min":0.63400000000000001,"15min":0.63400000000000001},"max":{"1min":1.7090000000000001,"5min":2.0510000000000002,"15min":2.1259999999999999},"last":1.248}]},{"osd":4,"last update":"Wed Aug 9 15:12:06 2023","interfaces":[{"interface":"back","average":{"1min":1.2070000000000001,"5min":1.3260000000000001,"15min":1.3959999999999999},"min":{"1min":0.752,"5min":0.748,"15min":0.72599999999999998},"max":{"1min":1.744,"5min":1.9179999999999999,"15min":2.2069999999999999},"last":1.04},{"interface":"front","average":{"1min":1.3220000000000001,"5min":1.4199999999999999,"15min":1.4570000000000001},"min":{"1min":0.77500000000000002,"5min":0.77500000000000002,"15min":0.77500000000000002},"max":{"1min":1.8939999999999999,"5min":1.9279999999999999,"15min":2.056},"last":1.4419999999999999}]},{"osd":5,"last update":"Wed Aug 9 15:12:06 2023","interfaces":[{"interface":"back","average":{"1min":1.4039999999999999,"5min":1.502,"15min":1.538},"min":{"1min":0.77600000000000002,"5min":0.77600000000000002,"15min":0.77600000000000002},"max":{"1min":2.1890000000000001,"5min":2.1890000000000001,"15min":2.1890000000000001},"last":1.327},{"interface":"front","average":{"1min":1.3740000000000001,"5min":1.5580000000000001,"15min":1.6000000000000001},"min":{"1min":0.88800000000000001,"5min":0.85799999999999998,"15min":0.85799999999999998},"max":{"1min":1.7330000000000001,"5min":2.2469999999999999,"15min":2.4049999999999998},"last":1.49}]},{"osd":6,"last update":"Wed Aug 9 15:12:06 2023","interfaces":[{"interface":"back","average":{"1min":1.3560000000000001,"5min":1.4890000000000001,"15min":1.5780000000000001},"min":{"1min":0.69599999999999995,"5min":0.69599999999999995,"15min":0.69599999999999995},"max":{"1min":1.9670000000000001,"5min":2.4860000000000002,"15min":2.4860000000000002},"last":0.96799999999999997},{"interface":"front","average":{"1min":1.458,"5min":1.5940000000000001,"15min":1.653},"min":{"1min":0.89700000000000002,"5min":0.89700000000000002,"15min":0.89700000000000002},"max":{"1min":2.04,"5min":2.4039999999999999,"15min":2.4039999999999999},"last":1.1279999999999999}]},{"osd":7,"last update":"Wed Aug 9 15:12:06 2023","interfaces":[{"interface":"back","average":{"1min":1.5620000000000001,"5min":1.696,"15min":1.7490000000000001},"min":{"1min":0.94799999999999995,"5min":0.94799999999999995,"15min":0.94799999999999995},"max":{"1min":1.8680000000000001,"5min":2.347,"15min":2.347},"last":1.3620000000000001},{"interface":"front","average":{"1min":1.4179999999999999,"5min":1.6220000000000001,"15min":1.694},"min":{"1min":0.81100000000000005,"5min":0.81100000000000005,"15min":0.81100000000000005},"max":{"1min":2.1309999999999998,"5min":3.3220000000000001,"15min":3.3220000000000001},"last":0.998}]}]},{"osd":2,"up_from":13,"seq":55834576021,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":46220,"kb_used_data":2808,"kb_used_omap":1,"kb_used_meta":43390,"kb_avail":94325620,"statfs":{"total":96636764160,"available":96589434880,"internally_reserved":0,"allocated":2875392,"data_stored":1567372,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1781,"internal_metadata":44431627},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Aug 9 15:11:52 2023","interfaces":[{"interface":"back","average":{"1min":0.874,"5min":0.98499999999999999,"15min":0.95899999999999996},"min":{"1min":0.48199999999999998,"5min":0.44800000000000001,"15min":0.44800000000000001},"max":{"1min":1.591,"5min":1.591,"15min":1.591},"last":1.742},{"interface":"front","average":{"1min":0.96399999999999997,"5min":1.0760000000000001,"15min":1.085},"min":{"1min":0.38900000000000001,"5min":0.38900000000000001,"15min":0.38900000000000001},"max":{"1min":1.5980000000000001,"5min":1.764,"15min":2.052},"last":1.8149999999999999}]},{"osd":1,"last update":"Wed Aug 9 15:11:52 2023","interfaces":[{"interface":"back","average":{"1min":0.98299999999999998,"5min":1.2330000000000001,"15min":1.2230000000000001},"min":{"1min":0.55400000000000005,"5min":0.55400000000000005,"15min":0.55400000000000005},"max":{"1min":1.403,"5min":2.0179999999999998,"15min":2.0179999999999998},"last":1.9570000000000001},{"interface":"front","average":{"1min":1.1359999999999999,"5min":1.292,"15min":1.381},"min":{"1min":0.51200000000000001,"5min":0.51200000000000001,"15min":0.51200000000000001},"max":{"1min":1.6639999999999999,"5min":1.95,"15min":1.95},"last":1.927}]},{"osd":3,"last update":"Wed Aug 9 15:11:52 2023","interfaces":[{"interface":"back","average":{"1min":0.99099999999999999,"5min":1.089,"15min":1.1080000000000001},"min":{"1min":0.53500000000000003,"5min":0.52100000000000002,"15min":0.52100000000000002},"max":{"1min":1.5649999999999999,"5min":1.871,"15min":1.871},"last":0.71899999999999997},{"interface":"front","average":{"1min":1.111,"5min":1.339,"15min":1.369},"min":{"1min":0.57099999999999995,"5min":0.57099999999999995,"15min":0.57099999999999995},"max":{"1min":1.6930000000000001,"5min":1.9299999999999999,"15min":2.0470000000000002},"last":1.782}]},{"osd":4,"last update":"Wed Aug 9 15:11:52 2023","interfaces":[{"interface":"back","average":{"1min":1.369,"5min":1.5229999999999999,"15min":1.5469999999999999},"min":{"1min":0.86399999999999999,"5min":0.86399999999999999,"15min":0.86399999999999999},"max":{"1min":1.7350000000000001,"5min":1.994,"15min":2.464},"last":1.8600000000000001},{"interface":"front","average":{"1min":1.395,"5min":1.4399999999999999,"15min":1.464},"min":{"1min":0.93600000000000005,"5min":0.81799999999999995,"15min":0.81799999999999995},"max":{"1min":1.9370000000000001,"5min":1.98,"15min":2.0720000000000001},"last":1.4830000000000001}]},{"osd":5,"last update":"Wed Aug 9 15:11:52 2023","interfaces":[{"interface":"back","average":{"1min":1.4279999999999999,"5min":1.5009999999999999,"15min":1.5169999999999999},"min":{"1min":0.64800000000000002,"5min":0.64800000000000002,"15min":0.64800000000000002},"max":{"1min":1.913,"5min":2.0369999999999999,"15min":2.1139999999999999},"last":1.6000000000000001},{"interface":"front","average":{"1min":1.351,"5min":1.419,"15min":1.4870000000000001},"min":{"1min":0.83399999999999996,"5min":0.83399999999999996,"15min":0.83399999999999996},"max":{"1min":2.1520000000000001,"5min":2.1520000000000001,"15min":2.214},"last":1.677}]},{"osd":6,"last update":"Wed Aug 9 15:11:52 2023","interfaces":[{"interface":"back","average":{"1min":1.407,"5min":1.4890000000000001,"15min":1.5229999999999999},"min":{"1min":0.89500000000000002,"5min":0.89500000000000002,"15min":0.89500000000000002},"max":{"1min":2.0680000000000001,"5min":2.1110000000000002,"15min":2.2999999999999998},"last":1.5209999999999999},{"interface":"front","average":{"1min":1.512,"5min":1.6479999999999999,"15min":1.6379999999999999},"min":{"1min":0.998,"5min":0.998,"15min":0.998},"max":{"1min":2.4140000000000001,"5min":2.4140000000000001,"15min":2.4140000000000001},"last":1.9099999999999999}]},{"osd":7,"last update":"Wed Aug 9 15:11:52 2023","interfaces":[{"interface":"back","average":{"1min":1.5249999999999999,"5min":1.605,"15min":1.6419999999999999},"min":{"1min":0.96299999999999997,"5min":0.87,"15min":0.87},"max":{"1min":2.1000000000000001,"5min":2.1749999999999998,"15min":2.302},"last":1.698},{"interface":"front","average":{"1min":1.4650000000000001,"5min":1.5800000000000001,"15min":1.6299999999999999},"min":{"1min":0.71399999999999997,"5min":0.71399999999999997,"15min":0.71399999999999997},"max":{"1min":2.2360000000000002,"5min":2.242,"15min":2.3809999999999998},"last":1.466}]}]},{"osd":1,"up_from":13,"seq":55834576021,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":56656,"kb_used_data":3388,"kb_used_omap":1,"kb_used_meta":53246,"kb_avail":94315184,"statfs":{"total":96636764160,"available":96578748416,"internally_reserved":0,"allocated":3469312,"data_stored":2157740,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1781,"internal_metadata":54524171},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Aug 9 15:12:03 2023","interfaces":[{"interface":"back","average":{"1min":0.83399999999999996,"5min":0.97399999999999998,"15min":1},"min":{"1min":0.32100000000000001,"5min":0.32100000000000001,"15min":0.32100000000000001},"max":{"1min":1.153,"5min":1.8799999999999999,"15min":2.0939999999999999},"last":0.81499999999999995},{"interface":"front","average":{"1min":0.94299999999999995,"5min":1.1060000000000001,"15min":1.1399999999999999},"min":{"1min":0.40500000000000003,"5min":0.40500000000000003,"15min":0.40500000000000003},"max":{"1min":1.641,"5min":1.78,"15min":1.8939999999999999},"last":0.46800000000000003}]},{"osd":2,"last update":"Wed Aug 9 15:12:03 2023","interfaces":[{"interface":"back","average":{"1min":0.93700000000000006,"5min":1.018,"15min":1.04},"min":{"1min":0.626,"5min":0.55800000000000005,"15min":0.55800000000000005},"max":{"1min":2.2240000000000002,"5min":2.2240000000000002,"15min":2.3940000000000001},"last":1.206},{"interface":"front","average":{"1min":0.97399999999999998,"5min":1.1759999999999999,"15min":1.254},"min":{"1min":0.49299999999999999,"5min":0.46700000000000003,"15min":0.46700000000000003},"max":{"1min":1.5149999999999999,"5min":2.169,"15min":2.169},"last":0.86299999999999999}]},{"osd":3,"last update":"Wed Aug 9 15:12:03 2023","interfaces":[{"interface":"back","average":{"1min":1.115,"5min":1.3340000000000001,"15min":1.381},"min":{"1min":0.54600000000000004,"5min":0.51000000000000001,"15min":0.51000000000000001},"max":{"1min":1.833,"5min":2.093,"15min":2.109},"last":0.69099999999999995},{"interface":"front","average":{"1min":1.0409999999999999,"5min":1.256,"15min":1.3089999999999999},"min":{"1min":0.59099999999999997,"5min":0.59099999999999997,"15min":0.59099999999999997},"max":{"1min":1.8680000000000001,"5min":2.052,"15min":2.052},"last":0.66200000000000003}]},{"osd":4,"last update":"Wed Aug 9 15:12:03 2023","interfaces":[{"interface":"back","average":{"1min":1.254,"5min":1.4339999999999999,"15min":1.4730000000000001},"min":{"1min":0.82099999999999995,"5min":0.753,"15min":0.753},"max":{"1min":1.887,"5min":2.0419999999999998,"15min":2.214},"last":1.3600000000000001},{"interface":"front","average":{"1min":1.202,"5min":1.381,"15min":1.4119999999999999},"min":{"1min":0.84499999999999997,"5min":0.69199999999999995,"15min":0.69199999999999995},"max":{"1min":1.528,"5min":2.0310000000000001,"15min":2.1190000000000002},"last":0.91100000000000003}]},{"osd":5,"last update":"Wed Aug 9 15:12:03 2023","interfaces":[{"interface":"back","average":{"1min":1.2450000000000001,"5min":1.415,"15min":1.446},"min":{"1min":0.83399999999999996,"5min":0.77800000000000002,"15min":0.77800000000000002},"max":{"1min":1.909,"5min":1.994,"15min":2.508},"last":1.0189999999999999},{"interface":"front","average":{"1min":1.2829999999999999,"5min":1.53,"15min":1.5800000000000001},"min":{"1min":0.74299999999999999,"5min":0.74299999999999999,"15min":0.74299999999999999},"max":{"1min":1.99,"5min":2.3809999999999998,"15min":2.3809999999999998},"last":1.3009999999999999}]},{"osd":6,"last update":"Wed Aug 9 15:12:03 2023","interfaces":[{"interface":"back","average":{"1min":1.321,"5min":1.575,"15min":1.613},"min":{"1min":0.80100000000000005,"5min":0.80100000000000005,"15min":0.80100000000000005},"max":{"1min":2.1669999999999998,"5min":2.347,"15min":2.347},"last":1.165},{"interface":"front","average":{"1min":1.288,"5min":1.522,"15min":1.5720000000000001},"min":{"1min":0.84999999999999998,"5min":0.73499999999999999,"15min":0.73499999999999999},"max":{"1min":1.8140000000000001,"5min":2.218,"15min":2.278},"last":1.101}]},{"osd":7,"last update":"Wed Aug 9 15:12:03 2023","interfaces":[{"interface":"back","average":{"1min":1.3959999999999999,"5min":1.6439999999999999,"15min":1.6910000000000001},"min":{"1min":0.90900000000000003,"5min":0.86399999999999999,"15min":0.86399999999999999},"max":{"1min":2.1299999999999999,"5min":2.3090000000000002,"15min":2.488},"last":1.5129999999999999},{"interface":"front","average":{"1min":1.385,"5min":1.633,"15min":1.698},"min":{"1min":0.75600000000000001,"5min":0.75600000000000001,"15min":0.75600000000000001},"max":{"1min":2.4980000000000002,"5min":2.4980000000000002,"15min":2.4980000000000002},"last":1.2629999999999999}]}]},{"osd":0,"up_from":13,"seq":55834576021,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":29824,"kb_used_data":2804,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94342016,"statfs":{"total":96636764160,"available":96606224384,"internally_reserved":0,"allocated":2871296,"data_stored":1566011,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1781,"internal_metadata":27654411},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Wed Aug 9 15:12:00 2023","interfaces":[{"interface":"back","average":{"1min":0.85099999999999998,"5min":0.89600000000000002,"15min":0.95099999999999996},"min":{"1min":0.44900000000000001,"5min":0.40000000000000002,"15min":0.40000000000000002},"max":{"1min":1.3380000000000001,"5min":1.617,"15min":1.6299999999999999},"last":0.86599999999999999},{"interface":"front","average":{"1min":0.98799999999999999,"5min":1.091,"15min":1.135},"min":{"1min":0.56599999999999995,"5min":0.56599999999999995,"15min":0.56599999999999995},"max":{"1min":1.635,"5min":2.1560000000000001,"15min":2.1560000000000001},"last":0.95099999999999996}]},{"osd":2,"last update":"Wed Aug 9 15:12:00 2023","interfaces":[{"interface":"back","average":{"1min":0.873,"5min":1.097,"15min":1.1719999999999999},"min":{"1min":0.54000000000000004,"5min":0.505,"15min":0.505},"max":{"1min":1.401,"5min":2.4769999999999999,"15min":2.4769999999999999},"last":0.97799999999999998},{"interface":"front","average":{"1min":1.0069999999999999,"5min":1.125,"15min":1.2150000000000001},"min":{"1min":0.55500000000000005,"5min":0.53100000000000003,"15min":0.53100000000000003},"max":{"1min":1.468,"5min":1.8180000000000001,"15min":1.9470000000000001},"last":0.67300000000000004}]},{"osd":3,"last update":"Wed Aug 9 15:12:00 2023","interfaces":[{"interface":"back","average":{"1min":0.91200000000000003,"5min":1.1040000000000001,"15min":1.1870000000000001},"min":{"1min":0.45500000000000002,"5min":0.45500000000000002,"15min":0.45500000000000002},"max":{"1min":1.629,"5min":2.1459999999999999,"15min":2.1459999999999999},"last":1.002},{"interface":"front","average":{"1min":1.079,"5min":1.284,"15min":1.3779999999999999},"min":{"1min":0.53100000000000003,"5min":0.53100000000000003,"15min":0.53100000000000003},"max":{"1min":1.5640000000000001,"5min":1.867,"15min":2.0760000000000001},"last":1.1060000000000001}]},{"osd":4,"last update":"Wed Aug 9 15:12:00 2023","interfaces":[{"interface":"back","average":{"1min":1.163,"5min":1.321,"15min":1.403},"min":{"1min":0.81799999999999995,"5min":0.81799999999999995,"15min":0.81799999999999995},"max":{"1min":1.5629999999999999,"5min":1.853,"15min":1.966},"last":1.29},{"interface":"front","average":{"1min":1.296,"5min":1.421,"15min":1.5069999999999999},"min":{"1min":0.83099999999999996,"5min":0.79400000000000004,"15min":0.79400000000000004},"max":{"1min":1.734,"5min":1.972,"15min":2.278},"last":1.4299999999999999}]},{"osd":5,"last update":"Wed Aug 9 15:12:00 2023","interfaces":[{"interface":"back","average":{"1min":1.256,"5min":1.476,"15min":1.5129999999999999},"min":{"1min":0.84699999999999998,"5min":0.84699999999999998,"15min":0.84699999999999998},"max":{"1min":1.79,"5min":2.355,"15min":2.355},"last":0.79800000000000004},{"interface":"front","average":{"1min":1.2849999999999999,"5min":1.4299999999999999,"15min":1.49},"min":{"1min":0.84799999999999998,"5min":0.82899999999999996,"15min":0.82899999999999996},"max":{"1min":2.089,"5min":2.089,"15min":2.3740000000000001},"last":1.802}]},{"osd":6,"last update":"Wed Aug 9 15:12:00 2023","interfaces":[{"interface":"back","average":{"1min":1.3129999999999999,"5min":1.5249999999999999,"15min":1.6160000000000001},"min":{"1min":0.79700000000000004,"5min":0.79700000000000004,"15min":0.79700000000000004},"max":{"1min":1.7889999999999999,"5min":2.7490000000000001,"15min":2.7490000000000001},"last":1.343},{"interface":"front","average":{"1min":1.3839999999999999,"5min":1.534,"15min":1.597},"min":{"1min":0.89200000000000002,"5min":0.84699999999999998,"15min":0.84699999999999998},"max":{"1min":2.0840000000000001,"5min":2.274,"15min":2.2839999999999998},"last":1.5109999999999999}]},{"osd":7,"last update":"Wed Aug 9 15:12:00 2023","interfaces":[{"interface":"back","average":{"1min":1.4710000000000001,"5min":1.599,"15min":1.665},"min":{"1min":0.98599999999999999,"5min":0.94899999999999995,"15min":0.94899999999999995},"max":{"1min":2.2989999999999999,"5min":2.645,"15min":2.645},"last":1.7410000000000001},{"interface":"front","average":{"1min":1.4019999999999999,"5min":1.54,"15min":1.609},"min":{"1min":0.82599999999999996,"5min":0.82599999999999996,"15min":0.82599999999999996},"max":{"1min":2.1829999999999998,"5min":2.1829999999999998,"15min":2.3759999999999999},"last":1.6499999999999999}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-08-09T15:12:20.104 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-08-09T15:12:20.310 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:20.308+0000 7fdc38e0d640 1 Processor -- start 2023-08-09T15:12:20.310 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:20.308+0000 7fdc38e0d640 1 -- start start 2023-08-09T15:12:20.312 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:20.310+0000 7fdc38e0d640 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7fdc340cdbe0 0x7fdc340afc70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T15:12:20.312 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:20.310+0000 7fdc38e0d640 1 -- --> v1:172.21.15.43:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fdc3406c2c0 con 0x7fdc3406fe60 2023-08-09T15:12:20.313 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:20.310+0000 7fdc38e0d640 1 -- --> v1:172.21.15.203:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fdc340cb690 con 0x7fdc3406b4b0 2023-08-09T15:12:20.313 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:20.310+0000 7fdc38e0d640 1 -- --> v2:172.21.15.203:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fdc340cdff0 con 0x7fdc340cdbe0 2023-08-09T15:12:20.313 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:20.311+0000 7fdc3259c640 1 --1- >> v1:172.21.15.43:6789/0 conn(0x7fdc3406fe60 0x7fdc3415be10 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.43:6789/0 says I am v1:172.21.15.43:32804/0 (socket says 172.21.15.43:32804) 2023-08-09T15:12:20.313 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:20.311+0000 7fdc3259c640 1 -- 172.21.15.43:0/1215923591 learned_addr learned my addr 172.21.15.43:0/1215923591 (peer_addr_for_me v1:172.21.15.43:0/0) 2023-08-09T15:12:20.316 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:20.311+0000 7fdc32d9d640 1 --2- 172.21.15.43:0/1215923591 >> v2:172.21.15.203:3300/0 conn(0x7fdc340cdbe0 0x7fdc340afc70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T15:12:20.316 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:20.311+0000 7fdc3159a640 1 -- 172.21.15.43:0/1215923591 <== mon.0 v1:172.21.15.43:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 230598764 0 0) 0x7fdc3406c2c0 con 0x7fdc3406fe60 2023-08-09T15:12:20.316 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:20.311+0000 7fdc3159a640 1 -- 172.21.15.43:0/1215923591 --> v1:172.21.15.43:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fdc1c003520 con 0x7fdc3406fe60 2023-08-09T15:12:20.317 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:20.312+0000 7fdc3159a640 1 -- 172.21.15.43:0/1215923591 <== mon.0 v1:172.21.15.43:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3921124556 0 0) 0x7fdc1c003520 con 0x7fdc3406fe60 2023-08-09T15:12:20.317 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:20.312+0000 7fdc3159a640 1 -- 172.21.15.43:0/1215923591 >> v2:172.21.15.203:3300/0 conn(0x7fdc340cdbe0 msgr2=0x7fdc340afc70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-09T15:12:20.317 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:20.312+0000 7fdc3159a640 1 --2- 172.21.15.43:0/1215923591 >> v2:172.21.15.203:3300/0 conn(0x7fdc340cdbe0 0x7fdc340afc70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T15:12:20.317 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:20.312+0000 7fdc3159a640 1 -- 172.21.15.43:0/1215923591 >> v1:172.21.15.203:6789/0 conn(0x7fdc3406b4b0 legacy=0x7fdc3415b310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T15:12:20.317 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:20.312+0000 7fdc3159a640 1 -- 172.21.15.43:0/1215923591 --> v1:172.21.15.43:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdc3406ba20 con 0x7fdc3406fe60 2023-08-09T15:12:20.317 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:20.312+0000 7fdc3159a640 1 -- 172.21.15.43:0/1215923591 <== mon.0 v1:172.21.15.43:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1285150740 0 0) 0x7fdc24001670 con 0x7fdc3406fe60 2023-08-09T15:12:20.318 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:20.312+0000 7fdc3159a640 1 -- 172.21.15.43:0/1215923591 >> v1:172.21.15.43:6789/0 conn(0x7fdc3406fe60 legacy=0x7fdc3415be10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T15:12:20.318 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:20.312+0000 7fdc3159a640 1 --2- 172.21.15.43:0/1215923591 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fdc1c003970 0x7fdc1c003d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T15:12:20.318 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:20.312+0000 7fdc3159a640 1 --2- 172.21.15.43:0/1215923591 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fdc1c0047c0 0x7fdc1c004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T15:12:20.319 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:20.312+0000 7fdc3159a640 1 --2- 172.21.15.43:0/1215923591 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fdc1c005180 0x7fdc1c00b980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T15:12:20.319 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:20.312+0000 7fdc3159a640 1 -- 172.21.15.43:0/1215923591 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdc340cdff0 con 0x7fdc1c005180 2023-08-09T15:12:20.319 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:20.312+0000 7fdc3159a640 1 -- 172.21.15.43:0/1215923591 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdc1c00c020 con 0x7fdc1c003970 2023-08-09T15:12:20.319 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:20.312+0000 7fdc3159a640 1 -- 172.21.15.43:0/1215923591 --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdc1c00c1e0 con 0x7fdc1c0047c0 2023-08-09T15:12:20.320 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:20.312+0000 7fdc32d9d640 1 --2- 172.21.15.43:0/1215923591 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fdc1c005180 0x7fdc1c00b980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T15:12:20.320 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:20.312+0000 7fdc3259c640 1 --2- 172.21.15.43:0/1215923591 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fdc1c003970 0x7fdc1c003d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T15:12:20.320 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:20.312+0000 7fdc31d9b640 1 --2- 172.21.15.43:0/1215923591 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fdc1c0047c0 0x7fdc1c004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T15:12:20.320 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:20.313+0000 7fdc32d9d640 1 -- 172.21.15.43:0/1215923591 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fdc1c0047c0 msgr2=0x7fdc1c004c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-09T15:12:20.321 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:20.313+0000 7fdc32d9d640 1 --2- 172.21.15.43:0/1215923591 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fdc1c0047c0 0x7fdc1c004c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T15:12:20.321 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:20.313+0000 7fdc32d9d640 1 -- 172.21.15.43:0/1215923591 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fdc1c003970 msgr2=0x7fdc1c003d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-09T15:12:20.321 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:20.313+0000 7fdc32d9d640 1 --2- 172.21.15.43:0/1215923591 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fdc1c003970 0x7fdc1c003d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T15:12:20.321 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:20.313+0000 7fdc32d9d640 1 -- 172.21.15.43:0/1215923591 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdc1c003520 con 0x7fdc1c005180 2023-08-09T15:12:20.322 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:20.313+0000 7fdc31d9b640 1 --2- 172.21.15.43:0/1215923591 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fdc1c0047c0 0x7fdc1c004c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-08-09T15:12:20.322 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:20.313+0000 7fdc3259c640 1 --2- 172.21.15.43:0/1215923591 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fdc1c003970 0x7fdc1c003d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-08-09T15:12:20.322 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:20.314+0000 7fdc32d9d640 1 -- 172.21.15.43:0/1215923591 reap_dead start 2023-08-09T15:12:20.322 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:20.314+0000 7fdc32d9d640 1 --2- 172.21.15.43:0/1215923591 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fdc1c005180 0x7fdc1c00b980 secure :-1 s=READY pgs=2560 cs=0 l=1 rev1=1 crypto rx=0x7fdc2c009fc0 tx=0x7fdc1c0057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f521a67f93f12cef server_cookie=0 in_seq=0 out_seq=0 2023-08-09T15:12:20.323 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:20.314+0000 7fdc3159a640 1 -- 172.21.15.43:0/1215923591 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdc2c02cc60 con 0x7fdc1c005180 2023-08-09T15:12:20.323 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:20.314+0000 7fdc3159a640 1 -- 172.21.15.43:0/1215923591 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdc2c00f040 con 0x7fdc1c005180 2023-08-09T15:12:20.323 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:20.314+0000 7fdc3159a640 1 -- 172.21.15.43:0/1215923591 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdc2c003990 con 0x7fdc1c005180 2023-08-09T15:12:20.324 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:20.315+0000 7fdc38e0d640 1 -- 172.21.15.43:0/1215923591 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fdc1c005180 msgr2=0x7fdc1c00b980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T15:12:20.324 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:20.315+0000 7fdc38e0d640 1 --2- 172.21.15.43:0/1215923591 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fdc1c005180 0x7fdc1c00b980 secure :-1 s=READY pgs=2560 cs=0 l=1 rev1=1 crypto rx=0x7fdc2c009fc0 tx=0x7fdc1c0057f0 comp rx=0 tx=0).stop 2023-08-09T15:12:20.324 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:20.315+0000 7fdc38e0d640 1 -- 172.21.15.43:0/1215923591 shutdown_connections 2023-08-09T15:12:20.325 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:20.315+0000 7fdc38e0d640 1 --2- 172.21.15.43:0/1215923591 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fdc1c005180 0x7fdc1c00b980 unknown :-1 s=CLOSED pgs=2560 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T15:12:20.325 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:20.315+0000 7fdc38e0d640 1 -- 172.21.15.43:0/1215923591 >> 172.21.15.43:0/1215923591 conn(0x7fdc34069d70 msgr2=0x7fdc340cac70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-09T15:12:20.325 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:20.315+0000 7fdc38e0d640 1 -- 172.21.15.43:0/1215923591 shutdown_connections 2023-08-09T15:12:20.326 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:20.315+0000 7fdc38e0d640 1 -- 172.21.15.43:0/1215923591 wait complete. 2023-08-09T15:12:20.329 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:20.317+0000 7fdc38e0d640 1 Processor -- start 2023-08-09T15:12:20.329 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:20.317+0000 7fdc38e0d640 1 -- start start 2023-08-09T15:12:20.329 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:20.318+0000 7fdc38e0d640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fdc1c005180 0x7fdc34146710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T15:12:20.330 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:20.318+0000 7fdc38e0d640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fdc340c2f20 0x7fdc340b2d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T15:12:20.330 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:20.318+0000 7fdc38e0d640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fdc340af740 0x7fdc340afbc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T15:12:20.330 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:20.318+0000 7fdc38e0d640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdc340b0100 con 0x7fdc1c005180 2023-08-09T15:12:20.330 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:20.318+0000 7fdc38e0d640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdc340c1e80 con 0x7fdc340af740 2023-08-09T15:12:20.330 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:20.318+0000 7fdc38e0d640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdc340c2160 con 0x7fdc340c2f20 2023-08-09T15:12:20.330 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:20.318+0000 7fdc3259c640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fdc1c005180 0x7fdc34146710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T15:12:20.330 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:20.318+0000 7fdc3259c640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fdc1c005180 0x7fdc34146710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:46270/0 (socket says 172.21.15.43:46270) 2023-08-09T15:12:20.331 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:20.318+0000 7fdc3259c640 1 -- 172.21.15.43:0/2070324984 learned_addr learned my addr 172.21.15.43:0/2070324984 (peer_addr_for_me v2:172.21.15.43:0/0) 2023-08-09T15:12:20.331 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:20.318+0000 7fdc3259c640 1 -- 172.21.15.43:0/2070324984 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fdc340c2f20 msgr2=0x7fdc340b2d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T15:12:20.331 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:20.318+0000 7fdc3259c640 1 --2- 172.21.15.43:0/2070324984 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fdc340c2f20 0x7fdc340b2d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T15:12:20.331 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:20.318+0000 7fdc3259c640 1 -- 172.21.15.43:0/2070324984 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fdc340af740 msgr2=0x7fdc340afbc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T15:12:20.331 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:20.318+0000 7fdc3259c640 1 --2- 172.21.15.43:0/2070324984 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fdc340af740 0x7fdc340afbc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T15:12:20.331 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:20.318+0000 7fdc3259c640 1 -- 172.21.15.43:0/2070324984 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdc340c2440 con 0x7fdc1c005180 2023-08-09T15:12:20.331 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:20.319+0000 7fdc3259c640 1 --2- 172.21.15.43:0/2070324984 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fdc1c005180 0x7fdc34146710 secure :-1 s=READY pgs=2561 cs=0 l=1 rev1=1 crypto rx=0x7fdc24000ff0 tx=0x7fdc2400dd40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-09T15:12:20.332 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:20.319+0000 7fdc22ffd640 1 -- 172.21.15.43:0/2070324984 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdc240074e0 con 0x7fdc1c005180 2023-08-09T15:12:20.332 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:20.319+0000 7fdc22ffd640 1 -- 172.21.15.43:0/2070324984 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdc24007640 con 0x7fdc1c005180 2023-08-09T15:12:20.332 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:20.319+0000 7fdc22ffd640 1 -- 172.21.15.43:0/2070324984 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdc240049e0 con 0x7fdc1c005180 2023-08-09T15:12:20.332 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:20.319+0000 7fdc38e0d640 1 -- 172.21.15.43:0/2070324984 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdc340c2720 con 0x7fdc1c005180 2023-08-09T15:12:20.332 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:20.319+0000 7fdc38e0d640 1 -- 172.21.15.43:0/2070324984 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdc340b32d0 con 0x7fdc1c005180 2023-08-09T15:12:20.332 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:20.321+0000 7fdc38e0d640 1 -- 172.21.15.43:0/2070324984 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdc3404db40 con 0x7fdc1c005180 2023-08-09T15:12:20.332 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:20.321+0000 7fdc22ffd640 1 -- 172.21.15.43:0/2070324984 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 129) v1 ==== 88443+0+0 (secure 0 0 0) 0x7fdc24004b40 con 0x7fdc1c005180 2023-08-09T15:12:20.333 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:20.322+0000 7fdc22ffd640 1 --2- 172.21.15.43:0/2070324984 >> [v2:172.21.15.203:6832/1456699667,v1:172.21.15.203:6833/1456699667] conn(0x7fdc1406a170 0x7fdc1406c630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T15:12:20.333 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:20.322+0000 7fdc22ffd640 1 -- 172.21.15.43:0/2070324984 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(229..229 src has 1..229) v4 ==== 6462+0+0 (secure 0 0 0) 0x7fdc24088280 con 0x7fdc1c005180 2023-08-09T15:12:20.333 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:20.323+0000 7fdc31d9b640 1 --2- 172.21.15.43:0/2070324984 >> [v2:172.21.15.203:6832/1456699667,v1:172.21.15.203:6833/1456699667] conn(0x7fdc1406a170 0x7fdc1406c630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T15:12:20.333 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:20.324+0000 7fdc31d9b640 1 --2- 172.21.15.43:0/2070324984 >> [v2:172.21.15.203:6832/1456699667,v1:172.21.15.203:6833/1456699667] conn(0x7fdc1406a170 0x7fdc1406c630 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7fdc28000f60 tx=0x7fdc28009040 comp rx=0 tx=0).ready entity=mgr.9232 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-09T15:12:20.333 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:20.327+0000 7fdc22ffd640 1 -- 172.21.15.43:0/2070324984 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178066 (secure 0 0 0) 0x7fdc24053670 con 0x7fdc1c005180 2023-08-09T15:12:20.520 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:20.519+0000 7fdc38e0d640 1 -- 172.21.15.43:0/2070324984 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fdc340ae000 con 0x7fdc1c005180 2023-08-09T15:12:20.522 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:20.521+0000 7fdc22ffd640 1 -- 172.21.15.43:0/2070324984 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v229) v1 ==== 74+0+16171 (secure 0 0 0) 0x7fdc24056c90 con 0x7fdc1c005180 2023-08-09T15:12:20.523 INFO:teuthology.orchestra.run.smithi043.stdout: 2023-08-09T15:12:20.523 INFO:teuthology.orchestra.run.smithi043.stdout:{"epoch":229,"fsid":"daac87ee-1337-4250-a955-aea4e408a39b","created":"2023-08-09T13:33:41.701198+0000","modified":"2023-08-09T15:12:15.631447+0000","last_up_change":"2023-08-09T13:33:53.783453+0000","last_in_change":"2023-08-09T13:33:45.148904+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":27,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":"rbd","create_time":"2023-08-09T13:33:54.365137+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":".mgr","create_time":"2023-08-09T13:33:54.831803+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"cbd49ecd-e896-4fbd-864e-5c37a829b165","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":221,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6800","nonce":502667335},{"type":"v1","addr":"172.21.15.43:6801","nonce":502667335}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6802","nonce":502667335},{"type":"v1","addr":"172.21.15.43:6803","nonce":502667335}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6806","nonce":502667335},{"type":"v1","addr":"172.21.15.43:6807","nonce":502667335}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6804","nonce":502667335},{"type":"v1","addr":"172.21.15.43:6805","nonce":502667335}]},"public_addr":"172.21.15.43:6801/502667335","cluster_addr":"172.21.15.43:6803/502667335","heartbeat_back_addr":"172.21.15.43:6807/502667335","heartbeat_front_addr":"172.21.15.43:6805/502667335","state":["exists","up"]},{"osd":1,"uuid":"75c7e0fe-1ec7-4fe8-a2f2-3661a65b8500","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":221,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6808","nonce":3289134610},{"type":"v1","addr":"172.21.15.43:6809","nonce":3289134610}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6810","nonce":3289134610},{"type":"v1","addr":"172.21.15.43:6812","nonce":3289134610}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6821","nonce":3289134610},{"type":"v1","addr":"172.21.15.43:6824","nonce":3289134610}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6815","nonce":3289134610},{"type":"v1","addr":"172.21.15.43:6818","nonce":3289134610}]},"public_addr":"172.21.15.43:6809/3289134610","cluster_addr":"172.21.15.43:6812/3289134610","heartbeat_back_addr":"172.21.15.43:6824/3289134610","heartbeat_front_addr":"172.21.15.43:6818/3289134610","state":["exists","up"]},{"osd":2,"uuid":"bb5c41e9-7f5f-4125-9b01-3bb8bfc2ea83","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":221,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6811","nonce":4079709736},{"type":"v1","addr":"172.21.15.43:6814","nonce":4079709736}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6817","nonce":4079709736},{"type":"v1","addr":"172.21.15.43:6820","nonce":4079709736}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6829","nonce":4079709736},{"type":"v1","addr":"172.21.15.43:6831","nonce":4079709736}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6823","nonce":4079709736},{"type":"v1","addr":"172.21.15.43:6826","nonce":4079709736}]},"public_addr":"172.21.15.43:6814/4079709736","cluster_addr":"172.21.15.43:6820/4079709736","heartbeat_back_addr":"172.21.15.43:6831/4079709736","heartbeat_front_addr":"172.21.15.43:6826/4079709736","state":["exists","up"]},{"osd":3,"uuid":"bfe4c8b4-694a-465c-87e6-2b2aab852613","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":221,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6813","nonce":1709538759},{"type":"v1","addr":"172.21.15.43:6816","nonce":1709538759}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6819","nonce":1709538759},{"type":"v1","addr":"172.21.15.43:6822","nonce":1709538759}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6828","nonce":1709538759},{"type":"v1","addr":"172.21.15.43:6830","nonce":1709538759}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6825","nonce":1709538759},{"type":"v1","addr":"172.21.15.43:6827","nonce":1709538759}]},"public_addr":"172.21.15.43:6816/1709538759","cluster_addr":"172.21.15.43:6822/1709538759","heartbeat_back_addr":"172.21.15.43:6830/1709538759","heartbeat_front_addr":"172.21.15.43:6827/1709538759","state":["exists","up"]},{"osd":4,"uuid":"fa2e8a77-aba1-4756-853e-0d174a5bdeba","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":221,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6800","nonce":4196866481},{"type":"v1","addr":"172.21.15.203:6801","nonce":4196866481}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6802","nonce":4196866481},{"type":"v1","addr":"172.21.15.203:6803","nonce":4196866481}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6806","nonce":4196866481},{"type":"v1","addr":"172.21.15.203:6807","nonce":4196866481}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6804","nonce":4196866481},{"type":"v1","addr":"172.21.15.203:6805","nonce":4196866481}]},"public_addr":"172.21.15.203:6801/4196866481","cluster_addr":"172.21.15.203:6803/4196866481","heartbeat_back_addr":"172.21.15.203:6807/4196866481","heartbeat_front_addr":"172.21.15.203:6805/4196866481","state":["exists","up"]},{"osd":5,"uuid":"81811e4b-4d24-4f51-a82c-f1d700a2a00d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":221,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6816","nonce":3059290081},{"type":"v1","addr":"172.21.15.203:6817","nonce":3059290081}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6818","nonce":3059290081},{"type":"v1","addr":"172.21.15.203:6819","nonce":3059290081}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6822","nonce":3059290081},{"type":"v1","addr":"172.21.15.203:6823","nonce":3059290081}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6820","nonce":3059290081},{"type":"v1","addr":"172.21.15.203:6821","nonce":3059290081}]},"public_addr":"172.21.15.203:6817/3059290081","cluster_addr":"172.21.15.203:6819/3059290081","heartbeat_back_addr":"172.21.15.203:6823/3059290081","heartbeat_front_addr":"172.21.15.203:6821/3059290081","state":["exists","up"]},{"osd":6,"uuid":"307bac4d-6a5b-443d-af75-56d695d7674f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":221,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6824","nonce":512155569},{"type":"v1","addr":"172.21.15.203:6825","nonce":512155569}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6826","nonce":512155569},{"type":"v1","addr":"172.21.15.203:6827","nonce":512155569}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6830","nonce":512155569},{"type":"v1","addr":"172.21.15.203:6831","nonce":512155569}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6828","nonce":512155569},{"type":"v1","addr":"172.21.15.203:6829","nonce":512155569}]},"public_addr":"172.21.15.203:6825/512155569","cluster_addr":"172.21.15.203:6827/512155569","heartbeat_back_addr":"172.21.15.203:6831/512155569","heartbeat_front_addr":"172.21.15.203:6829/512155569","state":["exists","up"]},{"osd":7,"uuid":"a95dc7ca-9f1b-4295-af70-68a22a2d7d5e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":221,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6808","nonce":3331450438},{"type":"v1","addr":"172.21.15.203:6809","nonce":3331450438}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6810","nonce":3331450438},{"type":"v1","addr":"172.21.15.203:6811","nonce":3331450438}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6814","nonce":3331450438},{"type":"v1","addr":"172.21.15.203:6815","nonce":3331450438}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6812","nonce":3331450438},{"type":"v1","addr":"172.21.15.203:6813","nonce":3331450438}]},"public_addr":"172.21.15.203:6809/3331450438","cluster_addr":"172.21.15.203:6811/3331450438","heartbeat_back_addr":"172.21.15.203:6815/3331450438","heartbeat_front_addr":"172.21.15.203:6813/3331450438","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-09T13:33:50.547652+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-09T13:33:50.589469+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-09T13:33:50.561724+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-09T13:33:50.599650+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-09T13:33:50.411855+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-09T13:33:50.219681+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-09T13:33:50.504664+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-09T13:33:50.441248+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.43:6832/1425989175":"2023-08-10T15:12:12.613679+0000","172.21.15.43:6833/1425989175":"2023-08-10T15:12:12.613679+0000","172.21.15.203:0/4004150203":"2023-08-10T15:12:07.324983+0000","172.21.15.203:6833/2375771024":"2023-08-10T15:12:07.324983+0000","172.21.15.203:6832/2375771024":"2023-08-10T15:12:07.324983+0000","172.21.15.203:0/1645319310":"2023-08-10T15:12:07.324983+0000","172.21.15.203:0/3324686399":"2023-08-10T15:12:07.324983+0000","172.21.15.203:0/286395424":"2023-08-10T15:12:07.324983+0000","172.21.15.203:0/1761748226":"2023-08-10T15:12:07.324983+0000","172.21.15.203:6832/4034093268":"2023-08-10T15:11:43.130145+0000","172.21.15.203:6833/4034093268":"2023-08-10T15:11:43.130145+0000","172.21.15.203:0/1654193324":"2023-08-10T15:11:43.130145+0000","172.21.15.203:0/3701038305":"2023-08-10T15:12:07.324983+0000","172.21.15.203:0/4140323243":"2023-08-10T15:11:43.130145+0000","172.21.15.203:0/450863258":"2023-08-10T15:11:43.130145+0000","172.21.15.203:0/134877587":"2023-08-10T15:11:43.130145+0000","172.21.15.203:0/3500453783":"2023-08-10T15:12:07.324983+0000","172.21.15.203:0/2330869600":"2023-08-10T15:11:43.130145+0000","172.21.15.203:0/1886393026":"2023-08-10T15:12:07.324983+0000","172.21.15.203:0/2203818837":"2023-08-10T15:11:43.130145+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":7,"snaps":[{"begin":104,"length":1},{"begin":221,"length":1}]},{"pool":27,"snaps":[{"begin":2,"length":2}]}],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-08-09T15:12:20.525 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:20.524+0000 7fdc38e0d640 1 -- 172.21.15.43:0/2070324984 >> [v2:172.21.15.203:6832/1456699667,v1:172.21.15.203:6833/1456699667] conn(0x7fdc1406a170 msgr2=0x7fdc1406c630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T15:12:20.525 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:20.524+0000 7fdc38e0d640 1 --2- 172.21.15.43:0/2070324984 >> [v2:172.21.15.203:6832/1456699667,v1:172.21.15.203:6833/1456699667] conn(0x7fdc1406a170 0x7fdc1406c630 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7fdc28000f60 tx=0x7fdc28009040 comp rx=0 tx=0).stop 2023-08-09T15:12:20.525 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:20.524+0000 7fdc38e0d640 1 -- 172.21.15.43:0/2070324984 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fdc1c005180 msgr2=0x7fdc34146710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T15:12:20.526 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:20.524+0000 7fdc38e0d640 1 --2- 172.21.15.43:0/2070324984 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fdc1c005180 0x7fdc34146710 secure :-1 s=READY pgs=2561 cs=0 l=1 rev1=1 crypto rx=0x7fdc24000ff0 tx=0x7fdc2400dd40 comp rx=0 tx=0).stop 2023-08-09T15:12:20.526 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:20.524+0000 7fdc38e0d640 1 -- 172.21.15.43:0/2070324984 shutdown_connections 2023-08-09T15:12:20.526 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:20.524+0000 7fdc38e0d640 1 --2- 172.21.15.43:0/2070324984 >> [v2:172.21.15.203:6832/1456699667,v1:172.21.15.203:6833/1456699667] conn(0x7fdc1406a170 0x7fdc1406c630 unknown :-1 s=CLOSED pgs=42 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T15:12:20.527 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:20.524+0000 7fdc38e0d640 1 --2- 172.21.15.43:0/2070324984 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fdc340af740 0x7fdc340afbc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T15:12:20.527 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:20.524+0000 7fdc38e0d640 1 --2- 172.21.15.43:0/2070324984 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fdc340c2f20 0x7fdc340b2d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T15:12:20.527 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:20.524+0000 7fdc38e0d640 1 --2- 172.21.15.43:0/2070324984 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fdc1c005180 0x7fdc34146710 unknown :-1 s=CLOSED pgs=2561 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T15:12:20.527 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:20.524+0000 7fdc38e0d640 1 -- 172.21.15.43:0/2070324984 >> 172.21.15.43:0/2070324984 conn(0x7fdc34069d70 msgr2=0x7fdc3415d220 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-09T15:12:20.527 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:20.524+0000 7fdc38e0d640 1 -- 172.21.15.43:0/2070324984 shutdown_connections 2023-08-09T15:12:20.527 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:20.524+0000 7fdc38e0d640 1 -- 172.21.15.43:0/2070324984 wait complete. 2023-08-09T15:12:21.545 INFO:tasks.ceph:Scrubbing osd.0 2023-08-09T15:12:21.546 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.0 config set osd_debug_deep_scrub_sleep 0 2023-08-09T15:12:21.783 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:21.782+0000 7fe0b2bab640 1 Processor -- start 2023-08-09T15:12:21.784 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:21.782+0000 7fe0b2bab640 1 -- start start 2023-08-09T15:12:21.785 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:21.784+0000 7fe0b2bab640 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7fe0ac0cb0e0 0x7fe0ac0abf70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T15:12:21.786 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:21.784+0000 7fe0b2bab640 1 -- --> v1:172.21.15.43:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe0ac160300 con 0x7fe0ac15c710 2023-08-09T15:12:21.786 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:21.784+0000 7fe0b2bab640 1 -- --> v1:172.21.15.203:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe0ac0cb530 con 0x7fe0ac0cac30 2023-08-09T15:12:21.786 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:21.784+0000 7fe0b2bab640 1 -- --> v2:172.21.15.203:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe0ac160550 con 0x7fe0ac0cb0e0 2023-08-09T15:12:21.787 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:21.785+0000 7fe0b0947640 1 --1- >> v1:172.21.15.43:6789/0 conn(0x7fe0ac15c710 0x7fe0ac0c9dd0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.43:6789/0 says I am v1:172.21.15.43:32808/0 (socket says 172.21.15.43:32808) 2023-08-09T15:12:21.787 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:21.785+0000 7fe0b0947640 1 -- 172.21.15.43:0/1323283457 learned_addr learned my addr 172.21.15.43:0/1323283457 (peer_addr_for_me v1:172.21.15.43:0/0) 2023-08-09T15:12:21.787 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:21.785+0000 7fe0b1148640 1 --2- 172.21.15.43:0/1323283457 >> v2:172.21.15.203:3300/0 conn(0x7fe0ac0cb0e0 0x7fe0ac0abf70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T15:12:21.787 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:21.785+0000 7fe0ab7fe640 1 -- 172.21.15.43:0/1323283457 <== mon.0 v1:172.21.15.43:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1123544741 0 0) 0x7fe0ac160300 con 0x7fe0ac15c710 2023-08-09T15:12:21.787 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:21.785+0000 7fe0ab7fe640 1 -- 172.21.15.43:0/1323283457 --> v1:172.21.15.43:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fe09c003520 con 0x7fe0ac15c710 2023-08-09T15:12:21.788 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:21.786+0000 7fe0ab7fe640 1 -- 172.21.15.43:0/1323283457 <== mon.0 v1:172.21.15.43:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1398804282 0 0) 0x7fe09c003520 con 0x7fe0ac15c710 2023-08-09T15:12:21.789 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:21.786+0000 7fe0ab7fe640 1 -- 172.21.15.43:0/1323283457 >> v2:172.21.15.203:3300/0 conn(0x7fe0ac0cb0e0 msgr2=0x7fe0ac0abf70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-09T15:12:21.789 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:21.786+0000 7fe0ab7fe640 1 --2- 172.21.15.43:0/1323283457 >> v2:172.21.15.203:3300/0 conn(0x7fe0ac0cb0e0 0x7fe0ac0abf70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T15:12:21.789 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:21.786+0000 7fe0ab7fe640 1 -- 172.21.15.43:0/1323283457 >> v1:172.21.15.203:6789/0 conn(0x7fe0ac0cac30 legacy=0x7fe0ac0ab860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T15:12:21.789 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:21.786+0000 7fe0ab7fe640 1 -- 172.21.15.43:0/1323283457 --> v1:172.21.15.43:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe0ac160830 con 0x7fe0ac15c710 2023-08-09T15:12:21.789 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:21.786+0000 7fe0ab7fe640 1 -- 172.21.15.43:0/1323283457 <== mon.0 v1:172.21.15.43:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1285150740 0 0) 0x7fe0a0001670 con 0x7fe0ac15c710 2023-08-09T15:12:21.789 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:21.786+0000 7fe0ab7fe640 1 -- 172.21.15.43:0/1323283457 >> v1:172.21.15.43:6789/0 conn(0x7fe0ac15c710 legacy=0x7fe0ac0c9dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T15:12:21.789 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:21.786+0000 7fe0ab7fe640 1 --2- 172.21.15.43:0/1323283457 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fe09c003970 0x7fe09c003d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T15:12:21.789 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:21.786+0000 7fe0ab7fe640 1 --2- 172.21.15.43:0/1323283457 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fe09c0047c0 0x7fe09c004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T15:12:21.790 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:21.786+0000 7fe0ab7fe640 1 --2- 172.21.15.43:0/1323283457 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fe09c0052a0 0x7fe09c00b980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T15:12:21.790 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:21.786+0000 7fe0ab7fe640 1 -- 172.21.15.43:0/1323283457 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe0ac160550 con 0x7fe09c0052a0 2023-08-09T15:12:21.790 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:21.786+0000 7fe0ab7fe640 1 -- 172.21.15.43:0/1323283457 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe09c00c170 con 0x7fe09c003970 2023-08-09T15:12:21.790 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:21.786+0000 7fe0ab7fe640 1 -- 172.21.15.43:0/1323283457 --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe09c00c330 con 0x7fe09c0047c0 2023-08-09T15:12:21.790 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:21.786+0000 7fe0b1148640 1 --2- 172.21.15.43:0/1323283457 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fe09c0052a0 0x7fe09c00b980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T15:12:21.790 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:21.787+0000 7fe0b0947640 1 --2- 172.21.15.43:0/1323283457 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fe09c003970 0x7fe09c003d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T15:12:21.790 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:21.787+0000 7fe0abfff640 1 --2- 172.21.15.43:0/1323283457 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fe09c0047c0 0x7fe09c004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T15:12:21.791 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:21.787+0000 7fe0b1148640 1 -- 172.21.15.43:0/1323283457 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fe09c0047c0 msgr2=0x7fe09c004c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-09T15:12:21.791 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:21.787+0000 7fe0b1148640 1 --2- 172.21.15.43:0/1323283457 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fe09c0047c0 0x7fe09c004c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T15:12:21.791 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:21.787+0000 7fe0b1148640 1 -- 172.21.15.43:0/1323283457 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fe09c003970 msgr2=0x7fe09c003d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-09T15:12:21.791 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:21.787+0000 7fe0b1148640 1 --2- 172.21.15.43:0/1323283457 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fe09c003970 0x7fe09c003d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T15:12:21.791 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:21.787+0000 7fe0b1148640 1 -- 172.21.15.43:0/1323283457 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe09c003520 con 0x7fe09c0052a0 2023-08-09T15:12:21.791 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:21.787+0000 7fe0abfff640 1 --2- 172.21.15.43:0/1323283457 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fe09c0047c0 0x7fe09c004c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-08-09T15:12:21.792 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:21.788+0000 7fe0b1148640 1 -- 172.21.15.43:0/1323283457 reap_dead start 2023-08-09T15:12:21.793 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:21.788+0000 7fe0b1148640 1 --2- 172.21.15.43:0/1323283457 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fe09c0052a0 0x7fe09c00b980 secure :-1 s=READY pgs=2563 cs=0 l=1 rev1=1 crypto rx=0x7fe0a4009f90 tx=0x7fe09c0057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=317b220350005591 server_cookie=0 in_seq=0 out_seq=0 2023-08-09T15:12:21.793 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:21.788+0000 7fe0ab7fe640 1 -- 172.21.15.43:0/1323283457 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe0a402cc00 con 0x7fe09c0052a0 2023-08-09T15:12:21.793 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:21.788+0000 7fe0ab7fe640 1 -- 172.21.15.43:0/1323283457 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe0a400f040 con 0x7fe09c0052a0 2023-08-09T15:12:21.793 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:21.788+0000 7fe0ab7fe640 1 -- 172.21.15.43:0/1323283457 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe0a40039e0 con 0x7fe09c0052a0 2023-08-09T15:12:21.793 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:21.789+0000 7fe0b2bab640 1 -- 172.21.15.43:0/1323283457 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fe09c0052a0 msgr2=0x7fe09c00b980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T15:12:21.793 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:21.789+0000 7fe0b2bab640 1 --2- 172.21.15.43:0/1323283457 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fe09c0052a0 0x7fe09c00b980 secure :-1 s=READY pgs=2563 cs=0 l=1 rev1=1 crypto rx=0x7fe0a4009f90 tx=0x7fe09c0057f0 comp rx=0 tx=0).stop 2023-08-09T15:12:21.793 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:21.789+0000 7fe0b2bab640 1 -- 172.21.15.43:0/1323283457 shutdown_connections 2023-08-09T15:12:21.794 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:21.789+0000 7fe0b2bab640 1 --2- 172.21.15.43:0/1323283457 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fe09c0052a0 0x7fe09c00b980 unknown :-1 s=CLOSED pgs=2563 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T15:12:21.794 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:21.789+0000 7fe0b2bab640 1 -- 172.21.15.43:0/1323283457 >> 172.21.15.43:0/1323283457 conn(0x7fe0ac0afdb0 msgr2=0x7fe0ac0b0190 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-09T15:12:21.794 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:21.789+0000 7fe0b2bab640 1 -- 172.21.15.43:0/1323283457 shutdown_connections 2023-08-09T15:12:21.794 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:21.789+0000 7fe0b2bab640 1 -- 172.21.15.43:0/1323283457 wait complete. 2023-08-09T15:12:21.794 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:21.791+0000 7fe0b2bab640 1 Processor -- start 2023-08-09T15:12:21.794 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:21.791+0000 7fe0b2bab640 1 -- start start 2023-08-09T15:12:21.795 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:21.792+0000 7fe0b2bab640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fe09c0052a0 0x7fe0ac15f7f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T15:12:21.796 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:21.792+0000 7fe0b2bab640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fe0ac1f3260 0x7fe0ac0b2de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T15:12:21.796 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:21.792+0000 7fe0b2bab640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fe0ac0ab860 0x7fe0ac0abce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T15:12:21.796 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:21.792+0000 7fe0b2bab640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe0ac1f20a0 con 0x7fe0ac1f3260 2023-08-09T15:12:21.796 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:21.792+0000 7fe0b2bab640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe0ac0ac220 con 0x7fe09c0052a0 2023-08-09T15:12:21.796 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:21.792+0000 7fe0b2bab640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe0ac0b1280 con 0x7fe0ac0ab860 2023-08-09T15:12:21.796 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:21.793+0000 7fe0abfff640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fe0ac1f3260 0x7fe0ac0b2de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T15:12:21.797 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:21.793+0000 7fe0abfff640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fe0ac1f3260 0x7fe0ac0b2de0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:46288/0 (socket says 172.21.15.43:46288) 2023-08-09T15:12:21.797 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:21.793+0000 7fe0abfff640 1 -- 172.21.15.43:0/795925641 learned_addr learned my addr 172.21.15.43:0/795925641 (peer_addr_for_me v2:172.21.15.43:0/0) 2023-08-09T15:12:21.797 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:21.793+0000 7fe0b0947640 1 --2- 172.21.15.43:0/795925641 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fe09c0052a0 0x7fe0ac15f7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T15:12:21.797 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:21.793+0000 7fe0b1148640 1 --2- 172.21.15.43:0/795925641 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fe0ac0ab860 0x7fe0ac0abce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T15:12:21.797 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:21.793+0000 7fe0abfff640 1 -- 172.21.15.43:0/795925641 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fe0ac0ab860 msgr2=0x7fe0ac0abce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T15:12:21.797 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:21.793+0000 7fe0abfff640 1 --2- 172.21.15.43:0/795925641 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fe0ac0ab860 0x7fe0ac0abce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T15:12:21.797 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:21.793+0000 7fe0abfff640 1 -- 172.21.15.43:0/795925641 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fe09c0052a0 msgr2=0x7fe0ac15f7f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T15:12:21.798 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:21.793+0000 7fe0abfff640 1 --2- 172.21.15.43:0/795925641 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fe09c0052a0 0x7fe0ac15f7f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T15:12:21.798 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:21.793+0000 7fe0abfff640 1 -- 172.21.15.43:0/795925641 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe0ac0b1560 con 0x7fe0ac1f3260 2023-08-09T15:12:21.798 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:21.793+0000 7fe0b0947640 1 --2- 172.21.15.43:0/795925641 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fe09c0052a0 0x7fe0ac15f7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-08-09T15:12:21.798 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:21.794+0000 7fe0abfff640 1 --2- 172.21.15.43:0/795925641 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fe0ac1f3260 0x7fe0ac0b2de0 secure :-1 s=READY pgs=2564 cs=0 l=1 rev1=1 crypto rx=0x7fe098001f00 tx=0x7fe09800d9d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-09T15:12:21.798 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:21.794+0000 7fe0a97fa640 1 -- 172.21.15.43:0/795925641 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe098005480 con 0x7fe0ac1f3260 2023-08-09T15:12:21.798 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:21.794+0000 7fe0a97fa640 1 -- 172.21.15.43:0/795925641 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe098003660 con 0x7fe0ac1f3260 2023-08-09T15:12:21.798 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:21.794+0000 7fe0a97fa640 1 -- 172.21.15.43:0/795925641 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe098003a10 con 0x7fe0ac1f3260 2023-08-09T15:12:21.798 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:21.794+0000 7fe0b2bab640 1 -- 172.21.15.43:0/795925641 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe0ac0b1840 con 0x7fe0ac1f3260 2023-08-09T15:12:21.799 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:21.794+0000 7fe0b2bab640 1 -- 172.21.15.43:0/795925641 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe0ac0b2960 con 0x7fe0ac1f3260 2023-08-09T15:12:21.799 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:21.796+0000 7fe0b2bab640 1 -- 172.21.15.43:0/795925641 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7fe0ac160df0 con 0x7fe0ac1f3260 2023-08-09T15:12:21.800 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:21.797+0000 7fe0a97fa640 1 -- 172.21.15.43:0/795925641 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 129) v1 ==== 88443+0+0 (secure 0 0 0) 0x7fe098003b70 con 0x7fe0ac1f3260 2023-08-09T15:12:21.800 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:21.798+0000 7fe0a97fa640 1 --2- 172.21.15.43:0/795925641 >> [v2:172.21.15.203:6832/1456699667,v1:172.21.15.203:6833/1456699667] conn(0x7fe09006a1a0 0x7fe09006c660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T15:12:21.800 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:21.798+0000 7fe0a97fa640 1 -- 172.21.15.43:0/795925641 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(229..229 src has 1..229) v4 ==== 6462+0+0 (secure 0 0 0) 0x7fe0980888d0 con 0x7fe0ac1f3260 2023-08-09T15:12:21.800 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:21.798+0000 7fe0a97fa640 1 --2- 172.21.15.43:0/795925641 >> [v2:172.21.15.43:6800/502667335,v1:172.21.15.43:6801/502667335] conn(0x7fe090071180 0x7fe0900735e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T15:12:21.800 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:21.798+0000 7fe0a97fa640 1 -- 172.21.15.43:0/795925641 --> [v2:172.21.15.43:6800/502667335,v1:172.21.15.43:6801/502667335] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7fe090073cb0 con 0x7fe090071180 2023-08-09T15:12:21.800 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:21.798+0000 7fe0a97fa640 1 -- 172.21.15.43:0/795925641 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_get_version_reply(handle=1 version=229) v2 ==== 24+0+0 (secure 0 0 0) 0x7fe09808d2a0 con 0x7fe0ac1f3260 2023-08-09T15:12:21.800 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:21.798+0000 7fe0b1148640 1 --2- 172.21.15.43:0/795925641 >> [v2:172.21.15.43:6800/502667335,v1:172.21.15.43:6801/502667335] conn(0x7fe090071180 0x7fe0900735e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T15:12:21.801 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:21.798+0000 7fe0b0947640 1 --2- 172.21.15.43:0/795925641 >> [v2:172.21.15.203:6832/1456699667,v1:172.21.15.203:6833/1456699667] conn(0x7fe09006a1a0 0x7fe09006c660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T15:12:21.801 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:21.799+0000 7fe0b1148640 1 --2- 172.21.15.43:0/795925641 >> [v2:172.21.15.43:6800/502667335,v1:172.21.15.43:6801/502667335] conn(0x7fe090071180 0x7fe0900735e0 crc :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-09T15:12:21.801 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:21.800+0000 7fe0b0947640 1 --2- 172.21.15.43:0/795925641 >> [v2:172.21.15.203:6832/1456699667,v1:172.21.15.203:6833/1456699667] conn(0x7fe09006a1a0 0x7fe09006c660 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7fe0a0001200 tx=0x7fe0a0009040 comp rx=0 tx=0).ready entity=mgr.9232 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-09T15:12:21.802 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:21.800+0000 7fe0a97fa640 1 -- 172.21.15.43:0/795925641 <== osd.0 v2:172.21.15.43:6800/502667335 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+25607 (crc 0 0 0) 0x7fe090073cb0 con 0x7fe090071180 2023-08-09T15:12:21.816 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:21.815+0000 7fe0b2bab640 1 -- 172.21.15.43:0/795925641 --> [v2:172.21.15.43:6800/502667335,v1:172.21.15.43:6801/502667335] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) v1 -- 0x7fe0ac160b10 con 0x7fe090071180 2023-08-09T15:12:21.816 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:21.815+0000 7fe0a97fa640 1 -- 172.21.15.43:0/795925641 <== osd.0 v2:172.21.15.43:6800/502667335 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+105 (crc 0 0 0) 0x7fe0a402cc80 con 0x7fe090071180 2023-08-09T15:12:21.817 INFO:teuthology.orchestra.run.smithi043.stdout:{ 2023-08-09T15:12:21.817 INFO:teuthology.orchestra.run.smithi043.stdout: "success": "osd_debug_deep_scrub_sleep = '0.000000' (not observed, change may require restart) " 2023-08-09T15:12:21.817 INFO:teuthology.orchestra.run.smithi043.stdout:} 2023-08-09T15:12:21.819 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:21.815+0000 7fe0b2bab640 1 -- 172.21.15.43:0/795925641 >> [v2:172.21.15.43:6800/502667335,v1:172.21.15.43:6801/502667335] conn(0x7fe090071180 msgr2=0x7fe0900735e0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T15:12:21.819 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:21.815+0000 7fe0b2bab640 1 --2- 172.21.15.43:0/795925641 >> [v2:172.21.15.43:6800/502667335,v1:172.21.15.43:6801/502667335] conn(0x7fe090071180 0x7fe0900735e0 crc :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T15:12:21.819 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:21.816+0000 7fe0b2bab640 1 -- 172.21.15.43:0/795925641 >> [v2:172.21.15.203:6832/1456699667,v1:172.21.15.203:6833/1456699667] conn(0x7fe09006a1a0 msgr2=0x7fe09006c660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T15:12:21.820 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:21.816+0000 7fe0b2bab640 1 --2- 172.21.15.43:0/795925641 >> [v2:172.21.15.203:6832/1456699667,v1:172.21.15.203:6833/1456699667] conn(0x7fe09006a1a0 0x7fe09006c660 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7fe0a0001200 tx=0x7fe0a0009040 comp rx=0 tx=0).stop 2023-08-09T15:12:21.820 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:21.816+0000 7fe0b2bab640 1 -- 172.21.15.43:0/795925641 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fe0ac1f3260 msgr2=0x7fe0ac0b2de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T15:12:21.820 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:21.816+0000 7fe0b2bab640 1 --2- 172.21.15.43:0/795925641 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fe0ac1f3260 0x7fe0ac0b2de0 secure :-1 s=READY pgs=2564 cs=0 l=1 rev1=1 crypto rx=0x7fe098001f00 tx=0x7fe09800d9d0 comp rx=0 tx=0).stop 2023-08-09T15:12:21.820 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:21.816+0000 7fe0b1148640 1 -- 172.21.15.43:0/795925641 reap_dead start 2023-08-09T15:12:21.820 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:21.816+0000 7fe0b2bab640 1 -- 172.21.15.43:0/795925641 shutdown_connections 2023-08-09T15:12:21.820 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:21.816+0000 7fe0b2bab640 1 -- 172.21.15.43:0/795925641 >> 172.21.15.43:0/795925641 conn(0x7fe0ac0afdb0 msgr2=0x7fe0ac0ce200 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-09T15:12:21.820 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:21.816+0000 7fe0b2bab640 1 -- 172.21.15.43:0/795925641 shutdown_connections 2023-08-09T15:12:21.821 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:21.816+0000 7fe0b2bab640 1 -- 172.21.15.43:0/795925641 wait complete. 2023-08-09T15:12:21.838 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 0 2023-08-09T15:12:22.066 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:22.065+0000 7f44fdce0640 1 Processor -- start 2023-08-09T15:12:22.066 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:22.065+0000 7f44fdce0640 1 -- start start 2023-08-09T15:12:22.068 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:22.066+0000 7f44fdce0640 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7f44f8159600 0x7f44f80cd590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T15:12:22.068 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:22.067+0000 7f44fdce0640 1 -- --> v1:172.21.15.43:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f44f815df30 con 0x7f44f8159220 2023-08-09T15:12:22.068 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:22.067+0000 7f44fdce0640 1 -- --> v1:172.21.15.203:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f44f80ad2a0 con 0x7f44f8070ec0 2023-08-09T15:12:22.068 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:22.067+0000 7f44fdce0640 1 -- --> v2:172.21.15.203:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f44f815d9b0 con 0x7f44f8159600 2023-08-09T15:12:22.069 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:22.067+0000 7f44fccde640 1 --1- >> v1:172.21.15.43:6789/0 conn(0x7f44f8159220 0x7f44f80707b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.43:6789/0 says I am v1:172.21.15.43:32816/0 (socket says 172.21.15.43:32816) 2023-08-09T15:12:22.069 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:22.067+0000 7f44fccde640 1 -- 172.21.15.43:0/1016119482 learned_addr learned my addr 172.21.15.43:0/1016119482 (peer_addr_for_me v1:172.21.15.43:0/0) 2023-08-09T15:12:22.071 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:22.067+0000 7f44fd4df640 1 --2- 172.21.15.43:0/1016119482 >> v2:172.21.15.203:3300/0 conn(0x7f44f8159600 0x7f44f80cd590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T15:12:22.071 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:22.067+0000 7f44f77fe640 1 -- 172.21.15.43:0/1016119482 <== mon.0 v1:172.21.15.43:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2448891576 0 0) 0x7f44f815df30 con 0x7f44f8159220 2023-08-09T15:12:22.071 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:22.067+0000 7f44f77fe640 1 -- 172.21.15.43:0/1016119482 --> v1:172.21.15.43:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f44e8003520 con 0x7f44f8159220 2023-08-09T15:12:22.071 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:22.068+0000 7f44f77fe640 1 -- 172.21.15.43:0/1016119482 <== mon.0 v1:172.21.15.43:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3530877683 0 0) 0x7f44e8003520 con 0x7f44f8159220 2023-08-09T15:12:22.071 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:22.068+0000 7f44f77fe640 1 -- 172.21.15.43:0/1016119482 >> v2:172.21.15.203:3300/0 conn(0x7f44f8159600 msgr2=0x7f44f80cd590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-09T15:12:22.072 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:22.068+0000 7f44f77fe640 1 --2- 172.21.15.43:0/1016119482 >> v2:172.21.15.203:3300/0 conn(0x7f44f8159600 0x7f44f80cd590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T15:12:22.072 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:22.068+0000 7f44f77fe640 1 -- 172.21.15.43:0/1016119482 >> v1:172.21.15.203:6789/0 conn(0x7f44f8070ec0 legacy=0x7f44f806c270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T15:12:22.072 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:22.068+0000 7f44f77fe640 1 -- 172.21.15.43:0/1016119482 --> v1:172.21.15.43:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f44f815dc50 con 0x7f44f8159220 2023-08-09T15:12:22.072 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:22.068+0000 7f44f77fe640 1 -- 172.21.15.43:0/1016119482 <== mon.0 v1:172.21.15.43:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1285150740 0 0) 0x7f44ec001670 con 0x7f44f8159220 2023-08-09T15:12:22.072 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:22.068+0000 7f44f77fe640 1 -- 172.21.15.43:0/1016119482 >> v1:172.21.15.43:6789/0 conn(0x7f44f8159220 legacy=0x7f44f80707b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T15:12:22.072 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:22.068+0000 7f44f77fe640 1 --2- 172.21.15.43:0/1016119482 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f44e8003970 0x7f44e8003d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T15:12:22.072 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:22.068+0000 7f44f77fe640 1 --2- 172.21.15.43:0/1016119482 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f44e80047c0 0x7f44e8004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T15:12:22.073 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:22.068+0000 7f44f77fe640 1 --2- 172.21.15.43:0/1016119482 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f44e80052a0 0x7f44e800b980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T15:12:22.073 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:22.068+0000 7f44f77fe640 1 -- 172.21.15.43:0/1016119482 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f44f815d9b0 con 0x7f44e80047c0 2023-08-09T15:12:22.073 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:22.068+0000 7f44f77fe640 1 -- 172.21.15.43:0/1016119482 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f44e800c170 con 0x7f44e8003970 2023-08-09T15:12:22.073 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:22.068+0000 7f44f77fe640 1 -- 172.21.15.43:0/1016119482 --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f44e800c330 con 0x7f44e80052a0 2023-08-09T15:12:22.073 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:22.068+0000 7f44f7fff640 1 --2- 172.21.15.43:0/1016119482 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f44e80047c0 0x7f44e8004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T15:12:22.073 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:22.069+0000 7f44fccde640 1 --2- 172.21.15.43:0/1016119482 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f44e8003970 0x7f44e8003d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T15:12:22.073 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:22.069+0000 7f44fd4df640 1 --2- 172.21.15.43:0/1016119482 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f44e80052a0 0x7f44e800b980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T15:12:22.074 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:22.069+0000 7f44f7fff640 1 -- 172.21.15.43:0/1016119482 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f44e80052a0 msgr2=0x7f44e800b980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-09T15:12:22.074 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:22.069+0000 7f44f7fff640 1 --2- 172.21.15.43:0/1016119482 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f44e80052a0 0x7f44e800b980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T15:12:22.074 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:22.069+0000 7f44f7fff640 1 -- 172.21.15.43:0/1016119482 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f44e8003970 msgr2=0x7f44e8003d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-09T15:12:22.074 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:22.069+0000 7f44f7fff640 1 --2- 172.21.15.43:0/1016119482 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f44e8003970 0x7f44e8003d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T15:12:22.074 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:22.069+0000 7f44f7fff640 1 -- 172.21.15.43:0/1016119482 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f44e8003520 con 0x7f44e80047c0 2023-08-09T15:12:22.074 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:22.069+0000 7f44fd4df640 1 -- 172.21.15.43:0/1016119482 reap_dead start 2023-08-09T15:12:22.074 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:22.069+0000 7f44fccde640 1 --2- 172.21.15.43:0/1016119482 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f44e8003970 0x7f44e8003d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-08-09T15:12:22.077 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:22.070+0000 7f44f7fff640 1 --2- 172.21.15.43:0/1016119482 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f44e80047c0 0x7f44e8004c40 secure :-1 s=READY pgs=2566 cs=0 l=1 rev1=1 crypto rx=0x7f44f806f4b0 tx=0x7f44e80057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cafa8b3461ab7b5f server_cookie=0 in_seq=0 out_seq=0 2023-08-09T15:12:22.077 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:22.070+0000 7f44f77fe640 1 -- 172.21.15.43:0/1016119482 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f44e40052d0 con 0x7f44e80047c0 2023-08-09T15:12:22.077 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:22.071+0000 7f44f77fe640 1 -- 172.21.15.43:0/1016119482 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f44e4030070 con 0x7f44e80047c0 2023-08-09T15:12:22.077 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:22.071+0000 7f44f77fe640 1 -- 172.21.15.43:0/1016119482 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f44e4003830 con 0x7f44e80047c0 2023-08-09T15:12:22.078 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:22.071+0000 7f44fdce0640 1 -- 172.21.15.43:0/1016119482 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f44e80047c0 msgr2=0x7f44e8004c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T15:12:22.078 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:22.071+0000 7f44fdce0640 1 --2- 172.21.15.43:0/1016119482 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f44e80047c0 0x7f44e8004c40 secure :-1 s=READY pgs=2566 cs=0 l=1 rev1=1 crypto rx=0x7f44f806f4b0 tx=0x7f44e80057f0 comp rx=0 tx=0).stop 2023-08-09T15:12:22.078 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:22.071+0000 7f44fdce0640 1 -- 172.21.15.43:0/1016119482 shutdown_connections 2023-08-09T15:12:22.078 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:22.071+0000 7f44fdce0640 1 --2- 172.21.15.43:0/1016119482 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f44e80047c0 0x7f44e8004c40 unknown :-1 s=CLOSED pgs=2566 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T15:12:22.078 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:22.071+0000 7f44fdce0640 1 -- 172.21.15.43:0/1016119482 >> 172.21.15.43:0/1016119482 conn(0x7f44f80ce020 msgr2=0x7f44f80ce430 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-09T15:12:22.078 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:22.071+0000 7f44fdce0640 1 -- 172.21.15.43:0/1016119482 shutdown_connections 2023-08-09T15:12:22.078 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:22.071+0000 7f44fdce0640 1 -- 172.21.15.43:0/1016119482 wait complete. 2023-08-09T15:12:22.078 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:22.073+0000 7f44fdce0640 1 Processor -- start 2023-08-09T15:12:22.079 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:22.073+0000 7f44fdce0640 1 -- start start 2023-08-09T15:12:22.079 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:22.074+0000 7f44fdce0640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f44e80047c0 0x7f44f81557c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T15:12:22.079 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:22.074+0000 7f44fdce0640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f44f80b2790 0x7f44f80b2c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T15:12:22.079 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:22.074+0000 7f44fdce0640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f44f80990d0 0x7f44f8099550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T15:12:22.079 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:22.074+0000 7f44fdce0640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f44f8099a90 con 0x7f44f80b2790 2023-08-09T15:12:22.079 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:22.074+0000 7f44fdce0640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f44f80c2bd0 con 0x7f44f80990d0 2023-08-09T15:12:22.079 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:22.074+0000 7f44fdce0640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f44f80b3270 con 0x7f44e80047c0 2023-08-09T15:12:22.080 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:22.075+0000 7f44f7fff640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f44f80b2790 0x7f44f80b2c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T15:12:22.080 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:22.075+0000 7f44f7fff640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f44f80b2790 0x7f44f80b2c10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:46304/0 (socket says 172.21.15.43:46304) 2023-08-09T15:12:22.080 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:22.075+0000 7f44f7fff640 1 -- 172.21.15.43:0/62721658 learned_addr learned my addr 172.21.15.43:0/62721658 (peer_addr_for_me v2:172.21.15.43:0/0) 2023-08-09T15:12:22.080 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:22.075+0000 7f44fccde640 1 --2- 172.21.15.43:0/62721658 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f44e80047c0 0x7f44f81557c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T15:12:22.080 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:22.075+0000 7f44fd4df640 1 --2- 172.21.15.43:0/62721658 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f44f80990d0 0x7f44f8099550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T15:12:22.080 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:22.075+0000 7f44f7fff640 1 -- 172.21.15.43:0/62721658 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f44e80047c0 msgr2=0x7f44f81557c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T15:12:22.080 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:22.075+0000 7f44f7fff640 1 --2- 172.21.15.43:0/62721658 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f44e80047c0 0x7f44f81557c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T15:12:22.081 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:22.075+0000 7f44f7fff640 1 -- 172.21.15.43:0/62721658 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f44f80990d0 msgr2=0x7f44f8099550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T15:12:22.081 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:22.075+0000 7f44f7fff640 1 --2- 172.21.15.43:0/62721658 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f44f80990d0 0x7f44f8099550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T15:12:22.081 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:22.075+0000 7f44f7fff640 1 -- 172.21.15.43:0/62721658 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f44f80b3550 con 0x7f44f80b2790 2023-08-09T15:12:22.082 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:22.075+0000 7f44f7fff640 1 --2- 172.21.15.43:0/62721658 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f44f80b2790 0x7f44f80b2c10 secure :-1 s=READY pgs=2567 cs=0 l=1 rev1=1 crypto rx=0x7f44e4001810 tx=0x7f44e4001120 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-09T15:12:22.082 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:22.076+0000 7f44f57fa640 1 -- 172.21.15.43:0/62721658 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f44e40069d0 con 0x7f44f80b2790 2023-08-09T15:12:22.082 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:22.076+0000 7f44f57fa640 1 -- 172.21.15.43:0/62721658 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f44e4006b50 con 0x7f44f80b2790 2023-08-09T15:12:22.082 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:22.076+0000 7f44f57fa640 1 -- 172.21.15.43:0/62721658 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f44e400f1d0 con 0x7f44f80b2790 2023-08-09T15:12:22.082 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:22.076+0000 7f44fdce0640 1 -- 172.21.15.43:0/62721658 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f44f80c2e40 con 0x7f44f80b2790 2023-08-09T15:12:22.083 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:22.076+0000 7f44fdce0640 1 -- 172.21.15.43:0/62721658 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f44f81f3030 con 0x7f44f80b2790 2023-08-09T15:12:22.083 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:22.077+0000 7f44fdce0640 1 -- 172.21.15.43:0/62721658 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f44f809bd40 con 0x7f44f80b2790 2023-08-09T15:12:22.083 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:22.078+0000 7f44f57fa640 1 -- 172.21.15.43:0/62721658 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 129) v1 ==== 88443+0+0 (secure 0 0 0) 0x7f44e40035f0 con 0x7f44f80b2790 2023-08-09T15:12:22.084 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:22.079+0000 7f44f57fa640 1 --2- 172.21.15.43:0/62721658 >> [v2:172.21.15.203:6832/1456699667,v1:172.21.15.203:6833/1456699667] conn(0x7f44dc06a1a0 0x7f44dc06c660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T15:12:22.084 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:22.080+0000 7f44f57fa640 1 -- 172.21.15.43:0/62721658 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(229..229 src has 1..229) v4 ==== 6462+0+0 (secure 0 0 0) 0x7f44e40a9aa0 con 0x7f44f80b2790 2023-08-09T15:12:22.084 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:22.080+0000 7f44fccde640 1 --2- 172.21.15.43:0/62721658 >> [v2:172.21.15.203:6832/1456699667,v1:172.21.15.203:6833/1456699667] conn(0x7f44dc06a1a0 0x7f44dc06c660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T15:12:22.084 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:22.081+0000 7f44fccde640 1 --2- 172.21.15.43:0/62721658 >> [v2:172.21.15.203:6832/1456699667,v1:172.21.15.203:6833/1456699667] conn(0x7f44dc06a1a0 0x7f44dc06c660 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7f44ec0010e0 tx=0x7f44ec009040 comp rx=0 tx=0).ready entity=mgr.9232 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-09T15:12:22.085 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:22.084+0000 7f44f57fa640 1 -- 172.21.15.43:0/62721658 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178066 (secure 0 0 0) 0x7f44e4074f10 con 0x7f44f80b2790 2023-08-09T15:12:22.284 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:22.282+0000 7f44fdce0640 1 -- 172.21.15.43:0/62721658 --> [v2:172.21.15.203:6832/1456699667,v1:172.21.15.203:6833/1456699667] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "0", "target": ["mon-mgr", ""]}) v1 -- 0x7f44f80cc000 con 0x7f44dc06a1a0 2023-08-09T15:12:22.284 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:22.283+0000 7f44f57fa640 1 -- 172.21.15.43:0/62721658 <== mgr.9232 v2:172.21.15.203:6832/1456699667 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 0 to deep-scrub) v1 ==== 41+0+0 (secure 0 0 0) 0x7f44ec0107e0 con 0x7f44dc06a1a0 2023-08-09T15:12:22.285 INFO:teuthology.orchestra.run.smithi043.stderr:instructed osd(s) 0 to deep-scrub 2023-08-09T15:12:22.287 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:22.285+0000 7f44fdce0640 1 -- 172.21.15.43:0/62721658 >> [v2:172.21.15.203:6832/1456699667,v1:172.21.15.203:6833/1456699667] conn(0x7f44dc06a1a0 msgr2=0x7f44dc06c660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T15:12:22.287 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:22.285+0000 7f44fdce0640 1 --2- 172.21.15.43:0/62721658 >> [v2:172.21.15.203:6832/1456699667,v1:172.21.15.203:6833/1456699667] conn(0x7f44dc06a1a0 0x7f44dc06c660 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7f44ec0010e0 tx=0x7f44ec009040 comp rx=0 tx=0).stop 2023-08-09T15:12:22.287 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:22.285+0000 7f44fdce0640 1 -- 172.21.15.43:0/62721658 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f44f80b2790 msgr2=0x7f44f80b2c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T15:12:22.287 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:22.285+0000 7f44fdce0640 1 --2- 172.21.15.43:0/62721658 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f44f80b2790 0x7f44f80b2c10 secure :-1 s=READY pgs=2567 cs=0 l=1 rev1=1 crypto rx=0x7f44e4001810 tx=0x7f44e4001120 comp rx=0 tx=0).stop 2023-08-09T15:12:22.288 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:22.286+0000 7f44fdce0640 1 -- 172.21.15.43:0/62721658 shutdown_connections 2023-08-09T15:12:22.288 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:22.286+0000 7f44fdce0640 1 --2- 172.21.15.43:0/62721658 >> [v2:172.21.15.203:6832/1456699667,v1:172.21.15.203:6833/1456699667] conn(0x7f44dc06a1a0 0x7f44dc06c660 unknown :-1 s=CLOSED pgs=44 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T15:12:22.288 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:22.286+0000 7f44fdce0640 1 --2- 172.21.15.43:0/62721658 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f44f80990d0 0x7f44f8099550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T15:12:22.288 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:22.286+0000 7f44fdce0640 1 --2- 172.21.15.43:0/62721658 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f44f80b2790 0x7f44f80b2c10 unknown :-1 s=CLOSED pgs=2567 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T15:12:22.288 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:22.286+0000 7f44fdce0640 1 --2- 172.21.15.43:0/62721658 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f44e80047c0 0x7f44f81557c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T15:12:22.289 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:22.286+0000 7f44fdce0640 1 -- 172.21.15.43:0/62721658 >> 172.21.15.43:0/62721658 conn(0x7f44f80ce020 msgr2=0x7f44f80c8c10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-09T15:12:22.289 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:22.286+0000 7f44fdce0640 1 -- 172.21.15.43:0/62721658 shutdown_connections 2023-08-09T15:12:22.289 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:22.286+0000 7f44fdce0640 1 -- 172.21.15.43:0/62721658 wait complete. 2023-08-09T15:12:22.307 INFO:tasks.ceph:Scrubbing osd.1 2023-08-09T15:12:22.308 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.1 config set osd_debug_deep_scrub_sleep 0 2023-08-09T15:12:22.532 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:22.530+0000 7fcaee99f640 1 Processor -- start 2023-08-09T15:12:22.532 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:22.531+0000 7fcaee99f640 1 -- start start 2023-08-09T15:12:22.534 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:22.532+0000 7fcaee99f640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7fcae81430b0 0x7fcae80c8640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T15:12:22.534 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:22.532+0000 7fcaee99f640 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7fcae80c8b80 0x7fcae80c3250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T15:12:22.535 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:22.532+0000 7fcaee99f640 1 -- --> v1:172.21.15.43:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fcae80c8fe0 con 0x7fcae80c3790 2023-08-09T15:12:22.535 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:22.532+0000 7fcaee99f640 1 -- --> v2:172.21.15.43:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fcae80c4820 con 0x7fcae81430b0 2023-08-09T15:12:22.536 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:22.532+0000 7fcaee99f640 1 -- --> v2:172.21.15.203:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fcae806ba70 con 0x7fcae80c8b80 2023-08-09T15:12:22.536 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:22.532+0000 7fcaed99d640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7fcae81430b0 0x7fcae80c8640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T15:12:22.536 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:22.532+0000 7fcaee19e640 1 --1- >> v1:172.21.15.43:6789/0 conn(0x7fcae80c3790 0x7fcae806be90 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.43:6789/0 says I am v1:172.21.15.43:32832/0 (socket says 172.21.15.43:32832) 2023-08-09T15:12:22.536 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:22.532+0000 7fcaee19e640 1 -- 172.21.15.43:0/2782709769 learned_addr learned my addr 172.21.15.43:0/2782709769 (peer_addr_for_me v1:172.21.15.43:0/0) 2023-08-09T15:12:22.537 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:22.533+0000 7fcaed19c640 1 --2- 172.21.15.43:0/2782709769 >> v2:172.21.15.203:3300/0 conn(0x7fcae80c8b80 0x7fcae80c3250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T15:12:22.537 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:22.533+0000 7fcaed99d640 1 -- 172.21.15.43:0/2782709769 >> v2:172.21.15.203:3300/0 conn(0x7fcae80c8b80 msgr2=0x7fcae80c3250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-09T15:12:22.537 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:22.533+0000 7fcaed99d640 1 --2- 172.21.15.43:0/2782709769 >> v2:172.21.15.203:3300/0 conn(0x7fcae80c8b80 0x7fcae80c3250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T15:12:22.537 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:22.533+0000 7fcaed99d640 1 -- 172.21.15.43:0/2782709769 >> v1:172.21.15.43:6789/0 conn(0x7fcae80c3790 legacy=0x7fcae806be90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T15:12:22.538 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:22.533+0000 7fcaed99d640 1 -- 172.21.15.43:0/2782709769 --> v2:172.21.15.43:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcae8143490 con 0x7fcae81430b0 2023-08-09T15:12:22.539 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:22.534+0000 7fcaed99d640 1 --2- 172.21.15.43:0/2782709769 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fcae81430b0 0x7fcae80c8640 secure :-1 s=READY pgs=2569 cs=0 l=1 rev1=1 crypto rx=0x7fcae806d9c0 tx=0x7fcae402b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a012f18bf716e635 server_cookie=0 in_seq=0 out_seq=0 2023-08-09T15:12:22.539 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:22.534+0000 7fcaec99b640 1 -- 172.21.15.43:0/2782709769 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcae4031830 con 0x7fcae81430b0 2023-08-09T15:12:22.539 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:22.534+0000 7fcaec99b640 1 -- 172.21.15.43:0/2782709769 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcae4031990 con 0x7fcae81430b0 2023-08-09T15:12:22.540 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:22.534+0000 7fcaec99b640 1 -- 172.21.15.43:0/2782709769 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcae4031d20 con 0x7fcae81430b0 2023-08-09T15:12:22.540 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:22.535+0000 7fcaee99f640 1 -- 172.21.15.43:0/2782709769 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fcae81430b0 msgr2=0x7fcae80c8640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T15:12:22.540 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:22.535+0000 7fcaee99f640 1 --2- 172.21.15.43:0/2782709769 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fcae81430b0 0x7fcae80c8640 secure :-1 s=READY pgs=2569 cs=0 l=1 rev1=1 crypto rx=0x7fcae806d9c0 tx=0x7fcae402b8d0 comp rx=0 tx=0).stop 2023-08-09T15:12:22.540 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:22.535+0000 7fcaee99f640 1 -- 172.21.15.43:0/2782709769 shutdown_connections 2023-08-09T15:12:22.540 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:22.535+0000 7fcaee99f640 1 --2- 172.21.15.43:0/2782709769 >> v2:172.21.15.203:3300/0 conn(0x7fcae80c8b80 0x7fcae80c3250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T15:12:22.540 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:22.535+0000 7fcaee99f640 1 --2- 172.21.15.43:0/2782709769 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fcae81430b0 0x7fcae80c8640 unknown :-1 s=CLOSED pgs=2569 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T15:12:22.541 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:22.535+0000 7fcaee99f640 1 -- 172.21.15.43:0/2782709769 >> 172.21.15.43:0/2782709769 conn(0x7fcae80cdff0 msgr2=0x7fcae80ce3d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-09T15:12:22.541 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:22.535+0000 7fcaee99f640 1 -- 172.21.15.43:0/2782709769 shutdown_connections 2023-08-09T15:12:22.541 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:22.535+0000 7fcaee99f640 1 -- 172.21.15.43:0/2782709769 wait complete. 2023-08-09T15:12:22.541 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:22.537+0000 7fcaee99f640 1 Processor -- start 2023-08-09T15:12:22.541 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:22.537+0000 7fcaee99f640 1 -- start start 2023-08-09T15:12:22.543 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:22.538+0000 7fcaee99f640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fcae80c3790 0x7fcae8159cd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T15:12:22.543 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:22.538+0000 7fcaee99f640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fcae80c8b80 0x7fcae80cd1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T15:12:22.544 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:22.538+0000 7fcaee99f640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fcae81430b0 0x7fcae80cd6e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T15:12:22.544 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:22.538+0000 7fcaee99f640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcae815a210 con 0x7fcae81430b0 2023-08-09T15:12:22.544 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:22.538+0000 7fcaee99f640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcae80b2b90 con 0x7fcae80c8b80 2023-08-09T15:12:22.544 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:22.538+0000 7fcaee99f640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcae80ab8f0 con 0x7fcae80c3790 2023-08-09T15:12:22.544 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:22.539+0000 7fcaee19e640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fcae81430b0 0x7fcae80cd6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T15:12:22.545 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:22.539+0000 7fcaee19e640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fcae81430b0 0x7fcae80cd6e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:46314/0 (socket says 172.21.15.43:46314) 2023-08-09T15:12:22.545 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:22.539+0000 7fcaee19e640 1 -- 172.21.15.43:0/2724840955 learned_addr learned my addr 172.21.15.43:0/2724840955 (peer_addr_for_me v2:172.21.15.43:0/0) 2023-08-09T15:12:22.545 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:22.539+0000 7fcaed99d640 1 --2- 172.21.15.43:0/2724840955 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fcae80c3790 0x7fcae8159cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T15:12:22.545 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:22.539+0000 7fcaee19e640 1 -- 172.21.15.43:0/2724840955 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fcae80c3790 msgr2=0x7fcae8159cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T15:12:22.546 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:22.539+0000 7fcaee19e640 1 --2- 172.21.15.43:0/2724840955 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fcae80c3790 0x7fcae8159cd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T15:12:22.546 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:22.539+0000 7fcaee19e640 1 -- 172.21.15.43:0/2724840955 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fcae80c8b80 msgr2=0x7fcae80cd1a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T15:12:22.546 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:22.539+0000 7fcaed19c640 1 --2- 172.21.15.43:0/2724840955 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fcae80c8b80 0x7fcae80cd1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T15:12:22.546 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:22.539+0000 7fcaee19e640 1 --2- 172.21.15.43:0/2724840955 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fcae80c8b80 0x7fcae80cd1a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T15:12:22.547 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:22.539+0000 7fcaee19e640 1 -- 172.21.15.43:0/2724840955 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcae4009400 con 0x7fcae81430b0 2023-08-09T15:12:22.547 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:22.539+0000 7fcaee19e640 1 --2- 172.21.15.43:0/2724840955 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fcae81430b0 0x7fcae80cd6e0 secure :-1 s=READY pgs=2570 cs=0 l=1 rev1=1 crypto rx=0x7fcae000cbc0 tx=0x7fcae000a430 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-09T15:12:22.547 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:22.540+0000 7fcade7fc640 1 -- 172.21.15.43:0/2724840955 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcae000ad40 con 0x7fcae81430b0 2023-08-09T15:12:22.548 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:22.540+0000 7fcade7fc640 1 -- 172.21.15.43:0/2724840955 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcae0012400 con 0x7fcae81430b0 2023-08-09T15:12:22.548 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:22.540+0000 7fcade7fc640 1 -- 172.21.15.43:0/2724840955 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcae000aea0 con 0x7fcae81430b0 2023-08-09T15:12:22.548 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:22.540+0000 7fcaee99f640 1 -- 172.21.15.43:0/2724840955 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcae81e3c70 con 0x7fcae81430b0 2023-08-09T15:12:22.548 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:22.540+0000 7fcaee99f640 1 -- 172.21.15.43:0/2724840955 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcae81e4230 con 0x7fcae81430b0 2023-08-09T15:12:22.549 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:22.541+0000 7fcaee99f640 1 -- 172.21.15.43:0/2724840955 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7fcae80c4260 con 0x7fcae81430b0 2023-08-09T15:12:22.549 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:22.542+0000 7fcade7fc640 1 -- 172.21.15.43:0/2724840955 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 129) v1 ==== 88443+0+0 (secure 0 0 0) 0x7fcae0007070 con 0x7fcae81430b0 2023-08-09T15:12:22.551 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:22.543+0000 7fcade7fc640 1 --2- 172.21.15.43:0/2724840955 >> [v2:172.21.15.203:6832/1456699667,v1:172.21.15.203:6833/1456699667] conn(0x7fcacc06a170 0x7fcacc06c630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T15:12:22.551 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:22.543+0000 7fcade7fc640 1 -- 172.21.15.43:0/2724840955 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(229..229 src has 1..229) v4 ==== 6462+0+0 (secure 0 0 0) 0x7fcae00887a0 con 0x7fcae81430b0 2023-08-09T15:12:22.552 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:22.543+0000 7fcade7fc640 1 --2- 172.21.15.43:0/2724840955 >> [v2:172.21.15.43:6808/3289134610,v1:172.21.15.43:6809/3289134610] conn(0x7fcacc071150 0x7fcacc0735b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T15:12:22.552 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:22.543+0000 7fcade7fc640 1 -- 172.21.15.43:0/2724840955 --> [v2:172.21.15.43:6808/3289134610,v1:172.21.15.43:6809/3289134610] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7fcacc073c80 con 0x7fcacc071150 2023-08-09T15:12:22.552 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:22.543+0000 7fcade7fc640 1 -- 172.21.15.43:0/2724840955 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_get_version_reply(handle=1 version=229) v2 ==== 24+0+0 (secure 0 0 0) 0x7fcae008ad80 con 0x7fcae81430b0 2023-08-09T15:12:22.553 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:22.544+0000 7fcaed19c640 1 --2- 172.21.15.43:0/2724840955 >> [v2:172.21.15.43:6808/3289134610,v1:172.21.15.43:6809/3289134610] conn(0x7fcacc071150 0x7fcacc0735b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T15:12:22.553 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:22.544+0000 7fcaed99d640 1 --2- 172.21.15.43:0/2724840955 >> [v2:172.21.15.203:6832/1456699667,v1:172.21.15.203:6833/1456699667] conn(0x7fcacc06a170 0x7fcacc06c630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T15:12:22.553 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:22.544+0000 7fcaed19c640 1 --2- 172.21.15.43:0/2724840955 >> [v2:172.21.15.43:6808/3289134610,v1:172.21.15.43:6809/3289134610] conn(0x7fcacc071150 0x7fcacc0735b0 crc :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-09T15:12:22.554 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:22.545+0000 7fcaed99d640 1 --2- 172.21.15.43:0/2724840955 >> [v2:172.21.15.203:6832/1456699667,v1:172.21.15.203:6833/1456699667] conn(0x7fcacc06a170 0x7fcacc06c630 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7fcae4036000 tx=0x7fcae403c040 comp rx=0 tx=0).ready entity=mgr.9232 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-09T15:12:22.554 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:22.545+0000 7fcade7fc640 1 -- 172.21.15.43:0/2724840955 <== osd.1 v2:172.21.15.43:6808/3289134610 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+25607 (crc 0 0 0) 0x7fcacc073c80 con 0x7fcacc071150 2023-08-09T15:12:22.561 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:22.560+0000 7fcaee99f640 1 -- 172.21.15.43:0/2724840955 --> [v2:172.21.15.43:6808/3289134610,v1:172.21.15.43:6809/3289134610] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) v1 -- 0x7fcae80c3f80 con 0x7fcacc071150 2023-08-09T15:12:22.562 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:22.560+0000 7fcade7fc640 1 -- 172.21.15.43:0/2724840955 <== osd.1 v2:172.21.15.43:6808/3289134610 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+105 (crc 0 0 0) 0x7fcad8009480 con 0x7fcacc071150 2023-08-09T15:12:22.562 INFO:teuthology.orchestra.run.smithi043.stdout:{ 2023-08-09T15:12:22.562 INFO:teuthology.orchestra.run.smithi043.stdout: "success": "osd_debug_deep_scrub_sleep = '0.000000' (not observed, change may require restart) " 2023-08-09T15:12:22.562 INFO:teuthology.orchestra.run.smithi043.stdout:} 2023-08-09T15:12:22.563 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:22.561+0000 7fcaee99f640 1 -- 172.21.15.43:0/2724840955 >> [v2:172.21.15.43:6808/3289134610,v1:172.21.15.43:6809/3289134610] conn(0x7fcacc071150 msgr2=0x7fcacc0735b0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T15:12:22.563 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:22.561+0000 7fcaee99f640 1 --2- 172.21.15.43:0/2724840955 >> [v2:172.21.15.43:6808/3289134610,v1:172.21.15.43:6809/3289134610] conn(0x7fcacc071150 0x7fcacc0735b0 crc :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T15:12:22.564 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:22.561+0000 7fcaee99f640 1 -- 172.21.15.43:0/2724840955 >> [v2:172.21.15.203:6832/1456699667,v1:172.21.15.203:6833/1456699667] conn(0x7fcacc06a170 msgr2=0x7fcacc06c630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T15:12:22.564 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:22.561+0000 7fcaee99f640 1 --2- 172.21.15.43:0/2724840955 >> [v2:172.21.15.203:6832/1456699667,v1:172.21.15.203:6833/1456699667] conn(0x7fcacc06a170 0x7fcacc06c630 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7fcae4036000 tx=0x7fcae403c040 comp rx=0 tx=0).stop 2023-08-09T15:12:22.564 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:22.561+0000 7fcaee99f640 1 -- 172.21.15.43:0/2724840955 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fcae81430b0 msgr2=0x7fcae80cd6e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T15:12:22.564 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:22.561+0000 7fcaee99f640 1 --2- 172.21.15.43:0/2724840955 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fcae81430b0 0x7fcae80cd6e0 secure :-1 s=READY pgs=2570 cs=0 l=1 rev1=1 crypto rx=0x7fcae000cbc0 tx=0x7fcae000a430 comp rx=0 tx=0).stop 2023-08-09T15:12:22.564 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:22.561+0000 7fcaee19e640 1 -- 172.21.15.43:0/2724840955 reap_dead start 2023-08-09T15:12:22.564 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:22.561+0000 7fcaee99f640 1 -- 172.21.15.43:0/2724840955 shutdown_connections 2023-08-09T15:12:22.564 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:22.561+0000 7fcaee99f640 1 -- 172.21.15.43:0/2724840955 >> 172.21.15.43:0/2724840955 conn(0x7fcae80cdff0 msgr2=0x7fcae8142270 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-09T15:12:22.565 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:22.561+0000 7fcaee99f640 1 -- 172.21.15.43:0/2724840955 shutdown_connections 2023-08-09T15:12:22.565 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:22.561+0000 7fcaee99f640 1 -- 172.21.15.43:0/2724840955 wait complete. 2023-08-09T15:12:22.579 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 1 2023-08-09T15:12:22.783 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:22.781+0000 7efe5a6fb640 1 Processor -- start 2023-08-09T15:12:22.783 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:22.782+0000 7efe5a6fb640 1 -- start start 2023-08-09T15:12:22.785 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:22.783+0000 7efe5a6fb640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7efe540cd3e0 0x7efe540c8820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T15:12:22.785 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:22.784+0000 7efe5a6fb640 1 -- --> v1:172.21.15.43:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7efe540ccf90 con 0x7efe5406fe20 2023-08-09T15:12:22.785 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:22.784+0000 7efe5a6fb640 1 -- --> v1:172.21.15.203:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7efe540cc6a0 con 0x7efe5415c630 2023-08-09T15:12:22.786 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:22.784+0000 7efe5a6fb640 1 -- --> v2:172.21.15.43:3300/0 -- mon_getmap magic: 0 v1 -- 0x7efe540cc940 con 0x7efe540cd3e0 2023-08-09T15:12:22.786 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:22.784+0000 7efe596f9640 1 --1- >> v1:172.21.15.203:6789/0 conn(0x7efe5415c630 0x7efe5406f710 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.203:6789/0 says I am v1:172.21.15.43:44280/0 (socket says 172.21.15.43:44280) 2023-08-09T15:12:22.787 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:22.784+0000 7efe596f9640 1 -- 172.21.15.43:0/3635757894 learned_addr learned my addr 172.21.15.43:0/3635757894 (peer_addr_for_me v1:172.21.15.43:0/0) 2023-08-09T15:12:22.787 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:22.784+0000 7efe58ef8640 1 --2- 172.21.15.43:0/3635757894 >> v2:172.21.15.43:3300/0 conn(0x7efe540cd3e0 0x7efe540c8820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T15:12:22.787 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:22.785+0000 7efe4bfff640 1 -- 172.21.15.43:0/3635757894 <== mon.0 v1:172.21.15.43:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4264617807 0 0) 0x7efe540ccf90 con 0x7efe5406fe20 2023-08-09T15:12:22.787 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:22.785+0000 7efe4bfff640 1 -- 172.21.15.43:0/3635757894 --> v1:172.21.15.43:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7efe40003520 con 0x7efe5406fe20 2023-08-09T15:12:22.787 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:22.785+0000 7efe4bfff640 1 -- 172.21.15.43:0/3635757894 <== mon.1 v1:172.21.15.203:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1579768622 0 0) 0x7efe540cc6a0 con 0x7efe5415c630 2023-08-09T15:12:22.787 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:22.785+0000 7efe4bfff640 1 -- 172.21.15.43:0/3635757894 --> v1:172.21.15.203:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7efe540ccf90 con 0x7efe5415c630 2023-08-09T15:12:22.787 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:22.785+0000 7efe58ef8640 1 -- 172.21.15.43:0/3635757894 >> v1:172.21.15.203:6789/0 conn(0x7efe5415c630 legacy=0x7efe5406f710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T15:12:22.788 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:22.785+0000 7efe58ef8640 1 -- 172.21.15.43:0/3635757894 >> v1:172.21.15.43:6789/0 conn(0x7efe5406fe20 legacy=0x7efe540c8d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T15:12:22.788 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:22.785+0000 7efe58ef8640 1 -- 172.21.15.43:0/3635757894 --> v2:172.21.15.43:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efe540ccc20 con 0x7efe540cd3e0 2023-08-09T15:12:22.788 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:22.786+0000 7efe58ef8640 1 --2- 172.21.15.43:0/3635757894 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7efe540cd3e0 0x7efe540c8820 secure :-1 s=READY pgs=2572 cs=0 l=1 rev1=1 crypto rx=0x7efe440097a0 tx=0x7efe4402b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1d91277a532978e4 server_cookie=0 in_seq=0 out_seq=0 2023-08-09T15:12:22.789 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:22.786+0000 7efe4bfff640 1 -- 172.21.15.43:0/3635757894 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efe44031830 con 0x7efe540cd3e0 2023-08-09T15:12:22.789 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:22.787+0000 7efe4bfff640 1 -- 172.21.15.43:0/3635757894 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efe44031990 con 0x7efe540cd3e0 2023-08-09T15:12:22.789 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:22.787+0000 7efe4bfff640 1 -- 172.21.15.43:0/3635757894 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efe44031d20 con 0x7efe540cd3e0 2023-08-09T15:12:22.789 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:22.787+0000 7efe5a6fb640 1 -- 172.21.15.43:0/3635757894 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7efe540cd3e0 msgr2=0x7efe540c8820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T15:12:22.789 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:22.787+0000 7efe5a6fb640 1 --2- 172.21.15.43:0/3635757894 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7efe540cd3e0 0x7efe540c8820 secure :-1 s=READY pgs=2572 cs=0 l=1 rev1=1 crypto rx=0x7efe440097a0 tx=0x7efe4402b8d0 comp rx=0 tx=0).stop 2023-08-09T15:12:22.789 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:22.787+0000 7efe5a6fb640 1 -- 172.21.15.43:0/3635757894 shutdown_connections 2023-08-09T15:12:22.790 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:22.787+0000 7efe5a6fb640 1 --2- 172.21.15.43:0/3635757894 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7efe540cd3e0 0x7efe540c8820 unknown :-1 s=CLOSED pgs=2572 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T15:12:22.790 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:22.787+0000 7efe5a6fb640 1 -- 172.21.15.43:0/3635757894 >> 172.21.15.43:0/3635757894 conn(0x7efe540ce020 msgr2=0x7efe540ce430 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-09T15:12:22.790 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:22.787+0000 7efe5a6fb640 1 -- 172.21.15.43:0/3635757894 shutdown_connections 2023-08-09T15:12:22.790 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:22.787+0000 7efe5a6fb640 1 -- 172.21.15.43:0/3635757894 wait complete. 2023-08-09T15:12:22.790 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:22.789+0000 7efe5a6fb640 1 Processor -- start 2023-08-09T15:12:22.791 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:22.790+0000 7efe5a6fb640 1 -- start start 2023-08-09T15:12:22.792 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:22.791+0000 7efe5a6fb640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7efe5406fe20 0x7efe541591b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T15:12:22.792 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:22.791+0000 7efe5a6fb640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7efe540cd3e0 0x7efe5415cd10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T15:12:22.793 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:22.791+0000 7efe5a6fb640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7efe5415c630 0x7efe540c5ca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T15:12:22.793 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:22.791+0000 7efe5a6fb640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efe540b20e0 con 0x7efe5415c630 2023-08-09T15:12:22.793 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:22.791+0000 7efe5a6fb640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efe5415fb70 con 0x7efe5406fe20 2023-08-09T15:12:22.793 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:22.791+0000 7efe5a6fb640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7efe540b28f0 con 0x7efe540cd3e0 2023-08-09T15:12:22.793 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:22.791+0000 7efe59efa640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7efe5415c630 0x7efe540c5ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T15:12:22.793 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:22.791+0000 7efe59efa640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7efe5415c630 0x7efe540c5ca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:46340/0 (socket says 172.21.15.43:46340) 2023-08-09T15:12:22.793 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:22.791+0000 7efe59efa640 1 -- 172.21.15.43:0/4197895516 learned_addr learned my addr 172.21.15.43:0/4197895516 (peer_addr_for_me v2:172.21.15.43:0/0) 2023-08-09T15:12:22.794 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:22.791+0000 7efe596f9640 1 --2- 172.21.15.43:0/4197895516 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7efe5406fe20 0x7efe541591b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T15:12:22.795 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:22.791+0000 7efe58ef8640 1 --2- 172.21.15.43:0/4197895516 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7efe540cd3e0 0x7efe5415cd10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T15:12:22.795 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:22.791+0000 7efe59efa640 1 -- 172.21.15.43:0/4197895516 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7efe540cd3e0 msgr2=0x7efe5415cd10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T15:12:22.795 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:22.792+0000 7efe59efa640 1 --2- 172.21.15.43:0/4197895516 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7efe540cd3e0 0x7efe5415cd10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T15:12:22.795 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:22.792+0000 7efe59efa640 1 -- 172.21.15.43:0/4197895516 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7efe5406fe20 msgr2=0x7efe541591b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T15:12:22.795 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:22.792+0000 7efe59efa640 1 --2- 172.21.15.43:0/4197895516 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7efe5406fe20 0x7efe541591b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T15:12:22.795 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:22.792+0000 7efe59efa640 1 -- 172.21.15.43:0/4197895516 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efe44009400 con 0x7efe5415c630 2023-08-09T15:12:22.795 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:22.792+0000 7efe59efa640 1 --2- 172.21.15.43:0/4197895516 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7efe5415c630 0x7efe540c5ca0 secure :-1 s=READY pgs=2573 cs=0 l=1 rev1=1 crypto rx=0x7efe4c002230 tx=0x7efe4c00cc00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-09T15:12:22.796 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:22.792+0000 7efe49ffb640 1 -- 172.21.15.43:0/4197895516 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efe4c003b70 con 0x7efe5415c630 2023-08-09T15:12:22.796 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:22.792+0000 7efe49ffb640 1 -- 172.21.15.43:0/4197895516 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efe4c003cd0 con 0x7efe5415c630 2023-08-09T15:12:22.796 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:22.792+0000 7efe5a6fb640 1 -- 172.21.15.43:0/4197895516 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7efe541596f0 con 0x7efe5415c630 2023-08-09T15:12:22.796 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:22.792+0000 7efe49ffb640 1 -- 172.21.15.43:0/4197895516 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efe4c00aa90 con 0x7efe5415c630 2023-08-09T15:12:22.796 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:22.792+0000 7efe5a6fb640 1 -- 172.21.15.43:0/4197895516 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7efe5415f890 con 0x7efe5415c630 2023-08-09T15:12:22.797 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:22.793+0000 7efe5a6fb640 1 -- 172.21.15.43:0/4197895516 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7efe5404e390 con 0x7efe5415c630 2023-08-09T15:12:22.797 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:22.794+0000 7efe49ffb640 1 -- 172.21.15.43:0/4197895516 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 129) v1 ==== 88443+0+0 (secure 0 0 0) 0x7efe4c00abf0 con 0x7efe5415c630 2023-08-09T15:12:22.797 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:22.796+0000 7efe49ffb640 1 --2- 172.21.15.43:0/4197895516 >> [v2:172.21.15.203:6832/1456699667,v1:172.21.15.203:6833/1456699667] conn(0x7efe3806a170 0x7efe3806c630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T15:12:22.797 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:22.796+0000 7efe49ffb640 1 -- 172.21.15.43:0/4197895516 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(229..229 src has 1..229) v4 ==== 6462+0+0 (secure 0 0 0) 0x7efe4c088710 con 0x7efe5415c630 2023-08-09T15:12:22.798 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:22.797+0000 7efe596f9640 1 --2- 172.21.15.43:0/4197895516 >> [v2:172.21.15.203:6832/1456699667,v1:172.21.15.203:6833/1456699667] conn(0x7efe3806a170 0x7efe3806c630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T15:12:22.799 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:22.798+0000 7efe596f9640 1 --2- 172.21.15.43:0/4197895516 >> [v2:172.21.15.203:6832/1456699667,v1:172.21.15.203:6833/1456699667] conn(0x7efe3806a170 0x7efe3806c630 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7efe500011e0 tx=0x7efe50009040 comp rx=0 tx=0).ready entity=mgr.9232 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-09T15:12:22.802 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:22.800+0000 7efe49ffb640 1 -- 172.21.15.43:0/4197895516 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178066 (secure 0 0 0) 0x7efe4c053b50 con 0x7efe5415c630 2023-08-09T15:12:22.996 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:22.995+0000 7efe5a6fb640 1 -- 172.21.15.43:0/4197895516 --> [v2:172.21.15.203:6832/1456699667,v1:172.21.15.203:6833/1456699667] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "1", "target": ["mon-mgr", ""]}) v1 -- 0x7efe540ad800 con 0x7efe3806a170 2023-08-09T15:12:22.998 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:22.997+0000 7efe49ffb640 1 -- 172.21.15.43:0/4197895516 <== mgr.9232 v2:172.21.15.203:6832/1456699667 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 1 to deep-scrub) v1 ==== 41+0+0 (secure 0 0 0) 0x7efe500058d0 con 0x7efe3806a170 2023-08-09T15:12:22.998 INFO:teuthology.orchestra.run.smithi043.stderr:instructed osd(s) 1 to deep-scrub 2023-08-09T15:12:23.000 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:22.999+0000 7efe5a6fb640 1 -- 172.21.15.43:0/4197895516 >> [v2:172.21.15.203:6832/1456699667,v1:172.21.15.203:6833/1456699667] conn(0x7efe3806a170 msgr2=0x7efe3806c630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T15:12:23.001 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:22.999+0000 7efe5a6fb640 1 --2- 172.21.15.43:0/4197895516 >> [v2:172.21.15.203:6832/1456699667,v1:172.21.15.203:6833/1456699667] conn(0x7efe3806a170 0x7efe3806c630 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7efe500011e0 tx=0x7efe50009040 comp rx=0 tx=0).stop 2023-08-09T15:12:23.001 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:22.999+0000 7efe5a6fb640 1 -- 172.21.15.43:0/4197895516 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7efe5415c630 msgr2=0x7efe540c5ca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T15:12:23.001 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:22.999+0000 7efe5a6fb640 1 --2- 172.21.15.43:0/4197895516 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7efe5415c630 0x7efe540c5ca0 secure :-1 s=READY pgs=2573 cs=0 l=1 rev1=1 crypto rx=0x7efe4c002230 tx=0x7efe4c00cc00 comp rx=0 tx=0).stop 2023-08-09T15:12:23.001 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:22.999+0000 7efe5a6fb640 1 -- 172.21.15.43:0/4197895516 shutdown_connections 2023-08-09T15:12:23.001 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:22.999+0000 7efe5a6fb640 1 --2- 172.21.15.43:0/4197895516 >> [v2:172.21.15.203:6832/1456699667,v1:172.21.15.203:6833/1456699667] conn(0x7efe3806a170 0x7efe3806c630 unknown :-1 s=CLOSED pgs=46 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T15:12:23.001 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:22.999+0000 7efe5a6fb640 1 --2- 172.21.15.43:0/4197895516 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7efe5415c630 0x7efe540c5ca0 unknown :-1 s=CLOSED pgs=2573 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T15:12:23.002 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:22.999+0000 7efe5a6fb640 1 --2- 172.21.15.43:0/4197895516 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7efe540cd3e0 0x7efe5415cd10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T15:12:23.002 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:22.999+0000 7efe5a6fb640 1 --2- 172.21.15.43:0/4197895516 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7efe5406fe20 0x7efe541591b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T15:12:23.002 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:22.999+0000 7efe5a6fb640 1 -- 172.21.15.43:0/4197895516 >> 172.21.15.43:0/4197895516 conn(0x7efe540ce020 msgr2=0x7efe540cd7c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-09T15:12:23.002 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:22.999+0000 7efe5a6fb640 1 -- 172.21.15.43:0/4197895516 shutdown_connections 2023-08-09T15:12:23.002 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:22.999+0000 7efe5a6fb640 1 -- 172.21.15.43:0/4197895516 wait complete. 2023-08-09T15:12:23.020 INFO:tasks.ceph:Scrubbing osd.2 2023-08-09T15:12:23.020 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.2 config set osd_debug_deep_scrub_sleep 0 2023-08-09T15:12:23.250 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:23.249+0000 7fd1476e7640 1 Processor -- start 2023-08-09T15:12:23.250 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:23.249+0000 7fd1476e7640 1 -- start start 2023-08-09T15:12:23.252 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:23.251+0000 7fd1476e7640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fd1400cac30 0x7fd1400cb0e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T15:12:23.252 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:23.251+0000 7fd1476e7640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7fd1400ab860 0x7fd1400abcb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T15:12:23.253 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:23.251+0000 7fd1476e7640 1 -- --> v1:172.21.15.43:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd140160300 con 0x7fd14015c710 2023-08-09T15:12:23.253 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:23.251+0000 7fd1476e7640 1 -- --> v2:172.21.15.43:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd1400cb620 con 0x7fd1400ab860 2023-08-09T15:12:23.253 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:23.251+0000 7fd1476e7640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd140160550 con 0x7fd1400cac30 2023-08-09T15:12:23.254 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:23.251+0000 7fd145c84640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7fd1400ab860 0x7fd1400abcb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T15:12:23.254 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:23.251+0000 7fd145483640 1 --1- >> v1:172.21.15.43:6789/0 conn(0x7fd14015c710 0x7fd1400c9dd0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.43:6789/0 says I am v1:172.21.15.43:32838/0 (socket says 172.21.15.43:32838) 2023-08-09T15:12:23.254 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:23.251+0000 7fd145483640 1 -- 172.21.15.43:0/1311485117 learned_addr learned my addr 172.21.15.43:0/1311485117 (peer_addr_for_me v1:172.21.15.43:0/0) 2023-08-09T15:12:23.254 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:23.252+0000 7fd144c82640 1 --2- 172.21.15.43:0/1311485117 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fd1400cac30 0x7fd1400cb0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T15:12:23.254 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:23.252+0000 7fd145c84640 1 -- 172.21.15.43:0/1311485117 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fd1400cac30 msgr2=0x7fd1400cb0e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-09T15:12:23.254 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:23.252+0000 7fd145c84640 1 --2- 172.21.15.43:0/1311485117 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fd1400cac30 0x7fd1400cb0e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T15:12:23.255 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:23.252+0000 7fd145c84640 1 -- 172.21.15.43:0/1311485117 >> v1:172.21.15.43:6789/0 conn(0x7fd14015c710 legacy=0x7fd1400c9dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T15:12:23.255 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:23.252+0000 7fd145c84640 1 -- 172.21.15.43:0/1311485117 --> v2:172.21.15.43:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd140160830 con 0x7fd1400ab860 2023-08-09T15:12:23.255 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:23.252+0000 7fd137fff640 1 -- 172.21.15.43:0/1311485117 <== mon.0 v1:172.21.15.43:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 799351573 0 0) 0x7fd140160300 con 0x7fd14015c710 2023-08-09T15:12:23.255 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:23.253+0000 7fd145c84640 1 --2- 172.21.15.43:0/1311485117 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fd1400ab860 0x7fd1400abcb0 secure :-1 s=READY pgs=2575 cs=0 l=1 rev1=1 crypto rx=0x7fd138009d90 tx=0x7fd13802bd80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6c4d88c2fb68510 server_cookie=0 in_seq=0 out_seq=0 2023-08-09T15:12:23.256 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:23.254+0000 7fd137fff640 1 -- 172.21.15.43:0/1311485117 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd13802cb30 con 0x7fd1400ab860 2023-08-09T15:12:23.256 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:23.254+0000 7fd137fff640 1 -- 172.21.15.43:0/1311485117 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd13802cc90 con 0x7fd1400ab860 2023-08-09T15:12:23.256 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:23.254+0000 7fd137fff640 1 -- 172.21.15.43:0/1311485117 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd138034440 con 0x7fd1400ab860 2023-08-09T15:12:23.256 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:23.254+0000 7fd1476e7640 1 -- 172.21.15.43:0/1311485117 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fd1400ab860 msgr2=0x7fd1400abcb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T15:12:23.256 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:23.254+0000 7fd1476e7640 1 --2- 172.21.15.43:0/1311485117 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fd1400ab860 0x7fd1400abcb0 secure :-1 s=READY pgs=2575 cs=0 l=1 rev1=1 crypto rx=0x7fd138009d90 tx=0x7fd13802bd80 comp rx=0 tx=0).stop 2023-08-09T15:12:23.257 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:23.255+0000 7fd1476e7640 1 -- 172.21.15.43:0/1311485117 shutdown_connections 2023-08-09T15:12:23.257 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:23.255+0000 7fd1476e7640 1 --2- 172.21.15.43:0/1311485117 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fd1400ab860 0x7fd1400abcb0 unknown :-1 s=CLOSED pgs=2575 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T15:12:23.257 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:23.255+0000 7fd1476e7640 1 --2- 172.21.15.43:0/1311485117 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fd1400cac30 0x7fd1400cb0e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T15:12:23.257 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:23.255+0000 7fd1476e7640 1 -- 172.21.15.43:0/1311485117 >> 172.21.15.43:0/1311485117 conn(0x7fd1400afdb0 msgr2=0x7fd1400b0190 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-09T15:12:23.257 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:23.255+0000 7fd1476e7640 1 -- 172.21.15.43:0/1311485117 shutdown_connections 2023-08-09T15:12:23.258 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:23.255+0000 7fd1476e7640 1 -- 172.21.15.43:0/1311485117 wait complete. 2023-08-09T15:12:23.258 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:23.256+0000 7fd1476e7640 1 Processor -- start 2023-08-09T15:12:23.258 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:23.257+0000 7fd1476e7640 1 -- start start 2023-08-09T15:12:23.259 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:23.258+0000 7fd1476e7640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fd1400ab860 0x7fd1401f20e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T15:12:23.259 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:23.258+0000 7fd1476e7640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fd1400cac30 0x7fd1400b37c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T15:12:23.259 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:23.258+0000 7fd1476e7640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fd14015c710 0x7fd1401f5910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T15:12:23.259 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:23.258+0000 7fd1476e7640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd1401f6520 con 0x7fd14015c710 2023-08-09T15:12:23.260 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:23.258+0000 7fd1476e7640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd1400b0ea0 con 0x7fd1400cac30 2023-08-09T15:12:23.260 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:23.258+0000 7fd1476e7640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd1400b1160 con 0x7fd1400ab860 2023-08-09T15:12:23.261 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:23.258+0000 7fd145c84640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fd14015c710 0x7fd1401f5910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T15:12:23.261 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:23.258+0000 7fd145c84640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fd14015c710 0x7fd1401f5910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:46362/0 (socket says 172.21.15.43:46362) 2023-08-09T15:12:23.261 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:23.258+0000 7fd145c84640 1 -- 172.21.15.43:0/1827817646 learned_addr learned my addr 172.21.15.43:0/1827817646 (peer_addr_for_me v2:172.21.15.43:0/0) 2023-08-09T15:12:23.261 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:23.258+0000 7fd144c82640 1 --2- 172.21.15.43:0/1827817646 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fd1400cac30 0x7fd1400b37c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T15:12:23.262 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:23.258+0000 7fd145483640 1 --2- 172.21.15.43:0/1827817646 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fd1400ab860 0x7fd1401f20e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T15:12:23.262 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:23.259+0000 7fd145c84640 1 -- 172.21.15.43:0/1827817646 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fd1400ab860 msgr2=0x7fd1401f20e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T15:12:23.262 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:23.259+0000 7fd145c84640 1 --2- 172.21.15.43:0/1827817646 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fd1400ab860 0x7fd1401f20e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T15:12:23.262 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:23.259+0000 7fd145c84640 1 -- 172.21.15.43:0/1827817646 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fd1400cac30 msgr2=0x7fd1400b37c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T15:12:23.262 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:23.259+0000 7fd145c84640 1 --2- 172.21.15.43:0/1827817646 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fd1400cac30 0x7fd1400b37c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T15:12:23.262 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:23.259+0000 7fd145c84640 1 -- 172.21.15.43:0/1827817646 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd138009a70 con 0x7fd14015c710 2023-08-09T15:12:23.262 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:23.259+0000 7fd144c82640 1 --2- 172.21.15.43:0/1827817646 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fd1400cac30 0x7fd1400b37c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-08-09T15:12:23.263 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:23.259+0000 7fd145483640 1 --2- 172.21.15.43:0/1827817646 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fd1400ab860 0x7fd1401f20e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-08-09T15:12:23.263 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:23.259+0000 7fd145c84640 1 --2- 172.21.15.43:0/1827817646 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fd14015c710 0x7fd1401f5910 secure :-1 s=READY pgs=2576 cs=0 l=1 rev1=1 crypto rx=0x7fd138009d90 tx=0x7fd138004a90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-09T15:12:23.263 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:23.259+0000 7fd135ffb640 1 -- 172.21.15.43:0/1827817646 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd138009450 con 0x7fd14015c710 2023-08-09T15:12:23.263 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:23.259+0000 7fd135ffb640 1 -- 172.21.15.43:0/1827817646 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd138035e60 con 0x7fd14015c710 2023-08-09T15:12:23.263 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:23.259+0000 7fd135ffb640 1 -- 172.21.15.43:0/1827817646 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd138003c00 con 0x7fd14015c710 2023-08-09T15:12:23.263 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:23.259+0000 7fd1476e7640 1 -- 172.21.15.43:0/1827817646 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd1400b1440 con 0x7fd14015c710 2023-08-09T15:12:23.263 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:23.260+0000 7fd1476e7640 1 -- 172.21.15.43:0/1827817646 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd1400b1a00 con 0x7fd14015c710 2023-08-09T15:12:23.264 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:23.260+0000 7fd1476e7640 1 -- 172.21.15.43:0/1827817646 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7fd140160df0 con 0x7fd14015c710 2023-08-09T15:12:23.264 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:23.262+0000 7fd135ffb640 1 -- 172.21.15.43:0/1827817646 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 129) v1 ==== 88443+0+0 (secure 0 0 0) 0x7fd13803b050 con 0x7fd14015c710 2023-08-09T15:12:23.265 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:23.263+0000 7fd135ffb640 1 --2- 172.21.15.43:0/1827817646 >> [v2:172.21.15.203:6832/1456699667,v1:172.21.15.203:6833/1456699667] conn(0x7fd12406a170 0x7fd12406c630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T15:12:23.265 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:23.263+0000 7fd135ffb640 1 -- 172.21.15.43:0/1827817646 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(229..229 src has 1..229) v4 ==== 6462+0+0 (secure 0 0 0) 0x7fd1380ab450 con 0x7fd14015c710 2023-08-09T15:12:23.265 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:23.263+0000 7fd135ffb640 1 --2- 172.21.15.43:0/1827817646 >> [v2:172.21.15.43:6811/4079709736,v1:172.21.15.43:6814/4079709736] conn(0x7fd124071150 0x7fd1240735b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T15:12:23.265 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:23.263+0000 7fd135ffb640 1 -- 172.21.15.43:0/1827817646 --> [v2:172.21.15.43:6811/4079709736,v1:172.21.15.43:6814/4079709736] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7fd124073c80 con 0x7fd124071150 2023-08-09T15:12:23.265 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:23.263+0000 7fd135ffb640 1 -- 172.21.15.43:0/1827817646 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_get_version_reply(handle=1 version=229) v2 ==== 24+0+0 (secure 0 0 0) 0x7fd1380ab7c0 con 0x7fd14015c710 2023-08-09T15:12:23.265 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:23.263+0000 7fd144c82640 1 --2- 172.21.15.43:0/1827817646 >> [v2:172.21.15.43:6811/4079709736,v1:172.21.15.43:6814/4079709736] conn(0x7fd124071150 0x7fd1240735b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T15:12:23.266 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:23.263+0000 7fd145483640 1 --2- 172.21.15.43:0/1827817646 >> [v2:172.21.15.203:6832/1456699667,v1:172.21.15.203:6833/1456699667] conn(0x7fd12406a170 0x7fd12406c630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T15:12:23.266 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:23.264+0000 7fd144c82640 1 --2- 172.21.15.43:0/1827817646 >> [v2:172.21.15.43:6811/4079709736,v1:172.21.15.43:6814/4079709736] conn(0x7fd124071150 0x7fd1240735b0 crc :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-09T15:12:23.266 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:23.265+0000 7fd135ffb640 1 -- 172.21.15.43:0/1827817646 <== osd.2 v2:172.21.15.43:6811/4079709736 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+25607 (crc 0 0 0) 0x7fd124073c80 con 0x7fd124071150 2023-08-09T15:12:23.266 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:23.265+0000 7fd145483640 1 --2- 172.21.15.43:0/1827817646 >> [v2:172.21.15.203:6832/1456699667,v1:172.21.15.203:6833/1456699667] conn(0x7fd12406a170 0x7fd12406c630 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7fd13c000f60 tx=0x7fd13c009040 comp rx=0 tx=0).ready entity=mgr.9232 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-09T15:12:23.280 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:23.279+0000 7fd1476e7640 1 -- 172.21.15.43:0/1827817646 --> [v2:172.21.15.43:6811/4079709736,v1:172.21.15.43:6814/4079709736] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) v1 -- 0x7fd140160b10 con 0x7fd124071150 2023-08-09T15:12:23.281 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:23.280+0000 7fd135ffb640 1 -- 172.21.15.43:0/1827817646 <== osd.2 v2:172.21.15.43:6811/4079709736 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+105 (crc 0 0 0) 0x7fd130009510 con 0x7fd124071150 2023-08-09T15:12:23.281 INFO:teuthology.orchestra.run.smithi043.stdout:{ 2023-08-09T15:12:23.282 INFO:teuthology.orchestra.run.smithi043.stdout: "success": "osd_debug_deep_scrub_sleep = '0.000000' (not observed, change may require restart) " 2023-08-09T15:12:23.282 INFO:teuthology.orchestra.run.smithi043.stdout:} 2023-08-09T15:12:23.282 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:23.280+0000 7fd1476e7640 1 -- 172.21.15.43:0/1827817646 >> [v2:172.21.15.43:6811/4079709736,v1:172.21.15.43:6814/4079709736] conn(0x7fd124071150 msgr2=0x7fd1240735b0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T15:12:23.283 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:23.280+0000 7fd1476e7640 1 --2- 172.21.15.43:0/1827817646 >> [v2:172.21.15.43:6811/4079709736,v1:172.21.15.43:6814/4079709736] conn(0x7fd124071150 0x7fd1240735b0 crc :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T15:12:23.283 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:23.280+0000 7fd1476e7640 1 -- 172.21.15.43:0/1827817646 >> [v2:172.21.15.203:6832/1456699667,v1:172.21.15.203:6833/1456699667] conn(0x7fd12406a170 msgr2=0x7fd12406c630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T15:12:23.283 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:23.280+0000 7fd1476e7640 1 --2- 172.21.15.43:0/1827817646 >> [v2:172.21.15.203:6832/1456699667,v1:172.21.15.203:6833/1456699667] conn(0x7fd12406a170 0x7fd12406c630 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7fd13c000f60 tx=0x7fd13c009040 comp rx=0 tx=0).stop 2023-08-09T15:12:23.283 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:23.280+0000 7fd1476e7640 1 -- 172.21.15.43:0/1827817646 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fd14015c710 msgr2=0x7fd1401f5910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T15:12:23.283 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:23.280+0000 7fd1476e7640 1 --2- 172.21.15.43:0/1827817646 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fd14015c710 0x7fd1401f5910 secure :-1 s=READY pgs=2576 cs=0 l=1 rev1=1 crypto rx=0x7fd138009d90 tx=0x7fd138004a90 comp rx=0 tx=0).stop 2023-08-09T15:12:23.283 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:23.280+0000 7fd145c84640 1 -- 172.21.15.43:0/1827817646 reap_dead start 2023-08-09T15:12:23.284 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:23.280+0000 7fd1476e7640 1 -- 172.21.15.43:0/1827817646 shutdown_connections 2023-08-09T15:12:23.284 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:23.280+0000 7fd1476e7640 1 -- 172.21.15.43:0/1827817646 >> 172.21.15.43:0/1827817646 conn(0x7fd1400afdb0 msgr2=0x7fd1400ad040 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-09T15:12:23.284 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:23.280+0000 7fd1476e7640 1 -- 172.21.15.43:0/1827817646 shutdown_connections 2023-08-09T15:12:23.284 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:23.280+0000 7fd1476e7640 1 -- 172.21.15.43:0/1827817646 wait complete. 2023-08-09T15:12:23.298 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 2 2023-08-09T15:12:23.509 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:23.508+0000 7fbbb55a2640 1 Processor -- start 2023-08-09T15:12:23.510 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:23.508+0000 7fbbb55a2640 1 -- start start 2023-08-09T15:12:23.511 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:23.510+0000 7fbbb55a2640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fbbb006b4d0 0x7fbbb006b950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T15:12:23.512 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:23.510+0000 7fbbb55a2640 1 -- --> v1:172.21.15.43:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbbb015de60 con 0x7fbbb01570b0 2023-08-09T15:12:23.512 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:23.510+0000 7fbbb55a2640 1 -- --> v1:172.21.15.203:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbbb015d2e0 con 0x7fbbb00709b0 2023-08-09T15:12:23.512 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:23.510+0000 7fbbb55a2640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbbb0070580 con 0x7fbbb006b4d0 2023-08-09T15:12:23.512 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:23.510+0000 7fbbaeffd640 1 --1- >> v1:172.21.15.43:6789/0 conn(0x7fbbb01570b0 0x7fbbb00cd950 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.43:6789/0 says I am v1:172.21.15.43:32852/0 (socket says 172.21.15.43:32852) 2023-08-09T15:12:23.512 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:23.510+0000 7fbbaeffd640 1 -- 172.21.15.43:0/2949587800 learned_addr learned my addr 172.21.15.43:0/2949587800 (peer_addr_for_me v1:172.21.15.43:0/0) 2023-08-09T15:12:23.515 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:23.511+0000 7fbbae7fc640 1 --2- 172.21.15.43:0/2949587800 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fbbb006b4d0 0x7fbbb006b950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T15:12:23.515 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:23.511+0000 7fbbadffb640 1 -- 172.21.15.43:0/2949587800 <== mon.0 v1:172.21.15.43:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3983351844 0 0) 0x7fbbb015de60 con 0x7fbbb01570b0 2023-08-09T15:12:23.516 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:23.511+0000 7fbbadffb640 1 -- 172.21.15.43:0/2949587800 --> v1:172.21.15.43:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fbb9c003520 con 0x7fbbb01570b0 2023-08-09T15:12:23.516 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:23.511+0000 7fbbadffb640 1 -- 172.21.15.43:0/2949587800 <== mon.0 v1:172.21.15.43:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 4139023847 0 0) 0x7fbb9c003520 con 0x7fbbb01570b0 2023-08-09T15:12:23.516 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:23.511+0000 7fbbadffb640 1 -- 172.21.15.43:0/2949587800 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fbbb006b4d0 msgr2=0x7fbbb006b950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-09T15:12:23.516 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:23.511+0000 7fbbadffb640 1 --2- 172.21.15.43:0/2949587800 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fbbb006b4d0 0x7fbbb006b950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T15:12:23.517 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:23.511+0000 7fbbadffb640 1 -- 172.21.15.43:0/2949587800 >> v1:172.21.15.203:6789/0 conn(0x7fbbb00709b0 legacy=0x7fbbb0070e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T15:12:23.517 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:23.511+0000 7fbbadffb640 1 -- 172.21.15.43:0/2949587800 --> v1:172.21.15.43:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbbb015d5c0 con 0x7fbbb01570b0 2023-08-09T15:12:23.517 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:23.511+0000 7fbbadffb640 1 -- 172.21.15.43:0/2949587800 <== mon.0 v1:172.21.15.43:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1285150740 0 0) 0x7fbba00019f0 con 0x7fbbb01570b0 2023-08-09T15:12:23.518 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:23.511+0000 7fbbadffb640 1 -- 172.21.15.43:0/2949587800 >> v1:172.21.15.43:6789/0 conn(0x7fbbb01570b0 legacy=0x7fbbb00cd950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T15:12:23.518 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:23.511+0000 7fbbadffb640 1 --2- 172.21.15.43:0/2949587800 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fbb9c0039a0 0x7fbb9c003dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T15:12:23.518 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:23.511+0000 7fbbadffb640 1 --2- 172.21.15.43:0/2949587800 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fbbb006b4d0 0x7fbb9c0047c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T15:12:23.518 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:23.511+0000 7fbbadffb640 1 --2- 172.21.15.43:0/2949587800 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fbb9c004d00 0x7fbb9c005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T15:12:23.519 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:23.511+0000 7fbbadffb640 1 -- 172.21.15.43:0/2949587800 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbb9c00dc40 con 0x7fbb9c0039a0 2023-08-09T15:12:23.519 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:23.511+0000 7fbbadffb640 1 -- 172.21.15.43:0/2949587800 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbb9c00dd80 con 0x7fbb9c004d00 2023-08-09T15:12:23.519 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:23.511+0000 7fbbadffb640 1 -- 172.21.15.43:0/2949587800 --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbb9c00df40 con 0x7fbbb006b4d0 2023-08-09T15:12:23.520 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:23.512+0000 7fbbaeffd640 1 --2- 172.21.15.43:0/2949587800 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fbb9c0039a0 0x7fbb9c003dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T15:12:23.520 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:23.512+0000 7fbbaf7fe640 1 --2- 172.21.15.43:0/2949587800 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fbb9c004d00 0x7fbb9c005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T15:12:23.520 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:23.512+0000 7fbbaeffd640 1 -- 172.21.15.43:0/2949587800 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fbbb006b4d0 msgr2=0x7fbb9c0047c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-09T15:12:23.520 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:23.512+0000 7fbbae7fc640 1 --2- 172.21.15.43:0/2949587800 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fbbb006b4d0 0x7fbb9c0047c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T15:12:23.521 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:23.512+0000 7fbbaeffd640 1 --2- 172.21.15.43:0/2949587800 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fbbb006b4d0 0x7fbb9c0047c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T15:12:23.521 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:23.512+0000 7fbbaeffd640 1 -- 172.21.15.43:0/2949587800 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fbb9c004d00 msgr2=0x7fbb9c005180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-09T15:12:23.521 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:23.512+0000 7fbbaeffd640 1 --2- 172.21.15.43:0/2949587800 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fbb9c004d00 0x7fbb9c005180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T15:12:23.522 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:23.512+0000 7fbbaeffd640 1 -- 172.21.15.43:0/2949587800 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbb9c003520 con 0x7fbb9c0039a0 2023-08-09T15:12:23.522 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:23.512+0000 7fbbaf7fe640 1 --2- 172.21.15.43:0/2949587800 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fbb9c004d00 0x7fbb9c005180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-08-09T15:12:23.522 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:23.512+0000 7fbbae7fc640 1 --2- 172.21.15.43:0/2949587800 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fbbb006b4d0 0x7fbb9c0047c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-08-09T15:12:23.522 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:23.513+0000 7fbbaeffd640 1 --2- 172.21.15.43:0/2949587800 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fbb9c0039a0 0x7fbb9c003dc0 secure :-1 s=READY pgs=2578 cs=0 l=1 rev1=1 crypto rx=0x7fbba000a800 tx=0x7fbb9c0057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1561f5ebcf2931ce server_cookie=0 in_seq=0 out_seq=0 2023-08-09T15:12:23.523 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:23.514+0000 7fbbadffb640 1 -- 172.21.15.43:0/2949587800 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbba00072d0 con 0x7fbb9c0039a0 2023-08-09T15:12:23.523 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:23.514+0000 7fbbadffb640 1 -- 172.21.15.43:0/2949587800 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbba002f070 con 0x7fbb9c0039a0 2023-08-09T15:12:23.523 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:23.514+0000 7fbbadffb640 1 -- 172.21.15.43:0/2949587800 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbba00076b0 con 0x7fbb9c0039a0 2023-08-09T15:12:23.527 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:23.514+0000 7fbbb55a2640 1 -- 172.21.15.43:0/2949587800 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fbb9c0039a0 msgr2=0x7fbb9c003dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T15:12:23.528 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:23.514+0000 7fbbb55a2640 1 --2- 172.21.15.43:0/2949587800 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fbb9c0039a0 0x7fbb9c003dc0 secure :-1 s=READY pgs=2578 cs=0 l=1 rev1=1 crypto rx=0x7fbba000a800 tx=0x7fbb9c0057f0 comp rx=0 tx=0).stop 2023-08-09T15:12:23.528 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:23.514+0000 7fbbaf7fe640 1 -- 172.21.15.43:0/2949587800 reap_dead start 2023-08-09T15:12:23.528 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:23.514+0000 7fbbb55a2640 1 -- 172.21.15.43:0/2949587800 shutdown_connections 2023-08-09T15:12:23.529 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:23.514+0000 7fbbb55a2640 1 -- 172.21.15.43:0/2949587800 >> 172.21.15.43:0/2949587800 conn(0x7fbbb0069dd0 msgr2=0x7fbbb00cac80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-09T15:12:23.529 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:23.514+0000 7fbbb55a2640 1 -- 172.21.15.43:0/2949587800 shutdown_connections 2023-08-09T15:12:23.529 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:23.515+0000 7fbbb55a2640 1 -- 172.21.15.43:0/2949587800 wait complete. 2023-08-09T15:12:23.529 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:23.516+0000 7fbbb55a2640 1 Processor -- start 2023-08-09T15:12:23.529 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:23.517+0000 7fbbb55a2640 1 -- start start 2023-08-09T15:12:23.530 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:23.518+0000 7fbbb55a2640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fbb9c0039a0 0x7fbbb00a9180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T15:12:23.530 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:23.518+0000 7fbbb55a2640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fbbb01ec3c0 0x7fbbb01f2e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T15:12:23.530 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:23.518+0000 7fbbb55a2640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fbbb0152c40 0x7fbbb01530c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T15:12:23.530 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:23.518+0000 7fbbb55a2640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbbb0153600 con 0x7fbbb01ec3c0 2023-08-09T15:12:23.531 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:23.518+0000 7fbbb55a2640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbbb01f3380 con 0x7fbb9c0039a0 2023-08-09T15:12:23.531 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:23.518+0000 7fbbb55a2640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbbb01ec840 con 0x7fbbb0152c40 2023-08-09T15:12:23.531 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:23.518+0000 7fbbae7fc640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fbbb01ec3c0 0x7fbbb01f2e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T15:12:23.531 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:23.518+0000 7fbbae7fc640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fbbb01ec3c0 0x7fbbb01f2e40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:46378/0 (socket says 172.21.15.43:46378) 2023-08-09T15:12:23.532 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:23.518+0000 7fbbae7fc640 1 -- 172.21.15.43:0/3114556922 learned_addr learned my addr 172.21.15.43:0/3114556922 (peer_addr_for_me v2:172.21.15.43:0/0) 2023-08-09T15:12:23.532 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:23.518+0000 7fbbae7fc640 1 -- 172.21.15.43:0/3114556922 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fbbb0152c40 msgr2=0x7fbbb01530c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T15:12:23.532 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:23.518+0000 7fbbae7fc640 1 --2- 172.21.15.43:0/3114556922 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fbbb0152c40 0x7fbbb01530c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T15:12:23.533 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:23.518+0000 7fbbae7fc640 1 -- 172.21.15.43:0/3114556922 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fbb9c0039a0 msgr2=0x7fbbb00a9180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T15:12:23.533 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:23.518+0000 7fbbae7fc640 1 --2- 172.21.15.43:0/3114556922 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fbb9c0039a0 0x7fbbb00a9180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T15:12:23.533 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:23.518+0000 7fbbae7fc640 1 -- 172.21.15.43:0/3114556922 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbbb00a9810 con 0x7fbbb01ec3c0 2023-08-09T15:12:23.534 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:23.519+0000 7fbbae7fc640 1 --2- 172.21.15.43:0/3114556922 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fbbb01ec3c0 0x7fbbb01f2e40 secure :-1 s=READY pgs=2579 cs=0 l=1 rev1=1 crypto rx=0x7fbb980108b0 tx=0x7fbb98010d70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-09T15:12:23.534 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:23.519+0000 7fbba77fe640 1 -- 172.21.15.43:0/3114556922 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbb9800eb60 con 0x7fbbb01ec3c0 2023-08-09T15:12:23.534 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:23.519+0000 7fbba77fe640 1 -- 172.21.15.43:0/3114556922 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbb9800ecc0 con 0x7fbbb01ec3c0 2023-08-09T15:12:23.534 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:23.519+0000 7fbbb55a2640 1 -- 172.21.15.43:0/3114556922 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbbb00a9af0 con 0x7fbbb01ec3c0 2023-08-09T15:12:23.535 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:23.519+0000 7fbba77fe640 1 -- 172.21.15.43:0/3114556922 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbb9801c400 con 0x7fbbb01ec3c0 2023-08-09T15:12:23.535 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:23.519+0000 7fbbb55a2640 1 -- 172.21.15.43:0/3114556922 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbbb00ab7a0 con 0x7fbbb01ec3c0 2023-08-09T15:12:23.535 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:23.520+0000 7fbbb55a2640 1 -- 172.21.15.43:0/3114556922 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbbb00ab320 con 0x7fbbb01ec3c0 2023-08-09T15:12:23.536 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:23.521+0000 7fbba77fe640 1 -- 172.21.15.43:0/3114556922 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 129) v1 ==== 88443+0+0 (secure 0 0 0) 0x7fbb9801c560 con 0x7fbbb01ec3c0 2023-08-09T15:12:23.536 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:23.522+0000 7fbba77fe640 1 --2- 172.21.15.43:0/3114556922 >> [v2:172.21.15.203:6832/1456699667,v1:172.21.15.203:6833/1456699667] conn(0x7fbb9006a1a0 0x7fbb9006c660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T15:12:23.536 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:23.523+0000 7fbba77fe640 1 -- 172.21.15.43:0/3114556922 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(229..229 src has 1..229) v4 ==== 6462+0+0 (secure 0 0 0) 0x7fbb98088e20 con 0x7fbbb01ec3c0 2023-08-09T15:12:23.537 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:23.523+0000 7fbbaeffd640 1 --2- 172.21.15.43:0/3114556922 >> [v2:172.21.15.203:6832/1456699667,v1:172.21.15.203:6833/1456699667] conn(0x7fbb9006a1a0 0x7fbb9006c660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T15:12:23.537 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:23.524+0000 7fbbaeffd640 1 --2- 172.21.15.43:0/3114556922 >> [v2:172.21.15.203:6832/1456699667,v1:172.21.15.203:6833/1456699667] conn(0x7fbb9006a1a0 0x7fbb9006c660 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7fbba0034000 tx=0x7fbba002f000 comp rx=0 tx=0).ready entity=mgr.9232 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-09T15:12:23.537 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:23.526+0000 7fbba77fe640 1 -- 172.21.15.43:0/3114556922 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178066 (secure 0 0 0) 0x7fbbb00ab320 con 0x7fbbb01ec3c0 2023-08-09T15:12:23.723 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:23.721+0000 7fbbb55a2640 1 -- 172.21.15.43:0/3114556922 --> [v2:172.21.15.203:6832/1456699667,v1:172.21.15.203:6833/1456699667] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "2", "target": ["mon-mgr", ""]}) v1 -- 0x7fbbb015db80 con 0x7fbb9006a1a0 2023-08-09T15:12:23.724 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:23.723+0000 7fbba77fe640 1 -- 172.21.15.43:0/3114556922 <== mgr.9232 v2:172.21.15.203:6832/1456699667 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 2 to deep-scrub) v1 ==== 41+0+0 (secure 0 0 0) 0x7fbba0001830 con 0x7fbb9006a1a0 2023-08-09T15:12:23.725 INFO:teuthology.orchestra.run.smithi043.stderr:instructed osd(s) 2 to deep-scrub 2023-08-09T15:12:23.727 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:23.725+0000 7fbbb55a2640 1 -- 172.21.15.43:0/3114556922 >> [v2:172.21.15.203:6832/1456699667,v1:172.21.15.203:6833/1456699667] conn(0x7fbb9006a1a0 msgr2=0x7fbb9006c660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T15:12:23.727 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:23.725+0000 7fbbb55a2640 1 --2- 172.21.15.43:0/3114556922 >> [v2:172.21.15.203:6832/1456699667,v1:172.21.15.203:6833/1456699667] conn(0x7fbb9006a1a0 0x7fbb9006c660 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7fbba0034000 tx=0x7fbba002f000 comp rx=0 tx=0).stop 2023-08-09T15:12:23.727 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:23.725+0000 7fbbb55a2640 1 -- 172.21.15.43:0/3114556922 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fbbb01ec3c0 msgr2=0x7fbbb01f2e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T15:12:23.727 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:23.725+0000 7fbbb55a2640 1 --2- 172.21.15.43:0/3114556922 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fbbb01ec3c0 0x7fbbb01f2e40 secure :-1 s=READY pgs=2579 cs=0 l=1 rev1=1 crypto rx=0x7fbb980108b0 tx=0x7fbb98010d70 comp rx=0 tx=0).stop 2023-08-09T15:12:23.728 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:23.725+0000 7fbbb55a2640 1 -- 172.21.15.43:0/3114556922 shutdown_connections 2023-08-09T15:12:23.728 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:23.725+0000 7fbbb55a2640 1 --2- 172.21.15.43:0/3114556922 >> [v2:172.21.15.203:6832/1456699667,v1:172.21.15.203:6833/1456699667] conn(0x7fbb9006a1a0 0x7fbb9006c660 unknown :-1 s=CLOSED pgs=48 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T15:12:23.728 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:23.725+0000 7fbbb55a2640 1 --2- 172.21.15.43:0/3114556922 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fbbb0152c40 0x7fbbb01530c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T15:12:23.728 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:23.725+0000 7fbbb55a2640 1 --2- 172.21.15.43:0/3114556922 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fbbb01ec3c0 0x7fbbb01f2e40 unknown :-1 s=CLOSED pgs=2579 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T15:12:23.728 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:23.725+0000 7fbbb55a2640 1 --2- 172.21.15.43:0/3114556922 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fbb9c0039a0 0x7fbbb00a9180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T15:12:23.728 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:23.725+0000 7fbbb55a2640 1 -- 172.21.15.43:0/3114556922 >> 172.21.15.43:0/3114556922 conn(0x7fbbb0069dd0 msgr2=0x7fbbb00cd2b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-09T15:12:23.728 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:23.726+0000 7fbbb55a2640 1 -- 172.21.15.43:0/3114556922 shutdown_connections 2023-08-09T15:12:23.729 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:23.726+0000 7fbbb55a2640 1 -- 172.21.15.43:0/3114556922 wait complete. 2023-08-09T15:12:23.747 INFO:tasks.ceph:Scrubbing osd.3 2023-08-09T15:12:23.748 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.3 config set osd_debug_deep_scrub_sleep 0 2023-08-09T15:12:23.986 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:23.984+0000 7f7893fff640 1 Processor -- start 2023-08-09T15:12:23.986 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:23.984+0000 7f7893fff640 1 -- start start 2023-08-09T15:12:23.988 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:23.986+0000 7f7893fff640 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7f789415fbe0 0x7f78940b5660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T15:12:23.988 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:23.986+0000 7f7893fff640 1 -- --> v1:172.21.15.43:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f789415ffc0 con 0x7f7894160be0 2023-08-09T15:12:23.988 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:23.986+0000 7f7893fff640 1 -- --> v1:172.21.15.203:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f78940cd460 con 0x7f78940b4690 2023-08-09T15:12:23.989 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:23.986+0000 7f7893fff640 1 -- --> v2:172.21.15.203:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f78940d1f50 con 0x7f789415fbe0 2023-08-09T15:12:23.989 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:23.986+0000 7f788a7fc640 1 --1- >> v1:172.21.15.43:6789/0 conn(0x7f7894160be0 0x7f78940b3f80 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.43:6789/0 says I am v1:172.21.15.43:32868/0 (socket says 172.21.15.43:32868) 2023-08-09T15:12:23.989 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:23.986+0000 7f788a7fc640 1 -- 172.21.15.43:0/1643491044 learned_addr learned my addr 172.21.15.43:0/1643491044 (peer_addr_for_me v1:172.21.15.43:0/0) 2023-08-09T15:12:23.993 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:23.987+0000 7f7892ffd640 1 --2- 172.21.15.43:0/1643491044 >> v2:172.21.15.203:3300/0 conn(0x7f789415fbe0 0x7f78940b5660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T15:12:23.993 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:23.987+0000 7f78927fc640 1 -- 172.21.15.43:0/1643491044 <== mon.0 v1:172.21.15.43:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3336991521 0 0) 0x7f787c001110 con 0x7f7894160be0 2023-08-09T15:12:23.994 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:23.987+0000 7f78927fc640 1 -- 172.21.15.43:0/1643491044 --> v1:172.21.15.43:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f7880003520 con 0x7f7894160be0 2023-08-09T15:12:23.994 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:23.987+0000 7f78927fc640 1 -- 172.21.15.43:0/1643491044 <== mon.0 v1:172.21.15.43:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3707257928 0 0) 0x7f7880003520 con 0x7f7894160be0 2023-08-09T15:12:23.994 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:23.988+0000 7f78927fc640 1 -- 172.21.15.43:0/1643491044 >> v2:172.21.15.203:3300/0 conn(0x7f789415fbe0 msgr2=0x7f78940b5660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-09T15:12:23.994 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:23.988+0000 7f78927fc640 1 --2- 172.21.15.43:0/1643491044 >> v2:172.21.15.203:3300/0 conn(0x7f789415fbe0 0x7f78940b5660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T15:12:23.995 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:23.988+0000 7f78927fc640 1 -- 172.21.15.43:0/1643491044 >> v1:172.21.15.203:6789/0 conn(0x7f78940b4690 legacy=0x7f78940cdf00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T15:12:23.995 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:23.988+0000 7f78927fc640 1 -- 172.21.15.43:0/1643491044 --> v1:172.21.15.43:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f78940d0e10 con 0x7f7894160be0 2023-08-09T15:12:23.995 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:23.988+0000 7f78927fc640 1 -- 172.21.15.43:0/1643491044 <== mon.0 v1:172.21.15.43:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1285150740 0 0) 0x7f787c001800 con 0x7f7894160be0 2023-08-09T15:12:23.996 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:23.988+0000 7f78927fc640 1 -- 172.21.15.43:0/1643491044 >> v1:172.21.15.43:6789/0 conn(0x7f7894160be0 legacy=0x7f78940b3f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T15:12:23.996 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:23.988+0000 7f78927fc640 1 --2- 172.21.15.43:0/1643491044 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f78800039e0 0x7f7880003e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T15:12:23.996 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:23.988+0000 7f78927fc640 1 --2- 172.21.15.43:0/1643491044 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f78800047c0 0x7f7880004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T15:12:23.996 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:23.988+0000 7f78927fc640 1 --2- 172.21.15.43:0/1643491044 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f7880005180 0x7f788000b980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T15:12:23.997 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:23.988+0000 7f78927fc640 1 -- 172.21.15.43:0/1643491044 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f788000c080 con 0x7f7880005180 2023-08-09T15:12:23.997 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:23.988+0000 7f78927fc640 1 -- 172.21.15.43:0/1643491044 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f788000c1f0 con 0x7f78800047c0 2023-08-09T15:12:23.997 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:23.988+0000 7f78927fc640 1 -- 172.21.15.43:0/1643491044 --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f788000c3b0 con 0x7f78800039e0 2023-08-09T15:12:23.997 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:23.988+0000 7f78937fe640 1 --2- 172.21.15.43:0/1643491044 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f7880005180 0x7f788000b980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T15:12:23.998 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:23.988+0000 7f788a7fc640 1 --2- 172.21.15.43:0/1643491044 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f78800047c0 0x7f7880004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T15:12:23.998 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:23.988+0000 7f7892ffd640 1 --2- 172.21.15.43:0/1643491044 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f78800039e0 0x7f7880003e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T15:12:23.998 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:23.989+0000 7f78937fe640 1 -- 172.21.15.43:0/1643491044 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f78800039e0 msgr2=0x7f7880003e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-09T15:12:23.998 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:23.989+0000 7f78937fe640 1 --2- 172.21.15.43:0/1643491044 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f78800039e0 0x7f7880003e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T15:12:23.999 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:23.989+0000 7f78937fe640 1 -- 172.21.15.43:0/1643491044 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f78800047c0 msgr2=0x7f7880004c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-09T15:12:23.999 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:23.989+0000 7f78937fe640 1 --2- 172.21.15.43:0/1643491044 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f78800047c0 0x7f7880004c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T15:12:23.999 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:23.989+0000 7f78937fe640 1 -- 172.21.15.43:0/1643491044 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7880003520 con 0x7f7880005180 2023-08-09T15:12:23.999 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:23.989+0000 7f788a7fc640 1 --2- 172.21.15.43:0/1643491044 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f78800047c0 0x7f7880004c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-08-09T15:12:24.000 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:23.989+0000 7f7892ffd640 1 --2- 172.21.15.43:0/1643491044 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f78800039e0 0x7f7880003e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-08-09T15:12:24.000 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:23.990+0000 7f78937fe640 1 -- 172.21.15.43:0/1643491044 reap_dead start 2023-08-09T15:12:24.000 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:23.990+0000 7f78937fe640 1 --2- 172.21.15.43:0/1643491044 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f7880005180 0x7f788000b980 secure :-1 s=READY pgs=2581 cs=0 l=1 rev1=1 crypto rx=0x7f788c001720 tx=0x7f78800057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c806a2c9373b46c4 server_cookie=0 in_seq=0 out_seq=0 2023-08-09T15:12:24.001 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:23.990+0000 7f78927fc640 1 -- 172.21.15.43:0/1643491044 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f788c0052d0 con 0x7f7880005180 2023-08-09T15:12:24.001 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:23.990+0000 7f78927fc640 1 -- 172.21.15.43:0/1643491044 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f788c02e050 con 0x7f7880005180 2023-08-09T15:12:24.001 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:23.990+0000 7f78927fc640 1 -- 172.21.15.43:0/1643491044 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f788c0035f0 con 0x7f7880005180 2023-08-09T15:12:24.001 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:23.991+0000 7f7893fff640 1 -- 172.21.15.43:0/1643491044 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f7880005180 msgr2=0x7f788000b980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T15:12:24.002 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:23.991+0000 7f7893fff640 1 --2- 172.21.15.43:0/1643491044 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f7880005180 0x7f788000b980 secure :-1 s=READY pgs=2581 cs=0 l=1 rev1=1 crypto rx=0x7f788c001720 tx=0x7f78800057f0 comp rx=0 tx=0).stop 2023-08-09T15:12:24.002 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:23.991+0000 7f7893fff640 1 -- 172.21.15.43:0/1643491044 shutdown_connections 2023-08-09T15:12:24.002 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:23.991+0000 7f7893fff640 1 --2- 172.21.15.43:0/1643491044 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f7880005180 0x7f788000b980 unknown :-1 s=CLOSED pgs=2581 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T15:12:24.002 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:23.991+0000 7f7893fff640 1 -- 172.21.15.43:0/1643491044 >> 172.21.15.43:0/1643491044 conn(0x7f78940714a0 msgr2=0x7f78940d2500 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-09T15:12:24.003 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:23.991+0000 7f7893fff640 1 -- 172.21.15.43:0/1643491044 shutdown_connections 2023-08-09T15:12:24.003 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:23.991+0000 7f7893fff640 1 -- 172.21.15.43:0/1643491044 wait complete. 2023-08-09T15:12:24.005 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:23.993+0000 7f7893fff640 1 Processor -- start 2023-08-09T15:12:24.006 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:23.993+0000 7f7893fff640 1 -- start start 2023-08-09T15:12:24.006 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:23.994+0000 7f7893fff640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f7880005180 0x7f7894151850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T15:12:24.006 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:23.994+0000 7f7893fff640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f7894154200 0x7f78941509f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T15:12:24.006 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:23.994+0000 7f7893fff640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f7894157170 0x7f7894157600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T15:12:24.006 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:23.994+0000 7f7893fff640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7894151d90 con 0x7f7880005180 2023-08-09T15:12:24.006 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:23.994+0000 7f7893fff640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7894150f30 con 0x7f7894157170 2023-08-09T15:12:24.006 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:23.994+0000 7f7893fff640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7894153fc0 con 0x7f7894154200 2023-08-09T15:12:24.007 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:23.994+0000 7f7892ffd640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f7880005180 0x7f7894151850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T15:12:24.007 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:23.994+0000 7f7892ffd640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f7880005180 0x7f7894151850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:46398/0 (socket says 172.21.15.43:46398) 2023-08-09T15:12:24.007 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:23.994+0000 7f7892ffd640 1 -- 172.21.15.43:0/55455826 learned_addr learned my addr 172.21.15.43:0/55455826 (peer_addr_for_me v2:172.21.15.43:0/0) 2023-08-09T15:12:24.007 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:23.994+0000 7f788a7fc640 1 --2- 172.21.15.43:0/55455826 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f7894154200 0x7f78941509f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T15:12:24.007 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:23.995+0000 7f78937fe640 1 --2- 172.21.15.43:0/55455826 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f7894157170 0x7f7894157600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T15:12:24.007 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:23.995+0000 7f7892ffd640 1 -- 172.21.15.43:0/55455826 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f7894154200 msgr2=0x7f78941509f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T15:12:24.008 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:23.995+0000 7f7892ffd640 1 --2- 172.21.15.43:0/55455826 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f7894154200 0x7f78941509f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T15:12:24.008 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:23.995+0000 7f7892ffd640 1 -- 172.21.15.43:0/55455826 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f7894157170 msgr2=0x7f7894157600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T15:12:24.008 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:23.995+0000 7f7892ffd640 1 --2- 172.21.15.43:0/55455826 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f7894157170 0x7f7894157600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T15:12:24.008 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:23.995+0000 7f7892ffd640 1 -- 172.21.15.43:0/55455826 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7894154770 con 0x7f7880005180 2023-08-09T15:12:24.008 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:23.995+0000 7f788a7fc640 1 --2- 172.21.15.43:0/55455826 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f7894154200 0x7f78941509f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-08-09T15:12:24.008 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:23.995+0000 7f7892ffd640 1 --2- 172.21.15.43:0/55455826 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f7880005180 0x7f7894151850 secure :-1 s=READY pgs=2582 cs=0 l=1 rev1=1 crypto rx=0x7f788400f350 tx=0x7f788400f810 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-09T15:12:24.008 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:23.995+0000 7f7890ff9640 1 -- 172.21.15.43:0/55455826 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7884003b90 con 0x7f7880005180 2023-08-09T15:12:24.009 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:23.995+0000 7f7890ff9640 1 -- 172.21.15.43:0/55455826 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7884011040 con 0x7f7880005180 2023-08-09T15:12:24.009 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:23.995+0000 7f7890ff9640 1 -- 172.21.15.43:0/55455826 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7884004020 con 0x7f7880005180 2023-08-09T15:12:24.009 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:23.995+0000 7f7893fff640 1 -- 172.21.15.43:0/55455826 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f78941563b0 con 0x7f7880005180 2023-08-09T15:12:24.009 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:23.995+0000 7f7893fff640 1 -- 172.21.15.43:0/55455826 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7894156970 con 0x7f7880005180 2023-08-09T15:12:24.009 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:23.996+0000 7f7893fff640 1 -- 172.21.15.43:0/55455826 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f78940cd700 con 0x7f7880005180 2023-08-09T15:12:24.009 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:23.998+0000 7f7890ff9640 1 -- 172.21.15.43:0/55455826 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 129) v1 ==== 88443+0+0 (secure 0 0 0) 0x7f7884004180 con 0x7f7880005180 2023-08-09T15:12:24.009 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:23.998+0000 7f7890ff9640 1 --2- 172.21.15.43:0/55455826 >> [v2:172.21.15.203:6832/1456699667,v1:172.21.15.203:6833/1456699667] conn(0x7f787406a1a0 0x7f787406c660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T15:12:24.009 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:23.999+0000 7f7890ff9640 1 -- 172.21.15.43:0/55455826 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(229..229 src has 1..229) v4 ==== 6462+0+0 (secure 0 0 0) 0x7f7884089940 con 0x7f7880005180 2023-08-09T15:12:24.010 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:23.999+0000 7f7890ff9640 1 --2- 172.21.15.43:0/55455826 >> [v2:172.21.15.43:6813/1709538759,v1:172.21.15.43:6816/1709538759] conn(0x7f7874071180 0x7f78740735e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T15:12:24.010 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:23.999+0000 7f7890ff9640 1 -- 172.21.15.43:0/55455826 --> [v2:172.21.15.43:6813/1709538759,v1:172.21.15.43:6816/1709538759] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f7874073cb0 con 0x7f7874071180 2023-08-09T15:12:24.010 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:23.999+0000 7f7890ff9640 1 -- 172.21.15.43:0/55455826 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_get_version_reply(handle=1 version=229) v2 ==== 24+0+0 (secure 0 0 0) 0x7f7884089cb0 con 0x7f7880005180 2023-08-09T15:12:24.010 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:23.999+0000 7f788a7fc640 1 --2- 172.21.15.43:0/55455826 >> [v2:172.21.15.203:6832/1456699667,v1:172.21.15.203:6833/1456699667] conn(0x7f787406a1a0 0x7f787406c660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T15:12:24.010 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:23.999+0000 7f78937fe640 1 --2- 172.21.15.43:0/55455826 >> [v2:172.21.15.43:6813/1709538759,v1:172.21.15.43:6816/1709538759] conn(0x7f7874071180 0x7f78740735e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T15:12:24.010 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:24.000+0000 7f78937fe640 1 --2- 172.21.15.43:0/55455826 >> [v2:172.21.15.43:6813/1709538759,v1:172.21.15.43:6816/1709538759] conn(0x7f7874071180 0x7f78740735e0 crc :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-09T15:12:24.010 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:24.000+0000 7f788a7fc640 1 --2- 172.21.15.43:0/55455826 >> [v2:172.21.15.203:6832/1456699667,v1:172.21.15.203:6833/1456699667] conn(0x7f787406a1a0 0x7f787406c660 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7f787c001270 tx=0x7f787c009040 comp rx=0 tx=0).ready entity=mgr.9232 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-09T15:12:24.011 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:24.001+0000 7f7890ff9640 1 -- 172.21.15.43:0/55455826 <== osd.3 v2:172.21.15.43:6813/1709538759 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+25607 (crc 0 0 0) 0x7f7874073cb0 con 0x7f7874071180 2023-08-09T15:12:24.016 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:24.015+0000 7f7893fff640 1 -- 172.21.15.43:0/55455826 --> [v2:172.21.15.43:6813/1709538759,v1:172.21.15.43:6816/1709538759] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) v1 -- 0x7f78940d10f0 con 0x7f7874071180 2023-08-09T15:12:24.017 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:24.016+0000 7f7890ff9640 1 -- 172.21.15.43:0/55455826 <== osd.3 v2:172.21.15.43:6813/1709538759 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+105 (crc 0 0 0) 0x7f788c005250 con 0x7f7874071180 2023-08-09T15:12:24.017 INFO:teuthology.orchestra.run.smithi043.stdout:{ 2023-08-09T15:12:24.018 INFO:teuthology.orchestra.run.smithi043.stdout: "success": "osd_debug_deep_scrub_sleep = '0.000000' (not observed, change may require restart) " 2023-08-09T15:12:24.018 INFO:teuthology.orchestra.run.smithi043.stdout:} 2023-08-09T15:12:24.019 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:24.016+0000 7f7893fff640 1 -- 172.21.15.43:0/55455826 >> [v2:172.21.15.43:6813/1709538759,v1:172.21.15.43:6816/1709538759] conn(0x7f7874071180 msgr2=0x7f78740735e0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T15:12:24.019 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:24.016+0000 7f7893fff640 1 --2- 172.21.15.43:0/55455826 >> [v2:172.21.15.43:6813/1709538759,v1:172.21.15.43:6816/1709538759] conn(0x7f7874071180 0x7f78740735e0 crc :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T15:12:24.019 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:24.017+0000 7f7893fff640 1 -- 172.21.15.43:0/55455826 >> [v2:172.21.15.203:6832/1456699667,v1:172.21.15.203:6833/1456699667] conn(0x7f787406a1a0 msgr2=0x7f787406c660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T15:12:24.019 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:24.017+0000 7f7893fff640 1 --2- 172.21.15.43:0/55455826 >> [v2:172.21.15.203:6832/1456699667,v1:172.21.15.203:6833/1456699667] conn(0x7f787406a1a0 0x7f787406c660 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7f787c001270 tx=0x7f787c009040 comp rx=0 tx=0).stop 2023-08-09T15:12:24.019 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:24.017+0000 7f7893fff640 1 -- 172.21.15.43:0/55455826 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f7880005180 msgr2=0x7f7894151850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T15:12:24.019 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:24.017+0000 7f7893fff640 1 --2- 172.21.15.43:0/55455826 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f7880005180 0x7f7894151850 secure :-1 s=READY pgs=2582 cs=0 l=1 rev1=1 crypto rx=0x7f788400f350 tx=0x7f788400f810 comp rx=0 tx=0).stop 2023-08-09T15:12:24.019 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:24.017+0000 7f78937fe640 1 -- 172.21.15.43:0/55455826 reap_dead start 2023-08-09T15:12:24.020 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:24.017+0000 7f7893fff640 1 -- 172.21.15.43:0/55455826 shutdown_connections 2023-08-09T15:12:24.020 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:24.017+0000 7f7893fff640 1 -- 172.21.15.43:0/55455826 >> 172.21.15.43:0/55455826 conn(0x7f78940714a0 msgr2=0x7f789415a3d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-09T15:12:24.020 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:24.017+0000 7f7893fff640 1 -- 172.21.15.43:0/55455826 shutdown_connections 2023-08-09T15:12:24.020 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:24.017+0000 7f7893fff640 1 -- 172.21.15.43:0/55455826 wait complete. 2023-08-09T15:12:24.038 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 3 2023-08-09T15:12:24.272 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:24.271+0000 7f5c45e8c640 1 Processor -- start 2023-08-09T15:12:24.272 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:24.271+0000 7f5c45e8c640 1 -- start start 2023-08-09T15:12:24.274 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:24.272+0000 7f5c45e8c640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f5c40159ec0 0x7f5c40071110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T15:12:24.274 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:24.272+0000 7f5c45e8c640 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7f5c40070570 0x7f5c4015a2a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T15:12:24.274 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:24.272+0000 7f5c45e8c640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f5c4015b300 0x7f5c4006b030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T15:12:24.274 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:24.272+0000 7f5c45e8c640 1 -- --> v2:172.21.15.43:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5c400717a0 con 0x7f5c40159ec0 2023-08-09T15:12:24.274 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:24.272+0000 7f5c45e8c640 1 -- --> v2:172.21.15.203:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5c4006f7b0 con 0x7f5c40070570 2023-08-09T15:12:24.274 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:24.272+0000 7f5c45e8c640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5c4006fa90 con 0x7f5c4015b300 2023-08-09T15:12:24.275 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:24.273+0000 7f5c44e8a640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f5c40159ec0 0x7f5c40071110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T15:12:24.276 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:24.273+0000 7f5c44e8a640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f5c40159ec0 0x7f5c40071110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:46406/0 (socket says 172.21.15.43:46406) 2023-08-09T15:12:24.276 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:24.273+0000 7f5c44e8a640 1 -- 172.21.15.43:0/1148122756 learned_addr learned my addr 172.21.15.43:0/1148122756 (peer_addr_for_me v2:172.21.15.43:0/0) 2023-08-09T15:12:24.276 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:24.273+0000 7f5c4568b640 1 --2- 172.21.15.43:0/1148122756 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f5c4015b300 0x7f5c4006b030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T15:12:24.276 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:24.273+0000 7f5c3ffff640 1 --2- 172.21.15.43:0/1148122756 >> v2:172.21.15.203:3300/0 conn(0x7f5c40070570 0x7f5c4015a2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T15:12:24.276 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:24.274+0000 7f5c44e8a640 1 -- 172.21.15.43:0/1148122756 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f5c4015b300 msgr2=0x7f5c4006b030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-09T15:12:24.276 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:24.274+0000 7f5c44e8a640 1 --2- 172.21.15.43:0/1148122756 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f5c4015b300 0x7f5c4006b030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T15:12:24.276 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:24.274+0000 7f5c44e8a640 1 -- 172.21.15.43:0/1148122756 >> v2:172.21.15.203:3300/0 conn(0x7f5c40070570 msgr2=0x7f5c4015a2a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-09T15:12:24.276 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:24.274+0000 7f5c44e8a640 1 --2- 172.21.15.43:0/1148122756 >> v2:172.21.15.203:3300/0 conn(0x7f5c40070570 0x7f5c4015a2a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T15:12:24.277 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:24.274+0000 7f5c44e8a640 1 -- 172.21.15.43:0/1148122756 --> v2:172.21.15.43:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5c4006fd70 con 0x7f5c40159ec0 2023-08-09T15:12:24.277 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:24.275+0000 7f5c44e8a640 1 --2- 172.21.15.43:0/1148122756 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f5c40159ec0 0x7f5c40071110 secure :-1 s=READY pgs=2583 cs=0 l=1 rev1=1 crypto rx=0x7f5c40066d60 tx=0x7f5c3402b910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6a71975fe54d972e server_cookie=0 in_seq=0 out_seq=0 2023-08-09T15:12:24.278 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:24.275+0000 7f5c3f7fe640 1 -- 172.21.15.43:0/1148122756 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5c34031870 con 0x7f5c40159ec0 2023-08-09T15:12:24.278 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:24.275+0000 7f5c3f7fe640 1 -- 172.21.15.43:0/1148122756 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5c340319d0 con 0x7f5c40159ec0 2023-08-09T15:12:24.278 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:24.275+0000 7f5c3f7fe640 1 -- 172.21.15.43:0/1148122756 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5c34031d60 con 0x7f5c40159ec0 2023-08-09T15:12:24.278 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:24.276+0000 7f5c45e8c640 1 -- 172.21.15.43:0/1148122756 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f5c40159ec0 msgr2=0x7f5c40071110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T15:12:24.278 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:24.276+0000 7f5c45e8c640 1 --2- 172.21.15.43:0/1148122756 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f5c40159ec0 0x7f5c40071110 secure :-1 s=READY pgs=2583 cs=0 l=1 rev1=1 crypto rx=0x7f5c40066d60 tx=0x7f5c3402b910 comp rx=0 tx=0).stop 2023-08-09T15:12:24.278 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:24.276+0000 7f5c45e8c640 1 -- 172.21.15.43:0/1148122756 shutdown_connections 2023-08-09T15:12:24.279 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:24.276+0000 7f5c45e8c640 1 --2- 172.21.15.43:0/1148122756 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f5c4015b300 0x7f5c4006b030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T15:12:24.279 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:24.276+0000 7f5c45e8c640 1 --2- 172.21.15.43:0/1148122756 >> v2:172.21.15.203:3300/0 conn(0x7f5c40070570 0x7f5c4015a2a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T15:12:24.279 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:24.276+0000 7f5c45e8c640 1 --2- 172.21.15.43:0/1148122756 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f5c40159ec0 0x7f5c40071110 unknown :-1 s=CLOSED pgs=2583 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T15:12:24.279 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:24.276+0000 7f5c45e8c640 1 -- 172.21.15.43:0/1148122756 >> 172.21.15.43:0/1148122756 conn(0x7f5c400ce020 msgr2=0x7f5c400ce430 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-09T15:12:24.279 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:24.276+0000 7f5c45e8c640 1 -- 172.21.15.43:0/1148122756 shutdown_connections 2023-08-09T15:12:24.279 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:24.276+0000 7f5c45e8c640 1 -- 172.21.15.43:0/1148122756 wait complete. 2023-08-09T15:12:24.280 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:24.278+0000 7f5c45e8c640 1 Processor -- start 2023-08-09T15:12:24.280 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:24.278+0000 7f5c45e8c640 1 -- start start 2023-08-09T15:12:24.281 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:24.279+0000 7f5c45e8c640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f5c40070570 0x7f5c4014c2e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T15:12:24.281 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:24.279+0000 7f5c45e8c640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f5c40159ec0 0x7f5c400c66e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T15:12:24.281 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:24.279+0000 7f5c45e8c640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f5c4015b300 0x7f5c400c6c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T15:12:24.281 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:24.279+0000 7f5c45e8c640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5c40098da0 con 0x7f5c4015b300 2023-08-09T15:12:24.282 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:24.279+0000 7f5c45e8c640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5c40099080 con 0x7f5c40159ec0 2023-08-09T15:12:24.282 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:24.279+0000 7f5c45e8c640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5c40099360 con 0x7f5c40070570 2023-08-09T15:12:24.283 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:24.280+0000 7f5c4568b640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f5c4015b300 0x7f5c400c6c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T15:12:24.283 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:24.280+0000 7f5c3ffff640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f5c40159ec0 0x7f5c400c66e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T15:12:24.283 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:24.280+0000 7f5c4568b640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f5c4015b300 0x7f5c400c6c20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:46416/0 (socket says 172.21.15.43:46416) 2023-08-09T15:12:24.284 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:24.280+0000 7f5c4568b640 1 -- 172.21.15.43:0/1742807727 learned_addr learned my addr 172.21.15.43:0/1742807727 (peer_addr_for_me v2:172.21.15.43:0/0) 2023-08-09T15:12:24.284 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:24.280+0000 7f5c44e8a640 1 --2- 172.21.15.43:0/1742807727 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f5c40070570 0x7f5c4014c2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T15:12:24.284 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:24.280+0000 7f5c4568b640 1 -- 172.21.15.43:0/1742807727 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f5c40070570 msgr2=0x7f5c4014c2e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T15:12:24.284 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:24.280+0000 7f5c4568b640 1 --2- 172.21.15.43:0/1742807727 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f5c40070570 0x7f5c4014c2e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T15:12:24.284 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:24.280+0000 7f5c4568b640 1 -- 172.21.15.43:0/1742807727 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f5c40159ec0 msgr2=0x7f5c400c66e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T15:12:24.284 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:24.280+0000 7f5c4568b640 1 --2- 172.21.15.43:0/1742807727 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f5c40159ec0 0x7f5c400c66e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T15:12:24.284 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:24.280+0000 7f5c4568b640 1 -- 172.21.15.43:0/1742807727 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5c340094d0 con 0x7f5c4015b300 2023-08-09T15:12:24.285 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:24.281+0000 7f5c4568b640 1 --2- 172.21.15.43:0/1742807727 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f5c4015b300 0x7f5c400c6c20 secure :-1 s=READY pgs=2584 cs=0 l=1 rev1=1 crypto rx=0x7f5c38007e40 tx=0x7f5c3800e4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-09T15:12:24.285 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:24.281+0000 7f5c3d7fa640 1 -- 172.21.15.43:0/1742807727 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5c38014620 con 0x7f5c4015b300 2023-08-09T15:12:24.285 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:24.281+0000 7f5c3d7fa640 1 -- 172.21.15.43:0/1742807727 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5c3800ee90 con 0x7f5c4015b300 2023-08-09T15:12:24.285 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:24.281+0000 7f5c45e8c640 1 -- 172.21.15.43:0/1742807727 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5c40099640 con 0x7f5c4015b300 2023-08-09T15:12:24.285 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:24.281+0000 7f5c3d7fa640 1 -- 172.21.15.43:0/1742807727 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5c380149e0 con 0x7f5c4015b300 2023-08-09T15:12:24.285 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:24.281+0000 7f5c45e8c640 1 -- 172.21.15.43:0/1742807727 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5c40159590 con 0x7f5c4015b300 2023-08-09T15:12:24.286 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:24.282+0000 7f5c45e8c640 1 -- 172.21.15.43:0/1742807727 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5c40099b60 con 0x7f5c4015b300 2023-08-09T15:12:24.286 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:24.283+0000 7f5c3d7fa640 1 -- 172.21.15.43:0/1742807727 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 129) v1 ==== 88443+0+0 (secure 0 0 0) 0x7f5c38014b40 con 0x7f5c4015b300 2023-08-09T15:12:24.286 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:24.285+0000 7f5c3d7fa640 1 --2- 172.21.15.43:0/1742807727 >> [v2:172.21.15.203:6832/1456699667,v1:172.21.15.203:6833/1456699667] conn(0x7f5c2406a170 0x7f5c2406c630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T15:12:24.286 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:24.285+0000 7f5c3d7fa640 1 -- 172.21.15.43:0/1742807727 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(229..229 src has 1..229) v4 ==== 6462+0+0 (secure 0 0 0) 0x7f5c38089120 con 0x7f5c4015b300 2023-08-09T15:12:24.287 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:24.285+0000 7f5c44e8a640 1 --2- 172.21.15.43:0/1742807727 >> [v2:172.21.15.203:6832/1456699667,v1:172.21.15.203:6833/1456699667] conn(0x7f5c2406a170 0x7f5c2406c630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T15:12:24.288 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:24.287+0000 7f5c44e8a640 1 --2- 172.21.15.43:0/1742807727 >> [v2:172.21.15.203:6832/1456699667,v1:172.21.15.203:6833/1456699667] conn(0x7f5c2406a170 0x7f5c2406c630 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7f5c34036000 tx=0x7f5c3403c040 comp rx=0 tx=0).ready entity=mgr.9232 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-09T15:12:24.291 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:24.289+0000 7f5c3d7fa640 1 -- 172.21.15.43:0/1742807727 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178066 (secure 0 0 0) 0x7f5c40099b60 con 0x7f5c4015b300 2023-08-09T15:12:24.487 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:24.485+0000 7f5c45e8c640 1 -- 172.21.15.43:0/1742807727 --> [v2:172.21.15.203:6832/1456699667,v1:172.21.15.203:6833/1456699667] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "3", "target": ["mon-mgr", ""]}) v1 -- 0x7f5c4015b790 con 0x7f5c2406a170 2023-08-09T15:12:24.488 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:24.486+0000 7f5c3d7fa640 1 -- 172.21.15.43:0/1742807727 <== mgr.9232 v2:172.21.15.203:6832/1456699667 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 3 to deep-scrub) v1 ==== 41+0+0 (secure 0 0 0) 0x7f5c34031bc0 con 0x7f5c2406a170 2023-08-09T15:12:24.488 INFO:teuthology.orchestra.run.smithi043.stderr:instructed osd(s) 3 to deep-scrub 2023-08-09T15:12:24.490 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:24.489+0000 7f5c45e8c640 1 -- 172.21.15.43:0/1742807727 >> [v2:172.21.15.203:6832/1456699667,v1:172.21.15.203:6833/1456699667] conn(0x7f5c2406a170 msgr2=0x7f5c2406c630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T15:12:24.491 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:24.489+0000 7f5c45e8c640 1 --2- 172.21.15.43:0/1742807727 >> [v2:172.21.15.203:6832/1456699667,v1:172.21.15.203:6833/1456699667] conn(0x7f5c2406a170 0x7f5c2406c630 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7f5c34036000 tx=0x7f5c3403c040 comp rx=0 tx=0).stop 2023-08-09T15:12:24.491 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:24.489+0000 7f5c45e8c640 1 -- 172.21.15.43:0/1742807727 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f5c4015b300 msgr2=0x7f5c400c6c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T15:12:24.491 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:24.489+0000 7f5c45e8c640 1 --2- 172.21.15.43:0/1742807727 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f5c4015b300 0x7f5c400c6c20 secure :-1 s=READY pgs=2584 cs=0 l=1 rev1=1 crypto rx=0x7f5c38007e40 tx=0x7f5c3800e4b0 comp rx=0 tx=0).stop 2023-08-09T15:12:24.492 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:24.489+0000 7f5c45e8c640 1 -- 172.21.15.43:0/1742807727 shutdown_connections 2023-08-09T15:12:24.492 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:24.489+0000 7f5c45e8c640 1 --2- 172.21.15.43:0/1742807727 >> [v2:172.21.15.203:6832/1456699667,v1:172.21.15.203:6833/1456699667] conn(0x7f5c2406a170 0x7f5c2406c630 unknown :-1 s=CLOSED pgs=50 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T15:12:24.492 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:24.489+0000 7f5c45e8c640 1 --2- 172.21.15.43:0/1742807727 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f5c4015b300 0x7f5c400c6c20 unknown :-1 s=CLOSED pgs=2584 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T15:12:24.492 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:24.489+0000 7f5c45e8c640 1 --2- 172.21.15.43:0/1742807727 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f5c40159ec0 0x7f5c400c66e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T15:12:24.492 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:24.489+0000 7f5c45e8c640 1 --2- 172.21.15.43:0/1742807727 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f5c40070570 0x7f5c4014c2e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T15:12:24.492 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:24.489+0000 7f5c45e8c640 1 -- 172.21.15.43:0/1742807727 >> 172.21.15.43:0/1742807727 conn(0x7f5c400ce020 msgr2=0x7f5c400b0c10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-09T15:12:24.492 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:24.489+0000 7f5c45e8c640 1 -- 172.21.15.43:0/1742807727 shutdown_connections 2023-08-09T15:12:24.493 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:24.489+0000 7f5c45e8c640 1 -- 172.21.15.43:0/1742807727 wait complete. 2023-08-09T15:12:24.511 INFO:tasks.ceph:Scrubbing osd.4 2023-08-09T15:12:24.511 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.4 config set osd_debug_deep_scrub_sleep 0 2023-08-09T15:12:24.747 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:24.745+0000 7fe986b64640 1 Processor -- start 2023-08-09T15:12:24.747 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:24.745+0000 7fe986b64640 1 -- start start 2023-08-09T15:12:24.748 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:24.747+0000 7fe986b64640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fe98015a650 0x7fe98015aa30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T15:12:24.749 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:24.747+0000 7fe986b64640 1 -- --> v1:172.21.15.43:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe9800c8c00 con 0x7fe9800ca580 2023-08-09T15:12:24.749 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:24.747+0000 7fe986b64640 1 -- --> v1:172.21.15.203:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe98015b2d0 con 0x7fe9800c9350 2023-08-09T15:12:24.749 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:24.747+0000 7fe986b64640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe98015b5b0 con 0x7fe98015a650 2023-08-09T15:12:24.750 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:24.748+0000 7fe97ffff640 1 --1- >> v1:172.21.15.43:6789/0 conn(0x7fe9800ca580 0x7fe98015bc40 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.43:6789/0 says I am v1:172.21.15.43:32874/0 (socket says 172.21.15.43:32874) 2023-08-09T15:12:24.750 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:24.748+0000 7fe97ffff640 1 -- 172.21.15.43:0/3495885881 learned_addr learned my addr 172.21.15.43:0/3495885881 (peer_addr_for_me v1:172.21.15.43:0/0) 2023-08-09T15:12:24.750 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:24.748+0000 7fe984900640 1 --2- 172.21.15.43:0/3495885881 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fe98015a650 0x7fe98015aa30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T15:12:24.750 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:24.748+0000 7fe97f7fe640 1 -- 172.21.15.43:0/3495885881 <== mon.0 v1:172.21.15.43:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1891656707 0 0) 0x7fe9800c8c00 con 0x7fe9800ca580 2023-08-09T15:12:24.750 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:24.748+0000 7fe97f7fe640 1 -- 172.21.15.43:0/3495885881 --> v1:172.21.15.43:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fe970003520 con 0x7fe9800ca580 2023-08-09T15:12:24.751 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:24.749+0000 7fe97f7fe640 1 -- 172.21.15.43:0/3495885881 <== mon.0 v1:172.21.15.43:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 633273112 0 0) 0x7fe970003520 con 0x7fe9800ca580 2023-08-09T15:12:24.751 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:24.749+0000 7fe984900640 1 -- 172.21.15.43:0/3495885881 >> v1:172.21.15.203:6789/0 conn(0x7fe9800c9350 legacy=0x7fe9800cac30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T15:12:24.751 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:24.749+0000 7fe984900640 1 -- 172.21.15.43:0/3495885881 >> v1:172.21.15.43:6789/0 conn(0x7fe9800ca580 legacy=0x7fe98015bc40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T15:12:24.751 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:24.749+0000 7fe984900640 1 -- 172.21.15.43:0/3495885881 --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe98015b890 con 0x7fe98015a650 2023-08-09T15:12:24.751 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:24.750+0000 7fe984900640 1 --2- 172.21.15.43:0/3495885881 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fe98015a650 0x7fe98015aa30 secure :-1 s=READY pgs=361 cs=0 l=1 rev1=1 crypto rx=0x7fe9740097d0 tx=0x7fe97402b770 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a81f4a621c1e5ff8 server_cookie=0 in_seq=0 out_seq=0 2023-08-09T15:12:24.752 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:24.751+0000 7fe97f7fe640 1 -- 172.21.15.43:0/3495885881 <== mon.2 v2:172.21.15.203:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe96c0017f0 con 0x7fe98015a650 2023-08-09T15:12:24.752 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:24.751+0000 7fe97f7fe640 1 -- 172.21.15.43:0/3495885881 <== mon.2 v2:172.21.15.203:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe97402e050 con 0x7fe98015a650 2023-08-09T15:12:24.752 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:24.751+0000 7fe97f7fe640 1 -- 172.21.15.43:0/3495885881 <== mon.2 v2:172.21.15.203:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe974030730 con 0x7fe98015a650 2023-08-09T15:12:24.753 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:24.751+0000 7fe986b64640 1 -- 172.21.15.43:0/3495885881 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fe98015a650 msgr2=0x7fe98015aa30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T15:12:24.753 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:24.751+0000 7fe986b64640 1 --2- 172.21.15.43:0/3495885881 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fe98015a650 0x7fe98015aa30 secure :-1 s=READY pgs=361 cs=0 l=1 rev1=1 crypto rx=0x7fe9740097d0 tx=0x7fe97402b770 comp rx=0 tx=0).stop 2023-08-09T15:12:24.753 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:24.751+0000 7fe986b64640 1 -- 172.21.15.43:0/3495885881 shutdown_connections 2023-08-09T15:12:24.753 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:24.751+0000 7fe986b64640 1 --2- 172.21.15.43:0/3495885881 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fe98015a650 0x7fe98015aa30 unknown :-1 s=CLOSED pgs=361 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T15:12:24.754 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:24.751+0000 7fe986b64640 1 -- 172.21.15.43:0/3495885881 >> 172.21.15.43:0/3495885881 conn(0x7fe9800afdb0 msgr2=0x7fe9800b0190 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-09T15:12:24.754 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:24.752+0000 7fe986b64640 1 -- 172.21.15.43:0/3495885881 shutdown_connections 2023-08-09T15:12:24.754 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:24.752+0000 7fe986b64640 1 -- 172.21.15.43:0/3495885881 wait complete. 2023-08-09T15:12:24.754 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:24.753+0000 7fe986b64640 1 Processor -- start 2023-08-09T15:12:24.755 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:24.753+0000 7fe986b64640 1 -- start start 2023-08-09T15:12:24.756 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:24.754+0000 7fe986b64640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fe9800c9350 0x7fe9801f15a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T15:12:24.756 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:24.754+0000 7fe986b64640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fe9800ca580 0x7fe9800cc7d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T15:12:24.756 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:24.754+0000 7fe986b64640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fe98015a650 0x7fe9800b2770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T15:12:24.756 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:24.754+0000 7fe986b64640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe9800abd90 con 0x7fe9800c9350 2023-08-09T15:12:24.756 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:24.754+0000 7fe986b64640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe9801f1ae0 con 0x7fe98015a650 2023-08-09T15:12:24.757 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:24.754+0000 7fe986b64640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe9800aba90 con 0x7fe9800ca580 2023-08-09T15:12:24.757 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:24.755+0000 7fe984900640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fe9800c9350 0x7fe9801f15a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T15:12:24.757 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:24.755+0000 7fe984900640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fe9800c9350 0x7fe9801f15a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:46430/0 (socket says 172.21.15.43:46430) 2023-08-09T15:12:24.757 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:24.755+0000 7fe984900640 1 -- 172.21.15.43:0/2059140356 learned_addr learned my addr 172.21.15.43:0/2059140356 (peer_addr_for_me v2:172.21.15.43:0/0) 2023-08-09T15:12:24.758 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:24.755+0000 7fe984900640 1 -- 172.21.15.43:0/2059140356 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fe9800ca580 msgr2=0x7fe9800cc7d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T15:12:24.758 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:24.755+0000 7fe984900640 1 --2- 172.21.15.43:0/2059140356 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fe9800ca580 0x7fe9800cc7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T15:12:24.758 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:24.755+0000 7fe984900640 1 -- 172.21.15.43:0/2059140356 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fe98015a650 msgr2=0x7fe9800b2770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T15:12:24.758 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:24.755+0000 7fe984900640 1 --2- 172.21.15.43:0/2059140356 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fe98015a650 0x7fe9800b2770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T15:12:24.759 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:24.755+0000 7fe984900640 1 -- 172.21.15.43:0/2059140356 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe974009400 con 0x7fe9800c9350 2023-08-09T15:12:24.759 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:24.756+0000 7fe984900640 1 --2- 172.21.15.43:0/2059140356 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fe9800c9350 0x7fe9801f15a0 secure :-1 s=READY pgs=2586 cs=0 l=1 rev1=1 crypto rx=0x7fe974035040 tx=0x7fe9740046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-09T15:12:24.759 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:24.756+0000 7fe97d7fa640 1 -- 172.21.15.43:0/2059140356 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe974004230 con 0x7fe9800c9350 2023-08-09T15:12:24.759 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:24.756+0000 7fe97d7fa640 1 -- 172.21.15.43:0/2059140356 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe9740043d0 con 0x7fe9800c9350 2023-08-09T15:12:24.759 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:24.756+0000 7fe986b64640 1 -- 172.21.15.43:0/2059140356 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe9800cd0c0 con 0x7fe9800c9350 2023-08-09T15:12:24.759 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:24.756+0000 7fe97d7fa640 1 -- 172.21.15.43:0/2059140356 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe97403c860 con 0x7fe9800c9350 2023-08-09T15:12:24.759 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:24.756+0000 7fe986b64640 1 -- 172.21.15.43:0/2059140356 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe9800cd680 con 0x7fe9800c9350 2023-08-09T15:12:24.760 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:24.757+0000 7fe986b64640 1 -- 172.21.15.43:0/2059140356 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7fe98015af70 con 0x7fe9800c9350 2023-08-09T15:12:24.760 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:24.758+0000 7fe97d7fa640 1 -- 172.21.15.43:0/2059140356 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 129) v1 ==== 88443+0+0 (secure 0 0 0) 0x7fe974039050 con 0x7fe9800c9350 2023-08-09T15:12:24.761 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:24.759+0000 7fe97d7fa640 1 --2- 172.21.15.43:0/2059140356 >> [v2:172.21.15.203:6832/1456699667,v1:172.21.15.203:6833/1456699667] conn(0x7fe96406a170 0x7fe96406c630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T15:12:24.761 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:24.760+0000 7fe97d7fa640 1 -- 172.21.15.43:0/2059140356 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(229..229 src has 1..229) v4 ==== 6462+0+0 (secure 0 0 0) 0x7fe9740aa1f0 con 0x7fe9800c9350 2023-08-09T15:12:24.761 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:24.760+0000 7fe97d7fa640 1 --2- 172.21.15.43:0/2059140356 >> [v2:172.21.15.203:6800/4196866481,v1:172.21.15.203:6801/4196866481] conn(0x7fe964071150 0x7fe9640735b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T15:12:24.762 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:24.760+0000 7fe97d7fa640 1 -- 172.21.15.43:0/2059140356 --> [v2:172.21.15.203:6800/4196866481,v1:172.21.15.203:6801/4196866481] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7fe964073c80 con 0x7fe964071150 2023-08-09T15:12:24.762 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:24.760+0000 7fe97d7fa640 1 -- 172.21.15.43:0/2059140356 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_get_version_reply(handle=1 version=229) v2 ==== 24+0+0 (secure 0 0 0) 0x7fe9740ac7d0 con 0x7fe9800c9350 2023-08-09T15:12:24.762 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:24.760+0000 7fe97ffff640 1 --2- 172.21.15.43:0/2059140356 >> [v2:172.21.15.203:6832/1456699667,v1:172.21.15.203:6833/1456699667] conn(0x7fe96406a170 0x7fe96406c630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T15:12:24.762 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:24.760+0000 7fe985101640 1 --2- 172.21.15.43:0/2059140356 >> [v2:172.21.15.203:6800/4196866481,v1:172.21.15.203:6801/4196866481] conn(0x7fe964071150 0x7fe9640735b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T15:12:24.763 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:24.761+0000 7fe97ffff640 1 --2- 172.21.15.43:0/2059140356 >> [v2:172.21.15.203:6832/1456699667,v1:172.21.15.203:6833/1456699667] conn(0x7fe96406a170 0x7fe96406c630 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7fe9800adc30 tx=0x7fe96c0011c0 comp rx=0 tx=0).ready entity=mgr.9232 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-09T15:12:24.763 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:24.762+0000 7fe985101640 1 --2- 172.21.15.43:0/2059140356 >> [v2:172.21.15.203:6800/4196866481,v1:172.21.15.203:6801/4196866481] conn(0x7fe964071150 0x7fe9640735b0 crc :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-09T15:12:24.765 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:24.764+0000 7fe97d7fa640 1 -- 172.21.15.43:0/2059140356 <== osd.4 v2:172.21.15.203:6800/4196866481 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+25607 (crc 0 0 0) 0x7fe964073c80 con 0x7fe964071150 2023-08-09T15:12:24.780 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:24.779+0000 7fe986b64640 1 -- 172.21.15.43:0/2059140356 --> [v2:172.21.15.203:6800/4196866481,v1:172.21.15.203:6801/4196866481] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) v1 -- 0x7fe9800c8e30 con 0x7fe964071150 2023-08-09T15:12:24.781 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:24.780+0000 7fe97d7fa640 1 -- 172.21.15.43:0/2059140356 <== osd.4 v2:172.21.15.203:6800/4196866481 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+105 (crc 0 0 0) 0x7fe978001d30 con 0x7fe964071150 2023-08-09T15:12:24.781 INFO:teuthology.orchestra.run.smithi043.stdout:{ 2023-08-09T15:12:24.782 INFO:teuthology.orchestra.run.smithi043.stdout: "success": "osd_debug_deep_scrub_sleep = '0.000000' (not observed, change may require restart) " 2023-08-09T15:12:24.782 INFO:teuthology.orchestra.run.smithi043.stdout:} 2023-08-09T15:12:24.782 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:24.780+0000 7fe986b64640 1 -- 172.21.15.43:0/2059140356 >> [v2:172.21.15.203:6800/4196866481,v1:172.21.15.203:6801/4196866481] conn(0x7fe964071150 msgr2=0x7fe9640735b0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T15:12:24.783 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:24.780+0000 7fe986b64640 1 --2- 172.21.15.43:0/2059140356 >> [v2:172.21.15.203:6800/4196866481,v1:172.21.15.203:6801/4196866481] conn(0x7fe964071150 0x7fe9640735b0 crc :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T15:12:24.783 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:24.780+0000 7fe986b64640 1 -- 172.21.15.43:0/2059140356 >> [v2:172.21.15.203:6832/1456699667,v1:172.21.15.203:6833/1456699667] conn(0x7fe96406a170 msgr2=0x7fe96406c630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T15:12:24.783 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:24.780+0000 7fe986b64640 1 --2- 172.21.15.43:0/2059140356 >> [v2:172.21.15.203:6832/1456699667,v1:172.21.15.203:6833/1456699667] conn(0x7fe96406a170 0x7fe96406c630 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7fe9800adc30 tx=0x7fe96c0011c0 comp rx=0 tx=0).stop 2023-08-09T15:12:24.783 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:24.780+0000 7fe986b64640 1 -- 172.21.15.43:0/2059140356 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fe9800c9350 msgr2=0x7fe9801f15a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T15:12:24.783 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:24.780+0000 7fe986b64640 1 --2- 172.21.15.43:0/2059140356 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fe9800c9350 0x7fe9801f15a0 secure :-1 s=READY pgs=2586 cs=0 l=1 rev1=1 crypto rx=0x7fe974035040 tx=0x7fe9740046c0 comp rx=0 tx=0).stop 2023-08-09T15:12:24.783 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:24.780+0000 7fe985101640 1 -- 172.21.15.43:0/2059140356 reap_dead start 2023-08-09T15:12:24.783 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:24.781+0000 7fe986b64640 1 -- 172.21.15.43:0/2059140356 shutdown_connections 2023-08-09T15:12:24.784 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:24.781+0000 7fe986b64640 1 -- 172.21.15.43:0/2059140356 >> 172.21.15.43:0/2059140356 conn(0x7fe9800afdb0 msgr2=0x7fe98015d610 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-09T15:12:24.784 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:24.781+0000 7fe986b64640 1 -- 172.21.15.43:0/2059140356 shutdown_connections 2023-08-09T15:12:24.784 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:24.781+0000 7fe986b64640 1 -- 172.21.15.43:0/2059140356 wait complete. 2023-08-09T15:12:24.803 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 4 2023-08-09T15:12:25.037 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:25.036+0000 7fdb5dcf3640 1 Processor -- start 2023-08-09T15:12:25.038 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:25.036+0000 7fdb5dcf3640 1 -- start start 2023-08-09T15:12:25.039 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:25.038+0000 7fdb5dcf3640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fdb58159220 0x7fdb580cd590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T15:12:25.039 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:25.038+0000 7fdb5dcf3640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7fdb58159600 0x7fdb580ad530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T15:12:25.039 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:25.038+0000 7fdb5dcf3640 1 -- --> v1:172.21.15.43:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fdb5815df30 con 0x7fdb580707b0 2023-08-09T15:12:25.040 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:25.038+0000 7fdb5dcf3640 1 -- --> v2:172.21.15.43:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fdb580ad2a0 con 0x7fdb58159600 2023-08-09T15:12:25.040 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:25.038+0000 7fdb5dcf3640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdb5815d9b0 con 0x7fdb58159220 2023-08-09T15:12:25.040 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:25.038+0000 7fdb5d4f2640 1 --1- >> v1:172.21.15.43:6789/0 conn(0x7fdb580707b0 0x7fdb5806f500 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.43:6789/0 says I am v1:172.21.15.43:45370/0 (socket says 172.21.15.43:45370) 2023-08-09T15:12:25.041 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:25.038+0000 7fdb5d4f2640 1 -- 172.21.15.43:0/671006366 learned_addr learned my addr 172.21.15.43:0/671006366 (peer_addr_for_me v1:172.21.15.43:0/0) 2023-08-09T15:12:25.041 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:25.038+0000 7fdb57fff640 1 --2- 172.21.15.43:0/671006366 >> v2:172.21.15.43:3300/0 conn(0x7fdb58159600 0x7fdb580ad530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T15:12:25.041 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:25.038+0000 7fdb5ccf1640 1 --2- 172.21.15.43:0/671006366 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fdb58159220 0x7fdb580cd590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T15:12:25.041 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:25.039+0000 7fdb57fff640 1 -- 172.21.15.43:0/671006366 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fdb58159220 msgr2=0x7fdb580cd590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-09T15:12:25.041 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:25.039+0000 7fdb57fff640 1 --2- 172.21.15.43:0/671006366 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fdb58159220 0x7fdb580cd590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T15:12:25.041 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:25.039+0000 7fdb57fff640 1 -- 172.21.15.43:0/671006366 >> v1:172.21.15.43:6789/0 conn(0x7fdb580707b0 legacy=0x7fdb5806f500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T15:12:25.041 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:25.039+0000 7fdb57fff640 1 -- 172.21.15.43:0/671006366 --> v2:172.21.15.43:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdb5815dc50 con 0x7fdb58159600 2023-08-09T15:12:25.042 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:25.040+0000 7fdb57fff640 1 --2- 172.21.15.43:0/671006366 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fdb58159600 0x7fdb580ad530 secure :-1 s=READY pgs=2588 cs=0 l=1 rev1=1 crypto rx=0x7fdb480097a0 tx=0x7fdb4802d9a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1af8c7a19a11f826 server_cookie=0 in_seq=0 out_seq=0 2023-08-09T15:12:25.042 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:25.040+0000 7fdb577fe640 1 -- 172.21.15.43:0/671006366 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdb48032860 con 0x7fdb58159600 2023-08-09T15:12:25.042 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:25.040+0000 7fdb577fe640 1 -- 172.21.15.43:0/671006366 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdb480329c0 con 0x7fdb58159600 2023-08-09T15:12:25.043 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:25.040+0000 7fdb577fe640 1 -- 172.21.15.43:0/671006366 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdb48032d50 con 0x7fdb58159600 2023-08-09T15:12:25.043 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:25.041+0000 7fdb5dcf3640 1 -- 172.21.15.43:0/671006366 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fdb58159600 msgr2=0x7fdb580ad530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T15:12:25.043 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:25.041+0000 7fdb5dcf3640 1 --2- 172.21.15.43:0/671006366 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fdb58159600 0x7fdb580ad530 secure :-1 s=READY pgs=2588 cs=0 l=1 rev1=1 crypto rx=0x7fdb480097a0 tx=0x7fdb4802d9a0 comp rx=0 tx=0).stop 2023-08-09T15:12:25.044 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:25.041+0000 7fdb5dcf3640 1 -- 172.21.15.43:0/671006366 shutdown_connections 2023-08-09T15:12:25.044 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:25.041+0000 7fdb5dcf3640 1 --2- 172.21.15.43:0/671006366 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fdb58159600 0x7fdb580ad530 unknown :-1 s=CLOSED pgs=2588 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T15:12:25.044 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:25.041+0000 7fdb5dcf3640 1 --2- 172.21.15.43:0/671006366 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fdb58159220 0x7fdb580cd590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T15:12:25.044 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:25.041+0000 7fdb5dcf3640 1 -- 172.21.15.43:0/671006366 >> 172.21.15.43:0/671006366 conn(0x7fdb580ce020 msgr2=0x7fdb580ce430 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-09T15:12:25.044 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:25.041+0000 7fdb5dcf3640 1 -- 172.21.15.43:0/671006366 shutdown_connections 2023-08-09T15:12:25.044 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:25.041+0000 7fdb5dcf3640 1 -- 172.21.15.43:0/671006366 wait complete. 2023-08-09T15:12:25.046 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:25.043+0000 7fdb5dcf3640 1 Processor -- start 2023-08-09T15:12:25.046 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:25.043+0000 7fdb5dcf3640 1 -- start start 2023-08-09T15:12:25.046 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:25.044+0000 7fdb5dcf3640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fdb580707b0 0x7fdb580c2cd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T15:12:25.046 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:25.044+0000 7fdb5dcf3640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fdb58159220 0x7fdb581f1990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T15:12:25.046 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:25.044+0000 7fdb5dcf3640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fdb58159600 0x7fdb58154eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T15:12:25.047 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:25.044+0000 7fdb5dcf3640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdb581f1ed0 con 0x7fdb580707b0 2023-08-09T15:12:25.047 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:25.044+0000 7fdb5dcf3640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdb581553f0 con 0x7fdb58159600 2023-08-09T15:12:25.047 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:25.044+0000 7fdb5dcf3640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdb5815b0f0 con 0x7fdb58159220 2023-08-09T15:12:25.047 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:25.044+0000 7fdb5ccf1640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fdb580707b0 0x7fdb580c2cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T15:12:25.047 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:25.045+0000 7fdb5ccf1640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fdb580707b0 0x7fdb580c2cd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:53094/0 (socket says 172.21.15.43:53094) 2023-08-09T15:12:25.047 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:25.045+0000 7fdb5ccf1640 1 -- 172.21.15.43:0/3336071183 learned_addr learned my addr 172.21.15.43:0/3336071183 (peer_addr_for_me v2:172.21.15.43:0/0) 2023-08-09T15:12:25.048 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:25.045+0000 7fdb57fff640 1 --2- 172.21.15.43:0/3336071183 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fdb58159220 0x7fdb581f1990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T15:12:25.049 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:25.045+0000 7fdb5ccf1640 1 -- 172.21.15.43:0/3336071183 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fdb58159220 msgr2=0x7fdb581f1990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T15:12:25.049 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:25.045+0000 7fdb5ccf1640 1 --2- 172.21.15.43:0/3336071183 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fdb58159220 0x7fdb581f1990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T15:12:25.049 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:25.045+0000 7fdb5ccf1640 1 -- 172.21.15.43:0/3336071183 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fdb58159600 msgr2=0x7fdb58154eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T15:12:25.049 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:25.045+0000 7fdb5d4f2640 1 --2- 172.21.15.43:0/3336071183 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fdb58159600 0x7fdb58154eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T15:12:25.049 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:25.045+0000 7fdb5ccf1640 1 --2- 172.21.15.43:0/3336071183 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fdb58159600 0x7fdb58154eb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T15:12:25.049 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:25.045+0000 7fdb5ccf1640 1 -- 172.21.15.43:0/3336071183 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdb48009400 con 0x7fdb580707b0 2023-08-09T15:12:25.049 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:25.045+0000 7fdb5ccf1640 1 --2- 172.21.15.43:0/3336071183 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fdb580707b0 0x7fdb580c2cd0 secure :-1 s=READY pgs=2589 cs=0 l=1 rev1=1 crypto rx=0x7fdb4000d800 tx=0x7fdb4000dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-09T15:12:25.050 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:25.046+0000 7fdb557fa640 1 -- 172.21.15.43:0/3336071183 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdb4000b6c0 con 0x7fdb580707b0 2023-08-09T15:12:25.050 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:25.046+0000 7fdb557fa640 1 -- 172.21.15.43:0/3336071183 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdb4000b820 con 0x7fdb580707b0 2023-08-09T15:12:25.050 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:25.046+0000 7fdb5dcf3640 1 -- 172.21.15.43:0/3336071183 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdb580c32b0 con 0x7fdb580707b0 2023-08-09T15:12:25.050 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:25.046+0000 7fdb557fa640 1 -- 172.21.15.43:0/3336071183 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdb4000bbb0 con 0x7fdb580707b0 2023-08-09T15:12:25.050 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:25.046+0000 7fdb5dcf3640 1 -- 172.21.15.43:0/3336071183 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdb580c3870 con 0x7fdb580707b0 2023-08-09T15:12:25.050 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:25.047+0000 7fdb5dcf3640 1 -- 172.21.15.43:0/3336071183 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdb5804e390 con 0x7fdb580707b0 2023-08-09T15:12:25.051 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:25.048+0000 7fdb557fa640 1 -- 172.21.15.43:0/3336071183 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 129) v1 ==== 88443+0+0 (secure 0 0 0) 0x7fdb4000bd10 con 0x7fdb580707b0 2023-08-09T15:12:25.051 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:25.049+0000 7fdb557fa640 1 --2- 172.21.15.43:0/3336071183 >> [v2:172.21.15.203:6832/1456699667,v1:172.21.15.203:6833/1456699667] conn(0x7fdb3806a170 0x7fdb3806c630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T15:12:25.051 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:25.049+0000 7fdb557fa640 1 -- 172.21.15.43:0/3336071183 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(229..229 src has 1..229) v4 ==== 6462+0+0 (secure 0 0 0) 0x7fdb40088d80 con 0x7fdb580707b0 2023-08-09T15:12:25.051 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:25.050+0000 7fdb57fff640 1 --2- 172.21.15.43:0/3336071183 >> [v2:172.21.15.203:6832/1456699667,v1:172.21.15.203:6833/1456699667] conn(0x7fdb3806a170 0x7fdb3806c630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T15:12:25.052 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:25.051+0000 7fdb57fff640 1 --2- 172.21.15.43:0/3336071183 >> [v2:172.21.15.203:6832/1456699667,v1:172.21.15.203:6833/1456699667] conn(0x7fdb3806a170 0x7fdb3806c630 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7fdb48038000 tx=0x7fdb4803d040 comp rx=0 tx=0).ready entity=mgr.9232 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-09T15:12:25.054 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:25.053+0000 7fdb557fa640 1 -- 172.21.15.43:0/3336071183 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178066 (secure 0 0 0) 0x7fdb40057cf0 con 0x7fdb580707b0 2023-08-09T15:12:25.251 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:25.250+0000 7fdb5dcf3640 1 -- 172.21.15.43:0/3336071183 --> [v2:172.21.15.203:6832/1456699667,v1:172.21.15.203:6833/1456699667] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "4", "target": ["mon-mgr", ""]}) v1 -- 0x7fdb580cc000 con 0x7fdb3806a170 2023-08-09T15:12:25.252 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:25.252+0000 7fdb557fa640 1 -- 172.21.15.43:0/3336071183 <== mgr.9232 v2:172.21.15.203:6832/1456699667 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 4 to deep-scrub) v1 ==== 41+0+0 (secure 0 0 0) 0x7fdb48032b30 con 0x7fdb3806a170 2023-08-09T15:12:25.253 INFO:teuthology.orchestra.run.smithi043.stderr:instructed osd(s) 4 to deep-scrub 2023-08-09T15:12:25.255 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:25.253+0000 7fdb5dcf3640 1 -- 172.21.15.43:0/3336071183 >> [v2:172.21.15.203:6832/1456699667,v1:172.21.15.203:6833/1456699667] conn(0x7fdb3806a170 msgr2=0x7fdb3806c630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T15:12:25.255 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:25.253+0000 7fdb5dcf3640 1 --2- 172.21.15.43:0/3336071183 >> [v2:172.21.15.203:6832/1456699667,v1:172.21.15.203:6833/1456699667] conn(0x7fdb3806a170 0x7fdb3806c630 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7fdb48038000 tx=0x7fdb4803d040 comp rx=0 tx=0).stop 2023-08-09T15:12:25.256 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:25.253+0000 7fdb5dcf3640 1 -- 172.21.15.43:0/3336071183 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fdb580707b0 msgr2=0x7fdb580c2cd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T15:12:25.256 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:25.253+0000 7fdb5dcf3640 1 --2- 172.21.15.43:0/3336071183 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fdb580707b0 0x7fdb580c2cd0 secure :-1 s=READY pgs=2589 cs=0 l=1 rev1=1 crypto rx=0x7fdb4000d800 tx=0x7fdb4000dcc0 comp rx=0 tx=0).stop 2023-08-09T15:12:25.256 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:25.254+0000 7fdb5dcf3640 1 -- 172.21.15.43:0/3336071183 shutdown_connections 2023-08-09T15:12:25.256 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:25.254+0000 7fdb5dcf3640 1 --2- 172.21.15.43:0/3336071183 >> [v2:172.21.15.203:6832/1456699667,v1:172.21.15.203:6833/1456699667] conn(0x7fdb3806a170 0x7fdb3806c630 unknown :-1 s=CLOSED pgs=52 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T15:12:25.256 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:25.254+0000 7fdb5dcf3640 1 --2- 172.21.15.43:0/3336071183 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fdb58159600 0x7fdb58154eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T15:12:25.256 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:25.254+0000 7fdb5dcf3640 1 --2- 172.21.15.43:0/3336071183 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fdb58159220 0x7fdb581f1990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T15:12:25.258 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:25.254+0000 7fdb5dcf3640 1 --2- 172.21.15.43:0/3336071183 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fdb580707b0 0x7fdb580c2cd0 unknown :-1 s=CLOSED pgs=2589 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T15:12:25.258 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:25.254+0000 7fdb5dcf3640 1 -- 172.21.15.43:0/3336071183 >> 172.21.15.43:0/3336071183 conn(0x7fdb580ce020 msgr2=0x7fdb58072af0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-09T15:12:25.258 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:25.254+0000 7fdb5dcf3640 1 -- 172.21.15.43:0/3336071183 shutdown_connections 2023-08-09T15:12:25.258 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:25.254+0000 7fdb5dcf3640 1 -- 172.21.15.43:0/3336071183 wait complete. 2023-08-09T15:12:25.273 INFO:tasks.ceph:Scrubbing osd.5 2023-08-09T15:12:25.274 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.5 config set osd_debug_deep_scrub_sleep 0 2023-08-09T15:12:25.508 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:25.506+0000 7f53b6700640 1 Processor -- start 2023-08-09T15:12:25.508 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:25.506+0000 7f53b6700640 1 -- start start 2023-08-09T15:12:25.509 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:25.508+0000 7f53b6700640 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7f53b015c710 0x7f53b00c9dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T15:12:25.509 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:25.508+0000 7f53b6700640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f53b00cb340 0x7f53b00ab860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T15:12:25.510 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:25.508+0000 7f53b6700640 1 -- --> v1:172.21.15.43:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f53b0160300 con 0x7f53b00ca310 2023-08-09T15:12:25.510 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:25.508+0000 7f53b6700640 1 -- --> v2:172.21.15.43:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f53b0160550 con 0x7f53b00cb340 2023-08-09T15:12:25.510 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:25.508+0000 7f53b6700640 1 -- --> v2:172.21.15.203:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f53b0160830 con 0x7f53b015c710 2023-08-09T15:12:25.510 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:25.508+0000 7f53af7fe640 1 --1- >> v1:172.21.15.43:6789/0 conn(0x7f53b00ca310 0x7f53b00cac30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.43:6789/0 says I am v1:172.21.15.43:45386/0 (socket says 172.21.15.43:45386) 2023-08-09T15:12:25.510 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:25.508+0000 7f53af7fe640 1 -- 172.21.15.43:0/3689642094 learned_addr learned my addr 172.21.15.43:0/3689642094 (peer_addr_for_me v1:172.21.15.43:0/0) 2023-08-09T15:12:25.512 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:25.508+0000 7f53b4c9d640 1 --2- 172.21.15.43:0/3689642094 >> v2:172.21.15.43:3300/0 conn(0x7f53b00cb340 0x7f53b00ab860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T15:12:25.513 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:25.509+0000 7f53affff640 1 --2- 172.21.15.43:0/3689642094 >> v2:172.21.15.203:3300/0 conn(0x7f53b015c710 0x7f53b00c9dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T15:12:25.513 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:25.509+0000 7f53b4c9d640 1 -- 172.21.15.43:0/3689642094 >> v2:172.21.15.203:3300/0 conn(0x7f53b015c710 msgr2=0x7f53b00c9dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-09T15:12:25.513 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:25.509+0000 7f53b4c9d640 1 --2- 172.21.15.43:0/3689642094 >> v2:172.21.15.203:3300/0 conn(0x7f53b015c710 0x7f53b00c9dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T15:12:25.513 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:25.509+0000 7f53b4c9d640 1 -- 172.21.15.43:0/3689642094 >> v1:172.21.15.43:6789/0 conn(0x7f53b00ca310 legacy=0x7f53b00cac30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T15:12:25.514 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:25.509+0000 7f53b4c9d640 1 -- 172.21.15.43:0/3689642094 --> v2:172.21.15.43:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f53b0160b10 con 0x7f53b00cb340 2023-08-09T15:12:25.514 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:25.509+0000 7f53aeffd640 1 -- 172.21.15.43:0/3689642094 <== mon.0 v1:172.21.15.43:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2609069622 0 0) 0x7f53b0160300 con 0x7f53b00ca310 2023-08-09T15:12:25.514 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:25.510+0000 7f53b4c9d640 1 --2- 172.21.15.43:0/3689642094 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f53b00cb340 0x7f53b00ab860 secure :-1 s=READY pgs=2591 cs=0 l=1 rev1=1 crypto rx=0x7f53a8009d90 tx=0x7f53a802bd90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=967776a2a040df0 server_cookie=0 in_seq=0 out_seq=0 2023-08-09T15:12:25.514 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:25.510+0000 7f53aeffd640 1 -- 172.21.15.43:0/3689642094 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f53a802cb40 con 0x7f53b00cb340 2023-08-09T15:12:25.514 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:25.510+0000 7f53aeffd640 1 -- 172.21.15.43:0/3689642094 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f53a802cca0 con 0x7f53b00cb340 2023-08-09T15:12:25.514 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:25.510+0000 7f53aeffd640 1 -- 172.21.15.43:0/3689642094 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f53a8034660 con 0x7f53b00cb340 2023-08-09T15:12:25.515 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:25.511+0000 7f53b6700640 1 -- 172.21.15.43:0/3689642094 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f53b00cb340 msgr2=0x7f53b00ab860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T15:12:25.515 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:25.511+0000 7f53b6700640 1 --2- 172.21.15.43:0/3689642094 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f53b00cb340 0x7f53b00ab860 secure :-1 s=READY pgs=2591 cs=0 l=1 rev1=1 crypto rx=0x7f53a8009d90 tx=0x7f53a802bd90 comp rx=0 tx=0).stop 2023-08-09T15:12:25.515 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:25.511+0000 7f53b6700640 1 -- 172.21.15.43:0/3689642094 shutdown_connections 2023-08-09T15:12:25.515 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:25.511+0000 7f53b6700640 1 --2- 172.21.15.43:0/3689642094 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f53b00cb340 0x7f53b00ab860 unknown :-1 s=CLOSED pgs=2591 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T15:12:25.515 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:25.511+0000 7f53b6700640 1 --2- 172.21.15.43:0/3689642094 >> v2:172.21.15.203:3300/0 conn(0x7f53b015c710 0x7f53b00c9dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T15:12:25.516 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:25.511+0000 7f53b6700640 1 -- 172.21.15.43:0/3689642094 >> 172.21.15.43:0/3689642094 conn(0x7f53b00afdb0 msgr2=0x7f53b00b0190 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-09T15:12:25.516 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:25.511+0000 7f53b6700640 1 -- 172.21.15.43:0/3689642094 shutdown_connections 2023-08-09T15:12:25.516 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:25.511+0000 7f53b6700640 1 -- 172.21.15.43:0/3689642094 wait complete. 2023-08-09T15:12:25.518 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:25.513+0000 7f53b6700640 1 Processor -- start 2023-08-09T15:12:25.519 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:25.514+0000 7f53b6700640 1 -- start start 2023-08-09T15:12:25.519 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:25.515+0000 7f53b6700640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f53b00ca310 0x7f53b0151aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T15:12:25.519 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:25.515+0000 7f53b6700640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f53b00cb340 0x7f53b00b2e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T15:12:25.519 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:25.515+0000 7f53b6700640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f53b015c710 0x7f53b00b33a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T15:12:25.519 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:25.515+0000 7f53b6700640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f53b00c3d60 con 0x7f53b015c710 2023-08-09T15:12:25.520 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:25.515+0000 7f53b6700640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f53b00c4040 con 0x7f53b00ca310 2023-08-09T15:12:25.520 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:25.515+0000 7f53b6700640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f53b00c4320 con 0x7f53b00cb340 2023-08-09T15:12:25.520 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:25.515+0000 7f53b4c9d640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f53b015c710 0x7f53b00b33a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T15:12:25.520 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:25.515+0000 7f53b4c9d640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f53b015c710 0x7f53b00b33a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:53102/0 (socket says 172.21.15.43:53102) 2023-08-09T15:12:25.520 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:25.515+0000 7f53b4c9d640 1 -- 172.21.15.43:0/2346908601 learned_addr learned my addr 172.21.15.43:0/2346908601 (peer_addr_for_me v2:172.21.15.43:0/0) 2023-08-09T15:12:25.520 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:25.516+0000 7f53affff640 1 --2- 172.21.15.43:0/2346908601 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f53b00ca310 0x7f53b0151aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T15:12:25.521 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:25.516+0000 7f53af7fe640 1 --2- 172.21.15.43:0/2346908601 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f53b00cb340 0x7f53b00b2e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T15:12:25.521 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:25.516+0000 7f53b4c9d640 1 -- 172.21.15.43:0/2346908601 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f53b00cb340 msgr2=0x7f53b00b2e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T15:12:25.521 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:25.516+0000 7f53b4c9d640 1 --2- 172.21.15.43:0/2346908601 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f53b00cb340 0x7f53b00b2e60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T15:12:25.521 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:25.516+0000 7f53b4c9d640 1 -- 172.21.15.43:0/2346908601 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f53b00ca310 msgr2=0x7f53b0151aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T15:12:25.521 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:25.516+0000 7f53b4c9d640 1 --2- 172.21.15.43:0/2346908601 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f53b00ca310 0x7f53b0151aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T15:12:25.522 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:25.516+0000 7f53b4c9d640 1 -- 172.21.15.43:0/2346908601 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f53a8009a70 con 0x7f53b015c710 2023-08-09T15:12:25.522 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:25.516+0000 7f53affff640 1 --2- 172.21.15.43:0/2346908601 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f53b00ca310 0x7f53b0151aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-08-09T15:12:25.522 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:25.516+0000 7f53b4c9d640 1 --2- 172.21.15.43:0/2346908601 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f53b015c710 0x7f53b00b33a0 secure :-1 s=READY pgs=2592 cs=0 l=1 rev1=1 crypto rx=0x7f53a8009d90 tx=0x7f53a8004a90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-09T15:12:25.522 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:25.517+0000 7f53acff9640 1 -- 172.21.15.43:0/2346908601 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f53a803e610 con 0x7f53b015c710 2023-08-09T15:12:25.522 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:25.517+0000 7f53acff9640 1 -- 172.21.15.43:0/2346908601 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f53a803e770 con 0x7f53b015c710 2023-08-09T15:12:25.523 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:25.517+0000 7f53b6700640 1 -- 172.21.15.43:0/2346908601 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f53b015cc70 con 0x7f53b015c710 2023-08-09T15:12:25.523 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:25.517+0000 7f53acff9640 1 -- 172.21.15.43:0/2346908601 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f53a803eae0 con 0x7f53b015c710 2023-08-09T15:12:25.523 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:25.517+0000 7f53b6700640 1 -- 172.21.15.43:0/2346908601 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f53b00c48e0 con 0x7f53b015c710 2023-08-09T15:12:25.524 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:25.518+0000 7f53b6700640 1 -- 172.21.15.43:0/2346908601 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f53b015caf0 con 0x7f53b015c710 2023-08-09T15:12:25.525 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:25.519+0000 7f53acff9640 1 -- 172.21.15.43:0/2346908601 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 129) v1 ==== 88443+0+0 (secure 0 0 0) 0x7f53a803b050 con 0x7f53b015c710 2023-08-09T15:12:25.525 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:25.520+0000 7f53acff9640 1 --2- 172.21.15.43:0/2346908601 >> [v2:172.21.15.203:6832/1456699667,v1:172.21.15.203:6833/1456699667] conn(0x7f539c06a170 0x7f539c06c630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T15:12:25.525 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:25.520+0000 7f53acff9640 1 -- 172.21.15.43:0/2346908601 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(229..229 src has 1..229) v4 ==== 6462+0+0 (secure 0 0 0) 0x7f53a80ab410 con 0x7f53b015c710 2023-08-09T15:12:25.525 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:25.520+0000 7f53acff9640 1 --2- 172.21.15.43:0/2346908601 >> [v2:172.21.15.203:6816/3059290081,v1:172.21.15.203:6817/3059290081] conn(0x7f539c071150 0x7f539c0735b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T15:12:25.525 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:25.520+0000 7f53acff9640 1 -- 172.21.15.43:0/2346908601 --> [v2:172.21.15.203:6816/3059290081,v1:172.21.15.203:6817/3059290081] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f539c073c80 con 0x7f539c071150 2023-08-09T15:12:25.526 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:25.520+0000 7f53acff9640 1 -- 172.21.15.43:0/2346908601 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_get_version_reply(handle=1 version=229) v2 ==== 24+0+0 (secure 0 0 0) 0x7f53a80ab780 con 0x7f53b015c710 2023-08-09T15:12:25.526 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:25.521+0000 7f53affff640 1 --2- 172.21.15.43:0/2346908601 >> [v2:172.21.15.203:6832/1456699667,v1:172.21.15.203:6833/1456699667] conn(0x7f539c06a170 0x7f539c06c630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T15:12:25.526 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:25.521+0000 7f53af7fe640 1 --2- 172.21.15.43:0/2346908601 >> [v2:172.21.15.203:6816/3059290081,v1:172.21.15.203:6817/3059290081] conn(0x7f539c071150 0x7f539c0735b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T15:12:25.526 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:25.522+0000 7f53affff640 1 --2- 172.21.15.43:0/2346908601 >> [v2:172.21.15.203:6832/1456699667,v1:172.21.15.203:6833/1456699667] conn(0x7f539c06a170 0x7f539c06c630 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f53a0006fd0 tx=0x7f53a0008040 comp rx=0 tx=0).ready entity=mgr.9232 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-09T15:12:25.526 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:25.522+0000 7f53af7fe640 1 --2- 172.21.15.43:0/2346908601 >> [v2:172.21.15.203:6816/3059290081,v1:172.21.15.203:6817/3059290081] conn(0x7f539c071150 0x7f539c0735b0 crc :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-09T15:12:25.527 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:25.525+0000 7f53acff9640 1 -- 172.21.15.43:0/2346908601 <== osd.5 v2:172.21.15.203:6816/3059290081 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+25607 (crc 0 0 0) 0x7f539c073c80 con 0x7f539c071150 2023-08-09T15:12:25.540 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:25.539+0000 7f53b6700640 1 -- 172.21.15.43:0/2346908601 --> [v2:172.21.15.203:6816/3059290081,v1:172.21.15.203:6817/3059290081] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) v1 -- 0x7f53b0160df0 con 0x7f539c071150 2023-08-09T15:12:25.542 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:25.540+0000 7f53acff9640 1 -- 172.21.15.43:0/2346908601 <== osd.5 v2:172.21.15.203:6816/3059290081 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+105 (crc 0 0 0) 0x7f53a40070f0 con 0x7f539c071150 2023-08-09T15:12:25.542 INFO:teuthology.orchestra.run.smithi043.stdout:{ 2023-08-09T15:12:25.542 INFO:teuthology.orchestra.run.smithi043.stdout: "success": "osd_debug_deep_scrub_sleep = '0.000000' (not observed, change may require restart) " 2023-08-09T15:12:25.542 INFO:teuthology.orchestra.run.smithi043.stdout:} 2023-08-09T15:12:25.543 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:25.541+0000 7f53b6700640 1 -- 172.21.15.43:0/2346908601 >> [v2:172.21.15.203:6816/3059290081,v1:172.21.15.203:6817/3059290081] conn(0x7f539c071150 msgr2=0x7f539c0735b0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T15:12:25.543 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:25.541+0000 7f53b6700640 1 --2- 172.21.15.43:0/2346908601 >> [v2:172.21.15.203:6816/3059290081,v1:172.21.15.203:6817/3059290081] conn(0x7f539c071150 0x7f539c0735b0 crc :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T15:12:25.544 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:25.541+0000 7f53b6700640 1 -- 172.21.15.43:0/2346908601 >> [v2:172.21.15.203:6832/1456699667,v1:172.21.15.203:6833/1456699667] conn(0x7f539c06a170 msgr2=0x7f539c06c630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T15:12:25.544 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:25.541+0000 7f53b6700640 1 --2- 172.21.15.43:0/2346908601 >> [v2:172.21.15.203:6832/1456699667,v1:172.21.15.203:6833/1456699667] conn(0x7f539c06a170 0x7f539c06c630 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f53a0006fd0 tx=0x7f53a0008040 comp rx=0 tx=0).stop 2023-08-09T15:12:25.544 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:25.541+0000 7f53b6700640 1 -- 172.21.15.43:0/2346908601 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f53b015c710 msgr2=0x7f53b00b33a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T15:12:25.544 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:25.541+0000 7f53b6700640 1 --2- 172.21.15.43:0/2346908601 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f53b015c710 0x7f53b00b33a0 secure :-1 s=READY pgs=2592 cs=0 l=1 rev1=1 crypto rx=0x7f53a8009d90 tx=0x7f53a8004a90 comp rx=0 tx=0).stop 2023-08-09T15:12:25.544 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:25.541+0000 7f53b4c9d640 1 -- 172.21.15.43:0/2346908601 reap_dead start 2023-08-09T15:12:25.544 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:25.542+0000 7f53b6700640 1 -- 172.21.15.43:0/2346908601 shutdown_connections 2023-08-09T15:12:25.544 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:25.542+0000 7f53b6700640 1 -- 172.21.15.43:0/2346908601 >> 172.21.15.43:0/2346908601 conn(0x7f53b00afdb0 msgr2=0x7f53b00adb60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-09T15:12:25.544 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:25.542+0000 7f53b6700640 1 -- 172.21.15.43:0/2346908601 shutdown_connections 2023-08-09T15:12:25.545 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:25.542+0000 7f53b6700640 1 -- 172.21.15.43:0/2346908601 wait complete. 2023-08-09T15:12:25.564 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 5 2023-08-09T15:12:25.797 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:25.795+0000 7fcf6750f640 1 Processor -- start 2023-08-09T15:12:25.797 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:25.795+0000 7fcf6750f640 1 -- start start 2023-08-09T15:12:25.798 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:25.797+0000 7fcf6750f640 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7fcf600ceda0 0x7fcf600b1cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T15:12:25.799 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:25.797+0000 7fcf6750f640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7fcf600b2200 0x7fcf6015a890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T15:12:25.799 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:25.797+0000 7fcf6750f640 1 -- --> v1:172.21.15.43:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fcf600b2610 con 0x7fcf60154ab0 2023-08-09T15:12:25.799 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:25.797+0000 7fcf6750f640 1 -- --> v2:172.21.15.43:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fcf600cfe40 con 0x7fcf600b2200 2023-08-09T15:12:25.799 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:25.797+0000 7fcf6750f640 1 -- --> v2:172.21.15.203:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fcf600ce2c0 con 0x7fcf600ceda0 2023-08-09T15:12:25.799 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:25.797+0000 7fcf6650d640 1 --1- >> v1:172.21.15.43:6789/0 conn(0x7fcf60154ab0 0x7fcf600cf3f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.43:6789/0 says I am v1:172.21.15.43:45400/0 (socket says 172.21.15.43:45400) 2023-08-09T15:12:25.799 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:25.797+0000 7fcf6650d640 1 -- 172.21.15.43:0/2871814415 learned_addr learned my addr 172.21.15.43:0/2871814415 (peer_addr_for_me v1:172.21.15.43:0/0) 2023-08-09T15:12:25.800 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:25.798+0000 7fcf66d0e640 1 --2- 172.21.15.43:0/2871814415 >> v2:172.21.15.43:3300/0 conn(0x7fcf600b2200 0x7fcf6015a890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T15:12:25.801 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:25.798+0000 7fcf65d0c640 1 --2- 172.21.15.43:0/2871814415 >> v2:172.21.15.203:3300/0 conn(0x7fcf600ceda0 0x7fcf600b1cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T15:12:25.801 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:25.798+0000 7fcf6550b640 1 -- 172.21.15.43:0/2871814415 <== mon.0 v1:172.21.15.43:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2003127964 0 0) 0x7fcf600b2610 con 0x7fcf60154ab0 2023-08-09T15:12:25.801 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:25.798+0000 7fcf6550b640 1 -- 172.21.15.43:0/2871814415 --> v1:172.21.15.43:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fcf4c003520 con 0x7fcf60154ab0 2023-08-09T15:12:25.801 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:25.798+0000 7fcf66d0e640 1 -- 172.21.15.43:0/2871814415 >> v2:172.21.15.203:3300/0 conn(0x7fcf600ceda0 msgr2=0x7fcf600b1cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-09T15:12:25.801 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:25.798+0000 7fcf66d0e640 1 --2- 172.21.15.43:0/2871814415 >> v2:172.21.15.203:3300/0 conn(0x7fcf600ceda0 0x7fcf600b1cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T15:12:25.801 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:25.798+0000 7fcf66d0e640 1 -- 172.21.15.43:0/2871814415 >> v1:172.21.15.43:6789/0 conn(0x7fcf60154ab0 legacy=0x7fcf600cf3f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T15:12:25.801 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:25.798+0000 7fcf6550b640 1 -- 172.21.15.43:0/2871814415 <== mon.0 v1:172.21.15.43:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 4015646618 0 0) 0x7fcf4c003520 con 0x7fcf60154ab0 2023-08-09T15:12:25.801 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:25.798+0000 7fcf66d0e640 1 -- 172.21.15.43:0/2871814415 --> v2:172.21.15.43:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcf600ce5a0 con 0x7fcf600b2200 2023-08-09T15:12:25.803 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:25.800+0000 7fcf66d0e640 1 --2- 172.21.15.43:0/2871814415 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fcf600b2200 0x7fcf6015a890 secure :-1 s=READY pgs=2594 cs=0 l=1 rev1=1 crypto rx=0x7fcf600afab0 tx=0x7fcf5802bd90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=582788d4b673ed98 server_cookie=0 in_seq=0 out_seq=0 2023-08-09T15:12:25.803 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:25.800+0000 7fcf6550b640 1 -- 172.21.15.43:0/2871814415 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcf5c001670 con 0x7fcf600b2200 2023-08-09T15:12:25.803 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:25.800+0000 7fcf6550b640 1 -- 172.21.15.43:0/2871814415 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcf5802c8c0 con 0x7fcf600b2200 2023-08-09T15:12:25.803 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:25.800+0000 7fcf6550b640 1 -- 172.21.15.43:0/2871814415 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcf5802cc30 con 0x7fcf600b2200 2023-08-09T15:12:25.803 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:25.800+0000 7fcf6750f640 1 -- 172.21.15.43:0/2871814415 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fcf600b2200 msgr2=0x7fcf6015a890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T15:12:25.803 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:25.800+0000 7fcf6750f640 1 --2- 172.21.15.43:0/2871814415 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fcf600b2200 0x7fcf6015a890 secure :-1 s=READY pgs=2594 cs=0 l=1 rev1=1 crypto rx=0x7fcf600afab0 tx=0x7fcf5802bd90 comp rx=0 tx=0).stop 2023-08-09T15:12:25.804 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:25.801+0000 7fcf6750f640 1 -- 172.21.15.43:0/2871814415 shutdown_connections 2023-08-09T15:12:25.804 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:25.801+0000 7fcf6750f640 1 --2- 172.21.15.43:0/2871814415 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fcf600b2200 0x7fcf6015a890 unknown :-1 s=CLOSED pgs=2594 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T15:12:25.804 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:25.801+0000 7fcf6750f640 1 --2- 172.21.15.43:0/2871814415 >> v2:172.21.15.203:3300/0 conn(0x7fcf600ceda0 0x7fcf600b1cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T15:12:25.804 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:25.801+0000 7fcf6750f640 1 -- 172.21.15.43:0/2871814415 >> 172.21.15.43:0/2871814415 conn(0x7fcf6006a150 msgr2=0x7fcf600cccc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-09T15:12:25.804 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:25.801+0000 7fcf6750f640 1 -- 172.21.15.43:0/2871814415 shutdown_connections 2023-08-09T15:12:25.804 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:25.801+0000 7fcf6750f640 1 -- 172.21.15.43:0/2871814415 wait complete. 2023-08-09T15:12:25.805 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:25.802+0000 7fcf6750f640 1 Processor -- start 2023-08-09T15:12:25.805 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:25.803+0000 7fcf6750f640 1 -- start start 2023-08-09T15:12:25.805 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:25.804+0000 7fcf6750f640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fcf600b2200 0x7fcf600c96c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T15:12:25.806 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:25.804+0000 7fcf6750f640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fcf600ceda0 0x7fcf60154ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T15:12:25.806 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:25.804+0000 7fcf6750f640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fcf60154ab0 0x7fcf60155430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T15:12:25.806 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:25.804+0000 7fcf6750f640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcf601567f0 con 0x7fcf60154ab0 2023-08-09T15:12:25.806 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:25.804+0000 7fcf6750f640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcf60155970 con 0x7fcf600ceda0 2023-08-09T15:12:25.806 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:25.804+0000 7fcf6750f640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcf60155be0 con 0x7fcf600b2200 2023-08-09T15:12:25.806 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:25.804+0000 7fcf66d0e640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fcf60154ab0 0x7fcf60155430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T15:12:25.806 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:25.804+0000 7fcf66d0e640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fcf60154ab0 0x7fcf60155430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:53116/0 (socket says 172.21.15.43:53116) 2023-08-09T15:12:25.807 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:25.804+0000 7fcf66d0e640 1 -- 172.21.15.43:0/1614781768 learned_addr learned my addr 172.21.15.43:0/1614781768 (peer_addr_for_me v2:172.21.15.43:0/0) 2023-08-09T15:12:25.807 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:25.804+0000 7fcf65d0c640 1 --2- 172.21.15.43:0/1614781768 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fcf600ceda0 0x7fcf60154ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T15:12:25.808 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:25.805+0000 7fcf66d0e640 1 -- 172.21.15.43:0/1614781768 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fcf600b2200 msgr2=0x7fcf600c96c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T15:12:25.808 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:25.805+0000 7fcf6650d640 1 --2- 172.21.15.43:0/1614781768 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fcf600b2200 0x7fcf600c96c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T15:12:25.808 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:25.805+0000 7fcf66d0e640 1 --2- 172.21.15.43:0/1614781768 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fcf600b2200 0x7fcf600c96c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T15:12:25.808 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:25.805+0000 7fcf66d0e640 1 -- 172.21.15.43:0/1614781768 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fcf600ceda0 msgr2=0x7fcf60154ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T15:12:25.808 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:25.805+0000 7fcf66d0e640 1 --2- 172.21.15.43:0/1614781768 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fcf600ceda0 0x7fcf60154ef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T15:12:25.808 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:25.805+0000 7fcf66d0e640 1 -- 172.21.15.43:0/1614781768 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcf58009a70 con 0x7fcf60154ab0 2023-08-09T15:12:25.809 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:25.805+0000 7fcf6650d640 1 --2- 172.21.15.43:0/1614781768 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fcf600b2200 0x7fcf600c96c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-08-09T15:12:25.809 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:25.805+0000 7fcf66d0e640 1 --2- 172.21.15.43:0/1614781768 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fcf60154ab0 0x7fcf60155430 secure :-1 s=READY pgs=2595 cs=0 l=1 rev1=1 crypto rx=0x7fcf5802c3a0 tx=0x7fcf58004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-09T15:12:25.809 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:25.805+0000 7fcf52ffd640 1 -- 172.21.15.43:0/1614781768 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcf58009450 con 0x7fcf60154ab0 2023-08-09T15:12:25.809 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:25.805+0000 7fcf52ffd640 1 -- 172.21.15.43:0/1614781768 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcf58034c50 con 0x7fcf60154ab0 2023-08-09T15:12:25.809 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:25.805+0000 7fcf52ffd640 1 -- 172.21.15.43:0/1614781768 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcf5803f420 con 0x7fcf60154ab0 2023-08-09T15:12:25.809 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:25.805+0000 7fcf6750f640 1 -- 172.21.15.43:0/1614781768 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcf60155ec0 con 0x7fcf60154ab0 2023-08-09T15:12:25.809 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:25.806+0000 7fcf6750f640 1 -- 172.21.15.43:0/1614781768 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcf60156480 con 0x7fcf60154ab0 2023-08-09T15:12:25.810 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:25.807+0000 7fcf6750f640 1 -- 172.21.15.43:0/1614781768 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcf601569a0 con 0x7fcf60154ab0 2023-08-09T15:12:25.810 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:25.808+0000 7fcf52ffd640 1 -- 172.21.15.43:0/1614781768 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 129) v1 ==== 88443+0+0 (secure 0 0 0) 0x7fcf5803f580 con 0x7fcf60154ab0 2023-08-09T15:12:25.810 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:25.809+0000 7fcf52ffd640 1 --2- 172.21.15.43:0/1614781768 >> [v2:172.21.15.203:6832/1456699667,v1:172.21.15.203:6833/1456699667] conn(0x7fcf4406a170 0x7fcf4406c630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T15:12:25.811 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:25.809+0000 7fcf52ffd640 1 -- 172.21.15.43:0/1614781768 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(229..229 src has 1..229) v4 ==== 6462+0+0 (secure 0 0 0) 0x7fcf580ab170 con 0x7fcf60154ab0 2023-08-09T15:12:25.811 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:25.810+0000 7fcf6650d640 1 --2- 172.21.15.43:0/1614781768 >> [v2:172.21.15.203:6832/1456699667,v1:172.21.15.203:6833/1456699667] conn(0x7fcf4406a170 0x7fcf4406c630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T15:12:25.812 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:25.811+0000 7fcf6650d640 1 --2- 172.21.15.43:0/1614781768 >> [v2:172.21.15.203:6832/1456699667,v1:172.21.15.203:6833/1456699667] conn(0x7fcf4406a170 0x7fcf4406c630 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7fcf5c0079b0 tx=0x7fcf5c009040 comp rx=0 tx=0).ready entity=mgr.9232 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-09T15:12:25.815 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:25.814+0000 7fcf52ffd640 1 -- 172.21.15.43:0/1614781768 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178066 (secure 0 0 0) 0x7fcf601569a0 con 0x7fcf60154ab0 2023-08-09T15:12:26.010 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:26.008+0000 7fcf6750f640 1 -- 172.21.15.43:0/1614781768 --> [v2:172.21.15.203:6832/1456699667,v1:172.21.15.203:6833/1456699667] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "5", "target": ["mon-mgr", ""]}) v1 -- 0x7fcf600cfb00 con 0x7fcf4406a170 2023-08-09T15:12:26.011 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:26.009+0000 7fcf52ffd640 1 -- 172.21.15.43:0/1614781768 <== mgr.9232 v2:172.21.15.203:6832/1456699667 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 5 to deep-scrub) v1 ==== 41+0+0 (secure 0 0 0) 0x7fcf5c0058d0 con 0x7fcf4406a170 2023-08-09T15:12:26.011 INFO:teuthology.orchestra.run.smithi043.stderr:instructed osd(s) 5 to deep-scrub 2023-08-09T15:12:26.013 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:26.012+0000 7fcf6750f640 1 -- 172.21.15.43:0/1614781768 >> [v2:172.21.15.203:6832/1456699667,v1:172.21.15.203:6833/1456699667] conn(0x7fcf4406a170 msgr2=0x7fcf4406c630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T15:12:26.014 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:26.012+0000 7fcf6750f640 1 --2- 172.21.15.43:0/1614781768 >> [v2:172.21.15.203:6832/1456699667,v1:172.21.15.203:6833/1456699667] conn(0x7fcf4406a170 0x7fcf4406c630 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7fcf5c0079b0 tx=0x7fcf5c009040 comp rx=0 tx=0).stop 2023-08-09T15:12:26.014 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:26.012+0000 7fcf6750f640 1 -- 172.21.15.43:0/1614781768 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fcf60154ab0 msgr2=0x7fcf60155430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T15:12:26.014 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:26.012+0000 7fcf6750f640 1 --2- 172.21.15.43:0/1614781768 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fcf60154ab0 0x7fcf60155430 secure :-1 s=READY pgs=2595 cs=0 l=1 rev1=1 crypto rx=0x7fcf5802c3a0 tx=0x7fcf58004a40 comp rx=0 tx=0).stop 2023-08-09T15:12:26.014 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:26.012+0000 7fcf6750f640 1 -- 172.21.15.43:0/1614781768 shutdown_connections 2023-08-09T15:12:26.014 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:26.012+0000 7fcf6750f640 1 --2- 172.21.15.43:0/1614781768 >> [v2:172.21.15.203:6832/1456699667,v1:172.21.15.203:6833/1456699667] conn(0x7fcf4406a170 0x7fcf4406c630 unknown :-1 s=CLOSED pgs=54 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T15:12:26.014 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:26.012+0000 7fcf6750f640 1 --2- 172.21.15.43:0/1614781768 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fcf60154ab0 0x7fcf60155430 unknown :-1 s=CLOSED pgs=2595 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T15:12:26.015 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:26.012+0000 7fcf6750f640 1 --2- 172.21.15.43:0/1614781768 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fcf600ceda0 0x7fcf60154ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T15:12:26.015 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:26.012+0000 7fcf6750f640 1 --2- 172.21.15.43:0/1614781768 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fcf600b2200 0x7fcf600c96c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T15:12:26.015 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:26.012+0000 7fcf6750f640 1 -- 172.21.15.43:0/1614781768 >> 172.21.15.43:0/1614781768 conn(0x7fcf6006a150 msgr2=0x7fcf60165740 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-09T15:12:26.015 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:26.013+0000 7fcf6750f640 1 -- 172.21.15.43:0/1614781768 shutdown_connections 2023-08-09T15:12:26.016 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:26.013+0000 7fcf6750f640 1 -- 172.21.15.43:0/1614781768 wait complete. 2023-08-09T15:12:26.032 INFO:tasks.ceph:Scrubbing osd.6 2023-08-09T15:12:26.032 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.6 config set osd_debug_deep_scrub_sleep 0 2023-08-09T15:12:26.251 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:26.250+0000 7fee35b02640 1 Processor -- start 2023-08-09T15:12:26.252 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:26.250+0000 7fee35b02640 1 -- start start 2023-08-09T15:12:26.253 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:26.252+0000 7fee35b02640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fee300cbbc0 0x7fee300c8640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T15:12:26.254 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:26.252+0000 7fee35b02640 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7fee30160ea0 0x7fee300c3c00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T15:12:26.254 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:26.252+0000 7fee35b02640 1 -- --> v1:172.21.15.203:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fee30161820 con 0x7fee300c9cb0 2023-08-09T15:12:26.254 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:26.252+0000 7fee35b02640 1 -- --> v2:172.21.15.203:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fee30161b00 con 0x7fee30160ea0 2023-08-09T15:12:26.254 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:26.252+0000 7fee35b02640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fee300c3400 con 0x7fee300cbbc0 2023-08-09T15:12:26.255 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:26.253+0000 7fee35301640 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7fee30160ea0 0x7fee300c3c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T15:12:26.255 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:26.253+0000 7fee35301640 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7fee30160ea0 0x7fee300c3c00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.203:3300/0 says I am v2:172.21.15.43:60172/0 (socket says 172.21.15.43:60172) 2023-08-09T15:12:26.255 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:26.253+0000 7fee35301640 1 -- 172.21.15.43:0/1285247387 learned_addr learned my addr 172.21.15.43:0/1285247387 (peer_addr_for_me v2:172.21.15.43:0/0) 2023-08-09T15:12:26.255 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:26.253+0000 7fee34b00640 1 --1- 172.21.15.43:0/1285247387 >> v1:172.21.15.203:6789/0 conn(0x7fee300c9cb0 0x7fee300ca090 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.203:6789/0 says I am v1:172.21.15.43:50970/0 (socket says 172.21.15.43:50970) 2023-08-09T15:12:26.256 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:26.253+0000 7fee2ffff640 1 --2- 172.21.15.43:0/1285247387 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fee300cbbc0 0x7fee300c8640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T15:12:26.256 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:26.254+0000 7fee35301640 1 -- 172.21.15.43:0/1285247387 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fee300cbbc0 msgr2=0x7fee300c8640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-09T15:12:26.256 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:26.254+0000 7fee35301640 1 --2- 172.21.15.43:0/1285247387 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fee300cbbc0 0x7fee300c8640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T15:12:26.256 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:26.254+0000 7fee35301640 1 -- 172.21.15.43:0/1285247387 >> v1:172.21.15.203:6789/0 conn(0x7fee300c9cb0 legacy=0x7fee300ca090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T15:12:26.256 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:26.254+0000 7fee35301640 1 -- 172.21.15.43:0/1285247387 --> v2:172.21.15.203:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fee300c36e0 con 0x7fee30160ea0 2023-08-09T15:12:26.257 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:26.255+0000 7fee35301640 1 --2- 172.21.15.43:0/1285247387 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fee30160ea0 0x7fee300c3c00 secure :-1 s=READY pgs=944 cs=0 l=1 rev1=1 crypto rx=0x7fee28009e50 tx=0x7fee2802bf50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=d35d84bdf3f90238 server_cookie=0 in_seq=0 out_seq=0 2023-08-09T15:12:26.257 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:26.256+0000 7fee2f7fe640 1 -- 172.21.15.43:0/1285247387 <== mon.1 v2:172.21.15.203:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fee2802caf0 con 0x7fee30160ea0 2023-08-09T15:12:26.257 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:26.256+0000 7fee2f7fe640 1 -- 172.21.15.43:0/1285247387 <== mon.1 v2:172.21.15.203:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fee2802cc50 con 0x7fee30160ea0 2023-08-09T15:12:26.258 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:26.256+0000 7fee2f7fe640 1 -- 172.21.15.43:0/1285247387 <== mon.1 v2:172.21.15.203:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fee280343f0 con 0x7fee30160ea0 2023-08-09T15:12:26.258 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:26.257+0000 7fee35b02640 1 -- 172.21.15.43:0/1285247387 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fee30160ea0 msgr2=0x7fee300c3c00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T15:12:26.258 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:26.257+0000 7fee35b02640 1 --2- 172.21.15.43:0/1285247387 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fee30160ea0 0x7fee300c3c00 secure :-1 s=READY pgs=944 cs=0 l=1 rev1=1 crypto rx=0x7fee28009e50 tx=0x7fee2802bf50 comp rx=0 tx=0).stop 2023-08-09T15:12:26.259 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:26.257+0000 7fee35b02640 1 -- 172.21.15.43:0/1285247387 shutdown_connections 2023-08-09T15:12:26.259 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:26.257+0000 7fee35b02640 1 --2- 172.21.15.43:0/1285247387 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fee30160ea0 0x7fee300c3c00 unknown :-1 s=CLOSED pgs=944 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T15:12:26.259 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:26.257+0000 7fee35b02640 1 --2- 172.21.15.43:0/1285247387 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fee300cbbc0 0x7fee300c8640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T15:12:26.259 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:26.257+0000 7fee35b02640 1 -- 172.21.15.43:0/1285247387 >> 172.21.15.43:0/1285247387 conn(0x7fee300cdff0 msgr2=0x7fee300ce3d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-09T15:12:26.259 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:26.257+0000 7fee35b02640 1 -- 172.21.15.43:0/1285247387 shutdown_connections 2023-08-09T15:12:26.259 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:26.257+0000 7fee35b02640 1 -- 172.21.15.43:0/1285247387 wait complete. 2023-08-09T15:12:26.260 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:26.259+0000 7fee35b02640 1 Processor -- start 2023-08-09T15:12:26.260 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:26.259+0000 7fee35b02640 1 -- start start 2023-08-09T15:12:26.262 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:26.260+0000 7fee35b02640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fee300c9cb0 0x7fee30163fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T15:12:26.262 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:26.260+0000 7fee35b02640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fee300cbbc0 0x7fee30152d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T15:12:26.262 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:26.260+0000 7fee35b02640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fee30160ea0 0x7fee30153290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T15:12:26.262 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:26.260+0000 7fee35b02640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fee300b1a10 con 0x7fee300cbbc0 2023-08-09T15:12:26.262 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:26.260+0000 7fee35b02640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fee300b0dd0 con 0x7fee300c9cb0 2023-08-09T15:12:26.263 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:26.260+0000 7fee35b02640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fee300b1070 con 0x7fee30160ea0 2023-08-09T15:12:26.263 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:26.260+0000 7fee2ffff640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fee300cbbc0 0x7fee30152d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T15:12:26.263 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:26.260+0000 7fee2ffff640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fee300cbbc0 0x7fee30152d50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:53120/0 (socket says 172.21.15.43:53120) 2023-08-09T15:12:26.263 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:26.260+0000 7fee2ffff640 1 -- 172.21.15.43:0/3099353451 learned_addr learned my addr 172.21.15.43:0/3099353451 (peer_addr_for_me v2:172.21.15.43:0/0) 2023-08-09T15:12:26.263 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:26.261+0000 7fee34b00640 1 --2- 172.21.15.43:0/3099353451 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fee300c9cb0 0x7fee30163fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T15:12:26.263 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:26.261+0000 7fee35301640 1 --2- 172.21.15.43:0/3099353451 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fee30160ea0 0x7fee30153290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T15:12:26.264 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:26.261+0000 7fee2ffff640 1 -- 172.21.15.43:0/3099353451 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fee30160ea0 msgr2=0x7fee30153290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T15:12:26.265 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:26.261+0000 7fee2ffff640 1 --2- 172.21.15.43:0/3099353451 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fee30160ea0 0x7fee30153290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T15:12:26.265 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:26.261+0000 7fee2ffff640 1 -- 172.21.15.43:0/3099353451 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fee300c9cb0 msgr2=0x7fee30163fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T15:12:26.265 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:26.261+0000 7fee2ffff640 1 --2- 172.21.15.43:0/3099353451 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fee300c9cb0 0x7fee30163fb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T15:12:26.265 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:26.261+0000 7fee2ffff640 1 -- 172.21.15.43:0/3099353451 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fee28009b30 con 0x7fee300cbbc0 2023-08-09T15:12:26.265 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:26.261+0000 7fee2ffff640 1 --2- 172.21.15.43:0/3099353451 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fee300cbbc0 0x7fee30152d50 secure :-1 s=READY pgs=2596 cs=0 l=1 rev1=1 crypto rx=0x7fee1c00e9b0 tx=0x7fee1c00ee70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-09T15:12:26.265 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:26.262+0000 7fee2d7fa640 1 -- 172.21.15.43:0/3099353451 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fee1c009670 con 0x7fee300cbbc0 2023-08-09T15:12:26.265 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:26.262+0000 7fee2d7fa640 1 -- 172.21.15.43:0/3099353451 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fee1c00c9a0 con 0x7fee300cbbc0 2023-08-09T15:12:26.266 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:26.262+0000 7fee2d7fa640 1 -- 172.21.15.43:0/3099353451 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fee1c00cd50 con 0x7fee300cbbc0 2023-08-09T15:12:26.266 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:26.262+0000 7fee35b02640 1 -- 172.21.15.43:0/3099353451 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fee300b1350 con 0x7fee300cbbc0 2023-08-09T15:12:26.266 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:26.262+0000 7fee35b02640 1 -- 172.21.15.43:0/3099353451 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fee301644f0 con 0x7fee300cbbc0 2023-08-09T15:12:26.266 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:26.263+0000 7fee35b02640 1 -- 172.21.15.43:0/3099353451 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7fee30161de0 con 0x7fee300cbbc0 2023-08-09T15:12:26.267 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:26.264+0000 7fee2d7fa640 1 -- 172.21.15.43:0/3099353451 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 129) v1 ==== 88443+0+0 (secure 0 0 0) 0x7fee1c013470 con 0x7fee300cbbc0 2023-08-09T15:12:26.267 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:26.265+0000 7fee2d7fa640 1 --2- 172.21.15.43:0/3099353451 >> [v2:172.21.15.203:6832/1456699667,v1:172.21.15.203:6833/1456699667] conn(0x7fee1406a1a0 0x7fee1406c660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T15:12:26.267 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:26.265+0000 7fee2d7fa640 1 -- 172.21.15.43:0/3099353451 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(229..229 src has 1..229) v4 ==== 6462+0+0 (secure 0 0 0) 0x7fee1c088ae0 con 0x7fee300cbbc0 2023-08-09T15:12:26.267 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:26.265+0000 7fee2d7fa640 1 --2- 172.21.15.43:0/3099353451 >> [v2:172.21.15.203:6824/512155569,v1:172.21.15.203:6825/512155569] conn(0x7fee14071180 0x7fee140735e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T15:12:26.268 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:26.265+0000 7fee2d7fa640 1 -- 172.21.15.43:0/3099353451 --> [v2:172.21.15.203:6824/512155569,v1:172.21.15.203:6825/512155569] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7fee14073cb0 con 0x7fee14071180 2023-08-09T15:12:26.268 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:26.265+0000 7fee2d7fa640 1 -- 172.21.15.43:0/3099353451 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_get_version_reply(handle=1 version=229) v2 ==== 24+0+0 (secure 0 0 0) 0x7fee1c08d2c0 con 0x7fee300cbbc0 2023-08-09T15:12:26.268 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:26.266+0000 7fee34b00640 1 --2- 172.21.15.43:0/3099353451 >> [v2:172.21.15.203:6832/1456699667,v1:172.21.15.203:6833/1456699667] conn(0x7fee1406a1a0 0x7fee1406c660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T15:12:26.268 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:26.266+0000 7fee35301640 1 --2- 172.21.15.43:0/3099353451 >> [v2:172.21.15.203:6824/512155569,v1:172.21.15.203:6825/512155569] conn(0x7fee14071180 0x7fee140735e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T15:12:26.269 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:26.267+0000 7fee34b00640 1 --2- 172.21.15.43:0/3099353451 >> [v2:172.21.15.203:6832/1456699667,v1:172.21.15.203:6833/1456699667] conn(0x7fee1406a1a0 0x7fee1406c660 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7fee24000f60 tx=0x7fee24007c80 comp rx=0 tx=0).ready entity=mgr.9232 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-09T15:12:26.269 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:26.267+0000 7fee35301640 1 --2- 172.21.15.43:0/3099353451 >> [v2:172.21.15.203:6824/512155569,v1:172.21.15.203:6825/512155569] conn(0x7fee14071180 0x7fee140735e0 crc :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-09T15:12:26.270 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:26.269+0000 7fee2d7fa640 1 -- 172.21.15.43:0/3099353451 <== osd.6 v2:172.21.15.203:6824/512155569 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+25607 (crc 0 0 0) 0x7fee14073cb0 con 0x7fee14071180 2023-08-09T15:12:26.285 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:26.284+0000 7fee35b02640 1 -- 172.21.15.43:0/3099353451 --> [v2:172.21.15.203:6824/512155569,v1:172.21.15.203:6825/512155569] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) v1 -- 0x7fee3004e390 con 0x7fee14071180 2023-08-09T15:12:26.286 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:26.285+0000 7fee2d7fa640 1 -- 172.21.15.43:0/3099353451 <== osd.6 v2:172.21.15.203:6824/512155569 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+105 (crc 0 0 0) 0x7fee3004e390 con 0x7fee14071180 2023-08-09T15:12:26.286 INFO:teuthology.orchestra.run.smithi043.stdout:{ 2023-08-09T15:12:26.287 INFO:teuthology.orchestra.run.smithi043.stdout: "success": "osd_debug_deep_scrub_sleep = '0.000000' (not observed, change may require restart) " 2023-08-09T15:12:26.287 INFO:teuthology.orchestra.run.smithi043.stdout:} 2023-08-09T15:12:26.287 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:26.285+0000 7fee35b02640 1 -- 172.21.15.43:0/3099353451 >> [v2:172.21.15.203:6824/512155569,v1:172.21.15.203:6825/512155569] conn(0x7fee14071180 msgr2=0x7fee140735e0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T15:12:26.288 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:26.285+0000 7fee35b02640 1 --2- 172.21.15.43:0/3099353451 >> [v2:172.21.15.203:6824/512155569,v1:172.21.15.203:6825/512155569] conn(0x7fee14071180 0x7fee140735e0 crc :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T15:12:26.288 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:26.285+0000 7fee35b02640 1 -- 172.21.15.43:0/3099353451 >> [v2:172.21.15.203:6832/1456699667,v1:172.21.15.203:6833/1456699667] conn(0x7fee1406a1a0 msgr2=0x7fee1406c660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T15:12:26.288 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:26.285+0000 7fee35b02640 1 --2- 172.21.15.43:0/3099353451 >> [v2:172.21.15.203:6832/1456699667,v1:172.21.15.203:6833/1456699667] conn(0x7fee1406a1a0 0x7fee1406c660 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7fee24000f60 tx=0x7fee24007c80 comp rx=0 tx=0).stop 2023-08-09T15:12:26.288 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:26.285+0000 7fee35b02640 1 -- 172.21.15.43:0/3099353451 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fee300cbbc0 msgr2=0x7fee30152d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T15:12:26.288 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:26.285+0000 7fee35b02640 1 --2- 172.21.15.43:0/3099353451 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fee300cbbc0 0x7fee30152d50 secure :-1 s=READY pgs=2596 cs=0 l=1 rev1=1 crypto rx=0x7fee1c00e9b0 tx=0x7fee1c00ee70 comp rx=0 tx=0).stop 2023-08-09T15:12:26.289 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:26.285+0000 7fee35301640 1 -- 172.21.15.43:0/3099353451 reap_dead start 2023-08-09T15:12:26.289 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:26.286+0000 7fee35b02640 1 -- 172.21.15.43:0/3099353451 shutdown_connections 2023-08-09T15:12:26.289 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:26.286+0000 7fee35b02640 1 -- 172.21.15.43:0/3099353451 >> 172.21.15.43:0/3099353451 conn(0x7fee300cdff0 msgr2=0x7fee3006f4f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-09T15:12:26.289 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:26.286+0000 7fee35b02640 1 -- 172.21.15.43:0/3099353451 shutdown_connections 2023-08-09T15:12:26.289 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:26.286+0000 7fee35b02640 1 -- 172.21.15.43:0/3099353451 wait complete. 2023-08-09T15:12:26.306 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 6 2023-08-09T15:12:26.541 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:26.539+0000 7fb0447b7640 1 Processor -- start 2023-08-09T15:12:26.541 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:26.539+0000 7fb0447b7640 1 -- start start 2023-08-09T15:12:26.542 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:26.540+0000 7fb0447b7640 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7fb03c0b0390 0x7fb03c0ab860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T15:12:26.542 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:26.540+0000 7fb0447b7640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7fb03c0ad570 0x7fb03c0abda0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T15:12:26.542 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:26.541+0000 7fb0447b7640 1 -- --> v1:172.21.15.43:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb03c0ac2e0 con 0x7fb03c15c980 2023-08-09T15:12:26.543 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:26.541+0000 7fb0447b7640 1 -- --> v2:172.21.15.43:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb03c15cd70 con 0x7fb03c0ad570 2023-08-09T15:12:26.543 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:26.541+0000 7fb0447b7640 1 -- --> v2:172.21.15.203:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb03c161220 con 0x7fb03c0b0390 2023-08-09T15:12:26.543 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:26.541+0000 7fb042553640 1 --1- >> v1:172.21.15.43:6789/0 conn(0x7fb03c15c980 0x7fb03c0afc80 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.43:6789/0 says I am v1:172.21.15.43:45416/0 (socket says 172.21.15.43:45416) 2023-08-09T15:12:26.543 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:26.541+0000 7fb042553640 1 -- 172.21.15.43:0/189523714 learned_addr learned my addr 172.21.15.43:0/189523714 (peer_addr_for_me v1:172.21.15.43:0/0) 2023-08-09T15:12:26.543 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:26.541+0000 7fb042d54640 1 --2- 172.21.15.43:0/189523714 >> v2:172.21.15.43:3300/0 conn(0x7fb03c0ad570 0x7fb03c0abda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T15:12:26.544 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:26.541+0000 7fb041d52640 1 --2- 172.21.15.43:0/189523714 >> v2:172.21.15.203:3300/0 conn(0x7fb03c0b0390 0x7fb03c0ab860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T15:12:26.544 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:26.541+0000 7fb042d54640 1 -- 172.21.15.43:0/189523714 >> v2:172.21.15.203:3300/0 conn(0x7fb03c0b0390 msgr2=0x7fb03c0ab860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-09T15:12:26.544 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:26.541+0000 7fb042d54640 1 --2- 172.21.15.43:0/189523714 >> v2:172.21.15.203:3300/0 conn(0x7fb03c0b0390 0x7fb03c0ab860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T15:12:26.544 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:26.541+0000 7fb042d54640 1 -- 172.21.15.43:0/189523714 >> v1:172.21.15.43:6789/0 conn(0x7fb03c15c980 legacy=0x7fb03c0afc80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T15:12:26.544 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:26.541+0000 7fb041551640 1 -- 172.21.15.43:0/189523714 <== mon.0 v1:172.21.15.43:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2127752518 0 0) 0x7fb03c0ac2e0 con 0x7fb03c15c980 2023-08-09T15:12:26.545 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:26.541+0000 7fb042d54640 1 -- 172.21.15.43:0/189523714 --> v2:172.21.15.43:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb03c1606a0 con 0x7fb03c0ad570 2023-08-09T15:12:26.545 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:26.543+0000 7fb042d54640 1 --2- 172.21.15.43:0/189523714 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb03c0ad570 0x7fb03c0abda0 secure :-1 s=READY pgs=2598 cs=0 l=1 rev1=1 crypto rx=0x7fb03c06c860 tx=0x7fb03402bed0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=18eb61efa72c536 server_cookie=0 in_seq=0 out_seq=0 2023-08-09T15:12:26.546 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:26.543+0000 7fb041551640 1 -- 172.21.15.43:0/189523714 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb03402cc80 con 0x7fb03c0ad570 2023-08-09T15:12:26.546 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:26.543+0000 7fb041551640 1 -- 172.21.15.43:0/189523714 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb03402cde0 con 0x7fb03c0ad570 2023-08-09T15:12:26.546 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:26.543+0000 7fb041551640 1 -- 172.21.15.43:0/189523714 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb034034660 con 0x7fb03c0ad570 2023-08-09T15:12:26.547 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:26.544+0000 7fb0447b7640 1 -- 172.21.15.43:0/189523714 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb03c0ad570 msgr2=0x7fb03c0abda0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T15:12:26.547 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:26.544+0000 7fb0447b7640 1 --2- 172.21.15.43:0/189523714 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb03c0ad570 0x7fb03c0abda0 secure :-1 s=READY pgs=2598 cs=0 l=1 rev1=1 crypto rx=0x7fb03c06c860 tx=0x7fb03402bed0 comp rx=0 tx=0).stop 2023-08-09T15:12:26.547 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:26.544+0000 7fb0447b7640 1 -- 172.21.15.43:0/189523714 shutdown_connections 2023-08-09T15:12:26.547 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:26.544+0000 7fb0447b7640 1 --2- 172.21.15.43:0/189523714 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb03c0ad570 0x7fb03c0abda0 unknown :-1 s=CLOSED pgs=2598 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T15:12:26.547 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:26.544+0000 7fb0447b7640 1 --2- 172.21.15.43:0/189523714 >> v2:172.21.15.203:3300/0 conn(0x7fb03c0b0390 0x7fb03c0ab860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T15:12:26.547 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:26.544+0000 7fb0447b7640 1 -- 172.21.15.43:0/189523714 >> 172.21.15.43:0/189523714 conn(0x7fb03c069dd0 msgr2=0x7fb03c0cac80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-09T15:12:26.547 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:26.544+0000 7fb0447b7640 1 -- 172.21.15.43:0/189523714 shutdown_connections 2023-08-09T15:12:26.548 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:26.544+0000 7fb0447b7640 1 -- 172.21.15.43:0/189523714 wait complete. 2023-08-09T15:12:26.548 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:26.546+0000 7fb0447b7640 1 Processor -- start 2023-08-09T15:12:26.549 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:26.546+0000 7fb0447b7640 1 -- start start 2023-08-09T15:12:26.549 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:26.547+0000 7fb0447b7640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fb03c0ad570 0x7fb03c0c3f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T15:12:26.549 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:26.547+0000 7fb0447b7640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb03c0b0390 0x7fb03c0b0d70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T15:12:26.549 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:26.547+0000 7fb0447b7640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fb03c0b12b0 0x7fb03c0ae670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T15:12:26.549 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:26.547+0000 7fb0447b7640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb03c0b1730 con 0x7fb03c0b0390 2023-08-09T15:12:26.549 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:26.547+0000 7fb0447b7640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb03c0c44d0 con 0x7fb03c0b12b0 2023-08-09T15:12:26.550 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:26.547+0000 7fb0447b7640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb03c15d670 con 0x7fb03c0ad570 2023-08-09T15:12:26.550 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:26.547+0000 7fb041d52640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb03c0b0390 0x7fb03c0b0d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T15:12:26.550 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:26.547+0000 7fb041d52640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb03c0b0390 0x7fb03c0b0d70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:53140/0 (socket says 172.21.15.43:53140) 2023-08-09T15:12:26.550 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:26.547+0000 7fb041d52640 1 -- 172.21.15.43:0/2486142124 learned_addr learned my addr 172.21.15.43:0/2486142124 (peer_addr_for_me v2:172.21.15.43:0/0) 2023-08-09T15:12:26.551 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:26.548+0000 7fb042d54640 1 --2- 172.21.15.43:0/2486142124 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fb03c0b12b0 0x7fb03c0ae670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T15:12:26.552 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:26.548+0000 7fb042553640 1 --2- 172.21.15.43:0/2486142124 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fb03c0ad570 0x7fb03c0c3f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T15:12:26.552 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:26.548+0000 7fb041d52640 1 -- 172.21.15.43:0/2486142124 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fb03c0ad570 msgr2=0x7fb03c0c3f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T15:12:26.552 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:26.548+0000 7fb041d52640 1 --2- 172.21.15.43:0/2486142124 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fb03c0ad570 0x7fb03c0c3f90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T15:12:26.553 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:26.548+0000 7fb041d52640 1 -- 172.21.15.43:0/2486142124 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fb03c0b12b0 msgr2=0x7fb03c0ae670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T15:12:26.553 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:26.548+0000 7fb041d52640 1 --2- 172.21.15.43:0/2486142124 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fb03c0b12b0 0x7fb03c0ae670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T15:12:26.553 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:26.548+0000 7fb041d52640 1 -- 172.21.15.43:0/2486142124 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb034009a70 con 0x7fb03c0b0390 2023-08-09T15:12:26.553 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:26.548+0000 7fb042d54640 1 --2- 172.21.15.43:0/2486142124 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fb03c0b12b0 0x7fb03c0ae670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-08-09T15:12:26.553 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:26.548+0000 7fb042553640 1 --2- 172.21.15.43:0/2486142124 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fb03c0ad570 0x7fb03c0c3f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-08-09T15:12:26.553 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:26.548+0000 7fb041d52640 1 --2- 172.21.15.43:0/2486142124 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb03c0b0390 0x7fb03c0b0d70 secure :-1 s=READY pgs=2599 cs=0 l=1 rev1=1 crypto rx=0x7fb02c00d800 tx=0x7fb02c00dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-09T15:12:26.554 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:26.548+0000 7fb032ffd640 1 -- 172.21.15.43:0/2486142124 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb02c00b660 con 0x7fb03c0b0390 2023-08-09T15:12:26.554 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:26.549+0000 7fb032ffd640 1 -- 172.21.15.43:0/2486142124 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb02c00b7c0 con 0x7fb03c0b0390 2023-08-09T15:12:26.554 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:26.549+0000 7fb032ffd640 1 -- 172.21.15.43:0/2486142124 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb02c00bb50 con 0x7fb03c0b0390 2023-08-09T15:12:26.554 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:26.549+0000 7fb0447b7640 1 -- 172.21.15.43:0/2486142124 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb03c15df10 con 0x7fb03c0b0390 2023-08-09T15:12:26.554 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:26.549+0000 7fb0447b7640 1 -- 172.21.15.43:0/2486142124 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb03c15d8c0 con 0x7fb03c0b0390 2023-08-09T15:12:26.554 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:26.550+0000 7fb0447b7640 1 -- 172.21.15.43:0/2486142124 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb03c15e0a0 con 0x7fb03c0b0390 2023-08-09T15:12:26.555 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:26.551+0000 7fb032ffd640 1 -- 172.21.15.43:0/2486142124 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 129) v1 ==== 88443+0+0 (secure 0 0 0) 0x7fb02c00bcb0 con 0x7fb03c0b0390 2023-08-09T15:12:26.555 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:26.552+0000 7fb032ffd640 1 --2- 172.21.15.43:0/2486142124 >> [v2:172.21.15.203:6832/1456699667,v1:172.21.15.203:6833/1456699667] conn(0x7fb02006a170 0x7fb02006c630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T15:12:26.556 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:26.552+0000 7fb032ffd640 1 -- 172.21.15.43:0/2486142124 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(229..229 src has 1..229) v4 ==== 6462+0+0 (secure 0 0 0) 0x7fb02c088d00 con 0x7fb03c0b0390 2023-08-09T15:12:26.556 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:26.552+0000 7fb042553640 1 --2- 172.21.15.43:0/2486142124 >> [v2:172.21.15.203:6832/1456699667,v1:172.21.15.203:6833/1456699667] conn(0x7fb02006a170 0x7fb02006c630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T15:12:26.556 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:26.554+0000 7fb042553640 1 --2- 172.21.15.43:0/2486142124 >> [v2:172.21.15.203:6832/1456699667,v1:172.21.15.203:6833/1456699667] conn(0x7fb02006a170 0x7fb02006c630 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7fb038000f60 tx=0x7fb038009040 comp rx=0 tx=0).ready entity=mgr.9232 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-09T15:12:26.557 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:26.556+0000 7fb032ffd640 1 -- 172.21.15.43:0/2486142124 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178066 (secure 0 0 0) 0x7fb02c0541e0 con 0x7fb03c0b0390 2023-08-09T15:12:26.751 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:26.750+0000 7fb0447b7640 1 -- 172.21.15.43:0/2486142124 --> [v2:172.21.15.203:6832/1456699667,v1:172.21.15.203:6833/1456699667] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "6", "target": ["mon-mgr", ""]}) v1 -- 0x7fb03c160c60 con 0x7fb02006a170 2023-08-09T15:12:26.752 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:26.752+0000 7fb032ffd640 1 -- 172.21.15.43:0/2486142124 <== mgr.9232 v2:172.21.15.203:6832/1456699667 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 6 to deep-scrub) v1 ==== 41+0+0 (secure 0 0 0) 0x7fb038001c90 con 0x7fb02006a170 2023-08-09T15:12:26.753 INFO:teuthology.orchestra.run.smithi043.stderr:instructed osd(s) 6 to deep-scrub 2023-08-09T15:12:26.755 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:26.753+0000 7fb0447b7640 1 -- 172.21.15.43:0/2486142124 >> [v2:172.21.15.203:6832/1456699667,v1:172.21.15.203:6833/1456699667] conn(0x7fb02006a170 msgr2=0x7fb02006c630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T15:12:26.756 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:26.753+0000 7fb0447b7640 1 --2- 172.21.15.43:0/2486142124 >> [v2:172.21.15.203:6832/1456699667,v1:172.21.15.203:6833/1456699667] conn(0x7fb02006a170 0x7fb02006c630 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7fb038000f60 tx=0x7fb038009040 comp rx=0 tx=0).stop 2023-08-09T15:12:26.756 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:26.753+0000 7fb0447b7640 1 -- 172.21.15.43:0/2486142124 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb03c0b0390 msgr2=0x7fb03c0b0d70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T15:12:26.756 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:26.753+0000 7fb0447b7640 1 --2- 172.21.15.43:0/2486142124 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb03c0b0390 0x7fb03c0b0d70 secure :-1 s=READY pgs=2599 cs=0 l=1 rev1=1 crypto rx=0x7fb02c00d800 tx=0x7fb02c00dcc0 comp rx=0 tx=0).stop 2023-08-09T15:12:26.756 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:26.754+0000 7fb0447b7640 1 -- 172.21.15.43:0/2486142124 shutdown_connections 2023-08-09T15:12:26.756 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:26.754+0000 7fb0447b7640 1 --2- 172.21.15.43:0/2486142124 >> [v2:172.21.15.203:6832/1456699667,v1:172.21.15.203:6833/1456699667] conn(0x7fb02006a170 0x7fb02006c630 unknown :-1 s=CLOSED pgs=56 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T15:12:26.757 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:26.754+0000 7fb0447b7640 1 --2- 172.21.15.43:0/2486142124 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fb03c0b12b0 0x7fb03c0ae670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T15:12:26.757 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:26.754+0000 7fb0447b7640 1 --2- 172.21.15.43:0/2486142124 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb03c0b0390 0x7fb03c0b0d70 unknown :-1 s=CLOSED pgs=2599 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T15:12:26.757 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:26.754+0000 7fb0447b7640 1 --2- 172.21.15.43:0/2486142124 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fb03c0ad570 0x7fb03c0c3f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T15:12:26.757 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:26.754+0000 7fb0447b7640 1 -- 172.21.15.43:0/2486142124 >> 172.21.15.43:0/2486142124 conn(0x7fb03c069dd0 msgr2=0x7fb03c0caba0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-09T15:12:26.758 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:26.754+0000 7fb0447b7640 1 -- 172.21.15.43:0/2486142124 shutdown_connections 2023-08-09T15:12:26.758 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:26.754+0000 7fb0447b7640 1 -- 172.21.15.43:0/2486142124 wait complete. 2023-08-09T15:12:26.775 INFO:tasks.ceph:Scrubbing osd.7 2023-08-09T15:12:26.776 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.7 config set osd_debug_deep_scrub_sleep 0 2023-08-09T15:12:27.013 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:27.012+0000 7fb51da04640 1 Processor -- start 2023-08-09T15:12:27.014 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:27.012+0000 7fb51da04640 1 -- start start 2023-08-09T15:12:27.015 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:27.013+0000 7fb51da04640 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7fb51815c750 0x7fb5180cf7c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T15:12:27.015 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:27.014+0000 7fb51da04640 1 -- --> v1:172.21.15.43:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb51806a800 con 0x7fb5180d2de0 2023-08-09T15:12:27.015 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:27.014+0000 7fb51da04640 1 -- --> v1:172.21.15.203:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb5180d0540 con 0x7fb5180d12c0 2023-08-09T15:12:27.015 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:27.014+0000 7fb51da04640 1 -- --> v2:172.21.15.203:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb51806ae30 con 0x7fb51815c750 2023-08-09T15:12:27.016 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:27.014+0000 7fb5167fc640 1 --1- >> v1:172.21.15.43:6789/0 conn(0x7fb5180d2de0 0x7fb5180d0bb0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.43:6789/0 says I am v1:172.21.15.43:45428/0 (socket says 172.21.15.43:45428) 2023-08-09T15:12:27.016 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:27.014+0000 7fb5167fc640 1 -- 172.21.15.43:0/4119364184 learned_addr learned my addr 172.21.15.43:0/4119364184 (peer_addr_for_me v1:172.21.15.43:0/0) 2023-08-09T15:12:27.016 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:27.014+0000 7fb516ffd640 1 --2- 172.21.15.43:0/4119364184 >> v2:172.21.15.203:3300/0 conn(0x7fb51815c750 0x7fb5180cf7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T15:12:27.016 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:27.015+0000 7fb515ffb640 1 -- 172.21.15.43:0/4119364184 <== mon.0 v1:172.21.15.43:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3660815027 0 0) 0x7fb51806a800 con 0x7fb5180d2de0 2023-08-09T15:12:27.016 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:27.015+0000 7fb515ffb640 1 -- 172.21.15.43:0/4119364184 --> v1:172.21.15.43:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fb500003520 con 0x7fb5180d2de0 2023-08-09T15:12:27.018 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:27.015+0000 7fb515ffb640 1 -- 172.21.15.43:0/4119364184 <== mon.0 v1:172.21.15.43:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 341246475 0 0) 0x7fb500003520 con 0x7fb5180d2de0 2023-08-09T15:12:27.018 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:27.015+0000 7fb515ffb640 1 -- 172.21.15.43:0/4119364184 >> v2:172.21.15.203:3300/0 conn(0x7fb51815c750 msgr2=0x7fb5180cf7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-09T15:12:27.018 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:27.015+0000 7fb515ffb640 1 --2- 172.21.15.43:0/4119364184 >> v2:172.21.15.203:3300/0 conn(0x7fb51815c750 0x7fb5180cf7c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T15:12:27.018 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:27.015+0000 7fb515ffb640 1 -- 172.21.15.43:0/4119364184 >> v1:172.21.15.203:6789/0 conn(0x7fb5180d12c0 legacy=0x7fb51806b230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T15:12:27.018 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:27.015+0000 7fb515ffb640 1 -- 172.21.15.43:0/4119364184 --> v1:172.21.15.43:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb5180d0820 con 0x7fb5180d2de0 2023-08-09T15:12:27.018 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:27.015+0000 7fb515ffb640 1 -- 172.21.15.43:0/4119364184 <== mon.0 v1:172.21.15.43:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1285150740 0 0) 0x7fb50c001670 con 0x7fb5180d2de0 2023-08-09T15:12:27.019 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:27.015+0000 7fb515ffb640 1 -- 172.21.15.43:0/4119364184 >> v1:172.21.15.43:6789/0 conn(0x7fb5180d2de0 legacy=0x7fb5180d0bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T15:12:27.019 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:27.015+0000 7fb515ffb640 1 --2- 172.21.15.43:0/4119364184 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fb500003970 0x7fb500003d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T15:12:27.019 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:27.015+0000 7fb515ffb640 1 --2- 172.21.15.43:0/4119364184 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb5000047c0 0x7fb500004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T15:12:27.019 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:27.015+0000 7fb515ffb640 1 --2- 172.21.15.43:0/4119364184 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fb5000052a0 0x7fb50000b980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T15:12:27.019 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:27.015+0000 7fb515ffb640 1 -- 172.21.15.43:0/4119364184 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb51806ae30 con 0x7fb5000047c0 2023-08-09T15:12:27.019 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:27.015+0000 7fb515ffb640 1 -- 172.21.15.43:0/4119364184 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb50000c170 con 0x7fb500003970 2023-08-09T15:12:27.019 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:27.015+0000 7fb515ffb640 1 -- 172.21.15.43:0/4119364184 --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb50000c330 con 0x7fb5000052a0 2023-08-09T15:12:27.020 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:27.015+0000 7fb516ffd640 1 --2- 172.21.15.43:0/4119364184 >> v2:172.21.15.203:3300/0 conn(0x7fb51815c750 0x7fb5180cf7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-08-09T15:12:27.020 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:27.015+0000 7fb5167fc640 1 --2- 172.21.15.43:0/4119364184 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb5000047c0 0x7fb500004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T15:12:27.020 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:27.016+0000 7fb5177fe640 1 --2- 172.21.15.43:0/4119364184 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fb5000052a0 0x7fb50000b980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T15:12:27.020 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:27.016+0000 7fb5167fc640 1 -- 172.21.15.43:0/4119364184 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fb5000052a0 msgr2=0x7fb50000b980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-09T15:12:27.020 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:27.016+0000 7fb5167fc640 1 --2- 172.21.15.43:0/4119364184 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fb5000052a0 0x7fb50000b980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T15:12:27.020 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:27.016+0000 7fb5167fc640 1 -- 172.21.15.43:0/4119364184 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fb500003970 msgr2=0x7fb500003d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-09T15:12:27.020 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:27.016+0000 7fb5167fc640 1 --2- 172.21.15.43:0/4119364184 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fb500003970 0x7fb500003d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T15:12:27.020 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:27.016+0000 7fb5167fc640 1 -- 172.21.15.43:0/4119364184 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb500003520 con 0x7fb5000047c0 2023-08-09T15:12:27.021 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:27.016+0000 7fb5177fe640 1 --2- 172.21.15.43:0/4119364184 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fb5000052a0 0x7fb50000b980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-08-09T15:12:27.021 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:27.016+0000 7fb5177fe640 1 -- 172.21.15.43:0/4119364184 reap_dead start 2023-08-09T15:12:27.022 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:27.017+0000 7fb5167fc640 1 --2- 172.21.15.43:0/4119364184 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb5000047c0 0x7fb500004c40 secure :-1 s=READY pgs=2601 cs=0 l=1 rev1=1 crypto rx=0x7fb51806a070 tx=0x7fb5000057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=70adfacb99a25d26 server_cookie=0 in_seq=0 out_seq=0 2023-08-09T15:12:27.022 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:27.017+0000 7fb515ffb640 1 -- 172.21.15.43:0/4119364184 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb50c005ad0 con 0x7fb5000047c0 2023-08-09T15:12:27.022 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:27.017+0000 7fb515ffb640 1 -- 172.21.15.43:0/4119364184 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb50c005c30 con 0x7fb5000047c0 2023-08-09T15:12:27.022 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:27.017+0000 7fb515ffb640 1 -- 172.21.15.43:0/4119364184 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb50c02ca50 con 0x7fb5000047c0 2023-08-09T15:12:27.022 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:27.018+0000 7fb51da04640 1 -- 172.21.15.43:0/4119364184 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb5000047c0 msgr2=0x7fb500004c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T15:12:27.022 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:27.018+0000 7fb51da04640 1 --2- 172.21.15.43:0/4119364184 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb5000047c0 0x7fb500004c40 secure :-1 s=READY pgs=2601 cs=0 l=1 rev1=1 crypto rx=0x7fb51806a070 tx=0x7fb5000057f0 comp rx=0 tx=0).stop 2023-08-09T15:12:27.023 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:27.018+0000 7fb51da04640 1 -- 172.21.15.43:0/4119364184 shutdown_connections 2023-08-09T15:12:27.023 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:27.018+0000 7fb51da04640 1 --2- 172.21.15.43:0/4119364184 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb5000047c0 0x7fb500004c40 unknown :-1 s=CLOSED pgs=2601 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T15:12:27.023 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:27.018+0000 7fb51da04640 1 -- 172.21.15.43:0/4119364184 >> 172.21.15.43:0/4119364184 conn(0x7fb51809b8c0 msgr2=0x7fb5180d25d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-09T15:12:27.023 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:27.018+0000 7fb51da04640 1 -- 172.21.15.43:0/4119364184 shutdown_connections 2023-08-09T15:12:27.023 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:27.018+0000 7fb51da04640 1 -- 172.21.15.43:0/4119364184 wait complete. 2023-08-09T15:12:27.023 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:27.020+0000 7fb51da04640 1 Processor -- start 2023-08-09T15:12:27.023 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:27.020+0000 7fb51da04640 1 -- start start 2023-08-09T15:12:27.025 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:27.021+0000 7fb51da04640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fb5000047c0 0x7fb5181f17a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T15:12:27.025 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:27.021+0000 7fb51da04640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fb5180b6c40 0x7fb5181f5cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T15:12:27.025 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:27.021+0000 7fb51da04640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb5180b3c30 0x7fb5180b40b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T15:12:27.025 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:27.021+0000 7fb51da04640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb5180b45f0 con 0x7fb5180b3c30 2023-08-09T15:12:27.025 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:27.021+0000 7fb51da04640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb518150520 con 0x7fb5000047c0 2023-08-09T15:12:27.026 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:27.021+0000 7fb51da04640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb5181f6230 con 0x7fb5180b6c40 2023-08-09T15:12:27.026 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:27.022+0000 7fb5177fe640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb5180b3c30 0x7fb5180b40b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T15:12:27.026 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:27.022+0000 7fb5177fe640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb5180b3c30 0x7fb5180b40b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:53160/0 (socket says 172.21.15.43:53160) 2023-08-09T15:12:27.026 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:27.022+0000 7fb5177fe640 1 -- 172.21.15.43:0/431135023 learned_addr learned my addr 172.21.15.43:0/431135023 (peer_addr_for_me v2:172.21.15.43:0/0) 2023-08-09T15:12:27.026 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:27.022+0000 7fb516ffd640 1 --2- 172.21.15.43:0/431135023 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fb5000047c0 0x7fb5181f17a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T15:12:27.026 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:27.022+0000 7fb5167fc640 1 --2- 172.21.15.43:0/431135023 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fb5180b6c40 0x7fb5181f5cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T15:12:27.026 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:27.022+0000 7fb5177fe640 1 -- 172.21.15.43:0/431135023 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fb5180b6c40 msgr2=0x7fb5181f5cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T15:12:27.026 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:27.022+0000 7fb5177fe640 1 --2- 172.21.15.43:0/431135023 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fb5180b6c40 0x7fb5181f5cf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T15:12:27.027 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:27.022+0000 7fb5177fe640 1 -- 172.21.15.43:0/431135023 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fb5000047c0 msgr2=0x7fb5181f17a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T15:12:27.027 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:27.022+0000 7fb5177fe640 1 --2- 172.21.15.43:0/431135023 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fb5000047c0 0x7fb5181f17a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T15:12:27.027 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:27.022+0000 7fb5177fe640 1 -- 172.21.15.43:0/431135023 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb5180b7200 con 0x7fb5180b3c30 2023-08-09T15:12:27.027 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:27.023+0000 7fb5177fe640 1 --2- 172.21.15.43:0/431135023 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb5180b3c30 0x7fb5180b40b0 secure :-1 s=READY pgs=2602 cs=0 l=1 rev1=1 crypto rx=0x7fb5180b2000 tx=0x7fb51000d810 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-09T15:12:27.027 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:27.023+0000 7fb4fffff640 1 -- 172.21.15.43:0/431135023 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb51000b070 con 0x7fb5180b3c30 2023-08-09T15:12:27.027 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:27.023+0000 7fb4fffff640 1 -- 172.21.15.43:0/431135023 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb5100054e0 con 0x7fb5180b3c30 2023-08-09T15:12:27.027 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:27.023+0000 7fb4fffff640 1 -- 172.21.15.43:0/431135023 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb510003810 con 0x7fb5180b3c30 2023-08-09T15:12:27.027 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:27.023+0000 7fb51da04640 1 -- 172.21.15.43:0/431135023 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb5180b74e0 con 0x7fb5180b3c30 2023-08-09T15:12:27.028 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:27.023+0000 7fb51da04640 1 -- 172.21.15.43:0/431135023 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb5180b7aa0 con 0x7fb5180b3c30 2023-08-09T15:12:27.028 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:27.024+0000 7fb51da04640 1 -- 172.21.15.43:0/431135023 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7fb51806aa50 con 0x7fb5180b3c30 2023-08-09T15:12:27.029 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:27.025+0000 7fb4fffff640 1 -- 172.21.15.43:0/431135023 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 129) v1 ==== 88443+0+0 (secure 0 0 0) 0x7fb510003970 con 0x7fb5180b3c30 2023-08-09T15:12:27.029 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:27.026+0000 7fb4fffff640 1 --2- 172.21.15.43:0/431135023 >> [v2:172.21.15.203:6832/1456699667,v1:172.21.15.203:6833/1456699667] conn(0x7fb50406a170 0x7fb50406c630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T15:12:27.029 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:27.026+0000 7fb4fffff640 1 -- 172.21.15.43:0/431135023 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(229..229 src has 1..229) v4 ==== 6462+0+0 (secure 0 0 0) 0x7fb510087f80 con 0x7fb5180b3c30 2023-08-09T15:12:27.029 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:27.026+0000 7fb4fffff640 1 --2- 172.21.15.43:0/431135023 >> [v2:172.21.15.203:6808/3331450438,v1:172.21.15.203:6809/3331450438] conn(0x7fb504071150 0x7fb5040735b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T15:12:27.029 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:27.026+0000 7fb4fffff640 1 -- 172.21.15.43:0/431135023 --> [v2:172.21.15.203:6808/3331450438,v1:172.21.15.203:6809/3331450438] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7fb504073c80 con 0x7fb504071150 2023-08-09T15:12:27.029 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:27.026+0000 7fb4fffff640 1 -- 172.21.15.43:0/431135023 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_get_version_reply(handle=1 version=229) v2 ==== 24+0+0 (secure 0 0 0) 0x7fb51008a560 con 0x7fb5180b3c30 2023-08-09T15:12:27.029 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:27.027+0000 7fb516ffd640 1 --2- 172.21.15.43:0/431135023 >> [v2:172.21.15.203:6832/1456699667,v1:172.21.15.203:6833/1456699667] conn(0x7fb50406a170 0x7fb50406c630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T15:12:27.030 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:27.027+0000 7fb5167fc640 1 --2- 172.21.15.43:0/431135023 >> [v2:172.21.15.203:6808/3331450438,v1:172.21.15.203:6809/3331450438] conn(0x7fb504071150 0x7fb5040735b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T15:12:27.030 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:27.028+0000 7fb516ffd640 1 --2- 172.21.15.43:0/431135023 >> [v2:172.21.15.203:6832/1456699667,v1:172.21.15.203:6833/1456699667] conn(0x7fb50406a170 0x7fb50406c630 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7fb508009fd0 tx=0x7fb50800b040 comp rx=0 tx=0).ready entity=mgr.9232 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-09T15:12:27.030 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:27.028+0000 7fb5167fc640 1 --2- 172.21.15.43:0/431135023 >> [v2:172.21.15.203:6808/3331450438,v1:172.21.15.203:6809/3331450438] conn(0x7fb504071150 0x7fb5040735b0 crc :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-09T15:12:27.032 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:27.030+0000 7fb4fffff640 1 -- 172.21.15.43:0/431135023 <== osd.7 v2:172.21.15.203:6808/3331450438 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+25607 (crc 0 0 0) 0x7fb504073c80 con 0x7fb504071150 2023-08-09T15:12:27.046 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:27.045+0000 7fb51da04640 1 -- 172.21.15.43:0/431135023 --> [v2:172.21.15.203:6808/3331450438,v1:172.21.15.203:6809/3331450438] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) v1 -- 0x7fb51804db40 con 0x7fb504071150 2023-08-09T15:12:27.047 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:27.046+0000 7fb4fffff640 1 -- 172.21.15.43:0/431135023 <== osd.7 v2:172.21.15.203:6808/3331450438 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+105 (crc 0 0 0) 0x7fb51804db40 con 0x7fb504071150 2023-08-09T15:12:27.048 INFO:teuthology.orchestra.run.smithi043.stdout:{ 2023-08-09T15:12:27.048 INFO:teuthology.orchestra.run.smithi043.stdout: "success": "osd_debug_deep_scrub_sleep = '0.000000' (not observed, change may require restart) " 2023-08-09T15:12:27.048 INFO:teuthology.orchestra.run.smithi043.stdout:} 2023-08-09T15:12:27.049 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:27.046+0000 7fb51da04640 1 -- 172.21.15.43:0/431135023 >> [v2:172.21.15.203:6808/3331450438,v1:172.21.15.203:6809/3331450438] conn(0x7fb504071150 msgr2=0x7fb5040735b0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T15:12:27.049 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:27.046+0000 7fb51da04640 1 --2- 172.21.15.43:0/431135023 >> [v2:172.21.15.203:6808/3331450438,v1:172.21.15.203:6809/3331450438] conn(0x7fb504071150 0x7fb5040735b0 crc :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T15:12:27.049 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:27.046+0000 7fb51da04640 1 -- 172.21.15.43:0/431135023 >> [v2:172.21.15.203:6832/1456699667,v1:172.21.15.203:6833/1456699667] conn(0x7fb50406a170 msgr2=0x7fb50406c630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T15:12:27.049 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:27.046+0000 7fb51da04640 1 --2- 172.21.15.43:0/431135023 >> [v2:172.21.15.203:6832/1456699667,v1:172.21.15.203:6833/1456699667] conn(0x7fb50406a170 0x7fb50406c630 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7fb508009fd0 tx=0x7fb50800b040 comp rx=0 tx=0).stop 2023-08-09T15:12:27.049 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:27.046+0000 7fb51da04640 1 -- 172.21.15.43:0/431135023 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb5180b3c30 msgr2=0x7fb5180b40b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T15:12:27.050 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:27.046+0000 7fb51da04640 1 --2- 172.21.15.43:0/431135023 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb5180b3c30 0x7fb5180b40b0 secure :-1 s=READY pgs=2602 cs=0 l=1 rev1=1 crypto rx=0x7fb5180b2000 tx=0x7fb51000d810 comp rx=0 tx=0).stop 2023-08-09T15:12:27.050 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:27.046+0000 7fb5177fe640 1 -- 172.21.15.43:0/431135023 reap_dead start 2023-08-09T15:12:27.050 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:27.047+0000 7fb51da04640 1 -- 172.21.15.43:0/431135023 shutdown_connections 2023-08-09T15:12:27.050 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:27.047+0000 7fb51da04640 1 -- 172.21.15.43:0/431135023 >> 172.21.15.43:0/431135023 conn(0x7fb51809b8c0 msgr2=0x7fb51815f680 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-09T15:12:27.051 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:27.047+0000 7fb51da04640 1 -- 172.21.15.43:0/431135023 shutdown_connections 2023-08-09T15:12:27.051 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:27.047+0000 7fb51da04640 1 -- 172.21.15.43:0/431135023 wait complete. 2023-08-09T15:12:27.065 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 7 2023-08-09T15:12:27.295 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:27.294+0000 7fb51da91640 1 Processor -- start 2023-08-09T15:12:27.296 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:27.294+0000 7fb51da91640 1 -- start start 2023-08-09T15:12:27.296 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:27.295+0000 7fb51da91640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fb518070ec0 0x7fb5180cd590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T15:12:27.296 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:27.295+0000 7fb51da91640 1 -- --> v1:172.21.15.43:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb51815df30 con 0x7fb518159600 2023-08-09T15:12:27.297 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:27.295+0000 7fb51da91640 1 -- --> v1:172.21.15.203:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb5180ad2a0 con 0x7fb518159220 2023-08-09T15:12:27.297 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:27.295+0000 7fb51da91640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb51815d9b0 con 0x7fb518070ec0 2023-08-09T15:12:27.298 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:27.295+0000 7fb51d290640 1 --1- >> v1:172.21.15.43:6789/0 conn(0x7fb518159600 0x7fb51806c270 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.43:6789/0 says I am v1:172.21.15.43:45440/0 (socket says 172.21.15.43:45440) 2023-08-09T15:12:27.298 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:27.295+0000 7fb51d290640 1 -- 172.21.15.43:0/2734836162 learned_addr learned my addr 172.21.15.43:0/2734836162 (peer_addr_for_me v1:172.21.15.43:0/0) 2023-08-09T15:12:27.298 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:27.296+0000 7fb5177fe640 1 -- 172.21.15.43:0/2734836162 <== mon.0 v1:172.21.15.43:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2259744867 0 0) 0x7fb51815df30 con 0x7fb518159600 2023-08-09T15:12:27.298 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:27.296+0000 7fb517fff640 1 --2- 172.21.15.43:0/2734836162 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fb518070ec0 0x7fb5180cd590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T15:12:27.299 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:27.296+0000 7fb5177fe640 1 -- 172.21.15.43:0/2734836162 --> v1:172.21.15.43:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fb500003520 con 0x7fb518159600 2023-08-09T15:12:27.299 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:27.296+0000 7fb5177fe640 1 -- 172.21.15.43:0/2734836162 <== mon.0 v1:172.21.15.43:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3017917082 0 0) 0x7fb500003520 con 0x7fb518159600 2023-08-09T15:12:27.299 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:27.296+0000 7fb5177fe640 1 -- 172.21.15.43:0/2734836162 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fb518070ec0 msgr2=0x7fb5180cd590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-09T15:12:27.299 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:27.296+0000 7fb5177fe640 1 --2- 172.21.15.43:0/2734836162 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fb518070ec0 0x7fb5180cd590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T15:12:27.299 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:27.296+0000 7fb5177fe640 1 -- 172.21.15.43:0/2734836162 >> v1:172.21.15.203:6789/0 conn(0x7fb518159220 legacy=0x7fb5180707b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T15:12:27.299 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:27.296+0000 7fb5177fe640 1 -- 172.21.15.43:0/2734836162 --> v1:172.21.15.43:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb51815dc50 con 0x7fb518159600 2023-08-09T15:12:27.299 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:27.296+0000 7fb5177fe640 1 -- 172.21.15.43:0/2734836162 <== mon.0 v1:172.21.15.43:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1285150740 0 0) 0x7fb510001670 con 0x7fb518159600 2023-08-09T15:12:27.299 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:27.296+0000 7fb5177fe640 1 -- 172.21.15.43:0/2734836162 >> v1:172.21.15.43:6789/0 conn(0x7fb518159600 legacy=0x7fb51806c270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T15:12:27.300 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:27.296+0000 7fb5177fe640 1 --2- 172.21.15.43:0/2734836162 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fb5000039a0 0x7fb500003dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T15:12:27.300 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:27.296+0000 7fb5177fe640 1 --2- 172.21.15.43:0/2734836162 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fb518070ec0 0x7fb5000047c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T15:12:27.300 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:27.297+0000 7fb5177fe640 1 --2- 172.21.15.43:0/2734836162 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb500004d50 0x7fb5000051d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T15:12:27.300 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:27.297+0000 7fb5177fe640 1 -- 172.21.15.43:0/2734836162 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb51815d9b0 con 0x7fb500004d50 2023-08-09T15:12:27.300 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:27.297+0000 7fb5177fe640 1 -- 172.21.15.43:0/2734836162 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb50000de10 con 0x7fb5000039a0 2023-08-09T15:12:27.300 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:27.297+0000 7fb5177fe640 1 -- 172.21.15.43:0/2734836162 --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb50000dfd0 con 0x7fb518070ec0 2023-08-09T15:12:27.300 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:27.297+0000 7fb51d290640 1 --2- 172.21.15.43:0/2734836162 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb500004d50 0x7fb5000051d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T15:12:27.302 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:27.297+0000 7fb51ca8f640 1 --2- 172.21.15.43:0/2734836162 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fb5000039a0 0x7fb500003dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T15:12:27.302 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:27.297+0000 7fb517fff640 1 --2- 172.21.15.43:0/2734836162 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fb518070ec0 0x7fb5000047c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T15:12:27.302 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:27.297+0000 7fb51d290640 1 -- 172.21.15.43:0/2734836162 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fb518070ec0 msgr2=0x7fb5000047c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-09T15:12:27.302 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:27.297+0000 7fb51d290640 1 --2- 172.21.15.43:0/2734836162 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fb518070ec0 0x7fb5000047c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T15:12:27.302 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:27.297+0000 7fb51d290640 1 -- 172.21.15.43:0/2734836162 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fb5000039a0 msgr2=0x7fb500003dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-09T15:12:27.302 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:27.297+0000 7fb51d290640 1 --2- 172.21.15.43:0/2734836162 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fb5000039a0 0x7fb500003dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T15:12:27.303 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:27.297+0000 7fb51d290640 1 -- 172.21.15.43:0/2734836162 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb500003520 con 0x7fb500004d50 2023-08-09T15:12:27.303 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:27.297+0000 7fb51ca8f640 1 --2- 172.21.15.43:0/2734836162 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fb5000039a0 0x7fb500003dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-08-09T15:12:27.303 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:27.298+0000 7fb51d290640 1 --2- 172.21.15.43:0/2734836162 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb500004d50 0x7fb5000051d0 secure :-1 s=READY pgs=2604 cs=0 l=1 rev1=1 crypto rx=0x7fb51000c210 tx=0x7fb5000057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3e4361657a51db01 server_cookie=0 in_seq=0 out_seq=0 2023-08-09T15:12:27.303 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:27.299+0000 7fb5177fe640 1 -- 172.21.15.43:0/2734836162 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb510005ad0 con 0x7fb500004d50 2023-08-09T15:12:27.303 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:27.299+0000 7fb5177fe640 1 -- 172.21.15.43:0/2734836162 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb510005c30 con 0x7fb500004d50 2023-08-09T15:12:27.303 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:27.299+0000 7fb5177fe640 1 -- 172.21.15.43:0/2734836162 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb51002c870 con 0x7fb500004d50 2023-08-09T15:12:27.303 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:27.299+0000 7fb51da91640 1 -- 172.21.15.43:0/2734836162 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb500004d50 msgr2=0x7fb5000051d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T15:12:27.303 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:27.299+0000 7fb51da91640 1 --2- 172.21.15.43:0/2734836162 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb500004d50 0x7fb5000051d0 secure :-1 s=READY pgs=2604 cs=0 l=1 rev1=1 crypto rx=0x7fb51000c210 tx=0x7fb5000057f0 comp rx=0 tx=0).stop 2023-08-09T15:12:27.304 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:27.299+0000 7fb51d290640 1 -- 172.21.15.43:0/2734836162 reap_dead start 2023-08-09T15:12:27.304 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:27.299+0000 7fb51da91640 1 -- 172.21.15.43:0/2734836162 shutdown_connections 2023-08-09T15:12:27.304 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:27.299+0000 7fb51da91640 1 -- 172.21.15.43:0/2734836162 >> 172.21.15.43:0/2734836162 conn(0x7fb5180ce020 msgr2=0x7fb5180ce430 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-09T15:12:27.304 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:27.300+0000 7fb51da91640 1 -- 172.21.15.43:0/2734836162 shutdown_connections 2023-08-09T15:12:27.304 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:27.300+0000 7fb51da91640 1 -- 172.21.15.43:0/2734836162 wait complete. 2023-08-09T15:12:27.306 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:27.301+0000 7fb51da91640 1 Processor -- start 2023-08-09T15:12:27.306 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:27.302+0000 7fb51da91640 1 -- start start 2023-08-09T15:12:27.306 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:27.303+0000 7fb51da91640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fb500004d50 0x7fb5180c5b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T15:12:27.306 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:27.303+0000 7fb51da91640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb5180ab500 0x7fb5180ab980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T15:12:27.306 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:27.303+0000 7fb51da91640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fb5180c7720 0x7fb5180c7ba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T15:12:27.307 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:27.303+0000 7fb51da91640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb51815a5a0 con 0x7fb5180ab500 2023-08-09T15:12:27.307 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:27.303+0000 7fb51da91640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb5180abec0 con 0x7fb500004d50 2023-08-09T15:12:27.307 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:27.303+0000 7fb51da91640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb5180c8450 con 0x7fb5180c7720 2023-08-09T15:12:27.307 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:27.303+0000 7fb517fff640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb5180ab500 0x7fb5180ab980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T15:12:27.307 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:27.303+0000 7fb517fff640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb5180ab500 0x7fb5180ab980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:53184/0 (socket says 172.21.15.43:53184) 2023-08-09T15:12:27.307 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:27.303+0000 7fb517fff640 1 -- 172.21.15.43:0/2301720490 learned_addr learned my addr 172.21.15.43:0/2301720490 (peer_addr_for_me v2:172.21.15.43:0/0) 2023-08-09T15:12:27.307 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:27.303+0000 7fb51ca8f640 1 --2- 172.21.15.43:0/2301720490 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fb500004d50 0x7fb5180c5b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T15:12:27.308 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:27.303+0000 7fb517fff640 1 -- 172.21.15.43:0/2301720490 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fb5180c7720 msgr2=0x7fb5180c7ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T15:12:27.308 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:27.303+0000 7fb517fff640 1 --2- 172.21.15.43:0/2301720490 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fb5180c7720 0x7fb5180c7ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T15:12:27.308 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:27.303+0000 7fb517fff640 1 -- 172.21.15.43:0/2301720490 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fb500004d50 msgr2=0x7fb5180c5b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T15:12:27.308 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:27.303+0000 7fb517fff640 1 --2- 172.21.15.43:0/2301720490 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fb500004d50 0x7fb5180c5b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T15:12:27.308 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:27.303+0000 7fb517fff640 1 -- 172.21.15.43:0/2301720490 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb518159c00 con 0x7fb5180ab500 2023-08-09T15:12:27.308 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:27.303+0000 7fb51ca8f640 1 --2- 172.21.15.43:0/2301720490 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fb500004d50 0x7fb5180c5b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-08-09T15:12:27.308 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:27.304+0000 7fb517fff640 1 --2- 172.21.15.43:0/2301720490 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb5180ab500 0x7fb5180ab980 secure :-1 s=READY pgs=2605 cs=0 l=1 rev1=1 crypto rx=0x7fb50c00e7e0 tx=0x7fb50c00eca0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-09T15:12:27.308 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:27.304+0000 7fb5157fa640 1 -- 172.21.15.43:0/2301720490 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb50c00c650 con 0x7fb5180ab500 2023-08-09T15:12:27.309 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:27.304+0000 7fb5157fa640 1 -- 172.21.15.43:0/2301720490 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb50c00c7b0 con 0x7fb5180ab500 2023-08-09T15:12:27.309 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:27.304+0000 7fb51da91640 1 -- 172.21.15.43:0/2301720490 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb518159ee0 con 0x7fb5180ab500 2023-08-09T15:12:27.309 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:27.304+0000 7fb5157fa640 1 -- 172.21.15.43:0/2301720490 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb50c00cb20 con 0x7fb5180ab500 2023-08-09T15:12:27.309 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:27.304+0000 7fb51da91640 1 -- 172.21.15.43:0/2301720490 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb5180c80e0 con 0x7fb5180ab500 2023-08-09T15:12:27.310 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:27.306+0000 7fb51da91640 1 -- 172.21.15.43:0/2301720490 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb5180ce430 con 0x7fb5180ab500 2023-08-09T15:12:27.310 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:27.307+0000 7fb5157fa640 1 -- 172.21.15.43:0/2301720490 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 129) v1 ==== 88443+0+0 (secure 0 0 0) 0x7fb50c00cc80 con 0x7fb5180ab500 2023-08-09T15:12:27.310 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:27.307+0000 7fb5157fa640 1 --2- 172.21.15.43:0/2301720490 >> [v2:172.21.15.203:6832/1456699667,v1:172.21.15.203:6833/1456699667] conn(0x7fb50406a1a0 0x7fb50406c660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T15:12:27.310 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:27.308+0000 7fb5157fa640 1 -- 172.21.15.43:0/2301720490 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(229..229 src has 1..229) v4 ==== 6462+0+0 (secure 0 0 0) 0x7fb50c088660 con 0x7fb5180ab500 2023-08-09T15:12:27.310 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:27.308+0000 7fb51ca8f640 1 --2- 172.21.15.43:0/2301720490 >> [v2:172.21.15.203:6832/1456699667,v1:172.21.15.203:6833/1456699667] conn(0x7fb50406a1a0 0x7fb50406c660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T15:12:27.311 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:27.309+0000 7fb51ca8f640 1 --2- 172.21.15.43:0/2301720490 >> [v2:172.21.15.203:6832/1456699667,v1:172.21.15.203:6833/1456699667] conn(0x7fb50406a1a0 0x7fb50406c660 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7fb508007fd0 tx=0x7fb508009040 comp rx=0 tx=0).ready entity=mgr.9232 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-09T15:12:27.313 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:27.312+0000 7fb5157fa640 1 -- 172.21.15.43:0/2301720490 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178066 (secure 0 0 0) 0x7fb50c053a30 con 0x7fb5180ab500 2023-08-09T15:12:27.509 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:27.507+0000 7fb51da91640 1 -- 172.21.15.43:0/2301720490 --> [v2:172.21.15.203:6832/1456699667,v1:172.21.15.203:6833/1456699667] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "7", "target": ["mon-mgr", ""]}) v1 -- 0x7fb5180cc000 con 0x7fb50406a1a0 2023-08-09T15:12:27.510 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:27.508+0000 7fb5157fa640 1 -- 172.21.15.43:0/2301720490 <== mgr.9232 v2:172.21.15.203:6832/1456699667 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 7 to deep-scrub) v1 ==== 41+0+0 (secure 0 0 0) 0x7fb50800d040 con 0x7fb50406a1a0 2023-08-09T15:12:27.549 INFO:teuthology.orchestra.run.smithi043.stderr:instructed osd(s) 7 to deep-scrub 2023-08-09T15:12:27.550 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:27.511+0000 7fb51da91640 1 -- 172.21.15.43:0/2301720490 >> [v2:172.21.15.203:6832/1456699667,v1:172.21.15.203:6833/1456699667] conn(0x7fb50406a1a0 msgr2=0x7fb50406c660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T15:12:27.550 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:27.511+0000 7fb51da91640 1 --2- 172.21.15.43:0/2301720490 >> [v2:172.21.15.203:6832/1456699667,v1:172.21.15.203:6833/1456699667] conn(0x7fb50406a1a0 0x7fb50406c660 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7fb508007fd0 tx=0x7fb508009040 comp rx=0 tx=0).stop 2023-08-09T15:12:27.550 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:27.511+0000 7fb51da91640 1 -- 172.21.15.43:0/2301720490 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb5180ab500 msgr2=0x7fb5180ab980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T15:12:27.551 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:27.511+0000 7fb51da91640 1 --2- 172.21.15.43:0/2301720490 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb5180ab500 0x7fb5180ab980 secure :-1 s=READY pgs=2605 cs=0 l=1 rev1=1 crypto rx=0x7fb50c00e7e0 tx=0x7fb50c00eca0 comp rx=0 tx=0).stop 2023-08-09T15:12:27.551 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:27.511+0000 7fb51da91640 1 -- 172.21.15.43:0/2301720490 shutdown_connections 2023-08-09T15:12:27.551 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:27.511+0000 7fb51da91640 1 --2- 172.21.15.43:0/2301720490 >> [v2:172.21.15.203:6832/1456699667,v1:172.21.15.203:6833/1456699667] conn(0x7fb50406a1a0 0x7fb50406c660 unknown :-1 s=CLOSED pgs=58 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T15:12:27.551 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:27.511+0000 7fb51da91640 1 --2- 172.21.15.43:0/2301720490 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fb5180c7720 0x7fb5180c7ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T15:12:27.551 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:27.511+0000 7fb51da91640 1 --2- 172.21.15.43:0/2301720490 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb5180ab500 0x7fb5180ab980 unknown :-1 s=CLOSED pgs=2605 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T15:12:27.551 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:27.511+0000 7fb51da91640 1 --2- 172.21.15.43:0/2301720490 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fb500004d50 0x7fb5180c5b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T15:12:27.551 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:27.511+0000 7fb51da91640 1 -- 172.21.15.43:0/2301720490 >> 172.21.15.43:0/2301720490 conn(0x7fb5180ce020 msgr2=0x7fb5180aad10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-09T15:12:27.552 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:27.511+0000 7fb51da91640 1 -- 172.21.15.43:0/2301720490 shutdown_connections 2023-08-09T15:12:27.552 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:27.511+0000 7fb51da91640 1 -- 172.21.15.43:0/2301720490 wait complete. 2023-08-09T15:12:27.552 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-08-09T15:12:27.781 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:27.779+0000 7f135012f640 1 Processor -- start 2023-08-09T15:12:27.781 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:27.779+0000 7f135012f640 1 -- start start 2023-08-09T15:12:27.782 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:27.781+0000 7f135012f640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f134806b4b0 0x7f13480cdbe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T15:12:27.783 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:27.781+0000 7f135012f640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f134815b310 0x7f134815b740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T15:12:27.783 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:27.781+0000 7f135012f640 1 -- --> v1:172.21.15.203:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f134806c2c0 con 0x7f134806fe60 2023-08-09T15:12:27.783 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:27.781+0000 7f135012f640 1 -- --> v2:172.21.15.43:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f134815bc80 con 0x7f134815b310 2023-08-09T15:12:27.783 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:27.781+0000 7f135012f640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f13480cb690 con 0x7f134806b4b0 2023-08-09T15:12:27.784 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:27.781+0000 7f134e6cc640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f134815b310 0x7f134815b740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T15:12:27.784 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:27.782+0000 7f134e6cc640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f134815b310 0x7f134815b740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:53190/0 (socket says 172.21.15.43:53190) 2023-08-09T15:12:27.784 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:27.782+0000 7f134e6cc640 1 -- 172.21.15.43:0/3320248033 learned_addr learned my addr 172.21.15.43:0/3320248033 (peer_addr_for_me v2:172.21.15.43:0/0) 2023-08-09T15:12:27.784 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:27.782+0000 7f134d6ca640 1 --2- 172.21.15.43:0/3320248033 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f134806b4b0 0x7f13480cdbe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T15:12:27.784 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:27.782+0000 7f134e6cc640 1 -- 172.21.15.43:0/3320248033 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f134806b4b0 msgr2=0x7f13480cdbe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-09T15:12:27.785 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:27.782+0000 7f134e6cc640 1 --2- 172.21.15.43:0/3320248033 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f134806b4b0 0x7f13480cdbe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T15:12:27.785 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:27.782+0000 7f134e6cc640 1 -- 172.21.15.43:0/3320248033 >> v1:172.21.15.203:6789/0 conn(0x7f134806fe60 legacy=0x7f134815be10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-09T15:12:27.785 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:27.782+0000 7f134e6cc640 1 -- 172.21.15.43:0/3320248033 --> v2:172.21.15.43:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f134806ba20 con 0x7f134815b310 2023-08-09T15:12:27.786 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:27.783+0000 7f134e6cc640 1 --2- 172.21.15.43:0/3320248033 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f134815b310 0x7f134815b740 secure :-1 s=READY pgs=2606 cs=0 l=1 rev1=1 crypto rx=0x7f1340009e50 tx=0x7f134002bed0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=de6e93656167007d server_cookie=0 in_seq=0 out_seq=0 2023-08-09T15:12:27.786 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:27.784+0000 7f134cec9640 1 -- 172.21.15.43:0/3320248033 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f134002ca70 con 0x7f134815b310 2023-08-09T15:12:27.786 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:27.784+0000 7f134cec9640 1 -- 172.21.15.43:0/3320248033 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f134002cbd0 con 0x7f134815b310 2023-08-09T15:12:27.786 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:27.784+0000 7f134cec9640 1 -- 172.21.15.43:0/3320248033 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f13400343f0 con 0x7f134815b310 2023-08-09T15:12:27.786 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:27.784+0000 7f135012f640 1 -- 172.21.15.43:0/3320248033 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f134815b310 msgr2=0x7f134815b740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T15:12:27.787 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:27.784+0000 7f135012f640 1 --2- 172.21.15.43:0/3320248033 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f134815b310 0x7f134815b740 secure :-1 s=READY pgs=2606 cs=0 l=1 rev1=1 crypto rx=0x7f1340009e50 tx=0x7f134002bed0 comp rx=0 tx=0).stop 2023-08-09T15:12:27.787 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:27.784+0000 7f135012f640 1 -- 172.21.15.43:0/3320248033 shutdown_connections 2023-08-09T15:12:27.787 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:27.785+0000 7f135012f640 1 --2- 172.21.15.43:0/3320248033 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f134815b310 0x7f134815b740 unknown :-1 s=CLOSED pgs=2606 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T15:12:27.787 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:27.785+0000 7f135012f640 1 --2- 172.21.15.43:0/3320248033 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f134806b4b0 0x7f13480cdbe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T15:12:27.787 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:27.785+0000 7f135012f640 1 -- 172.21.15.43:0/3320248033 >> 172.21.15.43:0/3320248033 conn(0x7f1348069d70 msgr2=0x7f13480cac70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-09T15:12:27.787 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:27.785+0000 7f135012f640 1 -- 172.21.15.43:0/3320248033 shutdown_connections 2023-08-09T15:12:27.787 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:27.785+0000 7f135012f640 1 -- 172.21.15.43:0/3320248033 wait complete. 2023-08-09T15:12:27.788 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:27.787+0000 7f135012f640 1 Processor -- start 2023-08-09T15:12:27.788 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:27.787+0000 7f135012f640 1 -- start start 2023-08-09T15:12:27.789 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:27.788+0000 7f135012f640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f134806b4b0 0x7f1348145790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T15:12:27.789 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:27.788+0000 7f135012f640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f134806fe60 0x7f1348146050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T15:12:27.789 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:27.788+0000 7f135012f640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f134815b310 0x7f13480c52c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T15:12:27.790 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:27.788+0000 7f135012f640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f13480c4db0 con 0x7f134806fe60 2023-08-09T15:12:27.790 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:27.788+0000 7f135012f640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f13480c6b80 con 0x7f134806b4b0 2023-08-09T15:12:27.790 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:27.788+0000 7f135012f640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f13480c7140 con 0x7f134815b310 2023-08-09T15:12:27.790 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:27.788+0000 7f134d6ca640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f134806fe60 0x7f1348146050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T15:12:27.790 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:27.788+0000 7f134d6ca640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f134806fe60 0x7f1348146050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:53192/0 (socket says 172.21.15.43:53192) 2023-08-09T15:12:27.790 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:27.788+0000 7f134d6ca640 1 -- 172.21.15.43:0/400155873 learned_addr learned my addr 172.21.15.43:0/400155873 (peer_addr_for_me v2:172.21.15.43:0/0) 2023-08-09T15:12:27.791 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:27.788+0000 7f134decb640 1 --2- 172.21.15.43:0/400155873 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f134806b4b0 0x7f1348145790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T15:12:27.791 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:27.789+0000 7f134d6ca640 1 -- 172.21.15.43:0/400155873 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f134815b310 msgr2=0x7f13480c52c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T15:12:27.792 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:27.789+0000 7f134d6ca640 1 --2- 172.21.15.43:0/400155873 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f134815b310 0x7f13480c52c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T15:12:27.792 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:27.789+0000 7f134d6ca640 1 -- 172.21.15.43:0/400155873 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f134806b4b0 msgr2=0x7f1348145790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T15:12:27.792 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:27.789+0000 7f134d6ca640 1 --2- 172.21.15.43:0/400155873 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f134806b4b0 0x7f1348145790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T15:12:27.792 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:27.789+0000 7f134d6ca640 1 -- 172.21.15.43:0/400155873 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1340009b30 con 0x7f134806fe60 2023-08-09T15:12:27.792 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:27.789+0000 7f134decb640 1 --2- 172.21.15.43:0/400155873 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f134806b4b0 0x7f1348145790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-08-09T15:12:27.792 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:27.789+0000 7f134d6ca640 1 --2- 172.21.15.43:0/400155873 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f134806fe60 0x7f1348146050 secure :-1 s=READY pgs=2607 cs=0 l=1 rev1=1 crypto rx=0x7f133800d620 tx=0x7f133800dae0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-09T15:12:27.793 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:27.789+0000 7f133e7fc640 1 -- 172.21.15.43:0/400155873 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f133800b620 con 0x7f134806fe60 2023-08-09T15:12:27.793 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:27.789+0000 7f133e7fc640 1 -- 172.21.15.43:0/400155873 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f133800b780 con 0x7f134806fe60 2023-08-09T15:12:27.793 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:27.789+0000 7f133e7fc640 1 -- 172.21.15.43:0/400155873 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f133800bb30 con 0x7f134806fe60 2023-08-09T15:12:27.793 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:27.789+0000 7f135012f640 1 -- 172.21.15.43:0/400155873 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f13480c5e30 con 0x7f134806fe60 2023-08-09T15:12:27.793 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:27.789+0000 7f135012f640 1 -- 172.21.15.43:0/400155873 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f13480c6dd0 con 0x7f134806fe60 2023-08-09T15:12:27.794 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:27.790+0000 7f135012f640 1 -- 172.21.15.43:0/400155873 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f134815add0 con 0x7f134806fe60 2023-08-09T15:12:27.794 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:27.791+0000 7f133e7fc640 1 -- 172.21.15.43:0/400155873 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 129) v1 ==== 88443+0+0 (secure 0 0 0) 0x7f133800bc90 con 0x7f134806fe60 2023-08-09T15:12:27.794 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:27.793+0000 7f133e7fc640 1 --2- 172.21.15.43:0/400155873 >> [v2:172.21.15.203:6832/1456699667,v1:172.21.15.203:6833/1456699667] conn(0x7f132c06a1a0 0x7f132c06c660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T15:12:27.795 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:27.793+0000 7f133e7fc640 1 -- 172.21.15.43:0/400155873 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(229..229 src has 1..229) v4 ==== 6462+0+0 (secure 0 0 0) 0x7f1338088df0 con 0x7f134806fe60 2023-08-09T15:12:27.795 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:27.794+0000 7f134decb640 1 --2- 172.21.15.43:0/400155873 >> [v2:172.21.15.203:6832/1456699667,v1:172.21.15.203:6833/1456699667] conn(0x7f132c06a1a0 0x7f132c06c660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T15:12:27.796 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:27.795+0000 7f134decb640 1 --2- 172.21.15.43:0/400155873 >> [v2:172.21.15.203:6832/1456699667,v1:172.21.15.203:6833/1456699667] conn(0x7f132c06a1a0 0x7f132c06c660 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f13480c81c0 tx=0x7f1344009040 comp rx=0 tx=0).ready entity=mgr.9232 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-09T15:12:27.799 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:27.797+0000 7f133e7fc640 1 -- 172.21.15.43:0/400155873 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178066 (secure 0 0 0) 0x7f134815add0 con 0x7f134806fe60 2023-08-09T15:12:27.990 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:27.988+0000 7f135012f640 1 -- 172.21.15.43:0/400155873 --> [v2:172.21.15.203:6832/1456699667,v1:172.21.15.203:6833/1456699667] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f134806bfe0 con 0x7f132c06a1a0 2023-08-09T15:12:27.997 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:27.996+0000 7f133e7fc640 1 -- 172.21.15.43:0/400155873 <== mgr.9232 v2:172.21.15.203:6832/1456699667 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+70502 (secure 0 0 0) 0x7f1344005050 con 0x7f132c06a1a0 2023-08-09T15:12:27.998 INFO:teuthology.orchestra.run.smithi043.stdout: 2023-08-09T15:12:27.999 INFO:teuthology.orchestra.run.smithi043.stderr:dumped all 2023-08-09T15:12:28.001 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:27.999+0000 7f135012f640 1 -- 172.21.15.43:0/400155873 >> [v2:172.21.15.203:6832/1456699667,v1:172.21.15.203:6833/1456699667] conn(0x7f132c06a1a0 msgr2=0x7f132c06c660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T15:12:28.001 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:27.999+0000 7f135012f640 1 --2- 172.21.15.43:0/400155873 >> [v2:172.21.15.203:6832/1456699667,v1:172.21.15.203:6833/1456699667] conn(0x7f132c06a1a0 0x7f132c06c660 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f13480c81c0 tx=0x7f1344009040 comp rx=0 tx=0).stop 2023-08-09T15:12:28.001 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:27.999+0000 7f135012f640 1 -- 172.21.15.43:0/400155873 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f134806fe60 msgr2=0x7f1348146050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T15:12:28.001 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:27.999+0000 7f135012f640 1 --2- 172.21.15.43:0/400155873 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f134806fe60 0x7f1348146050 secure :-1 s=READY pgs=2607 cs=0 l=1 rev1=1 crypto rx=0x7f133800d620 tx=0x7f133800dae0 comp rx=0 tx=0).stop 2023-08-09T15:12:28.001 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:27.999+0000 7f135012f640 1 -- 172.21.15.43:0/400155873 shutdown_connections 2023-08-09T15:12:28.001 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:27.999+0000 7f135012f640 1 --2- 172.21.15.43:0/400155873 >> [v2:172.21.15.203:6832/1456699667,v1:172.21.15.203:6833/1456699667] conn(0x7f132c06a1a0 0x7f132c06c660 unknown :-1 s=CLOSED pgs=59 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T15:12:28.002 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:27.999+0000 7f135012f640 1 --2- 172.21.15.43:0/400155873 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f134815b310 0x7f13480c52c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T15:12:28.002 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:27.999+0000 7f135012f640 1 --2- 172.21.15.43:0/400155873 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f134806fe60 0x7f1348146050 unknown :-1 s=CLOSED pgs=2607 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T15:12:28.002 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:27.999+0000 7f135012f640 1 --2- 172.21.15.43:0/400155873 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f134806b4b0 0x7f1348145790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T15:12:28.002 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:27.999+0000 7f135012f640 1 -- 172.21.15.43:0/400155873 >> 172.21.15.43:0/400155873 conn(0x7f1348069d70 msgr2=0x7f13480b3870 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-09T15:12:28.002 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:27.999+0000 7f135012f640 1 -- 172.21.15.43:0/400155873 shutdown_connections 2023-08-09T15:12:28.002 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:27.999+0000 7f135012f640 1 -- 172.21.15.43:0/400155873 wait complete. 2023-08-09T15:12:28.020 INFO:teuthology.orchestra.run.smithi043.stdout:{"pg_ready":true,"pg_map":{"version":17,"stamp":"2023-08-09T15:12:27.366401+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2155,"num_read_kb":4508,"num_write":583,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":351,"ondisk_log_size":351,"up":18,"acting":18,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":18,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":393136,"kb_used_data":23616,"kb_used_omap":13,"kb_used_meta":369394,"kb_avail":754581584,"statfs":{"total":773094113280,"available":772691542016,"internally_reserved":0,"allocated":24182784,"data_stored":13716990,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":14248,"internal_metadata":378259544},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"11.728769"},"pg_stats":[{"pgid":"1.0","version":"0'0","reported_seq":425,"reported_epoch":229,"state":"active+clean","last_fresh":"2023-08-09T15:12:15.642682+0000","last_change":"2023-08-09T13:33:56.401406+0000","last_active":"2023-08-09T15:12:15.642682+0000","last_peered":"2023-08-09T15:12:15.642682+0000","last_clean":"2023-08-09T15:12:15.642682+0000","last_became_active":"2023-08-09T13:33:56.400487+0000","last_became_peered":"2023-08-09T13:33:56.400487+0000","last_unstale":"2023-08-09T15:12:15.642682+0000","last_undegraded":"2023-08-09T15:12:15.642682+0000","last_fullsized":"2023-08-09T15:12:15.642682+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:33:54.845869+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:33:54.845869+0000","last_clean_scrub_stamp":"2023-08-09T13:33:54.845869+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T15:26:23.628880+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":432,"reported_epoch":229,"state":"active+clean","last_fresh":"2023-08-09T15:12:16.906007+0000","last_change":"2023-08-09T13:33:56.401123+0000","last_active":"2023-08-09T15:12:16.906007+0000","last_peered":"2023-08-09T15:12:16.906007+0000","last_clean":"2023-08-09T15:12:16.906007+0000","last_became_active":"2023-08-09T13:33:56.399933+0000","last_became_peered":"2023-08-09T13:33:56.399933+0000","last_unstale":"2023-08-09T15:12:16.906007+0000","last_undegraded":"2023-08-09T15:12:16.906007+0000","last_fullsized":"2023-08-09T15:12:16.906007+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:33:54.845869+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:33:54.845869+0000","last_clean_scrub_stamp":"2023-08-09T13:33:54.845869+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-11T00:04:51.245862+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.2","version":"0'0","reported_seq":424,"reported_epoch":229,"state":"active+clean","last_fresh":"2023-08-09T15:12:15.641337+0000","last_change":"2023-08-09T13:33:56.402794+0000","last_active":"2023-08-09T15:12:15.641337+0000","last_peered":"2023-08-09T15:12:15.641337+0000","last_clean":"2023-08-09T15:12:15.641337+0000","last_became_active":"2023-08-09T13:33:56.402038+0000","last_became_peered":"2023-08-09T13:33:56.402038+0000","last_unstale":"2023-08-09T15:12:15.641337+0000","last_undegraded":"2023-08-09T15:12:15.641337+0000","last_fullsized":"2023-08-09T15:12:15.641337+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:33:54.845869+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:33:54.845869+0000","last_clean_scrub_stamp":"2023-08-09T13:33:54.845869+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T20:39:01.972486+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.3","version":"0'0","reported_seq":439,"reported_epoch":229,"state":"active+clean","last_fresh":"2023-08-09T15:12:15.644227+0000","last_change":"2023-08-09T13:33:55.886749+0000","last_active":"2023-08-09T15:12:15.644227+0000","last_peered":"2023-08-09T15:12:15.644227+0000","last_clean":"2023-08-09T15:12:15.644227+0000","last_became_active":"2023-08-09T13:33:55.886116+0000","last_became_peered":"2023-08-09T13:33:55.886116+0000","last_unstale":"2023-08-09T15:12:15.644227+0000","last_undegraded":"2023-08-09T15:12:15.644227+0000","last_fullsized":"2023-08-09T15:12:15.644227+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:33:54.845869+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:33:54.845869+0000","last_clean_scrub_stamp":"2023-08-09T13:33:54.845869+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"226'351","reported_seq":1404,"reported_epoch":229,"state":"active+clean","last_fresh":"2023-08-09T15:12:15.642688+0000","last_change":"2023-08-09T13:33:55.886345+0000","last_active":"2023-08-09T15:12:15.642688+0000","last_peered":"2023-08-09T15:12:15.642688+0000","last_clean":"2023-08-09T15:12:15.642688+0000","last_became_active":"2023-08-09T13:33:55.885781+0000","last_became_peered":"2023-08-09T13:33:55.885781+0000","last_unstale":"2023-08-09T15:12:15.642688+0000","last_undegraded":"2023-08-09T15:12:15.642688+0000","last_fullsized":"2023-08-09T15:12:15.642688+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:33:54.845869+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:33:54.845869+0000","last_clean_scrub_stamp":"2023-08-09T13:33:54.845869+0000","objects_scrubbed":0,"log_size":351,"log_dups_size":0,"ondisk_log_size":351,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T18:20:34.406519+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2155,"num_read_kb":4508,"num_write":583,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":442,"reported_epoch":229,"state":"active+clean","last_fresh":"2023-08-09T15:12:15.646193+0000","last_change":"2023-08-09T13:33:55.889565+0000","last_active":"2023-08-09T15:12:15.646193+0000","last_peered":"2023-08-09T15:12:15.646193+0000","last_clean":"2023-08-09T15:12:15.646193+0000","last_became_active":"2023-08-09T13:33:55.889164+0000","last_became_peered":"2023-08-09T13:33:55.889164+0000","last_unstale":"2023-08-09T15:12:15.646193+0000","last_undegraded":"2023-08-09T15:12:15.646193+0000","last_fullsized":"2023-08-09T15:12:15.646193+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:33:54.845869+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:33:54.845869+0000","last_clean_scrub_stamp":"2023-08-09T13:33:54.845869+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-11T01:05:24.824606+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":430,"reported_epoch":229,"state":"active+clean","last_fresh":"2023-08-09T15:12:17.176452+0000","last_change":"2023-08-09T13:33:56.404236+0000","last_active":"2023-08-09T15:12:17.176452+0000","last_peered":"2023-08-09T15:12:17.176452+0000","last_clean":"2023-08-09T15:12:17.176452+0000","last_became_active":"2023-08-09T13:33:56.402485+0000","last_became_peered":"2023-08-09T13:33:56.402485+0000","last_unstale":"2023-08-09T15:12:17.176452+0000","last_undegraded":"2023-08-09T15:12:17.176452+0000","last_fullsized":"2023-08-09T15:12:17.176452+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:33:54.845869+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:33:54.845869+0000","last_clean_scrub_stamp":"2023-08-09T13:33:54.845869+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T14:46:58.528793+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":422,"reported_epoch":229,"state":"active+clean","last_fresh":"2023-08-09T15:12:17.182618+0000","last_change":"2023-08-09T13:33:56.401781+0000","last_active":"2023-08-09T15:12:17.182618+0000","last_peered":"2023-08-09T15:12:17.182618+0000","last_clean":"2023-08-09T15:12:17.182618+0000","last_became_active":"2023-08-09T13:33:56.400336+0000","last_became_peered":"2023-08-09T13:33:56.400336+0000","last_unstale":"2023-08-09T15:12:17.182618+0000","last_undegraded":"2023-08-09T15:12:17.182618+0000","last_fullsized":"2023-08-09T15:12:17.182618+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:33:54.845869+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:33:54.845869+0000","last_clean_scrub_stamp":"2023-08-09T13:33:54.845869+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T19:20:37.616910+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":422,"reported_epoch":229,"state":"active+clean","last_fresh":"2023-08-09T15:12:17.186106+0000","last_change":"2023-08-09T13:33:55.886389+0000","last_active":"2023-08-09T15:12:17.186106+0000","last_peered":"2023-08-09T15:12:17.186106+0000","last_clean":"2023-08-09T15:12:17.186106+0000","last_became_active":"2023-08-09T13:33:55.885638+0000","last_became_peered":"2023-08-09T13:33:55.885638+0000","last_unstale":"2023-08-09T15:12:17.186106+0000","last_undegraded":"2023-08-09T15:12:17.186106+0000","last_fullsized":"2023-08-09T15:12:17.186106+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T13:33:54.845869+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T13:33:54.845869+0000","last_clean_scrub_stamp":"2023-08-09T13:33:54.845869+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T17:59:55.723013+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2155,"num_read_kb":4508,"num_write":583,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":351,"ondisk_log_size":351,"up":2,"acting":2,"num_store_stats":2},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":8}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834576022,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":52552,"kb_used_data":3388,"kb_used_omap":1,"kb_used_meta":49150,"kb_avail":94319288,"statfs":{"total":96636764160,"available":96582950912,"internally_reserved":0,"allocated":3469312,"data_stored":2157740,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1781,"internal_metadata":50329867},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Aug 9 15:11:52 2023","interfaces":[{"interface":"back","average":{"1min":1.113,"5min":1.234,"15min":1.2290000000000001},"min":{"1min":0.52400000000000002,"5min":0.52400000000000002,"15min":0.52400000000000002},"max":{"1min":1.6080000000000001,"5min":2.056,"15min":2.056},"last":0.98899999999999999},{"interface":"front","average":{"1min":1.0900000000000001,"5min":1.2110000000000001,"15min":1.2529999999999999},"min":{"1min":0.55800000000000005,"5min":0.55800000000000005,"15min":0.55800000000000005},"max":{"1min":1.5960000000000001,"5min":1.7769999999999999,"15min":1.863},"last":1.0589999999999999}]},{"osd":1,"last update":"Wed Aug 9 15:11:52 2023","interfaces":[{"interface":"back","average":{"1min":1.1479999999999999,"5min":1.28,"15min":1.3049999999999999},"min":{"1min":0.52500000000000002,"5min":0.52500000000000002,"15min":0.52500000000000002},"max":{"1min":1.6619999999999999,"5min":1.853,"15min":1.853},"last":1.0920000000000001},{"interface":"front","average":{"1min":1.175,"5min":1.296,"15min":1.3100000000000001},"min":{"1min":0.748,"5min":0.748,"15min":0.748},"max":{"1min":1.605,"5min":2.1520000000000001,"15min":2.1520000000000001},"last":0.83099999999999996}]},{"osd":2,"last update":"Wed Aug 9 15:11:52 2023","interfaces":[{"interface":"back","average":{"1min":1.093,"5min":1.262,"15min":1.2969999999999999},"min":{"1min":0.70399999999999996,"5min":0.70399999999999996,"15min":0.70399999999999996},"max":{"1min":1.7050000000000001,"5min":1.861,"15min":1.861},"last":1.0189999999999999},{"interface":"front","average":{"1min":1.214,"5min":1.3480000000000001,"15min":1.393},"min":{"1min":0.58099999999999996,"5min":0.58099999999999996,"15min":0.58099999999999996},"max":{"1min":1.7889999999999999,"5min":1.9370000000000001,"15min":1.9370000000000001},"last":1.3169999999999999}]},{"osd":3,"last update":"Wed Aug 9 15:11:52 2023","interfaces":[{"interface":"back","average":{"1min":1.1950000000000001,"5min":1.3160000000000001,"15min":1.351},"min":{"1min":0.80700000000000005,"5min":0.80700000000000005,"15min":0.80700000000000005},"max":{"1min":1.798,"5min":1.798,"15min":1.9259999999999999},"last":1.26},{"interface":"front","average":{"1min":1.335,"5min":1.4379999999999999,"15min":1.472},"min":{"1min":0.76200000000000001,"5min":0.76200000000000001,"15min":0.76200000000000001},"max":{"1min":1.9179999999999999,"5min":1.9239999999999999,"15min":1.9239999999999999},"last":1.1779999999999999}]},{"osd":4,"last update":"Wed Aug 9 15:11:52 2023","interfaces":[{"interface":"back","average":{"1min":1.218,"5min":1.282,"15min":1.2989999999999999},"min":{"1min":0.56399999999999995,"5min":0.56399999999999995,"15min":0.56399999999999995},"max":{"1min":1.6080000000000001,"5min":2.2810000000000001,"15min":2.2810000000000001},"last":0.95999999999999996},{"interface":"front","average":{"1min":1.28,"5min":1.3560000000000001,"15min":1.385},"min":{"1min":0.622,"5min":0.622,"15min":0.622},"max":{"1min":2.0390000000000001,"5min":2.0390000000000001,"15min":2.0830000000000002},"last":1.139}]},{"osd":5,"last update":"Wed Aug 9 15:11:52 2023","interfaces":[{"interface":"back","average":{"1min":1.3620000000000001,"5min":1.4219999999999999,"15min":1.4410000000000001},"min":{"1min":0.59299999999999997,"5min":0.59299999999999997,"15min":0.59299999999999997},"max":{"1min":2.0899999999999999,"5min":2.0899999999999999,"15min":2.0899999999999999},"last":1.411},{"interface":"front","average":{"1min":1.3959999999999999,"5min":1.448,"15min":1.4730000000000001},"min":{"1min":0.65500000000000003,"5min":0.65500000000000003,"15min":0.65500000000000003},"max":{"1min":2.206,"5min":2.5310000000000001,"15min":2.5310000000000001},"last":1.202}]},{"osd":6,"last update":"Wed Aug 9 15:11:52 2023","interfaces":[{"interface":"back","average":{"1min":1.5069999999999999,"5min":1.587,"15min":1.589},"min":{"1min":0.69199999999999995,"5min":0.69199999999999995,"15min":0.69199999999999995},"max":{"1min":2.294,"5min":2.8959999999999999,"15min":2.8959999999999999},"last":1.371},{"interface":"front","average":{"1min":1.359,"5min":1.5089999999999999,"15min":1.534},"min":{"1min":0.60899999999999999,"5min":0.60899999999999999,"15min":0.60899999999999999},"max":{"1min":2.161,"5min":2.214,"15min":2.222},"last":1.349}]}]},{"osd":6,"up_from":13,"seq":55834576022,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":60160,"kb_used_data":2804,"kb_used_omap":1,"kb_used_meta":57342,"kb_avail":94311680,"statfs":{"total":96636764160,"available":96575160320,"internally_reserved":0,"allocated":2871296,"data_stored":1566011,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1781,"internal_metadata":58718475},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Aug 9 15:11:57 2023","interfaces":[{"interface":"back","average":{"1min":1.1639999999999999,"5min":1.2130000000000001,"15min":1.2190000000000001},"min":{"1min":0.76700000000000002,"5min":0.75600000000000001,"15min":0.75600000000000001},"max":{"1min":1.8169999999999999,"5min":1.8169999999999999,"15min":1.8859999999999999},"last":0.80100000000000005},{"interface":"front","average":{"1min":1.0720000000000001,"5min":1.21,"15min":1.234},"min":{"1min":0.64500000000000002,"5min":0.64500000000000002,"15min":0.64500000000000002},"max":{"1min":1.5109999999999999,"5min":2.0800000000000001,"15min":2.0800000000000001},"last":1.2070000000000001}]},{"osd":1,"last update":"Wed Aug 9 15:11:57 2023","interfaces":[{"interface":"back","average":{"1min":1.095,"5min":1.1879999999999999,"15min":1.1910000000000001},"min":{"1min":0.65900000000000003,"5min":0.65900000000000003,"15min":0.65900000000000003},"max":{"1min":1.776,"5min":2.6760000000000002,"15min":2.6760000000000002},"last":1.0129999999999999},{"interface":"front","average":{"1min":1.2869999999999999,"5min":1.3120000000000001,"15min":1.3460000000000001},"min":{"1min":0.72599999999999998,"5min":0.72599999999999998,"15min":0.72599999999999998},"max":{"1min":1.579,"5min":1.998,"15min":1.998},"last":1.1519999999999999}]},{"osd":2,"last update":"Wed Aug 9 15:11:57 2023","interfaces":[{"interface":"back","average":{"1min":1.2629999999999999,"5min":1.3120000000000001,"15min":1.3400000000000001},"min":{"1min":0.79800000000000004,"5min":0.79800000000000004,"15min":0.79800000000000004},"max":{"1min":1.6659999999999999,"5min":2.0089999999999999,"15min":2.081},"last":1.3560000000000001},{"interface":"front","average":{"1min":1.3080000000000001,"5min":1.337,"15min":1.3480000000000001},"min":{"1min":0.73799999999999999,"5min":0.73799999999999999,"15min":0.73799999999999999},"max":{"1min":1.75,"5min":1.9219999999999999,"15min":1.9219999999999999},"last":1.4319999999999999}]},{"osd":3,"last update":"Wed Aug 9 15:11:57 2023","interfaces":[{"interface":"back","average":{"1min":1.302,"5min":1.4099999999999999,"15min":1.4019999999999999},"min":{"1min":0.70299999999999996,"5min":0.70299999999999996,"15min":0.70299999999999996},"max":{"1min":2.2669999999999999,"5min":2.2669999999999999,"15min":2.2669999999999999},"last":1.617},{"interface":"front","average":{"1min":1.2270000000000001,"5min":1.319,"15min":1.3420000000000001},"min":{"1min":0.59699999999999998,"5min":0.59699999999999998,"15min":0.59699999999999998},"max":{"1min":1.855,"5min":2.0539999999999998,"15min":2.0539999999999998},"last":1.2969999999999999}]},{"osd":4,"last update":"Wed Aug 9 15:11:57 2023","interfaces":[{"interface":"back","average":{"1min":1.339,"5min":1.3500000000000001,"15min":1.339},"min":{"1min":0.46500000000000002,"5min":0.46500000000000002,"15min":0.46500000000000002},"max":{"1min":1.7270000000000001,"5min":2.0470000000000002,"15min":2.0470000000000002},"last":1.494},{"interface":"front","average":{"1min":1.288,"5min":1.4470000000000001,"15min":1.4330000000000001},"min":{"1min":0.53000000000000003,"5min":0.53000000000000003,"15min":0.53000000000000003},"max":{"1min":1.6559999999999999,"5min":2.1589999999999998,"15min":2.1589999999999998},"last":1.7130000000000001}]},{"osd":5,"last update":"Wed Aug 9 15:11:57 2023","interfaces":[{"interface":"back","average":{"1min":1.3109999999999999,"5min":1.431,"15min":1.381},"min":{"1min":0.51100000000000001,"5min":0.51100000000000001,"15min":0.51100000000000001},"max":{"1min":1.9610000000000001,"5min":2.1269999999999998,"15min":2.6059999999999999},"last":1.5489999999999999},{"interface":"front","average":{"1min":1.546,"5min":1.637,"15min":1.6299999999999999},"min":{"1min":0.67300000000000004,"5min":0.67300000000000004,"15min":0.67300000000000004},"max":{"1min":1.885,"5min":2.125,"15min":2.1899999999999999},"last":1.6870000000000001}]},{"osd":7,"last update":"Wed Aug 9 15:11:57 2023","interfaces":[{"interface":"back","average":{"1min":1.506,"5min":1.639,"15min":1.6180000000000001},"min":{"1min":0.626,"5min":0.626,"15min":0.626},"max":{"1min":1.9319999999999999,"5min":2.2280000000000002,"15min":2.4340000000000002},"last":1.772},{"interface":"front","average":{"1min":1.45,"5min":1.6180000000000001,"15min":1.6000000000000001},"min":{"1min":0.44600000000000001,"5min":0.44600000000000001,"15min":0.44600000000000001},"max":{"1min":2.0409999999999999,"5min":2.3300000000000001,"15min":2.7400000000000002},"last":1.657}]}]},{"osd":5,"up_from":13,"seq":55834576022,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":47876,"kb_used_data":2808,"kb_used_omap":1,"kb_used_meta":45054,"kb_avail":94323964,"statfs":{"total":96636764160,"available":96587739136,"internally_reserved":0,"allocated":2875392,"data_stored":1567372,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1781,"internal_metadata":46135563},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Aug 9 15:11:47 2023","interfaces":[{"interface":"back","average":{"1min":1.0669999999999999,"5min":1.1919999999999999,"15min":1.194},"min":{"1min":0.57699999999999996,"5min":0.57699999999999996,"15min":0.57699999999999996},"max":{"1min":1.494,"5min":1.7370000000000001,"15min":1.754},"last":1.5149999999999999},{"interface":"front","average":{"1min":1.0389999999999999,"5min":1.167,"15min":1.208},"min":{"1min":0.55600000000000005,"5min":0.55600000000000005,"15min":0.55600000000000005},"max":{"1min":1.3480000000000001,"5min":1.6870000000000001,"15min":1.976},"last":0.97399999999999998}]},{"osd":1,"last update":"Wed Aug 9 15:11:47 2023","interfaces":[{"interface":"back","average":{"1min":1.2490000000000001,"5min":1.28,"15min":1.2509999999999999},"min":{"1min":0.67500000000000004,"5min":0.65200000000000002,"15min":0.65200000000000002},"max":{"1min":1.9490000000000001,"5min":2.016,"15min":2.016},"last":0.90400000000000003},{"interface":"front","average":{"1min":1.2150000000000001,"5min":1.288,"15min":1.343},"min":{"1min":0.65000000000000002,"5min":0.65000000000000002,"15min":0.65000000000000002},"max":{"1min":2.0169999999999999,"5min":2.0169999999999999,"15min":2.0169999999999999},"last":1.403}]},{"osd":2,"last update":"Wed Aug 9 15:11:47 2023","interfaces":[{"interface":"back","average":{"1min":1.0269999999999999,"5min":1.2629999999999999,"15min":1.3009999999999999},"min":{"1min":0.60199999999999998,"5min":0.60199999999999998,"15min":0.60199999999999998},"max":{"1min":1.48,"5min":1.8859999999999999,"15min":2.032},"last":1.288},{"interface":"front","average":{"1min":1.3,"5min":1.339,"15min":1.3640000000000001},"min":{"1min":0.61899999999999999,"5min":0.61899999999999999,"15min":0.61899999999999999},"max":{"1min":1.7010000000000001,"5min":1.956,"15min":1.964},"last":1.3420000000000001}]},{"osd":3,"last update":"Wed Aug 9 15:11:47 2023","interfaces":[{"interface":"back","average":{"1min":1.3,"5min":1.3740000000000001,"15min":1.3939999999999999},"min":{"1min":0.746,"5min":0.746,"15min":0.746},"max":{"1min":1.6639999999999999,"5min":1.798,"15min":1.899},"last":1.1970000000000001},{"interface":"front","average":{"1min":1.278,"5min":1.389,"15min":1.4370000000000001},"min":{"1min":0.66400000000000003,"5min":0.66400000000000003,"15min":0.66400000000000003},"max":{"1min":1.6559999999999999,"5min":1.8819999999999999,"15min":2.0379999999999998},"last":1.4610000000000001}]},{"osd":4,"last update":"Wed Aug 9 15:11:47 2023","interfaces":[{"interface":"back","average":{"1min":1.347,"5min":1.323,"15min":1.3220000000000001},"min":{"1min":0.53000000000000003,"5min":0.53000000000000003,"15min":0.53000000000000003},"max":{"1min":1.833,"5min":2.0299999999999998,"15min":2.0299999999999998},"last":1.143},{"interface":"front","average":{"1min":1.4419999999999999,"5min":1.389,"15min":1.4059999999999999},"min":{"1min":0.69199999999999995,"5min":0.69199999999999995,"15min":0.69199999999999995},"max":{"1min":1.7709999999999999,"5min":1.9239999999999999,"15min":2.0390000000000001},"last":1.6259999999999999}]},{"osd":6,"last update":"Wed Aug 9 15:11:47 2023","interfaces":[{"interface":"back","average":{"1min":1.298,"5min":1.3660000000000001,"15min":1.411},"min":{"1min":0.501,"5min":0.501,"15min":0.501},"max":{"1min":1.728,"5min":2.032,"15min":2.1000000000000001},"last":1.0860000000000001},{"interface":"front","average":{"1min":1.48,"5min":1.4850000000000001,"15min":1.5029999999999999},"min":{"1min":0.70599999999999996,"5min":0.70599999999999996,"15min":0.70599999999999996},"max":{"1min":2.125,"5min":2.1349999999999998,"15min":2.1349999999999998},"last":1.0149999999999999}]},{"osd":7,"last update":"Wed Aug 9 15:11:47 2023","interfaces":[{"interface":"back","average":{"1min":1.5469999999999999,"5min":1.554,"15min":1.5800000000000001},"min":{"1min":0.71999999999999997,"5min":0.71999999999999997,"15min":0.71999999999999997},"max":{"1min":2.0760000000000001,"5min":2.0760000000000001,"15min":2.4689999999999999},"last":1.5760000000000001},{"interface":"front","average":{"1min":1.5309999999999999,"5min":1.595,"15min":1.637},"min":{"1min":0.63800000000000001,"5min":0.63800000000000001,"15min":0.63800000000000001},"max":{"1min":2.1709999999999998,"5min":2.1709999999999998,"15min":2.4239999999999999},"last":1.536}]}]},{"osd":4,"up_from":13,"seq":55834576022,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":51972,"kb_used_data":2808,"kb_used_omap":1,"kb_used_meta":49150,"kb_avail":94319868,"statfs":{"total":96636764160,"available":96583544832,"internally_reserved":0,"allocated":2875392,"data_stored":1567372,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1781,"internal_metadata":50329867},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Aug 9 15:12:13 2023","interfaces":[{"interface":"back","average":{"1min":1.2370000000000001,"5min":1.196,"15min":1.222},"min":{"1min":0.64700000000000002,"5min":0.60399999999999998,"15min":0.60399999999999998},"max":{"1min":1.8169999999999999,"5min":1.8169999999999999,"15min":2.2959999999999998},"last":0.93700000000000006},{"interface":"front","average":{"1min":1.2270000000000001,"5min":1.2829999999999999,"15min":1.3129999999999999},"min":{"1min":0.77000000000000002,"5min":0.66800000000000004,"15min":0.66800000000000004},"max":{"1min":1.772,"5min":2.0779999999999998,"15min":2.2709999999999999},"last":0.76100000000000001}]},{"osd":1,"last update":"Wed Aug 9 15:12:13 2023","interfaces":[{"interface":"back","average":{"1min":1.1439999999999999,"5min":1.288,"15min":1.3460000000000001},"min":{"1min":0.73199999999999998,"5min":0.73199999999999998,"15min":0.73199999999999998},"max":{"1min":1.9370000000000001,"5min":1.9370000000000001,"15min":1.9370000000000001},"last":1.077},{"interface":"front","average":{"1min":1.2430000000000001,"5min":1.2609999999999999,"15min":1.278},"min":{"1min":0.76400000000000001,"5min":0.73899999999999999,"15min":0.73899999999999999},"max":{"1min":1.865,"5min":1.865,"15min":2.117},"last":0.72699999999999998}]},{"osd":2,"last update":"Wed Aug 9 15:12:13 2023","interfaces":[{"interface":"back","average":{"1min":1.1859999999999999,"5min":1.2350000000000001,"15min":1.2829999999999999},"min":{"1min":0.70199999999999996,"5min":0.63100000000000001,"15min":0.63100000000000001},"max":{"1min":1.5469999999999999,"5min":1.7709999999999999,"15min":1.883},"last":0.84099999999999997},{"interface":"front","average":{"1min":1.298,"5min":1.3819999999999999,"15min":1.401},"min":{"1min":0.91400000000000003,"5min":0.82699999999999996,"15min":0.79200000000000004},"max":{"1min":1.903,"5min":2.343,"15min":2.343},"last":0.70299999999999996}]},{"osd":3,"last update":"Wed Aug 9 15:12:13 2023","interfaces":[{"interface":"back","average":{"1min":1.2789999999999999,"5min":1.3420000000000001,"15min":1.349},"min":{"1min":0.95299999999999996,"5min":0.90500000000000003,"15min":0.88700000000000001},"max":{"1min":1.6970000000000001,"5min":1.8009999999999999,"15min":2.4870000000000001},"last":0.78300000000000003},{"interface":"front","average":{"1min":1.3720000000000001,"5min":1.423,"15min":1.464},"min":{"1min":0.95299999999999996,"5min":0.79700000000000004,"15min":0.79700000000000004},"max":{"1min":1.8240000000000001,"5min":1.909,"15min":2.5459999999999998},"last":0.96899999999999997}]},{"osd":5,"last update":"Wed Aug 9 15:12:13 2023","interfaces":[{"interface":"back","average":{"1min":1.431,"5min":1.385,"15min":1.409},"min":{"1min":1.0029999999999999,"5min":0.82199999999999995,"15min":0.82199999999999995},"max":{"1min":2.004,"5min":2.633,"15min":2.633},"last":0.53200000000000003},{"interface":"front","average":{"1min":1.395,"5min":1.4059999999999999,"15min":1.393},"min":{"1min":1.048,"5min":0.68100000000000005,"15min":0.68100000000000005},"max":{"1min":1.8300000000000001,"5min":1.9930000000000001,"15min":2.4249999999999998},"last":0.65600000000000003}]},{"osd":6,"last update":"Wed Aug 9 15:12:13 2023","interfaces":[{"interface":"back","average":{"1min":1.3919999999999999,"5min":1.4330000000000001,"15min":1.4099999999999999},"min":{"1min":1.0509999999999999,"5min":0.71299999999999997,"15min":0.71299999999999997},"max":{"1min":1.73,"5min":2.0049999999999999,"15min":2.9119999999999999},"last":0.63},{"interface":"front","average":{"1min":1.528,"5min":1.5680000000000001,"15min":1.579},"min":{"1min":1.1699999999999999,"5min":0.77000000000000002,"15min":0.77000000000000002},"max":{"1min":1.8680000000000001,"5min":2.9529999999999998,"15min":2.9529999999999998},"last":0.58199999999999996}]},{"osd":7,"last update":"Wed Aug 9 15:12:13 2023","interfaces":[{"interface":"back","average":{"1min":1.5029999999999999,"5min":1.5309999999999999,"15min":1.5229999999999999},"min":{"1min":1.1719999999999999,"5min":0.83899999999999997,"15min":0.83899999999999997},"max":{"1min":1.9370000000000001,"5min":2.1589999999999998,"15min":2.5910000000000002},"last":0.59499999999999997},{"interface":"front","average":{"1min":1.599,"5min":1.6140000000000001,"15min":1.6060000000000001},"min":{"1min":1.206,"5min":0.90000000000000002,"15min":0.90000000000000002},"max":{"1min":1.8999999999999999,"5min":2.218,"15min":3.028},"last":0.68500000000000005}]}]},{"osd":3,"up_from":13,"seq":55834576023,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":47876,"kb_used_data":2808,"kb_used_omap":1,"kb_used_meta":45054,"kb_avail":94323964,"statfs":{"total":96636764160,"available":96587739136,"internally_reserved":0,"allocated":2875392,"data_stored":1567372,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1781,"internal_metadata":46135563},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Aug 9 15:12:06 2023","interfaces":[{"interface":"back","average":{"1min":0.75800000000000001,"5min":0.94699999999999995,"15min":0.96899999999999997},"min":{"1min":0.42799999999999999,"5min":0.42799999999999999,"15min":0.42799999999999999},"max":{"1min":1.24,"5min":1.6699999999999999,"15min":2.2090000000000001},"last":0.50700000000000001},{"interface":"front","average":{"1min":0.872,"5min":1.093,"15min":1.1439999999999999},"min":{"1min":0.434,"5min":0.434,"15min":0.434},"max":{"1min":1.4550000000000001,"5min":1.9630000000000001,"15min":1.9630000000000001},"last":0.65600000000000003}]},{"osd":1,"last update":"Wed Aug 9 15:12:06 2023","interfaces":[{"interface":"back","average":{"1min":0.90500000000000003,"5min":1.006,"15min":1.0660000000000001},"min":{"1min":0.437,"5min":0.437,"15min":0.437},"max":{"1min":1.595,"5min":1.956,"15min":2.27},"last":0.68100000000000005},{"interface":"front","average":{"1min":1.0960000000000001,"5min":1.2070000000000001,"15min":1.2569999999999999},"min":{"1min":0.63900000000000001,"5min":0.498,"15min":0.498},"max":{"1min":1.6140000000000001,"5min":1.78,"15min":1.9730000000000001},"last":0.71899999999999997}]},{"osd":2,"last update":"Wed Aug 9 15:12:06 2023","interfaces":[{"interface":"back","average":{"1min":1.024,"5min":1.236,"15min":1.3180000000000001},"min":{"1min":0.65400000000000003,"5min":0.65400000000000003,"15min":0.65400000000000003},"max":{"1min":1.3839999999999999,"5min":1.9650000000000001,"15min":2.4079999999999999},"last":0.74199999999999999},{"interface":"front","average":{"1min":1.2250000000000001,"5min":1.3720000000000001,"15min":1.4279999999999999},"min":{"1min":0.63400000000000001,"5min":0.63400000000000001,"15min":0.63400000000000001},"max":{"1min":1.7090000000000001,"5min":2.0510000000000002,"15min":2.1259999999999999},"last":0.79200000000000004}]},{"osd":4,"last update":"Wed Aug 9 15:12:06 2023","interfaces":[{"interface":"back","average":{"1min":1.2070000000000001,"5min":1.3260000000000001,"15min":1.3959999999999999},"min":{"1min":0.752,"5min":0.748,"15min":0.72599999999999998},"max":{"1min":1.744,"5min":1.9179999999999999,"15min":2.2069999999999999},"last":0.88300000000000001},{"interface":"front","average":{"1min":1.3220000000000001,"5min":1.4199999999999999,"15min":1.4570000000000001},"min":{"1min":0.77500000000000002,"5min":0.77500000000000002,"15min":0.77500000000000002},"max":{"1min":1.8939999999999999,"5min":1.9279999999999999,"15min":2.056},"last":1.179}]},{"osd":5,"last update":"Wed Aug 9 15:12:06 2023","interfaces":[{"interface":"back","average":{"1min":1.4039999999999999,"5min":1.502,"15min":1.538},"min":{"1min":0.77600000000000002,"5min":0.77600000000000002,"15min":0.77600000000000002},"max":{"1min":2.1890000000000001,"5min":2.1890000000000001,"15min":2.1890000000000001},"last":0.90500000000000003},{"interface":"front","average":{"1min":1.3740000000000001,"5min":1.5580000000000001,"15min":1.6000000000000001},"min":{"1min":0.88800000000000001,"5min":0.85799999999999998,"15min":0.85799999999999998},"max":{"1min":1.7330000000000001,"5min":2.2469999999999999,"15min":2.4049999999999998},"last":0.85499999999999998}]},{"osd":6,"last update":"Wed Aug 9 15:12:06 2023","interfaces":[{"interface":"back","average":{"1min":1.3560000000000001,"5min":1.4890000000000001,"15min":1.5780000000000001},"min":{"1min":0.69599999999999995,"5min":0.69599999999999995,"15min":0.69599999999999995},"max":{"1min":1.9670000000000001,"5min":2.4860000000000002,"15min":2.4860000000000002},"last":0.92900000000000005},{"interface":"front","average":{"1min":1.458,"5min":1.5940000000000001,"15min":1.653},"min":{"1min":0.89700000000000002,"5min":0.89700000000000002,"15min":0.89700000000000002},"max":{"1min":2.04,"5min":2.4039999999999999,"15min":2.4039999999999999},"last":0.99099999999999999}]},{"osd":7,"last update":"Wed Aug 9 15:12:06 2023","interfaces":[{"interface":"back","average":{"1min":1.5620000000000001,"5min":1.696,"15min":1.7490000000000001},"min":{"1min":0.94799999999999995,"5min":0.94799999999999995,"15min":0.94799999999999995},"max":{"1min":1.8680000000000001,"5min":2.347,"15min":2.347},"last":1.202},{"interface":"front","average":{"1min":1.4179999999999999,"5min":1.6220000000000001,"15min":1.694},"min":{"1min":0.81100000000000005,"5min":0.81100000000000005,"15min":0.81100000000000005},"max":{"1min":2.1309999999999998,"5min":3.3220000000000001,"15min":3.3220000000000001},"last":1.141}]}]},{"osd":2,"up_from":13,"seq":55834576022,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":46220,"kb_used_data":2808,"kb_used_omap":1,"kb_used_meta":43390,"kb_avail":94325620,"statfs":{"total":96636764160,"available":96589434880,"internally_reserved":0,"allocated":2875392,"data_stored":1567372,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1781,"internal_metadata":44431627},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Aug 9 15:11:52 2023","interfaces":[{"interface":"back","average":{"1min":0.874,"5min":0.98499999999999999,"15min":0.95899999999999996},"min":{"1min":0.48199999999999998,"5min":0.44800000000000001,"15min":0.44800000000000001},"max":{"1min":1.591,"5min":1.591,"15min":1.591},"last":0.81000000000000005},{"interface":"front","average":{"1min":0.96399999999999997,"5min":1.0760000000000001,"15min":1.085},"min":{"1min":0.38900000000000001,"5min":0.38900000000000001,"15min":0.38900000000000001},"max":{"1min":1.5980000000000001,"5min":1.764,"15min":2.052},"last":0.76300000000000001}]},{"osd":1,"last update":"Wed Aug 9 15:11:52 2023","interfaces":[{"interface":"back","average":{"1min":0.98299999999999998,"5min":1.2330000000000001,"15min":1.2230000000000001},"min":{"1min":0.55400000000000005,"5min":0.55400000000000005,"15min":0.55400000000000005},"max":{"1min":1.403,"5min":2.0179999999999998,"15min":2.0179999999999998},"last":1.0289999999999999},{"interface":"front","average":{"1min":1.1359999999999999,"5min":1.292,"15min":1.381},"min":{"1min":0.51200000000000001,"5min":0.51200000000000001,"15min":0.51200000000000001},"max":{"1min":1.6639999999999999,"5min":1.95,"15min":1.95},"last":1.1140000000000001}]},{"osd":3,"last update":"Wed Aug 9 15:11:52 2023","interfaces":[{"interface":"back","average":{"1min":0.99099999999999999,"5min":1.089,"15min":1.1080000000000001},"min":{"1min":0.53500000000000003,"5min":0.52100000000000002,"15min":0.52100000000000002},"max":{"1min":1.5649999999999999,"5min":1.871,"15min":1.871},"last":1.1990000000000001},{"interface":"front","average":{"1min":1.111,"5min":1.339,"15min":1.369},"min":{"1min":0.57099999999999995,"5min":0.57099999999999995,"15min":0.57099999999999995},"max":{"1min":1.6930000000000001,"5min":1.9299999999999999,"15min":2.0470000000000002},"last":1.262}]},{"osd":4,"last update":"Wed Aug 9 15:11:52 2023","interfaces":[{"interface":"back","average":{"1min":1.369,"5min":1.5229999999999999,"15min":1.5469999999999999},"min":{"1min":0.86399999999999999,"5min":0.86399999999999999,"15min":0.86399999999999999},"max":{"1min":1.7350000000000001,"5min":1.994,"15min":2.464},"last":1.4119999999999999},{"interface":"front","average":{"1min":1.395,"5min":1.4399999999999999,"15min":1.464},"min":{"1min":0.93600000000000005,"5min":0.81799999999999995,"15min":0.81799999999999995},"max":{"1min":1.9370000000000001,"5min":1.98,"15min":2.0720000000000001},"last":1.1830000000000001}]},{"osd":5,"last update":"Wed Aug 9 15:11:52 2023","interfaces":[{"interface":"back","average":{"1min":1.4279999999999999,"5min":1.5009999999999999,"15min":1.5169999999999999},"min":{"1min":0.64800000000000002,"5min":0.64800000000000002,"15min":0.64800000000000002},"max":{"1min":1.913,"5min":2.0369999999999999,"15min":2.1139999999999999},"last":1.329},{"interface":"front","average":{"1min":1.351,"5min":1.419,"15min":1.4870000000000001},"min":{"1min":0.83399999999999996,"5min":0.83399999999999996,"15min":0.83399999999999996},"max":{"1min":2.1520000000000001,"5min":2.1520000000000001,"15min":2.214},"last":1.359}]},{"osd":6,"last update":"Wed Aug 9 15:11:52 2023","interfaces":[{"interface":"back","average":{"1min":1.407,"5min":1.4890000000000001,"15min":1.5229999999999999},"min":{"1min":0.89500000000000002,"5min":0.89500000000000002,"15min":0.89500000000000002},"max":{"1min":2.0680000000000001,"5min":2.1110000000000002,"15min":2.2999999999999998},"last":1.6080000000000001},{"interface":"front","average":{"1min":1.512,"5min":1.6479999999999999,"15min":1.6379999999999999},"min":{"1min":0.998,"5min":0.998,"15min":0.998},"max":{"1min":2.4140000000000001,"5min":2.4140000000000001,"15min":2.4140000000000001},"last":1.548}]},{"osd":7,"last update":"Wed Aug 9 15:11:52 2023","interfaces":[{"interface":"back","average":{"1min":1.5249999999999999,"5min":1.605,"15min":1.6419999999999999},"min":{"1min":0.96299999999999997,"5min":0.87,"15min":0.87},"max":{"1min":2.1000000000000001,"5min":2.1749999999999998,"15min":2.302},"last":1.4710000000000001},{"interface":"front","average":{"1min":1.4650000000000001,"5min":1.5800000000000001,"15min":1.6299999999999999},"min":{"1min":0.71399999999999997,"5min":0.71399999999999997,"15min":0.71399999999999997},"max":{"1min":2.2360000000000002,"5min":2.242,"15min":2.3809999999999998},"last":1.762}]}]},{"osd":1,"up_from":13,"seq":55834576022,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":56656,"kb_used_data":3388,"kb_used_omap":1,"kb_used_meta":53246,"kb_avail":94315184,"statfs":{"total":96636764160,"available":96578748416,"internally_reserved":0,"allocated":3469312,"data_stored":2157740,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1781,"internal_metadata":54524171},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Aug 9 15:12:03 2023","interfaces":[{"interface":"back","average":{"1min":0.83399999999999996,"5min":0.97399999999999998,"15min":1},"min":{"1min":0.32100000000000001,"5min":0.32100000000000001,"15min":0.32100000000000001},"max":{"1min":1.153,"5min":1.8799999999999999,"15min":2.0939999999999999},"last":0.68000000000000005},{"interface":"front","average":{"1min":0.94299999999999995,"5min":1.1060000000000001,"15min":1.1399999999999999},"min":{"1min":0.40500000000000003,"5min":0.40500000000000003,"15min":0.40500000000000003},"max":{"1min":1.641,"5min":1.78,"15min":1.8939999999999999},"last":0.78800000000000003}]},{"osd":2,"last update":"Wed Aug 9 15:12:03 2023","interfaces":[{"interface":"back","average":{"1min":0.93700000000000006,"5min":1.018,"15min":1.04},"min":{"1min":0.626,"5min":0.55800000000000005,"15min":0.55800000000000005},"max":{"1min":2.2240000000000002,"5min":2.2240000000000002,"15min":2.3940000000000001},"last":0.57599999999999996},{"interface":"front","average":{"1min":0.97399999999999998,"5min":1.1759999999999999,"15min":1.254},"min":{"1min":0.49299999999999999,"5min":0.46700000000000003,"15min":0.46700000000000003},"max":{"1min":1.5149999999999999,"5min":2.169,"15min":2.169},"last":0.755}]},{"osd":3,"last update":"Wed Aug 9 15:12:03 2023","interfaces":[{"interface":"back","average":{"1min":1.115,"5min":1.3340000000000001,"15min":1.381},"min":{"1min":0.54600000000000004,"5min":0.51000000000000001,"15min":0.51000000000000001},"max":{"1min":1.833,"5min":2.093,"15min":2.109},"last":0.88300000000000001},{"interface":"front","average":{"1min":1.0409999999999999,"5min":1.256,"15min":1.3089999999999999},"min":{"1min":0.59099999999999997,"5min":0.59099999999999997,"15min":0.59099999999999997},"max":{"1min":1.8680000000000001,"5min":2.052,"15min":2.052},"last":0.71499999999999997}]},{"osd":4,"last update":"Wed Aug 9 15:12:03 2023","interfaces":[{"interface":"back","average":{"1min":1.254,"5min":1.4339999999999999,"15min":1.4730000000000001},"min":{"1min":0.82099999999999995,"5min":0.753,"15min":0.753},"max":{"1min":1.887,"5min":2.0419999999999998,"15min":2.214},"last":1.1120000000000001},{"interface":"front","average":{"1min":1.202,"5min":1.381,"15min":1.4119999999999999},"min":{"1min":0.84499999999999997,"5min":0.69199999999999995,"15min":0.69199999999999995},"max":{"1min":1.528,"5min":2.0310000000000001,"15min":2.1190000000000002},"last":0.92000000000000004}]},{"osd":5,"last update":"Wed Aug 9 15:12:03 2023","interfaces":[{"interface":"back","average":{"1min":1.2450000000000001,"5min":1.415,"15min":1.446},"min":{"1min":0.83399999999999996,"5min":0.77800000000000002,"15min":0.77800000000000002},"max":{"1min":1.909,"5min":1.994,"15min":2.508},"last":1.083},{"interface":"front","average":{"1min":1.2829999999999999,"5min":1.53,"15min":1.5800000000000001},"min":{"1min":0.74299999999999999,"5min":0.74299999999999999,"15min":0.74299999999999999},"max":{"1min":1.99,"5min":2.3809999999999998,"15min":2.3809999999999998},"last":1.1479999999999999}]},{"osd":6,"last update":"Wed Aug 9 15:12:03 2023","interfaces":[{"interface":"back","average":{"1min":1.321,"5min":1.575,"15min":1.613},"min":{"1min":0.80100000000000005,"5min":0.80100000000000005,"15min":0.80100000000000005},"max":{"1min":2.1669999999999998,"5min":2.347,"15min":2.347},"last":1.1830000000000001},{"interface":"front","average":{"1min":1.288,"5min":1.522,"15min":1.5720000000000001},"min":{"1min":0.84999999999999998,"5min":0.73499999999999999,"15min":0.73499999999999999},"max":{"1min":1.8140000000000001,"5min":2.218,"15min":2.278},"last":1.2030000000000001}]},{"osd":7,"last update":"Wed Aug 9 15:12:03 2023","interfaces":[{"interface":"back","average":{"1min":1.3959999999999999,"5min":1.6439999999999999,"15min":1.6910000000000001},"min":{"1min":0.90900000000000003,"5min":0.86399999999999999,"15min":0.86399999999999999},"max":{"1min":2.1299999999999999,"5min":2.3090000000000002,"15min":2.488},"last":1.048},{"interface":"front","average":{"1min":1.385,"5min":1.633,"15min":1.698},"min":{"1min":0.75600000000000001,"5min":0.75600000000000001,"15min":0.75600000000000001},"max":{"1min":2.4980000000000002,"5min":2.4980000000000002,"15min":2.4980000000000002},"last":0.94799999999999995}]}]},{"osd":0,"up_from":13,"seq":55834576022,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":29824,"kb_used_data":2804,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94342016,"statfs":{"total":96636764160,"available":96606224384,"internally_reserved":0,"allocated":2871296,"data_stored":1566011,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1781,"internal_metadata":27654411},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Wed Aug 9 15:12:00 2023","interfaces":[{"interface":"back","average":{"1min":0.85099999999999998,"5min":0.89600000000000002,"15min":0.95099999999999996},"min":{"1min":0.44900000000000001,"5min":0.40000000000000002,"15min":0.40000000000000002},"max":{"1min":1.3380000000000001,"5min":1.617,"15min":1.6299999999999999},"last":0.88200000000000001},{"interface":"front","average":{"1min":0.98799999999999999,"5min":1.091,"15min":1.135},"min":{"1min":0.56599999999999995,"5min":0.56599999999999995,"15min":0.56599999999999995},"max":{"1min":1.635,"5min":2.1560000000000001,"15min":2.1560000000000001},"last":1.546}]},{"osd":2,"last update":"Wed Aug 9 15:12:00 2023","interfaces":[{"interface":"back","average":{"1min":0.873,"5min":1.097,"15min":1.1719999999999999},"min":{"1min":0.54000000000000004,"5min":0.505,"15min":0.505},"max":{"1min":1.401,"5min":2.4769999999999999,"15min":2.4769999999999999},"last":1.1919999999999999},{"interface":"front","average":{"1min":1.0069999999999999,"5min":1.125,"15min":1.2150000000000001},"min":{"1min":0.55500000000000005,"5min":0.53100000000000003,"15min":0.53100000000000003},"max":{"1min":1.468,"5min":1.8180000000000001,"15min":1.9470000000000001},"last":1.3999999999999999}]},{"osd":3,"last update":"Wed Aug 9 15:12:00 2023","interfaces":[{"interface":"back","average":{"1min":0.91200000000000003,"5min":1.1040000000000001,"15min":1.1870000000000001},"min":{"1min":0.45500000000000002,"5min":0.45500000000000002,"15min":0.45500000000000002},"max":{"1min":1.629,"5min":2.1459999999999999,"15min":2.1459999999999999},"last":0.97499999999999998},{"interface":"front","average":{"1min":1.079,"5min":1.284,"15min":1.3779999999999999},"min":{"1min":0.53100000000000003,"5min":0.53100000000000003,"15min":0.53100000000000003},"max":{"1min":1.5640000000000001,"5min":1.867,"15min":2.0760000000000001},"last":1.5009999999999999}]},{"osd":4,"last update":"Wed Aug 9 15:12:00 2023","interfaces":[{"interface":"back","average":{"1min":1.163,"5min":1.321,"15min":1.403},"min":{"1min":0.81799999999999995,"5min":0.81799999999999995,"15min":0.81799999999999995},"max":{"1min":1.5629999999999999,"5min":1.853,"15min":1.966},"last":1.464},{"interface":"front","average":{"1min":1.296,"5min":1.421,"15min":1.5069999999999999},"min":{"1min":0.83099999999999996,"5min":0.79400000000000004,"15min":0.79400000000000004},"max":{"1min":1.734,"5min":1.972,"15min":2.278},"last":1.591}]},{"osd":5,"last update":"Wed Aug 9 15:12:00 2023","interfaces":[{"interface":"back","average":{"1min":1.256,"5min":1.476,"15min":1.5129999999999999},"min":{"1min":0.84699999999999998,"5min":0.84699999999999998,"15min":0.84699999999999998},"max":{"1min":1.79,"5min":2.355,"15min":2.355},"last":1.6890000000000001},{"interface":"front","average":{"1min":1.2849999999999999,"5min":1.4299999999999999,"15min":1.49},"min":{"1min":0.84799999999999998,"5min":0.82899999999999996,"15min":0.82899999999999996},"max":{"1min":2.089,"5min":2.089,"15min":2.3740000000000001},"last":1.123}]},{"osd":6,"last update":"Wed Aug 9 15:12:00 2023","interfaces":[{"interface":"back","average":{"1min":1.3129999999999999,"5min":1.5249999999999999,"15min":1.6160000000000001},"min":{"1min":0.79700000000000004,"5min":0.79700000000000004,"15min":0.79700000000000004},"max":{"1min":1.7889999999999999,"5min":2.7490000000000001,"15min":2.7490000000000001},"last":1.365},{"interface":"front","average":{"1min":1.3839999999999999,"5min":1.534,"15min":1.597},"min":{"1min":0.89200000000000002,"5min":0.84699999999999998,"15min":0.84699999999999998},"max":{"1min":2.0840000000000001,"5min":2.274,"15min":2.2839999999999998},"last":1.623}]},{"osd":7,"last update":"Wed Aug 9 15:12:00 2023","interfaces":[{"interface":"back","average":{"1min":1.4710000000000001,"5min":1.599,"15min":1.665},"min":{"1min":0.98599999999999999,"5min":0.94899999999999995,"15min":0.94899999999999995},"max":{"1min":2.2989999999999999,"5min":2.645,"15min":2.645},"last":1.746},{"interface":"front","average":{"1min":1.4019999999999999,"5min":1.54,"15min":1.609},"min":{"1min":0.82599999999999996,"5min":0.82599999999999996,"15min":0.82599999999999996},"max":{"1min":2.1829999999999998,"5min":2.1829999999999998,"15min":2.3759999999999999},"last":1.236}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-08-09T15:12:28.025 INFO:tasks.ceph:pgid 1.0 last_scrub_stamp 2023-08-09T13:33:54.845869+0000 time.struct_time(tm_year=2023, tm_mon=8, tm_mday=9, tm_hour=13, tm_min=33, tm_sec=54, tm_wday=2, tm_yday=221, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=8, tm_mday=9, tm_hour=15, tm_min=12, tm_sec=20, tm_wday=2, tm_yday=221, tm_isdst=0) 2023-08-09T15:12:28.026 INFO:tasks.ceph:pgid 1.1 last_scrub_stamp 2023-08-09T13:33:54.845869+0000 time.struct_time(tm_year=2023, tm_mon=8, tm_mday=9, tm_hour=13, tm_min=33, tm_sec=54, tm_wday=2, tm_yday=221, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=8, tm_mday=9, tm_hour=15, tm_min=12, tm_sec=20, tm_wday=2, tm_yday=221, tm_isdst=0) 2023-08-09T15:12:28.026 INFO:tasks.ceph:pgid 1.2 last_scrub_stamp 2023-08-09T13:33:54.845869+0000 time.struct_time(tm_year=2023, tm_mon=8, tm_mday=9, tm_hour=13, tm_min=33, tm_sec=54, tm_wday=2, tm_yday=221, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=8, tm_mday=9, tm_hour=15, tm_min=12, tm_sec=20, tm_wday=2, tm_yday=221, tm_isdst=0) 2023-08-09T15:12:28.026 INFO:tasks.ceph:pgid 1.3 last_scrub_stamp 2023-08-09T13:33:54.845869+0000 time.struct_time(tm_year=2023, tm_mon=8, tm_mday=9, tm_hour=13, tm_min=33, tm_sec=54, tm_wday=2, tm_yday=221, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=8, tm_mday=9, tm_hour=15, tm_min=12, tm_sec=20, tm_wday=2, tm_yday=221, tm_isdst=0) 2023-08-09T15:12:28.026 INFO:tasks.ceph:pgid 2.0 last_scrub_stamp 2023-08-09T13:33:54.845869+0000 time.struct_time(tm_year=2023, tm_mon=8, tm_mday=9, tm_hour=13, tm_min=33, tm_sec=54, tm_wday=2, tm_yday=221, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=8, tm_mday=9, tm_hour=15, tm_min=12, tm_sec=20, tm_wday=2, tm_yday=221, tm_isdst=0) 2023-08-09T15:12:28.026 INFO:tasks.ceph:pgid 1.4 last_scrub_stamp 2023-08-09T13:33:54.845869+0000 time.struct_time(tm_year=2023, tm_mon=8, tm_mday=9, tm_hour=13, tm_min=33, tm_sec=54, tm_wday=2, tm_yday=221, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=8, tm_mday=9, tm_hour=15, tm_min=12, tm_sec=20, tm_wday=2, tm_yday=221, tm_isdst=0) 2023-08-09T15:12:28.027 INFO:tasks.ceph:pgid 1.5 last_scrub_stamp 2023-08-09T13:33:54.845869+0000 time.struct_time(tm_year=2023, tm_mon=8, tm_mday=9, tm_hour=13, tm_min=33, tm_sec=54, tm_wday=2, tm_yday=221, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=8, tm_mday=9, tm_hour=15, tm_min=12, tm_sec=20, tm_wday=2, tm_yday=221, tm_isdst=0) 2023-08-09T15:12:28.027 INFO:tasks.ceph:pgid 1.6 last_scrub_stamp 2023-08-09T13:33:54.845869+0000 time.struct_time(tm_year=2023, tm_mon=8, tm_mday=9, tm_hour=13, tm_min=33, tm_sec=54, tm_wday=2, tm_yday=221, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=8, tm_mday=9, tm_hour=15, tm_min=12, tm_sec=20, tm_wday=2, tm_yday=221, tm_isdst=0) 2023-08-09T15:12:28.027 INFO:tasks.ceph:pgid 1.7 last_scrub_stamp 2023-08-09T13:33:54.845869+0000 time.struct_time(tm_year=2023, tm_mon=8, tm_mday=9, tm_hour=13, tm_min=33, tm_sec=54, tm_wday=2, tm_yday=221, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=8, tm_mday=9, tm_hour=15, tm_min=12, tm_sec=20, tm_wday=2, tm_yday=221, tm_isdst=0) 2023-08-09T15:12:28.027 INFO:tasks.ceph:Still waiting for all pgs to be scrubbed. 2023-08-09T15:12:32.527 DEBUG:teuthology.orchestra.run.smithi043:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-08-09T15:12:32.536 DEBUG:teuthology.orchestra.run.smithi203:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-08-09T15:12:48.028 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-08-09T15:12:48.268 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:48.267+0000 7f2ee5813640 1 Processor -- start 2023-08-09T15:12:48.268 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:48.267+0000 7f2ee5813640 1 -- start start 2023-08-09T15:12:48.270 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:48.269+0000 7f2ee5813640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f2ee0156e30 0x7f2ee00c8840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T15:12:48.270 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:48.269+0000 7f2ee5813640 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7f2ee015d060 0x7f2ee015a4f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T15:12:48.270 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:48.269+0000 7f2ee5813640 1 -- --> v1:172.21.15.203:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2ee015aa30 con 0x7f2ee0157210 2023-08-09T15:12:48.271 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:48.269+0000 7f2ee5813640 1 -- --> v2:172.21.15.203:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2ee00cd6b0 con 0x7f2ee015d060 2023-08-09T15:12:48.271 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:48.269+0000 7f2ee5813640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2ee015d550 con 0x7f2ee0156e30 2023-08-09T15:12:48.272 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:48.270+0000 7f2ed67fc640 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7f2ee015d060 0x7f2ee015a4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T15:12:48.272 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:48.270+0000 7f2ed67fc640 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7f2ee015d060 0x7f2ee015a4f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.203:3300/0 says I am v2:172.21.15.43:59606/0 (socket says 172.21.15.43:59606) 2023-08-09T15:12:48.272 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:48.270+0000 7f2ed67fc640 1 -- 172.21.15.43:0/3437161781 learned_addr learned my addr 172.21.15.43:0/3437161781 (peer_addr_for_me v2:172.21.15.43:0/0) 2023-08-09T15:12:48.272 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:48.270+0000 7f2edeffd640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f2ee0156e30 0x7f2ee00c8840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T15:12:48.273 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:48.271+0000 7f2ed67fc640 1 -- 172.21.15.43:0/3437161781 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f2ee0156e30 msgr2=0x7f2ee00c8840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-09T15:12:48.273 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:48.271+0000 7f2ed67fc640 1 --2- 172.21.15.43:0/3437161781 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f2ee0156e30 0x7f2ee00c8840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T15:12:48.273 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:48.271+0000 7f2ed67fc640 1 -- 172.21.15.43:0/3437161781 >> v1:172.21.15.203:6789/0 conn(0x7f2ee0157210 legacy=0x7f2ee006b4b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T15:12:48.273 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:48.271+0000 7f2ed67fc640 1 -- 172.21.15.43:0/3437161781 --> v2:172.21.15.203:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2ee015d830 con 0x7f2ee015d060 2023-08-09T15:12:48.273 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:48.271+0000 7f2edeffd640 1 --2- 172.21.15.43:0/3437161781 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f2ee0156e30 0x7f2ee00c8840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-08-09T15:12:48.274 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:48.272+0000 7f2ed67fc640 1 --2- 172.21.15.43:0/3437161781 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f2ee015d060 0x7f2ee015a4f0 secure :-1 s=READY pgs=949 cs=0 l=1 rev1=1 crypto rx=0x7f2ec8009870 tx=0x7f2ec802b8f0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=82f3b4ac0fe88f96 server_cookie=0 in_seq=0 out_seq=0 2023-08-09T15:12:48.274 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:48.273+0000 7f2ede7fc640 1 -- 172.21.15.43:0/3437161781 <== mon.1 v2:172.21.15.203:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2ec8031640 con 0x7f2ee015d060 2023-08-09T15:12:48.275 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:48.273+0000 7f2ede7fc640 1 -- 172.21.15.43:0/3437161781 <== mon.1 v2:172.21.15.203:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2ec80317a0 con 0x7f2ee015d060 2023-08-09T15:12:48.275 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:48.273+0000 7f2ede7fc640 1 -- 172.21.15.43:0/3437161781 <== mon.1 v2:172.21.15.203:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2ec8031b30 con 0x7f2ee015d060 2023-08-09T15:12:48.275 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:48.273+0000 7f2ee5813640 1 -- 172.21.15.43:0/3437161781 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f2ee015d060 msgr2=0x7f2ee015a4f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T15:12:48.275 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:48.273+0000 7f2ee5813640 1 --2- 172.21.15.43:0/3437161781 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f2ee015d060 0x7f2ee015a4f0 secure :-1 s=READY pgs=949 cs=0 l=1 rev1=1 crypto rx=0x7f2ec8009870 tx=0x7f2ec802b8f0 comp rx=0 tx=0).stop 2023-08-09T15:12:48.275 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:48.274+0000 7f2ee5813640 1 -- 172.21.15.43:0/3437161781 shutdown_connections 2023-08-09T15:12:48.275 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:48.274+0000 7f2ee5813640 1 --2- 172.21.15.43:0/3437161781 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f2ee015d060 0x7f2ee015a4f0 unknown :-1 s=CLOSED pgs=949 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T15:12:48.275 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:48.274+0000 7f2ee5813640 1 --2- 172.21.15.43:0/3437161781 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f2ee0156e30 0x7f2ee00c8840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T15:12:48.276 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:48.274+0000 7f2ee5813640 1 -- 172.21.15.43:0/3437161781 >> 172.21.15.43:0/3437161781 conn(0x7f2ee0069d70 msgr2=0x7f2ee00cac70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-09T15:12:48.276 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:48.274+0000 7f2ee5813640 1 -- 172.21.15.43:0/3437161781 shutdown_connections 2023-08-09T15:12:48.277 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:48.274+0000 7f2ee5813640 1 -- 172.21.15.43:0/3437161781 wait complete. 2023-08-09T15:12:48.283 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:48.275+0000 7f2ee5813640 1 Processor -- start 2023-08-09T15:12:48.283 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:48.276+0000 7f2ee5813640 1 -- start start 2023-08-09T15:12:48.284 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:48.277+0000 7f2ee5813640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f2ee0156e30 0x7f2ee014e8b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T15:12:48.284 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:48.277+0000 7f2ee5813640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f2ee0157210 0x7f2ee0157b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T15:12:48.284 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:48.277+0000 7f2ee5813640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f2ee015d060 0x7f2ee00ade60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T15:12:48.284 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:48.277+0000 7f2ee5813640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2ee0158d70 con 0x7f2ee015d060 2023-08-09T15:12:48.284 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:48.277+0000 7f2ee5813640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2ee00abd40 con 0x7f2ee0157210 2023-08-09T15:12:48.285 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:48.277+0000 7f2ee5813640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2ee01584d0 con 0x7f2ee0156e30 2023-08-09T15:12:48.285 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:48.277+0000 7f2edf7fe640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f2ee015d060 0x7f2ee00ade60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T15:12:48.285 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:48.277+0000 7f2edf7fe640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f2ee015d060 0x7f2ee00ade60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:46258/0 (socket says 172.21.15.43:46258) 2023-08-09T15:12:48.285 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:48.277+0000 7f2edf7fe640 1 -- 172.21.15.43:0/2155557506 learned_addr learned my addr 172.21.15.43:0/2155557506 (peer_addr_for_me v2:172.21.15.43:0/0) 2023-08-09T15:12:48.285 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:48.277+0000 7f2edeffd640 1 --2- 172.21.15.43:0/2155557506 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f2ee0156e30 0x7f2ee014e8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T15:12:48.285 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:48.277+0000 7f2ed67fc640 1 --2- 172.21.15.43:0/2155557506 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f2ee0157210 0x7f2ee0157b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T15:12:48.285 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:48.278+0000 7f2edf7fe640 1 -- 172.21.15.43:0/2155557506 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f2ee0156e30 msgr2=0x7f2ee014e8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T15:12:48.286 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:48.278+0000 7f2edf7fe640 1 --2- 172.21.15.43:0/2155557506 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f2ee0156e30 0x7f2ee014e8b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T15:12:48.286 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:48.278+0000 7f2edf7fe640 1 -- 172.21.15.43:0/2155557506 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f2ee0157210 msgr2=0x7f2ee0157b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T15:12:48.286 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:48.278+0000 7f2edf7fe640 1 --2- 172.21.15.43:0/2155557506 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f2ee0157210 0x7f2ee0157b60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T15:12:48.286 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:48.278+0000 7f2edf7fe640 1 -- 172.21.15.43:0/2155557506 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2ed0009a70 con 0x7f2ee015d060 2023-08-09T15:12:48.286 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:48.278+0000 7f2ed67fc640 1 --2- 172.21.15.43:0/2155557506 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f2ee0157210 0x7f2ee0157b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-08-09T15:12:48.286 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:48.278+0000 7f2edf7fe640 1 --2- 172.21.15.43:0/2155557506 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f2ee015d060 0x7f2ee00ade60 secure :-1 s=READY pgs=2608 cs=0 l=1 rev1=1 crypto rx=0x7f2ed800c3a0 tx=0x7f2ed800c860 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-09T15:12:48.286 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:48.278+0000 7f2edcff9640 1 -- 172.21.15.43:0/2155557506 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2ed80052d0 con 0x7f2ee015d060 2023-08-09T15:12:48.286 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:48.278+0000 7f2edcff9640 1 -- 172.21.15.43:0/2155557506 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2ed8005430 con 0x7f2ee015d060 2023-08-09T15:12:48.287 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:48.278+0000 7f2edcff9640 1 -- 172.21.15.43:0/2155557506 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2ed8003830 con 0x7f2ee015d060 2023-08-09T15:12:48.287 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:48.278+0000 7f2ee5813640 1 -- 172.21.15.43:0/2155557506 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2ec80094d0 con 0x7f2ee015d060 2023-08-09T15:12:48.287 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:48.278+0000 7f2ee5813640 1 -- 172.21.15.43:0/2155557506 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2ee0158a90 con 0x7f2ee015d060 2023-08-09T15:12:48.287 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:48.280+0000 7f2ee5813640 1 -- 172.21.15.43:0/2155557506 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2ee00aba80 con 0x7f2ee015d060 2023-08-09T15:12:48.287 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:48.281+0000 7f2edcff9640 1 -- 172.21.15.43:0/2155557506 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 129) v1 ==== 88443+0+0 (secure 0 0 0) 0x7f2ed8003990 con 0x7f2ee015d060 2023-08-09T15:12:48.287 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:48.281+0000 7f2edcff9640 1 --2- 172.21.15.43:0/2155557506 >> [v2:172.21.15.203:6832/1456699667,v1:172.21.15.203:6833/1456699667] conn(0x7f2ec006a170 0x7f2ec006c630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T15:12:48.287 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:48.282+0000 7f2edcff9640 1 -- 172.21.15.43:0/2155557506 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(229..229 src has 1..229) v4 ==== 6462+0+0 (secure 0 0 0) 0x7f2ed8087e20 con 0x7f2ee015d060 2023-08-09T15:12:48.288 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:48.282+0000 7f2edeffd640 1 --2- 172.21.15.43:0/2155557506 >> [v2:172.21.15.203:6832/1456699667,v1:172.21.15.203:6833/1456699667] conn(0x7f2ec006a170 0x7f2ec006c630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T15:12:48.288 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:48.283+0000 7f2edeffd640 1 --2- 172.21.15.43:0/2155557506 >> [v2:172.21.15.203:6832/1456699667,v1:172.21.15.203:6833/1456699667] conn(0x7f2ec006a170 0x7f2ec006c630 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f2ee00aeb30 tx=0x7f2ed000b040 comp rx=0 tx=0).ready entity=mgr.9232 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-09T15:12:48.289 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:48.286+0000 7f2edcff9640 1 -- 172.21.15.43:0/2155557506 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178066 (secure 0 0 0) 0x7f2ed8053320 con 0x7f2ee015d060 2023-08-09T15:12:48.478 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:48.478+0000 7f2ee5813640 1 -- 172.21.15.43:0/2155557506 --> [v2:172.21.15.203:6832/1456699667,v1:172.21.15.203:6833/1456699667] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f2ee015ddf0 con 0x7f2ec006a170 2023-08-09T15:12:48.486 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:48.486+0000 7f2edcff9640 1 -- 172.21.15.43:0/2155557506 <== mgr.9232 v2:172.21.15.203:6832/1456699667 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+70453 (secure 0 0 0) 0x7f2ed000f050 con 0x7f2ec006a170 2023-08-09T15:12:48.487 INFO:teuthology.orchestra.run.smithi043.stdout: 2023-08-09T15:12:48.488 INFO:teuthology.orchestra.run.smithi043.stderr:dumped all 2023-08-09T15:12:48.489 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:48.488+0000 7f2ee5813640 1 -- 172.21.15.43:0/2155557506 >> [v2:172.21.15.203:6832/1456699667,v1:172.21.15.203:6833/1456699667] conn(0x7f2ec006a170 msgr2=0x7f2ec006c630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T15:12:48.489 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:48.488+0000 7f2ee5813640 1 --2- 172.21.15.43:0/2155557506 >> [v2:172.21.15.203:6832/1456699667,v1:172.21.15.203:6833/1456699667] conn(0x7f2ec006a170 0x7f2ec006c630 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f2ee00aeb30 tx=0x7f2ed000b040 comp rx=0 tx=0).stop 2023-08-09T15:12:48.489 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:48.489+0000 7f2ee5813640 1 -- 172.21.15.43:0/2155557506 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f2ee015d060 msgr2=0x7f2ee00ade60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T15:12:48.489 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:48.489+0000 7f2ee5813640 1 --2- 172.21.15.43:0/2155557506 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f2ee015d060 0x7f2ee00ade60 secure :-1 s=READY pgs=2608 cs=0 l=1 rev1=1 crypto rx=0x7f2ed800c3a0 tx=0x7f2ed800c860 comp rx=0 tx=0).stop 2023-08-09T15:12:48.490 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:48.489+0000 7f2ee5813640 1 -- 172.21.15.43:0/2155557506 shutdown_connections 2023-08-09T15:12:48.490 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:48.489+0000 7f2ee5813640 1 --2- 172.21.15.43:0/2155557506 >> [v2:172.21.15.203:6832/1456699667,v1:172.21.15.203:6833/1456699667] conn(0x7f2ec006a170 0x7f2ec006c630 unknown :-1 s=CLOSED pgs=60 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T15:12:48.490 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:48.489+0000 7f2ee5813640 1 --2- 172.21.15.43:0/2155557506 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f2ee015d060 0x7f2ee00ade60 unknown :-1 s=CLOSED pgs=2608 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T15:12:48.491 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:48.489+0000 7f2ee5813640 1 --2- 172.21.15.43:0/2155557506 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f2ee0157210 0x7f2ee0157b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T15:12:48.491 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:48.489+0000 7f2ee5813640 1 --2- 172.21.15.43:0/2155557506 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f2ee0156e30 0x7f2ee014e8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T15:12:48.491 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:48.489+0000 7f2ee5813640 1 -- 172.21.15.43:0/2155557506 >> 172.21.15.43:0/2155557506 conn(0x7f2ee0069d70 msgr2=0x7f2ee00ce1d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-09T15:12:48.491 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:48.489+0000 7f2ee5813640 1 -- 172.21.15.43:0/2155557506 shutdown_connections 2023-08-09T15:12:48.491 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:48.489+0000 7f2ee5813640 1 -- 172.21.15.43:0/2155557506 wait complete. 2023-08-09T15:12:48.510 INFO:teuthology.orchestra.run.smithi043.stdout:{"pg_ready":true,"pg_map":{"version":27,"stamp":"2023-08-09T15:12:47.372429+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2155,"num_read_kb":4508,"num_write":583,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":352,"ondisk_log_size":352,"up":18,"acting":18,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":18,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":393100,"kb_used_data":23616,"kb_used_omap":13,"kb_used_meta":369394,"kb_avail":754581620,"statfs":{"total":773094113280,"available":772691578880,"internally_reserved":0,"allocated":24182784,"data_stored":13716990,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":14248,"internal_metadata":378259544},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.003600"},"pg_stats":[{"pgid":"1.0","version":"0'0","reported_seq":435,"reported_epoch":229,"state":"active+clean","last_fresh":"2023-08-09T15:12:33.542888+0000","last_change":"2023-08-09T15:12:33.542996+0000","last_active":"2023-08-09T15:12:33.542888+0000","last_peered":"2023-08-09T15:12:33.542888+0000","last_clean":"2023-08-09T15:12:33.542888+0000","last_became_active":"2023-08-09T13:33:56.400487+0000","last_became_peered":"2023-08-09T13:33:56.400487+0000","last_unstale":"2023-08-09T15:12:33.542888+0000","last_undegraded":"2023-08-09T15:12:33.542888+0000","last_fullsized":"2023-08-09T15:12:33.542888+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T15:12:33.542727+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T15:12:33.542727+0000","last_clean_scrub_stamp":"2023-08-09T15:12:33.542727+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T23:09:36.567773+0000","scrub_duration":0.0036903830000000002,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":441,"reported_epoch":229,"state":"active+clean","last_fresh":"2023-08-09T15:12:26.429098+0000","last_change":"2023-08-09T15:12:26.429243+0000","last_active":"2023-08-09T15:12:26.429098+0000","last_peered":"2023-08-09T15:12:26.429098+0000","last_clean":"2023-08-09T15:12:26.429098+0000","last_became_active":"2023-08-09T13:33:56.399933+0000","last_became_peered":"2023-08-09T13:33:56.399933+0000","last_unstale":"2023-08-09T15:12:26.429098+0000","last_undegraded":"2023-08-09T15:12:26.429098+0000","last_fullsized":"2023-08-09T15:12:26.429098+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T15:12:26.428939+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T15:12:26.428939+0000","last_clean_scrub_stamp":"2023-08-09T15:12:26.428939+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-08-11T00:10:07.528903+0000","scrub_duration":0.0035983590000000002,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.2","version":"0'0","reported_seq":433,"reported_epoch":229,"state":"active+clean","last_fresh":"2023-08-09T15:12:29.538444+0000","last_change":"2023-08-09T15:12:29.538558+0000","last_active":"2023-08-09T15:12:29.538444+0000","last_peered":"2023-08-09T15:12:29.538444+0000","last_clean":"2023-08-09T15:12:29.538444+0000","last_became_active":"2023-08-09T13:33:56.402038+0000","last_became_peered":"2023-08-09T13:33:56.402038+0000","last_unstale":"2023-08-09T15:12:29.538444+0000","last_undegraded":"2023-08-09T15:12:29.538444+0000","last_fullsized":"2023-08-09T15:12:29.538444+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T15:12:29.538290+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T15:12:29.538290+0000","last_clean_scrub_stamp":"2023-08-09T15:12:29.538290+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-08-11T01:14:09.425259+0000","scrub_duration":0.0044403960000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.3","version":"0'0","reported_seq":447,"reported_epoch":229,"state":"active+clean","last_fresh":"2023-08-09T15:12:24.997980+0000","last_change":"2023-08-09T15:12:24.998051+0000","last_active":"2023-08-09T15:12:24.997980+0000","last_peered":"2023-08-09T15:12:24.997980+0000","last_clean":"2023-08-09T15:12:24.997980+0000","last_became_active":"2023-08-09T13:33:55.886116+0000","last_became_peered":"2023-08-09T13:33:55.886116+0000","last_unstale":"2023-08-09T15:12:24.997980+0000","last_undegraded":"2023-08-09T15:12:24.997980+0000","last_fullsized":"2023-08-09T15:12:24.997980+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T15:12:24.997880+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T15:12:24.997880+0000","last_clean_scrub_stamp":"2023-08-09T15:12:24.997880+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T16:22:49.222948+0000","scrub_duration":0.0034978409999999998,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"229'352","reported_seq":1415,"reported_epoch":229,"state":"active+clean","last_fresh":"2023-08-09T15:12:31.571488+0000","last_change":"2023-08-09T15:12:31.571610+0000","last_active":"2023-08-09T15:12:31.571488+0000","last_peered":"2023-08-09T15:12:31.571488+0000","last_clean":"2023-08-09T15:12:31.571488+0000","last_became_active":"2023-08-09T13:33:55.885781+0000","last_became_peered":"2023-08-09T13:33:55.885781+0000","last_unstale":"2023-08-09T15:12:31.571488+0000","last_undegraded":"2023-08-09T15:12:31.571488+0000","last_fullsized":"2023-08-09T15:12:31.571488+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"229'352","last_scrub_stamp":"2023-08-09T15:12:31.571327+0000","last_deep_scrub":"229'352","last_deep_scrub_stamp":"2023-08-09T15:12:31.571327+0000","last_clean_scrub_stamp":"2023-08-09T15:12:31.571327+0000","objects_scrubbed":2,"log_size":352,"log_dups_size":0,"ondisk_log_size":352,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T19:38:18.059935+0000","scrub_duration":0.01162762,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2155,"num_read_kb":4508,"num_write":583,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":451,"reported_epoch":229,"state":"active+clean","last_fresh":"2023-08-09T15:12:28.214474+0000","last_change":"2023-08-09T15:12:28.214583+0000","last_active":"2023-08-09T15:12:28.214474+0000","last_peered":"2023-08-09T15:12:28.214474+0000","last_clean":"2023-08-09T15:12:28.214474+0000","last_became_active":"2023-08-09T13:33:55.889164+0000","last_became_peered":"2023-08-09T13:33:55.889164+0000","last_unstale":"2023-08-09T15:12:28.214474+0000","last_undegraded":"2023-08-09T15:12:28.214474+0000","last_fullsized":"2023-08-09T15:12:28.214474+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T15:12:28.214278+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T15:12:28.214278+0000","last_clean_scrub_stamp":"2023-08-09T15:12:28.214278+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T22:58:44.410404+0000","scrub_duration":0.0044052839999999998,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":439,"reported_epoch":229,"state":"active+clean","last_fresh":"2023-08-09T15:12:26.619912+0000","last_change":"2023-08-09T15:12:26.620013+0000","last_active":"2023-08-09T15:12:26.619912+0000","last_peered":"2023-08-09T15:12:26.619912+0000","last_clean":"2023-08-09T15:12:26.619912+0000","last_became_active":"2023-08-09T13:33:56.402485+0000","last_became_peered":"2023-08-09T13:33:56.402485+0000","last_unstale":"2023-08-09T15:12:26.619912+0000","last_undegraded":"2023-08-09T15:12:26.619912+0000","last_fullsized":"2023-08-09T15:12:26.619912+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T15:12:26.619758+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T15:12:26.619758+0000","last_clean_scrub_stamp":"2023-08-09T15:12:26.619758+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T19:13:04.352377+0000","scrub_duration":0.0038018240000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":431,"reported_epoch":229,"state":"active+clean","last_fresh":"2023-08-09T15:12:32.100650+0000","last_change":"2023-08-09T15:12:32.100826+0000","last_active":"2023-08-09T15:12:32.100650+0000","last_peered":"2023-08-09T15:12:32.100650+0000","last_clean":"2023-08-09T15:12:32.100650+0000","last_became_active":"2023-08-09T13:33:56.400336+0000","last_became_peered":"2023-08-09T13:33:56.400336+0000","last_unstale":"2023-08-09T15:12:32.100650+0000","last_undegraded":"2023-08-09T15:12:32.100650+0000","last_fullsized":"2023-08-09T15:12:32.100650+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T15:12:32.100476+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T15:12:32.100476+0000","last_clean_scrub_stamp":"2023-08-09T15:12:32.100476+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T21:17:34.779613+0000","scrub_duration":0.0041242429999999997,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":431,"reported_epoch":229,"state":"active+clean","last_fresh":"2023-08-09T15:12:27.285512+0000","last_change":"2023-08-09T15:12:27.285616+0000","last_active":"2023-08-09T15:12:27.285512+0000","last_peered":"2023-08-09T15:12:27.285512+0000","last_clean":"2023-08-09T15:12:27.285512+0000","last_became_active":"2023-08-09T13:33:55.885638+0000","last_became_peered":"2023-08-09T13:33:55.885638+0000","last_unstale":"2023-08-09T15:12:27.285512+0000","last_undegraded":"2023-08-09T15:12:27.285512+0000","last_fullsized":"2023-08-09T15:12:27.285512+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-09T15:12:27.285354+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-09T15:12:27.285354+0000","last_clean_scrub_stamp":"2023-08-09T15:12:27.285354+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-08-10T21:53:16.568036+0000","scrub_duration":0.0036268580000000002,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2155,"num_read_kb":4508,"num_write":583,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":352,"ondisk_log_size":352,"up":2,"acting":2,"num_store_stats":2},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":8}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834576026,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":52548,"kb_used_data":3388,"kb_used_omap":1,"kb_used_meta":49150,"kb_avail":94319292,"statfs":{"total":96636764160,"available":96582955008,"internally_reserved":0,"allocated":3469312,"data_stored":2157740,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1781,"internal_metadata":50329867},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Aug 9 15:11:52 2023","interfaces":[{"interface":"back","average":{"1min":1.113,"5min":1.234,"15min":1.2290000000000001},"min":{"1min":0.52400000000000002,"5min":0.52400000000000002,"15min":0.52400000000000002},"max":{"1min":1.6080000000000001,"5min":2.056,"15min":2.056},"last":1.119},{"interface":"front","average":{"1min":1.0900000000000001,"5min":1.2110000000000001,"15min":1.2529999999999999},"min":{"1min":0.55800000000000005,"5min":0.55800000000000005,"15min":0.55800000000000005},"max":{"1min":1.5960000000000001,"5min":1.7769999999999999,"15min":1.863},"last":1.2290000000000001}]},{"osd":1,"last update":"Wed Aug 9 15:11:52 2023","interfaces":[{"interface":"back","average":{"1min":1.1479999999999999,"5min":1.28,"15min":1.3049999999999999},"min":{"1min":0.52500000000000002,"5min":0.52500000000000002,"15min":0.52500000000000002},"max":{"1min":1.6619999999999999,"5min":1.853,"15min":1.853},"last":1.492},{"interface":"front","average":{"1min":1.175,"5min":1.296,"15min":1.3100000000000001},"min":{"1min":0.748,"5min":0.748,"15min":0.748},"max":{"1min":1.605,"5min":2.1520000000000001,"15min":2.1520000000000001},"last":1.3129999999999999}]},{"osd":2,"last update":"Wed Aug 9 15:11:52 2023","interfaces":[{"interface":"back","average":{"1min":1.093,"5min":1.262,"15min":1.2969999999999999},"min":{"1min":0.70399999999999996,"5min":0.70399999999999996,"15min":0.70399999999999996},"max":{"1min":1.7050000000000001,"5min":1.861,"15min":1.861},"last":1.165},{"interface":"front","average":{"1min":1.214,"5min":1.3480000000000001,"15min":1.393},"min":{"1min":0.58099999999999996,"5min":0.58099999999999996,"15min":0.58099999999999996},"max":{"1min":1.7889999999999999,"5min":1.9370000000000001,"15min":1.9370000000000001},"last":1.3520000000000001}]},{"osd":3,"last update":"Wed Aug 9 15:11:52 2023","interfaces":[{"interface":"back","average":{"1min":1.1950000000000001,"5min":1.3160000000000001,"15min":1.351},"min":{"1min":0.80700000000000005,"5min":0.80700000000000005,"15min":0.80700000000000005},"max":{"1min":1.798,"5min":1.798,"15min":1.9259999999999999},"last":1.6259999999999999},{"interface":"front","average":{"1min":1.335,"5min":1.4379999999999999,"15min":1.472},"min":{"1min":0.76200000000000001,"5min":0.76200000000000001,"15min":0.76200000000000001},"max":{"1min":1.9179999999999999,"5min":1.9239999999999999,"15min":1.9239999999999999},"last":1.2789999999999999}]},{"osd":4,"last update":"Wed Aug 9 15:11:52 2023","interfaces":[{"interface":"back","average":{"1min":1.218,"5min":1.282,"15min":1.2989999999999999},"min":{"1min":0.56399999999999995,"5min":0.56399999999999995,"15min":0.56399999999999995},"max":{"1min":1.6080000000000001,"5min":2.2810000000000001,"15min":2.2810000000000001},"last":1.0649999999999999},{"interface":"front","average":{"1min":1.28,"5min":1.3560000000000001,"15min":1.385},"min":{"1min":0.622,"5min":0.622,"15min":0.622},"max":{"1min":2.0390000000000001,"5min":2.0390000000000001,"15min":2.0830000000000002},"last":1.3959999999999999}]},{"osd":5,"last update":"Wed Aug 9 15:11:52 2023","interfaces":[{"interface":"back","average":{"1min":1.3620000000000001,"5min":1.4219999999999999,"15min":1.4410000000000001},"min":{"1min":0.59299999999999997,"5min":0.59299999999999997,"15min":0.59299999999999997},"max":{"1min":2.0899999999999999,"5min":2.0899999999999999,"15min":2.0899999999999999},"last":1},{"interface":"front","average":{"1min":1.3959999999999999,"5min":1.448,"15min":1.4730000000000001},"min":{"1min":0.65500000000000003,"5min":0.65500000000000003,"15min":0.65500000000000003},"max":{"1min":2.206,"5min":2.5310000000000001,"15min":2.5310000000000001},"last":1.4550000000000001}]},{"osd":6,"last update":"Wed Aug 9 15:11:52 2023","interfaces":[{"interface":"back","average":{"1min":1.5069999999999999,"5min":1.587,"15min":1.589},"min":{"1min":0.69199999999999995,"5min":0.69199999999999995,"15min":0.69199999999999995},"max":{"1min":2.294,"5min":2.8959999999999999,"15min":2.8959999999999999},"last":1.5589999999999999},{"interface":"front","average":{"1min":1.359,"5min":1.5089999999999999,"15min":1.534},"min":{"1min":0.60899999999999999,"5min":0.60899999999999999,"15min":0.60899999999999999},"max":{"1min":2.161,"5min":2.214,"15min":2.222},"last":1.518}]}]},{"osd":6,"up_from":13,"seq":55834576026,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":60156,"kb_used_data":2804,"kb_used_omap":1,"kb_used_meta":57342,"kb_avail":94311684,"statfs":{"total":96636764160,"available":96575164416,"internally_reserved":0,"allocated":2871296,"data_stored":1566011,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1781,"internal_metadata":58718475},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Aug 9 15:11:57 2023","interfaces":[{"interface":"back","average":{"1min":1.1639999999999999,"5min":1.2130000000000001,"15min":1.2190000000000001},"min":{"1min":0.76700000000000002,"5min":0.75600000000000001,"15min":0.75600000000000001},"max":{"1min":1.8169999999999999,"5min":1.8169999999999999,"15min":1.8859999999999999},"last":1.2509999999999999},{"interface":"front","average":{"1min":1.0720000000000001,"5min":1.21,"15min":1.234},"min":{"1min":0.64500000000000002,"5min":0.64500000000000002,"15min":0.64500000000000002},"max":{"1min":1.5109999999999999,"5min":2.0800000000000001,"15min":2.0800000000000001},"last":1.119}]},{"osd":1,"last update":"Wed Aug 9 15:11:57 2023","interfaces":[{"interface":"back","average":{"1min":1.095,"5min":1.1879999999999999,"15min":1.1910000000000001},"min":{"1min":0.65900000000000003,"5min":0.65900000000000003,"15min":0.65900000000000003},"max":{"1min":1.776,"5min":2.6760000000000002,"15min":2.6760000000000002},"last":1.073},{"interface":"front","average":{"1min":1.2869999999999999,"5min":1.3120000000000001,"15min":1.3460000000000001},"min":{"1min":0.72599999999999998,"5min":0.72599999999999998,"15min":0.72599999999999998},"max":{"1min":1.579,"5min":1.998,"15min":1.998},"last":1.4710000000000001}]},{"osd":2,"last update":"Wed Aug 9 15:11:57 2023","interfaces":[{"interface":"back","average":{"1min":1.2629999999999999,"5min":1.3120000000000001,"15min":1.3400000000000001},"min":{"1min":0.79800000000000004,"5min":0.79800000000000004,"15min":0.79800000000000004},"max":{"1min":1.6659999999999999,"5min":2.0089999999999999,"15min":2.081},"last":1.383},{"interface":"front","average":{"1min":1.3080000000000001,"5min":1.337,"15min":1.3480000000000001},"min":{"1min":0.73799999999999999,"5min":0.73799999999999999,"15min":0.73799999999999999},"max":{"1min":1.75,"5min":1.9219999999999999,"15min":1.9219999999999999},"last":1.6240000000000001}]},{"osd":3,"last update":"Wed Aug 9 15:11:57 2023","interfaces":[{"interface":"back","average":{"1min":1.302,"5min":1.4099999999999999,"15min":1.4019999999999999},"min":{"1min":0.70299999999999996,"5min":0.70299999999999996,"15min":0.70299999999999996},"max":{"1min":2.2669999999999999,"5min":2.2669999999999999,"15min":2.2669999999999999},"last":1.4330000000000001},{"interface":"front","average":{"1min":1.2270000000000001,"5min":1.319,"15min":1.3420000000000001},"min":{"1min":0.59699999999999998,"5min":0.59699999999999998,"15min":0.59699999999999998},"max":{"1min":1.855,"5min":2.0539999999999998,"15min":2.0539999999999998},"last":1.522}]},{"osd":4,"last update":"Wed Aug 9 15:11:57 2023","interfaces":[{"interface":"back","average":{"1min":1.339,"5min":1.3500000000000001,"15min":1.339},"min":{"1min":0.46500000000000002,"5min":0.46500000000000002,"15min":0.46500000000000002},"max":{"1min":1.7270000000000001,"5min":2.0470000000000002,"15min":2.0470000000000002},"last":1.0229999999999999},{"interface":"front","average":{"1min":1.288,"5min":1.4470000000000001,"15min":1.4330000000000001},"min":{"1min":0.53000000000000003,"5min":0.53000000000000003,"15min":0.53000000000000003},"max":{"1min":1.6559999999999999,"5min":2.1589999999999998,"15min":2.1589999999999998},"last":1.2949999999999999}]},{"osd":5,"last update":"Wed Aug 9 15:11:57 2023","interfaces":[{"interface":"back","average":{"1min":1.3109999999999999,"5min":1.431,"15min":1.381},"min":{"1min":0.51100000000000001,"5min":0.51100000000000001,"15min":0.51100000000000001},"max":{"1min":1.9610000000000001,"5min":2.1269999999999998,"15min":2.6059999999999999},"last":1.355},{"interface":"front","average":{"1min":1.546,"5min":1.637,"15min":1.6299999999999999},"min":{"1min":0.67300000000000004,"5min":0.67300000000000004,"15min":0.67300000000000004},"max":{"1min":1.885,"5min":2.125,"15min":2.1899999999999999},"last":1.575}]},{"osd":7,"last update":"Wed Aug 9 15:11:57 2023","interfaces":[{"interface":"back","average":{"1min":1.506,"5min":1.639,"15min":1.6180000000000001},"min":{"1min":0.626,"5min":0.626,"15min":0.626},"max":{"1min":1.9319999999999999,"5min":2.2280000000000002,"15min":2.4340000000000002},"last":1.74},{"interface":"front","average":{"1min":1.45,"5min":1.6180000000000001,"15min":1.6000000000000001},"min":{"1min":0.44600000000000001,"5min":0.44600000000000001,"15min":0.44600000000000001},"max":{"1min":2.0409999999999999,"5min":2.3300000000000001,"15min":2.7400000000000002},"last":1.1779999999999999}]}]},{"osd":5,"up_from":13,"seq":55834576026,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":47872,"kb_used_data":2808,"kb_used_omap":1,"kb_used_meta":45054,"kb_avail":94323968,"statfs":{"total":96636764160,"available":96587743232,"internally_reserved":0,"allocated":2875392,"data_stored":1567372,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1781,"internal_metadata":46135563},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Aug 9 15:11:47 2023","interfaces":[{"interface":"back","average":{"1min":1.0669999999999999,"5min":1.1919999999999999,"15min":1.194},"min":{"1min":0.57699999999999996,"5min":0.57699999999999996,"15min":0.57699999999999996},"max":{"1min":1.494,"5min":1.7370000000000001,"15min":1.754},"last":1.099},{"interface":"front","average":{"1min":1.0389999999999999,"5min":1.167,"15min":1.208},"min":{"1min":0.55600000000000005,"5min":0.55600000000000005,"15min":0.55600000000000005},"max":{"1min":1.3480000000000001,"5min":1.6870000000000001,"15min":1.976},"last":1.2729999999999999}]},{"osd":1,"last update":"Wed Aug 9 15:11:47 2023","interfaces":[{"interface":"back","average":{"1min":1.2490000000000001,"5min":1.28,"15min":1.2509999999999999},"min":{"1min":0.67500000000000004,"5min":0.65200000000000002,"15min":0.65200000000000002},"max":{"1min":1.9490000000000001,"5min":2.016,"15min":2.016},"last":1.4750000000000001},{"interface":"front","average":{"1min":1.2150000000000001,"5min":1.288,"15min":1.343},"min":{"1min":0.65000000000000002,"5min":0.65000000000000002,"15min":0.65000000000000002},"max":{"1min":2.0169999999999999,"5min":2.0169999999999999,"15min":2.0169999999999999},"last":1.504}]},{"osd":2,"last update":"Wed Aug 9 15:11:47 2023","interfaces":[{"interface":"back","average":{"1min":1.0269999999999999,"5min":1.2629999999999999,"15min":1.3009999999999999},"min":{"1min":0.60199999999999998,"5min":0.60199999999999998,"15min":0.60199999999999998},"max":{"1min":1.48,"5min":1.8859999999999999,"15min":2.032},"last":1.546},{"interface":"front","average":{"1min":1.3,"5min":1.339,"15min":1.3640000000000001},"min":{"1min":0.61899999999999999,"5min":0.61899999999999999,"15min":0.61899999999999999},"max":{"1min":1.7010000000000001,"5min":1.956,"15min":1.964},"last":1.5820000000000001}]},{"osd":3,"last update":"Wed Aug 9 15:11:47 2023","interfaces":[{"interface":"back","average":{"1min":1.3,"5min":1.3740000000000001,"15min":1.3939999999999999},"min":{"1min":0.746,"5min":0.746,"15min":0.746},"max":{"1min":1.6639999999999999,"5min":1.798,"15min":1.899},"last":1.22},{"interface":"front","average":{"1min":1.278,"5min":1.389,"15min":1.4370000000000001},"min":{"1min":0.66400000000000003,"5min":0.66400000000000003,"15min":0.66400000000000003},"max":{"1min":1.6559999999999999,"5min":1.8819999999999999,"15min":2.0379999999999998},"last":1.4299999999999999}]},{"osd":4,"last update":"Wed Aug 9 15:11:47 2023","interfaces":[{"interface":"back","average":{"1min":1.347,"5min":1.323,"15min":1.3220000000000001},"min":{"1min":0.53000000000000003,"5min":0.53000000000000003,"15min":0.53000000000000003},"max":{"1min":1.833,"5min":2.0299999999999998,"15min":2.0299999999999998},"last":1.165},{"interface":"front","average":{"1min":1.4419999999999999,"5min":1.389,"15min":1.4059999999999999},"min":{"1min":0.69199999999999995,"5min":0.69199999999999995,"15min":0.69199999999999995},"max":{"1min":1.7709999999999999,"5min":1.9239999999999999,"15min":2.0390000000000001},"last":1.6100000000000001}]},{"osd":6,"last update":"Wed Aug 9 15:11:47 2023","interfaces":[{"interface":"back","average":{"1min":1.298,"5min":1.3660000000000001,"15min":1.411},"min":{"1min":0.501,"5min":0.501,"15min":0.501},"max":{"1min":1.728,"5min":2.032,"15min":2.1000000000000001},"last":1.0640000000000001},{"interface":"front","average":{"1min":1.48,"5min":1.4850000000000001,"15min":1.5029999999999999},"min":{"1min":0.70599999999999996,"5min":0.70599999999999996,"15min":0.70599999999999996},"max":{"1min":2.125,"5min":2.1349999999999998,"15min":2.1349999999999998},"last":1.399}]},{"osd":7,"last update":"Wed Aug 9 15:11:47 2023","interfaces":[{"interface":"back","average":{"1min":1.5469999999999999,"5min":1.554,"15min":1.5800000000000001},"min":{"1min":0.71999999999999997,"5min":0.71999999999999997,"15min":0.71999999999999997},"max":{"1min":2.0760000000000001,"5min":2.0760000000000001,"15min":2.4689999999999999},"last":1.361},{"interface":"front","average":{"1min":1.5309999999999999,"5min":1.595,"15min":1.637},"min":{"1min":0.63800000000000001,"5min":0.63800000000000001,"15min":0.63800000000000001},"max":{"1min":2.1709999999999998,"5min":2.1709999999999998,"15min":2.4239999999999999},"last":1.6419999999999999}]}]},{"osd":4,"up_from":13,"seq":55834576026,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":51968,"kb_used_data":2808,"kb_used_omap":1,"kb_used_meta":49150,"kb_avail":94319872,"statfs":{"total":96636764160,"available":96583548928,"internally_reserved":0,"allocated":2875392,"data_stored":1567372,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1781,"internal_metadata":50329867},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Aug 9 15:12:13 2023","interfaces":[{"interface":"back","average":{"1min":1.2370000000000001,"5min":1.196,"15min":1.222},"min":{"1min":0.64700000000000002,"5min":0.60399999999999998,"15min":0.60399999999999998},"max":{"1min":1.8169999999999999,"5min":1.8169999999999999,"15min":2.2959999999999998},"last":1.444},{"interface":"front","average":{"1min":1.2270000000000001,"5min":1.2829999999999999,"15min":1.3129999999999999},"min":{"1min":0.77000000000000002,"5min":0.66800000000000004,"15min":0.66800000000000004},"max":{"1min":1.772,"5min":2.0779999999999998,"15min":2.2709999999999999},"last":1.0740000000000001}]},{"osd":1,"last update":"Wed Aug 9 15:12:13 2023","interfaces":[{"interface":"back","average":{"1min":1.1439999999999999,"5min":1.288,"15min":1.3460000000000001},"min":{"1min":0.73199999999999998,"5min":0.73199999999999998,"15min":0.73199999999999998},"max":{"1min":1.9370000000000001,"5min":1.9370000000000001,"15min":1.9370000000000001},"last":1.2669999999999999},{"interface":"front","average":{"1min":1.2430000000000001,"5min":1.2609999999999999,"15min":1.278},"min":{"1min":0.76400000000000001,"5min":0.73899999999999999,"15min":0.73899999999999999},"max":{"1min":1.865,"5min":1.865,"15min":2.117},"last":1.2250000000000001}]},{"osd":2,"last update":"Wed Aug 9 15:12:13 2023","interfaces":[{"interface":"back","average":{"1min":1.1859999999999999,"5min":1.2350000000000001,"15min":1.2829999999999999},"min":{"1min":0.70199999999999996,"5min":0.63100000000000001,"15min":0.63100000000000001},"max":{"1min":1.5469999999999999,"5min":1.7709999999999999,"15min":1.883},"last":1.3220000000000001},{"interface":"front","average":{"1min":1.298,"5min":1.3819999999999999,"15min":1.401},"min":{"1min":0.91400000000000003,"5min":0.82699999999999996,"15min":0.79200000000000004},"max":{"1min":1.903,"5min":2.343,"15min":2.343},"last":1.514}]},{"osd":3,"last update":"Wed Aug 9 15:12:13 2023","interfaces":[{"interface":"back","average":{"1min":1.2789999999999999,"5min":1.3420000000000001,"15min":1.349},"min":{"1min":0.95299999999999996,"5min":0.90500000000000003,"15min":0.88700000000000001},"max":{"1min":1.6970000000000001,"5min":1.8009999999999999,"15min":2.4870000000000001},"last":1.359},{"interface":"front","average":{"1min":1.3720000000000001,"5min":1.423,"15min":1.464},"min":{"1min":0.95299999999999996,"5min":0.79700000000000004,"15min":0.79700000000000004},"max":{"1min":1.8240000000000001,"5min":1.909,"15min":2.5459999999999998},"last":1.5489999999999999}]},{"osd":5,"last update":"Wed Aug 9 15:12:13 2023","interfaces":[{"interface":"back","average":{"1min":1.431,"5min":1.385,"15min":1.409},"min":{"1min":1.0029999999999999,"5min":0.82199999999999995,"15min":0.82199999999999995},"max":{"1min":2.004,"5min":2.633,"15min":2.633},"last":1.403},{"interface":"front","average":{"1min":1.395,"5min":1.4059999999999999,"15min":1.393},"min":{"1min":1.048,"5min":0.68100000000000005,"15min":0.68100000000000005},"max":{"1min":1.8300000000000001,"5min":1.9930000000000001,"15min":2.4249999999999998},"last":1.0089999999999999}]},{"osd":6,"last update":"Wed Aug 9 15:12:13 2023","interfaces":[{"interface":"back","average":{"1min":1.3919999999999999,"5min":1.4330000000000001,"15min":1.4099999999999999},"min":{"1min":1.0509999999999999,"5min":0.71299999999999997,"15min":0.71299999999999997},"max":{"1min":1.73,"5min":2.0049999999999999,"15min":2.9119999999999999},"last":1.173},{"interface":"front","average":{"1min":1.528,"5min":1.5680000000000001,"15min":1.579},"min":{"1min":1.1699999999999999,"5min":0.77000000000000002,"15min":0.77000000000000002},"max":{"1min":1.8680000000000001,"5min":2.9529999999999998,"15min":2.9529999999999998},"last":1.5800000000000001}]},{"osd":7,"last update":"Wed Aug 9 15:12:13 2023","interfaces":[{"interface":"back","average":{"1min":1.5029999999999999,"5min":1.5309999999999999,"15min":1.5229999999999999},"min":{"1min":1.1719999999999999,"5min":0.83899999999999997,"15min":0.83899999999999997},"max":{"1min":1.9370000000000001,"5min":2.1589999999999998,"15min":2.5910000000000002},"last":1.474},{"interface":"front","average":{"1min":1.599,"5min":1.6140000000000001,"15min":1.6060000000000001},"min":{"1min":1.206,"5min":0.90000000000000002,"15min":0.90000000000000002},"max":{"1min":1.8999999999999999,"5min":2.218,"15min":3.028},"last":1.615}]}]},{"osd":3,"up_from":13,"seq":55834576027,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":47872,"kb_used_data":2808,"kb_used_omap":1,"kb_used_meta":45054,"kb_avail":94323968,"statfs":{"total":96636764160,"available":96587743232,"internally_reserved":0,"allocated":2875392,"data_stored":1567372,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1781,"internal_metadata":46135563},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Aug 9 15:12:06 2023","interfaces":[{"interface":"back","average":{"1min":0.75800000000000001,"5min":0.94699999999999995,"15min":0.96899999999999997},"min":{"1min":0.42799999999999999,"5min":0.42799999999999999,"15min":0.42799999999999999},"max":{"1min":1.24,"5min":1.6699999999999999,"15min":2.2090000000000001},"last":0.84599999999999997},{"interface":"front","average":{"1min":0.872,"5min":1.093,"15min":1.1439999999999999},"min":{"1min":0.434,"5min":0.434,"15min":0.434},"max":{"1min":1.4550000000000001,"5min":1.9630000000000001,"15min":1.9630000000000001},"last":1.1899999999999999}]},{"osd":1,"last update":"Wed Aug 9 15:12:06 2023","interfaces":[{"interface":"back","average":{"1min":0.90500000000000003,"5min":1.006,"15min":1.0660000000000001},"min":{"1min":0.437,"5min":0.437,"15min":0.437},"max":{"1min":1.595,"5min":1.956,"15min":2.27},"last":0.999},{"interface":"front","average":{"1min":1.0960000000000001,"5min":1.2070000000000001,"15min":1.2569999999999999},"min":{"1min":0.63900000000000001,"5min":0.498,"15min":0.498},"max":{"1min":1.6140000000000001,"5min":1.78,"15min":1.9730000000000001},"last":1.421}]},{"osd":2,"last update":"Wed Aug 9 15:12:06 2023","interfaces":[{"interface":"back","average":{"1min":1.024,"5min":1.236,"15min":1.3180000000000001},"min":{"1min":0.65400000000000003,"5min":0.65400000000000003,"15min":0.65400000000000003},"max":{"1min":1.3839999999999999,"5min":1.9650000000000001,"15min":2.4079999999999999},"last":1.353},{"interface":"front","average":{"1min":1.2250000000000001,"5min":1.3720000000000001,"15min":1.4279999999999999},"min":{"1min":0.63400000000000001,"5min":0.63400000000000001,"15min":0.63400000000000001},"max":{"1min":1.7090000000000001,"5min":2.0510000000000002,"15min":2.1259999999999999},"last":1.2669999999999999}]},{"osd":4,"last update":"Wed Aug 9 15:12:06 2023","interfaces":[{"interface":"back","average":{"1min":1.2070000000000001,"5min":1.3260000000000001,"15min":1.3959999999999999},"min":{"1min":0.752,"5min":0.748,"15min":0.72599999999999998},"max":{"1min":1.744,"5min":1.9179999999999999,"15min":2.2069999999999999},"last":1.3169999999999999},{"interface":"front","average":{"1min":1.3220000000000001,"5min":1.4199999999999999,"15min":1.4570000000000001},"min":{"1min":0.77500000000000002,"5min":0.77500000000000002,"15min":0.77500000000000002},"max":{"1min":1.8939999999999999,"5min":1.9279999999999999,"15min":2.056},"last":1.73}]},{"osd":5,"last update":"Wed Aug 9 15:12:06 2023","interfaces":[{"interface":"back","average":{"1min":1.4039999999999999,"5min":1.502,"15min":1.538},"min":{"1min":0.77600000000000002,"5min":0.77600000000000002,"15min":0.77600000000000002},"max":{"1min":2.1890000000000001,"5min":2.1890000000000001,"15min":2.1890000000000001},"last":1.496},{"interface":"front","average":{"1min":1.3740000000000001,"5min":1.5580000000000001,"15min":1.6000000000000001},"min":{"1min":0.88800000000000001,"5min":0.85799999999999998,"15min":0.85799999999999998},"max":{"1min":1.7330000000000001,"5min":2.2469999999999999,"15min":2.4049999999999998},"last":1.544}]},{"osd":6,"last update":"Wed Aug 9 15:12:06 2023","interfaces":[{"interface":"back","average":{"1min":1.3560000000000001,"5min":1.4890000000000001,"15min":1.5780000000000001},"min":{"1min":0.69599999999999995,"5min":0.69599999999999995,"15min":0.69599999999999995},"max":{"1min":1.9670000000000001,"5min":2.4860000000000002,"15min":2.4860000000000002},"last":1.4650000000000001},{"interface":"front","average":{"1min":1.458,"5min":1.5940000000000001,"15min":1.653},"min":{"1min":0.89700000000000002,"5min":0.89700000000000002,"15min":0.89700000000000002},"max":{"1min":2.04,"5min":2.4039999999999999,"15min":2.4039999999999999},"last":1.6040000000000001}]},{"osd":7,"last update":"Wed Aug 9 15:12:06 2023","interfaces":[{"interface":"back","average":{"1min":1.5620000000000001,"5min":1.696,"15min":1.7490000000000001},"min":{"1min":0.94799999999999995,"5min":0.94799999999999995,"15min":0.94799999999999995},"max":{"1min":1.8680000000000001,"5min":2.347,"15min":2.347},"last":1.6559999999999999},{"interface":"front","average":{"1min":1.4179999999999999,"5min":1.6220000000000001,"15min":1.694},"min":{"1min":0.81100000000000005,"5min":0.81100000000000005,"15min":0.81100000000000005},"max":{"1min":2.1309999999999998,"5min":3.3220000000000001,"15min":3.3220000000000001},"last":1.5289999999999999}]}]},{"osd":2,"up_from":13,"seq":55834576026,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":46220,"kb_used_data":2808,"kb_used_omap":1,"kb_used_meta":43390,"kb_avail":94325620,"statfs":{"total":96636764160,"available":96589434880,"internally_reserved":0,"allocated":2875392,"data_stored":1567372,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1781,"internal_metadata":44431627},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Aug 9 15:11:52 2023","interfaces":[{"interface":"back","average":{"1min":0.874,"5min":0.98499999999999999,"15min":0.95899999999999996},"min":{"1min":0.48199999999999998,"5min":0.44800000000000001,"15min":0.44800000000000001},"max":{"1min":1.591,"5min":1.591,"15min":1.591},"last":0.76900000000000002},{"interface":"front","average":{"1min":0.96399999999999997,"5min":1.0760000000000001,"15min":1.085},"min":{"1min":0.38900000000000001,"5min":0.38900000000000001,"15min":0.38900000000000001},"max":{"1min":1.5980000000000001,"5min":1.764,"15min":2.052},"last":1.2130000000000001}]},{"osd":1,"last update":"Wed Aug 9 15:11:52 2023","interfaces":[{"interface":"back","average":{"1min":0.98299999999999998,"5min":1.2330000000000001,"15min":1.2230000000000001},"min":{"1min":0.55400000000000005,"5min":0.55400000000000005,"15min":0.55400000000000005},"max":{"1min":1.403,"5min":2.0179999999999998,"15min":2.0179999999999998},"last":0.89200000000000002},{"interface":"front","average":{"1min":1.1359999999999999,"5min":1.292,"15min":1.381},"min":{"1min":0.51200000000000001,"5min":0.51200000000000001,"15min":0.51200000000000001},"max":{"1min":1.6639999999999999,"5min":1.95,"15min":1.95},"last":1.599}]},{"osd":3,"last update":"Wed Aug 9 15:11:52 2023","interfaces":[{"interface":"back","average":{"1min":0.99099999999999999,"5min":1.089,"15min":1.1080000000000001},"min":{"1min":0.53500000000000003,"5min":0.52100000000000002,"15min":0.52100000000000002},"max":{"1min":1.5649999999999999,"5min":1.871,"15min":1.871},"last":1.0580000000000001},{"interface":"front","average":{"1min":1.111,"5min":1.339,"15min":1.369},"min":{"1min":0.57099999999999995,"5min":0.57099999999999995,"15min":0.57099999999999995},"max":{"1min":1.6930000000000001,"5min":1.9299999999999999,"15min":2.0470000000000002},"last":1.381}]},{"osd":4,"last update":"Wed Aug 9 15:11:52 2023","interfaces":[{"interface":"back","average":{"1min":1.369,"5min":1.5229999999999999,"15min":1.5469999999999999},"min":{"1min":0.86399999999999999,"5min":0.86399999999999999,"15min":0.86399999999999999},"max":{"1min":1.7350000000000001,"5min":1.994,"15min":2.464},"last":1.5409999999999999},{"interface":"front","average":{"1min":1.395,"5min":1.4399999999999999,"15min":1.464},"min":{"1min":0.93600000000000005,"5min":0.81799999999999995,"15min":0.81799999999999995},"max":{"1min":1.9370000000000001,"5min":1.98,"15min":2.0720000000000001},"last":1.718}]},{"osd":5,"last update":"Wed Aug 9 15:11:52 2023","interfaces":[{"interface":"back","average":{"1min":1.4279999999999999,"5min":1.5009999999999999,"15min":1.5169999999999999},"min":{"1min":0.64800000000000002,"5min":0.64800000000000002,"15min":0.64800000000000002},"max":{"1min":1.913,"5min":2.0369999999999999,"15min":2.1139999999999999},"last":1.7929999999999999},{"interface":"front","average":{"1min":1.351,"5min":1.419,"15min":1.4870000000000001},"min":{"1min":0.83399999999999996,"5min":0.83399999999999996,"15min":0.83399999999999996},"max":{"1min":2.1520000000000001,"5min":2.1520000000000001,"15min":2.214},"last":1.1879999999999999}]},{"osd":6,"last update":"Wed Aug 9 15:11:52 2023","interfaces":[{"interface":"back","average":{"1min":1.407,"5min":1.4890000000000001,"15min":1.5229999999999999},"min":{"1min":0.89500000000000002,"5min":0.89500000000000002,"15min":0.89500000000000002},"max":{"1min":2.0680000000000001,"5min":2.1110000000000002,"15min":2.2999999999999998},"last":1.49},{"interface":"front","average":{"1min":1.512,"5min":1.6479999999999999,"15min":1.6379999999999999},"min":{"1min":0.998,"5min":0.998,"15min":0.998},"max":{"1min":2.4140000000000001,"5min":2.4140000000000001,"15min":2.4140000000000001},"last":1.657}]},{"osd":7,"last update":"Wed Aug 9 15:11:52 2023","interfaces":[{"interface":"back","average":{"1min":1.5249999999999999,"5min":1.605,"15min":1.6419999999999999},"min":{"1min":0.96299999999999997,"5min":0.87,"15min":0.87},"max":{"1min":2.1000000000000001,"5min":2.1749999999999998,"15min":2.302},"last":1.855},{"interface":"front","average":{"1min":1.4650000000000001,"5min":1.5800000000000001,"15min":1.6299999999999999},"min":{"1min":0.71399999999999997,"5min":0.71399999999999997,"15min":0.71399999999999997},"max":{"1min":2.2360000000000002,"5min":2.242,"15min":2.3809999999999998},"last":1.583}]}]},{"osd":1,"up_from":13,"seq":55834576026,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":56644,"kb_used_data":3388,"kb_used_omap":1,"kb_used_meta":53246,"kb_avail":94315196,"statfs":{"total":96636764160,"available":96578760704,"internally_reserved":0,"allocated":3469312,"data_stored":2157740,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1781,"internal_metadata":54524171},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Aug 9 15:12:03 2023","interfaces":[{"interface":"back","average":{"1min":0.83399999999999996,"5min":0.97399999999999998,"15min":1},"min":{"1min":0.32100000000000001,"5min":0.32100000000000001,"15min":0.32100000000000001},"max":{"1min":1.153,"5min":1.8799999999999999,"15min":2.0939999999999999},"last":0.872},{"interface":"front","average":{"1min":0.94299999999999995,"5min":1.1060000000000001,"15min":1.1399999999999999},"min":{"1min":0.40500000000000003,"5min":0.40500000000000003,"15min":0.40500000000000003},"max":{"1min":1.641,"5min":1.78,"15min":1.8939999999999999},"last":1.077}]},{"osd":2,"last update":"Wed Aug 9 15:12:03 2023","interfaces":[{"interface":"back","average":{"1min":0.93700000000000006,"5min":1.018,"15min":1.04},"min":{"1min":0.626,"5min":0.55800000000000005,"15min":0.55800000000000005},"max":{"1min":2.2240000000000002,"5min":2.2240000000000002,"15min":2.3940000000000001},"last":1.2110000000000001},{"interface":"front","average":{"1min":0.97399999999999998,"5min":1.1759999999999999,"15min":1.254},"min":{"1min":0.49299999999999999,"5min":0.46700000000000003,"15min":0.46700000000000003},"max":{"1min":1.5149999999999999,"5min":2.169,"15min":2.169},"last":1.012}]},{"osd":3,"last update":"Wed Aug 9 15:12:03 2023","interfaces":[{"interface":"back","average":{"1min":1.115,"5min":1.3340000000000001,"15min":1.381},"min":{"1min":0.54600000000000004,"5min":0.51000000000000001,"15min":0.51000000000000001},"max":{"1min":1.833,"5min":2.093,"15min":2.109},"last":1.153},{"interface":"front","average":{"1min":1.0409999999999999,"5min":1.256,"15min":1.3089999999999999},"min":{"1min":0.59099999999999997,"5min":0.59099999999999997,"15min":0.59099999999999997},"max":{"1min":1.8680000000000001,"5min":2.052,"15min":2.052},"last":1.2809999999999999}]},{"osd":4,"last update":"Wed Aug 9 15:12:03 2023","interfaces":[{"interface":"back","average":{"1min":1.254,"5min":1.4339999999999999,"15min":1.4730000000000001},"min":{"1min":0.82099999999999995,"5min":0.753,"15min":0.753},"max":{"1min":1.887,"5min":2.0419999999999998,"15min":2.214},"last":1.581},{"interface":"front","average":{"1min":1.202,"5min":1.381,"15min":1.4119999999999999},"min":{"1min":0.84499999999999997,"5min":0.69199999999999995,"15min":0.69199999999999995},"max":{"1min":1.528,"5min":2.0310000000000001,"15min":2.1190000000000002},"last":1.343}]},{"osd":5,"last update":"Wed Aug 9 15:12:03 2023","interfaces":[{"interface":"back","average":{"1min":1.2450000000000001,"5min":1.415,"15min":1.446},"min":{"1min":0.83399999999999996,"5min":0.77800000000000002,"15min":0.77800000000000002},"max":{"1min":1.909,"5min":1.994,"15min":2.508},"last":1.5429999999999999},{"interface":"front","average":{"1min":1.2829999999999999,"5min":1.53,"15min":1.5800000000000001},"min":{"1min":0.74299999999999999,"5min":0.74299999999999999,"15min":0.74299999999999999},"max":{"1min":1.99,"5min":2.3809999999999998,"15min":2.3809999999999998},"last":1.496}]},{"osd":6,"last update":"Wed Aug 9 15:12:03 2023","interfaces":[{"interface":"back","average":{"1min":1.321,"5min":1.575,"15min":1.613},"min":{"1min":0.80100000000000005,"5min":0.80100000000000005,"15min":0.80100000000000005},"max":{"1min":2.1669999999999998,"5min":2.347,"15min":2.347},"last":1.4019999999999999},{"interface":"front","average":{"1min":1.288,"5min":1.522,"15min":1.5720000000000001},"min":{"1min":0.84999999999999998,"5min":0.73499999999999999,"15min":0.73499999999999999},"max":{"1min":1.8140000000000001,"5min":2.218,"15min":2.278},"last":1.4339999999999999}]},{"osd":7,"last update":"Wed Aug 9 15:12:03 2023","interfaces":[{"interface":"back","average":{"1min":1.3959999999999999,"5min":1.6439999999999999,"15min":1.6910000000000001},"min":{"1min":0.90900000000000003,"5min":0.86399999999999999,"15min":0.86399999999999999},"max":{"1min":2.1299999999999999,"5min":2.3090000000000002,"15min":2.488},"last":1.615},{"interface":"front","average":{"1min":1.385,"5min":1.633,"15min":1.698},"min":{"1min":0.75600000000000001,"5min":0.75600000000000001,"15min":0.75600000000000001},"max":{"1min":2.4980000000000002,"5min":2.4980000000000002,"15min":2.4980000000000002},"last":1.468}]}]},{"osd":0,"up_from":13,"seq":55834576026,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":29820,"kb_used_data":2804,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94342020,"statfs":{"total":96636764160,"available":96606228480,"internally_reserved":0,"allocated":2871296,"data_stored":1566011,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1781,"internal_metadata":27654411},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Wed Aug 9 15:12:00 2023","interfaces":[{"interface":"back","average":{"1min":0.85099999999999998,"5min":0.89600000000000002,"15min":0.95099999999999996},"min":{"1min":0.44900000000000001,"5min":0.40000000000000002,"15min":0.40000000000000002},"max":{"1min":1.3380000000000001,"5min":1.617,"15min":1.6299999999999999},"last":1.246},{"interface":"front","average":{"1min":0.98799999999999999,"5min":1.091,"15min":1.135},"min":{"1min":0.56599999999999995,"5min":0.56599999999999995,"15min":0.56599999999999995},"max":{"1min":1.635,"5min":2.1560000000000001,"15min":2.1560000000000001},"last":1.5149999999999999}]},{"osd":2,"last update":"Wed Aug 9 15:12:00 2023","interfaces":[{"interface":"back","average":{"1min":0.873,"5min":1.097,"15min":1.1719999999999999},"min":{"1min":0.54000000000000004,"5min":0.505,"15min":0.505},"max":{"1min":1.401,"5min":2.4769999999999999,"15min":2.4769999999999999},"last":0.92000000000000004},{"interface":"front","average":{"1min":1.0069999999999999,"5min":1.125,"15min":1.2150000000000001},"min":{"1min":0.55500000000000005,"5min":0.53100000000000003,"15min":0.53100000000000003},"max":{"1min":1.468,"5min":1.8180000000000001,"15min":1.9470000000000001},"last":1.1879999999999999}]},{"osd":3,"last update":"Wed Aug 9 15:12:00 2023","interfaces":[{"interface":"back","average":{"1min":0.91200000000000003,"5min":1.1040000000000001,"15min":1.1870000000000001},"min":{"1min":0.45500000000000002,"5min":0.45500000000000002,"15min":0.45500000000000002},"max":{"1min":1.629,"5min":2.1459999999999999,"15min":2.1459999999999999},"last":1.0289999999999999},{"interface":"front","average":{"1min":1.079,"5min":1.284,"15min":1.3779999999999999},"min":{"1min":0.53100000000000003,"5min":0.53100000000000003,"15min":0.53100000000000003},"max":{"1min":1.5640000000000001,"5min":1.867,"15min":2.0760000000000001},"last":1.4119999999999999}]},{"osd":4,"last update":"Wed Aug 9 15:12:00 2023","interfaces":[{"interface":"back","average":{"1min":1.163,"5min":1.321,"15min":1.403},"min":{"1min":0.81799999999999995,"5min":0.81799999999999995,"15min":0.81799999999999995},"max":{"1min":1.5629999999999999,"5min":1.853,"15min":1.966},"last":1.3240000000000001},{"interface":"front","average":{"1min":1.296,"5min":1.421,"15min":1.5069999999999999},"min":{"1min":0.83099999999999996,"5min":0.79400000000000004,"15min":0.79400000000000004},"max":{"1min":1.734,"5min":1.972,"15min":2.278},"last":1.736}]},{"osd":5,"last update":"Wed Aug 9 15:12:00 2023","interfaces":[{"interface":"back","average":{"1min":1.256,"5min":1.476,"15min":1.5129999999999999},"min":{"1min":0.84699999999999998,"5min":0.84699999999999998,"15min":0.84699999999999998},"max":{"1min":1.79,"5min":2.355,"15min":2.355},"last":1.6839999999999999},{"interface":"front","average":{"1min":1.2849999999999999,"5min":1.4299999999999999,"15min":1.49},"min":{"1min":0.84799999999999998,"5min":0.82899999999999996,"15min":0.82899999999999996},"max":{"1min":2.089,"5min":2.089,"15min":2.3740000000000001},"last":1.4710000000000001}]},{"osd":6,"last update":"Wed Aug 9 15:12:00 2023","interfaces":[{"interface":"back","average":{"1min":1.3129999999999999,"5min":1.5249999999999999,"15min":1.6160000000000001},"min":{"1min":0.79700000000000004,"5min":0.79700000000000004,"15min":0.79700000000000004},"max":{"1min":1.7889999999999999,"5min":2.7490000000000001,"15min":2.7490000000000001},"last":1.5780000000000001},{"interface":"front","average":{"1min":1.3839999999999999,"5min":1.534,"15min":1.597},"min":{"1min":0.89200000000000002,"5min":0.84699999999999998,"15min":0.84699999999999998},"max":{"1min":2.0840000000000001,"5min":2.274,"15min":2.2839999999999998},"last":1.827}]},{"osd":7,"last update":"Wed Aug 9 15:12:00 2023","interfaces":[{"interface":"back","average":{"1min":1.4710000000000001,"5min":1.599,"15min":1.665},"min":{"1min":0.98599999999999999,"5min":0.94899999999999995,"15min":0.94899999999999995},"max":{"1min":2.2989999999999999,"5min":2.645,"15min":2.645},"last":1.9079999999999999},{"interface":"front","average":{"1min":1.4019999999999999,"5min":1.54,"15min":1.609},"min":{"1min":0.82599999999999996,"5min":0.82599999999999996,"15min":0.82599999999999996},"max":{"1min":2.1829999999999998,"5min":2.1829999999999998,"15min":2.3759999999999999},"last":1.556}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-08-09T15:12:48.513 DEBUG:teuthology.orchestra.run.smithi043:> sudo ceph --cluster ceph config set global mon_health_to_clog false 2023-08-09T15:12:48.726 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:48.725+0000 7fa18a421640 1 Processor -- start 2023-08-09T15:12:48.726 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:48.726+0000 7fa18a421640 1 -- start start 2023-08-09T15:12:48.728 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:48.727+0000 7fa18a421640 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7fa1840c14b0 0x7fa1840cce20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T15:12:48.728 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:48.727+0000 7fa18a421640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7fa1840c86a0 0x7fa1840c8b10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T15:12:48.728 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:48.727+0000 7fa18a421640 1 -- --> v1:172.21.15.203:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa184160ec0 con 0x7fa1840cab40 2023-08-09T15:12:48.728 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:48.727+0000 7fa18a421640 1 -- --> v2:172.21.15.43:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa1840cca00 con 0x7fa1840c86a0 2023-08-09T15:12:48.729 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:48.727+0000 7fa18a421640 1 -- --> v2:172.21.15.203:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa1840c9050 con 0x7fa1840c14b0 2023-08-09T15:12:48.729 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:48.728+0000 7fa189c20640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7fa1840c86a0 0x7fa1840c8b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T15:12:48.729 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:48.728+0000 7fa189c20640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7fa1840c86a0 0x7fa1840c8b10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:46262/0 (socket says 172.21.15.43:46262) 2023-08-09T15:12:48.729 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:48.728+0000 7fa189c20640 1 -- 172.21.15.43:0/3546365887 learned_addr learned my addr 172.21.15.43:0/3546365887 (peer_addr_for_me v2:172.21.15.43:0/0) 2023-08-09T15:12:48.731 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:48.728+0000 7fa18941f640 1 --2- 172.21.15.43:0/3546365887 >> v2:172.21.15.203:3300/0 conn(0x7fa1840c14b0 0x7fa1840cce20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T15:12:48.731 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:48.728+0000 7fa189c20640 1 -- 172.21.15.43:0/3546365887 >> v2:172.21.15.203:3300/0 conn(0x7fa1840c14b0 msgr2=0x7fa1840cce20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-09T15:12:48.731 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:48.728+0000 7fa189c20640 1 --2- 172.21.15.43:0/3546365887 >> v2:172.21.15.203:3300/0 conn(0x7fa1840c14b0 0x7fa1840cce20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T15:12:48.732 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:48.728+0000 7fa189c20640 1 -- 172.21.15.43:0/3546365887 >> v1:172.21.15.203:6789/0 conn(0x7fa1840cab40 legacy=0x7fa18406c100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-09T15:12:48.732 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:48.728+0000 7fa189c20640 1 -- 172.21.15.43:0/3546365887 --> v2:172.21.15.43:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa1840cbda0 con 0x7fa1840c86a0 2023-08-09T15:12:48.732 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:48.729+0000 7fa189c20640 1 --2- 172.21.15.43:0/3546365887 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa1840c86a0 0x7fa1840c8b10 secure :-1 s=READY pgs=2609 cs=0 l=1 rev1=1 crypto rx=0x7fa1840cdf30 tx=0x7fa17c02be50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=20b1dd6cf7b92bb2 server_cookie=0 in_seq=0 out_seq=0 2023-08-09T15:12:48.733 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:48.730+0000 7fa17bfff640 1 -- 172.21.15.43:0/3546365887 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa17c02c9f0 con 0x7fa1840c86a0 2023-08-09T15:12:48.733 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:48.730+0000 7fa17bfff640 1 -- 172.21.15.43:0/3546365887 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa17c02cb50 con 0x7fa1840c86a0 2023-08-09T15:12:48.733 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:48.730+0000 7fa17bfff640 1 -- 172.21.15.43:0/3546365887 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa17c0343f0 con 0x7fa1840c86a0 2023-08-09T15:12:48.734 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:48.730+0000 7fa18a421640 1 -- 172.21.15.43:0/3546365887 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa1840c86a0 msgr2=0x7fa1840c8b10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T15:12:48.734 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:48.730+0000 7fa18a421640 1 --2- 172.21.15.43:0/3546365887 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa1840c86a0 0x7fa1840c8b10 secure :-1 s=READY pgs=2609 cs=0 l=1 rev1=1 crypto rx=0x7fa1840cdf30 tx=0x7fa17c02be50 comp rx=0 tx=0).stop 2023-08-09T15:12:48.738 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:48.731+0000 7fa18a421640 1 -- 172.21.15.43:0/3546365887 shutdown_connections 2023-08-09T15:12:48.738 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:48.731+0000 7fa18a421640 1 --2- 172.21.15.43:0/3546365887 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa1840c86a0 0x7fa1840c8b10 unknown :-1 s=CLOSED pgs=2609 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T15:12:48.738 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:48.731+0000 7fa18a421640 1 --2- 172.21.15.43:0/3546365887 >> v2:172.21.15.203:3300/0 conn(0x7fa1840c14b0 0x7fa1840cce20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T15:12:48.739 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:48.731+0000 7fa18a421640 1 -- 172.21.15.43:0/3546365887 >> 172.21.15.43:0/3546365887 conn(0x7fa1840afb60 msgr2=0x7fa1840aff40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-09T15:12:48.739 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:48.731+0000 7fa18a421640 1 -- 172.21.15.43:0/3546365887 shutdown_connections 2023-08-09T15:12:48.739 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:48.731+0000 7fa18a421640 1 -- 172.21.15.43:0/3546365887 wait complete. 2023-08-09T15:12:48.739 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:48.732+0000 7fa18a421640 1 Processor -- start 2023-08-09T15:12:48.740 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:48.733+0000 7fa18a421640 1 -- start start 2023-08-09T15:12:48.740 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:48.733+0000 7fa18a421640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fa1840c14b0 0x7fa184070330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T15:12:48.740 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:48.734+0000 7fa18a421640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa1840c86a0 0x7fa1840c7770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T15:12:48.740 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:48.734+0000 7fa18a421640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fa1840cab40 0x7fa1841f5330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T15:12:48.741 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:48.734+0000 7fa18a421640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa1841fa370 con 0x7fa1840c86a0 2023-08-09T15:12:48.741 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:48.734+0000 7fa18a421640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa1841f96f0 con 0x7fa1840cab40 2023-08-09T15:12:48.741 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:48.734+0000 7fa18a421640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa1841f99d0 con 0x7fa1840c14b0 2023-08-09T15:12:48.742 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:48.734+0000 7fa188c1e640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa1840c86a0 0x7fa1840c7770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T15:12:48.742 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:48.734+0000 7fa188c1e640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa1840c86a0 0x7fa1840c7770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:46272/0 (socket says 172.21.15.43:46272) 2023-08-09T15:12:48.742 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:48.734+0000 7fa188c1e640 1 -- 172.21.15.43:0/3009255598 learned_addr learned my addr 172.21.15.43:0/3009255598 (peer_addr_for_me v2:172.21.15.43:0/0) 2023-08-09T15:12:48.742 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:48.734+0000 7fa18941f640 1 --2- 172.21.15.43:0/3009255598 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fa1840c14b0 0x7fa184070330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T15:12:48.742 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:48.734+0000 7fa189c20640 1 --2- 172.21.15.43:0/3009255598 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fa1840cab40 0x7fa1841f5330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T15:12:48.743 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:48.734+0000 7fa188c1e640 1 -- 172.21.15.43:0/3009255598 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fa1840c14b0 msgr2=0x7fa184070330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T15:12:48.743 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:48.734+0000 7fa188c1e640 1 --2- 172.21.15.43:0/3009255598 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fa1840c14b0 0x7fa184070330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T15:12:48.743 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:48.734+0000 7fa188c1e640 1 -- 172.21.15.43:0/3009255598 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fa1840cab40 msgr2=0x7fa1841f5330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T15:12:48.743 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:48.734+0000 7fa188c1e640 1 --2- 172.21.15.43:0/3009255598 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fa1840cab40 0x7fa1841f5330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T15:12:48.743 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:48.735+0000 7fa188c1e640 1 -- 172.21.15.43:0/3009255598 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa17c009b30 con 0x7fa1840c86a0 2023-08-09T15:12:48.743 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:48.735+0000 7fa188c1e640 1 --2- 172.21.15.43:0/3009255598 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa1840c86a0 0x7fa1840c7770 secure :-1 s=READY pgs=2610 cs=0 l=1 rev1=1 crypto rx=0x7fa17400b3c0 tx=0x7fa17400b880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-09T15:12:48.744 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:48.735+0000 7fa179ffb640 1 -- 172.21.15.43:0/3009255598 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa174011830 con 0x7fa1840c86a0 2023-08-09T15:12:48.744 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:48.735+0000 7fa179ffb640 1 -- 172.21.15.43:0/3009255598 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa174011990 con 0x7fa1840c86a0 2023-08-09T15:12:48.744 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:48.735+0000 7fa18a421640 1 -- 172.21.15.43:0/3009255598 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa1841f9cb0 con 0x7fa1840c86a0 2023-08-09T15:12:48.744 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:48.735+0000 7fa179ffb640 1 -- 172.21.15.43:0/3009255598 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa174011d60 con 0x7fa1840c86a0 2023-08-09T15:12:48.744 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:48.735+0000 7fa18a421640 1 -- 172.21.15.43:0/3009255598 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa184154430 con 0x7fa1840c86a0 2023-08-09T15:12:48.744 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:48.736+0000 7fa18a421640 1 -- 172.21.15.43:0/3009255598 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa18404e390 con 0x7fa1840c86a0 2023-08-09T15:12:48.744 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:48.737+0000 7fa179ffb640 1 -- 172.21.15.43:0/3009255598 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 129) v1 ==== 88443+0+0 (secure 0 0 0) 0x7fa17401b460 con 0x7fa1840c86a0 2023-08-09T15:12:48.745 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:48.738+0000 7fa179ffb640 1 --2- 172.21.15.43:0/3009255598 >> [v2:172.21.15.203:6832/1456699667,v1:172.21.15.203:6833/1456699667] conn(0x7fa16806a170 0x7fa16806c630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T15:12:48.746 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:48.738+0000 7fa179ffb640 1 -- 172.21.15.43:0/3009255598 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(229..229 src has 1..229) v4 ==== 6462+0+0 (secure 0 0 0) 0x7fa1740886c0 con 0x7fa1840c86a0 2023-08-09T15:12:48.746 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:48.739+0000 7fa18941f640 1 --2- 172.21.15.43:0/3009255598 >> [v2:172.21.15.203:6832/1456699667,v1:172.21.15.203:6833/1456699667] conn(0x7fa16806a170 0x7fa16806c630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T15:12:48.746 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:48.740+0000 7fa18941f640 1 --2- 172.21.15.43:0/3009255598 >> [v2:172.21.15.203:6832/1456699667,v1:172.21.15.203:6833/1456699667] conn(0x7fa16806a170 0x7fa16806c630 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7fa180006fd0 tx=0x7fa180008040 comp rx=0 tx=0).ready entity=mgr.9232 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-09T15:12:48.746 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:48.742+0000 7fa179ffb640 1 -- 172.21.15.43:0/3009255598 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178066 (secure 0 0 0) 0x7fa174053a70 con 0x7fa1840c86a0 2023-08-09T15:12:48.941 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:48.940+0000 7fa18a421640 1 -- 172.21.15.43:0/3009255598 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command([{prefix=config set, name=mon_health_to_clog}] v 0) v1 -- 0x7fa18406c8b0 con 0x7fa1840c86a0 2023-08-09T15:12:48.948 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:48.948+0000 7fa179ffb640 1 -- 172.21.15.43:0/3009255598 <== mon.0 v2:172.21.15.43:3300/0 7 ==== config(1 keys) v1 ==== 35+0+0 (secure 0 0 0) 0x7fa174053a70 con 0x7fa1840c86a0 2023-08-09T15:12:48.948 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:48.948+0000 7fa179ffb640 1 -- 172.21.15.43:0/3009255598 <== mon.0 v2:172.21.15.43:3300/0 8 ==== mon_command_ack([{prefix=config set, name=mon_health_to_clog}]=0 v76)=0 v76) v1 ==== 123+0+0 (secure 0 0 0) 0x7fa174057090 con 0x7fa1840c86a0 2023-08-09T15:12:48.952 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:48.951+0000 7fa18a421640 1 -- 172.21.15.43:0/3009255598 >> [v2:172.21.15.203:6832/1456699667,v1:172.21.15.203:6833/1456699667] conn(0x7fa16806a170 msgr2=0x7fa16806c630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T15:12:48.952 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:48.951+0000 7fa18a421640 1 --2- 172.21.15.43:0/3009255598 >> [v2:172.21.15.203:6832/1456699667,v1:172.21.15.203:6833/1456699667] conn(0x7fa16806a170 0x7fa16806c630 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7fa180006fd0 tx=0x7fa180008040 comp rx=0 tx=0).stop 2023-08-09T15:12:48.953 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:48.951+0000 7fa18a421640 1 -- 172.21.15.43:0/3009255598 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa1840c86a0 msgr2=0x7fa1840c7770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T15:12:48.953 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:48.951+0000 7fa18a421640 1 --2- 172.21.15.43:0/3009255598 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa1840c86a0 0x7fa1840c7770 secure :-1 s=READY pgs=2610 cs=0 l=1 rev1=1 crypto rx=0x7fa17400b3c0 tx=0x7fa17400b880 comp rx=0 tx=0).stop 2023-08-09T15:12:48.953 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:48.951+0000 7fa18a421640 1 -- 172.21.15.43:0/3009255598 shutdown_connections 2023-08-09T15:12:48.953 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:48.951+0000 7fa18a421640 1 --2- 172.21.15.43:0/3009255598 >> [v2:172.21.15.203:6832/1456699667,v1:172.21.15.203:6833/1456699667] conn(0x7fa16806a170 0x7fa16806c630 unknown :-1 s=CLOSED pgs=61 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T15:12:48.953 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:48.951+0000 7fa18a421640 1 --2- 172.21.15.43:0/3009255598 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fa1840cab40 0x7fa1841f5330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T15:12:48.953 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:48.951+0000 7fa18a421640 1 --2- 172.21.15.43:0/3009255598 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa1840c86a0 0x7fa1840c7770 unknown :-1 s=CLOSED pgs=2610 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T15:12:48.953 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:48.951+0000 7fa18a421640 1 --2- 172.21.15.43:0/3009255598 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fa1840c14b0 0x7fa184070330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T15:12:48.954 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:48.951+0000 7fa18a421640 1 -- 172.21.15.43:0/3009255598 >> 172.21.15.43:0/3009255598 conn(0x7fa1840afb60 msgr2=0x7fa1841639f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-09T15:12:48.954 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:48.952+0000 7fa18a421640 1 -- 172.21.15.43:0/3009255598 shutdown_connections 2023-08-09T15:12:48.954 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:48.952+0000 7fa18a421640 1 -- 172.21.15.43:0/3009255598 wait complete. 2023-08-09T15:12:48.970 INFO:tasks.cephfs.filesystem:Destroying file system cephfs and related pools 2023-08-09T15:12:48.971 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-08-09T15:12:49.195 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:49.194+0000 7f4ef8b71640 1 Processor -- start 2023-08-09T15:12:49.195 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:49.194+0000 7f4ef8b71640 1 -- start start 2023-08-09T15:12:49.196 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:49.196+0000 7f4ef8b71640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f4ef406b4b0 0x7f4ef40cdbe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T15:12:49.196 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:49.196+0000 7f4ef8b71640 1 -- --> v1:172.21.15.43:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4ef406c2c0 con 0x7f4ef406fe60 2023-08-09T15:12:49.196 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:49.196+0000 7f4ef8b71640 1 -- --> v1:172.21.15.203:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4ef40cb690 con 0x7f4ef415b310 2023-08-09T15:12:49.197 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:49.196+0000 7f4ef8b71640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4ef406ba20 con 0x7f4ef406b4b0 2023-08-09T15:12:49.197 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:49.196+0000 7f4ef259c640 1 --1- >> v1:172.21.15.43:6789/0 conn(0x7f4ef406fe60 0x7f4ef415be10 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.43:6789/0 says I am v1:172.21.15.43:34200/0 (socket says 172.21.15.43:34200) 2023-08-09T15:12:49.197 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:49.196+0000 7f4ef259c640 1 -- 172.21.15.43:0/2700177342 learned_addr learned my addr 172.21.15.43:0/2700177342 (peer_addr_for_me v1:172.21.15.43:0/0) 2023-08-09T15:12:49.198 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:49.197+0000 7f4eebfff640 1 --2- 172.21.15.43:0/2700177342 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f4ef406b4b0 0x7f4ef40cdbe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T15:12:49.198 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:49.197+0000 7f4ef1d9b640 1 -- 172.21.15.43:0/2700177342 <== mon.0 v1:172.21.15.43:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 382736181 0 0) 0x7f4ef406c2c0 con 0x7f4ef406fe60 2023-08-09T15:12:49.198 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:49.197+0000 7f4ef1d9b640 1 -- 172.21.15.43:0/2700177342 --> v1:172.21.15.43:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f4ee0003520 con 0x7f4ef406fe60 2023-08-09T15:12:49.199 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:49.197+0000 7f4ef1d9b640 1 -- 172.21.15.43:0/2700177342 <== mon.0 v1:172.21.15.43:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 845465033 0 0) 0x7f4ee0003520 con 0x7f4ef406fe60 2023-08-09T15:12:49.199 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:49.197+0000 7f4ef1d9b640 1 -- 172.21.15.43:0/2700177342 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f4ef406b4b0 msgr2=0x7f4ef40cdbe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-09T15:12:49.200 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:49.197+0000 7f4ef1d9b640 1 --2- 172.21.15.43:0/2700177342 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f4ef406b4b0 0x7f4ef40cdbe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T15:12:49.200 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:49.197+0000 7f4ef1d9b640 1 -- 172.21.15.43:0/2700177342 >> v1:172.21.15.203:6789/0 conn(0x7f4ef415b310 legacy=0x7f4ef40afc70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T15:12:49.200 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:49.197+0000 7f4ef1d9b640 1 -- 172.21.15.43:0/2700177342 --> v1:172.21.15.43:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4ef406bd00 con 0x7f4ef406fe60 2023-08-09T15:12:49.200 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:49.197+0000 7f4ef1d9b640 1 -- 172.21.15.43:0/2700177342 <== mon.0 v1:172.21.15.43:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1285150740 0 0) 0x7f4ee4001670 con 0x7f4ef406fe60 2023-08-09T15:12:49.200 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:49.198+0000 7f4ef1d9b640 1 -- 172.21.15.43:0/2700177342 >> v1:172.21.15.43:6789/0 conn(0x7f4ef406fe60 legacy=0x7f4ef415be10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T15:12:49.200 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:49.198+0000 7f4ef1d9b640 1 --2- 172.21.15.43:0/2700177342 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f4ee00039a0 0x7f4ee0003d80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T15:12:49.201 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:49.198+0000 7f4ef1d9b640 1 --2- 172.21.15.43:0/2700177342 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f4ee0004350 0x7f4ee0004820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T15:12:49.201 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:49.198+0000 7f4ef1d9b640 1 --2- 172.21.15.43:0/2700177342 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f4ef406b4b0 0x7f4ee0005080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T15:12:49.201 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:49.198+0000 7f4ef1d9b640 1 -- 172.21.15.43:0/2700177342 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4ef406ba20 con 0x7f4ee0004350 2023-08-09T15:12:49.201 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:49.198+0000 7f4ef1d9b640 1 -- 172.21.15.43:0/2700177342 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4ee000b9b0 con 0x7f4ee00039a0 2023-08-09T15:12:49.201 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:49.198+0000 7f4ef1d9b640 1 -- 172.21.15.43:0/2700177342 --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4ee000bb70 con 0x7f4ef406b4b0 2023-08-09T15:12:49.201 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:49.198+0000 7f4eebfff640 1 --2- 172.21.15.43:0/2700177342 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f4ee0004350 0x7f4ee0004820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T15:12:49.201 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:49.198+0000 7f4ef259c640 1 --2- 172.21.15.43:0/2700177342 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f4ee00039a0 0x7f4ee0003d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T15:12:49.201 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:49.198+0000 7f4eebfff640 1 -- 172.21.15.43:0/2700177342 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f4ef406b4b0 msgr2=0x7f4ee0005080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-09T15:12:49.202 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:49.198+0000 7f4eebfff640 1 --2- 172.21.15.43:0/2700177342 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f4ef406b4b0 0x7f4ee0005080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T15:12:49.202 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:49.198+0000 7f4eebfff640 1 -- 172.21.15.43:0/2700177342 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f4ee00039a0 msgr2=0x7f4ee0003d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-09T15:12:49.202 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:49.198+0000 7f4eebfff640 1 --2- 172.21.15.43:0/2700177342 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f4ee00039a0 0x7f4ee0003d80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T15:12:49.202 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:49.198+0000 7f4eebfff640 1 -- 172.21.15.43:0/2700177342 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4ee0003520 con 0x7f4ee0004350 2023-08-09T15:12:49.202 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:49.198+0000 7f4ef259c640 1 --2- 172.21.15.43:0/2700177342 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f4ee00039a0 0x7f4ee0003d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-08-09T15:12:49.204 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:49.200+0000 7f4eebfff640 1 --2- 172.21.15.43:0/2700177342 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f4ee0004350 0x7f4ee0004820 secure :-1 s=READY pgs=2612 cs=0 l=1 rev1=1 crypto rx=0x7f4edc009ac0 tx=0x7f4ee00057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d7c3bb4087698ff4 server_cookie=0 in_seq=0 out_seq=0 2023-08-09T15:12:49.205 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:49.200+0000 7f4ef1d9b640 1 -- 172.21.15.43:0/2700177342 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4edc02c710 con 0x7f4ee0004350 2023-08-09T15:12:49.205 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:49.200+0000 7f4ef1d9b640 1 -- 172.21.15.43:0/2700177342 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(1 keys) v1 ==== 35+0+0 (secure 0 0 0) 0x7f4edc02c870 con 0x7f4ee0004350 2023-08-09T15:12:49.205 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:49.200+0000 7f4ef1d9b640 1 -- 172.21.15.43:0/2700177342 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4edc02cc20 con 0x7f4ee0004350 2023-08-09T15:12:49.205 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:49.201+0000 7f4ef8b71640 1 -- 172.21.15.43:0/2700177342 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f4ee0004350 msgr2=0x7f4ee0004820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T15:12:49.205 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:49.201+0000 7f4ef8b71640 1 --2- 172.21.15.43:0/2700177342 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f4ee0004350 0x7f4ee0004820 secure :-1 s=READY pgs=2612 cs=0 l=1 rev1=1 crypto rx=0x7f4edc009ac0 tx=0x7f4ee00057f0 comp rx=0 tx=0).stop 2023-08-09T15:12:49.205 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:49.201+0000 7f4ef2d9d640 1 -- 172.21.15.43:0/2700177342 reap_dead start 2023-08-09T15:12:49.205 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:49.201+0000 7f4ef8b71640 1 -- 172.21.15.43:0/2700177342 shutdown_connections 2023-08-09T15:12:49.206 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:49.201+0000 7f4ef8b71640 1 -- 172.21.15.43:0/2700177342 >> 172.21.15.43:0/2700177342 conn(0x7f4ef4069d70 msgr2=0x7f4ef40cac70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-09T15:12:49.206 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:49.201+0000 7f4ef8b71640 1 -- 172.21.15.43:0/2700177342 shutdown_connections 2023-08-09T15:12:49.206 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:49.201+0000 7f4ef8b71640 1 -- 172.21.15.43:0/2700177342 wait complete. 2023-08-09T15:12:49.206 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:49.203+0000 7f4ef8b71640 1 Processor -- start 2023-08-09T15:12:49.206 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:49.203+0000 7f4ef8b71640 1 -- start start 2023-08-09T15:12:49.206 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:49.204+0000 7f4ef8b71640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f4ee0004350 0x7f4ef40b2d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T15:12:49.206 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:49.204+0000 7f4ef8b71640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f4ef41f1ad0 0x7f4ef41503c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T15:12:49.206 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:49.204+0000 7f4ef8b71640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f4ef40b0cd0 0x7f4ef41524e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T15:12:49.207 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:49.204+0000 7f4ef8b71640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4ef40c3bc0 con 0x7f4ef40b0cd0 2023-08-09T15:12:49.207 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:49.204+0000 7f4ef8b71640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4ef41f2040 con 0x7f4ef41f1ad0 2023-08-09T15:12:49.207 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:49.204+0000 7f4ef8b71640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4ef41f2320 con 0x7f4ee0004350 2023-08-09T15:12:49.207 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:49.204+0000 7f4ef2d9d640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f4ef40b0cd0 0x7f4ef41524e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T15:12:49.207 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:49.204+0000 7f4ef2d9d640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f4ef40b0cd0 0x7f4ef41524e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:46298/0 (socket says 172.21.15.43:46298) 2023-08-09T15:12:49.207 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:49.204+0000 7f4ef2d9d640 1 -- 172.21.15.43:0/469917661 learned_addr learned my addr 172.21.15.43:0/469917661 (peer_addr_for_me v2:172.21.15.43:0/0) 2023-08-09T15:12:49.210 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:49.205+0000 7f4eebfff640 1 --2- 172.21.15.43:0/469917661 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f4ef41f1ad0 0x7f4ef41503c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T15:12:49.210 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:49.205+0000 7f4ef259c640 1 --2- 172.21.15.43:0/469917661 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f4ee0004350 0x7f4ef40b2d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T15:12:49.210 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:49.205+0000 7f4ef2d9d640 1 -- 172.21.15.43:0/469917661 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f4ee0004350 msgr2=0x7f4ef40b2d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T15:12:49.210 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:49.205+0000 7f4ef2d9d640 1 --2- 172.21.15.43:0/469917661 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f4ee0004350 0x7f4ef40b2d40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T15:12:49.211 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:49.205+0000 7f4ef2d9d640 1 -- 172.21.15.43:0/469917661 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f4ef41f1ad0 msgr2=0x7f4ef41503c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T15:12:49.211 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:49.205+0000 7f4ef2d9d640 1 --2- 172.21.15.43:0/469917661 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f4ef41f1ad0 0x7f4ef41503c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T15:12:49.211 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:49.205+0000 7f4ef2d9d640 1 -- 172.21.15.43:0/469917661 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4ef41f2600 con 0x7f4ef40b0cd0 2023-08-09T15:12:49.212 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:49.205+0000 7f4ef2d9d640 1 --2- 172.21.15.43:0/469917661 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f4ef40b0cd0 0x7f4ef41524e0 secure :-1 s=READY pgs=2613 cs=0 l=1 rev1=1 crypto rx=0x7f4eec0012e0 tx=0x7f4eec00cb90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-09T15:12:49.212 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:49.205+0000 7f4eeaffd640 1 -- 172.21.15.43:0/469917661 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4eec005d40 con 0x7f4ef40b0cd0 2023-08-09T15:12:49.212 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:49.205+0000 7f4eeaffd640 1 -- 172.21.15.43:0/469917661 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(1 keys) v1 ==== 35+0+0 (secure 0 0 0) 0x7f4eec00ced0 con 0x7f4ef40b0cd0 2023-08-09T15:12:49.213 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:49.205+0000 7f4eeaffd640 1 -- 172.21.15.43:0/469917661 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4eec00a870 con 0x7f4ef40b0cd0 2023-08-09T15:12:49.213 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:49.205+0000 7f4ef8b71640 1 -- 172.21.15.43:0/469917661 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4ef41f28e0 con 0x7f4ef40b0cd0 2023-08-09T15:12:49.213 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:49.205+0000 7f4ef8b71640 1 -- 172.21.15.43:0/469917661 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4ef40b1150 con 0x7f4ef40b0cd0 2023-08-09T15:12:49.213 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:49.207+0000 7f4ef8b71640 1 -- 172.21.15.43:0/469917661 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4ef41f2a70 con 0x7f4ef40b0cd0 2023-08-09T15:12:49.214 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:49.208+0000 7f4eeaffd640 1 -- 172.21.15.43:0/469917661 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 129) v1 ==== 88443+0+0 (secure 0 0 0) 0x7f4eec00a9d0 con 0x7f4ef40b0cd0 2023-08-09T15:12:49.214 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:49.209+0000 7f4eeaffd640 1 --2- 172.21.15.43:0/469917661 >> [v2:172.21.15.203:6832/1456699667,v1:172.21.15.203:6833/1456699667] conn(0x7f4ed406a1a0 0x7f4ed406c660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-09T15:12:49.214 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:49.209+0000 7f4eeaffd640 1 -- 172.21.15.43:0/469917661 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(229..229 src has 1..229) v4 ==== 6462+0+0 (secure 0 0 0) 0x7f4eec087950 con 0x7f4ef40b0cd0 2023-08-09T15:12:49.215 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:49.209+0000 7f4ef259c640 1 --2- 172.21.15.43:0/469917661 >> [v2:172.21.15.203:6832/1456699667,v1:172.21.15.203:6833/1456699667] conn(0x7f4ed406a1a0 0x7f4ed406c660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-09T15:12:49.215 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:49.211+0000 7f4ef259c640 1 --2- 172.21.15.43:0/469917661 >> [v2:172.21.15.203:6832/1456699667,v1:172.21.15.203:6833/1456699667] conn(0x7f4ed406a1a0 0x7f4ed406c660 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f4ee4000fd0 tx=0x7f4ee4009040 comp rx=0 tx=0).ready entity=mgr.9232 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-09T15:12:49.216 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:49.213+0000 7f4eeaffd640 1 -- 172.21.15.43:0/469917661 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178066 (secure 0 0 0) 0x7f4eec052c70 con 0x7f4ef40b0cd0 2023-08-09T15:12:49.447 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:49.446+0000 7f4ef8b71640 1 -- 172.21.15.43:0/469917661 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f4ef404db40 con 0x7f4ef40b0cd0 2023-08-09T15:12:49.447 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:49.447+0000 7f4eeaffd640 1 -- 172.21.15.43:0/469917661 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 106 v106) v1 ==== 95+0+3708 (secure 0 0 0) 0x7f4eec017e90 con 0x7f4ef40b0cd0 2023-08-09T15:12:49.448 INFO:teuthology.orchestra.run.smithi043.stdout: 2023-08-09T15:12:49.448 INFO:teuthology.orchestra.run.smithi043.stdout:{"epoch":106,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":8012,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.203:6837/2464832767","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6836","nonce":2464832767},{"type":"v1","addr":"172.21.15.203:6837","nonce":2464832767}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":100},{"gid":9010,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.43:6835/3989386049","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6834","nonce":3989386049},{"type":"v1","addr":"172.21.15.43:6835","nonce":3989386049}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":100},{"gid":9016,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.203:6835/2213455074","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6834","nonce":2213455074},{"type":"v1","addr":"172.21.15.203:6835","nonce":2213455074}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":100},{"gid":11865,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.43:6833/657828900","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6832","nonce":657828900},{"type":"v1","addr":"172.21.15.43:6833","nonce":657828900}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":106}],"filesystems":[]} 2023-08-09T15:12:49.448 INFO:teuthology.orchestra.run.smithi043.stderr:dumped fsmap epoch 106 2023-08-09T15:12:49.450 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:49.449+0000 7f4ef8b71640 1 -- 172.21.15.43:0/469917661 >> [v2:172.21.15.203:6832/1456699667,v1:172.21.15.203:6833/1456699667] conn(0x7f4ed406a1a0 msgr2=0x7f4ed406c660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T15:12:49.450 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:49.449+0000 7f4ef8b71640 1 --2- 172.21.15.43:0/469917661 >> [v2:172.21.15.203:6832/1456699667,v1:172.21.15.203:6833/1456699667] conn(0x7f4ed406a1a0 0x7f4ed406c660 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f4ee4000fd0 tx=0x7f4ee4009040 comp rx=0 tx=0).stop 2023-08-09T15:12:49.450 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:49.450+0000 7f4ef8b71640 1 -- 172.21.15.43:0/469917661 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f4ef40b0cd0 msgr2=0x7f4ef41524e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-09T15:12:49.450 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:49.450+0000 7f4ef8b71640 1 --2- 172.21.15.43:0/469917661 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f4ef40b0cd0 0x7f4ef41524e0 secure :-1 s=READY pgs=2613 cs=0 l=1 rev1=1 crypto rx=0x7f4eec0012e0 tx=0x7f4eec00cb90 comp rx=0 tx=0).stop 2023-08-09T15:12:49.451 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:49.450+0000 7f4ef8b71640 1 -- 172.21.15.43:0/469917661 shutdown_connections 2023-08-09T15:12:49.451 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:49.450+0000 7f4ef8b71640 1 --2- 172.21.15.43:0/469917661 >> [v2:172.21.15.203:6832/1456699667,v1:172.21.15.203:6833/1456699667] conn(0x7f4ed406a1a0 0x7f4ed406c660 unknown :-1 s=CLOSED pgs=62 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T15:12:49.451 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:49.450+0000 7f4ef8b71640 1 --2- 172.21.15.43:0/469917661 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f4ef40b0cd0 0x7f4ef41524e0 unknown :-1 s=CLOSED pgs=2613 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T15:12:49.451 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:49.450+0000 7f4ef8b71640 1 --2- 172.21.15.43:0/469917661 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f4ef41f1ad0 0x7f4ef41503c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T15:12:49.451 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:49.450+0000 7f4ef8b71640 1 --2- 172.21.15.43:0/469917661 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f4ee0004350 0x7f4ef40b2d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-09T15:12:49.452 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:49.450+0000 7f4ef8b71640 1 -- 172.21.15.43:0/469917661 >> 172.21.15.43:0/469917661 conn(0x7f4ef4069d70 msgr2=0x7f4ef4159700 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-09T15:12:49.452 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:49.450+0000 7f4ef8b71640 1 -- 172.21.15.43:0/469917661 shutdown_connections 2023-08-09T15:12:49.452 INFO:teuthology.orchestra.run.smithi043.stderr:2023-08-09T15:12:49.450+0000 7f4ef8b71640 1 -- 172.21.15.43:0/469917661 wait complete. 2023-08-09T15:12:49.471 DEBUG:tasks.cephfs.filesystem:already dead... 2023-08-09T15:12:49.471 INFO:teuthology.misc:Shutting down mds daemons... 2023-08-09T15:12:49.471 DEBUG:tasks.ceph.mds.a:waiting for process to exit 2023-08-09T15:12:49.472 INFO:teuthology.orchestra.run:waiting for 300 2023-08-09T15:12:49.487 INFO:tasks.ceph.mds.a:Stopped 2023-08-09T15:12:49.487 DEBUG:tasks.ceph.mds.c:waiting for process to exit 2023-08-09T15:12:49.488 INFO:teuthology.orchestra.run:waiting for 300 2023-08-09T15:12:49.561 INFO:tasks.ceph.mds.c:Stopped 2023-08-09T15:12:49.562 DEBUG:tasks.ceph.mds.b:waiting for process to exit 2023-08-09T15:12:49.562 INFO:teuthology.orchestra.run:waiting for 300 2023-08-09T15:12:49.583 INFO:tasks.ceph.mds.b:Stopped 2023-08-09T15:12:49.583 DEBUG:tasks.ceph.mds.d:waiting for process to exit 2023-08-09T15:12:49.584 INFO:teuthology.orchestra.run:waiting for 300 2023-08-09T15:12:49.645 INFO:tasks.ceph.mds.d:Stopped 2023-08-09T15:12:49.645 INFO:teuthology.misc:Shutting down osd daemons... 2023-08-09T15:12:49.645 DEBUG:tasks.ceph.osd.0:waiting for process to exit 2023-08-09T15:12:49.646 INFO:teuthology.orchestra.run:waiting for 300 2023-08-09T15:12:49.729 INFO:tasks.ceph.osd.0:Stopped 2023-08-09T15:12:49.729 DEBUG:tasks.ceph.osd.1:waiting for process to exit 2023-08-09T15:12:49.729 INFO:teuthology.orchestra.run:waiting for 300 2023-08-09T15:12:49.808 INFO:tasks.ceph.osd.1:Stopped 2023-08-09T15:12:49.809 DEBUG:tasks.ceph.osd.2:waiting for process to exit 2023-08-09T15:12:49.809 INFO:teuthology.orchestra.run:waiting for 300 2023-08-09T15:12:49.901 INFO:tasks.ceph.osd.2:Stopped 2023-08-09T15:12:49.901 DEBUG:tasks.ceph.osd.3:waiting for process to exit 2023-08-09T15:12:49.902 INFO:teuthology.orchestra.run:waiting for 300 2023-08-09T15:12:49.984 INFO:tasks.ceph.osd.3:Stopped 2023-08-09T15:12:49.985 DEBUG:tasks.ceph.osd.4:waiting for process to exit 2023-08-09T15:12:49.985 INFO:teuthology.orchestra.run:waiting for 300 2023-08-09T15:12:50.073 INFO:tasks.ceph.osd.4:Stopped 2023-08-09T15:12:50.074 DEBUG:tasks.ceph.osd.5:waiting for process to exit 2023-08-09T15:12:50.074 INFO:teuthology.orchestra.run:waiting for 300 2023-08-09T15:12:50.146 INFO:tasks.ceph.osd.5:Stopped 2023-08-09T15:12:50.147 DEBUG:tasks.ceph.osd.6:waiting for process to exit 2023-08-09T15:12:50.147 INFO:teuthology.orchestra.run:waiting for 300 2023-08-09T15:12:50.219 INFO:tasks.ceph.osd.6:Stopped 2023-08-09T15:12:50.220 DEBUG:tasks.ceph.osd.7:waiting for process to exit 2023-08-09T15:12:50.220 INFO:teuthology.orchestra.run:waiting for 300 2023-08-09T15:12:50.298 INFO:tasks.ceph.osd.7:Stopped 2023-08-09T15:12:50.299 INFO:teuthology.misc:Shutting down mgr daemons... 2023-08-09T15:12:50.299 DEBUG:tasks.ceph.mgr.y:waiting for process to exit 2023-08-09T15:12:50.299 INFO:teuthology.orchestra.run:waiting for 300 2023-08-09T15:12:53.808 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mgr.y is failed for ~0s 2023-08-09T15:12:56.401 INFO:tasks.ceph.mgr.y:Stopped 2023-08-09T15:12:56.402 DEBUG:tasks.ceph.mgr.x:waiting for process to exit 2023-08-09T15:12:56.403 INFO:teuthology.orchestra.run:waiting for 300 2023-08-09T15:12:56.487 INFO:tasks.ceph.mgr.x:Stopped 2023-08-09T15:12:56.488 INFO:teuthology.misc:Shutting down mon daemons... 2023-08-09T15:12:56.488 DEBUG:tasks.ceph.mon.a:waiting for process to exit 2023-08-09T15:12:56.489 INFO:teuthology.orchestra.run:waiting for 300 2023-08-09T15:12:56.570 INFO:tasks.ceph.mon.a:Stopped 2023-08-09T15:12:56.571 DEBUG:tasks.ceph.mon.b:waiting for process to exit 2023-08-09T15:12:56.572 INFO:teuthology.orchestra.run:waiting for 300 2023-08-09T15:12:56.630 INFO:tasks.ceph.mon.b:Stopped 2023-08-09T15:12:56.631 DEBUG:tasks.ceph.mon.c:waiting for process to exit 2023-08-09T15:12:56.631 INFO:teuthology.orchestra.run:waiting for 300 2023-08-09T15:12:56.682 INFO:tasks.ceph.mon.c:Stopped 2023-08-09T15:12:56.683 INFO:tasks.ceph:Checking cluster log for badness... 2023-08-09T15:12:56.683 DEBUG:teuthology.orchestra.run.smithi043:> sudo egrep '\[ERR\]|\[WRN\]|\[SEC\]' /var/log/ceph/ceph.log | egrep -v '\(MDS_ALL_DOWN\)' | egrep -v '\(MDS_UP_LESS_THAN_MAX\)' | egrep -v 'overall HEALTH_' | egrep -v '\(FS_DEGRADED\)' | egrep -v '\(MDS_FAILED\)' | egrep -v '\(MDS_DEGRADED\)' | egrep -v '\(FS_WITH_FAILED_MDS\)' | egrep -v '\(MDS_DAMAGE\)' | egrep -v '\(MDS_ALL_DOWN\)' | egrep -v '\(MDS_UP_LESS_THAN_MAX\)' | egrep -v '\(FS_INLINE_DATA_DEPRECATED\)' | egrep -v 'overall HEALTH_' | egrep -v '\(OSD_DOWN\)' | egrep -v '\(OSD_' | egrep -v 'but it is still running' | egrep -v 'is not responding' | head -n 1 2023-08-09T15:12:56.797 INFO:teuthology.orchestra.run.smithi043.stdout:2023-08-09T13:37:30.108177+0000 mon.a (mon.0) 437 : cluster [WRN] Health check failed: Reduced data availability: 2 pgs peering (PG_AVAILABILITY) 2023-08-09T15:12:56.798 WARNING:tasks.ceph:Found errors (ERR|WRN|SEC) in cluster log 2023-08-09T15:12:56.798 DEBUG:teuthology.orchestra.run.smithi043:> sudo egrep '\[SEC\]' /var/log/ceph/ceph.log | egrep -v '\(MDS_ALL_DOWN\)' | egrep -v '\(MDS_UP_LESS_THAN_MAX\)' | egrep -v 'overall HEALTH_' | egrep -v '\(FS_DEGRADED\)' | egrep -v '\(MDS_FAILED\)' | egrep -v '\(MDS_DEGRADED\)' | egrep -v '\(FS_WITH_FAILED_MDS\)' | egrep -v '\(MDS_DAMAGE\)' | egrep -v '\(MDS_ALL_DOWN\)' | egrep -v '\(MDS_UP_LESS_THAN_MAX\)' | egrep -v '\(FS_INLINE_DATA_DEPRECATED\)' | egrep -v 'overall HEALTH_' | egrep -v '\(OSD_DOWN\)' | egrep -v '\(OSD_' | egrep -v 'but it is still running' | egrep -v 'is not responding' | head -n 1 2023-08-09T15:12:56.885 DEBUG:teuthology.orchestra.run.smithi043:> sudo egrep '\[ERR\]' /var/log/ceph/ceph.log | egrep -v '\(MDS_ALL_DOWN\)' | egrep -v '\(MDS_UP_LESS_THAN_MAX\)' | egrep -v 'overall HEALTH_' | egrep -v '\(FS_DEGRADED\)' | egrep -v '\(MDS_FAILED\)' | egrep -v '\(MDS_DEGRADED\)' | egrep -v '\(FS_WITH_FAILED_MDS\)' | egrep -v '\(MDS_DAMAGE\)' | egrep -v '\(MDS_ALL_DOWN\)' | egrep -v '\(MDS_UP_LESS_THAN_MAX\)' | egrep -v '\(FS_INLINE_DATA_DEPRECATED\)' | egrep -v 'overall HEALTH_' | egrep -v '\(OSD_DOWN\)' | egrep -v '\(OSD_' | egrep -v 'but it is still running' | egrep -v 'is not responding' | head -n 1 2023-08-09T15:12:56.971 DEBUG:teuthology.orchestra.run.smithi043:> sudo egrep '\[WRN\]' /var/log/ceph/ceph.log | egrep -v '\(MDS_ALL_DOWN\)' | egrep -v '\(MDS_UP_LESS_THAN_MAX\)' | egrep -v 'overall HEALTH_' | egrep -v '\(FS_DEGRADED\)' | egrep -v '\(MDS_FAILED\)' | egrep -v '\(MDS_DEGRADED\)' | egrep -v '\(FS_WITH_FAILED_MDS\)' | egrep -v '\(MDS_DAMAGE\)' | egrep -v '\(MDS_ALL_DOWN\)' | egrep -v '\(MDS_UP_LESS_THAN_MAX\)' | egrep -v '\(FS_INLINE_DATA_DEPRECATED\)' | egrep -v 'overall HEALTH_' | egrep -v '\(OSD_DOWN\)' | egrep -v '\(OSD_' | egrep -v 'but it is still running' | egrep -v 'is not responding' | head -n 1 2023-08-09T15:12:57.053 INFO:teuthology.orchestra.run.smithi043.stdout:2023-08-09T13:37:30.108177+0000 mon.a (mon.0) 437 : cluster [WRN] Health check failed: Reduced data availability: 2 pgs peering (PG_AVAILABILITY) 2023-08-09T15:12:57.055 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-0 on ubuntu@smithi043.front.sepia.ceph.com 2023-08-09T15:12:57.055 DEBUG:teuthology.orchestra.run.smithi043:> sync && sudo umount -f /var/lib/ceph/osd/ceph-0 2023-08-09T15:12:57.453 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-1 on ubuntu@smithi043.front.sepia.ceph.com 2023-08-09T15:12:57.454 DEBUG:teuthology.orchestra.run.smithi043:> sync && sudo umount -f /var/lib/ceph/osd/ceph-1 2023-08-09T15:12:57.750 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-2 on ubuntu@smithi043.front.sepia.ceph.com 2023-08-09T15:12:57.751 DEBUG:teuthology.orchestra.run.smithi043:> sync && sudo umount -f /var/lib/ceph/osd/ceph-2 2023-08-09T15:12:58.003 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-3 on ubuntu@smithi043.front.sepia.ceph.com 2023-08-09T15:12:58.004 DEBUG:teuthology.orchestra.run.smithi043:> sync && sudo umount -f /var/lib/ceph/osd/ceph-3 2023-08-09T15:12:58.279 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-4 on ubuntu@smithi203.front.sepia.ceph.com 2023-08-09T15:12:58.279 DEBUG:teuthology.orchestra.run.smithi203:> sync && sudo umount -f /var/lib/ceph/osd/ceph-4 2023-08-09T15:12:58.607 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-5 on ubuntu@smithi203.front.sepia.ceph.com 2023-08-09T15:12:58.608 DEBUG:teuthology.orchestra.run.smithi203:> sync && sudo umount -f /var/lib/ceph/osd/ceph-5 2023-08-09T15:12:58.808 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mgr.y has been restored 2023-08-09T15:12:58.886 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-6 on ubuntu@smithi203.front.sepia.ceph.com 2023-08-09T15:12:58.887 DEBUG:teuthology.orchestra.run.smithi203:> sync && sudo umount -f /var/lib/ceph/osd/ceph-6 2023-08-09T15:12:59.167 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-7 on ubuntu@smithi203.front.sepia.ceph.com 2023-08-09T15:12:59.168 DEBUG:teuthology.orchestra.run.smithi203:> sync && sudo umount -f /var/lib/ceph/osd/ceph-7 2023-08-09T15:12:59.426 INFO:tasks.ceph:Archiving mon data... 2023-08-09T15:12:59.427 DEBUG:teuthology.misc:Transferring archived files from smithi043:/var/lib/ceph/mon/ceph-a to /home/teuthworker/archive/vshankar-2023-08-09_05:46:29-fs-wip-vshankar-testing-20230809.035933-testing-default-smithi/7364139/data/mon.a.tgz 2023-08-09T15:12:59.427 DEBUG:teuthology.orchestra.run.smithi043:> mktemp 2023-08-09T15:12:59.491 INFO:teuthology.orchestra.run.smithi043.stdout:/tmp/tmp.leanKBbscs 2023-08-09T15:12:59.492 DEBUG:teuthology.orchestra.run.smithi043:> sudo tar cz -f - -C /var/lib/ceph/mon/ceph-a -- . > /tmp/tmp.leanKBbscs 2023-08-09T15:13:00.124 DEBUG:teuthology.orchestra.run.smithi043:> sudo chmod 0666 /tmp/tmp.leanKBbscs 2023-08-09T15:13:00.281 DEBUG:teuthology.orchestra.remote:smithi043:/tmp/tmp.leanKBbscs is 3MB 2023-08-09T15:13:00.444 DEBUG:teuthology.orchestra.run.smithi043:> rm -fr /tmp/tmp.leanKBbscs 2023-08-09T15:13:00.508 DEBUG:teuthology.misc:Transferring archived files from smithi203:/var/lib/ceph/mon/ceph-b to /home/teuthworker/archive/vshankar-2023-08-09_05:46:29-fs-wip-vshankar-testing-20230809.035933-testing-default-smithi/7364139/data/mon.b.tgz 2023-08-09T15:13:00.508 DEBUG:teuthology.orchestra.run.smithi203:> mktemp 2023-08-09T15:13:00.569 INFO:teuthology.orchestra.run.smithi203.stdout:/tmp/tmp.nRXxcg52YV 2023-08-09T15:13:00.569 DEBUG:teuthology.orchestra.run.smithi203:> sudo tar cz -f - -C /var/lib/ceph/mon/ceph-b -- . > /tmp/tmp.nRXxcg52YV 2023-08-09T15:13:01.196 DEBUG:teuthology.orchestra.run.smithi203:> sudo chmod 0666 /tmp/tmp.nRXxcg52YV 2023-08-09T15:13:01.337 DEBUG:teuthology.orchestra.remote:smithi203:/tmp/tmp.nRXxcg52YV is 3MB 2023-08-09T15:13:01.483 DEBUG:teuthology.orchestra.run.smithi203:> rm -fr /tmp/tmp.nRXxcg52YV 2023-08-09T15:13:01.544 DEBUG:teuthology.misc:Transferring archived files from smithi203:/var/lib/ceph/mon/ceph-c to /home/teuthworker/archive/vshankar-2023-08-09_05:46:29-fs-wip-vshankar-testing-20230809.035933-testing-default-smithi/7364139/data/mon.c.tgz 2023-08-09T15:13:01.544 DEBUG:teuthology.orchestra.run.smithi203:> mktemp 2023-08-09T15:13:01.600 INFO:teuthology.orchestra.run.smithi203.stdout:/tmp/tmp.Teo70mUoGC 2023-08-09T15:13:01.600 DEBUG:teuthology.orchestra.run.smithi203:> sudo tar cz -f - -C /var/lib/ceph/mon/ceph-c -- . > /tmp/tmp.Teo70mUoGC 2023-08-09T15:13:02.154 DEBUG:teuthology.orchestra.run.smithi203:> sudo chmod 0666 /tmp/tmp.Teo70mUoGC 2023-08-09T15:13:02.298 DEBUG:teuthology.orchestra.remote:smithi203:/tmp/tmp.Teo70mUoGC is 3MB 2023-08-09T15:13:02.440 DEBUG:teuthology.orchestra.run.smithi203:> rm -fr /tmp/tmp.Teo70mUoGC 2023-08-09T15:13:02.502 INFO:tasks.ceph:Cleaning ceph cluster... 2023-08-09T15:13:02.503 DEBUG:teuthology.orchestra.run.smithi043:> sudo rm -rf -- /etc/ceph/ceph.conf /etc/ceph/ceph.keyring /home/ubuntu/cephtest/ceph.data /home/ubuntu/cephtest/ceph.monmap /home/ubuntu/cephtest/../*.pid 2023-08-09T15:13:02.512 DEBUG:teuthology.orchestra.run.smithi203:> sudo rm -rf -- /etc/ceph/ceph.conf /etc/ceph/ceph.keyring /home/ubuntu/cephtest/ceph.data /home/ubuntu/cephtest/ceph.monmap /home/ubuntu/cephtest/../*.pid 2023-08-09T15:13:02.614 INFO:tasks.ceph:Checking for errors in any valgrind logs... 2023-08-09T15:13:02.615 DEBUG:teuthology.orchestra.run.smithi043:> sudo zgrep '' /var/log/ceph/valgrind/* /dev/null | sort | uniq 2023-08-09T15:13:02.621 DEBUG:teuthology.orchestra.run.smithi203:> sudo zgrep '' /var/log/ceph/valgrind/* /dev/null | sort | uniq 2023-08-09T15:13:02.728 INFO:teuthology.orchestra.run.smithi043.stderr:gzip: /var/log/ceph/valgrind/*.gz: No such file or directory 2023-08-09T15:13:02.732 INFO:teuthology.orchestra.run.smithi203.stderr:gzip: /var/log/ceph/valgrind/*.gz: No such file or directory 2023-08-09T15:13:02.745 INFO:tasks.ceph:Archiving crash dumps... 2023-08-09T15:13:02.747 DEBUG:teuthology.misc:Transferring archived files from smithi043:/var/lib/ceph/crash to /home/teuthworker/archive/vshankar-2023-08-09_05:46:29-fs-wip-vshankar-testing-20230809.035933-testing-default-smithi/7364139/remote/smithi043/crash 2023-08-09T15:13:02.748 DEBUG:teuthology.orchestra.run.smithi043:> sudo tar c -f - -C /var/lib/ceph/crash -- . 2023-08-09T15:13:02.776 DEBUG:teuthology.orchestra.run.smithi043:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-08-09T15:13:02.798 DEBUG:teuthology.orchestra.run.smithi203:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-08-09T15:13:02.829 DEBUG:teuthology.misc:Transferring archived files from smithi203:/var/lib/ceph/crash to /home/teuthworker/archive/vshankar-2023-08-09_05:46:29-fs-wip-vshankar-testing-20230809.035933-testing-default-smithi/7364139/remote/smithi203/crash 2023-08-09T15:13:02.831 DEBUG:teuthology.orchestra.run.smithi203:> sudo tar c -f - -C /var/lib/ceph/crash -- . 2023-08-09T15:13:02.931 INFO:tasks.ceph:Shutting down logrotate 2023-08-09T15:13:02.941 DEBUG:teuthology.orchestra.run.smithi043:> sudo rm /etc/logrotate.d/ceph-test.conf 2023-08-09T15:13:03.030 DEBUG:teuthology.orchestra.run.smithi203:> sudo rm /etc/logrotate.d/ceph-test.conf 2023-08-09T15:13:03.134 INFO:tasks.ceph:Compressing logs... 2023-08-09T15:13:03.135 DEBUG:teuthology.orchestra.run.smithi043:> time sudo find /var/log/ceph -name '*.log' -print0 | sudo xargs --max-args=1 --max-procs=0 --verbose -0 --no-run-if-empty -- gzip -5 --verbose -- 2023-08-09T15:13:03.145 DEBUG:teuthology.orchestra.run.smithi203:> time sudo find /var/log/ceph -name '*.log' -print0 | sudo xargs --max-args=1 --max-procs=0 --verbose -0 --no-run-if-empty -- gzip -5 --verbose -- 2023-08-09T15:13:03.242 INFO:teuthology.orchestra.run.smithi203.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.89315.log 2023-08-09T15:13:03.243 INFO:teuthology.orchestra.run.smithi203.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.89436.log 2023-08-09T15:13:03.244 INFO:teuthology.orchestra.run.smithi043.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.100252.log 2023-08-09T15:13:03.245 INFO:teuthology.orchestra.run.smithi203.stderr:/var/log/ceph/ceph-client.admin.89315.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.89315.log.gz 2023-08-09T15:13:03.245 INFO:teuthology.orchestra.run.smithi203.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.89145.log 2023-08-09T15:13:03.246 INFO:teuthology.orchestra.run.smithi043.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.99538.log 2023-08-09T15:13:03.246 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.100252.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.100252.log.gz 2023-08-09T15:13:03.247 INFO:teuthology.orchestra.run.smithi043.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.139819.log 2023-08-09T15:13:03.247 INFO:teuthology.orchestra.run.smithi203.stderr:/var/log/ceph/ceph-client.admin.89436.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.89436.log.gz 2023-08-09T15:13:03.248 INFO:teuthology.orchestra.run.smithi203.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-osd.7.log 2023-08-09T15:13:03.248 INFO:teuthology.orchestra.run.smithi203.stderr:/var/log/ceph/ceph-client.admin.89145.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.89145.log.gz 2023-08-09T15:13:03.248 INFO:teuthology.orchestra.run.smithi203.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-osd.4.log 2023-08-09T15:13:03.249 INFO:teuthology.orchestra.run.smithi203.stderr:/var/log/ceph/ceph-osd.7.log: gzip -5 --verbose -- /var/log/ceph/ceph-osd.6.log 2023-08-09T15:13:03.250 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.99538.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.157142.log 2023-08-09T15:13:03.250 INFO:teuthology.orchestra.run.smithi043.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.99538.log.gz 2023-08-09T15:13:03.250 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.139819.log: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.139819.log.gz 2023-08-09T15:13:03.250 INFO:teuthology.orchestra.run.smithi043.stderr:gzip -5 --verbose -- /var/log/ceph/ceph.tmp-client.admin.83138.log 2023-08-09T15:13:03.251 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.157142.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.104431.log 2023-08-09T15:13:03.251 INFO:teuthology.orchestra.run.smithi043.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.157142.log.gz 2023-08-09T15:13:03.252 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph.tmp-client.admin.83138.log: 0.0% -- replaced with /var/log/ceph/ceph.tmp-client.admin.83138.log.gz 2023-08-09T15:13:03.252 INFO:teuthology.orchestra.run.smithi203.stderr:/var/log/ceph/ceph-osd.4.log: gzip -5 --verbose -- /var/log/ceph/ceph-mds.b.log 2023-08-09T15:13:03.252 INFO:teuthology.orchestra.run.smithi203.stderr:/var/log/ceph/ceph-osd.6.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.89396.log 2023-08-09T15:13:03.253 INFO:teuthology.orchestra.run.smithi203.stderr:/var/log/ceph/ceph-mds.b.log: gzip -5 --verbose -- /var/log/ceph/ceph.audit.log 2023-08-09T15:13:03.253 INFO:teuthology.orchestra.run.smithi043.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.107021.log 2023-08-09T15:13:03.253 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.104431.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.104431.log.gz 2023-08-09T15:13:03.254 INFO:teuthology.orchestra.run.smithi043.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.151949.log 2023-08-09T15:13:03.255 INFO:teuthology.orchestra.run.smithi203.stderr:/var/log/ceph/ceph-client.admin.89396.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.89396.log.gz 2023-08-09T15:13:03.255 INFO:teuthology.orchestra.run.smithi203.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-mds.d.log 2023-08-09T15:13:03.256 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.107021.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.153164.log 2023-08-09T15:13:03.256 INFO:teuthology.orchestra.run.smithi043.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.107021.log.gz 2023-08-09T15:13:03.257 INFO:teuthology.orchestra.run.smithi203.stderr:/var/log/ceph/ceph.audit.log: gzip -5 --verbose -- /var/log/ceph/ceph-osd.5.log 2023-08-09T15:13:03.257 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.151949.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.102853.log 2023-08-09T15:13:03.258 INFO:teuthology.orchestra.run.smithi043.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.151949.log.gz 2023-08-09T15:13:03.258 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.153164.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.153164.log.gz 2023-08-09T15:13:03.258 INFO:teuthology.orchestra.run.smithi043.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.132408.log 2023-08-09T15:13:03.259 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.102853.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.156000.log 2023-08-09T15:13:03.259 INFO:teuthology.orchestra.run.smithi043.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.102853.log.gz 2023-08-09T15:13:03.260 INFO:teuthology.orchestra.run.smithi203.stderr:/var/log/ceph/ceph-mds.d.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.89265.log 2023-08-09T15:13:03.260 INFO:teuthology.orchestra.run.smithi203.stderr:/var/log/ceph/ceph-osd.5.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.89356.log 2023-08-09T15:13:03.261 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.132408.log: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.132408.log.gz 2023-08-09T15:13:03.261 INFO:teuthology.orchestra.run.smithi043.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.137825.log 2023-08-09T15:13:03.262 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.156000.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.156000.log.gz 2023-08-09T15:13:03.262 INFO:teuthology.orchestra.run.smithi043.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.104083.log 2023-08-09T15:13:03.263 INFO:teuthology.orchestra.run.smithi203.stderr:/var/log/ceph/ceph-client.admin.89265.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.89265.log.gz 2023-08-09T15:13:03.263 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.137825.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.153290.log 2023-08-09T15:13:03.263 INFO:teuthology.orchestra.run.smithi043.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.137825.log.gz 2023-08-09T15:13:03.264 INFO:teuthology.orchestra.run.smithi203.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-mgr.x.log 2023-08-09T15:13:03.264 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.104083.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.104083.log.gz 2023-08-09T15:13:03.265 INFO:teuthology.orchestra.run.smithi043.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.119991.log 2023-08-09T15:13:03.266 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.153290.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.155553.log 2023-08-09T15:13:03.266 INFO:teuthology.orchestra.run.smithi043.stderr: 90.8% -- replaced with /var/log/ceph/ceph-client.admin.153290.log.gz 2023-08-09T15:13:03.267 INFO:teuthology.orchestra.run.smithi203.stderr:/var/log/ceph/ceph-client.admin.89356.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.89356.log.gz 2023-08-09T15:13:03.267 INFO:teuthology.orchestra.run.smithi203.stderr: 93.7% -- replaced with /var/log/ceph/ceph.audit.log.gz 2023-08-09T15:13:03.268 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.119991.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.119011.log 2023-08-09T15:13:03.268 INFO:teuthology.orchestra.run.smithi043.stderr: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.119991.log.gz 2023-08-09T15:13:03.269 INFO:teuthology.orchestra.run.smithi203.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-mon.b.log 2023-08-09T15:13:03.269 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.155553.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.155553.log.gz 2023-08-09T15:13:03.269 INFO:teuthology.orchestra.run.smithi043.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.130781.log 2023-08-09T15:13:03.270 INFO:teuthology.orchestra.run.smithi203.stderr:/var/log/ceph/ceph-mgr.x.log: gzip -5 --verbose -- /var/log/ceph/ceph-mon.c.log 2023-08-09T15:13:03.270 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.119011.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.119011.log.gz 2023-08-09T15:13:03.271 INFO:teuthology.orchestra.run.smithi043.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.151594.log 2023-08-09T15:13:03.271 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.130781.log: gzip -5 88.6% --verbose -- -- replaced with /var/log/ceph/ceph-client.admin.130781.log.gz 2023-08-09T15:13:03.271 INFO:teuthology.orchestra.run.smithi043.stderr: /var/log/ceph/ceph-osd.2.log 2023-08-09T15:13:03.272 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.151594.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.151594.log.gz 2023-08-09T15:13:03.273 INFO:teuthology.orchestra.run.smithi043.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.141345.log 2023-08-09T15:13:03.274 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-osd.2.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.146533.log 2023-08-09T15:13:03.275 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.141345.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.126985.log 2023-08-09T15:13:03.275 INFO:teuthology.orchestra.run.smithi043.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.141345.log.gz 2023-08-09T15:13:03.277 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.146533.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.146533.log.gz 2023-08-09T15:13:03.277 INFO:teuthology.orchestra.run.smithi043.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.96641.log 2023-08-09T15:13:03.278 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.126985.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.126985.log.gz 2023-08-09T15:13:03.279 INFO:teuthology.orchestra.run.smithi043.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.139753.log 2023-08-09T15:13:03.280 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.96641.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.96641.log.gz 2023-08-09T15:13:03.280 INFO:teuthology.orchestra.run.smithi203.stderr:/var/log/ceph/ceph-mon.b.log: gzip -5 --verbose -- /var/log/ceph/ceph.log 2023-08-09T15:13:03.281 INFO:teuthology.orchestra.run.smithi043.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.113051.log 2023-08-09T15:13:03.281 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.139753.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.99622.log 2023-08-09T15:13:03.282 INFO:teuthology.orchestra.run.smithi043.stderr: 91.4% -- replaced with /var/log/ceph/ceph-client.admin.139753.log.gz 2023-08-09T15:13:03.283 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.113051.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.113051.log.gz 2023-08-09T15:13:03.284 INFO:teuthology.orchestra.run.smithi043.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.131559.log 2023-08-09T15:13:03.284 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.99622.log: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.99622.log.gz 2023-08-09T15:13:03.285 INFO:teuthology.orchestra.run.smithi043.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.100042.log 2023-08-09T15:13:03.286 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.131559.log: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.131559.log.gz 2023-08-09T15:13:03.286 INFO:teuthology.orchestra.run.smithi043.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.140719.log 2023-08-09T15:13:03.287 INFO:teuthology.orchestra.run.smithi203.stderr:/var/log/ceph/ceph-mon.c.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.89225.log 2023-08-09T15:13:03.288 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.100042.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.95988.log 2023-08-09T15:13:03.288 INFO:teuthology.orchestra.run.smithi043.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.100042.log.gz 2023-08-09T15:13:03.289 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.140719.log: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.140719.log.gz 2023-08-09T15:13:03.290 INFO:teuthology.orchestra.run.smithi043.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.141784.log 2023-08-09T15:13:03.290 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.95988.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.95988.log.gz 2023-08-09T15:13:03.291 INFO:teuthology.orchestra.run.smithi043.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.120033.log 2023-08-09T15:13:03.292 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.141784.log: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.141784.log.gz 2023-08-09T15:13:03.293 INFO:teuthology.orchestra.run.smithi043.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.119232.log 2023-08-09T15:13:03.294 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.120033.log: gzip 88.8% -5 -- replaced with /var/log/ceph/ceph-client.admin.120033.log.gz 2023-08-09T15:13:03.294 INFO:teuthology.orchestra.run.smithi043.stderr: --verbose -- /var/log/ceph/ceph-client.admin.103441.log 2023-08-09T15:13:03.295 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.119232.log: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.119232.log.gz 2023-08-09T15:13:03.296 INFO:teuthology.orchestra.run.smithi043.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.132754.log 2023-08-09T15:13:03.297 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.103441.log: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.103441.log.gz 2023-08-09T15:13:03.297 INFO:teuthology.orchestra.run.smithi043.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.120625.log 2023-08-09T15:13:03.298 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.132754.log: 88.2%gzip -- replaced with /var/log/ceph/ceph-client.admin.132754.log.gz 2023-08-09T15:13:03.298 INFO:teuthology.orchestra.run.smithi043.stderr: -5 --verbose -- /var/log/ceph/ceph-client.admin.143564.log 2023-08-09T15:13:03.300 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.120625.log: gzip 88.8% -- replaced with /var/log/ceph/ceph-client.admin.120625.log.gz 2023-08-09T15:13:03.300 INFO:teuthology.orchestra.run.smithi043.stderr: -5 --verbose -- /var/log/ceph/ceph-client.admin.132364.log 2023-08-09T15:13:03.301 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.143564.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.100168.log 2023-08-09T15:13:03.301 INFO:teuthology.orchestra.run.smithi043.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.143564.log.gz 2023-08-09T15:13:03.302 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.132364.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.132364.log.gz 2023-08-09T15:13:03.303 INFO:teuthology.orchestra.run.smithi043.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.119931.log 2023-08-09T15:13:03.304 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.100168.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.100168.log.gz 2023-08-09T15:13:03.304 INFO:teuthology.orchestra.run.smithi043.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.97236.log 2023-08-09T15:13:03.306 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.119931.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.119931.log.gz 2023-08-09T15:13:03.306 INFO:teuthology.orchestra.run.smithi043.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.152977.log 2023-08-09T15:13:03.307 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.97236.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.120201.log 2023-08-09T15:13:03.307 INFO:teuthology.orchestra.run.smithi043.stderr: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.97236.log.gz 2023-08-09T15:13:03.308 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.152977.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.152977.log.gz 2023-08-09T15:13:03.309 INFO:teuthology.orchestra.run.smithi043.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.103999.log 2023-08-09T15:13:03.312 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.120201.log: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.120201.log.gz 2023-08-09T15:13:03.312 INFO:teuthology.orchestra.run.smithi043.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.148504.log 2023-08-09T15:13:03.312 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.103999.log: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.103999.log.gz 2023-08-09T15:13:03.313 INFO:teuthology.orchestra.run.smithi043.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.141952.log 2023-08-09T15:13:03.313 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.148504.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.148504.log.gz 2023-08-09T15:13:03.313 INFO:teuthology.orchestra.run.smithi043.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.151384.log 2023-08-09T15:13:03.314 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.141952.log: gzip 88.1% -- replaced with /var/log/ceph/ceph-client.admin.141952.log.gz -5 2023-08-09T15:13:03.314 INFO:teuthology.orchestra.run.smithi043.stderr: --verbose -- /var/log/ceph/ceph-client.admin.151342.log 2023-08-09T15:13:03.316 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.151384.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.123192.log 2023-08-09T15:13:03.316 INFO:teuthology.orchestra.run.smithi043.stderr: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.151384.log.gz 2023-08-09T15:13:03.317 INFO:teuthology.orchestra.run.smithi203.stderr:/var/log/ceph/ceph.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.1.94798.log 2023-08-09T15:13:03.317 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.151342.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.128772.log 87.4% 2023-08-09T15:13:03.317 INFO:teuthology.orchestra.run.smithi043.stderr: -- replaced with /var/log/ceph/ceph-client.admin.151342.log.gz 2023-08-09T15:13:03.318 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.123192.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.128622.log 2023-08-09T15:13:03.318 INFO:teuthology.orchestra.run.smithi043.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.123192.log.gz 2023-08-09T15:13:03.319 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.128772.log: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.128772.log.gz 2023-08-09T15:13:03.320 INFO:teuthology.orchestra.run.smithi043.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.133684.log 2023-08-09T15:13:03.320 INFO:teuthology.orchestra.run.smithi203.stderr:/var/log/ceph/ceph-client.admin.89225.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.89225.log.gz 2023-08-09T15:13:03.321 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.128622.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.120932.log 2023-08-09T15:13:03.321 INFO:teuthology.orchestra.run.smithi043.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.128622.log.gz 2023-08-09T15:13:03.322 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.133684.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.116608.log 2023-08-09T15:13:03.322 INFO:teuthology.orchestra.run.smithi043.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.133684.log.gz 2023-08-09T15:13:03.324 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.120932.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.123444.log 2023-08-09T15:13:03.324 INFO:teuthology.orchestra.run.smithi043.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.120932.log.gz 2023-08-09T15:13:03.325 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.116608.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.108313.log 2023-08-09T15:13:03.325 INFO:teuthology.orchestra.run.smithi043.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.116608.log.gz 2023-08-09T15:13:03.326 INFO:teuthology.orchestra.run.smithi203.stderr: 94.3%gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.89185.log 2023-08-09T15:13:03.326 INFO:teuthology.orchestra.run.smithi203.stderr: -- replaced with /var/log/ceph/ceph.log.gz/var/log/ceph/ceph-client.1.94798.log: 2023-08-09T15:13:03.327 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.123444.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.123444.log.gz 2023-08-09T15:13:03.327 INFO:teuthology.orchestra.run.smithi043.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.127027.log 2023-08-09T15:13:03.328 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.108313.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.130477.log 2023-08-09T15:13:03.328 INFO:teuthology.orchestra.run.smithi043.stderr: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.108313.log.gz 2023-08-09T15:13:03.329 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.127027.log: gzip -5 89.2% --verbose -- -- replaced with /var/log/ceph/ceph-client.admin.127027.log.gz 2023-08-09T15:13:03.329 INFO:teuthology.orchestra.run.smithi043.stderr: /var/log/ceph/ceph-client.admin.118780.log 2023-08-09T15:13:03.331 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.130477.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.130477.log.gz 2023-08-09T15:13:03.331 INFO:teuthology.orchestra.run.smithi043.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.104919.log 2023-08-09T15:13:03.332 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.118780.log: gzip 88.5% -5 -- replaced with /var/log/ceph/ceph-client.admin.118780.log.gz --verbose 2023-08-09T15:13:03.332 INFO:teuthology.orchestra.run.smithi043.stderr: -- /var/log/ceph/ceph-client.admin.103749.log 2023-08-09T15:13:03.333 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.104919.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.96648.log 2023-08-09T15:13:03.333 INFO:teuthology.orchestra.run.smithi043.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.104919.log.gz 2023-08-09T15:13:03.334 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.103749.log: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.103749.log.gz 2023-08-09T15:13:03.335 INFO:teuthology.orchestra.run.smithi043.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.101551.log 2023-08-09T15:13:03.336 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.96648.log: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.96648.log.gz 2023-08-09T15:13:03.336 INFO:teuthology.orchestra.run.smithi043.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.111115.log 2023-08-09T15:13:03.337 INFO:teuthology.orchestra.run.smithi203.stderr: 91.9%gzip -5 --verbose -- /var/log/ceph/ceph-client.3.95318.log 2023-08-09T15:13:03.338 INFO:teuthology.orchestra.run.smithi203.stderr:/var/log/ceph/ceph-client.admin.89185.log: 87.1% -- replaced with /var/log/ceph/ceph-client.admin.89185.log.gz 2023-08-09T15:13:03.338 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.101551.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.107754.log 2023-08-09T15:13:03.338 INFO:teuthology.orchestra.run.smithi043.stderr: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.101551.log.gz 2023-08-09T15:13:03.339 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.111115.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.111115.log.gz 2023-08-09T15:13:03.339 INFO:teuthology.orchestra.run.smithi043.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.157663.log 2023-08-09T15:13:03.340 INFO:teuthology.orchestra.run.smithi203.stderr: -- replaced with /var/log/ceph/ceph-client.1.94798.log.gz 2023-08-09T15:13:03.340 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.107754.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.145660.log 2023-08-09T15:13:03.340 INFO:teuthology.orchestra.run.smithi043.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.107754.log.gz 2023-08-09T15:13:03.341 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.157663.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.92948.log 2023-08-09T15:13:03.341 INFO:teuthology.orchestra.run.smithi043.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.157663.log.gz 2023-08-09T15:13:03.342 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.145660.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.145660.log.gz 2023-08-09T15:13:03.343 INFO:teuthology.orchestra.run.smithi043.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.94030.log 2023-08-09T15:13:03.344 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.92948.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.103273.log 2023-08-09T15:13:03.344 INFO:teuthology.orchestra.run.smithi043.stderr: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.92948.log.gz 2023-08-09T15:13:03.345 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.94030.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.99244.log 2023-08-09T15:13:03.346 INFO:teuthology.orchestra.run.smithi043.stderr: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.94030.log.gz 2023-08-09T15:13:03.347 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.103273.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.103273.log.gz 2023-08-09T15:13:03.347 INFO:teuthology.orchestra.run.smithi043.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.117583.log 2023-08-09T15:13:03.348 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.99244.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.96307.log 2023-08-09T15:13:03.349 INFO:teuthology.orchestra.run.smithi043.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.99244.log.gz 2023-08-09T15:13:03.350 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.117583.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.117583.log.gzgzip 2023-08-09T15:13:03.350 INFO:teuthology.orchestra.run.smithi043.stderr: -5 --verbose -- /var/log/ceph/ceph-client.admin.147378.log 2023-08-09T15:13:03.351 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.96307.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.96307.log.gz 2023-08-09T15:13:03.351 INFO:teuthology.orchestra.run.smithi043.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.118630.log 2023-08-09T15:13:03.352 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.147378.log: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.147378.log.gz 2023-08-09T15:13:03.352 INFO:teuthology.orchestra.run.smithi043.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.99832.log 2023-08-09T15:13:03.353 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.118630.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.97784.log 2023-08-09T15:13:03.354 INFO:teuthology.orchestra.run.smithi043.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.118630.log.gz 2023-08-09T15:13:03.355 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.99832.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.104685.log 2023-08-09T15:13:03.355 INFO:teuthology.orchestra.run.smithi043.stderr: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.99832.log.gz 2023-08-09T15:13:03.356 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.97784.log: gzip -5 --verbose 88.7% -- -- replaced with /var/log/ceph/ceph-client.admin.97784.log.gz /var/log/ceph/ceph-client.admin.138600.log 2023-08-09T15:13:03.356 INFO:teuthology.orchestra.run.smithi043.stderr: 2023-08-09T15:13:03.357 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.104685.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.139281.log 2023-08-09T15:13:03.357 INFO:teuthology.orchestra.run.smithi043.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.104685.log.gz 2023-08-09T15:13:03.358 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.138600.log: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.138600.log.gz 2023-08-09T15:13:03.359 INFO:teuthology.orchestra.run.smithi043.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.127069.log 2023-08-09T15:13:03.360 INFO:teuthology.orchestra.run.smithi203.stderr:/var/log/ceph/ceph-client.3.95318.log: 93.0% 92.3% -- replaced with /var/log/ceph/ceph-mds.d.log.gz 2023-08-09T15:13:03.360 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.139281.log: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.139281.log.gz 2023-08-09T15:13:03.360 INFO:teuthology.orchestra.run.smithi043.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.143039.log 2023-08-09T15:13:03.361 INFO:teuthology.orchestra.run.smithi203.stderr: -- replaced with /var/log/ceph/ceph-client.3.95318.log.gz 2023-08-09T15:13:03.362 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.127069.log: gzip 88.8% -5 -- replaced with /var/log/ceph/ceph-client.admin.127069.log.gz --verbose 2023-08-09T15:13:03.362 INFO:teuthology.orchestra.run.smithi043.stderr: -- /var/log/ceph/ceph-client.admin.97656.log 2023-08-09T15:13:03.362 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.143039.log: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.143039.log.gz 2023-08-09T15:13:03.363 INFO:teuthology.orchestra.run.smithi043.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.107695.log 2023-08-09T15:13:03.364 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.97656.log: gzip -5 --verbose -- /var/log/ceph/ceph-osd.0.log 2023-08-09T15:13:03.365 INFO:teuthology.orchestra.run.smithi043.stderr: 90.9% -- replaced with /var/log/ceph/ceph-client.admin.97656.log.gz 2023-08-09T15:13:03.365 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.107695.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.148701.log 2023-08-09T15:13:03.366 INFO:teuthology.orchestra.run.smithi043.stderr: 87.0% -- replaced with /var/log/ceph/ceph-client.admin.107695.log.gz 2023-08-09T15:13:03.367 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-osd.0.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.125955.log 2023-08-09T15:13:03.368 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.148701.log: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.148701.log.gz 2023-08-09T15:13:03.369 INFO:teuthology.orchestra.run.smithi043.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.123570.log 2023-08-09T15:13:03.370 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.125955.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.155416.log 2023-08-09T15:13:03.370 INFO:teuthology.orchestra.run.smithi043.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.125955.log.gz 2023-08-09T15:13:03.371 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.123570.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.120974.log 2023-08-09T15:13:03.371 INFO:teuthology.orchestra.run.smithi043.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.123570.log.gz 2023-08-09T15:13:03.372 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.155416.log: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.155416.log.gz 2023-08-09T15:13:03.373 INFO:teuthology.orchestra.run.smithi043.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.109891.log 2023-08-09T15:13:03.374 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.120974.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.107273.log 2023-08-09T15:13:03.374 INFO:teuthology.orchestra.run.smithi043.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.120974.log.gz 2023-08-09T15:13:03.375 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.109891.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.145082.log 2023-08-09T15:13:03.375 INFO:teuthology.orchestra.run.smithi043.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.109891.log.gz 2023-08-09T15:13:03.376 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.107273.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.107273.log.gz 2023-08-09T15:13:03.377 INFO:teuthology.orchestra.run.smithi043.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.115783.log 2023-08-09T15:13:03.378 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.145082.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.105760.log 2023-08-09T15:13:03.378 INFO:teuthology.orchestra.run.smithi043.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.145082.log.gz 2023-08-09T15:13:03.380 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.115783.log: gzip 88.7% -- replaced with /var/log/ceph/ceph-client.admin.115783.log.gz -5 2023-08-09T15:13:03.380 INFO:teuthology.orchestra.run.smithi043.stderr: --verbose -- /var/log/ceph/ceph-client.admin.140516.log 2023-08-09T15:13:03.380 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.105760.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.131853.log 2023-08-09T15:13:03.381 INFO:teuthology.orchestra.run.smithi043.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.105760.log.gz 2023-08-09T15:13:03.382 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.140516.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.140516.log.gz 2023-08-09T15:13:03.382 INFO:teuthology.orchestra.run.smithi043.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.140462.log 2023-08-09T15:13:03.383 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.131853.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.103831.log 2023-08-09T15:13:03.383 INFO:teuthology.orchestra.run.smithi043.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.131853.log.gz 2023-08-09T15:13:03.384 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.140462.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.140462.log.gz 2023-08-09T15:13:03.385 INFO:teuthology.orchestra.run.smithi043.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.152243.log 2023-08-09T15:13:03.386 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.103831.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.106383.log 2023-08-09T15:13:03.386 INFO:teuthology.orchestra.run.smithi043.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.103831.log.gz 2023-08-09T15:13:03.387 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.152243.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.152243.log.gz 2023-08-09T15:13:03.387 INFO:teuthology.orchestra.run.smithi043.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.95124.log 2023-08-09T15:13:03.388 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.106383.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.131246.log 2023-08-09T15:13:03.389 INFO:teuthology.orchestra.run.smithi043.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.106383.log.gz 2023-08-09T15:13:03.390 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.95124.log: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.95124.log.gz 2023-08-09T15:13:03.390 INFO:teuthology.orchestra.run.smithi043.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.128014.log 2023-08-09T15:13:03.391 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.131246.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.157394.log 2023-08-09T15:13:03.391 INFO:teuthology.orchestra.run.smithi043.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.131246.log.gz 2023-08-09T15:13:03.392 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.128014.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.128014.log.gz 2023-08-09T15:13:03.393 INFO:teuthology.orchestra.run.smithi043.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.145618.log 2023-08-09T15:13:03.393 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.157394.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.151063.log 2023-08-09T15:13:03.394 INFO:teuthology.orchestra.run.smithi043.stderr: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.157394.log.gz 2023-08-09T15:13:03.395 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.145618.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.145618.log.gz 2023-08-09T15:13:03.395 INFO:teuthology.orchestra.run.smithi043.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.125818.log 2023-08-09T15:13:03.396 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.151063.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.152754.log 2023-08-09T15:13:03.396 INFO:teuthology.orchestra.run.smithi043.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.151063.log.gz 2023-08-09T15:13:03.398 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.125818.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.125818.log.gz 2023-08-09T15:13:03.398 INFO:teuthology.orchestra.run.smithi043.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.156042.log 2023-08-09T15:13:03.399 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.152754.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.102229.log 2023-08-09T15:13:03.400 INFO:teuthology.orchestra.run.smithi043.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.152754.log.gz 2023-08-09T15:13:03.401 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.156042.log: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.156042.log.gz 2023-08-09T15:13:03.401 INFO:teuthology.orchestra.run.smithi043.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.107063.log 2023-08-09T15:13:03.402 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.102229.log: gzip -5 --verbose -- /var/log/ceph/ceph-osd.1.log 2023-08-09T15:13:03.402 INFO:teuthology.orchestra.run.smithi043.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.102229.log.gz 2023-08-09T15:13:03.403 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.107063.log: 88.8%gzip -- replaced with /var/log/ceph/ceph-client.admin.107063.log.gz -5 2023-08-09T15:13:03.404 INFO:teuthology.orchestra.run.smithi043.stderr: --verbose -- /var/log/ceph/ceph-client.admin.106509.log 2023-08-09T15:13:03.405 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-osd.1.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.132278.log 2023-08-09T15:13:03.406 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.106509.log: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.106509.log.gz 2023-08-09T15:13:03.406 INFO:teuthology.orchestra.run.smithi043.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.157016.log 2023-08-09T15:13:03.408 INFO:teuthology.orchestra.run.smithi043.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.112841.log 2023-08-09T15:13:03.409 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.132278.log: /var/log/ceph/ceph-client.admin.157016.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.132278.log.gz 2023-08-09T15:13:03.409 INFO:teuthology.orchestra.run.smithi043.stderr:gzip 88.0% -5 -- replaced with /var/log/ceph/ceph-client.admin.157016.log.gz 2023-08-09T15:13:03.410 INFO:teuthology.orchestra.run.smithi043.stderr: --verbose -- /var/log/ceph/ceph-client.admin.138684.log 2023-08-09T15:13:03.410 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.112841.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.100783.log 2023-08-09T15:13:03.411 INFO:teuthology.orchestra.run.smithi043.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.112841.log.gz 2023-08-09T15:13:03.411 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.138684.log: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.138684.log.gz 2023-08-09T15:13:03.412 INFO:teuthology.orchestra.run.smithi043.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.112883.log 2023-08-09T15:13:03.413 INFO:teuthology.orchestra.run.smithi043.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.141303.log 2023-08-09T15:13:03.415 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.100783.log: /var/log/ceph/ceph-client.admin.112883.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.100783.log.gz 2023-08-09T15:13:03.415 INFO:teuthology.orchestra.run.smithi043.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.112883.log.gz 2023-08-09T15:13:03.415 INFO:teuthology.orchestra.run.smithi043.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.97614.log 2023-08-09T15:13:03.416 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.141303.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.138726.log 2023-08-09T15:13:03.416 INFO:teuthology.orchestra.run.smithi043.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.141303.log.gz 2023-08-09T15:13:03.417 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.97614.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.97614.log.gz 2023-08-09T15:13:03.418 INFO:teuthology.orchestra.run.smithi043.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.152934.log 2023-08-09T15:13:03.419 INFO:teuthology.orchestra.run.smithi043.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.100294.log 2023-08-09T15:13:03.420 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.138726.log: /var/log/ceph/ceph-client.admin.152934.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.152934.log.gz 2023-08-09T15:13:03.421 INFO:teuthology.orchestra.run.smithi043.stderr: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.138726.log.gz 2023-08-09T15:13:03.422 INFO:teuthology.orchestra.run.smithi043.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.99076.log 2023-08-09T15:13:03.423 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.100294.log: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.100294.log.gz 2023-08-09T15:13:03.424 INFO:teuthology.orchestra.run.smithi043.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.126234.log 2023-08-09T15:13:03.425 INFO:teuthology.orchestra.run.smithi043.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.147336.log 2023-08-09T15:13:03.426 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.99076.log: /var/log/ceph/ceph-client.admin.126234.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.99076.log.gz 2023-08-09T15:13:03.427 INFO:teuthology.orchestra.run.smithi043.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.126234.log.gz 2023-08-09T15:13:03.427 INFO:teuthology.orchestra.run.smithi043.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.146575.log 2023-08-09T15:13:03.428 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.147336.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.94982.log 2023-08-09T15:13:03.428 INFO:teuthology.orchestra.run.smithi043.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.147336.log.gz 2023-08-09T15:13:03.429 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.146575.log: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.146575.log.gz 2023-08-09T15:13:03.429 INFO:teuthology.orchestra.run.smithi043.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.153856.log 2023-08-09T15:13:03.430 INFO:teuthology.orchestra.run.smithi043.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.112501.log 2023-08-09T15:13:03.431 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.94982.log: /var/log/ceph/ceph-client.admin.153856.log: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.94982.log.gz 2023-08-09T15:13:03.432 INFO:teuthology.orchestra.run.smithi043.stderr: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.153856.log.gz 2023-08-09T15:13:03.432 INFO:teuthology.orchestra.run.smithi043.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.116146.log 2023-08-09T15:13:03.433 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.112501.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.109102.log 2023-08-09T15:13:03.434 INFO:teuthology.orchestra.run.smithi043.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.112501.log.gz 2023-08-09T15:13:03.435 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.116146.log: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.116146.log.gz 2023-08-09T15:13:03.435 INFO:teuthology.orchestra.run.smithi043.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.138474.log 2023-08-09T15:13:03.436 INFO:teuthology.orchestra.run.smithi043.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.103357.log 2023-08-09T15:13:03.437 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.109102.log: /var/log/ceph/ceph-client.admin.138474.log: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.109102.log.gz 2023-08-09T15:13:03.438 INFO:teuthology.orchestra.run.smithi043.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.138474.log.gz 2023-08-09T15:13:03.438 INFO:teuthology.orchestra.run.smithi043.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.146370.log 2023-08-09T15:13:03.439 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.103357.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.95250.log 2023-08-09T15:13:03.439 INFO:teuthology.orchestra.run.smithi043.stderr: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.103357.log.gz 2023-08-09T15:13:03.440 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.146370.log: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.146370.log.gz 2023-08-09T15:13:03.441 INFO:teuthology.orchestra.run.smithi043.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.102727.log 2023-08-09T15:13:03.442 INFO:teuthology.orchestra.run.smithi043.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.128884.log 2023-08-09T15:13:03.442 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.95250.log: /var/log/ceph/ceph-client.admin.102727.log: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.95250.log.gz 2023-08-09T15:13:03.443 INFO:teuthology.orchestra.run.smithi043.stderr: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.102727.log.gz 2023-08-09T15:13:03.444 INFO:teuthology.orchestra.run.smithi043.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.110867.log 2023-08-09T15:13:03.444 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.128884.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.127668.log 2023-08-09T15:13:03.445 INFO:teuthology.orchestra.run.smithi043.stderr: 87.2% -- replaced with /var/log/ceph/ceph-client.admin.128884.log.gz 2023-08-09T15:13:03.445 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.110867.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.110867.log.gz 2023-08-09T15:13:03.446 INFO:teuthology.orchestra.run.smithi043.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.127804.log 2023-08-09T15:13:03.447 INFO:teuthology.orchestra.run.smithi043.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.120117.log 2023-08-09T15:13:03.448 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.127668.log: /var/log/ceph/ceph-client.admin.127804.log: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.127668.log.gz 2023-08-09T15:13:03.449 INFO:teuthology.orchestra.run.smithi043.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.127804.log.gz 2023-08-09T15:13:03.449 INFO:teuthology.orchestra.run.smithi043.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.116524.log 2023-08-09T15:13:03.450 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.120117.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.154288.log 2023-08-09T15:13:03.450 INFO:teuthology.orchestra.run.smithi043.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.120117.log.gz 2023-08-09T15:13:03.451 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.116524.log: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.116524.log.gz 2023-08-09T15:13:03.451 INFO:teuthology.orchestra.run.smithi043.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.142131.log 2023-08-09T15:13:03.452 INFO:teuthology.orchestra.run.smithi043.stderr:gzip -5 --verbose --/var/log/ceph/ceph-client.admin.154288.log: /var/log/ceph/ceph-client.0.125126.log 2023-08-09T15:13:03.453 INFO:teuthology.orchestra.run.smithi043.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.154288.log.gz 2023-08-09T15:13:03.454 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.142131.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.142131.log.gz 2023-08-09T15:13:03.454 INFO:teuthology.orchestra.run.smithi043.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.111241.log 2023-08-09T15:13:03.455 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.0.125126.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.131475.log 2023-08-09T15:13:03.457 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.111241.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.111241.log.gz 2023-08-09T15:13:03.457 INFO:teuthology.orchestra.run.smithi043.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.119053.log 2023-08-09T15:13:03.458 INFO:teuthology.orchestra.run.smithi043.stderr: 94.2% -- replaced with /var/log/ceph/ceph-client.0.125126.log.gz 2023-08-09T15:13:03.458 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.131475.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.123665.log 2023-08-09T15:13:03.459 INFO:teuthology.orchestra.run.smithi043.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.131475.log.gz 2023-08-09T15:13:03.459 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.119053.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.119053.log.gz 2023-08-09T15:13:03.460 INFO:teuthology.orchestra.run.smithi043.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.142565.log 2023-08-09T15:13:03.461 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.123665.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.123825.log 2023-08-09T15:13:03.461 INFO:teuthology.orchestra.run.smithi043.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.123665.log.gz 2023-08-09T15:13:03.462 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.142565.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.142565.log.gz 2023-08-09T15:13:03.463 INFO:teuthology.orchestra.run.smithi043.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.123867.log 2023-08-09T15:13:03.464 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.123825.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.119553.log 2023-08-09T15:13:03.464 INFO:teuthology.orchestra.run.smithi043.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.123825.log.gz 2023-08-09T15:13:03.465 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.123867.log: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.123867.log.gz 2023-08-09T15:13:03.466 INFO:teuthology.orchestra.run.smithi043.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.122080.log 2023-08-09T15:13:03.467 INFO:teuthology.orchestra.run.smithi203.stderr: 93.9% -- replaced with /var/log/ceph/ceph-mds.b.log.gz 2023-08-09T15:13:03.468 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.119553.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.146617.log 2023-08-09T15:13:03.468 INFO:teuthology.orchestra.run.smithi043.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.119553.log.gz 2023-08-09T15:13:03.469 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.122080.log: 87.0% -- replaced with /var/log/ceph/ceph-client.admin.122080.log.gz 2023-08-09T15:13:03.469 INFO:teuthology.orchestra.run.smithi043.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.93384.log 2023-08-09T15:13:03.470 INFO:teuthology.orchestra.run.smithi043.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.152668.log 2023-08-09T15:13:03.470 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.146617.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.146617.log.gz 2023-08-09T15:13:03.472 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.93384.log: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.93384.log.gz 2023-08-09T15:13:03.472 INFO:teuthology.orchestra.run.smithi043.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.127580.log 2023-08-09T15:13:03.473 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.152668.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.118908.log 2023-08-09T15:13:03.473 INFO:teuthology.orchestra.run.smithi043.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.152668.log.gz 2023-08-09T15:13:03.474 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.127580.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.127580.log.gz 2023-08-09T15:13:03.474 INFO:teuthology.orchestra.run.smithi043.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.89525.log 2023-08-09T15:13:03.474 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.118908.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.94814.log 2023-08-09T15:13:03.475 INFO:teuthology.orchestra.run.smithi043.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.118908.log.gz 2023-08-09T15:13:03.476 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.89525.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.89525.log.gz 2023-08-09T15:13:03.476 INFO:teuthology.orchestra.run.smithi043.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.107105.log 2023-08-09T15:13:03.477 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.94814.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.145124.log 2023-08-09T15:13:03.478 INFO:teuthology.orchestra.run.smithi043.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.94814.log.gz 2023-08-09T15:13:03.479 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.107105.log: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.107105.log.gz 2023-08-09T15:13:03.479 INFO:teuthology.orchestra.run.smithi043.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.100993.log 2023-08-09T15:13:03.480 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.145124.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.117119.log 2023-08-09T15:13:03.480 INFO:teuthology.orchestra.run.smithi043.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.145124.log.gz 2023-08-09T15:13:03.481 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.100993.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.100993.log.gz 2023-08-09T15:13:03.482 INFO:teuthology.orchestra.run.smithi043.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.112416.log 2023-08-09T15:13:03.483 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.117119.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.147006.log 2023-08-09T15:13:03.483 INFO:teuthology.orchestra.run.smithi043.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.117119.log.gz 2023-08-09T15:13:03.484 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.112416.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.112416.log.gz 2023-08-09T15:13:03.484 INFO:teuthology.orchestra.run.smithi043.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.155313.log 2023-08-09T15:13:03.485 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.147006.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.120583.log 2023-08-09T15:13:03.486 INFO:teuthology.orchestra.run.smithi043.stderr: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.147006.log.gz 2023-08-09T15:13:03.486 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.155313.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.155313.log.gz 2023-08-09T15:13:03.487 INFO:teuthology.orchestra.run.smithi043.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.95983.log 2023-08-09T15:13:03.488 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.120583.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.93520.log 2023-08-09T15:13:03.488 INFO:teuthology.orchestra.run.smithi043.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.120583.log.gz 2023-08-09T15:13:03.489 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.95983.log: 86.8% -- replaced with /var/log/ceph/ceph-client.admin.95983.log.gz 2023-08-09T15:13:03.490 INFO:teuthology.orchestra.run.smithi043.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.140295.log 2023-08-09T15:13:03.491 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.93520.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.154120.log 2023-08-09T15:13:03.491 INFO:teuthology.orchestra.run.smithi043.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.93520.log.gz 2023-08-09T15:13:03.492 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.140295.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.140295.log.gz 2023-08-09T15:13:03.492 INFO:teuthology.orchestra.run.smithi043.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.127760.log 2023-08-09T15:13:03.493 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.154120.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.101299.log 2023-08-09T15:13:03.494 INFO:teuthology.orchestra.run.smithi043.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.154120.log.gz 2023-08-09T15:13:03.495 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.127760.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.127760.log.gz 2023-08-09T15:13:03.495 INFO:teuthology.orchestra.run.smithi043.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.152892.log 2023-08-09T15:13:03.496 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.101299.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.112799.log 2023-08-09T15:13:03.496 INFO:teuthology.orchestra.run.smithi043.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.101299.log.gz 2023-08-09T15:13:03.497 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.152892.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.152892.log.gz 2023-08-09T15:13:03.498 INFO:teuthology.orchestra.run.smithi043.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.137741.log 2023-08-09T15:13:03.499 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.112799.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.101131.log 2023-08-09T15:13:03.499 INFO:teuthology.orchestra.run.smithi043.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.112799.log.gz 2023-08-09T15:13:03.500 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.137741.log: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.137741.log.gz 2023-08-09T15:13:03.500 INFO:teuthology.orchestra.run.smithi043.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.117201.log 2023-08-09T15:13:03.501 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.101131.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.107890.log 2023-08-09T15:13:03.502 INFO:teuthology.orchestra.run.smithi043.stderr: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.101131.log.gz 2023-08-09T15:13:03.503 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.117201.log: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.117201.log.gz 2023-08-09T15:13:03.503 INFO:teuthology.orchestra.run.smithi043.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.97488.log 2023-08-09T15:13:03.504 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.107890.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.93221.log 2023-08-09T15:13:03.504 INFO:teuthology.orchestra.run.smithi043.stderr: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.107890.log.gz 2023-08-09T15:13:03.505 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.97488.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.97488.log.gz 2023-08-09T15:13:03.506 INFO:teuthology.orchestra.run.smithi043.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.117285.log 2023-08-09T15:13:03.507 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.93221.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.104293.log 2023-08-09T15:13:03.507 INFO:teuthology.orchestra.run.smithi043.stderr: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.93221.log.gz 2023-08-09T15:13:03.508 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.117285.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.117285.log.gz 2023-08-09T15:13:03.509 INFO:teuthology.orchestra.run.smithi043.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.99496.log 2023-08-09T15:13:03.510 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.104293.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.156764.log 2023-08-09T15:13:03.510 INFO:teuthology.orchestra.run.smithi043.stderr: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.104293.log.gz 2023-08-09T15:13:03.511 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.99496.log: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.99496.log.gz 2023-08-09T15:13:03.512 INFO:teuthology.orchestra.run.smithi043.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.105537.log 2023-08-09T15:13:03.513 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.156764.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.116230.log 2023-08-09T15:13:03.513 INFO:teuthology.orchestra.run.smithi043.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.156764.log.gz 2023-08-09T15:13:03.514 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.105537.log: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.105537.log.gz 2023-08-09T15:13:03.514 INFO:teuthology.orchestra.run.smithi043.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.139669.log 2023-08-09T15:13:03.515 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.116230.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.100825.log 2023-08-09T15:13:03.516 INFO:teuthology.orchestra.run.smithi043.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.116230.log.gz 2023-08-09T15:13:03.517 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.139669.log: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.139669.log.gz 2023-08-09T15:13:03.517 INFO:teuthology.orchestra.run.smithi043.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.96320.log 2023-08-09T15:13:03.518 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.100825.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.102601.log 2023-08-09T15:13:03.519 INFO:teuthology.orchestra.run.smithi043.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.100825.log.gz 2023-08-09T15:13:03.519 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.96320.log: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.96320.log.gz 2023-08-09T15:13:03.520 INFO:teuthology.orchestra.run.smithi043.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.111637.log 2023-08-09T15:13:03.521 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.102601.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.151991.log 2023-08-09T15:13:03.522 INFO:teuthology.orchestra.run.smithi043.stderr: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.102601.log.gz 2023-08-09T15:13:03.522 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.111637.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.111637.log.gz 2023-08-09T15:13:03.523 INFO:teuthology.orchestra.run.smithi043.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.121930.log 2023-08-09T15:13:03.524 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.151991.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.142829.log 2023-08-09T15:13:03.524 INFO:teuthology.orchestra.run.smithi043.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.151991.log.gz 2023-08-09T15:13:03.525 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.121930.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.121930.log.gz 2023-08-09T15:13:03.526 INFO:teuthology.orchestra.run.smithi043.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.121144.log 2023-08-09T15:13:03.526 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.142829.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.155595.log 2023-08-09T15:13:03.527 INFO:teuthology.orchestra.run.smithi043.stderr: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.142829.log.gz 2023-08-09T15:13:03.528 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.121144.log: 88.8%gzip -- replaced with /var/log/ceph/ceph-client.admin.121144.log.gz -5 2023-08-09T15:13:03.528 INFO:teuthology.orchestra.run.smithi043.stderr: --verbose -- /var/log/ceph/ceph-client.admin.102061.log 2023-08-09T15:13:03.529 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.155595.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.155595.log.gz 2023-08-09T15:13:03.530 INFO:teuthology.orchestra.run.smithi043.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.126192.log 2023-08-09T15:13:03.531 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.102061.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.95040.log 2023-08-09T15:13:03.531 INFO:teuthology.orchestra.run.smithi043.stderr: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.102061.log.gz 2023-08-09T15:13:03.532 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.126192.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.126192.log.gz 2023-08-09T15:13:03.533 INFO:teuthology.orchestra.run.smithi043.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.111824.log 2023-08-09T15:13:03.534 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.95040.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.95040.log.gz 2023-08-09T15:13:03.534 INFO:teuthology.orchestra.run.smithi043.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.147466.log 2023-08-09T15:13:03.535 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.111824.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.132712.log 2023-08-09T15:13:03.536 INFO:teuthology.orchestra.run.smithi043.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.111824.log.gz 2023-08-09T15:13:03.537 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.147466.log: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.147466.log.gz 2023-08-09T15:13:03.537 INFO:teuthology.orchestra.run.smithi043.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.141219.log 2023-08-09T15:13:03.538 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.132712.log: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.132712.log.gz 2023-08-09T15:13:03.539 INFO:teuthology.orchestra.run.smithi043.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.97026.log 2023-08-09T15:13:03.540 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.141219.log: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.141219.log.gz 2023-08-09T15:13:03.540 INFO:teuthology.orchestra.run.smithi043.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.101467.log 2023-08-09T15:13:03.541 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.97026.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.121186.log 2023-08-09T15:13:03.542 INFO:teuthology.orchestra.run.smithi043.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.97026.log.gz 2023-08-09T15:13:03.543 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.101467.log: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.101467.log.gz 2023-08-09T15:13:03.543 INFO:teuthology.orchestra.run.smithi043.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.144754.log 2023-08-09T15:13:03.544 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.121186.log: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.121186.log.gz 2023-08-09T15:13:03.545 INFO:teuthology.orchestra.run.smithi043.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.124345.log 2023-08-09T15:13:03.546 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.144754.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.116566.log 2023-08-09T15:13:03.546 INFO:teuthology.orchestra.run.smithi043.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.144754.log.gz 2023-08-09T15:13:03.547 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.124345.log: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.124345.log.gz 2023-08-09T15:13:03.548 INFO:teuthology.orchestra.run.smithi043.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.132152.log 2023-08-09T15:13:03.549 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.116566.log: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.116566.log.gz 2023-08-09T15:13:03.549 INFO:teuthology.orchestra.run.smithi043.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.151510.log 2023-08-09T15:13:03.550 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.132152.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.137630.log 2023-08-09T15:13:03.551 INFO:teuthology.orchestra.run.smithi043.stderr: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.132152.log.gz 2023-08-09T15:13:03.552 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.151510.log: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.151510.log.gz 2023-08-09T15:13:03.552 INFO:teuthology.orchestra.run.smithi043.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.106114.log 2023-08-09T15:13:03.553 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.137630.log: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.137630.log.gz 2023-08-09T15:13:03.554 INFO:teuthology.orchestra.run.smithi043.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.132500.log 2023-08-09T15:13:03.555 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.106114.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.139711.log 2023-08-09T15:13:03.555 INFO:teuthology.orchestra.run.smithi043.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.106114.log.gz 2023-08-09T15:13:03.556 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.132500.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.132500.log.gz 2023-08-09T15:13:03.557 INFO:teuthology.orchestra.run.smithi043.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.95472.log 2023-08-09T15:13:03.558 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.139711.log: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.139711.log.gz 2023-08-09T15:13:03.558 INFO:teuthology.orchestra.run.smithi043.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.100084.log 2023-08-09T15:13:03.559 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.95472.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.152338.log 2023-08-09T15:13:03.560 INFO:teuthology.orchestra.run.smithi043.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.95472.log.gz 2023-08-09T15:13:03.561 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.100084.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.100084.log.gz 2023-08-09T15:13:03.561 INFO:teuthology.orchestra.run.smithi043.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.124615.log 2023-08-09T15:13:03.562 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.152338.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.152338.log.gz 2023-08-09T15:13:03.563 INFO:teuthology.orchestra.run.smithi043.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.112374.log 2023-08-09T15:13:03.564 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.124615.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.123995.log 2023-08-09T15:13:03.564 INFO:teuthology.orchestra.run.smithi043.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.124615.log.gz 2023-08-09T15:13:03.565 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.112374.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.112374.log.gz 2023-08-09T15:13:03.566 INFO:teuthology.orchestra.run.smithi043.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.130865.log 2023-08-09T15:13:03.567 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.123995.log: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.123995.log.gz 2023-08-09T15:13:03.568 INFO:teuthology.orchestra.run.smithi043.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-mds.a.log 2023-08-09T15:13:03.569 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.130865.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.130865.log.gz 2023-08-09T15:13:03.569 INFO:teuthology.orchestra.run.smithi043.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.106215.log 2023-08-09T15:13:03.571 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-mds.a.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.95208.log 2023-08-09T15:13:03.572 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.106215.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.138901.log 2023-08-09T15:13:03.573 INFO:teuthology.orchestra.run.smithi043.stderr: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.106215.log.gz 2023-08-09T15:13:03.574 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.95208.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.104517.log 2023-08-09T15:13:03.574 INFO:teuthology.orchestra.run.smithi043.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.95208.log.gz 2023-08-09T15:13:03.575 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.138901.log: gzip 87.5% -- replaced with /var/log/ceph/ceph-client.admin.138901.log.gz 2023-08-09T15:13:03.576 INFO:teuthology.orchestra.run.smithi043.stderr: -5 --verbose -- /var/log/ceph/ceph-client.admin.117039.log 2023-08-09T15:13:03.577 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.104517.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.102313.log 2023-08-09T15:13:03.577 INFO:teuthology.orchestra.run.smithi043.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.104517.log.gz 2023-08-09T15:13:03.579 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.117039.log: gzip -5 --verbose -- 88.5% /var/log/ceph/ceph-client.0.148280.log -- replaced with /var/log/ceph/ceph-client.admin.117039.log.gz 2023-08-09T15:13:03.579 INFO:teuthology.orchestra.run.smithi043.stderr: 2023-08-09T15:13:03.580 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.102313.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.93478.log 2023-08-09T15:13:03.581 INFO:teuthology.orchestra.run.smithi043.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.102313.log.gz 2023-08-09T15:13:03.582 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.0.148280.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.126444.log 2023-08-09T15:13:03.584 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.93478.log: gzip 88.4% -- replaced with /var/log/ceph/ceph-client.admin.93478.log.gz 2023-08-09T15:13:03.584 INFO:teuthology.orchestra.run.smithi043.stderr: -5 --verbose -- /var/log/ceph/ceph-client.admin.116692.log 2023-08-09T15:13:03.585 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.126444.log: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.126444.log.gz 2023-08-09T15:13:03.586 INFO:teuthology.orchestra.run.smithi043.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.122432.log 2023-08-09T15:13:03.587 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.116692.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.116692.log.gz 2023-08-09T15:13:03.588 INFO:teuthology.orchestra.run.smithi043.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.102895.log 2023-08-09T15:13:03.589 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.122432.log: 87.5%gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.127368.log 2023-08-09T15:13:03.589 INFO:teuthology.orchestra.run.smithi043.stderr: -- replaced with /var/log/ceph/ceph-client.admin.122432.log.gz 2023-08-09T15:13:03.591 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.102895.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.138982.log 2023-08-09T15:13:03.591 INFO:teuthology.orchestra.run.smithi043.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.102895.log.gz 2023-08-09T15:13:03.593 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.127368.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.112213.log 2023-08-09T15:13:03.593 INFO:teuthology.orchestra.run.smithi043.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.127368.log.gz 2023-08-09T15:13:03.594 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.138982.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.127718.log 2023-08-09T15:13:03.594 INFO:teuthology.orchestra.run.smithi043.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.138982.log.gz 2023-08-09T15:13:03.596 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.112213.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.127624.log 2023-08-09T15:13:03.596 INFO:teuthology.orchestra.run.smithi043.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.112213.log.gz 2023-08-09T15:13:03.597 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.127718.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.94772.log 2023-08-09T15:13:03.597 INFO:teuthology.orchestra.run.smithi043.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.127718.log.gz 2023-08-09T15:13:03.599 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.127624.log: 88.4%gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.155637.log 2023-08-09T15:13:03.599 INFO:teuthology.orchestra.run.smithi043.stderr: -- replaced with /var/log/ceph/ceph-client.admin.127624.log.gz 2023-08-09T15:13:03.601 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.94772.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.116734.log 2023-08-09T15:13:03.601 INFO:teuthology.orchestra.run.smithi043.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.94772.log.gz 2023-08-09T15:13:03.602 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.155637.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.152710.log 2023-08-09T15:13:03.602 INFO:teuthology.orchestra.run.smithi043.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.155637.log.gz 2023-08-09T15:13:03.603 INFO:teuthology.orchestra.run.smithi043.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.148004.log 2023-08-09T15:13:03.605 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.116734.log: /var/log/ceph/ceph-client.admin.152710.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.127410.log 2023-08-09T15:13:03.605 INFO:teuthology.orchestra.run.smithi043.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.152710.log.gz 2023-08-09T15:13:03.606 INFO:teuthology.orchestra.run.smithi043.stderr: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.116734.log.gz 2023-08-09T15:13:03.606 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.148004.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.148004.log.gz 2023-08-09T15:13:03.607 INFO:teuthology.orchestra.run.smithi043.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.131517.log 2023-08-09T15:13:03.608 INFO:teuthology.orchestra.run.smithi043.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.99580.log 2023-08-09T15:13:03.610 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.127410.log: /var/log/ceph/ceph-client.admin.131517.log: 89.4%gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.157184.log 2023-08-09T15:13:03.610 INFO:teuthology.orchestra.run.smithi043.stderr: -- replaced with /var/log/ceph/ceph-client.admin.127410.log.gz 2023-08-09T15:13:03.610 INFO:teuthology.orchestra.run.smithi043.stderr: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.131517.log.gz 2023-08-09T15:13:03.611 INFO:teuthology.orchestra.run.smithi043.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.98906.log 2023-08-09T15:13:03.613 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.99580.log: /var/log/ceph/ceph-client.admin.157184.log: 88.7%gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.151426.log 2023-08-09T15:13:03.613 INFO:teuthology.orchestra.run.smithi043.stderr: 86.8% -- replaced with /var/log/ceph/ceph-client.admin.157184.log.gz 2023-08-09T15:13:03.613 INFO:teuthology.orchestra.run.smithi043.stderr: -- replaced with /var/log/ceph/ceph-client.admin.99580.log.gz 2023-08-09T15:13:03.615 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.98906.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.156722.log 2023-08-09T15:13:03.615 INFO:teuthology.orchestra.run.smithi043.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.98906.log.gz 2023-08-09T15:13:03.616 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.151426.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.119343.log 2023-08-09T15:13:03.617 INFO:teuthology.orchestra.run.smithi043.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.151426.log.gz 2023-08-09T15:13:03.618 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.156722.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.131769.log 2023-08-09T15:13:03.618 INFO:teuthology.orchestra.run.smithi043.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.156722.log.gz 2023-08-09T15:13:03.619 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.119343.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.122795.log 2023-08-09T15:13:03.620 INFO:teuthology.orchestra.run.smithi043.stderr: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.119343.log.gz 2023-08-09T15:13:03.621 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.131769.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.111992.log 2023-08-09T15:13:03.621 INFO:teuthology.orchestra.run.smithi043.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.131769.log.gz 2023-08-09T15:13:03.622 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.122795.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.147422.log 2023-08-09T15:13:03.623 INFO:teuthology.orchestra.run.smithi043.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.122795.log.gz 2023-08-09T15:13:03.624 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.111992.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.101257.log 2023-08-09T15:13:03.624 INFO:teuthology.orchestra.run.smithi043.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.111992.log.gz 2023-08-09T15:13:03.626 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.147422.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.105937.log 2023-08-09T15:13:03.626 INFO:teuthology.orchestra.run.smithi043.stderr: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.147422.log.gz 2023-08-09T15:13:03.627 INFO:teuthology.orchestra.run.smithi043.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.140253.log 2023-08-09T15:13:03.629 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.101257.log: /var/log/ceph/ceph-client.admin.105937.log: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.101257.log.gz 2023-08-09T15:13:03.629 INFO:teuthology.orchestra.run.smithi043.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.95082.log 2023-08-09T15:13:03.630 INFO:teuthology.orchestra.run.smithi043.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.105937.log.gz 2023-08-09T15:13:03.631 INFO:teuthology.orchestra.run.smithi043.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.101173.log 2023-08-09T15:13:03.632 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.140253.log: /var/log/ceph/ceph-client.admin.95082.log: 88.1%gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.93650.log 2023-08-09T15:13:03.632 INFO:teuthology.orchestra.run.smithi043.stderr: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.95082.log.gz 2023-08-09T15:13:03.633 INFO:teuthology.orchestra.run.smithi043.stderr: -- replaced with /var/log/ceph/ceph-client.admin.140253.log.gz 2023-08-09T15:13:03.634 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.101173.log: gzip 88.6% -- replaced with /var/log/ceph/ceph-client.admin.101173.log.gz -5 2023-08-09T15:13:03.634 INFO:teuthology.orchestra.run.smithi043.stderr: --verbose -- /var/log/ceph/ceph-client.admin.155202.log 2023-08-09T15:13:03.635 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.93650.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.102271.log 2023-08-09T15:13:03.635 INFO:teuthology.orchestra.run.smithi043.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.93650.log.gz 2023-08-09T15:13:03.637 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.155202.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.130431.log 2023-08-09T15:13:03.637 INFO:teuthology.orchestra.run.smithi043.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.155202.log.gz 2023-08-09T15:13:03.638 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.102271.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.153206.log 2023-08-09T15:13:03.638 INFO:teuthology.orchestra.run.smithi043.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.102271.log.gz 2023-08-09T15:13:03.640 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.130431.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.126276.log 2023-08-09T15:13:03.640 INFO:teuthology.orchestra.run.smithi043.stderr: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.130431.log.gz 2023-08-09T15:13:03.641 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.153206.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.117937.log 2023-08-09T15:13:03.641 INFO:teuthology.orchestra.run.smithi043.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.153206.log.gz 2023-08-09T15:13:03.643 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.126276.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.136276.log 2023-08-09T15:13:03.643 INFO:teuthology.orchestra.run.smithi043.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.126276.log.gz 2023-08-09T15:13:03.644 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.117937.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.0.98188.log 2023-08-09T15:13:03.644 INFO:teuthology.orchestra.run.smithi043.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.117937.log.gz 2023-08-09T15:13:03.646 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.136276.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.156168.log 2023-08-09T15:13:03.646 INFO:teuthology.orchestra.run.smithi043.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.136276.log.gz 2023-08-09T15:13:03.647 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.0.98188.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.106467.log 2023-08-09T15:13:03.648 INFO:teuthology.orchestra.run.smithi043.stderr: 92.3% -- replaced with /var/log/ceph/ceph-client.0.98188.log.gz/var/log/ceph/ceph-client.admin.156168.log: 2023-08-09T15:13:03.649 INFO:teuthology.orchestra.run.smithi043.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.154472.log 2023-08-09T15:13:03.649 INFO:teuthology.orchestra.run.smithi043.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.156168.log.gz 2023-08-09T15:13:03.651 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.106467.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.152626.log 2023-08-09T15:13:03.651 INFO:teuthology.orchestra.run.smithi043.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.106467.log.gz 2023-08-09T15:13:03.653 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.154472.log: 89.0%gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.157478.log 2023-08-09T15:13:03.653 INFO:teuthology.orchestra.run.smithi043.stderr: -- replaced with /var/log/ceph/ceph-client.admin.154472.log.gz 2023-08-09T15:13:03.654 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.152626.log: gzip 88.6% -5 -- replaced with /var/log/ceph/ceph-client.admin.152626.log.gz --verbose 2023-08-09T15:13:03.654 INFO:teuthology.orchestra.run.smithi043.stderr: -- /var/log/ceph/ceph-client.admin.104769.log 2023-08-09T15:13:03.655 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.157478.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.153981.log 2023-08-09T15:13:03.656 INFO:teuthology.orchestra.run.smithi043.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.157478.log.gz 2023-08-09T15:13:03.657 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.104769.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.122132.log 2023-08-09T15:13:03.657 INFO:teuthology.orchestra.run.smithi043.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.104769.log.gz 2023-08-09T15:13:03.658 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.153981.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.99286.log 2023-08-09T15:13:03.659 INFO:teuthology.orchestra.run.smithi043.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.153981.log.gz 2023-08-09T15:13:03.660 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.122132.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.99916.log 2023-08-09T15:13:03.660 INFO:teuthology.orchestra.run.smithi043.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.122132.log.gz 2023-08-09T15:13:03.662 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.99286.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.0.153632.log 2023-08-09T15:13:03.662 INFO:teuthology.orchestra.run.smithi043.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.99286.log.gz 2023-08-09T15:13:03.663 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.99916.log: 87.6%gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.139193.log 2023-08-09T15:13:03.663 INFO:teuthology.orchestra.run.smithi043.stderr: -- replaced with /var/log/ceph/ceph-client.admin.99916.log.gz 2023-08-09T15:13:03.665 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.0.153632.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.131390.log 2023-08-09T15:13:03.666 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.139193.log: 88.7% 91.7% -- replaced with /var/log/ceph/ceph-client.0.153632.log.gzgzip 2023-08-09T15:13:03.667 INFO:teuthology.orchestra.run.smithi043.stderr: -5 --verbose -- /var/log/ceph/ceph-client.admin.141910.log 2023-08-09T15:13:03.667 INFO:teuthology.orchestra.run.smithi043.stderr: -- replaced with /var/log/ceph/ceph-client.admin.139193.log.gz 2023-08-09T15:13:03.668 INFO:teuthology.orchestra.run.smithi043.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.141742.log 2023-08-09T15:13:03.670 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.131390.log: /var/log/ceph/ceph-client.admin.141910.log: 87.9%gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.126733.log 2023-08-09T15:13:03.670 INFO:teuthology.orchestra.run.smithi043.stderr: -- replaced with /var/log/ceph/ceph-client.admin.131390.log.gz 2023-08-09T15:13:03.670 INFO:teuthology.orchestra.run.smithi043.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.141910.log.gz 2023-08-09T15:13:03.671 INFO:teuthology.orchestra.run.smithi043.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.95974.log 2023-08-09T15:13:03.673 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.141742.log: /var/log/ceph/ceph-client.admin.126733.log: 88.8%gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.116104.log 2023-08-09T15:13:03.673 INFO:teuthology.orchestra.run.smithi043.stderr: -- replaced with /var/log/ceph/ceph-client.admin.141742.log.gz 2023-08-09T15:13:03.673 INFO:teuthology.orchestra.run.smithi043.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.126733.log.gz 2023-08-09T15:13:03.674 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.95974.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.112757.log 2023-08-09T15:13:03.675 INFO:teuthology.orchestra.run.smithi043.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.95974.log.gz 2023-08-09T15:13:03.675 INFO:teuthology.orchestra.run.smithi043.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.156355.log 2023-08-09T15:13:03.677 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.116104.log: /var/log/ceph/ceph-client.admin.112757.log: 88.6%gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.143714.log 2023-08-09T15:13:03.678 INFO:teuthology.orchestra.run.smithi043.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.112757.log.gz 2023-08-09T15:13:03.678 INFO:teuthology.orchestra.run.smithi043.stderr: -- replaced with /var/log/ceph/ceph-client.admin.116104.log.gz 2023-08-09T15:13:03.679 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.156355.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.120710.log 2023-08-09T15:13:03.679 INFO:teuthology.orchestra.run.smithi043.stderr: 87.1% -- replaced with /var/log/ceph/ceph-client.admin.156355.log.gz 2023-08-09T15:13:03.681 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.143714.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.143790.log 2023-08-09T15:13:03.681 INFO:teuthology.orchestra.run.smithi043.stderr: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.143714.log.gz 2023-08-09T15:13:03.682 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.120710.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.103105.log 2023-08-09T15:13:03.683 INFO:teuthology.orchestra.run.smithi043.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.120710.log.gz 2023-08-09T15:13:03.684 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.143790.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.101877.log 2023-08-09T15:13:03.684 INFO:teuthology.orchestra.run.smithi043.stderr: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.143790.log.gz 2023-08-09T15:13:03.685 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.103105.log: 87.4%gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.155748.log 2023-08-09T15:13:03.686 INFO:teuthology.orchestra.run.smithi043.stderr: -- replaced with /var/log/ceph/ceph-client.admin.103105.log.gz 2023-08-09T15:13:03.687 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.101877.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.115646.log 2023-08-09T15:13:03.687 INFO:teuthology.orchestra.run.smithi043.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.101877.log.gz 2023-08-09T15:13:03.689 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.155748.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.139543.log 2023-08-09T15:13:03.689 INFO:teuthology.orchestra.run.smithi043.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.155748.log.gz 2023-08-09T15:13:03.689 INFO:teuthology.orchestra.run.smithi043.stderr: 94.2% -- replaced with /var/log/ceph/ceph-client.0.148280.log.gz 2023-08-09T15:13:03.690 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.115646.log: gzip -5 --verbose 89.2% -- /var/log/ceph/ceph-client.admin.127111.log 2023-08-09T15:13:03.690 INFO:teuthology.orchestra.run.smithi043.stderr: -- replaced with /var/log/ceph/ceph-client.admin.115646.log.gz 2023-08-09T15:13:03.691 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.139543.log: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.139543.log.gz 2023-08-09T15:13:03.691 INFO:teuthology.orchestra.run.smithi043.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.128056.log 2023-08-09T15:13:03.693 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.127111.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.121980.log 2023-08-09T15:13:03.693 INFO:teuthology.orchestra.run.smithi043.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.127111.log.gz 2023-08-09T15:13:03.694 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.128056.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.101509.log 2023-08-09T15:13:03.694 INFO:teuthology.orchestra.run.smithi043.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.128056.log.gz 2023-08-09T15:13:03.696 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.121980.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.119637.log 2023-08-09T15:13:03.696 INFO:teuthology.orchestra.run.smithi043.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.121980.log.gz 2023-08-09T15:13:03.697 INFO:teuthology.orchestra.run.smithi043.stderr:gzip/var/log/ceph/ceph-client.admin.101509.log: -5 --verbose -- /var/log/ceph/ceph-client.admin.146202.log 2023-08-09T15:13:03.698 INFO:teuthology.orchestra.run.smithi043.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.101509.log.gz 2023-08-09T15:13:03.699 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.119637.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.111451.log 2023-08-09T15:13:03.699 INFO:teuthology.orchestra.run.smithi043.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.119637.log.gz 2023-08-09T15:13:03.700 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.146202.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.121312.log 2023-08-09T15:13:03.701 INFO:teuthology.orchestra.run.smithi043.stderr: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.146202.log.gz 2023-08-09T15:13:03.702 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.111451.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.144796.log 2023-08-09T15:13:03.702 INFO:teuthology.orchestra.run.smithi043.stderr: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.111451.log.gz 2023-08-09T15:13:03.703 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.121312.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.121354.log 2023-08-09T15:13:03.704 INFO:teuthology.orchestra.run.smithi043.stderr: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.121312.log.gz 2023-08-09T15:13:03.705 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.144796.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.108008.log 2023-08-09T15:13:03.705 INFO:teuthology.orchestra.run.smithi043.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.144796.log.gz 2023-08-09T15:13:03.706 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.121354.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.124303.log 2023-08-09T15:13:03.707 INFO:teuthology.orchestra.run.smithi043.stderr: 91.4% -- replaced with /var/log/ceph/ceph-client.admin.121354.log.gz 2023-08-09T15:13:03.708 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.108008.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.102811.log 2023-08-09T15:13:03.708 INFO:teuthology.orchestra.run.smithi043.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.108008.log.gz 2023-08-09T15:13:03.709 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.124303.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.117709.log 2023-08-09T15:13:03.710 INFO:teuthology.orchestra.run.smithi043.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.124303.log.gz 2023-08-09T15:13:03.711 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.102811.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.130907.log 2023-08-09T15:13:03.712 INFO:teuthology.orchestra.run.smithi043.stderr: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.102811.log.gz 2023-08-09T15:13:03.712 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.117709.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.112629.log 2023-08-09T15:13:03.713 INFO:teuthology.orchestra.run.smithi043.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.117709.log.gz 2023-08-09T15:13:03.714 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.130907.log: 88.5%gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.104811.log 2023-08-09T15:13:03.714 INFO:teuthology.orchestra.run.smithi043.stderr: -- replaced with /var/log/ceph/ceph-client.admin.130907.log.gz 2023-08-09T15:13:03.715 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.112629.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.106341.log 2023-08-09T15:13:03.716 INFO:teuthology.orchestra.run.smithi043.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.112629.log.gz 2023-08-09T15:13:03.717 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.104811.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.119144.log 2023-08-09T15:13:03.717 INFO:teuthology.orchestra.run.smithi043.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.104811.log.gz 2023-08-09T15:13:03.718 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.106341.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.107813.log 2023-08-09T15:13:03.719 INFO:teuthology.orchestra.run.smithi043.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.106341.log.gz 2023-08-09T15:13:03.720 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.119144.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.107577.log 2023-08-09T15:13:03.720 INFO:teuthology.orchestra.run.smithi043.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.119144.log.gz 2023-08-09T15:13:03.721 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.107813.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.147602.log 2023-08-09T15:13:03.722 INFO:teuthology.orchestra.run.smithi043.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.107813.log.gz 2023-08-09T15:13:03.723 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.107577.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.147812.log 2023-08-09T15:13:03.723 INFO:teuthology.orchestra.run.smithi043.stderr: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.107577.log.gz 2023-08-09T15:13:03.725 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.147602.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.103483.log 2023-08-09T15:13:03.725 INFO:teuthology.orchestra.run.smithi043.stderr: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.147602.log.gz 2023-08-09T15:13:03.726 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.147812.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.96900.log 2023-08-09T15:13:03.726 INFO:teuthology.orchestra.run.smithi043.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.147812.log.gz 2023-08-09T15:13:03.727 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.103483.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.142745.log 2023-08-09T15:13:03.728 INFO:teuthology.orchestra.run.smithi043.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.103483.log.gz 2023-08-09T15:13:03.729 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.96900.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.126943.log 2023-08-09T15:13:03.729 INFO:teuthology.orchestra.run.smithi043.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.96900.log.gz 2023-08-09T15:13:03.731 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.142745.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.116818.log 2023-08-09T15:13:03.731 INFO:teuthology.orchestra.run.smithi043.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.142745.log.gz 2023-08-09T15:13:03.732 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.126943.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.155832.log 2023-08-09T15:13:03.732 INFO:teuthology.orchestra.run.smithi043.stderr: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.126943.log.gz 2023-08-09T15:13:03.733 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.116818.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.120502.log 2023-08-09T15:13:03.734 INFO:teuthology.orchestra.run.smithi043.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.116818.log.gz 2023-08-09T15:13:03.735 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.155832.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.98344.log 2023-08-09T15:13:03.735 INFO:teuthology.orchestra.run.smithi043.stderr: 87.2% -- replaced with /var/log/ceph/ceph-client.admin.155832.log.gz 2023-08-09T15:13:03.736 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.120502.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.113819.log 2023-08-09T15:13:03.737 INFO:teuthology.orchestra.run.smithi043.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.120502.log.gz 2023-08-09T15:13:03.738 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.98344.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.137951.log 2023-08-09T15:13:03.738 INFO:teuthology.orchestra.run.smithi043.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.98344.log.gz 2023-08-09T15:13:03.740 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.113819.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.106593.log 2023-08-09T15:13:03.740 INFO:teuthology.orchestra.run.smithi043.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.113819.log.gz 2023-08-09T15:13:03.741 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.137951.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.111698.log 2023-08-09T15:13:03.741 INFO:teuthology.orchestra.run.smithi043.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.137951.log.gz 2023-08-09T15:13:03.742 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.106593.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.95514.log 2023-08-09T15:13:03.743 INFO:teuthology.orchestra.run.smithi043.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.106593.log.gz 2023-08-09T15:13:03.744 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.111698.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.119511.log 2023-08-09T15:13:03.744 INFO:teuthology.orchestra.run.smithi043.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.111698.log.gz 2023-08-09T15:13:03.745 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.95514.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.132236.log 2023-08-09T15:13:03.746 INFO:teuthology.orchestra.run.smithi043.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.95514.log.gz 2023-08-09T15:13:03.747 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.119511.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.129865.log 2023-08-09T15:13:03.747 INFO:teuthology.orchestra.run.smithi043.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.119511.log.gz 2023-08-09T15:13:03.748 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.132236.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.137993.log 2023-08-09T15:13:03.749 INFO:teuthology.orchestra.run.smithi043.stderr: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.132236.log.gz 2023-08-09T15:13:03.750 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.129865.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.151147.log 2023-08-09T15:13:03.750 INFO:teuthology.orchestra.run.smithi043.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.129865.log.gz 2023-08-09T15:13:03.751 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.137993.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.103525.log 2023-08-09T15:13:03.752 INFO:teuthology.orchestra.run.smithi043.stderr: 87.1% -- replaced with /var/log/ceph/ceph-client.admin.137993.log.gz 2023-08-09T15:13:03.753 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.151147.log: gzip 88.5% -- replaced with /var/log/ceph/ceph-client.admin.151147.log.gz 2023-08-09T15:13:03.753 INFO:teuthology.orchestra.run.smithi043.stderr: -5 --verbose -- /var/log/ceph/ceph-client.admin.100515.log 2023-08-09T15:13:03.754 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.103525.log: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.103525.log.gz 2023-08-09T15:13:03.755 INFO:teuthology.orchestra.run.smithi043.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.100909.log 2023-08-09T15:13:03.756 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.100515.log: gzip 88.7% -- replaced with /var/log/ceph/ceph-client.admin.100515.log.gz 2023-08-09T15:13:03.757 INFO:teuthology.orchestra.run.smithi043.stderr: -5 --verbose -- /var/log/ceph/ceph-client.admin.125357.log 2023-08-09T15:13:03.758 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.100909.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.100909.log.gz 2023-08-09T15:13:03.758 INFO:teuthology.orchestra.run.smithi043.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.125860.log 2023-08-09T15:13:03.760 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.125357.log: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.125357.log.gz 2023-08-09T15:13:03.760 INFO:teuthology.orchestra.run.smithi043.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.146785.log 2023-08-09T15:13:03.761 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.125860.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.103915.log 2023-08-09T15:13:03.762 INFO:teuthology.orchestra.run.smithi043.stderr: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.125860.log.gz 2023-08-09T15:13:03.762 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.146785.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.103957.log 2023-08-09T15:13:03.763 INFO:teuthology.orchestra.run.smithi043.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.146785.log.gz 2023-08-09T15:13:03.764 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.103915.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.128664.log 2023-08-09T15:13:03.764 INFO:teuthology.orchestra.run.smithi043.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.103915.log.gz 2023-08-09T15:13:03.767 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.103957.log: gzip -5 --verbose 89.7% -- replaced with /var/log/ceph/ceph-client.admin.103957.log.gz 2023-08-09T15:13:03.767 INFO:teuthology.orchestra.run.smithi043.stderr: -- /var/log/ceph/ceph-client.admin.137867.log 2023-08-09T15:13:03.768 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.128664.log: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.128664.log.gz 2023-08-09T15:13:03.768 INFO:teuthology.orchestra.run.smithi043.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.124573.log 2023-08-09T15:13:03.769 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.137867.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.150823.log 2023-08-09T15:13:03.769 INFO:teuthology.orchestra.run.smithi043.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.137867.log.gz 2023-08-09T15:13:03.771 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.124573.log: gzip -5 --verbose -- 89.1% /var/log/ceph/ceph-client.admin.139627.log 2023-08-09T15:13:03.771 INFO:teuthology.orchestra.run.smithi043.stderr: -- replaced with /var/log/ceph/ceph-client.admin.124573.log.gz 2023-08-09T15:13:03.772 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.150823.log: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.150823.log.gz 2023-08-09T15:13:03.772 INFO:teuthology.orchestra.run.smithi043.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.95292.log 2023-08-09T15:13:03.773 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.139627.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.93904.log 2023-08-09T15:13:03.774 INFO:teuthology.orchestra.run.smithi043.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.139627.log.gz 2023-08-09T15:13:03.775 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.95292.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.96984.log 2023-08-09T15:13:03.775 INFO:teuthology.orchestra.run.smithi043.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.95292.log.gz 2023-08-09T15:13:03.777 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.93904.log: gzip -5 --verbose 88.5% -- /var/log/ceph/ceph-client.admin.155958.log 2023-08-09T15:13:03.777 INFO:teuthology.orchestra.run.smithi043.stderr: -- replaced with /var/log/ceph/ceph-client.admin.93904.log.gz 2023-08-09T15:13:03.778 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.96984.log: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.96984.log.gz 2023-08-09T15:13:03.778 INFO:teuthology.orchestra.run.smithi043.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.99454.log 2023-08-09T15:13:03.780 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.155958.log: gzip -5 --verbose 88.0% -- replaced with /var/log/ceph/ceph-client.admin.155958.log.gz 2023-08-09T15:13:03.780 INFO:teuthology.orchestra.run.smithi043.stderr: -- /var/log/ceph/ceph-client.admin.119763.log 2023-08-09T15:13:03.781 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.99454.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.99454.log.gz 2023-08-09T15:13:03.781 INFO:teuthology.orchestra.run.smithi043.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.131811.log 2023-08-09T15:13:03.783 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.119763.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.119763.log.gz 2023-08-09T15:13:03.783 INFO:teuthology.orchestra.run.smithi043.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.152499.log 2023-08-09T15:13:03.784 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.131811.log: gzip -5 --verbose -- 89.1% -- replaced with /var/log/ceph/ceph-client.admin.131811.log.gz 2023-08-09T15:13:03.785 INFO:teuthology.orchestra.run.smithi043.stderr: /var/log/ceph/ceph-client.admin.126528.log 2023-08-09T15:13:03.786 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.152499.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.152499.log.gz 2023-08-09T15:13:03.786 INFO:teuthology.orchestra.run.smithi043.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.100210.log 2023-08-09T15:13:03.788 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.126528.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.95556.log 2023-08-09T15:13:03.788 INFO:teuthology.orchestra.run.smithi043.stderr: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.126528.log.gz 2023-08-09T15:13:03.789 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.100210.log: gzip -5 88.5% --verbose -- /var/log/ceph/ceph-client.admin.119427.log 2023-08-09T15:13:03.789 INFO:teuthology.orchestra.run.smithi043.stderr: -- replaced with /var/log/ceph/ceph-client.admin.100210.log.gz 2023-08-09T15:13:03.791 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.95556.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.95556.log.gz 2023-08-09T15:13:03.791 INFO:teuthology.orchestra.run.smithi043.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.124261.log 2023-08-09T15:13:03.792 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.119427.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.106299.log 2023-08-09T15:13:03.792 INFO:teuthology.orchestra.run.smithi043.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.119427.log.gz 2023-08-09T15:13:03.794 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.124261.log: gzip -5 --verbose 88.9% -- replaced with /var/log/ceph/ceph-client.admin.124261.log.gz 2023-08-09T15:13:03.794 INFO:teuthology.orchestra.run.smithi043.stderr: -- /var/log/ceph/ceph-client.admin.146034.log 2023-08-09T15:13:03.795 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.106299.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.106299.log.gz 2023-08-09T15:13:03.796 INFO:teuthology.orchestra.run.smithi043.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.100683.log 2023-08-09T15:13:03.797 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.146034.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.95388.log 2023-08-09T15:13:03.797 INFO:teuthology.orchestra.run.smithi043.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.146034.log.gz 2023-08-09T15:13:03.799 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.100683.log: gzip -5 88.1% -- replaced with /var/log/ceph/ceph-client.admin.100683.log.gz 2023-08-09T15:13:03.799 INFO:teuthology.orchestra.run.smithi043.stderr: --verbose -- /var/log/ceph/ceph-client.admin.97068.log 2023-08-09T15:13:03.800 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.95388.log: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.95388.log.gz 2023-08-09T15:13:03.800 INFO:teuthology.orchestra.run.smithi043.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.146448.log 2023-08-09T15:13:03.802 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.97068.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.142437.log 2023-08-09T15:13:03.802 INFO:teuthology.orchestra.run.smithi043.stderr: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.97068.log.gz 2023-08-09T15:13:03.804 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.146448.log: gzip -5 --verbose 87.7% -- replaced with /var/log/ceph/ceph-client.admin.146448.log.gz 2023-08-09T15:13:03.804 INFO:teuthology.orchestra.run.smithi043.stderr: -- /var/log/ceph/ceph-client.admin.133642.log 2023-08-09T15:13:03.809 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.142437.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.142437.log.gz 2023-08-09T15:13:03.809 INFO:teuthology.orchestra.run.smithi043.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.154692.log 2023-08-09T15:13:03.809 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.133642.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.132458.log 2023-08-09T15:13:03.810 INFO:teuthology.orchestra.run.smithi043.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.133642.log.gz 2023-08-09T15:13:03.810 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.154692.log: gzip 88.3% -5 --verbose -- /var/log/ceph/ceph-client.admin.102433.log 2023-08-09T15:13:03.810 INFO:teuthology.orchestra.run.smithi043.stderr: -- replaced with /var/log/ceph/ceph-client.admin.154692.log.gz 2023-08-09T15:13:03.811 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.132458.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.132458.log.gz 2023-08-09T15:13:03.811 INFO:teuthology.orchestra.run.smithi043.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.95989.log 2023-08-09T15:13:03.812 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.102433.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.93862.log 2023-08-09T15:13:03.812 INFO:teuthology.orchestra.run.smithi043.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.102433.log.gz 2023-08-09T15:13:03.813 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.95989.log: gzip 87.9% -- replaced with /var/log/ceph/ceph-client.admin.95989.log.gz 2023-08-09T15:13:03.813 INFO:teuthology.orchestra.run.smithi043.stderr: -5 --verbose -- /var/log/ceph/ceph-client.admin.146701.log 2023-08-09T15:13:03.814 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.93862.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.93862.log.gz 2023-08-09T15:13:03.814 INFO:teuthology.orchestra.run.smithi043.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.117895.log 2023-08-09T15:13:03.816 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.146701.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.119805.log 2023-08-09T15:13:03.816 INFO:teuthology.orchestra.run.smithi043.stderr: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.146701.log.gz 2023-08-09T15:13:03.817 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.117895.log: gzip 87.6% -- replaced with /var/log/ceph/ceph-client.admin.117895.log.gz 2023-08-09T15:13:03.818 INFO:teuthology.orchestra.run.smithi043.stderr: -5 --verbose -- /var/log/ceph/ceph-client.admin.143081.log 2023-08-09T15:13:03.819 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.119805.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.119805.log.gz 2023-08-09T15:13:03.819 INFO:teuthology.orchestra.run.smithi043.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.146286.log 2023-08-09T15:13:03.821 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.143081.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.107357.log 2023-08-09T15:13:03.821 INFO:teuthology.orchestra.run.smithi043.stderr: 90.7% -- replaced with /var/log/ceph/ceph-client.admin.143081.log.gz 2023-08-09T15:13:03.822 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.146286.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.146286.log.gz 2023-08-09T15:13:03.822 INFO:teuthology.orchestra.run.smithi043.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.100725.log 2023-08-09T15:13:03.824 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.107357.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.107357.log.gz 2023-08-09T15:13:03.824 INFO:teuthology.orchestra.run.smithi043.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.136234.log 2023-08-09T15:13:03.826 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.100725.log: gzip 88.6% -- replaced with /var/log/ceph/ceph-client.admin.100725.log.gz 2023-08-09T15:13:03.826 INFO:teuthology.orchestra.run.smithi043.stderr: -5 --verbose -- /var/log/ceph/ceph-client.admin.103021.log 2023-08-09T15:13:03.827 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.136234.log: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.136234.log.gz 2023-08-09T15:13:03.827 INFO:teuthology.orchestra.run.smithi043.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.104125.log 2023-08-09T15:13:03.829 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.103021.log: 88.8%gzip -- replaced with /var/log/ceph/ceph-client.admin.103021.log.gz -5 2023-08-09T15:13:03.829 INFO:teuthology.orchestra.run.smithi043.stderr: --verbose -- /var/log/ceph/ceph-client.admin.145023.log 2023-08-09T15:13:03.831 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.104125.log: gzip -5 89.4% -- replaced with /var/log/ceph/ceph-client.admin.104125.log.gz 2023-08-09T15:13:03.831 INFO:teuthology.orchestra.run.smithi043.stderr: --verbose -- /var/log/ceph/ceph-client.0.113478.log 2023-08-09T15:13:03.832 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.145023.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.145023.log.gz 2023-08-09T15:13:03.832 INFO:teuthology.orchestra.run.smithi043.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.124037.log 2023-08-09T15:13:03.834 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.0.113478.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.107949.log 2023-08-09T15:13:03.836 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.124037.log: gzip -5 87.0% -- replaced with /var/log/ceph/ceph-client.admin.124037.log.gz 2023-08-09T15:13:03.836 INFO:teuthology.orchestra.run.smithi043.stderr: --verbose -- /var/log/ceph/ceph-client.admin.156848.log 2023-08-09T15:13:03.837 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.107949.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.107949.log.gz 2023-08-09T15:13:03.837 INFO:teuthology.orchestra.run.smithi043.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.124217.log 2023-08-09T15:13:03.839 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.156848.log: gzip -5 88.7% -- replaced with /var/log/ceph/ceph-client.admin.156848.log.gz 2023-08-09T15:13:03.839 INFO:teuthology.orchestra.run.smithi043.stderr: --verbose -- /var/log/ceph/ceph-client.admin.95973.log 2023-08-09T15:13:03.840 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.124217.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.124217.log.gz 2023-08-09T15:13:03.841 INFO:teuthology.orchestra.run.smithi043.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.132628.log 2023-08-09T15:13:03.843 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.95973.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.113009.log 88.3% -- replaced with /var/log/ceph/ceph-client.admin.95973.log.gz 2023-08-09T15:13:03.843 INFO:teuthology.orchestra.run.smithi043.stderr: 2023-08-09T15:13:03.844 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.132628.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.132628.log.gz 2023-08-09T15:13:03.845 INFO:teuthology.orchestra.run.smithi043.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.133530.log 2023-08-09T15:13:03.847 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.113009.log: gzip 88.7% -- replaced with /var/log/ceph/ceph-client.admin.113009.log.gz 2023-08-09T15:13:03.847 INFO:teuthology.orchestra.run.smithi043.stderr: -5 --verbose -- /var/log/ceph/ceph-client.admin.138306.log 2023-08-09T15:13:03.848 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.133530.log: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.133530.log.gz 2023-08-09T15:13:03.848 INFO:teuthology.orchestra.run.smithi043.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.110962.log 2023-08-09T15:13:03.850 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.138306.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.127930.log 2023-08-09T15:13:03.850 INFO:teuthology.orchestra.run.smithi043.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.138306.log.gz 2023-08-09T15:13:03.851 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.110962.log: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.110962.log.gz 2023-08-09T15:13:03.851 INFO:teuthology.orchestra.run.smithi043.stderr:gzip -5 --verbose -- /var/log/ceph/ceph.audit.log 2023-08-09T15:13:03.852 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.127930.log: gzip -5 88.4% -- replaced with /var/log/ceph/ceph-client.admin.127930.log.gz 2023-08-09T15:13:03.853 INFO:teuthology.orchestra.run.smithi043.stderr: --verbose -- /var/log/ceph/ceph-client.admin.126686.log 2023-08-09T15:13:03.854 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph.audit.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.138516.log 2023-08-09T15:13:03.856 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.126686.log: 87.1% -- replaced with /var/log/ceph/ceph-client.admin.126686.log.gz 2023-08-09T15:13:03.856 INFO:teuthology.orchestra.run.smithi043.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.119595.log 2023-08-09T15:13:03.857 INFO:teuthology.orchestra.run.smithi043.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.141616.log 2023-08-09T15:13:03.859 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.138516.log: /var/log/ceph/ceph-client.admin.119595.log: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.138516.log.gz 2023-08-09T15:13:03.859 INFO:teuthology.orchestra.run.smithi043.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.119595.log.gz 2023-08-09T15:13:03.860 INFO:teuthology.orchestra.run.smithi043.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.117751.log 2023-08-09T15:13:03.860 INFO:teuthology.orchestra.run.smithi043.stderr: 90.0% -- replaced with /var/log/ceph/ceph.audit.log.gz 2023-08-09T15:13:03.861 INFO:teuthology.orchestra.run.smithi043.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.139151.log 2023-08-09T15:13:03.863 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.141616.log: /var/log/ceph/ceph-client.admin.117751.log: 89.0%gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.120422.log 2023-08-09T15:13:03.864 INFO:teuthology.orchestra.run.smithi043.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.117751.log.gz 2023-08-09T15:13:03.864 INFO:teuthology.orchestra.run.smithi043.stderr: -- replaced with /var/log/ceph/ceph-client.admin.141616.log.gz 2023-08-09T15:13:03.864 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.139151.log: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.139151.log.gz 2023-08-09T15:13:03.865 INFO:teuthology.orchestra.run.smithi043.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.150926.log 2023-08-09T15:13:03.866 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.120422.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.154034.log 2023-08-09T15:13:03.867 INFO:teuthology.orchestra.run.smithi043.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.120422.log.gz 2023-08-09T15:13:03.868 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.150926.log: 87.8%gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.95808.log 2023-08-09T15:13:03.868 INFO:teuthology.orchestra.run.smithi043.stderr: -- replaced with /var/log/ceph/ceph-client.admin.150926.log.gz 2023-08-09T15:13:03.870 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.154034.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.139067.log 2023-08-09T15:13:03.870 INFO:teuthology.orchestra.run.smithi043.stderr: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.154034.log.gz 2023-08-09T15:13:03.871 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.95808.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.156313.log 2023-08-09T15:13:03.871 INFO:teuthology.orchestra.run.smithi043.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.95808.log.gz 2023-08-09T15:13:03.872 INFO:teuthology.orchestra.run.smithi043.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.93004.log 2023-08-09T15:13:03.874 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.139067.log: /var/log/ceph/ceph-client.admin.156313.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.139067.log.gzgzip -5 --verbose -- /var/log/ceph/ceph-client.admin.148462.log 2023-08-09T15:13:03.874 INFO:teuthology.orchestra.run.smithi043.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.156313.log.gz 2023-08-09T15:13:03.874 INFO:teuthology.orchestra.run.smithi043.stderr: 2023-08-09T15:13:03.875 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.93004.log: gzip 88.3% -5 --verbose -- /var/log/ceph/ceph-client.admin.106173.log 2023-08-09T15:13:03.875 INFO:teuthology.orchestra.run.smithi043.stderr: -- replaced with /var/log/ceph/ceph-client.admin.93004.log.gz 2023-08-09T15:13:03.876 INFO:teuthology.orchestra.run.smithi043.stderr:gzip -5 --verbose/var/log/ceph/ceph-client.admin.148462.log: -- /var/log/ceph/ceph-client.admin.157310.log 2023-08-09T15:13:03.877 INFO:teuthology.orchestra.run.smithi043.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.148462.log.gz 2023-08-09T15:13:03.878 INFO:teuthology.orchestra.run.smithi043.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.127454.log 2023-08-09T15:13:03.879 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.106173.log: 88.1%/var/log/ceph/ceph-client.admin.157310.log: -- replaced with /var/log/ceph/ceph-client.admin.106173.log.gz 2023-08-09T15:13:03.880 INFO:teuthology.orchestra.run.smithi043.stderr: 87.6%gzip -- replaced with /var/log/ceph/ceph-client.admin.157310.log.gz 2023-08-09T15:13:03.880 INFO:teuthology.orchestra.run.smithi043.stderr: -5 --verbose -- /var/log/ceph/ceph-client.admin.96586.log 2023-08-09T15:13:03.882 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.127454.log: gzip 87.4% -- replaced with /var/log/ceph/ceph-client.admin.127454.log.gz -5 2023-08-09T15:13:03.882 INFO:teuthology.orchestra.run.smithi043.stderr: --verbose -- /var/log/ceph/ceph-client.admin.126402.log 2023-08-09T15:13:03.883 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.96586.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.96586.log.gz 2023-08-09T15:13:03.883 INFO:teuthology.orchestra.run.smithi043.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.103873.log 2023-08-09T15:13:03.885 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.126402.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.147938.log 2023-08-09T15:13:03.885 INFO:teuthology.orchestra.run.smithi043.stderr: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.126402.log.gz 2023-08-09T15:13:03.886 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.103873.log: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.103873.log.gz 2023-08-09T15:13:03.886 INFO:teuthology.orchestra.run.smithi043.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.96858.log 2023-08-09T15:13:03.888 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.147938.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.156523.log 2023-08-09T15:13:03.888 INFO:teuthology.orchestra.run.smithi043.stderr: 91.0% -- replaced with /var/log/ceph/ceph-client.admin.147938.log.gz 2023-08-09T15:13:03.889 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.96858.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.103669.log 2023-08-09T15:13:03.890 INFO:teuthology.orchestra.run.smithi043.stderr: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.96858.log.gz 2023-08-09T15:13:03.891 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.156523.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.137475.log 2023-08-09T15:13:03.891 INFO:teuthology.orchestra.run.smithi043.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.156523.log.gz 2023-08-09T15:13:03.892 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.103669.log: gzip 88.9% -- replaced with /var/log/ceph/ceph-client.admin.103669.log.gz 2023-08-09T15:13:03.892 INFO:teuthology.orchestra.run.smithi043.stderr: -5 --verbose -- /var/log/ceph/ceph-client.admin.147252.log 2023-08-09T15:13:03.893 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.137475.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.137475.log.gz 2023-08-09T15:13:03.894 INFO:teuthology.orchestra.run.smithi043.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.130204.log 2023-08-09T15:13:03.896 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.147252.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.101961.log 2023-08-09T15:13:03.896 INFO:teuthology.orchestra.run.smithi043.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.147252.log.gz 2023-08-09T15:13:03.897 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.130204.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.96662.log 2023-08-09T15:13:03.897 INFO:teuthology.orchestra.run.smithi043.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.130204.log.gz 2023-08-09T15:13:03.898 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.101961.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.137433.log 2023-08-09T15:13:03.899 INFO:teuthology.orchestra.run.smithi043.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.101961.log.gz 2023-08-09T15:13:03.900 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.96662.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.142521.log 2023-08-09T15:13:03.900 INFO:teuthology.orchestra.run.smithi043.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.96662.log.gz 2023-08-09T15:13:03.901 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.137433.log: 88.3%gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.111283.log 2023-08-09T15:13:03.901 INFO:teuthology.orchestra.run.smithi043.stderr: -- replaced with /var/log/ceph/ceph-client.admin.137433.log.gz 2023-08-09T15:13:03.903 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.142521.log: gzip -5 --verbose -- 89.0% -- replaced with /var/log/ceph/ceph-client.admin.142521.log.gz 2023-08-09T15:13:03.903 INFO:teuthology.orchestra.run.smithi043.stderr: /var/log/ceph/ceph-client.admin.156270.log 2023-08-09T15:13:03.904 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.111283.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.111283.log.gz 2023-08-09T15:13:03.904 INFO:teuthology.orchestra.run.smithi043.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.102937.log 2023-08-09T15:13:03.906 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.156270.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.151552.log 2023-08-09T15:13:03.906 INFO:teuthology.orchestra.run.smithi043.stderr: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.156270.log.gz 2023-08-09T15:13:03.908 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.102937.log: 87.3%gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.152033.log 2023-08-09T15:13:03.908 INFO:teuthology.orchestra.run.smithi043.stderr: -- replaced with /var/log/ceph/ceph-client.admin.102937.log.gz 2023-08-09T15:13:03.908 INFO:teuthology.orchestra.run.smithi043.stderr: 94.0% -- replaced with /var/log/ceph/ceph-client.0.113478.log.gz 2023-08-09T15:13:03.909 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.151552.log: gzip -5 --verbose -- 89.1% -- replaced with /var/log/ceph/ceph-client.admin.151552.log.gz 2023-08-09T15:13:03.910 INFO:teuthology.orchestra.run.smithi043.stderr: /var/log/ceph/ceph-client.admin.141429.log 2023-08-09T15:13:03.910 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.152033.log: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.152033.log.gz 2023-08-09T15:13:03.911 INFO:teuthology.orchestra.run.smithi043.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.123047.log 2023-08-09T15:13:03.912 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.141429.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.120752.log 2023-08-09T15:13:03.913 INFO:teuthology.orchestra.run.smithi043.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.141429.log.gz 2023-08-09T15:13:03.914 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.123047.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.123150.log 2023-08-09T15:13:03.914 INFO:teuthology.orchestra.run.smithi043.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.123047.log.gz 2023-08-09T15:13:03.915 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.120752.log: gzip -5 89.1% -- replaced with /var/log/ceph/ceph-client.admin.120752.log.gz 2023-08-09T15:13:03.916 INFO:teuthology.orchestra.run.smithi043.stderr: --verbose -- /var/log/ceph/ceph-client.admin.108839.log 2023-08-09T15:13:03.917 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.123150.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.123150.log.gz 2023-08-09T15:13:03.917 INFO:teuthology.orchestra.run.smithi043.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.101425.log 2023-08-09T15:13:03.920 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.108839.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.99202.log 2023-08-09T15:13:03.920 INFO:teuthology.orchestra.run.smithi043.stderr: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.108839.log.gz 2023-08-09T15:13:03.921 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.101425.log: 88.7%gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.99706.log 2023-08-09T15:13:03.921 INFO:teuthology.orchestra.run.smithi043.stderr: -- replaced with /var/log/ceph/ceph-client.admin.101425.log.gz 2023-08-09T15:13:03.922 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.99202.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.111866.log 2023-08-09T15:13:03.922 INFO:teuthology.orchestra.run.smithi043.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.99202.log.gz 2023-08-09T15:13:03.923 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.99706.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.137546.log 2023-08-09T15:13:03.923 INFO:teuthology.orchestra.run.smithi043.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.99706.log.gz 2023-08-09T15:13:03.924 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.111866.log: gzip -5 --verbose -- /var/log/ceph/ceph-mds.c.log 2023-08-09T15:13:03.925 INFO:teuthology.orchestra.run.smithi043.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.111866.log.gz 2023-08-09T15:13:03.926 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.137546.log: gzip -5 --verbose 88.1% -- replaced with /var/log/ceph/ceph-client.admin.137546.log.gz 2023-08-09T15:13:03.926 INFO:teuthology.orchestra.run.smithi043.stderr: -- /var/log/ceph/ceph-client.admin.104643.log 2023-08-09T15:13:03.928 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-mds.c.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.116272.log 2023-08-09T15:13:03.930 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.104643.log: gzip 89.0% -- replaced with /var/log/ceph/ceph-client.admin.104643.log.gz 2023-08-09T15:13:03.930 INFO:teuthology.orchestra.run.smithi043.stderr: -5 --verbose -- /var/log/ceph/ceph-client.admin.95166.log 2023-08-09T15:13:03.931 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.116272.log: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.116272.log.gz 2023-08-09T15:13:03.931 INFO:teuthology.orchestra.run.smithi043.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.140558.log 2023-08-09T15:13:03.933 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.95166.log: gzip -5 --verbose 88.8% -- replaced with /var/log/ceph/ceph-client.admin.95166.log.gz 2023-08-09T15:13:03.933 INFO:teuthology.orchestra.run.smithi043.stderr: -- /var/log/ceph/ceph-client.admin.97152.log 2023-08-09T15:13:03.934 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.140558.log: 86.3% -- replaced with /var/log/ceph/ceph-client.admin.140558.log.gz 2023-08-09T15:13:03.935 INFO:teuthology.orchestra.run.smithi043.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.152584.log 2023-08-09T15:13:03.936 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.97152.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.97152.log.gz 2023-08-09T15:13:03.937 INFO:teuthology.orchestra.run.smithi043.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.101793.log 2023-08-09T15:13:03.938 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.152584.log: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.152584.log.gz 2023-08-09T15:13:03.938 INFO:teuthology.orchestra.run.smithi043.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.111367.log 2023-08-09T15:13:03.939 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.101793.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.101793.log.gz 2023-08-09T15:13:03.940 INFO:teuthology.orchestra.run.smithi043.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.151678.log 2023-08-09T15:13:03.942 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.111367.log: gzip -5 88.8% -- replaced with /var/log/ceph/ceph-client.admin.111367.log.gz 2023-08-09T15:13:03.942 INFO:teuthology.orchestra.run.smithi043.stderr: --verbose -- /var/log/ceph/ceph-client.admin.131727.log 2023-08-09T15:13:03.943 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.151678.log: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.151678.log.gz 2023-08-09T15:13:03.944 INFO:teuthology.orchestra.run.smithi043.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.95982.log 2023-08-09T15:13:03.945 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.131727.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.131727.log.gz 2023-08-09T15:13:03.945 INFO:teuthology.orchestra.run.smithi043.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.101035.log 2023-08-09T15:13:03.947 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.95982.log: gzip -5 88.8% -- replaced with /var/log/ceph/ceph-client.admin.95982.log.gz 2023-08-09T15:13:03.947 INFO:teuthology.orchestra.run.smithi043.stderr: --verbose -- /var/log/ceph/ceph-client.admin.102685.log 2023-08-09T15:13:03.948 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.101035.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.101035.log.gz 2023-08-09T15:13:03.948 INFO:teuthology.orchestra.run.smithi043.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.122963.log 2023-08-09T15:13:03.950 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.102685.log: gzip -5 88.2% -- replaced with /var/log/ceph/ceph-client.admin.102685.log.gz 2023-08-09T15:13:03.950 INFO:teuthology.orchestra.run.smithi043.stderr: --verbose -- /var/log/ceph/ceph-client.admin.146244.log 2023-08-09T15:13:03.952 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.122963.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.122963.log.gz 2023-08-09T15:13:03.952 INFO:teuthology.orchestra.run.smithi043.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.121888.log 2023-08-09T15:13:03.953 INFO:teuthology.orchestra.run.smithi043.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.127538.log 2023-08-09T15:13:03.955 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.146244.log: /var/log/ceph/ceph-client.admin.121888.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.0.133240.log 2023-08-09T15:13:03.955 INFO:teuthology.orchestra.run.smithi043.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.121888.log.gz 2023-08-09T15:13:03.955 INFO:teuthology.orchestra.run.smithi043.stderr: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.146244.log.gz 2023-08-09T15:13:03.957 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.127538.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.127538.log.gzgzip -5 --verbose -- /var/log/ceph/ceph-client.admin.105996.log 2023-08-09T15:13:03.958 INFO:teuthology.orchestra.run.smithi043.stderr: 2023-08-09T15:13:03.959 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.0.133240.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.103315.log 2023-08-09T15:13:03.960 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.105996.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.103567.log 2023-08-09T15:13:03.960 INFO:teuthology.orchestra.run.smithi043.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.105996.log.gz 2023-08-09T15:13:03.961 INFO:teuthology.orchestra.run.smithi043.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.93692.log 2023-08-09T15:13:03.962 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.103315.log: /var/log/ceph/ceph-client.admin.103567.log: 87.2% -- replaced with /var/log/ceph/ceph-client.admin.103315.log.gz 2023-08-09T15:13:03.963 INFO:teuthology.orchestra.run.smithi043.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.103567.log.gz 2023-08-09T15:13:03.963 INFO:teuthology.orchestra.run.smithi043.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.122390.log 2023-08-09T15:13:03.965 INFO:teuthology.orchestra.run.smithi043.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.110783.log 2023-08-09T15:13:03.966 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.93692.log: /var/log/ceph/ceph-client.admin.122390.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.93692.log.gz 2023-08-09T15:13:03.967 INFO:teuthology.orchestra.run.smithi043.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.122390.log.gz 2023-08-09T15:13:03.968 INFO:teuthology.orchestra.run.smithi043.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.103399.log 2023-08-09T15:13:03.969 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.110783.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.94072.log 2023-08-09T15:13:03.970 INFO:teuthology.orchestra.run.smithi043.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.110783.log.gz 2023-08-09T15:13:03.971 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.103399.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.99958.log 2023-08-09T15:13:03.972 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.94072.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.103399.log.gz 2023-08-09T15:13:03.973 INFO:teuthology.orchestra.run.smithi043.stderr: 91.4% -- replaced with /var/log/ceph/ceph-client.admin.94072.log.gz 2023-08-09T15:13:03.973 INFO:teuthology.orchestra.run.smithi043.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.111325.log 2023-08-09T15:13:03.975 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.99958.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.104473.log 2023-08-09T15:13:03.976 INFO:teuthology.orchestra.run.smithi043.stderr: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.99958.log.gz 2023-08-09T15:13:03.979 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.111325.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.111493.log 2023-08-09T15:13:03.979 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.104473.log: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.111325.log.gz 2023-08-09T15:13:03.979 INFO:teuthology.orchestra.run.smithi043.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.104473.log.gz 2023-08-09T15:13:03.980 INFO:teuthology.orchestra.run.smithi043.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.117327.log 2023-08-09T15:13:03.982 INFO:teuthology.orchestra.run.smithi043.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.155500.log 2023-08-09T15:13:03.982 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.111493.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.111493.log.gz 2023-08-09T15:13:03.983 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.117327.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.141513.log 2023-08-09T15:13:03.984 INFO:teuthology.orchestra.run.smithi043.stderr: 88.9%/var/log/ceph/ceph-client.admin.155500.log: -- replaced with /var/log/ceph/ceph-client.admin.117327.log.gz 2023-08-09T15:13:03.985 INFO:teuthology.orchestra.run.smithi043.stderr: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.155500.log.gz 2023-08-09T15:13:03.986 INFO:teuthology.orchestra.run.smithi043.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.146160.log 2023-08-09T15:13:03.987 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.141513.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.146743.log 2023-08-09T15:13:03.987 INFO:teuthology.orchestra.run.smithi043.stderr: 87.0% -- replaced with /var/log/ceph/ceph-client.admin.141513.log.gz 2023-08-09T15:13:03.989 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.146160.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.93778.log 2023-08-09T15:13:03.990 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.146743.log: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.146160.log.gz 2023-08-09T15:13:03.990 INFO:teuthology.orchestra.run.smithi043.stderr: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.146743.log.gz 2023-08-09T15:13:03.992 INFO:teuthology.orchestra.run.smithi043.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.102003.log 2023-08-09T15:13:03.993 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.93778.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.100867.log 2023-08-09T15:13:03.994 INFO:teuthology.orchestra.run.smithi043.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.93778.log.gz 2023-08-09T15:13:03.995 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.102003.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.138432.log 2023-08-09T15:13:03.996 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.100867.log: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.102003.log.gz 2023-08-09T15:13:03.997 INFO:teuthology.orchestra.run.smithi043.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.100867.log.gz 2023-08-09T15:13:03.997 INFO:teuthology.orchestra.run.smithi043.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.124175.log 2023-08-09T15:13:04.000 INFO:teuthology.orchestra.run.smithi043.stderr:gzip/var/log/ceph/ceph-client.admin.138432.log: -5 --verbose -- /var/log/ceph/ceph-client.admin.138077.log 2023-08-09T15:13:04.000 INFO:teuthology.orchestra.run.smithi043.stderr: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.138432.log.gz 2023-08-09T15:13:04.001 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.124175.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.138203.log 2023-08-09T15:13:04.002 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.138077.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.124175.log.gz 2023-08-09T15:13:04.003 INFO:teuthology.orchestra.run.smithi043.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.138077.log.gz 2023-08-09T15:13:04.003 INFO:teuthology.orchestra.run.smithi043.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.101341.log 2023-08-09T15:13:04.005 INFO:teuthology.orchestra.run.smithi043.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.127287.log 2023-08-09T15:13:04.006 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.138203.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.138203.log.gz 2023-08-09T15:13:04.008 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.101341.log: 89.0%gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.141261.log 2023-08-09T15:13:04.008 INFO:teuthology.orchestra.run.smithi043.stderr: -- replaced with /var/log/ceph/ceph-client.admin.101341.log.gz 2023-08-09T15:13:04.009 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.127287.log: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.127287.log.gz 2023-08-09T15:13:04.009 INFO:teuthology.orchestra.run.smithi043.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.122669.log 2023-08-09T15:13:04.011 INFO:teuthology.orchestra.run.smithi043.stderr:gzip/var/log/ceph/ceph-client.admin.141261.log: -5 --verbose -- /var/log/ceph/ceph-client.admin.117625.log 2023-08-09T15:13:04.012 INFO:teuthology.orchestra.run.smithi043.stderr: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.141261.log.gz 2023-08-09T15:13:04.013 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.122669.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.97530.log 2023-08-09T15:13:04.014 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.117625.log: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.122669.log.gz 2023-08-09T15:13:04.014 INFO:teuthology.orchestra.run.smithi043.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.117625.log.gz 2023-08-09T15:13:04.016 INFO:teuthology.orchestra.run.smithi043.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.120794.log 2023-08-09T15:13:04.017 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.97530.log: 88.5%gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.95986.log 2023-08-09T15:13:04.017 INFO:teuthology.orchestra.run.smithi043.stderr: -- replaced with /var/log/ceph/ceph-client.admin.97530.log.gz 2023-08-09T15:13:04.018 INFO:teuthology.orchestra.run.smithi043.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.140761.log 2023-08-09T15:13:04.019 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.120794.log: /var/log/ceph/ceph-client.admin.95986.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.120794.log.gz 2023-08-09T15:13:04.021 INFO:teuthology.orchestra.run.smithi043.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.95986.log.gz 2023-08-09T15:13:04.021 INFO:teuthology.orchestra.run.smithi043.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.133589.log 2023-08-09T15:13:04.022 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.140761.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.96643.log 2023-08-09T15:13:04.024 INFO:teuthology.orchestra.run.smithi043.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.140761.log.gz 2023-08-09T15:13:04.025 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.133589.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.132586.log 2023-08-09T15:13:04.025 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.96643.log: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.133589.log.gz 2023-08-09T15:13:04.026 INFO:teuthology.orchestra.run.smithi043.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.96643.log.gz 2023-08-09T15:13:04.028 INFO:teuthology.orchestra.run.smithi043.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.142609.log 2023-08-09T15:13:04.029 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.132586.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.139237.log 2023-08-09T15:13:04.029 INFO:teuthology.orchestra.run.smithi043.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.132586.log.gz 2023-08-09T15:13:04.031 INFO:teuthology.orchestra.run.smithi043.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.156397.log 2023-08-09T15:13:04.032 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.142609.log: /var/log/ceph/ceph-client.admin.139237.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.142609.log.gz 2023-08-09T15:13:04.033 INFO:teuthology.orchestra.run.smithi043.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.139237.log.gz 2023-08-09T15:13:04.034 INFO:teuthology.orchestra.run.smithi043.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.156210.log 2023-08-09T15:13:04.035 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.156397.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.118106.log 2023-08-09T15:13:04.035 INFO:teuthology.orchestra.run.smithi043.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.156397.log.gz 2023-08-09T15:13:04.037 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.156210.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.139331.log 2023-08-09T15:13:04.038 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.118106.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.156210.log.gz 2023-08-09T15:13:04.038 INFO:teuthology.orchestra.run.smithi043.stderr: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.118106.log.gz 2023-08-09T15:13:04.040 INFO:teuthology.orchestra.run.smithi043.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.133488.log 2023-08-09T15:13:04.041 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.139331.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.97362.log 2023-08-09T15:13:04.041 INFO:teuthology.orchestra.run.smithi043.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.139331.log.gz 2023-08-09T15:13:04.046 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.133488.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.137783.log 2023-08-09T15:13:04.046 INFO:teuthology.orchestra.run.smithi043.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.133488.log.gz/var/log/ceph/ceph-client.admin.97362.log: 2023-08-09T15:13:04.046 INFO:teuthology.orchestra.run.smithi043.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.97362.log.gz 2023-08-09T15:13:04.047 INFO:teuthology.orchestra.run.smithi043.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.116944.log 2023-08-09T15:13:04.047 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.137783.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.153248.log 2023-08-09T15:13:04.048 INFO:teuthology.orchestra.run.smithi043.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.137783.log.gz 2023-08-09T15:13:04.051 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.153248.log: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.153248.log.gz 2023-08-09T15:13:04.051 INFO:teuthology.orchestra.run.smithi043.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.151300.log 2023-08-09T15:13:04.053 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.116944.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.106055.log 2023-08-09T15:13:04.053 INFO:teuthology.orchestra.run.smithi043.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.116944.log.gz 2023-08-09T15:13:04.054 INFO:teuthology.orchestra.run.smithi043.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.125589.log 2023-08-09T15:13:04.056 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.151300.log: /var/log/ceph/ceph-client.admin.106055.log: 89.1% 87.4% -- replaced with /var/log/ceph/ceph-client.admin.106055.log.gz 2023-08-09T15:13:04.057 INFO:teuthology.orchestra.run.smithi043.stderr: -- replaced with /var/log/ceph/ceph-client.admin.151300.log.gz 2023-08-09T15:13:04.057 INFO:teuthology.orchestra.run.smithi043.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.104337.log 2023-08-09T15:13:04.058 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.125589.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.96942.log 2023-08-09T15:13:04.059 INFO:teuthology.orchestra.run.smithi043.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.125589.log.gz 2023-08-09T15:13:04.060 INFO:teuthology.orchestra.run.smithi043.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.145473.log 2023-08-09T15:13:04.061 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.104337.log: /var/log/ceph/ceph-client.admin.96942.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.104337.log.gz 89.2% -- replaced with /var/log/ceph/ceph-client.admin.96942.log.gz 2023-08-09T15:13:04.061 INFO:teuthology.orchestra.run.smithi043.stderr: 2023-08-09T15:13:04.063 INFO:teuthology.orchestra.run.smithi043.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.123360.log 2023-08-09T15:13:04.063 INFO:teuthology.orchestra.run.smithi043.stderr: 95.5% -- replaced with /var/log/ceph/ceph-client.0.133240.log.gz 2023-08-09T15:13:04.064 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.145473.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.145576.log 2023-08-09T15:13:04.065 INFO:teuthology.orchestra.run.smithi043.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.145473.log.gz 2023-08-09T15:13:04.065 INFO:teuthology.orchestra.run.smithi043.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.132544.log 2023-08-09T15:13:04.067 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.123360.log: /var/log/ceph/ceph-client.admin.145576.log: 89.3%gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.107189.log 2023-08-09T15:13:04.068 INFO:teuthology.orchestra.run.smithi043.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.145576.log.gz 2023-08-09T15:13:04.068 INFO:teuthology.orchestra.run.smithi043.stderr: -- replaced with /var/log/ceph/ceph-client.admin.123360.log.gz 2023-08-09T15:13:04.069 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.132544.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.113669.log 2023-08-09T15:13:04.069 INFO:teuthology.orchestra.run.smithi043.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.132544.log.gz 2023-08-09T15:13:04.070 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.107189.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.113135.log 2023-08-09T15:13:04.070 INFO:teuthology.orchestra.run.smithi043.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.107189.log.gz 2023-08-09T15:13:04.071 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.113669.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.113669.log.gz 2023-08-09T15:13:04.072 INFO:teuthology.orchestra.run.smithi043.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.146076.log 2023-08-09T15:13:04.073 INFO:teuthology.orchestra.run.smithi043.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.110154.log 2023-08-09T15:13:04.075 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.113135.log: /var/log/ceph/ceph-client.admin.146076.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.141573.log 2023-08-09T15:13:04.075 INFO:teuthology.orchestra.run.smithi043.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.146076.log.gz 2023-08-09T15:13:04.076 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.110154.log: 91.1% -- replaced with /var/log/ceph/ceph-client.admin.113135.log.gz 2023-08-09T15:13:04.077 INFO:teuthology.orchestra.run.smithi043.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.93432.log 2023-08-09T15:13:04.077 INFO:teuthology.orchestra.run.smithi043.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.110154.log.gz 2023-08-09T15:13:04.078 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.141573.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.115730.log 2023-08-09T15:13:04.078 INFO:teuthology.orchestra.run.smithi043.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.141573.log.gz 2023-08-09T15:13:04.079 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.93432.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.154598.log 2023-08-09T15:13:04.080 INFO:teuthology.orchestra.run.smithi043.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.93432.log.gz 2023-08-09T15:13:04.081 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.115730.log: 88.6%gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.142955.log 2023-08-09T15:13:04.081 INFO:teuthology.orchestra.run.smithi043.stderr: -- replaced with /var/log/ceph/ceph-client.admin.115730.log.gz 2023-08-09T15:13:04.083 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.154598.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.152117.log 2023-08-09T15:13:04.083 INFO:teuthology.orchestra.run.smithi043.stderr: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.154598.log.gz 2023-08-09T15:13:04.084 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.142955.log: gzip -5 88.3% --verbose -- -- replaced with /var/log/ceph/ceph-client.admin.142955.log.gz /var/log/ceph/ceph-client.admin.101635.log 2023-08-09T15:13:04.084 INFO:teuthology.orchestra.run.smithi043.stderr: 2023-08-09T15:13:04.086 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.152117.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.105699.log 2023-08-09T15:13:04.086 INFO:teuthology.orchestra.run.smithi043.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.152117.log.gz 2023-08-09T15:13:04.087 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.101635.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.138390.log 2023-08-09T15:13:04.087 INFO:teuthology.orchestra.run.smithi043.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.101635.log.gz 2023-08-09T15:13:04.088 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.105699.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.157436.log 2023-08-09T15:13:04.089 INFO:teuthology.orchestra.run.smithi043.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.105699.log.gz 2023-08-09T15:13:04.090 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.138390.log: gzip -5 89.3% -- replaced with /var/log/ceph/ceph-client.admin.138390.log.gz 2023-08-09T15:13:04.090 INFO:teuthology.orchestra.run.smithi043.stderr: --verbose -- /var/log/ceph/ceph-client.admin.142913.log 2023-08-09T15:13:04.091 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.157436.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.157436.log.gz 2023-08-09T15:13:04.092 INFO:teuthology.orchestra.run.smithi043.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.102979.log 2023-08-09T15:13:04.093 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.142913.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.142913.log.gz 2023-08-09T15:13:04.093 INFO:teuthology.orchestra.run.smithi043.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.131078.log 2023-08-09T15:13:04.095 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.102979.log: gzip 89.0% -- replaced with /var/log/ceph/ceph-client.admin.102979.log.gz 2023-08-09T15:13:04.095 INFO:teuthology.orchestra.run.smithi043.stderr: -5 --verbose -- /var/log/ceph/ceph-client.admin.151105.log 2023-08-09T15:13:04.096 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.131078.log: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.131078.log.gz 2023-08-09T15:13:04.097 INFO:teuthology.orchestra.run.smithi043.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.107518.log 2023-08-09T15:13:04.098 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.151105.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.151105.log.gz 2023-08-09T15:13:04.099 INFO:teuthology.orchestra.run.smithi043.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.97278.log 2023-08-09T15:13:04.101 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.107518.log: gzip -5 --verbose 87.9% -- replaced with /var/log/ceph/ceph-client.admin.107518.log.gz 2023-08-09T15:13:04.101 INFO:teuthology.orchestra.run.smithi043.stderr: -- /var/log/ceph/ceph-client.admin.153922.log 2023-08-09T15:13:04.102 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.97278.log: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.97278.log.gz 2023-08-09T15:13:04.102 INFO:teuthology.orchestra.run.smithi043.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.156439.log 2023-08-09T15:13:04.104 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.153922.log: gzip 88.4% -- replaced with /var/log/ceph/ceph-client.admin.153922.log.gz 2023-08-09T15:13:04.104 INFO:teuthology.orchestra.run.smithi043.stderr: -5 --verbose -- /var/log/ceph/ceph-client.admin.116860.log 2023-08-09T15:13:04.105 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.156439.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.156439.log.gz 2023-08-09T15:13:04.105 INFO:teuthology.orchestra.run.smithi043.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.122585.log 2023-08-09T15:13:04.107 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.116860.log: gzip -5 88.8% -- replaced with /var/log/ceph/ceph-client.admin.116860.log.gz 2023-08-09T15:13:04.107 INFO:teuthology.orchestra.run.smithi043.stderr: --verbose -- /var/log/ceph/ceph-client.admin.113771.log 2023-08-09T15:13:04.108 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.122585.log: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.122585.log.gz 2023-08-09T15:13:04.109 INFO:teuthology.orchestra.run.smithi043.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.95766.log 2023-08-09T15:13:04.110 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.113771.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.113771.log.gz 2023-08-09T15:13:04.111 INFO:teuthology.orchestra.run.smithi043.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.132320.log 2023-08-09T15:13:04.112 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.95766.log: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.95766.log.gz 2023-08-09T15:13:04.113 INFO:teuthology.orchestra.run.smithi043.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.157226.log 2023-08-09T15:13:04.114 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.132320.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.132320.log.gz 2023-08-09T15:13:04.115 INFO:teuthology.orchestra.run.smithi043.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.147686.log 2023-08-09T15:13:04.116 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.157226.log: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.157226.log.gz 2023-08-09T15:13:04.116 INFO:teuthology.orchestra.run.smithi043.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.151010.log 2023-08-09T15:13:04.117 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.147686.log: gzip 88.3% -5 --verbose -- /var/log/ceph/ceph-client.admin.96322.log 2023-08-09T15:13:04.118 INFO:teuthology.orchestra.run.smithi043.stderr: -- replaced with /var/log/ceph/ceph-client.admin.147686.log.gz 2023-08-09T15:13:04.119 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.151010.log: gzip -5 --verbose 88.5% -- replaced with /var/log/ceph/ceph-client.admin.151010.log.gz 2023-08-09T15:13:04.120 INFO:teuthology.orchestra.run.smithi043.stderr: -- /var/log/ceph/ceph-client.admin.111004.log 2023-08-09T15:13:04.121 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.96322.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.96322.log.gz 2023-08-09T15:13:04.121 INFO:teuthology.orchestra.run.smithi043.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.104209.log 2023-08-09T15:13:04.122 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.111004.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.111004.log.gz 2023-08-09T15:13:04.123 INFO:teuthology.orchestra.run.smithi043.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.147086.log 2023-08-09T15:13:04.124 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.104209.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.104209.log.gz 2023-08-09T15:13:04.125 INFO:teuthology.orchestra.run.smithi043.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.123107.log 2023-08-09T15:13:04.126 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.147086.log: gzip -5 88.0% -- replaced with /var/log/ceph/ceph-client.admin.147086.log.gz 2023-08-09T15:13:04.127 INFO:teuthology.orchestra.run.smithi043.stderr: --verbose -- /var/log/ceph/ceph-client.admin.140803.log 2023-08-09T15:13:04.128 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.123107.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.123107.log.gz 2023-08-09T15:13:04.128 INFO:teuthology.orchestra.run.smithi043.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.139417.log 2023-08-09T15:13:04.129 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.140803.log: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.140803.log.gz 2023-08-09T15:13:04.130 INFO:teuthology.orchestra.run.smithi043.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.120882.log 2023-08-09T15:13:04.131 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.139417.log: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.139417.log.gz 2023-08-09T15:13:04.132 INFO:teuthology.orchestra.run.smithi043.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.111577.log 2023-08-09T15:13:04.133 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.120882.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.120882.log.gz 2023-08-09T15:13:04.134 INFO:teuthology.orchestra.run.smithi043.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.113878.log 2023-08-09T15:13:04.135 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.111577.log: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.111577.log.gz 2023-08-09T15:13:04.136 INFO:teuthology.orchestra.run.smithi043.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.102517.log 2023-08-09T15:13:04.137 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.113878.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.113878.log.gz 2023-08-09T15:13:04.137 INFO:teuthology.orchestra.run.smithi043.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.107399.log 2023-08-09T15:13:04.139 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.102517.log: gzip 87.5% -- replaced with /var/log/ceph/ceph-client.admin.102517.log.gz 2023-08-09T15:13:04.139 INFO:teuthology.orchestra.run.smithi043.stderr: -5 --verbose -- /var/log/ceph/ceph-client.admin.141868.log 2023-08-09T15:13:04.140 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.107399.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.107399.log.gz 2023-08-09T15:13:04.141 INFO:teuthology.orchestra.run.smithi043.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.117369.log 2023-08-09T15:13:04.142 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.141868.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.141868.log.gz 2023-08-09T15:13:04.143 INFO:teuthology.orchestra.run.smithi043.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.118064.log 2023-08-09T15:13:04.144 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.117369.log: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.117369.log.gz 2023-08-09T15:13:04.144 INFO:teuthology.orchestra.run.smithi043.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.99412.log 2023-08-09T15:13:04.146 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.118064.log: gzip -5 --verbose 88.8% -- replaced with /var/log/ceph/ceph-client.admin.118064.log.gz 2023-08-09T15:13:04.146 INFO:teuthology.orchestra.run.smithi043.stderr: -- /var/log/ceph/ceph-client.admin.121102.log 2023-08-09T15:13:04.147 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.99412.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.99412.log.gz 2023-08-09T15:13:04.148 INFO:teuthology.orchestra.run.smithi043.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.107315.log 2023-08-09T15:13:04.149 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.121102.log: gzip -5 --verbose 89.1% -- replaced with /var/log/ceph/ceph-client.admin.121102.log.gz 2023-08-09T15:13:04.149 INFO:teuthology.orchestra.run.smithi043.stderr: -- /var/log/ceph/ceph-client.admin.100126.log 2023-08-09T15:13:04.150 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.107315.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.107315.log.gz 2023-08-09T15:13:04.151 INFO:teuthology.orchestra.run.smithi043.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.125631.log 2023-08-09T15:13:04.153 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.100126.log: gzip 87.4% -- replaced with /var/log/ceph/ceph-client.admin.100126.log.gz 2023-08-09T15:13:04.153 INFO:teuthology.orchestra.run.smithi043.stderr: -5 --verbose -- /var/log/ceph/ceph-client.admin.127846.log 2023-08-09T15:13:04.154 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.125631.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.125631.log.gz 2023-08-09T15:13:04.155 INFO:teuthology.orchestra.run.smithi043.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.126775.log 2023-08-09T15:13:04.156 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.127846.log: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.127846.log.gz 2023-08-09T15:13:04.156 INFO:teuthology.orchestra.run.smithi043.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.131120.log 2023-08-09T15:13:04.157 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.126775.log: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.126775.log.gz 2023-08-09T15:13:04.158 INFO:teuthology.orchestra.run.smithi043.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.155874.log 2023-08-09T15:13:04.160 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.131120.log: gzip 88.4% -- replaced with /var/log/ceph/ceph-client.admin.131120.log.gz 2023-08-09T15:13:04.160 INFO:teuthology.orchestra.run.smithi043.stderr: -5 --verbose -- /var/log/ceph/ceph-client.admin.111157.log 2023-08-09T15:13:04.161 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.155874.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.155874.log.gz 2023-08-09T15:13:04.162 INFO:teuthology.orchestra.run.smithi043.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.139585.log 2023-08-09T15:13:04.163 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.111157.log: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.111157.log.gz 2023-08-09T15:13:04.164 INFO:teuthology.orchestra.run.smithi043.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.139024.log 2023-08-09T15:13:04.165 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.139585.log: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.139585.log.gz 2023-08-09T15:13:04.165 INFO:teuthology.orchestra.run.smithi043.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.126150.log 2023-08-09T15:13:04.166 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.139024.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.139024.log.gz 2023-08-09T15:13:04.167 INFO:teuthology.orchestra.run.smithi043.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.148420.log 2023-08-09T15:13:04.168 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.126150.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.126150.log.gz 2023-08-09T15:13:04.169 INFO:teuthology.orchestra.run.smithi043.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.147167.log 2023-08-09T15:13:04.170 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.148420.log: gzip 88.2% -5 --verbose -- /var/log/ceph/ceph-client.admin.141658.log 2023-08-09T15:13:04.170 INFO:teuthology.orchestra.run.smithi043.stderr: -- replaced with /var/log/ceph/ceph-client.admin.148420.log.gz 2023-08-09T15:13:04.172 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.147167.log: gzip 87.7% -- replaced with /var/log/ceph/ceph-client.admin.147167.log.gz 2023-08-09T15:13:04.172 INFO:teuthology.orchestra.run.smithi043.stderr: -5 --verbose -- /var/log/ceph/ceph-client.admin.154430.log 2023-08-09T15:13:04.173 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.141658.log: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.141658.log.gz 2023-08-09T15:13:04.174 INFO:teuthology.orchestra.run.smithi043.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.126360.log 2023-08-09T15:13:04.175 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.154430.log: gzip -5 88.7% -- replaced with /var/log/ceph/ceph-client.admin.154430.log.gz 2023-08-09T15:13:04.176 INFO:teuthology.orchestra.run.smithi043.stderr: --verbose -- /var/log/ceph/ceph-client.admin.101593.log 2023-08-09T15:13:04.177 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.126360.log: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.126360.log.gz 2023-08-09T15:13:04.177 INFO:teuthology.orchestra.run.smithi043.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.151907.log 2023-08-09T15:13:04.179 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.101593.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.101593.log.gz 2023-08-09T15:13:04.179 INFO:teuthology.orchestra.run.smithi043.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.94557.log 2023-08-09T15:13:04.181 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.151907.log: 88.6%gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.142997.log 2023-08-09T15:13:04.181 INFO:teuthology.orchestra.run.smithi043.stderr: -- replaced with /var/log/ceph/ceph-client.admin.151907.log.gz 2023-08-09T15:13:04.182 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.94557.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.141051.log 2023-08-09T15:13:04.184 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.142997.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.142997.log.gzgzip 92.4% 2023-08-09T15:13:04.184 INFO:teuthology.orchestra.run.smithi043.stderr: -- replaced with /var/log/ceph/ceph-client.admin.94557.log.gz -5 2023-08-09T15:13:04.184 INFO:teuthology.orchestra.run.smithi043.stderr: --verbose -- /var/log/ceph/ceph-client.admin.142701.log 2023-08-09T15:13:04.185 INFO:teuthology.orchestra.run.smithi043.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.112925.log 2023-08-09T15:13:04.187 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.141051.log: /var/log/ceph/ceph-client.admin.142701.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.125997.log 2023-08-09T15:13:04.187 INFO:teuthology.orchestra.run.smithi043.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.142701.log.gz 88.7% -- replaced with /var/log/ceph/ceph-client.admin.141051.log.gz 2023-08-09T15:13:04.188 INFO:teuthology.orchestra.run.smithi043.stderr: 2023-08-09T15:13:04.188 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.112925.log: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.112925.log.gz 2023-08-09T15:13:04.189 INFO:teuthology.orchestra.run.smithi043.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.117853.log 2023-08-09T15:13:04.190 INFO:teuthology.orchestra.run.smithi043.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.124741.log 2023-08-09T15:13:04.192 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.125997.log: /var/log/ceph/ceph-client.admin.117853.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.138642.log 2023-08-09T15:13:04.192 INFO:teuthology.orchestra.run.smithi043.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.117853.log.gz 2023-08-09T15:13:04.192 INFO:teuthology.orchestra.run.smithi043.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.125997.log.gz 2023-08-09T15:13:04.193 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.124741.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.157520.log 2023-08-09T15:13:04.193 INFO:teuthology.orchestra.run.smithi043.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.124741.log.gz 2023-08-09T15:13:04.194 INFO:teuthology.orchestra.run.smithi043.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.108050.log 2023-08-09T15:13:04.196 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.138642.log: /var/log/ceph/ceph-client.admin.157520.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.118672.log 2023-08-09T15:13:04.196 INFO:teuthology.orchestra.run.smithi043.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.157520.log.gz 2023-08-09T15:13:04.197 INFO:teuthology.orchestra.run.smithi043.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.138642.log.gz 2023-08-09T15:13:04.198 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.108050.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.133422.log 2023-08-09T15:13:04.198 INFO:teuthology.orchestra.run.smithi043.stderr: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.108050.log.gz 2023-08-09T15:13:04.199 INFO:teuthology.orchestra.run.smithi043.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.116188.log 2023-08-09T15:13:04.200 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.118672.log: /var/log/ceph/ceph-client.admin.133422.log: 88.3%gzip -- replaced with /var/log/ceph/ceph-client.admin.118672.log.gz -5 --verbose -- /var/log/ceph/ceph-client.admin.102769.log 2023-08-09T15:13:04.201 INFO:teuthology.orchestra.run.smithi043.stderr: 2023-08-09T15:13:04.201 INFO:teuthology.orchestra.run.smithi043.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.133422.log.gz 2023-08-09T15:13:04.202 INFO:teuthology.orchestra.run.smithi043.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.111950.log 2023-08-09T15:13:04.203 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.116188.log: /var/log/ceph/ceph-client.admin.102769.log: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.116188.log.gzgzip -5 --verbose -- /var/log/ceph/ceph-client.admin.124657.log 2023-08-09T15:13:04.204 INFO:teuthology.orchestra.run.smithi043.stderr: 2023-08-09T15:13:04.204 INFO:teuthology.orchestra.run.smithi043.stderr: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.102769.log.gz 2023-08-09T15:13:04.205 INFO:teuthology.orchestra.run.smithi043.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.141093.log 2023-08-09T15:13:04.207 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.111950.log: /var/log/ceph/ceph-client.admin.124657.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.111950.log.gz 2023-08-09T15:13:04.207 INFO:teuthology.orchestra.run.smithi043.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.99790.log 2023-08-09T15:13:04.207 INFO:teuthology.orchestra.run.smithi043.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.124657.log.gz 2023-08-09T15:13:04.208 INFO:teuthology.orchestra.run.smithi043.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.0.140114.log 2023-08-09T15:13:04.210 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.141093.log: /var/log/ceph/ceph-client.admin.99790.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.141093.log.gzgzip -5 --verbose -- /var/log/ceph/ceph-client.admin.112076.log 2023-08-09T15:13:04.210 INFO:teuthology.orchestra.run.smithi043.stderr: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.99790.log.gz 2023-08-09T15:13:04.210 INFO:teuthology.orchestra.run.smithi043.stderr: 2023-08-09T15:13:04.211 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.0.140114.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.97194.log 2023-08-09T15:13:04.213 INFO:teuthology.orchestra.run.smithi043.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.95724.log 2023-08-09T15:13:04.214 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.112076.log: /var/log/ceph/ceph-client.admin.97194.log: 91.6% -- replaced with /var/log/ceph/ceph-client.0.140114.log.gz 2023-08-09T15:13:04.214 INFO:teuthology.orchestra.run.smithi043.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.112076.log.gz 2023-08-09T15:13:04.214 INFO:teuthology.orchestra.run.smithi043.stderr: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.97194.log.gz 2023-08-09T15:13:04.215 INFO:teuthology.orchestra.run.smithi043.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.101215.log 2023-08-09T15:13:04.216 INFO:teuthology.orchestra.run.smithi043.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.126628.log 2023-08-09T15:13:04.218 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.95724.log: /var/log/ceph/ceph-client.admin.101215.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.105429.log 2023-08-09T15:13:04.218 INFO:teuthology.orchestra.run.smithi043.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.101215.log.gz 2023-08-09T15:13:04.219 INFO:teuthology.orchestra.run.smithi043.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.95724.log.gz 2023-08-09T15:13:04.220 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.126628.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.154388.log 2023-08-09T15:13:04.220 INFO:teuthology.orchestra.run.smithi043.stderr: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.126628.log.gz 2023-08-09T15:13:04.220 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.105429.log: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.105429.log.gz 2023-08-09T15:13:04.221 INFO:teuthology.orchestra.run.smithi043.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.122295.log 2023-08-09T15:13:04.222 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.154388.log: 88.8%gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.105819.log 2023-08-09T15:13:04.222 INFO:teuthology.orchestra.run.smithi043.stderr: -- replaced with /var/log/ceph/ceph-client.admin.154388.log.gz 2023-08-09T15:13:04.224 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.122295.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.147728.log 2023-08-09T15:13:04.224 INFO:teuthology.orchestra.run.smithi043.stderr: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.122295.log.gz 2023-08-09T15:13:04.225 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.105819.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.132796.log 2023-08-09T15:13:04.226 INFO:teuthology.orchestra.run.smithi043.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.105819.log.gz 2023-08-09T15:13:04.227 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.147728.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.99118.log 2023-08-09T15:13:04.227 INFO:teuthology.orchestra.run.smithi043.stderr: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.147728.log.gz 2023-08-09T15:13:04.228 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.132796.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.151780.log 2023-08-09T15:13:04.229 INFO:teuthology.orchestra.run.smithi043.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.132796.log.gz 2023-08-09T15:13:04.230 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.99118.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.146491.log 2023-08-09T15:13:04.230 INFO:teuthology.orchestra.run.smithi043.stderr: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.99118.log.gz 2023-08-09T15:13:04.231 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.151780.log: gzip 88.7% -- replaced with /var/log/ceph/ceph-client.admin.151780.log.gz 2023-08-09T15:13:04.232 INFO:teuthology.orchestra.run.smithi043.stderr: -5 --verbose -- /var/log/ceph/ceph-client.admin.111740.log 2023-08-09T15:13:04.233 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.146491.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.146491.log.gz 2023-08-09T15:13:04.233 INFO:teuthology.orchestra.run.smithi043.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.106257.log 2023-08-09T15:13:04.235 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.111740.log: gzip -5 --verbose 88.6% -- replaced with /var/log/ceph/ceph-client.admin.111740.log.gz 2023-08-09T15:13:04.235 INFO:teuthology.orchestra.run.smithi043.stderr: -- /var/log/ceph/ceph-client.admin.116482.log 2023-08-09T15:13:04.237 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.106257.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.106257.log.gz 2023-08-09T15:13:04.238 INFO:teuthology.orchestra.run.smithi043.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.111908.log 2023-08-09T15:13:04.239 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.116482.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.116482.log.gz 2023-08-09T15:13:04.240 INFO:teuthology.orchestra.run.smithi043.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.110680.log 2023-08-09T15:13:04.240 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.111908.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.111908.log.gz 2023-08-09T15:13:04.241 INFO:teuthology.orchestra.run.smithi043.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.143672.log 2023-08-09T15:13:04.242 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.110680.log: 88.6%gzip -5 --verbose -- /var/log/ceph/ceph-client.0.121697.log 2023-08-09T15:13:04.242 INFO:teuthology.orchestra.run.smithi043.stderr: -- replaced with /var/log/ceph/ceph-client.admin.110680.log.gz 2023-08-09T15:13:04.244 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.143672.log: gzip -5 --verbose 88.7% -- replaced with /var/log/ceph/ceph-client.admin.143672.log.gz 2023-08-09T15:13:04.245 INFO:teuthology.orchestra.run.smithi043.stderr: -- /var/log/ceph/ceph-client.admin.111535.log 2023-08-09T15:13:04.247 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.0.121697.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.127972.log 2023-08-09T15:13:04.247 INFO:teuthology.orchestra.run.smithi043.stderr: 90.4% -- replaced with /var/log/ceph/ceph-client.0.121697.log.gz 2023-08-09T15:13:04.248 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.111535.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.111535.log.gz 2023-08-09T15:13:04.248 INFO:teuthology.orchestra.run.smithi043.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.132028.log 2023-08-09T15:13:04.249 INFO:teuthology.orchestra.run.smithi043.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.119469.log 2023-08-09T15:13:04.250 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.127972.log: /var/log/ceph/ceph-client.admin.132028.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.147209.log 2023-08-09T15:13:04.250 INFO:teuthology.orchestra.run.smithi043.stderr: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.132028.log.gz 2023-08-09T15:13:04.251 INFO:teuthology.orchestra.run.smithi043.stderr: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.127972.log.gz 2023-08-09T15:13:04.252 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.119469.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.95990.log 2023-08-09T15:13:04.252 INFO:teuthology.orchestra.run.smithi043.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.119469.log.gz 2023-08-09T15:13:04.253 INFO:teuthology.orchestra.run.smithi043.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.125415.log 2023-08-09T15:13:04.255 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.147209.log: /var/log/ceph/ceph-client.admin.95990.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.113093.log 2023-08-09T15:13:04.255 INFO:teuthology.orchestra.run.smithi043.stderr: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.95990.log.gz 2023-08-09T15:13:04.256 INFO:teuthology.orchestra.run.smithi043.stderr: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.147209.log.gz 2023-08-09T15:13:04.257 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.125415.log: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.125415.log.gz 2023-08-09T15:13:04.257 INFO:teuthology.orchestra.run.smithi043.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.94138.log 2023-08-09T15:13:04.258 INFO:teuthology.orchestra.run.smithi043.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.96313.log 2023-08-09T15:13:04.259 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.113093.log: /var/log/ceph/ceph-client.admin.94138.log: 88.7%gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.109365.log 2023-08-09T15:13:04.260 INFO:teuthology.orchestra.run.smithi043.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.94138.log.gz 2023-08-09T15:13:04.260 INFO:teuthology.orchestra.run.smithi043.stderr: -- replaced with /var/log/ceph/ceph-client.admin.113093.log.gz 2023-08-09T15:13:04.261 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.96313.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.104041.log 2023-08-09T15:13:04.261 INFO:teuthology.orchestra.run.smithi043.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.96313.log.gz 2023-08-09T15:13:04.262 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.109365.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.109628.log 2023-08-09T15:13:04.262 INFO:teuthology.orchestra.run.smithi043.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.109365.log.gz 2023-08-09T15:13:04.263 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.104041.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.157621.log 2023-08-09T15:13:04.264 INFO:teuthology.orchestra.run.smithi043.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.104041.log.gz 2023-08-09T15:13:04.265 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.109628.log: 87.2%gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.124783.log 2023-08-09T15:13:04.265 INFO:teuthology.orchestra.run.smithi043.stderr: -- replaced with /var/log/ceph/ceph-client.admin.109628.log.gz 2023-08-09T15:13:04.267 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.157621.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.122253.log 2023-08-09T15:13:04.267 INFO:teuthology.orchestra.run.smithi043.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.157621.log.gz 2023-08-09T15:13:04.268 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.124783.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.104727.log 2023-08-09T15:13:04.268 INFO:teuthology.orchestra.run.smithi043.stderr: 91.2% -- replaced with /var/log/ceph/ceph-client.admin.124783.log.gz 2023-08-09T15:13:04.269 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.122253.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.113949.log 2023-08-09T15:13:04.269 INFO:teuthology.orchestra.run.smithi043.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.122253.log.gz 2023-08-09T15:13:04.270 INFO:teuthology.orchestra.run.smithi043.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.138119.log 2023-08-09T15:13:04.273 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.104727.log: /var/log/ceph/ceph-client.admin.113949.log: 88.2%gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.126108.log 2023-08-09T15:13:04.273 INFO:teuthology.orchestra.run.smithi043.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.113949.log.gz 2023-08-09T15:13:04.273 INFO:teuthology.orchestra.run.smithi043.stderr: -- replaced with /var/log/ceph/ceph-client.admin.104727.log.gz 2023-08-09T15:13:04.274 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.138119.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.104251.log 2023-08-09T15:13:04.274 INFO:teuthology.orchestra.run.smithi043.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.138119.log.gz 2023-08-09T15:13:04.275 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.126108.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.121420.log 2023-08-09T15:13:04.276 INFO:teuthology.orchestra.run.smithi043.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.126108.log.gz 2023-08-09T15:13:04.277 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.104251.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.154246.log 2023-08-09T15:13:04.277 INFO:teuthology.orchestra.run.smithi043.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.104251.log.gz 2023-08-09T15:13:04.278 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.121420.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.148630.log 2023-08-09T15:13:04.279 INFO:teuthology.orchestra.run.smithi043.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.121420.log.gz 2023-08-09T15:13:04.280 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.154246.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.105495.log 2023-08-09T15:13:04.280 INFO:teuthology.orchestra.run.smithi043.stderr: 89.5% -- replaced with /var/log/ceph/ceph-client.admin.154246.log.gz 2023-08-09T15:13:04.281 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.148630.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.124531.log 2023-08-09T15:13:04.282 INFO:teuthology.orchestra.run.smithi043.stderr: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.148630.log.gz 2023-08-09T15:13:04.283 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.105495.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.151865.log 2023-08-09T15:13:04.283 INFO:teuthology.orchestra.run.smithi043.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.105495.log.gz 2023-08-09T15:13:04.284 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.124531.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.97446.log 2023-08-09T15:13:04.284 INFO:teuthology.orchestra.run.smithi043.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.124531.log.gz 2023-08-09T15:13:04.285 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.151865.log: 88.0%gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.99874.log 2023-08-09T15:13:04.286 INFO:teuthology.orchestra.run.smithi043.stderr: -- replaced with /var/log/ceph/ceph-client.admin.151865.log.gz 2023-08-09T15:13:04.287 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.97446.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.151720.log 2023-08-09T15:13:04.287 INFO:teuthology.orchestra.run.smithi043.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.97446.log.gz 2023-08-09T15:13:04.288 INFO:teuthology.orchestra.run.smithi043.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.100378.log 2023-08-09T15:13:04.290 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.99874.log: /var/log/ceph/ceph-client.admin.151720.log: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.99874.log.gzgzip -5 --verbose -- /var/log/ceph/ceph-client.admin.141700.log 2023-08-09T15:13:04.290 INFO:teuthology.orchestra.run.smithi043.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.151720.log.gz 2023-08-09T15:13:04.291 INFO:teuthology.orchestra.run.smithi043.stderr: 2023-08-09T15:13:04.292 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.100378.log: 88.6%gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.118839.log 2023-08-09T15:13:04.292 INFO:teuthology.orchestra.run.smithi043.stderr: -- replaced with /var/log/ceph/ceph-client.admin.100378.log.gz 2023-08-09T15:13:04.293 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.141700.log: 87.3%gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.146869.log 2023-08-09T15:13:04.293 INFO:teuthology.orchestra.run.smithi043.stderr: -- replaced with /var/log/ceph/ceph-client.admin.141700.log.gz 2023-08-09T15:13:04.295 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.118839.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.138035.log 2023-08-09T15:13:04.295 INFO:teuthology.orchestra.run.smithi043.stderr: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.118839.log.gz 2023-08-09T15:13:04.296 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.146869.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.146869.log.gz 2023-08-09T15:13:04.297 INFO:teuthology.orchestra.run.smithi043.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.112293.log 2023-08-09T15:13:04.298 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.138035.log: 89.1%gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.154162.log 2023-08-09T15:13:04.298 INFO:teuthology.orchestra.run.smithi043.stderr: -- replaced with /var/log/ceph/ceph-client.admin.138035.log.gz 2023-08-09T15:13:04.300 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.112293.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.131685.log 2023-08-09T15:13:04.300 INFO:teuthology.orchestra.run.smithi043.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.112293.log.gz 2023-08-09T15:13:04.301 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.154162.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.132964.log 2023-08-09T15:13:04.301 INFO:teuthology.orchestra.run.smithi043.stderr: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.154162.log.gz 2023-08-09T15:13:04.303 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.131685.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.118214.log 2023-08-09T15:13:04.303 INFO:teuthology.orchestra.run.smithi043.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.131685.log.gz 2023-08-09T15:13:04.304 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.132964.log: gzip -5 88.3% -- replaced with /var/log/ceph/ceph-client.admin.132964.log.gz 2023-08-09T15:13:04.304 INFO:teuthology.orchestra.run.smithi043.stderr: --verbose -- /var/log/ceph/ceph-client.admin.116020.log 2023-08-09T15:13:04.306 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.118214.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.118214.log.gz 2023-08-09T15:13:04.306 INFO:teuthology.orchestra.run.smithi043.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.119721.log 2023-08-09T15:13:04.307 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.116020.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.116020.log.gz 2023-08-09T15:13:04.307 INFO:teuthology.orchestra.run.smithi043.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.99748.log 2023-08-09T15:13:04.309 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.119721.log: gzip -5 88.9% -- replaced with /var/log/ceph/ceph-client.admin.119721.log.gz 2023-08-09T15:13:04.309 INFO:teuthology.orchestra.run.smithi043.stderr: --verbose -- /var/log/ceph/ceph-client.admin.125673.log 2023-08-09T15:13:04.310 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.99748.log: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.99748.log.gz 2023-08-09T15:13:04.311 INFO:teuthology.orchestra.run.smithi043.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.0.105196.log 2023-08-09T15:13:04.312 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.125673.log: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.125673.log.gz 2023-08-09T15:13:04.312 INFO:teuthology.orchestra.run.smithi043.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.128206.log 2023-08-09T15:13:04.314 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.0.105196.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.138558.log 2023-08-09T15:13:04.315 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.128206.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.128206.log.gz 2023-08-09T15:13:04.316 INFO:teuthology.orchestra.run.smithi043.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.153122.log 2023-08-09T15:13:04.317 INFO:teuthology.orchestra.run.smithi043.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.124387.log 2023-08-09T15:13:04.318 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.138558.log: /var/log/ceph/ceph-client.admin.153122.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.138558.log.gz 2023-08-09T15:13:04.319 INFO:teuthology.orchestra.run.smithi043.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.153122.log.gz 2023-08-09T15:13:04.319 INFO:teuthology.orchestra.run.smithi043.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.130823.log 2023-08-09T15:13:04.321 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.124387.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.123402.log 2023-08-09T15:13:04.322 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.130823.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.124387.log.gz 2023-08-09T15:13:04.323 INFO:teuthology.orchestra.run.smithi043.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.130823.log.gz 2023-08-09T15:13:04.323 INFO:teuthology.orchestra.run.smithi043.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.126901.log 2023-08-09T15:13:04.325 INFO:teuthology.orchestra.run.smithi043.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.97404.log 2023-08-09T15:13:04.326 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.123402.log: /var/log/ceph/ceph-client.admin.126901.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.123402.log.gz 2023-08-09T15:13:04.327 INFO:teuthology.orchestra.run.smithi043.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.126901.log.gz 2023-08-09T15:13:04.327 INFO:teuthology.orchestra.run.smithi043.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.125307.log 2023-08-09T15:13:04.330 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.97404.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.93048.log 2023-08-09T15:13:04.330 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.125307.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.97404.log.gz 2023-08-09T15:13:04.331 INFO:teuthology.orchestra.run.smithi043.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.125307.log.gz 2023-08-09T15:13:04.332 INFO:teuthology.orchestra.run.smithi043.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.102187.log 2023-08-09T15:13:04.335 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.93048.log: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.93048.log.gzgzip -5 --verbose -- /var/log/ceph/ceph-client.admin.113711.log 2023-08-09T15:13:04.335 INFO:teuthology.orchestra.run.smithi043.stderr: 2023-08-09T15:13:04.336 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.102187.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.102187.log.gzgzip -5 --verbose -- /var/log/ceph/ceph-client.admin.131433.log 2023-08-09T15:13:04.336 INFO:teuthology.orchestra.run.smithi043.stderr: 2023-08-09T15:13:04.338 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.113711.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.137391.log 2023-08-09T15:13:04.339 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.131433.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.113711.log.gz 2023-08-09T15:13:04.339 INFO:teuthology.orchestra.run.smithi043.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.131433.log.gz 2023-08-09T15:13:04.341 INFO:teuthology.orchestra.run.smithi043.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.105878.log 2023-08-09T15:13:04.342 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.137391.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.132670.log 2023-08-09T15:13:04.342 INFO:teuthology.orchestra.run.smithi043.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.137391.log.gz 2023-08-09T15:13:04.344 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.105878.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.103063.log 2023-08-09T15:13:04.345 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.132670.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.105878.log.gz 2023-08-09T15:13:04.345 INFO:teuthology.orchestra.run.smithi043.stderr: 89.6% -- replaced with /var/log/ceph/ceph-client.admin.132670.log.gz 2023-08-09T15:13:04.346 INFO:teuthology.orchestra.run.smithi043.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.102475.log 2023-08-09T15:13:04.348 INFO:teuthology.orchestra.run.smithi043.stderr:gzip -5 --verbose/var/log/ceph/ceph-client.admin.103063.log: -- /var/log/ceph/ceph-client.admin.121060.log 2023-08-09T15:13:04.348 INFO:teuthology.orchestra.run.smithi043.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.103063.log.gz 2023-08-09T15:13:04.350 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.102475.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.104601.log 2023-08-09T15:13:04.351 INFO:teuthology.orchestra.run.smithi043.stderr: 87.5%/var/log/ceph/ceph-client.admin.121060.log: -- replaced with /var/log/ceph/ceph-client.admin.102475.log.gz 2023-08-09T15:13:04.351 INFO:teuthology.orchestra.run.smithi043.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.121060.log.gz 2023-08-09T15:13:04.353 INFO:teuthology.orchestra.run.smithi043.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.126570.log 2023-08-09T15:13:04.354 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.104601.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.116062.log 2023-08-09T15:13:04.355 INFO:teuthology.orchestra.run.smithi043.stderr: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.104601.log.gz 2023-08-09T15:13:04.355 INFO:teuthology.orchestra.run.smithi043.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.155916.log 2023-08-09T15:13:04.357 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.126570.log: /var/log/ceph/ceph-client.admin.116062.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.126570.log.gz 2023-08-09T15:13:04.357 INFO:teuthology.orchestra.run.smithi043.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.116062.log.gz 2023-08-09T15:13:04.358 INFO:teuthology.orchestra.run.smithi043.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.127496.log 2023-08-09T15:13:04.359 INFO:teuthology.orchestra.run.smithi043.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.122837.log 2023-08-09T15:13:04.360 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.155916.log: /var/log/ceph/ceph-client.admin.127496.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.155916.log.gz 2023-08-09T15:13:04.361 INFO:teuthology.orchestra.run.smithi043.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.127496.log.gz 2023-08-09T15:13:04.362 INFO:teuthology.orchestra.run.smithi043.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.0.118491.log 2023-08-09T15:13:04.364 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.122837.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.103609.log 2023-08-09T15:13:04.365 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.0.118491.log: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.122837.log.gz 2023-08-09T15:13:04.368 INFO:teuthology.orchestra.run.smithi043.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.102391.log 2023-08-09T15:13:04.368 INFO:teuthology.orchestra.run.smithi043.stderr: 91.6% -- replaced with /var/log/ceph/ceph-client.0.118491.log.gz 2023-08-09T15:13:04.369 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.103609.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.132110.log 2023-08-09T15:13:04.370 INFO:teuthology.orchestra.run.smithi043.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.103609.log.gz 2023-08-09T15:13:04.370 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.102391.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.102391.log.gz 2023-08-09T15:13:04.371 INFO:teuthology.orchestra.run.smithi043.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.100951.log 2023-08-09T15:13:04.372 INFO:teuthology.orchestra.run.smithi043.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.117667.log 2023-08-09T15:13:04.373 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.132110.log: /var/log/ceph/ceph-client.admin.100951.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.132110.log.gz 2023-08-09T15:13:04.374 INFO:teuthology.orchestra.run.smithi043.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.100951.log.gz 2023-08-09T15:13:04.374 INFO:teuthology.orchestra.run.smithi043.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.93946.log 2023-08-09T15:13:04.376 INFO:teuthology.orchestra.run.smithi043.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.126318.log 2023-08-09T15:13:04.377 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.117667.log: /var/log/ceph/ceph-client.admin.93946.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.117667.log.gz 2023-08-09T15:13:04.378 INFO:teuthology.orchestra.run.smithi043.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.93946.log.gz 2023-08-09T15:13:04.378 INFO:teuthology.orchestra.run.smithi043.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.111782.log 2023-08-09T15:13:04.380 INFO:teuthology.orchestra.run.smithi043.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.95430.log 2023-08-09T15:13:04.381 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.126318.log: /var/log/ceph/ceph-client.admin.111782.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.126318.log.gz 2023-08-09T15:13:04.382 INFO:teuthology.orchestra.run.smithi043.stderr: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.111782.log.gz 2023-08-09T15:13:04.383 INFO:teuthology.orchestra.run.smithi043.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.128713.log 2023-08-09T15:13:04.384 INFO:teuthology.orchestra.run.smithi043.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.93276.log 2023-08-09T15:13:04.385 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.95430.log: /var/log/ceph/ceph-client.admin.128713.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.95430.log.gz 2023-08-09T15:13:04.386 INFO:teuthology.orchestra.run.smithi043.stderr: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.128713.log.gz 2023-08-09T15:13:04.386 INFO:teuthology.orchestra.run.smithi043.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.153020.log 2023-08-09T15:13:04.388 INFO:teuthology.orchestra.run.smithi043.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.130950.log 2023-08-09T15:13:04.389 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.93276.log: /var/log/ceph/ceph-client.admin.153020.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.93276.log.gz 2023-08-09T15:13:04.390 INFO:teuthology.orchestra.run.smithi043.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.153020.log.gz 2023-08-09T15:13:04.390 INFO:teuthology.orchestra.run.smithi043.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.119190.log 2023-08-09T15:13:04.392 INFO:teuthology.orchestra.run.smithi043.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.96324.log 2023-08-09T15:13:04.393 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.130950.log: /var/log/ceph/ceph-client.admin.119190.log: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.130950.log.gz 2023-08-09T15:13:04.394 INFO:teuthology.orchestra.run.smithi043.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.119190.log.gz 2023-08-09T15:13:04.394 INFO:teuthology.orchestra.run.smithi043.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.142311.log 2023-08-09T15:13:04.395 INFO:teuthology.orchestra.run.smithi043.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.108576.log 2023-08-09T15:13:04.397 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.96324.log: /var/log/ceph/ceph-client.admin.142311.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.96324.log.gz 2023-08-09T15:13:04.397 INFO:teuthology.orchestra.run.smithi043.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.142311.log.gz 2023-08-09T15:13:04.398 INFO:teuthology.orchestra.run.smithi043.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.122337.log 2023-08-09T15:13:04.399 INFO:teuthology.orchestra.run.smithi043.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.107231.log 2023-08-09T15:13:04.401 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.108576.log: /var/log/ceph/ceph-client.admin.122337.log: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.108576.log.gz 2023-08-09T15:13:04.401 INFO:teuthology.orchestra.run.smithi043.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.122337.log.gz 2023-08-09T15:13:04.402 INFO:teuthology.orchestra.run.smithi043.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.142211.log 2023-08-09T15:13:04.403 INFO:teuthology.orchestra.run.smithi043.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.125776.log 2023-08-09T15:13:04.404 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.107231.log: /var/log/ceph/ceph-client.admin.142211.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.107231.log.gz 2023-08-09T15:13:04.405 INFO:teuthology.orchestra.run.smithi043.stderr: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.142211.log.gz 2023-08-09T15:13:04.406 INFO:teuthology.orchestra.run.smithi043.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.145992.log 2023-08-09T15:13:04.407 INFO:teuthology.orchestra.run.smithi043.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.153772.log 2023-08-09T15:13:04.409 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.125776.log: /var/log/ceph/ceph-client.admin.145992.log: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.125776.log.gz 2023-08-09T15:13:04.409 INFO:teuthology.orchestra.run.smithi043.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.145992.log.gz 2023-08-09T15:13:04.410 INFO:teuthology.orchestra.run.smithi043.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.148743.log 2023-08-09T15:13:04.412 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.153772.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.152201.log 2023-08-09T15:13:04.413 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.148743.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.153772.log.gz 2023-08-09T15:13:04.414 INFO:teuthology.orchestra.run.smithi043.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.148743.log.gz 2023-08-09T15:13:04.414 INFO:teuthology.orchestra.run.smithi043.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.118738.log 2023-08-09T15:13:04.415 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.152201.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.151468.log 2023-08-09T15:13:04.417 INFO:teuthology.orchestra.run.smithi043.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.152201.log.gz 2023-08-09T15:13:04.418 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.118738.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.142395.log 2023-08-09T15:13:04.419 INFO:teuthology.orchestra.run.smithi043.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.118738.log.gz 2023-08-09T15:13:04.419 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.151468.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.151468.log.gz 2023-08-09T15:13:04.421 INFO:teuthology.orchestra.run.smithi043.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.152798.log 2023-08-09T15:13:04.422 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.142395.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.133380.log 2023-08-09T15:13:04.422 INFO:teuthology.orchestra.run.smithi043.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.142395.log.gz 2023-08-09T15:13:04.423 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.152798.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.116440.log 2023-08-09T15:13:04.424 INFO:teuthology.orchestra.run.smithi043.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.152798.log.gz 2023-08-09T15:13:04.426 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.133380.log: gzip -5 --verbose -- /var/log/ceph/ceph-mgr.y.log 2023-08-09T15:13:04.427 INFO:teuthology.orchestra.run.smithi043.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.133380.log.gz 2023-08-09T15:13:04.428 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.116440.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.116440.log.gz 2023-08-09T15:13:04.428 INFO:teuthology.orchestra.run.smithi043.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.153814.log 2023-08-09T15:13:04.430 INFO:teuthology.orchestra.run.smithi043.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.126486.log 2023-08-09T15:13:04.431 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-mgr.y.log: /var/log/ceph/ceph-client.admin.153814.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.153814.log.gz 2023-08-09T15:13:04.432 INFO:teuthology.orchestra.run.smithi043.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.156126.log 2023-08-09T15:13:04.434 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.126486.log: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.126486.log.gz 2023-08-09T15:13:04.435 INFO:teuthology.orchestra.run.smithi043.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.156481.log 2023-08-09T15:13:04.437 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.156126.log: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.156126.log.gz 2023-08-09T15:13:04.438 INFO:teuthology.orchestra.run.smithi043.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.119385.log 2023-08-09T15:13:04.440 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.156481.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.156481.log.gz 2023-08-09T15:13:04.441 INFO:teuthology.orchestra.run.smithi043.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.156974.log 2023-08-09T15:13:04.443 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.119385.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.119385.log.gz 2023-08-09T15:13:04.445 INFO:teuthology.orchestra.run.smithi043.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.115688.log 2023-08-09T15:13:04.446 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.156974.log: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.156974.log.gz 2023-08-09T15:13:04.448 INFO:teuthology.orchestra.run.smithi043.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.103231.log 2023-08-09T15:13:04.450 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.115688.log: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.115688.log.gz 2023-08-09T15:13:04.451 INFO:teuthology.orchestra.run.smithi043.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.104853.log 2023-08-09T15:13:04.453 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.103231.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.103231.log.gz 2023-08-09T15:13:04.454 INFO:teuthology.orchestra.run.smithi043.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.121228.log 2023-08-09T15:13:04.457 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.104853.log: 91.1% -- replaced with /var/log/ceph/ceph-client.admin.104853.log.gz 2023-08-09T15:13:04.458 INFO:teuthology.orchestra.run.smithi043.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.140898.log 2023-08-09T15:13:04.460 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.121228.log: 93.8% -- replaced with /var/log/ceph/ceph-client.0.105196.log.gz 2023-08-09T15:13:04.460 INFO:teuthology.orchestra.run.smithi043.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.121228.log.gz 2023-08-09T15:13:04.461 INFO:teuthology.orchestra.run.smithi043.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.120838.log 2023-08-09T15:13:04.462 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.140898.log: gzip -5 --verbose -- /var/log/ceph/ceph-osd.3.log 2023-08-09T15:13:04.463 INFO:teuthology.orchestra.run.smithi043.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.140898.log.gz 2023-08-09T15:13:04.465 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.120838.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.102103.log 2023-08-09T15:13:04.465 INFO:teuthology.orchestra.run.smithi043.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.120838.log.gz 2023-08-09T15:13:04.472 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-osd.3.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.110825.log 2023-08-09T15:13:04.474 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.102103.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.102103.log.gz 2023-08-09T15:13:04.475 INFO:teuthology.orchestra.run.smithi043.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.112967.log 2023-08-09T15:13:04.477 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.110825.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.110825.log.gz 2023-08-09T15:13:04.479 INFO:teuthology.orchestra.run.smithi043.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.157268.log 2023-08-09T15:13:04.480 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.112967.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.112967.log.gz 2023-08-09T15:13:04.482 INFO:teuthology.orchestra.run.smithi043.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.0.128482.log 2023-08-09T15:13:04.483 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.157268.log: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.157268.log.gz 2023-08-09T15:13:04.485 INFO:teuthology.orchestra.run.smithi043.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.112715.log 2023-08-09T15:13:04.498 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.0.128482.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.157058.log 2023-08-09T15:13:04.505 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.112715.log: 88.5%gzip -5 --verbose -- /var/log/ceph/ceph-client.0.143425.log 2023-08-09T15:13:04.511 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.157058.log: -- replaced with /var/log/ceph/ceph-client.admin.112715.log.gz 2023-08-09T15:13:04.519 INFO:teuthology.orchestra.run.smithi043.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.157058.log.gz 2023-08-09T15:13:04.525 INFO:teuthology.orchestra.run.smithi043.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.132838.log 2023-08-09T15:13:04.548 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.0.143425.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.117541.log 2023-08-09T15:13:04.562 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.132838.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.93128.log 2023-08-09T15:13:04.563 INFO:teuthology.orchestra.run.smithi043.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.132838.log.gz 2023-08-09T15:13:04.563 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.117541.log: 95.0% -- replaced with /var/log/ceph/ceph-client.0.143425.log.gz 2023-08-09T15:13:04.563 INFO:teuthology.orchestra.run.smithi043.stderr: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.117541.log.gz 2023-08-09T15:13:04.567 INFO:teuthology.orchestra.run.smithi043.stderr: 94.2% -- replaced with /var/log/ceph/ceph-client.0.128482.log.gz 2023-08-09T15:13:04.568 INFO:teuthology.orchestra.run.smithi043.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.116356.log 2023-08-09T15:13:04.570 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.93128.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.93128.log.gz 2023-08-09T15:13:04.571 INFO:teuthology.orchestra.run.smithi043.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.97572.log 2023-08-09T15:13:04.573 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.116356.log: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.116356.log.gz 2023-08-09T15:13:04.574 INFO:teuthology.orchestra.run.smithi043.stderr: 93.4% -- replaced with /var/log/ceph/ceph-mgr.y.log.gz 2023-08-09T15:13:04.575 INFO:teuthology.orchestra.run.smithi043.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.120668.log 2023-08-09T15:13:04.576 INFO:teuthology.orchestra.run.smithi043.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.103189.log 2023-08-09T15:13:04.577 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.97572.log: /var/log/ceph/ceph-client.admin.120668.log: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.97572.log.gz 2023-08-09T15:13:04.578 INFO:teuthology.orchestra.run.smithi043.stderr: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.120668.log.gz 2023-08-09T15:13:04.578 INFO:teuthology.orchestra.run.smithi043.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.142787.log 2023-08-09T15:13:04.580 INFO:teuthology.orchestra.run.smithi043.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.94940.log 2023-08-09T15:13:04.581 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.103189.log: /var/log/ceph/ceph-client.admin.142787.log: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.103189.log.gz 2023-08-09T15:13:04.582 INFO:teuthology.orchestra.run.smithi043.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.142787.log.gz 2023-08-09T15:13:04.582 INFO:teuthology.orchestra.run.smithi043.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.147516.log 2023-08-09T15:13:04.585 INFO:teuthology.orchestra.run.smithi043.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.151258.log 2023-08-09T15:13:04.587 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.94940.log: /var/log/ceph/ceph-client.admin.147516.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.94940.log.gz 2023-08-09T15:13:04.588 INFO:teuthology.orchestra.run.smithi043.stderr: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.147516.log.gz 2023-08-09T15:13:04.588 INFO:teuthology.orchestra.run.smithi043.stderr:gzip -5 --verbose -- /var/log/ceph/ceph.log 2023-08-09T15:13:04.589 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.151258.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.115414.log 2023-08-09T15:13:04.589 INFO:teuthology.orchestra.run.smithi043.stderr: 89.5% -- replaced with /var/log/ceph/ceph-client.admin.151258.log.gz 2023-08-09T15:13:04.590 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.100000.log 2023-08-09T15:13:04.592 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.115414.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.115414.log.gz 2023-08-09T15:13:04.593 INFO:teuthology.orchestra.run.smithi043.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.122474.log 2023-08-09T15:13:04.595 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.100000.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.100000.log.gz 2023-08-09T15:13:04.597 INFO:teuthology.orchestra.run.smithi043.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.117243.log 2023-08-09T15:13:04.599 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.122474.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.122474.log.gz 2023-08-09T15:13:04.600 INFO:teuthology.orchestra.run.smithi043.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.123276.log 2023-08-09T15:13:04.602 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.117243.log: 89.4% -- replaced with /var/log/ceph/ceph.log.gz 2023-08-09T15:13:04.602 INFO:teuthology.orchestra.run.smithi043.stderr: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.117243.log.gz 2023-08-09T15:13:04.603 INFO:teuthology.orchestra.run.smithi043.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.119847.log 2023-08-09T15:13:04.604 INFO:teuthology.orchestra.run.smithi043.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.156806.log 2023-08-09T15:13:04.606 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.123276.log: /var/log/ceph/ceph-client.admin.119847.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.123276.log.gz 2023-08-09T15:13:04.606 INFO:teuthology.orchestra.run.smithi043.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.119847.log.gz 2023-08-09T15:13:04.607 INFO:teuthology.orchestra.run.smithi043.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.120327.log 2023-08-09T15:13:04.609 INFO:teuthology.orchestra.run.smithi043.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.93562.log 2023-08-09T15:13:04.609 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.156806.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.156806.log.gz 2023-08-09T15:13:04.611 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.120327.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.116650.log 2023-08-09T15:13:04.611 INFO:teuthology.orchestra.run.smithi043.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.120327.log.gz/var/log/ceph/ceph-client.admin.93562.log: 2023-08-09T15:13:04.612 INFO:teuthology.orchestra.run.smithi043.stderr: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.93562.log.gz 2023-08-09T15:13:04.614 INFO:teuthology.orchestra.run.smithi043.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.97320.log 2023-08-09T15:13:04.615 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.116650.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.117411.log 2023-08-09T15:13:04.615 INFO:teuthology.orchestra.run.smithi043.stderr: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.116650.log.gz 2023-08-09T15:13:04.616 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.97320.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.138161.log 2023-08-09T15:13:04.617 INFO:teuthology.orchestra.run.smithi043.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.97320.log.gz 2023-08-09T15:13:04.618 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.117411.log: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.117411.log.gz 2023-08-09T15:13:04.620 INFO:teuthology.orchestra.run.smithi043.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.123745.log 2023-08-09T15:13:04.621 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.138161.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.98966.log 2023-08-09T15:13:04.622 INFO:teuthology.orchestra.run.smithi043.stderr: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.138161.log.gz 2023-08-09T15:13:04.623 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.123745.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.138348.log 2023-08-09T15:13:04.624 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.98966.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.123745.log.gz 2023-08-09T15:13:04.624 INFO:teuthology.orchestra.run.smithi043.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.98966.log.gz 2023-08-09T15:13:04.625 INFO:teuthology.orchestra.run.smithi043.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.104559.log 2023-08-09T15:13:04.627 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.138348.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.101835.log 2023-08-09T15:13:04.627 INFO:teuthology.orchestra.run.smithi043.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.138348.log.gz 2023-08-09T15:13:04.629 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.104559.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.124081.log 2023-08-09T15:13:04.629 INFO:teuthology.orchestra.run.smithi043.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.104559.log.gz 2023-08-09T15:13:04.630 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.101835.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.101835.log.gz 2023-08-09T15:13:04.632 INFO:teuthology.orchestra.run.smithi043.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.132194.log 2023-08-09T15:13:04.633 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.124081.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.152541.log 2023-08-09T15:13:04.634 INFO:teuthology.orchestra.run.smithi043.stderr: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.124081.log.gz 2023-08-09T15:13:04.635 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.132194.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.120075.log 2023-08-09T15:13:04.636 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.152541.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.132194.log.gz 2023-08-09T15:13:04.636 INFO:teuthology.orchestra.run.smithi043.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.152541.log.gz 2023-08-09T15:13:04.637 INFO:teuthology.orchestra.run.smithi043.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.140856.log 2023-08-09T15:13:04.639 INFO:teuthology.orchestra.run.smithi043.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.94856.log 2023-08-09T15:13:04.639 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.120075.log: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.120075.log.gz 2023-08-09T15:13:04.641 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.140856.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.141177.log 2023-08-09T15:13:04.642 INFO:teuthology.orchestra.run.smithi043.stderr: 88.0%/var/log/ceph/ceph-client.admin.94856.log: -- replaced with /var/log/ceph/ceph-client.admin.140856.log.gz 2023-08-09T15:13:04.642 INFO:teuthology.orchestra.run.smithi043.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.94856.log.gz 2023-08-09T15:13:04.643 INFO:teuthology.orchestra.run.smithi043.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.130628.log 2023-08-09T15:13:04.644 INFO:teuthology.orchestra.run.smithi043.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.140403.log 2023-08-09T15:13:04.645 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.141177.log: /var/log/ceph/ceph-client.admin.130628.log: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.141177.log.gz 2023-08-09T15:13:04.646 INFO:teuthology.orchestra.run.smithi043.stderr: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.130628.log.gz 2023-08-09T15:13:04.647 INFO:teuthology.orchestra.run.smithi043.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.121270.log 2023-08-09T15:13:04.648 INFO:teuthology.orchestra.run.smithi043.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.122879.log 2023-08-09T15:13:04.649 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.140403.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.140403.log.gz 2023-08-09T15:13:04.651 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.121270.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.148571.log 2023-08-09T15:13:04.651 INFO:teuthology.orchestra.run.smithi043.stderr: 88.3%/var/log/ceph/ceph-client.admin.122879.log: -- replaced with /var/log/ceph/ceph-client.admin.121270.log.gz 2023-08-09T15:13:04.652 INFO:teuthology.orchestra.run.smithi043.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.122879.log.gz 2023-08-09T15:13:04.653 INFO:teuthology.orchestra.run.smithi043.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.142659.log 2023-08-09T15:13:04.655 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.148571.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.0.154969.log 2023-08-09T15:13:04.655 INFO:teuthology.orchestra.run.smithi043.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.148571.log.gz 2023-08-09T15:13:04.656 INFO:teuthology.orchestra.run.smithi043.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.131204.log 2023-08-09T15:13:04.657 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.142659.log: /var/log/ceph/ceph-client.0.154969.log: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.142659.log.gz 2023-08-09T15:13:04.659 INFO:teuthology.orchestra.run.smithi043.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.156680.log 2023-08-09T15:13:04.661 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.131204.log: 92.4% -- replaced with /var/log/ceph/ceph-client.0.154969.log.gz 2023-08-09T15:13:04.661 INFO:teuthology.orchestra.run.smithi043.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.131204.log.gz 2023-08-09T15:13:04.662 INFO:teuthology.orchestra.run.smithi043.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.150968.log 2023-08-09T15:13:04.664 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.156680.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.156680.log.gz 2023-08-09T15:13:04.665 INFO:teuthology.orchestra.run.smithi043.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.157352.log 2023-08-09T15:13:04.666 INFO:teuthology.orchestra.run.smithi043.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.130533.log 2023-08-09T15:13:04.667 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.150968.log: /var/log/ceph/ceph-client.admin.157352.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.150968.log.gz 2023-08-09T15:13:04.668 INFO:teuthology.orchestra.run.smithi043.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.157352.log.gz 2023-08-09T15:13:04.668 INFO:teuthology.orchestra.run.smithi043.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.101677.log 2023-08-09T15:13:04.670 INFO:teuthology.orchestra.run.smithi043.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.120159.log 2023-08-09T15:13:04.671 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.130533.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.130533.log.gz 2023-08-09T15:13:04.672 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.101677.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.118022.log 2023-08-09T15:13:04.673 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.120159.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.101677.log.gz 2023-08-09T15:13:04.674 INFO:teuthology.orchestra.run.smithi043.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.120159.log.gz 2023-08-09T15:13:04.674 INFO:teuthology.orchestra.run.smithi043.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.143606.log 2023-08-09T15:13:04.676 INFO:teuthology.orchestra.run.smithi043.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.122038.log 2023-08-09T15:13:04.677 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.118022.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.118022.log.gz 2023-08-09T15:13:04.678 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.143606.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.105387.log 2023-08-09T15:13:04.679 INFO:teuthology.orchestra.run.smithi043.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.143606.log.gz 2023-08-09T15:13:04.680 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.122038.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.122038.log.gz 2023-08-09T15:13:04.680 INFO:teuthology.orchestra.run.smithi043.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.96326.log 2023-08-09T15:13:04.682 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.105387.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.125510.log 2023-08-09T15:13:04.683 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.96326.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.105387.log.gz 2023-08-09T15:13:04.684 INFO:teuthology.orchestra.run.smithi043.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.96326.log.gz 2023-08-09T15:13:04.684 INFO:teuthology.orchestra.run.smithi043.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.131643.log 2023-08-09T15:13:04.686 INFO:teuthology.orchestra.run.smithi043.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.124849.log 2023-08-09T15:13:04.687 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.125510.log: /var/log/ceph/ceph-client.admin.131643.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.125510.log.gz 2023-08-09T15:13:04.688 INFO:teuthology.orchestra.run.smithi043.stderr: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.131643.log.gz 2023-08-09T15:13:04.688 INFO:teuthology.orchestra.run.smithi043.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.131330.log 2023-08-09T15:13:04.690 INFO:teuthology.orchestra.run.smithi043.stderr:gzip -5/var/log/ceph/ceph-client.admin.124849.log: --verbose -- /var/log/ceph/ceph-client.admin.127888.log 2023-08-09T15:13:04.691 INFO:teuthology.orchestra.run.smithi043.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.124849.log.gz 2023-08-09T15:13:04.692 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.131330.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.125912.log 2023-08-09T15:13:04.692 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.127888.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.131330.log.gz 2023-08-09T15:13:04.693 INFO:teuthology.orchestra.run.smithi043.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.127888.log.gz 2023-08-09T15:13:04.694 INFO:teuthology.orchestra.run.smithi043.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.142036.log 2023-08-09T15:13:04.696 INFO:teuthology.orchestra.run.smithi043.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.115825.log 2023-08-09T15:13:04.696 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.125912.log: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.125912.log.gz 2023-08-09T15:13:04.698 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.142036.log: 88.5%gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.152848.log 2023-08-09T15:13:04.698 INFO:teuthology.orchestra.run.smithi043.stderr: -- replaced with /var/log/ceph/ceph-client.admin.142036.log.gz 2023-08-09T15:13:04.699 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.115825.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.115825.log.gz 2023-08-09T15:13:04.700 INFO:teuthology.orchestra.run.smithi043.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.146911.log 2023-08-09T15:13:04.701 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.152848.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.116776.log 2023-08-09T15:13:04.702 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.146911.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.152848.log.gz 2023-08-09T15:13:04.703 INFO:teuthology.orchestra.run.smithi043.stderr: 87.2% -- replaced with /var/log/ceph/ceph-client.admin.146911.log.gz 2023-08-09T15:13:04.703 INFO:teuthology.orchestra.run.smithi043.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.140616.log 2023-08-09T15:13:04.705 INFO:teuthology.orchestra.run.smithi043.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.154556.log 2023-08-09T15:13:04.706 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.116776.log: /var/log/ceph/ceph-client.admin.140616.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.116776.log.gz 2023-08-09T15:13:04.707 INFO:teuthology.orchestra.run.smithi043.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.140616.log.gz 2023-08-09T15:13:04.708 INFO:teuthology.orchestra.run.smithi043.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.145908.log 2023-08-09T15:13:04.709 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.154556.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.152159.log 2023-08-09T15:13:04.710 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.145908.log: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.154556.log.gz 2023-08-09T15:13:04.711 INFO:teuthology.orchestra.run.smithi043.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.145908.log.gz 2023-08-09T15:13:04.712 INFO:teuthology.orchestra.run.smithi043.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.93734.log 2023-08-09T15:13:04.713 INFO:teuthology.orchestra.run.smithi043.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.93820.log 2023-08-09T15:13:04.714 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.152159.log: /var/log/ceph/ceph-client.admin.93734.log: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.152159.log.gz 2023-08-09T15:13:04.715 INFO:teuthology.orchestra.run.smithi043.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.93734.log.gz 2023-08-09T15:13:04.716 INFO:teuthology.orchestra.run.smithi043.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.155790.log 2023-08-09T15:13:04.718 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.93820.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.115978.log 2023-08-09T15:13:04.718 INFO:teuthology.orchestra.run.smithi043.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.93820.log.gz 2023-08-09T15:13:04.719 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.155790.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.155790.log.gz 2023-08-09T15:13:04.720 INFO:teuthology.orchestra.run.smithi043.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.119889.log 2023-08-09T15:13:04.721 INFO:teuthology.orchestra.run.smithi043.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.122627.log 2023-08-09T15:13:04.722 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.115978.log: /var/log/ceph/ceph-client.admin.119889.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.115978.log.gz 2023-08-09T15:13:04.723 INFO:teuthology.orchestra.run.smithi043.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.119889.log.gz 2023-08-09T15:13:04.723 INFO:teuthology.orchestra.run.smithi043.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.132898.log 2023-08-09T15:13:04.725 INFO:teuthology.orchestra.run.smithi043.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.113201.log 2023-08-09T15:13:04.726 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.122627.log: /var/log/ceph/ceph-client.admin.132898.log: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.122627.log.gz 2023-08-09T15:13:04.727 INFO:teuthology.orchestra.run.smithi043.stderr: 90.8% -- replaced with /var/log/ceph/ceph-client.admin.132898.log.gzgzip -5 --verbose -- /var/log/ceph/ceph-client.admin.106425.log 2023-08-09T15:13:04.728 INFO:teuthology.orchestra.run.smithi043.stderr: 2023-08-09T15:13:04.729 INFO:teuthology.orchestra.run.smithi043.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.100641.log 2023-08-09T15:13:04.730 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.113201.log: /var/log/ceph/ceph-client.admin.106425.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.113201.log.gz 2023-08-09T15:13:04.731 INFO:teuthology.orchestra.run.smithi043.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.106425.log.gz 2023-08-09T15:13:04.731 INFO:teuthology.orchestra.run.smithi043.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.128140.log 2023-08-09T15:13:04.733 INFO:teuthology.orchestra.run.smithi043.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.139459.log 2023-08-09T15:13:04.734 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.100641.log: /var/log/ceph/ceph-client.admin.128140.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.100641.log.gz 2023-08-09T15:13:04.735 INFO:teuthology.orchestra.run.smithi043.stderr: 91.0% -- replaced with /var/log/ceph/ceph-client.admin.128140.log.gzgzip -5 --verbose -- /var/log/ceph/ceph-mon.a.log 2023-08-09T15:13:04.735 INFO:teuthology.orchestra.run.smithi043.stderr: 2023-08-09T15:13:04.737 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.139459.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.101919.log 2023-08-09T15:13:04.738 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-mon.a.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.139459.log.gz 2023-08-09T15:13:04.739 INFO:teuthology.orchestra.run.smithi043.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.103147.log 2023-08-09T15:13:04.741 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.101919.log: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.101919.log.gz 2023-08-09T15:13:04.743 INFO:teuthology.orchestra.run.smithi043.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.155458.log 2023-08-09T15:13:04.744 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.103147.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.103147.log.gz 2023-08-09T15:13:04.746 INFO:teuthology.orchestra.run.smithi043.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.99328.log 2023-08-09T15:13:04.748 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.155458.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.155458.log.gz 2023-08-09T15:13:04.750 INFO:teuthology.orchestra.run.smithi043.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.141387.log 2023-08-09T15:13:04.752 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.99328.log: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.99328.log.gz 2023-08-09T15:13:04.753 INFO:teuthology.orchestra.run.smithi043.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.157100.log 2023-08-09T15:13:04.755 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.141387.log: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.141387.log.gz 2023-08-09T15:13:04.757 INFO:teuthology.orchestra.run.smithi043.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.128831.log 2023-08-09T15:13:04.758 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.157100.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.157100.log.gz 2023-08-09T15:13:04.760 INFO:teuthology.orchestra.run.smithi043.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.156932.log 2023-08-09T15:13:04.762 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.128831.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.128831.log.gz 2023-08-09T15:13:04.764 INFO:teuthology.orchestra.run.smithi043.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.157581.log 2023-08-09T15:13:04.766 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.156932.log: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.156932.log.gz 2023-08-09T15:13:04.767 INFO:teuthology.orchestra.run.smithi043.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.89157.log 2023-08-09T15:13:04.769 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.157581.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.157581.log.gz 2023-08-09T15:13:04.771 INFO:teuthology.orchestra.run.smithi043.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.119679.log 2023-08-09T15:13:04.773 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.89157.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.89157.log.gz 2023-08-09T15:13:04.774 INFO:teuthology.orchestra.run.smithi043.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.125457.log 2023-08-09T15:13:04.776 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.119679.log: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.119679.log.gz 2023-08-09T15:13:04.778 INFO:teuthology.orchestra.run.smithi043.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.147854.log 2023-08-09T15:13:04.780 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.125457.log: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.125457.log.gz 2023-08-09T15:13:04.782 INFO:teuthology.orchestra.run.smithi043.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.107147.log 2023-08-09T15:13:04.783 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.147854.log: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.147854.log.gz 2023-08-09T15:13:04.785 INFO:teuthology.orchestra.run.smithi043.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.116398.log 2023-08-09T15:13:04.787 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.107147.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.107147.log.gz 2023-08-09T15:13:04.789 INFO:teuthology.orchestra.run.smithi043.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.117455.log 2023-08-09T15:13:04.791 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.116398.log: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.116398.log.gz 2023-08-09T15:13:04.793 INFO:teuthology.orchestra.run.smithi043.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.152418.log 2023-08-09T15:13:04.795 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.117455.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.117455.log.gz 2023-08-09T15:13:04.796 INFO:teuthology.orchestra.run.smithi043.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.113991.log 2023-08-09T15:13:04.798 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.152418.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.152418.log.gz 2023-08-09T15:13:04.800 INFO:teuthology.orchestra.run.smithi043.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.145950.log 2023-08-09T15:13:04.802 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.113991.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.113991.log.gz 2023-08-09T15:13:04.803 INFO:teuthology.orchestra.run.smithi043.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.141135.log 2023-08-09T15:13:04.805 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.145950.log: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.145950.log.gz 2023-08-09T15:13:04.807 INFO:teuthology.orchestra.run.smithi043.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.102145.log 2023-08-09T15:13:04.809 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.141135.log: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.141135.log.gz 2023-08-09T15:13:04.811 INFO:teuthology.orchestra.run.smithi043.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.107458.log 2023-08-09T15:13:04.813 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.102145.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.102145.log.gz 2023-08-09T15:13:04.814 INFO:teuthology.orchestra.run.smithi043.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.142871.log 2023-08-09T15:13:04.816 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.107458.log: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.107458.log.gz 2023-08-09T15:13:04.818 INFO:teuthology.orchestra.run.smithi043.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.152075.log 2023-08-09T15:13:04.820 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.142871.log: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.142871.log.gz 2023-08-09T15:13:04.822 INFO:teuthology.orchestra.run.smithi043.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.100473.log 2023-08-09T15:13:04.824 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.152075.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.152075.log.gz 2023-08-09T15:13:04.825 INFO:teuthology.orchestra.run.smithi043.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.139109.log 2023-08-09T15:13:04.827 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.100473.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.100473.log.gz 2023-08-09T15:13:04.829 INFO:teuthology.orchestra.run.smithi043.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.99160.log 2023-08-09T15:13:04.829 INFO:teuthology.orchestra.run.smithi043.stderr: 93.7% -- replaced with /var/log/ceph/ceph-mds.a.log.gz 2023-08-09T15:13:04.831 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.139109.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.154330.log 2023-08-09T15:13:04.833 INFO:teuthology.orchestra.run.smithi043.stderr: 87.6%/var/log/ceph/ceph-client.admin.99160.log: -- replaced with /var/log/ceph/ceph-client.admin.139109.log.gz 2023-08-09T15:13:04.833 INFO:teuthology.orchestra.run.smithi043.stderr: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.99160.log.gz 2023-08-09T15:13:04.833 INFO:teuthology.orchestra.run.smithi043.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.143843.log 2023-08-09T15:13:04.834 INFO:teuthology.orchestra.run.smithi043.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.127207.log 2023-08-09T15:13:04.835 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.154330.log: /var/log/ceph/ceph-client.admin.143843.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.154330.log.gz 2023-08-09T15:13:04.836 INFO:teuthology.orchestra.run.smithi043.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.143843.log.gz 2023-08-09T15:13:04.837 INFO:teuthology.orchestra.run.smithi043.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.156084.log 2023-08-09T15:13:04.838 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.127207.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.112585.log 2023-08-09T15:13:04.839 INFO:teuthology.orchestra.run.smithi043.stderr: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.127207.log.gz 2023-08-09T15:13:04.841 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.156084.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.104381.log 2023-08-09T15:13:04.841 INFO:teuthology.orchestra.run.smithi043.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.156084.log.gz 2023-08-09T15:13:04.842 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.112585.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.112585.log.gz 2023-08-09T15:13:04.844 INFO:teuthology.orchestra.run.smithi043.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.94898.log 2023-08-09T15:13:04.845 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.104381.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.123005.log 2023-08-09T15:13:04.846 INFO:teuthology.orchestra.run.smithi043.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.104381.log.gz 2023-08-09T15:13:04.847 INFO:teuthology.orchestra.run.smithi203.stderr: 92.1% -- replaced with /var/log/ceph/ceph-mgr.x.log.gz 2023-08-09T15:13:04.847 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.94898.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.123234.log 2023-08-09T15:13:04.848 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.123005.log: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.94898.log.gz 2023-08-09T15:13:04.848 INFO:teuthology.orchestra.run.smithi043.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.123005.log.gz 2023-08-09T15:13:04.849 INFO:teuthology.orchestra.run.smithi043.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.131288.log 2023-08-09T15:13:04.851 INFO:teuthology.orchestra.run.smithi043.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.93606.log 2023-08-09T15:13:04.852 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.123234.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.123234.log.gz 2023-08-09T15:13:04.853 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.131288.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.125265.log 2023-08-09T15:13:04.853 INFO:teuthology.orchestra.run.smithi043.stderr: 88.9%/var/log/ceph/ceph-client.admin.93606.log: -- replaced with /var/log/ceph/ceph-client.admin.131288.log.gz 2023-08-09T15:13:04.854 INFO:teuthology.orchestra.run.smithi043.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.93606.log.gz 2023-08-09T15:13:04.855 INFO:teuthology.orchestra.run.smithi043.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.96659.log 2023-08-09T15:13:04.856 INFO:teuthology.orchestra.run.smithi043.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.99664.log 2023-08-09T15:13:04.857 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.125265.log: /var/log/ceph/ceph-client.admin.96659.log: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.125265.log.gz 2023-08-09T15:13:04.858 INFO:teuthology.orchestra.run.smithi043.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.96659.log.gz 2023-08-09T15:13:04.858 INFO:teuthology.orchestra.run.smithi043.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.150730.log 2023-08-09T15:13:04.860 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.99664.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.123911.log 2023-08-09T15:13:04.861 INFO:teuthology.orchestra.run.smithi043.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.99664.log.gz 2023-08-09T15:13:04.862 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.150730.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.131601.log 2023-08-09T15:13:04.863 INFO:teuthology.orchestra.run.smithi043.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.150730.log.gz/var/log/ceph/ceph-client.admin.123911.log: 2023-08-09T15:13:04.863 INFO:teuthology.orchestra.run.smithi043.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.123911.log.gz 2023-08-09T15:13:04.865 INFO:teuthology.orchestra.run.smithi043.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.124447.log 2023-08-09T15:13:04.866 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.131601.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.128098.log 2023-08-09T15:13:04.867 INFO:teuthology.orchestra.run.smithi043.stderr: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.131601.log.gz 2023-08-09T15:13:04.868 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.124447.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.147294.log 2023-08-09T15:13:04.868 INFO:teuthology.orchestra.run.smithi043.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.124447.log.gz 2023-08-09T15:13:04.870 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.128098.log: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.128098.log.gz 2023-08-09T15:13:04.871 INFO:teuthology.orchestra.run.smithi043.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.112034.log 2023-08-09T15:13:04.872 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.147294.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.112118.log 2023-08-09T15:13:04.873 INFO:teuthology.orchestra.run.smithi043.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.147294.log.gz 2023-08-09T15:13:04.874 INFO:teuthology.orchestra.run.smithi043.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.116902.log 2023-08-09T15:13:04.875 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.112034.log: /var/log/ceph/ceph-client.admin.112118.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.112034.log.gz 2023-08-09T15:13:04.876 INFO:teuthology.orchestra.run.smithi043.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.112118.log.gz 2023-08-09T15:13:04.876 INFO:teuthology.orchestra.run.smithi043.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.153080.log 2023-08-09T15:13:04.878 INFO:teuthology.orchestra.run.smithi043.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.139373.log 2023-08-09T15:13:04.878 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.116902.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.116902.log.gz 2023-08-09T15:13:04.880 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.153080.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.122711.log 2023-08-09T15:13:04.881 INFO:teuthology.orchestra.run.smithi043.stderr: 89.2%/var/log/ceph/ceph-client.admin.139373.log: -- replaced with /var/log/ceph/ceph-client.admin.153080.log.gz 2023-08-09T15:13:04.881 INFO:teuthology.orchestra.run.smithi043.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.139373.log.gz 2023-08-09T15:13:04.883 INFO:teuthology.orchestra.run.smithi043.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.101383.log 2023-08-09T15:13:04.884 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.122711.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.146118.log 2023-08-09T15:13:04.884 INFO:teuthology.orchestra.run.smithi043.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.122711.log.gz 2023-08-09T15:13:04.886 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.101383.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.123486.log 2023-08-09T15:13:04.887 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.146118.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.101383.log.gz 2023-08-09T15:13:04.887 INFO:teuthology.orchestra.run.smithi043.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.146118.log.gz 2023-08-09T15:13:04.889 INFO:teuthology.orchestra.run.smithi043.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.146659.log 2023-08-09T15:13:04.890 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.123486.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.137909.log 2023-08-09T15:13:04.890 INFO:teuthology.orchestra.run.smithi043.stderr: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.123486.log.gz 2023-08-09T15:13:04.892 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.146659.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.95598.log 2023-08-09T15:13:04.893 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.137909.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.146659.log.gz 2023-08-09T15:13:04.894 INFO:teuthology.orchestra.run.smithi043.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.137909.log.gz 2023-08-09T15:13:04.895 INFO:teuthology.orchestra.run.smithi043.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.117793.log 2023-08-09T15:13:04.896 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.95598.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.153356.log 2023-08-09T15:13:04.897 INFO:teuthology.orchestra.run.smithi043.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.95598.log.gz 2023-08-09T15:13:04.898 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.117793.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.151823.log 2023-08-09T15:13:04.899 INFO:teuthology.orchestra.run.smithi043.stderr: 87.8%/var/log/ceph/ceph-client.admin.153356.log: -- replaced with /var/log/ceph/ceph-client.admin.117793.log.gz 2023-08-09T15:13:04.899 INFO:teuthology.orchestra.run.smithi043.stderr: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.153356.log.gz 2023-08-09T15:13:04.901 INFO:teuthology.orchestra.run.smithi043.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.154204.log 2023-08-09T15:13:04.903 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.151823.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.116314.log 2023-08-09T15:13:04.903 INFO:teuthology.orchestra.run.smithi043.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.151823.log.gz 2023-08-09T15:13:04.904 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.154204.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.117499.log 2023-08-09T15:13:04.905 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.116314.log: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.154204.log.gz 2023-08-09T15:13:04.906 INFO:teuthology.orchestra.run.smithi043.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.116314.log.gz 2023-08-09T15:13:04.907 INFO:teuthology.orchestra.run.smithi043.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.89426.log 2023-08-09T15:13:04.909 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.117499.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.97110.log 2023-08-09T15:13:04.909 INFO:teuthology.orchestra.run.smithi043.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.117499.log.gz 2023-08-09T15:13:04.911 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.89426.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.138263.log 2023-08-09T15:13:04.912 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.97110.log: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.89426.log.gz 2023-08-09T15:13:04.912 INFO:teuthology.orchestra.run.smithi043.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.97110.log.gz 2023-08-09T15:13:04.913 INFO:teuthology.orchestra.run.smithi043.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.112673.log 2023-08-09T15:13:04.914 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.138263.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.112543.log 2023-08-09T15:13:04.915 INFO:teuthology.orchestra.run.smithi043.stderr: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.138263.log.gz 2023-08-09T15:13:04.916 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.112673.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.100336.log 2023-08-09T15:13:04.917 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.112543.log: 87.1% -- replaced with /var/log/ceph/ceph-client.admin.112673.log.gz 2023-08-09T15:13:04.918 INFO:teuthology.orchestra.run.smithi043.stderr: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.112543.log.gz 2023-08-09T15:13:04.919 INFO:teuthology.orchestra.run.smithi043.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.124489.log 2023-08-09T15:13:04.920 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.100336.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.102643.log 2023-08-09T15:13:04.921 INFO:teuthology.orchestra.run.smithi043.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.100336.log.gz 2023-08-09T15:13:04.922 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.124489.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.111409.log 2023-08-09T15:13:04.923 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.102643.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.124489.log.gz 2023-08-09T15:13:04.924 INFO:teuthology.orchestra.run.smithi043.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.102643.log.gz 2023-08-09T15:13:04.926 INFO:teuthology.orchestra.run.smithi043.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.112459.log 2023-08-09T15:13:04.927 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.111409.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.146328.log 2023-08-09T15:13:04.927 INFO:teuthology.orchestra.run.smithi043.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.111409.log.gz 2023-08-09T15:13:04.929 INFO:teuthology.orchestra.run.smithi043.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.146827.log 2023-08-09T15:13:04.929 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.112459.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.112459.log.gz 2023-08-09T15:13:04.931 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.146328.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.126817.log 2023-08-09T15:13:04.932 INFO:teuthology.orchestra.run.smithi043.stderr: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.146328.log.gz 2023-08-09T15:13:04.932 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.146827.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.146827.log.gz 2023-08-09T15:13:04.933 INFO:teuthology.orchestra.run.smithi043.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.96305.log 2023-08-09T15:13:04.934 INFO:teuthology.orchestra.run.smithi043.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.93326.log 2023-08-09T15:13:04.935 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.126817.log: /var/log/ceph/ceph-client.admin.96305.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.126817.log.gz 2023-08-09T15:13:04.936 INFO:teuthology.orchestra.run.smithi043.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.96305.log.gz 2023-08-09T15:13:04.936 INFO:teuthology.orchestra.run.smithi043.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.141994.log 2023-08-09T15:13:04.938 INFO:teuthology.orchestra.run.smithi043.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.123528.log 2023-08-09T15:13:04.939 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.93326.log: /var/log/ceph/ceph-client.admin.141994.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.93326.log.gz 2023-08-09T15:13:04.940 INFO:teuthology.orchestra.run.smithi043.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.141994.log.gz 2023-08-09T15:13:04.940 INFO:teuthology.orchestra.run.smithi043.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.96646.log 2023-08-09T15:13:04.942 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.123528.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.142479.log 2023-08-09T15:13:04.943 INFO:teuthology.orchestra.run.smithi043.stderr: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.123528.log.gz 2023-08-09T15:13:04.945 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.96646.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.137588.log 2023-08-09T15:13:04.945 INFO:teuthology.orchestra.run.smithi043.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.96646.log.gz/var/log/ceph/ceph-client.admin.142479.log: 2023-08-09T15:13:04.945 INFO:teuthology.orchestra.run.smithi043.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.142479.log.gz 2023-08-09T15:13:04.946 INFO:teuthology.orchestra.run.smithi043.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.138821.log 2023-08-09T15:13:04.947 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.137588.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.96325.log 2023-08-09T15:13:04.948 INFO:teuthology.orchestra.run.smithi043.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.137588.log.gz 2023-08-09T15:13:04.950 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.138821.log: 87.2% -- replaced with /var/log/ceph/ceph-client.admin.138821.log.gz 2023-08-09T15:13:04.950 INFO:teuthology.orchestra.run.smithi043.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.145713.log 2023-08-09T15:13:04.952 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.96325.log: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.96325.log.gz 2023-08-09T15:13:04.952 INFO:teuthology.orchestra.run.smithi043.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.110919.log 2023-08-09T15:13:04.954 INFO:teuthology.orchestra.run.smithi043.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.122753.log 2023-08-09T15:13:04.955 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.145713.log: /var/log/ceph/ceph-client.admin.110919.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.145713.log.gz 2023-08-09T15:13:04.955 INFO:teuthology.orchestra.run.smithi043.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.110919.log.gz 2023-08-09T15:13:04.956 INFO:teuthology.orchestra.run.smithi043.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.137288.log 2023-08-09T15:13:04.957 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.122753.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.156890.log 2023-08-09T15:13:04.958 INFO:teuthology.orchestra.run.smithi043.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.122753.log.gz 2023-08-09T15:13:04.960 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.137288.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.106551.log 2023-08-09T15:13:04.960 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.156890.log: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.137288.log.gz 2023-08-09T15:13:04.961 INFO:teuthology.orchestra.run.smithi043.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.156890.log.gz 2023-08-09T15:13:04.963 INFO:teuthology.orchestra.run.smithi043.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.142353.log 2023-08-09T15:13:04.964 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.106551.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.94730.log 2023-08-09T15:13:04.964 INFO:teuthology.orchestra.run.smithi043.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.106551.log.gz 2023-08-09T15:13:04.966 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.142353.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.154514.log 2023-08-09T15:13:04.966 INFO:teuthology.orchestra.run.smithi043.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.142353.log.gz 2023-08-09T15:13:04.968 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.94730.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.140940.log 2023-08-09T15:13:04.968 INFO:teuthology.orchestra.run.smithi043.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.94730.log.gz/var/log/ceph/ceph-client.admin.154514.log: 2023-08-09T15:13:04.969 INFO:teuthology.orchestra.run.smithi043.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.154514.log.gz 2023-08-09T15:13:04.971 INFO:teuthology.orchestra.run.smithi043.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.93988.log 2023-08-09T15:13:04.972 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.140940.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.131948.log 2023-08-09T15:13:04.972 INFO:teuthology.orchestra.run.smithi043.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.140940.log.gz 2023-08-09T15:13:04.974 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.93988.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.124125.log 2023-08-09T15:13:04.974 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.131948.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.93988.log.gz 2023-08-09T15:13:04.975 INFO:teuthology.orchestra.run.smithi043.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.131948.log.gz 2023-08-09T15:13:04.977 INFO:teuthology.orchestra.run.smithi043.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.143147.log 2023-08-09T15:13:04.978 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.124125.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.145797.log 2023-08-09T15:13:04.978 INFO:teuthology.orchestra.run.smithi043.stderr: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.124125.log.gz 2023-08-09T15:13:04.980 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.143147.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.101751.log 2023-08-09T15:13:04.981 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.145797.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.143147.log.gz 2023-08-09T15:13:04.981 INFO:teuthology.orchestra.run.smithi043.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.145797.log.gz 2023-08-09T15:13:04.983 INFO:teuthology.orchestra.run.smithi043.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.155160.log 2023-08-09T15:13:04.984 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.101751.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.140361.log 2023-08-09T15:13:04.984 INFO:teuthology.orchestra.run.smithi043.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.101751.log.gz 2023-08-09T15:13:04.986 INFO:teuthology.orchestra.run.smithi043.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.2.98750.log 2023-08-09T15:13:04.986 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.155160.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.155160.log.gz 2023-08-09T15:13:04.988 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.140361.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.151636.log 2023-08-09T15:13:04.988 INFO:teuthology.orchestra.run.smithi043.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.140361.log.gz 2023-08-09T15:13:04.989 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.2.98750.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.115525.log 2023-08-09T15:13:04.990 INFO:teuthology.orchestra.run.smithi043.stderr: 93.2% -- replaced with /var/log/ceph/ceph-client.2.98750.log.gz 2023-08-09T15:13:04.992 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.151636.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.118148.log 2023-08-09T15:13:04.992 INFO:teuthology.orchestra.run.smithi043.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.151636.log.gz 2023-08-09T15:13:04.993 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.115525.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.115525.log.gz 2023-08-09T15:13:04.995 INFO:teuthology.orchestra.run.smithi043.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.105613.log 2023-08-09T15:13:04.996 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.118148.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.119095.log 2023-08-09T15:13:04.997 INFO:teuthology.orchestra.run.smithi043.stderr: 91.0% -- replaced with /var/log/ceph/ceph-client.admin.118148.log.gz 2023-08-09T15:13:04.998 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.105613.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.121018.log 2023-08-09T15:13:04.999 INFO:teuthology.orchestra.run.smithi043.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.105613.log.gz 2023-08-09T15:13:05.000 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.119095.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.119095.log.gz 2023-08-09T15:13:05.000 INFO:teuthology.orchestra.run.smithi043.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.95640.log 2023-08-09T15:13:05.002 INFO:teuthology.orchestra.run.smithi043.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.139501.log 2023-08-09T15:13:05.003 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.121018.log: /var/log/ceph/ceph-client.admin.95640.log: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.121018.log.gz 2023-08-09T15:13:05.003 INFO:teuthology.orchestra.run.smithi043.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.95640.log.gz 2023-08-09T15:13:05.004 INFO:teuthology.orchestra.run.smithi043.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.120285.log 2023-08-09T15:13:05.005 INFO:teuthology.orchestra.run.smithi043.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.99370.log 2023-08-09T15:13:05.007 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.139501.log: /var/log/ceph/ceph-client.admin.120285.log: 87.1% -- replaced with /var/log/ceph/ceph-client.admin.139501.log.gz 2023-08-09T15:13:05.007 INFO:teuthology.orchestra.run.smithi043.stderr: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.120285.log.gz 2023-08-09T15:13:05.008 INFO:teuthology.orchestra.run.smithi043.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.130584.log 2023-08-09T15:13:05.009 INFO:teuthology.orchestra.run.smithi043.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.145755.log 2023-08-09T15:13:05.010 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.99370.log: /var/log/ceph/ceph-client.admin.130584.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.99370.log.gz 2023-08-09T15:13:05.011 INFO:teuthology.orchestra.run.smithi043.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.130584.log.gz 2023-08-09T15:13:05.012 INFO:teuthology.orchestra.run.smithi043.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.147558.log 2023-08-09T15:13:05.014 INFO:teuthology.orchestra.run.smithi043.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.147770.log 2023-08-09T15:13:05.014 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.145755.log: /var/log/ceph/ceph-client.admin.147558.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.145755.log.gz 2023-08-09T15:13:05.015 INFO:teuthology.orchestra.run.smithi043.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.147558.log.gz 2023-08-09T15:13:05.016 INFO:teuthology.orchestra.run.smithi043.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.104167.log 2023-08-09T15:13:05.017 INFO:teuthology.orchestra.run.smithi043.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.130670.log 2023-08-09T15:13:05.018 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.147770.log: /var/log/ceph/ceph-client.admin.104167.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.147770.log.gz 2023-08-09T15:13:05.019 INFO:teuthology.orchestra.run.smithi043.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.104167.log.gz 2023-08-09T15:13:05.020 INFO:teuthology.orchestra.run.smithi043.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.147896.log 2023-08-09T15:13:05.021 INFO:teuthology.orchestra.run.smithi043.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.100557.log 2023-08-09T15:13:05.022 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.130670.log: /var/log/ceph/ceph-client.admin.147896.log: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.130670.log.gz 2023-08-09T15:13:05.023 INFO:teuthology.orchestra.run.smithi043.stderr: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.147896.log.gz 2023-08-09T15:13:05.024 INFO:teuthology.orchestra.run.smithi043.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.102559.log 2023-08-09T15:13:05.025 INFO:teuthology.orchestra.run.smithi043.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.124699.log 2023-08-09T15:13:05.026 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.100557.log: /var/log/ceph/ceph-client.admin.102559.log: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.100557.log.gz 2023-08-09T15:13:05.027 INFO:teuthology.orchestra.run.smithi043.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.102559.log.gz 2023-08-09T15:13:05.028 INFO:teuthology.orchestra.run.smithi043.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.122921.log 2023-08-09T15:13:05.030 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.124699.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.115867.log 2023-08-09T15:13:05.030 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.122921.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.124699.log.gz 2023-08-09T15:13:05.031 INFO:teuthology.orchestra.run.smithi043.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.122921.log.gz 2023-08-09T15:13:05.032 INFO:teuthology.orchestra.run.smithi043.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.123318.log 2023-08-09T15:13:05.033 INFO:teuthology.orchestra.run.smithi043.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.100599.log 2023-08-09T15:13:05.034 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.115867.log: /var/log/ceph/ceph-client.admin.123318.log: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.115867.log.gz 2023-08-09T15:13:05.035 INFO:teuthology.orchestra.run.smithi043.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.123318.log.gz 2023-08-09T15:13:05.035 INFO:teuthology.orchestra.run.smithi043.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.131162.log 2023-08-09T15:13:05.037 INFO:teuthology.orchestra.run.smithi043.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.123953.log 2023-08-09T15:13:05.038 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.100599.log: /var/log/ceph/ceph-client.admin.131162.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.100599.log.gz 2023-08-09T15:13:05.039 INFO:teuthology.orchestra.run.smithi043.stderr: 89.5% -- replaced with /var/log/ceph/ceph-client.admin.131162.log.gz 2023-08-09T15:13:05.039 INFO:teuthology.orchestra.run.smithi043.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.111199.log 2023-08-09T15:13:05.041 INFO:teuthology.orchestra.run.smithi043.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.141826.log 2023-08-09T15:13:05.042 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.123953.log: /var/log/ceph/ceph-client.admin.111199.log: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.123953.log.gz 2023-08-09T15:13:05.043 INFO:teuthology.orchestra.run.smithi043.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.111199.log.gz 2023-08-09T15:13:05.043 INFO:teuthology.orchestra.run.smithi043.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.95682.log 2023-08-09T15:13:05.045 INFO:teuthology.orchestra.run.smithi043.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.96661.log 2023-08-09T15:13:05.046 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.141826.log: /var/log/ceph/ceph-client.admin.95682.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.141826.log.gz 2023-08-09T15:13:05.047 INFO:teuthology.orchestra.run.smithi043.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.95682.log.gz 2023-08-09T15:13:05.047 INFO:teuthology.orchestra.run.smithi043.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.117979.log 2023-08-09T15:13:05.049 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.96661.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.126859.log 2023-08-09T15:13:05.049 INFO:teuthology.orchestra.run.smithi043.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.96661.log.gz 2023-08-09T15:13:05.051 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.117979.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.110417.log 2023-08-09T15:13:05.052 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.126859.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.117979.log.gz 2023-08-09T15:13:05.052 INFO:teuthology.orchestra.run.smithi043.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.126859.log.gz 2023-08-09T15:13:05.054 INFO:teuthology.orchestra.run.smithi043.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.120243.log 2023-08-09T15:13:05.055 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.110417.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.141471.log 2023-08-09T15:13:05.055 INFO:teuthology.orchestra.run.smithi043.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.110417.log.gz 2023-08-09T15:13:05.056 INFO:teuthology.orchestra.run.smithi043.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.147644.log 2023-08-09T15:13:05.058 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.120243.log: /var/log/ceph/ceph-client.admin.141471.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.120243.log.gz 2023-08-09T15:13:05.058 INFO:teuthology.orchestra.run.smithi043.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.141471.log.gz 2023-08-09T15:13:05.059 INFO:teuthology.orchestra.run.smithi043.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.107636.log 2023-08-09T15:13:05.061 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.147644.log: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.147644.log.gz 2023-08-09T15:13:05.062 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.107636.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.107636.log.gz 2023-08-09T15:13:05.142 INFO:teuthology.orchestra.run.smithi043.stderr: 93.5% -- replaced with /var/log/ceph/ceph-mds.c.log.gz 2023-08-09T15:13:05.196 INFO:teuthology.orchestra.run.smithi203.stderr: 92.3% -- replaced with /var/log/ceph/ceph-mon.c.log.gz 2023-08-09T15:13:05.494 INFO:teuthology.orchestra.run.smithi203.stderr: 92.2% -- replaced with /var/log/ceph/ceph-mon.b.log.gz 2023-08-09T15:13:08.158 INFO:teuthology.orchestra.run.smithi043.stderr: 93.2% -- replaced with /var/log/ceph/ceph-osd.0.log.gz 2023-08-09T15:13:08.214 INFO:teuthology.orchestra.run.smithi043.stderr: 90.6% -- replaced with /var/log/ceph/ceph-mon.a.log.gz 2023-08-09T15:13:08.291 INFO:teuthology.orchestra.run.smithi043.stderr: 93.2% -- replaced with /var/log/ceph/ceph-osd.1.log.gz 2023-08-09T15:13:08.354 INFO:teuthology.orchestra.run.smithi203.stderr: 93.2% -- replaced with /var/log/ceph/ceph-osd.6.log.gz 2023-08-09T15:13:08.449 INFO:teuthology.orchestra.run.smithi203.stderr: 93.0% -- replaced with /var/log/ceph/ceph-osd.5.log.gz 2023-08-09T15:13:08.877 INFO:teuthology.orchestra.run.smithi203.stderr: 93.0% -- replaced with /var/log/ceph/ceph-osd.4.log.gz 2023-08-09T15:13:08.912 INFO:teuthology.orchestra.run.smithi043.stderr: 93.3% -- replaced with /var/log/ceph/ceph-osd.2.log.gz 2023-08-09T15:13:09.008 INFO:teuthology.orchestra.run.smithi203.stderr: 92.9% -- replaced with /var/log/ceph/ceph-osd.7.log.gz 2023-08-09T15:13:09.012 INFO:teuthology.orchestra.run.smithi203.stderr: 2023-08-09T15:13:09.012 INFO:teuthology.orchestra.run.smithi203.stderr:real 0m5.818s 2023-08-09T15:13:09.013 INFO:teuthology.orchestra.run.smithi203.stderr:user 0m25.484s 2023-08-09T15:13:09.013 INFO:teuthology.orchestra.run.smithi203.stderr:sys 0m2.288s 2023-08-09T15:13:11.705 INFO:teuthology.orchestra.run.smithi043.stderr: 92.7% -- replaced with /var/log/ceph/ceph-osd.3.log.gz 2023-08-09T15:13:11.709 INFO:teuthology.orchestra.run.smithi043.stderr: 2023-08-09T15:13:11.710 INFO:teuthology.orchestra.run.smithi043.stderr:real 0m8.513s 2023-08-09T15:13:11.710 INFO:teuthology.orchestra.run.smithi043.stderr:user 0m27.359s 2023-08-09T15:13:11.710 INFO:teuthology.orchestra.run.smithi043.stderr:sys 0m5.210s 2023-08-09T15:13:11.711 INFO:tasks.ceph:Archiving logs... 2023-08-09T15:13:11.711 DEBUG:teuthology.misc:Transferring archived files from smithi043:/var/log/ceph to /home/teuthworker/archive/vshankar-2023-08-09_05:46:29-fs-wip-vshankar-testing-20230809.035933-testing-default-smithi/7364139/remote/smithi043/log 2023-08-09T15:13:11.725 DEBUG:teuthology.orchestra.run.smithi043:> sudo tar c -f - -C /var/log/ceph -- . 2023-08-09T15:13:15.337 DEBUG:teuthology.misc:Transferring archived files from smithi203:/var/log/ceph to /home/teuthworker/archive/vshankar-2023-08-09_05:46:29-fs-wip-vshankar-testing-20230809.035933-testing-default-smithi/7364139/remote/smithi203/log 2023-08-09T15:13:15.338 DEBUG:teuthology.orchestra.run.smithi203:> sudo tar c -f - -C /var/log/ceph -- . 2023-08-09T15:13:18.177 DEBUG:teuthology.run_tasks:Unwinding manager install 2023-08-09T15:13:18.186 INFO:teuthology.task.install.util:Removing shipped files: /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer... 2023-08-09T15:13:18.187 DEBUG:teuthology.orchestra.run.smithi043:> sudo rm -f -- /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer 2023-08-09T15:13:18.196 DEBUG:teuthology.orchestra.run.smithi203:> sudo rm -f -- /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer 2023-08-09T15:13:18.356 DEBUG:teuthology.run_tasks:Unwinding manager clock 2023-08-09T15:13:18.366 INFO:teuthology.task.clock:Checking final clock skew... 2023-08-09T15:13:18.366 DEBUG:teuthology.orchestra.run.smithi043:> PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2023-08-09T15:13:18.376 DEBUG:teuthology.orchestra.run.smithi203:> PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2023-08-09T15:13:18.424 INFO:teuthology.orchestra.run.smithi043.stderr:bash: line 1: ntpq: command not found 2023-08-09T15:13:18.431 INFO:teuthology.orchestra.run.smithi203.stderr:bash: line 1: ntpq: command not found 2023-08-09T15:13:18.434 INFO:teuthology.orchestra.run.smithi043.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2023-08-09T15:13:18.434 INFO:teuthology.orchestra.run.smithi043.stdout:=============================================================================== 2023-08-09T15:13:18.434 INFO:teuthology.orchestra.run.smithi043.stdout:^+ hv01.front.sepia.ceph.com 2 6 377 53 -3057us[-3057us] +/- 101ms 2023-08-09T15:13:18.434 INFO:teuthology.orchestra.run.smithi043.stdout:^* hv02.front.sepia.ceph.com 3 7 377 61 +719us[ +748us] +/- 80ms 2023-08-09T15:13:18.434 INFO:teuthology.orchestra.run.smithi043.stdout:^+ hv03.front.sepia.ceph.com 3 8 377 253 +1349us[+1446us] +/- 61ms 2023-08-09T15:13:18.434 INFO:teuthology.orchestra.run.smithi043.stdout:^? hv04.front.sepia.ceph.com 0 10 0 - +0ns[ +0ns] +/- 0ns 2023-08-09T15:13:18.440 INFO:teuthology.orchestra.run.smithi203.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2023-08-09T15:13:18.440 INFO:teuthology.orchestra.run.smithi203.stdout:=============================================================================== 2023-08-09T15:13:18.440 INFO:teuthology.orchestra.run.smithi203.stdout:^+ hv01.front.sepia.ceph.com 2 6 377 61 -3030us[-3030us] +/- 101ms 2023-08-09T15:13:18.440 INFO:teuthology.orchestra.run.smithi203.stdout:^* hv02.front.sepia.ceph.com 3 7 377 124 +717us[ +758us] +/- 79ms 2023-08-09T15:13:18.441 INFO:teuthology.orchestra.run.smithi203.stdout:^+ hv03.front.sepia.ceph.com 3 8 377 61 +1235us[+1235us] +/- 64ms 2023-08-09T15:13:18.441 INFO:teuthology.orchestra.run.smithi203.stdout:^? hv04.front.sepia.ceph.com 0 10 0 - +0ns[ +0ns] +/- 0ns 2023-08-09T15:13:18.442 DEBUG:teuthology.run_tasks:Unwinding manager ansible.cephlab 2023-08-09T15:13:18.452 INFO:teuthology.task.ansible:Skipping ansible cleanup... 2023-08-09T15:13:18.453 DEBUG:teuthology.run_tasks:Unwinding manager selinux 2023-08-09T15:13:18.463 DEBUG:teuthology.orchestra.run.smithi043:> mkdir /home/ubuntu/cephtest/archive/audit && sudo cp /var/log/audit/audit.log /home/ubuntu/cephtest/archive/audit && sudo chown $USER /home/ubuntu/cephtest/archive/audit/audit.log && gzip /home/ubuntu/cephtest/archive/audit/audit.log 2023-08-09T15:13:18.665 DEBUG:teuthology.orchestra.run.smithi203:> mkdir /home/ubuntu/cephtest/archive/audit && sudo cp /var/log/audit/audit.log /home/ubuntu/cephtest/archive/audit && sudo chown $USER /home/ubuntu/cephtest/archive/audit/audit.log && gzip /home/ubuntu/cephtest/archive/audit/audit.log 2023-08-09T15:13:18.932 DEBUG:teuthology.orchestra.run.smithi043:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 2023-08-09T15:13:19.045 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-08-09T15:13:19.046 DEBUG:teuthology.orchestra.run.smithi203:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 2023-08-09T15:13:19.161 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-08-09T15:13:19.162 DEBUG:teuthology.run_tasks:Unwinding manager pcp 2023-08-09T15:13:19.177 DEBUG:teuthology.run_tasks:Unwinding manager internal.timer 2023-08-09T15:13:19.192 INFO:teuthology.task.internal:Duration was 6830.613158 seconds 2023-08-09T15:13:19.193 DEBUG:teuthology.run_tasks:Unwinding manager internal.syslog 2023-08-09T15:13:19.207 INFO:teuthology.task.internal.syslog:Shutting down syslog monitoring... 2023-08-09T15:13:19.208 DEBUG:teuthology.orchestra.run.smithi043:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2023-08-09T15:13:19.217 DEBUG:teuthology.orchestra.run.smithi203:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2023-08-09T15:13:19.390 INFO:teuthology.orchestra.run.smithi203.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2023-08-09T15:13:19.399 INFO:teuthology.orchestra.run.smithi043.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2023-08-09T15:13:19.823 INFO:teuthology.task.internal.syslog:Checking logs for errors... 2023-08-09T15:13:19.823 DEBUG:teuthology.task.internal.syslog:Checking ubuntu@smithi043.front.sepia.ceph.com 2023-08-09T15:13:19.823 DEBUG:teuthology.orchestra.run.smithi043:> egrep --binary-files=text '\bBUG\b|\bINFO\b|\bDEADLOCK\b' /home/ubuntu/cephtest/archive/syslog/kern.log | grep -v 'task .* blocked for more than .* seconds' | grep -v 'lockdep is turned off' | grep -v 'trying to register non-static key' | grep -v 'DEBUG: fsize' | grep -v CRON | grep -v 'BUG: bad unlock balance detected' | grep -v 'inconsistent lock state' | grep -v '*** DEADLOCK ***' | grep -v 'INFO: possible irq lock inversion dependency detected' | grep -v 'INFO: NMI handler (perf_event_nmi_handler) took too long to run' | grep -v 'INFO: recovery required on readonly' | grep -v 'ceph-create-keys: INFO' | grep -v INFO:ceph-create-keys | grep -v 'Loaded datasource DataSourceOpenStack' | grep -v 'container-storage-setup: INFO: Volume group backing root filesystem could not be determined' | egrep -v '\bsalt-master\b|\bsalt-minion\b|\bsalt-api\b' | grep -v ceph-crash | egrep -v '\btcmu-runner\b.*\bINFO\b' | head -n 1 2023-08-09T15:13:19.903 DEBUG:teuthology.task.internal.syslog:Checking ubuntu@smithi203.front.sepia.ceph.com 2023-08-09T15:13:19.904 DEBUG:teuthology.orchestra.run.smithi203:> egrep --binary-files=text '\bBUG\b|\bINFO\b|\bDEADLOCK\b' /home/ubuntu/cephtest/archive/syslog/kern.log | grep -v 'task .* blocked for more than .* seconds' | grep -v 'lockdep is turned off' | grep -v 'trying to register non-static key' | grep -v 'DEBUG: fsize' | grep -v CRON | grep -v 'BUG: bad unlock balance detected' | grep -v 'inconsistent lock state' | grep -v '*** DEADLOCK ***' | grep -v 'INFO: possible irq lock inversion dependency detected' | grep -v 'INFO: NMI handler (perf_event_nmi_handler) took too long to run' | grep -v 'INFO: recovery required on readonly' | grep -v 'ceph-create-keys: INFO' | grep -v INFO:ceph-create-keys | grep -v 'Loaded datasource DataSourceOpenStack' | grep -v 'container-storage-setup: INFO: Volume group backing root filesystem could not be determined' | egrep -v '\bsalt-master\b|\bsalt-minion\b|\bsalt-api\b' | grep -v ceph-crash | egrep -v '\btcmu-runner\b.*\bINFO\b' | head -n 1 2023-08-09T15:13:19.993 INFO:teuthology.task.internal.syslog:Compressing syslogs... 2023-08-09T15:13:19.993 DEBUG:teuthology.orchestra.run.smithi043:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2023-08-09T15:13:20.002 DEBUG:teuthology.orchestra.run.smithi203:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2023-08-09T15:13:20.104 INFO:teuthology.task.internal.syslog:Gathering journactl -b0... 2023-08-09T15:13:20.105 DEBUG:teuthology.orchestra.run.smithi043:> sudo journalctl -b0 | gzip -9 > /home/ubuntu/cephtest/archive/syslog/journalctl-b0.gz 2023-08-09T15:13:20.110 DEBUG:teuthology.orchestra.run.smithi203:> sudo journalctl -b0 | gzip -9 > /home/ubuntu/cephtest/archive/syslog/journalctl-b0.gz 2023-08-09T15:13:22.153 DEBUG:teuthology.run_tasks:Unwinding manager internal.sudo 2023-08-09T15:13:22.169 INFO:teuthology.task.internal:Restoring /etc/sudoers... 2023-08-09T15:13:22.169 DEBUG:teuthology.orchestra.run.smithi043:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers 2023-08-09T15:13:22.268 DEBUG:teuthology.orchestra.run.smithi203:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers 2023-08-09T15:13:22.372 DEBUG:teuthology.run_tasks:Unwinding manager internal.coredump 2023-08-09T15:13:22.386 DEBUG:teuthology.orchestra.run.smithi043:> sudo sysctl -w kernel.core_pattern=core && sudo bash -c 'for f in `find /home/ubuntu/cephtest/archive/coredump -type f`; do file $f | grep -q systemd-sysusers && rm $f || true ; done' && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2023-08-09T15:13:22.395 DEBUG:teuthology.orchestra.run.smithi203:> sudo sysctl -w kernel.core_pattern=core && sudo bash -c 'for f in `find /home/ubuntu/cephtest/archive/coredump -type f`; do file $f | grep -q systemd-sysusers && rm $f || true ; done' && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2023-08-09T15:13:22.504 INFO:teuthology.orchestra.run.smithi043.stdout:kernel.core_pattern = core 2023-08-09T15:13:22.512 INFO:teuthology.orchestra.run.smithi203.stdout:kernel.core_pattern = core 2023-08-09T15:13:22.565 DEBUG:teuthology.orchestra.run.smithi043:> test -e /home/ubuntu/cephtest/archive/coredump 2023-08-09T15:13:22.620 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-08-09T15:13:22.621 DEBUG:teuthology.orchestra.run.smithi203:> test -e /home/ubuntu/cephtest/archive/coredump 2023-08-09T15:13:22.678 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-08-09T15:13:22.679 DEBUG:teuthology.run_tasks:Unwinding manager internal.archive 2023-08-09T15:13:22.693 INFO:teuthology.task.internal:Transferring archived files... 2023-08-09T15:13:22.694 DEBUG:teuthology.misc:Transferring archived files from smithi043:/home/ubuntu/cephtest/archive to /home/teuthworker/archive/vshankar-2023-08-09_05:46:29-fs-wip-vshankar-testing-20230809.035933-testing-default-smithi/7364139/remote/smithi043 2023-08-09T15:13:22.695 DEBUG:teuthology.orchestra.run.smithi043:> sudo tar c -f - -C /home/ubuntu/cephtest/archive -- . 2023-08-09T15:13:22.820 DEBUG:teuthology.misc:Transferring archived files from smithi203:/home/ubuntu/cephtest/archive to /home/teuthworker/archive/vshankar-2023-08-09_05:46:29-fs-wip-vshankar-testing-20230809.035933-testing-default-smithi/7364139/remote/smithi203 2023-08-09T15:13:22.820 DEBUG:teuthology.orchestra.run.smithi203:> sudo tar c -f - -C /home/ubuntu/cephtest/archive -- . 2023-08-09T15:13:22.936 INFO:teuthology.task.internal:Removing archive directory... 2023-08-09T15:13:22.937 DEBUG:teuthology.orchestra.run.smithi043:> rm -rf -- /home/ubuntu/cephtest/archive 2023-08-09T15:13:22.946 DEBUG:teuthology.orchestra.run.smithi203:> rm -rf -- /home/ubuntu/cephtest/archive 2023-08-09T15:13:23.036 DEBUG:teuthology.run_tasks:Unwinding manager internal.archive_upload 2023-08-09T15:13:23.047 INFO:teuthology.task.internal:Not uploading archives. 2023-08-09T15:13:23.048 DEBUG:teuthology.run_tasks:Unwinding manager internal.base 2023-08-09T15:13:23.058 INFO:teuthology.task.internal:Tidying up after the test... 2023-08-09T15:13:23.058 DEBUG:teuthology.orchestra.run.smithi043:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest 2023-08-09T15:13:23.065 DEBUG:teuthology.orchestra.run.smithi203:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest 2023-08-09T15:13:23.120 INFO:teuthology.orchestra.run.smithi043.stdout: 263230 4 drwxr-xr-x 2 ubuntu ubuntu 4096 Aug 9 15:13 /home/ubuntu/cephtest 2023-08-09T15:13:23.120 INFO:teuthology.orchestra.run.smithi203.stdout: 263230 4 drwxr-xr-x 2 ubuntu ubuntu 4096 Aug 9 15:13 /home/ubuntu/cephtest 2023-08-09T15:13:23.123 DEBUG:teuthology.run_tasks:Unwinding manager kernel 2023-08-09T15:13:23.133 DEBUG:teuthology.run_tasks:Unwinding manager console_log 2023-08-09T15:13:23.153 INFO:teuthology.nuke:Checking targets against current locks 2023-08-09T15:13:23.181 DEBUG:teuthology.nuke:shortname: smithi043 2023-08-09T15:13:23.181 INFO:teuthology.task.internal.check_lock:Checking locks... 2023-08-09T15:13:23.202 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi043.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/vshankar-2023-08-09_05:46:29-fs-wip-vshankar-testing-20230809.035933-testing-default-smithi/7364139', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '9', 'arch': 'x86_64', 'locked': True, 'locked_since': '2023-08-09 13:09:34.745232', 'locked_by': 'scheduled_vshankar@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBEKktPs1zHDi/KNgutOyVqOWoAarBLxJwqxEBET4kyZL78XfuBO2Mt+SP5ieccwraLOmO+jlznyREbzKXB1iudI='} 2023-08-09T15:13:23.204 DEBUG:teuthology.nuke:shortname: smithi203 2023-08-09T15:13:23.205 INFO:teuthology.task.internal.check_lock:Checking locks... 2023-08-09T15:13:23.229 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi203.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/vshankar-2023-08-09_05:46:29-fs-wip-vshankar-testing-20230809.035933-testing-default-smithi/7364139', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '9', 'arch': 'x86_64', 'locked': True, 'locked_since': '2023-08-09 13:09:34.746687', 'locked_by': 'scheduled_vshankar@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBD/W9aB9RM/Yz46ZrKxLIqCF48iawl+xUQvYLoz3OmwYWs/kun9L9UcFcFyB+qwqiBQJZebgK5RojyEKSWt07Cw='} 2023-08-09T15:13:23.262 INFO:teuthology.orchestra.console.smithi043:Power off 2023-08-09T15:13:23.263 DEBUG:teuthology.orchestra.console.smithi043:pexpect command: ipmitool -H smithi043.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power off 2023-08-09T15:13:23.287 DEBUG:teuthology.orchestra.console.smithi043:power off output: Chassis Power Control: Down/Off 2023-08-09T15:13:23.288 DEBUG:teuthology.orchestra.console.smithi043:pexpect command: ipmitool -H smithi043.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-08-09T15:13:23.306 INFO:teuthology.orchestra.console.smithi203:Power off 2023-08-09T15:13:23.307 DEBUG:teuthology.orchestra.console.smithi203:pexpect command: ipmitool -H smithi203.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power off 2023-08-09T15:13:23.317 DEBUG:teuthology.orchestra.console.smithi043:check power output: Chassis Power is on 2023-08-09T15:13:23.334 DEBUG:teuthology.orchestra.console.smithi203:power off output: Chassis Power Control: Down/Off 2023-08-09T15:13:23.335 DEBUG:teuthology.orchestra.console.smithi203:pexpect command: ipmitool -H smithi203.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-08-09T15:13:23.356 DEBUG:teuthology.orchestra.console.smithi203:check power output: Chassis Power is on 2023-08-09T15:13:27.319 DEBUG:teuthology.orchestra.console.smithi043:pexpect command: ipmitool -H smithi043.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-08-09T15:13:27.357 DEBUG:teuthology.orchestra.console.smithi203:pexpect command: ipmitool -H smithi203.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-08-09T15:13:27.429 DEBUG:teuthology.orchestra.console.smithi043:check power output: Chassis Power is on 2023-08-09T15:13:27.469 DEBUG:teuthology.orchestra.console.smithi203:check power output: Chassis Power is on 2023-08-09T15:13:31.432 DEBUG:teuthology.orchestra.console.smithi043:pexpect command: ipmitool -H smithi043.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-08-09T15:13:31.470 DEBUG:teuthology.orchestra.console.smithi203:pexpect command: ipmitool -H smithi203.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-08-09T15:13:31.543 DEBUG:teuthology.orchestra.console.smithi043:check power output: Chassis Power is off 2023-08-09T15:13:31.582 DEBUG:teuthology.orchestra.console.smithi203:check power output: Chassis Power is off 2023-08-09T15:13:31.644 INFO:teuthology.orchestra.console.smithi043:Power off completed 2023-08-09T15:13:31.683 INFO:teuthology.orchestra.console.smithi203:Power off completed 2023-08-09T15:13:31.789 INFO:teuthology.run:Summary data: description: fs/functional/{begin/{0-install 1-ceph 2-logrotate} clusters/1a3s-mds-4c-client conf/{client mds mon osd} distro/{centos_latest} mount/fuse objectstore/bluestore-ec-root overrides/{ignorelist_health ignorelist_wrongly_marked_down no_client_pidfile} tasks/snap-schedule} duration: 6830.613157749176 failure_reason: '"2023-08-09T13:37:30.108177+0000 mon.a (mon.0) 437 : cluster [WRN] Health check failed: Reduced data availability: 2 pgs peering (PG_AVAILABILITY)" in cluster log' flavor: default mon.a-kernel-sha1: a7fb1265323db972dd333f71b9a53e9479f62e37 mon.b-kernel-sha1: a7fb1265323db972dd333f71b9a53e9479f62e37 owner: scheduled_vshankar@teuthology success: false 2023-08-09T15:13:31.789 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-08-09T15:13:31.877 INFO:teuthology.run:FAIL